2024.10.22 00:14:17.489430 [ 1032 ] {} Application: Will watch for the process with pid 1034 2024.10.22 00:14:17.489707 [ 1034 ] {} Application: Forked a child process to watch 2024.10.22 00:14:17.491264 [ 1034 ] {} SentryWriter: Sending crash reports is disabled 2024.10.22 00:14:17.491714 [ 1034 ] {} Pipe: Pipe capacity is 1.00 MiB 2024.10.22 00:14:18.575140 [ 1034 ] {} Application: Starting ClickHouse 23.8.16.42.altinitystable (revision: 54477, git hash: 46d4f9bd6a40e43bf5543be25cc704364e0c737b, build id: 357BD09AC2B3A46F0B44F66168828D9D6CC8F70F), PID 1034 2024.10.22 00:14:18.578830 [ 1034 ] {} Application: starting up 2024.10.22 00:14:18.579015 [ 1034 ] {} Application: OS name: Linux, version: 5.15.0-122-generic, architecture: x86_64 2024.10.22 00:14:18.628457 [ 1034 ] {} Context: Server was built in debug mode. It will work slowly. 2024.10.22 00:14:18.642214 [ 1034 ] {} Context: ThreadFuzzer is enabled. Application will run slowly and unstable. 2024.10.22 00:14:18.647256 [ 1034 ] {} Application: Available RAM: 30.60 GiB; physical cores: 16; logical cores: 16. 2024.10.22 00:14:18.772418 [ 1034 ] {} Context: The setting 'allow_remote_fs_zero_copy_replication' is enabled for MergeTree tables. But the feature of 'zero-copy replication' is under development and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. 2024.10.22 00:14:18.929089 [ 1034 ] {} AsynchronousMetrics: Scanning /sys/class/thermal 2024.10.22 00:14:18.931799 [ 1034 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 00:14:18.953704 [ 1034 ] {} AsynchronousMetrics: Scanning /sys/devices/system/edac 2024.10.22 00:14:18.955580 [ 1034 ] {} AsynchronousMetrics: Scanning /sys/class/hwmon 2024.10.22 00:14:35.556141 [ 1034 ] {} Application: Integrity check of the executable successfully passed (checksum: 14165EF9ABB7482E2085E723F61605D4) 2024.10.22 00:14:35.559745 [ 1034 ] {} Application: Will do mlock to prevent executable memory from being paged out. It may take a few seconds. 2024.10.22 00:14:35.591244 [ 1034 ] {} Application: The memory map of clickhouse executable has been mlock'ed, total 471.31 MiB 2024.10.22 00:14:35.596738 [ 1034 ] {} Application: Set OOM score adjustment to 555 2024.10.22 00:14:35.763797 [ 1034 ] {} Application: Set max number of file descriptors to 1048576 (was 1024). 2024.10.22 00:14:35.765084 [ 1034 ] {} Application: rlimit on number of threads is 18446744073709551615 2024.10.22 00:14:35.767389 [ 1034 ] {} Application: Initializing DateLUT. 2024.10.22 00:14:35.768115 [ 1034 ] {} Application: Initialized DateLUT with time zone 'Europe/Amsterdam'. 2024.10.22 00:14:35.771567 [ 1034 ] {} Context: Setting up /var/lib/clickhouse/tmp/ to store temporary data in it 2024.10.22 00:14:35.782389 [ 1034 ] {} TLDListsHolder: public_suffix_list loading from /etc/clickhouse-server/top_level_domains/public_suffix_list.dat 2024.10.22 00:14:36.791761 [ 1034 ] {} TLDListsHolder: public_suffix_list was added (9084 hosts) 2024.10.22 00:14:36.793351 [ 1034 ] {} TLDListsHolder: no_new_line_list loading from /etc/clickhouse-server/top_level_domains/no_new_line_list.dat 2024.10.22 00:14:36.800782 [ 1034 ] {} TLDListsHolder: no_new_line_list was added (1 hosts) 2024.10.22 00:14:36.804208 [ 1034 ] {} Context: The setting 'merge_tree_metadata_cache' is enabled. But the feature of 'metadata cache in RocksDB' is experimental and is not ready for production. The usage of this feature can lead to data corruption and loss. The setting should be disabled in production. See the corresponding report at https://github.com/ClickHouse/ClickHouse/issues/51182 2024.10.22 00:14:36.807596 [ 1034 ] {} Application: Initializing MergeTree metadata cache, lru_cache_size: 256.00 MiB continue_if_corrupted: true 2024.10.22 00:14:37.865316 [ 1034 ] {} Application: Configuration parameter 'interserver_http_host' doesn't exist or exists and empty. Will use 'f111be1a9f3c' as replica host. 2024.10.22 00:14:37.867371 [ 1034 ] {} Application: Initializing interserver credentials. 2024.10.22 00:14:37.965712 [ 1034 ] {} NamedCollectionsUtils: Loaded 6 collections from config 2024.10.22 00:14:37.969653 [ 1034 ] {} NamedCollectionsUtils: Loaded 0 collections from SQL 2024.10.22 00:14:37.973512 [ 1034 ] {} SensitiveDataMaskerConfigRead: 1 query masking rules loaded. 2024.10.22 00:14:38.090485 [ 1034 ] {} ConfigReloader: Loading config '/etc/clickhouse-server/config.xml' 2024.10.22 00:14:40.714347 [ 1034 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performing update on configuration 2024.10.22 00:14:42.084040 [ 1034 ] {} Application: Setting max_server_memory_usage was set to 15.30 GiB (30.60 GiB available * 0.50 max_server_memory_usage_to_ram_ratio) 2024.10.22 00:14:42.085067 [ 1034 ] {} Application: Setting merges_mutations_memory_usage_soft_limit was set to 15.30 GiB (30.60 GiB available * 0.50 merges_mutations_memory_usage_to_ram_ratio) 2024.10.22 00:14:42.085822 [ 1034 ] {} Application: Merges and mutations memory limit is set to 15.30 GiB 2024.10.22 00:14:44.421683 [ 1034 ] {} BackgroundSchedulePool/BgBufSchPool: Create BackgroundSchedulePool with 16 threads 2024.10.22 00:14:44.551581 [ 1034 ] {} BackgroundSchedulePool/BgSchPool: Create BackgroundSchedulePool with 128 threads 2024.10.22 00:14:45.066334 [ 1034 ] {} BackgroundSchedulePool/BgMBSchPool: Create BackgroundSchedulePool with 16 threads 2024.10.22 00:14:45.254794 [ 1034 ] {} BackgroundSchedulePool/BgDistSchPool: Create BackgroundSchedulePool with 16 threads 2024.10.22 00:14:45.432477 [ 1034 ] {} CertificateReloader: Reloading certificate (/etc/clickhouse-server/server.crt) and key (/etc/clickhouse-server/server.key). 2024.10.22 00:14:45.567360 [ 1034 ] {} CertificateReloader: Reloaded certificate (/etc/clickhouse-server/server.crt) and key (/etc/clickhouse-server/server.key). 2024.10.22 00:14:45.569724 [ 1034 ] {} CertificateReloader: Initializing certificate reloader. 2024.10.22 00:14:46.119335 [ 1034 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/config.xml', performed update on configuration 2024.10.22 00:14:46.207790 [ 1034 ] {} Context: Keeper server is participant of the main zookeeper cluster, will try to connect to it 2024.10.22 00:14:46.271607 [ 1034 ] {} Context: Cannot connect to ZooKeeper (or Keeper) before internal Keeper start, will wait for Keeper synchronously 2024.10.22 00:14:46.318230 [ 1034 ] {} KeeperDispatcher: Initializing storage dispatcher 2024.10.22 00:14:46.365418 [ 1034 ] {} KeeperContext: Keeper feature flag FILTERED_LIST: enabled 2024.10.22 00:14:46.366391 [ 1034 ] {} KeeperContext: Keeper feature flag MULTI_READ: enabled 2024.10.22 00:14:46.367508 [ 1034 ] {} KeeperContext: Keeper feature flag CHECK_NOT_EXISTS: enabled 2024.10.22 00:14:46.371440 [ 1034 ] {} KeeperContext: Disk type 'azure_blob_storage' is not supported for Keeper 2024.10.22 00:14:46.373270 [ 1034 ] {} KeeperContext: Disk type 'cache' is not supported for Keeper 2024.10.22 00:14:50.505138 [ 1034 ] {} S3Client: Provider type: Unknown 2024.10.22 00:14:50.516922 [ 1034 ] {} S3Client: API mode of the S3 client: AWS 2024.10.22 00:14:52.618499 [ 1034 ] {} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/clickhouse_remove_objects_capability_fdaabc72-1a82-4137-b6d4-08f8e5beda7c 2024.10.22 00:14:52.621263 [ 1034 ] {} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/clickhouse_remove_objects_capability_fdaabc72-1a82-4137-b6d4-08f8e5beda7c. 2024.10.22 00:14:52.966547 [ 1341 ] {} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:14:53.026293 [ 1341 ] {} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/clickhouse_remove_objects_capability_fdaabc72-1a82-4137-b6d4-08f8e5beda7c, size 4 2024.10.22 00:14:53.030311 [ 1034 ] {} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/clickhouse_remove_objects_capability_fdaabc72-1a82-4137-b6d4-08f8e5beda7c. 2024.10.22 00:14:53.060057 [ 1034 ] {} S3ObjectStorage: Objects with paths [common/clickhouse_remove_objects_capability_fdaabc72-1a82-4137-b6d4-08f8e5beda7c] were removed from S3 2024.10.22 00:14:53.446305 [ 1034 ] {} S3ObjectStorage: Objects with paths [common/dro/vnmlhksehvtnfewbxctknffaogice] were removed from S3 2024.10.22 00:14:53.447788 [ 1034 ] {} DiskObjectStorage(DiskS3): Starting up disk s3_disk 2024.10.22 00:14:53.471137 [ 1034 ] {} DiskObjectStorage(DiskS3): Disk s3_disk started up 2024.10.22 00:14:53.473101 [ 1034 ] {} KeeperContext: Disk type 'cache' is not supported for Keeper 2024.10.22 00:14:53.475028 [ 1034 ] {} KeeperContext: Disk type 'local_blob_storage' is not supported for Keeper 2024.10.22 00:14:53.478714 [ 1034 ] {} KeeperContext: Disk type 'local_blob_storage' is not supported for Keeper 2024.10.22 00:14:53.494049 [ 1034 ] {} KeeperContext: Disk type 'local_blob_storage' is not supported for Keeper 2024.10.22 00:14:53.499698 [ 1034 ] {} KeeperContext: Disk type 'cache' is not supported for Keeper 2024.10.22 00:14:53.513655 [ 1034 ] {} KeeperContext: Disk type 'cache' is not supported for Keeper 2024.10.22 00:14:53.531612 [ 1034 ] {} KeeperContext: Disk type 'cache' is not supported for Keeper 2024.10.22 00:14:53.550011 [ 1034 ] {} KeeperContext: Disk type 'cache' is not supported for Keeper 2024.10.22 00:14:53.554544 [ 1034 ] {} KeeperContext: Disk type 'cache' is not supported for Keeper 2024.10.22 00:14:53.659677 [ 1034 ] {} KeeperSnapshotManager: Reading from disk LocalSnapshotDisk 2024.10.22 00:14:53.664410 [ 1034 ] {} KeeperSnapshotManager: Found snapshot_22.bin.zstd on LocalSnapshotDisk 2024.10.22 00:14:53.745434 [ 1034 ] {} KeeperLogStore: Reading from disk LocalLogDisk 2024.10.22 00:14:53.750512 [ 1034 ] {} KeeperLogStore: Found changelog_1_100000.bin.zstd on LocalLogDisk 2024.10.22 00:14:53.754787 [ 1034 ] {} KeeperLogStore: force_sync disabled 2024.10.22 00:14:53.755546 [ 1034 ] {} KeeperDispatcher: Waiting server to initialize 2024.10.22 00:14:53.756237 [ 1034 ] {} KeeperStateMachine: Totally have 1 snapshots 2024.10.22 00:14:53.756905 [ 1034 ] {} KeeperStateMachine: Trying to load state machine from snapshot up to log index 22 2024.10.22 00:14:53.763081 [ 1034 ] {} KeeperStateMachine: Loaded snapshot with last committed log index 22 2024.10.22 00:14:53.769212 [ 1034 ] {} KeeperLogStore: Totally read from changelog changelog_1_100000.bin.zstd 22 entries 2024.10.22 00:14:53.771837 [ 1034 ] {} KeeperLogStore: Continue to write into changelog_1_100000.bin.zstd 2024.10.22 00:14:53.773957 [ 1034 ] {} KeeperServer: Preprocessing 0 log entries 2024.10.22 00:14:53.774597 [ 1034 ] {} KeeperServer: Preprocessing done 2024.10.22 00:14:53.775829 [ 1034 ] {} KeeperServer: Will use config from log store with log index 1 2024.10.22 00:14:53.795364 [ 1034 ] {} KeeperServer: Got 1000000000000000 value for setting 'reserved_log_items' which is bigger than int32_t max value, lowering value to 2147483647. 2024.10.22 00:14:53.815649 [ 1034 ] {} RaftInstance: Raft ASIO listener initiated on :::9234, unsecured 2024.10.22 00:14:53.820335 [ 1034 ] {} KeeperStateManager: Read state from /var/lib/clickhouse/coordination/state 2024.10.22 00:14:53.836600 [ 1034 ] {} RaftInstance: invalid election timeout lower bound detected, adjusted to 0 2024.10.22 00:14:53.842137 [ 1034 ] {} RaftInstance: invalid election timeout upper bound detected, adjusted to 0 2024.10.22 00:14:53.848795 [ 1034 ] {} RaftInstance: parameters: election timeout range 0 - 0, heartbeat 0, leadership expiry 0, max batch 100, backoff 50, snapshot distance 10000, enable randomized snapshot creation NO, log sync stop gap 99999, reserved logs 2147483647, client timeout 10000, auto forwarding on, API call type async, custom commit quorum size 0, custom election quorum size 0, snapshot receiver included, leadership transfer wait time 0, grace period of lagging state machine 0, snapshot IO: blocking, parallel log appending: on 2024.10.22 00:14:53.854274 [ 1034 ] {} RaftInstance: new election timeout range: 0 - 0 2024.10.22 00:14:53.858068 [ 1034 ] {} RaftInstance: === INIT RAFT SERVER === commit index 22 term 1 election timer allowed log store start 1, end 22 config log idx 1, prev log idx 0 2024.10.22 00:14:53.870725 [ 1034 ] {} RaftInstance: peer 1: DC ID 0, localhost:9234, voting member, 1 my id: 1, voting_member num peers: 0 2024.10.22 00:14:53.872865 [ 1034 ] {} RaftInstance: global manager does not exist. will use local thread for commit and append 2024.10.22 00:14:53.874060 [ 1034 ] {} RaftInstance: wait for HB, for 50 + [0, 0] ms 2024.10.22 00:14:53.883983 [ 1366 ] {} RaftInstance: bg append_entries thread initiated 2024.10.22 00:14:53.934579 [ 1034 ] {} KeeperDispatcher: Server initialized, waiting for quorum 2024.10.22 00:14:53.945042 [ 1350 ] {} RaftInstance: Election timeout, initiate leader election 2024.10.22 00:14:53.959883 [ 1350 ] {} RaftInstance: [PRIORITY] decay, target 1 -> 1, mine 1 2024.10.22 00:14:53.967286 [ 1350 ] {} RaftInstance: [ELECTION TIMEOUT] current role: follower, log last term 1, state term 1, target p 1, my p 1, hb dead, pre-vote NOT done 2024.10.22 00:14:54.072756 [ 1350 ] {} RaftInstance: pre-vote term (0) is different, reset it to 1 2024.10.22 00:14:54.080075 [ 1350 ] {} IDisk: Copying from LocalStateFileDisk (path: /var/lib/clickhouse/coordination) state to LocalStateFileDisk (path: /var/lib/clickhouse/coordination) state-OLD. 2024.10.22 00:14:54.224011 [ 1350 ] {} RaftInstance: [VOTE INIT] my id 1, my role candidate, term 2, log idx 22, log term 1, priority (target 1 / mine 1) 2024.10.22 00:14:54.262326 [ 1350 ] {} RaftInstance: number of pending commit elements: 0 2024.10.22 00:14:54.346654 [ 1350 ] {} RaftInstance: state machine commit index 22, precommit index 22, last log index 22 2024.10.22 00:14:54.412673 [ 1350 ] {} RaftInstance: [BECOME LEADER] appended new config at 23 2024.10.22 00:14:54.752200 [ 1365 ] {} RaftInstance: config at index 23 is committed, prev config log idx 1 2024.10.22 00:14:54.812030 [ 1365 ] {} RaftInstance: new config log idx 23, prev log idx 1, cur config log idx 1, prev log idx 0 2024.10.22 00:14:54.819450 [ 1365 ] {} RaftInstance: new configuration: log idx 23, prev log idx 1 peer 1, DC ID 0, localhost:9234, voting member, 1 my id: 1, leader: 1, term: 2 2024.10.22 00:14:54.843725 [ 1034 ] {} KeeperDispatcher: Quorum initialized 2024.10.22 00:14:54.846313 [ 1034 ] {} KeeperDispatcher: Dispatcher initialized 2024.10.22 00:14:54.863880 [ 1034 ] {} Application: Listening for Keeper (tcp): [::]:9181 2024.10.22 00:14:54.864979 [ 1034 ] {} Application: Listening for replica communication (interserver): http://[::]:9009 2024.10.22 00:14:55.002456 [ 1034 ] {} ConfigReloader: Loading config '/etc/clickhouse-server/users.xml' 2024.10.22 00:14:55.735639 [ 1034 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performing update on configuration 2024.10.22 00:14:58.325162 [ 1034 ] {} ConfigReloader: Loaded config '/etc/clickhouse-server/users.xml', performed update on configuration 2024.10.22 00:14:58.448912 [ 1034 ] {} Access(user directories): Added users_xml access storage 'users_xml', path: /etc/clickhouse-server/users.xml 2024.10.22 00:14:58.458767 [ 1034 ] {} Access(user directories): Added local_directory access storage 'local_directory', path: /var/lib/clickhouse/access/ 2024.10.22 00:14:59.078423 [ 1034 ] {} Context: Initialized background executor for merges and mutations with num_threads=16, num_tasks=32, scheduling_policy=round_robin 2024.10.22 00:14:59.099533 [ 1034 ] {} Context: Initialized background executor for move operations with num_threads=8, num_tasks=8 2024.10.22 00:14:59.152436 [ 1034 ] {} Context: Initialized background executor for fetches with num_threads=8, num_tasks=8 2024.10.22 00:14:59.171455 [ 1034 ] {} Context: Initialized background executor for common operations (e.g. clearing old parts) with num_threads=8, num_tasks=8 2024.10.22 00:14:59.207557 [ 1034 ] {} DNSCacheUpdater: Update period 15 seconds 2024.10.22 00:14:59.209734 [ 1034 ] {} Application: Loading metadata from /var/lib/clickhouse/ 2024.10.22 00:14:59.218265 [ 1151 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:14:59.342334 [ 1151 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:15:00.136874 [ 1034 ] {} DatabaseAtomic (system): Metadata processed, database system has 9 tables and 0 dictionaries in total. 2024.10.22 00:15:00.140617 [ 1034 ] {} TablesLoader: Parsed metadata of 9 tables in 1 databases in 0.881460959 sec 2024.10.22 00:15:00.266250 [ 1034 ] {} ReferentialDeps: No tables 2024.10.22 00:15:00.267668 [ 1034 ] {} LoadingDeps: Table system.transactions_info_log has no dependencies (level 0) 2024.10.22 00:15:00.268513 [ 1034 ] {} LoadingDeps: Table system.text_log has no dependencies (level 0) 2024.10.22 00:15:00.269254 [ 1034 ] {} LoadingDeps: Table system.zookeeper_log has no dependencies (level 0) 2024.10.22 00:15:00.270243 [ 1034 ] {} LoadingDeps: Table system.session_log has no dependencies (level 0) 2024.10.22 00:15:00.271697 [ 1034 ] {} LoadingDeps: Table system.trace_log has no dependencies (level 0) 2024.10.22 00:15:00.273961 [ 1034 ] {} LoadingDeps: Table system.metric_log has no dependencies (level 0) 2024.10.22 00:15:00.285324 [ 1034 ] {} LoadingDeps: Table system.opentelemetry_span_log has no dependencies (level 0) 2024.10.22 00:15:00.287078 [ 1034 ] {} LoadingDeps: Table system.query_log has no dependencies (level 0) 2024.10.22 00:15:00.288829 [ 1034 ] {} LoadingDeps: Table system.asynchronous_metric_log has no dependencies (level 0) 2024.10.22 00:15:00.294997 [ 1034 ] {} TablesLoader: Loading 9 tables with dependency level 0 2024.10.22 00:15:01.919829 [ 1448 ] {} WriteBufferFromAzureBlobStorage: Staged block (id: ymaknbojwrvvjrjaguejjfneltnfgsetnajqpdkzzpplzqozddfrghceedobjbna) of size 4 (blob path: ftogfrtjyidslrsmazuuogferqzundps). 2024.10.22 00:15:01.951583 [ 1448 ] {} WriteBufferFromAzureBlobStorage: Committed 1 blocks for blob `ftogfrtjyidslrsmazuuogferqzundps` 2024.10.22 00:15:02.268549 [ 1448 ] {} DiskObjectStorage(DiskAzureBlobStorage): Starting up disk azure 2024.10.22 00:15:02.270553 [ 1448 ] {} DiskObjectStorage(DiskAzureBlobStorage): Disk azure started up 2024.10.22 00:15:02.285681 [ 1448 ] {} FileCache(cached_azure): Loading filesystem cache with 1 threads 2024.10.22 00:15:02.440745 [ 1448 ] {} DiskCache: Registered cached disk (`cached_azure`) with structure: DiskObjectStorage-azure(CachedObjectStorage-cached_azure(AzureObjectStorage)) 2024.10.22 00:15:03.711152 [ 1448 ] {} S3Client: Provider type: Unknown 2024.10.22 00:15:03.711607 [ 1448 ] {} S3Client: API mode of the S3 client: AWS 2024.10.22 00:15:04.755076 [ 1448 ] {} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/clickhouse_remove_objects_capability_fdaabc72-1a82-4137-b6d4-08f8e5beda7c (LogSeriesLimiter: on interval from 2024-10-22 00:14:52 to 2024-10-22 00:15:04 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:15:04.756824 [ 1448 ] {} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/clickhouse_remove_objects_capability_fdaabc72-1a82-4137-b6d4-08f8e5beda7c. 2024.10.22 00:15:04.775050 [ 1341 ] {} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/clickhouse_remove_objects_capability_fdaabc72-1a82-4137-b6d4-08f8e5beda7c, size 4 2024.10.22 00:15:04.808412 [ 1448 ] {} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/clickhouse_remove_objects_capability_fdaabc72-1a82-4137-b6d4-08f8e5beda7c. 2024.10.22 00:15:04.827006 [ 1448 ] {} S3ObjectStorage: Objects with paths [common/clickhouse_remove_objects_capability_fdaabc72-1a82-4137-b6d4-08f8e5beda7c] were removed from S3 2024.10.22 00:15:05.137079 [ 1448 ] {} S3ObjectStorage: Objects with paths [common/fmi/spzmdpyejfpazkrgoyllksirgibeu] were removed from S3 2024.10.22 00:15:05.143513 [ 1448 ] {} DiskObjectStorage(DiskS3): Starting up disk s3_disk 2024.10.22 00:15:05.174618 [ 1448 ] {} DiskObjectStorage(DiskS3): Disk s3_disk started up 2024.10.22 00:15:05.346972 [ 1448 ] {} FileCache(s3_cache): Loading filesystem cache with 1 threads 2024.10.22 00:15:05.467235 [ 1448 ] {} DiskCache: Registered cached disk (`s3_cache`) with structure: DiskObjectStorage-s3_disk(CachedObjectStorage-s3_cache(S3ObjectStorage)) 2024.10.22 00:15:05.603689 [ 1448 ] {} DiskObjectStorage(Local): Starting up disk local_disk 2024.10.22 00:15:05.606137 [ 1448 ] {} DiskObjectStorage(Local): Disk local_disk started up 2024.10.22 00:15:05.658652 [ 1448 ] {} DiskObjectStorage(Local): Starting up disk local_disk_2 2024.10.22 00:15:05.659892 [ 1448 ] {} DiskObjectStorage(Local): Disk local_disk_2 started up 2024.10.22 00:15:05.684631 [ 1448 ] {} DiskObjectStorage(Local): Starting up disk local_disk_3 2024.10.22 00:15:05.685785 [ 1448 ] {} DiskObjectStorage(Local): Disk local_disk_3 started up 2024.10.22 00:15:05.712595 [ 1448 ] {} FileCache(local_cache): Loading filesystem cache with 1 threads 2024.10.22 00:15:05.748337 [ 1448 ] {} DiskCache: Registered cached disk (`local_cache`) with structure: DiskObjectStorage-local_disk(CachedObjectStorage-local_cache(LocalObjectStorage)) 2024.10.22 00:15:05.759534 [ 1448 ] {} FileCache(local_cache_2): Loading filesystem cache with 1 threads 2024.10.22 00:15:05.773989 [ 1448 ] {} DiskCache: Registered cached disk (`local_cache_2`) with structure: DiskObjectStorage-local_disk(CachedObjectStorage-local_cache_2(LocalObjectStorage)) 2024.10.22 00:15:05.781396 [ 1448 ] {} FileCache(local_cache_3): Loading filesystem cache with 1 threads 2024.10.22 00:15:05.833023 [ 1448 ] {} DiskCache: Registered cached disk (`local_cache_3`) with structure: DiskObjectStorage-local_disk_3(CachedObjectStorage-local_cache_3(LocalObjectStorage)) 2024.10.22 00:15:05.871610 [ 1448 ] {} FileCache(s3_cache_multi): Loading filesystem cache with 1 threads 2024.10.22 00:15:05.897786 [ 1448 ] {} DiskCache: Registered cached disk (`s3_cache_multi`) with structure: DiskObjectStorage-s3_disk(CachedObjectStorage-s3_cache_multi(CachedObjectStorage-s3_cache(S3ObjectStorage))) 2024.10.22 00:15:06.033010 [ 1448 ] {} FileCache(s3_cache_multi_2): Loading filesystem cache with 1 threads 2024.10.22 00:15:06.374182 [ 1448 ] {} DiskCache: Registered cached disk (`s3_cache_multi_2`) with structure: DiskObjectStorage-s3_disk(CachedObjectStorage-s3_cache_multi_2(CachedObjectStorage-s3_cache_multi(CachedObjectStorage-s3_cache(S3ObjectStorage)))) 2024.10.22 00:15:06.392046 [ 1448 ] {} StoragePolicy (azure_cache): Storage policy azure_cache created, total volumes 1 2024.10.22 00:15:06.394093 [ 1448 ] {} StoragePolicySelector: Storage policy `azure_cache` loaded 2024.10.22 00:15:06.419340 [ 1448 ] {} StoragePolicy (s3_cache): Storage policy s3_cache created, total volumes 1 2024.10.22 00:15:06.422989 [ 1448 ] {} StoragePolicySelector: Storage policy `s3_cache` loaded 2024.10.22 00:15:06.456440 [ 1448 ] {} StoragePolicy (s3_cache_multi): Storage policy s3_cache_multi created, total volumes 1 2024.10.22 00:15:06.457743 [ 1448 ] {} StoragePolicySelector: Storage policy `s3_cache_multi` loaded 2024.10.22 00:15:06.486788 [ 1448 ] {} StoragePolicy (local_cache): Storage policy local_cache created, total volumes 1 2024.10.22 00:15:06.500589 [ 1448 ] {} StoragePolicySelector: Storage policy `local_cache` loaded 2024.10.22 00:15:06.508291 [ 1448 ] {} StoragePolicy (local_cache_2): Storage policy local_cache_2 created, total volumes 1 2024.10.22 00:15:06.509805 [ 1448 ] {} StoragePolicySelector: Storage policy `local_cache_2` loaded 2024.10.22 00:15:06.534289 [ 1448 ] {} StoragePolicy (local_cache_3): Storage policy local_cache_3 created, total volumes 1 2024.10.22 00:15:06.548084 [ 1448 ] {} StoragePolicySelector: Storage policy `local_cache_3` loaded 2024.10.22 00:15:06.551514 [ 1448 ] {} StoragePolicy (default): Storage policy default created, total volumes 1 2024.10.22 00:15:06.556197 [ 1448 ] {} system.opentelemetry_span_log (6ccc75a8-1522-4520-a414-6e81fb929787): Loading data parts 2024.10.22 00:15:06.556993 [ 1447 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading data parts 2024.10.22 00:15:06.556964 [ 1449 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading data parts 2024.10.22 00:15:06.557393 [ 1445 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Loading data parts 2024.10.22 00:15:06.558284 [ 1443 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Loading data parts 2024.10.22 00:15:06.557386 [ 1446 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading data parts 2024.10.22 00:15:06.556857 [ 1453 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loading data parts 2024.10.22 00:15:06.559481 [ 1444 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Loading data parts 2024.10.22 00:15:06.564333 [ 1442 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Loading data parts 2024.10.22 00:15:06.579489 [ 1447 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Will load 5 parts using up to 64 threads 2024.10.22 00:15:06.585351 [ 1448 ] {} system.opentelemetry_span_log (6ccc75a8-1522-4520-a414-6e81fb929787): Will load 1 parts using up to 64 threads 2024.10.22 00:15:06.588273 [ 1445 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Will load 2 parts using up to 64 threads 2024.10.22 00:15:06.586215 [ 1502 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading Active part 202410_9_9_0 from disk default 2024.10.22 00:15:06.592650 [ 1503 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading Active part 202410_7_7_0 from disk default 2024.10.22 00:15:06.597391 [ 1444 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Will load 3 parts using up to 64 threads 2024.10.22 00:15:06.603211 [ 1442 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Will load 2 parts using up to 64 threads 2024.10.22 00:15:06.611626 [ 1446 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Will load 3 parts using up to 64 threads 2024.10.22 00:15:06.611973 [ 1453 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Will load 2 parts using up to 64 threads 2024.10.22 00:15:06.623495 [ 1496 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Loading Active part 202410_1_1_0 from disk default 2024.10.22 00:15:06.625650 [ 1500 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading Active part 202410_8_8_0 from disk default 2024.10.22 00:15:06.597576 [ 1501 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading Active part 202410_1_6_1 from disk default 2024.10.22 00:15:06.653089 [ 1499 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Loading Active part 202410_2_2_0 from disk default 2024.10.22 00:15:06.604945 [ 1490 ] {} system.opentelemetry_span_log (6ccc75a8-1522-4520-a414-6e81fb929787): Loading Active part 202410_1_1_0 from disk default 2024.10.22 00:15:06.663324 [ 1508 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Loading Active part 202410_2_2_0 from disk default 2024.10.22 00:15:06.678115 [ 1514 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Loading Active part 202410_6_6_0 from disk default 2024.10.22 00:15:06.678561 [ 1512 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loading Active part 202410_7_7_0 from disk default 2024.10.22 00:15:06.686494 [ 1513 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loading Active part 202410_1_6_1 from disk default 2024.10.22 00:15:06.685281 [ 1510 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Active part 202410_12_12_0 from disk default 2024.10.22 00:15:06.682263 [ 1507 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Active part 202410_1_11_2 from disk default 2024.10.22 00:15:06.709458 [ 1443 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Will load 4 parts using up to 64 threads 2024.10.22 00:15:06.711745 [ 1449 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Will load 3 parts using up to 64 threads 2024.10.22 00:15:06.700046 [ 1511 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Active part 202410_13_13_0 from disk default 2024.10.22 00:15:06.698437 [ 1509 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Loading Active part 202410_3_3_0 from disk default 2024.10.22 00:15:06.719705 [ 1497 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Loading Active part 202410_1_1_0 from disk default 2024.10.22 00:15:06.704374 [ 1506 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Loading Active part 202410_1_5_1 from disk default 2024.10.22 00:15:06.739502 [ 1498 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading Active part 202410_10_10_0 from disk default 2024.10.22 00:15:06.755732 [ 1504 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Loading Active part 202410_1_1_0 from disk default 2024.10.22 00:15:06.771184 [ 1517 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Active part 202410_12_12_0 from disk default 2024.10.22 00:15:06.820112 [ 1516 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Loading Active part 202410_4_4_0 from disk default 2024.10.22 00:15:06.831597 [ 1515 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Active part 202410_13_13_0 from disk default 2024.10.22 00:15:06.855053 [ 1519 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Active part 202410_1_11_2 from disk default 2024.10.22 00:15:06.867589 [ 1514 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Finished loading Active part 202410_6_6_0 on disk default 2024.10.22 00:15:06.849103 [ 1518 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Loading Active part 202410_3_3_0 from disk default 2024.10.22 00:15:06.875636 [ 1507 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Active part 202410_1_11_2 on disk default 2024.10.22 00:15:06.877164 [ 1513 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Finished loading Active part 202410_1_6_1 on disk default 2024.10.22 00:15:06.887177 [ 1502 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Finished loading Active part 202410_9_9_0 on disk default 2024.10.22 00:15:06.945499 [ 1510 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Active part 202410_12_12_0 on disk default 2024.10.22 00:15:06.954439 [ 1511 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Active part 202410_13_13_0 on disk default 2024.10.22 00:15:06.959287 [ 1503 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Finished loading Active part 202410_7_7_0 on disk default 2024.10.22 00:15:06.965506 [ 1514 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Loading Active part 202410_2_2_0 from disk default 2024.10.22 00:15:06.971242 [ 1506 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Finished loading Active part 202410_1_5_1 on disk default 2024.10.22 00:15:06.983220 [ 1501 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Finished loading Active part 202410_1_6_1 on disk default 2024.10.22 00:15:06.984765 [ 1446 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 12 outdated data parts. They will be loaded asynchronously 2024.10.22 00:15:07.002408 [ 1446 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loaded data parts (3 items) 2024.10.22 00:15:07.003467 [ 1517 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Active part 202410_12_12_0 on disk default 2024.10.22 00:15:07.009254 [ 1499 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Finished loading Active part 202410_2_2_0 on disk default 2024.10.22 00:15:07.040912 [ 1446 ] {} TablesLoader: Processed: 11.11111111111111% 2024.10.22 00:15:07.046109 [ 1496 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Finished loading Active part 202410_1_1_0 on disk default 2024.10.22 00:15:07.050482 [ 1490 ] {} system.opentelemetry_span_log (6ccc75a8-1522-4520-a414-6e81fb929787): Finished loading Active part 202410_1_1_0 on disk default 2024.10.22 00:15:07.073540 [ 1442 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 5 outdated data parts. They will be loaded asynchronously 2024.10.22 00:15:07.075006 [ 1445 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Loaded data parts (2 items) 2024.10.22 00:15:07.075108 [ 1442 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Loaded data parts (2 items) 2024.10.22 00:15:07.075674 [ 1448 ] {} system.opentelemetry_span_log (6ccc75a8-1522-4520-a414-6e81fb929787): Loaded data parts (1 items) 2024.10.22 00:15:07.119017 [ 1515 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Active part 202410_13_13_0 on disk default 2024.10.22 00:15:07.153088 [ 1512 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Finished loading Active part 202410_7_7_0 on disk default 2024.10.22 00:15:07.191511 [ 1500 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Finished loading Active part 202410_8_8_0 on disk default 2024.10.22 00:15:07.205908 [ 1453 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 outdated data parts. They will be loaded asynchronously 2024.10.22 00:15:07.207955 [ 1453 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loaded data parts (2 items) 2024.10.22 00:15:07.232516 [ 1519 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Active part 202410_1_11_2 on disk default 2024.10.22 00:15:07.277643 [ 1449 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 12 outdated data parts. They will be loaded asynchronously 2024.10.22 00:15:07.286169 [ 1449 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loaded data parts (3 items) 2024.10.22 00:15:07.302944 [ 1498 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Finished loading Active part 202410_10_10_0 on disk default 2024.10.22 00:15:07.328388 [ 1516 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Finished loading Active part 202410_4_4_0 on disk default 2024.10.22 00:15:07.356345 [ 1504 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Finished loading Active part 202410_1_1_0 on disk default 2024.10.22 00:15:07.365462 [ 1447 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 outdated data parts. They will be loaded asynchronously 2024.10.22 00:15:07.459203 [ 1518 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Finished loading Active part 202410_3_3_0 on disk default 2024.10.22 00:15:07.467117 [ 1447 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loaded data parts (5 items) 2024.10.22 00:15:07.578882 [ 1514 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Finished loading Active part 202410_2_2_0 on disk default 2024.10.22 00:15:07.626379 [ 1443 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Loaded data parts (4 items) 2024.10.22 00:15:09.573097 [ 1508 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Finished loading Active part 202410_2_2_0 on disk default 2024.10.22 00:15:09.897179 [ 1509 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Finished loading Active part 202410_3_3_0 on disk default 2024.10.22 00:15:10.561039 [ 1497 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Finished loading Active part 202410_1_1_0 on disk default 2024.10.22 00:15:10.807878 [ 1444 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Loaded data parts (3 items) 2024.10.22 00:15:11.250632 [ 1034 ] {} DatabaseAtomic (system): Starting up tables. 2024.10.22 00:15:11.306435 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading 12 outdated data parts asynchronously 2024.10.22 00:15:11.317243 [ 1512 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_11_11_0 from disk default 2024.10.22 00:15:11.316722 [ 1166 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loading 6 outdated data parts asynchronously 2024.10.22 00:15:11.319001 [ 1500 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_10_10_0 from disk default 2024.10.22 00:15:11.306560 [ 1159 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading 6 outdated data parts asynchronously 2024.10.22 00:15:11.321623 [ 1510 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_9_9_0 from disk default 2024.10.22 00:15:11.323474 [ 1519 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_8_8_0 from disk default 2024.10.22 00:15:11.334235 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Loading 5 outdated data parts asynchronously 2024.10.22 00:15:11.434160 [ 1511 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_7_7_0 from disk default 2024.10.22 00:15:11.434643 [ 1498 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loading Outdated part 202410_6_6_0 from disk default 2024.10.22 00:15:11.435037 [ 1153 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_6_1 to 202410_10_10_0 2024.10.22 00:15:11.436157 [ 1510 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_9_9_0 on disk default 2024.10.22 00:15:11.436834 [ 1500 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_10_10_0 on disk default 2024.10.22 00:15:11.440726 [ 1516 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading Outdated part 202410_6_6_0 from disk default 2024.10.22 00:15:11.441282 [ 1503 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Loading Outdated part 202410_5_5_0 from disk default 2024.10.22 00:15:11.441994 [ 1504 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_6_6_0 from disk default 2024.10.22 00:15:11.447192 [ 1518 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Loading Outdated part 202410_4_4_0 from disk default 2024.10.22 00:15:11.463592 [ 1161 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading 12 outdated data parts asynchronously 2024.10.22 00:15:11.494816 [ 1034 ] {} SystemLog: Creating system.query_log from query_log 2024.10.22 00:15:11.551159 [ 1514 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_5_5_0 from disk default 2024.10.22 00:15:11.552259 [ 1506 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Loading Outdated part 202410_3_3_0 from disk default 2024.10.22 00:15:11.554411 [ 1153 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:15:11.556588 [ 1509 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_4_4_0 from disk default 2024.10.22 00:15:11.559547 [ 1512 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_11_11_0 on disk default 2024.10.22 00:15:11.559758 [ 1449 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Loading Outdated part 202410_2_2_0 from disk default 2024.10.22 00:15:11.561725 [ 1510 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loading Outdated part 202410_5_5_0 from disk default 2024.10.22 00:15:11.563286 [ 1502 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_3_3_0 from disk default 2024.10.22 00:15:11.563854 [ 1507 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_11_11_0 from disk default 2024.10.22 00:15:11.564699 [ 1445 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Loading Outdated part 202410_1_1_0 from disk default 2024.10.22 00:15:11.541573 [ 1167 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 00:15:11.566628 [ 1448 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loading Outdated part 202410_4_4_0 from disk default 2024.10.22 00:15:11.567405 [ 1442 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading Outdated part 202410_4_4_0 from disk default 2024.10.22 00:15:11.567471 [ 1500 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_2_2_0 from disk default 2024.10.22 00:15:11.567631 [ 1447 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_10_10_0 from disk default 2024.10.22 00:15:11.570755 [ 1446 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_1_1_0 from disk default 2024.10.22 00:15:11.571141 [ 1503 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Finished loading Outdated part 202410_5_5_0 on disk default 2024.10.22 00:15:11.573529 [ 1453 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading Outdated part 202410_5_5_0 from disk default 2024.10.22 00:15:11.573541 [ 1498 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Finished loading Outdated part 202410_6_6_0 on disk default 2024.10.22 00:15:11.574747 [ 1490 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loading Outdated part 202410_3_3_0 from disk default 2024.10.22 00:15:11.575454 [ 1511 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_7_7_0 on disk default 2024.10.22 00:15:11.576587 [ 1513 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loading Outdated part 202410_1_6_1 from disk default 2024.10.22 00:15:11.575629 [ 1499 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_9_9_0 from disk default 2024.10.22 00:15:11.576813 [ 1496 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading Outdated part 202410_2_2_0 from disk default 2024.10.22 00:15:11.577027 [ 1515 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loading Outdated part 202410_1_1_0 from disk default 2024.10.22 00:15:11.577113 [ 1443 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_8_8_0 from disk default 2024.10.22 00:15:11.558101 [ 1153 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:15:11.577819 [ 1517 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loading Outdated part 202410_2_2_0 from disk default 2024.10.22 00:15:11.577998 [ 1501 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading Outdated part 202410_3_3_0 from disk default 2024.10.22 00:15:11.578175 [ 1505 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loading Outdated part 202410_1_1_0 from disk default 2024.10.22 00:15:11.580105 [ 1508 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_7_7_0 from disk default 2024.10.22 00:15:11.583808 [ 1518 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Finished loading Outdated part 202410_4_4_0 on disk default 2024.10.22 00:15:11.593211 [ 1153 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 294 ms. 2024.10.22 00:15:11.604006 [ 1519 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_8_8_0 on disk default 2024.10.22 00:15:11.704055 [ 1514 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_5_5_0 on disk default 2024.10.22 00:15:11.719449 [ 1504 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_6_6_0 on disk default 2024.10.22 00:15:11.729492 [ 1445 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Finished loading Outdated part 202410_1_1_0 on disk default 2024.10.22 00:15:11.753705 [ 1446 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_1_1_0 on disk default 2024.10.22 00:15:11.764616 [ 1506 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Finished loading Outdated part 202410_3_3_0 on disk default 2024.10.22 00:15:11.772213 [ 1510 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Finished loading Outdated part 202410_5_5_0 on disk default 2024.10.22 00:15:11.782421 [ 1499 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_9_9_0 on disk default 2024.10.22 00:15:11.786777 [ 1502 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_3_3_0 on disk default 2024.10.22 00:15:11.793765 [ 1448 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Finished loading Outdated part 202410_4_4_0 on disk default 2024.10.22 00:15:11.809745 [ 1490 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Finished loading Outdated part 202410_3_3_0 on disk default 2024.10.22 00:15:11.826373 [ 1515 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Finished loading Outdated part 202410_1_1_0 on disk default 2024.10.22 00:15:11.829609 [ 1513 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_1_6_1 on disk default 2024.10.22 00:15:11.873723 [ 1447 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_10_10_0 on disk default 2024.10.22 00:15:11.874693 [ 1516 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Finished loading Outdated part 202410_6_6_0 on disk default 2024.10.22 00:15:11.885733 [ 1517 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Finished loading Outdated part 202410_2_2_0 on disk default 2024.10.22 00:15:11.908043 [ 1443 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_8_8_0 on disk default 2024.10.22 00:15:11.910409 [ 1507 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_11_11_0 on disk default 2024.10.22 00:15:11.932406 [ 1453 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Finished loading Outdated part 202410_5_5_0 on disk default 2024.10.22 00:15:11.935334 [ 1508 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_7_7_0 on disk default 2024.10.22 00:15:11.968049 [ 1442 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Finished loading Outdated part 202410_4_4_0 on disk default 2024.10.22 00:15:11.978322 [ 1449 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Finished loading Outdated part 202410_2_2_0 on disk default 2024.10.22 00:15:11.978632 [ 1034 ] {} SystemLog: Creating system.query_thread_log from query_thread_log 2024.10.22 00:15:12.033141 [ 1500 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_2_2_0 on disk default 2024.10.22 00:15:12.048298 [ 1505 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Finished loading Outdated part 202410_1_1_0 on disk default 2024.10.22 00:15:12.061349 [ 1509 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Finished loading Outdated part 202410_4_4_0 on disk default 2024.10.22 00:15:12.085697 [ 1501 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Finished loading Outdated part 202410_3_3_0 on disk default 2024.10.22 00:15:12.158750 [ 1496 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Finished loading Outdated part 202410_2_2_0 on disk default 2024.10.22 00:15:12.304411 [ 1034 ] {} SystemLog: Creating system.part_log from part_log 2024.10.22 00:15:12.353638 [ 1034 ] {} SystemLog: Creating system.trace_log from trace_log 2024.10.22 00:15:12.383387 [ 1034 ] {} SystemLog: Creating system.crash_log from crash_log 2024.10.22 00:15:12.398068 [ 1034 ] {} SystemLog: Creating system.text_log from text_log 2024.10.22 00:15:12.435425 [ 1034 ] {} SystemLog: Creating system.metric_log from metric_log 2024.10.22 00:15:12.538689 [ 1186 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 00:15:14.130460 [ 1034 ] {} SystemLog: Creating system.filesystem_cache_log from filesystem_cache_log 2024.10.22 00:15:14.182557 [ 1034 ] {} SystemLog: Not creating system.filesystem_read_prefetches_log since corresponding section 'filesystem_read_prefetches_log' is missing from config 2024.10.22 00:15:14.192433 [ 1034 ] {} SystemLog: Creating system.asynchronous_metric_log from asynchronous_metric_log 2024.10.22 00:15:14.330386 [ 1034 ] {} SystemLog: Creating system.opentelemetry_span_log from opentelemetry_span_log 2024.10.22 00:15:14.358293 [ 1212 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:15:14.379945 [ 1212 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:15:14.585122 [ 1034 ] {} SystemLog: Creating system.query_views_log from query_views_log 2024.10.22 00:15:14.824345 [ 1034 ] {} SystemLog: Creating system.zookeeper_log from zookeeper_log 2024.10.22 00:15:14.923355 [ 1204 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 204 ms. 2024.10.22 00:15:14.925797 [ 1218 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 00:15:15.294059 [ 1034 ] {} SystemLog: Creating system.session_log from session_log 2024.10.22 00:15:15.468430 [ 1034 ] {} SystemLog: Creating system.transactions_info_log from transactions_info_log 2024.10.22 00:15:15.506850 [ 1034 ] {} SystemLog: Creating system.processors_profile_log from processors_profile_log 2024.10.22 00:15:15.574473 [ 1034 ] {} SystemLog: Creating system.asynchronous_insert_log from asynchronous_insert_log 2024.10.22 00:15:15.836460 [ 1034 ] {} Context: Table system.session_log is enabled. It's unreliable and may contain garbage. Do not use it for any kind of security monitoring. 2024.10.22 00:15:15.957719 [ 1519 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_6_6_0 from disk default 2024.10.22 00:15:15.958430 [ 1498 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_5_5_0 from disk default 2024.10.22 00:15:15.959934 [ 1511 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_3_3_0 from disk default 2024.10.22 00:15:15.972503 [ 1518 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_4_4_0 from disk default 2024.10.22 00:15:15.974663 [ 1166 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Loaded 6 outdated data parts asynchronously 2024.10.22 00:15:15.975173 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Loaded 5 outdated data parts asynchronously 2024.10.22 00:15:15.975348 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Loaded 12 outdated data parts asynchronously 2024.10.22 00:15:15.975409 [ 1159 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Loaded 6 outdated data parts asynchronously 2024.10.22 00:15:15.976629 [ 1166 ] {} MergeTreeData::loadOutdatedDataParts: Execution took 4668 ms. 2024.10.22 00:15:15.978815 [ 1159 ] {} MergeTreeData::loadOutdatedDataParts: Execution took 4673 ms. 2024.10.22 00:15:15.979508 [ 1176 ] {} MergeTreeData::loadOutdatedDataParts: Execution took 4664 ms. 2024.10.22 00:15:15.979638 [ 1152 ] {} MergeTreeData::loadOutdatedDataParts: Execution took 4674 ms. 2024.10.22 00:15:15.980090 [ 1503 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_2_2_0 from disk default 2024.10.22 00:15:16.072653 [ 1560 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_1_1_0 from disk default 2024.10.22 00:15:16.102249 [ 1518 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_4_4_0 on disk default 2024.10.22 00:15:16.110428 [ 1498 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_5_5_0 on disk default 2024.10.22 00:15:16.115092 [ 1519 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_6_6_0 on disk default 2024.10.22 00:15:16.106463 [ 1561 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loading Outdated part 202410_1_6_1 from disk default 2024.10.22 00:15:16.184262 [ 1503 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_2_2_0 on disk default 2024.10.22 00:15:16.262101 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} bool DB::(anonymous namespace)::checkPermissionsImpl(): Code: 412. DB::Exception: Can't receive Netlink response: error -2. (NETLINK_ERROR) (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 00:15:16.271954 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_6_1 to 202410_10_10_0 into Compact with storage Full 2024.10.22 00:15:16.304018 [ 1511 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_3_3_0 on disk default 2024.10.22 00:15:16.335064 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:15:16.344693 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_6_1, total 32 rows starting from the beginning of the part 2024.10.22 00:15:16.402658 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_7_7_0, total 1 rows starting from the beginning of the part 2024.10.22 00:15:16.415808 [ 1561 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_1_6_1 on disk default 2024.10.22 00:15:16.431527 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_8_8_0, total 1 rows starting from the beginning of the part 2024.10.22 00:15:16.432470 [ 1560 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Finished loading Outdated part 202410_1_1_0 on disk default 2024.10.22 00:15:16.436989 [ 1161 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Loaded 12 outdated data parts asynchronously 2024.10.22 00:15:16.442364 [ 1161 ] {} MergeTreeData::loadOutdatedDataParts: Execution took 4989 ms. 2024.10.22 00:15:16.563141 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_9_9_0, total 1 rows starting from the beginning of the part 2024.10.22 00:15:16.689793 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_10_10_0, total 1 rows starting from the beginning of the part 2024.10.22 00:15:17.332305 [ 1220 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 210 ms. 2024.10.22 00:15:18.665929 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} MergeTask::MergeProjectionsStage: Merge sorted 36 rows, containing 32 columns (32 merged, 0 gathered) in 2.620809154 sec., 13.73621575804371 rows/sec., 2.40 KiB/sec. 2024.10.22 00:15:18.823313 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} MergedBlockOutputStream: filled checksums 202410_1_10_2 (state Temporary) 2024.10.22 00:15:18.981284 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_10_2 to 202410_1_10_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:15:18.992030 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_10_2} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_6_1, 202410_10_10_0] -> 202410_1_10_2 2024.10.22 00:15:19.014152 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.27 MiB. 2024.10.22 00:15:19.994773 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 414 entries to flush up to offset 414 2024.10.22 00:15:20.069898 [ 1548 ] {} SystemLog (system.text_log): Will use existing table system.text_log for TextLog 2024.10.22 00:15:20.245673 [ 1274 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 00:15:21.072889 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:15:21.076003 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:15:21.260356 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1_1_0 (state Temporary) 2024.10.22 00:15:21.400754 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1_1_0 to 202410_14_14_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:15:21.409964 [ 1275 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 200 ms. 2024.10.22 00:15:21.465291 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 414 2024.10.22 00:15:23.493353 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 8 entries to flush up to offset 8 2024.10.22 00:15:23.566403 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 46 entries to flush up to offset 46 2024.10.22 00:15:23.760632 [ 1556 ] {} SystemLog (system.transactions_info_log): Will use existing table system.transactions_info_log for TransactionsInfoLog 2024.10.22 00:15:24.430619 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:15:24.435109 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:15:24.525662 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_1_1_0 (state Temporary) 2024.10.22 00:15:24.901364 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_1_1_0 to 202410_14_14_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:15:24.972610 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 46 2024.10.22 00:15:25.103103 [ 1034 ] {} DatabaseCatalog: Found 0 partially dropped tables. Will load them and retry removal. 2024.10.22 00:15:25.143671 [ 1177 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 206 ms. 2024.10.22 00:15:25.387850 [ 1549 ] {} SystemLog (system.metric_log): Will use existing table system.metric_log for MetricLog 2024.10.22 00:15:25.463276 [ 1034 ] {} DatabaseOrdinary (datasets): Metadata processed, database datasets has 2 tables and 0 dictionaries in total. 2024.10.22 00:15:25.466804 [ 1034 ] {} DatabaseAtomic (default): Metadata processed, database default has 0 tables and 0 dictionaries in total. 2024.10.22 00:15:25.470872 [ 1034 ] {} DatabaseAtomic (test): Metadata processed, database test has 0 tables and 0 dictionaries in total. 2024.10.22 00:15:25.472110 [ 1034 ] {} TablesLoader: Parsed metadata of 2 tables in 3 databases in 0.21244685 sec 2024.10.22 00:15:25.476664 [ 1034 ] {} ReferentialDeps: No tables 2024.10.22 00:15:25.478231 [ 1034 ] {} LoadingDeps: Table datasets.visits_v1 has no dependencies (level 0) 2024.10.22 00:15:25.478957 [ 1034 ] {} LoadingDeps: Table datasets.hits_v1 has no dependencies (level 0) 2024.10.22 00:15:25.482766 [ 1034 ] {} TablesLoader: Loading 2 tables with dependency level 0 2024.10.22 00:15:25.502636 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 441 2024.10.22 00:15:25.901764 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:15:25.910757 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:15:26.178507 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_2_2_0 (state Temporary) 2024.10.22 00:15:26.217491 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_2_2_0 to 202410_15_15_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:15:26.388235 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 441 2024.10.22 00:15:26.706388 [ 1513 ] {} datasets.hits_v1: Loading data parts 2024.10.22 00:15:26.742768 [ 1513 ] {} datasets.hits_v1: Will load 1 parts using up to 64 threads 2024.10.22 00:15:26.760501 [ 1446 ] {} datasets.hits_v1: Loading Active part 201403_10_18_2 from disk default 2024.10.22 00:15:26.878873 [ 1502 ] {} datasets.visits_v1: Loading data parts 2024.10.22 00:15:27.054820 [ 1502 ] {} datasets.visits_v1: Will load 1 parts using up to 64 threads 2024.10.22 00:15:27.063250 [ 1515 ] {} datasets.visits_v1: Loading Active part 20140317_20140323_3_4_1 from disk default 2024.10.22 00:15:27.504035 [ 1446 ] {} datasets.hits_v1: Finished loading Active part 201403_10_18_2 on disk default 2024.10.22 00:15:27.651741 [ 1513 ] {} datasets.hits_v1: Loaded data parts (1 items) 2024.10.22 00:15:27.986841 [ 1515 ] {} datasets.visits_v1: Finished loading Active part 20140317_20140323_3_4_1 on disk default 2024.10.22 00:15:28.131361 [ 1502 ] {} datasets.visits_v1: Loaded data parts (1 items) 2024.10.22 00:15:28.239593 [ 1034 ] {} DatabaseOrdinary (datasets): Starting up tables. 2024.10.22 00:15:28.403955 [ 1034 ] {} DatabaseAtomic (default): Starting up tables. 2024.10.22 00:15:28.405068 [ 1034 ] {} DatabaseAtomic (test): Starting up tables. 2024.10.22 00:15:28.561038 [ 1034 ] {} UserDefinedSQLObjectsLoaderFromDisk: Loading user defined objects from /var/lib/clickhouse/user_defined/ 2024.10.22 00:15:28.664817 [ 1034 ] {} UserDefinedSQLObjectsLoaderFromDisk: User defined objects loaded 2024.10.22 00:15:28.666196 [ 1034 ] {} Application: Loaded metadata. 2024.10.22 00:15:28.668170 [ 1034 ] {} Pipe: Pipe capacity is 1.00 MiB 2024.10.22 00:15:28.669491 [ 1034 ] {} Application: Tasks stats provider: procfs 2024.10.22 00:15:29.328196 [ 1034 ] {} AsynchronousMetrics: MemoryTracking: was 71.26 MiB, peak 75.21 MiB, free memory in arenas 5.96 MiB, will set to 810.40 MiB (RSS), difference: 739.14 MiB 2024.10.22 00:15:29.384355 [ 1180 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:15:29.417091 [ 1180 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:15:29.561489 [ 1034 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.151765276 sec. 2024.10.22 00:15:29.756492 [ 1034 ] {} ExternalDictionariesLoader: Loading config file '/etc/clickhouse-server/decimals_dictionary.xml'. 2024.10.22 00:15:29.966773 [ 1034 ] {} ExternalDictionariesLoader: Loading config file '/etc/clickhouse-server/executable_dictionary.xml'. 2024.10.22 00:15:30.002265 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 148 entries to flush up to offset 148 2024.10.22 00:15:30.137210 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Will use existing table system.asynchronous_metric_log for AsynchronousMetricLog 2024.10.22 00:15:30.185753 [ 1034 ] {} ExternalDictionariesLoader: Loading config file '/etc/clickhouse-server/executable_pool_dictionary.xml'. 2024.10.22 00:15:30.271760 [ 1034 ] {} ExternalDictionariesLoader: Loading config file '/etc/clickhouse-server/ints_dictionary.xml'. 2024.10.22 00:15:30.390926 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 476 2024.10.22 00:15:30.394304 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:15:30.397943 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:15:30.472945 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_1_1_0 (state Temporary) 2024.10.22 00:15:30.600101 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_1_1_0 to 202410_8_8_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:15:30.647719 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 148 2024.10.22 00:15:30.861161 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:15:30.863355 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:15:30.866594 [ 1034 ] {} ExternalDictionariesLoader: Loading config file '/etc/clickhouse-server/strings_dictionary.xml'. 2024.10.22 00:15:30.885380 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 842.02 MiB, peak 845.10 MiB, free memory in arenas 2.03 MiB, will set to 818.52 MiB (RSS), difference: -23.50 MiB 2024.10.22 00:15:31.105161 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_3_3_0 (state Temporary) 2024.10.22 00:15:31.178322 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 17 entries to flush up to offset 17 2024.10.22 00:15:31.203092 [ 1034 ] {} ExternalUserDefinedExecutableFunctionsLoader: Loading config file '/etc/clickhouse-server/test_function.xml'. 2024.10.22 00:15:31.208405 [ 1546 ] {} SystemLog (system.trace_log): Will use existing table system.trace_log for TraceLog 2024.10.22 00:15:31.232558 [ 1034 ] {} ExternalUserDefinedExecutableFunctionsLoader: Will load 'test_function' because always_load_everything flag is set. 2024.10.22 00:15:31.234168 [ 1034 ] {} ExternalUserDefinedExecutableFunctionsLoader: Will load the object 'test_function' immediately, force = false, loading_id = 1 2024.10.22 00:15:31.235605 [ 1034 ] {} ExternalUserDefinedExecutableFunctionsLoader: Start loading object 'test_function' 2024.10.22 00:15:31.251114 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_3_3_0 to 202410_16_16_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:15:31.251521 [ 1034 ] {} ExternalUserDefinedExecutableFunctionsLoader: Supposed update time for 'test_function' is never (loaded, lifetime 0) 2024.10.22 00:15:31.253013 [ 1034 ] {} ExternalUserDefinedExecutableFunctionsLoader: Next update time for 'test_function' was set to 294247-01-10 05:00:54 2024.10.22 00:15:31.275426 [ 1445 ] {} DDLWorker: Starting DDLWorker thread 2024.10.22 00:15:31.276711 [ 1445 ] {} DDLWorker: Initializing DDLWorker thread 2024.10.22 00:15:31.276863 [ 1034 ] {} Application: Listening for http://[::]:8123 2024.10.22 00:15:31.276607 [ 1504 ] {} DDLWorker: Started DDLWorker cleanup thread 2024.10.22 00:15:31.278910 [ 1034 ] {} Application: Listening for https://[::]:8443 2024.10.22 00:15:31.282072 [ 1034 ] {} Application: Listening for native protocol (tcp): [::]:9000 2024.10.22 00:15:31.286295 [ 1034 ] {} Application: Listening for native protocol (tcp) with PROXY: [::]:9010 2024.10.22 00:15:31.288114 [ 1034 ] {} Application: Listening for secure native protocol (tcp_secure): [::]:9440 2024.10.22 00:15:31.288182 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 476 2024.10.22 00:15:31.289665 [ 1034 ] {} Application: Listening for MySQL compatibility protocol: [::]:9004 2024.10.22 00:15:31.292579 [ 1034 ] {} Application: Listening for PostgreSQL compatibility protocol: [::]:9005 2024.10.22 00:15:31.295537 [ 1034 ] {} Application: Listening for Prometheus: http://[::]:9988 2024.10.22 00:15:31.295810 [ 1229 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_11_2 to 202410_16_16_0 2024.10.22 00:15:31.331840 [ 1229 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:15:31.340985 [ 1229 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:15:31.359308 [ 1039 ] {} TCPHandlerFactory: TCP Request. Address: [::1]:33444 2024.10.22 00:15:31.372803 [ 1040 ] {} KeeperTCPHandlerFactory: Keeper request. Address: [::1]:41272 2024.10.22 00:15:31.394525 [ 1039 ] {} TCPHandler: Connected ClickHouse client version 23.8.0, revision: 54465, user: default. 2024.10.22 00:15:31.406490 [ 1039 ] {} TCP-Session: ff6d12eb-0d18-4847-ab27-7d335cc11f39 Authenticating user 'default' from [::1]:33444 2024.10.22 00:15:31.411124 [ 1039 ] {} TCP-Session: ff6d12eb-0d18-4847-ab27-7d335cc11f39 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:15:31.413173 [ 1039 ] {} TCP-Session: ff6d12eb-0d18-4847-ab27-7d335cc11f39 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:15:31.414005 [ 1040 ] {} KeeperTCPHandler: Requesting session ID for the new client 2024.10.22 00:15:31.437808 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:15:31.471665 [ 1365 ] {} KeeperStateMachine: Session ID response 3 with timeout 30000 2024.10.22 00:15:31.473087 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_11_2 to 202410_16_16_0 into Compact with storage Full 2024.10.22 00:15:31.500524 [ 1040 ] {} KeeperTCPHandler: Received session ID 3 2024.10.22 00:15:31.502890 [ 1445 ] {} ZooKeeperClient: Connected to ZooKeeper at [::1]:9181 with session_id 3 2024.10.22 00:15:31.503997 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:15:31.522501 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_11_2, total 655 rows starting from the beginning of the part 2024.10.22 00:15:31.526492 [ 1445 ] {} ZooKeeperClient: Keeper feature flag FILTERED_LIST: enabled 2024.10.22 00:15:31.527676 [ 1445 ] {} ZooKeeperClient: Keeper feature flag MULTI_READ: enabled 2024.10.22 00:15:31.529508 [ 1445 ] {} ZooKeeperClient: Keeper feature flag CHECK_NOT_EXISTS: enabled 2024.10.22 00:15:31.532595 [ 1445 ] {} ZooKeeper: Initialized, hosts: localhost:9181 2024.10.22 00:15:31.537676 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 1 2024.10.22 00:15:31.538205 [ 1034 ] {} ZooKeeperClient: ZK send fault: 0.002% 2024.10.22 00:15:31.544260 [ 1034 ] {} ZooKeeperClient: ZK recv fault: 0.002% 2024.10.22 00:15:31.545579 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_12_12_0, total 145 rows starting from the beginning of the part 2024.10.22 00:15:31.546104 [ 1034 ] {} ZooKeeperClient: ZK send sleep: 0.001% -> 10000ms 2024.10.22 00:15:31.550357 [ 1034 ] {} Application: Ready for connections. 2024.10.22 00:15:31.560747 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_13_13_0, total 31 rows starting from the beginning of the part 2024.10.22 00:15:31.570175 [ 1039 ] {} ContextAccess (default): Settings: readonly = 0, allow_ddl = true, allow_introspection_functions = false 2024.10.22 00:15:31.573421 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 89 2024.10.22 00:15:31.573901 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_14_14_0, total 414 rows starting from the beginning of the part 2024.10.22 00:15:31.574148 [ 1039 ] {} ContextAccess (default): List of all grants: GRANT ALL ON *.* WITH GRANT OPTION 2024.10.22 00:15:31.574976 [ 1039 ] {} ContextAccess (default): List of all grants including implicit: GRANT ALL ON *.* WITH GRANT OPTION 2024.10.22 00:15:31.583632 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_15_15_0, total 27 rows starting from the beginning of the part 2024.10.22 00:15:31.590682 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:15:31.593034 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:15:31.609581 [ 1445 ] {} DDLWorker: Initialized DDLWorker thread 2024.10.22 00:15:31.610898 [ 1445 ] {} DDLWorker: Scheduling tasks 2024.10.22 00:15:31.611577 [ 1504 ] {} DDLWorker: Cleaning queue 2024.10.22 00:15:31.612241 [ 1445 ] {} DDLWorker: Don't have unfinished tasks after restarting 2024.10.22 00:15:31.624697 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_16_16_0, total 35 rows starting from the beginning of the part 2024.10.22 00:15:31.634723 [ 1445 ] {} DDLWorker: scheduleTasks: initialized=true, size_before_filtering=0, queue_size=0, entries=none..none, first_failed_task_name=none, current_tasks_size=0, last_current_task=none, last_skipped_entry_name=none 2024.10.22 00:15:31.637051 [ 1445 ] {} DDLWorker: No tasks to schedule 2024.10.22 00:15:31.638316 [ 1445 ] {} DDLWorker: Waiting for queue updates 2024.10.22 00:15:31.676708 [ 1039 ] {} TCP-Session: ff6d12eb-0d18-4847-ab27-7d335cc11f39 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default 2024.10.22 00:15:31.710014 [ 1039 ] {dcdcc51f-d927-4dc3-a029-8cdff9b0d306} executeQuery: (from [::1]:33444) SELECT 1 (stage: Complete) 2024.10.22 00:15:31.739171 [ 1039 ] {dcdcc51f-d927-4dc3-a029-8cdff9b0d306} ContextAccess (default): Access granted: SELECT(dummy) ON system.one 2024.10.22 00:15:31.748174 [ 1039 ] {dcdcc51f-d927-4dc3-a029-8cdff9b0d306} InterpreterSelectQuery: FetchColumns -> Complete 2024.10.22 00:15:31.775566 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_1_1_0 (state Temporary) 2024.10.22 00:15:31.846384 [ 1039 ] {dcdcc51f-d927-4dc3-a029-8cdff9b0d306} executeQuery: Read 1 rows, 1.00 B in 0.130998 sec., 7.6337043313638375 rows/sec., 7.63 B/sec. 2024.10.22 00:15:31.854154 [ 1039 ] {dcdcc51f-d927-4dc3-a029-8cdff9b0d306} TCPHandler: Processed in 0.181454665 sec. 2024.10.22 00:15:32.059892 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_1_1_0 to 202410_7_7_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:15:32.098321 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 17 2024.10.22 00:15:32.099532 [ 1039 ] {} TCPHandler: Done processing connection. 2024.10.22 00:15:32.146351 [ 1039 ] {} TCP-Session: ff6d12eb-0d18-4847-ab27-7d335cc11f39 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:19:59.840786 [ 1040 ] {} KeeperTCPHandler: Session #3 expired 2024.10.22 00:19:59.849166 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 6 entries to flush up to offset 23 2024.10.22 00:19:59.850351 [ 1497 ] {} SystemLog (system.query_log): Flushing system log, 2 entries to flush up to offset 2 2024.10.22 00:19:59.851441 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 51 2024.10.22 00:19:59.851248 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 564 2024.10.22 00:19:59.851043 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 384 entries to flush up to offset 532 2024.10.22 00:19:59.866218 [ 1039 ] {} TCPHandlerFactory: TCP Request. Address: [::1]:50320 2024.10.22 00:19:59.867648 [ 1370 ] {} KeeperDispatcher: Found dead session 3, will try to close it 2024.10.22 00:19:59.867007 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 10 entries to flush up to offset 10 2024.10.22 00:19:59.871418 [ 1039 ] {} TCPHandler: Connected ClickHouse client version 23.8.0, revision: 54465, user: default. 2024.10.22 00:19:59.872838 [ 1039 ] {} TCP-Session: 7be04cf3-5ed7-4d18-bb41-beed3ef4b8c8 Authenticating user 'default' from [::1]:50320 2024.10.22 00:19:59.875884 [ 1555 ] {} SystemLog (system.session_log): Flushing system log, 2 entries to flush up to offset 2 2024.10.22 00:19:59.884850 [ 1039 ] {} TCP-Session: 7be04cf3-5ed7-4d18-bb41-beed3ef4b8c8 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:19:59.886938 [ 1039 ] {} TCP-Session: 7be04cf3-5ed7-4d18-bb41-beed3ef4b8c8 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:19:59.893802 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:19:59.895426 [ 1370 ] {} KeeperDispatcher: Dead session close request pushed 2024.10.22 00:19:59.903746 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_16_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:19:59.934517 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_1_1_0] 2024.10.22 00:19:59.942221 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_1_1_0] 2024.10.22 00:19:59.970012 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_1_1_0] 2024.10.22 00:19:59.972361 [ 1220 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:19:59.982354 [ 1555 ] {} SystemLog (system.session_log): Will use existing table system.session_log for SessionLog 2024.10.22 00:20:00.012375 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:00.018803 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:00.030928 [ 1220 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:20:00.032060 [ 1039 ] {} TCP-Session: 7be04cf3-5ed7-4d18-bb41-beed3ef4b8c8 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default 2024.10.22 00:20:00.045157 [ 1039 ] {52371b60-6715-4ddf-9d51-6e5b68acf5bd} executeQuery: (from [::1]:50320) SELECT 'Connected to clickhouse-server after attaching gdb' (stage: Complete) 2024.10.22 00:20:00.054949 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:00.058107 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:00.068844 [ 1039 ] {52371b60-6715-4ddf-9d51-6e5b68acf5bd} ContextAccess (default): Access granted: SELECT(dummy) ON system.one 2024.10.22 00:20:00.072255 [ 1039 ] {52371b60-6715-4ddf-9d51-6e5b68acf5bd} InterpreterSelectQuery: FetchColumns -> Complete 2024.10.22 00:20:00.077548 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 829.67 MiB, peak 845.10 MiB, free memory in arenas 1.04 MiB, will set to 843.75 MiB (RSS), difference: 14.09 MiB 2024.10.22 00:20:00.079504 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_2_2_0 (state Temporary) 2024.10.22 00:20:00.142543 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 00:20:00.145718 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:00.151843 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:00.183422 [ 1039 ] {52371b60-6715-4ddf-9d51-6e5b68acf5bd} executeQuery: Read 1 rows, 1.00 B in 0.136872 sec., 7.30609620667485 rows/sec., 7.31 B/sec. 2024.10.22 00:20:00.188882 [ 1497 ] {} SystemLog (system.query_log): Will use existing table system.query_log for QueryLog 2024.10.22 00:20:00.203542 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:00.206816 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_2_2_0 (state Temporary) 2024.10.22 00:20:00.208563 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:00.219177 [ 1554 ] {} SystemLog (system.zookeeper_log): Will use existing table system.zookeeper_log for ZooKeeperLog 2024.10.22 00:20:00.222290 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_2_2_0 to 202410_8_8_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:00.228438 [ 1039 ] {52371b60-6715-4ddf-9d51-6e5b68acf5bd} TCPHandler: Processed in 0.198147947 sec. 2024.10.22 00:20:00.288168 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 23 2024.10.22 00:20:00.299317 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_2_2_0 to 202410_15_15_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:00.314188 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_4_4_0 (state Temporary) 2024.10.22 00:20:00.366787 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 51 2024.10.22 00:20:00.408929 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_4_4_0 to 202410_17_17_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:00.487827 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 564 2024.10.22 00:20:00.527095 [ 1039 ] {} TCPHandler: Done processing connection. 2024.10.22 00:20:00.557390 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_2_2_0 (state Temporary) 2024.10.22 00:20:00.562515 [ 1039 ] {} TCP-Session: 7be04cf3-5ed7-4d18-bb41-beed3ef4b8c8 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:00.566238 [ 1555 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:00.661521 [ 1555 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:00.778955 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_2_2_0 to 202410_9_9_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:00.779474 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} MergeTask::MergeProjectionsStage: Merge sorted 1307 rows, containing 13 columns (13 merged, 0 gathered) in 269.317671557 sec., 4.8530049752913405 rows/sec., 725.95 B/sec. 2024.10.22 00:20:00.831844 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 532 2024.10.22 00:20:00.840518 [ 1555 ] {} MergedBlockOutputStream: filled checksums 202410_1_1_0 (state Temporary) 2024.10.22 00:20:00.874255 [ 1039 ] {} TCPHandlerFactory: TCP Request. Address: [::1]:39718 2024.10.22 00:20:00.879707 [ 1039 ] {} TCPHandler: Connected ClickHouse client version 23.8.0, revision: 54465, user: default. 2024.10.22 00:20:00.883303 [ 1039 ] {} TCP-Session: 27dbb03e-ab9b-4218-aa85-682701a88c1b Authenticating user 'default' from [::1]:39718 2024.10.22 00:20:00.894902 [ 1039 ] {} TCP-Session: 27dbb03e-ab9b-4218-aa85-682701a88c1b Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:00.899451 [ 1039 ] {} TCP-Session: 27dbb03e-ab9b-4218-aa85-682701a88c1b Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:00.965039 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} MergedBlockOutputStream: filled checksums 202410_1_16_3 (state Temporary) 2024.10.22 00:20:00.974909 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:00.977272 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:00.981891 [ 1555 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Renaming temporary part tmp_insert_202410_1_1_0 to 202410_3_3_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:01.052150 [ 1555 ] {} SystemLog (system.session_log): Flushed system log up to offset 2 2024.10.22 00:20:01.119525 [ 1234 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 00:20:01.122703 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_16_3 to 202410_1_16_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:01.129745 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_16_3} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_11_2, 202410_16_16_0] -> 202410_1_16_3 2024.10.22 00:20:01.142760 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_1_1_0 (state Temporary) 2024.10.22 00:20:01.166111 [ 1039 ] {} TCP-Session: 27dbb03e-ab9b-4218-aa85-682701a88c1b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default 2024.10.22 00:20:01.175789 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:20:01.184058 [ 1039 ] {e5526597-7f39-42da-bac7-aa3d44703869} executeQuery: (from [::1]:39718) SHOW TABLES FROM datasets (stage: Complete) 2024.10.22 00:20:01.203538 [ 1039 ] {e5526597-7f39-42da-bac7-aa3d44703869} executeQuery: (internal) SELECT name FROM system.tables WHERE database = 'datasets' ORDER BY name (stage: Complete) 2024.10.22 00:20:01.258262 [ 1039 ] {e5526597-7f39-42da-bac7-aa3d44703869} ContextAccess (default): Access granted: SELECT(database, name) ON system.tables 2024.10.22 00:20:01.340916 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_1_1_0 to 202410_11_11_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:01.383951 [ 1039 ] {e5526597-7f39-42da-bac7-aa3d44703869} InterpreterSelectQuery: FetchColumns -> Complete 2024.10.22 00:20:01.413419 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 10 2024.10.22 00:20:01.415338 [ 1498 ] {e5526597-7f39-42da-bac7-aa3d44703869} ContextAccess (default): Access granted: SHOW TABLES ON *.* 2024.10.22 00:20:01.702831 [ 1497 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:01.714361 [ 1497 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:01.716238 [ 1039 ] {e5526597-7f39-42da-bac7-aa3d44703869} executeQuery: Read 2 rows, 68.00 B in 0.511009 sec., 3.9138253925077637 rows/sec., 133.07 B/sec. 2024.10.22 00:20:01.729196 [ 1039 ] {e5526597-7f39-42da-bac7-aa3d44703869} TCPHandler: Processed in 0.572499665 sec. 2024.10.22 00:20:01.752079 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 840.83 MiB, peak 852.15 MiB, free memory in arenas 1.05 MiB, will set to 946.57 MiB (RSS), difference: 105.74 MiB 2024.10.22 00:20:01.774659 [ 1039 ] {} TCPHandler: Done processing connection. 2024.10.22 00:20:01.849022 [ 1039 ] {} TCP-Session: 27dbb03e-ab9b-4218-aa85-682701a88c1b Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:01.969168 [ 1497 ] {} MergedBlockOutputStream: filled checksums 202410_1_1_0 (state Temporary) 2024.10.22 00:20:02.162994 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.084617016 sec. 2024.10.22 00:20:02.205215 [ 1039 ] {} TCPHandlerFactory: TCP Request. Address: [::1]:39728 2024.10.22 00:20:02.212234 [ 1039 ] {} TCPHandler: Connected ClickHouse client version 23.8.0, revision: 54465, user: default. 2024.10.22 00:20:02.214380 [ 1039 ] {} TCP-Session: 5bc316b5-e5ef-41b8-ae7c-40207fbfada3 Authenticating user 'default' from [::1]:39728 2024.10.22 00:20:02.219957 [ 1039 ] {} TCP-Session: 5bc316b5-e5ef-41b8-ae7c-40207fbfada3 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:02.222493 [ 1039 ] {} TCP-Session: 5bc316b5-e5ef-41b8-ae7c-40207fbfada3 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:02.339836 [ 1447 ] {} ZooKeeperClient: Code: 210. DB::NetException: Connection reset by peer, while reading from socket ([::1]:9181). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type>, String&&, String&&) @ 0x0000000013cf152c in /usr/bin/clickhouse 6. /build/src/IO/ReadBufferFromPocoSocket.cpp:86: DB::ReadBufferFromPocoSocket::nextImpl() @ 0x0000000013cf07a0 in /usr/bin/clickhouse 7. /build/src/IO/ReadBuffer.h:71: DB::ReadBuffer::next() @ 0x000000000b21bd7c in /usr/bin/clickhouse 8. /build/src/IO/ReadBuffer.h:105: DB::ReadBuffer::read(char*, unsigned long) @ 0x000000000b21b973 in /usr/bin/clickhouse 9. /build/src/IO/ReadBuffer.h:196: DB::ReadBuffer::readStrict(char*, unsigned long) @ 0x000000000b5e4468 in /usr/bin/clickhouse 10. /build/src/IO/ReadHelpers.h:117: void DB::readPODBinary(int&, DB::ReadBuffer&) @ 0x0000000013999f02 in /usr/bin/clickhouse 11. /build/src/IO/ReadHelpers.h:1099: void DB::readBinary(int&, DB::ReadBuffer&) @ 0x0000000013999ebd in /usr/bin/clickhouse 12. /build/src/IO/ReadHelpers.h:1136: void DB::readBinaryEndian<(std::endian)64206, int>(int&, DB::ReadBuffer&) @ 0x000000001e60277d in /usr/bin/clickhouse 13. /build/src/IO/ReadHelpers.h:1149: void DB::readBinaryBigEndian(int&, DB::ReadBuffer&) @ 0x000000001e60273d in /usr/bin/clickhouse 14. /build/src/Common/ZooKeeper/ZooKeeperIO.h:51: void Coordination::read(int&, DB::ReadBuffer&) @ 0x000000001e5fdbbd in /usr/bin/clickhouse 15. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:284: void Coordination::ZooKeeper::read(int&) @ 0x000000001f30be14 in /usr/bin/clickhouse 16. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:744: Coordination::ZooKeeper::receiveEvent() @ 0x000000001f30124c in /usr/bin/clickhouse 17. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:710: Coordination::ZooKeeper::receiveThread() @ 0x000000001f300cc4 in /usr/bin/clickhouse 18. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:356: Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1::operator()() const @ 0x000000001f306af8 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&>()()) std::__invoke[abi:v15000]> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&) @ 0x000000001f306ad5 in /usr/bin/clickhouse 20. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001f306ab9 in /usr/bin/clickhouse 21. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&, std::tuple<>&) @ 0x000000001f306a1d in /usr/bin/clickhouse 22. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'()::operator()() @ 0x000000001f306945 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'()&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&) @ 0x000000001f3068b5 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'()&) @ 0x000000001f306895 in /usr/bin/clickhouse 25. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001f306875 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001f306840 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 29. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl::worker(std::__list_iterator) @ 0x0000000013a5c843 in /usr/bin/clickhouse 30. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a64ae4 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&) @ 0x0000000013a64a95 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 00:20:02.349427 [ 1447 ] {} ZooKeeperClient: Finalizing session 3. finalization_started: false, queue_finished: false, reason: 'Exception in receiveThread' 2024.10.22 00:20:02.353227 [ 1039 ] {} TCP-Session: 5bc316b5-e5ef-41b8-ae7c-40207fbfada3 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default 2024.10.22 00:20:02.370279 [ 1039 ] {8b4d7dd7-8fcb-4578-aecd-011927bba7a1} executeQuery: Query span trace_id for opentelemetry log: 6d4022c3-4e16-a653-7a39-57e3b5f44841 2024.10.22 00:20:02.377223 [ 1039 ] {8b4d7dd7-8fcb-4578-aecd-011927bba7a1} executeQuery: (from [::1]:39728) SHOW TABLES FROM test (stage: Complete) 2024.10.22 00:20:02.402753 [ 1039 ] {8b4d7dd7-8fcb-4578-aecd-011927bba7a1} executeQuery: (internal) SELECT name FROM system.tables WHERE database = 'test' ORDER BY name (stage: Complete) 2024.10.22 00:20:02.441128 [ 1039 ] {8b4d7dd7-8fcb-4578-aecd-011927bba7a1} ContextAccess (default): Access granted: SELECT(database, name) ON system.tables 2024.10.22 00:20:02.461046 [ 1039 ] {8b4d7dd7-8fcb-4578-aecd-011927bba7a1} InterpreterSelectQuery: FetchColumns -> Complete 2024.10.22 00:20:02.517196 [ 1509 ] {8b4d7dd7-8fcb-4578-aecd-011927bba7a1} ContextAccess (default): Access granted: SHOW TABLES ON *.* 2024.10.22 00:20:02.694565 [ 1497 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Renaming temporary part tmp_insert_202410_1_1_0 to 202410_5_5_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:02.811760 [ 1039 ] {8b4d7dd7-8fcb-4578-aecd-011927bba7a1} TCPHandler: Processed in 0.464934239 sec. 2024.10.22 00:20:02.851783 [ 1039 ] {} TCPHandler: Done processing connection. 2024.10.22 00:20:02.896375 [ 1039 ] {} TCP-Session: 5bc316b5-e5ef-41b8-ae7c-40207fbfada3 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:02.920563 [ 1497 ] {} SystemLog (system.query_log): Flushed system log up to offset 2 2024.10.22 00:20:03.257602 [ 1039 ] {} TCPHandlerFactory: TCP Request. Address: [::1]:39734 2024.10.22 00:20:03.262704 [ 1039 ] {} TCPHandler: Connected ClickHouse client version 23.8.0, revision: 54465, user: default. 2024.10.22 00:20:03.263982 [ 1039 ] {} TCP-Session: c1fd3d7a-7065-4280-984e-ece79b989fe2 Authenticating user 'default' from [::1]:39734 2024.10.22 00:20:03.267623 [ 1039 ] {} TCP-Session: c1fd3d7a-7065-4280-984e-ece79b989fe2 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:03.269469 [ 1039 ] {} TCP-Session: c1fd3d7a-7065-4280-984e-ece79b989fe2 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:03.518347 [ 1039 ] {} TCP-Session: c1fd3d7a-7065-4280-984e-ece79b989fe2 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default 2024.10.22 00:20:03.586473 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 950.62 MiB, peak 950.62 MiB, free memory in arenas 2.06 MiB, will set to 953.09 MiB (RSS), difference: 2.47 MiB 2024.10.22 00:20:03.621013 [ 1505 ] {} ZooKeeperClient: Code: 210. DB::NetException: I/O error: Broken pipe, while writing to socket ([::1]:41272 -> [::1]:9181). (NETWORK_ERROR), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/NetException.h:26: DB::NetException::NetException(int, FormatStringHelperImpl::type, std::type_identity::type, std::type_identity::type>, String&&, String&&, String&&) @ 0x0000000013cf30a8 in /usr/bin/clickhouse 6. /build/src/IO/WriteBufferFromPocoSocket.cpp:107: DB::WriteBufferFromPocoSocket::nextImpl() @ 0x0000000013cf26c4 in /usr/bin/clickhouse 7. /build/src/IO/WriteBuffer.h:48: DB::WriteBuffer::next() @ 0x000000000b207e8d in /usr/bin/clickhouse 8. /build/src/Common/ZooKeeper/ZooKeeperCommon.cpp:51: Coordination::ZooKeeperRequest::write(DB::WriteBuffer&) const @ 0x000000001f2cf9e5 in /usr/bin/clickhouse 9. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:645: Coordination::ZooKeeper::sendThread() @ 0x000000001f2fffbc in /usr/bin/clickhouse 10. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:355: Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0::operator()() const @ 0x000000001f3062b8 in /usr/bin/clickhouse 11. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&>()()) std::__invoke[abi:v15000]> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&) @ 0x000000001f306295 in /usr/bin/clickhouse 12. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001f306279 in /usr/bin/clickhouse 13. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&, std::tuple<>&) @ 0x000000001f3061dd in /usr/bin/clickhouse 14. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&&)::'lambda'()::operator()() @ 0x000000001f306105 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0>()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&&)::'lambda'()&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&&) @ 0x000000001f306075 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&&)::'lambda'()&) @ 0x000000001f306055 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001f306035 in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_0&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001f306000 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl::worker(std::__list_iterator) @ 0x0000000013a5c843 in /usr/bin/clickhouse 22. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a64ae4 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&) @ 0x0000000013a64a95 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x0000000013a649bd in /usr/bin/clickhouse 25. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000013a644a2 in /usr/bin/clickhouse 26. ? @ 0x00007fc9b4da9ac3 in ? 27. ? @ 0x00007fc9b4e3b850 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 00:20:03.676706 [ 1039 ] {3029e76d-f727-407a-b381-e2b0c9aa6b4e} executeQuery: (from [::1]:39734) CREATE TABLE test.hits_s3 (WatchID UInt64, JavaEnable UInt8, Title String, GoodEvent Int16, EventTime DateTime, EventDate Date, CounterID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RegionID UInt32, UserID UInt64, CounterClass Int8, OS UInt8, UserAgent UInt8, URL String, Referer String, URLDomain String, RefererDomain String, Refresh UInt8, IsRobot UInt8, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), ResolutionWidth UInt16, ResolutionHeight UInt16, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, FlashMinor2 String, NetMajor UInt8, NetMinor UInt8, UserAgentMajor UInt16, UserAgentMinor FixedString(2), CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, MobilePhone UInt8, MobilePhoneModel String, Params String, IPNetworkID UInt32, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, IsArtifical UInt8, WindowClientWidth UInt16, WindowClientHeight UInt16, ClientTimeZone Int16, ClientEventTime DateTime, SilverlightVersion1 UInt8, SilverlightVersion2 UInt8, SilverlightVersion3 UInt32, SilverlightVersion4 UInt16, PageCharset String, CodeVersion UInt32, IsLink UInt8, IsDownload UInt8, IsNotBounce UInt8, FUniqID UInt64, HID UInt32, IsOldCounter UInt8, IsEvent UInt8, IsParameter UInt8, DontCountHits UInt8, WithHash UInt8, HitColor FixedString(1), UTCEventTime DateTime, Age UInt8, Sex UInt8, Income UInt8, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), RemoteIP UInt32, RemoteIP6 FixedString(16), WindowName Int32, OpenerName Int32, HistoryLength Int16, BrowserLanguage FixedString(2), BrowserCountry FixedString(2), SocialNetwork String, SocialAction String, HTTPError UInt16, SendTiming Int32, DNSTiming Int32, ConnectTiming Int32, ResponseStartTiming Int32, ResponseEndTiming Int32, FetchTiming Int32, RedirectTiming Int32, DOMInteractiveTiming Int32, DOMContentLoadedTiming Int32, DOMCompleteTiming Int32, LoadEventStartTiming Int32, LoadEventEndTiming Int32, NSToDOMContentLoadedTiming Int32, FirstPaintTiming Int32, RedirectCount Int8, SocialSourceNetworkID UInt8, SocialSourcePage String, ParamPrice Int64, ParamOrderID String, ParamCurrency FixedString(3), ParamCurrencyID UInt16, GoalsReached Array(UInt32), OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, RefererHash UInt64, URLHash UInt64, CLID UInt32, YCLID UInt64, ShareService String, ShareURL String, ShareTitle String, ParsedParams Nested(Key1 String, Key2 String, Key3 String, Key4 String, Key5 String, ValueDouble Float64), IslandID FixedString(16), RequestNum UInt32, RequestTry UInt8) ENGINE = MergeTree() PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) SAMPLE BY intHash32(UserID) SETTINGS index_granularity = 8192, storage_policy='s3_cache' (stage: Complete) 2024.10.22 00:20:03.685404 [ 1039 ] {3029e76d-f727-407a-b381-e2b0c9aa6b4e} ContextAccess (default): Access granted: CREATE TABLE ON test.hits_s3 2024.10.22 00:20:04.501086 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 119 entries to flush up to offset 683 2024.10.22 00:20:04.541081 [ 1039 ] {3029e76d-f727-407a-b381-e2b0c9aa6b4e} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/etz/qnlyrmzvwzvcbsdnhhvcozbbepqru (LogSeriesLimiter: on interval from 2024-10-22 00:15:04 to 2024-10-22 00:20:04 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:20:04.666888 [ 1039 ] {3029e76d-f727-407a-b381-e2b0c9aa6b4e} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/etz/qnlyrmzvwzvcbsdnhhvcozbbepqru. 2024.10.22 00:20:04.744426 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:04.792593 [ 1341 ] {3029e76d-f727-407a-b381-e2b0c9aa6b4e} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/etz/qnlyrmzvwzvcbsdnhhvcozbbepqru, size 1 2024.10.22 00:20:04.845173 [ 1039 ] {3029e76d-f727-407a-b381-e2b0c9aa6b4e} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/etz/qnlyrmzvwzvcbsdnhhvcozbbepqru. 2024.10.22 00:20:04.849317 [ 1039 ] {3029e76d-f727-407a-b381-e2b0c9aa6b4e} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Loading data parts 2024.10.22 00:20:04.856283 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:04.881839 [ 1039 ] {3029e76d-f727-407a-b381-e2b0c9aa6b4e} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): There are no data parts 2024.10.22 00:20:05.006580 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_5_5_0 (state Temporary) 2024.10.22 00:20:05.098963 [ 1039 ] {3029e76d-f727-407a-b381-e2b0c9aa6b4e} MemoryTracker: Peak memory usage (for query): 2.36 MiB. 2024.10.22 00:20:05.100790 [ 1039 ] {3029e76d-f727-407a-b381-e2b0c9aa6b4e} TCPHandler: Processed in 1.688229792 sec. 2024.10.22 00:20:05.163992 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_5_5_0 to 202410_18_18_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:05.242689 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 683 2024.10.22 00:20:05.278250 [ 1039 ] {} TCPHandler: Done processing connection. 2024.10.22 00:20:05.420959 [ 1039 ] {} TCP-Session: c1fd3d7a-7065-4280-984e-ece79b989fe2 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:05.565427 [ 1039 ] {} TCPHandlerFactory: TCP Request. Address: [::1]:39740 2024.10.22 00:20:05.570737 [ 1039 ] {} TCPHandler: Connected ClickHouse client version 23.8.0, revision: 54465, user: default. 2024.10.22 00:20:05.572367 [ 1039 ] {} TCP-Session: 75c7d286-8baa-4bdc-9d90-8a1d4bd2e66f Authenticating user 'default' from [::1]:39740 2024.10.22 00:20:05.577093 [ 1039 ] {} TCP-Session: 75c7d286-8baa-4bdc-9d90-8a1d4bd2e66f Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:05.578072 [ 1039 ] {} TCP-Session: 75c7d286-8baa-4bdc-9d90-8a1d4bd2e66f Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:05.704873 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 947.30 MiB, peak 955.47 MiB, free memory in arenas 3.13 MiB, will set to 956.38 MiB (RSS), difference: 9.08 MiB 2024.10.22 00:20:05.716092 [ 1039 ] {} TCP-Session: 75c7d286-8baa-4bdc-9d90-8a1d4bd2e66f Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default 2024.10.22 00:20:05.869387 [ 1039 ] {f5924afe-f00c-41c7-a28e-9819f86915f3} executeQuery: (from [::1]:39740) CREATE TABLE test.hits (WatchID UInt64, JavaEnable UInt8, Title String, GoodEvent Int16, EventTime DateTime, EventDate Date, CounterID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RegionID UInt32, UserID UInt64, CounterClass Int8, OS UInt8, UserAgent UInt8, URL String, Referer String, URLDomain String, RefererDomain String, Refresh UInt8, IsRobot UInt8, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), ResolutionWidth UInt16, ResolutionHeight UInt16, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, FlashMinor2 String, NetMajor UInt8, NetMinor UInt8, UserAgentMajor UInt16, UserAgentMinor FixedString(2), CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, MobilePhone UInt8, MobilePhoneModel String, Params String, IPNetworkID UInt32, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, IsArtifical UInt8, WindowClientWidth UInt16, WindowClientHeight UInt16, ClientTimeZone Int16, ClientEventTime DateTime, SilverlightVersion1 UInt8, SilverlightVersion2 UInt8, SilverlightVersion3 UInt32, SilverlightVersion4 UInt16, PageCharset String, CodeVersion UInt32, IsLink UInt8, IsDownload UInt8, IsNotBounce UInt8, FUniqID UInt64, HID UInt32, IsOldCounter UInt8, IsEvent UInt8, IsParameter UInt8, DontCountHits UInt8, WithHash UInt8, HitColor FixedString(1), UTCEventTime DateTime, Age UInt8, Sex UInt8, Income UInt8, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), RemoteIP UInt32, RemoteIP6 FixedString(16), WindowName Int32, OpenerName Int32, HistoryLength Int16, BrowserLanguage FixedString(2), BrowserCountry FixedString(2), SocialNetwork String, SocialAction String, HTTPError UInt16, SendTiming Int32, DNSTiming Int32, ConnectTiming Int32, ResponseStartTiming Int32, ResponseEndTiming Int32, FetchTiming Int32, RedirectTiming Int32, DOMInteractiveTiming Int32, DOMContentLoadedTiming Int32, DOMCompleteTiming Int32, LoadEventStartTiming Int32, LoadEventEndTiming Int32, NSToDOMContentLoadedTiming Int32, FirstPaintTiming Int32, RedirectCount Int8, SocialSourceNetworkID UInt8, SocialSourcePage String, ParamPrice Int64, ParamOrderID String, ParamCurrency FixedString(3), ParamCurrencyID UInt16, GoalsReached Array(UInt32), OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, RefererHash UInt64, URLHash UInt64, CLID UInt32, YCLID UInt64, ShareService String, ShareURL String, ShareTitle String, ParsedParams Nested(Key1 String, Key2 String, Key3 String, Key4 String, Key5 String, ValueDouble Float64), IslandID FixedString(16), RequestNum UInt32, RequestTry UInt8) ENGINE = MergeTree() PARTITION BY toYYYYMM(EventDate) ORDER BY (CounterID, EventDate, intHash32(UserID)) SAMPLE BY intHash32(UserID) SETTINGS index_granularity = 8192, storage_policy='s3_cache' (stage: Complete) 2024.10.22 00:20:05.874605 [ 1039 ] {f5924afe-f00c-41c7-a28e-9819f86915f3} ContextAccess (default): Access granted: CREATE TABLE ON test.hits 2024.10.22 00:20:06.514355 [ 1447 ] {} ZooKeeperClient: Poco::Exception. Code: 1000, e.code() = 107, Net Exception: Socket is not connected, Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/base/poco/Foundation/src/Exception.cpp:145: Poco::RuntimeException::RuntimeException(String const&, int) @ 0x000000002495e5e7 in /usr/bin/clickhouse 4. /build/base/poco/Foundation/src/Exception.cpp:164: Poco::IOException::IOException(String const&, int) @ 0x0000000024962727 in /usr/bin/clickhouse 5. /build/base/poco/Net/src/NetException.cpp:26: Poco::Net::NetException::NetException(String const&, int) @ 0x0000000024769587 in /usr/bin/clickhouse 6. /build/base/poco/Net/src/SocketImpl.cpp:995: Poco::Net::SocketImpl::error(int, String const&) @ 0x0000000024778ecb in /usr/bin/clickhouse 7. /build/base/poco/Net/src/SocketImpl.cpp:914: Poco::Net::SocketImpl::error() @ 0x0000000024777869 in /usr/bin/clickhouse 8. /build/base/poco/Net/src/SocketImpl.cpp:272: Poco::Net::SocketImpl::shutdown() @ 0x000000002477a292 in /usr/bin/clickhouse 9. /build/base/poco/Net/src/StreamSocket.cpp:109: Poco::Net::StreamSocket::shutdown() @ 0x000000002477dfbe in /usr/bin/clickhouse 10. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:961: Coordination::ZooKeeper::finalize(bool, bool, String const&) @ 0x000000001f2f963a in /usr/bin/clickhouse 11. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:732: Coordination::ZooKeeper::receiveThread() @ 0x000000001f3010c3 in /usr/bin/clickhouse 12. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:356: Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1::operator()() const @ 0x000000001f306af8 in /usr/bin/clickhouse 13. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&>()()) std::__invoke[abi:v15000]> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&) @ 0x000000001f306ad5 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&, std::tuple<>&, std::__tuple_indices<>) @ 0x000000001f306ab9 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&, std::tuple<>&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&, std::tuple<>&) @ 0x000000001f306a1d in /usr/bin/clickhouse 16. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'()::operator()() @ 0x000000001f306945 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'()&>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&) @ 0x000000001f3068b5 in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'()&) @ 0x000000001f306895 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001f306875 in /usr/bin/clickhouse 20. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1>(Coordination::ZooKeeper::ZooKeeper(std::vector> const&, zkutil::ZooKeeperArgs const&, std::shared_ptr, std::optional>&&)::$_1&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001f306840 in /usr/bin/clickhouse 21. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 23. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl::worker(std::__list_iterator) @ 0x0000000013a5c843 in /usr/bin/clickhouse 24. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a64ae4 in /usr/bin/clickhouse 25. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&) @ 0x0000000013a64a95 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x0000000013a649bd in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000013a644a2 in /usr/bin/clickhouse 28. ? @ 0x00007fc9b4da9ac3 in ? 29. ? @ 0x00007fc9b4e3b850 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 00:20:06.518470 [ 1445 ] {} DDLWorker: Scheduling tasks 2024.10.22 00:20:06.999411 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:07.001707 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:07.051177 [ 1039 ] {f5924afe-f00c-41c7-a28e-9819f86915f3} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Loading data parts 2024.10.22 00:20:07.086694 [ 1039 ] {f5924afe-f00c-41c7-a28e-9819f86915f3} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): There are no data parts 2024.10.22 00:20:07.191130 [ 1039 ] {f5924afe-f00c-41c7-a28e-9819f86915f3} MemoryTracker: Peak memory usage (for query): 2.36 MiB. 2024.10.22 00:20:07.192520 [ 1039 ] {f5924afe-f00c-41c7-a28e-9819f86915f3} TCPHandler: Processed in 1.481974413 sec. 2024.10.22 00:20:07.254734 [ 1039 ] {} TCPHandler: Done processing connection. 2024.10.22 00:20:07.339034 [ 1039 ] {} TCP-Session: 75c7d286-8baa-4bdc-9d90-8a1d4bd2e66f Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:07.374327 [ 1551 ] {} SystemLog (system.opentelemetry_span_log): Flushing system log, 6 entries to flush up to offset 6 2024.10.22 00:20:07.569729 [ 1551 ] {} SystemLog (system.opentelemetry_span_log): Will use existing table system.opentelemetry_span_log for OpenTelemetrySpanLog 2024.10.22 00:20:07.610733 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 956.57 MiB, peak 960.83 MiB, free memory in arenas 3.29 MiB, will set to 958.76 MiB (RSS), difference: 2.19 MiB 2024.10.22 00:20:07.662693 [ 1039 ] {} TCPHandlerFactory: TCP Request. Address: [::1]:39750 2024.10.22 00:20:07.670178 [ 1039 ] {} TCPHandler: Connected ClickHouse client version 23.8.0, revision: 54465, user: default. 2024.10.22 00:20:07.672001 [ 1039 ] {} TCP-Session: fe2accc8-97b7-4310-adca-a9075aad652c Authenticating user 'default' from [::1]:39750 2024.10.22 00:20:07.682130 [ 1039 ] {} TCP-Session: fe2accc8-97b7-4310-adca-a9075aad652c Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:07.683695 [ 1039 ] {} TCP-Session: fe2accc8-97b7-4310-adca-a9075aad652c Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:07.793024 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 30 entries to flush up to offset 53 2024.10.22 00:20:07.836371 [ 1039 ] {} TCP-Session: fe2accc8-97b7-4310-adca-a9075aad652c Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default 2024.10.22 00:20:07.840096 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 2068 2024.10.22 00:20:07.875202 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 66 2024.10.22 00:20:07.878864 [ 1551 ] {} system.opentelemetry_span_log (6ccc75a8-1522-4520-a414-6e81fb929787): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:07.880564 [ 1551 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:07.912953 [ 1551 ] {} MergedBlockOutputStream: filled checksums 202410_1_1_0 (state Temporary) 2024.10.22 00:20:07.960657 [ 1551 ] {} system.opentelemetry_span_log (6ccc75a8-1522-4520-a414-6e81fb929787): Renaming temporary part tmp_insert_202410_1_1_0 to 202410_2_2_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:07.970984 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:07.972714 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:07.991299 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_3_3_0 (state Temporary) 2024.10.22 00:20:07.992279 [ 1445 ] {} DDLWorker: Lost ZooKeeper connection, will try to connect again: Code: 999. Coordination::Exception: Session expired. (KEEPER_EXCEPTION), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, char const (&) [16]) @ 0x00000000138d58c0 in /usr/bin/clickhouse 5. /build/src/Common/ZooKeeper/IKeeper.h:469: Coordination::Exception::Exception(char const (&) [16], Coordination::Error) @ 0x000000001f2c306d in /usr/bin/clickhouse 6. /build/src/Common/ZooKeeper/IKeeper.h:503: Coordination::Exception Coordination::Exception::fromMessage(Coordination::Error, char const (&) [16]) @ 0x000000001f2ae166 in /usr/bin/clickhouse 7. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:1116: Coordination::ZooKeeper::pushRequest(Coordination::ZooKeeper::RequestInfo&&) @ 0x000000001f302ba6 in /usr/bin/clickhouse 8. /build/src/Common/ZooKeeper/ZooKeeperImpl.cpp:1328: Coordination::ZooKeeper::list(String const&, Coordination::ListRequestType, std::function, std::shared_ptr>) @ 0x000000001f304add in /usr/bin/clickhouse 9. /build/src/Common/ZooKeeper/ZooKeeper.cpp:1080: zkutil::ZooKeeper::asyncTryGetChildrenNoThrow(String const&, std::shared_ptr>, Coordination::ListRequestType) @ 0x000000001f272e9c in /usr/bin/clickhouse 10. /build/src/Common/ZooKeeper/ZooKeeper.cpp:217: zkutil::ZooKeeper::getChildrenImpl(String const&, std::vector>&, Coordination::Stat*, std::shared_ptr>, Coordination::ListRequestType) @ 0x000000001f272a34 in /usr/bin/clickhouse 11. /build/src/Common/ZooKeeper/ZooKeeper.cpp:290: zkutil::ZooKeeper::tryGetChildrenWatch(String const&, std::vector>&, Coordination::Stat*, std::shared_ptr>, Coordination::ListRequestType) @ 0x000000001f2735db in /usr/bin/clickhouse 12. /build/src/Common/ZooKeeper/ZooKeeper.cpp:266: zkutil::ZooKeeper::tryGetChildren(String const&, std::vector>&, Coordination::Stat*, std::shared_ptr const&, Coordination::ListRequestType) @ 0x000000001f2731e2 in /usr/bin/clickhouse 13. /build/src/Common/ZooKeeper/ZooKeeper.cpp:241: zkutil::ZooKeeper::getChildren(String const&, Coordination::Stat*, std::shared_ptr const&, Coordination::ListRequestType) @ 0x000000001f272fdc in /usr/bin/clickhouse 14. /build/src/Interpreters/DDLWorker.cpp:314: DB::DDLWorker::scheduleTasks(bool) @ 0x000000001bec5a07 in /usr/bin/clickhouse 15. /build/src/Interpreters/DDLWorker.cpp:1126: DB::DDLWorker::runMainThread() @ 0x000000001bec0966 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:359: decltype(*std::declval().*std::declval()()) std::__invoke[abi:v15000](void (DB::DDLWorker::*&)(), DB::DDLWorker*&) @ 0x000000001bedc009 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]&, 0ul>(void (DB::DDLWorker::*&)(), std::tuple&, std::__tuple_indices<0ul>) @ 0x000000001bedbf8d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]&>(void (DB::DDLWorker::*&)(), std::tuple&) @ 0x000000001bedbedd in /usr/bin/clickhouse 19. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()::operator()() @ 0x000000001bedbdf5 in /usr/bin/clickhouse 20. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()(std::declval())) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()&>(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&) @ 0x000000001bedbd55 in /usr/bin/clickhouse 21. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'()&) @ 0x000000001bedbd35 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001bedbd15 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl(void (DB::DDLWorker::*&&)(), DB::DDLWorker*&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001bedbce0 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 25. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 26. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl::worker(std::__list_iterator) @ 0x0000000013a5c843 in /usr/bin/clickhouse 27. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a64ae4 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&) @ 0x0000000013a64a95 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/thread:285: void std::__thread_execute[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(std::tuple::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>&, std::__tuple_indices<>) @ 0x0000000013a649bd in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/thread:295: void* std::__thread_proxy[abi:v15000]>, void ThreadPoolImpl::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>>(void*) @ 0x0000000013a644a2 in /usr/bin/clickhouse 31. ? @ 0x00007fc9b4da9ac3 in ? (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 00:20:08.021380 [ 1551 ] {} SystemLog (system.opentelemetry_span_log): Flushed system log up to offset 6 2024.10.22 00:20:08.044352 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_3_3_0 to 202410_16_16_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:08.076385 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 66 2024.10.22 00:20:08.078987 [ 1169 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_11_2 to 202410_16_16_0 2024.10.22 00:20:08.083925 [ 1169 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:08.085250 [ 1169 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:08.114171 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_11_2 to 202410_16_16_0 into Compact with storage Full 2024.10.22 00:20:08.129682 [ 1039 ] {2ca2010f-efce-41e2-841f-5df0c5c0d163} executeQuery: (from [::1]:39750) CREATE TABLE test.visits (CounterID UInt32, StartDate Date, Sign Int8, IsNew UInt8, VisitID UInt64, UserID UInt64, StartTime DateTime, Duration UInt32, UTCStartTime DateTime, PageViews Int32, Hits Int32, IsBounce UInt8, Referer String, StartURL String, RefererDomain String, StartURLDomain String, EndURL String, LinkURL String, IsDownload UInt8, TraficSourceID Int8, SearchEngineID UInt16, SearchPhrase String, AdvEngineID UInt8, PlaceID Int32, RefererCategories Array(UInt16), URLCategories Array(UInt16), URLRegions Array(UInt32), RefererRegions Array(UInt32), IsYandex UInt8, GoalReachesDepth Int32, GoalReachesURL Int32, GoalReachesAny Int32, SocialSourceNetworkID UInt8, SocialSourcePage String, MobilePhoneModel String, ClientEventTime DateTime, RegionID UInt32, ClientIP UInt32, ClientIP6 FixedString(16), RemoteIP UInt32, RemoteIP6 FixedString(16), IPNetworkID UInt32, SilverlightVersion3 UInt32, CodeVersion UInt32, ResolutionWidth UInt16, ResolutionHeight UInt16, UserAgentMajor UInt16, UserAgentMinor UInt16, WindowClientWidth UInt16, WindowClientHeight UInt16, SilverlightVersion2 UInt8, SilverlightVersion4 UInt16, FlashVersion3 UInt16, FlashVersion4 UInt16, ClientTimeZone Int16, OS UInt8, UserAgent UInt8, ResolutionDepth UInt8, FlashMajor UInt8, FlashMinor UInt8, NetMajor UInt8, NetMinor UInt8, MobilePhone UInt8, SilverlightVersion1 UInt8, Age UInt8, Sex UInt8, Income UInt8, JavaEnable UInt8, CookieEnable UInt8, JavascriptEnable UInt8, IsMobile UInt8, BrowserLanguage UInt16, BrowserCountry UInt16, Interests UInt16, Robotness UInt8, GeneralInterests Array(UInt16), Params Array(String), Goals Nested(ID UInt32, Serial UInt32, EventTime DateTime, Price Int64, OrderID String, CurrencyID UInt32), WatchIDs Array(UInt64), ParamSumPrice Int64, ParamCurrency FixedString(3), ParamCurrencyID UInt16, ClickLogID UInt64, ClickEventID Int32, ClickGoodEvent Int32, ClickEventTime DateTime, ClickPriorityID Int32, ClickPhraseID Int32, ClickPageID Int32, ClickPlaceID Int32, ClickTypeID Int32, ClickResourceID Int32, ClickCost UInt32, ClickClientIP UInt32, ClickDomainID UInt32, ClickURL String, ClickAttempt UInt8, ClickOrderID UInt32, ClickBannerID UInt32, ClickMarketCategoryID UInt32, ClickMarketPP UInt32, ClickMarketCategoryName String, ClickMarketPPName String, ClickAWAPSCampaignName String, ClickPageName String, ClickTargetType UInt16, ClickTargetPhraseID UInt64, ClickContextType UInt8, ClickSelectType Int8, ClickOptions String, ClickGroupBannerID Int32, OpenstatServiceName String, OpenstatCampaignID String, OpenstatAdID String, OpenstatSourceID String, UTMSource String, UTMMedium String, UTMCampaign String, UTMContent String, UTMTerm String, FromTag String, HasGCLID UInt8, FirstVisit DateTime, PredLastVisit Date, LastVisit Date, TotalVisits UInt32, TraficSource Nested(ID Int8, SearchEngineID UInt16, AdvEngineID UInt8, PlaceID UInt16, SocialSourceNetworkID UInt8, Domain String, SearchPhrase String, SocialSourcePage String), Attendance FixedString(16), CLID UInt32, YCLID UInt64, NormalizedRefererHash UInt64, SearchPhraseHash UInt64, RefererDomainHash UInt64, NormalizedStartURLHash UInt64, StartURLDomainHash UInt64, NormalizedEndURLHash UInt64, TopLevelDomain UInt64, URLScheme UInt64, OpenstatServiceNameHash UInt64, OpenstatCampaignIDHash UInt64, OpenstatAdIDHash UInt64, OpenstatSourceIDHash UInt64, UTMSourceHash UInt64, UTMMediumHash UInt64, UTMCampaignHash UInt64, UTMContentHash UInt64, UTMTermHash UInt64, FromHash UInt64, WebVisorEnabled UInt8, WebVisorActivity UInt32, ParsedParams Nested(Key1 String, Key2 String, Key3 String, Key4 String, Key5 String, ValueDouble Float64), Market Nested(Type UInt8, GoalID UInt32, OrderID String, OrderPrice Int64, PP UInt32, DirectPlaceID UInt32, DirectOrderID UInt32, DirectBannerID UInt32, GoodID String, GoodName String, GoodQuantity Int32, GoodPrice Int64), IslandID FixedString(16)) ENGINE = CollapsingMergeTree(Sign) PARTITION BY toYYYYMM(StartDate) ORDER BY (CounterID, StartDate, intHash32(UserID), VisitID) SAMPLE BY intHash32(UserID) SETTINGS index_granularity = 8192, storage_policy='s3_cache' (stage: Complete) 2024.10.22 00:20:08.135504 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:20:08.139314 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_11_2, total 93 rows starting from the beginning of the part 2024.10.22 00:20:08.149931 [ 1039 ] {2ca2010f-efce-41e2-841f-5df0c5c0d163} ContextAccess (default): Access granted: CREATE TABLE ON test.visits 2024.10.22 00:20:08.151649 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_12_12_0, total 9 rows starting from the beginning of the part 2024.10.22 00:20:08.177244 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:08.180710 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:08.218881 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_3_3_0 (state Temporary) 2024.10.22 00:20:08.240473 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_3_3_0 to 202410_9_9_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:08.266524 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_13_13_0, total 4 rows starting from the beginning of the part 2024.10.22 00:20:08.269705 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 53 2024.10.22 00:20:08.273572 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_14_14_0, total 46 rows starting from the beginning of the part 2024.10.22 00:20:08.391279 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_15_15_0, total 5 rows starting from the beginning of the part 2024.10.22 00:20:08.407880 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:08.410154 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:08.415768 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_16_16_0, total 15 rows starting from the beginning of the part 2024.10.22 00:20:08.480724 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_3_3_0 (state Temporary) 2024.10.22 00:20:08.535908 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_3_3_0 to 202410_10_10_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:08.639539 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 2068 2024.10.22 00:20:08.993830 [ 1445 ] {} DDLWorker: Initializing DDLWorker thread 2024.10.22 00:20:08.997011 [ 1445 ] {} Context: Trying to establish a new connection with ZooKeeper 2024.10.22 00:20:09.011136 [ 1040 ] {} KeeperTCPHandlerFactory: Keeper request. Address: [::1]:48986 2024.10.22 00:20:09.024003 [ 1040 ] {} KeeperTCPHandler: Requesting session ID for the new client 2024.10.22 00:20:09.035789 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:20:09.056134 [ 1365 ] {} KeeperStateMachine: Session ID response 4 with timeout 30000 2024.10.22 00:20:09.163346 [ 1040 ] {} KeeperTCPHandler: Received session ID 4 2024.10.22 00:20:09.164497 [ 1445 ] {} ZooKeeperClient: Connected to ZooKeeper at [::1]:9181 with session_id 4 2024.10.22 00:20:09.069828 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} MergeTask::MergeProjectionsStage: Merge sorted 172 rows, containing 12 columns (12 merged, 0 gathered) in 0.958713996 sec., 179.40699803865178 rows/sec., 25.67 KiB/sec. 2024.10.22 00:20:09.212592 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} MergedBlockOutputStream: filled checksums 202410_1_16_3 (state Temporary) 2024.10.22 00:20:09.264354 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_16_3 to 202410_1_16_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:09.276715 [ 1445 ] {} ZooKeeperClient: Keeper feature flag FILTERED_LIST: enabled 2024.10.22 00:20:09.327724 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 974.11 MiB, peak 977.50 MiB, free memory in arenas 3.01 MiB, will set to 961.26 MiB (RSS), difference: -12.85 MiB 2024.10.22 00:20:09.372815 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 91 entries to flush up to offset 774 2024.10.22 00:20:09.373323 [ 1445 ] {} ZooKeeperClient: Keeper feature flag MULTI_READ: enabled 2024.10.22 00:20:09.375025 [ 1445 ] {} ZooKeeperClient: Keeper feature flag CHECK_NOT_EXISTS: enabled 2024.10.22 00:20:09.376546 [ 1445 ] {} ZooKeeper: Initialized, hosts: localhost:9181 2024.10.22 00:20:09.377679 [ 1445 ] {} ZooKeeperClient: ZK send fault: 0.002% 2024.10.22 00:20:09.378380 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 1 2024.10.22 00:20:09.378551 [ 1445 ] {} ZooKeeperClient: ZK recv fault: 0.002% 2024.10.22 00:20:09.379205 [ 1445 ] {} ZooKeeperClient: ZK send sleep: 0.001% -> 10000ms 2024.10.22 00:20:09.379898 [ 1445 ] {} Context: Establishing a new connection with ZooKeeper took 382 ms 2024.10.22 00:20:09.385147 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_16_3} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_11_2, 202410_16_16_0] -> 202410_1_16_3 2024.10.22 00:20:09.428500 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:20:09.493474 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:09.497594 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:09.504941 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 89 2024.10.22 00:20:09.641384 [ 1445 ] {} DDLWorker: Initialized DDLWorker thread 2024.10.22 00:20:09.643576 [ 1445 ] {} DDLWorker: Scheduling tasks 2024.10.22 00:20:09.644800 [ 1445 ] {} DDLWorker: Don't have unfinished tasks after restarting 2024.10.22 00:20:09.644912 [ 1504 ] {} DDLWorker: Cleaning queue 2024.10.22 00:20:09.658567 [ 1445 ] {} DDLWorker: scheduleTasks: initialized=true, size_before_filtering=0, queue_size=0, entries=none..none, first_failed_task_name=none, current_tasks_size=0, last_current_task=none, last_skipped_entry_name=none 2024.10.22 00:20:09.659375 [ 1445 ] {} DDLWorker: No tasks to schedule 2024.10.22 00:20:09.659935 [ 1445 ] {} DDLWorker: Waiting for queue updates 2024.10.22 00:20:09.661226 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_6_6_0 (state Temporary) 2024.10.22 00:20:09.714173 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_1_1_0 (state Temporary) 2024.10.22 00:20:09.717605 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_6_6_0 to 202410_19_19_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:09.768975 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 774 2024.10.22 00:20:09.942075 [ 1039 ] {2ca2010f-efce-41e2-841f-5df0c5c0d163} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vkd/kibsvtthwcdkicrcnbsepmflbnpki (LogSeriesLimiter: on interval from 2024-10-22 00:20:04 to 2024-10-22 00:20:09 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:20:10.076124 [ 1039 ] {2ca2010f-efce-41e2-841f-5df0c5c0d163} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vkd/kibsvtthwcdkicrcnbsepmflbnpki. 2024.10.22 00:20:10.307438 [ 1341 ] {2ca2010f-efce-41e2-841f-5df0c5c0d163} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vkd/kibsvtthwcdkicrcnbsepmflbnpki, size 1 2024.10.22 00:20:10.427458 [ 1497 ] {} SystemLog (system.query_log): Flushing system log, 10 entries to flush up to offset 12 2024.10.22 00:20:10.468031 [ 1039 ] {2ca2010f-efce-41e2-841f-5df0c5c0d163} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vkd/kibsvtthwcdkicrcnbsepmflbnpki. 2024.10.22 00:20:10.470092 [ 1039 ] {2ca2010f-efce-41e2-841f-5df0c5c0d163} test.visits (f12063af-8b93-4580-ac6b-01c8deb5ba5e): Loading data parts 2024.10.22 00:20:10.512429 [ 1039 ] {2ca2010f-efce-41e2-841f-5df0c5c0d163} test.visits (f12063af-8b93-4580-ac6b-01c8deb5ba5e): There are no data parts 2024.10.22 00:20:11.002531 [ 1039 ] {2ca2010f-efce-41e2-841f-5df0c5c0d163} MemoryTracker: Peak memory usage (for query): 2.48 MiB. 2024.10.22 00:20:11.004273 [ 1039 ] {2ca2010f-efce-41e2-841f-5df0c5c0d163} TCPHandler: Processed in 3.173706062 sec. 2024.10.22 00:20:11.106899 [ 1039 ] {} TCPHandler: Done processing connection. 2024.10.22 00:20:11.146395 [ 1039 ] {} TCP-Session: fe2accc8-97b7-4310-adca-a9075aad652c Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:11.360206 [ 1497 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:11.370535 [ 1497 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:11.474487 [ 1039 ] {} TCPHandlerFactory: TCP Request. Address: [::1]:58022 2024.10.22 00:20:11.479047 [ 1039 ] {} TCPHandler: Connected ClickHouse client version 23.8.0, revision: 54465, user: default. 2024.10.22 00:20:11.481020 [ 1039 ] {} TCP-Session: 026b32ed-8060-46ad-829a-7911ff45a875 Authenticating user 'default' from [::1]:58022 2024.10.22 00:20:11.482071 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 956.26 MiB, peak 977.50 MiB, free memory in arenas 2.76 MiB, will set to 965.25 MiB (RSS), difference: 8.99 MiB 2024.10.22 00:20:11.490422 [ 1039 ] {} TCP-Session: 026b32ed-8060-46ad-829a-7911ff45a875 Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:11.491778 [ 1039 ] {} TCP-Session: 026b32ed-8060-46ad-829a-7911ff45a875 Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 00:20:11.578828 [ 1039 ] {} TCP-Session: 026b32ed-8060-46ad-829a-7911ff45a875 Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default 2024.10.22 00:20:11.601155 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} executeQuery: (from [::1]:58022) INSERT INTO test.hits_s3 SELECT * FROM datasets.hits_v1 SETTINGS enable_filesystem_cache_on_write_operations=0 (stage: Complete) 2024.10.22 00:20:11.621440 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} ContextAccess (default): Access granted: INSERT(WatchID, JavaEnable, Title, GoodEvent, EventTime, EventDate, CounterID, ClientIP, ClientIP6, RegionID, UserID, CounterClass, OS, UserAgent, URL, Referer, URLDomain, RefererDomain, Refresh, IsRobot, RefererCategories, URLCategories, URLRegions, RefererRegions, ResolutionWidth, ResolutionHeight, ResolutionDepth, FlashMajor, FlashMinor, FlashMinor2, NetMajor, NetMinor, UserAgentMajor, UserAgentMinor, CookieEnable, JavascriptEnable, IsMobile, MobilePhone, MobilePhoneModel, Params, IPNetworkID, TraficSourceID, SearchEngineID, SearchPhrase, AdvEngineID, IsArtifical, WindowClientWidth, WindowClientHeight, ClientTimeZone, ClientEventTime, SilverlightVersion1, SilverlightVersion2, SilverlightVersion3, SilverlightVersion4, PageCharset, CodeVersion, IsLink, IsDownload, IsNotBounce, FUniqID, HID, IsOldCounter, IsEvent, IsParameter, DontCountHits, WithHash, HitColor, UTCEventTime, Age, Sex, Income, Interests, Robotness, GeneralInterests, RemoteIP, RemoteIP6, WindowName, OpenerName, HistoryLength, BrowserLanguage, BrowserCountry, SocialNetwork, SocialAction, HTTPError, SendTiming, DNSTiming, ConnectTiming, ResponseStartTiming, ResponseEndTiming, FetchTiming, RedirectTiming, DOMInteractiveTiming, DOMContentLoadedTiming, DOMCompleteTiming, LoadEventStartTiming, LoadEventEndTiming, NSToDOMContentLoadedTiming, FirstPaintTiming, RedirectCount, SocialSourceNetworkID, SocialSourcePage, ParamPrice, ParamOrderID, ParamCurrency, ParamCurrencyID, GoalsReached, OpenstatServiceName, OpenstatCampaignID, OpenstatAdID, OpenstatSourceID, UTMSource, UTMMedium, UTMCampaign, UTMContent, UTMTerm, FromTag, HasGCLID, RefererHash, URLHash, CLID, YCLID, ShareService, ShareURL, ShareTitle, `ParsedParams.Key1`, `ParsedParams.Key2`, `ParsedParams.Key3`, `ParsedParams.Key4`, `ParsedParams.Key5`, `ParsedParams.ValueDouble`, IslandID, RequestNum, RequestTry) ON test.hits_s3 2024.10.22 00:20:11.648290 [ 1497 ] {} MergedBlockOutputStream: filled checksums 202410_2_2_0 (state Temporary) 2024.10.22 00:20:11.779077 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} ContextAccess (default): Access granted: SELECT(WatchID, JavaEnable, Title, GoodEvent, EventTime, EventDate, CounterID, ClientIP, ClientIP6, RegionID, UserID, CounterClass, OS, UserAgent, URL, Referer, URLDomain, RefererDomain, Refresh, IsRobot, RefererCategories, URLCategories, URLRegions, RefererRegions, ResolutionWidth, ResolutionHeight, ResolutionDepth, FlashMajor, FlashMinor, FlashMinor2, NetMajor, NetMinor, UserAgentMajor, UserAgentMinor, CookieEnable, JavascriptEnable, IsMobile, MobilePhone, MobilePhoneModel, Params, IPNetworkID, TraficSourceID, SearchEngineID, SearchPhrase, AdvEngineID, IsArtifical, WindowClientWidth, WindowClientHeight, ClientTimeZone, ClientEventTime, SilverlightVersion1, SilverlightVersion2, SilverlightVersion3, SilverlightVersion4, PageCharset, CodeVersion, IsLink, IsDownload, IsNotBounce, FUniqID, HID, IsOldCounter, IsEvent, IsParameter, DontCountHits, WithHash, HitColor, UTCEventTime, Age, Sex, Income, Interests, Robotness, GeneralInterests, RemoteIP, RemoteIP6, WindowName, OpenerName, HistoryLength, BrowserLanguage, BrowserCountry, SocialNetwork, SocialAction, HTTPError, SendTiming, DNSTiming, ConnectTiming, ResponseStartTiming, ResponseEndTiming, FetchTiming, RedirectTiming, DOMInteractiveTiming, DOMContentLoadedTiming, DOMCompleteTiming, LoadEventStartTiming, LoadEventEndTiming, NSToDOMContentLoadedTiming, FirstPaintTiming, RedirectCount, SocialSourceNetworkID, SocialSourcePage, ParamPrice, ParamOrderID, ParamCurrency, ParamCurrencyID, GoalsReached, OpenstatServiceName, OpenstatCampaignID, OpenstatAdID, OpenstatSourceID, UTMSource, UTMMedium, UTMCampaign, UTMContent, UTMTerm, FromTag, HasGCLID, RefererHash, URLHash, CLID, YCLID, ShareService, ShareURL, ShareTitle, `ParsedParams.Key1`, `ParsedParams.Key2`, `ParsedParams.Key3`, `ParsedParams.Key4`, `ParsedParams.Key5`, `ParsedParams.ValueDouble`, IslandID, RequestNum, RequestTry) ON datasets.hits_v1 2024.10.22 00:20:11.797982 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} InterpreterSelectQuery: FetchColumns -> Complete 2024.10.22 00:20:11.829355 [ 1497 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Renaming temporary part tmp_insert_202410_2_2_0 to 202410_6_6_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:11.866371 [ 1192 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef) (MergerMutator): Selected 6 parts from 202410_1_1_0 to 202410_6_6_0 2024.10.22 00:20:11.873118 [ 1192 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:11.876379 [ 1192 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:11.908264 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_1_0 to 202410_6_6_0 into Compact with storage Full 2024.10.22 00:20:11.955224 [ 1497 ] {} SystemLog (system.query_log): Flushed system log up to offset 12 2024.10.22 00:20:11.961089 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:20:11.980049 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_1_0, total 2 rows starting from the beginning of the part 2024.10.22 00:20:12.015443 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_2_2_0, total 2 rows starting from the beginning of the part 2024.10.22 00:20:12.156469 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} datasets.hits_v1 (SelectExecutor): Key condition: unknown 2024.10.22 00:20:12.158760 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_3_3_0, total 2 rows starting from the beginning of the part 2024.10.22 00:20:12.159283 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} datasets.hits_v1 (SelectExecutor): MinMax index condition: unknown 2024.10.22 00:20:12.171147 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} datasets.hits_v1 (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1084/1084 marks by primary key, 1084 marks to read from 1 ranges 2024.10.22 00:20:12.175029 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} datasets.hits_v1 (SelectExecutor): Spreading mark ranges among streams (default reading) 2024.10.22 00:20:12.191579 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part 201403_10_18_2, approx. 8880128 rows starting from 0 2024.10.22 00:20:12.300295 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_4_4_0, total 2 rows starting from the beginning of the part 2024.10.22 00:20:12.453511 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_5_5_0, total 2 rows starting from the beginning of the part 2024.10.22 00:20:12.584954 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_6_6_0, total 10 rows starting from the beginning of the part 2024.10.22 00:20:13.470824 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 975.33 MiB, peak 977.50 MiB, free memory in arenas 3.12 MiB, will set to 967.97 MiB (RSS), difference: -7.37 MiB 2024.10.22 00:20:13.773489 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 846 2024.10.22 00:20:14.321930 [ 1252 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 202 ms. 2024.10.22 00:20:14.351703 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:14.353651 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:14.756237 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_7_7_0 (state Temporary) 2024.10.22 00:20:14.787650 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_7_7_0 to 202410_20_20_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:15.039375 [ 1259 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:20:15.049805 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 846 2024.10.22 00:20:15.079084 [ 1259 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:20:15.578280 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 80 2024.10.22 00:20:15.647579 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 3988 2024.10.22 00:20:15.758973 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:15.763870 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:15.773220 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 87 entries to flush up to offset 140 2024.10.22 00:20:15.790956 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_4_4_0 (state Temporary) 2024.10.22 00:20:15.805399 [ 1396 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} MergeTask::MergeProjectionsStage: Merge sorted 20 rows, containing 72 columns (72 merged, 0 gathered) in 3.90426302 sec., 5.122605699858817 rows/sec., 6.75 KiB/sec. 2024.10.22 00:20:15.850589 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_4_4_0 to 202410_17_17_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:15.861660 [ 1399 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} MergedBlockOutputStream: filled checksums 202410_1_6_1 (state Temporary) 2024.10.22 00:20:15.867300 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:15.868573 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:15.872545 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 80 2024.10.22 00:20:15.896325 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_4_4_0 (state Temporary) 2024.10.22 00:20:15.948459 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_4_4_0 to 202410_10_10_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:15.970994 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 140 2024.10.22 00:20:15.978056 [ 1267 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_5_1 to 202410_10_10_0 2024.10.22 00:20:15.988826 [ 1267 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:15.991440 [ 1267 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:16.007193 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_5_1 to 202410_10_10_0 into Compact with storage Full 2024.10.22 00:20:16.035703 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:20:16.044672 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_5_1, total 51 rows starting from the beginning of the part 2024.10.22 00:20:16.062659 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_6_6_0, total 53 rows starting from the beginning of the part 2024.10.22 00:20:16.087602 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_7_7_0, total 17 rows starting from the beginning of the part 2024.10.22 00:20:16.115486 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_8_8_0, total 6 rows starting from the beginning of the part 2024.10.22 00:20:16.127055 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_9_9_0, total 30 rows starting from the beginning of the part 2024.10.22 00:20:16.137580 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_10_10_0, total 87 rows starting from the beginning of the part 2024.10.22 00:20:16.167530 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:16.170943 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:16.195077 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_1_1_0 to 202410_4_4_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:16.217155 [ 1389 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Renaming temporary part tmp_merge_202410_1_6_1 to 202410_1_6_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:16.221119 [ 1389 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef) (MergerMutator): Merged 6 parts: [202410_1_1_0, 202410_6_6_0] -> 202410_1_6_1 2024.10.22 00:20:16.273186 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1019.68 MiB, peak 1019.68 MiB, free memory in arenas 2.27 MiB, will set to 1003.32 MiB (RSS), difference: -16.35 MiB 2024.10.22 00:20:16.385506 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_4_4_0 (state Temporary) 2024.10.22 00:20:16.415744 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 10 entries to flush up to offset 20 2024.10.22 00:20:16.452211 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_4_4_0 to 202410_11_11_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:16.505175 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 3988 2024.10.22 00:20:16.540067 [ 1275 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_6_1 to 202410_11_11_0 2024.10.22 00:20:16.650259 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MemoryTracker: Current memory usage (total): 1.00 GiB. 2024.10.22 00:20:16.655644 [ 1275 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:16.657748 [ 1275 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:16.680090 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:16.685744 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:16.703434 [ 1275 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 00:20:16.725308 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_6_1 to 202410_11_11_0 into Compact with storage Full 2024.10.22 00:20:16.742894 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:20:16.750658 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_6_1, total 12438 rows starting from the beginning of the part 2024.10.22 00:20:16.855788 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_7_7_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:20:16.855815 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_2_2_0 (state Temporary) 2024.10.22 00:20:16.863655 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_8_8_0, total 148 rows starting from the beginning of the part 2024.10.22 00:20:16.870556 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} MergeTask::MergeProjectionsStage: Merge sorted 244 rows, containing 13 columns (13 merged, 0 gathered) in 0.867346002 sec., 281.317950895449 rows/sec., 116.38 KiB/sec. 2024.10.22 00:20:16.871403 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_9_9_0, total 384 rows starting from the beginning of the part 2024.10.22 00:20:16.903083 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_10_10_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:20:16.991546 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} MergedBlockOutputStream: filled checksums 202410_1_10_2 (state Temporary) 2024.10.22 00:20:17.010896 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_11_11_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:20:17.080118 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_2_2_0 to 202410_12_12_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:17.125427 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_10_2 to 202410_1_10_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:17.129821 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_10_2} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_5_1, 202410_10_10_0] -> 202410_1_10_2 2024.10.22 00:20:17.250146 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 20 2024.10.22 00:20:17.265697 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:20:17.284453 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.00 GiB, peak 1.02 GiB, free memory in arenas 3.49 MiB, will set to 1.00 GiB (RSS), difference: -4.21 MiB 2024.10.22 00:20:17.489018 [ 1389 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_6_1}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 00:20:17.651791 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.35 MiB. 2024.10.22 00:20:17.944131 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} MergeTask::MergeProjectionsStage: Merge sorted 17578 rows, containing 4 columns (4 merged, 0 gathered) in 1.231747231 sec., 14270.785074734218 rows/sec., 219.50 KiB/sec. 2024.10.22 00:20:17.983235 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} MergedBlockOutputStream: filled checksums 202410_1_11_2 (state Temporary) 2024.10.22 00:20:17.995776 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 8 2024.10.22 00:20:17.999141 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_11_2 to 202410_1_11_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:18.002982 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_11_2} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_6_1, 202410_11_11_0] -> 202410_1_11_2 2024.10.22 00:20:18.140043 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:20:18.267955 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.03 GiB, peak 1.04 GiB, free memory in arenas 8.76 MiB, will set to 1.04 GiB (RSS), difference: 3.81 MiB 2024.10.22 00:20:19.057041 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 924 2024.10.22 00:20:19.168178 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:20:19.168317 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:20:19.248982 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:19.254529 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:19.302177 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_8_8_0 (state Temporary) 2024.10.22 00:20:19.319797 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.07 GiB, peak 1.07 GiB, free memory in arenas 8.95 MiB, will set to 1.06 GiB (RSS), difference: -10.46 MiB 2024.10.22 00:20:19.331849 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_8_8_0 to 202410_21_21_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:19.365711 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_16_3 to 202410_21_21_0 2024.10.22 00:20:19.373625 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:19.462213 [ 1497 ] {} SystemLog (system.query_log): Flushing system log, 3 entries to flush up to offset 15 2024.10.22 00:20:19.475179 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 924 2024.10.22 00:20:19.477524 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:19.506286 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_16_3 to 202410_21_21_0 into Compact with storage Full 2024.10.22 00:20:19.525798 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:20:19.531259 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_16_3, total 1307 rows starting from the beginning of the part 2024.10.22 00:20:19.542397 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_17_17_0, total 88 rows starting from the beginning of the part 2024.10.22 00:20:19.551506 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_18_18_0, total 119 rows starting from the beginning of the part 2024.10.22 00:20:19.564328 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_19_19_0, total 91 rows starting from the beginning of the part 2024.10.22 00:20:19.575636 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_20_20_0, total 72 rows starting from the beginning of the part 2024.10.22 00:20:19.584398 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_21_21_0, total 78 rows starting from the beginning of the part 2024.10.22 00:20:19.598147 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} SortDescription: Compile sort description Date 1 1DateTime 1 1 2024.10.22 00:20:20.165471 [ 1497 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:20.166624 [ 1497 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:20.300174 [ 1497 ] {} MergedBlockOutputStream: filled checksums 202410_3_3_0 (state Temporary) 2024.10.22 00:20:20.328081 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.08 GiB, peak 1.08 GiB, free memory in arenas 8.33 MiB, will set to 1.08 GiB (RSS), difference: 1.54 MiB 2024.10.22 00:20:20.363869 [ 1157 ] {} DatabaseCatalog: Execution took 282 ms. 2024.10.22 00:20:20.640730 [ 1497 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Renaming temporary part tmp_insert_202410_3_3_0 to 202410_7_7_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:20.865420 [ 1497 ] {} SystemLog (system.query_log): Flushed system log up to offset 15 2024.10.22 00:20:21.422479 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.14 GiB, peak 1.14 GiB, free memory in arenas 8.91 MiB, will set to 1.11 GiB (RSS), difference: -34.84 MiB 2024.10.22 00:20:22.152386 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.17 GiB, peak 1.17 GiB, free memory in arenas 9.00 MiB, will set to 1.16 GiB (RSS), difference: -11.31 MiB 2024.10.22 00:20:23.320162 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.19 GiB, peak 1.19 GiB, free memory in arenas 8.95 MiB, will set to 1.19 GiB (RSS), difference: -4.04 MiB 2024.10.22 00:20:23.373762 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 108 2024.10.22 00:20:23.481091 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 292 entries to flush up to offset 432 2024.10.22 00:20:23.485350 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 957 2024.10.22 00:20:23.507329 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:23.515551 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2760 entries to flush up to offset 6748 2024.10.22 00:20:23.610920 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:23.658286 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_5_5_0 (state Temporary) 2024.10.22 00:20:23.675003 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:23.677607 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:23.702351 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_9_9_0 (state Temporary) 2024.10.22 00:20:23.784860 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_5_5_0 to 202410_18_18_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:23.838819 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_9_9_0 to 202410_22_22_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:23.868244 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:23.869431 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:23.931791 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 108 2024.10.22 00:20:24.000570 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 957 2024.10.22 00:20:24.025717 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_5_5_0 (state Temporary) 2024.10.22 00:20:24.065580 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_5_5_0 to 202410_11_11_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:24.082720 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 432 2024.10.22 00:20:24.106860 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:24.108247 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:24.169354 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.22 GiB, peak 1.22 GiB, free memory in arenas 7.55 MiB, will set to 1.21 GiB (RSS), difference: -1.29 MiB 2024.10.22 00:20:24.544554 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_5_5_0 (state Temporary) 2024.10.22 00:20:24.588024 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_5_5_0 to 202410_12_12_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:24.637506 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 6748 2024.10.22 00:20:24.752506 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 21 2024.10.22 00:20:24.963956 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:24.967431 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:25.019005 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_3_3_0 (state Temporary) 2024.10.22 00:20:25.263920 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.23 GiB, peak 1.23 GiB, free memory in arenas 7.39 MiB, will set to 1.22 GiB (RSS), difference: -6.32 MiB 2024.10.22 00:20:25.318731 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_3_3_0 to 202410_13_13_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:25.508034 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 21 2024.10.22 00:20:25.504463 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 36 entries to flush up to offset 44 2024.10.22 00:20:26.264940 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.26 GiB, peak 1.26 GiB, free memory in arenas 7.04 MiB, will set to 1.24 GiB (RSS), difference: -17.84 MiB 2024.10.22 00:20:27.064546 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} MergeTask::MergeProjectionsStage: Merge sorted 1755 rows, containing 13 columns (13 merged, 0 gathered) in 7.562985247 sec., 232.05122616048388 rows/sec., 43.40 KiB/sec. 2024.10.22 00:20:27.087709 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_2_2_0] 2024.10.22 00:20:27.089018 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_2_2_0] 2024.10.22 00:20:27.093301 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_2_2_0] 2024.10.22 00:20:27.115069 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} MergedBlockOutputStream: filled checksums 202410_1_21_4 (state Temporary) 2024.10.22 00:20:27.182322 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_21_4 to 202410_1_21_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:27.184635 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_21_4} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_16_3, 202410_21_21_0] -> 202410_1_21_4 2024.10.22 00:20:27.203473 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.69 MiB. 2024.10.22 00:20:27.206895 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 1 old parts to remove. Parts: [202410_1_1_0] 2024.10.22 00:20:27.209119 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 1 parts from filesystem (serially): Parts: [202410_1_1_0] 2024.10.22 00:20:27.214737 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 1 parts from memory: Parts: [202410_1_1_0] 2024.10.22 00:20:27.412379 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 1.26 GiB, free memory in arenas 6.43 MiB, will set to 1.26 GiB (RSS), difference: 8.96 MiB 2024.10.22 00:20:28.004435 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 1002 2024.10.22 00:20:28.070146 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:28.159763 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:28.171038 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 1 old parts to remove. Parts: [202410_1_1_0] 2024.10.22 00:20:28.173652 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from filesystem (serially): Parts: [202410_1_1_0] 2024.10.22 00:20:28.178640 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from memory: Parts: [202410_1_1_0] 2024.10.22 00:20:28.208397 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_10_10_0 (state Temporary) 2024.10.22 00:20:28.251785 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.28 GiB, peak 1.28 GiB, free memory in arenas 6.52 MiB, will set to 1.27 GiB (RSS), difference: -4.77 MiB 2024.10.22 00:20:28.265222 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_10_10_0 to 202410_23_23_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:28.432608 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_1_1_0] 2024.10.22 00:20:28.436106 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_1_1_0] 2024.10.22 00:20:28.444598 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_1_1_0] 2024.10.22 00:20:28.501293 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1002 2024.10.22 00:20:29.027835 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 1 old parts to remove. Parts: [202410_1_1_0] 2024.10.22 00:20:29.029157 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from filesystem (serially): Parts: [202410_1_1_0] 2024.10.22 00:20:29.035479 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from memory: Parts: [202410_1_1_0] 2024.10.22 00:20:29.168872 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:20:29.170279 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:20:29.319487 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 1.30 GiB, free memory in arenas 6.48 MiB, will set to 1.29 GiB (RSS), difference: -12.70 MiB 2024.10.22 00:20:30.082387 [ 1266 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:20:30.119178 [ 1266 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:20:30.326395 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.34 GiB, free memory in arenas 6.56 MiB, will set to 1.32 GiB (RSS), difference: -22.03 MiB 2024.10.22 00:20:31.172893 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 1.37 GiB, free memory in arenas 6.75 MiB, will set to 1.37 GiB (RSS), difference: -4.73 MiB 2024.10.22 00:20:31.433251 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 121 2024.10.22 00:20:31.596566 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:31.598967 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:31.621039 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_3_3_0] 2024.10.22 00:20:31.624050 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_3_3_0] 2024.10.22 00:20:31.628292 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_3_3_0] 2024.10.22 00:20:31.639121 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 269 entries to flush up to offset 701 2024.10.22 00:20:31.646460 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 3384 entries to flush up to offset 10132 2024.10.22 00:20:31.657156 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_6_6_0 (state Temporary) 2024.10.22 00:20:31.690480 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_6_6_0 to 202410_19_19_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:31.722809 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 121 2024.10.22 00:20:32.133430 [ 1177 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 204 ms. 2024.10.22 00:20:32.169046 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:32.173238 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:32.239673 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:32.249807 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:32.323456 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_6_6_0 (state Temporary) 2024.10.22 00:20:32.406559 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_6_6_0 (state Temporary) 2024.10.22 00:20:32.456880 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_6_6_0 to 202410_12_12_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:32.491792 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_6_6_0 to 202410_13_13_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:32.504741 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 1045 2024.10.22 00:20:32.522058 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 10132 2024.10.22 00:20:32.588173 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:32.591140 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:32.692707 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 701 2024.10.22 00:20:32.741059 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_11_11_0 (state Temporary) 2024.10.22 00:20:32.811303 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 1.38 GiB, free memory in arenas 6.55 MiB, will set to 1.37 GiB (RSS), difference: 16.15 MiB 2024.10.22 00:20:32.884677 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_11_11_0 to 202410_24_24_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:33.010072 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 22 2024.10.22 00:20:33.027440 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1045 2024.10.22 00:20:33.197273 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:33.199329 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:33.348741 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 378.30 MiB using storage policy from min volume index 0 2024.10.22 00:20:33.374988 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_4_4_0 (state Temporary) 2024.10.22 00:20:33.467641 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_4_4_0 to 202410_14_14_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:33.506174 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 22 2024.10.22 00:20:34.282462 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hmd/tyrnontcbulbgbgdvntkenagwvjyf (LogSeriesLimiter: on interval from 2024-10-22 00:20:09 to 2024-10-22 00:20:34 accepted series 1 / 1 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:20:34.409469 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.40 GiB, free memory in arenas 4.82 MiB, will set to 1.38 GiB (RSS), difference: -27.06 MiB 2024.10.22 00:20:35.191722 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.44 GiB, free memory in arenas 4.52 MiB, will set to 1.38 GiB (RSS), difference: -63.92 MiB 2024.10.22 00:20:35.440578 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 1 old parts to remove. Parts: [202410_2_2_0] 2024.10.22 00:20:35.443484 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 1 parts from filesystem (serially): Parts: [202410_2_2_0] 2024.10.22 00:20:35.456676 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 1 parts from memory: Parts: [202410_2_2_0] 2024.10.22 00:20:36.576157 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_2_2_0] 2024.10.22 00:20:36.577789 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_2_2_0] 2024.10.22 00:20:36.582414 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_2_2_0] 2024.10.22 00:20:36.623534 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.52 GiB, free memory in arenas 4.53 MiB, will set to 1.38 GiB (RSS), difference: -146.49 MiB 2024.10.22 00:20:36.896376 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_1_1_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:20:37.034223 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 1072 2024.10.22 00:20:37.036337 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 1 old parts to remove. Parts: [202410_2_2_0] 2024.10.22 00:20:37.040212 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from filesystem (serially): Parts: [202410_2_2_0] 2024.10.22 00:20:37.050755 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from memory: Parts: [202410_2_2_0] 2024.10.22 00:20:37.348705 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:37.352842 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:37.412192 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_12_12_0 (state Temporary) 2024.10.22 00:20:37.446125 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_4_4_0] 2024.10.22 00:20:37.451670 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_4_4_0] 2024.10.22 00:20:37.460968 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_4_4_0] 2024.10.22 00:20:37.558212 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_12_12_0 to 202410_25_25_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:37.595710 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1072 2024.10.22 00:20:38.016136 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 1 old parts to remove. Parts: [202410_2_2_0] 2024.10.22 00:20:38.019294 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from filesystem (serially): Parts: [202410_2_2_0] 2024.10.22 00:20:38.024260 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from memory: Parts: [202410_2_2_0] 2024.10.22 00:20:38.415045 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.52 GiB, free memory in arenas 3.34 MiB, will set to 1.38 GiB (RSS), difference: -147.50 MiB 2024.10.22 00:20:39.011543 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hmo/qmolxgfjzjouudoskyiojcdximpdi (LogSeriesLimiter: on interval from 2024-10-22 00:20:34 to 2024-10-22 00:20:39 accepted series 1 / 286 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:20:39.170406 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:20:39.171854 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:20:39.200947 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MemoryTracker: Current memory usage (for query): 1.00 GiB. 2024.10.22 00:20:39.224200 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 127 2024.10.22 00:20:39.337144 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:39.346377 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:39.384785 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_7_7_0 (state Temporary) 2024.10.22 00:20:39.539066 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_7_7_0 to 202410_20_20_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:39.613579 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 127 2024.10.22 00:20:39.623643 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:39.631901 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:39.644302 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 12052 2024.10.22 00:20:40.267810 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.52 GiB, free memory in arenas 2.86 MiB, will set to 1.38 GiB (RSS), difference: -133.71 MiB 2024.10.22 00:20:40.339262 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 1046 entries to flush up to offset 1747 2024.10.22 00:20:40.727781 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:40.737808 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:41.004252 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_7_7_0 (state Temporary) 2024.10.22 00:20:41.008189 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 23 2024.10.22 00:20:41.035682 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_5_5_0] 2024.10.22 00:20:41.040101 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_5_5_0] 2024.10.22 00:20:41.058507 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_5_5_0] 2024.10.22 00:20:41.082522 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_7_7_0 to 202410_14_14_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:41.119275 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 12052 2024.10.22 00:20:41.221599 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:41.256365 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:41.359693 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_7_7_0 (state Temporary) 2024.10.22 00:20:41.435817 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:41.445234 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:41.604218 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 1117 2024.10.22 00:20:41.706855 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_5_5_0 (state Temporary) 2024.10.22 00:20:41.762465 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:41.765379 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:41.771966 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_7_7_0 to 202410_13_13_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:41.812851 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 1747 2024.10.22 00:20:41.820198 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_13_13_0 (state Temporary) 2024.10.22 00:20:41.828218 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_5_5_0 to 202410_15_15_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:41.873273 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_13_13_0 to 202410_26_26_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:41.906676 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 23 2024.10.22 00:20:41.977688 [ 1251 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_10_2 to 202410_15_15_0 2024.10.22 00:20:41.995720 [ 1251 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:41.999546 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:42.016015 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_10_2 to 202410_15_15_0 into Compact with storage Full 2024.10.22 00:20:42.018684 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1117 2024.10.22 00:20:42.066859 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:20:42.075277 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_10_2, total 36 rows starting from the beginning of the part 2024.10.22 00:20:42.104196 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_11_11_0, total 10 rows starting from the beginning of the part 2024.10.22 00:20:42.115923 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_21_4 to 202410_26_26_0 2024.10.22 00:20:42.126407 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_12_12_0, total 10 rows starting from the beginning of the part 2024.10.22 00:20:42.134417 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:42.140301 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:42.167594 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_21_4 to 202410_26_26_0 into Compact with storage Full 2024.10.22 00:20:42.169087 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_13_13_0, total 1 rows starting from the beginning of the part 2024.10.22 00:20:42.199520 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:20:42.206810 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_14_14_0, total 1 rows starting from the beginning of the part 2024.10.22 00:20:42.212581 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_21_4, total 1755 rows starting from the beginning of the part 2024.10.22 00:20:42.232772 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_22_22_0, total 33 rows starting from the beginning of the part 2024.10.22 00:20:42.248913 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_15_15_0, total 1 rows starting from the beginning of the part 2024.10.22 00:20:42.260437 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_23_23_0, total 45 rows starting from the beginning of the part 2024.10.22 00:20:42.272265 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_24_24_0, total 43 rows starting from the beginning of the part 2024.10.22 00:20:42.284007 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_25_25_0, total 27 rows starting from the beginning of the part 2024.10.22 00:20:42.339987 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_26_26_0, total 45 rows starting from the beginning of the part 2024.10.22 00:20:42.465031 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.52 GiB, free memory in arenas 2.95 MiB, will set to 1.39 GiB (RSS), difference: -44.12 MiB 2024.10.22 00:20:43.071729 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_2_2_0 (state Temporary) 2024.10.22 00:20:43.568615 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 1 old parts to remove. Parts: [202410_3_3_0] 2024.10.22 00:20:43.569674 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 1 parts from filesystem (serially): Parts: [202410_3_3_0] 2024.10.22 00:20:43.573142 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 1 parts from memory: Parts: [202410_3_3_0] 2024.10.22 00:20:43.699756 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_3_3_0] 2024.10.22 00:20:43.702521 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_3_3_0] 2024.10.22 00:20:43.712457 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_3_3_0] 2024.10.22 00:20:44.027649 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} MergeTask::MergeProjectionsStage: Merge sorted 59 rows, containing 32 columns (32 merged, 0 gathered) in 2.018402891 sec., 29.231032249844315 rows/sec., 4.68 KiB/sec. 2024.10.22 00:20:44.030356 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} MergeTask::MergeProjectionsStage: Merge sorted 1948 rows, containing 13 columns (13 merged, 0 gathered) in 1.869006748 sec., 1042.2648297468854 rows/sec., 188.73 KiB/sec. 2024.10.22 00:20:44.063174 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} MergedBlockOutputStream: filled checksums 202410_1_15_3 (state Temporary) 2024.10.22 00:20:44.104179 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} MergedBlockOutputStream: filled checksums 202410_1_26_5 (state Temporary) 2024.10.22 00:20:44.170584 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_15_3 to 202410_1_15_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:44.175579 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_15_3} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_10_2, 202410_15_15_0] -> 202410_1_15_3 2024.10.22 00:20:44.260412 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:20:44.270787 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_26_5 to 202410_1_26_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:44.273511 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_26_5} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_21_4, 202410_26_26_0] -> 202410_1_26_5 2024.10.22 00:20:44.309194 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:20:44.433033 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.52 GiB, free memory in arenas 2.99 MiB, will set to 1.40 GiB (RSS), difference: 4.55 MiB 2024.10.22 00:20:44.465271 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 1 old parts to remove. Parts: [202410_3_3_0] 2024.10.22 00:20:44.468651 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from filesystem (serially): Parts: [202410_3_3_0] 2024.10.22 00:20:44.573786 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from memory: Parts: [202410_3_3_0] 2024.10.22 00:20:45.128423 [ 1263 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:20:45.171191 [ 1263 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:20:45.336329 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 1 old parts to remove. Parts: [202410_3_3_0] 2024.10.22 00:20:45.339049 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from filesystem (serially): Parts: [202410_3_3_0] 2024.10.22 00:20:45.350728 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from memory: Parts: [202410_3_3_0] 2024.10.22 00:20:45.536871 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_6_6_0] 2024.10.22 00:20:45.539696 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_6_6_0] 2024.10.22 00:20:45.545705 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_6_6_0] 2024.10.22 00:20:46.021505 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 1180 2024.10.22 00:20:46.256511 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:46.258839 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:46.443376 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_14_14_0 (state Temporary) 2024.10.22 00:20:46.491826 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_14_14_0 to 202410_27_27_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:46.619480 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1180 2024.10.22 00:20:47.116910 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 147 2024.10.22 00:20:47.304507 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_2_2_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:20:47.458803 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:47.463348 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:47.700634 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_8_8_0 (state Temporary) 2024.10.22 00:20:47.866420 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_8_8_0 to 202410_21_21_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:47.953110 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 147 2024.10.22 00:20:47.985663 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_1_6_1] 2024.10.22 00:20:47.990642 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_1_6_1] 2024.10.22 00:20:47.998281 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_1_6_1] 2024.10.22 00:20:48.054469 [ 1185 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_16_3 to 202410_21_21_0 2024.10.22 00:20:48.070572 [ 1185 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:48.071912 [ 1185 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:48.077310 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_16_3 to 202410_21_21_0 into Compact with storage Full 2024.10.22 00:20:48.090569 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:20:48.123285 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 13588 2024.10.22 00:20:48.198724 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_16_3, total 172 rows starting from the beginning of the part 2024.10.22 00:20:48.212158 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_17_17_0, total 14 rows starting from the beginning of the part 2024.10.22 00:20:48.243612 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_18_18_0, total 28 rows starting from the beginning of the part 2024.10.22 00:20:48.258337 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_19_19_0, total 13 rows starting from the beginning of the part 2024.10.22 00:20:48.269811 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_20_20_0, total 6 rows starting from the beginning of the part 2024.10.22 00:20:48.278347 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_21_21_0, total 20 rows starting from the beginning of the part 2024.10.22 00:20:48.300436 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.52 GiB, free memory in arenas 2.44 MiB, will set to 1.42 GiB (RSS), difference: -7.47 MiB 2024.10.22 00:20:48.810420 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:48.812283 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:48.977810 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_8_8_0 (state Temporary) 2024.10.22 00:20:48.982401 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} MergeTask::MergeProjectionsStage: Merge sorted 253 rows, containing 12 columns (12 merged, 0 gathered) in 0.907034622 sec., 278.9309182511007 rows/sec., 40.95 KiB/sec. 2024.10.22 00:20:49.019875 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} MergedBlockOutputStream: filled checksums 202410_1_21_4 (state Temporary) 2024.10.22 00:20:49.068129 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_21_4 to 202410_1_21_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:49.068338 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_8_8_0 to 202410_15_15_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:49.069953 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_21_4} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_16_3, 202410_21_21_0] -> 202410_1_21_4 2024.10.22 00:20:49.086179 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:20:49.101551 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 13588 2024.10.22 00:20:49.172371 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:20:49.174536 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:20:49.475852 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 24 2024.10.22 00:20:49.482449 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.52 GiB, free memory in arenas 2.63 MiB, will set to 1.42 GiB (RSS), difference: 7.66 MiB 2024.10.22 00:20:50.005317 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 159 entries to flush up to offset 1906 2024.10.22 00:20:50.100686 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:50.101832 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_7_7_0] 2024.10.22 00:20:50.104755 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:50.105071 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_7_7_0] 2024.10.22 00:20:50.110191 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_7_7_0] 2024.10.22 00:20:50.160325 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_6_6_0 (state Temporary) 2024.10.22 00:20:50.257566 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_6_6_0 to 202410_16_16_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:50.413972 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:50.422137 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:50.620916 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_8_8_0 (state Temporary) 2024.10.22 00:20:50.635522 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 1233 2024.10.22 00:20:50.653631 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 24 2024.10.22 00:20:50.789126 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:50.796432 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.93 GiB. 2024.10.22 00:20:50.834775 [ 1200 ] {} DatabaseCatalog: Execution took 313 ms. 2024.10.22 00:20:50.861786 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_15_15_0 (state Temporary) 2024.10.22 00:20:50.911483 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_15_15_0 to 202410_28_28_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:50.974426 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_8_8_0 to 202410_14_14_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:51.029762 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_1_1_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:20:51.059757 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1233 2024.10.22 00:20:51.140364 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 1906 2024.10.22 00:20:51.259026 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_4_4_0] 2024.10.22 00:20:51.261026 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_4_4_0] 2024.10.22 00:20:51.265144 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_4_4_0] 2024.10.22 00:20:51.526735 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_2_2_0 to 202410_5_5_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:51.564303 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.52 GiB, free memory in arenas 4.33 MiB, will set to 1.43 GiB (RSS), difference: 7.93 MiB 2024.10.22 00:20:51.826293 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 1 old parts to remove. Parts: [202410_4_4_0] 2024.10.22 00:20:51.830213 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 1 parts from filesystem (serially): Parts: [202410_4_4_0] 2024.10.22 00:20:51.839043 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 1 parts from memory: Parts: [202410_4_4_0] 2024.10.22 00:20:52.658217 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 1 old parts to remove. Parts: [202410_4_4_0] 2024.10.22 00:20:52.660288 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from filesystem (serially): Parts: [202410_4_4_0] 2024.10.22 00:20:52.768787 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from memory: Parts: [202410_4_4_0] 2024.10.22 00:20:53.757145 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.52 GiB, free memory in arenas 4.73 MiB, will set to 1.43 GiB (RSS), difference: 6.52 MiB 2024.10.22 00:20:54.315093 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_8_8_0] 2024.10.22 00:20:54.317452 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_8_8_0] 2024.10.22 00:20:54.327315 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_8_8_0] 2024.10.22 00:20:55.068210 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 1260 2024.10.22 00:20:55.191907 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:55.204467 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:55.269333 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_16_16_0 (state Temporary) 2024.10.22 00:20:55.334862 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_16_16_0 to 202410_29_29_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:55.376155 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 44 2024.10.22 00:20:55.459854 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 161 2024.10.22 00:20:55.502834 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1260 2024.10.22 00:20:55.563220 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.52 GiB, free memory in arenas 6.23 MiB, will set to 1.43 GiB (RSS), difference: 6.74 MiB 2024.10.22 00:20:55.788226 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:55.796857 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:55.832342 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_9_9_0 (state Temporary) 2024.10.22 00:20:55.872865 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_9_9_0 to 202410_22_22_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:55.907796 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 161 2024.10.22 00:20:56.108220 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1875 entries to flush up to offset 15463 2024.10.22 00:20:56.823564 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:56.839174 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:57.071604 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_9_9_0 (state Temporary) 2024.10.22 00:20:57.256622 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_9_9_0 to 202410_16_16_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:57.271234 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 15463 2024.10.22 00:20:57.279272 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_11_2 to 202410_16_16_0 2024.10.22 00:20:57.287258 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:57.291309 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:57.325276 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_11_2 to 202410_16_16_0 into Compact with storage Full 2024.10.22 00:20:57.382562 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:20:57.390090 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_11_2, total 17578 rows starting from the beginning of the part 2024.10.22 00:20:57.405408 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_12_12_0, total 2760 rows starting from the beginning of the part 2024.10.22 00:20:57.406477 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.52 GiB, free memory in arenas 6.75 MiB, will set to 1.44 GiB (RSS), difference: 4.97 MiB 2024.10.22 00:20:57.415809 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_13_13_0, total 3384 rows starting from the beginning of the part 2024.10.22 00:20:57.437380 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_14_14_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:20:57.453833 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_15_15_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:20:57.461750 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_16_16_0, total 1875 rows starting from the beginning of the part 2024.10.22 00:20:58.118867 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_16_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:20:58.466804 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.52 GiB, free memory in arenas 7.88 MiB, will set to 1.44 GiB (RSS), difference: -1.03 MiB 2024.10.22 00:20:58.770453 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 43 entries to flush up to offset 1949 2024.10.22 00:20:58.866605 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} MergeTask::MergeProjectionsStage: Merge sorted 29053 rows, containing 4 columns (4 merged, 0 gathered) in 1.556983082 sec., 18659.804551428006 rows/sec., 280.53 KiB/sec. 2024.10.22 00:20:58.878861 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:58.881020 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:58.921218 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} MergedBlockOutputStream: filled checksums 202410_1_16_3 (state Temporary) 2024.10.22 00:20:58.942376 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_9_9_0 (state Temporary) 2024.10.22 00:20:58.987304 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_9_9_0 to 202410_15_15_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:59.058237 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_16_3 to 202410_1_16_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:59.059407 [ 1260 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_10_2 to 202410_15_15_0 2024.10.22 00:20:59.061086 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_16_3} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_11_2, 202410_16_16_0] -> 202410_1_16_3 2024.10.22 00:20:59.079861 [ 1260 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:59.082976 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:59.116234 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_10_2 to 202410_15_15_0 into Compact with storage Full 2024.10.22 00:20:59.116696 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 1 old parts to remove. Parts: [202410_5_5_0] 2024.10.22 00:20:59.117919 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 1949 2024.10.22 00:20:59.131878 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:20:59.150282 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:20:59.165055 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_10_2, total 244 rows starting from the beginning of the part 2024.10.22 00:20:59.171504 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:20:59.172841 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:20:59.220765 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 1 parts from filesystem (serially): Parts: [202410_5_5_0] 2024.10.22 00:20:59.228494 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 1 parts from memory: Parts: [202410_5_5_0] 2024.10.22 00:20:59.301142 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_11_11_0, total 292 rows starting from the beginning of the part 2024.10.22 00:20:59.385793 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.52 GiB, free memory in arenas 10.39 MiB, will set to 1.45 GiB (RSS), difference: 11.80 MiB 2024.10.22 00:20:59.413353 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_12_12_0, total 269 rows starting from the beginning of the part 2024.10.22 00:20:59.505603 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 1318 2024.10.22 00:20:59.643624 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_13_13_0, total 1046 rows starting from the beginning of the part 2024.10.22 00:20:59.649147 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:20:59.653243 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:20:59.656721 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_14_14_0, total 159 rows starting from the beginning of the part 2024.10.22 00:20:59.701068 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_17_17_0 (state Temporary) 2024.10.22 00:20:59.758722 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_17_17_0 to 202410_30_30_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:20:59.767683 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_15_15_0, total 43 rows starting from the beginning of the part 2024.10.22 00:20:59.841153 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1318 2024.10.22 00:20:59.870647 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_5_5_0] 2024.10.22 00:20:59.871811 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_5_5_0] 2024.10.22 00:20:59.875726 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_5_5_0] 2024.10.22 00:20:59.954850 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_9_9_0] 2024.10.22 00:20:59.957120 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_9_9_0] 2024.10.22 00:20:59.963853 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_9_9_0] 2024.10.22 00:21:00.126754 [ 1270 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 00:21:00.182350 [ 1274 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:21:00.241203 [ 1274 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:21:00.868081 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_15_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:21:00.891728 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} MergeTask::MergeProjectionsStage: Merge sorted 2053 rows, containing 13 columns (13 merged, 0 gathered) in 1.798584349 sec., 1141.453277485403 rows/sec., 500.69 KiB/sec. 2024.10.22 00:21:00.932422 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 1 old parts to remove. Parts: [202410_4_4_0] 2024.10.22 00:21:00.937443 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from filesystem (serially): Parts: [202410_4_4_0] 2024.10.22 00:21:01.049605 [ 1275 ] {} DatabaseCatalog: Execution took 209 ms. 2024.10.22 00:21:01.053576 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} MergedBlockOutputStream: filled checksums 202410_1_15_3 (state Temporary) 2024.10.22 00:21:01.055279 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from memory: Parts: [202410_4_4_0] 2024.10.22 00:21:01.224039 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_15_3 to 202410_1_15_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:01.232453 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_15_3} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_10_2, 202410_15_15_0] -> 202410_1_15_3 2024.10.22 00:21:01.272810 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.86 MiB. 2024.10.22 00:21:01.563888 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 1 old parts to remove. Parts: [202410_5_5_0] 2024.10.22 00:21:01.566049 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from filesystem (serially): Parts: [202410_5_5_0] 2024.10.22 00:21:01.571120 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from memory: Parts: [202410_5_5_0] 2024.10.22 00:21:03.054385 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 82 2024.10.22 00:21:03.346218 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_1_1_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:21:03.513287 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 179 2024.10.22 00:21:03.557819 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.52 GiB, free memory in arenas 9.17 MiB, will set to 1.45 GiB (RSS), difference: 1.75 MiB 2024.10.22 00:21:03.856224 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 1354 2024.10.22 00:21:03.962622 [ 1168 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 00:21:03.981317 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:03.986744 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:21:04.013789 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:04.017815 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:21:04.031023 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_18_18_0 (state Temporary) 2024.10.22 00:21:04.074872 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_18_18_0 to 202410_31_31_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:04.081441 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_10_10_0 (state Temporary) 2024.10.22 00:21:04.129172 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_26_5 to 202410_31_31_0 2024.10.22 00:21:04.163766 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1354 2024.10.22 00:21:04.174269 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_10_10_0 to 202410_23_23_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:04.217198 [ 1341 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:04.253717 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:04.256771 [ 1185 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.94 GiB. 2024.10.22 00:21:04.276678 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1836 entries to flush up to offset 17299 2024.10.22 00:21:04.347675 [ 1341 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:04.379526 [ 1678 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:04.383718 [ 1675 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:04.399018 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 179 2024.10.22 00:21:04.402770 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_26_5 to 202410_31_31_0 into Compact with storage Full 2024.10.22 00:21:04.464609 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:21:04.573469 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_26_5, total 1948 rows starting from the beginning of the part 2024.10.22 00:21:04.644026 [ 1680 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:04.707516 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_27_27_0, total 63 rows starting from the beginning of the part 2024.10.22 00:21:04.726849 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_28_28_0, total 53 rows starting from the beginning of the part 2024.10.22 00:21:04.968972 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_29_29_0, total 27 rows starting from the beginning of the part 2024.10.22 00:21:04.983857 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_30_30_0, total 58 rows starting from the beginning of the part 2024.10.22 00:21:05.000532 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_31_31_0, total 36 rows starting from the beginning of the part 2024.10.22 00:21:05.071982 [ 1341 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:05.323743 [ 1677 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:05.612335 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:05.636297 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.92 GiB. 2024.10.22 00:21:05.655300 [ 1681 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:05.656743 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 25 2024.10.22 00:21:05.658061 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_10_10_0] 2024.10.22 00:21:05.661545 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_10_10_0] 2024.10.22 00:21:05.663919 [ 1687 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:05.678697 [ 1696 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:05.680262 [ 1675 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:05.688907 [ 1680 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:05.725711 [ 1676 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:05.761048 [ 1694 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:05.772155 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_10_10_0] 2024.10.22 00:21:05.794694 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 1.52 GiB, free memory in arenas 6.34 MiB, will set to 1.45 GiB (RSS), difference: 86.67 MiB 2024.10.22 00:21:05.881127 [ 1691 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:21:06.108325 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_10_10_0 (state Temporary) 2024.10.22 00:21:06.247055 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:06.261569 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.92 GiB. 2024.10.22 00:21:06.299837 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_10_10_0 to 202410_17_17_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:06.485628 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_7_7_0 (state Temporary) 2024.10.22 00:21:06.575627 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 17299 2024.10.22 00:21:06.627480 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} MergeTask::MergeProjectionsStage: Merge sorted 2185 rows, containing 13 columns (13 merged, 0 gathered) in 2.363371432 sec., 924.5267038499093 rows/sec., 163.55 KiB/sec. 2024.10.22 00:21:06.635037 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 67 entries to flush up to offset 2016 2024.10.22 00:21:06.669055 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} MergedBlockOutputStream: filled checksums 202410_1_31_6 (state Temporary) 2024.10.22 00:21:06.876111 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_7_7_0 to 202410_17_17_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:06.963705 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:06.969299 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.92 GiB. 2024.10.22 00:21:07.015094 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_10_10_0 (state Temporary) 2024.10.22 00:21:07.098671 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 25 2024.10.22 00:21:07.108707 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_31_6 to 202410_1_31_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:07.114188 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_31_6} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_26_5, 202410_31_31_0] -> 202410_1_31_6 2024.10.22 00:21:07.276773 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:21:07.304049 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_10_10_0 to 202410_16_16_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:07.408555 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 2016 2024.10.22 00:21:07.513242 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_6_6_0] 2024.10.22 00:21:07.622434 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_6_6_0] 2024.10.22 00:21:07.628865 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_6_6_0] 2024.10.22 00:21:08.241778 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 1422 2024.10.22 00:21:08.378437 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_1_6_1] 2024.10.22 00:21:08.380197 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_1_6_1] 2024.10.22 00:21:08.380230 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 1 old parts to remove. Parts: [202410_11_11_0] 2024.10.22 00:21:08.383083 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from filesystem (serially): Parts: [202410_11_11_0] 2024.10.22 00:21:08.386283 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_1_6_1] 2024.10.22 00:21:08.389710 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 1 parts from memory: Parts: [202410_11_11_0] 2024.10.22 00:21:08.431122 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:08.436740 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.90 GiB. 2024.10.22 00:21:08.559611 [ 1213 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 346 ms. 2024.10.22 00:21:08.763951 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_19_19_0 (state Temporary) 2024.10.22 00:21:08.876651 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.24 GiB, peak 1.52 GiB, free memory in arenas 3.90 MiB, will set to 1.43 GiB (RSS), difference: 191.32 MiB 2024.10.22 00:21:09.049307 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_19_19_0 to 202410_32_32_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:09.158662 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1422 2024.10.22 00:21:09.174195 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:21:09.181312 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:21:09.417466 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 1 old parts to remove. Parts: [202410_6_6_0] 2024.10.22 00:21:09.420178 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from filesystem (serially): Parts: [202410_6_6_0] 2024.10.22 00:21:09.432112 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 1 parts from memory: Parts: [202410_6_6_0] 2024.10.22 00:21:09.606623 [ 1698 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hmo/qmolxgfjzjouudoskyiojcdximpdi, size 183 2024.10.22 00:21:09.850968 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_1_1_0 (state Temporary) 2024.10.22 00:21:09.866709 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vqq/fwxnxsymzfjehmjkoqtuqosgvgitd (LogSeriesLimiter: on interval from 2024-10-22 00:20:39 to 2024-10-22 00:21:09 accepted series 1 / 71 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:21:09.877720 [ 1695 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hmd/tyrnontcbulbgbgdvntkenagwvjyf, size 439 2024.10.22 00:21:09.939702 [ 1341 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vqq/fwxnxsymzfjehmjkoqtuqosgvgitd, size 4 2024.10.22 00:21:10.103839 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 1 old parts to remove. Parts: [202410_5_5_0] 2024.10.22 00:21:10.107177 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from filesystem (serially): Parts: [202410_5_5_0] 2024.10.22 00:21:10.115909 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from memory: Parts: [202410_5_5_0] 2024.10.22 00:21:10.691708 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.52 GiB, free memory in arenas 3.90 MiB, will set to 1.41 GiB (RSS), difference: 64.44 MiB 2024.10.22 00:21:11.239561 [ 1221 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 5 parts from 202410_1_1_0 to 202410_5_5_0 2024.10.22 00:21:11.251151 [ 1221 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:11.255417 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:11.268267 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_5_1} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_1_0 to 202410_5_5_0 into Compact with storage Full 2024.10.22 00:21:11.304327 [ 1190 ] {} DatabaseCatalog: Execution took 251 ms. 2024.10.22 00:21:11.904184 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 192 2024.10.22 00:21:12.305778 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:12.312948 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:12.366851 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_11_11_0 (state Temporary) 2024.10.22 00:21:12.464293 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 979.11 MiB, peak 1.52 GiB, free memory in arenas 4.04 MiB, will set to 1.04 GiB (RSS), difference: 82.00 MiB 2024.10.22 00:21:12.651555 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_11_11_0 to 202410_24_24_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:12.703207 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 192 2024.10.22 00:21:13.165783 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 1462 2024.10.22 00:21:13.580129 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1667 entries to flush up to offset 18966 2024.10.22 00:21:13.612334 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:13.718654 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:13.855222 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_5_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:21:13.864683 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_20_20_0 (state Temporary) 2024.10.22 00:21:13.938389 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_20_20_0 to 202410_33_33_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:14.098931 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1462 2024.10.22 00:21:14.238361 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:14.242617 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:14.374178 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.09 GiB, peak 1.52 GiB, free memory in arenas 2.99 MiB, will set to 1.08 GiB (RSS), difference: -15.47 MiB 2024.10.22 00:21:14.416057 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_5_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_1_0, total 8 rows starting from the beginning of the part 2024.10.22 00:21:14.446569 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_7_7_0] 2024.10.22 00:21:14.552617 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_7_7_0] 2024.10.22 00:21:14.619155 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 26 2024.10.22 00:21:14.660943 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_7_7_0] 2024.10.22 00:21:14.700255 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_11_11_0 (state Temporary) 2024.10.22 00:21:14.786665 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_11_11_0 to 202410_18_18_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:14.839383 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 18966 2024.10.22 00:21:14.917321 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 68 entries to flush up to offset 2084 2024.10.22 00:21:15.252531 [ 1264 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:21:15.278565 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:15.282711 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:15.282574 [ 1264 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:21:15.356673 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:15.362451 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:15.379791 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_8_8_0 (state Temporary) 2024.10.22 00:21:15.510865 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_11_11_0 (state Temporary) 2024.10.22 00:21:15.546387 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.08 GiB, peak 1.52 GiB, free memory in arenas 3.59 MiB, will set to 1.08 GiB (RSS), difference: -2.16 MiB 2024.10.22 00:21:15.591488 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_11_11_0 to 202410_17_17_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:15.675408 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_8_8_0 to 202410_18_18_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:15.781207 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 26 2024.10.22 00:21:15.847507 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 2084 2024.10.22 00:21:16.507757 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_5_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_2_2_0, total 42 rows starting from the beginning of the part 2024.10.22 00:21:17.774572 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.11 GiB, peak 1.52 GiB, free memory in arenas 3.39 MiB, will set to 1.10 GiB (RSS), difference: -5.86 MiB 2024.10.22 00:21:18.107581 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 1497 2024.10.22 00:21:18.220764 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:18.231127 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:18.277633 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 1 old parts to remove. Parts: [202410_6_6_0] 2024.10.22 00:21:18.279088 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from filesystem (serially): Parts: [202410_6_6_0] 2024.10.22 00:21:18.283872 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 1 parts from memory: Parts: [202410_6_6_0] 2024.10.22 00:21:18.403529 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_21_21_0 (state Temporary) 2024.10.22 00:21:18.462214 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_21_21_0 to 202410_34_34_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:18.515575 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1497 2024.10.22 00:21:18.971143 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_5_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_3_3_0, total 12 rows starting from the beginning of the part 2024.10.22 00:21:19.173197 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:21:19.175727 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:21:19.560111 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.18 GiB, peak 1.52 GiB, free memory in arenas 3.38 MiB, will set to 1.15 GiB (RSS), difference: -33.77 MiB 2024.10.22 00:21:20.030934 [ 1172 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 00:21:20.207112 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 198 2024.10.22 00:21:20.533345 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:20.638765 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:20.774541 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_12_12_0 (state Temporary) 2024.10.22 00:21:20.849732 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_12_12_0 to 202410_25_25_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:20.863765 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_5_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_4_4_0, total 8 rows starting from the beginning of the part 2024.10.22 00:21:20.907612 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 198 2024.10.22 00:21:21.059755 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_1_1_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:21:21.174057 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:21.176279 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:21.585740 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.21 GiB, peak 1.52 GiB, free memory in arenas 3.59 MiB, will set to 1.19 GiB (RSS), difference: -17.73 MiB 2024.10.22 00:21:21.635275 [ 1169 ] {} DatabaseCatalog: Execution took 324 ms. 2024.10.22 00:21:21.848432 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1703 entries to flush up to offset 20669 2024.10.22 00:21:22.524390 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 1524 2024.10.22 00:21:22.571567 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_8_8_0] 2024.10.22 00:21:22.585691 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_8_8_0] 2024.10.22 00:21:22.598670 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_8_8_0] 2024.10.22 00:21:22.820681 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:22.824068 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:22.877079 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_22_22_0 (state Temporary) 2024.10.22 00:21:23.147255 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_22_22_0 to 202410_35_35_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:23.219206 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:23.222871 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:23.248635 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_5_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_5_5_0, total 36 rows starting from the beginning of the part 2024.10.22 00:21:23.284138 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 27 2024.10.22 00:21:23.303956 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1524 2024.10.22 00:21:23.361653 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 37 entries to flush up to offset 2121 2024.10.22 00:21:23.376879 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_12_12_0 (state Temporary) 2024.10.22 00:21:23.476244 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_12_12_0 to 202410_19_19_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:23.549357 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:23.552111 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:23.620621 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_12_12_0 (state Temporary) 2024.10.22 00:21:23.627762 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:23.633258 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:23.634214 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 20669 2024.10.22 00:21:23.645549 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 1.52 GiB, free memory in arenas 3.73 MiB, will set to 1.23 GiB (RSS), difference: -65.73 MiB 2024.10.22 00:21:23.781714 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_12_12_0 to 202410_18_18_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:23.838976 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 2121 2024.10.22 00:21:24.129021 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_9_9_0 (state Temporary) 2024.10.22 00:21:24.364426 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_9_9_0 to 202410_19_19_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:24.453097 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 27 2024.10.22 00:21:25.342265 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 1.52 GiB, free memory in arenas 3.56 MiB, will set to 1.27 GiB (RSS), difference: -23.71 MiB 2024.10.22 00:21:25.406326 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_3_3_0 (state Temporary) 2024.10.22 00:21:27.309566 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 1554 2024.10.22 00:21:27.656685 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:27.658598 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:27.707362 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_23_23_0 (state Temporary) 2024.10.22 00:21:27.761146 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 1.52 GiB, free memory in arenas 3.70 MiB, will set to 1.30 GiB (RSS), difference: -76.64 MiB 2024.10.22 00:21:27.875037 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_23_23_0 to 202410_36_36_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:27.939331 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1554 2024.10.22 00:21:28.059652 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_31_6 to 202410_36_36_0 2024.10.22 00:21:28.073363 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:28.077074 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:28.183920 [ 1276 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 294 ms. 2024.10.22 00:21:28.302846 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_31_6 to 202410_36_36_0 into Compact with storage Full 2024.10.22 00:21:28.328441 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:21:28.334835 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_31_6, total 2185 rows starting from the beginning of the part 2024.10.22 00:21:28.410056 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 204 2024.10.22 00:21:28.567951 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_32_32_0, total 68 rows starting from the beginning of the part 2024.10.22 00:21:28.621629 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:28.624010 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:28.675425 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_13_13_0 (state Temporary) 2024.10.22 00:21:28.684926 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_33_33_0, total 40 rows starting from the beginning of the part 2024.10.22 00:21:28.715900 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_34_34_0, total 35 rows starting from the beginning of the part 2024.10.22 00:21:28.842784 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_13_13_0 to 202410_26_26_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:28.845303 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_35_35_0, total 27 rows starting from the beginning of the part 2024.10.22 00:21:28.859839 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_36_36_0, total 30 rows starting from the beginning of the part 2024.10.22 00:21:28.877602 [ 1223 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_21_4 to 202410_26_26_0 2024.10.22 00:21:28.997425 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 204 2024.10.22 00:21:29.011784 [ 1223 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:29.016790 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:29.073203 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_21_4 to 202410_26_26_0 into Compact with storage Full 2024.10.22 00:21:29.127795 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:21:29.136185 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_21_4, total 253 rows starting from the beginning of the part 2024.10.22 00:21:29.153757 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_22_22_0, total 14 rows starting from the beginning of the part 2024.10.22 00:21:29.174703 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:21:29.177545 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_23_23_0, total 18 rows starting from the beginning of the part 2024.10.22 00:21:29.179453 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:21:29.203630 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_24_24_0, total 13 rows starting from the beginning of the part 2024.10.22 00:21:29.225774 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_25_25_0, total 6 rows starting from the beginning of the part 2024.10.22 00:21:29.252542 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_26_26_0, total 6 rows starting from the beginning of the part 2024.10.22 00:21:29.605140 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.52 GiB, free memory in arenas 3.21 MiB, will set to 1.34 GiB (RSS), difference: -56.60 MiB 2024.10.22 00:21:30.141620 [ 1190 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 269 ms. 2024.10.22 00:21:30.287523 [ 1224 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:21:30.350847 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} MergeTask::MergeProjectionsStage: Merge sorted 2385 rows, containing 13 columns (13 merged, 0 gathered) in 2.161257307 sec., 1103.5243199758445 rows/sec., 191.09 KiB/sec. 2024.10.22 00:21:30.361635 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} MergeTask::MergeProjectionsStage: Merge sorted 310 rows, containing 12 columns (12 merged, 0 gathered) in 1.295760243 sec., 239.2417900415548 rows/sec., 35.27 KiB/sec. 2024.10.22 00:21:30.389549 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} MergedBlockOutputStream: filled checksums 202410_1_36_7 (state Temporary) 2024.10.22 00:21:30.503896 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} MergedBlockOutputStream: filled checksums 202410_1_26_5 (state Temporary) 2024.10.22 00:21:30.542110 [ 1224 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:21:30.544566 [ 1224 ] {} DNSCacheUpdater: Execution took 256 ms. 2024.10.22 00:21:30.556158 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_36_7 to 202410_1_36_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:30.562964 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_36_7} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_31_6, 202410_36_36_0] -> 202410_1_36_7 2024.10.22 00:21:30.644650 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1753 entries to flush up to offset 22422 2024.10.22 00:21:30.711008 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:21:30.723087 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_26_5 to 202410_1_26_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:30.727933 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_26_5} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_21_4, 202410_26_26_0] -> 202410_1_26_5 2024.10.22 00:21:30.776555 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:21:31.347126 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 61 entries to flush up to offset 2182 2024.10.22 00:21:31.366739 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:31.370755 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:31.472105 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 1.52 GiB, free memory in arenas 2.95 MiB, will set to 1.38 GiB (RSS), difference: 11.21 MiB 2024.10.22 00:21:31.568825 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:31.582664 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:31.734698 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_13_13_0 (state Temporary) 2024.10.22 00:21:31.830644 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_13_13_0 to 202410_20_20_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:31.858247 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_13_13_0 (state Temporary) 2024.10.22 00:21:31.878309 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 22422 2024.10.22 00:21:31.896641 [ 1231 ] {} DatabaseCatalog: Execution took 256 ms. 2024.10.22 00:21:31.932058 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_13_13_0 to 202410_19_19_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:31.946865 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 1620 2024.10.22 00:21:31.955365 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 28 2024.10.22 00:21:31.956587 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_9_9_0] 2024.10.22 00:21:31.959410 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_9_9_0] 2024.10.22 00:21:31.966214 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_9_9_0] 2024.10.22 00:21:32.181722 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 2182 2024.10.22 00:21:32.200841 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:32.207147 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:32.471901 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_24_24_0 (state Temporary) 2024.10.22 00:21:32.496266 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:32.502566 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:32.547413 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_24_24_0 to 202410_37_37_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:32.688894 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_10_10_0 (state Temporary) 2024.10.22 00:21:32.800386 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1620 2024.10.22 00:21:33.021808 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_10_10_0 to 202410_20_20_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:33.139076 [ 1252 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_15_3 to 202410_20_20_0 2024.10.22 00:21:33.144402 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 28 2024.10.22 00:21:33.155105 [ 1252 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:33.157478 [ 1252 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:33.176234 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_15_3 to 202410_20_20_0 into Compact with storage Full 2024.10.22 00:21:33.332214 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:21:33.351359 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_15_3, total 59 rows starting from the beginning of the part 2024.10.22 00:21:33.372132 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_16_16_0, total 1 rows starting from the beginning of the part 2024.10.22 00:21:33.394855 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_17_17_0, total 1 rows starting from the beginning of the part 2024.10.22 00:21:33.420680 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_18_18_0, total 1 rows starting from the beginning of the part 2024.10.22 00:21:33.551191 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_19_19_0, total 1 rows starting from the beginning of the part 2024.10.22 00:21:33.575587 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_20_20_0, total 1 rows starting from the beginning of the part 2024.10.22 00:21:33.793785 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.52 GiB, free memory in arenas 3.27 MiB, will set to 1.42 GiB (RSS), difference: -44.09 MiB 2024.10.22 00:21:34.883768 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_5_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:21:35.172908 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} MergeTask::MergeProjectionsStage: Merge sorted 64 rows, containing 32 columns (32 merged, 0 gathered) in 2.003927338 sec., 31.937285742044207 rows/sec., 5.10 KiB/sec. 2024.10.22 00:21:35.215798 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} MergedBlockOutputStream: filled checksums 202410_1_20_4 (state Temporary) 2024.10.22 00:21:35.503441 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.52 GiB, free memory in arenas 2.98 MiB, will set to 1.47 GiB (RSS), difference: -25.84 MiB 2024.10.22 00:21:35.631683 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_20_4 to 202410_1_20_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:35.651380 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_20_4} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_15_3, 202410_20_20_0] -> 202410_1_20_4 2024.10.22 00:21:35.686394 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:21:35.960244 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_3_3_0 to 202410_6_6_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:36.499888 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 231 2024.10.22 00:21:36.615397 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:36.622609 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:36.774931 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_14_14_0 (state Temporary) 2024.10.22 00:21:36.828376 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 1660 2024.10.22 00:21:36.933469 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_14_14_0 to 202410_27_27_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:36.980059 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 231 2024.10.22 00:21:36.992418 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:36.999069 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:37.074472 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_25_25_0 (state Temporary) 2024.10.22 00:21:37.130786 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_25_25_0 to 202410_38_38_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:37.292853 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1660 2024.10.22 00:21:37.376845 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.55 GiB, free memory in arenas 5.34 MiB, will set to 1.51 GiB (RSS), difference: -38.91 MiB 2024.10.22 00:21:38.740651 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_10_10_0] 2024.10.22 00:21:38.849658 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_10_10_0] 2024.10.22 00:21:38.858773 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_10_10_0] 2024.10.22 00:21:38.900185 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 23958 2024.10.22 00:21:39.173933 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:21:39.175184 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:21:39.661743 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 82 2024.10.22 00:21:39.696918 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 288 entries to flush up to offset 2470 2024.10.22 00:21:39.733675 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.78 GiB, free memory in arenas 6.95 MiB, will set to 1.63 GiB (RSS), difference: -159.86 MiB 2024.10.22 00:21:39.751749 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:39.753665 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:39.999413 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:40.005460 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:40.106532 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_14_14_0 (state Temporary) 2024.10.22 00:21:40.175304 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_14_14_0 to 202410_21_21_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:40.236067 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 23958 2024.10.22 00:21:40.285654 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_14_14_0 (state Temporary) 2024.10.22 00:21:40.339229 [ 1171 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_16_3 to 202410_21_21_0 2024.10.22 00:21:40.360838 [ 1171 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:40.363337 [ 1171 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:40.465095 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_14_14_0 to 202410_20_20_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:40.477200 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_16_3 to 202410_21_21_0 into Compact with storage Full 2024.10.22 00:21:40.521072 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:21:40.536847 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_16_3, total 29053 rows starting from the beginning of the part 2024.10.22 00:21:40.551457 [ 1180 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_15_3 to 202410_20_20_0 2024.10.22 00:21:40.556084 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_17_17_0, total 1836 rows starting from the beginning of the part 2024.10.22 00:21:40.564055 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_18_18_0, total 1667 rows starting from the beginning of the part 2024.10.22 00:21:40.574376 [ 1180 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:40.576394 [ 1180 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:40.590307 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_19_19_0, total 1703 rows starting from the beginning of the part 2024.10.22 00:21:40.612100 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_15_3 to 202410_20_20_0 into Compact with storage Full 2024.10.22 00:21:40.612520 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_20_20_0, total 1753 rows starting from the beginning of the part 2024.10.22 00:21:40.639845 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:21:40.648741 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 29 2024.10.22 00:21:40.650777 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_15_3, total 2053 rows starting from the beginning of the part 2024.10.22 00:21:40.684391 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 2470 2024.10.22 00:21:40.754247 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_21_21_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:21:40.785158 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_16_16_0, total 67 rows starting from the beginning of the part 2024.10.22 00:21:40.905204 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_17_17_0, total 68 rows starting from the beginning of the part 2024.10.22 00:21:40.917560 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_18_18_0, total 37 rows starting from the beginning of the part 2024.10.22 00:21:40.933032 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_19_19_0, total 61 rows starting from the beginning of the part 2024.10.22 00:21:40.952966 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_20_20_0, total 288 rows starting from the beginning of the part 2024.10.22 00:21:41.054388 [ 1555 ] {} SystemLog (system.session_log): Flushing system log, 13 entries to flush up to offset 15 2024.10.22 00:21:41.070841 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:41.073774 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:41.303363 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 1714 2024.10.22 00:21:41.468776 [ 1555 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:41.482204 [ 1555 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:41.535047 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.81 GiB, free memory in arenas 10.03 MiB, will set to 1.70 GiB (RSS), difference: 381.10 MiB 2024.10.22 00:21:41.607674 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:41.610943 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:41.680635 [ 1555 ] {} MergedBlockOutputStream: filled checksums 202410_2_2_0 (state Temporary) 2024.10.22 00:21:41.712094 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_11_11_0 (state Temporary) 2024.10.22 00:21:41.766858 [ 1555 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Renaming temporary part tmp_insert_202410_2_2_0 to 202410_4_4_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:41.819160 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_11_11_0 to 202410_21_21_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:41.851510 [ 1555 ] {} SystemLog (system.session_log): Flushed system log up to offset 15 2024.10.22 00:21:41.897652 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_26_26_0 (state Temporary) 2024.10.22 00:21:41.957681 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 29 2024.10.22 00:21:41.964753 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_26_26_0 to 202410_39_39_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:42.108331 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1714 2024.10.22 00:21:42.267132 [ 1187 ] {} DatabaseCatalog: Execution took 368 ms. 2024.10.22 00:21:42.267769 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} MergeTask::MergeProjectionsStage: Merge sorted 2574 rows, containing 13 columns (13 merged, 0 gathered) in 1.673101235 sec., 1538.4604028458564 rows/sec., 673.52 KiB/sec. 2024.10.22 00:21:42.539606 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} MergedBlockOutputStream: filled checksums 202410_1_20_4 (state Temporary) 2024.10.22 00:21:42.821596 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_20_4 to 202410_1_20_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:42.825391 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_20_4} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_15_3, 202410_20_20_0] -> 202410_1_20_4 2024.10.22 00:21:43.035589 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 499.68 MiB using storage policy from min volume index 0 2024.10.22 00:21:43.053106 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.53 MiB. 2024.10.22 00:21:43.168207 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} MergeTask::MergeProjectionsStage: Merge sorted 37548 rows, containing 4 columns (4 merged, 0 gathered) in 2.69541105 sec., 13930.342832125734 rows/sec., 207.14 KiB/sec. 2024.10.22 00:21:43.255619 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} MergedBlockOutputStream: filled checksums 202410_1_21_4 (state Temporary) 2024.10.22 00:21:43.274630 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_21_4 to 202410_1_21_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:43.278539 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_21_4} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_16_3, 202410_21_21_0] -> 202410_1_21_4 2024.10.22 00:21:43.523578 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.81 GiB, free memory in arenas 11.67 MiB, will set to 1.54 GiB (RSS), difference: -151.13 MiB 2024.10.22 00:21:43.533745 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:21:44.486013 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 252 2024.10.22 00:21:44.797636 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:44.803763 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:44.847836 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_15_15_0 (state Temporary) 2024.10.22 00:21:44.866531 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yiw/qddafdpisykehdkzqaqhgxvkuetjx (LogSeriesLimiter: on interval from 2024-10-22 00:21:09 to 2024-10-22 00:21:44 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:21:44.885766 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_15_15_0 to 202410_28_28_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:44.946920 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 252 2024.10.22 00:21:45.550981 [ 1212 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:21:45.567545 [ 1212 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:21:45.618350 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 11.71 MiB, will set to 1.55 GiB (RSS), difference: -68.57 MiB 2024.10.22 00:21:46.035170 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_5_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:21:46.158707 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 1753 2024.10.22 00:21:46.196455 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 1 old parts to remove. Parts: [202410_11_11_0] 2024.10.22 00:21:46.202958 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from filesystem (serially): Parts: [202410_11_11_0] 2024.10.22 00:21:46.212789 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 1 parts from memory: Parts: [202410_11_11_0] 2024.10.22 00:21:46.379224 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:46.382329 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:46.447877 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_27_27_0 (state Temporary) 2024.10.22 00:21:46.478399 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_27_27_0 to 202410_40_40_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:46.580635 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1753 2024.10.22 00:21:46.864801 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_2_2_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:21:47.203397 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 44 entries to flush up to offset 126 2024.10.22 00:21:47.247567 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 25494 2024.10.22 00:21:47.619221 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.81 GiB, free memory in arenas 12.56 MiB, will set to 1.55 GiB (RSS), difference: -142.10 MiB 2024.10.22 00:21:47.855003 [ 1161 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 00:21:48.001730 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:48.007011 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:48.369574 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_15_15_0 (state Temporary) 2024.10.22 00:21:48.560989 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_15_15_0 to 202410_22_22_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:48.608472 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 174 entries to flush up to offset 2644 2024.10.22 00:21:48.709384 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 25494 2024.10.22 00:21:49.103533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/liw/xawtepouskgsykdstujwboanzwmdw (LogSeriesLimiter: on interval from 2024-10-22 00:21:44 to 2024-10-22 00:21:49 accepted series 1 / 225 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:21:49.174985 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:21:49.178989 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:21:49.202351 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:49.206002 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:49.270150 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_15_15_0 (state Temporary) 2024.10.22 00:21:49.456086 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_15_15_0 to 202410_21_21_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:49.460535 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 30 2024.10.22 00:21:49.476847 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.81 GiB, free memory in arenas 11.29 MiB, will set to 1.55 GiB (RSS), difference: -156.74 MiB 2024.10.22 00:21:49.587686 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 2644 2024.10.22 00:21:50.092960 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:50.098621 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:50.366572 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_12_12_0 (state Temporary) 2024.10.22 00:21:50.445032 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_12_12_0 to 202410_22_22_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:50.535771 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 30 2024.10.22 00:21:50.591552 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 1788 2024.10.22 00:21:51.037212 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:51.040906 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:51.296648 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_28_28_0 (state Temporary) 2024.10.22 00:21:51.342884 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_28_28_0 to 202410_41_41_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:51.476513 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.81 GiB, free memory in arenas 9.59 MiB, will set to 1.55 GiB (RSS), difference: -173.36 MiB 2024.10.22 00:21:51.490198 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_36_7 to 202410_41_41_0 2024.10.22 00:21:51.500917 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1788 2024.10.22 00:21:51.508185 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:51.511154 [ 1234 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:51.559954 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_36_7 to 202410_41_41_0 into Compact with storage Full 2024.10.22 00:21:51.573481 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:21:51.594443 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_36_7, total 2385 rows starting from the beginning of the part 2024.10.22 00:21:51.627238 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_37_37_0, total 66 rows starting from the beginning of the part 2024.10.22 00:21:51.644552 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_38_38_0, total 40 rows starting from the beginning of the part 2024.10.22 00:21:51.663834 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_39_39_0, total 54 rows starting from the beginning of the part 2024.10.22 00:21:51.677375 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_40_40_0, total 39 rows starting from the beginning of the part 2024.10.22 00:21:51.712469 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_41_41_0, total 35 rows starting from the beginning of the part 2024.10.22 00:21:52.449733 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 258 2024.10.22 00:21:52.566667 [ 1250 ] {} DatabaseCatalog: Execution took 299 ms. 2024.10.22 00:21:52.601087 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:52.605282 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:52.622382 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} MergeTask::MergeProjectionsStage: Merge sorted 2619 rows, containing 13 columns (13 merged, 0 gathered) in 1.069386609 sec., 2449.067510251571 rows/sec., 417.54 KiB/sec. 2024.10.22 00:21:52.808364 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} MergedBlockOutputStream: filled checksums 202410_1_41_8 (state Temporary) 2024.10.22 00:21:52.855820 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_16_16_0 (state Temporary) 2024.10.22 00:21:52.860101 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_41_8 to 202410_1_41_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:52.865717 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_41_8} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_36_7, 202410_41_41_0] -> 202410_1_41_8 2024.10.22 00:21:52.956822 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_16_16_0 to 202410_29_29_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:53.027346 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 258 2024.10.22 00:21:53.158831 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:21:53.653708 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 11.42 MiB, will set to 1.56 GiB (RSS), difference: -6.30 MiB 2024.10.22 00:21:55.511510 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 1819 2024.10.22 00:21:55.550757 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 9.89 MiB, will set to 1.57 GiB (RSS), difference: -1.46 MiB 2024.10.22 00:21:55.714843 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1567 entries to flush up to offset 27061 2024.10.22 00:21:55.863028 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:55.865502 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:55.933423 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_29_29_0 (state Temporary) 2024.10.22 00:21:55.988175 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_29_29_0 to 202410_42_42_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:56.187738 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1819 2024.10.22 00:21:56.796869 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:56.803488 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:57.058703 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_16_16_0 (state Temporary) 2024.10.22 00:21:57.140553 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 164 entries to flush up to offset 2808 2024.10.22 00:21:57.154525 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_16_16_0 to 202410_23_23_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:57.177279 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 27061 2024.10.22 00:21:57.400696 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:21:57.407882 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:21:57.652338 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_16_16_0 (state Temporary) 2024.10.22 00:21:57.902977 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_16_16_0 to 202410_22_22_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:21:57.963889 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 2808 2024.10.22 00:21:59.178568 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:21:59.183597 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:21:59.375376 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 7.25 MiB, will set to 1.58 GiB (RSS), difference: -3.39 MiB 2024.10.22 00:22:00.072445 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_5_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:22:00.199990 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 23 entries to flush up to offset 1842 2024.10.22 00:22:00.453186 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:00.456609 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:00.522969 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_30_30_0 (state Temporary) 2024.10.22 00:22:00.532485 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 269 2024.10.22 00:22:00.566542 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_30_30_0 to 202410_43_43_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:00.572088 [ 1162 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:22:00.645390 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1842 2024.10.22 00:22:00.734639 [ 1162 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:22:00.859405 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:00.861190 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:00.998393 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_17_17_0 (state Temporary) 2024.10.22 00:22:01.162759 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_17_17_0 to 202410_30_30_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:01.225169 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 269 2024.10.22 00:22:01.528135 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 7.25 MiB, will set to 1.59 GiB (RSS), difference: 2.18 MiB 2024.10.22 00:22:01.954974 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.19589869 sec. 2024.10.22 00:22:03.456678 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 6.07 MiB, will set to 1.59 GiB (RSS), difference: -2.34 MiB 2024.10.22 00:22:03.468034 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:03.469739 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:04.185562 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1889 entries to flush up to offset 28950 2024.10.22 00:22:04.650415 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 20 entries to flush up to offset 1862 2024.10.22 00:22:04.860824 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:04.863644 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:04.885121 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_31_31_0 (state Temporary) 2024.10.22 00:22:04.924101 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:04.939568 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:04.939190 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_31_31_0 to 202410_44_44_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:05.014237 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1862 2024.10.22 00:22:05.079809 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_17_17_0 (state Temporary) 2024.10.22 00:22:05.156156 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_17_17_0 to 202410_24_24_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:05.286121 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 28950 2024.10.22 00:22:05.501098 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 47 entries to flush up to offset 2855 2024.10.22 00:22:05.538410 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 31 2024.10.22 00:22:05.638564 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:05.640331 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:05.680336 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_17_17_0 (state Temporary) 2024.10.22 00:22:06.045314 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_17_17_0 to 202410_23_23_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:06.073621 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 2855 2024.10.22 00:22:06.256027 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:06.259806 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:06.406239 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 7.97 MiB, will set to 1.60 GiB (RSS), difference: 3.36 MiB 2024.10.22 00:22:06.452667 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_13_13_0 (state Temporary) 2024.10.22 00:22:06.751859 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_13_13_0 to 202410_23_23_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:06.816228 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 31 2024.10.22 00:22:07.114296 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_4_4_0 (state Temporary) 2024.10.22 00:22:07.325515 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_5_1} MergeTask::MergeProjectionsStage: Merge sorted 106 rows, containing 679 columns (679 merged, 0 gathered) in 56.063359868 sec., 1.8907179350216392 rows/sec., 2.60 KiB/sec. 2024.10.22 00:22:08.167832 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_5_1} MergedBlockOutputStream: filled checksums 202410_1_5_1 (state Temporary) 2024.10.22 00:22:08.728981 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 275 2024.10.22 00:22:09.020073 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 28 entries to flush up to offset 1890 2024.10.22 00:22:09.023739 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:09.027468 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:09.080373 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_18_18_0 (state Temporary) 2024.10.22 00:22:09.131661 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_18_18_0 to 202410_31_31_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:09.179210 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:22:09.180261 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 275 2024.10.22 00:22:09.182099 [ 1217 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_26_5 to 202410_31_31_0 2024.10.22 00:22:09.183642 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:22:09.204831 [ 1217 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:09.210607 [ 1217 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:09.224091 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_26_5 to 202410_31_31_0 into Compact with storage Full 2024.10.22 00:22:09.247280 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:09.251007 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:09.261509 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:22:09.267392 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_26_5, total 310 rows starting from the beginning of the part 2024.10.22 00:22:09.276189 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_27_27_0, total 27 rows starting from the beginning of the part 2024.10.22 00:22:09.283926 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_28_28_0, total 21 rows starting from the beginning of the part 2024.10.22 00:22:09.296823 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_32_32_0 (state Temporary) 2024.10.22 00:22:09.313263 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_29_29_0, total 6 rows starting from the beginning of the part 2024.10.22 00:22:09.347348 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_30_30_0, total 11 rows starting from the beginning of the part 2024.10.22 00:22:09.441392 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_32_32_0 to 202410_45_45_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:09.476486 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_31_31_0, total 6 rows starting from the beginning of the part 2024.10.22 00:22:09.514948 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1890 2024.10.22 00:22:09.566658 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 7.52 MiB, will set to 1.61 GiB (RSS), difference: -6.48 MiB 2024.10.22 00:22:10.736809 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} MergeTask::MergeProjectionsStage: Merge sorted 381 rows, containing 12 columns (12 merged, 0 gathered) in 1.51826391 sec., 250.9445146463371 rows/sec., 37.35 KiB/sec. 2024.10.22 00:22:10.785665 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} MergedBlockOutputStream: filled checksums 202410_1_31_6 (state Temporary) 2024.10.22 00:22:10.891965 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_31_6 to 202410_1_31_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:10.899654 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_31_6} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_26_5, 202410_31_31_0] -> 202410_1_31_6 2024.10.22 00:22:11.047792 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:22:11.567468 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 5.36 MiB, will set to 1.61 GiB (RSS), difference: 3.72 MiB 2024.10.22 00:22:12.298557 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 30870 2024.10.22 00:22:12.881922 [ 1219 ] {} DatabaseCatalog: Execution took 217 ms. 2024.10.22 00:22:13.382165 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:13.383597 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:13.403340 [ 1220 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 00:22:13.525220 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 1926 2024.10.22 00:22:13.549370 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_18_18_0 (state Temporary) 2024.10.22 00:22:13.578709 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 45 entries to flush up to offset 2900 2024.10.22 00:22:13.680198 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:13.689376 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:13.732805 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_33_33_0 (state Temporary) 2024.10.22 00:22:13.765230 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_33_33_0 to 202410_46_46_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:13.788336 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_41_8 to 202410_46_46_0 2024.10.22 00:22:13.808781 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:13.811064 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:13.816170 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:13.819364 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1926 2024.10.22 00:22:13.819609 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:13.857797 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_18_18_0 to 202410_25_25_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:13.863166 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_18_18_0 (state Temporary) 2024.10.22 00:22:13.888931 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 30870 2024.10.22 00:22:13.911160 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_18_18_0 to 202410_24_24_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:13.925407 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_41_8 to 202410_46_46_0 into Compact with storage Full 2024.10.22 00:22:13.959836 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:22:13.963140 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_41_8, total 2619 rows starting from the beginning of the part 2024.10.22 00:22:13.975449 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_42_42_0, total 31 rows starting from the beginning of the part 2024.10.22 00:22:13.984132 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_43_43_0, total 23 rows starting from the beginning of the part 2024.10.22 00:22:14.003311 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_44_44_0, total 20 rows starting from the beginning of the part 2024.10.22 00:22:14.016006 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_45_45_0, total 28 rows starting from the beginning of the part 2024.10.22 00:22:14.052931 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_46_46_0, total 36 rows starting from the beginning of the part 2024.10.22 00:22:14.074688 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 2900 2024.10.22 00:22:14.319614 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 32 2024.10.22 00:22:14.548846 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 4.27 MiB, will set to 1.62 GiB (RSS), difference: 1.55 MiB 2024.10.22 00:22:14.596121 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:14.597642 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:14.759262 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_14_14_0 (state Temporary) 2024.10.22 00:22:14.883410 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} MergeTask::MergeProjectionsStage: Merge sorted 2757 rows, containing 13 columns (13 merged, 0 gathered) in 1.064770103 sec., 2589.291333624156 rows/sec., 437.13 KiB/sec. 2024.10.22 00:22:14.939034 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_14_14_0 to 202410_24_24_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:15.063605 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} MergedBlockOutputStream: filled checksums 202410_1_46_9 (state Temporary) 2024.10.22 00:22:15.116477 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 32 2024.10.22 00:22:15.216826 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_46_9 to 202410_1_46_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:15.221452 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_46_9} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_41_8, 202410_46_46_0] -> 202410_1_46_9 2024.10.22 00:22:15.264251 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:22:15.742564 [ 1241 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:22:15.783746 [ 1241 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:22:16.237110 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_5_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_5_1 to 202410_1_5_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:16.243894 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_5_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 5 parts: [202410_1_1_0, 202410_5_5_0] -> 202410_1_5_1 2024.10.22 00:22:16.654100 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 4.41 MiB, will set to 1.62 GiB (RSS), difference: 15.26 MiB 2024.10.22 00:22:16.683675 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 301 2024.10.22 00:22:16.784289 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:16.789401 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:16.856805 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_19_19_0 (state Temporary) 2024.10.22 00:22:16.866549 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_4_4_0 to 202410_7_7_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:17.006815 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_19_19_0 to 202410_32_32_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:17.077275 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 301 2024.10.22 00:22:17.123123 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.77 MiB. 2024.10.22 00:22:17.829821 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 1977 2024.10.22 00:22:17.984103 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:17.988341 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:18.157513 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_34_34_0 (state Temporary) 2024.10.22 00:22:18.297247 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_34_34_0 to 202410_47_47_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:18.428256 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 4.73 MiB, will set to 1.62 GiB (RSS), difference: 7.21 MiB 2024.10.22 00:22:18.461804 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 1977 2024.10.22 00:22:19.281460 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:22:19.283965 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:22:20.447343 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 6.91 MiB, will set to 1.62 GiB (RSS), difference: 3.86 MiB 2024.10.22 00:22:20.763538 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 126 2024.10.22 00:22:20.904001 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1753 entries to flush up to offset 32623 2024.10.22 00:22:21.582505 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 43 entries to flush up to offset 2943 2024.10.22 00:22:21.811404 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:21.813787 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:21.821119 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:21.823862 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:21.960722 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_19_19_0 (state Temporary) 2024.10.22 00:22:22.044375 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_19_19_0 (state Temporary) 2024.10.22 00:22:22.100690 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_19_19_0 to 202410_26_26_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:22.145809 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_19_19_0 to 202410_25_25_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:22.148253 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 32623 2024.10.22 00:22:22.188671 [ 1263 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_20_4 to 202410_25_25_0 2024.10.22 00:22:22.202567 [ 1263 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:22.206454 [ 1263 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:22.261023 [ 1238 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_21_4 to 202410_26_26_0 2024.10.22 00:22:22.277802 [ 1238 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:22.281749 [ 1238 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:22.315366 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 2943 2024.10.22 00:22:22.328659 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_20_4 to 202410_25_25_0 into Compact with storage Full 2024.10.22 00:22:22.329378 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_21_4 to 202410_26_26_0 into Compact with storage Full 2024.10.22 00:22:22.352898 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:22:22.362304 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_20_4, total 2574 rows starting from the beginning of the part 2024.10.22 00:22:22.363192 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:22:22.366904 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_21_4, total 37548 rows starting from the beginning of the part 2024.10.22 00:22:22.375070 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_22_22_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:22:22.390445 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_23_23_0, total 1567 rows starting from the beginning of the part 2024.10.22 00:22:22.391908 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_21_21_0, total 174 rows starting from the beginning of the part 2024.10.22 00:22:22.413018 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_24_24_0, total 1889 rows starting from the beginning of the part 2024.10.22 00:22:22.425294 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_22_22_0, total 164 rows starting from the beginning of the part 2024.10.22 00:22:22.440336 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_25_25_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:22:22.566846 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 2018 2024.10.22 00:22:22.619724 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 33 2024.10.22 00:22:22.661201 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_26_26_0, total 1753 rows starting from the beginning of the part 2024.10.22 00:22:22.662964 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_23_23_0, total 47 rows starting from the beginning of the part 2024.10.22 00:22:22.671762 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_24_24_0, total 45 rows starting from the beginning of the part 2024.10.22 00:22:22.683671 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_25_25_0, total 43 rows starting from the beginning of the part 2024.10.22 00:22:22.873134 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:22.875621 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:22.986955 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:22.994220 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:23.113838 [ 1151 ] {} DatabaseCatalog: Execution took 230 ms. 2024.10.22 00:22:23.152545 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_35_35_0 (state Temporary) 2024.10.22 00:22:23.206667 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_35_35_0 to 202410_48_48_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:23.210518 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_15_15_0 (state Temporary) 2024.10.22 00:22:23.276108 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2018 2024.10.22 00:22:23.344519 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_26_5/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:22:23.478715 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_15_15_0 to 202410_25_25_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:23.643996 [ 1162 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_20_4 to 202410_25_25_0 2024.10.22 00:22:23.661097 [ 1162 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:23.662645 [ 1162 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.87 GiB. 2024.10.22 00:22:23.675726 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_20_4 to 202410_25_25_0 into Compact with storage Full 2024.10.22 00:22:23.752611 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 33 2024.10.22 00:22:23.781627 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.81 GiB, free memory in arenas 8.34 MiB, will set to 1.63 GiB (RSS), difference: -8.68 MiB 2024.10.22 00:22:23.850709 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:22:23.860883 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_20_4, total 64 rows starting from the beginning of the part 2024.10.22 00:22:23.875986 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_21_21_0, total 1 rows starting from the beginning of the part 2024.10.22 00:22:24.010876 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_22_22_0, total 1 rows starting from the beginning of the part 2024.10.22 00:22:24.036648 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_2_2_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:22:24.162782 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} MergeTask::MergeProjectionsStage: Merge sorted 3047 rows, containing 13 columns (13 merged, 0 gathered) in 1.841261221 sec., 1654.8439543766399 rows/sec., 723.62 KiB/sec. 2024.10.22 00:22:24.171845 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_23_23_0, total 1 rows starting from the beginning of the part 2024.10.22 00:22:24.333333 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_24_24_0, total 1 rows starting from the beginning of the part 2024.10.22 00:22:24.371158 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} MergedBlockOutputStream: filled checksums 202410_1_25_5 (state Temporary) 2024.10.22 00:22:24.374380 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_25_25_0, total 1 rows starting from the beginning of the part 2024.10.22 00:22:24.579358 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 308 2024.10.22 00:22:24.676890 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:24.679713 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.86 GiB. 2024.10.22 00:22:24.867257 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_20_20_0 (state Temporary) 2024.10.22 00:22:24.875434 [ 1696 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/liw/xawtepouskgsykdstujwboanzwmdw, size 714721 2024.10.22 00:22:24.876180 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_25_5 to 202410_1_25_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:24.886342 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_25_5} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_20_4, 202410_25_25_0] -> 202410_1_25_5 2024.10.22 00:22:24.974379 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.62 MiB. 2024.10.22 00:22:25.154123 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_20_20_0 to 202410_33_33_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:25.518224 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 308 2024.10.22 00:22:25.594131 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 10.30 MiB, will set to 1.63 GiB (RSS), difference: 98.95 MiB 2024.10.22 00:22:25.858435 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} MergeTask::MergeProjectionsStage: Merge sorted 46213 rows, containing 4 columns (4 merged, 0 gathered) in 3.538218841 sec., 13061.091491711946 rows/sec., 193.72 KiB/sec. 2024.10.22 00:22:26.262113 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} MergedBlockOutputStream: filled checksums 202410_1_26_5 (state Temporary) 2024.10.22 00:22:26.298764 [ 1169 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 00:22:26.317617 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_26_5 to 202410_1_26_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:26.323721 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_26_5} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_21_4, 202410_26_26_0] -> 202410_1_26_5 2024.10.22 00:22:26.401636 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:22:26.828775 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} MergeTask::MergeProjectionsStage: Merge sorted 69 rows, containing 32 columns (32 merged, 0 gathered) in 3.160590239 sec., 21.83136527746519 rows/sec., 3.43 KiB/sec. 2024.10.22 00:22:26.859982 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} MergedBlockOutputStream: filled checksums 202410_1_25_5 (state Temporary) 2024.10.22 00:22:27.288089 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 2070 2024.10.22 00:22:27.378518 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_25_5 to 202410_1_25_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:27.385629 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_25_5} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_20_4, 202410_25_25_0] -> 202410_1_25_5 2024.10.22 00:22:27.456817 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:22:27.640990 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:27.656173 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.82 GiB. 2024.10.22 00:22:27.670385 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.81 GiB, free memory in arenas 10.07 MiB, will set to 1.62 GiB (RSS), difference: 155.34 MiB 2024.10.22 00:22:27.680948 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_36_36_0 (state Temporary) 2024.10.22 00:22:27.724093 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_36_36_0 to 202410_49_49_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:27.920002 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2070 2024.10.22 00:22:28.092764 [ 1698 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:22:28.478908 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 167 2024.10.22 00:22:28.868110 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_2_2_0 (state Temporary) 2024.10.22 00:22:28.876839 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/asw/efaccsffiihminvbjsgpicpynzjpx (LogSeriesLimiter: on interval from 2024-10-22 00:21:49 to 2024-10-22 00:22:28 accepted series 1 / 130 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:22:28.961081 [ 1691 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:22:28.962659 [ 1680 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:22:28.980470 [ 1673 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/asw/efaccsffiihminvbjsgpicpynzjpx, size 4 2024.10.22 00:22:29.000225 [ 1705 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yiw/qddafdpisykehdkzqaqhgxvkuetjx, size 600 2024.10.22 00:22:29.152417 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1703 entries to flush up to offset 34326 2024.10.22 00:22:29.283857 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:22:29.288504 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:22:29.725296 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.81 GiB, free memory in arenas 8.70 MiB, will set to 1.58 GiB (RSS), difference: 104.21 MiB 2024.10.22 00:22:29.825193 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 125 entries to flush up to offset 3068 2024.10.22 00:22:29.974215 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:29.978665 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:30.171040 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:30.173489 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:30.198571 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_20_20_0 (state Temporary) 2024.10.22 00:22:30.310343 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_20_20_0 to 202410_27_27_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:30.432957 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_20_20_0 (state Temporary) 2024.10.22 00:22:30.457822 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 34326 2024.10.22 00:22:30.601555 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_20_20_0 to 202410_26_26_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:30.661386 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3068 2024.10.22 00:22:30.788540 [ 1207 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:22:31.048061 [ 1207 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:22:31.055889 [ 1207 ] {} DNSCacheUpdater: Execution took 264 ms. 2024.10.22 00:22:31.255194 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 34 2024.10.22 00:22:31.390412 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 8.40 MiB, will set to 1.56 GiB (RSS), difference: -14.66 MiB 2024.10.22 00:22:31.421509 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:31.424712 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:31.595095 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_16_16_0 (state Temporary) 2024.10.22 00:22:31.676470 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_16_16_0 to 202410_26_26_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:31.935194 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 2112 2024.10.22 00:22:32.184510 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 34 2024.10.22 00:22:32.280594 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:32.384237 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:32.540000 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_37_37_0 (state Temporary) 2024.10.22 00:22:32.580143 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_37_37_0 to 202410_50_50_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:32.737495 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2112 2024.10.22 00:22:33.024629 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 335 2024.10.22 00:22:33.221072 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:33.225213 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:33.268486 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_21_21_0 (state Temporary) 2024.10.22 00:22:33.349094 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_21_21_0 to 202410_34_34_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:33.378407 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 335 2024.10.22 00:22:35.180166 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 00:22:36.755408 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 14 entries to flush up to offset 2126 2024.10.22 00:22:36.897700 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:36.905786 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:37.060311 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_38_38_0 (state Temporary) 2024.10.22 00:22:37.324944 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_38_38_0 to 202410_51_51_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:37.379402 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2126 2024.10.22 00:22:37.380765 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_46_9 to 202410_51_51_0 2024.10.22 00:22:37.402595 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:37.406390 [ 1240 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:37.434470 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_46_9 to 202410_51_51_0 into Compact with storage Full 2024.10.22 00:22:37.463780 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 36246 2024.10.22 00:22:37.574988 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:22:37.583062 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_46_9, total 2757 rows starting from the beginning of the part 2024.10.22 00:22:37.603850 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_47_47_0, total 51 rows starting from the beginning of the part 2024.10.22 00:22:37.619641 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_48_48_0, total 41 rows starting from the beginning of the part 2024.10.22 00:22:37.645687 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_49_49_0, total 52 rows starting from the beginning of the part 2024.10.22 00:22:37.663542 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_50_50_0, total 42 rows starting from the beginning of the part 2024.10.22 00:22:37.708434 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_51_51_0, total 14 rows starting from the beginning of the part 2024.10.22 00:22:38.068253 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:38.070459 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:38.175811 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 30 entries to flush up to offset 3098 2024.10.22 00:22:38.189204 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_21_21_0 (state Temporary) 2024.10.22 00:22:38.371198 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:38.372882 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:38.408072 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_21_21_0 to 202410_28_28_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:38.423230 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_21_21_0 (state Temporary) 2024.10.22 00:22:38.454743 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 36246 2024.10.22 00:22:38.464686 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_21_21_0 to 202410_27_27_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:38.490552 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3098 2024.10.22 00:22:38.678357 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} MergeTask::MergeProjectionsStage: Merge sorted 2957 rows, containing 13 columns (13 merged, 0 gathered) in 1.260679142 sec., 2345.561135650168 rows/sec., 392.17 KiB/sec. 2024.10.22 00:22:38.826388 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} MergedBlockOutputStream: filled checksums 202410_1_51_10 (state Temporary) 2024.10.22 00:22:39.168955 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_51_10 to 202410_1_51_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:39.173307 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_51_10} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_46_9, 202410_51_51_0] -> 202410_1_51_10 2024.10.22 00:22:39.231011 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:22:39.285361 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:22:39.291900 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:22:39.687430 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 35 2024.10.22 00:22:40.179902 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:40.182579 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:40.377264 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_17_17_0 (state Temporary) 2024.10.22 00:22:40.647167 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_17_17_0 to 202410_27_27_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:40.731896 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 35 2024.10.22 00:22:40.883272 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 347 2024.10.22 00:22:41.020323 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:41.023701 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:41.064563 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_22_22_0 (state Temporary) 2024.10.22 00:22:41.109547 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_22_22_0 to 202410_35_35_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:41.169244 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 347 2024.10.22 00:22:41.382649 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 2174 2024.10.22 00:22:41.558468 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:41.561623 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:41.648715 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_39_39_0 (state Temporary) 2024.10.22 00:22:41.677295 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 7.03 MiB, will set to 1.56 GiB (RSS), difference: -5.21 MiB 2024.10.22 00:22:41.729521 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_39_39_0 to 202410_52_52_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:41.782273 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2174 2024.10.22 00:22:43.379631 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 7.22 MiB, will set to 1.56 GiB (RSS), difference: 5.33 MiB 2024.10.22 00:22:43.648928 [ 1277 ] {} DatabaseCatalog: Execution took 292 ms. 2024.10.22 00:22:45.461236 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 37782 2024.10.22 00:22:45.793111 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 10 entries to flush up to offset 2184 2024.10.22 00:22:45.904699 [ 1159 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 00:22:46.005323 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 23 entries to flush up to offset 3121 2024.10.22 00:22:46.102278 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:46.105733 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:46.137333 [ 1158 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:22:46.150182 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_40_40_0 (state Temporary) 2024.10.22 00:22:46.164098 [ 1158 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:22:46.273802 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_40_40_0 to 202410_53_53_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:46.324708 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2184 2024.10.22 00:22:46.536044 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:46.539869 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:46.571339 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:46.574667 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:46.603332 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_22_22_0 (state Temporary) 2024.10.22 00:22:46.653298 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_22_22_0 (state Temporary) 2024.10.22 00:22:46.757074 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_22_22_0 to 202410_28_28_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:46.761226 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_22_22_0 to 202410_29_29_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:46.784914 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3121 2024.10.22 00:22:46.996403 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 37782 2024.10.22 00:22:47.266566 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:47.268757 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:48.671998 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 352 2024.10.22 00:22:49.118723 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:49.130867 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:49.271890 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_23_23_0 (state Temporary) 2024.10.22 00:22:49.293562 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:22:49.299234 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:22:49.432710 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_23_23_0 to 202410_36_36_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:49.471696 [ 1194 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_31_6 to 202410_36_36_0 2024.10.22 00:22:49.571014 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 6.24 MiB, will set to 1.56 GiB (RSS), difference: -5.29 MiB 2024.10.22 00:22:49.579675 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 352 2024.10.22 00:22:49.585911 [ 1194 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:49.592502 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:49.729809 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_31_6 to 202410_36_36_0 into Compact with storage Full 2024.10.22 00:22:49.759605 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:22:49.764395 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_31_6, total 381 rows starting from the beginning of the part 2024.10.22 00:22:49.775928 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_32_32_0, total 26 rows starting from the beginning of the part 2024.10.22 00:22:49.796729 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_33_33_0, total 7 rows starting from the beginning of the part 2024.10.22 00:22:49.814566 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_34_34_0, total 27 rows starting from the beginning of the part 2024.10.22 00:22:49.835818 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_35_35_0, total 12 rows starting from the beginning of the part 2024.10.22 00:22:49.854673 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_36_36_0, total 5 rows starting from the beginning of the part 2024.10.22 00:22:50.335984 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 2226 2024.10.22 00:22:50.708439 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} MergeTask::MergeProjectionsStage: Merge sorted 458 rows, containing 12 columns (12 merged, 0 gathered) in 1.103175226 sec., 415.1652332337637 rows/sec., 62.30 KiB/sec. 2024.10.22 00:22:50.854092 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} MergedBlockOutputStream: filled checksums 202410_1_36_7 (state Temporary) 2024.10.22 00:22:50.926980 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_36_7 to 202410_1_36_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:50.933747 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:51.035334 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_36_7} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_31_6, 202410_36_36_0] -> 202410_1_36_7 2024.10.22 00:22:51.042463 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:51.067528 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:22:51.213360 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_41_41_0 (state Temporary) 2024.10.22 00:22:51.283664 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_41_41_0 to 202410_54_54_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:51.356498 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2226 2024.10.22 00:22:51.824391 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_5_5_0 (state Temporary) 2024.10.22 00:22:51.847801 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 5.88 MiB, will set to 1.56 GiB (RSS), difference: -4.20 MiB 2024.10.22 00:22:54.003835 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1825 entries to flush up to offset 39607 2024.10.22 00:22:54.294968 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 30 entries to flush up to offset 3151 2024.10.22 00:22:54.443301 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:54.451023 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:54.583983 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_23_23_0 (state Temporary) 2024.10.22 00:22:54.637833 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_23_23_0 to 202410_29_29_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:54.680547 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3151 2024.10.22 00:22:55.104850 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:55.218796 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:55.366671 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 22 entries to flush up to offset 2248 2024.10.22 00:22:55.452499 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_23_23_0 (state Temporary) 2024.10.22 00:22:55.512302 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_23_23_0 to 202410_30_30_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:55.572854 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 39607 2024.10.22 00:22:55.584089 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:55.589741 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:55.654341 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_42_42_0 (state Temporary) 2024.10.22 00:22:55.736926 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 36 2024.10.22 00:22:55.790984 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_42_42_0 to 202410_55_55_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:55.863366 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2248 2024.10.22 00:22:56.016781 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:56.020346 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:56.109351 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_18_18_0 (state Temporary) 2024.10.22 00:22:56.175866 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_18_18_0 to 202410_28_28_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:56.280301 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 36 2024.10.22 00:22:57.081440 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 365 2024.10.22 00:22:57.248178 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hmo/qmolxgfjzjouudoskyiojcdximpdi. 2024.10.22 00:22:57.417336 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:22:57.422386 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:22:57.476613 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_24_24_0 (state Temporary) 2024.10.22 00:22:57.623387 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_24_24_0 to 202410_37_37_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:22:57.771499 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 365 2024.10.22 00:22:59.296370 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:22:59.301243 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:22:59.868192 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 24 entries to flush up to offset 2272 2024.10.22 00:23:00.097281 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:00.101447 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:23:00.165400 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:23:00.266971 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_43_43_0 (state Temporary) 2024.10.22 00:23:00.303999 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_43_43_0 to 202410_56_56_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:00.339639 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_51_10 to 202410_56_56_0 2024.10.22 00:23:00.363929 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:00.367078 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:23:00.368530 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2272 2024.10.22 00:23:00.396134 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_51_10 to 202410_56_56_0 into Compact with storage Full 2024.10.22 00:23:00.422216 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:23:00.435346 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_51_10, total 2957 rows starting from the beginning of the part 2024.10.22 00:23:00.457574 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_52_52_0, total 48 rows starting from the beginning of the part 2024.10.22 00:23:00.467422 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_53_53_0, total 10 rows starting from the beginning of the part 2024.10.22 00:23:00.482581 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_54_54_0, total 42 rows starting from the beginning of the part 2024.10.22 00:23:00.513572 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 5.82 MiB, will set to 1.56 GiB (RSS), difference: 22.90 MiB 2024.10.22 00:23:00.526525 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_55_55_0, total 22 rows starting from the beginning of the part 2024.10.22 00:23:00.655765 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_56_56_0, total 24 rows starting from the beginning of the part 2024.10.22 00:23:01.167472 [ 1190 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:23:01.369161 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} MergeTask::MergeProjectionsStage: Merge sorted 3103 rows, containing 13 columns (13 merged, 0 gathered) in 0.98486897 sec., 3150.672926572151 rows/sec., 521.57 KiB/sec. 2024.10.22 00:23:01.401693 [ 1190 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:23:01.404891 [ 1190 ] {} DNSCacheUpdater: Execution took 238 ms. 2024.10.22 00:23:01.422361 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} MergedBlockOutputStream: filled checksums 202410_1_56_11 (state Temporary) 2024.10.22 00:23:01.475595 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_56_11 to 202410_1_56_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:01.477298 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_56_11} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_51_10, 202410_56_56_0] -> 202410_1_56_11 2024.10.22 00:23:01.615119 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:23:02.183800 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 28 entries to flush up to offset 3179 2024.10.22 00:23:02.289730 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:02.296755 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:23:02.350302 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_24_24_0 (state Temporary) 2024.10.22 00:23:02.388209 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.81 GiB, free memory in arenas 3.75 MiB, will set to 1.55 GiB (RSS), difference: 98.76 MiB 2024.10.22 00:23:02.389582 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_24_24_0 to 202410_30_30_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:02.441755 [ 1232 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_25_5 to 202410_30_30_0 2024.10.22 00:23:02.548546 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3179 2024.10.22 00:23:02.558920 [ 1232 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:02.561830 [ 1232 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:23:02.570123 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_25_5 to 202410_30_30_0 into Compact with storage Full 2024.10.22 00:23:02.577410 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1679 entries to flush up to offset 41286 2024.10.22 00:23:02.583587 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:23:02.593532 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_25_5, total 3047 rows starting from the beginning of the part 2024.10.22 00:23:02.619422 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_26_26_0, total 125 rows starting from the beginning of the part 2024.10.22 00:23:02.642202 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_27_27_0, total 30 rows starting from the beginning of the part 2024.10.22 00:23:02.659853 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_28_28_0, total 23 rows starting from the beginning of the part 2024.10.22 00:23:02.769641 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_29_29_0, total 30 rows starting from the beginning of the part 2024.10.22 00:23:02.777029 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_30_30_0, total 28 rows starting from the beginning of the part 2024.10.22 00:23:03.175018 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:03.190624 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:23:03.337823 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_5_5_0 to 202410_8_8_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:03.395622 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.81 GiB, free memory in arenas 4.78 MiB, will set to 1.55 GiB (RSS), difference: 47.46 MiB 2024.10.22 00:23:03.571115 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} MergeTask::MergeProjectionsStage: Merge sorted 3283 rows, containing 13 columns (13 merged, 0 gathered) in 1.003873425 sec., 3270.33261190274 rows/sec., 1.39 MiB/sec. 2024.10.22 00:23:03.623117 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_24_24_0 (state Temporary) 2024.10.22 00:23:03.629558 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} MergedBlockOutputStream: filled checksums 202410_1_30_6 (state Temporary) 2024.10.22 00:23:03.670414 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_30_6 to 202410_1_30_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:03.673373 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_30_6} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_25_5, 202410_30_30_0] -> 202410_1_30_6 2024.10.22 00:23:03.715229 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.62 MiB. 2024.10.22 00:23:03.783903 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 37 2024.10.22 00:23:03.798610 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_24_24_0 to 202410_31_31_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:03.811151 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hmo/qmolxgfjzjouudoskyiojcdximpdi. 2024.10.22 00:23:03.829731 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 41286 2024.10.22 00:23:03.846609 [ 1256 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_26_5 to 202410_31_31_0 2024.10.22 00:23:03.859218 [ 1256 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:03.863043 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:23:03.875096 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_26_5 to 202410_31_31_0 into Compact with storage Full 2024.10.22 00:23:03.886543 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:23:03.913346 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_26_5, total 46213 rows starting from the beginning of the part 2024.10.22 00:23:03.968771 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:03.971729 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:23:04.081307 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_27_27_0, total 1703 rows starting from the beginning of the part 2024.10.22 00:23:04.093491 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_28_28_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:23:04.111381 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_29_29_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:23:04.136949 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_19_19_0 (state Temporary) 2024.10.22 00:23:04.150898 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_30_30_0, total 1825 rows starting from the beginning of the part 2024.10.22 00:23:04.159155 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_31_31_0, total 1679 rows starting from the beginning of the part 2024.10.22 00:23:04.232614 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_19_19_0 to 202410_29_29_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:04.398693 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 2347 2024.10.22 00:23:04.414782 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 37 2024.10.22 00:23:04.619961 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 5.61 MiB, will set to 1.55 GiB (RSS), difference: 18.84 MiB 2024.10.22 00:23:04.776493 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:04.879039 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.78 GiB. 2024.10.22 00:23:05.038350 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_44_44_0 (state Temporary) 2024.10.22 00:23:05.174177 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_44_44_0 to 202410_57_57_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:05.274108 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 386 2024.10.22 00:23:05.334269 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2347 2024.10.22 00:23:05.485325 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:05.489732 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.77 GiB. 2024.10.22 00:23:05.540425 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_25_25_0 (state Temporary) 2024.10.22 00:23:05.689758 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_25_25_0 to 202410_38_38_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:06.058816 [ 1264 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 00:23:06.085127 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 386 2024.10.22 00:23:06.267801 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 167 2024.10.22 00:23:06.559852 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 9.42 MiB, will set to 1.55 GiB (RSS), difference: 3.41 MiB 2024.10.22 00:23:07.400363 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} MergeTask::MergeProjectionsStage: Merge sorted 54876 rows, containing 4 columns (4 merged, 0 gathered) in 3.528323536 sec., 15552.995477906763 rows/sec., 230.22 KiB/sec. 2024.10.22 00:23:07.677182 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} MergedBlockOutputStream: filled checksums 202410_1_31_6 (state Temporary) 2024.10.22 00:23:07.735377 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_31_6 to 202410_1_31_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:07.761227 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_31_6} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_26_5, 202410_31_31_0] -> 202410_1_31_6 2024.10.22 00:23:07.813087 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:23:08.359385 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 11.27 MiB, will set to 1.55 GiB (RSS), difference: 7.42 MiB 2024.10.22 00:23:09.296307 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:23:09.300490 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:23:09.351159 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 25 entries to flush up to offset 2372 2024.10.22 00:23:09.428455 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.81 GiB, free memory in arenas 11.11 MiB, will set to 1.55 GiB (RSS), difference: -1.15 MiB 2024.10.22 00:23:09.655977 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:09.657593 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.77 GiB. 2024.10.22 00:23:09.902373 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_45_45_0 (state Temporary) 2024.10.22 00:23:10.055228 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 58 entries to flush up to offset 3237 2024.10.22 00:23:10.056164 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_45_45_0 to 202410_58_58_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:10.107367 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2372 2024.10.22 00:23:10.260743 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:10.262323 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.77 GiB. 2024.10.22 00:23:10.403454 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_25_25_0 (state Temporary) 2024.10.22 00:23:10.457255 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_25_25_0 to 202410_31_31_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:10.488920 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3237 2024.10.22 00:23:10.835869 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2256 entries to flush up to offset 43542 2024.10.22 00:23:11.883284 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:11.886223 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.77 GiB. 2024.10.22 00:23:11.919619 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 38 2024.10.22 00:23:12.007951 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_25_25_0 (state Temporary) 2024.10.22 00:23:12.118647 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_25_25_0 to 202410_32_32_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:12.162166 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 43542 2024.10.22 00:23:12.177737 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:12.181363 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.77 GiB. 2024.10.22 00:23:12.368413 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_20_20_0 (state Temporary) 2024.10.22 00:23:12.666711 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_20_20_0 to 202410_30_30_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:12.726442 [ 1168 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_25_5 to 202410_30_30_0 2024.10.22 00:23:12.761355 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 38 2024.10.22 00:23:12.851661 [ 1168 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:12.867268 [ 1168 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.77 GiB. 2024.10.22 00:23:12.905343 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_25_5 to 202410_30_30_0 into Compact with storage Full 2024.10.22 00:23:12.968917 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:23:12.978683 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_25_5, total 69 rows starting from the beginning of the part 2024.10.22 00:23:13.156560 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_26_26_0, total 1 rows starting from the beginning of the part 2024.10.22 00:23:13.208858 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_27_27_0, total 1 rows starting from the beginning of the part 2024.10.22 00:23:13.267103 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_28_28_0, total 1 rows starting from the beginning of the part 2024.10.22 00:23:13.520994 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_29_29_0, total 1 rows starting from the beginning of the part 2024.10.22 00:23:13.572429 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_30_30_0, total 1 rows starting from the beginning of the part 2024.10.22 00:23:13.591432 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 398 2024.10.22 00:23:13.908206 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 45 entries to flush up to offset 212 2024.10.22 00:23:13.921476 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:13.931500 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.77 GiB. 2024.10.22 00:23:14.074674 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_26_26_0 (state Temporary) 2024.10.22 00:23:14.240613 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 2413 2024.10.22 00:23:14.245132 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_26_26_0 to 202410_39_39_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:14.292433 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 398 2024.10.22 00:23:14.460861 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:14.464740 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.77 GiB. 2024.10.22 00:23:14.622745 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_46_46_0 (state Temporary) 2024.10.22 00:23:14.784191 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_46_46_0 to 202410_59_59_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:14.867147 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2413 2024.10.22 00:23:15.647113 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} MergeTask::MergeProjectionsStage: Merge sorted 74 rows, containing 32 columns (32 merged, 0 gathered) in 2.769978431 sec., 26.715009464274054 rows/sec., 3.96 KiB/sec. 2024.10.22 00:23:15.699144 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} MergedBlockOutputStream: filled checksums 202410_1_30_6 (state Temporary) 2024.10.22 00:23:15.832649 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.81 GiB, free memory in arenas 7.61 MiB, will set to 1.54 GiB (RSS), difference: -6.63 MiB 2024.10.22 00:23:15.944686 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_30_6 to 202410_1_30_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:15.948561 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_30_6} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_25_5, 202410_30_30_0] -> 202410_1_30_6 2024.10.22 00:23:15.985092 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:23:16.406638 [ 1206 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:23:16.448254 [ 1206 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:23:17.762985 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 6.23 MiB, will set to 1.54 GiB (RSS), difference: 4.41 MiB 2024.10.22 00:23:18.009470 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 28 entries to flush up to offset 3265 2024.10.22 00:23:18.226805 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:18.241212 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.76 GiB. 2024.10.22 00:23:18.270177 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_26_26_0 (state Temporary) 2024.10.22 00:23:18.331168 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_26_26_0 to 202410_32_32_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:18.548597 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3265 2024.10.22 00:23:18.872163 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_2_2_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:23:18.872280 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 23 entries to flush up to offset 2436 2024.10.22 00:23:19.083427 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:19.087878 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.76 GiB. 2024.10.22 00:23:19.168351 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 45078 2024.10.22 00:23:19.250517 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_47_47_0 (state Temporary) 2024.10.22 00:23:19.300035 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:23:19.302410 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:23:19.404738 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_47_47_0 to 202410_60_60_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:19.569242 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2436 2024.10.22 00:23:19.661518 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_6_1, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0, 202410_10_10_0] 2024.10.22 00:23:19.674425 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_6_1, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0, 202410_10_10_0] 2024.10.22 00:23:19.702992 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_6_1, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0, 202410_10_10_0] 2024.10.22 00:23:20.099121 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:20.102819 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.76 GiB. 2024.10.22 00:23:20.204361 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_26_26_0 (state Temporary) 2024.10.22 00:23:20.263598 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_26_26_0 to 202410_33_33_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:20.284528 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 45078 2024.10.22 00:23:20.364477 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 39 2024.10.22 00:23:20.704875 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:20.715456 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.76 GiB. 2024.10.22 00:23:20.776404 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_21_21_0 (state Temporary) 2024.10.22 00:23:20.966773 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_21_21_0 to 202410_31_31_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:21.153596 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 39 2024.10.22 00:23:21.523557 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 3.68 MiB, will set to 1.54 GiB (RSS), difference: -1.83 MiB 2024.10.22 00:23:21.800265 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 411 2024.10.22 00:23:22.240741 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:22.245705 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.76 GiB. 2024.10.22 00:23:22.398022 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_27_27_0 (state Temporary) 2024.10.22 00:23:22.463881 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_27_27_0 to 202410_40_40_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:22.510833 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 411 2024.10.22 00:23:23.571072 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 2467 2024.10.22 00:23:23.615692 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hmd/tyrnontcbulbgbgdvntkenagwvjyf. 2024.10.22 00:23:23.636155 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 3.79 MiB, will set to 1.55 GiB (RSS), difference: 2.19 MiB 2024.10.22 00:23:23.669718 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vqq/fwxnxsymzfjehmjkoqtuqosgvgitd. 2024.10.22 00:23:23.777790 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:23.887962 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:23.941376 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_48_48_0 (state Temporary) 2024.10.22 00:23:24.084774 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_48_48_0 to 202410_61_61_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:24.123977 [ 1232 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_56_11 to 202410_61_61_0 2024.10.22 00:23:24.143346 [ 1232 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:24.157634 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2467 2024.10.22 00:23:24.159602 [ 1232 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:24.180592 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_56_11 to 202410_61_61_0 into Compact with storage Full 2024.10.22 00:23:24.296323 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vqq/fwxnxsymzfjehmjkoqtuqosgvgitd. 2024.10.22 00:23:24.295871 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:23:24.300194 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_1_1_0 to 201403_1_1_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:24.308492 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_56_11, total 3103 rows starting from the beginning of the part 2024.10.22 00:23:24.322786 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hmd/tyrnontcbulbgbgdvntkenagwvjyf. 2024.10.22 00:23:24.328215 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_57_57_0, total 75 rows starting from the beginning of the part 2024.10.22 00:23:24.353125 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_58_58_0, total 25 rows starting from the beginning of the part 2024.10.22 00:23:24.479209 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_59_59_0, total 41 rows starting from the beginning of the part 2024.10.22 00:23:24.512896 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_60_60_0, total 23 rows starting from the beginning of the part 2024.10.22 00:23:24.524995 [ 1236 ] {} DatabaseCatalog: Execution took 589 ms. 2024.10.22 00:23:24.549167 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_61_61_0, total 31 rows starting from the beginning of the part 2024.10.22 00:23:25.262235 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.11 GiB, peak 1.81 GiB, free memory in arenas 3.93 MiB, will set to 1.37 GiB (RSS), difference: 256.71 MiB 2024.10.22 00:23:25.832395 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} MergeTask::MergeProjectionsStage: Merge sorted 3298 rows, containing 13 columns (13 merged, 0 gathered) in 1.660281467 sec., 1986.4101753537193 rows/sec., 326.14 KiB/sec. 2024.10.22 00:23:25.983755 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} MergedBlockOutputStream: filled checksums 202410_1_61_12 (state Temporary) 2024.10.22 00:23:26.043937 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_61_12 to 202410_1_61_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:26.050086 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_61_12} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_56_11, 202410_61_61_0] -> 202410_1_61_12 2024.10.22 00:23:26.055589 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 29 entries to flush up to offset 3294 2024.10.22 00:23:26.078627 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:23:26.155602 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:26.157706 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:26.297286 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_27_27_0 (state Temporary) 2024.10.22 00:23:26.353707 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_27_27_0 to 202410_33_33_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:26.376956 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3294 2024.10.22 00:23:26.564917 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 997.06 MiB, peak 1.81 GiB, free memory in arenas 2.95 MiB, will set to 1.05 GiB (RSS), difference: 81.95 MiB 2024.10.22 00:23:27.291739 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 46998 2024.10.22 00:23:28.102700 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:28.107997 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:28.160125 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 2507 2024.10.22 00:23:28.215695 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_27_27_0 (state Temporary) 2024.10.22 00:23:28.348708 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:28.455418 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:28.489090 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_27_27_0 to 202410_34_34_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:28.516266 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 46998 2024.10.22 00:23:28.584979 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_49_49_0 (state Temporary) 2024.10.22 00:23:28.732885 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_49_49_0 to 202410_62_62_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:28.789043 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2507 2024.10.22 00:23:28.853520 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.81 GiB, free memory in arenas 3.54 MiB, will set to 1.09 GiB (RSS), difference: -56.80 MiB 2024.10.22 00:23:29.304220 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:23:29.306300 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:23:29.882729 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 1 2024.10.22 00:23:29.885758 [ 1545 ] {} SystemLog (system.part_log): Creating new table system.part_log for PartLog 2024.10.22 00:23:30.014935 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 424 2024.10.22 00:23:30.140891 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:30.247670 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:30.395974 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_28_28_0 (state Temporary) 2024.10.22 00:23:30.566275 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_28_28_0 to 202410_41_41_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:30.622844 [ 1266 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_36_7 to 202410_41_41_0 2024.10.22 00:23:30.645516 [ 1266 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:30.649102 [ 1266 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:30.684768 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_36_7 to 202410_41_41_0 into Compact with storage Full 2024.10.22 00:23:30.743112 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 424 2024.10.22 00:23:30.755180 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Loading data parts 2024.10.22 00:23:30.767910 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.11 GiB, peak 1.81 GiB, free memory in arenas 3.46 MiB, will set to 1.11 GiB (RSS), difference: 2.75 MiB 2024.10.22 00:23:30.880824 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:23:30.885781 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_36_7, total 458 rows starting from the beginning of the part 2024.10.22 00:23:30.902722 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_37_37_0, total 13 rows starting from the beginning of the part 2024.10.22 00:23:30.920237 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_38_38_0, total 21 rows starting from the beginning of the part 2024.10.22 00:23:30.936842 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_39_39_0, total 12 rows starting from the beginning of the part 2024.10.22 00:23:30.956349 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_40_40_0, total 13 rows starting from the beginning of the part 2024.10.22 00:23:30.968285 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_41_41_0, total 13 rows starting from the beginning of the part 2024.10.22 00:23:31.064667 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): There are no data parts 2024.10.22 00:23:31.457735 [ 1156 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:23:31.535360 [ 1156 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:23:31.579861 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:31.581280 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:31.772544 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_1_1_0 (state Temporary) 2024.10.22 00:23:31.859072 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} MergeTask::MergeProjectionsStage: Merge sorted 530 rows, containing 12 columns (12 merged, 0 gathered) in 1.185037476 sec., 447.24323975725474 rows/sec., 67.49 KiB/sec. 2024.10.22 00:23:31.886582 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_1_1_0 to 202410_1_1_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:31.921589 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} MergedBlockOutputStream: filled checksums 202410_1_41_8 (state Temporary) 2024.10.22 00:23:32.173734 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 1 2024.10.22 00:23:32.275349 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_41_8 to 202410_1_41_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:32.279322 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_41_8} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_36_7, 202410_41_41_0] -> 202410_1_41_8 2024.10.22 00:23:32.450968 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:23:32.671290 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.20 GiB, peak 1.81 GiB, free memory in arenas 2.14 MiB, will set to 1.16 GiB (RSS), difference: -40.34 MiB 2024.10.22 00:23:32.801383 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 2554 2024.10.22 00:23:33.013180 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:33.017525 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:33.065702 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_50_50_0 (state Temporary) 2024.10.22 00:23:33.107298 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:33.110257 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:33.123598 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_50_50_0 to 202410_63_63_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:33.172205 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2554 2024.10.22 00:23:33.853285 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_2_2_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:23:33.884472 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 30 entries to flush up to offset 3324 2024.10.22 00:23:34.106606 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:34.110777 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:34.174686 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_28_28_0 (state Temporary) 2024.10.22 00:23:34.435818 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_28_28_0 to 202410_34_34_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:34.458604 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.24 GiB, peak 1.81 GiB, free memory in arenas 2.33 MiB, will set to 1.20 GiB (RSS), difference: -42.35 MiB 2024.10.22 00:23:34.510898 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3324 2024.10.22 00:23:35.523198 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 48534 2024.10.22 00:23:35.632548 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 1.81 GiB, free memory in arenas 2.46 MiB, will set to 1.22 GiB (RSS), difference: -27.71 MiB 2024.10.22 00:23:36.158827 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 40 2024.10.22 00:23:36.335885 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_6_6_0 (state Temporary) 2024.10.22 00:23:36.618176 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:36.626463 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:36.658961 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:36.671896 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:36.845895 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_22_22_0 (state Temporary) 2024.10.22 00:23:36.908024 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_28_28_0 (state Temporary) 2024.10.22 00:23:36.920819 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_22_22_0 to 202410_32_32_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:36.970587 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_28_28_0 to 202410_35_35_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:36.989444 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 40 2024.10.22 00:23:37.096881 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 48534 2024.10.22 00:23:37.179410 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 2584 2024.10.22 00:23:37.395983 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:37.400770 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:37.466778 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_51_51_0 (state Temporary) 2024.10.22 00:23:37.614319 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_51_51_0 to 202410_64_64_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:37.672847 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2584 2024.10.22 00:23:37.865416 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.81 GiB, free memory in arenas 3.72 MiB, will set to 1.29 GiB (RSS), difference: -103.38 MiB 2024.10.22 00:23:38.248091 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 438 2024.10.22 00:23:38.572147 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:38.576497 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:38.752504 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_29_29_0 (state Temporary) 2024.10.22 00:23:38.799046 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_29_29_0 to 202410_42_42_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:38.965182 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 438 2024.10.22 00:23:39.319351 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:23:39.324839 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:23:39.752481 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 1.81 GiB, free memory in arenas 3.15 MiB, will set to 1.31 GiB (RSS), difference: -71.93 MiB 2024.10.22 00:23:41.405625 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.81 GiB, free memory in arenas 2.77 MiB, will set to 1.36 GiB (RSS), difference: -29.14 MiB 2024.10.22 00:23:41.686282 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 17 entries to flush up to offset 2601 2024.10.22 00:23:42.022977 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 20 entries to flush up to offset 3344 2024.10.22 00:23:42.119423 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:42.123362 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:42.147337 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:42.152531 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:42.175267 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_52_52_0 (state Temporary) 2024.10.22 00:23:42.289824 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_29_29_0 (state Temporary) 2024.10.22 00:23:42.519387 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.81 GiB, free memory in arenas 3.26 MiB, will set to 1.38 GiB (RSS), difference: -18.32 MiB 2024.10.22 00:23:42.547073 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_52_52_0 to 202410_65_65_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:42.554743 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_29_29_0 to 202410_35_35_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:42.629523 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3344 2024.10.22 00:23:42.636218 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_30_6 to 202410_35_35_0 2024.10.22 00:23:42.653371 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:42.657138 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:42.678977 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_30_6 to 202410_35_35_0 into Compact with storage Full 2024.10.22 00:23:42.747687 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2601 2024.10.22 00:23:42.817398 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:23:42.822657 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_30_6, total 3283 rows starting from the beginning of the part 2024.10.22 00:23:43.047550 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_31_31_0, total 58 rows starting from the beginning of the part 2024.10.22 00:23:43.059206 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_32_32_0, total 28 rows starting from the beginning of the part 2024.10.22 00:23:43.274930 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_33_33_0, total 29 rows starting from the beginning of the part 2024.10.22 00:23:43.301695 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_34_34_0, total 30 rows starting from the beginning of the part 2024.10.22 00:23:43.438208 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_35_35_0, total 20 rows starting from the beginning of the part 2024.10.22 00:23:44.104317 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 50454 2024.10.22 00:23:44.379050 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.81 GiB, free memory in arenas 3.45 MiB, will set to 1.40 GiB (RSS), difference: -33.44 MiB 2024.10.22 00:23:44.497688 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 41 2024.10.22 00:23:44.751754 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} MergeTask::MergeProjectionsStage: Merge sorted 3448 rows, containing 13 columns (13 merged, 0 gathered) in 2.07867086 sec., 1658.75226634004 rows/sec., 720.47 KiB/sec. 2024.10.22 00:23:44.821747 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} MergedBlockOutputStream: filled checksums 202410_1_35_7 (state Temporary) 2024.10.22 00:23:44.867352 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:44.871180 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:44.950556 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_6_6_0 to 202410_9_9_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:45.123248 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_35_7 to 202410_1_35_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:45.129418 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_35_7} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_30_6, 202410_35_35_0] -> 202410_1_35_7 2024.10.22 00:23:45.211538 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_29_29_0 (state Temporary) 2024.10.22 00:23:45.277440 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_29_29_0 to 202410_36_36_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:45.308429 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:45.311003 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:45.313244 [ 1240 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_31_6 to 202410_36_36_0 2024.10.22 00:23:45.315641 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 50454 2024.10.22 00:23:45.321638 [ 1240 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:45.328150 [ 1240 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:45.374919 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.74 MiB. 2024.10.22 00:23:45.399926 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.81 GiB, free memory in arenas 3.32 MiB, will set to 1.43 GiB (RSS), difference: 12.02 MiB 2024.10.22 00:23:45.416381 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_31_6 to 202410_36_36_0 into Compact with storage Full 2024.10.22 00:23:45.452331 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:23:45.457592 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_31_6, total 54876 rows starting from the beginning of the part 2024.10.22 00:23:45.464919 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_32_32_0, total 2256 rows starting from the beginning of the part 2024.10.22 00:23:45.470068 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_33_33_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:23:45.478831 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_34_34_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:23:45.490547 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_35_35_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:23:45.619985 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_36_36_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:23:45.663702 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_23_23_0 (state Temporary) 2024.10.22 00:23:45.736763 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_23_23_0 to 202410_33_33_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:45.823493 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 41 2024.10.22 00:23:46.467577 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 451 2024.10.22 00:23:46.481432 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.81 GiB, free memory in arenas 3.89 MiB, will set to 1.43 GiB (RSS), difference: -22.08 MiB 2024.10.22 00:23:46.553833 [ 1256 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:23:46.581778 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:46.586911 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:46.716005 [ 1256 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:23:46.735696 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_30_30_0 (state Temporary) 2024.10.22 00:23:46.853754 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 2663 2024.10.22 00:23:46.896893 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_30_30_0 to 202410_43_43_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:47.087647 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:47.104260 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:47.160681 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 451 2024.10.22 00:23:47.172499 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_53_53_0 (state Temporary) 2024.10.22 00:23:47.213092 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_53_53_0 to 202410_66_66_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:47.272848 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2663 2024.10.22 00:23:47.277533 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_61_12 to 202410_66_66_0 2024.10.22 00:23:47.299850 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:47.303424 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:47.341683 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_61_12 to 202410_66_66_0 into Compact with storage Full 2024.10.22 00:23:47.472493 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:23:47.477730 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_61_12, total 3298 rows starting from the beginning of the part 2024.10.22 00:23:47.487125 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_62_62_0, total 40 rows starting from the beginning of the part 2024.10.22 00:23:47.504504 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_63_63_0, total 47 rows starting from the beginning of the part 2024.10.22 00:23:47.529019 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_64_64_0, total 30 rows starting from the beginning of the part 2024.10.22 00:23:47.758703 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_65_65_0, total 17 rows starting from the beginning of the part 2024.10.22 00:23:47.768047 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_66_66_0, total 62 rows starting from the beginning of the part 2024.10.22 00:23:48.181490 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_36_7/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:23:48.427791 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 212 2024.10.22 00:23:48.476177 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.81 GiB, free memory in arenas 8.95 MiB, will set to 1.50 GiB (RSS), difference: -53.98 MiB 2024.10.22 00:23:48.640778 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} MergeTask::MergeProjectionsStage: Merge sorted 64044 rows, containing 4 columns (4 merged, 0 gathered) in 3.13358455 sec., 20437.935845707434 rows/sec., 302.24 KiB/sec. 2024.10.22 00:23:48.708265 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} MergedBlockOutputStream: filled checksums 202410_1_36_7 (state Temporary) 2024.10.22 00:23:48.853186 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_36_7 to 202410_1_36_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:48.855871 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_2_2_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:23:48.856958 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_36_7} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_31_6, 202410_36_36_0] -> 202410_1_36_7 2024.10.22 00:23:48.896240 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:23:49.283251 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} MergeTask::MergeProjectionsStage: Merge sorted 3494 rows, containing 13 columns (13 merged, 0 gathered) in 1.95313112 sec., 1788.922394519012 rows/sec., 291.44 KiB/sec. 2024.10.22 00:23:49.312140 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:23:49.360046 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} MergedBlockOutputStream: filled checksums 202410_1_66_13 (state Temporary) 2024.10.22 00:23:49.418647 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:23:49.428697 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_66_13 to 202410_1_66_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:49.433177 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_66_13} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_61_12, 202410_66_66_0] -> 202410_1_66_13 2024.10.22 00:23:49.468150 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:23:50.133278 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 65 entries to flush up to offset 3409 2024.10.22 00:23:50.363633 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.81 GiB, free memory in arenas 10.30 MiB, will set to 1.66 GiB (RSS), difference: -97.35 MiB 2024.10.22 00:23:50.539571 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:50.545371 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:50.583868 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_30_30_0 (state Temporary) 2024.10.22 00:23:50.741805 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_30_30_0 to 202410_36_36_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:50.776389 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3409 2024.10.22 00:23:51.283205 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 2705 2024.10.22 00:23:51.375506 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:51.376959 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:51.626715 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_54_54_0 (state Temporary) 2024.10.22 00:23:51.675399 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_54_54_0 to 202410_67_67_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:51.735808 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2705 2024.10.22 00:23:52.321683 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 52374 2024.10.22 00:23:52.379715 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.24 GiB, peak 1.81 GiB, free memory in arenas 8.01 MiB, will set to 1.50 GiB (RSS), difference: 263.44 MiB 2024.10.22 00:23:53.163610 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 451.34 MiB using storage policy from min volume index 0 2024.10.22 00:23:53.327007 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 42 2024.10.22 00:23:53.377236 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:53.380822 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:53.723921 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_30_30_0 (state Temporary) 2024.10.22 00:23:53.772881 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:53.775225 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:53.802384 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_30_30_0 to 202410_37_37_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:53.852716 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_24_24_0 (state Temporary) 2024.10.22 00:23:53.956267 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 52374 2024.10.22 00:23:54.275514 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_24_24_0 to 202410_34_34_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:54.355997 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 42 2024.10.22 00:23:54.662675 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 471 2024.10.22 00:23:54.719891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ntb/bxjaxbtvedbkcfcvsqexgzlnsmebt (LogSeriesLimiter: on interval from 2024-10-22 00:22:28 to 2024-10-22 00:23:54 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:23:54.836214 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:54.839889 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:54.973719 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_31_31_0 (state Temporary) 2024.10.22 00:23:55.015858 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_31_31_0 to 202410_44_44_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:55.077648 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 471 2024.10.22 00:23:55.747992 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 2732 2024.10.22 00:23:55.867180 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:55.870105 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:55.911389 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_55_55_0 (state Temporary) 2024.10.22 00:23:55.956431 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_55_55_0 to 202410_68_68_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:55.965249 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 254 2024.10.22 00:23:55.991627 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2732 2024.10.22 00:23:56.255476 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 8.11 MiB, will set to 1.50 GiB (RSS), difference: -116.92 MiB 2024.10.22 00:23:57.365545 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 6.81 MiB, will set to 1.50 GiB (RSS), difference: -123.48 MiB 2024.10.22 00:23:58.284338 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 21 entries to flush up to offset 3430 2024.10.22 00:23:58.314995 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 6.94 MiB, will set to 1.50 GiB (RSS), difference: -102.64 MiB 2024.10.22 00:23:58.367979 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:23:58.369834 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:23:58.505667 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_31_31_0 (state Temporary) 2024.10.22 00:23:58.658094 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_31_31_0 to 202410_37_37_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:23:58.686848 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3430 2024.10.22 00:23:58.871614 [ 1199 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 207 ms. 2024.10.22 00:23:59.020520 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ezr/yybvfddfgzesqmbkyzofkzljhjxly (LogSeriesLimiter: on interval from 2024-10-22 00:23:54 to 2024-10-22 00:23:59 accepted series 1 / 271 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:23:59.313750 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:23:59.318731 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:23:59.478818 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 6.96 MiB, will set to 1.50 GiB (RSS), difference: -88.61 MiB 2024.10.22 00:23:59.995312 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 21 entries to flush up to offset 2753 2024.10.22 00:24:00.309419 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:00.317193 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:00.358069 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_56_56_0 (state Temporary) 2024.10.22 00:24:00.485020 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_56_56_0 to 202410_69_69_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:00.653760 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2753 2024.10.22 00:24:00.960640 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2304 entries to flush up to offset 54678 2024.10.22 00:24:01.339968 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 6.64 MiB, will set to 1.51 GiB (RSS), difference: -105.79 MiB 2024.10.22 00:24:01.723080 [ 1218 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:24:01.759533 [ 1218 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:24:01.797198 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.173856118 sec. 2024.10.22 00:24:01.859541 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 43 2024.10.22 00:24:02.040236 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:02.049605 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:02.215149 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:02.223640 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:02.333627 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_31_31_0 (state Temporary) 2024.10.22 00:24:02.402784 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_31_31_0 to 202410_38_38_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:02.432052 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 54678 2024.10.22 00:24:02.470817 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:24:02.547745 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_25_25_0 (state Temporary) 2024.10.22 00:24:02.579453 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 476 2024.10.22 00:24:02.748156 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_25_25_0 to 202410_35_35_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:02.785722 [ 1222 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_30_6 to 202410_35_35_0 2024.10.22 00:24:02.801558 [ 1222 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:02.805512 [ 1222 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:02.828498 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 43 2024.10.22 00:24:02.927969 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:02.932619 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_30_6 to 202410_35_35_0 into Compact with storage Full 2024.10.22 00:24:02.936510 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:02.989405 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:24:02.998787 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_30_6, total 74 rows starting from the beginning of the part 2024.10.22 00:24:03.038987 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_31_31_0, total 1 rows starting from the beginning of the part 2024.10.22 00:24:03.041429 [ 1233 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 221 ms. 2024.10.22 00:24:03.074317 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_32_32_0 (state Temporary) 2024.10.22 00:24:03.075722 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_32_32_0, total 1 rows starting from the beginning of the part 2024.10.22 00:24:03.136373 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_33_33_0, total 1 rows starting from the beginning of the part 2024.10.22 00:24:03.262589 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_34_34_0, total 1 rows starting from the beginning of the part 2024.10.22 00:24:03.281498 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_35_35_0, total 1 rows starting from the beginning of the part 2024.10.22 00:24:03.431118 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_32_32_0 to 202410_45_45_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:03.866492 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 476 2024.10.22 00:24:03.874697 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.81 GiB, free memory in arenas 4.95 MiB, will set to 1.52 GiB (RSS), difference: -5.65 MiB 2024.10.22 00:24:04.659060 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 2795 2024.10.22 00:24:04.780750 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} MergeTask::MergeProjectionsStage: Merge sorted 79 rows, containing 32 columns (32 merged, 0 gathered) in 1.959773464 sec., 40.31078155265807 rows/sec., 5.89 KiB/sec. 2024.10.22 00:24:04.885404 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:04.890709 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:04.970369 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} MergedBlockOutputStream: filled checksums 202410_1_35_7 (state Temporary) 2024.10.22 00:24:05.035577 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_57_57_0 (state Temporary) 2024.10.22 00:24:05.078472 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_57_57_0 to 202410_70_70_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:05.219162 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_35_7 to 202410_1_35_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:05.244769 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2795 2024.10.22 00:24:05.302954 [ 1223 ] {} DatabaseCatalog: Execution took 395 ms. 2024.10.22 00:24:05.329217 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_35_7} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_30_6, 202410_35_35_0] -> 202410_1_35_7 2024.10.22 00:24:05.328356 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.81 GiB, free memory in arenas 5.43 MiB, will set to 1.52 GiB (RSS), difference: 5.53 MiB 2024.10.22 00:24:05.480768 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:24:06.202858 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 29 entries to flush up to offset 3459 2024.10.22 00:24:06.411511 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:06.413805 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:06.664743 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_32_32_0 (state Temporary) 2024.10.22 00:24:06.703469 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_32_32_0 to 202410_38_38_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:06.744174 [ 1190 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 00:24:06.755962 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3459 2024.10.22 00:24:07.500311 [ 1234 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 5 parts from 202410_1_5_1 to 202410_9_9_0 2024.10.22 00:24:07.515067 [ 1234 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:07.518945 [ 1234 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:07.629220 [ 1234 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 257 ms. 2024.10.22 00:24:07.662718 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_9_2} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_5_1 to 202410_9_9_0 into Compact with storage Full 2024.10.22 00:24:07.676097 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 5.57 MiB, will set to 1.53 GiB (RSS), difference: -4.52 MiB 2024.10.22 00:24:09.255122 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 2821 2024.10.22 00:24:09.314810 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:24:09.349226 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:24:09.440238 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 56214 2024.10.22 00:24:09.510445 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:09.519685 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:09.578675 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_58_58_0 (state Temporary) 2024.10.22 00:24:09.646418 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_58_58_0 to 202410_71_71_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:09.656975 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 5.52 MiB, will set to 1.54 GiB (RSS), difference: 3.09 MiB 2024.10.22 00:24:09.676006 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_66_13 to 202410_71_71_0 2024.10.22 00:24:09.694403 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:09.702556 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:09.710804 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2821 2024.10.22 00:24:09.726920 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_66_13 to 202410_71_71_0 into Compact with storage Full 2024.10.22 00:24:09.763167 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:24:09.784923 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_66_13, total 3494 rows starting from the beginning of the part 2024.10.22 00:24:10.006165 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_67_67_0, total 42 rows starting from the beginning of the part 2024.10.22 00:24:10.026587 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_68_68_0, total 27 rows starting from the beginning of the part 2024.10.22 00:24:10.070444 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_69_69_0, total 21 rows starting from the beginning of the part 2024.10.22 00:24:10.083837 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_70_70_0, total 42 rows starting from the beginning of the part 2024.10.22 00:24:10.223159 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_71_71_0, total 26 rows starting from the beginning of the part 2024.10.22 00:24:10.332867 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 44 2024.10.22 00:24:10.372218 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_9_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:24:10.386448 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:10.393715 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:10.492171 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_32_32_0 (state Temporary) 2024.10.22 00:24:10.578815 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_32_32_0 to 202410_39_39_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:10.630428 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 56214 2024.10.22 00:24:10.853009 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:10.863271 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:10.907801 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_5_1, total 106 rows starting from the beginning of the part 2024.10.22 00:24:11.022665 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_26_26_0 (state Temporary) 2024.10.22 00:24:11.109638 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_26_26_0 to 202410_36_36_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:11.314210 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 44 2024.10.22 00:24:11.367892 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 490 2024.10.22 00:24:11.538983 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 4.00 MiB, will set to 1.54 GiB (RSS), difference: 4.43 MiB 2024.10.22 00:24:11.555636 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} MergeTask::MergeProjectionsStage: Merge sorted 3652 rows, containing 13 columns (13 merged, 0 gathered) in 1.838030735 sec., 1986.90910356295 rows/sec., 322.02 KiB/sec. 2024.10.22 00:24:11.633674 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:11.640502 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:11.675807 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_33_33_0 (state Temporary) 2024.10.22 00:24:11.725219 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_33_33_0 to 202410_46_46_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:11.738659 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} MergedBlockOutputStream: filled checksums 202410_1_71_14 (state Temporary) 2024.10.22 00:24:11.784679 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_41_8 to 202410_46_46_0 2024.10.22 00:24:11.788245 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 490 2024.10.22 00:24:11.803894 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:11.807987 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:11.832769 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_41_8 to 202410_46_46_0 into Compact with storage Full 2024.10.22 00:24:11.855070 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:24:11.857720 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_41_8, total 530 rows starting from the beginning of the part 2024.10.22 00:24:11.867257 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_42_42_0, total 14 rows starting from the beginning of the part 2024.10.22 00:24:11.935542 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_71_14 to 202410_1_71_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:11.941455 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_71_14} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_66_13, 202410_71_71_0] -> 202410_1_71_14 2024.10.22 00:24:11.978005 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_43_43_0, total 13 rows starting from the beginning of the part 2024.10.22 00:24:11.995728 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_44_44_0, total 20 rows starting from the beginning of the part 2024.10.22 00:24:12.006691 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:24:12.012838 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_45_45_0, total 5 rows starting from the beginning of the part 2024.10.22 00:24:12.030339 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_46_46_0, total 14 rows starting from the beginning of the part 2024.10.22 00:24:12.476414 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_6_6_0, total 38 rows starting from the beginning of the part 2024.10.22 00:24:12.667442 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} MergeTask::MergeProjectionsStage: Merge sorted 596 rows, containing 12 columns (12 merged, 0 gathered) in 0.847735331 sec., 703.0496172631503 rows/sec., 106.18 KiB/sec. 2024.10.22 00:24:12.690332 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} MergedBlockOutputStream: filled checksums 202410_1_46_9 (state Temporary) 2024.10.22 00:24:12.770190 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_46_9 to 202410_1_46_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:12.776553 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_46_9} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_41_8, 202410_46_46_0] -> 202410_1_46_9 2024.10.22 00:24:12.818097 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:24:13.465420 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 3.63 MiB, will set to 1.55 GiB (RSS), difference: 6.93 MiB 2024.10.22 00:24:13.629780 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:13.640347 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:13.716998 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 2887 2024.10.22 00:24:14.167266 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:14.169371 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:14.260613 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 58 entries to flush up to offset 3517 2024.10.22 00:24:14.310867 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_7_7_0, total 44 rows starting from the beginning of the part 2024.10.22 00:24:14.335284 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_59_59_0 (state Temporary) 2024.10.22 00:24:14.499341 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_59_59_0 to 202410_72_72_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:14.505462 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:14.510687 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:14.548909 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2887 2024.10.22 00:24:14.579338 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_33_33_0 (state Temporary) 2024.10.22 00:24:14.631436 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_33_33_0 to 202410_39_39_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:14.810259 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3517 2024.10.22 00:24:15.553057 [ 1151 ] {} DatabaseCatalog: Execution took 248 ms. 2024.10.22 00:24:15.564210 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 2.78 MiB, will set to 1.55 GiB (RSS), difference: -3.36 MiB 2024.10.22 00:24:15.804998 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_8_8_0, total 41 rows starting from the beginning of the part 2024.10.22 00:24:16.764968 [ 1172 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:24:16.803264 [ 1172 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:24:17.583084 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 2.92 MiB, will set to 1.56 GiB (RSS), difference: 6.91 MiB 2024.10.22 00:24:17.642298 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 57750 2024.10.22 00:24:17.918451 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_7_7_0 (state Temporary) 2024.10.22 00:24:18.372648 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_9_9_0, total 45 rows starting from the beginning of the part 2024.10.22 00:24:18.563748 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 22 entries to flush up to offset 2909 2024.10.22 00:24:18.674362 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:18.683066 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:18.780476 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:18.783461 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:18.939046 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_33_33_0 (state Temporary) 2024.10.22 00:24:19.002986 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_33_33_0 to 202410_40_40_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:19.033134 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_60_60_0 (state Temporary) 2024.10.22 00:24:19.149420 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_60_60_0 to 202410_73_73_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:19.159378 [ 1169 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 00:24:19.159946 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 57750 2024.10.22 00:24:19.166353 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:24:19.256780 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2909 2024.10.22 00:24:19.294466 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 509 2024.10.22 00:24:19.315583 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:24:19.320411 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:24:19.620406 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:19.625038 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:19.665241 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_34_34_0 (state Temporary) 2024.10.22 00:24:19.706582 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_34_34_0 to 202410_47_47_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:19.722388 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 3.03 MiB, will set to 1.57 GiB (RSS), difference: 5.07 MiB 2024.10.22 00:24:19.759277 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 509 2024.10.22 00:24:21.623127 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 2.76 MiB, will set to 1.57 GiB (RSS), difference: 1.49 MiB 2024.10.22 00:24:21.738937 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:24:22.423089 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 42 entries to flush up to offset 3559 2024.10.22 00:24:22.679669 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:22.687082 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:22.858815 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_34_34_0 (state Temporary) 2024.10.22 00:24:22.883892 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_34_34_0 to 202410_40_40_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:22.923344 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3559 2024.10.22 00:24:22.943941 [ 1203 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_35_7 to 202410_40_40_0 2024.10.22 00:24:22.960041 [ 1203 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:22.962396 [ 1203 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:23.088731 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_35_7 to 202410_40_40_0 into Compact with storage Full 2024.10.22 00:24:23.149271 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:24:23.258474 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_35_7, total 3448 rows starting from the beginning of the part 2024.10.22 00:24:23.261457 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 2945 2024.10.22 00:24:23.268561 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_36_36_0, total 65 rows starting from the beginning of the part 2024.10.22 00:24:23.280923 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_37_37_0, total 21 rows starting from the beginning of the part 2024.10.22 00:24:23.296611 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_38_38_0, total 29 rows starting from the beginning of the part 2024.10.22 00:24:23.318459 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_39_39_0, total 58 rows starting from the beginning of the part 2024.10.22 00:24:23.351667 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_40_40_0, total 42 rows starting from the beginning of the part 2024.10.22 00:24:23.404626 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:23.408780 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.75 GiB. 2024.10.22 00:24:23.495859 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 3.12 MiB, will set to 1.58 GiB (RSS), difference: -2.62 MiB 2024.10.22 00:24:23.554632 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_61_61_0 (state Temporary) 2024.10.22 00:24:23.811629 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_61_61_0 to 202410_74_74_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:23.962053 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2945 2024.10.22 00:24:24.441346 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 3.14 MiB, will set to 1.58 GiB (RSS), difference: -3.55 MiB 2024.10.22 00:24:24.482769 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} MergeTask::MergeProjectionsStage: Merge sorted 3663 rows, containing 13 columns (13 merged, 0 gathered) in 1.401499154 sec., 2613.629833129389 rows/sec., 1.11 MiB/sec. 2024.10.22 00:24:24.653800 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} MergedBlockOutputStream: filled checksums 202410_1_40_8 (state Temporary) 2024.10.22 00:24:24.855646 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_40_8 to 202410_1_40_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:24.860382 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_40_8} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_35_7, 202410_40_40_0] -> 202410_1_40_8 2024.10.22 00:24:24.898482 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.74 MiB. 2024.10.22 00:24:25.723694 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 2.87 MiB, will set to 1.58 GiB (RSS), difference: -3.47 MiB 2024.10.22 00:24:26.164279 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2027 entries to flush up to offset 59777 2024.10.22 00:24:26.317048 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 45 2024.10.22 00:24:26.669607 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:26.672158 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.74 GiB. 2024.10.22 00:24:26.945234 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_27_27_0 (state Temporary) 2024.10.22 00:24:27.020973 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_7_7_0 to 202410_10_10_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:27.024616 [ 1683 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ezr/yybvfddfgzesqmbkyzofkzljhjxly, size 24026 2024.10.22 00:24:27.261025 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 520 2024.10.22 00:24:27.263680 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:27.271008 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.73 GiB. 2024.10.22 00:24:27.279517 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_27_27_0 to 202410_37_37_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:27.380159 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:27.381588 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.73 GiB. 2024.10.22 00:24:27.414361 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_35_35_0 (state Temporary) 2024.10.22 00:24:27.529508 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_34_34_0 (state Temporary) 2024.10.22 00:24:27.531466 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 45 2024.10.22 00:24:27.641069 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.81 GiB, free memory in arenas 3.05 MiB, will set to 1.57 GiB (RSS), difference: 111.45 MiB 2024.10.22 00:24:27.675948 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_34_34_0 to 202410_41_41_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:27.691539 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_35_35_0 to 202410_48_48_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:27.712809 [ 1192 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_36_7 to 202410_41_41_0 2024.10.22 00:24:27.720747 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 59777 2024.10.22 00:24:27.730191 [ 1192 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:27.733414 [ 1192 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.73 GiB. 2024.10.22 00:24:27.751968 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 520 2024.10.22 00:24:27.852793 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_36_7 to 202410_41_41_0 into Compact with storage Full 2024.10.22 00:24:27.876254 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:24:27.880286 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_36_7, total 64044 rows starting from the beginning of the part 2024.10.22 00:24:27.887681 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_37_37_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:24:27.906671 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_38_38_0, total 2304 rows starting from the beginning of the part 2024.10.22 00:24:27.914740 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_39_39_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:24:27.933331 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_40_40_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:24:27.943809 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_41_41_0, total 2027 rows starting from the beginning of the part 2024.10.22 00:24:27.964380 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 2996 2024.10.22 00:24:28.239991 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:28.243926 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.73 GiB. 2024.10.22 00:24:28.524111 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_62_62_0 (state Temporary) 2024.10.22 00:24:28.572568 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_62_62_0 to 202410_75_75_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:28.754439 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 2996 2024.10.22 00:24:29.319590 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:24:29.327368 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:24:29.888695 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 1.81 GiB, free memory in arenas 5.82 MiB, will set to 1.56 GiB (RSS), difference: 185.97 MiB 2024.10.22 00:24:30.295401 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_3_3_0 (state Temporary) 2024.10.22 00:24:30.324192 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nuo/ayzmhzhrerdinzqpjlarwqitqvhnu (LogSeriesLimiter: on interval from 2024-10-22 00:23:59 to 2024-10-22 00:24:30 accepted series 1 / 90 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:24:30.328615 [ 1693 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ntb/bxjaxbtvedbkcfcvsqexgzlnsmebt, size 549 2024.10.22 00:24:30.452664 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 107 entries to flush up to offset 3666 2024.10.22 00:24:30.480747 [ 1711 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nuo/ayzmhzhrerdinzqpjlarwqitqvhnu, size 4 2024.10.22 00:24:31.077772 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:31.083601 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:31.140595 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_35_35_0 (state Temporary) 2024.10.22 00:24:31.177184 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_35_35_0 to 202410_41_41_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:31.222699 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3666 2024.10.22 00:24:31.391225 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 254 2024.10.22 00:24:31.738533 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.81 GiB, free memory in arenas 9.54 MiB, will set to 1.53 GiB (RSS), difference: 52.46 MiB 2024.10.22 00:24:31.808718 [ 1250 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:24:31.860470 [ 1250 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:24:31.954416 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} MergeTask::MergeProjectionsStage: Merge sorted 73367 rows, containing 4 columns (4 merged, 0 gathered) in 4.211657977 sec., 17419.980539887038 rows/sec., 257.34 KiB/sec. 2024.10.22 00:24:32.136839 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} MergedBlockOutputStream: filled checksums 202410_1_41_8 (state Temporary) 2024.10.22 00:24:32.368433 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_41_8 to 202410_1_41_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:32.370855 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_36_7, 202410_41_41_0] -> 202410_1_41_8 2024.10.22 00:24:32.527992 [ 1391 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_41_8}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 00:24:32.659839 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:24:32.877760 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 29 entries to flush up to offset 3025 2024.10.22 00:24:33.198938 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:33.206959 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:33.348333 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_63_63_0 (state Temporary) 2024.10.22 00:24:33.475531 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 10.11 MiB, will set to 1.53 GiB (RSS), difference: 4.78 MiB 2024.10.22 00:24:33.476005 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_63_63_0 to 202410_76_76_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:33.512891 [ 1237 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_71_14 to 202410_76_76_0 2024.10.22 00:24:33.525622 [ 1237 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:33.536122 [ 1237 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:33.561758 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_71_14 to 202410_76_76_0 into Compact with storage Full 2024.10.22 00:24:33.582226 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:24:33.590519 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_71_14, total 3652 rows starting from the beginning of the part 2024.10.22 00:24:33.607954 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_72_72_0, total 66 rows starting from the beginning of the part 2024.10.22 00:24:33.629352 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3025 2024.10.22 00:24:33.729275 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_73_73_0, total 22 rows starting from the beginning of the part 2024.10.22 00:24:33.751799 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_74_74_0, total 36 rows starting from the beginning of the part 2024.10.22 00:24:33.760646 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_75_75_0, total 51 rows starting from the beginning of the part 2024.10.22 00:24:33.768607 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_76_76_0, total 29 rows starting from the beginning of the part 2024.10.22 00:24:34.052871 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_9_2/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:24:34.557362 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 10.78 MiB, will set to 1.53 GiB (RSS), difference: 1.85 MiB 2024.10.22 00:24:34.836649 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1929 entries to flush up to offset 61706 2024.10.22 00:24:34.878832 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} MergeTask::MergeProjectionsStage: Merge sorted 3856 rows, containing 13 columns (13 merged, 0 gathered) in 1.324382491 sec., 2911.5455891360016 rows/sec., 477.02 KiB/sec. 2024.10.22 00:24:34.907299 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/liw/xawtepouskgsykdstujwboanzwmdw. 2024.10.22 00:24:35.035164 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 46 2024.10.22 00:24:35.152046 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} MergedBlockOutputStream: filled checksums 202410_1_76_15 (state Temporary) 2024.10.22 00:24:35.253980 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 533 2024.10.22 00:24:35.317732 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_76_15 to 202410_1_76_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:35.321197 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_76_15} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_71_14, 202410_76_76_0] -> 202410_1_76_15 2024.10.22 00:24:35.374511 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:24:35.448386 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:35.451357 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:35.475122 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_36_36_0 (state Temporary) 2024.10.22 00:24:35.525485 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_36_36_0 to 202410_49_49_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:35.572135 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:35.574555 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:35.590035 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 533 2024.10.22 00:24:35.623099 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:35.634818 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:35.693107 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_28_28_0 (state Temporary) 2024.10.22 00:24:35.800397 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_35_35_0 (state Temporary) 2024.10.22 00:24:35.810324 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_28_28_0 to 202410_38_38_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:35.897445 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 46 2024.10.22 00:24:35.994724 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_35_35_0 to 202410_42_42_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:36.128925 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 61706 2024.10.22 00:24:36.420569 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 9.02 MiB, will set to 1.53 GiB (RSS), difference: 2.79 MiB 2024.10.22 00:24:37.299741 [ 1155 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 00:24:37.637540 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 3071 2024.10.22 00:24:37.856481 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:37.862279 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:37.932755 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_64_64_0 (state Temporary) 2024.10.22 00:24:37.978508 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_64_64_0 to 202410_77_77_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:38.014246 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3071 2024.10.22 00:24:38.754451 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 3716 2024.10.22 00:24:38.923212 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 297 2024.10.22 00:24:38.960094 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:38.961609 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:39.012774 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_36_36_0 (state Temporary) 2024.10.22 00:24:39.063628 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_36_36_0 to 202410_42_42_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:39.167910 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3716 2024.10.22 00:24:39.318511 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:24:39.323424 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:24:42.024404 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 15 entries to flush up to offset 3086 2024.10.22 00:24:42.324464 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:42.331013 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:42.373647 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_65_65_0 (state Temporary) 2024.10.22 00:24:42.533118 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_65_65_0 to 202410_78_78_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:42.563805 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 8.16 MiB, will set to 1.53 GiB (RSS), difference: 1.28 MiB 2024.10.22 00:24:42.574124 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3086 2024.10.22 00:24:43.097289 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 546 2024.10.22 00:24:43.137343 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2190 entries to flush up to offset 63896 2024.10.22 00:24:43.402139 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 47 2024.10.22 00:24:43.524182 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:43.527985 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:43.563190 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_37_37_0 (state Temporary) 2024.10.22 00:24:43.577092 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:43.578518 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:43.610682 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_37_37_0 to 202410_50_50_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:43.632471 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:43.661837 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 546 2024.10.22 00:24:43.738763 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:43.836647 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_36_36_0 (state Temporary) 2024.10.22 00:24:43.855816 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_29_29_0 (state Temporary) 2024.10.22 00:24:43.890329 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_36_36_0 to 202410_43_43_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:43.918652 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 63896 2024.10.22 00:24:44.130842 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_29_29_0 to 202410_39_39_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:44.248043 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 47 2024.10.22 00:24:44.488619 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 8.97 MiB, will set to 1.54 GiB (RSS), difference: 1.77 MiB 2024.10.22 00:24:46.576393 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 3112 2024.10.22 00:24:46.772475 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 30 entries to flush up to offset 3746 2024.10.22 00:24:46.865132 [ 1210 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:24:46.904130 [ 1210 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:24:46.915347 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:46.922521 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:46.979285 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_66_66_0 (state Temporary) 2024.10.22 00:24:47.131899 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_66_66_0 to 202410_79_79_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:47.194188 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3112 2024.10.22 00:24:47.223782 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:47.227727 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:47.273290 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_37_37_0 (state Temporary) 2024.10.22 00:24:47.313088 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_37_37_0 to 202410_43_43_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:47.470207 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3746 2024.10.22 00:24:48.476985 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 8.75 MiB, will set to 1.54 GiB (RSS), difference: 3.30 MiB 2024.10.22 00:24:49.320622 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:24:49.323979 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:24:49.776369 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:24:50.230311 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_9_2/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:24:50.927755 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 65048 2024.10.22 00:24:51.164553 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 551 2024.10.22 00:24:51.204152 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 21 entries to flush up to offset 3133 2024.10.22 00:24:51.318892 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:51.336543 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:51.375931 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_67_67_0 (state Temporary) 2024.10.22 00:24:51.478610 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:51.480785 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:51.505987 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_67_67_0 to 202410_80_80_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:51.519912 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_38_38_0 (state Temporary) 2024.10.22 00:24:51.562718 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3133 2024.10.22 00:24:51.569788 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_38_38_0 to 202410_51_51_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:51.657647 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 551 2024.10.22 00:24:51.670612 [ 1182 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_46_9 to 202410_51_51_0 2024.10.22 00:24:51.683510 [ 1182 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:51.687089 [ 1182 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:51.719590 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_46_9 to 202410_51_51_0 into Compact with storage Full 2024.10.22 00:24:51.751265 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 48 2024.10.22 00:24:51.768893 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:24:51.773673 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_46_9, total 596 rows starting from the beginning of the part 2024.10.22 00:24:51.787974 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_47_47_0, total 19 rows starting from the beginning of the part 2024.10.22 00:24:51.995937 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_48_48_0, total 11 rows starting from the beginning of the part 2024.10.22 00:24:52.009821 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_49_49_0, total 13 rows starting from the beginning of the part 2024.10.22 00:24:52.023374 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_50_50_0, total 13 rows starting from the beginning of the part 2024.10.22 00:24:52.053185 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_51_51_0, total 5 rows starting from the beginning of the part 2024.10.22 00:24:52.168422 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:52.171341 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:52.505286 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:52.508551 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:52.532501 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_30_30_0 (state Temporary) 2024.10.22 00:24:52.586494 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 7.57 MiB, will set to 1.53 GiB (RSS), difference: -6.68 MiB 2024.10.22 00:24:52.608187 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_30_30_0 to 202410_40_40_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:52.667682 [ 1215 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_35_7 to 202410_40_40_0 2024.10.22 00:24:52.679688 [ 1215 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:52.683096 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:52.705391 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_35_7 to 202410_40_40_0 into Compact with storage Full 2024.10.22 00:24:52.836389 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_37_37_0 (state Temporary) 2024.10.22 00:24:52.868220 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:24:52.874275 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_35_7, total 79 rows starting from the beginning of the part 2024.10.22 00:24:52.904148 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_36_36_0, total 1 rows starting from the beginning of the part 2024.10.22 00:24:52.907750 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 48 2024.10.22 00:24:52.958531 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_37_37_0, total 1 rows starting from the beginning of the part 2024.10.22 00:24:52.966746 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} MergeTask::MergeProjectionsStage: Merge sorted 657 rows, containing 12 columns (12 merged, 0 gathered) in 1.255279316 sec., 523.3894891963631 rows/sec., 79.21 KiB/sec. 2024.10.22 00:24:53.007491 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} MergedBlockOutputStream: filled checksums 202410_1_51_10 (state Temporary) 2024.10.22 00:24:53.024750 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_37_37_0 to 202410_44_44_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:53.076381 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_51_10 to 202410_1_51_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:53.076392 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_38_38_0, total 1 rows starting from the beginning of the part 2024.10.22 00:24:53.077529 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 65048 2024.10.22 00:24:53.078792 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_51_10} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_46_9, 202410_51_51_0] -> 202410_1_51_10 2024.10.22 00:24:53.106967 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_39_39_0, total 1 rows starting from the beginning of the part 2024.10.22 00:24:53.113310 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:24:53.133813 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_40_40_0, total 1 rows starting from the beginning of the part 2024.10.22 00:24:54.123020 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:54.131259 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:54.325720 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 4.71 MiB, will set to 1.53 GiB (RSS), difference: 1.19 MiB 2024.10.22 00:24:54.967586 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} MergeTask::MergeProjectionsStage: Merge sorted 84 rows, containing 32 columns (32 merged, 0 gathered) in 2.270050225 sec., 37.00358656161451 rows/sec., 5.35 KiB/sec. 2024.10.22 00:24:54.982184 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 3796 2024.10.22 00:24:55.052594 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} MergedBlockOutputStream: filled checksums 202410_1_40_8 (state Temporary) 2024.10.22 00:24:55.274567 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_40_8 to 202410_1_40_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:55.279263 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_40_8} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_35_7, 202410_40_40_0] -> 202410_1_40_8 2024.10.22 00:24:55.302598 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:24:55.404551 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:55.409245 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:55.567433 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 3194 2024.10.22 00:24:55.575021 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_38_38_0 (state Temporary) 2024.10.22 00:24:55.718640 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_38_38_0 to 202410_44_44_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:55.880075 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3796 2024.10.22 00:24:55.912110 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:55.916100 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:55.967069 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_68_68_0 (state Temporary) 2024.10.22 00:24:55.998967 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_68_68_0 to 202410_81_81_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:56.064573 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_76_15 to 202410_81_81_0 2024.10.22 00:24:56.074577 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:56.076297 [ 1249 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:56.205751 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_76_15 to 202410_81_81_0 into Compact with storage Full 2024.10.22 00:24:56.243876 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:24:56.247800 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3194 2024.10.22 00:24:56.253144 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_76_15, total 3856 rows starting from the beginning of the part 2024.10.22 00:24:56.264459 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_77_77_0, total 46 rows starting from the beginning of the part 2024.10.22 00:24:56.373179 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_78_78_0, total 15 rows starting from the beginning of the part 2024.10.22 00:24:56.386582 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_79_79_0, total 26 rows starting from the beginning of the part 2024.10.22 00:24:56.408988 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_80_80_0, total 21 rows starting from the beginning of the part 2024.10.22 00:24:56.418624 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_81_81_0, total 61 rows starting from the beginning of the part 2024.10.22 00:24:56.754561 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 5.80 MiB, will set to 1.53 GiB (RSS), difference: -5.52 MiB 2024.10.22 00:24:57.465561 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} MergeTask::MergeProjectionsStage: Merge sorted 4025 rows, containing 13 columns (13 merged, 0 gathered) in 1.264883085 sec., 3182.112281942643 rows/sec., 517.92 KiB/sec. 2024.10.22 00:24:57.524304 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} MergedBlockOutputStream: filled checksums 202410_1_81_16 (state Temporary) 2024.10.22 00:24:57.787579 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_81_16 to 202410_1_81_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:57.793209 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_81_16} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_76_15, 202410_81_81_0] -> 202410_1_81_16 2024.10.22 00:24:57.841666 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:24:58.151467 [ 1245 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 246 ms. 2024.10.22 00:24:58.235069 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_8_8_0 (state Temporary) 2024.10.22 00:24:58.559802 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 5.41 MiB, will set to 1.53 GiB (RSS), difference: 1.37 MiB 2024.10.22 00:24:59.160598 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 578 2024.10.22 00:24:59.322482 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:24:59.334137 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:24:59.378952 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:24:59.383296 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:24:59.445503 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_39_39_0 (state Temporary) 2024.10.22 00:24:59.538864 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_39_39_0 to 202410_52_52_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:24:59.569810 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 578 2024.10.22 00:24:59.622037 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 5.29 MiB, will set to 1.53 GiB (RSS), difference: -1.16 MiB 2024.10.22 00:25:00.089739 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2244 entries to flush up to offset 67292 2024.10.22 00:25:00.262888 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 3233 2024.10.22 00:25:00.383320 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:00.386926 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:25:00.411281 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 49 2024.10.22 00:25:00.552611 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_69_69_0 (state Temporary) 2024.10.22 00:25:00.606909 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_69_69_0 to 202410_82_82_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:00.668868 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3233 2024.10.22 00:25:00.678577 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:00.680525 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:25:00.768358 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:00.771012 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:25:00.846516 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_31_31_0 (state Temporary) 2024.10.22 00:25:00.888763 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_38_38_0 (state Temporary) 2024.10.22 00:25:00.890914 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_31_31_0 to 202410_41_41_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:00.966676 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_38_38_0 to 202410_45_45_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:01.030926 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 67292 2024.10.22 00:25:01.104179 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 49 2024.10.22 00:25:01.453271 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 00:25:01.910365 [ 1156 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:25:01.940032 [ 1156 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:25:03.415957 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 49 entries to flush up to offset 3845 2024.10.22 00:25:03.463858 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/liw/xawtepouskgsykdstujwboanzwmdw. 2024.10.22 00:25:03.509123 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:03.513327 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:25:03.551094 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_39_39_0 (state Temporary) 2024.10.22 00:25:03.579838 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.81 GiB, free memory in arenas 5.86 MiB, will set to 1.53 GiB (RSS), difference: 34.03 MiB 2024.10.22 00:25:03.680003 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_39_39_0 to 202410_45_45_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:03.744630 [ 1167 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_40_8 to 202410_45_45_0 2024.10.22 00:25:03.755590 [ 1167 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:03.757190 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:25:03.769422 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_40_8 to 202410_45_45_0 into Compact with storage Full 2024.10.22 00:25:03.787398 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:25:03.798290 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_40_8, total 3663 rows starting from the beginning of the part 2024.10.22 00:25:03.838753 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_41_41_0, total 107 rows starting from the beginning of the part 2024.10.22 00:25:03.958968 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_42_42_0, total 50 rows starting from the beginning of the part 2024.10.22 00:25:03.968457 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_43_43_0, total 30 rows starting from the beginning of the part 2024.10.22 00:25:03.982871 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_44_44_0, total 50 rows starting from the beginning of the part 2024.10.22 00:25:04.005521 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_45_45_0, total 49 rows starting from the beginning of the part 2024.10.22 00:25:04.025711 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3845 2024.10.22 00:25:04.550852 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_9_2} MergeTask::MergeProjectionsStage: Merge sorted 274 rows, containing 679 columns (679 merged, 0 gathered) in 56.898049297 sec., 4.815630823646654 rows/sec., 5.73 KiB/sec. 2024.10.22 00:25:04.679324 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 3273 2024.10.22 00:25:04.828234 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:04.831627 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:25:04.873122 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_70_70_0 (state Temporary) 2024.10.22 00:25:04.877079 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} MergeTask::MergeProjectionsStage: Merge sorted 3949 rows, containing 13 columns (13 merged, 0 gathered) in 1.112937682 sec., 3548.266954986613 rows/sec., 1.50 MiB/sec. 2024.10.22 00:25:04.946828 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} MergedBlockOutputStream: filled checksums 202410_1_45_9 (state Temporary) 2024.10.22 00:25:05.144077 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_70_70_0 to 202410_83_83_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:05.217386 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_45_9 to 202410_1_45_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:05.284463 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3273 2024.10.22 00:25:05.300114 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.81 GiB, free memory in arenas 7.98 MiB, will set to 1.52 GiB (RSS), difference: 97.55 MiB 2024.10.22 00:25:05.324160 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_45_9} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_40_8, 202410_45_45_0] -> 202410_1_45_9 2024.10.22 00:25:05.364093 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.74 MiB. 2024.10.22 00:25:05.981063 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_9_2} MergedBlockOutputStream: filled checksums 202410_1_9_2 (state Temporary) 2024.10.22 00:25:07.060367 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_8_8_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:25:07.071880 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 591 2024.10.22 00:25:07.357026 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:07.360985 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:25:07.400070 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_40_40_0 (state Temporary) 2024.10.22 00:25:07.559197 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_40_40_0 to 202410_53_53_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:07.577121 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.81 GiB, free memory in arenas 10.82 MiB, will set to 1.52 GiB (RSS), difference: 53.75 MiB 2024.10.22 00:25:07.627082 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_8_8_0 to 202410_11_11_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:07.709538 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 591 2024.10.22 00:25:08.097636 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1308 entries to flush up to offset 68600 2024.10.22 00:25:08.827857 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:08.837419 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:25:09.062909 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_insert_202410_39_39_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:25:09.078220 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_39_39_0 (state Temporary) 2024.10.22 00:25:09.162812 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_39_39_0 to 202410_46_46_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:09.283881 [ 1206 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_41_8 to 202410_46_46_0 2024.10.22 00:25:09.294655 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 68600 2024.10.22 00:25:09.300722 [ 1206 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:09.304193 [ 1206 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.68 GiB. 2024.10.22 00:25:09.324963 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:25:09.327067 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 3305 2024.10.22 00:25:09.340191 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:25:09.345817 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_41_8 to 202410_46_46_0 into Compact with storage Full 2024.10.22 00:25:09.365563 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:25:09.375014 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_41_8, total 73367 rows starting from the beginning of the part 2024.10.22 00:25:09.486615 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_42_42_0, total 1929 rows starting from the beginning of the part 2024.10.22 00:25:09.500718 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_43_43_0, total 2190 rows starting from the beginning of the part 2024.10.22 00:25:09.512965 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_44_44_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:25:09.526230 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_45_45_0, total 2244 rows starting from the beginning of the part 2024.10.22 00:25:09.560045 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_46_46_0, total 1308 rows starting from the beginning of the part 2024.10.22 00:25:09.732795 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:09.746987 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.67 GiB. 2024.10.22 00:25:09.880124 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_71_71_0 (state Temporary) 2024.10.22 00:25:10.027558 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_71_71_0 to 202410_84_84_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:10.095473 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3305 2024.10.22 00:25:10.856958 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 297 2024.10.22 00:25:11.533270 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 61 entries to flush up to offset 3906 2024.10.22 00:25:11.617328 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.81 GiB, free memory in arenas 16.07 MiB, will set to 1.52 GiB (RSS), difference: 7.10 MiB 2024.10.22 00:25:12.056188 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:12.058512 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.67 GiB. 2024.10.22 00:25:12.227690 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_40_40_0 (state Temporary) 2024.10.22 00:25:12.470446 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_40_40_0 to 202410_46_46_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:12.534742 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3906 2024.10.22 00:25:12.721592 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} MergeTask::MergeProjectionsStage: Merge sorted 82190 rows, containing 4 columns (4 merged, 0 gathered) in 3.408889434 sec., 24110.491581288406 rows/sec., 355.71 KiB/sec. 2024.10.22 00:25:12.766101 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} MergedBlockOutputStream: filled checksums 202410_1_46_9 (state Temporary) 2024.10.22 00:25:12.787357 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_46_9 to 202410_1_46_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:12.792125 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_46_9} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_41_8, 202410_46_46_0] -> 202410_1_46_9 2024.10.22 00:25:12.943655 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:25:13.342176 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.81 GiB, free memory in arenas 20.45 MiB, will set to 1.52 GiB (RSS), difference: 9.16 MiB 2024.10.22 00:25:13.771518 [ 1183 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 00:25:14.110974 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 3336 2024.10.22 00:25:14.328081 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:14.333560 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.67 GiB. 2024.10.22 00:25:14.474481 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_72_72_0 (state Temporary) 2024.10.22 00:25:14.523380 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_72_72_0 to 202410_85_85_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:14.579700 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3336 2024.10.22 00:25:15.131628 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_9_2} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_9_2 to 202410_1_9_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:15.140339 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_9_2} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 5 parts: [202410_1_5_1, 202410_9_9_0] -> 202410_1_9_2 2024.10.22 00:25:15.211470 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 610 2024.10.22 00:25:15.396387 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:15.400730 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.67 GiB. 2024.10.22 00:25:15.435581 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_41_41_0 (state Temporary) 2024.10.22 00:25:15.475557 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.81 GiB, free memory in arenas 16.73 MiB, will set to 1.52 GiB (RSS), difference: -3.42 MiB 2024.10.22 00:25:15.492261 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_41_41_0 to 202410_54_54_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:15.656650 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 610 2024.10.22 00:25:16.008168 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.76 MiB. 2024.10.22 00:25:16.107508 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 50 2024.10.22 00:25:16.302744 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1824 entries to flush up to offset 70424 2024.10.22 00:25:16.378218 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.81 GiB, free memory in arenas 13.01 MiB, will set to 1.52 GiB (RSS), difference: 8.79 MiB 2024.10.22 00:25:16.465443 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:16.466982 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.67 GiB. 2024.10.22 00:25:16.532362 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_32_32_0 (state Temporary) 2024.10.22 00:25:16.708566 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_32_32_0 to 202410_42_42_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:16.803806 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 50 2024.10.22 00:25:16.944855 [ 1231 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:25:16.970709 [ 1231 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:25:17.038963 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:17.047689 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.67 GiB. 2024.10.22 00:25:17.173657 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_40_40_0 (state Temporary) 2024.10.22 00:25:17.271786 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_40_40_0 to 202410_47_47_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:17.340803 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 70424 2024.10.22 00:25:18.479618 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 337 2024.10.22 00:25:18.584658 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 3368 2024.10.22 00:25:18.616536 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.81 GiB, free memory in arenas 11.35 MiB, will set to 1.52 GiB (RSS), difference: -2.49 MiB 2024.10.22 00:25:18.773417 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:18.777758 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.66 GiB. 2024.10.22 00:25:19.034256 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_73_73_0 (state Temporary) 2024.10.22 00:25:19.072916 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_73_73_0 to 202410_86_86_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:19.100073 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_81_16 to 202410_86_86_0 2024.10.22 00:25:19.120489 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:19.123254 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3368 2024.10.22 00:25:19.123328 [ 1249 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.66 GiB. 2024.10.22 00:25:19.183704 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_81_16 to 202410_86_86_0 into Compact with storage Full 2024.10.22 00:25:19.234427 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:25:19.240215 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_81_16, total 4025 rows starting from the beginning of the part 2024.10.22 00:25:19.258007 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_82_82_0, total 39 rows starting from the beginning of the part 2024.10.22 00:25:19.271889 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_83_83_0, total 40 rows starting from the beginning of the part 2024.10.22 00:25:19.284268 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_84_84_0, total 32 rows starting from the beginning of the part 2024.10.22 00:25:19.310655 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_85_85_0, total 31 rows starting from the beginning of the part 2024.10.22 00:25:19.328881 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:25:19.333485 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:25:19.426775 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_86_86_0, total 32 rows starting from the beginning of the part 2024.10.22 00:25:20.042143 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 34 entries to flush up to offset 3940 2024.10.22 00:25:20.267420 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:20.270644 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.66 GiB. 2024.10.22 00:25:20.307881 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_41_41_0 (state Temporary) 2024.10.22 00:25:20.345414 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.81 GiB, free memory in arenas 10.48 MiB, will set to 1.51 GiB (RSS), difference: -9.97 MiB 2024.10.22 00:25:20.448315 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_41_41_0 to 202410_47_47_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:20.490087 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3940 2024.10.22 00:25:20.960124 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} MergeTask::MergeProjectionsStage: Merge sorted 4199 rows, containing 13 columns (13 merged, 0 gathered) in 1.832675113 sec., 2291.1862392928124 rows/sec., 371.09 KiB/sec. 2024.10.22 00:25:21.136589 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} MergedBlockOutputStream: filled checksums 202410_1_86_17 (state Temporary) 2024.10.22 00:25:21.171904 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_86_17 to 202410_1_86_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:21.175549 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_86_17} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_81_16, 202410_86_86_0] -> 202410_1_86_17 2024.10.22 00:25:21.205373 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:25:22.577770 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.81 GiB, free memory in arenas 9.12 MiB, will set to 1.51 GiB (RSS), difference: 8.51 MiB 2024.10.22 00:25:23.133677 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 3401 2024.10.22 00:25:23.159541 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 622 2024.10.22 00:25:23.241212 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:23.245447 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.66 GiB. 2024.10.22 00:25:23.257742 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:23.353811 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.66 GiB. 2024.10.22 00:25:23.390366 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_42_42_0 (state Temporary) 2024.10.22 00:25:23.480782 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_74_74_0 (state Temporary) 2024.10.22 00:25:23.538207 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_74_74_0 to 202410_87_87_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:23.547893 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_42_42_0 to 202410_55_55_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:23.587921 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 622 2024.10.22 00:25:23.587975 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3401 2024.10.22 00:25:24.307349 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 51 2024.10.22 00:25:24.350706 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 71960 2024.10.22 00:25:24.483890 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:24.492113 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.66 GiB. 2024.10.22 00:25:24.518550 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.81 GiB, free memory in arenas 9.03 MiB, will set to 1.51 GiB (RSS), difference: -3.00 MiB 2024.10.22 00:25:24.584205 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_33_33_0 (state Temporary) 2024.10.22 00:25:24.802780 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_33_33_0 to 202410_43_43_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:24.971784 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 51 2024.10.22 00:25:25.290782 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:25.297018 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.66 GiB. 2024.10.22 00:25:25.371379 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_41_41_0 (state Temporary) 2024.10.22 00:25:25.552035 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_41_41_0 to 202410_48_48_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:25.574333 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 71960 2024.10.22 00:25:26.664497 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.81 GiB, free memory in arenas 6.87 MiB, will set to 1.51 GiB (RSS), difference: -2.09 MiB 2024.10.22 00:25:26.841041 [ 1166 ] {} DatabaseCatalog: Execution took 497 ms. 2024.10.22 00:25:27.593761 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 3428 2024.10.22 00:25:27.909063 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:27.911911 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:27.965343 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_75_75_0 (state Temporary) 2024.10.22 00:25:28.002141 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 22 entries to flush up to offset 3962 2024.10.22 00:25:28.092900 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_75_75_0 to 202410_88_88_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:28.115864 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:28.119871 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:28.175289 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3428 2024.10.22 00:25:28.182544 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_42_42_0 (state Temporary) 2024.10.22 00:25:28.342887 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.81 GiB, free memory in arenas 6.95 MiB, will set to 1.51 GiB (RSS), difference: -5.96 MiB 2024.10.22 00:25:28.426550 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_42_42_0 to 202410_48_48_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:28.469537 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3962 2024.10.22 00:25:29.260551 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yiw/qddafdpisykehdkzqaqhgxvkuetjx. 2024.10.22 00:25:29.284048 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/asw/efaccsffiihminvbjsgpicpynzjpx. 2024.10.22 00:25:29.326261 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:25:29.332368 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:25:29.828479 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/asw/efaccsffiihminvbjsgpicpynzjpx. 2024.10.22 00:25:29.836983 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_2_2_0 to 201403_2_2_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:29.864614 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yiw/qddafdpisykehdkzqaqhgxvkuetjx. 2024.10.22 00:25:29.880697 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:25:30.452857 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.07 GiB, peak 1.81 GiB, free memory in arenas 6.89 MiB, will set to 1.47 GiB (RSS), difference: 413.85 MiB 2024.10.22 00:25:31.091988 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 629 2024.10.22 00:25:31.192965 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:31.198400 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:31.236933 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_43_43_0 (state Temporary) 2024.10.22 00:25:31.282616 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_43_43_0 to 202410_56_56_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:31.328519 [ 1152 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_51_10 to 202410_56_56_0 2024.10.22 00:25:31.332634 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 629 2024.10.22 00:25:31.359630 [ 1152 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:31.362779 [ 1152 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:31.382208 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_51_10 to 202410_56_56_0 into Compact with storage Full 2024.10.22 00:25:31.431705 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:25:31.439615 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_51_10, total 657 rows starting from the beginning of the part 2024.10.22 00:25:31.454904 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_52_52_0, total 27 rows starting from the beginning of the part 2024.10.22 00:25:31.467158 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_53_53_0, total 13 rows starting from the beginning of the part 2024.10.22 00:25:31.479553 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_54_54_0, total 19 rows starting from the beginning of the part 2024.10.22 00:25:31.511073 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_55_55_0, total 12 rows starting from the beginning of the part 2024.10.22 00:25:31.539102 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_56_56_0, total 7 rows starting from the beginning of the part 2024.10.22 00:25:31.977013 [ 1157 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:25:32.010742 [ 1157 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:25:32.178373 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 3469 2024.10.22 00:25:32.198414 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 2 2024.10.22 00:25:32.426090 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} MergeTask::MergeProjectionsStage: Merge sorted 735 rows, containing 12 columns (12 merged, 0 gathered) in 1.047844946 sec., 701.4396574662678 rows/sec., 106.55 KiB/sec. 2024.10.22 00:25:32.428905 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:32.432898 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:32.473277 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 52 2024.10.22 00:25:32.504071 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_2_2_0 (state Temporary) 2024.10.22 00:25:32.564423 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.81 GiB, free memory in arenas 7.16 MiB, will set to 1.08 GiB (RSS), difference: -80.29 MiB 2024.10.22 00:25:32.576972 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 73496 2024.10.22 00:25:32.661074 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:32.664892 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:32.664997 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} MergedBlockOutputStream: filled checksums 202410_1_56_11 (state Temporary) 2024.10.22 00:25:32.682867 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_2_2_0 to 202410_2_2_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:32.715332 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_76_76_0 (state Temporary) 2024.10.22 00:25:32.766429 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_76_76_0 to 202410_89_89_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:32.856104 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_56_11 to 202410_1_56_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:32.858622 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_56_11} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_51_10, 202410_56_56_0] -> 202410_1_56_11 2024.10.22 00:25:32.865208 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:32.867372 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:32.873119 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 2 2024.10.22 00:25:32.885374 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:25:32.940285 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3469 2024.10.22 00:25:33.004153 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:33.007003 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:33.041093 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_34_34_0 (state Temporary) 2024.10.22 00:25:33.109825 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_34_34_0 to 202410_44_44_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:33.325620 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_42_42_0 (state Temporary) 2024.10.22 00:25:33.389228 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 52 2024.10.22 00:25:33.472902 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_42_42_0 to 202410_49_49_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:33.507722 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 73496 2024.10.22 00:25:34.461231 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.08 GiB, peak 1.81 GiB, free memory in arenas 6.63 MiB, will set to 1.10 GiB (RSS), difference: 20.57 MiB 2024.10.22 00:25:34.944783 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:34.949373 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:35.574481 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.81 GiB, free memory in arenas 6.63 MiB, will set to 1.12 GiB (RSS), difference: -33.57 MiB 2024.10.22 00:25:35.971461 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 28 entries to flush up to offset 3990 2024.10.22 00:25:36.112549 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:36.118970 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:36.371710 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_43_43_0 (state Temporary) 2024.10.22 00:25:36.412280 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_43_43_0 to 202410_49_49_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:36.768616 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 3990 2024.10.22 00:25:36.949648 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 3509 2024.10.22 00:25:37.073546 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:37.075476 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:37.125469 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_77_77_0 (state Temporary) 2024.10.22 00:25:37.269722 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_77_77_0 to 202410_90_90_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:37.335375 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3509 2024.10.22 00:25:37.467054 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.12 GiB, peak 1.81 GiB, free memory in arenas 6.68 MiB, will set to 1.14 GiB (RSS), difference: 19.04 MiB 2024.10.22 00:25:37.962644 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_9_9_0 (state Temporary) 2024.10.22 00:25:38.836103 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 643 2024.10.22 00:25:39.040776 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:39.044273 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:39.086373 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_44_44_0 (state Temporary) 2024.10.22 00:25:39.129805 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_44_44_0 to 202410_57_57_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:39.184995 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 643 2024.10.22 00:25:39.331543 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:25:39.336201 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:25:39.588464 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.23 GiB, peak 1.81 GiB, free memory in arenas 7.59 MiB, will set to 1.17 GiB (RSS), difference: -55.60 MiB 2024.10.22 00:25:40.513197 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 75416 2024.10.22 00:25:40.897246 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 53 2024.10.22 00:25:41.056562 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:41.060092 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:41.302910 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 1.81 GiB, free memory in arenas 6.35 MiB, will set to 1.23 GiB (RSS), difference: -21.11 MiB 2024.10.22 00:25:41.312588 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_35_35_0 (state Temporary) 2024.10.22 00:25:41.357175 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 23 entries to flush up to offset 3532 2024.10.22 00:25:41.375621 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:41.378446 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:25:41.484255 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_35_35_0 to 202410_45_45_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:41.527209 [ 1243 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_40_8 to 202410_45_45_0 2024.10.22 00:25:41.542578 [ 1243 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:41.567466 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 53 2024.10.22 00:25:41.575312 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_43_43_0 (state Temporary) 2024.10.22 00:25:41.591959 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:41.601009 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:25:41.626440 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_43_43_0 to 202410_50_50_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:41.650997 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:25:41.664883 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_40_8 to 202410_45_45_0 into Compact with storage Full 2024.10.22 00:25:41.746323 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_78_78_0 (state Temporary) 2024.10.22 00:25:41.762534 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 75416 2024.10.22 00:25:41.820625 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:25:41.870793 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_78_78_0 to 202410_91_91_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:41.894331 [ 1256 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_86_17 to 202410_91_91_0 2024.10.22 00:25:41.904681 [ 1256 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:41.908461 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:25:41.931417 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3532 2024.10.22 00:25:41.936804 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_40_8, total 84 rows starting from the beginning of the part 2024.10.22 00:25:41.977500 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_41_41_0, total 1 rows starting from the beginning of the part 2024.10.22 00:25:42.018176 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_42_42_0, total 1 rows starting from the beginning of the part 2024.10.22 00:25:42.028432 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_86_17 to 202410_91_91_0 into Compact with storage Full 2024.10.22 00:25:42.082895 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:25:42.088572 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_86_17, total 4199 rows starting from the beginning of the part 2024.10.22 00:25:42.109132 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_87_87_0, total 33 rows starting from the beginning of the part 2024.10.22 00:25:42.133704 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_88_88_0, total 27 rows starting from the beginning of the part 2024.10.22 00:25:42.162617 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_43_43_0, total 1 rows starting from the beginning of the part 2024.10.22 00:25:42.254134 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_89_89_0, total 41 rows starting from the beginning of the part 2024.10.22 00:25:42.266449 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_90_90_0, total 40 rows starting from the beginning of the part 2024.10.22 00:25:42.294230 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_44_44_0, total 1 rows starting from the beginning of the part 2024.10.22 00:25:42.359983 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_45_45_0, total 1 rows starting from the beginning of the part 2024.10.22 00:25:42.379279 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_91_91_0, total 23 rows starting from the beginning of the part 2024.10.22 00:25:42.668946 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:25:42.776871 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:25:43.409620 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 1.81 GiB, free memory in arenas 5.11 MiB, will set to 1.27 GiB (RSS), difference: -30.73 MiB 2024.10.22 00:25:43.475000 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} MergeTask::MergeProjectionsStage: Merge sorted 4363 rows, containing 13 columns (13 merged, 0 gathered) in 1.354226914 sec., 3221.7643549210984 rows/sec., 518.55 KiB/sec. 2024.10.22 00:25:43.564305 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} MergedBlockOutputStream: filled checksums 202410_1_91_18 (state Temporary) 2024.10.22 00:25:43.850811 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_91_18 to 202410_1_91_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:43.855044 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_91_18} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_86_17, 202410_91_91_0] -> 202410_1_91_18 2024.10.22 00:25:43.882302 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:25:43.914292 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} MergeTask::MergeProjectionsStage: Merge sorted 89 rows, containing 32 columns (32 merged, 0 gathered) in 2.253188808 sec., 39.499574862081424 rows/sec., 5.65 KiB/sec. 2024.10.22 00:25:43.968475 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} MergedBlockOutputStream: filled checksums 202410_1_45_9 (state Temporary) 2024.10.22 00:25:44.272638 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 42 entries to flush up to offset 4032 2024.10.22 00:25:44.472631 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:44.475522 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:44.531931 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_44_44_0 (state Temporary) 2024.10.22 00:25:44.560528 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_45_9 to 202410_1_45_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:44.563841 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_45_9} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_40_8, 202410_45_45_0] -> 202410_1_45_9 2024.10.22 00:25:44.563882 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_44_44_0 to 202410_50_50_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:44.589519 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_45_9 to 202410_50_50_0 2024.10.22 00:25:44.599379 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:44.602314 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:44.616967 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_45_9 to 202410_50_50_0 into Compact with storage Full 2024.10.22 00:25:44.660734 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:25:44.700639 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4032 2024.10.22 00:25:44.772298 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:25:44.772878 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_45_9, total 3949 rows starting from the beginning of the part 2024.10.22 00:25:44.885115 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_46_46_0, total 61 rows starting from the beginning of the part 2024.10.22 00:25:45.020614 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_47_47_0, total 34 rows starting from the beginning of the part 2024.10.22 00:25:45.057688 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_48_48_0, total 22 rows starting from the beginning of the part 2024.10.22 00:25:45.079329 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_49_49_0, total 28 rows starting from the beginning of the part 2024.10.22 00:25:45.103485 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_50_50_0, total 42 rows starting from the beginning of the part 2024.10.22 00:25:45.377078 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.81 GiB, free memory in arenas 4.76 MiB, will set to 1.32 GiB (RSS), difference: -14.44 MiB 2024.10.22 00:25:45.936779 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 3598 2024.10.22 00:25:45.958560 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} MergeTask::MergeProjectionsStage: Merge sorted 4136 rows, containing 13 columns (13 merged, 0 gathered) in 1.347418377 sec., 3069.5736903995044 rows/sec., 1.29 MiB/sec. 2024.10.22 00:25:46.205210 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} MergedBlockOutputStream: filled checksums 202410_1_50_10 (state Temporary) 2024.10.22 00:25:46.264659 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:46.267758 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:46.375385 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_9_9_0 to 202410_12_12_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:46.515986 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_79_79_0 (state Temporary) 2024.10.22 00:25:46.620234 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.81 GiB, free memory in arenas 4.68 MiB, will set to 1.34 GiB (RSS), difference: -46.52 MiB 2024.10.22 00:25:46.673376 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_50_10 to 202410_1_50_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:46.675646 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_50_10} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_45_9, 202410_50_50_0] -> 202410_1_50_10 2024.10.22 00:25:46.687539 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 670 2024.10.22 00:25:46.696719 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.04 MiB. 2024.10.22 00:25:46.776434 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_79_79_0 to 202410_92_92_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:46.785511 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:46.795255 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:46.843339 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_45_45_0 (state Temporary) 2024.10.22 00:25:46.851445 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3598 2024.10.22 00:25:46.885434 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_45_45_0 to 202410_58_58_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:46.956308 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 670 2024.10.22 00:25:47.015490 [ 1156 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:25:47.043050 [ 1156 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:25:47.213273 [ 1263 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 00:25:48.264089 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 00:25:48.601418 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.81 GiB, free memory in arenas 5.59 MiB, will set to 1.38 GiB (RSS), difference: -4.20 MiB 2024.10.22 00:25:48.767082 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 76952 2024.10.22 00:25:49.163580 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 337 2024.10.22 00:25:49.330335 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:25:49.334389 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:25:49.493834 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:49.500419 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:49.778455 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_44_44_0 (state Temporary) 2024.10.22 00:25:49.856430 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_44_44_0 to 202410_51_51_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:49.884021 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 76952 2024.10.22 00:25:49.887571 [ 1188 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_46_9 to 202410_51_51_0 2024.10.22 00:25:49.910303 [ 1188 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:49.914926 [ 1188 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:49.955928 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_46_9 to 202410_51_51_0 into Compact with storage Full 2024.10.22 00:25:50.041440 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:25:50.150787 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} MergeTreeSequentialSource: Reading 12 marks from part 202410_1_46_9, total 82190 rows starting from the beginning of the part 2024.10.22 00:25:50.162728 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_47_47_0, total 1824 rows starting from the beginning of the part 2024.10.22 00:25:50.169253 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_48_48_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:25:50.176148 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_49_49_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:25:50.182962 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_50_50_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:25:50.196789 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_51_51_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:25:50.303387 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.81 GiB, free memory in arenas 7.51 MiB, will set to 1.40 GiB (RSS), difference: -6.24 MiB 2024.10.22 00:25:50.758814 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 345.34 MiB using storage policy from min volume index 0 2024.10.22 00:25:50.862788 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 3644 2024.10.22 00:25:51.300131 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:51.304805 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:25:51.461454 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_80_80_0 (state Temporary) 2024.10.22 00:25:51.499272 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_80_80_0 to 202410_93_93_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:51.562112 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3644 2024.10.22 00:25:51.771662 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zpc/obwdaacwakhhpjihuklhalldnaiqq (LogSeriesLimiter: on interval from 2024-10-22 00:24:30 to 2024-10-22 00:25:51 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:25:52.222986 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 4082 2024.10.22 00:25:52.432839 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:52.445528 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:52.797371 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_45_45_0 (state Temporary) 2024.10.22 00:25:52.976936 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_45_45_0 to 202410_51_51_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:53.254813 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4082 2024.10.22 00:25:53.459904 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.81 GiB, free memory in arenas 10.90 MiB, will set to 1.41 GiB (RSS), difference: -111.67 MiB 2024.10.22 00:25:53.780202 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_51_10/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:25:54.326835 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.81 GiB, free memory in arenas 13.42 MiB, will set to 1.41 GiB (RSS), difference: -73.33 MiB 2024.10.22 00:25:54.459605 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 675 2024.10.22 00:25:54.638962 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:54.629352 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} MergeTask::MergeProjectionsStage: Merge sorted 90542 rows, containing 4 columns (4 merged, 0 gathered) in 4.687801344 sec., 19314.38500820567 rows/sec., 284.67 KiB/sec. 2024.10.22 00:25:54.644548 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:54.670046 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_46_46_0 (state Temporary) 2024.10.22 00:25:54.724510 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_46_46_0 to 202410_59_59_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:54.813551 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} MergedBlockOutputStream: filled checksums 202410_1_51_10 (state Temporary) 2024.10.22 00:25:54.849512 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_51_10 to 202410_1_51_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:54.857106 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_51_10} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_46_9, 202410_51_51_0] -> 202410_1_51_10 2024.10.22 00:25:54.975730 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 675 2024.10.22 00:25:54.990745 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:25:55.428003 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.81 GiB, free memory in arenas 13.64 MiB, will set to 1.41 GiB (RSS), difference: -79.46 MiB 2024.10.22 00:25:55.672998 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 28 entries to flush up to offset 3672 2024.10.22 00:25:55.865784 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:55.968920 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:56.003797 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_81_81_0 (state Temporary) 2024.10.22 00:25:56.049300 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_81_81_0 to 202410_94_94_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:56.067525 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kjh/bzivnfiqgcbdiwvewjlbnnguynseq (LogSeriesLimiter: on interval from 2024-10-22 00:25:51 to 2024-10-22 00:25:56 accepted series 1 / 219 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:25:56.201438 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3672 2024.10.22 00:25:56.569546 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 54 2024.10.22 00:25:56.683296 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 375 2024.10.22 00:25:56.897988 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2304 entries to flush up to offset 79256 2024.10.22 00:25:56.967596 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:56.969773 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:57.066047 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_36_36_0 (state Temporary) 2024.10.22 00:25:57.264946 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_36_36_0 to 202410_46_46_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:57.376148 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 11.33 MiB, will set to 1.41 GiB (RSS), difference: -153.26 MiB 2024.10.22 00:25:57.437677 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 54 2024.10.22 00:25:57.492992 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:25:57.499743 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:25:57.782239 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_45_45_0 (state Temporary) 2024.10.22 00:25:57.845364 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_45_45_0 to 202410_52_52_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:25:57.910452 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 79256 2024.10.22 00:25:58.708953 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 10.11 MiB, will set to 1.41 GiB (RSS), difference: -121.34 MiB 2024.10.22 00:25:59.333019 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:25:59.337752 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:25:59.908543 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:26:00.210461 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 25 entries to flush up to offset 3697 2024.10.22 00:26:00.412974 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:00.417581 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:26:00.474654 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_82_82_0 (state Temporary) 2024.10.22 00:26:00.532456 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_82_82_0 to 202410_95_95_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:00.758947 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3697 2024.10.22 00:26:00.867973 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 29 entries to flush up to offset 4111 2024.10.22 00:26:01.082381 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:01.083970 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:26:01.342439 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_46_46_0 (state Temporary) 2024.10.22 00:26:01.381301 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_46_46_0 to 202410_52_52_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:01.579259 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4111 2024.10.22 00:26:02.054349 [ 1227 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:26:02.083734 [ 1227 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:26:02.380290 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 7.69 MiB, will set to 1.43 GiB (RSS), difference: 3.45 MiB 2024.10.22 00:26:02.477761 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 688 2024.10.22 00:26:02.754182 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.077178618 sec. 2024.10.22 00:26:02.913753 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:02.917636 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:26:03.059381 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_47_47_0 (state Temporary) 2024.10.22 00:26:03.101051 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_47_47_0 to 202410_60_60_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:03.150633 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 688 2024.10.22 00:26:04.412545 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.81 GiB, free memory in arenas 7.66 MiB, will set to 1.44 GiB (RSS), difference: -1.08 MiB 2024.10.22 00:26:04.605581 [ 1249 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 00:26:04.763985 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 24 entries to flush up to offset 3721 2024.10.22 00:26:04.875228 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:04.877553 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:26:04.920696 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1915 entries to flush up to offset 81171 2024.10.22 00:26:04.946239 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 55 2024.10.22 00:26:05.014132 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_83_83_0 (state Temporary) 2024.10.22 00:26:05.135966 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_83_83_0 to 202410_96_96_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:05.174737 [ 1237 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_91_18 to 202410_96_96_0 2024.10.22 00:26:05.190257 [ 1237 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:05.196290 [ 1237 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.65 GiB. 2024.10.22 00:26:05.300902 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3721 2024.10.22 00:26:05.319894 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_91_18 to 202410_96_96_0 into Compact with storage Full 2024.10.22 00:26:05.374216 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:26:05.379982 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_91_18, total 4363 rows starting from the beginning of the part 2024.10.22 00:26:05.397182 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_92_92_0, total 66 rows starting from the beginning of the part 2024.10.22 00:26:05.480340 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.81 GiB, free memory in arenas 7.28 MiB, will set to 1.44 GiB (RSS), difference: -2.83 MiB 2024.10.22 00:26:05.523959 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_93_93_0, total 46 rows starting from the beginning of the part 2024.10.22 00:26:05.657496 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_94_94_0, total 28 rows starting from the beginning of the part 2024.10.22 00:26:05.737725 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:05.741802 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:05.770985 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_95_95_0, total 25 rows starting from the beginning of the part 2024.10.22 00:26:05.805134 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_96_96_0, total 24 rows starting from the beginning of the part 2024.10.22 00:26:05.925788 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_37_37_0 (state Temporary) 2024.10.22 00:26:05.989146 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_37_37_0 to 202410_47_47_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:06.083507 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 55 2024.10.22 00:26:06.121518 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:06.125354 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:06.637410 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.81 GiB, free memory in arenas 7.06 MiB, will set to 1.45 GiB (RSS), difference: -21.61 MiB 2024.10.22 00:26:06.718316 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_46_46_0 (state Temporary) 2024.10.22 00:26:06.892654 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_46_46_0 to 202410_53_53_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:06.927679 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 81171 2024.10.22 00:26:07.647461 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} MergeTask::MergeProjectionsStage: Merge sorted 4552 rows, containing 13 columns (13 merged, 0 gathered) in 2.335593355 sec., 1948.9694086751674 rows/sec., 312.60 KiB/sec. 2024.10.22 00:26:07.662745 [ 1268 ] {} DatabaseCatalog: Execution took 344 ms. 2024.10.22 00:26:07.783144 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} MergedBlockOutputStream: filled checksums 202410_1_96_19 (state Temporary) 2024.10.22 00:26:07.947651 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_96_19 to 202410_1_96_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:07.952566 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_96_19} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_91_18, 202410_96_96_0] -> 202410_1_96_19 2024.10.22 00:26:07.976755 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:26:08.377018 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.81 GiB, free memory in arenas 6.64 MiB, will set to 1.45 GiB (RSS), difference: 8.39 MiB 2024.10.22 00:26:09.085539 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 33 entries to flush up to offset 4144 2024.10.22 00:26:09.245437 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:09.254348 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:09.302425 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_47_47_0 (state Temporary) 2024.10.22 00:26:09.304999 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 3762 2024.10.22 00:26:09.336046 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:26:09.340444 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:26:09.415505 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_47_47_0 to 202410_53_53_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:09.567686 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4144 2024.10.22 00:26:09.908168 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:09.911575 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:09.962258 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_84_84_0 (state Temporary) 2024.10.22 00:26:10.113760 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_84_84_0 to 202410_97_97_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:10.271021 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3762 2024.10.22 00:26:10.460333 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.81 GiB, free memory in arenas 6.68 MiB, will set to 1.45 GiB (RSS), difference: 3.43 MiB 2024.10.22 00:26:10.653698 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 701 2024.10.22 00:26:10.957385 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:10.960799 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:11.115144 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_48_48_0 (state Temporary) 2024.10.22 00:26:11.258857 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_48_48_0 to 202410_61_61_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:11.385711 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 701 2024.10.22 00:26:11.498885 [ 1151 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_56_11 to 202410_61_61_0 2024.10.22 00:26:11.614857 [ 1151 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:11.617303 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:11.626566 [ 1151 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 257 ms. 2024.10.22 00:26:11.664843 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_56_11 to 202410_61_61_0 into Compact with storage Full 2024.10.22 00:26:11.684644 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:26:11.696800 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_56_11, total 735 rows starting from the beginning of the part 2024.10.22 00:26:11.713244 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_57_57_0, total 14 rows starting from the beginning of the part 2024.10.22 00:26:11.727316 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_58_58_0, total 27 rows starting from the beginning of the part 2024.10.22 00:26:11.875690 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_59_59_0, total 5 rows starting from the beginning of the part 2024.10.22 00:26:11.908173 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_60_60_0, total 13 rows starting from the beginning of the part 2024.10.22 00:26:11.952359 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_61_61_0, total 13 rows starting from the beginning of the part 2024.10.22 00:26:12.066750 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:12.070293 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:12.357197 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.81 GiB, free memory in arenas 7.04 MiB, will set to 1.46 GiB (RSS), difference: 1.39 MiB 2024.10.22 00:26:12.755546 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} MergeTask::MergeProjectionsStage: Merge sorted 807 rows, containing 12 columns (12 merged, 0 gathered) in 1.095472832 sec., 736.6682006405066 rows/sec., 112.09 KiB/sec. 2024.10.22 00:26:12.806874 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} MergedBlockOutputStream: filled checksums 202410_1_61_12 (state Temporary) 2024.10.22 00:26:13.120305 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_61_12 to 202410_1_61_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:13.124892 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_61_12} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_56_11, 202410_61_61_0] -> 202410_1_61_12 2024.10.22 00:26:13.160413 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:26:13.588157 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 56 2024.10.22 00:26:13.936613 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 83096 2024.10.22 00:26:13.877784 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:13.982218 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:14.239232 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_10_10_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:26:14.271528 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_38_38_0 (state Temporary) 2024.10.22 00:26:14.275688 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 3805 2024.10.22 00:26:14.475895 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_38_38_0 to 202410_48_48_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:14.514324 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.81 GiB, free memory in arenas 6.47 MiB, will set to 1.46 GiB (RSS), difference: -2.39 MiB 2024.10.22 00:26:14.623870 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:14.629261 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:14.671465 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 56 2024.10.22 00:26:14.754596 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:14.758657 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:14.774104 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_85_85_0 (state Temporary) 2024.10.22 00:26:14.832193 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_85_85_0 to 202410_98_98_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:14.878000 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_47_47_0 (state Temporary) 2024.10.22 00:26:14.885075 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3805 2024.10.22 00:26:14.964130 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:26:15.145117 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_10_10_0 (state Temporary) 2024.10.22 00:26:15.158275 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_47_47_0 to 202410_54_54_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:15.182303 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 83096 2024.10.22 00:26:16.571980 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.81 GiB, free memory in arenas 7.30 MiB, will set to 1.47 GiB (RSS), difference: 8.32 MiB 2024.10.22 00:26:16.566853 [ 1198 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 00:26:17.093801 [ 1157 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:26:17.171324 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 27 entries to flush up to offset 4171 2024.10.22 00:26:17.321627 [ 1157 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:26:17.325001 [ 1157 ] {} DNSCacheUpdater: Execution took 232 ms. 2024.10.22 00:26:17.464199 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:17.569068 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:17.598426 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_48_48_0 (state Temporary) 2024.10.22 00:26:17.662748 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_48_48_0 to 202410_54_54_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:17.799818 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4171 2024.10.22 00:26:18.888425 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 713 2024.10.22 00:26:18.907803 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 28 entries to flush up to offset 3833 2024.10.22 00:26:19.063687 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:19.067480 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:19.201041 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_49_49_0 (state Temporary) 2024.10.22 00:26:19.252588 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_49_49_0 to 202410_62_62_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:19.278983 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:19.280834 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.64 GiB. 2024.10.22 00:26:19.298438 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 713 2024.10.22 00:26:19.343060 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_86_86_0 (state Temporary) 2024.10.22 00:26:19.343761 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:26:19.352304 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:26:19.392018 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_86_86_0 to 202410_99_99_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:19.458396 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3833 2024.10.22 00:26:20.668162 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.81 GiB, free memory in arenas 6.34 MiB, will set to 1.48 GiB (RSS), difference: 26.37 MiB 2024.10.22 00:26:20.683566 [ 1713 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kjh/bzivnfiqgcbdiwvewjlbnnguynseq, size 361826 2024.10.22 00:26:22.173262 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 57 2024.10.22 00:26:22.187212 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1538 entries to flush up to offset 84634 2024.10.22 00:26:22.414933 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.81 GiB, free memory in arenas 5.93 MiB, will set to 1.47 GiB (RSS), difference: 132.82 MiB 2024.10.22 00:26:22.509379 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:22.512082 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.63 GiB. 2024.10.22 00:26:22.675083 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_39_39_0 (state Temporary) 2024.10.22 00:26:22.775470 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_39_39_0 to 202410_49_49_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:22.800248 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:22.809261 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.63 GiB. 2024.10.22 00:26:22.907492 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 57 2024.10.22 00:26:23.285037 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_48_48_0 (state Temporary) 2024.10.22 00:26:23.464706 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 3860 2024.10.22 00:26:23.495384 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_48_48_0 to 202410_55_55_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:23.541687 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 84634 2024.10.22 00:26:23.777085 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_10_10_0 to 202410_13_13_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:23.923873 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:23.933335 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.62 GiB. 2024.10.22 00:26:24.204995 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_87_87_0 (state Temporary) 2024.10.22 00:26:24.275251 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_87_87_0 to 202410_100_100_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:24.382304 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3860 2024.10.22 00:26:24.760263 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 1.81 GiB, free memory in arenas 3.93 MiB, will set to 1.46 GiB (RSS), difference: 167.47 MiB 2024.10.22 00:26:25.169487 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_4_4_0 (state Temporary) 2024.10.22 00:26:25.273257 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uah/pxkafxonowbkaplsypmhrdkhkdjsn (LogSeriesLimiter: on interval from 2024-10-22 00:25:56 to 2024-10-22 00:26:25 accepted series 1 / 136 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:26:25.289317 [ 1704 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zpc/obwdaacwakhhpjihuklhalldnaiqq, size 305 2024.10.22 00:26:25.311579 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 60 entries to flush up to offset 4231 2024.10.22 00:26:25.343618 [ 1711 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uah/pxkafxonowbkaplsypmhrdkhkdjsn, size 4 2024.10.22 00:26:25.451366 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:25.456610 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:25.710758 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_49_49_0 (state Temporary) 2024.10.22 00:26:25.853368 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_49_49_0 to 202410_55_55_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:25.881299 [ 1225 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_50_10 to 202410_55_55_0 2024.10.22 00:26:25.993384 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4231 2024.10.22 00:26:26.002728 [ 1225 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:26.008345 [ 1225 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:26.060056 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_50_10 to 202410_55_55_0 into Compact with storage Full 2024.10.22 00:26:26.082161 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:26:26.195235 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_50_10, total 4136 rows starting from the beginning of the part 2024.10.22 00:26:26.206305 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_51_51_0, total 50 rows starting from the beginning of the part 2024.10.22 00:26:26.229067 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_52_52_0, total 29 rows starting from the beginning of the part 2024.10.22 00:26:26.257791 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_53_53_0, total 33 rows starting from the beginning of the part 2024.10.22 00:26:26.267022 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_54_54_0, total 27 rows starting from the beginning of the part 2024.10.22 00:26:26.278544 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_55_55_0, total 60 rows starting from the beginning of the part 2024.10.22 00:26:26.473190 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.81 GiB, free memory in arenas 6.63 MiB, will set to 1.42 GiB (RSS), difference: 25.39 MiB 2024.10.22 00:26:26.566371 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 375 2024.10.22 00:26:26.803674 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 720 2024.10.22 00:26:26.855329 [ 1232 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 00:26:27.152936 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:27.156050 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:27.198791 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_50_50_0 (state Temporary) 2024.10.22 00:26:27.458227 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_50_50_0 to 202410_63_63_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:27.493144 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 720 2024.10.22 00:26:27.547708 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} MergeTask::MergeProjectionsStage: Merge sorted 4335 rows, containing 13 columns (13 merged, 0 gathered) in 1.497639125 sec., 2894.5557895998477 rows/sec., 1.22 MiB/sec. 2024.10.22 00:26:27.806968 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} MergedBlockOutputStream: filled checksums 202410_1_55_11 (state Temporary) 2024.10.22 00:26:27.886452 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_55_11 to 202410_1_55_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:27.891528 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_55_11} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_50_10, 202410_55_55_0] -> 202410_1_55_11 2024.10.22 00:26:28.057200 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.04 MiB. 2024.10.22 00:26:28.062564 [ 1253 ] {} DatabaseCatalog: Execution took 295 ms. 2024.10.22 00:26:28.355104 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 7.76 MiB, will set to 1.42 GiB (RSS), difference: 5.19 MiB 2024.10.22 00:26:28.389622 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 3907 2024.10.22 00:26:28.693694 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:28.696962 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:28.865806 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_88_88_0 (state Temporary) 2024.10.22 00:26:28.919703 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_88_88_0 to 202410_101_101_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:28.962608 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3907 2024.10.22 00:26:28.972671 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_96_19 to 202410_101_101_0 2024.10.22 00:26:28.980986 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:28.983301 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:29.017766 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_96_19 to 202410_101_101_0 into Compact with storage Full 2024.10.22 00:26:29.056192 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:26:29.064689 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_96_19, total 4552 rows starting from the beginning of the part 2024.10.22 00:26:29.088238 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_97_97_0, total 41 rows starting from the beginning of the part 2024.10.22 00:26:29.114986 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_98_98_0, total 43 rows starting from the beginning of the part 2024.10.22 00:26:29.139928 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_99_99_0, total 28 rows starting from the beginning of the part 2024.10.22 00:26:29.155372 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_100_100_0, total 27 rows starting from the beginning of the part 2024.10.22 00:26:29.169093 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_101_101_0, total 47 rows starting from the beginning of the part 2024.10.22 00:26:29.440610 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:26:29.444157 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:26:29.471100 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 7.78 MiB, will set to 1.42 GiB (RSS), difference: -3.48 MiB 2024.10.22 00:26:30.385246 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} MergeTask::MergeProjectionsStage: Merge sorted 4738 rows, containing 13 columns (13 merged, 0 gathered) in 1.373739913 sec., 3448.9789189083563 rows/sec., 550.51 KiB/sec. 2024.10.22 00:26:30.409503 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 58 2024.10.22 00:26:30.475248 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 7.78 MiB, will set to 1.42 GiB (RSS), difference: 1.09 MiB 2024.10.22 00:26:30.547319 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 86554 2024.10.22 00:26:30.567152 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} MergedBlockOutputStream: filled checksums 202410_1_101_20 (state Temporary) 2024.10.22 00:26:30.611882 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_101_20 to 202410_1_101_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:30.615261 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_101_20} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_96_19, 202410_101_101_0] -> 202410_1_101_20 2024.10.22 00:26:30.675095 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:30.677656 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:30.767340 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:26:30.860865 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_40_40_0 (state Temporary) 2024.10.22 00:26:31.284729 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_40_40_0 to 202410_50_50_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:31.373537 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 58 2024.10.22 00:26:31.423682 [ 1265 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_45_9 to 202410_50_50_0 2024.10.22 00:26:31.438702 [ 1265 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:31.440852 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:31.474678 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_45_9 to 202410_50_50_0 into Compact with storage Full 2024.10.22 00:26:31.677188 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:26:31.686205 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_45_9, total 89 rows starting from the beginning of the part 2024.10.22 00:26:31.764543 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_46_46_0, total 1 rows starting from the beginning of the part 2024.10.22 00:26:31.783353 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:31.786667 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:31.810526 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ezr/yybvfddfgzesqmbkyzofkzljhjxly. 2024.10.22 00:26:31.814418 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_47_47_0, total 1 rows starting from the beginning of the part 2024.10.22 00:26:31.873303 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_48_48_0, total 1 rows starting from the beginning of the part 2024.10.22 00:26:32.014726 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_49_49_0 (state Temporary) 2024.10.22 00:26:32.164002 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_49_49_0, total 1 rows starting from the beginning of the part 2024.10.22 00:26:32.175762 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_49_49_0 to 202410_56_56_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:32.213525 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 86554 2024.10.22 00:26:32.229397 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_50_50_0, total 1 rows starting from the beginning of the part 2024.10.22 00:26:32.232776 [ 1262 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_51_10 to 202410_56_56_0 2024.10.22 00:26:32.253395 [ 1262 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:32.255793 [ 1262 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:32.326734 [ 1272 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:26:32.366955 [ 1272 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:26:32.371452 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_51_10 to 202410_56_56_0 into Compact with storage Full 2024.10.22 00:26:32.384946 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:26:32.397078 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} MergeTreeSequentialSource: Reading 13 marks from part 202410_1_51_10, total 90542 rows starting from the beginning of the part 2024.10.22 00:26:32.419161 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_52_52_0, total 2304 rows starting from the beginning of the part 2024.10.22 00:26:32.429368 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_53_53_0, total 1915 rows starting from the beginning of the part 2024.10.22 00:26:32.443103 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_54_54_0, total 1925 rows starting from the beginning of the part 2024.10.22 00:26:32.461377 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_55_55_0, total 1538 rows starting from the beginning of the part 2024.10.22 00:26:32.498854 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 6.75 MiB, will set to 1.42 GiB (RSS), difference: -1.07 MiB 2024.10.22 00:26:32.564052 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_56_56_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:26:32.966371 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 3971 2024.10.22 00:26:33.446560 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:33.451488 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:33.498272 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.81 GiB, free memory in arenas 8.12 MiB, will set to 1.42 GiB (RSS), difference: -5.42 MiB 2024.10.22 00:26:33.507456 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 68 entries to flush up to offset 4299 2024.10.22 00:26:33.595812 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_89_89_0 (state Temporary) 2024.10.22 00:26:33.641011 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_89_89_0 to 202410_102_102_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:33.749624 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:33.754210 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:33.790716 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 3971 2024.10.22 00:26:33.796981 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_50_50_0 (state Temporary) 2024.10.22 00:26:33.843369 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_50_50_0 to 202410_56_56_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:33.977104 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4299 2024.10.22 00:26:34.120570 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 413 2024.10.22 00:26:34.705131 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 8.64 MiB, will set to 1.42 GiB (RSS), difference: 1.11 MiB 2024.10.22 00:26:34.848923 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_56_11/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:26:34.891009 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} MergeTask::MergeProjectionsStage: Merge sorted 94 rows, containing 32 columns (32 merged, 0 gathered) in 3.433866088 sec., 27.374393057578082 rows/sec., 3.88 KiB/sec. 2024.10.22 00:26:34.930548 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} MergedBlockOutputStream: filled checksums 202410_1_50_10 (state Temporary) 2024.10.22 00:26:35.005606 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 740 2024.10.22 00:26:35.189197 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:35.203698 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:35.238322 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_51_51_0 (state Temporary) 2024.10.22 00:26:35.271867 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_50_10 to 202410_1_50_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:35.275898 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_50_10} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_45_9, 202410_50_50_0] -> 202410_1_50_10 2024.10.22 00:26:35.425267 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:26:35.485045 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_51_51_0 to 202410_64_64_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:35.513437 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} MergeTask::MergeProjectionsStage: Merge sorted 100144 rows, containing 4 columns (4 merged, 0 gathered) in 3.145510101 sec., 31837.125548623375 rows/sec., 469.35 KiB/sec. 2024.10.22 00:26:35.591122 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} MergedBlockOutputStream: filled checksums 202410_1_56_11 (state Temporary) 2024.10.22 00:26:35.649344 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 740 2024.10.22 00:26:35.758477 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_56_11 to 202410_1_56_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:35.762095 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_56_11} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_51_10, 202410_56_56_0] -> 202410_1_56_11 2024.10.22 00:26:35.792755 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:26:36.583442 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.81 GiB, free memory in arenas 12.71 MiB, will set to 1.43 GiB (RSS), difference: 15.60 MiB 2024.10.22 00:26:36.683998 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_4_4_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:26:37.827817 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 4005 2024.10.22 00:26:38.049420 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:38.052279 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:38.320845 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_90_90_0 (state Temporary) 2024.10.22 00:26:38.371057 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_90_90_0 to 202410_103_103_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:38.384682 [ 1169 ] {} DatabaseCatalog: Execution took 320 ms. 2024.10.22 00:26:38.515145 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4005 2024.10.22 00:26:39.225369 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2105 entries to flush up to offset 88659 2024.10.22 00:26:39.307801 [ 1160 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 00:26:39.342679 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:26:39.347391 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:26:40.177425 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:40.182737 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:40.517475 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_50_50_0 (state Temporary) 2024.10.22 00:26:40.571608 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.81 GiB, free memory in arenas 7.28 MiB, will set to 1.43 GiB (RSS), difference: -11.59 MiB 2024.10.22 00:26:40.611707 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_50_50_0 to 202410_57_57_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:40.650553 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 88659 2024.10.22 00:26:41.758199 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 28 entries to flush up to offset 4327 2024.10.22 00:26:41.979683 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:41.987213 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:42.046835 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_51_51_0 (state Temporary) 2024.10.22 00:26:42.086249 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_51_51_0 to 202410_57_57_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:42.355916 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4327 2024.10.22 00:26:42.528113 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 23 entries to flush up to offset 4028 2024.10.22 00:26:42.659931 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 7.29 MiB, will set to 1.43 GiB (RSS), difference: 8.94 MiB 2024.10.22 00:26:42.757141 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:42.760124 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:42.854770 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_91_91_0 (state Temporary) 2024.10.22 00:26:42.932220 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_91_91_0 to 202410_104_104_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:43.153305 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 759 2024.10.22 00:26:43.249617 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4028 2024.10.22 00:26:43.575623 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:43.578867 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:43.631022 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_52_52_0 (state Temporary) 2024.10.22 00:26:43.670447 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_52_52_0 to 202410_65_65_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:43.710943 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 759 2024.10.22 00:26:44.398062 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.81 GiB, free memory in arenas 6.53 MiB, will set to 1.43 GiB (RSS), difference: -1.20 MiB 2024.10.22 00:26:46.377727 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 59 2024.10.22 00:26:46.756126 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:46.758602 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:46.915782 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_41_41_0 (state Temporary) 2024.10.22 00:26:47.108718 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_41_41_0 to 202410_51_51_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:47.173783 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 59 2024.10.22 00:26:47.258443 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 20 entries to flush up to offset 4048 2024.10.22 00:26:47.371748 [ 1219 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:26:47.409351 [ 1219 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:26:47.486607 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:47.499028 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:47.552710 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_92_92_0 (state Temporary) 2024.10.22 00:26:47.658868 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1735 entries to flush up to offset 90394 2024.10.22 00:26:47.796529 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_92_92_0 to 202410_105_105_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:47.858377 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4048 2024.10.22 00:26:48.636311 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:48.643482 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:48.697599 [ 1223 ] {} DatabaseCatalog: Execution took 312 ms. 2024.10.22 00:26:48.980919 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_51_51_0 (state Temporary) 2024.10.22 00:26:49.057171 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_51_51_0 to 202410_58_58_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:49.080287 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 90394 2024.10.22 00:26:49.344908 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:26:49.349300 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:26:49.861437 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 19 entries to flush up to offset 4346 2024.10.22 00:26:50.167484 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:50.170225 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:50.230739 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_52_52_0 (state Temporary) 2024.10.22 00:26:50.275258 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_52_52_0 to 202410_58_58_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:50.312128 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4346 2024.10.22 00:26:51.212729 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 764 2024.10.22 00:26:51.528650 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:51.534710 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:51.676733 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_53_53_0 (state Temporary) 2024.10.22 00:26:51.743440 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_53_53_0 to 202410_66_66_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:51.863720 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 28 entries to flush up to offset 4076 2024.10.22 00:26:51.885156 [ 1246 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_61_12 to 202410_66_66_0 2024.10.22 00:26:51.905273 [ 1246 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:51.908814 [ 1246 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.58 GiB. 2024.10.22 00:26:51.933221 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_61_12 to 202410_66_66_0 into Compact with storage Full 2024.10.22 00:26:51.968642 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:26:52.072942 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_61_12, total 807 rows starting from the beginning of the part 2024.10.22 00:26:52.082914 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_62_62_0, total 12 rows starting from the beginning of the part 2024.10.22 00:26:52.088956 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 764 2024.10.22 00:26:52.205237 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_63_63_0, total 7 rows starting from the beginning of the part 2024.10.22 00:26:52.228377 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_64_64_0, total 20 rows starting from the beginning of the part 2024.10.22 00:26:52.252606 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_65_65_0, total 19 rows starting from the beginning of the part 2024.10.22 00:26:52.277452 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_66_66_0, total 5 rows starting from the beginning of the part 2024.10.22 00:26:52.444517 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:52.449721 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:26:52.501530 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:52.506562 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:26:52.522035 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.81 GiB, free memory in arenas 6.46 MiB, will set to 1.43 GiB (RSS), difference: -1.93 MiB 2024.10.22 00:26:52.572202 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_93_93_0 (state Temporary) 2024.10.22 00:26:52.615421 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_93_93_0 to 202410_106_106_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:52.667721 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_101_20 to 202410_106_106_0 2024.10.22 00:26:52.680166 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:52.682987 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:26:52.684026 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4076 2024.10.22 00:26:52.703321 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_101_20 to 202410_106_106_0 into Compact with storage Full 2024.10.22 00:26:52.836050 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:26:52.846122 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_101_20, total 4738 rows starting from the beginning of the part 2024.10.22 00:26:52.862849 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_102_102_0, total 64 rows starting from the beginning of the part 2024.10.22 00:26:52.876157 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_103_103_0, total 34 rows starting from the beginning of the part 2024.10.22 00:26:52.898664 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_104_104_0, total 23 rows starting from the beginning of the part 2024.10.22 00:26:53.011542 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_105_105_0, total 20 rows starting from the beginning of the part 2024.10.22 00:26:53.033743 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_106_106_0, total 28 rows starting from the beginning of the part 2024.10.22 00:26:53.260041 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} MergeTask::MergeProjectionsStage: Merge sorted 870 rows, containing 12 columns (12 merged, 0 gathered) in 1.344415457 sec., 647.1213905420012 rows/sec., 98.56 KiB/sec. 2024.10.22 00:26:53.414058 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} MergedBlockOutputStream: filled checksums 202410_1_66_13 (state Temporary) 2024.10.22 00:26:53.479168 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_66_13 to 202410_1_66_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:53.481210 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_66_13} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_61_12, 202410_66_66_0] -> 202410_1_66_13 2024.10.22 00:26:53.523535 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:26:54.062080 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} MergeTask::MergeProjectionsStage: Merge sorted 4907 rows, containing 13 columns (13 merged, 0 gathered) in 1.373258339 sec., 3573.253378947805 rows/sec., 567.78 KiB/sec. 2024.10.22 00:26:54.134208 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_106_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:26:54.328347 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} MergedBlockOutputStream: filled checksums 202410_1_106_21 (state Temporary) 2024.10.22 00:26:54.370777 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_106_21 to 202410_1_106_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:54.384241 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_106_21} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_101_20, 202410_106_106_0] -> 202410_1_106_21 2024.10.22 00:26:54.436513 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:26:54.676698 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 60 2024.10.22 00:26:55.088884 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:55.098574 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:26:55.472435 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_42_42_0 (state Temporary) 2024.10.22 00:26:55.568195 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_42_42_0 to 202410_52_52_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:55.884594 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 60 2024.10.22 00:26:56.092821 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 92314 2024.10.22 00:26:56.692229 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 4126 2024.10.22 00:26:56.741531 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_11_11_0 (state Temporary) 2024.10.22 00:26:56.867847 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:56.912944 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.81 GiB, free memory in arenas 6.34 MiB, will set to 1.43 GiB (RSS), difference: -4.94 MiB 2024.10.22 00:26:56.972302 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:26:57.122291 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_94_94_0 (state Temporary) 2024.10.22 00:26:57.172936 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_94_94_0 to 202410_107_107_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:57.244613 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:57.254561 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:26:57.350839 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4126 2024.10.22 00:26:57.402797 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_52_52_0 (state Temporary) 2024.10.22 00:26:57.449795 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_52_52_0 to 202410_59_59_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:57.580898 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 92314 2024.10.22 00:26:57.917473 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 37 entries to flush up to offset 4383 2024.10.22 00:26:58.256364 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:26:58.259228 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:26:58.304893 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_53_53_0 (state Temporary) 2024.10.22 00:26:58.456647 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_53_53_0 to 202410_59_59_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:26:58.497753 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4383 2024.10.22 00:26:59.349317 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:26:59.352388 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:26:59.592708 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 784 2024.10.22 00:27:00.012276 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:00.017200 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:27:00.068682 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_54_54_0 (state Temporary) 2024.10.22 00:27:00.333288 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_54_54_0 to 202410_67_67_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:00.390106 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 784 2024.10.22 00:27:00.535646 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 6.05 MiB, will set to 1.43 GiB (RSS), difference: 8.58 MiB 2024.10.22 00:27:00.928177 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ezr/yybvfddfgzesqmbkyzofkzljhjxly. 2024.10.22 00:27:01.389073 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 1.81 GiB, free memory in arenas 5.66 MiB, will set to 1.43 GiB (RSS), difference: 45.48 MiB 2024.10.22 00:27:01.464904 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 29 entries to flush up to offset 4155 2024.10.22 00:27:01.659947 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:01.664163 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:27:01.692375 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_95_95_0 (state Temporary) 2024.10.22 00:27:01.840344 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_95_95_0 to 202410_108_108_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:01.975903 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4155 2024.10.22 00:27:02.416419 [ 1185 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:27:02.457283 [ 1185 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:27:02.485651 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 1.81 GiB, free memory in arenas 6.36 MiB, will set to 1.42 GiB (RSS), difference: 62.72 MiB 2024.10.22 00:27:03.233775 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:27:03.393249 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 61 2024.10.22 00:27:03.757005 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:03.762052 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:27:03.808259 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_43_43_0 (state Temporary) 2024.10.22 00:27:03.879801 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_43_43_0 to 202410_53_53_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:04.152443 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 61 2024.10.22 00:27:04.253643 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 1.81 GiB, free memory in arenas 7.72 MiB, will set to 1.42 GiB (RSS), difference: 63.35 MiB 2024.10.22 00:27:04.612769 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1938 entries to flush up to offset 94252 2024.10.22 00:27:05.380713 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:05.383636 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:27:05.670743 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.81 GiB, free memory in arenas 6.81 MiB, will set to 1.41 GiB (RSS), difference: 2.77 MiB 2024.10.22 00:27:05.785828 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_53_53_0 (state Temporary) 2024.10.22 00:27:05.849243 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_53_53_0 to 202410_60_60_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:05.977828 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 24 entries to flush up to offset 4179 2024.10.22 00:27:06.009066 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 94252 2024.10.22 00:27:06.010139 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 19 entries to flush up to offset 4402 2024.10.22 00:27:06.014283 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_11_11_0 to 202410_14_14_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:06.068032 [ 1200 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_9_2 to 202410_14_14_0 2024.10.22 00:27:06.077340 [ 1200 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:06.183150 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:27:06.196993 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_9_2 to 202410_14_14_0 into Compact with storage Full 2024.10.22 00:27:06.237835 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:06.242481 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:27:06.278654 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_54_54_0 (state Temporary) 2024.10.22 00:27:06.316992 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:06.321200 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:27:06.413303 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_54_54_0 to 202410_60_60_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:06.422619 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_96_96_0 (state Temporary) 2024.10.22 00:27:06.464242 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4402 2024.10.22 00:27:06.466867 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_96_96_0 to 202410_109_109_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:06.502119 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4179 2024.10.22 00:27:06.552748 [ 1206 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_55_11 to 202410_60_60_0 2024.10.22 00:27:06.567869 [ 1206 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:06.572050 [ 1206 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.57 GiB. 2024.10.22 00:27:06.808973 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_55_11 to 202410_60_60_0 into Compact with storage Full 2024.10.22 00:27:06.876959 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:27:06.891883 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_55_11, total 4335 rows starting from the beginning of the part 2024.10.22 00:27:07.008807 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_56_56_0, total 68 rows starting from the beginning of the part 2024.10.22 00:27:07.038252 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_57_57_0, total 28 rows starting from the beginning of the part 2024.10.22 00:27:07.068942 [ 1203 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 00:27:07.068541 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_58_58_0, total 19 rows starting from the beginning of the part 2024.10.22 00:27:07.181407 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_59_59_0, total 37 rows starting from the beginning of the part 2024.10.22 00:27:07.207417 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_60_60_0, total 19 rows starting from the beginning of the part 2024.10.22 00:27:07.213259 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:27:07.535303 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.81 GiB, free memory in arenas 8.32 MiB, will set to 1.41 GiB (RSS), difference: 1.20 MiB 2024.10.22 00:27:07.895265 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 791 2024.10.22 00:27:08.017262 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:08.020926 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.56 GiB. 2024.10.22 00:27:08.051829 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_55_55_0 (state Temporary) 2024.10.22 00:27:08.086903 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} MergeTask::MergeProjectionsStage: Merge sorted 4506 rows, containing 13 columns (13 merged, 0 gathered) in 1.493380651 sec., 3017.3151078277897 rows/sec., 1.27 MiB/sec. 2024.10.22 00:27:08.213599 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_55_55_0 to 202410_68_68_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:08.259639 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 791 2024.10.22 00:27:08.268190 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} MergedBlockOutputStream: filled checksums 202410_1_60_12 (state Temporary) 2024.10.22 00:27:08.427208 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_60_12 to 202410_1_60_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:08.432404 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_60_12} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_55_11, 202410_60_60_0] -> 202410_1_60_12 2024.10.22 00:27:08.481141 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.04 MiB. 2024.10.22 00:27:08.604778 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:27:08.766710 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 413 2024.10.22 00:27:08.945149 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_9_2, total 274 rows starting from the beginning of the part 2024.10.22 00:27:09.115478 [ 1235 ] {} DatabaseCatalog: Execution took 220 ms. 2024.10.22 00:27:09.356203 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:27:09.361505 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:27:09.677612 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.81 GiB, free memory in arenas 10.68 MiB, will set to 1.41 GiB (RSS), difference: 11.20 MiB 2024.10.22 00:27:10.508095 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 4229 2024.10.22 00:27:10.966129 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_10_10_0, total 42 rows starting from the beginning of the part 2024.10.22 00:27:11.075521 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:11.078638 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.56 GiB. 2024.10.22 00:27:11.355563 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_97_97_0 (state Temporary) 2024.10.22 00:27:11.391680 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_97_97_0 to 202410_110_110_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:11.554514 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4229 2024.10.22 00:27:11.655386 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 62 2024.10.22 00:27:11.989787 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:11.995367 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.56 GiB. 2024.10.22 00:27:12.165081 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_44_44_0 (state Temporary) 2024.10.22 00:27:12.758527 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_44_44_0 to 202410_54_54_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:12.812832 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 62 2024.10.22 00:27:13.122514 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1902 entries to flush up to offset 96154 2024.10.22 00:27:13.410862 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_11_11_0, total 43 rows starting from the beginning of the part 2024.10.22 00:27:13.805325 [ 1240 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 248 ms. 2024.10.22 00:27:13.966372 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 48 entries to flush up to offset 4450 2024.10.22 00:27:14.099555 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:14.179901 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:14.203787 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.56 GiB. 2024.10.22 00:27:14.314976 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.56 GiB. 2024.10.22 00:27:14.338400 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_55_55_0 (state Temporary) 2024.10.22 00:27:14.411523 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_55_55_0 to 202410_61_61_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:14.428277 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_54_54_0 (state Temporary) 2024.10.22 00:27:14.467719 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4450 2024.10.22 00:27:14.480119 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_54_54_0 to 202410_61_61_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:14.498727 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_56_11 to 202410_61_61_0 2024.10.22 00:27:14.507755 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 96154 2024.10.22 00:27:14.529151 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:14.533772 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.56 GiB. 2024.10.22 00:27:14.566208 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_56_11 to 202410_61_61_0 into Compact with storage Full 2024.10.22 00:27:14.668891 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:27:14.672582 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} MergeTreeSequentialSource: Reading 14 marks from part 202410_1_56_11, total 100144 rows starting from the beginning of the part 2024.10.22 00:27:14.779953 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_57_57_0, total 2105 rows starting from the beginning of the part 2024.10.22 00:27:14.820072 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_58_58_0, total 1735 rows starting from the beginning of the part 2024.10.22 00:27:14.833272 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_59_59_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:27:14.854748 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_60_60_0, total 1938 rows starting from the beginning of the part 2024.10.22 00:27:14.864115 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_61_61_0, total 1902 rows starting from the beginning of the part 2024.10.22 00:27:15.360716 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_12_12_0, total 40 rows starting from the beginning of the part 2024.10.22 00:27:15.561296 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 4268 2024.10.22 00:27:15.604171 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 8.82 MiB, will set to 1.41 GiB (RSS), difference: -9.60 MiB 2024.10.22 00:27:15.691293 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:15.763671 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 803 2024.10.22 00:27:15.800049 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.55 GiB. 2024.10.22 00:27:15.877562 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:15.879706 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.55 GiB. 2024.10.22 00:27:15.976485 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_98_98_0 (state Temporary) 2024.10.22 00:27:16.034939 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_56_56_0 (state Temporary) 2024.10.22 00:27:16.036964 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_98_98_0 to 202410_111_111_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:16.080628 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_106_21 to 202410_111_111_0 2024.10.22 00:27:16.092165 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:16.102541 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.55 GiB. 2024.10.22 00:27:16.118223 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4268 2024.10.22 00:27:16.121819 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_106_21 to 202410_111_111_0 into Compact with storage Full 2024.10.22 00:27:16.176240 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:27:16.179420 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_106_21, total 4907 rows starting from the beginning of the part 2024.10.22 00:27:16.184859 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_56_56_0 to 202410_69_69_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:16.195101 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_107_107_0, total 50 rows starting from the beginning of the part 2024.10.22 00:27:16.213792 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_108_108_0, total 29 rows starting from the beginning of the part 2024.10.22 00:27:16.242495 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_109_109_0, total 24 rows starting from the beginning of the part 2024.10.22 00:27:16.259716 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_110_110_0, total 50 rows starting from the beginning of the part 2024.10.22 00:27:16.261597 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 803 2024.10.22 00:27:16.266255 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_111_111_0, total 39 rows starting from the beginning of the part 2024.10.22 00:27:16.459652 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 455 2024.10.22 00:27:17.072198 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_13_13_0, total 38 rows starting from the beginning of the part 2024.10.22 00:27:17.460747 [ 1261 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:27:17.483595 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} MergeTask::MergeProjectionsStage: Merge sorted 5099 rows, containing 13 columns (13 merged, 0 gathered) in 1.37263976 sec., 3714.7401296316816 rows/sec., 588.21 KiB/sec. 2024.10.22 00:27:17.499288 [ 1261 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:27:17.857495 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} MergedBlockOutputStream: filled checksums 202410_1_111_22 (state Temporary) 2024.10.22 00:27:17.929144 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_111_22 to 202410_1_111_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:17.932468 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_111_22} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_106_21, 202410_111_111_0] -> 202410_1_111_22 2024.10.22 00:27:17.977076 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:27:18.722917 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_14_14_0, total 38 rows starting from the beginning of the part 2024.10.22 00:27:19.299747 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_61_12/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:27:19.349802 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:27:19.352148 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:27:19.571365 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 13.68 MiB, will set to 1.42 GiB (RSS), difference: 7.83 MiB 2024.10.22 00:27:20.058693 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} MergeTask::MergeProjectionsStage: Merge sorted 109744 rows, containing 4 columns (4 merged, 0 gathered) in 5.503814723 sec., 19939.62470091674 rows/sec., 293.79 KiB/sec. 2024.10.22 00:27:20.087143 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} MergedBlockOutputStream: filled checksums 202410_1_61_12 (state Temporary) 2024.10.22 00:27:20.128919 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 4308 2024.10.22 00:27:20.254888 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_61_12 to 202410_1_61_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:20.260978 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_61_12} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_56_11, 202410_61_61_0] -> 202410_1_61_12 2024.10.22 00:27:20.286442 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:27:20.315342 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 63 2024.10.22 00:27:20.371235 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:20.376350 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.55 GiB. 2024.10.22 00:27:20.640344 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_99_99_0 (state Temporary) 2024.10.22 00:27:20.701495 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_99_99_0 to 202410_112_112_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:20.760591 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4308 2024.10.22 00:27:20.813082 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:20.816423 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.55 GiB. 2024.10.22 00:27:20.902616 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_45_45_0 (state Temporary) 2024.10.22 00:27:20.977706 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_45_45_0 to 202410_55_55_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:20.996428 [ 1156 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_50_10 to 202410_55_55_0 2024.10.22 00:27:21.013699 [ 1156 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:21.019277 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.55 GiB. 2024.10.22 00:27:21.035998 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 63 2024.10.22 00:27:21.036316 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_50_10 to 202410_55_55_0 into Compact with storage Full 2024.10.22 00:27:21.220424 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:27:21.240194 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_50_10, total 94 rows starting from the beginning of the part 2024.10.22 00:27:21.337509 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_51_51_0, total 1 rows starting from the beginning of the part 2024.10.22 00:27:21.403753 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_52_52_0, total 1 rows starting from the beginning of the part 2024.10.22 00:27:21.461001 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 10.89 MiB, will set to 1.42 GiB (RSS), difference: 5.24 MiB 2024.10.22 00:27:21.471064 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_53_53_0, total 1 rows starting from the beginning of the part 2024.10.22 00:27:21.512204 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 97690 2024.10.22 00:27:21.654216 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_54_54_0, total 1 rows starting from the beginning of the part 2024.10.22 00:27:21.709326 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_55_55_0, total 1 rows starting from the beginning of the part 2024.10.22 00:27:21.958243 [ 1159 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 222 ms. 2024.10.22 00:27:21.973616 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 4529 2024.10.22 00:27:22.120329 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:22.128386 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:22.359049 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_55_55_0 (state Temporary) 2024.10.22 00:27:22.376106 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:22.379804 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:22.631478 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_55_55_0 to 202410_62_62_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:22.647195 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_56_56_0 (state Temporary) 2024.10.22 00:27:22.672019 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 97690 2024.10.22 00:27:22.891651 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_56_56_0 to 202410_62_62_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:23.038268 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4529 2024.10.22 00:27:23.322873 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} MergeTask::MergeProjectionsStage: Merge sorted 99 rows, containing 32 columns (32 merged, 0 gathered) in 2.294324301 sec., 43.149959208839846 rows/sec., 6.06 KiB/sec. 2024.10.22 00:27:23.376608 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} MergedBlockOutputStream: filled checksums 202410_1_55_11 (state Temporary) 2024.10.22 00:27:23.460866 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.81 GiB, free memory in arenas 7.29 MiB, will set to 1.42 GiB (RSS), difference: -8.35 MiB 2024.10.22 00:27:23.514899 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_55_11 to 202410_1_55_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:23.517783 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_55_11} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_50_10, 202410_55_55_0] -> 202410_1_55_11 2024.10.22 00:27:23.684957 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:27:23.866455 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 830 2024.10.22 00:27:24.068656 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:24.071066 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:24.239826 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_57_57_0 (state Temporary) 2024.10.22 00:27:24.298147 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_57_57_0 to 202410_70_70_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:24.367673 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 830 2024.10.22 00:27:24.768309 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 4360 2024.10.22 00:27:24.945526 [ 1168 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 00:27:25.094826 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ntb/bxjaxbtvedbkcfcvsqexgzlnsmebt. 2024.10.22 00:27:25.238585 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nuo/ayzmhzhrerdinzqpjlarwqitqvhnu. 2024.10.22 00:27:25.344081 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:25.347975 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:25.410220 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_100_100_0 (state Temporary) 2024.10.22 00:27:25.461110 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 5.75 MiB, will set to 1.42 GiB (RSS), difference: -1.14 MiB 2024.10.22 00:27:25.559347 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_100_100_0 to 202410_113_113_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:25.848811 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4360 2024.10.22 00:27:25.972849 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nuo/ayzmhzhrerdinzqpjlarwqitqvhnu. 2024.10.22 00:27:25.977029 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_3_3_0 to 201403_3_3_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:25.987400 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ntb/bxjaxbtvedbkcfcvsqexgzlnsmebt. 2024.10.22 00:27:26.023887 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_4_4_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:27:26.521634 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 966.39 MiB, peak 1.81 GiB, free memory in arenas 4.63 MiB, will set to 1.42 GiB (RSS), difference: 485.41 MiB 2024.10.22 00:27:28.386830 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 1.81 GiB, free memory in arenas 4.48 MiB, will set to 1.09 GiB (RSS), difference: -278.61 MiB 2024.10.22 00:27:29.353739 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:27:29.367725 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:27:29.681597 [ 1209 ] {} DatabaseCatalog: Execution took 377 ms. 2024.10.22 00:27:29.821608 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 99610 2024.10.22 00:27:29.863732 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_14_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:27:29.871826 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 20 entries to flush up to offset 4380 2024.10.22 00:27:30.171909 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:30.174912 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:30.216212 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_101_101_0 (state Temporary) 2024.10.22 00:27:30.367694 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_101_101_0 to 202410_114_114_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:30.496588 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4380 2024.10.22 00:27:30.559928 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 54 entries to flush up to offset 4583 2024.10.22 00:27:30.621058 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:30.625371 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:30.764749 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:30.767175 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:30.776498 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.17 GiB, peak 1.81 GiB, free memory in arenas 3.82 MiB, will set to 1.12 GiB (RSS), difference: -58.85 MiB 2024.10.22 00:27:30.822953 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_57_57_0 (state Temporary) 2024.10.22 00:27:30.865630 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_57_57_0 to 202410_63_63_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:30.980944 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_56_56_0 (state Temporary) 2024.10.22 00:27:31.014968 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4583 2024.10.22 00:27:31.160493 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_56_56_0 to 202410_63_63_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:31.287036 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 99610 2024.10.22 00:27:31.870267 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 836 2024.10.22 00:27:32.385225 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:32.388257 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:32.465472 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.81 GiB, free memory in arenas 4.66 MiB, will set to 1.15 GiB (RSS), difference: -1.85 MiB 2024.10.22 00:27:32.505727 [ 1183 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:27:32.541612 [ 1183 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:27:32.547655 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_58_58_0 (state Temporary) 2024.10.22 00:27:32.596809 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_58_58_0 to 202410_71_71_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:32.847472 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 836 2024.10.22 00:27:32.849203 [ 1211 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_66_13 to 202410_71_71_0 2024.10.22 00:27:32.863557 [ 1211 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:32.866809 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:32.875646 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_66_13 to 202410_71_71_0 into Compact with storage Full 2024.10.22 00:27:32.892620 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 3 2024.10.22 00:27:32.898751 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:27:32.908321 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_66_13, total 870 rows starting from the beginning of the part 2024.10.22 00:27:33.099230 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_67_67_0, total 20 rows starting from the beginning of the part 2024.10.22 00:27:33.109635 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_68_68_0, total 7 rows starting from the beginning of the part 2024.10.22 00:27:33.136428 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_69_69_0, total 12 rows starting from the beginning of the part 2024.10.22 00:27:33.178696 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_70_70_0, total 27 rows starting from the beginning of the part 2024.10.22 00:27:33.208258 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_71_71_0, total 6 rows starting from the beginning of the part 2024.10.22 00:27:33.379703 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:33.382113 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:33.446396 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_3_3_0 (state Temporary) 2024.10.22 00:27:33.639550 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_3_3_0 to 202410_3_3_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:33.698237 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 3 2024.10.22 00:27:33.772682 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:33.774828 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:34.027223 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} MergeTask::MergeProjectionsStage: Merge sorted 942 rows, containing 12 columns (12 merged, 0 gathered) in 1.044052595 sec., 902.2533965350664 rows/sec., 137.73 KiB/sec. 2024.10.22 00:27:34.080082 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} MergedBlockOutputStream: filled checksums 202410_1_71_14 (state Temporary) 2024.10.22 00:27:34.185824 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_71_14 to 202410_1_71_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:34.188821 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_71_14} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_66_13, 202410_71_71_0] -> 202410_1_71_14 2024.10.22 00:27:34.337024 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:27:34.508935 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 4432 2024.10.22 00:27:34.866209 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:34.867595 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:35.020556 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_102_102_0 (state Temporary) 2024.10.22 00:27:35.069600 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_102_102_0 to 202410_115_115_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:35.163200 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4432 2024.10.22 00:27:35.345231 [ 1228 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 00:27:35.594319 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.21 GiB, peak 1.81 GiB, free memory in arenas 3.57 MiB, will set to 1.20 GiB (RSS), difference: -11.68 MiB 2024.10.22 00:27:36.040776 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 64 2024.10.22 00:27:36.191543 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:36.194796 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:36.371222 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_46_46_0 (state Temporary) 2024.10.22 00:27:36.556882 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_46_46_0 to 202410_56_56_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:36.596301 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 64 2024.10.22 00:27:37.111768 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_12_12_0 (state Temporary) 2024.10.22 00:27:37.671259 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 1.81 GiB, free memory in arenas 5.80 MiB, will set to 1.24 GiB (RSS), difference: -12.98 MiB 2024.10.22 00:27:38.339757 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 101530 2024.10.22 00:27:38.532564 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 43 entries to flush up to offset 4626 2024.10.22 00:27:38.756557 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:38.760070 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:38.840458 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_58_58_0 (state Temporary) 2024.10.22 00:27:38.887490 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:38.899369 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:38.979733 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_58_58_0 to 202410_64_64_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:39.022287 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_57_57_0 (state Temporary) 2024.10.22 00:27:39.045380 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4626 2024.10.22 00:27:39.081481 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_57_57_0 to 202410_64_64_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:39.117838 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 101530 2024.10.22 00:27:39.271650 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 28 entries to flush up to offset 4460 2024.10.22 00:27:39.352947 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:27:39.355305 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:27:39.581706 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:39.584729 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:39.650745 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_103_103_0 (state Temporary) 2024.10.22 00:27:39.651504 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 1.81 GiB, free memory in arenas 6.11 MiB, will set to 1.30 GiB (RSS), difference: 5.29 MiB 2024.10.22 00:27:39.819284 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_103_103_0 to 202410_116_116_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:39.863760 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_111_22 to 202410_116_116_0 2024.10.22 00:27:39.883918 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:39.889014 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:39.971318 [ 1260 ] {} DatabaseCatalog: Execution took 201 ms. 2024.10.22 00:27:39.994561 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4460 2024.10.22 00:27:40.041735 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_111_22 to 202410_116_116_0 into Compact with storage Full 2024.10.22 00:27:40.077153 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:27:40.185364 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_111_22, total 5099 rows starting from the beginning of the part 2024.10.22 00:27:40.206432 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_112_112_0, total 40 rows starting from the beginning of the part 2024.10.22 00:27:40.224687 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_113_113_0, total 52 rows starting from the beginning of the part 2024.10.22 00:27:40.348280 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_114_114_0, total 20 rows starting from the beginning of the part 2024.10.22 00:27:40.351023 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 850 2024.10.22 00:27:40.368042 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_115_115_0, total 52 rows starting from the beginning of the part 2024.10.22 00:27:40.382408 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_116_116_0, total 28 rows starting from the beginning of the part 2024.10.22 00:27:40.453393 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:40.455609 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:40.477014 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_59_59_0 (state Temporary) 2024.10.22 00:27:40.528140 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_59_59_0 to 202410_72_72_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:40.577416 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 850 2024.10.22 00:27:41.606497 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} MergeTask::MergeProjectionsStage: Merge sorted 5291 rows, containing 13 columns (13 merged, 0 gathered) in 1.577116935 sec., 3354.8558655227425 rows/sec., 529.40 KiB/sec. 2024.10.22 00:27:41.663038 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.81 GiB, free memory in arenas 5.83 MiB, will set to 1.34 GiB (RSS), difference: -7.12 MiB 2024.10.22 00:27:41.882991 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} MergedBlockOutputStream: filled checksums 202410_1_116_23 (state Temporary) 2024.10.22 00:27:42.056910 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_116_23 to 202410_1_116_23 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:42.059425 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_116_23} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_111_22, 202410_116_116_0] -> 202410_1_116_23 2024.10.22 00:27:42.090362 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:27:43.711475 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.81 GiB, free memory in arenas 5.38 MiB, will set to 1.38 GiB (RSS), difference: -108.85 MiB 2024.10.22 00:27:44.006983 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 4494 2024.10.22 00:27:44.098146 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 65 2024.10.22 00:27:44.348086 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:44.352113 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:44.583366 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_104_104_0 (state Temporary) 2024.10.22 00:27:44.599013 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_12_12_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:27:44.655350 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_104_104_0 to 202410_117_117_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:44.663241 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:44.665004 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:44.785630 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4494 2024.10.22 00:27:44.823373 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_47_47_0 (state Temporary) 2024.10.22 00:27:45.111313 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_47_47_0 to 202410_57_57_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:45.290912 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_4_4_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:27:45.439357 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 65 2024.10.22 00:27:45.518198 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 5.39 MiB, will set to 1.41 GiB (RSS), difference: -12.81 MiB 2024.10.22 00:27:46.123891 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1458 entries to flush up to offset 102988 2024.10.22 00:27:46.561211 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 43 entries to flush up to offset 4669 2024.10.22 00:27:46.611177 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_12_12_0 to 202410_15_15_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:46.669126 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:46.671944 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:46.718164 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_59_59_0 (state Temporary) 2024.10.22 00:27:46.752171 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:46.755467 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:46.872117 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_59_59_0 to 202410_65_65_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:46.934600 [ 1178 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_60_12 to 202410_65_65_0 2024.10.22 00:27:46.942912 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4669 2024.10.22 00:27:46.953353 [ 1178 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:46.959556 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:46.982962 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_60_12 to 202410_65_65_0 into Compact with storage Full 2024.10.22 00:27:47.045675 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:27:47.057395 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_60_12, total 4506 rows starting from the beginning of the part 2024.10.22 00:27:47.072153 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_61_61_0, total 48 rows starting from the beginning of the part 2024.10.22 00:27:47.087138 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_62_62_0, total 79 rows starting from the beginning of the part 2024.10.22 00:27:47.111052 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_58_58_0 (state Temporary) 2024.10.22 00:27:47.117796 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_63_63_0, total 54 rows starting from the beginning of the part 2024.10.22 00:27:47.147057 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_64_64_0, total 43 rows starting from the beginning of the part 2024.10.22 00:27:47.166421 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_65_65_0, total 43 rows starting from the beginning of the part 2024.10.22 00:27:47.237161 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_58_58_0 to 202410_65_65_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:47.272480 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 102988 2024.10.22 00:27:47.554284 [ 1185 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:27:47.675775 [ 1185 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:27:47.784951 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.81 GiB, free memory in arenas 7.95 MiB, will set to 1.50 GiB (RSS), difference: -56.32 MiB 2024.10.22 00:27:48.078681 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 863 2024.10.22 00:27:48.124750 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} MergeTask::MergeProjectionsStage: Merge sorted 4773 rows, containing 13 columns (13 merged, 0 gathered) in 1.149762892 sec., 4151.2906993349025 rows/sec., 1.74 MiB/sec. 2024.10.22 00:27:48.377623 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:48.380964 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:48.430755 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_60_60_0 (state Temporary) 2024.10.22 00:27:48.479581 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_60_60_0 to 202410_73_73_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:48.487848 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} MergedBlockOutputStream: filled checksums 202410_1_65_13 (state Temporary) 2024.10.22 00:27:48.547180 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 863 2024.10.22 00:27:48.753312 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_65_13 to 202410_1_65_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:48.758750 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_65_13} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_60_12, 202410_65_65_0] -> 202410_1_65_13 2024.10.22 00:27:48.779885 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.22 MiB. 2024.10.22 00:27:48.797293 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 4547 2024.10.22 00:27:49.206942 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:49.210977 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:49.273889 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_105_105_0 (state Temporary) 2024.10.22 00:27:49.352092 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:27:49.354883 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:27:49.441354 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_105_105_0 to 202410_118_118_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:49.494761 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4547 2024.10.22 00:27:49.760806 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 9.78 MiB, will set to 1.55 GiB (RSS), difference: -38.72 MiB 2024.10.22 00:27:49.779739 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 455 2024.10.22 00:27:51.752137 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.81 GiB, free memory in arenas 11.62 MiB, will set to 1.60 GiB (RSS), difference: -48.52 MiB 2024.10.22 00:27:52.945337 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 66 2024.10.22 00:27:53.087231 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:53.089287 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:53.263199 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 12.53 MiB, will set to 1.61 GiB (RSS), difference: 26.73 MiB 2024.10.22 00:27:53.277803 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 246.38 MiB using storage policy from min volume index 0 2024.10.22 00:27:53.360795 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_48_48_0 (state Temporary) 2024.10.22 00:27:53.506947 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 17 entries to flush up to offset 4564 2024.10.22 00:27:53.638688 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_48_48_0 to 202410_58_58_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:53.868132 [ 1207 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 213 ms. 2024.10.22 00:27:53.877244 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 66 2024.10.22 00:27:53.885189 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:53.887865 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:54.025528 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_106_106_0 (state Temporary) 2024.10.22 00:27:54.182436 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fqd/pomcijlwqluvxjwtfdoletcbymlpl (LogSeriesLimiter: on interval from 2024-10-22 00:26:25 to 2024-10-22 00:27:54 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:27:54.186673 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_106_106_0 to 202410_119_119_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:54.274448 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 11.88 MiB, will set to 1.61 GiB (RSS), difference: -3.14 MiB 2024.10.22 00:27:54.276310 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1614 entries to flush up to offset 104602 2024.10.22 00:27:54.366914 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4564 2024.10.22 00:27:54.454579 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 45 entries to flush up to offset 4714 2024.10.22 00:27:54.681395 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:54.683784 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:54.823058 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_60_60_0 (state Temporary) 2024.10.22 00:27:54.964851 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_60_60_0 to 202410_66_66_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:55.102950 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4714 2024.10.22 00:27:55.178938 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:55.181700 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:55.398889 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_59_59_0 (state Temporary) 2024.10.22 00:27:55.484643 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.81 GiB, free memory in arenas 11.17 MiB, will set to 1.61 GiB (RSS), difference: -126.64 MiB 2024.10.22 00:27:55.798031 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_59_59_0 to 202410_66_66_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:55.828197 [ 1211 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_61_12 to 202410_66_66_0 2024.10.22 00:27:55.841111 [ 1211 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:55.844215 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:55.866757 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_61_12 to 202410_66_66_0 into Compact with storage Full 2024.10.22 00:27:55.883587 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:27:55.887222 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} MergeTreeSequentialSource: Reading 15 marks from part 202410_1_61_12, total 109744 rows starting from the beginning of the part 2024.10.22 00:27:55.909100 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_62_62_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:27:55.923356 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_63_63_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:27:55.942506 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_64_64_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:27:55.949440 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 104602 2024.10.22 00:27:55.957408 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_65_65_0, total 1458 rows starting from the beginning of the part 2024.10.22 00:27:55.972394 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_66_66_0, total 1614 rows starting from the beginning of the part 2024.10.22 00:27:56.051534 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 876 2024.10.22 00:27:56.259023 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:56.261101 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.53 GiB. 2024.10.22 00:27:56.282058 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_61_61_0 (state Temporary) 2024.10.22 00:27:56.560825 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_61_61_0 to 202410_74_74_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:56.584185 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 876 2024.10.22 00:27:56.710970 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_14_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:27:57.272289 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.81 GiB, free memory in arenas 10.93 MiB, will set to 1.61 GiB (RSS), difference: -119.11 MiB 2024.10.22 00:27:57.325262 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 496 2024.10.22 00:27:57.784086 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_66_13/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:27:58.262721 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.81 GiB, free memory in arenas 16.81 MiB, will set to 1.62 GiB (RSS), difference: -119.43 MiB 2024.10.22 00:27:58.370590 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 4610 2024.10.22 00:27:58.711460 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:27:58.725365 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:27:58.781174 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_107_107_0 (state Temporary) 2024.10.22 00:27:58.822890 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_107_107_0 to 202410_120_120_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:27:58.873188 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4610 2024.10.22 00:27:59.158047 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rwd/tvfhzvwufstoolmjlbiaskycpoovl (LogSeriesLimiter: on interval from 2024-10-22 00:27:54 to 2024-10-22 00:27:59 accepted series 1 / 275 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:27:59.456095 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:27:59.470308 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.81 GiB, free memory in arenas 19.89 MiB, will set to 1.62 GiB (RSS), difference: -84.77 MiB 2024.10.22 00:27:59.453817 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:28:00.379973 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} MergeTask::MergeProjectionsStage: Merge sorted 118192 rows, containing 4 columns (4 merged, 0 gathered) in 4.517978811 sec., 26160.370587005837 rows/sec., 385.03 KiB/sec. 2024.10.22 00:28:00.434561 [ 1232 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 00:28:00.543662 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} MergedBlockOutputStream: filled checksums 202410_1_66_13 (state Temporary) 2024.10.22 00:28:00.575982 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_66_13 to 202410_1_66_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:00.579683 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_66_13} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_61_12, 202410_66_66_0] -> 202410_1_66_13 2024.10.22 00:28:00.621431 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:28:01.228828 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.81 GiB, free memory in arenas 23.84 MiB, will set to 1.63 GiB (RSS), difference: -80.20 MiB 2024.10.22 00:28:01.379082 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 67 2024.10.22 00:28:01.663515 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:01.664628 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:01.703965 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_49_49_0 (state Temporary) 2024.10.22 00:28:01.767453 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_11_2, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0] 2024.10.22 00:28:01.769151 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_11_2, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0] 2024.10.22 00:28:01.776171 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_49_49_0 to 202410_59_59_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:01.788743 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_11_2, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0] 2024.10.22 00:28:01.903584 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 67 2024.10.22 00:28:02.319279 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.81 GiB, free memory in arenas 22.13 MiB, will set to 1.63 GiB (RSS), difference: -2.94 MiB 2024.10.22 00:28:02.500735 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.040669429 sec. 2024.10.22 00:28:02.613381 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 4764 2024.10.22 00:28:02.678220 [ 1259 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:28:02.696346 [ 1259 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:28:02.717408 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:02.824111 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:02.865840 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_61_61_0 (state Temporary) 2024.10.22 00:28:02.877893 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 4644 2024.10.22 00:28:02.910527 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_61_61_0 to 202410_67_67_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:02.956011 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2593 entries to flush up to offset 107195 2024.10.22 00:28:03.057786 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4764 2024.10.22 00:28:03.176514 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:03.260400 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:03.397102 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_108_108_0 (state Temporary) 2024.10.22 00:28:03.444740 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_108_108_0 to 202410_121_121_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:03.488516 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4644 2024.10.22 00:28:03.589712 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_116_23 to 202410_121_121_0 2024.10.22 00:28:03.610702 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:03.614960 [ 1264 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:03.745276 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_116_23 to 202410_121_121_0 into Compact with storage Full 2024.10.22 00:28:03.766100 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:28:03.770380 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_116_23, total 5291 rows starting from the beginning of the part 2024.10.22 00:28:03.778325 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_117_117_0, total 34 rows starting from the beginning of the part 2024.10.22 00:28:03.788429 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_118_118_0, total 53 rows starting from the beginning of the part 2024.10.22 00:28:03.813967 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_119_119_0, total 17 rows starting from the beginning of the part 2024.10.22 00:28:03.839190 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_120_120_0, total 46 rows starting from the beginning of the part 2024.10.22 00:28:03.873802 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_121_121_0, total 34 rows starting from the beginning of the part 2024.10.22 00:28:04.085774 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 888 2024.10.22 00:28:04.214665 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:04.219243 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:04.264275 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_62_62_0 (state Temporary) 2024.10.22 00:28:04.312004 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_62_62_0 to 202410_75_75_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:04.318320 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:04.332226 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:04.367967 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 888 2024.10.22 00:28:04.524093 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.81 GiB, free memory in arenas 19.29 MiB, will set to 1.64 GiB (RSS), difference: -1.54 MiB 2024.10.22 00:28:04.763033 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_60_60_0 (state Temporary) 2024.10.22 00:28:05.062434 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_60_60_0 to 202410_67_67_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:05.083211 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 107195 2024.10.22 00:28:05.359222 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} MergeTask::MergeProjectionsStage: Merge sorted 5475 rows, containing 13 columns (13 merged, 0 gathered) in 1.734237361 sec., 3157.0072950354343 rows/sec., 496.93 KiB/sec. 2024.10.22 00:28:05.361844 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_121_24/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:28:05.431318 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} MergedBlockOutputStream: filled checksums 202410_1_121_24 (state Temporary) 2024.10.22 00:28:05.615978 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:28:05.810888 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_121_24 to 202410_1_121_24 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:05.816585 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_121_24} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_116_23, 202410_121_121_0] -> 202410_1_121_24 2024.10.22 00:28:05.869072 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:28:06.656872 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.81 GiB, free memory in arenas 17.84 MiB, will set to 1.64 GiB (RSS), difference: 5.01 MiB 2024.10.22 00:28:07.501585 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 4684 2024.10.22 00:28:07.757996 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:07.762271 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:07.816661 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_109_109_0 (state Temporary) 2024.10.22 00:28:07.978013 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_109_109_0 to 202410_122_122_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:08.312471 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4684 2024.10.22 00:28:08.412379 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.81 GiB, free memory in arenas 14.31 MiB, will set to 1.65 GiB (RSS), difference: -3.45 MiB 2024.10.22 00:28:09.258416 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.81 GiB, free memory in arenas 12.41 MiB, will set to 1.66 GiB (RSS), difference: 4.49 MiB 2024.10.22 00:28:09.454464 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:28:09.457146 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:28:10.567950 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 40 entries to flush up to offset 4804 2024.10.22 00:28:10.676688 [ 1180 ] {} DatabaseCatalog: Execution took 398 ms. 2024.10.22 00:28:10.768287 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:10.771485 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:10.812191 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_62_62_0 (state Temporary) 2024.10.22 00:28:10.860666 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_62_62_0 to 202410_68_68_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:10.877094 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_11_2, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0] 2024.10.22 00:28:10.879659 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_11_2, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0] 2024.10.22 00:28:10.920930 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_11_2, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0] 2024.10.22 00:28:10.940822 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4804 2024.10.22 00:28:11.528484 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.81 GiB, free memory in arenas 11.23 MiB, will set to 1.66 GiB (RSS), difference: 4.24 MiB 2024.10.22 00:28:11.870704 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 899 2024.10.22 00:28:12.089065 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2003 entries to flush up to offset 109198 2024.10.22 00:28:12.091598 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:12.098919 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:12.173702 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_63_63_0 (state Temporary) 2024.10.22 00:28:12.230625 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_63_63_0 to 202410_76_76_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:12.280144 [ 1189 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_71_14 to 202410_76_76_0 2024.10.22 00:28:12.294045 [ 1189 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:12.299295 [ 1189 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:12.312635 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 899 2024.10.22 00:28:12.315944 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 4714 2024.10.22 00:28:12.415208 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_71_14 to 202410_76_76_0 into Compact with storage Full 2024.10.22 00:28:12.545445 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:28:12.562461 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_71_14, total 942 rows starting from the beginning of the part 2024.10.22 00:28:12.572525 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_72_72_0, total 14 rows starting from the beginning of the part 2024.10.22 00:28:12.586856 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_73_73_0, total 13 rows starting from the beginning of the part 2024.10.22 00:28:12.614754 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_74_74_0, total 13 rows starting from the beginning of the part 2024.10.22 00:28:12.636022 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_14_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:28:12.521020 [ 1196 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 211 ms. 2024.10.22 00:28:12.654090 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_75_75_0, total 12 rows starting from the beginning of the part 2024.10.22 00:28:12.682632 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_76_76_0, total 11 rows starting from the beginning of the part 2024.10.22 00:28:12.718453 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:12.721572 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:12.766947 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_110_110_0 (state Temporary) 2024.10.22 00:28:12.913786 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_110_110_0 to 202410_123_123_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:12.974913 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4714 2024.10.22 00:28:13.079419 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:13.081754 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:13.276936 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} MergeTask::MergeProjectionsStage: Merge sorted 475 rows, containing 679 columns (679 merged, 0 gathered) in 67.085534919 sec., 7.080512968608234 rows/sec., 8.58 KiB/sec. 2024.10.22 00:28:13.295125 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_61_61_0 (state Temporary) 2024.10.22 00:28:13.346272 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.81 GiB, free memory in arenas 9.12 MiB, will set to 1.66 GiB (RSS), difference: 1.09 MiB 2024.10.22 00:28:13.571445 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_61_61_0 to 202410_68_68_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:13.628622 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 109198 2024.10.22 00:28:14.170583 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} MergeTask::MergeProjectionsStage: Merge sorted 1005 rows, containing 12 columns (12 merged, 0 gathered) in 1.760142125 sec., 570.976619288627 rows/sec., 87.26 KiB/sec. 2024.10.22 00:28:14.270291 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} MergedBlockOutputStream: filled checksums 202410_1_76_15 (state Temporary) 2024.10.22 00:28:14.372861 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:14.376389 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:14.482561 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_76_15 to 202410_1_76_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:14.488099 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_76_15} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_71_14, 202410_76_76_0] -> 202410_1_76_15 2024.10.22 00:28:14.836932 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:28:14.879962 [ 1202 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_55_11 to 202410_59_59_0 2024.10.22 00:28:14.892109 [ 1202 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:14.899066 [ 1202 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:14.932577 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} MergedBlockOutputStream: filled checksums 202410_1_14_3 (state Temporary) 2024.10.22 00:28:15.059645 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_59_12} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_55_11 to 202410_59_59_0 into Compact with storage Full 2024.10.22 00:28:15.239051 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_59_12} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:28:15.265060 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_59_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_55_11, total 99 rows starting from the beginning of the part 2024.10.22 00:28:15.329329 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_59_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_56_56_0, total 1 rows starting from the beginning of the part 2024.10.22 00:28:15.364556 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.81 GiB, free memory in arenas 8.69 MiB, will set to 1.67 GiB (RSS), difference: 13.35 MiB 2024.10.22 00:28:15.375519 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_59_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_57_57_0, total 1 rows starting from the beginning of the part 2024.10.22 00:28:15.364760 [ 1201 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 00:28:15.543132 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_59_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_58_58_0, total 1 rows starting from the beginning of the part 2024.10.22 00:28:15.580730 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_59_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_59_59_0, total 1 rows starting from the beginning of the part 2024.10.22 00:28:16.417586 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.81 GiB, free memory in arenas 8.30 MiB, will set to 1.67 GiB (RSS), difference: -9.69 MiB 2024.10.22 00:28:16.906433 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 68 2024.10.22 00:28:16.978752 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 4760 2024.10.22 00:28:17.087580 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:17.093257 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.54 GiB. 2024.10.22 00:28:17.172675 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_50_50_0 (state Temporary) 2024.10.22 00:28:17.362882 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_59_12} MergeTask::MergeProjectionsStage: Merge sorted 103 rows, containing 32 columns (32 merged, 0 gathered) in 2.312612975 sec., 44.53836466086592 rows/sec., 6.14 KiB/sec. 2024.10.22 00:28:17.380960 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:17.385384 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.53 GiB. 2024.10.22 00:28:17.392730 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_59_12} MergedBlockOutputStream: filled checksums 202410_1_59_12 (state Temporary) 2024.10.22 00:28:17.413785 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_6_1, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0, 202410_10_10_0, 202410_11_11_0] 2024.10.22 00:28:17.527645 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_6_1, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0, 202410_10_10_0, 202410_11_11_0] 2024.10.22 00:28:17.555476 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_111_111_0 (state Temporary) 2024.10.22 00:28:17.562193 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_50_50_0 to 202410_60_60_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:17.571603 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_6_1, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0, 202410_10_10_0, 202410_11_11_0] 2024.10.22 00:28:17.579064 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_59_12} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_59_12 to 202410_1_59_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:17.584818 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_59_12} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_55_11, 202410_59_59_0] -> 202410_1_59_12 2024.10.22 00:28:17.616466 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_111_111_0 to 202410_124_124_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:17.682313 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 68 2024.10.22 00:28:17.713211 [ 1222 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:28:17.763705 [ 1222 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:28:17.767666 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4760 2024.10.22 00:28:17.821770 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:28:17.977476 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_5_1, 202410_6_6_0, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0, 202410_10_10_0] 2024.10.22 00:28:17.979491 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_5_1, 202410_6_6_0, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0, 202410_10_10_0] 2024.10.22 00:28:18.098340 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_5_1, 202410_6_6_0, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0, 202410_10_10_0] 2024.10.22 00:28:18.191618 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_13_13_0 (state Temporary) 2024.10.22 00:28:18.654705 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 64 entries to flush up to offset 4868 2024.10.22 00:28:18.774501 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 4.20 MiB, will set to 1.66 GiB (RSS), difference: 104.62 MiB 2024.10.22 00:28:19.075802 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:19.079246 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.52 GiB. 2024.10.22 00:28:19.171924 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_63_63_0 (state Temporary) 2024.10.22 00:28:19.247546 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_63_63_0 to 202410_69_69_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:19.363258 [ 1685 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rwd/tvfhzvwufstoolmjlbiaskycpoovl, size 117 2024.10.22 00:28:19.391546 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4868 2024.10.22 00:28:19.456768 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:28:19.461048 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:28:19.826872 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 918 2024.10.22 00:28:19.971297 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:19.974233 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.51 GiB. 2024.10.22 00:28:20.008651 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_64_64_0 (state Temporary) 2024.10.22 00:28:20.221769 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_64_64_0 to 202410_77_77_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:20.444111 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 918 2024.10.22 00:28:20.508759 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.81 GiB, free memory in arenas 4.15 MiB, will set to 1.65 GiB (RSS), difference: 150.54 MiB 2024.10.22 00:28:20.651659 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1548 entries to flush up to offset 110746 2024.10.22 00:28:20.776185 [ 1676 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:28:20.810197 [ 1706 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:28:20.903340 [ 1224 ] {} DatabaseCatalog: Execution took 220 ms. 2024.10.22 00:28:21.159419 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_5_5_0 (state Temporary) 2024.10.22 00:28:21.175035 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wmz/npkvrwtmdxyhkbsgptodrpfthhcnh (LogSeriesLimiter: on interval from 2024-10-22 00:27:59 to 2024-10-22 00:28:21 accepted series 1 / 84 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:28:21.239662 [ 1690 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wmz/npkvrwtmdxyhkbsgptodrpfthhcnh, size 4 2024.10.22 00:28:21.283342 [ 1707 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fqd/pomcijlwqluvxjwtfdoletcbymlpl, size 249 2024.10.22 00:28:21.775422 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 4810 2024.10.22 00:28:21.841806 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:21.854768 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:22.179360 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_62_62_0 (state Temporary) 2024.10.22 00:28:22.184912 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:22.191962 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:22.274393 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_14_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:28:22.352294 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_62_62_0 to 202410_69_69_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:22.355424 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_112_112_0 (state Temporary) 2024.10.22 00:28:22.395400 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 110746 2024.10.22 00:28:22.440786 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 4.90 MiB, will set to 1.62 GiB (RSS), difference: 60.96 MiB 2024.10.22 00:28:22.488234 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_112_112_0 to 202410_125_125_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:22.546454 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4810 2024.10.22 00:28:23.485120 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 4.76 MiB, will set to 1.62 GiB (RSS), difference: 5.07 MiB 2024.10.22 00:28:23.762882 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:28:24.044938 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_14_3 to 202410_1_14_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:24.053044 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_14_3} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_9_2, 202410_14_14_0] -> 202410_1_14_3 2024.10.22 00:28:24.125199 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kjh/bzivnfiqgcbdiwvewjlbnnguynseq. 2024.10.22 00:28:24.689808 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.60 MiB. 2024.10.22 00:28:25.183777 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 69 2024.10.22 00:28:25.362373 [ 1260 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 00:28:25.468904 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:25.476539 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:25.513502 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 5.32 MiB, will set to 1.62 GiB (RSS), difference: 5.23 MiB 2024.10.22 00:28:25.657982 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_51_51_0 (state Temporary) 2024.10.22 00:28:26.012628 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_51_51_0 to 202410_61_61_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:26.193320 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 69 2024.10.22 00:28:26.559627 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 4837 2024.10.22 00:28:26.884047 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:26.897509 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:26.913739 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 64 entries to flush up to offset 4932 2024.10.22 00:28:27.160857 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_113_113_0 (state Temporary) 2024.10.22 00:28:27.184792 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_113_113_0 to 202410_126_126_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:27.186736 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:27.197807 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:27.259540 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_121_24 to 202410_126_126_0 2024.10.22 00:28:27.267776 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4837 2024.10.22 00:28:27.269785 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:27.276143 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:27.313011 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_121_24 to 202410_126_126_0 into Compact with storage Full 2024.10.22 00:28:27.355616 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:28:27.364583 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_121_24, total 5475 rows starting from the beginning of the part 2024.10.22 00:28:27.368709 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_64_64_0 (state Temporary) 2024.10.22 00:28:27.372787 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_122_122_0, total 40 rows starting from the beginning of the part 2024.10.22 00:28:27.381062 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 3.81 MiB, will set to 1.62 GiB (RSS), difference: -5.05 MiB 2024.10.22 00:28:27.390326 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_123_123_0, total 30 rows starting from the beginning of the part 2024.10.22 00:28:27.411974 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_16_3, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 00:28:27.418606 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_16_3, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 00:28:27.428303 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_124_124_0, total 46 rows starting from the beginning of the part 2024.10.22 00:28:27.446527 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_125_125_0, total 50 rows starting from the beginning of the part 2024.10.22 00:28:27.472484 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_126_126_0, total 27 rows starting from the beginning of the part 2024.10.22 00:28:27.535296 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_64_64_0 to 202410_70_70_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:27.568429 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_16_3, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 00:28:27.599456 [ 1278 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_65_13 to 202410_70_70_0 2024.10.22 00:28:27.614180 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4932 2024.10.22 00:28:27.630294 [ 1278 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:27.640756 [ 1278 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:27.659726 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_65_13 to 202410_70_70_0 into Compact with storage Full 2024.10.22 00:28:27.693147 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:28:27.709118 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_65_13, total 4773 rows starting from the beginning of the part 2024.10.22 00:28:27.728834 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_66_66_0, total 45 rows starting from the beginning of the part 2024.10.22 00:28:27.854072 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_67_67_0, total 50 rows starting from the beginning of the part 2024.10.22 00:28:27.869098 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_68_68_0, total 40 rows starting from the beginning of the part 2024.10.22 00:28:27.882354 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_69_69_0, total 64 rows starting from the beginning of the part 2024.10.22 00:28:27.912595 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_70_70_0, total 64 rows starting from the beginning of the part 2024.10.22 00:28:27.915804 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_13_13_0 to 202410_16_16_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:27.951149 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 932 2024.10.22 00:28:28.073043 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:28.111112 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} MergeTask::MergeProjectionsStage: Merge sorted 5668 rows, containing 13 columns (13 merged, 0 gathered) in 0.811601836 sec., 6983.720031899977 rows/sec., 1.07 MiB/sec. 2024.10.22 00:28:28.187237 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:28.211848 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} MergedBlockOutputStream: filled checksums 202410_1_126_25 (state Temporary) 2024.10.22 00:28:28.232227 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_65_65_0 (state Temporary) 2024.10.22 00:28:28.379637 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_126_25 to 202410_1_126_25 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:28.382637 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_126_25} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_121_24, 202410_126_126_0] -> 202410_1_126_25 2024.10.22 00:28:28.421214 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_65_65_0 to 202410_78_78_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:28.428533 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:28:28.466632 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 932 2024.10.22 00:28:29.104707 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} MergeTask::MergeProjectionsStage: Merge sorted 5036 rows, containing 13 columns (13 merged, 0 gathered) in 1.450818321 sec., 3471.14447557352 rows/sec., 1.46 MiB/sec. 2024.10.22 00:28:29.366944 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} MergedBlockOutputStream: filled checksums 202410_1_70_14 (state Temporary) 2024.10.22 00:28:29.408857 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 112666 2024.10.22 00:28:29.461567 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:28:29.463926 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_70_14 to 202410_1_70_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:29.464005 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:28:29.468642 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_70_14} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_65_13, 202410_70_70_0] -> 202410_1_70_14 2024.10.22 00:28:29.510225 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.31 MiB. 2024.10.22 00:28:29.688342 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 4.16 MiB, will set to 1.62 GiB (RSS), difference: 13.55 MiB 2024.10.22 00:28:30.000855 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:30.005023 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:30.268305 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_63_63_0 (state Temporary) 2024.10.22 00:28:30.337803 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_63_63_0 to 202410_70_70_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:30.399781 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 496 2024.10.22 00:28:30.579228 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 112666 2024.10.22 00:28:31.270395 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 4902 2024.10.22 00:28:31.363589 [ 1167 ] {} DatabaseCatalog: Execution took 348 ms. 2024.10.22 00:28:31.496560 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:31.502544 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:31.571902 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_114_114_0 (state Temporary) 2024.10.22 00:28:31.611354 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_114_114_0 to 202410_127_127_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:31.627743 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 7.16 MiB, will set to 1.62 GiB (RSS), difference: -3.04 MiB 2024.10.22 00:28:31.658326 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4902 2024.10.22 00:28:32.769272 [ 1194 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:28:32.813372 [ 1194 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:28:33.696532 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 70 2024.10.22 00:28:33.868941 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:33.872185 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:34.043655 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_52_52_0 (state Temporary) 2024.10.22 00:28:34.458296 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 6.88 MiB, will set to 1.62 GiB (RSS), difference: -4.33 MiB 2024.10.22 00:28:34.635495 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_52_52_0 to 202410_62_62_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:34.700760 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 70 2024.10.22 00:28:35.178842 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 4982 2024.10.22 00:28:35.500031 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:35.511527 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:35.627072 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 4.87 MiB, will set to 1.62 GiB (RSS), difference: 3.30 MiB 2024.10.22 00:28:35.667246 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 21 entries to flush up to offset 4923 2024.10.22 00:28:35.768051 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_65_65_0 (state Temporary) 2024.10.22 00:28:35.793133 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_65_65_0 to 202410_71_71_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:35.831503 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 4982 2024.10.22 00:28:35.873428 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:35.876906 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:35.911176 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_115_115_0 (state Temporary) 2024.10.22 00:28:35.958733 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_115_115_0 to 202410_128_128_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:35.967865 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 952 2024.10.22 00:28:36.037639 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4923 2024.10.22 00:28:36.056283 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:36.060496 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:36.187760 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_66_66_0 (state Temporary) 2024.10.22 00:28:36.243833 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_66_66_0 to 202410_79_79_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:36.382981 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 952 2024.10.22 00:28:37.820231 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 114586 2024.10.22 00:28:38.069447 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 536 2024.10.22 00:28:38.909064 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:38.912941 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:39.417227 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_64_64_0 (state Temporary) 2024.10.22 00:28:39.460503 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:28:39.464668 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:28:39.568295 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_64_64_0 to 202410_71_71_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:39.594161 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 114586 2024.10.22 00:28:39.601296 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 3.61 MiB, will set to 1.61 GiB (RSS), difference: -1.49 MiB 2024.10.22 00:28:39.604825 [ 1213 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_66_13 to 202410_71_71_0 2024.10.22 00:28:39.629235 [ 1213 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:39.638555 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:39.662289 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_66_13 to 202410_71_71_0 into Compact with storage Full 2024.10.22 00:28:39.784494 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:28:39.788874 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} MergeTreeSequentialSource: Reading 16 marks from part 202410_1_66_13, total 118192 rows starting from the beginning of the part 2024.10.22 00:28:39.924105 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_67_67_0, total 2593 rows starting from the beginning of the part 2024.10.22 00:28:39.936420 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_68_68_0, total 2003 rows starting from the beginning of the part 2024.10.22 00:28:39.957327 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_69_69_0, total 1548 rows starting from the beginning of the part 2024.10.22 00:28:40.046088 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 4957 2024.10.22 00:28:40.067087 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_70_70_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:28:40.085701 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_71_71_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:28:40.181467 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:40.183516 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:40.233669 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_116_116_0 (state Temporary) 2024.10.22 00:28:40.266240 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_116_116_0 to 202410_129_129_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:40.312737 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4957 2024.10.22 00:28:41.360124 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 5.79 MiB, will set to 1.62 GiB (RSS), difference: -5.10 MiB 2024.10.22 00:28:41.499863 [ 1192 ] {} DatabaseCatalog: Execution took 233 ms. 2024.10.22 00:28:42.203490 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 71 2024.10.22 00:28:42.398047 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 8.66 MiB, will set to 1.62 GiB (RSS), difference: 4.30 MiB 2024.10.22 00:28:42.419784 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:42.428230 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:42.434693 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_71_14/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:28:42.495283 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_53_53_0 (state Temporary) 2024.10.22 00:28:42.902188 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_53_53_0 to 202410_63_63_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:43.087646 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 71 2024.10.22 00:28:43.344809 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 39 entries to flush up to offset 5021 2024.10.22 00:28:43.629152 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:43.633446 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:43.680602 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_66_66_0 (state Temporary) 2024.10.22 00:28:43.718673 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_66_66_0 to 202410_72_72_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:43.772129 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5021 2024.10.22 00:28:43.885325 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 957 2024.10.22 00:28:44.112758 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:44.115264 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:44.155667 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_67_67_0 (state Temporary) 2024.10.22 00:28:44.311753 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_67_67_0 to 202410_80_80_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:44.316195 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 29 entries to flush up to offset 4986 2024.10.22 00:28:44.413028 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_10_2, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0] 2024.10.22 00:28:44.416563 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_10_2, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0] 2024.10.22 00:28:44.449673 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 00:28:44.465586 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_10_2, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0] 2024.10.22 00:28:44.471690 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 957 2024.10.22 00:28:44.577802 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_21_4, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0, 202410_26_26_0] 2024.10.22 00:28:44.579412 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_21_4, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0, 202410_26_26_0] 2024.10.22 00:28:44.590281 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 12.93 MiB, will set to 1.62 GiB (RSS), difference: 3.38 MiB 2024.10.22 00:28:44.619180 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_21_4, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0, 202410_26_26_0] 2024.10.22 00:28:44.756461 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:44.760839 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:44.882764 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} MergeTask::MergeProjectionsStage: Merge sorted 128176 rows, containing 4 columns (4 merged, 0 gathered) in 5.228049405 sec., 24516.983308806357 rows/sec., 361.00 KiB/sec. 2024.10.22 00:28:44.902633 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_117_117_0 (state Temporary) 2024.10.22 00:28:44.955968 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} MergedBlockOutputStream: filled checksums 202410_1_71_14 (state Temporary) 2024.10.22 00:28:44.966737 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_117_117_0 to 202410_130_130_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:44.982831 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_71_14 to 202410_1_71_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:44.988643 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_71_14} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_66_13, 202410_71_71_0] -> 202410_1_71_14 2024.10.22 00:28:45.217798 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 4986 2024.10.22 00:28:45.259896 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:28:46.604951 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 116506 2024.10.22 00:28:46.689218 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 13.91 MiB, will set to 1.62 GiB (RSS), difference: 7.78 MiB 2024.10.22 00:28:47.569292 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:47.572303 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:47.774331 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_65_65_0 (state Temporary) 2024.10.22 00:28:47.818668 [ 1259 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:28:47.830725 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_65_65_0 to 202410_72_72_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:47.842187 [ 1259 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:28:47.863795 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 116506 2024.10.22 00:28:49.225358 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 29 entries to flush up to offset 5015 2024.10.22 00:28:49.266871 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:28:49.452254 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:49.460887 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:28:49.463823 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:28:49.467147 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:49.617308 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_118_118_0 (state Temporary) 2024.10.22 00:28:49.664139 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_118_118_0 to 202410_131_131_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:49.683157 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_126_25 to 202410_131_131_0 2024.10.22 00:28:49.706292 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:49.710344 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:49.751984 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_16_3, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 00:28:49.754522 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_16_3, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 00:28:49.823196 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5015 2024.10.22 00:28:49.830391 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_126_25 to 202410_131_131_0 into Compact with storage Full 2024.10.22 00:28:49.883745 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:28:49.895081 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_126_25, total 5668 rows starting from the beginning of the part 2024.10.22 00:28:49.920875 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_127_127_0, total 65 rows starting from the beginning of the part 2024.10.22 00:28:49.988082 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_16_3, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 00:28:50.151093 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_128_128_0, total 21 rows starting from the beginning of the part 2024.10.22 00:28:50.259417 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_129_129_0, total 34 rows starting from the beginning of the part 2024.10.22 00:28:50.274472 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_130_130_0, total 29 rows starting from the beginning of the part 2024.10.22 00:28:50.293622 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_131_131_0, total 29 rows starting from the beginning of the part 2024.10.22 00:28:50.590368 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 72 2024.10.22 00:28:50.718161 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.81 GiB, free memory in arenas 12.49 MiB, will set to 1.62 GiB (RSS), difference: -4.83 MiB 2024.10.22 00:28:50.860805 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:50.863844 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:51.012253 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_54_54_0 (state Temporary) 2024.10.22 00:28:51.080660 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_54_54_0 to 202410_64_64_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:51.118570 [ 1272 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_59_12 to 202410_64_64_0 2024.10.22 00:28:51.141422 [ 1272 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:51.152296 [ 1272 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:51.164196 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_59_12 to 202410_64_64_0 into Compact with storage Full 2024.10.22 00:28:51.168413 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 72 2024.10.22 00:28:51.212693 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:28:51.232736 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_59_12, total 103 rows starting from the beginning of the part 2024.10.22 00:28:51.266284 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_60_60_0, total 1 rows starting from the beginning of the part 2024.10.22 00:28:51.277766 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 34 entries to flush up to offset 5055 2024.10.22 00:28:51.384748 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_61_61_0, total 1 rows starting from the beginning of the part 2024.10.22 00:28:51.446503 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_62_62_0, total 1 rows starting from the beginning of the part 2024.10.22 00:28:51.480095 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_63_63_0, total 1 rows starting from the beginning of the part 2024.10.22 00:28:51.580802 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:51.583713 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:51.622594 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_67_67_0 (state Temporary) 2024.10.22 00:28:51.640548 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_64_64_0, total 1 rows starting from the beginning of the part 2024.10.22 00:28:51.865580 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_67_67_0 to 202410_73_73_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:51.953420 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5055 2024.10.22 00:28:51.973165 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 970 2024.10.22 00:28:52.077198 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:52.079125 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:52.113675 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_68_68_0 (state Temporary) 2024.10.22 00:28:52.159170 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} MergeTask::MergeProjectionsStage: Merge sorted 5846 rows, containing 13 columns (13 merged, 0 gathered) in 2.442093694 sec., 2393.8475474397583 rows/sec., 374.88 KiB/sec. 2024.10.22 00:28:52.172136 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_68_68_0 to 202410_81_81_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:52.267196 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 970 2024.10.22 00:28:52.272531 [ 1153 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_76_15 to 202410_81_81_0 2024.10.22 00:28:52.279610 [ 1153 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:52.281557 [ 1153 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:52.324353 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_76_15 to 202410_81_81_0 into Compact with storage Full 2024.10.22 00:28:52.362998 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} MergedBlockOutputStream: filled checksums 202410_1_131_26 (state Temporary) 2024.10.22 00:28:52.447839 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.81 GiB, free memory in arenas 10.63 MiB, will set to 1.62 GiB (RSS), difference: -4.05 MiB 2024.10.22 00:28:52.477335 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:28:52.500636 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_76_15, total 1005 rows starting from the beginning of the part 2024.10.22 00:28:52.604165 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_131_26 to 202410_1_131_26 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:52.618845 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_131_26} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_126_25, 202410_131_131_0] -> 202410_1_131_26 2024.10.22 00:28:52.619387 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_77_77_0, total 19 rows starting from the beginning of the part 2024.10.22 00:28:52.635975 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_78_78_0, total 14 rows starting from the beginning of the part 2024.10.22 00:28:52.662849 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_79_79_0, total 20 rows starting from the beginning of the part 2024.10.22 00:28:52.682977 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_80_80_0, total 5 rows starting from the beginning of the part 2024.10.22 00:28:52.697381 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_81_81_0, total 13 rows starting from the beginning of the part 2024.10.22 00:28:52.783000 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.57 MiB. 2024.10.22 00:28:52.989062 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kjh/bzivnfiqgcbdiwvewjlbnnguynseq. 2024.10.22 00:28:53.343937 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} MergeTask::MergeProjectionsStage: Merge sorted 108 rows, containing 32 columns (32 merged, 0 gathered) in 2.183778357 sec., 49.455568443478256 rows/sec., 6.86 KiB/sec. 2024.10.22 00:28:53.371636 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} MergedBlockOutputStream: filled checksums 202410_1_64_13 (state Temporary) 2024.10.22 00:28:53.492815 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_64_13 to 202410_1_64_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:53.497175 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_64_13} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_59_12, 202410_64_64_0] -> 202410_1_64_13 2024.10.22 00:28:53.558681 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:53.561033 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:53.562987 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:28:53.567861 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_81_16/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:28:53.766111 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} MergeTask::MergeProjectionsStage: Merge sorted 1076 rows, containing 12 columns (12 merged, 0 gathered) in 1.475012429 sec., 729.4853784584618 rows/sec., 111.59 KiB/sec. 2024.10.22 00:28:53.784040 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} MergedBlockOutputStream: filled checksums 202410_1_81_16 (state Temporary) 2024.10.22 00:28:53.841336 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 5096 2024.10.22 00:28:54.005267 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_81_16 to 202410_1_81_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:54.012581 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_81_16} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_76_15, 202410_81_81_0] -> 202410_1_81_16 2024.10.22 00:28:54.066854 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:28:54.194487 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:54.202650 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:54.482507 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_119_119_0 (state Temporary) 2024.10.22 00:28:54.657296 [ 1172 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 00:28:54.730302 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_119_119_0 to 202410_132_132_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:54.780398 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5096 2024.10.22 00:28:54.819642 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.81 GiB, free memory in arenas 9.78 MiB, will set to 1.62 GiB (RSS), difference: 116.06 MiB 2024.10.22 00:28:54.867831 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 118042 2024.10.22 00:28:55.432235 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:55.552240 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:55.783587 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_66_66_0 (state Temporary) 2024.10.22 00:28:55.956789 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_66_66_0 to 202410_73_73_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:56.011367 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 118042 2024.10.22 00:28:56.273452 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_14_14_0 (state Temporary) 2024.10.22 00:28:56.662522 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 11.77 MiB, will set to 1.61 GiB (RSS), difference: 73.60 MiB 2024.10.22 00:28:58.271378 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 10.41 MiB, will set to 1.61 GiB (RSS), difference: -3.65 MiB 2024.10.22 00:28:58.474206 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:28:58.788709 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 21 entries to flush up to offset 5117 2024.10.22 00:28:59.010990 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:59.015632 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:59.159253 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_120_120_0 (state Temporary) 2024.10.22 00:28:59.218597 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_120_120_0 to 202410_133_133_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:28:59.286512 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5117 2024.10.22 00:28:59.459421 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:28:59.460217 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 41 entries to flush up to offset 5096 2024.10.22 00:28:59.463194 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:28:59.769174 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 995 2024.10.22 00:28:59.776180 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:59.779039 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.48 GiB. 2024.10.22 00:28:59.825421 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_68_68_0 (state Temporary) 2024.10.22 00:28:59.884313 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:28:59.889600 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.47 GiB. 2024.10.22 00:28:59.968474 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_68_68_0 to 202410_74_74_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:00.011588 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5096 2024.10.22 00:29:00.040929 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_69_69_0 (state Temporary) 2024.10.22 00:29:00.163760 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_69_69_0 to 202410_82_82_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:00.233110 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_11_2, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0] 2024.10.22 00:29:00.238155 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_11_2, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0] 2024.10.22 00:29:00.264366 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_11_2, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0] 2024.10.22 00:29:00.291325 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 995 2024.10.22 00:29:01.799672 [ 1216 ] {} DatabaseCatalog: Execution took 209 ms. 2024.10.22 00:29:02.383381 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_10_2, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0] 2024.10.22 00:29:02.385197 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_10_2, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0] 2024.10.22 00:29:02.428174 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_10_2, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0, 202410_15_15_0] 2024.10.22 00:29:02.853030 [ 1211 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:29:02.890753 [ 1211 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:29:03.019005 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1772 entries to flush up to offset 119814 2024.10.22 00:29:03.289807 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 5147 2024.10.22 00:29:03.411773 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:03.415114 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.47 GiB. 2024.10.22 00:29:03.667341 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_121_121_0 (state Temporary) 2024.10.22 00:29:03.814795 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_121_121_0 to 202410_134_134_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:03.971632 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5147 2024.10.22 00:29:04.082639 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:04.084395 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.47 GiB. 2024.10.22 00:29:04.333384 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_67_67_0 (state Temporary) 2024.10.22 00:29:04.611833 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_67_67_0 to 202410_74_74_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:04.719695 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 7.48 MiB, will set to 1.61 GiB (RSS), difference: -8.35 MiB 2024.10.22 00:29:04.805275 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_14_14_0 to 202410_17_17_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:04.867468 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 119814 2024.10.22 00:29:06.170371 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 73 2024.10.22 00:29:06.409128 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 8.91 MiB, will set to 1.61 GiB (RSS), difference: 15.62 MiB 2024.10.22 00:29:06.666793 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 536 2024.10.22 00:29:06.839484 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:06.844772 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.47 GiB. 2024.10.22 00:29:07.007580 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_55_55_0 (state Temporary) 2024.10.22 00:29:07.492491 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_55_55_0 to 202410_65_65_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:07.537435 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 19 entries to flush up to offset 5115 2024.10.22 00:29:07.682403 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 73 2024.10.22 00:29:07.894933 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 1001 2024.10.22 00:29:07.975433 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 23 entries to flush up to offset 5170 2024.10.22 00:29:08.071565 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:08.074318 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.47 GiB. 2024.10.22 00:29:08.121092 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:08.126422 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.47 GiB. 2024.10.22 00:29:08.167993 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_70_70_0 (state Temporary) 2024.10.22 00:29:08.213353 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_69_69_0 (state Temporary) 2024.10.22 00:29:08.250120 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_69_69_0 to 202410_75_75_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:08.255527 [ 1251 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 00:29:08.277135 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:08.278704 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.47 GiB. 2024.10.22 00:29:08.307760 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5115 2024.10.22 00:29:08.315301 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_70_70_0 to 202410_83_83_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:08.324841 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 11.29 MiB, will set to 1.60 GiB (RSS), difference: -6.04 MiB 2024.10.22 00:29:08.372005 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1001 2024.10.22 00:29:08.437040 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_122_122_0 (state Temporary) 2024.10.22 00:29:08.475035 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_122_122_0 to 202410_135_135_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:08.520835 [ 1253 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_70_14 to 202410_75_75_0 2024.10.22 00:29:08.541723 [ 1253 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:08.545236 [ 1253 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.47 GiB. 2024.10.22 00:29:08.551861 [ 1253 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 289 ms. 2024.10.22 00:29:08.558842 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_70_14 to 202410_75_75_0 into Compact with storage Full 2024.10.22 00:29:08.586113 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:29:08.632981 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5170 2024.10.22 00:29:08.692083 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_70_14, total 5036 rows starting from the beginning of the part 2024.10.22 00:29:08.715237 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_71_71_0, total 50 rows starting from the beginning of the part 2024.10.22 00:29:08.746487 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_72_72_0, total 39 rows starting from the beginning of the part 2024.10.22 00:29:08.752482 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_26_5, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0, 202410_31_31_0] 2024.10.22 00:29:08.756957 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_26_5, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0, 202410_31_31_0] 2024.10.22 00:29:08.767390 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_73_73_0, total 34 rows starting from the beginning of the part 2024.10.22 00:29:08.780129 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_26_5, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0, 202410_31_31_0] 2024.10.22 00:29:08.781524 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_74_74_0, total 41 rows starting from the beginning of the part 2024.10.22 00:29:08.799468 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_75_75_0, total 19 rows starting from the beginning of the part 2024.10.22 00:29:09.459901 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:29:09.462895 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:29:10.082807 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} MergeTask::MergeProjectionsStage: Merge sorted 5219 rows, containing 13 columns (13 merged, 0 gathered) in 1.529761399 sec., 3411.6431512859735 rows/sec., 1.43 MiB/sec. 2024.10.22 00:29:10.270497 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} MergedBlockOutputStream: filled checksums 202410_1_75_15 (state Temporary) 2024.10.22 00:29:10.451877 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_75_15 to 202410_1_75_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:10.457242 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_75_15} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_70_14, 202410_75_75_0] -> 202410_1_75_15 2024.10.22 00:29:10.475891 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.36 MiB. 2024.10.22 00:29:11.322961 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 10.87 MiB, will set to 1.60 GiB (RSS), difference: 2.67 MiB 2024.10.22 00:29:11.871667 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2068 entries to flush up to offset 121882 2024.10.22 00:29:12.171421 [ 1274 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 209 ms. 2024.10.22 00:29:12.570476 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:12.571908 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:12.637032 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 5215 2024.10.22 00:29:12.764360 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:12.766882 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:12.801384 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_68_68_0 (state Temporary) 2024.10.22 00:29:12.812834 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_123_123_0 (state Temporary) 2024.10.22 00:29:12.860063 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_123_123_0 to 202410_136_136_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:12.869486 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_68_68_0 to 202410_75_75_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:12.910512 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 121882 2024.10.22 00:29:13.016088 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5215 2024.10.22 00:29:13.114682 [ 1154 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_131_26 to 202410_136_136_0 2024.10.22 00:29:13.130472 [ 1154 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:13.132917 [ 1154 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:13.140017 [ 1154 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 269 ms. 2024.10.22 00:29:13.156680 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_131_26 to 202410_136_136_0 into Compact with storage Full 2024.10.22 00:29:13.183872 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:29:13.228288 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_131_26, total 5846 rows starting from the beginning of the part 2024.10.22 00:29:13.249251 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_132_132_0, total 81 rows starting from the beginning of the part 2024.10.22 00:29:13.266799 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_133_133_0, total 21 rows starting from the beginning of the part 2024.10.22 00:29:13.280232 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_134_134_0, total 30 rows starting from the beginning of the part 2024.10.22 00:29:13.294233 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_135_135_0, total 23 rows starting from the beginning of the part 2024.10.22 00:29:13.318076 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_136_136_0, total 45 rows starting from the beginning of the part 2024.10.22 00:29:13.377052 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 11.45 MiB, will set to 1.60 GiB (RSS), difference: 1.04 MiB 2024.10.22 00:29:14.202748 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 573 2024.10.22 00:29:14.474562 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} MergeTask::MergeProjectionsStage: Merge sorted 6046 rows, containing 13 columns (13 merged, 0 gathered) in 1.33020977 sec., 4545.147792742494 rows/sec., 709.96 KiB/sec. 2024.10.22 00:29:14.492516 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 11.50 MiB, will set to 1.61 GiB (RSS), difference: -4.59 MiB 2024.10.22 00:29:14.673149 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} MergedBlockOutputStream: filled checksums 202410_1_136_27 (state Temporary) 2024.10.22 00:29:15.017359 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_136_27 to 202410_1_136_27 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:15.021732 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_136_27} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_131_26, 202410_136_136_0] -> 202410_1_136_27 2024.10.22 00:29:15.167657 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.59 MiB. 2024.10.22 00:29:15.183805 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 74 2024.10.22 00:29:15.473649 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zpc/obwdaacwakhhpjihuklhalldnaiqq. 2024.10.22 00:29:15.492979 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uah/pxkafxonowbkaplsypmhrdkhkdjsn. 2024.10.22 00:29:15.563873 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:15.567080 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:15.627849 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_56_56_0 (state Temporary) 2024.10.22 00:29:15.714395 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_56_56_0 to 202410_66_66_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:15.813355 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 45 entries to flush up to offset 5160 2024.10.22 00:29:15.874447 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 1021 2024.10.22 00:29:15.883057 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 74 2024.10.22 00:29:15.923519 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:15.932405 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:15.963158 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:15.968130 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:15.977338 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_70_70_0 (state Temporary) 2024.10.22 00:29:16.004293 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uah/pxkafxonowbkaplsypmhrdkhkdjsn. 2024.10.22 00:29:16.008128 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_4_4_0 to 201403_4_4_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:16.013350 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_70_70_0 to 202410_76_76_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:16.021570 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zpc/obwdaacwakhhpjihuklhalldnaiqq. 2024.10.22 00:29:16.068528 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:29:16.090545 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_71_71_0 (state Temporary) 2024.10.22 00:29:16.140812 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5160 2024.10.22 00:29:16.257112 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_71_71_0 to 202410_84_84_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:16.286376 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1021 2024.10.22 00:29:16.299966 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.81 GiB, free memory in arenas 10.21 MiB, will set to 1.60 GiB (RSS), difference: 84.74 MiB 2024.10.22 00:29:17.021400 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 5269 2024.10.22 00:29:17.279626 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:17.283489 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:17.347074 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_124_124_0 (state Temporary) 2024.10.22 00:29:17.381131 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_124_124_0 to 202410_137_137_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:17.417831 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 10.16 MiB, will set to 1.36 GiB (RSS), difference: -53.76 MiB 2024.10.22 00:29:17.424850 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5269 2024.10.22 00:29:17.900036 [ 1197 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:29:17.946745 [ 1197 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:29:18.486592 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 1.81 GiB, free memory in arenas 10.29 MiB, will set to 1.38 GiB (RSS), difference: 12.77 MiB 2024.10.22 00:29:18.828718 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 4 2024.10.22 00:29:19.084441 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:19.095125 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:19.145570 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_4_4_0 (state Temporary) 2024.10.22 00:29:19.312532 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_4_4_0 to 202410_4_4_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:19.385826 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 4 2024.10.22 00:29:19.476142 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:29:19.564346 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:29:20.280492 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2304 entries to flush up to offset 124186 2024.10.22 00:29:20.485632 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.81 GiB, free memory in arenas 8.42 MiB, will set to 1.40 GiB (RSS), difference: -83.58 MiB 2024.10.22 00:29:21.306854 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:21.310833 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:21.415145 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_69_69_0 (state Temporary) 2024.10.22 00:29:21.435708 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 23 entries to flush up to offset 5292 2024.10.22 00:29:21.468858 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_69_69_0 to 202410_76_76_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:21.497318 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 124186 2024.10.22 00:29:21.511184 [ 1205 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_71_14 to 202410_76_76_0 2024.10.22 00:29:21.535106 [ 1205 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:21.541249 [ 1205 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:21.573250 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_71_14 to 202410_76_76_0 into Compact with storage Full 2024.10.22 00:29:21.600861 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:29:21.612637 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} MergeTreeSequentialSource: Reading 17 marks from part 202410_1_71_14, total 128176 rows starting from the beginning of the part 2024.10.22 00:29:21.623831 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_72_72_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:29:21.632305 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_73_73_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:29:21.660737 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:21.664422 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:21.665309 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_74_74_0, total 1772 rows starting from the beginning of the part 2024.10.22 00:29:21.677219 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_75_75_0, total 2068 rows starting from the beginning of the part 2024.10.22 00:29:21.687564 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_76_76_0, total 2304 rows starting from the beginning of the part 2024.10.22 00:29:21.699538 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_125_125_0 (state Temporary) 2024.10.22 00:29:21.865946 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_125_125_0 to 202410_138_138_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:22.059551 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5292 2024.10.22 00:29:22.302095 [ 1212 ] {} DatabaseCatalog: Execution took 315 ms. 2024.10.22 00:29:22.651717 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.81 GiB, free memory in arenas 5.79 MiB, will set to 1.45 GiB (RSS), difference: -13.79 MiB 2024.10.22 00:29:23.384879 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 75 2024.10.22 00:29:23.563641 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:23.566089 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:23.614556 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_57_57_0 (state Temporary) 2024.10.22 00:29:23.647471 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 39 entries to flush up to offset 5199 2024.10.22 00:29:23.752585 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:23.760647 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:23.788645 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 1028 2024.10.22 00:29:23.797288 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_71_71_0 (state Temporary) 2024.10.22 00:29:23.864995 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_71_71_0 to 202410_77_77_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:23.908046 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5199 2024.10.22 00:29:24.020614 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_57_57_0 to 202410_67_67_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:24.109236 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 75 2024.10.22 00:29:24.303966 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:24.309146 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:24.366149 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_72_72_0 (state Temporary) 2024.10.22 00:29:24.409052 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_72_72_0 to 202410_85_85_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:24.467579 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1028 2024.10.22 00:29:24.529616 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.81 GiB, free memory in arenas 6.78 MiB, will set to 1.50 GiB (RSS), difference: -18.19 MiB 2024.10.22 00:29:26.069762 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 5332 2024.10.22 00:29:26.175139 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:26.176704 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:26.347470 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_126_126_0 (state Temporary) 2024.10.22 00:29:26.377424 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_126_126_0 to 202410_139_139_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:26.472243 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5332 2024.10.22 00:29:26.954204 [ 1240 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 342 ms. 2024.10.22 00:29:26.977298 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} MergeTask::MergeProjectionsStage: Merge sorted 137776 rows, containing 4 columns (4 merged, 0 gathered) in 5.413614819 sec., 25449.908167912454 rows/sec., 374.53 KiB/sec. 2024.10.22 00:29:27.223424 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} MergedBlockOutputStream: filled checksums 202410_1_76_15 (state Temporary) 2024.10.22 00:29:27.265625 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_76_15 to 202410_1_76_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:27.270570 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_76_15} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_71_14, 202410_76_76_0] -> 202410_1_76_15 2024.10.22 00:29:27.302683 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:29:28.359496 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 288.12 MiB using storage policy from min volume index 0 2024.10.22 00:29:28.508376 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 125722 2024.10.22 00:29:28.579614 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 9.05 MiB, will set to 1.57 GiB (RSS), difference: -10.38 MiB 2024.10.22 00:29:29.462526 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:29:29.464984 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:29:29.466758 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mow/zeaxjecsosukrdahmatflrqfmeuja (LogSeriesLimiter: on interval from 2024-10-22 00:28:21 to 2024-10-22 00:29:29 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:29:29.556351 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:29.558513 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:29.657401 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_70_70_0 (state Temporary) 2024.10.22 00:29:29.727906 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_70_70_0 to 202410_77_77_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:29.762713 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 125722 2024.10.22 00:29:29.783624 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:29.785839 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:30.279559 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.81 GiB, free memory in arenas 6.65 MiB, will set to 1.57 GiB (RSS), difference: -76.94 MiB 2024.10.22 00:29:30.477253 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 5358 2024.10.22 00:29:30.594969 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:30.604801 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:30.645111 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_127_127_0 (state Temporary) 2024.10.22 00:29:30.686865 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_31_6, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0, 202410_36_36_0] 2024.10.22 00:29:30.687197 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_127_127_0 to 202410_140_140_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:30.687909 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_31_6, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0, 202410_36_36_0] 2024.10.22 00:29:30.746865 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5358 2024.10.22 00:29:30.824109 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_31_6, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0, 202410_36_36_0] 2024.10.22 00:29:31.416383 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 22 entries to flush up to offset 5221 2024.10.22 00:29:31.456756 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.81 GiB, free memory in arenas 5.04 MiB, will set to 1.57 GiB (RSS), difference: -100.97 MiB 2024.10.22 00:29:31.507341 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:31.510437 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:31.531334 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_21_4, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0, 202410_26_26_0] 2024.10.22 00:29:31.540535 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_21_4, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0, 202410_26_26_0] 2024.10.22 00:29:31.554067 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_72_72_0 (state Temporary) 2024.10.22 00:29:31.575114 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_21_4, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0, 202410_26_26_0] 2024.10.22 00:29:31.612909 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 76 2024.10.22 00:29:31.944550 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_72_72_0 to 202410_78_78_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:31.965135 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:31.967440 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:31.969350 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 1042 2024.10.22 00:29:31.988918 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5221 2024.10.22 00:29:32.034933 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_58_58_0 (state Temporary) 2024.10.22 00:29:32.101097 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:32.130742 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_58_58_0 to 202410_68_68_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:32.210540 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:32.359591 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_73_73_0 (state Temporary) 2024.10.22 00:29:32.406984 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_73_73_0 to 202410_86_86_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:32.475005 [ 1271 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_81_16 to 202410_86_86_0 2024.10.22 00:29:32.481031 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1042 2024.10.22 00:29:32.484564 [ 1271 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:32.488444 [ 1271 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:32.503017 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 76 2024.10.22 00:29:32.508304 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_81_16 to 202410_86_86_0 into Compact with storage Full 2024.10.22 00:29:32.583494 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:29:32.602844 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_81_16, total 1076 rows starting from the beginning of the part 2024.10.22 00:29:32.645547 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_15_15_0 (state Temporary) 2024.10.22 00:29:32.747749 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_82_82_0, total 25 rows starting from the beginning of the part 2024.10.22 00:29:32.773613 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_83_83_0, total 6 rows starting from the beginning of the part 2024.10.22 00:29:32.812845 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_84_84_0, total 20 rows starting from the beginning of the part 2024.10.22 00:29:32.957583 [ 1278 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:29:32.961345 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_85_85_0, total 7 rows starting from the beginning of the part 2024.10.22 00:29:32.989078 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_86_86_0, total 14 rows starting from the beginning of the part 2024.10.22 00:29:33.010119 [ 1278 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:29:33.388018 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:29:33.583485 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.81 GiB, free memory in arenas 4.75 MiB, will set to 1.57 GiB (RSS), difference: -172.21 MiB 2024.10.22 00:29:34.012281 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bfq/zynblfbdreqbbjwegzpryojummamh (LogSeriesLimiter: on interval from 2024-10-22 00:29:29 to 2024-10-22 00:29:34 accepted series 1 / 239 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:29:34.153570 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} MergeTask::MergeProjectionsStage: Merge sorted 1148 rows, containing 12 columns (12 merged, 0 gathered) in 1.653599879 sec., 694.2429148544949 rows/sec., 106.34 KiB/sec. 2024.10.22 00:29:34.174330 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} MergedBlockOutputStream: filled checksums 202410_1_86_17 (state Temporary) 2024.10.22 00:29:34.228915 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_86_17 to 202410_1_86_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:34.235703 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_86_17} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_81_16, 202410_86_86_0] -> 202410_1_86_17 2024.10.22 00:29:34.271829 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:29:34.859084 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 5411 2024.10.22 00:29:35.092875 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_15_3, 202410_16_16_0, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0] 2024.10.22 00:29:35.096307 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_15_3, 202410_16_16_0, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0] 2024.10.22 00:29:35.131051 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_15_3, 202410_16_16_0, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0] 2024.10.22 00:29:35.211444 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:35.215938 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:35.271917 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_128_128_0 (state Temporary) 2024.10.22 00:29:35.391598 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.81 GiB, free memory in arenas 2.64 MiB, will set to 1.57 GiB (RSS), difference: -133.69 MiB 2024.10.22 00:29:35.607280 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_128_128_0 to 202410_141_141_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:35.770785 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_136_27 to 202410_141_141_0 2024.10.22 00:29:35.775912 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:35.776874 [ 1158 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:35.780789 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5411 2024.10.22 00:29:35.882626 [ 1158 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 255 ms. 2024.10.22 00:29:35.908390 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_136_27 to 202410_141_141_0 into Compact with storage Full 2024.10.22 00:29:35.981118 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:29:36.002918 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_136_27, total 6046 rows starting from the beginning of the part 2024.10.22 00:29:36.061018 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_137_137_0, total 54 rows starting from the beginning of the part 2024.10.22 00:29:36.104312 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_138_138_0, total 23 rows starting from the beginning of the part 2024.10.22 00:29:36.263447 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_139_139_0, total 40 rows starting from the beginning of the part 2024.10.22 00:29:36.452902 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_140_140_0, total 26 rows starting from the beginning of the part 2024.10.22 00:29:36.489685 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_141_141_0, total 53 rows starting from the beginning of the part 2024.10.22 00:29:36.767346 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 127642 2024.10.22 00:29:37.444143 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.81 GiB, free memory in arenas 2.30 MiB, will set to 1.57 GiB (RSS), difference: -99.92 MiB 2024.10.22 00:29:37.483436 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:37.485609 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:37.727310 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_71_71_0 (state Temporary) 2024.10.22 00:29:37.910696 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_71_71_0 to 202410_78_78_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:37.956828 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 127642 2024.10.22 00:29:38.268343 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} MergeTask::MergeProjectionsStage: Merge sorted 6242 rows, containing 13 columns (13 merged, 0 gathered) in 2.373230129 sec., 2630.1705526678825 rows/sec., 409.95 KiB/sec. 2024.10.22 00:29:38.340559 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} MergedBlockOutputStream: filled checksums 202410_1_141_28 (state Temporary) 2024.10.22 00:29:38.371726 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 2.55 MiB, will set to 1.58 GiB (RSS), difference: 1.74 MiB 2024.10.22 00:29:38.518569 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_141_28 to 202410_1_141_28 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:38.529793 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_141_28} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_136_27, 202410_141_141_0] -> 202410_1_141_28 2024.10.22 00:29:38.617095 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.61 MiB. 2024.10.22 00:29:39.463663 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:29:39.466279 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:29:39.496479 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 45 entries to flush up to offset 5266 2024.10.22 00:29:39.510993 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 2.49 MiB, will set to 1.58 GiB (RSS), difference: 2.78 MiB 2024.10.22 00:29:39.783976 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 5450 2024.10.22 00:29:39.820109 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:39.823733 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:39.881176 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_73_73_0 (state Temporary) 2024.10.22 00:29:39.983017 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 1060 2024.10.22 00:29:40.006447 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 77 2024.10.22 00:29:40.042669 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_73_73_0 to 202410_79_79_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:40.096005 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:40.100093 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5266 2024.10.22 00:29:40.101544 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:40.231362 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:40.239422 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:40.249678 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_74_74_0 (state Temporary) 2024.10.22 00:29:40.290557 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_74_74_0 to 202410_87_87_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:40.290696 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:40.296350 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:40.328219 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1060 2024.10.22 00:29:40.390115 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_129_129_0 (state Temporary) 2024.10.22 00:29:40.453483 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_129_129_0 to 202410_142_142_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:40.461038 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_59_59_0 (state Temporary) 2024.10.22 00:29:40.501051 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5450 2024.10.22 00:29:40.528298 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_59_59_0 to 202410_69_69_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:40.760355 [ 1152 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_64_13 to 202410_69_69_0 2024.10.22 00:29:40.769378 [ 1152 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:40.771265 [ 1152 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:40.793284 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_64_13 to 202410_69_69_0 into Compact with storage Full 2024.10.22 00:29:40.928248 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 77 2024.10.22 00:29:41.066935 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:29:41.072516 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_64_13, total 108 rows starting from the beginning of the part 2024.10.22 00:29:41.090468 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_65_65_0, total 1 rows starting from the beginning of the part 2024.10.22 00:29:41.136458 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_66_66_0, total 1 rows starting from the beginning of the part 2024.10.22 00:29:41.275555 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_67_67_0, total 1 rows starting from the beginning of the part 2024.10.22 00:29:41.302890 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_68_68_0, total 1 rows starting from the beginning of the part 2024.10.22 00:29:41.444778 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_69_69_0, total 1 rows starting from the beginning of the part 2024.10.22 00:29:42.385171 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 3.11 MiB, will set to 1.59 GiB (RSS), difference: -2.94 MiB 2024.10.22 00:29:42.635413 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_15_3, 202410_16_16_0, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0] 2024.10.22 00:29:42.639806 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_15_3, 202410_16_16_0, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0] 2024.10.22 00:29:42.663635 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_15_3, 202410_16_16_0, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0] 2024.10.22 00:29:42.844117 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_15_15_0 to 202410_18_18_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:43.064350 [ 1210 ] {} DatabaseCatalog: Execution took 623 ms. 2024.10.22 00:29:43.094325 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} MergeTask::MergeProjectionsStage: Merge sorted 113 rows, containing 32 columns (32 merged, 0 gathered) in 2.308988609 sec., 48.93917603558 rows/sec., 6.74 KiB/sec. 2024.10.22 00:29:43.160216 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} MergedBlockOutputStream: filled checksums 202410_1_69_14 (state Temporary) 2024.10.22 00:29:43.231566 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 3.18 MiB, will set to 1.59 GiB (RSS), difference: 5.08 MiB 2024.10.22 00:29:43.804069 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_16_3, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 00:29:43.804568 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_69_14 to 202410_1_69_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:43.808991 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_16_3, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 00:29:43.819343 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_69_14} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_64_13, 202410_69_69_0] -> 202410_1_69_14 2024.10.22 00:29:43.876282 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:29:43.933548 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_16_3, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 00:29:44.378518 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 4.06 MiB, will set to 1.60 GiB (RSS), difference: 7.19 MiB 2024.10.22 00:29:44.507444 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 5500 2024.10.22 00:29:44.663203 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:44.666631 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:44.806735 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_130_130_0 (state Temporary) 2024.10.22 00:29:44.964566 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2688 entries to flush up to offset 130330 2024.10.22 00:29:45.078905 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_130_130_0 to 202410_143_143_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:45.116523 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 573 2024.10.22 00:29:45.144176 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5500 2024.10.22 00:29:45.850478 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 6.53 MiB, will set to 1.60 GiB (RSS), difference: 5.70 MiB 2024.10.22 00:29:45.867422 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:45.870911 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:46.130692 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_72_72_0 (state Temporary) 2024.10.22 00:29:46.386552 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_72_72_0 to 202410_79_79_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:46.645032 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 130330 2024.10.22 00:29:47.368471 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 6.73 MiB, will set to 1.60 GiB (RSS), difference: 3.96 MiB 2024.10.22 00:29:47.615275 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 29 entries to flush up to offset 5295 2024.10.22 00:29:47.831319 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 1074 2024.10.22 00:29:47.861148 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:47.870247 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:47.945305 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:47.957187 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:47.980188 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_75_75_0 (state Temporary) 2024.10.22 00:29:48.016208 [ 1240 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:29:48.028060 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_74_74_0 (state Temporary) 2024.10.22 00:29:48.170471 [ 1240 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:29:48.170542 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_74_74_0 to 202410_80_80_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:48.248798 [ 1236 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_75_15 to 202410_80_80_0 2024.10.22 00:29:48.257552 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5295 2024.10.22 00:29:48.267072 [ 1236 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:48.269212 [ 1236 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:48.303197 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_75_15 to 202410_80_80_0 into Compact with storage Full 2024.10.22 00:29:48.330982 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:29:48.261809 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_75_75_0 to 202410_88_88_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:48.439491 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_75_15, total 5219 rows starting from the beginning of the part 2024.10.22 00:29:48.458628 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_76_76_0, total 45 rows starting from the beginning of the part 2024.10.22 00:29:48.468928 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_77_77_0, total 39 rows starting from the beginning of the part 2024.10.22 00:29:48.477626 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_78_78_0, total 22 rows starting from the beginning of the part 2024.10.22 00:29:48.483863 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1074 2024.10.22 00:29:48.489208 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_79_79_0, total 45 rows starting from the beginning of the part 2024.10.22 00:29:48.507818 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_80_80_0, total 29 rows starting from the beginning of the part 2024.10.22 00:29:49.156616 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 5540 2024.10.22 00:29:49.384286 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:49.387790 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.46 GiB. 2024.10.22 00:29:49.448500 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_131_131_0 (state Temporary) 2024.10.22 00:29:49.464623 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:29:49.466204 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:29:49.478198 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 6.24 MiB, will set to 1.61 GiB (RSS), difference: -7.74 MiB 2024.10.22 00:29:49.486362 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} MergeTask::MergeProjectionsStage: Merge sorted 5399 rows, containing 13 columns (13 merged, 0 gathered) in 1.203606385 sec., 4485.685741854883 rows/sec., 1.88 MiB/sec. 2024.10.22 00:29:49.615074 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_131_131_0 to 202410_144_144_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:49.701580 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5540 2024.10.22 00:29:49.760104 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} MergedBlockOutputStream: filled checksums 202410_1_80_16 (state Temporary) 2024.10.22 00:29:49.875524 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_80_16 to 202410_1_80_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:49.887545 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_80_16} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_75_15, 202410_80_80_0] -> 202410_1_80_16 2024.10.22 00:29:49.927137 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.42 MiB. 2024.10.22 00:29:50.039468 [ 1712 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bfq/zynblfbdreqbbjwegzpryojummamh, size 81046 2024.10.22 00:29:50.072860 [ 1246 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 349 ms. 2024.10.22 00:29:50.709607 [ 1687 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:29:50.827346 [ 1678 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:29:50.849406 [ 1673 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:29:50.850685 [ 1680 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:29:50.982517 [ 1690 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:29:51.827361 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 4.00 MiB, will set to 1.60 GiB (RSS), difference: 189.86 MiB 2024.10.22 00:29:52.663123 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 611 2024.10.22 00:29:52.737952 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_36_7, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0] 2024.10.22 00:29:52.748470 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_36_7, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0] 2024.10.22 00:29:52.778801 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_36_7, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0] 2024.10.22 00:29:52.821161 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:29:53.363532 [ 1270 ] {} DatabaseCatalog: Execution took 247 ms. 2024.10.22 00:29:53.483941 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.81 GiB, free memory in arenas 3.54 MiB, will set to 1.60 GiB (RSS), difference: 102.63 MiB 2024.10.22 00:29:53.657716 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 131866 2024.10.22 00:29:53.719399 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 5570 2024.10.22 00:29:53.958164 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_6_6_0 (state Temporary) 2024.10.22 00:29:53.983649 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/igj/qxlpmpoibbmwllrhvtkyaqgzexrjs (LogSeriesLimiter: on interval from 2024-10-22 00:29:34 to 2024-10-22 00:29:53 accepted series 1 / 128 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:29:54.042911 [ 1686 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mow/zeaxjecsosukrdahmatflrqfmeuja, size 245 2024.10.22 00:29:54.077351 [ 1748 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/igj/qxlpmpoibbmwllrhvtkyaqgzexrjs, size 4 2024.10.22 00:29:54.128975 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:54.135452 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:29:54.379511 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:54.471097 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:29:54.475502 [ 1269 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 212 ms. 2024.10.22 00:29:54.513797 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_132_132_0 (state Temporary) 2024.10.22 00:29:54.557312 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_132_132_0 to 202410_145_145_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:54.563684 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_73_73_0 (state Temporary) 2024.10.22 00:29:54.605386 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_73_73_0 to 202410_80_80_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:54.685218 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 131866 2024.10.22 00:29:54.710307 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5570 2024.10.22 00:29:55.578952 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 2.47 MiB, will set to 1.59 GiB (RSS), difference: 59.01 MiB 2024.10.22 00:29:55.763912 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 101 entries to flush up to offset 5396 2024.10.22 00:29:55.869676 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:55.873488 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:29:55.917327 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_75_75_0 (state Temporary) 2024.10.22 00:29:55.935356 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 78 2024.10.22 00:29:55.972267 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_75_75_0 to 202410_81_81_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:55.985699 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 1087 2024.10.22 00:29:56.109448 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5396 2024.10.22 00:29:56.295375 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:56.303935 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:29:56.349697 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_76_76_0 (state Temporary) 2024.10.22 00:29:56.563863 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:56.568776 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:29:56.620731 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_76_76_0 to 202410_89_89_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:56.778779 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1087 2024.10.22 00:29:56.831960 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_60_60_0 (state Temporary) 2024.10.22 00:29:57.016999 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_60_60_0 to 202410_70_70_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:57.179105 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 78 2024.10.22 00:29:57.387703 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 2.45 MiB, will set to 1.59 GiB (RSS), difference: 1.36 MiB 2024.10.22 00:29:58.850621 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 5606 2024.10.22 00:29:59.080203 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:59.084316 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:29:59.144775 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_133_133_0 (state Temporary) 2024.10.22 00:29:59.281827 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_133_133_0 to 202410_146_146_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:29:59.465025 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:29:59.465199 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5606 2024.10.22 00:29:59.466763 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:29:59.560384 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_141_28 to 202410_146_146_0 2024.10.22 00:29:59.569485 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:29:59.570978 [ 1186 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:29:59.612591 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_141_28 to 202410_146_146_0 into Compact with storage Full 2024.10.22 00:29:59.736585 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:29:59.759571 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_141_28, total 6242 rows starting from the beginning of the part 2024.10.22 00:29:59.878263 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_142_142_0, total 39 rows starting from the beginning of the part 2024.10.22 00:29:59.895292 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_143_143_0, total 50 rows starting from the beginning of the part 2024.10.22 00:29:59.922322 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_144_144_0, total 40 rows starting from the beginning of the part 2024.10.22 00:29:59.942243 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_145_145_0, total 30 rows starting from the beginning of the part 2024.10.22 00:29:59.964463 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_146_146_0, total 36 rows starting from the beginning of the part 2024.10.22 00:30:00.428685 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 3.14 MiB, will set to 1.59 GiB (RSS), difference: -4.71 MiB 2024.10.22 00:30:01.310738 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} MergeTask::MergeProjectionsStage: Merge sorted 6437 rows, containing 13 columns (13 merged, 0 gathered) in 1.731561073 sec., 3717.4547871116297 rows/sec., 578.34 KiB/sec. 2024.10.22 00:30:01.576742 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} MergedBlockOutputStream: filled checksums 202410_1_146_29 (state Temporary) 2024.10.22 00:30:01.739411 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_146_29 to 202410_1_146_29 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:01.746530 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_146_29} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_141_28, 202410_146_146_0] -> 202410_1_146_29 2024.10.22 00:30:01.763529 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 133786 2024.10.22 00:30:01.776076 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.69 MiB. 2024.10.22 00:30:02.268216 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:02.269542 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:30:02.375540 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_74_74_0 (state Temporary) 2024.10.22 00:30:02.547697 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_74_74_0 to 202410_81_81_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:02.558836 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 1.84 MiB, will set to 1.59 GiB (RSS), difference: -2.36 MiB 2024.10.22 00:30:02.566918 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 00:30:02.583626 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 133786 2024.10.22 00:30:02.586795 [ 1187 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_76_15 to 202410_81_81_0 2024.10.22 00:30:02.607238 [ 1187 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:02.616372 [ 1187 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:30:02.655834 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_76_15 to 202410_81_81_0 into Compact with storage Full 2024.10.22 00:30:02.667380 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:30:02.670399 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} MergeTreeSequentialSource: Reading 18 marks from part 202410_1_76_15, total 137776 rows starting from the beginning of the part 2024.10.22 00:30:02.678591 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_77_77_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:30:02.690765 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_78_78_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:30:02.710847 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_79_79_0, total 2688 rows starting from the beginning of the part 2024.10.22 00:30:02.727566 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_80_80_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:30:02.746964 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_81_81_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:30:02.865424 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.101169223 sec. 2024.10.22 00:30:03.175283 [ 1160 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:30:03.211377 [ 1160 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:30:03.468334 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 5653 2024.10.22 00:30:03.630063 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 51 entries to flush up to offset 5447 2024.10.22 00:30:03.795135 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:03.801320 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:30:03.825792 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:03.828548 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:03.951439 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_134_134_0 (state Temporary) 2024.10.22 00:30:03.865814 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_76_76_0 (state Temporary) 2024.10.22 00:30:04.010553 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_134_134_0 to 202410_147_147_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:04.152172 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_76_76_0 to 202410_82_82_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:04.170927 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5653 2024.10.22 00:30:04.188519 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5447 2024.10.22 00:30:04.281962 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 1100 2024.10.22 00:30:04.394307 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 4.26 MiB, will set to 1.59 GiB (RSS), difference: 3.06 MiB 2024.10.22 00:30:04.584148 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:04.587807 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:30:04.680570 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 79 2024.10.22 00:30:04.731706 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_77_77_0 (state Temporary) 2024.10.22 00:30:04.876548 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_77_77_0 to 202410_90_90_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:05.135433 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1100 2024.10.22 00:30:05.188398 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:05.195650 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:30:05.364521 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_61_61_0 (state Temporary) 2024.10.22 00:30:05.440551 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_61_61_0 to 202410_71_71_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:05.513728 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 79 2024.10.22 00:30:06.878938 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 10.99 MiB, will set to 1.60 GiB (RSS), difference: 6.58 MiB 2024.10.22 00:30:08.182375 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 5679 2024.10.22 00:30:08.268351 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_81_16/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:30:08.305400 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:08.311875 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:30:08.359697 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_135_135_0 (state Temporary) 2024.10.22 00:30:08.416599 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} MergeTask::MergeProjectionsStage: Merge sorted 147376 rows, containing 4 columns (4 merged, 0 gathered) in 5.775057701 sec., 25519.39870219489 rows/sec., 375.44 KiB/sec. 2024.10.22 00:30:08.495128 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} MergedBlockOutputStream: filled checksums 202410_1_81_16 (state Temporary) 2024.10.22 00:30:08.499488 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_135_135_0 to 202410_148_148_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:08.663464 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5679 2024.10.22 00:30:08.691483 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:08.705605 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:30:08.730544 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_81_16 to 202410_1_81_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:08.735654 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_81_16} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_76_15, 202410_81_81_0] -> 202410_1_81_16 2024.10.22 00:30:08.892622 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:30:09.073816 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rwd/tvfhzvwufstoolmjlbiaskycpoovl. 2024.10.22 00:30:09.325063 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:30:09.468282 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:30:09.571347 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:30:09.587701 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 135322 2024.10.22 00:30:10.132492 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:10.138181 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:10.396916 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 14.40 MiB, will set to 1.60 GiB (RSS), difference: 5.16 MiB 2024.10.22 00:30:10.493800 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_26_5, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0, 202410_31_31_0] 2024.10.22 00:30:10.498725 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_26_5, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0, 202410_31_31_0] 2024.10.22 00:30:10.570592 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_75_75_0 (state Temporary) 2024.10.22 00:30:10.630184 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_26_5, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0, 202410_31_31_0] 2024.10.22 00:30:10.655200 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_75_75_0 to 202410_82_82_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:10.695201 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 135322 2024.10.22 00:30:11.698679 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 28 entries to flush up to offset 5475 2024.10.22 00:30:11.886516 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:11.890634 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:11.919801 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_77_77_0 (state Temporary) 2024.10.22 00:30:12.104710 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_77_77_0 to 202410_83_83_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:12.168876 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5475 2024.10.22 00:30:12.340581 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_16_16_0 (state Temporary) 2024.10.22 00:30:12.623896 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 13.53 MiB, will set to 1.60 GiB (RSS), difference: -3.55 MiB 2024.10.22 00:30:12.643273 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 1112 2024.10.22 00:30:12.666702 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 37 entries to flush up to offset 5716 2024.10.22 00:30:12.756036 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:12.758718 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:12.783931 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_78_78_0 (state Temporary) 2024.10.22 00:30:12.890891 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:12.896563 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:12.947705 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_136_136_0 (state Temporary) 2024.10.22 00:30:12.954215 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_78_78_0 to 202410_91_91_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:12.982850 [ 1247 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_86_17 to 202410_91_91_0 2024.10.22 00:30:12.984351 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1112 2024.10.22 00:30:12.988573 [ 1247 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:12.994469 [ 1247 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:13.015563 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 80 2024.10.22 00:30:13.104919 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_136_136_0 to 202410_149_149_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:13.131836 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_86_17 to 202410_91_91_0 into Compact with storage Full 2024.10.22 00:30:13.312591 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:30:13.332745 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_86_17, total 1148 rows starting from the beginning of the part 2024.10.22 00:30:13.348485 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_87_87_0, total 18 rows starting from the beginning of the part 2024.10.22 00:30:13.370538 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_88_88_0, total 14 rows starting from the beginning of the part 2024.10.22 00:30:13.375858 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5716 2024.10.22 00:30:13.379814 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_89_89_0, total 13 rows starting from the beginning of the part 2024.10.22 00:30:13.379942 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:13.382260 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:13.388781 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_90_90_0, total 13 rows starting from the beginning of the part 2024.10.22 00:30:13.397264 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_91_91_0, total 12 rows starting from the beginning of the part 2024.10.22 00:30:13.544845 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_62_62_0 (state Temporary) 2024.10.22 00:30:13.739909 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_62_62_0 to 202410_72_72_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:13.758027 [ 1246 ] {} DatabaseCatalog: Execution took 240 ms. 2024.10.22 00:30:13.952031 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 80 2024.10.22 00:30:14.030142 [ 1249 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 258 ms. 2024.10.22 00:30:14.048865 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} MergeTask::MergeProjectionsStage: Merge sorted 1218 rows, containing 12 columns (12 merged, 0 gathered) in 0.927643987 sec., 1313.0037137835725 rows/sec., 201.45 KiB/sec. 2024.10.22 00:30:14.176725 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} MergedBlockOutputStream: filled checksums 202410_1_91_18 (state Temporary) 2024.10.22 00:30:14.280532 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_91_18 to 202410_1_91_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:14.286905 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_91_18} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_86_17, 202410_91_91_0] -> 202410_1_91_18 2024.10.22 00:30:14.336896 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:30:14.384399 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 10.02 MiB, will set to 1.60 GiB (RSS), difference: -2.70 MiB 2024.10.22 00:30:16.777593 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_41_8, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 00:30:16.780685 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_41_8, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 00:30:17.023392 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_41_8, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 00:30:17.379432 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 5755 2024.10.22 00:30:17.608465 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:17.624337 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:17.676346 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_137_137_0 (state Temporary) 2024.10.22 00:30:17.702822 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 137242 2024.10.22 00:30:17.718713 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_137_137_0 to 202410_150_150_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:17.769573 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5755 2024.10.22 00:30:18.228382 [ 1262 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:30:18.379630 [ 1262 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:30:18.502811 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:18.506650 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:18.816778 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_76_76_0 (state Temporary) 2024.10.22 00:30:18.877236 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_76_76_0 to 202410_83_83_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:19.105636 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 137242 2024.10.22 00:30:19.466682 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:30:19.475611 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:30:19.673130 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 28 entries to flush up to offset 5503 2024.10.22 00:30:19.777633 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:19.780575 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:19.830400 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_78_78_0 (state Temporary) 2024.10.22 00:30:19.891164 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_78_78_0 to 202410_84_84_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:19.968980 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5503 2024.10.22 00:30:20.485535 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 1125 2024.10.22 00:30:20.599130 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:20.615835 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:20.770176 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_79_79_0 (state Temporary) 2024.10.22 00:30:20.930435 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_79_79_0 to 202410_92_92_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:21.185036 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1125 2024.10.22 00:30:21.192986 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_16_16_0 to 202410_19_19_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:21.219853 [ 1151 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_14_3 to 202410_19_19_0 2024.10.22 00:30:21.244192 [ 1151 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:21.247546 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:30:21.281599 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_14_3 to 202410_19_19_0 into Compact with storage Full 2024.10.22 00:30:21.284490 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 5 old parts to remove. Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0, 202410_5_5_0] 2024.10.22 00:30:21.286856 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from filesystem (serially): Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0, 202410_5_5_0] 2024.10.22 00:30:21.423711 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from memory: Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0, 202410_5_5_0] 2024.10.22 00:30:21.459260 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 81 2024.10.22 00:30:21.499493 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 8.89 MiB, will set to 1.60 GiB (RSS), difference: 3.15 MiB 2024.10.22 00:30:21.779482 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 5793 2024.10.22 00:30:21.847996 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:21.855846 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:22.112880 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_63_63_0 (state Temporary) 2024.10.22 00:30:22.191760 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:22.200420 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:22.288467 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_63_63_0 to 202410_73_73_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:22.358816 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_138_138_0 (state Temporary) 2024.10.22 00:30:22.474238 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 81 2024.10.22 00:30:22.493082 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_138_138_0 to 202410_151_151_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:22.527075 [ 1188 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_146_29 to 202410_151_151_0 2024.10.22 00:30:22.544359 [ 1188 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:22.547440 [ 1188 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:22.647843 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5793 2024.10.22 00:30:22.655027 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_146_29 to 202410_151_151_0 into Compact with storage Full 2024.10.22 00:30:22.785824 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:30:22.795387 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_146_29, total 6437 rows starting from the beginning of the part 2024.10.22 00:30:22.917972 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_147_147_0, total 47 rows starting from the beginning of the part 2024.10.22 00:30:22.931337 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_148_148_0, total 26 rows starting from the beginning of the part 2024.10.22 00:30:23.060813 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_149_149_0, total 37 rows starting from the beginning of the part 2024.10.22 00:30:23.174155 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_150_150_0, total 39 rows starting from the beginning of the part 2024.10.22 00:30:23.196572 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_151_151_0, total 38 rows starting from the beginning of the part 2024.10.22 00:30:24.063405 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 611 2024.10.22 00:30:24.277062 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_20_4, 202410_21_21_0, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0] 2024.10.22 00:30:24.280506 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_20_4, 202410_21_21_0, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0] 2024.10.22 00:30:24.310301 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_20_4, 202410_21_21_0, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0] 2024.10.22 00:30:24.505079 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} MergeTask::MergeProjectionsStage: Merge sorted 6624 rows, containing 13 columns (13 merged, 0 gathered) in 1.945261473 sec., 3405.197754615685 rows/sec., 528.13 KiB/sec. 2024.10.22 00:30:24.688695 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} MergedBlockOutputStream: filled checksums 202410_1_151_30 (state Temporary) 2024.10.22 00:30:24.740573 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_151_30 to 202410_1_151_30 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:24.761679 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_151_30} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_146_29, 202410_151_151_0] -> 202410_1_151_30 2024.10.22 00:30:24.895283 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.70 MiB. 2024.10.22 00:30:25.359750 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 10.21 MiB, will set to 1.59 GiB (RSS), difference: 4.92 MiB 2024.10.22 00:30:26.112196 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1839 entries to flush up to offset 139081 2024.10.22 00:30:26.656769 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 5826 2024.10.22 00:30:26.845697 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:26.851303 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:26.876907 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:26.879679 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:26.981502 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_77_77_0 (state Temporary) 2024.10.22 00:30:27.002118 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_139_139_0 (state Temporary) 2024.10.22 00:30:27.029534 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_77_77_0 to 202410_84_84_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:27.067723 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 139081 2024.10.22 00:30:27.115329 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_21_4, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0, 202410_26_26_0] 2024.10.22 00:30:27.116926 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_21_4, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0, 202410_26_26_0] 2024.10.22 00:30:27.147335 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_139_139_0 to 202410_152_152_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:27.162354 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_21_4, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0, 202410_26_26_0] 2024.10.22 00:30:27.213518 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_20_4, 202410_21_21_0, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0] 2024.10.22 00:30:27.215491 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_20_4, 202410_21_21_0, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0] 2024.10.22 00:30:27.224549 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5826 2024.10.22 00:30:27.242234 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_20_4, 202410_21_21_0, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0] 2024.10.22 00:30:27.469669 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 9.18 MiB, will set to 1.59 GiB (RSS), difference: 66.47 MiB 2024.10.22 00:30:27.471193 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 40 entries to flush up to offset 5543 2024.10.22 00:30:27.680005 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:27.683847 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:30:27.725477 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_79_79_0 (state Temporary) 2024.10.22 00:30:27.863290 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_79_79_0 to 202410_85_85_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:27.891214 [ 1212 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_80_16 to 202410_85_85_0 2024.10.22 00:30:27.906192 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5543 2024.10.22 00:30:27.907835 [ 1212 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:27.911595 [ 1212 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.43 GiB. 2024.10.22 00:30:27.918553 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rwd/tvfhzvwufstoolmjlbiaskycpoovl. 2024.10.22 00:30:27.937101 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_80_16 to 202410_85_85_0 into Compact with storage Full 2024.10.22 00:30:28.274898 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:30:28.278159 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_80_16, total 5399 rows starting from the beginning of the part 2024.10.22 00:30:28.293736 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_81_81_0, total 101 rows starting from the beginning of the part 2024.10.22 00:30:28.310332 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_82_82_0, total 51 rows starting from the beginning of the part 2024.10.22 00:30:28.330555 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_83_83_0, total 28 rows starting from the beginning of the part 2024.10.22 00:30:28.359619 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_84_84_0, total 28 rows starting from the beginning of the part 2024.10.22 00:30:28.475706 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_85_85_0, total 40 rows starting from the beginning of the part 2024.10.22 00:30:28.687594 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 1139 2024.10.22 00:30:28.968294 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:28.971335 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:29.093585 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_80_80_0 (state Temporary) 2024.10.22 00:30:29.257354 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_80_80_0 to 202410_93_93_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:29.305363 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1139 2024.10.22 00:30:29.322443 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} MergeTask::MergeProjectionsStage: Merge sorted 5647 rows, containing 13 columns (13 merged, 0 gathered) in 1.390656043 sec., 4060.6733983034223 rows/sec., 1.70 MiB/sec. 2024.10.22 00:30:29.380624 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} MergedBlockOutputStream: filled checksums 202410_1_85_17 (state Temporary) 2024.10.22 00:30:29.468018 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:30:29.469265 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:30:29.659154 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_85_17 to 202410_1_85_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:29.662988 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_85_17} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_80_16, 202410_85_85_0] -> 202410_1_85_17 2024.10.22 00:30:29.686317 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.57 MiB. 2024.10.22 00:30:29.707135 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.81 GiB, free memory in arenas 7.86 MiB, will set to 1.59 GiB (RSS), difference: 105.23 MiB 2024.10.22 00:30:29.976463 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 82 2024.10.22 00:30:30.097247 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:30.101408 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:30.277764 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_64_64_0 (state Temporary) 2024.10.22 00:30:30.337791 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_64_64_0 to 202410_74_74_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:30.381626 [ 1220 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_69_14 to 202410_74_74_0 2024.10.22 00:30:30.389573 [ 1220 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:30.391668 [ 1220 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:30.413705 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_69_14 to 202410_74_74_0 into Compact with storage Full 2024.10.22 00:30:30.483686 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 82 2024.10.22 00:30:30.629677 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:30:30.644727 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_69_14, total 113 rows starting from the beginning of the part 2024.10.22 00:30:30.689725 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_70_70_0, total 1 rows starting from the beginning of the part 2024.10.22 00:30:30.795736 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_71_71_0, total 1 rows starting from the beginning of the part 2024.10.22 00:30:31.048090 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_72_72_0, total 1 rows starting from the beginning of the part 2024.10.22 00:30:31.083134 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_73_73_0, total 1 rows starting from the beginning of the part 2024.10.22 00:30:31.141157 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_74_74_0, total 1 rows starting from the beginning of the part 2024.10.22 00:30:31.230846 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 5893 2024.10.22 00:30:31.284312 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 8.60 MiB, will set to 1.58 GiB (RSS), difference: 9.23 MiB 2024.10.22 00:30:31.486916 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:31.489443 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:31.584456 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 650 2024.10.22 00:30:31.644860 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_140_140_0 (state Temporary) 2024.10.22 00:30:31.685902 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_140_140_0 to 202410_153_153_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:31.818835 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:30:31.951560 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5893 2024.10.22 00:30:32.508686 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:30:32.709613 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_14_3, total 475 rows starting from the beginning of the part 2024.10.22 00:30:32.935919 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} MergeTask::MergeProjectionsStage: Merge sorted 118 rows, containing 32 columns (32 merged, 0 gathered) in 2.532126435 sec., 46.60114849280818 rows/sec., 6.38 KiB/sec. 2024.10.22 00:30:32.992702 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} MergedBlockOutputStream: filled checksums 202410_1_74_15 (state Temporary) 2024.10.22 00:30:33.171688 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_74_15/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:30:33.392110 [ 1246 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:30:33.419754 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_74_15 to 202410_1_74_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:33.439724 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_74_15} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_69_14, 202410_74_74_0] -> 202410_1_74_15 2024.10.22 00:30:33.470883 [ 1246 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:30:33.487947 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:30:33.655118 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 5.19 MiB, will set to 1.58 GiB (RSS), difference: 1.74 MiB 2024.10.22 00:30:34.073075 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1513 entries to flush up to offset 140594 2024.10.22 00:30:34.720559 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:34.723717 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:34.864563 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_78_78_0 (state Temporary) 2024.10.22 00:30:34.968523 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_78_78_0 to 202410_85_85_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:34.987407 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 140594 2024.10.22 00:30:35.417386 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 59 entries to flush up to offset 5602 2024.10.22 00:30:35.433542 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_15_15_0, total 42 rows starting from the beginning of the part 2024.10.22 00:30:35.749769 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:35.858800 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:35.890943 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_80_80_0 (state Temporary) 2024.10.22 00:30:35.932544 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_80_80_0 to 202410_86_86_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:35.964497 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 5925 2024.10.22 00:30:35.977423 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5602 2024.10.22 00:30:36.177046 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:36.179022 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:36.331439 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_141_141_0 (state Temporary) 2024.10.22 00:30:36.376845 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_141_141_0 to 202410_154_154_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:36.436908 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5925 2024.10.22 00:30:36.810221 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 1159 2024.10.22 00:30:36.892576 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_16_16_0, total 41 rows starting from the beginning of the part 2024.10.22 00:30:37.013110 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:37.016570 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:37.060472 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_81_81_0 (state Temporary) 2024.10.22 00:30:37.131164 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_81_81_0 to 202410_94_94_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:37.283315 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1159 2024.10.22 00:30:38.707820 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_17_17_0, total 40 rows starting from the beginning of the part 2024.10.22 00:30:39.425112 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_46_9, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0, 202410_51_51_0] 2024.10.22 00:30:39.468481 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:30:39.470462 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:30:39.529261 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_46_9, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0, 202410_51_51_0] 2024.10.22 00:30:39.652371 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_46_9, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0, 202410_51_51_0] 2024.10.22 00:30:40.046909 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_18_18_0, total 37 rows starting from the beginning of the part 2024.10.22 00:30:40.454178 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 21 entries to flush up to offset 5946 2024.10.22 00:30:40.771966 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:40.773705 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:40.908837 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_142_142_0 (state Temporary) 2024.10.22 00:30:41.068920 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_142_142_0 to 202410_155_155_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:41.104470 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5946 2024.10.22 00:30:41.980121 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_19_19_0, total 38 rows starting from the beginning of the part 2024.10.22 00:30:42.065555 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1578 entries to flush up to offset 142172 2024.10.22 00:30:42.645797 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:42.649264 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.42 GiB. 2024.10.22 00:30:42.835451 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_79_79_0 (state Temporary) 2024.10.22 00:30:43.228438 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_79_79_0 to 202410_86_86_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:43.376019 [ 1193 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_81_16 to 202410_86_86_0 2024.10.22 00:30:43.381328 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 142172 2024.10.22 00:30:43.385068 [ 1193 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:43.420099 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:43.463320 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_81_16 to 202410_86_86_0 into Compact with storage Full 2024.10.22 00:30:43.472914 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:30:43.478698 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} MergeTreeSequentialSource: Reading 19 marks from part 202410_1_81_16, total 147376 rows starting from the beginning of the part 2024.10.22 00:30:43.485482 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 38 entries to flush up to offset 5640 2024.10.22 00:30:43.491926 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_82_82_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:30:43.610963 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_83_83_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:30:43.629042 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_84_84_0, total 1839 rows starting from the beginning of the part 2024.10.22 00:30:43.676737 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:43.679773 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:43.780264 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_85_85_0, total 1513 rows starting from the beginning of the part 2024.10.22 00:30:43.790875 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_86_86_0, total 1578 rows starting from the beginning of the part 2024.10.22 00:30:43.907158 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_81_81_0 (state Temporary) 2024.10.22 00:30:44.079637 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_81_81_0 to 202410_87_87_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:44.134502 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5640 2024.10.22 00:30:44.446073 [ 1170 ] {} DatabaseCatalog: Execution took 451 ms. 2024.10.22 00:30:44.786045 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 4 entries to flush up to offset 1163 2024.10.22 00:30:45.097536 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:45.111013 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:45.111799 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 5979 2024.10.22 00:30:45.156479 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_86_17/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:30:45.261045 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_82_82_0 (state Temporary) 2024.10.22 00:30:45.271743 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:45.277330 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:45.309674 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_82_82_0 to 202410_95_95_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:45.456841 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1163 2024.10.22 00:30:45.485739 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 83 2024.10.22 00:30:45.519329 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_143_143_0 (state Temporary) 2024.10.22 00:30:45.589134 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_143_143_0 to 202410_156_156_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:45.672107 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 5979 2024.10.22 00:30:45.675383 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 2.86 MiB, will set to 1.58 GiB (RSS), difference: -6.46 MiB 2024.10.22 00:30:45.740172 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_151_30 to 202410_156_156_0 2024.10.22 00:30:45.768043 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:45.848864 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:45.858749 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:45.865522 [ 1157 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:45.913633 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_65_65_0 (state Temporary) 2024.10.22 00:30:45.976126 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_151_30 to 202410_156_156_0 into Compact with storage Full 2024.10.22 00:30:46.024160 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:30:46.033461 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_151_30, total 6624 rows starting from the beginning of the part 2024.10.22 00:30:46.071336 [ 1157 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 457 ms. 2024.10.22 00:30:46.156933 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_65_65_0 to 202410_75_75_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:46.232810 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_152_152_0, total 33 rows starting from the beginning of the part 2024.10.22 00:30:46.262967 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_153_153_0, total 67 rows starting from the beginning of the part 2024.10.22 00:30:46.274945 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_154_154_0, total 32 rows starting from the beginning of the part 2024.10.22 00:30:46.331834 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_155_155_0, total 21 rows starting from the beginning of the part 2024.10.22 00:30:46.362561 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 83 2024.10.22 00:30:46.367681 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_156_156_0, total 33 rows starting from the beginning of the part 2024.10.22 00:30:46.443185 [ 1202 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 334 ms. 2024.10.22 00:30:47.676635 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 8.53 MiB, will set to 1.59 GiB (RSS), difference: -3.05 MiB 2024.10.22 00:30:48.474509 [ 1207 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:30:48.459367 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} MergeTask::MergeProjectionsStage: Merge sorted 6810 rows, containing 13 columns (13 merged, 0 gathered) in 2.487313978 sec., 2737.8931892932096 rows/sec., 424.29 KiB/sec. 2024.10.22 00:30:48.597650 [ 1207 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:30:48.718883 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} MergedBlockOutputStream: filled checksums 202410_1_156_31 (state Temporary) 2024.10.22 00:30:48.773360 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_156_31 to 202410_1_156_31 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:48.775747 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_156_31} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_151_30, 202410_156_156_0] -> 202410_1_156_31 2024.10.22 00:30:48.890476 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 7 file segments (while reserving 450266 bytes for 62f858f0f0d0a1f73e78159443a825d8:0) 2024.10.22 00:30:48.892759 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ac5a66928351316152f8e12bdb5bbd5e, offset: 0, size: 3159312 2024.10.22 00:30:48.900451 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ac5a66928351316152f8e12bdb5bbd5e for removal 2024.10.22 00:30:48.901339 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.72 MiB. 2024.10.22 00:30:48.906321 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f0549eb26015f3f2e7a3b76ef8c52827, offset: 0, size: 269 2024.10.22 00:30:48.911824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f0549eb26015f3f2e7a3b76ef8c52827 for removal 2024.10.22 00:30:48.917795 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bac2b430bf0fa08048de64a7b8a9374a, offset: 0, size: 4 2024.10.22 00:30:48.926693 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bac2b430bf0fa08048de64a7b8a9374a for removal 2024.10.22 00:30:48.927674 [ 1476 ] {} CacheMetadata: Key ac5a66928351316152f8e12bdb5bbd5e is removed from metadata 2024.10.22 00:30:48.933060 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0ebc1d7d491922f8ebc2390208721e88, offset: 0, size: 1 2024.10.22 00:30:48.939239 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0ebc1d7d491922f8ebc2390208721e88 for removal 2024.10.22 00:30:48.943220 [ 1476 ] {} CacheMetadata: Key bac2b430bf0fa08048de64a7b8a9374a is removed from metadata 2024.10.22 00:30:48.943831 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} MergeTask::MergeProjectionsStage: Merge sorted 155762 rows, containing 4 columns (4 merged, 0 gathered) in 5.488973184 sec., 28377.256506560483 rows/sec., 417.17 KiB/sec. 2024.10.22 00:30:48.943221 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 40932cff1c3f64586db39612a315f855, offset: 0, size: 1 2024.10.22 00:30:48.952849 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 40932cff1c3f64586db39612a315f855 for removal 2024.10.22 00:30:48.956615 [ 1476 ] {} CacheMetadata: Key 0ebc1d7d491922f8ebc2390208721e88 is removed from metadata 2024.10.22 00:30:48.958780 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2dce1dc019579827f69909fcbf4bc913, offset: 0, size: 4 2024.10.22 00:30:48.963929 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2dce1dc019579827f69909fcbf4bc913 for removal 2024.10.22 00:30:48.964277 [ 1476 ] {} CacheMetadata: Key 40932cff1c3f64586db39612a315f855 is removed from metadata 2024.10.22 00:30:48.967866 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 38a98c26e3531a1c3cb4044b02e38019, offset: 0, size: 1 2024.10.22 00:30:48.970584 [ 1476 ] {} CacheMetadata: Key 2dce1dc019579827f69909fcbf4bc913 is removed from metadata 2024.10.22 00:30:48.972841 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} MergedBlockOutputStream: filled checksums 202410_1_86_17 (state Temporary) 2024.10.22 00:30:48.975734 [ 1476 ] {} CacheMetadata: Key f0549eb26015f3f2e7a3b76ef8c52827 is removed from metadata 2024.10.22 00:30:49.001367 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_86_17 to 202410_1_86_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:49.008177 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_86_17} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_81_16, 202410_86_86_0] -> 202410_1_86_17 2024.10.22 00:30:49.047326 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:30:49.072480 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 38a98c26e3531a1c3cb4044b02e38019 for removal 2024.10.22 00:30:49.074910 [ 1476 ] {} CacheMetadata: Key 38a98c26e3531a1c3cb4044b02e38019 is removed from metadata 2024.10.22 00:30:49.469097 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:30:49.470593 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:30:49.666477 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 10.77 MiB, will set to 1.59 GiB (RSS), difference: 15.93 MiB 2024.10.22 00:30:49.689141 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 6048 2024.10.22 00:30:49.791985 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fqd/pomcijlwqluvxjwtfdoletcbymlpl. 2024.10.22 00:30:49.828006 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/wmz/npkvrwtmdxyhkbsgptodrpfthhcnh. 2024.10.22 00:30:50.072430 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:50.077459 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:50.148894 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_144_144_0 (state Temporary) 2024.10.22 00:30:50.177532 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_144_144_0 to 202410_157_157_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:50.183733 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:50.185434 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:50.215636 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6048 2024.10.22 00:30:50.385441 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1594 entries to flush up to offset 143766 2024.10.22 00:30:50.730692 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/wmz/npkvrwtmdxyhkbsgptodrpfthhcnh. 2024.10.22 00:30:50.747325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_5_5_0 to 201403_5_5_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:50.769248 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fqd/pomcijlwqluvxjwtfdoletcbymlpl. 2024.10.22 00:30:50.809165 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:30:51.120059 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_31_6, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0, 202410_36_36_0] 2024.10.22 00:30:51.122609 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_31_6, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0, 202410_36_36_0] 2024.10.22 00:30:51.154855 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_31_6, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0, 202410_36_36_0] 2024.10.22 00:30:51.214834 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:51.216734 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:51.527229 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_80_80_0 (state Temporary) 2024.10.22 00:30:51.559669 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.08 GiB, peak 1.81 GiB, free memory in arenas 9.99 MiB, will set to 1.51 GiB (RSS), difference: 443.51 MiB 2024.10.22 00:30:51.644615 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 80 entries to flush up to offset 5720 2024.10.22 00:30:51.727813 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_80_80_0 to 202410_87_87_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:51.765567 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 143766 2024.10.22 00:30:51.855531 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:51.860241 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:51.888059 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_19_4/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:30:51.898261 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_82_82_0 (state Temporary) 2024.10.22 00:30:51.966334 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_82_82_0 to 202410_88_88_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:52.031788 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5720 2024.10.22 00:30:52.962406 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 1184 2024.10.22 00:30:53.384763 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:53.387126 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:53.419299 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_83_83_0 (state Temporary) 2024.10.22 00:30:53.487613 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.81 GiB, free memory in arenas 8.57 MiB, will set to 1.32 GiB (RSS), difference: -356.74 MiB 2024.10.22 00:30:53.671800 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_83_83_0 to 202410_96_96_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:53.745526 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1184 2024.10.22 00:30:53.819588 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_91_18 to 202410_96_96_0 2024.10.22 00:30:53.835326 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:53.839141 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:53.865683 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 84 2024.10.22 00:30:53.874656 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_91_18 to 202410_96_96_0 into Compact with storage Full 2024.10.22 00:30:54.022148 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:30:54.033220 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_91_18, total 1218 rows starting from the beginning of the part 2024.10.22 00:30:54.057253 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_92_92_0, total 13 rows starting from the beginning of the part 2024.10.22 00:30:54.132321 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:54.136417 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:54.167130 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_93_93_0, total 14 rows starting from the beginning of the part 2024.10.22 00:30:54.175211 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_17_17_0 (state Temporary) 2024.10.22 00:30:54.191420 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_66_66_0 (state Temporary) 2024.10.22 00:30:54.228631 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 6099 2024.10.22 00:30:54.233688 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_94_94_0, total 20 rows starting from the beginning of the part 2024.10.22 00:30:54.287732 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_95_95_0, total 4 rows starting from the beginning of the part 2024.10.22 00:30:54.338190 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_96_96_0, total 21 rows starting from the beginning of the part 2024.10.22 00:30:54.387785 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:54.391944 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:54.428341 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_66_66_0 to 202410_76_76_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:54.566732 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_145_145_0 (state Temporary) 2024.10.22 00:30:54.616252 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_145_145_0 to 202410_158_158_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:54.653272 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.81 GiB, free memory in arenas 8.86 MiB, will set to 1.35 GiB (RSS), difference: -53.16 MiB 2024.10.22 00:30:54.665842 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 84 2024.10.22 00:30:54.772339 [ 1234 ] {} DatabaseCatalog: Execution took 322 ms. 2024.10.22 00:30:54.883682 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6099 2024.10.22 00:30:55.460154 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} MergeTask::MergeProjectionsStage: Merge sorted 1290 rows, containing 12 columns (12 merged, 0 gathered) in 1.590780336 sec., 810.9227721809104 rows/sec., 124.54 KiB/sec. 2024.10.22 00:30:55.611779 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} MergedBlockOutputStream: filled checksums 202410_1_96_19 (state Temporary) 2024.10.22 00:30:55.756512 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_96_19 to 202410_1_96_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:55.761325 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_96_19} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_91_18, 202410_96_96_0] -> 202410_1_96_19 2024.10.22 00:30:55.883011 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:30:56.699868 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.81 GiB, free memory in arenas 6.11 MiB, will set to 1.39 GiB (RSS), difference: -24.13 MiB 2024.10.22 00:30:56.911251 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 5 2024.10.22 00:30:57.060133 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:57.063578 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:57.113663 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_5_5_0 (state Temporary) 2024.10.22 00:30:57.189096 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_5_5_0 to 202410_5_5_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:57.368919 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 5 2024.10.22 00:30:58.271477 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.81 GiB, free memory in arenas 5.17 MiB, will set to 1.45 GiB (RSS), difference: -31.12 MiB 2024.10.22 00:30:58.774138 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1802 entries to flush up to offset 145568 2024.10.22 00:30:58.891705 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 6126 2024.10.22 00:30:59.059614 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:59.062092 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:59.101653 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_146_146_0 (state Temporary) 2024.10.22 00:30:59.158808 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_146_146_0 to 202410_159_159_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:30:59.305478 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6126 2024.10.22 00:30:59.305437 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.81 GiB, free memory in arenas 5.64 MiB, will set to 1.48 GiB (RSS), difference: -33.27 MiB 2024.10.22 00:30:59.471741 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:30:59.480523 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:30:59.566391 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 47 entries to flush up to offset 5767 2024.10.22 00:30:59.619804 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:59.627688 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:59.789697 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:30:59.793440 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:30:59.838409 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_83_83_0 (state Temporary) 2024.10.22 00:30:59.953196 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_81_81_0 (state Temporary) 2024.10.22 00:30:59.975528 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_83_83_0 to 202410_89_89_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:00.022154 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_17_17_0 to 202410_20_20_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:00.140905 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5767 2024.10.22 00:31:00.307382 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_81_81_0 to 202410_88_88_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:00.358660 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 145568 2024.10.22 00:31:01.249455 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 1199 2024.10.22 00:31:01.571343 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:01.573683 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:01.620892 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_84_84_0 (state Temporary) 2024.10.22 00:31:01.686702 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_84_84_0 to 202410_97_97_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:01.747603 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1199 2024.10.22 00:31:01.782420 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.81 GiB, free memory in arenas 6.79 MiB, will set to 1.54 GiB (RSS), difference: -94.70 MiB 2024.10.22 00:31:01.854378 [ 1277 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 454 ms. 2024.10.22 00:31:02.171747 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 85 2024.10.22 00:31:02.616351 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 650 2024.10.22 00:31:02.703708 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:02.710948 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:02.780335 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_67_67_0 (state Temporary) 2024.10.22 00:31:02.918614 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_51_10, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0, 202410_56_56_0] 2024.10.22 00:31:02.921533 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_51_10, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0, 202410_56_56_0] 2024.10.22 00:31:02.973536 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_67_67_0 to 202410_77_77_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:03.058381 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_51_10, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0, 202410_56_56_0] 2024.10.22 00:31:03.076695 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 85 2024.10.22 00:31:03.311308 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 6165 2024.10.22 00:31:03.421828 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_25_5, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0] 2024.10.22 00:31:03.426545 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_25_5, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0] 2024.10.22 00:31:03.451919 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:03.466855 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:03.548280 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_147_147_0 (state Temporary) 2024.10.22 00:31:03.558628 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.81 GiB, free memory in arenas 7.64 MiB, will set to 1.64 GiB (RSS), difference: -18.77 MiB 2024.10.22 00:31:03.614907 [ 1171 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:31:03.665712 [ 1171 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:31:03.674677 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_25_5, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0] 2024.10.22 00:31:03.966808 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_147_147_0 to 202410_160_160_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:04.021467 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6165 2024.10.22 00:31:04.093134 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_19_4/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:31:05.609300 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.81 GiB, free memory in arenas 5.61 MiB, will set to 1.69 GiB (RSS), difference: 552.91 MiB 2024.10.22 00:31:07.362491 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1658 entries to flush up to offset 147226 2024.10.22 00:31:07.456621 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.81 GiB, free memory in arenas 7.59 MiB, will set to 1.55 GiB (RSS), difference: -137.89 MiB 2024.10.22 00:31:07.597302 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 465.91 MiB using storage policy from min volume index 0 2024.10.22 00:31:07.700935 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_26_5, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0, 202410_31_31_0] 2024.10.22 00:31:07.707199 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_26_5, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0, 202410_31_31_0] 2024.10.22 00:31:07.754967 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 29 entries to flush up to offset 5796 2024.10.22 00:31:07.879873 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:07.882321 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:07.912132 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_26_5, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0, 202410_31_31_0] 2024.10.22 00:31:07.916573 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_84_84_0 (state Temporary) 2024.10.22 00:31:08.028097 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 24 entries to flush up to offset 6189 2024.10.22 00:31:08.066410 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_84_84_0 to 202410_90_90_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:08.090293 [ 1199 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_85_17 to 202410_90_90_0 2024.10.22 00:31:08.200415 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5796 2024.10.22 00:31:08.202792 [ 1199 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:08.205720 [ 1199 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:08.337045 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_85_17 to 202410_90_90_0 into Compact with storage Full 2024.10.22 00:31:08.378503 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:31:08.386216 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_85_17, total 5647 rows starting from the beginning of the part 2024.10.22 00:31:08.398318 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_86_86_0, total 59 rows starting from the beginning of the part 2024.10.22 00:31:08.415068 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_87_87_0, total 38 rows starting from the beginning of the part 2024.10.22 00:31:08.427024 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_88_88_0, total 80 rows starting from the beginning of the part 2024.10.22 00:31:08.444495 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:08.455851 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:08.464052 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_89_89_0, total 47 rows starting from the beginning of the part 2024.10.22 00:31:08.477564 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_90_90_0, total 29 rows starting from the beginning of the part 2024.10.22 00:31:08.491776 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:08.494774 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:08.515317 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_148_148_0 (state Temporary) 2024.10.22 00:31:08.685746 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_82_82_0 (state Temporary) 2024.10.22 00:31:08.735993 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_82_82_0 to 202410_89_89_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:08.759399 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_148_148_0 to 202410_161_161_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:08.871270 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yhz/ueiaizunikhmvbuxmfbadcmphsxgh (LogSeriesLimiter: on interval from 2024-10-22 00:29:53 to 2024-10-22 00:31:08 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:31:08.884472 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 147226 2024.10.22 00:31:08.887182 [ 1203 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_156_31 to 202410_161_161_0 2024.10.22 00:31:08.895098 [ 1203 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:08.898584 [ 1203 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:08.920242 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6189 2024.10.22 00:31:09.055736 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_156_31 to 202410_161_161_0 into Compact with storage Full 2024.10.22 00:31:09.180922 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:31:09.181147 [ 1181 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Selected 5 parts from 202410_1_1_0 to 202410_5_5_0 2024.10.22 00:31:09.186889 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_156_31, total 6810 rows starting from the beginning of the part 2024.10.22 00:31:09.191063 [ 1181 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:09.194826 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:09.210610 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_157_157_0, total 69 rows starting from the beginning of the part 2024.10.22 00:31:09.217028 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_5_1} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_1_0 to 202410_5_5_0 into Compact with storage Full 2024.10.22 00:31:09.242178 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_158_158_0, total 51 rows starting from the beginning of the part 2024.10.22 00:31:09.261826 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_90_18/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:31:09.264160 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_159_159_0, total 27 rows starting from the beginning of the part 2024.10.22 00:31:09.273467 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_5_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:31:09.280372 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_5_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_1_0, total 1 rows starting from the beginning of the part 2024.10.22 00:31:09.315765 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_5_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_2_2_0, total 1 rows starting from the beginning of the part 2024.10.22 00:31:09.346644 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 6.68 MiB, will set to 1.56 GiB (RSS), difference: -40.06 MiB 2024.10.22 00:31:09.249809 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 1205 2024.10.22 00:31:09.358731 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_5_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_3_3_0, total 1 rows starting from the beginning of the part 2024.10.22 00:31:09.385126 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_5_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_4_4_0, total 1 rows starting from the beginning of the part 2024.10.22 00:31:09.410703 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_160_160_0, total 39 rows starting from the beginning of the part 2024.10.22 00:31:09.420568 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_5_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_5_5_0, total 1 rows starting from the beginning of the part 2024.10.22 00:31:09.470650 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:31:09.472165 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:31:09.480456 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:09.482196 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:09.629321 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_85_85_0 (state Temporary) 2024.10.22 00:31:09.680178 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_161_161_0, total 24 rows starting from the beginning of the part 2024.10.22 00:31:09.780919 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_85_85_0 to 202410_98_98_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:09.808237 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:31:09.836610 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} MergeTask::MergeProjectionsStage: Merge sorted 5900 rows, containing 13 columns (13 merged, 0 gathered) in 1.508958691 sec., 3909.9811248577116 rows/sec., 1.63 MiB/sec. 2024.10.22 00:31:09.919693 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1205 2024.10.22 00:31:09.996449 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} MergedBlockOutputStream: filled checksums 202410_1_90_18 (state Temporary) 2024.10.22 00:31:10.061012 [ 1230 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 271 ms. 2024.10.22 00:31:10.077533 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_90_18 to 202410_1_90_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:10.081168 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_90_18} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_85_17, 202410_90_90_0] -> 202410_1_90_18 2024.10.22 00:31:10.112859 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.64 MiB. 2024.10.22 00:31:10.147896 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 689 2024.10.22 00:31:10.578414 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 86 2024.10.22 00:31:10.637331 [ 1422 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): /var/lib/clickhouse/store/2d8/2d8f770f-485b-41a3-9b48-1e3c86e7559f/tmp_merge_202410_1_5_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:31:10.884589 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} MergeTask::MergeProjectionsStage: Merge sorted 7020 rows, containing 13 columns (13 merged, 0 gathered) in 1.939683344 sec., 3619.147435438307 rows/sec., 559.37 KiB/sec. 2024.10.22 00:31:11.057551 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:11.061625 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:11.069567 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} MergedBlockOutputStream: filled checksums 202410_1_161_32 (state Temporary) 2024.10.22 00:31:11.073382 [ 1392 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_5_1} MergeTask::MergeProjectionsStage: Merge sorted 5 rows, containing 27 columns (27 merged, 0 gathered) in 1.861974993 sec., 2.6853207045192575 rows/sec., 8.53 KiB/sec. 2024.10.22 00:31:11.084896 [ 1397 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_5_1} MergedBlockOutputStream: filled checksums 202410_1_5_1 (state Temporary) 2024.10.22 00:31:11.113078 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_68_68_0 (state Temporary) 2024.10.22 00:31:11.311436 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_68_68_0 to 202410_78_78_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:11.335606 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_161_32 to 202410_1_161_32 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:11.341775 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_161_32} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_156_31, 202410_161_161_0] -> 202410_1_161_32 2024.10.22 00:31:11.396441 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.74 MiB. 2024.10.22 00:31:11.472157 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.81 GiB, free memory in arenas 3.57 MiB, will set to 1.56 GiB (RSS), difference: -150.21 MiB 2024.10.22 00:31:11.475411 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 86 2024.10.22 00:31:11.518584 [ 1386 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_5_1} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_merge_202410_1_5_1 to 202410_1_5_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:11.521436 [ 1386 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_5_1} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Merged 5 parts: [202410_1_1_0, 202410_5_5_0] -> 202410_1_5_1 2024.10.22 00:31:11.681600 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.24 MiB. 2024.10.22 00:31:12.924800 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 6271 2024.10.22 00:31:13.107796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xcm/wrpwdfvkxfyilugemfxwvcnakiggi (LogSeriesLimiter: on interval from 2024-10-22 00:31:08 to 2024-10-22 00:31:13 accepted series 1 / 206 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:31:13.180681 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:13.183702 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:13.353390 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_149_149_0 (state Temporary) 2024.10.22 00:31:13.488466 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_149_149_0 to 202410_162_162_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:13.562451 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.81 GiB, free memory in arenas 5.28 MiB, will set to 1.56 GiB (RSS), difference: -162.24 MiB 2024.10.22 00:31:13.663101 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6271 2024.10.22 00:31:14.389295 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 5.46 MiB, will set to 1.56 GiB (RSS), difference: -57.01 MiB 2024.10.22 00:31:15.157409 [ 1232 ] {} DatabaseCatalog: Execution took 285 ms. 2024.10.22 00:31:15.444119 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.81 GiB, free memory in arenas 5.54 MiB, will set to 1.56 GiB (RSS), difference: -72.09 MiB 2024.10.22 00:31:15.706867 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 68 entries to flush up to offset 5864 2024.10.22 00:31:15.831880 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:15.837238 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:15.887652 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2688 entries to flush up to offset 149914 2024.10.22 00:31:15.981334 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_85_85_0 (state Temporary) 2024.10.22 00:31:16.129587 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_85_85_0 to 202410_91_91_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:16.179056 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5864 2024.10.22 00:31:16.419768 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 5.70 MiB, will set to 1.56 GiB (RSS), difference: -53.38 MiB 2024.10.22 00:31:16.676735 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_25_5, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0] 2024.10.22 00:31:16.678119 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_25_5, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0] 2024.10.22 00:31:16.729596 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_25_5, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0] 2024.10.22 00:31:17.256014 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:17.262965 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:17.422343 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 24 entries to flush up to offset 1229 2024.10.22 00:31:17.522340 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_83_83_0 (state Temporary) 2024.10.22 00:31:17.530706 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:17.536371 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:17.579717 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_86_86_0 (state Temporary) 2024.10.22 00:31:17.691063 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 29 entries to flush up to offset 6300 2024.10.22 00:31:17.736100 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_86_86_0 to 202410_99_99_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:17.780390 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:17.785116 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:17.821086 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_150_150_0 (state Temporary) 2024.10.22 00:31:17.879384 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1229 2024.10.22 00:31:17.958574 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_83_83_0 to 202410_90_90_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:17.963860 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_150_150_0 to 202410_163_163_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:18.071589 [ 1248 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 00:31:18.082985 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 149914 2024.10.22 00:31:18.319615 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6300 2024.10.22 00:31:18.348732 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_19_4/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:31:18.456143 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 6.78 MiB, will set to 1.57 GiB (RSS), difference: -2.06 MiB 2024.10.22 00:31:18.716495 [ 1268 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:31:18.845473 [ 1268 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:31:19.472506 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:31:19.475279 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:31:20.339187 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 6.45 MiB, will set to 1.58 GiB (RSS), difference: 3.50 MiB 2024.10.22 00:31:21.671986 [ 1238 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 00:31:22.276432 [ 1277 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 211 ms. 2024.10.22 00:31:22.291382 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 6.58 MiB, will set to 1.60 GiB (RSS), difference: -3.13 MiB 2024.10.22 00:31:22.322723 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 21 entries to flush up to offset 6321 2024.10.22 00:31:22.630436 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:22.634761 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:22.684633 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_151_151_0 (state Temporary) 2024.10.22 00:31:22.724737 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_151_151_0 to 202410_164_164_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:22.971118 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6321 2024.10.22 00:31:23.071399 [ 1151 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 209 ms. 2024.10.22 00:31:23.527938 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 6.92 MiB, will set to 1.60 GiB (RSS), difference: -6.64 MiB 2024.10.22 00:31:23.686814 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 29 entries to flush up to offset 5893 2024.10.22 00:31:23.800727 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:23.806940 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:23.955902 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_86_86_0 (state Temporary) 2024.10.22 00:31:24.096138 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_86_86_0 to 202410_92_92_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:24.132551 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5893 2024.10.22 00:31:24.470598 [ 1167 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_74_15 to 202410_78_78_0 2024.10.22 00:31:24.480424 [ 1167 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:24.482734 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:24.519516 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_78_16} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_74_15 to 202410_78_78_0 into Compact with storage Full 2024.10.22 00:31:24.672362 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_78_16} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:31:24.677082 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_78_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_74_15, total 118 rows starting from the beginning of the part 2024.10.22 00:31:24.694281 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_78_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_75_75_0, total 1 rows starting from the beginning of the part 2024.10.22 00:31:24.764081 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_78_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_76_76_0, total 1 rows starting from the beginning of the part 2024.10.22 00:31:24.810659 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_78_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_77_77_0, total 1 rows starting from the beginning of the part 2024.10.22 00:31:24.858142 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_78_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_78_78_0, total 1 rows starting from the beginning of the part 2024.10.22 00:31:25.088142 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1922 entries to flush up to offset 151836 2024.10.22 00:31:25.383152 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 1234 2024.10.22 00:31:25.516932 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:25.531192 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:25.563750 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 7.14 MiB, will set to 1.61 GiB (RSS), difference: 5.21 MiB 2024.10.22 00:31:25.575534 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_87_87_0 (state Temporary) 2024.10.22 00:31:25.682062 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:25.684507 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:25.784424 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:25.785660 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:25.989346 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_87_87_0 to 202410_100_100_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:26.131264 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_84_84_0 (state Temporary) 2024.10.22 00:31:26.158283 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1234 2024.10.22 00:31:26.289279 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_84_84_0 to 202410_91_91_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:26.339814 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 151836 2024.10.22 00:31:26.426844 [ 1189 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_86_17 to 202410_91_91_0 2024.10.22 00:31:26.450582 [ 1189 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:26.454237 [ 1189 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:26.466531 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_86_17 to 202410_91_91_0 into Compact with storage Full 2024.10.22 00:31:26.478795 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 87 2024.10.22 00:31:26.481352 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:31:26.486864 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} MergeTreeSequentialSource: Reading 21 marks from part 202410_1_86_17, total 155762 rows starting from the beginning of the part 2024.10.22 00:31:26.499154 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_87_87_0, total 1594 rows starting from the beginning of the part 2024.10.22 00:31:26.515183 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_78_16} MergeTask::MergeProjectionsStage: Merge sorted 122 rows, containing 32 columns (32 merged, 0 gathered) in 2.001468717 sec., 60.95523700358691 rows/sec., 8.21 KiB/sec. 2024.10.22 00:31:26.515911 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_88_88_0, total 1802 rows starting from the beginning of the part 2024.10.22 00:31:26.524960 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_89_89_0, total 1658 rows starting from the beginning of the part 2024.10.22 00:31:26.571113 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_78_16} MergedBlockOutputStream: filled checksums 202410_1_78_16 (state Temporary) 2024.10.22 00:31:26.607245 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_56_11, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0, 202410_61_61_0] 2024.10.22 00:31:26.609004 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_56_11, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0, 202410_61_61_0] 2024.10.22 00:31:26.643935 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_56_11, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0, 202410_61_61_0] 2024.10.22 00:31:26.650445 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_90_90_0, total 2688 rows starting from the beginning of the part 2024.10.22 00:31:26.662631 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_91_91_0, total 1922 rows starting from the beginning of the part 2024.10.22 00:31:26.852551 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:26.857203 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:26.893600 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_78_16} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_78_16 to 202410_1_78_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:26.896119 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_78_16} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_74_15, 202410_78_78_0] -> 202410_1_78_16 2024.10.22 00:31:26.920037 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_69_69_0 (state Temporary) 2024.10.22 00:31:26.973426 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 6382 2024.10.22 00:31:27.157549 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_69_69_0 to 202410_79_79_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:27.158359 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:31:27.395932 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:27.405091 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:27.478659 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 87 2024.10.22 00:31:27.495301 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_152_152_0 (state Temporary) 2024.10.22 00:31:27.516157 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 6.96 MiB, will set to 1.62 GiB (RSS), difference: -8.04 MiB 2024.10.22 00:31:27.661181 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_152_152_0 to 202410_165_165_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:27.849729 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6382 2024.10.22 00:31:28.350628 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:31:29.215124 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_18_18_0 (state Temporary) 2024.10.22 00:31:29.429214 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 11.70 MiB, will set to 1.63 GiB (RSS), difference: 9.68 MiB 2024.10.22 00:31:29.472079 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:31:29.473359 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:31:29.758806 [ 1205 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 00:31:29.761589 [ 1420 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Found 6 old parts to remove. Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0, 202410_5_5_0, 202410_6_6_0] 2024.10.22 00:31:29.764592 [ 1420 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Removing 6 parts from filesystem (serially): Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0, 202410_5_5_0, 202410_6_6_0] 2024.10.22 00:31:29.785730 [ 1420 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Removing 6 parts from memory: Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0, 202410_5_5_0, 202410_6_6_0] 2024.10.22 00:31:30.402912 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.81 GiB, free memory in arenas 8.54 MiB, will set to 1.63 GiB (RSS), difference: 2.80 MiB 2024.10.22 00:31:31.642253 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 61 entries to flush up to offset 5954 2024.10.22 00:31:31.871301 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 21 entries to flush up to offset 6403 2024.10.22 00:31:31.984597 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_91_18/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:31:31.987827 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:31.990689 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:32.131128 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:32.133502 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:32.186102 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_153_153_0 (state Temporary) 2024.10.22 00:31:32.206317 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_87_87_0 (state Temporary) 2024.10.22 00:31:32.223599 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_36_7, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0] 2024.10.22 00:31:32.226888 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_36_7, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0] 2024.10.22 00:31:32.249309 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_87_87_0 to 202410_93_93_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:32.258356 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_153_153_0 to 202410_166_166_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:32.268497 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_36_7, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0] 2024.10.22 00:31:32.284526 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_161_32 to 202410_166_166_0 2024.10.22 00:31:32.284832 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6403 2024.10.22 00:31:32.285227 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 5954 2024.10.22 00:31:32.292182 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:32.294631 [ 1183 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:32.321830 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_161_32 to 202410_166_166_0 into Compact with storage Full 2024.10.22 00:31:32.370586 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:31:32.377679 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_161_32, total 7020 rows starting from the beginning of the part 2024.10.22 00:31:32.386510 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_162_162_0, total 82 rows starting from the beginning of the part 2024.10.22 00:31:32.396961 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_163_163_0, total 29 rows starting from the beginning of the part 2024.10.22 00:31:32.464406 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_19_4/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:31:32.515112 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_164_164_0, total 21 rows starting from the beginning of the part 2024.10.22 00:31:32.534978 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_165_165_0, total 61 rows starting from the beginning of the part 2024.10.22 00:31:32.559293 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_166_166_0, total 21 rows starting from the beginning of the part 2024.10.22 00:31:33.261552 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} MergeTask::MergeProjectionsStage: Merge sorted 7234 rows, containing 13 columns (13 merged, 0 gathered) in 0.946804996 sec., 7640.4328563555655 rows/sec., 1.15 MiB/sec. 2024.10.22 00:31:33.336501 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} MergedBlockOutputStream: filled checksums 202410_1_166_33 (state Temporary) 2024.10.22 00:31:33.382724 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1713 entries to flush up to offset 153549 2024.10.22 00:31:33.393767 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_166_33 to 202410_1_166_33 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:33.400264 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_166_33} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_161_32, 202410_166_166_0] -> 202410_1_166_33 2024.10.22 00:31:33.556442 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.76 MiB. 2024.10.22 00:31:33.662661 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 1253 2024.10.22 00:31:33.736362 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:33.744428 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:33.780682 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_88_88_0 (state Temporary) 2024.10.22 00:31:33.788320 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} MergeTask::MergeProjectionsStage: Merge sorted 165426 rows, containing 4 columns (4 merged, 0 gathered) in 7.326381561 sec., 22579.49556990047 rows/sec., 332.04 KiB/sec. 2024.10.22 00:31:33.860163 [ 1228 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:31:33.948562 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} MergedBlockOutputStream: filled checksums 202410_1_91_18 (state Temporary) 2024.10.22 00:31:33.979213 [ 1228 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:31:34.076814 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_91_18 to 202410_1_91_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:34.080538 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_91_18} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_86_17, 202410_91_91_0] -> 202410_1_91_18 2024.10.22 00:31:34.130955 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_88_88_0 to 202410_101_101_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:34.228376 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1253 2024.10.22 00:31:34.294950 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:34.296817 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:34.326027 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_96_19 to 202410_101_101_0 2024.10.22 00:31:34.428830 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:31:34.452169 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:34.454708 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:34.487811 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_96_19 to 202410_101_101_0 into Compact with storage Full 2024.10.22 00:31:34.564828 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:31:34.571283 [ 1224 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 257 ms. 2024.10.22 00:31:34.572278 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_96_19, total 1290 rows starting from the beginning of the part 2024.10.22 00:31:34.584783 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_97_97_0, total 15 rows starting from the beginning of the part 2024.10.22 00:31:34.603412 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_85_85_0 (state Temporary) 2024.10.22 00:31:34.606294 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_98_98_0, total 6 rows starting from the beginning of the part 2024.10.22 00:31:34.725069 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_99_99_0, total 24 rows starting from the beginning of the part 2024.10.22 00:31:34.758767 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_100_100_0, total 5 rows starting from the beginning of the part 2024.10.22 00:31:34.787032 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_101_101_0, total 19 rows starting from the beginning of the part 2024.10.22 00:31:34.926241 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_85_85_0 to 202410_92_92_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:34.980964 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 88 2024.10.22 00:31:35.168775 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 153549 2024.10.22 00:31:35.332375 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:35.335519 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:35.403464 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_70_70_0 (state Temporary) 2024.10.22 00:31:35.585232 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_70_70_0 to 202410_80_80_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:35.676247 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 88 2024.10.22 00:31:36.207626 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} MergeTask::MergeProjectionsStage: Merge sorted 1359 rows, containing 12 columns (12 merged, 0 gathered) in 1.734414083 sec., 783.5499107856356 rows/sec., 120.33 KiB/sec. 2024.10.22 00:31:36.235329 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} MergedBlockOutputStream: filled checksums 202410_1_101_20 (state Temporary) 2024.10.22 00:31:36.286006 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 6474 2024.10.22 00:31:36.312615 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_101_20 to 202410_1_101_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:36.318139 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_101_20} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_96_19, 202410_101_101_0] -> 202410_1_101_20 2024.10.22 00:31:36.350808 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:31:36.608937 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:36.623558 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.81 GiB, free memory in arenas 7.58 MiB, will set to 1.64 GiB (RSS), difference: 7.73 MiB 2024.10.22 00:31:36.626583 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:36.774888 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_154_154_0 (state Temporary) 2024.10.22 00:31:36.976612 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_154_154_0 to 202410_167_167_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:37.220515 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6474 2024.10.22 00:31:38.719246 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.81 GiB, free memory in arenas 5.38 MiB, will set to 1.63 GiB (RSS), difference: -3.98 MiB 2024.10.22 00:31:39.434917 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_18_18_0 to 202410_21_21_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:39.472966 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:31:39.474449 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:31:39.791428 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 59 entries to flush up to offset 6013 2024.10.22 00:31:39.943779 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:39.947096 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.41 GiB. 2024.10.22 00:31:40.080958 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_88_88_0 (state Temporary) 2024.10.22 00:31:40.215808 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_88_88_0 to 202410_94_94_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:40.263036 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6013 2024.10.22 00:31:40.276390 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.81 GiB, free memory in arenas 3.39 MiB, will set to 1.64 GiB (RSS), difference: 5.44 MiB 2024.10.22 00:31:41.255508 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 21 entries to flush up to offset 6495 2024.10.22 00:31:41.733928 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 1273 2024.10.22 00:31:41.774989 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:41.895905 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.40 GiB. 2024.10.22 00:31:41.957825 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_155_155_0 (state Temporary) 2024.10.22 00:31:41.974231 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:41.977647 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.40 GiB. 2024.10.22 00:31:42.050701 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_155_155_0 to 202410_168_168_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:42.173058 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1743 entries to flush up to offset 155292 2024.10.22 00:31:42.272185 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_89_89_0 (state Temporary) 2024.10.22 00:31:42.323342 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6495 2024.10.22 00:31:42.333006 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_89_89_0 to 202410_102_102_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:42.396803 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1273 2024.10.22 00:31:42.606531 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.81 GiB, free memory in arenas 7.15 MiB, will set to 1.63 GiB (RSS), difference: 124.68 MiB 2024.10.22 00:31:42.710445 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 689 2024.10.22 00:31:43.177761 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 89 2024.10.22 00:31:43.632377 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:43.677498 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:43.759498 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.39 GiB. 2024.10.22 00:31:43.763747 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.39 GiB. 2024.10.22 00:31:43.835840 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_71_71_0 (state Temporary) 2024.10.22 00:31:43.877791 [ 1159 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 215 ms. 2024.10.22 00:31:43.961033 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} MergeTask::MergeProjectionsStage: Merge sorted 673 rows, containing 679 columns (679 merged, 0 gathered) in 82.681804282 sec., 8.139638531648655 rows/sec., 10.69 KiB/sec. 2024.10.22 00:31:43.991324 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_71_71_0 to 202410_81_81_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:44.060457 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_86_86_0 (state Temporary) 2024.10.22 00:31:44.108510 [ 1717 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xcm/wrpwdfvkxfyilugemfxwvcnakiggi, size 216 2024.10.22 00:31:44.151080 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:31:44.317059 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 89 2024.10.22 00:31:44.327963 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_86_86_0 to 202410_93_93_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:44.366040 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 155292 2024.10.22 00:31:44.585045 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.81 GiB, free memory in arenas 8.54 MiB, will set to 1.61 GiB (RSS), difference: 150.86 MiB 2024.10.22 00:31:45.463038 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_7_7_0 (state Temporary) 2024.10.22 00:31:45.477535 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tts/aeaktfpvhzxnvwmngisqjmkkshyaq (LogSeriesLimiter: on interval from 2024-10-22 00:31:13 to 2024-10-22 00:31:45 accepted series 1 / 143 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:31:45.590102 [ 1739 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yhz/ueiaizunikhmvbuxmfbadcmphsxgh, size 575 2024.10.22 00:31:45.607673 [ 1743 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tts/aeaktfpvhzxnvwmngisqjmkkshyaq, size 4 2024.10.22 00:31:45.659232 [ 1172 ] {} DatabaseCatalog: Execution took 285 ms. 2024.10.22 00:31:46.252985 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_30_6, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0] 2024.10.22 00:31:46.256386 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_30_6, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0] 2024.10.22 00:31:46.336753 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 6533 2024.10.22 00:31:46.380533 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_30_6, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0] 2024.10.22 00:31:46.488285 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} MergedBlockOutputStream: filled checksums 202410_1_19_4 (state Temporary) 2024.10.22 00:31:46.571625 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.81 GiB, free memory in arenas 11.73 MiB, will set to 1.57 GiB (RSS), difference: 69.05 MiB 2024.10.22 00:31:46.592650 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:46.597653 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:31:46.649452 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_156_156_0 (state Temporary) 2024.10.22 00:31:46.887562 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_156_156_0 to 202410_169_169_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:47.064944 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6533 2024.10.22 00:31:47.772303 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 56 entries to flush up to offset 6069 2024.10.22 00:31:47.891957 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:47.896673 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:31:48.053394 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_89_89_0 (state Temporary) 2024.10.22 00:31:48.081051 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_89_89_0 to 202410_95_95_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:48.157672 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6069 2024.10.22 00:31:48.162633 [ 1196 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_90_18 to 202410_95_95_0 2024.10.22 00:31:48.168890 [ 1196 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:48.170366 [ 1196 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:31:48.171967 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_31_6, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0, 202410_36_36_0] 2024.10.22 00:31:48.175428 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_31_6, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0, 202410_36_36_0] 2024.10.22 00:31:48.190992 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_90_18 to 202410_95_95_0 into Compact with storage Full 2024.10.22 00:31:48.230785 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:31:48.236670 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_90_18, total 5900 rows starting from the beginning of the part 2024.10.22 00:31:48.258942 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_91_91_0, total 68 rows starting from the beginning of the part 2024.10.22 00:31:48.273689 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_92_92_0, total 29 rows starting from the beginning of the part 2024.10.22 00:31:48.286015 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_93_93_0, total 61 rows starting from the beginning of the part 2024.10.22 00:31:48.299953 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_94_94_0, total 59 rows starting from the beginning of the part 2024.10.22 00:31:48.312399 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_31_6, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0, 202410_36_36_0] 2024.10.22 00:31:48.319670 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_95_95_0, total 56 rows starting from the beginning of the part 2024.10.22 00:31:48.986443 [ 1202 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:31:49.046701 [ 1202 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:31:49.266849 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} MergeTask::MergeProjectionsStage: Merge sorted 6173 rows, containing 13 columns (13 merged, 0 gathered) in 1.082546966 sec., 5702.2930125693965 rows/sec., 2.38 MiB/sec. 2024.10.22 00:31:49.425804 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} MergedBlockOutputStream: filled checksums 202410_1_95_19 (state Temporary) 2024.10.22 00:31:49.472254 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:31:49.472870 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:31:49.731321 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_95_19 to 202410_1_95_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:49.734958 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_95_19} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_90_18, 202410_95_95_0] -> 202410_1_95_19 2024.10.22 00:31:49.761441 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.78 MiB. 2024.10.22 00:31:49.904584 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 1286 2024.10.22 00:31:49.912882 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_61_12, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0, 202410_66_66_0] 2024.10.22 00:31:49.917733 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_61_12, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0, 202410_66_66_0] 2024.10.22 00:31:49.964737 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_61_12, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0, 202410_66_66_0] 2024.10.22 00:31:50.016612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/bfq/zynblfbdreqbbjwegzpryojummamh. 2024.10.22 00:31:50.219846 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:50.223586 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:31:50.267541 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_90_90_0 (state Temporary) 2024.10.22 00:31:50.312362 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_90_90_0 to 202410_103_103_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:50.350069 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1286 2024.10.22 00:31:50.476067 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 729 2024.10.22 00:31:51.067742 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 6582 2024.10.22 00:31:51.402694 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 157212 2024.10.22 00:31:51.500760 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:51.507013 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:31:51.670607 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_157_157_0 (state Temporary) 2024.10.22 00:31:51.708207 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_157_157_0 to 202410_170_170_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:51.819493 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 90 2024.10.22 00:31:51.886973 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6582 2024.10.22 00:31:52.060147 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:52.063108 [ 1222 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 00:31:52.167101 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:31:52.247669 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_72_72_0 (state Temporary) 2024.10.22 00:31:52.326518 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_72_72_0 to 202410_82_82_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:52.498854 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 90 2024.10.22 00:31:52.709396 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:52.714438 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:31:52.800909 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 8.50 MiB, will set to 1.57 GiB (RSS), difference: -8.31 MiB 2024.10.22 00:31:52.866464 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_87_87_0 (state Temporary) 2024.10.22 00:31:52.965277 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_87_87_0 to 202410_94_94_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:52.985136 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 157212 2024.10.22 00:31:54.395930 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 8.50 MiB, will set to 1.57 GiB (RSS), difference: 5.18 MiB 2024.10.22 00:31:55.332979 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_19_4/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:31:55.667134 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 46 entries to flush up to offset 6115 2024.10.22 00:31:56.010650 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 23 entries to flush up to offset 6605 2024.10.22 00:31:56.322763 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:56.322827 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:56.325294 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:31:56.328940 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:31:56.397419 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_90_90_0 (state Temporary) 2024.10.22 00:31:56.489527 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_90_90_0 to 202410_96_96_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:56.513058 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_158_158_0 (state Temporary) 2024.10.22 00:31:56.675557 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6115 2024.10.22 00:31:56.712545 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_158_158_0 to 202410_171_171_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:56.866769 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_166_33 to 202410_171_171_0 2024.10.22 00:31:56.880717 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:56.893798 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:31:57.008387 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6605 2024.10.22 00:31:57.061760 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_166_33 to 202410_171_171_0 into Compact with storage Full 2024.10.22 00:31:57.104920 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 8.65 MiB, will set to 1.57 GiB (RSS), difference: 1.05 MiB 2024.10.22 00:31:57.191840 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:31:57.197639 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_166_33, total 7234 rows starting from the beginning of the part 2024.10.22 00:31:57.213496 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_167_167_0, total 71 rows starting from the beginning of the part 2024.10.22 00:31:57.232244 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_168_168_0, total 21 rows starting from the beginning of the part 2024.10.22 00:31:57.251241 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_169_169_0, total 38 rows starting from the beginning of the part 2024.10.22 00:31:57.274783 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_170_170_0, total 49 rows starting from the beginning of the part 2024.10.22 00:31:57.285432 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_171_171_0, total 23 rows starting from the beginning of the part 2024.10.22 00:31:57.854784 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 1292 2024.10.22 00:31:58.146925 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:31:58.151219 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:31:58.298923 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_91_91_0 (state Temporary) 2024.10.22 00:31:58.367798 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_91_91_0 to 202410_104_104_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:58.525770 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 8.42 MiB, will set to 1.57 GiB (RSS), difference: -3.20 MiB 2024.10.22 00:31:58.532164 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} MergeTask::MergeProjectionsStage: Merge sorted 7436 rows, containing 13 columns (13 merged, 0 gathered) in 1.499295642 sec., 4959.662251856262 rows/sec., 764.91 KiB/sec. 2024.10.22 00:31:58.544575 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1292 2024.10.22 00:31:58.723807 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} MergedBlockOutputStream: filled checksums 202410_1_171_34 (state Temporary) 2024.10.22 00:31:58.781481 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_171_34 to 202410_1_171_34 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:31:58.785120 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_171_34} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_166_33, 202410_171_171_0] -> 202410_1_171_34 2024.10.22 00:31:58.828103 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.78 MiB. 2024.10.22 00:31:59.472974 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:31:59.474945 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:31:59.991463 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 158748 2024.10.22 00:32:00.003354 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 91 2024.10.22 00:32:00.535578 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 7.83 MiB, will set to 1.57 GiB (RSS), difference: 2.00 MiB 2024.10.22 00:32:00.579518 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:00.586798 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:00.663815 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_73_73_0 (state Temporary) 2024.10.22 00:32:00.880564 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_73_73_0 to 202410_83_83_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:00.918484 [ 1267 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_78_16 to 202410_83_83_0 2024.10.22 00:32:00.935454 [ 1267 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:00.941215 [ 1267 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:00.967326 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:00.970709 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:01.062502 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_78_16 to 202410_83_83_0 into Compact with storage Full 2024.10.22 00:32:01.079268 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 6654 2024.10.22 00:32:01.086274 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 91 2024.10.22 00:32:01.216428 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_88_88_0 (state Temporary) 2024.10.22 00:32:01.292576 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_88_88_0 to 202410_95_95_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:01.323182 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 158748 2024.10.22 00:32:01.395312 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:32:01.524057 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:01.524327 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_78_16, total 122 rows starting from the beginning of the part 2024.10.22 00:32:01.530746 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:01.588089 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_159_159_0 (state Temporary) 2024.10.22 00:32:01.703339 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_79_79_0, total 1 rows starting from the beginning of the part 2024.10.22 00:32:01.732580 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_159_159_0 to 202410_172_172_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:01.780510 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6654 2024.10.22 00:32:01.784649 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_80_80_0, total 1 rows starting from the beginning of the part 2024.10.22 00:32:01.958786 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_81_81_0, total 1 rows starting from the beginning of the part 2024.10.22 00:32:02.023295 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_82_82_0, total 1 rows starting from the beginning of the part 2024.10.22 00:32:02.184414 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_83_83_0, total 1 rows starting from the beginning of the part 2024.10.22 00:32:02.490975 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_19_4 to 202410_1_19_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:02.504113 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_19_4} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_14_3, 202410_19_19_0] -> 202410_1_19_4 2024.10.22 00:32:02.590798 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 7.07 MiB, will set to 1.57 GiB (RSS), difference: -2.07 MiB 2024.10.22 00:32:02.719714 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.043508462 sec. 2024.10.22 00:32:02.925623 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.74 MiB. 2024.10.22 00:32:03.757104 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_83_17/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:32:04.057281 [ 1154 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:32:04.081234 [ 1154 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:32:04.165436 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} MergeTask::MergeProjectionsStage: Merge sorted 127 rows, containing 32 columns (32 merged, 0 gathered) in 3.215491384 sec., 39.49629615925601 rows/sec., 5.36 KiB/sec. 2024.10.22 00:32:04.181585 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} MergedBlockOutputStream: filled checksums 202410_1_83_17 (state Temporary) 2024.10.22 00:32:04.182329 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 6165 2024.10.22 00:32:04.504297 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:04.611535 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:04.778851 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_83_17 to 202410_1_83_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:04.783498 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_91_91_0 (state Temporary) 2024.10.22 00:32:04.882786 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_83_17} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_78_16, 202410_83_83_0] -> 202410_1_83_17 2024.10.22 00:32:05.058836 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 4.00 MiB, will set to 1.57 GiB (RSS), difference: 2.10 MiB 2024.10.22 00:32:05.165589 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_91_91_0 to 202410_97_97_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:05.223578 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6165 2024.10.22 00:32:05.287104 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:32:05.784711 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 6692 2024.10.22 00:32:05.876648 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_30_6, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0] 2024.10.22 00:32:05.879624 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_30_6, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0] 2024.10.22 00:32:05.915697 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:05.918434 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:05.970501 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_160_160_0 (state Temporary) 2024.10.22 00:32:06.002952 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_30_6, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0, 202410_35_35_0] 2024.10.22 00:32:06.023895 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_160_160_0 to 202410_173_173_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:06.047556 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 1319 2024.10.22 00:32:06.295527 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:06.298686 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:06.361352 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6692 2024.10.22 00:32:06.432117 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_92_92_0 (state Temporary) 2024.10.22 00:32:06.716469 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_92_92_0 to 202410_105_105_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:06.786728 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1319 2024.10.22 00:32:07.020828 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 259 ms. 2024.10.22 00:32:07.895240 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:32:08.428384 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 160284 2024.10.22 00:32:08.770492 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 3.33 MiB, will set to 1.57 GiB (RSS), difference: 3.48 MiB 2024.10.22 00:32:09.402640 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:09.413332 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:09.474734 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:32:09.578768 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:32:09.628526 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_89_89_0 (state Temporary) 2024.10.22 00:32:09.847972 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:09.852197 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:09.896060 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_89_89_0 to 202410_96_96_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:10.054866 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 160284 2024.10.22 00:32:10.156647 [ 1189 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_91_18 to 202410_96_96_0 2024.10.22 00:32:10.181608 [ 1189 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:10.187367 [ 1189 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:10.306771 [ 1189 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 277 ms. 2024.10.22 00:32:10.323093 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_91_18 to 202410_96_96_0 into Compact with storage Full 2024.10.22 00:32:10.367180 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:32:10.367719 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 6725 2024.10.22 00:32:10.372620 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} MergeTreeSequentialSource: Reading 22 marks from part 202410_1_91_18, total 165426 rows starting from the beginning of the part 2024.10.22 00:32:10.386315 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.81 GiB, free memory in arenas 3.21 MiB, will set to 1.57 GiB (RSS), difference: 1.79 MiB 2024.10.22 00:32:10.480837 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_92_92_0, total 1713 rows starting from the beginning of the part 2024.10.22 00:32:10.491420 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_93_93_0, total 1743 rows starting from the beginning of the part 2024.10.22 00:32:10.505424 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_94_94_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:32:10.516292 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:10.524423 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:10.526147 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_95_95_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:32:10.658489 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_96_96_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:32:10.771830 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_161_161_0 (state Temporary) 2024.10.22 00:32:10.833574 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_161_161_0 to 202410_174_174_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:10.882436 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6725 2024.10.22 00:32:11.597338 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_66_13, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0, 202410_71_71_0] 2024.10.22 00:32:11.601646 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_66_13, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0, 202410_71_71_0] 2024.10.22 00:32:11.834954 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_66_13, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0, 202410_71_71_0] 2024.10.22 00:32:12.486663 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 7.63 MiB, will set to 1.57 GiB (RSS), difference: -8.68 MiB 2024.10.22 00:32:12.747087 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 40 entries to flush up to offset 6205 2024.10.22 00:32:12.966730 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_19_19_0 (state Temporary) 2024.10.22 00:32:12.974145 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:12.975495 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:13.109476 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_92_92_0 (state Temporary) 2024.10.22 00:32:13.368324 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_92_92_0 to 202410_98_98_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:13.399244 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_41_8, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 00:32:13.402153 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_41_8, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 00:32:13.418271 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6205 2024.10.22 00:32:13.437528 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_41_8, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 00:32:14.292473 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 4 entries to flush up to offset 1323 2024.10.22 00:32:14.540775 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:14.545592 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:14.572309 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 9.29 MiB, will set to 1.57 GiB (RSS), difference: -3.64 MiB 2024.10.22 00:32:14.789075 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_93_93_0 (state Temporary) 2024.10.22 00:32:14.887414 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 6758 2024.10.22 00:32:14.951281 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_93_93_0 to 202410_106_106_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:14.998175 [ 1213 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_101_20 to 202410_106_106_0 2024.10.22 00:32:15.013638 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1323 2024.10.22 00:32:15.016156 [ 1213 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:15.025423 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:15.045092 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_101_20 to 202410_106_106_0 into Compact with storage Full 2024.10.22 00:32:15.080117 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:32:15.087021 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_101_20, total 1359 rows starting from the beginning of the part 2024.10.22 00:32:15.100405 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:15.110985 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:15.112466 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_102_102_0, total 20 rows starting from the beginning of the part 2024.10.22 00:32:15.157743 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_103_103_0, total 13 rows starting from the beginning of the part 2024.10.22 00:32:15.167137 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_162_162_0 (state Temporary) 2024.10.22 00:32:15.188657 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_104_104_0, total 6 rows starting from the beginning of the part 2024.10.22 00:32:15.216213 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_105_105_0, total 27 rows starting from the beginning of the part 2024.10.22 00:32:15.266859 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_106_106_0, total 4 rows starting from the beginning of the part 2024.10.22 00:32:15.523828 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_162_162_0 to 202410_175_175_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:15.718040 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6758 2024.10.22 00:32:15.995118 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_96_19/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:32:16.088509 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 92 2024.10.22 00:32:16.266706 [ 1183 ] {} DatabaseCatalog: Execution took 265 ms. 2024.10.22 00:32:16.486396 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} MergeTask::MergeProjectionsStage: Merge sorted 1429 rows, containing 12 columns (12 merged, 0 gathered) in 1.450121495 sec., 985.4346721479361 rows/sec., 151.51 KiB/sec. 2024.10.22 00:32:16.540494 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} MergedBlockOutputStream: filled checksums 202410_1_106_21 (state Temporary) 2024.10.22 00:32:16.593302 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 9.43 MiB, will set to 1.57 GiB (RSS), difference: -4.45 MiB 2024.10.22 00:32:16.600913 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:16.707065 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:16.779571 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_74_74_0 (state Temporary) 2024.10.22 00:32:16.896056 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_106_21 to 202410_1_106_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:16.903381 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_106_21} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_101_20, 202410_106_106_0] -> 202410_1_106_21 2024.10.22 00:32:16.950737 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:32:17.017119 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_74_74_0 to 202410_84_84_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:17.107117 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1741 entries to flush up to offset 162025 2024.10.22 00:32:17.337505 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 92 2024.10.22 00:32:17.880984 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} MergeTask::MergeProjectionsStage: Merge sorted 173874 rows, containing 4 columns (4 merged, 0 gathered) in 7.568087344 sec., 22974.629136362673 rows/sec., 337.63 KiB/sec. 2024.10.22 00:32:17.943155 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} MergedBlockOutputStream: filled checksums 202410_1_96_19 (state Temporary) 2024.10.22 00:32:18.192733 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_96_19 to 202410_1_96_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:18.216258 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_96_19} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_91_18, 202410_96_96_0] -> 202410_1_96_19 2024.10.22 00:32:18.219319 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:18.224573 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:18.463439 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:32:18.464539 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_90_90_0 (state Temporary) 2024.10.22 00:32:18.478582 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 10.44 MiB, will set to 1.57 GiB (RSS), difference: 13.24 MiB 2024.10.22 00:32:18.794802 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/bfq/zynblfbdreqbbjwegzpryojummamh. 2024.10.22 00:32:18.856064 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_90_90_0 to 202410_97_97_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:18.986425 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 162025 2024.10.22 00:32:19.085643 [ 1240 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:32:19.111278 [ 1240 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:32:19.475624 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:32:19.477376 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:32:19.732666 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 6808 2024.10.22 00:32:20.076302 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:20.078561 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:20.129205 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_163_163_0 (state Temporary) 2024.10.22 00:32:20.172543 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_163_163_0 to 202410_176_176_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:20.192807 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_171_34 to 202410_176_176_0 2024.10.22 00:32:20.211880 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:20.219646 [ 1247 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:20.234197 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6808 2024.10.22 00:32:20.272391 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_171_34 to 202410_176_176_0 into Compact with storage Full 2024.10.22 00:32:20.295123 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:32:20.314462 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_171_34, total 7436 rows starting from the beginning of the part 2024.10.22 00:32:20.328725 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_172_172_0, total 49 rows starting from the beginning of the part 2024.10.22 00:32:20.350371 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_173_173_0, total 38 rows starting from the beginning of the part 2024.10.22 00:32:20.365327 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_174_174_0, total 33 rows starting from the beginning of the part 2024.10.22 00:32:20.463148 [ 1243 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 00:32:20.477303 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_175_175_0, total 33 rows starting from the beginning of the part 2024.10.22 00:32:20.477752 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.81 GiB, free memory in arenas 9.11 MiB, will set to 1.57 GiB (RSS), difference: 85.15 MiB 2024.10.22 00:32:20.496015 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_176_176_0, total 50 rows starting from the beginning of the part 2024.10.22 00:32:20.888313 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:32:20.925343 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 47 entries to flush up to offset 6252 2024.10.22 00:32:21.130409 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:21.133554 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:21.180927 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_93_93_0 (state Temporary) 2024.10.22 00:32:21.330533 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_93_93_0 to 202410_99_99_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:21.383913 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6252 2024.10.22 00:32:21.688924 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} MergeTask::MergeProjectionsStage: Merge sorted 7639 rows, containing 13 columns (13 merged, 0 gathered) in 1.422036363 sec., 5371.873883649768 rows/sec., 827.30 KiB/sec. 2024.10.22 00:32:21.979935 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} MergedBlockOutputStream: filled checksums 202410_1_176_35 (state Temporary) 2024.10.22 00:32:22.162811 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_176_35 to 202410_1_176_35 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:22.167162 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_176_35} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_171_34, 202410_176_176_0] -> 202410_1_176_35 2024.10.22 00:32:22.200849 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.80 MiB. 2024.10.22 00:32:22.517392 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 1350 2024.10.22 00:32:22.604062 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.81 GiB, free memory in arenas 9.36 MiB, will set to 1.57 GiB (RSS), difference: 105.79 MiB 2024.10.22 00:32:22.816303 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:22.821103 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:22.863680 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_94_94_0 (state Temporary) 2024.10.22 00:32:22.893937 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_94_94_0 to 202410_107_107_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:22.949118 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1350 2024.10.22 00:32:23.667093 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_19_19_0 to 202410_22_22_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:24.055335 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 948078 bytes for 0bcc259e2342d61aa7a94e549f9f8a00:0) 2024.10.22 00:32:24.058821 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c707f04a8064ce6b4360b282d5c5a0d1, offset: 0, size: 1549222 2024.10.22 00:32:24.064635 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c707f04a8064ce6b4360b282d5c5a0d1 for removal 2024.10.22 00:32:24.068194 [ 1476 ] {} CacheMetadata: Key c707f04a8064ce6b4360b282d5c5a0d1 is removed from metadata 2024.10.22 00:32:24.068412 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ea695be58b18929001804edf3f2246b7, offset: 0, size: 3516 2024.10.22 00:32:24.072387 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ea695be58b18929001804edf3f2246b7 for removal 2024.10.22 00:32:24.075400 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 56ae9251aa51a37d167080a7b6b1dfa1, offset: 0, size: 154 2024.10.22 00:32:24.079030 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 56ae9251aa51a37d167080a7b6b1dfa1 for removal 2024.10.22 00:32:24.080980 [ 1476 ] {} CacheMetadata: Key ea695be58b18929001804edf3f2246b7 is removed from metadata 2024.10.22 00:32:24.083101 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d3363fc31c712c9030edb9757b371d7c, offset: 0, size: 201 2024.10.22 00:32:24.086208 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d3363fc31c712c9030edb9757b371d7c for removal 2024.10.22 00:32:24.087456 [ 1476 ] {} CacheMetadata: Key 56ae9251aa51a37d167080a7b6b1dfa1 is removed from metadata 2024.10.22 00:32:24.089336 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 580224acb25c79934e2b074325ee9280, offset: 0, size: 72751 2024.10.22 00:32:24.111323 [ 1476 ] {} CacheMetadata: Key d3363fc31c712c9030edb9757b371d7c is removed from metadata 2024.10.22 00:32:24.199344 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 580224acb25c79934e2b074325ee9280 for removal 2024.10.22 00:32:24.206141 [ 1476 ] {} CacheMetadata: Key 580224acb25c79934e2b074325ee9280 is removed from metadata 2024.10.22 00:32:24.207544 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a4abd27d826fd3a2cabe6d8072b0135e, offset: 0, size: 146 2024.10.22 00:32:24.212099 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a4abd27d826fd3a2cabe6d8072b0135e for removal 2024.10.22 00:32:24.215057 [ 1476 ] {} CacheMetadata: Key a4abd27d826fd3a2cabe6d8072b0135e is removed from metadata 2024.10.22 00:32:24.243660 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 6866 2024.10.22 00:32:24.448993 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:24.452946 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:24.663467 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_35_7, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0] 2024.10.22 00:32:24.665997 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_35_7, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0] 2024.10.22 00:32:24.690349 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 9.11 MiB, will set to 1.56 GiB (RSS), difference: 20.54 MiB 2024.10.22 00:32:24.700884 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_164_164_0 (state Temporary) 2024.10.22 00:32:24.754590 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_164_164_0 to 202410_177_177_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:24.818828 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6866 2024.10.22 00:32:24.843927 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 93 2024.10.22 00:32:24.889808 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_35_7, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0] 2024.10.22 00:32:25.199890 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:25.207390 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:25.570936 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_75_75_0 (state Temporary) 2024.10.22 00:32:25.990713 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1715 entries to flush up to offset 163740 2024.10.22 00:32:26.054423 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_75_75_0 to 202410_85_85_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:26.105234 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 93 2024.10.22 00:32:26.487211 [ 1272 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 00:32:26.651568 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 9.05 MiB, will set to 1.56 GiB (RSS), difference: -2.79 MiB 2024.10.22 00:32:27.268858 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:27.271712 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:27.455498 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_91_91_0 (state Temporary) 2024.10.22 00:32:27.619223 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_91_91_0 to 202410_98_98_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:27.663583 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 163740 2024.10.22 00:32:27.687061 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 729 2024.10.22 00:32:27.778750 [ 1263 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 00:32:28.831862 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 6892 2024.10.22 00:32:28.872679 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.81 GiB, free memory in arenas 10.47 MiB, will set to 1.55 GiB (RSS), difference: 3.17 MiB 2024.10.22 00:32:28.890036 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 21 entries to flush up to offset 6273 2024.10.22 00:32:29.133813 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:29.140519 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:29.182916 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_94_94_0 (state Temporary) 2024.10.22 00:32:29.223188 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:29.228695 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:29.233178 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_94_94_0 to 202410_100_100_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:29.277727 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_165_165_0 (state Temporary) 2024.10.22 00:32:29.283230 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_95_19 to 202410_100_100_0 2024.10.22 00:32:29.318669 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_165_165_0 to 202410_178_178_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:29.323766 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6273 2024.10.22 00:32:29.406369 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:29.411112 [ 1176 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:29.430451 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_95_19 to 202410_100_100_0 into Compact with storage Full 2024.10.22 00:32:29.475260 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:32:29.476109 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6892 2024.10.22 00:32:29.479155 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:32:29.485674 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:32:29.488778 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_95_19, total 6173 rows starting from the beginning of the part 2024.10.22 00:32:29.604886 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_96_96_0, total 46 rows starting from the beginning of the part 2024.10.22 00:32:29.622970 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_97_97_0, total 50 rows starting from the beginning of the part 2024.10.22 00:32:29.648799 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_98_98_0, total 40 rows starting from the beginning of the part 2024.10.22 00:32:29.672567 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_99_99_0, total 47 rows starting from the beginning of the part 2024.10.22 00:32:29.687804 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_100_100_0, total 21 rows starting from the beginning of the part 2024.10.22 00:32:30.270871 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 10 file segments (while reserving 963962 bytes for bc4d99afb18560565cf3a508d873a38c:0) 2024.10.22 00:32:30.272787 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5be898308263fd0d7124b270442f02b3, offset: 0, size: 648126 2024.10.22 00:32:30.277214 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5be898308263fd0d7124b270442f02b3 for removal 2024.10.22 00:32:30.280974 [ 1476 ] {} CacheMetadata: Key 5be898308263fd0d7124b270442f02b3 is removed from metadata 2024.10.22 00:32:30.280742 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4185ec02c3a2abf39fa86fb3aed0d300, offset: 0, size: 230 2024.10.22 00:32:30.284775 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4185ec02c3a2abf39fa86fb3aed0d300 for removal 2024.10.22 00:32:30.287584 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1fa490e94f84421269a77acd2c1b297f, offset: 0, size: 439560 2024.10.22 00:32:30.290581 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1fa490e94f84421269a77acd2c1b297f for removal 2024.10.22 00:32:30.292115 [ 1476 ] {} CacheMetadata: Key 4185ec02c3a2abf39fa86fb3aed0d300 is removed from metadata 2024.10.22 00:32:30.293184 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ebe06e90e7e2495bc6a180ef32b5354b, offset: 0, size: 17280 2024.10.22 00:32:30.297766 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ebe06e90e7e2495bc6a180ef32b5354b for removal 2024.10.22 00:32:30.299981 [ 1476 ] {} CacheMetadata: Key 1fa490e94f84421269a77acd2c1b297f is removed from metadata 2024.10.22 00:32:30.311408 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e0f585b27e0fe8b971ed656fe3794635, offset: 0, size: 176 2024.10.22 00:32:30.317304 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e0f585b27e0fe8b971ed656fe3794635 for removal 2024.10.22 00:32:30.320969 [ 1476 ] {} CacheMetadata: Key ebe06e90e7e2495bc6a180ef32b5354b is removed from metadata 2024.10.22 00:32:30.321102 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5ea7b5be3fbeeb40e2f0f051fc8d09df, offset: 0, size: 183 2024.10.22 00:32:30.328299 [ 1476 ] {} CacheMetadata: Key e0f585b27e0fe8b971ed656fe3794635 is removed from metadata 2024.10.22 00:32:30.329553 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5ea7b5be3fbeeb40e2f0f051fc8d09df for removal 2024.10.22 00:32:30.337016 [ 1476 ] {} CacheMetadata: Key 5ea7b5be3fbeeb40e2f0f051fc8d09df is removed from metadata 2024.10.22 00:32:30.341439 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 06ba3f6cc44aeeb95fab2c7f1075ef1d, offset: 0, size: 27711 2024.10.22 00:32:30.347993 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 06ba3f6cc44aeeb95fab2c7f1075ef1d for removal 2024.10.22 00:32:30.353514 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: baed1271867a864b19c2ea76c9974e4c, offset: 0, size: 294061 2024.10.22 00:32:30.357722 [ 1476 ] {} CacheMetadata: Key 06ba3f6cc44aeeb95fab2c7f1075ef1d is removed from metadata 2024.10.22 00:32:30.358163 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key baed1271867a864b19c2ea76c9974e4c for removal 2024.10.22 00:32:30.363356 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7809d1ea24e155c6a8e1b2243dcd32f7, offset: 0, size: 186 2024.10.22 00:32:30.368653 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7809d1ea24e155c6a8e1b2243dcd32f7 for removal 2024.10.22 00:32:30.377171 [ 1476 ] {} CacheMetadata: Key 7809d1ea24e155c6a8e1b2243dcd32f7 is removed from metadata 2024.10.22 00:32:30.372206 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 11554a5c8f8ca3378795e9b29ad9786f, offset: 0, size: 158 2024.10.22 00:32:30.382590 [ 1476 ] {} CacheMetadata: Key baed1271867a864b19c2ea76c9974e4c is removed from metadata 2024.10.22 00:32:30.386702 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 11554a5c8f8ca3378795e9b29ad9786f for removal 2024.10.22 00:32:30.391856 [ 1476 ] {} CacheMetadata: Key 11554a5c8f8ca3378795e9b29ad9786f is removed from metadata 2024.10.22 00:32:30.452861 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 1357 2024.10.22 00:32:30.772853 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:30.775682 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:30.807767 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_95_95_0 (state Temporary) 2024.10.22 00:32:30.874814 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_95_95_0 to 202410_108_108_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:31.016016 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1357 2024.10.22 00:32:31.093569 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} MergeTask::MergeProjectionsStage: Merge sorted 6377 rows, containing 13 columns (13 merged, 0 gathered) in 1.671337178 sec., 3815.507776612147 rows/sec., 1.59 MiB/sec. 2024.10.22 00:32:31.283511 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} MergedBlockOutputStream: filled checksums 202410_1_100_20 (state Temporary) 2024.10.22 00:32:31.350981 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_100_20 to 202410_1_100_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:31.356081 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_100_20} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_95_19, 202410_100_100_0] -> 202410_1_100_20 2024.10.22 00:32:31.491622 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.29 MiB. 2024.10.22 00:32:33.000998 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 12 file segments (while reserving 949412 bytes for e063039339d57402b8f24ef2fb1307a0:0) 2024.10.22 00:32:33.005073 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: abf6f54e4db9216857dbd9de51587cf6, offset: 0, size: 94103 2024.10.22 00:32:33.009442 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key abf6f54e4db9216857dbd9de51587cf6 for removal 2024.10.22 00:32:33.013235 [ 1476 ] {} CacheMetadata: Key abf6f54e4db9216857dbd9de51587cf6 is removed from metadata 2024.10.22 00:32:33.013627 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0bcb215fd68d772b68429a9183ddee61, offset: 0, size: 145 2024.10.22 00:32:33.019210 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0bcb215fd68d772b68429a9183ddee61 for removal 2024.10.22 00:32:33.022452 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4ea221c5f3ea790e10fda947b2933849, offset: 0, size: 161 2024.10.22 00:32:33.025324 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4ea221c5f3ea790e10fda947b2933849 for removal 2024.10.22 00:32:33.028420 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0c14e7d5d09834ecb3083e89b9a0389a, offset: 0, size: 145 2024.10.22 00:32:33.029302 [ 1476 ] {} CacheMetadata: Key 4ea221c5f3ea790e10fda947b2933849 is removed from metadata 2024.10.22 00:32:33.032962 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0c14e7d5d09834ecb3083e89b9a0389a for removal 2024.10.22 00:32:33.139117 [ 1476 ] {} CacheMetadata: Key 0bcb215fd68d772b68429a9183ddee61 is removed from metadata 2024.10.22 00:32:33.140012 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e6a867427fdd86b3ec5f0ea3d991bc0b, offset: 0, size: 104358 2024.10.22 00:32:33.144611 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e6a867427fdd86b3ec5f0ea3d991bc0b for removal 2024.10.22 00:32:33.149073 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5af5e02aabc5b8a972e527fd477ebad7, offset: 0, size: 189182 2024.10.22 00:32:33.153449 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5af5e02aabc5b8a972e527fd477ebad7 for removal 2024.10.22 00:32:33.158475 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 48b73727ac77949d91832fbf9b01331c, offset: 0, size: 122248 2024.10.22 00:32:33.161380 [ 1476 ] {} CacheMetadata: Key 5af5e02aabc5b8a972e527fd477ebad7 is removed from metadata 2024.10.22 00:32:33.163136 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 48b73727ac77949d91832fbf9b01331c for removal 2024.10.22 00:32:33.169224 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 62757405283f54849784595f87794d59, offset: 0, size: 189067 2024.10.22 00:32:33.174351 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 62757405283f54849784595f87794d59 for removal 2024.10.22 00:32:33.179992 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 400236102ecdfce0f3dd9c241bc25958, offset: 0, size: 158 2024.10.22 00:32:33.185106 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 400236102ecdfce0f3dd9c241bc25958 for removal 2024.10.22 00:32:33.187334 [ 1476 ] {} CacheMetadata: Key 62757405283f54849784595f87794d59 is removed from metadata 2024.10.22 00:32:33.197091 [ 1476 ] {} CacheMetadata: Key e6a867427fdd86b3ec5f0ea3d991bc0b is removed from metadata 2024.10.22 00:32:33.211242 [ 1476 ] {} CacheMetadata: Key 48b73727ac77949d91832fbf9b01331c is removed from metadata 2024.10.22 00:32:33.218692 [ 1476 ] {} CacheMetadata: Key 0c14e7d5d09834ecb3083e89b9a0389a is removed from metadata 2024.10.22 00:32:33.288271 [ 1476 ] {} CacheMetadata: Key 400236102ecdfce0f3dd9c241bc25958 is removed from metadata 2024.10.22 00:32:33.288788 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1a7aa139eb0784c4766679792e47bdcb, offset: 0, size: 5380 2024.10.22 00:32:33.293034 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1a7aa139eb0784c4766679792e47bdcb for removal 2024.10.22 00:32:33.303158 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 482fdbcdc48ea81ae73e4942d84fe9b1, offset: 0, size: 145 2024.10.22 00:32:33.310526 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 482fdbcdc48ea81ae73e4942d84fe9b1 for removal 2024.10.22 00:32:33.311567 [ 1476 ] {} CacheMetadata: Key 1a7aa139eb0784c4766679792e47bdcb is removed from metadata 2024.10.22 00:32:33.313387 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 785e7950b3203354b58f15c98f52c704, offset: 0, size: 144 2024.10.22 00:32:33.321257 [ 1476 ] {} CacheMetadata: Key 482fdbcdc48ea81ae73e4942d84fe9b1 is removed from metadata 2024.10.22 00:32:33.321407 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 785e7950b3203354b58f15c98f52c704 for removal 2024.10.22 00:32:33.336139 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_36_7, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0] 2024.10.22 00:32:33.336722 [ 1476 ] {} CacheMetadata: Key 785e7950b3203354b58f15c98f52c704 is removed from metadata 2024.10.22 00:32:33.343760 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_36_7, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0] 2024.10.22 00:32:33.482434 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 107 entries to flush up to offset 6999 2024.10.22 00:32:33.483792 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_36_7, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0] 2024.10.22 00:32:33.611266 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 94 2024.10.22 00:32:33.672443 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 34378 bytes for b740cb8bdf952b093089306894ad67bc:0) 2024.10.22 00:32:33.675069 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: be1989143a1ff8427617888bb9f01eed, offset: 0, size: 86419 2024.10.22 00:32:33.677306 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key be1989143a1ff8427617888bb9f01eed for removal 2024.10.22 00:32:33.680815 [ 1476 ] {} CacheMetadata: Key be1989143a1ff8427617888bb9f01eed is removed from metadata 2024.10.22 00:32:33.682859 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c7e12d158e0bffc9a97a276c4f6fbe91, offset: 0, size: 145 2024.10.22 00:32:33.686627 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c7e12d158e0bffc9a97a276c4f6fbe91 for removal 2024.10.22 00:32:33.688899 [ 1476 ] {} CacheMetadata: Key c7e12d158e0bffc9a97a276c4f6fbe91 is removed from metadata 2024.10.22 00:32:33.900818 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:33.904078 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:34.118661 [ 1173 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:32:34.157299 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_166_166_0 (state Temporary) 2024.10.22 00:32:34.181303 [ 1173 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:32:34.283187 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_166_166_0 to 202410_179_179_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:34.373011 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 35839 bytes for 0e6ec3c18eaab6e71acc14ab6833a086:0) 2024.10.22 00:32:34.374482 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b9b2034184f6bee7d951c5b888ab1216, offset: 0, size: 56367 2024.10.22 00:32:34.380766 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b9b2034184f6bee7d951c5b888ab1216 for removal 2024.10.22 00:32:34.385126 [ 1476 ] {} CacheMetadata: Key b9b2034184f6bee7d951c5b888ab1216 is removed from metadata 2024.10.22 00:32:34.385580 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 72c9159f0dc81ed1131852250c593388, offset: 0, size: 144 2024.10.22 00:32:34.488790 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 72c9159f0dc81ed1131852250c593388 for removal 2024.10.22 00:32:34.491204 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 6.29 MiB, will set to 1.55 GiB (RSS), difference: -4.30 MiB 2024.10.22 00:32:34.492889 [ 1476 ] {} CacheMetadata: Key 72c9159f0dc81ed1131852250c593388 is removed from metadata 2024.10.22 00:32:34.521406 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:34.525254 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:34.640524 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 6999 2024.10.22 00:32:34.667438 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 164892 2024.10.22 00:32:34.674811 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_76_76_0 (state Temporary) 2024.10.22 00:32:34.864147 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_76_76_0 to 202410_86_86_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:34.936049 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 94 2024.10.22 00:32:35.192043 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 72482 bytes for 2fffc05cae0424f8187591f599f4323b:0) 2024.10.22 00:32:35.194887 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f929a37d20e88fe62cdc454190390c10, offset: 0, size: 185307 2024.10.22 00:32:35.198121 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f929a37d20e88fe62cdc454190390c10 for removal 2024.10.22 00:32:35.201246 [ 1476 ] {} CacheMetadata: Key f929a37d20e88fe62cdc454190390c10 is removed from metadata 2024.10.22 00:32:35.202353 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cd97ebd86f5d5e6f5aff6c0f62fd263d, offset: 0, size: 155 2024.10.22 00:32:35.210562 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cd97ebd86f5d5e6f5aff6c0f62fd263d for removal 2024.10.22 00:32:35.224667 [ 1476 ] {} CacheMetadata: Key cd97ebd86f5d5e6f5aff6c0f62fd263d is removed from metadata 2024.10.22 00:32:35.231931 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 45 entries to flush up to offset 774 2024.10.22 00:32:35.673577 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:35.679647 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:35.686432 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_71_14, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0, 202410_76_76_0] 2024.10.22 00:32:35.690723 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_71_14, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0, 202410_76_76_0] 2024.10.22 00:32:35.722992 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_71_14, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0, 202410_76_76_0] 2024.10.22 00:32:36.322646 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_92_92_0 (state Temporary) 2024.10.22 00:32:36.348556 [ 1212 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 00:32:36.513837 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_92_92_0 to 202410_99_99_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:36.556337 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 164892 2024.10.22 00:32:36.636877 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.81 GiB, free memory in arenas 6.32 MiB, will set to 1.55 GiB (RSS), difference: 3.99 MiB 2024.10.22 00:32:36.667278 [ 1216 ] {} DatabaseCatalog: Execution took 216 ms. 2024.10.22 00:32:36.831048 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 34 entries to flush up to offset 6307 2024.10.22 00:32:37.277552 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:37.279886 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 72053 bytes for f355cfa527c1d9645f22bb2c09b082b3:0) 2024.10.22 00:32:37.280773 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:37.281798 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9b8170c11a5c09f1337a6dc6c3e7aebb, offset: 0, size: 2411 2024.10.22 00:32:37.284671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9b8170c11a5c09f1337a6dc6c3e7aebb for removal 2024.10.22 00:32:37.286120 [ 1476 ] {} CacheMetadata: Key 9b8170c11a5c09f1337a6dc6c3e7aebb is removed from metadata 2024.10.22 00:32:37.286884 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8a031c024e0e99fff2406d5146bf59ae, offset: 0, size: 144 2024.10.22 00:32:37.289013 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8a031c024e0e99fff2406d5146bf59ae for removal 2024.10.22 00:32:37.303685 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_95_95_0 (state Temporary) 2024.10.22 00:32:37.311234 [ 1476 ] {} CacheMetadata: Key 8a031c024e0e99fff2406d5146bf59ae is removed from metadata 2024.10.22 00:32:37.354935 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_95_95_0 to 202410_101_101_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:37.497067 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6307 2024.10.22 00:32:37.678081 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 75761 bytes for 3f986617f6cbae070648738b65865e75:0) 2024.10.22 00:32:37.680009 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5647ea269b176d9cdf31ec3a6b5e98dd, offset: 0, size: 144 2024.10.22 00:32:37.682477 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5647ea269b176d9cdf31ec3a6b5e98dd for removal 2024.10.22 00:32:37.685108 [ 1476 ] {} CacheMetadata: Key 5647ea269b176d9cdf31ec3a6b5e98dd is removed from metadata 2024.10.22 00:32:37.686441 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 479de26efed97b55fe39590f81432023, offset: 0, size: 37952 2024.10.22 00:32:37.689371 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 479de26efed97b55fe39590f81432023 for removal 2024.10.22 00:32:37.692455 [ 1476 ] {} CacheMetadata: Key 479de26efed97b55fe39590f81432023 is removed from metadata 2024.10.22 00:32:37.693169 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 74d1bc6cd15dc4c1174db17579f4ad61, offset: 0, size: 42662 2024.10.22 00:32:37.697087 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 74d1bc6cd15dc4c1174db17579f4ad61 for removal 2024.10.22 00:32:37.709337 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 820af61752f408acf679e0863b1ae9e2, offset: 0, size: 2805 2024.10.22 00:32:37.714405 [ 1476 ] {} CacheMetadata: Key 74d1bc6cd15dc4c1174db17579f4ad61 is removed from metadata 2024.10.22 00:32:37.815663 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 820af61752f408acf679e0863b1ae9e2 for removal 2024.10.22 00:32:37.819052 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d3a3a7224821af218a1f9ce0a6b2dc91, offset: 0, size: 144 2024.10.22 00:32:37.819398 [ 1476 ] {} CacheMetadata: Key 820af61752f408acf679e0863b1ae9e2 is removed from metadata 2024.10.22 00:32:37.821386 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d3a3a7224821af218a1f9ce0a6b2dc91 for removal 2024.10.22 00:32:37.824144 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1f1365f4f24a3070db437cdb88ffde5a, offset: 0, size: 144 2024.10.22 00:32:37.827906 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1f1365f4f24a3070db437cdb88ffde5a for removal 2024.10.22 00:32:37.927847 [ 1476 ] {} CacheMetadata: Key d3a3a7224821af218a1f9ce0a6b2dc91 is removed from metadata 2024.10.22 00:32:37.946269 [ 1476 ] {} CacheMetadata: Key 1f1365f4f24a3070db437cdb88ffde5a is removed from metadata 2024.10.22 00:32:38.160489 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 82610 bytes for 7afd549b433ed4bcb207afd76875794c:0) 2024.10.22 00:32:38.164859 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a43371e105144ab3caaecb749983c6a1, offset: 0, size: 335436 2024.10.22 00:32:38.169498 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a43371e105144ab3caaecb749983c6a1 for removal 2024.10.22 00:32:38.172504 [ 1476 ] {} CacheMetadata: Key a43371e105144ab3caaecb749983c6a1 is removed from metadata 2024.10.22 00:32:38.173071 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1ab5c7bbce96fdcb9acd53cc0cc83d83, offset: 0, size: 185 2024.10.22 00:32:38.176181 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1ab5c7bbce96fdcb9acd53cc0cc83d83 for removal 2024.10.22 00:32:38.181348 [ 1476 ] {} CacheMetadata: Key 1ab5c7bbce96fdcb9acd53cc0cc83d83 is removed from metadata 2024.10.22 00:32:38.522860 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 1369 2024.10.22 00:32:38.603286 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.81 GiB, free memory in arenas 4.57 MiB, will set to 1.56 GiB (RSS), difference: 1.05 MiB 2024.10.22 00:32:38.672112 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 91 entries to flush up to offset 7090 2024.10.22 00:32:38.761807 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:38.775609 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:38.914477 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_96_96_0 (state Temporary) 2024.10.22 00:32:38.973899 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:38.976364 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:38.977117 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 84026 bytes for 46eee315ef49f4017b0b9cd674208be6:0) 2024.10.22 00:32:38.980325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ff721979cb63c30f088bdb2a0b9de250, offset: 0, size: 162160 2024.10.22 00:32:38.986408 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ff721979cb63c30f088bdb2a0b9de250 for removal 2024.10.22 00:32:38.987900 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_96_96_0 to 202410_109_109_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:38.988408 [ 1476 ] {} CacheMetadata: Key ff721979cb63c30f088bdb2a0b9de250 is removed from metadata 2024.10.22 00:32:38.999163 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 12ef118a302fc249d006a0fdad306820, offset: 0, size: 147 2024.10.22 00:32:39.004659 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 12ef118a302fc249d006a0fdad306820 for removal 2024.10.22 00:32:39.009321 [ 1476 ] {} CacheMetadata: Key 12ef118a302fc249d006a0fdad306820 is removed from metadata 2024.10.22 00:32:39.139107 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1369 2024.10.22 00:32:39.144777 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_167_167_0 (state Temporary) 2024.10.22 00:32:39.266272 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_167_167_0 to 202410_180_180_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:39.307768 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7090 2024.10.22 00:32:39.476256 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:32:39.477795 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:32:39.899230 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 62294 bytes for 5724a01beb75ecf01f486bd68c9d3b3b:0) 2024.10.22 00:32:39.903875 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a77f85359f86edcc68f7681b5bcecd96, offset: 0, size: 83811 2024.10.22 00:32:39.911286 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a77f85359f86edcc68f7681b5bcecd96 for removal 2024.10.22 00:32:39.915650 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 282f9402ba23f73bf279b11dad8e4508, offset: 0, size: 156 2024.10.22 00:32:39.917135 [ 1476 ] {} CacheMetadata: Key a77f85359f86edcc68f7681b5bcecd96 is removed from metadata 2024.10.22 00:32:39.920178 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 282f9402ba23f73bf279b11dad8e4508 for removal 2024.10.22 00:32:39.926499 [ 1476 ] {} CacheMetadata: Key 282f9402ba23f73bf279b11dad8e4508 is removed from metadata 2024.10.22 00:32:40.163991 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 62658 bytes for 037ffea62a802d61ca935830ad39f72d:0) 2024.10.22 00:32:40.166737 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f848351b923954a612919bfdb0005e0b, offset: 0, size: 42325 2024.10.22 00:32:40.171885 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f848351b923954a612919bfdb0005e0b for removal 2024.10.22 00:32:40.174811 [ 1476 ] {} CacheMetadata: Key f848351b923954a612919bfdb0005e0b is removed from metadata 2024.10.22 00:32:40.175508 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 073e0f42c57a86c437e8f5e3b8823af4, offset: 0, size: 144 2024.10.22 00:32:40.180258 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 073e0f42c57a86c437e8f5e3b8823af4 for removal 2024.10.22 00:32:40.182471 [ 1476 ] {} CacheMetadata: Key 073e0f42c57a86c437e8f5e3b8823af4 is removed from metadata 2024.10.22 00:32:40.538847 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 52561 bytes for 54de15e0a232a94471bd180a57dfa53e:0) 2024.10.22 00:32:40.546675 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 53966c3c137a6ec48f10d26593ab0490, offset: 0, size: 271692 2024.10.22 00:32:40.550497 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 53966c3c137a6ec48f10d26593ab0490 for removal 2024.10.22 00:32:40.552980 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c57a3f3c793de51a08feafef07f14c42, offset: 0, size: 164 2024.10.22 00:32:40.553058 [ 1476 ] {} CacheMetadata: Key 53966c3c137a6ec48f10d26593ab0490 is removed from metadata 2024.10.22 00:32:40.557213 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c57a3f3c793de51a08feafef07f14c42 for removal 2024.10.22 00:32:40.561367 [ 1476 ] {} CacheMetadata: Key c57a3f3c793de51a08feafef07f14c42 is removed from metadata 2024.10.22 00:32:41.820363 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 236606 bytes for a6a165c0d77d77b701055e53eb5b8ed6:0) 2024.10.22 00:32:41.823386 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 52d4fee9bb58866359f32364b9fb0b59, offset: 0, size: 287647 2024.10.22 00:32:41.827305 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 52d4fee9bb58866359f32364b9fb0b59 for removal 2024.10.22 00:32:41.831529 [ 1476 ] {} CacheMetadata: Key 52d4fee9bb58866359f32364b9fb0b59 is removed from metadata 2024.10.22 00:32:41.832397 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 78713759ee77bd551757d26e9766a91b, offset: 0, size: 165 2024.10.22 00:32:41.846836 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 78713759ee77bd551757d26e9766a91b for removal 2024.10.22 00:32:41.872661 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:32:41.943047 [ 1476 ] {} CacheMetadata: Key 78713759ee77bd551757d26e9766a91b is removed from metadata 2024.10.22 00:32:42.198595 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 7bde876bfca3d8dea83a43bc5eefc71b:0) 2024.10.22 00:32:42.207329 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fb3312a52e1b9946aa191bbb94cb933a, offset: 0, size: 1553147 2024.10.22 00:32:42.214418 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fb3312a52e1b9946aa191bbb94cb933a for removal 2024.10.22 00:32:42.217017 [ 1476 ] {} CacheMetadata: Key fb3312a52e1b9946aa191bbb94cb933a is removed from metadata 2024.10.22 00:32:42.217653 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d3fb5aeb20edee218374dea5cb100f64, offset: 0, size: 201 2024.10.22 00:32:42.221366 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d3fb5aeb20edee218374dea5cb100f64 for removal 2024.10.22 00:32:42.224427 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d42418135c9603b1480d1f9853fe9669, offset: 0, size: 157967 2024.10.22 00:32:42.224627 [ 1476 ] {} CacheMetadata: Key d3fb5aeb20edee218374dea5cb100f64 is removed from metadata 2024.10.22 00:32:42.233081 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d42418135c9603b1480d1f9853fe9669 for removal 2024.10.22 00:32:42.238818 [ 1476 ] {} CacheMetadata: Key d42418135c9603b1480d1f9853fe9669 is removed from metadata 2024.10.22 00:32:42.241582 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1a226138fc62862b1c6e1c3dd48ee8cc, offset: 0, size: 158 2024.10.22 00:32:42.246800 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1a226138fc62862b1c6e1c3dd48ee8cc for removal 2024.10.22 00:32:42.355959 [ 1476 ] {} CacheMetadata: Key 1a226138fc62862b1c6e1c3dd48ee8cc is removed from metadata 2024.10.22 00:32:42.441341 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 95 2024.10.22 00:32:42.576681 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 5.01 MiB, will set to 1.55 GiB (RSS), difference: -2.15 MiB 2024.10.22 00:32:42.776062 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:42.848830 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 322124 bytes for c83b0ad8973ddd91dbf2f016a24b5667:0) 2024.10.22 00:32:42.852656 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 88d8290b9bc06c3bef0fc87beb16b430, offset: 0, size: 148994 2024.10.22 00:32:42.859343 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 88d8290b9bc06c3bef0fc87beb16b430 for removal 2024.10.22 00:32:42.863110 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f7635aca62b2315ab27aa478b3519dec, offset: 0, size: 180 2024.10.22 00:32:42.864217 [ 1476 ] {} CacheMetadata: Key 88d8290b9bc06c3bef0fc87beb16b430 is removed from metadata 2024.10.22 00:32:42.869201 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f7635aca62b2315ab27aa478b3519dec for removal 2024.10.22 00:32:42.872316 [ 1476 ] {} CacheMetadata: Key f7635aca62b2315ab27aa478b3519dec is removed from metadata 2024.10.22 00:32:42.872468 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a085e54a3f20b57cbf5cea2966319855, offset: 0, size: 146 2024.10.22 00:32:42.877087 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a085e54a3f20b57cbf5cea2966319855 for removal 2024.10.22 00:32:42.878943 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:42.880050 [ 1476 ] {} CacheMetadata: Key a085e54a3f20b57cbf5cea2966319855 is removed from metadata 2024.10.22 00:32:42.880695 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4baacefc9f93eccb2751babe907d809b, offset: 0, size: 73677 2024.10.22 00:32:42.884111 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4baacefc9f93eccb2751babe907d809b for removal 2024.10.22 00:32:42.887797 [ 1476 ] {} CacheMetadata: Key 4baacefc9f93eccb2751babe907d809b is removed from metadata 2024.10.22 00:32:42.888447 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dbee7f7a24260f877282ec67da14ce59, offset: 0, size: 84103 2024.10.22 00:32:42.893095 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dbee7f7a24260f877282ec67da14ce59 for removal 2024.10.22 00:32:42.899486 [ 1476 ] {} CacheMetadata: Key dbee7f7a24260f877282ec67da14ce59 is removed from metadata 2024.10.22 00:32:42.900109 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6a8cfb8a57fbd360b3af664f8fadc99c, offset: 0, size: 145 2024.10.22 00:32:42.911047 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6a8cfb8a57fbd360b3af664f8fadc99c for removal 2024.10.22 00:32:42.916922 [ 1476 ] {} CacheMetadata: Key 6a8cfb8a57fbd360b3af664f8fadc99c is removed from metadata 2024.10.22 00:32:43.057695 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_77_77_0 (state Temporary) 2024.10.22 00:32:43.193374 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mow/zeaxjecsosukrdahmatflrqfmeuja. 2024.10.22 00:32:43.236228 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/igj/qxlpmpoibbmwllrhvtkyaqgzexrjs. 2024.10.22 00:32:43.240848 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_77_77_0 to 202410_87_87_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:43.315975 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 7179 2024.10.22 00:32:43.518443 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 95 2024.10.22 00:32:43.663339 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 166812 2024.10.22 00:32:43.785400 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:43.791993 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:43.865260 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_168_168_0 (state Temporary) 2024.10.22 00:32:43.914316 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_168_168_0 to 202410_181_181_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:43.966690 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_176_35 to 202410_181_181_0 2024.10.22 00:32:43.978216 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7179 2024.10.22 00:32:43.978486 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:43.979950 [ 1253 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:43.983539 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/igj/qxlpmpoibbmwllrhvtkyaqgzexrjs. 2024.10.22 00:32:43.986254 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_6_6_0 to 201403_6_6_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:43.992652 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_176_35 to 202410_181_181_0 into Compact with storage Full 2024.10.22 00:32:44.008009 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mow/zeaxjecsosukrdahmatflrqfmeuja. 2024.10.22 00:32:44.044678 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:32:44.160702 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_176_35, total 7639 rows starting from the beginning of the part 2024.10.22 00:32:44.177171 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_177_177_0, total 58 rows starting from the beginning of the part 2024.10.22 00:32:44.188751 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_178_178_0, total 26 rows starting from the beginning of the part 2024.10.22 00:32:44.311641 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_179_179_0, total 107 rows starting from the beginning of the part 2024.10.22 00:32:44.439683 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_180_180_0, total 91 rows starting from the beginning of the part 2024.10.22 00:32:44.467729 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.27 GiB, peak 1.81 GiB, free memory in arenas 4.51 MiB, will set to 1.55 GiB (RSS), difference: 290.92 MiB 2024.10.22 00:32:44.476030 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_181_181_0, total 89 rows starting from the beginning of the part 2024.10.22 00:32:44.566671 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:44.572092 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:32:44.685472 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_93_93_0 (state Temporary) 2024.10.22 00:32:44.889038 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_93_93_0 to 202410_100_100_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:45.030717 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 166812 2024.10.22 00:32:45.005662 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 31 entries to flush up to offset 6338 2024.10.22 00:32:45.325062 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:45.330692 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:32:45.387717 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_96_96_0 (state Temporary) 2024.10.22 00:32:45.572106 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_96_96_0 to 202410_102_102_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:45.729742 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6338 2024.10.22 00:32:45.974964 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_181_36/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:32:46.186392 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} MergeTask::MergeProjectionsStage: Merge sorted 8010 rows, containing 13 columns (13 merged, 0 gathered) in 2.202311619 sec., 3637.087472497233 rows/sec., 558.47 KiB/sec. 2024.10.22 00:32:46.485612 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} MergedBlockOutputStream: filled checksums 202410_1_181_36 (state Temporary) 2024.10.22 00:32:46.531349 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_181_36 to 202410_1_181_36 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:46.541290 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_181_36} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_176_35, 202410_181_181_0] -> 202410_1_181_36 2024.10.22 00:32:46.589309 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.82 MiB. 2024.10.22 00:32:46.723768 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 1383 2024.10.22 00:32:46.762488 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 1.81 GiB, free memory in arenas 4.16 MiB, will set to 1.12 GiB (RSS), difference: -264.56 MiB 2024.10.22 00:32:46.841013 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:46.945110 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:46.995547 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_97_97_0 (state Temporary) 2024.10.22 00:32:47.050568 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_97_97_0 to 202410_110_110_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:47.097247 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1383 2024.10.22 00:32:47.998932 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 7226 2024.10.22 00:32:48.256626 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:48.266465 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:48.447702 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_169_169_0 (state Temporary) 2024.10.22 00:32:48.578846 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 1.81 GiB, free memory in arenas 3.49 MiB, will set to 1.16 GiB (RSS), difference: -93.79 MiB 2024.10.22 00:32:48.728185 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_169_169_0 to 202410_182_182_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:48.940836 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7226 2024.10.22 00:32:49.189602 [ 1263 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:32:49.443419 [ 1263 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:32:49.451403 [ 1263 ] {} DNSCacheUpdater: Execution took 261 ms. 2024.10.22 00:32:49.477579 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:32:49.479577 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:32:49.873124 [ 1164 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 00:32:49.985718 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 6 2024.10.22 00:32:50.395920 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:50.408634 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:50.674882 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_6_6_0 (state Temporary) 2024.10.22 00:32:50.769652 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_6_6_0 to 202410_6_6_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:50.870335 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.81 GiB, free memory in arenas 2.97 MiB, will set to 1.22 GiB (RSS), difference: -124.19 MiB 2024.10.22 00:32:50.870768 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 6 2024.10.22 00:32:51.022099 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 96 2024.10.22 00:32:51.559388 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:51.564694 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:51.637258 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_78_78_0 (state Temporary) 2024.10.22 00:32:51.818714 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_78_78_0 to 202410_88_88_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:51.914195 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 96 2024.10.22 00:32:51.973460 [ 1172 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_83_17 to 202410_88_88_0 2024.10.22 00:32:51.982140 [ 1172 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:51.985286 [ 1172 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:32:52.024139 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_83_17 to 202410_88_88_0 into Compact with storage Full 2024.10.22 00:32:52.048924 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1401 entries to flush up to offset 168213 2024.10.22 00:32:52.206487 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:32:52.225594 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_83_17, total 127 rows starting from the beginning of the part 2024.10.22 00:32:52.374339 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_84_84_0, total 1 rows starting from the beginning of the part 2024.10.22 00:32:52.426653 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_85_85_0, total 1 rows starting from the beginning of the part 2024.10.22 00:32:52.481103 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_86_86_0, total 1 rows starting from the beginning of the part 2024.10.22 00:32:52.528344 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_87_87_0, total 1 rows starting from the beginning of the part 2024.10.22 00:32:52.560869 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:52.566736 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:32:52.590591 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_88_88_0, total 1 rows starting from the beginning of the part 2024.10.22 00:32:52.596950 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:52.600301 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:32:52.787553 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_94_94_0 (state Temporary) 2024.10.22 00:32:52.850299 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_94_94_0 to 202410_101_101_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:52.880876 [ 1193 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_96_19 to 202410_101_101_0 2024.10.22 00:32:52.888238 [ 1193 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:52.890872 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:32:52.911038 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_96_19 to 202410_101_101_0 into Compact with storage Full 2024.10.22 00:32:52.952047 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 7274 2024.10.22 00:32:52.979078 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 168213 2024.10.22 00:32:53.068312 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:32:53.073714 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} MergeTreeSequentialSource: Reading 23 marks from part 202410_1_96_19, total 173874 rows starting from the beginning of the part 2024.10.22 00:32:53.090872 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_97_97_0, total 1741 rows starting from the beginning of the part 2024.10.22 00:32:53.105585 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_98_98_0, total 1715 rows starting from the beginning of the part 2024.10.22 00:32:53.137528 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_99_99_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:32:53.162552 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_100_100_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:32:53.177518 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_101_101_0, total 1401 rows starting from the beginning of the part 2024.10.22 00:32:53.343479 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 38 entries to flush up to offset 6376 2024.10.22 00:32:53.480415 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:53.483946 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:32:53.501409 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:53.506988 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:32:53.637574 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_170_170_0 (state Temporary) 2024.10.22 00:32:53.676930 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_97_97_0 (state Temporary) 2024.10.22 00:32:53.734798 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 1.81 GiB, free memory in arenas 3.38 MiB, will set to 1.32 GiB (RSS), difference: -60.80 MiB 2024.10.22 00:32:53.775928 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_170_170_0 to 202410_183_183_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:53.852033 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_97_97_0 to 202410_103_103_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:54.023690 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_46_9, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0, 202410_51_51_0] 2024.10.22 00:32:54.030767 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_46_9, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0, 202410_51_51_0] 2024.10.22 00:32:54.052878 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6376 2024.10.22 00:32:54.066689 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7274 2024.10.22 00:32:54.182544 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_46_9, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0, 202410_51_51_0] 2024.10.22 00:32:54.343694 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} MergeTask::MergeProjectionsStage: Merge sorted 132 rows, containing 32 columns (32 merged, 0 gathered) in 2.329680578 sec., 56.66012810791437 rows/sec., 7.65 KiB/sec. 2024.10.22 00:32:54.391736 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} MergedBlockOutputStream: filled checksums 202410_1_88_18 (state Temporary) 2024.10.22 00:32:54.601460 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 1390 2024.10.22 00:32:54.692384 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_88_18 to 202410_1_88_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:54.697693 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_88_18} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_83_17, 202410_88_88_0] -> 202410_1_88_18 2024.10.22 00:32:54.779642 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:32:54.975603 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:54.981670 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:32:55.093387 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_insert_202410_98_98_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:32:55.113526 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_98_98_0 (state Temporary) 2024.10.22 00:32:55.191210 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_98_98_0 to 202410_111_111_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:55.264735 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1390 2024.10.22 00:32:55.344730 [ 1200 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_106_21 to 202410_111_111_0 2024.10.22 00:32:55.370917 [ 1200 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:55.375685 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:32:55.409653 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_106_21 to 202410_111_111_0 into Compact with storage Full 2024.10.22 00:32:55.433410 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.81 GiB, free memory in arenas 4.31 MiB, will set to 1.34 GiB (RSS), difference: 14.92 MiB 2024.10.22 00:32:55.557566 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:32:55.567459 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_106_21, total 1429 rows starting from the beginning of the part 2024.10.22 00:32:55.581611 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_107_107_0, total 27 rows starting from the beginning of the part 2024.10.22 00:32:55.601591 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_35_7, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0] 2024.10.22 00:32:55.609280 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_35_7, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0] 2024.10.22 00:32:55.623938 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_108_108_0, total 7 rows starting from the beginning of the part 2024.10.22 00:32:55.760100 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_35_7, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0, 202410_40_40_0] 2024.10.22 00:32:55.856339 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_109_109_0, total 12 rows starting from the beginning of the part 2024.10.22 00:32:55.873945 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_110_110_0, total 14 rows starting from the beginning of the part 2024.10.22 00:32:55.891892 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_111_111_0, total 7 rows starting from the beginning of the part 2024.10.22 00:32:55.932816 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_101_20/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:32:56.820216 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_20_20_0 (state Temporary) 2024.10.22 00:32:57.085106 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:32:57.177792 [ 1209 ] {} DatabaseCatalog: Execution took 391 ms. 2024.10.22 00:32:57.203973 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} MergeTask::MergeProjectionsStage: Merge sorted 1496 rows, containing 12 columns (12 merged, 0 gathered) in 1.809893846 sec., 826.567814077202 rows/sec., 127.07 KiB/sec. 2024.10.22 00:32:57.281770 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} MergedBlockOutputStream: filled checksums 202410_1_111_22 (state Temporary) 2024.10.22 00:32:57.572428 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_111_22 to 202410_1_111_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:57.587004 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_111_22} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_106_21, 202410_111_111_0] -> 202410_1_111_22 2024.10.22 00:32:57.640169 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:32:57.676846 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.81 GiB, free memory in arenas 7.78 MiB, will set to 1.38 GiB (RSS), difference: -53.80 MiB 2024.10.22 00:32:57.935812 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:32:58.075467 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 7336 2024.10.22 00:32:58.492374 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:32:58.496706 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:32:58.673543 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_171_171_0 (state Temporary) 2024.10.22 00:32:58.779568 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_76_15, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0] 2024.10.22 00:32:58.782450 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_76_15, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0] 2024.10.22 00:32:58.818686 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_171_171_0 to 202410_184_184_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:32:58.921142 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_76_15, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0] 2024.10.22 00:32:59.039441 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7336 2024.10.22 00:32:59.317428 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.81 GiB, free memory in arenas 11.01 MiB, will set to 1.41 GiB (RSS), difference: -11.34 MiB 2024.10.22 00:32:59.479073 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:32:59.480301 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:33:00.037458 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1586 entries to flush up to offset 169799 2024.10.22 00:33:00.027050 [ 1244 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 222 ms. 2024.10.22 00:33:00.774563 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:00.778847 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:00.925307 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_95_95_0 (state Temporary) 2024.10.22 00:33:01.195941 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_95_95_0 to 202410_102_102_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:01.346653 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 169799 2024.10.22 00:33:01.399432 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.81 GiB, free memory in arenas 14.00 MiB, will set to 1.46 GiB (RSS), difference: -67.06 MiB 2024.10.22 00:33:01.554850 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} MergeTask::MergeProjectionsStage: Merge sorted 181803 rows, containing 4 columns (4 merged, 0 gathered) in 8.650705003 sec., 21015.974991281295 rows/sec., 308.75 KiB/sec. 2024.10.22 00:33:01.570459 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 45 entries to flush up to offset 6421 2024.10.22 00:33:01.610705 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} MergedBlockOutputStream: filled checksums 202410_1_101_20 (state Temporary) 2024.10.22 00:33:01.646939 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_101_20 to 202410_1_101_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:01.653696 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_101_20} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_96_19, 202410_101_101_0] -> 202410_1_101_20 2024.10.22 00:33:01.716783 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:33:01.783038 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:01.784717 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:01.829536 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_98_98_0 (state Temporary) 2024.10.22 00:33:01.879261 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_98_98_0 to 202410_104_104_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:02.070798 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6421 2024.10.22 00:33:02.767688 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 1415 2024.10.22 00:33:02.896029 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:02.899155 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:02.971554 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_99_99_0 (state Temporary) 2024.10.22 00:33:03.057600 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 7371 2024.10.22 00:33:03.153797 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_99_99_0 to 202410_112_112_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:03.214699 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1415 2024.10.22 00:33:03.259909 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:03.262750 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:03.290626 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_172_172_0 (state Temporary) 2024.10.22 00:33:03.476721 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_172_172_0 to 202410_185_185_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:03.523966 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 13.63 MiB, will set to 1.52 GiB (RSS), difference: -46.15 MiB 2024.10.22 00:33:03.625263 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7371 2024.10.22 00:33:04.461255 [ 1249 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:33:04.531973 [ 1249 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:33:05.451640 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 12.33 MiB, will set to 1.56 GiB (RSS), difference: -26.98 MiB 2024.10.22 00:33:06.071030 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_40_8, 202410_41_41_0, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0] 2024.10.22 00:33:06.076619 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_40_8, 202410_41_41_0, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0] 2024.10.22 00:33:06.112014 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_40_8, 202410_41_41_0, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0] 2024.10.22 00:33:06.601063 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_20_20_0 to 202410_23_23_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:06.920714 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 97 2024.10.22 00:33:07.407806 [ 1270 ] {} DatabaseCatalog: Execution took 226 ms. 2024.10.22 00:33:07.528919 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:07.534539 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:07.635295 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 20 entries to flush up to offset 7391 2024.10.22 00:33:07.644696 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.81 GiB, free memory in arenas 9.51 MiB, will set to 1.60 GiB (RSS), difference: -69.16 MiB 2024.10.22 00:33:07.777627 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:07.780400 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:07.823172 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_79_79_0 (state Temporary) 2024.10.22 00:33:07.911307 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_79_79_0 to 202410_89_89_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:08.034771 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_173_173_0 (state Temporary) 2024.10.22 00:33:08.075166 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_173_173_0 to 202410_186_186_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:08.091507 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 97 2024.10.22 00:33:08.132278 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_181_36 to 202410_186_186_0 2024.10.22 00:33:08.135670 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7391 2024.10.22 00:33:08.148766 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:08.154474 [ 1269 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:08.273341 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_181_36 to 202410_186_186_0 into Compact with storage Full 2024.10.22 00:33:08.331841 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:33:08.346724 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_181_36, total 8010 rows starting from the beginning of the part 2024.10.22 00:33:08.363025 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_182_182_0, total 47 rows starting from the beginning of the part 2024.10.22 00:33:08.388458 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_183_183_0, total 48 rows starting from the beginning of the part 2024.10.22 00:33:08.422832 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1560 entries to flush up to offset 171359 2024.10.22 00:33:08.510676 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_184_184_0, total 62 rows starting from the beginning of the part 2024.10.22 00:33:08.771319 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_185_185_0, total 35 rows starting from the beginning of the part 2024.10.22 00:33:08.786593 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_186_186_0, total 20 rows starting from the beginning of the part 2024.10.22 00:33:09.344385 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 245.84 MiB using storage policy from min volume index 0 2024.10.22 00:33:09.369552 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:09.373659 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:09.477410 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:33:09.479480 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:33:09.583142 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.81 GiB, free memory in arenas 6.98 MiB, will set to 1.60 GiB (RSS), difference: 15.22 MiB 2024.10.22 00:33:09.585110 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 34 entries to flush up to offset 6455 2024.10.22 00:33:09.777008 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_96_96_0 (state Temporary) 2024.10.22 00:33:09.801174 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:09.809161 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:09.876304 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_99_99_0 (state Temporary) 2024.10.22 00:33:09.971154 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_99_99_0 to 202410_105_105_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:09.989815 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_96_96_0 to 202410_103_103_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:10.036846 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6455 2024.10.22 00:33:10.045224 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 774 2024.10.22 00:33:10.069922 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 171359 2024.10.22 00:33:10.139586 [ 1277 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_100_20 to 202410_105_105_0 2024.10.22 00:33:10.155945 [ 1277 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:10.159572 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:10.182193 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_100_20 to 202410_105_105_0 into Compact with storage Full 2024.10.22 00:33:10.419881 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:33:10.434751 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_100_20, total 6377 rows starting from the beginning of the part 2024.10.22 00:33:10.444396 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_101_101_0, total 34 rows starting from the beginning of the part 2024.10.22 00:33:10.462932 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_102_102_0, total 31 rows starting from the beginning of the part 2024.10.22 00:33:10.474193 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_103_103_0, total 38 rows starting from the beginning of the part 2024.10.22 00:33:10.486584 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_104_104_0, total 45 rows starting from the beginning of the part 2024.10.22 00:33:10.527167 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_105_105_0, total 34 rows starting from the beginning of the part 2024.10.22 00:33:10.609335 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} MergeTask::MergeProjectionsStage: Merge sorted 8222 rows, containing 13 columns (13 merged, 0 gathered) in 2.341518626 sec., 3511.396368452377 rows/sec., 538.61 KiB/sec. 2024.10.22 00:33:10.664891 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} MergedBlockOutputStream: filled checksums 202410_1_186_37 (state Temporary) 2024.10.22 00:33:10.717716 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 1422 2024.10.22 00:33:10.720254 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lpe/qgsqzrpjqpbymlprhjbatsdunljtr (LogSeriesLimiter: on interval from 2024-10-22 00:31:45 to 2024-10-22 00:33:10 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:33:10.748724 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_186_37 to 202410_1_186_37 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:10.753026 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_186_37} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_181_36, 202410_186_186_0] -> 202410_1_186_37 2024.10.22 00:33:10.802742 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.85 MiB. 2024.10.22 00:33:10.964155 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:10.966607 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:11.112011 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_100_100_0 (state Temporary) 2024.10.22 00:33:11.172458 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_100_100_0 to 202410_113_113_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:11.217515 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1422 2024.10.22 00:33:11.630181 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.81 GiB, free memory in arenas 9.46 MiB, will set to 1.60 GiB (RSS), difference: -72.02 MiB 2024.10.22 00:33:11.762439 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} MergeTask::MergeProjectionsStage: Merge sorted 6559 rows, containing 13 columns (13 merged, 0 gathered) in 1.593762236 sec., 4115.419384300181 rows/sec., 1.72 MiB/sec. 2024.10.22 00:33:12.138375 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 7454 2024.10.22 00:33:12.145394 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} MergedBlockOutputStream: filled checksums 202410_1_105_21 (state Temporary) 2024.10.22 00:33:12.542504 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_105_21 to 202410_1_105_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:12.548119 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_105_21} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_100_20, 202410_105_105_0] -> 202410_1_105_21 2024.10.22 00:33:12.577719 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.14 MiB. 2024.10.22 00:33:12.592239 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:12.594880 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:12.783307 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_174_174_0 (state Temporary) 2024.10.22 00:33:12.831078 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_174_174_0 to 202410_187_187_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:12.896166 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7454 2024.10.22 00:33:13.180299 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:33:13.220422 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_41_8, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 00:33:13.222836 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_41_8, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 00:33:13.258018 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_41_8, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 00:33:13.705477 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.81 GiB, free memory in arenas 8.17 MiB, will set to 1.60 GiB (RSS), difference: -112.26 MiB 2024.10.22 00:33:15.139860 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uzj/aseogqargoydernmxrqhfmopslhdm (LogSeriesLimiter: on interval from 2024-10-22 00:33:10 to 2024-10-22 00:33:15 accepted series 1 / 185 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:33:15.346519 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.81 GiB, free memory in arenas 8.19 MiB, will set to 1.60 GiB (RSS), difference: -132.01 MiB 2024.10.22 00:33:15.593145 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 98 2024.10.22 00:33:16.184679 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:16.190380 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:16.268959 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_80_80_0 (state Temporary) 2024.10.22 00:33:16.438425 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_80_80_0 to 202410_90_90_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:16.500671 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 98 2024.10.22 00:33:17.075876 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1597 entries to flush up to offset 172956 2024.10.22 00:33:17.150960 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 23 entries to flush up to offset 7477 2024.10.22 00:33:17.272402 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:17.276989 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:17.347474 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_175_175_0 (state Temporary) 2024.10.22 00:33:17.558039 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 221 entries to flush up to offset 6676 2024.10.22 00:33:17.591159 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_175_175_0 to 202410_188_188_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:17.655241 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.81 GiB, free memory in arenas 7.11 MiB, will set to 1.60 GiB (RSS), difference: -133.32 MiB 2024.10.22 00:33:17.675757 [ 1208 ] {} DatabaseCatalog: Execution took 263 ms. 2024.10.22 00:33:17.784205 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 5 old parts to remove. Parts: [202410_1_5_1, 202410_6_6_0, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0] 2024.10.22 00:33:17.788058 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from filesystem (serially): Parts: [202410_1_5_1, 202410_6_6_0, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0] 2024.10.22 00:33:17.888968 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7477 2024.10.22 00:33:17.691950 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 816 2024.10.22 00:33:17.925729 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from memory: Parts: [202410_1_5_1, 202410_6_6_0, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0] 2024.10.22 00:33:17.946890 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:17.952978 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:17.996770 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_100_100_0 (state Temporary) 2024.10.22 00:33:18.096148 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:18.104026 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:18.184081 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_100_100_0 to 202410_106_106_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:18.270928 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6676 2024.10.22 00:33:18.380480 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_97_97_0 (state Temporary) 2024.10.22 00:33:18.497055 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_97_97_0 to 202410_104_104_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:18.548217 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 172956 2024.10.22 00:33:18.720221 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 1442 2024.10.22 00:33:18.920383 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:19.024830 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:19.134186 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_101_101_0 (state Temporary) 2024.10.22 00:33:19.219196 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_101_101_0 to 202410_114_114_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:19.476420 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1442 2024.10.22 00:33:19.480715 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:33:19.482850 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:33:19.540186 [ 1212 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:33:19.659114 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.81 GiB, free memory in arenas 6.71 MiB, will set to 1.60 GiB (RSS), difference: -101.00 MiB 2024.10.22 00:33:19.675106 [ 1212 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:33:21.160209 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_81_16, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0] 2024.10.22 00:33:21.162285 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_81_16, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0] 2024.10.22 00:33:21.200912 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_81_16, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0] 2024.10.22 00:33:21.678565 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 7.30 MiB, will set to 1.60 GiB (RSS), difference: 4.47 MiB 2024.10.22 00:33:21.900556 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 7516 2024.10.22 00:33:22.158981 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:22.163251 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:22.409779 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_176_176_0 (state Temporary) 2024.10.22 00:33:22.677105 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_176_176_0 to 202410_189_189_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:22.841197 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7516 2024.10.22 00:33:23.534822 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 7.33 MiB, will set to 1.61 GiB (RSS), difference: 3.48 MiB 2024.10.22 00:33:24.107118 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 99 2024.10.22 00:33:24.513220 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:24.525283 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:24.692147 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_81_81_0 (state Temporary) 2024.10.22 00:33:24.873125 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_81_81_0 to 202410_91_91_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:24.974632 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 99 2024.10.22 00:33:25.564619 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 174492 2024.10.22 00:33:25.803343 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 7.42 MiB, will set to 1.61 GiB (RSS), difference: 1.55 MiB 2024.10.22 00:33:25.821443 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 230 entries to flush up to offset 6906 2024.10.22 00:33:26.333052 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:26.339919 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:26.496846 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_101_101_0 (state Temporary) 2024.10.22 00:33:26.537108 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:26.539092 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:26.678104 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_101_101_0 to 202410_107_107_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:26.709612 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6906 2024.10.22 00:33:26.786439 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_98_98_0 (state Temporary) 2024.10.22 00:33:26.853164 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 24 entries to flush up to offset 7540 2024.10.22 00:33:26.916028 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_98_98_0 to 202410_105_105_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:26.928649 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:33:26.978739 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 4 entries to flush up to offset 1446 2024.10.22 00:33:27.144946 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:27.156436 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:27.196465 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:27.209585 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_102_102_0 (state Temporary) 2024.10.22 00:33:27.213356 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:27.375104 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_177_177_0 (state Temporary) 2024.10.22 00:33:27.375110 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 174492 2024.10.22 00:33:27.380792 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_102_102_0 to 202410_115_115_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:27.500637 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 7.95 MiB, will set to 1.61 GiB (RSS), difference: 1.82 MiB 2024.10.22 00:33:27.529685 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1446 2024.10.22 00:33:27.558687 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_177_177_0 to 202410_190_190_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:27.604844 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7540 2024.10.22 00:33:29.428165 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 7.75 MiB, will set to 1.62 GiB (RSS), difference: 5.33 MiB 2024.10.22 00:33:29.480089 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:33:29.493569 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:33:31.680510 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 7.94 MiB, will set to 1.62 GiB (RSS), difference: 1.72 MiB 2024.10.22 00:33:31.715637 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 19 entries to flush up to offset 7559 2024.10.22 00:33:31.971360 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:31.973311 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.34 GiB. 2024.10.22 00:33:32.029814 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_178_178_0 (state Temporary) 2024.10.22 00:33:32.173436 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_178_178_0 to 202410_191_191_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:32.225048 [ 1154 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_186_37 to 202410_191_191_0 2024.10.22 00:33:32.261126 [ 1266 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 00:33:32.266756 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7559 2024.10.22 00:33:32.382386 [ 1154 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:32.388436 [ 1154 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:32.409007 [ 1154 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 00:33:32.426813 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_186_37 to 202410_191_191_0 into Compact with storage Full 2024.10.22 00:33:32.476501 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 100 2024.10.22 00:33:32.479806 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:33:32.488308 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_186_37, total 8222 rows starting from the beginning of the part 2024.10.22 00:33:32.509523 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_187_187_0, total 63 rows starting from the beginning of the part 2024.10.22 00:33:32.536192 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_51_10, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0, 202410_56_56_0] 2024.10.22 00:33:32.547286 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_51_10, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0, 202410_56_56_0] 2024.10.22 00:33:32.666951 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_188_188_0, total 23 rows starting from the beginning of the part 2024.10.22 00:33:32.682879 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_51_10, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0, 202410_56_56_0] 2024.10.22 00:33:32.689686 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_189_189_0, total 39 rows starting from the beginning of the part 2024.10.22 00:33:32.731155 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_190_190_0, total 24 rows starting from the beginning of the part 2024.10.22 00:33:32.763950 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_191_191_0, total 19 rows starting from the beginning of the part 2024.10.22 00:33:32.849437 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:32.863992 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:32.873472 [ 1696 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uzj/aseogqargoydernmxrqhfmopslhdm, size 36372 2024.10.22 00:33:33.063977 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_82_82_0 (state Temporary) 2024.10.22 00:33:33.184705 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_82_82_0 to 202410_92_92_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:33.499413 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 100 2024.10.22 00:33:33.716435 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.81 GiB, free memory in arenas 7.95 MiB, will set to 1.62 GiB (RSS), difference: 110.47 MiB 2024.10.22 00:33:34.238575 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} MergeTask::MergeProjectionsStage: Merge sorted 8390 rows, containing 13 columns (13 merged, 0 gathered) in 1.829963227 sec., 4584.791582809221 rows/sec., 702.01 KiB/sec. 2024.10.22 00:33:34.284201 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 48 entries to flush up to offset 6954 2024.10.22 00:33:34.321541 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} MergedBlockOutputStream: filled checksums 202410_1_191_38 (state Temporary) 2024.10.22 00:33:34.379343 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1878 entries to flush up to offset 176370 2024.10.22 00:33:34.379273 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_191_38 to 202410_1_191_38 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:34.388177 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_191_38} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_186_37, 202410_191_191_0] -> 202410_1_191_38 2024.10.22 00:33:34.449359 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.69 MiB. 2024.10.22 00:33:34.528651 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:34.534719 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.33 GiB. 2024.10.22 00:33:34.595492 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_102_102_0 (state Temporary) 2024.10.22 00:33:34.668787 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_102_102_0 to 202410_108_108_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:34.678656 [ 1171 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:33:34.722708 [ 1171 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:33:34.763124 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 6954 2024.10.22 00:33:35.031876 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 1459 2024.10.22 00:33:35.195549 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:35.197706 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.32 GiB. 2024.10.22 00:33:35.327313 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:35.342580 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.32 GiB. 2024.10.22 00:33:35.376027 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.81 GiB, free memory in arenas 6.79 MiB, will set to 1.61 GiB (RSS), difference: 109.63 MiB 2024.10.22 00:33:35.376375 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_99_99_0 (state Temporary) 2024.10.22 00:33:35.490463 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_103_103_0 (state Temporary) 2024.10.22 00:33:35.582593 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_103_103_0 to 202410_116_116_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:35.617802 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_99_99_0 to 202410_106_106_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:35.663120 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1459 2024.10.22 00:33:35.673475 [ 1172 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_101_20 to 202410_106_106_0 2024.10.22 00:33:35.683114 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 176370 2024.10.22 00:33:35.683103 [ 1172 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:35.686295 [ 1172 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.32 GiB. 2024.10.22 00:33:35.698624 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_101_20 to 202410_106_106_0 into Compact with storage Full 2024.10.22 00:33:35.722366 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:33:35.731640 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} MergeTreeSequentialSource: Reading 24 marks from part 202410_1_101_20, total 181803 rows starting from the beginning of the part 2024.10.22 00:33:35.731640 [ 1178 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_111_22 to 202410_116_116_0 2024.10.22 00:33:35.759786 [ 1178 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:35.767517 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.32 GiB. 2024.10.22 00:33:35.875758 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_102_102_0, total 1586 rows starting from the beginning of the part 2024.10.22 00:33:35.898849 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_111_22 to 202410_116_116_0 into Compact with storage Full 2024.10.22 00:33:35.934137 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_103_103_0, total 1560 rows starting from the beginning of the part 2024.10.22 00:33:35.963982 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:35.965900 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.32 GiB. 2024.10.22 00:33:36.069775 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_104_104_0, total 1597 rows starting from the beginning of the part 2024.10.22 00:33:36.103341 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_105_105_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:33:36.132182 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:33:36.139421 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_106_106_0, total 1878 rows starting from the beginning of the part 2024.10.22 00:33:36.163781 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_111_22, total 1496 rows starting from the beginning of the part 2024.10.22 00:33:36.183446 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_112_112_0, total 25 rows starting from the beginning of the part 2024.10.22 00:33:36.269034 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_113_113_0, total 7 rows starting from the beginning of the part 2024.10.22 00:33:36.307242 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_114_114_0, total 20 rows starting from the beginning of the part 2024.10.22 00:33:36.341393 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_115_115_0, total 4 rows starting from the beginning of the part 2024.10.22 00:33:36.443705 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 7638 2024.10.22 00:33:36.480439 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_116_116_0, total 13 rows starting from the beginning of the part 2024.10.22 00:33:36.840610 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_8_8_0 (state Temporary) 2024.10.22 00:33:36.852170 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:36.864500 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:36.874494 [ 1696 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lpe/qgsqzrpjqpbymlprhjbatsdunljtr, size 245 2024.10.22 00:33:36.900489 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_179_179_0 (state Temporary) 2024.10.22 00:33:36.946591 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_179_179_0 to 202410_192_192_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:36.865798 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/oyb/vlbjnmhomyzjnszfhllwufomgypne (LogSeriesLimiter: on interval from 2024-10-22 00:33:15 to 2024-10-22 00:33:36 accepted series 1 / 174 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:33:37.022950 [ 1679 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/oyb/vlbjnmhomyzjnszfhllwufomgypne, size 4 2024.10.22 00:33:37.031231 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7638 2024.10.22 00:33:37.355567 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} MergeTask::MergeProjectionsStage: Merge sorted 1565 rows, containing 12 columns (12 merged, 0 gathered) in 1.577665804 sec., 991.9718079913457 rows/sec., 152.72 KiB/sec. 2024.10.22 00:33:37.379248 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} MergedBlockOutputStream: filled checksums 202410_1_116_23 (state Temporary) 2024.10.22 00:33:37.476057 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_116_23 to 202410_1_116_23 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:37.478577 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_116_23} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_111_22, 202410_116_116_0] -> 202410_1_116_23 2024.10.22 00:33:37.600481 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:33:37.695068 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.81 GiB, free memory in arenas 6.56 MiB, will set to 1.61 GiB (RSS), difference: 99.21 MiB 2024.10.22 00:33:39.480104 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:33:39.482316 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:33:39.482590 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 13.07 MiB, will set to 1.62 GiB (RSS), difference: 10.50 MiB 2024.10.22 00:33:39.902051 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_106_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:33:40.383332 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_21_21_0 (state Temporary) 2024.10.22 00:33:41.003505 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 101 2024.10.22 00:33:41.060139 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 24 entries to flush up to offset 7662 2024.10.22 00:33:41.187635 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:41.189790 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:41.218981 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:41.222220 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:41.273452 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_180_180_0 (state Temporary) 2024.10.22 00:33:41.414326 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_180_180_0 to 202410_193_193_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:41.491785 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7662 2024.10.22 00:33:41.503022 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_83_83_0 (state Temporary) 2024.10.22 00:33:41.684901 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_83_83_0 to 202410_93_93_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:41.758622 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.81 GiB, free memory in arenas 16.93 MiB, will set to 1.62 GiB (RSS), difference: -4.97 MiB 2024.10.22 00:33:41.898760 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 101 2024.10.22 00:33:41.989408 [ 1210 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_88_18 to 202410_93_93_0 2024.10.22 00:33:42.117104 [ 1210 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:42.124280 [ 1210 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:42.132330 [ 1210 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 434 ms. 2024.10.22 00:33:42.144056 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_88_18 to 202410_93_93_0 into Compact with storage Full 2024.10.22 00:33:42.276322 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 76 entries to flush up to offset 7030 2024.10.22 00:33:42.302884 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:33:42.412992 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_88_18, total 132 rows starting from the beginning of the part 2024.10.22 00:33:42.497348 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:42.500707 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:42.554505 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_103_103_0 (state Temporary) 2024.10.22 00:33:42.677435 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_103_103_0 to 202410_109_109_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:42.687774 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_89_89_0, total 1 rows starting from the beginning of the part 2024.10.22 00:33:42.694427 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1360 entries to flush up to offset 177730 2024.10.22 00:33:42.791436 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7030 2024.10.22 00:33:42.796755 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_90_90_0, total 1 rows starting from the beginning of the part 2024.10.22 00:33:42.865630 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_91_91_0, total 1 rows starting from the beginning of the part 2024.10.22 00:33:43.159915 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_92_92_0, total 1 rows starting from the beginning of the part 2024.10.22 00:33:43.167698 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 1472 2024.10.22 00:33:43.317329 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:43.320951 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:43.324024 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_93_93_0, total 1 rows starting from the beginning of the part 2024.10.22 00:33:43.475154 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_104_104_0 (state Temporary) 2024.10.22 00:33:43.588759 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:43.592077 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:43.629459 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} MergeTask::MergeProjectionsStage: Merge sorted 189960 rows, containing 4 columns (4 merged, 0 gathered) in 7.937595179 sec., 23931.68153782462 rows/sec., 351.57 KiB/sec. 2024.10.22 00:33:43.695296 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} MergedBlockOutputStream: filled checksums 202410_1_106_21 (state Temporary) 2024.10.22 00:33:43.720408 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_106_21 to 202410_1_106_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:43.723855 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_106_21} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_101_20, 202410_106_106_0] -> 202410_1_106_21 2024.10.22 00:33:43.724707 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_104_104_0 to 202410_117_117_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:43.779227 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1472 2024.10.22 00:33:43.873765 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:33:43.931270 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_100_100_0 (state Temporary) 2024.10.22 00:33:44.040606 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_100_100_0 to 202410_107_107_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:44.075796 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_86_17, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0] 2024.10.22 00:33:44.076423 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 177730 2024.10.22 00:33:44.077451 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_86_17, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0] 2024.10.22 00:33:44.115050 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_86_17, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0] 2024.10.22 00:33:44.166939 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_21_21_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:33:44.445330 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 19.65 MiB, will set to 1.62 GiB (RSS), difference: 10.63 MiB 2024.10.22 00:33:45.035790 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} MergeTask::MergeProjectionsStage: Merge sorted 137 rows, containing 32 columns (32 merged, 0 gathered) in 2.900819011 sec., 47.22804128092499 rows/sec., 5.41 KiB/sec. 2024.10.22 00:33:45.088146 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} MergedBlockOutputStream: filled checksums 202410_1_93_19 (state Temporary) 2024.10.22 00:33:45.497071 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 7716 2024.10.22 00:33:45.538827 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_93_19 to 202410_1_93_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:45.548297 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_93_19} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_88_18, 202410_93_93_0] -> 202410_1_93_19 2024.10.22 00:33:45.591112 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_40_8, 202410_41_41_0, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0] 2024.10.22 00:33:45.593457 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_40_8, 202410_41_41_0, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0] 2024.10.22 00:33:45.634337 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_40_8, 202410_41_41_0, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0] 2024.10.22 00:33:45.646973 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:33:45.666457 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:45.670922 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:45.823347 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_181_181_0 (state Temporary) 2024.10.22 00:33:45.885384 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_181_181_0 to 202410_194_194_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:46.197189 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_8_8_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:33:46.248869 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7716 2024.10.22 00:33:46.524766 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 16.80 MiB, will set to 1.62 GiB (RSS), difference: 1.68 MiB 2024.10.22 00:33:47.763231 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_45_9, 202410_46_46_0, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0] 2024.10.22 00:33:47.766150 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_45_9, 202410_46_46_0, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0] 2024.10.22 00:33:47.995585 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_45_9, 202410_46_46_0, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0] 2024.10.22 00:33:48.322597 [ 1237 ] {} DatabaseCatalog: Execution took 240 ms. 2024.10.22 00:33:49.376225 [ 1257 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 00:33:49.480945 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:33:49.484632 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:33:49.732842 [ 1242 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:33:49.789189 [ 1242 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:33:50.313171 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 36 entries to flush up to offset 7066 2024.10.22 00:33:50.362656 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 23 entries to flush up to offset 7739 2024.10.22 00:33:50.483168 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 15.90 MiB, will set to 1.62 GiB (RSS), difference: -1.41 MiB 2024.10.22 00:33:50.596792 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:50.603687 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:50.604428 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:50.612496 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:50.673393 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_182_182_0 (state Temporary) 2024.10.22 00:33:50.687647 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_104_104_0 (state Temporary) 2024.10.22 00:33:50.823190 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_182_182_0 to 202410_195_195_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:50.865731 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_104_104_0 to 202410_110_110_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:50.889387 [ 1267 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_105_21 to 202410_110_110_0 2024.10.22 00:33:50.891950 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7739 2024.10.22 00:33:51.004984 [ 1267 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:51.019986 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7066 2024.10.22 00:33:51.021826 [ 1267 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:51.080702 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1621 entries to flush up to offset 179351 2024.10.22 00:33:51.155399 [ 1245 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 00:33:51.155156 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_105_21 to 202410_110_110_0 into Compact with storage Full 2024.10.22 00:33:51.178958 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:33:51.184940 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_105_21, total 6559 rows starting from the beginning of the part 2024.10.22 00:33:51.281365 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 1491 2024.10.22 00:33:51.308036 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_106_106_0, total 221 rows starting from the beginning of the part 2024.10.22 00:33:51.323944 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_107_107_0, total 230 rows starting from the beginning of the part 2024.10.22 00:33:51.323952 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_21_21_0 to 202410_24_24_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:51.363245 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_108_108_0, total 48 rows starting from the beginning of the part 2024.10.22 00:33:51.386953 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_109_109_0, total 76 rows starting from the beginning of the part 2024.10.22 00:33:51.388100 [ 1268 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_19_4 to 202410_24_24_0 2024.10.22 00:33:51.394564 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:51.398409 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:51.399404 [ 1268 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:51.408407 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:51.453660 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_19_4 to 202410_24_24_0 into Compact with storage Full 2024.10.22 00:33:51.521182 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_110_110_0, total 36 rows starting from the beginning of the part 2024.10.22 00:33:51.561672 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_105_105_0 (state Temporary) 2024.10.22 00:33:51.614362 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_105_105_0 to 202410_118_118_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:51.670542 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1491 2024.10.22 00:33:51.985034 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:51.988391 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:52.206716 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_101_101_0 (state Temporary) 2024.10.22 00:33:52.285132 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_110_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:33:52.334040 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_101_101_0 to 202410_108_108_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:52.385609 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 179351 2024.10.22 00:33:52.471528 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 14.62 MiB, will set to 1.62 GiB (RSS), difference: -3.21 MiB 2024.10.22 00:33:52.590488 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} MergeTask::MergeProjectionsStage: Merge sorted 7170 rows, containing 13 columns (13 merged, 0 gathered) in 1.55303552 sec., 4616.764979077878 rows/sec., 1.93 MiB/sec. 2024.10.22 00:33:52.880134 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} MergedBlockOutputStream: filled checksums 202410_1_110_22 (state Temporary) 2024.10.22 00:33:53.271197 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_110_22 to 202410_1_110_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:53.276982 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_110_22} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_105_21, 202410_110_110_0] -> 202410_1_110_22 2024.10.22 00:33:53.538737 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.51 MiB. 2024.10.22 00:33:53.568624 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 11.76 MiB, will set to 1.62 GiB (RSS), difference: 3.69 MiB 2024.10.22 00:33:54.500563 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_46_9, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0, 202410_51_51_0] 2024.10.22 00:33:54.511023 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_46_9, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0, 202410_51_51_0] 2024.10.22 00:33:54.543212 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 816 2024.10.22 00:33:54.578828 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_46_9, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0, 202410_51_51_0] 2024.10.22 00:33:54.894441 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 7793 2024.10.22 00:33:55.266897 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:55.270747 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:55.323272 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_183_183_0 (state Temporary) 2024.10.22 00:33:55.375140 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_183_183_0 to 202410_196_196_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:55.406340 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_191_38 to 202410_196_196_0 2024.10.22 00:33:55.426483 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:55.435258 [ 1176 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:55.479562 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_191_38 to 202410_196_196_0 into Compact with storage Full 2024.10.22 00:33:55.514214 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:33:55.612289 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:33:55.620612 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7793 2024.10.22 00:33:55.624634 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_191_38, total 8390 rows starting from the beginning of the part 2024.10.22 00:33:55.647035 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_192_192_0, total 79 rows starting from the beginning of the part 2024.10.22 00:33:55.670455 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_193_193_0, total 24 rows starting from the beginning of the part 2024.10.22 00:33:55.684778 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_194_194_0, total 54 rows starting from the beginning of the part 2024.10.22 00:33:55.719595 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_195_195_0, total 23 rows starting from the beginning of the part 2024.10.22 00:33:55.740979 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_196_196_0, total 54 rows starting from the beginning of the part 2024.10.22 00:33:56.251115 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_19_4, total 673 rows starting from the beginning of the part 2024.10.22 00:33:56.910422 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 102 2024.10.22 00:33:57.319756 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} MergeTask::MergeProjectionsStage: Merge sorted 8624 rows, containing 13 columns (13 merged, 0 gathered) in 1.849949323 sec., 4661.749320794773 rows/sec., 713.82 KiB/sec. 2024.10.22 00:33:57.337940 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:57.344798 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:57.430874 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_84_84_0 (state Temporary) 2024.10.22 00:33:57.603879 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} MergedBlockOutputStream: filled checksums 202410_1_196_39 (state Temporary) 2024.10.22 00:33:57.681536 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_84_84_0 to 202410_94_94_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:57.819408 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.81 GiB, free memory in arenas 13.89 MiB, will set to 1.61 GiB (RSS), difference: -3.92 MiB 2024.10.22 00:33:58.001730 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 102 2024.10.22 00:33:58.302904 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_20_20_0, total 39 rows starting from the beginning of the part 2024.10.22 00:33:58.535362 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 62 entries to flush up to offset 7128 2024.10.22 00:33:58.656294 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:58.663305 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:58.680899 [ 1170 ] {} DatabaseCatalog: Execution took 357 ms. 2024.10.22 00:33:58.715749 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_105_105_0 (state Temporary) 2024.10.22 00:33:58.878663 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_196_39 to 202410_1_196_39 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:58.882337 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_196_39} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_191_38, 202410_196_196_0] -> 202410_1_196_39 2024.10.22 00:33:58.941835 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.76 MiB. 2024.10.22 00:33:58.968679 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_105_105_0 to 202410_111_111_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:59.010507 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7128 2024.10.22 00:33:59.174586 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 1511 2024.10.22 00:33:59.391209 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1669 entries to flush up to offset 181020 2024.10.22 00:33:59.481519 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:33:59.588712 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:59.590519 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:33:59.592192 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:33:59.631208 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 7838 2024.10.22 00:33:59.731400 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_106_106_0 (state Temporary) 2024.10.22 00:33:59.795433 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_106_106_0 to 202410_119_119_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:33:59.849671 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1511 2024.10.22 00:33:59.854744 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:33:59.860808 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:00.100448 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_184_184_0 (state Temporary) 2024.10.22 00:34:00.101623 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_21_21_0, total 39 rows starting from the beginning of the part 2024.10.22 00:34:00.264453 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_184_184_0 to 202410_197_197_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:00.328494 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7838 2024.10.22 00:34:00.351736 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:00.358536 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:00.818145 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_102_102_0 (state Temporary) 2024.10.22 00:34:00.717840 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xcm/wrpwdfvkxfyilugemfxwvcnakiggi. 2024.10.22 00:34:00.993815 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_102_102_0 to 202410_109_109_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:01.049241 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 181020 2024.10.22 00:34:01.565273 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 11.45 MiB, will set to 1.61 GiB (RSS), difference: 1.23 MiB 2024.10.22 00:34:02.080957 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 45 entries to flush up to offset 861 2024.10.22 00:34:02.307950 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_22_22_0, total 40 rows starting from the beginning of the part 2024.10.22 00:34:03.261316 [ 1222 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 00:34:03.860865 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.077839137 sec. 2024.10.22 00:34:04.021324 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_23_23_0, total 45 rows starting from the beginning of the part 2024.10.22 00:34:04.343271 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 22 entries to flush up to offset 7860 2024.10.22 00:34:04.462719 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:04.469725 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:04.524527 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_185_185_0 (state Temporary) 2024.10.22 00:34:04.687114 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_185_185_0 to 202410_198_198_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:04.796542 [ 1191 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:34:04.960312 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7860 2024.10.22 00:34:04.960589 [ 1191 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:34:05.423191 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 9.46 MiB, will set to 1.61 GiB (RSS), difference: -1.26 MiB 2024.10.22 00:34:05.504387 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 103 2024.10.22 00:34:05.819610 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_24_24_0, total 42 rows starting from the beginning of the part 2024.10.22 00:34:05.979816 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:05.982379 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:06.276130 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_85_85_0 (state Temporary) 2024.10.22 00:34:06.458200 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_85_85_0 to 202410_95_95_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:06.521176 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 37 entries to flush up to offset 7165 2024.10.22 00:34:06.522364 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 103 2024.10.22 00:34:07.125401 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:07.133831 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:07.290041 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_106_106_0 (state Temporary) 2024.10.22 00:34:07.352696 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 1516 2024.10.22 00:34:07.436344 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_106_106_0 to 202410_112_112_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:07.599321 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7165 2024.10.22 00:34:07.680329 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:07.714622 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_91_18, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0] 2024.10.22 00:34:07.718995 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:07.719690 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_91_18, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0] 2024.10.22 00:34:07.761421 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_91_18, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0] 2024.10.22 00:34:07.771194 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_107_107_0 (state Temporary) 2024.10.22 00:34:07.924420 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_107_107_0 to 202410_120_120_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:07.983166 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1516 2024.10.22 00:34:08.062206 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1688 entries to flush up to offset 182708 2024.10.22 00:34:08.872088 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:08.883263 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:08.963220 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 7894 2024.10.22 00:34:09.053012 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_103_103_0 (state Temporary) 2024.10.22 00:34:09.234642 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:09.240023 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:09.244559 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_103_103_0 to 202410_110_110_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:09.290359 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 182708 2024.10.22 00:34:09.322624 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_186_186_0 (state Temporary) 2024.10.22 00:34:09.480232 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:34:09.482836 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:34:09.484182 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_186_186_0 to 202410_199_199_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:09.543958 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7894 2024.10.22 00:34:11.589615 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.81 GiB, free memory in arenas 8.55 MiB, will set to 1.61 GiB (RSS), difference: 72.48 MiB 2024.10.22 00:34:13.263576 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xcm/wrpwdfvkxfyilugemfxwvcnakiggi. 2024.10.22 00:34:13.376853 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.81 GiB, free memory in arenas 8.93 MiB, will set to 1.61 GiB (RSS), difference: 62.18 MiB 2024.10.22 00:34:13.552677 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 14 entries to flush up to offset 7908 2024.10.22 00:34:13.615032 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:13.691093 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_56_11, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0, 202410_61_61_0] 2024.10.22 00:34:13.693066 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_56_11, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0, 202410_61_61_0] 2024.10.22 00:34:13.703777 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:13.724909 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_56_11, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0, 202410_61_61_0] 2024.10.22 00:34:13.765324 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_187_187_0 (state Temporary) 2024.10.22 00:34:13.885248 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_187_187_0 to 202410_200_200_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:14.025805 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 104 2024.10.22 00:34:14.061743 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 7908 2024.10.22 00:34:14.292252 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:34:14.428998 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.81 GiB, free memory in arenas 9.23 MiB, will set to 1.60 GiB (RSS), difference: 47.89 MiB 2024.10.22 00:34:14.781262 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:14.784160 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:14.861440 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_86_86_0 (state Temporary) 2024.10.22 00:34:14.955966 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 0112ceefeef4d9eea8435dcc8cf8fce8:0) 2024.10.22 00:34:14.961483 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8dab3b70cfcba66ed47075430d6a61ed, offset: 0, size: 1526761 2024.10.22 00:34:14.973668 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8dab3b70cfcba66ed47075430d6a61ed for removal 2024.10.22 00:34:14.978298 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_86_86_0 to 202410_96_96_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:14.978573 [ 1476 ] {} CacheMetadata: Key 8dab3b70cfcba66ed47075430d6a61ed is removed from metadata 2024.10.22 00:34:14.978923 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 380b4099c1f6c7e0bb65a1d89fc0a9c7, offset: 0, size: 229 2024.10.22 00:34:14.982142 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 380b4099c1f6c7e0bb65a1d89fc0a9c7 for removal 2024.10.22 00:34:14.984121 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e5665deb2ab03443f9321c3aa653666c, offset: 0, size: 201 2024.10.22 00:34:14.988060 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e5665deb2ab03443f9321c3aa653666c for removal 2024.10.22 00:34:14.990463 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 20a24803fee6d3447680af890f988492, offset: 0, size: 574754 2024.10.22 00:34:14.991107 [ 1476 ] {} CacheMetadata: Key e5665deb2ab03443f9321c3aa653666c is removed from metadata 2024.10.22 00:34:14.992532 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 20a24803fee6d3447680af890f988492 for removal 2024.10.22 00:34:14.996898 [ 1476 ] {} CacheMetadata: Key 380b4099c1f6c7e0bb65a1d89fc0a9c7 is removed from metadata 2024.10.22 00:34:15.041405 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 104 2024.10.22 00:34:15.098428 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5d2e109abe082a9de9df3ab6be1f1e9d, offset: 0, size: 214418 2024.10.22 00:34:15.104549 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5d2e109abe082a9de9df3ab6be1f1e9d for removal 2024.10.22 00:34:15.109146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 825c927ffb97692ab813483d300d34ff, offset: 0, size: 181 2024.10.22 00:34:15.112160 [ 1476 ] {} CacheMetadata: Key 5d2e109abe082a9de9df3ab6be1f1e9d is removed from metadata 2024.10.22 00:34:15.116272 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 45 entries to flush up to offset 7210 2024.10.22 00:34:15.116807 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 825c927ffb97692ab813483d300d34ff for removal 2024.10.22 00:34:15.227635 [ 1476 ] {} CacheMetadata: Key 825c927ffb97692ab813483d300d34ff is removed from metadata 2024.10.22 00:34:15.245142 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:15.243748 [ 1476 ] {} CacheMetadata: Key 20a24803fee6d3447680af890f988492 is removed from metadata 2024.10.22 00:34:15.251399 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:15.415282 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_107_107_0 (state Temporary) 2024.10.22 00:34:15.484360 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 1522 2024.10.22 00:34:15.520698 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 0112ceefeef4d9eea8435dcc8cf8fce8:0) 2024.10.22 00:34:15.525056 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 06cc236f1449dfdc976a75a06bd7cc8a, offset: 0, size: 1545574 2024.10.22 00:34:15.535280 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 06cc236f1449dfdc976a75a06bd7cc8a for removal 2024.10.22 00:34:15.539976 [ 1476 ] {} CacheMetadata: Key 06cc236f1449dfdc976a75a06bd7cc8a is removed from metadata 2024.10.22 00:34:15.540623 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7985412f5e24fceec315cc0de52700eb, offset: 0, size: 144 2024.10.22 00:34:15.547596 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7985412f5e24fceec315cc0de52700eb for removal 2024.10.22 00:34:15.554539 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bc94e7a77d1f6086a119c4a72f98d4c5, offset: 0, size: 201 2024.10.22 00:34:15.563267 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bc94e7a77d1f6086a119c4a72f98d4c5 for removal 2024.10.22 00:34:15.569077 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 927aa38c19ca6bc555d92be768a069ae, offset: 0, size: 5485 2024.10.22 00:34:15.572682 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 927aa38c19ca6bc555d92be768a069ae for removal 2024.10.22 00:34:15.575002 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b68f44848465aa724420ea5e8511c375, offset: 0, size: 144 2024.10.22 00:34:15.579031 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b68f44848465aa724420ea5e8511c375 for removal 2024.10.22 00:34:15.588163 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:15.590528 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:15.591855 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5875c231345d9358f5cf79f5d25cf002, offset: 0, size: 46043 2024.10.22 00:34:15.595046 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5875c231345d9358f5cf79f5d25cf002 for removal 2024.10.22 00:34:15.637395 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_108_108_0 (state Temporary) 2024.10.22 00:34:15.659490 [ 1476 ] {} CacheMetadata: Key 7985412f5e24fceec315cc0de52700eb is removed from metadata 2024.10.22 00:34:15.669124 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_107_107_0 to 202410_113_113_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:15.670785 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 935050 bytes for 0112ceefeef4d9eea8435dcc8cf8fce8:0) 2024.10.22 00:34:15.673370 [ 1476 ] {} CacheMetadata: Key 5875c231345d9358f5cf79f5d25cf002 is removed from metadata 2024.10.22 00:34:15.674435 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eeab6cadbd660accaf451d3212234e65, offset: 0, size: 129673 2024.10.22 00:34:15.678415 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eeab6cadbd660accaf451d3212234e65 for removal 2024.10.22 00:34:15.683660 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 28f26f37aa47c4c879b187a75b1ecd4d, offset: 0, size: 145 2024.10.22 00:34:15.687329 [ 1476 ] {} CacheMetadata: Key eeab6cadbd660accaf451d3212234e65 is removed from metadata 2024.10.22 00:34:15.688309 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 28f26f37aa47c4c879b187a75b1ecd4d for removal 2024.10.22 00:34:15.697584 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_108_108_0 to 202410_121_121_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:15.701285 [ 1476 ] {} CacheMetadata: Key 28f26f37aa47c4c879b187a75b1ecd4d is removed from metadata 2024.10.22 00:34:15.727353 [ 1476 ] {} CacheMetadata: Key b68f44848465aa724420ea5e8511c375 is removed from metadata 2024.10.22 00:34:15.740824 [ 1476 ] {} CacheMetadata: Key 927aa38c19ca6bc555d92be768a069ae is removed from metadata 2024.10.22 00:34:15.758559 [ 1277 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_116_23 to 202410_121_121_0 2024.10.22 00:34:15.770795 [ 1277 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:15.774776 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:15.777732 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1522 2024.10.22 00:34:15.789641 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_116_23 to 202410_121_121_0 into Compact with storage Full 2024.10.22 00:34:15.834762 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7210 2024.10.22 00:34:15.848035 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:34:15.856663 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_116_23, total 1565 rows starting from the beginning of the part 2024.10.22 00:34:15.862213 [ 1476 ] {} CacheMetadata: Key bc94e7a77d1f6086a119c4a72f98d4c5 is removed from metadata 2024.10.22 00:34:15.883110 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_117_117_0, total 13 rows starting from the beginning of the part 2024.10.22 00:34:15.913035 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_118_118_0, total 19 rows starting from the beginning of the part 2024.10.22 00:34:15.936075 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_119_119_0, total 20 rows starting from the beginning of the part 2024.10.22 00:34:15.956323 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_120_120_0, total 5 rows starting from the beginning of the part 2024.10.22 00:34:16.078847 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_121_121_0, total 6 rows starting from the beginning of the part 2024.10.22 00:34:16.174993 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 72042 bytes for cd3db5ca3c546920afcd9a122e6a8bc2:0) 2024.10.22 00:34:16.176612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 18e9b0363a3d87e8553e70facfc8bbd1, offset: 0, size: 105849 2024.10.22 00:34:16.179410 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 18e9b0363a3d87e8553e70facfc8bbd1 for removal 2024.10.22 00:34:16.182689 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c73db013536afdf8b2d5803578628ef4, offset: 0, size: 145 2024.10.22 00:34:16.182997 [ 1476 ] {} CacheMetadata: Key 18e9b0363a3d87e8553e70facfc8bbd1 is removed from metadata 2024.10.22 00:34:16.189146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c73db013536afdf8b2d5803578628ef4 for removal 2024.10.22 00:34:16.191615 [ 1476 ] {} CacheMetadata: Key c73db013536afdf8b2d5803578628ef4 is removed from metadata 2024.10.22 00:34:16.295876 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1768 entries to flush up to offset 184476 2024.10.22 00:34:16.571813 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 7.70 MiB, will set to 1.60 GiB (RSS), difference: -6.08 MiB 2024.10.22 00:34:16.998413 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} MergeTask::MergeProjectionsStage: Merge sorted 1628 rows, containing 12 columns (12 merged, 0 gathered) in 1.213981078 sec., 1341.042318947907 rows/sec., 206.60 KiB/sec. 2024.10.22 00:34:17.004987 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 16 file segments (while reserving 1048576 bytes for 54efd73cbc3c2e3cafa9408cfe8c9b30:0) 2024.10.22 00:34:17.120825 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7a2e564854688a2ab93f16fa080793a7, offset: 0, size: 92298 2024.10.22 00:34:17.128099 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7a2e564854688a2ab93f16fa080793a7 for removal 2024.10.22 00:34:17.133472 [ 1476 ] {} CacheMetadata: Key 7a2e564854688a2ab93f16fa080793a7 is removed from metadata 2024.10.22 00:34:17.136816 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 468c86745a178c676c0462331555d02f, offset: 0, size: 178 2024.10.22 00:34:17.156170 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 468c86745a178c676c0462331555d02f for removal 2024.10.22 00:34:17.162539 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ce0a23bbec4640ffbe7b6827ff2814dd, offset: 0, size: 91400 2024.10.22 00:34:17.164129 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} MergedBlockOutputStream: filled checksums 202410_1_121_24 (state Temporary) 2024.10.22 00:34:17.165036 [ 1476 ] {} CacheMetadata: Key 468c86745a178c676c0462331555d02f is removed from metadata 2024.10.22 00:34:17.178954 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ce0a23bbec4640ffbe7b6827ff2814dd for removal 2024.10.22 00:34:17.181065 [ 1476 ] {} CacheMetadata: Key ce0a23bbec4640ffbe7b6827ff2814dd is removed from metadata 2024.10.22 00:34:17.181781 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5d981002e032ff8e257014ab3de0f2dd, offset: 0, size: 155 2024.10.22 00:34:17.185168 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5d981002e032ff8e257014ab3de0f2dd for removal 2024.10.22 00:34:17.189080 [ 1476 ] {} CacheMetadata: Key 5d981002e032ff8e257014ab3de0f2dd is removed from metadata 2024.10.22 00:34:17.190811 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 92012eaa1a1dd3d520e1f125c75b792d, offset: 0, size: 7008 2024.10.22 00:34:17.193445 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 92012eaa1a1dd3d520e1f125c75b792d for removal 2024.10.22 00:34:17.198254 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 07061b48357c62856c9c5fc85c571dea, offset: 0, size: 178 2024.10.22 00:34:17.198348 [ 1476 ] {} CacheMetadata: Key 92012eaa1a1dd3d520e1f125c75b792d is removed from metadata 2024.10.22 00:34:17.206653 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 07061b48357c62856c9c5fc85c571dea for removal 2024.10.22 00:34:17.215957 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 141ee6fe757c2572943775e3843365a9, offset: 0, size: 106290 2024.10.22 00:34:17.219698 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:17.221637 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 141ee6fe757c2572943775e3843365a9 for removal 2024.10.22 00:34:17.224685 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0e5089160c21f73c9f7fc8512d9053c3, offset: 0, size: 326257 2024.10.22 00:34:17.225005 [ 1476 ] {} CacheMetadata: Key 07061b48357c62856c9c5fc85c571dea is removed from metadata 2024.10.22 00:34:17.225492 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:17.227386 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0e5089160c21f73c9f7fc8512d9053c3 for removal 2024.10.22 00:34:17.230131 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 01e9e3d08e6b24c651cc2a7fc5fec137, offset: 0, size: 176 2024.10.22 00:34:17.231110 [ 1476 ] {} CacheMetadata: Key 0e5089160c21f73c9f7fc8512d9053c3 is removed from metadata 2024.10.22 00:34:17.234206 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 01e9e3d08e6b24c651cc2a7fc5fec137 for removal 2024.10.22 00:34:17.236391 [ 1476 ] {} CacheMetadata: Key 01e9e3d08e6b24c651cc2a7fc5fec137 is removed from metadata 2024.10.22 00:34:17.245729 [ 1476 ] {} CacheMetadata: Key 141ee6fe757c2572943775e3843365a9 is removed from metadata 2024.10.22 00:34:17.248797 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c6ccd3d2ae02c19692a52c8add2faf5e, offset: 0, size: 118836 2024.10.22 00:34:17.258257 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c6ccd3d2ae02c19692a52c8add2faf5e for removal 2024.10.22 00:34:17.262699 [ 1476 ] {} CacheMetadata: Key c6ccd3d2ae02c19692a52c8add2faf5e is removed from metadata 2024.10.22 00:34:17.265467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3b16ba96dc749721cd1b39aedede783d, offset: 0, size: 183 2024.10.22 00:34:17.274845 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3b16ba96dc749721cd1b39aedede783d for removal 2024.10.22 00:34:17.278474 [ 1476 ] {} CacheMetadata: Key 3b16ba96dc749721cd1b39aedede783d is removed from metadata 2024.10.22 00:34:17.278626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 531a9078bbef54a85dc49a328cd9f273, offset: 0, size: 82741 2024.10.22 00:34:17.279665 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_121_24 to 202410_1_121_24 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:17.282643 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 531a9078bbef54a85dc49a328cd9f273 for removal 2024.10.22 00:34:17.284808 [ 1476 ] {} CacheMetadata: Key 531a9078bbef54a85dc49a328cd9f273 is removed from metadata 2024.10.22 00:34:17.285102 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_121_24} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_116_23, 202410_121_121_0] -> 202410_1_121_24 2024.10.22 00:34:17.285340 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c77d938ed35f8cf2b6922ed633367904, offset: 0, size: 145 2024.10.22 00:34:17.290121 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c77d938ed35f8cf2b6922ed633367904 for removal 2024.10.22 00:34:17.291943 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6512c12c5dfa2146f89d544b17bd8913, offset: 0, size: 145 2024.10.22 00:34:17.292154 [ 1476 ] {} CacheMetadata: Key c77d938ed35f8cf2b6922ed633367904 is removed from metadata 2024.10.22 00:34:17.297372 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6512c12c5dfa2146f89d544b17bd8913 for removal 2024.10.22 00:34:17.303422 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 33e1ae86fa00b59c91eaa08c34c99ddf, offset: 0, size: 189046 2024.10.22 00:34:17.309571 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 33e1ae86fa00b59c91eaa08c34c99ddf for removal 2024.10.22 00:34:17.317523 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5cd34a5d0e0882c661ab8b0cce3de470, offset: 0, size: 158 2024.10.22 00:34:17.321235 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5cd34a5d0e0882c661ab8b0cce3de470 for removal 2024.10.22 00:34:17.403176 [ 1476 ] {} CacheMetadata: Key 6512c12c5dfa2146f89d544b17bd8913 is removed from metadata 2024.10.22 00:34:17.414688 [ 1476 ] {} CacheMetadata: Key 5cd34a5d0e0882c661ab8b0cce3de470 is removed from metadata 2024.10.22 00:34:17.420582 [ 1476 ] {} CacheMetadata: Key 33e1ae86fa00b59c91eaa08c34c99ddf is removed from metadata 2024.10.22 00:34:17.477810 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:34:17.482460 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 594802 bytes for 54efd73cbc3c2e3cafa9408cfe8c9b30:0) 2024.10.22 00:34:17.484144 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eb35190435366c13636ceb3eaf4d713d, offset: 0, size: 437620 2024.10.22 00:34:17.487570 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eb35190435366c13636ceb3eaf4d713d for removal 2024.10.22 00:34:17.490903 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_104_104_0 (state Temporary) 2024.10.22 00:34:17.491383 [ 1476 ] {} CacheMetadata: Key eb35190435366c13636ceb3eaf4d713d is removed from metadata 2024.10.22 00:34:17.492651 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e7e767412f1691ed1b4dff60d69ef862, offset: 0, size: 186 2024.10.22 00:34:17.584104 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_104_104_0 to 202410_111_111_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:17.603111 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e7e767412f1691ed1b4dff60d69ef862 for removal 2024.10.22 00:34:17.608365 [ 1476 ] {} CacheMetadata: Key e7e767412f1691ed1b4dff60d69ef862 is removed from metadata 2024.10.22 00:34:17.607553 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e4db31da0d280cd0928dd7887c67accc, offset: 0, size: 280410 2024.10.22 00:34:17.614979 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e4db31da0d280cd0928dd7887c67accc for removal 2024.10.22 00:34:17.620129 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 184476 2024.10.22 00:34:17.621244 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6609b140769a6928b2eb8626cb592517, offset: 0, size: 183 2024.10.22 00:34:17.624630 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6609b140769a6928b2eb8626cb592517 for removal 2024.10.22 00:34:17.710836 [ 1171 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_106_21 to 202410_111_111_0 2024.10.22 00:34:17.722902 [ 1476 ] {} CacheMetadata: Key e4db31da0d280cd0928dd7887c67accc is removed from metadata 2024.10.22 00:34:17.729429 [ 1171 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:17.740561 [ 1171 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:17.743288 [ 1476 ] {} CacheMetadata: Key 6609b140769a6928b2eb8626cb592517 is removed from metadata 2024.10.22 00:34:17.771048 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_106_21 to 202410_111_111_0 into Compact with storage Full 2024.10.22 00:34:17.784245 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:34:17.790169 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} MergeTreeSequentialSource: Reading 25 marks from part 202410_1_106_21, total 189960 rows starting from the beginning of the part 2024.10.22 00:34:17.804613 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_107_107_0, total 1360 rows starting from the beginning of the part 2024.10.22 00:34:17.817270 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:17.821345 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_108_108_0, total 1621 rows starting from the beginning of the part 2024.10.22 00:34:17.956192 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_109_109_0, total 1669 rows starting from the beginning of the part 2024.10.22 00:34:17.972202 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_110_110_0, total 1688 rows starting from the beginning of the part 2024.10.22 00:34:17.978806 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:17.985517 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_111_111_0, total 1768 rows starting from the beginning of the part 2024.10.22 00:34:18.067549 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 179 entries to flush up to offset 8087 2024.10.22 00:34:18.511377 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 6.85 MiB, will set to 1.60 GiB (RSS), difference: -2.42 MiB 2024.10.22 00:34:18.557612 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:18.565231 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:18.584410 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 393592 bytes for af88dede9591ea25764a6f433b45fbb6:0) 2024.10.22 00:34:18.587182 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dc06dedce81961fb812d9c72d695d286, offset: 0, size: 834544 2024.10.22 00:34:18.589301 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dc06dedce81961fb812d9c72d695d286 for removal 2024.10.22 00:34:18.591205 [ 1476 ] {} CacheMetadata: Key dc06dedce81961fb812d9c72d695d286 is removed from metadata 2024.10.22 00:34:18.591505 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 83878a09f6fd27ec2e4e3149988f7733, offset: 0, size: 193 2024.10.22 00:34:18.595197 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 83878a09f6fd27ec2e4e3149988f7733 for removal 2024.10.22 00:34:18.606634 [ 1476 ] {} CacheMetadata: Key 83878a09f6fd27ec2e4e3149988f7733 is removed from metadata 2024.10.22 00:34:18.736076 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_188_188_0 (state Temporary) 2024.10.22 00:34:18.807330 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_188_188_0 to 202410_201_201_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:18.826269 [ 1193 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_196_39 to 202410_201_201_0 2024.10.22 00:34:18.837790 [ 1193 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:18.842555 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:18.847965 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8087 2024.10.22 00:34:18.886851 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_196_39 to 202410_201_201_0 into Compact with storage Full 2024.10.22 00:34:18.899235 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:34:18.927854 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_196_39, total 8624 rows starting from the beginning of the part 2024.10.22 00:34:18.938627 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_197_197_0, total 45 rows starting from the beginning of the part 2024.10.22 00:34:18.979406 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_198_198_0, total 22 rows starting from the beginning of the part 2024.10.22 00:34:19.090736 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_199_199_0, total 34 rows starting from the beginning of the part 2024.10.22 00:34:19.096081 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_24_5/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:34:19.104438 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_200_200_0, total 14 rows starting from the beginning of the part 2024.10.22 00:34:19.136737 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_201_201_0, total 179 rows starting from the beginning of the part 2024.10.22 00:34:19.480349 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:34:19.481691 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:34:19.511702 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 564530 bytes for 0087a1c9439c9416ef1f6532d04354ed:0) 2024.10.22 00:34:19.520690 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c15e27a284db3b636cbb610b15f287ad, offset: 0, size: 830243 2024.10.22 00:34:19.525221 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c15e27a284db3b636cbb610b15f287ad for removal 2024.10.22 00:34:19.528882 [ 1476 ] {} CacheMetadata: Key c15e27a284db3b636cbb610b15f287ad is removed from metadata 2024.10.22 00:34:19.529455 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 995d16ccd713578ba13fc574da35aec1, offset: 0, size: 193 2024.10.22 00:34:19.534483 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 995d16ccd713578ba13fc574da35aec1 for removal 2024.10.22 00:34:19.665696 [ 1476 ] {} CacheMetadata: Key 995d16ccd713578ba13fc574da35aec1 is removed from metadata 2024.10.22 00:34:19.688644 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_111_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:34:19.970628 [ 1189 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:34:20.008380 [ 1189 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:34:20.326583 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} MergeTask::MergeProjectionsStage: Merge sorted 8918 rows, containing 13 columns (13 merged, 0 gathered) in 1.444120142 sec., 6175.386479721297 rows/sec., 943.57 KiB/sec. 2024.10.22 00:34:20.351749 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} MergedBlockOutputStream: filled checksums 202410_1_201_40 (state Temporary) 2024.10.22 00:34:20.399598 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 10.37 MiB, will set to 1.61 GiB (RSS), difference: -4.32 MiB 2024.10.22 00:34:20.513366 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_201_40 to 202410_1_201_40 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:20.517182 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_201_40} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_196_39, 202410_201_201_0] -> 202410_1_201_40 2024.10.22 00:34:20.779853 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.66 MiB. 2024.10.22 00:34:20.816188 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 176964 bytes for 7690d09ea8e5735388d1f284ca46cf7c:0) 2024.10.22 00:34:20.820702 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5b05beb105638a699c0d54d9f10f3a9f, offset: 0, size: 473939 2024.10.22 00:34:20.826303 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5b05beb105638a699c0d54d9f10f3a9f for removal 2024.10.22 00:34:20.830719 [ 1476 ] {} CacheMetadata: Key 5b05beb105638a699c0d54d9f10f3a9f is removed from metadata 2024.10.22 00:34:20.832129 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6c19690499c2f54b676d709b07ac06ee, offset: 0, size: 187 2024.10.22 00:34:20.838871 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6c19690499c2f54b676d709b07ac06ee for removal 2024.10.22 00:34:20.937613 [ 1476 ] {} CacheMetadata: Key 6c19690499c2f54b676d709b07ac06ee is removed from metadata 2024.10.22 00:34:21.800556 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_22_22_0 (state Temporary) 2024.10.22 00:34:22.093521 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 179985 bytes for defea7f0f93fd17c801f78ddc8afc3dc:0) 2024.10.22 00:34:22.095365 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 06d154aa068d2b395cf218b9b98a2b69, offset: 0, size: 241709 2024.10.22 00:34:22.097560 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 06d154aa068d2b395cf218b9b98a2b69 for removal 2024.10.22 00:34:22.100242 [ 1476 ] {} CacheMetadata: Key 06d154aa068d2b395cf218b9b98a2b69 is removed from metadata 2024.10.22 00:34:22.101716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a9415f0df4bd423c013687010d701cc9, offset: 0, size: 181 2024.10.22 00:34:22.106838 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a9415f0df4bd423c013687010d701cc9 for removal 2024.10.22 00:34:22.112437 [ 1476 ] {} CacheMetadata: Key a9415f0df4bd423c013687010d701cc9 is removed from metadata 2024.10.22 00:34:22.518932 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 14.37 MiB, will set to 1.61 GiB (RSS), difference: 2.64 MiB 2024.10.22 00:34:22.597410 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 105 2024.10.22 00:34:22.788049 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:22.792465 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:22.979247 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 8150 2024.10.22 00:34:23.051248 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_87_87_0 (state Temporary) 2024.10.22 00:34:23.118498 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_87_87_0 to 202410_97_97_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:23.216414 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 105 2024.10.22 00:34:23.279850 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 1541 2024.10.22 00:34:23.287943 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:23.290966 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:23.352676 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 86 entries to flush up to offset 7296 2024.10.22 00:34:23.427894 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:23.430956 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:23.448993 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_189_189_0 (state Temporary) 2024.10.22 00:34:23.476296 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_109_109_0 (state Temporary) 2024.10.22 00:34:23.515205 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 300484 bytes for 5bf056c95400bf4e3d587128b4b1c409:0) 2024.10.22 00:34:23.519872 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 94b38e61c3705b9844815958b4b60bff, offset: 0, size: 655060 2024.10.22 00:34:23.523568 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 94b38e61c3705b9844815958b4b60bff for removal 2024.10.22 00:34:23.525721 [ 1476 ] {} CacheMetadata: Key 94b38e61c3705b9844815958b4b60bff is removed from metadata 2024.10.22 00:34:23.526230 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 508223320f19000b3ccef032b3a24544, offset: 0, size: 190 2024.10.22 00:34:23.528670 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 508223320f19000b3ccef032b3a24544 for removal 2024.10.22 00:34:23.533107 [ 1476 ] {} CacheMetadata: Key 508223320f19000b3ccef032b3a24544 is removed from metadata 2024.10.22 00:34:23.657820 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_109_109_0 to 202410_122_122_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:23.703158 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1541 2024.10.22 00:34:23.777307 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:23.779448 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:23.892099 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_189_189_0 to 202410_202_202_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:23.920276 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_108_108_0 (state Temporary) 2024.10.22 00:34:23.982316 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_108_108_0 to 202410_114_114_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:24.042923 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8150 2024.10.22 00:34:24.088796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 192654 bytes for e78a0e8a5df7ce29088c6cfce344a11b:0) 2024.10.22 00:34:24.091375 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 38a50a29ae43e8bc99bfb21827adc869, offset: 0, size: 578452 2024.10.22 00:34:24.094532 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 38a50a29ae43e8bc99bfb21827adc869 for removal 2024.10.22 00:34:24.100077 [ 1476 ] {} CacheMetadata: Key 38a50a29ae43e8bc99bfb21827adc869 is removed from metadata 2024.10.22 00:34:24.101168 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1e64312a5b9297e83a4b3d9147ba175b, offset: 0, size: 189 2024.10.22 00:34:24.107921 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1e64312a5b9297e83a4b3d9147ba175b for removal 2024.10.22 00:34:24.127151 [ 1476 ] {} CacheMetadata: Key 1e64312a5b9297e83a4b3d9147ba175b is removed from metadata 2024.10.22 00:34:24.308901 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7296 2024.10.22 00:34:24.726947 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 186012 2024.10.22 00:34:25.003314 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 345724 bytes for fc12b8df80c9d41efbd9aba346830edf:0) 2024.10.22 00:34:25.008875 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a89838c8477de8e15ed06b72f8545e70, offset: 0, size: 141046 2024.10.22 00:34:25.015755 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a89838c8477de8e15ed06b72f8545e70 for removal 2024.10.22 00:34:25.024382 [ 1476 ] {} CacheMetadata: Key a89838c8477de8e15ed06b72f8545e70 is removed from metadata 2024.10.22 00:34:25.026955 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a01db6b9f6c9b7101b9b8ca4f860e881, offset: 0, size: 180 2024.10.22 00:34:25.037267 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a01db6b9f6c9b7101b9b8ca4f860e881 for removal 2024.10.22 00:34:25.043166 [ 1476 ] {} CacheMetadata: Key a01db6b9f6c9b7101b9b8ca4f860e881 is removed from metadata 2024.10.22 00:34:25.045302 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aaf5546210eadd8f426b7d5d0020b4b3, offset: 0, size: 141915 2024.10.22 00:34:25.052237 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aaf5546210eadd8f426b7d5d0020b4b3 for removal 2024.10.22 00:34:25.056105 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1bfb972599eed9a6f9243027faa31571, offset: 0, size: 180 2024.10.22 00:34:25.064834 [ 1476 ] {} CacheMetadata: Key aaf5546210eadd8f426b7d5d0020b4b3 is removed from metadata 2024.10.22 00:34:25.079525 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1bfb972599eed9a6f9243027faa31571 for removal 2024.10.22 00:34:25.082360 [ 1476 ] {} CacheMetadata: Key 1bfb972599eed9a6f9243027faa31571 is removed from metadata 2024.10.22 00:34:25.377053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 141843 bytes for b2957510ca5a0e69c8c03daca0812395:0) 2024.10.22 00:34:25.379366 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6ad9b244319034f4ece62827358b0e0b, offset: 0, size: 120371 2024.10.22 00:34:25.382748 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6ad9b244319034f4ece62827358b0e0b for removal 2024.10.22 00:34:25.385392 [ 1476 ] {} CacheMetadata: Key 6ad9b244319034f4ece62827358b0e0b is removed from metadata 2024.10.22 00:34:25.386391 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a17107e23d5125dec92ff9871b36c7ca, offset: 0, size: 178 2024.10.22 00:34:25.389123 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a17107e23d5125dec92ff9871b36c7ca for removal 2024.10.22 00:34:25.405271 [ 1476 ] {} CacheMetadata: Key a17107e23d5125dec92ff9871b36c7ca is removed from metadata 2024.10.22 00:34:25.581110 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:25.585339 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:25.633760 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for d9ea42c80049b7912b860d68d09265d2:0) 2024.10.22 00:34:25.638985 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 96d42cb67ce088712eb123c63c5e4ae4, offset: 0, size: 481210 2024.10.22 00:34:25.646353 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 96d42cb67ce088712eb123c63c5e4ae4 for removal 2024.10.22 00:34:25.652313 [ 1476 ] {} CacheMetadata: Key 96d42cb67ce088712eb123c63c5e4ae4 is removed from metadata 2024.10.22 00:34:25.754113 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f74ff3d8e39b08b1d98f4c2305aeca0e, offset: 0, size: 187 2024.10.22 00:34:25.766600 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f74ff3d8e39b08b1d98f4c2305aeca0e for removal 2024.10.22 00:34:25.768895 [ 1476 ] {} CacheMetadata: Key f74ff3d8e39b08b1d98f4c2305aeca0e is removed from metadata 2024.10.22 00:34:25.770706 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5b7bd554273879a67c716fe1f2c61598, offset: 0, size: 628635 2024.10.22 00:34:25.773609 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5b7bd554273879a67c716fe1f2c61598 for removal 2024.10.22 00:34:25.776459 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: def0fb1e549c582d29ff9807b45e9b95, offset: 0, size: 190 2024.10.22 00:34:25.776981 [ 1476 ] {} CacheMetadata: Key 5b7bd554273879a67c716fe1f2c61598 is removed from metadata 2024.10.22 00:34:25.781153 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key def0fb1e549c582d29ff9807b45e9b95 for removal 2024.10.22 00:34:25.783737 [ 1476 ] {} CacheMetadata: Key def0fb1e549c582d29ff9807b45e9b95 is removed from metadata 2024.10.22 00:34:25.935394 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 613815 bytes for d9ea42c80049b7912b860d68d09265d2:0) 2024.10.22 00:34:25.937811 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 558a37812a1a9b3ff16db71877a6a64c, offset: 0, size: 1465076 2024.10.22 00:34:25.947778 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 558a37812a1a9b3ff16db71877a6a64c for removal 2024.10.22 00:34:25.952412 [ 1476 ] {} CacheMetadata: Key 558a37812a1a9b3ff16db71877a6a64c is removed from metadata 2024.10.22 00:34:25.953338 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c98321295d3cf586f5148cdcfdea0a7c, offset: 0, size: 243 2024.10.22 00:34:25.961232 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c98321295d3cf586f5148cdcfdea0a7c for removal 2024.10.22 00:34:25.965268 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: de7a9bfba2021d0d584e8cc94ef9739c, offset: 0, size: 1752 2024.10.22 00:34:25.973045 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key de7a9bfba2021d0d584e8cc94ef9739c for removal 2024.10.22 00:34:25.978399 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6fe07f467e5e6a124db74a7acb0d0559, offset: 0, size: 144 2024.10.22 00:34:25.984666 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6fe07f467e5e6a124db74a7acb0d0559 for removal 2024.10.22 00:34:26.063982 [ 1476 ] {} CacheMetadata: Key 6fe07f467e5e6a124db74a7acb0d0559 is removed from metadata 2024.10.22 00:34:26.083554 [ 1476 ] {} CacheMetadata: Key de7a9bfba2021d0d584e8cc94ef9739c is removed from metadata 2024.10.22 00:34:26.095790 [ 1476 ] {} CacheMetadata: Key c98321295d3cf586f5148cdcfdea0a7c is removed from metadata 2024.10.22 00:34:26.152339 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_105_105_0 (state Temporary) 2024.10.22 00:34:26.178225 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} MergeTask::MergeProjectionsStage: Merge sorted 198066 rows, containing 4 columns (4 merged, 0 gathered) in 8.416283319 sec., 23533.665929812552 rows/sec., 345.67 KiB/sec. 2024.10.22 00:34:26.373651 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_105_105_0 to 202410_112_112_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:26.387898 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} MergedBlockOutputStream: filled checksums 202410_1_111_22 (state Temporary) 2024.10.22 00:34:26.421816 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 186012 2024.10.22 00:34:26.546262 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 15.63 MiB, will set to 1.61 GiB (RSS), difference: 1.54 MiB 2024.10.22 00:34:26.635847 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_111_22 to 202410_1_111_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:26.665293 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_111_22} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_106_21, 202410_111_111_0] -> 202410_1_111_22 2024.10.22 00:34:26.705284 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:34:28.059298 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 8243 2024.10.22 00:34:28.107533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 501683 bytes for add4ce26eb98bcd83fc7b2110e043ead:0) 2024.10.22 00:34:28.115577 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1add26c32ed6615ebcf1a5282a62a6db, offset: 0, size: 1711687 2024.10.22 00:34:28.121464 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1add26c32ed6615ebcf1a5282a62a6db for removal 2024.10.22 00:34:28.127049 [ 1476 ] {} CacheMetadata: Key 1add26c32ed6615ebcf1a5282a62a6db is removed from metadata 2024.10.22 00:34:28.126818 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1089c4bca77341c0f7b4243d9dc71a02, offset: 0, size: 247 2024.10.22 00:34:28.133709 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1089c4bca77341c0f7b4243d9dc71a02 for removal 2024.10.22 00:34:28.194624 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_50_10, 202410_51_51_0, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0] 2024.10.22 00:34:28.195816 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_50_10, 202410_51_51_0, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0] 2024.10.22 00:34:28.235847 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_50_10, 202410_51_51_0, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0] 2024.10.22 00:34:28.244787 [ 1476 ] {} CacheMetadata: Key 1089c4bca77341c0f7b4243d9dc71a02 is removed from metadata 2024.10.22 00:34:28.312252 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:28.424345 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:28.545503 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 14.49 MiB, will set to 1.61 GiB (RSS), difference: 2.01 MiB 2024.10.22 00:34:28.738520 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_190_190_0 (state Temporary) 2024.10.22 00:34:28.785331 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_190_190_0 to 202410_203_203_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:29.078597 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8243 2024.10.22 00:34:29.170908 [ 1226 ] {} DatabaseCatalog: Execution took 275 ms. 2024.10.22 00:34:29.481172 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:34:29.482593 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:34:29.676757 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 7 file segments (while reserving 1048576 bytes for 218d6b0eb2b5dcda351d367f6974b026:0) 2024.10.22 00:34:29.680145 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7af61883a9e4fc239b67ecd8459e4be0, offset: 0, size: 4278216 2024.10.22 00:34:29.686772 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7af61883a9e4fc239b67ecd8459e4be0 for removal 2024.10.22 00:34:29.689514 [ 1476 ] {} CacheMetadata: Key 7af61883a9e4fc239b67ecd8459e4be0 is removed from metadata 2024.10.22 00:34:29.690431 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5fda5b4861b391ba0ada074e0c72a57e, offset: 0, size: 274 2024.10.22 00:34:29.693500 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5fda5b4861b391ba0ada074e0c72a57e for removal 2024.10.22 00:34:29.697505 [ 1476 ] {} CacheMetadata: Key 5fda5b4861b391ba0ada074e0c72a57e is removed from metadata 2024.10.22 00:34:29.800116 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d0cecd2c2a0da11917e48697efbe323a, offset: 0, size: 4 2024.10.22 00:34:29.816572 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d0cecd2c2a0da11917e48697efbe323a for removal 2024.10.22 00:34:29.923478 [ 1476 ] {} CacheMetadata: Key d0cecd2c2a0da11917e48697efbe323a is removed from metadata 2024.10.22 00:34:29.924418 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4b3950ca9e9de42f2fd7211c5ce83ce5, offset: 0, size: 144 2024.10.22 00:34:29.928941 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4b3950ca9e9de42f2fd7211c5ce83ce5 for removal 2024.10.22 00:34:29.933203 [ 1476 ] {} CacheMetadata: Key 4b3950ca9e9de42f2fd7211c5ce83ce5 is removed from metadata 2024.10.22 00:34:29.933301 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a8c7a5597c1dc23899940191c4ec0d91, offset: 0, size: 47329 2024.10.22 00:34:29.937421 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a8c7a5597c1dc23899940191c4ec0d91 for removal 2024.10.22 00:34:29.941595 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6a2adb923f8d2989c7e7e1e0c509e3d8, offset: 0, size: 682432 2024.10.22 00:34:29.945485 [ 1476 ] {} CacheMetadata: Key a8c7a5597c1dc23899940191c4ec0d91 is removed from metadata 2024.10.22 00:34:29.948256 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6a2adb923f8d2989c7e7e1e0c509e3d8 for removal 2024.10.22 00:34:29.951479 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 33c3b7c6d24085f51e3d4b646b382dfd, offset: 0, size: 191 2024.10.22 00:34:29.954773 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 33c3b7c6d24085f51e3d4b646b382dfd for removal 2024.10.22 00:34:29.957594 [ 1476 ] {} CacheMetadata: Key 6a2adb923f8d2989c7e7e1e0c509e3d8 is removed from metadata 2024.10.22 00:34:29.978465 [ 1476 ] {} CacheMetadata: Key 33c3b7c6d24085f51e3d4b646b382dfd is removed from metadata 2024.10.22 00:34:30.401336 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_96_19, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0] 2024.10.22 00:34:30.404051 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_96_19, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0] 2024.10.22 00:34:30.538011 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_96_19, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0] 2024.10.22 00:34:30.671813 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_22_22_0 to 202410_25_25_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:30.719524 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 106 2024.10.22 00:34:30.739448 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 11.88 MiB, will set to 1.61 GiB (RSS), difference: 1.02 MiB 2024.10.22 00:34:30.888845 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:30.893674 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:30.976350 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_88_88_0 (state Temporary) 2024.10.22 00:34:31.206183 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 1554 2024.10.22 00:34:31.265052 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_88_88_0 to 202410_98_98_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:31.287305 [ 1251 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_93_19 to 202410_98_98_0 2024.10.22 00:34:31.293255 [ 1251 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:31.295027 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:31.310557 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_93_19 to 202410_98_98_0 into Compact with storage Full 2024.10.22 00:34:31.334498 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:31.335297 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 106 2024.10.22 00:34:31.336786 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:31.370107 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_110_110_0 (state Temporary) 2024.10.22 00:34:31.480337 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:34:31.495840 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_93_19, total 137 rows starting from the beginning of the part 2024.10.22 00:34:31.505470 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_110_110_0 to 202410_123_123_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:31.520074 [ 1250 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 221 ms. 2024.10.22 00:34:31.552116 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1554 2024.10.22 00:34:31.563545 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_94_94_0, total 1 rows starting from the beginning of the part 2024.10.22 00:34:31.714247 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_95_95_0, total 1 rows starting from the beginning of the part 2024.10.22 00:34:31.834862 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 37 entries to flush up to offset 7333 2024.10.22 00:34:31.905454 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_96_96_0, total 1 rows starting from the beginning of the part 2024.10.22 00:34:32.038263 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_24_5/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:34:32.067596 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_97_97_0, total 1 rows starting from the beginning of the part 2024.10.22 00:34:32.123264 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_98_98_0, total 1 rows starting from the beginning of the part 2024.10.22 00:34:32.287221 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:32.289562 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:32.339094 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_109_109_0 (state Temporary) 2024.10.22 00:34:32.385133 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_109_109_0 to 202410_115_115_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:32.428396 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7333 2024.10.22 00:34:32.527996 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_110_22 to 202410_115_115_0 2024.10.22 00:34:32.552088 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:32.559597 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:32.584352 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_110_22 to 202410_115_115_0 into Compact with storage Full 2024.10.22 00:34:32.608142 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 6.88 MiB, will set to 1.61 GiB (RSS), difference: -5.68 MiB 2024.10.22 00:34:32.625802 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:34:32.636150 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_110_22, total 7170 rows starting from the beginning of the part 2024.10.22 00:34:32.762109 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_111_111_0, total 62 rows starting from the beginning of the part 2024.10.22 00:34:32.774036 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_112_112_0, total 37 rows starting from the beginning of the part 2024.10.22 00:34:32.784266 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_113_113_0, total 45 rows starting from the beginning of the part 2024.10.22 00:34:32.804913 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_114_114_0, total 86 rows starting from the beginning of the part 2024.10.22 00:34:32.833487 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_115_115_0, total 37 rows starting from the beginning of the part 2024.10.22 00:34:33.084793 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 90 entries to flush up to offset 8333 2024.10.22 00:34:33.477206 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:33.479914 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:33.522815 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} MergeTask::MergeProjectionsStage: Merge sorted 142 rows, containing 32 columns (32 merged, 0 gathered) in 2.195970618 sec., 64.66388886811599 rows/sec., 7.36 KiB/sec. 2024.10.22 00:34:33.546803 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_191_191_0 (state Temporary) 2024.10.22 00:34:33.629614 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 187932 2024.10.22 00:34:33.667373 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} MergedBlockOutputStream: filled checksums 202410_1_98_20 (state Temporary) 2024.10.22 00:34:33.690618 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_191_191_0 to 202410_204_204_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:33.896941 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 861 2024.10.22 00:34:34.027332 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_98_20 to 202410_1_98_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:34.033808 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_98_20} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_93_19, 202410_98_98_0] -> 202410_1_98_20 2024.10.22 00:34:34.079275 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:34:34.080887 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8333 2024.10.22 00:34:34.103876 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} MergeTask::MergeProjectionsStage: Merge sorted 7437 rows, containing 13 columns (13 merged, 0 gathered) in 1.523587592 sec., 4881.242167532696 rows/sec., 2.04 MiB/sec. 2024.10.22 00:34:34.175145 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} MergedBlockOutputStream: filled checksums 202410_1_115_23 (state Temporary) 2024.10.22 00:34:34.220624 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_115_23 to 202410_1_115_23 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:34.224909 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_115_23} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_110_22, 202410_115_115_0] -> 202410_1_115_23 2024.10.22 00:34:34.481642 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.51 MiB. 2024.10.22 00:34:34.535726 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:34.543235 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:34.589587 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 11.42 MiB, will set to 1.61 GiB (RSS), difference: 9.14 MiB 2024.10.22 00:34:34.776312 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_106_106_0 (state Temporary) 2024.10.22 00:34:35.019919 [ 1211 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:34:35.036930 [ 1211 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:34:35.091958 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_106_106_0 to 202410_113_113_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:35.239406 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 187932 2024.10.22 00:34:35.240714 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 616206 bytes for ba0f32b2f99edf3225947a86d8005812:0) 2024.10.22 00:34:35.248725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c28a44b55eb77fa1ce5ed8e485716aaa, offset: 0, size: 4789 2024.10.22 00:34:35.252877 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c28a44b55eb77fa1ce5ed8e485716aaa for removal 2024.10.22 00:34:35.267946 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 416055f0edc29bc0e612ead817c8eb18, offset: 0, size: 1998216 2024.10.22 00:34:35.274876 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 416055f0edc29bc0e612ead817c8eb18 for removal 2024.10.22 00:34:35.277448 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5b5b38bced8abc6744bff673fdd28774, offset: 0, size: 258 2024.10.22 00:34:35.279640 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5b5b38bced8abc6744bff673fdd28774 for removal 2024.10.22 00:34:35.279420 [ 1476 ] {} CacheMetadata: Key c28a44b55eb77fa1ce5ed8e485716aaa is removed from metadata 2024.10.22 00:34:35.283670 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1f1d0bfa5bd880c01b2bbfc43ae1100e, offset: 0, size: 103207 2024.10.22 00:34:35.286735 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1f1d0bfa5bd880c01b2bbfc43ae1100e for removal 2024.10.22 00:34:35.289606 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 958383d022643da88033a5321f2b221c, offset: 0, size: 186 2024.10.22 00:34:35.289686 [ 1476 ] {} CacheMetadata: Key 5b5b38bced8abc6744bff673fdd28774 is removed from metadata 2024.10.22 00:34:35.291318 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 958383d022643da88033a5321f2b221c for removal 2024.10.22 00:34:35.292869 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bea7247767ff7e8391d692d3b5f554c6, offset: 0, size: 164 2024.10.22 00:34:35.293446 [ 1476 ] {} CacheMetadata: Key 958383d022643da88033a5321f2b221c is removed from metadata 2024.10.22 00:34:35.295739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bea7247767ff7e8391d692d3b5f554c6 for removal 2024.10.22 00:34:35.305630 [ 1476 ] {} CacheMetadata: Key bea7247767ff7e8391d692d3b5f554c6 is removed from metadata 2024.10.22 00:34:35.322553 [ 1476 ] {} CacheMetadata: Key 1f1d0bfa5bd880c01b2bbfc43ae1100e is removed from metadata 2024.10.22 00:34:35.333801 [ 1476 ] {} CacheMetadata: Key 416055f0edc29bc0e612ead817c8eb18 is removed from metadata 2024.10.22 00:34:35.658707 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_45_9, 202410_46_46_0, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0] 2024.10.22 00:34:35.664676 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_45_9, 202410_46_46_0, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0] 2024.10.22 00:34:35.694842 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_45_9, 202410_46_46_0, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0, 202410_50_50_0] 2024.10.22 00:34:35.953131 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_51_10, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0, 202410_56_56_0] 2024.10.22 00:34:35.957776 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_51_10, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0, 202410_56_56_0] 2024.10.22 00:34:35.997018 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_51_10, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0, 202410_56_56_0] 2024.10.22 00:34:36.476273 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 9.90 MiB, will set to 1.60 GiB (RSS), difference: -3.25 MiB 2024.10.22 00:34:37.425626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 5 file segments (while reserving 258721 bytes for 1b1eb7803a99d75a57627e5d746b3c40:0) 2024.10.22 00:34:37.427652 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 137e87816dd898f4657310cf66b834c7, offset: 0, size: 26157 2024.10.22 00:34:37.430486 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 137e87816dd898f4657310cf66b834c7 for removal 2024.10.22 00:34:37.434203 [ 1476 ] {} CacheMetadata: Key 137e87816dd898f4657310cf66b834c7 is removed from metadata 2024.10.22 00:34:37.434981 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a83c5fbf17a6e2d164a6e5823d1a8dc7, offset: 0, size: 221 2024.10.22 00:34:37.437505 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a83c5fbf17a6e2d164a6e5823d1a8dc7 for removal 2024.10.22 00:34:37.441521 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8e0171209c7b49e1d142aa47437959ea, offset: 0, size: 39536 2024.10.22 00:34:37.451565 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8e0171209c7b49e1d142aa47437959ea for removal 2024.10.22 00:34:37.454882 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d3953569f61c17665779c12c3ec38394, offset: 0, size: 238 2024.10.22 00:34:37.465559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d3953569f61c17665779c12c3ec38394 for removal 2024.10.22 00:34:37.467714 [ 1476 ] {} CacheMetadata: Key 8e0171209c7b49e1d142aa47437959ea is removed from metadata 2024.10.22 00:34:37.469630 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cccaded85c56935da3583b17c76b2654, offset: 0, size: 181 2024.10.22 00:34:37.475869 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cccaded85c56935da3583b17c76b2654 for removal 2024.10.22 00:34:37.476547 [ 1476 ] {} CacheMetadata: Key d3953569f61c17665779c12c3ec38394 is removed from metadata 2024.10.22 00:34:37.484599 [ 1476 ] {} CacheMetadata: Key cccaded85c56935da3583b17c76b2654 is removed from metadata 2024.10.22 00:34:37.492036 [ 1476 ] {} CacheMetadata: Key a83c5fbf17a6e2d164a6e5823d1a8dc7 is removed from metadata 2024.10.22 00:34:37.873756 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 215 bytes for b76ab27cf7807544e481db6a441425d4:0) 2024.10.22 00:34:37.876550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5ca2c9e17b123b7a4694f94b655ce476, offset: 0, size: 579934 2024.10.22 00:34:37.878599 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5ca2c9e17b123b7a4694f94b655ce476 for removal 2024.10.22 00:34:37.880489 [ 1476 ] {} CacheMetadata: Key 5ca2c9e17b123b7a4694f94b655ce476 is removed from metadata 2024.10.22 00:34:38.088482 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 8405 2024.10.22 00:34:38.222414 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:38.225504 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:38.378909 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_192_192_0 (state Temporary) 2024.10.22 00:34:38.602993 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 7.87 MiB, will set to 1.60 GiB (RSS), difference: -5.53 MiB 2024.10.22 00:34:38.761062 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 419214 bytes for cfc68467b4fe8815b488ab829e0691fe:0) 2024.10.22 00:34:38.763591 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 95c7cae03694cabfb915b344c353be8f, offset: 0, size: 389715 2024.10.22 00:34:38.766424 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 95c7cae03694cabfb915b344c353be8f for removal 2024.10.22 00:34:38.766933 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_192_192_0 to 202410_205_205_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:38.769702 [ 1476 ] {} CacheMetadata: Key 95c7cae03694cabfb915b344c353be8f is removed from metadata 2024.10.22 00:34:38.771560 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c8011b93f6132290cb6f68af7480b848, offset: 0, size: 233 2024.10.22 00:34:38.775577 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c8011b93f6132290cb6f68af7480b848 for removal 2024.10.22 00:34:38.798290 [ 1476 ] {} CacheMetadata: Key c8011b93f6132290cb6f68af7480b848 is removed from metadata 2024.10.22 00:34:38.815643 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8405 2024.10.22 00:34:39.056359 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 1574 2024.10.22 00:34:39.176065 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 302755 bytes for 36d4a3d319ff49a6fa78e00f82087109:0) 2024.10.22 00:34:39.178412 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ae7041f8207608e7b4cbe8aa2a963a9f, offset: 0, size: 835871 2024.10.22 00:34:39.181667 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ae7041f8207608e7b4cbe8aa2a963a9f for removal 2024.10.22 00:34:39.184835 [ 1476 ] {} CacheMetadata: Key ae7041f8207608e7b4cbe8aa2a963a9f is removed from metadata 2024.10.22 00:34:39.185765 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ca5b55df74e4e1f57e76f764520eb5f8, offset: 0, size: 229 2024.10.22 00:34:39.190693 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ca5b55df74e4e1f57e76f764520eb5f8 for removal 2024.10.22 00:34:39.204799 [ 1476 ] {} CacheMetadata: Key ca5b55df74e4e1f57e76f764520eb5f8 is removed from metadata 2024.10.22 00:34:39.369416 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:39.372745 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:39.376004 [ 1167 ] {} DatabaseCatalog: Execution took 204 ms. 2024.10.22 00:34:39.425509 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_111_111_0 (state Temporary) 2024.10.22 00:34:39.481169 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:34:39.483375 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:34:39.563870 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_111_111_0 to 202410_124_124_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:39.605189 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1574 2024.10.22 00:34:39.883507 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 8 file segments (while reserving 1048576 bytes for c46273aec6d1cd294f12494c25ae15e8:0) 2024.10.22 00:34:39.885915 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5e0b23b2c5359af0c1383a52336f3357, offset: 0, size: 193 2024.10.22 00:34:39.888427 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5e0b23b2c5359af0c1383a52336f3357 for removal 2024.10.22 00:34:39.890799 [ 1476 ] {} CacheMetadata: Key 5e0b23b2c5359af0c1383a52336f3357 is removed from metadata 2024.10.22 00:34:39.890869 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cdfc21e47e713940e3142c69114277bc, offset: 0, size: 168358 2024.10.22 00:34:39.895175 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cdfc21e47e713940e3142c69114277bc for removal 2024.10.22 00:34:39.898831 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 16f9de3c0356800ccabba85f22df18fc, offset: 0, size: 167 2024.10.22 00:34:39.907355 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 16f9de3c0356800ccabba85f22df18fc for removal 2024.10.22 00:34:39.915214 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c4b5370fc17b976ce3b7e7bba40caff4, offset: 0, size: 259788 2024.10.22 00:34:39.915313 [ 1476 ] {} CacheMetadata: Key 16f9de3c0356800ccabba85f22df18fc is removed from metadata 2024.10.22 00:34:39.918706 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c4b5370fc17b976ce3b7e7bba40caff4 for removal 2024.10.22 00:34:39.921274 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aab7682d1a3756f2124a4e58c4e17c40, offset: 0, size: 8518 2024.10.22 00:34:39.923411 [ 1476 ] {} CacheMetadata: Key c4b5370fc17b976ce3b7e7bba40caff4 is removed from metadata 2024.10.22 00:34:39.926481 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aab7682d1a3756f2124a4e58c4e17c40 for removal 2024.10.22 00:34:39.930292 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5e02cf6d76183ef48e9d114e7ab1b97c, offset: 0, size: 145894 2024.10.22 00:34:39.935007 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 61 entries to flush up to offset 7394 2024.10.22 00:34:39.937310 [ 1476 ] {} CacheMetadata: Key aab7682d1a3756f2124a4e58c4e17c40 is removed from metadata 2024.10.22 00:34:39.936940 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5e02cf6d76183ef48e9d114e7ab1b97c for removal 2024.10.22 00:34:39.943798 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2f30688d66932d10fa02ce433e4a00aa, offset: 0, size: 167 2024.10.22 00:34:39.951900 [ 1476 ] {} CacheMetadata: Key 5e02cf6d76183ef48e9d114e7ab1b97c is removed from metadata 2024.10.22 00:34:39.955626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2f30688d66932d10fa02ce433e4a00aa for removal 2024.10.22 00:34:39.959694 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ecf82576fe5cfbfd6e01552308c8c0dd, offset: 0, size: 160 2024.10.22 00:34:39.968796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ecf82576fe5cfbfd6e01552308c8c0dd for removal 2024.10.22 00:34:39.970589 [ 1476 ] {} CacheMetadata: Key 2f30688d66932d10fa02ce433e4a00aa is removed from metadata 2024.10.22 00:34:39.980505 [ 1476 ] {} CacheMetadata: Key ecf82576fe5cfbfd6e01552308c8c0dd is removed from metadata 2024.10.22 00:34:39.988339 [ 1476 ] {} CacheMetadata: Key cdfc21e47e713940e3142c69114277bc is removed from metadata 2024.10.22 00:34:40.167951 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 576651 bytes for c46273aec6d1cd294f12494c25ae15e8:0) 2024.10.22 00:34:40.169697 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 817c549053818cb9e9713b83fbd2790f, offset: 0, size: 127879 2024.10.22 00:34:40.178819 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 817c549053818cb9e9713b83fbd2790f for removal 2024.10.22 00:34:40.180337 [ 1476 ] {} CacheMetadata: Key 817c549053818cb9e9713b83fbd2790f is removed from metadata 2024.10.22 00:34:40.181058 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d183dd271fc74c05c0d8993883704307, offset: 0, size: 165 2024.10.22 00:34:40.187087 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d183dd271fc74c05c0d8993883704307 for removal 2024.10.22 00:34:40.189055 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5a12fafd964ea042cfd12c20df435cb7, offset: 0, size: 254418 2024.10.22 00:34:40.189055 [ 1476 ] {} CacheMetadata: Key d183dd271fc74c05c0d8993883704307 is removed from metadata 2024.10.22 00:34:40.191861 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5a12fafd964ea042cfd12c20df435cb7 for removal 2024.10.22 00:34:40.206328 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4d6d7f6ea34835b43db30b5a6c984a5f, offset: 0, size: 161 2024.10.22 00:34:40.208534 [ 1476 ] {} CacheMetadata: Key 5a12fafd964ea042cfd12c20df435cb7 is removed from metadata 2024.10.22 00:34:40.214434 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4d6d7f6ea34835b43db30b5a6c984a5f for removal 2024.10.22 00:34:40.218630 [ 1476 ] {} CacheMetadata: Key 4d6d7f6ea34835b43db30b5a6c984a5f is removed from metadata 2024.10.22 00:34:40.218433 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 51f929e2d23d95a7e2ef70727080be37, offset: 0, size: 120074 2024.10.22 00:34:40.239729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 51f929e2d23d95a7e2ef70727080be37 for removal 2024.10.22 00:34:40.251380 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6d96a473689856da57a2f0d0b70b0108, offset: 0, size: 194 2024.10.22 00:34:40.252685 [ 1476 ] {} CacheMetadata: Key 51f929e2d23d95a7e2ef70727080be37 is removed from metadata 2024.10.22 00:34:40.256931 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6d96a473689856da57a2f0d0b70b0108 for removal 2024.10.22 00:34:40.270556 [ 1476 ] {} CacheMetadata: Key 6d96a473689856da57a2f0d0b70b0108 is removed from metadata 2024.10.22 00:34:40.297442 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:40.300669 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:40.351350 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_110_110_0 (state Temporary) 2024.10.22 00:34:40.385293 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 7.17 MiB, will set to 1.61 GiB (RSS), difference: 6.67 MiB 2024.10.22 00:34:40.494910 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_110_110_0 to 202410_116_116_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:40.687746 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 16 file segments (while reserving 1048576 bytes for f373f0e79569bd9283457080fd8d15fd:0) 2024.10.22 00:34:40.690731 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e7912845718f5b2c9f02a4b9d6ac9f65, offset: 0, size: 258931 2024.10.22 00:34:40.693584 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e7912845718f5b2c9f02a4b9d6ac9f65 for removal 2024.10.22 00:34:40.698352 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 83ed5f579b2384b090373dd8e69c83d1, offset: 0, size: 166 2024.10.22 00:34:40.697794 [ 1476 ] {} CacheMetadata: Key e7912845718f5b2c9f02a4b9d6ac9f65 is removed from metadata 2024.10.22 00:34:40.774151 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7394 2024.10.22 00:34:40.802697 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 83ed5f579b2384b090373dd8e69c83d1 for removal 2024.10.22 00:34:40.811929 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4a2115d5a2d0d916b4570771317929a1, offset: 0, size: 235 2024.10.22 00:34:40.815282 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4a2115d5a2d0d916b4570771317929a1 for removal 2024.10.22 00:34:40.819723 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: be65826be60466e187d920a4f6e53745, offset: 0, size: 56453 2024.10.22 00:34:40.823881 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key be65826be60466e187d920a4f6e53745 for removal 2024.10.22 00:34:40.828412 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4ee3edbdd9fb49050e5093ea1d3739ce, offset: 0, size: 159 2024.10.22 00:34:40.834874 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4ee3edbdd9fb49050e5093ea1d3739ce for removal 2024.10.22 00:34:40.840949 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 42d4a610530f0bb6fc1b858eb722adf1, offset: 0, size: 63986 2024.10.22 00:34:40.849394 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 42d4a610530f0bb6fc1b858eb722adf1 for removal 2024.10.22 00:34:40.856070 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 22ed3fbd942a214995a5a7f1e091939c, offset: 0, size: 192 2024.10.22 00:34:40.911551 [ 1476 ] {} CacheMetadata: Key 83ed5f579b2384b090373dd8e69c83d1 is removed from metadata 2024.10.22 00:34:40.928311 [ 1476 ] {} CacheMetadata: Key 4ee3edbdd9fb49050e5093ea1d3739ce is removed from metadata 2024.10.22 00:34:40.943219 [ 1476 ] {} CacheMetadata: Key 42d4a610530f0bb6fc1b858eb722adf1 is removed from metadata 2024.10.22 00:34:40.952088 [ 1476 ] {} CacheMetadata: Key be65826be60466e187d920a4f6e53745 is removed from metadata 2024.10.22 00:34:40.963568 [ 1476 ] {} CacheMetadata: Key 4a2115d5a2d0d916b4570771317929a1 is removed from metadata 2024.10.22 00:34:40.865747 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 22ed3fbd942a214995a5a7f1e091939c for removal 2024.10.22 00:34:40.969749 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9f9db9c8eb037db6c815a0b2cfb15acb, offset: 0, size: 161 2024.10.22 00:34:40.972529 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9f9db9c8eb037db6c815a0b2cfb15acb for removal 2024.10.22 00:34:40.973270 [ 1476 ] {} CacheMetadata: Key 22ed3fbd942a214995a5a7f1e091939c is removed from metadata 2024.10.22 00:34:40.975376 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 03326f74d02ab0280084521ec24b5b3f, offset: 0, size: 4313 2024.10.22 00:34:40.979754 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 03326f74d02ab0280084521ec24b5b3f for removal 2024.10.22 00:34:40.979754 [ 1476 ] {} CacheMetadata: Key 9f9db9c8eb037db6c815a0b2cfb15acb is removed from metadata 2024.10.22 00:34:40.984197 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f33e26169585a99324ea01d99f76e721, offset: 0, size: 159 2024.10.22 00:34:40.987950 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f33e26169585a99324ea01d99f76e721 for removal 2024.10.22 00:34:40.988164 [ 1476 ] {} CacheMetadata: Key 03326f74d02ab0280084521ec24b5b3f is removed from metadata 2024.10.22 00:34:40.991061 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9b1e60a7612edff86bf9e46a4dd4f760, offset: 0, size: 161 2024.10.22 00:34:40.996971 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9b1e60a7612edff86bf9e46a4dd4f760 for removal 2024.10.22 00:34:41.000174 [ 1476 ] {} CacheMetadata: Key f33e26169585a99324ea01d99f76e721 is removed from metadata 2024.10.22 00:34:41.000267 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2c5ebc04b6851a1fa3e3d751aaf914c5, offset: 0, size: 160 2024.10.22 00:34:41.008671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2c5ebc04b6851a1fa3e3d751aaf914c5 for removal 2024.10.22 00:34:41.015580 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 386cebef29f4913801be89a04a8dce8c, offset: 0, size: 3795 2024.10.22 00:34:41.017541 [ 1476 ] {} CacheMetadata: Key 2c5ebc04b6851a1fa3e3d751aaf914c5 is removed from metadata 2024.10.22 00:34:41.022789 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 386cebef29f4913801be89a04a8dce8c for removal 2024.10.22 00:34:41.026396 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c3f3912cbc64b37f1954bdd3868e353c, offset: 0, size: 451077 2024.10.22 00:34:41.027852 [ 1476 ] {} CacheMetadata: Key 386cebef29f4913801be89a04a8dce8c is removed from metadata 2024.10.22 00:34:41.030595 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c3f3912cbc64b37f1954bdd3868e353c for removal 2024.10.22 00:34:41.037773 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8471e54a8fb471ab075775e0f211339f, offset: 0, size: 74999 2024.10.22 00:34:41.040321 [ 1476 ] {} CacheMetadata: Key c3f3912cbc64b37f1954bdd3868e353c is removed from metadata 2024.10.22 00:34:41.045789 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8471e54a8fb471ab075775e0f211339f for removal 2024.10.22 00:34:41.052006 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ac3b5419b8f09529f7ee44e9161c51e7, offset: 0, size: 257505 2024.10.22 00:34:41.054571 [ 1476 ] {} CacheMetadata: Key 8471e54a8fb471ab075775e0f211339f is removed from metadata 2024.10.22 00:34:41.057298 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ac3b5419b8f09529f7ee44e9161c51e7 for removal 2024.10.22 00:34:41.067562 [ 1476 ] {} CacheMetadata: Key ac3b5419b8f09529f7ee44e9161c51e7 is removed from metadata 2024.10.22 00:34:41.077102 [ 1476 ] {} CacheMetadata: Key 9b1e60a7612edff86bf9e46a4dd4f760 is removed from metadata 2024.10.22 00:34:41.351384 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 8 file segments (while reserving 967208 bytes for f373f0e79569bd9283457080fd8d15fd:0) 2024.10.22 00:34:41.356182 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7bca83807092b75e80503a1ec3c6695d, offset: 0, size: 411936 2024.10.22 00:34:41.463253 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7bca83807092b75e80503a1ec3c6695d for removal 2024.10.22 00:34:41.466424 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 93456f06ed240d9fa04fea440f2cb12d, offset: 0, size: 196 2024.10.22 00:34:41.466543 [ 1476 ] {} CacheMetadata: Key 7bca83807092b75e80503a1ec3c6695d is removed from metadata 2024.10.22 00:34:41.470584 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 93456f06ed240d9fa04fea440f2cb12d for removal 2024.10.22 00:34:41.474956 [ 1476 ] {} CacheMetadata: Key 93456f06ed240d9fa04fea440f2cb12d is removed from metadata 2024.10.22 00:34:41.476313 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 76a16b3ff3194c0df74ac9f0697bb1ab, offset: 0, size: 193 2024.10.22 00:34:41.479699 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 76a16b3ff3194c0df74ac9f0697bb1ab for removal 2024.10.22 00:34:41.481959 [ 1476 ] {} CacheMetadata: Key 76a16b3ff3194c0df74ac9f0697bb1ab is removed from metadata 2024.10.22 00:34:41.482458 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 809c1a2910798941e76806634c58592e, offset: 0, size: 122157 2024.10.22 00:34:41.486770 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 809c1a2910798941e76806634c58592e for removal 2024.10.22 00:34:41.488947 [ 1476 ] {} CacheMetadata: Key 809c1a2910798941e76806634c58592e is removed from metadata 2024.10.22 00:34:41.489023 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1bd25b16bf7e9d7d1ba40470fdb40540, offset: 0, size: 164 2024.10.22 00:34:41.491367 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1bd25b16bf7e9d7d1ba40470fdb40540 for removal 2024.10.22 00:34:41.493252 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6b2b4c88c82242d1260dc73a202a067a, offset: 0, size: 142921 2024.10.22 00:34:41.493582 [ 1476 ] {} CacheMetadata: Key 1bd25b16bf7e9d7d1ba40470fdb40540 is removed from metadata 2024.10.22 00:34:41.496232 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6b2b4c88c82242d1260dc73a202a067a for removal 2024.10.22 00:34:41.500122 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aa472ec419d2b9a2914d0011f3de3c4f, offset: 0, size: 385742 2024.10.22 00:34:41.505490 [ 1476 ] {} CacheMetadata: Key 6b2b4c88c82242d1260dc73a202a067a is removed from metadata 2024.10.22 00:34:41.554842 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 900 2024.10.22 00:34:41.608510 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aa472ec419d2b9a2914d0011f3de3c4f for removal 2024.10.22 00:34:41.614707 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 001a9d1acefdda36d29be0b58f07c87f, offset: 0, size: 190 2024.10.22 00:34:41.617733 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 001a9d1acefdda36d29be0b58f07c87f for removal 2024.10.22 00:34:41.716169 [ 1476 ] {} CacheMetadata: Key aa472ec419d2b9a2914d0011f3de3c4f is removed from metadata 2024.10.22 00:34:41.722898 [ 1476 ] {} CacheMetadata: Key 001a9d1acefdda36d29be0b58f07c87f is removed from metadata 2024.10.22 00:34:41.883750 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 981588 bytes for 832fa4a4ce6948100a7f87eb9015bc07:0) 2024.10.22 00:34:41.888902 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8a2aa46e0fee65e624915fe945f64e46, offset: 0, size: 2020073 2024.10.22 00:34:41.893441 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8a2aa46e0fee65e624915fe945f64e46 for removal 2024.10.22 00:34:41.895866 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d96ec48fac8739682461529b1a99546f, offset: 0, size: 198752 2024.10.22 00:34:41.895895 [ 1476 ] {} CacheMetadata: Key 8a2aa46e0fee65e624915fe945f64e46 is removed from metadata 2024.10.22 00:34:42.004346 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d96ec48fac8739682461529b1a99546f for removal 2024.10.22 00:34:42.012614 [ 1476 ] {} CacheMetadata: Key d96ec48fac8739682461529b1a99546f is removed from metadata 2024.10.22 00:34:42.012284 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dbb5da86b28fbfc5b7a8b02c1caaf360, offset: 0, size: 258 2024.10.22 00:34:42.017844 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dbb5da86b28fbfc5b7a8b02c1caaf360 for removal 2024.10.22 00:34:42.122180 [ 1476 ] {} CacheMetadata: Key dbb5da86b28fbfc5b7a8b02c1caaf360 is removed from metadata 2024.10.22 00:34:42.122244 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1f2531310a1e54e7f6dde6b880af8ea7, offset: 0, size: 192 2024.10.22 00:34:42.146844 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1f2531310a1e54e7f6dde6b880af8ea7 for removal 2024.10.22 00:34:42.229674 [ 1476 ] {} CacheMetadata: Key 1f2531310a1e54e7f6dde6b880af8ea7 is removed from metadata 2024.10.22 00:34:42.261650 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1516 entries to flush up to offset 189448 2024.10.22 00:34:42.736624 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yhz/ueiaizunikhmvbuxmfbadcmphsxgh. 2024.10.22 00:34:42.781496 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tts/aeaktfpvhzxnvwmngisqjmkkshyaq. 2024.10.22 00:34:42.825578 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 172 entries to flush up to offset 8577 2024.10.22 00:34:43.295691 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:43.297943 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:43.419876 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_107_107_0 (state Temporary) 2024.10.22 00:34:43.538691 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.81 GiB, free memory in arenas 4.94 MiB, will set to 1.60 GiB (RSS), difference: -10.08 MiB 2024.10.22 00:34:43.556030 [ 1199 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 00:34:43.696900 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_107_107_0 to 202410_114_114_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:43.700938 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tts/aeaktfpvhzxnvwmngisqjmkkshyaq. 2024.10.22 00:34:43.709445 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_7_7_0 to 201403_7_7_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:43.794788 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yhz/ueiaizunikhmvbuxmfbadcmphsxgh. 2024.10.22 00:34:43.807174 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 189448 2024.10.22 00:34:43.816753 [ 1175 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937) (MergerMutator): Selected 7 parts from 201403_1_1_0 to 201403_7_7_0 2024.10.22 00:34:43.831252 [ 1175 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 437.23 MiB using storage policy from min volume index 0 2024.10.22 00:34:44.060451 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTask::PrepareStage: Merging 7 parts: from 201403_1_1_0 to 201403_7_7_0 into Wide with storage Full 2024.10.22 00:34:44.096527 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_8_8_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:34:44.408901 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:44.414395 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:44.449404 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Vertical 2024.10.22 00:34:44.478120 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part 2024.10.22 00:34:44.478675 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_193_193_0 (state Temporary) 2024.10.22 00:34:44.489394 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part 2024.10.22 00:34:44.525333 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part 2024.10.22 00:34:44.641161 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part 2024.10.22 00:34:44.679844 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part 2024.10.22 00:34:44.696841 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part 2024.10.22 00:34:44.736096 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part 2024.10.22 00:34:44.792690 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/equ/aviniscxwlanrpdldqzjxjlgorshd (LogSeriesLimiter: on interval from 2024-10-22 00:33:36 to 2024-10-22 00:34:44 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:34:44.891755 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_193_193_0 to 202410_206_206_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:44.903580 [ 1217 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_201_40 to 202410_206_206_0 2024.10.22 00:34:44.927309 [ 1217 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:44.930438 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8577 2024.10.22 00:34:44.932349 [ 1217 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:44.963680 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_201_40 to 202410_206_206_0 into Compact with storage Full 2024.10.22 00:34:45.093465 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:34:45.103354 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_201_40, total 8918 rows starting from the beginning of the part 2024.10.22 00:34:45.133382 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_202_202_0, total 63 rows starting from the beginning of the part 2024.10.22 00:34:45.163299 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_203_203_0, total 93 rows starting from the beginning of the part 2024.10.22 00:34:45.189166 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_204_204_0, total 90 rows starting from the beginning of the part 2024.10.22 00:34:45.218426 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_205_205_0, total 72 rows starting from the beginning of the part 2024.10.22 00:34:45.240611 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_206_206_0, total 172 rows starting from the beginning of the part 2024.10.22 00:34:45.683366 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.06 GiB, peak 1.81 GiB, free memory in arenas 2.72 MiB, will set to 1.36 GiB (RSS), difference: 309.33 MiB 2024.10.22 00:34:46.340455 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 107 2024.10.22 00:34:46.967218 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:46.984348 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:47.110846 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 1579 2024.10.22 00:34:47.170241 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_89_89_0 (state Temporary) 2024.10.22 00:34:47.464157 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:47.470742 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:47.481377 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_89_89_0 to 202410_99_99_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:47.508091 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_112_112_0 (state Temporary) 2024.10.22 00:34:47.569591 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_112_112_0 to 202410_125_125_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:47.604611 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1579 2024.10.22 00:34:47.673931 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 107 2024.10.22 00:34:47.775844 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} MergeTask::MergeProjectionsStage: Merge sorted 9408 rows, containing 13 columns (13 merged, 0 gathered) in 2.820010397 sec., 3336.157912753965 rows/sec., 508.75 KiB/sec. 2024.10.22 00:34:47.819899 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.81 GiB, free memory in arenas 2.17 MiB, will set to 1.40 GiB (RSS), difference: -7.08 MiB 2024.10.22 00:34:47.935344 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} MergedBlockOutputStream: filled checksums 202410_1_206_41 (state Temporary) 2024.10.22 00:34:47.994635 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_206_41 to 202410_1_206_41 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:47.997590 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_206_41} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_201_40, 202410_206_206_0] -> 202410_1_206_41 2024.10.22 00:34:48.080691 [ 1191 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 00:34:48.285600 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 7494 2024.10.22 00:34:48.534513 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.66 MiB. 2024.10.22 00:34:48.561524 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_24_5/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:34:48.767821 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:48.775680 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:48.971259 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 8641 2024.10.22 00:34:49.033250 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_111_111_0 (state Temporary) 2024.10.22 00:34:49.120626 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:49.125531 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:49.314190 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_111_111_0 to 202410_117_117_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:49.376469 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7494 2024.10.22 00:34:49.385062 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_194_194_0 (state Temporary) 2024.10.22 00:34:49.481586 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:34:49.485052 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:34:49.496786 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_194_194_0 to 202410_207_207_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:49.624947 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.81 GiB, free memory in arenas 2.75 MiB, will set to 1.43 GiB (RSS), difference: -45.67 MiB 2024.10.22 00:34:49.769090 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8641 2024.10.22 00:34:50.046798 [ 1190 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:34:50.072985 [ 1190 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:34:50.817313 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1556 entries to flush up to offset 191004 2024.10.22 00:34:50.885118 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 7 2024.10.22 00:34:51.164004 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:51.170468 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:51.320380 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_7_7_0 (state Temporary) 2024.10.22 00:34:51.385478 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_7_7_0 to 202410_7_7_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:51.459706 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 7 2024.10.22 00:34:51.640356 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.81 GiB, free memory in arenas 2.25 MiB, will set to 1.45 GiB (RSS), difference: -9.75 MiB 2024.10.22 00:34:51.673464 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:51.675646 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:51.988496 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_108_108_0 (state Temporary) 2024.10.22 00:34:52.068540 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_108_108_0 to 202410_115_115_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:52.089101 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 191004 2024.10.22 00:34:53.532224 [ 1257 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 00:34:53.564765 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.81 GiB, free memory in arenas 2.69 MiB, will set to 1.52 GiB (RSS), difference: -82.98 MiB 2024.10.22 00:34:53.592641 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_61_12, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0, 202410_66_66_0] 2024.10.22 00:34:53.594502 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_61_12, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0, 202410_66_66_0] 2024.10.22 00:34:53.728401 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_61_12, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0, 202410_66_66_0] 2024.10.22 00:34:53.766440 [ 1258 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 00:34:53.776711 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 8674 2024.10.22 00:34:54.108707 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:54.114855 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:54.265222 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_195_195_0 (state Temporary) 2024.10.22 00:34:54.501477 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_195_195_0 to 202410_208_208_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:54.572882 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8674 2024.10.22 00:34:55.109392 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 1593 2024.10.22 00:34:55.175732 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 108 2024.10.22 00:34:55.337548 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:55.341009 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:55.520994 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.81 GiB, free memory in arenas 5.06 MiB, will set to 1.56 GiB (RSS), difference: -23.66 MiB 2024.10.22 00:34:55.585047 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:55.585372 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_113_113_0 (state Temporary) 2024.10.22 00:34:55.587805 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:55.633293 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_90_90_0 (state Temporary) 2024.10.22 00:34:55.802910 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_90_90_0 to 202410_100_100_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:55.856210 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_113_113_0 to 202410_126_126_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:55.876759 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_101_20, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0] 2024.10.22 00:34:55.877780 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_101_20, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0] 2024.10.22 00:34:55.878526 [ 1278 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_121_24 to 202410_126_126_0 2024.10.22 00:34:55.885607 [ 1278 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:55.888781 [ 1278 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:55.892360 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1593 2024.10.22 00:34:55.912142 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_101_20, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0] 2024.10.22 00:34:55.922424 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_121_24 to 202410_126_126_0 into Compact with storage Full 2024.10.22 00:34:55.961403 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:34:55.972957 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_121_24, total 1628 rows starting from the beginning of the part 2024.10.22 00:34:55.979387 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 108 2024.10.22 00:34:56.029285 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_122_122_0, total 19 rows starting from the beginning of the part 2024.10.22 00:34:56.045496 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_123_123_0, total 13 rows starting from the beginning of the part 2024.10.22 00:34:56.061832 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_124_124_0, total 20 rows starting from the beginning of the part 2024.10.22 00:34:56.085056 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_125_125_0, total 5 rows starting from the beginning of the part 2024.10.22 00:34:56.107518 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_126_126_0, total 14 rows starting from the beginning of the part 2024.10.22 00:34:56.996243 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 53 entries to flush up to offset 7547 2024.10.22 00:34:57.017285 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} MergeTask::MergeProjectionsStage: Merge sorted 1699 rows, containing 12 columns (12 merged, 0 gathered) in 1.101709508 sec., 1542.1488038932312 rows/sec., 237.89 KiB/sec. 2024.10.22 00:34:57.051245 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_8_8_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:34:57.096135 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} MergedBlockOutputStream: filled checksums 202410_1_126_25 (state Temporary) 2024.10.22 00:34:57.161827 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:57.168651 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:57.324021 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_112_112_0 (state Temporary) 2024.10.22 00:34:57.617793 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.81 GiB, free memory in arenas 5.83 MiB, will set to 1.62 GiB (RSS), difference: -51.49 MiB 2024.10.22 00:34:57.678136 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_126_25 to 202410_1_126_25 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:57.681689 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_126_25} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_121_24, 202410_126_126_0] -> 202410_1_126_25 2024.10.22 00:34:57.778434 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_112_112_0 to 202410_118_118_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:57.823971 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7547 2024.10.22 00:34:57.937404 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:34:58.576665 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 8720 2024.10.22 00:34:58.701389 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:58.709811 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:58.884667 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_196_196_0 (state Temporary) 2024.10.22 00:34:58.952589 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_196_196_0 to 202410_209_209_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:34:58.991078 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8720 2024.10.22 00:34:59.136937 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 192540 2024.10.22 00:34:59.223215 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 261.76 MiB using storage policy from min volume index 0 2024.10.22 00:34:59.482287 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:34:59.540594 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 10 file segments (while reserving 1048576 bytes for a55d1fd35893ea10476be244c2ea8973:0) 2024.10.22 00:34:59.543933 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1d4fad01761a89cc186224a263257926, offset: 0, size: 97639 2024.10.22 00:34:59.548596 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1d4fad01761a89cc186224a263257926 for removal 2024.10.22 00:34:59.569144 [ 1476 ] {} CacheMetadata: Key 1d4fad01761a89cc186224a263257926 is removed from metadata 2024.10.22 00:34:59.572443 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c8b06e4702c659271030f3d6b5abcc65, offset: 0, size: 161 2024.10.22 00:34:59.576210 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c8b06e4702c659271030f3d6b5abcc65 for removal 2024.10.22 00:34:59.578286 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7b295fc984386807b5e823786c62b294, offset: 0, size: 271939 2024.10.22 00:34:59.582565 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7b295fc984386807b5e823786c62b294 for removal 2024.10.22 00:34:59.583695 [ 1476 ] {} CacheMetadata: Key c8b06e4702c659271030f3d6b5abcc65 is removed from metadata 2024.10.22 00:34:59.584289 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 716ad2fa0de51b8fa3c3407768070788, offset: 0, size: 201229 2024.10.22 00:34:59.586363 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 716ad2fa0de51b8fa3c3407768070788 for removal 2024.10.22 00:34:59.590648 [ 1476 ] {} CacheMetadata: Key 7b295fc984386807b5e823786c62b294 is removed from metadata 2024.10.22 00:34:59.484131 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:34:59.591203 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e7f0086dba085b1194ff259e48320136, offset: 0, size: 116382 2024.10.22 00:34:59.594202 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e7f0086dba085b1194ff259e48320136 for removal 2024.10.22 00:34:59.595540 [ 1476 ] {} CacheMetadata: Key 716ad2fa0de51b8fa3c3407768070788 is removed from metadata 2024.10.22 00:34:59.600157 [ 1476 ] {} CacheMetadata: Key e7f0086dba085b1194ff259e48320136 is removed from metadata 2024.10.22 00:34:59.607074 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a5da9f06f13dc540809cda307b4c5092, offset: 0, size: 163 2024.10.22 00:34:59.611150 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a5da9f06f13dc540809cda307b4c5092 for removal 2024.10.22 00:34:59.616470 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4f83e980ccde60f97129d98c3b096030, offset: 0, size: 232 2024.10.22 00:34:59.621343 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4f83e980ccde60f97129d98c3b096030 for removal 2024.10.22 00:34:59.625210 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0ed38a47f267335fb4b26aaeebb5d9a7, offset: 0, size: 101110 2024.10.22 00:34:59.647420 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.81 GiB, free memory in arenas 7.89 MiB, will set to 1.64 GiB (RSS), difference: -34.64 MiB 2024.10.22 00:34:59.704875 [ 1476 ] {} CacheMetadata: Key 4f83e980ccde60f97129d98c3b096030 is removed from metadata 2024.10.22 00:34:59.713517 [ 1476 ] {} CacheMetadata: Key a5da9f06f13dc540809cda307b4c5092 is removed from metadata 2024.10.22 00:34:59.729341 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0ed38a47f267335fb4b26aaeebb5d9a7 for removal 2024.10.22 00:34:59.734771 [ 1476 ] {} CacheMetadata: Key 0ed38a47f267335fb4b26aaeebb5d9a7 is removed from metadata 2024.10.22 00:34:59.734921 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8c051ad46e3904c98e93fc1036aba01b, offset: 0, size: 230 2024.10.22 00:34:59.739642 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8c051ad46e3904c98e93fc1036aba01b for removal 2024.10.22 00:34:59.744713 [ 1476 ] {} CacheMetadata: Key 8c051ad46e3904c98e93fc1036aba01b is removed from metadata 2024.10.22 00:34:59.744939 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0dba3921bf17f340f5852b872661e696, offset: 0, size: 164 2024.10.22 00:34:59.750087 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0dba3921bf17f340f5852b872661e696 for removal 2024.10.22 00:34:59.752733 [ 1476 ] {} CacheMetadata: Key 0dba3921bf17f340f5852b872661e696 is removed from metadata 2024.10.22 00:34:59.798623 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:34:59.803307 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:34:59.803505 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/crk/ijyqbozfrkpompfysqlxifkabynvc (LogSeriesLimiter: on interval from 2024-10-22 00:34:44 to 2024-10-22 00:34:59 accepted series 1 / 7 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:35:00.049707 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_109_109_0 (state Temporary) 2024.10.22 00:35:00.100152 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_109_109_0 to 202410_116_116_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:00.122830 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 192540 2024.10.22 00:35:00.149089 [ 1165 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_111_22 to 202410_116_116_0 2024.10.22 00:35:00.167652 [ 1165 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:00.172643 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:00.216588 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:00.221680 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:00.287184 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_111_22 to 202410_116_116_0 into Compact with storage Full 2024.10.22 00:35:00.311945 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:35:00.318984 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} MergeTreeSequentialSource: Reading 26 marks from part 202410_1_111_22, total 198066 rows starting from the beginning of the part 2024.10.22 00:35:00.333258 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_112_112_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:35:00.347166 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_113_113_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:35:00.362123 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_114_114_0, total 1516 rows starting from the beginning of the part 2024.10.22 00:35:00.371908 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_115_115_0, total 1556 rows starting from the beginning of the part 2024.10.22 00:35:00.382561 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_116_116_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:35:01.460259 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.81 GiB, free memory in arenas 9.79 MiB, will set to 1.64 GiB (RSS), difference: -112.73 MiB 2024.10.22 00:35:02.621557 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.81 GiB, free memory in arenas 11.06 MiB, will set to 1.64 GiB (RSS), difference: -83.80 MiB 2024.10.22 00:35:02.640385 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 00:35:02.983097 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_23_23_0 (state Temporary) 2024.10.22 00:35:03.002316 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 8785 2024.10.22 00:35:03.198403 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_24_5/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:35:03.234686 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:03.237681 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:03.395193 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 1605 2024.10.22 00:35:03.398392 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_197_197_0 (state Temporary) 2024.10.22 00:35:03.460956 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_197_197_0 to 202410_210_210_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:03.480978 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 109 2024.10.22 00:35:03.505372 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8785 2024.10.22 00:35:03.599193 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:03.603889 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:03.752317 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_114_114_0 (state Temporary) 2024.10.22 00:35:03.794258 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_114_114_0 to 202410_127_127_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:03.882456 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_116_23/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:35:03.983311 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1605 2024.10.22 00:35:04.002567 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:04.005464 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:04.073026 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_91_91_0 (state Temporary) 2024.10.22 00:35:04.111123 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pje/vceumghggjbnpbedpwqdttxntsunu (LogSeriesLimiter: on interval from 2024-10-22 00:34:59 to 2024-10-22 00:35:04 accepted series 1 / 184 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:35:04.375049 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_91_91_0 to 202410_101_101_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:04.375964 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.81 GiB, free memory in arenas 13.45 MiB, will set to 1.65 GiB (RSS), difference: -112.70 MiB 2024.10.22 00:35:04.547937 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 109 2024.10.22 00:35:05.077392 [ 1206 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:35:05.092215 [ 1206 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:35:05.171702 [ 1200 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 00:35:05.336778 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 68 entries to flush up to offset 7615 2024.10.22 00:35:05.419983 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.81 GiB, free memory in arenas 13.08 MiB, will set to 1.65 GiB (RSS), difference: -49.41 MiB 2024.10.22 00:35:05.787186 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:05.790815 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:05.825200 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_113_113_0 (state Temporary) 2024.10.22 00:35:05.978965 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_113_113_0 to 202410_119_119_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:06.027168 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7615 2024.10.22 00:35:07.126760 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 194460 2024.10.22 00:35:07.328867 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.83 GiB, peak 1.83 GiB, free memory in arenas 14.75 MiB, will set to 1.65 GiB (RSS), difference: -183.92 MiB 2024.10.22 00:35:07.523798 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 8819 2024.10.22 00:35:07.694917 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:07.697719 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:07.876326 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_198_198_0 (state Temporary) 2024.10.22 00:35:08.166279 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_198_198_0 to 202410_211_211_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:08.170407 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:08.173787 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:08.224081 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_206_41 to 202410_211_211_0 2024.10.22 00:35:08.227529 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8819 2024.10.22 00:35:08.251511 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:08.255962 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:08.287446 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_206_41 to 202410_211_211_0 into Compact with storage Full 2024.10.22 00:35:08.351493 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:35:08.415345 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_206_41, total 9408 rows starting from the beginning of the part 2024.10.22 00:35:08.470875 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_110_110_0 (state Temporary) 2024.10.22 00:35:08.476242 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_207_207_0, total 64 rows starting from the beginning of the part 2024.10.22 00:35:08.565453 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_208_208_0, total 33 rows starting from the beginning of the part 2024.10.22 00:35:08.606149 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_209_209_0, total 46 rows starting from the beginning of the part 2024.10.22 00:35:08.687385 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_110_110_0 to 202410_117_117_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:08.693935 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_210_210_0, total 65 rows starting from the beginning of the part 2024.10.22 00:35:08.747626 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 194460 2024.10.22 00:35:08.791413 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_211_211_0, total 34 rows starting from the beginning of the part 2024.10.22 00:35:09.122246 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_55_11, 202410_56_56_0, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0] 2024.10.22 00:35:09.124825 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_55_11, 202410_56_56_0, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0] 2024.10.22 00:35:09.355264 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} MergeTask::MergeProjectionsStage: Merge sorted 206130 rows, containing 4 columns (4 merged, 0 gathered) in 9.071840625 sec., 22721.96002120573 rows/sec., 333.71 KiB/sec. 2024.10.22 00:35:09.369543 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_55_11, 202410_56_56_0, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0] 2024.10.22 00:35:09.483299 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:35:09.485834 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:35:09.510929 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} MergedBlockOutputStream: filled checksums 202410_1_116_23 (state Temporary) 2024.10.22 00:35:09.586280 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_116_23 to 202410_1_116_23 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:09.590721 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_116_23} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_111_22, 202410_116_116_0] -> 202410_1_116_23 2024.10.22 00:35:09.778259 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.83 GiB, free memory in arenas 9.40 MiB, will set to 1.66 GiB (RSS), difference: -8.84 MiB 2024.10.22 00:35:09.828135 [ 1182 ] {} DatabaseCatalog: Execution took 235 ms. 2024.10.22 00:35:09.875489 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:35:10.856985 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 00:35:11.178851 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} MergeTask::MergeProjectionsStage: Merge sorted 9650 rows, containing 13 columns (13 merged, 0 gathered) in 2.909052509 sec., 3317.2312875566595 rows/sec., 505.34 KiB/sec. 2024.10.22 00:35:11.270801 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} MergedBlockOutputStream: filled checksums 202410_1_211_42 (state Temporary) 2024.10.22 00:35:11.484759 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 1618 2024.10.22 00:35:11.528904 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.83 GiB, free memory in arenas 9.93 MiB, will set to 1.66 GiB (RSS), difference: 8.19 MiB 2024.10.22 00:35:11.608714 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:11.617176 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:11.636787 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_211_42 to 202410_1_211_42 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:11.641695 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_211_42} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_206_41, 202410_211_211_0] -> 202410_1_211_42 2024.10.22 00:35:11.772439 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_115_115_0 (state Temporary) 2024.10.22 00:35:11.922649 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_115_115_0 to 202410_128_128_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:11.951088 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.69 MiB. 2024.10.22 00:35:11.963956 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1618 2024.10.22 00:35:12.056721 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 110 2024.10.22 00:35:12.216408 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} MergeTask::MergeProjectionsStage: Merge sorted 878 rows, containing 679 columns (679 merged, 0 gathered) in 80.774629085 sec., 10.869749696728057 rows/sec., 14.04 KiB/sec. 2024.10.22 00:35:12.248792 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 8868 2024.10.22 00:35:12.372223 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:12.475976 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:12.647041 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_92_92_0 (state Temporary) 2024.10.22 00:35:12.658198 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:12.674980 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:12.768876 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_92_92_0 to 202410_102_102_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:12.877521 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 110 2024.10.22 00:35:12.898797 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_199_199_0 (state Temporary) 2024.10.22 00:35:12.920589 [ 1236 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 203 ms. 2024.10.22 00:35:12.929643 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_23_23_0 to 202410_26_26_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:12.964179 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_199_199_0 to 202410_212_212_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:13.009490 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8868 2024.10.22 00:35:13.082611 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_24_5/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:35:13.335638 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.83 GiB, free memory in arenas 7.81 MiB, will set to 1.66 GiB (RSS), difference: 9.82 MiB 2024.10.22 00:35:13.542156 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 83 entries to flush up to offset 7698 2024.10.22 00:35:13.758378 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} MergedBlockOutputStream: filled checksums 202410_1_24_5 (state Temporary) 2024.10.22 00:35:13.873765 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:13.875733 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:14.020871 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_114_114_0 (state Temporary) 2024.10.22 00:35:14.076916 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column WatchID 2024.10.22 00:35:14.082642 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column WatchID 2024.10.22 00:35:14.089332 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column WatchID 2024.10.22 00:35:14.090720 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_114_114_0 to 202410_120_120_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:14.149576 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7698 2024.10.22 00:35:14.155786 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column WatchID 2024.10.22 00:35:14.158035 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_115_23 to 202410_120_120_0 2024.10.22 00:35:14.167832 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column WatchID 2024.10.22 00:35:14.177186 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:14.179325 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:14.179298 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column WatchID 2024.10.22 00:35:14.184080 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column WatchID 2024.10.22 00:35:14.190081 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_115_23 to 202410_120_120_0 into Compact with storage Full 2024.10.22 00:35:14.278557 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:35:14.305423 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_115_23, total 7437 rows starting from the beginning of the part 2024.10.22 00:35:14.465053 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_116_116_0, total 61 rows starting from the beginning of the part 2024.10.22 00:35:14.499261 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_117_117_0, total 100 rows starting from the beginning of the part 2024.10.22 00:35:14.550881 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_118_118_0, total 53 rows starting from the beginning of the part 2024.10.22 00:35:14.566767 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lfq/racxrqmhsbfxlzedeshjmogapfweh (LogSeriesLimiter: on interval from 2024-10-22 00:35:04 to 2024-10-22 00:35:14 accepted series 1 / 169 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:35:14.739174 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.83 GiB, free memory in arenas 11.34 MiB, will set to 1.67 GiB (RSS), difference: -1.25 MiB 2024.10.22 00:35:14.797450 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_119_119_0, total 68 rows starting from the beginning of the part 2024.10.22 00:35:15.066476 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_120_120_0, total 83 rows starting from the beginning of the part 2024.10.22 00:35:15.766554 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 196380 2024.10.22 00:35:16.081453 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} MergeTask::MergeProjectionsStage: Merge sorted 7802 rows, containing 13 columns (13 merged, 0 gathered) in 1.895261609 sec., 4116.58209238807 rows/sec., 1.72 MiB/sec. 2024.10.22 00:35:16.530474 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for cbd3c930dd72e3d362befb72c68430d5:0) 2024.10.22 00:35:16.533245 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 82c864d4cb21c0182369117ad3ada414, offset: 0, size: 1901758 2024.10.22 00:35:16.540221 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 82c864d4cb21c0182369117ad3ada414 for removal 2024.10.22 00:35:16.550545 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c428194428b11407a7139396af7ec311, offset: 0, size: 257 2024.10.22 00:35:16.557815 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c428194428b11407a7139396af7ec311 for removal 2024.10.22 00:35:16.563470 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} MergedBlockOutputStream: filled checksums 202410_1_120_24 (state Temporary) 2024.10.22 00:35:16.563235 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 839958788a41913b096b2b9644a37328, offset: 0, size: 746773 2024.10.22 00:35:16.639460 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.83 GiB, free memory in arenas 12.13 MiB, will set to 1.68 GiB (RSS), difference: -7.05 MiB 2024.10.22 00:35:16.652035 [ 1476 ] {} CacheMetadata: Key 82c864d4cb21c0182369117ad3ada414 is removed from metadata 2024.10.22 00:35:16.667955 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 839958788a41913b096b2b9644a37328 for removal 2024.10.22 00:35:16.670537 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5c6036693b3e8dfb6357c20ab8100ab0, offset: 0, size: 227 2024.10.22 00:35:16.673555 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5c6036693b3e8dfb6357c20ab8100ab0 for removal 2024.10.22 00:35:16.760554 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_120_24 to 202410_1_120_24 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:16.764858 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_120_24} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_115_23, 202410_120_120_0] -> 202410_1_120_24 2024.10.22 00:35:16.797460 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.72 MiB. 2024.10.22 00:35:16.872115 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:16.876768 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:16.920001 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 900 2024.10.22 00:35:17.017280 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 8930 2024.10.22 00:35:17.101486 [ 1476 ] {} CacheMetadata: Key 5c6036693b3e8dfb6357c20ab8100ab0 is removed from metadata 2024.10.22 00:35:17.108235 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_111_111_0 (state Temporary) 2024.10.22 00:35:17.109427 [ 1476 ] {} CacheMetadata: Key 839958788a41913b096b2b9644a37328 is removed from metadata 2024.10.22 00:35:17.121230 [ 1476 ] {} CacheMetadata: Key c428194428b11407a7139396af7ec311 is removed from metadata 2024.10.22 00:35:17.291652 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_111_111_0 to 202410_118_118_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:17.363792 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:17.369636 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:17.433192 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 196380 2024.10.22 00:35:17.455065 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_200_200_0 (state Temporary) 2024.10.22 00:35:17.643928 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_106_21, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0] 2024.10.22 00:35:17.644578 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_200_200_0 to 202410_213_213_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:17.647252 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_106_21, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0] 2024.10.22 00:35:17.705285 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8930 2024.10.22 00:35:17.780483 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_106_21, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0] 2024.10.22 00:35:18.791300 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.83 GiB, free memory in arenas 13.07 MiB, will set to 1.68 GiB (RSS), difference: 4.62 MiB 2024.10.22 00:35:18.875964 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 2097152 bytes for cbd3c930dd72e3d362befb72c68430d5:0) 2024.10.22 00:35:18.878756 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cde70b4cbfe6073b35d3ffe8818dc35b, offset: 0, size: 2001597 2024.10.22 00:35:18.883216 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cde70b4cbfe6073b35d3ffe8818dc35b for removal 2024.10.22 00:35:18.885650 [ 1476 ] {} CacheMetadata: Key cde70b4cbfe6073b35d3ffe8818dc35b is removed from metadata 2024.10.22 00:35:18.886985 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 19658c94b2ecbb554ceeaf7b6593b6d8, offset: 0, size: 159 2024.10.22 00:35:18.892119 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 19658c94b2ecbb554ceeaf7b6593b6d8 for removal 2024.10.22 00:35:18.893998 [ 1476 ] {} CacheMetadata: Key 19658c94b2ecbb554ceeaf7b6593b6d8 is removed from metadata 2024.10.22 00:35:18.894162 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d6f20792d985baa8ce7cac6ff34bbf6a, offset: 0, size: 128980 2024.10.22 00:35:18.898804 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d6f20792d985baa8ce7cac6ff34bbf6a for removal 2024.10.22 00:35:18.908203 [ 1476 ] {} CacheMetadata: Key d6f20792d985baa8ce7cac6ff34bbf6a is removed from metadata 2024.10.22 00:35:18.910191 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ecfe93b7702990de2a1401c08bc7416a, offset: 0, size: 5035 2024.10.22 00:35:18.922529 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ecfe93b7702990de2a1401c08bc7416a for removal 2024.10.22 00:35:18.928818 [ 1476 ] {} CacheMetadata: Key ecfe93b7702990de2a1401c08bc7416a is removed from metadata 2024.10.22 00:35:18.928890 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 73f5df1b2d39f52b8552b4ec68e4917b, offset: 0, size: 258 2024.10.22 00:35:18.941228 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 73f5df1b2d39f52b8552b4ec68e4917b for removal 2024.10.22 00:35:18.948519 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fc6e6cbd460879ddc05cfbfcd60339f1, offset: 0, size: 164 2024.10.22 00:35:18.949471 [ 1476 ] {} CacheMetadata: Key 73f5df1b2d39f52b8552b4ec68e4917b is removed from metadata 2024.10.22 00:35:18.953749 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fc6e6cbd460879ddc05cfbfcd60339f1 for removal 2024.10.22 00:35:18.959593 [ 1476 ] {} CacheMetadata: Key fc6e6cbd460879ddc05cfbfcd60339f1 is removed from metadata 2024.10.22 00:35:19.471110 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 1639 2024.10.22 00:35:19.485305 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:35:19.581042 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:19.584174 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:19.592308 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:35:19.625360 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_116_116_0 (state Temporary) 2024.10.22 00:35:19.774327 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_116_116_0 to 202410_129_129_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:20.022348 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1639 2024.10.22 00:35:20.097490 [ 1156 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:35:20.223052 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_56_11, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0, 202410_61_61_0] 2024.10.22 00:35:20.226312 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_56_11, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0, 202410_61_61_0] 2024.10.22 00:35:20.228977 [ 1156 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:35:20.383096 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 111 2024.10.22 00:35:20.388466 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_56_11, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0, 202410_60_60_0, 202410_61_61_0] 2024.10.22 00:35:20.453829 [ 1155 ] {} DatabaseCatalog: Execution took 624 ms. 2024.10.22 00:35:20.482823 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.83 GiB, free memory in arenas 12.58 MiB, will set to 1.68 GiB (RSS), difference: -1.40 MiB 2024.10.22 00:35:20.749074 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:20.754085 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:20.925549 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_93_93_0 (state Temporary) 2024.10.22 00:35:20.998461 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_93_93_0 to 202410_103_103_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:21.058683 [ 1162 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_98_20 to 202410_103_103_0 2024.10.22 00:35:21.081897 [ 1162 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:21.087378 [ 1162 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:21.127252 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_98_20 to 202410_103_103_0 into Compact with storage Full 2024.10.22 00:35:21.356504 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 111 2024.10.22 00:35:21.587356 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:35:21.723413 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 8991 2024.10.22 00:35:21.725047 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_98_20, total 142 rows starting from the beginning of the part 2024.10.22 00:35:21.772499 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 85 entries to flush up to offset 7783 2024.10.22 00:35:21.883293 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_99_99_0, total 1 rows starting from the beginning of the part 2024.10.22 00:35:22.124306 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:22.134686 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:22.139987 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 26 file segments (while reserving 4194304 bytes for cbd3c930dd72e3d362befb72c68430d5:0) 2024.10.22 00:35:22.174058 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:22.185761 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.31 GiB. 2024.10.22 00:35:22.192750 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_201_201_0 (state Temporary) 2024.10.22 00:35:22.244347 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 571248166b17cbcf2b1c64bf35a1bfe1, offset: 0, size: 1024724 2024.10.22 00:35:22.260865 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 571248166b17cbcf2b1c64bf35a1bfe1 for removal 2024.10.22 00:35:22.269671 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_115_115_0 (state Temporary) 2024.10.22 00:35:22.271424 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 90ebecfcc1adbfbbf1dbe7814888674e, offset: 0, size: 246 2024.10.22 00:35:22.275466 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 90ebecfcc1adbfbbf1dbe7814888674e for removal 2024.10.22 00:35:22.275665 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_201_201_0 to 202410_214_214_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:22.338714 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 8991 2024.10.22 00:35:22.369699 [ 1476 ] {} CacheMetadata: Key 571248166b17cbcf2b1c64bf35a1bfe1 is removed from metadata 2024.10.22 00:35:22.371345 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c0927d8d611b1e12c1f8344baec927e1, offset: 0, size: 605806 2024.10.22 00:35:22.378307 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c0927d8d611b1e12c1f8344baec927e1 for removal 2024.10.22 00:35:22.386484 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d69d3481755e5cf3c387fecdc6442637, offset: 0, size: 192 2024.10.22 00:35:22.394384 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d69d3481755e5cf3c387fecdc6442637 for removal 2024.10.22 00:35:22.395386 [ 1476 ] {} CacheMetadata: Key c0927d8d611b1e12c1f8344baec927e1 is removed from metadata 2024.10.22 00:35:22.406054 [ 1476 ] {} CacheMetadata: Key d69d3481755e5cf3c387fecdc6442637 is removed from metadata 2024.10.22 00:35:22.405673 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0408ad7bfb783126fc2cc58550200035, offset: 0, size: 238 2024.10.22 00:35:22.415000 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0408ad7bfb783126fc2cc58550200035 for removal 2024.10.22 00:35:22.419206 [ 1476 ] {} CacheMetadata: Key 90ebecfcc1adbfbbf1dbe7814888674e is removed from metadata 2024.10.22 00:35:22.419824 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.83 GiB, free memory in arenas 9.06 MiB, will set to 1.68 GiB (RSS), difference: 5.97 MiB 2024.10.22 00:35:22.422212 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6559d47e651896d7fb728d24104de209, offset: 0, size: 238176 2024.10.22 00:35:22.427111 [ 1476 ] {} CacheMetadata: Key 0408ad7bfb783126fc2cc58550200035 is removed from metadata 2024.10.22 00:35:22.428641 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6559d47e651896d7fb728d24104de209 for removal 2024.10.22 00:35:22.438498 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a3eb330e04b05fcc2960f952d54df123, offset: 0, size: 9508 2024.10.22 00:35:22.440680 [ 1476 ] {} CacheMetadata: Key 6559d47e651896d7fb728d24104de209 is removed from metadata 2024.10.22 00:35:22.444241 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a3eb330e04b05fcc2960f952d54df123 for removal 2024.10.22 00:35:22.453272 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f0227ef0626ef2006fe43cf7c2bab79c, offset: 0, size: 224 2024.10.22 00:35:22.455781 [ 1476 ] {} CacheMetadata: Key a3eb330e04b05fcc2960f952d54df123 is removed from metadata 2024.10.22 00:35:22.456637 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f0227ef0626ef2006fe43cf7c2bab79c for removal 2024.10.22 00:35:22.462189 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4288dce0fe90535f600fb598b8533af2, offset: 0, size: 246671 2024.10.22 00:35:22.468614 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_115_115_0 to 202410_121_121_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:22.468746 [ 1476 ] {} CacheMetadata: Key f0227ef0626ef2006fe43cf7c2bab79c is removed from metadata 2024.10.22 00:35:22.469044 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4288dce0fe90535f600fb598b8533af2 for removal 2024.10.22 00:35:22.473342 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d3ec04cbe663885de0dde92619617d68, offset: 0, size: 355250 2024.10.22 00:35:22.476986 [ 1476 ] {} CacheMetadata: Key 4288dce0fe90535f600fb598b8533af2 is removed from metadata 2024.10.22 00:35:22.477195 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d3ec04cbe663885de0dde92619617d68 for removal 2024.10.22 00:35:22.481043 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6c41cb2eb504441fcdcfafa37c844209, offset: 0, size: 233 2024.10.22 00:35:22.485404 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_100_100_0, total 1 rows starting from the beginning of the part 2024.10.22 00:35:22.485590 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6c41cb2eb504441fcdcfafa37c844209 for removal 2024.10.22 00:35:22.489194 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d9a74cea044535c54e8d75db4e24ac32, offset: 0, size: 572144 2024.10.22 00:35:22.492433 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d9a74cea044535c54e8d75db4e24ac32 for removal 2024.10.22 00:35:22.495256 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a81442e082214b3a41ba20bcaad6302d, offset: 0, size: 238 2024.10.22 00:35:22.497342 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7783 2024.10.22 00:35:22.551596 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_101_101_0, total 1 rows starting from the beginning of the part 2024.10.22 00:35:22.587143 [ 1476 ] {} CacheMetadata: Key d9a74cea044535c54e8d75db4e24ac32 is removed from metadata 2024.10.22 00:35:22.596305 [ 1476 ] {} CacheMetadata: Key 6c41cb2eb504441fcdcfafa37c844209 is removed from metadata 2024.10.22 00:35:22.497817 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a81442e082214b3a41ba20bcaad6302d for removal 2024.10.22 00:35:22.602434 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c91aa77fae88e2c9017c6579af8dab0f, offset: 0, size: 113044 2024.10.22 00:35:22.604135 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_102_102_0, total 1 rows starting from the beginning of the part 2024.10.22 00:35:22.606965 [ 1476 ] {} CacheMetadata: Key a81442e082214b3a41ba20bcaad6302d is removed from metadata 2024.10.22 00:35:22.618629 [ 1476 ] {} CacheMetadata: Key d3ec04cbe663885de0dde92619617d68 is removed from metadata 2024.10.22 00:35:22.622256 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c91aa77fae88e2c9017c6579af8dab0f for removal 2024.10.22 00:35:22.627687 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e747d754f19ff0f266b86d586b98eaaf, offset: 0, size: 465694 2024.10.22 00:35:22.628864 [ 1476 ] {} CacheMetadata: Key c91aa77fae88e2c9017c6579af8dab0f is removed from metadata 2024.10.22 00:35:22.633296 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e747d754f19ff0f266b86d586b98eaaf for removal 2024.10.22 00:35:22.639806 [ 1476 ] {} CacheMetadata: Key e747d754f19ff0f266b86d586b98eaaf is removed from metadata 2024.10.22 00:35:22.640434 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2e31c1454c7833cffbb92d63e915b968, offset: 0, size: 232 2024.10.22 00:35:22.648398 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2e31c1454c7833cffbb92d63e915b968 for removal 2024.10.22 00:35:22.650876 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_103_103_0, total 1 rows starting from the beginning of the part 2024.10.22 00:35:22.660885 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fd690acebc31a4e06006db0574eecb3b, offset: 0, size: 165 2024.10.22 00:35:22.665817 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fd690acebc31a4e06006db0574eecb3b for removal 2024.10.22 00:35:22.670555 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3da4a4a2a40e61fb5e2048e6d5f2ed28, offset: 0, size: 257283 2024.10.22 00:35:22.673187 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3da4a4a2a40e61fb5e2048e6d5f2ed28 for removal 2024.10.22 00:35:22.675629 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fecc0b8c0265b2decd8e86d6552a89d2, offset: 0, size: 168 2024.10.22 00:35:22.679202 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fecc0b8c0265b2decd8e86d6552a89d2 for removal 2024.10.22 00:35:22.682517 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 535c10a8dea2d7e4d2b61c9c5ecc6fa2, offset: 0, size: 190 2024.10.22 00:35:22.685129 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 535c10a8dea2d7e4d2b61c9c5ecc6fa2 for removal 2024.10.22 00:35:22.690322 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e55292308acbc2cf9232b197e13a9765, offset: 0, size: 171552 2024.10.22 00:35:22.694074 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e55292308acbc2cf9232b197e13a9765 for removal 2024.10.22 00:35:22.697123 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cfb0dd9aa1b9f9322eca7504a6135977, offset: 0, size: 144760 2024.10.22 00:35:22.699258 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cfb0dd9aa1b9f9322eca7504a6135977 for removal 2024.10.22 00:35:22.702539 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3e117e9e79911fd18d72379de7832434, offset: 0, size: 162 2024.10.22 00:35:22.708886 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3e117e9e79911fd18d72379de7832434 for removal 2024.10.22 00:35:22.715498 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1cbd7e19b439f7058b5d6a730dc5be76, offset: 0, size: 145348 2024.10.22 00:35:22.720699 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1cbd7e19b439f7058b5d6a730dc5be76 for removal 2024.10.22 00:35:22.727128 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e3b665fc27ad7416315a9bb44d448a6f, offset: 0, size: 230 2024.10.22 00:35:22.733116 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e3b665fc27ad7416315a9bb44d448a6f for removal 2024.10.22 00:35:22.746120 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a389ad31da15b3ecf44fa5658655b079, offset: 0, size: 167 2024.10.22 00:35:22.762275 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a389ad31da15b3ecf44fa5658655b079 for removal 2024.10.22 00:35:22.764047 [ 1476 ] {} CacheMetadata: Key e3b665fc27ad7416315a9bb44d448a6f is removed from metadata 2024.10.22 00:35:22.779724 [ 1476 ] {} CacheMetadata: Key a389ad31da15b3ecf44fa5658655b079 is removed from metadata 2024.10.22 00:35:22.799041 [ 1476 ] {} CacheMetadata: Key cfb0dd9aa1b9f9322eca7504a6135977 is removed from metadata 2024.10.22 00:35:22.837716 [ 1476 ] {} CacheMetadata: Key e55292308acbc2cf9232b197e13a9765 is removed from metadata 2024.10.22 00:35:22.954311 [ 1476 ] {} CacheMetadata: Key fd690acebc31a4e06006db0574eecb3b is removed from metadata 2024.10.22 00:35:22.979045 [ 1476 ] {} CacheMetadata: Key 3da4a4a2a40e61fb5e2048e6d5f2ed28 is removed from metadata 2024.10.22 00:35:22.992458 [ 1476 ] {} CacheMetadata: Key 535c10a8dea2d7e4d2b61c9c5ecc6fa2 is removed from metadata 2024.10.22 00:35:23.016056 [ 1476 ] {} CacheMetadata: Key 1cbd7e19b439f7058b5d6a730dc5be76 is removed from metadata 2024.10.22 00:35:23.027485 [ 1476 ] {} CacheMetadata: Key 3e117e9e79911fd18d72379de7832434 is removed from metadata 2024.10.22 00:35:23.046581 [ 1476 ] {} CacheMetadata: Key fecc0b8c0265b2decd8e86d6552a89d2 is removed from metadata 2024.10.22 00:35:23.065466 [ 1476 ] {} CacheMetadata: Key 2e31c1454c7833cffbb92d63e915b968 is removed from metadata 2024.10.22 00:35:23.955636 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_103_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:35:24.056001 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_50_10, 202410_51_51_0, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0] 2024.10.22 00:35:24.166839 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_50_10, 202410_51_51_0, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0] 2024.10.22 00:35:24.209059 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_50_10, 202410_51_51_0, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0, 202410_55_55_0] 2024.10.22 00:35:24.534438 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.83 GiB, free memory in arenas 8.31 MiB, will set to 1.68 GiB (RSS), difference: 118.26 MiB 2024.10.22 00:35:24.551160 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 197916 2024.10.22 00:35:24.568967 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 943 2024.10.22 00:35:25.313333 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} MergeTask::MergeProjectionsStage: Merge sorted 147 rows, containing 32 columns (32 merged, 0 gathered) in 4.216098505 sec., 34.866358038283074 rows/sec., 3.94 KiB/sec. 2024.10.22 00:35:25.376607 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} MergedBlockOutputStream: filled checksums 202410_1_103_21 (state Temporary) 2024.10.22 00:35:25.596025 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:25.600326 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.29 GiB. 2024.10.22 00:35:25.643756 [ 1737 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pje/vceumghggjbnpbedpwqdttxntsunu, size 156 2024.10.22 00:35:25.687035 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_103_21 to 202410_1_103_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:25.690563 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_103_21} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_98_20, 202410_103_103_0] -> 202410_1_103_21 2024.10.22 00:35:25.744678 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:35:25.792332 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_112_112_0 (state Temporary) 2024.10.22 00:35:25.983144 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_112_112_0 to 202410_119_119_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:26.025599 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 197916 2024.10.22 00:35:26.355986 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 116 entries to flush up to offset 9107 2024.10.22 00:35:26.679171 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.83 GiB, free memory in arenas 4.63 MiB, will set to 1.67 GiB (RSS), difference: 158.55 MiB 2024.10.22 00:35:26.686648 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:26.710655 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.29 GiB. 2024.10.22 00:35:26.852009 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_202_202_0 (state Temporary) 2024.10.22 00:35:26.977561 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_202_202_0 to 202410_215_215_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:27.156891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_9_9_0 (state Temporary) 2024.10.22 00:35:27.193005 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9107 2024.10.22 00:35:27.228212 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_24_5/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:35:27.271981 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uum/buvvnrtypiocjupyegkhpxhmgrjmz (LogSeriesLimiter: on interval from 2024-10-22 00:35:14 to 2024-10-22 00:35:27 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:35:27.315300 [ 1746 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/crk/ijyqbozfrkpompfysqlxifkabynvc, size 350 2024.10.22 00:35:27.348849 [ 1685 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uum/buvvnrtypiocjupyegkhpxhmgrjmz, size 4 2024.10.22 00:35:27.524762 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 1652 2024.10.22 00:35:27.728309 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:27.731493 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.27 GiB. 2024.10.22 00:35:27.764785 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_117_117_0 (state Temporary) 2024.10.22 00:35:27.935907 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_117_117_0 to 202410_130_130_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:27.995304 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1652 2024.10.22 00:35:28.401219 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.83 GiB, free memory in arenas 3.92 MiB, will set to 1.65 GiB (RSS), difference: 55.42 MiB 2024.10.22 00:35:29.371533 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 00:35:29.484818 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:35:29.589103 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:35:30.012545 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 7874 2024.10.22 00:35:30.315037 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 26 file segments (while reserving 8388608 bytes for cbd3c930dd72e3d362befb72c68430d5:0) 2024.10.22 00:35:30.320388 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cdd979f63fcef9342cae47bfbf5942bd, offset: 0, size: 2516969 2024.10.22 00:35:30.328283 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cdd979f63fcef9342cae47bfbf5942bd for removal 2024.10.22 00:35:30.338712 [ 1476 ] {} CacheMetadata: Key cdd979f63fcef9342cae47bfbf5942bd is removed from metadata 2024.10.22 00:35:30.339484 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4c4d6938728fa2a037672eb5ab2f6259, offset: 0, size: 1986806 2024.10.22 00:35:30.347816 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4c4d6938728fa2a037672eb5ab2f6259 for removal 2024.10.22 00:35:30.353921 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f8570f07811f0dff9afaaf3d0bff217c, offset: 0, size: 2404 2024.10.22 00:35:30.361049 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f8570f07811f0dff9afaaf3d0bff217c for removal 2024.10.22 00:35:30.366420 [ 1476 ] {} CacheMetadata: Key f8570f07811f0dff9afaaf3d0bff217c is removed from metadata 2024.10.22 00:35:30.367077 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3b107b52b24dd6b925bbf2b45ad67a28, offset: 0, size: 161 2024.10.22 00:35:30.369470 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3b107b52b24dd6b925bbf2b45ad67a28 for removal 2024.10.22 00:35:30.375308 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c4ebc53954b702413c16068372ec67ee, offset: 0, size: 226 2024.10.22 00:35:30.378609 [ 1476 ] {} CacheMetadata: Key 3b107b52b24dd6b925bbf2b45ad67a28 is removed from metadata 2024.10.22 00:35:30.382861 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c4ebc53954b702413c16068372ec67ee for removal 2024.10.22 00:35:30.387822 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1b28a7708149617e72880ace8271dddb, offset: 0, size: 535029 2024.10.22 00:35:30.391021 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1b28a7708149617e72880ace8271dddb for removal 2024.10.22 00:35:30.393374 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fffe498e0c0a5fd589c11ec650c0aa38, offset: 0, size: 620437 2024.10.22 00:35:30.399609 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fffe498e0c0a5fd589c11ec650c0aa38 for removal 2024.10.22 00:35:30.505120 [ 1476 ] {} CacheMetadata: Key fffe498e0c0a5fd589c11ec650c0aa38 is removed from metadata 2024.10.22 00:35:30.505742 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2575d6b49aa11e40b93b8e105c65459c, offset: 0, size: 252134 2024.10.22 00:35:30.513542 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2575d6b49aa11e40b93b8e105c65459c for removal 2024.10.22 00:35:30.515585 [ 1476 ] {} CacheMetadata: Key 1b28a7708149617e72880ace8271dddb is removed from metadata 2024.10.22 00:35:30.528465 [ 1476 ] {} CacheMetadata: Key 2575d6b49aa11e40b93b8e105c65459c is removed from metadata 2024.10.22 00:35:30.542506 [ 1476 ] {} CacheMetadata: Key c4ebc53954b702413c16068372ec67ee is removed from metadata 2024.10.22 00:35:30.567984 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:30.571205 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.28 GiB. 2024.10.22 00:35:30.517684 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fc5a0d24e51d85c0e2a6b68c9feffc4b, offset: 0, size: 1023075 2024.10.22 00:35:30.625696 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fc5a0d24e51d85c0e2a6b68c9feffc4b for removal 2024.10.22 00:35:30.630058 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a14c9a8a3202f45d1467ad2604954f86, offset: 0, size: 543667 2024.10.22 00:35:30.636962 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a14c9a8a3202f45d1467ad2604954f86 for removal 2024.10.22 00:35:30.641362 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c3e4045a7bf3a392b46e50175cc80c78, offset: 0, size: 232 2024.10.22 00:35:30.647631 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c3e4045a7bf3a392b46e50175cc80c78 for removal 2024.10.22 00:35:30.651851 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3d54cf19e3d748c8c6840e77ad8b444d, offset: 0, size: 413871 2024.10.22 00:35:30.655745 [ 1476 ] {} CacheMetadata: Key c3e4045a7bf3a392b46e50175cc80c78 is removed from metadata 2024.10.22 00:35:30.660013 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3d54cf19e3d748c8c6840e77ad8b444d for removal 2024.10.22 00:35:30.667562 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7d5ec0addd78a8c4cb6bc1c361130bdc, offset: 0, size: 252 2024.10.22 00:35:30.668911 [ 1476 ] {} CacheMetadata: Key 3d54cf19e3d748c8c6840e77ad8b444d is removed from metadata 2024.10.22 00:35:30.672817 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7d5ec0addd78a8c4cb6bc1c361130bdc for removal 2024.10.22 00:35:30.677971 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5f47f15811798008ffd34513d3a36706, offset: 0, size: 229 2024.10.22 00:35:30.678234 [ 1476 ] {} CacheMetadata: Key 7d5ec0addd78a8c4cb6bc1c361130bdc is removed from metadata 2024.10.22 00:35:30.680069 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5f47f15811798008ffd34513d3a36706 for removal 2024.10.22 00:35:30.683011 [ 1476 ] {} CacheMetadata: Key 5f47f15811798008ffd34513d3a36706 is removed from metadata 2024.10.22 00:35:30.689746 [ 1476 ] {} CacheMetadata: Key fc5a0d24e51d85c0e2a6b68c9feffc4b is removed from metadata 2024.10.22 00:35:30.696636 [ 1476 ] {} CacheMetadata: Key a14c9a8a3202f45d1467ad2604954f86 is removed from metadata 2024.10.22 00:35:30.705208 [ 1476 ] {} CacheMetadata: Key 4c4d6938728fa2a037672eb5ab2f6259 is removed from metadata 2024.10.22 00:35:30.743827 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_116_116_0 (state Temporary) 2024.10.22 00:35:30.774083 [ 1183 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 247 ms. 2024.10.22 00:35:30.784374 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9cb87467853e8ce137c6a9eb24db4854, offset: 0, size: 237 2024.10.22 00:35:30.787720 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9cb87467853e8ce137c6a9eb24db4854 for removal 2024.10.22 00:35:30.788614 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_116_116_0 to 202410_122_122_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:30.790728 [ 1476 ] {} CacheMetadata: Key 9cb87467853e8ce137c6a9eb24db4854 is removed from metadata 2024.10.22 00:35:30.790995 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a34f1a9f826aaeb9a82080f928e77f89, offset: 0, size: 308676 2024.10.22 00:35:30.794321 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a34f1a9f826aaeb9a82080f928e77f89 for removal 2024.10.22 00:35:30.799810 [ 1476 ] {} CacheMetadata: Key a34f1a9f826aaeb9a82080f928e77f89 is removed from metadata 2024.10.22 00:35:30.799759 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cbe0ce7e6846cdce95eb40d86b5fb4b0, offset: 0, size: 238 2024.10.22 00:35:30.805117 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cbe0ce7e6846cdce95eb40d86b5fb4b0 for removal 2024.10.22 00:35:30.808445 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1be27558537f148122acc0cb13af3c01, offset: 0, size: 241 2024.10.22 00:35:30.813736 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1be27558537f148122acc0cb13af3c01 for removal 2024.10.22 00:35:30.820578 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d020c0e4d01327f53c86c5655477ee63, offset: 0, size: 260 2024.10.22 00:35:30.827888 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d020c0e4d01327f53c86c5655477ee63 for removal 2024.10.22 00:35:30.837498 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ac9c3c9a56d20ea9ca352d098e37ee21, offset: 0, size: 235 2024.10.22 00:35:30.846730 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ac9c3c9a56d20ea9ca352d098e37ee21 for removal 2024.10.22 00:35:30.851783 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5bd61b469ee8c22137e7ba0e1e6ce584, offset: 0, size: 239 2024.10.22 00:35:30.855445 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5bd61b469ee8c22137e7ba0e1e6ce584 for removal 2024.10.22 00:35:30.858467 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 41648cc877d28dd78033d5b7121a19f3, offset: 0, size: 656233 2024.10.22 00:35:30.941340 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7874 2024.10.22 00:35:30.967975 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 41648cc877d28dd78033d5b7121a19f3 for removal 2024.10.22 00:35:30.972054 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 77b13f3008e0b0fa76c71ac14b92a680, offset: 0, size: 307683 2024.10.22 00:35:30.976288 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 77b13f3008e0b0fa76c71ac14b92a680 for removal 2024.10.22 00:35:30.979767 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a726f702060498dd2a76261d53276406, offset: 0, size: 714721 2024.10.22 00:35:30.984067 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a726f702060498dd2a76261d53276406 for removal 2024.10.22 00:35:30.989814 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 43d71074a8880b0200656a0ad8680a6c, offset: 0, size: 239 2024.10.22 00:35:30.999096 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 43d71074a8880b0200656a0ad8680a6c for removal 2024.10.22 00:35:31.004640 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 70c8d38e4610d399f54b95df67f04132, offset: 0, size: 246 2024.10.22 00:35:31.010968 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 70c8d38e4610d399f54b95df67f04132 for removal 2024.10.22 00:35:31.015071 [ 1476 ] {} CacheMetadata: Key 43d71074a8880b0200656a0ad8680a6c is removed from metadata 2024.10.22 00:35:31.027085 [ 1476 ] {} CacheMetadata: Key 77b13f3008e0b0fa76c71ac14b92a680 is removed from metadata 2024.10.22 00:35:31.038153 [ 1476 ] {} CacheMetadata: Key 41648cc877d28dd78033d5b7121a19f3 is removed from metadata 2024.10.22 00:35:31.048110 [ 1476 ] {} CacheMetadata: Key a726f702060498dd2a76261d53276406 is removed from metadata 2024.10.22 00:35:31.058235 [ 1476 ] {} CacheMetadata: Key 5bd61b469ee8c22137e7ba0e1e6ce584 is removed from metadata 2024.10.22 00:35:31.170325 [ 1476 ] {} CacheMetadata: Key ac9c3c9a56d20ea9ca352d098e37ee21 is removed from metadata 2024.10.22 00:35:31.179558 [ 1476 ] {} CacheMetadata: Key 70c8d38e4610d399f54b95df67f04132 is removed from metadata 2024.10.22 00:35:31.189286 [ 1476 ] {} CacheMetadata: Key d020c0e4d01327f53c86c5655477ee63 is removed from metadata 2024.10.22 00:35:31.198357 [ 1476 ] {} CacheMetadata: Key 1be27558537f148122acc0cb13af3c01 is removed from metadata 2024.10.22 00:35:31.216046 [ 1476 ] {} CacheMetadata: Key cbe0ce7e6846cdce95eb40d86b5fb4b0 is removed from metadata 2024.10.22 00:35:31.225597 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 107 entries to flush up to offset 9214 2024.10.22 00:35:31.504538 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_24_5 to 202410_1_24_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:31.511120 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_24_5} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_19_4, 202410_24_24_0] -> 202410_1_24_5 2024.10.22 00:35:31.583530 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:31.585449 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.27 GiB. 2024.10.22 00:35:31.817125 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_203_203_0 (state Temporary) 2024.10.22 00:35:31.846937 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.83 GiB, free memory in arenas 3.53 MiB, will set to 1.65 GiB (RSS), difference: -14.12 MiB 2024.10.22 00:35:31.884378 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_203_203_0 to 202410_216_216_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:32.044926 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9214 2024.10.22 00:35:32.099022 [ 1228 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_211_42 to 202410_216_216_0 2024.10.22 00:35:32.124229 [ 1228 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:32.127791 [ 1228 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.27 GiB. 2024.10.22 00:35:32.135629 [ 1228 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 00:35:32.175958 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.61 MiB. 2024.10.22 00:35:32.374453 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_211_42 to 202410_216_216_0 into Compact with storage Full 2024.10.22 00:35:32.401042 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:35:32.414071 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_211_42, total 9650 rows starting from the beginning of the part 2024.10.22 00:35:32.432049 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_212_212_0, total 49 rows starting from the beginning of the part 2024.10.22 00:35:32.450888 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_213_213_0, total 62 rows starting from the beginning of the part 2024.10.22 00:35:32.576427 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_214_214_0, total 61 rows starting from the beginning of the part 2024.10.22 00:35:32.591869 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_215_215_0, total 116 rows starting from the beginning of the part 2024.10.22 00:35:32.828740 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_216_216_0, total 107 rows starting from the beginning of the part 2024.10.22 00:35:33.034206 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1922 entries to flush up to offset 199838 2024.10.22 00:35:33.306483 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.83 GiB, free memory in arenas 4.38 MiB, will set to 1.65 GiB (RSS), difference: 2.61 MiB 2024.10.22 00:35:33.354239 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_216_43/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:35:34.036042 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:34.043543 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.27 GiB. 2024.10.22 00:35:34.204364 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uzj/aseogqargoydernmxrqhfmopslhdm. 2024.10.22 00:35:34.400748 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_113_113_0 (state Temporary) 2024.10.22 00:35:34.604028 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_113_113_0 to 202410_120_120_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:34.636055 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 199838 2024.10.22 00:35:34.770074 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} MergeTask::MergeProjectionsStage: Merge sorted 10045 rows, containing 13 columns (13 merged, 0 gathered) in 2.529714863 sec., 3970.8032501685148 rows/sec., 605.30 KiB/sec. 2024.10.22 00:35:34.952110 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} MergedBlockOutputStream: filled checksums 202410_1_216_43 (state Temporary) 2024.10.22 00:35:35.133044 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_66_13, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0, 202410_71_71_0] 2024.10.22 00:35:35.133833 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_216_43 to 202410_1_216_43 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:35.136784 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_66_13, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0, 202410_71_71_0] 2024.10.22 00:35:35.235917 [ 1251 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:35:35.239897 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_216_43} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_211_42, 202410_216_216_0] -> 202410_1_216_43 2024.10.22 00:35:35.286474 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_66_13, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0, 202410_71_71_0] 2024.10.22 00:35:35.383585 [ 1251 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:35:35.387381 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.69 MiB. 2024.10.22 00:35:35.498981 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 1670 2024.10.22 00:35:35.789468 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.83 GiB, free memory in arenas 4.07 MiB, will set to 1.66 GiB (RSS), difference: 9.53 MiB 2024.10.22 00:35:35.826592 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:35.830302 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.27 GiB. 2024.10.22 00:35:35.972113 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_118_118_0 (state Temporary) 2024.10.22 00:35:36.060831 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 9261 2024.10.22 00:35:36.183624 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:36.185771 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.27 GiB. 2024.10.22 00:35:36.225254 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_118_118_0 to 202410_131_131_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:36.245352 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_204_204_0 (state Temporary) 2024.10.22 00:35:36.366231 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 112 2024.10.22 00:35:36.379371 [ 1255 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_126_25 to 202410_131_131_0 2024.10.22 00:35:36.379890 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_204_204_0 to 202410_217_217_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:36.381630 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1670 2024.10.22 00:35:36.384948 [ 1255 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:36.386688 [ 1255 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.27 GiB. 2024.10.22 00:35:36.397574 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_126_25 to 202410_131_131_0 into Compact with storage Full 2024.10.22 00:35:36.422915 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9261 2024.10.22 00:35:36.488532 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:36.491096 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.27 GiB. 2024.10.22 00:35:36.677792 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:35:36.678641 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_94_94_0 (state Temporary) 2024.10.22 00:35:36.686654 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_126_25, total 1699 rows starting from the beginning of the part 2024.10.22 00:35:36.702698 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 13 file segments (while reserving 6193400 bytes for cbd3c930dd72e3d362befb72c68430d5:0) 2024.10.22 00:35:36.706523 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bc298583c03686cfe176bd284bcc697a, offset: 0, size: 705989 2024.10.22 00:35:36.712802 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bc298583c03686cfe176bd284bcc697a for removal 2024.10.22 00:35:36.717170 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_127_127_0, total 12 rows starting from the beginning of the part 2024.10.22 00:35:36.717703 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c208856296cfadf11568b415d2825750, offset: 0, size: 250 2024.10.22 00:35:36.717235 [ 1476 ] {} CacheMetadata: Key bc298583c03686cfe176bd284bcc697a is removed from metadata 2024.10.22 00:35:36.721324 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c208856296cfadf11568b415d2825750 for removal 2024.10.22 00:35:36.725121 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8057e810074378e30efb8c8bb16ffc77, offset: 0, size: 109739 2024.10.22 00:35:36.728103 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8057e810074378e30efb8c8bb16ffc77 for removal 2024.10.22 00:35:36.731855 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 474b2297db88e06f9d00d8982975a1de, offset: 0, size: 4 2024.10.22 00:35:36.737397 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 474b2297db88e06f9d00d8982975a1de for removal 2024.10.22 00:35:36.741777 [ 1476 ] {} CacheMetadata: Key 8057e810074378e30efb8c8bb16ffc77 is removed from metadata 2024.10.22 00:35:36.751808 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 197ade428e896503e81217766dca9171, offset: 0, size: 5082 2024.10.22 00:35:36.753145 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_128_128_0, total 13 rows starting from the beginning of the part 2024.10.22 00:35:36.759741 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 197ade428e896503e81217766dca9171 for removal 2024.10.22 00:35:36.764019 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 86381566cd279b8ee8cbc5ba37bb65ec, offset: 0, size: 164 2024.10.22 00:35:36.764360 [ 1476 ] {} CacheMetadata: Key 197ade428e896503e81217766dca9171 is removed from metadata 2024.10.22 00:35:36.768505 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 86381566cd279b8ee8cbc5ba37bb65ec for removal 2024.10.22 00:35:36.772519 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 95d2c0a5e7992602f47204c919ae3c47, offset: 0, size: 2181441 2024.10.22 00:35:36.775982 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 95d2c0a5e7992602f47204c919ae3c47 for removal 2024.10.22 00:35:36.778229 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 527fc5e97ad96946e6d19f5232311606, offset: 0, size: 269 2024.10.22 00:35:36.780121 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 527fc5e97ad96946e6d19f5232311606 for removal 2024.10.22 00:35:36.781574 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 867513332c819a1c362b6abf2df823f0, offset: 0, size: 930285 2024.10.22 00:35:36.789544 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 867513332c819a1c362b6abf2df823f0 for removal 2024.10.22 00:35:36.792077 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a2a9197b5675851190ddead28ea984ea, offset: 0, size: 166 2024.10.22 00:35:36.874556 [ 1476 ] {} CacheMetadata: Key 867513332c819a1c362b6abf2df823f0 is removed from metadata 2024.10.22 00:35:36.880544 [ 1476 ] {} CacheMetadata: Key 527fc5e97ad96946e6d19f5232311606 is removed from metadata 2024.10.22 00:35:36.884140 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_129_129_0, total 21 rows starting from the beginning of the part 2024.10.22 00:35:36.894849 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a2a9197b5675851190ddead28ea984ea for removal 2024.10.22 00:35:36.898438 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_94_94_0 to 202410_104_104_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:36.898671 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b31c2d9648ad006716221f13e8dc743c, offset: 0, size: 137258 2024.10.22 00:35:36.899873 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_130_130_0, total 13 rows starting from the beginning of the part 2024.10.22 00:35:36.900301 [ 1476 ] {} CacheMetadata: Key a2a9197b5675851190ddead28ea984ea is removed from metadata 2024.10.22 00:35:36.903126 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b31c2d9648ad006716221f13e8dc743c for removal 2024.10.22 00:35:36.909551 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4e5ca785ab43f381499180d2deef6cfd, offset: 0, size: 192 2024.10.22 00:35:36.915435 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4e5ca785ab43f381499180d2deef6cfd for removal 2024.10.22 00:35:36.917326 [ 1476 ] {} CacheMetadata: Key b31c2d9648ad006716221f13e8dc743c is removed from metadata 2024.10.22 00:35:36.920025 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8bf8a8a15cdbb3edebbbe6f154026058, offset: 0, size: 245 2024.10.22 00:35:36.923802 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8bf8a8a15cdbb3edebbbe6f154026058 for removal 2024.10.22 00:35:36.926083 [ 1476 ] {} CacheMetadata: Key 4e5ca785ab43f381499180d2deef6cfd is removed from metadata 2024.10.22 00:35:36.939936 [ 1476 ] {} CacheMetadata: Key 8bf8a8a15cdbb3edebbbe6f154026058 is removed from metadata 2024.10.22 00:35:36.954859 [ 1476 ] {} CacheMetadata: Key 95d2c0a5e7992602f47204c919ae3c47 is removed from metadata 2024.10.22 00:35:36.959758 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_131_131_0, total 18 rows starting from the beginning of the part 2024.10.22 00:35:37.068897 [ 1476 ] {} CacheMetadata: Key 86381566cd279b8ee8cbc5ba37bb65ec is removed from metadata 2024.10.22 00:35:37.083415 [ 1476 ] {} CacheMetadata: Key 474b2297db88e06f9d00d8982975a1de is removed from metadata 2024.10.22 00:35:37.091186 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 112 2024.10.22 00:35:37.097109 [ 1476 ] {} CacheMetadata: Key c208856296cfadf11568b415d2825750 is removed from metadata 2024.10.22 00:35:37.256081 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:35:37.275673 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:35:37.285697 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:35:37.296815 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:35:37.314409 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:35:37.420880 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:35:37.439510 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:35:37.503700 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lax/fvyfahspymghmuwtwwqniejiqmxsx (LogSeriesLimiter: on interval from 2024-10-22 00:35:27 to 2024-10-22 00:35:37 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:35:37.671854 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.83 GiB, free memory in arenas 4.09 MiB, will set to 1.66 GiB (RSS), difference: -2.16 MiB 2024.10.22 00:35:37.935302 [ 1748 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lfq/racxrqmhsbfxlzedeshjmogapfweh, size 22970616 2024.10.22 00:35:38.089731 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} MergeTask::MergeProjectionsStage: Merge sorted 1776 rows, containing 12 columns (12 merged, 0 gathered) in 1.695459509 sec., 1047.5036357827876 rows/sec., 161.84 KiB/sec. 2024.10.22 00:35:38.112933 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} MergedBlockOutputStream: filled checksums 202410_1_131_26 (state Temporary) 2024.10.22 00:35:38.421355 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_131_26 to 202410_1_131_26 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:38.427022 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_131_26} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_126_25, 202410_131_131_0] -> 202410_1_131_26 2024.10.22 00:35:38.472766 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 62 entries to flush up to offset 7936 2024.10.22 00:35:38.477229 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:35:38.567773 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:38.570755 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:38.736510 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_117_117_0 (state Temporary) 2024.10.22 00:35:38.983148 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_117_117_0 to 202410_123_123_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:39.028940 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7936 2024.10.22 00:35:39.434818 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.83 GiB, free memory in arenas 3.67 MiB, will set to 1.64 GiB (RSS), difference: 17.78 MiB 2024.10.22 00:35:39.484699 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:35:39.485586 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:35:40.429042 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 9350 2024.10.22 00:35:40.782917 [ 1275 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 00:35:40.803952 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:40.809258 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:40.966903 [ 1274 ] {} DatabaseCatalog: Execution took 421 ms. 2024.10.22 00:35:41.096432 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:41.100346 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:41.178331 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_205_205_0 (state Temporary) 2024.10.22 00:35:41.250866 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_205_205_0 to 202410_218_218_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:41.299927 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9350 2024.10.22 00:35:41.643970 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 201758 2024.10.22 00:35:42.503984 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 3.63 MiB, will set to 1.64 GiB (RSS), difference: -5.03 MiB 2024.10.22 00:35:42.864789 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:42.868319 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:43.190674 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_114_114_0 (state Temporary) 2024.10.22 00:35:43.620127 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_114_114_0 to 202410_121_121_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:43.655531 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 201758 2024.10.22 00:35:43.674952 [ 1166 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_116_23 to 202410_121_121_0 2024.10.22 00:35:43.685574 [ 1166 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:43.688486 [ 1166 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:43.731000 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_116_23 to 202410_121_121_0 into Compact with storage Full 2024.10.22 00:35:43.766520 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:35:43.772108 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} MergeTreeSequentialSource: Reading 27 marks from part 202410_1_116_23, total 206130 rows starting from the beginning of the part 2024.10.22 00:35:43.779114 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_117_117_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:35:43.787675 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_118_118_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:35:43.796051 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_119_119_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:35:43.818650 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_120_120_0, total 1922 rows starting from the beginning of the part 2024.10.22 00:35:43.838781 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_121_121_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:35:43.878730 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_111_22, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0] 2024.10.22 00:35:43.880887 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_111_22, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0] 2024.10.22 00:35:43.883348 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 1683 2024.10.22 00:35:43.911530 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_111_22, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0] 2024.10.22 00:35:44.097801 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:44.101741 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:44.161520 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_119_119_0 (state Temporary) 2024.10.22 00:35:44.214128 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_119_119_0 to 202410_132_132_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:44.269751 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1683 2024.10.22 00:35:44.450441 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_24_24_0 (state Temporary) 2024.10.22 00:35:44.558655 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.83 GiB, free memory in arenas 4.80 MiB, will set to 1.64 GiB (RSS), difference: -8.93 MiB 2024.10.22 00:35:44.594302 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 113 2024.10.22 00:35:44.988660 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:44.991326 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:45.214579 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_95_95_0 (state Temporary) 2024.10.22 00:35:45.320979 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 9393 2024.10.22 00:35:45.569757 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:45.575097 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:45.614934 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_95_95_0 to 202410_105_105_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:45.653728 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:35:45.755471 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_206_206_0 (state Temporary) 2024.10.22 00:35:45.809446 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 113 2024.10.22 00:35:45.817386 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_206_206_0 to 202410_219_219_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:45.927324 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9393 2024.10.22 00:35:46.535923 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 44 entries to flush up to offset 7980 2024.10.22 00:35:46.680201 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:46.683385 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:46.720275 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_118_118_0 (state Temporary) 2024.10.22 00:35:46.763445 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_118_118_0 to 202410_124_124_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:46.997177 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 7980 2024.10.22 00:35:48.563075 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_121_24/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:35:48.920658 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 10.94 MiB, will set to 1.65 GiB (RSS), difference: 7.11 MiB 2024.10.22 00:35:49.487898 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:35:49.493240 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:35:49.972976 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 19 entries to flush up to offset 9412 2024.10.22 00:35:50.083713 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_60_12, 202410_61_61_0, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0] 2024.10.22 00:35:50.085517 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_60_12, 202410_61_61_0, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0] 2024.10.22 00:35:50.258457 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_60_12, 202410_61_61_0, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0] 2024.10.22 00:35:50.332381 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:50.338658 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:50.388008 [ 1175 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:35:50.518804 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_207_207_0 (state Temporary) 2024.10.22 00:35:50.577529 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_207_207_0 to 202410_220_220_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:50.651527 [ 1175 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:35:50.662578 [ 1175 ] {} DNSCacheUpdater: Execution took 273 ms. 2024.10.22 00:35:50.667930 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 203294 2024.10.22 00:35:50.676605 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9412 2024.10.22 00:35:50.876203 [ 1205 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 250 ms. 2024.10.22 00:35:51.307130 [ 1181 ] {} DatabaseCatalog: Execution took 328 ms. 2024.10.22 00:35:51.772764 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 1688 2024.10.22 00:35:51.794174 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:51.796113 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:52.082902 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_115_115_0 (state Temporary) 2024.10.22 00:35:52.095983 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:52.098810 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:52.166573 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_120_120_0 (state Temporary) 2024.10.22 00:35:52.184544 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_24_24_0 to 202410_27_27_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:52.219123 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_120_120_0 to 202410_133_133_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:52.273620 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1688 2024.10.22 00:35:52.301777 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_115_115_0 to 202410_122_122_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:52.449473 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 203294 2024.10.22 00:35:52.758442 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 10.65 MiB, will set to 1.65 GiB (RSS), difference: 4.05 MiB 2024.10.22 00:35:53.197400 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 466500 bytes for aa398d336148aad0340593613a964984:0) 2024.10.22 00:35:53.203321 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a103283e385d2f0f31f1bcdce975fe1b, offset: 0, size: 412443 2024.10.22 00:35:53.208293 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a103283e385d2f0f31f1bcdce975fe1b for removal 2024.10.22 00:35:53.213656 [ 1476 ] {} CacheMetadata: Key a103283e385d2f0f31f1bcdce975fe1b is removed from metadata 2024.10.22 00:35:53.214962 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9e6fc878b04c3927e96e82c71d02e572, offset: 0, size: 241 2024.10.22 00:35:53.223054 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9e6fc878b04c3927e96e82c71d02e572 for removal 2024.10.22 00:35:53.249568 [ 1476 ] {} CacheMetadata: Key 9e6fc878b04c3927e96e82c71d02e572 is removed from metadata 2024.10.22 00:35:53.297613 [ 1731 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lax/fvyfahspymghmuwtwwqniejiqmxsx, size 466500 2024.10.22 00:35:53.313455 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 114 2024.10.22 00:35:53.359683 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Title 2024.10.22 00:35:53.391478 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Title 2024.10.22 00:35:53.402260 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Title 2024.10.22 00:35:53.426145 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Title 2024.10.22 00:35:53.556139 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Title 2024.10.22 00:35:53.559580 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} MergeTask::MergeProjectionsStage: Merge sorted 215348 rows, containing 4 columns (4 merged, 0 gathered) in 9.836813366 sec., 21892.04897841507 rows/sec., 321.60 KiB/sec. 2024.10.22 00:35:53.585309 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Title 2024.10.22 00:35:53.632452 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Title 2024.10.22 00:35:53.720787 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} MergedBlockOutputStream: filled checksums 202410_1_121_24 (state Temporary) 2024.10.22 00:35:53.797568 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gat/lfrxnnjxbadvnbtyugoavuyywbgcb (LogSeriesLimiter: on interval from 2024-10-22 00:35:37 to 2024-10-22 00:35:53 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:35:53.897129 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:53.899418 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:53.996139 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_121_24 to 202410_1_121_24 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:54.013289 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_121_24} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_116_23, 202410_121_121_0] -> 202410_1_121_24 2024.10.22 00:35:54.167972 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:35:54.280261 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_96_96_0 (state Temporary) 2024.10.22 00:35:54.367612 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_96_96_0 to 202410_106_106_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:54.396869 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.83 GiB, free memory in arenas 10.26 MiB, will set to 1.65 GiB (RSS), difference: -10.05 MiB 2024.10.22 00:35:54.481115 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 114 2024.10.22 00:35:54.504053 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 32 entries to flush up to offset 8012 2024.10.22 00:35:54.601046 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:54.606987 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:54.660254 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_119_119_0 (state Temporary) 2024.10.22 00:35:54.679893 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 9472 2024.10.22 00:35:54.695709 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_119_119_0 to 202410_125_125_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:54.739778 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_120_24 to 202410_125_125_0 2024.10.22 00:35:54.745689 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8012 2024.10.22 00:35:54.767618 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:54.772091 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:54.785149 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_120_24 to 202410_125_125_0 into Compact with storage Full 2024.10.22 00:35:54.908293 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:35:54.926803 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_120_24, total 7802 rows starting from the beginning of the part 2024.10.22 00:35:54.951015 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:54.954463 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:55.133162 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_insert_202410_208_208_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:35:55.184813 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_121_121_0, total 85 rows starting from the beginning of the part 2024.10.22 00:35:55.228358 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_122_122_0, total 91 rows starting from the beginning of the part 2024.10.22 00:35:55.266473 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_208_208_0 (state Temporary) 2024.10.22 00:35:55.315005 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_123_123_0, total 62 rows starting from the beginning of the part 2024.10.22 00:35:55.362631 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_124_124_0, total 44 rows starting from the beginning of the part 2024.10.22 00:35:55.415037 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_125_125_0, total 32 rows starting from the beginning of the part 2024.10.22 00:35:55.432473 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_208_208_0 to 202410_221_221_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:55.491204 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9472 2024.10.22 00:35:55.577421 [ 1228 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_216_43 to 202410_221_221_0 2024.10.22 00:35:55.586640 [ 1228 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:55.593494 [ 1228 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:55.627134 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_216_43 to 202410_221_221_0 into Compact with storage Full 2024.10.22 00:35:55.712939 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:35:55.733487 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_216_43, total 10045 rows starting from the beginning of the part 2024.10.22 00:35:55.901646 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_217_217_0, total 47 rows starting from the beginning of the part 2024.10.22 00:35:56.018298 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_218_218_0, total 89 rows starting from the beginning of the part 2024.10.22 00:35:56.206537 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_219_219_0, total 43 rows starting from the beginning of the part 2024.10.22 00:35:56.288983 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_220_220_0, total 19 rows starting from the beginning of the part 2024.10.22 00:35:56.379193 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_221_221_0, total 60 rows starting from the beginning of the part 2024.10.22 00:35:56.385813 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 943 2024.10.22 00:35:56.482917 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 1048576 bytes for 854f10257937f71db2f581a95fb90b19:0) 2024.10.22 00:35:56.483950 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f3f72f19f94b58a91f45c479f9f96b4d, offset: 0, size: 286896 2024.10.22 00:35:56.485830 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f3f72f19f94b58a91f45c479f9f96b4d for removal 2024.10.22 00:35:56.487969 [ 1476 ] {} CacheMetadata: Key f3f72f19f94b58a91f45c479f9f96b4d is removed from metadata 2024.10.22 00:35:56.488769 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5250b2acb493ce055b2a1d466192e0d0, offset: 0, size: 291707 2024.10.22 00:35:56.494617 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5250b2acb493ce055b2a1d466192e0d0 for removal 2024.10.22 00:35:56.497280 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5913562384aab29f7e3b41a28906a86f, offset: 0, size: 202 2024.10.22 00:35:56.499734 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5913562384aab29f7e3b41a28906a86f for removal 2024.10.22 00:35:56.500146 [ 1476 ] {} CacheMetadata: Key 5250b2acb493ce055b2a1d466192e0d0 is removed from metadata 2024.10.22 00:35:56.503563 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 536feba6776efdd2ced66aa38ee1688b, offset: 0, size: 202 2024.10.22 00:35:56.508134 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 536feba6776efdd2ced66aa38ee1688b for removal 2024.10.22 00:35:56.514574 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3c6d5741152a6aaa43bc8b77774aeb42, offset: 0, size: 169 2024.10.22 00:35:56.515488 [ 1476 ] {} CacheMetadata: Key 536feba6776efdd2ced66aa38ee1688b is removed from metadata 2024.10.22 00:35:56.518545 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3c6d5741152a6aaa43bc8b77774aeb42 for removal 2024.10.22 00:35:56.523869 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cec5c06f97f9559aa5ea5e47278c0497, offset: 0, size: 167623 2024.10.22 00:35:56.524407 [ 1476 ] {} CacheMetadata: Key 3c6d5741152a6aaa43bc8b77774aeb42 is removed from metadata 2024.10.22 00:35:56.526301 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cec5c06f97f9559aa5ea5e47278c0497 for removal 2024.10.22 00:35:56.529094 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 769c549c31e294103310ceb5cb94a715, offset: 0, size: 195732 2024.10.22 00:35:56.534568 [ 1476 ] {} CacheMetadata: Key cec5c06f97f9559aa5ea5e47278c0497 is removed from metadata 2024.10.22 00:35:56.543845 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 769c549c31e294103310ceb5cb94a715 for removal 2024.10.22 00:35:56.545737 [ 1476 ] {} CacheMetadata: Key 5913562384aab29f7e3b41a28906a86f is removed from metadata 2024.10.22 00:35:56.554438 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ef055850b586b65bf7fb82078d2287ee, offset: 0, size: 170 2024.10.22 00:35:56.562665 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ef055850b586b65bf7fb82078d2287ee for removal 2024.10.22 00:35:56.568742 [ 1476 ] {} CacheMetadata: Key ef055850b586b65bf7fb82078d2287ee is removed from metadata 2024.10.22 00:35:56.579661 [ 1476 ] {} CacheMetadata: Key 769c549c31e294103310ceb5cb94a715 is removed from metadata 2024.10.22 00:35:56.617058 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} MergeTask::MergeProjectionsStage: Merge sorted 8116 rows, containing 13 columns (13 merged, 0 gathered) in 1.83850273 sec., 4414.461761500893 rows/sec., 1.85 MiB/sec. 2024.10.22 00:35:56.684831 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.83 GiB, free memory in arenas 11.13 MiB, will set to 1.67 GiB (RSS), difference: -21.34 MiB 2024.10.22 00:35:56.797101 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} MergedBlockOutputStream: filled checksums 202410_1_125_25 (state Temporary) 2024.10.22 00:35:56.904056 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_125_25 to 202410_1_125_25 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:56.910835 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_125_25} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_120_24, 202410_125_125_0] -> 202410_1_125_25 2024.10.22 00:35:56.956259 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.79 MiB. 2024.10.22 00:35:57.493170 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 18 file segments (while reserving 1048576 bytes for 854f10257937f71db2f581a95fb90b19:0) 2024.10.22 00:35:57.495933 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d5387959221d6ac9183cff0758f7c46f, offset: 0, size: 495594 2024.10.22 00:35:57.497606 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d5387959221d6ac9183cff0758f7c46f for removal 2024.10.22 00:35:57.499574 [ 1476 ] {} CacheMetadata: Key d5387959221d6ac9183cff0758f7c46f is removed from metadata 2024.10.22 00:35:57.499962 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c22581936e553f3852469665d520bd09, offset: 0, size: 246 2024.10.22 00:35:57.502809 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c22581936e553f3852469665d520bd09 for removal 2024.10.22 00:35:57.516651 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ade25180bb140e5050ffcab99738ecf1, offset: 0, size: 58250 2024.10.22 00:35:57.518651 [ 1476 ] {} CacheMetadata: Key c22581936e553f3852469665d520bd09 is removed from metadata 2024.10.22 00:35:57.521611 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ade25180bb140e5050ffcab99738ecf1 for removal 2024.10.22 00:35:57.525562 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d69b9f5cdf390a2f962288d00d28cc6a, offset: 0, size: 162 2024.10.22 00:35:57.526422 [ 1476 ] {} CacheMetadata: Key ade25180bb140e5050ffcab99738ecf1 is removed from metadata 2024.10.22 00:35:57.529126 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d69b9f5cdf390a2f962288d00d28cc6a for removal 2024.10.22 00:35:57.532860 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6e593cb9728fc868a7c8f82a861d4a1f, offset: 0, size: 164 2024.10.22 00:35:57.533364 [ 1476 ] {} CacheMetadata: Key d69b9f5cdf390a2f962288d00d28cc6a is removed from metadata 2024.10.22 00:35:57.537220 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6e593cb9728fc868a7c8f82a861d4a1f for removal 2024.10.22 00:35:57.540832 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 29a4d75480f3500e64badd5fe28eeba8, offset: 0, size: 6175 2024.10.22 00:35:57.542106 [ 1476 ] {} CacheMetadata: Key 6e593cb9728fc868a7c8f82a861d4a1f is removed from metadata 2024.10.22 00:35:57.545211 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 29a4d75480f3500e64badd5fe28eeba8 for removal 2024.10.22 00:35:57.552299 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4b6228261a9a39ceeb308509dd7bfddb, offset: 0, size: 162 2024.10.22 00:35:57.557811 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4b6228261a9a39ceeb308509dd7bfddb for removal 2024.10.22 00:35:57.565812 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ea77c7a658112723aec4a49d9a4c201b, offset: 0, size: 3988 2024.10.22 00:35:57.573605 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ea77c7a658112723aec4a49d9a4c201b for removal 2024.10.22 00:35:57.577517 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 831982b2126bd7b8eb1ee3dfdbe71dcb, offset: 0, size: 168 2024.10.22 00:35:57.580745 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 831982b2126bd7b8eb1ee3dfdbe71dcb for removal 2024.10.22 00:35:57.582700 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 95f27a959cdf1654fcfc97cdf303282a, offset: 0, size: 162 2024.10.22 00:35:57.588745 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 95f27a959cdf1654fcfc97cdf303282a for removal 2024.10.22 00:35:57.590756 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 33420e53f83bdd65ee004f10e781a9a6, offset: 0, size: 294284 2024.10.22 00:35:57.593603 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 33420e53f83bdd65ee004f10e781a9a6 for removal 2024.10.22 00:35:57.595879 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ee938492e5b92940c6acfdb6af5f311c, offset: 0, size: 83184 2024.10.22 00:35:57.599784 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ee938492e5b92940c6acfdb6af5f311c for removal 2024.10.22 00:35:57.608690 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 68468aeeedafab232119c8f95d7d7732, offset: 0, size: 53956 2024.10.22 00:35:57.620347 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 68468aeeedafab232119c8f95d7d7732 for removal 2024.10.22 00:35:57.623327 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 641d1a47746fa0f1abae974b6b262780, offset: 0, size: 142502 2024.10.22 00:35:57.626389 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 641d1a47746fa0f1abae974b6b262780 for removal 2024.10.22 00:35:57.628879 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 13db04201b60c46bea7add72d8258b3f, offset: 0, size: 200 2024.10.22 00:35:57.634147 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 13db04201b60c46bea7add72d8258b3f for removal 2024.10.22 00:35:57.639964 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3b432d1d577c15a3c2e06d083349a343, offset: 0, size: 130648 2024.10.22 00:35:57.648174 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3b432d1d577c15a3c2e06d083349a343 for removal 2024.10.22 00:35:57.655290 [ 1476 ] {} CacheMetadata: Key 29a4d75480f3500e64badd5fe28eeba8 is removed from metadata 2024.10.22 00:35:57.655335 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 86c115ec2b617732276a08c6b0aab0e4, offset: 0, size: 169 2024.10.22 00:35:57.664226 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 86c115ec2b617732276a08c6b0aab0e4 for removal 2024.10.22 00:35:57.665090 [ 1476 ] {} CacheMetadata: Key 3b432d1d577c15a3c2e06d083349a343 is removed from metadata 2024.10.22 00:35:57.671089 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e9bc6da28f0ff3aeeed451f00c3e3fb8, offset: 0, size: 170 2024.10.22 00:35:57.673823 [ 1476 ] {} CacheMetadata: Key 86c115ec2b617732276a08c6b0aab0e4 is removed from metadata 2024.10.22 00:35:57.683346 [ 1476 ] {} CacheMetadata: Key 13db04201b60c46bea7add72d8258b3f is removed from metadata 2024.10.22 00:35:57.777143 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e9bc6da28f0ff3aeeed451f00c3e3fb8 for removal 2024.10.22 00:35:57.794710 [ 1476 ] {} CacheMetadata: Key e9bc6da28f0ff3aeeed451f00c3e3fb8 is removed from metadata 2024.10.22 00:35:57.803794 [ 1476 ] {} CacheMetadata: Key 641d1a47746fa0f1abae974b6b262780 is removed from metadata 2024.10.22 00:35:57.828720 [ 1476 ] {} CacheMetadata: Key 68468aeeedafab232119c8f95d7d7732 is removed from metadata 2024.10.22 00:35:57.846624 [ 1476 ] {} CacheMetadata: Key ee938492e5b92940c6acfdb6af5f311c is removed from metadata 2024.10.22 00:35:57.862883 [ 1476 ] {} CacheMetadata: Key 33420e53f83bdd65ee004f10e781a9a6 is removed from metadata 2024.10.22 00:35:57.874175 [ 1476 ] {} CacheMetadata: Key 95f27a959cdf1654fcfc97cdf303282a is removed from metadata 2024.10.22 00:35:57.884278 [ 1476 ] {} CacheMetadata: Key 831982b2126bd7b8eb1ee3dfdbe71dcb is removed from metadata 2024.10.22 00:35:57.995258 [ 1476 ] {} CacheMetadata: Key ea77c7a658112723aec4a49d9a4c201b is removed from metadata 2024.10.22 00:35:58.102578 [ 1476 ] {} CacheMetadata: Key 4b6228261a9a39ceeb308509dd7bfddb is removed from metadata 2024.10.22 00:35:58.376806 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} MergeTask::MergeProjectionsStage: Merge sorted 10303 rows, containing 13 columns (13 merged, 0 gathered) in 2.759564044 sec., 3733.560749351465 rows/sec., 568.98 KiB/sec. 2024.10.22 00:35:58.441041 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.83 GiB, free memory in arenas 10.89 MiB, will set to 1.67 GiB (RSS), difference: 9.29 MiB 2024.10.22 00:35:58.768579 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} MergedBlockOutputStream: filled checksums 202410_1_221_44 (state Temporary) 2024.10.22 00:35:58.967699 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_221_44 to 202410_1_221_44 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:35:58.996520 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_221_44} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_216_43, 202410_221_221_0] -> 202410_1_221_44 2024.10.22 00:35:59.059504 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.66 MiB. 2024.10.22 00:35:59.282424 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 12 file segments (while reserving 2097152 bytes for 854f10257937f71db2f581a95fb90b19:0) 2024.10.22 00:35:59.283447 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fb3909f94a3f1236f2191401e23a0c90, offset: 0, size: 2206018 2024.10.22 00:35:59.284861 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fb3909f94a3f1236f2191401e23a0c90 for removal 2024.10.22 00:35:59.286074 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8fd5b109ec8917eea575392fffc7e16a, offset: 0, size: 202 2024.10.22 00:35:59.287297 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8fd5b109ec8917eea575392fffc7e16a for removal 2024.10.22 00:35:59.290247 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 203a0e2ed5620c72429921fbd5dfbca9, offset: 0, size: 231944 2024.10.22 00:35:59.293262 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 203a0e2ed5620c72429921fbd5dfbca9 for removal 2024.10.22 00:35:59.298574 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 72980d755ce48df525f45c96ea76b715, offset: 0, size: 422090 2024.10.22 00:35:59.301612 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 72980d755ce48df525f45c96ea76b715 for removal 2024.10.22 00:35:59.305142 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ee20797c6839ab2c23455d1a30b95fce, offset: 0, size: 202 2024.10.22 00:35:59.309542 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ee20797c6839ab2c23455d1a30b95fce for removal 2024.10.22 00:35:59.317112 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2cbc198acee25b4248a29216541bfbaf, offset: 0, size: 392631 2024.10.22 00:35:59.386482 [ 1476 ] {} CacheMetadata: Key fb3909f94a3f1236f2191401e23a0c90 is removed from metadata 2024.10.22 00:35:59.423580 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2cbc198acee25b4248a29216541bfbaf for removal 2024.10.22 00:35:59.433222 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5aa68b79df5d7cf0d1117df77ce89a53, offset: 0, size: 199 2024.10.22 00:35:59.436219 [ 1476 ] {} CacheMetadata: Key 2cbc198acee25b4248a29216541bfbaf is removed from metadata 2024.10.22 00:35:59.438909 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5aa68b79df5d7cf0d1117df77ce89a53 for removal 2024.10.22 00:35:59.448493 [ 1476 ] {} CacheMetadata: Key 5aa68b79df5d7cf0d1117df77ce89a53 is removed from metadata 2024.10.22 00:35:59.455066 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7d38b727e6bab1f848e42257d006dd53, offset: 0, size: 170 2024.10.22 00:35:59.456215 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 205214 2024.10.22 00:35:59.465834 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7d38b727e6bab1f848e42257d006dd53 for removal 2024.10.22 00:35:59.467523 [ 1476 ] {} CacheMetadata: Key ee20797c6839ab2c23455d1a30b95fce is removed from metadata 2024.10.22 00:35:59.471517 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c1d8237875d611aff295945c2669f2c5, offset: 0, size: 170 2024.10.22 00:35:59.475961 [ 1476 ] {} CacheMetadata: Key 7d38b727e6bab1f848e42257d006dd53 is removed from metadata 2024.10.22 00:35:59.475791 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c1d8237875d611aff295945c2669f2c5 for removal 2024.10.22 00:35:59.478157 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4ff7408e544aa6d3f6fcb614af512c02, offset: 0, size: 269 2024.10.22 00:35:59.480297 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4ff7408e544aa6d3f6fcb614af512c02 for removal 2024.10.22 00:35:59.480993 [ 1476 ] {} CacheMetadata: Key c1d8237875d611aff295945c2669f2c5 is removed from metadata 2024.10.22 00:35:59.488353 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7065baf971eff1239486634b9ead3723, offset: 0, size: 196401 2024.10.22 00:35:59.489391 [ 1476 ] {} CacheMetadata: Key 4ff7408e544aa6d3f6fcb614af512c02 is removed from metadata 2024.10.22 00:35:59.490517 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7065baf971eff1239486634b9ead3723 for removal 2024.10.22 00:35:59.491160 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:35:59.492371 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f0a14190b49306258d4ad7fb37f8fe8a, offset: 0, size: 174433 2024.10.22 00:35:59.495129 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 156 entries to flush up to offset 9628 2024.10.22 00:35:59.496630 [ 1476 ] {} CacheMetadata: Key 7065baf971eff1239486634b9ead3723 is removed from metadata 2024.10.22 00:35:59.497763 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f0a14190b49306258d4ad7fb37f8fe8a for removal 2024.10.22 00:35:59.519291 [ 1476 ] {} CacheMetadata: Key f0a14190b49306258d4ad7fb37f8fe8a is removed from metadata 2024.10.22 00:35:59.529249 [ 1476 ] {} CacheMetadata: Key 72980d755ce48df525f45c96ea76b715 is removed from metadata 2024.10.22 00:35:59.549001 [ 1476 ] {} CacheMetadata: Key 203a0e2ed5620c72429921fbd5dfbca9 is removed from metadata 2024.10.22 00:35:59.485787 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:35:59.667482 [ 1476 ] {} CacheMetadata: Key 8fd5b109ec8917eea575392fffc7e16a is removed from metadata 2024.10.22 00:35:59.778770 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 1715 2024.10.22 00:35:59.829166 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:59.831232 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:35:59.868992 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:35:59.870613 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:00.081642 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_121_121_0 (state Temporary) 2024.10.22 00:36:00.121830 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_209_209_0 (state Temporary) 2024.10.22 00:36:00.132133 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_121_121_0 to 202410_134_134_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:00.199963 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:00.206348 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:00.339806 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_209_209_0 to 202410_222_222_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:00.469404 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_116_116_0 (state Temporary) 2024.10.22 00:36:00.491507 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9628 2024.10.22 00:36:00.553737 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.83 GiB, free memory in arenas 9.35 MiB, will set to 1.67 GiB (RSS), difference: -10.68 MiB 2024.10.22 00:36:00.572620 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1715 2024.10.22 00:36:00.575952 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_116_116_0 to 202410_123_123_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:00.599461 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 205214 2024.10.22 00:36:00.651364 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_61_12, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0, 202410_66_66_0] 2024.10.22 00:36:00.656319 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_61_12, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0, 202410_66_66_0] 2024.10.22 00:36:00.784155 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_61_12, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0, 202410_65_65_0, 202410_66_66_0] 2024.10.22 00:36:01.502034 [ 1245 ] {} DatabaseCatalog: Execution took 200 ms. 2024.10.22 00:36:01.984341 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 115 2024.10.22 00:36:02.258328 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 89 entries to flush up to offset 8101 2024.10.22 00:36:02.284291 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:02.288141 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:02.337339 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_97_97_0 (state Temporary) 2024.10.22 00:36:02.456747 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:02.458783 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:02.507922 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_120_120_0 (state Temporary) 2024.10.22 00:36:02.577364 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_120_120_0 to 202410_126_126_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:02.604453 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8101 2024.10.22 00:36:02.660447 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_97_97_0 to 202410_107_107_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:02.715834 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 115 2024.10.22 00:36:02.781804 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uzj/aseogqargoydernmxrqhfmopslhdm. 2024.10.22 00:36:02.986756 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 14 file segments (while reserving 4194304 bytes for 854f10257937f71db2f581a95fb90b19:0) 2024.10.22 00:36:02.987946 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7fa6fb86127f32a467196ff94476794f, offset: 0, size: 2085005 2024.10.22 00:36:02.992568 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7fa6fb86127f32a467196ff94476794f for removal 2024.10.22 00:36:02.994710 [ 1476 ] {} CacheMetadata: Key 7fa6fb86127f32a467196ff94476794f is removed from metadata 2024.10.22 00:36:02.995053 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: deaf5d18f38d45171fe306d39489d643, offset: 0, size: 238 2024.10.22 00:36:02.997045 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key deaf5d18f38d45171fe306d39489d643 for removal 2024.10.22 00:36:03.001519 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 62c540d61fc17e4752cfaae08d6f5aa3, offset: 0, size: 873705 2024.10.22 00:36:03.005546 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 62c540d61fc17e4752cfaae08d6f5aa3 for removal 2024.10.22 00:36:03.015125 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9d6d1f1e2c13f1eac361208b7e904640, offset: 0, size: 170 2024.10.22 00:36:03.017145 [ 1476 ] {} CacheMetadata: Key 62c540d61fc17e4752cfaae08d6f5aa3 is removed from metadata 2024.10.22 00:36:03.019129 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9d6d1f1e2c13f1eac361208b7e904640 for removal 2024.10.22 00:36:03.021016 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 522de562ddef0ff0e465c8acf45de842, offset: 0, size: 243 2024.10.22 00:36:03.022493 [ 1476 ] {} CacheMetadata: Key 9d6d1f1e2c13f1eac361208b7e904640 is removed from metadata 2024.10.22 00:36:03.025576 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 522de562ddef0ff0e465c8acf45de842 for removal 2024.10.22 00:36:03.029351 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 85d05dca9fc1e31a147d091864eeeeb2, offset: 0, size: 187848 2024.10.22 00:36:03.029784 [ 1476 ] {} CacheMetadata: Key 522de562ddef0ff0e465c8acf45de842 is removed from metadata 2024.10.22 00:36:03.033747 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 85d05dca9fc1e31a147d091864eeeeb2 for removal 2024.10.22 00:36:03.038326 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0442cd14fcfe250e13f6309c0eb3c3ba, offset: 0, size: 338788 2024.10.22 00:36:03.039062 [ 1476 ] {} CacheMetadata: Key 85d05dca9fc1e31a147d091864eeeeb2 is removed from metadata 2024.10.22 00:36:03.043715 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0442cd14fcfe250e13f6309c0eb3c3ba for removal 2024.10.22 00:36:03.047694 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5e47b43697b522fde84abe2280135109, offset: 0, size: 164 2024.10.22 00:36:03.052226 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5e47b43697b522fde84abe2280135109 for removal 2024.10.22 00:36:03.056492 [ 1476 ] {} CacheMetadata: Key 0442cd14fcfe250e13f6309c0eb3c3ba is removed from metadata 2024.10.22 00:36:03.056882 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 346cd86cf4d45469b2cb08509aa918fc, offset: 0, size: 105997 2024.10.22 00:36:03.064369 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 346cd86cf4d45469b2cb08509aa918fc for removal 2024.10.22 00:36:03.068725 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 25acf73ddf9c413f4182fb02c61a8d6b, offset: 0, size: 235 2024.10.22 00:36:03.068940 [ 1476 ] {} CacheMetadata: Key 346cd86cf4d45469b2cb08509aa918fc is removed from metadata 2024.10.22 00:36:03.071995 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 25acf73ddf9c413f4182fb02c61a8d6b for removal 2024.10.22 00:36:03.075166 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a025f500ecde10bc7c3f1563872b65ae, offset: 0, size: 123640 2024.10.22 00:36:03.076391 [ 1476 ] {} CacheMetadata: Key 25acf73ddf9c413f4182fb02c61a8d6b is removed from metadata 2024.10.22 00:36:03.082448 [ 1476 ] {} CacheMetadata: Key 5e47b43697b522fde84abe2280135109 is removed from metadata 2024.10.22 00:36:03.089592 [ 1476 ] {} CacheMetadata: Key deaf5d18f38d45171fe306d39489d643 is removed from metadata 2024.10.22 00:36:03.077805 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a025f500ecde10bc7c3f1563872b65ae for removal 2024.10.22 00:36:03.181722 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 442b0332f520669a2f5d7beb47ef9f9a, offset: 0, size: 269 2024.10.22 00:36:03.181445 [ 1476 ] {} CacheMetadata: Key a025f500ecde10bc7c3f1563872b65ae is removed from metadata 2024.10.22 00:36:03.184680 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 442b0332f520669a2f5d7beb47ef9f9a for removal 2024.10.22 00:36:03.187033 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f260d8354c7105abfe777a5527fdde16, offset: 0, size: 193205 2024.10.22 00:36:03.190114 [ 1476 ] {} CacheMetadata: Key 442b0332f520669a2f5d7beb47ef9f9a is removed from metadata 2024.10.22 00:36:03.192980 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f260d8354c7105abfe777a5527fdde16 for removal 2024.10.22 00:36:03.195305 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 77efff47569efbd2756cca6c14fccbfb, offset: 0, size: 164 2024.10.22 00:36:03.199332 [ 1476 ] {} CacheMetadata: Key f260d8354c7105abfe777a5527fdde16 is removed from metadata 2024.10.22 00:36:03.198885 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 77efff47569efbd2756cca6c14fccbfb for removal 2024.10.22 00:36:03.316762 [ 1476 ] {} CacheMetadata: Key 77efff47569efbd2756cca6c14fccbfb is removed from metadata 2024.10.22 00:36:04.021372 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 982 2024.10.22 00:36:04.455764 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.83 GiB, free memory in arenas 5.84 MiB, will set to 1.67 GiB (RSS), difference: 119.52 MiB 2024.10.22 00:36:04.544522 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 9716 2024.10.22 00:36:04.897223 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:04.907175 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:04.979471 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.20428248 sec. 2024.10.22 00:36:04.986866 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_210_210_0 (state Temporary) 2024.10.22 00:36:05.042000 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_210_210_0 to 202410_223_223_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:05.106878 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9716 2024.10.22 00:36:05.227710 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_116_23, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0] 2024.10.22 00:36:05.233216 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_116_23, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0] 2024.10.22 00:36:05.260912 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_116_23, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0] 2024.10.22 00:36:05.665313 [ 1162 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:36:05.686689 [ 1162 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:36:06.614906 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 523ed9fa8342d36b82499fde53c9f4ab:0) 2024.10.22 00:36:06.621091 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 152a93d82e3301429b40e9374caa501d, offset: 0, size: 2180932 2024.10.22 00:36:06.631027 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.83 GiB, free memory in arenas 7.08 MiB, will set to 1.66 GiB (RSS), difference: 89.28 MiB 2024.10.22 00:36:06.728245 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 152a93d82e3301429b40e9374caa501d for removal 2024.10.22 00:36:06.732235 [ 1476 ] {} CacheMetadata: Key 152a93d82e3301429b40e9374caa501d is removed from metadata 2024.10.22 00:36:06.732431 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5f06304ed7a532cadf91dd35981cf942, offset: 0, size: 269 2024.10.22 00:36:06.738237 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5f06304ed7a532cadf91dd35981cf942 for removal 2024.10.22 00:36:06.741955 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 07fe18728b58062883909d7215949af3, offset: 0, size: 201693 2024.10.22 00:36:06.762308 [ 1476 ] {} CacheMetadata: Key 5f06304ed7a532cadf91dd35981cf942 is removed from metadata 2024.10.22 00:36:06.763452 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 07fe18728b58062883909d7215949af3 for removal 2024.10.22 00:36:06.770751 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2b03aa4d7ba1119f2ffe486c493d51ae, offset: 0, size: 171 2024.10.22 00:36:06.779243 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2b03aa4d7ba1119f2ffe486c493d51ae for removal 2024.10.22 00:36:06.779364 [ 1476 ] {} CacheMetadata: Key 07fe18728b58062883909d7215949af3 is removed from metadata 2024.10.22 00:36:06.795960 [ 1476 ] {} CacheMetadata: Key 2b03aa4d7ba1119f2ffe486c493d51ae is removed from metadata 2024.10.22 00:36:07.606163 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1509 entries to flush up to offset 206723 2024.10.22 00:36:08.074605 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 1721 2024.10.22 00:36:08.172544 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:08.175625 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:08.200277 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_122_122_0 (state Temporary) 2024.10.22 00:36:08.357119 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_122_122_0 to 202410_135_135_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:08.365632 [ 1193 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 00:36:08.495137 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1721 2024.10.22 00:36:08.496621 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:08.545037 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.83 GiB, free memory in arenas 7.13 MiB, will set to 1.67 GiB (RSS), difference: 4.67 MiB 2024.10.22 00:36:08.599884 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:08.639368 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 34 file segments (while reserving 8388608 bytes for 854f10257937f71db2f581a95fb90b19:0) 2024.10.22 00:36:08.642484 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 74046a6d13a71dbedd135217795e1545, offset: 0, size: 780415 2024.10.22 00:36:08.654709 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 74046a6d13a71dbedd135217795e1545 for removal 2024.10.22 00:36:08.660218 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 42dcd7526298b38d5544b8ff42054bbd, offset: 0, size: 248 2024.10.22 00:36:08.660430 [ 1476 ] {} CacheMetadata: Key 74046a6d13a71dbedd135217795e1545 is removed from metadata 2024.10.22 00:36:08.664796 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 42dcd7526298b38d5544b8ff42054bbd for removal 2024.10.22 00:36:08.668739 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 495ffa99393a8df9513ad75f9eee0357, offset: 0, size: 492054 2024.10.22 00:36:08.669445 [ 1476 ] {} CacheMetadata: Key 42dcd7526298b38d5544b8ff42054bbd is removed from metadata 2024.10.22 00:36:08.672747 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 495ffa99393a8df9513ad75f9eee0357 for removal 2024.10.22 00:36:08.675650 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3eb0109f4bdf758e4441b2032625400e, offset: 0, size: 252 2024.10.22 00:36:08.676432 [ 1476 ] {} CacheMetadata: Key 495ffa99393a8df9513ad75f9eee0357 is removed from metadata 2024.10.22 00:36:08.677698 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3eb0109f4bdf758e4441b2032625400e for removal 2024.10.22 00:36:08.679785 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c5e6a13580d889d5a7e592d0f3c10dd4, offset: 0, size: 250 2024.10.22 00:36:08.680508 [ 1476 ] {} CacheMetadata: Key 3eb0109f4bdf758e4441b2032625400e is removed from metadata 2024.10.22 00:36:08.681832 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c5e6a13580d889d5a7e592d0f3c10dd4 for removal 2024.10.22 00:36:08.683695 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 52c611c9da018f66f9e73b41774d96b2, offset: 0, size: 981887 2024.10.22 00:36:08.684460 [ 1476 ] {} CacheMetadata: Key c5e6a13580d889d5a7e592d0f3c10dd4 is removed from metadata 2024.10.22 00:36:08.690034 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_117_117_0 (state Temporary) 2024.10.22 00:36:08.696185 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 52c611c9da018f66f9e73b41774d96b2 for removal 2024.10.22 00:36:08.699377 [ 1476 ] {} CacheMetadata: Key 52c611c9da018f66f9e73b41774d96b2 is removed from metadata 2024.10.22 00:36:08.699536 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9cb2ea6d6257d2c2a040367bfbd61237, offset: 0, size: 254 2024.10.22 00:36:08.702364 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9cb2ea6d6257d2c2a040367bfbd61237 for removal 2024.10.22 00:36:08.707806 [ 1476 ] {} CacheMetadata: Key 9cb2ea6d6257d2c2a040367bfbd61237 is removed from metadata 2024.10.22 00:36:08.730536 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_117_117_0 to 202410_124_124_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:08.770531 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 206723 2024.10.22 00:36:08.705791 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cff5d36a1da48ea9d95dc2fe0fdc5c0a, offset: 0, size: 605630 2024.10.22 00:36:08.813353 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cff5d36a1da48ea9d95dc2fe0fdc5c0a for removal 2024.10.22 00:36:08.820435 [ 1476 ] {} CacheMetadata: Key cff5d36a1da48ea9d95dc2fe0fdc5c0a is removed from metadata 2024.10.22 00:36:08.821826 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 96b2ee2ca662dadfcd51b2c6b2d210ae, offset: 0, size: 234 2024.10.22 00:36:08.827254 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 96b2ee2ca662dadfcd51b2c6b2d210ae for removal 2024.10.22 00:36:08.830663 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4c69d8322270c250fb39159c8135a89c, offset: 0, size: 249 2024.10.22 00:36:08.836098 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4c69d8322270c250fb39159c8135a89c for removal 2024.10.22 00:36:08.840666 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0e34404c4b81015b7f96ee862a01717b, offset: 0, size: 197 2024.10.22 00:36:08.848530 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0e34404c4b81015b7f96ee862a01717b for removal 2024.10.22 00:36:08.856162 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9338643e2c64b9eda6f14be4226a717e, offset: 0, size: 10092 2024.10.22 00:36:08.861005 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9338643e2c64b9eda6f14be4226a717e for removal 2024.10.22 00:36:08.867741 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1ef81bc29704b7dec3145d43bccf0147, offset: 0, size: 690636 2024.10.22 00:36:08.935317 [ 1476 ] {} CacheMetadata: Key 0e34404c4b81015b7f96ee862a01717b is removed from metadata 2024.10.22 00:36:08.951261 [ 1476 ] {} CacheMetadata: Key 9338643e2c64b9eda6f14be4226a717e is removed from metadata 2024.10.22 00:36:08.959818 [ 1476 ] {} CacheMetadata: Key 4c69d8322270c250fb39159c8135a89c is removed from metadata 2024.10.22 00:36:08.970465 [ 1476 ] {} CacheMetadata: Key 96b2ee2ca662dadfcd51b2c6b2d210ae is removed from metadata 2024.10.22 00:36:08.978370 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1ef81bc29704b7dec3145d43bccf0147 for removal 2024.10.22 00:36:08.980624 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e8234cd1b194f8a49bc63f34d4d4230a, offset: 0, size: 102605 2024.10.22 00:36:08.980613 [ 1476 ] {} CacheMetadata: Key 1ef81bc29704b7dec3145d43bccf0147 is removed from metadata 2024.10.22 00:36:08.981968 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e8234cd1b194f8a49bc63f34d4d4230a for removal 2024.10.22 00:36:08.983709 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b669b8a7a5425ef13fbddfafcc2224ea, offset: 0, size: 150297 2024.10.22 00:36:08.984614 [ 1476 ] {} CacheMetadata: Key e8234cd1b194f8a49bc63f34d4d4230a is removed from metadata 2024.10.22 00:36:08.985435 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b669b8a7a5425ef13fbddfafcc2224ea for removal 2024.10.22 00:36:08.988099 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4e2d67e319caf18cd86d4fe17f77ce08, offset: 0, size: 166 2024.10.22 00:36:08.988732 [ 1476 ] {} CacheMetadata: Key b669b8a7a5425ef13fbddfafcc2224ea is removed from metadata 2024.10.22 00:36:08.990279 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4e2d67e319caf18cd86d4fe17f77ce08 for removal 2024.10.22 00:36:08.992331 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9ca956033e2a9dd38eaf91f6ca4fd7a7, offset: 0, size: 238 2024.10.22 00:36:08.993155 [ 1476 ] {} CacheMetadata: Key 4e2d67e319caf18cd86d4fe17f77ce08 is removed from metadata 2024.10.22 00:36:08.994472 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9ca956033e2a9dd38eaf91f6ca4fd7a7 for removal 2024.10.22 00:36:08.996251 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1acc34068806ef8ba5c83a5b704e716b, offset: 0, size: 998053 2024.10.22 00:36:08.999445 [ 1476 ] {} CacheMetadata: Key 9ca956033e2a9dd38eaf91f6ca4fd7a7 is removed from metadata 2024.10.22 00:36:08.999930 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1acc34068806ef8ba5c83a5b704e716b for removal 2024.10.22 00:36:09.005740 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e0e1e7a93ee7574b2952f4d2be13f5a9, offset: 0, size: 248 2024.10.22 00:36:09.009338 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e0e1e7a93ee7574b2952f4d2be13f5a9 for removal 2024.10.22 00:36:09.009616 [ 1476 ] {} CacheMetadata: Key 1acc34068806ef8ba5c83a5b704e716b is removed from metadata 2024.10.22 00:36:09.015737 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2e196c77d7976c60c0d704cb403f37ef, offset: 0, size: 709526 2024.10.22 00:36:09.020361 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2e196c77d7976c60c0d704cb403f37ef for removal 2024.10.22 00:36:09.020939 [ 1476 ] {} CacheMetadata: Key e0e1e7a93ee7574b2952f4d2be13f5a9 is removed from metadata 2024.10.22 00:36:09.025086 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c7de7e817213da929581ab895001468f, offset: 0, size: 251 2024.10.22 00:36:09.029728 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c7de7e817213da929581ab895001468f for removal 2024.10.22 00:36:09.034658 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 73d06317a0716bc12245af27554ac1ea, offset: 0, size: 164 2024.10.22 00:36:09.038887 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 73d06317a0716bc12245af27554ac1ea for removal 2024.10.22 00:36:09.043846 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7efcf438a698fb30004b348e9e43410a, offset: 0, size: 764361 2024.10.22 00:36:09.048712 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7efcf438a698fb30004b348e9e43410a for removal 2024.10.22 00:36:09.053428 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d176441ec629ad41553ae5d11cb308bf, offset: 0, size: 681154 2024.10.22 00:36:09.064241 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d176441ec629ad41553ae5d11cb308bf for removal 2024.10.22 00:36:09.070522 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ae540a57bd83e822e12556cab390bc4a, offset: 0, size: 267818 2024.10.22 00:36:09.076914 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ae540a57bd83e822e12556cab390bc4a for removal 2024.10.22 00:36:09.083921 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a331f5ab3a9be38475a0012d77644722, offset: 0, size: 175184 2024.10.22 00:36:09.090327 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a331f5ab3a9be38475a0012d77644722 for removal 2024.10.22 00:36:09.116677 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bd91b084866ea3fa3d026e0b2bdfad48, offset: 0, size: 165 2024.10.22 00:36:09.126516 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 112 entries to flush up to offset 9828 2024.10.22 00:36:09.129765 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bd91b084866ea3fa3d026e0b2bdfad48 for removal 2024.10.22 00:36:09.131074 [ 1476 ] {} CacheMetadata: Key a331f5ab3a9be38475a0012d77644722 is removed from metadata 2024.10.22 00:36:09.134918 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e78465313d715a12f9f614adc25e0bfc, offset: 0, size: 200 2024.10.22 00:36:09.140448 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e78465313d715a12f9f614adc25e0bfc for removal 2024.10.22 00:36:09.143193 [ 1476 ] {} CacheMetadata: Key bd91b084866ea3fa3d026e0b2bdfad48 is removed from metadata 2024.10.22 00:36:09.149370 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3576a0942d9bd006be413e203ed5e539, offset: 0, size: 128605 2024.10.22 00:36:09.155644 [ 1476 ] {} CacheMetadata: Key e78465313d715a12f9f614adc25e0bfc is removed from metadata 2024.10.22 00:36:09.156075 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3576a0942d9bd006be413e203ed5e539 for removal 2024.10.22 00:36:09.165041 [ 1476 ] {} CacheMetadata: Key 3576a0942d9bd006be413e203ed5e539 is removed from metadata 2024.10.22 00:36:09.164204 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a9ed74ce08fede5dbadbc04ed875e231, offset: 0, size: 246 2024.10.22 00:36:09.168844 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a9ed74ce08fede5dbadbc04ed875e231 for removal 2024.10.22 00:36:09.172158 [ 1476 ] {} CacheMetadata: Key ae540a57bd83e822e12556cab390bc4a is removed from metadata 2024.10.22 00:36:09.173785 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 77c5f1e340a578490c1dfc596f1faf46, offset: 0, size: 250 2024.10.22 00:36:09.176770 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 77c5f1e340a578490c1dfc596f1faf46 for removal 2024.10.22 00:36:09.178613 [ 1476 ] {} CacheMetadata: Key 7efcf438a698fb30004b348e9e43410a is removed from metadata 2024.10.22 00:36:09.179603 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 21b13839623686382078a2c8ac9401c9, offset: 0, size: 209710 2024.10.22 00:36:09.184002 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 21b13839623686382078a2c8ac9401c9 for removal 2024.10.22 00:36:09.190866 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 09204e1035658e0255d9e4add13167a7, offset: 0, size: 170663 2024.10.22 00:36:09.194781 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 09204e1035658e0255d9e4add13167a7 for removal 2024.10.22 00:36:09.198966 [ 1476 ] {} CacheMetadata: Key 21b13839623686382078a2c8ac9401c9 is removed from metadata 2024.10.22 00:36:09.225574 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:36:09.299886 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 45b70817b2dbc1ac90373970c43c4539, offset: 0, size: 169 2024.10.22 00:36:09.309304 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 45b70817b2dbc1ac90373970c43c4539 for removal 2024.10.22 00:36:09.314907 [ 1476 ] {} CacheMetadata: Key 09204e1035658e0255d9e4add13167a7 is removed from metadata 2024.10.22 00:36:09.342324 [ 1476 ] {} CacheMetadata: Key 45b70817b2dbc1ac90373970c43c4539 is removed from metadata 2024.10.22 00:36:09.360485 [ 1476 ] {} CacheMetadata: Key 77c5f1e340a578490c1dfc596f1faf46 is removed from metadata 2024.10.22 00:36:09.372895 [ 1476 ] {} CacheMetadata: Key d176441ec629ad41553ae5d11cb308bf is removed from metadata 2024.10.22 00:36:09.380554 [ 1476 ] {} CacheMetadata: Key 73d06317a0716bc12245af27554ac1ea is removed from metadata 2024.10.22 00:36:09.385587 [ 1476 ] {} CacheMetadata: Key a9ed74ce08fede5dbadbc04ed875e231 is removed from metadata 2024.10.22 00:36:09.486919 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:36:09.489495 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:36:09.493553 [ 1476 ] {} CacheMetadata: Key c7de7e817213da929581ab895001468f is removed from metadata 2024.10.22 00:36:09.511132 [ 1476 ] {} CacheMetadata: Key 2e196c77d7976c60c0d704cb403f37ef is removed from metadata 2024.10.22 00:36:09.545298 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:09.551365 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:09.709721 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_211_211_0 (state Temporary) 2024.10.22 00:36:09.926762 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_211_211_0 to 202410_224_224_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:10.100403 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9828 2024.10.22 00:36:10.128208 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 29 entries to flush up to offset 8130 2024.10.22 00:36:10.322705 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 116 2024.10.22 00:36:10.467668 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:10.470330 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:10.495668 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_121_121_0 (state Temporary) 2024.10.22 00:36:10.536858 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_121_121_0 to 202410_127_127_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:10.681375 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8130 2024.10.22 00:36:10.812546 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:10.818356 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:10.943785 [ 1202 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 00:36:11.088697 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_98_98_0 (state Temporary) 2024.10.22 00:36:11.276640 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_98_98_0 to 202410_108_108_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:11.297054 [ 1206 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_103_21 to 202410_108_108_0 2024.10.22 00:36:11.315237 [ 1206 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:11.319851 [ 1206 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:11.348956 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_103_21 to 202410_108_108_0 into Compact with storage Full 2024.10.22 00:36:11.401547 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 116 2024.10.22 00:36:11.405375 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.83 GiB, free memory in arenas 7.60 MiB, will set to 1.67 GiB (RSS), difference: -14.21 MiB 2024.10.22 00:36:11.567936 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:36:11.588658 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_103_21, total 147 rows starting from the beginning of the part 2024.10.22 00:36:11.694109 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_104_104_0, total 1 rows starting from the beginning of the part 2024.10.22 00:36:11.761344 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_105_105_0, total 1 rows starting from the beginning of the part 2024.10.22 00:36:11.913325 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_106_106_0, total 1 rows starting from the beginning of the part 2024.10.22 00:36:12.025668 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_107_107_0, total 1 rows starting from the beginning of the part 2024.10.22 00:36:12.200033 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_108_108_0, total 1 rows starting from the beginning of the part 2024.10.22 00:36:13.470886 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_108_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:36:13.504325 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.83 GiB, free memory in arenas 6.09 MiB, will set to 1.67 GiB (RSS), difference: -3.52 MiB 2024.10.22 00:36:13.930497 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 189675 bytes for 3a3f76295ca3ecfd1fff228d299c33ef:0) 2024.10.22 00:36:13.933305 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a5e97ab309dc056dc4cb489f19bcc8d2, offset: 0, size: 652350 2024.10.22 00:36:13.941370 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a5e97ab309dc056dc4cb489f19bcc8d2 for removal 2024.10.22 00:36:13.956585 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3e14bccd521e38edc4456ff5e22a3f91, offset: 0, size: 249 2024.10.22 00:36:13.953798 [ 1476 ] {} CacheMetadata: Key a5e97ab309dc056dc4cb489f19bcc8d2 is removed from metadata 2024.10.22 00:36:13.965723 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3e14bccd521e38edc4456ff5e22a3f91 for removal 2024.10.22 00:36:13.977211 [ 1476 ] {} CacheMetadata: Key 3e14bccd521e38edc4456ff5e22a3f91 is removed from metadata 2024.10.22 00:36:14.120530 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 9902 2024.10.22 00:36:14.365794 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} MergeTask::MergeProjectionsStage: Merge sorted 152 rows, containing 32 columns (32 merged, 0 gathered) in 3.028606695 sec., 50.18809482622503 rows/sec., 5.64 KiB/sec. 2024.10.22 00:36:14.492371 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.83 GiB, free memory in arenas 6.14 MiB, will set to 1.67 GiB (RSS), difference: 2.22 MiB 2024.10.22 00:36:14.501430 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} MergedBlockOutputStream: filled checksums 202410_1_108_22 (state Temporary) 2024.10.22 00:36:14.513577 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:14.516439 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:14.579906 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_212_212_0 (state Temporary) 2024.10.22 00:36:14.723148 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_212_212_0 to 202410_225_225_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:14.878251 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9902 2024.10.22 00:36:14.985360 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 559184 bytes for 1069ca3278bee12138d37f978dfd58a4:0) 2024.10.22 00:36:14.987232 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0160d77711956c0f320251cf6c45b742, offset: 0, size: 550058 2024.10.22 00:36:14.989624 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0160d77711956c0f320251cf6c45b742 for removal 2024.10.22 00:36:14.991414 [ 1476 ] {} CacheMetadata: Key 0160d77711956c0f320251cf6c45b742 is removed from metadata 2024.10.22 00:36:14.991929 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eeb5e263d7dc66c3fb776f32545ac5b3, offset: 0, size: 247 2024.10.22 00:36:14.996030 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eeb5e263d7dc66c3fb776f32545ac5b3 for removal 2024.10.22 00:36:14.998481 [ 1476 ] {} CacheMetadata: Key eeb5e263d7dc66c3fb776f32545ac5b3 is removed from metadata 2024.10.22 00:36:15.156386 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_108_22 to 202410_1_108_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:15.164462 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_108_22} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_103_21, 202410_108_108_0] -> 202410_1_108_22 2024.10.22 00:36:15.202162 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:36:15.272882 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_71_14, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0, 202410_76_76_0] 2024.10.22 00:36:15.275439 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_71_14, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0, 202410_76_76_0] 2024.10.22 00:36:15.494661 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_71_14, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0, 202410_76_76_0] 2024.10.22 00:36:15.623848 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.83 GiB, free memory in arenas 5.96 MiB, will set to 1.67 GiB (RSS), difference: 10.48 MiB 2024.10.22 00:36:15.875460 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1563 entries to flush up to offset 208286 2024.10.22 00:36:15.996680 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 1734 2024.10.22 00:36:16.333378 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:16.338506 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:16.479092 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_123_123_0 (state Temporary) 2024.10.22 00:36:16.625897 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_123_123_0 to 202410_136_136_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:16.674799 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 116789 bytes for d398a1c1237aa54ec5174294d679ed0d:0) 2024.10.22 00:36:16.677098 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1fac47fe827348a734c7a822b8780da2, offset: 0, size: 547839 2024.10.22 00:36:16.680583 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1734 2024.10.22 00:36:16.680766 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1fac47fe827348a734c7a822b8780da2 for removal 2024.10.22 00:36:16.683291 [ 1476 ] {} CacheMetadata: Key 1fac47fe827348a734c7a822b8780da2 is removed from metadata 2024.10.22 00:36:16.673196 [ 1183 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_131_26 to 202410_136_136_0 2024.10.22 00:36:16.687266 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:16.690636 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:16.693221 [ 1183 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:16.694564 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 48145c8dd2839f79a7c2823015f0d3ae, offset: 0, size: 247 2024.10.22 00:36:16.705045 [ 1183 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:16.707522 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 48145c8dd2839f79a7c2823015f0d3ae for removal 2024.10.22 00:36:16.718504 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_131_26 to 202410_136_136_0 into Compact with storage Full 2024.10.22 00:36:16.810857 [ 1476 ] {} CacheMetadata: Key 48145c8dd2839f79a7c2823015f0d3ae is removed from metadata 2024.10.22 00:36:16.848679 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:36:16.858827 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_131_26, total 1776 rows starting from the beginning of the part 2024.10.22 00:36:16.880175 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_132_132_0, total 13 rows starting from the beginning of the part 2024.10.22 00:36:16.902361 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_133_133_0, total 5 rows starting from the beginning of the part 2024.10.22 00:36:17.055393 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_134_134_0, total 27 rows starting from the beginning of the part 2024.10.22 00:36:17.216500 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_135_135_0, total 6 rows starting from the beginning of the part 2024.10.22 00:36:17.252262 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_136_136_0, total 13 rows starting from the beginning of the part 2024.10.22 00:36:17.469671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 489594 bytes for b3afeeca8e54198f93ea96f47e74701b:0) 2024.10.22 00:36:17.510383 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.83 GiB, free memory in arenas 6.61 MiB, will set to 1.67 GiB (RSS), difference: -9.43 MiB 2024.10.22 00:36:17.564702 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_118_118_0 (state Temporary) 2024.10.22 00:36:17.473804 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ad0ac0b09a3271317e000b47ec3cb505, offset: 0, size: 426297 2024.10.22 00:36:17.577398 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ad0ac0b09a3271317e000b47ec3cb505 for removal 2024.10.22 00:36:17.579036 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0e9b20aba4cc6d3d855cf42f3c1f2ab9, offset: 0, size: 240 2024.10.22 00:36:17.579385 [ 1476 ] {} CacheMetadata: Key ad0ac0b09a3271317e000b47ec3cb505 is removed from metadata 2024.10.22 00:36:17.586572 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0e9b20aba4cc6d3d855cf42f3c1f2ab9 for removal 2024.10.22 00:36:17.592609 [ 1476 ] {} CacheMetadata: Key 0e9b20aba4cc6d3d855cf42f3c1f2ab9 is removed from metadata 2024.10.22 00:36:17.635673 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_118_118_0 to 202410_125_125_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:17.670249 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 208286 2024.10.22 00:36:17.843718 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_55_11, 202410_56_56_0, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0] 2024.10.22 00:36:17.848134 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_55_11, 202410_56_56_0, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0] 2024.10.22 00:36:17.969573 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_55_11, 202410_56_56_0, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0] 2024.10.22 00:36:18.196040 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} MergeTask::MergeProjectionsStage: Merge sorted 1840 rows, containing 12 columns (12 merged, 0 gathered) in 1.482154661 sec., 1241.435896277224 rows/sec., 191.83 KiB/sec. 2024.10.22 00:36:18.260637 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} MergedBlockOutputStream: filled checksums 202410_1_136_27 (state Temporary) 2024.10.22 00:36:18.286512 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 49 entries to flush up to offset 8179 2024.10.22 00:36:18.424566 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_136_27 to 202410_1_136_27 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:18.428867 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_136_27} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_131_26, 202410_136_136_0] -> 202410_1_136_27 2024.10.22 00:36:18.506220 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:36:18.616727 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:18.623110 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:18.683588 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_122_122_0 (state Temporary) 2024.10.22 00:36:18.725436 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_122_122_0 to 202410_128_128_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:18.886554 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 9976 2024.10.22 00:36:18.899507 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8179 2024.10.22 00:36:19.015223 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:19.019620 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:19.174726 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_213_213_0 (state Temporary) 2024.10.22 00:36:19.323802 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_213_213_0 to 202410_226_226_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:19.482421 [ 1236 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_221_44 to 202410_226_226_0 2024.10.22 00:36:19.484502 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.83 GiB, free memory in arenas 6.30 MiB, will set to 1.67 GiB (RSS), difference: 9.33 MiB 2024.10.22 00:36:19.489484 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:36:19.493795 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:36:19.495120 [ 1236 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:19.498100 [ 1236 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:19.588473 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 9976 2024.10.22 00:36:19.620484 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_221_44 to 202410_226_226_0 into Compact with storage Full 2024.10.22 00:36:19.704281 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:36:19.723374 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_221_44, total 10303 rows starting from the beginning of the part 2024.10.22 00:36:19.766750 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_222_222_0, total 156 rows starting from the beginning of the part 2024.10.22 00:36:19.781452 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_223_223_0, total 88 rows starting from the beginning of the part 2024.10.22 00:36:19.791976 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_224_224_0, total 112 rows starting from the beginning of the part 2024.10.22 00:36:19.817790 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_225_225_0, total 74 rows starting from the beginning of the part 2024.10.22 00:36:19.839591 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_226_226_0, total 74 rows starting from the beginning of the part 2024.10.22 00:36:20.690337 [ 1249 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:36:20.713251 [ 1249 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:36:20.807047 [ 1251 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 217 ms. 2024.10.22 00:36:21.736501 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} MergeTask::MergeProjectionsStage: Merge sorted 10807 rows, containing 13 columns (13 merged, 0 gathered) in 2.223535556 sec., 4860.277574981131 rows/sec., 740.52 KiB/sec. 2024.10.22 00:36:21.848056 [ 1242 ] {} DatabaseCatalog: Execution took 226 ms. 2024.10.22 00:36:21.921431 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} MergedBlockOutputStream: filled checksums 202410_1_226_45 (state Temporary) 2024.10.22 00:36:21.984103 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_226_45 to 202410_1_226_45 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:21.987153 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_226_45} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_221_44, 202410_226_226_0] -> 202410_1_226_45 2024.10.22 00:36:22.028035 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.66 MiB. 2024.10.22 00:36:22.416549 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:22.422285 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:22.703387 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 109 file segments (while reserving 16777216 bytes for 854f10257937f71db2f581a95fb90b19:0) 2024.10.22 00:36:22.705551 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a7334f718852ed5a49b3bc77d968f726, offset: 0, size: 171 2024.10.22 00:36:22.714677 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a7334f718852ed5a49b3bc77d968f726 for removal 2024.10.22 00:36:22.722757 [ 1476 ] {} CacheMetadata: Key a7334f718852ed5a49b3bc77d968f726 is removed from metadata 2024.10.22 00:36:22.723285 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 803ab54a3cbcf17bc31c569268a3a95f, offset: 0, size: 361826 2024.10.22 00:36:22.728406 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 803ab54a3cbcf17bc31c569268a3a95f for removal 2024.10.22 00:36:22.736104 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: eb5d77764dd94ce884e12f476cba22f3, offset: 0, size: 170 2024.10.22 00:36:22.740094 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key eb5d77764dd94ce884e12f476cba22f3 for removal 2024.10.22 00:36:22.741743 [ 1476 ] {} CacheMetadata: Key 803ab54a3cbcf17bc31c569268a3a95f is removed from metadata 2024.10.22 00:36:22.757122 [ 1476 ] {} CacheMetadata: Key eb5d77764dd94ce884e12f476cba22f3 is removed from metadata 2024.10.22 00:36:22.851846 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 20e5399a52d9c005a174fa3a2a0b8050, offset: 0, size: 340102 2024.10.22 00:36:22.859020 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 20e5399a52d9c005a174fa3a2a0b8050 for removal 2024.10.22 00:36:22.863719 [ 1476 ] {} CacheMetadata: Key 20e5399a52d9c005a174fa3a2a0b8050 is removed from metadata 2024.10.22 00:36:22.865310 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7c04d42586544845432979c39d6efbf1, offset: 0, size: 170 2024.10.22 00:36:22.870685 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7c04d42586544845432979c39d6efbf1 for removal 2024.10.22 00:36:22.874388 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6d398f066689a82929a716ee7278976d, offset: 0, size: 350351 2024.10.22 00:36:22.878393 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6d398f066689a82929a716ee7278976d for removal 2024.10.22 00:36:22.881417 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 282cc48e5b091150cc0b0da19222667b, offset: 0, size: 170 2024.10.22 00:36:22.883678 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 282cc48e5b091150cc0b0da19222667b for removal 2024.10.22 00:36:22.887034 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7e66ed25fdad9b6ddc0575fe5ed79c4d, offset: 0, size: 150 2024.10.22 00:36:22.889508 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7e66ed25fdad9b6ddc0575fe5ed79c4d for removal 2024.10.22 00:36:22.892523 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b256b0e680804b7814300805ff09ec3c, offset: 0, size: 147 2024.10.22 00:36:22.896081 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b256b0e680804b7814300805ff09ec3c for removal 2024.10.22 00:36:22.905637 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 719249a363a613c933959ca30c6184ee, offset: 0, size: 161535 2024.10.22 00:36:22.916877 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 719249a363a613c933959ca30c6184ee for removal 2024.10.22 00:36:22.924941 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cb3ed3d8ce8e7130b998a4bbe5140dd0, offset: 0, size: 161 2024.10.22 00:36:22.928195 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cb3ed3d8ce8e7130b998a4bbe5140dd0 for removal 2024.10.22 00:36:22.936480 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dc3f24be5f70ac09a038cc23f0f45d58, offset: 0, size: 173 2024.10.22 00:36:22.952103 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dc3f24be5f70ac09a038cc23f0f45d58 for removal 2024.10.22 00:36:22.957200 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f781d8ccd33143ad42ef31c7a28687e5, offset: 0, size: 315234 2024.10.22 00:36:22.965240 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f781d8ccd33143ad42ef31c7a28687e5 for removal 2024.10.22 00:36:22.972230 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f9cfdeca3e3ebf9f31923862fb531ae4, offset: 0, size: 147 2024.10.22 00:36:22.974489 [ 1476 ] {} CacheMetadata: Key f781d8ccd33143ad42ef31c7a28687e5 is removed from metadata 2024.10.22 00:36:22.975653 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f9cfdeca3e3ebf9f31923862fb531ae4 for removal 2024.10.22 00:36:22.980300 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f86f0428f0db16d342e6c80840b8d9ce, offset: 0, size: 86245 2024.10.22 00:36:22.983094 [ 1476 ] {} CacheMetadata: Key f9cfdeca3e3ebf9f31923862fb531ae4 is removed from metadata 2024.10.22 00:36:22.983823 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f86f0428f0db16d342e6c80840b8d9ce for removal 2024.10.22 00:36:22.985649 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8f6ce9c9ccfb9bacd4db10daa8dd8166, offset: 0, size: 177 2024.10.22 00:36:22.989579 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8f6ce9c9ccfb9bacd4db10daa8dd8166 for removal 2024.10.22 00:36:22.989743 [ 1476 ] {} CacheMetadata: Key f86f0428f0db16d342e6c80840b8d9ce is removed from metadata 2024.10.22 00:36:22.994158 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9a8ae86f70a4fa46646ed51183eb074e, offset: 0, size: 77094 2024.10.22 00:36:22.996069 [ 1476 ] {} CacheMetadata: Key 8f6ce9c9ccfb9bacd4db10daa8dd8166 is removed from metadata 2024.10.22 00:36:22.996917 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9a8ae86f70a4fa46646ed51183eb074e for removal 2024.10.22 00:36:22.999757 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d3cd3eb54893b6c9e57d734fb76f68aa, offset: 0, size: 4 2024.10.22 00:36:23.005478 [ 1476 ] {} CacheMetadata: Key 9a8ae86f70a4fa46646ed51183eb074e is removed from metadata 2024.10.22 00:36:23.009326 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d3cd3eb54893b6c9e57d734fb76f68aa for removal 2024.10.22 00:36:23.019784 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2b71f935b10855a81607ffb8a2e78a5b, offset: 0, size: 71599 2024.10.22 00:36:23.022562 [ 1476 ] {} CacheMetadata: Key dc3f24be5f70ac09a038cc23f0f45d58 is removed from metadata 2024.10.22 00:36:23.022978 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2b71f935b10855a81607ffb8a2e78a5b for removal 2024.10.22 00:36:23.027446 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5cae3f05088d93e47eb66bf71e456e22, offset: 0, size: 109921 2024.10.22 00:36:23.031152 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5cae3f05088d93e47eb66bf71e456e22 for removal 2024.10.22 00:36:23.037775 [ 1476 ] {} CacheMetadata: Key 2b71f935b10855a81607ffb8a2e78a5b is removed from metadata 2024.10.22 00:36:23.037720 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e1f413528acbfcdfd66a8f527b5f9176, offset: 0, size: 332539 2024.10.22 00:36:23.042777 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e1f413528acbfcdfd66a8f527b5f9176 for removal 2024.10.22 00:36:23.048593 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6e17d023abe75ea762e52a9d959c7334, offset: 0, size: 177 2024.10.22 00:36:23.048934 [ 1476 ] {} CacheMetadata: Key e1f413528acbfcdfd66a8f527b5f9176 is removed from metadata 2024.10.22 00:36:23.057062 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6e17d023abe75ea762e52a9d959c7334 for removal 2024.10.22 00:36:23.066561 [ 1476 ] {} CacheMetadata: Key 719249a363a613c933959ca30c6184ee is removed from metadata 2024.10.22 00:36:23.068095 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5b0c3196b438226f6ef959de940c3e86, offset: 0, size: 1337874 2024.10.22 00:36:23.072636 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5b0c3196b438226f6ef959de940c3e86 for removal 2024.10.22 00:36:23.075982 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d1a00ce83af3694fc5c32c790f8c2113, offset: 0, size: 184 2024.10.22 00:36:23.076858 [ 1476 ] {} CacheMetadata: Key 5b0c3196b438226f6ef959de940c3e86 is removed from metadata 2024.10.22 00:36:23.078440 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d1a00ce83af3694fc5c32c790f8c2113 for removal 2024.10.22 00:36:23.081178 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 75206f25a12c5d4ea5387437f48be537, offset: 0, size: 317338 2024.10.22 00:36:23.082482 [ 1476 ] {} CacheMetadata: Key d1a00ce83af3694fc5c32c790f8c2113 is removed from metadata 2024.10.22 00:36:23.083761 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 75206f25a12c5d4ea5387437f48be537 for removal 2024.10.22 00:36:23.086306 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 663852eec8aa30f507b623ac2fe2d610, offset: 0, size: 177 2024.10.22 00:36:23.086196 [ 1476 ] {} CacheMetadata: Key 75206f25a12c5d4ea5387437f48be537 is removed from metadata 2024.10.22 00:36:23.091884 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 663852eec8aa30f507b623ac2fe2d610 for removal 2024.10.22 00:36:23.092840 [ 1476 ] {} CacheMetadata: Key d3cd3eb54893b6c9e57d734fb76f68aa is removed from metadata 2024.10.22 00:36:23.095704 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1a19ddca9267c4da8ee10fd79db16aa7, offset: 0, size: 32457 2024.10.22 00:36:23.101397 [ 1476 ] {} CacheMetadata: Key 663852eec8aa30f507b623ac2fe2d610 is removed from metadata 2024.10.22 00:36:23.101064 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1a19ddca9267c4da8ee10fd79db16aa7 for removal 2024.10.22 00:36:23.110132 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1391bd9658433a17eccf9ab64f4f0ec6, offset: 0, size: 176 2024.10.22 00:36:23.120067 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1391bd9658433a17eccf9ab64f4f0ec6 for removal 2024.10.22 00:36:23.121475 [ 1476 ] {} CacheMetadata: Key 7e66ed25fdad9b6ddc0575fe5ed79c4d is removed from metadata 2024.10.22 00:36:23.125301 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9aeaea159846a5914a12631e1cb622a4, offset: 0, size: 179 2024.10.22 00:36:23.128635 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9aeaea159846a5914a12631e1cb622a4 for removal 2024.10.22 00:36:23.130754 [ 1476 ] {} CacheMetadata: Key 1391bd9658433a17eccf9ab64f4f0ec6 is removed from metadata 2024.10.22 00:36:23.133253 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5ba52570be0c4386c48c41567cc1da55, offset: 0, size: 854086 2024.10.22 00:36:23.139025 [ 1476 ] {} CacheMetadata: Key 9aeaea159846a5914a12631e1cb622a4 is removed from metadata 2024.10.22 00:36:23.140154 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5ba52570be0c4386c48c41567cc1da55 for removal 2024.10.22 00:36:23.145261 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1ab85e8a155d4c5fe5c6ffe034647045, offset: 0, size: 207 2024.10.22 00:36:23.151629 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1ab85e8a155d4c5fe5c6ffe034647045 for removal 2024.10.22 00:36:23.151647 [ 1476 ] {} CacheMetadata: Key 5ba52570be0c4386c48c41567cc1da55 is removed from metadata 2024.10.22 00:36:23.155025 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 399977314bb8e493f77ca2d54b2f82e6, offset: 0, size: 178 2024.10.22 00:36:23.161253 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 399977314bb8e493f77ca2d54b2f82e6 for removal 2024.10.22 00:36:23.166639 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e2db9e4bbe2ab4c53abc8bb108b2be14, offset: 0, size: 90976 2024.10.22 00:36:23.170581 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e2db9e4bbe2ab4c53abc8bb108b2be14 for removal 2024.10.22 00:36:23.176503 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 17b66669cc263f5ddd0581b550ce9aa4, offset: 0, size: 136551 2024.10.22 00:36:23.181222 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 17b66669cc263f5ddd0581b550ce9aa4 for removal 2024.10.22 00:36:23.187223 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 985d7449182875bcb751919aa81c67ad, offset: 0, size: 164 2024.10.22 00:36:23.190686 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 985d7449182875bcb751919aa81c67ad for removal 2024.10.22 00:36:23.267130 [ 1476 ] {} CacheMetadata: Key 399977314bb8e493f77ca2d54b2f82e6 is removed from metadata 2024.10.22 00:36:23.275689 [ 1476 ] {} CacheMetadata: Key 7c04d42586544845432979c39d6efbf1 is removed from metadata 2024.10.22 00:36:23.279981 [ 1476 ] {} CacheMetadata: Key 1ab85e8a155d4c5fe5c6ffe034647045 is removed from metadata 2024.10.22 00:36:23.284589 [ 1476 ] {} CacheMetadata: Key e2db9e4bbe2ab4c53abc8bb108b2be14 is removed from metadata 2024.10.22 00:36:23.289562 [ 1476 ] {} CacheMetadata: Key b256b0e680804b7814300805ff09ec3c is removed from metadata 2024.10.22 00:36:23.294695 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8b76666f73396dafc4784f21d7ffb193, offset: 0, size: 56560 2024.10.22 00:36:23.298335 [ 1476 ] {} CacheMetadata: Key 282cc48e5b091150cc0b0da19222667b is removed from metadata 2024.10.22 00:36:23.299018 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8b76666f73396dafc4784f21d7ffb193 for removal 2024.10.22 00:36:23.307606 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 77c7bcb4ca6b0a7657a93a6519233a37, offset: 0, size: 66780 2024.10.22 00:36:23.307606 [ 1476 ] {} CacheMetadata: Key 8b76666f73396dafc4784f21d7ffb193 is removed from metadata 2024.10.22 00:36:23.313187 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 77c7bcb4ca6b0a7657a93a6519233a37 for removal 2024.10.22 00:36:23.317753 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4e303a4a558b96b412502f9fe100b8ad, offset: 0, size: 168371 2024.10.22 00:36:23.319906 [ 1476 ] {} CacheMetadata: Key 77c7bcb4ca6b0a7657a93a6519233a37 is removed from metadata 2024.10.22 00:36:23.323064 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4e303a4a558b96b412502f9fe100b8ad for removal 2024.10.22 00:36:23.327150 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7e8853b34311e70871a2840637ecf24e, offset: 0, size: 165 2024.10.22 00:36:23.327678 [ 1476 ] {} CacheMetadata: Key 4e303a4a558b96b412502f9fe100b8ad is removed from metadata 2024.10.22 00:36:23.330494 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7e8853b34311e70871a2840637ecf24e for removal 2024.10.22 00:36:23.338301 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d28299f49bed3411a1482d43a4d8a646, offset: 0, size: 731594 2024.10.22 00:36:23.340443 [ 1476 ] {} CacheMetadata: Key 7e8853b34311e70871a2840637ecf24e is removed from metadata 2024.10.22 00:36:23.345172 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d28299f49bed3411a1482d43a4d8a646 for removal 2024.10.22 00:36:23.350826 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2e9c3cdaa3f1639231c9955384ae744c, offset: 0, size: 1346529 2024.10.22 00:36:23.353170 [ 1476 ] {} CacheMetadata: Key d28299f49bed3411a1482d43a4d8a646 is removed from metadata 2024.10.22 00:36:23.367452 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2e9c3cdaa3f1639231c9955384ae744c for removal 2024.10.22 00:36:23.367954 [ 1476 ] {} CacheMetadata: Key 5cae3f05088d93e47eb66bf71e456e22 is removed from metadata 2024.10.22 00:36:23.372025 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4c7e83bb91fe85d3c17524b6483ea97b, offset: 0, size: 169 2024.10.22 00:36:23.373409 [ 1476 ] {} CacheMetadata: Key 2e9c3cdaa3f1639231c9955384ae744c is removed from metadata 2024.10.22 00:36:23.375642 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4c7e83bb91fe85d3c17524b6483ea97b for removal 2024.10.22 00:36:23.378157 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a7069c2b7c63752ac77a38cdea0626a9, offset: 0, size: 184 2024.10.22 00:36:23.379500 [ 1476 ] {} CacheMetadata: Key 6d398f066689a82929a716ee7278976d is removed from metadata 2024.10.22 00:36:23.380612 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a7069c2b7c63752ac77a38cdea0626a9 for removal 2024.10.22 00:36:23.384028 [ 1476 ] {} CacheMetadata: Key 17b66669cc263f5ddd0581b550ce9aa4 is removed from metadata 2024.10.22 00:36:23.388369 [ 1476 ] {} CacheMetadata: Key 1a19ddca9267c4da8ee10fd79db16aa7 is removed from metadata 2024.10.22 00:36:23.394236 [ 1476 ] {} CacheMetadata: Key 4c7e83bb91fe85d3c17524b6483ea97b is removed from metadata 2024.10.22 00:36:23.397036 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.83 GiB, free memory in arenas 5.65 MiB, will set to 1.68 GiB (RSS), difference: 4.42 MiB 2024.10.22 00:36:23.400137 [ 1476 ] {} CacheMetadata: Key 985d7449182875bcb751919aa81c67ad is removed from metadata 2024.10.22 00:36:23.412914 [ 1476 ] {} CacheMetadata: Key 6e17d023abe75ea762e52a9d959c7334 is removed from metadata 2024.10.22 00:36:23.421181 [ 1476 ] {} CacheMetadata: Key cb3ed3d8ce8e7130b998a4bbe5140dd0 is removed from metadata 2024.10.22 00:36:23.484215 [ 1476 ] {} CacheMetadata: Key a7069c2b7c63752ac77a38cdea0626a9 is removed from metadata 2024.10.22 00:36:23.484506 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fe3ec25cec8d0518732765191cf5397b, offset: 0, size: 146 2024.10.22 00:36:23.487818 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fe3ec25cec8d0518732765191cf5397b for removal 2024.10.22 00:36:23.491211 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 732ce819db4979bc2961e0edc33bf9bb, offset: 0, size: 177 2024.10.22 00:36:23.494638 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 732ce819db4979bc2961e0edc33bf9bb for removal 2024.10.22 00:36:23.508941 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 755315b455a251ca2ba0b25f9abafc80, offset: 0, size: 149 2024.10.22 00:36:23.515868 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 755315b455a251ca2ba0b25f9abafc80 for removal 2024.10.22 00:36:23.520875 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4a9befec9bac1bfd0aa05172502d2e21, offset: 0, size: 165 2024.10.22 00:36:23.524694 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4a9befec9bac1bfd0aa05172502d2e21 for removal 2024.10.22 00:36:23.528119 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 34fb2d7fa361613196a9e1fba4e852a1, offset: 0, size: 90286 2024.10.22 00:36:23.538713 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 34fb2d7fa361613196a9e1fba4e852a1 for removal 2024.10.22 00:36:23.545627 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d69bd5a09e25acdff4b3c41ac49baa63, offset: 0, size: 183529 2024.10.22 00:36:23.549037 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d69bd5a09e25acdff4b3c41ac49baa63 for removal 2024.10.22 00:36:23.553566 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 177278fffe36cc4d672375ee3309ad9b, offset: 0, size: 6132 2024.10.22 00:36:23.558715 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 177278fffe36cc4d672375ee3309ad9b for removal 2024.10.22 00:36:23.565067 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5ac268256a409a615032020f3923a506, offset: 0, size: 146 2024.10.22 00:36:23.569749 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5ac268256a409a615032020f3923a506 for removal 2024.10.22 00:36:23.573054 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 56909094b1eea3dc3c346d68cbcba209, offset: 0, size: 186354 2024.10.22 00:36:23.575718 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 56909094b1eea3dc3c346d68cbcba209 for removal 2024.10.22 00:36:23.589631 [ 1476 ] {} CacheMetadata: Key 5ac268256a409a615032020f3923a506 is removed from metadata 2024.10.22 00:36:23.670209 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 181 entries to flush up to offset 10157 2024.10.22 00:36:23.676159 [ 1476 ] {} CacheMetadata: Key 177278fffe36cc4d672375ee3309ad9b is removed from metadata 2024.10.22 00:36:23.679062 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 044971822796dac42df0edd03bf2f9fd, offset: 0, size: 176 2024.10.22 00:36:23.680661 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 044971822796dac42df0edd03bf2f9fd for removal 2024.10.22 00:36:23.687695 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a47bdbd16020a8f3791c701348f10447, offset: 0, size: 204310 2024.10.22 00:36:23.700884 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a47bdbd16020a8f3791c701348f10447 for removal 2024.10.22 00:36:23.704909 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4c374dc325ae835f27025d24c1b4edd0, offset: 0, size: 178 2024.10.22 00:36:23.712221 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4c374dc325ae835f27025d24c1b4edd0 for removal 2024.10.22 00:36:23.720451 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7a05d96c6540be6d7dd6321f97766084, offset: 0, size: 167 2024.10.22 00:36:23.725716 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7a05d96c6540be6d7dd6321f97766084 for removal 2024.10.22 00:36:23.729334 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e1c7fbef0b0187aa66054ac556af206d, offset: 0, size: 20500 2024.10.22 00:36:23.737010 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e1c7fbef0b0187aa66054ac556af206d for removal 2024.10.22 00:36:23.746704 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0a5da0b239ade88d3bad93460c41ac54, offset: 0, size: 435026 2024.10.22 00:36:23.782632 [ 1476 ] {} CacheMetadata: Key 34fb2d7fa361613196a9e1fba4e852a1 is removed from metadata 2024.10.22 00:36:23.794484 [ 1476 ] {} CacheMetadata: Key e1c7fbef0b0187aa66054ac556af206d is removed from metadata 2024.10.22 00:36:23.803454 [ 1476 ] {} CacheMetadata: Key 7a05d96c6540be6d7dd6321f97766084 is removed from metadata 2024.10.22 00:36:23.814162 [ 1476 ] {} CacheMetadata: Key 4c374dc325ae835f27025d24c1b4edd0 is removed from metadata 2024.10.22 00:36:23.826883 [ 1476 ] {} CacheMetadata: Key a47bdbd16020a8f3791c701348f10447 is removed from metadata 2024.10.22 00:36:23.838131 [ 1476 ] {} CacheMetadata: Key 044971822796dac42df0edd03bf2f9fd is removed from metadata 2024.10.22 00:36:23.855119 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0a5da0b239ade88d3bad93460c41ac54 for removal 2024.10.22 00:36:23.857062 [ 1476 ] {} CacheMetadata: Key 4a9befec9bac1bfd0aa05172502d2e21 is removed from metadata 2024.10.22 00:36:23.863278 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0306ad3a0a0858c8d56b0fb093de6adc, offset: 0, size: 184 2024.10.22 00:36:23.867591 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0306ad3a0a0858c8d56b0fb093de6adc for removal 2024.10.22 00:36:23.867824 [ 1476 ] {} CacheMetadata: Key 0a5da0b239ade88d3bad93460c41ac54 is removed from metadata 2024.10.22 00:36:23.870514 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fa1f821b83bedecde582e5ca97b7c95f, offset: 0, size: 176 2024.10.22 00:36:23.873156 [ 1476 ] {} CacheMetadata: Key 0306ad3a0a0858c8d56b0fb093de6adc is removed from metadata 2024.10.22 00:36:23.873240 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fa1f821b83bedecde582e5ca97b7c95f for removal 2024.10.22 00:36:23.878226 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3088e0929065a687a412d4c091e9ff98, offset: 0, size: 177 2024.10.22 00:36:23.879419 [ 1476 ] {} CacheMetadata: Key fa1f821b83bedecde582e5ca97b7c95f is removed from metadata 2024.10.22 00:36:23.880464 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3088e0929065a687a412d4c091e9ff98 for removal 2024.10.22 00:36:23.881603 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f5e8515160c04002ec232b20c2b37df5, offset: 0, size: 176 2024.10.22 00:36:23.883092 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f5e8515160c04002ec232b20c2b37df5 for removal 2024.10.22 00:36:23.884455 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e3b8403a2155e10d47dc5672b100032c, offset: 0, size: 181567 2024.10.22 00:36:23.887523 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e3b8403a2155e10d47dc5672b100032c for removal 2024.10.22 00:36:23.889720 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bd071f7fadda816ce17770d2a1eb6c0e, offset: 0, size: 177 2024.10.22 00:36:23.893411 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bd071f7fadda816ce17770d2a1eb6c0e for removal 2024.10.22 00:36:23.898923 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cdcdabb382a4a4f280a16dca9198024d, offset: 0, size: 134296 2024.10.22 00:36:23.909397 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cdcdabb382a4a4f280a16dca9198024d for removal 2024.10.22 00:36:23.916937 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1587e0199f41437d0cfbbf2eefdeeb6c, offset: 0, size: 147 2024.10.22 00:36:23.921950 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1587e0199f41437d0cfbbf2eefdeeb6c for removal 2024.10.22 00:36:23.927831 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: af72b647d9216ac1a91344dc5fa7185b, offset: 0, size: 176 2024.10.22 00:36:23.932052 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key af72b647d9216ac1a91344dc5fa7185b for removal 2024.10.22 00:36:23.938760 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: eda2551c2e907c5287cdb7c45d7adaa4, offset: 0, size: 1337949 2024.10.22 00:36:23.945799 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key eda2551c2e907c5287cdb7c45d7adaa4 for removal 2024.10.22 00:36:23.953822 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b0e488235ae4ae70e93c4205b0fcfe3a, offset: 0, size: 249 2024.10.22 00:36:23.963401 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b0e488235ae4ae70e93c4205b0fcfe3a for removal 2024.10.22 00:36:23.967312 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 42a9a331f2c0cde0c79d3f8de8d2ff25, offset: 0, size: 223012 2024.10.22 00:36:23.969759 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 42a9a331f2c0cde0c79d3f8de8d2ff25 for removal 2024.10.22 00:36:23.972891 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: aad016391d6a682b24fcc01923f5f36e, offset: 0, size: 3092 2024.10.22 00:36:23.974830 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key aad016391d6a682b24fcc01923f5f36e for removal 2024.10.22 00:36:23.976175 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 04a2115d19e36ca52da4330ac6c9c442, offset: 0, size: 212934 2024.10.22 00:36:23.982430 [ 1476 ] {} CacheMetadata: Key 3088e0929065a687a412d4c091e9ff98 is removed from metadata 2024.10.22 00:36:23.985157 [ 1476 ] {} CacheMetadata: Key aad016391d6a682b24fcc01923f5f36e is removed from metadata 2024.10.22 00:36:23.989125 [ 1476 ] {} CacheMetadata: Key 42a9a331f2c0cde0c79d3f8de8d2ff25 is removed from metadata 2024.10.22 00:36:23.998744 [ 1476 ] {} CacheMetadata: Key b0e488235ae4ae70e93c4205b0fcfe3a is removed from metadata 2024.10.22 00:36:24.013064 [ 1476 ] {} CacheMetadata: Key eda2551c2e907c5287cdb7c45d7adaa4 is removed from metadata 2024.10.22 00:36:23.977838 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 04a2115d19e36ca52da4330ac6c9c442 for removal 2024.10.22 00:36:24.082866 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cad646fbfefcde2b6b4f162ecb88b210, offset: 0, size: 253 2024.10.22 00:36:24.085352 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cad646fbfefcde2b6b4f162ecb88b210 for removal 2024.10.22 00:36:24.041728 [ 1476 ] {} CacheMetadata: Key af72b647d9216ac1a91344dc5fa7185b is removed from metadata 2024.10.22 00:36:24.181798 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 1752 2024.10.22 00:36:24.189553 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:24.190102 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2231d99c4fb2ddcffeec5a18660a0c00, offset: 0, size: 100547 2024.10.22 00:36:24.192181 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2231d99c4fb2ddcffeec5a18660a0c00 for removal 2024.10.22 00:36:24.192274 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.26 GiB. 2024.10.22 00:36:24.193310 [ 1476 ] {} CacheMetadata: Key 04a2115d19e36ca52da4330ac6c9c442 is removed from metadata 2024.10.22 00:36:24.195557 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1d34d0833196a02a24e62e35713c81e3, offset: 0, size: 177 2024.10.22 00:36:24.198458 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1d34d0833196a02a24e62e35713c81e3 for removal 2024.10.22 00:36:24.200893 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b2e0341fcd2d4811a563df0a5c7e8b95, offset: 0, size: 149 2024.10.22 00:36:24.215247 [ 1476 ] {} CacheMetadata: Key 1d34d0833196a02a24e62e35713c81e3 is removed from metadata 2024.10.22 00:36:24.226270 [ 1476 ] {} CacheMetadata: Key cdcdabb382a4a4f280a16dca9198024d is removed from metadata 2024.10.22 00:36:24.236965 [ 1476 ] {} CacheMetadata: Key bd071f7fadda816ce17770d2a1eb6c0e is removed from metadata 2024.10.22 00:36:24.252808 [ 1476 ] {} CacheMetadata: Key cad646fbfefcde2b6b4f162ecb88b210 is removed from metadata 2024.10.22 00:36:24.268430 [ 1476 ] {} CacheMetadata: Key e3b8403a2155e10d47dc5672b100032c is removed from metadata 2024.10.22 00:36:24.277922 [ 1476 ] {} CacheMetadata: Key 2231d99c4fb2ddcffeec5a18660a0c00 is removed from metadata 2024.10.22 00:36:24.279464 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_214_214_0 (state Temporary) 2024.10.22 00:36:24.292998 [ 1476 ] {} CacheMetadata: Key 1587e0199f41437d0cfbbf2eefdeeb6c is removed from metadata 2024.10.22 00:36:24.295784 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:24.297424 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.26 GiB. 2024.10.22 00:36:24.299173 [ 1476 ] {} CacheMetadata: Key f5e8515160c04002ec232b20c2b37df5 is removed from metadata 2024.10.22 00:36:24.304153 [ 1476 ] {} CacheMetadata: Key d69bd5a09e25acdff4b3c41ac49baa63 is removed from metadata 2024.10.22 00:36:24.205770 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b2e0341fcd2d4811a563df0a5c7e8b95 for removal 2024.10.22 00:36:24.318588 [ 1476 ] {} CacheMetadata: Key b2e0341fcd2d4811a563df0a5c7e8b95 is removed from metadata 2024.10.22 00:36:24.318808 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7db185722660e3959f2d45e19777b77c, offset: 0, size: 171 2024.10.22 00:36:24.322559 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7db185722660e3959f2d45e19777b77c for removal 2024.10.22 00:36:24.327549 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8aabde91cb2f26c7c7d277cd77daa3c3, offset: 0, size: 80083 2024.10.22 00:36:24.327916 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_124_124_0 (state Temporary) 2024.10.22 00:36:24.330715 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8aabde91cb2f26c7c7d277cd77daa3c3 for removal 2024.10.22 00:36:24.331118 [ 1476 ] {} CacheMetadata: Key 7db185722660e3959f2d45e19777b77c is removed from metadata 2024.10.22 00:36:24.335052 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0cdc5b3d69a838cb0d317b6bc654f5e7, offset: 0, size: 143704 2024.10.22 00:36:24.338867 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_214_214_0 to 202410_227_227_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:24.350388 [ 1476 ] {} CacheMetadata: Key 8aabde91cb2f26c7c7d277cd77daa3c3 is removed from metadata 2024.10.22 00:36:24.364970 [ 1476 ] {} CacheMetadata: Key 755315b455a251ca2ba0b25f9abafc80 is removed from metadata 2024.10.22 00:36:24.375997 [ 1476 ] {} CacheMetadata: Key 732ce819db4979bc2961e0edc33bf9bb is removed from metadata 2024.10.22 00:36:24.384466 [ 1476 ] {} CacheMetadata: Key 56909094b1eea3dc3c346d68cbcba209 is removed from metadata 2024.10.22 00:36:24.386201 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_124_124_0 to 202410_137_137_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:24.390566 [ 1476 ] {} CacheMetadata: Key fe3ec25cec8d0518732765191cf5397b is removed from metadata 2024.10.22 00:36:24.391021 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 10157 2024.10.22 00:36:24.430686 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1752 2024.10.22 00:36:24.440955 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0cdc5b3d69a838cb0d317b6bc654f5e7 for removal 2024.10.22 00:36:24.447227 [ 1476 ] {} CacheMetadata: Key 0cdc5b3d69a838cb0d317b6bc654f5e7 is removed from metadata 2024.10.22 00:36:24.447666 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 992ab7992b7c03195d01f0e41cca10b3, offset: 0, size: 245 2024.10.22 00:36:24.453685 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 992ab7992b7c03195d01f0e41cca10b3 for removal 2024.10.22 00:36:24.460719 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 436d2d4ab43146fdd08f953e23188776, offset: 0, size: 167 2024.10.22 00:36:24.460958 [ 1476 ] {} CacheMetadata: Key 992ab7992b7c03195d01f0e41cca10b3 is removed from metadata 2024.10.22 00:36:24.467718 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 436d2d4ab43146fdd08f953e23188776 for removal 2024.10.22 00:36:24.471102 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e876ab29b3212c55d4e5ebfc08b96690, offset: 0, size: 1995 2024.10.22 00:36:24.474334 [ 1476 ] {} CacheMetadata: Key 436d2d4ab43146fdd08f953e23188776 is removed from metadata 2024.10.22 00:36:24.475064 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e876ab29b3212c55d4e5ebfc08b96690 for removal 2024.10.22 00:36:24.480214 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 51bfda3947029924d89a341d45ea568f, offset: 0, size: 66154 2024.10.22 00:36:24.480799 [ 1476 ] {} CacheMetadata: Key e876ab29b3212c55d4e5ebfc08b96690 is removed from metadata 2024.10.22 00:36:24.482272 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 51bfda3947029924d89a341d45ea568f for removal 2024.10.22 00:36:24.486076 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 71a883010d50f4fbce7f67a470e56a9d, offset: 0, size: 598827 2024.10.22 00:36:24.486352 [ 1476 ] {} CacheMetadata: Key 51bfda3947029924d89a341d45ea568f is removed from metadata 2024.10.22 00:36:24.493828 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 71a883010d50f4fbce7f67a470e56a9d for removal 2024.10.22 00:36:24.505221 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ab06cfc1fa1d694649c3c8b0d93df5a8, offset: 0, size: 177 2024.10.22 00:36:24.515840 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ab06cfc1fa1d694649c3c8b0d93df5a8 for removal 2024.10.22 00:36:24.517802 [ 1476 ] {} CacheMetadata: Key 71a883010d50f4fbce7f67a470e56a9d is removed from metadata 2024.10.22 00:36:24.522497 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c049d0b8d24ef41e5c5b207e86854ee8, offset: 0, size: 243 2024.10.22 00:36:24.525757 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c049d0b8d24ef41e5c5b207e86854ee8 for removal 2024.10.22 00:36:24.529480 [ 1476 ] {} CacheMetadata: Key ab06cfc1fa1d694649c3c8b0d93df5a8 is removed from metadata 2024.10.22 00:36:24.539965 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: df00325251754b44256ee7676a0a4415, offset: 0, size: 176 2024.10.22 00:36:24.549741 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key df00325251754b44256ee7676a0a4415 for removal 2024.10.22 00:36:24.553423 [ 1476 ] {} CacheMetadata: Key c049d0b8d24ef41e5c5b207e86854ee8 is removed from metadata 2024.10.22 00:36:24.562527 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b2971dbaf2e168012040876d46edd5a7, offset: 0, size: 45339 2024.10.22 00:36:24.567301 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b2971dbaf2e168012040876d46edd5a7 for removal 2024.10.22 00:36:24.568846 [ 1476 ] {} CacheMetadata: Key df00325251754b44256ee7676a0a4415 is removed from metadata 2024.10.22 00:36:24.572644 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c2d731e7ae7578db8eba0635a34e44d4, offset: 0, size: 1054476 2024.10.22 00:36:24.578607 [ 1476 ] {} CacheMetadata: Key b2971dbaf2e168012040876d46edd5a7 is removed from metadata 2024.10.22 00:36:24.679791 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c2d731e7ae7578db8eba0635a34e44d4 for removal 2024.10.22 00:36:24.682217 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 209822 2024.10.22 00:36:24.683204 [ 1476 ] {} CacheMetadata: Key c2d731e7ae7578db8eba0635a34e44d4 is removed from metadata 2024.10.22 00:36:24.683747 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7734959867f226e55ba41039fa24d032, offset: 0, size: 176 2024.10.22 00:36:24.686988 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7734959867f226e55ba41039fa24d032 for removal 2024.10.22 00:36:24.691725 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 203a74ba35b0565653f0f0b910ced822, offset: 0, size: 168 2024.10.22 00:36:24.692930 [ 1476 ] {} CacheMetadata: Key 7734959867f226e55ba41039fa24d032 is removed from metadata 2024.10.22 00:36:24.696103 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 203a74ba35b0565653f0f0b910ced822 for removal 2024.10.22 00:36:24.700573 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0a71f6ac1a77c9fc6f67b10b5549553e, offset: 0, size: 139706 2024.10.22 00:36:24.700999 [ 1476 ] {} CacheMetadata: Key 203a74ba35b0565653f0f0b910ced822 is removed from metadata 2024.10.22 00:36:24.705300 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0a71f6ac1a77c9fc6f67b10b5549553e for removal 2024.10.22 00:36:24.715170 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 75e7994221a23349cc5784a508fa3d04, offset: 0, size: 505386 2024.10.22 00:36:24.715620 [ 1476 ] {} CacheMetadata: Key 0a71f6ac1a77c9fc6f67b10b5549553e is removed from metadata 2024.10.22 00:36:24.820657 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 75e7994221a23349cc5784a508fa3d04 for removal 2024.10.22 00:36:24.823757 [ 1476 ] {} CacheMetadata: Key 75e7994221a23349cc5784a508fa3d04 is removed from metadata 2024.10.22 00:36:24.823757 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0b767c3b443a4487494d3af539bec4f3, offset: 0, size: 144 2024.10.22 00:36:24.825722 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0b767c3b443a4487494d3af539bec4f3 for removal 2024.10.22 00:36:24.828229 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e675369acfc5f3026766ed649a4ffeba, offset: 0, size: 178 2024.10.22 00:36:24.828631 [ 1476 ] {} CacheMetadata: Key 0b767c3b443a4487494d3af539bec4f3 is removed from metadata 2024.10.22 00:36:24.834849 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e675369acfc5f3026766ed649a4ffeba for removal 2024.10.22 00:36:24.840062 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f6dee03dd480903b1ea5036a961ceaf0, offset: 0, size: 113607 2024.10.22 00:36:24.841557 [ 1476 ] {} CacheMetadata: Key e675369acfc5f3026766ed649a4ffeba is removed from metadata 2024.10.22 00:36:24.845587 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f6dee03dd480903b1ea5036a961ceaf0 for removal 2024.10.22 00:36:24.852170 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 83079f1f6e36573a37ef5801b20e7422, offset: 0, size: 836494 2024.10.22 00:36:24.854073 [ 1476 ] {} CacheMetadata: Key f6dee03dd480903b1ea5036a961ceaf0 is removed from metadata 2024.10.22 00:36:24.858708 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 83079f1f6e36573a37ef5801b20e7422 for removal 2024.10.22 00:36:24.866931 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e95d246fb96fd920acd52b4464bb6471, offset: 0, size: 163 2024.10.22 00:36:24.867998 [ 1476 ] {} CacheMetadata: Key 83079f1f6e36573a37ef5801b20e7422 is removed from metadata 2024.10.22 00:36:24.869711 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e95d246fb96fd920acd52b4464bb6471 for removal 2024.10.22 00:36:24.872619 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 37bc45bec617ba6cff21aa09afa8f9af, offset: 0, size: 200888 2024.10.22 00:36:24.872920 [ 1476 ] {} CacheMetadata: Key e95d246fb96fd920acd52b4464bb6471 is removed from metadata 2024.10.22 00:36:24.877400 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 37bc45bec617ba6cff21aa09afa8f9af for removal 2024.10.22 00:36:24.880833 [ 1476 ] {} CacheMetadata: Key 37bc45bec617ba6cff21aa09afa8f9af is removed from metadata 2024.10.22 00:36:24.880958 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dd7f39f1f7dbbc66c518676df6e01cab, offset: 0, size: 2532 2024.10.22 00:36:24.883371 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dd7f39f1f7dbbc66c518676df6e01cab for removal 2024.10.22 00:36:24.885528 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f51ae5a9e8d4ce1a8b6d88c3fb35b0f6, offset: 0, size: 432497 2024.10.22 00:36:24.887001 [ 1476 ] {} CacheMetadata: Key dd7f39f1f7dbbc66c518676df6e01cab is removed from metadata 2024.10.22 00:36:24.899012 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f51ae5a9e8d4ce1a8b6d88c3fb35b0f6 for removal 2024.10.22 00:36:24.903053 [ 1476 ] {} CacheMetadata: Key f51ae5a9e8d4ce1a8b6d88c3fb35b0f6 is removed from metadata 2024.10.22 00:36:24.903523 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9f3dcc3163ef28004be3289331c4c4fc, offset: 0, size: 1154912 2024.10.22 00:36:24.908231 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9f3dcc3163ef28004be3289331c4c4fc for removal 2024.10.22 00:36:24.911668 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6cf90ff639b0fdaefdb4dc8bff3b910b, offset: 0, size: 130934 2024.10.22 00:36:24.913633 [ 1476 ] {} CacheMetadata: Key 9f3dcc3163ef28004be3289331c4c4fc is removed from metadata 2024.10.22 00:36:24.915503 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6cf90ff639b0fdaefdb4dc8bff3b910b for removal 2024.10.22 00:36:25.021481 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 04fe2cb22772c06075c4b2f401226925, offset: 0, size: 178 2024.10.22 00:36:25.028647 [ 1476 ] {} CacheMetadata: Key 6cf90ff639b0fdaefdb4dc8bff3b910b is removed from metadata 2024.10.22 00:36:25.032304 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 04fe2cb22772c06075c4b2f401226925 for removal 2024.10.22 00:36:25.037445 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f6e7f377c00e13e0f7f4335e90f706cb, offset: 0, size: 183076 2024.10.22 00:36:25.038598 [ 1476 ] {} CacheMetadata: Key 04fe2cb22772c06075c4b2f401226925 is removed from metadata 2024.10.22 00:36:25.040279 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f6e7f377c00e13e0f7f4335e90f706cb for removal 2024.10.22 00:36:25.047644 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 31b1b8094fbed6a6da9ee9374f7359a1, offset: 0, size: 176 2024.10.22 00:36:25.048964 [ 1476 ] {} CacheMetadata: Key f6e7f377c00e13e0f7f4335e90f706cb is removed from metadata 2024.10.22 00:36:25.053463 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 31b1b8094fbed6a6da9ee9374f7359a1 for removal 2024.10.22 00:36:25.058909 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4a50d5a2ebacfbe4028d98209dc80725, offset: 0, size: 249 2024.10.22 00:36:25.066619 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4a50d5a2ebacfbe4028d98209dc80725 for removal 2024.10.22 00:36:25.067775 [ 1476 ] {} CacheMetadata: Key 31b1b8094fbed6a6da9ee9374f7359a1 is removed from metadata 2024.10.22 00:36:25.069474 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 35f2637dcddd95ada5d2789ebc4a8602, offset: 0, size: 401203 2024.10.22 00:36:25.074516 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 35f2637dcddd95ada5d2789ebc4a8602 for removal 2024.10.22 00:36:25.078485 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 251d828f9dbd6a077fcee7fa0048a2e3, offset: 0, size: 3095 2024.10.22 00:36:25.081935 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 251d828f9dbd6a077fcee7fa0048a2e3 for removal 2024.10.22 00:36:25.174841 [ 1476 ] {} CacheMetadata: Key 251d828f9dbd6a077fcee7fa0048a2e3 is removed from metadata 2024.10.22 00:36:25.179632 [ 1476 ] {} CacheMetadata: Key 35f2637dcddd95ada5d2789ebc4a8602 is removed from metadata 2024.10.22 00:36:25.187070 [ 1476 ] {} CacheMetadata: Key 4a50d5a2ebacfbe4028d98209dc80725 is removed from metadata 2024.10.22 00:36:25.334460 [ 1211 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 00:36:25.560354 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:25.564408 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:25.640753 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_25_25_0 (state Temporary) 2024.10.22 00:36:25.756907 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 49261 bytes for 57df92c36162da7619aa037fe8adfbc1:0) 2024.10.22 00:36:25.763718 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 961eaa32daf11b579de173ad1fcbafa5, offset: 0, size: 395244 2024.10.22 00:36:25.769062 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 961eaa32daf11b579de173ad1fcbafa5 for removal 2024.10.22 00:36:25.771983 [ 1476 ] {} CacheMetadata: Key 961eaa32daf11b579de173ad1fcbafa5 is removed from metadata 2024.10.22 00:36:25.774548 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8d808ecfd3117384f927e98704bf303e, offset: 0, size: 171 2024.10.22 00:36:25.788073 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8d808ecfd3117384f927e98704bf303e for removal 2024.10.22 00:36:25.801375 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_119_119_0 (state Temporary) 2024.10.22 00:36:25.877787 [ 1476 ] {} CacheMetadata: Key 8d808ecfd3117384f927e98704bf303e is removed from metadata 2024.10.22 00:36:25.987374 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_119_119_0 to 202410_126_126_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:26.012279 [ 1275 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_121_24 to 202410_126_126_0 2024.10.22 00:36:26.024485 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 209822 2024.10.22 00:36:26.034545 [ 1275 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:26.038994 [ 1275 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:26.171132 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_121_24 to 202410_126_126_0 into Compact with storage Full 2024.10.22 00:36:26.193167 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:36:26.201634 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} MergeTreeSequentialSource: Reading 28 marks from part 202410_1_121_24, total 215348 rows starting from the beginning of the part 2024.10.22 00:36:26.217733 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_122_122_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:36:26.231936 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_123_123_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:36:26.251542 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_124_124_0, total 1509 rows starting from the beginning of the part 2024.10.22 00:36:26.281901 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.83 GiB, free memory in arenas 6.55 MiB, will set to 1.68 GiB (RSS), difference: -25.29 MiB 2024.10.22 00:36:26.372322 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_125_125_0, total 1563 rows starting from the beginning of the part 2024.10.22 00:36:26.382073 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_126_126_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:36:26.404951 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 117 2024.10.22 00:36:26.406369 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 45 entries to flush up to offset 8224 2024.10.22 00:36:26.564280 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:26.569026 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:26.611914 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_123_123_0 (state Temporary) 2024.10.22 00:36:26.806557 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:26.816233 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.24 GiB. 2024.10.22 00:36:26.880141 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_123_123_0 to 202410_129_129_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:26.890507 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_99_99_0 (state Temporary) 2024.10.22 00:36:26.930760 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8224 2024.10.22 00:36:27.079299 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_99_99_0 to 202410_109_109_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:27.899478 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 117 2024.10.22 00:36:28.198253 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 432915 bytes for 32b7bceed9f67092d92024e6fa1cb480:0) 2024.10.22 00:36:28.204967 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4ce8b93406799f9e6dbcbc321563f484, offset: 0, size: 438111 2024.10.22 00:36:28.212520 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4ce8b93406799f9e6dbcbc321563f484 for removal 2024.10.22 00:36:28.218780 [ 1476 ] {} CacheMetadata: Key 4ce8b93406799f9e6dbcbc321563f484 is removed from metadata 2024.10.22 00:36:28.218480 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dc169e70f75e643f68914288fd7a7762, offset: 0, size: 171 2024.10.22 00:36:28.225608 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dc169e70f75e643f68914288fd7a7762 for removal 2024.10.22 00:36:28.235522 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4e5fab2c633fb34fbdd8c0ee24130379, offset: 0, size: 326462 2024.10.22 00:36:28.246325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4e5fab2c633fb34fbdd8c0ee24130379 for removal 2024.10.22 00:36:28.255323 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2c653b0350a879f3fef51830dea60da4, offset: 0, size: 168 2024.10.22 00:36:28.256801 [ 1476 ] {} CacheMetadata: Key 4e5fab2c633fb34fbdd8c0ee24130379 is removed from metadata 2024.10.22 00:36:28.263256 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2c653b0350a879f3fef51830dea60da4 for removal 2024.10.22 00:36:28.271655 [ 1476 ] {} CacheMetadata: Key 2c653b0350a879f3fef51830dea60da4 is removed from metadata 2024.10.22 00:36:28.277562 [ 1476 ] {} CacheMetadata: Key dc169e70f75e643f68914288fd7a7762 is removed from metadata 2024.10.22 00:36:28.411571 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 244 entries to flush up to offset 10401 2024.10.22 00:36:28.636109 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 534861 bytes for b4d37d2dd25774db3bcef3dc515bfb0e:0) 2024.10.22 00:36:28.638478 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1171ed2be714d3c3f2ea1754339f8078, offset: 0, size: 390581 2024.10.22 00:36:28.649478 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1171ed2be714d3c3f2ea1754339f8078 for removal 2024.10.22 00:36:28.656410 [ 1476 ] {} CacheMetadata: Key 1171ed2be714d3c3f2ea1754339f8078 is removed from metadata 2024.10.22 00:36:28.657246 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9efa1c2463177bce135960dd439096a0, offset: 0, size: 170 2024.10.22 00:36:28.664736 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9efa1c2463177bce135960dd439096a0 for removal 2024.10.22 00:36:28.678806 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_126_25/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:36:28.685476 [ 1476 ] {} CacheMetadata: Key 9efa1c2463177bce135960dd439096a0 is removed from metadata 2024.10.22 00:36:28.855716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 864677 bytes for f1217972affb92f68b5817526455f2d7:0) 2024.10.22 00:36:28.859072 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 48e39e57b50638a584fde32ec29d1169, offset: 0, size: 373311 2024.10.22 00:36:28.868117 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 48e39e57b50638a584fde32ec29d1169 for removal 2024.10.22 00:36:28.871879 [ 1476 ] {} CacheMetadata: Key 48e39e57b50638a584fde32ec29d1169 is removed from metadata 2024.10.22 00:36:28.871941 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c999e0c96e03bd42cc66df8dd6e41d3e, offset: 0, size: 170 2024.10.22 00:36:28.876980 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c999e0c96e03bd42cc66df8dd6e41d3e for removal 2024.10.22 00:36:28.879086 [ 1476 ] {} CacheMetadata: Key c999e0c96e03bd42cc66df8dd6e41d3e is removed from metadata 2024.10.22 00:36:28.879515 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 84ada0f8273a02fbe4bde405a393dda4, offset: 0, size: 373069 2024.10.22 00:36:28.882403 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 84ada0f8273a02fbe4bde405a393dda4 for removal 2024.10.22 00:36:28.885431 [ 1476 ] {} CacheMetadata: Key 84ada0f8273a02fbe4bde405a393dda4 is removed from metadata 2024.10.22 00:36:28.885304 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e57e5cbf1e5392114475810669b0426d, offset: 0, size: 170 2024.10.22 00:36:28.890872 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e57e5cbf1e5392114475810669b0426d for removal 2024.10.22 00:36:28.895594 [ 1476 ] {} CacheMetadata: Key e57e5cbf1e5392114475810669b0426d is removed from metadata 2024.10.22 00:36:29.087760 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:29.090393 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:29.308121 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_215_215_0 (state Temporary) 2024.10.22 00:36:29.339918 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.83 GiB, free memory in arenas 9.34 MiB, will set to 1.69 GiB (RSS), difference: 5.84 MiB 2024.10.22 00:36:29.368696 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lpe/qgsqzrpjqpbymlprhjbatsdunljtr. 2024.10.22 00:36:29.377173 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_215_215_0 to 202410_228_228_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:29.446419 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 10401 2024.10.22 00:36:29.488966 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:36:29.491282 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:36:29.503313 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/oyb/vlbjnmhomyzjnszfhllwufomgypne. 2024.10.22 00:36:29.582521 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_121_24, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0] 2024.10.22 00:36:29.584049 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_121_24, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0] 2024.10.22 00:36:29.614915 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_121_24, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0] 2024.10.22 00:36:30.230992 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_65_13, 202410_66_66_0, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0] 2024.10.22 00:36:30.236943 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_65_13, 202410_66_66_0, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0] 2024.10.22 00:36:30.385415 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_65_13, 202410_66_66_0, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0] 2024.10.22 00:36:30.527402 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/oyb/vlbjnmhomyzjnszfhllwufomgypne. 2024.10.22 00:36:30.635637 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_8_8_0 to 201403_8_8_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:30.661126 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lpe/qgsqzrpjqpbymlprhjbatsdunljtr. 2024.10.22 00:36:30.688559 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:36:31.491613 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.83 GiB, free memory in arenas 9.28 MiB, will set to 1.69 GiB (RSS), difference: 292.56 MiB 2024.10.22 00:36:31.768759 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_25_25_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:36:31.786137 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_9_2, 202410_10_10_0, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0] 2024.10.22 00:36:31.788156 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_9_2, 202410_10_10_0, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0] 2024.10.22 00:36:31.827748 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_9_2, 202410_10_10_0, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0, 202410_14_14_0] 2024.10.22 00:36:31.933344 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 1759 2024.10.22 00:36:32.091055 [ 1187 ] {} DatabaseCatalog: Execution took 249 ms. 2024.10.22 00:36:32.167276 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:32.171946 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:32.308645 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_125_125_0 (state Temporary) 2024.10.22 00:36:32.468191 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_125_125_0 to 202410_138_138_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:32.502213 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1759 2024.10.22 00:36:33.027930 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 211742 2024.10.22 00:36:33.469657 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 10456 2024.10.22 00:36:33.579728 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:33.581303 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:33.612446 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.83 GiB, free memory in arenas 12.61 MiB, will set to 1.45 GiB (RSS), difference: -31.49 MiB 2024.10.22 00:36:33.743406 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_216_216_0 (state Temporary) 2024.10.22 00:36:33.790328 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_216_216_0 to 202410_229_229_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:33.891404 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 10456 2024.10.22 00:36:33.979809 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} MergeTask::MergeProjectionsStage: Merge sorted 223412 rows, containing 4 columns (4 merged, 0 gathered) in 7.91292024 sec., 28233.824330826315 rows/sec., 414.58 KiB/sec. 2024.10.22 00:36:34.004364 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:34.114452 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:34.158453 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} MergedBlockOutputStream: filled checksums 202410_1_126_25 (state Temporary) 2024.10.22 00:36:34.275030 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_126_25 to 202410_1_126_25 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:34.378528 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_126_25} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_121_24, 202410_126_126_0] -> 202410_1_126_25 2024.10.22 00:36:34.378886 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_120_120_0 (state Temporary) 2024.10.22 00:36:34.398551 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:36:34.436514 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 49 entries to flush up to offset 8273 2024.10.22 00:36:34.521738 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:34.524883 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:34.570993 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_124_124_0 (state Temporary) 2024.10.22 00:36:34.614682 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_124_124_0 to 202410_130_130_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:34.624171 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_120_120_0 to 202410_127_127_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:34.663860 [ 1199 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_125_25 to 202410_130_130_0 2024.10.22 00:36:34.670995 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 211742 2024.10.22 00:36:34.672310 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8273 2024.10.22 00:36:34.674803 [ 1199 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:34.676174 [ 1199 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:34.689524 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_125_25 to 202410_130_130_0 into Compact with storage Full 2024.10.22 00:36:34.716438 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:36:34.726399 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_125_25, total 8116 rows starting from the beginning of the part 2024.10.22 00:36:34.768269 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_126_126_0, total 89 rows starting from the beginning of the part 2024.10.22 00:36:34.815627 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_127_127_0, total 29 rows starting from the beginning of the part 2024.10.22 00:36:34.889308 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_128_128_0, total 49 rows starting from the beginning of the part 2024.10.22 00:36:35.033338 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_129_129_0, total 45 rows starting from the beginning of the part 2024.10.22 00:36:35.192959 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_130_130_0, total 49 rows starting from the beginning of the part 2024.10.22 00:36:35.243638 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 131 file segments (while reserving 16777216 bytes for 854f10257937f71db2f581a95fb90b19:33554432) 2024.10.22 00:36:35.246133 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e381530f53eb987e4b210fa709f313fa, offset: 0, size: 175 2024.10.22 00:36:35.247931 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.83 GiB, free memory in arenas 9.48 MiB, will set to 1.49 GiB (RSS), difference: -46.44 MiB 2024.10.22 00:36:35.253222 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e381530f53eb987e4b210fa709f313fa for removal 2024.10.22 00:36:35.261376 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 17491e5e3da83752376e2ff1b9ed97b0, offset: 0, size: 144 2024.10.22 00:36:35.264048 [ 1476 ] {} CacheMetadata: Key e381530f53eb987e4b210fa709f313fa is removed from metadata 2024.10.22 00:36:35.267753 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 17491e5e3da83752376e2ff1b9ed97b0 for removal 2024.10.22 00:36:35.270299 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6f1e23935ffd34d08eb07f08394447e2, offset: 0, size: 167085 2024.10.22 00:36:35.272947 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6f1e23935ffd34d08eb07f08394447e2 for removal 2024.10.22 00:36:35.275565 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1ddcc482df9b2fceaf0b5bfb92e125e5, offset: 0, size: 145 2024.10.22 00:36:35.276877 [ 1476 ] {} CacheMetadata: Key 6f1e23935ffd34d08eb07f08394447e2 is removed from metadata 2024.10.22 00:36:35.277567 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1ddcc482df9b2fceaf0b5bfb92e125e5 for removal 2024.10.22 00:36:35.282098 [ 1476 ] {} CacheMetadata: Key 17491e5e3da83752376e2ff1b9ed97b0 is removed from metadata 2024.10.22 00:36:35.379777 [ 1476 ] {} CacheMetadata: Key 1ddcc482df9b2fceaf0b5bfb92e125e5 is removed from metadata 2024.10.22 00:36:35.380180 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 892d4121d3c553ce6dca001edeb8e252, offset: 0, size: 320561 2024.10.22 00:36:35.382553 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 892d4121d3c553ce6dca001edeb8e252 for removal 2024.10.22 00:36:35.384349 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: de9d3389b40534ab82d4598e99ca97c8, offset: 0, size: 225611 2024.10.22 00:36:35.384557 [ 1476 ] {} CacheMetadata: Key 892d4121d3c553ce6dca001edeb8e252 is removed from metadata 2024.10.22 00:36:35.386507 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key de9d3389b40534ab82d4598e99ca97c8 for removal 2024.10.22 00:36:35.387981 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: defb4436657b10832545013d27bc5189, offset: 0, size: 146 2024.10.22 00:36:35.389052 [ 1476 ] {} CacheMetadata: Key de9d3389b40534ab82d4598e99ca97c8 is removed from metadata 2024.10.22 00:36:35.389438 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key defb4436657b10832545013d27bc5189 for removal 2024.10.22 00:36:35.391613 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3a86a42717d9b47b8718d0928248b8ed, offset: 0, size: 265759 2024.10.22 00:36:35.398514 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3a86a42717d9b47b8718d0928248b8ed for removal 2024.10.22 00:36:35.399673 [ 1476 ] {} CacheMetadata: Key defb4436657b10832545013d27bc5189 is removed from metadata 2024.10.22 00:36:35.401070 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 118 2024.10.22 00:36:35.405274 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3aba59282ced9b24c236e0198a1e3924, offset: 0, size: 297351 2024.10.22 00:36:35.408766 [ 1476 ] {} CacheMetadata: Key 3a86a42717d9b47b8718d0928248b8ed is removed from metadata 2024.10.22 00:36:35.510981 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3aba59282ced9b24c236e0198a1e3924 for removal 2024.10.22 00:36:35.523475 [ 1476 ] {} CacheMetadata: Key 3aba59282ced9b24c236e0198a1e3924 is removed from metadata 2024.10.22 00:36:35.524048 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7174d7c8a90fd9c59df0512cc8b70f8e, offset: 0, size: 322313 2024.10.22 00:36:35.529427 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7174d7c8a90fd9c59df0512cc8b70f8e for removal 2024.10.22 00:36:35.535544 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 04e1da46a27206df7fcc93ec2f511186, offset: 0, size: 146 2024.10.22 00:36:35.535191 [ 1476 ] {} CacheMetadata: Key 7174d7c8a90fd9c59df0512cc8b70f8e is removed from metadata 2024.10.22 00:36:35.542973 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 04e1da46a27206df7fcc93ec2f511186 for removal 2024.10.22 00:36:35.548887 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 615bcb68d7f2a85984b5f70835e5dd2c, offset: 0, size: 148 2024.10.22 00:36:35.555266 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 615bcb68d7f2a85984b5f70835e5dd2c for removal 2024.10.22 00:36:35.556086 [ 1476 ] {} CacheMetadata: Key 04e1da46a27206df7fcc93ec2f511186 is removed from metadata 2024.10.22 00:36:35.559334 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 66344649326d6be877be4082ba96a898, offset: 0, size: 335206 2024.10.22 00:36:35.568336 [ 1476 ] {} CacheMetadata: Key 615bcb68d7f2a85984b5f70835e5dd2c is removed from metadata 2024.10.22 00:36:35.574830 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 66344649326d6be877be4082ba96a898 for removal 2024.10.22 00:36:35.679673 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c831b9ad5293af57203b523ffc7b4658, offset: 0, size: 265639 2024.10.22 00:36:35.680008 [ 1476 ] {} CacheMetadata: Key 66344649326d6be877be4082ba96a898 is removed from metadata 2024.10.22 00:36:35.681386 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c831b9ad5293af57203b523ffc7b4658 for removal 2024.10.22 00:36:35.683723 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f4a4de9bca1f3b590185afcd002de772, offset: 0, size: 148 2024.10.22 00:36:35.685603 [ 1476 ] {} CacheMetadata: Key c831b9ad5293af57203b523ffc7b4658 is removed from metadata 2024.10.22 00:36:35.686247 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f4a4de9bca1f3b590185afcd002de772 for removal 2024.10.22 00:36:35.688485 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 64d1768e5606a02c8b7ea3bf66129812, offset: 0, size: 331841 2024.10.22 00:36:35.691377 [ 1476 ] {} CacheMetadata: Key f4a4de9bca1f3b590185afcd002de772 is removed from metadata 2024.10.22 00:36:35.695697 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 64d1768e5606a02c8b7ea3bf66129812 for removal 2024.10.22 00:36:35.699628 [ 1476 ] {} CacheMetadata: Key 64d1768e5606a02c8b7ea3bf66129812 is removed from metadata 2024.10.22 00:36:35.699902 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 075e5e1fe1d7dbd84695cff07261cee0, offset: 0, size: 148 2024.10.22 00:36:35.706545 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 075e5e1fe1d7dbd84695cff07261cee0 for removal 2024.10.22 00:36:35.716290 [ 1476 ] {} CacheMetadata: Key 075e5e1fe1d7dbd84695cff07261cee0 is removed from metadata 2024.10.22 00:36:35.716657 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 29a52c4c62392ba4dc9ac2a789098e95, offset: 0, size: 261506 2024.10.22 00:36:35.725404 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 29a52c4c62392ba4dc9ac2a789098e95 for removal 2024.10.22 00:36:35.726674 [ 1181 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:36:35.727837 [ 1476 ] {} CacheMetadata: Key 29a52c4c62392ba4dc9ac2a789098e95 is removed from metadata 2024.10.22 00:36:35.729435 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a036232dcfebb8b8900b993e33faebe9, offset: 0, size: 146 2024.10.22 00:36:35.739723 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a036232dcfebb8b8900b993e33faebe9 for removal 2024.10.22 00:36:35.745095 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e057cc5b7f077e194aaa2495a15dffee, offset: 0, size: 236001 2024.10.22 00:36:35.751970 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e057cc5b7f077e194aaa2495a15dffee for removal 2024.10.22 00:36:35.760202 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:35.760333 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5ed0b4b2cf911ae776dd007c36faf0d5, offset: 0, size: 157 2024.10.22 00:36:35.770136 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5ed0b4b2cf911ae776dd007c36faf0d5 for removal 2024.10.22 00:36:35.774116 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0d8e4002c5292eb9a8d065e1a06bd5eb, offset: 0, size: 1398 2024.10.22 00:36:35.776113 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0d8e4002c5292eb9a8d065e1a06bd5eb for removal 2024.10.22 00:36:35.777613 [ 1181 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:36:35.779232 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fa6bde06c34f5aef76b620656f60e8ac, offset: 0, size: 374859 2024.10.22 00:36:35.782688 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fa6bde06c34f5aef76b620656f60e8ac for removal 2024.10.22 00:36:35.786657 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9dd8aaa57b129777abf32c20bccfd265, offset: 0, size: 41177 2024.10.22 00:36:35.790618 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9dd8aaa57b129777abf32c20bccfd265 for removal 2024.10.22 00:36:35.792768 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 558463bad830da2e5ccc252a4bc8dea4, offset: 0, size: 241971 2024.10.22 00:36:35.795065 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 558463bad830da2e5ccc252a4bc8dea4 for removal 2024.10.22 00:36:35.798487 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 304b57f3c58b84585f897b26087a25a7, offset: 0, size: 146 2024.10.22 00:36:35.804466 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 304b57f3c58b84585f897b26087a25a7 for removal 2024.10.22 00:36:35.806858 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 41003ab9cbe0f128eb83504d500775ec, offset: 0, size: 96926 2024.10.22 00:36:35.809424 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_25_25_0 to 202410_28_28_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:35.817038 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 41003ab9cbe0f128eb83504d500775ec for removal 2024.10.22 00:36:35.823130 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 460d48265228ee338e235edc45d0b904, offset: 0, size: 167 2024.10.22 00:36:35.828378 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 460d48265228ee338e235edc45d0b904 for removal 2024.10.22 00:36:35.833622 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2ab20dc407cdc18f059fbabebd394c9e, offset: 0, size: 144 2024.10.22 00:36:35.839352 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2ab20dc407cdc18f059fbabebd394c9e for removal 2024.10.22 00:36:35.844008 [ 1476 ] {} CacheMetadata: Key 2ab20dc407cdc18f059fbabebd394c9e is removed from metadata 2024.10.22 00:36:35.843853 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7aea37b2189e2faca1a7a013f21144f9, offset: 0, size: 141 2024.10.22 00:36:35.851678 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7aea37b2189e2faca1a7a013f21144f9 for removal 2024.10.22 00:36:35.856342 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7f884cdddeced0a4f845c90f39325c9c, offset: 0, size: 149 2024.10.22 00:36:35.859308 [ 1476 ] {} CacheMetadata: Key 7aea37b2189e2faca1a7a013f21144f9 is removed from metadata 2024.10.22 00:36:35.862347 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7f884cdddeced0a4f845c90f39325c9c for removal 2024.10.22 00:36:35.867082 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 220d8d6646ee10479076c752d0da09f4, offset: 0, size: 145 2024.10.22 00:36:35.867561 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.25 GiB. 2024.10.22 00:36:35.870739 [ 1476 ] {} CacheMetadata: Key 460d48265228ee338e235edc45d0b904 is removed from metadata 2024.10.22 00:36:35.874284 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 220d8d6646ee10479076c752d0da09f4 for removal 2024.10.22 00:36:35.877144 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ab4c8e90e015fe78a027f2fb033fd59e, offset: 0, size: 52865 2024.10.22 00:36:35.878133 [ 1476 ] {} CacheMetadata: Key 41003ab9cbe0f128eb83504d500775ec is removed from metadata 2024.10.22 00:36:35.879231 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ab4c8e90e015fe78a027f2fb033fd59e for removal 2024.10.22 00:36:35.880529 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 279199f6b0e3c85acfa9d30526dbb6aa, offset: 0, size: 144 2024.10.22 00:36:35.882679 [ 1476 ] {} CacheMetadata: Key ab4c8e90e015fe78a027f2fb033fd59e is removed from metadata 2024.10.22 00:36:35.883595 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 279199f6b0e3c85acfa9d30526dbb6aa for removal 2024.10.22 00:36:35.885778 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c736518574b4355ee1196af75ddb4a98, offset: 0, size: 142463 2024.10.22 00:36:35.888215 [ 1476 ] {} CacheMetadata: Key 304b57f3c58b84585f897b26087a25a7 is removed from metadata 2024.10.22 00:36:35.889178 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c736518574b4355ee1196af75ddb4a98 for removal 2024.10.22 00:36:35.899149 [ 1476 ] {} CacheMetadata: Key 558463bad830da2e5ccc252a4bc8dea4 is removed from metadata 2024.10.22 00:36:35.899776 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4c2396147e2264c003ed216f205d20ed, offset: 0, size: 128520 2024.10.22 00:36:35.908191 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4c2396147e2264c003ed216f205d20ed for removal 2024.10.22 00:36:35.923753 [ 1476 ] {} CacheMetadata: Key c736518574b4355ee1196af75ddb4a98 is removed from metadata 2024.10.22 00:36:35.925114 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 48ff1ae2751fa2afdcfe43bf7f999537, offset: 0, size: 148 2024.10.22 00:36:35.929543 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 48ff1ae2751fa2afdcfe43bf7f999537 for removal 2024.10.22 00:36:35.934871 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d76f1f83fbbf29b516e0aff04ab02b7d, offset: 0, size: 630854 2024.10.22 00:36:35.940851 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d76f1f83fbbf29b516e0aff04ab02b7d for removal 2024.10.22 00:36:35.949173 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8cec21a4660928f2cbaeeef7b755b990, offset: 0, size: 144 2024.10.22 00:36:35.949360 [ 1476 ] {} CacheMetadata: Key 48ff1ae2751fa2afdcfe43bf7f999537 is removed from metadata 2024.10.22 00:36:35.963312 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8cec21a4660928f2cbaeeef7b755b990 for removal 2024.10.22 00:36:35.965488 [ 1476 ] {} CacheMetadata: Key d76f1f83fbbf29b516e0aff04ab02b7d is removed from metadata 2024.10.22 00:36:35.967749 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 01ab98fd53c2bdcb0cc9d590428233db, offset: 0, size: 294992 2024.10.22 00:36:35.972060 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 01ab98fd53c2bdcb0cc9d590428233db for removal 2024.10.22 00:36:35.974822 [ 1476 ] {} CacheMetadata: Key 8cec21a4660928f2cbaeeef7b755b990 is removed from metadata 2024.10.22 00:36:35.976339 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 98a4ea20ed95ced92c72e1718b91ae0b, offset: 0, size: 113591 2024.10.22 00:36:35.979962 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 98a4ea20ed95ced92c72e1718b91ae0b for removal 2024.10.22 00:36:35.985797 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: db878e09fcbabb5c34a32a1baff24216, offset: 0, size: 145 2024.10.22 00:36:35.993738 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key db878e09fcbabb5c34a32a1baff24216 for removal 2024.10.22 00:36:35.998192 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 02d874f8b615240dc0051eefb3fcadef, offset: 0, size: 144 2024.10.22 00:36:36.000098 [ 1476 ] {} CacheMetadata: Key db878e09fcbabb5c34a32a1baff24216 is removed from metadata 2024.10.22 00:36:36.004830 [ 1476 ] {} CacheMetadata: Key 98a4ea20ed95ced92c72e1718b91ae0b is removed from metadata 2024.10.22 00:36:36.013292 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 02d874f8b615240dc0051eefb3fcadef for removal 2024.10.22 00:36:36.019381 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dabf7d8cb5c521764494a837567ca612, offset: 0, size: 102009 2024.10.22 00:36:36.019756 [ 1476 ] {} CacheMetadata: Key 01ab98fd53c2bdcb0cc9d590428233db is removed from metadata 2024.10.22 00:36:36.022610 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dabf7d8cb5c521764494a837567ca612 for removal 2024.10.22 00:36:36.026636 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7885c84e942ff0ae202079643f98e3dd, offset: 0, size: 85895 2024.10.22 00:36:36.029121 [ 1476 ] {} CacheMetadata: Key dabf7d8cb5c521764494a837567ca612 is removed from metadata 2024.10.22 00:36:36.029453 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7885c84e942ff0ae202079643f98e3dd for removal 2024.10.22 00:36:36.035428 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4df81b2847ae846e51c28f07b34db717, offset: 0, size: 145 2024.10.22 00:36:36.040232 [ 1476 ] {} CacheMetadata: Key 7885c84e942ff0ae202079643f98e3dd is removed from metadata 2024.10.22 00:36:36.038611 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4df81b2847ae846e51c28f07b34db717 for removal 2024.10.22 00:36:36.051025 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 04fd9f740ef4df21de2876daa5c22b06, offset: 0, size: 143 2024.10.22 00:36:36.054928 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 04fd9f740ef4df21de2876daa5c22b06 for removal 2024.10.22 00:36:36.056348 [ 1476 ] {} CacheMetadata: Key 4df81b2847ae846e51c28f07b34db717 is removed from metadata 2024.10.22 00:36:36.059247 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a1fd6f5fbcfe9025fdbaa56860b1b8ec, offset: 0, size: 145 2024.10.22 00:36:36.070492 [ 1476 ] {} CacheMetadata: Key 04fd9f740ef4df21de2876daa5c22b06 is removed from metadata 2024.10.22 00:36:36.086980 [ 1476 ] {} CacheMetadata: Key 02d874f8b615240dc0051eefb3fcadef is removed from metadata 2024.10.22 00:36:36.093062 [ 1476 ] {} CacheMetadata: Key 4c2396147e2264c003ed216f205d20ed is removed from metadata 2024.10.22 00:36:36.103243 [ 1476 ] {} CacheMetadata: Key 7f884cdddeced0a4f845c90f39325c9c is removed from metadata 2024.10.22 00:36:36.104898 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_100_100_0 (state Temporary) 2024.10.22 00:36:36.122063 [ 1476 ] {} CacheMetadata: Key fa6bde06c34f5aef76b620656f60e8ac is removed from metadata 2024.10.22 00:36:36.131982 [ 1476 ] {} CacheMetadata: Key 279199f6b0e3c85acfa9d30526dbb6aa is removed from metadata 2024.10.22 00:36:36.142674 [ 1476 ] {} CacheMetadata: Key 9dd8aaa57b129777abf32c20bccfd265 is removed from metadata 2024.10.22 00:36:36.154648 [ 1476 ] {} CacheMetadata: Key 5ed0b4b2cf911ae776dd007c36faf0d5 is removed from metadata 2024.10.22 00:36:36.165262 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a1fd6f5fbcfe9025fdbaa56860b1b8ec for removal 2024.10.22 00:36:36.167798 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ccaf31aae3dc4cf8f209a2204aaebc66, offset: 0, size: 49588 2024.10.22 00:36:36.171157 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ccaf31aae3dc4cf8f209a2204aaebc66 for removal 2024.10.22 00:36:36.171433 [ 1476 ] {} CacheMetadata: Key a1fd6f5fbcfe9025fdbaa56860b1b8ec is removed from metadata 2024.10.22 00:36:36.172886 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0cd3d3fea4aa735761f376a56708d953, offset: 0, size: 144 2024.10.22 00:36:36.175449 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0cd3d3fea4aa735761f376a56708d953 for removal 2024.10.22 00:36:36.178879 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 54be4bf2f59347974d8e258ae578fa3a, offset: 0, size: 153 2024.10.22 00:36:36.179470 [ 1476 ] {} CacheMetadata: Key 0cd3d3fea4aa735761f376a56708d953 is removed from metadata 2024.10.22 00:36:36.181040 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 54be4bf2f59347974d8e258ae578fa3a for removal 2024.10.22 00:36:36.183290 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8faa078c2141926e231fa5a54785a6f1, offset: 0, size: 991871 2024.10.22 00:36:36.184936 [ 1476 ] {} CacheMetadata: Key 54be4bf2f59347974d8e258ae578fa3a is removed from metadata 2024.10.22 00:36:36.192937 [ 1476 ] {} CacheMetadata: Key ccaf31aae3dc4cf8f209a2204aaebc66 is removed from metadata 2024.10.22 00:36:36.199916 [ 1476 ] {} CacheMetadata: Key 0d8e4002c5292eb9a8d065e1a06bd5eb is removed from metadata 2024.10.22 00:36:36.214908 [ 1476 ] {} CacheMetadata: Key e057cc5b7f077e194aaa2495a15dffee is removed from metadata 2024.10.22 00:36:36.286730 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8faa078c2141926e231fa5a54785a6f1 for removal 2024.10.22 00:36:36.291784 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5282c3ea45b24bc119ee32eb059afd9a, offset: 0, size: 146 2024.10.22 00:36:36.293726 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5282c3ea45b24bc119ee32eb059afd9a for removal 2024.10.22 00:36:36.295892 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fbc6721e732f5798b37758f82080ca22, offset: 0, size: 144 2024.10.22 00:36:36.298974 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fbc6721e732f5798b37758f82080ca22 for removal 2024.10.22 00:36:36.304554 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8119c813ce8417a0bd4e5de8f132c462, offset: 0, size: 143 2024.10.22 00:36:36.311498 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8119c813ce8417a0bd4e5de8f132c462 for removal 2024.10.22 00:36:36.327375 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f7cdac981da430470d6cd5def3fccf71, offset: 0, size: 67615 2024.10.22 00:36:36.331701 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f7cdac981da430470d6cd5def3fccf71 for removal 2024.10.22 00:36:36.331441 [ 1476 ] {} CacheMetadata: Key 8119c813ce8417a0bd4e5de8f132c462 is removed from metadata 2024.10.22 00:36:36.336879 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 60d00f63d84bdcec3f18880e0716d2a5, offset: 0, size: 240498 2024.10.22 00:36:36.342778 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 60d00f63d84bdcec3f18880e0716d2a5 for removal 2024.10.22 00:36:36.352558 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2d0e228487db13df3f7d84d078e6c2d3, offset: 0, size: 155 2024.10.22 00:36:36.363644 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2d0e228487db13df3f7d84d078e6c2d3 for removal 2024.10.22 00:36:36.367125 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 56dd685aef6996c1c0afab83158cc7cd, offset: 0, size: 148437 2024.10.22 00:36:36.370682 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 56dd685aef6996c1c0afab83158cc7cd for removal 2024.10.22 00:36:36.372293 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 118c6b6c6fe1d1fe5673c6e36aad233e, offset: 0, size: 155 2024.10.22 00:36:36.364010 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} MergeTask::MergeProjectionsStage: Merge sorted 8377 rows, containing 13 columns (13 merged, 0 gathered) in 1.684192182 sec., 4973.897925385334 rows/sec., 2.08 MiB/sec. 2024.10.22 00:36:36.378249 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 118c6b6c6fe1d1fe5673c6e36aad233e for removal 2024.10.22 00:36:36.383521 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: afc7bcf377dc6a00c185824c2af5e149, offset: 0, size: 129499 2024.10.22 00:36:36.386735 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key afc7bcf377dc6a00c185824c2af5e149 for removal 2024.10.22 00:36:36.393004 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ca90495c2f9ae56da9a712cdcbf66f45, offset: 0, size: 154 2024.10.22 00:36:36.395157 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_100_100_0 to 202410_110_110_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:36.395943 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ca90495c2f9ae56da9a712cdcbf66f45 for removal 2024.10.22 00:36:36.399468 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e2108e05d8ec6dc89ef43240aeef2bbd, offset: 0, size: 168 2024.10.22 00:36:36.402850 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} MergedBlockOutputStream: filled checksums 202410_1_130_26 (state Temporary) 2024.10.22 00:36:36.402819 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e2108e05d8ec6dc89ef43240aeef2bbd for removal 2024.10.22 00:36:36.406292 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0bf670981897a9218760c0419a827bc5, offset: 0, size: 2280 2024.10.22 00:36:36.413444 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0bf670981897a9218760c0419a827bc5 for removal 2024.10.22 00:36:36.426361 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ae658320cf1c80734c8daab46a47132f, offset: 0, size: 165762 2024.10.22 00:36:36.431060 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ae658320cf1c80734c8daab46a47132f for removal 2024.10.22 00:36:36.435374 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c26abd138c010bf08fec724d334d7f1a, offset: 0, size: 241889 2024.10.22 00:36:36.439795 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c26abd138c010bf08fec724d334d7f1a for removal 2024.10.22 00:36:36.448070 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ed858810746b6368e3a427280f735fe7, offset: 0, size: 146 2024.10.22 00:36:36.449209 [ 1476 ] {} CacheMetadata: Key ae658320cf1c80734c8daab46a47132f is removed from metadata 2024.10.22 00:36:36.454576 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ed858810746b6368e3a427280f735fe7 for removal 2024.10.22 00:36:36.461533 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: aa3ab49ad90361a41905b210fe2d4803, offset: 0, size: 155 2024.10.22 00:36:36.467580 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key aa3ab49ad90361a41905b210fe2d4803 for removal 2024.10.22 00:36:36.473150 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 07c51dbdafaef1b94d533ae7bc9b1983, offset: 0, size: 67105 2024.10.22 00:36:36.473676 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 8 2024.10.22 00:36:36.485654 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 07c51dbdafaef1b94d533ae7bc9b1983 for removal 2024.10.22 00:36:36.485938 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 118 2024.10.22 00:36:36.478809 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_130_26 to 202410_1_130_26 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:36.490200 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cb2df8b3c7ac2865c8d62ea8bdf82c6e, offset: 0, size: 105399 2024.10.22 00:36:36.491847 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_130_26} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_125_25, 202410_130_130_0] -> 202410_1_130_26 2024.10.22 00:36:36.492484 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cb2df8b3c7ac2865c8d62ea8bdf82c6e for removal 2024.10.22 00:36:36.495728 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c5c1f5c60a50a288d72cb017906af6e8, offset: 0, size: 157 2024.10.22 00:36:36.499523 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c5c1f5c60a50a288d72cb017906af6e8 for removal 2024.10.22 00:36:36.503510 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 37f9a13b222096474ef122aae3d3206d, offset: 0, size: 32575 2024.10.22 00:36:36.531651 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.91 MiB. 2024.10.22 00:36:36.566872 [ 1476 ] {} CacheMetadata: Key cb2df8b3c7ac2865c8d62ea8bdf82c6e is removed from metadata 2024.10.22 00:36:36.575996 [ 1476 ] {} CacheMetadata: Key aa3ab49ad90361a41905b210fe2d4803 is removed from metadata 2024.10.22 00:36:36.583809 [ 1476 ] {} CacheMetadata: Key ed858810746b6368e3a427280f735fe7 is removed from metadata 2024.10.22 00:36:36.591837 [ 1476 ] {} CacheMetadata: Key 0bf670981897a9218760c0419a827bc5 is removed from metadata 2024.10.22 00:36:36.600140 [ 1476 ] {} CacheMetadata: Key e2108e05d8ec6dc89ef43240aeef2bbd is removed from metadata 2024.10.22 00:36:36.609421 [ 1476 ] {} CacheMetadata: Key afc7bcf377dc6a00c185824c2af5e149 is removed from metadata 2024.10.22 00:36:36.614301 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 37f9a13b222096474ef122aae3d3206d for removal 2024.10.22 00:36:36.619633 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1490d33cea4fe77403bacacdc40455b8, offset: 0, size: 148 2024.10.22 00:36:36.623469 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1490d33cea4fe77403bacacdc40455b8 for removal 2024.10.22 00:36:36.627046 [ 1476 ] {} CacheMetadata: Key 37f9a13b222096474ef122aae3d3206d is removed from metadata 2024.10.22 00:36:36.628420 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6ef345f703616ebbd887995304b0f7ce, offset: 0, size: 133936 2024.10.22 00:36:36.639830 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6ef345f703616ebbd887995304b0f7ce for removal 2024.10.22 00:36:36.641762 [ 1476 ] {} CacheMetadata: Key 1490d33cea4fe77403bacacdc40455b8 is removed from metadata 2024.10.22 00:36:36.647539 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5310bd19c1b3f4646cd6de013d93d08b, offset: 0, size: 2023 2024.10.22 00:36:36.654338 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5310bd19c1b3f4646cd6de013d93d08b for removal 2024.10.22 00:36:36.659305 [ 1476 ] {} CacheMetadata: Key 5310bd19c1b3f4646cd6de013d93d08b is removed from metadata 2024.10.22 00:36:36.661233 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 98904e930e23201c412b7bcff8bca02b, offset: 0, size: 1077818 2024.10.22 00:36:36.668809 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 98904e930e23201c412b7bcff8bca02b for removal 2024.10.22 00:36:36.670795 [ 1476 ] {} CacheMetadata: Key c26abd138c010bf08fec724d334d7f1a is removed from metadata 2024.10.22 00:36:36.673011 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 64016dcc6199130f42751956a438e09c, offset: 0, size: 97323 2024.10.22 00:36:36.678234 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 64016dcc6199130f42751956a438e09c for removal 2024.10.22 00:36:36.680456 [ 1476 ] {} CacheMetadata: Key 64016dcc6199130f42751956a438e09c is removed from metadata 2024.10.22 00:36:36.681125 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7fe058087d696a15c4c9ff365f00e364, offset: 0, size: 144 2024.10.22 00:36:36.684067 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7fe058087d696a15c4c9ff365f00e364 for removal 2024.10.22 00:36:36.692766 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6895fb25cd18a9d1e63429b91204f5a6, offset: 0, size: 230 2024.10.22 00:36:36.694083 [ 1476 ] {} CacheMetadata: Key 7fe058087d696a15c4c9ff365f00e364 is removed from metadata 2024.10.22 00:36:36.698762 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6895fb25cd18a9d1e63429b91204f5a6 for removal 2024.10.22 00:36:36.701549 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8006c8dcaeb62726d6315e59079a4ca9, offset: 0, size: 74074 2024.10.22 00:36:36.704508 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8006c8dcaeb62726d6315e59079a4ca9 for removal 2024.10.22 00:36:36.709091 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 851b0541715c02056069a49e30fdab98, offset: 0, size: 4544 2024.10.22 00:36:36.711182 [ 1476 ] {} CacheMetadata: Key 6895fb25cd18a9d1e63429b91204f5a6 is removed from metadata 2024.10.22 00:36:36.720718 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 851b0541715c02056069a49e30fdab98 for removal 2024.10.22 00:36:36.724019 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4ca9bf1a9928401692d9fbbf458d3b73, offset: 0, size: 84855 2024.10.22 00:36:36.724575 [ 1476 ] {} CacheMetadata: Key 8006c8dcaeb62726d6315e59079a4ca9 is removed from metadata 2024.10.22 00:36:36.726217 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4ca9bf1a9928401692d9fbbf458d3b73 for removal 2024.10.22 00:36:36.728244 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c683867e79f87637c5e6a83448c3f178, offset: 0, size: 988861 2024.10.22 00:36:36.733970 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c683867e79f87637c5e6a83448c3f178 for removal 2024.10.22 00:36:36.734989 [ 1476 ] {} CacheMetadata: Key 4ca9bf1a9928401692d9fbbf458d3b73 is removed from metadata 2024.10.22 00:36:36.740262 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 55ecb8fd7f1f389eb9f906e2738a6609, offset: 0, size: 57409 2024.10.22 00:36:36.747963 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 55ecb8fd7f1f389eb9f906e2738a6609 for removal 2024.10.22 00:36:36.754413 [ 1476 ] {} CacheMetadata: Key c683867e79f87637c5e6a83448c3f178 is removed from metadata 2024.10.22 00:36:36.758218 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 57fc617c2e5407271474a62c77dff6d5, offset: 0, size: 155 2024.10.22 00:36:36.765207 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 57fc617c2e5407271474a62c77dff6d5 for removal 2024.10.22 00:36:36.766945 [ 1476 ] {} CacheMetadata: Key 55ecb8fd7f1f389eb9f906e2738a6609 is removed from metadata 2024.10.22 00:36:36.769308 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9778b089363d35e46280affc82d47e7d, offset: 0, size: 148 2024.10.22 00:36:36.772229 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9778b089363d35e46280affc82d47e7d for removal 2024.10.22 00:36:36.773583 [ 1476 ] {} CacheMetadata: Key 98904e930e23201c412b7bcff8bca02b is removed from metadata 2024.10.22 00:36:36.775261 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a1c1f39dd6d02cc271541fd8b664786f, offset: 0, size: 135291 2024.10.22 00:36:36.777634 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a1c1f39dd6d02cc271541fd8b664786f for removal 2024.10.22 00:36:36.780137 [ 1476 ] {} CacheMetadata: Key 9778b089363d35e46280affc82d47e7d is removed from metadata 2024.10.22 00:36:36.782220 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5801da8f4ac87d874da4ac7788b0c437, offset: 0, size: 81896 2024.10.22 00:36:36.786885 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5801da8f4ac87d874da4ac7788b0c437 for removal 2024.10.22 00:36:36.788779 [ 1476 ] {} CacheMetadata: Key 2d0e228487db13df3f7d84d078e6c2d3 is removed from metadata 2024.10.22 00:36:36.790707 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bb9d20c88207e72c977f167fcf98ef25, offset: 0, size: 144 2024.10.22 00:36:36.793199 [ 1476 ] {} CacheMetadata: Key 5801da8f4ac87d874da4ac7788b0c437 is removed from metadata 2024.10.22 00:36:36.793182 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bb9d20c88207e72c977f167fcf98ef25 for removal 2024.10.22 00:36:36.795112 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 49dec8a80f1b4a52fe87d4c78bde5a3a, offset: 0, size: 146 2024.10.22 00:36:36.797533 [ 1476 ] {} CacheMetadata: Key a1c1f39dd6d02cc271541fd8b664786f is removed from metadata 2024.10.22 00:36:36.797782 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 49dec8a80f1b4a52fe87d4c78bde5a3a for removal 2024.10.22 00:36:36.801363 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 581ca397527a121a649d2ee57dd26045, offset: 0, size: 286123 2024.10.22 00:36:36.807005 [ 1476 ] {} CacheMetadata: Key 49dec8a80f1b4a52fe87d4c78bde5a3a is removed from metadata 2024.10.22 00:36:36.819279 [ 1476 ] {} CacheMetadata: Key 851b0541715c02056069a49e30fdab98 is removed from metadata 2024.10.22 00:36:36.827182 [ 1476 ] {} CacheMetadata: Key 60d00f63d84bdcec3f18880e0716d2a5 is removed from metadata 2024.10.22 00:36:36.837513 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 581ca397527a121a649d2ee57dd26045 for removal 2024.10.22 00:36:36.838915 [ 1476 ] {} CacheMetadata: Key bb9d20c88207e72c977f167fcf98ef25 is removed from metadata 2024.10.22 00:36:36.847135 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: afae7ee4287ca136a5d64c56e93d5a04, offset: 0, size: 1560 2024.10.22 00:36:36.853015 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key afae7ee4287ca136a5d64c56e93d5a04 for removal 2024.10.22 00:36:36.856544 [ 1476 ] {} CacheMetadata: Key 581ca397527a121a649d2ee57dd26045 is removed from metadata 2024.10.22 00:36:36.859200 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9c8e172042c858a6d0298c0dc9ea587a, offset: 0, size: 144 2024.10.22 00:36:36.868383 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9c8e172042c858a6d0298c0dc9ea587a for removal 2024.10.22 00:36:36.871314 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d68501183abb1dfe478ccdc3d5be8a9f, offset: 0, size: 257111 2024.10.22 00:36:36.871640 [ 1476 ] {} CacheMetadata: Key 9c8e172042c858a6d0298c0dc9ea587a is removed from metadata 2024.10.22 00:36:36.875466 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d68501183abb1dfe478ccdc3d5be8a9f for removal 2024.10.22 00:36:36.879288 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cd35140fbfb484a979add836241d156d, offset: 0, size: 168 2024.10.22 00:36:36.882453 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cd35140fbfb484a979add836241d156d for removal 2024.10.22 00:36:36.882689 [ 1476 ] {} CacheMetadata: Key afae7ee4287ca136a5d64c56e93d5a04 is removed from metadata 2024.10.22 00:36:36.885802 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3d9f96a28aa3b3cc5df7d34bac11aa53, offset: 0, size: 157 2024.10.22 00:36:36.889703 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3d9f96a28aa3b3cc5df7d34bac11aa53 for removal 2024.10.22 00:36:36.890436 [ 1476 ] {} CacheMetadata: Key c5c1f5c60a50a288d72cb017906af6e8 is removed from metadata 2024.10.22 00:36:36.892947 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e8c300a23d2342df2d3f2779b3229910, offset: 0, size: 988825 2024.10.22 00:36:36.909314 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e8c300a23d2342df2d3f2779b3229910 for removal 2024.10.22 00:36:36.915173 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3beb98e8ba6fbdce56e127aed14be91e, offset: 0, size: 145 2024.10.22 00:36:36.916140 [ 1476 ] {} CacheMetadata: Key e8c300a23d2342df2d3f2779b3229910 is removed from metadata 2024.10.22 00:36:36.921309 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3beb98e8ba6fbdce56e127aed14be91e for removal 2024.10.22 00:36:36.924496 [ 1476 ] {} CacheMetadata: Key 3d9f96a28aa3b3cc5df7d34bac11aa53 is removed from metadata 2024.10.22 00:36:36.924704 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 50d077b0288e1574fbef1f7ed8927b65, offset: 0, size: 144 2024.10.22 00:36:36.931653 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 50d077b0288e1574fbef1f7ed8927b65 for removal 2024.10.22 00:36:36.935074 [ 1476 ] {} CacheMetadata: Key f7cdac981da430470d6cd5def3fccf71 is removed from metadata 2024.10.22 00:36:36.938798 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 987d1aaddb3cd23b0b81b0b236485e92, offset: 0, size: 181184 2024.10.22 00:36:36.946331 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 987d1aaddb3cd23b0b81b0b236485e92 for removal 2024.10.22 00:36:36.954813 [ 1476 ] {} CacheMetadata: Key 987d1aaddb3cd23b0b81b0b236485e92 is removed from metadata 2024.10.22 00:36:36.962790 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a728087837d74a677cfbcd71bb87cabe, offset: 0, size: 157 2024.10.22 00:36:36.971155 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a728087837d74a677cfbcd71bb87cabe for removal 2024.10.22 00:36:36.971704 [ 1476 ] {} CacheMetadata: Key 50d077b0288e1574fbef1f7ed8927b65 is removed from metadata 2024.10.22 00:36:36.975671 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 34f2c966d6a4d025c0cf8def51e1015f, offset: 0, size: 171 2024.10.22 00:36:36.978575 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 34f2c966d6a4d025c0cf8def51e1015f for removal 2024.10.22 00:36:36.979451 [ 1476 ] {} CacheMetadata: Key a728087837d74a677cfbcd71bb87cabe is removed from metadata 2024.10.22 00:36:36.982534 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 59e6cbd923906be6407d5e242565385a, offset: 0, size: 421058 2024.10.22 00:36:36.986922 [ 1476 ] {} CacheMetadata: Key 34f2c966d6a4d025c0cf8def51e1015f is removed from metadata 2024.10.22 00:36:36.989568 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 59e6cbd923906be6407d5e242565385a for removal 2024.10.22 00:36:36.994964 [ 1476 ] {} CacheMetadata: Key 59e6cbd923906be6407d5e242565385a is removed from metadata 2024.10.22 00:36:36.995004 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 385264a5ab27ae2ff27f66816b5fb171, offset: 0, size: 171 2024.10.22 00:36:37.003382 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 385264a5ab27ae2ff27f66816b5fb171 for removal 2024.10.22 00:36:37.016754 [ 1476 ] {} CacheMetadata: Key 385264a5ab27ae2ff27f66816b5fb171 is removed from metadata 2024.10.22 00:36:37.018549 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c5bfa3c343ae8f3aa60657ef001a0c3f, offset: 0, size: 1136 2024.10.22 00:36:37.022481 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c5bfa3c343ae8f3aa60657ef001a0c3f for removal 2024.10.22 00:36:37.025240 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e2ae4f5304941af7ce85e92d80fa280b, offset: 0, size: 144 2024.10.22 00:36:37.030908 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:37.032573 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e2ae4f5304941af7ce85e92d80fa280b for removal 2024.10.22 00:36:37.035944 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.26 GiB. 2024.10.22 00:36:37.037799 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cf6d52bbf538fcc704737ee01f457453, offset: 0, size: 145 2024.10.22 00:36:37.050659 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cf6d52bbf538fcc704737ee01f457453 for removal 2024.10.22 00:36:37.060638 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: db8098569d9f902cd4de41dd98372e15, offset: 0, size: 422951 2024.10.22 00:36:37.068426 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key db8098569d9f902cd4de41dd98372e15 for removal 2024.10.22 00:36:37.071480 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5ee04007081134ac3aaffebef5f7241d, offset: 0, size: 148 2024.10.22 00:36:37.077153 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5ee04007081134ac3aaffebef5f7241d for removal 2024.10.22 00:36:37.079956 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5d370f50f58d93d34e401297c7e406a0, offset: 0, size: 167 2024.10.22 00:36:37.082847 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5d370f50f58d93d34e401297c7e406a0 for removal 2024.10.22 00:36:37.085140 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c9a852725da7e70526e430d56d131946, offset: 0, size: 178 2024.10.22 00:36:37.088931 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c9a852725da7e70526e430d56d131946 for removal 2024.10.22 00:36:37.091256 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: beb918b5289b724e5b43d12aa16301ab, offset: 0, size: 101903 2024.10.22 00:36:37.094090 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key beb918b5289b724e5b43d12aa16301ab for removal 2024.10.22 00:36:37.100465 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f3fd6e1fde468e45c06960d3ff56c159, offset: 0, size: 156 2024.10.22 00:36:37.105660 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f3fd6e1fde468e45c06960d3ff56c159 for removal 2024.10.22 00:36:37.114285 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 66072ccedcd38aa12c0eac551700a036, offset: 0, size: 176 2024.10.22 00:36:37.118230 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 66072ccedcd38aa12c0eac551700a036 for removal 2024.10.22 00:36:37.122979 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 14138e4b43213a18d952a78b2a8f09d4, offset: 0, size: 148 2024.10.22 00:36:37.025750 [ 1476 ] {} CacheMetadata: Key c5bfa3c343ae8f3aa60657ef001a0c3f is removed from metadata 2024.10.22 00:36:37.126878 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 14138e4b43213a18d952a78b2a8f09d4 for removal 2024.10.22 00:36:37.132904 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6afd6758ae5a961a3b6a9671a43455a9, offset: 0, size: 155 2024.10.22 00:36:37.137516 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6afd6758ae5a961a3b6a9671a43455a9 for removal 2024.10.22 00:36:37.144988 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1a06ad2f2624606fad908fc24c396e87, offset: 0, size: 217 2024.10.22 00:36:37.150711 [ 1476 ] {} CacheMetadata: Key 6afd6758ae5a961a3b6a9671a43455a9 is removed from metadata 2024.10.22 00:36:37.150828 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1a06ad2f2624606fad908fc24c396e87 for removal 2024.10.22 00:36:37.154728 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f97bfebb3f0c34a75c2bee331a19528b, offset: 0, size: 47062 2024.10.22 00:36:37.161526 [ 1476 ] {} CacheMetadata: Key 66072ccedcd38aa12c0eac551700a036 is removed from metadata 2024.10.22 00:36:37.166574 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f97bfebb3f0c34a75c2bee331a19528b for removal 2024.10.22 00:36:37.171560 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6dd29ea79d2540fe95146fb4a93e0a0e, offset: 0, size: 1964477 2024.10.22 00:36:37.176840 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6dd29ea79d2540fe95146fb4a93e0a0e for removal 2024.10.22 00:36:37.181104 [ 1476 ] {} CacheMetadata: Key f97bfebb3f0c34a75c2bee331a19528b is removed from metadata 2024.10.22 00:36:37.187723 [ 1476 ] {} CacheMetadata: Key 6dd29ea79d2540fe95146fb4a93e0a0e is removed from metadata 2024.10.22 00:36:37.189087 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_8_8_0 (state Temporary) 2024.10.22 00:36:37.196109 [ 1476 ] {} CacheMetadata: Key f3fd6e1fde468e45c06960d3ff56c159 is removed from metadata 2024.10.22 00:36:37.211611 [ 1476 ] {} CacheMetadata: Key beb918b5289b724e5b43d12aa16301ab is removed from metadata 2024.10.22 00:36:37.236015 [ 1476 ] {} CacheMetadata: Key 3beb98e8ba6fbdce56e127aed14be91e is removed from metadata 2024.10.22 00:36:37.282884 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e504219684cb5a6a9c95f2aad03733f4, offset: 0, size: 235561 2024.10.22 00:36:37.286563 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e504219684cb5a6a9c95f2aad03733f4 for removal 2024.10.22 00:36:37.289507 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8333879f2c21b9aac19ac794112fd3ce, offset: 0, size: 61979 2024.10.22 00:36:37.293281 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8333879f2c21b9aac19ac794112fd3ce for removal 2024.10.22 00:36:37.298615 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0f338ebde14743e99aa5616252cedb9d, offset: 0, size: 144 2024.10.22 00:36:37.302842 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0f338ebde14743e99aa5616252cedb9d for removal 2024.10.22 00:36:37.306606 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bf2e207f66620149078545b8dec58e5e, offset: 0, size: 229721 2024.10.22 00:36:37.315703 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bf2e207f66620149078545b8dec58e5e for removal 2024.10.22 00:36:37.319228 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8ced85c12b6c75f2c859b60cb4fb5862, offset: 0, size: 241004 2024.10.22 00:36:37.322129 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8ced85c12b6c75f2c859b60cb4fb5862 for removal 2024.10.22 00:36:37.352722 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_8_8_0 to 202410_8_8_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:37.426904 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e36f42878cc929b01405ead72057878d, offset: 0, size: 52747 2024.10.22 00:36:37.434846 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e36f42878cc929b01405ead72057878d for removal 2024.10.22 00:36:37.439907 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 269843704866fd490163731535cb810d, offset: 0, size: 4 2024.10.22 00:36:37.441176 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 8 2024.10.22 00:36:37.443611 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 269843704866fd490163731535cb810d for removal 2024.10.22 00:36:37.546814 [ 1476 ] {} CacheMetadata: Key 269843704866fd490163731535cb810d is removed from metadata 2024.10.22 00:36:37.552157 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 747208504427087c0e91e0d590b5707e, offset: 0, size: 313717 2024.10.22 00:36:37.558688 [ 1476 ] {} CacheMetadata: Key e36f42878cc929b01405ead72057878d is removed from metadata 2024.10.22 00:36:37.566608 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 747208504427087c0e91e0d590b5707e for removal 2024.10.22 00:36:37.572154 [ 1476 ] {} CacheMetadata: Key 747208504427087c0e91e0d590b5707e is removed from metadata 2024.10.22 00:36:37.577195 [ 1476 ] {} CacheMetadata: Key bf2e207f66620149078545b8dec58e5e is removed from metadata 2024.10.22 00:36:37.589967 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 56d8eb713dd5b02f4d70e0f440b2bc53, offset: 0, size: 144 2024.10.22 00:36:37.593647 [ 1476 ] {} CacheMetadata: Key 8ced85c12b6c75f2c859b60cb4fb5862 is removed from metadata 2024.10.22 00:36:37.596234 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 56d8eb713dd5b02f4d70e0f440b2bc53 for removal 2024.10.22 00:36:37.604301 [ 1476 ] {} CacheMetadata: Key 0f338ebde14743e99aa5616252cedb9d is removed from metadata 2024.10.22 00:36:37.620646 [ 1476 ] {} CacheMetadata: Key 8333879f2c21b9aac19ac794112fd3ce is removed from metadata 2024.10.22 00:36:37.623858 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 968c644bc26a99801e54a648bd162cb7, offset: 0, size: 763782 2024.10.22 00:36:37.627903 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 968c644bc26a99801e54a648bd162cb7 for removal 2024.10.22 00:36:37.628479 [ 1476 ] {} CacheMetadata: Key 56d8eb713dd5b02f4d70e0f440b2bc53 is removed from metadata 2024.10.22 00:36:37.630680 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 396b03b389a34194b0ef211eec3cbc76, offset: 0, size: 155 2024.10.22 00:36:37.634722 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 396b03b389a34194b0ef211eec3cbc76 for removal 2024.10.22 00:36:37.635987 [ 1476 ] {} CacheMetadata: Key 968c644bc26a99801e54a648bd162cb7 is removed from metadata 2024.10.22 00:36:37.739913 [ 1476 ] {} CacheMetadata: Key 396b03b389a34194b0ef211eec3cbc76 is removed from metadata 2024.10.22 00:36:37.742582 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 87e8dc000d5aeef7d9ffaebaa4deedd9, offset: 0, size: 145 2024.10.22 00:36:37.745829 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 87e8dc000d5aeef7d9ffaebaa4deedd9 for removal 2024.10.22 00:36:37.753726 [ 1476 ] {} CacheMetadata: Key e504219684cb5a6a9c95f2aad03733f4 is removed from metadata 2024.10.22 00:36:37.767565 [ 1476 ] {} CacheMetadata: Key c9a852725da7e70526e430d56d131946 is removed from metadata 2024.10.22 00:36:37.786525 [ 1476 ] {} CacheMetadata: Key 5ee04007081134ac3aaffebef5f7241d is removed from metadata 2024.10.22 00:36:37.794878 [ 1476 ] {} CacheMetadata: Key db8098569d9f902cd4de41dd98372e15 is removed from metadata 2024.10.22 00:36:37.814467 [ 1476 ] {} CacheMetadata: Key 14138e4b43213a18d952a78b2a8f09d4 is removed from metadata 2024.10.22 00:36:37.835566 [ 1476 ] {} CacheMetadata: Key cf6d52bbf538fcc704737ee01f457453 is removed from metadata 2024.10.22 00:36:37.853221 [ 1476 ] {} CacheMetadata: Key d68501183abb1dfe478ccdc3d5be8a9f is removed from metadata 2024.10.22 00:36:37.866551 [ 1476 ] {} CacheMetadata: Key 5d370f50f58d93d34e401297c7e406a0 is removed from metadata 2024.10.22 00:36:37.877439 [ 1476 ] {} CacheMetadata: Key e2ae4f5304941af7ce85e92d80fa280b is removed from metadata 2024.10.22 00:36:37.885512 [ 1476 ] {} CacheMetadata: Key 07c51dbdafaef1b94d533ae7bc9b1983 is removed from metadata 2024.10.22 00:36:37.891646 [ 1476 ] {} CacheMetadata: Key 1a06ad2f2624606fad908fc24c396e87 is removed from metadata 2024.10.22 00:36:37.897248 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 437 entries to flush up to offset 10893 2024.10.22 00:36:37.897017 [ 1476 ] {} CacheMetadata: Key a036232dcfebb8b8900b993e33faebe9 is removed from metadata 2024.10.22 00:36:37.909028 [ 1476 ] {} CacheMetadata: Key ca90495c2f9ae56da9a712cdcbf66f45 is removed from metadata 2024.10.22 00:36:37.919074 [ 1476 ] {} CacheMetadata: Key cd35140fbfb484a979add836241d156d is removed from metadata 2024.10.22 00:36:37.924608 [ 1476 ] {} CacheMetadata: Key fbc6721e732f5798b37758f82080ca22 is removed from metadata 2024.10.22 00:36:37.932682 [ 1476 ] {} CacheMetadata: Key 220d8d6646ee10479076c752d0da09f4 is removed from metadata 2024.10.22 00:36:37.938375 [ 1476 ] {} CacheMetadata: Key 5282c3ea45b24bc119ee32eb059afd9a is removed from metadata 2024.10.22 00:36:37.946650 [ 1476 ] {} CacheMetadata: Key 56dd685aef6996c1c0afab83158cc7cd is removed from metadata 2024.10.22 00:36:37.968088 [ 1476 ] {} CacheMetadata: Key 87e8dc000d5aeef7d9ffaebaa4deedd9 is removed from metadata 2024.10.22 00:36:37.975144 [ 1476 ] {} CacheMetadata: Key 8faa078c2141926e231fa5a54785a6f1 is removed from metadata 2024.10.22 00:36:37.979187 [ 1476 ] {} CacheMetadata: Key 57fc617c2e5407271474a62c77dff6d5 is removed from metadata 2024.10.22 00:36:37.993176 [ 1476 ] {} CacheMetadata: Key 6ef345f703616ebbd887995304b0f7ce is removed from metadata 2024.10.22 00:36:38.003220 [ 1476 ] {} CacheMetadata: Key 118c6b6c6fe1d1fe5673c6e36aad233e is removed from metadata 2024.10.22 00:36:38.004226 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/gat/lfrxnnjxbadvnbtyugoavuyywbgcb, upload id 4d6cbce2-f572-4168-98ed-b2afcb69cafc 2024.10.22 00:36:39.010545 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 982 2024.10.22 00:36:39.044860 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:39.048041 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.16 GiB. 2024.10.22 00:36:39.270593 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_217_217_0 (state Temporary) 2024.10.22 00:36:39.343590 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_217_217_0 to 202410_230_230_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:39.491727 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:36:39.505776 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:36:39.588560 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 10893 2024.10.22 00:36:39.751614 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 10.04 MiB, will set to 1.51 GiB (RSS), difference: -115.31 MiB 2024.10.22 00:36:40.007388 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 1781 2024.10.22 00:36:40.108024 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:40.113166 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.16 GiB. 2024.10.22 00:36:40.153198 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_126_126_0 (state Temporary) 2024.10.22 00:36:40.339051 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_126_126_0 to 202410_139_139_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:40.397395 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1781 2024.10.22 00:36:41.595535 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.83 GiB, free memory in arenas 10.30 MiB, will set to 1.56 GiB (RSS), difference: -12.68 MiB 2024.10.22 00:36:41.676364 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 212894 2024.10.22 00:36:42.169698 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:42.172794 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.16 GiB. 2024.10.22 00:36:42.178380 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 8323 2024.10.22 00:36:42.503565 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:42.511104 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.15 GiB. 2024.10.22 00:36:42.553089 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_121_121_0 (state Temporary) 2024.10.22 00:36:42.573841 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_125_125_0 (state Temporary) 2024.10.22 00:36:42.706747 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:36:42.716796 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_125_125_0 to 202410_131_131_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:42.774446 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8323 2024.10.22 00:36:42.783538 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_121_121_0 to 202410_128_128_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:43.079282 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 212894 2024.10.22 00:36:43.548419 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.83 GiB, free memory in arenas 10.00 MiB, will set to 1.60 GiB (RSS), difference: -44.66 MiB 2024.10.22 00:36:43.879889 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 10937 2024.10.22 00:36:43.989315 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 119 2024.10.22 00:36:44.286567 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:44.367307 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.16 GiB. 2024.10.22 00:36:44.417068 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_218_218_0 (state Temporary) 2024.10.22 00:36:44.432645 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:44.438487 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.15 GiB. 2024.10.22 00:36:44.475654 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_218_218_0 to 202410_231_231_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:44.516309 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_226_45 to 202410_231_231_0 2024.10.22 00:36:44.554753 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:44.558823 [ 1267 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.15 GiB. 2024.10.22 00:36:44.561969 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 10937 2024.10.22 00:36:44.589520 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_226_45 to 202410_231_231_0 into Compact with storage Full 2024.10.22 00:36:44.712719 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_101_101_0 (state Temporary) 2024.10.22 00:36:44.834944 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_101_101_0 to 202410_111_111_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:44.991391 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:36:44.998058 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_226_45, total 10807 rows starting from the beginning of the part 2024.10.22 00:36:45.024669 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 119 2024.10.22 00:36:45.128114 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_227_227_0, total 181 rows starting from the beginning of the part 2024.10.22 00:36:45.152351 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_228_228_0, total 244 rows starting from the beginning of the part 2024.10.22 00:36:45.179299 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_229_229_0, total 55 rows starting from the beginning of the part 2024.10.22 00:36:45.304205 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_230_230_0, total 437 rows starting from the beginning of the part 2024.10.22 00:36:45.316661 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_66_13, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0, 202410_71_71_0] 2024.10.22 00:36:45.319631 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_66_13, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0, 202410_71_71_0] 2024.10.22 00:36:45.442997 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_231_231_0, total 44 rows starting from the beginning of the part 2024.10.22 00:36:45.448914 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_66_13, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0, 202410_70_70_0, 202410_71_71_0] 2024.10.22 00:36:45.637592 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.83 GiB, free memory in arenas 9.57 MiB, will set to 1.62 GiB (RSS), difference: -51.89 MiB 2024.10.22 00:36:45.970538 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 113 file segments (while reserving 7317775 bytes for 854f10257937f71db2f581a95fb90b19:33554432) 2024.10.22 00:36:45.973018 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 781a2ebf5dedcc218e08e234158a6765, offset: 0, size: 139 2024.10.22 00:36:45.976500 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 781a2ebf5dedcc218e08e234158a6765 for removal 2024.10.22 00:36:45.979525 [ 1476 ] {} CacheMetadata: Key 781a2ebf5dedcc218e08e234158a6765 is removed from metadata 2024.10.22 00:36:45.979787 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: be94253cf23409241954aba6c89b6a27, offset: 0, size: 137 2024.10.22 00:36:45.982414 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key be94253cf23409241954aba6c89b6a27 for removal 2024.10.22 00:36:45.984253 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 037ffea62a802d61ca935830ad39f72d, offset: 0, size: 62658 2024.10.22 00:36:45.989021 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 037ffea62a802d61ca935830ad39f72d for removal 2024.10.22 00:36:45.998287 [ 1476 ] {} CacheMetadata: Key 037ffea62a802d61ca935830ad39f72d is removed from metadata 2024.10.22 00:36:46.004262 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 93b36e2cf56b647cdae46de3ff1373f0, offset: 0, size: 137 2024.10.22 00:36:46.010350 [ 1476 ] {} CacheMetadata: Key be94253cf23409241954aba6c89b6a27 is removed from metadata 2024.10.22 00:36:46.010979 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 93b36e2cf56b647cdae46de3ff1373f0 for removal 2024.10.22 00:36:46.018996 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5724a01beb75ecf01f486bd68c9d3b3b, offset: 0, size: 62294 2024.10.22 00:36:46.026042 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5724a01beb75ecf01f486bd68c9d3b3b for removal 2024.10.22 00:36:46.028752 [ 1476 ] {} CacheMetadata: Key 93b36e2cf56b647cdae46de3ff1373f0 is removed from metadata 2024.10.22 00:36:46.047495 [ 1476 ] {} CacheMetadata: Key 5724a01beb75ecf01f486bd68c9d3b3b is removed from metadata 2024.10.22 00:36:46.076906 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 272.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:46.133638 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 854ab948d321789b1d63bfcf041c1ca9, offset: 0, size: 137 2024.10.22 00:36:46.139235 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 854ab948d321789b1d63bfcf041c1ca9 for removal 2024.10.22 00:36:46.142885 [ 1476 ] {} CacheMetadata: Key 854ab948d321789b1d63bfcf041c1ca9 is removed from metadata 2024.10.22 00:36:46.143982 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 08d5a4ff6ea60b228ca8ae33c8803bd8, offset: 0, size: 139 2024.10.22 00:36:46.149129 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 08d5a4ff6ea60b228ca8ae33c8803bd8 for removal 2024.10.22 00:36:46.157007 [ 1476 ] {} CacheMetadata: Key 08d5a4ff6ea60b228ca8ae33c8803bd8 is removed from metadata 2024.10.22 00:36:46.157053 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e8a351c783f8b1b9f55bb384a8af620f, offset: 0, size: 70321 2024.10.22 00:36:46.165604 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e8a351c783f8b1b9f55bb384a8af620f for removal 2024.10.22 00:36:46.173603 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c711dd111690207cc1591a682d7b1b85, offset: 0, size: 16568 2024.10.22 00:36:46.173084 [ 1476 ] {} CacheMetadata: Key e8a351c783f8b1b9f55bb384a8af620f is removed from metadata 2024.10.22 00:36:46.177728 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c711dd111690207cc1591a682d7b1b85 for removal 2024.10.22 00:36:46.180544 [ 1476 ] {} CacheMetadata: Key c711dd111690207cc1591a682d7b1b85 is removed from metadata 2024.10.22 00:36:46.180571 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0f604eb22e087bbc07055f8d88311fb9, offset: 0, size: 139 2024.10.22 00:36:46.186293 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0f604eb22e087bbc07055f8d88311fb9 for removal 2024.10.22 00:36:46.188139 [ 1476 ] {} CacheMetadata: Key 0f604eb22e087bbc07055f8d88311fb9 is removed from metadata 2024.10.22 00:36:46.188667 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: de31cd1cb10c7e79d415d4e0fb35e8a8, offset: 0, size: 137 2024.10.22 00:36:46.190661 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key de31cd1cb10c7e79d415d4e0fb35e8a8 for removal 2024.10.22 00:36:46.193082 [ 1476 ] {} CacheMetadata: Key de31cd1cb10c7e79d415d4e0fb35e8a8 is removed from metadata 2024.10.22 00:36:46.193359 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c4b87e5719a6cd85f090c25edbd05fe7, offset: 0, size: 139 2024.10.22 00:36:46.199353 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c4b87e5719a6cd85f090c25edbd05fe7 for removal 2024.10.22 00:36:46.204669 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f5d7cf2186448486be5867b14eb0ae0c, offset: 0, size: 4672 2024.10.22 00:36:46.218907 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f5d7cf2186448486be5867b14eb0ae0c for removal 2024.10.22 00:36:46.227697 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 213c50424926d39332435310273ae096, offset: 0, size: 15909 2024.10.22 00:36:46.233044 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 213c50424926d39332435310273ae096 for removal 2024.10.22 00:36:46.243616 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c21fd79667bc44ef56f375bcfabc2e12, offset: 0, size: 73822 2024.10.22 00:36:46.249486 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c21fd79667bc44ef56f375bcfabc2e12 for removal 2024.10.22 00:36:46.263769 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 595f554298bbfd25d7c93cbab46fabcb, offset: 0, size: 139 2024.10.22 00:36:46.269294 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 595f554298bbfd25d7c93cbab46fabcb for removal 2024.10.22 00:36:46.278361 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2fffc05cae0424f8187591f599f4323b, offset: 0, size: 72482 2024.10.22 00:36:46.286685 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2fffc05cae0424f8187591f599f4323b for removal 2024.10.22 00:36:46.292407 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 70d27cdbb8296b0fc99f5369f9422476, offset: 0, size: 141 2024.10.22 00:36:46.300046 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 70d27cdbb8296b0fc99f5369f9422476 for removal 2024.10.22 00:36:46.305260 [ 1476 ] {} CacheMetadata: Key 2fffc05cae0424f8187591f599f4323b is removed from metadata 2024.10.22 00:36:46.316665 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0e6ec3c18eaab6e71acc14ab6833a086, offset: 0, size: 35839 2024.10.22 00:36:46.332868 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0e6ec3c18eaab6e71acc14ab6833a086 for removal 2024.10.22 00:36:46.427023 [ 1476 ] {} CacheMetadata: Key 70d27cdbb8296b0fc99f5369f9422476 is removed from metadata 2024.10.22 00:36:46.438985 [ 1476 ] {} CacheMetadata: Key 0e6ec3c18eaab6e71acc14ab6833a086 is removed from metadata 2024.10.22 00:36:46.438895 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0bcc259e2342d61aa7a94e549f9f8a00, offset: 0, size: 948078 2024.10.22 00:36:46.443713 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0bcc259e2342d61aa7a94e549f9f8a00 for removal 2024.10.22 00:36:46.459909 [ 1476 ] {} CacheMetadata: Key 0bcc259e2342d61aa7a94e549f9f8a00 is removed from metadata 2024.10.22 00:36:46.460776 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a6c6a725b241965ffa030aa191ab4443, offset: 0, size: 141 2024.10.22 00:36:46.465828 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a6c6a725b241965ffa030aa191ab4443 for removal 2024.10.22 00:36:46.470401 [ 1476 ] {} CacheMetadata: Key a6c6a725b241965ffa030aa191ab4443 is removed from metadata 2024.10.22 00:36:46.474885 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b740cb8bdf952b093089306894ad67bc, offset: 0, size: 34378 2024.10.22 00:36:46.482738 [ 1476 ] {} CacheMetadata: Key c21fd79667bc44ef56f375bcfabc2e12 is removed from metadata 2024.10.22 00:36:46.483744 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b740cb8bdf952b093089306894ad67bc for removal 2024.10.22 00:36:46.488647 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 339adad940e5f89e694738a239f3913f, offset: 0, size: 24609 2024.10.22 00:36:46.491704 [ 1476 ] {} CacheMetadata: Key b740cb8bdf952b093089306894ad67bc is removed from metadata 2024.10.22 00:36:46.492373 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 339adad940e5f89e694738a239f3913f for removal 2024.10.22 00:36:46.497573 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cb7c00523ecee8fd0bb11bf1acd2d27f, offset: 0, size: 141 2024.10.22 00:36:46.500172 [ 1476 ] {} CacheMetadata: Key 339adad940e5f89e694738a239f3913f is removed from metadata 2024.10.22 00:36:46.503101 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cb7c00523ecee8fd0bb11bf1acd2d27f for removal 2024.10.22 00:36:46.512582 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 710a55d59454baaf083caf452a6e31a4, offset: 0, size: 4 2024.10.22 00:36:46.513834 [ 1476 ] {} CacheMetadata: Key cb7c00523ecee8fd0bb11bf1acd2d27f is removed from metadata 2024.10.22 00:36:46.521584 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 710a55d59454baaf083caf452a6e31a4 for removal 2024.10.22 00:36:46.530814 [ 1476 ] {} CacheMetadata: Key 710a55d59454baaf083caf452a6e31a4 is removed from metadata 2024.10.22 00:36:46.531120 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e063039339d57402b8f24ef2fb1307a0, offset: 0, size: 949412 2024.10.22 00:36:46.543867 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e063039339d57402b8f24ef2fb1307a0 for removal 2024.10.22 00:36:46.552358 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7afd549b433ed4bcb207afd76875794c, offset: 0, size: 82610 2024.10.22 00:36:46.557120 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7afd549b433ed4bcb207afd76875794c for removal 2024.10.22 00:36:46.562378 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f355cfa527c1d9645f22bb2c09b082b3, offset: 0, size: 72053 2024.10.22 00:36:46.565017 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f355cfa527c1d9645f22bb2c09b082b3 for removal 2024.10.22 00:36:46.568928 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 65ad4191ab395717183d3c38b9384931, offset: 0, size: 139802 2024.10.22 00:36:46.661126 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 1025 2024.10.22 00:36:46.673601 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 65ad4191ab395717183d3c38b9384931 for removal 2024.10.22 00:36:46.675582 [ 1476 ] {} CacheMetadata: Key f355cfa527c1d9645f22bb2c09b082b3 is removed from metadata 2024.10.22 00:36:46.678670 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f93d3858d948e670ec7ffd322233e747, offset: 0, size: 141 2024.10.22 00:36:46.682292 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f93d3858d948e670ec7ffd322233e747 for removal 2024.10.22 00:36:46.683702 [ 1476 ] {} CacheMetadata: Key 65ad4191ab395717183d3c38b9384931 is removed from metadata 2024.10.22 00:36:46.686505 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6d7aa407e60e511d744a15fa9aa398ab, offset: 0, size: 107869 2024.10.22 00:36:46.691158 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6d7aa407e60e511d744a15fa9aa398ab for removal 2024.10.22 00:36:46.692857 [ 1476 ] {} CacheMetadata: Key f93d3858d948e670ec7ffd322233e747 is removed from metadata 2024.10.22 00:36:46.714265 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 590d855fe3b06aa5652147cd52e7348d, offset: 0, size: 12348 2024.10.22 00:36:46.721440 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 590d855fe3b06aa5652147cd52e7348d for removal 2024.10.22 00:36:46.722259 [ 1476 ] {} CacheMetadata: Key 7afd549b433ed4bcb207afd76875794c is removed from metadata 2024.10.22 00:36:46.827445 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 881093e70dae20de6b1c41e6b92ad6e8, offset: 0, size: 152 2024.10.22 00:36:46.833377 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 881093e70dae20de6b1c41e6b92ad6e8 for removal 2024.10.22 00:36:46.838181 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8ea44c30371150c9035b98e5fddb1d23, offset: 0, size: 137 2024.10.22 00:36:46.850400 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8ea44c30371150c9035b98e5fddb1d23 for removal 2024.10.22 00:36:46.852194 [ 1476 ] {} CacheMetadata: Key 881093e70dae20de6b1c41e6b92ad6e8 is removed from metadata 2024.10.22 00:36:46.858503 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9239f8de2572c0c0c0c4107a2088b594, offset: 0, size: 139 2024.10.22 00:36:46.866638 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9239f8de2572c0c0c0c4107a2088b594 for removal 2024.10.22 00:36:46.867645 [ 1476 ] {} CacheMetadata: Key 8ea44c30371150c9035b98e5fddb1d23 is removed from metadata 2024.10.22 00:36:46.875314 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8d5fe1fb1509ba15f4a8bdc281b82acb, offset: 0, size: 137 2024.10.22 00:36:46.876254 [ 1476 ] {} CacheMetadata: Key 9239f8de2572c0c0c0c4107a2088b594 is removed from metadata 2024.10.22 00:36:46.879373 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8d5fe1fb1509ba15f4a8bdc281b82acb for removal 2024.10.22 00:36:46.883953 [ 1476 ] {} CacheMetadata: Key 8d5fe1fb1509ba15f4a8bdc281b82acb is removed from metadata 2024.10.22 00:36:46.884194 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 31da56647a8445b8d3ec4656546925a9, offset: 0, size: 15837 2024.10.22 00:36:46.889444 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 31da56647a8445b8d3ec4656546925a9 for removal 2024.10.22 00:36:46.892597 [ 1476 ] {} CacheMetadata: Key 31da56647a8445b8d3ec4656546925a9 is removed from metadata 2024.10.22 00:36:46.893019 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 33099c7c561a6a22e008d22d11d87a8f, offset: 0, size: 148 2024.10.22 00:36:46.895246 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 33099c7c561a6a22e008d22d11d87a8f for removal 2024.10.22 00:36:46.910766 [ 1476 ] {} CacheMetadata: Key 590d855fe3b06aa5652147cd52e7348d is removed from metadata 2024.10.22 00:36:46.917232 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 59b8d6305d0533ba6a8352466eb19e1a, offset: 0, size: 168 2024.10.22 00:36:46.920944 [ 1476 ] {} CacheMetadata: Key 33099c7c561a6a22e008d22d11d87a8f is removed from metadata 2024.10.22 00:36:46.930548 [ 1476 ] {} CacheMetadata: Key e063039339d57402b8f24ef2fb1307a0 is removed from metadata 2024.10.22 00:36:46.931653 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 59b8d6305d0533ba6a8352466eb19e1a for removal 2024.10.22 00:36:46.940294 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a75ff4cecd106c76306d6cc4e7b43fc3, offset: 0, size: 19109 2024.10.22 00:36:46.950423 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a75ff4cecd106c76306d6cc4e7b43fc3 for removal 2024.10.22 00:36:46.955642 [ 1476 ] {} CacheMetadata: Key a75ff4cecd106c76306d6cc4e7b43fc3 is removed from metadata 2024.10.22 00:36:46.957205 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e7e9ca3fdf9228a3a20df630d31c6383, offset: 0, size: 141 2024.10.22 00:36:46.966468 [ 1476 ] {} CacheMetadata: Key 59b8d6305d0533ba6a8352466eb19e1a is removed from metadata 2024.10.22 00:36:46.977308 [ 1476 ] {} CacheMetadata: Key 6d7aa407e60e511d744a15fa9aa398ab is removed from metadata 2024.10.22 00:36:46.983938 [ 1476 ] {} CacheMetadata: Key 213c50424926d39332435310273ae096 is removed from metadata 2024.10.22 00:36:46.990106 [ 1476 ] {} CacheMetadata: Key f5d7cf2186448486be5867b14eb0ae0c is removed from metadata 2024.10.22 00:36:46.997400 [ 1476 ] {} CacheMetadata: Key 595f554298bbfd25d7c93cbab46fabcb is removed from metadata 2024.10.22 00:36:47.011650 [ 1476 ] {} CacheMetadata: Key c4b87e5719a6cd85f090c25edbd05fe7 is removed from metadata 2024.10.22 00:36:47.074198 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e7e9ca3fdf9228a3a20df630d31c6383 for removal 2024.10.22 00:36:47.084709 [ 1476 ] {} CacheMetadata: Key e7e9ca3fdf9228a3a20df630d31c6383 is removed from metadata 2024.10.22 00:36:47.086375 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3e54e8b7cff862cd46df062e0e3977e2, offset: 0, size: 11152 2024.10.22 00:36:47.099154 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3e54e8b7cff862cd46df062e0e3977e2 for removal 2024.10.22 00:36:47.106366 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 05f85c50a49d4c5b59c1133612717f45, offset: 0, size: 24589 2024.10.22 00:36:47.110125 [ 1476 ] {} CacheMetadata: Key 3e54e8b7cff862cd46df062e0e3977e2 is removed from metadata 2024.10.22 00:36:47.119734 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 05f85c50a49d4c5b59c1133612717f45 for removal 2024.10.22 00:36:47.129985 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4d773b705c9135d6d87231e536cb3b7f, offset: 0, size: 154 2024.10.22 00:36:47.146603 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4d773b705c9135d6d87231e536cb3b7f for removal 2024.10.22 00:36:47.155304 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ab97f3260a9e554e2cd2e825a9beb565, offset: 0, size: 141 2024.10.22 00:36:47.161519 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ab97f3260a9e554e2cd2e825a9beb565 for removal 2024.10.22 00:36:47.172677 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cf2003b03e5fbf78f7919878b7a3ab1a, offset: 0, size: 154 2024.10.22 00:36:47.179429 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cf2003b03e5fbf78f7919878b7a3ab1a for removal 2024.10.22 00:36:47.186858 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5eab043e078e7272132f93372a7a07ee, offset: 0, size: 48180 2024.10.22 00:36:47.199872 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5eab043e078e7272132f93372a7a07ee for removal 2024.10.22 00:36:47.208021 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fd71457803676dc92b0837def738d58b, offset: 0, size: 45759 2024.10.22 00:36:47.221514 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fd71457803676dc92b0837def738d58b for removal 2024.10.22 00:36:47.230581 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 36a7971c7827efc23c3ea04a68d9f78d, offset: 0, size: 43581 2024.10.22 00:36:47.237507 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 36a7971c7827efc23c3ea04a68d9f78d for removal 2024.10.22 00:36:47.248576 [ 1476 ] {} CacheMetadata: Key fd71457803676dc92b0837def738d58b is removed from metadata 2024.10.22 00:36:47.260364 [ 1476 ] {} CacheMetadata: Key 5eab043e078e7272132f93372a7a07ee is removed from metadata 2024.10.22 00:36:47.275263 [ 1476 ] {} CacheMetadata: Key cf2003b03e5fbf78f7919878b7a3ab1a is removed from metadata 2024.10.22 00:36:47.283697 [ 1476 ] {} CacheMetadata: Key ab97f3260a9e554e2cd2e825a9beb565 is removed from metadata 2024.10.22 00:36:47.289666 [ 1476 ] {} CacheMetadata: Key 4d773b705c9135d6d87231e536cb3b7f is removed from metadata 2024.10.22 00:36:47.296399 [ 1476 ] {} CacheMetadata: Key 05f85c50a49d4c5b59c1133612717f45 is removed from metadata 2024.10.22 00:36:47.353501 [ 1476 ] {} CacheMetadata: Key 36a7971c7827efc23c3ea04a68d9f78d is removed from metadata 2024.10.22 00:36:47.354632 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bc4d99afb18560565cf3a508d873a38c, offset: 0, size: 963962 2024.10.22 00:36:47.464574 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bc4d99afb18560565cf3a508d873a38c for removal 2024.10.22 00:36:47.571746 [ 1476 ] {} CacheMetadata: Key bc4d99afb18560565cf3a508d873a38c is removed from metadata 2024.10.22 00:36:47.573323 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ea0706c0902a25bf509ade4c0d8c54e0, offset: 0, size: 29346 2024.10.22 00:36:47.576464 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} MergeTask::MergeProjectionsStage: Merge sorted 11768 rows, containing 13 columns (13 merged, 0 gathered) in 2.994832045 sec., 3929.435715651293 rows/sec., 598.20 KiB/sec. 2024.10.22 00:36:47.577528 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ea0706c0902a25bf509ade4c0d8c54e0 for removal 2024.10.22 00:36:47.638799 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qqa/uvffilwlwxrllldpsvznikrurrauv (LogSeriesLimiter: on interval from 2024-10-22 00:35:53 to 2024-10-22 00:36:47 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:36:47.685625 [ 1476 ] {} CacheMetadata: Key ea0706c0902a25bf509ade4c0d8c54e0 is removed from metadata 2024.10.22 00:36:47.686901 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 74f5d6873b185fea1709232f2f2f6253, offset: 0, size: 27239 2024.10.22 00:36:47.692680 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 74f5d6873b185fea1709232f2f2f6253 for removal 2024.10.22 00:36:47.696922 [ 1476 ] {} CacheMetadata: Key 74f5d6873b185fea1709232f2f2f6253 is removed from metadata 2024.10.22 00:36:47.698170 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 000dea7d5d3ad3a37b01b63a432cf912, offset: 0, size: 152 2024.10.22 00:36:47.703212 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 000dea7d5d3ad3a37b01b63a432cf912 for removal 2024.10.22 00:36:47.715116 [ 1476 ] {} CacheMetadata: Key 000dea7d5d3ad3a37b01b63a432cf912 is removed from metadata 2024.10.22 00:36:47.717632 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 76e6cf1b8ed66506df6e6f90cc1c9b31, offset: 0, size: 141 2024.10.22 00:36:47.724515 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 76e6cf1b8ed66506df6e6f90cc1c9b31 for removal 2024.10.22 00:36:47.730766 [ 1476 ] {} CacheMetadata: Key 76e6cf1b8ed66506df6e6f90cc1c9b31 is removed from metadata 2024.10.22 00:36:47.732424 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0cdcf3aca7ec87470896d0df6a21b12c, offset: 0, size: 47572 2024.10.22 00:36:47.738738 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0cdcf3aca7ec87470896d0df6a21b12c for removal 2024.10.22 00:36:47.745674 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a9febe926637e97192582dca5c42da66, offset: 0, size: 139 2024.10.22 00:36:47.752047 [ 1476 ] {} CacheMetadata: Key 0cdcf3aca7ec87470896d0df6a21b12c is removed from metadata 2024.10.22 00:36:47.757604 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a9febe926637e97192582dca5c42da66 for removal 2024.10.22 00:36:47.768640 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2fe5b343bb904d0cbc6de7faa536249c, offset: 0, size: 20732 2024.10.22 00:36:47.770791 [ 1476 ] {} CacheMetadata: Key a9febe926637e97192582dca5c42da66 is removed from metadata 2024.10.22 00:36:47.774032 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2fe5b343bb904d0cbc6de7faa536249c for removal 2024.10.22 00:36:47.778608 [ 1476 ] {} CacheMetadata: Key 2fe5b343bb904d0cbc6de7faa536249c is removed from metadata 2024.10.22 00:36:47.778343 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 84e6462c039be6c743748cead4f50546, offset: 0, size: 141 2024.10.22 00:36:47.779068 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} MergedBlockOutputStream: filled checksums 202410_1_231_46 (state Temporary) 2024.10.22 00:36:47.782283 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 84e6462c039be6c743748cead4f50546 for removal 2024.10.22 00:36:47.785287 [ 1476 ] {} CacheMetadata: Key 84e6462c039be6c743748cead4f50546 is removed from metadata 2024.10.22 00:36:47.786575 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b04ee3baf4c3087d8751ea809ac0e82d, offset: 0, size: 152 2024.10.22 00:36:47.836437 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_231_46 to 202410_1_231_46 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:47.848725 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_231_46} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_226_45, 202410_231_231_0] -> 202410_1_231_46 2024.10.22 00:36:47.895402 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b04ee3baf4c3087d8751ea809ac0e82d for removal 2024.10.22 00:36:47.906378 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 1793 2024.10.22 00:36:47.907565 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3850ef56c18a1316c846b05b360187f5, offset: 0, size: 137 2024.10.22 00:36:47.912833 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3850ef56c18a1316c846b05b360187f5 for removal 2024.10.22 00:36:47.920050 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 19ac271c9adf25cd66235f00be2f097f, offset: 0, size: 137 2024.10.22 00:36:47.923928 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 19ac271c9adf25cd66235f00be2f097f for removal 2024.10.22 00:36:47.931640 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f413db2a3461fd1cabc8019cda3c1eab, offset: 0, size: 141 2024.10.22 00:36:47.936525 [ 1476 ] {} CacheMetadata: Key b04ee3baf4c3087d8751ea809ac0e82d is removed from metadata 2024.10.22 00:36:47.939944 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f413db2a3461fd1cabc8019cda3c1eab for removal 2024.10.22 00:36:47.952188 [ 1476 ] {} CacheMetadata: Key 19ac271c9adf25cd66235f00be2f097f is removed from metadata 2024.10.22 00:36:47.954370 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ce0fc91ff9b6b17b5c3bb14d579d9bea, offset: 0, size: 141 2024.10.22 00:36:47.960058 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.66 MiB. 2024.10.22 00:36:47.963740 [ 1476 ] {} CacheMetadata: Key f413db2a3461fd1cabc8019cda3c1eab is removed from metadata 2024.10.22 00:36:47.960782 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ce0fc91ff9b6b17b5c3bb14d579d9bea for removal 2024.10.22 00:36:47.972174 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2be3b08b2119c77a312e4267bb7cf959, offset: 0, size: 152 2024.10.22 00:36:47.973476 [ 1476 ] {} CacheMetadata: Key ce0fc91ff9b6b17b5c3bb14d579d9bea is removed from metadata 2024.10.22 00:36:47.975052 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2be3b08b2119c77a312e4267bb7cf959 for removal 2024.10.22 00:36:47.980793 [ 1476 ] {} CacheMetadata: Key 2be3b08b2119c77a312e4267bb7cf959 is removed from metadata 2024.10.22 00:36:47.985440 [ 1476 ] {} CacheMetadata: Key 3850ef56c18a1316c846b05b360187f5 is removed from metadata 2024.10.22 00:36:48.078437 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e2f3652bb452bd339db48a0844b68b6e, offset: 0, size: 152 2024.10.22 00:36:48.081387 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e2f3652bb452bd339db48a0844b68b6e for removal 2024.10.22 00:36:48.083951 [ 1476 ] {} CacheMetadata: Key e2f3652bb452bd339db48a0844b68b6e is removed from metadata 2024.10.22 00:36:48.084706 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 06058e1b07419f6c2c7997456a60733c, offset: 0, size: 21015 2024.10.22 00:36:48.088069 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 06058e1b07419f6c2c7997456a60733c for removal 2024.10.22 00:36:48.091988 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1107129527394733015cae06506af3ae, offset: 0, size: 1273 2024.10.22 00:36:48.095095 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1107129527394733015cae06506af3ae for removal 2024.10.22 00:36:48.102945 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c5bef9c89ed359201a3ae81353522fd4, offset: 0, size: 26745 2024.10.22 00:36:48.111731 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c5bef9c89ed359201a3ae81353522fd4 for removal 2024.10.22 00:36:48.122166 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 46eee315ef49f4017b0b9cd674208be6, offset: 0, size: 84026 2024.10.22 00:36:48.127442 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 46eee315ef49f4017b0b9cd674208be6 for removal 2024.10.22 00:36:48.131221 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:48.135163 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.16 GiB. 2024.10.22 00:36:48.133791 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0fe0b2c854de01c4ebddc415d5fbd73c, offset: 0, size: 1275 2024.10.22 00:36:48.140084 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0fe0b2c854de01c4ebddc415d5fbd73c for removal 2024.10.22 00:36:48.147317 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b73975a09801bb72a181939b382cad65, offset: 0, size: 36270 2024.10.22 00:36:48.157709 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b73975a09801bb72a181939b382cad65 for removal 2024.10.22 00:36:48.172558 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1bca150b0b710e80ce59e5e336298b45, offset: 0, size: 152 2024.10.22 00:36:48.176305 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1bca150b0b710e80ce59e5e336298b45 for removal 2024.10.22 00:36:48.179897 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_127_127_0 (state Temporary) 2024.10.22 00:36:48.184180 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 76dd34291f14f892a9e308509e7943af, offset: 0, size: 152 2024.10.22 00:36:48.208003 [ 1476 ] {} CacheMetadata: Key 1bca150b0b710e80ce59e5e336298b45 is removed from metadata 2024.10.22 00:36:48.220739 [ 1476 ] {} CacheMetadata: Key b73975a09801bb72a181939b382cad65 is removed from metadata 2024.10.22 00:36:48.231541 [ 1476 ] {} CacheMetadata: Key 0fe0b2c854de01c4ebddc415d5fbd73c is removed from metadata 2024.10.22 00:36:48.288353 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 76dd34291f14f892a9e308509e7943af for removal 2024.10.22 00:36:48.292824 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b1cce5a6b8323661ccd309b4937de011, offset: 0, size: 151 2024.10.22 00:36:48.296775 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b1cce5a6b8323661ccd309b4937de011 for removal 2024.10.22 00:36:48.306651 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2cf94b4ab0164f2f14ab11383c589b3d, offset: 0, size: 11932 2024.10.22 00:36:48.322512 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2cf94b4ab0164f2f14ab11383c589b3d for removal 2024.10.22 00:36:48.329584 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f9fe49a6c4ba94c7d96acdced8566e74, offset: 0, size: 139 2024.10.22 00:36:48.334315 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_127_127_0 to 202410_140_140_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:48.336682 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f9fe49a6c4ba94c7d96acdced8566e74 for removal 2024.10.22 00:36:48.340975 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9a38f8e936deadf714113f4862d1cd69, offset: 0, size: 16671 2024.10.22 00:36:48.348009 [ 1476 ] {} CacheMetadata: Key f9fe49a6c4ba94c7d96acdced8566e74 is removed from metadata 2024.10.22 00:36:48.352480 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9a38f8e936deadf714113f4862d1cd69 for removal 2024.10.22 00:36:48.368629 [ 1476 ] {} CacheMetadata: Key 9a38f8e936deadf714113f4862d1cd69 is removed from metadata 2024.10.22 00:36:48.368660 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 979a84a0e5389f3396fe762d70a4ed57, offset: 0, size: 139 2024.10.22 00:36:48.375330 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 979a84a0e5389f3396fe762d70a4ed57 for removal 2024.10.22 00:36:48.380363 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f855ea1422ca592f699ef14aa4656545, offset: 0, size: 139 2024.10.22 00:36:48.380709 [ 1476 ] {} CacheMetadata: Key 979a84a0e5389f3396fe762d70a4ed57 is removed from metadata 2024.10.22 00:36:48.384758 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f855ea1422ca592f699ef14aa4656545 for removal 2024.10.22 00:36:48.391691 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1de519437d6f8c9fd12e385693c61473, offset: 0, size: 185 2024.10.22 00:36:48.393134 [ 1476 ] {} CacheMetadata: Key f855ea1422ca592f699ef14aa4656545 is removed from metadata 2024.10.22 00:36:48.410001 [ 1476 ] {} CacheMetadata: Key 2cf94b4ab0164f2f14ab11383c589b3d is removed from metadata 2024.10.22 00:36:48.425523 [ 1476 ] {} CacheMetadata: Key b1cce5a6b8323661ccd309b4937de011 is removed from metadata 2024.10.22 00:36:48.440969 [ 1476 ] {} CacheMetadata: Key 76dd34291f14f892a9e308509e7943af is removed from metadata 2024.10.22 00:36:48.456961 [ 1476 ] {} CacheMetadata: Key 46eee315ef49f4017b0b9cd674208be6 is removed from metadata 2024.10.22 00:36:48.497585 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1793 2024.10.22 00:36:48.498102 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1de519437d6f8c9fd12e385693c61473 for removal 2024.10.22 00:36:48.504496 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 154d0416ac16501defff856e47b3b65c, offset: 0, size: 141 2024.10.22 00:36:48.519574 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 154d0416ac16501defff856e47b3b65c for removal 2024.10.22 00:36:48.524485 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dcf40f9bbed494b22e6c53dbd4cdef5a, offset: 0, size: 146 2024.10.22 00:36:48.528690 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dcf40f9bbed494b22e6c53dbd4cdef5a for removal 2024.10.22 00:36:48.535198 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1a7d0dd5ec0f721cb2aad856781eec7a, offset: 0, size: 34009 2024.10.22 00:36:48.547160 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1a7d0dd5ec0f721cb2aad856781eec7a for removal 2024.10.22 00:36:48.555793 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e5491a44de1243074e125809030b6bf4, offset: 0, size: 19774 2024.10.22 00:36:48.565517 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e5491a44de1243074e125809030b6bf4 for removal 2024.10.22 00:36:48.572142 [ 1476 ] {} CacheMetadata: Key dcf40f9bbed494b22e6c53dbd4cdef5a is removed from metadata 2024.10.22 00:36:48.573019 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 285 entries to flush up to offset 11222 2024.10.22 00:36:48.575829 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 19d968506d5968715abed2af792eacbd, offset: 0, size: 57242 2024.10.22 00:36:48.586154 [ 1476 ] {} CacheMetadata: Key e5491a44de1243074e125809030b6bf4 is removed from metadata 2024.10.22 00:36:48.595720 [ 1476 ] {} CacheMetadata: Key 1a7d0dd5ec0f721cb2aad856781eec7a is removed from metadata 2024.10.22 00:36:48.682939 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 19d968506d5968715abed2af792eacbd for removal 2024.10.22 00:36:48.686946 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5d66324f71e4b8d172b8465d3da6405e, offset: 0, size: 137 2024.10.22 00:36:48.691174 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5d66324f71e4b8d172b8465d3da6405e for removal 2024.10.22 00:36:48.693737 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 54de15e0a232a94471bd180a57dfa53e, offset: 0, size: 52561 2024.10.22 00:36:48.697035 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 54de15e0a232a94471bd180a57dfa53e for removal 2024.10.22 00:36:48.714062 [ 1476 ] {} CacheMetadata: Key 54de15e0a232a94471bd180a57dfa53e is removed from metadata 2024.10.22 00:36:48.718018 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3f986617f6cbae070648738b65865e75, offset: 0, size: 75761 2024.10.22 00:36:48.721742 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3f986617f6cbae070648738b65865e75 for removal 2024.10.22 00:36:48.726678 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 927d1f9a0121e1640a3835ccb74ed998, offset: 0, size: 141 2024.10.22 00:36:48.730612 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 927d1f9a0121e1640a3835ccb74ed998 for removal 2024.10.22 00:36:48.735946 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 86866cde6775b7beaedb7b93629985d5, offset: 0, size: 152 2024.10.22 00:36:48.739748 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 86866cde6775b7beaedb7b93629985d5 for removal 2024.10.22 00:36:48.755156 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d11a4b11c6fe1f06e2b50c379e004998, offset: 0, size: 35345 2024.10.22 00:36:48.765485 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d11a4b11c6fe1f06e2b50c379e004998 for removal 2024.10.22 00:36:48.771363 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d6dc52ea69218f6f2f0017e69c8c0bf6, offset: 0, size: 85201 2024.10.22 00:36:48.776417 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d6dc52ea69218f6f2f0017e69c8c0bf6 for removal 2024.10.22 00:36:48.783020 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2df14b44ffee9c610bd2c958bf2d9872, offset: 0, size: 137 2024.10.22 00:36:48.787055 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2df14b44ffee9c610bd2c958bf2d9872 for removal 2024.10.22 00:36:48.791747 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4ceeaa966b5ded3d9b47be93cbaec6c1, offset: 0, size: 141 2024.10.22 00:36:48.794629 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4ceeaa966b5ded3d9b47be93cbaec6c1 for removal 2024.10.22 00:36:48.811971 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6ab713ecd16fd1a7a775bcfb045fa593, offset: 0, size: 27573 2024.10.22 00:36:48.817654 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6ab713ecd16fd1a7a775bcfb045fa593 for removal 2024.10.22 00:36:48.826351 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 61265ac8d95b361b2f4d27dfda1f6feb, offset: 0, size: 154 2024.10.22 00:36:48.830464 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 61265ac8d95b361b2f4d27dfda1f6feb for removal 2024.10.22 00:36:48.838914 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6a381f93fa46e867b412353216755a8c, offset: 0, size: 83180 2024.10.22 00:36:48.843398 [ 1476 ] {} CacheMetadata: Key 61265ac8d95b361b2f4d27dfda1f6feb is removed from metadata 2024.10.22 00:36:48.845238 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6a381f93fa46e867b412353216755a8c for removal 2024.10.22 00:36:48.853301 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d2958d6eee9af984a3cc189c98499c3d, offset: 0, size: 81046 2024.10.22 00:36:48.854852 [ 1476 ] {} CacheMetadata: Key 6a381f93fa46e867b412353216755a8c is removed from metadata 2024.10.22 00:36:48.857279 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d2958d6eee9af984a3cc189c98499c3d for removal 2024.10.22 00:36:48.870334 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 71323f47ec36d3991bed844fab0c466c, offset: 0, size: 32879 2024.10.22 00:36:48.877049 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 71323f47ec36d3991bed844fab0c466c for removal 2024.10.22 00:36:48.877098 [ 1476 ] {} CacheMetadata: Key d2958d6eee9af984a3cc189c98499c3d is removed from metadata 2024.10.22 00:36:48.883648 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b9abfa64fa116cdc5952582fa09f18fb, offset: 0, size: 141 2024.10.22 00:36:48.887801 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b9abfa64fa116cdc5952582fa09f18fb for removal 2024.10.22 00:36:48.892544 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 53888bf74bc6a4041a47bca26e815e83, offset: 0, size: 323966 2024.10.22 00:36:48.897131 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 53888bf74bc6a4041a47bca26e815e83 for removal 2024.10.22 00:36:48.898674 [ 1476 ] {} CacheMetadata: Key b9abfa64fa116cdc5952582fa09f18fb is removed from metadata 2024.10.22 00:36:48.903556 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: eee82ad0eae378dbae495cb84517ef01, offset: 0, size: 141 2024.10.22 00:36:48.906964 [ 1476 ] {} CacheMetadata: Key 53888bf74bc6a4041a47bca26e815e83 is removed from metadata 2024.10.22 00:36:48.911489 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key eee82ad0eae378dbae495cb84517ef01 for removal 2024.10.22 00:36:48.918430 [ 1476 ] {} CacheMetadata: Key eee82ad0eae378dbae495cb84517ef01 is removed from metadata 2024.10.22 00:36:48.918598 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 998203e4f38f8c17fdfe5c7cfc3b6d17, offset: 0, size: 72249 2024.10.22 00:36:48.926303 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 998203e4f38f8c17fdfe5c7cfc3b6d17 for removal 2024.10.22 00:36:48.929125 [ 1476 ] {} CacheMetadata: Key 71323f47ec36d3991bed844fab0c466c is removed from metadata 2024.10.22 00:36:48.936849 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ba28ca375329a2433912f80362e36dd4, offset: 0, size: 145 2024.10.22 00:36:48.945716 [ 1476 ] {} CacheMetadata: Key 998203e4f38f8c17fdfe5c7cfc3b6d17 is removed from metadata 2024.10.22 00:36:48.946150 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ba28ca375329a2433912f80362e36dd4 for removal 2024.10.22 00:36:48.953549 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3908c676cf4cd9fcc9d01d370e41a6c8, offset: 0, size: 137 2024.10.22 00:36:48.955716 [ 1476 ] {} CacheMetadata: Key ba28ca375329a2433912f80362e36dd4 is removed from metadata 2024.10.22 00:36:48.965838 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3908c676cf4cd9fcc9d01d370e41a6c8 for removal 2024.10.22 00:36:48.969564 [ 1476 ] {} CacheMetadata: Key 6ab713ecd16fd1a7a775bcfb045fa593 is removed from metadata 2024.10.22 00:36:48.976340 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5e3040d364a7ae51a2106aba6ea9fda5, offset: 0, size: 141 2024.10.22 00:36:48.978471 [ 1476 ] {} CacheMetadata: Key 4ceeaa966b5ded3d9b47be93cbaec6c1 is removed from metadata 2024.10.22 00:36:48.980374 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5e3040d364a7ae51a2106aba6ea9fda5 for removal 2024.10.22 00:36:48.984256 [ 1476 ] {} CacheMetadata: Key 5e3040d364a7ae51a2106aba6ea9fda5 is removed from metadata 2024.10.22 00:36:48.985320 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d340fbef219bf0eacaa145e75c7a749f, offset: 0, size: 126612 2024.10.22 00:36:48.988797 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d340fbef219bf0eacaa145e75c7a749f for removal 2024.10.22 00:36:48.988983 [ 1476 ] {} CacheMetadata: Key 2df14b44ffee9c610bd2c958bf2d9872 is removed from metadata 2024.10.22 00:36:48.992604 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 62f858f0f0d0a1f73e78159443a825d8, offset: 0, size: 1498842 2024.10.22 00:36:48.997090 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 62f858f0f0d0a1f73e78159443a825d8 for removal 2024.10.22 00:36:49.011233 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 80c72655c214d2f1c30ea2181e743837, offset: 0, size: 152 2024.10.22 00:36:49.014836 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 80c72655c214d2f1c30ea2181e743837 for removal 2024.10.22 00:36:49.019233 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4ce330bf8599485fb3bcf6f671bb0ce7, offset: 0, size: 2344 2024.10.22 00:36:49.024245 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4ce330bf8599485fb3bcf6f671bb0ce7 for removal 2024.10.22 00:36:49.027228 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 16d7aa27ec357f3330ba7a9283744f3b, offset: 0, size: 139 2024.10.22 00:36:49.031593 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 16d7aa27ec357f3330ba7a9283744f3b for removal 2024.10.22 00:36:49.037110 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9797867e7206af8febc3e51a88a94b96, offset: 0, size: 236130 2024.10.22 00:36:49.051610 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9797867e7206af8febc3e51a88a94b96 for removal 2024.10.22 00:36:49.056099 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e8eed129badedd040ea27ee2956de411, offset: 0, size: 23285 2024.10.22 00:36:49.069190 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e8eed129badedd040ea27ee2956de411 for removal 2024.10.22 00:36:49.096872 [ 1476 ] {} CacheMetadata: Key e8eed129badedd040ea27ee2956de411 is removed from metadata 2024.10.22 00:36:49.111681 [ 1476 ] {} CacheMetadata: Key 9797867e7206af8febc3e51a88a94b96 is removed from metadata 2024.10.22 00:36:49.127585 [ 1476 ] {} CacheMetadata: Key 16d7aa27ec357f3330ba7a9283744f3b is removed from metadata 2024.10.22 00:36:49.134444 [ 1476 ] {} CacheMetadata: Key 4ce330bf8599485fb3bcf6f671bb0ce7 is removed from metadata 2024.10.22 00:36:49.152649 [ 1476 ] {} CacheMetadata: Key 80c72655c214d2f1c30ea2181e743837 is removed from metadata 2024.10.22 00:36:49.164181 [ 1476 ] {} CacheMetadata: Key 62f858f0f0d0a1f73e78159443a825d8 is removed from metadata 2024.10.22 00:36:49.173567 [ 1476 ] {} CacheMetadata: Key d6dc52ea69218f6f2f0017e69c8c0bf6 is removed from metadata 2024.10.22 00:36:49.184683 [ 1476 ] {} CacheMetadata: Key 86866cde6775b7beaedb7b93629985d5 is removed from metadata 2024.10.22 00:36:49.192455 [ 1476 ] {} CacheMetadata: Key d340fbef219bf0eacaa145e75c7a749f is removed from metadata 2024.10.22 00:36:49.206339 [ 1476 ] {} CacheMetadata: Key 3f986617f6cbae070648738b65865e75 is removed from metadata 2024.10.22 00:36:49.222965 [ 1476 ] {} CacheMetadata: Key 154d0416ac16501defff856e47b3b65c is removed from metadata 2024.10.22 00:36:49.230705 [ 1476 ] {} CacheMetadata: Key 1de519437d6f8c9fd12e385693c61473 is removed from metadata 2024.10.22 00:36:49.240481 [ 1476 ] {} CacheMetadata: Key c5bef9c89ed359201a3ae81353522fd4 is removed from metadata 2024.10.22 00:36:49.260419 [ 1476 ] {} CacheMetadata: Key 1107129527394733015cae06506af3ae is removed from metadata 2024.10.22 00:36:49.270472 [ 1476 ] {} CacheMetadata: Key 3908c676cf4cd9fcc9d01d370e41a6c8 is removed from metadata 2024.10.22 00:36:49.278508 [ 1476 ] {} CacheMetadata: Key d11a4b11c6fe1f06e2b50c379e004998 is removed from metadata 2024.10.22 00:36:49.288441 [ 1476 ] {} CacheMetadata: Key 927d1f9a0121e1640a3835ccb74ed998 is removed from metadata 2024.10.22 00:36:49.309521 [ 1476 ] {} CacheMetadata: Key 5d66324f71e4b8d172b8465d3da6405e is removed from metadata 2024.10.22 00:36:49.326994 [ 1476 ] {} CacheMetadata: Key 19d968506d5968715abed2af792eacbd is removed from metadata 2024.10.22 00:36:49.345618 [ 1476 ] {} CacheMetadata: Key 06058e1b07419f6c2c7997456a60733c is removed from metadata 2024.10.22 00:36:49.460697 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:36:49.476293 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:49.479090 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:36:49.481517 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.16 GiB. 2024.10.22 00:36:49.493314 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:36:49.496337 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:36:49.496804 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:36:49.627257 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:36:49.684135 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:36:49.711551 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:36:49.741110 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:36:49.882757 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_219_219_0 (state Temporary) 2024.10.22 00:36:50.015086 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_219_219_0 to 202410_232_232_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:50.065293 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 11222 2024.10.22 00:36:50.082365 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 214046 2024.10.22 00:36:50.280792 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 47 entries to flush up to offset 8370 2024.10.22 00:36:50.599206 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:50.605284 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:50.615699 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:36:50.618523 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:36:50.756006 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.83 GiB, free memory in arenas 5.75 MiB, will set to 1.61 GiB (RSS), difference: -171.88 MiB 2024.10.22 00:36:50.782730 [ 1168 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:36:50.820663 [ 1168 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:36:50.875070 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_122_122_0 (state Temporary) 2024.10.22 00:36:50.876284 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_126_126_0 (state Temporary) 2024.10.22 00:36:50.934432 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_126_126_0 to 202410_132_132_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:51.053461 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_122_122_0 to 202410_129_129_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:51.091740 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 214046 2024.10.22 00:36:51.231855 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8370 2024.10.22 00:36:52.072048 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bgr/onekilfvpngmqlhsnyflsnagkybfy (LogSeriesLimiter: on interval from 2024-10-22 00:36:47 to 2024-10-22 00:36:52 accepted series 1 / 182 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:36:52.526566 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 120 2024.10.22 00:36:52.626970 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.83 GiB, free memory in arenas 5.80 MiB, will set to 1.61 GiB (RSS), difference: -134.74 MiB 2024.10.22 00:36:52.801128 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:52.815528 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:36:52.881221 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_102_102_0 (state Temporary) 2024.10.22 00:36:52.917285 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_126_25, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0] 2024.10.22 00:36:52.922488 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_126_25, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0] 2024.10.22 00:36:52.984849 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_102_102_0 to 202410_112_112_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:53.070032 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_126_25, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0] 2024.10.22 00:36:53.278181 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 120 2024.10.22 00:36:54.107039 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 137 entries to flush up to offset 11359 2024.10.22 00:36:54.231483 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_76_15, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0] 2024.10.22 00:36:54.234094 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_76_15, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0] 2024.10.22 00:36:54.377216 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_76_15, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0] 2024.10.22 00:36:54.454782 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.83 GiB, free memory in arenas 3.77 MiB, will set to 1.61 GiB (RSS), difference: -162.21 MiB 2024.10.22 00:36:54.831123 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:54.836166 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:36:54.907246 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_59_12, 202410_60_60_0, 202410_61_61_0, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0] 2024.10.22 00:36:54.911005 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_59_12, 202410_60_60_0, 202410_61_61_0, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0] 2024.10.22 00:36:54.978721 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_220_220_0 (state Temporary) 2024.10.22 00:36:55.047894 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_59_12, 202410_60_60_0, 202410_61_61_0, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0] 2024.10.22 00:36:55.075134 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_220_220_0 to 202410_233_233_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:55.144828 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 11359 2024.10.22 00:36:56.000534 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 1799 2024.10.22 00:36:56.257305 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:56.366793 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:36:56.412376 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.83 GiB, free memory in arenas 4.25 MiB, will set to 1.62 GiB (RSS), difference: -73.98 MiB 2024.10.22 00:36:56.537406 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_128_128_0 (state Temporary) 2024.10.22 00:36:56.697676 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_128_128_0 to 202410_141_141_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:56.802045 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1799 2024.10.22 00:36:56.809742 [ 1235 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_136_27 to 202410_141_141_0 2024.10.22 00:36:56.831288 [ 1235 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:56.842697 [ 1235 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:36:57.112734 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_136_27 to 202410_141_141_0 into Compact with storage Full 2024.10.22 00:36:57.385139 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:36:57.393264 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_136_27, total 1840 rows starting from the beginning of the part 2024.10.22 00:36:57.508161 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_137_137_0, total 18 rows starting from the beginning of the part 2024.10.22 00:36:57.631181 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_138_138_0, total 7 rows starting from the beginning of the part 2024.10.22 00:36:57.661122 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_139_139_0, total 22 rows starting from the beginning of the part 2024.10.22 00:36:57.673670 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_140_140_0, total 12 rows starting from the beginning of the part 2024.10.22 00:36:57.693369 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_141_141_0, total 6 rows starting from the beginning of the part 2024.10.22 00:36:58.112320 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 215582 2024.10.22 00:36:58.554696 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 00:36:58.739628 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 46 entries to flush up to offset 8416 2024.10.22 00:36:58.978399 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:58.981195 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:36:58.987848 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:58.990648 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:36:59.133773 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_127_127_0 (state Temporary) 2024.10.22 00:36:59.172343 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 11398 2024.10.22 00:36:59.176423 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} MergeTask::MergeProjectionsStage: Merge sorted 1905 rows, containing 12 columns (12 merged, 0 gathered) in 2.08572124 sec., 913.3531190390524 rows/sec., 141.18 KiB/sec. 2024.10.22 00:36:59.278695 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_127_127_0 to 202410_133_133_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:59.305968 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} MergedBlockOutputStream: filled checksums 202410_1_141_28 (state Temporary) 2024.10.22 00:36:59.337686 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_123_123_0 (state Temporary) 2024.10.22 00:36:59.472907 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_123_123_0 to 202410_130_130_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:59.488134 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:36:59.491785 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:36:59.492216 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:36:59.495427 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:36:59.537472 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8416 2024.10.22 00:36:59.661233 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_221_221_0 (state Temporary) 2024.10.22 00:36:59.679522 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 3.89 MiB, will set to 1.63 GiB (RSS), difference: -13.95 MiB 2024.10.22 00:36:59.698435 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_221_221_0 to 202410_234_234_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:59.705581 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 215582 2024.10.22 00:36:59.785100 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_141_28 to 202410_1_141_28 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:36:59.788583 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_141_28} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_136_27, 202410_141_141_0] -> 202410_1_141_28 2024.10.22 00:36:59.855494 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 11398 2024.10.22 00:36:59.931445 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:37:00.780349 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 121 2024.10.22 00:37:01.091877 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:01.096420 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:01.284322 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_103_103_0 (state Temporary) 2024.10.22 00:37:01.478349 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_103_103_0 to 202410_113_113_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:01.520267 [ 1256 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_108_22 to 202410_113_113_0 2024.10.22 00:37:01.534719 [ 1256 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:01.538557 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:01.557443 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 3.33 MiB, will set to 1.64 GiB (RSS), difference: 11.33 MiB 2024.10.22 00:37:01.581282 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_108_22 to 202410_113_113_0 into Compact with storage Full 2024.10.22 00:37:01.672320 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 121 2024.10.22 00:37:01.759394 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:37:01.773086 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_108_22, total 152 rows starting from the beginning of the part 2024.10.22 00:37:01.883739 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_109_109_0, total 1 rows starting from the beginning of the part 2024.10.22 00:37:01.990891 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_110_110_0, total 1 rows starting from the beginning of the part 2024.10.22 00:37:02.191811 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_111_111_0, total 1 rows starting from the beginning of the part 2024.10.22 00:37:02.291550 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_112_112_0, total 1 rows starting from the beginning of the part 2024.10.22 00:37:02.394621 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_113_113_0, total 1 rows starting from the beginning of the part 2024.10.22 00:37:02.682738 [ 1250 ] {} DatabaseCatalog: Execution took 217 ms. 2024.10.22 00:37:03.586466 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_113_23/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:37:03.868748 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 11437 2024.10.22 00:37:04.178444 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:04.181768 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:04.304039 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 1811 2024.10.22 00:37:04.358146 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_222_222_0 (state Temporary) 2024.10.22 00:37:04.437524 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:04.440458 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:04.482555 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_129_129_0 (state Temporary) 2024.10.22 00:37:04.554768 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_129_129_0 to 202410_142_142_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:04.558548 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_222_222_0 to 202410_235_235_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:04.623542 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1811 2024.10.22 00:37:04.632537 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 11437 2024.10.22 00:37:04.679679 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} MergeTask::MergeProjectionsStage: Merge sorted 157 rows, containing 32 columns (32 merged, 0 gathered) in 3.1203748 sec., 50.314468633703875 rows/sec., 5.63 KiB/sec. 2024.10.22 00:37:04.718231 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} MergedBlockOutputStream: filled checksums 202410_1_113_23 (state Temporary) 2024.10.22 00:37:04.735193 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 25592 bytes for 3daea68137f87ac49d90e680cc354a43:0) 2024.10.22 00:37:04.740032 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a6a165c0d77d77b701055e53eb5b8ed6, offset: 0, size: 236606 2024.10.22 00:37:04.754456 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a6a165c0d77d77b701055e53eb5b8ed6 for removal 2024.10.22 00:37:04.760399 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cf9f431a4dc698b21ba174d0c921cfed, offset: 0, size: 170 2024.10.22 00:37:04.762085 [ 1476 ] {} CacheMetadata: Key a6a165c0d77d77b701055e53eb5b8ed6 is removed from metadata 2024.10.22 00:37:04.767144 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cf9f431a4dc698b21ba174d0c921cfed for removal 2024.10.22 00:37:04.771607 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d60faacc241b1717574cfe11cf4ac2ba, offset: 0, size: 1168 2024.10.22 00:37:04.776915 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d60faacc241b1717574cfe11cf4ac2ba for removal 2024.10.22 00:37:04.779119 [ 1476 ] {} CacheMetadata: Key cf9f431a4dc698b21ba174d0c921cfed is removed from metadata 2024.10.22 00:37:04.782066 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c887ea267024be58abe8b822994a3d49, offset: 0, size: 141 2024.10.22 00:37:04.784334 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c887ea267024be58abe8b822994a3d49 for removal 2024.10.22 00:37:04.786364 [ 1476 ] {} CacheMetadata: Key d60faacc241b1717574cfe11cf4ac2ba is removed from metadata 2024.10.22 00:37:04.793224 [ 1476 ] {} CacheMetadata: Key c887ea267024be58abe8b822994a3d49 is removed from metadata 2024.10.22 00:37:05.167196 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_113_23 to 202410_1_113_23 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:05.171139 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_113_23} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_108_22, 202410_113_113_0] -> 202410_1_113_23 2024.10.22 00:37:05.228281 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:37:05.268698 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column EventTime 2024.10.22 00:37:05.279087 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column EventTime 2024.10.22 00:37:05.286567 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column EventTime 2024.10.22 00:37:05.329622 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column EventTime 2024.10.22 00:37:05.345211 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column EventTime 2024.10.22 00:37:05.363955 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column EventTime 2024.10.22 00:37:05.375669 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column EventTime 2024.10.22 00:37:05.576199 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/amw/ncxhweuvwcqggkwegaukcfczjbzks (LogSeriesLimiter: on interval from 2024-10-22 00:36:52 to 2024-10-22 00:37:05 accepted series 1 / 183 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:37:05.826529 [ 1204 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:37:05.870288 [ 1204 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:37:05.877141 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 3.61 MiB, will set to 1.65 GiB (RSS), difference: 1.37 MiB 2024.10.22 00:37:06.098331 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:06.101668 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:06.716320 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 217118 2024.10.22 00:37:07.048382 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 53 entries to flush up to offset 8469 2024.10.22 00:37:07.299789 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:07.308631 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:07.457752 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_128_128_0 (state Temporary) 2024.10.22 00:37:07.593752 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_128_128_0 to 202410_134_134_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:07.620754 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:07.623816 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:07.656259 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.83 GiB, free memory in arenas 4.07 MiB, will set to 1.66 GiB (RSS), difference: -4.01 MiB 2024.10.22 00:37:07.690096 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8469 2024.10.22 00:37:07.858341 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_124_124_0 (state Temporary) 2024.10.22 00:37:08.150765 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_124_124_0 to 202410_131_131_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:08.187394 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 217118 2024.10.22 00:37:08.188954 [ 1277 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_126_25 to 202410_131_131_0 2024.10.22 00:37:08.197223 [ 1277 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:08.206695 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:08.240496 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_126_25 to 202410_131_131_0 into Compact with storage Full 2024.10.22 00:37:08.285414 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:37:08.290489 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} MergeTreeSequentialSource: Reading 29 marks from part 202410_1_126_25, total 223412 rows starting from the beginning of the part 2024.10.22 00:37:08.317548 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_127_127_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:37:08.342774 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_128_128_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:37:08.367450 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_129_129_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:37:08.491396 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_130_130_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:37:08.554636 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 1ccf6191ac7f5aa9a6c26d32a0a37ae9:0) 2024.10.22 00:37:08.566212 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7bde876bfca3d8dea83a43bc5eefc71b, offset: 0, size: 1794232 2024.10.22 00:37:08.571050 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7bde876bfca3d8dea83a43bc5eefc71b for removal 2024.10.22 00:37:08.575276 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6f6dcfef2ee37f74b79a6809cdd9fb52, offset: 0, size: 194 2024.10.22 00:37:08.578320 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6f6dcfef2ee37f74b79a6809cdd9fb52 for removal 2024.10.22 00:37:08.621385 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_131_131_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:37:08.649541 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 11509 2024.10.22 00:37:08.674915 [ 1476 ] {} CacheMetadata: Key 7bde876bfca3d8dea83a43bc5eefc71b is removed from metadata 2024.10.22 00:37:08.684977 [ 1476 ] {} CacheMetadata: Key 6f6dcfef2ee37f74b79a6809cdd9fb52 is removed from metadata 2024.10.22 00:37:09.147028 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:09.151792 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:09.343155 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_223_223_0 (state Temporary) 2024.10.22 00:37:09.492992 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:37:09.494980 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:37:09.528505 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_26_26_0 (state Temporary) 2024.10.22 00:37:09.597150 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_223_223_0 to 202410_236_236_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:09.677017 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 11509 2024.10.22 00:37:09.702355 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.83 GiB, free memory in arenas 2.89 MiB, will set to 1.67 GiB (RSS), difference: -4.43 MiB 2024.10.22 00:37:09.742395 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_231_46 to 202410_236_236_0 2024.10.22 00:37:09.756865 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:09.763866 [ 1166 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:09.854748 [ 1155 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 00:37:09.868970 [ 1166 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 00:37:09.887978 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_231_46 to 202410_236_236_0 into Compact with storage Full 2024.10.22 00:37:09.939260 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:37:09.968549 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_231_46, total 11768 rows starting from the beginning of the part 2024.10.22 00:37:10.024918 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_232_232_0, total 285 rows starting from the beginning of the part 2024.10.22 00:37:10.080330 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_233_233_0, total 137 rows starting from the beginning of the part 2024.10.22 00:37:10.103626 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_234_234_0, total 39 rows starting from the beginning of the part 2024.10.22 00:37:10.163045 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_235_235_0, total 39 rows starting from the beginning of the part 2024.10.22 00:37:10.182290 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_236_236_0, total 72 rows starting from the beginning of the part 2024.10.22 00:37:10.313459 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 1ccf6191ac7f5aa9a6c26d32a0a37ae9:0) 2024.10.22 00:37:10.419094 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c83b0ad8973ddd91dbf2f016a24b5667, offset: 0, size: 322124 2024.10.22 00:37:10.425678 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c83b0ad8973ddd91dbf2f016a24b5667 for removal 2024.10.22 00:37:10.434892 [ 1476 ] {} CacheMetadata: Key c83b0ad8973ddd91dbf2f016a24b5667 is removed from metadata 2024.10.22 00:37:10.439961 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 55f33ebb5e2cb7e14cf5ee5d227449e6, offset: 0, size: 144 2024.10.22 00:37:10.445722 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 55f33ebb5e2cb7e14cf5ee5d227449e6 for removal 2024.10.22 00:37:10.569793 [ 1476 ] {} CacheMetadata: Key 55f33ebb5e2cb7e14cf5ee5d227449e6 is removed from metadata 2024.10.22 00:37:11.267599 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_70_14, 202410_71_71_0, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0] 2024.10.22 00:37:11.267697 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_236_47/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:37:11.278339 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_70_14, 202410_71_71_0, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0] 2024.10.22 00:37:11.321229 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_70_14, 202410_71_71_0, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0] 2024.10.22 00:37:11.635640 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:37:12.126307 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 1823 2024.10.22 00:37:12.337591 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} MergeTask::MergeProjectionsStage: Merge sorted 12340 rows, containing 13 columns (13 merged, 0 gathered) in 2.456620895 sec., 5023.160075335923 rows/sec., 765.15 KiB/sec. 2024.10.22 00:37:12.491774 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:12.493571 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:12.540482 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_130_130_0 (state Temporary) 2024.10.22 00:37:12.544161 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} MergedBlockOutputStream: filled checksums 202410_1_236_47 (state Temporary) 2024.10.22 00:37:12.665736 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_130_130_0 to 202410_143_143_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:12.696255 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1823 2024.10.22 00:37:12.927168 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_236_47 to 202410_1_236_47 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:12.966510 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_236_47} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_231_46, 202410_236_236_0] -> 202410_1_236_47 2024.10.22 00:37:12.980424 [ 1179 ] {} DatabaseCatalog: Execution took 296 ms. 2024.10.22 00:37:13.007583 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.71 MiB. 2024.10.22 00:37:13.412913 [ 1721 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bgr/onekilfvpngmqlhsnyflsnagkybfy, size 154 2024.10.22 00:37:13.447970 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_26_26_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:37:13.665623 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_131_26/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:37:13.690175 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 11561 2024.10.22 00:37:13.774743 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 9.96 MiB, will set to 1.68 GiB (RSS), difference: 54.34 MiB 2024.10.22 00:37:13.973664 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:13.976133 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.14 GiB. 2024.10.22 00:37:13.992422 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 35 file segments (while reserving 2097152 bytes for 1ccf6191ac7f5aa9a6c26d32a0a37ae9:0) 2024.10.22 00:37:13.996556 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 647e619df1653ffbc3135a76f77a6bdb, offset: 0, size: 217 2024.10.22 00:37:13.999521 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 647e619df1653ffbc3135a76f77a6bdb for removal 2024.10.22 00:37:14.004252 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 62203f4e55cada11184938b1345c4a51, offset: 0, size: 40431 2024.10.22 00:37:14.004549 [ 1476 ] {} CacheMetadata: Key 647e619df1653ffbc3135a76f77a6bdb is removed from metadata 2024.10.22 00:37:14.011202 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 62203f4e55cada11184938b1345c4a51 for removal 2024.10.22 00:37:14.021198 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 16e97d44e54362f4eda6fe43e2ce89ad, offset: 0, size: 43604 2024.10.22 00:37:14.031957 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 16e97d44e54362f4eda6fe43e2ce89ad for removal 2024.10.22 00:37:14.038297 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f452487423a01156714fdd50de91a137, offset: 0, size: 155 2024.10.22 00:37:14.054849 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f452487423a01156714fdd50de91a137 for removal 2024.10.22 00:37:14.066335 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 277b6f30be6141dfe896ab049b11f652, offset: 0, size: 4410 2024.10.22 00:37:14.073674 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 277b6f30be6141dfe896ab049b11f652 for removal 2024.10.22 00:37:14.079051 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 22997af84849f45c5e7ffab6a85a3f29, offset: 0, size: 154 2024.10.22 00:37:14.085587 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 22997af84849f45c5e7ffab6a85a3f29 for removal 2024.10.22 00:37:14.091937 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: defea7f0f93fd17c801f78ddc8afc3dc, offset: 0, size: 179985 2024.10.22 00:37:14.100211 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key defea7f0f93fd17c801f78ddc8afc3dc for removal 2024.10.22 00:37:14.110655 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f041df0c940b1d76a92a101c34c09a27, offset: 0, size: 187 2024.10.22 00:37:14.126256 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f041df0c940b1d76a92a101c34c09a27 for removal 2024.10.22 00:37:14.139554 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6df6a11135ec0d97fe6a7564a8d8b602, offset: 0, size: 153 2024.10.22 00:37:14.154543 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6df6a11135ec0d97fe6a7564a8d8b602 for removal 2024.10.22 00:37:14.259423 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ff3dd65c0fc733a5a8f00cc611ba4da1, offset: 0, size: 181218 2024.10.22 00:37:14.281769 [ 1476 ] {} CacheMetadata: Key 6df6a11135ec0d97fe6a7564a8d8b602 is removed from metadata 2024.10.22 00:37:14.290279 [ 1476 ] {} CacheMetadata: Key f041df0c940b1d76a92a101c34c09a27 is removed from metadata 2024.10.22 00:37:14.296825 [ 1476 ] {} CacheMetadata: Key defea7f0f93fd17c801f78ddc8afc3dc is removed from metadata 2024.10.22 00:37:14.315895 [ 1476 ] {} CacheMetadata: Key 22997af84849f45c5e7ffab6a85a3f29 is removed from metadata 2024.10.22 00:37:14.328440 [ 1476 ] {} CacheMetadata: Key 277b6f30be6141dfe896ab049b11f652 is removed from metadata 2024.10.22 00:37:14.343451 [ 1476 ] {} CacheMetadata: Key f452487423a01156714fdd50de91a137 is removed from metadata 2024.10.22 00:37:14.351506 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_224_224_0 (state Temporary) 2024.10.22 00:37:14.355736 [ 1476 ] {} CacheMetadata: Key 16e97d44e54362f4eda6fe43e2ce89ad is removed from metadata 2024.10.22 00:37:14.265725 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ff3dd65c0fc733a5a8f00cc611ba4da1 for removal 2024.10.22 00:37:14.374013 [ 1476 ] {} CacheMetadata: Key 62203f4e55cada11184938b1345c4a51 is removed from metadata 2024.10.22 00:37:14.384155 [ 1476 ] {} CacheMetadata: Key ff3dd65c0fc733a5a8f00cc611ba4da1 is removed from metadata 2024.10.22 00:37:14.403227 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_224_224_0 to 202410_237_237_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:14.471469 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f09e206a0e02a54824d0d2f528228f5d, offset: 0, size: 84070 2024.10.22 00:37:14.475254 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f09e206a0e02a54824d0d2f528228f5d for removal 2024.10.22 00:37:14.481676 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 322b9cb56e942b94b80f1dd941f87815, offset: 0, size: 89477 2024.10.22 00:37:14.486701 [ 1476 ] {} CacheMetadata: Key f09e206a0e02a54824d0d2f528228f5d is removed from metadata 2024.10.22 00:37:14.487125 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 322b9cb56e942b94b80f1dd941f87815 for removal 2024.10.22 00:37:14.495017 [ 1476 ] {} CacheMetadata: Key 322b9cb56e942b94b80f1dd941f87815 is removed from metadata 2024.10.22 00:37:14.495380 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5bf056c95400bf4e3d587128b4b1c409, offset: 0, size: 300484 2024.10.22 00:37:14.512809 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5bf056c95400bf4e3d587128b4b1c409 for removal 2024.10.22 00:37:14.524048 [ 1476 ] {} CacheMetadata: Key 5bf056c95400bf4e3d587128b4b1c409 is removed from metadata 2024.10.22 00:37:14.525264 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6a7dc223d9d96bf1e9a0490d0ec44b39, offset: 0, size: 216 2024.10.22 00:37:14.533573 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6a7dc223d9d96bf1e9a0490d0ec44b39 for removal 2024.10.22 00:37:14.536789 [ 1476 ] {} CacheMetadata: Key 6a7dc223d9d96bf1e9a0490d0ec44b39 is removed from metadata 2024.10.22 00:37:14.536789 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1b0a77bc9953d8c24aacdb8a4229e6e2, offset: 0, size: 153 2024.10.22 00:37:14.544892 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1b0a77bc9953d8c24aacdb8a4229e6e2 for removal 2024.10.22 00:37:14.549193 [ 1476 ] {} CacheMetadata: Key 1b0a77bc9953d8c24aacdb8a4229e6e2 is removed from metadata 2024.10.22 00:37:14.549592 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 562b697101a97ae04b877fd8ab397d32, offset: 0, size: 185 2024.10.22 00:37:14.555363 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 562b697101a97ae04b877fd8ab397d32 for removal 2024.10.22 00:37:14.563108 [ 1476 ] {} CacheMetadata: Key 562b697101a97ae04b877fd8ab397d32 is removed from metadata 2024.10.22 00:37:14.563071 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7690d09ea8e5735388d1f284ca46cf7c, offset: 0, size: 176964 2024.10.22 00:37:14.669348 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7690d09ea8e5735388d1f284ca46cf7c for removal 2024.10.22 00:37:14.670382 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 11561 2024.10.22 00:37:14.671901 [ 1476 ] {} CacheMetadata: Key 7690d09ea8e5735388d1f284ca46cf7c is removed from metadata 2024.10.22 00:37:14.672877 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9bd43181785dce7a573cc777ec395447, offset: 0, size: 181 2024.10.22 00:37:14.675612 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9bd43181785dce7a573cc777ec395447 for removal 2024.10.22 00:37:14.679625 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 683a61885627f745ba4dcb3873d694f3, offset: 0, size: 156 2024.10.22 00:37:14.681013 [ 1476 ] {} CacheMetadata: Key 9bd43181785dce7a573cc777ec395447 is removed from metadata 2024.10.22 00:37:14.682559 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 683a61885627f745ba4dcb3873d694f3 for removal 2024.10.22 00:37:14.687426 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9ad01681ed32f5b8f404185dee023660, offset: 0, size: 121709 2024.10.22 00:37:14.691268 [ 1476 ] {} CacheMetadata: Key 683a61885627f745ba4dcb3873d694f3 is removed from metadata 2024.10.22 00:37:14.697452 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9ad01681ed32f5b8f404185dee023660 for removal 2024.10.22 00:37:14.709565 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 50964af4f453e349bb00db8a56b350db, offset: 0, size: 6444 2024.10.22 00:37:14.710391 [ 1476 ] {} CacheMetadata: Key 9ad01681ed32f5b8f404185dee023660 is removed from metadata 2024.10.22 00:37:14.720388 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 50964af4f453e349bb00db8a56b350db for removal 2024.10.22 00:37:14.728277 [ 1476 ] {} CacheMetadata: Key 50964af4f453e349bb00db8a56b350db is removed from metadata 2024.10.22 00:37:14.734308 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 87e58950b6188625b1a028805462f6d3, offset: 0, size: 155 2024.10.22 00:37:14.746682 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 87e58950b6188625b1a028805462f6d3 for removal 2024.10.22 00:37:14.762933 [ 1476 ] {} CacheMetadata: Key 87e58950b6188625b1a028805462f6d3 is removed from metadata 2024.10.22 00:37:14.769440 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4cb27f46a60bfa78eb54e6a3fa54e9f9, offset: 0, size: 154 2024.10.22 00:37:14.774238 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4cb27f46a60bfa78eb54e6a3fa54e9f9 for removal 2024.10.22 00:37:14.781636 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bea39a1440548db8a6f252196ddd0c59, offset: 0, size: 221 2024.10.22 00:37:14.787752 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bea39a1440548db8a6f252196ddd0c59 for removal 2024.10.22 00:37:14.792312 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d9f41e04c8b9baed8c74c910661157c8, offset: 0, size: 4 2024.10.22 00:37:14.802202 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d9f41e04c8b9baed8c74c910661157c8 for removal 2024.10.22 00:37:14.816206 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b3fd9d1412cffb0aa69188fbae31ccdc, offset: 0, size: 103847 2024.10.22 00:37:14.820215 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b3fd9d1412cffb0aa69188fbae31ccdc for removal 2024.10.22 00:37:14.825764 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ff937e66fa80361b573262109c581e9a, offset: 0, size: 3440 2024.10.22 00:37:14.831708 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ff937e66fa80361b573262109c581e9a for removal 2024.10.22 00:37:14.836047 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: af88dede9591ea25764a6f433b45fbb6, offset: 0, size: 393592 2024.10.22 00:37:14.855995 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key af88dede9591ea25764a6f433b45fbb6 for removal 2024.10.22 00:37:14.873427 [ 1476 ] {} CacheMetadata: Key ff937e66fa80361b573262109c581e9a is removed from metadata 2024.10.22 00:37:14.880657 [ 1476 ] {} CacheMetadata: Key b3fd9d1412cffb0aa69188fbae31ccdc is removed from metadata 2024.10.22 00:37:14.968353 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ede61ef5aceea8505d8b857231205ed6, offset: 0, size: 157 2024.10.22 00:37:14.972864 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ede61ef5aceea8505d8b857231205ed6 for removal 2024.10.22 00:37:14.981103 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b5a1c38e927ddb944612886786e9b28d, offset: 0, size: 51869 2024.10.22 00:37:14.985597 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b5a1c38e927ddb944612886786e9b28d for removal 2024.10.22 00:37:14.990630 [ 1476 ] {} CacheMetadata: Key ede61ef5aceea8505d8b857231205ed6 is removed from metadata 2024.10.22 00:37:14.997460 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2135df912478af1aa7751233b0b27d6c, offset: 0, size: 152 2024.10.22 00:37:15.006760 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2135df912478af1aa7751233b0b27d6c for removal 2024.10.22 00:37:15.015311 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c658da3ecc0aa07006b799a8e3c0297e, offset: 0, size: 256981 2024.10.22 00:37:15.022477 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c658da3ecc0aa07006b799a8e3c0297e for removal 2024.10.22 00:37:15.027605 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ac36f5d3d4b926ecda34036290146b91, offset: 0, size: 141 2024.10.22 00:37:15.035237 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ac36f5d3d4b926ecda34036290146b91 for removal 2024.10.22 00:37:15.043466 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8b34300229acc194f2f3595475372a07, offset: 0, size: 45026 2024.10.22 00:37:15.058901 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8b34300229acc194f2f3595475372a07 for removal 2024.10.22 00:37:15.075609 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 64a0356121238aae07392ba46f65c7d7, offset: 0, size: 157 2024.10.22 00:37:15.081795 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 64a0356121238aae07392ba46f65c7d7 for removal 2024.10.22 00:37:15.113680 [ 1476 ] {} CacheMetadata: Key 64a0356121238aae07392ba46f65c7d7 is removed from metadata 2024.10.22 00:37:15.127783 [ 1476 ] {} CacheMetadata: Key 8b34300229acc194f2f3595475372a07 is removed from metadata 2024.10.22 00:37:15.152086 [ 1476 ] {} CacheMetadata: Key ac36f5d3d4b926ecda34036290146b91 is removed from metadata 2024.10.22 00:37:15.195502 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 218270 2024.10.22 00:37:15.197763 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 96 entries to flush up to offset 8565 2024.10.22 00:37:15.271377 [ 1476 ] {} CacheMetadata: Key c658da3ecc0aa07006b799a8e3c0297e is removed from metadata 2024.10.22 00:37:15.292069 [ 1476 ] {} CacheMetadata: Key 2135df912478af1aa7751233b0b27d6c is removed from metadata 2024.10.22 00:37:15.302226 [ 1476 ] {} CacheMetadata: Key b5a1c38e927ddb944612886786e9b28d is removed from metadata 2024.10.22 00:37:15.322954 [ 1476 ] {} CacheMetadata: Key af88dede9591ea25764a6f433b45fbb6 is removed from metadata 2024.10.22 00:37:15.333739 [ 1476 ] {} CacheMetadata: Key d9f41e04c8b9baed8c74c910661157c8 is removed from metadata 2024.10.22 00:37:15.347988 [ 1476 ] {} CacheMetadata: Key bea39a1440548db8a6f252196ddd0c59 is removed from metadata 2024.10.22 00:37:15.390386 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:15.395415 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.13 GiB. 2024.10.22 00:37:15.472875 [ 1476 ] {} CacheMetadata: Key 4cb27f46a60bfa78eb54e6a3fa54e9f9 is removed from metadata 2024.10.22 00:37:15.479977 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_129_129_0 (state Temporary) 2024.10.22 00:37:15.559172 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_131_26, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0] 2024.10.22 00:37:15.592953 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_131_26, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0] 2024.10.22 00:37:15.636564 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_131_26, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0] 2024.10.22 00:37:15.694904 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_129_129_0 to 202410_135_135_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:15.774893 [ 1203 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_130_26 to 202410_135_135_0 2024.10.22 00:37:15.789322 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8565 2024.10.22 00:37:15.791656 [ 1203 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:15.800323 [ 1203 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.13 GiB. 2024.10.22 00:37:15.822826 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_130_26 to 202410_135_135_0 into Compact with storage Full 2024.10.22 00:37:15.906367 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:37:15.913362 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:15.920077 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.13 GiB. 2024.10.22 00:37:15.926358 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_130_26, total 8377 rows starting from the beginning of the part 2024.10.22 00:37:15.980033 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_131_131_0, total 50 rows starting from the beginning of the part 2024.10.22 00:37:16.009790 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_132_132_0, total 47 rows starting from the beginning of the part 2024.10.22 00:37:16.197138 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_133_133_0, total 46 rows starting from the beginning of the part 2024.10.22 00:37:16.292610 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_125_125_0 (state Temporary) 2024.10.22 00:37:16.294744 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_134_134_0, total 53 rows starting from the beginning of the part 2024.10.22 00:37:16.346723 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_135_135_0, total 96 rows starting from the beginning of the part 2024.10.22 00:37:16.522776 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_125_125_0 to 202410_132_132_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:16.571392 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 218270 2024.10.22 00:37:16.676132 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 122 2024.10.22 00:37:16.691425 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.83 GiB, free memory in arenas 10.00 MiB, will set to 1.67 GiB (RSS), difference: 176.97 MiB 2024.10.22 00:37:17.070664 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:17.085577 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.12 GiB. 2024.10.22 00:37:17.287656 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_104_104_0 (state Temporary) 2024.10.22 00:37:17.423743 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_104_104_0 to 202410_114_114_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:17.559508 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 122 2024.10.22 00:37:18.555286 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_10_10_0 (state Temporary) 2024.10.22 00:37:18.573317 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hql/kcpwljkgzmiszsbtygnzxgqswurau (LogSeriesLimiter: on interval from 2024-10-22 00:37:05 to 2024-10-22 00:37:18 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:37:18.585461 [ 1707 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qqa/uvffilwlwxrllldpsvznikrurrauv, size 312 2024.10.22 00:37:18.599916 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.83 GiB, free memory in arenas 13.18 MiB, will set to 1.66 GiB (RSS), difference: 138.44 MiB 2024.10.22 00:37:18.626626 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} MergeTask::MergeProjectionsStage: Merge sorted 8669 rows, containing 13 columns (13 merged, 0 gathered) in 2.810283147 sec., 3084.742549609006 rows/sec., 1.29 MiB/sec. 2024.10.22 00:37:18.640776 [ 1707 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hql/kcpwljkgzmiszsbtygnzxgqswurau, size 4 2024.10.22 00:37:18.777587 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 152 entries to flush up to offset 11713 2024.10.22 00:37:18.780424 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} MergedBlockOutputStream: filled checksums 202410_1_135_27 (state Temporary) 2024.10.22 00:37:18.855998 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_135_27 to 202410_1_135_27 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:18.865309 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_135_27} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_130_26, 202410_135_135_0] -> 202410_1_135_27 2024.10.22 00:37:18.901279 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.88 MiB. 2024.10.22 00:37:19.083405 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} MergeTask::MergeProjectionsStage: Merge sorted 230708 rows, containing 4 columns (4 merged, 0 gathered) in 10.856592186 sec., 21250.498871782896 rows/sec., 311.94 KiB/sec. 2024.10.22 00:37:19.177670 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} MergedBlockOutputStream: filled checksums 202410_1_131_26 (state Temporary) 2024.10.22 00:37:19.275726 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:19.281110 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.10 GiB. 2024.10.22 00:37:19.332490 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_131_26 to 202410_1_131_26 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:19.385026 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_131_26} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_126_25, 202410_131_131_0] -> 202410_1_131_26 2024.10.22 00:37:19.450266 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:37:19.573057 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_225_225_0 (state Temporary) 2024.10.22 00:37:19.597729 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:37:19.609429 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:37:19.728201 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_225_225_0 to 202410_238_238_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:19.911177 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 11713 2024.10.22 00:37:20.095442 [ 1244 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 00:37:20.199078 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 1850 2024.10.22 00:37:20.256735 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_26_26_0 to 202410_29_29_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:20.283764 [ 1183 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_24_5 to 202410_29_29_0 2024.10.22 00:37:20.290958 [ 1183 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:20.293326 [ 1183 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.10 GiB. 2024.10.22 00:37:20.420006 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_24_5 to 202410_29_29_0 into Compact with storage Full 2024.10.22 00:37:20.421071 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:20.427243 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.10 GiB. 2024.10.22 00:37:20.483710 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_131_131_0 (state Temporary) 2024.10.22 00:37:20.576220 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_131_131_0 to 202410_144_144_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:20.640521 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1850 2024.10.22 00:37:20.666706 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 10.45 MiB, will set to 1.64 GiB (RSS), difference: 7.68 MiB 2024.10.22 00:37:20.876275 [ 1184 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:37:21.011430 [ 1184 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:37:21.843371 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 28 file segments (while reserving 4194304 bytes for 1ccf6191ac7f5aa9a6c26d32a0a37ae9:0) 2024.10.22 00:37:21.846899 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: add4ce26eb98bcd83fc7b2110e043ead, offset: 0, size: 501683 2024.10.22 00:37:21.854375 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key add4ce26eb98bcd83fc7b2110e043ead for removal 2024.10.22 00:37:21.860800 [ 1476 ] {} CacheMetadata: Key add4ce26eb98bcd83fc7b2110e043ead is removed from metadata 2024.10.22 00:37:21.861034 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fa3d1d8104f78b7d0802f42696102f38, offset: 0, size: 214 2024.10.22 00:37:21.971169 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fa3d1d8104f78b7d0802f42696102f38 for removal 2024.10.22 00:37:21.974445 [ 1476 ] {} CacheMetadata: Key fa3d1d8104f78b7d0802f42696102f38 is removed from metadata 2024.10.22 00:37:21.974325 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e61ce98fb6f70dd29deae624e999c4c9, offset: 0, size: 81637 2024.10.22 00:37:21.978531 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e61ce98fb6f70dd29deae624e999c4c9 for removal 2024.10.22 00:37:21.983801 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 994adaf79427f2c7911f2bf02afb2457, offset: 0, size: 80895 2024.10.22 00:37:21.984379 [ 1476 ] {} CacheMetadata: Key e61ce98fb6f70dd29deae624e999c4c9 is removed from metadata 2024.10.22 00:37:21.990596 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 994adaf79427f2c7911f2bf02afb2457 for removal 2024.10.22 00:37:21.994844 [ 1476 ] {} CacheMetadata: Key 994adaf79427f2c7911f2bf02afb2457 is removed from metadata 2024.10.22 00:37:21.995442 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 16aed25e13ecc641c7ccf99dd92e0302, offset: 0, size: 158 2024.10.22 00:37:22.012056 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 16aed25e13ecc641c7ccf99dd92e0302 for removal 2024.10.22 00:37:22.019523 [ 1476 ] {} CacheMetadata: Key 16aed25e13ecc641c7ccf99dd92e0302 is removed from metadata 2024.10.22 00:37:22.021447 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bd21609623b502989354c0edd1ce363d, offset: 0, size: 192707 2024.10.22 00:37:22.026885 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bd21609623b502989354c0edd1ce363d for removal 2024.10.22 00:37:22.032080 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b0c43ce8890fc4adca029ed5b84fb6a0, offset: 0, size: 187 2024.10.22 00:37:22.032629 [ 1476 ] {} CacheMetadata: Key bd21609623b502989354c0edd1ce363d is removed from metadata 2024.10.22 00:37:22.038190 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b0c43ce8890fc4adca029ed5b84fb6a0 for removal 2024.10.22 00:37:22.046623 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c31e5a3ed99d8926c127d8f54dcf57cd, offset: 0, size: 214 2024.10.22 00:37:22.054310 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c31e5a3ed99d8926c127d8f54dcf57cd for removal 2024.10.22 00:37:22.055547 [ 1476 ] {} CacheMetadata: Key b0c43ce8890fc4adca029ed5b84fb6a0 is removed from metadata 2024.10.22 00:37:22.066805 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6883cf503f1d75bd4d78658d8986324e, offset: 0, size: 156 2024.10.22 00:37:22.071683 [ 1476 ] {} CacheMetadata: Key c31e5a3ed99d8926c127d8f54dcf57cd is removed from metadata 2024.10.22 00:37:22.072152 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6883cf503f1d75bd4d78658d8986324e for removal 2024.10.22 00:37:22.076556 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 17366dbe4b3ed1543b584601888861ed, offset: 0, size: 128748 2024.10.22 00:37:22.082903 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 17366dbe4b3ed1543b584601888861ed for removal 2024.10.22 00:37:22.085132 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a95323d61bb6085990b93eb3d9951768, offset: 0, size: 212 2024.10.22 00:37:22.087334 [ 1476 ] {} CacheMetadata: Key 17366dbe4b3ed1543b584601888861ed is removed from metadata 2024.10.22 00:37:22.088298 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a95323d61bb6085990b93eb3d9951768 for removal 2024.10.22 00:37:22.091662 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 36f46783e4ce3f7047f108af3470a44c, offset: 0, size: 241 2024.10.22 00:37:22.095390 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 36f46783e4ce3f7047f108af3470a44c for removal 2024.10.22 00:37:22.098330 [ 1476 ] {} CacheMetadata: Key 36f46783e4ce3f7047f108af3470a44c is removed from metadata 2024.10.22 00:37:22.098330 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b335bfa6c011cdfc4823cc172e25babf, offset: 0, size: 158 2024.10.22 00:37:22.103450 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b335bfa6c011cdfc4823cc172e25babf for removal 2024.10.22 00:37:22.108024 [ 1476 ] {} CacheMetadata: Key b335bfa6c011cdfc4823cc172e25babf is removed from metadata 2024.10.22 00:37:22.108029 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e78a0e8a5df7ce29088c6cfce344a11b, offset: 0, size: 192654 2024.10.22 00:37:22.114485 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e78a0e8a5df7ce29088c6cfce344a11b for removal 2024.10.22 00:37:22.119374 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fc12b8df80c9d41efbd9aba346830edf, offset: 0, size: 345724 2024.10.22 00:37:22.123819 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fc12b8df80c9d41efbd9aba346830edf for removal 2024.10.22 00:37:22.127853 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1d34f785297a58eb3de950013801ee15, offset: 0, size: 189 2024.10.22 00:37:22.133529 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1d34f785297a58eb3de950013801ee15 for removal 2024.10.22 00:37:22.137591 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8ac6969a2ebd018e0fe4134e30ae5104, offset: 0, size: 156 2024.10.22 00:37:22.141261 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8ac6969a2ebd018e0fe4134e30ae5104 for removal 2024.10.22 00:37:22.149758 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fdf0e36cd15cb22658bf767b38f39582, offset: 0, size: 107507 2024.10.22 00:37:22.159056 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fdf0e36cd15cb22658bf767b38f39582 for removal 2024.10.22 00:37:22.165050 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d6e9e0b21d53f7e663eafd3ad6a9fc01, offset: 0, size: 69219 2024.10.22 00:37:22.173162 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d6e9e0b21d53f7e663eafd3ad6a9fc01 for removal 2024.10.22 00:37:22.223449 [ 1476 ] {} CacheMetadata: Key fdf0e36cd15cb22658bf767b38f39582 is removed from metadata 2024.10.22 00:37:22.240472 [ 1476 ] {} CacheMetadata: Key 8ac6969a2ebd018e0fe4134e30ae5104 is removed from metadata 2024.10.22 00:37:22.277013 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: df769d7d89629c4791d1a8b21a0816c1, offset: 0, size: 190 2024.10.22 00:37:22.291089 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key df769d7d89629c4791d1a8b21a0816c1 for removal 2024.10.22 00:37:22.295839 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 161f112ba2d3aa357177f7afdc6642ad, offset: 0, size: 179 2024.10.22 00:37:22.301398 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 161f112ba2d3aa357177f7afdc6642ad for removal 2024.10.22 00:37:22.314623 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d9ea42c80049b7912b860d68d09265d2, offset: 0, size: 1662391 2024.10.22 00:37:22.321566 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d9ea42c80049b7912b860d68d09265d2 for removal 2024.10.22 00:37:22.327358 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b2957510ca5a0e69c8c03daca0812395, offset: 0, size: 141843 2024.10.22 00:37:22.368186 [ 1476 ] {} CacheMetadata: Key d9ea42c80049b7912b860d68d09265d2 is removed from metadata 2024.10.22 00:37:22.383466 [ 1476 ] {} CacheMetadata: Key 161f112ba2d3aa357177f7afdc6642ad is removed from metadata 2024.10.22 00:37:22.391395 [ 1476 ] {} CacheMetadata: Key df769d7d89629c4791d1a8b21a0816c1 is removed from metadata 2024.10.22 00:37:22.398782 [ 1476 ] {} CacheMetadata: Key d6e9e0b21d53f7e663eafd3ad6a9fc01 is removed from metadata 2024.10.22 00:37:22.435128 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b2957510ca5a0e69c8c03daca0812395 for removal 2024.10.22 00:37:22.438129 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 786627c4b98ed7f01f24dc27da6f6c33, offset: 0, size: 330042 2024.10.22 00:37:22.442728 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 786627c4b98ed7f01f24dc27da6f6c33 for removal 2024.10.22 00:37:22.448213 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c077b6d0c6399908a0f327e5f6e1621c, offset: 0, size: 254065 2024.10.22 00:37:22.497456 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 8.55 MiB, will set to 1.64 GiB (RSS), difference: -1.28 MiB 2024.10.22 00:37:22.529337 [ 1476 ] {} CacheMetadata: Key 1d34f785297a58eb3de950013801ee15 is removed from metadata 2024.10.22 00:37:22.548295 [ 1476 ] {} CacheMetadata: Key 786627c4b98ed7f01f24dc27da6f6c33 is removed from metadata 2024.10.22 00:37:22.567470 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c077b6d0c6399908a0f327e5f6e1621c for removal 2024.10.22 00:37:22.571500 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f4c1f5a6a311432f6871612ca6907cc9, offset: 0, size: 106456 2024.10.22 00:37:22.574900 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f4c1f5a6a311432f6871612ca6907cc9 for removal 2024.10.22 00:37:22.577659 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b331d14c78d39e943e58d8814685cb33, offset: 0, size: 155 2024.10.22 00:37:22.581314 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b331d14c78d39e943e58d8814685cb33 for removal 2024.10.22 00:37:22.584321 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cea91d6416420160c6776659e317e646, offset: 0, size: 161 2024.10.22 00:37:22.588670 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cea91d6416420160c6776659e317e646 for removal 2024.10.22 00:37:22.665509 [ 1476 ] {} CacheMetadata: Key cea91d6416420160c6776659e317e646 is removed from metadata 2024.10.22 00:37:22.679053 [ 1476 ] {} CacheMetadata: Key b331d14c78d39e943e58d8814685cb33 is removed from metadata 2024.10.22 00:37:22.689117 [ 1476 ] {} CacheMetadata: Key f4c1f5a6a311432f6871612ca6907cc9 is removed from metadata 2024.10.22 00:37:22.698367 [ 1476 ] {} CacheMetadata: Key c077b6d0c6399908a0f327e5f6e1621c is removed from metadata 2024.10.22 00:37:22.714723 [ 1476 ] {} CacheMetadata: Key b2957510ca5a0e69c8c03daca0812395 is removed from metadata 2024.10.22 00:37:22.731252 [ 1476 ] {} CacheMetadata: Key fc12b8df80c9d41efbd9aba346830edf is removed from metadata 2024.10.22 00:37:22.742859 [ 1476 ] {} CacheMetadata: Key e78a0e8a5df7ce29088c6cfce344a11b is removed from metadata 2024.10.22 00:37:22.759852 [ 1476 ] {} CacheMetadata: Key a95323d61bb6085990b93eb3d9951768 is removed from metadata 2024.10.22 00:37:22.771880 [ 1476 ] {} CacheMetadata: Key 6883cf503f1d75bd4d78658d8986324e is removed from metadata 2024.10.22 00:37:23.313772 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 82 entries to flush up to offset 8647 2024.10.22 00:37:23.441326 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:23.456732 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.10 GiB. 2024.10.22 00:37:23.516668 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_130_130_0 (state Temporary) 2024.10.22 00:37:23.577526 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 220190 2024.10.22 00:37:23.669588 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_130_130_0 to 202410_136_136_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:23.717355 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8647 2024.10.22 00:37:23.819352 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1025 2024.10.22 00:37:23.942501 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 126 entries to flush up to offset 11839 2024.10.22 00:37:24.184452 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:24.187241 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.10 GiB. 2024.10.22 00:37:24.389206 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_226_226_0 (state Temporary) 2024.10.22 00:37:24.433589 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:24.494485 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.10 GiB. 2024.10.22 00:37:24.553092 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.83 GiB, free memory in arenas 9.84 MiB, will set to 1.64 GiB (RSS), difference: -8.08 MiB 2024.10.22 00:37:24.568586 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_226_226_0 to 202410_239_239_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:24.641189 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 11839 2024.10.22 00:37:24.865457 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_126_126_0 (state Temporary) 2024.10.22 00:37:25.063543 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 123 2024.10.22 00:37:25.171779 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_126_126_0 to 202410_133_133_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:25.224194 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 220190 2024.10.22 00:37:25.590799 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:25.593630 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.10 GiB. 2024.10.22 00:37:25.797347 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_105_105_0 (state Temporary) 2024.10.22 00:37:26.323351 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_105_105_0 to 202410_115_115_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:26.428472 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 123 2024.10.22 00:37:26.637722 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:37:26.659126 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 9.22 MiB, will set to 1.64 GiB (RSS), difference: 3.89 MiB 2024.10.22 00:37:27.200583 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 10 file segments (while reserving 2260176 bytes for 1ccf6191ac7f5aa9a6c26d32a0a37ae9:0) 2024.10.22 00:37:27.203850 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 218d6b0eb2b5dcda351d367f6974b026, offset: 0, size: 1648287 2024.10.22 00:37:27.210987 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 218d6b0eb2b5dcda351d367f6974b026 for removal 2024.10.22 00:37:27.217575 [ 1476 ] {} CacheMetadata: Key 218d6b0eb2b5dcda351d367f6974b026 is removed from metadata 2024.10.22 00:37:27.219420 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 30ca1ead426235bccff02d55011860af, offset: 0, size: 5805 2024.10.22 00:37:27.224389 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 30ca1ead426235bccff02d55011860af for removal 2024.10.22 00:37:27.232848 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 655ecb6980489001c3a7b8c06f94efe5, offset: 0, size: 1159556 2024.10.22 00:37:27.242133 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 655ecb6980489001c3a7b8c06f94efe5 for removal 2024.10.22 00:37:27.253371 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5620615a32ca959bf57ab014f7240c24, offset: 0, size: 158 2024.10.22 00:37:27.259484 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5620615a32ca959bf57ab014f7240c24 for removal 2024.10.22 00:37:27.259859 [ 1476 ] {} CacheMetadata: Key 655ecb6980489001c3a7b8c06f94efe5 is removed from metadata 2024.10.22 00:37:27.268698 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 00e3f27e652963239da83c4675aa231a, offset: 0, size: 76722 2024.10.22 00:37:27.274533 [ 1476 ] {} CacheMetadata: Key 5620615a32ca959bf57ab014f7240c24 is removed from metadata 2024.10.22 00:37:27.276646 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 00e3f27e652963239da83c4675aa231a for removal 2024.10.22 00:37:27.281601 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c5265640a48dc3745087fa50ea6a99e2, offset: 0, size: 241 2024.10.22 00:37:27.290938 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c5265640a48dc3745087fa50ea6a99e2 for removal 2024.10.22 00:37:27.293686 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 57fa6cbdcaf9dd23fc60ad01f5b1cdf0, offset: 0, size: 152 2024.10.22 00:37:27.295543 [ 1476 ] {} CacheMetadata: Key c5265640a48dc3745087fa50ea6a99e2 is removed from metadata 2024.10.22 00:37:27.297618 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 57fa6cbdcaf9dd23fc60ad01f5b1cdf0 for removal 2024.10.22 00:37:27.303493 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b4eee2c473f8e3806a26533b9673996e, offset: 0, size: 154 2024.10.22 00:37:27.313217 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b4eee2c473f8e3806a26533b9673996e for removal 2024.10.22 00:37:27.317033 [ 1476 ] {} CacheMetadata: Key 57fa6cbdcaf9dd23fc60ad01f5b1cdf0 is removed from metadata 2024.10.22 00:37:27.320801 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8a13ffbed82bb94ce09a7312d84d0826, offset: 0, size: 196149 2024.10.22 00:37:27.325221 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8a13ffbed82bb94ce09a7312d84d0826 for removal 2024.10.22 00:37:27.330318 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 76f999c26dda51a277c96087a1552a66, offset: 0, size: 234 2024.10.22 00:37:27.339291 [ 1476 ] {} CacheMetadata: Key 8a13ffbed82bb94ce09a7312d84d0826 is removed from metadata 2024.10.22 00:37:27.335887 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 76f999c26dda51a277c96087a1552a66 for removal 2024.10.22 00:37:27.355974 [ 1476 ] {} CacheMetadata: Key 76f999c26dda51a277c96087a1552a66 is removed from metadata 2024.10.22 00:37:27.374502 [ 1476 ] {} CacheMetadata: Key b4eee2c473f8e3806a26533b9673996e is removed from metadata 2024.10.22 00:37:27.382637 [ 1476 ] {} CacheMetadata: Key 00e3f27e652963239da83c4675aa231a is removed from metadata 2024.10.22 00:37:27.416123 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_71_14, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0, 202410_76_76_0] 2024.10.22 00:37:27.421018 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_71_14, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0, 202410_76_76_0] 2024.10.22 00:37:27.491767 [ 1476 ] {} CacheMetadata: Key 30ca1ead426235bccff02d55011860af is removed from metadata 2024.10.22 00:37:27.570884 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_71_14, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0, 202410_75_75_0, 202410_76_76_0] 2024.10.22 00:37:27.600108 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:37:27.608018 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_24_5, total 878 rows starting from the beginning of the part 2024.10.22 00:37:27.618547 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:37:27.673134 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:37:27.691149 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:37:27.806210 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:37:27.825710 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:37:27.851834 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:37:27.901842 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/dlb/dcmwipuqozbpxptksqwgxblqttniq (LogSeriesLimiter: on interval from 2024-10-22 00:37:18 to 2024-10-22 00:37:27 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:37:28.145654 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 1856 2024.10.22 00:37:28.173416 [ 1734 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/amw/ncxhweuvwcqggkwegaukcfczjbzks, size 10648784 2024.10.22 00:37:28.351324 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:28.355424 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:28.392568 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_132_132_0 (state Temporary) 2024.10.22 00:37:28.645185 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_132_132_0 to 202410_145_145_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:28.664285 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 11908 2024.10.22 00:37:28.692996 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1856 2024.10.22 00:37:28.927103 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 9.29 MiB, will set to 1.64 GiB (RSS), difference: 8.44 MiB 2024.10.22 00:37:28.996960 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:29.003861 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:29.175951 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_227_227_0 (state Temporary) 2024.10.22 00:37:29.219335 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_227_227_0 to 202410_240_240_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:29.496400 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:37:29.503067 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:37:29.529589 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 11908 2024.10.22 00:37:29.750875 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_25_25_0, total 45 rows starting from the beginning of the part 2024.10.22 00:37:30.438564 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 8.11 MiB, will set to 1.63 GiB (RSS), difference: -7.30 MiB 2024.10.22 00:37:31.061057 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_10_10_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:37:31.236513 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 40 entries to flush up to offset 8687 2024.10.22 00:37:31.353373 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:31.358263 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:31.391177 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_131_131_0 (state Temporary) 2024.10.22 00:37:31.477671 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 45 entries to flush up to offset 1070 2024.10.22 00:37:31.534848 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_131_131_0 to 202410_137_137_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:31.695098 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8687 2024.10.22 00:37:31.792347 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_26_26_0, total 39 rows starting from the beginning of the part 2024.10.22 00:37:32.236042 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 221726 2024.10.22 00:37:33.004417 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:33.020013 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:33.361766 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_127_127_0 (state Temporary) 2024.10.22 00:37:33.366457 [ 1176 ] {} DatabaseCatalog: Execution took 238 ms. 2024.10.22 00:37:33.630198 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_127_127_0 to 202410_134_134_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:33.642177 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 11934 2024.10.22 00:37:33.680833 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 221726 2024.10.22 00:37:33.698404 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_27_27_0, total 43 rows starting from the beginning of the part 2024.10.22 00:37:33.887726 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:33.890702 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:33.930851 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 124 2024.10.22 00:37:33.939569 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_228_228_0 (state Temporary) 2024.10.22 00:37:33.993211 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_228_228_0 to 202410_241_241_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:34.023644 [ 1178 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_236_47 to 202410_241_241_0 2024.10.22 00:37:34.038882 [ 1178 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:34.042897 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:34.066391 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_236_47 to 202410_241_241_0 into Compact with storage Full 2024.10.22 00:37:34.096744 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:37:34.111502 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_236_47, total 12340 rows starting from the beginning of the part 2024.10.22 00:37:34.138065 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_237_237_0, total 52 rows starting from the beginning of the part 2024.10.22 00:37:34.150878 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 11934 2024.10.22 00:37:34.170272 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_238_238_0, total 152 rows starting from the beginning of the part 2024.10.22 00:37:34.181519 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_239_239_0, total 126 rows starting from the beginning of the part 2024.10.22 00:37:34.212666 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_240_240_0, total 69 rows starting from the beginning of the part 2024.10.22 00:37:34.240380 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:34.238353 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_241_241_0, total 26 rows starting from the beginning of the part 2024.10.22 00:37:34.249801 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:34.315094 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_106_106_0 (state Temporary) 2024.10.22 00:37:34.382906 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_81_16, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0] 2024.10.22 00:37:34.383999 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_81_16, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0] 2024.10.22 00:37:34.450230 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_81_16, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0] 2024.10.22 00:37:34.580207 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 7.55 MiB, will set to 1.63 GiB (RSS), difference: -9.33 MiB 2024.10.22 00:37:34.877168 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_106_106_0 to 202410_116_116_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:35.170496 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 124 2024.10.22 00:37:35.680032 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_28_28_0, total 39 rows starting from the beginning of the part 2024.10.22 00:37:36.025964 [ 1160 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:37:36.048910 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} MergeTask::MergeProjectionsStage: Merge sorted 12765 rows, containing 13 columns (13 merged, 0 gathered) in 1.990185471 sec., 6413.975072175572 rows/sec., 976.89 KiB/sec. 2024.10.22 00:37:36.058708 [ 1160 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:37:36.194549 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 1862 2024.10.22 00:37:36.248659 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} MergedBlockOutputStream: filled checksums 202410_1_241_48 (state Temporary) 2024.10.22 00:37:36.384348 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_241_48 to 202410_1_241_48 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:36.388046 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_241_48} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_236_47, 202410_241_241_0] -> 202410_1_241_48 2024.10.22 00:37:36.429507 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.83 MiB. 2024.10.22 00:37:36.581014 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:36.585015 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:36.594389 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 3.68 MiB, will set to 1.64 GiB (RSS), difference: 9.16 MiB 2024.10.22 00:37:36.623404 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_133_133_0 (state Temporary) 2024.10.22 00:37:36.673050 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_133_133_0 to 202410_146_146_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:36.794614 [ 1200 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_141_28 to 202410_146_146_0 2024.10.22 00:37:36.811687 [ 1200 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:36.824820 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:36.832721 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1862 2024.10.22 00:37:36.866969 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_141_28 to 202410_146_146_0 into Compact with storage Full 2024.10.22 00:37:37.087668 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:37:37.092826 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_141_28, total 1905 rows starting from the beginning of the part 2024.10.22 00:37:37.120167 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_142_142_0, total 12 rows starting from the beginning of the part 2024.10.22 00:37:37.154935 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_143_143_0, total 12 rows starting from the beginning of the part 2024.10.22 00:37:37.174575 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_144_144_0, total 27 rows starting from the beginning of the part 2024.10.22 00:37:37.188144 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_145_145_0, total 6 rows starting from the beginning of the part 2024.10.22 00:37:37.217541 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_146_146_0, total 6 rows starting from the beginning of the part 2024.10.22 00:37:37.610920 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_29_29_0, total 43 rows starting from the beginning of the part 2024.10.22 00:37:38.161257 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 11991 2024.10.22 00:37:38.330576 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:38.335249 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:38.384213 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 2.85 MiB, will set to 1.63 GiB (RSS), difference: -3.78 MiB 2024.10.22 00:37:38.477264 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_229_229_0 (state Temporary) 2024.10.22 00:37:38.542939 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} MergeTask::MergeProjectionsStage: Merge sorted 1968 rows, containing 12 columns (12 merged, 0 gathered) in 1.690706617 sec., 1164.0103494076525 rows/sec., 179.99 KiB/sec. 2024.10.22 00:37:38.582736 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_229_229_0 to 202410_242_242_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:38.595243 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} MergedBlockOutputStream: filled checksums 202410_1_146_29 (state Temporary) 2024.10.22 00:37:38.641815 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 11991 2024.10.22 00:37:39.009374 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_146_29 to 202410_1_146_29 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:39.015454 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_146_29} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_141_28, 202410_146_146_0] -> 202410_1_146_29 2024.10.22 00:37:39.051014 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:37:39.213104 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 8766 2024.10.22 00:37:39.500793 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:37:39.512083 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:37:39.585355 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:39.588839 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:39.707640 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 14 file segments (while reserving 1048576 bytes for 0b8ed379d7c7c95912b944418ca13ab5:0) 2024.10.22 00:37:39.713722 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 429bd513857f76dce16661fa666fca12, offset: 0, size: 319046 2024.10.22 00:37:39.722142 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 429bd513857f76dce16661fa666fca12 for removal 2024.10.22 00:37:39.726914 [ 1476 ] {} CacheMetadata: Key 429bd513857f76dce16661fa666fca12 is removed from metadata 2024.10.22 00:37:39.729293 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2b08eda02ba1f75fed85bc06216919cc, offset: 0, size: 217 2024.10.22 00:37:39.737341 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2b08eda02ba1f75fed85bc06216919cc for removal 2024.10.22 00:37:39.745737 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c349f51ca6ada74fdfde973437b15927, offset: 0, size: 218 2024.10.22 00:37:39.751243 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_132_132_0 (state Temporary) 2024.10.22 00:37:39.750974 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_136_27, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0] 2024.10.22 00:37:39.751721 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c349f51ca6ada74fdfde973437b15927 for removal 2024.10.22 00:37:39.754758 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_136_27, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0] 2024.10.22 00:37:39.760647 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3879ed4b5d3222388bb222cbabefd10f, offset: 0, size: 84910 2024.10.22 00:37:39.760903 [ 1476 ] {} CacheMetadata: Key c349f51ca6ada74fdfde973437b15927 is removed from metadata 2024.10.22 00:37:39.767232 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3879ed4b5d3222388bb222cbabefd10f for removal 2024.10.22 00:37:39.771606 [ 1476 ] {} CacheMetadata: Key 2b08eda02ba1f75fed85bc06216919cc is removed from metadata 2024.10.22 00:37:39.775382 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3d29d1bd0f71145162bbf3ee8732d716, offset: 0, size: 379328 2024.10.22 00:37:39.782930 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3d29d1bd0f71145162bbf3ee8732d716 for removal 2024.10.22 00:37:39.783537 [ 1476 ] {} CacheMetadata: Key 3879ed4b5d3222388bb222cbabefd10f is removed from metadata 2024.10.22 00:37:39.787251 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 61cdebac7725d0ba233d345b7f94e327, offset: 0, size: 175815 2024.10.22 00:37:39.791323 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_136_27, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0] 2024.10.22 00:37:39.792231 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 61cdebac7725d0ba233d345b7f94e327 for removal 2024.10.22 00:37:39.793560 [ 1476 ] {} CacheMetadata: Key 3d29d1bd0f71145162bbf3ee8732d716 is removed from metadata 2024.10.22 00:37:39.793702 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_132_132_0 to 202410_138_138_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:39.796043 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: aa0a1d7b2548f96bd8da6ada6dd31ef3, offset: 0, size: 185 2024.10.22 00:37:39.899825 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key aa0a1d7b2548f96bd8da6ada6dd31ef3 for removal 2024.10.22 00:37:39.909502 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bc7facb93751720099ac856f66aedc4e, offset: 0, size: 101934 2024.10.22 00:37:39.910690 [ 1476 ] {} CacheMetadata: Key aa0a1d7b2548f96bd8da6ada6dd31ef3 is removed from metadata 2024.10.22 00:37:39.916493 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bc7facb93751720099ac856f66aedc4e for removal 2024.10.22 00:37:39.923998 [ 1476 ] {} CacheMetadata: Key bc7facb93751720099ac856f66aedc4e is removed from metadata 2024.10.22 00:37:39.925250 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1dcef1a92457a5613af12092d57c9015, offset: 0, size: 106567 2024.10.22 00:37:39.935936 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1dcef1a92457a5613af12092d57c9015 for removal 2024.10.22 00:37:39.940896 [ 1476 ] {} CacheMetadata: Key 61cdebac7725d0ba233d345b7f94e327 is removed from metadata 2024.10.22 00:37:39.948593 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e14e95288d9d8c36349d9fb102c57171, offset: 0, size: 156 2024.10.22 00:37:39.953639 [ 1476 ] {} CacheMetadata: Key 1dcef1a92457a5613af12092d57c9015 is removed from metadata 2024.10.22 00:37:39.956978 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e14e95288d9d8c36349d9fb102c57171 for removal 2024.10.22 00:37:39.973273 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 27769a822f48bcada68a43858b33825d, offset: 0, size: 158 2024.10.22 00:37:39.975831 [ 1476 ] {} CacheMetadata: Key e14e95288d9d8c36349d9fb102c57171 is removed from metadata 2024.10.22 00:37:39.978908 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 27769a822f48bcada68a43858b33825d for removal 2024.10.22 00:37:39.983505 [ 1476 ] {} CacheMetadata: Key 27769a822f48bcada68a43858b33825d is removed from metadata 2024.10.22 00:37:39.984310 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 32e013fd7e41abe9d7f2291eafc83a18, offset: 0, size: 156 2024.10.22 00:37:39.991243 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 32e013fd7e41abe9d7f2291eafc83a18 for removal 2024.10.22 00:37:39.995643 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8766 2024.10.22 00:37:39.996252 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2f671a7285049e0ed62300d6c93b110d, offset: 0, size: 154 2024.10.22 00:37:39.999844 [ 1476 ] {} CacheMetadata: Key 32e013fd7e41abe9d7f2291eafc83a18 is removed from metadata 2024.10.22 00:37:40.003314 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2f671a7285049e0ed62300d6c93b110d for removal 2024.10.22 00:37:40.011267 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1977c389b1efe4ca38f0cac2866e0389, offset: 0, size: 130603 2024.10.22 00:37:40.020677 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1977c389b1efe4ca38f0cac2866e0389 for removal 2024.10.22 00:37:40.118535 [ 1476 ] {} CacheMetadata: Key 1977c389b1efe4ca38f0cac2866e0389 is removed from metadata 2024.10.22 00:37:40.137023 [ 1476 ] {} CacheMetadata: Key 2f671a7285049e0ed62300d6c93b110d is removed from metadata 2024.10.22 00:37:40.674886 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 2.95 MiB, will set to 1.63 GiB (RSS), difference: 1.33 MiB 2024.10.22 00:37:40.683846 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 223262 2024.10.22 00:37:40.722551 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pje/vceumghggjbnpbedpwqdttxntsunu. 2024.10.22 00:37:41.896787 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:41.901290 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:42.462804 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 2.00 MiB, will set to 1.63 GiB (RSS), difference: -7.71 MiB 2024.10.22 00:37:42.471523 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_128_128_0 (state Temporary) 2024.10.22 00:37:42.663861 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 12064 2024.10.22 00:37:42.674111 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 125 2024.10.22 00:37:42.727857 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_128_128_0 to 202410_135_135_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:42.833352 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 223262 2024.10.22 00:37:42.941590 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:42.950473 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:43.012659 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_230_230_0 (state Temporary) 2024.10.22 00:37:43.151131 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:43.160829 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:43.270694 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_230_230_0 to 202410_243_243_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:43.318691 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12064 2024.10.22 00:37:43.319098 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_107_107_0 (state Temporary) 2024.10.22 00:37:43.468011 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_64_13, 202410_65_65_0, 202410_66_66_0, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0] 2024.10.22 00:37:43.470273 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_64_13, 202410_65_65_0, 202410_66_66_0, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0] 2024.10.22 00:37:43.611398 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_64_13, 202410_65_65_0, 202410_66_66_0, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0] 2024.10.22 00:37:43.617037 [ 1184 ] {} DatabaseCatalog: Execution took 242 ms. 2024.10.22 00:37:43.622673 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_107_107_0 to 202410_117_117_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:43.799625 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 125 2024.10.22 00:37:44.336462 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 1882 2024.10.22 00:37:44.541575 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 2.23 MiB, will set to 1.64 GiB (RSS), difference: 4.91 MiB 2024.10.22 00:37:44.571747 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:44.576442 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.09 GiB. 2024.10.22 00:37:44.743001 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_134_134_0 (state Temporary) 2024.10.22 00:37:44.794879 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_134_134_0 to 202410_147_147_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:44.866819 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1882 2024.10.22 00:37:45.615327 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 649637 bytes for 0b8ed379d7c7c95912b944418ca13ab5:0) 2024.10.22 00:37:45.617808 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b3594ec6568305ccd847b0a4a7378e13, offset: 0, size: 236910 2024.10.22 00:37:45.622907 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b3594ec6568305ccd847b0a4a7378e13 for removal 2024.10.22 00:37:45.628443 [ 1476 ] {} CacheMetadata: Key b3594ec6568305ccd847b0a4a7378e13 is removed from metadata 2024.10.22 00:37:45.627873 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 075091faf0bef2ba2ddf2be913226f20, offset: 0, size: 216 2024.10.22 00:37:45.657463 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 075091faf0bef2ba2ddf2be913226f20 for removal 2024.10.22 00:37:45.666208 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fd5e1cfee026a50a33c8e48ff1fc875b, offset: 0, size: 213427 2024.10.22 00:37:45.667438 [ 1476 ] {} CacheMetadata: Key 075091faf0bef2ba2ddf2be913226f20 is removed from metadata 2024.10.22 00:37:45.670239 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fd5e1cfee026a50a33c8e48ff1fc875b for removal 2024.10.22 00:37:45.673803 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 46c2f3d5e77128a08b4109385ee3fae9, offset: 0, size: 186 2024.10.22 00:37:45.677073 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 46c2f3d5e77128a08b4109385ee3fae9 for removal 2024.10.22 00:37:45.679703 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e71db72b5360bc5b5b67e1527980c2b4, offset: 0, size: 9052 2024.10.22 00:37:45.682784 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e71db72b5360bc5b5b67e1527980c2b4 for removal 2024.10.22 00:37:45.686847 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 78095f1784e6f07e59115dd282576d93, offset: 0, size: 210 2024.10.22 00:37:45.687646 [ 1476 ] {} CacheMetadata: Key e71db72b5360bc5b5b67e1527980c2b4 is removed from metadata 2024.10.22 00:37:45.693640 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 78095f1784e6f07e59115dd282576d93 for removal 2024.10.22 00:37:45.698243 [ 1476 ] {} CacheMetadata: Key 78095f1784e6f07e59115dd282576d93 is removed from metadata 2024.10.22 00:37:45.715326 [ 1476 ] {} CacheMetadata: Key 46c2f3d5e77128a08b4109385ee3fae9 is removed from metadata 2024.10.22 00:37:45.739312 [ 1476 ] {} CacheMetadata: Key fd5e1cfee026a50a33c8e48ff1fc875b is removed from metadata 2024.10.22 00:37:45.991540 [ 1689 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/dlb/dcmwipuqozbpxptksqwgxblqttniq, size 2746789 2024.10.22 00:37:46.480453 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 2.27 MiB, will set to 1.63 GiB (RSS), difference: 1.56 MiB 2024.10.22 00:37:46.481033 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:37:46.485361 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:37:46.606039 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:37:46.716066 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:37:46.729080 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:37:46.748497 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:37:46.765197 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:37:46.789738 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ffk/axsehijcqcdwmxptpzhyswvuiudsw (LogSeriesLimiter: on interval from 2024-10-22 00:37:27 to 2024-10-22 00:37:46 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:37:47.336467 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 12118 2024.10.22 00:37:47.529803 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 58 entries to flush up to offset 8824 2024.10.22 00:37:47.673057 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:47.676552 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:47.720298 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_133_133_0 (state Temporary) 2024.10.22 00:37:47.766939 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:47.769111 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:47.770221 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_133_133_0 to 202410_139_139_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:47.797930 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_231_231_0 (state Temporary) 2024.10.22 00:37:47.811588 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8824 2024.10.22 00:37:48.063713 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_231_231_0 to 202410_244_244_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:48.100146 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12118 2024.10.22 00:37:48.693230 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.83 GiB, free memory in arenas 2.65 MiB, will set to 1.64 GiB (RSS), difference: -7.53 MiB 2024.10.22 00:37:49.266157 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_75_15, 202410_76_76_0, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0] 2024.10.22 00:37:49.268277 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_75_15, 202410_76_76_0, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0] 2024.10.22 00:37:49.323615 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_75_15, 202410_76_76_0, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0] 2024.10.22 00:37:49.501441 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:37:49.506785 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:37:49.838159 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 225182 2024.10.22 00:37:50.657519 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 3.58 MiB, will set to 1.64 GiB (RSS), difference: 1.62 MiB 2024.10.22 00:37:50.804387 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:50.811159 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:51.045301 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:51.050782 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:51.063817 [ 1278 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:37:51.079594 [ 1278 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:37:51.084958 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_129_129_0 (state Temporary) 2024.10.22 00:37:51.278829 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_29_6/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:37:51.285385 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_129_129_0 to 202410_136_136_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:51.302905 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 126 2024.10.22 00:37:51.320339 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 225182 2024.10.22 00:37:51.430454 [ 1231 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_131_26 to 202410_136_136_0 2024.10.22 00:37:51.454885 [ 1231 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:51.458835 [ 1231 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:51.471962 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_131_26 to 202410_136_136_0 into Compact with storage Full 2024.10.22 00:37:51.482363 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:51.485524 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:51.491945 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:37:51.499005 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} MergeTreeSequentialSource: Reading 30 marks from part 202410_1_131_26, total 230708 rows starting from the beginning of the part 2024.10.22 00:37:51.511136 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_132_132_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:37:51.526436 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_133_133_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:37:51.559557 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_134_134_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:37:51.569319 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_108_108_0 (state Temporary) 2024.10.22 00:37:51.580792 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_135_135_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:37:51.595827 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_136_136_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:37:51.643567 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_108_108_0 to 202410_118_118_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:51.678069 [ 1274 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_113_23 to 202410_118_118_0 2024.10.22 00:37:51.689431 [ 1274 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:51.690976 [ 1274 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:51.706203 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 126 2024.10.22 00:37:51.717682 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_113_23 to 202410_118_118_0 into Compact with storage Full 2024.10.22 00:37:51.997063 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:37:52.029619 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_113_23, total 157 rows starting from the beginning of the part 2024.10.22 00:37:52.111789 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 12171 2024.10.22 00:37:52.125431 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_114_114_0, total 1 rows starting from the beginning of the part 2024.10.22 00:37:52.289363 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:52.290702 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:52.311109 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_115_115_0, total 1 rows starting from the beginning of the part 2024.10.22 00:37:52.369826 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 1887 2024.10.22 00:37:52.374546 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.83 GiB, free memory in arenas 5.00 MiB, will set to 1.65 GiB (RSS), difference: -7.83 MiB 2024.10.22 00:37:52.385164 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_232_232_0 (state Temporary) 2024.10.22 00:37:52.450697 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_232_232_0 to 202410_245_245_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:52.491507 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_116_116_0, total 1 rows starting from the beginning of the part 2024.10.22 00:37:52.551221 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12171 2024.10.22 00:37:52.594127 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 4899080f045eb5a9089d4d61d4ffe08d:0) 2024.10.22 00:37:52.601194 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_117_117_0, total 1 rows starting from the beginning of the part 2024.10.22 00:37:52.602326 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 368bdcb6f675814e19ed7014d21a2cb7, offset: 0, size: 225 2024.10.22 00:37:52.609414 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 368bdcb6f675814e19ed7014d21a2cb7 for removal 2024.10.22 00:37:52.615049 [ 1476 ] {} CacheMetadata: Key 368bdcb6f675814e19ed7014d21a2cb7 is removed from metadata 2024.10.22 00:37:52.618380 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1878b2b6987d2d96ac19235e79f60326, offset: 0, size: 431911 2024.10.22 00:37:52.622272 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1878b2b6987d2d96ac19235e79f60326 for removal 2024.10.22 00:37:52.626073 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 208fcdf2d55dd2834b8f5f1e09e53653, offset: 0, size: 220 2024.10.22 00:37:52.631749 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 208fcdf2d55dd2834b8f5f1e09e53653 for removal 2024.10.22 00:37:52.640441 [ 1476 ] {} CacheMetadata: Key 208fcdf2d55dd2834b8f5f1e09e53653 is removed from metadata 2024.10.22 00:37:52.642230 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fd103a2b5e491f3df3432af412a759e6, offset: 0, size: 623197 2024.10.22 00:37:52.646701 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fd103a2b5e491f3df3432af412a759e6 for removal 2024.10.22 00:37:52.653147 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3c872d38d186fb6da7d45e38a6978e02, offset: 0, size: 357456 2024.10.22 00:37:52.658706 [ 1476 ] {} CacheMetadata: Key fd103a2b5e491f3df3432af412a759e6 is removed from metadata 2024.10.22 00:37:52.663107 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3c872d38d186fb6da7d45e38a6978e02 for removal 2024.10.22 00:37:52.671631 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a6ec20e225a56cc331f5f56421964ced, offset: 0, size: 219 2024.10.22 00:37:52.676179 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a6ec20e225a56cc331f5f56421964ced for removal 2024.10.22 00:37:52.676050 [ 1476 ] {} CacheMetadata: Key 3c872d38d186fb6da7d45e38a6978e02 is removed from metadata 2024.10.22 00:37:52.686242 [ 1476 ] {} CacheMetadata: Key a6ec20e225a56cc331f5f56421964ced is removed from metadata 2024.10.22 00:37:52.694494 [ 1476 ] {} CacheMetadata: Key 1878b2b6987d2d96ac19235e79f60326 is removed from metadata 2024.10.22 00:37:52.782385 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:52.783625 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:52.895074 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_118_118_0, total 1 rows starting from the beginning of the part 2024.10.22 00:37:53.073480 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_135_135_0 (state Temporary) 2024.10.22 00:37:53.281482 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_135_135_0 to 202410_148_148_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:53.522319 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1887 2024.10.22 00:37:54.535016 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.83 GiB, free memory in arenas 10.55 MiB, will set to 1.65 GiB (RSS), difference: 83.13 MiB 2024.10.22 00:37:54.640987 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_27_27_0 (state Temporary) 2024.10.22 00:37:54.745534 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} MergeTask::MergeProjectionsStage: Merge sorted 162 rows, containing 32 columns (32 merged, 0 gathered) in 3.035340833 sec., 53.37127160111578 rows/sec., 5.94 KiB/sec. 2024.10.22 00:37:54.787807 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} MergedBlockOutputStream: filled checksums 202410_1_118_24 (state Temporary) 2024.10.22 00:37:55.111655 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_118_24 to 202410_1_118_24 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:55.218953 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_118_24} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_113_23, 202410_118_118_0] -> 202410_1_118_24 2024.10.22 00:37:55.363938 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 67 entries to flush up to offset 8891 2024.10.22 00:37:55.375552 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:37:55.418117 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pje/vceumghggjbnpbedpwqdttxntsunu. 2024.10.22 00:37:55.568367 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:55.580638 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:55.670848 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_134_134_0 (state Temporary) 2024.10.22 00:37:55.818965 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_134_134_0 to 202410_140_140_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:55.861298 [ 1178 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_135_27 to 202410_140_140_0 2024.10.22 00:37:55.878733 [ 1178 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:55.880819 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:55.883863 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8891 2024.10.22 00:37:55.906616 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_135_27 to 202410_140_140_0 into Compact with storage Full 2024.10.22 00:37:55.939741 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:37:55.953763 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_135_27, total 8669 rows starting from the beginning of the part 2024.10.22 00:37:55.981455 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_136_136_0, total 82 rows starting from the beginning of the part 2024.10.22 00:37:56.113829 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_137_137_0, total 40 rows starting from the beginning of the part 2024.10.22 00:37:56.147215 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_138_138_0, total 79 rows starting from the beginning of the part 2024.10.22 00:37:56.296011 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_139_139_0, total 58 rows starting from the beginning of the part 2024.10.22 00:37:56.326799 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_140_140_0, total 67 rows starting from the beginning of the part 2024.10.22 00:37:56.565826 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 12233 2024.10.22 00:37:56.869717 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:56.872202 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:56.942789 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_233_233_0 (state Temporary) 2024.10.22 00:37:56.971537 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.83 GiB, free memory in arenas 13.29 MiB, will set to 1.65 GiB (RSS), difference: 104.08 MiB 2024.10.22 00:37:56.988513 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_233_233_0 to 202410_246_246_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:57.029242 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_241_48 to 202410_246_246_0 2024.10.22 00:37:57.052136 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:57.066903 [ 1180 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:57.103023 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_241_48 to 202410_246_246_0 into Compact with storage Full 2024.10.22 00:37:57.126416 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for dd15ba7eb48779f0905d69195f1e95c5:0) 2024.10.22 00:37:57.130648 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 68aa59d75e6817c24d1c0c215aca06a1, offset: 0, size: 426741 2024.10.22 00:37:57.137111 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 68aa59d75e6817c24d1c0c215aca06a1 for removal 2024.10.22 00:37:57.148314 [ 1476 ] {} CacheMetadata: Key 68aa59d75e6817c24d1c0c215aca06a1 is removed from metadata 2024.10.22 00:37:57.151378 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 18784c61af692a5228c64beaeb33531f, offset: 0, size: 220 2024.10.22 00:37:57.161196 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 18784c61af692a5228c64beaeb33531f for removal 2024.10.22 00:37:57.171425 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ba0f32b2f99edf3225947a86d8005812, offset: 0, size: 616206 2024.10.22 00:37:57.175983 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ba0f32b2f99edf3225947a86d8005812 for removal 2024.10.22 00:37:57.177148 [ 1476 ] {} CacheMetadata: Key 18784c61af692a5228c64beaeb33531f is removed from metadata 2024.10.22 00:37:57.180558 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5efc2fd03b24b47f1b120983af2cfda2, offset: 0, size: 224 2024.10.22 00:37:57.183148 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:37:57.184052 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5efc2fd03b24b47f1b120983af2cfda2 for removal 2024.10.22 00:37:57.184382 [ 1476 ] {} CacheMetadata: Key ba0f32b2f99edf3225947a86d8005812 is removed from metadata 2024.10.22 00:37:57.195071 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_241_48, total 12765 rows starting from the beginning of the part 2024.10.22 00:37:57.244723 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_242_242_0, total 57 rows starting from the beginning of the part 2024.10.22 00:37:57.291617 [ 1476 ] {} CacheMetadata: Key 5efc2fd03b24b47f1b120983af2cfda2 is removed from metadata 2024.10.22 00:37:57.365620 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for dd15ba7eb48779f0905d69195f1e95c5:0) 2024.10.22 00:37:57.369439 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b8c259f99dc3df003895953b982c285e, offset: 0, size: 472899 2024.10.22 00:37:57.372278 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b8c259f99dc3df003895953b982c285e for removal 2024.10.22 00:37:57.374302 [ 1476 ] {} CacheMetadata: Key b8c259f99dc3df003895953b982c285e is removed from metadata 2024.10.22 00:37:57.374664 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2cea597979c6b09dba59849bab454681, offset: 0, size: 222 2024.10.22 00:37:57.377199 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2cea597979c6b09dba59849bab454681 for removal 2024.10.22 00:37:57.379155 [ 1476 ] {} CacheMetadata: Key 2cea597979c6b09dba59849bab454681 is removed from metadata 2024.10.22 00:37:57.379598 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fc601f6270bb3410de13317ada5a965a, offset: 0, size: 322652 2024.10.22 00:37:57.381456 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fc601f6270bb3410de13317ada5a965a for removal 2024.10.22 00:37:57.383009 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 466c1e46316a2ee3d08f1ab62ecae366, offset: 0, size: 217 2024.10.22 00:37:57.384120 [ 1476 ] {} CacheMetadata: Key fc601f6270bb3410de13317ada5a965a is removed from metadata 2024.10.22 00:37:57.385094 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 466c1e46316a2ee3d08f1ab62ecae366 for removal 2024.10.22 00:37:57.387818 [ 1476 ] {} CacheMetadata: Key 466c1e46316a2ee3d08f1ab62ecae366 is removed from metadata 2024.10.22 00:37:57.418362 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_243_243_0, total 73 rows starting from the beginning of the part 2024.10.22 00:37:57.461888 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12233 2024.10.22 00:37:57.471826 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_244_244_0, total 54 rows starting from the beginning of the part 2024.10.22 00:37:57.635379 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_245_245_0, total 53 rows starting from the beginning of the part 2024.10.22 00:37:57.650738 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 206286 bytes for dd15ba7eb48779f0905d69195f1e95c5:0) 2024.10.22 00:37:57.675920 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c1b6da431e6b3b4e311683be0fb725a1, offset: 0, size: 383217 2024.10.22 00:37:57.678811 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c1b6da431e6b3b4e311683be0fb725a1 for removal 2024.10.22 00:37:57.682389 [ 1476 ] {} CacheMetadata: Key c1b6da431e6b3b4e311683be0fb725a1 is removed from metadata 2024.10.22 00:37:57.683674 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_136_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:37:57.684176 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 77b91111ac63f85599222f2d699d570c, offset: 0, size: 219 2024.10.22 00:37:57.684443 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_246_246_0, total 62 rows starting from the beginning of the part 2024.10.22 00:37:57.688517 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 77b91111ac63f85599222f2d699d570c for removal 2024.10.22 00:37:57.701440 [ 1476 ] {} CacheMetadata: Key 77b91111ac63f85599222f2d699d570c is removed from metadata 2024.10.22 00:37:57.742918 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} MergeTask::MergeProjectionsStage: Merge sorted 8995 rows, containing 13 columns (13 merged, 0 gathered) in 1.845806906 sec., 4873.207468647319 rows/sec., 2.04 MiB/sec. 2024.10.22 00:37:57.792456 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} MergedBlockOutputStream: filled checksums 202410_1_140_28 (state Temporary) 2024.10.22 00:37:57.876694 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 4899080f045eb5a9089d4d61d4ffe08d:0) 2024.10.22 00:37:57.878622 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 28819fa4b1be67354fb7c3a146adab33, offset: 0, size: 217034 2024.10.22 00:37:57.881713 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 28819fa4b1be67354fb7c3a146adab33 for removal 2024.10.22 00:37:57.886684 [ 1476 ] {} CacheMetadata: Key 28819fa4b1be67354fb7c3a146adab33 is removed from metadata 2024.10.22 00:37:57.888127 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3c64a48a7998b188d1002d98e601fc83, offset: 0, size: 257551 2024.10.22 00:37:57.894513 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3c64a48a7998b188d1002d98e601fc83 for removal 2024.10.22 00:37:57.896382 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e1f028bbd4add3e39d00c8b0bd48ba3c, offset: 0, size: 215 2024.10.22 00:37:57.900959 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e1f028bbd4add3e39d00c8b0bd48ba3c for removal 2024.10.22 00:37:57.906520 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b76ab27cf7807544e481db6a441425d4, offset: 0, size: 215 2024.10.22 00:37:57.906520 [ 1476 ] {} CacheMetadata: Key 3c64a48a7998b188d1002d98e601fc83 is removed from metadata 2024.10.22 00:37:57.909744 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b76ab27cf7807544e481db6a441425d4 for removal 2024.10.22 00:37:57.914846 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1b1eb7803a99d75a57627e5d746b3c40, offset: 0, size: 258721 2024.10.22 00:37:57.919209 [ 1476 ] {} CacheMetadata: Key b76ab27cf7807544e481db6a441425d4 is removed from metadata 2024.10.22 00:37:57.920388 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1b1eb7803a99d75a57627e5d746b3c40 for removal 2024.10.22 00:37:57.926428 [ 1476 ] {} CacheMetadata: Key e1f028bbd4add3e39d00c8b0bd48ba3c is removed from metadata 2024.10.22 00:37:57.934540 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 89e318d9738d3ba77df9ff9df5dc3827, offset: 0, size: 215 2024.10.22 00:37:57.937696 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 89e318d9738d3ba77df9ff9df5dc3827 for removal 2024.10.22 00:37:57.938668 [ 1476 ] {} CacheMetadata: Key 1b1eb7803a99d75a57627e5d746b3c40 is removed from metadata 2024.10.22 00:37:58.053447 [ 1476 ] {} CacheMetadata: Key 89e318d9738d3ba77df9ff9df5dc3827 is removed from metadata 2024.10.22 00:37:58.123977 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 37404 bytes for 9d6636d60e3a1572f167a6ef11701bc7:0) 2024.10.22 00:37:58.128832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cfc68467b4fe8815b488ab829e0691fe, offset: 0, size: 419214 2024.10.22 00:37:58.133430 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cfc68467b4fe8815b488ab829e0691fe for removal 2024.10.22 00:37:58.143171 [ 1476 ] {} CacheMetadata: Key cfc68467b4fe8815b488ab829e0691fe is removed from metadata 2024.10.22 00:37:58.145794 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5576572ca0b479fd87e84d7c014f9825, offset: 0, size: 221 2024.10.22 00:37:58.157485 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5576572ca0b479fd87e84d7c014f9825 for removal 2024.10.22 00:37:58.172943 [ 1476 ] {} CacheMetadata: Key 5576572ca0b479fd87e84d7c014f9825 is removed from metadata 2024.10.22 00:37:58.271429 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_140_28 to 202410_1_140_28 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:58.285329 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_140_28} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_135_27, 202410_140_140_0] -> 202410_1_140_28 2024.10.22 00:37:58.314781 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.09 MiB. 2024.10.22 00:37:58.338257 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 226718 2024.10.22 00:37:58.455282 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 14.95 MiB, will set to 1.64 GiB (RSS), difference: 3.42 MiB 2024.10.22 00:37:58.723927 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 922184 bytes for 63781a22d7e02ced7ff2fe91ba5103b0:0) 2024.10.22 00:37:58.726474 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3a0d3fbf1c02f6dd37a5706f0fc60d93, offset: 0, size: 239 2024.10.22 00:37:58.737097 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3a0d3fbf1c02f6dd37a5706f0fc60d93 for removal 2024.10.22 00:37:58.742072 [ 1476 ] {} CacheMetadata: Key 3a0d3fbf1c02f6dd37a5706f0fc60d93 is removed from metadata 2024.10.22 00:37:58.744166 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 151190866afae7c29f555a4d3e5f56ed, offset: 0, size: 2272 2024.10.22 00:37:58.748567 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 151190866afae7c29f555a4d3e5f56ed for removal 2024.10.22 00:37:58.753163 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bf8fda0bf7a79ba62d0902962a7881e4, offset: 0, size: 155 2024.10.22 00:37:58.753601 [ 1476 ] {} CacheMetadata: Key 151190866afae7c29f555a4d3e5f56ed is removed from metadata 2024.10.22 00:37:58.759102 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bf8fda0bf7a79ba62d0902962a7881e4 for removal 2024.10.22 00:37:58.765309 [ 1476 ] {} CacheMetadata: Key bf8fda0bf7a79ba62d0902962a7881e4 is removed from metadata 2024.10.22 00:37:58.774645 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 36d4a3d319ff49a6fa78e00f82087109, offset: 0, size: 302755 2024.10.22 00:37:58.777325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 36d4a3d319ff49a6fa78e00f82087109 for removal 2024.10.22 00:37:58.781391 [ 1476 ] {} CacheMetadata: Key 36d4a3d319ff49a6fa78e00f82087109 is removed from metadata 2024.10.22 00:37:58.782237 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c46273aec6d1cd294f12494c25ae15e8, offset: 0, size: 1625227 2024.10.22 00:37:58.788848 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c46273aec6d1cd294f12494c25ae15e8 for removal 2024.10.22 00:37:58.796395 [ 1476 ] {} CacheMetadata: Key c46273aec6d1cd294f12494c25ae15e8 is removed from metadata 2024.10.22 00:37:58.796750 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 158d90958454252498d1e3b768962d11, offset: 0, size: 217 2024.10.22 00:37:58.800171 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 158d90958454252498d1e3b768962d11 for removal 2024.10.22 00:37:58.828842 [ 1476 ] {} CacheMetadata: Key 158d90958454252498d1e3b768962d11 is removed from metadata 2024.10.22 00:37:59.235762 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:37:59.240106 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:37:59.305447 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} MergeTask::MergeProjectionsStage: Merge sorted 13064 rows, containing 13 columns (13 merged, 0 gathered) in 2.220144467 sec., 5884.301762422202 rows/sec., 896.27 KiB/sec. 2024.10.22 00:37:59.296344 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:37:59.465656 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_130_130_0 (state Temporary) 2024.10.22 00:37:59.475583 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} MergedBlockOutputStream: filled checksums 202410_1_246_49 (state Temporary) 2024.10.22 00:37:59.488738 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.83 GiB, free memory in arenas 16.00 MiB, will set to 1.65 GiB (RSS), difference: -2.25 MiB 2024.10.22 00:37:59.501425 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:37:59.505681 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:37:59.676078 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_130_130_0 to 202410_137_137_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:59.771633 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_246_49 to 202410_1_246_49 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:37:59.773977 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_246_49} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_241_48, 202410_246_246_0] -> 202410_1_246_49 2024.10.22 00:37:59.801279 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 226718 2024.10.22 00:37:59.834686 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.80 MiB. 2024.10.22 00:38:01.012535 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} MergeTask::MergeProjectionsStage: Merge sorted 238772 rows, containing 4 columns (4 merged, 0 gathered) in 9.546056147 sec., 25012.633104513836 rows/sec., 367.22 KiB/sec. 2024.10.22 00:38:01.025077 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 1913 2024.10.22 00:38:01.163107 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} MergedBlockOutputStream: filled checksums 202410_1_136_27 (state Temporary) 2024.10.22 00:38:01.191366 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_136_27 to 202410_1_136_27 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:01.195010 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_136_27} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_131_26, 202410_136_136_0] -> 202410_1_136_27 2024.10.22 00:38:01.346846 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:38:01.428073 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:01.435738 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:01.471031 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 126 entries to flush up to offset 12359 2024.10.22 00:38:01.479600 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_136_136_0 (state Temporary) 2024.10.22 00:38:01.519448 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 14.72 MiB, will set to 1.65 GiB (RSS), difference: 14.49 MiB 2024.10.22 00:38:01.541672 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_136_136_0 to 202410_149_149_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:01.616255 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1913 2024.10.22 00:38:01.955411 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:01.960016 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:02.132415 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_27_27_0 to 202410_30_30_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:02.177463 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 656275 bytes for 4899080f045eb5a9089d4d61d4ffe08d:0) 2024.10.22 00:38:02.178659 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f373f0e79569bd9283457080fd8d15fd, offset: 0, size: 2015784 2024.10.22 00:38:02.186803 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f373f0e79569bd9283457080fd8d15fd for removal 2024.10.22 00:38:02.188864 [ 1476 ] {} CacheMetadata: Key f373f0e79569bd9283457080fd8d15fd is removed from metadata 2024.10.22 00:38:02.190403 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c1682928065bb9d5309a615e73570936, offset: 0, size: 245 2024.10.22 00:38:02.193693 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c1682928065bb9d5309a615e73570936 for removal 2024.10.22 00:38:02.194208 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_234_234_0 (state Temporary) 2024.10.22 00:38:02.274300 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_234_234_0 to 202410_247_247_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:02.304380 [ 1476 ] {} CacheMetadata: Key c1682928065bb9d5309a615e73570936 is removed from metadata 2024.10.22 00:38:02.311832 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12359 2024.10.22 00:38:02.415159 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_141_28, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0] 2024.10.22 00:38:02.418521 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_141_28, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0] 2024.10.22 00:38:02.420775 [ 1723 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ffk/axsehijcqcdwmxptpzhyswvuiudsw, size 2753427 2024.10.22 00:38:02.527897 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RegionID 2024.10.22 00:38:02.545093 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RegionID 2024.10.22 00:38:02.564116 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RegionID 2024.10.22 00:38:02.572891 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RegionID 2024.10.22 00:38:02.578808 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RegionID 2024.10.22 00:38:02.649374 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_141_28, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0] 2024.10.22 00:38:02.696006 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RegionID 2024.10.22 00:38:02.715639 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RegionID 2024.10.22 00:38:02.863987 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jwc/towfughomidftgvoqqxuvwouuvnia (LogSeriesLimiter: on interval from 2024-10-22 00:37:46 to 2024-10-22 00:38:02 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:38:03.394196 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 8982 2024.10.22 00:38:03.599274 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:03.604056 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:03.675552 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_135_135_0 (state Temporary) 2024.10.22 00:38:03.685355 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 10.50 MiB, will set to 1.64 GiB (RSS), difference: -3.89 MiB 2024.10.22 00:38:03.725142 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_135_135_0 to 202410_141_141_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:03.882621 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 8982 2024.10.22 00:38:04.088594 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_29_6/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:38:05.115307 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1070 2024.10.22 00:38:05.426017 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 13.38 MiB, will set to 1.64 GiB (RSS), difference: 10.30 MiB 2024.10.22 00:38:05.867576 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 925497 bytes for dfe65364208d5efbd42a8e6294d4f1b9:0) 2024.10.22 00:38:05.869486 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 832fa4a4ce6948100a7f87eb9015bc07, offset: 0, size: 981588 2024.10.22 00:38:05.874691 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 832fa4a4ce6948100a7f87eb9015bc07 for removal 2024.10.22 00:38:05.877173 [ 1476 ] {} CacheMetadata: Key 832fa4a4ce6948100a7f87eb9015bc07 is removed from metadata 2024.10.22 00:38:05.889566 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.377024519 sec. 2024.10.22 00:38:05.978857 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 69252873b8ea6699eee68ae4428020fd, offset: 0, size: 230 2024.10.22 00:38:05.981605 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 69252873b8ea6699eee68ae4428020fd for removal 2024.10.22 00:38:05.983268 [ 1476 ] {} CacheMetadata: Key 69252873b8ea6699eee68ae4428020fd is removed from metadata 2024.10.22 00:38:06.084025 [ 1224 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:38:06.096632 [ 1224 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:38:06.542942 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 12409 2024.10.22 00:38:06.711678 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 127 2024.10.22 00:38:06.811888 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 228638 2024.10.22 00:38:06.902405 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:06.911976 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:06.960797 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_235_235_0 (state Temporary) 2024.10.22 00:38:07.098303 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_235_235_0 to 202410_248_248_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:07.204926 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:07.212490 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:07.265114 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12409 2024.10.22 00:38:07.272835 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_109_109_0 (state Temporary) 2024.10.22 00:38:07.484946 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_109_109_0 to 202410_119_119_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:07.520830 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:07.524353 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:07.565521 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 127 2024.10.22 00:38:07.570302 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 11.56 MiB, will set to 1.64 GiB (RSS), difference: -1.03 MiB 2024.10.22 00:38:07.636767 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_131_131_0 (state Temporary) 2024.10.22 00:38:07.802889 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_131_131_0 to 202410_138_138_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:07.863567 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 228638 2024.10.22 00:38:07.886485 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 20 file segments (while reserving 732670 bytes for 740e9123c8e8c994a9858eb092875a51:0) 2024.10.22 00:38:07.888496 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bc2cf6bf6907d366e7a69e08d017b553, offset: 0, size: 25310 2024.10.22 00:38:07.892682 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bc2cf6bf6907d366e7a69e08d017b553 for removal 2024.10.22 00:38:07.895053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1f5da6efb998a2288d928bc40f3c9f44, offset: 0, size: 32556 2024.10.22 00:38:07.895230 [ 1476 ] {} CacheMetadata: Key bc2cf6bf6907d366e7a69e08d017b553 is removed from metadata 2024.10.22 00:38:07.898458 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1f5da6efb998a2288d928bc40f3c9f44 for removal 2024.10.22 00:38:07.906890 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 592db496fa8fb7f51e6a6740f8073f9b, offset: 0, size: 210 2024.10.22 00:38:07.911337 [ 1476 ] {} CacheMetadata: Key 1f5da6efb998a2288d928bc40f3c9f44 is removed from metadata 2024.10.22 00:38:08.011586 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 592db496fa8fb7f51e6a6740f8073f9b for removal 2024.10.22 00:38:08.017589 [ 1476 ] {} CacheMetadata: Key 592db496fa8fb7f51e6a6740f8073f9b is removed from metadata 2024.10.22 00:38:08.018403 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 06df9f327481e4b795ed483fdd4ffd0b, offset: 0, size: 216 2024.10.22 00:38:08.022768 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 06df9f327481e4b795ed483fdd4ffd0b for removal 2024.10.22 00:38:08.025827 [ 1476 ] {} CacheMetadata: Key 06df9f327481e4b795ed483fdd4ffd0b is removed from metadata 2024.10.22 00:38:08.026680 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f4c8ad1a39c22a45c26d0fea1b928f05, offset: 0, size: 168 2024.10.22 00:38:08.031569 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f4c8ad1a39c22a45c26d0fea1b928f05 for removal 2024.10.22 00:38:08.035264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3962a904ddde171a61a8b8ac2245422c, offset: 0, size: 137 2024.10.22 00:38:08.035450 [ 1476 ] {} CacheMetadata: Key f4c8ad1a39c22a45c26d0fea1b928f05 is removed from metadata 2024.10.22 00:38:08.038127 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3962a904ddde171a61a8b8ac2245422c for removal 2024.10.22 00:38:08.046624 [ 1476 ] {} CacheMetadata: Key 3962a904ddde171a61a8b8ac2245422c is removed from metadata 2024.10.22 00:38:08.046375 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 78db3617ee2e62f36ee48386ebdb5ec2, offset: 0, size: 177 2024.10.22 00:38:08.050252 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 78db3617ee2e62f36ee48386ebdb5ec2 for removal 2024.10.22 00:38:08.053428 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c3ab0fdb03e31b5c78f36291b79f05b9, offset: 0, size: 168 2024.10.22 00:38:08.057084 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c3ab0fdb03e31b5c78f36291b79f05b9 for removal 2024.10.22 00:38:08.063063 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a4e3524ba15f068efb8700f8e9d78509, offset: 0, size: 193 2024.10.22 00:38:08.066664 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a4e3524ba15f068efb8700f8e9d78509 for removal 2024.10.22 00:38:08.070854 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e78353210533a7e41b7b9d9d46658fb3, offset: 0, size: 152 2024.10.22 00:38:08.070877 [ 1476 ] {} CacheMetadata: Key c3ab0fdb03e31b5c78f36291b79f05b9 is removed from metadata 2024.10.22 00:38:08.072439 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e78353210533a7e41b7b9d9d46658fb3 for removal 2024.10.22 00:38:08.075366 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4382b9ec6c2fc0db07d4d7cd30de4696, offset: 0, size: 154 2024.10.22 00:38:08.076382 [ 1476 ] {} CacheMetadata: Key a4e3524ba15f068efb8700f8e9d78509 is removed from metadata 2024.10.22 00:38:08.079885 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4382b9ec6c2fc0db07d4d7cd30de4696 for removal 2024.10.22 00:38:08.082300 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f86180a6b550f6221d32cfba5b567831, offset: 0, size: 923938 2024.10.22 00:38:08.086264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f86180a6b550f6221d32cfba5b567831 for removal 2024.10.22 00:38:08.089796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e46946962ba1b9521956af2023b9bdef, offset: 0, size: 239 2024.10.22 00:38:08.094499 [ 1476 ] {} CacheMetadata: Key f86180a6b550f6221d32cfba5b567831 is removed from metadata 2024.10.22 00:38:08.094487 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e46946962ba1b9521956af2023b9bdef for removal 2024.10.22 00:38:08.096624 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 95f7cd7afe7b38d663442ae702298e92, offset: 0, size: 145 2024.10.22 00:38:08.098884 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 95f7cd7afe7b38d663442ae702298e92 for removal 2024.10.22 00:38:08.100821 [ 1476 ] {} CacheMetadata: Key 95f7cd7afe7b38d663442ae702298e92 is removed from metadata 2024.10.22 00:38:08.101160 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d74f70c52907fa6c8320379e9470fd38, offset: 0, size: 234 2024.10.22 00:38:08.104319 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d74f70c52907fa6c8320379e9470fd38 for removal 2024.10.22 00:38:08.109219 [ 1476 ] {} CacheMetadata: Key d74f70c52907fa6c8320379e9470fd38 is removed from metadata 2024.10.22 00:38:08.111164 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 89794e74586445a6c14073ea7218cc7b, offset: 0, size: 141 2024.10.22 00:38:08.115230 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 89794e74586445a6c14073ea7218cc7b for removal 2024.10.22 00:38:08.117257 [ 1476 ] {} CacheMetadata: Key 89794e74586445a6c14073ea7218cc7b is removed from metadata 2024.10.22 00:38:08.118739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3b2e0892b903d54c014782cb33f724bd, offset: 0, size: 157 2024.10.22 00:38:08.121031 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3b2e0892b903d54c014782cb33f724bd for removal 2024.10.22 00:38:08.123972 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3c7b282883192e5a790dab1a2f78173f, offset: 0, size: 207 2024.10.22 00:38:08.124347 [ 1476 ] {} CacheMetadata: Key 3b2e0892b903d54c014782cb33f724bd is removed from metadata 2024.10.22 00:38:08.128200 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3c7b282883192e5a790dab1a2f78173f for removal 2024.10.22 00:38:08.131381 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 84f8a72e319b71c57acc78c26ec8a879, offset: 0, size: 161 2024.10.22 00:38:08.134585 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 84f8a72e319b71c57acc78c26ec8a879 for removal 2024.10.22 00:38:08.134144 [ 1476 ] {} CacheMetadata: Key 3c7b282883192e5a790dab1a2f78173f is removed from metadata 2024.10.22 00:38:08.137693 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8a3c25ba764e0d5fe34977f2f7823487, offset: 0, size: 160951 2024.10.22 00:38:08.140503 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8a3c25ba764e0d5fe34977f2f7823487 for removal 2024.10.22 00:38:08.144337 [ 1476 ] {} CacheMetadata: Key 84f8a72e319b71c57acc78c26ec8a879 is removed from metadata 2024.10.22 00:38:08.151559 [ 1476 ] {} CacheMetadata: Key 8a3c25ba764e0d5fe34977f2f7823487 is removed from metadata 2024.10.22 00:38:08.163240 [ 1476 ] {} CacheMetadata: Key e46946962ba1b9521956af2023b9bdef is removed from metadata 2024.10.22 00:38:08.174963 [ 1476 ] {} CacheMetadata: Key 4382b9ec6c2fc0db07d4d7cd30de4696 is removed from metadata 2024.10.22 00:38:08.181092 [ 1476 ] {} CacheMetadata: Key e78353210533a7e41b7b9d9d46658fb3 is removed from metadata 2024.10.22 00:38:08.189311 [ 1476 ] {} CacheMetadata: Key 78db3617ee2e62f36ee48386ebdb5ec2 is removed from metadata 2024.10.22 00:38:08.900111 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 923031 bytes for 053ebdb417631176c909e387158b6d6d:0) 2024.10.22 00:38:08.904891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fecef6e71038b4a9223ec7420e57bfb9, offset: 0, size: 409198 2024.10.22 00:38:08.909552 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fecef6e71038b4a9223ec7420e57bfb9 for removal 2024.10.22 00:38:08.913665 [ 1476 ] {} CacheMetadata: Key fecef6e71038b4a9223ec7420e57bfb9 is removed from metadata 2024.10.22 00:38:08.913476 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1c1781e5eee3a4ec1ca29de552afdd25, offset: 0, size: 6281 2024.10.22 00:38:08.919269 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1c1781e5eee3a4ec1ca29de552afdd25 for removal 2024.10.22 00:38:08.921262 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2d140be0784062030ab418a5f0852b04, offset: 0, size: 301380 2024.10.22 00:38:08.921485 [ 1476 ] {} CacheMetadata: Key 1c1781e5eee3a4ec1ca29de552afdd25 is removed from metadata 2024.10.22 00:38:08.923945 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2d140be0784062030ab418a5f0852b04 for removal 2024.10.22 00:38:08.925838 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 53abf12adcfbad78a388e2aa04806db4, offset: 0, size: 5312 2024.10.22 00:38:08.928759 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 53abf12adcfbad78a388e2aa04806db4 for removal 2024.10.22 00:38:08.930670 [ 1476 ] {} CacheMetadata: Key 2d140be0784062030ab418a5f0852b04 is removed from metadata 2024.10.22 00:38:08.932715 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3c2404b0e5c520b7a1e465730771ee3b, offset: 0, size: 7422 2024.10.22 00:38:08.936907 [ 1476 ] {} CacheMetadata: Key 53abf12adcfbad78a388e2aa04806db4 is removed from metadata 2024.10.22 00:38:09.020875 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_76_15, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0] 2024.10.22 00:38:09.035985 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3c2404b0e5c520b7a1e465730771ee3b for removal 2024.10.22 00:38:09.038318 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f4fb28030f292d8323f1f07dc8877599, offset: 0, size: 4818 2024.10.22 00:38:09.039397 [ 1476 ] {} CacheMetadata: Key 3c2404b0e5c520b7a1e465730771ee3b is removed from metadata 2024.10.22 00:38:09.042855 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f4fb28030f292d8323f1f07dc8877599 for removal 2024.10.22 00:38:09.052072 [ 1476 ] {} CacheMetadata: Key f4fb28030f292d8323f1f07dc8877599 is removed from metadata 2024.10.22 00:38:09.120914 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 1927 2024.10.22 00:38:09.129171 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_76_15, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0] 2024.10.22 00:38:09.163716 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_76_15, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0] 2024.10.22 00:38:09.213174 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:09.216626 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:09.249111 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_137_137_0 (state Temporary) 2024.10.22 00:38:09.477562 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 12.37 MiB, will set to 1.64 GiB (RSS), difference: 1.18 MiB 2024.10.22 00:38:09.485478 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_137_137_0 to 202410_150_150_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:09.500705 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:38:09.508280 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:38:09.717385 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1927 2024.10.22 00:38:10.350895 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 12 file segments (while reserving 1048576 bytes for 91d10ca0919c1e97aea31d4fe7e98383:0) 2024.10.22 00:38:10.354146 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 24218aeffcb8697cb5cfeecc5d4304fa, offset: 0, size: 4541381 2024.10.22 00:38:10.361685 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 24218aeffcb8697cb5cfeecc5d4304fa for removal 2024.10.22 00:38:10.365588 [ 1476 ] {} CacheMetadata: Key 24218aeffcb8697cb5cfeecc5d4304fa is removed from metadata 2024.10.22 00:38:10.365528 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c192e8f86c5891f04b392daa7ba69e6e, offset: 0, size: 243 2024.10.22 00:38:10.369751 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c192e8f86c5891f04b392daa7ba69e6e for removal 2024.10.22 00:38:10.372437 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 225a4a49704c71b40109e8bc16825f28, offset: 0, size: 265 2024.10.22 00:38:10.375226 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 225a4a49704c71b40109e8bc16825f28 for removal 2024.10.22 00:38:10.375887 [ 1476 ] {} CacheMetadata: Key c192e8f86c5891f04b392daa7ba69e6e is removed from metadata 2024.10.22 00:38:10.377312 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3fc16d0b52db5465a92659429a3d2af2, offset: 0, size: 4937 2024.10.22 00:38:10.379481 [ 1476 ] {} CacheMetadata: Key 225a4a49704c71b40109e8bc16825f28 is removed from metadata 2024.10.22 00:38:10.483205 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3fc16d0b52db5465a92659429a3d2af2 for removal 2024.10.22 00:38:10.487689 [ 1476 ] {} CacheMetadata: Key 3fc16d0b52db5465a92659429a3d2af2 is removed from metadata 2024.10.22 00:38:10.488940 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1bca27a22ff4502c1886822dd9ac2a07, offset: 0, size: 195 2024.10.22 00:38:10.492240 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1bca27a22ff4502c1886822dd9ac2a07 for removal 2024.10.22 00:38:10.493590 [ 1476 ] {} CacheMetadata: Key 1bca27a22ff4502c1886822dd9ac2a07 is removed from metadata 2024.10.22 00:38:10.495245 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5e556ba05f158f470eced75feeb6a58b, offset: 0, size: 199 2024.10.22 00:38:10.510244 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5e556ba05f158f470eced75feeb6a58b for removal 2024.10.22 00:38:10.512753 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: aa66d255286fc1e46efefa723f3a413b, offset: 0, size: 286 2024.10.22 00:38:10.512475 [ 1476 ] {} CacheMetadata: Key 5e556ba05f158f470eced75feeb6a58b is removed from metadata 2024.10.22 00:38:10.516176 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key aa66d255286fc1e46efefa723f3a413b for removal 2024.10.22 00:38:10.518874 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0087a1c9439c9416ef1f6532d04354ed, offset: 0, size: 564530 2024.10.22 00:38:10.522639 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0087a1c9439c9416ef1f6532d04354ed for removal 2024.10.22 00:38:10.525797 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4a2b872e7794ff9413ea9dd75104eee3, offset: 0, size: 42963 2024.10.22 00:38:10.532853 [ 1476 ] {} CacheMetadata: Key 0087a1c9439c9416ef1f6532d04354ed is removed from metadata 2024.10.22 00:38:10.532911 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4a2b872e7794ff9413ea9dd75104eee3 for removal 2024.10.22 00:38:10.539783 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 47cc6007cddfde7637e6abac71345427, offset: 0, size: 109600 2024.10.22 00:38:10.543487 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 47cc6007cddfde7637e6abac71345427 for removal 2024.10.22 00:38:10.544585 [ 1476 ] {} CacheMetadata: Key 4a2b872e7794ff9413ea9dd75104eee3 is removed from metadata 2024.10.22 00:38:10.546764 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 82d5aa63db26d907fa7befdb376b5163, offset: 0, size: 31191 2024.10.22 00:38:10.552715 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 82d5aa63db26d907fa7befdb376b5163 for removal 2024.10.22 00:38:10.553579 [ 1476 ] {} CacheMetadata: Key 47cc6007cddfde7637e6abac71345427 is removed from metadata 2024.10.22 00:38:10.560597 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 549edd884ae9a8409e4045cf39506e25, offset: 0, size: 2667 2024.10.22 00:38:10.563704 [ 1476 ] {} CacheMetadata: Key 82d5aa63db26d907fa7befdb376b5163 is removed from metadata 2024.10.22 00:38:10.565121 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 549edd884ae9a8409e4045cf39506e25 for removal 2024.10.22 00:38:10.570371 [ 1476 ] {} CacheMetadata: Key 549edd884ae9a8409e4045cf39506e25 is removed from metadata 2024.10.22 00:38:10.574168 [ 1476 ] {} CacheMetadata: Key aa66d255286fc1e46efefa723f3a413b is removed from metadata 2024.10.22 00:38:11.267514 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 148 entries to flush up to offset 12557 2024.10.22 00:38:11.391905 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 38 entries to flush up to offset 9020 2024.10.22 00:38:11.608479 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:11.616867 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:11.659854 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_136_136_0 (state Temporary) 2024.10.22 00:38:11.698133 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_136_136_0 to 202410_142_142_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:11.770084 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9020 2024.10.22 00:38:11.807213 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 11.23 MiB, will set to 1.64 GiB (RSS), difference: 1.82 MiB 2024.10.22 00:38:12.253280 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:12.259382 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:12.430692 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_236_236_0 (state Temporary) 2024.10.22 00:38:12.542800 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_236_236_0 to 202410_249_249_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:12.618402 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12557 2024.10.22 00:38:12.919933 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 1111 2024.10.22 00:38:14.313363 [ 1262 ] {} DatabaseCatalog: Execution took 428 ms. 2024.10.22 00:38:14.868712 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 230174 2024.10.22 00:38:14.918552 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_86_17, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0] 2024.10.22 00:38:14.922983 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_86_17, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0] 2024.10.22 00:38:14.954775 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_86_17, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0] 2024.10.22 00:38:15.070512 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 128 2024.10.22 00:38:15.480847 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:15.484371 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:15.784137 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:15.786401 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:15.875608 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_110_110_0 (state Temporary) 2024.10.22 00:38:15.932045 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_132_132_0 (state Temporary) 2024.10.22 00:38:16.015020 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_132_132_0 to 202410_139_139_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:16.076511 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 230174 2024.10.22 00:38:16.179760 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_110_110_0 to 202410_120_120_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:16.252986 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 128 2024.10.22 00:38:16.639117 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 12587 2024.10.22 00:38:16.797159 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:16.805414 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:16.959896 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_237_237_0 (state Temporary) 2024.10.22 00:38:17.195240 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_237_237_0 to 202410_250_250_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:17.221412 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 1933 2024.10.22 00:38:17.269385 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12587 2024.10.22 00:38:17.438690 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:17.454778 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:17.485352 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_138_138_0 (state Temporary) 2024.10.22 00:38:17.545377 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_138_138_0 to 202410_151_151_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:17.577163 [ 1151 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_146_29 to 202410_151_151_0 2024.10.22 00:38:17.581465 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 823748 bytes for f98ae42a94578360ce3bbda6fb78ed47:0) 2024.10.22 00:38:17.582771 [ 1151 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:17.583004 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1d1e156bacd9b3893104bf85e4469719, offset: 0, size: 2764374 2024.10.22 00:38:17.585728 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1d1e156bacd9b3893104bf85e4469719 for removal 2024.10.22 00:38:17.587162 [ 1476 ] {} CacheMetadata: Key 1d1e156bacd9b3893104bf85e4469719 is removed from metadata 2024.10.22 00:38:17.589720 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:17.649435 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.83 GiB, free memory in arenas 7.26 MiB, will set to 1.64 GiB (RSS), difference: 2.25 MiB 2024.10.22 00:38:17.650916 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_146_29 to 202410_151_151_0 into Compact with storage Full 2024.10.22 00:38:17.690145 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1933 2024.10.22 00:38:17.767802 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_29_6/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:38:17.888922 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:38:17.897616 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_146_29, total 1968 rows starting from the beginning of the part 2024.10.22 00:38:17.930676 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_147_147_0, total 20 rows starting from the beginning of the part 2024.10.22 00:38:17.986378 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_148_148_0, total 5 rows starting from the beginning of the part 2024.10.22 00:38:18.021835 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_149_149_0, total 26 rows starting from the beginning of the part 2024.10.22 00:38:18.163134 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_150_150_0, total 14 rows starting from the beginning of the part 2024.10.22 00:38:18.187644 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_151_151_0, total 6 rows starting from the beginning of the part 2024.10.22 00:38:18.565171 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/crk/ijyqbozfrkpompfysqlxifkabynvc. 2024.10.22 00:38:18.594973 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uum/buvvnrtypiocjupyegkhpxhmgrjmz. 2024.10.22 00:38:19.061075 [ 1736 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jwc/towfughomidftgvoqqxuvwouuvnia, size 1717842 2024.10.22 00:38:19.273840 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 45 entries to flush up to offset 9065 2024.10.22 00:38:19.278329 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} MergeTask::MergeProjectionsStage: Merge sorted 2039 rows, containing 12 columns (12 merged, 0 gathered) in 1.654482292 sec., 1232.4096848054992 rows/sec., 190.72 KiB/sec. 2024.10.22 00:38:19.284398 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:38:19.293666 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:38:19.326290 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:38:19.369969 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:38:19.387199 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:19.389079 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:19.479132 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.83 GiB, free memory in arenas 5.62 MiB, will set to 1.64 GiB (RSS), difference: 1.97 MiB 2024.10.22 00:38:19.484643 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:38:19.500512 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:38:19.506483 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:38:19.511749 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:38:19.512567 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_137_137_0 (state Temporary) 2024.10.22 00:38:19.523958 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uum/buvvnrtypiocjupyegkhpxhmgrjmz. 2024.10.22 00:38:19.530650 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_9_9_0 to 201403_9_9_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:19.564963 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_137_137_0 to 202410_143_143_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:19.567933 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/crk/ijyqbozfrkpompfysqlxifkabynvc. 2024.10.22 00:38:19.573653 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:38:19.589576 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_10_10_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:38:19.596719 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} MergedBlockOutputStream: filled checksums 202410_1_151_30 (state Temporary) 2024.10.22 00:38:19.637446 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9065 2024.10.22 00:38:19.669044 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yek/kglxnvmsdximxkjzxumezotfodekj (LogSeriesLimiter: on interval from 2024-10-22 00:38:02 to 2024-10-22 00:38:19 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:38:20.225689 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_151_30 to 202410_1_151_30 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:20.336854 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_151_30} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_146_29, 202410_151_151_0] -> 202410_1_151_30 2024.10.22 00:38:20.481641 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:38:21.103662 [ 1189 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:38:21.123811 [ 1189 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:38:21.271224 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 12647 2024.10.22 00:38:21.531763 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.13 GiB, peak 1.83 GiB, free memory in arenas 6.79 MiB, will set to 1.37 GiB (RSS), difference: 242.70 MiB 2024.10.22 00:38:21.561096 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:21.563066 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:21.715700 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_238_238_0 (state Temporary) 2024.10.22 00:38:21.964892 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_238_238_0 to 202410_251_251_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:21.991009 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_246_49 to 202410_251_251_0 2024.10.22 00:38:21.992456 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12647 2024.10.22 00:38:21.998677 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:22.001726 [ 1198 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:22.138763 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_246_49 to 202410_251_251_0 into Compact with storage Full 2024.10.22 00:38:22.246674 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:38:22.260807 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_246_49, total 13064 rows starting from the beginning of the part 2024.10.22 00:38:22.276890 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_247_247_0, total 126 rows starting from the beginning of the part 2024.10.22 00:38:22.465293 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 9 2024.10.22 00:38:22.483841 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.83 GiB, free memory in arenas 6.90 MiB, will set to 1.40 GiB (RSS), difference: -65.81 MiB 2024.10.22 00:38:22.488348 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_248_248_0, total 50 rows starting from the beginning of the part 2024.10.22 00:38:22.616786 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_249_249_0, total 148 rows starting from the beginning of the part 2024.10.22 00:38:22.654961 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_250_250_0, total 30 rows starting from the beginning of the part 2024.10.22 00:38:22.680128 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_251_251_0, total 60 rows starting from the beginning of the part 2024.10.22 00:38:22.906927 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:22.911361 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:23.080701 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 232094 2024.10.22 00:38:23.140928 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_9_9_0 (state Temporary) 2024.10.22 00:38:23.216427 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_9_9_0 to 202410_9_9_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:23.395573 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 9 2024.10.22 00:38:23.758102 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 129 2024.10.22 00:38:23.854320 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:23.863314 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:23.994895 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_133_133_0 (state Temporary) 2024.10.22 00:38:24.071522 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_133_133_0 to 202410_140_140_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:24.153121 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 232094 2024.10.22 00:38:24.236494 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} MergeTask::MergeProjectionsStage: Merge sorted 13478 rows, containing 13 columns (13 merged, 0 gathered) in 2.087537996 sec., 6456.409428631066 rows/sec., 983.18 KiB/sec. 2024.10.22 00:38:24.332503 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:24.337536 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:24.497224 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_111_111_0 (state Temporary) 2024.10.22 00:38:24.681358 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_111_111_0 to 202410_121_121_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:24.682408 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} MergedBlockOutputStream: filled checksums 202410_1_251_50 (state Temporary) 2024.10.22 00:38:24.765010 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 129 2024.10.22 00:38:24.880744 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_251_50 to 202410_1_251_50 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:24.985114 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_251_50} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_246_49, 202410_251_251_0] -> 202410_1_251_50 2024.10.22 00:38:25.178419 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_146_29, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:38:25.179991 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_146_29, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:38:25.216742 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_146_29, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:38:25.335143 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.82 MiB. 2024.10.22 00:38:25.360567 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 1954 2024.10.22 00:38:25.469671 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:25.471862 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:25.507665 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_139_139_0 (state Temporary) 2024.10.22 00:38:25.562655 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_139_139_0 to 202410_152_152_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:25.692711 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1954 2024.10.22 00:38:26.015829 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 12698 2024.10.22 00:38:26.168932 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:26.171417 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:26.367561 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.83 GiB, free memory in arenas 12.42 MiB, will set to 1.46 GiB (RSS), difference: -23.61 MiB 2024.10.22 00:38:26.419797 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_239_239_0 (state Temporary) 2024.10.22 00:38:26.601164 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_239_239_0 to 202410_252_252_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:26.678784 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12698 2024.10.22 00:38:27.156020 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 60 entries to flush up to offset 9125 2024.10.22 00:38:27.419984 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:27.422174 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:27.774280 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_138_138_0 (state Temporary) 2024.10.22 00:38:27.920984 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_138_138_0 to 202410_144_144_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:27.987745 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9125 2024.10.22 00:38:28.980462 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.83 GiB, free memory in arenas 11.79 MiB, will set to 1.52 GiB (RSS), difference: -79.86 MiB 2024.10.22 00:38:29.347609 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_80_16, 202410_81_81_0, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0] 2024.10.22 00:38:29.351179 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_80_16, 202410_81_81_0, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0] 2024.10.22 00:38:29.484917 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_80_16, 202410_81_81_0, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0] 2024.10.22 00:38:29.501771 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:38:29.521650 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:38:29.779958 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:29.783013 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:30.438940 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.83 GiB, free memory in arenas 11.68 MiB, will set to 1.57 GiB (RSS), difference: -27.57 MiB 2024.10.22 00:38:30.687224 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 22 entries to flush up to offset 12720 2024.10.22 00:38:31.148936 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_29_6/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:38:31.165095 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1472 entries to flush up to offset 233566 2024.10.22 00:38:31.224820 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:31.231647 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:31.278819 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_240_240_0 (state Temporary) 2024.10.22 00:38:31.325683 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_240_240_0 to 202410_253_253_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:31.382355 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12720 2024.10.22 00:38:31.677760 [ 1251 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 322 ms. 2024.10.22 00:38:31.801715 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:31.806558 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:32.325107 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_134_134_0 (state Temporary) 2024.10.22 00:38:32.367622 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 130 2024.10.22 00:38:32.508127 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_134_134_0 to 202410_141_141_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:32.655672 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_10_10_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:38:32.657773 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 233566 2024.10.22 00:38:32.663007 [ 1242 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_136_27 to 202410_141_141_0 2024.10.22 00:38:32.681143 [ 1242 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:32.691337 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.83 GiB, free memory in arenas 12.11 MiB, will set to 1.63 GiB (RSS), difference: -163.83 MiB 2024.10.22 00:38:32.739109 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:32.685793 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:32.788963 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:32.792616 [ 1242 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 266 ms. 2024.10.22 00:38:32.850311 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_136_27 to 202410_141_141_0 into Compact with storage Full 2024.10.22 00:38:32.889517 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_112_112_0 (state Temporary) 2024.10.22 00:38:32.991312 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:38:33.002626 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} MergeTreeSequentialSource: Reading 31 marks from part 202410_1_136_27, total 238772 rows starting from the beginning of the part 2024.10.22 00:38:33.015450 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_112_112_0 to 202410_122_122_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:33.039577 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_137_137_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:38:33.087494 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_138_138_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:38:33.123872 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_139_139_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:38:33.138477 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 130 2024.10.22 00:38:33.155975 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_140_140_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:38:33.194781 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 1960 2024.10.22 00:38:33.227103 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_69_14, 202410_70_70_0, 202410_71_71_0, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0] 2024.10.22 00:38:33.231818 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_69_14, 202410_70_70_0, 202410_71_71_0, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0] 2024.10.22 00:38:33.284251 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_141_141_0, total 1472 rows starting from the beginning of the part 2024.10.22 00:38:33.346268 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:33.366793 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:33.374058 [ 1248 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 00:38:33.400008 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_140_140_0 (state Temporary) 2024.10.22 00:38:33.472860 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_140_140_0 to 202410_153_153_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:33.495273 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_28_28_0 (state Temporary) 2024.10.22 00:38:33.511795 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_69_14, 202410_70_70_0, 202410_71_71_0, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0] 2024.10.22 00:38:33.525420 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1960 2024.10.22 00:38:34.618341 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 12.26 MiB, will set to 1.81 GiB (RSS), difference: 89.34 MiB 2024.10.22 00:38:35.079573 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_141_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:38:35.387787 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 12767 2024.10.22 00:38:35.508326 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 75 entries to flush up to offset 9200 2024.10.22 00:38:35.871534 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:35.875635 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:36.115012 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_139_139_0 (state Temporary) 2024.10.22 00:38:36.131958 [ 1274 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:38:36.183929 [ 1274 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:38:36.232689 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_139_139_0 to 202410_145_145_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:36.243633 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:36.248719 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:36.274329 [ 1272 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Selected 5 parts from 202410_1_5_1 to 202410_9_9_0 2024.10.22 00:38:36.285436 [ 1272 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:36.289370 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_140_28 to 202410_145_145_0 2024.10.22 00:38:36.291399 [ 1272 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:36.303021 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_241_241_0 (state Temporary) 2024.10.22 00:38:36.303806 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9200 2024.10.22 00:38:36.306690 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:36.318583 [ 1150 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:36.360270 [ 1394 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_9_2} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_5_1 to 202410_9_9_0 into Compact with storage Full 2024.10.22 00:38:36.378860 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_140_28 to 202410_145_145_0 into Compact with storage Full 2024.10.22 00:38:36.412844 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 9.71 MiB, will set to 1.61 GiB (RSS), difference: 187.46 MiB 2024.10.22 00:38:36.424287 [ 1394 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_9_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:38:36.439599 [ 1394 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_5_1, total 5 rows starting from the beginning of the part 2024.10.22 00:38:36.445770 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:38:36.459846 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_140_28, total 8995 rows starting from the beginning of the part 2024.10.22 00:38:36.474391 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_241_241_0 to 202410_254_254_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:36.480075 [ 1394 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_6_6_0, total 1 rows starting from the beginning of the part 2024.10.22 00:38:36.480493 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_141_141_0, total 91 rows starting from the beginning of the part 2024.10.22 00:38:36.532079 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_142_142_0, total 38 rows starting from the beginning of the part 2024.10.22 00:38:36.561314 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12767 2024.10.22 00:38:36.572636 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_143_143_0, total 45 rows starting from the beginning of the part 2024.10.22 00:38:36.604417 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_144_144_0, total 60 rows starting from the beginning of the part 2024.10.22 00:38:36.628731 [ 1394 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_7_7_0, total 1 rows starting from the beginning of the part 2024.10.22 00:38:36.648201 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_145_145_0, total 75 rows starting from the beginning of the part 2024.10.22 00:38:36.660185 [ 1680 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yek/kglxnvmsdximxkjzxumezotfodekj, size 30642 2024.10.22 00:38:36.681420 [ 1394 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_8_8_0, total 1 rows starting from the beginning of the part 2024.10.22 00:38:36.730952 [ 1394 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_9_9_0, total 1 rows starting from the beginning of the part 2024.10.22 00:38:37.252052 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OS 2024.10.22 00:38:37.283020 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OS 2024.10.22 00:38:37.293073 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OS 2024.10.22 00:38:37.375706 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 489.89 MiB using storage policy from min volume index 0 2024.10.22 00:38:37.410770 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OS 2024.10.22 00:38:37.442737 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OS 2024.10.22 00:38:37.464161 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OS 2024.10.22 00:38:37.476224 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OS 2024.10.22 00:38:37.568795 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lxg/fzsnpecoohsliawklraeucgpslnsg (LogSeriesLimiter: on interval from 2024-10-22 00:38:19 to 2024-10-22 00:38:37 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:38:38.572592 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 9.41 MiB, will set to 1.62 GiB (RSS), difference: -4.82 MiB 2024.10.22 00:38:38.717615 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} MergeTask::MergeProjectionsStage: Merge sorted 9304 rows, containing 13 columns (13 merged, 0 gathered) in 2.347518888 sec., 3963.3333932093155 rows/sec., 1.66 MiB/sec. 2024.10.22 00:38:38.728713 [ 1398 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_9_2} MergeTask::MergeProjectionsStage: Merge sorted 9 rows, containing 27 columns (27 merged, 0 gathered) in 2.394518716 sec., 3.758584111229808 rows/sec., 12.07 KiB/sec. 2024.10.22 00:38:38.747537 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} MergedBlockOutputStream: filled checksums 202410_1_145_29 (state Temporary) 2024.10.22 00:38:38.762840 [ 1400 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_9_2} MergedBlockOutputStream: filled checksums 202410_1_9_2 (state Temporary) 2024.10.22 00:38:38.983086 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_145_29 to 202410_1_145_29 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:38.987602 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_145_29} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_140_28, 202410_145_145_0] -> 202410_1_145_29 2024.10.22 00:38:39.049675 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.39 MiB. 2024.10.22 00:38:39.281046 [ 1388 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_9_2} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_merge_202410_1_9_2 to 202410_1_9_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:39.288147 [ 1388 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_9_2} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Merged 5 parts: [202410_1_5_1, 202410_9_9_0] -> 202410_1_9_2 2024.10.22 00:38:39.443586 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.24 MiB. 2024.10.22 00:38:39.509597 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:38:39.515917 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:38:39.684673 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1600 entries to flush up to offset 235166 2024.10.22 00:38:40.413836 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 12.39 MiB, will set to 1.62 GiB (RSS), difference: -105.02 MiB 2024.10.22 00:38:40.475678 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:40.477533 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:40.571399 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 12830 2024.10.22 00:38:40.643803 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 131 2024.10.22 00:38:40.788925 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:40.791192 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:40.820949 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_135_135_0 (state Temporary) 2024.10.22 00:38:40.854555 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_242_242_0 (state Temporary) 2024.10.22 00:38:40.893605 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_242_242_0 to 202410_255_255_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:40.899139 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_135_135_0 to 202410_142_142_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:40.948138 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12830 2024.10.22 00:38:41.031070 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 1978 2024.10.22 00:38:41.118299 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:41.125362 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:41.147584 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 235166 2024.10.22 00:38:41.162143 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_141_141_0 (state Temporary) 2024.10.22 00:38:41.215791 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:41.220452 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_141_141_0 to 202410_154_154_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:41.229220 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:41.256558 [ 1169 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 00:38:41.291753 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_113_113_0 (state Temporary) 2024.10.22 00:38:41.485033 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_113_113_0 to 202410_123_123_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:41.591102 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 1978 2024.10.22 00:38:41.625041 [ 1187 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_118_24 to 202410_123_123_0 2024.10.22 00:38:41.656211 [ 1187 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:41.664053 [ 1187 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:41.689553 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_118_24 to 202410_123_123_0 into Compact with storage Full 2024.10.22 00:38:41.768156 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 131 2024.10.22 00:38:41.775573 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:38:41.786786 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_118_24, total 162 rows starting from the beginning of the part 2024.10.22 00:38:41.895534 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_119_119_0, total 1 rows starting from the beginning of the part 2024.10.22 00:38:42.006695 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gjr/rlolkbcgrgvtrxaawjpqpajysjeof (LogSeriesLimiter: on interval from 2024-10-22 00:38:37 to 2024-10-22 00:38:42 accepted series 1 / 137 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:38:42.055534 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_120_120_0, total 1 rows starting from the beginning of the part 2024.10.22 00:38:42.068408 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_29_6/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:38:42.134137 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_121_121_0, total 1 rows starting from the beginning of the part 2024.10.22 00:38:42.177993 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_28_28_0 to 202410_31_31_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:42.325060 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_122_122_0, total 1 rows starting from the beginning of the part 2024.10.22 00:38:42.394574 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_123_123_0, total 1 rows starting from the beginning of the part 2024.10.22 00:38:42.543226 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.88 GiB, free memory in arenas 9.16 MiB, will set to 1.62 GiB (RSS), difference: -140.20 MiB 2024.10.22 00:38:42.994833 [ 1173 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 213 ms. 2024.10.22 00:38:43.928096 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} MergeTask::MergeProjectionsStage: Merge sorted 247156 rows, containing 4 columns (4 merged, 0 gathered) in 11.12920834 sec., 22207.86892017155 rows/sec., 326.05 KiB/sec. 2024.10.22 00:38:43.929278 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 87 entries to flush up to offset 9287 2024.10.22 00:38:43.917757 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} MergeTask::MergeProjectionsStage: Merge sorted 167 rows, containing 32 columns (32 merged, 0 gathered) in 2.232088923 sec., 74.81780778498133 rows/sec., 8.29 KiB/sec. 2024.10.22 00:38:43.973738 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} MergedBlockOutputStream: filled checksums 202410_1_123_25 (state Temporary) 2024.10.22 00:38:43.978210 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} MergedBlockOutputStream: filled checksums 202410_1_141_28 (state Temporary) 2024.10.22 00:38:44.137063 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_141_28 to 202410_1_141_28 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:44.139094 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:44.143018 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_141_28} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_136_27, 202410_141_141_0] -> 202410_1_141_28 2024.10.22 00:38:44.146688 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:44.188910 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:38:44.190539 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_140_140_0 (state Temporary) 2024.10.22 00:38:44.246587 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_140_140_0 to 202410_146_146_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:44.335323 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9287 2024.10.22 00:38:44.367664 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_123_25 to 202410_1_123_25 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:44.371860 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_123_25} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_118_24, 202410_123_123_0] -> 202410_1_123_25 2024.10.22 00:38:44.420685 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:38:44.431339 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.88 GiB, free memory in arenas 7.11 MiB, will set to 1.61 GiB (RSS), difference: -133.10 MiB 2024.10.22 00:38:44.771237 [ 1181 ] {} DatabaseCatalog: Execution took 201 ms. 2024.10.22 00:38:44.922304 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1111 2024.10.22 00:38:44.977653 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 12887 2024.10.22 00:38:45.205052 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:45.210730 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:45.281764 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_243_243_0 (state Temporary) 2024.10.22 00:38:45.431329 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_243_243_0 to 202410_256_256_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:45.471606 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_251_50 to 202410_256_256_0 2024.10.22 00:38:45.478191 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:45.480219 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:45.506788 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12887 2024.10.22 00:38:45.565043 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 10.11 MiB, will set to 1.61 GiB (RSS), difference: -76.82 MiB 2024.10.22 00:38:45.566832 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} MergeTask::MergeProjectionsStage: Merge sorted 1087 rows, containing 679 columns (679 merged, 0 gathered) in 85.263628565 sec., 12.748695056665747 rows/sec., 16.20 KiB/sec. 2024.10.22 00:38:45.621994 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_251_50 to 202410_256_256_0 into Compact with storage Full 2024.10.22 00:38:45.692836 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:38:45.705776 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_251_50, total 13478 rows starting from the beginning of the part 2024.10.22 00:38:45.727276 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_252_252_0, total 51 rows starting from the beginning of the part 2024.10.22 00:38:45.750103 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_253_253_0, total 22 rows starting from the beginning of the part 2024.10.22 00:38:45.769083 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_254_254_0, total 47 rows starting from the beginning of the part 2024.10.22 00:38:45.779422 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_255_255_0, total 63 rows starting from the beginning of the part 2024.10.22 00:38:45.791458 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_256_256_0, total 57 rows starting from the beginning of the part 2024.10.22 00:38:45.814905 [ 1235 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 202 ms. 2024.10.22 00:38:47.318550 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} MergedBlockOutputStream: filled checksums 202410_1_29_6 (state Temporary) 2024.10.22 00:38:47.370476 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} MergeTask::MergeProjectionsStage: Merge sorted 13718 rows, containing 13 columns (13 merged, 0 gathered) in 1.874752232 sec., 7317.233587376788 rows/sec., 1.09 MiB/sec. 2024.10.22 00:38:47.428737 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} MergedBlockOutputStream: filled checksums 202410_1_256_51 (state Temporary) 2024.10.22 00:38:47.783417 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_256_51 to 202410_1_256_51 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:47.800993 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_256_51} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_251_50, 202410_256_256_0] -> 202410_1_256_51 2024.10.22 00:38:47.843011 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 8.64 MiB, will set to 1.61 GiB (RSS), difference: -70.15 MiB 2024.10.22 00:38:47.975963 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.78 MiB. 2024.10.22 00:38:48.152735 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1555 entries to flush up to offset 236721 2024.10.22 00:38:48.343883 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_151_30, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:38:48.347554 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_151_30, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:38:48.393096 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_151_30, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:38:49.093780 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 2004 2024.10.22 00:38:49.289546 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:49.455069 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:49.477412 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_81_16, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0] 2024.10.22 00:38:49.484656 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_81_16, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0] 2024.10.22 00:38:49.487722 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:49.494361 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:49.511442 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:38:49.517555 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:38:49.549648 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 8.11 MiB, will set to 1.62 GiB (RSS), difference: -5.06 MiB 2024.10.22 00:38:49.633361 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 12928 2024.10.22 00:38:49.712207 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_81_16, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0] 2024.10.22 00:38:49.766302 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_142_142_0 (state Temporary) 2024.10.22 00:38:49.797044 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:49.801579 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:49.811098 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_142_142_0 to 202410_155_155_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:49.861661 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_136_136_0 (state Temporary) 2024.10.22 00:38:49.864889 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_244_244_0 (state Temporary) 2024.10.22 00:38:49.891766 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2004 2024.10.22 00:38:49.916696 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_244_244_0 to 202410_257_257_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:49.955878 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 00:38:50.080290 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_136_136_0 to 202410_143_143_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:50.088150 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12928 2024.10.22 00:38:50.223238 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 236721 2024.10.22 00:38:51.188057 [ 1237 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:38:51.299053 [ 1237 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:38:51.695833 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 8.66 MiB, will set to 1.63 GiB (RSS), difference: -6.29 MiB 2024.10.22 00:38:51.859144 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 53 entries to flush up to offset 9340 2024.10.22 00:38:52.085110 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:52.087414 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:52.094842 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 679840 bytes for 11092f76ce2a13549fee4ceefa8d58c4:0) 2024.10.22 00:38:52.099192 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 423c886e239cc355f4e684dc5b60cf30, offset: 0, size: 2040360 2024.10.22 00:38:52.110267 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 423c886e239cc355f4e684dc5b60cf30 for removal 2024.10.22 00:38:52.113212 [ 1476 ] {} CacheMetadata: Key 423c886e239cc355f4e684dc5b60cf30 is removed from metadata 2024.10.22 00:38:52.129836 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_141_141_0 (state Temporary) 2024.10.22 00:38:52.175713 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_141_141_0 to 202410_147_147_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:52.281270 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9340 2024.10.22 00:38:52.293749 [ 1718 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lxg/fzsnpecoohsliawklraeucgpslnsg, size 679840 2024.10.22 00:38:52.562558 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 1151 2024.10.22 00:38:52.594797 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UserAgent 2024.10.22 00:38:52.608704 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UserAgent 2024.10.22 00:38:52.709840 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UserAgent 2024.10.22 00:38:52.729503 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UserAgent 2024.10.22 00:38:52.753236 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UserAgent 2024.10.22 00:38:52.769612 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UserAgent 2024.10.22 00:38:52.801367 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UserAgent 2024.10.22 00:38:52.851756 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/cqv/nupspueycgrtlvvjjrkworsphwcyb (LogSeriesLimiter: on interval from 2024-10-22 00:38:42 to 2024-10-22 00:38:52 accepted series 1 / 218 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:38:53.440395 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 9.12 MiB, will set to 1.63 GiB (RSS), difference: -3.19 MiB 2024.10.22 00:38:54.117332 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 12966 2024.10.22 00:38:54.314982 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:54.317151 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:54.567472 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_245_245_0 (state Temporary) 2024.10.22 00:38:54.608877 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_245_245_0 to 202410_258_258_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:54.867812 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 12966 2024.10.22 00:38:55.088881 [ 1204 ] {} DatabaseCatalog: Execution took 318 ms. 2024.10.22 00:38:55.553727 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 7.97 MiB, will set to 1.65 GiB (RSS), difference: -12.68 MiB 2024.10.22 00:38:56.770904 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 132 2024.10.22 00:38:56.773786 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_29_6/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:38:57.087945 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:57.090367 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:57.148951 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_91_18, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0] 2024.10.22 00:38:57.152199 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_91_18, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0] 2024.10.22 00:38:57.160937 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_114_114_0 (state Temporary) 2024.10.22 00:38:57.194403 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_91_18, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0] 2024.10.22 00:38:57.229718 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1901 entries to flush up to offset 238622 2024.10.22 00:38:57.254576 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_114_114_0 to 202410_124_124_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:57.325467 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 132 2024.10.22 00:38:57.393698 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 2010 2024.10.22 00:38:57.414248 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 8.22 MiB, will set to 1.65 GiB (RSS), difference: 4.23 MiB 2024.10.22 00:38:57.652655 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:57.655454 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:57.684929 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_143_143_0 (state Temporary) 2024.10.22 00:38:57.876520 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_143_143_0 to 202410_156_156_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:58.000971 [ 1158 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_151_30 to 202410_156_156_0 2024.10.22 00:38:58.031045 [ 1158 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:58.036862 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2010 2024.10.22 00:38:58.047555 [ 1158 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:58.169057 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_151_30 to 202410_156_156_0 into Compact with storage Full 2024.10.22 00:38:58.334641 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:38:58.365419 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_151_30, total 2039 rows starting from the beginning of the part 2024.10.22 00:38:58.389358 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_152_152_0, total 21 rows starting from the beginning of the part 2024.10.22 00:38:58.394108 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:58.400442 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:58.423874 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_153_153_0, total 6 rows starting from the beginning of the part 2024.10.22 00:38:58.459877 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_154_154_0, total 18 rows starting from the beginning of the part 2024.10.22 00:38:58.489735 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_155_155_0, total 26 rows starting from the beginning of the part 2024.10.22 00:38:58.538927 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_156_156_0, total 6 rows starting from the beginning of the part 2024.10.22 00:38:58.611501 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 7.27 MiB, will set to 1.65 GiB (RSS), difference: -3.96 MiB 2024.10.22 00:38:58.651117 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_137_137_0 (state Temporary) 2024.10.22 00:38:58.755814 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_137_137_0 to 202410_144_144_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:58.809394 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 238622 2024.10.22 00:38:58.959441 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 13009 2024.10.22 00:38:59.062598 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:38:59.065205 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:38:59.104823 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_246_246_0 (state Temporary) 2024.10.22 00:38:59.161344 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_246_246_0 to 202410_259_259_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:38:59.203917 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 13009 2024.10.22 00:38:59.514403 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:38:59.517476 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:38:59.784935 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 44 entries to flush up to offset 9384 2024.10.22 00:39:00.111947 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} MergeTask::MergeProjectionsStage: Merge sorted 2116 rows, containing 12 columns (12 merged, 0 gathered) in 1.948178844 sec., 1086.1425820924271 rows/sec., 168.25 KiB/sec. 2024.10.22 00:39:00.146788 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} MergedBlockOutputStream: filled checksums 202410_1_156_31 (state Temporary) 2024.10.22 00:39:00.228919 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:00.231843 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:39:00.318547 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_156_31 to 202410_1_156_31 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:00.321371 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_156_31} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_151_30, 202410_156_156_0] -> 202410_1_156_31 2024.10.22 00:39:00.378638 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:39:00.380921 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_142_142_0 (state Temporary) 2024.10.22 00:39:00.534949 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_142_142_0 to 202410_148_148_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:00.573282 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 3.48 MiB, will set to 1.66 GiB (RSS), difference: 10.02 MiB 2024.10.22 00:39:00.589931 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9384 2024.10.22 00:39:01.095067 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_29_6 to 202410_1_29_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:01.105280 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_29_6} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_24_5, 202410_29_29_0] -> 202410_1_29_6 2024.10.22 00:39:02.222649 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.60 MiB. 2024.10.22 00:39:02.491742 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 3.72 MiB, will set to 1.66 GiB (RSS), difference: 10.35 MiB 2024.10.22 00:39:03.212647 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 24 entries to flush up to offset 13033 2024.10.22 00:39:03.323102 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:03.327429 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:39:03.378225 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_247_247_0 (state Temporary) 2024.10.22 00:39:03.420867 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_247_247_0 to 202410_260_260_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:03.487930 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 13033 2024.10.22 00:39:04.774193 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 3.89 MiB, will set to 1.67 GiB (RSS), difference: 3.64 MiB 2024.10.22 00:39:04.829360 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 133 2024.10.22 00:39:04.873447 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:39:05.025583 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:05.030836 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:39:05.303840 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_115_115_0 (state Temporary) 2024.10.22 00:39:05.439300 [ 1203 ] {} DatabaseCatalog: Execution took 345 ms. 2024.10.22 00:39:05.487612 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_115_115_0 to 202410_125_125_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:05.543406 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 2030 2024.10.22 00:39:05.617095 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 133 2024.10.22 00:39:05.758417 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:05.762569 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:39:05.790381 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_144_144_0 (state Temporary) 2024.10.22 00:39:05.842010 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_144_144_0 to 202410_157_157_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:05.940982 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 240542 2024.10.22 00:39:06.241034 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2030 2024.10.22 00:39:06.314518 [ 1217 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:39:06.325507 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 3.33 MiB, will set to 1.67 GiB (RSS), difference: 4.73 MiB 2024.10.22 00:39:06.353470 [ 1217 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:39:06.775068 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:06.778439 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:39:06.817498 [ 1728 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/cqv/nupspueycgrtlvvjjrkworsphwcyb, size 809752 2024.10.22 00:39:07.155666 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column URL 2024.10.22 00:39:07.174235 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column URL 2024.10.22 00:39:07.180152 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column URL 2024.10.22 00:39:07.188597 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column URL 2024.10.22 00:39:07.200315 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column URL 2024.10.22 00:39:07.217319 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column URL 2024.10.22 00:39:07.257826 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column URL 2024.10.22 00:39:07.303727 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nha/fqnkkorkipggbolblpoywpaztgkzw (LogSeriesLimiter: on interval from 2024-10-22 00:38:52 to 2024-10-22 00:39:07 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:39:07.367794 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:07.371842 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:39:07.495964 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 13071 2024.10.22 00:39:07.543637 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_138_138_0 (state Temporary) 2024.10.22 00:39:07.718759 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_138_138_0 to 202410_145_145_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:07.872867 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 240542 2024.10.22 00:39:07.928839 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:07.935491 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:39:07.996219 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_248_248_0 (state Temporary) 2024.10.22 00:39:08.073813 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_248_248_0 to 202410_261_261_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:08.096051 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 27 entries to flush up to offset 9411 2024.10.22 00:39:08.121500 [ 1222 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_256_51 to 202410_261_261_0 2024.10.22 00:39:08.151853 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 13071 2024.10.22 00:39:08.231528 [ 1222 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:08.235444 [ 1222 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:39:08.348178 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:08.359333 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.07 GiB. 2024.10.22 00:39:08.486490 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_143_143_0 (state Temporary) 2024.10.22 00:39:08.547646 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_143_143_0 to 202410_149_149_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:08.581588 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 4.41 MiB, will set to 1.68 GiB (RSS), difference: -21.73 MiB 2024.10.22 00:39:08.481738 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_256_51 to 202410_261_261_0 into Compact with storage Full 2024.10.22 00:39:08.595137 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9411 2024.10.22 00:39:08.672353 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:39:08.685374 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_256_51, total 13718 rows starting from the beginning of the part 2024.10.22 00:39:08.696687 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_257_257_0, total 41 rows starting from the beginning of the part 2024.10.22 00:39:08.716550 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_258_258_0, total 38 rows starting from the beginning of the part 2024.10.22 00:39:08.737307 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_259_259_0, total 43 rows starting from the beginning of the part 2024.10.22 00:39:08.768263 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_260_260_0, total 24 rows starting from the beginning of the part 2024.10.22 00:39:08.791955 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_261_261_0, total 38 rows starting from the beginning of the part 2024.10.22 00:39:09.409275 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_29_29_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:39:09.516428 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:39:09.521179 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:39:09.722456 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 1048576 bytes for 7a2eb8cea9c17e12aec9d150ec4c29b7:0) 2024.10.22 00:39:09.727611 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4500e28349710cd3e88526f5c4b934fe, offset: 0, size: 2106195 2024.10.22 00:39:09.832837 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4500e28349710cd3e88526f5c4b934fe for removal 2024.10.22 00:39:09.835873 [ 1476 ] {} CacheMetadata: Key 4500e28349710cd3e88526f5c4b934fe is removed from metadata 2024.10.22 00:39:10.438262 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 3.99 MiB, will set to 1.71 GiB (RSS), difference: 4.17 MiB 2024.10.22 00:39:10.506865 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_29_29_0 (state Temporary) 2024.10.22 00:39:10.577270 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} MergeTask::MergeProjectionsStage: Merge sorted 13902 rows, containing 13 columns (13 merged, 0 gathered) in 2.057249904 sec., 6757.565025507956 rows/sec., 1.00 MiB/sec. 2024.10.22 00:39:10.632472 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_85_17, 202410_86_86_0, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0] 2024.10.22 00:39:10.635947 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_85_17, 202410_86_86_0, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0] 2024.10.22 00:39:10.665640 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_85_17, 202410_86_86_0, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0] 2024.10.22 00:39:10.759881 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} MergedBlockOutputStream: filled checksums 202410_1_261_52 (state Temporary) 2024.10.22 00:39:10.988636 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_261_52 to 202410_1_261_52 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:10.992400 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_261_52} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_256_51, 202410_261_261_0] -> 202410_1_261_52 2024.10.22 00:39:11.048597 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.75 MiB. 2024.10.22 00:39:11.192395 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_156_31, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:39:11.193743 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_156_31, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:39:11.218065 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_156_31, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:39:11.531301 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 4.68 MiB, will set to 1.71 GiB (RSS), difference: 2.84 MiB 2024.10.22 00:39:12.230228 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 13119 2024.10.22 00:39:12.376655 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 2097152 bytes for 7a2eb8cea9c17e12aec9d150ec4c29b7:0) 2024.10.22 00:39:12.379224 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0112ceefeef4d9eea8435dcc8cf8fce8, offset: 0, size: 4080778 2024.10.22 00:39:12.381580 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0112ceefeef4d9eea8435dcc8cf8fce8 for removal 2024.10.22 00:39:12.388071 [ 1476 ] {} CacheMetadata: Key 0112ceefeef4d9eea8435dcc8cf8fce8 is removed from metadata 2024.10.22 00:39:12.481303 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:12.494854 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:39:12.661210 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_249_249_0 (state Temporary) 2024.10.22 00:39:12.696277 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_249_249_0 to 202410_262_262_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:12.764213 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 13119 2024.10.22 00:39:13.119368 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 134 2024.10.22 00:39:13.388771 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:13.391873 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:39:13.562481 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_116_116_0 (state Temporary) 2024.10.22 00:39:13.581127 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 4.11 MiB, will set to 1.71 GiB (RSS), difference: -10.76 MiB 2024.10.22 00:39:13.662605 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_116_116_0 to 202410_126_126_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:13.746385 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 2043 2024.10.22 00:39:13.849124 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:13.852532 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.08 GiB. 2024.10.22 00:39:13.878232 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_145_145_0 (state Temporary) 2024.10.22 00:39:13.931088 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 134 2024.10.22 00:39:14.025218 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_145_145_0 to 202410_158_158_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:14.089672 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2043 2024.10.22 00:39:14.734249 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 10 file segments (while reserving 4194304 bytes for 7a2eb8cea9c17e12aec9d150ec4c29b7:0) 2024.10.22 00:39:14.737762 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 854f10257937f71db2f581a95fb90b19, offset: 0, size: 33554432 2024.10.22 00:39:14.756657 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cd3db5ca3c546920afcd9a122e6a8bc2, offset: 0, size: 72042 2024.10.22 00:39:14.763427 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cd3db5ca3c546920afcd9a122e6a8bc2 for removal 2024.10.22 00:39:14.766864 [ 1476 ] {} CacheMetadata: Key cd3db5ca3c546920afcd9a122e6a8bc2 is removed from metadata 2024.10.22 00:39:14.867428 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2f803c801dce23c32124c8d263f570a6, offset: 0, size: 14274 2024.10.22 00:39:14.877254 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 242462 2024.10.22 00:39:14.882304 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2f803c801dce23c32124c8d263f570a6 for removal 2024.10.22 00:39:14.884576 [ 1476 ] {} CacheMetadata: Key 2f803c801dce23c32124c8d263f570a6 is removed from metadata 2024.10.22 00:39:14.887121 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8aac696270e5e0e0ad88083e1d4fd580, offset: 0, size: 39251 2024.10.22 00:39:14.889471 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8aac696270e5e0e0ad88083e1d4fd580 for removal 2024.10.22 00:39:14.891870 [ 1476 ] {} CacheMetadata: Key 8aac696270e5e0e0ad88083e1d4fd580 is removed from metadata 2024.10.22 00:39:14.894351 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1f41b1b1b4e8ac24b22452cf64a4b5ab, offset: 0, size: 55242 2024.10.22 00:39:14.904448 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1f41b1b1b4e8ac24b22452cf64a4b5ab for removal 2024.10.22 00:39:14.910454 [ 1476 ] {} CacheMetadata: Key 1f41b1b1b4e8ac24b22452cf64a4b5ab is removed from metadata 2024.10.22 00:39:14.923023 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f8781c12ddb23be0a5df9b65435dc3cf, offset: 0, size: 154 2024.10.22 00:39:14.926630 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f8781c12ddb23be0a5df9b65435dc3cf for removal 2024.10.22 00:39:14.931244 [ 1476 ] {} CacheMetadata: Key f8781c12ddb23be0a5df9b65435dc3cf is removed from metadata 2024.10.22 00:39:14.930618 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a75572f8af3dec3c906906ece6f860ba, offset: 0, size: 176 2024.10.22 00:39:14.955614 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a75572f8af3dec3c906906ece6f860ba for removal 2024.10.22 00:39:14.959905 [ 1476 ] {} CacheMetadata: Key a75572f8af3dec3c906906ece6f860ba is removed from metadata 2024.10.22 00:39:14.960657 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 560477d5cf448a0bb1d5d66a4c74a07f, offset: 0, size: 140 2024.10.22 00:39:14.964430 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 560477d5cf448a0bb1d5d66a4c74a07f for removal 2024.10.22 00:39:14.966851 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 53a39c7ead794d53bcc5a577eee50771, offset: 0, size: 144 2024.10.22 00:39:14.967520 [ 1476 ] {} CacheMetadata: Key 560477d5cf448a0bb1d5d66a4c74a07f is removed from metadata 2024.10.22 00:39:14.971291 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 53a39c7ead794d53bcc5a577eee50771 for removal 2024.10.22 00:39:14.973631 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7f35485e98f61f851b059ff70ff5a6a2, offset: 0, size: 4 2024.10.22 00:39:14.978413 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7f35485e98f61f851b059ff70ff5a6a2 for removal 2024.10.22 00:39:15.073158 [ 1476 ] {} CacheMetadata: Key 7f35485e98f61f851b059ff70ff5a6a2 is removed from metadata 2024.10.22 00:39:15.076257 [ 1476 ] {} CacheMetadata: Key 53a39c7ead794d53bcc5a577eee50771 is removed from metadata 2024.10.22 00:39:15.561298 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:15.565611 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.11 GiB. 2024.10.22 00:39:15.624280 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 3.99 MiB, will set to 1.71 GiB (RSS), difference: -4.42 MiB 2024.10.22 00:39:15.671227 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_139_139_0 (state Temporary) 2024.10.22 00:39:15.860743 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_139_139_0 to 202410_146_146_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:15.881816 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 242462 2024.10.22 00:39:15.910199 [ 1254 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_141_28 to 202410_146_146_0 2024.10.22 00:39:15.932948 [ 1254 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:15.941822 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.11 GiB. 2024.10.22 00:39:15.973661 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_141_28 to 202410_146_146_0 into Compact with storage Full 2024.10.22 00:39:15.989501 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:39:15.991943 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} MergeTreeSequentialSource: Reading 32 marks from part 202410_1_141_28, total 247156 rows starting from the beginning of the part 2024.10.22 00:39:15.998369 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_142_142_0, total 1600 rows starting from the beginning of the part 2024.10.22 00:39:16.008833 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_143_143_0, total 1555 rows starting from the beginning of the part 2024.10.22 00:39:16.031276 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_144_144_0, total 1901 rows starting from the beginning of the part 2024.10.22 00:39:16.052841 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_145_145_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:39:16.099764 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 48 entries to flush up to offset 9459 2024.10.22 00:39:16.174570 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_146_146_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:39:16.274298 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:16.275936 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.10 GiB. 2024.10.22 00:39:16.300484 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_144_144_0 (state Temporary) 2024.10.22 00:39:16.673191 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_144_144_0 to 202410_150_150_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:16.756711 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9459 2024.10.22 00:39:16.768843 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 13195 2024.10.22 00:39:16.809939 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_145_29 to 202410_150_150_0 2024.10.22 00:39:16.829555 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:16.836710 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.10 GiB. 2024.10.22 00:39:16.866932 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_145_29 to 202410_150_150_0 into Compact with storage Full 2024.10.22 00:39:16.998142 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:39:17.137170 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_145_29, total 9304 rows starting from the beginning of the part 2024.10.22 00:39:17.375830 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_146_146_0, total 87 rows starting from the beginning of the part 2024.10.22 00:39:17.422462 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:17.432814 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.10 GiB. 2024.10.22 00:39:17.538462 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_250_250_0 (state Temporary) 2024.10.22 00:39:17.582238 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_147_147_0, total 53 rows starting from the beginning of the part 2024.10.22 00:39:17.688930 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_250_250_0 to 202410_263_263_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:17.729176 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_148_148_0, total 44 rows starting from the beginning of the part 2024.10.22 00:39:17.805161 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 13195 2024.10.22 00:39:17.812517 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 5.82 MiB, will set to 1.72 GiB (RSS), difference: 65.42 MiB 2024.10.22 00:39:17.879966 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_149_149_0, total 27 rows starting from the beginning of the part 2024.10.22 00:39:17.988560 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_150_150_0, total 48 rows starting from the beginning of the part 2024.10.22 00:39:18.431507 [ 1726 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gjr/rlolkbcgrgvtrxaawjpqpajysjeof, size 20725 2024.10.22 00:39:19.487832 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_150_30/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:39:19.517614 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:39:19.535178 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:39:19.629279 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.88 GiB, free memory in arenas 7.48 MiB, will set to 1.71 GiB (RSS), difference: 116.52 MiB 2024.10.22 00:39:20.267124 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_146_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:39:20.662979 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} MergeTask::MergeProjectionsStage: Merge sorted 9563 rows, containing 13 columns (13 merged, 0 gathered) in 3.812204892 sec., 2508.522041946952 rows/sec., 1.05 MiB/sec. 2024.10.22 00:39:20.764726 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} MergedBlockOutputStream: filled checksums 202410_1_150_30 (state Temporary) 2024.10.22 00:39:21.167327 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_29_29_0 to 202410_32_32_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:21.312883 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_150_30 to 202410_1_150_30 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:21.319430 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_150_30} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_145_29, 202410_150_150_0] -> 202410_1_150_30 2024.10.22 00:39:21.360777 [ 1162 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:39:21.375755 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.44 MiB. 2024.10.22 00:39:21.397788 [ 1162 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:39:21.434406 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 135 2024.10.22 00:39:21.569969 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_11_11_0 (state Temporary) 2024.10.22 00:39:21.594717 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 2055 2024.10.22 00:39:21.596573 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gjd/bddedolrepuaeczszbmtkkhqmjwjw (LogSeriesLimiter: on interval from 2024-10-22 00:39:07 to 2024-10-22 00:39:21 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:39:21.658563 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.88 GiB, free memory in arenas 9.13 MiB, will set to 1.69 GiB (RSS), difference: 183.71 MiB 2024.10.22 00:39:21.660682 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:21.665756 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.02 GiB. 2024.10.22 00:39:21.670334 [ 1687 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gjd/bddedolrepuaeczszbmtkkhqmjwjw, size 4 2024.10.22 00:39:21.767755 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:21.772787 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.02 GiB. 2024.10.22 00:39:21.820080 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_146_146_0 (state Temporary) 2024.10.22 00:39:21.930423 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 13237 2024.10.22 00:39:21.949653 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_117_117_0 (state Temporary) 2024.10.22 00:39:22.040603 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_146_146_0 to 202410_159_159_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:22.096764 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2055 2024.10.22 00:39:22.121321 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_117_117_0 to 202410_127_127_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:22.286754 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:22.294589 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.01 GiB. 2024.10.22 00:39:22.225745 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 135 2024.10.22 00:39:22.434796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/bgr/onekilfvpngmqlhsnyflsnagkybfy. 2024.10.22 00:39:22.449306 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_251_251_0 (state Temporary) 2024.10.22 00:39:22.544873 [ 1421 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Found 5 old parts to remove. Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0, 202410_5_5_0] 2024.10.22 00:39:22.546316 [ 1421 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from filesystem (serially): Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0, 202410_5_5_0] 2024.10.22 00:39:22.665519 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_251_251_0 to 202410_264_264_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:22.672148 [ 1421 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from memory: Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0, 202410_5_5_0] 2024.10.22 00:39:22.700840 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 13237 2024.10.22 00:39:22.888256 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1538 entries to flush up to offset 244000 2024.10.22 00:39:23.571039 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 12.23 MiB, will set to 1.66 GiB (RSS), difference: -9.63 MiB 2024.10.22 00:39:23.907467 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:23.915178 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.01 GiB. 2024.10.22 00:39:24.011450 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_140_140_0 (state Temporary) 2024.10.22 00:39:24.163333 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_140_140_0 to 202410_147_147_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:24.226544 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 244000 2024.10.22 00:39:24.270523 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 111 entries to flush up to offset 9570 2024.10.22 00:39:24.340311 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1151 2024.10.22 00:39:24.413734 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:24.419704 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.01 GiB. 2024.10.22 00:39:24.476229 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_145_145_0 (state Temporary) 2024.10.22 00:39:24.648739 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_145_145_0 to 202410_151_151_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:24.694631 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9570 2024.10.22 00:39:25.413337 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 19.76 MiB, will set to 1.66 GiB (RSS), difference: 8.45 MiB 2024.10.22 00:39:25.743604 [ 1157 ] {} DatabaseCatalog: Execution took 200 ms. 2024.10.22 00:39:26.479298 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:39:26.516883 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 19.82 MiB, will set to 1.66 GiB (RSS), difference: -1.08 MiB 2024.10.22 00:39:26.709428 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 13271 2024.10.22 00:39:27.093145 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} MergeTask::MergeProjectionsStage: Merge sorted 256052 rows, containing 4 columns (4 merged, 0 gathered) in 11.128253749 sec., 23009.180575434773 rows/sec., 337.83 KiB/sec. 2024.10.22 00:39:27.283104 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:27.285597 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.01 GiB. 2024.10.22 00:39:27.345451 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_252_252_0 (state Temporary) 2024.10.22 00:39:27.390810 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_74_15, 202410_75_75_0, 202410_76_76_0, 202410_77_77_0, 202410_78_78_0] 2024.10.22 00:39:27.395517 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_74_15, 202410_75_75_0, 202410_76_76_0, 202410_77_77_0, 202410_78_78_0] 2024.10.22 00:39:27.430113 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_74_15, 202410_75_75_0, 202410_76_76_0, 202410_77_77_0, 202410_78_78_0] 2024.10.22 00:39:27.480842 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} MergedBlockOutputStream: filled checksums 202410_1_146_29 (state Temporary) 2024.10.22 00:39:27.484307 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_252_252_0 to 202410_265_265_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:27.517585 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_146_29 to 202410_1_146_29 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:27.536553 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_146_29} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_141_28, 202410_146_146_0] -> 202410_1_146_29 2024.10.22 00:39:27.547285 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 13271 2024.10.22 00:39:27.587844 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:39:28.601365 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 19.25 MiB, will set to 1.66 GiB (RSS), difference: 10.41 MiB 2024.10.22 00:39:29.522827 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:39:29.527192 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:39:29.709313 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 2068 2024.10.22 00:39:29.828556 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 136 2024.10.22 00:39:29.972332 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:29.974849 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.01 GiB. 2024.10.22 00:39:30.008928 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_147_147_0 (state Temporary) 2024.10.22 00:39:30.063713 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_147_147_0 to 202410_160_160_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:30.095882 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2068 2024.10.22 00:39:30.234117 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:30.240939 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.01 GiB. 2024.10.22 00:39:30.610533 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 19.23 MiB, will set to 1.67 GiB (RSS), difference: 2.49 MiB 2024.10.22 00:39:30.623336 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_118_118_0 (state Temporary) 2024.10.22 00:39:30.821063 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_118_118_0 to 202410_128_128_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:30.890627 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 136 2024.10.22 00:39:30.968197 [ 1219 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_123_25 to 202410_128_128_0 2024.10.22 00:39:30.987418 [ 1219 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:30.988715 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.01 GiB. 2024.10.22 00:39:31.144682 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_123_25 to 202410_128_128_0 into Compact with storage Full 2024.10.22 00:39:31.235046 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1788 entries to flush up to offset 245788 2024.10.22 00:39:31.566208 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:39:31.573057 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 13306 2024.10.22 00:39:31.582771 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_123_25, total 167 rows starting from the beginning of the part 2024.10.22 00:39:31.635293 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_124_124_0, total 1 rows starting from the beginning of the part 2024.10.22 00:39:31.700425 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_125_125_0, total 1 rows starting from the beginning of the part 2024.10.22 00:39:31.784176 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_126_126_0, total 1 rows starting from the beginning of the part 2024.10.22 00:39:31.853313 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_127_127_0, total 1 rows starting from the beginning of the part 2024.10.22 00:39:31.886820 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 1190 2024.10.22 00:39:32.005051 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_128_128_0, total 1 rows starting from the beginning of the part 2024.10.22 00:39:32.029218 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:32.035310 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.00 GiB. 2024.10.22 00:39:32.095607 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_253_253_0 (state Temporary) 2024.10.22 00:39:32.246191 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 36 entries to flush up to offset 9606 2024.10.22 00:39:32.266284 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:32.269554 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.00 GiB. 2024.10.22 00:39:32.375185 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_253_253_0 to 202410_266_266_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:32.388719 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:32.392237 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.00 GiB. 2024.10.22 00:39:32.445556 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_261_52 to 202410_266_266_0 2024.10.22 00:39:32.460071 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:32.468868 [ 1184 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.00 GiB. 2024.10.22 00:39:32.519460 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_261_52 to 202410_266_266_0 into Compact with storage Full 2024.10.22 00:39:32.530884 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_141_141_0 (state Temporary) 2024.10.22 00:39:32.576169 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_146_146_0 (state Temporary) 2024.10.22 00:39:32.593360 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 17.25 MiB, will set to 1.67 GiB (RSS), difference: -21.05 MiB 2024.10.22 00:39:32.619901 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_146_146_0 to 202410_152_152_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:32.627940 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:39:32.656772 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 13306 2024.10.22 00:39:32.662986 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_261_52, total 13902 rows starting from the beginning of the part 2024.10.22 00:39:32.696800 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_141_141_0 to 202410_148_148_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:32.771300 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 245788 2024.10.22 00:39:32.785395 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_262_262_0, total 48 rows starting from the beginning of the part 2024.10.22 00:39:32.882657 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9606 2024.10.22 00:39:32.952351 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_263_263_0, total 76 rows starting from the beginning of the part 2024.10.22 00:39:33.015257 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_264_264_0, total 42 rows starting from the beginning of the part 2024.10.22 00:39:33.078342 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_265_265_0, total 34 rows starting from the beginning of the part 2024.10.22 00:39:33.124369 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_266_266_0, total 35 rows starting from the beginning of the part 2024.10.22 00:39:33.435044 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_161_32, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0] 2024.10.22 00:39:33.439887 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_161_32, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0] 2024.10.22 00:39:33.483211 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_161_32, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0] 2024.10.22 00:39:34.219516 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} MergeTask::MergeProjectionsStage: Merge sorted 172 rows, containing 32 columns (32 merged, 0 gathered) in 3.096371101 sec., 55.54889720565184 rows/sec., 6.12 KiB/sec. 2024.10.22 00:39:34.371177 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} MergedBlockOutputStream: filled checksums 202410_1_128_26 (state Temporary) 2024.10.22 00:39:34.843883 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 11.86 MiB, will set to 1.68 GiB (RSS), difference: 5.67 MiB 2024.10.22 00:39:34.858928 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_128_26 to 202410_1_128_26 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:34.862778 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_128_26} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_123_25, 202410_128_128_0] -> 202410_1_128_26 2024.10.22 00:39:34.889657 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:39:34.904198 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} MergeTask::MergeProjectionsStage: Merge sorted 14137 rows, containing 13 columns (13 merged, 0 gathered) in 2.429252636 sec., 5819.48529786426 rows/sec., 885.81 KiB/sec. 2024.10.22 00:39:35.167292 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_86_17, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0] 2024.10.22 00:39:35.170407 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_86_17, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0] 2024.10.22 00:39:35.195221 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_86_17, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0] 2024.10.22 00:39:35.286875 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} MergedBlockOutputStream: filled checksums 202410_1_266_53 (state Temporary) 2024.10.22 00:39:35.629456 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_266_53 to 202410_1_266_53 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:35.635753 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_266_53} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_261_52, 202410_266_266_0] -> 202410_1_266_53 2024.10.22 00:39:35.672329 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.77 MiB. 2024.10.22 00:39:36.277651 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_96_19, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0] 2024.10.22 00:39:36.279222 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_96_19, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0] 2024.10.22 00:39:36.319830 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_96_19, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0] 2024.10.22 00:39:36.418115 [ 1270 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:39:36.477519 [ 1270 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:39:36.652289 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 9.54 MiB, will set to 1.67 GiB (RSS), difference: 8.92 MiB 2024.10.22 00:39:36.739053 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 13366 2024.10.22 00:39:37.293596 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:37.302497 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.00 GiB. 2024.10.22 00:39:37.379548 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_254_254_0 (state Temporary) 2024.10.22 00:39:37.434223 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_254_254_0 to 202410_267_267_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:37.599487 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 2088 2024.10.22 00:39:37.788145 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:37.791025 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.00 GiB. 2024.10.22 00:39:37.827520 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_148_148_0 (state Temporary) 2024.10.22 00:39:37.828431 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 13366 2024.10.22 00:39:38.217299 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_148_148_0 to 202410_161_161_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:38.299278 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 6.21 MiB, will set to 1.68 GiB (RSS), difference: 4.14 MiB 2024.10.22 00:39:38.360674 [ 1269 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_156_31 to 202410_161_161_0 2024.10.22 00:39:38.371610 [ 1269 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:38.372454 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2088 2024.10.22 00:39:38.373172 [ 1269 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.00 GiB. 2024.10.22 00:39:38.384558 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_156_31 to 202410_161_161_0 into Compact with storage Full 2024.10.22 00:39:38.458110 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:39:38.464279 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_156_31, total 2116 rows starting from the beginning of the part 2024.10.22 00:39:38.486471 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_157_157_0, total 20 rows starting from the beginning of the part 2024.10.22 00:39:38.518652 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_158_158_0, total 13 rows starting from the beginning of the part 2024.10.22 00:39:38.555781 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_159_159_0, total 12 rows starting from the beginning of the part 2024.10.22 00:39:38.575954 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_160_160_0, total 13 rows starting from the beginning of the part 2024.10.22 00:39:38.594080 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_161_161_0, total 20 rows starting from the beginning of the part 2024.10.22 00:39:39.022402 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 138 file segments (while reserving 16777216 bytes for 7a2eb8cea9c17e12aec9d150ec4c29b7:33554432) 2024.10.22 00:39:39.024625 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dd15ba7eb48779f0905d69195f1e95c5, offset: 0, size: 2303438 2024.10.22 00:39:39.029981 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dd15ba7eb48779f0905d69195f1e95c5 for removal 2024.10.22 00:39:39.034525 [ 1476 ] {} CacheMetadata: Key dd15ba7eb48779f0905d69195f1e95c5 is removed from metadata 2024.10.22 00:39:39.035254 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 38bc4b2d53099a18b9bff04716b994bf, offset: 0, size: 241 2024.10.22 00:39:39.038415 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 38bc4b2d53099a18b9bff04716b994bf for removal 2024.10.22 00:39:39.040823 [ 1476 ] {} CacheMetadata: Key 38bc4b2d53099a18b9bff04716b994bf is removed from metadata 2024.10.22 00:39:39.041269 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 825177d71c2b40c6a1e321543bdbab73, offset: 0, size: 4560 2024.10.22 00:39:39.044546 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 825177d71c2b40c6a1e321543bdbab73 for removal 2024.10.22 00:39:39.048454 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f1217972affb92f68b5817526455f2d7, offset: 0, size: 864677 2024.10.22 00:39:39.048938 [ 1476 ] {} CacheMetadata: Key 825177d71c2b40c6a1e321543bdbab73 is removed from metadata 2024.10.22 00:39:39.053174 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f1217972affb92f68b5817526455f2d7 for removal 2024.10.22 00:39:39.061472 [ 1476 ] {} CacheMetadata: Key f1217972affb92f68b5817526455f2d7 is removed from metadata 2024.10.22 00:39:39.063679 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2c52f814ab3ffd5adfa061df3372a153, offset: 0, size: 158 2024.10.22 00:39:39.066856 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2c52f814ab3ffd5adfa061df3372a153 for removal 2024.10.22 00:39:39.069405 [ 1476 ] {} CacheMetadata: Key 2c52f814ab3ffd5adfa061df3372a153 is removed from metadata 2024.10.22 00:39:39.171483 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: df7ecaeae98ddf0de0d51d66e87604ff, offset: 0, size: 181 2024.10.22 00:39:39.172590 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} MergeTask::MergeProjectionsStage: Merge sorted 2194 rows, containing 12 columns (12 merged, 0 gathered) in 0.793979964 sec., 2763.293911028717 rows/sec., 428.56 KiB/sec. 2024.10.22 00:39:39.173535 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key df7ecaeae98ddf0de0d51d66e87604ff for removal 2024.10.22 00:39:39.175787 [ 1476 ] {} CacheMetadata: Key df7ecaeae98ddf0de0d51d66e87604ff is removed from metadata 2024.10.22 00:39:39.176152 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: facf76177d4075c30debdb854f4e40d2, offset: 0, size: 1268 2024.10.22 00:39:39.178386 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key facf76177d4075c30debdb854f4e40d2 for removal 2024.10.22 00:39:39.183945 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 48f994682755da81cbb7dc0dc9cd0e61, offset: 0, size: 154 2024.10.22 00:39:39.186807 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 48f994682755da81cbb7dc0dc9cd0e61 for removal 2024.10.22 00:39:39.191271 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} MergedBlockOutputStream: filled checksums 202410_1_161_32 (state Temporary) 2024.10.22 00:39:39.191552 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 042afbb992a6c51e4ca33b1f85c92f95, offset: 0, size: 142 2024.10.22 00:39:39.193657 [ 1476 ] {} CacheMetadata: Key facf76177d4075c30debdb854f4e40d2 is removed from metadata 2024.10.22 00:39:39.195274 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 042afbb992a6c51e4ca33b1f85c92f95 for removal 2024.10.22 00:39:39.200157 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 63d9c1f60708f4d403e7a342b9624671, offset: 0, size: 142 2024.10.22 00:39:39.206783 [ 1476 ] {} CacheMetadata: Key 042afbb992a6c51e4ca33b1f85c92f95 is removed from metadata 2024.10.22 00:39:39.211720 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 63d9c1f60708f4d403e7a342b9624671 for removal 2024.10.22 00:39:39.218546 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6d40179a8b5fe49ef24fc8ffddea30e0, offset: 0, size: 51766 2024.10.22 00:39:39.223115 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6d40179a8b5fe49ef24fc8ffddea30e0 for removal 2024.10.22 00:39:39.224455 [ 1476 ] {} CacheMetadata: Key 63d9c1f60708f4d403e7a342b9624671 is removed from metadata 2024.10.22 00:39:39.227658 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fa666c0d679f8b0dd24c4cfce9330498, offset: 0, size: 141 2024.10.22 00:39:39.232874 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fa666c0d679f8b0dd24c4cfce9330498 for removal 2024.10.22 00:39:39.245058 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e65e0eaedee339fbed98b496110aceac, offset: 0, size: 142 2024.10.22 00:39:39.248407 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e65e0eaedee339fbed98b496110aceac for removal 2024.10.22 00:39:39.251891 [ 1476 ] {} CacheMetadata: Key fa666c0d679f8b0dd24c4cfce9330498 is removed from metadata 2024.10.22 00:39:39.252823 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bfe9e45735a952f6a2b73f5eb25d318e, offset: 0, size: 47567 2024.10.22 00:39:39.258378 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bfe9e45735a952f6a2b73f5eb25d318e for removal 2024.10.22 00:39:39.267689 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 470afb06e85073bc5b35270a846ff7b1, offset: 0, size: 142 2024.10.22 00:39:39.271625 [ 1476 ] {} CacheMetadata: Key bfe9e45735a952f6a2b73f5eb25d318e is removed from metadata 2024.10.22 00:39:39.272229 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 470afb06e85073bc5b35270a846ff7b1 for removal 2024.10.22 00:39:39.275585 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: eb794725d28a949a40bf7ca9d7d667a4, offset: 0, size: 142 2024.10.22 00:39:39.277320 [ 1476 ] {} CacheMetadata: Key 470afb06e85073bc5b35270a846ff7b1 is removed from metadata 2024.10.22 00:39:39.279648 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key eb794725d28a949a40bf7ca9d7d667a4 for removal 2024.10.22 00:39:39.283762 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 54a81cec15a65d649dbfd7f8635c2c29, offset: 0, size: 142 2024.10.22 00:39:39.290146 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 54a81cec15a65d649dbfd7f8635c2c29 for removal 2024.10.22 00:39:39.298891 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9705cb40f00823cf9d949832d4edc061, offset: 0, size: 51317 2024.10.22 00:39:39.311650 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9705cb40f00823cf9d949832d4edc061 for removal 2024.10.22 00:39:39.319069 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8ee6428913ee919e0539ae6bf81b19bf, offset: 0, size: 142 2024.10.22 00:39:39.323349 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8ee6428913ee919e0539ae6bf81b19bf for removal 2024.10.22 00:39:39.341790 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 51f31be9ea2d20b816b65981cc688522, offset: 0, size: 180 2024.10.22 00:39:39.351572 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 51f31be9ea2d20b816b65981cc688522 for removal 2024.10.22 00:39:39.358660 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f0085f219fb234be71106ab799c8d860, offset: 0, size: 50030 2024.10.22 00:39:39.363442 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f0085f219fb234be71106ab799c8d860 for removal 2024.10.22 00:39:39.367959 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ef2448a3928a41daa17d3ca38760595f, offset: 0, size: 212 2024.10.22 00:39:39.370567 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ef2448a3928a41daa17d3ca38760595f for removal 2024.10.22 00:39:39.374506 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 58ab01a7e307a72d11ccf774f6ee235d, offset: 0, size: 142 2024.10.22 00:39:39.378008 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 58ab01a7e307a72d11ccf774f6ee235d for removal 2024.10.22 00:39:39.382011 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 32b7bceed9f67092d92024e6fa1cb480, offset: 0, size: 432915 2024.10.22 00:39:39.386451 [ 1476 ] {} CacheMetadata: Key 58ab01a7e307a72d11ccf774f6ee235d is removed from metadata 2024.10.22 00:39:39.384956 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 32b7bceed9f67092d92024e6fa1cb480 for removal 2024.10.22 00:39:39.402256 [ 1476 ] {} CacheMetadata: Key f0085f219fb234be71106ab799c8d860 is removed from metadata 2024.10.22 00:39:39.415010 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6fb5bc727c49b3d92d35fe17f472b6a3, offset: 0, size: 142 2024.10.22 00:39:39.419518 [ 1476 ] {} CacheMetadata: Key 32b7bceed9f67092d92024e6fa1cb480 is removed from metadata 2024.10.22 00:39:39.428311 [ 1476 ] {} CacheMetadata: Key 51f31be9ea2d20b816b65981cc688522 is removed from metadata 2024.10.22 00:39:39.429162 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6fb5bc727c49b3d92d35fe17f472b6a3 for removal 2024.10.22 00:39:39.433312 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 552ca2f776075edad658070524ae19c3, offset: 0, size: 45363 2024.10.22 00:39:39.439824 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 552ca2f776075edad658070524ae19c3 for removal 2024.10.22 00:39:39.446920 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 49cef2ee5ccfe31050451abe96709c60, offset: 0, size: 142 2024.10.22 00:39:39.450341 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 49cef2ee5ccfe31050451abe96709c60 for removal 2024.10.22 00:39:39.460078 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 11da9fb272b9dbdbb7e693aaf1714975, offset: 0, size: 164 2024.10.22 00:39:39.468162 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 11da9fb272b9dbdbb7e693aaf1714975 for removal 2024.10.22 00:39:39.471166 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 64ed2c1e1047a3da543177aeacd3db56, offset: 0, size: 154 2024.10.22 00:39:39.474736 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 64ed2c1e1047a3da543177aeacd3db56 for removal 2024.10.22 00:39:39.477590 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 677ed4e48bc55a00637be9637967dd93, offset: 0, size: 166 2024.10.22 00:39:39.480203 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 677ed4e48bc55a00637be9637967dd93 for removal 2024.10.22 00:39:39.484027 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ffe80b947773c0c2cd6265c01530ce74, offset: 0, size: 154 2024.10.22 00:39:39.487537 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ffe80b947773c0c2cd6265c01530ce74 for removal 2024.10.22 00:39:39.492472 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 57df92c36162da7619aa037fe8adfbc1, offset: 0, size: 49261 2024.10.22 00:39:39.497427 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 57df92c36162da7619aa037fe8adfbc1 for removal 2024.10.22 00:39:39.515186 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b3afeeca8e54198f93ea96f47e74701b, offset: 0, size: 489594 2024.10.22 00:39:39.518600 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:39:39.520403 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b3afeeca8e54198f93ea96f47e74701b for removal 2024.10.22 00:39:39.521333 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:39:39.525645 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ebcf2cac03c73cfd05697ceb89329bb2, offset: 0, size: 155 2024.10.22 00:39:39.532012 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ebcf2cac03c73cfd05697ceb89329bb2 for removal 2024.10.22 00:39:39.546257 [ 1476 ] {} CacheMetadata: Key ebcf2cac03c73cfd05697ceb89329bb2 is removed from metadata 2024.10.22 00:39:39.546775 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b1c6652cc005ae5c92c800ef19f7f4c3, offset: 0, size: 127136 2024.10.22 00:39:39.552691 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b1c6652cc005ae5c92c800ef19f7f4c3 for removal 2024.10.22 00:39:39.559829 [ 1476 ] {} CacheMetadata: Key b3afeeca8e54198f93ea96f47e74701b is removed from metadata 2024.10.22 00:39:39.566077 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1256f4c0a89baf35eb2560b334703e1c, offset: 0, size: 144 2024.10.22 00:39:39.570445 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1256f4c0a89baf35eb2560b334703e1c for removal 2024.10.22 00:39:39.572444 [ 1476 ] {} CacheMetadata: Key 57df92c36162da7619aa037fe8adfbc1 is removed from metadata 2024.10.22 00:39:39.573487 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fc9937599aec50cbe9b429578323516a, offset: 0, size: 47615 2024.10.22 00:39:39.590706 [ 1476 ] {} CacheMetadata: Key 1256f4c0a89baf35eb2560b334703e1c is removed from metadata 2024.10.22 00:39:39.591932 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fc9937599aec50cbe9b429578323516a for removal 2024.10.22 00:39:39.602992 [ 1476 ] {} CacheMetadata: Key fc9937599aec50cbe9b429578323516a is removed from metadata 2024.10.22 00:39:39.611280 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cb1d3953dc9374a5555b263db2ebfea1, offset: 0, size: 155 2024.10.22 00:39:39.615612 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cb1d3953dc9374a5555b263db2ebfea1 for removal 2024.10.22 00:39:39.618973 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d08fd6abc17b2fbcb982998bec419e34, offset: 0, size: 44212 2024.10.22 00:39:39.620810 [ 1476 ] {} CacheMetadata: Key ffe80b947773c0c2cd6265c01530ce74 is removed from metadata 2024.10.22 00:39:39.624739 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d08fd6abc17b2fbcb982998bec419e34 for removal 2024.10.22 00:39:39.635195 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 68722d1060e4f17467a6cbf34a627a3e, offset: 0, size: 174 2024.10.22 00:39:39.638287 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 68722d1060e4f17467a6cbf34a627a3e for removal 2024.10.22 00:39:39.645637 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cf336f1d643238a8b67de15c686063a6, offset: 0, size: 27464 2024.10.22 00:39:39.653063 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cf336f1d643238a8b67de15c686063a6 for removal 2024.10.22 00:39:39.660003 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 34b5d220ae9f4b6b8dddc079db505f18, offset: 0, size: 32624 2024.10.22 00:39:39.668135 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 34b5d220ae9f4b6b8dddc079db505f18 for removal 2024.10.22 00:39:39.670838 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5db0466f84a41d08bc7563cc703adbcb, offset: 0, size: 14222 2024.10.22 00:39:39.672872 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5db0466f84a41d08bc7563cc703adbcb for removal 2024.10.22 00:39:39.677246 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 578107d5db43a6b2a8693261aef5f66e, offset: 0, size: 164 2024.10.22 00:39:39.681423 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 578107d5db43a6b2a8693261aef5f66e for removal 2024.10.22 00:39:39.685061 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1b7c68dbd1d5785b12036ee4061febef, offset: 0, size: 142 2024.10.22 00:39:39.688563 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1b7c68dbd1d5785b12036ee4061febef for removal 2024.10.22 00:39:39.692783 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7b835f0eb473b7653b7dd8abffdf7b8d, offset: 0, size: 154 2024.10.22 00:39:39.697324 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7b835f0eb473b7653b7dd8abffdf7b8d for removal 2024.10.22 00:39:39.704979 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 873f460335d84a18a36d2144ba408b69, offset: 0, size: 154 2024.10.22 00:39:39.709370 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 873f460335d84a18a36d2144ba408b69 for removal 2024.10.22 00:39:39.714807 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a6873fedc3b0e84bd3944194cd97a412, offset: 0, size: 164 2024.10.22 00:39:39.719278 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a6873fedc3b0e84bd3944194cd97a412 for removal 2024.10.22 00:39:39.724251 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 19eb2b3b3d9c061566c7a3ee0bbb2670, offset: 0, size: 2500 2024.10.22 00:39:39.727953 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 19eb2b3b3d9c061566c7a3ee0bbb2670 for removal 2024.10.22 00:39:39.731075 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 54efd73cbc3c2e3cafa9408cfe8c9b30, offset: 0, size: 1643378 2024.10.22 00:39:39.735828 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 54efd73cbc3c2e3cafa9408cfe8c9b30 for removal 2024.10.22 00:39:39.744479 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 40f024775fe7360f7200e880c878ece8, offset: 0, size: 142 2024.10.22 00:39:39.750987 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 40f024775fe7360f7200e880c878ece8 for removal 2024.10.22 00:39:39.761046 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1069ca3278bee12138d37f978dfd58a4, offset: 0, size: 559184 2024.10.22 00:39:39.767167 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1069ca3278bee12138d37f978dfd58a4 for removal 2024.10.22 00:39:39.775851 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1284 entries to flush up to offset 247072 2024.10.22 00:39:39.780410 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_161_32 to 202410_1_161_32 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:39.780497 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9af17c27e6164a49689d1aaba6b7d9de, offset: 0, size: 154 2024.10.22 00:39:39.783468 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_161_32} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_156_31, 202410_161_161_0] -> 202410_1_161_32 2024.10.22 00:39:39.783771 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9af17c27e6164a49689d1aaba6b7d9de for removal 2024.10.22 00:39:39.787635 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 52c4ecb330a94a74800135068bb55ef1, offset: 0, size: 142 2024.10.22 00:39:39.832850 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:39:39.875222 [ 1476 ] {} CacheMetadata: Key 9af17c27e6164a49689d1aaba6b7d9de is removed from metadata 2024.10.22 00:39:39.881351 [ 1476 ] {} CacheMetadata: Key 54efd73cbc3c2e3cafa9408cfe8c9b30 is removed from metadata 2024.10.22 00:39:39.885252 [ 1476 ] {} CacheMetadata: Key 19eb2b3b3d9c061566c7a3ee0bbb2670 is removed from metadata 2024.10.22 00:39:39.889515 [ 1476 ] {} CacheMetadata: Key a6873fedc3b0e84bd3944194cd97a412 is removed from metadata 2024.10.22 00:39:39.890940 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 52c4ecb330a94a74800135068bb55ef1 for removal 2024.10.22 00:39:39.895067 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dd1624befbef0ec5fa8f7bc9bf19775d, offset: 0, size: 9036 2024.10.22 00:39:39.897411 [ 1476 ] {} CacheMetadata: Key 52c4ecb330a94a74800135068bb55ef1 is removed from metadata 2024.10.22 00:39:39.900760 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dd1624befbef0ec5fa8f7bc9bf19775d for removal 2024.10.22 00:39:39.906645 [ 1476 ] {} CacheMetadata: Key dd1624befbef0ec5fa8f7bc9bf19775d is removed from metadata 2024.10.22 00:39:39.905830 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 44107a7c8cf8f44627b48637d077dbab, offset: 0, size: 27358 2024.10.22 00:39:39.912973 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 44107a7c8cf8f44627b48637d077dbab for removal 2024.10.22 00:39:39.916527 [ 1476 ] {} CacheMetadata: Key 873f460335d84a18a36d2144ba408b69 is removed from metadata 2024.10.22 00:39:39.919719 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 090c1a2e051bbb11f1d67c39f7cd76d0, offset: 0, size: 14524 2024.10.22 00:39:39.922150 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 090c1a2e051bbb11f1d67c39f7cd76d0 for removal 2024.10.22 00:39:39.924815 [ 1476 ] {} CacheMetadata: Key 1b7c68dbd1d5785b12036ee4061febef is removed from metadata 2024.10.22 00:39:39.927313 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1993591cbf9df4ae3335747e82ab2dce, offset: 0, size: 192130 2024.10.22 00:39:39.932822 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1993591cbf9df4ae3335747e82ab2dce for removal 2024.10.22 00:39:39.942368 [ 1476 ] {} CacheMetadata: Key 578107d5db43a6b2a8693261aef5f66e is removed from metadata 2024.10.22 00:39:39.944996 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 647a05c3a495a9cacf0091b0abd82846, offset: 0, size: 142 2024.10.22 00:39:39.950292 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 647a05c3a495a9cacf0091b0abd82846 for removal 2024.10.22 00:39:39.955900 [ 1476 ] {} CacheMetadata: Key 1993591cbf9df4ae3335747e82ab2dce is removed from metadata 2024.10.22 00:39:39.961265 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c3c050ebfe8ee2b057f3972f003eac45, offset: 0, size: 142 2024.10.22 00:39:39.965626 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c3c050ebfe8ee2b057f3972f003eac45 for removal 2024.10.22 00:39:39.970816 [ 1476 ] {} CacheMetadata: Key 40f024775fe7360f7200e880c878ece8 is removed from metadata 2024.10.22 00:39:39.971783 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a4ab880f27911b67ef5027a1b2ad23eb, offset: 0, size: 5890 2024.10.22 00:39:39.977736 [ 1476 ] {} CacheMetadata: Key c3c050ebfe8ee2b057f3972f003eac45 is removed from metadata 2024.10.22 00:39:39.977739 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a4ab880f27911b67ef5027a1b2ad23eb for removal 2024.10.22 00:39:39.980561 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0a9022b0b1f6552ef3ae4f0b3f032de4, offset: 0, size: 4964 2024.10.22 00:39:39.983930 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0a9022b0b1f6552ef3ae4f0b3f032de4 for removal 2024.10.22 00:39:39.984264 [ 1476 ] {} CacheMetadata: Key a4ab880f27911b67ef5027a1b2ad23eb is removed from metadata 2024.10.22 00:39:39.986412 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e04b2125d0fd13f9f33e31b0e7b0ce58, offset: 0, size: 52584 2024.10.22 00:39:39.988177 [ 1476 ] {} CacheMetadata: Key 0a9022b0b1f6552ef3ae4f0b3f032de4 is removed from metadata 2024.10.22 00:39:39.988397 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e04b2125d0fd13f9f33e31b0e7b0ce58 for removal 2024.10.22 00:39:39.993620 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3a3f76295ca3ecfd1fff228d299c33ef, offset: 0, size: 189675 2024.10.22 00:39:39.994256 [ 1476 ] {} CacheMetadata: Key e04b2125d0fd13f9f33e31b0e7b0ce58 is removed from metadata 2024.10.22 00:39:39.995696 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3a3f76295ca3ecfd1fff228d299c33ef for removal 2024.10.22 00:39:40.002097 [ 1476 ] {} CacheMetadata: Key 3a3f76295ca3ecfd1fff228d299c33ef is removed from metadata 2024.10.22 00:39:40.006632 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8f7eaf98715a85c5af1bc2ca0b174637, offset: 0, size: 166 2024.10.22 00:39:40.011804 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8f7eaf98715a85c5af1bc2ca0b174637 for removal 2024.10.22 00:39:40.016537 [ 1476 ] {} CacheMetadata: Key 8f7eaf98715a85c5af1bc2ca0b174637 is removed from metadata 2024.10.22 00:39:40.017990 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f6a32994bb1b92b0e67d20d80a50a21f, offset: 0, size: 124274 2024.10.22 00:39:40.020844 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f6a32994bb1b92b0e67d20d80a50a21f for removal 2024.10.22 00:39:40.021739 [ 1476 ] {} CacheMetadata: Key 5db0466f84a41d08bc7563cc703adbcb is removed from metadata 2024.10.22 00:39:40.027102 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b4d37d2dd25774db3bcef3dc515bfb0e, offset: 0, size: 534861 2024.10.22 00:39:40.027692 [ 1476 ] {} CacheMetadata: Key cf336f1d643238a8b67de15c686063a6 is removed from metadata 2024.10.22 00:39:40.036260 [ 1476 ] {} CacheMetadata: Key 44107a7c8cf8f44627b48637d077dbab is removed from metadata 2024.10.22 00:39:40.046999 [ 1476 ] {} CacheMetadata: Key 68722d1060e4f17467a6cbf34a627a3e is removed from metadata 2024.10.22 00:39:40.056735 [ 1476 ] {} CacheMetadata: Key 1069ca3278bee12138d37f978dfd58a4 is removed from metadata 2024.10.22 00:39:40.064501 [ 1476 ] {} CacheMetadata: Key d08fd6abc17b2fbcb982998bec419e34 is removed from metadata 2024.10.22 00:39:40.071959 [ 1476 ] {} CacheMetadata: Key f6a32994bb1b92b0e67d20d80a50a21f is removed from metadata 2024.10.22 00:39:40.076482 [ 1476 ] {} CacheMetadata: Key 677ed4e48bc55a00637be9637967dd93 is removed from metadata 2024.10.22 00:39:40.083161 [ 1476 ] {} CacheMetadata: Key cb1d3953dc9374a5555b263db2ebfea1 is removed from metadata 2024.10.22 00:39:40.087199 [ 1476 ] {} CacheMetadata: Key 64ed2c1e1047a3da543177aeacd3db56 is removed from metadata 2024.10.22 00:39:40.094889 [ 1476 ] {} CacheMetadata: Key 11da9fb272b9dbdbb7e693aaf1714975 is removed from metadata 2024.10.22 00:39:40.113353 [ 1476 ] {} CacheMetadata: Key 49cef2ee5ccfe31050451abe96709c60 is removed from metadata 2024.10.22 00:39:40.122870 [ 1476 ] {} CacheMetadata: Key 647a05c3a495a9cacf0091b0abd82846 is removed from metadata 2024.10.22 00:39:40.132334 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b4d37d2dd25774db3bcef3dc515bfb0e for removal 2024.10.22 00:39:40.144280 [ 1476 ] {} CacheMetadata: Key b4d37d2dd25774db3bcef3dc515bfb0e is removed from metadata 2024.10.22 00:39:40.142753 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6419989b680dbc37929b0688d5ea03e7, offset: 0, size: 154 2024.10.22 00:39:40.153761 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6419989b680dbc37929b0688d5ea03e7 for removal 2024.10.22 00:39:40.162321 [ 1476 ] {} CacheMetadata: Key b1c6652cc005ae5c92c800ef19f7f4c3 is removed from metadata 2024.10.22 00:39:40.164224 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 586faf91fad9d703effa193c02875482, offset: 0, size: 142 2024.10.22 00:39:40.168245 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 586faf91fad9d703effa193c02875482 for removal 2024.10.22 00:39:40.169552 [ 1476 ] {} CacheMetadata: Key 6fb5bc727c49b3d92d35fe17f472b6a3 is removed from metadata 2024.10.22 00:39:40.174231 [ 1476 ] {} CacheMetadata: Key 090c1a2e051bbb11f1d67c39f7cd76d0 is removed from metadata 2024.10.22 00:39:40.178493 [ 1476 ] {} CacheMetadata: Key 8ee6428913ee919e0539ae6bf81b19bf is removed from metadata 2024.10.22 00:39:40.184233 [ 1476 ] {} CacheMetadata: Key 9705cb40f00823cf9d949832d4edc061 is removed from metadata 2024.10.22 00:39:40.192690 [ 1476 ] {} CacheMetadata: Key 34b5d220ae9f4b6b8dddc079db505f18 is removed from metadata 2024.10.22 00:39:40.196787 [ 1476 ] {} CacheMetadata: Key 54a81cec15a65d649dbfd7f8635c2c29 is removed from metadata 2024.10.22 00:39:40.209344 [ 1476 ] {} CacheMetadata: Key 552ca2f776075edad658070524ae19c3 is removed from metadata 2024.10.22 00:39:40.220699 [ 1476 ] {} CacheMetadata: Key eb794725d28a949a40bf7ca9d7d667a4 is removed from metadata 2024.10.22 00:39:40.226922 [ 1476 ] {} CacheMetadata: Key e65e0eaedee339fbed98b496110aceac is removed from metadata 2024.10.22 00:39:40.236276 [ 1476 ] {} CacheMetadata: Key ef2448a3928a41daa17d3ca38760595f is removed from metadata 2024.10.22 00:39:40.262912 [ 1476 ] {} CacheMetadata: Key 6d40179a8b5fe49ef24fc8ffddea30e0 is removed from metadata 2024.10.22 00:39:40.270486 [ 1476 ] {} CacheMetadata: Key 6419989b680dbc37929b0688d5ea03e7 is removed from metadata 2024.10.22 00:39:40.271952 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 53f64b62194c3fd5f15848948b559a00, offset: 0, size: 142 2024.10.22 00:39:40.278312 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 53f64b62194c3fd5f15848948b559a00 for removal 2024.10.22 00:39:40.280392 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 40ff83358dc41ab07a60c70f36c224a3, offset: 0, size: 154 2024.10.22 00:39:40.282798 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 40ff83358dc41ab07a60c70f36c224a3 for removal 2024.10.22 00:39:40.284850 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5ecebcc6992a5b96668edbe4e8b56325, offset: 0, size: 142 2024.10.22 00:39:40.287329 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5ecebcc6992a5b96668edbe4e8b56325 for removal 2024.10.22 00:39:40.292646 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 88a9a2d42fe62aa414858a3d153568a1, offset: 0, size: 1268 2024.10.22 00:39:40.298313 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 88a9a2d42fe62aa414858a3d153568a1 for removal 2024.10.22 00:39:40.304422 [ 1476 ] {} CacheMetadata: Key 5ecebcc6992a5b96668edbe4e8b56325 is removed from metadata 2024.10.22 00:39:40.307279 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: acd489cc2db77c2379c1d72a6ba8c5f3, offset: 0, size: 154 2024.10.22 00:39:40.314849 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key acd489cc2db77c2379c1d72a6ba8c5f3 for removal 2024.10.22 00:39:40.315247 [ 1476 ] {} CacheMetadata: Key 88a9a2d42fe62aa414858a3d153568a1 is removed from metadata 2024.10.22 00:39:40.320906 [ 1476 ] {} CacheMetadata: Key acd489cc2db77c2379c1d72a6ba8c5f3 is removed from metadata 2024.10.22 00:39:40.320918 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4943165587ece2acb7c54412c08a18d4, offset: 0, size: 154 2024.10.22 00:39:40.327048 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4943165587ece2acb7c54412c08a18d4 for removal 2024.10.22 00:39:40.328336 [ 1476 ] {} CacheMetadata: Key 40ff83358dc41ab07a60c70f36c224a3 is removed from metadata 2024.10.22 00:39:40.331611 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5c1ae2cb9a2c99c74e56a21c48a73a0b, offset: 0, size: 18053 2024.10.22 00:39:40.338295 [ 1476 ] {} CacheMetadata: Key 4943165587ece2acb7c54412c08a18d4 is removed from metadata 2024.10.22 00:39:40.344924 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5c1ae2cb9a2c99c74e56a21c48a73a0b for removal 2024.10.22 00:39:40.359587 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1a1ffc8aef72da70e4541b0605528f17, offset: 0, size: 53620 2024.10.22 00:39:40.364708 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1a1ffc8aef72da70e4541b0605528f17 for removal 2024.10.22 00:39:40.369802 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5f3fee3767f8d01eb7c5ecf3d24358c9, offset: 0, size: 164 2024.10.22 00:39:40.374595 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5f3fee3767f8d01eb7c5ecf3d24358c9 for removal 2024.10.22 00:39:40.377976 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dc883dcc315d339ac9e5131b9c442da1, offset: 0, size: 5015 2024.10.22 00:39:40.379756 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dc883dcc315d339ac9e5131b9c442da1 for removal 2024.10.22 00:39:40.381550 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7fd2f7c5bb521be81ef2f51c6753ffa0, offset: 0, size: 2455 2024.10.22 00:39:40.384177 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7fd2f7c5bb521be81ef2f51c6753ffa0 for removal 2024.10.22 00:39:40.386866 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 73 entries to flush up to offset 9679 2024.10.22 00:39:40.489996 [ 1476 ] {} CacheMetadata: Key 7fd2f7c5bb521be81ef2f51c6753ffa0 is removed from metadata 2024.10.22 00:39:40.491067 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b4544c247a78451d10196c5976e5e33e, offset: 0, size: 10377 2024.10.22 00:39:40.496622 [ 1476 ] {} CacheMetadata: Key 5f3fee3767f8d01eb7c5ecf3d24358c9 is removed from metadata 2024.10.22 00:39:40.506625 [ 1476 ] {} CacheMetadata: Key dc883dcc315d339ac9e5131b9c442da1 is removed from metadata 2024.10.22 00:39:40.515555 [ 1476 ] {} CacheMetadata: Key 1a1ffc8aef72da70e4541b0605528f17 is removed from metadata 2024.10.22 00:39:40.493789 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b4544c247a78451d10196c5976e5e33e for removal 2024.10.22 00:39:40.600797 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c193aa1c46d0c29627cbcc9bbf6a3cbc, offset: 0, size: 155 2024.10.22 00:39:40.605155 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c193aa1c46d0c29627cbcc9bbf6a3cbc for removal 2024.10.22 00:39:40.623798 [ 1476 ] {} CacheMetadata: Key b4544c247a78451d10196c5976e5e33e is removed from metadata 2024.10.22 00:39:40.731597 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:40.732745 [ 1476 ] {} CacheMetadata: Key 5c1ae2cb9a2c99c74e56a21c48a73a0b is removed from metadata 2024.10.22 00:39:40.736100 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.01 GiB. 2024.10.22 00:39:40.742861 [ 1476 ] {} CacheMetadata: Key 53f64b62194c3fd5f15848948b559a00 is removed from metadata 2024.10.22 00:39:40.756370 [ 1476 ] {} CacheMetadata: Key 586faf91fad9d703effa193c02875482 is removed from metadata 2024.10.22 00:39:40.765110 [ 1476 ] {} CacheMetadata: Key 7b835f0eb473b7653b7dd8abffdf7b8d is removed from metadata 2024.10.22 00:39:40.771288 [ 1476 ] {} CacheMetadata: Key 48f994682755da81cbb7dc0dc9cd0e61 is removed from metadata 2024.10.22 00:39:40.817917 [ 1476 ] {} CacheMetadata: Key c193aa1c46d0c29627cbcc9bbf6a3cbc is removed from metadata 2024.10.22 00:39:40.818655 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 66936621f35d308bf4f6d22664f0c814, offset: 0, size: 154 2024.10.22 00:39:40.821461 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 66936621f35d308bf4f6d22664f0c814 for removal 2024.10.22 00:39:40.826992 [ 1476 ] {} CacheMetadata: Key 66936621f35d308bf4f6d22664f0c814 is removed from metadata 2024.10.22 00:39:40.826850 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 15439f1fe772915e1ede39e1a9ca2e9d, offset: 0, size: 153 2024.10.22 00:39:40.834210 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 15439f1fe772915e1ede39e1a9ca2e9d for removal 2024.10.22 00:39:40.839972 [ 1476 ] {} CacheMetadata: Key 15439f1fe772915e1ede39e1a9ca2e9d is removed from metadata 2024.10.22 00:39:40.842179 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: eddbaccca9da1e74dae86a23f5f6fe47, offset: 0, size: 142 2024.10.22 00:39:40.855516 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key eddbaccca9da1e74dae86a23f5f6fe47 for removal 2024.10.22 00:39:40.861337 [ 1476 ] {} CacheMetadata: Key eddbaccca9da1e74dae86a23f5f6fe47 is removed from metadata 2024.10.22 00:39:40.862295 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f9f45dc5d3276afe497e6a82d9782b35, offset: 0, size: 7742 2024.10.22 00:39:40.864976 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f9f45dc5d3276afe497e6a82d9782b35 for removal 2024.10.22 00:39:40.867594 [ 1476 ] {} CacheMetadata: Key f9f45dc5d3276afe497e6a82d9782b35 is removed from metadata 2024.10.22 00:39:40.869179 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ce5a76b9159f377ca89a7fedcf03eb6f, offset: 0, size: 164 2024.10.22 00:39:40.873278 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ce5a76b9159f377ca89a7fedcf03eb6f for removal 2024.10.22 00:39:40.875766 [ 1476 ] {} CacheMetadata: Key ce5a76b9159f377ca89a7fedcf03eb6f is removed from metadata 2024.10.22 00:39:40.876123 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: deed314271b957ebfa767c5d2b316e39, offset: 0, size: 22571 2024.10.22 00:39:40.878889 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key deed314271b957ebfa767c5d2b316e39 for removal 2024.10.22 00:39:40.901422 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 77b8bd39ca37ad5cf3bda909213e38e8, offset: 0, size: 11281 2024.10.22 00:39:40.911011 [ 1476 ] {} CacheMetadata: Key deed314271b957ebfa767c5d2b316e39 is removed from metadata 2024.10.22 00:39:40.960386 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:41.012374 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 77b8bd39ca37ad5cf3bda909213e38e8 for removal 2024.10.22 00:39:41.013400 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 299.01 GiB. 2024.10.22 00:39:41.019273 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 434934eddafcc0351e5104e3c5b2eeb0, offset: 0, size: 17727 2024.10.22 00:39:41.022856 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 434934eddafcc0351e5104e3c5b2eeb0 for removal 2024.10.22 00:39:41.032383 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 64b313f41014dd5a8fdb427faa60e528, offset: 0, size: 9618 2024.10.22 00:39:41.039381 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 64b313f41014dd5a8fdb427faa60e528 for removal 2024.10.22 00:39:41.047557 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: af97391f6ff42774ceb441927c7484f1, offset: 0, size: 154 2024.10.22 00:39:41.062611 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key af97391f6ff42774ceb441927c7484f1 for removal 2024.10.22 00:39:41.064194 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_147_147_0 (state Temporary) 2024.10.22 00:39:41.068038 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 338f4467dc337dc2d23b820803e5ee7c, offset: 0, size: 54180 2024.10.22 00:39:41.073668 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 338f4467dc337dc2d23b820803e5ee7c for removal 2024.10.22 00:39:41.079555 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 196722745482467e1363381a174f0835, offset: 0, size: 154 2024.10.22 00:39:41.082636 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 196722745482467e1363381a174f0835 for removal 2024.10.22 00:39:41.085437 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 56de5f77bf8bb2cf3d7121a2dc2a8cb2, offset: 0, size: 154 2024.10.22 00:39:41.088715 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 56de5f77bf8bb2cf3d7121a2dc2a8cb2 for removal 2024.10.22 00:39:41.093661 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 615c00eff23a402d426e02537917d31c, offset: 0, size: 17211 2024.10.22 00:39:41.100890 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 615c00eff23a402d426e02537917d31c for removal 2024.10.22 00:39:41.105786 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_142_142_0 (state Temporary) 2024.10.22 00:39:41.107671 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 18c224cc63ecb45fc104db5e2293ba18, offset: 0, size: 164 2024.10.22 00:39:41.111358 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_147_147_0 to 202410_153_153_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:41.113598 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 18c224cc63ecb45fc104db5e2293ba18 for removal 2024.10.22 00:39:41.117834 [ 1476 ] {} CacheMetadata: Key 18c224cc63ecb45fc104db5e2293ba18 is removed from metadata 2024.10.22 00:39:41.118685 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: eaa90026aace844ee00014b910f0d65d, offset: 0, size: 489543 2024.10.22 00:39:41.121747 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key eaa90026aace844ee00014b910f0d65d for removal 2024.10.22 00:39:41.127048 [ 1476 ] {} CacheMetadata: Key eaa90026aace844ee00014b910f0d65d is removed from metadata 2024.10.22 00:39:41.128838 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3dc941844fe80f9ff669819a7a8ee604, offset: 0, size: 142 2024.10.22 00:39:41.133255 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3dc941844fe80f9ff669819a7a8ee604 for removal 2024.10.22 00:39:41.136792 [ 1476 ] {} CacheMetadata: Key 3dc941844fe80f9ff669819a7a8ee604 is removed from metadata 2024.10.22 00:39:41.138176 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bbf2afb44fa9148e62b513989d649c55, offset: 0, size: 177 2024.10.22 00:39:41.145498 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bbf2afb44fa9148e62b513989d649c55 for removal 2024.10.22 00:39:41.151570 [ 1476 ] {} CacheMetadata: Key bbf2afb44fa9148e62b513989d649c55 is removed from metadata 2024.10.22 00:39:41.157036 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8659a455e34222c3b482f81b704c65e9, offset: 0, size: 2569 2024.10.22 00:39:41.159513 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9679 2024.10.22 00:39:41.162156 [ 1476 ] {} CacheMetadata: Key 615c00eff23a402d426e02537917d31c is removed from metadata 2024.10.22 00:39:41.163451 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8659a455e34222c3b482f81b704c65e9 for removal 2024.10.22 00:39:41.166095 [ 1476 ] {} CacheMetadata: Key 56de5f77bf8bb2cf3d7121a2dc2a8cb2 is removed from metadata 2024.10.22 00:39:41.167141 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a89f8f7ffec59a5deeb25045ef5a17a0, offset: 0, size: 1268 2024.10.22 00:39:41.173389 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_142_142_0 to 202410_149_149_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:41.173196 [ 1476 ] {} CacheMetadata: Key 196722745482467e1363381a174f0835 is removed from metadata 2024.10.22 00:39:41.174589 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a89f8f7ffec59a5deeb25045ef5a17a0 for removal 2024.10.22 00:39:41.176325 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6ee0dca5028d7453643660aa444edb26, offset: 0, size: 18660 2024.10.22 00:39:41.179605 [ 1476 ] {} CacheMetadata: Key a89f8f7ffec59a5deeb25045ef5a17a0 is removed from metadata 2024.10.22 00:39:41.180424 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6ee0dca5028d7453643660aa444edb26 for removal 2024.10.22 00:39:41.183206 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6ecd9e8de1c1a481898ae16a54cc3224, offset: 0, size: 48828 2024.10.22 00:39:41.187699 [ 1476 ] {} CacheMetadata: Key 6ee0dca5028d7453643660aa444edb26 is removed from metadata 2024.10.22 00:39:41.189426 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6ecd9e8de1c1a481898ae16a54cc3224 for removal 2024.10.22 00:39:41.194184 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 88f441d8fe08413becb744155cd8ef96, offset: 0, size: 4964 2024.10.22 00:39:41.195103 [ 1476 ] {} CacheMetadata: Key 6ecd9e8de1c1a481898ae16a54cc3224 is removed from metadata 2024.10.22 00:39:41.202455 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 88f441d8fe08413becb744155cd8ef96 for removal 2024.10.22 00:39:41.214656 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ead185d5c34c464a8892a1850e7584a0, offset: 0, size: 18321 2024.10.22 00:39:41.215278 [ 1476 ] {} CacheMetadata: Key 88f441d8fe08413becb744155cd8ef96 is removed from metadata 2024.10.22 00:39:41.221057 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ead185d5c34c464a8892a1850e7584a0 for removal 2024.10.22 00:39:41.224022 [ 1476 ] {} CacheMetadata: Key ead185d5c34c464a8892a1850e7584a0 is removed from metadata 2024.10.22 00:39:41.224910 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cd06409eb492ab92ac614b996311a1bc, offset: 0, size: 154 2024.10.22 00:39:41.230289 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cd06409eb492ab92ac614b996311a1bc for removal 2024.10.22 00:39:41.236104 [ 1476 ] {} CacheMetadata: Key cd06409eb492ab92ac614b996311a1bc is removed from metadata 2024.10.22 00:39:41.235753 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7b510ce21b9d6dd71bafa6b08e9dd53f, offset: 0, size: 37465 2024.10.22 00:39:41.243837 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7b510ce21b9d6dd71bafa6b08e9dd53f for removal 2024.10.22 00:39:41.248938 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9c8746cf1d1c49d89cf39bac2569d74c, offset: 0, size: 14973 2024.10.22 00:39:41.258923 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9c8746cf1d1c49d89cf39bac2569d74c for removal 2024.10.22 00:39:41.276048 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b3fe64de0b7e067e97030cc4f8c6dbd8, offset: 0, size: 2521 2024.10.22 00:39:41.279690 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b3fe64de0b7e067e97030cc4f8c6dbd8 for removal 2024.10.22 00:39:41.283521 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 223cdecbb63190493ecd46563b1587fb, offset: 0, size: 111357 2024.10.22 00:39:41.287142 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 223cdecbb63190493ecd46563b1587fb for removal 2024.10.22 00:39:41.316231 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 247072 2024.10.22 00:39:41.392329 [ 1476 ] {} CacheMetadata: Key 223cdecbb63190493ecd46563b1587fb is removed from metadata 2024.10.22 00:39:41.393412 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9292934b034c4b4bf346e9587c804ee9, offset: 0, size: 208 2024.10.22 00:39:41.401422 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9292934b034c4b4bf346e9587c804ee9 for removal 2024.10.22 00:39:41.401694 [ 1476 ] {} CacheMetadata: Key b3fe64de0b7e067e97030cc4f8c6dbd8 is removed from metadata 2024.10.22 00:39:41.412536 [ 1476 ] {} CacheMetadata: Key 9292934b034c4b4bf346e9587c804ee9 is removed from metadata 2024.10.22 00:39:41.417813 [ 1476 ] {} CacheMetadata: Key 9c8746cf1d1c49d89cf39bac2569d74c is removed from metadata 2024.10.22 00:39:41.423053 [ 1476 ] {} CacheMetadata: Key 7b510ce21b9d6dd71bafa6b08e9dd53f is removed from metadata 2024.10.22 00:39:41.432984 [ 1476 ] {} CacheMetadata: Key 338f4467dc337dc2d23b820803e5ee7c is removed from metadata 2024.10.22 00:39:41.452572 [ 1476 ] {} CacheMetadata: Key af97391f6ff42774ceb441927c7484f1 is removed from metadata 2024.10.22 00:39:41.466436 [ 1476 ] {} CacheMetadata: Key 64b313f41014dd5a8fdb427faa60e528 is removed from metadata 2024.10.22 00:39:41.471843 [ 1476 ] {} CacheMetadata: Key 434934eddafcc0351e5104e3c5b2eeb0 is removed from metadata 2024.10.22 00:39:41.478829 [ 1476 ] {} CacheMetadata: Key 8659a455e34222c3b482f81b704c65e9 is removed from metadata 2024.10.22 00:39:41.487183 [ 1476 ] {} CacheMetadata: Key 77b8bd39ca37ad5cf3bda909213e38e8 is removed from metadata 2024.10.22 00:39:41.509689 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 04326957c0abb10a836322e8fa807329, offset: 0, size: 53621 2024.10.22 00:39:41.516759 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 04326957c0abb10a836322e8fa807329 for removal 2024.10.22 00:39:41.524247 [ 1476 ] {} CacheMetadata: Key 04326957c0abb10a836322e8fa807329 is removed from metadata 2024.10.22 00:39:41.524129 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0643bae73bf875fc36a9fbbd9cb54acd, offset: 0, size: 164 2024.10.22 00:39:41.528624 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0643bae73bf875fc36a9fbbd9cb54acd for removal 2024.10.22 00:39:41.536167 [ 1476 ] {} CacheMetadata: Key 0643bae73bf875fc36a9fbbd9cb54acd is removed from metadata 2024.10.22 00:39:41.535810 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 99d3bc5f4a5d32cb9bcb075b1eee5e1a, offset: 0, size: 153 2024.10.22 00:39:41.547299 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 99d3bc5f4a5d32cb9bcb075b1eee5e1a for removal 2024.10.22 00:39:41.553157 [ 1476 ] {} CacheMetadata: Key 99d3bc5f4a5d32cb9bcb075b1eee5e1a is removed from metadata 2024.10.22 00:39:41.555193 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ee63d4b407c27df64c7f6a6e38f18e95, offset: 0, size: 142 2024.10.22 00:39:41.562611 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ee63d4b407c27df64c7f6a6e38f18e95 for removal 2024.10.22 00:39:41.567797 [ 1476 ] {} CacheMetadata: Key ee63d4b407c27df64c7f6a6e38f18e95 is removed from metadata 2024.10.22 00:39:41.569255 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c112cee6db065b3b10f782a61c19ba21, offset: 0, size: 21304 2024.10.22 00:39:41.573793 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c112cee6db065b3b10f782a61c19ba21 for removal 2024.10.22 00:39:41.578385 [ 1476 ] {} CacheMetadata: Key c112cee6db065b3b10f782a61c19ba21 is removed from metadata 2024.10.22 00:39:41.579542 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f8c622c54b92ff4a31603189c1febedd, offset: 0, size: 154 2024.10.22 00:39:41.582256 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f8c622c54b92ff4a31603189c1febedd for removal 2024.10.22 00:39:41.594605 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b9072ff12b0856939be372d32e427fee, offset: 0, size: 174 2024.10.22 00:39:41.596827 [ 1476 ] {} CacheMetadata: Key f8c622c54b92ff4a31603189c1febedd is removed from metadata 2024.10.22 00:39:41.600205 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b9072ff12b0856939be372d32e427fee for removal 2024.10.22 00:39:41.610347 [ 1476 ] {} CacheMetadata: Key b9072ff12b0856939be372d32e427fee is removed from metadata 2024.10.22 00:39:41.610995 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: baf9e0246979885187d2b3f7c3bca9b9, offset: 0, size: 4964 2024.10.22 00:39:41.615655 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key baf9e0246979885187d2b3f7c3bca9b9 for removal 2024.10.22 00:39:41.618156 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 876460ec3f0a28f060ef83dfcb5bdc46, offset: 0, size: 1268 2024.10.22 00:39:41.620519 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 876460ec3f0a28f060ef83dfcb5bdc46 for removal 2024.10.22 00:39:41.622228 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a35cadfd2654d4cb9bcc3a2d2930a814, offset: 0, size: 37020 2024.10.22 00:39:41.629154 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a35cadfd2654d4cb9bcc3a2d2930a814 for removal 2024.10.22 00:39:41.638214 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 54037dab6ff57274d79dcc6a606fa8b7, offset: 0, size: 46049 2024.10.22 00:39:41.647068 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 54037dab6ff57274d79dcc6a606fa8b7 for removal 2024.10.22 00:39:41.651510 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 523ed9fa8342d36b82499fde53c9f4ab, offset: 0, size: 2237846 2024.10.22 00:39:41.666241 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 523ed9fa8342d36b82499fde53c9f4ab for removal 2024.10.22 00:39:41.673056 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ab12301993151ffb841f61cb85732fcc, offset: 0, size: 6985 2024.10.22 00:39:41.675752 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ab12301993151ffb841f61cb85732fcc for removal 2024.10.22 00:39:41.683679 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f9ddf0c1d0f288be542f2f45816d2162, offset: 0, size: 154 2024.10.22 00:39:41.687132 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f9ddf0c1d0f288be542f2f45816d2162 for removal 2024.10.22 00:39:41.690734 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 53e5d8d134de084bacc806b5e8cb4820, offset: 0, size: 27959 2024.10.22 00:39:41.703235 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 53e5d8d134de084bacc806b5e8cb4820 for removal 2024.10.22 00:39:41.709093 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 15a74b2a70c2351b0a0b58458928bb33, offset: 0, size: 2509 2024.10.22 00:39:41.713606 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 15a74b2a70c2351b0a0b58458928bb33 for removal 2024.10.22 00:39:41.718493 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 710bed01eebc66d65659846b6888617f, offset: 0, size: 154 2024.10.22 00:39:41.721597 [ 1476 ] {} CacheMetadata: Key 15a74b2a70c2351b0a0b58458928bb33 is removed from metadata 2024.10.22 00:39:41.722365 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 710bed01eebc66d65659846b6888617f for removal 2024.10.22 00:39:41.730239 [ 1476 ] {} CacheMetadata: Key f9ddf0c1d0f288be542f2f45816d2162 is removed from metadata 2024.10.22 00:39:41.730580 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d398a1c1237aa54ec5174294d679ed0d, offset: 0, size: 116789 2024.10.22 00:39:41.734712 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d398a1c1237aa54ec5174294d679ed0d for removal 2024.10.22 00:39:41.738124 [ 1476 ] {} CacheMetadata: Key d398a1c1237aa54ec5174294d679ed0d is removed from metadata 2024.10.22 00:39:41.739658 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 66bc3ab9001f1ad78c8cd4a3c3d34b87, offset: 0, size: 154 2024.10.22 00:39:41.745756 [ 1476 ] {} CacheMetadata: Key 53e5d8d134de084bacc806b5e8cb4820 is removed from metadata 2024.10.22 00:39:41.746879 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 66bc3ab9001f1ad78c8cd4a3c3d34b87 for removal 2024.10.22 00:39:41.751256 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 34d26f2f9a418b7a032bce9cdafa7318, offset: 0, size: 155 2024.10.22 00:39:41.752770 [ 1476 ] {} CacheMetadata: Key 66bc3ab9001f1ad78c8cd4a3c3d34b87 is removed from metadata 2024.10.22 00:39:41.754796 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 34d26f2f9a418b7a032bce9cdafa7318 for removal 2024.10.22 00:39:41.757381 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 52d54f4ad8db44f8f2e190c69d8f889c, offset: 0, size: 5015 2024.10.22 00:39:41.759903 [ 1476 ] {} CacheMetadata: Key 34d26f2f9a418b7a032bce9cdafa7318 is removed from metadata 2024.10.22 00:39:41.761638 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 52d54f4ad8db44f8f2e190c69d8f889c for removal 2024.10.22 00:39:41.764131 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9f7e16374a560b4318b103f0a42a0268, offset: 0, size: 51814 2024.10.22 00:39:41.767005 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9f7e16374a560b4318b103f0a42a0268 for removal 2024.10.22 00:39:41.767151 [ 1476 ] {} CacheMetadata: Key 52d54f4ad8db44f8f2e190c69d8f889c is removed from metadata 2024.10.22 00:39:41.769974 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5515a5cfce6d309927825a19092439a7, offset: 0, size: 142 2024.10.22 00:39:41.770682 [ 1476 ] {} CacheMetadata: Key 9f7e16374a560b4318b103f0a42a0268 is removed from metadata 2024.10.22 00:39:41.771403 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5515a5cfce6d309927825a19092439a7 for removal 2024.10.22 00:39:41.772842 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8cddac17600e6866a7bc332b6e536e0b, offset: 0, size: 164 2024.10.22 00:39:41.773303 [ 1476 ] {} CacheMetadata: Key 5515a5cfce6d309927825a19092439a7 is removed from metadata 2024.10.22 00:39:41.774372 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8cddac17600e6866a7bc332b6e536e0b for removal 2024.10.22 00:39:41.775887 [ 1476 ] {} CacheMetadata: Key 8cddac17600e6866a7bc332b6e536e0b is removed from metadata 2024.10.22 00:39:41.775897 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: aa830d4025af35b63e7b239575fbcef9, offset: 0, size: 127400 2024.10.22 00:39:41.778798 [ 1476 ] {} CacheMetadata: Key 523ed9fa8342d36b82499fde53c9f4ab is removed from metadata 2024.10.22 00:39:41.782658 [ 1476 ] {} CacheMetadata: Key 54037dab6ff57274d79dcc6a606fa8b7 is removed from metadata 2024.10.22 00:39:41.785986 [ 1476 ] {} CacheMetadata: Key a35cadfd2654d4cb9bcc3a2d2930a814 is removed from metadata 2024.10.22 00:39:41.795144 [ 1476 ] {} CacheMetadata: Key ab12301993151ffb841f61cb85732fcc is removed from metadata 2024.10.22 00:39:41.807149 [ 1476 ] {} CacheMetadata: Key 876460ec3f0a28f060ef83dfcb5bdc46 is removed from metadata 2024.10.22 00:39:41.813793 [ 1476 ] {} CacheMetadata: Key 710bed01eebc66d65659846b6888617f is removed from metadata 2024.10.22 00:39:41.834020 [ 1476 ] {} CacheMetadata: Key baf9e0246979885187d2b3f7c3bca9b9 is removed from metadata 2024.10.22 00:39:41.845488 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 452 entries to flush up to offset 13818 2024.10.22 00:39:41.878614 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key aa830d4025af35b63e7b239575fbcef9 for removal 2024.10.22 00:39:41.880780 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7542590eaa8cbebc32ef19183e08ef4b, offset: 0, size: 154 2024.10.22 00:39:41.884880 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7542590eaa8cbebc32ef19183e08ef4b for removal 2024.10.22 00:39:41.958750 [ 1476 ] {} CacheMetadata: Key 7542590eaa8cbebc32ef19183e08ef4b is removed from metadata 2024.10.22 00:39:41.969102 [ 1476 ] {} CacheMetadata: Key aa830d4025af35b63e7b239575fbcef9 is removed from metadata 2024.10.22 00:39:42.162642 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/nha/fqnkkorkipggbolblpoywpaztgkzw, upload id eaf0db76-1ffc-45ab-b8b1-239f2c196ee2 2024.10.22 00:39:42.942802 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:42.950532 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:43.172846 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_255_255_0 (state Temporary) 2024.10.22 00:39:43.354425 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_255_255_0 to 202410_268_268_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:43.415323 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 13818 2024.10.22 00:39:43.738864 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 5.99 MiB, will set to 1.63 GiB (RSS), difference: -14.32 MiB 2024.10.22 00:39:45.418757 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 6.15 MiB, will set to 1.64 GiB (RSS), difference: 8.53 MiB 2024.10.22 00:39:45.873752 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 2099 2024.10.22 00:39:45.892653 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 137 2024.10.22 00:39:46.247905 [ 1157 ] {} DatabaseCatalog: Execution took 369 ms. 2024.10.22 00:39:46.359907 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:46.366428 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:46.575696 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:46.579779 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:46.596076 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_149_149_0 (state Temporary) 2024.10.22 00:39:46.649154 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_149_149_0 to 202410_162_162_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:46.688984 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2099 2024.10.22 00:39:46.690813 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 6.00 MiB, will set to 1.64 GiB (RSS), difference: 1.84 MiB 2024.10.22 00:39:46.749502 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_119_119_0 (state Temporary) 2024.10.22 00:39:47.074463 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_119_119_0 to 202410_129_129_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:47.264327 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 137 2024.10.22 00:39:47.683829 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 29 entries to flush up to offset 13847 2024.10.22 00:39:47.749450 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:47.752888 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:48.021770 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:48.026409 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:48.078434 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_256_256_0 (state Temporary) 2024.10.22 00:39:48.132631 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_256_256_0 to 202410_269_269_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:48.182927 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 13847 2024.10.22 00:39:48.321136 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 248608 2024.10.22 00:39:48.670854 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 38 entries to flush up to offset 9717 2024.10.22 00:39:48.797189 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:48.807643 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:48.884452 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_148_148_0 (state Temporary) 2024.10.22 00:39:48.952848 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_148_148_0 to 202410_154_154_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:49.008510 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 5.24 MiB, will set to 1.64 GiB (RSS), difference: 2.01 MiB 2024.10.22 00:39:49.097000 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9717 2024.10.22 00:39:49.232725 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_90_18, 202410_91_91_0, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0] 2024.10.22 00:39:49.237408 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_90_18, 202410_91_91_0, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0] 2024.10.22 00:39:49.337207 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:49.357337 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:49.367414 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_90_18, 202410_91_91_0, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0] 2024.10.22 00:39:49.517546 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_143_143_0 (state Temporary) 2024.10.22 00:39:49.520008 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:39:49.522266 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:39:49.577655 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_143_143_0 to 202410_150_150_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:49.607529 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 248608 2024.10.22 00:39:51.016217 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 135 file segments (while reserving 16777216 bytes for 7a2eb8cea9c17e12aec9d150ec4c29b7:33554432) 2024.10.22 00:39:51.020149 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7a2eb8cea9c17e12aec9d150ec4c29b7, offset: 0, size: 33554432 2024.10.22 00:39:51.044663 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 148a39c23f06641abee1eb526a52ec6f, offset: 0, size: 576010 2024.10.22 00:39:51.056766 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 148a39c23f06641abee1eb526a52ec6f for removal 2024.10.22 00:39:51.060080 [ 1476 ] {} CacheMetadata: Key 148a39c23f06641abee1eb526a52ec6f is removed from metadata 2024.10.22 00:39:51.063712 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 32fe03495c00c6821d737a5e021ae4fd, offset: 0, size: 192 2024.10.22 00:39:51.067085 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 32fe03495c00c6821d737a5e021ae4fd for removal 2024.10.22 00:39:51.071879 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f98ae42a94578360ce3bbda6fb78ed47, offset: 0, size: 823748 2024.10.22 00:39:51.075185 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f98ae42a94578360ce3bbda6fb78ed47 for removal 2024.10.22 00:39:51.075885 [ 1476 ] {} CacheMetadata: Key 32fe03495c00c6821d737a5e021ae4fd is removed from metadata 2024.10.22 00:39:51.077522 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2b5daff9500fc2fee0deb520b39a9aa9, offset: 0, size: 1300 2024.10.22 00:39:51.082101 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2b5daff9500fc2fee0deb520b39a9aa9 for removal 2024.10.22 00:39:51.086274 [ 1476 ] {} CacheMetadata: Key f98ae42a94578360ce3bbda6fb78ed47 is removed from metadata 2024.10.22 00:39:51.086756 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: eb3b1444c8a3bea43e2879f6b1eb62ea, offset: 0, size: 156 2024.10.22 00:39:51.089326 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key eb3b1444c8a3bea43e2879f6b1eb62ea for removal 2024.10.22 00:39:51.092238 [ 1476 ] {} CacheMetadata: Key eb3b1444c8a3bea43e2879f6b1eb62ea is removed from metadata 2024.10.22 00:39:51.093695 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f17b3e5b36eeae91d6cb7af2e8edb336, offset: 0, size: 155 2024.10.22 00:39:51.097030 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f17b3e5b36eeae91d6cb7af2e8edb336 for removal 2024.10.22 00:39:51.097619 [ 1476 ] {} CacheMetadata: Key 2b5daff9500fc2fee0deb520b39a9aa9 is removed from metadata 2024.10.22 00:39:51.099895 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 887d9a063374239a8590d14d4d8b9fd6, offset: 0, size: 220545 2024.10.22 00:39:51.105615 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 887d9a063374239a8590d14d4d8b9fd6 for removal 2024.10.22 00:39:51.113383 [ 1476 ] {} CacheMetadata: Key f17b3e5b36eeae91d6cb7af2e8edb336 is removed from metadata 2024.10.22 00:39:51.116944 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6ffca5c7f419a8377da7472706740978, offset: 0, size: 96140 2024.10.22 00:39:51.119866 [ 1476 ] {} CacheMetadata: Key 887d9a063374239a8590d14d4d8b9fd6 is removed from metadata 2024.10.22 00:39:51.120818 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6ffca5c7f419a8377da7472706740978 for removal 2024.10.22 00:39:51.122821 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5e486b540bfc0070972accb96e36c282, offset: 0, size: 112548 2024.10.22 00:39:51.127388 [ 1476 ] {} CacheMetadata: Key 6ffca5c7f419a8377da7472706740978 is removed from metadata 2024.10.22 00:39:51.128717 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5e486b540bfc0070972accb96e36c282 for removal 2024.10.22 00:39:51.148255 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b996c03848e7104885aa4f4e022f5cda, offset: 0, size: 113057 2024.10.22 00:39:51.150476 [ 1476 ] {} CacheMetadata: Key 5e486b540bfc0070972accb96e36c282 is removed from metadata 2024.10.22 00:39:51.154352 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b996c03848e7104885aa4f4e022f5cda for removal 2024.10.22 00:39:51.171161 [ 1476 ] {} CacheMetadata: Key b996c03848e7104885aa4f4e022f5cda is removed from metadata 2024.10.22 00:39:51.171990 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2ef20f695fc1c01b6e20a42098d713c4, offset: 0, size: 153 2024.10.22 00:39:51.174310 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2ef20f695fc1c01b6e20a42098d713c4 for removal 2024.10.22 00:39:51.175941 [ 1476 ] {} CacheMetadata: Key 2ef20f695fc1c01b6e20a42098d713c4 is removed from metadata 2024.10.22 00:39:51.176319 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c82c2a8d224dc3aeedb23c9825a5b4db, offset: 0, size: 155 2024.10.22 00:39:51.179820 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c82c2a8d224dc3aeedb23c9825a5b4db for removal 2024.10.22 00:39:51.182968 [ 1476 ] {} CacheMetadata: Key c82c2a8d224dc3aeedb23c9825a5b4db is removed from metadata 2024.10.22 00:39:51.183611 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a0bcfb66faaa0b5fcb76ffe8099cf5cf, offset: 0, size: 155 2024.10.22 00:39:51.185162 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a0bcfb66faaa0b5fcb76ffe8099cf5cf for removal 2024.10.22 00:39:51.186638 [ 1476 ] {} CacheMetadata: Key a0bcfb66faaa0b5fcb76ffe8099cf5cf is removed from metadata 2024.10.22 00:39:51.188699 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7911a5d5b884034d263101dfa99a49f1, offset: 0, size: 244085 2024.10.22 00:39:51.190267 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7911a5d5b884034d263101dfa99a49f1 for removal 2024.10.22 00:39:51.192105 [ 1476 ] {} CacheMetadata: Key 7911a5d5b884034d263101dfa99a49f1 is removed from metadata 2024.10.22 00:39:51.193015 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6a40ac6d14a12ea1305f9ea506706af2, offset: 0, size: 159 2024.10.22 00:39:51.194717 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6a40ac6d14a12ea1305f9ea506706af2 for removal 2024.10.22 00:39:51.197063 [ 1476 ] {} CacheMetadata: Key 6a40ac6d14a12ea1305f9ea506706af2 is removed from metadata 2024.10.22 00:39:51.196834 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c2d586d07c729733e6a140490be21657, offset: 0, size: 383691 2024.10.22 00:39:51.202518 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c2d586d07c729733e6a140490be21657 for removal 2024.10.22 00:39:51.208640 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a223451c8e3447505cb138422a995a11, offset: 0, size: 159 2024.10.22 00:39:51.216807 [ 1476 ] {} CacheMetadata: Key c2d586d07c729733e6a140490be21657 is removed from metadata 2024.10.22 00:39:51.222913 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a223451c8e3447505cb138422a995a11 for removal 2024.10.22 00:39:51.227205 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 45dd818dafccb28e9340e403923c469a, offset: 0, size: 155 2024.10.22 00:39:51.228532 [ 1476 ] {} CacheMetadata: Key a223451c8e3447505cb138422a995a11 is removed from metadata 2024.10.22 00:39:51.231672 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 45dd818dafccb28e9340e403923c469a for removal 2024.10.22 00:39:51.236286 [ 1476 ] {} CacheMetadata: Key 45dd818dafccb28e9340e403923c469a is removed from metadata 2024.10.22 00:39:51.237412 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d241a5cc09701c2b1f9962c64f9f9cc1, offset: 0, size: 168 2024.10.22 00:39:51.241442 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d241a5cc09701c2b1f9962c64f9f9cc1 for removal 2024.10.22 00:39:51.247806 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 140dc4ad684568c5ac31391e1ab415fa, offset: 0, size: 167718 2024.10.22 00:39:51.257348 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 140dc4ad684568c5ac31391e1ab415fa for removal 2024.10.22 00:39:51.273772 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 49bf767c9cdc02dd65dd4f2365852041, offset: 0, size: 207786 2024.10.22 00:39:51.276942 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 49bf767c9cdc02dd65dd4f2365852041 for removal 2024.10.22 00:39:51.279922 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d9a7214102c91f3dafb2aed3307fc77d, offset: 0, size: 156 2024.10.22 00:39:51.284170 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d9a7214102c91f3dafb2aed3307fc77d for removal 2024.10.22 00:39:51.290884 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bfe54f1ccd0ba0f98624a835fefb89ef, offset: 0, size: 98795 2024.10.22 00:39:51.292870 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bfe54f1ccd0ba0f98624a835fefb89ef for removal 2024.10.22 00:39:51.294997 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9d83ed1ff5acca495207d8c5cc75cd19, offset: 0, size: 157 2024.10.22 00:39:51.299353 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9d83ed1ff5acca495207d8c5cc75cd19 for removal 2024.10.22 00:39:51.304643 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a5757c4db3608ee0eb36ab4846b6b207, offset: 0, size: 48318 2024.10.22 00:39:51.354326 [ 1476 ] {} CacheMetadata: Key 9d83ed1ff5acca495207d8c5cc75cd19 is removed from metadata 2024.10.22 00:39:51.367293 [ 1476 ] {} CacheMetadata: Key bfe54f1ccd0ba0f98624a835fefb89ef is removed from metadata 2024.10.22 00:39:51.380399 [ 1476 ] {} CacheMetadata: Key d9a7214102c91f3dafb2aed3307fc77d is removed from metadata 2024.10.22 00:39:51.398761 [ 1476 ] {} CacheMetadata: Key 49bf767c9cdc02dd65dd4f2365852041 is removed from metadata 2024.10.22 00:39:51.414442 [ 1476 ] {} CacheMetadata: Key 140dc4ad684568c5ac31391e1ab415fa is removed from metadata 2024.10.22 00:39:51.415996 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a5757c4db3608ee0eb36ab4846b6b207 for removal 2024.10.22 00:39:51.419471 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: efefb9da5814f73f0d9e4eb7c0b7bd31, offset: 0, size: 167 2024.10.22 00:39:51.421802 [ 1476 ] {} CacheMetadata: Key a5757c4db3608ee0eb36ab4846b6b207 is removed from metadata 2024.10.22 00:39:51.423222 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key efefb9da5814f73f0d9e4eb7c0b7bd31 for removal 2024.10.22 00:39:51.428866 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 388af86b3becc6198344b2409b93bc13, offset: 0, size: 168 2024.10.22 00:39:51.429253 [ 1476 ] {} CacheMetadata: Key efefb9da5814f73f0d9e4eb7c0b7bd31 is removed from metadata 2024.10.22 00:39:51.432258 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 388af86b3becc6198344b2409b93bc13 for removal 2024.10.22 00:39:51.436892 [ 1476 ] {} CacheMetadata: Key d241a5cc09701c2b1f9962c64f9f9cc1 is removed from metadata 2024.10.22 00:39:51.448594 [ 1476 ] {} CacheMetadata: Key 388af86b3becc6198344b2409b93bc13 is removed from metadata 2024.10.22 00:39:51.452719 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c8d973c07385e8caee74780d158e4174, offset: 0, size: 77121 2024.10.22 00:39:51.462798 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c8d973c07385e8caee74780d158e4174 for removal 2024.10.22 00:39:51.466859 [ 1476 ] {} CacheMetadata: Key c8d973c07385e8caee74780d158e4174 is removed from metadata 2024.10.22 00:39:51.468374 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 76a39623c9e1b3ed0ebc30bd2fa3b3d5, offset: 0, size: 59552 2024.10.22 00:39:51.470289 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 76a39623c9e1b3ed0ebc30bd2fa3b3d5 for removal 2024.10.22 00:39:51.472036 [ 1476 ] {} CacheMetadata: Key 76a39623c9e1b3ed0ebc30bd2fa3b3d5 is removed from metadata 2024.10.22 00:39:51.472197 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 71d79b823b0be8bf44e49ae2d93d3ae4, offset: 0, size: 59365 2024.10.22 00:39:51.474713 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 71d79b823b0be8bf44e49ae2d93d3ae4 for removal 2024.10.22 00:39:51.475532 [ 1476 ] {} CacheMetadata: Key 71d79b823b0be8bf44e49ae2d93d3ae4 is removed from metadata 2024.10.22 00:39:51.475688 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0ed8a2fd596f2a56960fcc87b12cd980, offset: 0, size: 268627 2024.10.22 00:39:51.477492 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0ed8a2fd596f2a56960fcc87b12cd980 for removal 2024.10.22 00:39:51.478795 [ 1476 ] {} CacheMetadata: Key 0ed8a2fd596f2a56960fcc87b12cd980 is removed from metadata 2024.10.22 00:39:51.479111 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 62496262a259665fb8ec45af3f53e744, offset: 0, size: 156 2024.10.22 00:39:51.481174 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 62496262a259665fb8ec45af3f53e744 for removal 2024.10.22 00:39:51.481581 [ 1191 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:39:51.484015 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9d2e012820e932e1eb90358a6f17993b, offset: 0, size: 151 2024.10.22 00:39:51.484128 [ 1476 ] {} CacheMetadata: Key 62496262a259665fb8ec45af3f53e744 is removed from metadata 2024.10.22 00:39:51.486245 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9d2e012820e932e1eb90358a6f17993b for removal 2024.10.22 00:39:51.488797 [ 1476 ] {} CacheMetadata: Key 9d2e012820e932e1eb90358a6f17993b is removed from metadata 2024.10.22 00:39:51.488350 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d933343d7f79d08c7038ae2a1664c1e6, offset: 0, size: 157 2024.10.22 00:39:51.499458 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d933343d7f79d08c7038ae2a1664c1e6 for removal 2024.10.22 00:39:51.503922 [ 1191 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:39:51.515458 [ 1476 ] {} CacheMetadata: Key d933343d7f79d08c7038ae2a1664c1e6 is removed from metadata 2024.10.22 00:39:51.516995 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b35e1dec02b57d5272d55e26a4ccc8b9, offset: 0, size: 156 2024.10.22 00:39:51.536480 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b35e1dec02b57d5272d55e26a4ccc8b9 for removal 2024.10.22 00:39:51.544886 [ 1476 ] {} CacheMetadata: Key b35e1dec02b57d5272d55e26a4ccc8b9 is removed from metadata 2024.10.22 00:39:51.545522 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a1ac0ff709801d3bab0f5c55295a4930, offset: 0, size: 167 2024.10.22 00:39:51.554253 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a1ac0ff709801d3bab0f5c55295a4930 for removal 2024.10.22 00:39:51.560036 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 917e97b9d24191d45e56b2efbb8bbef4, offset: 0, size: 192987 2024.10.22 00:39:51.562346 [ 1476 ] {} CacheMetadata: Key a1ac0ff709801d3bab0f5c55295a4930 is removed from metadata 2024.10.22 00:39:51.563301 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 917e97b9d24191d45e56b2efbb8bbef4 for removal 2024.10.22 00:39:51.568769 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e9298c1bc23d40fcee010d7cd280a5d1, offset: 0, size: 96373 2024.10.22 00:39:51.570370 [ 1476 ] {} CacheMetadata: Key 917e97b9d24191d45e56b2efbb8bbef4 is removed from metadata 2024.10.22 00:39:51.571608 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e9298c1bc23d40fcee010d7cd280a5d1 for removal 2024.10.22 00:39:51.573488 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 62151225b84628448c2f8b2a5157bdd2, offset: 0, size: 5128 2024.10.22 00:39:51.576014 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 62151225b84628448c2f8b2a5157bdd2 for removal 2024.10.22 00:39:51.578224 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fda3215412aab1b5a224f56497df2a2e, offset: 0, size: 157 2024.10.22 00:39:51.580273 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fda3215412aab1b5a224f56497df2a2e for removal 2024.10.22 00:39:51.583942 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 21b4901e9c46cb6832a3faa7c507d2db, offset: 0, size: 180 2024.10.22 00:39:51.586272 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 21b4901e9c46cb6832a3faa7c507d2db for removal 2024.10.22 00:39:51.587739 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 949acc828312a249cbb482a670125f85, offset: 0, size: 156 2024.10.22 00:39:51.589440 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 949acc828312a249cbb482a670125f85 for removal 2024.10.22 00:39:51.591998 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c6233d7f7e69949710493c4135a95c88, offset: 0, size: 156 2024.10.22 00:39:51.594564 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c6233d7f7e69949710493c4135a95c88 for removal 2024.10.22 00:39:51.598417 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7b40b8f8d0c2091c6abdd961cd1c9aaf, offset: 0, size: 261296 2024.10.22 00:39:51.603039 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7b40b8f8d0c2091c6abdd961cd1c9aaf for removal 2024.10.22 00:39:51.612444 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8813e8b6ca2ea0d29f3a95df02654f24, offset: 0, size: 108267 2024.10.22 00:39:51.616209 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8813e8b6ca2ea0d29f3a95df02654f24 for removal 2024.10.22 00:39:51.626668 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 660c06985dbb1725a7f45d3b06c66d80, offset: 0, size: 160601 2024.10.22 00:39:51.633377 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 660c06985dbb1725a7f45d3b06c66d80 for removal 2024.10.22 00:39:51.675131 [ 1476 ] {} CacheMetadata: Key e9298c1bc23d40fcee010d7cd280a5d1 is removed from metadata 2024.10.22 00:39:51.681060 [ 1476 ] {} CacheMetadata: Key 8813e8b6ca2ea0d29f3a95df02654f24 is removed from metadata 2024.10.22 00:39:51.686756 [ 1476 ] {} CacheMetadata: Key 7b40b8f8d0c2091c6abdd961cd1c9aaf is removed from metadata 2024.10.22 00:39:51.692378 [ 1476 ] {} CacheMetadata: Key c6233d7f7e69949710493c4135a95c88 is removed from metadata 2024.10.22 00:39:51.713612 [ 1476 ] {} CacheMetadata: Key 949acc828312a249cbb482a670125f85 is removed from metadata 2024.10.22 00:39:51.730610 [ 1476 ] {} CacheMetadata: Key fda3215412aab1b5a224f56497df2a2e is removed from metadata 2024.10.22 00:39:51.741160 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fbdd0de2377119c357d16a507c8f1d73, offset: 0, size: 6109 2024.10.22 00:39:51.741818 [ 1476 ] {} CacheMetadata: Key 660c06985dbb1725a7f45d3b06c66d80 is removed from metadata 2024.10.22 00:39:51.751323 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fbdd0de2377119c357d16a507c8f1d73 for removal 2024.10.22 00:39:51.759164 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 27b3a51cb59b08628bea23460af80a9f, offset: 0, size: 156 2024.10.22 00:39:51.759427 [ 1476 ] {} CacheMetadata: Key 21b4901e9c46cb6832a3faa7c507d2db is removed from metadata 2024.10.22 00:39:51.767645 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 27b3a51cb59b08628bea23460af80a9f for removal 2024.10.22 00:39:51.770722 [ 1476 ] {} CacheMetadata: Key 27b3a51cb59b08628bea23460af80a9f is removed from metadata 2024.10.22 00:39:51.770689 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5e9111597116bbad70fa06f0572eeb1b, offset: 0, size: 156 2024.10.22 00:39:51.773312 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5e9111597116bbad70fa06f0572eeb1b for removal 2024.10.22 00:39:51.775462 [ 1476 ] {} CacheMetadata: Key 5e9111597116bbad70fa06f0572eeb1b is removed from metadata 2024.10.22 00:39:51.775497 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6e256354b2b321e4b907132d031b80ef, offset: 0, size: 65823 2024.10.22 00:39:51.778544 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6e256354b2b321e4b907132d031b80ef for removal 2024.10.22 00:39:51.782321 [ 1476 ] {} CacheMetadata: Key fbdd0de2377119c357d16a507c8f1d73 is removed from metadata 2024.10.22 00:39:51.782946 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cd97c1898172e70052c12b5b4303dafd, offset: 0, size: 163 2024.10.22 00:39:51.786358 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cd97c1898172e70052c12b5b4303dafd for removal 2024.10.22 00:39:51.790100 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2ff366c7c7192159cce1813beffaa11e, offset: 0, size: 151 2024.10.22 00:39:51.791174 [ 1476 ] {} CacheMetadata: Key 6e256354b2b321e4b907132d031b80ef is removed from metadata 2024.10.22 00:39:51.796380 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2ff366c7c7192159cce1813beffaa11e for removal 2024.10.22 00:39:51.800756 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5d892ed614a6761b2eabebcebd1d441d, offset: 0, size: 152 2024.10.22 00:39:51.802333 [ 1476 ] {} CacheMetadata: Key 2ff366c7c7192159cce1813beffaa11e is removed from metadata 2024.10.22 00:39:51.805200 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5d892ed614a6761b2eabebcebd1d441d for removal 2024.10.22 00:39:51.812593 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 41de00bce36a741777b1355d613689a4, offset: 0, size: 36495 2024.10.22 00:39:51.814884 [ 1476 ] {} CacheMetadata: Key 5d892ed614a6761b2eabebcebd1d441d is removed from metadata 2024.10.22 00:39:51.821498 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 41de00bce36a741777b1355d613689a4 for removal 2024.10.22 00:39:51.824297 [ 1476 ] {} CacheMetadata: Key 41de00bce36a741777b1355d613689a4 is removed from metadata 2024.10.22 00:39:51.827208 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9ce766f9f63f3dcadcb945fb14b9512d, offset: 0, size: 156 2024.10.22 00:39:51.833036 [ 1476 ] {} CacheMetadata: Key cd97c1898172e70052c12b5b4303dafd is removed from metadata 2024.10.22 00:39:51.834290 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9ce766f9f63f3dcadcb945fb14b9512d for removal 2024.10.22 00:39:51.840773 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 41ea6749f28568afdd4c83bd32c5d614, offset: 0, size: 44592 2024.10.22 00:39:51.845329 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 41ea6749f28568afdd4c83bd32c5d614 for removal 2024.10.22 00:39:51.850009 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1609c2c459560de994fa84cc8d6d397f, offset: 0, size: 167 2024.10.22 00:39:51.851468 [ 1476 ] {} CacheMetadata: Key 41ea6749f28568afdd4c83bd32c5d614 is removed from metadata 2024.10.22 00:39:51.854784 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1609c2c459560de994fa84cc8d6d397f for removal 2024.10.22 00:39:51.861577 [ 1476 ] {} CacheMetadata: Key 1609c2c459560de994fa84cc8d6d397f is removed from metadata 2024.10.22 00:39:51.861082 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 50fb7ba44879af6055d674bab2ada7fc, offset: 0, size: 167 2024.10.22 00:39:51.865145 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 50fb7ba44879af6055d674bab2ada7fc for removal 2024.10.22 00:39:51.867068 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ab4e5feb42dd5ea55d90a13b6bfcd7ed, offset: 0, size: 99887 2024.10.22 00:39:51.867765 [ 1476 ] {} CacheMetadata: Key 50fb7ba44879af6055d674bab2ada7fc is removed from metadata 2024.10.22 00:39:51.869554 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ab4e5feb42dd5ea55d90a13b6bfcd7ed for removal 2024.10.22 00:39:51.873308 [ 1476 ] {} CacheMetadata: Key 9ce766f9f63f3dcadcb945fb14b9512d is removed from metadata 2024.10.22 00:39:51.876000 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b61659b279041fd2509ef235bac69903, offset: 0, size: 73357 2024.10.22 00:39:51.879618 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b61659b279041fd2509ef235bac69903 for removal 2024.10.22 00:39:51.882588 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 141d821fbff73325d6042cc2f631a2ca, offset: 0, size: 73279 2024.10.22 00:39:51.884558 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 141d821fbff73325d6042cc2f631a2ca for removal 2024.10.22 00:39:51.887359 [ 1476 ] {} CacheMetadata: Key 141d821fbff73325d6042cc2f631a2ca is removed from metadata 2024.10.22 00:39:51.887242 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f1a13cb6291703863e714976948999a9, offset: 0, size: 157 2024.10.22 00:39:51.889182 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f1a13cb6291703863e714976948999a9 for removal 2024.10.22 00:39:51.891482 [ 1476 ] {} CacheMetadata: Key f1a13cb6291703863e714976948999a9 is removed from metadata 2024.10.22 00:39:51.891493 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d9cbb5ba21017e673d477189b5048de5, offset: 0, size: 81804 2024.10.22 00:39:51.899388 [ 1476 ] {} CacheMetadata: Key b61659b279041fd2509ef235bac69903 is removed from metadata 2024.10.22 00:39:51.921006 [ 1476 ] {} CacheMetadata: Key ab4e5feb42dd5ea55d90a13b6bfcd7ed is removed from metadata 2024.10.22 00:39:51.930910 [ 1476 ] {} CacheMetadata: Key 62151225b84628448c2f8b2a5157bdd2 is removed from metadata 2024.10.22 00:39:51.971717 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_30_30_0 (state Temporary) 2024.10.22 00:39:51.997684 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d9cbb5ba21017e673d477189b5048de5 for removal 2024.10.22 00:39:52.001089 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f27cc964a65d47e8673210097cb0cfda, offset: 0, size: 236143 2024.10.22 00:39:52.005344 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f27cc964a65d47e8673210097cb0cfda for removal 2024.10.22 00:39:52.014501 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8384df8584d1b13dd132c72cf0876342, offset: 0, size: 65031 2024.10.22 00:39:52.019791 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8384df8584d1b13dd132c72cf0876342 for removal 2024.10.22 00:39:52.027152 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bc0a6c19e882da547daae8571d665330, offset: 0, size: 157 2024.10.22 00:39:52.031818 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bc0a6c19e882da547daae8571d665330 for removal 2024.10.22 00:39:52.037276 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e991dc5a6a77309e8b5b01ef1272bf15, offset: 0, size: 294167 2024.10.22 00:39:52.038997 [ 1476 ] {} CacheMetadata: Key bc0a6c19e882da547daae8571d665330 is removed from metadata 2024.10.22 00:39:52.046794 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e991dc5a6a77309e8b5b01ef1272bf15 for removal 2024.10.22 00:39:52.049961 [ 1476 ] {} CacheMetadata: Key 8384df8584d1b13dd132c72cf0876342 is removed from metadata 2024.10.22 00:39:52.051449 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2d6469e3ee8ae830175b753b54c1e7ea, offset: 0, size: 169 2024.10.22 00:39:52.056300 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2d6469e3ee8ae830175b753b54c1e7ea for removal 2024.10.22 00:39:52.057713 [ 1476 ] {} CacheMetadata: Key e991dc5a6a77309e8b5b01ef1272bf15 is removed from metadata 2024.10.22 00:39:52.061979 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 351c031f3683af4e661b933646413b6e, offset: 0, size: 156 2024.10.22 00:39:52.067357 [ 1476 ] {} CacheMetadata: Key 2d6469e3ee8ae830175b753b54c1e7ea is removed from metadata 2024.10.22 00:39:52.068784 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 351c031f3683af4e661b933646413b6e for removal 2024.10.22 00:39:52.072483 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: aadb9e6a41bd3186ffaf93b9c187f2d0, offset: 0, size: 1042948 2024.10.22 00:39:52.073424 [ 1476 ] {} CacheMetadata: Key 351c031f3683af4e661b933646413b6e is removed from metadata 2024.10.22 00:39:52.083116 [ 1476 ] {} CacheMetadata: Key f27cc964a65d47e8673210097cb0cfda is removed from metadata 2024.10.22 00:39:52.088664 [ 1476 ] {} CacheMetadata: Key d9cbb5ba21017e673d477189b5048de5 is removed from metadata 2024.10.22 00:39:52.178274 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key aadb9e6a41bd3186ffaf93b9c187f2d0 for removal 2024.10.22 00:39:52.181467 [ 1476 ] {} CacheMetadata: Key aadb9e6a41bd3186ffaf93b9c187f2d0 is removed from metadata 2024.10.22 00:39:52.182573 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b79b53469c1b09158523e114d80f6f82, offset: 0, size: 156 2024.10.22 00:39:52.186084 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b79b53469c1b09158523e114d80f6f82 for removal 2024.10.22 00:39:52.188513 [ 1476 ] {} CacheMetadata: Key b79b53469c1b09158523e114d80f6f82 is removed from metadata 2024.10.22 00:39:52.188602 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7c1967e86b161fbbd11e8130bbdc5356, offset: 0, size: 2846 2024.10.22 00:39:52.191709 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 243 entries to flush up to offset 14090 2024.10.22 00:39:52.194769 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7c1967e86b161fbbd11e8130bbdc5356 for removal 2024.10.22 00:39:52.203425 [ 1476 ] {} CacheMetadata: Key 7c1967e86b161fbbd11e8130bbdc5356 is removed from metadata 2024.10.22 00:39:52.205084 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 33582a661fced7c55727f4a1e578e44e, offset: 0, size: 152 2024.10.22 00:39:52.210461 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 33582a661fced7c55727f4a1e578e44e for removal 2024.10.22 00:39:52.222942 [ 1476 ] {} CacheMetadata: Key 33582a661fced7c55727f4a1e578e44e is removed from metadata 2024.10.22 00:39:52.224462 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 758a58675461e04ddc3540460035edce, offset: 0, size: 156 2024.10.22 00:39:52.232741 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 758a58675461e04ddc3540460035edce for removal 2024.10.22 00:39:52.240178 [ 1476 ] {} CacheMetadata: Key 758a58675461e04ddc3540460035edce is removed from metadata 2024.10.22 00:39:52.241792 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 740e9123c8e8c994a9858eb092875a51, offset: 0, size: 732670 2024.10.22 00:39:52.250394 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 740e9123c8e8c994a9858eb092875a51 for removal 2024.10.22 00:39:52.257544 [ 1476 ] {} CacheMetadata: Key 740e9123c8e8c994a9858eb092875a51 is removed from metadata 2024.10.22 00:39:52.258274 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c049d44724360863a2ddef954699ff57, offset: 0, size: 181 2024.10.22 00:39:52.264599 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c049d44724360863a2ddef954699ff57 for removal 2024.10.22 00:39:52.269273 [ 1476 ] {} CacheMetadata: Key c049d44724360863a2ddef954699ff57 is removed from metadata 2024.10.22 00:39:52.269687 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 06b253bfbce68b05dd45efb9d172c215, offset: 0, size: 168 2024.10.22 00:39:52.373083 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 06b253bfbce68b05dd45efb9d172c215 for removal 2024.10.22 00:39:52.376011 [ 1476 ] {} CacheMetadata: Key 06b253bfbce68b05dd45efb9d172c215 is removed from metadata 2024.10.22 00:39:52.376922 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3105d6c1b439ce3280283882fbe0edf2, offset: 0, size: 27402 2024.10.22 00:39:52.382772 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3105d6c1b439ce3280283882fbe0edf2 for removal 2024.10.22 00:39:52.385322 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c6291d4ab3cb1d572595cad732096f3e, offset: 0, size: 24126 2024.10.22 00:39:52.385816 [ 1476 ] {} CacheMetadata: Key 3105d6c1b439ce3280283882fbe0edf2 is removed from metadata 2024.10.22 00:39:52.388138 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c6291d4ab3cb1d572595cad732096f3e for removal 2024.10.22 00:39:52.390627 [ 1476 ] {} CacheMetadata: Key c6291d4ab3cb1d572595cad732096f3e is removed from metadata 2024.10.22 00:39:52.391005 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2975654172728f34fd8e3f6cdfbe186a, offset: 0, size: 156 2024.10.22 00:39:52.394526 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2975654172728f34fd8e3f6cdfbe186a for removal 2024.10.22 00:39:52.399566 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 94f264e3094b451bb14093efd0fb8ac9, offset: 0, size: 156 2024.10.22 00:39:52.406839 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 94f264e3094b451bb14093efd0fb8ac9 for removal 2024.10.22 00:39:52.413547 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a04c375d603a4fb1d36771e253b2b29c, offset: 0, size: 155 2024.10.22 00:39:52.420948 [ 1476 ] {} CacheMetadata: Key 94f264e3094b451bb14093efd0fb8ac9 is removed from metadata 2024.10.22 00:39:52.421724 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a04c375d603a4fb1d36771e253b2b29c for removal 2024.10.22 00:39:52.430663 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 13bb9aa2db6ec73e0c7f6939875347f9, offset: 0, size: 41515 2024.10.22 00:39:52.438457 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 13bb9aa2db6ec73e0c7f6939875347f9 for removal 2024.10.22 00:39:52.444798 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9d0604e9e7b4ea19afa753ee4d73a416, offset: 0, size: 25282 2024.10.22 00:39:52.447992 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9d0604e9e7b4ea19afa753ee4d73a416 for removal 2024.10.22 00:39:52.452096 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c838464a61277b1bcabfb757587502c4, offset: 0, size: 167 2024.10.22 00:39:52.460821 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c838464a61277b1bcabfb757587502c4 for removal 2024.10.22 00:39:52.465713 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f0779e324125cd5810a0b2e763ae3192, offset: 0, size: 153 2024.10.22 00:39:52.470450 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f0779e324125cd5810a0b2e763ae3192 for removal 2024.10.22 00:39:52.472496 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ea9f822def10d6c09b868d00809e40ae, offset: 0, size: 80934 2024.10.22 00:39:52.477228 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ea9f822def10d6c09b868d00809e40ae for removal 2024.10.22 00:39:52.481023 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 03a946517d588134cbb0debf6b51f7df, offset: 0, size: 139777 2024.10.22 00:39:52.485792 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 03a946517d588134cbb0debf6b51f7df for removal 2024.10.22 00:39:52.489220 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c5014c5fe03d1a8c40b3cb3cbf0465e5, offset: 0, size: 156 2024.10.22 00:39:52.491856 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c5014c5fe03d1a8c40b3cb3cbf0465e5 for removal 2024.10.22 00:39:52.495364 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 643a7c8e9eeb97c7745e00b82121c785, offset: 0, size: 375980 2024.10.22 00:39:52.500159 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 643a7c8e9eeb97c7745e00b82121c785 for removal 2024.10.22 00:39:52.505511 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bfdaea51dd8a8412f59dd3d172d103c7, offset: 0, size: 97770 2024.10.22 00:39:52.510938 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bfdaea51dd8a8412f59dd3d172d103c7 for removal 2024.10.22 00:39:52.514165 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6b87bb9e71613c50b2728dc7f4dc1e2b, offset: 0, size: 195309 2024.10.22 00:39:52.516900 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6b87bb9e71613c50b2728dc7f4dc1e2b for removal 2024.10.22 00:39:52.520450 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b452b56c9e01f919859f677e6cf91b70, offset: 0, size: 191333 2024.10.22 00:39:52.528744 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b452b56c9e01f919859f677e6cf91b70 for removal 2024.10.22 00:39:52.532771 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c32537efc09a8ad4235ddc60e0fe5ab7, offset: 0, size: 156 2024.10.22 00:39:52.536973 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c32537efc09a8ad4235ddc60e0fe5ab7 for removal 2024.10.22 00:39:52.544028 [ 1476 ] {} CacheMetadata: Key c32537efc09a8ad4235ddc60e0fe5ab7 is removed from metadata 2024.10.22 00:39:52.544148 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 33334a4d981f7f74a31dbdce6d0293c0, offset: 0, size: 168 2024.10.22 00:39:52.549113 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 33334a4d981f7f74a31dbdce6d0293c0 for removal 2024.10.22 00:39:52.553195 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c0cbc9c154d08270df940f0beae727d3, offset: 0, size: 189 2024.10.22 00:39:52.559703 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c0cbc9c154d08270df940f0beae727d3 for removal 2024.10.22 00:39:52.574584 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 37b4d67a949d7e9d51ac6c0f4d3ca48e, offset: 0, size: 200916 2024.10.22 00:39:52.591223 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 37b4d67a949d7e9d51ac6c0f4d3ca48e for removal 2024.10.22 00:39:52.595350 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 22be28484ba3cbd602fbb5a824ef9d80, offset: 0, size: 47550 2024.10.22 00:39:52.602783 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 22be28484ba3cbd602fbb5a824ef9d80 for removal 2024.10.22 00:39:52.608030 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f2cd99c132a57dd7a025cceb003fc285, offset: 0, size: 98802 2024.10.22 00:39:52.659014 [ 1476 ] {} CacheMetadata: Key 37b4d67a949d7e9d51ac6c0f4d3ca48e is removed from metadata 2024.10.22 00:39:52.667091 [ 1476 ] {} CacheMetadata: Key c0cbc9c154d08270df940f0beae727d3 is removed from metadata 2024.10.22 00:39:52.673322 [ 1476 ] {} CacheMetadata: Key 33334a4d981f7f74a31dbdce6d0293c0 is removed from metadata 2024.10.22 00:39:52.677422 [ 1476 ] {} CacheMetadata: Key b452b56c9e01f919859f677e6cf91b70 is removed from metadata 2024.10.22 00:39:52.681785 [ 1476 ] {} CacheMetadata: Key bfdaea51dd8a8412f59dd3d172d103c7 is removed from metadata 2024.10.22 00:39:52.686893 [ 1476 ] {} CacheMetadata: Key 643a7c8e9eeb97c7745e00b82121c785 is removed from metadata 2024.10.22 00:39:52.690526 [ 1476 ] {} CacheMetadata: Key 03a946517d588134cbb0debf6b51f7df is removed from metadata 2024.10.22 00:39:52.700166 [ 1476 ] {} CacheMetadata: Key ea9f822def10d6c09b868d00809e40ae is removed from metadata 2024.10.22 00:39:52.713232 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f2cd99c132a57dd7a025cceb003fc285 for removal 2024.10.22 00:39:52.715890 [ 1476 ] {} CacheMetadata: Key f0779e324125cd5810a0b2e763ae3192 is removed from metadata 2024.10.22 00:39:52.719871 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7b833701ee0a8bde9e6a72ee3d9e4f0a, offset: 0, size: 156 2024.10.22 00:39:52.725515 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7b833701ee0a8bde9e6a72ee3d9e4f0a for removal 2024.10.22 00:39:52.733759 [ 1476 ] {} CacheMetadata: Key f2cd99c132a57dd7a025cceb003fc285 is removed from metadata 2024.10.22 00:39:52.834830 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a75b667412ea47f7ac0df05b4e03eaec, offset: 0, size: 137397 2024.10.22 00:39:52.844988 [ 1476 ] {} CacheMetadata: Key 7b833701ee0a8bde9e6a72ee3d9e4f0a is removed from metadata 2024.10.22 00:39:52.847746 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a75b667412ea47f7ac0df05b4e03eaec for removal 2024.10.22 00:39:52.856217 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4816fa35109ff4ce744dd6729f6093ad, offset: 0, size: 2134 2024.10.22 00:39:52.862901 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4816fa35109ff4ce744dd6729f6093ad for removal 2024.10.22 00:39:52.867127 [ 1476 ] {} CacheMetadata: Key a75b667412ea47f7ac0df05b4e03eaec is removed from metadata 2024.10.22 00:39:52.871206 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7a24fb3847bc289a4d9ed3c2700f9557, offset: 0, size: 59379 2024.10.22 00:39:52.875586 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7a24fb3847bc289a4d9ed3c2700f9557 for removal 2024.10.22 00:39:52.877141 [ 1476 ] {} CacheMetadata: Key 4816fa35109ff4ce744dd6729f6093ad is removed from metadata 2024.10.22 00:39:52.883384 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c70fe48f4f613b9396c341bd0a9a1e99, offset: 0, size: 156 2024.10.22 00:39:52.887722 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c70fe48f4f613b9396c341bd0a9a1e99 for removal 2024.10.22 00:39:52.889409 [ 1476 ] {} CacheMetadata: Key 7a24fb3847bc289a4d9ed3c2700f9557 is removed from metadata 2024.10.22 00:39:52.892744 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3b91094f9bafe6003fa949f6c64d6f25, offset: 0, size: 17129 2024.10.22 00:39:52.897503 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3b91094f9bafe6003fa949f6c64d6f25 for removal 2024.10.22 00:39:52.906928 [ 1476 ] {} CacheMetadata: Key 3b91094f9bafe6003fa949f6c64d6f25 is removed from metadata 2024.10.22 00:39:52.907766 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 560b438862e8462a9f43ba65b4c96774, offset: 0, size: 156 2024.10.22 00:39:52.916139 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 560b438862e8462a9f43ba65b4c96774 for removal 2024.10.22 00:39:52.921131 [ 1476 ] {} CacheMetadata: Key c70fe48f4f613b9396c341bd0a9a1e99 is removed from metadata 2024.10.22 00:39:52.922402 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 18d56c579cf32fdbf918241622ec7a0d, offset: 0, size: 156 2024.10.22 00:39:52.926723 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 18d56c579cf32fdbf918241622ec7a0d for removal 2024.10.22 00:39:52.933401 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dfe65364208d5efbd42a8e6294d4f1b9, offset: 0, size: 925497 2024.10.22 00:39:52.933468 [ 1476 ] {} CacheMetadata: Key 18d56c579cf32fdbf918241622ec7a0d is removed from metadata 2024.10.22 00:39:52.944909 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:52.949827 [ 1476 ] {} CacheMetadata: Key c838464a61277b1bcabfb757587502c4 is removed from metadata 2024.10.22 00:39:52.951639 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.94 GiB. 2024.10.22 00:39:52.965798 [ 1476 ] {} CacheMetadata: Key 22be28484ba3cbd602fbb5a824ef9d80 is removed from metadata 2024.10.22 00:39:52.973582 [ 1476 ] {} CacheMetadata: Key c5014c5fe03d1a8c40b3cb3cbf0465e5 is removed from metadata 2024.10.22 00:39:52.982389 [ 1476 ] {} CacheMetadata: Key 9d0604e9e7b4ea19afa753ee4d73a416 is removed from metadata 2024.10.22 00:39:52.988490 [ 1476 ] {} CacheMetadata: Key 6b87bb9e71613c50b2728dc7f4dc1e2b is removed from metadata 2024.10.22 00:39:52.993529 [ 1476 ] {} CacheMetadata: Key 13bb9aa2db6ec73e0c7f6939875347f9 is removed from metadata 2024.10.22 00:39:53.010455 [ 1476 ] {} CacheMetadata: Key a04c375d603a4fb1d36771e253b2b29c is removed from metadata 2024.10.22 00:39:53.020486 [ 1476 ] {} CacheMetadata: Key 560b438862e8462a9f43ba65b4c96774 is removed from metadata 2024.10.22 00:39:53.031216 [ 1476 ] {} CacheMetadata: Key 2975654172728f34fd8e3f6cdfbe186a is removed from metadata 2024.10.22 00:39:53.043424 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dfe65364208d5efbd42a8e6294d4f1b9 for removal 2024.10.22 00:39:53.048430 [ 1476 ] {} CacheMetadata: Key dfe65364208d5efbd42a8e6294d4f1b9 is removed from metadata 2024.10.22 00:39:53.048419 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f0cc74126f915f451e1ba0b781b31c47, offset: 0, size: 220387 2024.10.22 00:39:53.052806 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f0cc74126f915f451e1ba0b781b31c47 for removal 2024.10.22 00:39:53.059647 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c1236d83683deaf0870ae8bf8ccb49e9, offset: 0, size: 3262 2024.10.22 00:39:53.061327 [ 1476 ] {} CacheMetadata: Key f0cc74126f915f451e1ba0b781b31c47 is removed from metadata 2024.10.22 00:39:53.064385 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c1236d83683deaf0870ae8bf8ccb49e9 for removal 2024.10.22 00:39:53.070196 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 393a2f20497d0e5f79f0cf7d5896c083, offset: 0, size: 156 2024.10.22 00:39:53.082318 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 393a2f20497d0e5f79f0cf7d5896c083 for removal 2024.10.22 00:39:53.100041 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8c5937ce5833a2fe5ab2e756cf474c29, offset: 0, size: 153 2024.10.22 00:39:53.115150 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8c5937ce5833a2fe5ab2e756cf474c29 for removal 2024.10.22 00:39:53.122667 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9b6bcd0744cb8c2d6227c58fe92f3e00, offset: 0, size: 2573 2024.10.22 00:39:53.127089 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9b6bcd0744cb8c2d6227c58fe92f3e00 for removal 2024.10.22 00:39:53.135667 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9d6636d60e3a1572f167a6ef11701bc7, offset: 0, size: 37404 2024.10.22 00:39:53.142661 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9d6636d60e3a1572f167a6ef11701bc7 for removal 2024.10.22 00:39:53.147112 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 83f7acf2951564ab24acf8788c094576, offset: 0, size: 155 2024.10.22 00:39:53.160964 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 83f7acf2951564ab24acf8788c094576 for removal 2024.10.22 00:39:53.164913 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 053ebdb417631176c909e387158b6d6d, offset: 0, size: 923031 2024.10.22 00:39:53.169263 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 053ebdb417631176c909e387158b6d6d for removal 2024.10.22 00:39:53.169490 [ 1476 ] {} CacheMetadata: Key 83f7acf2951564ab24acf8788c094576 is removed from metadata 2024.10.22 00:39:53.172568 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a174b70252322c1d9a1b62861f1295ea, offset: 0, size: 165 2024.10.22 00:39:53.176475 [ 1476 ] {} CacheMetadata: Key 053ebdb417631176c909e387158b6d6d is removed from metadata 2024.10.22 00:39:53.177705 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a174b70252322c1d9a1b62861f1295ea for removal 2024.10.22 00:39:53.180701 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7396b03979ffd067b1a375fff8ed6317, offset: 0, size: 169 2024.10.22 00:39:53.183206 [ 1476 ] {} CacheMetadata: Key a174b70252322c1d9a1b62861f1295ea is removed from metadata 2024.10.22 00:39:53.183243 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7396b03979ffd067b1a375fff8ed6317 for removal 2024.10.22 00:39:53.185968 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 63781a22d7e02ced7ff2fe91ba5103b0, offset: 0, size: 922184 2024.10.22 00:39:53.188779 [ 1476 ] {} CacheMetadata: Key 7396b03979ffd067b1a375fff8ed6317 is removed from metadata 2024.10.22 00:39:53.189016 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 63781a22d7e02ced7ff2fe91ba5103b0 for removal 2024.10.22 00:39:53.191751 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f7bc8d44e8fe5a4aa1f44c8718d6c7f9, offset: 0, size: 168 2024.10.22 00:39:53.193793 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f7bc8d44e8fe5a4aa1f44c8718d6c7f9 for removal 2024.10.22 00:39:53.194512 [ 1476 ] {} CacheMetadata: Key 63781a22d7e02ced7ff2fe91ba5103b0 is removed from metadata 2024.10.22 00:39:53.198298 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d29d491c8da1be7e233e1c805ce1cafa, offset: 0, size: 23187 2024.10.22 00:39:53.200206 [ 1476 ] {} CacheMetadata: Key f7bc8d44e8fe5a4aa1f44c8718d6c7f9 is removed from metadata 2024.10.22 00:39:53.201404 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d29d491c8da1be7e233e1c805ce1cafa for removal 2024.10.22 00:39:53.207728 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d055370b8fe02dc5bf575e7152867c34, offset: 0, size: 156 2024.10.22 00:39:53.209583 [ 1476 ] {} CacheMetadata: Key d29d491c8da1be7e233e1c805ce1cafa is removed from metadata 2024.10.22 00:39:53.213183 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d055370b8fe02dc5bf575e7152867c34 for removal 2024.10.22 00:39:53.220445 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_257_257_0 (state Temporary) 2024.10.22 00:39:53.318632 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3caccad52393929d1521bc93a0e91372, offset: 0, size: 151 2024.10.22 00:39:53.323118 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3caccad52393929d1521bc93a0e91372 for removal 2024.10.22 00:39:53.323567 [ 1476 ] {} CacheMetadata: Key d055370b8fe02dc5bf575e7152867c34 is removed from metadata 2024.10.22 00:39:53.346701 [ 1476 ] {} CacheMetadata: Key 3caccad52393929d1521bc93a0e91372 is removed from metadata 2024.10.22 00:39:53.356905 [ 1476 ] {} CacheMetadata: Key 9d6636d60e3a1572f167a6ef11701bc7 is removed from metadata 2024.10.22 00:39:53.408301 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_257_257_0 to 202410_270_270_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:53.469087 [ 1476 ] {} CacheMetadata: Key 9b6bcd0744cb8c2d6227c58fe92f3e00 is removed from metadata 2024.10.22 00:39:53.475198 [ 1476 ] {} CacheMetadata: Key 8c5937ce5833a2fe5ab2e756cf474c29 is removed from metadata 2024.10.22 00:39:53.493195 [ 1476 ] {} CacheMetadata: Key 393a2f20497d0e5f79f0cf7d5896c083 is removed from metadata 2024.10.22 00:39:53.542989 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b894d6e7c722507d936660aa6078f69e, offset: 0, size: 168 2024.10.22 00:39:53.547028 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b894d6e7c722507d936660aa6078f69e for removal 2024.10.22 00:39:53.555281 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 55085481a87c24ef50025907c0a66a50, offset: 0, size: 156 2024.10.22 00:39:53.563526 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 55085481a87c24ef50025907c0a66a50 for removal 2024.10.22 00:39:53.570655 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7c0b3357447e716fc6cdc253cda1d630, offset: 0, size: 94866 2024.10.22 00:39:53.575027 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14090 2024.10.22 00:39:53.576746 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7c0b3357447e716fc6cdc253cda1d630 for removal 2024.10.22 00:39:53.578837 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: eceb51bc08b356c8bb1e2a11f2203b01, offset: 0, size: 155 2024.10.22 00:39:53.580589 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key eceb51bc08b356c8bb1e2a11f2203b01 for removal 2024.10.22 00:39:53.585830 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 110130cb749580ac914f2611c1dd8385, offset: 0, size: 156 2024.10.22 00:39:53.588501 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 110130cb749580ac914f2611c1dd8385 for removal 2024.10.22 00:39:53.607259 [ 1476 ] {} CacheMetadata: Key 110130cb749580ac914f2611c1dd8385 is removed from metadata 2024.10.22 00:39:53.607746 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9aa09a27be00a369360191669970bc1f, offset: 0, size: 156 2024.10.22 00:39:53.612026 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9aa09a27be00a369360191669970bc1f for removal 2024.10.22 00:39:53.615221 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2fb5dff65e8127cf5b1b397fd7f9f8fd, offset: 0, size: 189872 2024.10.22 00:39:53.620494 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2fb5dff65e8127cf5b1b397fd7f9f8fd for removal 2024.10.22 00:39:53.623411 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2af397206bae73efb704d7b40d53f944, offset: 0, size: 48871 2024.10.22 00:39:53.627796 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2af397206bae73efb704d7b40d53f944 for removal 2024.10.22 00:39:53.636308 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7622065069644f2259d86bea9329416b, offset: 0, size: 57912 2024.10.22 00:39:53.644519 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7622065069644f2259d86bea9329416b for removal 2024.10.22 00:39:53.649475 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5efcc56c79a43522e8453202d5bf1ecf, offset: 0, size: 167 2024.10.22 00:39:53.653532 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5efcc56c79a43522e8453202d5bf1ecf for removal 2024.10.22 00:39:53.657561 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 96f8c130da6d847cc671e6ac34a3f07b, offset: 0, size: 162731 2024.10.22 00:39:53.662293 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 96f8c130da6d847cc671e6ac34a3f07b for removal 2024.10.22 00:39:53.732619 [ 1476 ] {} CacheMetadata: Key 96f8c130da6d847cc671e6ac34a3f07b is removed from metadata 2024.10.22 00:39:53.749153 [ 1476 ] {} CacheMetadata: Key 7622065069644f2259d86bea9329416b is removed from metadata 2024.10.22 00:39:53.765277 [ 1476 ] {} CacheMetadata: Key 5efcc56c79a43522e8453202d5bf1ecf is removed from metadata 2024.10.22 00:39:53.775464 [ 1476 ] {} CacheMetadata: Key 2af397206bae73efb704d7b40d53f944 is removed from metadata 2024.10.22 00:39:53.788469 [ 1476 ] {} CacheMetadata: Key 2fb5dff65e8127cf5b1b397fd7f9f8fd is removed from metadata 2024.10.22 00:39:53.803623 [ 1476 ] {} CacheMetadata: Key 9aa09a27be00a369360191669970bc1f is removed from metadata 2024.10.22 00:39:53.817116 [ 1476 ] {} CacheMetadata: Key eceb51bc08b356c8bb1e2a11f2203b01 is removed from metadata 2024.10.22 00:39:53.822913 [ 1476 ] {} CacheMetadata: Key 7c0b3357447e716fc6cdc253cda1d630 is removed from metadata 2024.10.22 00:39:53.829487 [ 1476 ] {} CacheMetadata: Key 55085481a87c24ef50025907c0a66a50 is removed from metadata 2024.10.22 00:39:53.839927 [ 1476 ] {} CacheMetadata: Key b894d6e7c722507d936660aa6078f69e is removed from metadata 2024.10.22 00:39:53.990336 [ 1476 ] {} CacheMetadata: Key c1236d83683deaf0870ae8bf8ccb49e9 is removed from metadata 2024.10.22 00:39:54.192853 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 2105 2024.10.22 00:39:54.449049 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:54.454837 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.93 GiB. 2024.10.22 00:39:54.498705 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_150_150_0 (state Temporary) 2024.10.22 00:39:54.672670 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_150_150_0 to 202410_163_163_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:54.674686 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/bgr/onekilfvpngmqlhsnyflsnagkybfy. 2024.10.22 00:39:54.712162 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2105 2024.10.22 00:39:54.767603 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 138 2024.10.22 00:39:55.166870 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 00:39:55.268073 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:55.271583 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:55.330923 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_120_120_0 (state Temporary) 2024.10.22 00:39:55.404169 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_120_120_0 to 202410_130_130_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:55.416422 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 4.31 MiB, will set to 1.62 GiB (RSS), difference: 3.48 MiB 2024.10.22 00:39:55.582921 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 138 2024.10.22 00:39:56.570176 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.88 GiB, free memory in arenas 4.66 MiB, will set to 1.62 GiB (RSS), difference: 39.23 MiB 2024.10.22 00:39:56.611531 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 30 entries to flush up to offset 9747 2024.10.22 00:39:56.615229 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 249760 2024.10.22 00:39:56.666558 [ 1227 ] {} DatabaseCatalog: Execution took 417 ms. 2024.10.22 00:39:56.915914 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:57.019875 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:57.058516 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_149_149_0 (state Temporary) 2024.10.22 00:39:57.105567 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_149_149_0 to 202410_155_155_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:57.123474 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:39:57.139249 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_30_30_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:39:57.166423 [ 1251 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 4 parts from 202410_1_150_30 to 202410_153_153_0 2024.10.22 00:39:57.178110 [ 1251 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:57.179887 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:57.188314 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9747 2024.10.22 00:39:57.191211 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_153_31} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_150_30 to 202410_153_153_0 into Compact with storage Full 2024.10.22 00:39:57.256054 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_153_31} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:39:57.262227 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_153_31} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_150_30, total 9563 rows starting from the beginning of the part 2024.10.22 00:39:57.272084 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_153_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_151_151_0, total 111 rows starting from the beginning of the part 2024.10.22 00:39:57.282740 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_153_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_152_152_0, total 36 rows starting from the beginning of the part 2024.10.22 00:39:57.295660 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_153_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_153_153_0, total 73 rows starting from the beginning of the part 2024.10.22 00:39:57.331801 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:57.334697 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:57.516755 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_144_144_0 (state Temporary) 2024.10.22 00:39:57.613111 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_144_144_0 to 202410_151_151_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:57.637314 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.88 GiB, free memory in arenas 4.88 MiB, will set to 1.62 GiB (RSS), difference: 15.48 MiB 2024.10.22 00:39:57.707403 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 234 entries to flush up to offset 14324 2024.10.22 00:39:57.749681 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 249760 2024.10.22 00:39:57.758989 [ 1250 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_146_29 to 202410_151_151_0 2024.10.22 00:39:57.775342 [ 1250 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:57.778483 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:57.811411 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_146_29 to 202410_151_151_0 into Compact with storage Full 2024.10.22 00:39:57.853628 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:39:57.866688 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} MergeTreeSequentialSource: Reading 33 marks from part 202410_1_146_29, total 256052 rows starting from the beginning of the part 2024.10.22 00:39:57.881894 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_147_147_0, total 1538 rows starting from the beginning of the part 2024.10.22 00:39:57.926639 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_148_148_0, total 1788 rows starting from the beginning of the part 2024.10.22 00:39:57.962212 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_149_149_0, total 1284 rows starting from the beginning of the part 2024.10.22 00:39:57.996128 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_150_150_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:39:58.128964 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_151_151_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:39:58.287055 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:58.288313 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:58.568399 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_258_258_0 (state Temporary) 2024.10.22 00:39:58.575150 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_153_31} MergeTask::MergeProjectionsStage: Merge sorted 9783 rows, containing 13 columns (13 merged, 0 gathered) in 1.390188845 sec., 7037.17342804603 rows/sec., 2.94 MiB/sec. 2024.10.22 00:39:58.580910 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_153_31/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:39:58.676487 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_258_258_0 to 202410_271_271_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:58.698541 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_153_31} MergedBlockOutputStream: filled checksums 202410_1_153_31 (state Temporary) 2024.10.22 00:39:58.699804 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_266_53 to 202410_271_271_0 2024.10.22 00:39:58.717477 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:39:58.719302 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.90 GiB. 2024.10.22 00:39:58.738754 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_266_53 to 202410_271_271_0 into Compact with storage Full 2024.10.22 00:39:58.825501 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14324 2024.10.22 00:39:58.895209 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_153_31} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_153_31 to 202410_1_153_31 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:39:58.899677 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_153_31} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 4 parts: [202410_1_150_30, 202410_153_153_0] -> 202410_1_153_31 2024.10.22 00:39:58.918626 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:39:58.925038 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_266_53, total 14137 rows starting from the beginning of the part 2024.10.22 00:39:59.063844 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.39 MiB. 2024.10.22 00:39:59.067428 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_267_267_0, total 60 rows starting from the beginning of the part 2024.10.22 00:39:59.116046 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_268_268_0, total 452 rows starting from the beginning of the part 2024.10.22 00:39:59.182311 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_269_269_0, total 29 rows starting from the beginning of the part 2024.10.22 00:39:59.228956 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_270_270_0, total 243 rows starting from the beginning of the part 2024.10.22 00:39:59.288985 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_271_271_0, total 234 rows starting from the beginning of the part 2024.10.22 00:39:59.352240 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_166_33, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0] 2024.10.22 00:39:59.366102 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_166_33, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0] 2024.10.22 00:39:59.423822 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.88 GiB, free memory in arenas 8.23 MiB, will set to 1.62 GiB (RSS), difference: 108.23 MiB 2024.10.22 00:39:59.433604 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_166_33, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0] 2024.10.22 00:39:59.521410 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:39:59.523779 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:40:00.756596 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_30_30_0 to 202410_33_33_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:00.968374 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_151_30/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:40:01.751390 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.88 GiB, free memory in arenas 12.11 MiB, will set to 1.63 GiB (RSS), difference: 12.82 MiB 2024.10.22 00:40:02.084405 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} MergeTask::MergeProjectionsStage: Merge sorted 15155 rows, containing 13 columns (13 merged, 0 gathered) in 3.355201845 sec., 4516.86685335618 rows/sec., 687.42 KiB/sec. 2024.10.22 00:40:02.214006 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 2116 2024.10.22 00:40:02.250985 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} MergedBlockOutputStream: filled checksums 202410_1_271_54 (state Temporary) 2024.10.22 00:40:02.389017 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_271_54 to 202410_1_271_54 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:02.394400 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_271_54} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_266_53, 202410_271_271_0] -> 202410_1_271_54 2024.10.22 00:40:02.445307 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.71 MiB. 2024.10.22 00:40:02.572078 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:02.573652 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.89 GiB. 2024.10.22 00:40:02.713678 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_151_151_0 (state Temporary) 2024.10.22 00:40:02.809407 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_151_151_0 to 202410_164_164_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:02.866363 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Referer 2024.10.22 00:40:02.871917 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2116 2024.10.22 00:40:02.951412 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 14380 2024.10.22 00:40:02.961321 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Referer 2024.10.22 00:40:02.984450 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Referer 2024.10.22 00:40:02.991336 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Referer 2024.10.22 00:40:03.013791 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Referer 2024.10.22 00:40:03.084729 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 139 2024.10.22 00:40:03.127088 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_14_3, 202410_15_15_0, 202410_16_16_0, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0] 2024.10.22 00:40:03.135236 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_14_3, 202410_15_15_0, 202410_16_16_0, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0] 2024.10.22 00:40:03.144718 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Referer 2024.10.22 00:40:03.170069 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Referer 2024.10.22 00:40:03.206369 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ald/bmlzonhgrdbacakfdxkosgvvurtwk (LogSeriesLimiter: on interval from 2024-10-22 00:39:21 to 2024-10-22 00:40:03 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:40:03.280232 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_14_3, 202410_15_15_0, 202410_16_16_0, 202410_17_17_0, 202410_18_18_0, 202410_19_19_0] 2024.10.22 00:40:03.429127 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:03.434804 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.86 GiB. 2024.10.22 00:40:03.578608 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_259_259_0 (state Temporary) 2024.10.22 00:40:03.627027 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:03.633563 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_259_259_0 to 202410_272_272_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:03.634170 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.86 GiB. 2024.10.22 00:40:03.639375 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.88 GiB, free memory in arenas 12.12 MiB, will set to 1.62 GiB (RSS), difference: 22.30 MiB 2024.10.22 00:40:03.665716 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 00:40:03.667109 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1190 2024.10.22 00:40:03.679249 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14380 2024.10.22 00:40:03.716063 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_121_121_0 (state Temporary) 2024.10.22 00:40:03.779910 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_121_121_0 to 202410_131_131_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:03.911713 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 139 2024.10.22 00:40:04.707445 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 96 entries to flush up to offset 9843 2024.10.22 00:40:04.753922 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 251680 2024.10.22 00:40:04.927738 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_78_16, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0, 202410_82_82_0, 202410_83_83_0] 2024.10.22 00:40:04.930279 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_78_16, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0, 202410_82_82_0, 202410_83_83_0] 2024.10.22 00:40:04.944901 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:04.956738 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.86 GiB. 2024.10.22 00:40:04.973367 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_78_16, 202410_79_79_0, 202410_80_80_0, 202410_81_81_0, 202410_82_82_0, 202410_83_83_0] 2024.10.22 00:40:05.423431 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_150_150_0 (state Temporary) 2024.10.22 00:40:05.479247 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_150_150_0 to 202410_156_156_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:05.534564 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9843 2024.10.22 00:40:05.662622 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 13.39 MiB, will set to 1.63 GiB (RSS), difference: -18.42 MiB 2024.10.22 00:40:05.868494 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.082140883 sec. 2024.10.22 00:40:05.898665 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:05.903045 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.86 GiB. 2024.10.22 00:40:06.338294 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_145_145_0 (state Temporary) 2024.10.22 00:40:06.416537 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_145_145_0 to 202410_152_152_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:06.447988 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 251680 2024.10.22 00:40:06.521304 [ 1179 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:40:06.780864 [ 1179 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:40:06.783834 [ 1179 ] {} DNSCacheUpdater: Execution took 263 ms. 2024.10.22 00:40:07.074378 [ 1170 ] {} DatabaseCatalog: Execution took 315 ms. 2024.10.22 00:40:07.605682 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 16.13 MiB, will set to 1.63 GiB (RSS), difference: 3.38 MiB 2024.10.22 00:40:07.689031 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 14429 2024.10.22 00:40:08.041727 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:08.049024 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.85 GiB. 2024.10.22 00:40:08.289618 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_260_260_0 (state Temporary) 2024.10.22 00:40:08.466466 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_260_260_0 to 202410_273_273_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:08.538915 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} MergeTask::MergeProjectionsStage: Merge sorted 263350 rows, containing 4 columns (4 merged, 0 gathered) in 10.742008059 sec., 24515.90043067943 rows/sec., 359.78 KiB/sec. 2024.10.22 00:40:08.822987 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} MergedBlockOutputStream: filled checksums 202410_1_151_30 (state Temporary) 2024.10.22 00:40:08.891842 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_151_30 to 202410_1_151_30 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:08.895372 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_151_30} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_146_29, 202410_151_151_0] -> 202410_1_151_30 2024.10.22 00:40:08.936271 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:40:09.083889 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14429 2024.10.22 00:40:09.521593 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:40:09.524610 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:40:10.452833 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 2136 2024.10.22 00:40:10.564754 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:10.568641 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.85 GiB. 2024.10.22 00:40:10.601108 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_152_152_0 (state Temporary) 2024.10.22 00:40:10.680621 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_152_152_0 to 202410_165_165_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:11.102452 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2136 2024.10.22 00:40:11.228217 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 1230 2024.10.22 00:40:11.418827 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 140 2024.10.22 00:40:11.430543 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 13.93 MiB, will set to 1.64 GiB (RSS), difference: 3.24 MiB 2024.10.22 00:40:11.563578 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:11.616490 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.84 GiB. 2024.10.22 00:40:11.876536 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_122_122_0 (state Temporary) 2024.10.22 00:40:11.935915 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_122_122_0 to 202410_132_132_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:12.036759 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 140 2024.10.22 00:40:12.779404 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 8388608 bytes for a3a42c9839a9416ca2091653b4a3c6fc:0) 2024.10.22 00:40:12.780959 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7a2eb8cea9c17e12aec9d150ec4c29b7, offset: 33554432, size: 33554432 2024.10.22 00:40:13.058863 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 35 entries to flush up to offset 9878 2024.10.22 00:40:13.091780 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 14459 2024.10.22 00:40:13.194405 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:13.200828 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.87 GiB. 2024.10.22 00:40:13.346783 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_261_261_0 (state Temporary) 2024.10.22 00:40:13.375482 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:13.377737 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.87 GiB. 2024.10.22 00:40:13.382230 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_261_261_0 to 202410_274_274_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:13.410159 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_151_151_0 (state Temporary) 2024.10.22 00:40:13.453174 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_151_151_0 to 202410_157_157_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:13.455063 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 253216 2024.10.22 00:40:13.522623 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14459 2024.10.22 00:40:13.553058 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 13.18 MiB, will set to 1.64 GiB (RSS), difference: -17.35 MiB 2024.10.22 00:40:13.579046 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9878 2024.10.22 00:40:14.533502 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:14.537618 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.87 GiB. 2024.10.22 00:40:14.658660 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_146_146_0 (state Temporary) 2024.10.22 00:40:14.729282 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_146_146_0 to 202410_153_153_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:14.765715 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 253216 2024.10.22 00:40:15.771806 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 10.19 MiB, will set to 1.64 GiB (RSS), difference: 6.67 MiB 2024.10.22 00:40:17.237240 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_101_20, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0] 2024.10.22 00:40:17.252045 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_101_20, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0] 2024.10.22 00:40:17.282065 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_101_20, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0] 2024.10.22 00:40:17.506430 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 9.96 MiB, will set to 1.65 GiB (RSS), difference: 6.23 MiB 2024.10.22 00:40:17.526473 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 23 entries to flush up to offset 14482 2024.10.22 00:40:17.752969 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:17.759576 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.86 GiB. 2024.10.22 00:40:18.011384 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_262_262_0 (state Temporary) 2024.10.22 00:40:18.056739 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_262_262_0 to 202410_275_275_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:18.122344 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14482 2024.10.22 00:40:18.475218 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qqa/uvffilwlwxrllldpsvznikrurrauv. 2024.10.22 00:40:18.497687 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_91_18, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0] 2024.10.22 00:40:18.505062 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_91_18, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0] 2024.10.22 00:40:18.605634 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 2142 2024.10.22 00:40:18.620658 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hql/kcpwljkgzmiszsbtygnzxgqswurau. 2024.10.22 00:40:18.749152 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_91_18, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0] 2024.10.22 00:40:18.844039 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 10.13 MiB, will set to 1.65 GiB (RSS), difference: 11.98 MiB 2024.10.22 00:40:18.852109 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:18.858824 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.86 GiB. 2024.10.22 00:40:18.887693 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_153_153_0 (state Temporary) 2024.10.22 00:40:19.087029 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_153_153_0 to 202410_166_166_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:19.325439 [ 1250 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_161_32 to 202410_166_166_0 2024.10.22 00:40:19.346339 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2142 2024.10.22 00:40:19.347468 [ 1250 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:19.351485 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.86 GiB. 2024.10.22 00:40:19.367712 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_161_32 to 202410_166_166_0 into Compact with storage Full 2024.10.22 00:40:19.523444 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:40:19.530923 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:40:19.574355 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hql/kcpwljkgzmiszsbtygnzxgqswurau. 2024.10.22 00:40:19.578283 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_10_10_0 to 201403_10_10_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:19.623058 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:40:19.628799 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:40:19.640433 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_161_32, total 2194 rows starting from the beginning of the part 2024.10.22 00:40:19.674223 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_162_162_0, total 11 rows starting from the beginning of the part 2024.10.22 00:40:19.695313 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qqa/uvffilwlwxrllldpsvznikrurrauv. 2024.10.22 00:40:19.714363 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_163_163_0, total 6 rows starting from the beginning of the part 2024.10.22 00:40:19.761659 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_164_164_0, total 11 rows starting from the beginning of the part 2024.10.22 00:40:19.776969 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_165_165_0, total 20 rows starting from the beginning of the part 2024.10.22 00:40:19.787748 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_166_166_0, total 6 rows starting from the beginning of the part 2024.10.22 00:40:20.471573 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.88 GiB, free memory in arenas 10.33 MiB, will set to 1.58 GiB (RSS), difference: 438.43 MiB 2024.10.22 00:40:21.095297 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 36 entries to flush up to offset 9914 2024.10.22 00:40:21.194487 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} MergeTask::MergeProjectionsStage: Merge sorted 2248 rows, containing 12 columns (12 merged, 0 gathered) in 1.834309427 sec., 1225.5293283187166 rows/sec., 189.98 KiB/sec. 2024.10.22 00:40:21.337033 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:21.341700 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.85 GiB. 2024.10.22 00:40:21.360942 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} MergedBlockOutputStream: filled checksums 202410_1_166_33 (state Temporary) 2024.10.22 00:40:21.381715 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_152_152_0 (state Temporary) 2024.10.22 00:40:21.428117 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_152_152_0 to 202410_158_158_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:21.462718 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 5 parts from 202410_1_153_31 to 202410_157_157_0 2024.10.22 00:40:21.471830 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9914 2024.10.22 00:40:21.478681 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:21.480886 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.85 GiB. 2024.10.22 00:40:21.509778 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_157_32} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_153_31 to 202410_157_157_0 into Compact with storage Full 2024.10.22 00:40:21.564142 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_157_32} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:40:21.573333 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_157_32} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_153_31, total 9783 rows starting from the beginning of the part 2024.10.22 00:40:21.769414 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 255136 2024.10.22 00:40:21.784941 [ 1204 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:40:21.800851 [ 1204 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:40:21.830803 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_166_33 to 202410_1_166_33 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:21.835016 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_166_33} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_161_32, 202410_166_166_0] -> 202410_1_166_33 2024.10.22 00:40:21.844968 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_157_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_154_154_0, total 38 rows starting from the beginning of the part 2024.10.22 00:40:21.890639 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:40:21.943572 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_157_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_155_155_0, total 30 rows starting from the beginning of the part 2024.10.22 00:40:22.099478 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_157_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_156_156_0, total 96 rows starting from the beginning of the part 2024.10.22 00:40:22.124961 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 14541 2024.10.22 00:40:22.199189 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_157_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_157_157_0, total 35 rows starting from the beginning of the part 2024.10.22 00:40:22.276827 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:22.280951 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.85 GiB. 2024.10.22 00:40:22.333549 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_263_263_0 (state Temporary) 2024.10.22 00:40:22.379237 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_263_263_0 to 202410_276_276_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:22.422022 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_271_54 to 202410_276_276_0 2024.10.22 00:40:22.440900 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:22.448737 [ 1231 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.85 GiB. 2024.10.22 00:40:22.473468 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14541 2024.10.22 00:40:22.480203 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_271_54 to 202410_276_276_0 into Compact with storage Full 2024.10.22 00:40:22.554920 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:22.560199 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.84 GiB. 2024.10.22 00:40:22.577544 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:40:22.588732 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_271_54, total 15155 rows starting from the beginning of the part 2024.10.22 00:40:22.599515 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 1.88 GiB, free memory in arenas 8.79 MiB, will set to 1.18 GiB (RSS), difference: -66.01 MiB 2024.10.22 00:40:22.733166 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_147_147_0 (state Temporary) 2024.10.22 00:40:22.769490 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_272_272_0, total 56 rows starting from the beginning of the part 2024.10.22 00:40:22.804584 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_273_273_0, total 49 rows starting from the beginning of the part 2024.10.22 00:40:22.884054 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_274_274_0, total 30 rows starting from the beginning of the part 2024.10.22 00:40:22.909659 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_147_147_0 to 202410_154_154_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:23.004279 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 255136 2024.10.22 00:40:23.027512 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_275_275_0, total 23 rows starting from the beginning of the part 2024.10.22 00:40:23.177238 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_276_276_0, total 59 rows starting from the beginning of the part 2024.10.22 00:40:23.223875 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_171_34, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0] 2024.10.22 00:40:23.227579 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_171_34, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0] 2024.10.22 00:40:23.299685 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_171_34, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0] 2024.10.22 00:40:23.423468 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 10 2024.10.22 00:40:23.698491 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:23.703919 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.85 GiB. 2024.10.22 00:40:23.785306 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_10_10_0 (state Temporary) 2024.10.22 00:40:23.978172 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_10_10_0 to 202410_10_10_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:23.978242 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_157_32} MergeTask::MergeProjectionsStage: Merge sorted 9982 rows, containing 13 columns (13 merged, 0 gathered) in 2.488673767 sec., 4010.971679921276 rows/sec., 1.68 MiB/sec. 2024.10.22 00:40:24.141232 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_157_32} MergedBlockOutputStream: filled checksums 202410_1_157_32 (state Temporary) 2024.10.22 00:40:24.264522 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 10 2024.10.22 00:40:24.518964 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.27 GiB, peak 1.88 GiB, free memory in arenas 5.62 MiB, will set to 1.22 GiB (RSS), difference: -59.21 MiB 2024.10.22 00:40:24.768328 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_157_32} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_157_32 to 202410_1_157_32 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:24.773185 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_157_32} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 5 parts: [202410_1_153_31, 202410_157_157_0] -> 202410_1_157_32 2024.10.22 00:40:24.818425 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.97 MiB. 2024.10.22 00:40:25.069049 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} MergeTask::MergeProjectionsStage: Merge sorted 15372 rows, containing 13 columns (13 merged, 0 gathered) in 2.591711833 sec., 5931.214961582497 rows/sec., 902.64 KiB/sec. 2024.10.22 00:40:25.126302 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} MergedBlockOutputStream: filled checksums 202410_1_276_55 (state Temporary) 2024.10.22 00:40:25.343192 [ 1186 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_128_26 to 202410_132_132_0 2024.10.22 00:40:25.378752 [ 1186 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:25.380467 [ 1186 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.85 GiB. 2024.10.22 00:40:25.391447 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_132_27} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_128_26 to 202410_132_132_0 into Compact with storage Full 2024.10.22 00:40:25.445538 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_132_27} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:40:25.571084 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_276_55 to 202410_1_276_55 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:25.574826 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_276_55} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_271_54, 202410_276_276_0] -> 202410_1_276_55 2024.10.22 00:40:25.585303 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_132_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_128_26, total 172 rows starting from the beginning of the part 2024.10.22 00:40:25.608467 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.03 MiB. 2024.10.22 00:40:25.802397 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_132_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_129_129_0, total 1 rows starting from the beginning of the part 2024.10.22 00:40:26.026446 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_132_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_130_130_0, total 1 rows starting from the beginning of the part 2024.10.22 00:40:26.113622 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_132_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_131_131_0, total 1 rows starting from the beginning of the part 2024.10.22 00:40:26.275820 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_132_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_132_132_0, total 1 rows starting from the beginning of the part 2024.10.22 00:40:26.479522 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 14595 2024.10.22 00:40:26.699682 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:26.708621 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.84 GiB. 2024.10.22 00:40:26.778908 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_264_264_0 (state Temporary) 2024.10.22 00:40:26.849405 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 2168 2024.10.22 00:40:26.916041 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_264_264_0 to 202410_277_277_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:27.028996 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14595 2024.10.22 00:40:27.040711 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 141 2024.10.22 00:40:27.051785 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:27.058973 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.84 GiB. 2024.10.22 00:40:27.130481 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_154_154_0 (state Temporary) 2024.10.22 00:40:27.288555 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_154_154_0 to 202410_167_167_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:27.348158 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2168 2024.10.22 00:40:27.461712 [ 1197 ] {} DatabaseCatalog: Execution took 246 ms. 2024.10.22 00:40:27.471133 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:27.472942 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.85 GiB. 2024.10.22 00:40:27.516873 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_123_123_0 (state Temporary) 2024.10.22 00:40:27.609527 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 1.88 GiB, free memory in arenas 3.04 MiB, will set to 1.26 GiB (RSS), difference: -31.71 MiB 2024.10.22 00:40:27.618355 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_123_123_0 to 202410_133_133_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:27.629316 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_132_27} MergeTask::MergeProjectionsStage: Merge sorted 176 rows, containing 32 columns (32 merged, 0 gathered) in 2.242471342 sec., 78.48483800155516 rows/sec., 8.53 KiB/sec. 2024.10.22 00:40:27.724475 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 141 2024.10.22 00:40:27.775936 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_132_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:40:27.795263 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_132_27} MergedBlockOutputStream: filled checksums 202410_1_132_27 (state Temporary) 2024.10.22 00:40:28.094400 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_132_27} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_132_27 to 202410_1_132_27 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:28.101359 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_132_27} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_128_26, 202410_132_132_0] -> 202410_1_132_27 2024.10.22 00:40:28.150863 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:40:28.976743 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 9998 2024.10.22 00:40:29.283338 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:29.287181 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.85 GiB. 2024.10.22 00:40:29.390086 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:29.394193 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.84 GiB. 2024.10.22 00:40:29.433805 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_153_153_0 (state Temporary) 2024.10.22 00:40:29.526313 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:40:29.529770 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:40:29.609698 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.88 GiB, free memory in arenas 3.08 MiB, will set to 1.31 GiB (RSS), difference: -28.05 MiB 2024.10.22 00:40:29.897176 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_153_153_0 to 202410_159_159_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:30.020655 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1612 entries to flush up to offset 256748 2024.10.22 00:40:30.063403 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 9998 2024.10.22 00:40:31.046408 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 14633 2024.10.22 00:40:31.100497 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:31.112235 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.84 GiB. 2024.10.22 00:40:31.300823 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:31.307712 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.84 GiB. 2024.10.22 00:40:31.343022 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_148_148_0 (state Temporary) 2024.10.22 00:40:31.463844 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_265_265_0 (state Temporary) 2024.10.22 00:40:31.473821 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.88 GiB, free memory in arenas 3.17 MiB, will set to 1.36 GiB (RSS), difference: -36.47 MiB 2024.10.22 00:40:31.507272 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_265_265_0 to 202410_278_278_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:31.515309 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_148_148_0 to 202410_155_155_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:31.566078 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 256748 2024.10.22 00:40:31.571027 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14633 2024.10.22 00:40:31.744917 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_95_19, 202410_96_96_0, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0] 2024.10.22 00:40:31.752039 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_95_19, 202410_96_96_0, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0] 2024.10.22 00:40:31.776690 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_95_19, 202410_96_96_0, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0] 2024.10.22 00:40:32.747258 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_31_31_0 (state Temporary) 2024.10.22 00:40:33.168923 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:40:33.287074 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_31_31_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:40:33.394942 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 2.51 MiB, will set to 1.40 GiB (RSS), difference: -30.29 MiB 2024.10.22 00:40:34.857698 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 2180 2024.10.22 00:40:35.080268 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:35.083284 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.84 GiB. 2024.10.22 00:40:35.125075 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_155_155_0 (state Temporary) 2024.10.22 00:40:35.137079 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 33 file segments (while reserving 16777216 bytes for a3a42c9839a9416ca2091653b4a3c6fc:33554432) 2024.10.22 00:40:35.146830 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a3a42c9839a9416ca2091653b4a3c6fc, offset: 0, size: 33554432 2024.10.22 00:40:35.173201 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 37c7f550952df17a517de9c9689e2b86, offset: 0, size: 154 2024.10.22 00:40:35.176995 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 37c7f550952df17a517de9c9689e2b86 for removal 2024.10.22 00:40:35.180495 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: af0d8031c220377988dd519a037b8efd, offset: 0, size: 32797 2024.10.22 00:40:35.181589 [ 1476 ] {} CacheMetadata: Key 37c7f550952df17a517de9c9689e2b86 is removed from metadata 2024.10.22 00:40:35.186684 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key af0d8031c220377988dd519a037b8efd for removal 2024.10.22 00:40:35.190460 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 670abfebb390ab3dc2cc04952b61399f, offset: 0, size: 154 2024.10.22 00:40:35.195511 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 670abfebb390ab3dc2cc04952b61399f for removal 2024.10.22 00:40:35.201696 [ 1476 ] {} CacheMetadata: Key af0d8031c220377988dd519a037b8efd is removed from metadata 2024.10.22 00:40:35.205758 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e8e4a65ed5157e633f8acda0cc226d25, offset: 0, size: 164 2024.10.22 00:40:35.212566 [ 1476 ] {} CacheMetadata: Key 670abfebb390ab3dc2cc04952b61399f is removed from metadata 2024.10.22 00:40:35.215141 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e8e4a65ed5157e633f8acda0cc226d25 for removal 2024.10.22 00:40:35.220458 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5486583e922ccc42ecb1e24199f0f2c0, offset: 0, size: 64848 2024.10.22 00:40:35.225231 [ 1476 ] {} CacheMetadata: Key e8e4a65ed5157e633f8acda0cc226d25 is removed from metadata 2024.10.22 00:40:35.226373 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5486583e922ccc42ecb1e24199f0f2c0 for removal 2024.10.22 00:40:35.227682 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 142 2024.10.22 00:40:35.235563 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 38280ab7d994610760caf3e03b0f632e, offset: 0, size: 163 2024.10.22 00:40:35.248050 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 38280ab7d994610760caf3e03b0f632e for removal 2024.10.22 00:40:35.260636 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 765d7e776106e46cdc1337ebc2d0fb54, offset: 0, size: 47078 2024.10.22 00:40:35.267763 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 765d7e776106e46cdc1337ebc2d0fb54 for removal 2024.10.22 00:40:35.272926 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0ce2b1796bd7b9637048e243b7b4cc26, offset: 0, size: 39547 2024.10.22 00:40:35.275415 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0ce2b1796bd7b9637048e243b7b4cc26 for removal 2024.10.22 00:40:35.277288 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8662df0510845a93e527a9a2795c5605, offset: 0, size: 154 2024.10.22 00:40:35.292585 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8662df0510845a93e527a9a2795c5605 for removal 2024.10.22 00:40:35.302536 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7076c65e4367f9bef97f87df4c0732be, offset: 0, size: 144 2024.10.22 00:40:35.311869 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7076c65e4367f9bef97f87df4c0732be for removal 2024.10.22 00:40:35.312711 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_155_155_0 to 202410_168_168_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:35.352822 [ 1476 ] {} CacheMetadata: Key 8662df0510845a93e527a9a2795c5605 is removed from metadata 2024.10.22 00:40:35.420845 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e01c5630ca548a931c676a9429c9d5ab, offset: 0, size: 202898 2024.10.22 00:40:35.423584 [ 1476 ] {} CacheMetadata: Key 7076c65e4367f9bef97f87df4c0732be is removed from metadata 2024.10.22 00:40:35.426599 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e01c5630ca548a931c676a9429c9d5ab for removal 2024.10.22 00:40:35.437772 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: afd729377ed3633b9d54ea2f6a060122, offset: 0, size: 176 2024.10.22 00:40:35.440271 [ 1476 ] {} CacheMetadata: Key e01c5630ca548a931c676a9429c9d5ab is removed from metadata 2024.10.22 00:40:35.446774 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key afd729377ed3633b9d54ea2f6a060122 for removal 2024.10.22 00:40:35.450739 [ 1476 ] {} CacheMetadata: Key 0ce2b1796bd7b9637048e243b7b4cc26 is removed from metadata 2024.10.22 00:40:35.460170 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8a075f71ca3ae2674541cf482b9d460b, offset: 0, size: 14190 2024.10.22 00:40:35.462739 [ 1476 ] {} CacheMetadata: Key afd729377ed3633b9d54ea2f6a060122 is removed from metadata 2024.10.22 00:40:35.465092 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2180 2024.10.22 00:40:35.468494 [ 1476 ] {} CacheMetadata: Key 765d7e776106e46cdc1337ebc2d0fb54 is removed from metadata 2024.10.22 00:40:35.473022 [ 1476 ] {} CacheMetadata: Key 38280ab7d994610760caf3e03b0f632e is removed from metadata 2024.10.22 00:40:35.481112 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:35.486992 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.88 GiB. 2024.10.22 00:40:35.488544 [ 1476 ] {} CacheMetadata: Key 5486583e922ccc42ecb1e24199f0f2c0 is removed from metadata 2024.10.22 00:40:35.562073 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_124_124_0 (state Temporary) 2024.10.22 00:40:35.564639 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8a075f71ca3ae2674541cf482b9d460b for removal 2024.10.22 00:40:35.571445 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4181d4ebd829ba88a2b00834c24a52ac, offset: 0, size: 142 2024.10.22 00:40:35.574689 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4181d4ebd829ba88a2b00834c24a52ac for removal 2024.10.22 00:40:35.581931 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bf4a60cb9db9b3be226703796690f208, offset: 0, size: 154 2024.10.22 00:40:35.587537 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bf4a60cb9db9b3be226703796690f208 for removal 2024.10.22 00:40:35.592623 [ 1476 ] {} CacheMetadata: Key 4181d4ebd829ba88a2b00834c24a52ac is removed from metadata 2024.10.22 00:40:35.595391 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 22bfd7629d1f6bb6406b7bb95b0629f0, offset: 0, size: 17234 2024.10.22 00:40:35.600954 [ 1476 ] {} CacheMetadata: Key bf4a60cb9db9b3be226703796690f208 is removed from metadata 2024.10.22 00:40:35.603745 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 22bfd7629d1f6bb6406b7bb95b0629f0 for removal 2024.10.22 00:40:35.612298 [ 1476 ] {} CacheMetadata: Key 8a075f71ca3ae2674541cf482b9d460b is removed from metadata 2024.10.22 00:40:35.620398 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 396320a5575db36dc79dad0c1e21669e, offset: 0, size: 160 2024.10.22 00:40:35.621740 [ 1476 ] {} CacheMetadata: Key 22bfd7629d1f6bb6406b7bb95b0629f0 is removed from metadata 2024.10.22 00:40:35.626482 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 396320a5575db36dc79dad0c1e21669e for removal 2024.10.22 00:40:35.634566 [ 1476 ] {} CacheMetadata: Key 396320a5575db36dc79dad0c1e21669e is removed from metadata 2024.10.22 00:40:35.633454 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0867eae6ebf3c35ec396d7f255011e3d, offset: 0, size: 142865 2024.10.22 00:40:35.642374 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0867eae6ebf3c35ec396d7f255011e3d for removal 2024.10.22 00:40:35.652760 [ 1476 ] {} CacheMetadata: Key 0867eae6ebf3c35ec396d7f255011e3d is removed from metadata 2024.10.22 00:40:35.653078 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2044f33ca6a2cd7e909e19860c15efb2, offset: 0, size: 2237842 2024.10.22 00:40:35.659796 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2044f33ca6a2cd7e909e19860c15efb2 for removal 2024.10.22 00:40:35.666122 [ 1476 ] {} CacheMetadata: Key 2044f33ca6a2cd7e909e19860c15efb2 is removed from metadata 2024.10.22 00:40:35.666149 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 582f8e4faed5955e91f9c15fe5b67625, offset: 0, size: 164 2024.10.22 00:40:35.670878 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 582f8e4faed5955e91f9c15fe5b67625 for removal 2024.10.22 00:40:35.675165 [ 1476 ] {} CacheMetadata: Key 582f8e4faed5955e91f9c15fe5b67625 is removed from metadata 2024.10.22 00:40:35.678642 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 69679831f50bedb68ea2c15e0692b24e, offset: 0, size: 154 2024.10.22 00:40:35.682727 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 91 entries to flush up to offset 14724 2024.10.22 00:40:35.687716 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 69679831f50bedb68ea2c15e0692b24e for removal 2024.10.22 00:40:35.691940 [ 1476 ] {} CacheMetadata: Key 69679831f50bedb68ea2c15e0692b24e is removed from metadata 2024.10.22 00:40:35.692198 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2b2376cd5e6cabe538260b7374eea317, offset: 0, size: 1102137 2024.10.22 00:40:35.694880 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.88 GiB, free memory in arenas 2.54 MiB, will set to 1.44 GiB (RSS), difference: -38.02 MiB 2024.10.22 00:40:35.698263 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2b2376cd5e6cabe538260b7374eea317 for removal 2024.10.22 00:40:35.709168 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 61df3bbc11aa06c85e3531e7761d07fa, offset: 0, size: 146 2024.10.22 00:40:35.718996 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 61df3bbc11aa06c85e3531e7761d07fa for removal 2024.10.22 00:40:35.732745 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0c40beeb368fb08ed6ccbd6fa8de0f8b, offset: 0, size: 154 2024.10.22 00:40:35.739448 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0c40beeb368fb08ed6ccbd6fa8de0f8b for removal 2024.10.22 00:40:35.750348 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: df4086f4be6f5e7d1460b91cb39b039d, offset: 0, size: 92334 2024.10.22 00:40:35.759456 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key df4086f4be6f5e7d1460b91cb39b039d for removal 2024.10.22 00:40:35.766285 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 255e3793035ad1656eac49152e050ea1, offset: 0, size: 34892 2024.10.22 00:40:35.769577 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 255e3793035ad1656eac49152e050ea1 for removal 2024.10.22 00:40:35.772520 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 96be2d6aa3d5d1c4298a2f4b8a82a55f, offset: 0, size: 28036 2024.10.22 00:40:35.777386 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 96be2d6aa3d5d1c4298a2f4b8a82a55f for removal 2024.10.22 00:40:35.782187 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9f076bbd145b46863a4609a46ab7a3c7, offset: 0, size: 164 2024.10.22 00:40:35.786650 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9f076bbd145b46863a4609a46ab7a3c7 for removal 2024.10.22 00:40:35.791453 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 19e43f6335e313791c00c2b2cdb45d86, offset: 0, size: 2509 2024.10.22 00:40:35.795711 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 19e43f6335e313791c00c2b2cdb45d86 for removal 2024.10.22 00:40:35.808489 [ 1476 ] {} CacheMetadata: Key 19e43f6335e313791c00c2b2cdb45d86 is removed from metadata 2024.10.22 00:40:35.816763 [ 1476 ] {} CacheMetadata: Key 9f076bbd145b46863a4609a46ab7a3c7 is removed from metadata 2024.10.22 00:40:35.831106 [ 1476 ] {} CacheMetadata: Key 96be2d6aa3d5d1c4298a2f4b8a82a55f is removed from metadata 2024.10.22 00:40:35.844152 [ 1476 ] {} CacheMetadata: Key 255e3793035ad1656eac49152e050ea1 is removed from metadata 2024.10.22 00:40:35.852784 [ 1476 ] {} CacheMetadata: Key df4086f4be6f5e7d1460b91cb39b039d is removed from metadata 2024.10.22 00:40:35.897086 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_124_124_0 to 202410_134_134_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:35.912255 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 84047fa3f645eafcb9f8b360adbfb506, offset: 0, size: 208 2024.10.22 00:40:35.964108 [ 1476 ] {} CacheMetadata: Key 0c40beeb368fb08ed6ccbd6fa8de0f8b is removed from metadata 2024.10.22 00:40:36.008299 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 84047fa3f645eafcb9f8b360adbfb506 for removal 2024.10.22 00:40:36.012401 [ 1476 ] {} CacheMetadata: Key 61df3bbc11aa06c85e3531e7761d07fa is removed from metadata 2024.10.22 00:40:36.016904 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 840f22a79e1bf7bf7e092caa1f9703fb, offset: 0, size: 62909 2024.10.22 00:40:36.022505 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 840f22a79e1bf7bf7e092caa1f9703fb for removal 2024.10.22 00:40:36.026809 [ 1476 ] {} CacheMetadata: Key 84047fa3f645eafcb9f8b360adbfb506 is removed from metadata 2024.10.22 00:40:36.031285 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3c098150b072759a0dd323bc273c5784, offset: 0, size: 4 2024.10.22 00:40:36.034969 [ 1476 ] {} CacheMetadata: Key 840f22a79e1bf7bf7e092caa1f9703fb is removed from metadata 2024.10.22 00:40:36.035396 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3c098150b072759a0dd323bc273c5784 for removal 2024.10.22 00:40:36.047812 [ 1476 ] {} CacheMetadata: Key 3c098150b072759a0dd323bc273c5784 is removed from metadata 2024.10.22 00:40:36.067800 [ 1476 ] {} CacheMetadata: Key 2b2376cd5e6cabe538260b7374eea317 is removed from metadata 2024.10.22 00:40:36.144062 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:36.150986 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.86 GiB. 2024.10.22 00:40:36.174442 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/ald/bmlzonhgrdbacakfdxkosgvvurtwk, upload id 8921de21-1d62-418e-a7cf-780809c64759 2024.10.22 00:40:36.199559 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 142 2024.10.22 00:40:36.362156 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_266_266_0 (state Temporary) 2024.10.22 00:40:36.611720 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_266_266_0 to 202410_279_279_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:36.661934 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14724 2024.10.22 00:40:36.812594 [ 1225 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:40:36.929813 [ 1225 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:40:37.024519 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 262.61 MiB using storage policy from min volume index 0 2024.10.22 00:40:37.605610 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 34 entries to flush up to offset 10032 2024.10.22 00:40:37.656049 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 2.05 MiB, will set to 1.43 GiB (RSS), difference: 22.64 MiB 2024.10.22 00:40:37.783492 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yea/ckaywwgvtguglvzonslximtbmmrlk (LogSeriesLimiter: on interval from 2024-10-22 00:40:03 to 2024-10-22 00:40:37 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:40:37.786287 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:37.788188 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.77 GiB. 2024.10.22 00:40:37.844685 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_154_154_0 (state Temporary) 2024.10.22 00:40:37.879810 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_154_154_0 to 202410_160_160_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:38.030395 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10032 2024.10.22 00:40:38.572184 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1844 entries to flush up to offset 258592 2024.10.22 00:40:38.962861 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_31_31_0 to 202410_34_34_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:38.993724 [ 1252 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_29_6 to 202410_34_34_0 2024.10.22 00:40:39.015841 [ 1252 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:39.021880 [ 1252 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.77 GiB. 2024.10.22 00:40:39.058680 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_29_6 to 202410_34_34_0 into Compact with storage Full 2024.10.22 00:40:39.527743 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:40:39.529486 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.88 GiB, free memory in arenas 2.46 MiB, will set to 1.39 GiB (RSS), difference: -195.13 MiB 2024.10.22 00:40:39.531437 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:40:39.785290 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:39.787159 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.77 GiB. 2024.10.22 00:40:40.032971 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_149_149_0 (state Temporary) 2024.10.22 00:40:40.187774 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_149_149_0 to 202410_156_156_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:40.229127 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 258592 2024.10.22 00:40:40.230277 [ 1260 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_151_30 to 202410_156_156_0 2024.10.22 00:40:40.240453 [ 1260 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:40.244156 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.77 GiB. 2024.10.22 00:40:40.370290 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_151_30 to 202410_156_156_0 into Compact with storage Full 2024.10.22 00:40:40.489008 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:40:40.496771 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} MergeTreeSequentialSource: Reading 34 marks from part 202410_1_151_30, total 263350 rows starting from the beginning of the part 2024.10.22 00:40:40.510436 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_152_152_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:40:40.519203 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_153_153_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:40:40.526540 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_154_154_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:40:40.534714 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_155_155_0, total 1612 rows starting from the beginning of the part 2024.10.22 00:40:40.542957 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_156_156_0, total 1844 rows starting from the beginning of the part 2024.10.22 00:40:40.711612 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 14806 2024.10.22 00:40:41.241915 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:41.245640 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.77 GiB. 2024.10.22 00:40:41.288999 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_267_267_0 (state Temporary) 2024.10.22 00:40:41.339818 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_267_267_0 to 202410_280_280_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:41.371798 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_156_31/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:40:41.428159 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14806 2024.10.22 00:40:41.696124 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.88 GiB, free memory in arenas 5.34 MiB, will set to 1.39 GiB (RSS), difference: -149.44 MiB 2024.10.22 00:40:41.771360 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1230 2024.10.22 00:40:42.024007 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fzt/nyzlhfndbahircpdmxtaihhndodtj (LogSeriesLimiter: on interval from 2024-10-22 00:40:37 to 2024-10-22 00:40:42 accepted series 1 / 214 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:40:42.136616 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column URLDomain 2024.10.22 00:40:42.144670 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column URLDomain 2024.10.22 00:40:42.174453 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column URLDomain 2024.10.22 00:40:42.179290 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column URLDomain 2024.10.22 00:40:42.188324 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column URLDomain 2024.10.22 00:40:42.210183 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column URLDomain 2024.10.22 00:40:42.231004 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column URLDomain 2024.10.22 00:40:42.966832 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 2187 2024.10.22 00:40:43.140203 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:43.143898 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:43.182189 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_156_156_0 (state Temporary) 2024.10.22 00:40:43.250400 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_156_156_0 to 202410_169_169_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:43.385512 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2187 2024.10.22 00:40:43.508310 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.88 GiB, free memory in arenas 12.32 MiB, will set to 1.38 GiB (RSS), difference: -83.88 MiB 2024.10.22 00:40:43.703794 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 143 2024.10.22 00:40:43.871358 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:43.874216 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:43.925109 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_125_125_0 (state Temporary) 2024.10.22 00:40:44.176854 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_125_125_0 to 202410_135_135_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:44.329558 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 143 2024.10.22 00:40:45.430114 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.88 GiB, free memory in arenas 13.15 MiB, will set to 1.39 GiB (RSS), difference: -143.40 MiB 2024.10.22 00:40:45.441636 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 14837 2024.10.22 00:40:45.539598 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 55 entries to flush up to offset 10087 2024.10.22 00:40:45.744508 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:45.752093 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:45.790287 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:45.796039 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:45.835553 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_268_268_0 (state Temporary) 2024.10.22 00:40:45.886958 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_268_268_0 to 202410_281_281_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:45.907678 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_155_155_0 (state Temporary) 2024.10.22 00:40:45.924657 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 5 parts from 202410_1_276_55 to 202410_280_280_0 2024.10.22 00:40:45.937030 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:45.941519 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:45.954286 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_155_155_0 to 202410_161_161_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:45.967005 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_280_56} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_276_55 to 202410_280_280_0 into Compact with storage Full 2024.10.22 00:40:45.997262 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10087 2024.10.22 00:40:46.015192 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_280_56} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:40:46.046025 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_280_56} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_276_55, total 15372 rows starting from the beginning of the part 2024.10.22 00:40:46.065410 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14837 2024.10.22 00:40:46.210117 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_280_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_277_277_0, total 54 rows starting from the beginning of the part 2024.10.22 00:40:46.263570 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_280_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_278_278_0, total 38 rows starting from the beginning of the part 2024.10.22 00:40:46.390978 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_280_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_279_279_0, total 91 rows starting from the beginning of the part 2024.10.22 00:40:46.458676 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_280_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_280_280_0, total 82 rows starting from the beginning of the part 2024.10.22 00:40:46.975078 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:40:47.235422 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 260128 2024.10.22 00:40:47.757805 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_280_56/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:40:47.757805 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 12.71 MiB, will set to 1.40 GiB (RSS), difference: 1.94 MiB 2024.10.22 00:40:47.824421 [ 1168 ] {} DatabaseCatalog: Execution took 212 ms. 2024.10.22 00:40:47.833491 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_280_56} MergeTask::MergeProjectionsStage: Merge sorted 15637 rows, containing 13 columns (13 merged, 0 gathered) in 1.871782553 sec., 8354.0686790449 rows/sec., 1.24 MiB/sec. 2024.10.22 00:40:47.892152 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_176_35, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0] 2024.10.22 00:40:47.905638 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_176_35, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0] 2024.10.22 00:40:47.948149 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_176_35, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0] 2024.10.22 00:40:48.072939 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_280_56} MergedBlockOutputStream: filled checksums 202410_1_280_56 (state Temporary) 2024.10.22 00:40:48.174569 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:48.177243 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:48.264174 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_29_6, total 1087 rows starting from the beginning of the part 2024.10.22 00:40:48.352111 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_280_56} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_280_56 to 202410_1_280_56 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:48.378624 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_280_56} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 5 parts: [202410_1_276_55, 202410_280_280_0] -> 202410_1_280_56 2024.10.22 00:40:48.413574 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.99 MiB. 2024.10.22 00:40:48.426779 [ 1187 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 00:40:48.644254 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_150_150_0 (state Temporary) 2024.10.22 00:40:49.210434 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_150_150_0 to 202410_157_157_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:49.367257 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 260128 2024.10.22 00:40:49.418820 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 1268 2024.10.22 00:40:49.527882 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:40:49.530318 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:40:49.560009 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 14.91 MiB, will set to 1.41 GiB (RSS), difference: 4.72 MiB 2024.10.22 00:40:50.068972 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 14883 2024.10.22 00:40:50.368645 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:50.374797 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.74 GiB. 2024.10.22 00:40:50.593823 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_269_269_0 (state Temporary) 2024.10.22 00:40:50.687049 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} MergeTask::MergeProjectionsStage: Merge sorted 272182 rows, containing 4 columns (4 merged, 0 gathered) in 10.437006299 sec., 26078.55089884142 rows/sec., 382.84 KiB/sec. 2024.10.22 00:40:50.731762 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_269_269_0 to 202410_282_282_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:50.863724 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 00:40:50.887097 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 2199 2024.10.22 00:40:50.912779 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14883 2024.10.22 00:40:50.960003 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} MergedBlockOutputStream: filled checksums 202410_1_156_31 (state Temporary) 2024.10.22 00:40:51.095921 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_156_31 to 202410_1_156_31 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:51.103852 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_156_31} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_151_30, 202410_156_156_0] -> 202410_1_156_31 2024.10.22 00:40:51.115222 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:51.118846 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:51.145396 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:40:51.271939 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_30_30_0, total 45 rows starting from the beginning of the part 2024.10.22 00:40:51.363251 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_157_157_0 (state Temporary) 2024.10.22 00:40:51.428377 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_157_157_0 to 202410_170_170_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:51.489754 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.88 GiB, free memory in arenas 13.85 MiB, will set to 1.41 GiB (RSS), difference: 3.46 MiB 2024.10.22 00:40:51.531436 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2199 2024.10.22 00:40:51.832509 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 144 2024.10.22 00:40:52.046608 [ 1208 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:40:52.103120 [ 1208 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:40:52.185660 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:52.290906 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:52.462899 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_126_126_0 (state Temporary) 2024.10.22 00:40:52.547760 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_126_126_0 to 202410_136_136_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:52.626262 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 144 2024.10.22 00:40:53.088418 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_31_31_0, total 41 rows starting from the beginning of the part 2024.10.22 00:40:53.310407 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.88 GiB, free memory in arenas 12.10 MiB, will set to 1.42 GiB (RSS), difference: 4.36 MiB 2024.10.22 00:40:53.515153 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 64 entries to flush up to offset 10151 2024.10.22 00:40:53.812057 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:53.815841 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:53.864339 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_156_156_0 (state Temporary) 2024.10.22 00:40:54.210809 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_156_156_0 to 202410_162_162_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:54.274718 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10151 2024.10.22 00:40:54.285139 [ 1230 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_157_32 to 202410_162_162_0 2024.10.22 00:40:54.306686 [ 1230 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:54.415010 [ 1230 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:54.577834 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_157_32 to 202410_162_162_0 into Compact with storage Full 2024.10.22 00:40:54.611709 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_32_32_0, total 40 rows starting from the beginning of the part 2024.10.22 00:40:54.670549 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:40:54.705218 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_157_32, total 9982 rows starting from the beginning of the part 2024.10.22 00:40:54.775027 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_158_158_0, total 36 rows starting from the beginning of the part 2024.10.22 00:40:54.790645 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_83_17, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0, 202410_87_87_0, 202410_88_88_0] 2024.10.22 00:40:54.792213 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_83_17, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0, 202410_87_87_0, 202410_88_88_0] 2024.10.22 00:40:54.818479 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_159_159_0, total 84 rows starting from the beginning of the part 2024.10.22 00:40:54.830066 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_83_17, 202410_84_84_0, 202410_85_85_0, 202410_86_86_0, 202410_87_87_0, 202410_88_88_0] 2024.10.22 00:40:54.929380 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 14931 2024.10.22 00:40:54.982423 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_160_160_0, total 34 rows starting from the beginning of the part 2024.10.22 00:40:55.045212 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_161_161_0, total 55 rows starting from the beginning of the part 2024.10.22 00:40:55.105026 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_162_162_0, total 64 rows starting from the beginning of the part 2024.10.22 00:40:55.280527 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:55.284963 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:55.343641 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_270_270_0 (state Temporary) 2024.10.22 00:40:55.432707 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 10.15 MiB, will set to 1.42 GiB (RSS), difference: -2.87 MiB 2024.10.22 00:40:55.601040 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_270_270_0 to 202410_283_283_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:55.675176 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14931 2024.10.22 00:40:56.371824 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 262048 2024.10.22 00:40:56.389608 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_33_33_0, total 39 rows starting from the beginning of the part 2024.10.22 00:40:56.884253 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} MergeTask::MergeProjectionsStage: Merge sorted 10255 rows, containing 13 columns (13 merged, 0 gathered) in 2.311354976 sec., 4436.791451976435 rows/sec., 1.86 MiB/sec. 2024.10.22 00:40:57.082384 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} MergedBlockOutputStream: filled checksums 202410_1_162_33 (state Temporary) 2024.10.22 00:40:57.318823 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_162_33 to 202410_1_162_33 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:57.322940 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_162_33} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_157_32, 202410_162_162_0] -> 202410_1_162_33 2024.10.22 00:40:57.372117 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.03 MiB. 2024.10.22 00:40:57.396143 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:57.400742 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:57.511123 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 9.18 MiB, will set to 1.43 GiB (RSS), difference: 12.60 MiB 2024.10.22 00:40:57.559019 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_151_151_0 (state Temporary) 2024.10.22 00:40:57.647250 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_151_151_0 to 202410_158_158_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:57.693767 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 262048 2024.10.22 00:40:57.984138 [ 1234 ] {} DatabaseCatalog: Execution took 211 ms. 2024.10.22 00:40:58.177562 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_34_34_0, total 40 rows starting from the beginning of the part 2024.10.22 00:40:58.583898 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 8.95 MiB, will set to 1.43 GiB (RSS), difference: 4.67 MiB 2024.10.22 00:40:59.041681 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 2218 2024.10.22 00:40:59.269408 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:59.273520 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:59.404779 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_158_158_0 (state Temporary) 2024.10.22 00:40:59.530582 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:40:59.535130 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:40:59.587478 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_158_158_0 to 202410_171_171_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:40:59.641266 [ 1247 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_166_33 to 202410_171_171_0 2024.10.22 00:40:59.649408 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2218 2024.10.22 00:40:59.658726 [ 1247 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:59.663037 [ 1247 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:59.679754 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 37 entries to flush up to offset 14968 2024.10.22 00:40:59.711276 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_166_33 to 202410_171_171_0 into Compact with storage Full 2024.10.22 00:40:59.900135 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_106_21, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0] 2024.10.22 00:40:59.907012 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_106_21, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0] 2024.10.22 00:40:59.908634 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:40:59.922940 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_166_33, total 2248 rows starting from the beginning of the part 2024.10.22 00:40:59.932905 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:40:59.936079 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:40:59.956323 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_106_21, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0] 2024.10.22 00:41:00.079080 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_167_167_0, total 26 rows starting from the beginning of the part 2024.10.22 00:41:00.129138 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 145 2024.10.22 00:41:00.190515 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_271_271_0 (state Temporary) 2024.10.22 00:41:00.214423 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_168_168_0, total 12 rows starting from the beginning of the part 2024.10.22 00:41:00.274564 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_169_169_0, total 7 rows starting from the beginning of the part 2024.10.22 00:41:00.326052 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_170_170_0, total 12 rows starting from the beginning of the part 2024.10.22 00:41:00.350894 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_271_271_0 to 202410_284_284_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:00.369511 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_171_171_0, total 19 rows starting from the beginning of the part 2024.10.22 00:41:00.398547 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 14968 2024.10.22 00:41:00.663949 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 7.02 MiB, will set to 1.43 GiB (RSS), difference: -1.39 MiB 2024.10.22 00:41:00.735887 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:00.752101 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:41:01.019501 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_127_127_0 (state Temporary) 2024.10.22 00:41:01.271795 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_127_127_0 to 202410_137_137_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:01.310658 [ 1242 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_132_27 to 202410_137_137_0 2024.10.22 00:41:01.326961 [ 1242 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:01.332300 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:41:01.386511 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_132_27 to 202410_137_137_0 into Compact with storage Full 2024.10.22 00:41:01.484650 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 145 2024.10.22 00:41:01.570622 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} MergeTask::MergeProjectionsStage: Merge sorted 2324 rows, containing 12 columns (12 merged, 0 gathered) in 1.853848805 sec., 1253.6081657425132 rows/sec., 194.50 KiB/sec. 2024.10.22 00:41:01.594731 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_96_19, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0] 2024.10.22 00:41:01.606096 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_96_19, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0] 2024.10.22 00:41:01.637004 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} MergedBlockOutputStream: filled checksums 202410_1_171_34 (state Temporary) 2024.10.22 00:41:01.704899 [ 1733 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fzt/nyzlhfndbahircpdmxtaihhndodtj, size 147 2024.10.22 00:41:01.765452 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_96_19, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0] 2024.10.22 00:41:01.785420 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 78 entries to flush up to offset 10229 2024.10.22 00:41:01.822342 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:41:01.952888 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_171_34 to 202410_1_171_34 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:01.954700 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_132_27, total 176 rows starting from the beginning of the part 2024.10.22 00:41:01.961133 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_171_34} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_166_33, 202410_171_171_0] -> 202410_1_171_34 2024.10.22 00:41:02.062653 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:41:02.162250 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:02.165818 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.73 GiB. 2024.10.22 00:41:02.283759 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_133_133_0, total 1 rows starting from the beginning of the part 2024.10.22 00:41:02.333354 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_157_157_0 (state Temporary) 2024.10.22 00:41:02.499775 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_157_157_0 to 202410_163_163_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:02.582851 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_134_134_0, total 1 rows starting from the beginning of the part 2024.10.22 00:41:02.633399 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10229 2024.10.22 00:41:02.776445 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.88 GiB, free memory in arenas 3.70 MiB, will set to 1.43 GiB (RSS), difference: 108.18 MiB 2024.10.22 00:41:03.050952 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_135_135_0, total 1 rows starting from the beginning of the part 2024.10.22 00:41:03.391414 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RefererDomain 2024.10.22 00:41:03.409454 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RefererDomain 2024.10.22 00:41:03.422287 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RefererDomain 2024.10.22 00:41:03.432568 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RefererDomain 2024.10.22 00:41:03.448117 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_136_136_0, total 1 rows starting from the beginning of the part 2024.10.22 00:41:03.471037 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RefererDomain 2024.10.22 00:41:03.485111 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RefererDomain 2024.10.22 00:41:03.497579 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RefererDomain 2024.10.22 00:41:03.624797 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_137_137_0, total 1 rows starting from the beginning of the part 2024.10.22 00:41:03.776382 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wie/picmyhdsnaaiuybivsjkeyshtsuqw (LogSeriesLimiter: on interval from 2024-10-22 00:40:42 to 2024-10-22 00:41:03 accepted series 1 / 143 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:41:04.404579 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 15027 2024.10.22 00:41:04.692757 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.88 GiB, free memory in arenas 5.02 MiB, will set to 1.43 GiB (RSS), difference: 88.06 MiB 2024.10.22 00:41:04.704013 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 263584 2024.10.22 00:41:04.879738 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:04.882807 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.71 GiB. 2024.10.22 00:41:04.969443 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_34_7/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:41:05.033777 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_272_272_0 (state Temporary) 2024.10.22 00:41:05.166992 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_272_272_0 to 202410_285_285_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:05.243163 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_280_56 to 202410_285_285_0 2024.10.22 00:41:05.278507 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:05.283704 [ 1158 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.70 GiB. 2024.10.22 00:41:05.310561 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_280_56 to 202410_285_285_0 into Compact with storage Full 2024.10.22 00:41:05.387019 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15027 2024.10.22 00:41:05.396837 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:41:05.412535 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_280_56, total 15637 rows starting from the beginning of the part 2024.10.22 00:41:05.660140 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_281_281_0, total 31 rows starting from the beginning of the part 2024.10.22 00:41:05.677916 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_282_282_0, total 46 rows starting from the beginning of the part 2024.10.22 00:41:05.703384 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_283_283_0, total 48 rows starting from the beginning of the part 2024.10.22 00:41:05.743479 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_284_284_0, total 37 rows starting from the beginning of the part 2024.10.22 00:41:05.885297 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_285_285_0, total 59 rows starting from the beginning of the part 2024.10.22 00:41:05.934323 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:41:05.955043 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:05.963176 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.70 GiB. 2024.10.22 00:41:06.407695 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_152_152_0 (state Temporary) 2024.10.22 00:41:06.469692 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_137_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:41:06.556381 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_12_12_0 (state Temporary) 2024.10.22 00:41:06.581379 [ 1676 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yea/ckaywwgvtguglvzonslximtbmmrlk, size 359 2024.10.22 00:41:06.594982 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 1.88 GiB, free memory in arenas 3.66 MiB, will set to 1.42 GiB (RSS), difference: 130.82 MiB 2024.10.22 00:41:06.739821 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_152_152_0 to 202410_159_159_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:06.781365 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 263584 2024.10.22 00:41:06.933713 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} MergeTask::MergeProjectionsStage: Merge sorted 181 rows, containing 32 columns (32 merged, 0 gathered) in 5.562246077 sec., 32.54081130075109 rows/sec., 3.56 KiB/sec. 2024.10.22 00:41:06.993078 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} MergedBlockOutputStream: filled checksums 202410_1_137_28 (state Temporary) 2024.10.22 00:41:07.116861 [ 1186 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:41:07.155295 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 2231 2024.10.22 00:41:07.157676 [ 1186 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:41:07.337043 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:07.342510 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.69 GiB. 2024.10.22 00:41:07.392951 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_159_159_0 (state Temporary) 2024.10.22 00:41:07.571422 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_159_159_0 to 202410_172_172_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:07.587155 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_137_28 to 202410_1_137_28 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:07.593167 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_137_28} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_132_27, 202410_137_137_0] -> 202410_1_137_28 2024.10.22 00:41:07.631240 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2231 2024.10.22 00:41:07.658995 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:41:07.781341 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} MergeTask::MergeProjectionsStage: Merge sorted 15858 rows, containing 13 columns (13 merged, 0 gathered) in 2.48421195 sec., 6383.513290804353 rows/sec., 971.71 KiB/sec. 2024.10.22 00:41:07.871451 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} MergedBlockOutputStream: filled checksums 202410_1_285_57 (state Temporary) 2024.10.22 00:41:07.959046 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_285_57 to 202410_1_285_57 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:07.963386 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_285_57} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_280_56, 202410_285_285_0] -> 202410_1_285_57 2024.10.22 00:41:08.008592 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.15 MiB. 2024.10.22 00:41:08.225367 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:08.332274 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.69 GiB. 2024.10.22 00:41:08.399936 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 2.70 MiB, will set to 1.41 GiB (RSS), difference: 6.27 MiB 2024.10.22 00:41:08.427186 [ 1193 ] {} DatabaseCatalog: Execution took 437 ms. 2024.10.22 00:41:09.429210 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 15079 2024.10.22 00:41:09.468149 [ 1197 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 269 ms. 2024.10.22 00:41:09.533540 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:41:09.535294 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:41:09.879050 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:09.882960 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:09.958295 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_273_273_0 (state Temporary) 2024.10.22 00:41:10.015561 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_273_273_0 to 202410_286_286_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:10.075941 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15079 2024.10.22 00:41:10.154960 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 139 entries to flush up to offset 10368 2024.10.22 00:41:10.285300 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:10.292227 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_181_36, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0] 2024.10.22 00:41:10.294169 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_181_36, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0] 2024.10.22 00:41:10.329049 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_181_36, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0] 2024.10.22 00:41:10.388161 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.69 GiB. 2024.10.22 00:41:10.441082 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_158_158_0 (state Temporary) 2024.10.22 00:41:10.472144 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.88 GiB, free memory in arenas 2.98 MiB, will set to 1.40 GiB (RSS), difference: -6.72 MiB 2024.10.22 00:41:10.492407 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_158_158_0 to 202410_164_164_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:10.678564 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10368 2024.10.22 00:41:10.863360 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_32_32_0 (state Temporary) 2024.10.22 00:41:12.522008 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.88 GiB, free memory in arenas 3.54 MiB, will set to 1.41 GiB (RSS), difference: -3.66 MiB 2024.10.22 00:41:13.455286 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_100_20, 202410_101_101_0, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0] 2024.10.22 00:41:13.457569 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_100_20, 202410_101_101_0, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0] 2024.10.22 00:41:13.493333 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_100_20, 202410_101_101_0, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0] 2024.10.22 00:41:13.787196 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 265504 2024.10.22 00:41:14.052059 [ 1181 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 00:41:14.084587 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 15105 2024.10.22 00:41:14.574148 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:14.576666 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:14.608448 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_274_274_0 (state Temporary) 2024.10.22 00:41:14.719200 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:14.720794 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:14.740882 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_274_274_0 to 202410_287_287_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:14.779786 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15105 2024.10.22 00:41:15.135853 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 2249 2024.10.22 00:41:15.174009 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_153_153_0 (state Temporary) 2024.10.22 00:41:15.261090 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_153_153_0 to 202410_160_160_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:15.390840 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 265504 2024.10.22 00:41:15.305825 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:15.420860 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:15.568826 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gjr/rlolkbcgrgvtrxaawjpqpajysjeof. 2024.10.22 00:41:15.569483 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_160_160_0 (state Temporary) 2024.10.22 00:41:15.725248 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_160_160_0 to 202410_173_173_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:15.770679 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2249 2024.10.22 00:41:16.349405 [ 1239 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 207 ms. 2024.10.22 00:41:16.499345 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 146 2024.10.22 00:41:16.884669 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:16.888155 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:16.953213 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_128_128_0 (state Temporary) 2024.10.22 00:41:17.128117 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_128_128_0 to 202410_138_138_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:17.320701 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 146 2024.10.22 00:41:18.319087 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 58 entries to flush up to offset 10426 2024.10.22 00:41:18.431593 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:18.436935 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:18.488671 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_159_159_0 (state Temporary) 2024.10.22 00:41:18.781580 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_159_159_0 to 202410_165_165_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:18.782650 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 29 entries to flush up to offset 15134 2024.10.22 00:41:18.953315 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:18.963187 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:18.983406 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_32_32_0 to 202410_35_35_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:18.995189 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_275_275_0 (state Temporary) 2024.10.22 00:41:19.018185 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10426 2024.10.22 00:41:19.041359 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 5.58 MiB, will set to 1.41 GiB (RSS), difference: -6.03 MiB 2024.10.22 00:41:19.065500 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_275_275_0 to 202410_288_288_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:19.094088 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15134 2024.10.22 00:41:19.232685 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_34_7/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:41:19.535120 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:41:19.542946 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:41:20.719078 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 6.21 MiB, will set to 1.41 GiB (RSS), difference: 8.59 MiB 2024.10.22 00:41:22.168132 [ 1257 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:41:22.280537 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1268 2024.10.22 00:41:22.291613 [ 1257 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:41:22.395737 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 267424 2024.10.22 00:41:22.448431 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.88 GiB, free memory in arenas 7.89 MiB, will set to 1.41 GiB (RSS), difference: 3.21 MiB 2024.10.22 00:41:22.582208 [ 1701 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wie/picmyhdsnaaiuybivsjkeyshtsuqw, size 2833854 2024.10.22 00:41:22.762833 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Refresh 2024.10.22 00:41:22.770299 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Refresh 2024.10.22 00:41:22.798713 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Refresh 2024.10.22 00:41:22.830571 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Refresh 2024.10.22 00:41:22.865398 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Refresh 2024.10.22 00:41:22.894280 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Refresh 2024.10.22 00:41:22.923452 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Refresh 2024.10.22 00:41:22.983326 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fdg/veasgpcdltcourddxamuvkcbzldjz (LogSeriesLimiter: on interval from 2024-10-22 00:41:03 to 2024-10-22 00:41:22 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:41:23.110530 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 28 entries to flush up to offset 15162 2024.10.22 00:41:23.112927 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:23.121266 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:23.247740 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:23.251077 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:23.268472 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_154_154_0 (state Temporary) 2024.10.22 00:41:23.274629 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 2255 2024.10.22 00:41:23.316845 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_276_276_0 (state Temporary) 2024.10.22 00:41:23.365320 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_276_276_0 to 202410_289_289_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:23.367134 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_154_154_0 to 202410_161_161_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:23.386310 [ 1248 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_156_31 to 202410_161_161_0 2024.10.22 00:41:23.389997 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 267424 2024.10.22 00:41:23.411828 [ 1248 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:23.415719 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:23.418844 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15162 2024.10.22 00:41:23.461307 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_156_31 to 202410_161_161_0 into Compact with storage Full 2024.10.22 00:41:23.473243 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:23.476472 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:23.484952 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.88 GiB, free memory in arenas 8.87 MiB, will set to 1.41 GiB (RSS), difference: -2.22 MiB 2024.10.22 00:41:23.527943 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_161_161_0 (state Temporary) 2024.10.22 00:41:23.595811 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:41:23.603169 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} MergeTreeSequentialSource: Reading 35 marks from part 202410_1_156_31, total 272182 rows starting from the beginning of the part 2024.10.22 00:41:23.614761 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_157_157_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:41:23.630231 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_158_158_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:41:23.652049 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_159_159_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:41:23.666923 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_160_160_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:41:23.678337 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_161_161_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:41:23.786723 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_161_161_0 to 202410_174_174_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:23.850946 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2255 2024.10.22 00:41:24.026795 [ 1270 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 00:41:24.823660 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 147 2024.10.22 00:41:25.483251 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:25.485599 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:25.610819 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 8.60 MiB, will set to 1.41 GiB (RSS), difference: -7.01 MiB 2024.10.22 00:41:25.638864 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_129_129_0 (state Temporary) 2024.10.22 00:41:25.719844 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_129_129_0 to 202410_139_139_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:25.788311 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 147 2024.10.22 00:41:26.275663 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_161_32/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:41:26.651011 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 60 entries to flush up to offset 10486 2024.10.22 00:41:26.873371 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:26.878490 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:26.920566 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_160_160_0 (state Temporary) 2024.10.22 00:41:26.971882 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_160_160_0 to 202410_166_166_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:27.114485 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10486 2024.10.22 00:41:27.285497 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_12_12_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:41:27.428812 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 15207 2024.10.22 00:41:27.531747 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.88 GiB, free memory in arenas 16.64 MiB, will set to 1.42 GiB (RSS), difference: 8.11 MiB 2024.10.22 00:41:27.753660 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:27.758893 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:27.897973 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_277_277_0 (state Temporary) 2024.10.22 00:41:27.940782 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_277_277_0 to 202410_290_290_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:27.989746 [ 1164 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_285_57 to 202410_290_290_0 2024.10.22 00:41:27.992112 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15207 2024.10.22 00:41:28.012972 [ 1164 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:28.016526 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:28.055112 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_285_57 to 202410_290_290_0 into Compact with storage Full 2024.10.22 00:41:28.108163 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:41:28.124427 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_285_57, total 15858 rows starting from the beginning of the part 2024.10.22 00:41:28.169239 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_286_286_0, total 52 rows starting from the beginning of the part 2024.10.22 00:41:28.189446 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_287_287_0, total 26 rows starting from the beginning of the part 2024.10.22 00:41:28.235956 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_288_288_0, total 29 rows starting from the beginning of the part 2024.10.22 00:41:28.266688 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_289_289_0, total 28 rows starting from the beginning of the part 2024.10.22 00:41:28.289815 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_290_290_0, total 45 rows starting from the beginning of the part 2024.10.22 00:41:29.548453 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:41:29.556342 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:41:29.692689 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} MergeTask::MergeProjectionsStage: Merge sorted 16038 rows, containing 13 columns (13 merged, 0 gathered) in 1.663428448 sec., 9641.53283496087 rows/sec., 1.43 MiB/sec. 2024.10.22 00:41:29.817185 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 1308 2024.10.22 00:41:30.074394 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} MergedBlockOutputStream: filled checksums 202410_1_290_58 (state Temporary) 2024.10.22 00:41:30.206945 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_290_58 to 202410_1_290_58 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:30.212005 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_290_58} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_285_57, 202410_290_290_0] -> 202410_1_290_58 2024.10.22 00:41:30.347752 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.15 MiB. 2024.10.22 00:41:30.363611 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 18.02 MiB, will set to 1.42 GiB (RSS), difference: 7.22 MiB 2024.10.22 00:41:30.412414 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 269344 2024.10.22 00:41:31.354010 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 2268 2024.10.22 00:41:31.412332 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:31.417624 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:31.674959 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:31.678984 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:31.721102 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_162_162_0 (state Temporary) 2024.10.22 00:41:31.816025 [ 1178 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 00:41:31.896899 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_162_162_0 to 202410_175_175_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:32.017183 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 15243 2024.10.22 00:41:32.065168 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_155_155_0 (state Temporary) 2024.10.22 00:41:32.081190 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2268 2024.10.22 00:41:32.160511 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:32.167438 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:32.177755 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_155_155_0 to 202410_162_162_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:32.200811 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_278_278_0 (state Temporary) 2024.10.22 00:41:32.203031 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 269344 2024.10.22 00:41:32.347095 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_278_278_0 to 202410_291_291_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:32.379395 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 17.30 MiB, will set to 1.42 GiB (RSS), difference: -5.84 MiB 2024.10.22 00:41:32.628302 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15243 2024.10.22 00:41:32.681616 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_34_7/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:41:33.291549 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 148 2024.10.22 00:41:33.603559 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 18.04 MiB, will set to 1.42 GiB (RSS), difference: 7.80 MiB 2024.10.22 00:41:33.732426 [ 1170 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 217 ms. 2024.10.22 00:41:33.753195 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} MergeTask::MergeProjectionsStage: Merge sorted 281014 rows, containing 4 columns (4 merged, 0 gathered) in 10.297964802 sec., 27288.30457309617 rows/sec., 400.57 KiB/sec. 2024.10.22 00:41:33.791414 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} MergedBlockOutputStream: filled checksums 202410_1_161_32 (state Temporary) 2024.10.22 00:41:33.864924 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_161_32 to 202410_1_161_32 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:33.869031 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_161_32} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_156_31, 202410_161_161_0] -> 202410_1_161_32 2024.10.22 00:41:33.913961 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:33.919906 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:34.002238 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_130_130_0 (state Temporary) 2024.10.22 00:41:34.034162 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:41:34.279799 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_130_130_0 to 202410_140_140_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:34.508687 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_186_37, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0, 202410_191_191_0] 2024.10.22 00:41:34.511011 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_186_37, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0, 202410_191_191_0] 2024.10.22 00:41:34.563840 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_186_37, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0, 202410_191_191_0] 2024.10.22 00:41:34.572963 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 148 2024.10.22 00:41:34.623285 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 67 entries to flush up to offset 10553 2024.10.22 00:41:34.881054 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:34.884836 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:34.929778 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_161_161_0 (state Temporary) 2024.10.22 00:41:34.977819 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_161_161_0 to 202410_167_167_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:35.120944 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10553 2024.10.22 00:41:35.229807 [ 1202 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_162_33 to 202410_167_167_0 2024.10.22 00:41:35.246589 [ 1202 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:35.251326 [ 1202 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:35.281189 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_162_33 to 202410_167_167_0 into Compact with storage Full 2024.10.22 00:41:35.419837 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:41:35.450326 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_162_33, total 10255 rows starting from the beginning of the part 2024.10.22 00:41:35.495895 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_163_163_0, total 78 rows starting from the beginning of the part 2024.10.22 00:41:35.536762 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_164_164_0, total 139 rows starting from the beginning of the part 2024.10.22 00:41:35.584705 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_165_165_0, total 58 rows starting from the beginning of the part 2024.10.22 00:41:35.648581 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_166_166_0, total 60 rows starting from the beginning of the part 2024.10.22 00:41:35.683956 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 13.96 MiB, will set to 1.42 GiB (RSS), difference: 4.57 MiB 2024.10.22 00:41:35.715082 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_167_167_0, total 67 rows starting from the beginning of the part 2024.10.22 00:41:36.644225 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 15289 2024.10.22 00:41:36.785045 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:36.789604 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:36.974957 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_279_279_0 (state Temporary) 2024.10.22 00:41:37.300124 [ 1207 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:41:37.322853 [ 1207 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:41:37.331026 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_279_279_0 to 202410_292_292_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:37.417070 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15289 2024.10.22 00:41:37.418288 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_167_34/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:41:37.474473 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 10.66 MiB, will set to 1.43 GiB (RSS), difference: -4.83 MiB 2024.10.22 00:41:37.774986 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} MergeTask::MergeProjectionsStage: Merge sorted 10657 rows, containing 13 columns (13 merged, 0 gathered) in 2.489980468 sec., 4279.95325142446 rows/sec., 1.79 MiB/sec. 2024.10.22 00:41:37.819093 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} MergedBlockOutputStream: filled checksums 202410_1_167_34 (state Temporary) 2024.10.22 00:41:37.903318 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_167_34 to 202410_1_167_34 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:37.909360 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_167_34} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_162_33, 202410_167_167_0] -> 202410_1_167_34 2024.10.22 00:41:38.060447 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.26 MiB. 2024.10.22 00:41:38.072953 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_111_22, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0] 2024.10.22 00:41:38.075057 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_111_22, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0] 2024.10.22 00:41:38.106865 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_111_22, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0] 2024.10.22 00:41:39.313071 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 271264 2024.10.22 00:41:39.512670 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 9.29 MiB, will set to 1.42 GiB (RSS), difference: 18.61 MiB 2024.10.22 00:41:39.544247 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:41:39.548326 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:41:39.583492 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 2288 2024.10.22 00:41:39.694723 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:39.700356 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:39.748443 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_163_163_0 (state Temporary) 2024.10.22 00:41:40.110689 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_163_163_0 to 202410_176_176_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:40.138938 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:40.143042 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:40.169840 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2288 2024.10.22 00:41:40.244193 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_156_156_0 (state Temporary) 2024.10.22 00:41:40.250957 [ 1219 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_171_34 to 202410_176_176_0 2024.10.22 00:41:40.266253 [ 1219 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:40.267807 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:40.283067 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_171_34 to 202410_176_176_0 into Compact with storage Full 2024.10.22 00:41:40.307408 [ 1742 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fdg/veasgpcdltcourddxamuvkcbzldjz, size 76659 2024.10.22 00:41:40.345702 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_156_156_0 to 202410_163_163_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:40.349790 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:41:40.363055 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_171_34, total 2324 rows starting from the beginning of the part 2024.10.22 00:41:40.378031 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 271264 2024.10.22 00:41:40.396162 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gjr/rlolkbcgrgvtrxaawjpqpajysjeof. 2024.10.22 00:41:40.436792 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 1.88 GiB, free memory in arenas 8.91 MiB, will set to 1.41 GiB (RSS), difference: 38.81 MiB 2024.10.22 00:41:40.487282 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_172_172_0, total 13 rows starting from the beginning of the part 2024.10.22 00:41:40.521319 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_173_173_0, total 18 rows starting from the beginning of the part 2024.10.22 00:41:40.576273 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_174_174_0, total 6 rows starting from the beginning of the part 2024.10.22 00:41:40.607857 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_175_175_0, total 13 rows starting from the beginning of the part 2024.10.22 00:41:40.809512 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsRobot 2024.10.22 00:41:40.848780 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_176_176_0, total 20 rows starting from the beginning of the part 2024.10.22 00:41:40.848673 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsRobot 2024.10.22 00:41:40.885040 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsRobot 2024.10.22 00:41:40.921090 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsRobot 2024.10.22 00:41:40.965394 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsRobot 2024.10.22 00:41:41.005332 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsRobot 2024.10.22 00:41:41.112729 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsRobot 2024.10.22 00:41:41.274622 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rpg/tcfjgywptvvrystypquzggxsekbow (LogSeriesLimiter: on interval from 2024-10-22 00:41:22 to 2024-10-22 00:41:41 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:41:41.426818 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 15344 2024.10.22 00:41:41.885251 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} MergeTask::MergeProjectionsStage: Merge sorted 2394 rows, containing 12 columns (12 merged, 0 gathered) in 1.609660448 sec., 1487.2701897934687 rows/sec., 230.90 KiB/sec. 2024.10.22 00:41:41.917611 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} MergedBlockOutputStream: filled checksums 202410_1_176_35 (state Temporary) 2024.10.22 00:41:41.976869 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:42.074461 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 149 2024.10.22 00:41:42.079011 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:42.224887 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_280_280_0 (state Temporary) 2024.10.22 00:41:42.238296 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_176_35 to 202410_1_176_35 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:42.241211 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_176_35} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_171_34, 202410_176_176_0] -> 202410_1_176_35 2024.10.22 00:41:42.268440 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:41:42.551444 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.88 GiB, free memory in arenas 5.52 MiB, will set to 1.40 GiB (RSS), difference: 79.92 MiB 2024.10.22 00:41:42.565525 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_280_280_0 to 202410_293_293_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:42.589686 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:42.594005 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:42.603393 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15344 2024.10.22 00:41:42.644433 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 10624 2024.10.22 00:41:42.650426 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_131_131_0 (state Temporary) 2024.10.22 00:41:42.867697 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:42.870898 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:41:43.008214 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_162_162_0 (state Temporary) 2024.10.22 00:41:43.130917 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_131_131_0 to 202410_141_141_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:43.232099 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 149 2024.10.22 00:41:43.277131 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_162_162_0 to 202410_168_168_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:43.324484 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10624 2024.10.22 00:41:44.212703 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_101_20, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0] 2024.10.22 00:41:44.215783 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_101_20, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0] 2024.10.22 00:41:44.351471 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_101_20, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0] 2024.10.22 00:41:44.530885 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 1.88 GiB, free memory in arenas 3.89 MiB, will set to 1.40 GiB (RSS), difference: 34.61 MiB 2024.10.22 00:41:44.887001 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 968811 bytes for 995d320a2aee64bec80ceb8017ce2cc9:0) 2024.10.22 00:41:44.888908 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 54d45cca0235b14ad812971d9d632885, offset: 0, size: 64751 2024.10.22 00:41:44.892660 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 54d45cca0235b14ad812971d9d632885 for removal 2024.10.22 00:41:44.899606 [ 1476 ] {} CacheMetadata: Key 54d45cca0235b14ad812971d9d632885 is removed from metadata 2024.10.22 00:41:44.901641 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 40c3c0756da0f3d19cddbb0052a36fcc, offset: 0, size: 164 2024.10.22 00:41:44.909533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 40c3c0756da0f3d19cddbb0052a36fcc for removal 2024.10.22 00:41:44.913269 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ff4740e4910d91e8b3f92a20a74c8f1c, offset: 0, size: 20228 2024.10.22 00:41:44.917011 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ff4740e4910d91e8b3f92a20a74c8f1c for removal 2024.10.22 00:41:44.921781 [ 1476 ] {} CacheMetadata: Key ff4740e4910d91e8b3f92a20a74c8f1c is removed from metadata 2024.10.22 00:41:44.935641 [ 1476 ] {} CacheMetadata: Key 40c3c0756da0f3d19cddbb0052a36fcc is removed from metadata 2024.10.22 00:41:44.971072 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_34_7/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:41:45.713283 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 10 file segments (while reserving 79785 bytes for 88818bb666aae4885f0de933829cec27:0) 2024.10.22 00:41:45.715279 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a08b3869d23cc52716dffbada2e61f12, offset: 0, size: 154 2024.10.22 00:41:45.718578 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a08b3869d23cc52716dffbada2e61f12 for removal 2024.10.22 00:41:45.725728 [ 1476 ] {} CacheMetadata: Key a08b3869d23cc52716dffbada2e61f12 is removed from metadata 2024.10.22 00:41:45.728875 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 978d6d8960a00ed1fe333590d009a289, offset: 0, size: 17786 2024.10.22 00:41:45.732836 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_88_18, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0, 202410_92_92_0, 202410_93_93_0] 2024.10.22 00:41:45.733347 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 978d6d8960a00ed1fe333590d009a289 for removal 2024.10.22 00:41:45.737494 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_88_18, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0, 202410_92_92_0, 202410_93_93_0] 2024.10.22 00:41:45.739534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c9eea76d202f490561ba9e0a8895ec1b, offset: 0, size: 109749 2024.10.22 00:41:45.745044 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c9eea76d202f490561ba9e0a8895ec1b for removal 2024.10.22 00:41:45.745269 [ 1476 ] {} CacheMetadata: Key 978d6d8960a00ed1fe333590d009a289 is removed from metadata 2024.10.22 00:41:45.752202 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dead82567efee2688f59e01450444052, offset: 0, size: 15685 2024.10.22 00:41:45.758832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dead82567efee2688f59e01450444052 for removal 2024.10.22 00:41:45.761776 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0b4b314e9c7129f4d743fff9e452bfdd, offset: 0, size: 154 2024.10.22 00:41:45.765197 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0b4b314e9c7129f4d743fff9e452bfdd for removal 2024.10.22 00:41:45.768873 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b652d94790983f6865bf416e31eeb8cf, offset: 0, size: 144 2024.10.22 00:41:45.769090 [ 1476 ] {} CacheMetadata: Key 0b4b314e9c7129f4d743fff9e452bfdd is removed from metadata 2024.10.22 00:41:45.771709 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b652d94790983f6865bf416e31eeb8cf for removal 2024.10.22 00:41:45.774340 [ 1476 ] {} CacheMetadata: Key b652d94790983f6865bf416e31eeb8cf is removed from metadata 2024.10.22 00:41:45.774376 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 119ec30d74bef0dd9bab5c35aecaf4cd, offset: 0, size: 1742 2024.10.22 00:41:45.777829 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 119ec30d74bef0dd9bab5c35aecaf4cd for removal 2024.10.22 00:41:45.780022 [ 1476 ] {} CacheMetadata: Key 119ec30d74bef0dd9bab5c35aecaf4cd is removed from metadata 2024.10.22 00:41:45.780076 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c01c29eb2d46c1f5ced71f66be55581b, offset: 0, size: 1831 2024.10.22 00:41:45.782784 [ 1476 ] {} CacheMetadata: Key dead82567efee2688f59e01450444052 is removed from metadata 2024.10.22 00:41:45.785170 [ 1476 ] {} CacheMetadata: Key c9eea76d202f490561ba9e0a8895ec1b is removed from metadata 2024.10.22 00:41:45.871833 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_88_18, 202410_89_89_0, 202410_90_90_0, 202410_91_91_0, 202410_92_92_0, 202410_93_93_0] 2024.10.22 00:41:45.882651 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c01c29eb2d46c1f5ced71f66be55581b for removal 2024.10.22 00:41:45.885254 [ 1476 ] {} CacheMetadata: Key c01c29eb2d46c1f5ced71f66be55581b is removed from metadata 2024.10.22 00:41:45.885642 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ab9abf73fc9db65e69bdef1fb8ec8c9d, offset: 0, size: 154 2024.10.22 00:41:45.889133 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ab9abf73fc9db65e69bdef1fb8ec8c9d for removal 2024.10.22 00:41:45.891223 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 745a79d28d7cc599c189e3ccc3b1c4fc, offset: 0, size: 154 2024.10.22 00:41:45.891314 [ 1476 ] {} CacheMetadata: Key ab9abf73fc9db65e69bdef1fb8ec8c9d is removed from metadata 2024.10.22 00:41:45.899637 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 745a79d28d7cc599c189e3ccc3b1c4fc for removal 2024.10.22 00:41:45.916498 [ 1476 ] {} CacheMetadata: Key 745a79d28d7cc599c189e3ccc3b1c4fc is removed from metadata 2024.10.22 00:41:46.080663 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:46.082594 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:46.576524 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 4.68 MiB, will set to 1.40 GiB (RSS), difference: -2.85 MiB 2024.10.22 00:41:46.615676 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 15420 2024.10.22 00:41:46.896912 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:46.901161 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:46.922632 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 12 file segments (while reserving 573236 bytes for 7749d04b5fae7df26d159e1ecab0f6de:0) 2024.10.22 00:41:46.928415 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 814765e15fc5e88e7883eca95da37093, offset: 0, size: 1103750 2024.10.22 00:41:46.938331 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 814765e15fc5e88e7883eca95da37093 for removal 2024.10.22 00:41:46.944250 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2f5975a3e09dfac527d78d62758c65d4, offset: 0, size: 160 2024.10.22 00:41:46.955325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2f5975a3e09dfac527d78d62758c65d4 for removal 2024.10.22 00:41:46.962494 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 17b1680d2ea311042a32735cd129001b, offset: 0, size: 48500 2024.10.22 00:41:46.967495 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 17b1680d2ea311042a32735cd129001b for removal 2024.10.22 00:41:46.971326 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d399c5b511a2303650f33912f43f7f30, offset: 0, size: 165 2024.10.22 00:41:46.975076 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d399c5b511a2303650f33912f43f7f30 for removal 2024.10.22 00:41:46.991518 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 39e2e9554d06d9517ad4c480c0e36050, offset: 0, size: 90484 2024.10.22 00:41:46.993712 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 39e2e9554d06d9517ad4c480c0e36050 for removal 2024.10.22 00:41:46.997413 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8591984c9ba687b99b3280110dcd1459, offset: 0, size: 165 2024.10.22 00:41:47.005228 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8591984c9ba687b99b3280110dcd1459 for removal 2024.10.22 00:41:47.009722 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 619b08c35170f5c359899b92a009a507, offset: 0, size: 32976 2024.10.22 00:41:47.013512 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 619b08c35170f5c359899b92a009a507 for removal 2024.10.22 00:41:47.018667 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c2ffb560096119bc83617e0e0c589c5e, offset: 0, size: 154 2024.10.22 00:41:47.021360 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c2ffb560096119bc83617e0e0c589c5e for removal 2024.10.22 00:41:47.026991 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4fe03931436cde823323eb335357930b, offset: 0, size: 164 2024.10.22 00:41:47.031200 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4fe03931436cde823323eb335357930b for removal 2024.10.22 00:41:47.044048 [ 1476 ] {} CacheMetadata: Key 814765e15fc5e88e7883eca95da37093 is removed from metadata 2024.10.22 00:41:47.045215 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 421bc8eec41ccd1dd5b76d919ac086b7, offset: 0, size: 86549 2024.10.22 00:41:47.051367 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 421bc8eec41ccd1dd5b76d919ac086b7 for removal 2024.10.22 00:41:47.063074 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aa6c7a0ae10cb9b3298087d284aac9bf, offset: 0, size: 91245 2024.10.22 00:41:47.066998 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aa6c7a0ae10cb9b3298087d284aac9bf for removal 2024.10.22 00:41:47.070410 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0e16e33499312b787fefa93faaa171e0, offset: 0, size: 155 2024.10.22 00:41:47.074227 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0e16e33499312b787fefa93faaa171e0 for removal 2024.10.22 00:41:47.080039 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_281_281_0 (state Temporary) 2024.10.22 00:41:47.082036 [ 1476 ] {} CacheMetadata: Key 421bc8eec41ccd1dd5b76d919ac086b7 is removed from metadata 2024.10.22 00:41:47.089665 [ 1476 ] {} CacheMetadata: Key 0e16e33499312b787fefa93faaa171e0 is removed from metadata 2024.10.22 00:41:47.102458 [ 1476 ] {} CacheMetadata: Key 4fe03931436cde823323eb335357930b is removed from metadata 2024.10.22 00:41:47.115267 [ 1476 ] {} CacheMetadata: Key c2ffb560096119bc83617e0e0c589c5e is removed from metadata 2024.10.22 00:41:47.124716 [ 1476 ] {} CacheMetadata: Key 619b08c35170f5c359899b92a009a507 is removed from metadata 2024.10.22 00:41:47.132459 [ 1476 ] {} CacheMetadata: Key 8591984c9ba687b99b3280110dcd1459 is removed from metadata 2024.10.22 00:41:47.148612 [ 1476 ] {} CacheMetadata: Key 39e2e9554d06d9517ad4c480c0e36050 is removed from metadata 2024.10.22 00:41:47.164814 [ 1476 ] {} CacheMetadata: Key d399c5b511a2303650f33912f43f7f30 is removed from metadata 2024.10.22 00:41:47.174713 [ 1476 ] {} CacheMetadata: Key aa6c7a0ae10cb9b3298087d284aac9bf is removed from metadata 2024.10.22 00:41:47.183405 [ 1476 ] {} CacheMetadata: Key 17b1680d2ea311042a32735cd129001b is removed from metadata 2024.10.22 00:41:47.189690 [ 1476 ] {} CacheMetadata: Key 2f5975a3e09dfac527d78d62758c65d4 is removed from metadata 2024.10.22 00:41:47.277918 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_281_281_0 to 202410_294_294_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:47.323616 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15420 2024.10.22 00:41:47.382437 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1744 entries to flush up to offset 273008 2024.10.22 00:41:47.672461 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 2301 2024.10.22 00:41:47.877600 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:47.880611 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:47.891316 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 10 file segments (while reserving 810686 bytes for 723ff4671bf16a6cdeb6fecab3397e2a:0) 2024.10.22 00:41:47.894268 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 82d2569e8b3f1310b337d03c229d462b, offset: 0, size: 179306 2024.10.22 00:41:47.905409 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 82d2569e8b3f1310b337d03c229d462b for removal 2024.10.22 00:41:47.911104 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d45d9c7288272b0b3cb923cc7e822415, offset: 0, size: 71759 2024.10.22 00:41:47.912550 [ 1476 ] {} CacheMetadata: Key 82d2569e8b3f1310b337d03c229d462b is removed from metadata 2024.10.22 00:41:47.918483 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d45d9c7288272b0b3cb923cc7e822415 for removal 2024.10.22 00:41:47.920808 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_164_164_0 (state Temporary) 2024.10.22 00:41:47.923509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f033a8ad577dfa83d8a0efc0cb214d0a, offset: 0, size: 144 2024.10.22 00:41:47.927402 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f033a8ad577dfa83d8a0efc0cb214d0a for removal 2024.10.22 00:41:47.944315 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a4fba9a32e99a2a92d7ea405dc2a623c, offset: 0, size: 176 2024.10.22 00:41:47.950340 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a4fba9a32e99a2a92d7ea405dc2a623c for removal 2024.10.22 00:41:47.950603 [ 1476 ] {} CacheMetadata: Key f033a8ad577dfa83d8a0efc0cb214d0a is removed from metadata 2024.10.22 00:41:47.959115 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 015764fc22015b47cc438fbae50324ec, offset: 0, size: 154 2024.10.22 00:41:47.963996 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 015764fc22015b47cc438fbae50324ec for removal 2024.10.22 00:41:47.964926 [ 1476 ] {} CacheMetadata: Key a4fba9a32e99a2a92d7ea405dc2a623c is removed from metadata 2024.10.22 00:41:47.967513 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f215ed7dfedff8ed3156d93d57560607, offset: 0, size: 30847 2024.10.22 00:41:47.970644 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f215ed7dfedff8ed3156d93d57560607 for removal 2024.10.22 00:41:47.973686 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 25b24039156f7a399180376636e2f4de, offset: 0, size: 142 2024.10.22 00:41:47.976885 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 25b24039156f7a399180376636e2f4de for removal 2024.10.22 00:41:47.978944 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f083233eb957b905d86e32678a8eb10c, offset: 0, size: 26198 2024.10.22 00:41:47.986524 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f083233eb957b905d86e32678a8eb10c for removal 2024.10.22 00:41:47.994551 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 333b57197459b142588cd828084cf066, offset: 0, size: 46450 2024.10.22 00:41:47.996489 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_164_164_0 to 202410_177_177_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:47.999859 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 333b57197459b142588cd828084cf066 for removal 2024.10.22 00:41:48.010367 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bc29b1ac6d1faf7c5146cb57d3d33130, offset: 0, size: 154 2024.10.22 00:41:48.017559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bc29b1ac6d1faf7c5146cb57d3d33130 for removal 2024.10.22 00:41:48.070289 [ 1476 ] {} CacheMetadata: Key bc29b1ac6d1faf7c5146cb57d3d33130 is removed from metadata 2024.10.22 00:41:48.070506 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2301 2024.10.22 00:41:48.073457 [ 1476 ] {} CacheMetadata: Key 25b24039156f7a399180376636e2f4de is removed from metadata 2024.10.22 00:41:48.076642 [ 1476 ] {} CacheMetadata: Key f215ed7dfedff8ed3156d93d57560607 is removed from metadata 2024.10.22 00:41:48.080537 [ 1476 ] {} CacheMetadata: Key 333b57197459b142588cd828084cf066 is removed from metadata 2024.10.22 00:41:48.088394 [ 1476 ] {} CacheMetadata: Key f083233eb957b905d86e32678a8eb10c is removed from metadata 2024.10.22 00:41:48.103536 [ 1476 ] {} CacheMetadata: Key 015764fc22015b47cc438fbae50324ec is removed from metadata 2024.10.22 00:41:48.114609 [ 1476 ] {} CacheMetadata: Key d45d9c7288272b0b3cb923cc7e822415 is removed from metadata 2024.10.22 00:41:48.465319 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:48.469337 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:48.497548 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 150406 bytes for b740edbe388279d32a3661bd26d9be02:0) 2024.10.22 00:41:48.505079 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e811fd988a1a33d6eead2be114523bba, offset: 0, size: 1070127 2024.10.22 00:41:48.518752 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e811fd988a1a33d6eead2be114523bba for removal 2024.10.22 00:41:48.522528 [ 1476 ] {} CacheMetadata: Key e811fd988a1a33d6eead2be114523bba is removed from metadata 2024.10.22 00:41:48.523434 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 20ac9238badf073d2ef20f5eb87c6772, offset: 0, size: 160 2024.10.22 00:41:48.531444 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 20ac9238badf073d2ef20f5eb87c6772 for removal 2024.10.22 00:41:48.540412 [ 1476 ] {} CacheMetadata: Key 20ac9238badf073d2ef20f5eb87c6772 is removed from metadata 2024.10.22 00:41:48.578988 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 4.32 MiB, will set to 1.40 GiB (RSS), difference: 1.70 MiB 2024.10.22 00:41:48.736987 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_157_157_0 (state Temporary) 2024.10.22 00:41:48.888718 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_157_157_0 to 202410_164_164_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:48.932307 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 273008 2024.10.22 00:41:49.549521 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:41:49.553438 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:41:50.248788 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_33_33_0 (state Temporary) 2024.10.22 00:41:50.486183 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 5.60 MiB, will set to 1.39 GiB (RSS), difference: -9.72 MiB 2024.10.22 00:41:50.667027 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 71180 bytes for 8f2c10bd2ecf80c48fa314a65e30348b:0) 2024.10.22 00:41:50.669681 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d80baca18c7fb7bf40c67ccec509fc37, offset: 0, size: 154 2024.10.22 00:41:50.671781 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d80baca18c7fb7bf40c67ccec509fc37 for removal 2024.10.22 00:41:50.673400 [ 1476 ] {} CacheMetadata: Key d80baca18c7fb7bf40c67ccec509fc37 is removed from metadata 2024.10.22 00:41:50.734711 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 150 2024.10.22 00:41:50.848723 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 48 entries to flush up to offset 10672 2024.10.22 00:41:51.050525 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 192 bytes for f5a88c323932e0f3d7fa2757b8016ed6:0) 2024.10.22 00:41:51.054578 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 37b7245faf56086b4145bf31c7a4379c, offset: 0, size: 39788 2024.10.22 00:41:51.057669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 37b7245faf56086b4145bf31c7a4379c for removal 2024.10.22 00:41:51.061246 [ 1476 ] {} CacheMetadata: Key 37b7245faf56086b4145bf31c7a4379c is removed from metadata 2024.10.22 00:41:51.072864 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:51.075248 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:51.128807 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_163_163_0 (state Temporary) 2024.10.22 00:41:51.175302 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_163_163_0 to 202410_169_169_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:51.183153 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:51.185144 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:51.259924 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_132_132_0 (state Temporary) 2024.10.22 00:41:51.205499 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 253673 bytes for 17d2debdc4945145a3f89347492d7249:0) 2024.10.22 00:41:51.312454 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 20ce1f319f24703d9dd1002373c051e2, offset: 0, size: 1100421 2024.10.22 00:41:51.312503 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10672 2024.10.22 00:41:51.315765 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 20ce1f319f24703d9dd1002373c051e2 for removal 2024.10.22 00:41:51.318494 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 115551f2b893fd77d5fcde5ea663622a, offset: 0, size: 160 2024.10.22 00:41:51.318842 [ 1476 ] {} CacheMetadata: Key 20ce1f319f24703d9dd1002373c051e2 is removed from metadata 2024.10.22 00:41:51.322415 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 115551f2b893fd77d5fcde5ea663622a for removal 2024.10.22 00:41:51.328770 [ 1476 ] {} CacheMetadata: Key 115551f2b893fd77d5fcde5ea663622a is removed from metadata 2024.10.22 00:41:51.423526 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_132_132_0 to 202410_142_142_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:51.452609 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 122 entries to flush up to offset 15542 2024.10.22 00:41:51.502580 [ 1153 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_137_28 to 202410_142_142_0 2024.10.22 00:41:51.510162 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 150 2024.10.22 00:41:51.519435 [ 1153 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:51.525345 [ 1153 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:51.566128 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_137_28 to 202410_142_142_0 into Compact with storage Full 2024.10.22 00:41:51.628251 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:41:51.668165 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_137_28, total 181 rows starting from the beginning of the part 2024.10.22 00:41:51.690334 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_138_138_0, total 1 rows starting from the beginning of the part 2024.10.22 00:41:51.835709 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_139_139_0, total 1 rows starting from the beginning of the part 2024.10.22 00:41:51.884614 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_140_140_0, total 1 rows starting from the beginning of the part 2024.10.22 00:41:51.922083 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_141_141_0, total 1 rows starting from the beginning of the part 2024.10.22 00:41:51.964335 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:51.966974 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:51.982426 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_142_142_0, total 1 rows starting from the beginning of the part 2024.10.22 00:41:52.124406 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_282_282_0 (state Temporary) 2024.10.22 00:41:52.318874 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_282_282_0 to 202410_295_295_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:52.328971 [ 1165 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:41:52.370733 [ 1165 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:41:52.375637 [ 1177 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_290_58 to 202410_295_295_0 2024.10.22 00:41:52.381811 [ 1177 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:52.384723 [ 1177 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:52.386049 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15542 2024.10.22 00:41:52.465935 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_290_58 to 202410_295_295_0 into Compact with storage Full 2024.10.22 00:41:52.642319 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:41:52.662252 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_290_58, total 16038 rows starting from the beginning of the part 2024.10.22 00:41:52.791032 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_291_291_0, total 36 rows starting from the beginning of the part 2024.10.22 00:41:52.843064 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_292_292_0, total 46 rows starting from the beginning of the part 2024.10.22 00:41:52.875231 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_293_293_0, total 55 rows starting from the beginning of the part 2024.10.22 00:41:52.918688 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_294_294_0, total 76 rows starting from the beginning of the part 2024.10.22 00:41:52.967906 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_295_295_0, total 122 rows starting from the beginning of the part 2024.10.22 00:41:53.361061 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 165094 bytes for cd5348f62303cfe79927f733cb2c32b6:0) 2024.10.22 00:41:53.363432 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 021f3d7b8248319bde6b26d130243377, offset: 0, size: 48550 2024.10.22 00:41:53.366282 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 021f3d7b8248319bde6b26d130243377 for removal 2024.10.22 00:41:53.367297 [ 1476 ] {} CacheMetadata: Key 021f3d7b8248319bde6b26d130243377 is removed from metadata 2024.10.22 00:41:53.368115 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 414f2f333bb2800c676a1cb607b2aedb, offset: 0, size: 154 2024.10.22 00:41:53.370213 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 414f2f333bb2800c676a1cb607b2aedb for removal 2024.10.22 00:41:53.371243 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 367e20847ca5c613708a818a2483bf01, offset: 0, size: 58410 2024.10.22 00:41:53.373298 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 367e20847ca5c613708a818a2483bf01 for removal 2024.10.22 00:41:53.375096 [ 1476 ] {} CacheMetadata: Key 414f2f333bb2800c676a1cb607b2aedb is removed from metadata 2024.10.22 00:41:53.376146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d4875c1da34900563273c21fe804a26f, offset: 0, size: 154 2024.10.22 00:41:53.378049 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d4875c1da34900563273c21fe804a26f for removal 2024.10.22 00:41:53.378433 [ 1476 ] {} CacheMetadata: Key 367e20847ca5c613708a818a2483bf01 is removed from metadata 2024.10.22 00:41:53.380956 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 337b82dbac70d4333bfba6ed8a15efd8, offset: 0, size: 47353 2024.10.22 00:41:53.384432 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 337b82dbac70d4333bfba6ed8a15efd8 for removal 2024.10.22 00:41:53.384435 [ 1476 ] {} CacheMetadata: Key d4875c1da34900563273c21fe804a26f is removed from metadata 2024.10.22 00:41:53.387716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b1676f9ed2c473b3f632ee52c94eb794, offset: 0, size: 154 2024.10.22 00:41:53.388359 [ 1476 ] {} CacheMetadata: Key 337b82dbac70d4333bfba6ed8a15efd8 is removed from metadata 2024.10.22 00:41:53.392769 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b1676f9ed2c473b3f632ee52c94eb794 for removal 2024.10.22 00:41:53.398758 [ 1476 ] {} CacheMetadata: Key b1676f9ed2c473b3f632ee52c94eb794 is removed from metadata 2024.10.22 00:41:53.731190 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} MergeTask::MergeProjectionsStage: Merge sorted 186 rows, containing 32 columns (32 merged, 0 gathered) in 2.174532881 sec., 85.53561163649287 rows/sec., 9.32 KiB/sec. 2024.10.22 00:41:53.760351 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} MergedBlockOutputStream: filled checksums 202410_1_142_29 (state Temporary) 2024.10.22 00:41:53.778751 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_105_21, 202410_106_106_0, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0] 2024.10.22 00:41:53.780813 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_105_21, 202410_106_106_0, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0] 2024.10.22 00:41:53.923140 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_105_21, 202410_106_106_0, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0] 2024.10.22 00:41:54.070183 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 164385 bytes for 271f68104e974afb2e10145359871f40:0) 2024.10.22 00:41:54.072146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1a50d29a6f0f66806a7e40bc31c2ce18, offset: 0, size: 146 2024.10.22 00:41:54.074628 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1a50d29a6f0f66806a7e40bc31c2ce18 for removal 2024.10.22 00:41:54.077194 [ 1476 ] {} CacheMetadata: Key 1a50d29a6f0f66806a7e40bc31c2ce18 is removed from metadata 2024.10.22 00:41:54.077710 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5193d1f1e8d3d461547ce7b3986c510f, offset: 0, size: 46208 2024.10.22 00:41:54.081834 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5193d1f1e8d3d461547ce7b3986c510f for removal 2024.10.22 00:41:54.085245 [ 1476 ] {} CacheMetadata: Key 5193d1f1e8d3d461547ce7b3986c510f is removed from metadata 2024.10.22 00:41:54.085446 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c5471cb08b83cfe08feff27a7e059ed4, offset: 0, size: 154 2024.10.22 00:41:54.090490 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c5471cb08b83cfe08feff27a7e059ed4 for removal 2024.10.22 00:41:54.095616 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0ecf0c776600d516d9b73cce2cefa09a, offset: 0, size: 160781 2024.10.22 00:41:54.100952 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0ecf0c776600d516d9b73cce2cefa09a for removal 2024.10.22 00:41:54.110871 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3a80741e2bd1db2fdaf6e2ccd7bd9572, offset: 0, size: 70403 2024.10.22 00:41:54.115195 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3a80741e2bd1db2fdaf6e2ccd7bd9572 for removal 2024.10.22 00:41:54.118477 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 369aea2cdfdd4465cb46cd710d77477b, offset: 0, size: 164 2024.10.22 00:41:54.121224 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 369aea2cdfdd4465cb46cd710d77477b for removal 2024.10.22 00:41:54.195462 [ 1476 ] {} CacheMetadata: Key c5471cb08b83cfe08feff27a7e059ed4 is removed from metadata 2024.10.22 00:41:54.202544 [ 1476 ] {} CacheMetadata: Key 369aea2cdfdd4465cb46cd710d77477b is removed from metadata 2024.10.22 00:41:54.209428 [ 1476 ] {} CacheMetadata: Key 3a80741e2bd1db2fdaf6e2ccd7bd9572 is removed from metadata 2024.10.22 00:41:54.220556 [ 1476 ] {} CacheMetadata: Key 0ecf0c776600d516d9b73cce2cefa09a is removed from metadata 2024.10.22 00:41:54.322625 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.88 GiB, free memory in arenas 9.32 MiB, will set to 1.40 GiB (RSS), difference: -4.81 MiB 2024.10.22 00:41:54.455205 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_142_29 to 202410_1_142_29 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:54.460542 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_142_29} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_137_28, 202410_142_142_0] -> 202410_1_142_29 2024.10.22 00:41:54.662670 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} MergeTask::MergeProjectionsStage: Merge sorted 16373 rows, containing 13 columns (13 merged, 0 gathered) in 2.266654438 sec., 7223.421323299215 rows/sec., 1.07 MiB/sec. 2024.10.22 00:41:54.667835 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 10 file segments (while reserving 345773 bytes for 1dae2dd476fd3c2811ad56a3bc97d4d0:0) 2024.10.22 00:41:54.670646 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 63099c9f909abd314c57d359a75af6a5, offset: 0, size: 146962 2024.10.22 00:41:54.672315 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 63099c9f909abd314c57d359a75af6a5 for removal 2024.10.22 00:41:54.674220 [ 1476 ] {} CacheMetadata: Key 63099c9f909abd314c57d359a75af6a5 is removed from metadata 2024.10.22 00:41:54.674328 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cfd51d51242b3bb6895013f8382a8c44, offset: 0, size: 146 2024.10.22 00:41:54.677306 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cfd51d51242b3bb6895013f8382a8c44 for removal 2024.10.22 00:41:54.678960 [ 1476 ] {} CacheMetadata: Key cfd51d51242b3bb6895013f8382a8c44 is removed from metadata 2024.10.22 00:41:54.679744 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f7c951f9d7b03d2004c1792c45305c5d, offset: 0, size: 35063 2024.10.22 00:41:54.681108 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:41:54.682052 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f7c951f9d7b03d2004c1792c45305c5d for removal 2024.10.22 00:41:54.683394 [ 1476 ] {} CacheMetadata: Key f7c951f9d7b03d2004c1792c45305c5d is removed from metadata 2024.10.22 00:41:54.683752 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 74607e4d134f445cba49257f4869b62e, offset: 0, size: 142 2024.10.22 00:41:54.689479 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 74607e4d134f445cba49257f4869b62e for removal 2024.10.22 00:41:54.692534 [ 1476 ] {} CacheMetadata: Key 74607e4d134f445cba49257f4869b62e is removed from metadata 2024.10.22 00:41:54.694717 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d4728658f4d0987fff504d168bd1546e, offset: 0, size: 142 2024.10.22 00:41:54.702938 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d4728658f4d0987fff504d168bd1546e for removal 2024.10.22 00:41:54.708553 [ 1476 ] {} CacheMetadata: Key d4728658f4d0987fff504d168bd1546e is removed from metadata 2024.10.22 00:41:54.714500 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8ad504f963fed8f6c88ee674ebc35df9, offset: 0, size: 164 2024.10.22 00:41:54.721584 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8ad504f963fed8f6c88ee674ebc35df9 for removal 2024.10.22 00:41:54.724891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 46a4391e2c3f6f3246506472945b7ada, offset: 0, size: 4964 2024.10.22 00:41:54.725102 [ 1476 ] {} CacheMetadata: Key 8ad504f963fed8f6c88ee674ebc35df9 is removed from metadata 2024.10.22 00:41:54.731488 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 46a4391e2c3f6f3246506472945b7ada for removal 2024.10.22 00:41:54.735926 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 810a8b60101a029fb4086117debea4c7, offset: 0, size: 66509 2024.10.22 00:41:54.827478 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} MergedBlockOutputStream: filled checksums 202410_1_295_59 (state Temporary) 2024.10.22 00:41:54.839212 [ 1476 ] {} CacheMetadata: Key 46a4391e2c3f6f3246506472945b7ada is removed from metadata 2024.10.22 00:41:54.841607 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 810a8b60101a029fb4086117debea4c7 for removal 2024.10.22 00:41:54.847066 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9e507e91e6d8f1dab4afd0b9461a9184, offset: 0, size: 40591 2024.10.22 00:41:54.849540 [ 1476 ] {} CacheMetadata: Key 810a8b60101a029fb4086117debea4c7 is removed from metadata 2024.10.22 00:41:54.884598 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_295_59 to 202410_1_295_59 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:54.887179 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_290_58, 202410_295_295_0] -> 202410_1_295_59 2024.10.22 00:41:54.919999 [ 1398 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_295_59}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 00:41:54.946010 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.15 MiB. 2024.10.22 00:41:54.954264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9e507e91e6d8f1dab4afd0b9461a9184 for removal 2024.10.22 00:41:54.959239 [ 1476 ] {} CacheMetadata: Key 9e507e91e6d8f1dab4afd0b9461a9184 is removed from metadata 2024.10.22 00:41:54.960174 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1b2083af85fd72087b0a1c3dc834f747, offset: 0, size: 144 2024.10.22 00:41:54.967198 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1b2083af85fd72087b0a1c3dc834f747 for removal 2024.10.22 00:41:54.969679 [ 1476 ] {} CacheMetadata: Key 1b2083af85fd72087b0a1c3dc834f747 is removed from metadata 2024.10.22 00:41:55.295149 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.88 GiB, free memory in arenas 9.50 MiB, will set to 1.40 GiB (RSS), difference: 8.00 MiB 2024.10.22 00:41:55.388689 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 386320 bytes for 43fb93bc82e797c059e140f806fca972:0) 2024.10.22 00:41:55.390136 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3463583b60fac8cedde1e1326f1b02a5, offset: 0, size: 599639 2024.10.22 00:41:55.393375 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3463583b60fac8cedde1e1326f1b02a5 for removal 2024.10.22 00:41:55.396878 [ 1476 ] {} CacheMetadata: Key 3463583b60fac8cedde1e1326f1b02a5 is removed from metadata 2024.10.22 00:41:55.397617 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: edf91a67ca0cbe2c106dca03ddfb5169, offset: 0, size: 155 2024.10.22 00:41:55.404629 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key edf91a67ca0cbe2c106dca03ddfb5169 for removal 2024.10.22 00:41:55.408746 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a6863c6c213a9cc2ced9c91ae2ce336e, offset: 0, size: 249188 2024.10.22 00:41:55.412687 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a6863c6c213a9cc2ced9c91ae2ce336e for removal 2024.10.22 00:41:55.421401 [ 1476 ] {} CacheMetadata: Key a6863c6c213a9cc2ced9c91ae2ce336e is removed from metadata 2024.10.22 00:41:55.416725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1fe1b2a1a7e062885d6924405c668d8f, offset: 0, size: 147 2024.10.22 00:41:55.430265 [ 1476 ] {} CacheMetadata: Key edf91a67ca0cbe2c106dca03ddfb5169 is removed from metadata 2024.10.22 00:41:55.430794 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1fe1b2a1a7e062885d6924405c668d8f for removal 2024.10.22 00:41:55.441832 [ 1476 ] {} CacheMetadata: Key 1fe1b2a1a7e062885d6924405c668d8f is removed from metadata 2024.10.22 00:41:55.571896 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 2320 2024.10.22 00:41:55.765498 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:55.769548 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:55.891897 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_165_165_0 (state Temporary) 2024.10.22 00:41:55.941284 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1886 entries to flush up to offset 274894 2024.10.22 00:41:55.950529 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_165_165_0 to 202410_178_178_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:55.969720 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for a375da93c91235b6252fca985c4da76a:0) 2024.10.22 00:41:55.971283 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4e7bed7edb030644381475530b6ece8a, offset: 0, size: 149 2024.10.22 00:41:55.973270 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4e7bed7edb030644381475530b6ece8a for removal 2024.10.22 00:41:55.975848 [ 1476 ] {} CacheMetadata: Key 4e7bed7edb030644381475530b6ece8a is removed from metadata 2024.10.22 00:41:55.976034 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 274ac02d7ecaca9d55bf4233aad439fe, offset: 0, size: 290868 2024.10.22 00:41:55.978754 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 274ac02d7ecaca9d55bf4233aad439fe for removal 2024.10.22 00:41:55.980385 [ 1476 ] {} CacheMetadata: Key 274ac02d7ecaca9d55bf4233aad439fe is removed from metadata 2024.10.22 00:41:55.980509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7f757fc4e24e1278857aae5af2ba61e4, offset: 0, size: 592024 2024.10.22 00:41:55.984578 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7f757fc4e24e1278857aae5af2ba61e4 for removal 2024.10.22 00:41:55.986915 [ 1476 ] {} CacheMetadata: Key 7f757fc4e24e1278857aae5af2ba61e4 is removed from metadata 2024.10.22 00:41:55.988085 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3526b8d2c123c0c17ba5f3c2d0394367, offset: 0, size: 155 2024.10.22 00:41:55.991240 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3526b8d2c123c0c17ba5f3c2d0394367 for removal 2024.10.22 00:41:55.994321 [ 1476 ] {} CacheMetadata: Key 3526b8d2c123c0c17ba5f3c2d0394367 is removed from metadata 2024.10.22 00:41:56.090510 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2320 2024.10.22 00:41:56.227822 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 1000934 bytes for a375da93c91235b6252fca985c4da76a:0) 2024.10.22 00:41:56.232722 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fde5c7c8d0f17cf8559ce6dfd389c1d9, offset: 0, size: 381310 2024.10.22 00:41:56.243104 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fde5c7c8d0f17cf8559ce6dfd389c1d9 for removal 2024.10.22 00:41:56.249277 [ 1476 ] {} CacheMetadata: Key fde5c7c8d0f17cf8559ce6dfd389c1d9 is removed from metadata 2024.10.22 00:41:56.250053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cb97bc9f6db83d1c368544af8ddb9d04, offset: 0, size: 149 2024.10.22 00:41:56.255847 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cb97bc9f6db83d1c368544af8ddb9d04 for removal 2024.10.22 00:41:56.263431 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0b6d5efc621fa416e1f62a197d98459d, offset: 0, size: 153 2024.10.22 00:41:56.263584 [ 1476 ] {} CacheMetadata: Key cb97bc9f6db83d1c368544af8ddb9d04 is removed from metadata 2024.10.22 00:41:56.269066 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0b6d5efc621fa416e1f62a197d98459d for removal 2024.10.22 00:41:56.271919 [ 1476 ] {} CacheMetadata: Key 0b6d5efc621fa416e1f62a197d98459d is removed from metadata 2024.10.22 00:41:56.271913 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ed092db5b5a21752271da2583144ccd2, offset: 0, size: 472964 2024.10.22 00:41:56.275029 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ed092db5b5a21752271da2583144ccd2 for removal 2024.10.22 00:41:56.278913 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3cabb371b84730b774392888721785e6, offset: 0, size: 175039 2024.10.22 00:41:56.281791 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3cabb371b84730b774392888721785e6 for removal 2024.10.22 00:41:56.284900 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6d6a95d4e2665795b6ca97d1c39752d4, offset: 0, size: 146 2024.10.22 00:41:56.287621 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6d6a95d4e2665795b6ca97d1c39752d4 for removal 2024.10.22 00:41:56.299042 [ 1476 ] {} CacheMetadata: Key 6d6a95d4e2665795b6ca97d1c39752d4 is removed from metadata 2024.10.22 00:41:56.312376 [ 1476 ] {} CacheMetadata: Key 3cabb371b84730b774392888721785e6 is removed from metadata 2024.10.22 00:41:56.324462 [ 1476 ] {} CacheMetadata: Key ed092db5b5a21752271da2583144ccd2 is removed from metadata 2024.10.22 00:41:56.393037 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 170 entries to flush up to offset 15712 2024.10.22 00:41:57.200193 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:57.212119 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:57.347364 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:57.357690 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:57.397493 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_283_283_0 (state Temporary) 2024.10.22 00:41:57.467238 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_283_283_0 to 202410_296_296_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:57.489539 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_158_158_0 (state Temporary) 2024.10.22 00:41:57.568870 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_158_158_0 to 202410_165_165_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:57.593259 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 274894 2024.10.22 00:41:57.614426 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15712 2024.10.22 00:41:57.653802 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 172736 bytes for 0d005f02f4fc666aa71e3fb619caf304:0) 2024.10.22 00:41:57.660299 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4179100ece592b72e10b533f0e93ae83, offset: 0, size: 51370 2024.10.22 00:41:57.666688 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4179100ece592b72e10b533f0e93ae83 for removal 2024.10.22 00:41:57.668986 [ 1476 ] {} CacheMetadata: Key 4179100ece592b72e10b533f0e93ae83 is removed from metadata 2024.10.22 00:41:57.669641 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bded500d387d08bf8482a2e39fe3ea2b, offset: 0, size: 142 2024.10.22 00:41:57.672317 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bded500d387d08bf8482a2e39fe3ea2b for removal 2024.10.22 00:41:57.675741 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 70065ebe3c56752c094251fabbcb43af, offset: 0, size: 142 2024.10.22 00:41:57.678599 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 70065ebe3c56752c094251fabbcb43af for removal 2024.10.22 00:41:57.681309 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1c80737a20ac165367e8c4e23c617bd1, offset: 0, size: 63147 2024.10.22 00:41:57.686389 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1c80737a20ac165367e8c4e23c617bd1 for removal 2024.10.22 00:41:57.690162 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e443fc660f6aa592dd940908f1fe56df, offset: 0, size: 63538 2024.10.22 00:41:57.692501 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e443fc660f6aa592dd940908f1fe56df for removal 2024.10.22 00:41:57.700053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6b7d6e9fc1554858735cf09555a1c1ee, offset: 0, size: 142 2024.10.22 00:41:57.704248 [ 1476 ] {} CacheMetadata: Key e443fc660f6aa592dd940908f1fe56df is removed from metadata 2024.10.22 00:41:57.704563 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6b7d6e9fc1554858735cf09555a1c1ee for removal 2024.10.22 00:41:57.721557 [ 1476 ] {} CacheMetadata: Key 6b7d6e9fc1554858735cf09555a1c1ee is removed from metadata 2024.10.22 00:41:57.732738 [ 1476 ] {} CacheMetadata: Key 1c80737a20ac165367e8c4e23c617bd1 is removed from metadata 2024.10.22 00:41:57.746469 [ 1476 ] {} CacheMetadata: Key 70065ebe3c56752c094251fabbcb43af is removed from metadata 2024.10.22 00:41:57.760918 [ 1476 ] {} CacheMetadata: Key bded500d387d08bf8482a2e39fe3ea2b is removed from metadata 2024.10.22 00:41:58.246501 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:41:58.278361 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 296994 bytes for 2e209366b79dc36b74229bd5bb1281ae:0) 2024.10.22 00:41:58.279474 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5830508834a688c25f9072841ca6cbbf, offset: 0, size: 403823 2024.10.22 00:41:58.285090 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5830508834a688c25f9072841ca6cbbf for removal 2024.10.22 00:41:58.288573 [ 1476 ] {} CacheMetadata: Key 5830508834a688c25f9072841ca6cbbf is removed from metadata 2024.10.22 00:41:58.288573 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1d1199ffce5649429455c176ad2d4669, offset: 0, size: 151 2024.10.22 00:41:58.293514 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1d1199ffce5649429455c176ad2d4669 for removal 2024.10.22 00:41:58.297371 [ 1476 ] {} CacheMetadata: Key 1d1199ffce5649429455c176ad2d4669 is removed from metadata 2024.10.22 00:41:58.679630 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_191_38, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0] 2024.10.22 00:41:58.681438 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_191_38, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0] 2024.10.22 00:41:58.682784 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RefererCategories 2024.10.22 00:41:58.707516 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RefererCategories 2024.10.22 00:41:58.710959 [ 1699 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rpg/tcfjgywptvvrystypquzggxsekbow, size 15581 2024.10.22 00:41:58.730125 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RefererCategories 2024.10.22 00:41:58.762552 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RefererCategories 2024.10.22 00:41:58.790107 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RefererCategories 2024.10.22 00:41:58.816223 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RefererCategories 2024.10.22 00:41:58.824509 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 63 entries to flush up to offset 10735 2024.10.22 00:41:58.859976 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RefererCategories 2024.10.22 00:41:58.904623 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_191_38, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0] 2024.10.22 00:41:58.971519 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 147763 bytes for 775ac2ab5e31a34de716c752f20fae38:0) 2024.10.22 00:41:59.037257 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/sps/hsoindaragffpvwnrpqruyvucvpme (LogSeriesLimiter: on interval from 2024-10-22 00:41:41 to 2024-10-22 00:41:59 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:41:59.075553 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 110ba3bd63b8314e31b6e96747ef3166, offset: 0, size: 397840 2024.10.22 00:41:59.077625 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 110ba3bd63b8314e31b6e96747ef3166 for removal 2024.10.22 00:41:59.079801 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: af7298693229f40a21b7f8398d582e86, offset: 0, size: 152 2024.10.22 00:41:59.082409 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key af7298693229f40a21b7f8398d582e86 for removal 2024.10.22 00:41:59.101793 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:41:59.106988 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:41:59.170583 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_164_164_0 (state Temporary) 2024.10.22 00:41:59.180325 [ 1476 ] {} CacheMetadata: Key af7298693229f40a21b7f8398d582e86 is removed from metadata 2024.10.22 00:41:59.257388 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_164_164_0 to 202410_170_170_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:41:59.291022 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10735 2024.10.22 00:41:59.308668 [ 1476 ] {} CacheMetadata: Key 110ba3bd63b8314e31b6e96747ef3166 is removed from metadata 2024.10.22 00:41:59.462703 [ 1222 ] {} DatabaseCatalog: Execution took 420 ms. 2024.10.22 00:41:59.550177 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:41:59.553764 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:41:59.597461 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 681616 bytes for 9d491f91eef09af81228c1edfd9475f1:0) 2024.10.22 00:41:59.616966 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0f48e36422180e05783989d8616e401d, offset: 0, size: 974317 2024.10.22 00:41:59.628519 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0f48e36422180e05783989d8616e401d for removal 2024.10.22 00:41:59.633485 [ 1476 ] {} CacheMetadata: Key 0f48e36422180e05783989d8616e401d is removed from metadata 2024.10.22 00:41:59.634025 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f13fe973e1d12a5a0a16f944a3e9b60f, offset: 0, size: 1268 2024.10.22 00:41:59.637692 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f13fe973e1d12a5a0a16f944a3e9b60f for removal 2024.10.22 00:41:59.642984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4f23a189311a8f56711087781dcfc6df, offset: 0, size: 188 2024.10.22 00:41:59.648352 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4f23a189311a8f56711087781dcfc6df for removal 2024.10.22 00:41:59.655395 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 10.56 MiB, will set to 1.40 GiB (RSS), difference: -5.95 MiB 2024.10.22 00:41:59.656158 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 42a388eea2b5cfeda9eba1d350dcceb1, offset: 0, size: 154 2024.10.22 00:41:59.659532 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 42a388eea2b5cfeda9eba1d350dcceb1 for removal 2024.10.22 00:41:59.660139 [ 1476 ] {} CacheMetadata: Key 4f23a189311a8f56711087781dcfc6df is removed from metadata 2024.10.22 00:41:59.667976 [ 1476 ] {} CacheMetadata: Key 42a388eea2b5cfeda9eba1d350dcceb1 is removed from metadata 2024.10.22 00:41:59.676095 [ 1476 ] {} CacheMetadata: Key f13fe973e1d12a5a0a16f944a3e9b60f is removed from metadata 2024.10.22 00:42:00.079180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 971803485c923501475fcf71babe9c4a:0) 2024.10.22 00:42:00.081366 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 87a7ec36760fc4e2d79de4cdc673a449, offset: 0, size: 1449189 2024.10.22 00:42:00.085088 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 87a7ec36760fc4e2d79de4cdc673a449 for removal 2024.10.22 00:42:00.087310 [ 1476 ] {} CacheMetadata: Key 87a7ec36760fc4e2d79de4cdc673a449 is removed from metadata 2024.10.22 00:42:00.088052 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2e170b90a31798e4bff5c269de58c726, offset: 0, size: 196 2024.10.22 00:42:00.092637 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2e170b90a31798e4bff5c269de58c726 for removal 2024.10.22 00:42:00.097551 [ 1476 ] {} CacheMetadata: Key 2e170b90a31798e4bff5c269de58c726 is removed from metadata 2024.10.22 00:42:00.555718 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_33_33_0 to 202410_36_36_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:00.630551 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_34_7/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:42:00.967340 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 178930 bytes for bbf75c07e3841935f9b7ef80e9e93fa8:0) 2024.10.22 00:42:00.968669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1cb9fb46bb1b4387bea7f5ade9b35254, offset: 0, size: 443053 2024.10.22 00:42:00.970499 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1cb9fb46bb1b4387bea7f5ade9b35254 for removal 2024.10.22 00:42:00.972362 [ 1476 ] {} CacheMetadata: Key 1cb9fb46bb1b4387bea7f5ade9b35254 is removed from metadata 2024.10.22 00:42:00.972543 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f3e73d6e2c2db93b4d7133f43f7e5596, offset: 0, size: 152 2024.10.22 00:42:00.974626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f3e73d6e2c2db93b4d7133f43f7e5596 for removal 2024.10.22 00:42:00.984551 [ 1476 ] {} CacheMetadata: Key f3e73d6e2c2db93b4d7133f43f7e5596 is removed from metadata 2024.10.22 00:42:01.618786 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 5.43 MiB, will set to 1.40 GiB (RSS), difference: -1.07 MiB 2024.10.22 00:42:01.660998 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 15808 2024.10.22 00:42:01.776570 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 9a642bdfd1a0013c531965b90d449890:0) 2024.10.22 00:42:01.778333 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 220034b850efd45e8438680b624e8bf6, offset: 0, size: 4278235 2024.10.22 00:42:01.784453 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 220034b850efd45e8438680b624e8bf6 for removal 2024.10.22 00:42:01.787163 [ 1476 ] {} CacheMetadata: Key 220034b850efd45e8438680b624e8bf6 is removed from metadata 2024.10.22 00:42:01.787159 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 070baad7778634b769dc28f4c8b25f1b, offset: 0, size: 274 2024.10.22 00:42:01.793115 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 070baad7778634b769dc28f4c8b25f1b for removal 2024.10.22 00:42:01.802320 [ 1476 ] {} CacheMetadata: Key 070baad7778634b769dc28f4c8b25f1b is removed from metadata 2024.10.22 00:42:01.804153 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bd7aa1f8f813a6d6f7e1ffc07da3ff3a, offset: 0, size: 40280 2024.10.22 00:42:01.816954 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bd7aa1f8f813a6d6f7e1ffc07da3ff3a for removal 2024.10.22 00:42:01.821567 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 595b8bb68918686a68635c0c783e5de5, offset: 0, size: 154 2024.10.22 00:42:01.824722 [ 1476 ] {} CacheMetadata: Key bd7aa1f8f813a6d6f7e1ffc07da3ff3a is removed from metadata 2024.10.22 00:42:01.931274 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 595b8bb68918686a68635c0c783e5de5 for removal 2024.10.22 00:42:01.937077 [ 1476 ] {} CacheMetadata: Key 595b8bb68918686a68635c0c783e5de5 is removed from metadata 2024.10.22 00:42:01.968934 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:01.971016 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:42:02.021639 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_284_284_0 (state Temporary) 2024.10.22 00:42:02.168603 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_284_284_0 to 202410_297_297_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:02.222063 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15808 2024.10.22 00:42:02.468055 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} MergeTask::MergeProjectionsStage: Merge sorted 1292 rows, containing 679 columns (679 merged, 0 gathered) in 83.40153515 sec., 15.491321564720621 rows/sec., 20.70 KiB/sec. 2024.10.22 00:42:03.595335 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 2326 2024.10.22 00:42:03.616331 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 6.05 MiB, will set to 1.40 GiB (RSS), difference: 4.87 MiB 2024.10.22 00:42:03.725000 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:03.728215 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.68 GiB. 2024.10.22 00:42:03.865782 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_166_166_0 (state Temporary) 2024.10.22 00:42:03.923440 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1308 2024.10.22 00:42:04.158856 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} MergedBlockOutputStream: filled checksums 202410_1_34_7 (state Temporary) 2024.10.22 00:42:04.237397 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_166_166_0 to 202410_179_179_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:04.286652 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2326 2024.10.22 00:42:04.604575 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1746 entries to flush up to offset 276640 2024.10.22 00:42:05.777073 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 523495 bytes for d4a31693302a1be3e97c9348ba0a3df7:0) 2024.10.22 00:42:05.778679 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 995d320a2aee64bec80ceb8017ce2cc9, offset: 0, size: 2017387 2024.10.22 00:42:05.783007 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 995d320a2aee64bec80ceb8017ce2cc9 for removal 2024.10.22 00:42:05.785508 [ 1476 ] {} CacheMetadata: Key 995d320a2aee64bec80ceb8017ce2cc9 is removed from metadata 2024.10.22 00:42:05.785379 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4009303814c5578c06c1d38e449c5374, offset: 0, size: 258 2024.10.22 00:42:05.790989 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4009303814c5578c06c1d38e449c5374 for removal 2024.10.22 00:42:05.802426 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c9e965681a8c8d37704f1e2d43ea8846, offset: 0, size: 186 2024.10.22 00:42:05.811602 [ 1476 ] {} CacheMetadata: Key 4009303814c5578c06c1d38e449c5374 is removed from metadata 2024.10.22 00:42:05.812473 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c9e965681a8c8d37704f1e2d43ea8846 for removal 2024.10.22 00:42:05.816190 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2ea3a904203c3b6aae7facff815c6f1d, offset: 0, size: 104131 2024.10.22 00:42:05.818509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2ea3a904203c3b6aae7facff815c6f1d for removal 2024.10.22 00:42:05.819239 [ 1476 ] {} CacheMetadata: Key c9e965681a8c8d37704f1e2d43ea8846 is removed from metadata 2024.10.22 00:42:05.821478 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3d3103019f411765f2dbe8587878b412, offset: 0, size: 4789 2024.10.22 00:42:05.826226 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3d3103019f411765f2dbe8587878b412 for removal 2024.10.22 00:42:05.828787 [ 1476 ] {} CacheMetadata: Key 2ea3a904203c3b6aae7facff815c6f1d is removed from metadata 2024.10.22 00:42:05.833090 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 41b3186bc4d5b9b78b28b1cc8c75ae98, offset: 0, size: 162 2024.10.22 00:42:05.840117 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 41b3186bc4d5b9b78b28b1cc8c75ae98 for removal 2024.10.22 00:42:05.844401 [ 1476 ] {} CacheMetadata: Key 41b3186bc4d5b9b78b28b1cc8c75ae98 is removed from metadata 2024.10.22 00:42:05.856212 [ 1476 ] {} CacheMetadata: Key 3d3103019f411765f2dbe8587878b412 is removed from metadata 2024.10.22 00:42:05.863962 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:05.871457 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:06.082765 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.164805848 sec. 2024.10.22 00:42:06.084470 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_159_159_0 (state Temporary) 2024.10.22 00:42:06.233802 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 15862 2024.10.22 00:42:06.280745 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_159_159_0 to 202410_166_166_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:06.379979 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:06.382758 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:06.408755 [ 1249 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_161_32 to 202410_166_166_0 2024.10.22 00:42:06.409206 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 276640 2024.10.22 00:42:06.437772 [ 1249 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:06.442944 [ 1249 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:06.470759 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_161_32 to 202410_166_166_0 into Compact with storage Full 2024.10.22 00:42:06.516876 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 151 2024.10.22 00:42:06.563200 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_285_285_0 (state Temporary) 2024.10.22 00:42:06.584984 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:42:06.591025 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} MergeTreeSequentialSource: Reading 36 marks from part 202410_1_161_32, total 281014 rows starting from the beginning of the part 2024.10.22 00:42:06.597293 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_162_162_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:42:06.602561 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_285_285_0 to 202410_298_298_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:06.614244 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_163_163_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:42:06.628494 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_164_164_0, total 1744 rows starting from the beginning of the part 2024.10.22 00:42:06.653339 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_165_165_0, total 1886 rows starting from the beginning of the part 2024.10.22 00:42:06.671977 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_166_166_0, total 1746 rows starting from the beginning of the part 2024.10.22 00:42:06.717578 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15862 2024.10.22 00:42:06.802203 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 55 entries to flush up to offset 10790 2024.10.22 00:42:07.115571 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:07.120184 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:07.169686 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_165_165_0 (state Temporary) 2024.10.22 00:42:07.292081 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_165_165_0 to 202410_171_171_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:07.348585 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10790 2024.10.22 00:42:07.376714 [ 1242 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:42:07.407672 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:07.414717 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:07.416183 [ 1242 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:42:07.373667 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 7.53 MiB, will set to 1.39 GiB (RSS), difference: -6.81 MiB 2024.10.22 00:42:07.661909 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_133_133_0 (state Temporary) 2024.10.22 00:42:07.750714 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_133_133_0 to 202410_143_143_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:07.951371 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 151 2024.10.22 00:42:07.986483 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 1009484 bytes for e82df546095a90ca463b83ad73f90e0d:0) 2024.10.22 00:42:07.988027 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7749d04b5fae7df26d159e1ecab0f6de, offset: 0, size: 573236 2024.10.22 00:42:07.990147 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7749d04b5fae7df26d159e1ecab0f6de for removal 2024.10.22 00:42:07.992334 [ 1476 ] {} CacheMetadata: Key 7749d04b5fae7df26d159e1ecab0f6de is removed from metadata 2024.10.22 00:42:07.992648 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0b195f639e0dee9dce988c8c1841a4d8, offset: 0, size: 238 2024.10.22 00:42:07.998910 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0b195f639e0dee9dce988c8c1841a4d8 for removal 2024.10.22 00:42:08.006235 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 63daf6c522ef0b7d87e7ffe48be7212c, offset: 0, size: 224 2024.10.22 00:42:08.015591 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 63daf6c522ef0b7d87e7ffe48be7212c for removal 2024.10.22 00:42:08.018490 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 88818bb666aae4885f0de933829cec27, offset: 0, size: 79785 2024.10.22 00:42:08.019810 [ 1476 ] {} CacheMetadata: Key 63daf6c522ef0b7d87e7ffe48be7212c is removed from metadata 2024.10.22 00:42:08.021716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 88818bb666aae4885f0de933829cec27 for removal 2024.10.22 00:42:08.024957 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ccd7625518c57fa84d4dcb3a1d4b2016, offset: 0, size: 60813 2024.10.22 00:42:08.026582 [ 1476 ] {} CacheMetadata: Key 88818bb666aae4885f0de933829cec27 is removed from metadata 2024.10.22 00:42:08.031964 [ 1476 ] {} CacheMetadata: Key 0b195f639e0dee9dce988c8c1841a4d8 is removed from metadata 2024.10.22 00:42:08.128461 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ccd7625518c57fa84d4dcb3a1d4b2016 for removal 2024.10.22 00:42:08.133159 [ 1476 ] {} CacheMetadata: Key ccd7625518c57fa84d4dcb3a1d4b2016 is removed from metadata 2024.10.22 00:42:08.134604 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e06ed7356a6b96377f21913fa88e5467, offset: 0, size: 188 2024.10.22 00:42:08.140385 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e06ed7356a6b96377f21913fa88e5467 for removal 2024.10.22 00:42:08.142439 [ 1476 ] {} CacheMetadata: Key e06ed7356a6b96377f21913fa88e5467 is removed from metadata 2024.10.22 00:42:08.885016 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 657386 bytes for 502d2c78448b54d19b33fde64d6ef0f1:0) 2024.10.22 00:42:08.891138 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 723ff4671bf16a6cdeb6fecab3397e2a, offset: 0, size: 810686 2024.10.22 00:42:08.899972 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 723ff4671bf16a6cdeb6fecab3397e2a for removal 2024.10.22 00:42:09.005216 [ 1476 ] {} CacheMetadata: Key 723ff4671bf16a6cdeb6fecab3397e2a is removed from metadata 2024.10.22 00:42:09.006178 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cf2b2ff2d4faedfc88c255d52e5e8d70, offset: 0, size: 228 2024.10.22 00:42:09.014398 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cf2b2ff2d4faedfc88c255d52e5e8d70 for removal 2024.10.22 00:42:09.017241 [ 1476 ] {} CacheMetadata: Key cf2b2ff2d4faedfc88c255d52e5e8d70 is removed from metadata 2024.10.22 00:42:09.019165 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1c943d05d8335e972533a27a84b94441, offset: 0, size: 370683 2024.10.22 00:42:09.023125 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1c943d05d8335e972533a27a84b94441 for removal 2024.10.22 00:42:09.027624 [ 1476 ] {} CacheMetadata: Key 1c943d05d8335e972533a27a84b94441 is removed from metadata 2024.10.22 00:42:09.028672 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f9729529d1930204f62f77f2de1f06ab, offset: 0, size: 233 2024.10.22 00:42:09.042205 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f9729529d1930204f62f77f2de1f06ab for removal 2024.10.22 00:42:09.047622 [ 1476 ] {} CacheMetadata: Key f9729529d1930204f62f77f2de1f06ab is removed from metadata 2024.10.22 00:42:09.553612 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:42:09.557497 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:42:09.786963 [ 1273 ] {} DatabaseCatalog: Execution took 322 ms. 2024.10.22 00:42:10.025658 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.88 GiB, free memory in arenas 10.82 MiB, will set to 1.40 GiB (RSS), difference: 6.05 MiB 2024.10.22 00:42:10.373772 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 12 file segments (while reserving 1048576 bytes for 258b4ab77e04857fb74fc7d5ffd4b711:0) 2024.10.22 00:42:10.376461 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dd86988838b9917d635645df6c140851, offset: 0, size: 123822 2024.10.22 00:42:10.379893 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dd86988838b9917d635645df6c140851 for removal 2024.10.22 00:42:10.383146 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d55c37dfc417d78e414db2c7c8fb2487, offset: 0, size: 258327 2024.10.22 00:42:10.383576 [ 1476 ] {} CacheMetadata: Key dd86988838b9917d635645df6c140851 is removed from metadata 2024.10.22 00:42:10.385617 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d55c37dfc417d78e414db2c7c8fb2487 for removal 2024.10.22 00:42:10.389234 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e7a3a40d8e096d1e7efd53391fe2552e, offset: 0, size: 191 2024.10.22 00:42:10.403103 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e7a3a40d8e096d1e7efd53391fe2552e for removal 2024.10.22 00:42:10.408391 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b740edbe388279d32a3661bd26d9be02, offset: 0, size: 150406 2024.10.22 00:42:10.415340 [ 1476 ] {} CacheMetadata: Key e7a3a40d8e096d1e7efd53391fe2552e is removed from metadata 2024.10.22 00:42:10.417332 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b740edbe388279d32a3661bd26d9be02 for removal 2024.10.22 00:42:10.420317 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a64dd30bd82682296f9339e1b9fed858, offset: 0, size: 192 2024.10.22 00:42:10.428262 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a64dd30bd82682296f9339e1b9fed858 for removal 2024.10.22 00:42:10.431171 [ 1476 ] {} CacheMetadata: Key b740edbe388279d32a3661bd26d9be02 is removed from metadata 2024.10.22 00:42:10.434783 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 24c31739d7441d186f9c65d1c2ccbbe2, offset: 0, size: 260110 2024.10.22 00:42:10.438867 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 24c31739d7441d186f9c65d1c2ccbbe2 for removal 2024.10.22 00:42:10.441335 [ 1476 ] {} CacheMetadata: Key a64dd30bd82682296f9339e1b9fed858 is removed from metadata 2024.10.22 00:42:10.442148 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: aa8364a4abec9a14eb2d5345be75f413, offset: 0, size: 163 2024.10.22 00:42:10.446524 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key aa8364a4abec9a14eb2d5345be75f413 for removal 2024.10.22 00:42:10.451299 [ 1476 ] {} CacheMetadata: Key aa8364a4abec9a14eb2d5345be75f413 is removed from metadata 2024.10.22 00:42:10.453561 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 92bd0cb1d98544ff2448e3bef3840a39, offset: 0, size: 164 2024.10.22 00:42:10.459084 [ 1476 ] {} CacheMetadata: Key 24c31739d7441d186f9c65d1c2ccbbe2 is removed from metadata 2024.10.22 00:42:10.459951 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 92bd0cb1d98544ff2448e3bef3840a39 for removal 2024.10.22 00:42:10.471733 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: faf3470557c74318edb18b8ff17611ca, offset: 0, size: 163 2024.10.22 00:42:10.474266 [ 1476 ] {} CacheMetadata: Key 92bd0cb1d98544ff2448e3bef3840a39 is removed from metadata 2024.10.22 00:42:10.474379 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key faf3470557c74318edb18b8ff17611ca for removal 2024.10.22 00:42:10.477152 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f58c4d2d1317dceb809fd573d88a755d, offset: 0, size: 170004 2024.10.22 00:42:10.481837 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f58c4d2d1317dceb809fd573d88a755d for removal 2024.10.22 00:42:10.481721 [ 1476 ] {} CacheMetadata: Key faf3470557c74318edb18b8ff17611ca is removed from metadata 2024.10.22 00:42:10.484093 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 34c04555dc2326052835f8e91c0accee, offset: 0, size: 10223 2024.10.22 00:42:10.487751 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 34c04555dc2326052835f8e91c0accee for removal 2024.10.22 00:42:10.490580 [ 1476 ] {} CacheMetadata: Key 34c04555dc2326052835f8e91c0accee is removed from metadata 2024.10.22 00:42:10.491091 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 66967dcf0417eecb36f803cf2d8b9c34, offset: 0, size: 159 2024.10.22 00:42:10.498082 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 66967dcf0417eecb36f803cf2d8b9c34 for removal 2024.10.22 00:42:10.509167 [ 1476 ] {} CacheMetadata: Key 66967dcf0417eecb36f803cf2d8b9c34 is removed from metadata 2024.10.22 00:42:10.517655 [ 1476 ] {} CacheMetadata: Key f58c4d2d1317dceb809fd573d88a755d is removed from metadata 2024.10.22 00:42:10.576662 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_166_33/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:42:10.627068 [ 1476 ] {} CacheMetadata: Key d55c37dfc417d78e414db2c7c8fb2487 is removed from metadata 2024.10.22 00:42:10.724861 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 108 entries to flush up to offset 15970 2024.10.22 00:42:11.084504 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:11.088237 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:11.227626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 16 file segments (while reserving 902343 bytes for 662b16d70c5cf28002a65d3d73454522:0) 2024.10.22 00:42:11.334348 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ff0b50c4dae7f98c3426a47a59ff7efe, offset: 0, size: 234 2024.10.22 00:42:11.347649 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ff0b50c4dae7f98c3426a47a59ff7efe for removal 2024.10.22 00:42:11.352848 [ 1476 ] {} CacheMetadata: Key ff0b50c4dae7f98c3426a47a59ff7efe is removed from metadata 2024.10.22 00:42:11.354536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c9840543173d46c82a7e294c07b7106a, offset: 0, size: 164 2024.10.22 00:42:11.359286 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c9840543173d46c82a7e294c07b7106a for removal 2024.10.22 00:42:11.363418 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8f38bb917110e6a7368ac7462fb71521, offset: 0, size: 4723 2024.10.22 00:42:11.364255 [ 1476 ] {} CacheMetadata: Key c9840543173d46c82a7e294c07b7106a is removed from metadata 2024.10.22 00:42:11.366484 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8f38bb917110e6a7368ac7462fb71521 for removal 2024.10.22 00:42:11.369439 [ 1476 ] {} CacheMetadata: Key 8f38bb917110e6a7368ac7462fb71521 is removed from metadata 2024.10.22 00:42:11.370022 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 460a48bc901d66e01c8195024644d966, offset: 0, size: 161 2024.10.22 00:42:11.373587 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 460a48bc901d66e01c8195024644d966 for removal 2024.10.22 00:42:11.376113 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eaf3d995a8e3b0abd4275f73bb67f371, offset: 0, size: 159 2024.10.22 00:42:11.377406 [ 1476 ] {} CacheMetadata: Key 460a48bc901d66e01c8195024644d966 is removed from metadata 2024.10.22 00:42:11.378920 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eaf3d995a8e3b0abd4275f73bb67f371 for removal 2024.10.22 00:42:11.380829 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 851b6c0f0ba4fd749031890e212f9dab, offset: 0, size: 436223 2024.10.22 00:42:11.382733 [ 1476 ] {} CacheMetadata: Key eaf3d995a8e3b0abd4275f73bb67f371 is removed from metadata 2024.10.22 00:42:11.383233 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 851b6c0f0ba4fd749031890e212f9dab for removal 2024.10.22 00:42:11.385219 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0979123e631cd3eb413b05c22813165b, offset: 0, size: 160 2024.10.22 00:42:11.387076 [ 1476 ] {} CacheMetadata: Key 851b6c0f0ba4fd749031890e212f9dab is removed from metadata 2024.10.22 00:42:11.387246 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0979123e631cd3eb413b05c22813165b for removal 2024.10.22 00:42:11.391489 [ 1476 ] {} CacheMetadata: Key 0979123e631cd3eb413b05c22813165b is removed from metadata 2024.10.22 00:42:11.490681 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 17d2debdc4945145a3f89347492d7249, offset: 0, size: 253673 2024.10.22 00:42:11.493833 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 17d2debdc4945145a3f89347492d7249 for removal 2024.10.22 00:42:11.498610 [ 1476 ] {} CacheMetadata: Key 17d2debdc4945145a3f89347492d7249 is removed from metadata 2024.10.22 00:42:11.499009 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f5a88c323932e0f3d7fa2757b8016ed6, offset: 0, size: 192 2024.10.22 00:42:11.510620 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f5a88c323932e0f3d7fa2757b8016ed6 for removal 2024.10.22 00:42:11.513686 [ 1476 ] {} CacheMetadata: Key f5a88c323932e0f3d7fa2757b8016ed6 is removed from metadata 2024.10.22 00:42:11.514377 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8f2c10bd2ecf80c48fa314a65e30348b, offset: 0, size: 71180 2024.10.22 00:42:11.524894 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_286_286_0 (state Temporary) 2024.10.22 00:42:11.531371 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8f2c10bd2ecf80c48fa314a65e30348b for removal 2024.10.22 00:42:11.534257 [ 1476 ] {} CacheMetadata: Key 8f2c10bd2ecf80c48fa314a65e30348b is removed from metadata 2024.10.22 00:42:11.578391 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 1350 2024.10.22 00:42:11.639082 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a4272c03040bb8cf3de2153c0a9a8663, offset: 0, size: 68485 2024.10.22 00:42:11.648381 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a4272c03040bb8cf3de2153c0a9a8663 for removal 2024.10.22 00:42:11.651694 [ 1476 ] {} CacheMetadata: Key a4272c03040bb8cf3de2153c0a9a8663 is removed from metadata 2024.10.22 00:42:11.652538 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 99adfe42390af0fff02d5ee9fde99aac, offset: 0, size: 5154 2024.10.22 00:42:11.657435 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 99adfe42390af0fff02d5ee9fde99aac for removal 2024.10.22 00:42:11.661791 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 826cc6fd5d9ac8ed623642b4b0d5cf56, offset: 0, size: 63891 2024.10.22 00:42:11.665177 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 826cc6fd5d9ac8ed623642b4b0d5cf56 for removal 2024.10.22 00:42:11.671359 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8cbef104fb03a76674356a968ae8e9c2, offset: 0, size: 161 2024.10.22 00:42:11.673172 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8cbef104fb03a76674356a968ae8e9c2 for removal 2024.10.22 00:42:11.673413 [ 1476 ] {} CacheMetadata: Key 826cc6fd5d9ac8ed623642b4b0d5cf56 is removed from metadata 2024.10.22 00:42:11.675666 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1c78eb9805d4d79a3f0caa28ebf2a5cd, offset: 0, size: 116777 2024.10.22 00:42:11.677998 [ 1476 ] {} CacheMetadata: Key 8cbef104fb03a76674356a968ae8e9c2 is removed from metadata 2024.10.22 00:42:11.678467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1c78eb9805d4d79a3f0caa28ebf2a5cd for removal 2024.10.22 00:42:11.682423 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a4fb7ca85228cba51e60f0d7b3e26af8, offset: 0, size: 160 2024.10.22 00:42:11.684174 [ 1476 ] {} CacheMetadata: Key 1c78eb9805d4d79a3f0caa28ebf2a5cd is removed from metadata 2024.10.22 00:42:11.684281 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a4fb7ca85228cba51e60f0d7b3e26af8 for removal 2024.10.22 00:42:11.692821 [ 1476 ] {} CacheMetadata: Key a4fb7ca85228cba51e60f0d7b3e26af8 is removed from metadata 2024.10.22 00:42:11.704469 [ 1476 ] {} CacheMetadata: Key 99adfe42390af0fff02d5ee9fde99aac is removed from metadata 2024.10.22 00:42:11.788279 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 2331 2024.10.22 00:42:11.974857 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_286_286_0 to 202410_299_299_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:11.998845 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:12.006976 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:12.007528 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 15970 2024.10.22 00:42:12.102227 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 579269 bytes for e9e8c49e56ef9a63fcf94809ca6e50c8:0) 2024.10.22 00:42:12.110710 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 271f68104e974afb2e10145359871f40, offset: 0, size: 164385 2024.10.22 00:42:12.115665 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 271f68104e974afb2e10145359871f40 for removal 2024.10.22 00:42:12.117372 [ 1476 ] {} CacheMetadata: Key 271f68104e974afb2e10145359871f40 is removed from metadata 2024.10.22 00:42:12.118077 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a48a94f1c44b7b3e067f4854f396b2de, offset: 0, size: 166 2024.10.22 00:42:12.121113 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a48a94f1c44b7b3e067f4854f396b2de for removal 2024.10.22 00:42:12.123266 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e64ff80c547211526255d8695be8ff81, offset: 0, size: 272947 2024.10.22 00:42:12.126870 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e64ff80c547211526255d8695be8ff81 for removal 2024.10.22 00:42:12.130751 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cd5348f62303cfe79927f733cb2c32b6, offset: 0, size: 165094 2024.10.22 00:42:12.133464 [ 1476 ] {} CacheMetadata: Key e64ff80c547211526255d8695be8ff81 is removed from metadata 2024.10.22 00:42:12.135063 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cd5348f62303cfe79927f733cb2c32b6 for removal 2024.10.22 00:42:12.147308 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4f5e59364e1856dce3b7911f9ebac1ea, offset: 0, size: 192 2024.10.22 00:42:12.153001 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4f5e59364e1856dce3b7911f9ebac1ea for removal 2024.10.22 00:42:12.156970 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_167_167_0 (state Temporary) 2024.10.22 00:42:12.161023 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2b198dcfca5fdb9e7ed6b303cde503c6, offset: 0, size: 169 2024.10.22 00:42:12.168866 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2b198dcfca5fdb9e7ed6b303cde503c6 for removal 2024.10.22 00:42:12.245845 [ 1476 ] {} CacheMetadata: Key 2b198dcfca5fdb9e7ed6b303cde503c6 is removed from metadata 2024.10.22 00:42:12.418605 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_167_167_0 to 202410_180_180_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:12.459662 [ 1476 ] {} CacheMetadata: Key 4f5e59364e1856dce3b7911f9ebac1ea is removed from metadata 2024.10.22 00:42:12.464635 [ 1476 ] {} CacheMetadata: Key cd5348f62303cfe79927f733cb2c32b6 is removed from metadata 2024.10.22 00:42:12.468988 [ 1476 ] {} CacheMetadata: Key a48a94f1c44b7b3e067f4854f396b2de is removed from metadata 2024.10.22 00:42:12.478850 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2331 2024.10.22 00:42:12.604368 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 12.24 MiB, will set to 1.40 GiB (RSS), difference: -1.04 MiB 2024.10.22 00:42:13.172674 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 329086 bytes for 34978f75c5206cde86e62438a78c653b:0) 2024.10.22 00:42:13.175137 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1dae2dd476fd3c2811ad56a3bc97d4d0, offset: 0, size: 345773 2024.10.22 00:42:13.178190 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1dae2dd476fd3c2811ad56a3bc97d4d0 for removal 2024.10.22 00:42:13.181148 [ 1476 ] {} CacheMetadata: Key 1dae2dd476fd3c2811ad56a3bc97d4d0 is removed from metadata 2024.10.22 00:42:13.181149 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ee1b528ff83f0b52758d6e57b0e1cc09, offset: 0, size: 193 2024.10.22 00:42:13.185452 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ee1b528ff83f0b52758d6e57b0e1cc09 for removal 2024.10.22 00:42:13.211476 [ 1476 ] {} CacheMetadata: Key ee1b528ff83f0b52758d6e57b0e1cc09 is removed from metadata 2024.10.22 00:42:13.421185 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 278176 2024.10.22 00:42:13.666108 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 326823 bytes for b791e10e4cbc44454557dfc0e7c809c1:0) 2024.10.22 00:42:13.771380 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 43fb93bc82e797c059e140f806fca972, offset: 0, size: 386320 2024.10.22 00:42:13.773833 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 43fb93bc82e797c059e140f806fca972 for removal 2024.10.22 00:42:13.778870 [ 1476 ] {} CacheMetadata: Key 43fb93bc82e797c059e140f806fca972 is removed from metadata 2024.10.22 00:42:13.779963 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b0f5c267fde7585b267f6e30fdad4faf, offset: 0, size: 196 2024.10.22 00:42:13.782326 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b0f5c267fde7585b267f6e30fdad4faf for removal 2024.10.22 00:42:13.783787 [ 1476 ] {} CacheMetadata: Key b0f5c267fde7585b267f6e30fdad4faf is removed from metadata 2024.10.22 00:42:14.274267 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 276155 bytes for 6c4a5db52ca1970c012ffcc136382a18:0) 2024.10.22 00:42:14.275469 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4bced909a8123cd2fcc645c436a595df, offset: 0, size: 197352 2024.10.22 00:42:14.378849 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4bced909a8123cd2fcc645c436a595df for removal 2024.10.22 00:42:14.381081 [ 1476 ] {} CacheMetadata: Key 4bced909a8123cd2fcc645c436a595df is removed from metadata 2024.10.22 00:42:14.381795 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2025817698847cc0781ec0df973508b6, offset: 0, size: 191 2024.10.22 00:42:14.386805 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2025817698847cc0781ec0df973508b6 for removal 2024.10.22 00:42:14.398922 [ 1476 ] {} CacheMetadata: Key 2025817698847cc0781ec0df973508b6 is removed from metadata 2024.10.22 00:42:14.563587 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:14.567679 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:14.777338 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_160_160_0 (state Temporary) 2024.10.22 00:42:14.858402 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 53 entries to flush up to offset 10843 2024.10.22 00:42:14.888721 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 651502 bytes for 486b06a4a7e24a25e79ca8279b1df483:0) 2024.10.22 00:42:14.890193 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a375da93c91235b6252fca985c4da76a, offset: 0, size: 2049510 2024.10.22 00:42:14.897407 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a375da93c91235b6252fca985c4da76a for removal 2024.10.22 00:42:14.905428 [ 1476 ] {} CacheMetadata: Key a375da93c91235b6252fca985c4da76a is removed from metadata 2024.10.22 00:42:14.907627 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4b824d468a6dfc46d6dc3b6df9b5dff4, offset: 0, size: 258 2024.10.22 00:42:14.913741 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4b824d468a6dfc46d6dc3b6df9b5dff4 for removal 2024.10.22 00:42:14.925480 [ 1476 ] {} CacheMetadata: Key 4b824d468a6dfc46d6dc3b6df9b5dff4 is removed from metadata 2024.10.22 00:42:14.930726 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.88 GiB, free memory in arenas 13.94 MiB, will set to 1.40 GiB (RSS), difference: -5.72 MiB 2024.10.22 00:42:15.040963 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_160_160_0 to 202410_167_167_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:15.069545 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:15.079481 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:15.157620 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_166_166_0 (state Temporary) 2024.10.22 00:42:15.178624 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_34_7/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:42:15.216217 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 278176 2024.10.22 00:42:15.271757 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_166_166_0 to 202410_172_172_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:15.293442 [ 1186 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_167_34 to 202410_172_172_0 2024.10.22 00:42:15.315572 [ 1186 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:15.316176 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10843 2024.10.22 00:42:15.317389 [ 1186 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:15.336288 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_167_34 to 202410_172_172_0 into Compact with storage Full 2024.10.22 00:42:15.456940 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 152 2024.10.22 00:42:15.578393 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:42:15.603081 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_167_34, total 10657 rows starting from the beginning of the part 2024.10.22 00:42:15.648696 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:15.652659 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:15.661625 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_168_168_0, total 71 rows starting from the beginning of the part 2024.10.22 00:42:15.714251 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_169_169_0, total 48 rows starting from the beginning of the part 2024.10.22 00:42:15.731710 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_134_134_0 (state Temporary) 2024.10.22 00:42:15.814256 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_134_134_0 to 202410_144_144_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:15.898082 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 152 2024.10.22 00:42:16.021385 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 138 entries to flush up to offset 16108 2024.10.22 00:42:16.082218 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_170_170_0, total 63 rows starting from the beginning of the part 2024.10.22 00:42:16.095060 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} MergeTask::MergeProjectionsStage: Merge sorted 290230 rows, containing 4 columns (4 merged, 0 gathered) in 9.628418555 sec., 30143.06018606604 rows/sec., 442.48 KiB/sec. 2024.10.22 00:42:16.264882 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_171_171_0, total 55 rows starting from the beginning of the part 2024.10.22 00:42:16.283470 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} MergedBlockOutputStream: filled checksums 202410_1_166_33 (state Temporary) 2024.10.22 00:42:16.288060 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_172_172_0, total 53 rows starting from the beginning of the part 2024.10.22 00:42:16.340644 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 12 file segments (while reserving 1048576 bytes for 7579f0fdbd8d7487e29bd26de19fcb21:0) 2024.10.22 00:42:16.343324 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_166_33 to 202410_1_166_33 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:16.346511 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9d491f91eef09af81228c1edfd9475f1, offset: 0, size: 681616 2024.10.22 00:42:16.348982 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_166_33} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_161_32, 202410_166_166_0] -> 202410_1_166_33 2024.10.22 00:42:16.351611 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9d491f91eef09af81228c1edfd9475f1 for removal 2024.10.22 00:42:16.356249 [ 1476 ] {} CacheMetadata: Key 9d491f91eef09af81228c1edfd9475f1 is removed from metadata 2024.10.22 00:42:16.357067 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 775ac2ab5e31a34de716c752f20fae38, offset: 0, size: 147763 2024.10.22 00:42:16.362504 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 775ac2ab5e31a34de716c752f20fae38 for removal 2024.10.22 00:42:16.366095 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0d005f02f4fc666aa71e3fb619caf304, offset: 0, size: 172736 2024.10.22 00:42:16.371717 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0d005f02f4fc666aa71e3fb619caf304 for removal 2024.10.22 00:42:16.372359 [ 1476 ] {} CacheMetadata: Key 775ac2ab5e31a34de716c752f20fae38 is removed from metadata 2024.10.22 00:42:16.375960 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 03e567f3b1fd6a2072e2b098e025013d, offset: 0, size: 92819 2024.10.22 00:42:16.379420 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 03e567f3b1fd6a2072e2b098e025013d for removal 2024.10.22 00:42:16.380803 [ 1476 ] {} CacheMetadata: Key 0d005f02f4fc666aa71e3fb619caf304 is removed from metadata 2024.10.22 00:42:16.381506 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:42:16.383493 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dbf162e86d945fbb7ea68a61aecae36b, offset: 0, size: 163 2024.10.22 00:42:16.383674 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:16.385447 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dbf162e86d945fbb7ea68a61aecae36b for removal 2024.10.22 00:42:16.387072 [ 1476 ] {} CacheMetadata: Key 03e567f3b1fd6a2072e2b098e025013d is removed from metadata 2024.10.22 00:42:16.387448 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:16.388828 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8b07fcc8a48b4f8d2dcf8837692aa782, offset: 0, size: 106044 2024.10.22 00:42:16.391755 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8b07fcc8a48b4f8d2dcf8837692aa782 for removal 2024.10.22 00:42:16.393593 [ 1476 ] {} CacheMetadata: Key dbf162e86d945fbb7ea68a61aecae36b is removed from metadata 2024.10.22 00:42:16.395196 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ec2e2afa0307929173dea6936be49803, offset: 0, size: 226 2024.10.22 00:42:16.408039 [ 1476 ] {} CacheMetadata: Key 8b07fcc8a48b4f8d2dcf8837692aa782 is removed from metadata 2024.10.22 00:42:16.459395 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_287_287_0 (state Temporary) 2024.10.22 00:42:16.501650 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ec2e2afa0307929173dea6936be49803 for removal 2024.10.22 00:42:16.504880 [ 1476 ] {} CacheMetadata: Key ec2e2afa0307929173dea6936be49803 is removed from metadata 2024.10.22 00:42:16.506542 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cc2fa0b20b7dadb26fcac1b277e9e88f, offset: 0, size: 229 2024.10.22 00:42:16.508892 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.88 GiB, free memory in arenas 13.27 MiB, will set to 1.40 GiB (RSS), difference: 6.08 MiB 2024.10.22 00:42:16.510265 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cc2fa0b20b7dadb26fcac1b277e9e88f for removal 2024.10.22 00:42:16.514147 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 36683f32b241c58cba4041c4fc2fb694, offset: 0, size: 167 2024.10.22 00:42:16.515665 [ 1476 ] {} CacheMetadata: Key cc2fa0b20b7dadb26fcac1b277e9e88f is removed from metadata 2024.10.22 00:42:16.517838 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 36683f32b241c58cba4041c4fc2fb694 for removal 2024.10.22 00:42:16.522824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2e209366b79dc36b74229bd5bb1281ae, offset: 0, size: 296994 2024.10.22 00:42:16.529583 [ 1476 ] {} CacheMetadata: Key 36683f32b241c58cba4041c4fc2fb694 is removed from metadata 2024.10.22 00:42:16.531882 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2e209366b79dc36b74229bd5bb1281ae for removal 2024.10.22 00:42:16.535277 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c5b7e03c6c2a89d9f37cde0fe5839f2c, offset: 0, size: 232 2024.10.22 00:42:16.546612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c5b7e03c6c2a89d9f37cde0fe5839f2c for removal 2024.10.22 00:42:16.548010 [ 1476 ] {} CacheMetadata: Key 2e209366b79dc36b74229bd5bb1281ae is removed from metadata 2024.10.22 00:42:16.553559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4d54a165fc29c330f1025ba15c66401e, offset: 0, size: 160 2024.10.22 00:42:16.559635 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4d54a165fc29c330f1025ba15c66401e for removal 2024.10.22 00:42:16.560136 [ 1476 ] {} CacheMetadata: Key c5b7e03c6c2a89d9f37cde0fe5839f2c is removed from metadata 2024.10.22 00:42:16.568385 [ 1476 ] {} CacheMetadata: Key 4d54a165fc29c330f1025ba15c66401e is removed from metadata 2024.10.22 00:42:16.594769 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_287_287_0 to 202410_300_300_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:16.628525 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_295_59 to 202410_300_300_0 2024.10.22 00:42:16.656187 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:16.659094 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16108 2024.10.22 00:42:16.662489 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:16.685457 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_295_59 to 202410_300_300_0 into Compact with storage Full 2024.10.22 00:42:16.755978 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:42:16.767041 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_295_59, total 16373 rows starting from the beginning of the part 2024.10.22 00:42:16.798654 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_296_296_0, total 170 rows starting from the beginning of the part 2024.10.22 00:42:16.856615 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_297_297_0, total 96 rows starting from the beginning of the part 2024.10.22 00:42:16.882273 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_298_298_0, total 54 rows starting from the beginning of the part 2024.10.22 00:42:16.902586 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 821723 bytes for 7579f0fdbd8d7487e29bd26de19fcb21:0) 2024.10.22 00:42:16.909372 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 971803485c923501475fcf71babe9c4a, offset: 0, size: 1942466 2024.10.22 00:42:16.916296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 971803485c923501475fcf71babe9c4a for removal 2024.10.22 00:42:16.919068 [ 1476 ] {} CacheMetadata: Key 971803485c923501475fcf71babe9c4a is removed from metadata 2024.10.22 00:42:16.919548 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e448b707747218a6d64a821ec331f466, offset: 0, size: 258 2024.10.22 00:42:16.923739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e448b707747218a6d64a821ec331f466 for removal 2024.10.22 00:42:16.924258 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_299_299_0, total 108 rows starting from the beginning of the part 2024.10.22 00:42:16.928993 [ 1476 ] {} CacheMetadata: Key e448b707747218a6d64a821ec331f466 is removed from metadata 2024.10.22 00:42:17.087274 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_300_300_0, total 138 rows starting from the beginning of the part 2024.10.22 00:42:17.272331 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_116_23, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0] 2024.10.22 00:42:17.274751 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_116_23, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0] 2024.10.22 00:42:17.305385 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_116_23, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0] 2024.10.22 00:42:17.656593 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 959459820ef9368f3bf7a8d3471208de:0) 2024.10.22 00:42:17.661531 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9a642bdfd1a0013c531965b90d449890, offset: 0, size: 2027649 2024.10.22 00:42:17.665524 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9a642bdfd1a0013c531965b90d449890 for removal 2024.10.22 00:42:17.668384 [ 1476 ] {} CacheMetadata: Key 9a642bdfd1a0013c531965b90d449890 is removed from metadata 2024.10.22 00:42:17.668832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8a088a689ec96dfb36de53e15118d1af, offset: 0, size: 258 2024.10.22 00:42:17.672868 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8a088a689ec96dfb36de53e15118d1af for removal 2024.10.22 00:42:17.675662 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6a1d8c5c837312caa06ae907e4b8c651, offset: 0, size: 8525 2024.10.22 00:42:17.675624 [ 1476 ] {} CacheMetadata: Key 8a088a689ec96dfb36de53e15118d1af is removed from metadata 2024.10.22 00:42:17.678343 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6a1d8c5c837312caa06ae907e4b8c651 for removal 2024.10.22 00:42:17.680709 [ 1476 ] {} CacheMetadata: Key 6a1d8c5c837312caa06ae907e4b8c651 is removed from metadata 2024.10.22 00:42:17.681832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bbf75c07e3841935f9b7ef80e9e93fa8, offset: 0, size: 178930 2024.10.22 00:42:17.685184 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bbf75c07e3841935f9b7ef80e9e93fa8 for removal 2024.10.22 00:42:17.686986 [ 1476 ] {} CacheMetadata: Key bbf75c07e3841935f9b7ef80e9e93fa8 is removed from metadata 2024.10.22 00:42:17.687498 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 338525f9037266386812517f83fae01d, offset: 0, size: 160 2024.10.22 00:42:17.692055 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 338525f9037266386812517f83fae01d for removal 2024.10.22 00:42:17.706884 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a7cbdfca883e3147509e92d2b4e39ea9, offset: 0, size: 165 2024.10.22 00:42:17.707962 [ 1476 ] {} CacheMetadata: Key 338525f9037266386812517f83fae01d is removed from metadata 2024.10.22 00:42:17.711269 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a7cbdfca883e3147509e92d2b4e39ea9 for removal 2024.10.22 00:42:17.733368 [ 1476 ] {} CacheMetadata: Key a7cbdfca883e3147509e92d2b4e39ea9 is removed from metadata 2024.10.22 00:42:17.861234 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} MergeTask::MergeProjectionsStage: Merge sorted 10947 rows, containing 13 columns (13 merged, 0 gathered) in 2.533093659 sec., 4321.59306905438 rows/sec., 1.81 MiB/sec. 2024.10.22 00:42:17.911128 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} MergedBlockOutputStream: filled checksums 202410_1_172_35 (state Temporary) 2024.10.22 00:42:18.065802 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_172_35 to 202410_1_172_35 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:18.070251 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_172_35} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_167_34, 202410_172_172_0] -> 202410_1_172_35 2024.10.22 00:42:18.117036 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.19 MiB. 2024.10.22 00:42:18.470461 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_34_7 to 202410_1_34_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:18.474963 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_34_7} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_29_6, 202410_34_34_0] -> 202410_1_34_7 2024.10.22 00:42:18.652355 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} MergeTask::MergeProjectionsStage: Merge sorted 16939 rows, containing 13 columns (13 merged, 0 gathered) in 1.973312352 sec., 8584.043971970232 rows/sec., 1.28 MiB/sec. 2024.10.22 00:42:18.678160 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} MergedBlockOutputStream: filled checksums 202410_1_300_60 (state Temporary) 2024.10.22 00:42:18.723228 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 258b4ab77e04857fb74fc7d5ffd4b711:0) 2024.10.22 00:42:18.725246 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cca0823b361652377f118f9ac5aec95c, offset: 0, size: 142725 2024.10.22 00:42:18.728123 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cca0823b361652377f118f9ac5aec95c for removal 2024.10.22 00:42:18.731283 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ba45960df171af27382571bacdf8d263, offset: 0, size: 166 2024.10.22 00:42:18.731636 [ 1476 ] {} CacheMetadata: Key cca0823b361652377f118f9ac5aec95c is removed from metadata 2024.10.22 00:42:18.740717 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ba45960df171af27382571bacdf8d263 for removal 2024.10.22 00:42:18.746916 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8e06bb5b4dc333f85dda168420495694, offset: 0, size: 167 2024.10.22 00:42:18.749682 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8e06bb5b4dc333f85dda168420495694 for removal 2024.10.22 00:42:18.757112 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 47bbaf4d164eb11656c73b48568cb59f, offset: 0, size: 144793 2024.10.22 00:42:18.760777 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 47bbaf4d164eb11656c73b48568cb59f for removal 2024.10.22 00:42:18.762822 [ 1476 ] {} CacheMetadata: Key 8e06bb5b4dc333f85dda168420495694 is removed from metadata 2024.10.22 00:42:18.763705 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7a194615966440cd667b331d22aaaa76, offset: 0, size: 173756 2024.10.22 00:42:18.766118 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7a194615966440cd667b331d22aaaa76 for removal 2024.10.22 00:42:18.768453 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c573f86821fde4944203cd0472d5a5c4, offset: 0, size: 167 2024.10.22 00:42:18.772265 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c573f86821fde4944203cd0472d5a5c4 for removal 2024.10.22 00:42:18.779969 [ 1476 ] {} CacheMetadata: Key c573f86821fde4944203cd0472d5a5c4 is removed from metadata 2024.10.22 00:42:18.793748 [ 1476 ] {} CacheMetadata: Key 7a194615966440cd667b331d22aaaa76 is removed from metadata 2024.10.22 00:42:18.808421 [ 1476 ] {} CacheMetadata: Key 47bbaf4d164eb11656c73b48568cb59f is removed from metadata 2024.10.22 00:42:18.816433 [ 1476 ] {} CacheMetadata: Key ba45960df171af27382571bacdf8d263 is removed from metadata 2024.10.22 00:42:18.889795 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.88 GiB, free memory in arenas 10.28 MiB, will set to 1.40 GiB (RSS), difference: 9.23 MiB 2024.10.22 00:42:18.915322 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_300_60 to 202410_1_300_60 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:18.919923 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_300_60} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_295_59, 202410_300_300_0] -> 202410_1_300_60 2024.10.22 00:42:18.971704 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.72 MiB. 2024.10.22 00:42:18.998901 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 191427 bytes for 9834235100db4d3c3ae017fec27aafbb:0) 2024.10.22 00:42:19.105791 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 462eae0c5fdfefe502bfeef1be9dc035, offset: 0, size: 241298 2024.10.22 00:42:19.115833 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 462eae0c5fdfefe502bfeef1be9dc035 for removal 2024.10.22 00:42:19.120163 [ 1476 ] {} CacheMetadata: Key 462eae0c5fdfefe502bfeef1be9dc035 is removed from metadata 2024.10.22 00:42:19.120802 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 64ef899d796164a03590d99fa2b92a3b, offset: 0, size: 192 2024.10.22 00:42:19.131603 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 64ef899d796164a03590d99fa2b92a3b for removal 2024.10.22 00:42:19.135399 [ 1476 ] {} CacheMetadata: Key 64ef899d796164a03590d99fa2b92a3b is removed from metadata 2024.10.22 00:42:19.205754 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gjd/bddedolrepuaeczszbmtkkhqmjwjw. 2024.10.22 00:42:19.253255 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 27.64 MiB. 2024.10.22 00:42:19.556511 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:42:19.620631 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gjd/bddedolrepuaeczszbmtkkhqmjwjw. 2024.10.22 00:42:19.624817 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_11_11_0 to 201403_11_11_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:19.660742 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:42:19.675743 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_12_12_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:42:19.980870 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 35 entries to flush up to offset 2366 2024.10.22 00:42:20.026055 [ 1201 ] {} DatabaseCatalog: Execution took 237 ms. 2024.10.22 00:42:20.185093 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:20.189294 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:20.388486 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_196_39, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0] 2024.10.22 00:42:20.390857 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_196_39, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0] 2024.10.22 00:42:20.457707 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_168_168_0 (state Temporary) 2024.10.22 00:42:20.510350 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_168_168_0 to 202410_181_181_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:20.538695 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_196_39, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0] 2024.10.22 00:42:20.569548 [ 1205 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_176_35 to 202410_181_181_0 2024.10.22 00:42:20.577326 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2366 2024.10.22 00:42:20.578284 [ 1205 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:20.581955 [ 1205 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:20.589577 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 946.04 MiB, peak 1.88 GiB, free memory in arenas 8.43 MiB, will set to 1.27 GiB (RSS), difference: 357.03 MiB 2024.10.22 00:42:20.621234 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_176_35 to 202410_181_181_0 into Compact with storage Full 2024.10.22 00:42:20.655460 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:42:20.661589 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_176_35, total 2394 rows starting from the beginning of the part 2024.10.22 00:42:20.670117 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 154 entries to flush up to offset 16262 2024.10.22 00:42:20.680547 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_177_177_0, total 13 rows starting from the beginning of the part 2024.10.22 00:42:20.720651 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_178_178_0, total 19 rows starting from the beginning of the part 2024.10.22 00:42:20.753788 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_179_179_0, total 6 rows starting from the beginning of the part 2024.10.22 00:42:20.773273 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_180_180_0, total 5 rows starting from the beginning of the part 2024.10.22 00:42:20.790942 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_181_181_0, total 35 rows starting from the beginning of the part 2024.10.22 00:42:21.190887 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:21.194920 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:21.362290 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_288_288_0 (state Temporary) 2024.10.22 00:42:21.397313 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_288_288_0 to 202410_301_301_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:21.425114 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16262 2024.10.22 00:42:22.035118 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} MergeTask::MergeProjectionsStage: Merge sorted 2472 rows, containing 12 columns (12 merged, 0 gathered) in 1.431315643 sec., 1727.0823609659942 rows/sec., 268.36 KiB/sec. 2024.10.22 00:42:22.242781 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 279712 2024.10.22 00:42:22.281449 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} MergedBlockOutputStream: filled checksums 202410_1_181_36 (state Temporary) 2024.10.22 00:42:22.429052 [ 1213 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:42:22.480986 [ 1213 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:42:22.572988 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.26 GiB, peak 1.88 GiB, free memory in arenas 7.35 MiB, will set to 1.15 GiB (RSS), difference: -118.64 MiB 2024.10.22 00:42:22.594327 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_181_36 to 202410_1_181_36 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:22.601285 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_181_36} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_176_35, 202410_181_181_0] -> 202410_1_181_36 2024.10.22 00:42:22.760590 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:42:22.813343 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:22.827991 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 89 entries to flush up to offset 10932 2024.10.22 00:42:22.917675 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:23.011981 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:23.014128 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:23.072915 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_167_167_0 (state Temporary) 2024.10.22 00:42:23.090658 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_161_161_0 (state Temporary) 2024.10.22 00:42:23.299120 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_167_167_0 to 202410_173_173_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:23.366913 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_161_161_0 to 202410_168_168_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:23.401023 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 153 2024.10.22 00:42:23.497336 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10932 2024.10.22 00:42:23.618577 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 279712 2024.10.22 00:42:23.669234 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:23.671580 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:23.720397 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_135_135_0 (state Temporary) 2024.10.22 00:42:24.015779 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_135_135_0 to 202410_145_145_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:24.083245 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 153 2024.10.22 00:42:24.274792 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 11 2024.10.22 00:42:24.316435 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.23 GiB, peak 1.88 GiB, free memory in arenas 5.93 MiB, will set to 1.18 GiB (RSS), difference: -55.07 MiB 2024.10.22 00:42:24.429824 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:24.437202 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:24.695943 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_11_11_0 (state Temporary) 2024.10.22 00:42:25.067331 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_11_11_0 to 202410_11_11_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:25.108567 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 11 2024.10.22 00:42:25.646934 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 16307 2024.10.22 00:42:25.757645 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.21 GiB, peak 1.88 GiB, free memory in arenas 6.36 MiB, will set to 1.19 GiB (RSS), difference: -15.68 MiB 2024.10.22 00:42:25.972608 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 450327 bytes for e720c25457e0ec7d4cadf23d1d570803:0) 2024.10.22 00:42:25.975879 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 21dea0627e3747599909d24f4bdbb8f2, offset: 0, size: 104866 2024.10.22 00:42:25.978469 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 21dea0627e3747599909d24f4bdbb8f2 for removal 2024.10.22 00:42:25.981054 [ 1476 ] {} CacheMetadata: Key 21dea0627e3747599909d24f4bdbb8f2 is removed from metadata 2024.10.22 00:42:25.981417 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4f510d5071b8344f83310e34948ffee5, offset: 0, size: 547382 2024.10.22 00:42:25.984532 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4f510d5071b8344f83310e34948ffee5 for removal 2024.10.22 00:42:25.986617 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6cdb1e68920541ce9036e3602a6f158f, offset: 0, size: 238 2024.10.22 00:42:25.988996 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6cdb1e68920541ce9036e3602a6f158f for removal 2024.10.22 00:42:26.080271 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:26.083808 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:26.091688 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2680671b835fc8cc96282a57f885cee1, offset: 0, size: 161 2024.10.22 00:42:26.092373 [ 1476 ] {} CacheMetadata: Key 6cdb1e68920541ce9036e3602a6f158f is removed from metadata 2024.10.22 00:42:26.097005 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2680671b835fc8cc96282a57f885cee1 for removal 2024.10.22 00:42:26.104139 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 39d3afa4804728a231f1a1217c2b15e9, offset: 0, size: 4 2024.10.22 00:42:26.104587 [ 1476 ] {} CacheMetadata: Key 2680671b835fc8cc96282a57f885cee1 is removed from metadata 2024.10.22 00:42:26.112487 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 39d3afa4804728a231f1a1217c2b15e9 for removal 2024.10.22 00:42:26.115409 [ 1476 ] {} CacheMetadata: Key 39d3afa4804728a231f1a1217c2b15e9 is removed from metadata 2024.10.22 00:42:26.128330 [ 1476 ] {} CacheMetadata: Key 4f510d5071b8344f83310e34948ffee5 is removed from metadata 2024.10.22 00:42:26.160890 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_289_289_0 (state Temporary) 2024.10.22 00:42:26.227830 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_289_289_0 to 202410_302_302_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:26.283821 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16307 2024.10.22 00:42:26.418979 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 598344 bytes for 258b4ab77e04857fb74fc7d5ffd4b711:0) 2024.10.22 00:42:26.422545 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 921936a9819bcaf94552803e4c0c959c, offset: 0, size: 382501 2024.10.22 00:42:26.428541 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 921936a9819bcaf94552803e4c0c959c for removal 2024.10.22 00:42:26.433261 [ 1476 ] {} CacheMetadata: Key 921936a9819bcaf94552803e4c0c959c is removed from metadata 2024.10.22 00:42:26.435422 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 551c2eab3c250124a2612d86e99763b3, offset: 0, size: 234 2024.10.22 00:42:26.444557 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 551c2eab3c250124a2612d86e99763b3 for removal 2024.10.22 00:42:26.463630 [ 1476 ] {} CacheMetadata: Key 551c2eab3c250124a2612d86e99763b3 is removed from metadata 2024.10.22 00:42:26.670578 [ 1691 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/sps/hsoindaragffpvwnrpqruyvucvpme, size 2695496 2024.10.22 00:42:26.714371 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:26.719161 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:26.881798 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column URLCategories 2024.10.22 00:42:26.914163 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column URLCategories 2024.10.22 00:42:26.928875 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column URLCategories 2024.10.22 00:42:26.958253 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column URLCategories 2024.10.22 00:42:26.987128 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column URLCategories 2024.10.22 00:42:27.109977 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column URLCategories 2024.10.22 00:42:27.154691 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column URLCategories 2024.10.22 00:42:27.321308 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/eyz/utljejbsdarftyeqidczugpccclul (LogSeriesLimiter: on interval from 2024-10-22 00:41:59 to 2024-10-22 00:42:27 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:42:27.691266 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_106_21, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0] 2024.10.22 00:42:27.692560 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_106_21, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0] 2024.10.22 00:42:27.693659 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.22 GiB, peak 1.88 GiB, free memory in arenas 5.49 MiB, will set to 1.21 GiB (RSS), difference: -10.64 MiB 2024.10.22 00:42:27.709455 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_106_21, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0] 2024.10.22 00:42:28.080411 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 2380 2024.10.22 00:42:28.292936 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:28.299643 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:28.401674 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_169_169_0 (state Temporary) 2024.10.22 00:42:28.571808 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_169_169_0 to 202410_182_182_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:28.619224 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2380 2024.10.22 00:42:28.966567 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_34_34_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:42:29.370285 [ 1258 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 340 ms. 2024.10.22 00:42:29.490608 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 1.88 GiB, free memory in arenas 5.45 MiB, will set to 1.25 GiB (RSS), difference: -42.67 MiB 2024.10.22 00:42:29.659967 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:42:29.663380 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:42:29.833312 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_34_34_0 (state Temporary) 2024.10.22 00:42:30.262219 [ 1248 ] {} DatabaseCatalog: Execution took 232 ms. 2024.10.22 00:42:30.289106 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 16365 2024.10.22 00:42:30.624196 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 281632 2024.10.22 00:42:30.759863 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:30.763425 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:30.895862 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_290_290_0 (state Temporary) 2024.10.22 00:42:31.015287 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 29 entries to flush up to offset 10961 2024.10.22 00:42:31.071232 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_290_290_0 to 202410_303_303_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:31.175928 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16365 2024.10.22 00:42:31.421058 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:31.429749 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:31.583645 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_168_168_0 (state Temporary) 2024.10.22 00:42:31.585083 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 154 2024.10.22 00:42:31.597274 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 1.88 GiB, free memory in arenas 5.49 MiB, will set to 1.30 GiB (RSS), difference: -49.83 MiB 2024.10.22 00:42:31.615470 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_12_12_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:42:31.633477 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_168_168_0 to 202410_174_174_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:31.781623 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10961 2024.10.22 00:42:31.873504 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:31.877131 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:31.933842 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:32.069277 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_136_136_0 (state Temporary) 2024.10.22 00:42:32.140356 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:32.256413 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_162_162_0 (state Temporary) 2024.10.22 00:42:32.273028 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_136_136_0 to 202410_146_146_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:32.307079 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_162_162_0 to 202410_169_169_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:32.341406 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 281632 2024.10.22 00:42:32.461406 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 154 2024.10.22 00:42:33.324347 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.88 GiB, free memory in arenas 4.26 MiB, will set to 1.34 GiB (RSS), difference: -51.84 MiB 2024.10.22 00:42:34.207121 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_110_22, 202410_111_111_0, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0] 2024.10.22 00:42:34.210799 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_110_22, 202410_111_111_0, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0] 2024.10.22 00:42:34.258602 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_110_22, 202410_111_111_0, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0] 2024.10.22 00:42:34.279853 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_93_19, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0, 202410_97_97_0, 202410_98_98_0] 2024.10.22 00:42:34.282183 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_93_19, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0, 202410_97_97_0, 202410_98_98_0] 2024.10.22 00:42:34.429404 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_93_19, 202410_94_94_0, 202410_95_95_0, 202410_96_96_0, 202410_97_97_0, 202410_98_98_0] 2024.10.22 00:42:34.668584 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 4.30 MiB, will set to 1.36 GiB (RSS), difference: -80.29 MiB 2024.10.22 00:42:35.180105 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 16399 2024.10.22 00:42:35.276001 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:35.277605 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:35.303972 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_291_291_0 (state Temporary) 2024.10.22 00:42:35.344889 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_291_291_0 to 202410_304_304_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:35.487798 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16399 2024.10.22 00:42:36.120711 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 2386 2024.10.22 00:42:36.383699 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 4.72 MiB, will set to 1.38 GiB (RSS), difference: -19.01 MiB 2024.10.22 00:42:36.431541 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:36.435035 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:36.470382 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_170_170_0 (state Temporary) 2024.10.22 00:42:36.617012 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_170_170_0 to 202410_183_183_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:36.672861 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2386 2024.10.22 00:42:37.484675 [ 1185 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:42:37.516810 [ 1185 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:42:38.392471 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_34_34_0 to 202410_37_37_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:38.444569 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.88 GiB, free memory in arenas 4.56 MiB, will set to 1.43 GiB (RSS), difference: -49.47 MiB 2024.10.22 00:42:39.285570 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 24 entries to flush up to offset 10985 2024.10.22 00:42:39.379423 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1880 entries to flush up to offset 283512 2024.10.22 00:42:39.493548 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 19 entries to flush up to offset 16418 2024.10.22 00:42:39.509147 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:39.515421 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:39.558031 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:42:39.561245 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_169_169_0 (state Temporary) 2024.10.22 00:42:39.562314 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:42:39.609619 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_169_169_0 to 202410_175_175_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:39.770904 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 10985 2024.10.22 00:42:39.919414 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:39.923676 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:39.963846 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 155 2024.10.22 00:42:40.175169 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_292_292_0 (state Temporary) 2024.10.22 00:42:40.218908 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_292_292_0 to 202410_305_305_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:40.266751 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:40.271983 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:40.276031 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1_300_60 to 202410_303_303_0 2024.10.22 00:42:40.285715 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:40.286556 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16418 2024.10.22 00:42:40.374765 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.88 GiB, free memory in arenas 4.17 MiB, will set to 1.44 GiB (RSS), difference: -12.21 MiB 2024.10.22 00:42:40.384706 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:40.388405 [ 1202 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:40.429589 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_137_137_0 (state Temporary) 2024.10.22 00:42:40.496917 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:40.497505 [ 1202 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 00:42:40.516091 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_303_61} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_300_60 to 202410_303_303_0 into Compact with storage Full 2024.10.22 00:42:40.586339 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_137_137_0 to 202410_147_147_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:40.656683 [ 1218 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_142_29 to 202410_147_147_0 2024.10.22 00:42:40.673930 [ 1218 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:40.678425 [ 1218 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:40.706669 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_142_29 to 202410_147_147_0 into Compact with storage Full 2024.10.22 00:42:40.768886 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 155 2024.10.22 00:42:40.832752 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_163_163_0 (state Temporary) 2024.10.22 00:42:40.858632 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:42:40.869636 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_142_29, total 186 rows starting from the beginning of the part 2024.10.22 00:42:40.906742 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_163_163_0 to 202410_170_170_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:40.943080 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 283512 2024.10.22 00:42:40.959049 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_303_61} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:42:40.973347 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_303_61} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_300_60, total 16939 rows starting from the beginning of the part 2024.10.22 00:42:41.016424 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_143_143_0, total 1 rows starting from the beginning of the part 2024.10.22 00:42:41.049038 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_303_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_301_301_0, total 154 rows starting from the beginning of the part 2024.10.22 00:42:41.121827 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_303_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_302_302_0, total 45 rows starting from the beginning of the part 2024.10.22 00:42:41.162615 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_144_144_0, total 1 rows starting from the beginning of the part 2024.10.22 00:42:41.178399 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_145_145_0, total 1 rows starting from the beginning of the part 2024.10.22 00:42:41.222653 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_146_146_0, total 1 rows starting from the beginning of the part 2024.10.22 00:42:41.307186 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1350 2024.10.22 00:42:41.382057 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_147_147_0, total 1 rows starting from the beginning of the part 2024.10.22 00:42:41.472723 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_303_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_303_303_0, total 58 rows starting from the beginning of the part 2024.10.22 00:42:42.722413 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_147_30/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:42:42.863593 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.88 GiB, free memory in arenas 6.13 MiB, will set to 1.49 GiB (RSS), difference: -76.13 MiB 2024.10.22 00:42:43.446809 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} MergeTask::MergeProjectionsStage: Merge sorted 191 rows, containing 32 columns (32 merged, 0 gathered) in 2.753253952 sec., 69.3724601253201 rows/sec., 7.54 KiB/sec. 2024.10.22 00:42:43.583598 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} MergedBlockOutputStream: filled checksums 202410_1_147_30 (state Temporary) 2024.10.22 00:42:44.036987 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_303_61} MergeTask::MergeProjectionsStage: Merge sorted 17196 rows, containing 13 columns (13 merged, 0 gathered) in 3.537578891 sec., 4860.951664922178 rows/sec., 741.68 KiB/sec. 2024.10.22 00:42:44.081234 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_303_61} MergedBlockOutputStream: filled checksums 202410_1_303_61 (state Temporary) 2024.10.22 00:42:44.137342 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_303_61} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_303_61 to 202410_1_303_61 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:44.145479 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_303_61} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1_300_60, 202410_303_303_0] -> 202410_1_303_61 2024.10.22 00:42:44.174371 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 2397 2024.10.22 00:42:44.178753 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.67 MiB. 2024.10.22 00:42:44.248432 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:44.255181 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:44.294373 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_171_171_0 (state Temporary) 2024.10.22 00:42:44.388174 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 16478 2024.10.22 00:42:44.412898 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_147_30 to 202410_1_147_30 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:44.439540 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_147_30} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_142_29, 202410_147_147_0] -> 202410_1_147_30 2024.10.22 00:42:44.571528 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:42:44.583820 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.88 GiB, free memory in arenas 5.75 MiB, will set to 1.52 GiB (RSS), difference: -31.59 MiB 2024.10.22 00:42:44.584778 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:44.585781 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:44.619316 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_171_171_0 to 202410_184_184_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:44.642850 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_293_293_0 (state Temporary) 2024.10.22 00:42:44.683373 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2397 2024.10.22 00:42:44.820199 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_293_293_0 to 202410_306_306_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:44.895251 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16478 2024.10.22 00:42:45.069018 [ 1220 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 00:42:46.508309 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.88 GiB, free memory in arenas 5.59 MiB, will set to 1.56 GiB (RSS), difference: -21.68 MiB 2024.10.22 00:42:47.263718 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 1048576 bytes for bf4de651f62646b449a06eb48638cf68:0) 2024.10.22 00:42:47.265356 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 24cdf3f2dc2bd0c3173be6dc37cbd12a, offset: 0, size: 236 2024.10.22 00:42:47.270394 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 24cdf3f2dc2bd0c3173be6dc37cbd12a for removal 2024.10.22 00:42:47.273825 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 61 entries to flush up to offset 11046 2024.10.22 00:42:47.275714 [ 1476 ] {} CacheMetadata: Key 24cdf3f2dc2bd0c3173be6dc37cbd12a is removed from metadata 2024.10.22 00:42:47.276295 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 28d9fae756b057627beb2f288bbd1ca8, offset: 0, size: 233 2024.10.22 00:42:47.278970 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 28d9fae756b057627beb2f288bbd1ca8 for removal 2024.10.22 00:42:47.281206 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 003f8279d6a3762d2672aeb9267693a5, offset: 0, size: 230660 2024.10.22 00:42:47.286114 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 003f8279d6a3762d2672aeb9267693a5 for removal 2024.10.22 00:42:47.289104 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d4a31693302a1be3e97c9348ba0a3df7, offset: 0, size: 523495 2024.10.22 00:42:47.293276 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d4a31693302a1be3e97c9348ba0a3df7 for removal 2024.10.22 00:42:47.301724 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e1dff52ed25dc3f650af6dba71c07d70, offset: 0, size: 351867 2024.10.22 00:42:47.310694 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e1dff52ed25dc3f650af6dba71c07d70 for removal 2024.10.22 00:42:47.327661 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1acbd9a963cda775ecf123c4e7c2a5cb, offset: 0, size: 192 2024.10.22 00:42:47.343555 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1acbd9a963cda775ecf123c4e7c2a5cb for removal 2024.10.22 00:42:47.347779 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3c0b1a01d6daaa8ebe47369713b484a9, offset: 0, size: 9508 2024.10.22 00:42:47.356631 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3c0b1a01d6daaa8ebe47369713b484a9 for removal 2024.10.22 00:42:47.365234 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 50535df74e1480cf27dc1c130cade3c0, offset: 0, size: 224 2024.10.22 00:42:47.371288 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 50535df74e1480cf27dc1c130cade3c0 for removal 2024.10.22 00:42:47.388130 [ 1476 ] {} CacheMetadata: Key 50535df74e1480cf27dc1c130cade3c0 is removed from metadata 2024.10.22 00:42:47.392866 [ 1476 ] {} CacheMetadata: Key 3c0b1a01d6daaa8ebe47369713b484a9 is removed from metadata 2024.10.22 00:42:47.405345 [ 1476 ] {} CacheMetadata: Key 1acbd9a963cda775ecf123c4e7c2a5cb is removed from metadata 2024.10.22 00:42:47.415588 [ 1476 ] {} CacheMetadata: Key e1dff52ed25dc3f650af6dba71c07d70 is removed from metadata 2024.10.22 00:42:47.426913 [ 1476 ] {} CacheMetadata: Key d4a31693302a1be3e97c9348ba0a3df7 is removed from metadata 2024.10.22 00:42:47.483385 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:47.485819 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:47.539654 [ 1476 ] {} CacheMetadata: Key 003f8279d6a3762d2672aeb9267693a5 is removed from metadata 2024.10.22 00:42:47.558000 [ 1476 ] {} CacheMetadata: Key 28d9fae756b057627beb2f288bbd1ca8 is removed from metadata 2024.10.22 00:42:47.627060 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_170_170_0 (state Temporary) 2024.10.22 00:42:47.771702 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_170_170_0 to 202410_176_176_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:47.916583 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11046 2024.10.22 00:42:47.954588 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1576 entries to flush up to offset 285088 2024.10.22 00:42:48.277514 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 5.88 MiB, will set to 1.62 GiB (RSS), difference: -80.37 MiB 2024.10.22 00:42:48.820719 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 943c26921857b165baf6c892b7ec2804:0) 2024.10.22 00:42:48.826305 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3786b65a78e8c836e915773b6720ff0c, offset: 0, size: 246 2024.10.22 00:42:48.830200 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3786b65a78e8c836e915773b6720ff0c for removal 2024.10.22 00:42:48.832783 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6e60822dba2af1c2d6c0c54e272da506, offset: 0, size: 642951 2024.10.22 00:42:48.840722 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6e60822dba2af1c2d6c0c54e272da506 for removal 2024.10.22 00:42:48.844966 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 922da56e4e09cac7f900bdc25117b7b7, offset: 0, size: 1011269 2024.10.22 00:42:48.937071 [ 1476 ] {} CacheMetadata: Key 3786b65a78e8c836e915773b6720ff0c is removed from metadata 2024.10.22 00:42:48.943847 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 16531 2024.10.22 00:42:48.950987 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 922da56e4e09cac7f900bdc25117b7b7 for removal 2024.10.22 00:42:48.955204 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 1387 2024.10.22 00:42:48.956201 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 60be7c3c6047ec50eb8c887b1193d5d2, offset: 0, size: 239 2024.10.22 00:42:48.959665 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 60be7c3c6047ec50eb8c887b1193d5d2 for removal 2024.10.22 00:42:48.972206 [ 1476 ] {} CacheMetadata: Key 60be7c3c6047ec50eb8c887b1193d5d2 is removed from metadata 2024.10.22 00:42:48.976984 [ 1476 ] {} CacheMetadata: Key 922da56e4e09cac7f900bdc25117b7b7 is removed from metadata 2024.10.22 00:42:49.022401 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:49.025089 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:49.083287 [ 1476 ] {} CacheMetadata: Key 6e60822dba2af1c2d6c0c54e272da506 is removed from metadata 2024.10.22 00:42:49.282860 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:49.286308 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:49.352401 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_294_294_0 (state Temporary) 2024.10.22 00:42:49.386120 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 6.37 MiB, will set to 1.67 GiB (RSS), difference: -51.56 MiB 2024.10.22 00:42:49.388531 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_164_164_0 (state Temporary) 2024.10.22 00:42:49.389098 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_294_294_0 to 202410_307_307_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:49.430511 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_12_12_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:42:49.472769 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_164_164_0 to 202410_171_171_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:49.489626 [ 1257 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_166_33 to 202410_171_171_0 2024.10.22 00:42:49.511683 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_201_40, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0, 202410_205_205_0, 202410_206_206_0] 2024.10.22 00:42:49.515369 [ 1257 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:49.515670 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_201_40, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0, 202410_205_205_0, 202410_206_206_0] 2024.10.22 00:42:49.521480 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:49.543002 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16531 2024.10.22 00:42:49.560583 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_201_40, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0, 202410_205_205_0, 202410_206_206_0] 2024.10.22 00:42:49.562486 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:42:49.567881 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:42:49.573042 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_166_33 to 202410_171_171_0 into Compact with storage Full 2024.10.22 00:42:49.596485 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 285088 2024.10.22 00:42:49.689523 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:42:49.694788 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} MergeTreeSequentialSource: Reading 37 marks from part 202410_1_166_33, total 290230 rows starting from the beginning of the part 2024.10.22 00:42:49.712609 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_167_167_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:42:49.743636 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_168_168_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:42:49.767807 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_169_169_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:42:49.779392 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_170_170_0, total 1880 rows starting from the beginning of the part 2024.10.22 00:42:49.988801 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_171_171_0, total 1576 rows starting from the beginning of the part 2024.10.22 00:42:50.545602 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.82 GiB, peak 1.88 GiB, free memory in arenas 6.41 MiB, will set to 1.78 GiB (RSS), difference: -39.93 MiB 2024.10.22 00:42:50.627988 [ 1248 ] {} DatabaseCatalog: Execution took 257 ms. 2024.10.22 00:42:51.745148 [ 1695 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/eyz/utljejbsdarftyeqidczugpccclul, size 1148739 2024.10.22 00:42:51.948386 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column URLRegions 2024.10.22 00:42:51.964990 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column URLRegions 2024.10.22 00:42:51.997159 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column URLRegions 2024.10.22 00:42:52.040603 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column URLRegions 2024.10.22 00:42:52.072306 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column URLRegions 2024.10.22 00:42:52.093490 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column URLRegions 2024.10.22 00:42:52.144074 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column URLRegions 2024.10.22 00:42:52.184961 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 2409 2024.10.22 00:42:52.244856 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kcc/ildtfbiqlmhpzdffgnligrvwlcyqd (LogSeriesLimiter: on interval from 2024-10-22 00:42:27 to 2024-10-22 00:42:52 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:42:52.288199 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:52.363288 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:52.484048 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_172_172_0 (state Temporary) 2024.10.22 00:42:52.521735 [ 1211 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:42:52.586067 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_172_172_0 to 202410_185_185_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:52.615571 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.88 GiB, free memory in arenas 9.27 MiB, will set to 1.61 GiB (RSS), difference: 292.41 MiB 2024.10.22 00:42:52.561826 [ 1211 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:42:52.956473 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2409 2024.10.22 00:42:53.125515 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 495.61 MiB using storage policy from min volume index 0 2024.10.22 00:42:53.556316 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 16589 2024.10.22 00:42:53.888064 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:53.899496 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:53.977925 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_171_34/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:42:54.068289 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_295_295_0 (state Temporary) 2024.10.22 00:42:54.145098 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_295_295_0 to 202410_308_308_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:54.180500 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_303_61 to 202410_308_308_0 2024.10.22 00:42:54.189637 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:54.192733 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:54.222662 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_303_61 to 202410_308_308_0 into Compact with storage Full 2024.10.22 00:42:54.305751 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16589 2024.10.22 00:42:54.356443 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:42:54.363644 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_303_61, total 17196 rows starting from the beginning of the part 2024.10.22 00:42:54.502556 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_304_304_0, total 34 rows starting from the beginning of the part 2024.10.22 00:42:54.536789 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_305_305_0, total 19 rows starting from the beginning of the part 2024.10.22 00:42:54.555775 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 12.36 MiB, will set to 1.62 GiB (RSS), difference: 4.93 MiB 2024.10.22 00:42:54.708480 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_306_306_0, total 60 rows starting from the beginning of the part 2024.10.22 00:42:54.750941 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_307_307_0, total 53 rows starting from the beginning of the part 2024.10.22 00:42:54.778366 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_308_308_0, total 58 rows starting from the beginning of the part 2024.10.22 00:42:55.422840 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 66 entries to flush up to offset 11112 2024.10.22 00:42:55.764734 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:55.767933 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:42:55.770365 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 156 2024.10.22 00:42:55.795906 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_171_171_0 (state Temporary) 2024.10.22 00:42:55.849591 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_171_171_0 to 202410_177_177_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:55.870493 [ 1151 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_172_35 to 202410_177_177_0 2024.10.22 00:42:55.883154 [ 1151 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:55.886823 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:55.906599 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_172_35 to 202410_177_177_0 into Compact with storage Full 2024.10.22 00:42:55.937443 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11112 2024.10.22 00:42:55.973200 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:42:55.981038 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_172_35, total 10947 rows starting from the beginning of the part 2024.10.22 00:42:55.996764 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_173_173_0, total 89 rows starting from the beginning of the part 2024.10.22 00:42:56.029367 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_174_174_0, total 29 rows starting from the beginning of the part 2024.10.22 00:42:56.068374 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_175_175_0, total 24 rows starting from the beginning of the part 2024.10.22 00:42:56.108062 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_176_176_0, total 61 rows starting from the beginning of the part 2024.10.22 00:42:56.158596 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:56.162878 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:42:56.220809 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_138_138_0 (state Temporary) 2024.10.22 00:42:56.312139 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_138_138_0 to 202410_148_148_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:56.368394 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} MergeTask::MergeProjectionsStage: Merge sorted 17420 rows, containing 13 columns (13 merged, 0 gathered) in 2.151420352 sec., 8096.976485234996 rows/sec., 1.21 MiB/sec. 2024.10.22 00:42:56.370159 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_177_177_0, total 66 rows starting from the beginning of the part 2024.10.22 00:42:56.392049 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} MergedBlockOutputStream: filled checksums 202410_1_308_62 (state Temporary) 2024.10.22 00:42:56.392066 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 156 2024.10.22 00:42:56.481264 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_308_62 to 202410_1_308_62 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:56.483863 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_308_62} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_303_61, 202410_308_308_0] -> 202410_1_308_62 2024.10.22 00:42:56.519793 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.73 MiB. 2024.10.22 00:42:56.590438 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.88 GiB, free memory in arenas 12.37 MiB, will set to 1.63 GiB (RSS), difference: -164.50 MiB 2024.10.22 00:42:56.600580 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 287008 2024.10.22 00:42:57.007345 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rwr/dfuzwtdqnahonoqtqngfqgfthdsid (LogSeriesLimiter: on interval from 2024-10-22 00:42:52 to 2024-10-22 00:42:57 accepted series 1 / 139 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:42:57.816597 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:57.824194 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:42:57.838374 [ 1172 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 00:42:58.043180 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_165_165_0 (state Temporary) 2024.10.22 00:42:58.109226 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_165_165_0 to 202410_172_172_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:58.160502 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 287008 2024.10.22 00:42:58.292127 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.88 GiB, free memory in arenas 14.95 MiB, will set to 1.63 GiB (RSS), difference: -154.51 MiB 2024.10.22 00:42:58.314771 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 16646 2024.10.22 00:42:58.603344 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} MergeTask::MergeProjectionsStage: Merge sorted 11216 rows, containing 13 columns (13 merged, 0 gathered) in 2.708046884 sec., 4141.7303615634155 rows/sec., 1.73 MiB/sec. 2024.10.22 00:42:58.626770 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_121_24, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0] 2024.10.22 00:42:58.633123 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_121_24, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0] 2024.10.22 00:42:58.671812 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:42:58.674304 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.67 GiB. 2024.10.22 00:42:58.678763 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_121_24, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0] 2024.10.22 00:42:58.761307 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} MergedBlockOutputStream: filled checksums 202410_1_177_36 (state Temporary) 2024.10.22 00:42:58.762304 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_296_296_0 (state Temporary) 2024.10.22 00:42:58.805516 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_296_296_0 to 202410_309_309_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:58.945708 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_177_36 to 202410_1_177_36 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:42:58.962795 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_177_36} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_172_35, 202410_177_177_0] -> 202410_1_177_36 2024.10.22 00:42:58.988545 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.17 MiB. 2024.10.22 00:42:59.062154 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16646 2024.10.22 00:42:59.379540 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.88 GiB, free memory in arenas 13.94 MiB, will set to 1.63 GiB (RSS), difference: -117.33 MiB 2024.10.22 00:42:59.562050 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:42:59.564019 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:43:00.459765 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 2429 2024.10.22 00:43:00.546148 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:00.551618 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:00.593717 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_173_173_0 (state Temporary) 2024.10.22 00:43:00.642837 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_173_173_0 to 202410_186_186_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:00.681466 [ 1173 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_181_36 to 202410_186_186_0 2024.10.22 00:43:00.686300 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2429 2024.10.22 00:43:00.687227 [ 1173 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:00.689734 [ 1173 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:00.828100 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_181_36 to 202410_186_186_0 into Compact with storage Full 2024.10.22 00:43:00.973764 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:43:00.980297 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_181_36, total 2472 rows starting from the beginning of the part 2024.10.22 00:43:00.989781 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_182_182_0, total 14 rows starting from the beginning of the part 2024.10.22 00:43:01.000429 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_183_183_0, total 6 rows starting from the beginning of the part 2024.10.22 00:43:01.017463 [ 1160 ] {} DatabaseCatalog: Execution took 387 ms. 2024.10.22 00:43:01.021213 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_184_184_0, total 11 rows starting from the beginning of the part 2024.10.22 00:43:01.063037 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_185_185_0, total 12 rows starting from the beginning of the part 2024.10.22 00:43:01.114680 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_186_186_0, total 20 rows starting from the beginning of the part 2024.10.22 00:43:01.591824 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.88 GiB, free memory in arenas 13.36 MiB, will set to 1.63 GiB (RSS), difference: -109.63 MiB 2024.10.22 00:43:02.072064 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} MergeTask::MergeProjectionsStage: Merge sorted 298678 rows, containing 4 columns (4 merged, 0 gathered) in 12.511846386 sec., 23871.616609216257 rows/sec., 350.34 KiB/sec. 2024.10.22 00:43:02.171888 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} MergeTask::MergeProjectionsStage: Merge sorted 2535 rows, containing 12 columns (12 merged, 0 gathered) in 1.450354032 sec., 1747.8491072309441 rows/sec., 271.52 KiB/sec. 2024.10.22 00:43:02.262583 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} MergedBlockOutputStream: filled checksums 202410_1_171_34 (state Temporary) 2024.10.22 00:43:02.404333 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_171_34 to 202410_1_171_34 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:02.411716 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_171_34} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_166_33, 202410_171_171_0] -> 202410_1_171_34 2024.10.22 00:43:02.412001 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} MergedBlockOutputStream: filled checksums 202410_1_186_37 (state Temporary) 2024.10.22 00:43:02.481505 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:43:02.604635 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_186_37 to 202410_1_186_37 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:02.633299 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_186_37} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_181_36, 202410_186_186_0] -> 202410_1_186_37 2024.10.22 00:43:02.676958 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:43:03.076437 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 16692 2024.10.22 00:43:03.339899 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:03.342262 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:03.382342 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_297_297_0 (state Temporary) 2024.10.22 00:43:03.416958 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_297_297_0 to 202410_310_310_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:03.482869 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16692 2024.10.22 00:43:03.502384 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 14.11 MiB, will set to 1.64 GiB (RSS), difference: 6.02 MiB 2024.10.22 00:43:03.508635 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 11183 2024.10.22 00:43:03.829773 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:03.834466 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:03.883316 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_172_172_0 (state Temporary) 2024.10.22 00:43:03.897357 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 157 2024.10.22 00:43:03.954425 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_172_172_0 to 202410_178_178_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:04.002771 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11183 2024.10.22 00:43:04.082693 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:04.085078 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:04.160830 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_139_139_0 (state Temporary) 2024.10.22 00:43:04.278726 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:43:04.353158 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_139_139_0 to 202410_149_149_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:04.760028 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 157 2024.10.22 00:43:05.167366 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 288928 2024.10.22 00:43:05.888329 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 9.70 MiB, will set to 1.64 GiB (RSS), difference: 2.67 MiB 2024.10.22 00:43:06.195486 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:06.206482 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:06.366114 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_166_166_0 (state Temporary) 2024.10.22 00:43:06.522568 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 00:43:06.564654 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_166_166_0 to 202410_173_173_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:06.584010 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 288928 2024.10.22 00:43:07.486569 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 28 entries to flush up to offset 16720 2024.10.22 00:43:07.656002 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:07.660185 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:07.667278 [ 1224 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:43:07.679866 [ 1224 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:43:07.704038 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_298_298_0 (state Temporary) 2024.10.22 00:43:07.755122 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_298_298_0 to 202410_311_311_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:07.916150 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16720 2024.10.22 00:43:07.967397 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:07.971110 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:08.189074 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 2449 2024.10.22 00:43:08.404635 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:08.410359 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:08.488010 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_174_174_0 (state Temporary) 2024.10.22 00:43:08.649726 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_174_174_0 to 202410_187_187_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:08.708077 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2449 2024.10.22 00:43:09.561758 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:43:09.563847 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:43:09.582807 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 9.88 MiB, will set to 1.66 GiB (RSS), difference: -12.38 MiB 2024.10.22 00:43:10.029591 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_111_22, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0] 2024.10.22 00:43:10.031354 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_111_22, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0] 2024.10.22 00:43:10.169032 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_111_22, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0] 2024.10.22 00:43:11.309419 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 8.94 MiB, will set to 1.67 GiB (RSS), difference: 1.52 MiB 2024.10.22 00:43:11.320934 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_206_41, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0, 202410_210_210_0, 202410_211_211_0] 2024.10.22 00:43:11.328719 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_206_41, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0, 202410_210_210_0, 202410_211_211_0] 2024.10.22 00:43:11.354455 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_206_41, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0, 202410_210_210_0, 202410_211_211_0] 2024.10.22 00:43:11.414861 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_35_35_0 (state Temporary) 2024.10.22 00:43:11.422797 [ 1249 ] {} DatabaseCatalog: Execution took 306 ms. 2024.10.22 00:43:11.526871 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 26 entries to flush up to offset 11209 2024.10.22 00:43:11.626367 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:11.634569 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:11.678381 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_173_173_0 (state Temporary) 2024.10.22 00:43:11.815333 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_173_173_0 to 202410_179_179_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:11.901174 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11209 2024.10.22 00:43:11.939649 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 16754 2024.10.22 00:43:12.113002 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:12.117502 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:12.183321 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_299_299_0 (state Temporary) 2024.10.22 00:43:12.259581 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_299_299_0 to 202410_312_312_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:12.264579 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 158 2024.10.22 00:43:12.307110 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16754 2024.10.22 00:43:12.483633 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 9.52 MiB, will set to 1.67 GiB (RSS), difference: 1.50 MiB 2024.10.22 00:43:12.752007 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:12.755740 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 764813 bytes for daedfdfd044c677a7992dc95673d365c:0) 2024.10.22 00:43:12.756170 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:12.758942 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e82df546095a90ca463b83ad73f90e0d, offset: 0, size: 1009484 2024.10.22 00:43:12.762473 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e82df546095a90ca463b83ad73f90e0d for removal 2024.10.22 00:43:12.764153 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4f5013ff1704e133845674b4813df2f2, offset: 0, size: 246 2024.10.22 00:43:12.764701 [ 1476 ] {} CacheMetadata: Key e82df546095a90ca463b83ad73f90e0d is removed from metadata 2024.10.22 00:43:12.773069 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4f5013ff1704e133845674b4813df2f2 for removal 2024.10.22 00:43:12.782803 [ 1476 ] {} CacheMetadata: Key 4f5013ff1704e133845674b4813df2f2 is removed from metadata 2024.10.22 00:43:12.803733 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_140_140_0 (state Temporary) 2024.10.22 00:43:12.980037 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 907024 bytes for 3a885f4b6d48fa8efa56ba9fc5f37fb4:0) 2024.10.22 00:43:12.981635 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 502d2c78448b54d19b33fde64d6ef0f1, offset: 0, size: 657386 2024.10.22 00:43:12.985538 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 502d2c78448b54d19b33fde64d6ef0f1 for removal 2024.10.22 00:43:12.988114 [ 1476 ] {} CacheMetadata: Key 502d2c78448b54d19b33fde64d6ef0f1 is removed from metadata 2024.10.22 00:43:12.988292 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5a4cca3805ef61c6c9d40fe60525fb91, offset: 0, size: 238 2024.10.22 00:43:12.990940 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5a4cca3805ef61c6c9d40fe60525fb91 for removal 2024.10.22 00:43:12.993344 [ 1476 ] {} CacheMetadata: Key 5a4cca3805ef61c6c9d40fe60525fb91 is removed from metadata 2024.10.22 00:43:13.116325 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_140_140_0 to 202410_150_150_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:13.179055 [ 1698 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kcc/ildtfbiqlmhpzdffgnligrvwlcyqd, size 907024 2024.10.22 00:43:13.222849 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 158 2024.10.22 00:43:13.287587 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RefererRegions 2024.10.22 00:43:13.326920 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RefererRegions 2024.10.22 00:43:13.355788 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RefererRegions 2024.10.22 00:43:13.381326 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RefererRegions 2024.10.22 00:43:13.505763 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RefererRegions 2024.10.22 00:43:13.531373 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RefererRegions 2024.10.22 00:43:13.588510 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 290848 2024.10.22 00:43:13.677533 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RefererRegions 2024.10.22 00:43:13.718993 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pfz/kzxmviuodqttakisytcqnkiruljxn (LogSeriesLimiter: on interval from 2024-10-22 00:42:57 to 2024-10-22 00:43:13 accepted series 1 / 220 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:43:14.419717 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 7.57 MiB, will set to 1.67 GiB (RSS), difference: -4.36 MiB 2024.10.22 00:43:14.676612 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:14.679325 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:14.952797 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_167_167_0 (state Temporary) 2024.10.22 00:43:15.122525 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_167_167_0 to 202410_174_174_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:15.276352 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 290848 2024.10.22 00:43:16.210266 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 2454 2024.10.22 00:43:16.396604 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:16.405222 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:16.414825 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 16798 2024.10.22 00:43:16.559593 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_175_175_0 (state Temporary) 2024.10.22 00:43:16.719048 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_115_23, 202410_116_116_0, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0] 2024.10.22 00:43:16.722224 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_115_23, 202410_116_116_0, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0] 2024.10.22 00:43:16.732321 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_175_175_0 to 202410_188_188_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:16.792368 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2454 2024.10.22 00:43:16.861526 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_115_23, 202410_116_116_0, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0] 2024.10.22 00:43:16.877651 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:16.885473 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:16.936261 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_300_300_0 (state Temporary) 2024.10.22 00:43:17.078606 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_300_300_0 to 202410_313_313_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:17.125119 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_308_62 to 202410_313_313_0 2024.10.22 00:43:17.147963 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:17.157580 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:17.177608 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_308_62 to 202410_313_313_0 into Compact with storage Full 2024.10.22 00:43:17.259137 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16798 2024.10.22 00:43:17.375967 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:43:17.599375 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_308_62, total 17420 rows starting from the beginning of the part 2024.10.22 00:43:17.970189 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_35_35_0 to 202410_38_38_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:18.172036 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_309_309_0, total 57 rows starting from the beginning of the part 2024.10.22 00:43:18.255460 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_310_310_0, total 46 rows starting from the beginning of the part 2024.10.22 00:43:18.306932 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_311_311_0, total 28 rows starting from the beginning of the part 2024.10.22 00:43:18.313123 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 7.18 MiB, will set to 1.68 GiB (RSS), difference: 2.46 MiB 2024.10.22 00:43:18.384853 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_312_312_0, total 34 rows starting from the beginning of the part 2024.10.22 00:43:18.593765 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_313_313_0, total 44 rows starting from the beginning of the part 2024.10.22 00:43:19.357713 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_313_63/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:43:19.376643 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 6.61 MiB, will set to 1.68 GiB (RSS), difference: -5.26 MiB 2024.10.22 00:43:19.409729 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 51 entries to flush up to offset 11260 2024.10.22 00:43:19.519170 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:19.526871 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:19.563865 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:43:19.566646 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:43:19.574252 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_174_174_0 (state Temporary) 2024.10.22 00:43:19.718822 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_174_174_0 to 202410_180_180_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:19.768564 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11260 2024.10.22 00:43:20.831371 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 159 2024.10.22 00:43:21.084464 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:21.085486 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} MergeTask::MergeProjectionsStage: Merge sorted 17629 rows, containing 13 columns (13 merged, 0 gathered) in 3.917268518 sec., 4500.329737161 rows/sec., 686.25 KiB/sec. 2024.10.22 00:43:21.088748 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:21.158587 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} MergedBlockOutputStream: filled checksums 202410_1_313_63 (state Temporary) 2024.10.22 00:43:21.180101 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_141_141_0 (state Temporary) 2024.10.22 00:43:21.266915 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 16841 2024.10.22 00:43:21.358932 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_141_141_0 to 202410_151_151_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:21.373402 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 7.19 MiB, will set to 1.69 GiB (RSS), difference: 7.53 MiB 2024.10.22 00:43:21.389041 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:21.395645 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:21.428274 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 159 2024.10.22 00:43:21.436190 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_301_301_0 (state Temporary) 2024.10.22 00:43:21.489499 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_301_301_0 to 202410_314_314_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:21.523809 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1387 2024.10.22 00:43:21.545253 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_313_63 to 202410_1_313_63 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:21.555140 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_313_63} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_308_62, 202410_313_313_0] -> 202410_1_313_63 2024.10.22 00:43:21.568108 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16841 2024.10.22 00:43:21.576511 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.73 MiB. 2024.10.22 00:43:21.625228 [ 1180 ] {} DatabaseCatalog: Execution took 201 ms. 2024.10.22 00:43:22.280642 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 292768 2024.10.22 00:43:22.782791 [ 1157 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:43:22.820070 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 13edbad82cf5b37bc07f1066b7c65045:0) 2024.10.22 00:43:22.821912 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 662b16d70c5cf28002a65d3d73454522, offset: 0, size: 902343 2024.10.22 00:43:22.827158 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 662b16d70c5cf28002a65d3d73454522 for removal 2024.10.22 00:43:22.831340 [ 1476 ] {} CacheMetadata: Key 662b16d70c5cf28002a65d3d73454522 is removed from metadata 2024.10.22 00:43:22.918265 [ 1157 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:43:22.933113 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c9ba865289a363a1d5188528a2f3003b, offset: 0, size: 244 2024.10.22 00:43:22.943130 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c9ba865289a363a1d5188528a2f3003b for removal 2024.10.22 00:43:22.952025 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ade196c515678e4eca5063d5ce826578, offset: 0, size: 465506 2024.10.22 00:43:22.956002 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ade196c515678e4eca5063d5ce826578 for removal 2024.10.22 00:43:23.059398 [ 1476 ] {} CacheMetadata: Key ade196c515678e4eca5063d5ce826578 is removed from metadata 2024.10.22 00:43:23.060038 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0ec3425e7061e15eb6a266dc08f0c1a8, offset: 0, size: 235 2024.10.22 00:43:23.064194 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0ec3425e7061e15eb6a266dc08f0c1a8 for removal 2024.10.22 00:43:23.067283 [ 1476 ] {} CacheMetadata: Key 0ec3425e7061e15eb6a266dc08f0c1a8 is removed from metadata 2024.10.22 00:43:23.074459 [ 1476 ] {} CacheMetadata: Key c9ba865289a363a1d5188528a2f3003b is removed from metadata 2024.10.22 00:43:23.295335 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:23.306818 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:23.392064 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 8.72 MiB, will set to 1.69 GiB (RSS), difference: 6.52 MiB 2024.10.22 00:43:23.526947 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_168_168_0 (state Temporary) 2024.10.22 00:43:23.619681 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_168_168_0 to 202410_175_175_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:23.961732 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 292768 2024.10.22 00:43:24.397186 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 2468 2024.10.22 00:43:24.483732 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:24.486310 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:24.644647 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_176_176_0 (state Temporary) 2024.10.22 00:43:24.785409 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_176_176_0 to 202410_189_189_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:24.848701 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2468 2024.10.22 00:43:25.128062 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_98_20, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0, 202410_102_102_0, 202410_103_103_0] 2024.10.22 00:43:25.133481 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_98_20, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0, 202410_102_102_0, 202410_103_103_0] 2024.10.22 00:43:25.368395 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_98_20, 202410_99_99_0, 202410_100_100_0, 202410_101_101_0, 202410_102_102_0, 202410_103_103_0] 2024.10.22 00:43:25.421419 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 9.19 MiB, will set to 1.70 GiB (RSS), difference: 8.43 MiB 2024.10.22 00:43:25.572698 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 16887 2024.10.22 00:43:25.690497 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:25.696554 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:25.766369 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_302_302_0 (state Temporary) 2024.10.22 00:43:25.803663 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_302_302_0 to 202410_315_315_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:25.942516 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:43:26.174097 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16887 2024.10.22 00:43:27.275263 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 27 entries to flush up to offset 11287 2024.10.22 00:43:27.319862 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 8.42 MiB, will set to 1.70 GiB (RSS), difference: 2.11 MiB 2024.10.22 00:43:27.380479 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:27.385681 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:27.429509 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_175_175_0 (state Temporary) 2024.10.22 00:43:27.568119 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_175_175_0 to 202410_181_181_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:27.699081 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11287 2024.10.22 00:43:28.767394 [ 1221 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 00:43:29.075853 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 1428 2024.10.22 00:43:29.564535 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:43:29.566818 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:43:30.176051 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 18 entries to flush up to offset 16905 2024.10.22 00:43:30.389161 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:30.392170 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:30.562255 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_303_303_0 (state Temporary) 2024.10.22 00:43:30.670521 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_303_303_0 to 202410_316_316_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:30.695577 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16905 2024.10.22 00:43:30.978318 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 294688 2024.10.22 00:43:31.526614 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 8.79 MiB, will set to 1.71 GiB (RSS), difference: 14.07 MiB 2024.10.22 00:43:31.811194 [ 1677 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rwr/dfuzwtdqnahonoqtqngfqgfthdsid, size 278500 2024.10.22 00:43:31.842453 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:31.857085 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.66 GiB. 2024.10.22 00:43:31.884716 [ 1243 ] {} DatabaseCatalog: Execution took 247 ms. 2024.10.22 00:43:32.091772 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 13edbad82cf5b37bc07f1066b7c65045:0) 2024.10.22 00:43:32.094617 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e9e8c49e56ef9a63fcf94809ca6e50c8, offset: 0, size: 579269 2024.10.22 00:43:32.103840 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e9e8c49e56ef9a63fcf94809ca6e50c8 for removal 2024.10.22 00:43:32.108217 [ 1476 ] {} CacheMetadata: Key e9e8c49e56ef9a63fcf94809ca6e50c8 is removed from metadata 2024.10.22 00:43:32.109456 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 41a33ee1ba533e5476f1ea231bf3711c, offset: 0, size: 238 2024.10.22 00:43:32.115658 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 41a33ee1ba533e5476f1ea231bf3711c for removal 2024.10.22 00:43:32.157169 [ 1476 ] {} CacheMetadata: Key 41a33ee1ba533e5476f1ea231bf3711c is removed from metadata 2024.10.22 00:43:32.354239 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 4 entries to flush up to offset 2472 2024.10.22 00:43:32.470232 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_169_169_0 (state Temporary) 2024.10.22 00:43:32.657604 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_169_169_0 to 202410_176_176_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:32.718452 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_171_34 to 202410_176_176_0 2024.10.22 00:43:32.735043 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 294688 2024.10.22 00:43:32.760245 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:32.766400 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.65 GiB. 2024.10.22 00:43:32.901160 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_171_34 to 202410_176_176_0 into Compact with storage Full 2024.10.22 00:43:32.932988 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:32.936093 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:43:32.942416 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.64 GiB. 2024.10.22 00:43:32.946470 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} MergeTreeSequentialSource: Reading 38 marks from part 202410_1_171_34, total 298678 rows starting from the beginning of the part 2024.10.22 00:43:32.960907 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_172_172_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:43:32.982117 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_173_173_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:43:33.065266 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_177_177_0 (state Temporary) 2024.10.22 00:43:33.065522 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_174_174_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:43:33.103847 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_175_175_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:43:33.159656 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_176_176_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:43:33.269410 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_177_177_0 to 202410_190_190_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:33.330588 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2472 2024.10.22 00:43:33.771598 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.88 GiB, free memory in arenas 6.35 MiB, will set to 1.70 GiB (RSS), difference: 147.59 MiB 2024.10.22 00:43:34.713764 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 16945 2024.10.22 00:43:34.883162 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_19_4, 202410_20_20_0, 202410_21_21_0, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0] 2024.10.22 00:43:34.885515 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_19_4, 202410_20_20_0, 202410_21_21_0, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0] 2024.10.22 00:43:35.026944 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_19_4, 202410_20_20_0, 202410_21_21_0, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0] 2024.10.22 00:43:35.205042 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 78 entries to flush up to offset 11365 2024.10.22 00:43:35.301420 [ 1264 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_147_30 to 202410_151_151_0 2024.10.22 00:43:35.337236 [ 1264 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:35.450945 [ 1264 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.61 GiB. 2024.10.22 00:43:35.489069 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_151_31} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_147_30 to 202410_151_151_0 into Compact with storage Full 2024.10.22 00:43:35.506117 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:35.518446 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.60 GiB. 2024.10.22 00:43:35.606222 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_176_176_0 (state Temporary) 2024.10.22 00:43:35.697319 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_176_176_0 to 202410_182_182_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:35.734066 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:35.744289 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.59 GiB. 2024.10.22 00:43:35.765307 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_151_31} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:43:35.787828 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_151_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_147_30, total 191 rows starting from the beginning of the part 2024.10.22 00:43:35.795333 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_304_304_0 (state Temporary) 2024.10.22 00:43:35.797610 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.88 GiB, free memory in arenas 8.10 MiB, will set to 1.67 GiB (RSS), difference: 146.59 MiB 2024.10.22 00:43:35.811928 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 872160 bytes for eab7361f011deb475088b433a4812d2f:0) 2024.10.22 00:43:35.819557 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d099bfa1196adfbcfb6ba292724af018, offset: 0, size: 232 2024.10.22 00:43:35.828819 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d099bfa1196adfbcfb6ba292724af018 for removal 2024.10.22 00:43:35.834977 [ 1476 ] {} CacheMetadata: Key d099bfa1196adfbcfb6ba292724af018 is removed from metadata 2024.10.22 00:43:35.835595 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 34978f75c5206cde86e62438a78c653b, offset: 0, size: 329086 2024.10.22 00:43:35.846340 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 34978f75c5206cde86e62438a78c653b for removal 2024.10.22 00:43:35.844177 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11365 2024.10.22 00:43:35.864336 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b791e10e4cbc44454557dfc0e7c809c1, offset: 0, size: 326823 2024.10.22 00:43:35.868929 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b791e10e4cbc44454557dfc0e7c809c1 for removal 2024.10.22 00:43:35.869724 [ 1476 ] {} CacheMetadata: Key 34978f75c5206cde86e62438a78c653b is removed from metadata 2024.10.22 00:43:35.872059 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5184bbb7102b7932f86829d8e7dfa79c, offset: 0, size: 232 2024.10.22 00:43:35.873725 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_304_304_0 to 202410_317_317_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:35.874653 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5184bbb7102b7932f86829d8e7dfa79c for removal 2024.10.22 00:43:35.874952 [ 1270 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_177_36 to 202410_182_182_0 2024.10.22 00:43:35.890393 [ 1476 ] {} CacheMetadata: Key 5184bbb7102b7932f86829d8e7dfa79c is removed from metadata 2024.10.22 00:43:35.920502 [ 1270 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:35.922295 [ 1476 ] {} CacheMetadata: Key b791e10e4cbc44454557dfc0e7c809c1 is removed from metadata 2024.10.22 00:43:35.931373 [ 1270 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:35.949068 [ 1270 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 204 ms. 2024.10.22 00:43:36.010888 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_13_13_0 (state Temporary) 2024.10.22 00:43:36.032550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nqm/wlwhibesgsqvkkuvesdcaxaabwtpy (LogSeriesLimiter: on interval from 2024-10-22 00:43:13 to 2024-10-22 00:43:36 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:43:36.056546 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 16945 2024.10.22 00:43:36.075064 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_177_36 to 202410_182_182_0 into Compact with storage Full 2024.10.22 00:43:36.110528 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_151_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_148_148_0, total 1 rows starting from the beginning of the part 2024.10.22 00:43:36.144613 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 417003 bytes for 13edbad82cf5b37bc07f1066b7c65045:0) 2024.10.22 00:43:36.164335 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 486b06a4a7e24a25e79ca8279b1df483, offset: 0, size: 651502 2024.10.22 00:43:36.189954 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_211_42, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0, 202410_215_215_0, 202410_216_216_0] 2024.10.22 00:43:36.191481 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_211_42, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0, 202410_215_215_0, 202410_216_216_0] 2024.10.22 00:43:36.248186 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_211_42, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0, 202410_215_215_0, 202410_216_216_0] 2024.10.22 00:43:36.267051 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_151_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_149_149_0, total 1 rows starting from the beginning of the part 2024.10.22 00:43:36.274968 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 486b06a4a7e24a25e79ca8279b1df483 for removal 2024.10.22 00:43:36.278829 [ 1476 ] {} CacheMetadata: Key 486b06a4a7e24a25e79ca8279b1df483 is removed from metadata 2024.10.22 00:43:36.279342 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c04452fc01aec3356e1224a0bf27b783, offset: 0, size: 239 2024.10.22 00:43:36.282221 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c04452fc01aec3356e1224a0bf27b783 for removal 2024.10.22 00:43:36.285249 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6c4a5db52ca1970c012ffcc136382a18, offset: 0, size: 276155 2024.10.22 00:43:36.286291 [ 1476 ] {} CacheMetadata: Key c04452fc01aec3356e1224a0bf27b783 is removed from metadata 2024.10.22 00:43:36.289637 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6c4a5db52ca1970c012ffcc136382a18 for removal 2024.10.22 00:43:36.297467 [ 1476 ] {} CacheMetadata: Key 6c4a5db52ca1970c012ffcc136382a18 is removed from metadata 2024.10.22 00:43:36.298257 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 745dd8cece9cb88578bf462322b22b9d, offset: 0, size: 231 2024.10.22 00:43:36.306481 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:43:36.312457 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 745dd8cece9cb88578bf462322b22b9d for removal 2024.10.22 00:43:36.319157 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_177_36, total 11216 rows starting from the beginning of the part 2024.10.22 00:43:36.323888 [ 1476 ] {} CacheMetadata: Key 745dd8cece9cb88578bf462322b22b9d is removed from metadata 2024.10.22 00:43:36.348153 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_178_178_0, total 71 rows starting from the beginning of the part 2024.10.22 00:43:36.371890 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_179_179_0, total 26 rows starting from the beginning of the part 2024.10.22 00:43:36.383535 [ 1728 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nqm/wlwhibesgsqvkkuvesdcaxaabwtpy, size 4 2024.10.22 00:43:36.437164 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 160 2024.10.22 00:43:36.484718 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_180_180_0, total 51 rows starting from the beginning of the part 2024.10.22 00:43:36.538799 [ 1732 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pfz/kzxmviuodqttakisytcqnkiruljxn, size 2514155 2024.10.22 00:43:36.581173 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_151_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_150_150_0, total 1 rows starting from the beginning of the part 2024.10.22 00:43:36.632126 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_181_181_0, total 27 rows starting from the beginning of the part 2024.10.22 00:43:36.679622 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_151_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_151_151_0, total 1 rows starting from the beginning of the part 2024.10.22 00:43:36.689564 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:36.696507 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:36.766477 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_182_182_0, total 78 rows starting from the beginning of the part 2024.10.22 00:43:36.777780 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_142_142_0 (state Temporary) 2024.10.22 00:43:36.968337 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 00:43:36.987064 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 00:43:37.002962 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_142_142_0 to 202410_152_152_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:37.014854 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 00:43:37.045613 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 00:43:37.074088 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 00:43:37.105749 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 00:43:37.117225 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 160 2024.10.22 00:43:37.129839 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 00:43:37.262491 [ 1211 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 00:43:37.665118 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_176_35/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:43:37.667324 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_151_31/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:43:37.724498 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 11.25 MiB, will set to 1.64 GiB (RSS), difference: 3.33 MiB 2024.10.22 00:43:37.925373 [ 1272 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:43:37.958773 [ 1272 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:43:38.145543 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} MergeTask::MergeProjectionsStage: Merge sorted 11469 rows, containing 13 columns (13 merged, 0 gathered) in 2.077421076 sec., 5520.787351442083 rows/sec., 2.31 MiB/sec. 2024.10.22 00:43:38.219040 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} MergedBlockOutputStream: filled checksums 202410_1_182_37 (state Temporary) 2024.10.22 00:43:38.284344 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_126_25, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0] 2024.10.22 00:43:38.286265 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_126_25, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0] 2024.10.22 00:43:38.322181 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_126_25, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0] 2024.10.22 00:43:38.515687 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_182_37 to 202410_1_182_37 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:38.519958 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_182_37} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_177_36, 202410_182_182_0] -> 202410_1_182_37 2024.10.22 00:43:38.555888 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.28 MiB. 2024.10.22 00:43:38.861233 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_151_31} MergeTask::MergeProjectionsStage: Merge sorted 195 rows, containing 32 columns (32 merged, 0 gathered) in 3.384446953 sec., 57.616503584773426 rows/sec., 6.18 KiB/sec. 2024.10.22 00:43:38.887453 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_151_31} MergedBlockOutputStream: filled checksums 202410_1_151_31 (state Temporary) 2024.10.22 00:43:39.409838 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 13.20 MiB, will set to 1.64 GiB (RSS), difference: 4.16 MiB 2024.10.22 00:43:39.565210 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:43:39.567232 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:43:39.571835 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_151_31} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_151_31 to 202410_1_151_31 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:39.573606 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_151_31} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_147_30, 202410_151_151_0] -> 202410_1_151_31 2024.10.22 00:43:39.713774 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:43:39.748333 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1904 entries to flush up to offset 296592 2024.10.22 00:43:40.163141 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 107 entries to flush up to offset 17052 2024.10.22 00:43:40.367072 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fzt/nyzlhfndbahircpdmxtaihhndodtj. 2024.10.22 00:43:40.405393 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:40.413565 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:40.544741 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 13.02 MiB, will set to 1.64 GiB (RSS), difference: 6.39 MiB 2024.10.22 00:43:40.655103 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:40.658270 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:40.674687 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_170_170_0 (state Temporary) 2024.10.22 00:43:40.720482 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_170_170_0 to 202410_177_177_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:40.763070 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 296592 2024.10.22 00:43:40.822913 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_305_305_0 (state Temporary) 2024.10.22 00:43:40.837635 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 2491 2024.10.22 00:43:40.941041 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:40.946352 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:40.981098 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_305_305_0 to 202410_318_318_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:41.020850 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_313_63 to 202410_318_318_0 2024.10.22 00:43:41.044929 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:41.048976 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:41.084058 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_313_63 to 202410_318_318_0 into Compact with storage Full 2024.10.22 00:43:41.087503 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_178_178_0 (state Temporary) 2024.10.22 00:43:41.145094 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17052 2024.10.22 00:43:41.149171 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:43:41.247161 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_178_178_0 to 202410_191_191_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:41.276491 [ 1169 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_186_37 to 202410_191_191_0 2024.10.22 00:43:41.276481 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_313_63, total 17629 rows starting from the beginning of the part 2024.10.22 00:43:41.291579 [ 1169 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:41.294477 [ 1169 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:41.321242 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_314_314_0, total 43 rows starting from the beginning of the part 2024.10.22 00:43:41.334593 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_186_37 to 202410_191_191_0 into Compact with storage Full 2024.10.22 00:43:41.377625 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_315_315_0, total 46 rows starting from the beginning of the part 2024.10.22 00:43:41.398346 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2491 2024.10.22 00:43:41.408882 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:43:41.417607 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_186_37, total 2535 rows starting from the beginning of the part 2024.10.22 00:43:41.483961 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 14.13 MiB, will set to 1.65 GiB (RSS), difference: 1.32 MiB 2024.10.22 00:43:41.532477 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_187_187_0, total 20 rows starting from the beginning of the part 2024.10.22 00:43:41.533648 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_316_316_0, total 18 rows starting from the beginning of the part 2024.10.22 00:43:41.562329 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_188_188_0, total 5 rows starting from the beginning of the part 2024.10.22 00:43:41.678943 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_189_189_0, total 14 rows starting from the beginning of the part 2024.10.22 00:43:41.685518 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_317_317_0, total 40 rows starting from the beginning of the part 2024.10.22 00:43:41.690554 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_190_190_0, total 4 rows starting from the beginning of the part 2024.10.22 00:43:41.716580 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_191_191_0, total 19 rows starting from the beginning of the part 2024.10.22 00:43:41.746426 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_318_318_0, total 107 rows starting from the beginning of the part 2024.10.22 00:43:43.145455 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} MergeTask::MergeProjectionsStage: Merge sorted 2597 rows, containing 12 columns (12 merged, 0 gathered) in 1.823713488 sec., 1424.017542825784 rows/sec., 221.25 KiB/sec. 2024.10.22 00:43:43.270172 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} MergedBlockOutputStream: filled checksums 202410_1_191_38 (state Temporary) 2024.10.22 00:43:43.363227 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 11480 2024.10.22 00:43:43.568818 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 12.75 MiB, will set to 1.65 GiB (RSS), difference: -9.35 MiB 2024.10.22 00:43:43.570959 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_191_38 to 202410_1_191_38 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:43.577722 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_191_38} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_186_37, 202410_191_191_0] -> 202410_1_191_38 2024.10.22 00:43:43.606135 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:43:43.795819 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:43.807878 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:43.992209 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_177_177_0 (state Temporary) 2024.10.22 00:43:44.247969 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_177_177_0 to 202410_183_183_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:44.299068 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11480 2024.10.22 00:43:44.368137 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} MergeTask::MergeProjectionsStage: Merge sorted 17883 rows, containing 13 columns (13 merged, 0 gathered) in 3.29133035 sec., 5433.365265203476 rows/sec., 828.66 KiB/sec. 2024.10.22 00:43:44.432163 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} MergedBlockOutputStream: filled checksums 202410_1_318_64 (state Temporary) 2024.10.22 00:43:44.619713 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 161 2024.10.22 00:43:44.758750 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_318_64 to 202410_1_318_64 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:44.763653 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_318_64} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_313_63, 202410_318_318_0] -> 202410_1_318_64 2024.10.22 00:43:44.828833 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.73 MiB. 2024.10.22 00:43:45.067621 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} MergeTask::MergeProjectionsStage: Merge sorted 308278 rows, containing 4 columns (4 merged, 0 gathered) in 12.177773957 sec., 25314.807212593758 rows/sec., 371.59 KiB/sec. 2024.10.22 00:43:45.172480 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 17113 2024.10.22 00:43:45.229185 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} MergedBlockOutputStream: filled checksums 202410_1_176_35 (state Temporary) 2024.10.22 00:43:45.271669 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_176_35 to 202410_1_176_35 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:45.274351 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_176_35} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_171_34, 202410_176_176_0] -> 202410_1_176_35 2024.10.22 00:43:45.312342 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:43:45.315853 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:45.320619 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:45.343533 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:45.348808 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:45.478965 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_143_143_0 (state Temporary) 2024.10.22 00:43:45.532270 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 8.63 MiB, will set to 1.64 GiB (RSS), difference: 9.17 MiB 2024.10.22 00:43:45.591733 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_306_306_0 (state Temporary) 2024.10.22 00:43:45.675556 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_143_143_0 to 202410_153_153_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:45.879652 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_306_306_0 to 202410_319_319_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:45.988066 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 161 2024.10.22 00:43:46.144483 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17113 2024.10.22 00:43:47.293771 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 9.15 MiB, will set to 1.64 GiB (RSS), difference: 3.74 MiB 2024.10.22 00:43:47.768296 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:47.770270 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:47.807744 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1718 entries to flush up to offset 298310 2024.10.22 00:43:48.654663 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:48.662456 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:48.906160 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 2517 2024.10.22 00:43:48.983061 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:48.985174 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:49.021595 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_179_179_0 (state Temporary) 2024.10.22 00:43:49.065227 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_179_179_0 to 202410_192_192_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:49.138443 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_171_171_0 (state Temporary) 2024.10.22 00:43:49.262471 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2517 2024.10.22 00:43:49.372066 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_171_171_0 to 202410_178_178_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:49.378305 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 17bc03913bdfe997e86579287443c676:0) 2024.10.22 00:43:49.382232 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7579f0fdbd8d7487e29bd26de19fcb21, offset: 0, size: 1870299 2024.10.22 00:43:49.386426 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7579f0fdbd8d7487e29bd26de19fcb21 for removal 2024.10.22 00:43:49.388365 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 298310 2024.10.22 00:43:49.391260 [ 1476 ] {} CacheMetadata: Key 7579f0fdbd8d7487e29bd26de19fcb21 is removed from metadata 2024.10.22 00:43:49.392180 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e67abeafa26f116e45080c0e212845a3, offset: 0, size: 236 2024.10.22 00:43:49.394212 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e67abeafa26f116e45080c0e212845a3 for removal 2024.10.22 00:43:49.403354 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 01331264a3f30124827613eabddb6ef5, offset: 0, size: 250 2024.10.22 00:43:49.414674 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 01331264a3f30124827613eabddb6ef5 for removal 2024.10.22 00:43:49.417085 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 7.87 MiB, will set to 1.64 GiB (RSS), difference: -7.34 MiB 2024.10.22 00:43:49.421177 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 35187ba7fc901f1baee60561192f16a8, offset: 0, size: 161 2024.10.22 00:43:49.431019 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 35187ba7fc901f1baee60561192f16a8 for removal 2024.10.22 00:43:49.433550 [ 1476 ] {} CacheMetadata: Key 01331264a3f30124827613eabddb6ef5 is removed from metadata 2024.10.22 00:43:49.435087 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b81b9882c658d6571bd26e6d73a4f4c8, offset: 0, size: 465534 2024.10.22 00:43:49.446210 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b81b9882c658d6571bd26e6d73a4f4c8 for removal 2024.10.22 00:43:49.453457 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fd6762f0efff28e391b954b542edef0c, offset: 0, size: 2404 2024.10.22 00:43:49.454508 [ 1476 ] {} CacheMetadata: Key b81b9882c658d6571bd26e6d73a4f4c8 is removed from metadata 2024.10.22 00:43:49.462698 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fd6762f0efff28e391b954b542edef0c for removal 2024.10.22 00:43:49.465258 [ 1476 ] {} CacheMetadata: Key 35187ba7fc901f1baee60561192f16a8 is removed from metadata 2024.10.22 00:43:49.470480 [ 1476 ] {} CacheMetadata: Key fd6762f0efff28e391b954b542edef0c is removed from metadata 2024.10.22 00:43:49.475137 [ 1476 ] {} CacheMetadata: Key e67abeafa26f116e45080c0e212845a3 is removed from metadata 2024.10.22 00:43:49.566951 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:43:49.569828 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:43:50.164158 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 17166 2024.10.22 00:43:50.293356 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:50.305282 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:50.344713 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_307_307_0 (state Temporary) 2024.10.22 00:43:50.402293 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_307_307_0 to 202410_320_320_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:50.571229 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17166 2024.10.22 00:43:50.669385 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 244 ms. 2024.10.22 00:43:51.316395 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_36_36_0 (state Temporary) 2024.10.22 00:43:51.526458 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 00:43:51.534804 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 00:43:51.564699 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 00:43:51.580035 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 00:43:51.581208 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 6.79 MiB, will set to 1.64 GiB (RSS), difference: -2.54 MiB 2024.10.22 00:43:51.586769 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 00:43:51.601404 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 00:43:51.615796 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 00:43:51.661178 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/iij/hcwetkcbspnnvrvhttntongbnirtx (LogSeriesLimiter: on interval from 2024-10-22 00:43:36 to 2024-10-22 00:43:51 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:43:51.830413 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 30 entries to flush up to offset 11510 2024.10.22 00:43:52.162886 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:52.165310 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:52.190094 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_178_178_0 (state Temporary) 2024.10.22 00:43:52.249176 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_178_178_0 to 202410_184_184_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:52.287843 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11510 2024.10.22 00:43:52.964069 [ 1251 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:43:53.075983 [ 1251 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:43:53.488754 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 6.56 MiB, will set to 1.64 GiB (RSS), difference: -8.21 MiB 2024.10.22 00:43:53.492831 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 162 2024.10.22 00:43:53.660960 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:53.664752 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:53.822869 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_144_144_0 (state Temporary) 2024.10.22 00:43:54.050444 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_144_144_0 to 202410_154_154_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:54.102449 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 162 2024.10.22 00:43:54.574295 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 17198 2024.10.22 00:43:54.779025 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:54.782921 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:54.828215 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_308_308_0 (state Temporary) 2024.10.22 00:43:54.889509 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_308_308_0 to 202410_321_321_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:55.055642 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17198 2024.10.22 00:43:55.991236 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_116_23, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0] 2024.10.22 00:43:55.993060 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_116_23, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0] 2024.10.22 00:43:56.125328 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_116_23, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0] 2024.10.22 00:43:56.396748 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1758 entries to flush up to offset 300068 2024.10.22 00:43:56.720941 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 5.16 MiB, will set to 1.64 GiB (RSS), difference: 4.59 MiB 2024.10.22 00:43:56.766595 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 2523 2024.10.22 00:43:56.912700 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:56.916915 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:56.956694 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_180_180_0 (state Temporary) 2024.10.22 00:43:57.168593 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_180_180_0 to 202410_193_193_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:57.187944 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_120_24, 202410_121_121_0, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0] 2024.10.22 00:43:57.190314 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_120_24, 202410_121_121_0, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0] 2024.10.22 00:43:57.261807 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2523 2024.10.22 00:43:57.292725 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:43:57.329017 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_120_24, 202410_121_121_0, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0] 2024.10.22 00:43:57.735691 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:57.745107 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:58.068280 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_172_172_0 (state Temporary) 2024.10.22 00:43:58.154529 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_172_172_0 to 202410_179_179_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:58.180603 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 300068 2024.10.22 00:43:59.065323 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 17224 2024.10.22 00:43:59.281321 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:59.283625 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:59.317595 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_309_309_0 (state Temporary) 2024.10.22 00:43:59.460161 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_309_309_0 to 202410_322_322_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:59.568592 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:43:59.652480 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17224 2024.10.22 00:43:59.664807 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:43:59.720628 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_36_36_0 to 202410_39_39_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:43:59.767627 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_216_43, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0, 202410_220_220_0, 202410_221_221_0] 2024.10.22 00:43:59.768774 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_216_43, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0, 202410_220_220_0, 202410_221_221_0] 2024.10.22 00:43:59.791635 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 29 entries to flush up to offset 11539 2024.10.22 00:43:59.791947 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_216_43, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0, 202410_220_220_0, 202410_221_221_0] 2024.10.22 00:43:59.821358 [ 1159 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_34_7 to 202410_39_39_0 2024.10.22 00:43:59.837348 [ 1159 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:43:59.844968 [ 1159 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:43:59.870781 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_34_7 to 202410_39_39_0 into Compact with storage Full 2024.10.22 00:44:00.020100 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:00.022922 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:44:00.066411 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_179_179_0 (state Temporary) 2024.10.22 00:44:00.095550 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_179_179_0 to 202410_185_185_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:00.177816 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11539 2024.10.22 00:44:00.728016 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 2.75 MiB, will set to 1.63 GiB (RSS), difference: 4.16 MiB 2024.10.22 00:44:01.605486 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 163 2024.10.22 00:44:01.968527 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:01.973735 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:44:02.053782 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_145_145_0 (state Temporary) 2024.10.22 00:44:02.139222 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_145_145_0 to 202410_155_155_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:02.193177 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 163 2024.10.22 00:44:02.690109 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1428 2024.10.22 00:44:03.655676 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 17254 2024.10.22 00:44:03.752332 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:03.759869 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:44:03.892119 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_310_310_0 (state Temporary) 2024.10.22 00:44:03.964000 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:04.031237 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_310_310_0 to 202410_323_323_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:04.068471 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_318_64 to 202410_319_319_0 2024.10.22 00:44:04.078512 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:04.082647 [ 1196 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:44:04.090693 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17254 2024.10.22 00:44:04.125529 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_319_65} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_318_64 to 202410_319_319_0 into Compact with storage Full 2024.10.22 00:44:04.166991 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_319_65} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:04.173931 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_319_65} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_318_64, total 17883 rows starting from the beginning of the part 2024.10.22 00:44:04.316357 [ 1200 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 00:44:04.357480 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_319_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_319_319_0, total 61 rows starting from the beginning of the part 2024.10.22 00:44:04.595098 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_34_7, total 1292 rows starting from the beginning of the part 2024.10.22 00:44:04.763953 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 2530 2024.10.22 00:44:04.868769 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 5.11 MiB, will set to 1.63 GiB (RSS), difference: -10.34 MiB 2024.10.22 00:44:04.891192 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:04.895477 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.58 GiB. 2024.10.22 00:44:05.032616 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_181_181_0 (state Temporary) 2024.10.22 00:44:05.121721 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_181_181_0 to 202410_194_194_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:05.212080 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 301604 2024.10.22 00:44:05.294420 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2530 2024.10.22 00:44:05.719746 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_319_65} MergeTask::MergeProjectionsStage: Merge sorted 17944 rows, containing 13 columns (13 merged, 0 gathered) in 1.602959923 sec., 11194.291100189908 rows/sec., 1.67 MiB/sec. 2024.10.22 00:44:05.789492 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_319_65} MergedBlockOutputStream: filled checksums 202410_1_319_65 (state Temporary) 2024.10.22 00:44:05.891102 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_319_65} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_319_65 to 202410_1_319_65 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:05.895268 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_319_65} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_318_64, 202410_319_319_0] -> 202410_1_319_65 2024.10.22 00:44:05.927519 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.60 MiB. 2024.10.22 00:44:06.289238 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:06.291742 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:06.520924 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_173_173_0 (state Temporary) 2024.10.22 00:44:06.560439 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fzt/nyzlhfndbahircpdmxtaihhndodtj. 2024.10.22 00:44:06.598480 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_173_173_0 to 202410_180_180_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:06.745221 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_35_35_0, total 38 rows starting from the beginning of the part 2024.10.22 00:44:06.845004 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 301604 2024.10.22 00:44:07.578564 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.88 GiB, free memory in arenas 6.16 MiB, will set to 1.63 GiB (RSS), difference: 81.45 MiB 2024.10.22 00:44:07.683819 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 60 entries to flush up to offset 11599 2024.10.22 00:44:07.826178 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.059094208 sec. 2024.10.22 00:44:07.981388 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:07.983110 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:08.038175 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_180_180_0 (state Temporary) 2024.10.22 00:44:08.059071 [ 1738 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/iij/hcwetkcbspnnvrvhttntongbnirtx, size 1173061 2024.10.22 00:44:08.084729 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_180_180_0 to 202410_186_186_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:08.096078 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 17298 2024.10.22 00:44:08.182516 [ 1233 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:44:08.229740 [ 1233 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:44:08.237087 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:08.241195 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:08.311159 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11599 2024.10.22 00:44:08.388102 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 00:44:08.406958 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 00:44:08.484380 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_311_311_0 (state Temporary) 2024.10.22 00:44:08.486879 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_36_36_0, total 40 rows starting from the beginning of the part 2024.10.22 00:44:08.531384 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 00:44:08.660737 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 00:44:08.687969 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 00:44:08.715401 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 00:44:08.734992 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_311_311_0 to 202410_324_324_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:08.936510 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 00:44:08.985433 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17298 2024.10.22 00:44:08.985626 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_319_65 to 202410_324_324_0 2024.10.22 00:44:09.007115 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:09.009280 [ 1235 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:09.027160 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_319_65 to 202410_324_324_0 into Compact with storage Full 2024.10.22 00:44:09.112256 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hzs/bodvccjbatlljdbphsquxvmuwxnss (LogSeriesLimiter: on interval from 2024-10-22 00:43:51 to 2024-10-22 00:44:09 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:44:09.145709 [ 1276 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 00:44:09.168675 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:09.172673 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_319_65, total 17944 rows starting from the beginning of the part 2024.10.22 00:44:09.186284 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_320_320_0, total 53 rows starting from the beginning of the part 2024.10.22 00:44:09.209656 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_321_321_0, total 32 rows starting from the beginning of the part 2024.10.22 00:44:09.261534 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_322_322_0, total 26 rows starting from the beginning of the part 2024.10.22 00:44:09.570249 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:44:09.578267 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:44:09.694906 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 164 2024.10.22 00:44:09.818999 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_323_323_0, total 30 rows starting from the beginning of the part 2024.10.22 00:44:09.862988 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_324_324_0, total 44 rows starting from the beginning of the part 2024.10.22 00:44:09.875539 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.88 GiB, free memory in arenas 5.50 MiB, will set to 1.62 GiB (RSS), difference: 110.61 MiB 2024.10.22 00:44:09.996932 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:10.007180 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:10.097290 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_37_37_0, total 42 rows starting from the beginning of the part 2024.10.22 00:44:10.214070 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 1469 2024.10.22 00:44:10.267411 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_146_146_0 (state Temporary) 2024.10.22 00:44:10.316589 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_146_146_0 to 202410_156_156_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:10.364741 [ 1183 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_151_31 to 202410_156_156_0 2024.10.22 00:44:10.372332 [ 1183 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:10.374753 [ 1183 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:10.398570 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_151_31 to 202410_156_156_0 into Compact with storage Full 2024.10.22 00:44:10.514202 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 164 2024.10.22 00:44:10.970053 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:10.990760 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_151_31, total 195 rows starting from the beginning of the part 2024.10.22 00:44:11.145052 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 3b99aa022bb83f1bdc2066fb3ba8f81f:0) 2024.10.22 00:44:11.150550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 959459820ef9368f3bf7a8d3471208de, offset: 0, size: 2324838 2024.10.22 00:44:11.158189 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 959459820ef9368f3bf7a8d3471208de for removal 2024.10.22 00:44:11.162193 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f31346990006026a1352b741c0e6555b, offset: 0, size: 257 2024.10.22 00:44:11.165479 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f31346990006026a1352b741c0e6555b for removal 2024.10.22 00:44:11.250577 [ 1476 ] {} CacheMetadata: Key 959459820ef9368f3bf7a8d3471208de is removed from metadata 2024.10.22 00:44:11.259809 [ 1476 ] {} CacheMetadata: Key f31346990006026a1352b741c0e6555b is removed from metadata 2024.10.22 00:44:11.285346 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_152_152_0, total 1 rows starting from the beginning of the part 2024.10.22 00:44:11.562884 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_153_153_0, total 1 rows starting from the beginning of the part 2024.10.22 00:44:11.707707 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_154_154_0, total 1 rows starting from the beginning of the part 2024.10.22 00:44:11.771756 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} MergeTask::MergeProjectionsStage: Merge sorted 18129 rows, containing 13 columns (13 merged, 0 gathered) in 2.751716548 sec., 6588.251254721894 rows/sec., 1004.22 KiB/sec. 2024.10.22 00:44:11.939207 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} MergedBlockOutputStream: filled checksums 202410_1_324_66 (state Temporary) 2024.10.22 00:44:12.072983 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_155_155_0, total 1 rows starting from the beginning of the part 2024.10.22 00:44:12.119878 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_324_66 to 202410_1_324_66 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:12.137386 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_324_66} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_319_65, 202410_324_324_0] -> 202410_1_324_66 2024.10.22 00:44:12.170187 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 08270a49df28f2baf414637586dcf7b1:0) 2024.10.22 00:44:12.172396 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c54d775970b843825b3e3ec8a4c1abec, offset: 0, size: 856761 2024.10.22 00:44:12.175180 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_38_38_0, total 37 rows starting from the beginning of the part 2024.10.22 00:44:12.176115 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c54d775970b843825b3e3ec8a4c1abec for removal 2024.10.22 00:44:12.176837 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.73 MiB. 2024.10.22 00:44:12.179391 [ 1476 ] {} CacheMetadata: Key c54d775970b843825b3e3ec8a4c1abec is removed from metadata 2024.10.22 00:44:12.180184 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fa8454b6d2b31509708e8f50fa790502, offset: 0, size: 243 2024.10.22 00:44:12.184394 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fa8454b6d2b31509708e8f50fa790502 for removal 2024.10.22 00:44:12.220399 [ 1476 ] {} CacheMetadata: Key fa8454b6d2b31509708e8f50fa790502 is removed from metadata 2024.10.22 00:44:12.397621 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_156_156_0, total 1 rows starting from the beginning of the part 2024.10.22 00:44:12.517826 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 4.00 MiB, will set to 1.62 GiB (RSS), difference: 1.55 MiB 2024.10.22 00:44:12.799874 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 2545 2024.10.22 00:44:13.007241 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 17372 2024.10.22 00:44:13.130492 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:13.138479 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:13.260995 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_182_182_0 (state Temporary) 2024.10.22 00:44:13.322441 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_182_182_0 to 202410_195_195_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:13.392258 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:13.399818 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:13.511784 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2545 2024.10.22 00:44:13.596633 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_312_312_0 (state Temporary) 2024.10.22 00:44:13.664613 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_312_312_0 to 202410_325_325_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:13.852223 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 303140 2024.10.22 00:44:13.917154 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17372 2024.10.22 00:44:13.978297 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 155360 bytes for 4dffcbe561ed62a777b6753005693e07:0) 2024.10.22 00:44:13.979953 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9834235100db4d3c3ae017fec27aafbb, offset: 0, size: 191427 2024.10.22 00:44:13.986302 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9834235100db4d3c3ae017fec27aafbb for removal 2024.10.22 00:44:13.989616 [ 1476 ] {} CacheMetadata: Key 9834235100db4d3c3ae017fec27aafbb is removed from metadata 2024.10.22 00:44:13.990927 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 07374f34e279053eefe1a5cc9e952e5a, offset: 0, size: 168 2024.10.22 00:44:13.997142 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 07374f34e279053eefe1a5cc9e952e5a for removal 2024.10.22 00:44:14.088686 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} MergeTask::MergeProjectionsStage: Merge sorted 200 rows, containing 32 columns (32 merged, 0 gathered) in 3.697419258 sec., 54.09178295571046 rows/sec., 5.84 KiB/sec. 2024.10.22 00:44:14.118480 [ 1476 ] {} CacheMetadata: Key 07374f34e279053eefe1a5cc9e952e5a is removed from metadata 2024.10.22 00:44:14.228343 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} MergedBlockOutputStream: filled checksums 202410_1_156_32 (state Temporary) 2024.10.22 00:44:14.439673 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_39_39_0, total 41 rows starting from the beginning of the part 2024.10.22 00:44:14.483198 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 2.98 MiB, will set to 1.62 GiB (RSS), difference: -6.66 MiB 2024.10.22 00:44:14.669669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 335695 bytes for 9e74aa5cc1028f032d5a6265b5ef6b18:0) 2024.10.22 00:44:14.672685 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3b99aa022bb83f1bdc2066fb3ba8f81f, offset: 0, size: 2237846 2024.10.22 00:44:14.677305 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3b99aa022bb83f1bdc2066fb3ba8f81f for removal 2024.10.22 00:44:14.679667 [ 1476 ] {} CacheMetadata: Key 3b99aa022bb83f1bdc2066fb3ba8f81f is removed from metadata 2024.10.22 00:44:14.680004 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2978772d78eaef00fdf9b9980b58688e, offset: 0, size: 208 2024.10.22 00:44:14.681609 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_156_32 to 202410_1_156_32 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:14.682513 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2978772d78eaef00fdf9b9980b58688e for removal 2024.10.22 00:44:14.683665 [ 1476 ] {} CacheMetadata: Key 2978772d78eaef00fdf9b9980b58688e is removed from metadata 2024.10.22 00:44:14.683925 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_156_32} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_151_31, 202410_156_156_0] -> 202410_1_156_32 2024.10.22 00:44:14.726589 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:44:15.244214 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:15.247929 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:15.475796 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_174_174_0 (state Temporary) 2024.10.22 00:44:15.688188 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_174_174_0 to 202410_181_181_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:15.724641 [ 1257 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_176_35 to 202410_181_181_0 2024.10.22 00:44:15.745675 [ 1257 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:15.748829 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:15.768773 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_176_35 to 202410_181_181_0 into Compact with storage Full 2024.10.22 00:44:15.785419 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:15.791902 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} MergeTreeSequentialSource: Reading 39 marks from part 202410_1_176_35, total 308278 rows starting from the beginning of the part 2024.10.22 00:44:15.809592 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_177_177_0, total 1904 rows starting from the beginning of the part 2024.10.22 00:44:15.826255 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_178_178_0, total 1718 rows starting from the beginning of the part 2024.10.22 00:44:15.830477 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 303140 2024.10.22 00:44:15.831847 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 82 entries to flush up to offset 11681 2024.10.22 00:44:15.849757 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_179_179_0, total 1758 rows starting from the beginning of the part 2024.10.22 00:44:15.868619 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_180_180_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:44:15.879834 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_181_181_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:44:15.991296 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:15.996144 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:16.054026 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_181_181_0 (state Temporary) 2024.10.22 00:44:16.088037 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_181_181_0 to 202410_187_187_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:16.256845 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11681 2024.10.22 00:44:16.264698 [ 1259 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_182_37 to 202410_187_187_0 2024.10.22 00:44:16.273719 [ 1259 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:16.276313 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:16.285728 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_182_37 to 202410_187_187_0 into Compact with storage Full 2024.10.22 00:44:16.338827 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:16.346551 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_182_37, total 11469 rows starting from the beginning of the part 2024.10.22 00:44:16.368352 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_183_183_0, total 115 rows starting from the beginning of the part 2024.10.22 00:44:16.389064 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_184_184_0, total 30 rows starting from the beginning of the part 2024.10.22 00:44:16.428010 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_185_185_0, total 29 rows starting from the beginning of the part 2024.10.22 00:44:16.461023 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_186_186_0, total 60 rows starting from the beginning of the part 2024.10.22 00:44:16.495802 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_187_187_0, total 82 rows starting from the beginning of the part 2024.10.22 00:44:16.687830 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 2.37 MiB, will set to 1.62 GiB (RSS), difference: -6.32 MiB 2024.10.22 00:44:16.735614 [ 1265 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 00:44:16.791066 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_103_21, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0, 202410_107_107_0, 202410_108_108_0] 2024.10.22 00:44:16.794840 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_103_21, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0, 202410_107_107_0, 202410_108_108_0] 2024.10.22 00:44:16.956733 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_103_21, 202410_104_104_0, 202410_105_105_0, 202410_106_106_0, 202410_107_107_0, 202410_108_108_0] 2024.10.22 00:44:17.528906 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_181_36/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:44:17.704353 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} MergeTask::MergeProjectionsStage: Merge sorted 11785 rows, containing 13 columns (13 merged, 0 gathered) in 1.423888616 sec., 8276.630536668326 rows/sec., 3.46 MiB/sec. 2024.10.22 00:44:17.780457 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} MergedBlockOutputStream: filled checksums 202410_1_187_38 (state Temporary) 2024.10.22 00:44:17.932132 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 17446 2024.10.22 00:44:18.085082 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:18.086809 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:18.134100 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_187_38 to 202410_1_187_38 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:18.239305 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_187_38} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_182_37, 202410_187_187_0] -> 202410_1_187_38 2024.10.22 00:44:18.267882 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_313_313_0 (state Temporary) 2024.10.22 00:44:18.280976 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.34 MiB. 2024.10.22 00:44:18.514674 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_313_313_0 to 202410_326_326_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:18.592660 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17446 2024.10.22 00:44:18.617627 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 5.39 MiB, will set to 1.63 GiB (RSS), difference: 5.31 MiB 2024.10.22 00:44:18.804602 [ 1270 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 221 ms. 2024.10.22 00:44:18.984724 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_131_26, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0] 2024.10.22 00:44:18.986007 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_131_26, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0] 2024.10.22 00:44:19.027705 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_131_26, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0] 2024.10.22 00:44:19.568546 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:44:19.571636 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:44:21.016321 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 2564 2024.10.22 00:44:21.113813 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 645dda777abea8b309e7ce434451485a:0) 2024.10.22 00:44:21.123784 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 08270a49df28f2baf414637586dcf7b1, offset: 0, size: 1077091 2024.10.22 00:44:21.231330 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 08270a49df28f2baf414637586dcf7b1 for removal 2024.10.22 00:44:21.233147 [ 1476 ] {} CacheMetadata: Key 08270a49df28f2baf414637586dcf7b1 is removed from metadata 2024.10.22 00:44:21.233682 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:21.236768 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:21.233533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ee3ab5516cbb781e0a1d5bfd59e5cc84, offset: 0, size: 164 2024.10.22 00:44:21.345289 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ee3ab5516cbb781e0a1d5bfd59e5cc84 for removal 2024.10.22 00:44:21.351478 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 245c8c22927aa7d06e3205a14bc533d6, offset: 0, size: 160 2024.10.22 00:44:21.357322 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 245c8c22927aa7d06e3205a14bc533d6 for removal 2024.10.22 00:44:21.361449 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4f0f71295c23c6ea89da47bd682c08fd, offset: 0, size: 2509 2024.10.22 00:44:21.365301 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4f0f71295c23c6ea89da47bd682c08fd for removal 2024.10.22 00:44:21.371037 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_183_183_0 (state Temporary) 2024.10.22 00:44:21.374114 [ 1476 ] {} CacheMetadata: Key 4f0f71295c23c6ea89da47bd682c08fd is removed from metadata 2024.10.22 00:44:21.370807 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f03ecd851142db9a7733779e890b255e, offset: 0, size: 41211 2024.10.22 00:44:21.384946 [ 1476 ] {} CacheMetadata: Key 245c8c22927aa7d06e3205a14bc533d6 is removed from metadata 2024.10.22 00:44:21.387566 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f03ecd851142db9a7733779e890b255e for removal 2024.10.22 00:44:21.392292 [ 1476 ] {} CacheMetadata: Key f03ecd851142db9a7733779e890b255e is removed from metadata 2024.10.22 00:44:21.405475 [ 1476 ] {} CacheMetadata: Key ee3ab5516cbb781e0a1d5bfd59e5cc84 is removed from metadata 2024.10.22 00:44:21.389710 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e01b4fff2b351577c6d06dcdc6c40783, offset: 0, size: 154 2024.10.22 00:44:21.495179 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e01b4fff2b351577c6d06dcdc6c40783 for removal 2024.10.22 00:44:21.498313 [ 1476 ] {} CacheMetadata: Key e01b4fff2b351577c6d06dcdc6c40783 is removed from metadata 2024.10.22 00:44:21.525503 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_183_183_0 to 202410_196_196_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:21.574513 [ 1155 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_191_38 to 202410_196_196_0 2024.10.22 00:44:21.574799 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2564 2024.10.22 00:44:21.583546 [ 1155 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:21.585412 [ 1155 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:21.616199 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_191_38 to 202410_196_196_0 into Compact with storage Full 2024.10.22 00:44:21.775759 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:21.784373 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_191_38, total 2597 rows starting from the beginning of the part 2024.10.22 00:44:21.808589 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_192_192_0, total 26 rows starting from the beginning of the part 2024.10.22 00:44:21.835569 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_193_193_0, total 6 rows starting from the beginning of the part 2024.10.22 00:44:21.864069 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_194_194_0, total 7 rows starting from the beginning of the part 2024.10.22 00:44:21.878691 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_195_195_0, total 15 rows starting from the beginning of the part 2024.10.22 00:44:21.896804 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_196_196_0, total 19 rows starting from the beginning of the part 2024.10.22 00:44:22.558430 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 10.63 MiB, will set to 1.63 GiB (RSS), difference: -2.03 MiB 2024.10.22 00:44:22.733693 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 17499 2024.10.22 00:44:22.677147 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_196_39/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:44:22.940096 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1730 entries to flush up to offset 304870 2024.10.22 00:44:23.065384 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_221_44, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0, 202410_225_225_0, 202410_226_226_0] 2024.10.22 00:44:23.068429 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_221_44, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0, 202410_225_225_0, 202410_226_226_0] 2024.10.22 00:44:23.089709 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:23.093266 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:23.098881 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_221_44, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0, 202410_225_225_0, 202410_226_226_0] 2024.10.22 00:44:23.115826 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} MergeTask::MergeProjectionsStage: Merge sorted 2670 rows, containing 12 columns (12 merged, 0 gathered) in 1.506975412 sec., 1771.7608255177024 rows/sec., 275.44 KiB/sec. 2024.10.22 00:44:23.236086 [ 1172 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:44:23.256997 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_314_314_0 (state Temporary) 2024.10.22 00:44:23.271089 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} MergedBlockOutputStream: filled checksums 202410_1_196_39 (state Temporary) 2024.10.22 00:44:23.286477 [ 1172 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:44:23.286357 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 272820 bytes for ba5c3f71d1b3131e0b15830a18bde471:0) 2024.10.22 00:44:23.290782 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0dfd3888b968688ab3a72b6ea16e7e42, offset: 0, size: 241200 2024.10.22 00:44:23.294948 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0dfd3888b968688ab3a72b6ea16e7e42 for removal 2024.10.22 00:44:23.301652 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_314_314_0 to 202410_327_327_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:23.305120 [ 1476 ] {} CacheMetadata: Key 0dfd3888b968688ab3a72b6ea16e7e42 is removed from metadata 2024.10.22 00:44:23.308946 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8de89df45032a3ae3a3dbf41c5fca839, offset: 0, size: 25399 2024.10.22 00:44:23.316227 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8de89df45032a3ae3a3dbf41c5fca839 for removal 2024.10.22 00:44:23.319753 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fc4b9079b211588d8cf63860db12d7ee, offset: 0, size: 35001 2024.10.22 00:44:23.324741 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fc4b9079b211588d8cf63860db12d7ee for removal 2024.10.22 00:44:23.329813 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1c9bb89715ecb454d47f4b9c0aaff556, offset: 0, size: 142 2024.10.22 00:44:23.343052 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1c9bb89715ecb454d47f4b9c0aaff556 for removal 2024.10.22 00:44:23.350777 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 361689d3f471cc76a24ab7f73b299316, offset: 0, size: 146 2024.10.22 00:44:23.359058 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 361689d3f471cc76a24ab7f73b299316 for removal 2024.10.22 00:44:23.361573 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 48771c002216655671073297cfc00266, offset: 0, size: 164 2024.10.22 00:44:23.365249 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 48771c002216655671073297cfc00266 for removal 2024.10.22 00:44:23.366699 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17499 2024.10.22 00:44:23.415012 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:23.433373 [ 1476 ] {} CacheMetadata: Key 48771c002216655671073297cfc00266 is removed from metadata 2024.10.22 00:44:23.442610 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:23.447913 [ 1476 ] {} CacheMetadata: Key 361689d3f471cc76a24ab7f73b299316 is removed from metadata 2024.10.22 00:44:23.458561 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 10.59 MiB, will set to 1.63 GiB (RSS), difference: -1.37 MiB 2024.10.22 00:44:23.462398 [ 1476 ] {} CacheMetadata: Key 1c9bb89715ecb454d47f4b9c0aaff556 is removed from metadata 2024.10.22 00:44:23.467839 [ 1476 ] {} CacheMetadata: Key fc4b9079b211588d8cf63860db12d7ee is removed from metadata 2024.10.22 00:44:23.474529 [ 1476 ] {} CacheMetadata: Key 8de89df45032a3ae3a3dbf41c5fca839 is removed from metadata 2024.10.22 00:44:23.635319 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_175_175_0 (state Temporary) 2024.10.22 00:44:23.718314 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_196_39 to 202410_1_196_39 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:23.722982 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_196_39} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_191_38, 202410_196_196_0] -> 202410_1_196_39 2024.10.22 00:44:23.769137 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 113 entries to flush up to offset 11794 2024.10.22 00:44:23.770654 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:44:23.884722 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_175_175_0 to 202410_182_182_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:23.937420 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 304870 2024.10.22 00:44:23.938694 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 14 file segments (while reserving 967547 bytes for 2ec49605e26879ff7c545bc74f7058f8:0) 2024.10.22 00:44:23.948178 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fde25be03eeb2e822b1734507a78a020, offset: 0, size: 107613 2024.10.22 00:44:23.955101 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fde25be03eeb2e822b1734507a78a020 for removal 2024.10.22 00:44:23.958229 [ 1476 ] {} CacheMetadata: Key fde25be03eeb2e822b1734507a78a020 is removed from metadata 2024.10.22 00:44:23.960312 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b6cac93c2a13e0feb39dda898e01675e, offset: 0, size: 165 2024.10.22 00:44:23.963959 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b6cac93c2a13e0feb39dda898e01675e for removal 2024.10.22 00:44:23.967801 [ 1476 ] {} CacheMetadata: Key b6cac93c2a13e0feb39dda898e01675e is removed from metadata 2024.10.22 00:44:23.967826 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 581fb34f0024144d2f48d2799d4e444c, offset: 0, size: 165 2024.10.22 00:44:23.970301 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 581fb34f0024144d2f48d2799d4e444c for removal 2024.10.22 00:44:23.972669 [ 1476 ] {} CacheMetadata: Key 581fb34f0024144d2f48d2799d4e444c is removed from metadata 2024.10.22 00:44:23.972690 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9de11bf2d1e07ad84c3fb6db9bb431e9, offset: 0, size: 178 2024.10.22 00:44:23.977481 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9de11bf2d1e07ad84c3fb6db9bb431e9 for removal 2024.10.22 00:44:23.979671 [ 1476 ] {} CacheMetadata: Key 9de11bf2d1e07ad84c3fb6db9bb431e9 is removed from metadata 2024.10.22 00:44:23.980694 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 40892fcf4916ec79298ad17438cb82f8, offset: 0, size: 155 2024.10.22 00:44:23.982912 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 40892fcf4916ec79298ad17438cb82f8 for removal 2024.10.22 00:44:23.985141 [ 1476 ] {} CacheMetadata: Key 40892fcf4916ec79298ad17438cb82f8 is removed from metadata 2024.10.22 00:44:23.985199 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8818408361eabb4e933bc79528a56b33, offset: 0, size: 154 2024.10.22 00:44:24.081716 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:24.090581 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:24.154709 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_182_182_0 (state Temporary) 2024.10.22 00:44:24.191372 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8818408361eabb4e933bc79528a56b33 for removal 2024.10.22 00:44:24.194394 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 15b02f36d46fc61eb317a4ed83b556f5, offset: 0, size: 110966 2024.10.22 00:44:24.195611 [ 1476 ] {} CacheMetadata: Key 8818408361eabb4e933bc79528a56b33 is removed from metadata 2024.10.22 00:44:24.201186 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 15b02f36d46fc61eb317a4ed83b556f5 for removal 2024.10.22 00:44:24.204198 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_182_182_0 to 202410_188_188_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:24.204791 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 04260347b8ab0842d5770f641805e38b, offset: 0, size: 71505 2024.10.22 00:44:24.218228 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 04260347b8ab0842d5770f641805e38b for removal 2024.10.22 00:44:24.220884 [ 1476 ] {} CacheMetadata: Key 04260347b8ab0842d5770f641805e38b is removed from metadata 2024.10.22 00:44:24.221017 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 512efe776c9b0e153fa24ac1715cdbbe, offset: 0, size: 62839 2024.10.22 00:44:24.227578 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 512efe776c9b0e153fa24ac1715cdbbe for removal 2024.10.22 00:44:24.231164 [ 1476 ] {} CacheMetadata: Key 15b02f36d46fc61eb317a4ed83b556f5 is removed from metadata 2024.10.22 00:44:24.233551 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dddc7064ea86ea9a72e82dc25cde8cef, offset: 0, size: 153 2024.10.22 00:44:24.246825 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dddc7064ea86ea9a72e82dc25cde8cef for removal 2024.10.22 00:44:24.250963 [ 1476 ] {} CacheMetadata: Key 512efe776c9b0e153fa24ac1715cdbbe is removed from metadata 2024.10.22 00:44:24.259843 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9e74aa5cc1028f032d5a6265b5ef6b18, offset: 0, size: 335695 2024.10.22 00:44:24.264257 [ 1476 ] {} CacheMetadata: Key dddc7064ea86ea9a72e82dc25cde8cef is removed from metadata 2024.10.22 00:44:24.266974 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9e74aa5cc1028f032d5a6265b5ef6b18 for removal 2024.10.22 00:44:24.271589 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11794 2024.10.22 00:44:24.280327 [ 1476 ] {} CacheMetadata: Key 9e74aa5cc1028f032d5a6265b5ef6b18 is removed from metadata 2024.10.22 00:44:24.282076 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a27649aab94ff098714d815a16ff3535, offset: 0, size: 146 2024.10.22 00:44:24.285031 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a27649aab94ff098714d815a16ff3535 for removal 2024.10.22 00:44:24.290114 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4dffcbe561ed62a777b6753005693e07, offset: 0, size: 155360 2024.10.22 00:44:24.293934 [ 1476 ] {} CacheMetadata: Key a27649aab94ff098714d815a16ff3535 is removed from metadata 2024.10.22 00:44:24.296931 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4dffcbe561ed62a777b6753005693e07 for removal 2024.10.22 00:44:24.304631 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 16085bfb4a1a9c19f454130df015bff1, offset: 0, size: 5460 2024.10.22 00:44:24.313155 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 16085bfb4a1a9c19f454130df015bff1 for removal 2024.10.22 00:44:24.317302 [ 1476 ] {} CacheMetadata: Key 4dffcbe561ed62a777b6753005693e07 is removed from metadata 2024.10.22 00:44:24.332148 [ 1476 ] {} CacheMetadata: Key 16085bfb4a1a9c19f454130df015bff1 is removed from metadata 2024.10.22 00:44:24.630138 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 20 file segments (while reserving 598170 bytes for d4ec29e3bc6fa874c24c81651e5a3d39:0) 2024.10.22 00:44:24.746555 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 65ae824613e5ac668db42fc84f07c8a0, offset: 0, size: 128697 2024.10.22 00:44:24.751163 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 65ae824613e5ac668db42fc84f07c8a0 for removal 2024.10.22 00:44:24.754597 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6760a1c2a155298fc5c467778387fbe4, offset: 0, size: 155 2024.10.22 00:44:24.755091 [ 1476 ] {} CacheMetadata: Key 65ae824613e5ac668db42fc84f07c8a0 is removed from metadata 2024.10.22 00:44:24.758670 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6760a1c2a155298fc5c467778387fbe4 for removal 2024.10.22 00:44:24.763843 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 971c26b46d4a888f8e560944b3c91f39, offset: 0, size: 26576 2024.10.22 00:44:24.768039 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 971c26b46d4a888f8e560944b3c91f39 for removal 2024.10.22 00:44:24.770890 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1b4829efac47ee9696e1b72f5727cece, offset: 0, size: 154 2024.10.22 00:44:24.772197 [ 1476 ] {} CacheMetadata: Key 971c26b46d4a888f8e560944b3c91f39 is removed from metadata 2024.10.22 00:44:24.772919 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1b4829efac47ee9696e1b72f5727cece for removal 2024.10.22 00:44:24.775676 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 371b62974644473ca11eeb1919fc59be, offset: 0, size: 27023 2024.10.22 00:44:24.777275 [ 1476 ] {} CacheMetadata: Key 1b4829efac47ee9696e1b72f5727cece is removed from metadata 2024.10.22 00:44:24.784781 [ 1476 ] {} CacheMetadata: Key 6760a1c2a155298fc5c467778387fbe4 is removed from metadata 2024.10.22 00:44:24.878735 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 371b62974644473ca11eeb1919fc59be for removal 2024.10.22 00:44:24.881257 [ 1476 ] {} CacheMetadata: Key 371b62974644473ca11eeb1919fc59be is removed from metadata 2024.10.22 00:44:24.882754 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ee8aa3dbe3c808f1e8eba12d424486ff, offset: 0, size: 2373 2024.10.22 00:44:24.885010 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ee8aa3dbe3c808f1e8eba12d424486ff for removal 2024.10.22 00:44:24.887295 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 262d76f26f9845f596f99d7a943db26e, offset: 0, size: 154 2024.10.22 00:44:24.890268 [ 1476 ] {} CacheMetadata: Key ee8aa3dbe3c808f1e8eba12d424486ff is removed from metadata 2024.10.22 00:44:24.891411 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 262d76f26f9845f596f99d7a943db26e for removal 2024.10.22 00:44:24.893465 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e66725e6dfa090dc53c202908dcb0480, offset: 0, size: 105682 2024.10.22 00:44:24.898216 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e66725e6dfa090dc53c202908dcb0480 for removal 2024.10.22 00:44:24.903501 [ 1476 ] {} CacheMetadata: Key e66725e6dfa090dc53c202908dcb0480 is removed from metadata 2024.10.22 00:44:24.911878 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fa0582d9a35a3d54ac03b38982106315, offset: 0, size: 154 2024.10.22 00:44:24.915794 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fa0582d9a35a3d54ac03b38982106315 for removal 2024.10.22 00:44:24.919068 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6a15c95e98a2ea56f02ada57cd3d656d, offset: 0, size: 49882 2024.10.22 00:44:24.923658 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6a15c95e98a2ea56f02ada57cd3d656d for removal 2024.10.22 00:44:24.932048 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c41c221a8a1be39844cdee955bc95036, offset: 0, size: 2107 2024.10.22 00:44:24.960697 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c41c221a8a1be39844cdee955bc95036 for removal 2024.10.22 00:44:24.966295 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 96a1f21fe726a30437e43818778c40bd, offset: 0, size: 165 2024.10.22 00:44:24.972037 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 96a1f21fe726a30437e43818778c40bd for removal 2024.10.22 00:44:24.975194 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f4bb596a00c6ecce950c82949bd222fe, offset: 0, size: 30275 2024.10.22 00:44:24.978454 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f4bb596a00c6ecce950c82949bd222fe for removal 2024.10.22 00:44:24.980381 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 733ea1da6d52e1d6b91effb64da35d43, offset: 0, size: 52814 2024.10.22 00:44:24.983678 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 733ea1da6d52e1d6b91effb64da35d43 for removal 2024.10.22 00:44:24.986354 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a4b195ae81c07908de913f72ffb25396, offset: 0, size: 154 2024.10.22 00:44:24.989111 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a4b195ae81c07908de913f72ffb25396 for removal 2024.10.22 00:44:24.992184 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 76788e677f431232f8019da440ad5c99, offset: 0, size: 154 2024.10.22 00:44:24.997418 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 76788e677f431232f8019da440ad5c99 for removal 2024.10.22 00:44:25.011686 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d14110175bfb939b8aa104a85fd05160, offset: 0, size: 183492 2024.10.22 00:44:25.017109 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d14110175bfb939b8aa104a85fd05160 for removal 2024.10.22 00:44:25.017212 [ 1476 ] {} CacheMetadata: Key 76788e677f431232f8019da440ad5c99 is removed from metadata 2024.10.22 00:44:25.021062 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 83d4c57133aa3e0e27e0d38fb19f231b, offset: 0, size: 154 2024.10.22 00:44:25.026398 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 83d4c57133aa3e0e27e0d38fb19f231b for removal 2024.10.22 00:44:25.033627 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2406912f28f5207a80c4b831748f8237, offset: 0, size: 146 2024.10.22 00:44:25.128206 [ 1476 ] {} CacheMetadata: Key a4b195ae81c07908de913f72ffb25396 is removed from metadata 2024.10.22 00:44:25.146614 [ 1476 ] {} CacheMetadata: Key 733ea1da6d52e1d6b91effb64da35d43 is removed from metadata 2024.10.22 00:44:25.153274 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2406912f28f5207a80c4b831748f8237 for removal 2024.10.22 00:44:25.161749 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 79a919c2c473f97928fbd18c41ffac41, offset: 0, size: 154 2024.10.22 00:44:25.163050 [ 1476 ] {} CacheMetadata: Key 2406912f28f5207a80c4b831748f8237 is removed from metadata 2024.10.22 00:44:25.164853 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 79a919c2c473f97928fbd18c41ffac41 for removal 2024.10.22 00:44:25.170590 [ 1476 ] {} CacheMetadata: Key 79a919c2c473f97928fbd18c41ffac41 is removed from metadata 2024.10.22 00:44:25.175763 [ 1476 ] {} CacheMetadata: Key f4bb596a00c6ecce950c82949bd222fe is removed from metadata 2024.10.22 00:44:25.183118 [ 1476 ] {} CacheMetadata: Key 96a1f21fe726a30437e43818778c40bd is removed from metadata 2024.10.22 00:44:25.192860 [ 1476 ] {} CacheMetadata: Key 83d4c57133aa3e0e27e0d38fb19f231b is removed from metadata 2024.10.22 00:44:25.223695 [ 1476 ] {} CacheMetadata: Key c41c221a8a1be39844cdee955bc95036 is removed from metadata 2024.10.22 00:44:25.237244 [ 1476 ] {} CacheMetadata: Key 6a15c95e98a2ea56f02ada57cd3d656d is removed from metadata 2024.10.22 00:44:25.263716 [ 1476 ] {} CacheMetadata: Key d14110175bfb939b8aa104a85fd05160 is removed from metadata 2024.10.22 00:44:25.271763 [ 1734 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hzs/bodvccjbatlljdbphsquxvmuwxnss, size 598170 2024.10.22 00:44:25.272453 [ 1476 ] {} CacheMetadata: Key fa0582d9a35a3d54ac03b38982106315 is removed from metadata 2024.10.22 00:44:25.281261 [ 1476 ] {} CacheMetadata: Key 262d76f26f9845f596f99d7a943db26e is removed from metadata 2024.10.22 00:44:25.340394 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 11.04 MiB, will set to 1.63 GiB (RSS), difference: 2.97 MiB 2024.10.22 00:44:25.517141 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 165 2024.10.22 00:44:25.563519 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 95821 bytes for 204c1d1e7f156899355c31787e642d8f:0) 2024.10.22 00:44:25.567129 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 618ea670947a3caa007a55746bc0e773, offset: 0, size: 92885 2024.10.22 00:44:25.569501 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 618ea670947a3caa007a55746bc0e773 for removal 2024.10.22 00:44:25.572789 [ 1476 ] {} CacheMetadata: Key 618ea670947a3caa007a55746bc0e773 is removed from metadata 2024.10.22 00:44:25.573257 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5c59d1f63fbae64dfef664e422e34513, offset: 0, size: 67236 2024.10.22 00:44:25.577095 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5c59d1f63fbae64dfef664e422e34513 for removal 2024.10.22 00:44:25.580056 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bdba31021bc8f8bf7566772025bf6a12, offset: 0, size: 155 2024.10.22 00:44:25.583293 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bdba31021bc8f8bf7566772025bf6a12 for removal 2024.10.22 00:44:25.585516 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 87149c38b30473525dbd9d8f51ee006d, offset: 0, size: 165 2024.10.22 00:44:25.585710 [ 1476 ] {} CacheMetadata: Key bdba31021bc8f8bf7566772025bf6a12 is removed from metadata 2024.10.22 00:44:25.591143 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 87149c38b30473525dbd9d8f51ee006d for removal 2024.10.22 00:44:25.598322 [ 1476 ] {} CacheMetadata: Key 87149c38b30473525dbd9d8f51ee006d is removed from metadata 2024.10.22 00:44:25.614651 [ 1476 ] {} CacheMetadata: Key 5c59d1f63fbae64dfef664e422e34513 is removed from metadata 2024.10.22 00:44:25.759513 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FlashMajor 2024.10.22 00:44:25.773367 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FlashMajor 2024.10.22 00:44:25.780772 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FlashMajor 2024.10.22 00:44:25.910788 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FlashMajor 2024.10.22 00:44:26.016800 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:26.019789 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:26.020543 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FlashMajor 2024.10.22 00:44:26.040519 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FlashMajor 2024.10.22 00:44:26.067603 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FlashMajor 2024.10.22 00:44:26.119588 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/iak/zkpczqbryvkqmuoanokgzsjmqbure (LogSeriesLimiter: on interval from 2024-10-22 00:44:09 to 2024-10-22 00:44:26 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:44:26.216110 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_147_147_0 (state Temporary) 2024.10.22 00:44:26.273635 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:26.275914 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:26.418486 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_147_147_0 to 202410_157_157_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:26.500639 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 8 file segments (while reserving 1048576 bytes for a24350ddbdee64eed3f0ad81f7abe0e7:0) 2024.10.22 00:44:26.503690 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e6bcf2fbc7dd7b3e229819dba4012d1e, offset: 0, size: 160 2024.10.22 00:44:26.508059 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e6bcf2fbc7dd7b3e229819dba4012d1e for removal 2024.10.22 00:44:26.512686 [ 1476 ] {} CacheMetadata: Key e6bcf2fbc7dd7b3e229819dba4012d1e is removed from metadata 2024.10.22 00:44:26.515331 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 22b8135d731d078e0cbaeb20fa282994, offset: 0, size: 78326 2024.10.22 00:44:26.518574 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 22b8135d731d078e0cbaeb20fa282994 for removal 2024.10.22 00:44:26.526461 [ 1476 ] {} CacheMetadata: Key 22b8135d731d078e0cbaeb20fa282994 is removed from metadata 2024.10.22 00:44:26.521083 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0f4c9b4b4597ed589ed30e03a3af2d41, offset: 0, size: 160923 2024.10.22 00:44:26.534492 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0f4c9b4b4597ed589ed30e03a3af2d41 for removal 2024.10.22 00:44:26.538498 [ 1476 ] {} CacheMetadata: Key 0f4c9b4b4597ed589ed30e03a3af2d41 is removed from metadata 2024.10.22 00:44:26.539221 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ce4e1556c3c0d06e4a0fafb7c66f9497, offset: 0, size: 165 2024.10.22 00:44:26.546349 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ce4e1556c3c0d06e4a0fafb7c66f9497 for removal 2024.10.22 00:44:26.551924 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cc86743c4946424a72a9cadfa3788b8d, offset: 0, size: 166 2024.10.22 00:44:26.558848 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cc86743c4946424a72a9cadfa3788b8d for removal 2024.10.22 00:44:26.562239 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 645dda777abea8b309e7ce434451485a, offset: 0, size: 1106975 2024.10.22 00:44:26.567210 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 645dda777abea8b309e7ce434451485a for removal 2024.10.22 00:44:26.569624 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 86f38cb2819558ba8732d4ae5d6eac68, offset: 0, size: 151385 2024.10.22 00:44:26.573144 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 86f38cb2819558ba8732d4ae5d6eac68 for removal 2024.10.22 00:44:26.575278 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8d064cae456305aecde4da3496b39d22, offset: 0, size: 166 2024.10.22 00:44:26.578342 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8d064cae456305aecde4da3496b39d22 for removal 2024.10.22 00:44:26.619896 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 165 2024.10.22 00:44:26.651773 [ 1476 ] {} CacheMetadata: Key ce4e1556c3c0d06e4a0fafb7c66f9497 is removed from metadata 2024.10.22 00:44:26.661258 [ 1476 ] {} CacheMetadata: Key 8d064cae456305aecde4da3496b39d22 is removed from metadata 2024.10.22 00:44:26.670527 [ 1476 ] {} CacheMetadata: Key 86f38cb2819558ba8732d4ae5d6eac68 is removed from metadata 2024.10.22 00:44:26.675713 [ 1476 ] {} CacheMetadata: Key 645dda777abea8b309e7ce434451485a is removed from metadata 2024.10.22 00:44:26.685497 [ 1476 ] {} CacheMetadata: Key cc86743c4946424a72a9cadfa3788b8d is removed from metadata 2024.10.22 00:44:27.028733 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_39_8/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:44:27.225066 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} MergeTask::MergeProjectionsStage: Merge sorted 316730 rows, containing 4 columns (4 merged, 0 gathered) in 11.461499868 sec., 27634.254124479477 rows/sec., 405.52 KiB/sec. 2024.10.22 00:44:27.325207 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} MergedBlockOutputStream: filled checksums 202410_1_181_36 (state Temporary) 2024.10.22 00:44:27.369561 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_181_36 to 202410_1_181_36 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:27.372141 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 212 entries to flush up to offset 17711 2024.10.22 00:44:27.376156 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_181_36} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_176_35, 202410_181_181_0] -> 202410_1_181_36 2024.10.22 00:44:27.409771 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:44:27.586813 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 11.53 MiB, will set to 1.64 GiB (RSS), difference: 1.95 MiB 2024.10.22 00:44:27.740396 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:27.745365 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:27.808675 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_315_315_0 (state Temporary) 2024.10.22 00:44:28.276893 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_315_315_0 to 202410_328_328_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:28.326393 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17711 2024.10.22 00:44:28.421127 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 240524 bytes for 575371f222fb3037f259b69f8027d2be:0) 2024.10.22 00:44:28.424714 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d4f682d63663ce5a910810b5e5334709, offset: 0, size: 40496 2024.10.22 00:44:28.430309 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d4f682d63663ce5a910810b5e5334709 for removal 2024.10.22 00:44:28.434951 [ 1476 ] {} CacheMetadata: Key d4f682d63663ce5a910810b5e5334709 is removed from metadata 2024.10.22 00:44:28.435226 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4b4c91570a3c8f13a5b1a435e1c2ce5c, offset: 0, size: 154 2024.10.22 00:44:28.440584 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4b4c91570a3c8f13a5b1a435e1c2ce5c for removal 2024.10.22 00:44:28.445671 [ 1476 ] {} CacheMetadata: Key 4b4c91570a3c8f13a5b1a435e1c2ce5c is removed from metadata 2024.10.22 00:44:28.448123 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a8186b57c5aa9ac14e2006c99e60755c, offset: 0, size: 46530 2024.10.22 00:44:28.454225 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a8186b57c5aa9ac14e2006c99e60755c for removal 2024.10.22 00:44:28.457535 [ 1476 ] {} CacheMetadata: Key a8186b57c5aa9ac14e2006c99e60755c is removed from metadata 2024.10.22 00:44:28.457696 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 72fd3b0462109e8675a96e433ea12ee3, offset: 0, size: 154 2024.10.22 00:44:28.460642 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 72fd3b0462109e8675a96e433ea12ee3 for removal 2024.10.22 00:44:28.464590 [ 1476 ] {} CacheMetadata: Key 72fd3b0462109e8675a96e433ea12ee3 is removed from metadata 2024.10.22 00:44:28.874764 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 220577 bytes for e1209af38c7787f63c866d66addf0e7f:0) 2024.10.22 00:44:28.877339 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e0296975614a6d460c283c1d3fe65bb4, offset: 0, size: 125145 2024.10.22 00:44:28.881136 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e0296975614a6d460c283c1d3fe65bb4 for removal 2024.10.22 00:44:28.882960 [ 1476 ] {} CacheMetadata: Key e0296975614a6d460c283c1d3fe65bb4 is removed from metadata 2024.10.22 00:44:28.883773 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 506fd29a4f60ba0f905325ae99875058, offset: 0, size: 155 2024.10.22 00:44:28.886744 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 506fd29a4f60ba0f905325ae99875058 for removal 2024.10.22 00:44:28.890521 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1d0ee3ca5aa8c4fe328a910d26dadfdd, offset: 0, size: 144 2024.10.22 00:44:28.892077 [ 1476 ] {} CacheMetadata: Key 506fd29a4f60ba0f905325ae99875058 is removed from metadata 2024.10.22 00:44:28.898189 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1d0ee3ca5aa8c4fe328a910d26dadfdd for removal 2024.10.22 00:44:28.905040 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 446c4be1b749eb53e283d6dec0cdcf88, offset: 0, size: 75060 2024.10.22 00:44:28.907548 [ 1476 ] {} CacheMetadata: Key 1d0ee3ca5aa8c4fe328a910d26dadfdd is removed from metadata 2024.10.22 00:44:28.908731 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 446c4be1b749eb53e283d6dec0cdcf88 for removal 2024.10.22 00:44:28.912128 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 78728fe779403378e68eebbef3ff7662, offset: 0, size: 91312 2024.10.22 00:44:28.916207 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 78728fe779403378e68eebbef3ff7662 for removal 2024.10.22 00:44:28.920653 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8d2392f3d1fdce66a1516c70f7c35305, offset: 0, size: 144 2024.10.22 00:44:28.921168 [ 1476 ] {} CacheMetadata: Key 78728fe779403378e68eebbef3ff7662 is removed from metadata 2024.10.22 00:44:28.924633 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8d2392f3d1fdce66a1516c70f7c35305 for removal 2024.10.22 00:44:29.028601 [ 1476 ] {} CacheMetadata: Key 8d2392f3d1fdce66a1516c70f7c35305 is removed from metadata 2024.10.22 00:44:29.076591 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 2584 2024.10.22 00:44:29.145746 [ 1476 ] {} CacheMetadata: Key 446c4be1b749eb53e283d6dec0cdcf88 is removed from metadata 2024.10.22 00:44:29.257682 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:29.260991 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:29.295999 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_184_184_0 (state Temporary) 2024.10.22 00:44:29.362789 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_184_184_0 to 202410_197_197_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:29.408665 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2584 2024.10.22 00:44:29.568464 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:44:29.571105 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:44:29.681421 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 9.56 MiB, will set to 1.64 GiB (RSS), difference: -4.38 MiB 2024.10.22 00:44:29.756544 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_37_37_0 (state Temporary) 2024.10.22 00:44:30.366249 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 163351 bytes for 841b061292c1dd350f5f10879e02a1e7:0) 2024.10.22 00:44:30.369672 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ba5c3f71d1b3131e0b15830a18bde471, offset: 0, size: 272820 2024.10.22 00:44:30.372409 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ba5c3f71d1b3131e0b15830a18bde471 for removal 2024.10.22 00:44:30.374797 [ 1476 ] {} CacheMetadata: Key ba5c3f71d1b3131e0b15830a18bde471 is removed from metadata 2024.10.22 00:44:30.375437 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 395629a84dd4da5efb707d34aeaaf769, offset: 0, size: 176 2024.10.22 00:44:30.377644 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 395629a84dd4da5efb707d34aeaaf769 for removal 2024.10.22 00:44:30.380102 [ 1476 ] {} CacheMetadata: Key 395629a84dd4da5efb707d34aeaaf769 is removed from metadata 2024.10.22 00:44:31.047395 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1726 entries to flush up to offset 306596 2024.10.22 00:44:31.280446 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 212475 bytes for 71d197ba70cd08f8f84a2384a2516fbf:0) 2024.10.22 00:44:31.282800 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2ec49605e26879ff7c545bc74f7058f8, offset: 0, size: 967547 2024.10.22 00:44:31.285350 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2ec49605e26879ff7c545bc74f7058f8 for removal 2024.10.22 00:44:31.292483 [ 1476 ] {} CacheMetadata: Key 2ec49605e26879ff7c545bc74f7058f8 is removed from metadata 2024.10.22 00:44:31.293338 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2a2a8d8561a58f64dcd0df13c184ef61, offset: 0, size: 159 2024.10.22 00:44:31.299748 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2a2a8d8561a58f64dcd0df13c184ef61 for removal 2024.10.22 00:44:31.405001 [ 1476 ] {} CacheMetadata: Key 2a2a8d8561a58f64dcd0df13c184ef61 is removed from metadata 2024.10.22 00:44:31.435537 [ 1239 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 00:44:31.548032 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 10.57 MiB, will set to 1.64 GiB (RSS), difference: -2.69 MiB 2024.10.22 00:44:31.787458 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 60 entries to flush up to offset 11854 2024.10.22 00:44:31.961675 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:31.965931 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:32.084791 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:32.090232 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_176_176_0 (state Temporary) 2024.10.22 00:44:32.089436 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:32.256810 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_183_183_0 (state Temporary) 2024.10.22 00:44:32.313288 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_183_183_0 to 202410_189_189_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:32.460973 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 17788 2024.10.22 00:44:32.471865 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11854 2024.10.22 00:44:32.582654 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_176_176_0 to 202410_183_183_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:32.676282 [ 1183 ] {} DatabaseCatalog: Execution took 228 ms. 2024.10.22 00:44:32.683836 [ 1244 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 248 ms. 2024.10.22 00:44:32.697234 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:32.702425 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:32.724204 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 306596 2024.10.22 00:44:32.772737 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 523421 bytes for acd3a2b5417c269c03f90030e3b7a9ec:0) 2024.10.22 00:44:32.776260 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fb8bee09699c145403a81059d52d49d2, offset: 0, size: 3754 2024.10.22 00:44:32.778717 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_316_316_0 (state Temporary) 2024.10.22 00:44:32.779377 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fb8bee09699c145403a81059d52d49d2 for removal 2024.10.22 00:44:32.786959 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a24350ddbdee64eed3f0ad81f7abe0e7, offset: 0, size: 1078314 2024.10.22 00:44:32.788168 [ 1476 ] {} CacheMetadata: Key fb8bee09699c145403a81059d52d49d2 is removed from metadata 2024.10.22 00:44:32.791975 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a24350ddbdee64eed3f0ad81f7abe0e7 for removal 2024.10.22 00:44:32.797303 [ 1476 ] {} CacheMetadata: Key a24350ddbdee64eed3f0ad81f7abe0e7 is removed from metadata 2024.10.22 00:44:32.801071 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7c80b32dc5b81a92d2108e1f2c3296d7, offset: 0, size: 154 2024.10.22 00:44:32.809799 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7c80b32dc5b81a92d2108e1f2c3296d7 for removal 2024.10.22 00:44:32.815092 [ 1476 ] {} CacheMetadata: Key 7c80b32dc5b81a92d2108e1f2c3296d7 is removed from metadata 2024.10.22 00:44:32.815736 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cf8e55721569d137d125c77d0852f8f8, offset: 0, size: 160 2024.10.22 00:44:32.835038 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_316_316_0 to 202410_329_329_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:32.868694 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_324_66 to 202410_329_329_0 2024.10.22 00:44:32.883599 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:32.885047 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:32.921468 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cf8e55721569d137d125c77d0852f8f8 for removal 2024.10.22 00:44:32.923267 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_324_66 to 202410_329_329_0 into Compact with storage Full 2024.10.22 00:44:32.925416 [ 1476 ] {} CacheMetadata: Key cf8e55721569d137d125c77d0852f8f8 is removed from metadata 2024.10.22 00:44:32.927223 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 204c1d1e7f156899355c31787e642d8f, offset: 0, size: 95821 2024.10.22 00:44:32.932180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 204c1d1e7f156899355c31787e642d8f for removal 2024.10.22 00:44:32.950936 [ 1476 ] {} CacheMetadata: Key 204c1d1e7f156899355c31787e642d8f is removed from metadata 2024.10.22 00:44:32.951662 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7d39ec43eece287040f8c6c8bd71ded2, offset: 0, size: 155 2024.10.22 00:44:32.957424 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7d39ec43eece287040f8c6c8bd71ded2 for removal 2024.10.22 00:44:32.961085 [ 1476 ] {} CacheMetadata: Key 7d39ec43eece287040f8c6c8bd71ded2 is removed from metadata 2024.10.22 00:44:32.976622 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17788 2024.10.22 00:44:33.074132 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:33.093019 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_324_66, total 18129 rows starting from the beginning of the part 2024.10.22 00:44:33.317950 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_325_325_0, total 74 rows starting from the beginning of the part 2024.10.22 00:44:33.339156 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_326_326_0, total 74 rows starting from the beginning of the part 2024.10.22 00:44:33.562319 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_327_327_0, total 53 rows starting from the beginning of the part 2024.10.22 00:44:33.579289 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_328_328_0, total 212 rows starting from the beginning of the part 2024.10.22 00:44:33.620994 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 7.54 MiB, will set to 1.64 GiB (RSS), difference: -3.19 MiB 2024.10.22 00:44:33.695293 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_329_329_0, total 77 rows starting from the beginning of the part 2024.10.22 00:44:34.122313 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 166 2024.10.22 00:44:34.366283 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 214430 bytes for bf2b02055fa2831910d7d508e71e8637:0) 2024.10.22 00:44:34.369781 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0f472615f0c70ccd66115c347d7d5513, offset: 0, size: 72240 2024.10.22 00:44:34.375336 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0f472615f0c70ccd66115c347d7d5513 for removal 2024.10.22 00:44:34.377306 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7eebb4016444c6f6393d40e5d9484341, offset: 0, size: 155 2024.10.22 00:44:34.377279 [ 1476 ] {} CacheMetadata: Key 0f472615f0c70ccd66115c347d7d5513 is removed from metadata 2024.10.22 00:44:34.380438 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:34.382543 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7eebb4016444c6f6393d40e5d9484341 for removal 2024.10.22 00:44:34.383843 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:34.403008 [ 1476 ] {} CacheMetadata: Key 7eebb4016444c6f6393d40e5d9484341 is removed from metadata 2024.10.22 00:44:34.441432 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_148_148_0 (state Temporary) 2024.10.22 00:44:34.783771 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_148_148_0 to 202410_158_158_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:34.964082 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 10 file segments (while reserving 484664 bytes for 0c518df460082d4a3bf51cdac9b0523a:0) 2024.10.22 00:44:34.967514 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 575371f222fb3037f259b69f8027d2be, offset: 0, size: 240524 2024.10.22 00:44:34.970275 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 575371f222fb3037f259b69f8027d2be for removal 2024.10.22 00:44:34.972946 [ 1476 ] {} CacheMetadata: Key 575371f222fb3037f259b69f8027d2be is removed from metadata 2024.10.22 00:44:34.973520 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 957af705cc6f3f2c764d87a3558fb91f, offset: 0, size: 154 2024.10.22 00:44:34.977931 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 957af705cc6f3f2c764d87a3558fb91f for removal 2024.10.22 00:44:34.979559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6fad91d5cd92f400332243732d58a443, offset: 0, size: 148 2024.10.22 00:44:34.979585 [ 1476 ] {} CacheMetadata: Key 957af705cc6f3f2c764d87a3558fb91f is removed from metadata 2024.10.22 00:44:34.981418 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6fad91d5cd92f400332243732d58a443 for removal 2024.10.22 00:44:34.983463 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 29ec2dd3ff792e4e9c9726cee97c5456, offset: 0, size: 94124 2024.10.22 00:44:34.983463 [ 1476 ] {} CacheMetadata: Key 6fad91d5cd92f400332243732d58a443 is removed from metadata 2024.10.22 00:44:34.985593 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 29ec2dd3ff792e4e9c9726cee97c5456 for removal 2024.10.22 00:44:34.987916 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ea19f279dfe1fd4818962f580c13eeca, offset: 0, size: 165 2024.10.22 00:44:34.988600 [ 1476 ] {} CacheMetadata: Key 29ec2dd3ff792e4e9c9726cee97c5456 is removed from metadata 2024.10.22 00:44:34.991797 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ea19f279dfe1fd4818962f580c13eeca for removal 2024.10.22 00:44:34.995224 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5cc2f548868f8d6f990f081864ad8188, offset: 0, size: 62514 2024.10.22 00:44:34.997328 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 166 2024.10.22 00:44:34.999397 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5cc2f548868f8d6f990f081864ad8188 for removal 2024.10.22 00:44:35.008156 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 999a27b35ed00688ba0eda04cd96d9f2, offset: 0, size: 43324 2024.10.22 00:44:35.013148 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 999a27b35ed00688ba0eda04cd96d9f2 for removal 2024.10.22 00:44:35.014905 [ 1476 ] {} CacheMetadata: Key 5cc2f548868f8d6f990f081864ad8188 is removed from metadata 2024.10.22 00:44:35.017325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a4b5b6c4ffeff11045127d4ed7b32a5e, offset: 0, size: 154 2024.10.22 00:44:35.021193 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a4b5b6c4ffeff11045127d4ed7b32a5e for removal 2024.10.22 00:44:35.022480 [ 1476 ] {} CacheMetadata: Key 999a27b35ed00688ba0eda04cd96d9f2 is removed from metadata 2024.10.22 00:44:35.029257 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5675a9f770f2680c87c03e7744512915, offset: 0, size: 61455 2024.10.22 00:44:35.037335 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5675a9f770f2680c87c03e7744512915 for removal 2024.10.22 00:44:35.040659 [ 1476 ] {} CacheMetadata: Key a4b5b6c4ffeff11045127d4ed7b32a5e is removed from metadata 2024.10.22 00:44:35.045829 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2f38e428d9d3d5e78b31495aaa59fdaa, offset: 0, size: 154 2024.10.22 00:44:35.049777 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2f38e428d9d3d5e78b31495aaa59fdaa for removal 2024.10.22 00:44:35.055778 [ 1476 ] {} CacheMetadata: Key 2f38e428d9d3d5e78b31495aaa59fdaa is removed from metadata 2024.10.22 00:44:35.068715 [ 1476 ] {} CacheMetadata: Key 5675a9f770f2680c87c03e7744512915 is removed from metadata 2024.10.22 00:44:35.078551 [ 1476 ] {} CacheMetadata: Key ea19f279dfe1fd4818962f580c13eeca is removed from metadata 2024.10.22 00:44:35.111633 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_121_24, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0] 2024.10.22 00:44:35.114124 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_121_24, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0] 2024.10.22 00:44:35.152576 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_121_24, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0] 2024.10.22 00:44:35.520537 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 6.78 MiB, will set to 1.64 GiB (RSS), difference: -1.47 MiB 2024.10.22 00:44:35.606915 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} MergeTask::MergeProjectionsStage: Merge sorted 18619 rows, containing 13 columns (13 merged, 0 gathered) in 2.717951105 sec., 6850.380776073601 rows/sec., 1.02 MiB/sec. 2024.10.22 00:44:35.661738 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} MergedBlockOutputStream: filled checksums 202410_1_329_67 (state Temporary) 2024.10.22 00:44:35.718869 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_329_67 to 202410_1_329_67 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:35.724759 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_329_67} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_324_66, 202410_329_329_0] -> 202410_1_329_67 2024.10.22 00:44:35.867644 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.73 MiB. 2024.10.22 00:44:35.961674 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 403641 bytes for 81ebd8c138c31e965000fe9add4b4883:0) 2024.10.22 00:44:35.965549 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 841b061292c1dd350f5f10879e02a1e7, offset: 0, size: 163351 2024.10.22 00:44:36.070614 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 841b061292c1dd350f5f10879e02a1e7 for removal 2024.10.22 00:44:36.074282 [ 1476 ] {} CacheMetadata: Key 841b061292c1dd350f5f10879e02a1e7 is removed from metadata 2024.10.22 00:44:36.075804 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 47847248c5bbc987b0694f2143ab106a, offset: 0, size: 166 2024.10.22 00:44:36.078770 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 47847248c5bbc987b0694f2143ab106a for removal 2024.10.22 00:44:36.081297 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b94e49bf3f38faff2e2718cf98b9f8ad, offset: 0, size: 4964 2024.10.22 00:44:36.084210 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b94e49bf3f38faff2e2718cf98b9f8ad for removal 2024.10.22 00:44:36.084291 [ 1476 ] {} CacheMetadata: Key 47847248c5bbc987b0694f2143ab106a is removed from metadata 2024.10.22 00:44:36.086896 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 29c44e40e924b7279e1d8e74bef06291, offset: 0, size: 142 2024.10.22 00:44:36.089818 [ 1476 ] {} CacheMetadata: Key b94e49bf3f38faff2e2718cf98b9f8ad is removed from metadata 2024.10.22 00:44:36.099977 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 29c44e40e924b7279e1d8e74bef06291 for removal 2024.10.22 00:44:36.106324 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e1209af38c7787f63c866d66addf0e7f, offset: 0, size: 220577 2024.10.22 00:44:36.105799 [ 1476 ] {} CacheMetadata: Key 29c44e40e924b7279e1d8e74bef06291 is removed from metadata 2024.10.22 00:44:36.115195 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e1209af38c7787f63c866d66addf0e7f for removal 2024.10.22 00:44:36.118510 [ 1476 ] {} CacheMetadata: Key e1209af38c7787f63c866d66addf0e7f is removed from metadata 2024.10.22 00:44:36.119190 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e19d86da8663fa3a4561686afb175276, offset: 0, size: 147 2024.10.22 00:44:36.127914 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e19d86da8663fa3a4561686afb175276 for removal 2024.10.22 00:44:36.138536 [ 1476 ] {} CacheMetadata: Key e19d86da8663fa3a4561686afb175276 is removed from metadata 2024.10.22 00:44:36.573312 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 183100 bytes for c47476d8ec5b72fdc450fdc2093e236c:0) 2024.10.22 00:44:36.576138 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 71d197ba70cd08f8f84a2384a2516fbf, offset: 0, size: 212475 2024.10.22 00:44:36.578320 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 71d197ba70cd08f8f84a2384a2516fbf for removal 2024.10.22 00:44:36.580238 [ 1476 ] {} CacheMetadata: Key 71d197ba70cd08f8f84a2384a2516fbf is removed from metadata 2024.10.22 00:44:36.580689 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8e6bb6583dc380ba359dcb3c11b4bcfb, offset: 0, size: 147 2024.10.22 00:44:36.582515 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8e6bb6583dc380ba359dcb3c11b4bcfb for removal 2024.10.22 00:44:36.584139 [ 1476 ] {} CacheMetadata: Key 8e6bb6583dc380ba359dcb3c11b4bcfb is removed from metadata 2024.10.22 00:44:36.912769 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 2596 2024.10.22 00:44:36.983442 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 122 entries to flush up to offset 17910 2024.10.22 00:44:37.020468 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 182342 bytes for febe1ee8f271fd6e8204620406e8e25f:0) 2024.10.22 00:44:37.022534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b024d80c3455e45eb9204806a61a1103, offset: 0, size: 257880 2024.10.22 00:44:37.028869 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b024d80c3455e45eb9204806a61a1103 for removal 2024.10.22 00:44:37.032955 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fd075af6ee626c1817f8285821ae3af0, offset: 0, size: 147 2024.10.22 00:44:37.033031 [ 1476 ] {} CacheMetadata: Key b024d80c3455e45eb9204806a61a1103 is removed from metadata 2024.10.22 00:44:37.041011 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fd075af6ee626c1817f8285821ae3af0 for removal 2024.10.22 00:44:37.104722 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_125_25, 202410_126_126_0, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0] 2024.10.22 00:44:37.107238 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_125_25, 202410_126_126_0, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0] 2024.10.22 00:44:37.132452 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_125_25, 202410_126_126_0, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0] 2024.10.22 00:44:37.149605 [ 1476 ] {} CacheMetadata: Key fd075af6ee626c1817f8285821ae3af0 is removed from metadata 2024.10.22 00:44:37.344665 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:37.347258 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:37.476721 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_185_185_0 (state Temporary) 2024.10.22 00:44:37.520146 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_185_185_0 to 202410_198_198_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:37.566447 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2596 2024.10.22 00:44:37.929338 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:37.933167 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:37.994654 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_37_37_0 to 202410_40_40_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:38.131872 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 429253 bytes for 07cba649fd2ba1baaeee0e9904ea6d72:0) 2024.10.22 00:44:38.141110 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bfcfd266a029d71b3702a4a0956d1241, offset: 0, size: 153 2024.10.22 00:44:38.149417 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bfcfd266a029d71b3702a4a0956d1241 for removal 2024.10.22 00:44:38.158512 [ 1476 ] {} CacheMetadata: Key bfcfd266a029d71b3702a4a0956d1241 is removed from metadata 2024.10.22 00:44:38.160900 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: acd3a2b5417c269c03f90030e3b7a9ec, offset: 0, size: 523421 2024.10.22 00:44:38.164896 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key acd3a2b5417c269c03f90030e3b7a9ec for removal 2024.10.22 00:44:38.168184 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 66973a2a59cb0c0334ae472908b062de, offset: 0, size: 333887 2024.10.22 00:44:38.172861 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 66973a2a59cb0c0334ae472908b062de for removal 2024.10.22 00:44:38.172912 [ 1476 ] {} CacheMetadata: Key acd3a2b5417c269c03f90030e3b7a9ec is removed from metadata 2024.10.22 00:44:38.181290 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f52b4890cdff7a804960806a8c9af8dc, offset: 0, size: 150 2024.10.22 00:44:38.184632 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f52b4890cdff7a804960806a8c9af8dc for removal 2024.10.22 00:44:38.185043 [ 1476 ] {} CacheMetadata: Key 66973a2a59cb0c0334ae472908b062de is removed from metadata 2024.10.22 00:44:38.192056 [ 1476 ] {} CacheMetadata: Key f52b4890cdff7a804960806a8c9af8dc is removed from metadata 2024.10.22 00:44:38.289742 [ 1260 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:44:38.310990 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_317_317_0 (state Temporary) 2024.10.22 00:44:38.326914 [ 1260 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:44:38.364603 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_317_317_0 to 202410_330_330_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:38.423824 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 17910 2024.10.22 00:44:39.087406 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 966414 bytes for 608f7e2dcaba466c1c5eb7fbe329e863:0) 2024.10.22 00:44:39.090120 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e3d928668325bd71df18ca2cd6e15852, offset: 0, size: 149 2024.10.22 00:44:39.092467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e3d928668325bd71df18ca2cd6e15852 for removal 2024.10.22 00:44:39.095808 [ 1476 ] {} CacheMetadata: Key e3d928668325bd71df18ca2cd6e15852 is removed from metadata 2024.10.22 00:44:39.097734 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aba04db95c93cc04154ecff1c245fa48, offset: 0, size: 300928 2024.10.22 00:44:39.100880 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aba04db95c93cc04154ecff1c245fa48 for removal 2024.10.22 00:44:39.103603 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 75dd9093709234637f49191c6f95a67d, offset: 0, size: 513985 2024.10.22 00:44:39.107635 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 75dd9093709234637f49191c6f95a67d for removal 2024.10.22 00:44:39.110599 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d707bdcd9c718acf3a2e229092dd7efb, offset: 0, size: 153 2024.10.22 00:44:39.113759 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d707bdcd9c718acf3a2e229092dd7efb for removal 2024.10.22 00:44:39.210775 [ 1476 ] {} CacheMetadata: Key d707bdcd9c718acf3a2e229092dd7efb is removed from metadata 2024.10.22 00:44:39.218767 [ 1476 ] {} CacheMetadata: Key 75dd9093709234637f49191c6f95a67d is removed from metadata 2024.10.22 00:44:39.230072 [ 1476 ] {} CacheMetadata: Key aba04db95c93cc04154ecff1c245fa48 is removed from metadata 2024.10.22 00:44:39.551050 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 4.53 MiB, will set to 1.63 GiB (RSS), difference: 4.53 MiB 2024.10.22 00:44:39.569365 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:44:39.572107 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:44:39.684866 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 5757cd667c7156ac24d7af32fad0db03:0) 2024.10.22 00:44:39.686477 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 81ebd8c138c31e965000fe9add4b4883, offset: 0, size: 403641 2024.10.22 00:44:39.689448 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 81ebd8c138c31e965000fe9add4b4883 for removal 2024.10.22 00:44:39.692264 [ 1476 ] {} CacheMetadata: Key 81ebd8c138c31e965000fe9add4b4883 is removed from metadata 2024.10.22 00:44:39.692264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5b2ac666647936fd4272f50e45110c51, offset: 0, size: 152 2024.10.22 00:44:39.695268 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5b2ac666647936fd4272f50e45110c51 for removal 2024.10.22 00:44:39.702135 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0c518df460082d4a3bf51cdac9b0523a, offset: 0, size: 484664 2024.10.22 00:44:39.708815 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0c518df460082d4a3bf51cdac9b0523a for removal 2024.10.22 00:44:39.713772 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c4f38cdeaddab6150a4e509522e10a67, offset: 0, size: 147 2024.10.22 00:44:39.718779 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c4f38cdeaddab6150a4e509522e10a67 for removal 2024.10.22 00:44:39.722253 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e453483c9faeb602a2b8da351d182e34, offset: 0, size: 153 2024.10.22 00:44:39.728138 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e453483c9faeb602a2b8da351d182e34 for removal 2024.10.22 00:44:39.730176 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 308132 2024.10.22 00:44:39.732046 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bf2b02055fa2831910d7d508e71e8637, offset: 0, size: 214430 2024.10.22 00:44:39.736728 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bf2b02055fa2831910d7d508e71e8637 for removal 2024.10.22 00:44:39.803881 [ 1476 ] {} CacheMetadata: Key bf2b02055fa2831910d7d508e71e8637 is removed from metadata 2024.10.22 00:44:39.813168 [ 1476 ] {} CacheMetadata: Key e453483c9faeb602a2b8da351d182e34 is removed from metadata 2024.10.22 00:44:39.819883 [ 1476 ] {} CacheMetadata: Key c4f38cdeaddab6150a4e509522e10a67 is removed from metadata 2024.10.22 00:44:39.830086 [ 1476 ] {} CacheMetadata: Key 0c518df460082d4a3bf51cdac9b0523a is removed from metadata 2024.10.22 00:44:39.837771 [ 1476 ] {} CacheMetadata: Key 5b2ac666647936fd4272f50e45110c51 is removed from metadata 2024.10.22 00:44:39.980470 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 61 entries to flush up to offset 11915 2024.10.22 00:44:39.973672 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1469 2024.10.22 00:44:40.123254 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:40.126695 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:40.179304 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 551240 bytes for 8d2a43fee02ca1b204056c772b9f9766:0) 2024.10.22 00:44:40.180717 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e1505c5361195dbf76b945fae3519b90, offset: 0, size: 153582 2024.10.22 00:44:40.183828 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e1505c5361195dbf76b945fae3519b90 for removal 2024.10.22 00:44:40.186636 [ 1476 ] {} CacheMetadata: Key e1505c5361195dbf76b945fae3519b90 is removed from metadata 2024.10.22 00:44:40.188217 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 422cdbc61bc819f5139eca78befb875e, offset: 0, size: 146 2024.10.22 00:44:40.195717 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_39_8/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:44:40.263898 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_184_184_0 (state Temporary) 2024.10.22 00:44:40.287373 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_184_184_0 to 202410_190_190_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:40.292253 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 422cdbc61bc819f5139eca78befb875e for removal 2024.10.22 00:44:40.308040 [ 1476 ] {} CacheMetadata: Key 422cdbc61bc819f5139eca78befb875e is removed from metadata 2024.10.22 00:44:40.308573 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8bb48caa8d3ffbd1b74f56591e86389b, offset: 0, size: 146 2024.10.22 00:44:40.312822 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8bb48caa8d3ffbd1b74f56591e86389b for removal 2024.10.22 00:44:40.320300 [ 1476 ] {} CacheMetadata: Key 8bb48caa8d3ffbd1b74f56591e86389b is removed from metadata 2024.10.22 00:44:40.320645 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: febe1ee8f271fd6e8204620406e8e25f, offset: 0, size: 182342 2024.10.22 00:44:40.324739 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key febe1ee8f271fd6e8204620406e8e25f for removal 2024.10.22 00:44:40.329190 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c47476d8ec5b72fdc450fdc2093e236c, offset: 0, size: 183100 2024.10.22 00:44:40.329811 [ 1476 ] {} CacheMetadata: Key febe1ee8f271fd6e8204620406e8e25f is removed from metadata 2024.10.22 00:44:40.334652 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c47476d8ec5b72fdc450fdc2093e236c for removal 2024.10.22 00:44:40.437715 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:40.442578 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 23a52d50571aaf78d094ea3c9011e5ec, offset: 0, size: 146 2024.10.22 00:44:40.445828 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:40.447891 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 23a52d50571aaf78d094ea3c9011e5ec for removal 2024.10.22 00:44:40.466375 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11915 2024.10.22 00:44:40.547298 [ 1476 ] {} CacheMetadata: Key c47476d8ec5b72fdc450fdc2093e236c is removed from metadata 2024.10.22 00:44:40.554722 [ 1476 ] {} CacheMetadata: Key 23a52d50571aaf78d094ea3c9011e5ec is removed from metadata 2024.10.22 00:44:40.572844 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_177_177_0 (state Temporary) 2024.10.22 00:44:40.716380 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_177_177_0 to 202410_184_184_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:40.766546 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 165095 bytes for 5757cd667c7156ac24d7af32fad0db03:0) 2024.10.22 00:44:40.768897 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 07cba649fd2ba1baaeee0e9904ea6d72, offset: 0, size: 429253 2024.10.22 00:44:40.772207 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 07cba649fd2ba1baaeee0e9904ea6d72 for removal 2024.10.22 00:44:40.778270 [ 1476 ] {} CacheMetadata: Key 07cba649fd2ba1baaeee0e9904ea6d72 is removed from metadata 2024.10.22 00:44:40.778332 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 37808c1e8b7d1465ccd0b456142f174c, offset: 0, size: 152 2024.10.22 00:44:40.782351 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 37808c1e8b7d1465ccd0b456142f174c for removal 2024.10.22 00:44:40.787234 [ 1476 ] {} CacheMetadata: Key 37808c1e8b7d1465ccd0b456142f174c is removed from metadata 2024.10.22 00:44:40.848007 [ 1735 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/iak/zkpczqbryvkqmuoanokgzsjmqbure, size 551240 2024.10.22 00:44:41.057324 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 308132 2024.10.22 00:44:41.396232 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FlashMinor 2024.10.22 00:44:41.420023 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FlashMinor 2024.10.22 00:44:41.440775 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FlashMinor 2024.10.22 00:44:41.476237 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FlashMinor 2024.10.22 00:44:41.483523 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FlashMinor 2024.10.22 00:44:41.521951 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FlashMinor 2024.10.22 00:44:41.541600 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FlashMinor 2024.10.22 00:44:41.589585 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 434264 bytes for 06e90f4caafd8156c77b38e6e9545ca5:0) 2024.10.22 00:44:41.590953 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 54c7a34b1eb84f8d73a97364637dc406, offset: 0, size: 258578 2024.10.22 00:44:41.595503 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 54c7a34b1eb84f8d73a97364637dc406 for removal 2024.10.22 00:44:41.600134 [ 1476 ] {} CacheMetadata: Key 54c7a34b1eb84f8d73a97364637dc406 is removed from metadata 2024.10.22 00:44:41.602695 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3414b4b6f508e248a0802d56a83345c5, offset: 0, size: 147 2024.10.22 00:44:41.688063 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/srn/kmehfddtofizdujocydrgespvqvsv (LogSeriesLimiter: on interval from 2024-10-22 00:44:26 to 2024-10-22 00:44:41 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:44:41.716872 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3414b4b6f508e248a0802d56a83345c5 for removal 2024.10.22 00:44:41.719438 [ 1476 ] {} CacheMetadata: Key 3414b4b6f508e248a0802d56a83345c5 is removed from metadata 2024.10.22 00:44:41.934865 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yea/ckaywwgvtguglvzonslximtbmmrlk. 2024.10.22 00:44:42.381955 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:44:42.475108 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 8.39 MiB, will set to 1.63 GiB (RSS), difference: 2.86 MiB 2024.10.22 00:44:42.501567 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 167 2024.10.22 00:44:42.541839 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 130 entries to flush up to offset 18040 2024.10.22 00:44:42.720546 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_12_12_0 to 201403_12_12_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:42.740889 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yea/ckaywwgvtguglvzonslximtbmmrlk. 2024.10.22 00:44:42.921748 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:42.926983 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:43.094698 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_149_149_0 (state Temporary) 2024.10.22 00:44:43.158380 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:43.163303 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:43.275097 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_149_149_0 to 202410_159_159_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:43.346337 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_318_318_0 (state Temporary) 2024.10.22 00:44:43.544507 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 167 2024.10.22 00:44:43.587430 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_318_318_0 to 202410_331_331_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:43.729767 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18040 2024.10.22 00:44:44.419871 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 741.07 MiB, peak 1.88 GiB, free memory in arenas 7.72 MiB, will set to 1.33 GiB (RSS), difference: 623.66 MiB 2024.10.22 00:44:45.068426 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 2604 2024.10.22 00:44:45.259981 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:45.266351 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:45.430825 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_186_186_0 (state Temporary) 2024.10.22 00:44:45.483957 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_186_186_0 to 202410_199_199_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:45.532359 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2604 2024.10.22 00:44:46.356583 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 7.76 MiB, will set to 1.17 GiB (RSS), difference: -239.66 MiB 2024.10.22 00:44:47.473809 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 1.88 GiB, free memory in arenas 8.28 MiB, will set to 1.18 GiB (RSS), difference: -71.12 MiB 2024.10.22 00:44:47.505058 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 1506 2024.10.22 00:44:47.629297 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 12 2024.10.22 00:44:47.751638 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 18066 2024.10.22 00:44:47.850891 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:47.858136 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:47.910148 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_319_319_0 (state Temporary) 2024.10.22 00:44:47.971831 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 38 entries to flush up to offset 11953 2024.10.22 00:44:47.976558 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_319_319_0 to 202410_332_332_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:48.040483 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18066 2024.10.22 00:44:48.149126 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1720 entries to flush up to offset 309852 2024.10.22 00:44:48.172639 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:48.175240 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:48.215301 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_12_12_0 (state Temporary) 2024.10.22 00:44:48.275653 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:48.276725 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:48.291771 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_12_12_0 to 202410_12_12_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:48.398605 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_185_185_0 (state Temporary) 2024.10.22 00:44:48.469718 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_226_45, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0, 202410_230_230_0, 202410_231_231_0] 2024.10.22 00:44:48.471792 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_226_45, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0, 202410_230_230_0, 202410_231_231_0] 2024.10.22 00:44:48.493407 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_226_45, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0, 202410_230_230_0, 202410_231_231_0] 2024.10.22 00:44:48.566112 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_185_185_0 to 202410_191_191_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:48.620368 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11953 2024.10.22 00:44:49.076408 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 12 2024.10.22 00:44:49.523258 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:49.530421 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:49.579915 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:44:49.581766 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:44:49.717476 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.22 GiB, peak 1.88 GiB, free memory in arenas 7.26 MiB, will set to 1.22 GiB (RSS), difference: -9.34 MiB 2024.10.22 00:44:49.739068 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_178_178_0 (state Temporary) 2024.10.22 00:44:49.819449 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_178_178_0 to 202410_185_185_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:49.966295 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 309852 2024.10.22 00:44:51.047010 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 168 2024.10.22 00:44:51.416667 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:51.422632 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:51.483837 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_150_150_0 (state Temporary) 2024.10.22 00:44:51.557628 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.28 GiB, peak 1.88 GiB, free memory in arenas 6.85 MiB, will set to 1.24 GiB (RSS), difference: -33.85 MiB 2024.10.22 00:44:51.707077 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_150_150_0 to 202410_160_160_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:52.045263 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 18101 2024.10.22 00:44:52.169147 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:52.174886 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:52.215894 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_320_320_0 (state Temporary) 2024.10.22 00:44:52.238134 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 168 2024.10.22 00:44:52.262383 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_320_320_0 to 202410_333_333_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:52.294631 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18101 2024.10.22 00:44:53.037022 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 2611 2024.10.22 00:44:53.192336 [ 1192 ] {} DatabaseCatalog: Execution took 399 ms. 2024.10.22 00:44:53.256528 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:53.261193 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:53.333575 [ 1219 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:44:53.367773 [ 1219 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:44:53.384216 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_187_187_0 (state Temporary) 2024.10.22 00:44:53.641637 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_187_187_0 to 202410_200_200_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:53.673827 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 1.88 GiB, free memory in arenas 6.63 MiB, will set to 1.29 GiB (RSS), difference: -24.83 MiB 2024.10.22 00:44:53.694155 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2611 2024.10.22 00:44:54.838123 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 356072 bytes for fa4aae25ca2d41247b0717ffb653fc96:0) 2024.10.22 00:44:54.843857 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 608f7e2dcaba466c1c5eb7fbe329e863, offset: 0, size: 966414 2024.10.22 00:44:54.860352 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 608f7e2dcaba466c1c5eb7fbe329e863 for removal 2024.10.22 00:44:54.864187 [ 1476 ] {} CacheMetadata: Key 608f7e2dcaba466c1c5eb7fbe329e863 is removed from metadata 2024.10.22 00:44:54.864856 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a56d02367daaac965c89b1d4b611b2a1, offset: 0, size: 188 2024.10.22 00:44:54.868730 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a56d02367daaac965c89b1d4b611b2a1 for removal 2024.10.22 00:44:54.870740 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ecc44fb8a7471e07544112199063835f, offset: 0, size: 1268 2024.10.22 00:44:54.872969 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ecc44fb8a7471e07544112199063835f for removal 2024.10.22 00:44:54.881136 [ 1476 ] {} CacheMetadata: Key ecc44fb8a7471e07544112199063835f is removed from metadata 2024.10.22 00:44:54.875485 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5307439ecd0fdd334754b6284a57c5cc, offset: 0, size: 154 2024.10.22 00:44:54.884412 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5307439ecd0fdd334754b6284a57c5cc for removal 2024.10.22 00:44:54.884644 [ 1476 ] {} CacheMetadata: Key a56d02367daaac965c89b1d4b611b2a1 is removed from metadata 2024.10.22 00:44:54.886878 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9951ee343b351a669abb3d02ebb257c4, offset: 0, size: 4 2024.10.22 00:44:54.889496 [ 1476 ] {} CacheMetadata: Key 5307439ecd0fdd334754b6284a57c5cc is removed from metadata 2024.10.22 00:44:54.889671 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9951ee343b351a669abb3d02ebb257c4 for removal 2024.10.22 00:44:54.897385 [ 1476 ] {} CacheMetadata: Key 9951ee343b351a669abb3d02ebb257c4 is removed from metadata 2024.10.22 00:44:55.171013 [ 1731 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/srn/kmehfddtofizdujocydrgespvqvsv, size 356072 2024.10.22 00:44:55.512305 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 1.88 GiB, free memory in arenas 7.12 MiB, will set to 1.33 GiB (RSS), difference: -17.67 MiB 2024.10.22 00:44:55.681087 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 00:44:55.814218 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 00:44:55.829947 [ 1234 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803) (MergerMutator): Selected 4 parts from 202410_1_1_0 to 202410_4_4_0 2024.10.22 00:44:55.843132 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 00:44:55.848244 [ 1234 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:55.857085 [ 1234 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:55.892702 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 00:44:55.905135 [ 1393 ] {ed97c24a-3021-4d92-aee2-1ef2de3fa803::202410_1_4_1} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_1_0 to 202410_4_4_0 into Compact with storage Full 2024.10.22 00:44:55.943863 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 00:44:56.070069 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 00:44:56.085139 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 00:44:56.168074 [ 1393 ] {ed97c24a-3021-4d92-aee2-1ef2de3fa803::202410_1_4_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:56.224628 [ 1393 ] {ed97c24a-3021-4d92-aee2-1ef2de3fa803::202410_1_4_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_1_0, total 2 rows starting from the beginning of the part 2024.10.22 00:44:56.301201 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 18149 2024.10.22 00:44:56.373267 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/htf/qekbpubdtnbirivsslpxpghtydzxe (LogSeriesLimiter: on interval from 2024-10-22 00:44:41 to 2024-10-22 00:44:56 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:44:56.376531 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_39_8/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:44:56.396484 [ 1393 ] {ed97c24a-3021-4d92-aee2-1ef2de3fa803::202410_1_4_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_2_2_0, total 6 rows starting from the beginning of the part 2024.10.22 00:44:56.463556 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:56.465603 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:56.632821 [ 1393 ] {ed97c24a-3021-4d92-aee2-1ef2de3fa803::202410_1_4_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_3_3_0, total 2 rows starting from the beginning of the part 2024.10.22 00:44:56.751362 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_321_321_0 (state Temporary) 2024.10.22 00:44:56.759318 [ 1393 ] {ed97c24a-3021-4d92-aee2-1ef2de3fa803::202410_1_4_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_4_4_0, total 13 rows starting from the beginning of the part 2024.10.22 00:44:56.864854 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 38 entries to flush up to offset 11991 2024.10.22 00:44:56.936465 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_321_321_0 to 202410_334_334_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:56.971221 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1736 entries to flush up to offset 311588 2024.10.22 00:44:56.971591 [ 1151 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_329_67 to 202410_334_334_0 2024.10.22 00:44:56.995859 [ 1151 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:57.007827 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:57.160002 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:57.165755 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_329_67 to 202410_334_334_0 into Compact with storage Full 2024.10.22 00:44:57.166728 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:57.223030 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18149 2024.10.22 00:44:57.233802 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:57.246953 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_329_67, total 18619 rows starting from the beginning of the part 2024.10.22 00:44:57.277662 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_330_330_0, total 122 rows starting from the beginning of the part 2024.10.22 00:44:57.290234 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_331_331_0, total 130 rows starting from the beginning of the part 2024.10.22 00:44:57.304699 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_186_186_0 (state Temporary) 2024.10.22 00:44:57.318879 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_332_332_0, total 26 rows starting from the beginning of the part 2024.10.22 00:44:57.364262 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_333_333_0, total 35 rows starting from the beginning of the part 2024.10.22 00:44:57.370817 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_186_186_0 to 202410_192_192_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:57.409699 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 4 parts from 202410_1_187_38 to 202410_190_190_0 2024.10.22 00:44:57.419183 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:57.422827 [ 1237 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:57.441599 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 11991 2024.10.22 00:44:57.450113 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_334_334_0, total 48 rows starting from the beginning of the part 2024.10.22 00:44:57.451350 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_190_39} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_187_38 to 202410_190_190_0 into Compact with storage Full 2024.10.22 00:44:57.471164 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 4.34 MiB, will set to 1.37 GiB (RSS), difference: -52.63 MiB 2024.10.22 00:44:57.647785 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_190_39} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:57.655926 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_190_39} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_187_38, total 11785 rows starting from the beginning of the part 2024.10.22 00:44:57.681149 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_190_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_188_188_0, total 113 rows starting from the beginning of the part 2024.10.22 00:44:57.720873 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_190_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_189_189_0, total 60 rows starting from the beginning of the part 2024.10.22 00:44:57.765047 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_190_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_190_190_0, total 61 rows starting from the beginning of the part 2024.10.22 00:44:58.031607 [ 1392 ] {ed97c24a-3021-4d92-aee2-1ef2de3fa803::202410_1_4_1} MergeTask::MergeProjectionsStage: Merge sorted 23 rows, containing 21 columns (21 merged, 0 gathered) in 2.137935136 sec., 10.758043877342406 rows/sec., 3.55 KiB/sec. 2024.10.22 00:44:58.072493 [ 1388 ] {ed97c24a-3021-4d92-aee2-1ef2de3fa803::202410_1_4_1} MergedBlockOutputStream: filled checksums 202410_1_4_1 (state Temporary) 2024.10.22 00:44:58.125708 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:58.129330 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:58.248434 [ 1388 ] {ed97c24a-3021-4d92-aee2-1ef2de3fa803::202410_1_4_1} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Renaming temporary part tmp_merge_202410_1_4_1 to 202410_1_4_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:58.255811 [ 1388 ] {ed97c24a-3021-4d92-aee2-1ef2de3fa803::202410_1_4_1} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803) (MergerMutator): Merged 4 parts: [202410_1_1_0, 202410_4_4_0] -> 202410_1_4_1 2024.10.22 00:44:58.304174 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.23 MiB. 2024.10.22 00:44:58.377590 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_179_179_0 (state Temporary) 2024.10.22 00:44:58.544303 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_179_179_0 to 202410_186_186_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:58.681485 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 311588 2024.10.22 00:44:58.695892 [ 1259 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_181_36 to 202410_186_186_0 2024.10.22 00:44:58.716082 [ 1259 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:44:58.720711 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:44:58.771500 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_181_36 to 202410_186_186_0 into Compact with storage Full 2024.10.22 00:44:58.778144 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:44:58.783609 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} MergeTreeSequentialSource: Reading 40 marks from part 202410_1_181_36, total 316730 rows starting from the beginning of the part 2024.10.22 00:44:58.823110 [ 1242 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 320 ms. 2024.10.22 00:44:58.856018 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_182_182_0, total 1730 rows starting from the beginning of the part 2024.10.22 00:44:58.879915 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_183_183_0, total 1726 rows starting from the beginning of the part 2024.10.22 00:44:58.899921 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_184_184_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:44:58.919750 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_185_185_0, total 1720 rows starting from the beginning of the part 2024.10.22 00:44:58.946128 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_186_186_0, total 1736 rows starting from the beginning of the part 2024.10.22 00:44:58.984045 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_190_39} MergeTask::MergeProjectionsStage: Merge sorted 12019 rows, containing 13 columns (13 merged, 0 gathered) in 1.540046745 sec., 7804.308563374159 rows/sec., 3.27 MiB/sec. 2024.10.22 00:44:59.033068 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_190_39} MergedBlockOutputStream: filled checksums 202410_1_190_39 (state Temporary) 2024.10.22 00:44:59.331679 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_190_39} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_190_39 to 202410_1_190_39 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:59.337078 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_190_39} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 4 parts: [202410_1_187_38, 202410_190_190_0] -> 202410_1_190_39 2024.10.22 00:44:59.410004 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} MergeTask::MergeProjectionsStage: Merge sorted 18980 rows, containing 13 columns (13 merged, 0 gathered) in 2.258200035 sec., 8404.924145703504 rows/sec., 1.25 MiB/sec. 2024.10.22 00:44:59.418536 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.88 GiB, free memory in arenas 5.63 MiB, will set to 1.40 GiB (RSS), difference: -121.59 MiB 2024.10.22 00:44:59.469533 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.26 MiB. 2024.10.22 00:44:59.476783 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} MergedBlockOutputStream: filled checksums 202410_1_334_68 (state Temporary) 2024.10.22 00:44:59.580115 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:44:59.582494 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:44:59.742979 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 169 2024.10.22 00:44:59.879262 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_334_68 to 202410_1_334_68 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:44:59.882636 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_334_68} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_329_67, 202410_334_334_0] -> 202410_1_334_68 2024.10.22 00:44:59.952709 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.73 MiB. 2024.10.22 00:45:00.064293 [ 1265 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 00:45:00.099583 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:00.106829 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:00.176499 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_136_27, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0] 2024.10.22 00:45:00.182766 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_136_27, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0] 2024.10.22 00:45:00.194715 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_151_151_0 (state Temporary) 2024.10.22 00:45:00.453939 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_136_27, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0] 2024.10.22 00:45:00.492525 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_151_151_0 to 202410_161_161_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:00.533364 [ 1152 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_156_32 to 202410_161_161_0 2024.10.22 00:45:00.667691 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 169 2024.10.22 00:45:00.757927 [ 1152 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:00.762682 [ 1152 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:00.768447 [ 1152 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 257 ms. 2024.10.22 00:45:00.781515 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_156_32 to 202410_161_161_0 into Compact with storage Full 2024.10.22 00:45:01.046434 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:45:01.068638 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_156_32, total 200 rows starting from the beginning of the part 2024.10.22 00:45:01.197754 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 2633 2024.10.22 00:45:01.232253 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 18241 2024.10.22 00:45:01.259405 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_157_157_0, total 1 rows starting from the beginning of the part 2024.10.22 00:45:01.432889 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:01.446632 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:01.568472 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_158_158_0, total 1 rows starting from the beginning of the part 2024.10.22 00:45:01.601355 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_188_188_0 (state Temporary) 2024.10.22 00:45:01.661102 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_159_159_0, total 1 rows starting from the beginning of the part 2024.10.22 00:45:01.663427 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_188_188_0 to 202410_201_201_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:01.770277 [ 1273 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_196_39 to 202410_201_201_0 2024.10.22 00:45:01.771842 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_160_160_0, total 1 rows starting from the beginning of the part 2024.10.22 00:45:01.776430 [ 1273 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:01.779302 [ 1273 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:01.797792 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2633 2024.10.22 00:45:01.847681 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_161_161_0, total 1 rows starting from the beginning of the part 2024.10.22 00:45:01.868109 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_196_39 to 202410_201_201_0 into Compact with storage Full 2024.10.22 00:45:01.869236 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_186_37/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:45:01.894618 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:01.903853 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:01.905714 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:45:01.914111 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_196_39, total 2670 rows starting from the beginning of the part 2024.10.22 00:45:01.922835 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_197_197_0, total 20 rows starting from the beginning of the part 2024.10.22 00:45:01.953760 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.88 GiB, free memory in arenas 10.31 MiB, will set to 1.46 GiB (RSS), difference: -30.32 MiB 2024.10.22 00:45:01.989558 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_322_322_0 (state Temporary) 2024.10.22 00:45:02.060962 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_198_198_0, total 12 rows starting from the beginning of the part 2024.10.22 00:45:02.073313 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_199_199_0, total 8 rows starting from the beginning of the part 2024.10.22 00:45:02.086485 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_200_200_0, total 7 rows starting from the beginning of the part 2024.10.22 00:45:02.160699 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_322_322_0 to 202410_335_335_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:02.198870 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_201_201_0, total 22 rows starting from the beginning of the part 2024.10.22 00:45:02.284396 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:45:02.415710 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18241 2024.10.22 00:45:03.424237 [ 1263 ] {} DatabaseCatalog: Execution took 231 ms. 2024.10.22 00:45:03.474958 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} MergeTask::MergeProjectionsStage: Merge sorted 2739 rows, containing 12 columns (12 merged, 0 gathered) in 1.624416617 sec., 1686.143795462048 rows/sec., 262.16 KiB/sec. 2024.10.22 00:45:03.491489 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} MergedBlockOutputStream: filled checksums 202410_1_201_40 (state Temporary) 2024.10.22 00:45:03.756545 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_201_40 to 202410_1_201_40 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:03.762601 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_201_40} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_196_39, 202410_201_201_0] -> 202410_1_201_40 2024.10.22 00:45:03.883590 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:45:04.019578 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} MergeTask::MergeProjectionsStage: Merge sorted 205 rows, containing 32 columns (32 merged, 0 gathered) in 3.240773971 sec., 63.25649423083447 rows/sec., 6.81 KiB/sec. 2024.10.22 00:45:04.177169 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} MergedBlockOutputStream: filled checksums 202410_1_161_33 (state Temporary) 2024.10.22 00:45:04.420187 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_161_33 to 202410_1_161_33 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:04.424551 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_161_33} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_156_32, 202410_161_161_0] -> 202410_1_161_33 2024.10.22 00:45:04.488454 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.88 GiB, free memory in arenas 14.62 MiB, will set to 1.51 GiB (RSS), difference: -39.70 MiB 2024.10.22 00:45:04.612767 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 00:45:04.680440 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:45:04.954069 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 156 entries to flush up to offset 12147 2024.10.22 00:45:05.216912 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_108_22, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0, 202410_112_112_0, 202410_113_113_0] 2024.10.22 00:45:05.218211 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_108_22, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0, 202410_112_112_0, 202410_113_113_0] 2024.10.22 00:45:05.228842 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:05.236465 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:05.262166 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_108_22, 202410_109_109_0, 202410_110_110_0, 202410_111_111_0, 202410_112_112_0, 202410_113_113_0] 2024.10.22 00:45:05.296078 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:05.299534 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:05.310780 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_187_187_0 (state Temporary) 2024.10.22 00:45:05.468698 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_187_187_0 to 202410_193_193_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:05.644591 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12147 2024.10.22 00:45:05.684775 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 313124 2024.10.22 00:45:06.348989 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.88 GiB, free memory in arenas 20.11 MiB, will set to 1.56 GiB (RSS), difference: -16.56 MiB 2024.10.22 00:45:06.434896 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 18297 2024.10.22 00:45:06.771864 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:06.782729 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:06.914674 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:06.926747 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:06.947309 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_323_323_0 (state Temporary) 2024.10.22 00:45:07.167040 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_180_180_0 (state Temporary) 2024.10.22 00:45:07.221191 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_323_323_0 to 202410_336_336_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:07.379196 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_180_180_0 to 202410_187_187_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:07.386245 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18297 2024.10.22 00:45:07.520365 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 313124 2024.10.22 00:45:07.652722 [ 1170 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 255 ms. 2024.10.22 00:45:07.785929 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_38_38_0 (state Temporary) 2024.10.22 00:45:08.372482 [ 1206 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:45:08.384542 [ 1206 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:45:08.460384 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 21.65 MiB, will set to 1.60 GiB (RSS), difference: -26.65 MiB 2024.10.22 00:45:09.304459 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 2652 2024.10.22 00:45:09.542655 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:09.547196 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:09.581112 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:45:09.583738 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:45:09.686943 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_189_189_0 (state Temporary) 2024.10.22 00:45:09.749711 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_189_189_0 to 202410_202_202_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:09.868119 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2652 2024.10.22 00:45:10.011820 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} MergeTask::MergeProjectionsStage: Merge sorted 325178 rows, containing 4 columns (4 merged, 0 gathered) in 11.246495994 sec., 28913.71678551989 rows/sec., 424.27 KiB/sec. 2024.10.22 00:45:10.155635 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} MergedBlockOutputStream: filled checksums 202410_1_186_37 (state Temporary) 2024.10.22 00:45:10.181374 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_186_37 to 202410_1_186_37 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:10.184522 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_186_37} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_181_36, 202410_186_186_0] -> 202410_1_186_37 2024.10.22 00:45:10.226134 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:45:10.374820 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 19.26 MiB, will set to 1.62 GiB (RSS), difference: -16.76 MiB 2024.10.22 00:45:10.985424 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_38_38_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:45:11.392027 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 18328 2024.10.22 00:45:11.455589 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 15.39 MiB, will set to 1.65 GiB (RSS), difference: -26.79 MiB 2024.10.22 00:45:11.532561 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:11.537669 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:11.582052 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_324_324_0 (state Temporary) 2024.10.22 00:45:11.950545 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_324_324_0 to 202410_337_337_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:12.071654 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18328 2024.10.22 00:45:13.032413 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_231_46, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0, 202410_236_236_0] 2024.10.22 00:45:13.142764 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_231_46, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0, 202410_236_236_0] 2024.10.22 00:45:13.178721 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 12197 2024.10.22 00:45:13.279146 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_231_46, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0, 202410_236_236_0] 2024.10.22 00:45:13.385200 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:13.390808 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:13.429267 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_188_188_0 (state Temporary) 2024.10.22 00:45:13.478579 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_188_188_0 to 202410_194_194_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:13.518893 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 12.62 MiB, will set to 1.67 GiB (RSS), difference: -43.86 MiB 2024.10.22 00:45:13.705594 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12197 2024.10.22 00:45:14.242524 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 867630 bytes for 812709cfcf7bca5aad2b6d9e2e88def7:0) 2024.10.22 00:45:14.248156 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5757cd667c7156ac24d7af32fad0db03, offset: 0, size: 1213671 2024.10.22 00:45:14.255507 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5757cd667c7156ac24d7af32fad0db03 for removal 2024.10.22 00:45:14.266735 [ 1476 ] {} CacheMetadata: Key 5757cd667c7156ac24d7af32fad0db03 is removed from metadata 2024.10.22 00:45:14.267814 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d15c7bbc09ec3f51acb4d6e751fffc1b, offset: 0, size: 192 2024.10.22 00:45:14.271493 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d15c7bbc09ec3f51acb4d6e751fffc1b for removal 2024.10.22 00:45:14.283217 [ 1476 ] {} CacheMetadata: Key d15c7bbc09ec3f51acb4d6e751fffc1b is removed from metadata 2024.10.22 00:45:14.325790 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 254.08 MiB using storage policy from min volume index 0 2024.10.22 00:45:14.526430 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 315044 2024.10.22 00:45:14.570753 [ 1725 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/htf/qekbpubdtnbirivsslpxpghtydzxe, size 867630 2024.10.22 00:45:14.873276 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column NetMajor 2024.10.22 00:45:14.882867 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column NetMajor 2024.10.22 00:45:15.072390 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column NetMajor 2024.10.22 00:45:15.186034 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column NetMajor 2024.10.22 00:45:15.206956 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:15.212925 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:15.215832 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column NetMajor 2024.10.22 00:45:15.248772 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column NetMajor 2024.10.22 00:45:15.282745 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column NetMajor 2024.10.22 00:45:15.341109 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zuw/okdwmuxafpkoihsvgzsqvcflntrja (LogSeriesLimiter: on interval from 2024-10-22 00:44:56 to 2024-10-22 00:45:15 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:45:15.548748 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_181_181_0 (state Temporary) 2024.10.22 00:45:15.627723 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_181_181_0 to 202410_188_188_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:15.664037 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 315044 2024.10.22 00:45:15.669505 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 170 2024.10.22 00:45:15.690378 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 8.53 MiB, will set to 1.66 GiB (RSS), difference: -45.01 MiB 2024.10.22 00:45:15.831047 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_38_38_0 to 202410_41_41_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:16.116994 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:16.176844 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 18371 2024.10.22 00:45:16.178751 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:16.233246 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_152_152_0 (state Temporary) 2024.10.22 00:45:16.408304 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:16.445567 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:16.491911 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_325_325_0 (state Temporary) 2024.10.22 00:45:16.528803 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_325_325_0 to 202410_338_338_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:16.560823 [ 1226 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 333 ms. 2024.10.22 00:45:16.576875 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_152_152_0 to 202410_162_162_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:16.583270 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18371 2024.10.22 00:45:16.677827 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 170 2024.10.22 00:45:17.371239 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 2666 2024.10.22 00:45:17.578546 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:17.581667 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:17.713071 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_190_190_0 (state Temporary) 2024.10.22 00:45:17.847882 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_190_190_0 to 202410_203_203_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:17.891201 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2666 2024.10.22 00:45:17.915988 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.83 GiB, peak 1.88 GiB, free memory in arenas 6.57 MiB, will set to 1.66 GiB (RSS), difference: -171.86 MiB 2024.10.22 00:45:18.368062 [ 1253 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 335 ms. 2024.10.22 00:45:18.412280 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} MergeTask::MergeProjectionsStage: Merge sorted 1490 rows, containing 679 columns (679 merged, 0 gathered) in 78.543532922 sec., 18.970371519698368 rows/sec., 25.48 KiB/sec. 2024.10.22 00:45:18.761233 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1506 2024.10.22 00:45:19.214754 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_130_26, 202410_131_131_0, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0] 2024.10.22 00:45:19.218348 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_130_26, 202410_131_131_0, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0] 2024.10.22 00:45:19.263607 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_130_26, 202410_131_131_0, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0] 2024.10.22 00:45:19.392147 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.88 GiB, free memory in arenas 9.04 MiB, will set to 1.66 GiB (RSS), difference: -137.60 MiB 2024.10.22 00:45:19.581003 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:45:19.583929 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:45:19.651999 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_126_25, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0] 2024.10.22 00:45:19.655811 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_126_25, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0] 2024.10.22 00:45:19.689653 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_126_25, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0] 2024.10.22 00:45:20.090354 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rmf/yslnhhowyadzqzbrnocvlmrtyizoc (LogSeriesLimiter: on interval from 2024-10-22 00:45:15 to 2024-10-22 00:45:20 accepted series 1 / 276 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:45:20.358618 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.88 GiB, free memory in arenas 7.91 MiB, will set to 1.65 GiB (RSS), difference: -90.62 MiB 2024.10.22 00:45:20.586425 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 18404 2024.10.22 00:45:21.182741 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:21.188005 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:21.218233 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 40 entries to flush up to offset 12237 2024.10.22 00:45:21.241277 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} MergedBlockOutputStream: filled checksums 202410_1_39_8 (state Temporary) 2024.10.22 00:45:21.257263 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_326_326_0 (state Temporary) 2024.10.22 00:45:21.315727 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_326_326_0 to 202410_339_339_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:21.334615 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 3 parts from 202410_1_334_68 to 202410_336_336_0 2024.10.22 00:45:21.344780 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:21.347914 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:21.350972 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18404 2024.10.22 00:45:21.364602 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_336_69} MergeTask::PrepareStage: Merging 3 parts: from 202410_1_334_68 to 202410_336_336_0 into Compact with storage Full 2024.10.22 00:45:21.504146 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_336_69} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:45:21.540660 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_336_69} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_334_68, total 18980 rows starting from the beginning of the part 2024.10.22 00:45:21.660292 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.88 GiB, free memory in arenas 8.04 MiB, will set to 1.65 GiB (RSS), difference: -92.69 MiB 2024.10.22 00:45:21.675465 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:21.678624 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:21.679973 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:45:21.695644 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_336_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_335_335_0, total 92 rows starting from the beginning of the part 2024.10.22 00:45:21.730453 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_189_189_0 (state Temporary) 2024.10.22 00:45:21.787472 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_189_189_0 to 202410_195_195_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:21.866318 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12237 2024.10.22 00:45:21.882626 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_336_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_336_336_0, total 56 rows starting from the beginning of the part 2024.10.22 00:45:21.947585 [ 1204 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 4 parts from 202410_1_190_39 to 202410_193_193_0 2024.10.22 00:45:21.965094 [ 1204 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:21.967135 [ 1204 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:21.990532 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_193_40} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_190_39 to 202410_193_193_0 into Compact with storage Full 2024.10.22 00:45:22.056741 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_193_40} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:45:22.164368 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_193_40} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_190_39, total 12019 rows starting from the beginning of the part 2024.10.22 00:45:22.382714 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_39_8/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:45:22.384941 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_193_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_191_191_0, total 38 rows starting from the beginning of the part 2024.10.22 00:45:22.407224 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_193_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_192_192_0, total 38 rows starting from the beginning of the part 2024.10.22 00:45:22.455040 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_193_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_193_193_0, total 156 rows starting from the beginning of the part 2024.10.22 00:45:22.668182 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 316964 2024.10.22 00:45:23.318482 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:23.321362 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:23.386243 [ 1272 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:45:23.408231 [ 1272 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:45:23.473091 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 7.21 MiB, will set to 1.66 GiB (RSS), difference: -20.71 MiB 2024.10.22 00:45:23.680935 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_182_182_0 (state Temporary) 2024.10.22 00:45:23.688435 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_336_69} MergeTask::MergeProjectionsStage: Merge sorted 19128 rows, containing 13 columns (13 merged, 0 gathered) in 2.329560362 sec., 8210.991357862054 rows/sec., 1.22 MiB/sec. 2024.10.22 00:45:23.691856 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_193_40} MergeTask::MergeProjectionsStage: Merge sorted 12251 rows, containing 13 columns (13 merged, 0 gathered) in 1.707191347 sec., 7176.114160564569 rows/sec., 3.01 MiB/sec. 2024.10.22 00:45:23.769369 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_336_69} MergedBlockOutputStream: filled checksums 202410_1_336_69 (state Temporary) 2024.10.22 00:45:23.862520 [ 1159 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 00:45:23.882832 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_182_182_0 to 202410_189_189_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:23.937785 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 316964 2024.10.22 00:45:23.976189 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_193_40} MergedBlockOutputStream: filled checksums 202410_1_193_40 (state Temporary) 2024.10.22 00:45:24.046660 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_336_69} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_336_69 to 202410_1_336_69 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:24.056277 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_336_69} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 3 parts: [202410_1_334_68, 202410_336_336_0] -> 202410_1_336_69 2024.10.22 00:45:24.119786 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_193_40} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_193_40 to 202410_1_193_40 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:24.126361 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_193_40} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 4 parts: [202410_1_190_39, 202410_193_193_0] -> 202410_1_193_40 2024.10.22 00:45:24.173260 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.63 MiB. 2024.10.22 00:45:24.182825 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 171 2024.10.22 00:45:24.266183 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.22 MiB. 2024.10.22 00:45:24.495654 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:24.505750 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:24.603161 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_153_153_0 (state Temporary) 2024.10.22 00:45:24.816561 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_153_153_0 to 202410_163_163_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:24.893638 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 171 2024.10.22 00:45:25.358683 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 18463 2024.10.22 00:45:25.392962 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 2680 2024.10.22 00:45:25.538356 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 7.11 MiB, will set to 1.66 GiB (RSS), difference: 8.74 MiB 2024.10.22 00:45:25.584806 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:25.586651 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:25.587199 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:25.590471 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:25.645326 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_191_191_0 (state Temporary) 2024.10.22 00:45:25.713419 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_191_191_0 to 202410_204_204_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:25.857464 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2680 2024.10.22 00:45:26.058809 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_327_327_0 (state Temporary) 2024.10.22 00:45:26.211696 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_327_327_0 to 202410_340_340_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:26.276516 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18463 2024.10.22 00:45:26.391866 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 1545 2024.10.22 00:45:28.659839 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 6.77 MiB, will set to 1.67 GiB (RSS), difference: -6.19 MiB 2024.10.22 00:45:29.370472 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 72 entries to flush up to offset 12309 2024.10.22 00:45:29.582677 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:45:29.585175 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:45:29.900412 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:29.906774 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:29.953715 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_190_190_0 (state Temporary) 2024.10.22 00:45:30.088168 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_190_190_0 to 202410_196_196_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:30.284189 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 22 entries to flush up to offset 18485 2024.10.22 00:45:30.363745 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12309 2024.10.22 00:45:30.499129 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:30.562406 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:30.584741 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 6.75 MiB, will set to 1.68 GiB (RSS), difference: 4.78 MiB 2024.10.22 00:45:30.613587 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_328_328_0 (state Temporary) 2024.10.22 00:45:30.773743 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_328_328_0 to 202410_341_341_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:30.925613 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 5 parts from 202410_1_336_69 to 202410_340_340_0 2024.10.22 00:45:30.946699 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1685 entries to flush up to offset 318649 2024.10.22 00:45:30.948628 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:31.025740 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18485 2024.10.22 00:45:31.057711 [ 1201 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:31.069963 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_340_70} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_336_69 to 202410_340_340_0 into Compact with storage Full 2024.10.22 00:45:31.215767 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_340_70} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:45:31.243546 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_340_70} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_336_69, total 19128 rows starting from the beginning of the part 2024.10.22 00:45:31.427860 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_340_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_337_337_0, total 31 rows starting from the beginning of the part 2024.10.22 00:45:31.489562 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_340_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_338_338_0, total 43 rows starting from the beginning of the part 2024.10.22 00:45:31.678602 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_340_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_339_339_0, total 33 rows starting from the beginning of the part 2024.10.22 00:45:31.868574 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_340_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_340_340_0, total 59 rows starting from the beginning of the part 2024.10.22 00:45:32.024114 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column NetMinor 2024.10.22 00:45:32.039984 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:32.045556 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:32.059933 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column NetMinor 2024.10.22 00:45:32.091517 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column NetMinor 2024.10.22 00:45:32.125187 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column NetMinor 2024.10.22 00:45:32.173409 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column NetMinor 2024.10.22 00:45:32.224061 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column NetMinor 2024.10.22 00:45:32.266644 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column NetMinor 2024.10.22 00:45:32.282603 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_183_183_0 (state Temporary) 2024.10.22 00:45:32.398513 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/goa/hnxerplqhzmhdlislgkzgtvaxaosw (LogSeriesLimiter: on interval from 2024-10-22 00:45:20 to 2024-10-22 00:45:32 accepted series 1 / 81 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:45:32.402130 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 172 2024.10.22 00:45:32.584856 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_183_183_0 to 202410_190_190_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:32.590549 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:32.592641 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:32.614815 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 7.59 MiB, will set to 1.68 GiB (RSS), difference: -15.05 MiB 2024.10.22 00:45:32.628124 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 318649 2024.10.22 00:45:32.753832 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_154_154_0 (state Temporary) 2024.10.22 00:45:32.966999 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_154_154_0 to 202410_164_164_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:33.071769 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 172 2024.10.22 00:45:33.362425 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 2686 2024.10.22 00:45:33.591820 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:33.594790 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:33.640479 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_192_192_0 (state Temporary) 2024.10.22 00:45:33.684142 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_192_192_0 to 202410_205_205_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:33.759591 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2686 2024.10.22 00:45:33.817424 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_340_70} MergeTask::MergeProjectionsStage: Merge sorted 19294 rows, containing 13 columns (13 merged, 0 gathered) in 2.751433278 sec., 7012.345221769176 rows/sec., 1.04 MiB/sec. 2024.10.22 00:45:33.861543 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_340_70/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:45:33.924382 [ 1230 ] {} DatabaseCatalog: Execution took 292 ms. 2024.10.22 00:45:33.924951 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_340_70} MergedBlockOutputStream: filled checksums 202410_1_340_70 (state Temporary) 2024.10.22 00:45:34.221162 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_340_70} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_340_70 to 202410_1_340_70 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:34.224451 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_340_70} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 5 parts: [202410_1_336_69, 202410_340_340_0] -> 202410_1_340_70 2024.10.22 00:45:34.269225 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.70 MiB. 2024.10.22 00:45:34.488190 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 6.37 MiB, will set to 1.69 GiB (RSS), difference: 13.40 MiB 2024.10.22 00:45:35.062502 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 18538 2024.10.22 00:45:35.182695 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:35.184857 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:35.226389 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_329_329_0 (state Temporary) 2024.10.22 00:45:35.637781 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_329_329_0 to 202410_342_342_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:35.877590 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18538 2024.10.22 00:45:36.331785 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 6.07 MiB, will set to 1.69 GiB (RSS), difference: 3.81 MiB 2024.10.22 00:45:36.473504 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_39_8/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:45:36.578947 [ 1220 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 00:45:37.570778 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_236_47, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0, 202410_241_241_0] 2024.10.22 00:45:37.572404 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_236_47, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0, 202410_241_241_0] 2024.10.22 00:45:37.599003 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_236_47, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0, 202410_241_241_0] 2024.10.22 00:45:37.870608 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 67 entries to flush up to offset 12376 2024.10.22 00:45:38.096122 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:38.099866 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.57 GiB. 2024.10.22 00:45:38.101117 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:45:38.257295 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_191_191_0 (state Temporary) 2024.10.22 00:45:38.287381 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_191_191_0 to 202410_197_197_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:38.318910 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_39_8 to 202410_1_39_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:38.323225 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_39_8} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_34_7, 202410_39_39_0] -> 202410_1_39_8 2024.10.22 00:45:38.417397 [ 1241 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:45:38.447587 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12376 2024.10.22 00:45:38.470399 [ 1241 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:45:39.076210 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 28.62 MiB. 2024.10.22 00:45:39.583904 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:45:39.549394 [ 1227 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 259 ms. 2024.10.22 00:45:39.588849 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:45:39.618185 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 3.29 MiB, will set to 1.70 GiB (RSS), difference: 35.16 MiB 2024.10.22 00:45:39.635897 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1771 entries to flush up to offset 320420 2024.10.22 00:45:39.858996 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_141_28, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0] 2024.10.22 00:45:39.862043 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_141_28, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0] 2024.10.22 00:45:39.905598 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 18569 2024.10.22 00:45:39.957271 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_141_28, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0] 2024.10.22 00:45:40.205543 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:40.212276 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.56 GiB. 2024.10.22 00:45:40.386869 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_330_330_0 (state Temporary) 2024.10.22 00:45:40.469338 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_330_330_0 to 202410_343_343_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:40.552092 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18569 2024.10.22 00:45:40.574004 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 173 2024.10.22 00:45:40.979903 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:40.986456 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.56 GiB. 2024.10.22 00:45:41.169537 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:41.171721 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.56 GiB. 2024.10.22 00:45:41.253266 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_184_184_0 (state Temporary) 2024.10.22 00:45:41.261766 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 2703 2024.10.22 00:45:41.342989 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_155_155_0 (state Temporary) 2024.10.22 00:45:41.358684 [ 1341 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rmf/yslnhhowyadzqzbrnocvlmrtyizoc, size 151 2024.10.22 00:45:41.416041 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_184_184_0 to 202410_191_191_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:41.485459 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 320420 2024.10.22 00:45:41.489330 [ 1258 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_186_37 to 202410_191_191_0 2024.10.22 00:45:41.505054 [ 1258 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:41.508452 [ 1258 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.56 GiB. 2024.10.22 00:45:41.517317 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_155_155_0 to 202410_165_165_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:41.525752 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.88 GiB, free memory in arenas 2.96 MiB, will set to 1.69 GiB (RSS), difference: 128.95 MiB 2024.10.22 00:45:41.532182 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_186_37 to 202410_191_191_0 into Compact with storage Full 2024.10.22 00:45:41.561031 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:41.563759 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.56 GiB. 2024.10.22 00:45:41.597017 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:45:41.606894 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} MergeTreeSequentialSource: Reading 41 marks from part 202410_1_186_37, total 325178 rows starting from the beginning of the part 2024.10.22 00:45:41.609761 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 173 2024.10.22 00:45:41.613748 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_193_193_0 (state Temporary) 2024.10.22 00:45:41.622430 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_187_187_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:45:41.656726 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_188_188_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:45:41.675886 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_189_189_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:45:41.682770 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_190_190_0, total 1685 rows starting from the beginning of the part 2024.10.22 00:45:41.694888 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_193_193_0 to 202410_206_206_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:41.704185 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_191_191_0, total 1771 rows starting from the beginning of the part 2024.10.22 00:45:41.771237 [ 1267 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_201_40 to 202410_206_206_0 2024.10.22 00:45:41.883713 [ 1267 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:41.886762 [ 1267 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.55 GiB. 2024.10.22 00:45:41.892077 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2703 2024.10.22 00:45:42.108634 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_201_40 to 202410_206_206_0 into Compact with storage Full 2024.10.22 00:45:42.417587 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:45:42.466740 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_201_40, total 2739 rows starting from the beginning of the part 2024.10.22 00:45:42.516185 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_202_202_0, total 19 rows starting from the beginning of the part 2024.10.22 00:45:42.562912 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_203_203_0, total 14 rows starting from the beginning of the part 2024.10.22 00:45:42.703258 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_204_204_0, total 14 rows starting from the beginning of the part 2024.10.22 00:45:42.771903 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_205_205_0, total 6 rows starting from the beginning of the part 2024.10.22 00:45:42.852784 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_206_206_0, total 17 rows starting from the beginning of the part 2024.10.22 00:45:43.471490 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_14_14_0 (state Temporary) 2024.10.22 00:45:43.477718 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fkx/vjaekrbzrijgdhuqawssewxvngcew (LogSeriesLimiter: on interval from 2024-10-22 00:45:32 to 2024-10-22 00:45:43 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:45:43.555266 [ 1154 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 00:45:43.555617 [ 1696 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zuw/okdwmuxafpkoihsvgzsqvcflntrja, size 395 2024.10.22 00:45:43.562062 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.88 GiB, free memory in arenas 3.71 MiB, will set to 1.67 GiB (RSS), difference: 175.66 MiB 2024.10.22 00:45:43.579117 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_206_41/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:45:43.681212 [ 1720 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fkx/vjaekrbzrijgdhuqawssewxvngcew, size 4 2024.10.22 00:45:43.782744 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} MergeTask::MergeProjectionsStage: Merge sorted 2809 rows, containing 12 columns (12 merged, 0 gathered) in 1.73167089 sec., 1622.1327136821017 rows/sec., 252.35 KiB/sec. 2024.10.22 00:45:43.818213 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} MergedBlockOutputStream: filled checksums 202410_1_206_41 (state Temporary) 2024.10.22 00:45:43.978442 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_206_41 to 202410_1_206_41 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:43.980919 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_206_41} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_201_40, 202410_206_206_0] -> 202410_1_206_41 2024.10.22 00:45:44.007890 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:45:44.363269 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:44.367654 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:44.559252 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 18631 2024.10.22 00:45:45.181396 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:45.185301 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:45.288889 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_331_331_0 (state Temporary) 2024.10.22 00:45:45.449505 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_331_331_0 to 202410_344_344_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:45.567186 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 8.67 MiB, will set to 1.67 GiB (RSS), difference: 9.33 MiB 2024.10.22 00:45:45.615899 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18631 2024.10.22 00:45:46.066302 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 101 entries to flush up to offset 12477 2024.10.22 00:45:46.275800 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:46.278156 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:46.334132 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_192_192_0 (state Temporary) 2024.10.22 00:45:46.340550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rwr/dfuzwtdqnahonoqtqngfqgfthdsid. 2024.10.22 00:45:46.390371 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_192_192_0 to 202410_198_198_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:46.548411 [ 1272 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_193_40 to 202410_198_198_0 2024.10.22 00:45:46.567152 [ 1272 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:46.569510 [ 1272 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:46.572198 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12477 2024.10.22 00:45:46.582065 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_193_40 to 202410_198_198_0 into Compact with storage Full 2024.10.22 00:45:46.630328 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:45:46.651065 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_193_40, total 12251 rows starting from the beginning of the part 2024.10.22 00:45:46.684873 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_194_194_0, total 50 rows starting from the beginning of the part 2024.10.22 00:45:46.772591 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_191_38/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:45:46.825652 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_195_195_0, total 40 rows starting from the beginning of the part 2024.10.22 00:45:46.868573 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_196_196_0, total 72 rows starting from the beginning of the part 2024.10.22 00:45:46.885920 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_197_197_0, total 67 rows starting from the beginning of the part 2024.10.22 00:45:46.921463 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_198_198_0, total 101 rows starting from the beginning of the part 2024.10.22 00:45:47.689102 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 12.39 MiB, will set to 1.67 GiB (RSS), difference: -16.50 MiB 2024.10.22 00:45:47.770807 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_39_39_0 (state Temporary) 2024.10.22 00:45:47.877642 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_198_41/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:45:48.391402 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} MergeTask::MergeProjectionsStage: Merge sorted 12581 rows, containing 13 columns (13 merged, 0 gathered) in 1.804095843 sec., 6973.576292420957 rows/sec., 2.92 MiB/sec. 2024.10.22 00:45:48.489037 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1888 entries to flush up to offset 322308 2024.10.22 00:45:48.553542 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 00:45:48.577387 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} MergedBlockOutputStream: filled checksums 202410_1_198_41 (state Temporary) 2024.10.22 00:45:48.971863 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_198_41 to 202410_1_198_41 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:48.980671 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_198_41} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_193_40, 202410_198_198_0] -> 202410_1_198_41 2024.10.22 00:45:49.144988 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.37 MiB. 2024.10.22 00:45:49.395691 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 2722 2024.10.22 00:45:49.585522 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:45:49.590644 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:45:49.631730 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 18670 2024.10.22 00:45:49.695615 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:49.699205 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 14.95 MiB, will set to 1.67 GiB (RSS), difference: 15.74 MiB 2024.10.22 00:45:49.699380 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:49.720285 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:49.728680 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:49.830649 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_185_185_0 (state Temporary) 2024.10.22 00:45:49.872342 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:49.877634 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:49.877818 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_194_194_0 (state Temporary) 2024.10.22 00:45:49.933309 [ 1683 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/goa/hnxerplqhzmhdlislgkzgtvaxaosw, size 37171 2024.10.22 00:45:49.943172 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_185_185_0 to 202410_192_192_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:49.977293 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 322308 2024.10.22 00:45:49.985117 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_332_332_0 (state Temporary) 2024.10.22 00:45:50.033711 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_332_332_0 to 202410_345_345_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:50.074556 [ 1167 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 5 parts from 202410_1_340_70 to 202410_344_344_0 2024.10.22 00:45:50.079743 [ 1167 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:50.081423 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_194_194_0 to 202410_207_207_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:50.081423 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:50.091694 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_344_71} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_340_70 to 202410_344_344_0 into Compact with storage Full 2024.10.22 00:45:50.114208 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_344_71} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:45:50.119625 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_344_71} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_340_70, total 19294 rows starting from the beginning of the part 2024.10.22 00:45:50.121411 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2722 2024.10.22 00:45:50.132435 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_344_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_341_341_0, total 22 rows starting from the beginning of the part 2024.10.22 00:45:50.167666 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_344_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_342_342_0, total 53 rows starting from the beginning of the part 2024.10.22 00:45:50.204564 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_344_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_343_343_0, total 31 rows starting from the beginning of the part 2024.10.22 00:45:50.230795 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_344_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_344_344_0, total 62 rows starting from the beginning of the part 2024.10.22 00:45:50.331837 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18670 2024.10.22 00:45:50.478609 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 00:45:50.509026 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 00:45:50.528242 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 00:45:50.661253 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 00:45:50.676031 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 00:45:50.698818 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 00:45:50.742073 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 00:45:50.857148 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ncj/puwvecbvdyowysebtutqtongjaedg (LogSeriesLimiter: on interval from 2024-10-22 00:45:43 to 2024-10-22 00:45:50 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:45:51.344542 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 12.10 MiB, will set to 1.68 GiB (RSS), difference: -11.66 MiB 2024.10.22 00:45:52.341016 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_344_71} MergeTask::MergeProjectionsStage: Merge sorted 19462 rows, containing 13 columns (13 merged, 0 gathered) in 2.253984941 sec., 8634.485371213488 rows/sec., 1.28 MiB/sec. 2024.10.22 00:45:52.494972 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_344_71} MergedBlockOutputStream: filled checksums 202410_1_344_71 (state Temporary) 2024.10.22 00:45:52.524547 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} MergeTask::MergeProjectionsStage: Merge sorted 334010 rows, containing 4 columns (4 merged, 0 gathered) in 10.999378454 sec., 30366.26127529369 rows/sec., 445.60 KiB/sec. 2024.10.22 00:45:52.583084 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} MergedBlockOutputStream: filled checksums 202410_1_191_38 (state Temporary) 2024.10.22 00:45:52.607996 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_191_38 to 202410_1_191_38 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:52.612914 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_191_38} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_186_37, 202410_191_191_0] -> 202410_1_191_38 2024.10.22 00:45:52.674433 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:45:52.683265 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_344_71} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_344_71 to 202410_1_344_71 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:52.685695 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_344_71} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 5 parts: [202410_1_340_70, 202410_344_344_0] -> 202410_1_344_71 2024.10.22 00:45:52.797016 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.70 MiB. 2024.10.22 00:45:53.471737 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 14.37 MiB, will set to 1.67 GiB (RSS), difference: 12.97 MiB 2024.10.22 00:45:53.473120 [ 1203 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:45:53.501327 [ 1203 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:45:54.088978 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 12561 2024.10.22 00:45:54.300223 [ 1199 ] {} DatabaseCatalog: Execution took 229 ms. 2024.10.22 00:45:54.354905 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 18722 2024.10.22 00:45:54.401031 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:54.407599 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:54.592107 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:54.600037 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:54.670749 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_193_193_0 (state Temporary) 2024.10.22 00:45:54.748999 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_333_333_0 (state Temporary) 2024.10.22 00:45:54.750149 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_193_193_0 to 202410_199_199_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:54.785486 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_333_333_0 to 202410_346_346_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:54.798581 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12561 2024.10.22 00:45:54.956997 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18722 2024.10.22 00:45:55.621441 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_39_39_0 to 202410_42_42_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:55.664486 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 12.09 MiB, will set to 1.67 GiB (RSS), difference: -2.67 MiB 2024.10.22 00:45:55.868896 [ 1181 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_161_33 to 202410_165_165_0 2024.10.22 00:45:55.881478 [ 1181 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:55.886785 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:56.011486 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_113_23, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0, 202410_117_117_0, 202410_118_118_0] 2024.10.22 00:45:56.013653 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_113_23, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0, 202410_117_117_0, 202410_118_118_0] 2024.10.22 00:45:56.044170 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_165_34} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_161_33 to 202410_165_165_0 into Compact with storage Full 2024.10.22 00:45:56.157265 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_113_23, 202410_114_114_0, 202410_115_115_0, 202410_116_116_0, 202410_117_117_0, 202410_118_118_0] 2024.10.22 00:45:56.397693 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_165_34} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:45:56.442904 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_165_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_161_33, total 205 rows starting from the beginning of the part 2024.10.22 00:45:56.617836 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_165_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_162_162_0, total 1 rows starting from the beginning of the part 2024.10.22 00:45:56.619450 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 174 2024.10.22 00:45:56.807802 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_165_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_163_163_0, total 1 rows starting from the beginning of the part 2024.10.22 00:45:56.887140 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_165_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_164_164_0, total 1 rows starting from the beginning of the part 2024.10.22 00:45:56.982653 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1568 entries to flush up to offset 323876 2024.10.22 00:45:57.042721 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_165_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_165_165_0, total 1 rows starting from the beginning of the part 2024.10.22 00:45:57.227564 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:57.231457 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:57.295622 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_156_156_0 (state Temporary) 2024.10.22 00:45:57.625241 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 2741 2024.10.22 00:45:57.659721 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 9.00 MiB, will set to 1.67 GiB (RSS), difference: -2.15 MiB 2024.10.22 00:45:57.699717 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_156_156_0 to 202410_166_166_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:57.731760 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:57.749384 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:57.751429 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:57.757456 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:57.779855 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_195_195_0 (state Temporary) 2024.10.22 00:45:57.861439 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_195_195_0 to 202410_208_208_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:57.884063 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 174 2024.10.22 00:45:57.921508 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_186_186_0 (state Temporary) 2024.10.22 00:45:58.040041 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2741 2024.10.22 00:45:58.081376 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_186_186_0 to 202410_193_193_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:58.108944 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 323876 2024.10.22 00:45:58.278210 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1545 2024.10.22 00:45:58.498642 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_135_27, 202410_136_136_0, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0] 2024.10.22 00:45:58.507482 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_135_27, 202410_136_136_0, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0] 2024.10.22 00:45:58.536785 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_135_27, 202410_136_136_0, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0] 2024.10.22 00:45:58.676434 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_165_34} MergeTask::MergeProjectionsStage: Merge sorted 209 rows, containing 32 columns (32 merged, 0 gathered) in 2.752947747 sec., 75.9186222214918 rows/sec., 8.07 KiB/sec. 2024.10.22 00:45:58.829572 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_165_34} MergedBlockOutputStream: filled checksums 202410_1_165_34 (state Temporary) 2024.10.22 00:45:58.967923 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 18773 2024.10.22 00:45:59.387918 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:45:59.391638 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:45:59.401548 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_165_34} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_165_34 to 202410_1_165_34 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:59.411920 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_165_34} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_161_33, 202410_165_165_0] -> 202410_1_165_34 2024.10.22 00:45:59.444970 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_334_334_0 (state Temporary) 2024.10.22 00:45:59.471401 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:45:59.476637 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_334_334_0 to 202410_347_347_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:45:59.511864 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 9.66 MiB, will set to 1.66 GiB (RSS), difference: 2.10 MiB 2024.10.22 00:45:59.523118 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18773 2024.10.22 00:45:59.586199 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:45:59.588472 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:46:00.925760 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_241_48, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 00:46:00.933435 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_241_48, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 00:46:00.970843 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_241_48, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 00:46:01.175490 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:46:01.304074 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_131_26, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0] 2024.10.22 00:46:01.311963 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_131_26, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0] 2024.10.22 00:46:01.448934 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_131_26, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0] 2024.10.22 00:46:02.455225 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 38 entries to flush up to offset 12599 2024.10.22 00:46:02.873095 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:02.875373 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:02.998029 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_194_194_0 (state Temporary) 2024.10.22 00:46:03.146969 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_194_194_0 to 202410_200_200_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:03.183481 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12599 2024.10.22 00:46:03.535420 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 25 entries to flush up to offset 18798 2024.10.22 00:46:03.671900 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:03.674862 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:03.725467 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_335_335_0 (state Temporary) 2024.10.22 00:46:03.789168 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_335_335_0 to 202410_348_348_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:03.858888 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18798 2024.10.22 00:46:04.374945 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for fdcf8b4491799d98da6fb0108d8ff2f8:0) 2024.10.22 00:46:04.379144 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 06e90f4caafd8156c77b38e6e9545ca5, offset: 0, size: 434264 2024.10.22 00:46:04.383159 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 06e90f4caafd8156c77b38e6e9545ca5 for removal 2024.10.22 00:46:04.385513 [ 1476 ] {} CacheMetadata: Key 06e90f4caafd8156c77b38e6e9545ca5 is removed from metadata 2024.10.22 00:46:04.386185 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9b3fe8c29abcb0af9f6103b149f5fff9, offset: 0, size: 152 2024.10.22 00:46:04.388909 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9b3fe8c29abcb0af9f6103b149f5fff9 for removal 2024.10.22 00:46:04.421698 [ 1476 ] {} CacheMetadata: Key 9b3fe8c29abcb0af9f6103b149f5fff9 is removed from metadata 2024.10.22 00:46:04.489739 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 7.76 MiB, will set to 1.66 GiB (RSS), difference: -3.24 MiB 2024.10.22 00:46:05.114892 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1876 entries to flush up to offset 325752 2024.10.22 00:46:05.387029 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 175 2024.10.22 00:46:05.543088 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 2753 2024.10.22 00:46:05.675643 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:05.679818 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:05.679992 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:05.684506 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:05.752605 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_196_196_0 (state Temporary) 2024.10.22 00:46:05.757004 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_157_157_0 (state Temporary) 2024.10.22 00:46:05.819001 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 1584 2024.10.22 00:46:05.850840 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_157_157_0 to 202410_167_167_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:05.984349 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_196_196_0 to 202410_209_209_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:06.085643 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 175 2024.10.22 00:46:06.154458 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2753 2024.10.22 00:46:06.226371 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:06.231889 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:06.416151 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 6.20 MiB, will set to 1.66 GiB (RSS), difference: -5.24 MiB 2024.10.22 00:46:06.468574 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_187_187_0 (state Temporary) 2024.10.22 00:46:06.536908 [ 1274 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 248 ms. 2024.10.22 00:46:06.740711 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_187_187_0 to 202410_194_194_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:06.775148 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 325752 2024.10.22 00:46:07.016559 [ 1698 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ncj/puwvecbvdyowysebtutqtongjaedg, size 1202203 2024.10.22 00:46:07.241375 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 00:46:07.257422 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 00:46:07.268274 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 00:46:07.279975 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 00:46:07.292681 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 00:46:07.351987 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 00:46:07.377696 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 00:46:07.637186 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/abm/ftpigiyfpnnrylmdglzhjewlkdvwd (LogSeriesLimiter: on interval from 2024-10-22 00:45:50 to 2024-10-22 00:46:07 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:46:07.865506 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 18842 2024.10.22 00:46:08.180400 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:08.182314 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:08.343450 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_336_336_0 (state Temporary) 2024.10.22 00:46:08.479968 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_336_336_0 to 202410_349_349_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:08.507145 [ 1159 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 3 parts from 202410_1_344_71 to 202410_346_346_0 2024.10.22 00:46:08.509280 [ 1263 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:46:08.552853 [ 1263 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:46:08.618525 [ 1159 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:08.623987 [ 1159 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:08.667526 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_346_72} MergeTask::PrepareStage: Merging 3 parts: from 202410_1_344_71 to 202410_346_346_0 into Compact with storage Full 2024.10.22 00:46:08.696134 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18842 2024.10.22 00:46:08.697821 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 6.84 MiB, will set to 1.66 GiB (RSS), difference: 2.18 MiB 2024.10.22 00:46:08.755795 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_346_72} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:46:08.770615 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_346_72} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_344_71, total 19462 rows starting from the beginning of the part 2024.10.22 00:46:08.853817 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_346_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_345_345_0, total 39 rows starting from the beginning of the part 2024.10.22 00:46:08.933570 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_346_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_346_346_0, total 52 rows starting from the beginning of the part 2024.10.22 00:46:09.068453 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.057157446 sec. 2024.10.22 00:46:09.585728 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:46:09.586853 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:46:10.689437 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 51 entries to flush up to offset 12650 2024.10.22 00:46:10.697101 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_346_72} MergeTask::MergeProjectionsStage: Merge sorted 19553 rows, containing 13 columns (13 merged, 0 gathered) in 2.03372428 sec., 9614.380962202014 rows/sec., 1.43 MiB/sec. 2024.10.22 00:46:10.859136 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_346_72} MergedBlockOutputStream: filled checksums 202410_1_346_72 (state Temporary) 2024.10.22 00:46:10.932286 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:10.937671 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:10.951057 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_346_72} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_346_72 to 202410_1_346_72 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:10.957643 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_346_72} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 3 parts: [202410_1_344_71, 202410_346_346_0] -> 202410_1_346_72 2024.10.22 00:46:10.986467 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_195_195_0 (state Temporary) 2024.10.22 00:46:11.080187 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.63 MiB. 2024.10.22 00:46:11.133423 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_195_195_0 to 202410_201_201_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:11.172904 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12650 2024.10.22 00:46:12.512665 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 5.75 MiB, will set to 1.66 GiB (RSS), difference: 1.25 MiB 2024.10.22 00:46:12.714882 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 18874 2024.10.22 00:46:13.024607 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:13.028851 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:13.220687 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_337_337_0 (state Temporary) 2024.10.22 00:46:13.568425 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_337_337_0 to 202410_350_350_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:13.587576 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 176 2024.10.22 00:46:13.657568 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 2763 2024.10.22 00:46:13.732862 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18874 2024.10.22 00:46:13.780305 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1580 entries to flush up to offset 327332 2024.10.22 00:46:13.846585 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:13.849492 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:13.878850 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_197_197_0 (state Temporary) 2024.10.22 00:46:13.939322 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_197_197_0 to 202410_210_210_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:14.083066 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2763 2024.10.22 00:46:14.094456 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:14.097288 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:14.228366 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:14.230585 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:14.283667 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_158_158_0 (state Temporary) 2024.10.22 00:46:14.371755 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_158_158_0 to 202410_168_168_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:14.452501 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_188_188_0 (state Temporary) 2024.10.22 00:46:14.455223 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 176 2024.10.22 00:46:14.505589 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_188_188_0 to 202410_195_195_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:14.539845 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 327332 2024.10.22 00:46:14.622222 [ 1203 ] {} DatabaseCatalog: Execution took 220 ms. 2024.10.22 00:46:17.287661 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rwr/dfuzwtdqnahonoqtqngfqgfthdsid. 2024.10.22 00:46:17.741945 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 18900 2024.10.22 00:46:17.982425 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:17.987654 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:18.232738 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_338_338_0 (state Temporary) 2024.10.22 00:46:18.372669 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_338_338_0 to 202410_351_351_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:18.397313 [ 1212 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_346_72 to 202410_347_347_0 2024.10.22 00:46:18.406985 [ 1212 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:18.411885 [ 1212 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:18.438989 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_347_73} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_346_72 to 202410_347_347_0 into Compact with storage Full 2024.10.22 00:46:18.459754 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.88 GiB, free memory in arenas 6.27 MiB, will set to 1.66 GiB (RSS), difference: 76.02 MiB 2024.10.22 00:46:18.578775 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_347_73} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:46:18.590241 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_347_73} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_346_72, total 19553 rows starting from the beginning of the part 2024.10.22 00:46:18.599851 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18900 2024.10.22 00:46:18.619910 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_347_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_347_347_0, total 51 rows starting from the beginning of the part 2024.10.22 00:46:18.676852 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 25 entries to flush up to offset 12675 2024.10.22 00:46:18.834861 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:18.838721 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:18.976240 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_196_196_0 (state Temporary) 2024.10.22 00:46:19.132976 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_196_196_0 to 202410_202_202_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:19.272314 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12675 2024.10.22 00:46:19.587136 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:46:19.590106 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:46:19.604836 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_347_73} MergeTask::MergeProjectionsStage: Merge sorted 19604 rows, containing 13 columns (13 merged, 0 gathered) in 1.174265387 sec., 16694.692883764616 rows/sec., 2.48 MiB/sec. 2024.10.22 00:46:19.773045 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_347_73} MergedBlockOutputStream: filled checksums 202410_1_347_73 (state Temporary) 2024.10.22 00:46:19.949059 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_347_73} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_347_73 to 202410_1_347_73 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:19.959368 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_347_73} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_346_72, 202410_347_347_0] -> 202410_1_347_73 2024.10.22 00:46:20.070799 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.60 MiB. 2024.10.22 00:46:20.527282 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:20.531910 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:20.674683 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.88 GiB, free memory in arenas 5.77 MiB, will set to 1.65 GiB (RSS), difference: 98.75 MiB 2024.10.22 00:46:21.023031 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(ed8f4dca1a53d9bc1a751cbf568b0a0a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ed8f4dca1a53d9bc1a751cbf568b0a0a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 00:46:21.029238 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 633185, file segment info: File segment: [0, 33554431], key: ed8f4dca1a53d9bc1a751cbf568b0a0a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 00:46:21.032264 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:46:21.097805 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_146_29, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:46:21.105542 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_146_29, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:46:21.193404 [ 1724 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/abm/ftpigiyfpnnrylmdglzhjewlkdvwd, size 633185 2024.10.22 00:46:21.228291 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column CookieEnable 2024.10.22 00:46:21.249960 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column CookieEnable 2024.10.22 00:46:21.262706 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_146_29, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:46:21.266959 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column CookieEnable 2024.10.22 00:46:21.278184 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column CookieEnable 2024.10.22 00:46:21.288474 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column CookieEnable 2024.10.22 00:46:21.320629 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column CookieEnable 2024.10.22 00:46:21.333332 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column CookieEnable 2024.10.22 00:46:21.472860 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uam/irzfmglctjozblifrtfptmzuevyiu (LogSeriesLimiter: on interval from 2024-10-22 00:46:07 to 2024-10-22 00:46:21 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:46:21.548108 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 328868 2024.10.22 00:46:21.584549 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 2771 2024.10.22 00:46:21.787686 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:21.792862 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:21.847669 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_198_198_0 (state Temporary) 2024.10.22 00:46:21.958290 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 177 2024.10.22 00:46:22.081507 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c28e39b1e67b1a3e5880b65109db8ae6) : [0, 4409847]: Setting download as failed: File segment: [0, 4409847], key: c28e39b1e67b1a3e5880b65109db8ae6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:22.083875 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c28e39b1e67b1a3e5880b65109db8ae6): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:46:22.084060 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_198_198_0 to 202410_211_211_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:22.220078 [ 1223 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_206_41 to 202410_211_211_0 2024.10.22 00:46:22.237337 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2771 2024.10.22 00:46:22.314889 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:22.347782 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:22.363279 [ 1223 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:22.367209 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:22.387855 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:22.390614 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:22.437663 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_206_41 to 202410_211_211_0 into Compact with storage Full 2024.10.22 00:46:22.459793 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_159_159_0 (state Temporary) 2024.10.22 00:46:22.491944 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c28e39b1e67b1a3e5880b65109db8ae6, offset: 0, size: 0 2024.10.22 00:46:22.496626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c28e39b1e67b1a3e5880b65109db8ae6 for removal 2024.10.22 00:46:22.503864 [ 1476 ] {} CacheMetadata: Key c28e39b1e67b1a3e5880b65109db8ae6 is removed from metadata 2024.10.22 00:46:22.582303 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:46:22.583136 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 5.86 MiB, will set to 1.65 GiB (RSS), difference: -1.76 MiB 2024.10.22 00:46:22.608215 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 18969 2024.10.22 00:46:22.640650 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_159_159_0 to 202410_169_169_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:22.679991 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_189_189_0 (state Temporary) 2024.10.22 00:46:22.680588 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_206_41, total 2809 rows starting from the beginning of the part 2024.10.22 00:46:22.687957 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_207_207_0, total 19 rows starting from the beginning of the part 2024.10.22 00:46:22.732279 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_208_208_0, total 19 rows starting from the beginning of the part 2024.10.22 00:46:22.761456 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 177 2024.10.22 00:46:22.774916 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_209_209_0, total 12 rows starting from the beginning of the part 2024.10.22 00:46:22.783888 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_189_189_0 to 202410_196_196_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:22.783740 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_210_210_0, total 10 rows starting from the beginning of the part 2024.10.22 00:46:22.799510 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_211_211_0, total 8 rows starting from the beginning of the part 2024.10.22 00:46:22.821575 [ 1234 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_191_38 to 202410_196_196_0 2024.10.22 00:46:22.844438 [ 1234 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:22.856548 [ 1234 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:22.920852 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 328868 2024.10.22 00:46:22.924095 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:22.930594 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:22.990447 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_339_339_0 (state Temporary) 2024.10.22 00:46:22.993523 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_191_38 to 202410_196_196_0 into Compact with storage Full 2024.10.22 00:46:23.014883 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:46:23.020557 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} MergeTreeSequentialSource: Reading 42 marks from part 202410_1_191_38, total 334010 rows starting from the beginning of the part 2024.10.22 00:46:23.031602 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_192_192_0, total 1888 rows starting from the beginning of the part 2024.10.22 00:46:23.060981 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_193_193_0, total 1568 rows starting from the beginning of the part 2024.10.22 00:46:23.081102 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_194_194_0, total 1876 rows starting from the beginning of the part 2024.10.22 00:46:23.114428 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_195_195_0, total 1580 rows starting from the beginning of the part 2024.10.22 00:46:23.152570 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_196_196_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:46:23.248253 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_339_339_0 to 202410_352_352_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:23.290925 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_347_73 to 202410_352_352_0 2024.10.22 00:46:23.307268 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:23.311456 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:23.313161 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 18969 2024.10.22 00:46:23.349698 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_347_73 to 202410_352_352_0 into Compact with storage Full 2024.10.22 00:46:23.380196 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:46:23.386097 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_347_73, total 19604 rows starting from the beginning of the part 2024.10.22 00:46:23.453711 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_348_348_0, total 25 rows starting from the beginning of the part 2024.10.22 00:46:23.471435 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_349_349_0, total 44 rows starting from the beginning of the part 2024.10.22 00:46:23.484695 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_350_350_0, total 32 rows starting from the beginning of the part 2024.10.22 00:46:23.514313 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_351_351_0, total 26 rows starting from the beginning of the part 2024.10.22 00:46:23.543067 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_352_352_0, total 69 rows starting from the beginning of the part 2024.10.22 00:46:23.561123 [ 1256 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:46:23.679170 [ 1256 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:46:23.790492 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} MergeTask::MergeProjectionsStage: Merge sorted 2877 rows, containing 12 columns (12 merged, 0 gathered) in 1.361044547 sec., 2113.817660370818 rows/sec., 328.99 KiB/sec. 2024.10.22 00:46:23.884960 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} MergedBlockOutputStream: filled checksums 202410_1_211_42 (state Temporary) 2024.10.22 00:46:24.085509 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_211_42 to 202410_1_211_42 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:24.089029 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_211_42} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_206_41, 202410_211_211_0] -> 202410_1_211_42 2024.10.22 00:46:24.098929 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c3574c63cc43f227c0ce8e5d2b52e468) : [0, 2146313]: Setting download as failed: File segment: [0, 2146313], key: c3574c63cc43f227c0ce8e5d2b52e468, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:24.103479 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c3574c63cc43f227c0ce8e5d2b52e468): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:46:24.147088 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:46:24.373479 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c3574c63cc43f227c0ce8e5d2b52e468, offset: 0, size: 0 2024.10.22 00:46:24.376054 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c3574c63cc43f227c0ce8e5d2b52e468 for removal 2024.10.22 00:46:24.379190 [ 1476 ] {} CacheMetadata: Key c3574c63cc43f227c0ce8e5d2b52e468 is removed from metadata 2024.10.22 00:46:24.423496 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 6.17 MiB, will set to 1.65 GiB (RSS), difference: -4.97 MiB 2024.10.22 00:46:24.476253 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_196_39/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:46:24.774294 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_40_40_0 (state Temporary) 2024.10.22 00:46:24.928988 [ 1251 ] {} DatabaseCatalog: Execution took 315 ms. 2024.10.22 00:46:25.288159 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 72852 bytes for 0017117b445d9ce5b2be5d8135d1a1a0:0) 2024.10.22 00:46:25.290220 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aabb4efee39eed9930dbae0ec4bed3c0, offset: 0, size: 105219 2024.10.22 00:46:25.304491 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aabb4efee39eed9930dbae0ec4bed3c0 for removal 2024.10.22 00:46:25.304956 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_40_40_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:46:25.308618 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 19fc476a4086ebc4a11cadee5bf4c921, offset: 0, size: 155 2024.10.22 00:46:25.308498 [ 1476 ] {} CacheMetadata: Key aabb4efee39eed9930dbae0ec4bed3c0 is removed from metadata 2024.10.22 00:46:25.313787 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 19fc476a4086ebc4a11cadee5bf4c921 for removal 2024.10.22 00:46:25.335578 [ 1476 ] {} CacheMetadata: Key 19fc476a4086ebc4a11cadee5bf4c921 is removed from metadata 2024.10.22 00:46:25.856564 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} MergeTask::MergeProjectionsStage: Merge sorted 19800 rows, containing 13 columns (13 merged, 0 gathered) in 2.513598262 sec., 7877.153759744285 rows/sec., 1.17 MiB/sec. 2024.10.22 00:46:26.122228 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} MergedBlockOutputStream: filled checksums 202410_1_352_74 (state Temporary) 2024.10.22 00:46:26.168014 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_246_49, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0, 202410_250_250_0, 202410_251_251_0] 2024.10.22 00:46:26.170275 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_246_49, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0, 202410_250_250_0, 202410_251_251_0] 2024.10.22 00:46:26.181328 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_352_74 to 202410_1_352_74 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:26.286098 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_352_74} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_347_73, 202410_352_352_0] -> 202410_1_352_74 2024.10.22 00:46:26.350994 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_246_49, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0, 202410_250_250_0, 202410_251_251_0] 2024.10.22 00:46:26.354276 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.73 MiB. 2024.10.22 00:46:26.370486 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(78d5a4383d74d0003080ae103b98d290) : [0, 565533]: Setting download as failed: File segment: [0, 565533], key: 78d5a4383d74d0003080ae103b98d290, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:26.371185 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(78d5a4383d74d0003080ae103b98d290): No space left in cache to reserve 565534 bytes, will continue without cache download 2024.10.22 00:46:26.528813 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 78d5a4383d74d0003080ae103b98d290, offset: 0, size: 0 2024.10.22 00:46:26.533684 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 78d5a4383d74d0003080ae103b98d290 for removal 2024.10.22 00:46:26.543536 [ 1476 ] {} CacheMetadata: Key 78d5a4383d74d0003080ae103b98d290 is removed from metadata 2024.10.22 00:46:26.615002 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 8.43 MiB, will set to 1.66 GiB (RSS), difference: 9.31 MiB 2024.10.22 00:46:26.776468 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 102 entries to flush up to offset 12777 2024.10.22 00:46:26.984545 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:26.986981 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:27.048697 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_197_197_0 (state Temporary) 2024.10.22 00:46:27.122995 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_197_197_0 to 202410_203_203_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:27.181714 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12777 2024.10.22 00:46:27.163967 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_198_41 to 202410_203_203_0 2024.10.22 00:46:27.188437 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:27.192417 [ 1154 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:27.247279 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_198_41 to 202410_203_203_0 into Compact with storage Full 2024.10.22 00:46:27.308537 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3ffec417c0d9904335f9654fb7ee94ab) : [0, 366937]: Setting download as failed: File segment: [0, 366937], key: 3ffec417c0d9904335f9654fb7ee94ab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:27.311539 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3ffec417c0d9904335f9654fb7ee94ab): No space left in cache to reserve 366938 bytes, will continue without cache download 2024.10.22 00:46:27.325600 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 19058 2024.10.22 00:46:27.467621 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3ffec417c0d9904335f9654fb7ee94ab, offset: 0, size: 0 2024.10.22 00:46:27.469045 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3ffec417c0d9904335f9654fb7ee94ab for removal 2024.10.22 00:46:27.471284 [ 1476 ] {} CacheMetadata: Key 3ffec417c0d9904335f9654fb7ee94ab is removed from metadata 2024.10.22 00:46:27.603938 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:27.610285 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:27.632645 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:46:27.649791 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_198_41, total 12581 rows starting from the beginning of the part 2024.10.22 00:46:27.690919 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_199_199_0, total 84 rows starting from the beginning of the part 2024.10.22 00:46:27.771593 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_200_200_0, total 38 rows starting from the beginning of the part 2024.10.22 00:46:27.775257 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_340_340_0 (state Temporary) 2024.10.22 00:46:27.847922 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_340_340_0 to 202410_353_353_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:27.897554 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19058 2024.10.22 00:46:27.919324 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_201_201_0, total 51 rows starting from the beginning of the part 2024.10.22 00:46:28.046823 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_202_202_0, total 25 rows starting from the beginning of the part 2024.10.22 00:46:28.406133 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c97da123611b10dd03360a43471fcfac) : [0, 804403]: Setting download as failed: File segment: [0, 804403], key: c97da123611b10dd03360a43471fcfac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:28.407757 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c97da123611b10dd03360a43471fcfac): No space left in cache to reserve 804404 bytes, will continue without cache download 2024.10.22 00:46:28.424121 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_203_203_0, total 102 rows starting from the beginning of the part 2024.10.22 00:46:28.675792 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c97da123611b10dd03360a43471fcfac, offset: 0, size: 0 2024.10.22 00:46:28.677599 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c97da123611b10dd03360a43471fcfac for removal 2024.10.22 00:46:28.680668 [ 1476 ] {} CacheMetadata: Key c97da123611b10dd03360a43471fcfac is removed from metadata 2024.10.22 00:46:29.473548 [ 1262 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 00:46:29.569151 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 11065 bytes for 866e4cb3ca20c8c210415874ba29ece8:0) 2024.10.22 00:46:29.570069 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: be7fb5b15a72c730b156a1d9b0c0c91b, offset: 0, size: 92527 2024.10.22 00:46:29.574379 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key be7fb5b15a72c730b156a1d9b0c0c91b for removal 2024.10.22 00:46:29.582604 [ 1476 ] {} CacheMetadata: Key be7fb5b15a72c730b156a1d9b0c0c91b is removed from metadata 2024.10.22 00:46:29.584522 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e32b006bce6b527b8b75dfd9871229e1, offset: 0, size: 166 2024.10.22 00:46:29.587726 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:46:29.589622 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e32b006bce6b527b8b75dfd9871229e1 for removal 2024.10.22 00:46:29.591245 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:46:29.593045 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9a6bfc40b04c5396f7a4034dd06fdb70, offset: 0, size: 280 2024.10.22 00:46:29.599781 [ 1476 ] {} CacheMetadata: Key e32b006bce6b527b8b75dfd9871229e1 is removed from metadata 2024.10.22 00:46:29.600689 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9a6bfc40b04c5396f7a4034dd06fdb70 for removal 2024.10.22 00:46:29.616662 [ 1476 ] {} CacheMetadata: Key 9a6bfc40b04c5396f7a4034dd06fdb70 is removed from metadata 2024.10.22 00:46:29.848701 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 2791 2024.10.22 00:46:29.933677 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 330404 2024.10.22 00:46:29.961557 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:29.964334 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:30.082281 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} MergeTask::MergeProjectionsStage: Merge sorted 12881 rows, containing 13 columns (13 merged, 0 gathered) in 2.855389466 sec., 4511.118414275189 rows/sec., 1.89 MiB/sec. 2024.10.22 00:46:30.123643 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_199_199_0 (state Temporary) 2024.10.22 00:46:30.267025 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 178 2024.10.22 00:46:30.285653 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_199_199_0 to 202410_212_212_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:30.339860 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2791 2024.10.22 00:46:30.364369 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} MergedBlockOutputStream: filled checksums 202410_1_203_42 (state Temporary) 2024.10.22 00:46:30.410611 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:30.416486 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:30.464923 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 15.05 MiB, will set to 1.66 GiB (RSS), difference: 2.61 MiB 2024.10.22 00:46:30.521196 [ 1238 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 00:46:30.580489 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_203_42 to 202410_1_203_42 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:30.583153 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_203_42} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_198_41, 202410_203_203_0] -> 202410_1_203_42 2024.10.22 00:46:30.585635 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_160_160_0 (state Temporary) 2024.10.22 00:46:30.622152 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 5 file segments (while reserving 144926 bytes for 8dbd323fd338ce0c386a2ccec639630f:0) 2024.10.22 00:46:30.631022 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0017117b445d9ce5b2be5d8135d1a1a0, offset: 0, size: 72852 2024.10.22 00:46:30.642502 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0017117b445d9ce5b2be5d8135d1a1a0 for removal 2024.10.22 00:46:30.646614 [ 1476 ] {} CacheMetadata: Key 0017117b445d9ce5b2be5d8135d1a1a0 is removed from metadata 2024.10.22 00:46:30.649626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3328ec72c11b1e26aedaa4c6e82346c4, offset: 0, size: 264 2024.10.22 00:46:30.654672 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.62 MiB. 2024.10.22 00:46:30.658454 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3328ec72c11b1e26aedaa4c6e82346c4 for removal 2024.10.22 00:46:30.663026 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fc7269b15a134b170bd0d1c2adec3a4b, offset: 0, size: 185 2024.10.22 00:46:30.665940 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fc7269b15a134b170bd0d1c2adec3a4b for removal 2024.10.22 00:46:30.669779 [ 1476 ] {} CacheMetadata: Key fc7269b15a134b170bd0d1c2adec3a4b is removed from metadata 2024.10.22 00:46:30.669824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3e169c325eb88b62ed0ad5e0d87ef196, offset: 0, size: 4917 2024.10.22 00:46:30.673589 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3e169c325eb88b62ed0ad5e0d87ef196 for removal 2024.10.22 00:46:30.675233 [ 1476 ] {} CacheMetadata: Key 3328ec72c11b1e26aedaa4c6e82346c4 is removed from metadata 2024.10.22 00:46:30.675855 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 952abc79f05f32f4163be1fe176e48b3, offset: 0, size: 183 2024.10.22 00:46:30.678346 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 952abc79f05f32f4163be1fe176e48b3 for removal 2024.10.22 00:46:30.680076 [ 1476 ] {} CacheMetadata: Key 3e169c325eb88b62ed0ad5e0d87ef196 is removed from metadata 2024.10.22 00:46:30.686384 [ 1476 ] {} CacheMetadata: Key 952abc79f05f32f4163be1fe176e48b3 is removed from metadata 2024.10.22 00:46:30.771479 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:30.773582 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:30.811931 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_160_160_0 to 202410_170_170_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:30.883227 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_190_190_0 (state Temporary) 2024.10.22 00:46:30.966152 [ 1171 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_165_34 to 202410_170_170_0 2024.10.22 00:46:30.984121 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_190_190_0 to 202410_197_197_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:30.993003 [ 1171 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:30.997364 [ 1171 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:31.018209 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 178 2024.10.22 00:46:31.040427 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_165_34 to 202410_170_170_0 into Compact with storage Full 2024.10.22 00:46:31.134523 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 330404 2024.10.22 00:46:31.222432 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 9 file segments (while reserving 163275 bytes for acd17b1834bf0293c0f10df823291374:0) 2024.10.22 00:46:31.225523 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8dbd323fd338ce0c386a2ccec639630f, offset: 0, size: 144926 2024.10.22 00:46:31.327053 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:46:31.349114 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8dbd323fd338ce0c386a2ccec639630f for removal 2024.10.22 00:46:31.350884 [ 1476 ] {} CacheMetadata: Key 8dbd323fd338ce0c386a2ccec639630f is removed from metadata 2024.10.22 00:46:31.353023 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d39589a9a45d7e0be9d8511725d9cbf8, offset: 0, size: 167 2024.10.22 00:46:31.359348 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d39589a9a45d7e0be9d8511725d9cbf8 for removal 2024.10.22 00:46:31.364197 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 866e4cb3ca20c8c210415874ba29ece8, offset: 0, size: 11065 2024.10.22 00:46:31.369720 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 866e4cb3ca20c8c210415874ba29ece8 for removal 2024.10.22 00:46:31.370377 [ 1476 ] {} CacheMetadata: Key d39589a9a45d7e0be9d8511725d9cbf8 is removed from metadata 2024.10.22 00:46:31.372361 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 31ecd0fde83c64004dc6f578ff64a281, offset: 0, size: 52220 2024.10.22 00:46:31.374832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 31ecd0fde83c64004dc6f578ff64a281 for removal 2024.10.22 00:46:31.375835 [ 1476 ] {} CacheMetadata: Key 866e4cb3ca20c8c210415874ba29ece8 is removed from metadata 2024.10.22 00:46:31.377268 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7789e67d75ed5f6a1fd405d7a931ffd9, offset: 0, size: 239 2024.10.22 00:46:31.380280 [ 1476 ] {} CacheMetadata: Key 31ecd0fde83c64004dc6f578ff64a281 is removed from metadata 2024.10.22 00:46:31.380237 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7789e67d75ed5f6a1fd405d7a931ffd9 for removal 2024.10.22 00:46:31.383210 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b544118a3c9e9ea68cb88ed90f466611, offset: 0, size: 233 2024.10.22 00:46:31.386531 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b544118a3c9e9ea68cb88ed90f466611 for removal 2024.10.22 00:46:31.388242 [ 1476 ] {} CacheMetadata: Key 7789e67d75ed5f6a1fd405d7a931ffd9 is removed from metadata 2024.10.22 00:46:31.389234 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 56e7a41d901dd0d7f1e5d676802efa7b, offset: 0, size: 163 2024.10.22 00:46:31.396140 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 56e7a41d901dd0d7f1e5d676802efa7b for removal 2024.10.22 00:46:31.396531 [ 1476 ] {} CacheMetadata: Key b544118a3c9e9ea68cb88ed90f466611 is removed from metadata 2024.10.22 00:46:31.413349 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8112d4a2b02645428c26b7a91868bd3a, offset: 0, size: 243 2024.10.22 00:46:31.416037 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8112d4a2b02645428c26b7a91868bd3a for removal 2024.10.22 00:46:31.416867 [ 1476 ] {} CacheMetadata: Key 56e7a41d901dd0d7f1e5d676802efa7b is removed from metadata 2024.10.22 00:46:31.419883 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 36f46b32a703e6f44ec9e75f5d3a0b8c, offset: 0, size: 229 2024.10.22 00:46:31.423914 [ 1476 ] {} CacheMetadata: Key 8112d4a2b02645428c26b7a91868bd3a is removed from metadata 2024.10.22 00:46:31.428316 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 36f46b32a703e6f44ec9e75f5d3a0b8c for removal 2024.10.22 00:46:31.432781 [ 1476 ] {} CacheMetadata: Key 36f46b32a703e6f44ec9e75f5d3a0b8c is removed from metadata 2024.10.22 00:46:31.582362 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_165_34, total 209 rows starting from the beginning of the part 2024.10.22 00:46:31.828336 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_166_166_0, total 1 rows starting from the beginning of the part 2024.10.22 00:46:32.118644 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0a7dc0e9e8fa4ec7982965926d249bde) : [0, 260304]: Setting download as failed: File segment: [0, 260304], key: 0a7dc0e9e8fa4ec7982965926d249bde, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:32.131590 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 110 entries to flush up to offset 19168 2024.10.22 00:46:32.132024 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0a7dc0e9e8fa4ec7982965926d249bde): No space left in cache to reserve 260305 bytes, will continue without cache download 2024.10.22 00:46:32.176999 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0a7dc0e9e8fa4ec7982965926d249bde, offset: 0, size: 0 2024.10.22 00:46:32.181673 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0a7dc0e9e8fa4ec7982965926d249bde for removal 2024.10.22 00:46:32.183819 [ 1476 ] {} CacheMetadata: Key 0a7dc0e9e8fa4ec7982965926d249bde is removed from metadata 2024.10.22 00:46:32.220898 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_167_167_0, total 1 rows starting from the beginning of the part 2024.10.22 00:46:32.344327 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:32.348343 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:32.398472 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_341_341_0 (state Temporary) 2024.10.22 00:46:32.399571 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 14.20 MiB, will set to 1.66 GiB (RSS), difference: 1.31 MiB 2024.10.22 00:46:32.564022 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_341_341_0 to 202410_354_354_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:32.592989 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19168 2024.10.22 00:46:32.601418 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_168_168_0, total 1 rows starting from the beginning of the part 2024.10.22 00:46:32.851553 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(95987864f77364468c8b1c352dfcd4b2) : [0, 251186]: Setting download as failed: File segment: [0, 251186], key: 95987864f77364468c8b1c352dfcd4b2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:32.855657 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(95987864f77364468c8b1c352dfcd4b2): No space left in cache to reserve 251187 bytes, will continue without cache download 2024.10.22 00:46:32.877783 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 95987864f77364468c8b1c352dfcd4b2, offset: 0, size: 0 2024.10.22 00:46:32.879775 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 95987864f77364468c8b1c352dfcd4b2 for removal 2024.10.22 00:46:32.882499 [ 1476 ] {} CacheMetadata: Key 95987864f77364468c8b1c352dfcd4b2 is removed from metadata 2024.10.22 00:46:32.934505 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_169_169_0, total 1 rows starting from the beginning of the part 2024.10.22 00:46:33.065811 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_170_170_0, total 1 rows starting from the beginning of the part 2024.10.22 00:46:33.596476 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 121685 bytes for 084421d4ca34e31079cb47a5b054a8ce:0) 2024.10.22 00:46:33.600220 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: acd17b1834bf0293c0f10df823291374, offset: 0, size: 163275 2024.10.22 00:46:33.606874 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key acd17b1834bf0293c0f10df823291374 for removal 2024.10.22 00:46:33.611489 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a11fe9c4988242541a99fa47b43770d6, offset: 0, size: 168 2024.10.22 00:46:33.612854 [ 1476 ] {} CacheMetadata: Key acd17b1834bf0293c0f10df823291374 is removed from metadata 2024.10.22 00:46:33.619513 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a11fe9c4988242541a99fa47b43770d6 for removal 2024.10.22 00:46:33.728203 [ 1476 ] {} CacheMetadata: Key a11fe9c4988242541a99fa47b43770d6 is removed from metadata 2024.10.22 00:46:34.108445 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} MergeTask::MergeProjectionsStage: Merge sorted 342458 rows, containing 4 columns (4 merged, 0 gathered) in 11.240910722 sec., 30465.325138626253 rows/sec., 447.00 KiB/sec. 2024.10.22 00:46:34.256102 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} MergedBlockOutputStream: filled checksums 202410_1_196_39 (state Temporary) 2024.10.22 00:46:34.356763 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_196_39 to 202410_1_196_39 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:34.364217 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_196_39} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_191_38, 202410_196_196_0] -> 202410_1_196_39 2024.10.22 00:46:34.396153 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:46:34.417279 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 12.08 MiB, will set to 1.66 GiB (RSS), difference: 3.73 MiB 2024.10.22 00:46:34.673029 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 113708 bytes for 248d5432cda7e6046347506782014dd3:0) 2024.10.22 00:46:34.674545 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 084421d4ca34e31079cb47a5b054a8ce, offset: 0, size: 121685 2024.10.22 00:46:34.676483 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 084421d4ca34e31079cb47a5b054a8ce for removal 2024.10.22 00:46:34.678421 [ 1476 ] {} CacheMetadata: Key 084421d4ca34e31079cb47a5b054a8ce is removed from metadata 2024.10.22 00:46:34.678838 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d9e04d11643bc8156897ab90c447e8bd, offset: 0, size: 162 2024.10.22 00:46:34.680978 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d9e04d11643bc8156897ab90c447e8bd for removal 2024.10.22 00:46:34.683127 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ed631c6e040bf34bafcabeb096acd2ee, offset: 0, size: 195 2024.10.22 00:46:34.687886 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ed631c6e040bf34bafcabeb096acd2ee for removal 2024.10.22 00:46:34.689722 [ 1476 ] {} CacheMetadata: Key d9e04d11643bc8156897ab90c447e8bd is removed from metadata 2024.10.22 00:46:34.692605 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e9b43c2469c5602ce81c57108c901a07, offset: 0, size: 195 2024.10.22 00:46:34.697671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e9b43c2469c5602ce81c57108c901a07 for removal 2024.10.22 00:46:34.698642 [ 1476 ] {} CacheMetadata: Key ed631c6e040bf34bafcabeb096acd2ee is removed from metadata 2024.10.22 00:46:34.714156 [ 1476 ] {} CacheMetadata: Key e9b43c2469c5602ce81c57108c901a07 is removed from metadata 2024.10.22 00:46:34.795833 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 76 entries to flush up to offset 12853 2024.10.22 00:46:34.983816 [ 1692 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uam/irzfmglctjozblifrtfptmzuevyiu, size 18211 2024.10.22 00:46:34.985649 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_40_40_0 to 202410_43_43_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:34.988384 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} MergeTask::MergeProjectionsStage: Merge sorted 214 rows, containing 32 columns (32 merged, 0 gathered) in 3.955986157 sec., 54.09523479280466 rows/sec., 5.79 KiB/sec. 2024.10.22 00:46:35.041196 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} MergedBlockOutputStream: filled checksums 202410_1_170_35 (state Temporary) 2024.10.22 00:46:35.059620 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 00:46:35.070347 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 00:46:35.076983 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 00:46:35.100138 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 00:46:35.116053 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 00:46:35.151638 [ 1187 ] {} DatabaseCatalog: Execution took 221 ms. 2024.10.22 00:46:35.231389 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 00:46:35.257789 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:35.261114 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 00:46:35.262821 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:35.390426 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mcd/ojjnnhpojycmdwjlesokiplijqefh (LogSeriesLimiter: on interval from 2024-10-22 00:46:21 to 2024-10-22 00:46:35 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:46:35.413749 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_198_198_0 (state Temporary) 2024.10.22 00:46:35.548444 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_198_198_0 to 202410_204_204_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:35.556623 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:46:35.673647 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_170_35 to 202410_1_170_35 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:35.676789 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_170_35} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_165_34, 202410_170_170_0] -> 202410_1_170_35 2024.10.22 00:46:35.696696 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12853 2024.10.22 00:46:35.699452 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2b3080add6d2381fe3ad7136d09197e3) : [0, 248810]: Setting download as failed: File segment: [0, 248810], key: 2b3080add6d2381fe3ad7136d09197e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:35.703150 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2b3080add6d2381fe3ad7136d09197e3): No space left in cache to reserve 248811 bytes, will continue without cache download 2024.10.22 00:46:35.728966 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:46:35.932915 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2b3080add6d2381fe3ad7136d09197e3, offset: 0, size: 0 2024.10.22 00:46:35.959184 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2b3080add6d2381fe3ad7136d09197e3 for removal 2024.10.22 00:46:35.963885 [ 1476 ] {} CacheMetadata: Key 2b3080add6d2381fe3ad7136d09197e3 is removed from metadata 2024.10.22 00:46:36.610826 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 19243 2024.10.22 00:46:36.883564 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:36.886454 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:37.052160 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_342_342_0 (state Temporary) 2024.10.22 00:46:37.077004 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 59294 bytes for c96dbc4d71eaaf2861fee49126fb1b51:0) 2024.10.22 00:46:37.080727 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 248d5432cda7e6046347506782014dd3, offset: 0, size: 113708 2024.10.22 00:46:37.083868 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 248d5432cda7e6046347506782014dd3 for removal 2024.10.22 00:46:37.085718 [ 1476 ] {} CacheMetadata: Key 248d5432cda7e6046347506782014dd3 is removed from metadata 2024.10.22 00:46:37.086753 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 62892e5fb8335fe1a4c785f773db9210, offset: 0, size: 163 2024.10.22 00:46:37.090513 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 62892e5fb8335fe1a4c785f773db9210 for removal 2024.10.22 00:46:37.109559 [ 1476 ] {} CacheMetadata: Key 62892e5fb8335fe1a4c785f773db9210 is removed from metadata 2024.10.22 00:46:37.202271 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_342_342_0 to 202410_355_355_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:37.262456 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19243 2024.10.22 00:46:37.845464 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 2819 2024.10.22 00:46:38.053316 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:38.059203 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:38.089614 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_200_200_0 (state Temporary) 2024.10.22 00:46:38.139823 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 331940 2024.10.22 00:46:38.146295 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_200_200_0 to 202410_213_213_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:38.279231 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2819 2024.10.22 00:46:38.385359 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7962e7a4417cbf9efddf78e895cd6393) : [0, 428703]: Setting download as failed: File segment: [0, 428703], key: 7962e7a4417cbf9efddf78e895cd6393, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:38.387148 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7962e7a4417cbf9efddf78e895cd6393): No space left in cache to reserve 428704 bytes, will continue without cache download 2024.10.22 00:46:38.423910 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7962e7a4417cbf9efddf78e895cd6393, offset: 0, size: 0 2024.10.22 00:46:38.505739 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 9.14 MiB, will set to 1.65 GiB (RSS), difference: 1.07 MiB 2024.10.22 00:46:38.495470 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1584 2024.10.22 00:46:38.527365 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7962e7a4417cbf9efddf78e895cd6393 for removal 2024.10.22 00:46:38.534010 [ 1476 ] {} CacheMetadata: Key 7962e7a4417cbf9efddf78e895cd6393 is removed from metadata 2024.10.22 00:46:38.684222 [ 1207 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:46:38.711339 [ 1207 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:46:39.193239 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:39.199162 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:39.250806 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(816a0a2ac7db6953cb323b4a4490ac49) : [0, 268327]: Setting download as failed: File segment: [0, 268327], key: 816a0a2ac7db6953cb323b4a4490ac49, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:39.254760 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(816a0a2ac7db6953cb323b4a4490ac49): No space left in cache to reserve 268328 bytes, will continue without cache download 2024.10.22 00:46:39.372009 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 816a0a2ac7db6953cb323b4a4490ac49, offset: 0, size: 0 2024.10.22 00:46:39.373016 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 816a0a2ac7db6953cb323b4a4490ac49 for removal 2024.10.22 00:46:39.375003 [ 1476 ] {} CacheMetadata: Key 816a0a2ac7db6953cb323b4a4490ac49 is removed from metadata 2024.10.22 00:46:39.397076 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_191_191_0 (state Temporary) 2024.10.22 00:46:39.508766 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_191_191_0 to 202410_198_198_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:39.546368 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 331940 2024.10.22 00:46:39.557320 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_140_28, 202410_141_141_0, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0] 2024.10.22 00:46:39.560722 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_140_28, 202410_141_141_0, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0] 2024.10.22 00:46:39.579328 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_140_28, 202410_141_141_0, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0] 2024.10.22 00:46:39.588406 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:46:39.593662 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:46:40.194090 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 9 file segments (while reserving 151173 bytes for 7c492daabe232dbfb5720f6966de5a30:0) 2024.10.22 00:46:40.203296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c96dbc4d71eaaf2861fee49126fb1b51, offset: 0, size: 59294 2024.10.22 00:46:40.209803 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c96dbc4d71eaaf2861fee49126fb1b51 for removal 2024.10.22 00:46:40.215110 [ 1476 ] {} CacheMetadata: Key c96dbc4d71eaaf2861fee49126fb1b51 is removed from metadata 2024.10.22 00:46:40.217079 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 02e7d1f7f426da6aafff4cea6980dcca, offset: 0, size: 161 2024.10.22 00:46:40.224837 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 02e7d1f7f426da6aafff4cea6980dcca for removal 2024.10.22 00:46:40.230910 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e027434b32305743dae50766274ce2e6, offset: 0, size: 5062 2024.10.22 00:46:40.237333 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e027434b32305743dae50766274ce2e6 for removal 2024.10.22 00:46:40.242488 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3dea6e12e3651d3b6fa5c3fec3c41128, offset: 0, size: 162 2024.10.22 00:46:40.242737 [ 1476 ] {} CacheMetadata: Key e027434b32305743dae50766274ce2e6 is removed from metadata 2024.10.22 00:46:40.246491 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3dea6e12e3651d3b6fa5c3fec3c41128 for removal 2024.10.22 00:46:40.250450 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 86e12728ed718d53cf07ce31b71f144b, offset: 0, size: 4680 2024.10.22 00:46:40.355810 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 86e12728ed718d53cf07ce31b71f144b for removal 2024.10.22 00:46:40.356289 [ 1476 ] {} CacheMetadata: Key 3dea6e12e3651d3b6fa5c3fec3c41128 is removed from metadata 2024.10.22 00:46:40.361385 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 26892290d32dbc00ccb46c7059b39b1b, offset: 0, size: 163 2024.10.22 00:46:40.364127 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 26892290d32dbc00ccb46c7059b39b1b for removal 2024.10.22 00:46:40.365343 [ 1476 ] {} CacheMetadata: Key 86e12728ed718d53cf07ce31b71f144b is removed from metadata 2024.10.22 00:46:40.367011 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c609cf264c36fbb25c6e89b731ccf41f, offset: 0, size: 195 2024.10.22 00:46:40.370352 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c609cf264c36fbb25c6e89b731ccf41f for removal 2024.10.22 00:46:40.371664 [ 1476 ] {} CacheMetadata: Key 26892290d32dbc00ccb46c7059b39b1b is removed from metadata 2024.10.22 00:46:40.372880 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 15a771db1c5f460fb2b5497ef41be1e2, offset: 0, size: 70007 2024.10.22 00:46:40.376000 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 15a771db1c5f460fb2b5497ef41be1e2 for removal 2024.10.22 00:46:40.377193 [ 1476 ] {} CacheMetadata: Key c609cf264c36fbb25c6e89b731ccf41f is removed from metadata 2024.10.22 00:46:40.379741 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 16b1b7ef973b9153456f936096631258, offset: 0, size: 162 2024.10.22 00:46:40.381686 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 16b1b7ef973b9153456f936096631258 for removal 2024.10.22 00:46:40.390688 [ 1476 ] {} CacheMetadata: Key 16b1b7ef973b9153456f936096631258 is removed from metadata 2024.10.22 00:46:40.403069 [ 1476 ] {} CacheMetadata: Key 15a771db1c5f460fb2b5497ef41be1e2 is removed from metadata 2024.10.22 00:46:40.434903 [ 1476 ] {} CacheMetadata: Key 02e7d1f7f426da6aafff4cea6980dcca is removed from metadata 2024.10.22 00:46:40.849778 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 190450 bytes for bccfe88190f99df8785938729111f8f2:0) 2024.10.22 00:46:40.854364 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7c492daabe232dbfb5720f6966de5a30, offset: 0, size: 151173 2024.10.22 00:46:40.858612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7c492daabe232dbfb5720f6966de5a30 for removal 2024.10.22 00:46:40.862557 [ 1476 ] {} CacheMetadata: Key 7c492daabe232dbfb5720f6966de5a30 is removed from metadata 2024.10.22 00:46:40.863854 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 122cd577d1c2487d956f6689d63b2925, offset: 0, size: 55127 2024.10.22 00:46:40.868996 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 122cd577d1c2487d956f6689d63b2925 for removal 2024.10.22 00:46:40.871023 [ 1476 ] {} CacheMetadata: Key 122cd577d1c2487d956f6689d63b2925 is removed from metadata 2024.10.22 00:46:40.871358 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0648aaf8ad03cdad5b817d0ca8ca05e8, offset: 0, size: 171 2024.10.22 00:46:40.877009 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0648aaf8ad03cdad5b817d0ca8ca05e8 for removal 2024.10.22 00:46:40.879761 [ 1476 ] {} CacheMetadata: Key 0648aaf8ad03cdad5b817d0ca8ca05e8 is removed from metadata 2024.10.22 00:46:40.880287 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 42aaf400f3ec0ac36828a1064f80c234, offset: 0, size: 237 2024.10.22 00:46:40.883334 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 42aaf400f3ec0ac36828a1064f80c234 for removal 2024.10.22 00:46:40.888403 [ 1476 ] {} CacheMetadata: Key 42aaf400f3ec0ac36828a1064f80c234 is removed from metadata 2024.10.22 00:46:40.888440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c3654ec61833441ba22e65d1c53a3bca, offset: 0, size: 192 2024.10.22 00:46:40.893391 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c3654ec61833441ba22e65d1c53a3bca for removal 2024.10.22 00:46:40.901552 [ 1476 ] {} CacheMetadata: Key c3654ec61833441ba22e65d1c53a3bca is removed from metadata 2024.10.22 00:46:40.897393 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cb8ea65c847579a214f7cabb630cd5f5, offset: 0, size: 163 2024.10.22 00:46:41.007546 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cb8ea65c847579a214f7cabb630cd5f5 for removal 2024.10.22 00:46:41.011301 [ 1476 ] {} CacheMetadata: Key cb8ea65c847579a214f7cabb630cd5f5 is removed from metadata 2024.10.22 00:46:41.282844 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 91 entries to flush up to offset 19334 2024.10.22 00:46:41.713315 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:41.724792 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:41.877267 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(84613112aae0ea95387388c355b2e5fb) : [0, 375312]: Setting download as failed: File segment: [0, 375312], key: 84613112aae0ea95387388c355b2e5fb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:41.878246 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(84613112aae0ea95387388c355b2e5fb): No space left in cache to reserve 375313 bytes, will continue without cache download 2024.10.22 00:46:41.900678 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_343_343_0 (state Temporary) 2024.10.22 00:46:41.926759 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 84613112aae0ea95387388c355b2e5fb, offset: 0, size: 0 2024.10.22 00:46:41.929723 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 84613112aae0ea95387388c355b2e5fb for removal 2024.10.22 00:46:41.935201 [ 1476 ] {} CacheMetadata: Key 84613112aae0ea95387388c355b2e5fb is removed from metadata 2024.10.22 00:46:42.177656 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_343_343_0 to 202410_356_356_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:42.332284 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19334 2024.10.22 00:46:42.767969 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e54137c9fc8d580c4738447937aef0c7) : [0, 377120]: Setting download as failed: File segment: [0, 377120], key: e54137c9fc8d580c4738447937aef0c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:42.769776 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e54137c9fc8d580c4738447937aef0c7): No space left in cache to reserve 377121 bytes, will continue without cache download 2024.10.22 00:46:42.789771 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e54137c9fc8d580c4738447937aef0c7, offset: 0, size: 0 2024.10.22 00:46:42.893779 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e54137c9fc8d580c4738447937aef0c7 for removal 2024.10.22 00:46:43.000463 [ 1476 ] {} CacheMetadata: Key e54137c9fc8d580c4738447937aef0c7 is removed from metadata 2024.10.22 00:46:43.217161 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 26 entries to flush up to offset 12879 2024.10.22 00:46:43.517547 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:43.520823 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:43.559506 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_199_199_0 (state Temporary) 2024.10.22 00:46:43.601693 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_199_199_0 to 202410_205_205_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:43.750232 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12879 2024.10.22 00:46:43.850481 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 176821 bytes for e0e53fac05d0b064c34fcfc6328165af:0) 2024.10.22 00:46:43.853689 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bccfe88190f99df8785938729111f8f2, offset: 0, size: 190450 2024.10.22 00:46:43.858211 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bccfe88190f99df8785938729111f8f2 for removal 2024.10.22 00:46:43.862315 [ 1476 ] {} CacheMetadata: Key bccfe88190f99df8785938729111f8f2 is removed from metadata 2024.10.22 00:46:43.864000 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3f9e23270cca7d72187ba4f751185451, offset: 0, size: 169 2024.10.22 00:46:43.867903 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3f9e23270cca7d72187ba4f751185451 for removal 2024.10.22 00:46:43.870139 [ 1476 ] {} CacheMetadata: Key 3f9e23270cca7d72187ba4f751185451 is removed from metadata 2024.10.22 00:46:44.486537 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f3d8b36f2c41d855866450035b097702) : [0, 2177824]: Setting download as failed: File segment: [0, 2177824], key: f3d8b36f2c41d855866450035b097702, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:44.489197 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f3d8b36f2c41d855866450035b097702): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:46:44.532359 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f3d8b36f2c41d855866450035b097702, offset: 0, size: 0 2024.10.22 00:46:44.534080 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f3d8b36f2c41d855866450035b097702 for removal 2024.10.22 00:46:44.539802 [ 1476 ] {} CacheMetadata: Key f3d8b36f2c41d855866450035b097702 is removed from metadata 2024.10.22 00:46:44.681069 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_118_24, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0, 202410_122_122_0, 202410_123_123_0] 2024.10.22 00:46:44.684370 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_118_24, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0, 202410_122_122_0, 202410_123_123_0] 2024.10.22 00:46:44.772546 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_136_27, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0] 2024.10.22 00:46:44.773599 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_136_27, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0] 2024.10.22 00:46:44.823449 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_118_24, 202410_119_119_0, 202410_120_120_0, 202410_121_121_0, 202410_122_122_0, 202410_123_123_0] 2024.10.22 00:46:44.893332 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_136_27, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0] 2024.10.22 00:46:44.992795 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 111928 bytes for ac58111b25f935a5397acf55e7daceb8:0) 2024.10.22 00:46:44.995008 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e0e53fac05d0b064c34fcfc6328165af, offset: 0, size: 176821 2024.10.22 00:46:45.001410 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e0e53fac05d0b064c34fcfc6328165af for removal 2024.10.22 00:46:45.010628 [ 1476 ] {} CacheMetadata: Key e0e53fac05d0b064c34fcfc6328165af is removed from metadata 2024.10.22 00:46:45.011101 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c2c1a2fc86d1b3158131fb13edf1b1b3, offset: 0, size: 194 2024.10.22 00:46:45.015102 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c2c1a2fc86d1b3158131fb13edf1b1b3 for removal 2024.10.22 00:46:45.017250 [ 1476 ] {} CacheMetadata: Key c2c1a2fc86d1b3158131fb13edf1b1b3 is removed from metadata 2024.10.22 00:46:45.017639 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 88255dbb97a91a36509951b8c2b5a88d, offset: 0, size: 196 2024.10.22 00:46:45.020812 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 88255dbb97a91a36509951b8c2b5a88d for removal 2024.10.22 00:46:45.024013 [ 1476 ] {} CacheMetadata: Key 88255dbb97a91a36509951b8c2b5a88d is removed from metadata 2024.10.22 00:46:45.025225 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ce8d79bf865ab069f4f1e3885206094, offset: 0, size: 198 2024.10.22 00:46:45.030800 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ce8d79bf865ab069f4f1e3885206094 for removal 2024.10.22 00:46:45.035451 [ 1476 ] {} CacheMetadata: Key 7ce8d79bf865ab069f4f1e3885206094 is removed from metadata 2024.10.22 00:46:45.783366 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 4 entries to flush up to offset 2823 2024.10.22 00:46:45.865804 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:45.867906 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:45.888934 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_201_201_0 (state Temporary) 2024.10.22 00:46:45.925186 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 5 file segments (while reserving 187006 bytes for dfeee55c7b335ae11e9f375a9e14cd2b:0) 2024.10.22 00:46:45.933340 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e9d3253aacb50c3098ae7224a6eeafeb, offset: 0, size: 98105 2024.10.22 00:46:45.939240 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e9d3253aacb50c3098ae7224a6eeafeb for removal 2024.10.22 00:46:45.947419 [ 1476 ] {} CacheMetadata: Key e9d3253aacb50c3098ae7224a6eeafeb is removed from metadata 2024.10.22 00:46:45.949019 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d531bf9426d1066a8a49203d187cea2c, offset: 0, size: 163 2024.10.22 00:46:45.954291 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d531bf9426d1066a8a49203d187cea2c for removal 2024.10.22 00:46:45.958414 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ac58111b25f935a5397acf55e7daceb8, offset: 0, size: 111928 2024.10.22 00:46:45.962652 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ac58111b25f935a5397acf55e7daceb8 for removal 2024.10.22 00:46:45.966120 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aa883e5dffdec69b9107ecbd1e6d3f5d, offset: 0, size: 164 2024.10.22 00:46:45.968959 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aa883e5dffdec69b9107ecbd1e6d3f5d for removal 2024.10.22 00:46:45.969064 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_201_201_0 to 202410_214_214_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:45.972932 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 43e3f7ea670662de6f546ec26a2d6af1, offset: 0, size: 264 2024.10.22 00:46:46.031774 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 179 2024.10.22 00:46:46.035615 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 1625 2024.10.22 00:46:46.058244 [ 1476 ] {} CacheMetadata: Key aa883e5dffdec69b9107ecbd1e6d3f5d is removed from metadata 2024.10.22 00:46:46.073541 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 43e3f7ea670662de6f546ec26a2d6af1 for removal 2024.10.22 00:46:46.079903 [ 1476 ] {} CacheMetadata: Key 43e3f7ea670662de6f546ec26a2d6af1 is removed from metadata 2024.10.22 00:46:46.088069 [ 1476 ] {} CacheMetadata: Key ac58111b25f935a5397acf55e7daceb8 is removed from metadata 2024.10.22 00:46:46.104601 [ 1476 ] {} CacheMetadata: Key d531bf9426d1066a8a49203d187cea2c is removed from metadata 2024.10.22 00:46:46.113368 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2823 2024.10.22 00:46:46.315171 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:46.317769 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:46.347453 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 19413 2024.10.22 00:46:46.380116 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_161_161_0 (state Temporary) 2024.10.22 00:46:46.550958 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_161_161_0 to 202410_171_171_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:46.552149 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 333860 2024.10.22 00:46:46.617675 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 179 2024.10.22 00:46:46.688821 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:46.693078 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:46.754608 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_344_344_0 (state Temporary) 2024.10.22 00:46:46.800394 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(86dcc163498ad8cb1578d8f59cdb5b69) : [0, 278499]: Setting download as failed: File segment: [0, 278499], key: 86dcc163498ad8cb1578d8f59cdb5b69, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:46.802338 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_344_344_0 to 202410_357_357_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:46.804834 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(86dcc163498ad8cb1578d8f59cdb5b69): No space left in cache to reserve 278500 bytes, will continue without cache download 2024.10.22 00:46:46.945457 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_352_74 to 202410_357_357_0 2024.10.22 00:46:46.954339 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 86dcc163498ad8cb1578d8f59cdb5b69, offset: 0, size: 0 2024.10.22 00:46:46.957256 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 86dcc163498ad8cb1578d8f59cdb5b69 for removal 2024.10.22 00:46:46.960353 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19413 2024.10.22 00:46:46.964366 [ 1476 ] {} CacheMetadata: Key 86dcc163498ad8cb1578d8f59cdb5b69 is removed from metadata 2024.10.22 00:46:46.973266 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:46.976080 [ 1253 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:47.002974 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_352_74 to 202410_357_357_0 into Compact with storage Full 2024.10.22 00:46:47.181032 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:46:47.188848 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_352_74, total 19800 rows starting from the beginning of the part 2024.10.22 00:46:47.228379 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_353_353_0, total 89 rows starting from the beginning of the part 2024.10.22 00:46:47.470885 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_354_354_0, total 110 rows starting from the beginning of the part 2024.10.22 00:46:47.535609 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_355_355_0, total 75 rows starting from the beginning of the part 2024.10.22 00:46:47.551982 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 8.17 MiB, will set to 1.64 GiB (RSS), difference: -2.02 MiB 2024.10.22 00:46:47.574226 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_356_356_0, total 91 rows starting from the beginning of the part 2024.10.22 00:46:47.609180 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:47.613591 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:46:47.615187 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_357_357_0, total 79 rows starting from the beginning of the part 2024.10.22 00:46:47.628409 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 185137 bytes for dc75ae18f869f00cfa64cb63cafe9c0d:0) 2024.10.22 00:46:47.632075 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dfeee55c7b335ae11e9f375a9e14cd2b, offset: 0, size: 187006 2024.10.22 00:46:47.638223 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dfeee55c7b335ae11e9f375a9e14cd2b for removal 2024.10.22 00:46:47.641510 [ 1476 ] {} CacheMetadata: Key dfeee55c7b335ae11e9f375a9e14cd2b is removed from metadata 2024.10.22 00:46:47.644218 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0396a79f566164c68fef2aa01213f19f, offset: 0, size: 230 2024.10.22 00:46:47.751853 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0396a79f566164c68fef2aa01213f19f for removal 2024.10.22 00:46:47.758945 [ 1476 ] {} CacheMetadata: Key 0396a79f566164c68fef2aa01213f19f is removed from metadata 2024.10.22 00:46:48.068171 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_251_50, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0] 2024.10.22 00:46:48.069507 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_251_50, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0] 2024.10.22 00:46:48.103682 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_192_192_0 (state Temporary) 2024.10.22 00:46:48.114585 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_251_50, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0] 2024.10.22 00:46:48.151885 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(53ef7020d92d37da2e3e7fb5db83b8c1) : [0, 629559]: Setting download as failed: File segment: [0, 629559], key: 53ef7020d92d37da2e3e7fb5db83b8c1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:48.154999 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(53ef7020d92d37da2e3e7fb5db83b8c1): No space left in cache to reserve 629560 bytes, will continue without cache download 2024.10.22 00:46:48.289069 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 53ef7020d92d37da2e3e7fb5db83b8c1, offset: 0, size: 0 2024.10.22 00:46:48.291556 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 53ef7020d92d37da2e3e7fb5db83b8c1 for removal 2024.10.22 00:46:48.297096 [ 1476 ] {} CacheMetadata: Key 53ef7020d92d37da2e3e7fb5db83b8c1 is removed from metadata 2024.10.22 00:46:48.312241 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_192_192_0 to 202410_199_199_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:48.564834 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 333860 2024.10.22 00:46:48.815292 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ea428303a611fed64845c1f8d0a61fcb) : [0, 1984713]: Setting download as failed: File segment: [0, 1984713], key: ea428303a611fed64845c1f8d0a61fcb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:48.816984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ea428303a611fed64845c1f8d0a61fcb): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:46:49.072309 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ea428303a611fed64845c1f8d0a61fcb, offset: 0, size: 0 2024.10.22 00:46:49.074218 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ea428303a611fed64845c1f8d0a61fcb for removal 2024.10.22 00:46:49.076204 [ 1476 ] {} CacheMetadata: Key ea428303a611fed64845c1f8d0a61fcb is removed from metadata 2024.10.22 00:46:49.521505 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 202180 bytes for 1b7d7319e6f158d044ae41a31fca7e85:0) 2024.10.22 00:46:49.527709 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6733c72de2b2a04f538e78c2611d744e, offset: 0, size: 166 2024.10.22 00:46:49.535694 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6733c72de2b2a04f538e78c2611d744e for removal 2024.10.22 00:46:49.541073 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dc75ae18f869f00cfa64cb63cafe9c0d, offset: 0, size: 185137 2024.10.22 00:46:49.541453 [ 1476 ] {} CacheMetadata: Key 6733c72de2b2a04f538e78c2611d744e is removed from metadata 2024.10.22 00:46:49.550474 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dc75ae18f869f00cfa64cb63cafe9c0d for removal 2024.10.22 00:46:49.554599 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 75a5a7401d0592c3528a662edc4be8b3, offset: 0, size: 237 2024.10.22 00:46:49.563223 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 75a5a7401d0592c3528a662edc4be8b3 for removal 2024.10.22 00:46:49.589542 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:46:49.594910 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:46:49.655653 [ 1476 ] {} CacheMetadata: Key 75a5a7401d0592c3528a662edc4be8b3 is removed from metadata 2024.10.22 00:46:49.675117 [ 1476 ] {} CacheMetadata: Key dc75ae18f869f00cfa64cb63cafe9c0d is removed from metadata 2024.10.22 00:46:49.683626 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 7.60 MiB, will set to 1.65 GiB (RSS), difference: 1.75 MiB 2024.10.22 00:46:49.852743 [ 1245 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 00:46:50.044976 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} MergeTask::MergeProjectionsStage: Merge sorted 20244 rows, containing 13 columns (13 merged, 0 gathered) in 3.064764075 sec., 6605.402407687777 rows/sec., 1007.47 KiB/sec. 2024.10.22 00:46:50.098587 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} MergedBlockOutputStream: filled checksums 202410_1_357_75 (state Temporary) 2024.10.22 00:46:50.160711 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_357_75 to 202410_1_357_75 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:50.165464 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_357_75} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_352_74, 202410_357_357_0] -> 202410_1_357_75 2024.10.22 00:46:50.186710 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.75 MiB. 2024.10.22 00:46:50.494474 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 24042 bytes for 4b1fbf802cc14b0fe3c29ad58b5118f6:0) 2024.10.22 00:46:50.497439 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1b7d7319e6f158d044ae41a31fca7e85, offset: 0, size: 202180 2024.10.22 00:46:50.507311 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1b7d7319e6f158d044ae41a31fca7e85 for removal 2024.10.22 00:46:50.510736 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 37b3e993e33257c8784af0060ef77b7e, offset: 0, size: 167 2024.10.22 00:46:50.511506 [ 1476 ] {} CacheMetadata: Key 1b7d7319e6f158d044ae41a31fca7e85 is removed from metadata 2024.10.22 00:46:50.514575 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 37b3e993e33257c8784af0060ef77b7e for removal 2024.10.22 00:46:50.518353 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2965cbc615d84df43ed3eb3ac158ac2e, offset: 0, size: 263 2024.10.22 00:46:50.518929 [ 1476 ] {} CacheMetadata: Key 37b3e993e33257c8784af0060ef77b7e is removed from metadata 2024.10.22 00:46:50.527140 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2965cbc615d84df43ed3eb3ac158ac2e for removal 2024.10.22 00:46:50.533812 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ff56bd0740a9f7070a682abde2e4cbae, offset: 0, size: 228 2024.10.22 00:46:50.535122 [ 1476 ] {} CacheMetadata: Key 2965cbc615d84df43ed3eb3ac158ac2e is removed from metadata 2024.10.22 00:46:50.538956 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ff56bd0740a9f7070a682abde2e4cbae for removal 2024.10.22 00:46:50.546861 [ 1476 ] {} CacheMetadata: Key ff56bd0740a9f7070a682abde2e4cbae is removed from metadata 2024.10.22 00:46:50.969089 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 19497 2024.10.22 00:46:50.986269 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c1c35c66b2a9bbd6b8e3515373faf842) : [0, 2153402]: Setting download as failed: File segment: [0, 2153402], key: c1c35c66b2a9bbd6b8e3515373faf842, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:50.988029 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c1c35c66b2a9bbd6b8e3515373faf842): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:46:51.039552 [ 1748 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mcd/ojjnnhpojycmdwjlesokiplijqefh, size 24042 2024.10.22 00:46:51.067885 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c1c35c66b2a9bbd6b8e3515373faf842, offset: 0, size: 0 2024.10.22 00:46:51.071371 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c1c35c66b2a9bbd6b8e3515373faf842 for removal 2024.10.22 00:46:51.073391 [ 1476 ] {} CacheMetadata: Key c1c35c66b2a9bbd6b8e3515373faf842 is removed from metadata 2024.10.22 00:46:51.193048 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:51.198439 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:51.258921 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 12929 2024.10.22 00:46:51.316183 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_345_345_0 (state Temporary) 2024.10.22 00:46:51.373628 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_345_345_0 to 202410_358_358_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:51.436258 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19497 2024.10.22 00:46:51.492913 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:51.496588 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:51.647845 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_200_200_0 (state Temporary) 2024.10.22 00:46:51.748627 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsMobile 2024.10.22 00:46:51.757782 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsMobile 2024.10.22 00:46:51.773605 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsMobile 2024.10.22 00:46:51.785320 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsMobile 2024.10.22 00:46:51.812680 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsMobile 2024.10.22 00:46:51.832706 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsMobile 2024.10.22 00:46:51.919952 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_200_200_0 to 202410_206_206_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:51.961268 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsMobile 2024.10.22 00:46:51.983590 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12929 2024.10.22 00:46:51.985554 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mye/vnuaqwvmcmrltvzzpbgubgdcxleio (LogSeriesLimiter: on interval from 2024-10-22 00:46:35 to 2024-10-22 00:46:51 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:46:52.265284 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 5 file segments (while reserving 141224 bytes for c9c580d492b80df5b93f8066c30a74ed:0) 2024.10.22 00:46:52.268144 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fcc23884c064d7a838340db3f7a410f9, offset: 0, size: 161481 2024.10.22 00:46:52.271032 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fcc23884c064d7a838340db3f7a410f9 for removal 2024.10.22 00:46:52.273625 [ 1476 ] {} CacheMetadata: Key fcc23884c064d7a838340db3f7a410f9 is removed from metadata 2024.10.22 00:46:52.273830 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: db384d40fe720dc8c408b01e46a5b9ae, offset: 0, size: 168 2024.10.22 00:46:52.280357 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key db384d40fe720dc8c408b01e46a5b9ae for removal 2024.10.22 00:46:52.282733 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 776351a34f40a373b607b077d4e8954b, offset: 0, size: 7701 2024.10.22 00:46:52.285253 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 776351a34f40a373b607b077d4e8954b for removal 2024.10.22 00:46:52.288292 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2d499979be247e4510a498b1e03dede5, offset: 0, size: 264 2024.10.22 00:46:52.290480 [ 1476 ] {} CacheMetadata: Key 776351a34f40a373b607b077d4e8954b is removed from metadata 2024.10.22 00:46:52.291114 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2d499979be247e4510a498b1e03dede5 for removal 2024.10.22 00:46:52.292893 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: da700ea4e0a9c2e44383a8e246e3ce90, offset: 0, size: 161 2024.10.22 00:46:52.299424 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key da700ea4e0a9c2e44383a8e246e3ce90 for removal 2024.10.22 00:46:52.302739 [ 1476 ] {} CacheMetadata: Key 2d499979be247e4510a498b1e03dede5 is removed from metadata 2024.10.22 00:46:52.310900 [ 1476 ] {} CacheMetadata: Key da700ea4e0a9c2e44383a8e246e3ce90 is removed from metadata 2024.10.22 00:46:52.320927 [ 1476 ] {} CacheMetadata: Key db384d40fe720dc8c408b01e46a5b9ae is removed from metadata 2024.10.22 00:46:52.771058 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 137518 bytes for feeae4c1663ed082141db56c79e6e7a9:0) 2024.10.22 00:46:52.772629 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c9c580d492b80df5b93f8066c30a74ed, offset: 0, size: 141224 2024.10.22 00:46:52.774697 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c9c580d492b80df5b93f8066c30a74ed for removal 2024.10.22 00:46:52.776372 [ 1476 ] {} CacheMetadata: Key c9c580d492b80df5b93f8066c30a74ed is removed from metadata 2024.10.22 00:46:52.776627 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b079cf249078e43bb51ef79fd7bb983d, offset: 0, size: 167 2024.10.22 00:46:52.780832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b079cf249078e43bb51ef79fd7bb983d for removal 2024.10.22 00:46:52.785350 [ 1476 ] {} CacheMetadata: Key b079cf249078e43bb51ef79fd7bb983d is removed from metadata 2024.10.22 00:46:53.571960 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b896d1d070888134ed583091459448cc) : [0, 207156]: Setting download as failed: File segment: [0, 207156], key: b896d1d070888134ed583091459448cc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:53.573625 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b896d1d070888134ed583091459448cc): No space left in cache to reserve 207157 bytes, will continue without cache download 2024.10.22 00:46:53.587472 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b896d1d070888134ed583091459448cc, offset: 0, size: 0 2024.10.22 00:46:53.588862 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b896d1d070888134ed583091459448cc for removal 2024.10.22 00:46:53.591487 [ 1476 ] {} CacheMetadata: Key b896d1d070888134ed583091459448cc is removed from metadata 2024.10.22 00:46:53.617368 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 2836 2024.10.22 00:46:53.720264 [ 1166 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:46:53.727839 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:53.733057 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:53.879474 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_202_202_0 (state Temporary) 2024.10.22 00:46:53.933563 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_202_202_0 to 202410_215_215_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:53.967904 [ 1166 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:46:53.969801 [ 1166 ] {} DNSCacheUpdater: Execution took 253 ms. 2024.10.22 00:46:53.972355 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2836 2024.10.22 00:46:54.120810 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 180 2024.10.22 00:46:54.153027 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 84153 bytes for 1085aa794214826b06b2de3e8798a04e:0) 2024.10.22 00:46:54.156526 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: feeae4c1663ed082141db56c79e6e7a9, offset: 0, size: 137518 2024.10.22 00:46:54.161493 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key feeae4c1663ed082141db56c79e6e7a9 for removal 2024.10.22 00:46:54.165470 [ 1476 ] {} CacheMetadata: Key feeae4c1663ed082141db56c79e6e7a9 is removed from metadata 2024.10.22 00:46:54.166794 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1c8446927855b3a1db95eedb67d6cdc1, offset: 0, size: 164 2024.10.22 00:46:54.170729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1c8446927855b3a1db95eedb67d6cdc1 for removal 2024.10.22 00:46:54.172636 [ 1476 ] {} CacheMetadata: Key 1c8446927855b3a1db95eedb67d6cdc1 is removed from metadata 2024.10.22 00:46:54.381153 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:54.384387 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:54.457660 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_162_162_0 (state Temporary) 2024.10.22 00:46:54.619652 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_162_162_0 to 202410_172_172_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:54.661949 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 6.60 MiB, will set to 1.65 GiB (RSS), difference: -6.41 MiB 2024.10.22 00:46:54.687976 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 180 2024.10.22 00:46:54.986536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bea7c7656dbc78993ffb6a92d3d7c869) : [0, 523059]: Setting download as failed: File segment: [0, 523059], key: bea7c7656dbc78993ffb6a92d3d7c869, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:54.988292 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bea7c7656dbc78993ffb6a92d3d7c869): No space left in cache to reserve 523060 bytes, will continue without cache download 2024.10.22 00:46:55.025596 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bea7c7656dbc78993ffb6a92d3d7c869, offset: 0, size: 0 2024.10.22 00:46:55.029231 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bea7c7656dbc78993ffb6a92d3d7c869 for removal 2024.10.22 00:46:55.032686 [ 1476 ] {} CacheMetadata: Key bea7c7656dbc78993ffb6a92d3d7c869 is removed from metadata 2024.10.22 00:46:55.448591 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 19579 2024.10.22 00:46:55.569305 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 335396 2024.10.22 00:46:55.624345 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(04480aae540a71b1ec483d268dfb48a4) : [0, 430758]: Setting download as failed: File segment: [0, 430758], key: 04480aae540a71b1ec483d268dfb48a4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:55.627167 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(04480aae540a71b1ec483d268dfb48a4): No space left in cache to reserve 430759 bytes, will continue without cache download 2024.10.22 00:46:55.767838 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 04480aae540a71b1ec483d268dfb48a4, offset: 0, size: 0 2024.10.22 00:46:55.769185 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 04480aae540a71b1ec483d268dfb48a4 for removal 2024.10.22 00:46:55.771768 [ 1476 ] {} CacheMetadata: Key 04480aae540a71b1ec483d268dfb48a4 is removed from metadata 2024.10.22 00:46:55.796787 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:55.803744 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:55.970252 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_346_346_0 (state Temporary) 2024.10.22 00:46:56.155658 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:56.159399 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:56.216685 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_346_346_0 to 202410_359_359_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:56.484367 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19579 2024.10.22 00:46:56.571887 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_193_193_0 (state Temporary) 2024.10.22 00:46:56.663112 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 5.25 MiB, will set to 1.65 GiB (RSS), difference: -8.04 MiB 2024.10.22 00:46:56.765447 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_193_193_0 to 202410_200_200_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:46:56.803011 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 335396 2024.10.22 00:46:57.077537 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(51ae740bbcff5bd0e3305ba12f44e7d6) : [0, 275500]: Setting download as failed: File segment: [0, 275500], key: 51ae740bbcff5bd0e3305ba12f44e7d6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:57.079953 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(51ae740bbcff5bd0e3305ba12f44e7d6): No space left in cache to reserve 275501 bytes, will continue without cache download 2024.10.22 00:46:57.134960 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 51ae740bbcff5bd0e3305ba12f44e7d6, offset: 0, size: 0 2024.10.22 00:46:57.140299 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 51ae740bbcff5bd0e3305ba12f44e7d6 for removal 2024.10.22 00:46:57.151654 [ 1476 ] {} CacheMetadata: Key 51ae740bbcff5bd0e3305ba12f44e7d6 is removed from metadata 2024.10.22 00:46:57.870223 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5f4a6595b27cc06a520fc7b1cdaa1f01) : [0, 458779]: Setting download as failed: File segment: [0, 458779], key: 5f4a6595b27cc06a520fc7b1cdaa1f01, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:57.872291 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5f4a6595b27cc06a520fc7b1cdaa1f01): No space left in cache to reserve 458780 bytes, will continue without cache download 2024.10.22 00:46:57.893816 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5f4a6595b27cc06a520fc7b1cdaa1f01, offset: 0, size: 0 2024.10.22 00:46:57.896670 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5f4a6595b27cc06a520fc7b1cdaa1f01 for removal 2024.10.22 00:46:57.904159 [ 1476 ] {} CacheMetadata: Key 5f4a6595b27cc06a520fc7b1cdaa1f01 is removed from metadata 2024.10.22 00:46:58.955563 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(06d9975016f806ecc59bbde122da1a2b) : [0, 549484]: Setting download as failed: File segment: [0, 549484], key: 06d9975016f806ecc59bbde122da1a2b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:58.957323 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(06d9975016f806ecc59bbde122da1a2b): No space left in cache to reserve 549485 bytes, will continue without cache download 2024.10.22 00:46:58.994596 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 06d9975016f806ecc59bbde122da1a2b, offset: 0, size: 0 2024.10.22 00:46:58.998377 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 06d9975016f806ecc59bbde122da1a2b for removal 2024.10.22 00:46:59.007575 [ 1476 ] {} CacheMetadata: Key 06d9975016f806ecc59bbde122da1a2b is removed from metadata 2024.10.22 00:46:59.490762 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 37 entries to flush up to offset 12966 2024.10.22 00:46:59.562129 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 5.60 MiB, will set to 1.65 GiB (RSS), difference: 8.89 MiB 2024.10.22 00:46:59.589611 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:46:59.606707 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:46:59.696463 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:46:59.777193 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(88c8d802cbabc5faf7991c12c16ce604) : [0, 677870]: Setting download as failed: File segment: [0, 677870], key: 88c8d802cbabc5faf7991c12c16ce604, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:46:59.808511 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:46:59.809194 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(88c8d802cbabc5faf7991c12c16ce604): No space left in cache to reserve 677871 bytes, will continue without cache download 2024.10.22 00:46:59.869032 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 88c8d802cbabc5faf7991c12c16ce604, offset: 0, size: 0 2024.10.22 00:46:59.872736 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 88c8d802cbabc5faf7991c12c16ce604 for removal 2024.10.22 00:46:59.877497 [ 1476 ] {} CacheMetadata: Key 88c8d802cbabc5faf7991c12c16ce604 is removed from metadata 2024.10.22 00:46:59.885394 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_201_201_0 (state Temporary) 2024.10.22 00:46:59.927798 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_201_201_0 to 202410_207_207_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:00.177353 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 12966 2024.10.22 00:47:00.594865 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 19626 2024.10.22 00:47:00.879653 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:00.882081 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:47:00.917114 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b4dd53f4a71573e053037ec6dabab0ed) : [0, 1104758]: Setting download as failed: File segment: [0, 1104758], key: b4dd53f4a71573e053037ec6dabab0ed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:00.919458 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b4dd53f4a71573e053037ec6dabab0ed): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:47:00.928561 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_347_347_0 (state Temporary) 2024.10.22 00:47:00.989047 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b4dd53f4a71573e053037ec6dabab0ed, offset: 0, size: 0 2024.10.22 00:47:00.991951 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b4dd53f4a71573e053037ec6dabab0ed for removal 2024.10.22 00:47:00.997022 [ 1476 ] {} CacheMetadata: Key b4dd53f4a71573e053037ec6dabab0ed is removed from metadata 2024.10.22 00:47:01.075963 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_347_347_0 to 202410_360_360_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:01.250659 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_151_30, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:47:01.252782 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_151_30, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:47:01.271585 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_151_30, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:47:01.273319 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19626 2024.10.22 00:47:01.330925 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 5.50 MiB, will set to 1.65 GiB (RSS), difference: -1.88 MiB 2024.10.22 00:47:01.473407 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 2842 2024.10.22 00:47:01.690848 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:01.695430 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:47:01.746590 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_203_203_0 (state Temporary) 2024.10.22 00:47:01.985439 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_203_203_0 to 202410_216_216_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:02.020216 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c88a94c700e8840a098a4974783449c5) : [0, 1093777]: Setting download as failed: File segment: [0, 1093777], key: c88a94c700e8840a098a4974783449c5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:02.021597 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c88a94c700e8840a098a4974783449c5): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:47:02.082841 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c88a94c700e8840a098a4974783449c5, offset: 0, size: 0 2024.10.22 00:47:02.084479 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c88a94c700e8840a098a4974783449c5 for removal 2024.10.22 00:47:02.086676 [ 1233 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_211_42 to 202410_216_216_0 2024.10.22 00:47:02.087050 [ 1476 ] {} CacheMetadata: Key c88a94c700e8840a098a4974783449c5 is removed from metadata 2024.10.22 00:47:02.094690 [ 1233 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:02.096639 [ 1233 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:47:02.121759 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_211_42 to 202410_216_216_0 into Compact with storage Full 2024.10.22 00:47:02.191292 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2842 2024.10.22 00:47:02.191290 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 181 2024.10.22 00:47:02.205690 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:02.220178 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_211_42, total 2877 rows starting from the beginning of the part 2024.10.22 00:47:02.398254 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_212_212_0, total 20 rows starting from the beginning of the part 2024.10.22 00:47:02.452583 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:02.459486 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:02.471318 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_213_213_0, total 28 rows starting from the beginning of the part 2024.10.22 00:47:02.547902 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_214_214_0, total 4 rows starting from the beginning of the part 2024.10.22 00:47:02.694132 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_215_215_0, total 13 rows starting from the beginning of the part 2024.10.22 00:47:02.712966 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_163_163_0 (state Temporary) 2024.10.22 00:47:02.787540 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_216_216_0, total 6 rows starting from the beginning of the part 2024.10.22 00:47:02.804473 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(307f1e924b0783d7b94834538048ad17) : [0, 678373]: Setting download as failed: File segment: [0, 678373], key: 307f1e924b0783d7b94834538048ad17, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:02.808107 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(307f1e924b0783d7b94834538048ad17): No space left in cache to reserve 678374 bytes, will continue without cache download 2024.10.22 00:47:02.853175 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 307f1e924b0783d7b94834538048ad17, offset: 0, size: 0 2024.10.22 00:47:02.856657 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 307f1e924b0783d7b94834538048ad17 for removal 2024.10.22 00:47:02.860493 [ 1476 ] {} CacheMetadata: Key 307f1e924b0783d7b94834538048ad17 is removed from metadata 2024.10.22 00:47:02.884459 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_24_5, 202410_25_25_0, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0] 2024.10.22 00:47:02.885576 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_24_5, 202410_25_25_0, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0] 2024.10.22 00:47:02.914635 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_24_5, 202410_25_25_0, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0] 2024.10.22 00:47:02.929169 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_163_163_0 to 202410_173_173_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:03.118204 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 181 2024.10.22 00:47:03.374886 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:03.377765 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:47:03.394011 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 3.47 MiB, will set to 1.64 GiB (RSS), difference: -3.28 MiB 2024.10.22 00:47:03.466533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3423196a35f79583a6508e39411c3d20) : [0, 496997]: Setting download as failed: File segment: [0, 496997], key: 3423196a35f79583a6508e39411c3d20, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:03.467689 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3423196a35f79583a6508e39411c3d20): No space left in cache to reserve 496998 bytes, will continue without cache download 2024.10.22 00:47:03.498484 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3423196a35f79583a6508e39411c3d20, offset: 0, size: 0 2024.10.22 00:47:03.500868 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3423196a35f79583a6508e39411c3d20 for removal 2024.10.22 00:47:03.507971 [ 1476 ] {} CacheMetadata: Key 3423196a35f79583a6508e39411c3d20 is removed from metadata 2024.10.22 00:47:03.817739 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 336548 2024.10.22 00:47:04.165160 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} MergeTask::MergeProjectionsStage: Merge sorted 2948 rows, containing 12 columns (12 merged, 0 gathered) in 2.050472595 sec., 1437.7173375487127 rows/sec., 223.82 KiB/sec. 2024.10.22 00:47:04.190624 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} MergedBlockOutputStream: filled checksums 202410_1_216_43 (state Temporary) 2024.10.22 00:47:04.573367 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_216_43 to 202410_1_216_43 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:04.576361 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_216_43} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_211_42, 202410_216_216_0] -> 202410_1_216_43 2024.10.22 00:47:04.612744 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(031d12d30daa0150e8c3fe9e757bc1d6) : [0, 961005]: Setting download as failed: File segment: [0, 961005], key: 031d12d30daa0150e8c3fe9e757bc1d6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:04.614822 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(031d12d30daa0150e8c3fe9e757bc1d6): No space left in cache to reserve 961006 bytes, will continue without cache download 2024.10.22 00:47:04.615905 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:47:04.763580 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 031d12d30daa0150e8c3fe9e757bc1d6, offset: 0, size: 0 2024.10.22 00:47:04.765420 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 031d12d30daa0150e8c3fe9e757bc1d6 for removal 2024.10.22 00:47:04.768826 [ 1476 ] {} CacheMetadata: Key 031d12d30daa0150e8c3fe9e757bc1d6 is removed from metadata 2024.10.22 00:47:04.861064 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:04.966275 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:47:05.185706 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_194_194_0 (state Temporary) 2024.10.22 00:47:05.275578 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_194_194_0 to 202410_201_201_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:05.279007 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 19699 2024.10.22 00:47:05.418982 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 336548 2024.10.22 00:47:05.420671 [ 1223 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_196_39 to 202410_201_201_0 2024.10.22 00:47:05.448038 [ 1223 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:05.458763 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.52 GiB. 2024.10.22 00:47:05.494067 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_196_39 to 202410_201_201_0 into Compact with storage Full 2024.10.22 00:47:05.572976 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:05.576556 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:05.609125 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7ca3028483fefd9a532d54f60e23ca9e) : [0, 719063]: Setting download as failed: File segment: [0, 719063], key: 7ca3028483fefd9a532d54f60e23ca9e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:05.611171 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7ca3028483fefd9a532d54f60e23ca9e): No space left in cache to reserve 719064 bytes, will continue without cache download 2024.10.22 00:47:05.645509 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:05.655494 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} MergeTreeSequentialSource: Reading 43 marks from part 202410_1_196_39, total 342458 rows starting from the beginning of the part 2024.10.22 00:47:05.565676 [ 1228 ] {} DatabaseCatalog: Execution took 232 ms. 2024.10.22 00:47:05.670098 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ca3028483fefd9a532d54f60e23ca9e, offset: 0, size: 0 2024.10.22 00:47:05.671261 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_197_197_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:47:05.672176 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ca3028483fefd9a532d54f60e23ca9e for removal 2024.10.22 00:47:05.674555 [ 1476 ] {} CacheMetadata: Key 7ca3028483fefd9a532d54f60e23ca9e is removed from metadata 2024.10.22 00:47:05.680659 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_198_198_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:47:05.714419 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_199_199_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:47:05.760399 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_200_200_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:47:05.764592 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_348_348_0 (state Temporary) 2024.10.22 00:47:05.783959 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_201_201_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:47:06.012484 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_348_348_0 to 202410_361_361_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:06.265543 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19699 2024.10.22 00:47:06.521153 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(871691a59833afe91d5a4985ded3181a) : [0, 454084]: Setting download as failed: File segment: [0, 454084], key: 871691a59833afe91d5a4985ded3181a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:06.523452 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(871691a59833afe91d5a4985ded3181a): No space left in cache to reserve 454085 bytes, will continue without cache download 2024.10.22 00:47:06.872434 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 871691a59833afe91d5a4985ded3181a, offset: 0, size: 0 2024.10.22 00:47:06.874518 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 871691a59833afe91d5a4985ded3181a for removal 2024.10.22 00:47:06.878564 [ 1476 ] {} CacheMetadata: Key 871691a59833afe91d5a4985ded3181a is removed from metadata 2024.10.22 00:47:07.170732 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_41_41_0 (state Temporary) 2024.10.22 00:47:07.411636 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b2d4087ba937701f4bc3ff6ee1cb8108) : [0, 451939]: Setting download as failed: File segment: [0, 451939], key: b2d4087ba937701f4bc3ff6ee1cb8108, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:07.413269 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b2d4087ba937701f4bc3ff6ee1cb8108): No space left in cache to reserve 451940 bytes, will continue without cache download 2024.10.22 00:47:07.658143 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b2d4087ba937701f4bc3ff6ee1cb8108, offset: 0, size: 0 2024.10.22 00:47:07.660552 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b2d4087ba937701f4bc3ff6ee1cb8108 for removal 2024.10.22 00:47:07.663528 [ 1476 ] {} CacheMetadata: Key b2d4087ba937701f4bc3ff6ee1cb8108 is removed from metadata 2024.10.22 00:47:07.680581 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 6.21 MiB, will set to 1.65 GiB (RSS), difference: -7.96 MiB 2024.10.22 00:47:07.688938 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 55 entries to flush up to offset 13021 2024.10.22 00:47:07.805379 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(0d6673e58e8ced9bc6e813d7994ee8e3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0d6673e58e8ced9bc6e813d7994ee8e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 00:47:07.809208 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 246447, file segment info: File segment: [0, 33554431], key: 0d6673e58e8ced9bc6e813d7994ee8e3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 00:47:07.813016 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:47:07.824766 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:07.828120 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:07.873612 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_202_202_0 (state Temporary) 2024.10.22 00:47:08.017786 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_202_202_0 to 202410_208_208_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:08.058837 [ 1249 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_203_42 to 202410_204_204_0 2024.10.22 00:47:08.070286 [ 1249 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:08.074610 [ 1249 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:08.117192 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_204_43} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_203_42 to 202410_204_204_0 into Compact with storage Full 2024.10.22 00:47:08.184918 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13021 2024.10.22 00:47:08.259330 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_204_43} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:08.271585 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_204_43} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_203_42, total 12881 rows starting from the beginning of the part 2024.10.22 00:47:08.433067 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_204_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_204_204_0, total 76 rows starting from the beginning of the part 2024.10.22 00:47:08.519017 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b19a77016659a24791a0ec4cba013836) : [0, 373295]: Setting download as failed: File segment: [0, 373295], key: b19a77016659a24791a0ec4cba013836, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:08.524147 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b19a77016659a24791a0ec4cba013836): No space left in cache to reserve 373296 bytes, will continue without cache download 2024.10.22 00:47:08.566711 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b19a77016659a24791a0ec4cba013836, offset: 0, size: 0 2024.10.22 00:47:08.569055 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b19a77016659a24791a0ec4cba013836 for removal 2024.10.22 00:47:08.671713 [ 1476 ] {} CacheMetadata: Key b19a77016659a24791a0ec4cba013836 is removed from metadata 2024.10.22 00:47:08.672542 [ 1693 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mye/vnuaqwvmcmrltvzzpbgubgdcxleio, size 246447 2024.10.22 00:47:08.804529 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_41_41_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:47:08.970815 [ 1258 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:47:09.058348 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column MobilePhone 2024.10.22 00:47:09.067312 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column MobilePhone 2024.10.22 00:47:09.075218 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column MobilePhone 2024.10.22 00:47:09.111687 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column MobilePhone 2024.10.22 00:47:09.112507 [ 1258 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:47:09.127520 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column MobilePhone 2024.10.22 00:47:09.354567 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column MobilePhone 2024.10.22 00:47:09.384771 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column MobilePhone 2024.10.22 00:47:09.592583 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:47:09.619976 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:47:09.620746 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nrn/dnbatncuxxncwzshvlqavhhjbnjnl (LogSeriesLimiter: on interval from 2024-10-22 00:46:51 to 2024-10-22 00:47:09 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:47:09.673226 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 11.16 MiB, will set to 1.65 GiB (RSS), difference: -10.60 MiB 2024.10.22 00:47:09.692774 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 2854 2024.10.22 00:47:09.694260 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8437fcbcf1584853021d9a03f1542ce5) : [0, 795560]: Setting download as failed: File segment: [0, 795560], key: 8437fcbcf1584853021d9a03f1542ce5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:09.696605 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8437fcbcf1584853021d9a03f1542ce5): No space left in cache to reserve 795561 bytes, will continue without cache download 2024.10.22 00:47:09.876681 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_204_43} MergeTask::MergeProjectionsStage: Merge sorted 12957 rows, containing 13 columns (13 merged, 0 gathered) in 1.772750828 sec., 7308.9798043518385 rows/sec., 3.06 MiB/sec. 2024.10.22 00:47:09.929065 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_204_43} MergedBlockOutputStream: filled checksums 202410_1_204_43 (state Temporary) 2024.10.22 00:47:09.963331 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8437fcbcf1584853021d9a03f1542ce5, offset: 0, size: 0 2024.10.22 00:47:09.965186 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8437fcbcf1584853021d9a03f1542ce5 for removal 2024.10.22 00:47:09.967163 [ 1476 ] {} CacheMetadata: Key 8437fcbcf1584853021d9a03f1542ce5 is removed from metadata 2024.10.22 00:47:09.987302 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:09.999714 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:10.161511 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_204_204_0 (state Temporary) 2024.10.22 00:47:10.273740 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 19783 2024.10.22 00:47:10.316871 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_204_204_0 to 202410_217_217_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:10.367214 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2854 2024.10.22 00:47:10.512001 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:10.517060 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:10.526539 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_204_43} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_204_43 to 202410_1_204_43 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:10.569452 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_204_43} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_203_42, 202410_204_204_0] -> 202410_1_204_43 2024.10.22 00:47:10.594384 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.39 MiB. 2024.10.22 00:47:10.620142 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 182 2024.10.22 00:47:10.680143 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_201_40/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:47:10.682685 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_349_349_0 (state Temporary) 2024.10.22 00:47:10.760959 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_256_51, 202410_257_257_0, 202410_258_258_0, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0] 2024.10.22 00:47:10.763524 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_256_51, 202410_257_257_0, 202410_258_258_0, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0] 2024.10.22 00:47:10.779586 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:10.787018 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:10.792594 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_256_51, 202410_257_257_0, 202410_258_258_0, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0] 2024.10.22 00:47:10.850361 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_349_349_0 to 202410_362_362_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:10.853693 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(194febeb39c06a228cc425592693d5cf) : [0, 490495]: Setting download as failed: File segment: [0, 490495], key: 194febeb39c06a228cc425592693d5cf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:10.855533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(194febeb39c06a228cc425592693d5cf): No space left in cache to reserve 490496 bytes, will continue without cache download 2024.10.22 00:47:10.883587 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 194febeb39c06a228cc425592693d5cf, offset: 0, size: 0 2024.10.22 00:47:10.886202 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 194febeb39c06a228cc425592693d5cf for removal 2024.10.22 00:47:10.890102 [ 1476 ] {} CacheMetadata: Key 194febeb39c06a228cc425592693d5cf is removed from metadata 2024.10.22 00:47:10.970380 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_164_164_0 (state Temporary) 2024.10.22 00:47:10.975924 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_357_75 to 202410_362_362_0 2024.10.22 00:47:10.989387 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:11.039659 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19783 2024.10.22 00:47:11.094487 [ 1150 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:11.234612 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_164_164_0 to 202410_174_174_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:11.256522 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_357_75 to 202410_362_362_0 into Compact with storage Full 2024.10.22 00:47:11.403830 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:11.415271 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_357_75, total 20244 rows starting from the beginning of the part 2024.10.22 00:47:11.451026 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_358_358_0, total 84 rows starting from the beginning of the part 2024.10.22 00:47:11.472250 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_359_359_0, total 82 rows starting from the beginning of the part 2024.10.22 00:47:11.490388 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_360_360_0, total 47 rows starting from the beginning of the part 2024.10.22 00:47:11.518776 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_361_361_0, total 73 rows starting from the beginning of the part 2024.10.22 00:47:11.520258 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 182 2024.10.22 00:47:11.554809 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_362_362_0, total 84 rows starting from the beginning of the part 2024.10.22 00:47:11.684617 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 10.54 MiB, will set to 1.65 GiB (RSS), difference: 10.37 MiB 2024.10.22 00:47:11.958710 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(85cc480d292258890b79f2c6bd4b4823) : [0, 1923872]: Setting download as failed: File segment: [0, 1923872], key: 85cc480d292258890b79f2c6bd4b4823, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:12.063092 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(85cc480d292258890b79f2c6bd4b4823): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:47:12.272656 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 85cc480d292258890b79f2c6bd4b4823, offset: 0, size: 0 2024.10.22 00:47:12.275194 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 85cc480d292258890b79f2c6bd4b4823 for removal 2024.10.22 00:47:12.278419 [ 1476 ] {} CacheMetadata: Key 85cc480d292258890b79f2c6bd4b4823 is removed from metadata 2024.10.22 00:47:12.484113 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1877 entries to flush up to offset 338425 2024.10.22 00:47:12.988173 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7de29dc868acfc6400e446bc062f7601) : [0, 2300176]: Setting download as failed: File segment: [0, 2300176], key: 7de29dc868acfc6400e446bc062f7601, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:12.989559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7de29dc868acfc6400e446bc062f7601): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:47:13.255168 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} MergeTask::MergeProjectionsStage: Merge sorted 20614 rows, containing 13 columns (13 merged, 0 gathered) in 2.011269846 sec., 10249.24628636828 rows/sec., 1.53 MiB/sec. 2024.10.22 00:47:13.413771 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:13.419899 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:13.423853 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7de29dc868acfc6400e446bc062f7601, offset: 0, size: 0 2024.10.22 00:47:13.425529 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7de29dc868acfc6400e446bc062f7601 for removal 2024.10.22 00:47:13.429028 [ 1476 ] {} CacheMetadata: Key 7de29dc868acfc6400e446bc062f7601 is removed from metadata 2024.10.22 00:47:13.434863 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} MergedBlockOutputStream: filled checksums 202410_1_362_76 (state Temporary) 2024.10.22 00:47:13.453667 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 11.04 MiB, will set to 1.66 GiB (RSS), difference: 4.54 MiB 2024.10.22 00:47:13.581198 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_195_195_0 (state Temporary) 2024.10.22 00:47:13.664526 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_195_195_0 to 202410_202_202_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:13.688119 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_362_76 to 202410_1_362_76 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:13.692436 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_362_76} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_357_75, 202410_362_362_0] -> 202410_1_362_76 2024.10.22 00:47:13.717509 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 338425 2024.10.22 00:47:13.739663 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.74 MiB. 2024.10.22 00:47:14.462380 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(71091bfa700f63632db7f83b96bd2e35) : [0, 883087]: Setting download as failed: File segment: [0, 883087], key: 71091bfa700f63632db7f83b96bd2e35, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:14.463812 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(71091bfa700f63632db7f83b96bd2e35): No space left in cache to reserve 883088 bytes, will continue without cache download 2024.10.22 00:47:14.829161 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 71091bfa700f63632db7f83b96bd2e35, offset: 0, size: 0 2024.10.22 00:47:14.832065 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 71091bfa700f63632db7f83b96bd2e35 for removal 2024.10.22 00:47:14.834660 [ 1476 ] {} CacheMetadata: Key 71091bfa700f63632db7f83b96bd2e35 is removed from metadata 2024.10.22 00:47:15.062975 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 19848 2024.10.22 00:47:15.285409 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:15.289410 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:15.522768 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_41_41_0 to 202410_44_44_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:15.572222 [ 1159 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_39_8 to 202410_44_44_0 2024.10.22 00:47:15.573254 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_350_350_0 (state Temporary) 2024.10.22 00:47:15.593350 [ 1159 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.07 MiB using storage policy from min volume index 0 2024.10.22 00:47:15.612585 [ 1159 ] {} DiskLocal: Reserved 1.07 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:15.617569 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_350_350_0 to 202410_363_363_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:15.665794 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_39_8 to 202410_44_44_0 into Compact with storage Full 2024.10.22 00:47:15.692213 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 13111 2024.10.22 00:47:15.779104 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19848 2024.10.22 00:47:15.788238 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:15.792228 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:15.840452 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_203_203_0 (state Temporary) 2024.10.22 00:47:15.885602 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_203_203_0 to 202410_209_209_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:15.959721 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13111 2024.10.22 00:47:16.059626 [ 1188 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_204_43 to 202410_209_209_0 2024.10.22 00:47:16.074759 [ 1188 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:16.076867 [ 1188 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:16.103693 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bb6ef661e4bb7726bd4aec18f77fc315) : [0, 207942]: Setting download as failed: File segment: [0, 207942], key: bb6ef661e4bb7726bd4aec18f77fc315, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:47:16.107990 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bb6ef661e4bb7726bd4aec18f77fc315): No space left in cache to reserve 207943 bytes, will continue without cache download 2024.10.22 00:47:16.180463 [ 1188 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 284 ms. 2024.10.22 00:47:16.227201 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_204_43 to 202410_209_209_0 into Compact with storage Full 2024.10.22 00:47:16.270934 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bb6ef661e4bb7726bd4aec18f77fc315, offset: 0, size: 0 2024.10.22 00:47:16.272741 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bb6ef661e4bb7726bd4aec18f77fc315 for removal 2024.10.22 00:47:16.277070 [ 1476 ] {} CacheMetadata: Key bb6ef661e4bb7726bd4aec18f77fc315 is removed from metadata 2024.10.22 00:47:16.294304 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:16.310548 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_204_43, total 12957 rows starting from the beginning of the part 2024.10.22 00:47:16.339510 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_205_205_0, total 26 rows starting from the beginning of the part 2024.10.22 00:47:16.370107 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_206_206_0, total 50 rows starting from the beginning of the part 2024.10.22 00:47:16.433311 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nqm/wlwhibesgsqvkkuvesdcaxaabwtpy. 2024.10.22 00:47:16.452926 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_207_207_0, total 37 rows starting from the beginning of the part 2024.10.22 00:47:16.593168 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_208_208_0, total 55 rows starting from the beginning of the part 2024.10.22 00:47:16.650085 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_209_209_0, total 90 rows starting from the beginning of the part 2024.10.22 00:47:17.588970 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 14.19 MiB, will set to 1.66 GiB (RSS), difference: -2.87 MiB 2024.10.22 00:47:17.819429 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nqm/wlwhibesgsqvkkuvesdcaxaabwtpy. 2024.10.22 00:47:17.822404 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_13_13_0 to 201403_13_13_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:17.868754 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 2872 2024.10.22 00:47:17.887825 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:47:18.074691 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:18.076090 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:18.079327 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} MergeTask::MergeProjectionsStage: Merge sorted 350138 rows, containing 4 columns (4 merged, 0 gathered) in 12.600251915 sec., 27788.174582698415 rows/sec., 407.65 KiB/sec. 2024.10.22 00:47:18.130670 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_205_205_0 (state Temporary) 2024.10.22 00:47:18.184890 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_205_205_0 to 202410_218_218_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:18.227957 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2872 2024.10.22 00:47:18.345214 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} MergedBlockOutputStream: filled checksums 202410_1_201_40 (state Temporary) 2024.10.22 00:47:18.412256 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_201_40 to 202410_1_201_40 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:18.426412 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_201_40} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_196_39, 202410_201_201_0] -> 202410_1_201_40 2024.10.22 00:47:18.473798 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:47:18.595311 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} MergeTask::MergeProjectionsStage: Merge sorted 13215 rows, containing 13 columns (13 merged, 0 gathered) in 2.396769657 sec., 5513.6712705805085 rows/sec., 2.31 MiB/sec. 2024.10.22 00:47:18.584287 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1625 2024.10.22 00:47:18.768949 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} MergedBlockOutputStream: filled checksums 202410_1_209_44 (state Temporary) 2024.10.22 00:47:18.870437 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_209_44 to 202410_1_209_44 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:18.872743 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_209_44} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_204_43, 202410_209_209_0] -> 202410_1_209_44 2024.10.22 00:47:18.913637 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.59 MiB. 2024.10.22 00:47:19.093425 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 13 2024.10.22 00:47:19.432455 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:19.538687 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:19.581794 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_13_13_0 (state Temporary) 2024.10.22 00:47:19.592199 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:47:19.691240 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:47:19.691577 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.88 GiB, free memory in arenas 14.81 MiB, will set to 1.41 GiB (RSS), difference: 247.44 MiB 2024.10.22 00:47:19.906700 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 19911 2024.10.22 00:47:19.966519 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_13_13_0 to 202410_13_13_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:20.048208 [ 1421 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Found 5 old parts to remove. Parts: [202410_1_5_1, 202410_6_6_0, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0] 2024.10.22 00:47:20.051465 [ 1421 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from filesystem (serially): Parts: [202410_1_5_1, 202410_6_6_0, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0] 2024.10.22 00:47:20.071831 [ 1421 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from memory: Parts: [202410_1_5_1, 202410_6_6_0, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0] 2024.10.22 00:47:20.072449 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 13 2024.10.22 00:47:20.087869 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:20.091233 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:20.240866 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_351_351_0 (state Temporary) 2024.10.22 00:47:20.387338 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_351_351_0 to 202410_364_364_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:20.634440 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19911 2024.10.22 00:47:20.830889 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1579 entries to flush up to offset 340004 2024.10.22 00:47:20.898746 [ 1207 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 214 ms. 2024.10.22 00:47:21.313758 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 12.64 MiB, will set to 1.43 GiB (RSS), difference: 4.64 MiB 2024.10.22 00:47:21.506563 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_145_29, 202410_146_146_0, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0] 2024.10.22 00:47:21.510484 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_145_29, 202410_146_146_0, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0] 2024.10.22 00:47:21.541286 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_145_29, 202410_146_146_0, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0] 2024.10.22 00:47:21.757100 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:21.760898 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:21.971791 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_196_196_0 (state Temporary) 2024.10.22 00:47:22.162760 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_196_196_0 to 202410_203_203_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:22.299542 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 340004 2024.10.22 00:47:23.459245 [ 1192 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 259 ms. 2024.10.22 00:47:23.468472 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 66 entries to flush up to offset 13177 2024.10.22 00:47:23.488559 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.88 GiB, free memory in arenas 10.80 MiB, will set to 1.46 GiB (RSS), difference: -69.05 MiB 2024.10.22 00:47:23.860605 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:23.863424 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:23.916291 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_204_204_0 (state Temporary) 2024.10.22 00:47:23.967523 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_204_204_0 to 202410_210_210_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:24.002877 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13177 2024.10.22 00:47:24.120632 [ 1220 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:47:24.161301 [ 1220 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:47:24.647016 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 19943 2024.10.22 00:47:24.778204 [ 1224 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_170_35 to 202410_174_174_0 2024.10.22 00:47:24.784240 [ 1224 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:24.786548 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:24.877156 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:24.879153 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:24.913099 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_352_352_0 (state Temporary) 2024.10.22 00:47:24.937566 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_174_36} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_170_35 to 202410_174_174_0 into Compact with storage Full 2024.10.22 00:47:25.077496 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(b1515df9bd43a94426139b71a8410581) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b1515df9bd43a94426139b71a8410581, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:47:25.080007 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 230914, file segment info: File segment: [0, 33554431], key: b1515df9bd43a94426139b71a8410581, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:47:25.081382 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:47:25.148623 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_352_352_0 to 202410_365_365_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:25.172828 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_174_36} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:25.177144 [ 1709 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nrn/dnbatncuxxncwzshvlqavhhjbnjnl, size 230914 2024.10.22 00:47:25.191128 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_174_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_170_35, total 214 rows starting from the beginning of the part 2024.10.22 00:47:25.235194 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19943 2024.10.22 00:47:25.289631 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_174_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_171_171_0, total 1 rows starting from the beginning of the part 2024.10.22 00:47:25.398955 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_174_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_172_172_0, total 1 rows starting from the beginning of the part 2024.10.22 00:47:25.584972 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_174_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_173_173_0, total 1 rows starting from the beginning of the part 2024.10.22 00:47:25.685488 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 00:47:25.693799 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 00:47:25.732042 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 2892 2024.10.22 00:47:25.810325 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.88 GiB, free memory in arenas 8.67 MiB, will set to 1.50 GiB (RSS), difference: -42.38 MiB 2024.10.22 00:47:25.842400 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 00:47:25.845373 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:25.849807 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:25.870882 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 00:47:25.911458 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_206_206_0 (state Temporary) 2024.10.22 00:47:25.919172 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 00:47:25.970884 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 00:47:26.064622 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_174_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_174_174_0, total 1 rows starting from the beginning of the part 2024.10.22 00:47:26.091176 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_206_206_0 to 202410_219_219_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:26.159316 [ 1232 ] {} DatabaseCatalog: Execution took 415 ms. 2024.10.22 00:47:26.165040 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 1665 2024.10.22 00:47:26.232589 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 00:47:26.373724 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2892 2024.10.22 00:47:26.374485 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vbo/twhlrjmqorxwehyeldpfjgccguzyz (LogSeriesLimiter: on interval from 2024-10-22 00:47:09 to 2024-10-22 00:47:26 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:47:26.522074 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 183 2024.10.22 00:47:26.899963 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:26.906505 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:27.473989 [ 1249 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 00:47:27.491518 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_165_165_0 (state Temporary) 2024.10.22 00:47:27.643212 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_174_36/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:47:27.658736 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.88 GiB, free memory in arenas 8.10 MiB, will set to 1.54 GiB (RSS), difference: -26.15 MiB 2024.10.22 00:47:27.766525 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_165_165_0 to 202410_175_175_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:27.863536 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_174_36} MergeTask::MergeProjectionsStage: Merge sorted 218 rows, containing 32 columns (32 merged, 0 gathered) in 3.045996865 sec., 71.56934483581617 rows/sec., 7.57 KiB/sec. 2024.10.22 00:47:27.914492 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_174_36} MergedBlockOutputStream: filled checksums 202410_1_174_36 (state Temporary) 2024.10.22 00:47:28.220539 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 183 2024.10.22 00:47:28.316286 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_141_28, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0] 2024.10.22 00:47:28.370290 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_174_36} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_174_36 to 202410_1_174_36 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:28.373129 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_174_36} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_170_35, 202410_174_174_0] -> 202410_1_174_36 2024.10.22 00:47:28.419071 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:47:28.425325 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_141_28, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0] 2024.10.22 00:47:28.458975 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_141_28, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0] 2024.10.22 00:47:29.241502 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 19997 2024.10.22 00:47:29.268674 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.88 GiB, free memory in arenas 4.29 MiB, will set to 1.57 GiB (RSS), difference: -18.39 MiB 2024.10.22 00:47:29.309087 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 341540 2024.10.22 00:47:29.567773 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:29.591873 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:47:29.593573 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:47:29.569839 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:29.973141 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_353_353_0 (state Temporary) 2024.10.22 00:47:30.165206 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_353_353_0 to 202410_366_366_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:30.232446 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 19997 2024.10.22 00:47:30.444550 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:30.451214 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:30.599305 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_197_197_0 (state Temporary) 2024.10.22 00:47:30.778720 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_197_197_0 to 202410_204_204_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:30.813696 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 341540 2024.10.22 00:47:31.306944 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 4.61 MiB, will set to 1.62 GiB (RSS), difference: -42.44 MiB 2024.10.22 00:47:31.619914 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 58 entries to flush up to offset 13235 2024.10.22 00:47:31.761219 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:31.765206 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:47:31.765645 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:31.797732 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_205_205_0 (state Temporary) 2024.10.22 00:47:31.833402 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_205_205_0 to 202410_211_211_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:31.991369 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13235 2024.10.22 00:47:33.179272 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 264.11 MiB using storage policy from min volume index 0 2024.10.22 00:47:33.334692 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 4.41 MiB, will set to 1.64 GiB (RSS), difference: -52.78 MiB 2024.10.22 00:47:33.876405 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 2903 2024.10.22 00:47:34.195483 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:34.204794 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:34.251168 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 28 entries to flush up to offset 20025 2024.10.22 00:47:34.363299 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_207_207_0 (state Temporary) 2024.10.22 00:47:34.383658 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:34.387248 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:34.412549 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_207_207_0 to 202410_220_220_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:34.444342 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_354_354_0 (state Temporary) 2024.10.22 00:47:34.484095 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2903 2024.10.22 00:47:34.484364 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:34.498704 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 4.18 MiB, will set to 1.64 GiB (RSS), difference: -5.99 MiB 2024.10.22 00:47:34.532347 [ 1186 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Selected 5 parts from 202410_1_9_2 to 202410_13_13_0 2024.10.22 00:47:34.552147 [ 1186 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:34.556288 [ 1186 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:34.569163 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_13_3} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_9_2 to 202410_13_13_0 into Compact with storage Full 2024.10.22 00:47:34.573747 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gsk/emfiqjgqrzbhajzimijqqxuxubzbe (LogSeriesLimiter: on interval from 2024-10-22 00:47:26 to 2024-10-22 00:47:34 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:47:34.583619 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_354_354_0 to 202410_367_367_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:34.613296 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 5 parts from 202410_1_362_76 to 202410_366_366_0 2024.10.22 00:47:34.618500 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_13_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:34.624946 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:34.630577 [ 1176 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:34.632633 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_13_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_9_2, total 9 rows starting from the beginning of the part 2024.10.22 00:47:34.647025 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20025 2024.10.22 00:47:34.669590 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_366_77} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_362_76 to 202410_366_366_0 into Compact with storage Full 2024.10.22 00:47:34.686927 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_13_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_10_10_0, total 1 rows starting from the beginning of the part 2024.10.22 00:47:34.720818 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_366_77} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:34.730669 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_366_77} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_362_76, total 20614 rows starting from the beginning of the part 2024.10.22 00:47:34.769186 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_366_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_363_363_0, total 65 rows starting from the beginning of the part 2024.10.22 00:47:34.800096 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_366_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_364_364_0, total 63 rows starting from the beginning of the part 2024.10.22 00:47:34.868795 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_366_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_365_365_0, total 32 rows starting from the beginning of the part 2024.10.22 00:47:34.880783 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_13_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_11_11_0, total 1 rows starting from the beginning of the part 2024.10.22 00:47:34.917667 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_366_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_366_366_0, total 54 rows starting from the beginning of the part 2024.10.22 00:47:35.079724 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_13_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_12_12_0, total 1 rows starting from the beginning of the part 2024.10.22 00:47:35.124650 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_13_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_13_13_0, total 1 rows starting from the beginning of the part 2024.10.22 00:47:35.285302 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_123_25, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0, 202410_127_127_0, 202410_128_128_0] 2024.10.22 00:47:35.287451 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_123_25, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0, 202410_127_127_0, 202410_128_128_0] 2024.10.22 00:47:35.316147 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_123_25, 202410_124_124_0, 202410_125_125_0, 202410_126_126_0, 202410_127_127_0, 202410_128_128_0] 2024.10.22 00:47:35.560321 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_261_52, 202410_262_262_0, 202410_263_263_0, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0] 2024.10.22 00:47:35.563097 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_261_52, 202410_262_262_0, 202410_263_263_0, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0] 2024.10.22 00:47:35.606341 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_261_52, 202410_262_262_0, 202410_263_263_0, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0] 2024.10.22 00:47:35.722477 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 184 2024.10.22 00:47:35.822436 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_39_8, total 1490 rows starting from the beginning of the part 2024.10.22 00:47:35.987629 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:35.991605 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:36.165445 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_166_166_0 (state Temporary) 2024.10.22 00:47:36.432459 [ 1397 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_13_3} MergeTask::MergeProjectionsStage: Merge sorted 13 rows, containing 27 columns (27 merged, 0 gathered) in 1.868715231 sec., 6.956651170999098 rows/sec., 22.41 KiB/sec. 2024.10.22 00:47:36.447992 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.81 GiB, peak 1.88 GiB, free memory in arenas 2.82 MiB, will set to 1.64 GiB (RSS), difference: -177.10 MiB 2024.10.22 00:47:36.460365 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_166_166_0 to 202410_176_176_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:36.467350 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_13_3} MergedBlockOutputStream: filled checksums 202410_1_13_3 (state Temporary) 2024.10.22 00:47:36.535655 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 184 2024.10.22 00:47:37.099592 [ 1388 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_13_3} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_merge_202410_1_13_3 to 202410_1_13_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:37.104288 [ 1388 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_13_3} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Merged 5 parts: [202410_1_9_2, 202410_13_13_0] -> 202410_1_13_3 2024.10.22 00:47:37.215164 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.24 MiB. 2024.10.22 00:47:37.482755 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_366_77} MergeTask::MergeProjectionsStage: Merge sorted 20828 rows, containing 13 columns (13 merged, 0 gathered) in 2.821570452 sec., 7381.704747168936 rows/sec., 1.10 MiB/sec. 2024.10.22 00:47:37.539571 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_366_77} MergedBlockOutputStream: filled checksums 202410_1_366_77 (state Temporary) 2024.10.22 00:47:37.628925 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_366_77} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_366_77 to 202410_1_366_77 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:37.632963 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_366_77} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 5 parts: [202410_1_362_76, 202410_366_366_0] -> 202410_1_366_77 2024.10.22 00:47:37.780776 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.73 MiB. 2024.10.22 00:47:37.820293 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 343460 2024.10.22 00:47:38.323370 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.88 GiB, free memory in arenas 1.96 MiB, will set to 1.64 GiB (RSS), difference: -135.22 MiB 2024.10.22 00:47:38.530574 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_40_40_0, total 41 rows starting from the beginning of the part 2024.10.22 00:47:38.651580 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 20084 2024.10.22 00:47:38.669084 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:38.671945 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:38.894423 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:38.897554 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_198_198_0 (state Temporary) 2024.10.22 00:47:38.900588 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:38.988823 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_198_198_0 to 202410_205_205_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:39.016407 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/sxm/jdipbkbkvtrofeizdwmkutustatxm (LogSeriesLimiter: on interval from 2024-10-22 00:47:34 to 2024-10-22 00:47:39 accepted series 1 / 248 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:47:39.065348 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_355_355_0 (state Temporary) 2024.10.22 00:47:39.133663 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 343460 2024.10.22 00:47:39.162941 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_355_355_0 to 202410_368_368_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:39.167577 [ 1209 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:47:39.183425 [ 1209 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:47:39.233553 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20084 2024.10.22 00:47:39.509246 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 77 entries to flush up to offset 13312 2024.10.22 00:47:39.592328 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:47:39.593678 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:47:39.651627 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:39.656828 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:39.691289 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_206_206_0 (state Temporary) 2024.10.22 00:47:39.751514 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_206_206_0 to 202410_212_212_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:39.890802 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13312 2024.10.22 00:47:40.343834 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_41_41_0, total 37 rows starting from the beginning of the part 2024.10.22 00:47:40.386532 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_156_31, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:47:40.395181 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_156_31, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:47:40.430780 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_156_31, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:47:40.578733 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.86 GiB, peak 1.88 GiB, free memory in arenas 2.28 MiB, will set to 1.64 GiB (RSS), difference: -224.99 MiB 2024.10.22 00:47:40.762270 [ 1277 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 00:47:41.368988 [ 1276 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 248 ms. 2024.10.22 00:47:41.985764 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 2921 2024.10.22 00:47:42.221694 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_42_42_0, total 39 rows starting from the beginning of the part 2024.10.22 00:47:42.223814 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:42.243587 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:42.387991 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_208_208_0 (state Temporary) 2024.10.22 00:47:42.519608 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 2.18 MiB, will set to 1.65 GiB (RSS), difference: -4.32 MiB 2024.10.22 00:47:42.559821 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_208_208_0 to 202410_221_221_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:42.608405 [ 1219 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_216_43 to 202410_221_221_0 2024.10.22 00:47:42.618872 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2921 2024.10.22 00:47:42.624012 [ 1219 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:42.649347 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:42.677087 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_216_43 to 202410_221_221_0 into Compact with storage Full 2024.10.22 00:47:42.727822 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:42.736050 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_216_43, total 2948 rows starting from the beginning of the part 2024.10.22 00:47:42.861694 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_217_217_0, total 12 rows starting from the beginning of the part 2024.10.22 00:47:42.885828 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_218_218_0, total 18 rows starting from the beginning of the part 2024.10.22 00:47:42.912817 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_219_219_0, total 20 rows starting from the beginning of the part 2024.10.22 00:47:42.961585 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_220_220_0, total 11 rows starting from the beginning of the part 2024.10.22 00:47:42.998483 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_221_221_0, total 18 rows starting from the beginning of the part 2024.10.22 00:47:43.246922 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 20132 2024.10.22 00:47:43.587309 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:43.590189 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:43.649161 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_356_356_0 (state Temporary) 2024.10.22 00:47:43.726751 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_356_356_0 to 202410_369_369_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:43.789825 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20132 2024.10.22 00:47:43.834420 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:43.840804 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:44.041196 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 185 2024.10.22 00:47:44.098666 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_43_43_0, total 39 rows starting from the beginning of the part 2024.10.22 00:47:44.268767 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:44.272878 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:44.460086 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_167_167_0 (state Temporary) 2024.10.22 00:47:44.522946 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_167_167_0 to 202410_177_177_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:44.720556 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 185 2024.10.22 00:47:44.847755 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 1.83 MiB, will set to 1.66 GiB (RSS), difference: -4.56 MiB 2024.10.22 00:47:44.919771 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} MergeTask::MergeProjectionsStage: Merge sorted 3027 rows, containing 12 columns (12 merged, 0 gathered) in 2.247558872 sec., 1346.7945323747676 rows/sec., 209.81 KiB/sec. 2024.10.22 00:47:45.176836 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} MergedBlockOutputStream: filled checksums 202410_1_221_44 (state Temporary) 2024.10.22 00:47:45.417606 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(76ea6eece23bcbb309d8d41e310572f2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 76ea6eece23bcbb309d8d41e310572f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:47:45.423014 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 355500, file segment info: File segment: [0, 33554431], key: 76ea6eece23bcbb309d8d41e310572f2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:47:45.432495 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:47:45.552419 [ 1733 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vbo/twhlrjmqorxwehyeldpfjgccguzyz, size 355500 2024.10.22 00:47:45.771293 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Params 2024.10.22 00:47:45.780381 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Params 2024.10.22 00:47:45.784621 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_221_44 to 202410_1_221_44 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:45.787681 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_221_44} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_216_43, 202410_221_221_0] -> 202410_1_221_44 2024.10.22 00:47:45.794872 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Params 2024.10.22 00:47:45.854444 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:47:46.051244 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Params 2024.10.22 00:47:46.073541 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Params 2024.10.22 00:47:46.142643 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 344996 2024.10.22 00:47:46.190905 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Params 2024.10.22 00:47:46.287846 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_44_44_0, total 41 rows starting from the beginning of the part 2024.10.22 00:47:46.356720 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Params 2024.10.22 00:47:46.398378 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mle/rgcyjafqgecqzwhmzkampwynkvauv (LogSeriesLimiter: on interval from 2024-10-22 00:47:39 to 2024-10-22 00:47:46 accepted series 1 / 105 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:47:46.682952 [ 1237 ] {} DatabaseCatalog: Execution took 317 ms. 2024.10.22 00:47:46.812694 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_42_42_0 (state Temporary) 2024.10.22 00:47:47.116185 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:47.122349 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:47.410248 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 59 entries to flush up to offset 13371 2024.10.22 00:47:47.455360 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_199_199_0 (state Temporary) 2024.10.22 00:47:47.538402 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_199_199_0 to 202410_206_206_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:47.556421 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:47.559761 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:47.564984 [ 1258 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_201_40 to 202410_206_206_0 2024.10.22 00:47:47.575394 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 344996 2024.10.22 00:47:47.585400 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_207_207_0 (state Temporary) 2024.10.22 00:47:47.633142 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 3.04 MiB, will set to 1.67 GiB (RSS), difference: -2.87 MiB 2024.10.22 00:47:47.680285 [ 1258 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:47.698515 [ 1258 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:47.740098 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_201_40 to 202410_206_206_0 into Compact with storage Full 2024.10.22 00:47:47.804731 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 20183 2024.10.22 00:47:47.819620 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_207_207_0 to 202410_213_213_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:47.873445 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13371 2024.10.22 00:47:47.875393 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:47.884652 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} MergeTreeSequentialSource: Reading 44 marks from part 202410_1_201_40, total 350138 rows starting from the beginning of the part 2024.10.22 00:47:47.911043 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_202_202_0, total 1877 rows starting from the beginning of the part 2024.10.22 00:47:47.940710 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_203_203_0, total 1579 rows starting from the beginning of the part 2024.10.22 00:47:47.967808 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_204_204_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:47:47.977202 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_205_205_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:47:48.009621 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_206_206_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:47:48.084531 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:48.087210 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:48.163104 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_357_357_0 (state Temporary) 2024.10.22 00:47:48.187324 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_357_357_0 to 202410_370_370_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:48.270000 [ 1260 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 00:47:48.363590 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20183 2024.10.22 00:47:48.497146 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_42_42_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:47:49.416678 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 3.97 MiB, will set to 1.68 GiB (RSS), difference: 1.92 MiB 2024.10.22 00:47:49.594842 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:47:49.599247 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:47:49.887831 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 00:47:50.121822 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 2934 2024.10.22 00:47:50.349265 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:50.356486 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:50.487371 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_209_209_0 (state Temporary) 2024.10.22 00:47:50.549310 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_209_209_0 to 202410_222_222_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:50.704798 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2934 2024.10.22 00:47:51.435061 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 7.21 MiB, will set to 1.69 GiB (RSS), difference: 5.53 MiB 2024.10.22 00:47:51.583682 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_206_41/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:47:52.225681 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 186 2024.10.22 00:47:52.375887 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 20213 2024.10.22 00:47:52.620554 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:52.628958 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:52.662800 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:52.666560 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:52.702456 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_168_168_0 (state Temporary) 2024.10.22 00:47:52.820948 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_358_358_0 (state Temporary) 2024.10.22 00:47:52.880160 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_168_168_0 to 202410_178_178_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:52.957529 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_358_358_0 to 202410_371_371_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:52.982662 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 5 parts from 202410_1_366_77 to 202410_370_370_0 2024.10.22 00:47:52.991083 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:52.993916 [ 1171 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:53.053301 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20213 2024.10.22 00:47:53.068598 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 186 2024.10.22 00:47:53.160095 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_370_78} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_366_77 to 202410_370_370_0 into Compact with storage Full 2024.10.22 00:47:53.188772 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_370_78} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:53.198145 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_370_78} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_366_77, total 20828 rows starting from the beginning of the part 2024.10.22 00:47:53.217724 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_370_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_367_367_0, total 28 rows starting from the beginning of the part 2024.10.22 00:47:53.256745 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_370_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_368_368_0, total 59 rows starting from the beginning of the part 2024.10.22 00:47:53.271985 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_370_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_369_369_0, total 48 rows starting from the beginning of the part 2024.10.22 00:47:53.294794 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_370_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_370_370_0, total 51 rows starting from the beginning of the part 2024.10.22 00:47:53.468349 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 9.52 MiB, will set to 1.70 GiB (RSS), difference: 8.81 MiB 2024.10.22 00:47:54.186376 [ 1178 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:47:54.258227 [ 1178 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:47:54.493817 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_370_78/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:47:54.669777 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 346532 2024.10.22 00:47:55.384121 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 106 entries to flush up to offset 13477 2024.10.22 00:47:55.510663 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 9.54 MiB, will set to 1.71 GiB (RSS), difference: -3.85 MiB 2024.10.22 00:47:55.777838 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:55.781663 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:56.030929 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_200_200_0 (state Temporary) 2024.10.22 00:47:56.064308 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_370_78} MergeTask::MergeProjectionsStage: Merge sorted 21014 rows, containing 13 columns (13 merged, 0 gathered) in 3.015905501 sec., 6967.724947957512 rows/sec., 1.04 MiB/sec. 2024.10.22 00:47:56.147465 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_370_78} MergedBlockOutputStream: filled checksums 202410_1_370_78 (state Temporary) 2024.10.22 00:47:56.176476 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:56.182897 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:56.261149 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_200_200_0 to 202410_207_207_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:56.373768 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 346532 2024.10.22 00:47:56.464665 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_208_208_0 (state Temporary) 2024.10.22 00:47:56.575939 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_208_208_0 to 202410_214_214_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:56.637521 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13477 2024.10.22 00:47:56.625182 [ 1189 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_209_44 to 202410_214_214_0 2024.10.22 00:47:56.660475 [ 1189 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:56.666871 [ 1189 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:56.752456 [ 1185 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 217 ms. 2024.10.22 00:47:56.786749 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_370_78} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_370_78 to 202410_1_370_78 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:56.793259 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_209_44 to 202410_214_214_0 into Compact with storage Full 2024.10.22 00:47:56.799020 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_370_78} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 5 parts: [202410_1_366_77, 202410_370_370_0] -> 202410_1_370_78 2024.10.22 00:47:56.834436 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:47:56.866668 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.71 MiB. 2024.10.22 00:47:56.869473 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_209_44, total 13215 rows starting from the beginning of the part 2024.10.22 00:47:56.878953 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_210_210_0, total 66 rows starting from the beginning of the part 2024.10.22 00:47:56.887370 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_42_42_0 to 202410_45_45_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:56.899498 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_211_211_0, total 58 rows starting from the beginning of the part 2024.10.22 00:47:56.929049 [ 1187 ] {} DatabaseCatalog: Execution took 244 ms. 2024.10.22 00:47:57.033268 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_212_212_0, total 77 rows starting from the beginning of the part 2024.10.22 00:47:57.062816 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 20268 2024.10.22 00:47:57.070177 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_213_213_0, total 59 rows starting from the beginning of the part 2024.10.22 00:47:57.183956 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_214_214_0, total 106 rows starting from the beginning of the part 2024.10.22 00:47:57.201664 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:57.212583 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.51 GiB. 2024.10.22 00:47:57.286646 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_359_359_0 (state Temporary) 2024.10.22 00:47:57.440837 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_359_359_0 to 202410_372_372_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:57.611772 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20268 2024.10.22 00:47:57.674225 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 8.56 MiB, will set to 1.71 GiB (RSS), difference: 5.16 MiB 2024.10.22 00:47:58.208959 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 2947 2024.10.22 00:47:58.586055 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:47:58.591408 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.50 GiB. 2024.10.22 00:47:58.671101 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_210_210_0 (state Temporary) 2024.10.22 00:47:58.769416 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_210_210_0 to 202410_223_223_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:58.896630 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2947 2024.10.22 00:47:59.144929 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 4 old parts to remove. Parts: [202410_1_150_30, 202410_151_151_0, 202410_152_152_0, 202410_153_153_0] 2024.10.22 00:47:59.155385 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from filesystem (serially): Parts: [202410_1_150_30, 202410_151_151_0, 202410_152_152_0, 202410_153_153_0] 2024.10.22 00:47:59.172752 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} MergeTask::MergeProjectionsStage: Merge sorted 13581 rows, containing 13 columns (13 merged, 0 gathered) in 2.395760725 sec., 5668.763102375342 rows/sec., 2.38 MiB/sec. 2024.10.22 00:47:59.184800 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from memory: Parts: [202410_1_150_30, 202410_151_151_0, 202410_152_152_0, 202410_153_153_0] 2024.10.22 00:47:59.301305 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} MergedBlockOutputStream: filled checksums 202410_1_214_45 (state Temporary) 2024.10.22 00:47:59.540072 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_214_45 to 202410_1_214_45 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:47:59.553315 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_214_45} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_209_44, 202410_214_214_0] -> 202410_1_214_45 2024.10.22 00:47:59.593791 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:47:59.658299 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.67 MiB. 2024.10.22 00:47:59.678402 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:47:59.919096 [ 1734 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/sxm/jdipbkbkvtrofeizdwmkutustatxm, size 141 2024.10.22 00:48:00.032043 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.88 GiB, free memory in arenas 11.46 MiB, will set to 1.70 GiB (RSS), difference: 193.02 MiB 2024.10.22 00:48:00.671405 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 187 2024.10.22 00:48:00.833655 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(77b288dbc3877447fa79e7cde8fbb95c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 77b288dbc3877447fa79e7cde8fbb95c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:48:00.848440 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 77b288dbc3877447fa79e7cde8fbb95c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:48:00.851857 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:48:00.911667 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1665 2024.10.22 00:48:01.100013 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:01.110624 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.48 GiB. 2024.10.22 00:48:01.268087 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_169_169_0 (state Temporary) 2024.10.22 00:48:01.559572 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_169_169_0 to 202410_179_179_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:01.577588 [ 1161 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_174_36 to 202410_179_179_0 2024.10.22 00:48:01.586664 [ 1161 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:01.592671 [ 1161 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.48 GiB. 2024.10.22 00:48:01.640613 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 20307 2024.10.22 00:48:01.651809 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_174_36 to 202410_179_179_0 into Compact with storage Full 2024.10.22 00:48:01.676558 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 187 2024.10.22 00:48:01.815262 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_15_15_0 (state Temporary) 2024.10.22 00:48:01.854024 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gcc/zqugthxmjnzhrpzgnvcvsytaqqnmy (LogSeriesLimiter: on interval from 2024-10-22 00:47:46 to 2024-10-22 00:48:01 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:48:01.951929 [ 1341 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gsk/emfiqjgqrzbhajzimijqqxuxubzbe, size 365 2024.10.22 00:48:01.991559 [ 1676 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gcc/zqugthxmjnzhrpzgnvcvsytaqqnmy, size 4 2024.10.22 00:48:01.996470 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:01.999902 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.47 GiB. 2024.10.22 00:48:02.077149 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:48:02.090605 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_174_36, total 218 rows starting from the beginning of the part 2024.10.22 00:48:02.176149 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_360_360_0 (state Temporary) 2024.10.22 00:48:02.197471 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_175_175_0, total 1 rows starting from the beginning of the part 2024.10.22 00:48:02.247340 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_360_360_0 to 202410_373_373_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:02.390835 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20307 2024.10.22 00:48:02.471377 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.88 GiB, free memory in arenas 16.78 MiB, will set to 1.67 GiB (RSS), difference: 145.62 MiB 2024.10.22 00:48:02.529826 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_206_41/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:48:02.632767 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} MergeTask::MergeProjectionsStage: Merge sorted 358586 rows, containing 4 columns (4 merged, 0 gathered) in 14.903805824 sec., 24060.028977468242 rows/sec., 353.00 KiB/sec. 2024.10.22 00:48:02.672405 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_176_176_0, total 1 rows starting from the beginning of the part 2024.10.22 00:48:02.954632 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_44_9/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:48:02.986780 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_177_177_0, total 1 rows starting from the beginning of the part 2024.10.22 00:48:02.989377 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} MergedBlockOutputStream: filled checksums 202410_1_206_41 (state Temporary) 2024.10.22 00:48:03.076232 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_178_178_0, total 1 rows starting from the beginning of the part 2024.10.22 00:48:03.152996 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_179_179_0, total 1 rows starting from the beginning of the part 2024.10.22 00:48:03.184084 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_206_41 to 202410_1_206_41 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:03.186328 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_206_41} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_201_40, 202410_206_206_0] -> 202410_1_206_41 2024.10.22 00:48:03.235990 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:48:03.379633 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 348068 2024.10.22 00:48:03.474340 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_266_53, 202410_267_267_0, 202410_268_268_0, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0] 2024.10.22 00:48:03.476916 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_266_53, 202410_267_267_0, 202410_268_268_0, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0] 2024.10.22 00:48:03.504445 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_266_53, 202410_267_267_0, 202410_268_268_0, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0] 2024.10.22 00:48:04.255767 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 152 entries to flush up to offset 13629 2024.10.22 00:48:04.320246 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:04.323839 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.47 GiB. 2024.10.22 00:48:04.411212 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 13.72 MiB, will set to 1.68 GiB (RSS), difference: -1.10 MiB 2024.10.22 00:48:04.486928 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:04.490889 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.47 GiB. 2024.10.22 00:48:04.553776 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_209_209_0 (state Temporary) 2024.10.22 00:48:04.571839 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_201_201_0 (state Temporary) 2024.10.22 00:48:04.664176 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} MergeTask::MergeProjectionsStage: Merge sorted 223 rows, containing 32 columns (32 merged, 0 gathered) in 3.04533872 sec., 73.22666557104688 rows/sec., 7.80 KiB/sec. 2024.10.22 00:48:04.730760 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_209_209_0 to 202410_215_215_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:04.797037 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13629 2024.10.22 00:48:04.815290 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} MergedBlockOutputStream: filled checksums 202410_1_179_37 (state Temporary) 2024.10.22 00:48:04.890946 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_201_201_0 to 202410_208_208_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:04.947534 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 348068 2024.10.22 00:48:05.523873 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_179_37 to 202410_1_179_37 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:05.529138 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_179_37} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_174_36, 202410_179_179_0] -> 202410_1_179_37 2024.10.22 00:48:05.570329 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:48:06.415527 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 2973 2024.10.22 00:48:06.503322 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 20355 2024.10.22 00:48:06.551208 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_15_15_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:48:06.688679 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 12.66 MiB, will set to 1.68 GiB (RSS), difference: 6.75 MiB 2024.10.22 00:48:06.782055 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:06.785475 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.47 GiB. 2024.10.22 00:48:06.849201 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_361_361_0 (state Temporary) 2024.10.22 00:48:06.863787 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:06.867972 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.47 GiB. 2024.10.22 00:48:06.920372 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_211_211_0 (state Temporary) 2024.10.22 00:48:07.010801 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_361_361_0 to 202410_374_374_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:07.090932 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_211_211_0 to 202410_224_224_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:07.169295 [ 1151 ] {} DatabaseCatalog: Execution took 238 ms. 2024.10.22 00:48:07.177479 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20355 2024.10.22 00:48:07.201521 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2973 2024.10.22 00:48:07.277338 [ 1691 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mle/rgcyjafqgecqzwhmzkampwynkvauv, size 2147943 2024.10.22 00:48:07.791941 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 00:48:07.804419 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 00:48:07.837267 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 00:48:07.954561 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 00:48:07.967623 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 00:48:07.980331 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 00:48:07.991373 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 00:48:08.365616 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_146_29, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:48:08.370379 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_146_29, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:48:08.376777 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pey/mharanpaqmoajmttyfkvfxwiminre (LogSeriesLimiter: on interval from 2024-10-22 00:48:01 to 2024-10-22 00:48:08 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:48:08.528002 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_146_29, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:48:08.578678 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 1707 2024.10.22 00:48:08.652208 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.077783211 sec. 2024.10.22 00:48:09.265532 [ 1261 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:48:09.304223 [ 1261 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:48:09.594135 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:48:09.610842 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:48:10.570847 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 7.81 MiB, will set to 1.67 GiB (RSS), difference: -5.26 MiB 2024.10.22 00:48:11.183397 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 20389 2024.10.22 00:48:11.517273 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:11.523440 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:11.572414 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_362_362_0 (state Temporary) 2024.10.22 00:48:11.733810 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_362_362_0 to 202410_375_375_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:11.820304 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20389 2024.10.22 00:48:11.960747 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 349604 2024.10.22 00:48:11.972942 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 5 parts from 202410_1_370_78 to 202410_374_374_0 2024.10.22 00:48:11.988490 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:11.995014 [ 1271 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:12.006179 [ 1271 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 249 ms. 2024.10.22 00:48:12.049743 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_374_79} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_370_78 to 202410_374_374_0 into Compact with storage Full 2024.10.22 00:48:12.258901 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_374_79} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:48:12.266516 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_374_79} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_370_78, total 21014 rows starting from the beginning of the part 2024.10.22 00:48:12.286461 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_374_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_371_371_0, total 30 rows starting from the beginning of the part 2024.10.22 00:48:12.346579 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 47 entries to flush up to offset 13676 2024.10.22 00:48:12.440984 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_374_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_372_372_0, total 55 rows starting from the beginning of the part 2024.10.22 00:48:12.516864 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 7.69 MiB, will set to 1.67 GiB (RSS), difference: 3.28 MiB 2024.10.22 00:48:12.553593 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:12.557276 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:12.577709 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_374_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_373_373_0, total 39 rows starting from the beginning of the part 2024.10.22 00:48:12.590209 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_210_210_0 (state Temporary) 2024.10.22 00:48:12.704185 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_374_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_374_374_0, total 48 rows starting from the beginning of the part 2024.10.22 00:48:12.762140 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_210_210_0 to 202410_216_216_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:12.813492 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13676 2024.10.22 00:48:13.037029 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:13.048994 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:13.304136 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_202_202_0 (state Temporary) 2024.10.22 00:48:13.505004 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_202_202_0 to 202410_209_209_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:13.827866 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 349604 2024.10.22 00:48:14.634351 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 8.63 MiB, will set to 1.68 GiB (RSS), difference: 1.45 MiB 2024.10.22 00:48:15.118988 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_374_79} MergeTask::MergeProjectionsStage: Merge sorted 21186 rows, containing 13 columns (13 merged, 0 gathered) in 3.08124606 sec., 6875.789725147754 rows/sec., 1.03 MiB/sec. 2024.10.22 00:48:15.231776 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 2978 2024.10.22 00:48:15.295708 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_374_79} MergedBlockOutputStream: filled checksums 202410_1_374_79 (state Temporary) 2024.10.22 00:48:15.365740 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:15.370782 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:15.497405 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_212_212_0 (state Temporary) 2024.10.22 00:48:15.570497 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_374_79} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_374_79 to 202410_1_374_79 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:15.573531 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_374_79} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 5 parts: [202410_1_370_78, 202410_374_374_0] -> 202410_1_374_79 2024.10.22 00:48:15.606782 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.71 MiB. 2024.10.22 00:48:15.660814 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_212_212_0 to 202410_225_225_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:15.698161 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2978 2024.10.22 00:48:15.945322 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 20431 2024.10.22 00:48:16.282967 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:16.386541 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:16.417811 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_363_363_0 (state Temporary) 2024.10.22 00:48:16.469365 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_363_363_0 to 202410_376_376_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:16.679429 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 188 2024.10.22 00:48:16.779118 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20431 2024.10.22 00:48:16.788663 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 6.09 MiB, will set to 1.67 GiB (RSS), difference: -3.49 MiB 2024.10.22 00:48:17.094794 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(535d5511e222b0a3092f01fea7bce0d3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 535d5511e222b0a3092f01fea7bce0d3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 00:48:17.108822 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 535d5511e222b0a3092f01fea7bce0d3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 00:48:17.115066 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:48:17.376716 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:17.379605 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:17.475461 [ 1169 ] {} DatabaseCatalog: Execution took 335 ms. 2024.10.22 00:48:17.552289 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_170_170_0 (state Temporary) 2024.10.22 00:48:17.921200 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_170_170_0 to 202410_180_180_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:17.996784 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 188 2024.10.22 00:48:19.467907 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rmf/yslnhhowyadzqzbrnocvlmrtyizoc. 2024.10.22 00:48:19.596494 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:48:19.602952 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:48:19.624070 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_44_9/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:48:20.328216 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 62 entries to flush up to offset 13738 2024.10.22 00:48:20.473370 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 6.54 MiB, will set to 1.67 GiB (RSS), difference: 2.56 MiB 2024.10.22 00:48:20.740404 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:20.742961 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:20.768314 [ 1205 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 00:48:20.789763 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_211_211_0 (state Temporary) 2024.10.22 00:48:20.832294 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 351524 2024.10.22 00:48:20.887507 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 20457 2024.10.22 00:48:21.005694 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:21.010311 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:21.118210 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_211_211_0 to 202410_217_217_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:21.152003 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_364_364_0 (state Temporary) 2024.10.22 00:48:21.184008 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_364_364_0 to 202410_377_377_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:21.199419 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13738 2024.10.22 00:48:21.336782 [ 1175 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 00:48:21.350297 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20457 2024.10.22 00:48:21.364682 [ 1217 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 00:48:21.375115 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 6.19 MiB, will set to 1.67 GiB (RSS), difference: -2.82 MiB 2024.10.22 00:48:21.958047 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:21.963889 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:22.043232 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_161_32, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0] 2024.10.22 00:48:22.048138 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_161_32, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0] 2024.10.22 00:48:22.085324 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_161_32, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0] 2024.10.22 00:48:22.292771 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_203_203_0 (state Temporary) 2024.10.22 00:48:22.497449 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_203_203_0 to 202410_210_210_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:22.538838 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 351524 2024.10.22 00:48:23.201281 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 2990 2024.10.22 00:48:23.328536 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:23.334425 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:23.404039 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 6.90 MiB, will set to 1.67 GiB (RSS), difference: 2.49 MiB 2024.10.22 00:48:23.485297 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_213_213_0 (state Temporary) 2024.10.22 00:48:23.744626 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_213_213_0 to 202410_226_226_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:23.770863 [ 1235 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_221_44 to 202410_226_226_0 2024.10.22 00:48:23.778844 [ 1235 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:23.781420 [ 1235 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:23.818750 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_221_44 to 202410_226_226_0 into Compact with storage Full 2024.10.22 00:48:23.895230 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:48:23.896064 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 2990 2024.10.22 00:48:23.906972 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_221_44, total 3027 rows starting from the beginning of the part 2024.10.22 00:48:23.951119 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_222_222_0, total 13 rows starting from the beginning of the part 2024.10.22 00:48:23.975284 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_223_223_0, total 13 rows starting from the beginning of the part 2024.10.22 00:48:23.995093 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_224_224_0, total 26 rows starting from the beginning of the part 2024.10.22 00:48:24.034651 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_225_225_0, total 5 rows starting from the beginning of the part 2024.10.22 00:48:24.093433 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_226_226_0, total 12 rows starting from the beginning of the part 2024.10.22 00:48:24.313407 [ 1191 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:48:24.444828 [ 1191 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:48:24.456189 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 5 old parts to remove. Parts: [202410_1_153_31, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0, 202410_157_157_0] 2024.10.22 00:48:24.461119 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from filesystem (serially): Parts: [202410_1_153_31, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0, 202410_157_157_0] 2024.10.22 00:48:24.487054 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from memory: Parts: [202410_1_153_31, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0, 202410_157_157_0] 2024.10.22 00:48:25.148954 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} MergeTask::MergeProjectionsStage: Merge sorted 3096 rows, containing 12 columns (12 merged, 0 gathered) in 1.342297995 sec., 2306.4923076190694 rows/sec., 359.38 KiB/sec. 2024.10.22 00:48:25.156639 [ 1684 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pey/mharanpaqmoajmttyfkvfxwiminre, size 2027602 2024.10.22 00:48:25.324791 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} MergedBlockOutputStream: filled checksums 202410_1_226_45 (state Temporary) 2024.10.22 00:48:25.356294 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 20504 2024.10.22 00:48:25.473572 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:25.477407 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:25.505809 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 189 2024.10.22 00:48:25.603581 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 5.61 MiB, will set to 1.67 GiB (RSS), difference: -2.53 MiB 2024.10.22 00:48:25.612130 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 00:48:25.623193 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 00:48:25.648928 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 00:48:25.651422 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_365_365_0 (state Temporary) 2024.10.22 00:48:25.670611 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_271_54, 202410_272_272_0, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0] 2024.10.22 00:48:25.673361 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_271_54, 202410_272_272_0, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0] 2024.10.22 00:48:25.673504 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 00:48:25.699836 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_226_45 to 202410_1_226_45 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:25.708198 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 00:48:25.710460 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_226_45} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_221_44, 202410_226_226_0] -> 202410_1_226_45 2024.10.22 00:48:25.716871 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_365_365_0 to 202410_378_378_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:25.768607 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_271_54, 202410_272_272_0, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0] 2024.10.22 00:48:25.851999 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 00:48:25.872477 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 00:48:25.879256 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:48:25.886732 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20504 2024.10.22 00:48:25.909498 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/sdx/qcrhvrabqfaqzwpmsviahefjllxgi (LogSeriesLimiter: on interval from 2024-10-22 00:48:08 to 2024-10-22 00:48:25 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:48:26.388570 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:26.392413 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:26.589415 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_171_171_0 (state Temporary) 2024.10.22 00:48:26.675125 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_171_171_0 to 202410_181_181_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:26.885294 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 189 2024.10.22 00:48:27.515291 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:27.520365 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:28.717037 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 47 entries to flush up to offset 13785 2024.10.22 00:48:29.047093 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:29.051509 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:29.183664 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_212_212_0 (state Temporary) 2024.10.22 00:48:29.321080 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_128_26, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0, 202410_132_132_0] 2024.10.22 00:48:29.322125 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_212_212_0 to 202410_218_218_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:29.325276 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_128_26, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0, 202410_132_132_0] 2024.10.22 00:48:29.370201 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13785 2024.10.22 00:48:29.456849 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_128_26, 202410_129_129_0, 202410_130_130_0, 202410_131_131_0, 202410_132_132_0] 2024.10.22 00:48:29.550502 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 353060 2024.10.22 00:48:29.595759 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:48:29.597902 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:48:29.814363 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 5.18 MiB, will set to 1.67 GiB (RSS), difference: -6.56 MiB 2024.10.22 00:48:29.911211 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 20546 2024.10.22 00:48:30.077538 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:48:30.593209 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:30.613069 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:30.709124 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:30.715876 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:30.874712 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_366_366_0 (state Temporary) 2024.10.22 00:48:30.921060 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_366_366_0 to 202410_379_379_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:30.983377 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20546 2024.10.22 00:48:31.052976 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_204_204_0 (state Temporary) 2024.10.22 00:48:31.065417 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_374_79 to 202410_375_375_0 2024.10.22 00:48:31.071377 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:31.073468 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:31.128541 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_204_204_0 to 202410_211_211_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:31.203408 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_375_80} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_374_79 to 202410_375_375_0 into Compact with storage Full 2024.10.22 00:48:31.215968 [ 1268 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_206_41 to 202410_211_211_0 2024.10.22 00:48:31.224569 [ 1268 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:31.229177 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:31.243030 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_375_80} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:48:31.263103 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_375_80} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_374_79, total 21186 rows starting from the beginning of the part 2024.10.22 00:48:31.281791 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_375_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_375_375_0, total 34 rows starting from the beginning of the part 2024.10.22 00:48:31.283415 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_206_41 to 202410_211_211_0 into Compact with storage Full 2024.10.22 00:48:31.316840 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 353060 2024.10.22 00:48:31.316531 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:48:31.333304 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} MergeTreeSequentialSource: Reading 45 marks from part 202410_1_206_41, total 358586 rows starting from the beginning of the part 2024.10.22 00:48:31.384811 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_207_207_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:48:31.398817 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 3003 2024.10.22 00:48:31.402538 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_208_208_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:48:31.426166 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_209_209_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:48:31.560531 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_210_210_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:48:31.580408 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_211_211_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:48:31.766605 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:31.772417 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:31.916297 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_214_214_0 (state Temporary) 2024.10.22 00:48:31.989265 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_214_214_0 to 202410_227_227_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:32.184880 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3003 2024.10.22 00:48:32.278221 [ 1155 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 251 ms. 2024.10.22 00:48:32.341405 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_43_43_0 (state Temporary) 2024.10.22 00:48:32.477046 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_375_80/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:48:33.648204 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 5.37 MiB, will set to 1.68 GiB (RSS), difference: -10.02 MiB 2024.10.22 00:48:33.795888 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_375_80} MergeTask::MergeProjectionsStage: Merge sorted 21220 rows, containing 13 columns (13 merged, 0 gathered) in 2.604537804 sec., 8147.318870707396 rows/sec., 1.22 MiB/sec. 2024.10.22 00:48:33.864652 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_375_80} MergedBlockOutputStream: filled checksums 202410_1_375_80 (state Temporary) 2024.10.22 00:48:34.128594 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_375_80} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_375_80 to 202410_1_375_80 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:34.132022 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_375_80} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_374_79, 202410_375_375_0] -> 202410_1_375_80 2024.10.22 00:48:34.168336 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.71 MiB. 2024.10.22 00:48:34.387545 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 190 2024.10.22 00:48:34.886845 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:34.891120 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:34.988284 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 20594 2024.10.22 00:48:35.266109 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_172_172_0 (state Temporary) 2024.10.22 00:48:35.372330 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_172_172_0 to 202410_182_182_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:35.381800 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:35.385918 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:35.453398 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 190 2024.10.22 00:48:35.668168 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_367_367_0 (state Temporary) 2024.10.22 00:48:35.784588 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_367_367_0 to 202410_380_380_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:35.825370 [ 1164 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_375_80 to 202410_376_376_0 2024.10.22 00:48:35.835805 [ 1164 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:35.841231 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:35.985118 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_376_81} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_375_80 to 202410_376_376_0 into Compact with storage Full 2024.10.22 00:48:36.005739 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_376_81} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:48:36.015543 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_376_81} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_375_80, total 21220 rows starting from the beginning of the part 2024.10.22 00:48:36.023070 [ 1238 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 00:48:36.070574 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20594 2024.10.22 00:48:36.069807 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_376_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_376_376_0, total 42 rows starting from the beginning of the part 2024.10.22 00:48:36.379506 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_211_42/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:48:36.882437 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 106 entries to flush up to offset 13891 2024.10.22 00:48:36.976495 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_43_43_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:48:37.139049 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:37.142760 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:37.203976 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_213_213_0 (state Temporary) 2024.10.22 00:48:37.367713 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_213_213_0 to 202410_219_219_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:37.389619 [ 1178 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_214_45 to 202410_219_219_0 2024.10.22 00:48:37.466476 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13891 2024.10.22 00:48:37.510834 [ 1178 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:37.515141 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:37.564606 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_214_45 to 202410_219_219_0 into Compact with storage Full 2024.10.22 00:48:37.577395 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_376_81} MergeTask::MergeProjectionsStage: Merge sorted 21262 rows, containing 13 columns (13 merged, 0 gathered) in 1.597366555 sec., 13310.65805368637 rows/sec., 1.99 MiB/sec. 2024.10.22 00:48:37.602199 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:48:37.616606 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_214_45, total 13581 rows starting from the beginning of the part 2024.10.22 00:48:37.639641 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_215_215_0, total 152 rows starting from the beginning of the part 2024.10.22 00:48:37.671557 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_216_216_0, total 47 rows starting from the beginning of the part 2024.10.22 00:48:37.691602 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_217_217_0, total 62 rows starting from the beginning of the part 2024.10.22 00:48:37.734892 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_218_218_0, total 47 rows starting from the beginning of the part 2024.10.22 00:48:37.778077 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_219_219_0, total 106 rows starting from the beginning of the part 2024.10.22 00:48:37.830166 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 10.80 MiB, will set to 1.68 GiB (RSS), difference: 5.31 MiB 2024.10.22 00:48:37.887414 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_376_81} MergedBlockOutputStream: filled checksums 202410_1_376_81 (state Temporary) 2024.10.22 00:48:38.099956 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_376_81} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_376_81 to 202410_1_376_81 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:38.133027 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_376_81} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_375_80, 202410_376_376_0] -> 202410_1_376_81 2024.10.22 00:48:38.183439 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.71 MiB. 2024.10.22 00:48:38.322112 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1727 entries to flush up to offset 354787 2024.10.22 00:48:39.458443 [ 1185 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:48:39.480204 [ 1185 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:48:39.600800 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:48:39.613524 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:48:39.631262 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.88 GiB, free memory in arenas 12.32 MiB, will set to 1.69 GiB (RSS), difference: 82.68 MiB 2024.10.22 00:48:39.687493 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 3013 2024.10.22 00:48:39.795310 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:39.795293 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rmf/yslnhhowyadzqzbrnocvlmrtyizoc. 2024.10.22 00:48:39.797712 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:39.848473 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(927ceebccecc3160f7e9ec9243418846) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 927ceebccecc3160f7e9ec9243418846, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:48:39.853691 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 927ceebccecc3160f7e9ec9243418846, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:48:39.860008 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:48:39.971295 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} MergeTask::MergeProjectionsStage: Merge sorted 13995 rows, containing 13 columns (13 merged, 0 gathered) in 2.42304183 sec., 5775.797935770676 rows/sec., 2.42 MiB/sec. 2024.10.22 00:48:40.014741 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_205_205_0 (state Temporary) 2024.10.22 00:48:40.045600 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} MergedBlockOutputStream: filled checksums 202410_1_219_46 (state Temporary) 2024.10.22 00:48:40.082661 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 20652 2024.10.22 00:48:40.084042 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_219_46 to 202410_1_219_46 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:40.093426 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_219_46} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_214_45, 202410_219_219_0] -> 202410_1_219_46 2024.10.22 00:48:40.133330 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:40.138544 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:40.263158 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.68 MiB. 2024.10.22 00:48:40.282888 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_215_215_0 (state Temporary) 2024.10.22 00:48:40.320337 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_205_205_0 to 202410_212_212_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:40.362398 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_215_215_0 to 202410_228_228_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:40.472494 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:40.474666 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:40.483558 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 354787 2024.10.22 00:48:40.606664 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3013 2024.10.22 00:48:40.748099 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_368_368_0 (state Temporary) 2024.10.22 00:48:40.785214 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_368_368_0 to 202410_381_381_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:40.800929 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_376_81 to 202410_381_381_0 2024.10.22 00:48:40.826644 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:40.834724 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:40.923597 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20652 2024.10.22 00:48:41.187890 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_376_81 to 202410_381_381_0 into Compact with storage Full 2024.10.22 00:48:41.333676 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:48:41.347299 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_376_81, total 21262 rows starting from the beginning of the part 2024.10.22 00:48:41.370542 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_377_377_0, total 26 rows starting from the beginning of the part 2024.10.22 00:48:41.386133 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_378_378_0, total 47 rows starting from the beginning of the part 2024.10.22 00:48:41.411838 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_379_379_0, total 42 rows starting from the beginning of the part 2024.10.22 00:48:41.432703 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.88 GiB, free memory in arenas 8.90 MiB, will set to 1.67 GiB (RSS), difference: 102.80 MiB 2024.10.22 00:48:41.434171 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_380_380_0, total 48 rows starting from the beginning of the part 2024.10.22 00:48:41.471011 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_381_381_0, total 58 rows starting from the beginning of the part 2024.10.22 00:48:41.494701 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_43_43_0 to 202410_46_46_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:42.587630 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8b07ca41f36cbaffad50dc90200ccf37) : [0, 2303664]: Setting download as failed: File segment: [0, 2303664], key: 8b07ca41f36cbaffad50dc90200ccf37, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:48:42.592735 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8b07ca41f36cbaffad50dc90200ccf37): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:48:42.671602 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8b07ca41f36cbaffad50dc90200ccf37, offset: 0, size: 0 2024.10.22 00:48:42.688197 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8b07ca41f36cbaffad50dc90200ccf37 for removal 2024.10.22 00:48:42.697135 [ 1720 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/sdx/qcrhvrabqfaqzwpmsviahefjllxgi, size 1217502 2024.10.22 00:48:42.698831 [ 1476 ] {} CacheMetadata: Key 8b07ca41f36cbaffad50dc90200ccf37 is removed from metadata 2024.10.22 00:48:42.831205 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 9.27 MiB, will set to 1.67 GiB (RSS), difference: -2.94 MiB 2024.10.22 00:48:42.959212 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 191 2024.10.22 00:48:43.182402 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 00:48:43.186804 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 00:48:43.199745 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:43.202477 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:43.223867 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 00:48:43.267436 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 00:48:43.294501 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 00:48:43.307379 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_173_173_0 (state Temporary) 2024.10.22 00:48:43.352315 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_381_82/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:48:43.422396 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_173_173_0 to 202410_183_183_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:43.454433 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 00:48:43.467956 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 00:48:43.527361 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lwv/xhsqvgwyoqsxwpqcfaoigetpxquud (LogSeriesLimiter: on interval from 2024-10-22 00:48:25 to 2024-10-22 00:48:43 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:48:43.608251 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 191 2024.10.22 00:48:44.345491 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(440d3e368a1ca22e74ec51b5468e4b43) : [0, 1084076]: Setting download as failed: File segment: [0, 1084076], key: 440d3e368a1ca22e74ec51b5468e4b43, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:48:44.351572 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(440d3e368a1ca22e74ec51b5468e4b43): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:48:44.378780 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 440d3e368a1ca22e74ec51b5468e4b43, offset: 0, size: 0 2024.10.22 00:48:44.380266 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 440d3e368a1ca22e74ec51b5468e4b43 for removal 2024.10.22 00:48:44.382537 [ 1476 ] {} CacheMetadata: Key 440d3e368a1ca22e74ec51b5468e4b43 is removed from metadata 2024.10.22 00:48:44.421700 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 10.89 MiB, will set to 1.67 GiB (RSS), difference: 9.34 MiB 2024.10.22 00:48:44.704842 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} MergeTask::MergeProjectionsStage: Merge sorted 21483 rows, containing 13 columns (13 merged, 0 gathered) in 3.522476862 sec., 6098.833531528816 rows/sec., 932.85 KiB/sec. 2024.10.22 00:48:44.870216 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} MergedBlockOutputStream: filled checksums 202410_1_381_82 (state Temporary) 2024.10.22 00:48:44.976233 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 106 entries to flush up to offset 13997 2024.10.22 00:48:45.009154 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_381_82 to 202410_1_381_82 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:45.041456 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 20711 2024.10.22 00:48:45.048272 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_381_82} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_376_81, 202410_381_381_0] -> 202410_1_381_82 2024.10.22 00:48:45.157724 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1707 2024.10.22 00:48:45.190692 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.76 MiB. 2024.10.22 00:48:45.272040 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:45.274423 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:45.274873 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:45.278964 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:45.281017 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 48111 bytes for a446ffc8129c99ba84e8eb8ed1c7084f:0) 2024.10.22 00:48:45.282201 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1085aa794214826b06b2de3e8798a04e, offset: 0, size: 84153 2024.10.22 00:48:45.285192 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1085aa794214826b06b2de3e8798a04e for removal 2024.10.22 00:48:45.287191 [ 1476 ] {} CacheMetadata: Key 1085aa794214826b06b2de3e8798a04e is removed from metadata 2024.10.22 00:48:45.287663 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 88fdeaca499c121fbc3b9c12ef3c9ee8, offset: 0, size: 193 2024.10.22 00:48:45.290436 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 88fdeaca499c121fbc3b9c12ef3c9ee8 for removal 2024.10.22 00:48:45.296122 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6467f48c0b67a41abdf139adf646fc57, offset: 0, size: 164 2024.10.22 00:48:45.303261 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6467f48c0b67a41abdf139adf646fc57 for removal 2024.10.22 00:48:45.312317 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dfb655ceba986eeb051d897ecb7a6191, offset: 0, size: 4 2024.10.22 00:48:45.319470 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dfb655ceba986eeb051d897ecb7a6191 for removal 2024.10.22 00:48:45.330177 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_214_214_0 (state Temporary) 2024.10.22 00:48:45.419517 [ 1476 ] {} CacheMetadata: Key dfb655ceba986eeb051d897ecb7a6191 is removed from metadata 2024.10.22 00:48:45.441006 [ 1476 ] {} CacheMetadata: Key 6467f48c0b67a41abdf139adf646fc57 is removed from metadata 2024.10.22 00:48:45.460157 [ 1476 ] {} CacheMetadata: Key 88fdeaca499c121fbc3b9c12ef3c9ee8 is removed from metadata 2024.10.22 00:48:45.523557 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_214_214_0 to 202410_220_220_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:45.626575 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_369_369_0 (state Temporary) 2024.10.22 00:48:45.673654 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_369_369_0 to 202410_382_382_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:45.677405 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 13997 2024.10.22 00:48:45.740928 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20711 2024.10.22 00:48:46.069373 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} MergeTask::MergeProjectionsStage: Merge sorted 366650 rows, containing 4 columns (4 merged, 0 gathered) in 14.817729551 sec., 24744.00674800115 rows/sec., 362.99 KiB/sec. 2024.10.22 00:48:46.102705 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} MergedBlockOutputStream: filled checksums 202410_1_211_42 (state Temporary) 2024.10.22 00:48:46.177624 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_211_42 to 202410_1_211_42 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:46.179550 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_211_42} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_206_41, 202410_211_211_0] -> 202410_1_211_42 2024.10.22 00:48:46.209814 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:48:46.482667 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 22.50 MiB, will set to 1.67 GiB (RSS), difference: 17.18 MiB 2024.10.22 00:48:46.523375 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(40ef83627c7bdd8058134646e52f1b8c) : [0, 310565]: Setting download as failed: File segment: [0, 310565], key: 40ef83627c7bdd8058134646e52f1b8c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:48:46.525207 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(40ef83627c7bdd8058134646e52f1b8c): No space left in cache to reserve 310566 bytes, will continue without cache download 2024.10.22 00:48:46.644493 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 40ef83627c7bdd8058134646e52f1b8c, offset: 0, size: 0 2024.10.22 00:48:46.646150 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 40ef83627c7bdd8058134646e52f1b8c for removal 2024.10.22 00:48:46.650640 [ 1476 ] {} CacheMetadata: Key 40ef83627c7bdd8058134646e52f1b8c is removed from metadata 2024.10.22 00:48:47.488880 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2113 entries to flush up to offset 356900 2024.10.22 00:48:47.590615 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e9644d08bfb7e662ed4763af365707ad) : [0, 200433]: Setting download as failed: File segment: [0, 200433], key: e9644d08bfb7e662ed4763af365707ad, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:48:47.594487 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e9644d08bfb7e662ed4763af365707ad): No space left in cache to reserve 200434 bytes, will continue without cache download 2024.10.22 00:48:47.835493 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e9644d08bfb7e662ed4763af365707ad, offset: 0, size: 0 2024.10.22 00:48:47.837447 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e9644d08bfb7e662ed4763af365707ad for removal 2024.10.22 00:48:47.969518 [ 1476 ] {} CacheMetadata: Key e9644d08bfb7e662ed4763af365707ad is removed from metadata 2024.10.22 00:48:48.110220 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 3041 2024.10.22 00:48:48.179750 [ 1220 ] {} DatabaseCatalog: Execution took 357 ms. 2024.10.22 00:48:48.306689 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 5 old parts to remove. Parts: [202410_1_276_55, 202410_277_277_0, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0] 2024.10.22 00:48:48.415283 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from filesystem (serially): Parts: [202410_1_276_55, 202410_277_277_0, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0] 2024.10.22 00:48:48.437467 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:48.443435 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:48.465158 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 21.53 MiB, will set to 1.67 GiB (RSS), difference: -3.34 MiB 2024.10.22 00:48:48.479420 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_216_216_0 (state Temporary) 2024.10.22 00:48:48.651893 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_216_216_0 to 202410_229_229_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:48.662876 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from memory: Parts: [202410_1_276_55, 202410_277_277_0, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0] 2024.10.22 00:48:48.690615 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3041 2024.10.22 00:48:48.774309 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:48.776781 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:48.911002 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_206_206_0 (state Temporary) 2024.10.22 00:48:48.978878 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_206_206_0 to 202410_213_213_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:49.034700 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(09a84fcb359a9883c8891e1d29f51814) : [0, 435244]: Setting download as failed: File segment: [0, 435244], key: 09a84fcb359a9883c8891e1d29f51814, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:48:49.043311 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(09a84fcb359a9883c8891e1d29f51814): No space left in cache to reserve 435245 bytes, will continue without cache download 2024.10.22 00:48:49.074622 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 09a84fcb359a9883c8891e1d29f51814, offset: 0, size: 0 2024.10.22 00:48:49.076268 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 09a84fcb359a9883c8891e1d29f51814 for removal 2024.10.22 00:48:49.078631 [ 1476 ] {} CacheMetadata: Key 09a84fcb359a9883c8891e1d29f51814 is removed from metadata 2024.10.22 00:48:49.114258 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 356900 2024.10.22 00:48:49.605671 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:48:49.612009 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:48:49.773960 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 20780 2024.10.22 00:48:50.065717 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:50.072271 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:50.234687 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_370_370_0 (state Temporary) 2024.10.22 00:48:50.252491 [ 1227 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 00:48:50.376396 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_370_370_0 to 202410_383_383_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:50.423802 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20780 2024.10.22 00:48:50.809312 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 28 file segments (while reserving 78679 bytes for b150b81e530d68841259ca1996bd0d1c:0) 2024.10.22 00:48:50.932353 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1cb4ce1354aa0e871e62222ce5c9f008, offset: 0, size: 52706 2024.10.22 00:48:50.943128 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1cb4ce1354aa0e871e62222ce5c9f008 for removal 2024.10.22 00:48:50.947747 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e11df730070553f2681ace7ada529fd0, offset: 0, size: 212 2024.10.22 00:48:50.947911 [ 1476 ] {} CacheMetadata: Key 1cb4ce1354aa0e871e62222ce5c9f008 is removed from metadata 2024.10.22 00:48:50.958735 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e11df730070553f2681ace7ada529fd0 for removal 2024.10.22 00:48:50.963274 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5e1932fde18d43de9450d1d3dfd54ff4, offset: 0, size: 240 2024.10.22 00:48:50.966125 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5e1932fde18d43de9450d1d3dfd54ff4 for removal 2024.10.22 00:48:50.968399 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4caff12245cc7bbdbb541b5dd97f1808, offset: 0, size: 240 2024.10.22 00:48:50.972663 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4caff12245cc7bbdbb541b5dd97f1808 for removal 2024.10.22 00:48:50.974527 [ 1476 ] {} CacheMetadata: Key 5e1932fde18d43de9450d1d3dfd54ff4 is removed from metadata 2024.10.22 00:48:50.976902 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: daa577b14c06d31c9b550dd038395196, offset: 0, size: 255 2024.10.22 00:48:50.980221 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key daa577b14c06d31c9b550dd038395196 for removal 2024.10.22 00:48:50.983942 [ 1476 ] {} CacheMetadata: Key daa577b14c06d31c9b550dd038395196 is removed from metadata 2024.10.22 00:48:50.984215 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 770cc915e1f695ad90044bf5439b4628, offset: 0, size: 241 2024.10.22 00:48:50.987316 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 770cc915e1f695ad90044bf5439b4628 for removal 2024.10.22 00:48:50.989531 [ 1476 ] {} CacheMetadata: Key 770cc915e1f695ad90044bf5439b4628 is removed from metadata 2024.10.22 00:48:50.989681 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 135148d52add73f6689783343b8d5366, offset: 0, size: 156 2024.10.22 00:48:50.991816 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 135148d52add73f6689783343b8d5366 for removal 2024.10.22 00:48:50.994198 [ 1476 ] {} CacheMetadata: Key 135148d52add73f6689783343b8d5366 is removed from metadata 2024.10.22 00:48:50.994385 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5e2afee23df5ee48f22298603b0bfc16, offset: 0, size: 2468 2024.10.22 00:48:50.997499 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5e2afee23df5ee48f22298603b0bfc16 for removal 2024.10.22 00:48:51.004581 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 80e64ae382a65c5814a9d252b6d5c0b7, offset: 0, size: 247 2024.10.22 00:48:51.004986 [ 1476 ] {} CacheMetadata: Key 5e2afee23df5ee48f22298603b0bfc16 is removed from metadata 2024.10.22 00:48:51.013168 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 80e64ae382a65c5814a9d252b6d5c0b7 for removal 2024.10.22 00:48:51.018680 [ 1476 ] {} CacheMetadata: Key 80e64ae382a65c5814a9d252b6d5c0b7 is removed from metadata 2024.10.22 00:48:51.018078 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 32bbcfc841a35e3d4573e6b4a09d6e9e, offset: 0, size: 249 2024.10.22 00:48:51.024429 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 32bbcfc841a35e3d4573e6b4a09d6e9e for removal 2024.10.22 00:48:51.028780 [ 1476 ] {} CacheMetadata: Key 32bbcfc841a35e3d4573e6b4a09d6e9e is removed from metadata 2024.10.22 00:48:51.029293 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a5a84f026cd09d0c2ff2974a0989d8e8, offset: 0, size: 163 2024.10.22 00:48:51.032965 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a5a84f026cd09d0c2ff2974a0989d8e8 for removal 2024.10.22 00:48:51.039820 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9e395a4c98302ac450bbfafad9e59fa6, offset: 0, size: 252 2024.10.22 00:48:51.044267 [ 1476 ] {} CacheMetadata: Key a5a84f026cd09d0c2ff2974a0989d8e8 is removed from metadata 2024.10.22 00:48:51.044981 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9e395a4c98302ac450bbfafad9e59fa6 for removal 2024.10.22 00:48:51.057372 [ 1476 ] {} CacheMetadata: Key 9e395a4c98302ac450bbfafad9e59fa6 is removed from metadata 2024.10.22 00:48:51.056254 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5a45718a61e6e199d4ce1740ab96e8b4, offset: 0, size: 246 2024.10.22 00:48:51.062541 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5a45718a61e6e199d4ce1740ab96e8b4 for removal 2024.10.22 00:48:51.066921 [ 1476 ] {} CacheMetadata: Key 5a45718a61e6e199d4ce1740ab96e8b4 is removed from metadata 2024.10.22 00:48:51.065813 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0229a52b19f6361dda0850990045586d, offset: 0, size: 238 2024.10.22 00:48:51.069675 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0229a52b19f6361dda0850990045586d for removal 2024.10.22 00:48:51.073407 [ 1476 ] {} CacheMetadata: Key 0229a52b19f6361dda0850990045586d is removed from metadata 2024.10.22 00:48:51.073454 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6fe74f5f7eaa54677dd841ccf84678f7, offset: 0, size: 240 2024.10.22 00:48:51.077060 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6fe74f5f7eaa54677dd841ccf84678f7 for removal 2024.10.22 00:48:51.081332 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 53710c5d050ad94e6f34fa3df59318de, offset: 0, size: 250 2024.10.22 00:48:51.082081 [ 1476 ] {} CacheMetadata: Key 6fe74f5f7eaa54677dd841ccf84678f7 is removed from metadata 2024.10.22 00:48:51.085334 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 53710c5d050ad94e6f34fa3df59318de for removal 2024.10.22 00:48:51.090705 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b541100f35ad50d2b72db1cc2f270ef2, offset: 0, size: 241 2024.10.22 00:48:51.095947 [ 1476 ] {} CacheMetadata: Key 53710c5d050ad94e6f34fa3df59318de is removed from metadata 2024.10.22 00:48:51.098378 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b541100f35ad50d2b72db1cc2f270ef2 for removal 2024.10.22 00:48:51.104099 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 49ae087e486d46e2aef32f4da5b0e65b, offset: 0, size: 246 2024.10.22 00:48:51.105701 [ 1476 ] {} CacheMetadata: Key b541100f35ad50d2b72db1cc2f270ef2 is removed from metadata 2024.10.22 00:48:51.111075 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 49ae087e486d46e2aef32f4da5b0e65b for removal 2024.10.22 00:48:51.113321 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 192 2024.10.22 00:48:51.115444 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a697a4b804101340073ce8b758e052f1, offset: 0, size: 196 2024.10.22 00:48:51.117486 [ 1476 ] {} CacheMetadata: Key 49ae087e486d46e2aef32f4da5b0e65b is removed from metadata 2024.10.22 00:48:51.220565 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a697a4b804101340073ce8b758e052f1 for removal 2024.10.22 00:48:51.226905 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5908eeb6dbf9ab600eaaf43492f7a89d, offset: 0, size: 262 2024.10.22 00:48:51.233319 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5908eeb6dbf9ab600eaaf43492f7a89d for removal 2024.10.22 00:48:51.241182 [ 1476 ] {} CacheMetadata: Key a697a4b804101340073ce8b758e052f1 is removed from metadata 2024.10.22 00:48:51.245474 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 417674482a8822fae70f2ac84f8cd59d, offset: 0, size: 252 2024.10.22 00:48:51.253627 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 417674482a8822fae70f2ac84f8cd59d for removal 2024.10.22 00:48:51.255338 [ 1476 ] {} CacheMetadata: Key 5908eeb6dbf9ab600eaaf43492f7a89d is removed from metadata 2024.10.22 00:48:51.260603 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_151_30, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:48:51.260853 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ed4449b621398e1f9b5f52c49c119956, offset: 0, size: 246 2024.10.22 00:48:51.263494 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_151_30, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:48:51.264848 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ed4449b621398e1f9b5f52c49c119956 for removal 2024.10.22 00:48:51.269708 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2629032485f8464b254529a8a27ecfc4, offset: 0, size: 241 2024.10.22 00:48:51.274996 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2629032485f8464b254529a8a27ecfc4 for removal 2024.10.22 00:48:51.278725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a067b5472379fca7a117e264c5366c70, offset: 0, size: 9811 2024.10.22 00:48:51.282263 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a067b5472379fca7a117e264c5366c70 for removal 2024.10.22 00:48:51.285800 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_151_30, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:48:51.293409 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 43f7349d9b48d646a8890d57ee06afee, offset: 0, size: 238 2024.10.22 00:48:51.297293 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 43f7349d9b48d646a8890d57ee06afee for removal 2024.10.22 00:48:51.302543 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b6c6891fd9828c86f262429cdc75e647, offset: 0, size: 229 2024.10.22 00:48:51.309181 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b6c6891fd9828c86f262429cdc75e647 for removal 2024.10.22 00:48:51.315393 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e910754cb67a9080ad46d77621e2bdcb, offset: 0, size: 170 2024.10.22 00:48:51.318545 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e910754cb67a9080ad46d77621e2bdcb for removal 2024.10.22 00:48:51.321481 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5bac8b1cef3c06034093bec8ef80fa2b, offset: 0, size: 236 2024.10.22 00:48:51.328123 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5bac8b1cef3c06034093bec8ef80fa2b for removal 2024.10.22 00:48:51.367994 [ 1476 ] {} CacheMetadata: Key 5bac8b1cef3c06034093bec8ef80fa2b is removed from metadata 2024.10.22 00:48:51.375062 [ 1476 ] {} CacheMetadata: Key e910754cb67a9080ad46d77621e2bdcb is removed from metadata 2024.10.22 00:48:51.382728 [ 1476 ] {} CacheMetadata: Key b6c6891fd9828c86f262429cdc75e647 is removed from metadata 2024.10.22 00:48:51.386915 [ 1476 ] {} CacheMetadata: Key 43f7349d9b48d646a8890d57ee06afee is removed from metadata 2024.10.22 00:48:51.393727 [ 1476 ] {} CacheMetadata: Key a067b5472379fca7a117e264c5366c70 is removed from metadata 2024.10.22 00:48:51.416057 [ 1476 ] {} CacheMetadata: Key 2629032485f8464b254529a8a27ecfc4 is removed from metadata 2024.10.22 00:48:51.425165 [ 1476 ] {} CacheMetadata: Key ed4449b621398e1f9b5f52c49c119956 is removed from metadata 2024.10.22 00:48:51.434516 [ 1476 ] {} CacheMetadata: Key 417674482a8822fae70f2ac84f8cd59d is removed from metadata 2024.10.22 00:48:51.452184 [ 1476 ] {} CacheMetadata: Key 4caff12245cc7bbdbb541b5dd97f1808 is removed from metadata 2024.10.22 00:48:51.467130 [ 1476 ] {} CacheMetadata: Key e11df730070553f2681ace7ada529fd0 is removed from metadata 2024.10.22 00:48:51.509666 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_44_9/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:48:51.594919 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:51.598649 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:51.768121 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_174_174_0 (state Temporary) 2024.10.22 00:48:51.830739 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_174_174_0 to 202410_184_184_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:51.864619 [ 1257 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_179_37 to 202410_184_184_0 2024.10.22 00:48:51.872233 [ 1257 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:51.874743 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:51.905126 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_179_37 to 202410_184_184_0 into Compact with storage Full 2024.10.22 00:48:52.078226 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:48:52.084811 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 7 file segments (while reserving 91051 bytes for d0eed966c39db71fce3199964341c38f:0) 2024.10.22 00:48:52.085653 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_179_37, total 223 rows starting from the beginning of the part 2024.10.22 00:48:52.086951 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 33021037fc367bc0bafa87d9ecbe9cc6, offset: 0, size: 151 2024.10.22 00:48:52.088673 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 33021037fc367bc0bafa87d9ecbe9cc6 for removal 2024.10.22 00:48:52.091025 [ 1476 ] {} CacheMetadata: Key 33021037fc367bc0bafa87d9ecbe9cc6 is removed from metadata 2024.10.22 00:48:52.091295 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a446ffc8129c99ba84e8eb8ed1c7084f, offset: 0, size: 48111 2024.10.22 00:48:52.094461 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a446ffc8129c99ba84e8eb8ed1c7084f for removal 2024.10.22 00:48:52.100348 [ 1476 ] {} CacheMetadata: Key a446ffc8129c99ba84e8eb8ed1c7084f is removed from metadata 2024.10.22 00:48:52.103066 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 58ba4339efa04f0f693a4c05753c8ff5, offset: 0, size: 167 2024.10.22 00:48:52.110510 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 58ba4339efa04f0f693a4c05753c8ff5 for removal 2024.10.22 00:48:52.116028 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4c2fa87ceb2515e2c8b40901ca6a3f07, offset: 0, size: 33673 2024.10.22 00:48:52.119210 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_180_180_0, total 1 rows starting from the beginning of the part 2024.10.22 00:48:52.119935 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4c2fa87ceb2515e2c8b40901ca6a3f07 for removal 2024.10.22 00:48:52.121554 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 73044309daa498bc7cb1394de2f4b32d, offset: 0, size: 169 2024.10.22 00:48:52.127288 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 73044309daa498bc7cb1394de2f4b32d for removal 2024.10.22 00:48:52.129668 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fa9393730e8b071ae13acb2e8a45b7d8, offset: 0, size: 2573 2024.10.22 00:48:52.143699 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fa9393730e8b071ae13acb2e8a45b7d8 for removal 2024.10.22 00:48:52.213603 [ 1476 ] {} CacheMetadata: Key 73044309daa498bc7cb1394de2f4b32d is removed from metadata 2024.10.22 00:48:52.216058 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 192 2024.10.22 00:48:52.219688 [ 1476 ] {} CacheMetadata: Key 4c2fa87ceb2515e2c8b40901ca6a3f07 is removed from metadata 2024.10.22 00:48:52.228333 [ 1476 ] {} CacheMetadata: Key 58ba4339efa04f0f693a4c05753c8ff5 is removed from metadata 2024.10.22 00:48:52.250053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c2a9c2524081241ccdb99988684fec0d, offset: 0, size: 167 2024.10.22 00:48:52.251194 [ 1476 ] {} CacheMetadata: Key fa9393730e8b071ae13acb2e8a45b7d8 is removed from metadata 2024.10.22 00:48:52.256491 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c2a9c2524081241ccdb99988684fec0d for removal 2024.10.22 00:48:52.262281 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_181_181_0, total 1 rows starting from the beginning of the part 2024.10.22 00:48:52.365401 [ 1476 ] {} CacheMetadata: Key c2a9c2524081241ccdb99988684fec0d is removed from metadata 2024.10.22 00:48:52.392035 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_182_182_0, total 1 rows starting from the beginning of the part 2024.10.22 00:48:52.525139 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_183_183_0, total 1 rows starting from the beginning of the part 2024.10.22 00:48:52.572031 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_184_184_0, total 1 rows starting from the beginning of the part 2024.10.22 00:48:52.650566 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 9 file segments (while reserving 142676 bytes for 74a43b1dc7c93f2f97fef1672eb84d8f:0) 2024.10.22 00:48:52.657085 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d0eed966c39db71fce3199964341c38f, offset: 0, size: 91051 2024.10.22 00:48:52.661796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d0eed966c39db71fce3199964341c38f for removal 2024.10.22 00:48:52.664393 [ 1476 ] {} CacheMetadata: Key d0eed966c39db71fce3199964341c38f is removed from metadata 2024.10.22 00:48:52.664974 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bf3ec88147eb817e4f6668eb8c92de49, offset: 0, size: 157 2024.10.22 00:48:52.668824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bf3ec88147eb817e4f6668eb8c92de49 for removal 2024.10.22 00:48:52.671380 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b150b81e530d68841259ca1996bd0d1c, offset: 0, size: 78679 2024.10.22 00:48:52.673593 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b150b81e530d68841259ca1996bd0d1c for removal 2024.10.22 00:48:52.674977 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d778db2f6dcc38450fb04b0db76b70ab, offset: 0, size: 157 2024.10.22 00:48:52.687999 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d778db2f6dcc38450fb04b0db76b70ab for removal 2024.10.22 00:48:52.691653 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 983dd26569db8fe50e1782351cc27985, offset: 0, size: 183 2024.10.22 00:48:52.695048 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 983dd26569db8fe50e1782351cc27985 for removal 2024.10.22 00:48:52.700377 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7d7b6be32fa55b7e5cdbe81cdb2d3535, offset: 0, size: 156 2024.10.22 00:48:52.708027 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7d7b6be32fa55b7e5cdbe81cdb2d3535 for removal 2024.10.22 00:48:52.711760 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 841e8a0e330d4ad54c2528168b8d5b09, offset: 0, size: 155 2024.10.22 00:48:52.718452 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 841e8a0e330d4ad54c2528168b8d5b09 for removal 2024.10.22 00:48:52.721167 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c3b25c34f33f7bda7354775b6177634c, offset: 0, size: 7208 2024.10.22 00:48:52.727366 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c3b25c34f33f7bda7354775b6177634c for removal 2024.10.22 00:48:52.731825 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0d66565322108dc574335045dba4187b, offset: 0, size: 158 2024.10.22 00:48:52.734678 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0d66565322108dc574335045dba4187b for removal 2024.10.22 00:48:52.733590 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 44 entries to flush up to offset 1751 2024.10.22 00:48:52.781262 [ 1476 ] {} CacheMetadata: Key 0d66565322108dc574335045dba4187b is removed from metadata 2024.10.22 00:48:52.886405 [ 1476 ] {} CacheMetadata: Key c3b25c34f33f7bda7354775b6177634c is removed from metadata 2024.10.22 00:48:52.915112 [ 1476 ] {} CacheMetadata: Key 841e8a0e330d4ad54c2528168b8d5b09 is removed from metadata 2024.10.22 00:48:52.928893 [ 1476 ] {} CacheMetadata: Key 7d7b6be32fa55b7e5cdbe81cdb2d3535 is removed from metadata 2024.10.22 00:48:52.939331 [ 1476 ] {} CacheMetadata: Key 983dd26569db8fe50e1782351cc27985 is removed from metadata 2024.10.22 00:48:52.951485 [ 1476 ] {} CacheMetadata: Key d778db2f6dcc38450fb04b0db76b70ab is removed from metadata 2024.10.22 00:48:53.064936 [ 1476 ] {} CacheMetadata: Key b150b81e530d68841259ca1996bd0d1c is removed from metadata 2024.10.22 00:48:53.073776 [ 1476 ] {} CacheMetadata: Key bf3ec88147eb817e4f6668eb8c92de49 is removed from metadata 2024.10.22 00:48:53.291402 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 48 entries to flush up to offset 14045 2024.10.22 00:48:53.364160 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 8.02 MiB, will set to 1.66 GiB (RSS), difference: -8.95 MiB 2024.10.22 00:48:53.395723 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:53.399929 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:53.660783 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_215_215_0 (state Temporary) 2024.10.22 00:48:53.693426 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_215_215_0 to 202410_221_221_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:53.844140 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 137920 bytes for f206027e3739dfa95d4c1f3118411d1a:0) 2024.10.22 00:48:53.948448 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 74a43b1dc7c93f2f97fef1672eb84d8f, offset: 0, size: 142676 2024.10.22 00:48:53.951469 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14045 2024.10.22 00:48:53.965486 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 74a43b1dc7c93f2f97fef1672eb84d8f for removal 2024.10.22 00:48:53.969035 [ 1476 ] {} CacheMetadata: Key 74a43b1dc7c93f2f97fef1672eb84d8f is removed from metadata 2024.10.22 00:48:53.970007 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5d0973290855dbe7ef119636eae9b2c0, offset: 0, size: 168 2024.10.22 00:48:53.975197 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5d0973290855dbe7ef119636eae9b2c0 for removal 2024.10.22 00:48:53.977294 [ 1476 ] {} CacheMetadata: Key 5d0973290855dbe7ef119636eae9b2c0 is removed from metadata 2024.10.22 00:48:54.434447 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 178 entries to flush up to offset 20958 2024.10.22 00:48:54.448757 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} MergeTask::MergeProjectionsStage: Merge sorted 228 rows, containing 32 columns (32 merged, 0 gathered) in 2.554073448 sec., 89.2691634136592 rows/sec., 9.48 KiB/sec. 2024.10.22 00:48:54.473372 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} MergedBlockOutputStream: filled checksums 202410_1_184_38 (state Temporary) 2024.10.22 00:48:54.484478 [ 1157 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:48:54.523047 [ 1157 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:48:54.823405 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 67051 bytes for ae8a2d6494c893dd4909b0c3d6f731a7:0) 2024.10.22 00:48:54.827427 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f206027e3739dfa95d4c1f3118411d1a, offset: 0, size: 137920 2024.10.22 00:48:54.833303 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f206027e3739dfa95d4c1f3118411d1a for removal 2024.10.22 00:48:54.835231 [ 1476 ] {} CacheMetadata: Key f206027e3739dfa95d4c1f3118411d1a is removed from metadata 2024.10.22 00:48:54.835833 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e0dcf44eb2ee968b815203dceaafd5e6, offset: 0, size: 168 2024.10.22 00:48:54.839326 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e0dcf44eb2ee968b815203dceaafd5e6 for removal 2024.10.22 00:48:54.858372 [ 1476 ] {} CacheMetadata: Key e0dcf44eb2ee968b815203dceaafd5e6 is removed from metadata 2024.10.22 00:48:55.042792 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:55.048771 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:55.082550 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_184_38 to 202410_1_184_38 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:55.086344 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_184_38} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_179_37, 202410_184_184_0] -> 202410_1_184_38 2024.10.22 00:48:55.245460 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:48:55.465410 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 6.85 MiB, will set to 1.66 GiB (RSS), difference: -1.88 MiB 2024.10.22 00:48:55.484163 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_371_371_0 (state Temporary) 2024.10.22 00:48:55.545826 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_371_371_0 to 202410_384_384_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:55.685023 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 20958 2024.10.22 00:48:56.122389 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1478 entries to flush up to offset 358378 2024.10.22 00:48:56.193621 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 3054 2024.10.22 00:48:56.343838 [ 1271 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 221 ms. 2024.10.22 00:48:56.485695 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 136739 bytes for 98804b9dde9125c638bdf27230636fd1:0) 2024.10.22 00:48:56.487506 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e488e3c8cea930009a113f486d8277e5, offset: 0, size: 64014 2024.10.22 00:48:56.492266 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e488e3c8cea930009a113f486d8277e5 for removal 2024.10.22 00:48:56.496210 [ 1476 ] {} CacheMetadata: Key e488e3c8cea930009a113f486d8277e5 is removed from metadata 2024.10.22 00:48:56.498462 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2c0387a25cb607441bb18486dd11be69, offset: 0, size: 156 2024.10.22 00:48:56.505295 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2c0387a25cb607441bb18486dd11be69 for removal 2024.10.22 00:48:56.511041 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ae8a2d6494c893dd4909b0c3d6f731a7, offset: 0, size: 67051 2024.10.22 00:48:56.515044 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ae8a2d6494c893dd4909b0c3d6f731a7 for removal 2024.10.22 00:48:56.519362 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8bb98c24a22cb312769351b0cd7746b9, offset: 0, size: 156 2024.10.22 00:48:56.522095 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8bb98c24a22cb312769351b0cd7746b9 for removal 2024.10.22 00:48:56.621080 [ 1476 ] {} CacheMetadata: Key 8bb98c24a22cb312769351b0cd7746b9 is removed from metadata 2024.10.22 00:48:56.633786 [ 1476 ] {} CacheMetadata: Key ae8a2d6494c893dd4909b0c3d6f731a7 is removed from metadata 2024.10.22 00:48:56.659210 [ 1476 ] {} CacheMetadata: Key 2c0387a25cb607441bb18486dd11be69 is removed from metadata 2024.10.22 00:48:56.710244 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:56.713830 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:56.758189 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_217_217_0 (state Temporary) 2024.10.22 00:48:56.807398 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_217_217_0 to 202410_230_230_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:56.863294 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3054 2024.10.22 00:48:57.035884 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:48:57.043218 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:48:57.389023 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_207_207_0 (state Temporary) 2024.10.22 00:48:57.458489 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_207_207_0 to 202410_214_214_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:48:57.490538 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 7.29 MiB, will set to 1.65 GiB (RSS), difference: -9.48 MiB 2024.10.22 00:48:57.521627 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 358378 2024.10.22 00:48:57.785495 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(26e836a289e24534b2ce9c925e2b3cce) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 26e836a289e24534b2ce9c925e2b3cce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 00:48:57.814598 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 459400, file segment info: File segment: [0, 33554431], key: 26e836a289e24534b2ce9c925e2b3cce, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 00:48:57.817290 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:48:57.999725 [ 1726 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lwv/xhsqvgwyoqsxwpqcfaoigetpxquud, size 459400 2024.10.22 00:48:58.271236 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 00:48:58.289391 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 00:48:58.310905 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 00:48:58.344541 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 32400 bytes for c552854d2800a44ab39295cd70c5f956:0) 2024.10.22 00:48:58.348834 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0ece5dc35f204c685d0e8d863046b636, offset: 0, size: 39997 2024.10.22 00:48:58.357170 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0ece5dc35f204c685d0e8d863046b636 for removal 2024.10.22 00:48:58.361342 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9ed7f39ed5ccce3003c6f8c34f69c8db, offset: 0, size: 156 2024.10.22 00:48:58.361840 [ 1476 ] {} CacheMetadata: Key 0ece5dc35f204c685d0e8d863046b636 is removed from metadata 2024.10.22 00:48:58.365067 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9ed7f39ed5ccce3003c6f8c34f69c8db for removal 2024.10.22 00:48:58.375780 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_157_32, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0, 202410_162_162_0] 2024.10.22 00:48:58.378825 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_157_32, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0, 202410_162_162_0] 2024.10.22 00:48:58.411285 [ 1272 ] {} DatabaseCatalog: Execution took 228 ms. 2024.10.22 00:48:58.414749 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 00:48:58.433249 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 00:48:58.467222 [ 1476 ] {} CacheMetadata: Key 9ed7f39ed5ccce3003c6f8c34f69c8db is removed from metadata 2024.10.22 00:48:58.612884 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_157_32, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0, 202410_162_162_0] 2024.10.22 00:48:58.687014 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 00:48:58.702713 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 00:48:58.854371 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pjo/vunlpdxneexaxeytfeuhdhxsplhpt (LogSeriesLimiter: on interval from 2024-10-22 00:48:43 to 2024-10-22 00:48:58 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:48:59.022888 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 30298 bytes for d40c35f21585259952f79a586b412b61:0) 2024.10.22 00:48:59.026624 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 98804b9dde9125c638bdf27230636fd1, offset: 0, size: 136739 2024.10.22 00:48:59.031787 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 98804b9dde9125c638bdf27230636fd1 for removal 2024.10.22 00:48:59.039963 [ 1476 ] {} CacheMetadata: Key 98804b9dde9125c638bdf27230636fd1 is removed from metadata 2024.10.22 00:48:59.037814 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: da82f9ab08485864239443a2ec9924a7, offset: 0, size: 168 2024.10.22 00:48:59.143359 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key da82f9ab08485864239443a2ec9924a7 for removal 2024.10.22 00:48:59.251325 [ 1476 ] {} CacheMetadata: Key da82f9ab08485864239443a2ec9924a7 is removed from metadata 2024.10.22 00:48:59.351708 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} MergeTask::MergeProjectionsStage: Merge sorted 1687 rows, containing 679 columns (679 merged, 0 gathered) in 103.692212728 sec., 16.26930273370914 rows/sec., 22.21 KiB/sec. 2024.10.22 00:48:59.418828 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 8.80 MiB, will set to 1.65 GiB (RSS), difference: 6.05 MiB 2024.10.22 00:48:59.605036 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:48:59.608222 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:48:59.717665 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 193 2024.10.22 00:48:59.799696 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 21041 2024.10.22 00:49:00.104185 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(81ca61acdc1a1e3352375a3094b0c14a) : [0, 234098]: Setting download as failed: File segment: [0, 234098], key: 81ca61acdc1a1e3352375a3094b0c14a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:00.113380 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(81ca61acdc1a1e3352375a3094b0c14a): No space left in cache to reserve 234099 bytes, will continue without cache download 2024.10.22 00:49:00.125308 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:00.131043 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:00.248727 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 81ca61acdc1a1e3352375a3094b0c14a, offset: 0, size: 0 2024.10.22 00:49:00.250849 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 81ca61acdc1a1e3352375a3094b0c14a for removal 2024.10.22 00:49:00.254772 [ 1476 ] {} CacheMetadata: Key 81ca61acdc1a1e3352375a3094b0c14a is removed from metadata 2024.10.22 00:49:00.327072 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_175_175_0 (state Temporary) 2024.10.22 00:49:00.381451 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:00.384231 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:00.408051 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_175_175_0 to 202410_185_185_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:00.594597 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 193 2024.10.22 00:49:00.668703 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_372_372_0 (state Temporary) 2024.10.22 00:49:00.715838 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_372_372_0 to 202410_385_385_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:00.869506 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21041 2024.10.22 00:49:00.999277 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} MergedBlockOutputStream: filled checksums 202410_1_44_9 (state Temporary) 2024.10.22 00:49:01.023572 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 147000 bytes for fb6b78e4814acdfbb5478108407bbd8d:0) 2024.10.22 00:49:01.026248 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c552854d2800a44ab39295cd70c5f956, offset: 0, size: 32400 2024.10.22 00:49:01.029529 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c552854d2800a44ab39295cd70c5f956 for removal 2024.10.22 00:49:01.033140 [ 1476 ] {} CacheMetadata: Key c552854d2800a44ab39295cd70c5f956 is removed from metadata 2024.10.22 00:49:01.034929 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 47f2f4172381a4f7f6bfc42aa8362611, offset: 0, size: 2508 2024.10.22 00:49:01.041296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 47f2f4172381a4f7f6bfc42aa8362611 for removal 2024.10.22 00:49:01.047636 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fff985072c68715a59a7c79c1dd1efeb, offset: 0, size: 156 2024.10.22 00:49:01.052303 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fff985072c68715a59a7c79c1dd1efeb for removal 2024.10.22 00:49:01.061082 [ 1476 ] {} CacheMetadata: Key fff985072c68715a59a7c79c1dd1efeb is removed from metadata 2024.10.22 00:49:01.061106 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: de22d2828b1d15dc8d15e4d5d5d5a1c4, offset: 0, size: 156 2024.10.22 00:49:01.066474 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key de22d2828b1d15dc8d15e4d5d5d5a1c4 for removal 2024.10.22 00:49:01.068270 [ 1476 ] {} CacheMetadata: Key 47f2f4172381a4f7f6bfc42aa8362611 is removed from metadata 2024.10.22 00:49:01.069669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f1789fe63e17a01da01e5cb3b9b72b2f, offset: 0, size: 2070 2024.10.22 00:49:01.073225 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f1789fe63e17a01da01e5cb3b9b72b2f for removal 2024.10.22 00:49:01.075438 [ 1476 ] {} CacheMetadata: Key de22d2828b1d15dc8d15e4d5d5d5a1c4 is removed from metadata 2024.10.22 00:49:01.076829 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: be9c71900585eb9290d88a62f663e719, offset: 0, size: 156 2024.10.22 00:49:01.079302 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key be9c71900585eb9290d88a62f663e719 for removal 2024.10.22 00:49:01.084592 [ 1476 ] {} CacheMetadata: Key be9c71900585eb9290d88a62f663e719 is removed from metadata 2024.10.22 00:49:01.093834 [ 1476 ] {} CacheMetadata: Key f1789fe63e17a01da01e5cb3b9b72b2f is removed from metadata 2024.10.22 00:49:01.466622 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 70 entries to flush up to offset 14115 2024.10.22 00:49:01.509771 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 9.16 MiB, will set to 1.65 GiB (RSS), difference: -1.97 MiB 2024.10.22 00:49:01.584754 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 5 file segments (while reserving 85316 bytes for f0b82ca4d7bc6ee450c669bea0a49ce2:0) 2024.10.22 00:49:01.586725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0c49f4332148c9b04d3404d852bb1273, offset: 0, size: 158 2024.10.22 00:49:01.589357 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0c49f4332148c9b04d3404d852bb1273 for removal 2024.10.22 00:49:01.592908 [ 1476 ] {} CacheMetadata: Key 0c49f4332148c9b04d3404d852bb1273 is removed from metadata 2024.10.22 00:49:01.593413 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fb6b78e4814acdfbb5478108407bbd8d, offset: 0, size: 147000 2024.10.22 00:49:01.597621 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fb6b78e4814acdfbb5478108407bbd8d for removal 2024.10.22 00:49:01.603390 [ 1476 ] {} CacheMetadata: Key fb6b78e4814acdfbb5478108407bbd8d is removed from metadata 2024.10.22 00:49:01.608534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d40c35f21585259952f79a586b412b61, offset: 0, size: 30298 2024.10.22 00:49:01.612071 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d40c35f21585259952f79a586b412b61 for removal 2024.10.22 00:49:01.619653 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c1d74eb52235a358235d8aab0f175da7, offset: 0, size: 156 2024.10.22 00:49:01.623518 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:01.625789 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c1d74eb52235a358235d8aab0f175da7 for removal 2024.10.22 00:49:01.626930 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:01.629158 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dcd481359aab5da652f9434b1b46a457, offset: 0, size: 156 2024.10.22 00:49:01.632932 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dcd481359aab5da652f9434b1b46a457 for removal 2024.10.22 00:49:01.715687 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_216_216_0 (state Temporary) 2024.10.22 00:49:01.720045 [ 1476 ] {} CacheMetadata: Key d40c35f21585259952f79a586b412b61 is removed from metadata 2024.10.22 00:49:01.740775 [ 1476 ] {} CacheMetadata: Key dcd481359aab5da652f9434b1b46a457 is removed from metadata 2024.10.22 00:49:01.759424 [ 1476 ] {} CacheMetadata: Key c1d74eb52235a358235d8aab0f175da7 is removed from metadata 2024.10.22 00:49:01.881833 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_216_216_0 to 202410_222_222_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:02.071424 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_44_9/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:49:02.093204 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14115 2024.10.22 00:49:02.687247 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_166_33, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0] 2024.10.22 00:49:02.691452 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_166_33, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0] 2024.10.22 00:49:02.924483 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_166_33, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0] 2024.10.22 00:49:03.037563 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1da8f0d63403865fe2a82ad2285e27aa) : [0, 195421]: Setting download as failed: File segment: [0, 195421], key: 1da8f0d63403865fe2a82ad2285e27aa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:03.047196 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1da8f0d63403865fe2a82ad2285e27aa): No space left in cache to reserve 195422 bytes, will continue without cache download 2024.10.22 00:49:03.082669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1da8f0d63403865fe2a82ad2285e27aa, offset: 0, size: 0 2024.10.22 00:49:03.084849 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1da8f0d63403865fe2a82ad2285e27aa for removal 2024.10.22 00:49:03.086881 [ 1476 ] {} CacheMetadata: Key 1da8f0d63403865fe2a82ad2285e27aa is removed from metadata 2024.10.22 00:49:03.599034 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 5.76 MiB, will set to 1.65 GiB (RSS), difference: -1.62 MiB 2024.10.22 00:49:03.728151 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ce11774d80014dc9d6b02ddc380bae64) : [0, 237174]: Setting download as failed: File segment: [0, 237174], key: ce11774d80014dc9d6b02ddc380bae64, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:03.731365 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ce11774d80014dc9d6b02ddc380bae64): No space left in cache to reserve 237175 bytes, will continue without cache download 2024.10.22 00:49:03.755033 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ce11774d80014dc9d6b02ddc380bae64, offset: 0, size: 0 2024.10.22 00:49:03.758172 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ce11774d80014dc9d6b02ddc380bae64 for removal 2024.10.22 00:49:03.760859 [ 1476 ] {} CacheMetadata: Key ce11774d80014dc9d6b02ddc380bae64 is removed from metadata 2024.10.22 00:49:03.865259 [ 1205 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 00:49:04.009062 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:49:04.270264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 103272 bytes for 7b3f721a5d6e07a4a08f4125384a139e:0) 2024.10.22 00:49:04.272058 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1d933d2d5b52c0264ea98989fdde0cef, offset: 0, size: 94767 2024.10.22 00:49:04.274736 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1d933d2d5b52c0264ea98989fdde0cef for removal 2024.10.22 00:49:04.277247 [ 1476 ] {} CacheMetadata: Key 1d933d2d5b52c0264ea98989fdde0cef is removed from metadata 2024.10.22 00:49:04.277419 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 149d2f4a791fc26641abc683268ef0dd, offset: 0, size: 157 2024.10.22 00:49:04.280383 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 149d2f4a791fc26641abc683268ef0dd for removal 2024.10.22 00:49:04.284459 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f0b82ca4d7bc6ee450c669bea0a49ce2, offset: 0, size: 85316 2024.10.22 00:49:04.289524 [ 1476 ] {} CacheMetadata: Key 149d2f4a791fc26641abc683268ef0dd is removed from metadata 2024.10.22 00:49:04.364761 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 3059 2024.10.22 00:49:04.389036 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f0b82ca4d7bc6ee450c669bea0a49ce2 for removal 2024.10.22 00:49:04.392640 [ 1476 ] {} CacheMetadata: Key f0b82ca4d7bc6ee450c669bea0a49ce2 is removed from metadata 2024.10.22 00:49:04.393265 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 234ef888f8dd9b73081f3e2e78547a0b, offset: 0, size: 168 2024.10.22 00:49:04.398818 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 234ef888f8dd9b73081f3e2e78547a0b for removal 2024.10.22 00:49:04.405102 [ 1476 ] {} CacheMetadata: Key 234ef888f8dd9b73081f3e2e78547a0b is removed from metadata 2024.10.22 00:49:04.526313 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1594 entries to flush up to offset 359972 2024.10.22 00:49:04.783728 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:04.785650 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:04.877079 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 21134 2024.10.22 00:49:05.026119 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_218_218_0 (state Temporary) 2024.10.22 00:49:05.072740 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_218_218_0 to 202410_231_231_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:05.094238 [ 1218 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_226_45 to 202410_231_231_0 2024.10.22 00:49:05.111187 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f3fc49240b34f60f7b272a30540f79a4) : [0, 1099711]: Setting download as failed: File segment: [0, 1099711], key: f3fc49240b34f60f7b272a30540f79a4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:05.114796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f3fc49240b34f60f7b272a30540f79a4): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:49:05.116728 [ 1218 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:05.120647 [ 1218 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:05.128866 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3059 2024.10.22 00:49:05.156141 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:05.161633 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:05.168030 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_226_45 to 202410_231_231_0 into Compact with storage Full 2024.10.22 00:49:05.214518 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_373_373_0 (state Temporary) 2024.10.22 00:49:05.219133 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:05.266077 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f3fc49240b34f60f7b272a30540f79a4, offset: 0, size: 0 2024.10.22 00:49:05.269002 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f3fc49240b34f60f7b272a30540f79a4 for removal 2024.10.22 00:49:05.333325 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_226_45, total 3096 rows starting from the beginning of the part 2024.10.22 00:49:05.359094 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_227_227_0, total 13 rows starting from the beginning of the part 2024.10.22 00:49:05.375866 [ 1476 ] {} CacheMetadata: Key f3fc49240b34f60f7b272a30540f79a4 is removed from metadata 2024.10.22 00:49:05.379275 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 4.97 MiB, will set to 1.65 GiB (RSS), difference: -4.19 MiB 2024.10.22 00:49:05.441705 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:05.476924 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_228_228_0, total 10 rows starting from the beginning of the part 2024.10.22 00:49:05.483139 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_373_373_0 to 202410_386_386_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:05.498178 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_229_229_0, total 28 rows starting from the beginning of the part 2024.10.22 00:49:05.524049 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_230_230_0, total 13 rows starting from the beginning of the part 2024.10.22 00:49:05.548124 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:05.560566 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_231_231_0, total 5 rows starting from the beginning of the part 2024.10.22 00:49:05.639031 [ 1181 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_381_82 to 202410_386_386_0 2024.10.22 00:49:05.654630 [ 1181 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:05.660317 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:05.695362 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_381_82 to 202410_386_386_0 into Compact with storage Full 2024.10.22 00:49:05.702126 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_208_208_0 (state Temporary) 2024.10.22 00:49:05.785239 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_208_208_0 to 202410_215_215_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:05.809170 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:05.853484 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_381_82, total 21483 rows starting from the beginning of the part 2024.10.22 00:49:05.862344 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21134 2024.10.22 00:49:05.932027 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 359972 2024.10.22 00:49:05.962490 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_382_382_0, total 59 rows starting from the beginning of the part 2024.10.22 00:49:06.045590 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_383_383_0, total 69 rows starting from the beginning of the part 2024.10.22 00:49:06.168400 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_384_384_0, total 178 rows starting from the beginning of the part 2024.10.22 00:49:06.327745 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_385_385_0, total 83 rows starting from the beginning of the part 2024.10.22 00:49:06.431322 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_386_386_0, total 93 rows starting from the beginning of the part 2024.10.22 00:49:06.755800 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 53888 bytes for b912751b502f6ca0f62a067839860e3f:0) 2024.10.22 00:49:06.761282 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7b3f721a5d6e07a4a08f4125384a139e, offset: 0, size: 103272 2024.10.22 00:49:06.767410 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7b3f721a5d6e07a4a08f4125384a139e for removal 2024.10.22 00:49:06.771536 [ 1476 ] {} CacheMetadata: Key 7b3f721a5d6e07a4a08f4125384a139e is removed from metadata 2024.10.22 00:49:06.772105 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 79829c606e0cb7fb8762dac95c1d80da, offset: 0, size: 168 2024.10.22 00:49:06.777382 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 79829c606e0cb7fb8762dac95c1d80da for removal 2024.10.22 00:49:06.780710 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 696dfb2a7510e3a6d70c4475b67bcbe1, offset: 0, size: 170 2024.10.22 00:49:06.786598 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 696dfb2a7510e3a6d70c4475b67bcbe1 for removal 2024.10.22 00:49:06.788192 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2abf2960578f25a2de2cbd9ab064329e, offset: 0, size: 169 2024.10.22 00:49:06.788588 [ 1476 ] {} CacheMetadata: Key 696dfb2a7510e3a6d70c4475b67bcbe1 is removed from metadata 2024.10.22 00:49:06.790643 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} MergeTask::MergeProjectionsStage: Merge sorted 3165 rows, containing 12 columns (12 merged, 0 gathered) in 1.634010366 sec., 1936.9522163729039 rows/sec., 301.90 KiB/sec. 2024.10.22 00:49:06.792102 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2abf2960578f25a2de2cbd9ab064329e for removal 2024.10.22 00:49:06.798326 [ 1476 ] {} CacheMetadata: Key 2abf2960578f25a2de2cbd9ab064329e is removed from metadata 2024.10.22 00:49:06.815029 [ 1476 ] {} CacheMetadata: Key 79829c606e0cb7fb8762dac95c1d80da is removed from metadata 2024.10.22 00:49:06.914662 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} MergedBlockOutputStream: filled checksums 202410_1_231_46 (state Temporary) 2024.10.22 00:49:07.033515 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_231_46/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:49:07.426926 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 4.76 MiB, will set to 1.64 GiB (RSS), difference: -11.01 MiB 2024.10.22 00:49:07.474302 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_231_46 to 202410_1_231_46 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:07.479652 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_231_46} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_226_45, 202410_231_231_0] -> 202410_1_231_46 2024.10.22 00:49:07.528360 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:49:07.535487 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(d92f793922a845cb0c146612349d02b3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d92f793922a845cb0c146612349d02b3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 00:49:07.540460 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: d92f793922a845cb0c146612349d02b3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 00:49:07.545519 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:49:08.336569 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 100967 bytes for 156c479edf0dc67aba5d0be21391d469:0) 2024.10.22 00:49:08.343052 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 91d636ad3222056017f636380751fffb, offset: 0, size: 62559 2024.10.22 00:49:08.351384 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 91d636ad3222056017f636380751fffb for removal 2024.10.22 00:49:08.359644 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: beeddb8c325859e6e314bc6a56777002, offset: 0, size: 156 2024.10.22 00:49:08.359452 [ 1476 ] {} CacheMetadata: Key 91d636ad3222056017f636380751fffb is removed from metadata 2024.10.22 00:49:08.367900 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key beeddb8c325859e6e314bc6a56777002 for removal 2024.10.22 00:49:08.371143 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 65636d8a10210c45c35efcc696d0e9e0, offset: 0, size: 169 2024.10.22 00:49:08.371591 [ 1476 ] {} CacheMetadata: Key beeddb8c325859e6e314bc6a56777002 is removed from metadata 2024.10.22 00:49:08.375348 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 65636d8a10210c45c35efcc696d0e9e0 for removal 2024.10.22 00:49:08.379172 [ 1476 ] {} CacheMetadata: Key 65636d8a10210c45c35efcc696d0e9e0 is removed from metadata 2024.10.22 00:49:08.385338 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_132_27, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0, 202410_137_137_0] 2024.10.22 00:49:08.388290 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_132_27, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0, 202410_137_137_0] 2024.10.22 00:49:08.441318 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_132_27, 202410_133_133_0, 202410_134_134_0, 202410_135_135_0, 202410_136_136_0, 202410_137_137_0] 2024.10.22 00:49:08.476756 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_280_56, 202410_281_281_0, 202410_282_282_0, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0] 2024.10.22 00:49:08.494362 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_280_56, 202410_281_281_0, 202410_282_282_0, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0] 2024.10.22 00:49:08.534384 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 4.24 MiB, will set to 1.65 GiB (RSS), difference: 6.80 MiB 2024.10.22 00:49:08.643421 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_280_56, 202410_281_281_0, 202410_282_282_0, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0] 2024.10.22 00:49:09.340503 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 153478 bytes for 03a28e852d766c5e490ac5a7fafefe6b:0) 2024.10.22 00:49:09.346121 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 156c479edf0dc67aba5d0be21391d469, offset: 0, size: 100967 2024.10.22 00:49:09.352828 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 156c479edf0dc67aba5d0be21391d469 for removal 2024.10.22 00:49:09.358179 [ 1476 ] {} CacheMetadata: Key 156c479edf0dc67aba5d0be21391d469 is removed from metadata 2024.10.22 00:49:09.361401 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 69392ba14fb5b94778bfb8e7f1aafee9, offset: 0, size: 153 2024.10.22 00:49:09.364988 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 69392ba14fb5b94778bfb8e7f1aafee9 for removal 2024.10.22 00:49:09.370490 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b912751b502f6ca0f62a067839860e3f, offset: 0, size: 53888 2024.10.22 00:49:09.374445 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b912751b502f6ca0f62a067839860e3f for removal 2024.10.22 00:49:09.376831 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 65e8b1e04915893e1139309cae935879, offset: 0, size: 156 2024.10.22 00:49:09.381723 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 65e8b1e04915893e1139309cae935879 for removal 2024.10.22 00:49:09.383135 [ 1476 ] {} CacheMetadata: Key b912751b502f6ca0f62a067839860e3f is removed from metadata 2024.10.22 00:49:09.399687 [ 1476 ] {} CacheMetadata: Key 65e8b1e04915893e1139309cae935879 is removed from metadata 2024.10.22 00:49:09.417287 [ 1476 ] {} CacheMetadata: Key 69392ba14fb5b94778bfb8e7f1aafee9 is removed from metadata 2024.10.22 00:49:09.527947 [ 1225 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:49:09.559504 [ 1225 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:49:09.568916 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} MergeTask::MergeProjectionsStage: Merge sorted 21965 rows, containing 13 columns (13 merged, 0 gathered) in 3.882091006 sec., 5658.033252196252 rows/sec., 865.64 KiB/sec. 2024.10.22 00:49:09.603835 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:49:09.624627 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:49:09.636164 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 74 entries to flush up to offset 14189 2024.10.22 00:49:09.867964 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} MergedBlockOutputStream: filled checksums 202410_1_386_83 (state Temporary) 2024.10.22 00:49:09.876437 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 101 entries to flush up to offset 21235 2024.10.22 00:49:09.881575 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:09.884448 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:10.121560 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_217_217_0 (state Temporary) 2024.10.22 00:49:10.155464 [ 1236 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 206 ms. 2024.10.22 00:49:10.170610 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_217_217_0 to 202410_223_223_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:10.241218 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_386_83 to 202410_1_386_83 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:10.247182 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_386_83} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_381_82, 202410_386_386_0] -> 202410_1_386_83 2024.10.22 00:49:10.285628 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.82 MiB. 2024.10.22 00:49:10.356947 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:10.362820 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:10.371469 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14189 2024.10.22 00:49:10.420923 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_374_374_0 (state Temporary) 2024.10.22 00:49:10.479097 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 86102 bytes for 73f6591c3004d711b4cf0750e4c7e055:0) 2024.10.22 00:49:10.481343 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 03a28e852d766c5e490ac5a7fafefe6b, offset: 0, size: 153478 2024.10.22 00:49:10.572309 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_374_374_0 to 202410_387_387_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:10.589640 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 03a28e852d766c5e490ac5a7fafefe6b for removal 2024.10.22 00:49:10.592840 [ 1476 ] {} CacheMetadata: Key 03a28e852d766c5e490ac5a7fafefe6b is removed from metadata 2024.10.22 00:49:10.593341 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4c8cd75a2f7bfd8d4bd95a3471e61d6b, offset: 0, size: 155 2024.10.22 00:49:10.597543 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4c8cd75a2f7bfd8d4bd95a3471e61d6b for removal 2024.10.22 00:49:10.633399 [ 1476 ] {} CacheMetadata: Key 4c8cd75a2f7bfd8d4bd95a3471e61d6b is removed from metadata 2024.10.22 00:49:10.634695 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 2.85 MiB, will set to 1.65 GiB (RSS), difference: 7.29 MiB 2024.10.22 00:49:10.795729 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:10.824666 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21235 2024.10.22 00:49:10.899140 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:11.556411 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d3a4b4c2f687b5353dedecce5cb2c352) : [0, 347124]: Setting download as failed: File segment: [0, 347124], key: d3a4b4c2f687b5353dedecce5cb2c352, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:11.560499 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d3a4b4c2f687b5353dedecce5cb2c352): No space left in cache to reserve 347125 bytes, will continue without cache download 2024.10.22 00:49:11.579550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d3a4b4c2f687b5353dedecce5cb2c352, offset: 0, size: 0 2024.10.22 00:49:11.582419 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d3a4b4c2f687b5353dedecce5cb2c352 for removal 2024.10.22 00:49:11.585480 [ 1476 ] {} CacheMetadata: Key d3a4b4c2f687b5353dedecce5cb2c352 is removed from metadata 2024.10.22 00:49:12.305768 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(dadd77dd1315675b04080f62686b71e3) : [0, 1032497]: Setting download as failed: File segment: [0, 1032497], key: dadd77dd1315675b04080f62686b71e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:12.310913 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(dadd77dd1315675b04080f62686b71e3): No space left in cache to reserve 1032498 bytes, will continue without cache download 2024.10.22 00:49:12.435377 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dadd77dd1315675b04080f62686b71e3, offset: 0, size: 0 2024.10.22 00:49:12.444860 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dadd77dd1315675b04080f62686b71e3 for removal 2024.10.22 00:49:12.449196 [ 1476 ] {} CacheMetadata: Key dadd77dd1315675b04080f62686b71e3 is removed from metadata 2024.10.22 00:49:12.460390 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_44_9 to 202410_1_44_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:12.467125 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_44_9} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_39_8, 202410_44_44_0] -> 202410_1_44_9 2024.10.22 00:49:12.635165 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 3085 2024.10.22 00:49:12.953687 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:12.960020 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:12.987879 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_219_219_0 (state Temporary) 2024.10.22 00:49:13.003906 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1595 entries to flush up to offset 361567 2024.10.22 00:49:13.027418 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 29.64 MiB. 2024.10.22 00:49:13.046384 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_219_219_0 to 202410_232_232_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:13.079059 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3085 2024.10.22 00:49:13.221984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 98218 bytes for e07737e536e98c2df8003ca881f27322:0) 2024.10.22 00:49:13.224984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 73f6591c3004d711b4cf0750e4c7e055, offset: 0, size: 86102 2024.10.22 00:49:13.232197 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 73f6591c3004d711b4cf0750e4c7e055 for removal 2024.10.22 00:49:13.235591 [ 1476 ] {} CacheMetadata: Key 73f6591c3004d711b4cf0750e4c7e055 is removed from metadata 2024.10.22 00:49:13.239693 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: de10f481fa0c35c262bbcd13ab336dd9, offset: 0, size: 157 2024.10.22 00:49:13.250333 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key de10f481fa0c35c262bbcd13ab336dd9 for removal 2024.10.22 00:49:13.258976 [ 1476 ] {} CacheMetadata: Key de10f481fa0c35c262bbcd13ab336dd9 is removed from metadata 2024.10.22 00:49:13.494367 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:13.499965 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:13.742240 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_209_209_0 (state Temporary) 2024.10.22 00:49:13.892988 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_209_209_0 to 202410_216_216_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:13.934820 [ 1242 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_211_42 to 202410_216_216_0 2024.10.22 00:49:13.957203 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 361567 2024.10.22 00:49:13.975748 [ 1242 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:13.977303 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:13.985737 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_211_42 to 202410_216_216_0 into Compact with storage Full 2024.10.22 00:49:14.017751 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:14.024752 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} MergeTreeSequentialSource: Reading 46 marks from part 202410_1_211_42, total 366650 rows starting from the beginning of the part 2024.10.22 00:49:14.039312 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_212_212_0, total 1727 rows starting from the beginning of the part 2024.10.22 00:49:14.054511 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_213_213_0, total 2113 rows starting from the beginning of the part 2024.10.22 00:49:14.066261 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_214_214_0, total 1478 rows starting from the beginning of the part 2024.10.22 00:49:14.074557 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_215_215_0, total 1594 rows starting from the beginning of the part 2024.10.22 00:49:14.085980 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_216_216_0, total 1595 rows starting from the beginning of the part 2024.10.22 00:49:14.173976 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8d8a3dc705b57f7ea051d876e70f86e9) : [0, 1083150]: Setting download as failed: File segment: [0, 1083150], key: 8d8a3dc705b57f7ea051d876e70f86e9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:14.176448 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8d8a3dc705b57f7ea051d876e70f86e9): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:49:14.212260 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8d8a3dc705b57f7ea051d876e70f86e9, offset: 0, size: 0 2024.10.22 00:49:14.215461 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8d8a3dc705b57f7ea051d876e70f86e9 for removal 2024.10.22 00:49:14.219174 [ 1476 ] {} CacheMetadata: Key 8d8a3dc705b57f7ea051d876e70f86e9 is removed from metadata 2024.10.22 00:49:14.251320 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_44_44_0 (state Temporary) 2024.10.22 00:49:14.423463 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 3.90 MiB, will set to 1.64 GiB (RSS), difference: -5.31 MiB 2024.10.22 00:49:14.854526 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 21311 2024.10.22 00:49:15.319319 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:15.322621 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:15.421499 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 90544 bytes for 568fdb3f8416e592a8cdcfeb7a93b471:0) 2024.10.22 00:49:15.427453 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e07737e536e98c2df8003ca881f27322, offset: 0, size: 98218 2024.10.22 00:49:15.427762 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_375_375_0 (state Temporary) 2024.10.22 00:49:15.434145 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e07737e536e98c2df8003ca881f27322 for removal 2024.10.22 00:49:15.440986 [ 1476 ] {} CacheMetadata: Key e07737e536e98c2df8003ca881f27322 is removed from metadata 2024.10.22 00:49:15.442566 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f6bbab89d78f24b97d7236310c0228d0, offset: 0, size: 168 2024.10.22 00:49:15.457460 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f6bbab89d78f24b97d7236310c0228d0 for removal 2024.10.22 00:49:15.464149 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6e64550d0263cee39dd77016fa74b613, offset: 0, size: 157 2024.10.22 00:49:15.470916 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6e64550d0263cee39dd77016fa74b613 for removal 2024.10.22 00:49:15.482358 [ 1476 ] {} CacheMetadata: Key 6e64550d0263cee39dd77016fa74b613 is removed from metadata 2024.10.22 00:49:15.492801 [ 1476 ] {} CacheMetadata: Key f6bbab89d78f24b97d7236310c0228d0 is removed from metadata 2024.10.22 00:49:15.515864 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_375_375_0 to 202410_388_388_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:15.569490 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21311 2024.10.22 00:49:15.575583 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b5a93933763f5907883820629bf5242e, offset: 0, size: 182 2024.10.22 00:49:15.578337 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b5a93933763f5907883820629bf5242e for removal 2024.10.22 00:49:15.580654 [ 1476 ] {} CacheMetadata: Key b5a93933763f5907883820629bf5242e is removed from metadata 2024.10.22 00:49:15.599417 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 194 2024.10.22 00:49:15.762328 [ 1153 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 00:49:15.859411 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:15.862199 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:16.017115 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_176_176_0 (state Temporary) 2024.10.22 00:49:16.396502 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 8.20 MiB, will set to 1.65 GiB (RSS), difference: -1.13 MiB 2024.10.22 00:49:16.462246 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_176_176_0 to 202410_186_186_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:16.562907 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 194 2024.10.22 00:49:16.698746 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 5 file segments (while reserving 75653 bytes for 5a7f49a94cc0abb6ccd7b4779db96fe2:0) 2024.10.22 00:49:16.702134 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 568fdb3f8416e592a8cdcfeb7a93b471, offset: 0, size: 90544 2024.10.22 00:49:16.709486 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 568fdb3f8416e592a8cdcfeb7a93b471 for removal 2024.10.22 00:49:16.713254 [ 1476 ] {} CacheMetadata: Key 568fdb3f8416e592a8cdcfeb7a93b471 is removed from metadata 2024.10.22 00:49:16.714181 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: af732b5492b61f03bbdeed96fa88f14c, offset: 0, size: 157 2024.10.22 00:49:16.719655 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key af732b5492b61f03bbdeed96fa88f14c for removal 2024.10.22 00:49:16.723789 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 73b737fc78937068cb1006ba0d4ad04a, offset: 0, size: 4850 2024.10.22 00:49:16.728442 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 73b737fc78937068cb1006ba0d4ad04a for removal 2024.10.22 00:49:16.731804 [ 1476 ] {} CacheMetadata: Key af732b5492b61f03bbdeed96fa88f14c is removed from metadata 2024.10.22 00:49:16.735276 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b55f48adcc4e8a7719a17c657f81640b, offset: 0, size: 169 2024.10.22 00:49:16.742820 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b55f48adcc4e8a7719a17c657f81640b for removal 2024.10.22 00:49:16.745139 [ 1476 ] {} CacheMetadata: Key 73b737fc78937068cb1006ba0d4ad04a is removed from metadata 2024.10.22 00:49:16.756307 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b6c9c34a3de8f8df74f857a8f24137f1, offset: 0, size: 156 2024.10.22 00:49:16.760660 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b6c9c34a3de8f8df74f857a8f24137f1 for removal 2024.10.22 00:49:16.763636 [ 1476 ] {} CacheMetadata: Key b55f48adcc4e8a7719a17c657f81640b is removed from metadata 2024.10.22 00:49:16.769393 [ 1476 ] {} CacheMetadata: Key b6c9c34a3de8f8df74f857a8f24137f1 is removed from metadata 2024.10.22 00:49:17.432174 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 121188 bytes for 85ec6e03fb6145338b91b1444122acff:0) 2024.10.22 00:49:17.439114 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5a7f49a94cc0abb6ccd7b4779db96fe2, offset: 0, size: 75653 2024.10.22 00:49:17.444081 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5a7f49a94cc0abb6ccd7b4779db96fe2 for removal 2024.10.22 00:49:17.448539 [ 1476 ] {} CacheMetadata: Key 5a7f49a94cc0abb6ccd7b4779db96fe2 is removed from metadata 2024.10.22 00:49:17.449541 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6fb6b844a6ec4514c129a27042f32afd, offset: 0, size: 157 2024.10.22 00:49:17.456301 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6fb6b844a6ec4514c129a27042f32afd for removal 2024.10.22 00:49:17.559418 [ 1476 ] {} CacheMetadata: Key 6fb6b844a6ec4514c129a27042f32afd is removed from metadata 2024.10.22 00:49:17.561673 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eda84c2147cd563f16a957fc4b5cb349, offset: 0, size: 78249 2024.10.22 00:49:17.567893 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eda84c2147cd563f16a957fc4b5cb349 for removal 2024.10.22 00:49:17.571274 [ 1476 ] {} CacheMetadata: Key eda84c2147cd563f16a957fc4b5cb349 is removed from metadata 2024.10.22 00:49:17.573903 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 621430cb4c12119c071e01078725f980, offset: 0, size: 157 2024.10.22 00:49:17.578800 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 621430cb4c12119c071e01078725f980 for removal 2024.10.22 00:49:17.587150 [ 1476 ] {} CacheMetadata: Key 621430cb4c12119c071e01078725f980 is removed from metadata 2024.10.22 00:49:18.112398 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 56 entries to flush up to offset 14245 2024.10.22 00:49:18.275783 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:18.278207 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:18.415678 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_218_218_0 (state Temporary) 2024.10.22 00:49:18.479057 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a98ea46951d177bd8425d852272cc9b9) : [0, 302488]: Setting download as failed: File segment: [0, 302488], key: a98ea46951d177bd8425d852272cc9b9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:18.480669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a98ea46951d177bd8425d852272cc9b9): No space left in cache to reserve 302489 bytes, will continue without cache download 2024.10.22 00:49:18.493512 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a98ea46951d177bd8425d852272cc9b9, offset: 0, size: 0 2024.10.22 00:49:18.497429 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a98ea46951d177bd8425d852272cc9b9 for removal 2024.10.22 00:49:18.502865 [ 1476 ] {} CacheMetadata: Key a98ea46951d177bd8425d852272cc9b9 is removed from metadata 2024.10.22 00:49:18.572144 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 9.80 MiB, will set to 1.65 GiB (RSS), difference: 4.96 MiB 2024.10.22 00:49:18.590282 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_218_218_0 to 202410_224_224_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:18.625316 [ 1271 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 4 parts from 202410_1_219_46 to 202410_222_222_0 2024.10.22 00:49:18.671971 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14245 2024.10.22 00:49:18.756546 [ 1271 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:18.763142 [ 1271 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:18.773770 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_222_47} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_219_46 to 202410_222_222_0 into Compact with storage Full 2024.10.22 00:49:18.912063 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_222_47} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:18.927768 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_222_47} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_219_46, total 13995 rows starting from the beginning of the part 2024.10.22 00:49:19.047675 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_222_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_220_220_0, total 106 rows starting from the beginning of the part 2024.10.22 00:49:19.169619 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_222_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_221_221_0, total 48 rows starting from the beginning of the part 2024.10.22 00:49:19.193118 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_222_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_222_222_0, total 70 rows starting from the beginning of the part 2024.10.22 00:49:19.404902 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(00c4a0e54e2f7d4299d31e3688d90c35) : [0, 230573]: Setting download as failed: File segment: [0, 230573], key: 00c4a0e54e2f7d4299d31e3688d90c35, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:19.410080 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(00c4a0e54e2f7d4299d31e3688d90c35): No space left in cache to reserve 230574 bytes, will continue without cache download 2024.10.22 00:49:19.548551 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 00c4a0e54e2f7d4299d31e3688d90c35, offset: 0, size: 0 2024.10.22 00:49:19.553129 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 00c4a0e54e2f7d4299d31e3688d90c35 for removal 2024.10.22 00:49:19.557050 [ 1476 ] {} CacheMetadata: Key 00c4a0e54e2f7d4299d31e3688d90c35 is removed from metadata 2024.10.22 00:49:19.575896 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 21393 2024.10.22 00:49:19.611864 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:49:19.614501 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:49:19.684245 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 00:49:19.965530 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_216_43/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:49:20.060678 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_44_44_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:49:20.091033 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:20.094902 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:20.270666 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_376_376_0 (state Temporary) 2024.10.22 00:49:20.421279 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_222_47} MergeTask::MergeProjectionsStage: Merge sorted 14219 rows, containing 13 columns (13 merged, 0 gathered) in 1.65176622 sec., 8608.361054871311 rows/sec., 3.61 MiB/sec. 2024.10.22 00:49:20.495440 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 12.62 MiB, will set to 1.67 GiB (RSS), difference: 3.46 MiB 2024.10.22 00:49:20.558832 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_376_376_0 to 202410_389_389_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:20.584835 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 3091 2024.10.22 00:49:20.633561 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21393 2024.10.22 00:49:20.661063 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 132376 bytes for 2173f17eb0c1f324914482492fc3fb9a:0) 2024.10.22 00:49:20.665699 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4d88ec4e2fc7ccea43bc0b0072956a82, offset: 0, size: 47561 2024.10.22 00:49:20.668901 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4d88ec4e2fc7ccea43bc0b0072956a82 for removal 2024.10.22 00:49:20.672487 [ 1476 ] {} CacheMetadata: Key 4d88ec4e2fc7ccea43bc0b0072956a82 is removed from metadata 2024.10.22 00:49:20.673174 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 029635568a1b89c747290f8edfa7ac4a, offset: 0, size: 156 2024.10.22 00:49:20.675891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 029635568a1b89c747290f8edfa7ac4a for removal 2024.10.22 00:49:20.680997 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 85ec6e03fb6145338b91b1444122acff, offset: 0, size: 121188 2024.10.22 00:49:20.685952 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 85ec6e03fb6145338b91b1444122acff for removal 2024.10.22 00:49:20.691390 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6847d6018c5ef757f649323ed378a602, offset: 0, size: 168 2024.10.22 00:49:20.696969 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6847d6018c5ef757f649323ed378a602 for removal 2024.10.22 00:49:20.698217 [ 1476 ] {} CacheMetadata: Key 85ec6e03fb6145338b91b1444122acff is removed from metadata 2024.10.22 00:49:20.719323 [ 1476 ] {} CacheMetadata: Key 6847d6018c5ef757f649323ed378a602 is removed from metadata 2024.10.22 00:49:20.728000 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_222_47} MergedBlockOutputStream: filled checksums 202410_1_222_47 (state Temporary) 2024.10.22 00:49:20.738418 [ 1476 ] {} CacheMetadata: Key 029635568a1b89c747290f8edfa7ac4a is removed from metadata 2024.10.22 00:49:20.883924 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_222_47} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_222_47 to 202410_1_222_47 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:20.897369 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_222_47} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 4 parts: [202410_1_219_46, 202410_222_222_0] -> 202410_1_222_47 2024.10.22 00:49:20.937225 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.78 MiB. 2024.10.22 00:49:21.008589 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1503 entries to flush up to offset 363070 2024.10.22 00:49:21.032203 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:21.038871 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:21.077506 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_220_220_0 (state Temporary) 2024.10.22 00:49:21.261597 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_220_220_0 to 202410_233_233_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:21.510666 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3091 2024.10.22 00:49:21.533225 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:21.541441 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:21.673370 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_210_210_0 (state Temporary) 2024.10.22 00:49:21.770369 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_210_210_0 to 202410_217_217_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:21.793663 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(27a1ae10b5c1e788a008b1e7d0b4dac4) : [0, 205305]: Setting download as failed: File segment: [0, 205305], key: 27a1ae10b5c1e788a008b1e7d0b4dac4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:21.798961 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 363070 2024.10.22 00:49:21.796982 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(27a1ae10b5c1e788a008b1e7d0b4dac4): No space left in cache to reserve 205306 bytes, will continue without cache download 2024.10.22 00:49:21.968060 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 27a1ae10b5c1e788a008b1e7d0b4dac4, offset: 0, size: 0 2024.10.22 00:49:21.969230 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 27a1ae10b5c1e788a008b1e7d0b4dac4 for removal 2024.10.22 00:49:21.972162 [ 1476 ] {} CacheMetadata: Key 27a1ae10b5c1e788a008b1e7d0b4dac4 is removed from metadata 2024.10.22 00:49:22.511315 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 13.86 MiB, will set to 1.66 GiB (RSS), difference: 2.88 MiB 2024.10.22 00:49:22.883413 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(edc841105b50525d0594f66ddec7fcef) : [0, 273030]: Setting download as failed: File segment: [0, 273030], key: edc841105b50525d0594f66ddec7fcef, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:22.884704 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(edc841105b50525d0594f66ddec7fcef): No space left in cache to reserve 273031 bytes, will continue without cache download 2024.10.22 00:49:22.935997 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: edc841105b50525d0594f66ddec7fcef, offset: 0, size: 0 2024.10.22 00:49:22.937210 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key edc841105b50525d0594f66ddec7fcef for removal 2024.10.22 00:49:22.940713 [ 1476 ] {} CacheMetadata: Key edc841105b50525d0594f66ddec7fcef is removed from metadata 2024.10.22 00:49:23.891914 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f45ab4569c6523cd2d25a483c539073d) : [0, 318387]: Setting download as failed: File segment: [0, 318387], key: f45ab4569c6523cd2d25a483c539073d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:23.893655 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f45ab4569c6523cd2d25a483c539073d): No space left in cache to reserve 318388 bytes, will continue without cache download 2024.10.22 00:49:24.033573 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f45ab4569c6523cd2d25a483c539073d, offset: 0, size: 0 2024.10.22 00:49:24.037564 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f45ab4569c6523cd2d25a483c539073d for removal 2024.10.22 00:49:24.042391 [ 1476 ] {} CacheMetadata: Key f45ab4569c6523cd2d25a483c539073d is removed from metadata 2024.10.22 00:49:24.065574 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 195 2024.10.22 00:49:24.439624 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 00:49:24.479902 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 00:49:24.544662 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 00:49:24.564716 [ 1170 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:49:24.575939 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 00:49:24.627751 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 00:49:24.634076 [ 1170 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:49:24.664321 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 21459 2024.10.22 00:49:24.668193 [ 1680 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pjo/vunlpdxneexaxeytfeuhdhxsplhpt, size 3120927 2024.10.22 00:49:24.674987 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 00:49:24.708847 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:24.710123 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 00:49:24.718597 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:24.772918 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 14.30 MiB, will set to 1.66 GiB (RSS), difference: 6.56 MiB 2024.10.22 00:49:24.885556 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(71adf85dd30421e65a8f50e0b89a7a4a) : [0, 524474]: Setting download as failed: File segment: [0, 524474], key: 71adf85dd30421e65a8f50e0b89a7a4a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:24.888173 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(71adf85dd30421e65a8f50e0b89a7a4a): No space left in cache to reserve 524475 bytes, will continue without cache download 2024.10.22 00:49:24.921089 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 71adf85dd30421e65a8f50e0b89a7a4a, offset: 0, size: 0 2024.10.22 00:49:24.923662 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 71adf85dd30421e65a8f50e0b89a7a4a for removal 2024.10.22 00:49:24.926956 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_177_177_0 (state Temporary) 2024.10.22 00:49:24.932767 [ 1476 ] {} CacheMetadata: Key 71adf85dd30421e65a8f50e0b89a7a4a is removed from metadata 2024.10.22 00:49:24.980559 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uoj/wiijhvsvrwqejvyfuacfdztoidqxp (LogSeriesLimiter: on interval from 2024-10-22 00:48:58 to 2024-10-22 00:49:24 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:49:25.014992 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_177_177_0 to 202410_187_187_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:25.032473 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:25.145744 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:25.235989 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_377_377_0 (state Temporary) 2024.10.22 00:49:25.284759 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_377_377_0 to 202410_390_390_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:25.352730 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21459 2024.10.22 00:49:25.439189 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 195 2024.10.22 00:49:25.821811 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e9739a574e919f4bde29d7165966ee24) : [0, 523924]: Setting download as failed: File segment: [0, 523924], key: e9739a574e919f4bde29d7165966ee24, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:25.829574 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e9739a574e919f4bde29d7165966ee24): No space left in cache to reserve 523925 bytes, will continue without cache download 2024.10.22 00:49:25.873390 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_44_44_0 to 202410_47_47_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:25.965751 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e9739a574e919f4bde29d7165966ee24, offset: 0, size: 0 2024.10.22 00:49:25.966854 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e9739a574e919f4bde29d7165966ee24 for removal 2024.10.22 00:49:25.969126 [ 1476 ] {} CacheMetadata: Key e9739a574e919f4bde29d7165966ee24 is removed from metadata 2024.10.22 00:49:26.184089 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 67 entries to flush up to offset 14312 2024.10.22 00:49:26.431504 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 15.09 MiB, will set to 1.65 GiB (RSS), difference: -5.73 MiB 2024.10.22 00:49:26.501461 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:26.508979 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:26.568655 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5ff4936d42f689665ae139806c33b904) : [0, 337022]: Setting download as failed: File segment: [0, 337022], key: 5ff4936d42f689665ae139806c33b904, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:26.572242 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5ff4936d42f689665ae139806c33b904): No space left in cache to reserve 337023 bytes, will continue without cache download 2024.10.22 00:49:26.582492 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_219_219_0 (state Temporary) 2024.10.22 00:49:26.601571 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5ff4936d42f689665ae139806c33b904, offset: 0, size: 0 2024.10.22 00:49:26.606273 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5ff4936d42f689665ae139806c33b904 for removal 2024.10.22 00:49:26.612120 [ 1476 ] {} CacheMetadata: Key 5ff4936d42f689665ae139806c33b904 is removed from metadata 2024.10.22 00:49:26.731295 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_219_219_0 to 202410_225_225_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:26.914393 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14312 2024.10.22 00:49:27.407496 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1581c1a7d85e8ff4368377b355435d0c) : [0, 239202]: Setting download as failed: File segment: [0, 239202], key: 1581c1a7d85e8ff4368377b355435d0c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:27.409181 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1581c1a7d85e8ff4368377b355435d0c): No space left in cache to reserve 239203 bytes, will continue without cache download 2024.10.22 00:49:27.426758 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1581c1a7d85e8ff4368377b355435d0c, offset: 0, size: 0 2024.10.22 00:49:27.428386 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1581c1a7d85e8ff4368377b355435d0c for removal 2024.10.22 00:49:27.431575 [ 1476 ] {} CacheMetadata: Key 1581c1a7d85e8ff4368377b355435d0c is removed from metadata 2024.10.22 00:49:28.215266 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(27c2bc7b1ddb3e3bc6c9a74821069f65) : [0, 452249]: Setting download as failed: File segment: [0, 452249], key: 27c2bc7b1ddb3e3bc6c9a74821069f65, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:28.216884 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(27c2bc7b1ddb3e3bc6c9a74821069f65): No space left in cache to reserve 452250 bytes, will continue without cache download 2024.10.22 00:49:28.259173 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 27c2bc7b1ddb3e3bc6c9a74821069f65, offset: 0, size: 0 2024.10.22 00:49:28.262636 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 27c2bc7b1ddb3e3bc6c9a74821069f65 for removal 2024.10.22 00:49:28.267036 [ 1476 ] {} CacheMetadata: Key 27c2bc7b1ddb3e3bc6c9a74821069f65 is removed from metadata 2024.10.22 00:49:28.737163 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} MergeTask::MergeProjectionsStage: Merge sorted 375157 rows, containing 4 columns (4 merged, 0 gathered) in 14.755517398 sec., 25424.86243490518 rows/sec., 373.00 KiB/sec. 2024.10.22 00:49:28.807266 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1613 entries to flush up to offset 364683 2024.10.22 00:49:28.997023 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} MergedBlockOutputStream: filled checksums 202410_1_216_43 (state Temporary) 2024.10.22 00:49:29.012740 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 3102 2024.10.22 00:49:29.048341 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_216_43 to 202410_1_216_43 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:29.063215 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_216_43} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_211_42, 202410_216_216_0] -> 202410_1_216_43 2024.10.22 00:49:29.103970 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 00:49:29.223857 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(34b3a9e64bc1654c60be7b2e708bac93) : [0, 363187]: Setting download as failed: File segment: [0, 363187], key: 34b3a9e64bc1654c60be7b2e708bac93, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:29.225517 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(34b3a9e64bc1654c60be7b2e708bac93): No space left in cache to reserve 363188 bytes, will continue without cache download 2024.10.22 00:49:29.261299 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 34b3a9e64bc1654c60be7b2e708bac93, offset: 0, size: 0 2024.10.22 00:49:29.263665 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 34b3a9e64bc1654c60be7b2e708bac93 for removal 2024.10.22 00:49:29.268157 [ 1476 ] {} CacheMetadata: Key 34b3a9e64bc1654c60be7b2e708bac93 is removed from metadata 2024.10.22 00:49:29.316054 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:29.320666 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:29.363039 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 21522 2024.10.22 00:49:29.513655 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:29.516700 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:29.547270 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1751 2024.10.22 00:49:29.592151 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:29.596416 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:29.610837 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:49:29.702514 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:49:29.705763 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_221_221_0 (state Temporary) 2024.10.22 00:49:29.751626 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_211_211_0 (state Temporary) 2024.10.22 00:49:29.838614 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_211_211_0 to 202410_218_218_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:29.854317 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_378_378_0 (state Temporary) 2024.10.22 00:49:29.888163 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_221_221_0 to 202410_234_234_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:29.894528 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_378_378_0 to 202410_391_391_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:29.931655 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_386_83 to 202410_391_391_0 2024.10.22 00:49:29.945430 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:29.951232 [ 1182 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:29.972770 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_386_83 to 202410_391_391_0 into Compact with storage Full 2024.10.22 00:49:29.980241 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 364683 2024.10.22 00:49:30.008452 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(50f875bfcafd47e0b8c5b5577d61a67a) : [0, 191677]: Setting download as failed: File segment: [0, 191677], key: 50f875bfcafd47e0b8c5b5577d61a67a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:30.011207 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(50f875bfcafd47e0b8c5b5577d61a67a): No space left in cache to reserve 191678 bytes, will continue without cache download 2024.10.22 00:49:30.030729 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:30.037302 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3102 2024.10.22 00:49:30.053500 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 50f875bfcafd47e0b8c5b5577d61a67a, offset: 0, size: 0 2024.10.22 00:49:30.058899 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 50f875bfcafd47e0b8c5b5577d61a67a for removal 2024.10.22 00:49:30.062456 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_386_83, total 21965 rows starting from the beginning of the part 2024.10.22 00:49:30.064229 [ 1476 ] {} CacheMetadata: Key 50f875bfcafd47e0b8c5b5577d61a67a is removed from metadata 2024.10.22 00:49:30.076559 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21522 2024.10.22 00:49:30.173182 [ 1182 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 211 ms. 2024.10.22 00:49:30.192112 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_387_387_0, total 101 rows starting from the beginning of the part 2024.10.22 00:49:30.237714 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_388_388_0, total 76 rows starting from the beginning of the part 2024.10.22 00:49:30.287760 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_389_389_0, total 82 rows starting from the beginning of the part 2024.10.22 00:49:30.429511 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_390_390_0, total 66 rows starting from the beginning of the part 2024.10.22 00:49:30.434696 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 11.85 MiB, will set to 1.65 GiB (RSS), difference: 7.97 MiB 2024.10.22 00:49:30.465993 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_391_391_0, total 63 rows starting from the beginning of the part 2024.10.22 00:49:30.950240 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bbd4b03b656fc3de9a404f50b262a105) : [0, 191413]: Setting download as failed: File segment: [0, 191413], key: bbd4b03b656fc3de9a404f50b262a105, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:30.964055 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bbd4b03b656fc3de9a404f50b262a105): No space left in cache to reserve 191414 bytes, will continue without cache download 2024.10.22 00:49:30.986342 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bbd4b03b656fc3de9a404f50b262a105, offset: 0, size: 0 2024.10.22 00:49:30.988524 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bbd4b03b656fc3de9a404f50b262a105 for removal 2024.10.22 00:49:30.993462 [ 1476 ] {} CacheMetadata: Key bbd4b03b656fc3de9a404f50b262a105 is removed from metadata 2024.10.22 00:49:31.452600 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_285_57, 202410_286_286_0, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0] 2024.10.22 00:49:31.456945 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_285_57, 202410_286_286_0, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0] 2024.10.22 00:49:31.479686 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_285_57, 202410_286_286_0, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0] 2024.10.22 00:49:31.700452 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 160852 bytes for 2521c155cbde82ccc63f10ea3e554ff8:0) 2024.10.22 00:49:31.704908 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2173f17eb0c1f324914482492fc3fb9a, offset: 0, size: 132376 2024.10.22 00:49:31.715931 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2173f17eb0c1f324914482492fc3fb9a for removal 2024.10.22 00:49:31.720987 [ 1476 ] {} CacheMetadata: Key 2173f17eb0c1f324914482492fc3fb9a is removed from metadata 2024.10.22 00:49:31.723982 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2981cbf383d6aaf2d84b64017d2fcdf1, offset: 0, size: 168 2024.10.22 00:49:31.832343 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2981cbf383d6aaf2d84b64017d2fcdf1 for removal 2024.10.22 00:49:31.838611 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b7ef2c998f46d3cef95a33887b33a817, offset: 0, size: 151 2024.10.22 00:49:31.873580 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_391_84/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:49:31.944508 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b7ef2c998f46d3cef95a33887b33a817 for removal 2024.10.22 00:49:31.953545 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f155f6405c5731f84bc83abebaf2594c, offset: 0, size: 5128 2024.10.22 00:49:31.960433 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f155f6405c5731f84bc83abebaf2594c for removal 2024.10.22 00:49:31.966544 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d303bfdcfb1f74b4ae6487af259b2899, offset: 0, size: 156 2024.10.22 00:49:31.969502 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d303bfdcfb1f74b4ae6487af259b2899 for removal 2024.10.22 00:49:31.972710 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0d4fd97637a79387b7165e11a44e3a72, offset: 0, size: 158 2024.10.22 00:49:31.975146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0d4fd97637a79387b7165e11a44e3a72 for removal 2024.10.22 00:49:32.028472 [ 1476 ] {} CacheMetadata: Key 0d4fd97637a79387b7165e11a44e3a72 is removed from metadata 2024.10.22 00:49:32.035936 [ 1476 ] {} CacheMetadata: Key d303bfdcfb1f74b4ae6487af259b2899 is removed from metadata 2024.10.22 00:49:32.048742 [ 1476 ] {} CacheMetadata: Key f155f6405c5731f84bc83abebaf2594c is removed from metadata 2024.10.22 00:49:32.163816 [ 1476 ] {} CacheMetadata: Key b7ef2c998f46d3cef95a33887b33a817 is removed from metadata 2024.10.22 00:49:32.174335 [ 1476 ] {} CacheMetadata: Key 2981cbf383d6aaf2d84b64017d2fcdf1 is removed from metadata 2024.10.22 00:49:32.420246 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 10.90 MiB, will set to 1.65 GiB (RSS), difference: -3.56 MiB 2024.10.22 00:49:32.943187 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 196 2024.10.22 00:49:33.083772 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:33.085887 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:33.120290 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} MergeTask::MergeProjectionsStage: Merge sorted 22353 rows, containing 13 columns (13 merged, 0 gathered) in 3.156803738 sec., 7080.896329070427 rows/sec., 1.06 MiB/sec. 2024.10.22 00:49:33.149238 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5c6d65a9819adde918ffd44cc09eb7ca) : [0, 393924]: Setting download as failed: File segment: [0, 393924], key: 5c6d65a9819adde918ffd44cc09eb7ca, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:33.156773 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5c6d65a9819adde918ffd44cc09eb7ca): No space left in cache to reserve 393925 bytes, will continue without cache download 2024.10.22 00:49:33.182319 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5c6d65a9819adde918ffd44cc09eb7ca, offset: 0, size: 0 2024.10.22 00:49:33.183517 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5c6d65a9819adde918ffd44cc09eb7ca for removal 2024.10.22 00:49:33.185063 [ 1476 ] {} CacheMetadata: Key 5c6d65a9819adde918ffd44cc09eb7ca is removed from metadata 2024.10.22 00:49:33.195271 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} MergedBlockOutputStream: filled checksums 202410_1_391_84 (state Temporary) 2024.10.22 00:49:33.232711 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_178_178_0 (state Temporary) 2024.10.22 00:49:33.580114 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_391_84 to 202410_1_391_84 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:33.612859 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_178_178_0 to 202410_188_188_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:33.687021 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_391_84} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_386_83, 202410_391_391_0] -> 202410_1_391_84 2024.10.22 00:49:33.722055 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 196 2024.10.22 00:49:33.820763 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.82 MiB. 2024.10.22 00:49:33.946614 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(78afbb54517382fa38efaaeb2ac71ed9) : [0, 258148]: Setting download as failed: File segment: [0, 258148], key: 78afbb54517382fa38efaaeb2ac71ed9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:33.948168 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(78afbb54517382fa38efaaeb2ac71ed9): No space left in cache to reserve 258149 bytes, will continue without cache download 2024.10.22 00:49:34.168604 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 78afbb54517382fa38efaaeb2ac71ed9, offset: 0, size: 0 2024.10.22 00:49:34.184833 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 21604 2024.10.22 00:49:34.184876 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 78afbb54517382fa38efaaeb2ac71ed9 for removal 2024.10.22 00:49:34.188793 [ 1476 ] {} CacheMetadata: Key 78afbb54517382fa38efaaeb2ac71ed9 is removed from metadata 2024.10.22 00:49:34.427522 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 57 entries to flush up to offset 14369 2024.10.22 00:49:34.455644 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 10.47 MiB, will set to 1.65 GiB (RSS), difference: 1.93 MiB 2024.10.22 00:49:34.564592 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:34.667804 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:34.677159 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_156_31, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:49:34.678540 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_156_31, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:49:34.698684 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_156_31, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:49:34.719036 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_220_220_0 (state Temporary) 2024.10.22 00:49:34.764274 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_220_220_0 to 202410_226_226_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:34.999889 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14369 2024.10.22 00:49:35.175213 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:35.177654 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:35.355515 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_379_379_0 (state Temporary) 2024.10.22 00:49:35.398830 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_379_379_0 to 202410_392_392_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:35.573788 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21604 2024.10.22 00:49:35.699288 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4058665604290229f86a1adb23ff7c69) : [0, 985787]: Setting download as failed: File segment: [0, 985787], key: 4058665604290229f86a1adb23ff7c69, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:35.701179 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4058665604290229f86a1adb23ff7c69): No space left in cache to reserve 985788 bytes, will continue without cache download 2024.10.22 00:49:35.858330 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4058665604290229f86a1adb23ff7c69, offset: 0, size: 0 2024.10.22 00:49:35.860415 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4058665604290229f86a1adb23ff7c69 for removal 2024.10.22 00:49:35.864428 [ 1476 ] {} CacheMetadata: Key 4058665604290229f86a1adb23ff7c69 is removed from metadata 2024.10.22 00:49:36.750448 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f332abfb602fedf35f424d5c446afe35) : [0, 1116533]: Setting download as failed: File segment: [0, 1116533], key: f332abfb602fedf35f424d5c446afe35, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:36.752776 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f332abfb602fedf35f424d5c446afe35): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:49:36.791287 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f332abfb602fedf35f424d5c446afe35, offset: 0, size: 0 2024.10.22 00:49:36.793569 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f332abfb602fedf35f424d5c446afe35 for removal 2024.10.22 00:49:36.797762 [ 1476 ] {} CacheMetadata: Key f332abfb602fedf35f424d5c446afe35 is removed from metadata 2024.10.22 00:49:36.985011 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1433 entries to flush up to offset 366116 2024.10.22 00:49:37.095654 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 45 entries to flush up to offset 1796 2024.10.22 00:49:37.203050 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_162_33, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0, 202410_167_167_0] 2024.10.22 00:49:37.205471 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_162_33, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0, 202410_167_167_0] 2024.10.22 00:49:37.236233 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_162_33, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0, 202410_167_167_0] 2024.10.22 00:49:37.542699 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 3122 2024.10.22 00:49:37.558729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(203d57d6b325c88586492950608e26c5) : [0, 428733]: Setting download as failed: File segment: [0, 428733], key: 203d57d6b325c88586492950608e26c5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:49:37.560234 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(203d57d6b325c88586492950608e26c5): No space left in cache to reserve 428734 bytes, will continue without cache download 2024.10.22 00:49:37.584850 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 203d57d6b325c88586492950608e26c5, offset: 0, size: 0 2024.10.22 00:49:37.585704 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 203d57d6b325c88586492950608e26c5 for removal 2024.10.22 00:49:37.587461 [ 1476 ] {} CacheMetadata: Key 203d57d6b325c88586492950608e26c5 is removed from metadata 2024.10.22 00:49:37.898310 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:37.905491 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:37.945698 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:37.951025 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:38.080541 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 13 file segments (while reserving 101540 bytes for 8bb385825c4db872c8b0f214ad0e3ec0:0) 2024.10.22 00:49:38.083870 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2521c155cbde82ccc63f10ea3e554ff8, offset: 0, size: 160852 2024.10.22 00:49:38.086182 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2521c155cbde82ccc63f10ea3e554ff8 for removal 2024.10.22 00:49:38.087986 [ 1476 ] {} CacheMetadata: Key 2521c155cbde82ccc63f10ea3e554ff8 is removed from metadata 2024.10.22 00:49:38.088044 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aa890d44e391e01323fc22449ec13d2b, offset: 0, size: 155 2024.10.22 00:49:38.092322 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aa890d44e391e01323fc22449ec13d2b for removal 2024.10.22 00:49:38.095760 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 911e303bc27a75762e0068516e50edcf, offset: 0, size: 162 2024.10.22 00:49:38.108409 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 911e303bc27a75762e0068516e50edcf for removal 2024.10.22 00:49:38.113112 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6c3093661a1a95c1ffd54544e7d1ed98, offset: 0, size: 156 2024.10.22 00:49:38.115046 [ 1476 ] {} CacheMetadata: Key 911e303bc27a75762e0068516e50edcf is removed from metadata 2024.10.22 00:49:38.116511 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6c3093661a1a95c1ffd54544e7d1ed98 for removal 2024.10.22 00:49:38.119755 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3e08dd5b6115ef71c078939ce3065acb, offset: 0, size: 158 2024.10.22 00:49:38.122772 [ 1476 ] {} CacheMetadata: Key 6c3093661a1a95c1ffd54544e7d1ed98 is removed from metadata 2024.10.22 00:49:38.130904 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3e08dd5b6115ef71c078939ce3065acb for removal 2024.10.22 00:49:38.134294 [ 1476 ] {} CacheMetadata: Key aa890d44e391e01323fc22449ec13d2b is removed from metadata 2024.10.22 00:49:38.135579 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fae8f366279d7ffe67466adfa25dd396, offset: 0, size: 158 2024.10.22 00:49:38.143724 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fae8f366279d7ffe67466adfa25dd396 for removal 2024.10.22 00:49:38.147592 [ 1476 ] {} CacheMetadata: Key fae8f366279d7ffe67466adfa25dd396 is removed from metadata 2024.10.22 00:49:38.148659 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 57b3dbebcae799ac55585a0a4cba477f, offset: 0, size: 161 2024.10.22 00:49:38.157713 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 57b3dbebcae799ac55585a0a4cba477f for removal 2024.10.22 00:49:38.159976 [ 1476 ] {} CacheMetadata: Key 3e08dd5b6115ef71c078939ce3065acb is removed from metadata 2024.10.22 00:49:38.163294 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1d9e5c34037b90c12c2a000e31698415, offset: 0, size: 159 2024.10.22 00:49:38.169146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1d9e5c34037b90c12c2a000e31698415 for removal 2024.10.22 00:49:38.171319 [ 1476 ] {} CacheMetadata: Key 57b3dbebcae799ac55585a0a4cba477f is removed from metadata 2024.10.22 00:49:38.178672 [ 1476 ] {} CacheMetadata: Key 1d9e5c34037b90c12c2a000e31698415 is removed from metadata 2024.10.22 00:49:38.172329 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 01f4347eff5c6455ef0618ca54f943ce, offset: 0, size: 158 2024.10.22 00:49:38.182494 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_222_222_0 (state Temporary) 2024.10.22 00:49:38.184821 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 01f4347eff5c6455ef0618ca54f943ce for removal 2024.10.22 00:49:38.187683 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cc09397139c2b1699b998420916df995, offset: 0, size: 155 2024.10.22 00:49:38.189584 [ 1476 ] {} CacheMetadata: Key 01f4347eff5c6455ef0618ca54f943ce is removed from metadata 2024.10.22 00:49:38.191895 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cc09397139c2b1699b998420916df995 for removal 2024.10.22 00:49:38.195931 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2f30a2383c8a4bb7354109100043eb84, offset: 0, size: 160 2024.10.22 00:49:38.204446 [ 1476 ] {} CacheMetadata: Key cc09397139c2b1699b998420916df995 is removed from metadata 2024.10.22 00:49:38.205322 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2f30a2383c8a4bb7354109100043eb84 for removal 2024.10.22 00:49:38.216704 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b9be34530a674946688ce86b7e583d5e, offset: 0, size: 155 2024.10.22 00:49:38.219243 [ 1476 ] {} CacheMetadata: Key 2f30a2383c8a4bb7354109100043eb84 is removed from metadata 2024.10.22 00:49:38.224270 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_222_222_0 to 202410_235_235_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:38.321027 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b9be34530a674946688ce86b7e583d5e for removal 2024.10.22 00:49:38.323810 [ 1476 ] {} CacheMetadata: Key b9be34530a674946688ce86b7e583d5e is removed from metadata 2024.10.22 00:49:38.323889 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2aabdee09a15646e676e15c5c7bb2b6a, offset: 0, size: 156 2024.10.22 00:49:38.329389 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2aabdee09a15646e676e15c5c7bb2b6a for removal 2024.10.22 00:49:38.334906 [ 1476 ] {} CacheMetadata: Key 2aabdee09a15646e676e15c5c7bb2b6a is removed from metadata 2024.10.22 00:49:38.387949 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_212_212_0 (state Temporary) 2024.10.22 00:49:38.395706 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3122 2024.10.22 00:49:38.428206 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_212_212_0 to 202410_219_219_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:38.479430 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zuw/okdwmuxafpkoihsvgzsqvcflntrja. 2024.10.22 00:49:38.575323 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 366116 2024.10.22 00:49:38.721086 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 6.89 MiB, will set to 1.64 GiB (RSS), difference: -3.49 MiB 2024.10.22 00:49:38.724507 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fkx/vjaekrbzrijgdhuqawssewxvngcew. 2024.10.22 00:49:39.565200 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fkx/vjaekrbzrijgdhuqawssewxvngcew. 2024.10.22 00:49:39.568664 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_14_14_0 to 201403_14_14_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:39.619303 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zuw/okdwmuxafpkoihsvgzsqvcflntrja. 2024.10.22 00:49:39.647578 [ 1154 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:49:39.669227 [ 1154 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:49:39.683691 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 98 entries to flush up to offset 21702 2024.10.22 00:49:39.713041 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:49:39.716201 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:49:39.764559 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_15_15_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:49:39.818499 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:39.821637 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:40.019880 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_380_380_0 (state Temporary) 2024.10.22 00:49:40.179916 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_380_380_0 to 202410_393_393_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:40.313166 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21702 2024.10.22 00:49:40.672282 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.10 GiB, peak 1.88 GiB, free memory in arenas 6.02 MiB, will set to 1.49 GiB (RSS), difference: 399.48 MiB 2024.10.22 00:49:40.969773 [ 1721 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uoj/wiijhvsvrwqejvyfuacfdztoidqxp, size 7427 2024.10.22 00:49:41.226743 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 197 2024.10.22 00:49:41.602104 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsArtifical 2024.10.22 00:49:41.623616 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:41.631129 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:41.641779 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsArtifical 2024.10.22 00:49:41.662172 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsArtifical 2024.10.22 00:49:41.675926 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsArtifical 2024.10.22 00:49:41.694313 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsArtifical 2024.10.22 00:49:41.801097 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsArtifical 2024.10.22 00:49:41.827169 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsArtifical 2024.10.22 00:49:41.978138 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fqn/azxsbsrwhgejnvehfmsrwobeawppt (LogSeriesLimiter: on interval from 2024-10-22 00:49:24 to 2024-10-22 00:49:41 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:49:41.982973 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_179_179_0 (state Temporary) 2024.10.22 00:49:42.175569 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_179_179_0 to 202410_189_189_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:42.180226 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_171_34, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0] 2024.10.22 00:49:42.182261 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_171_34, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0] 2024.10.22 00:49:42.210821 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_184_38 to 202410_189_189_0 2024.10.22 00:49:42.213748 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_171_34, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0] 2024.10.22 00:49:42.227136 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:42.237347 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:42.379292 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_184_38 to 202410_189_189_0 into Compact with storage Full 2024.10.22 00:49:42.521587 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 197 2024.10.22 00:49:42.549027 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 27 entries to flush up to offset 14396 2024.10.22 00:49:42.570587 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:42.579211 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_184_38, total 228 rows starting from the beginning of the part 2024.10.22 00:49:42.690916 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 14 2024.10.22 00:49:42.713745 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_185_185_0, total 1 rows starting from the beginning of the part 2024.10.22 00:49:42.789606 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:42.792376 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:42.857658 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_221_221_0 (state Temporary) 2024.10.22 00:49:42.871004 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_186_186_0, total 1 rows starting from the beginning of the part 2024.10.22 00:49:42.872342 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 6.00 MiB, will set to 1.41 GiB (RSS), difference: -244.96 MiB 2024.10.22 00:49:42.892727 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_221_221_0 to 202410_227_227_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:42.897607 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:42.898674 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_187_187_0, total 1 rows starting from the beginning of the part 2024.10.22 00:49:42.902475 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:42.933074 [ 1171 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_222_47 to 202410_223_223_0 2024.10.22 00:49:42.966297 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14396 2024.10.22 00:49:43.039907 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_188_188_0, total 1 rows starting from the beginning of the part 2024.10.22 00:49:43.042600 [ 1171 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:43.049001 [ 1171 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:43.067504 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_14_14_0 (state Temporary) 2024.10.22 00:49:43.094215 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_223_48} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_222_47 to 202410_223_223_0 into Compact with storage Full 2024.10.22 00:49:43.104841 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_189_189_0, total 1 rows starting from the beginning of the part 2024.10.22 00:49:43.184567 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_223_48} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:43.222623 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_14_14_0 to 202410_14_14_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:43.309738 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_223_48} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_222_47, total 14219 rows starting from the beginning of the part 2024.10.22 00:49:43.404282 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 14 2024.10.22 00:49:43.476557 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_223_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_223_223_0, total 74 rows starting from the beginning of the part 2024.10.22 00:49:44.326106 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 21761 2024.10.22 00:49:44.472284 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_223_48} MergeTask::MergeProjectionsStage: Merge sorted 14293 rows, containing 13 columns (13 merged, 0 gathered) in 1.39486335 sec., 10246.881889899823 rows/sec., 4.30 MiB/sec. 2024.10.22 00:49:44.555624 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.88 GiB, free memory in arenas 2.89 MiB, will set to 1.45 GiB (RSS), difference: -10.96 MiB 2024.10.22 00:49:44.594652 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:44.598372 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.46 GiB. 2024.10.22 00:49:44.659490 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_223_48} MergedBlockOutputStream: filled checksums 202410_1_223_48 (state Temporary) 2024.10.22 00:49:44.667699 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_381_381_0 (state Temporary) 2024.10.22 00:49:44.803736 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_381_381_0 to 202410_394_394_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:44.864473 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21761 2024.10.22 00:49:44.915204 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} MergeTask::MergeProjectionsStage: Merge sorted 233 rows, containing 32 columns (32 merged, 0 gathered) in 2.542652482 sec., 91.63658881796023 rows/sec., 9.71 KiB/sec. 2024.10.22 00:49:44.971806 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_223_48} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_223_48 to 202410_1_223_48 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:44.974741 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_223_48} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_222_47, 202410_223_223_0] -> 202410_1_223_48 2024.10.22 00:49:44.975338 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} MergedBlockOutputStream: filled checksums 202410_1_189_39 (state Temporary) 2024.10.22 00:49:44.995584 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.64 MiB. 2024.10.22 00:49:45.579037 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 368036 2024.10.22 00:49:45.900386 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 3133 2024.10.22 00:49:45.984238 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_189_39 to 202410_1_189_39 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:46.103913 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_189_39} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_184_38, 202410_189_189_0] -> 202410_1_189_39 2024.10.22 00:49:46.135119 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:46.144964 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:46.160617 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:49:46.188009 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_223_223_0 (state Temporary) 2024.10.22 00:49:46.437002 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:46.441697 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:46.442704 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_223_223_0 to 202410_236_236_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:46.486362 [ 1187 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_231_46 to 202410_236_236_0 2024.10.22 00:49:46.493461 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.88 GiB, free memory in arenas 3.24 MiB, will set to 1.49 GiB (RSS), difference: -48.87 MiB 2024.10.22 00:49:46.570986 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3133 2024.10.22 00:49:46.598904 [ 1187 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:46.612484 [ 1187 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:46.634516 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_231_46 to 202410_236_236_0 into Compact with storage Full 2024.10.22 00:49:46.680508 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:46.683394 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_213_213_0 (state Temporary) 2024.10.22 00:49:46.765325 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_213_213_0 to 202410_220_220_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:46.874909 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 368036 2024.10.22 00:49:46.880761 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_231_46, total 3165 rows starting from the beginning of the part 2024.10.22 00:49:46.909181 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_232_232_0, total 26 rows starting from the beginning of the part 2024.10.22 00:49:46.933725 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_233_233_0, total 6 rows starting from the beginning of the part 2024.10.22 00:49:46.971825 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_234_234_0, total 11 rows starting from the beginning of the part 2024.10.22 00:49:46.990681 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_235_235_0, total 20 rows starting from the beginning of the part 2024.10.22 00:49:47.017352 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_236_236_0, total 11 rows starting from the beginning of the part 2024.10.22 00:49:48.270627 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} MergeTask::MergeProjectionsStage: Merge sorted 3239 rows, containing 12 columns (12 merged, 0 gathered) in 1.645703422 sec., 1968.1553533282986 rows/sec., 306.81 KiB/sec. 2024.10.22 00:49:48.421621 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} MergedBlockOutputStream: filled checksums 202410_1_236_47 (state Temporary) 2024.10.22 00:49:48.444446 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.88 GiB, free memory in arenas 3.36 MiB, will set to 1.53 GiB (RSS), difference: -30.12 MiB 2024.10.22 00:49:48.822938 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_236_47 to 202410_1_236_47 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:48.827533 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_236_47} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_231_46, 202410_236_236_0] -> 202410_1_236_47 2024.10.22 00:49:48.884192 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 21807 2024.10.22 00:49:48.972544 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:49:49.289692 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:49.292921 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:49.345713 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_382_382_0 (state Temporary) 2024.10.22 00:49:49.486571 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_382_382_0 to 202410_395_395_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:49.672858 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21807 2024.10.22 00:49:49.716278 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:49:49.720605 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:49:50.030249 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 198 2024.10.22 00:49:50.314326 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.88 GiB, free memory in arenas 3.37 MiB, will set to 1.55 GiB (RSS), difference: 7.75 MiB 2024.10.22 00:49:50.439644 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:50.445634 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:50.470785 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 70 entries to flush up to offset 14466 2024.10.22 00:49:50.618687 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_180_180_0 (state Temporary) 2024.10.22 00:49:50.683031 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:50.686992 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:50.748848 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_222_222_0 (state Temporary) 2024.10.22 00:49:50.889498 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_222_222_0 to 202410_228_228_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:50.896866 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_180_180_0 to 202410_190_190_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:50.926464 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_223_48 to 202410_224_224_0 2024.10.22 00:49:50.945251 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:50.952912 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:50.990690 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 198 2024.10.22 00:49:51.128231 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_224_49} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_223_48 to 202410_224_224_0 into Compact with storage Full 2024.10.22 00:49:51.156409 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14466 2024.10.22 00:49:51.186454 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_224_49} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:51.210887 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_224_49} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_223_48, total 14293 rows starting from the beginning of the part 2024.10.22 00:49:51.249491 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_224_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_224_224_0, total 56 rows starting from the beginning of the part 2024.10.22 00:49:51.794772 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_224_49} MergeTask::MergeProjectionsStage: Merge sorted 14349 rows, containing 13 columns (13 merged, 0 gathered) in 0.780911969 sec., 18374.670346485622 rows/sec., 7.71 MiB/sec. 2024.10.22 00:49:51.979113 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_224_49} MergedBlockOutputStream: filled checksums 202410_1_224_49 (state Temporary) 2024.10.22 00:49:52.246314 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_224_49} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_224_49 to 202410_1_224_49 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:52.252095 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_224_49} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_223_48, 202410_224_224_0] -> 202410_1_224_49 2024.10.22 00:49:52.271866 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.64 MiB. 2024.10.22 00:49:52.503694 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 2.34 MiB, will set to 1.59 GiB (RSS), difference: -68.95 MiB 2024.10.22 00:49:53.574420 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.88 GiB, free memory in arenas 2.43 MiB, will set to 1.62 GiB (RSS), difference: -143.55 MiB 2024.10.22 00:49:53.683806 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 21843 2024.10.22 00:49:53.849041 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_15_15_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:49:53.880208 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1583 entries to flush up to offset 369619 2024.10.22 00:49:53.930881 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:53.934543 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:53.985412 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_383_383_0 (state Temporary) 2024.10.22 00:49:54.072284 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 3155 2024.10.22 00:49:54.141700 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_383_383_0 to 202410_396_396_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:54.183131 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:54.189048 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:54.193446 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 3 parts from 202410_1_391_84 to 202410_393_393_0 2024.10.22 00:49:54.207470 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21843 2024.10.22 00:49:54.220247 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:54.225030 [ 1241 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:54.281069 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_393_85} MergeTask::PrepareStage: Merging 3 parts: from 202410_1_391_84 to 202410_393_393_0 into Compact with storage Full 2024.10.22 00:49:54.334274 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_224_224_0 (state Temporary) 2024.10.22 00:49:54.386237 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_224_224_0 to 202410_237_237_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:54.444841 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:54.452157 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:54.462546 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_393_85} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:54.491111 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_393_85} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_391_84, total 22353 rows starting from the beginning of the part 2024.10.22 00:49:54.520471 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_393_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_392_392_0, total 82 rows starting from the beginning of the part 2024.10.22 00:49:54.556511 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_393_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_393_393_0, total 98 rows starting from the beginning of the part 2024.10.22 00:49:54.619880 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3155 2024.10.22 00:49:54.675821 [ 1255 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:49:54.719855 [ 1255 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:49:54.720930 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:54.725074 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:54.781739 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_290_58, 202410_291_291_0, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0] 2024.10.22 00:49:54.784203 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_290_58, 202410_291_291_0, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0] 2024.10.22 00:49:54.836924 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_290_58, 202410_291_291_0, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0] 2024.10.22 00:49:55.185641 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_214_214_0 (state Temporary) 2024.10.22 00:49:55.235115 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_214_214_0 to 202410_221_221_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:55.268283 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 369619 2024.10.22 00:49:55.367588 [ 1252 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_216_43 to 202410_221_221_0 2024.10.22 00:49:55.378530 [ 1252 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:55.381003 [ 1252 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:55.394917 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(09fff66073688a8c95ab8b1df6521cc0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 09fff66073688a8c95ab8b1df6521cc0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 00:49:55.397086 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 915037, file segment info: File segment: [0, 33554431], key: 09fff66073688a8c95ab8b1df6521cc0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 00:49:55.400121 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_216_43 to 202410_221_221_0 into Compact with storage Full 2024.10.22 00:49:55.403656 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:49:55.470268 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:55.479271 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} MergeTreeSequentialSource: Reading 47 marks from part 202410_1_216_43, total 375157 rows starting from the beginning of the part 2024.10.22 00:49:55.490001 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_217_217_0, total 1503 rows starting from the beginning of the part 2024.10.22 00:49:55.490648 [ 1733 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fqn/azxsbsrwhgejnvehfmsrwobeawppt, size 915037 2024.10.22 00:49:55.512393 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_218_218_0, total 1613 rows starting from the beginning of the part 2024.10.22 00:49:55.532864 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_219_219_0, total 1433 rows starting from the beginning of the part 2024.10.22 00:49:55.543785 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.88 GiB, free memory in arenas 2.40 MiB, will set to 1.78 GiB (RSS), difference: 183.23 MiB 2024.10.22 00:49:55.567767 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_220_220_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:49:55.585326 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_221_221_0, total 1583 rows starting from the beginning of the part 2024.10.22 00:49:55.620483 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 00:49:55.628003 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 00:49:55.671590 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 00:49:55.686889 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 00:49:55.706044 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 00:49:55.743436 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 00:49:55.867649 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 00:49:55.896252 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_137_28, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0, 202410_142_142_0] 2024.10.22 00:49:55.902480 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_137_28, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0, 202410_142_142_0] 2024.10.22 00:49:55.945478 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_137_28, 202410_138_138_0, 202410_139_139_0, 202410_140_140_0, 202410_141_141_0, 202410_142_142_0] 2024.10.22 00:49:55.992579 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wbu/uhizqurvulkkgcemctabgydvlsbzz (LogSeriesLimiter: on interval from 2024-10-22 00:49:41 to 2024-10-22 00:49:55 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:49:56.298848 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_393_85} MergeTask::MergeProjectionsStage: Merge sorted 22533 rows, containing 13 columns (13 merged, 0 gathered) in 2.039940216 sec., 11045.911945490072 rows/sec., 1.65 MiB/sec. 2024.10.22 00:49:56.371047 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_393_85} MergedBlockOutputStream: filled checksums 202410_1_393_85 (state Temporary) 2024.10.22 00:49:56.533786 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_393_85} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_393_85 to 202410_1_393_85 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:56.548599 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_393_85} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 3 parts: [202410_1_391_84, 202410_393_393_0] -> 202410_1_393_85 2024.10.22 00:49:56.588248 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.65 MiB. 2024.10.22 00:49:57.393809 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.88 GiB, free memory in arenas 5.96 MiB, will set to 1.62 GiB (RSS), difference: 116.99 MiB 2024.10.22 00:49:57.396161 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_45_45_0 (state Temporary) 2024.10.22 00:49:58.050434 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 491.09 MiB using storage policy from min volume index 0 2024.10.22 00:49:58.213213 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 21912 2024.10.22 00:49:58.628340 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_45_45_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:49:58.667746 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 95 entries to flush up to offset 14561 2024.10.22 00:49:58.761596 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_221_44/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:49:58.880573 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:58.883109 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:58.935380 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_223_223_0 (state Temporary) 2024.10.22 00:49:58.947212 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:58.952433 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:59.087597 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_223_223_0 to 202410_229_229_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:59.135810 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_384_384_0 (state Temporary) 2024.10.22 00:49:59.140787 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_224_49 to 202410_225_225_0 2024.10.22 00:49:59.151573 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:49:59.161605 [ 1150 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:49:59.170784 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_225_50} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_224_49 to 202410_225_225_0 into Compact with storage Full 2024.10.22 00:49:59.277782 [ 1158 ] {} DatabaseCatalog: Execution took 215 ms. 2024.10.22 00:49:59.282725 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_384_384_0 to 202410_397_397_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:49:59.286688 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14561 2024.10.22 00:49:59.381526 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_225_50} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:49:59.450382 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21912 2024.10.22 00:49:59.466699 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 10.65 MiB, will set to 1.63 GiB (RSS), difference: 1.37 MiB 2024.10.22 00:49:59.487811 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_225_50} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_224_49, total 14349 rows starting from the beginning of the part 2024.10.22 00:49:59.499412 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_225_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_225_225_0, total 67 rows starting from the beginning of the part 2024.10.22 00:49:59.717581 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:49:59.719934 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:50:00.222289 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_225_50} MergeTask::MergeProjectionsStage: Merge sorted 14416 rows, containing 13 columns (13 merged, 0 gathered) in 1.054474046 sec., 13671.270577673375 rows/sec., 5.74 MiB/sec. 2024.10.22 00:50:00.231711 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ech/gnqlxuxuamziemjkojorbcvelgycp (LogSeriesLimiter: on interval from 2024-10-22 00:49:55 to 2024-10-22 00:50:00 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:50:00.424259 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_225_50} MergedBlockOutputStream: filled checksums 202410_1_225_50 (state Temporary) 2024.10.22 00:50:00.801020 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_225_50} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_225_50 to 202410_1_225_50 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:00.806852 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_225_50} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_224_49, 202410_225_225_0] -> 202410_1_225_50 2024.10.22 00:50:00.842566 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.64 MiB. 2024.10.22 00:50:01.484432 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.88 GiB, free memory in arenas 11.50 MiB, will set to 1.63 GiB (RSS), difference: -113.20 MiB 2024.10.22 00:50:02.121804 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 3167 2024.10.22 00:50:02.260492 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:02.268264 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:02.271836 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1873 entries to flush up to offset 371492 2024.10.22 00:50:02.300634 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_225_225_0 (state Temporary) 2024.10.22 00:50:02.473173 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_225_225_0 to 202410_238_238_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:02.504139 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3167 2024.10.22 00:50:03.297234 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:03.306810 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:03.420791 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_215_215_0 (state Temporary) 2024.10.22 00:50:03.464933 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 21954 2024.10.22 00:50:03.509093 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(6a2c5d5778e4af3f611d8881ba1a400b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6a2c5d5778e4af3f611d8881ba1a400b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 00:50:03.515585 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 6a2c5d5778e4af3f611d8881ba1a400b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 00:50:03.521556 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:50:03.526741 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_215_215_0 to 202410_222_222_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:03.572429 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 371492 2024.10.22 00:50:03.611315 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.83 GiB, peak 1.88 GiB, free memory in arenas 11.12 MiB, will set to 1.63 GiB (RSS), difference: -199.27 MiB 2024.10.22 00:50:03.690793 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:03.693136 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:03.916373 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_385_385_0 (state Temporary) 2024.10.22 00:50:03.971845 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_385_385_0 to 202410_398_398_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:03.991454 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 5 parts from 202410_1_393_85 to 202410_397_397_0 2024.10.22 00:50:04.008777 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 21954 2024.10.22 00:50:04.009827 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:04.013593 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:04.031978 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_397_86} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_393_85 to 202410_397_397_0 into Compact with storage Full 2024.10.22 00:50:04.113562 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_397_86} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:04.226121 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_397_86} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_393_85, total 22533 rows starting from the beginning of the part 2024.10.22 00:50:04.272988 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_397_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_394_394_0, total 59 rows starting from the beginning of the part 2024.10.22 00:50:04.312495 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_397_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_395_395_0, total 46 rows starting from the beginning of the part 2024.10.22 00:50:04.361509 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_397_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_396_396_0, total 36 rows starting from the beginning of the part 2024.10.22 00:50:04.427774 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_397_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_397_397_0, total 69 rows starting from the beginning of the part 2024.10.22 00:50:05.117403 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mvg/niiynvsppfukcsdhuoxgnkhrzdcdz (LogSeriesLimiter: on interval from 2024-10-22 00:50:00 to 2024-10-22 00:50:05 accepted series 1 / 289 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:50:05.369823 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.88 GiB, free memory in arenas 10.70 MiB, will set to 1.64 GiB (RSS), difference: -161.30 MiB 2024.10.22 00:50:05.381839 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 00:50:05.622888 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_45_45_0 to 202410_48_48_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:05.984919 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_397_86} MergeTask::MergeProjectionsStage: Merge sorted 22743 rows, containing 13 columns (13 merged, 0 gathered) in 1.959540358 sec., 11606.293234609664 rows/sec., 1.74 MiB/sec. 2024.10.22 00:50:05.995292 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 199 2024.10.22 00:50:06.269172 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_397_86} MergedBlockOutputStream: filled checksums 202410_1_397_86 (state Temporary) 2024.10.22 00:50:06.366895 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_397_86} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_397_86 to 202410_1_397_86 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:06.370578 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_397_86} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 5 parts: [202410_1_393_85, 202410_397_397_0] -> 202410_1_397_86 2024.10.22 00:50:06.432387 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:06.448912 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:06.493527 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.73 MiB. 2024.10.22 00:50:06.523301 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_181_181_0 (state Temporary) 2024.10.22 00:50:06.597825 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_181_181_0 to 202410_191_191_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:06.675903 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 199 2024.10.22 00:50:06.800435 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 83 entries to flush up to offset 14644 2024.10.22 00:50:07.199241 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:07.204533 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:07.279696 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_224_224_0 (state Temporary) 2024.10.22 00:50:07.412564 [ 1201 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 00:50:07.433347 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_224_224_0 to 202410_230_230_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:07.463084 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 9.01 MiB, will set to 1.64 GiB (RSS), difference: -59.74 MiB 2024.10.22 00:50:07.470434 [ 1199 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_225_50 to 202410_226_226_0 2024.10.22 00:50:07.476759 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14644 2024.10.22 00:50:07.475347 [ 1199 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:07.489583 [ 1199 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:07.689663 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_226_51} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_225_50 to 202410_226_226_0 into Compact with storage Full 2024.10.22 00:50:07.820412 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1796 2024.10.22 00:50:07.984133 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_226_51} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:07.996222 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_226_51} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_225_50, total 14416 rows starting from the beginning of the part 2024.10.22 00:50:08.019113 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 22006 2024.10.22 00:50:08.059399 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_226_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_226_226_0, total 57 rows starting from the beginning of the part 2024.10.22 00:50:08.178518 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:08.183603 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:08.460162 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_386_386_0 (state Temporary) 2024.10.22 00:50:08.504477 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_386_386_0 to 202410_399_399_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:08.660791 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22006 2024.10.22 00:50:09.438463 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_221_44/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:50:09.665725 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 18.05 MiB, will set to 1.66 GiB (RSS), difference: -6.06 MiB 2024.10.22 00:50:09.730326 [ 1213 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:50:09.752850 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:50:09.753571 [ 1222 ] {} DatabaseCatalog: Execution took 471 ms. 2024.10.22 00:50:09.763505 [ 1213 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:50:09.769181 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_226_51} MergeTask::MergeProjectionsStage: Merge sorted 14473 rows, containing 13 columns (13 merged, 0 gathered) in 2.082672292 sec., 6949.24499432482 rows/sec., 2.92 MiB/sec. 2024.10.22 00:50:09.819623 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:50:09.891433 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.172422129 sec. 2024.10.22 00:50:10.009177 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 3180 2024.10.22 00:50:10.063974 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_226_51} MergedBlockOutputStream: filled checksums 202410_1_226_51 (state Temporary) 2024.10.22 00:50:10.074199 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} MergeTask::MergeProjectionsStage: Merge sorted 383209 rows, containing 4 columns (4 merged, 0 gathered) in 14.678911083 sec., 26106.091782503103 rows/sec., 382.95 KiB/sec. 2024.10.22 00:50:10.320403 [ 1704 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wbu/uhizqurvulkkgcemctabgydvlsbzz, size 1740305 2024.10.22 00:50:10.357761 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} MergedBlockOutputStream: filled checksums 202410_1_221_44 (state Temporary) 2024.10.22 00:50:10.405241 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_221_44 to 202410_1_221_44 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:10.410881 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_221_44} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_216_43, 202410_221_221_0] -> 202410_1_221_44 2024.10.22 00:50:10.562712 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:50:10.565629 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:10.567508 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:10.573540 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_226_51} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_226_51 to 202410_1_226_51 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:10.575583 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_226_51} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_225_50, 202410_226_226_0] -> 202410_1_226_51 2024.10.22 00:50:10.580346 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1484 entries to flush up to offset 372976 2024.10.22 00:50:10.586757 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.64 MiB. 2024.10.22 00:50:10.587133 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_226_226_0 (state Temporary) 2024.10.22 00:50:10.771413 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_226_226_0 to 202410_239_239_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:10.931057 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3180 2024.10.22 00:50:10.975393 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 00:50:10.982821 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 00:50:10.989411 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 00:50:11.000487 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 00:50:11.035926 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 00:50:11.067458 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 00:50:11.195671 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 00:50:11.234826 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rfq/fglcubdxqgsfqomsavhnupwneemnh (LogSeriesLimiter: on interval from 2024-10-22 00:50:05 to 2024-10-22 00:50:11 accepted series 1 / 64 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:50:11.479481 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:11.482028 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:11.587157 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_216_216_0 (state Temporary) 2024.10.22 00:50:11.587122 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 16.78 MiB, will set to 1.66 GiB (RSS), difference: 4.83 MiB 2024.10.22 00:50:11.654133 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_216_216_0 to 202410_223_223_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:11.696088 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 372976 2024.10.22 00:50:12.352408 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 00:50:12.671726 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 22054 2024.10.22 00:50:12.781089 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:12.785355 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:12.860235 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_387_387_0 (state Temporary) 2024.10.22 00:50:13.134423 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_387_387_0 to 202410_400_400_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:13.205830 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22054 2024.10.22 00:50:13.559304 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 16.29 MiB, will set to 1.66 GiB (RSS), difference: 1.40 MiB 2024.10.22 00:50:14.180180 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 200 2024.10.22 00:50:14.540040 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:14.543719 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:14.560651 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 16.22 MiB, will set to 1.67 GiB (RSS), difference: 1.72 MiB 2024.10.22 00:50:14.591035 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_182_182_0 (state Temporary) 2024.10.22 00:50:14.987491 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 63 entries to flush up to offset 14707 2024.10.22 00:50:15.168843 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_182_182_0 to 202410_192_192_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:15.220269 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:15.225439 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:15.235773 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 200 2024.10.22 00:50:15.363260 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 1834 2024.10.22 00:50:15.521733 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_225_225_0 (state Temporary) 2024.10.22 00:50:15.564941 [ 1151 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 00:50:15.597523 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_225_225_0 to 202410_231_231_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:15.656508 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_226_51 to 202410_227_227_0 2024.10.22 00:50:15.660996 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:15.663260 [ 1237 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:15.689899 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_227_52} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_226_51 to 202410_227_227_0 into Compact with storage Full 2024.10.22 00:50:15.761074 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_227_52} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:15.770890 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_227_52} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_226_51, total 14473 rows starting from the beginning of the part 2024.10.22 00:50:15.786437 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_227_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_227_227_0, total 27 rows starting from the beginning of the part 2024.10.22 00:50:15.895145 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14707 2024.10.22 00:50:16.500941 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 14.61 MiB, will set to 1.68 GiB (RSS), difference: -11.62 MiB 2024.10.22 00:50:16.574569 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_227_52} MergeTask::MergeProjectionsStage: Merge sorted 14500 rows, containing 13 columns (13 merged, 0 gathered) in 0.891066159 sec., 16272.641322472218 rows/sec., 6.83 MiB/sec. 2024.10.22 00:50:16.657110 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_227_52} MergedBlockOutputStream: filled checksums 202410_1_227_52 (state Temporary) 2024.10.22 00:50:17.018706 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_227_52} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_227_52 to 202410_1_227_52 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:17.021370 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_227_52} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_226_51, 202410_227_227_0] -> 202410_1_227_52 2024.10.22 00:50:17.145073 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.61 MiB. 2024.10.22 00:50:17.219805 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 22089 2024.10.22 00:50:17.319470 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_161_32, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0] 2024.10.22 00:50:17.321811 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_161_32, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0] 2024.10.22 00:50:17.339001 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:17.350650 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:17.394435 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_388_388_0 (state Temporary) 2024.10.22 00:50:17.447991 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_388_388_0 to 202410_401_401_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:17.472705 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_161_32, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0, 202410_166_166_0] 2024.10.22 00:50:17.491974 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22089 2024.10.22 00:50:18.384597 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 12.98 MiB, will set to 1.68 GiB (RSS), difference: 10.08 MiB 2024.10.22 00:50:18.433243 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 3199 2024.10.22 00:50:18.444088 [ 1231 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 00:50:18.641250 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:18.648590 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:18.686372 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_227_227_0 (state Temporary) 2024.10.22 00:50:18.701572 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1822 entries to flush up to offset 374798 2024.10.22 00:50:18.764423 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_227_227_0 to 202410_240_240_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:18.784273 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3199 2024.10.22 00:50:18.825688 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_167_34, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0, 202410_172_172_0] 2024.10.22 00:50:18.832165 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_167_34, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0, 202410_172_172_0] 2024.10.22 00:50:18.975565 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_167_34, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0, 202410_172_172_0] 2024.10.22 00:50:19.425709 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(c30984cb930c9d84b49870a9566d0beb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c30984cb930c9d84b49870a9566d0beb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 00:50:19.429169 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: c30984cb930c9d84b49870a9566d0beb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 00:50:19.436012 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:50:19.544170 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:19.550886 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:19.719710 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:50:19.722803 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:50:19.873826 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_217_217_0 (state Temporary) 2024.10.22 00:50:19.882864 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_295_59, 202410_296_296_0, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0] 2024.10.22 00:50:19.884269 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_295_59, 202410_296_296_0, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0] 2024.10.22 00:50:20.033691 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_295_59, 202410_296_296_0, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0] 2024.10.22 00:50:20.045951 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_217_217_0 to 202410_224_224_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:20.082323 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 374798 2024.10.22 00:50:20.136417 [ 1154 ] {} DatabaseCatalog: Execution took 382 ms. 2024.10.22 00:50:21.502865 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 22124 2024.10.22 00:50:21.567918 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 11.71 MiB, will set to 1.68 GiB (RSS), difference: 5.20 MiB 2024.10.22 00:50:21.692271 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:21.695157 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:21.841132 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_389_389_0 (state Temporary) 2024.10.22 00:50:22.088588 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_389_389_0 to 202410_402_402_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:22.205221 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 00:50:22.220745 [ 1275 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_397_86 to 202410_398_398_0 2024.10.22 00:50:22.225315 [ 1275 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:22.232370 [ 1275 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:22.290078 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22124 2024.10.22 00:50:22.305261 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_398_87} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_397_86 to 202410_398_398_0 into Compact with storage Full 2024.10.22 00:50:22.470281 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_398_87} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:22.479950 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_398_87} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_397_86, total 22743 rows starting from the beginning of the part 2024.10.22 00:50:22.498802 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_398_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_398_398_0, total 42 rows starting from the beginning of the part 2024.10.22 00:50:22.724595 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_29_6, 202410_30_30_0, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0] 2024.10.22 00:50:22.727773 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_29_6, 202410_30_30_0, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0] 2024.10.22 00:50:22.740848 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 201 2024.10.22 00:50:22.876184 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_29_6, 202410_30_30_0, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0] 2024.10.22 00:50:23.122172 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:23.132929 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:23.314857 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_176_35, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0] 2024.10.22 00:50:23.319551 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_176_35, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0] 2024.10.22 00:50:23.359868 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_176_35, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0] 2024.10.22 00:50:23.405381 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_183_183_0 (state Temporary) 2024.10.22 00:50:23.415151 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 66 entries to flush up to offset 14773 2024.10.22 00:50:23.510279 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 10.62 MiB, will set to 1.69 GiB (RSS), difference: -1.58 MiB 2024.10.22 00:50:23.751740 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_183_183_0 to 202410_193_193_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:23.765150 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:23.870365 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:23.915976 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_226_226_0 (state Temporary) 2024.10.22 00:50:23.930802 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_398_87} MergeTask::MergeProjectionsStage: Merge sorted 22785 rows, containing 13 columns (13 merged, 0 gathered) in 1.634837206 sec., 13937.167515136673 rows/sec., 2.09 MiB/sec. 2024.10.22 00:50:23.996488 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_398_87} MergedBlockOutputStream: filled checksums 202410_1_398_87 (state Temporary) 2024.10.22 00:50:24.041469 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 201 2024.10.22 00:50:24.067162 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_226_226_0 to 202410_232_232_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:24.086785 [ 1166 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_227_52 to 202410_228_228_0 2024.10.22 00:50:24.088889 [ 1186 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 00:50:24.092412 [ 1166 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:24.097604 [ 1166 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:24.101319 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14773 2024.10.22 00:50:24.131521 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_228_53} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_227_52 to 202410_228_228_0 into Compact with storage Full 2024.10.22 00:50:24.211976 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_228_53} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:24.320215 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_228_53} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_227_52, total 14500 rows starting from the beginning of the part 2024.10.22 00:50:24.368949 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_398_87} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_398_87 to 202410_1_398_87 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:24.372347 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_398_87} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_397_86, 202410_398_398_0] -> 202410_1_398_87 2024.10.22 00:50:24.456872 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_228_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_228_228_0, total 70 rows starting from the beginning of the part 2024.10.22 00:50:24.476294 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.73 MiB. 2024.10.22 00:50:24.871586 [ 1172 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:50:24.908067 [ 1172 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:50:25.541320 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 10.70 MiB, will set to 1.70 GiB (RSS), difference: 11.12 MiB 2024.10.22 00:50:25.635257 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_228_53} MergeTask::MergeProjectionsStage: Merge sorted 14570 rows, containing 13 columns (13 merged, 0 gathered) in 1.517159188 sec., 9603.47478052514 rows/sec., 4.03 MiB/sec. 2024.10.22 00:50:25.911632 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_228_53} MergedBlockOutputStream: filled checksums 202410_1_228_53 (state Temporary) 2024.10.22 00:50:26.023214 [ 1189 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 00:50:26.084769 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_228_53} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_228_53 to 202410_1_228_53 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:26.112200 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_228_53} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_227_52, 202410_228_228_0] -> 202410_1_228_53 2024.10.22 00:50:26.145246 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.64 MiB. 2024.10.22 00:50:26.286292 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 3210 2024.10.22 00:50:26.303659 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 22181 2024.10.22 00:50:26.508325 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:26.515923 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:26.565278 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_228_228_0 (state Temporary) 2024.10.22 00:50:26.611665 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_228_228_0 to 202410_241_241_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:26.684313 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3210 2024.10.22 00:50:26.684512 [ 1197 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_236_47 to 202410_241_241_0 2024.10.22 00:50:26.707362 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:26.715347 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:26.796660 [ 1197 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:26.801315 [ 1197 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:26.852990 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_236_47 to 202410_241_241_0 into Compact with storage Full 2024.10.22 00:50:26.878934 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_390_390_0 (state Temporary) 2024.10.22 00:50:26.924355 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_390_390_0 to 202410_403_403_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:26.977263 [ 1695 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rfq/fglcubdxqgsfqomsavhnupwneemnh, size 1863307 2024.10.22 00:50:26.992097 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:26.996912 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22181 2024.10.22 00:50:27.000653 [ 1203 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_398_87 to 202410_399_399_0 2024.10.22 00:50:27.010408 [ 1203 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:27.014631 [ 1203 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:27.015086 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_236_47, total 3239 rows starting from the beginning of the part 2024.10.22 00:50:27.036303 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_399_88} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_398_87 to 202410_399_399_0 into Compact with storage Full 2024.10.22 00:50:27.080130 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_399_88} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:27.080977 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_237_237_0, total 22 rows starting from the beginning of the part 2024.10.22 00:50:27.096594 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1686 entries to flush up to offset 376484 2024.10.22 00:50:27.105585 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_399_88} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_398_87, total 22785 rows starting from the beginning of the part 2024.10.22 00:50:27.164592 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_238_238_0, total 12 rows starting from the beginning of the part 2024.10.22 00:50:27.174180 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_399_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_399_399_0, total 52 rows starting from the beginning of the part 2024.10.22 00:50:27.177379 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 00:50:27.182993 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 00:50:27.188356 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 00:50:27.231009 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_239_239_0, total 13 rows starting from the beginning of the part 2024.10.22 00:50:27.267060 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 00:50:27.275722 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 00:50:27.286137 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 00:50:27.298391 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 00:50:27.408767 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_240_240_0, total 19 rows starting from the beginning of the part 2024.10.22 00:50:27.471345 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mbo/teorykxdgumrjrrtwrramjgmmovic (LogSeriesLimiter: on interval from 2024-10-22 00:50:11 to 2024-10-22 00:50:27 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:50:27.482728 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_241_241_0, total 11 rows starting from the beginning of the part 2024.10.22 00:50:27.634934 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 11.08 MiB, will set to 1.69 GiB (RSS), difference: -4.56 MiB 2024.10.22 00:50:28.631516 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_399_88} MergeTask::MergeProjectionsStage: Merge sorted 22837 rows, containing 13 columns (13 merged, 0 gathered) in 1.606352645 sec., 14216.679053060605 rows/sec., 2.13 MiB/sec. 2024.10.22 00:50:28.770094 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_399_88} MergedBlockOutputStream: filled checksums 202410_1_399_88 (state Temporary) 2024.10.22 00:50:28.932366 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_241_48/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:50:28.986260 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} MergeTask::MergeProjectionsStage: Merge sorted 3316 rows, containing 12 columns (12 merged, 0 gathered) in 2.15212303 sec., 1540.8041054232854 rows/sec., 240.36 KiB/sec. 2024.10.22 00:50:29.121531 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} MergedBlockOutputStream: filled checksums 202410_1_241_48 (state Temporary) 2024.10.22 00:50:29.151121 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:29.159723 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:29.213472 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:50:29.365965 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_218_218_0 (state Temporary) 2024.10.22 00:50:29.429100 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_241_48 to 202410_1_241_48 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:29.430600 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_399_88} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_399_88 to 202410_1_399_88 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:29.433586 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_241_48} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_236_47, 202410_241_241_0] -> 202410_1_241_48 2024.10.22 00:50:29.434764 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_399_88} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_398_87, 202410_399_399_0] -> 202410_1_399_88 2024.10.22 00:50:29.467898 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.08 MiB. 2024.10.22 00:50:29.475556 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:50:29.584244 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_218_218_0 to 202410_225_225_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:29.592524 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 7.67 MiB, will set to 1.70 GiB (RSS), difference: 3.54 MiB 2024.10.22 00:50:29.622820 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 376484 2024.10.22 00:50:29.723909 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:50:29.727965 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:50:31.020027 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 22241 2024.10.22 00:50:31.277664 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:31.289652 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:31.362375 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_391_391_0 (state Temporary) 2024.10.22 00:50:31.548568 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 202 2024.10.22 00:50:31.616244 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_391_391_0 to 202410_404_404_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:31.632039 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 89 entries to flush up to offset 14862 2024.10.22 00:50:31.676159 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_399_88 to 202410_404_404_0 2024.10.22 00:50:31.680339 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22241 2024.10.22 00:50:31.694718 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:31.699221 [ 1244 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.45 GiB. 2024.10.22 00:50:31.717689 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_399_88 to 202410_404_404_0 into Compact with storage Full 2024.10.22 00:50:31.882108 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:31.891492 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_399_88, total 22837 rows starting from the beginning of the part 2024.10.22 00:50:31.915228 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:31.918664 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.44 GiB. 2024.10.22 00:50:31.931123 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_400_400_0, total 48 rows starting from the beginning of the part 2024.10.22 00:50:31.951538 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 7.63 MiB, will set to 1.71 GiB (RSS), difference: 12.75 MiB 2024.10.22 00:50:31.970934 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_401_401_0, total 35 rows starting from the beginning of the part 2024.10.22 00:50:31.982717 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_227_227_0 (state Temporary) 2024.10.22 00:50:31.994870 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_402_402_0, total 35 rows starting from the beginning of the part 2024.10.22 00:50:32.039373 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_403_403_0, total 57 rows starting from the beginning of the part 2024.10.22 00:50:32.187569 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_404_404_0, total 60 rows starting from the beginning of the part 2024.10.22 00:50:32.200473 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:32.207003 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.44 GiB. 2024.10.22 00:50:32.234713 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_227_227_0 to 202410_233_233_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:32.274393 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_228_53 to 202410_229_229_0 2024.10.22 00:50:32.278559 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:32.279729 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.44 GiB. 2024.10.22 00:50:32.283530 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14862 2024.10.22 00:50:32.301380 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_229_54} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_228_53 to 202410_229_229_0 into Compact with storage Full 2024.10.22 00:50:32.372830 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_184_184_0 (state Temporary) 2024.10.22 00:50:32.482089 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_229_54} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:32.509114 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_229_54} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_228_53, total 14570 rows starting from the beginning of the part 2024.10.22 00:50:32.618008 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:32.620400 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.44 GiB. 2024.10.22 00:50:32.643246 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_229_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_229_229_0, total 95 rows starting from the beginning of the part 2024.10.22 00:50:32.710883 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_184_184_0 to 202410_194_194_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:32.748075 [ 1184 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_189_39 to 202410_194_194_0 2024.10.22 00:50:32.773681 [ 1184 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:32.777206 [ 1184 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.44 GiB. 2024.10.22 00:50:32.819308 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_189_39 to 202410_194_194_0 into Compact with storage Full 2024.10.22 00:50:32.883343 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 202 2024.10.22 00:50:32.914582 [ 1236 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 214 ms. 2024.10.22 00:50:33.022935 [ 1234 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 00:50:33.058779 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:33.079148 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_189_39, total 233 rows starting from the beginning of the part 2024.10.22 00:50:33.165203 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_190_190_0, total 1 rows starting from the beginning of the part 2024.10.22 00:50:33.393511 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_191_191_0, total 1 rows starting from the beginning of the part 2024.10.22 00:50:33.496796 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 6.01 MiB, will set to 1.72 GiB (RSS), difference: -4.61 MiB 2024.10.22 00:50:33.582220 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_192_192_0, total 1 rows starting from the beginning of the part 2024.10.22 00:50:33.626836 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_229_54} MergeTask::MergeProjectionsStage: Merge sorted 14665 rows, containing 13 columns (13 merged, 0 gathered) in 1.341546146 sec., 10931.416741590043 rows/sec., 4.59 MiB/sec. 2024.10.22 00:50:33.649700 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_193_193_0, total 1 rows starting from the beginning of the part 2024.10.22 00:50:33.697158 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_194_194_0, total 1 rows starting from the beginning of the part 2024.10.22 00:50:33.915514 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} MergeTask::MergeProjectionsStage: Merge sorted 23072 rows, containing 13 columns (13 merged, 0 gathered) in 2.206581745 sec., 10455.991513697581 rows/sec., 1.57 MiB/sec. 2024.10.22 00:50:33.930987 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_229_54} MergedBlockOutputStream: filled checksums 202410_1_229_54 (state Temporary) 2024.10.22 00:50:33.988103 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_229_54} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_229_54 to 202410_1_229_54 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:33.991792 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_229_54} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_228_53, 202410_229_229_0] -> 202410_1_229_54 2024.10.22 00:50:34.042448 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.64 MiB. 2024.10.22 00:50:34.187119 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 3229 2024.10.22 00:50:34.215181 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} MergedBlockOutputStream: filled checksums 202410_1_404_89 (state Temporary) 2024.10.22 00:50:34.332364 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:34.338586 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.44 GiB. 2024.10.22 00:50:34.383323 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_229_229_0 (state Temporary) 2024.10.22 00:50:34.457698 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_229_229_0 to 202410_242_242_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:34.496147 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3229 2024.10.22 00:50:34.571118 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_404_89 to 202410_1_404_89 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:34.574193 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_404_89} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_399_88, 202410_404_404_0] -> 202410_1_404_89 2024.10.22 00:50:34.716981 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.78 MiB. 2024.10.22 00:50:35.346533 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} MergeTask::MergeProjectionsStage: Merge sorted 238 rows, containing 32 columns (32 merged, 0 gathered) in 2.547988467 sec., 93.40701619431623 rows/sec., 9.87 KiB/sec. 2024.10.22 00:50:35.392369 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} MergedBlockOutputStream: filled checksums 202410_1_194_40 (state Temporary) 2024.10.22 00:50:35.420157 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 6.14 MiB, will set to 1.71 GiB (RSS), difference: -3.83 MiB 2024.10.22 00:50:35.668889 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_46_46_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:50:35.687731 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 22314 2024.10.22 00:50:35.712504 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_194_40 to 202410_1_194_40 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:35.720182 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_194_40} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_189_39, 202410_194_194_0] -> 202410_1_194_40 2024.10.22 00:50:35.794925 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:50:36.234594 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_46_46_0 (state Temporary) 2024.10.22 00:50:36.343683 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:36.361095 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.44 GiB. 2024.10.22 00:50:36.523799 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_392_392_0 (state Temporary) 2024.10.22 00:50:36.632584 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1922 entries to flush up to offset 378406 2024.10.22 00:50:36.682176 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_392_392_0 to 202410_405_405_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:36.852240 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22314 2024.10.22 00:50:37.706058 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:37.713380 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.44 GiB. 2024.10.22 00:50:37.969131 [ 1264 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 340 ms. 2024.10.22 00:50:38.046935 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_219_219_0 (state Temporary) 2024.10.22 00:50:38.163444 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_219_219_0 to 202410_226_226_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:38.189010 [ 1198 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_221_44 to 202410_226_226_0 2024.10.22 00:50:38.192184 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 378406 2024.10.22 00:50:38.211141 [ 1198 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:38.215913 [ 1198 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.44 GiB. 2024.10.22 00:50:38.257314 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_221_44 to 202410_226_226_0 into Compact with storage Full 2024.10.22 00:50:38.374843 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:38.389126 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} MergeTreeSequentialSource: Reading 48 marks from part 202410_1_221_44, total 383209 rows starting from the beginning of the part 2024.10.22 00:50:38.517569 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_222_222_0, total 1873 rows starting from the beginning of the part 2024.10.22 00:50:38.633485 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_223_223_0, total 1484 rows starting from the beginning of the part 2024.10.22 00:50:38.661353 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_224_224_0, total 1822 rows starting from the beginning of the part 2024.10.22 00:50:38.675493 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_225_225_0, total 1686 rows starting from the beginning of the part 2024.10.22 00:50:38.692755 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_226_226_0, total 1922 rows starting from the beginning of the part 2024.10.22 00:50:39.447251 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 4.49 MiB, will set to 1.71 GiB (RSS), difference: -2.65 MiB 2024.10.22 00:50:39.722731 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:50:39.729840 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:50:39.793376 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 99 entries to flush up to offset 14961 2024.10.22 00:50:39.923262 [ 1153 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:50:39.990135 [ 1153 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:50:40.204314 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:40.213445 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.43 GiB. 2024.10.22 00:50:40.272411 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_228_228_0 (state Temporary) 2024.10.22 00:50:40.386776 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 203 2024.10.22 00:50:40.633357 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_228_228_0 to 202410_234_234_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:40.682083 [ 1152 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_229_54 to 202410_230_230_0 2024.10.22 00:50:40.693314 [ 1152 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:40.696619 [ 1152 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.43 GiB. 2024.10.22 00:50:40.734410 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 14961 2024.10.22 00:50:40.831258 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_230_55} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_229_54 to 202410_230_230_0 into Compact with storage Full 2024.10.22 00:50:40.882504 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 22358 2024.10.22 00:50:40.904876 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_230_55} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:40.920936 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_230_55} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_229_54, total 14665 rows starting from the beginning of the part 2024.10.22 00:50:40.980212 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_230_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_230_230_0, total 83 rows starting from the beginning of the part 2024.10.22 00:50:41.083138 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:41.091075 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.42 GiB. 2024.10.22 00:50:41.265553 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:41.271046 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.42 GiB. 2024.10.22 00:50:41.280159 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_185_185_0 (state Temporary) 2024.10.22 00:50:41.416090 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_393_393_0 (state Temporary) 2024.10.22 00:50:41.481075 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_185_185_0 to 202410_195_195_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:41.572846 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_393_393_0 to 202410_406_406_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:41.579514 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 203 2024.10.22 00:50:41.682573 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.88 GiB, free memory in arenas 7.68 MiB, will set to 1.70 GiB (RSS), difference: 119.98 MiB 2024.10.22 00:50:41.777366 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22358 2024.10.22 00:50:42.001084 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 3249 2024.10.22 00:50:42.277228 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:42.285099 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.42 GiB. 2024.10.22 00:50:42.588021 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_230_230_0 (state Temporary) 2024.10.22 00:50:42.687049 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_230_55} MergeTask::MergeProjectionsStage: Merge sorted 14748 rows, containing 13 columns (13 merged, 0 gathered) in 1.973980083 sec., 7471.20000197084 rows/sec., 3.14 MiB/sec. 2024.10.22 00:50:42.743862 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_230_230_0 to 202410_243_243_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:42.801539 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3249 2024.10.22 00:50:43.054953 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_230_55} MergedBlockOutputStream: filled checksums 202410_1_230_55 (state Temporary) 2024.10.22 00:50:43.179688 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_230_55} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_230_55 to 202410_1_230_55 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:43.184526 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_230_55} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_229_54, 202410_230_230_0] -> 202410_1_230_55 2024.10.22 00:50:43.599597 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.64 MiB. 2024.10.22 00:50:43.719186 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.88 GiB, free memory in arenas 9.04 MiB, will set to 1.68 GiB (RSS), difference: 160.35 MiB 2024.10.22 00:50:43.786624 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_226_45/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:50:43.913512 [ 1725 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mvg/niiynvsppfukcsdhuoxgnkhrzdcdz, size 6502 2024.10.22 00:50:44.228013 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_16_16_0 (state Temporary) 2024.10.22 00:50:44.287705 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1_300_60, 202410_301_301_0, 202410_302_302_0, 202410_303_303_0] 2024.10.22 00:50:44.359307 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1_300_60, 202410_301_301_0, 202410_302_302_0, 202410_303_303_0] 2024.10.22 00:50:44.391296 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1_300_60, 202410_301_301_0, 202410_302_302_0, 202410_303_303_0] 2024.10.22 00:50:44.466176 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pnx/eooflzhvcxezkhttkegbnvrnyvwpf (LogSeriesLimiter: on interval from 2024-10-22 00:50:27 to 2024-10-22 00:50:44 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:50:44.468659 [ 1722 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ech/gnqlxuxuamziemjkojorbcvelgycp, size 717 2024.10.22 00:50:44.632808 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(d27d7e01cc034014964b0ad12436d011) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d27d7e01cc034014964b0ad12436d011, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 00:50:44.659619 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 958171, file segment info: File segment: [0, 33554431], key: d27d7e01cc034014964b0ad12436d011, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 00:50:44.667515 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:50:44.683322 [ 1708 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pnx/eooflzhvcxezkhttkegbnvrnyvwpf, size 4 2024.10.22 00:50:44.734572 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_142_29, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0, 202410_147_147_0] 2024.10.22 00:50:44.738347 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_142_29, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0, 202410_147_147_0] 2024.10.22 00:50:44.772697 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_142_29, 202410_143_143_0, 202410_144_144_0, 202410_145_145_0, 202410_146_146_0, 202410_147_147_0] 2024.10.22 00:50:45.184040 [ 1738 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mbo/teorykxdgumrjrrtwrramjgmmovic, size 958171 2024.10.22 00:50:45.264996 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 00:50:45.286694 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 00:50:45.295261 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 00:50:45.197827 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1538 entries to flush up to offset 379944 2024.10.22 00:50:45.314648 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 00:50:45.334347 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 00:50:45.365053 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 00:50:45.375030 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 00:50:45.621709 [ 1185 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 00:50:45.783055 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 22410 2024.10.22 00:50:45.881353 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.88 GiB, free memory in arenas 8.30 MiB, will set to 1.65 GiB (RSS), difference: 61.55 MiB 2024.10.22 00:50:45.978375 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_46_46_0 to 202410_49_49_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:46.060625 [ 1189 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_44_9 to 202410_49_49_0 2024.10.22 00:50:46.074030 [ 1189 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.14 MiB using storage policy from min volume index 0 2024.10.22 00:50:46.077588 [ 1189 ] {} DiskLocal: Reserved 1.14 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:46.089390 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_44_9 to 202410_49_49_0 into Compact with storage Full 2024.10.22 00:50:46.154839 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:46.170476 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:46.321300 [ 1170 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 263 ms. 2024.10.22 00:50:46.376005 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:46.378933 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:46.479623 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_220_220_0 (state Temporary) 2024.10.22 00:50:46.634211 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_394_394_0 (state Temporary) 2024.10.22 00:50:46.674961 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_220_220_0 to 202410_227_227_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:46.727463 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 379944 2024.10.22 00:50:46.788943 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_394_394_0 to 202410_407_407_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:46.849324 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22410 2024.10.22 00:50:47.535414 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 8.54 MiB, will set to 1.66 GiB (RSS), difference: -2.01 MiB 2024.10.22 00:50:48.265717 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1834 2024.10.22 00:50:48.273075 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 141 entries to flush up to offset 15102 2024.10.22 00:50:48.725763 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:48.758187 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:48.971417 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_229_229_0 (state Temporary) 2024.10.22 00:50:49.023049 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_229_229_0 to 202410_235_235_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:49.268878 [ 1213 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_230_55 to 202410_235_235_0 2024.10.22 00:50:49.276794 [ 1213 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:49.278616 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:49.454998 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_230_55 to 202410_235_235_0 into Compact with storage Full 2024.10.22 00:50:49.515756 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(cdd6d67d2ea9bee81a92e4b0a93e3247) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cdd6d67d2ea9bee81a92e4b0a93e3247, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 00:50:49.520634 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: cdd6d67d2ea9bee81a92e4b0a93e3247, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 00:50:49.525556 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15102 2024.10.22 00:50:49.523534 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:50:49.576138 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:49.587435 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_230_55, total 14748 rows starting from the beginning of the part 2024.10.22 00:50:49.721210 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:50:49.724179 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:50:49.829058 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_231_231_0, total 63 rows starting from the beginning of the part 2024.10.22 00:50:49.875826 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 17.07 MiB, will set to 1.66 GiB (RSS), difference: 8.27 MiB 2024.10.22 00:50:49.967748 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_232_232_0, total 66 rows starting from the beginning of the part 2024.10.22 00:50:50.024281 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_233_233_0, total 89 rows starting from the beginning of the part 2024.10.22 00:50:50.078114 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_234_234_0, total 99 rows starting from the beginning of the part 2024.10.22 00:50:50.110783 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_235_235_0, total 141 rows starting from the beginning of the part 2024.10.22 00:50:50.319189 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 3257 2024.10.22 00:50:50.461219 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:50.465084 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:50.525763 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_231_231_0 (state Temporary) 2024.10.22 00:50:50.819978 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_231_231_0 to 202410_244_244_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:50.857587 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 22458 2024.10.22 00:50:50.887347 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3257 2024.10.22 00:50:50.939382 [ 1191 ] {} DatabaseCatalog: Execution took 457 ms. 2024.10.22 00:50:51.073208 [ 1215 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 00:50:51.188329 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:51.191874 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:51.307674 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:51.324368 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_395_395_0 (state Temporary) 2024.10.22 00:50:51.376393 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_395_395_0 to 202410_408_408_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:51.524074 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22458 2024.10.22 00:50:51.532494 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 18.02 MiB, will set to 1.67 GiB (RSS), difference: -7.63 MiB 2024.10.22 00:50:51.995472 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_44_9, total 1687 rows starting from the beginning of the part 2024.10.22 00:50:52.353836 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:50:52.609820 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} MergeTask::MergeProjectionsStage: Merge sorted 15206 rows, containing 13 columns (13 merged, 0 gathered) in 3.18915495 sec., 4768.034240543879 rows/sec., 2.00 MiB/sec. 2024.10.22 00:50:52.715289 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} MergedBlockOutputStream: filled checksums 202410_1_235_56 (state Temporary) 2024.10.22 00:50:53.226926 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_235_56 to 202410_1_235_56 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:53.331915 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_235_56} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_230_55, 202410_235_235_0] -> 202410_1_235_56 2024.10.22 00:50:53.380470 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.97 MiB. 2024.10.22 00:50:53.758932 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_45_45_0, total 40 rows starting from the beginning of the part 2024.10.22 00:50:53.834773 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 381480 2024.10.22 00:50:53.865277 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 15.77 MiB, will set to 1.66 GiB (RSS), difference: 11.63 MiB 2024.10.22 00:50:54.674434 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} MergeTask::MergeProjectionsStage: Merge sorted 391996 rows, containing 4 columns (4 merged, 0 gathered) in 16.427915585 sec., 23861.578662963406 rows/sec., 350.06 KiB/sec. 2024.10.22 00:50:54.763556 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:54.778508 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:54.828707 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} MergedBlockOutputStream: filled checksums 202410_1_226_45 (state Temporary) 2024.10.22 00:50:54.862518 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_226_45 to 202410_1_226_45 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:54.868715 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_226_45} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_221_44, 202410_226_226_0] -> 202410_1_226_45 2024.10.22 00:50:54.897825 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:50:54.997405 [ 1151 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:50:55.040373 [ 1151 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:50:55.121174 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_221_221_0 (state Temporary) 2024.10.22 00:50:55.279204 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_221_221_0 to 202410_228_228_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:55.280875 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_46_46_0, total 42 rows starting from the beginning of the part 2024.10.22 00:50:55.422177 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 381480 2024.10.22 00:50:55.551256 [ 1252 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 00:50:55.584194 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 22492 2024.10.22 00:50:55.616650 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 17.39 MiB, will set to 1.66 GiB (RSS), difference: -1.85 MiB 2024.10.22 00:50:55.725691 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:55.730180 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:55.784355 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 1874 2024.10.22 00:50:55.977354 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_396_396_0 (state Temporary) 2024.10.22 00:50:56.121084 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_396_396_0 to 202410_409_409_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:56.167252 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1_404_89 to 202410_407_407_0 2024.10.22 00:50:56.179258 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22492 2024.10.22 00:50:56.180924 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:56.183506 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:56.314627 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_407_90} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_404_89 to 202410_407_407_0 into Compact with storage Full 2024.10.22 00:50:56.363610 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_407_90} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:50:56.370783 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_407_90} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_404_89, total 23072 rows starting from the beginning of the part 2024.10.22 00:50:56.384653 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_407_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_405_405_0, total 73 rows starting from the beginning of the part 2024.10.22 00:50:56.447826 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_407_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_406_406_0, total 44 rows starting from the beginning of the part 2024.10.22 00:50:56.583369 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 204 2024.10.22 00:50:56.604931 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_407_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_407_407_0, total 52 rows starting from the beginning of the part 2024.10.22 00:50:57.072567 [ 1267 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 00:50:57.116007 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:57.124898 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:57.156107 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 87 entries to flush up to offset 15189 2024.10.22 00:50:57.369662 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_186_186_0 (state Temporary) 2024.10.22 00:50:57.560202 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:57.563173 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:57.596060 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_186_186_0 to 202410_196_196_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:57.599645 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_47_47_0, total 44 rows starting from the beginning of the part 2024.10.22 00:50:57.605319 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 13.97 MiB, will set to 1.66 GiB (RSS), difference: -13.20 MiB 2024.10.22 00:50:57.613455 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_230_230_0 (state Temporary) 2024.10.22 00:50:57.673402 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_230_230_0 to 202410_236_236_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:57.695406 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 204 2024.10.22 00:50:57.724664 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15189 2024.10.22 00:50:57.846330 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_303_61, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0, 202410_307_307_0, 202410_308_308_0] 2024.10.22 00:50:57.853029 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_303_61, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0, 202410_307_307_0, 202410_308_308_0] 2024.10.22 00:50:58.006002 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_303_61, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0, 202410_307_307_0, 202410_308_308_0] 2024.10.22 00:50:58.389643 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 3277 2024.10.22 00:50:58.566676 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:50:58.570558 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:50:58.604839 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_232_232_0 (state Temporary) 2024.10.22 00:50:58.777277 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_232_232_0 to 202410_245_245_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:58.817311 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_172_35, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0, 202410_177_177_0] 2024.10.22 00:50:58.820913 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_172_35, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0, 202410_177_177_0] 2024.10.22 00:50:58.925441 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3277 2024.10.22 00:50:58.954979 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_172_35, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0, 202410_177_177_0] 2024.10.22 00:50:59.012730 [ 1159 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 00:50:59.172165 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_407_90} MergeTask::MergeProjectionsStage: Merge sorted 23241 rows, containing 13 columns (13 merged, 0 gathered) in 2.86440982 sec., 8113.713281432612 rows/sec., 1.22 MiB/sec. 2024.10.22 00:50:59.532425 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_48_48_0, total 45 rows starting from the beginning of the part 2024.10.22 00:50:59.571983 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_407_90} MergedBlockOutputStream: filled checksums 202410_1_407_90 (state Temporary) 2024.10.22 00:50:59.678569 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 10.14 MiB, will set to 1.66 GiB (RSS), difference: 7.06 MiB 2024.10.22 00:50:59.721939 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:50:59.725631 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:50:59.886833 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_407_90} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_407_90 to 202410_1_407_90 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:50:59.899158 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_407_90} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1_404_89, 202410_407_407_0] -> 202410_1_407_90 2024.10.22 00:50:59.967572 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.68 MiB. 2024.10.22 00:51:00.185142 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 22547 2024.10.22 00:51:00.824944 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:00.832942 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:51:01.048795 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_397_397_0 (state Temporary) 2024.10.22 00:51:01.106972 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_397_397_0 to 202410_410_410_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:01.171943 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22547 2024.10.22 00:51:01.258789 [ 1248 ] {} DatabaseCatalog: Execution took 310 ms. 2024.10.22 00:51:01.432549 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_49_49_0, total 38 rows starting from the beginning of the part 2024.10.22 00:51:01.539150 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 6.90 MiB, will set to 1.66 GiB (RSS), difference: -4.04 MiB 2024.10.22 00:51:02.166255 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_181_36, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0] 2024.10.22 00:51:02.168726 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_181_36, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0] 2024.10.22 00:51:02.194071 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_181_36, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0] 2024.10.22 00:51:02.442601 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_166_33, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0] 2024.10.22 00:51:02.557723 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1683 entries to flush up to offset 383163 2024.10.22 00:51:02.473794 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_166_33, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0] 2024.10.22 00:51:02.618584 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_166_33, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0, 202410_171_171_0] 2024.10.22 00:51:03.097172 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/sxm/jdipbkbkvtrofeizdwmkutustatxm. 2024.10.22 00:51:03.140069 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:03.147253 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:51:03.374632 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_222_222_0 (state Temporary) 2024.10.22 00:51:03.543331 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_222_222_0 to 202410_229_229_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:03.584451 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 383163 2024.10.22 00:51:05.187534 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 22 entries to flush up to offset 22569 2024.10.22 00:51:05.197538 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 205 2024.10.22 00:51:05.343503 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 69 entries to flush up to offset 15258 2024.10.22 00:51:05.509043 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:05.511557 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:51:05.565897 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_398_398_0 (state Temporary) 2024.10.22 00:51:05.584906 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:05.590314 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:51:05.659303 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_231_231_0 (state Temporary) 2024.10.22 00:51:05.677625 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:05.680585 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.36 GiB. 2024.10.22 00:51:05.680634 [ 1194 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 283 ms. 2024.10.22 00:51:05.692669 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_231_231_0 to 202410_237_237_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:05.777488 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_187_187_0 (state Temporary) 2024.10.22 00:51:05.828294 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 00:51:05.853804 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_187_187_0 to 202410_197_197_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:05.861587 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_398_398_0 to 202410_411_411_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:05.875720 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 00:51:05.876529 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 5.55 MiB, will set to 1.66 GiB (RSS), difference: 19.65 MiB 2024.10.22 00:51:05.888037 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 00:51:05.898193 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22569 2024.10.22 00:51:05.918510 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15258 2024.10.22 00:51:06.002202 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 00:51:06.021063 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 00:51:06.026996 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 205 2024.10.22 00:51:06.060353 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 00:51:06.262482 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 00:51:06.284003 [ 1180 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 00:51:06.431784 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 3288 2024.10.22 00:51:06.451899 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qcg/gbkhkcchmtswqcnxawgzxzhavbsdv (LogSeriesLimiter: on interval from 2024-10-22 00:50:44 to 2024-10-22 00:51:06 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:51:06.572828 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:06.575679 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:06.828711 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_233_233_0 (state Temporary) 2024.10.22 00:51:06.877797 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_233_233_0 to 202410_246_246_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:07.000923 [ 1197 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_241_48 to 202410_246_246_0 2024.10.22 00:51:07.022458 [ 1197 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:07.034679 [ 1197 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:07.064450 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_241_48 to 202410_246_246_0 into Compact with storage Full 2024.10.22 00:51:07.280189 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:51:07.310495 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3288 2024.10.22 00:51:07.395115 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_241_48, total 3316 rows starting from the beginning of the part 2024.10.22 00:51:07.674260 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_242_242_0, total 19 rows starting from the beginning of the part 2024.10.22 00:51:07.690719 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 4.36 MiB, will set to 1.64 GiB (RSS), difference: -19.75 MiB 2024.10.22 00:51:07.968552 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_243_243_0, total 20 rows starting from the beginning of the part 2024.10.22 00:51:08.013152 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_244_244_0, total 8 rows starting from the beginning of the part 2024.10.22 00:51:08.066799 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_245_245_0, total 20 rows starting from the beginning of the part 2024.10.22 00:51:08.120419 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_246_246_0, total 11 rows starting from the beginning of the part 2024.10.22 00:51:09.690891 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 3.02 MiB, will set to 1.64 GiB (RSS), difference: -6.25 MiB 2024.10.22 00:51:09.724760 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:51:09.727815 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:51:09.886425 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} MergeTask::MergeProjectionsStage: Merge sorted 3394 rows, containing 12 columns (12 merged, 0 gathered) in 2.833053464 sec., 1198.0006883484639 rows/sec., 186.98 KiB/sec. 2024.10.22 00:51:09.908292 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 22620 2024.10.22 00:51:09.936914 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} MergedBlockOutputStream: filled checksums 202410_1_246_49 (state Temporary) 2024.10.22 00:51:10.060779 [ 1209 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:51:10.093303 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_49_10/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:51:10.097273 [ 1209 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:51:10.200520 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:10.207710 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:10.375496 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_399_399_0 (state Temporary) 2024.10.22 00:51:10.388865 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_246_49 to 202410_1_246_49 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:10.396219 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_246_49} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_241_48, 202410_246_246_0] -> 202410_1_246_49 2024.10.22 00:51:10.475002 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_399_399_0 to 202410_412_412_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:10.488199 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:51:10.523242 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_407_90 to 202410_408_408_0 2024.10.22 00:51:10.531522 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:10.572328 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22620 2024.10.22 00:51:10.589371 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1773 entries to flush up to offset 384936 2024.10.22 00:51:10.678807 [ 1235 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:10.736676 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_408_91} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_407_90 to 202410_408_408_0 into Compact with storage Full 2024.10.22 00:51:10.782333 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_408_91} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:51:10.791690 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_408_91} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_407_90, total 23241 rows starting from the beginning of the part 2024.10.22 00:51:10.806119 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_408_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_408_408_0, total 48 rows starting from the beginning of the part 2024.10.22 00:51:11.429450 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 1.88 MiB, will set to 1.64 GiB (RSS), difference: -5.52 MiB 2024.10.22 00:51:11.579202 [ 1239 ] {} DatabaseCatalog: Execution took 222 ms. 2024.10.22 00:51:11.720634 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:11.728149 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:12.156679 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_insert_202410_223_223_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:51:12.208467 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_223_223_0 (state Temporary) 2024.10.22 00:51:12.316516 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_223_223_0 to 202410_230_230_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:12.571651 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 384936 2024.10.22 00:51:12.576062 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_408_91} MergeTask::MergeProjectionsStage: Merge sorted 23289 rows, containing 13 columns (13 merged, 0 gathered) in 1.845844015 sec., 12616.992449386358 rows/sec., 1.89 MiB/sec. 2024.10.22 00:51:12.657304 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_408_91} MergedBlockOutputStream: filled checksums 202410_1_408_91 (state Temporary) 2024.10.22 00:51:12.761158 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_408_91} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_408_91 to 202410_1_408_91 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:12.765494 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_408_91} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_407_90, 202410_408_408_0] -> 202410_1_408_91 2024.10.22 00:51:12.793481 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.07 MiB. 2024.10.22 00:51:12.933795 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:12.941599 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:13.420574 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 1.99 MiB, will set to 1.65 GiB (RSS), difference: 9.81 MiB 2024.10.22 00:51:13.430378 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 88 entries to flush up to offset 15346 2024.10.22 00:51:13.635744 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 206 2024.10.22 00:51:13.759626 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:13.763529 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:13.796622 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_232_232_0 (state Temporary) 2024.10.22 00:51:13.971548 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_232_232_0 to 202410_238_238_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:13.999870 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15346 2024.10.22 00:51:14.484078 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:14.490496 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:14.579030 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 22666 2024.10.22 00:51:14.787910 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:14.789668 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:14.814691 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 3302 2024.10.22 00:51:14.867112 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_400_400_0 (state Temporary) 2024.10.22 00:51:14.891497 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_188_188_0 (state Temporary) 2024.10.22 00:51:14.921119 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_400_400_0 to 202410_413_413_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:14.985484 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_408_91 to 202410_409_409_0 2024.10.22 00:51:15.043002 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:15.072171 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22666 2024.10.22 00:51:15.085789 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_188_188_0 to 202410_198_198_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:15.095789 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:15.103045 [ 1234 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:15.131261 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_409_92} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_408_91 to 202410_409_409_0 into Compact with storage Full 2024.10.22 00:51:15.169090 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_409_92} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:51:15.176590 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_409_92} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_408_91, total 23289 rows starting from the beginning of the part 2024.10.22 00:51:15.193237 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_409_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_409_409_0, total 34 rows starting from the beginning of the part 2024.10.22 00:51:15.195325 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:15.258278 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_234_234_0 (state Temporary) 2024.10.22 00:51:15.268643 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 206 2024.10.22 00:51:15.302754 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_234_234_0 to 202410_247_247_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:15.474077 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3302 2024.10.22 00:51:15.573189 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 2.21 MiB, will set to 1.65 GiB (RSS), difference: -12.14 MiB 2024.10.22 00:51:16.328818 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_47_47_0 (state Temporary) 2024.10.22 00:51:16.497037 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_409_92} MergeTask::MergeProjectionsStage: Merge sorted 23323 rows, containing 13 columns (13 merged, 0 gathered) in 1.373857032 sec., 16976.29335277151 rows/sec., 2.54 MiB/sec. 2024.10.22 00:51:16.773755 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_409_92} MergedBlockOutputStream: filled checksums 202410_1_409_92 (state Temporary) 2024.10.22 00:51:17.048286 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_409_92} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_409_92 to 202410_1_409_92 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:17.061002 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_409_92} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_408_91, 202410_409_409_0] -> 202410_1_409_92 2024.10.22 00:51:17.303782 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.07 MiB. 2024.10.22 00:51:17.651501 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 3.49 MiB, will set to 1.65 GiB (RSS), difference: 1.71 MiB 2024.10.22 00:51:18.333369 [ 1268 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 221 ms. 2024.10.22 00:51:19.075002 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 22697 2024.10.22 00:51:19.384110 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:19.388158 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:19.448883 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_401_401_0 (state Temporary) 2024.10.22 00:51:19.496539 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_401_401_0 to 202410_414_414_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:19.575708 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 386472 2024.10.22 00:51:19.632511 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_409_92 to 202410_410_410_0 2024.10.22 00:51:19.645008 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:19.657049 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22697 2024.10.22 00:51:19.679595 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 4.06 MiB, will set to 1.66 GiB (RSS), difference: 4.64 MiB 2024.10.22 00:51:19.735739 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:51:19.753230 [ 1198 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:19.830964 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:51:19.873308 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_410_93} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_409_92 to 202410_410_410_0 into Compact with storage Full 2024.10.22 00:51:19.968176 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_410_93} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:51:19.983911 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_410_93} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_409_92, total 23323 rows starting from the beginning of the part 2024.10.22 00:51:20.196563 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_410_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_410_410_0, total 55 rows starting from the beginning of the part 2024.10.22 00:51:20.668346 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:20.671376 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:21.069284 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_224_224_0 (state Temporary) 2024.10.22 00:51:21.193671 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_224_224_0 to 202410_231_231_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:21.226606 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_226_45 to 202410_231_231_0 2024.10.22 00:51:21.233557 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 386472 2024.10.22 00:51:21.236680 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:21.241706 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:21.281495 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_226_45 to 202410_231_231_0 into Compact with storage Full 2024.10.22 00:51:21.422433 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:51:21.448550 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} MergeTreeSequentialSource: Reading 49 marks from part 202410_1_226_45, total 391996 rows starting from the beginning of the part 2024.10.22 00:51:21.491810 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_227_227_0, total 1538 rows starting from the beginning of the part 2024.10.22 00:51:21.531613 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 94 entries to flush up to offset 15440 2024.10.22 00:51:21.549327 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_228_228_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:51:21.623393 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 4.48 MiB, will set to 1.66 GiB (RSS), difference: 43.49 MiB 2024.10.22 00:51:21.687472 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_229_229_0, total 1683 rows starting from the beginning of the part 2024.10.22 00:51:21.809700 [ 1173 ] {} DatabaseCatalog: Execution took 229 ms. 2024.10.22 00:51:21.836177 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_230_230_0, total 1773 rows starting from the beginning of the part 2024.10.22 00:51:21.880166 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:21.885069 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:21.891099 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_231_231_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:51:21.972006 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(e52189212a55d7efadf77f2c6b2d8024) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e52189212a55d7efadf77f2c6b2d8024, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 00:51:21.974460 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 540553, file segment info: File segment: [0, 33554431], key: e52189212a55d7efadf77f2c6b2d8024, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 00:51:21.977079 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:51:22.067646 [ 1727 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qcg/gbkhkcchmtswqcnxawgzxzhavbsdv, size 540553 2024.10.22 00:51:22.096037 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 00:51:22.227639 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 00:51:22.274109 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_233_233_0 (state Temporary) 2024.10.22 00:51:22.340805 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 00:51:22.363708 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 00:51:22.391074 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_410_93} MergeTask::MergeProjectionsStage: Merge sorted 23378 rows, containing 13 columns (13 merged, 0 gathered) in 2.524325497 sec., 9261.087774846494 rows/sec., 1.39 MiB/sec. 2024.10.22 00:51:22.398146 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 00:51:22.427873 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 00:51:22.458748 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 00:51:22.469768 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_233_233_0 to 202410_239_239_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:22.496913 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hfe/yxmrdaljbjhvgofbltweptehwbifu (LogSeriesLimiter: on interval from 2024-10-22 00:51:06 to 2024-10-22 00:51:22 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:51:22.523732 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15440 2024.10.22 00:51:22.596933 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_410_93} MergedBlockOutputStream: filled checksums 202410_1_410_93 (state Temporary) 2024.10.22 00:51:22.691126 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_308_62, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0, 202410_312_312_0, 202410_313_313_0] 2024.10.22 00:51:22.712154 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_308_62, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0, 202410_312_312_0, 202410_313_313_0] 2024.10.22 00:51:22.753022 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_308_62, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0, 202410_312_312_0, 202410_313_313_0] 2024.10.22 00:51:22.771448 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 207 2024.10.22 00:51:22.933834 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_49_10/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:51:22.980268 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 3311 2024.10.22 00:51:23.137112 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/sxm/jdipbkbkvtrofeizdwmkutustatxm. 2024.10.22 00:51:23.224506 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:23.231429 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:23.311164 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:23.316719 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:23.368277 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_235_235_0 (state Temporary) 2024.10.22 00:51:23.379207 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_410_93} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_410_93 to 202410_1_410_93 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:23.384441 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_410_93} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_409_92, 202410_410_410_0] -> 202410_1_410_93 2024.10.22 00:51:23.390069 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_189_189_0 (state Temporary) 2024.10.22 00:51:23.417729 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.07 MiB. 2024.10.22 00:51:23.453211 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_231_46/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:51:23.529560 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_235_235_0 to 202410_248_248_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:23.582842 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3311 2024.10.22 00:51:23.590024 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_189_189_0 to 202410_199_199_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:23.663089 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 22772 2024.10.22 00:51:23.745709 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_194_40 to 202410_199_199_0 2024.10.22 00:51:23.874209 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:23.876965 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:23.895743 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_194_40 to 202410_199_199_0 into Compact with storage Full 2024.10.22 00:51:23.925124 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 207 2024.10.22 00:51:24.094610 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:51:24.127706 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_194_40, total 238 rows starting from the beginning of the part 2024.10.22 00:51:24.184619 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:24.187513 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:24.328977 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_195_195_0, total 1 rows starting from the beginning of the part 2024.10.22 00:51:24.376392 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.88 GiB, free memory in arenas 9.02 MiB, will set to 1.65 GiB (RSS), difference: 101.82 MiB 2024.10.22 00:51:24.529467 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_196_196_0, total 1 rows starting from the beginning of the part 2024.10.22 00:51:24.588971 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_402_402_0 (state Temporary) 2024.10.22 00:51:24.856586 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_197_197_0, total 1 rows starting from the beginning of the part 2024.10.22 00:51:24.957368 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_198_198_0, total 1 rows starting from the beginning of the part 2024.10.22 00:51:25.106518 [ 1186 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:51:25.156743 [ 1186 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:51:25.156655 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_402_402_0 to 202410_415_415_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:25.270527 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_199_199_0, total 1 rows starting from the beginning of the part 2024.10.22 00:51:25.276369 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_410_93 to 202410_415_415_0 2024.10.22 00:51:25.281747 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:25.284910 [ 1176 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:25.304990 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_410_93 to 202410_415_415_0 into Compact with storage Full 2024.10.22 00:51:25.322586 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22772 2024.10.22 00:51:25.367389 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:51:25.388146 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_410_93, total 23378 rows starting from the beginning of the part 2024.10.22 00:51:25.418865 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_411_411_0, total 22 rows starting from the beginning of the part 2024.10.22 00:51:25.680529 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_412_412_0, total 51 rows starting from the beginning of the part 2024.10.22 00:51:25.813613 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_413_413_0, total 46 rows starting from the beginning of the part 2024.10.22 00:51:25.864044 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_414_414_0, total 31 rows starting from the beginning of the part 2024.10.22 00:51:25.870748 [ 1172 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 325 ms. 2024.10.22 00:51:25.994565 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:51:26.036054 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_415_415_0, total 75 rows starting from the beginning of the part 2024.10.22 00:51:26.070668 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2d0b4e6d7eddf5e6cfbda206331eb06f) : [0, 2303664]: Setting download as failed: File segment: [0, 2303664], key: 2d0b4e6d7eddf5e6cfbda206331eb06f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:51:26.071435 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2d0b4e6d7eddf5e6cfbda206331eb06f): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:51:26.114494 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2d0b4e6d7eddf5e6cfbda206331eb06f, offset: 0, size: 0 2024.10.22 00:51:26.119094 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2d0b4e6d7eddf5e6cfbda206331eb06f for removal 2024.10.22 00:51:26.122260 [ 1476 ] {} CacheMetadata: Key 2d0b4e6d7eddf5e6cfbda206331eb06f is removed from metadata 2024.10.22 00:51:26.645355 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_47_47_0 to 202410_50_50_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:26.771906 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 12.04 MiB, will set to 1.65 GiB (RSS), difference: 27.29 MiB 2024.10.22 00:51:27.380978 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a8a751e92a909e172ca478012a9b8e48) : [0, 1056991]: Setting download as failed: File segment: [0, 1056991], key: a8a751e92a909e172ca478012a9b8e48, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:51:27.383704 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a8a751e92a909e172ca478012a9b8e48): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:51:27.631496 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a8a751e92a909e172ca478012a9b8e48, offset: 0, size: 0 2024.10.22 00:51:27.633336 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a8a751e92a909e172ca478012a9b8e48 for removal 2024.10.22 00:51:27.642801 [ 1476 ] {} CacheMetadata: Key a8a751e92a909e172ca478012a9b8e48 is removed from metadata 2024.10.22 00:51:27.697023 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} MergeTask::MergeProjectionsStage: Merge sorted 243 rows, containing 32 columns (32 merged, 0 gathered) in 3.805844536 sec., 63.84916611843443 rows/sec., 6.73 KiB/sec. 2024.10.22 00:51:27.785637 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} MergedBlockOutputStream: filled checksums 202410_1_199_41 (state Temporary) 2024.10.22 00:51:28.248414 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 388008 2024.10.22 00:51:28.348200 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_199_41 to 202410_1_199_41 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:28.353490 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_199_41} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_194_40, 202410_199_199_0] -> 202410_1_199_41 2024.10.22 00:51:28.376949 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} MergeTask::MergeProjectionsStage: Merge sorted 23603 rows, containing 13 columns (13 merged, 0 gathered) in 3.087536645 sec., 7644.605623781997 rows/sec., 1.14 MiB/sec. 2024.10.22 00:51:28.443829 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:51:28.568606 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} MergedBlockOutputStream: filled checksums 202410_1_415_94 (state Temporary) 2024.10.22 00:51:28.674966 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 6.45 MiB, will set to 1.65 GiB (RSS), difference: 15.31 MiB 2024.10.22 00:51:28.779820 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 10 file segments (while reserving 34151 bytes for 4c674117d240f1c408c4a735384bae08:0) 2024.10.22 00:51:28.783643 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 87d863f435bc32126f8ab6f975d48ff2, offset: 0, size: 156 2024.10.22 00:51:28.786786 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 87d863f435bc32126f8ab6f975d48ff2 for removal 2024.10.22 00:51:28.788469 [ 1476 ] {} CacheMetadata: Key 87d863f435bc32126f8ab6f975d48ff2 is removed from metadata 2024.10.22 00:51:28.788927 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 87f837619ac0cc975c7b6e34a7505bf3, offset: 0, size: 160 2024.10.22 00:51:28.791990 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 87f837619ac0cc975c7b6e34a7505bf3 for removal 2024.10.22 00:51:28.796246 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4113a0886970dae209821d0b88dc8844, offset: 0, size: 194 2024.10.22 00:51:28.801135 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4113a0886970dae209821d0b88dc8844 for removal 2024.10.22 00:51:28.809296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8bb385825c4db872c8b0f214ad0e3ec0, offset: 0, size: 101540 2024.10.22 00:51:28.817238 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8bb385825c4db872c8b0f214ad0e3ec0 for removal 2024.10.22 00:51:28.822459 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f8e513d65ff8a1317f11be6932ed7d7e, offset: 0, size: 192 2024.10.22 00:51:28.832124 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f8e513d65ff8a1317f11be6932ed7d7e for removal 2024.10.22 00:51:28.841818 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f1e008e328b5d665e95d07fa59fb4ed7, offset: 0, size: 159 2024.10.22 00:51:28.847153 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_415_94 to 202410_1_415_94 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:28.855236 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f1e008e328b5d665e95d07fa59fb4ed7 for removal 2024.10.22 00:51:28.866603 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_415_94} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_410_93, 202410_415_415_0] -> 202410_1_415_94 2024.10.22 00:51:28.866984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e7767b1a978f7e8ec2b76a383781160b, offset: 0, size: 156 2024.10.22 00:51:28.873682 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e7767b1a978f7e8ec2b76a383781160b for removal 2024.10.22 00:51:28.878689 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 70d6daed3ae23b8ad2ccec5f4ad0ba2e, offset: 0, size: 156 2024.10.22 00:51:28.880021 [ 1476 ] {} CacheMetadata: Key e7767b1a978f7e8ec2b76a383781160b is removed from metadata 2024.10.22 00:51:28.892944 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 70d6daed3ae23b8ad2ccec5f4ad0ba2e for removal 2024.10.22 00:51:28.896859 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ada807c602a203809f871bf926318810, offset: 0, size: 1300 2024.10.22 00:51:28.897416 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.90 MiB. 2024.10.22 00:51:28.899145 [ 1476 ] {} CacheMetadata: Key 70d6daed3ae23b8ad2ccec5f4ad0ba2e is removed from metadata 2024.10.22 00:51:28.904300 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ada807c602a203809f871bf926318810 for removal 2024.10.22 00:51:28.916309 [ 1476 ] {} CacheMetadata: Key ada807c602a203809f871bf926318810 is removed from metadata 2024.10.22 00:51:28.916487 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0781623fadcaf965e5df7062bfa0a463, offset: 0, size: 4 2024.10.22 00:51:28.922789 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0781623fadcaf965e5df7062bfa0a463 for removal 2024.10.22 00:51:28.931518 [ 1476 ] {} CacheMetadata: Key 0781623fadcaf965e5df7062bfa0a463 is removed from metadata 2024.10.22 00:51:28.964459 [ 1476 ] {} CacheMetadata: Key f1e008e328b5d665e95d07fa59fb4ed7 is removed from metadata 2024.10.22 00:51:28.980013 [ 1476 ] {} CacheMetadata: Key f8e513d65ff8a1317f11be6932ed7d7e is removed from metadata 2024.10.22 00:51:28.989507 [ 1476 ] {} CacheMetadata: Key 8bb385825c4db872c8b0f214ad0e3ec0 is removed from metadata 2024.10.22 00:51:29.000546 [ 1476 ] {} CacheMetadata: Key 4113a0886970dae209821d0b88dc8844 is removed from metadata 2024.10.22 00:51:29.019470 [ 1476 ] {} CacheMetadata: Key 87f837619ac0cc975c7b6e34a7505bf3 is removed from metadata 2024.10.22 00:51:29.334337 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:29.340838 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:29.440569 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 91 entries to flush up to offset 22863 2024.10.22 00:51:29.595600 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_225_225_0 (state Temporary) 2024.10.22 00:51:29.729697 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:51:29.735914 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:51:29.926764 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_225_225_0 to 202410_232_232_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:30.061841 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:30.064348 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 104 entries to flush up to offset 15544 2024.10.22 00:51:30.065656 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:30.072722 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ae29251816b605ec38de15ea8e2a2716) : [0, 314712]: Setting download as failed: File segment: [0, 314712], key: ae29251816b605ec38de15ea8e2a2716, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:51:30.075252 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ae29251816b605ec38de15ea8e2a2716): No space left in cache to reserve 314713 bytes, will continue without cache download 2024.10.22 00:51:30.086170 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 388008 2024.10.22 00:51:30.315959 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ae29251816b605ec38de15ea8e2a2716, offset: 0, size: 0 2024.10.22 00:51:30.319253 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ae29251816b605ec38de15ea8e2a2716 for removal 2024.10.22 00:51:30.323422 [ 1476 ] {} CacheMetadata: Key ae29251816b605ec38de15ea8e2a2716 is removed from metadata 2024.10.22 00:51:30.330952 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:30.333392 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:30.371228 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_403_403_0 (state Temporary) 2024.10.22 00:51:30.537252 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_403_403_0 to 202410_416_416_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:30.584102 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_234_234_0 (state Temporary) 2024.10.22 00:51:30.745132 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_234_234_0 to 202410_240_240_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:30.753613 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 10.63 MiB, will set to 1.65 GiB (RSS), difference: 3.46 MiB 2024.10.22 00:51:30.858108 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22863 2024.10.22 00:51:30.874098 [ 1181 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_235_56 to 202410_240_240_0 2024.10.22 00:51:30.933092 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1874 2024.10.22 00:51:30.981094 [ 1181 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:30.984171 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:30.995234 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15544 2024.10.22 00:51:31.084976 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 24 entries to flush up to offset 3335 2024.10.22 00:51:31.104853 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_235_56 to 202410_240_240_0 into Compact with storage Full 2024.10.22 00:51:31.149762 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:51:31.167749 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_235_56, total 15206 rows starting from the beginning of the part 2024.10.22 00:51:31.181392 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_236_236_0, total 87 rows starting from the beginning of the part 2024.10.22 00:51:31.207624 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_237_237_0, total 69 rows starting from the beginning of the part 2024.10.22 00:51:31.353495 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_238_238_0, total 88 rows starting from the beginning of the part 2024.10.22 00:51:31.430792 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 208 2024.10.22 00:51:31.451463 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2745e04daef119e0bf9fae9826b6fc25) : [0, 209182]: Setting download as failed: File segment: [0, 209182], key: 2745e04daef119e0bf9fae9826b6fc25, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:51:31.454480 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2745e04daef119e0bf9fae9826b6fc25): No space left in cache to reserve 209183 bytes, will continue without cache download 2024.10.22 00:51:31.482331 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2745e04daef119e0bf9fae9826b6fc25, offset: 0, size: 0 2024.10.22 00:51:31.482579 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_239_239_0, total 94 rows starting from the beginning of the part 2024.10.22 00:51:31.484220 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2745e04daef119e0bf9fae9826b6fc25 for removal 2024.10.22 00:51:31.487274 [ 1476 ] {} CacheMetadata: Key 2745e04daef119e0bf9fae9826b6fc25 is removed from metadata 2024.10.22 00:51:31.599504 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:31.600962 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_240_240_0, total 104 rows starting from the beginning of the part 2024.10.22 00:51:31.601747 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:31.673707 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:31.676841 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:51:31.725799 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_236_236_0 (state Temporary) 2024.10.22 00:51:31.880587 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_190_190_0 (state Temporary) 2024.10.22 00:51:32.019479 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_236_236_0 to 202410_249_249_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:32.071523 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_190_190_0 to 202410_200_200_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:32.180179 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3335 2024.10.22 00:51:32.292768 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 208 2024.10.22 00:51:32.498285 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a9168a05b0b263ac5280301e0f8ef2a7) : [0, 458219]: Setting download as failed: File segment: [0, 458219], key: a9168a05b0b263ac5280301e0f8ef2a7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:51:32.500669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a9168a05b0b263ac5280301e0f8ef2a7): No space left in cache to reserve 458220 bytes, will continue without cache download 2024.10.22 00:51:32.682315 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_240_57/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:51:32.743729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a9168a05b0b263ac5280301e0f8ef2a7, offset: 0, size: 0 2024.10.22 00:51:32.747516 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a9168a05b0b263ac5280301e0f8ef2a7 for removal 2024.10.22 00:51:32.766597 [ 1476 ] {} CacheMetadata: Key a9168a05b0b263ac5280301e0f8ef2a7 is removed from metadata 2024.10.22 00:51:33.562999 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 15.98 MiB, will set to 1.66 GiB (RSS), difference: 6.83 MiB 2024.10.22 00:51:33.676985 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} MergeTask::MergeProjectionsStage: Merge sorted 15648 rows, containing 13 columns (13 merged, 0 gathered) in 2.581288699 sec., 6062.08829181412 rows/sec., 2.55 MiB/sec. 2024.10.22 00:51:33.728475 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 77404 bytes for d701c49db6dd533f5dca8895d39c7473:0) 2024.10.22 00:51:33.733455 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 49e5e407d468077154301167ef24dcc8, offset: 0, size: 51112 2024.10.22 00:51:33.742796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 49e5e407d468077154301167ef24dcc8 for removal 2024.10.22 00:51:33.749601 [ 1476 ] {} CacheMetadata: Key 49e5e407d468077154301167ef24dcc8 is removed from metadata 2024.10.22 00:51:33.756014 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 994b8b3d8ddb081ed89f0519df6bd314, offset: 0, size: 156 2024.10.22 00:51:33.763184 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 994b8b3d8ddb081ed89f0519df6bd314 for removal 2024.10.22 00:51:33.767144 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 30db520fa190cc860f9e9f18c5021432, offset: 0, size: 212 2024.10.22 00:51:33.770817 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 30db520fa190cc860f9e9f18c5021432 for removal 2024.10.22 00:51:33.773328 [ 1476 ] {} CacheMetadata: Key 994b8b3d8ddb081ed89f0519df6bd314 is removed from metadata 2024.10.22 00:51:33.779485 [ 1476 ] {} CacheMetadata: Key 30db520fa190cc860f9e9f18c5021432 is removed from metadata 2024.10.22 00:51:34.091935 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} MergedBlockOutputStream: filled checksums 202410_1_240_57 (state Temporary) 2024.10.22 00:51:34.530140 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_240_57 to 202410_1_240_57 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:34.541076 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_240_57} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_235_56, 202410_240_240_0] -> 202410_1_240_57 2024.10.22 00:51:34.764202 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 12 file segments (while reserving 91267 bytes for 7ca41b8dba4d34790fa36e31f4e13a02:0) 2024.10.22 00:51:34.765789 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ac8ccc0beb4561f6dda6ea3c71f9fe7, offset: 0, size: 156 2024.10.22 00:51:34.769532 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ac8ccc0beb4561f6dda6ea3c71f9fe7 for removal 2024.10.22 00:51:34.772348 [ 1476 ] {} CacheMetadata: Key 7ac8ccc0beb4561f6dda6ea3c71f9fe7 is removed from metadata 2024.10.22 00:51:34.772672 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4c674117d240f1c408c4a735384bae08, offset: 0, size: 34151 2024.10.22 00:51:34.776045 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4c674117d240f1c408c4a735384bae08 for removal 2024.10.22 00:51:34.780003 [ 1476 ] {} CacheMetadata: Key 4c674117d240f1c408c4a735384bae08 is removed from metadata 2024.10.22 00:51:34.780808 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 08c81f42a44b991da9db4fe52a96521f, offset: 0, size: 183 2024.10.22 00:51:34.785534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 08c81f42a44b991da9db4fe52a96521f for removal 2024.10.22 00:51:34.788857 [ 1476 ] {} CacheMetadata: Key 08c81f42a44b991da9db4fe52a96521f is removed from metadata 2024.10.22 00:51:34.789377 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 41ce131300d642fd7ed9d460e4857e19, offset: 0, size: 169 2024.10.22 00:51:34.792187 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 41ce131300d642fd7ed9d460e4857e19 for removal 2024.10.22 00:51:34.795294 [ 1476 ] {} CacheMetadata: Key 41ce131300d642fd7ed9d460e4857e19 is removed from metadata 2024.10.22 00:51:34.795737 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 54228dd89998b7ac29d2dc87c3b4640b, offset: 0, size: 156 2024.10.22 00:51:34.800802 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 54228dd89998b7ac29d2dc87c3b4640b for removal 2024.10.22 00:51:34.820503 [ 1476 ] {} CacheMetadata: Key 54228dd89998b7ac29d2dc87c3b4640b is removed from metadata 2024.10.22 00:51:34.819922 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 89861401165290aa6061c2b03270eb38, offset: 0, size: 24865 2024.10.22 00:51:34.868497 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 90 entries to flush up to offset 22953 2024.10.22 00:51:34.889064 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.93 MiB. 2024.10.22 00:51:34.931646 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 89861401165290aa6061c2b03270eb38 for removal 2024.10.22 00:51:34.935296 [ 1476 ] {} CacheMetadata: Key 89861401165290aa6061c2b03270eb38 is removed from metadata 2024.10.22 00:51:34.936446 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e0c6aaffce7ab6860cf2a848a9cf146a, offset: 0, size: 2573 2024.10.22 00:51:34.953393 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e0c6aaffce7ab6860cf2a848a9cf146a for removal 2024.10.22 00:51:35.064114 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0ab30119504effb16a3db0790f20d5a3, offset: 0, size: 167 2024.10.22 00:51:35.068789 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0ab30119504effb16a3db0790f20d5a3 for removal 2024.10.22 00:51:35.073325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bf487d9cf231677cf5db880b3f29de63, offset: 0, size: 5373 2024.10.22 00:51:35.074428 [ 1476 ] {} CacheMetadata: Key 0ab30119504effb16a3db0790f20d5a3 is removed from metadata 2024.10.22 00:51:35.075847 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bf487d9cf231677cf5db880b3f29de63 for removal 2024.10.22 00:51:35.083249 [ 1476 ] {} CacheMetadata: Key bf487d9cf231677cf5db880b3f29de63 is removed from metadata 2024.10.22 00:51:35.090289 [ 1476 ] {} CacheMetadata: Key e0c6aaffce7ab6860cf2a848a9cf146a is removed from metadata 2024.10.22 00:51:35.159086 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:35.162730 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:35.178798 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 925bfa3dab52d6878fa45ab843a9dc64, offset: 0, size: 151 2024.10.22 00:51:35.180756 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 925bfa3dab52d6878fa45ab843a9dc64 for removal 2024.10.22 00:51:35.182415 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6f02c9c37aa42468985089d0de12dd2b, offset: 0, size: 158 2024.10.22 00:51:35.183360 [ 1476 ] {} CacheMetadata: Key 925bfa3dab52d6878fa45ab843a9dc64 is removed from metadata 2024.10.22 00:51:35.184972 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6f02c9c37aa42468985089d0de12dd2b for removal 2024.10.22 00:51:35.288702 [ 1476 ] {} CacheMetadata: Key 6f02c9c37aa42468985089d0de12dd2b is removed from metadata 2024.10.22 00:51:35.289271 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 381cdfe731722b730363b36f820a411d, offset: 0, size: 167 2024.10.22 00:51:35.293478 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 381cdfe731722b730363b36f820a411d for removal 2024.10.22 00:51:35.297353 [ 1476 ] {} CacheMetadata: Key 381cdfe731722b730363b36f820a411d is removed from metadata 2024.10.22 00:51:35.329048 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_404_404_0 (state Temporary) 2024.10.22 00:51:35.418405 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_404_404_0 to 202410_417_417_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:35.589119 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 22953 2024.10.22 00:51:35.675092 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_231_46/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:51:36.687834 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 140576 bytes for 2de32028846db5647ebc6ecf37266172:0) 2024.10.22 00:51:36.703652 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e61e81acafc74655ca325962fd66c128, offset: 0, size: 157 2024.10.22 00:51:36.712540 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e61e81acafc74655ca325962fd66c128 for removal 2024.10.22 00:51:36.718275 [ 1476 ] {} CacheMetadata: Key e61e81acafc74655ca325962fd66c128 is removed from metadata 2024.10.22 00:51:36.719928 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d701c49db6dd533f5dca8895d39c7473, offset: 0, size: 77404 2024.10.22 00:51:36.726443 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d701c49db6dd533f5dca8895d39c7473 for removal 2024.10.22 00:51:36.732651 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ca41b8dba4d34790fa36e31f4e13a02, offset: 0, size: 91267 2024.10.22 00:51:36.738719 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ca41b8dba4d34790fa36e31f4e13a02 for removal 2024.10.22 00:51:36.764805 [ 1476 ] {} CacheMetadata: Key d701c49db6dd533f5dca8895d39c7473 is removed from metadata 2024.10.22 00:51:36.851665 [ 1476 ] {} CacheMetadata: Key 7ca41b8dba4d34790fa36e31f4e13a02 is removed from metadata 2024.10.22 00:51:36.852714 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d716c1b922a290cbc846582a2812b034, offset: 0, size: 157 2024.10.22 00:51:36.859293 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d716c1b922a290cbc846582a2812b034 for removal 2024.10.22 00:51:36.864067 [ 1476 ] {} CacheMetadata: Key d716c1b922a290cbc846582a2812b034 is removed from metadata 2024.10.22 00:51:37.093501 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 389544 2024.10.22 00:51:37.197357 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_49_10/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:51:37.981164 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 134820 bytes for ce700021718a72f2d7d94137539ad37e:0) 2024.10.22 00:51:37.983117 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2de32028846db5647ebc6ecf37266172, offset: 0, size: 140576 2024.10.22 00:51:37.986832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2de32028846db5647ebc6ecf37266172 for removal 2024.10.22 00:51:37.988633 [ 1476 ] {} CacheMetadata: Key 2de32028846db5647ebc6ecf37266172 is removed from metadata 2024.10.22 00:51:37.989164 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6cbaff95257607fb2e207ea8f314aa81, offset: 0, size: 168 2024.10.22 00:51:37.994479 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6cbaff95257607fb2e207ea8f314aa81 for removal 2024.10.22 00:51:37.997722 [ 1476 ] {} CacheMetadata: Key 6cbaff95257607fb2e207ea8f314aa81 is removed from metadata 2024.10.22 00:51:38.478194 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 1917 2024.10.22 00:51:38.529336 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 75 entries to flush up to offset 15619 2024.10.22 00:51:38.666971 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:38.670962 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:38.691671 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:38.696047 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:51:38.871268 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_235_235_0 (state Temporary) 2024.10.22 00:51:38.894862 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_226_226_0 (state Temporary) 2024.10.22 00:51:39.064320 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 67896 bytes for 926c0becc76618c8e0131e3720cee956:0) 2024.10.22 00:51:39.067388 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ce700021718a72f2d7d94137539ad37e, offset: 0, size: 134820 2024.10.22 00:51:39.071339 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ce700021718a72f2d7d94137539ad37e for removal 2024.10.22 00:51:39.074975 [ 1476 ] {} CacheMetadata: Key ce700021718a72f2d7d94137539ad37e is removed from metadata 2024.10.22 00:51:39.076018 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 28faf65547085c86976c20a319cea7ac, offset: 0, size: 168 2024.10.22 00:51:39.082897 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 28faf65547085c86976c20a319cea7ac for removal 2024.10.22 00:51:39.087095 [ 1476 ] {} CacheMetadata: Key 28faf65547085c86976c20a319cea7ac is removed from metadata 2024.10.22 00:51:39.107616 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_226_226_0 to 202410_233_233_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:39.160316 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 389544 2024.10.22 00:51:39.331546 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_177_36, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0, 202410_182_182_0] 2024.10.22 00:51:39.331844 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_235_235_0 to 202410_241_241_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:39.335991 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_177_36, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0, 202410_182_182_0] 2024.10.22 00:51:39.440885 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_147_30, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:51:39.445469 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_147_30, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:51:39.478919 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_177_36, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0, 202410_182_182_0] 2024.10.22 00:51:39.491405 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_147_30, 202410_148_148_0, 202410_149_149_0, 202410_150_150_0, 202410_151_151_0] 2024.10.22 00:51:39.510347 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15619 2024.10.22 00:51:39.600812 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 23028 2024.10.22 00:51:39.682842 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 3347 2024.10.22 00:51:39.728090 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:51:39.730500 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:51:39.766530 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:39.770920 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:39.792066 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:39.796314 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:39.798476 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 209 2024.10.22 00:51:39.832579 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_405_405_0 (state Temporary) 2024.10.22 00:51:39.848751 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_237_237_0 (state Temporary) 2024.10.22 00:51:39.894936 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_237_237_0 to 202410_250_250_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:39.966299 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3347 2024.10.22 00:51:39.986800 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_405_405_0 to 202410_418_418_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:40.067245 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23028 2024.10.22 00:51:40.071627 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} MergeTask::MergeProjectionsStage: Merge sorted 400062 rows, containing 4 columns (4 merged, 0 gathered) in 18.797403896 sec., 21282.832576956618 rows/sec., 312.18 KiB/sec. 2024.10.22 00:51:40.171330 [ 1268 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:51:40.188535 [ 1268 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:51:40.203731 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} MergedBlockOutputStream: filled checksums 202410_1_231_46 (state Temporary) 2024.10.22 00:51:40.331142 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:40.379030 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_231_46 to 202410_1_231_46 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:40.399923 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_231_46} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_226_45, 202410_231_231_0] -> 202410_1_231_46 2024.10.22 00:51:40.436270 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:40.445449 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:51:40.524658 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_191_191_0 (state Temporary) 2024.10.22 00:51:40.592980 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(9fe04f7c0cf72ff745a95c4749293eb8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9fe04f7c0cf72ff745a95c4749293eb8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:51:40.596274 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 463523, file segment info: File segment: [0, 33554431], key: 9fe04f7c0cf72ff745a95c4749293eb8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:51:40.598909 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:51:40.774944 [ 1726 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hfe/yxmrdaljbjhvgofbltweptehwbifu, size 463523 2024.10.22 00:51:40.829781 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_191_191_0 to 202410_201_201_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:40.890485 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 13.06 MiB, will set to 1.65 GiB (RSS), difference: 1.90 MiB 2024.10.22 00:51:40.987349 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 209 2024.10.22 00:51:41.062465 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 00:51:41.069749 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 00:51:41.177475 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 00:51:41.296104 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 00:51:41.315446 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 00:51:41.351545 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 00:51:41.379758 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 00:51:41.397701 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 40507 bytes for 084c1dc7b06c693319e213270f83fd88:0) 2024.10.22 00:51:41.403250 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 926c0becc76618c8e0131e3720cee956, offset: 0, size: 67896 2024.10.22 00:51:41.417518 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 926c0becc76618c8e0131e3720cee956 for removal 2024.10.22 00:51:41.417704 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zsg/uwrzinqzglxowihrtfhnyeeuoaydx (LogSeriesLimiter: on interval from 2024-10-22 00:51:22 to 2024-10-22 00:51:41 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:51:41.421145 [ 1476 ] {} CacheMetadata: Key 926c0becc76618c8e0131e3720cee956 is removed from metadata 2024.10.22 00:51:41.422987 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f4ca11f3cfbd854b5b6749dbfe3d7519, offset: 0, size: 156 2024.10.22 00:51:41.448417 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f4ca11f3cfbd854b5b6749dbfe3d7519 for removal 2024.10.22 00:51:41.549800 [ 1476 ] {} CacheMetadata: Key f4ca11f3cfbd854b5b6749dbfe3d7519 is removed from metadata 2024.10.22 00:51:42.272298 [ 1164 ] {} DatabaseCatalog: Execution took 307 ms. 2024.10.22 00:51:42.279022 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 140327 bytes for 3ada5e5b35eaa90c6f524b0f0b7ce4a2:0) 2024.10.22 00:51:42.280208 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 01f593a9c80c89f39ee05713149d24c1, offset: 0, size: 156 2024.10.22 00:51:42.300930 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 01f593a9c80c89f39ee05713149d24c1 for removal 2024.10.22 00:51:42.308648 [ 1476 ] {} CacheMetadata: Key 01f593a9c80c89f39ee05713149d24c1 is removed from metadata 2024.10.22 00:51:42.311150 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0922f005f837cebe6869649bdcb78c38, offset: 0, size: 61700 2024.10.22 00:51:42.317642 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0922f005f837cebe6869649bdcb78c38 for removal 2024.10.22 00:51:42.321825 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 084c1dc7b06c693319e213270f83fd88, offset: 0, size: 40507 2024.10.22 00:51:42.326004 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 084c1dc7b06c693319e213270f83fd88 for removal 2024.10.22 00:51:42.335247 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a936286ffaade127b56a54b33dbf90fb, offset: 0, size: 156 2024.10.22 00:51:42.341016 [ 1476 ] {} CacheMetadata: Key 084c1dc7b06c693319e213270f83fd88 is removed from metadata 2024.10.22 00:51:42.359733 [ 1476 ] {} CacheMetadata: Key 0922f005f837cebe6869649bdcb78c38 is removed from metadata 2024.10.22 00:51:42.345779 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a936286ffaade127b56a54b33dbf90fb for removal 2024.10.22 00:51:42.456332 [ 1476 ] {} CacheMetadata: Key a936286ffaade127b56a54b33dbf90fb is removed from metadata 2024.10.22 00:51:43.632821 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 12.03 MiB, will set to 1.65 GiB (RSS), difference: -1.97 MiB 2024.10.22 00:51:43.827220 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 29314 bytes for 9d2fb0392dda161bf9e37a32943684e3:0) 2024.10.22 00:51:43.831328 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3ada5e5b35eaa90c6f524b0f0b7ce4a2, offset: 0, size: 140327 2024.10.22 00:51:43.836253 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3ada5e5b35eaa90c6f524b0f0b7ce4a2 for removal 2024.10.22 00:51:43.838535 [ 1476 ] {} CacheMetadata: Key 3ada5e5b35eaa90c6f524b0f0b7ce4a2 is removed from metadata 2024.10.22 00:51:43.844235 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aa203d4560b53fd14f8d9dcb03b11164, offset: 0, size: 168 2024.10.22 00:51:43.853010 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aa203d4560b53fd14f8d9dcb03b11164 for removal 2024.10.22 00:51:43.861252 [ 1476 ] {} CacheMetadata: Key aa203d4560b53fd14f8d9dcb03b11164 is removed from metadata 2024.10.22 00:51:44.088153 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 23097 2024.10.22 00:51:44.297660 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_186_37, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0, 202410_191_191_0] 2024.10.22 00:51:44.304647 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_186_37, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0, 202410_191_191_0] 2024.10.22 00:51:44.332926 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_186_37, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0, 202410_191_191_0] 2024.10.22 00:51:44.778918 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:44.783062 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:44.851397 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_406_406_0 (state Temporary) 2024.10.22 00:51:44.982717 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_313_63, 202410_314_314_0, 202410_315_315_0, 202410_316_316_0, 202410_317_317_0, 202410_318_318_0] 2024.10.22 00:51:44.985469 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_313_63, 202410_314_314_0, 202410_315_315_0, 202410_316_316_0, 202410_317_317_0, 202410_318_318_0] 2024.10.22 00:51:44.993239 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_406_406_0 to 202410_419_419_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:45.020884 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_313_63, 202410_314_314_0, 202410_315_315_0, 202410_316_316_0, 202410_317_317_0, 202410_318_318_0] 2024.10.22 00:51:45.185103 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2f935145a70a7dfa23eb6e4d6fe6c02d) : [0, 242888]: Setting download as failed: File segment: [0, 242888], key: 2f935145a70a7dfa23eb6e4d6fe6c02d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:51:45.186330 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2f935145a70a7dfa23eb6e4d6fe6c02d): No space left in cache to reserve 242889 bytes, will continue without cache download 2024.10.22 00:51:45.191030 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23097 2024.10.22 00:51:45.223494 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2f935145a70a7dfa23eb6e4d6fe6c02d, offset: 0, size: 0 2024.10.22 00:51:45.329141 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2f935145a70a7dfa23eb6e4d6fe6c02d for removal 2024.10.22 00:51:45.334884 [ 1476 ] {} CacheMetadata: Key 2f935145a70a7dfa23eb6e4d6fe6c02d is removed from metadata 2024.10.22 00:51:45.564918 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 6.30 MiB, will set to 1.65 GiB (RSS), difference: -3.29 MiB 2024.10.22 00:51:46.090730 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_171_34, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0] 2024.10.22 00:51:46.093588 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_171_34, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0] 2024.10.22 00:51:46.244932 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_171_34, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0, 202410_175_175_0, 202410_176_176_0] 2024.10.22 00:51:46.262421 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1450 entries to flush up to offset 390994 2024.10.22 00:51:46.388184 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 126762 bytes for c12645cdca9a428f446f204883467d04:0) 2024.10.22 00:51:46.390989 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9d2fb0392dda161bf9e37a32943684e3, offset: 0, size: 29314 2024.10.22 00:51:46.395389 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9d2fb0392dda161bf9e37a32943684e3 for removal 2024.10.22 00:51:46.501390 [ 1476 ] {} CacheMetadata: Key 9d2fb0392dda161bf9e37a32943684e3 is removed from metadata 2024.10.22 00:51:46.508667 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1a79d0a8af5774964fdf311af9213182, offset: 0, size: 2333 2024.10.22 00:51:46.520385 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1a79d0a8af5774964fdf311af9213182 for removal 2024.10.22 00:51:46.523699 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ca5283fa77dadd3b400b8d84187d44c5, offset: 0, size: 156 2024.10.22 00:51:46.529650 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ca5283fa77dadd3b400b8d84187d44c5 for removal 2024.10.22 00:51:46.534751 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 72c325deccf701d64ae782dbe915cd6f, offset: 0, size: 156 2024.10.22 00:51:46.541612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 72c325deccf701d64ae782dbe915cd6f for removal 2024.10.22 00:51:46.548430 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4df8c0b293dbfae3faa3b9e5be042930, offset: 0, size: 2019 2024.10.22 00:51:46.552447 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4df8c0b293dbfae3faa3b9e5be042930 for removal 2024.10.22 00:51:46.559703 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a9962a04cbf3a53d20320db7c8d8811e, offset: 0, size: 156 2024.10.22 00:51:46.566829 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a9962a04cbf3a53d20320db7c8d8811e for removal 2024.10.22 00:51:46.620724 [ 1476 ] {} CacheMetadata: Key a9962a04cbf3a53d20320db7c8d8811e is removed from metadata 2024.10.22 00:51:46.631124 [ 1476 ] {} CacheMetadata: Key 4df8c0b293dbfae3faa3b9e5be042930 is removed from metadata 2024.10.22 00:51:46.643965 [ 1476 ] {} CacheMetadata: Key 72c325deccf701d64ae782dbe915cd6f is removed from metadata 2024.10.22 00:51:46.653390 [ 1476 ] {} CacheMetadata: Key ca5283fa77dadd3b400b8d84187d44c5 is removed from metadata 2024.10.22 00:51:46.672519 [ 1476 ] {} CacheMetadata: Key 1a79d0a8af5774964fdf311af9213182 is removed from metadata 2024.10.22 00:51:47.029123 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 44 entries to flush up to offset 15663 2024.10.22 00:51:47.277182 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:47.280782 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:47.349342 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:47.354476 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:47.468129 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 3358 2024.10.22 00:51:47.473285 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_236_236_0 (state Temporary) 2024.10.22 00:51:47.642712 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_227_227_0 (state Temporary) 2024.10.22 00:51:47.769793 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_236_236_0 to 202410_242_242_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:47.790587 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:47.794830 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:47.868351 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_238_238_0 (state Temporary) 2024.10.22 00:51:47.897594 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_227_227_0 to 202410_234_234_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:47.971328 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 5 file segments (while reserving 73720 bytes for fa7983075b319d7dddecd654fc4b2617:0) 2024.10.22 00:51:47.974687 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c12645cdca9a428f446f204883467d04, offset: 0, size: 126762 2024.10.22 00:51:47.975634 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 8.93 MiB, will set to 1.65 GiB (RSS), difference: -3.01 MiB 2024.10.22 00:51:47.978992 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c12645cdca9a428f446f204883467d04 for removal 2024.10.22 00:51:47.984550 [ 1476 ] {} CacheMetadata: Key c12645cdca9a428f446f204883467d04 is removed from metadata 2024.10.22 00:51:47.986108 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1344a6842159470ce90387b6237712bc, offset: 0, size: 156 2024.10.22 00:51:47.990353 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1344a6842159470ce90387b6237712bc for removal 2024.10.22 00:51:47.996050 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 01215ef75b11fc3f48c9557d4cdde0ae, offset: 0, size: 157 2024.10.22 00:51:48.009606 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_238_238_0 to 202410_251_251_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:48.012349 [ 1476 ] {} CacheMetadata: Key 1344a6842159470ce90387b6237712bc is removed from metadata 2024.10.22 00:51:48.030488 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15663 2024.10.22 00:51:48.039904 [ 1188 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_246_49 to 202410_251_251_0 2024.10.22 00:51:48.055981 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3358 2024.10.22 00:51:48.057155 [ 1188 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:48.061784 [ 1188 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:48.075122 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 390994 2024.10.22 00:51:48.087900 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_246_49 to 202410_251_251_0 into Compact with storage Full 2024.10.22 00:51:48.106655 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 01215ef75b11fc3f48c9557d4cdde0ae for removal 2024.10.22 00:51:48.113328 [ 1476 ] {} CacheMetadata: Key 01215ef75b11fc3f48c9557d4cdde0ae is removed from metadata 2024.10.22 00:51:48.115620 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1b32c48864013af462c2b9e799410b9e, offset: 0, size: 26232 2024.10.22 00:51:48.120855 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1b32c48864013af462c2b9e799410b9e for removal 2024.10.22 00:51:48.126338 [ 1476 ] {} CacheMetadata: Key 1b32c48864013af462c2b9e799410b9e is removed from metadata 2024.10.22 00:51:48.126101 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5242f3a59aac4e5e12d7ee6ff6da027f, offset: 0, size: 156 2024.10.22 00:51:48.158478 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5242f3a59aac4e5e12d7ee6ff6da027f for removal 2024.10.22 00:51:48.163099 [ 1476 ] {} CacheMetadata: Key 5242f3a59aac4e5e12d7ee6ff6da027f is removed from metadata 2024.10.22 00:51:48.291454 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:51:48.307186 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_246_49, total 3394 rows starting from the beginning of the part 2024.10.22 00:51:48.468367 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_247_247_0, total 14 rows starting from the beginning of the part 2024.10.22 00:51:48.673983 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_248_248_0, total 9 rows starting from the beginning of the part 2024.10.22 00:51:48.713553 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_249_249_0, total 24 rows starting from the beginning of the part 2024.10.22 00:51:48.880296 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_250_250_0, total 12 rows starting from the beginning of the part 2024.10.22 00:51:48.960204 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_251_251_0, total 11 rows starting from the beginning of the part 2024.10.22 00:51:49.207812 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 23183 2024.10.22 00:51:49.435508 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(10dd0efed35164f7b130c65683c7fbf6) : [0, 176981]: Setting download as failed: File segment: [0, 176981], key: 10dd0efed35164f7b130c65683c7fbf6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:51:49.439650 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(10dd0efed35164f7b130c65683c7fbf6): No space left in cache to reserve 176982 bytes, will continue without cache download 2024.10.22 00:51:49.488822 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:49.492866 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:49.571737 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_407_407_0 (state Temporary) 2024.10.22 00:51:49.577531 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 10dd0efed35164f7b130c65683c7fbf6, offset: 0, size: 0 2024.10.22 00:51:49.579386 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 10dd0efed35164f7b130c65683c7fbf6 for removal 2024.10.22 00:51:49.581317 [ 1476 ] {} CacheMetadata: Key 10dd0efed35164f7b130c65683c7fbf6 is removed from metadata 2024.10.22 00:51:49.729063 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:51:49.741211 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:51:49.797128 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_407_407_0 to 202410_420_420_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:49.978123 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23183 2024.10.22 00:51:50.003397 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} MergeTask::MergeProjectionsStage: Merge sorted 3464 rows, containing 12 columns (12 merged, 0 gathered) in 1.917384111 sec., 1806.6280929976892 rows/sec., 282.10 KiB/sec. 2024.10.22 00:51:50.037558 [ 1189 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 5 parts from 202410_1_415_94 to 202410_419_419_0 2024.10.22 00:51:50.057563 [ 1189 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:50.064733 [ 1189 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:50.080557 [ 1189 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 253 ms. 2024.10.22 00:51:50.080551 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_419_95} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_415_94 to 202410_419_419_0 into Compact with storage Full 2024.10.22 00:51:50.196084 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5d1edad818368ab176617620ed66b1c9) : [0, 192780]: Setting download as failed: File segment: [0, 192780], key: 5d1edad818368ab176617620ed66b1c9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:51:50.199105 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5d1edad818368ab176617620ed66b1c9): No space left in cache to reserve 192781 bytes, will continue without cache download 2024.10.22 00:51:50.213507 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_419_95} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:51:50.228306 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5d1edad818368ab176617620ed66b1c9, offset: 0, size: 0 2024.10.22 00:51:50.231748 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5d1edad818368ab176617620ed66b1c9 for removal 2024.10.22 00:51:50.234628 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_419_95} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_415_94, total 23603 rows starting from the beginning of the part 2024.10.22 00:51:50.244560 [ 1476 ] {} CacheMetadata: Key 5d1edad818368ab176617620ed66b1c9 is removed from metadata 2024.10.22 00:51:50.278216 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} MergedBlockOutputStream: filled checksums 202410_1_251_50 (state Temporary) 2024.10.22 00:51:50.377519 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_419_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_416_416_0, total 91 rows starting from the beginning of the part 2024.10.22 00:51:50.539716 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_419_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_417_417_0, total 90 rows starting from the beginning of the part 2024.10.22 00:51:50.685298 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_419_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_418_418_0, total 75 rows starting from the beginning of the part 2024.10.22 00:51:50.683859 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_251_50 to 202410_1_251_50 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:50.719034 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_251_50} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_246_49, 202410_251_251_0] -> 202410_1_251_50 2024.10.22 00:51:50.721211 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_419_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_419_419_0, total 69 rows starting from the beginning of the part 2024.10.22 00:51:50.756274 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:51:51.066756 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 103794 bytes for e432a53983749e9877703adb3eac14de:0) 2024.10.22 00:51:51.070274 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ff0b680e7c8db71c6f271932295cf94a, offset: 0, size: 87503 2024.10.22 00:51:51.073315 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ff0b680e7c8db71c6f271932295cf94a for removal 2024.10.22 00:51:51.075246 [ 1476 ] {} CacheMetadata: Key ff0b680e7c8db71c6f271932295cf94a is removed from metadata 2024.10.22 00:51:51.076037 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 71a34efa7d89c95ba9fe69027dc9bf12, offset: 0, size: 157 2024.10.22 00:51:51.127435 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 71a34efa7d89c95ba9fe69027dc9bf12 for removal 2024.10.22 00:51:51.130808 [ 1476 ] {} CacheMetadata: Key 71a34efa7d89c95ba9fe69027dc9bf12 is removed from metadata 2024.10.22 00:51:51.131910 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fa7983075b319d7dddecd654fc4b2617, offset: 0, size: 73720 2024.10.22 00:51:51.140386 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fa7983075b319d7dddecd654fc4b2617 for removal 2024.10.22 00:51:51.147965 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 792fb9e1ae5b1e74ed3e8840df10969d, offset: 0, size: 167 2024.10.22 00:51:51.153488 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 792fb9e1ae5b1e74ed3e8840df10969d for removal 2024.10.22 00:51:51.156151 [ 1476 ] {} CacheMetadata: Key fa7983075b319d7dddecd654fc4b2617 is removed from metadata 2024.10.22 00:51:51.163198 [ 1476 ] {} CacheMetadata: Key 792fb9e1ae5b1e74ed3e8840df10969d is removed from metadata 2024.10.22 00:51:51.623388 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 6.55 MiB, will set to 1.65 GiB (RSS), difference: -8.55 MiB 2024.10.22 00:51:52.008858 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a0398261a63a49798abcb1f9784b5d9f) : [0, 1070994]: Setting download as failed: File segment: [0, 1070994], key: a0398261a63a49798abcb1f9784b5d9f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:51:52.012125 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a0398261a63a49798abcb1f9784b5d9f): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:51:52.068349 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a0398261a63a49798abcb1f9784b5d9f, offset: 0, size: 0 2024.10.22 00:51:52.070103 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a0398261a63a49798abcb1f9784b5d9f for removal 2024.10.22 00:51:52.071846 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_49_10/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:51:52.076043 [ 1476 ] {} CacheMetadata: Key a0398261a63a49798abcb1f9784b5d9f is removed from metadata 2024.10.22 00:51:52.118116 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_419_95/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:51:52.489503 [ 1199 ] {} DatabaseCatalog: Execution took 218 ms. 2024.10.22 00:51:53.070918 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 54839 bytes for e22d51a3ecae9910ef5013fd1cb82d7d:0) 2024.10.22 00:51:53.072268 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e432a53983749e9877703adb3eac14de, offset: 0, size: 103794 2024.10.22 00:51:53.075078 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e432a53983749e9877703adb3eac14de for removal 2024.10.22 00:51:53.078555 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 599d05021b33b2f4e11cdaaa2199c790, offset: 0, size: 168 2024.10.22 00:51:53.083408 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 599d05021b33b2f4e11cdaaa2199c790 for removal 2024.10.22 00:51:53.086984 [ 1476 ] {} CacheMetadata: Key e432a53983749e9877703adb3eac14de is removed from metadata 2024.10.22 00:51:53.089824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 102a6d53ac4c42f809813216823d2e4d, offset: 0, size: 169 2024.10.22 00:51:53.092746 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 102a6d53ac4c42f809813216823d2e4d for removal 2024.10.22 00:51:53.095817 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9680f4d83a0c6775d1a9bae137789020, offset: 0, size: 169 2024.10.22 00:51:53.103247 [ 1476 ] {} CacheMetadata: Key 102a6d53ac4c42f809813216823d2e4d is removed from metadata 2024.10.22 00:51:53.105735 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9680f4d83a0c6775d1a9bae137789020 for removal 2024.10.22 00:51:53.116072 [ 1476 ] {} CacheMetadata: Key 9680f4d83a0c6775d1a9bae137789020 is removed from metadata 2024.10.22 00:51:53.123805 [ 1476 ] {} CacheMetadata: Key 599d05021b33b2f4e11cdaaa2199c790 is removed from metadata 2024.10.22 00:51:53.169521 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_419_95} MergeTask::MergeProjectionsStage: Merge sorted 23928 rows, containing 13 columns (13 merged, 0 gathered) in 3.093905708 sec., 7733.913783515991 rows/sec., 1.16 MiB/sec. 2024.10.22 00:51:53.477027 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_419_95} MergedBlockOutputStream: filled checksums 202410_1_419_95 (state Temporary) 2024.10.22 00:51:53.490236 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 5.83 MiB, will set to 1.65 GiB (RSS), difference: 5.88 MiB 2024.10.22 00:51:53.570998 [ 1233 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 00:51:53.682253 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_419_95} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_419_95 to 202410_1_419_95 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:53.684946 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_419_95} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 5 parts: [202410_1_415_94, 202410_419_419_0] -> 202410_1_419_95 2024.10.22 00:51:53.734996 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.88 MiB. 2024.10.22 00:51:54.087276 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 23259 2024.10.22 00:51:54.097047 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:54.103579 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:54.284194 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 108241 bytes for fab28845c960b3ff6673ea0812a146db:0) 2024.10.22 00:51:54.285729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8d6fc893a6d5e6a9df82915094de3166, offset: 0, size: 63560 2024.10.22 00:51:54.289211 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8d6fc893a6d5e6a9df82915094de3166 for removal 2024.10.22 00:51:54.292412 [ 1476 ] {} CacheMetadata: Key 8d6fc893a6d5e6a9df82915094de3166 is removed from metadata 2024.10.22 00:51:54.293638 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 91fa90f0dfd47d151cec717c06c848dc, offset: 0, size: 156 2024.10.22 00:51:54.301534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 91fa90f0dfd47d151cec717c06c848dc for removal 2024.10.22 00:51:54.310283 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 852c8bf357b3a016521378f4bed1e179, offset: 0, size: 168 2024.10.22 00:51:54.322496 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 852c8bf357b3a016521378f4bed1e179 for removal 2024.10.22 00:51:54.325591 [ 1476 ] {} CacheMetadata: Key 852c8bf357b3a016521378f4bed1e179 is removed from metadata 2024.10.22 00:51:54.347315 [ 1476 ] {} CacheMetadata: Key 91fa90f0dfd47d151cec717c06c848dc is removed from metadata 2024.10.22 00:51:54.567477 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:54.570901 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:54.626570 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_408_408_0 (state Temporary) 2024.10.22 00:51:54.669609 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_408_408_0 to 202410_421_421_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:54.816432 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23259 2024.10.22 00:51:55.080488 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1622 entries to flush up to offset 392616 2024.10.22 00:51:55.192797 [ 1277 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:51:55.244232 [ 1277 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:51:55.466968 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 147563 bytes for 3ce03a67cb9c2a0bb3f06b22b68024a3:0) 2024.10.22 00:51:55.471233 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fab28845c960b3ff6673ea0812a146db, offset: 0, size: 108241 2024.10.22 00:51:55.473753 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fab28845c960b3ff6673ea0812a146db for removal 2024.10.22 00:51:55.475572 [ 1476 ] {} CacheMetadata: Key fab28845c960b3ff6673ea0812a146db is removed from metadata 2024.10.22 00:51:55.478606 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e8cfad8b3a9e6c7b021b2138730e037c, offset: 0, size: 153 2024.10.22 00:51:55.481224 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e8cfad8b3a9e6c7b021b2138730e037c for removal 2024.10.22 00:51:55.483932 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e22d51a3ecae9910ef5013fd1cb82d7d, offset: 0, size: 54839 2024.10.22 00:51:55.487451 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e22d51a3ecae9910ef5013fd1cb82d7d for removal 2024.10.22 00:51:55.490791 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8b995d5239e94f55389b6b3d20b88d6f, offset: 0, size: 156 2024.10.22 00:51:55.497655 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8b995d5239e94f55389b6b3d20b88d6f for removal 2024.10.22 00:51:55.559631 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 89 entries to flush up to offset 15752 2024.10.22 00:51:55.560129 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 3376 2024.10.22 00:51:55.586982 [ 1476 ] {} CacheMetadata: Key 8b995d5239e94f55389b6b3d20b88d6f is removed from metadata 2024.10.22 00:51:55.596384 [ 1476 ] {} CacheMetadata: Key e22d51a3ecae9910ef5013fd1cb82d7d is removed from metadata 2024.10.22 00:51:55.629219 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 6.18 MiB, will set to 1.65 GiB (RSS), difference: 1.36 MiB 2024.10.22 00:51:55.717492 [ 1476 ] {} CacheMetadata: Key e8cfad8b3a9e6c7b021b2138730e037c is removed from metadata 2024.10.22 00:51:55.776518 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:55.778358 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:55.833350 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_237_237_0 (state Temporary) 2024.10.22 00:51:55.884662 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:55.889125 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_237_237_0 to 202410_243_243_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:55.889596 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:55.941294 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_239_239_0 (state Temporary) 2024.10.22 00:51:55.969625 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15752 2024.10.22 00:51:55.990425 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 210 2024.10.22 00:51:56.209382 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_239_239_0 to 202410_252_252_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:56.287021 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3376 2024.10.22 00:51:56.345998 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:56.461162 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:56.548415 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:56.662345 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:56.684298 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 83261 bytes for 148ad92b4b80560e8280bfa57e68f16b:0) 2024.10.22 00:51:56.687612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3ce03a67cb9c2a0bb3f06b22b68024a3, offset: 0, size: 147563 2024.10.22 00:51:56.692842 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3ce03a67cb9c2a0bb3f06b22b68024a3 for removal 2024.10.22 00:51:56.698766 [ 1476 ] {} CacheMetadata: Key 3ce03a67cb9c2a0bb3f06b22b68024a3 is removed from metadata 2024.10.22 00:51:56.704002 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f815d4ec81b83860f2b7fa561c77a653, offset: 0, size: 155 2024.10.22 00:51:56.711528 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f815d4ec81b83860f2b7fa561c77a653 for removal 2024.10.22 00:51:56.819313 [ 1476 ] {} CacheMetadata: Key f815d4ec81b83860f2b7fa561c77a653 is removed from metadata 2024.10.22 00:51:56.904788 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_192_192_0 (state Temporary) 2024.10.22 00:51:57.144949 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_192_192_0 to 202410_202_202_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:57.156699 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_228_228_0 (state Temporary) 2024.10.22 00:51:57.651256 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 5.43 MiB, will set to 1.64 GiB (RSS), difference: -11.13 MiB 2024.10.22 00:51:57.690977 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 210 2024.10.22 00:51:57.864884 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(343502a9cbf41fe676f7dd5859af66b3) : [0, 404348]: Setting download as failed: File segment: [0, 404348], key: 343502a9cbf41fe676f7dd5859af66b3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:51:57.866841 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(343502a9cbf41fe676f7dd5859af66b3): No space left in cache to reserve 404349 bytes, will continue without cache download 2024.10.22 00:51:57.873747 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_228_228_0 to 202410_235_235_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:57.913610 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 392616 2024.10.22 00:51:57.915642 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 343502a9cbf41fe676f7dd5859af66b3, offset: 0, size: 0 2024.10.22 00:51:57.917093 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 343502a9cbf41fe676f7dd5859af66b3 for removal 2024.10.22 00:51:57.918700 [ 1476 ] {} CacheMetadata: Key 343502a9cbf41fe676f7dd5859af66b3 is removed from metadata 2024.10.22 00:51:58.613572 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_48_48_0 (state Temporary) 2024.10.22 00:51:58.802748 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7f3588c89f0a8f3ae3ac8e72a743e6fb) : [0, 805560]: Setting download as failed: File segment: [0, 805560], key: 7f3588c89f0a8f3ae3ac8e72a743e6fb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:51:58.805337 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7f3588c89f0a8f3ae3ac8e72a743e6fb): No space left in cache to reserve 805561 bytes, will continue without cache download 2024.10.22 00:51:58.844354 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 23333 2024.10.22 00:51:58.951531 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7f3588c89f0a8f3ae3ac8e72a743e6fb, offset: 0, size: 0 2024.10.22 00:51:58.955559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7f3588c89f0a8f3ae3ac8e72a743e6fb for removal 2024.10.22 00:51:58.960792 [ 1476 ] {} CacheMetadata: Key 7f3588c89f0a8f3ae3ac8e72a743e6fb is removed from metadata 2024.10.22 00:51:59.338176 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:51:59.346607 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:51:59.389433 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_409_409_0 (state Temporary) 2024.10.22 00:51:59.470439 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 5.78 MiB, will set to 1.64 GiB (RSS), difference: 1.86 MiB 2024.10.22 00:51:59.520019 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(2d64afe9d4bc84dcec8d5ac421197cce) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2d64afe9d4bc84dcec8d5ac421197cce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 00:51:59.522052 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 852584, file segment info: File segment: [0, 33554431], key: 2d64afe9d4bc84dcec8d5ac421197cce, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 00:51:59.524140 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:51:59.607454 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_409_409_0 to 202410_422_422_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:51:59.623425 [ 1703 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zsg/uwrzinqzglxowihrtfhnyeeuoaydx, size 852584 2024.10.22 00:51:59.687322 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23333 2024.10.22 00:51:59.730504 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:51:59.734561 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:51:59.867333 [ 1234 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 00:51:59.934339 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 00:52:00.068135 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 00:52:00.080674 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 89883 bytes for 6a4784b6fe82b51c060964241ee4de57:0) 2024.10.22 00:52:00.083812 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 148ad92b4b80560e8280bfa57e68f16b, offset: 0, size: 83261 2024.10.22 00:52:00.084212 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 00:52:00.087226 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 148ad92b4b80560e8280bfa57e68f16b for removal 2024.10.22 00:52:00.090325 [ 1476 ] {} CacheMetadata: Key 148ad92b4b80560e8280bfa57e68f16b is removed from metadata 2024.10.22 00:52:00.090604 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 710e1bd234e116a89c38ac87eea28f68, offset: 0, size: 157 2024.10.22 00:52:00.094380 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 710e1bd234e116a89c38ac87eea28f68 for removal 2024.10.22 00:52:00.120408 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 00:52:00.130740 [ 1476 ] {} CacheMetadata: Key 710e1bd234e116a89c38ac87eea28f68 is removed from metadata 2024.10.22 00:52:00.171380 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 00:52:00.340714 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 00:52:00.472040 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 00:52:00.625729 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jwa/aeadmywzhbibkxbpudwvdxmiouvxh (LogSeriesLimiter: on interval from 2024-10-22 00:51:41 to 2024-10-22 00:52:00 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:52:01.670720 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3928b79c09d381f09ee97c81868f1fba) : [0, 1057126]: Setting download as failed: File segment: [0, 1057126], key: 3928b79c09d381f09ee97c81868f1fba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:01.672500 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3928b79c09d381f09ee97c81868f1fba): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:52:01.852011 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 2.98 MiB, will set to 1.64 GiB (RSS), difference: -4.61 MiB 2024.10.22 00:52:01.900951 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3928b79c09d381f09ee97c81868f1fba, offset: 0, size: 0 2024.10.22 00:52:01.903039 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3928b79c09d381f09ee97c81868f1fba for removal 2024.10.22 00:52:01.915522 [ 1476 ] {} CacheMetadata: Key 3928b79c09d381f09ee97c81868f1fba is removed from metadata 2024.10.22 00:52:02.692896 [ 1251 ] {} DatabaseCatalog: Execution took 203 ms. 2024.10.22 00:52:03.264245 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 92899 bytes for 09e7f3f3bb88724ad4adb8e208b2b278:0) 2024.10.22 00:52:03.267792 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 02bfe3997b5f5f275c6b832ebf4085dc, offset: 0, size: 157 2024.10.22 00:52:03.272485 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 02bfe3997b5f5f275c6b832ebf4085dc for removal 2024.10.22 00:52:03.275084 [ 1476 ] {} CacheMetadata: Key 02bfe3997b5f5f275c6b832ebf4085dc is removed from metadata 2024.10.22 00:52:03.275827 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f1502a0519b31e9336638288f73dcdbd, offset: 0, size: 166 2024.10.22 00:52:03.289072 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f1502a0519b31e9336638288f73dcdbd for removal 2024.10.22 00:52:03.291922 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6a4784b6fe82b51c060964241ee4de57, offset: 0, size: 89883 2024.10.22 00:52:03.315346 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6a4784b6fe82b51c060964241ee4de57 for removal 2024.10.22 00:52:03.322819 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 08c629bec2068831b96f22925bb072be, offset: 0, size: 182 2024.10.22 00:52:03.329320 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 08c629bec2068831b96f22925bb072be for removal 2024.10.22 00:52:03.344126 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 7.11 MiB, will set to 1.64 GiB (RSS), difference: 2.57 MiB 2024.10.22 00:52:03.402813 [ 1476 ] {} CacheMetadata: Key f1502a0519b31e9336638288f73dcdbd is removed from metadata 2024.10.22 00:52:03.417216 [ 1476 ] {} CacheMetadata: Key 08c629bec2068831b96f22925bb072be is removed from metadata 2024.10.22 00:52:03.432216 [ 1476 ] {} CacheMetadata: Key 6a4784b6fe82b51c060964241ee4de57 is removed from metadata 2024.10.22 00:52:03.476016 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 47 entries to flush up to offset 15799 2024.10.22 00:52:03.494436 [ 1250 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 00:52:03.600231 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:03.617835 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:52:03.692168 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 23390 2024.10.22 00:52:03.788862 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 3381 2024.10.22 00:52:03.979076 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_238_238_0 (state Temporary) 2024.10.22 00:52:04.001702 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:04.007745 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:04.116787 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:04.130535 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:04.139799 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_240_240_0 (state Temporary) 2024.10.22 00:52:04.193818 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_240_240_0 to 202410_253_253_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:04.247401 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_238_238_0 to 202410_244_244_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:04.265659 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3381 2024.10.22 00:52:04.297188 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_410_410_0 (state Temporary) 2024.10.22 00:52:04.300428 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15799 2024.10.22 00:52:04.475273 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_410_410_0 to 202410_423_423_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:04.714246 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23390 2024.10.22 00:52:04.796365 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 5 file segments (while reserving 79316 bytes for 38b8001231574df5d64738d5d7f6076e:0) 2024.10.22 00:52:04.800162 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 479a71e44b58c2bdf4f092a62ea11ec1, offset: 0, size: 157 2024.10.22 00:52:04.812775 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 479a71e44b58c2bdf4f092a62ea11ec1 for removal 2024.10.22 00:52:04.818802 [ 1476 ] {} CacheMetadata: Key 479a71e44b58c2bdf4f092a62ea11ec1 is removed from metadata 2024.10.22 00:52:04.820027 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 09e7f3f3bb88724ad4adb8e208b2b278, offset: 0, size: 92899 2024.10.22 00:52:04.925465 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 394536 2024.10.22 00:52:04.927299 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 09e7f3f3bb88724ad4adb8e208b2b278 for removal 2024.10.22 00:52:05.032162 [ 1476 ] {} CacheMetadata: Key 09e7f3f3bb88724ad4adb8e208b2b278 is removed from metadata 2024.10.22 00:52:05.034305 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3e43789d383004ef1b4e76f0aac8d103, offset: 0, size: 169 2024.10.22 00:52:05.042323 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3e43789d383004ef1b4e76f0aac8d103 for removal 2024.10.22 00:52:05.047806 [ 1476 ] {} CacheMetadata: Key 3e43789d383004ef1b4e76f0aac8d103 is removed from metadata 2024.10.22 00:52:05.049369 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2d7800619ce03244614da9f4fc13f264, offset: 0, size: 4604 2024.10.22 00:52:05.055785 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2d7800619ce03244614da9f4fc13f264 for removal 2024.10.22 00:52:05.063813 [ 1476 ] {} CacheMetadata: Key 2d7800619ce03244614da9f4fc13f264 is removed from metadata 2024.10.22 00:52:05.171122 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b95fc85d8fa5f5aae95387d14ade3cc0, offset: 0, size: 156 2024.10.22 00:52:05.174875 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b95fc85d8fa5f5aae95387d14ade3cc0 for removal 2024.10.22 00:52:05.176950 [ 1476 ] {} CacheMetadata: Key b95fc85d8fa5f5aae95387d14ade3cc0 is removed from metadata 2024.10.22 00:52:05.194812 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 211 2024.10.22 00:52:05.429093 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 11.07 MiB, will set to 1.65 GiB (RSS), difference: 4.20 MiB 2024.10.22 00:52:05.490774 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:05.498578 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:52:05.615416 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:05.680649 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_229_229_0 (state Temporary) 2024.10.22 00:52:05.723205 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:05.793355 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_193_193_0 (state Temporary) 2024.10.22 00:52:05.885382 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_193_193_0 to 202410_203_203_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:05.919254 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_229_229_0 to 202410_236_236_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:05.967217 [ 1155 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_231_46 to 202410_236_236_0 2024.10.22 00:52:05.971731 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 394536 2024.10.22 00:52:05.975640 [ 1155 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:05.977107 [ 1155 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:52:05.991147 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_318_64, 202410_319_319_0] 2024.10.22 00:52:05.993005 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_318_64, 202410_319_319_0] 2024.10.22 00:52:05.996481 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_231_46 to 202410_236_236_0 into Compact with storage Full 2024.10.22 00:52:06.006251 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_318_64, 202410_319_319_0] 2024.10.22 00:52:06.078275 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 211 2024.10.22 00:52:06.145393 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:52:06.163478 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} MergeTreeSequentialSource: Reading 50 marks from part 202410_1_231_46, total 400062 rows starting from the beginning of the part 2024.10.22 00:52:06.217777 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 124300 bytes for 512cea3cc4c17b93b7aee76ddeb9ec8a:0) 2024.10.22 00:52:06.222832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8ed5211a87b13f3380d3adb489d149b0, offset: 0, size: 75584 2024.10.22 00:52:06.228253 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8ed5211a87b13f3380d3adb489d149b0 for removal 2024.10.22 00:52:06.232565 [ 1476 ] {} CacheMetadata: Key 8ed5211a87b13f3380d3adb489d149b0 is removed from metadata 2024.10.22 00:52:06.235115 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 38b8001231574df5d64738d5d7f6076e, offset: 0, size: 79316 2024.10.22 00:52:06.251978 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 38b8001231574df5d64738d5d7f6076e for removal 2024.10.22 00:52:06.260206 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 234db92c541c5dda4365a1554e7a757d, offset: 0, size: 157 2024.10.22 00:52:06.265491 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 234db92c541c5dda4365a1554e7a757d for removal 2024.10.22 00:52:06.270493 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 186420ebd9979705c26a79ca0d665eb7, offset: 0, size: 157 2024.10.22 00:52:06.273423 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_232_232_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:52:06.273390 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 186420ebd9979705c26a79ca0d665eb7 for removal 2024.10.22 00:52:06.318236 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_233_233_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:52:06.331125 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_234_234_0, total 1450 rows starting from the beginning of the part 2024.10.22 00:52:06.352948 [ 1476 ] {} CacheMetadata: Key 186420ebd9979705c26a79ca0d665eb7 is removed from metadata 2024.10.22 00:52:06.359940 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_235_235_0, total 1622 rows starting from the beginning of the part 2024.10.22 00:52:06.368239 [ 1476 ] {} CacheMetadata: Key 234db92c541c5dda4365a1554e7a757d is removed from metadata 2024.10.22 00:52:06.377166 [ 1476 ] {} CacheMetadata: Key 38b8001231574df5d64738d5d7f6076e is removed from metadata 2024.10.22 00:52:06.489680 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_236_236_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:52:07.368053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 52311 bytes for cfd4e9eb240892cac6bf774a65895308:0) 2024.10.22 00:52:07.397067 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 512cea3cc4c17b93b7aee76ddeb9ec8a, offset: 0, size: 124300 2024.10.22 00:52:07.405532 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 512cea3cc4c17b93b7aee76ddeb9ec8a for removal 2024.10.22 00:52:07.410886 [ 1476 ] {} CacheMetadata: Key 512cea3cc4c17b93b7aee76ddeb9ec8a is removed from metadata 2024.10.22 00:52:07.412180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: febb9163c2b1e067ea6d8dd59282704b, offset: 0, size: 168 2024.10.22 00:52:07.417689 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key febb9163c2b1e067ea6d8dd59282704b for removal 2024.10.22 00:52:07.439600 [ 1476 ] {} CacheMetadata: Key febb9163c2b1e067ea6d8dd59282704b is removed from metadata 2024.10.22 00:52:07.598800 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 12.53 MiB, will set to 1.65 GiB (RSS), difference: -5.98 MiB 2024.10.22 00:52:08.232639 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6f2647fc92c195e0eb72bda5a75e4b0e) : [0, 307904]: Setting download as failed: File segment: [0, 307904], key: 6f2647fc92c195e0eb72bda5a75e4b0e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:08.335465 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6f2647fc92c195e0eb72bda5a75e4b0e): No space left in cache to reserve 307905 bytes, will continue without cache download 2024.10.22 00:52:08.378024 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6f2647fc92c195e0eb72bda5a75e4b0e, offset: 0, size: 0 2024.10.22 00:52:08.379539 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6f2647fc92c195e0eb72bda5a75e4b0e for removal 2024.10.22 00:52:08.381377 [ 1476 ] {} CacheMetadata: Key 6f2647fc92c195e0eb72bda5a75e4b0e is removed from metadata 2024.10.22 00:52:08.721315 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 23474 2024.10.22 00:52:08.974612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5868994137d86e3b0a4722b4fce82bdd) : [0, 233194]: Setting download as failed: File segment: [0, 233194], key: 5868994137d86e3b0a4722b4fce82bdd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:08.975989 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5868994137d86e3b0a4722b4fce82bdd): No space left in cache to reserve 233195 bytes, will continue without cache download 2024.10.22 00:52:08.993533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5868994137d86e3b0a4722b4fce82bdd, offset: 0, size: 0 2024.10.22 00:52:08.996565 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5868994137d86e3b0a4722b4fce82bdd for removal 2024.10.22 00:52:09.003084 [ 1476 ] {} CacheMetadata: Key 5868994137d86e3b0a4722b4fce82bdd is removed from metadata 2024.10.22 00:52:09.072468 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:09.075686 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:09.124760 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_411_411_0 (state Temporary) 2024.10.22 00:52:09.185159 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_411_411_0 to 202410_424_424_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:09.234506 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 3 parts from 202410_1_419_95 to 202410_421_421_0 2024.10.22 00:52:09.263990 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:09.278215 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:09.418465 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_421_96} MergeTask::PrepareStage: Merging 3 parts: from 202410_1_419_95 to 202410_421_421_0 into Compact with storage Full 2024.10.22 00:52:09.471332 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23474 2024.10.22 00:52:09.624605 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 16.48 MiB, will set to 1.65 GiB (RSS), difference: 3.75 MiB 2024.10.22 00:52:09.699047 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_421_96} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:52:09.732775 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:52:09.735888 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:52:09.860106 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_421_96} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_419_95, total 23928 rows starting from the beginning of the part 2024.10.22 00:52:10.079665 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_421_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_420_420_0, total 86 rows starting from the beginning of the part 2024.10.22 00:52:10.265524 [ 1269 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:52:10.341200 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_421_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_421_421_0, total 76 rows starting from the beginning of the part 2024.10.22 00:52:10.364740 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 141243 bytes for 0f18ec41df31d8ad37bfd2c08b581342:0) 2024.10.22 00:52:10.368592 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cfd4e9eb240892cac6bf774a65895308, offset: 0, size: 52311 2024.10.22 00:52:10.373648 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cfd4e9eb240892cac6bf774a65895308 for removal 2024.10.22 00:52:10.378527 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4390ea9bd0a49eca582adda8da041c38, offset: 0, size: 156 2024.10.22 00:52:10.377803 [ 1476 ] {} CacheMetadata: Key cfd4e9eb240892cac6bf774a65895308 is removed from metadata 2024.10.22 00:52:10.382163 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4390ea9bd0a49eca582adda8da041c38 for removal 2024.10.22 00:52:10.430627 [ 1269 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:52:10.535475 [ 1476 ] {} CacheMetadata: Key 4390ea9bd0a49eca582adda8da041c38 is removed from metadata 2024.10.22 00:52:10.789096 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.202029636 sec. 2024.10.22 00:52:11.387855 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b4a124e7ecde2306734535dc4fe5b861) : [0, 203275]: Setting download as failed: File segment: [0, 203275], key: b4a124e7ecde2306734535dc4fe5b861, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:11.387507 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_236_47/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:52:11.390748 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b4a124e7ecde2306734535dc4fe5b861): No space left in cache to reserve 203276 bytes, will continue without cache download 2024.10.22 00:52:11.538509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b4a124e7ecde2306734535dc4fe5b861, offset: 0, size: 0 2024.10.22 00:52:11.543724 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b4a124e7ecde2306734535dc4fe5b861 for removal 2024.10.22 00:52:11.566955 [ 1476 ] {} CacheMetadata: Key b4a124e7ecde2306734535dc4fe5b861 is removed from metadata 2024.10.22 00:52:11.768560 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 3387 2024.10.22 00:52:11.821136 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 76 entries to flush up to offset 15875 2024.10.22 00:52:11.881584 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_48_48_0 to 202410_51_51_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:12.029511 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:12.036721 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:12.094096 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_241_241_0 (state Temporary) 2024.10.22 00:52:12.158382 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_49_10/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:52:12.173450 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_241_241_0 to 202410_254_254_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:12.200177 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:12.210948 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:12.354106 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3387 2024.10.22 00:52:12.472397 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_239_239_0 (state Temporary) 2024.10.22 00:52:12.535349 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_239_239_0 to 202410_245_245_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:12.582488 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_240_57 to 202410_241_241_0 2024.10.22 00:52:12.588129 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:12.591778 [ 1176 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:52:12.591779 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15875 2024.10.22 00:52:12.614915 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_241_58} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_240_57 to 202410_241_241_0 into Compact with storage Full 2024.10.22 00:52:12.681062 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4dbf64350e7f1f1e9c05b2eb7d900f16) : [0, 257459]: Setting download as failed: File segment: [0, 257459], key: 4dbf64350e7f1f1e9c05b2eb7d900f16, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:12.684156 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4dbf64350e7f1f1e9c05b2eb7d900f16): No space left in cache to reserve 257460 bytes, will continue without cache download 2024.10.22 00:52:12.719981 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4dbf64350e7f1f1e9c05b2eb7d900f16, offset: 0, size: 0 2024.10.22 00:52:12.722956 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4dbf64350e7f1f1e9c05b2eb7d900f16 for removal 2024.10.22 00:52:12.726368 [ 1476 ] {} CacheMetadata: Key 4dbf64350e7f1f1e9c05b2eb7d900f16 is removed from metadata 2024.10.22 00:52:12.780483 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_241_58} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:52:12.791862 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_241_58} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_240_57, total 15648 rows starting from the beginning of the part 2024.10.22 00:52:12.809331 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 8.16 MiB, will set to 1.66 GiB (RSS), difference: 7.80 MiB 2024.10.22 00:52:12.899871 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_421_96} MergeTask::MergeProjectionsStage: Merge sorted 24090 rows, containing 13 columns (13 merged, 0 gathered) in 3.61151634 sec., 6670.328397295857 rows/sec., 1023.37 KiB/sec. 2024.10.22 00:52:12.948402 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_241_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_241_241_0, total 75 rows starting from the beginning of the part 2024.10.22 00:52:12.975495 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 395688 2024.10.22 00:52:13.057039 [ 1194 ] {} DatabaseCatalog: Execution took 261 ms. 2024.10.22 00:52:13.134861 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_319_65, 202410_320_320_0, 202410_321_321_0, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0] 2024.10.22 00:52:13.139379 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_319_65, 202410_320_320_0, 202410_321_321_0, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0] 2024.10.22 00:52:13.185134 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_319_65, 202410_320_320_0, 202410_321_321_0, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0] 2024.10.22 00:52:13.325793 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_421_96} MergedBlockOutputStream: filled checksums 202410_1_421_96 (state Temporary) 2024.10.22 00:52:13.474750 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 23546 2024.10.22 00:52:13.565201 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_421_96} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_421_96 to 202410_1_421_96 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:13.569595 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_421_96} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 3 parts: [202410_1_419_95, 202410_421_421_0] -> 202410_1_421_96 2024.10.22 00:52:13.578777 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(23447a1f241b5ae31138bec0de5446dd) : [0, 314194]: Setting download as failed: File segment: [0, 314194], key: 23447a1f241b5ae31138bec0de5446dd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:13.579944 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 212 2024.10.22 00:52:13.580180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(23447a1f241b5ae31138bec0de5446dd): No space left in cache to reserve 314195 bytes, will continue without cache download 2024.10.22 00:52:13.605090 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.76 MiB. 2024.10.22 00:52:13.661065 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 23447a1f241b5ae31138bec0de5446dd, offset: 0, size: 0 2024.10.22 00:52:13.665611 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:13.668056 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 23447a1f241b5ae31138bec0de5446dd for removal 2024.10.22 00:52:13.669377 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:13.671240 [ 1476 ] {} CacheMetadata: Key 23447a1f241b5ae31138bec0de5446dd is removed from metadata 2024.10.22 00:52:13.737693 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:13.746869 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:13.883041 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_230_230_0 (state Temporary) 2024.10.22 00:52:13.933439 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_412_412_0 (state Temporary) 2024.10.22 00:52:13.976224 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:13.999633 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_230_230_0 to 202410_237_237_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:14.055959 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:52:14.148072 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 395688 2024.10.22 00:52:14.152344 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_412_412_0 to 202410_425_425_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:14.234552 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23546 2024.10.22 00:52:14.273552 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_194_194_0 (state Temporary) 2024.10.22 00:52:14.279249 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_241_58} MergeTask::MergeProjectionsStage: Merge sorted 15723 rows, containing 13 columns (13 merged, 0 gathered) in 1.678902272 sec., 9365.047782840811 rows/sec., 3.94 MiB/sec. 2024.10.22 00:52:14.471061 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_194_194_0 to 202410_204_204_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:14.478079 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_241_58} MergedBlockOutputStream: filled checksums 202410_1_241_58 (state Temporary) 2024.10.22 00:52:14.498291 [ 1200 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_199_41 to 202410_204_204_0 2024.10.22 00:52:14.570490 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 212 2024.10.22 00:52:14.614390 [ 1200 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:14.635796 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:14.663828 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_199_41 to 202410_204_204_0 into Compact with storage Full 2024.10.22 00:52:14.819174 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_241_58} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_241_58 to 202410_1_241_58 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:14.826995 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_241_58} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_240_57, 202410_241_241_0] -> 202410_1_241_58 2024.10.22 00:52:14.992915 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:52:15.008121 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3c1db59852ba71c5e6f60d3b7f3151cb) : [0, 452453]: Setting download as failed: File segment: [0, 452453], key: 3c1db59852ba71c5e6f60d3b7f3151cb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:15.011867 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3c1db59852ba71c5e6f60d3b7f3151cb): No space left in cache to reserve 452454 bytes, will continue without cache download 2024.10.22 00:52:15.055059 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_199_41, total 243 rows starting from the beginning of the part 2024.10.22 00:52:15.067741 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.64 MiB. 2024.10.22 00:52:15.158985 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3c1db59852ba71c5e6f60d3b7f3151cb, offset: 0, size: 0 2024.10.22 00:52:15.162700 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3c1db59852ba71c5e6f60d3b7f3151cb for removal 2024.10.22 00:52:15.166494 [ 1476 ] {} CacheMetadata: Key 3c1db59852ba71c5e6f60d3b7f3151cb is removed from metadata 2024.10.22 00:52:15.388958 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1917 2024.10.22 00:52:15.440203 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_151_31, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:52:15.446967 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_151_31, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:52:15.491052 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 13.32 MiB, will set to 1.66 GiB (RSS), difference: 6.82 MiB 2024.10.22 00:52:15.569819 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_200_200_0, total 1 rows starting from the beginning of the part 2024.10.22 00:52:15.591421 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_151_31, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0, 202410_155_155_0, 202410_156_156_0] 2024.10.22 00:52:15.622859 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:52:15.774268 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_201_201_0, total 1 rows starting from the beginning of the part 2024.10.22 00:52:16.011327 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4a155024bead095e5a118a32cdddb491) : [0, 450881]: Setting download as failed: File segment: [0, 450881], key: 4a155024bead095e5a118a32cdddb491, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:16.014495 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4a155024bead095e5a118a32cdddb491): No space left in cache to reserve 450882 bytes, will continue without cache download 2024.10.22 00:52:16.059758 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_202_202_0, total 1 rows starting from the beginning of the part 2024.10.22 00:52:16.276835 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4a155024bead095e5a118a32cdddb491, offset: 0, size: 0 2024.10.22 00:52:16.279148 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4a155024bead095e5a118a32cdddb491 for removal 2024.10.22 00:52:16.282815 [ 1476 ] {} CacheMetadata: Key 4a155024bead095e5a118a32cdddb491 is removed from metadata 2024.10.22 00:52:16.531869 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_203_203_0, total 1 rows starting from the beginning of the part 2024.10.22 00:52:16.977370 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_204_204_0, total 1 rows starting from the beginning of the part 2024.10.22 00:52:17.601442 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(76ebe3ced553175568643dbcf6714e38) : [0, 306573]: Setting download as failed: File segment: [0, 306573], key: 76ebe3ced553175568643dbcf6714e38, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:17.605314 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(76ebe3ced553175568643dbcf6714e38): No space left in cache to reserve 306574 bytes, will continue without cache download 2024.10.22 00:52:17.859468 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 76ebe3ced553175568643dbcf6714e38, offset: 0, size: 0 2024.10.22 00:52:17.861788 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 76ebe3ced553175568643dbcf6714e38 for removal 2024.10.22 00:52:17.967545 [ 1476 ] {} CacheMetadata: Key 76ebe3ced553175568643dbcf6714e38 is removed from metadata 2024.10.22 00:52:18.257683 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 23608 2024.10.22 00:52:18.619480 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:18.623068 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:52:18.626482 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 13.59 MiB, will set to 1.65 GiB (RSS), difference: -10.98 MiB 2024.10.22 00:52:18.720794 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_413_413_0 (state Temporary) 2024.10.22 00:52:18.788118 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_182_37, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0, 202410_187_187_0] 2024.10.22 00:52:18.793076 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_182_37, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0, 202410_187_187_0] 2024.10.22 00:52:18.836243 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_182_37, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0, 202410_187_187_0] 2024.10.22 00:52:18.877344 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_413_413_0 to 202410_426_426_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:18.949339 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_421_96 to 202410_426_426_0 2024.10.22 00:52:18.968680 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:18.971703 [ 1214 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:52:19.005657 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8366c174642324e26061b03f46699982) : [0, 219188]: Setting download as failed: File segment: [0, 219188], key: 8366c174642324e26061b03f46699982, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:19.008570 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8366c174642324e26061b03f46699982): No space left in cache to reserve 219189 bytes, will continue without cache download 2024.10.22 00:52:19.055453 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23608 2024.10.22 00:52:19.114467 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_421_96 to 202410_426_426_0 into Compact with storage Full 2024.10.22 00:52:19.144484 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8366c174642324e26061b03f46699982, offset: 0, size: 0 2024.10.22 00:52:19.147899 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8366c174642324e26061b03f46699982 for removal 2024.10.22 00:52:19.158384 [ 1476 ] {} CacheMetadata: Key 8366c174642324e26061b03f46699982 is removed from metadata 2024.10.22 00:52:19.313736 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:52:19.331433 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_421_96, total 24090 rows starting from the beginning of the part 2024.10.22 00:52:19.375456 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_422_422_0, total 74 rows starting from the beginning of the part 2024.10.22 00:52:19.377066 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} MergeTask::MergeProjectionsStage: Merge sorted 248 rows, containing 32 columns (32 merged, 0 gathered) in 4.725452064 sec., 52.48175129938214 rows/sec., 5.52 KiB/sec. 2024.10.22 00:52:19.416680 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_423_423_0, total 57 rows starting from the beginning of the part 2024.10.22 00:52:19.470905 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_424_424_0, total 84 rows starting from the beginning of the part 2024.10.22 00:52:19.500847 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_425_425_0, total 72 rows starting from the beginning of the part 2024.10.22 00:52:19.518456 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} MergedBlockOutputStream: filled checksums 202410_1_204_42 (state Temporary) 2024.10.22 00:52:19.575157 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_426_426_0, total 62 rows starting from the beginning of the part 2024.10.22 00:52:19.736684 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:52:19.744566 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:52:19.862729 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 3401 2024.10.22 00:52:20.121837 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 101 entries to flush up to offset 15976 2024.10.22 00:52:20.132862 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:20.140806 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:52:20.194939 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_242_242_0 (state Temporary) 2024.10.22 00:52:20.223164 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e5be7d72b4cc025f911e3fbe5989e480) : [0, 429868]: Setting download as failed: File segment: [0, 429868], key: e5be7d72b4cc025f911e3fbe5989e480, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:20.225213 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e5be7d72b4cc025f911e3fbe5989e480): No space left in cache to reserve 429869 bytes, will continue without cache download 2024.10.22 00:52:20.273782 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} MergeTask::MergeProjectionsStage: Merge sorted 1896 rows, containing 679 columns (679 merged, 0 gathered) in 94.187604249 sec., 20.130037440888938 rows/sec., 26.77 KiB/sec. 2024.10.22 00:52:20.292446 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_204_42 to 202410_1_204_42 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:20.297415 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_204_42} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_199_41, 202410_204_204_0] -> 202410_1_204_42 2024.10.22 00:52:20.372243 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:52:20.397148 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 11.79 MiB, will set to 1.64 GiB (RSS), difference: -11.59 MiB 2024.10.22 00:52:20.460578 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_242_242_0 to 202410_255_255_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:20.484492 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e5be7d72b4cc025f911e3fbe5989e480, offset: 0, size: 0 2024.10.22 00:52:20.488128 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e5be7d72b4cc025f911e3fbe5989e480 for removal 2024.10.22 00:52:20.491301 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:20.497130 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:52:20.511495 [ 1476 ] {} CacheMetadata: Key e5be7d72b4cc025f911e3fbe5989e480 is removed from metadata 2024.10.22 00:52:20.620455 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3401 2024.10.22 00:52:20.662363 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_240_240_0 (state Temporary) 2024.10.22 00:52:20.796329 [ 1718 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jwa/aeadmywzhbibkxbpudwvdxmiouvxh, size 136 2024.10.22 00:52:20.840009 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_240_240_0 to 202410_246_246_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:20.925534 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 15976 2024.10.22 00:52:20.935961 [ 1226 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_241_58 to 202410_246_246_0 2024.10.22 00:52:21.159620 [ 1226 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:21.163018 [ 1226 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:52:21.164881 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1495 entries to flush up to offset 397183 2024.10.22 00:52:21.169628 [ 1226 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 294 ms. 2024.10.22 00:52:21.196236 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_241_58 to 202410_246_246_0 into Compact with storage Full 2024.10.22 00:52:21.297830 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:52:21.417365 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_241_58, total 15723 rows starting from the beginning of the part 2024.10.22 00:52:21.461768 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column PageCharset 2024.10.22 00:52:21.537071 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e4c4f4bab593fbef0bfbf8d8dd770121) : [0, 338556]: Setting download as failed: File segment: [0, 338556], key: e4c4f4bab593fbef0bfbf8d8dd770121, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:21.543623 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e4c4f4bab593fbef0bfbf8d8dd770121): No space left in cache to reserve 338557 bytes, will continue without cache download 2024.10.22 00:52:21.588575 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column PageCharset 2024.10.22 00:52:21.605222 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column PageCharset 2024.10.22 00:52:21.606825 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_242_242_0, total 44 rows starting from the beginning of the part 2024.10.22 00:52:21.651569 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column PageCharset 2024.10.22 00:52:21.663999 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_243_243_0, total 89 rows starting from the beginning of the part 2024.10.22 00:52:21.695511 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column PageCharset 2024.10.22 00:52:21.695938 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e4c4f4bab593fbef0bfbf8d8dd770121, offset: 0, size: 0 2024.10.22 00:52:21.704564 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e4c4f4bab593fbef0bfbf8d8dd770121 for removal 2024.10.22 00:52:21.711591 [ 1476 ] {} CacheMetadata: Key e4c4f4bab593fbef0bfbf8d8dd770121 is removed from metadata 2024.10.22 00:52:21.737181 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_244_244_0, total 47 rows starting from the beginning of the part 2024.10.22 00:52:21.754797 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column PageCharset 2024.10.22 00:52:21.820526 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_245_245_0, total 76 rows starting from the beginning of the part 2024.10.22 00:52:21.868211 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} MergeTask::MergeProjectionsStage: Merge sorted 24439 rows, containing 13 columns (13 merged, 0 gathered) in 2.766503736 sec., 8833.893727299126 rows/sec., 1.32 MiB/sec. 2024.10.22 00:52:21.874468 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column PageCharset 2024.10.22 00:52:21.880802 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_246_246_0, total 101 rows starting from the beginning of the part 2024.10.22 00:52:21.940690 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mgf/kvtrmpbyoiivlqxhifcbzfovpslrp (LogSeriesLimiter: on interval from 2024-10-22 00:52:00 to 2024-10-22 00:52:21 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:52:22.067845 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} MergedBlockOutputStream: filled checksums 202410_1_426_97 (state Temporary) 2024.10.22 00:52:22.072404 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 213 2024.10.22 00:52:22.230270 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_426_97 to 202410_1_426_97 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:22.235789 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_426_97} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_421_96, 202410_426_426_0] -> 202410_1_426_97 2024.10.22 00:52:22.274798 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_236_47/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:52:22.382379 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:22.388820 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:22.419319 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.93 MiB. 2024.10.22 00:52:22.571261 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} MergedBlockOutputStream: filled checksums 202410_1_49_10 (state Temporary) 2024.10.22 00:52:22.641211 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8e5ebb0f4b0e2c8905719a541601923b) : [0, 212436]: Setting download as failed: File segment: [0, 212436], key: 8e5ebb0f4b0e2c8905719a541601923b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:22.655127 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8e5ebb0f4b0e2c8905719a541601923b): No space left in cache to reserve 212437 bytes, will continue without cache download 2024.10.22 00:52:22.658745 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_231_231_0 (state Temporary) 2024.10.22 00:52:22.698594 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:22.707124 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:22.733306 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_231_231_0 to 202410_238_238_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:22.782638 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_195_195_0 (state Temporary) 2024.10.22 00:52:22.784600 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 397183 2024.10.22 00:52:22.793167 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8e5ebb0f4b0e2c8905719a541601923b, offset: 0, size: 0 2024.10.22 00:52:22.794129 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8e5ebb0f4b0e2c8905719a541601923b for removal 2024.10.22 00:52:22.796271 [ 1476 ] {} CacheMetadata: Key 8e5ebb0f4b0e2c8905719a541601923b is removed from metadata 2024.10.22 00:52:22.880369 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_195_195_0 to 202410_205_205_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:23.064586 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 101 entries to flush up to offset 23709 2024.10.22 00:52:23.067842 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 45 entries to flush up to offset 1962 2024.10.22 00:52:23.149641 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_191_38, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0] 2024.10.22 00:52:23.152006 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_191_38, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0] 2024.10.22 00:52:23.171605 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 213 2024.10.22 00:52:23.183006 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_191_38, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0] 2024.10.22 00:52:23.200211 [ 1253 ] {} DatabaseCatalog: Execution took 233 ms. 2024.10.22 00:52:23.231071 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} MergeTask::MergeProjectionsStage: Merge sorted 408126 rows, containing 4 columns (4 merged, 0 gathered) in 17.240504672 sec., 23672.50888327128 rows/sec., 347.22 KiB/sec. 2024.10.22 00:52:23.370251 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_246_59/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:52:23.467895 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:23.471565 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.35 GiB. 2024.10.22 00:52:23.530147 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} MergedBlockOutputStream: filled checksums 202410_1_236_47 (state Temporary) 2024.10.22 00:52:23.570709 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_414_414_0 (state Temporary) 2024.10.22 00:52:23.661640 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_414_414_0 to 202410_427_427_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:23.675588 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_236_47 to 202410_1_236_47 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:23.679100 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_236_47} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_231_46, 202410_236_236_0] -> 202410_1_236_47 2024.10.22 00:52:23.713057 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23709 2024.10.22 00:52:23.781575 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} MergeTask::MergeProjectionsStage: Merge sorted 16080 rows, containing 13 columns (13 merged, 0 gathered) in 2.597805119 sec., 6189.8407553334255 rows/sec., 2.60 MiB/sec. 2024.10.22 00:52:23.843727 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:52:23.877216 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} MergedBlockOutputStream: filled checksums 202410_1_246_59 (state Temporary) 2024.10.22 00:52:23.969358 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6f5593d1e7773d3bbe57785c838a7753) : [0, 211289]: Setting download as failed: File segment: [0, 211289], key: 6f5593d1e7773d3bbe57785c838a7753, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:23.972542 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6f5593d1e7773d3bbe57785c838a7753): No space left in cache to reserve 211290 bytes, will continue without cache download 2024.10.22 00:52:24.091023 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6f5593d1e7773d3bbe57785c838a7753, offset: 0, size: 0 2024.10.22 00:52:24.094148 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6f5593d1e7773d3bbe57785c838a7753 for removal 2024.10.22 00:52:24.097125 [ 1476 ] {} CacheMetadata: Key 6f5593d1e7773d3bbe57785c838a7753 is removed from metadata 2024.10.22 00:52:24.120183 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_246_59 to 202410_1_246_59 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:24.125036 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_246_59} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_241_58, 202410_246_246_0] -> 202410_1_246_59 2024.10.22 00:52:24.186189 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.89 MiB. 2024.10.22 00:52:24.662604 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 13.00 MiB, will set to 1.64 GiB (RSS), difference: 10.19 MiB 2024.10.22 00:52:25.358600 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2618efb7d89052e83e3e1585ddd63b73) : [0, 176850]: Setting download as failed: File segment: [0, 176850], key: 2618efb7d89052e83e3e1585ddd63b73, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:25.360790 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2618efb7d89052e83e3e1585ddd63b73): No space left in cache to reserve 176851 bytes, will continue without cache download 2024.10.22 00:52:25.444938 [ 1231 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:52:25.462064 [ 1231 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:52:25.586725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2618efb7d89052e83e3e1585ddd63b73, offset: 0, size: 0 2024.10.22 00:52:25.589216 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2618efb7d89052e83e3e1585ddd63b73 for removal 2024.10.22 00:52:25.591537 [ 1476 ] {} CacheMetadata: Key 2618efb7d89052e83e3e1585ddd63b73 is removed from metadata 2024.10.22 00:52:26.230420 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_49_10/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:52:26.386725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(71c8cb3f8689a2d735b112b9dbc4cc3a) : [0, 363125]: Setting download as failed: File segment: [0, 363125], key: 71c8cb3f8689a2d735b112b9dbc4cc3a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:26.387316 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(71c8cb3f8689a2d735b112b9dbc4cc3a): No space left in cache to reserve 363126 bytes, will continue without cache download 2024.10.22 00:52:26.417812 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 71c8cb3f8689a2d735b112b9dbc4cc3a, offset: 0, size: 0 2024.10.22 00:52:26.423452 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 71c8cb3f8689a2d735b112b9dbc4cc3a for removal 2024.10.22 00:52:26.428184 [ 1476 ] {} CacheMetadata: Key 71c8cb3f8689a2d735b112b9dbc4cc3a is removed from metadata 2024.10.22 00:52:27.663749 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 10.09 MiB, will set to 1.63 GiB (RSS), difference: -2.13 MiB 2024.10.22 00:52:28.079953 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ae04d55837e6863b897ec5c5629f7376) : [0, 222947]: Setting download as failed: File segment: [0, 222947], key: ae04d55837e6863b897ec5c5629f7376, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:28.126186 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 33 entries to flush up to offset 3434 2024.10.22 00:52:28.185652 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 23752 2024.10.22 00:52:28.186748 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ae04d55837e6863b897ec5c5629f7376): No space left in cache to reserve 222948 bytes, will continue without cache download 2024.10.22 00:52:28.247791 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_176_35, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0] 2024.10.22 00:52:28.251386 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_176_35, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0] 2024.10.22 00:52:28.332881 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ae04d55837e6863b897ec5c5629f7376, offset: 0, size: 0 2024.10.22 00:52:28.335185 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ae04d55837e6863b897ec5c5629f7376 for removal 2024.10.22 00:52:28.341653 [ 1476 ] {} CacheMetadata: Key ae04d55837e6863b897ec5c5629f7376 is removed from metadata 2024.10.22 00:52:28.388512 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_176_35, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0, 202410_180_180_0, 202410_181_181_0] 2024.10.22 00:52:28.444043 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 16060 2024.10.22 00:52:28.504519 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:28.510690 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:28.556089 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_243_243_0 (state Temporary) 2024.10.22 00:52:28.619231 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_243_243_0 to 202410_256_256_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:28.777584 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:28.780374 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:28.784734 [ 1150 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_251_50 to 202410_256_256_0 2024.10.22 00:52:28.873266 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:28.877473 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:28.917296 [ 1150 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:28.919701 [ 1150 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:28.928452 [ 1150 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 273 ms. 2024.10.22 00:52:28.979130 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3434 2024.10.22 00:52:29.070423 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_251_50 to 202410_256_256_0 into Compact with storage Full 2024.10.22 00:52:29.073369 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_241_241_0 (state Temporary) 2024.10.22 00:52:29.156199 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_415_415_0 (state Temporary) 2024.10.22 00:52:29.200094 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:52:29.227023 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_241_241_0 to 202410_247_247_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:29.275165 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16060 2024.10.22 00:52:29.311165 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_251_50, total 3464 rows starting from the beginning of the part 2024.10.22 00:52:29.312025 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_415_415_0 to 202410_428_428_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:29.329722 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_252_252_0, total 18 rows starting from the beginning of the part 2024.10.22 00:52:29.364115 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_253_253_0, total 5 rows starting from the beginning of the part 2024.10.22 00:52:29.379003 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23752 2024.10.22 00:52:29.385758 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_254_254_0, total 6 rows starting from the beginning of the part 2024.10.22 00:52:29.400954 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_255_255_0, total 14 rows starting from the beginning of the part 2024.10.22 00:52:29.447342 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_256_256_0, total 33 rows starting from the beginning of the part 2024.10.22 00:52:29.584147 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 5.75 MiB, will set to 1.63 GiB (RSS), difference: -8.66 MiB 2024.10.22 00:52:29.646889 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(036b8de517704044c4edac91f8fc3ec8) : [0, 858819]: Setting download as failed: File segment: [0, 858819], key: 036b8de517704044c4edac91f8fc3ec8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:29.652647 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(036b8de517704044c4edac91f8fc3ec8): No space left in cache to reserve 858820 bytes, will continue without cache download 2024.10.22 00:52:29.734702 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:52:29.737567 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:52:29.792530 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1193 entries to flush up to offset 398376 2024.10.22 00:52:29.811601 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 036b8de517704044c4edac91f8fc3ec8, offset: 0, size: 0 2024.10.22 00:52:29.814216 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 036b8de517704044c4edac91f8fc3ec8 for removal 2024.10.22 00:52:29.821381 [ 1476 ] {} CacheMetadata: Key 036b8de517704044c4edac91f8fc3ec8 is removed from metadata 2024.10.22 00:52:30.578785 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} MergeTask::MergeProjectionsStage: Merge sorted 3540 rows, containing 12 columns (12 merged, 0 gathered) in 1.522483071 sec., 2325.1490065336825 rows/sec., 363.21 KiB/sec. 2024.10.22 00:52:30.625439 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} MergedBlockOutputStream: filled checksums 202410_1_256_51 (state Temporary) 2024.10.22 00:52:30.674097 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 214 2024.10.22 00:52:30.890376 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f773fffa1c9bbea322781815a58ed3e2) : [0, 1175704]: Setting download as failed: File segment: [0, 1175704], key: f773fffa1c9bbea322781815a58ed3e2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:30.892551 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f773fffa1c9bbea322781815a58ed3e2): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:52:30.970650 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_256_51 to 202410_1_256_51 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:30.972175 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f773fffa1c9bbea322781815a58ed3e2, offset: 0, size: 0 2024.10.22 00:52:30.976136 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_256_51} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_251_50, 202410_256_256_0] -> 202410_1_256_51 2024.10.22 00:52:30.977066 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f773fffa1c9bbea322781815a58ed3e2 for removal 2024.10.22 00:52:30.979518 [ 1476 ] {} CacheMetadata: Key f773fffa1c9bbea322781815a58ed3e2 is removed from metadata 2024.10.22 00:52:31.082834 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:31.097244 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:31.176415 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:31.179392 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:31.213394 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:52:31.283557 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_196_196_0 (state Temporary) 2024.10.22 00:52:31.393313 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 6.21 MiB, will set to 1.63 GiB (RSS), difference: -5.07 MiB 2024.10.22 00:52:31.422530 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_232_232_0 (state Temporary) 2024.10.22 00:52:31.487767 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_196_196_0 to 202410_206_206_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:31.540890 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_232_232_0 to 202410_239_239_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:31.677429 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 214 2024.10.22 00:52:31.791896 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 398376 2024.10.22 00:52:31.991043 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(59255020ce25b8a4b697ebbcf46d5965) : [0, 518002]: Setting download as failed: File segment: [0, 518002], key: 59255020ce25b8a4b697ebbcf46d5965, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:52:31.994861 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(59255020ce25b8a4b697ebbcf46d5965): No space left in cache to reserve 518003 bytes, will continue without cache download 2024.10.22 00:52:32.137404 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 59255020ce25b8a4b697ebbcf46d5965, offset: 0, size: 0 2024.10.22 00:52:32.144254 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 59255020ce25b8a4b697ebbcf46d5965 for removal 2024.10.22 00:52:32.154547 [ 1476 ] {} CacheMetadata: Key 59255020ce25b8a4b697ebbcf46d5965 is removed from metadata 2024.10.22 00:52:33.263263 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_49_10 to 202410_1_49_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:33.267564 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_49_10} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_44_9, 202410_49_49_0] -> 202410_1_49_10 2024.10.22 00:52:33.386933 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 23828 2024.10.22 00:52:33.392542 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 93558 bytes for ae0f3d226f107a31c91ec01f99b74639:0) 2024.10.22 00:52:33.406081 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0f18ec41df31d8ad37bfd2c08b581342, offset: 0, size: 141243 2024.10.22 00:52:33.409539 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0f18ec41df31d8ad37bfd2c08b581342 for removal 2024.10.22 00:52:33.415559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 644b50c5a1916be168b740cc1b3235d7, offset: 0, size: 168 2024.10.22 00:52:33.416213 [ 1476 ] {} CacheMetadata: Key 0f18ec41df31d8ad37bfd2c08b581342 is removed from metadata 2024.10.22 00:52:33.423183 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 644b50c5a1916be168b740cc1b3235d7 for removal 2024.10.22 00:52:33.429606 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cb235a660f09aac87653f30b77cbc07c, offset: 0, size: 5128 2024.10.22 00:52:33.437543 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cb235a660f09aac87653f30b77cbc07c for removal 2024.10.22 00:52:33.443105 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 6.29 MiB, will set to 1.63 GiB (RSS), difference: 10.68 MiB 2024.10.22 00:52:33.447110 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 57ece72b00843c618b30641cc6c8571e, offset: 0, size: 156 2024.10.22 00:52:33.560534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 57ece72b00843c618b30641cc6c8571e for removal 2024.10.22 00:52:33.566365 [ 1476 ] {} CacheMetadata: Key 57ece72b00843c618b30641cc6c8571e is removed from metadata 2024.10.22 00:52:33.579405 [ 1476 ] {} CacheMetadata: Key cb235a660f09aac87653f30b77cbc07c is removed from metadata 2024.10.22 00:52:33.587538 [ 1476 ] {} CacheMetadata: Key 644b50c5a1916be168b740cc1b3235d7 is removed from metadata 2024.10.22 00:52:33.666952 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1c23bf1c14dfa1886cfc56a9e3dae981, offset: 0, size: 151 2024.10.22 00:52:33.772151 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1c23bf1c14dfa1886cfc56a9e3dae981 for removal 2024.10.22 00:52:33.775887 [ 1476 ] {} CacheMetadata: Key 1c23bf1c14dfa1886cfc56a9e3dae981 is removed from metadata 2024.10.22 00:52:33.776059 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 92cf673f006e0471e1ea9b135f80175e, offset: 0, size: 157 2024.10.22 00:52:33.779240 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 92cf673f006e0471e1ea9b135f80175e for removal 2024.10.22 00:52:33.783257 [ 1476 ] {} CacheMetadata: Key 92cf673f006e0471e1ea9b135f80175e is removed from metadata 2024.10.22 00:52:33.800432 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:33.808284 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:33.907523 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gsk/emfiqjgqrzbhajzimijqqxuxubzbe. 2024.10.22 00:52:33.970391 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gcc/zqugthxmjnzhrpzgnvcvsytaqqnmy. 2024.10.22 00:52:34.116821 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_416_416_0 (state Temporary) 2024.10.22 00:52:34.380268 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 30.69 MiB. 2024.10.22 00:52:34.388754 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_416_416_0 to 202410_429_429_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:34.572264 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23828 2024.10.22 00:52:35.064603 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gcc/zqugthxmjnzhrpzgnvcvsytaqqnmy. 2024.10.22 00:52:35.068202 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_15_15_0 to 201403_15_15_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:35.085478 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gsk/emfiqjgqrzbhajzimijqqxuxubzbe. 2024.10.22 00:52:35.157442 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:52:35.607088 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.88 GiB, free memory in arenas 4.64 MiB, will set to 1.63 GiB (RSS), difference: 303.45 MiB 2024.10.22 00:52:35.883585 [ 1201 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 00:52:35.963168 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 15 2024.10.22 00:52:36.481310 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 3455 2024.10.22 00:52:36.519916 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:36.524781 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:36.724012 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_15_15_0 (state Temporary) 2024.10.22 00:52:36.812711 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 47 entries to flush up to offset 16107 2024.10.22 00:52:36.827857 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:36.944463 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:36.989554 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_244_244_0 (state Temporary) 2024.10.22 00:52:37.048522 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_15_15_0 to 202410_15_15_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:37.050907 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_324_66, 202410_325_325_0, 202410_326_326_0, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0] 2024.10.22 00:52:37.058821 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_324_66, 202410_325_325_0, 202410_326_326_0, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0] 2024.10.22 00:52:37.060124 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_244_244_0 to 202410_257_257_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:37.079951 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_324_66, 202410_325_325_0, 202410_326_326_0, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0] 2024.10.22 00:52:37.148034 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 15 2024.10.22 00:52:37.210674 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:37.214453 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:37.222351 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3455 2024.10.22 00:52:37.364121 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_242_242_0 (state Temporary) 2024.10.22 00:52:37.395682 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.09 GiB, peak 1.88 GiB, free memory in arenas 5.10 MiB, will set to 1.16 GiB (RSS), difference: 67.34 MiB 2024.10.22 00:52:37.397025 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_242_242_0 to 202410_248_248_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:37.638635 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16107 2024.10.22 00:52:38.770907 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 23885 2024.10.22 00:52:38.796361 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 400296 2024.10.22 00:52:39.233247 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:39.237061 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:39.317282 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_417_417_0 (state Temporary) 2024.10.22 00:52:39.477385 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_417_417_0 to 202410_430_430_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:39.596202 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23885 2024.10.22 00:52:39.748076 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:52:39.758717 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:52:40.027082 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 1.88 GiB, free memory in arenas 5.08 MiB, will set to 1.20 GiB (RSS), difference: -49.27 MiB 2024.10.22 00:52:40.079318 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:40.084166 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:40.229032 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_233_233_0 (state Temporary) 2024.10.22 00:52:40.372942 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_233_233_0 to 202410_240_240_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:40.427766 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 400296 2024.10.22 00:52:40.468114 [ 1224 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:52:40.588821 [ 1224 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:52:40.626296 [ 1223 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 00:52:41.181837 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 00:52:41.286674 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:41.289087 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:41.366912 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.26 GiB, peak 1.88 GiB, free memory in arenas 5.93 MiB, will set to 1.22 GiB (RSS), difference: -47.43 MiB 2024.10.22 00:52:42.570775 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.23 GiB, peak 1.88 GiB, free memory in arenas 5.93 MiB, will set to 1.23 GiB (RSS), difference: -4.29 MiB 2024.10.22 00:52:43.217765 [ 1241 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 00:52:43.387230 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 22 file segments (while reserving 97679 bytes for 3b5191fef1d3c47a60f394896e6c41d2:0) 2024.10.22 00:52:43.389291 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5989db2fead7f8171579fe9cbdf69600, offset: 0, size: 1300 2024.10.22 00:52:43.392629 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5989db2fead7f8171579fe9cbdf69600 for removal 2024.10.22 00:52:43.396270 [ 1476 ] {} CacheMetadata: Key 5989db2fead7f8171579fe9cbdf69600 is removed from metadata 2024.10.22 00:52:43.396956 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 988aa57bdf137e7ed96d4b842bb63668, offset: 0, size: 156 2024.10.22 00:52:43.403843 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 988aa57bdf137e7ed96d4b842bb63668 for removal 2024.10.22 00:52:43.411984 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ae0f3d226f107a31c91ec01f99b74639, offset: 0, size: 93558 2024.10.22 00:52:43.417602 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ae0f3d226f107a31c91ec01f99b74639 for removal 2024.10.22 00:52:43.428500 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8407402ff807f7666b5350b627aa4746, offset: 0, size: 156 2024.10.22 00:52:43.430288 [ 1476 ] {} CacheMetadata: Key ae0f3d226f107a31c91ec01f99b74639 is removed from metadata 2024.10.22 00:52:43.436721 [ 1476 ] {} CacheMetadata: Key 988aa57bdf137e7ed96d4b842bb63668 is removed from metadata 2024.10.22 00:52:43.546868 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8407402ff807f7666b5350b627aa4746 for removal 2024.10.22 00:52:43.553478 [ 1476 ] {} CacheMetadata: Key 8407402ff807f7666b5350b627aa4746 is removed from metadata 2024.10.22 00:52:43.554004 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2fa669d982486d41f221a1285cda8ca4, offset: 0, size: 162 2024.10.22 00:52:43.562788 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2fa669d982486d41f221a1285cda8ca4 for removal 2024.10.22 00:52:43.566701 [ 1227 ] {} DatabaseCatalog: Execution took 235 ms. 2024.10.22 00:52:43.620124 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 23925 2024.10.22 00:52:43.668178 [ 1476 ] {} CacheMetadata: Key 2fa669d982486d41f221a1285cda8ca4 is removed from metadata 2024.10.22 00:52:43.668924 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8a41d87015474d10549d85d9bd4406a1, offset: 0, size: 158 2024.10.22 00:52:43.672005 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8a41d87015474d10549d85d9bd4406a1 for removal 2024.10.22 00:52:43.676786 [ 1476 ] {} CacheMetadata: Key 8a41d87015474d10549d85d9bd4406a1 is removed from metadata 2024.10.22 00:52:43.679310 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 380c10de10921cc449e47fec1f913772, offset: 0, size: 155 2024.10.22 00:52:43.683996 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 380c10de10921cc449e47fec1f913772 for removal 2024.10.22 00:52:43.689181 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a097f043cff5bf25d5f27dcb9c0d6f5b, offset: 0, size: 156 2024.10.22 00:52:43.688912 [ 1476 ] {} CacheMetadata: Key 380c10de10921cc449e47fec1f913772 is removed from metadata 2024.10.22 00:52:43.693475 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a097f043cff5bf25d5f27dcb9c0d6f5b for removal 2024.10.22 00:52:43.705786 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 672fc26bb9fde57f9737008773f85eb6, offset: 0, size: 160 2024.10.22 00:52:43.716807 [ 1476 ] {} CacheMetadata: Key a097f043cff5bf25d5f27dcb9c0d6f5b is removed from metadata 2024.10.22 00:52:43.719167 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 672fc26bb9fde57f9737008773f85eb6 for removal 2024.10.22 00:52:43.726546 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 094566fc72cc4c1b06b499cae4841251, offset: 0, size: 158 2024.10.22 00:52:43.733839 [ 1476 ] {} CacheMetadata: Key 672fc26bb9fde57f9737008773f85eb6 is removed from metadata 2024.10.22 00:52:43.735499 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 094566fc72cc4c1b06b499cae4841251 for removal 2024.10.22 00:52:43.742503 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bd0dcf92903a5928ed65c43dcebfd33a, offset: 0, size: 158 2024.10.22 00:52:43.741292 [ 1476 ] {} CacheMetadata: Key 094566fc72cc4c1b06b499cae4841251 is removed from metadata 2024.10.22 00:52:43.759255 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bd0dcf92903a5928ed65c43dcebfd33a for removal 2024.10.22 00:52:43.768942 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ed4c44dbb980052aca02ee0d658aa595, offset: 0, size: 157 2024.10.22 00:52:43.773079 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ed4c44dbb980052aca02ee0d658aa595 for removal 2024.10.22 00:52:43.785084 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6ff00facd4e24cd16b756f583515fb85, offset: 0, size: 160 2024.10.22 00:52:43.788112 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:43.793221 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:43.796001 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6ff00facd4e24cd16b756f583515fb85 for removal 2024.10.22 00:52:43.802646 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c3e12029971e456bf6937a9b5f45b3c3, offset: 0, size: 160 2024.10.22 00:52:43.812040 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c3e12029971e456bf6937a9b5f45b3c3 for removal 2024.10.22 00:52:43.820144 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1f3b0f7cf6f69c184af5806d6573134c, offset: 0, size: 4 2024.10.22 00:52:43.826975 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1f3b0f7cf6f69c184af5806d6573134c for removal 2024.10.22 00:52:43.834616 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 755ff74aa0a9e76bc3c869985d3eecb9, offset: 0, size: 155 2024.10.22 00:52:43.838598 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 755ff74aa0a9e76bc3c869985d3eecb9 for removal 2024.10.22 00:52:43.844120 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3cb49bc86cfb6136c9d8eab94728d451, offset: 0, size: 156 2024.10.22 00:52:43.850147 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3cb49bc86cfb6136c9d8eab94728d451 for removal 2024.10.22 00:52:43.854662 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bea444b9be6b2aca8b6a9c301ec778b5, offset: 0, size: 190 2024.10.22 00:52:43.860509 [ 1476 ] {} CacheMetadata: Key 3cb49bc86cfb6136c9d8eab94728d451 is removed from metadata 2024.10.22 00:52:43.862573 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bea444b9be6b2aca8b6a9c301ec778b5 for removal 2024.10.22 00:52:43.869033 [ 1476 ] {} CacheMetadata: Key bea444b9be6b2aca8b6a9c301ec778b5 is removed from metadata 2024.10.22 00:52:43.876389 [ 1476 ] {} CacheMetadata: Key 755ff74aa0a9e76bc3c869985d3eecb9 is removed from metadata 2024.10.22 00:52:43.881657 [ 1476 ] {} CacheMetadata: Key 1f3b0f7cf6f69c184af5806d6573134c is removed from metadata 2024.10.22 00:52:43.887428 [ 1476 ] {} CacheMetadata: Key c3e12029971e456bf6937a9b5f45b3c3 is removed from metadata 2024.10.22 00:52:43.893248 [ 1476 ] {} CacheMetadata: Key 6ff00facd4e24cd16b756f583515fb85 is removed from metadata 2024.10.22 00:52:43.903080 [ 1476 ] {} CacheMetadata: Key ed4c44dbb980052aca02ee0d658aa595 is removed from metadata 2024.10.22 00:52:43.912495 [ 1476 ] {} CacheMetadata: Key bd0dcf92903a5928ed65c43dcebfd33a is removed from metadata 2024.10.22 00:52:43.968730 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 811a8f9ab9e83f72d8d780fb53811919, offset: 0, size: 156 2024.10.22 00:52:44.067441 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_418_418_0 (state Temporary) 2024.10.22 00:52:44.074808 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 811a8f9ab9e83f72d8d780fb53811919 for removal 2024.10.22 00:52:44.078933 [ 1476 ] {} CacheMetadata: Key 811a8f9ab9e83f72d8d780fb53811919 is removed from metadata 2024.10.22 00:52:44.079369 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a4f4c3144e25a034fdc9a915b4cafa06, offset: 0, size: 158 2024.10.22 00:52:44.087327 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a4f4c3144e25a034fdc9a915b4cafa06 for removal 2024.10.22 00:52:44.089990 [ 1476 ] {} CacheMetadata: Key a4f4c3144e25a034fdc9a915b4cafa06 is removed from metadata 2024.10.22 00:52:44.094117 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3f27cf82bde9e311e6ac28defae6e4ac, offset: 0, size: 193 2024.10.22 00:52:44.101563 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3f27cf82bde9e311e6ac28defae6e4ac for removal 2024.10.22 00:52:44.105231 [ 1476 ] {} CacheMetadata: Key 3f27cf82bde9e311e6ac28defae6e4ac is removed from metadata 2024.10.22 00:52:44.105587 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1b48b7ab1088d129042952b2e5f7bf32, offset: 0, size: 156 2024.10.22 00:52:44.113643 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1b48b7ab1088d129042952b2e5f7bf32 for removal 2024.10.22 00:52:44.133815 [ 1476 ] {} CacheMetadata: Key 1b48b7ab1088d129042952b2e5f7bf32 is removed from metadata 2024.10.22 00:52:44.215979 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_418_418_0 to 202410_431_431_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:44.249684 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1_426_97 to 202410_429_429_0 2024.10.22 00:52:44.256200 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 23925 2024.10.22 00:52:44.265736 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:44.268947 [ 1255 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:44.295024 [ 1743 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mgf/kvtrmpbyoiivlqxhifcbzfovpslrp, size 97679 2024.10.22 00:52:44.300339 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_429_98} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_426_97 to 202410_429_429_0 into Compact with storage Full 2024.10.22 00:52:44.373299 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_429_98} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:52:44.377150 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_429_98} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_426_97, total 24439 rows starting from the beginning of the part 2024.10.22 00:52:44.387573 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_429_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_427_427_0, total 101 rows starting from the beginning of the part 2024.10.22 00:52:44.646551 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_429_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_428_428_0, total 43 rows starting from the beginning of the part 2024.10.22 00:52:44.657562 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column CodeVersion 2024.10.22 00:52:44.726960 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 3461 2024.10.22 00:52:44.785697 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column CodeVersion 2024.10.22 00:52:44.805650 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column CodeVersion 2024.10.22 00:52:44.813401 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_429_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_429_429_0, total 76 rows starting from the beginning of the part 2024.10.22 00:52:44.835315 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column CodeVersion 2024.10.22 00:52:44.868680 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column CodeVersion 2024.10.22 00:52:44.889430 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column CodeVersion 2024.10.22 00:52:44.921105 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column CodeVersion 2024.10.22 00:52:45.055543 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:45.063159 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:45.082785 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ark/tbxbxekhrwtvddorcdyaqrxjqoysp (LogSeriesLimiter: on interval from 2024-10-22 00:52:21 to 2024-10-22 00:52:45 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:52:45.124414 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_49_49_0 (state Temporary) 2024.10.22 00:52:45.154688 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 39 entries to flush up to offset 16146 2024.10.22 00:52:45.319804 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_245_245_0 (state Temporary) 2024.10.22 00:52:45.388386 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:45.390690 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:45.391988 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_245_245_0 to 202410_258_258_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:45.493662 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_243_243_0 (state Temporary) 2024.10.22 00:52:45.778405 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3461 2024.10.22 00:52:45.788818 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_243_243_0 to 202410_249_249_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:45.866225 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16146 2024.10.22 00:52:46.296514 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_429_98/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:52:46.479982 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_429_98} MergeTask::MergeProjectionsStage: Merge sorted 24659 rows, containing 13 columns (13 merged, 0 gathered) in 2.188443566 sec., 11267.82540025526 rows/sec., 1.69 MiB/sec. 2024.10.22 00:52:46.629972 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.88 GiB, free memory in arenas 6.91 MiB, will set to 1.31 GiB (RSS), difference: -79.39 MiB 2024.10.22 00:52:46.680159 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 215 2024.10.22 00:52:46.939348 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_429_98} MergedBlockOutputStream: filled checksums 202410_1_429_98 (state Temporary) 2024.10.22 00:52:47.298205 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_429_98} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_429_98 to 202410_1_429_98 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:47.304457 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_429_98} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1_426_97, 202410_429_429_0] -> 202410_1_429_98 2024.10.22 00:52:47.329545 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:47.336833 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:47.375004 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.83 MiB. 2024.10.22 00:52:47.434873 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1538 entries to flush up to offset 401834 2024.10.22 00:52:47.516516 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_197_197_0 (state Temporary) 2024.10.22 00:52:47.713272 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_197_197_0 to 202410_207_207_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:47.821578 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 215 2024.10.22 00:52:48.120331 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:48.124458 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:48.269573 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 105 entries to flush up to offset 24030 2024.10.22 00:52:48.340671 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_234_234_0 (state Temporary) 2024.10.22 00:52:48.439014 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 1.88 GiB, free memory in arenas 5.64 MiB, will set to 1.34 GiB (RSS), difference: -15.05 MiB 2024.10.22 00:52:48.553238 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_234_234_0 to 202410_241_241_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:48.583876 [ 1163 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_236_47 to 202410_241_241_0 2024.10.22 00:52:48.592746 [ 1163 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:48.597807 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:48.609229 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 401834 2024.10.22 00:52:48.690008 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_236_47 to 202410_241_241_0 into Compact with storage Full 2024.10.22 00:52:48.743775 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:52:48.755889 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} MergeTreeSequentialSource: Reading 51 marks from part 202410_1_236_47, total 408126 rows starting from the beginning of the part 2024.10.22 00:52:48.805335 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:48.814749 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:48.869548 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_237_237_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:52:48.879970 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_238_238_0, total 1495 rows starting from the beginning of the part 2024.10.22 00:52:48.895100 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_239_239_0, total 1193 rows starting from the beginning of the part 2024.10.22 00:52:48.919243 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_240_240_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:52:48.935652 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_241_241_0, total 1538 rows starting from the beginning of the part 2024.10.22 00:52:49.028489 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_419_419_0 (state Temporary) 2024.10.22 00:52:49.189350 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_419_419_0 to 202410_432_432_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:49.252342 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24030 2024.10.22 00:52:49.748208 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:52:49.752521 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:52:50.088340 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:52:50.372642 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_241_48/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:52:50.769941 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.88 GiB, free memory in arenas 7.23 MiB, will set to 1.40 GiB (RSS), difference: -84.11 MiB 2024.10.22 00:52:51.880914 [ 1152 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 00:52:51.893128 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_49_49_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:52:52.513512 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 11.03 MiB, will set to 1.43 GiB (RSS), difference: 2.65 MiB 2024.10.22 00:52:53.214826 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 277.20 MiB using storage policy from min volume index 0 2024.10.22 00:52:53.281169 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 3471 2024.10.22 00:52:53.375758 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 63 entries to flush up to offset 16209 2024.10.22 00:52:53.381328 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:53.382193 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 24061 2024.10.22 00:52:53.387971 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:53.589670 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_246_246_0 (state Temporary) 2024.10.22 00:52:53.624102 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:53.629926 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:53.675160 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_246_246_0 to 202410_259_259_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:53.736493 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3471 2024.10.22 00:52:53.783914 [ 1269 ] {} DatabaseCatalog: Execution took 215 ms. 2024.10.22 00:52:53.787934 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_244_244_0 (state Temporary) 2024.10.22 00:52:53.836196 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_244_244_0 to 202410_250_250_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:53.870560 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:53.874353 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:53.880007 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16209 2024.10.22 00:52:53.920056 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_420_420_0 (state Temporary) 2024.10.22 00:52:53.986168 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_420_420_0 to 202410_433_433_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:54.168911 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yoi/otbfrghsffeslnywenpiqwqccgixe (LogSeriesLimiter: on interval from 2024-10-22 00:52:45 to 2024-10-22 00:52:54 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:52:54.382663 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24061 2024.10.22 00:52:54.530415 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.88 GiB, free memory in arenas 11.58 MiB, will set to 1.43 GiB (RSS), difference: -27.59 MiB 2024.10.22 00:52:54.602949 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_49_49_0 to 202410_52_52_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:55.324604 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 216 2024.10.22 00:52:55.591879 [ 1193 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:52:55.592045 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:55.595232 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:55.652988 [ 1193 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:52:55.722759 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1467 entries to flush up to offset 403301 2024.10.22 00:52:55.872751 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_198_198_0 (state Temporary) 2024.10.22 00:52:55.956105 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_198_198_0 to 202410_208_208_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:56.031971 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 216 2024.10.22 00:52:56.440166 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.88 GiB, free memory in arenas 11.09 MiB, will set to 1.43 GiB (RSS), difference: -121.01 MiB 2024.10.22 00:52:56.713563 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:56.719799 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:57.097779 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_235_235_0 (state Temporary) 2024.10.22 00:52:57.330162 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 1962 2024.10.22 00:52:57.375686 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_235_235_0 to 202410_242_242_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:57.523345 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 403301 2024.10.22 00:52:58.288710 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(c840e110eff0a55c1a489e1390710c8e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c840e110eff0a55c1a489e1390710c8e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:52:58.293262 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: c840e110eff0a55c1a489e1390710c8e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 00:52:58.297262 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:52:58.386687 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 24101 2024.10.22 00:52:58.656427 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:58.660944 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:58.728959 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.88 GiB, free memory in arenas 17.13 MiB, will set to 1.43 GiB (RSS), difference: -162.39 MiB 2024.10.22 00:52:59.051843 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_421_421_0 (state Temporary) 2024.10.22 00:52:59.090050 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_421_421_0 to 202410_434_434_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:52:59.089821 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/elr/fhkzpojidfabjsxhcrwyplogsiawu (LogSeriesLimiter: on interval from 2024-10-22 00:52:54 to 2024-10-22 00:52:59 accepted series 1 / 224 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:52:59.138741 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1_429_98 to 202410_432_432_0 2024.10.22 00:52:59.159829 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:52:59.164430 [ 1218 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:52:59.176124 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24101 2024.10.22 00:52:59.184446 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_432_99} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_429_98 to 202410_432_432_0 into Compact with storage Full 2024.10.22 00:52:59.243514 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_432_99} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:52:59.264805 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_432_99} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_429_98, total 24659 rows starting from the beginning of the part 2024.10.22 00:52:59.285023 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_432_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_430_430_0, total 57 rows starting from the beginning of the part 2024.10.22 00:52:59.349398 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_329_67, 202410_330_330_0, 202410_331_331_0, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0] 2024.10.22 00:52:59.352268 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_329_67, 202410_330_330_0, 202410_331_331_0, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0] 2024.10.22 00:52:59.379632 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_329_67, 202410_330_330_0, 202410_331_331_0, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0] 2024.10.22 00:52:59.430158 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_432_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_431_431_0, total 40 rows starting from the beginning of the part 2024.10.22 00:52:59.475725 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_432_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_432_432_0, total 105 rows starting from the beginning of the part 2024.10.22 00:52:59.760271 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:52:59.767409 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:53:00.771793 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 4 old parts to remove. Parts: [202410_1_187_38, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0] 2024.10.22 00:53:00.782195 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from filesystem (serially): Parts: [202410_1_187_38, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0] 2024.10.22 00:53:00.796578 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from memory: Parts: [202410_1_187_38, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0] 2024.10.22 00:53:00.863026 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.88 GiB, free memory in arenas 18.12 MiB, will set to 1.44 GiB (RSS), difference: -156.52 MiB 2024.10.22 00:53:01.240885 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 3478 2024.10.22 00:53:01.275404 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_241_48/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:53:01.389003 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_432_99} MergeTask::MergeProjectionsStage: Merge sorted 24861 rows, containing 13 columns (13 merged, 0 gathered) in 2.210395046 sec., 11247.310766909854 rows/sec., 1.69 MiB/sec. 2024.10.22 00:53:01.396805 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 57 entries to flush up to offset 16266 2024.10.22 00:53:01.554690 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_432_99} MergedBlockOutputStream: filled checksums 202410_1_432_99 (state Temporary) 2024.10.22 00:53:01.589535 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:01.592022 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:01.616362 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_247_247_0 (state Temporary) 2024.10.22 00:53:01.727869 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:01.731587 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:01.767531 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_247_247_0 to 202410_260_260_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:01.776807 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_245_245_0 (state Temporary) 2024.10.22 00:53:01.985650 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_432_99} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_432_99 to 202410_1_432_99 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:02.013003 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_432_99} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1_429_98, 202410_432_432_0] -> 202410_1_432_99 2024.10.22 00:53:02.047312 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3478 2024.10.22 00:53:02.060237 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_245_245_0 to 202410_251_251_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:02.080425 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.84 MiB. 2024.10.22 00:53:02.181416 [ 1214 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_246_59 to 202410_247_247_0 2024.10.22 00:53:02.189775 [ 1214 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:02.199041 [ 1214 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:02.220802 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16266 2024.10.22 00:53:02.248309 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_247_60} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_246_59 to 202410_247_247_0 into Compact with storage Full 2024.10.22 00:53:02.283069 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_247_60} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:02.306045 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_247_60} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_246_59, total 16080 rows starting from the beginning of the part 2024.10.22 00:53:02.468170 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_247_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_247_247_0, total 84 rows starting from the beginning of the part 2024.10.22 00:53:02.566826 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.88 GiB, free memory in arenas 15.74 MiB, will set to 1.43 GiB (RSS), difference: -66.38 MiB 2024.10.22 00:53:03.179024 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 24153 2024.10.22 00:53:03.440095 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:03.448950 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:03.493381 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_422_422_0 (state Temporary) 2024.10.22 00:53:03.524719 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_422_422_0 to 202410_435_435_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:03.534854 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 217 2024.10.22 00:53:03.674584 [ 1705 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ark/tbxbxekhrwtvddorcdyaqrxjqoysp, size 1321100 2024.10.22 00:53:03.678235 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_247_60} MergeTask::MergeProjectionsStage: Merge sorted 16164 rows, containing 13 columns (13 merged, 0 gathered) in 1.442798986 sec., 11203.223842576224 rows/sec., 4.71 MiB/sec. 2024.10.22 00:53:03.712662 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24153 2024.10.22 00:53:03.774438 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_247_60} MergedBlockOutputStream: filled checksums 202410_1_247_60 (state Temporary) 2024.10.22 00:53:03.892306 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:03.894620 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:03.970508 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_199_199_0 (state Temporary) 2024.10.22 00:53:03.987402 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_196_39, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0] 2024.10.22 00:53:03.989821 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_196_39, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0] 2024.10.22 00:53:04.021397 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_196_39, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0] 2024.10.22 00:53:04.037664 [ 1220 ] {} DatabaseCatalog: Execution took 252 ms. 2024.10.22 00:53:04.125012 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_199_199_0 to 202410_209_209_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:04.182284 [ 1226 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_204_42 to 202410_209_209_0 2024.10.22 00:53:04.221016 [ 1226 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:04.225183 [ 1226 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:04.229681 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 217 2024.10.22 00:53:04.265362 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_204_42 to 202410_209_209_0 into Compact with storage Full 2024.10.22 00:53:04.390518 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsLink 2024.10.22 00:53:04.412298 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_247_60} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_247_60 to 202410_1_247_60 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:04.417031 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_156_32, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:53:04.421161 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_247_60} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_246_59, 202410_247_247_0] -> 202410_1_247_60 2024.10.22 00:53:04.421620 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_156_32, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:53:04.430267 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsLink 2024.10.22 00:53:04.471763 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsLink 2024.10.22 00:53:04.495487 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsLink 2024.10.22 00:53:04.496343 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 15.57 MiB, will set to 1.44 GiB (RSS), difference: 1.37 MiB 2024.10.22 00:53:04.531229 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1605 entries to flush up to offset 404906 2024.10.22 00:53:04.570078 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_156_32, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0, 202410_160_160_0, 202410_161_161_0] 2024.10.22 00:53:04.662281 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsLink 2024.10.22 00:53:04.671637 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.64 MiB. 2024.10.22 00:53:04.780225 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsLink 2024.10.22 00:53:04.831209 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsLink 2024.10.22 00:53:04.885403 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 2003 2024.10.22 00:53:05.077646 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:05.086788 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gzz/uszgjvzmzmpsnbwmsivgxvlnpqenh (LogSeriesLimiter: on interval from 2024-10-22 00:52:59 to 2024-10-22 00:53:05 accepted series 1 / 133 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:53:05.109169 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_204_42, total 248 rows starting from the beginning of the part 2024.10.22 00:53:05.276582 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:05.278981 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:05.306978 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} MergeTask::MergeProjectionsStage: Merge sorted 415424 rows, containing 4 columns (4 merged, 0 gathered) in 16.62415355 sec., 24989.182080792318 rows/sec., 366.48 KiB/sec. 2024.10.22 00:53:05.395577 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} MergedBlockOutputStream: filled checksums 202410_1_241_48 (state Temporary) 2024.10.22 00:53:05.419469 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_236_236_0 (state Temporary) 2024.10.22 00:53:05.445678 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_241_48 to 202410_1_241_48 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:05.452286 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_205_205_0, total 1 rows starting from the beginning of the part 2024.10.22 00:53:05.452900 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_241_48} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_236_47, 202410_241_241_0] -> 202410_1_241_48 2024.10.22 00:53:05.513804 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:53:05.524166 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_236_236_0 to 202410_243_243_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:05.641839 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_206_206_0, total 1 rows starting from the beginning of the part 2024.10.22 00:53:05.672661 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 404906 2024.10.22 00:53:05.886623 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_207_207_0, total 1 rows starting from the beginning of the part 2024.10.22 00:53:06.372546 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 13.25 MiB, will set to 1.44 GiB (RSS), difference: 1.50 MiB 2024.10.22 00:53:06.614277 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_208_208_0, total 1 rows starting from the beginning of the part 2024.10.22 00:53:06.726880 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_209_209_0, total 1 rows starting from the beginning of the part 2024.10.22 00:53:07.718901 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_209_43/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:53:07.725666 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 24211 2024.10.22 00:53:07.971604 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:07.978593 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:08.029329 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_423_423_0 (state Temporary) 2024.10.22 00:53:08.072775 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.88 GiB, free memory in arenas 13.07 MiB, will set to 1.45 GiB (RSS), difference: -12.34 MiB 2024.10.22 00:53:08.488563 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_423_423_0 to 202410_436_436_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:08.539093 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24211 2024.10.22 00:53:08.980822 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} MergeTask::MergeProjectionsStage: Merge sorted 253 rows, containing 32 columns (32 merged, 0 gathered) in 4.724702468 sec., 53.54834547012157 rows/sec., 5.62 KiB/sec. 2024.10.22 00:53:09.143139 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} MergedBlockOutputStream: filled checksums 202410_1_209_43 (state Temporary) 2024.10.22 00:53:09.553161 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 3499 2024.10.22 00:53:09.730283 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 77 entries to flush up to offset 16343 2024.10.22 00:53:09.757504 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:53:09.764707 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:53:09.914405 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_209_43 to 202410_1_209_43 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:09.917297 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_209_43} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_204_42, 202410_209_209_0] -> 202410_1_209_43 2024.10.22 00:53:09.981755 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:53:09.989605 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:09.993664 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:10.074377 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_246_246_0 (state Temporary) 2024.10.22 00:53:10.168750 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:10.172018 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:10.303303 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_248_248_0 (state Temporary) 2024.10.22 00:53:10.356090 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_246_246_0 to 202410_252_252_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:10.401174 [ 1252 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_247_60 to 202410_248_248_0 2024.10.22 00:53:10.412495 [ 1252 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:10.417276 [ 1252 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:10.444678 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_248_61} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_247_60 to 202410_248_248_0 into Compact with storage Full 2024.10.22 00:53:10.480619 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_248_248_0 to 202410_261_261_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:10.501392 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16343 2024.10.22 00:53:10.523061 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_248_61} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:10.537370 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_248_61} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_247_60, total 16164 rows starting from the beginning of the part 2024.10.22 00:53:10.556157 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3499 2024.10.22 00:53:10.583497 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_248_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_248_248_0, total 47 rows starting from the beginning of the part 2024.10.22 00:53:10.590623 [ 1163 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 00:53:10.601122 [ 1168 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_256_51 to 202410_261_261_0 2024.10.22 00:53:10.638563 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_181_36, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0] 2024.10.22 00:53:10.645617 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_181_36, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0] 2024.10.22 00:53:10.660560 [ 1169 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:53:10.699975 [ 1169 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:53:10.719989 [ 1168 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:10.723937 [ 1168 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:10.734923 [ 1168 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 00:53:10.779831 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_181_36, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0, 202410_185_185_0, 202410_186_186_0] 2024.10.22 00:53:10.847734 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_256_51 to 202410_261_261_0 into Compact with storage Full 2024.10.22 00:53:10.915328 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:10.933174 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_256_51, total 3540 rows starting from the beginning of the part 2024.10.22 00:53:11.181215 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_257_257_0, total 21 rows starting from the beginning of the part 2024.10.22 00:53:11.220674 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_258_258_0, total 6 rows starting from the beginning of the part 2024.10.22 00:53:11.267516 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_259_259_0, total 10 rows starting from the beginning of the part 2024.10.22 00:53:11.298353 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_260_260_0, total 7 rows starting from the beginning of the part 2024.10.22 00:53:11.380299 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_261_261_0, total 21 rows starting from the beginning of the part 2024.10.22 00:53:11.544137 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_248_61} MergeTask::MergeProjectionsStage: Merge sorted 16211 rows, containing 13 columns (13 merged, 0 gathered) in 1.11148063 sec., 14585.049493844981 rows/sec., 6.13 MiB/sec. 2024.10.22 00:53:11.733237 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 218 2024.10.22 00:53:11.755444 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_248_61} MergedBlockOutputStream: filled checksums 202410_1_248_61 (state Temporary) 2024.10.22 00:53:12.048608 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:12.056348 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:12.124700 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_200_200_0 (state Temporary) 2024.10.22 00:53:12.284123 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_248_61} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_248_61 to 202410_1_248_61 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:12.288626 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_248_61} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_247_60, 202410_248_248_0] -> 202410_1_248_61 2024.10.22 00:53:12.319536 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.62 MiB. 2024.10.22 00:53:12.440729 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_200_200_0 to 202410_210_210_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:12.564142 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 24273 2024.10.22 00:53:12.590501 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 00:53:12.665525 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 218 2024.10.22 00:53:12.676758 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 406826 2024.10.22 00:53:12.720509 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:12.726938 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:12.936806 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} MergeTask::MergeProjectionsStage: Merge sorted 3605 rows, containing 12 columns (12 merged, 0 gathered) in 2.203751839 sec., 1635.84662129464 rows/sec., 255.48 KiB/sec. 2024.10.22 00:53:12.943674 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_424_424_0 (state Temporary) 2024.10.22 00:53:13.084460 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} MergedBlockOutputStream: filled checksums 202410_1_261_52 (state Temporary) 2024.10.22 00:53:13.194016 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_424_424_0 to 202410_437_437_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:13.232498 [ 1270 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_432_99 to 202410_433_433_0 2024.10.22 00:53:13.243085 [ 1270 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:13.247987 [ 1270 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:13.275098 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_433_100} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_432_99 to 202410_433_433_0 into Compact with storage Full 2024.10.22 00:53:13.288552 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24273 2024.10.22 00:53:13.318604 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_433_100} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:13.336089 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_433_100} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_432_99, total 24861 rows starting from the beginning of the part 2024.10.22 00:53:13.374573 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_433_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_433_433_0, total 31 rows starting from the beginning of the part 2024.10.22 00:53:13.562774 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_261_52 to 202410_1_261_52 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:13.567861 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_261_52} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_256_51, 202410_261_261_0] -> 202410_1_261_52 2024.10.22 00:53:13.596627 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:53:13.618836 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.88 GiB, free memory in arenas 7.41 MiB, will set to 1.46 GiB (RSS), difference: 6.54 MiB 2024.10.22 00:53:13.719609 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:13.732529 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:13.986968 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_237_237_0 (state Temporary) 2024.10.22 00:53:14.068144 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_237_237_0 to 202410_244_244_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:14.102872 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 406826 2024.10.22 00:53:14.297646 [ 1269 ] {} DatabaseCatalog: Execution took 260 ms. 2024.10.22 00:53:15.584245 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.88 GiB, free memory in arenas 7.27 MiB, will set to 1.47 GiB (RSS), difference: 7.39 MiB 2024.10.22 00:53:15.616234 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_433_100} MergeTask::MergeProjectionsStage: Merge sorted 24892 rows, containing 13 columns (13 merged, 0 gathered) in 2.348650519 sec., 10598.426542659241 rows/sec., 1.59 MiB/sec. 2024.10.22 00:53:15.786928 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_433_100} MergedBlockOutputStream: filled checksums 202410_1_433_100 (state Temporary) 2024.10.22 00:53:15.974952 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_433_100} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_433_100 to 202410_1_433_100 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:15.978289 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_433_100} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_432_99, 202410_433_433_0] -> 202410_1_433_100 2024.10.22 00:53:16.008118 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.95 MiB. 2024.10.22 00:53:17.331351 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 24307 2024.10.22 00:53:17.708659 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:17.713104 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:17.746328 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.88 GiB, free memory in arenas 6.33 MiB, will set to 1.47 GiB (RSS), difference: 4.00 MiB 2024.10.22 00:53:17.864791 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_425_425_0 (state Temporary) 2024.10.22 00:53:17.897226 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_425_425_0 to 202410_438_438_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:17.968408 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24307 2024.10.22 00:53:18.018704 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 86 entries to flush up to offset 16429 2024.10.22 00:53:18.038908 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_433_100 to 202410_434_434_0 2024.10.22 00:53:18.075619 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 3525 2024.10.22 00:53:18.150165 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:18.153494 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:18.164569 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:18.177701 [ 1202 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:18.181320 [ 1202 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 261 ms. 2024.10.22 00:53:18.182117 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:18.193668 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_434_101} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_433_100 to 202410_434_434_0 into Compact with storage Full 2024.10.22 00:53:18.194811 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:18.229197 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_247_247_0 (state Temporary) 2024.10.22 00:53:18.352400 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_247_247_0 to 202410_253_253_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:18.388363 [ 1165 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_248_61 to 202410_249_249_0 2024.10.22 00:53:18.391428 [ 1165 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:18.394246 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:18.416628 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16429 2024.10.22 00:53:18.435473 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_434_101} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:18.464729 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_249_249_0 (state Temporary) 2024.10.22 00:53:18.466812 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_434_101} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_433_100, total 24892 rows starting from the beginning of the part 2024.10.22 00:53:18.471955 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_249_62} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_248_61 to 202410_249_249_0 into Compact with storage Full 2024.10.22 00:53:18.582520 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_434_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_434_434_0, total 40 rows starting from the beginning of the part 2024.10.22 00:53:18.597290 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_249_62} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:18.634041 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_249_249_0 to 202410_262_262_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:18.720221 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_249_62} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_248_61, total 16211 rows starting from the beginning of the part 2024.10.22 00:53:18.744339 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3525 2024.10.22 00:53:18.757421 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_249_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_249_249_0, total 39 rows starting from the beginning of the part 2024.10.22 00:53:19.716972 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.88 GiB, free memory in arenas 3.98 MiB, will set to 1.47 GiB (RSS), difference: -21.50 MiB 2024.10.22 00:53:19.759729 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:53:19.763066 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:53:20.168187 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 219 2024.10.22 00:53:20.185080 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_249_62} MergeTask::MergeProjectionsStage: Merge sorted 16250 rows, containing 13 columns (13 merged, 0 gathered) in 1.762329236 sec., 9220.751530447855 rows/sec., 3.88 MiB/sec. 2024.10.22 00:53:20.400965 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_249_62} MergedBlockOutputStream: filled checksums 202410_1_249_62 (state Temporary) 2024.10.22 00:53:20.577937 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:20.580678 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.34 GiB. 2024.10.22 00:53:20.733434 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_201_201_0 (state Temporary) 2024.10.22 00:53:20.734901 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_434_101} MergeTask::MergeProjectionsStage: Merge sorted 24932 rows, containing 13 columns (13 merged, 0 gathered) in 2.552471173 sec., 9767.789060158762 rows/sec., 1.47 MiB/sec. 2024.10.22 00:53:20.773415 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_434_101} MergedBlockOutputStream: filled checksums 202410_1_434_101 (state Temporary) 2024.10.22 00:53:20.909092 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_201_201_0 to 202410_211_211_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:20.968471 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_249_62} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_249_62 to 202410_1_249_62 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:20.975616 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_249_62} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_248_61, 202410_249_249_0] -> 202410_1_249_62 2024.10.22 00:53:21.017457 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.61 MiB. 2024.10.22 00:53:21.027990 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 219 2024.10.22 00:53:21.111201 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 408362 2024.10.22 00:53:21.199074 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_434_101} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_434_101 to 202410_1_434_101 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:21.206191 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_434_101} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_433_100, 202410_434_434_0] -> 202410_1_434_101 2024.10.22 00:53:21.268735 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.96 MiB. 2024.10.22 00:53:21.700942 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 4.37 MiB, will set to 1.48 GiB (RSS), difference: 48.96 MiB 2024.10.22 00:53:21.925447 [ 1721 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/elr/fhkzpojidfabjsxhcrwyplogsiawu, size 155 2024.10.22 00:53:21.976649 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 24363 2024.10.22 00:53:22.227814 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:22.232043 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.33 GiB. 2024.10.22 00:53:22.574225 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:22.578985 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.33 GiB. 2024.10.22 00:53:22.624455 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_238_238_0 (state Temporary) 2024.10.22 00:53:22.744727 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_426_426_0 (state Temporary) 2024.10.22 00:53:22.884175 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_238_238_0 to 202410_245_245_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:22.914027 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_426_426_0 to 202410_439_439_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:22.970133 [ 1192 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_434_101 to 202410_435_435_0 2024.10.22 00:53:22.977723 [ 1192 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:22.988920 [ 1192 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.33 GiB. 2024.10.22 00:53:22.999571 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24363 2024.10.22 00:53:23.055035 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 408362 2024.10.22 00:53:23.065166 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_435_102} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_434_101 to 202410_435_435_0 into Compact with storage Full 2024.10.22 00:53:23.217361 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_435_102} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:23.324130 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(0fa096cd89a6503ffa2847e3a956777e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0fa096cd89a6503ffa2847e3a956777e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 00:53:23.327177 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 374482, file segment info: File segment: [0, 33554431], key: 0fa096cd89a6503ffa2847e3a956777e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 00:53:23.352743 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_435_102} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_434_101, total 24932 rows starting from the beginning of the part 2024.10.22 00:53:23.358754 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:53:23.471404 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_435_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_435_435_0, total 52 rows starting from the beginning of the part 2024.10.22 00:53:23.553547 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.88 GiB, free memory in arenas 4.15 MiB, will set to 1.47 GiB (RSS), difference: 144.13 MiB 2024.10.22 00:53:23.633639 [ 1713 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gzz/uszgjvzmzmpsnbwmsivgxvlnpqenh, size 374482 2024.10.22 00:53:24.138918 [ 1240 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 00:53:24.454367 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsDownload 2024.10.22 00:53:24.478111 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsDownload 2024.10.22 00:53:24.519934 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsDownload 2024.10.22 00:53:24.676790 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsDownload 2024.10.22 00:53:24.687403 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_435_102/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:53:24.818123 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsDownload 2024.10.22 00:53:24.828828 [ 1220 ] {} DatabaseCatalog: Execution took 513 ms. 2024.10.22 00:53:24.862777 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 3 old parts to remove. Parts: [202410_1_334_68, 202410_335_335_0, 202410_336_336_0] 2024.10.22 00:53:24.873078 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 3 parts from filesystem (serially): Parts: [202410_1_334_68, 202410_335_335_0, 202410_336_336_0] 2024.10.22 00:53:24.880972 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsDownload 2024.10.22 00:53:24.900076 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 3 parts from memory: Parts: [202410_1_334_68, 202410_335_335_0, 202410_336_336_0] 2024.10.22 00:53:24.949149 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsDownload 2024.10.22 00:53:25.095573 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fxb/whoiefrlualzpxzyyqhcbbimsfjjj (LogSeriesLimiter: on interval from 2024-10-22 00:53:05 to 2024-10-22 00:53:25 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:53:25.511575 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.88 GiB, free memory in arenas 4.21 MiB, will set to 1.46 GiB (RSS), difference: 129.42 MiB 2024.10.22 00:53:25.710736 [ 1247 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:53:25.785367 [ 1247 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:53:25.971535 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 162 entries to flush up to offset 16591 2024.10.22 00:53:25.995875 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_17_17_0 (state Temporary) 2024.10.22 00:53:26.005674 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 4 old parts to remove. Parts: [202410_1_190_39, 202410_191_191_0, 202410_192_192_0, 202410_193_193_0] 2024.10.22 00:53:26.052966 [ 1743 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yoi/otbfrghsffeslnywenpiqwqccgixe, size 384 2024.10.22 00:53:26.060829 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from filesystem (serially): Parts: [202410_1_190_39, 202410_191_191_0, 202410_192_192_0, 202410_193_193_0] 2024.10.22 00:53:26.090542 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from memory: Parts: [202410_1_190_39, 202410_191_191_0, 202410_192_192_0, 202410_193_193_0] 2024.10.22 00:53:26.247988 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 3536 2024.10.22 00:53:26.379548 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_435_102} MergeTask::MergeProjectionsStage: Merge sorted 24984 rows, containing 13 columns (13 merged, 0 gathered) in 3.329839317 sec., 7503.064749235166 rows/sec., 1.13 MiB/sec. 2024.10.22 00:53:26.398476 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:26.409532 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:26.433057 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_435_102} MergedBlockOutputStream: filled checksums 202410_1_435_102 (state Temporary) 2024.10.22 00:53:26.496124 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_435_102} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_435_102 to 202410_1_435_102 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:26.519511 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_435_102} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_434_101, 202410_435_435_0] -> 202410_1_435_102 2024.10.22 00:53:26.526910 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:26.531863 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:26.592547 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:26.595675 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.97 MiB. 2024.10.22 00:53:26.598989 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:26.662229 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_250_250_0 (state Temporary) 2024.10.22 00:53:26.664647 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_248_248_0 (state Temporary) 2024.10.22 00:53:26.823492 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_248_248_0 to 202410_254_254_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:26.825219 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_250_250_0 to 202410_263_263_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:26.967351 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_249_62 to 202410_254_254_0 2024.10.22 00:53:26.994932 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:26.999014 [ 1237 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:27.001751 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16591 2024.10.22 00:53:27.022212 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 24429 2024.10.22 00:53:27.041163 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_249_62 to 202410_254_254_0 into Compact with storage Full 2024.10.22 00:53:27.086458 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3536 2024.10.22 00:53:27.114948 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:27.160203 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_249_62, total 16250 rows starting from the beginning of the part 2024.10.22 00:53:27.330169 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_250_250_0, total 63 rows starting from the beginning of the part 2024.10.22 00:53:27.440447 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:27.447084 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:27.497533 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_251_251_0, total 57 rows starting from the beginning of the part 2024.10.22 00:53:27.576465 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_252_252_0, total 77 rows starting from the beginning of the part 2024.10.22 00:53:27.619804 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_427_427_0 (state Temporary) 2024.10.22 00:53:27.676110 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 3.49 MiB, will set to 1.43 GiB (RSS), difference: 35.11 MiB 2024.10.22 00:53:27.748446 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_253_253_0, total 86 rows starting from the beginning of the part 2024.10.22 00:53:27.963257 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_254_254_0, total 162 rows starting from the beginning of the part 2024.10.22 00:53:27.975067 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_427_427_0 to 202410_440_440_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:27.992515 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_435_102 to 202410_440_440_0 2024.10.22 00:53:28.016753 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:28.020152 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:28.040842 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_435_102 to 202410_440_440_0 into Compact with storage Full 2024.10.22 00:53:28.062665 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24429 2024.10.22 00:53:28.090850 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:28.099066 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_435_102, total 24984 rows starting from the beginning of the part 2024.10.22 00:53:28.220451 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_436_436_0, total 58 rows starting from the beginning of the part 2024.10.22 00:53:28.253547 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_437_437_0, total 62 rows starting from the beginning of the part 2024.10.22 00:53:28.376677 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_438_438_0, total 34 rows starting from the beginning of the part 2024.10.22 00:53:28.389095 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_439_439_0, total 56 rows starting from the beginning of the part 2024.10.22 00:53:28.416510 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_440_440_0, total 66 rows starting from the beginning of the part 2024.10.22 00:53:29.761349 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:53:29.764609 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:53:29.768686 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.88 GiB, free memory in arenas 3.94 MiB, will set to 1.45 GiB (RSS), difference: -17.14 MiB 2024.10.22 00:53:29.996903 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_50_50_0 (state Temporary) 2024.10.22 00:53:30.067278 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1591 entries to flush up to offset 409953 2024.10.22 00:53:30.487603 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} MergeTask::MergeProjectionsStage: Merge sorted 16695 rows, containing 13 columns (13 merged, 0 gathered) in 3.450292851 sec., 4838.719703216288 rows/sec., 2.04 MiB/sec. 2024.10.22 00:53:30.515501 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} MergedBlockOutputStream: filled checksums 202410_1_254_63 (state Temporary) 2024.10.22 00:53:30.637249 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_254_63 to 202410_1_254_63 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:30.643601 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_254_63} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_249_62, 202410_254_254_0] -> 202410_1_254_63 2024.10.22 00:53:30.694610 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 19.93 MiB. 2024.10.22 00:53:31.027634 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:31.031967 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:31.316860 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_239_239_0 (state Temporary) 2024.10.22 00:53:31.452831 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} MergeTask::MergeProjectionsStage: Merge sorted 25260 rows, containing 13 columns (13 merged, 0 gathered) in 3.422705271 sec., 7380.127121672931 rows/sec., 1.11 MiB/sec. 2024.10.22 00:53:31.484075 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} MergedBlockOutputStream: filled checksums 202410_1_440_103 (state Temporary) 2024.10.22 00:53:31.519559 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_239_239_0 to 202410_246_246_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:31.562927 [ 1163 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_241_48 to 202410_246_246_0 2024.10.22 00:53:31.572073 [ 1163 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:31.575492 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:31.597252 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_241_48 to 202410_246_246_0 into Compact with storage Full 2024.10.22 00:53:31.662100 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 409953 2024.10.22 00:53:31.688759 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 4.34 MiB, will set to 1.44 GiB (RSS), difference: 4.75 MiB 2024.10.22 00:53:31.714827 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:31.721451 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} MergeTreeSequentialSource: Reading 52 marks from part 202410_1_241_48, total 415424 rows starting from the beginning of the part 2024.10.22 00:53:31.732438 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_242_242_0, total 1467 rows starting from the beginning of the part 2024.10.22 00:53:31.781447 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_17_17_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:53:31.839892 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_440_103 to 202410_1_440_103 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:31.854957 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_243_243_0, total 1605 rows starting from the beginning of the part 2024.10.22 00:53:31.872600 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_244_244_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:53:31.963892 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_440_103} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_435_102, 202410_440_440_0] -> 202410_1_440_103 2024.10.22 00:53:31.992580 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_245_245_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:53:32.007165 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_246_246_0, total 1591 rows starting from the beginning of the part 2024.10.22 00:53:32.071979 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 24488 2024.10.22 00:53:32.090281 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.21 MiB. 2024.10.22 00:53:32.484824 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:32.487672 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:32.534148 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_428_428_0 (state Temporary) 2024.10.22 00:53:32.579694 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_428_428_0 to 202410_441_441_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:32.651183 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24488 2024.10.22 00:53:33.495658 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 5.23 MiB, will set to 1.44 GiB (RSS), difference: -3.64 MiB 2024.10.22 00:53:34.536010 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 16691 2024.10.22 00:53:34.588201 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 3558 2024.10.22 00:53:34.812812 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:34.818532 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:34.876648 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_251_251_0 (state Temporary) 2024.10.22 00:53:34.943270 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_50_50_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:53:34.978647 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:34.981113 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:35.056614 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_249_249_0 (state Temporary) 2024.10.22 00:53:35.067774 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_251_251_0 to 202410_264_264_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:35.080691 [ 1248 ] {} DatabaseCatalog: Execution took 267 ms. 2024.10.22 00:53:35.117825 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3558 2024.10.22 00:53:35.131362 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_249_249_0 to 202410_255_255_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:35.221477 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16691 2024.10.22 00:53:35.489208 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 8.59 MiB, will set to 1.44 GiB (RSS), difference: 4.75 MiB 2024.10.22 00:53:35.784357 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_246_49/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:53:35.889723 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 5 old parts to remove. Parts: [202410_1_336_69, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0, 202410_340_340_0] 2024.10.22 00:53:35.891853 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from filesystem (serially): Parts: [202410_1_336_69, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0, 202410_340_340_0] 2024.10.22 00:53:36.019842 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from memory: Parts: [202410_1_336_69, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0, 202410_340_340_0] 2024.10.22 00:53:36.123042 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 220 2024.10.22 00:53:36.405133 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:36.411032 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:36.495677 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 9.74 MiB, will set to 1.44 GiB (RSS), difference: 1.23 MiB 2024.10.22 00:53:36.587801 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_202_202_0 (state Temporary) 2024.10.22 00:53:36.669144 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 24520 2024.10.22 00:53:36.864875 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_202_202_0 to 202410_212_212_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:37.004171 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:37.022159 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:37.117402 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 220 2024.10.22 00:53:37.178393 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_429_429_0 (state Temporary) 2024.10.22 00:53:37.463981 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_429_429_0 to 202410_442_442_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:37.634931 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24520 2024.10.22 00:53:38.199133 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_50_50_0 to 202410_53_53_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:38.282938 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_34_7, 202410_35_35_0, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0] 2024.10.22 00:53:38.284805 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_34_7, 202410_35_35_0, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0] 2024.10.22 00:53:38.452521 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_34_7, 202410_35_35_0, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0, 202410_39_39_0] 2024.10.22 00:53:38.667323 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1867 entries to flush up to offset 411820 2024.10.22 00:53:39.766164 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:53:39.770317 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:53:40.067776 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:40.085229 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:40.337391 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_240_240_0 (state Temporary) 2024.10.22 00:53:40.506802 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 11.50 MiB, will set to 1.44 GiB (RSS), difference: -2.56 MiB 2024.10.22 00:53:40.788727 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_240_240_0 to 202410_247_247_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:40.796875 [ 1201 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:53:40.943693 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 411820 2024.10.22 00:53:40.948923 [ 1201 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:53:41.060734 [ 1201 ] {} DNSCacheUpdater: Execution took 264 ms. 2024.10.22 00:53:41.746477 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 24 entries to flush up to offset 24544 2024.10.22 00:53:42.107856 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:42.118513 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:42.174302 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_430_430_0 (state Temporary) 2024.10.22 00:53:42.180213 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2003 2024.10.22 00:53:42.214504 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_430_430_0 to 202410_443_443_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:42.271970 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24544 2024.10.22 00:53:42.536316 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 17.09 MiB, will set to 1.44 GiB (RSS), difference: 14.47 MiB 2024.10.22 00:53:42.624651 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 3565 2024.10.22 00:53:42.746170 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 30 entries to flush up to offset 16721 2024.10.22 00:53:42.941605 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:42.947744 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:43.075775 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:43.077730 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:43.087804 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_252_252_0 (state Temporary) 2024.10.22 00:53:43.117325 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_250_250_0 (state Temporary) 2024.10.22 00:53:43.285164 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_252_252_0 to 202410_265_265_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:43.347286 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3565 2024.10.22 00:53:43.384448 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_250_250_0 to 202410_256_256_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:43.558682 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16721 2024.10.22 00:53:44.620377 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 221 2024.10.22 00:53:44.658662 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_201_40, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0, 202410_205_205_0, 202410_206_206_0] 2024.10.22 00:53:44.662646 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_201_40, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0, 202410_205_205_0, 202410_206_206_0] 2024.10.22 00:53:44.684287 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_201_40, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0, 202410_205_205_0, 202410_206_206_0] 2024.10.22 00:53:44.900343 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:44.903454 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:44.992766 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_203_203_0 (state Temporary) 2024.10.22 00:53:45.007192 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.88 GiB, free memory in arenas 19.89 MiB, will set to 1.44 GiB (RSS), difference: -4.51 MiB 2024.10.22 00:53:45.047716 [ 1736 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fxb/whoiefrlualzpxzyyqhcbbimsfjjj, size 11122 2024.10.22 00:53:45.071670 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_203_203_0 to 202410_213_213_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:45.366836 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 221 2024.10.22 00:53:45.513564 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 00:53:45.531402 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 00:53:45.544394 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 00:53:45.570087 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 00:53:45.593725 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 00:53:45.622801 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 00:53:45.847842 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 00:53:46.101435 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gqq/nxnupxruawrwcslgrvuzmlkxjntzo (LogSeriesLimiter: on interval from 2024-10-22 00:53:25 to 2024-10-22 00:53:46 accepted series 1 / 12 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:53:46.274809 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 24583 2024.10.22 00:53:46.355830 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_246_49/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:53:46.527335 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 17.79 MiB, will set to 1.44 GiB (RSS), difference: 3.87 MiB 2024.10.22 00:53:46.696385 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:46.701224 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:46.754564 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_431_431_0 (state Temporary) 2024.10.22 00:53:46.792759 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_431_431_0 to 202410_444_444_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:46.871279 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24583 2024.10.22 00:53:48.155095 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 413740 2024.10.22 00:53:48.310334 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_193_40, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0, 202410_197_197_0, 202410_198_198_0] 2024.10.22 00:53:48.312517 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_193_40, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0, 202410_197_197_0, 202410_198_198_0] 2024.10.22 00:53:48.346402 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_193_40, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0, 202410_197_197_0, 202410_198_198_0] 2024.10.22 00:53:49.296217 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:49.300689 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:49.603220 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_241_241_0 (state Temporary) 2024.10.22 00:53:49.691103 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_241_241_0 to 202410_248_248_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:49.719748 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 45 entries to flush up to offset 2048 2024.10.22 00:53:49.721593 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 413740 2024.10.22 00:53:49.765516 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:53:49.767880 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:53:50.182436 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} MergeTask::MergeProjectionsStage: Merge sorted 423543 rows, containing 4 columns (4 merged, 0 gathered) in 18.59139064 sec., 22781.67395873728 rows/sec., 334.14 KiB/sec. 2024.10.22 00:53:50.247587 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} MergedBlockOutputStream: filled checksums 202410_1_246_49 (state Temporary) 2024.10.22 00:53:50.429113 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_246_49 to 202410_1_246_49 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:50.459064 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_246_49} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_241_48, 202410_246_246_0] -> 202410_1_246_49 2024.10.22 00:53:50.476648 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 21.72 MiB, will set to 1.45 GiB (RSS), difference: 2.31 MiB 2024.10.22 00:53:50.603282 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:53:50.679105 [ 1267 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 00:53:50.850241 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 3577 2024.10.22 00:53:50.876041 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 28 entries to flush up to offset 24611 2024.10.22 00:53:51.072015 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 49 entries to flush up to offset 16770 2024.10.22 00:53:51.291079 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:51.293926 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:51.333432 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_253_253_0 (state Temporary) 2024.10.22 00:53:51.386271 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_253_253_0 to 202410_266_266_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:51.420433 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:51.429195 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:51.436799 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:51.489082 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_251_251_0 (state Temporary) 2024.10.22 00:53:51.528864 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_261_52 to 202410_266_266_0 2024.10.22 00:53:51.540423 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_251_251_0 to 202410_257_257_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:51.604620 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16770 2024.10.22 00:53:51.647782 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:51.653766 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:51.662672 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:51.688439 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3577 2024.10.22 00:53:51.710859 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_261_52 to 202410_266_266_0 into Compact with storage Full 2024.10.22 00:53:51.814367 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_432_432_0 (state Temporary) 2024.10.22 00:53:51.967803 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_432_432_0 to 202410_445_445_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:52.060590 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:52.074250 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_261_52, total 3605 rows starting from the beginning of the part 2024.10.22 00:53:52.087186 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_262_262_0, total 26 rows starting from the beginning of the part 2024.10.22 00:53:52.100611 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24611 2024.10.22 00:53:52.120576 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_263_263_0, total 11 rows starting from the beginning of the part 2024.10.22 00:53:52.149800 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_264_264_0, total 22 rows starting from the beginning of the part 2024.10.22 00:53:52.178541 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_265_265_0, total 7 rows starting from the beginning of the part 2024.10.22 00:53:52.192212 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_266_266_0, total 12 rows starting from the beginning of the part 2024.10.22 00:53:52.290168 [ 1156 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_440_103 to 202410_441_441_0 2024.10.22 00:53:52.296059 [ 1156 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:52.299371 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:52.309716 [ 1156 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 329 ms. 2024.10.22 00:53:52.336999 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_441_104} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_440_103 to 202410_441_441_0 into Compact with storage Full 2024.10.22 00:53:52.424110 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_186_37, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0, 202410_191_191_0] 2024.10.22 00:53:52.427932 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_186_37, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0, 202410_191_191_0] 2024.10.22 00:53:52.467778 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_186_37, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0, 202410_190_190_0, 202410_191_191_0] 2024.10.22 00:53:52.583067 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_441_104} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:52.597444 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_441_104} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_440_103, total 25260 rows starting from the beginning of the part 2024.10.22 00:53:52.775862 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_441_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_441_441_0, total 59 rows starting from the beginning of the part 2024.10.22 00:53:52.872034 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 222 2024.10.22 00:53:53.171023 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:53.175635 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:53.368778 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} MergeTask::MergeProjectionsStage: Merge sorted 3683 rows, containing 12 columns (12 merged, 0 gathered) in 1.680979597 sec., 2190.9843561295766 rows/sec., 342.40 KiB/sec. 2024.10.22 00:53:53.403665 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} MergedBlockOutputStream: filled checksums 202410_1_266_53 (state Temporary) 2024.10.22 00:53:53.462474 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_204_204_0 (state Temporary) 2024.10.22 00:53:53.529758 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 5 old parts to remove. Parts: [202410_1_340_70, 202410_341_341_0, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0] 2024.10.22 00:53:53.534182 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from filesystem (serially): Parts: [202410_1_340_70, 202410_341_341_0, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0] 2024.10.22 00:53:53.635908 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_204_204_0 to 202410_214_214_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:53.693195 [ 1241 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_209_43 to 202410_214_214_0 2024.10.22 00:53:53.711780 [ 1241 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:53.719414 [ 1241 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:53.732221 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_209_43 to 202410_214_214_0 into Compact with storage Full 2024.10.22 00:53:53.759077 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 222 2024.10.22 00:53:53.778175 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from memory: Parts: [202410_1_340_70, 202410_341_341_0, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0] 2024.10.22 00:53:53.999691 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_266_53 to 202410_1_266_53 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:54.005547 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_266_53} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_261_52, 202410_266_266_0] -> 202410_1_266_53 2024.10.22 00:53:54.066870 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:53:54.184070 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:54.233417 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_209_43, total 253 rows starting from the beginning of the part 2024.10.22 00:53:54.291947 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_210_210_0, total 1 rows starting from the beginning of the part 2024.10.22 00:53:54.349368 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_211_211_0, total 1 rows starting from the beginning of the part 2024.10.22 00:53:54.383041 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_212_212_0, total 1 rows starting from the beginning of the part 2024.10.22 00:53:54.392695 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 14.46 MiB, will set to 1.45 GiB (RSS), difference: 3.82 MiB 2024.10.22 00:53:54.430834 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_213_213_0, total 1 rows starting from the beginning of the part 2024.10.22 00:53:54.453784 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_441_104} MergeTask::MergeProjectionsStage: Merge sorted 25319 rows, containing 13 columns (13 merged, 0 gathered) in 1.991989583 sec., 12710.407833493173 rows/sec., 1.91 MiB/sec. 2024.10.22 00:53:54.476798 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_214_214_0, total 1 rows starting from the beginning of the part 2024.10.22 00:53:54.478396 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_441_104} MergedBlockOutputStream: filled checksums 202410_1_441_104 (state Temporary) 2024.10.22 00:53:54.867242 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_441_104} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_441_104 to 202410_1_441_104 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:54.972248 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_441_104} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_440_103, 202410_441_441_0] -> 202410_1_441_104 2024.10.22 00:53:55.002622 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.07 MiB. 2024.10.22 00:53:55.498509 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.88 GiB, free memory in arenas 12.45 MiB, will set to 1.44 GiB (RSS), difference: -6.48 MiB 2024.10.22 00:53:55.601559 [ 1161 ] {} DatabaseCatalog: Execution took 340 ms. 2024.10.22 00:53:56.059863 [ 1271 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:53:56.105031 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 24684 2024.10.22 00:53:56.192996 [ 1271 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:53:56.397165 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:56.402384 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:56.476277 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_433_433_0 (state Temporary) 2024.10.22 00:53:56.477956 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} MergeTask::MergeProjectionsStage: Merge sorted 258 rows, containing 32 columns (32 merged, 0 gathered) in 2.748412062 sec., 93.87238673820083 rows/sec., 9.83 KiB/sec. 2024.10.22 00:53:56.494556 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} MergedBlockOutputStream: filled checksums 202410_1_214_44 (state Temporary) 2024.10.22 00:53:56.522720 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 8.43 MiB, will set to 1.44 GiB (RSS), difference: -3.93 MiB 2024.10.22 00:53:56.555802 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_433_433_0 to 202410_446_446_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:56.588099 [ 1275 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_441_104 to 202410_442_442_0 2024.10.22 00:53:56.592573 [ 1275 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:56.596250 [ 1275 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:56.620827 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_442_105} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_441_104 to 202410_442_442_0 into Compact with storage Full 2024.10.22 00:53:56.632992 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24684 2024.10.22 00:53:56.659077 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_442_105} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:53:56.674122 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_442_105} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_441_104, total 25319 rows starting from the beginning of the part 2024.10.22 00:53:56.718174 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_442_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_442_442_0, total 32 rows starting from the beginning of the part 2024.10.22 00:53:56.741318 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1999 entries to flush up to offset 415739 2024.10.22 00:53:57.134537 [ 1262 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 00:53:57.294578 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_214_44 to 202410_1_214_44 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:57.297575 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_214_44} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_209_43, 202410_214_214_0] -> 202410_1_214_44 2024.10.22 00:53:57.355706 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:53:57.614991 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:57.623039 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:57.784793 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_242_242_0 (state Temporary) 2024.10.22 00:53:57.976415 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mvg/niiynvsppfukcsdhuoxgnkhrzdcdz. 2024.10.22 00:53:57.980558 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_242_242_0 to 202410_249_249_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:58.019254 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 415739 2024.10.22 00:53:58.671189 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 5.75 MiB, will set to 1.44 GiB (RSS), difference: 3.47 MiB 2024.10.22 00:53:59.123428 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 103 entries to flush up to offset 16873 2024.10.22 00:53:59.190802 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 3600 2024.10.22 00:53:59.330789 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_442_105} MergeTask::MergeProjectionsStage: Merge sorted 25351 rows, containing 13 columns (13 merged, 0 gathered) in 2.727786257 sec., 9293.616732229179 rows/sec., 1.40 MiB/sec. 2024.10.22 00:53:59.383473 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:59.388792 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:59.489558 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_442_105} MergedBlockOutputStream: filled checksums 202410_1_442_105 (state Temporary) 2024.10.22 00:53:59.676257 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_252_252_0 (state Temporary) 2024.10.22 00:53:59.715208 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:53:59.720525 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:53:59.756198 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_442_105} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_442_105 to 202410_1_442_105 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:59.763488 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_442_105} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_441_104, 202410_442_442_0] -> 202410_1_442_105 2024.10.22 00:53:59.767942 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:53:59.772773 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:53:59.799287 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_254_254_0 (state Temporary) 2024.10.22 00:53:59.807783 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 00:53:59.832464 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_252_252_0 to 202410_258_258_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:59.846442 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_254_254_0 to 202410_267_267_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:53:59.897126 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_17_17_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:53:59.988765 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3600 2024.10.22 00:53:59.990823 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16873 2024.10.22 00:54:00.338449 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_161_33, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0] 2024.10.22 00:54:00.342749 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_161_33, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0] 2024.10.22 00:54:00.481084 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_161_33, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0, 202410_165_165_0] 2024.10.22 00:54:00.655644 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 24736 2024.10.22 00:54:00.761415 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 3.70 MiB, will set to 1.44 GiB (RSS), difference: -2.18 MiB 2024.10.22 00:54:00.832126 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:00.836901 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:54:01.045441 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_434_434_0 (state Temporary) 2024.10.22 00:54:01.262901 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 223 2024.10.22 00:54:01.388896 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_434_434_0 to 202410_447_447_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:01.423661 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_442_105 to 202410_443_443_0 2024.10.22 00:54:01.432284 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:01.449732 [ 1201 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:54:01.489318 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_443_106} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_442_105 to 202410_443_443_0 into Compact with storage Full 2024.10.22 00:54:01.505116 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(a51bbb62503d58ba797de11bf9336705) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a51bbb62503d58ba797de11bf9336705, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 00:54:01.513127 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 717780, file segment info: File segment: [0, 33554431], key: a51bbb62503d58ba797de11bf9336705, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 00:54:01.548747 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:54:01.552243 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24736 2024.10.22 00:54:01.590037 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:01.592723 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_443_106} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:01.599138 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:54:01.603079 [ 1680 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gqq/nxnupxruawrwcslgrvuzmlkxjntzo, size 717780 2024.10.22 00:54:01.612676 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_443_106} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_442_105, total 25351 rows starting from the beginning of the part 2024.10.22 00:54:01.744605 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_443_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_443_443_0, total 24 rows starting from the beginning of the part 2024.10.22 00:54:01.918360 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_205_205_0 (state Temporary) 2024.10.22 00:54:02.161333 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FUniqID 2024.10.22 00:54:02.175070 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FUniqID 2024.10.22 00:54:02.197220 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FUniqID 2024.10.22 00:54:02.235275 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FUniqID 2024.10.22 00:54:02.304626 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_205_205_0 to 202410_215_215_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:02.363136 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FUniqID 2024.10.22 00:54:02.392863 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FUniqID 2024.10.22 00:54:02.426909 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FUniqID 2024.10.22 00:54:02.492968 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bgo/ukpnjcifiwygwyujttniikuumxvur (LogSeriesLimiter: on interval from 2024-10-22 00:53:46 to 2024-10-22 00:54:02 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:54:02.496720 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_443_106/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:54:02.545660 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 223 2024.10.22 00:54:03.584510 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 1.88 GiB, free memory in arenas 4.73 MiB, will set to 1.43 GiB (RSS), difference: 87.90 MiB 2024.10.22 00:54:03.966430 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_443_106} MergeTask::MergeProjectionsStage: Merge sorted 25375 rows, containing 13 columns (13 merged, 0 gathered) in 2.489218458 sec., 10193.962654602814 rows/sec., 1.53 MiB/sec. 2024.10.22 00:54:04.013465 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_443_106} MergedBlockOutputStream: filled checksums 202410_1_443_106 (state Temporary) 2024.10.22 00:54:04.314370 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_443_106} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_443_106 to 202410_1_443_106 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:04.320009 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_443_106} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_442_105, 202410_443_443_0] -> 202410_1_443_106 2024.10.22 00:54:04.457318 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 00:54:04.956878 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mvg/niiynvsppfukcsdhuoxgnkhrzdcdz. 2024.10.22 00:54:05.029238 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1457 entries to flush up to offset 417196 2024.10.22 00:54:05.562754 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.88 GiB, free memory in arenas 5.84 MiB, will set to 1.42 GiB (RSS), difference: 87.19 MiB 2024.10.22 00:54:05.565316 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 24777 2024.10.22 00:54:05.942877 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:05.941796 [ 1191 ] {} DatabaseCatalog: Execution took 349 ms. 2024.10.22 00:54:05.955144 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:54:06.017755 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:06.023059 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:54:06.198578 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_435_435_0 (state Temporary) 2024.10.22 00:54:06.491307 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_435_435_0 to 202410_448_448_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:06.525365 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_443_106 to 202410_444_444_0 2024.10.22 00:54:06.535164 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:06.572661 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_243_243_0 (state Temporary) 2024.10.22 00:54:06.620730 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b0227540cee0622267526ae23c8fc9de) : [0, 4541506]: Setting download as failed: File segment: [0, 4541506], key: b0227540cee0622267526ae23c8fc9de, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:06.624311 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b0227540cee0622267526ae23c8fc9de): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:54:06.645519 [ 1240 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:54:06.662048 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24777 2024.10.22 00:54:06.701595 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_444_107} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_443_106 to 202410_444_444_0 into Compact with storage Full 2024.10.22 00:54:06.731752 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b0227540cee0622267526ae23c8fc9de, offset: 0, size: 0 2024.10.22 00:54:06.734068 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b0227540cee0622267526ae23c8fc9de for removal 2024.10.22 00:54:06.737145 [ 1476 ] {} CacheMetadata: Key b0227540cee0622267526ae23c8fc9de is removed from metadata 2024.10.22 00:54:06.745490 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_444_107} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:06.761756 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_444_107} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_443_106, total 25375 rows starting from the beginning of the part 2024.10.22 00:54:06.880288 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_444_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_444_444_0, total 39 rows starting from the beginning of the part 2024.10.22 00:54:06.991767 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_243_243_0 to 202410_250_250_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:07.023426 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 417196 2024.10.22 00:54:07.275372 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 5.71 MiB, will set to 1.42 GiB (RSS), difference: -5.84 MiB 2024.10.22 00:54:07.490319 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 3612 2024.10.22 00:54:07.500945 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 16964 2024.10.22 00:54:07.726728 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:07.732067 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:54:07.777564 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_253_253_0 (state Temporary) 2024.10.22 00:54:07.823956 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_253_253_0 to 202410_259_259_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:07.834583 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:07.837720 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:54:07.872315 [ 1236 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_254_63 to 202410_255_255_0 2024.10.22 00:54:07.877180 [ 1236 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:07.879863 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_255_255_0 (state Temporary) 2024.10.22 00:54:07.881734 [ 1236 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:54:07.888190 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 16964 2024.10.22 00:54:07.891431 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_255_64} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_254_63 to 202410_255_255_0 into Compact with storage Full 2024.10.22 00:54:07.894846 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(2f989c75d76c902f630aebb13ca5ec17) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2f989c75d76c902f630aebb13ca5ec17, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 00:54:07.897631 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 2f989c75d76c902f630aebb13ca5ec17, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 00:54:07.899688 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:54:07.949083 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_255_64} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:07.969086 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_255_64} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_254_63, total 16695 rows starting from the beginning of the part 2024.10.22 00:54:08.132020 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_255_255_0 to 202410_268_268_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:08.201380 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3612 2024.10.22 00:54:08.261358 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_255_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_255_255_0, total 100 rows starting from the beginning of the part 2024.10.22 00:54:08.270760 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:08.276347 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:54:08.455623 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ae01bfe27c5ed196a44d7960edbdf82e) : [0, 88091]: Setting download as failed: File segment: [0, 88091], key: ae01bfe27c5ed196a44d7960edbdf82e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:08.459743 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ae01bfe27c5ed196a44d7960edbdf82e): No space left in cache to reserve 88092 bytes, will continue without cache download 2024.10.22 00:54:08.584273 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ae01bfe27c5ed196a44d7960edbdf82e, offset: 0, size: 0 2024.10.22 00:54:08.586755 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ae01bfe27c5ed196a44d7960edbdf82e for removal 2024.10.22 00:54:08.590405 [ 1476 ] {} CacheMetadata: Key ae01bfe27c5ed196a44d7960edbdf82e is removed from metadata 2024.10.22 00:54:08.915081 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_444_107} MergeTask::MergeProjectionsStage: Merge sorted 25414 rows, containing 13 columns (13 merged, 0 gathered) in 2.219563598 sec., 11449.99855958171 rows/sec., 1.72 MiB/sec. 2024.10.22 00:54:08.975012 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_444_107} MergedBlockOutputStream: filled checksums 202410_1_444_107 (state Temporary) 2024.10.22 00:54:09.302939 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_255_64} MergeTask::MergeProjectionsStage: Merge sorted 16795 rows, containing 13 columns (13 merged, 0 gathered) in 1.41480816 sec., 11870.867354906972 rows/sec., 4.99 MiB/sec. 2024.10.22 00:54:09.327930 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_444_107} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_444_107 to 202410_1_444_107 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:09.335838 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_444_107} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_443_106, 202410_444_444_0] -> 202410_1_444_107 2024.10.22 00:54:09.468797 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_255_64} MergedBlockOutputStream: filled checksums 202410_1_255_64 (state Temporary) 2024.10.22 00:54:09.578622 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 00:54:09.670703 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_255_64} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_255_64 to 202410_1_255_64 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:09.672982 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_255_64} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_254_63, 202410_255_255_0] -> 202410_1_255_64 2024.10.22 00:54:09.725524 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.44 MiB. 2024.10.22 00:54:09.769657 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:54:09.872964 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:54:10.050641 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 224 2024.10.22 00:54:10.281814 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4048c87837cb137bb272e6216dcafcce) : [0, 2144509]: Setting download as failed: File segment: [0, 2144509], key: 4048c87837cb137bb272e6216dcafcce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:10.283426 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4048c87837cb137bb272e6216dcafcce): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:54:10.326742 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:10.336268 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.29 GiB. 2024.10.22 00:54:10.454404 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4048c87837cb137bb272e6216dcafcce, offset: 0, size: 0 2024.10.22 00:54:10.457450 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4048c87837cb137bb272e6216dcafcce for removal 2024.10.22 00:54:10.461284 [ 1476 ] {} CacheMetadata: Key 4048c87837cb137bb272e6216dcafcce is removed from metadata 2024.10.22 00:54:10.619597 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_206_206_0 (state Temporary) 2024.10.22 00:54:10.626239 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 4.39 MiB, will set to 1.42 GiB (RSS), difference: -3.61 MiB 2024.10.22 00:54:10.677134 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 24854 2024.10.22 00:54:10.873222 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_206_206_0 to 202410_216_216_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:11.024224 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:11.028383 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:11.050467 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 224 2024.10.22 00:54:11.200563 [ 1250 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:54:11.238966 [ 1250 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:54:11.324086 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_436_436_0 (state Temporary) 2024.10.22 00:54:11.386566 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_436_436_0 to 202410_449_449_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:11.422872 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_444_107 to 202410_449_449_0 2024.10.22 00:54:11.443328 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:11.443505 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24854 2024.10.22 00:54:11.453019 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:11.475902 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c340c3b240d514792a32b9bf03c5d4d0) : [0, 83146]: Setting download as failed: File segment: [0, 83146], key: c340c3b240d514792a32b9bf03c5d4d0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:11.477350 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c340c3b240d514792a32b9bf03c5d4d0): No space left in cache to reserve 83147 bytes, will continue without cache download 2024.10.22 00:54:11.495938 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c340c3b240d514792a32b9bf03c5d4d0, offset: 0, size: 0 2024.10.22 00:54:11.498182 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c340c3b240d514792a32b9bf03c5d4d0 for removal 2024.10.22 00:54:11.503461 [ 1476 ] {} CacheMetadata: Key c340c3b240d514792a32b9bf03c5d4d0 is removed from metadata 2024.10.22 00:54:11.581503 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_444_107 to 202410_449_449_0 into Compact with storage Full 2024.10.22 00:54:11.592562 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 3 old parts to remove. Parts: [202410_1_344_71, 202410_345_345_0, 202410_346_346_0] 2024.10.22 00:54:11.594557 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 3 parts from filesystem (serially): Parts: [202410_1_344_71, 202410_345_345_0, 202410_346_346_0] 2024.10.22 00:54:11.639092 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_51_51_0 (state Temporary) 2024.10.22 00:54:11.658726 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:11.666725 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_444_107, total 25414 rows starting from the beginning of the part 2024.10.22 00:54:11.717544 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_445_445_0, total 28 rows starting from the beginning of the part 2024.10.22 00:54:11.744877 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 3 parts from memory: Parts: [202410_1_344_71, 202410_345_345_0, 202410_346_346_0] 2024.10.22 00:54:11.758165 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_446_446_0, total 73 rows starting from the beginning of the part 2024.10.22 00:54:11.791933 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_447_447_0, total 52 rows starting from the beginning of the part 2024.10.22 00:54:11.882078 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.158144717 sec. 2024.10.22 00:54:11.936258 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_448_448_0, total 41 rows starting from the beginning of the part 2024.10.22 00:54:11.984767 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_449_449_0, total 77 rows starting from the beginning of the part 2024.10.22 00:54:12.113791 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(cd1cb30534332ddaa365a2ef0eb1d6b6) : [0, 60828]: Setting download as failed: File segment: [0, 60828], key: cd1cb30534332ddaa365a2ef0eb1d6b6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:12.117008 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(cd1cb30534332ddaa365a2ef0eb1d6b6): No space left in cache to reserve 60829 bytes, will continue without cache download 2024.10.22 00:54:12.166371 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cd1cb30534332ddaa365a2ef0eb1d6b6, offset: 0, size: 0 2024.10.22 00:54:12.169798 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cd1cb30534332ddaa365a2ef0eb1d6b6 for removal 2024.10.22 00:54:12.177188 [ 1476 ] {} CacheMetadata: Key cd1cb30534332ddaa365a2ef0eb1d6b6 is removed from metadata 2024.10.22 00:54:12.976209 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a64229fa38ee020fca4dbc926b7c0f98) : [0, 523626]: Setting download as failed: File segment: [0, 523626], key: a64229fa38ee020fca4dbc926b7c0f98, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:13.078416 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a64229fa38ee020fca4dbc926b7c0f98): No space left in cache to reserve 523627 bytes, will continue without cache download 2024.10.22 00:54:13.102740 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a64229fa38ee020fca4dbc926b7c0f98, offset: 0, size: 0 2024.10.22 00:54:13.106524 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a64229fa38ee020fca4dbc926b7c0f98 for removal 2024.10.22 00:54:13.113487 [ 1476 ] {} CacheMetadata: Key a64229fa38ee020fca4dbc926b7c0f98 is removed from metadata 2024.10.22 00:54:13.449767 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_449_108/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:54:13.584182 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 4.78 MiB, will set to 1.43 GiB (RSS), difference: -6.12 MiB 2024.10.22 00:54:13.777301 [ 1252 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 00:54:14.030250 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 418348 2024.10.22 00:54:14.330518 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f19815705044d200afef190e8558c408) : [0, 335265]: Setting download as failed: File segment: [0, 335265], key: f19815705044d200afef190e8558c408, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:14.335584 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f19815705044d200afef190e8558c408): No space left in cache to reserve 335266 bytes, will continue without cache download 2024.10.22 00:54:14.476568 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f19815705044d200afef190e8558c408, offset: 0, size: 0 2024.10.22 00:54:14.478902 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f19815705044d200afef190e8558c408 for removal 2024.10.22 00:54:14.482379 [ 1476 ] {} CacheMetadata: Key f19815705044d200afef190e8558c408 is removed from metadata 2024.10.22 00:54:14.489566 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} MergeTask::MergeProjectionsStage: Merge sorted 25685 rows, containing 13 columns (13 merged, 0 gathered) in 2.913059303 sec., 8817.190907699143 rows/sec., 1.33 MiB/sec. 2024.10.22 00:54:14.539514 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} MergedBlockOutputStream: filled checksums 202410_1_449_108 (state Temporary) 2024.10.22 00:54:14.675197 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:14.676577 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:14.779507 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_244_244_0 (state Temporary) 2024.10.22 00:54:14.988978 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_244_244_0 to 202410_251_251_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:15.039277 [ 1170 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_246_49 to 202410_251_251_0 2024.10.22 00:54:15.041604 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 418348 2024.10.22 00:54:15.048972 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_449_108 to 202410_1_449_108 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:15.053367 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_449_108} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_444_107, 202410_449_449_0] -> 202410_1_449_108 2024.10.22 00:54:15.066622 [ 1170 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:15.072264 [ 1170 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:15.082764 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_246_49 to 202410_251_251_0 into Compact with storage Full 2024.10.22 00:54:15.098988 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.29 MiB. 2024.10.22 00:54:15.124861 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:15.137169 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} MergeTreeSequentialSource: Reading 53 marks from part 202410_1_246_49, total 423543 rows starting from the beginning of the part 2024.10.22 00:54:15.152495 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_247_247_0, total 1867 rows starting from the beginning of the part 2024.10.22 00:54:15.182420 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_248_248_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:54:15.299691 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_249_249_0, total 1999 rows starting from the beginning of the part 2024.10.22 00:54:15.403390 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(06ab8c537c819e9a5b02e1356d83a988) : [0, 735762]: Setting download as failed: File segment: [0, 735762], key: 06ab8c537c819e9a5b02e1356d83a988, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:15.405397 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(06ab8c537c819e9a5b02e1356d83a988): No space left in cache to reserve 735763 bytes, will continue without cache download 2024.10.22 00:54:15.421738 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_250_250_0, total 1457 rows starting from the beginning of the part 2024.10.22 00:54:15.456042 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_251_251_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:54:15.468509 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 24926 2024.10.22 00:54:15.474680 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 17048 2024.10.22 00:54:15.587092 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 06ab8c537c819e9a5b02e1356d83a988, offset: 0, size: 0 2024.10.22 00:54:15.590445 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 06ab8c537c819e9a5b02e1356d83a988 for removal 2024.10.22 00:54:15.596043 [ 1476 ] {} CacheMetadata: Key 06ab8c537c819e9a5b02e1356d83a988 is removed from metadata 2024.10.22 00:54:15.641557 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:15.651916 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:15.706491 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 3630 2024.10.22 00:54:15.714395 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 3.71 MiB, will set to 1.42 GiB (RSS), difference: 5.55 MiB 2024.10.22 00:54:15.859707 [ 1190 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 257 ms. 2024.10.22 00:54:15.908866 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_254_254_0 (state Temporary) 2024.10.22 00:54:15.973080 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:15.975916 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:15.987903 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_254_254_0 to 202410_260_260_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:16.069347 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:16.082010 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:16.116948 [ 1167 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_255_64 to 202410_256_256_0 2024.10.22 00:54:16.119371 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_256_256_0 (state Temporary) 2024.10.22 00:54:16.132013 [ 1167 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:16.142730 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:16.179511 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_256_65} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_255_64 to 202410_256_256_0 into Compact with storage Full 2024.10.22 00:54:16.189636 [ 1153 ] {} DatabaseCatalog: Execution took 244 ms. 2024.10.22 00:54:16.200049 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_256_256_0 to 202410_269_269_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:16.268361 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 17048 2024.10.22 00:54:16.271640 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_437_437_0 (state Temporary) 2024.10.22 00:54:16.285592 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3630 2024.10.22 00:54:16.299243 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_256_65} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:16.308853 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_256_65} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_255_64, total 16795 rows starting from the beginning of the part 2024.10.22 00:54:16.330836 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_437_437_0 to 202410_450_450_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:16.368383 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_256_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_256_256_0, total 30 rows starting from the beginning of the part 2024.10.22 00:54:16.496682 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24926 2024.10.22 00:54:17.182355 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4261f86937d8df6523c26f19c7b1befa) : [0, 137539]: Setting download as failed: File segment: [0, 137539], key: 4261f86937d8df6523c26f19c7b1befa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:17.184365 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4261f86937d8df6523c26f19c7b1befa): No space left in cache to reserve 137540 bytes, will continue without cache download 2024.10.22 00:54:17.323162 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4261f86937d8df6523c26f19c7b1befa, offset: 0, size: 0 2024.10.22 00:54:17.325506 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4261f86937d8df6523c26f19c7b1befa for removal 2024.10.22 00:54:17.332649 [ 1476 ] {} CacheMetadata: Key 4261f86937d8df6523c26f19c7b1befa is removed from metadata 2024.10.22 00:54:17.711640 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 6.11 MiB, will set to 1.42 GiB (RSS), difference: -14.02 MiB 2024.10.22 00:54:17.841022 [ 1274 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 00:54:17.866685 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_256_65/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:54:18.192291 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_51_51_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:54:18.519800 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_256_65} MergeTask::MergeProjectionsStage: Merge sorted 16825 rows, containing 13 columns (13 merged, 0 gathered) in 2.325063731 sec., 7236.3607825767585 rows/sec., 3.04 MiB/sec. 2024.10.22 00:54:18.522873 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(56c2c24258912ed2592fbc24c75fb4ff) : [0, 150078]: Setting download as failed: File segment: [0, 150078], key: 56c2c24258912ed2592fbc24c75fb4ff, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:18.525086 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(56c2c24258912ed2592fbc24c75fb4ff): No space left in cache to reserve 150079 bytes, will continue without cache download 2024.10.22 00:54:18.571453 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_256_65} MergedBlockOutputStream: filled checksums 202410_1_256_65 (state Temporary) 2024.10.22 00:54:18.595372 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_251_50/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:54:18.670748 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_256_65} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_256_65 to 202410_1_256_65 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:18.674194 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_256_65} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_255_64, 202410_256_256_0] -> 202410_1_256_65 2024.10.22 00:54:18.674455 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 56c2c24258912ed2592fbc24c75fb4ff, offset: 0, size: 0 2024.10.22 00:54:18.678064 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 56c2c24258912ed2592fbc24c75fb4ff for removal 2024.10.22 00:54:18.680314 [ 1476 ] {} CacheMetadata: Key 56c2c24258912ed2592fbc24c75fb4ff is removed from metadata 2024.10.22 00:54:18.695320 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.40 MiB. 2024.10.22 00:54:18.961137 [ 1709 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bgo/ukpnjcifiwygwyujttniikuumxvur, size 3500979 2024.10.22 00:54:19.502754 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column HID 2024.10.22 00:54:19.531143 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column HID 2024.10.22 00:54:19.530411 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_346_72, 202410_347_347_0] 2024.10.22 00:54:19.536448 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_346_72, 202410_347_347_0] 2024.10.22 00:54:19.550554 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_346_72, 202410_347_347_0] 2024.10.22 00:54:19.666393 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column HID 2024.10.22 00:54:19.677952 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column HID 2024.10.22 00:54:19.686946 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column HID 2024.10.22 00:54:19.700895 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 8.48 MiB, will set to 1.42 GiB (RSS), difference: 18.78 MiB 2024.10.22 00:54:19.720213 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column HID 2024.10.22 00:54:19.750560 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column HID 2024.10.22 00:54:19.771779 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:54:19.776434 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:54:19.803349 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lfy/adieuqaykmzqiujpwuoxrrupvsqpr (LogSeriesLimiter: on interval from 2024-10-22 00:54:02 to 2024-10-22 00:54:19 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:54:20.327509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9230638ae333014f0eba2aa7cae005b5) : [0, 241424]: Setting download as failed: File segment: [0, 241424], key: 9230638ae333014f0eba2aa7cae005b5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:20.332021 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9230638ae333014f0eba2aa7cae005b5): No space left in cache to reserve 241425 bytes, will continue without cache download 2024.10.22 00:54:20.385693 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9230638ae333014f0eba2aa7cae005b5, offset: 0, size: 0 2024.10.22 00:54:20.388112 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9230638ae333014f0eba2aa7cae005b5 for removal 2024.10.22 00:54:20.390257 [ 1476 ] {} CacheMetadata: Key 9230638ae333014f0eba2aa7cae005b5 is removed from metadata 2024.10.22 00:54:20.520386 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 24998 2024.10.22 00:54:20.852228 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:20.857330 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:20.921509 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_438_438_0 (state Temporary) 2024.10.22 00:54:21.072537 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_438_438_0 to 202410_451_451_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:21.074336 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_51_51_0 to 202410_54_54_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:21.131307 [ 1185 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_49_10 to 202410_54_54_0 2024.10.22 00:54:21.132850 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 24998 2024.10.22 00:54:21.147264 [ 1185 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.21 MiB using storage policy from min volume index 0 2024.10.22 00:54:21.151727 [ 1185 ] {} DiskLocal: Reserved 1.21 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:21.287033 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_49_10 to 202410_54_54_0 into Compact with storage Full 2024.10.22 00:54:21.659235 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 10.95 MiB, will set to 1.43 GiB (RSS), difference: -1.42 MiB 2024.10.22 00:54:21.720282 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ea4d5cb24ebd9b992a5d71c306252478) : [0, 233006]: Setting download as failed: File segment: [0, 233006], key: ea4d5cb24ebd9b992a5d71c306252478, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:21.723542 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ea4d5cb24ebd9b992a5d71c306252478): No space left in cache to reserve 233007 bytes, will continue without cache download 2024.10.22 00:54:21.777257 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ea4d5cb24ebd9b992a5d71c306252478, offset: 0, size: 0 2024.10.22 00:54:21.780830 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ea4d5cb24ebd9b992a5d71c306252478 for removal 2024.10.22 00:54:21.784444 [ 1476 ] {} CacheMetadata: Key ea4d5cb24ebd9b992a5d71c306252478 is removed from metadata 2024.10.22 00:54:22.047682 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1653 entries to flush up to offset 420001 2024.10.22 00:54:23.022013 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:23.025157 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:23.168941 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ff3833ea72525c0d5fe1bcb3b30090a3) : [0, 114877]: Setting download as failed: File segment: [0, 114877], key: ff3833ea72525c0d5fe1bcb3b30090a3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:23.171733 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ff3833ea72525c0d5fe1bcb3b30090a3): No space left in cache to reserve 114878 bytes, will continue without cache download 2024.10.22 00:54:23.215431 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ff3833ea72525c0d5fe1bcb3b30090a3, offset: 0, size: 0 2024.10.22 00:54:23.218122 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ff3833ea72525c0d5fe1bcb3b30090a3 for removal 2024.10.22 00:54:23.223295 [ 1476 ] {} CacheMetadata: Key ff3833ea72525c0d5fe1bcb3b30090a3 is removed from metadata 2024.10.22 00:54:23.270826 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_245_245_0 (state Temporary) 2024.10.22 00:54:23.476084 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 11.02 MiB, will set to 1.43 GiB (RSS), difference: -7.40 MiB 2024.10.22 00:54:23.476204 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_245_245_0 to 202410_252_252_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:23.502302 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 420001 2024.10.22 00:54:23.777815 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 17138 2024.10.22 00:54:23.789640 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 3639 2024.10.22 00:54:23.932070 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:23.940020 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:23.998051 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_257_257_0 (state Temporary) 2024.10.22 00:54:24.057491 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_257_257_0 to 202410_270_270_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:24.127249 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:24.135278 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:24.172640 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(b4988f305950f6bb511fe3f66cdc6d67) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b4988f305950f6bb511fe3f66cdc6d67, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 00:54:24.176775 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: b4988f305950f6bb511fe3f66cdc6d67, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 00:54:24.180744 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:54:24.203853 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3639 2024.10.22 00:54:24.290153 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_255_255_0 (state Temporary) 2024.10.22 00:54:24.437606 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_255_255_0 to 202410_261_261_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:24.479277 [ 1209 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_256_65 to 202410_257_257_0 2024.10.22 00:54:24.484100 [ 1209 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:24.497841 [ 1209 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:24.529318 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 17138 2024.10.22 00:54:24.561631 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_257_66} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_256_65 to 202410_257_257_0 into Compact with storage Full 2024.10.22 00:54:24.684144 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_257_66} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:24.704113 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_257_66} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_256_65, total 16825 rows starting from the beginning of the part 2024.10.22 00:54:24.733365 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_257_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_257_257_0, total 49 rows starting from the beginning of the part 2024.10.22 00:54:25.095772 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3ff9d6e583366a9551e882adf34094f2) : [0, 110360]: Setting download as failed: File segment: [0, 110360], key: 3ff9d6e583366a9551e882adf34094f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:25.097249 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3ff9d6e583366a9551e882adf34094f2): No space left in cache to reserve 110361 bytes, will continue without cache download 2024.10.22 00:54:25.166186 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 25051 2024.10.22 00:54:25.229229 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3ff9d6e583366a9551e882adf34094f2, offset: 0, size: 0 2024.10.22 00:54:25.232610 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3ff9d6e583366a9551e882adf34094f2 for removal 2024.10.22 00:54:25.235886 [ 1476 ] {} CacheMetadata: Key 3ff9d6e583366a9551e882adf34094f2 is removed from metadata 2024.10.22 00:54:25.142498 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_206_41, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0, 202410_210_210_0, 202410_211_211_0] 2024.10.22 00:54:25.272000 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_206_41, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0, 202410_210_210_0, 202410_211_211_0] 2024.10.22 00:54:25.392758 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_206_41, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0, 202410_210_210_0, 202410_211_211_0] 2024.10.22 00:54:25.428436 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:25.432009 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:25.423484 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2048 2024.10.22 00:54:25.600264 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_439_439_0 (state Temporary) 2024.10.22 00:54:25.756417 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_439_439_0 to 202410_452_452_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:25.782460 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 13.87 MiB, will set to 1.43 GiB (RSS), difference: -13.49 MiB 2024.10.22 00:54:26.040522 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25051 2024.10.22 00:54:26.054861 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 225 2024.10.22 00:54:26.248992 [ 1192 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:54:26.279821 [ 1192 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:54:26.506889 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_257_66} MergeTask::MergeProjectionsStage: Merge sorted 16874 rows, containing 13 columns (13 merged, 0 gathered) in 1.968914715 sec., 8570.203610876057 rows/sec., 3.60 MiB/sec. 2024.10.22 00:54:26.577003 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_257_66} MergedBlockOutputStream: filled checksums 202410_1_257_66 (state Temporary) 2024.10.22 00:54:26.612831 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c9a374677cdb6a877015c25a4c403901) : [0, 69216]: Setting download as failed: File segment: [0, 69216], key: c9a374677cdb6a877015c25a4c403901, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:26.615133 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c9a374677cdb6a877015c25a4c403901): No space left in cache to reserve 69217 bytes, will continue without cache download 2024.10.22 00:54:26.639854 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c9a374677cdb6a877015c25a4c403901, offset: 0, size: 0 2024.10.22 00:54:26.645111 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c9a374677cdb6a877015c25a4c403901 for removal 2024.10.22 00:54:26.653471 [ 1476 ] {} CacheMetadata: Key c9a374677cdb6a877015c25a4c403901 is removed from metadata 2024.10.22 00:54:26.676646 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_257_66} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_257_66 to 202410_1_257_66 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:26.679346 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_257_66} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_256_65, 202410_257_257_0] -> 202410_1_257_66 2024.10.22 00:54:26.695199 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:26.698206 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:26.807530 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.41 MiB. 2024.10.22 00:54:27.080205 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_207_207_0 (state Temporary) 2024.10.22 00:54:27.273949 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_347_73, 202410_348_348_0, 202410_349_349_0, 202410_350_350_0, 202410_351_351_0, 202410_352_352_0] 2024.10.22 00:54:27.277457 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_347_73, 202410_348_348_0, 202410_349_349_0, 202410_350_350_0, 202410_351_351_0, 202410_352_352_0] 2024.10.22 00:54:27.342604 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_207_207_0 to 202410_217_217_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:27.437372 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_347_73, 202410_348_348_0, 202410_349_349_0, 202410_350_350_0, 202410_351_351_0, 202410_352_352_0] 2024.10.22 00:54:27.476921 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 225 2024.10.22 00:54:27.592865 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(40ed0339cbfd20cd13b43a5e2a4a5d00) : [0, 228201]: Setting download as failed: File segment: [0, 228201], key: 40ed0339cbfd20cd13b43a5e2a4a5d00, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:27.594289 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(40ed0339cbfd20cd13b43a5e2a4a5d00): No space left in cache to reserve 228202 bytes, will continue without cache download 2024.10.22 00:54:27.618444 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 40ed0339cbfd20cd13b43a5e2a4a5d00, offset: 0, size: 0 2024.10.22 00:54:27.620697 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 40ed0339cbfd20cd13b43a5e2a4a5d00 for removal 2024.10.22 00:54:27.623642 [ 1476 ] {} CacheMetadata: Key 40ed0339cbfd20cd13b43a5e2a4a5d00 is removed from metadata 2024.10.22 00:54:28.651889 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.88 GiB, free memory in arenas 13.00 MiB, will set to 1.43 GiB (RSS), difference: 26.18 MiB 2024.10.22 00:54:29.020792 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 5712 bytes for 7ad8e3a73ce69e5310a0dd057011fb1a:0) 2024.10.22 00:54:29.023058 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c56b778ae3c520988c4a0e621f65a753, offset: 0, size: 168 2024.10.22 00:54:29.026245 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c56b778ae3c520988c4a0e621f65a753 for removal 2024.10.22 00:54:29.029427 [ 1476 ] {} CacheMetadata: Key c56b778ae3c520988c4a0e621f65a753 is removed from metadata 2024.10.22 00:54:29.030504 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b9dfd10a578bf6b67699d23037659552, offset: 0, size: 286 2024.10.22 00:54:29.047250 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b9dfd10a578bf6b67699d23037659552 for removal 2024.10.22 00:54:29.153456 [ 1476 ] {} CacheMetadata: Key b9dfd10a578bf6b67699d23037659552 is removed from metadata 2024.10.22 00:54:29.666891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 162 bytes for 1435fe972a9a0744fd5f3d6b25b96c04:0) 2024.10.22 00:54:29.668458 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ab521fe03523c479234d9838af8191a6, offset: 0, size: 192 2024.10.22 00:54:29.672014 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ab521fe03523c479234d9838af8191a6 for removal 2024.10.22 00:54:29.673954 [ 1476 ] {} CacheMetadata: Key ab521fe03523c479234d9838af8191a6 is removed from metadata 2024.10.22 00:54:29.772642 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:54:29.776780 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:54:29.793682 [ 1419 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Found 4 old parts to remove. Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0] 2024.10.22 00:54:29.895914 [ 1419 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Removing 4 parts from filesystem (serially): Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0] 2024.10.22 00:54:30.016373 [ 1419 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Removing 4 parts from memory: Parts: [202410_1_1_0, 202410_2_2_0, 202410_3_3_0, 202410_4_4_0] 2024.10.22 00:54:30.065692 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 25108 2024.10.22 00:54:30.252553 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:30.267085 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:30.308167 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_440_440_0 (state Temporary) 2024.10.22 00:54:30.335304 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_251_50/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:54:30.491984 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 9.38 MiB, will set to 1.43 GiB (RSS), difference: -11.54 MiB 2024.10.22 00:54:30.507791 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1419 entries to flush up to offset 421420 2024.10.22 00:54:30.599790 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_198_41, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0, 202410_202_202_0, 202410_203_203_0] 2024.10.22 00:54:30.604440 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_198_41, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0, 202410_202_202_0, 202410_203_203_0] 2024.10.22 00:54:30.611396 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_440_440_0 to 202410_453_453_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:30.676667 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25108 2024.10.22 00:54:30.739420 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_198_41, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0, 202410_202_202_0, 202410_203_203_0] 2024.10.22 00:54:31.048258 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(966c4ab25129f6fd8a12455706de9f32) : [0, 58156]: Setting download as failed: File segment: [0, 58156], key: 966c4ab25129f6fd8a12455706de9f32, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:31.053784 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(966c4ab25129f6fd8a12455706de9f32): No space left in cache to reserve 58157 bytes, will continue without cache download 2024.10.22 00:54:31.079679 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 966c4ab25129f6fd8a12455706de9f32, offset: 0, size: 0 2024.10.22 00:54:31.081670 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 966c4ab25129f6fd8a12455706de9f32 for removal 2024.10.22 00:54:31.084900 [ 1476 ] {} CacheMetadata: Key 966c4ab25129f6fd8a12455706de9f32 is removed from metadata 2024.10.22 00:54:31.567028 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:31.569392 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:31.710375 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 3647 2024.10.22 00:54:31.784390 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_246_246_0 (state Temporary) 2024.10.22 00:54:31.918114 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 162 bytes for 0f25658a46abe68afb650f4f5db5e817:0) 2024.10.22 00:54:31.922686 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 18a14948bea560eed77f3162467a432d, offset: 0, size: 5082 2024.10.22 00:54:31.927091 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 18a14948bea560eed77f3162467a432d for removal 2024.10.22 00:54:31.932295 [ 1476 ] {} CacheMetadata: Key 18a14948bea560eed77f3162467a432d is removed from metadata 2024.10.22 00:54:31.982880 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:31.985579 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:32.002581 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_246_246_0 to 202410_253_253_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:32.044504 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 421420 2024.10.22 00:54:32.053471 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 81 entries to flush up to offset 17219 2024.10.22 00:54:32.100895 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_258_258_0 (state Temporary) 2024.10.22 00:54:32.260732 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:32.267147 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:32.304066 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_256_256_0 (state Temporary) 2024.10.22 00:54:32.377897 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_258_258_0 to 202410_271_271_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:32.426494 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3647 2024.10.22 00:54:32.506593 [ 1261 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_266_53 to 202410_271_271_0 2024.10.22 00:54:32.518785 [ 1261 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:32.523434 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:32.548031 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_266_53 to 202410_271_271_0 into Compact with storage Full 2024.10.22 00:54:32.564176 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_256_256_0 to 202410_262_262_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:32.581734 [ 1231 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_257_66 to 202410_258_258_0 2024.10.22 00:54:32.584532 [ 1231 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:32.586348 [ 1231 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:32.599974 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_258_67} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_257_66 to 202410_258_258_0 into Compact with storage Full 2024.10.22 00:54:32.633035 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_258_67} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:32.641380 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_258_67} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_257_66, total 16874 rows starting from the beginning of the part 2024.10.22 00:54:32.682210 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_258_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_258_258_0, total 103 rows starting from the beginning of the part 2024.10.22 00:54:32.704955 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:32.722683 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_266_53, total 3683 rows starting from the beginning of the part 2024.10.22 00:54:32.757258 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 17219 2024.10.22 00:54:32.795183 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(351545d64a66e03101f7438e281cadb7) : [0, 53660]: Setting download as failed: File segment: [0, 53660], key: 351545d64a66e03101f7438e281cadb7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:32.796875 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(351545d64a66e03101f7438e281cadb7): No space left in cache to reserve 53661 bytes, will continue without cache download 2024.10.22 00:54:32.866960 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_267_267_0, total 23 rows starting from the beginning of the part 2024.10.22 00:54:32.888214 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_268_268_0, total 12 rows starting from the beginning of the part 2024.10.22 00:54:32.914784 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_269_269_0, total 18 rows starting from the beginning of the part 2024.10.22 00:54:32.917167 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 351545d64a66e03101f7438e281cadb7, offset: 0, size: 0 2024.10.22 00:54:32.920509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 351545d64a66e03101f7438e281cadb7 for removal 2024.10.22 00:54:32.924481 [ 1476 ] {} CacheMetadata: Key 351545d64a66e03101f7438e281cadb7 is removed from metadata 2024.10.22 00:54:32.946199 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_270_270_0, total 9 rows starting from the beginning of the part 2024.10.22 00:54:32.970849 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_271_271_0, total 8 rows starting from the beginning of the part 2024.10.22 00:54:32.974814 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 2091 2024.10.22 00:54:33.580124 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 11.60 MiB, will set to 1.43 GiB (RSS), difference: -14.53 MiB 2024.10.22 00:54:34.078290 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a9efd81c0314efb2052a065abe07ae7b) : [0, 378516]: Setting download as failed: File segment: [0, 378516], key: a9efd81c0314efb2052a065abe07ae7b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:34.081003 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a9efd81c0314efb2052a065abe07ae7b): No space left in cache to reserve 378517 bytes, will continue without cache download 2024.10.22 00:54:34.232404 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:34.247693 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a9efd81c0314efb2052a065abe07ae7b, offset: 0, size: 0 2024.10.22 00:54:34.272216 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a9efd81c0314efb2052a065abe07ae7b for removal 2024.10.22 00:54:34.287078 [ 1476 ] {} CacheMetadata: Key a9efd81c0314efb2052a065abe07ae7b is removed from metadata 2024.10.22 00:54:34.380469 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_191_38, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0] 2024.10.22 00:54:34.384471 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_191_38, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0] 2024.10.22 00:54:34.444966 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_191_38, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0, 202410_195_195_0, 202410_196_196_0] 2024.10.22 00:54:34.488660 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_258_67/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:54:34.525573 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_258_67} MergeTask::MergeProjectionsStage: Merge sorted 16977 rows, containing 13 columns (13 merged, 0 gathered) in 1.929727495 sec., 8797.615230123463 rows/sec., 3.70 MiB/sec. 2024.10.22 00:54:34.576544 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_258_67} MergedBlockOutputStream: filled checksums 202410_1_258_67 (state Temporary) 2024.10.22 00:54:34.586898 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} MergeTask::MergeProjectionsStage: Merge sorted 3753 rows, containing 12 columns (12 merged, 0 gathered) in 2.045361902 sec., 1834.8831061780481 rows/sec., 286.81 KiB/sec. 2024.10.22 00:54:34.680073 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 25184 2024.10.22 00:54:34.761026 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} MergedBlockOutputStream: filled checksums 202410_1_271_54 (state Temporary) 2024.10.22 00:54:34.761556 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} MergeTask::MergeProjectionsStage: Merge sorted 431938 rows, containing 4 columns (4 merged, 0 gathered) in 19.681667391 sec., 21946.20970972794 rows/sec., 321.89 KiB/sec. 2024.10.22 00:54:34.980852 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 226 2024.10.22 00:54:34.982568 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_258_67} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_258_67 to 202410_1_258_67 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:34.986781 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_258_67} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_257_66, 202410_258_258_0] -> 202410_1_258_67 2024.10.22 00:54:35.001384 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.59 MiB. 2024.10.22 00:54:35.062174 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} MergedBlockOutputStream: filled checksums 202410_1_251_50 (state Temporary) 2024.10.22 00:54:35.087523 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_271_54 to 202410_1_271_54 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:35.091697 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:35.092511 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_271_54} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_266_53, 202410_271_271_0] -> 202410_1_271_54 2024.10.22 00:54:35.095688 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:35.137138 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:54:35.183086 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_441_441_0 (state Temporary) 2024.10.22 00:54:35.199973 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_251_50 to 202410_1_251_50 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:35.206168 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_251_50} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_246_49, 202410_251_251_0] -> 202410_1_251_50 2024.10.22 00:54:35.281789 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_441_441_0 to 202410_454_454_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:35.374445 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:54:35.389077 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:35.392301 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:35.451114 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 3 parts from 202410_1_449_108 to 202410_451_451_0 2024.10.22 00:54:35.463052 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:35.476006 [ 1153 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:35.477569 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25184 2024.10.22 00:54:35.587481 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.88 GiB, free memory in arenas 11.76 MiB, will set to 1.43 GiB (RSS), difference: 28.55 MiB 2024.10.22 00:54:35.600526 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_451_109} MergeTask::PrepareStage: Merging 3 parts: from 202410_1_449_108 to 202410_451_451_0 into Compact with storage Full 2024.10.22 00:54:35.653001 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_451_109} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:35.676531 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_451_109} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_449_108, total 25685 rows starting from the beginning of the part 2024.10.22 00:54:35.684536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d627e97ea3c6c4b4f31b98e529c23c5e) : [0, 284203]: Setting download as failed: File segment: [0, 284203], key: d627e97ea3c6c4b4f31b98e529c23c5e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:35.687066 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d627e97ea3c6c4b4f31b98e529c23c5e): No space left in cache to reserve 284204 bytes, will continue without cache download 2024.10.22 00:54:35.688987 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_451_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_450_450_0, total 72 rows starting from the beginning of the part 2024.10.22 00:54:35.723135 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_451_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_451_451_0, total 72 rows starting from the beginning of the part 2024.10.22 00:54:35.779112 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_208_208_0 (state Temporary) 2024.10.22 00:54:35.824622 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d627e97ea3c6c4b4f31b98e529c23c5e, offset: 0, size: 0 2024.10.22 00:54:35.828445 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d627e97ea3c6c4b4f31b98e529c23c5e for removal 2024.10.22 00:54:35.833278 [ 1476 ] {} CacheMetadata: Key d627e97ea3c6c4b4f31b98e529c23c5e is removed from metadata 2024.10.22 00:54:35.970406 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_49_10, total 1896 rows starting from the beginning of the part 2024.10.22 00:54:36.093087 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_208_208_0 to 202410_218_218_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:36.188727 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_165_34, 202410_166_166_0, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0] 2024.10.22 00:54:36.195074 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_165_34, 202410_166_166_0, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0] 2024.10.22 00:54:36.218669 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 226 2024.10.22 00:54:36.230106 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_165_34, 202410_166_166_0, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0, 202410_170_170_0] 2024.10.22 00:54:37.176671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(46d1999fa6bd23b98b3e3c82930927c1) : [0, 143613]: Setting download as failed: File segment: [0, 143613], key: 46d1999fa6bd23b98b3e3c82930927c1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:37.178824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(46d1999fa6bd23b98b3e3c82930927c1): No space left in cache to reserve 143614 bytes, will continue without cache download 2024.10.22 00:54:37.301039 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 46d1999fa6bd23b98b3e3c82930927c1, offset: 0, size: 0 2024.10.22 00:54:37.308921 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 46d1999fa6bd23b98b3e3c82930927c1 for removal 2024.10.22 00:54:37.315509 [ 1476 ] {} CacheMetadata: Key 46d1999fa6bd23b98b3e3c82930927c1 is removed from metadata 2024.10.22 00:54:37.641601 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_451_109} MergeTask::MergeProjectionsStage: Merge sorted 25829 rows, containing 13 columns (13 merged, 0 gathered) in 2.160079809 sec., 11957.428559992619 rows/sec., 1.80 MiB/sec. 2024.10.22 00:54:37.710109 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_451_109} MergedBlockOutputStream: filled checksums 202410_1_451_109 (state Temporary) 2024.10.22 00:54:37.768316 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_451_109} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_451_109 to 202410_1_451_109 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:37.772343 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_451_109} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 3 parts: [202410_1_449_108, 202410_451_451_0] -> 202410_1_451_109 2024.10.22 00:54:37.810982 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.22 MiB. 2024.10.22 00:54:38.739981 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_50_50_0, total 40 rows starting from the beginning of the part 2024.10.22 00:54:38.905482 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5f94c8405e7e899edd6998697d529815) : [0, 207337]: Setting download as failed: File segment: [0, 207337], key: 5f94c8405e7e899edd6998697d529815, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:38.909422 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5f94c8405e7e899edd6998697d529815): No space left in cache to reserve 207338 bytes, will continue without cache download 2024.10.22 00:54:39.039080 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5f94c8405e7e899edd6998697d529815, offset: 0, size: 0 2024.10.22 00:54:39.041381 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5f94c8405e7e899edd6998697d529815 for removal 2024.10.22 00:54:39.046546 [ 1476 ] {} CacheMetadata: Key 5f94c8405e7e899edd6998697d529815 is removed from metadata 2024.10.22 00:54:39.053753 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 422956 2024.10.22 00:54:39.357078 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 6.79 MiB, will set to 1.43 GiB (RSS), difference: -2.71 MiB 2024.10.22 00:54:39.492559 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 25244 2024.10.22 00:54:39.772395 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:54:39.774022 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:54:39.805692 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:39.814197 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:39.873549 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_442_442_0 (state Temporary) 2024.10.22 00:54:39.905295 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_442_442_0 to 202410_455_455_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:39.929363 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 3674 2024.10.22 00:54:39.960739 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25244 2024.10.22 00:54:40.104604 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:40.176739 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:40.109805 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:40.221220 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:40.256932 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_259_259_0 (state Temporary) 2024.10.22 00:54:40.273258 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 106 entries to flush up to offset 17325 2024.10.22 00:54:40.293242 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(663b984dd800037a3af20044a3468600) : [0, 334083]: Setting download as failed: File segment: [0, 334083], key: 663b984dd800037a3af20044a3468600, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:40.296192 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(663b984dd800037a3af20044a3468600): No space left in cache to reserve 334084 bytes, will continue without cache download 2024.10.22 00:54:40.323568 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_259_259_0 to 202410_272_272_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:40.388524 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3674 2024.10.22 00:54:40.420580 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_247_247_0 (state Temporary) 2024.10.22 00:54:40.488891 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 3.85 MiB, will set to 1.42 GiB (RSS), difference: -7.68 MiB 2024.10.22 00:54:40.586074 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_51_51_0, total 43 rows starting from the beginning of the part 2024.10.22 00:54:40.669181 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 663b984dd800037a3af20044a3468600, offset: 0, size: 0 2024.10.22 00:54:40.671300 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 663b984dd800037a3af20044a3468600 for removal 2024.10.22 00:54:40.678445 [ 1476 ] {} CacheMetadata: Key 663b984dd800037a3af20044a3468600 is removed from metadata 2024.10.22 00:54:40.720746 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_247_247_0 to 202410_254_254_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:40.771587 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 422956 2024.10.22 00:54:40.851399 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:40.856041 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:41.020275 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_257_257_0 (state Temporary) 2024.10.22 00:54:41.275959 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_257_257_0 to 202410_263_263_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:41.283896 [ 1202 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:54:41.298442 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_258_67 to 202410_259_259_0 2024.10.22 00:54:41.299410 [ 1202 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:54:41.303545 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:41.311845 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.28 GiB. 2024.10.22 00:54:41.325456 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 17325 2024.10.22 00:54:41.329623 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_259_68} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_258_67 to 202410_259_259_0 into Compact with storage Full 2024.10.22 00:54:41.370574 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_259_68} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:41.378870 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_259_68} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_258_67, total 16977 rows starting from the beginning of the part 2024.10.22 00:54:41.403614 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_259_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_259_259_0, total 91 rows starting from the beginning of the part 2024.10.22 00:54:41.499610 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:54:41.606539 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9ced19b29909ae1a1e0cddbb433b8f65) : [0, 365019]: Setting download as failed: File segment: [0, 365019], key: 9ced19b29909ae1a1e0cddbb433b8f65, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:41.611578 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9ced19b29909ae1a1e0cddbb433b8f65): No space left in cache to reserve 365020 bytes, will continue without cache download 2024.10.22 00:54:41.763249 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9ced19b29909ae1a1e0cddbb433b8f65, offset: 0, size: 0 2024.10.22 00:54:41.767851 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9ced19b29909ae1a1e0cddbb433b8f65 for removal 2024.10.22 00:54:41.784790 [ 1476 ] {} CacheMetadata: Key 9ced19b29909ae1a1e0cddbb433b8f65 is removed from metadata 2024.10.22 00:54:42.080361 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 00:54:42.181783 [ 1710 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lfy/adieuqaykmzqiujpwuoxrrupvsqpr, size 8293612 2024.10.22 00:54:42.208342 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 00:54:42.337273 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 00:54:42.476187 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 00:54:42.625760 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 00:54:42.644836 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 4.77 MiB, will set to 1.43 GiB (RSS), difference: 18.63 MiB 2024.10.22 00:54:42.660577 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_52_52_0, total 45 rows starting from the beginning of the part 2024.10.22 00:54:42.776868 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 00:54:42.812778 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 00:54:42.913039 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zed/vyfgpsanlrpdgpbmkdiokllmguthg (LogSeriesLimiter: on interval from 2024-10-22 00:54:19 to 2024-10-22 00:54:42 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:54:43.174991 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_259_68} MergeTask::MergeProjectionsStage: Merge sorted 17068 rows, containing 13 columns (13 merged, 0 gathered) in 1.853650824 sec., 9207.775153234577 rows/sec., 3.87 MiB/sec. 2024.10.22 00:54:43.237263 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_259_68} MergedBlockOutputStream: filled checksums 202410_1_259_68 (state Temporary) 2024.10.22 00:54:43.381517 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b93e7e77fea4b07d058b0968fdc031dd) : [0, 176966]: Setting download as failed: File segment: [0, 176966], key: b93e7e77fea4b07d058b0968fdc031dd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:43.387722 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b93e7e77fea4b07d058b0968fdc031dd): No space left in cache to reserve 176967 bytes, will continue without cache download 2024.10.22 00:54:43.548922 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b93e7e77fea4b07d058b0968fdc031dd, offset: 0, size: 0 2024.10.22 00:54:43.553817 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b93e7e77fea4b07d058b0968fdc031dd for removal 2024.10.22 00:54:43.560014 [ 1476 ] {} CacheMetadata: Key b93e7e77fea4b07d058b0968fdc031dd is removed from metadata 2024.10.22 00:54:43.623246 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_259_68} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_259_68 to 202410_1_259_68 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:43.722658 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 227 2024.10.22 00:54:43.727061 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_259_68} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_258_67, 202410_259_259_0] -> 202410_1_259_68 2024.10.22 00:54:43.784571 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.59 MiB. 2024.10.22 00:54:44.112024 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:44.188705 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 25313 2024.10.22 00:54:44.190493 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:44.369824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fe58b921b5ce2a9836d6d5d65afb4897) : [0, 2164209]: Setting download as failed: File segment: [0, 2164209], key: fe58b921b5ce2a9836d6d5d65afb4897, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:44.371682 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fe58b921b5ce2a9836d6d5d65afb4897): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:54:44.425828 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fe58b921b5ce2a9836d6d5d65afb4897, offset: 0, size: 0 2024.10.22 00:54:44.430054 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fe58b921b5ce2a9836d6d5d65afb4897 for removal 2024.10.22 00:54:44.434782 [ 1476 ] {} CacheMetadata: Key fe58b921b5ce2a9836d6d5d65afb4897 is removed from metadata 2024.10.22 00:54:44.465075 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_209_209_0 (state Temporary) 2024.10.22 00:54:44.674987 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:44.677067 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:44.763968 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_209_209_0 to 202410_219_219_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:44.766632 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_53_53_0, total 41 rows starting from the beginning of the part 2024.10.22 00:54:44.783186 [ 1213 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_214_44 to 202410_219_219_0 2024.10.22 00:54:44.822991 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_443_443_0 (state Temporary) 2024.10.22 00:54:44.825280 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 227 2024.10.22 00:54:44.865108 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_443_443_0 to 202410_456_456_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:44.892173 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_451_109 to 202410_456_456_0 2024.10.22 00:54:44.894967 [ 1213 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:44.898420 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:44.916479 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:44.919817 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:44.922886 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_214_44 to 202410_219_219_0 into Compact with storage Full 2024.10.22 00:54:44.945333 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_451_109 to 202410_456_456_0 into Compact with storage Full 2024.10.22 00:54:44.978527 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:44.983806 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_451_109, total 25829 rows starting from the beginning of the part 2024.10.22 00:54:45.036845 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25313 2024.10.22 00:54:45.092127 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:45.092669 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_452_452_0, total 53 rows starting from the beginning of the part 2024.10.22 00:54:45.099937 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_214_44, total 258 rows starting from the beginning of the part 2024.10.22 00:54:45.147883 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_215_215_0, total 1 rows starting from the beginning of the part 2024.10.22 00:54:45.187335 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_216_216_0, total 1 rows starting from the beginning of the part 2024.10.22 00:54:45.206111 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_453_453_0, total 57 rows starting from the beginning of the part 2024.10.22 00:54:45.210632 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_217_217_0, total 1 rows starting from the beginning of the part 2024.10.22 00:54:45.228231 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_454_454_0, total 76 rows starting from the beginning of the part 2024.10.22 00:54:45.254964 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_455_455_0, total 60 rows starting from the beginning of the part 2024.10.22 00:54:45.281471 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_456_456_0, total 69 rows starting from the beginning of the part 2024.10.22 00:54:45.358773 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_218_218_0, total 1 rows starting from the beginning of the part 2024.10.22 00:54:45.387109 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_219_219_0, total 1 rows starting from the beginning of the part 2024.10.22 00:54:45.407924 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e1856497587fe51b369312fc769b1490) : [0, 103009]: Setting download as failed: File segment: [0, 103009], key: e1856497587fe51b369312fc769b1490, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:45.411474 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e1856497587fe51b369312fc769b1490): No space left in cache to reserve 103010 bytes, will continue without cache download 2024.10.22 00:54:45.441111 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 4.33 MiB, will set to 1.42 GiB (RSS), difference: -20.65 MiB 2024.10.22 00:54:45.539866 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e1856497587fe51b369312fc769b1490, offset: 0, size: 0 2024.10.22 00:54:45.542152 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e1856497587fe51b369312fc769b1490 for removal 2024.10.22 00:54:45.544635 [ 1476 ] {} CacheMetadata: Key e1856497587fe51b369312fc769b1490 is removed from metadata 2024.10.22 00:54:46.633948 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3eef3bc417ef0b68743352ba9ed8fee5) : [0, 89002]: Setting download as failed: File segment: [0, 89002], key: 3eef3bc417ef0b68743352ba9ed8fee5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:46.636715 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3eef3bc417ef0b68743352ba9ed8fee5): No space left in cache to reserve 89003 bytes, will continue without cache download 2024.10.22 00:54:46.667122 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3eef3bc417ef0b68743352ba9ed8fee5, offset: 0, size: 0 2024.10.22 00:54:46.668914 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3eef3bc417ef0b68743352ba9ed8fee5 for removal 2024.10.22 00:54:46.671068 [ 1476 ] {} CacheMetadata: Key 3eef3bc417ef0b68743352ba9ed8fee5 is removed from metadata 2024.10.22 00:54:47.194965 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_54_54_0, total 45 rows starting from the beginning of the part 2024.10.22 00:54:47.292109 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} MergeTask::MergeProjectionsStage: Merge sorted 263 rows, containing 32 columns (32 merged, 0 gathered) in 2.377206043 sec., 110.63407851180528 rows/sec., 11.57 KiB/sec. 2024.10.22 00:54:47.349826 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} MergedBlockOutputStream: filled checksums 202410_1_219_45 (state Temporary) 2024.10.22 00:54:47.776489 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 424492 2024.10.22 00:54:47.776772 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(accbe7a170942fdcb707269f2449a610) : [0, 165105]: Setting download as failed: File segment: [0, 165105], key: accbe7a170942fdcb707269f2449a610, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:47.778092 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(accbe7a170942fdcb707269f2449a610): No space left in cache to reserve 165106 bytes, will continue without cache download 2024.10.22 00:54:47.804534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: accbe7a170942fdcb707269f2449a610, offset: 0, size: 0 2024.10.22 00:54:47.824844 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key accbe7a170942fdcb707269f2449a610 for removal 2024.10.22 00:54:47.830548 [ 1476 ] {} CacheMetadata: Key accbe7a170942fdcb707269f2449a610 is removed from metadata 2024.10.22 00:54:47.891095 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 3682 2024.10.22 00:54:48.079907 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_219_45 to 202410_1_219_45 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:48.082416 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_219_45} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_214_44, 202410_219_219_0] -> 202410_1_219_45 2024.10.22 00:54:48.119365 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:54:48.166222 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:48.169841 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:48.197307 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_260_260_0 (state Temporary) 2024.10.22 00:54:48.463788 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_260_260_0 to 202410_273_273_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:48.494289 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:48.496724 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:48.590991 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3682 2024.10.22 00:54:48.622889 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} MergeTask::MergeProjectionsStage: Merge sorted 26144 rows, containing 13 columns (13 merged, 0 gathered) in 3.684527704 sec., 7095.617701996793 rows/sec., 1.07 MiB/sec. 2024.10.22 00:54:48.765766 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} MergedBlockOutputStream: filled checksums 202410_1_456_110 (state Temporary) 2024.10.22 00:54:48.835345 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_248_248_0 (state Temporary) 2024.10.22 00:54:48.898911 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_248_248_0 to 202410_255_255_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:48.951214 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 106 entries to flush up to offset 17431 2024.10.22 00:54:49.015533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(cf5117907cf331a013ec764c8c358262) : [0, 269611]: Setting download as failed: File segment: [0, 269611], key: cf5117907cf331a013ec764c8c358262, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:49.020001 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(cf5117907cf331a013ec764c8c358262): No space left in cache to reserve 269612 bytes, will continue without cache download 2024.10.22 00:54:49.049688 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 424492 2024.10.22 00:54:49.078678 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_456_110 to 202410_1_456_110 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:49.175926 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cf5117907cf331a013ec764c8c358262, offset: 0, size: 0 2024.10.22 00:54:49.320823 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 25390 2024.10.22 00:54:49.323811 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cf5117907cf331a013ec764c8c358262 for removal 2024.10.22 00:54:49.323811 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_456_110} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_451_109, 202410_456_456_0] -> 202410_1_456_110 2024.10.22 00:54:49.327962 [ 1476 ] {} CacheMetadata: Key cf5117907cf331a013ec764c8c358262 is removed from metadata 2024.10.22 00:54:49.540892 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.44 MiB. 2024.10.22 00:54:49.583657 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:49.589139 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:49.639216 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_258_258_0 (state Temporary) 2024.10.22 00:54:49.722872 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_258_258_0 to 202410_264_264_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:49.773500 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:54:49.776095 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:54:49.784182 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_259_68 to 202410_260_260_0 2024.10.22 00:54:49.791478 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:49.796432 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:49.800894 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 17431 2024.10.22 00:54:49.844322 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_260_69} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_259_68 to 202410_260_260_0 into Compact with storage Full 2024.10.22 00:54:49.956123 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:49.960682 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:49.989326 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:49.995461 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:50.016823 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_444_444_0 (state Temporary) 2024.10.22 00:54:50.079394 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_444_444_0 to 202410_457_457_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:50.082676 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_260_69} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:50.090717 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_260_69} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_259_68, total 17068 rows starting from the beginning of the part 2024.10.22 00:54:50.119495 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25390 2024.10.22 00:54:50.173920 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_260_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_260_260_0, total 84 rows starting from the beginning of the part 2024.10.22 00:54:50.781245 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(088fbc18d4267b9b12a872c07062aebe) : [0, 188040]: Setting download as failed: File segment: [0, 188040], key: 088fbc18d4267b9b12a872c07062aebe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:50.783325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(088fbc18d4267b9b12a872c07062aebe): No space left in cache to reserve 188041 bytes, will continue without cache download 2024.10.22 00:54:50.914995 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 088fbc18d4267b9b12a872c07062aebe, offset: 0, size: 0 2024.10.22 00:54:50.919085 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 088fbc18d4267b9b12a872c07062aebe for removal 2024.10.22 00:54:50.923661 [ 1476 ] {} CacheMetadata: Key 088fbc18d4267b9b12a872c07062aebe is removed from metadata 2024.10.22 00:54:51.581440 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 2.73 MiB, will set to 1.43 GiB (RSS), difference: -8.66 MiB 2024.10.22 00:54:51.670101 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_260_69/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:54:51.698905 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_352_74, 202410_353_353_0, 202410_354_354_0, 202410_355_355_0, 202410_356_356_0, 202410_357_357_0] 2024.10.22 00:54:51.702078 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_352_74, 202410_353_353_0, 202410_354_354_0, 202410_355_355_0, 202410_356_356_0, 202410_357_357_0] 2024.10.22 00:54:51.732527 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_352_74, 202410_353_353_0, 202410_354_354_0, 202410_355_355_0, 202410_356_356_0, 202410_357_357_0] 2024.10.22 00:54:51.876135 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(09c163601a52869225f3c61ac0a49750) : [0, 534465]: Setting download as failed: File segment: [0, 534465], key: 09c163601a52869225f3c61ac0a49750, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:51.877107 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(09c163601a52869225f3c61ac0a49750): No space left in cache to reserve 534466 bytes, will continue without cache download 2024.10.22 00:54:51.879261 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_260_69} MergeTask::MergeProjectionsStage: Merge sorted 17152 rows, containing 13 columns (13 merged, 0 gathered) in 2.0563627 sec., 8340.94102173707 rows/sec., 3.51 MiB/sec. 2024.10.22 00:54:51.937830 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_260_69} MergedBlockOutputStream: filled checksums 202410_1_260_69 (state Temporary) 2024.10.22 00:54:51.945020 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 09c163601a52869225f3c61ac0a49750, offset: 0, size: 0 2024.10.22 00:54:51.952463 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 09c163601a52869225f3c61ac0a49750 for removal 2024.10.22 00:54:51.959930 [ 1476 ] {} CacheMetadata: Key 09c163601a52869225f3c61ac0a49750 is removed from metadata 2024.10.22 00:54:52.022673 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_260_69} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_260_69 to 202410_1_260_69 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:52.028565 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_260_69} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_259_68, 202410_260_260_0] -> 202410_1_260_69 2024.10.22 00:54:52.270431 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.59 MiB. 2024.10.22 00:54:52.329152 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 228 2024.10.22 00:54:52.582637 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:52.584637 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:52.645543 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_210_210_0 (state Temporary) 2024.10.22 00:54:52.716853 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_210_210_0 to 202410_220_220_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:52.792687 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 228 2024.10.22 00:54:53.421765 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c0145a7918ce5be7431b9348bc8dff06) : [0, 2007078]: Setting download as failed: File segment: [0, 2007078], key: c0145a7918ce5be7431b9348bc8dff06, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:53.426309 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c0145a7918ce5be7431b9348bc8dff06): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:54:53.460056 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 2.07 MiB, will set to 1.42 GiB (RSS), difference: 1.44 MiB 2024.10.22 00:54:53.690710 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c0145a7918ce5be7431b9348bc8dff06, offset: 0, size: 0 2024.10.22 00:54:53.691890 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c0145a7918ce5be7431b9348bc8dff06 for removal 2024.10.22 00:54:53.693716 [ 1476 ] {} CacheMetadata: Key c0145a7918ce5be7431b9348bc8dff06 is removed from metadata 2024.10.22 00:54:53.874968 [ 1160 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 00:54:53.877609 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_52_52_0 (state Temporary) 2024.10.22 00:54:54.136077 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 25453 2024.10.22 00:54:54.351307 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:54.358286 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:54.511937 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_445_445_0 (state Temporary) 2024.10.22 00:54:54.679714 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_445_445_0 to 202410_458_458_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:54.869527 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25453 2024.10.22 00:54:54.945043 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(de9cf7c81923e0094fe8dd4c3dbe8e1f) : [0, 218533]: Setting download as failed: File segment: [0, 218533], key: de9cf7c81923e0094fe8dd4c3dbe8e1f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:54.947640 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(de9cf7c81923e0094fe8dd4c3dbe8e1f): No space left in cache to reserve 218534 bytes, will continue without cache download 2024.10.22 00:54:55.007845 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: de9cf7c81923e0094fe8dd4c3dbe8e1f, offset: 0, size: 0 2024.10.22 00:54:55.011367 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key de9cf7c81923e0094fe8dd4c3dbe8e1f for removal 2024.10.22 00:54:55.016676 [ 1476 ] {} CacheMetadata: Key de9cf7c81923e0094fe8dd4c3dbe8e1f is removed from metadata 2024.10.22 00:54:55.394321 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 1.98 MiB, will set to 1.42 GiB (RSS), difference: -3.16 MiB 2024.10.22 00:54:55.604331 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 8 file segments (while reserving 7038 bytes for 64b135042af36a68eb8e394f02192bfa:0) 2024.10.22 00:54:55.612620 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c1f9f76590dcb7d1a1d1605e65523630, offset: 0, size: 9968 2024.10.22 00:54:55.619725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c1f9f76590dcb7d1a1d1605e65523630 for removal 2024.10.22 00:54:55.625004 [ 1476 ] {} CacheMetadata: Key c1f9f76590dcb7d1a1d1605e65523630 is removed from metadata 2024.10.22 00:54:55.625749 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2a3d97f75919beb66f0172c17756f46e, offset: 0, size: 163 2024.10.22 00:54:55.633805 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2a3d97f75919beb66f0172c17756f46e for removal 2024.10.22 00:54:55.643736 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4018fa50db3af63bfbd417c25fb92851, offset: 0, size: 242 2024.10.22 00:54:55.650204 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4018fa50db3af63bfbd417c25fb92851 for removal 2024.10.22 00:54:55.654076 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7403b74a1ca0cf63e8974942b30b14a2, offset: 0, size: 246 2024.10.22 00:54:55.676027 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7403b74a1ca0cf63e8974942b30b14a2 for removal 2024.10.22 00:54:55.678729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: abd8176dfc77b6c439237c624df27000, offset: 0, size: 231 2024.10.22 00:54:55.681429 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key abd8176dfc77b6c439237c624df27000 for removal 2024.10.22 00:54:55.684593 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eb19f426fbd6a0497cc9213004df520e, offset: 0, size: 196 2024.10.22 00:54:55.687910 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eb19f426fbd6a0497cc9213004df520e for removal 2024.10.22 00:54:55.691682 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 136e3adafd4517f4f9ab59414e1325e9, offset: 0, size: 236 2024.10.22 00:54:55.695348 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 136e3adafd4517f4f9ab59414e1325e9 for removal 2024.10.22 00:54:55.699543 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3c2b0d327752678b23f16c3145402fe0, offset: 0, size: 269 2024.10.22 00:54:55.703316 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3c2b0d327752678b23f16c3145402fe0 for removal 2024.10.22 00:54:55.661780 [ 1476 ] {} CacheMetadata: Key 4018fa50db3af63bfbd417c25fb92851 is removed from metadata 2024.10.22 00:54:55.774204 [ 1476 ] {} CacheMetadata: Key 3c2b0d327752678b23f16c3145402fe0 is removed from metadata 2024.10.22 00:54:55.782596 [ 1476 ] {} CacheMetadata: Key 136e3adafd4517f4f9ab59414e1325e9 is removed from metadata 2024.10.22 00:54:55.790918 [ 1476 ] {} CacheMetadata: Key eb19f426fbd6a0497cc9213004df520e is removed from metadata 2024.10.22 00:54:55.899950 [ 1476 ] {} CacheMetadata: Key abd8176dfc77b6c439237c624df27000 is removed from metadata 2024.10.22 00:54:55.907992 [ 1476 ] {} CacheMetadata: Key 7403b74a1ca0cf63e8974942b30b14a2 is removed from metadata 2024.10.22 00:54:56.020378 [ 1476 ] {} CacheMetadata: Key 2a3d97f75919beb66f0172c17756f46e is removed from metadata 2024.10.22 00:54:56.093006 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 3705 2024.10.22 00:54:56.302923 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:56.310027 [ 1190 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:54:56.311336 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:56.323318 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1374 entries to flush up to offset 425866 2024.10.22 00:54:56.331042 [ 1190 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:54:56.349619 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_261_261_0 (state Temporary) 2024.10.22 00:54:56.411861 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_261_261_0 to 202410_274_274_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:56.678320 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3705 2024.10.22 00:54:56.768910 [ 1153 ] {} DatabaseCatalog: Execution took 250 ms. 2024.10.22 00:54:56.986501 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:56.989333 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:57.275489 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_249_249_0 (state Temporary) 2024.10.22 00:54:57.316751 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b459d4dde4d051f600f8d2df5c3d4f51) : [0, 2138879]: Setting download as failed: File segment: [0, 2138879], key: b459d4dde4d051f600f8d2df5c3d4f51, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:57.320711 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b459d4dde4d051f600f8d2df5c3d4f51): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:54:57.332296 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 77 entries to flush up to offset 17508 2024.10.22 00:54:57.391056 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_249_249_0 to 202410_256_256_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:57.433712 [ 1161 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_251_50 to 202410_256_256_0 2024.10.22 00:54:57.439340 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 425866 2024.10.22 00:54:57.452028 [ 1161 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:57.470588 [ 1161 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:57.488325 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_251_50 to 202410_256_256_0 into Compact with storage Full 2024.10.22 00:54:57.624896 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b459d4dde4d051f600f8d2df5c3d4f51, offset: 0, size: 0 2024.10.22 00:54:57.632774 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b459d4dde4d051f600f8d2df5c3d4f51 for removal 2024.10.22 00:54:57.641182 [ 1476 ] {} CacheMetadata: Key b459d4dde4d051f600f8d2df5c3d4f51 is removed from metadata 2024.10.22 00:54:57.756431 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.88 GiB, free memory in arenas 2.39 MiB, will set to 1.43 GiB (RSS), difference: 1.72 MiB 2024.10.22 00:54:57.766882 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:57.771400 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:57.861417 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:57.877741 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} MergeTreeSequentialSource: Reading 54 marks from part 202410_1_251_50, total 431938 rows starting from the beginning of the part 2024.10.22 00:54:57.887885 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_252_252_0, total 1653 rows starting from the beginning of the part 2024.10.22 00:54:57.904302 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_253_253_0, total 1419 rows starting from the beginning of the part 2024.10.22 00:54:57.925681 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_254_254_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:54:57.952306 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_255_255_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:54:57.975491 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_256_256_0, total 1374 rows starting from the beginning of the part 2024.10.22 00:54:58.036208 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_259_259_0 (state Temporary) 2024.10.22 00:54:58.095945 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_259_259_0 to 202410_265_265_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:58.140677 [ 1152 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_260_69 to 202410_261_261_0 2024.10.22 00:54:58.156264 [ 1152 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:58.163676 [ 1152 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:58.172879 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 17508 2024.10.22 00:54:58.389273 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_261_70} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_260_69 to 202410_261_261_0 into Compact with storage Full 2024.10.22 00:54:58.435789 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_261_70} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:54:58.445297 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_261_70} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_260_69, total 17152 rows starting from the beginning of the part 2024.10.22 00:54:58.467642 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_261_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_261_261_0, total 90 rows starting from the beginning of the part 2024.10.22 00:54:58.878634 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 25535 2024.10.22 00:54:58.933560 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7eca9230dcb1fcdd7b864c089820a4e5) : [0, 155802]: Setting download as failed: File segment: [0, 155802], key: 7eca9230dcb1fcdd7b864c089820a4e5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:58.948148 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7eca9230dcb1fcdd7b864c089820a4e5): No space left in cache to reserve 155803 bytes, will continue without cache download 2024.10.22 00:54:58.988493 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7eca9230dcb1fcdd7b864c089820a4e5, offset: 0, size: 0 2024.10.22 00:54:58.992214 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7eca9230dcb1fcdd7b864c089820a4e5 for removal 2024.10.22 00:54:58.994714 [ 1476 ] {} CacheMetadata: Key 7eca9230dcb1fcdd7b864c089820a4e5 is removed from metadata 2024.10.22 00:54:59.200380 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:54:59.210553 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:54:59.392073 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_446_446_0 (state Temporary) 2024.10.22 00:54:59.578953 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_54_11/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:54:59.712199 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.88 GiB, free memory in arenas 6.44 MiB, will set to 1.45 GiB (RSS), difference: -1.29 MiB 2024.10.22 00:54:59.737304 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_446_446_0 to 202410_459_459_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:54:59.773461 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:54:59.776371 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:54:59.894700 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25535 2024.10.22 00:54:59.942831 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_261_70} MergeTask::MergeProjectionsStage: Merge sorted 17242 rows, containing 13 columns (13 merged, 0 gathered) in 1.561230345 sec., 11043.854006053156 rows/sec., 4.65 MiB/sec. 2024.10.22 00:54:59.954829 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(47d3d7c9f4ce3f2db906a9fff3f13f30) : [0, 131216]: Setting download as failed: File segment: [0, 131216], key: 47d3d7c9f4ce3f2db906a9fff3f13f30, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:54:59.957054 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(47d3d7c9f4ce3f2db906a9fff3f13f30): No space left in cache to reserve 131217 bytes, will continue without cache download 2024.10.22 00:54:59.988811 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_261_70} MergedBlockOutputStream: filled checksums 202410_1_261_70 (state Temporary) 2024.10.22 00:55:00.171427 [ 1698 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zed/vyfgpsanlrpdgpbmkdiokllmguthg, size 54 2024.10.22 00:55:00.189776 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 47d3d7c9f4ce3f2db906a9fff3f13f30, offset: 0, size: 0 2024.10.22 00:55:00.192253 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 47d3d7c9f4ce3f2db906a9fff3f13f30 for removal 2024.10.22 00:55:00.194130 [ 1476 ] {} CacheMetadata: Key 47d3d7c9f4ce3f2db906a9fff3f13f30 is removed from metadata 2024.10.22 00:55:00.295655 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 229 2024.10.22 00:55:00.696698 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_261_70} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_261_70 to 202410_1_261_70 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:00.700445 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_261_70} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_260_69, 202410_261_261_0] -> 202410_1_261_70 2024.10.22 00:55:00.847064 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.62 MiB. 2024.10.22 00:55:00.876028 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsEvent 2024.10.22 00:55:00.882822 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:00.891381 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:00.927006 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsEvent 2024.10.22 00:55:01.169830 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsEvent 2024.10.22 00:55:01.186081 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_211_211_0 (state Temporary) 2024.10.22 00:55:01.205521 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsEvent 2024.10.22 00:55:01.354121 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsEvent 2024.10.22 00:55:01.398139 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsEvent 2024.10.22 00:55:01.483484 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_211_211_0 to 202410_221_221_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:01.648746 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsEvent 2024.10.22 00:55:01.717293 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ekj/lnabsvhcfbtyoelkwqzpihausxrjl (LogSeriesLimiter: on interval from 2024-10-22 00:54:42 to 2024-10-22 00:55:01 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:55:01.773169 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 229 2024.10.22 00:55:01.826927 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(38e69340ed05b626dcdb41292dfa1795) : [0, 130356]: Setting download as failed: File segment: [0, 130356], key: 38e69340ed05b626dcdb41292dfa1795, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:01.829757 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(38e69340ed05b626dcdb41292dfa1795): No space left in cache to reserve 130357 bytes, will continue without cache download 2024.10.22 00:55:01.847568 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 38e69340ed05b626dcdb41292dfa1795, offset: 0, size: 0 2024.10.22 00:55:01.851010 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 38e69340ed05b626dcdb41292dfa1795 for removal 2024.10.22 00:55:01.857462 [ 1476 ] {} CacheMetadata: Key 38e69340ed05b626dcdb41292dfa1795 is removed from metadata 2024.10.22 00:55:02.465534 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_52_52_0 to 202410_55_55_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:02.699904 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(38899dbf63d640fddd38a1cfc24a5db1) : [0, 187287]: Setting download as failed: File segment: [0, 187287], key: 38899dbf63d640fddd38a1cfc24a5db1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:02.700651 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(38899dbf63d640fddd38a1cfc24a5db1): No space left in cache to reserve 187288 bytes, will continue without cache download 2024.10.22 00:55:02.742743 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 38899dbf63d640fddd38a1cfc24a5db1, offset: 0, size: 0 2024.10.22 00:55:02.749366 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 38899dbf63d640fddd38a1cfc24a5db1 for removal 2024.10.22 00:55:02.752902 [ 1476 ] {} CacheMetadata: Key 38899dbf63d640fddd38a1cfc24a5db1 is removed from metadata 2024.10.22 00:55:03.444735 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_256_51/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:55:03.909038 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 25587 2024.10.22 00:55:04.089755 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fb6e9f0f7b7cf5a59392ce12ea10e18f) : [0, 77878]: Setting download as failed: File segment: [0, 77878], key: fb6e9f0f7b7cf5a59392ce12ea10e18f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:04.091014 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fb6e9f0f7b7cf5a59392ce12ea10e18f): No space left in cache to reserve 77879 bytes, will continue without cache download 2024.10.22 00:55:04.122616 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:04.126351 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:04.179930 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 3714 2024.10.22 00:55:04.187674 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_447_447_0 (state Temporary) 2024.10.22 00:55:04.303782 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_211_42, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0, 202410_215_215_0, 202410_216_216_0] 2024.10.22 00:55:04.307476 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_211_42, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0, 202410_215_215_0, 202410_216_216_0] 2024.10.22 00:55:04.335121 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fb6e9f0f7b7cf5a59392ce12ea10e18f, offset: 0, size: 0 2024.10.22 00:55:04.339544 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fb6e9f0f7b7cf5a59392ce12ea10e18f for removal 2024.10.22 00:55:04.344847 [ 1476 ] {} CacheMetadata: Key fb6e9f0f7b7cf5a59392ce12ea10e18f is removed from metadata 2024.10.22 00:55:04.346223 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_211_42, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0, 202410_215_215_0, 202410_216_216_0] 2024.10.22 00:55:04.373488 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_447_447_0 to 202410_460_460_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:04.453104 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1377 entries to flush up to offset 427243 2024.10.22 00:55:04.472097 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:04.477284 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:04.486013 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25587 2024.10.22 00:55:04.527677 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_262_262_0 (state Temporary) 2024.10.22 00:55:04.579307 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_262_262_0 to 202410_275_275_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:04.629283 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3714 2024.10.22 00:55:05.120613 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:05.123963 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:05.212174 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2091 2024.10.22 00:55:05.247792 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_250_250_0 (state Temporary) 2024.10.22 00:55:05.409260 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.88 GiB, free memory in arenas 14.87 MiB, will set to 1.44 GiB (RSS), difference: -2.83 MiB 2024.10.22 00:55:05.416341 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 00:55:05.683155 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 103 entries to flush up to offset 17611 2024.10.22 00:55:05.737075 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8a1ea1824a3b5c7ea5f07448354f5637) : [0, 505770]: Setting download as failed: File segment: [0, 505770], key: 8a1ea1824a3b5c7ea5f07448354f5637, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:05.741076 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8a1ea1824a3b5c7ea5f07448354f5637): No space left in cache to reserve 505771 bytes, will continue without cache download 2024.10.22 00:55:05.771416 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_250_250_0 to 202410_257_257_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:05.816592 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 427243 2024.10.22 00:55:05.982559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8a1ea1824a3b5c7ea5f07448354f5637, offset: 0, size: 0 2024.10.22 00:55:05.993634 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8a1ea1824a3b5c7ea5f07448354f5637 for removal 2024.10.22 00:55:05.996688 [ 1476 ] {} CacheMetadata: Key 8a1ea1824a3b5c7ea5f07448354f5637 is removed from metadata 2024.10.22 00:55:06.053413 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:06.060807 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:06.239453 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_260_260_0 (state Temporary) 2024.10.22 00:55:06.288415 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_260_260_0 to 202410_266_266_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:06.425669 [ 1277 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_261_70 to 202410_262_262_0 2024.10.22 00:55:06.438953 [ 1277 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:06.541591 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 17611 2024.10.22 00:55:06.444319 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:06.592145 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_262_71} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_261_70 to 202410_262_262_0 into Compact with storage Full 2024.10.22 00:55:06.661565 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_262_71} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:06.674983 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_262_71} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_261_70, total 17242 rows starting from the beginning of the part 2024.10.22 00:55:06.702667 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_262_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_262_262_0, total 81 rows starting from the beginning of the part 2024.10.22 00:55:07.051714 [ 1182 ] {} DatabaseCatalog: Execution took 281 ms. 2024.10.22 00:55:07.328471 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(32216bfdb461de661e9d55c898e95a1a) : [0, 440781]: Setting download as failed: File segment: [0, 440781], key: 32216bfdb461de661e9d55c898e95a1a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:07.330982 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(32216bfdb461de661e9d55c898e95a1a): No space left in cache to reserve 440782 bytes, will continue without cache download 2024.10.22 00:55:07.387071 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.88 GiB, free memory in arenas 15.78 MiB, will set to 1.45 GiB (RSS), difference: -3.34 MiB 2024.10.22 00:55:07.394109 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 32216bfdb461de661e9d55c898e95a1a, offset: 0, size: 0 2024.10.22 00:55:07.396645 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 32216bfdb461de661e9d55c898e95a1a for removal 2024.10.22 00:55:07.399156 [ 1476 ] {} CacheMetadata: Key 32216bfdb461de661e9d55c898e95a1a is removed from metadata 2024.10.22 00:55:07.847271 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_262_71} MergeTask::MergeProjectionsStage: Merge sorted 17323 rows, containing 13 columns (13 merged, 0 gathered) in 1.260652512 sec., 13741.296538978379 rows/sec., 5.78 MiB/sec. 2024.10.22 00:55:07.989087 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_262_71} MergedBlockOutputStream: filled checksums 202410_1_262_71 (state Temporary) 2024.10.22 00:55:08.032729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 10092 bytes for d9a15360d0eb501bf8d0dbf8a8ab8979:0) 2024.10.22 00:55:08.036015 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 690f82c694571dcd51cd26f7cd332735, offset: 0, size: 24591 2024.10.22 00:55:08.041273 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 690f82c694571dcd51cd26f7cd332735 for removal 2024.10.22 00:55:08.047076 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 61f9a72819b1ef963d7c8e1c7ad319f9, offset: 0, size: 170 2024.10.22 00:55:08.055806 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 61f9a72819b1ef963d7c8e1c7ad319f9 for removal 2024.10.22 00:55:08.064991 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9655f088e01d17633ba128a40d0890aa, offset: 0, size: 163 2024.10.22 00:55:08.069819 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9655f088e01d17633ba128a40d0890aa for removal 2024.10.22 00:55:08.072658 [ 1476 ] {} CacheMetadata: Key 690f82c694571dcd51cd26f7cd332735 is removed from metadata 2024.10.22 00:55:08.073342 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 141e26fc6a8ef1abc8626259612a6a2d, offset: 0, size: 168 2024.10.22 00:55:08.077159 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 141e26fc6a8ef1abc8626259612a6a2d for removal 2024.10.22 00:55:08.081834 [ 1476 ] {} CacheMetadata: Key 141e26fc6a8ef1abc8626259612a6a2d is removed from metadata 2024.10.22 00:55:08.090256 [ 1476 ] {} CacheMetadata: Key 9655f088e01d17633ba128a40d0890aa is removed from metadata 2024.10.22 00:55:08.122004 [ 1476 ] {} CacheMetadata: Key 61f9a72819b1ef963d7c8e1c7ad319f9 is removed from metadata 2024.10.22 00:55:08.230721 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_262_71} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_262_71 to 202410_1_262_71 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:08.245378 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_262_71} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_261_70, 202410_262_262_0] -> 202410_1_262_71 2024.10.22 00:55:08.414746 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.87 MiB. 2024.10.22 00:55:08.494560 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 25659 2024.10.22 00:55:08.652694 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:08.658678 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:08.721009 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_448_448_0 (state Temporary) 2024.10.22 00:55:08.789240 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_448_448_0 to 202410_461_461_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:08.820843 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_456_110 to 202410_457_457_0 2024.10.22 00:55:08.825762 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:08.829110 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:08.876802 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_457_111} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_456_110 to 202410_457_457_0 into Compact with storage Full 2024.10.22 00:55:08.939478 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25659 2024.10.22 00:55:09.007205 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_457_111} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:09.045175 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_457_111} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_456_110, total 26144 rows starting from the beginning of the part 2024.10.22 00:55:09.167414 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_457_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_457_457_0, total 77 rows starting from the beginning of the part 2024.10.22 00:55:09.331459 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 00:55:09.699918 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 18.12 MiB, will set to 1.45 GiB (RSS), difference: 5.06 MiB 2024.10.22 00:55:09.775276 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:55:09.776996 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:55:09.785026 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5d282a87d723b92365dee04739f522c9) : [0, 267719]: Setting download as failed: File segment: [0, 267719], key: 5d282a87d723b92365dee04739f522c9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:09.787320 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5d282a87d723b92365dee04739f522c9): No space left in cache to reserve 267720 bytes, will continue without cache download 2024.10.22 00:55:09.928336 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5d282a87d723b92365dee04739f522c9, offset: 0, size: 0 2024.10.22 00:55:09.930903 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5d282a87d723b92365dee04739f522c9 for removal 2024.10.22 00:55:10.036518 [ 1476 ] {} CacheMetadata: Key 5d282a87d723b92365dee04739f522c9 is removed from metadata 2024.10.22 00:55:11.174248 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6fee7da5dad6a1135bd4d1de30e0b859) : [0, 466784]: Setting download as failed: File segment: [0, 466784], key: 6fee7da5dad6a1135bd4d1de30e0b859, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:11.175334 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6fee7da5dad6a1135bd4d1de30e0b859): No space left in cache to reserve 466785 bytes, will continue without cache download 2024.10.22 00:55:11.317381 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6fee7da5dad6a1135bd4d1de30e0b859, offset: 0, size: 0 2024.10.22 00:55:11.319867 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6fee7da5dad6a1135bd4d1de30e0b859 for removal 2024.10.22 00:55:11.341239 [ 1227 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:55:11.423100 [ 1476 ] {} CacheMetadata: Key 6fee7da5dad6a1135bd4d1de30e0b859 is removed from metadata 2024.10.22 00:55:11.483144 [ 1227 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:55:11.598541 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_457_111} MergeTask::MergeProjectionsStage: Merge sorted 26221 rows, containing 13 columns (13 merged, 0 gathered) in 2.726363328 sec., 9617.573611964312 rows/sec., 1.45 MiB/sec. 2024.10.22 00:55:11.619212 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_203_42, 202410_204_204_0] 2024.10.22 00:55:11.622450 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_203_42, 202410_204_204_0] 2024.10.22 00:55:11.748982 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_203_42, 202410_204_204_0] 2024.10.22 00:55:11.754207 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_457_111} MergedBlockOutputStream: filled checksums 202410_1_457_111 (state Temporary) 2024.10.22 00:55:11.689731 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 17.60 MiB, will set to 1.45 GiB (RSS), difference: 11.60 MiB 2024.10.22 00:55:11.915965 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_457_111} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_457_111 to 202410_1_457_111 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:11.921036 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_457_111} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_456_110, 202410_457_457_0] -> 202410_1_457_111 2024.10.22 00:55:11.960541 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.21 MiB. 2024.10.22 00:55:12.133000 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 3725 2024.10.22 00:55:12.411724 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:12.416309 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:12.456041 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7f9232850a0b4d48022d003938428d43) : [0, 583057]: Setting download as failed: File segment: [0, 583057], key: 7f9232850a0b4d48022d003938428d43, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:12.461489 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7f9232850a0b4d48022d003938428d43): No space left in cache to reserve 583058 bytes, will continue without cache download 2024.10.22 00:55:12.555133 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_263_263_0 (state Temporary) 2024.10.22 00:55:12.593404 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7f9232850a0b4d48022d003938428d43, offset: 0, size: 0 2024.10.22 00:55:12.597524 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7f9232850a0b4d48022d003938428d43 for removal 2024.10.22 00:55:12.598141 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_263_263_0 to 202410_276_276_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:12.599128 [ 1476 ] {} CacheMetadata: Key 7f9232850a0b4d48022d003938428d43 is removed from metadata 2024.10.22 00:55:12.646508 [ 1257 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_271_54 to 202410_276_276_0 2024.10.22 00:55:12.665207 [ 1257 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:12.742609 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3725 2024.10.22 00:55:12.763912 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 2131 2024.10.22 00:55:12.768535 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:12.772569 [ 1257 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 279 ms. 2024.10.22 00:55:12.788769 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_271_54 to 202410_276_276_0 into Compact with storage Full 2024.10.22 00:55:12.821259 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1761 entries to flush up to offset 429004 2024.10.22 00:55:12.890795 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_54_11/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:55:12.949299 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 25716 2024.10.22 00:55:12.951333 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:12.959309 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_271_54, total 3753 rows starting from the beginning of the part 2024.10.22 00:55:12.984105 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_272_272_0, total 27 rows starting from the beginning of the part 2024.10.22 00:55:13.112471 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_273_273_0, total 8 rows starting from the beginning of the part 2024.10.22 00:55:13.142878 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_274_274_0, total 23 rows starting from the beginning of the part 2024.10.22 00:55:13.240920 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_275_275_0, total 9 rows starting from the beginning of the part 2024.10.22 00:55:13.284440 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:13.358975 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:13.461274 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:13.467797 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:13.479471 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_276_276_0, total 11 rows starting from the beginning of the part 2024.10.22 00:55:13.518420 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_449_449_0 (state Temporary) 2024.10.22 00:55:13.591541 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_251_251_0 (state Temporary) 2024.10.22 00:55:13.595226 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bf0700fcceaa299d561dad4656e37807) : [0, 700834]: Setting download as failed: File segment: [0, 700834], key: bf0700fcceaa299d561dad4656e37807, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:13.598553 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bf0700fcceaa299d561dad4656e37807): No space left in cache to reserve 700835 bytes, will continue without cache download 2024.10.22 00:55:13.667748 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_449_449_0 to 202410_462_462_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:13.701518 [ 1260 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_457_111 to 202410_458_458_0 2024.10.22 00:55:13.705335 [ 1260 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:13.722069 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25716 2024.10.22 00:55:13.769567 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_251_251_0 to 202410_258_258_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:13.770440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bf0700fcceaa299d561dad4656e37807, offset: 0, size: 0 2024.10.22 00:55:13.771730 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bf0700fcceaa299d561dad4656e37807 for removal 2024.10.22 00:55:13.773403 [ 1476 ] {} CacheMetadata: Key bf0700fcceaa299d561dad4656e37807 is removed from metadata 2024.10.22 00:55:13.708497 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:13.891526 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 429004 2024.10.22 00:55:13.903538 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.88 GiB, free memory in arenas 18.70 MiB, will set to 1.45 GiB (RSS), difference: -2.77 MiB 2024.10.22 00:55:13.940875 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_458_112} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_457_111 to 202410_458_458_0 into Compact with storage Full 2024.10.22 00:55:13.986607 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_458_112} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:13.996190 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_458_112} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_457_111, total 26221 rows starting from the beginning of the part 2024.10.22 00:55:13.996256 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_357_75, 202410_358_358_0, 202410_359_359_0, 202410_360_360_0, 202410_361_361_0, 202410_362_362_0] 2024.10.22 00:55:13.999008 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_357_75, 202410_358_358_0, 202410_359_359_0, 202410_360_360_0, 202410_361_361_0, 202410_362_362_0] 2024.10.22 00:55:14.022570 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_458_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_458_458_0, total 63 rows starting from the beginning of the part 2024.10.22 00:55:14.032642 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_357_75, 202410_358_358_0, 202410_359_359_0, 202410_360_360_0, 202410_361_361_0, 202410_362_362_0] 2024.10.22 00:55:14.068992 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 107 entries to flush up to offset 17718 2024.10.22 00:55:14.487676 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:14.490276 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:14.662225 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_261_261_0 (state Temporary) 2024.10.22 00:55:14.812320 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_261_261_0 to 202410_267_267_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:14.839690 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} MergeTask::MergeProjectionsStage: Merge sorted 3831 rows, containing 12 columns (12 merged, 0 gathered) in 2.055635185 sec., 1863.6575341553128 rows/sec., 291.46 KiB/sec. 2024.10.22 00:55:14.844984 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} MergeTask::MergeProjectionsStage: Merge sorted 439456 rows, containing 4 columns (4 merged, 0 gathered) in 17.364042126 sec., 25308.392873683584 rows/sec., 371.15 KiB/sec. 2024.10.22 00:55:14.883544 [ 1160 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_262_71 to 202410_263_263_0 2024.10.22 00:55:14.898729 [ 1160 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:14.911513 [ 1160 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:14.931569 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_263_72} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_262_71 to 202410_263_263_0 into Compact with storage Full 2024.10.22 00:55:15.016896 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} MergedBlockOutputStream: filled checksums 202410_1_276_55 (state Temporary) 2024.10.22 00:55:15.038502 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0228585cd9d5a01a521b3c95bf4bc4ab) : [0, 1095114]: Setting download as failed: File segment: [0, 1095114], key: 0228585cd9d5a01a521b3c95bf4bc4ab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:15.041248 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0228585cd9d5a01a521b3c95bf4bc4ab): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:55:15.063726 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} MergedBlockOutputStream: filled checksums 202410_1_256_51 (state Temporary) 2024.10.22 00:55:15.076609 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_263_72} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:15.180478 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_256_51 to 202410_1_256_51 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:15.183958 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_256_51} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_251_50, 202410_256_256_0] -> 202410_1_256_51 2024.10.22 00:55:15.191106 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_263_72} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_262_71, total 17323 rows starting from the beginning of the part 2024.10.22 00:55:15.201756 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 17718 2024.10.22 00:55:15.240344 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:55:15.311552 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0228585cd9d5a01a521b3c95bf4bc4ab, offset: 0, size: 0 2024.10.22 00:55:15.313470 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0228585cd9d5a01a521b3c95bf4bc4ab for removal 2024.10.22 00:55:15.317133 [ 1476 ] {} CacheMetadata: Key 0228585cd9d5a01a521b3c95bf4bc4ab is removed from metadata 2024.10.22 00:55:15.338196 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_263_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_263_263_0, total 106 rows starting from the beginning of the part 2024.10.22 00:55:15.375955 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_276_55 to 202410_1_276_55 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:15.378802 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_276_55} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_271_54, 202410_276_276_0] -> 202410_1_276_55 2024.10.22 00:55:15.400990 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:55:15.784458 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.88 GiB, free memory in arenas 14.19 MiB, will set to 1.46 GiB (RSS), difference: -6.76 MiB 2024.10.22 00:55:16.420771 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_263_72} MergeTask::MergeProjectionsStage: Merge sorted 17429 rows, containing 13 columns (13 merged, 0 gathered) in 1.496013868 sec., 11650.293070678941 rows/sec., 4.90 MiB/sec. 2024.10.22 00:55:16.425489 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_458_112} MergeTask::MergeProjectionsStage: Merge sorted 26284 rows, containing 13 columns (13 merged, 0 gathered) in 2.592077278 sec., 10140.129780498002 rows/sec., 1.53 MiB/sec. 2024.10.22 00:55:16.468210 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_263_72} MergedBlockOutputStream: filled checksums 202410_1_263_72 (state Temporary) 2024.10.22 00:55:16.470487 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_458_112} MergedBlockOutputStream: filled checksums 202410_1_458_112 (state Temporary) 2024.10.22 00:55:16.514850 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ff117976447ea3dc9197c236886d3968) : [0, 1099245]: Setting download as failed: File segment: [0, 1099245], key: ff117976447ea3dc9197c236886d3968, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:16.517075 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ff117976447ea3dc9197c236886d3968): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:55:16.616795 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_263_72} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_263_72 to 202410_1_263_72 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:16.621006 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_263_72} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_262_71, 202410_263_263_0] -> 202410_1_263_72 2024.10.22 00:55:16.764422 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.90 MiB. 2024.10.22 00:55:16.775429 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_458_112} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_458_112 to 202410_1_458_112 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:16.776147 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 230 2024.10.22 00:55:16.779401 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_458_112} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_457_111, 202410_458_458_0] -> 202410_1_458_112 2024.10.22 00:55:16.809671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ff117976447ea3dc9197c236886d3968, offset: 0, size: 0 2024.10.22 00:55:16.811365 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ff117976447ea3dc9197c236886d3968 for removal 2024.10.22 00:55:16.813500 [ 1476 ] {} CacheMetadata: Key ff117976447ea3dc9197c236886d3968 is removed from metadata 2024.10.22 00:55:17.023783 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.63 MiB. 2024.10.22 00:55:17.307042 [ 1175 ] {} DatabaseCatalog: Execution took 254 ms. 2024.10.22 00:55:17.321038 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:17.325600 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:17.517688 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_212_212_0 (state Temporary) 2024.10.22 00:55:17.598255 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_212_212_0 to 202410_222_222_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:17.726278 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 25800 2024.10.22 00:55:17.995085 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 230 2024.10.22 00:55:18.072731 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:18.075401 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:18.164838 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_196_39, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0] 2024.10.22 00:55:18.168749 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_196_39, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0] 2024.10.22 00:55:18.171641 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_450_450_0 (state Temporary) 2024.10.22 00:55:18.198713 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_196_39, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0, 202410_200_200_0, 202410_201_201_0] 2024.10.22 00:55:18.221480 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_450_450_0 to 202410_463_463_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:18.367237 [ 1167 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_458_112 to 202410_459_459_0 2024.10.22 00:55:18.376146 [ 1167 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:18.380108 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:18.388042 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25800 2024.10.22 00:55:18.403641 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_459_113} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_458_112 to 202410_459_459_0 into Compact with storage Full 2024.10.22 00:55:18.415370 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(55cabb232ba2a35e7301c7fff50481ad) : [0, 698684]: Setting download as failed: File segment: [0, 698684], key: 55cabb232ba2a35e7301c7fff50481ad, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:18.420288 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(55cabb232ba2a35e7301c7fff50481ad): No space left in cache to reserve 698685 bytes, will continue without cache download 2024.10.22 00:55:18.481625 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_459_113} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:18.483298 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.88 GiB, free memory in arenas 10.38 MiB, will set to 1.44 GiB (RSS), difference: 9.76 MiB 2024.10.22 00:55:18.490502 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_459_113} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_458_112, total 26284 rows starting from the beginning of the part 2024.10.22 00:55:18.504939 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_459_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_459_459_0, total 82 rows starting from the beginning of the part 2024.10.22 00:55:18.700660 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 55cabb232ba2a35e7301c7fff50481ad, offset: 0, size: 0 2024.10.22 00:55:18.703799 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 55cabb232ba2a35e7301c7fff50481ad for removal 2024.10.22 00:55:18.755970 [ 1675 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ekj/lnabsvhcfbtyoelkwqzpihausxrjl, size 54 2024.10.22 00:55:18.812660 [ 1476 ] {} CacheMetadata: Key 55cabb232ba2a35e7301c7fff50481ad is removed from metadata 2024.10.22 00:55:19.017353 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsParameter 2024.10.22 00:55:19.071773 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_204_43, 202410_205_205_0, 202410_206_206_0, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0] 2024.10.22 00:55:19.121563 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_204_43, 202410_205_205_0, 202410_206_206_0, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0] 2024.10.22 00:55:19.168502 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_204_43, 202410_205_205_0, 202410_206_206_0, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0] 2024.10.22 00:55:19.272969 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsParameter 2024.10.22 00:55:19.279634 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsParameter 2024.10.22 00:55:19.301611 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsParameter 2024.10.22 00:55:19.382167 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_459_113/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:55:19.524268 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsParameter 2024.10.22 00:55:19.655644 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsParameter 2024.10.22 00:55:19.690279 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsParameter 2024.10.22 00:55:19.745842 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uzm/nyfgaisqshulvgvsgndfmfidwdoav (LogSeriesLimiter: on interval from 2024-10-22 00:55:01 to 2024-10-22 00:55:19 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:55:19.776502 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:55:19.779651 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:55:19.958642 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d6d357738f15cef4eac9675a33d4be3e) : [0, 524206]: Setting download as failed: File segment: [0, 524206], key: d6d357738f15cef4eac9675a33d4be3e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:19.962376 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d6d357738f15cef4eac9675a33d4be3e): No space left in cache to reserve 524207 bytes, will continue without cache download 2024.10.22 00:55:20.092949 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d6d357738f15cef4eac9675a33d4be3e, offset: 0, size: 0 2024.10.22 00:55:20.107648 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d6d357738f15cef4eac9675a33d4be3e for removal 2024.10.22 00:55:20.111198 [ 1476 ] {} CacheMetadata: Key d6d357738f15cef4eac9675a33d4be3e is removed from metadata 2024.10.22 00:55:20.245124 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 3751 2024.10.22 00:55:20.643672 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.88 GiB, free memory in arenas 9.61 MiB, will set to 1.44 GiB (RSS), difference: -12.52 MiB 2024.10.22 00:55:20.671598 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:20.675827 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:20.699688 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_264_264_0 (state Temporary) 2024.10.22 00:55:20.895322 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1248 entries to flush up to offset 430252 2024.10.22 00:55:20.970419 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_264_264_0 to 202410_277_277_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:20.977308 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_459_113} MergeTask::MergeProjectionsStage: Merge sorted 26366 rows, containing 13 columns (13 merged, 0 gathered) in 2.579581278 sec., 10221.03867199799 rows/sec., 1.54 MiB/sec. 2024.10.22 00:55:20.999196 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3751 2024.10.22 00:55:20.999294 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ecd0473e169b7f3f8a091ada93dbecc0) : [0, 1020774]: Setting download as failed: File segment: [0, 1020774], key: ecd0473e169b7f3f8a091ada93dbecc0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:21.000940 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ecd0473e169b7f3f8a091ada93dbecc0): No space left in cache to reserve 1020775 bytes, will continue without cache download 2024.10.22 00:55:21.128203 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_459_113} MergedBlockOutputStream: filled checksums 202410_1_459_113 (state Temporary) 2024.10.22 00:55:21.279971 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ecd0473e169b7f3f8a091ada93dbecc0, offset: 0, size: 0 2024.10.22 00:55:21.281579 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ecd0473e169b7f3f8a091ada93dbecc0 for removal 2024.10.22 00:55:21.283137 [ 1476 ] {} CacheMetadata: Key ecd0473e169b7f3f8a091ada93dbecc0 is removed from metadata 2024.10.22 00:55:21.365371 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_459_113} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_459_113 to 202410_1_459_113 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:21.368463 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_459_113} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_458_112, 202410_459_459_0] -> 202410_1_459_113 2024.10.22 00:55:21.485351 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.63 MiB. 2024.10.22 00:55:21.535139 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:21.545206 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:21.675941 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_252_252_0 (state Temporary) 2024.10.22 00:55:21.835072 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_252_252_0 to 202410_259_259_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:21.968165 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 430252 2024.10.22 00:55:22.402042 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 25866 2024.10.22 00:55:22.407461 [ 1272 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 216 ms. 2024.10.22 00:55:22.412257 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0bb43d5ba066ea34cb2f8159b74f0efe) : [0, 862730]: Setting download as failed: File segment: [0, 862730], key: 0bb43d5ba066ea34cb2f8159b74f0efe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:22.418440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0bb43d5ba066ea34cb2f8159b74f0efe): No space left in cache to reserve 862731 bytes, will continue without cache download 2024.10.22 00:55:22.585313 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0bb43d5ba066ea34cb2f8159b74f0efe, offset: 0, size: 0 2024.10.22 00:55:22.587547 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_54_11/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:55:22.587870 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0bb43d5ba066ea34cb2f8159b74f0efe for removal 2024.10.22 00:55:22.590193 [ 1476 ] {} CacheMetadata: Key 0bb43d5ba066ea34cb2f8159b74f0efe is removed from metadata 2024.10.22 00:55:22.603491 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:22.607624 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:22.773765 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_451_451_0 (state Temporary) 2024.10.22 00:55:22.813545 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_451_451_0 to 202410_464_464_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:22.843314 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_459_113 to 202410_464_464_0 2024.10.22 00:55:22.867703 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:22.872628 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25866 2024.10.22 00:55:22.882632 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.27 GiB. 2024.10.22 00:55:22.903925 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_459_113 to 202410_464_464_0 into Compact with storage Full 2024.10.22 00:55:22.909105 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 131 entries to flush up to offset 17849 2024.10.22 00:55:22.950182 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:23.065450 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_459_113, total 26366 rows starting from the beginning of the part 2024.10.22 00:55:23.196417 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_460_460_0, total 52 rows starting from the beginning of the part 2024.10.22 00:55:23.235751 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_461_0, total 72 rows starting from the beginning of the part 2024.10.22 00:55:23.333362 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:23.347426 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:23.491563 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_462_462_0, total 57 rows starting from the beginning of the part 2024.10.22 00:55:23.499261 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_262_262_0 (state Temporary) 2024.10.22 00:55:23.534004 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_463_463_0, total 84 rows starting from the beginning of the part 2024.10.22 00:55:23.564376 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_262_262_0 to 202410_268_268_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:23.582932 [ 1197 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_263_72 to 202410_268_268_0 2024.10.22 00:55:23.589214 [ 1197 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.01 MiB using storage policy from min volume index 0 2024.10.22 00:55:23.590825 [ 1197 ] {} DiskLocal: Reserved 1.01 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:23.637507 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 17849 2024.10.22 00:55:23.697426 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(019526d99662297c4cc0386cb60c2957) : [0, 514176]: Setting download as failed: File segment: [0, 514176], key: 019526d99662297c4cc0386cb60c2957, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:23.723940 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_263_72 to 202410_268_268_0 into Compact with storage Full 2024.10.22 00:55:23.763308 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_464_464_0, total 66 rows starting from the beginning of the part 2024.10.22 00:55:23.799805 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(019526d99662297c4cc0386cb60c2957): No space left in cache to reserve 514177 bytes, will continue without cache download 2024.10.22 00:55:23.841622 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 019526d99662297c4cc0386cb60c2957, offset: 0, size: 0 2024.10.22 00:55:23.844116 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 019526d99662297c4cc0386cb60c2957 for removal 2024.10.22 00:55:23.848137 [ 1476 ] {} CacheMetadata: Key 019526d99662297c4cc0386cb60c2957 is removed from metadata 2024.10.22 00:55:23.860929 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:23.872372 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_263_72, total 17429 rows starting from the beginning of the part 2024.10.22 00:55:23.917063 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_264_264_0, total 106 rows starting from the beginning of the part 2024.10.22 00:55:24.187146 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_265_265_0, total 77 rows starting from the beginning of the part 2024.10.22 00:55:24.263042 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_266_266_0, total 103 rows starting from the beginning of the part 2024.10.22 00:55:24.392506 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.88 GiB, free memory in arenas 8.98 MiB, will set to 1.44 GiB (RSS), difference: -6.71 MiB 2024.10.22 00:55:24.425517 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_267_267_0, total 107 rows starting from the beginning of the part 2024.10.22 00:55:24.501294 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_268_268_0, total 131 rows starting from the beginning of the part 2024.10.22 00:55:24.863631 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bd1f65a64c3710b29da54ea96f219ba4) : [0, 512978]: Setting download as failed: File segment: [0, 512978], key: bd1f65a64c3710b29da54ea96f219ba4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:24.866348 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bd1f65a64c3710b29da54ea96f219ba4): No space left in cache to reserve 512979 bytes, will continue without cache download 2024.10.22 00:55:25.302922 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bd1f65a64c3710b29da54ea96f219ba4, offset: 0, size: 0 2024.10.22 00:55:25.313389 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bd1f65a64c3710b29da54ea96f219ba4 for removal 2024.10.22 00:55:25.318628 [ 1476 ] {} CacheMetadata: Key bd1f65a64c3710b29da54ea96f219ba4 is removed from metadata 2024.10.22 00:55:25.497535 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 231 2024.10.22 00:55:25.796128 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:25.802703 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:25.990508 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_213_213_0 (state Temporary) 2024.10.22 00:55:26.096373 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_insert_202410_213_213_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:55:26.120058 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} MergeTask::MergeProjectionsStage: Merge sorted 26697 rows, containing 13 columns (13 merged, 0 gathered) in 3.227142831 sec., 8272.642829300294 rows/sec., 1.25 MiB/sec. 2024.10.22 00:55:26.284299 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} MergedBlockOutputStream: filled checksums 202410_1_464_114 (state Temporary) 2024.10.22 00:55:26.288619 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_213_213_0 to 202410_223_223_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:26.387104 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(17ea50db7ae0a91770b780a891721897) : [0, 431564]: Setting download as failed: File segment: [0, 431564], key: 17ea50db7ae0a91770b780a891721897, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:26.388158 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(17ea50db7ae0a91770b780a891721897): No space left in cache to reserve 431565 bytes, will continue without cache download 2024.10.22 00:55:26.397199 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_464_114 to 202410_1_464_114 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:26.400935 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_464_114} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_459_113, 202410_464_464_0] -> 202410_1_464_114 2024.10.22 00:55:26.473436 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 8.02 MiB, will set to 1.45 GiB (RSS), difference: 6.15 MiB 2024.10.22 00:55:26.488490 [ 1212 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:55:26.500124 [ 1212 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:55:26.571698 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.87 MiB. 2024.10.22 00:55:26.625406 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 231 2024.10.22 00:55:26.876705 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 17ea50db7ae0a91770b780a891721897, offset: 0, size: 0 2024.10.22 00:55:26.898651 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 25935 2024.10.22 00:55:26.899428 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 17ea50db7ae0a91770b780a891721897 for removal 2024.10.22 00:55:26.901251 [ 1476 ] {} CacheMetadata: Key 17ea50db7ae0a91770b780a891721897 is removed from metadata 2024.10.22 00:55:27.048546 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} MergeTask::MergeProjectionsStage: Merge sorted 17953 rows, containing 13 columns (13 merged, 0 gathered) in 3.332052227 sec., 5387.97076904281 rows/sec., 2.27 MiB/sec. 2024.10.22 00:55:27.136293 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:27.140657 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:27.192857 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_452_452_0 (state Temporary) 2024.10.22 00:55:27.207541 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} MergedBlockOutputStream: filled checksums 202410_1_268_73 (state Temporary) 2024.10.22 00:55:27.450795 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_452_452_0 to 202410_465_465_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:27.501385 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25935 2024.10.22 00:55:27.610581 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_268_73 to 202410_1_268_73 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:27.616109 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_268_73} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_263_72, 202410_268_268_0] -> 202410_1_268_73 2024.10.22 00:55:27.759451 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.27 MiB. 2024.10.22 00:55:27.990906 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(dbbeb2a63b834418dd4f4ae8ef2688a8) : [0, 978197]: Setting download as failed: File segment: [0, 978197], key: dbbeb2a63b834418dd4f4ae8ef2688a8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:27.992024 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(dbbeb2a63b834418dd4f4ae8ef2688a8): No space left in cache to reserve 978198 bytes, will continue without cache download 2024.10.22 00:55:28.037384 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dbbeb2a63b834418dd4f4ae8ef2688a8, offset: 0, size: 0 2024.10.22 00:55:28.040252 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dbbeb2a63b834418dd4f4ae8ef2688a8 for removal 2024.10.22 00:55:28.042605 [ 1476 ] {} CacheMetadata: Key dbbeb2a63b834418dd4f4ae8ef2688a8 is removed from metadata 2024.10.22 00:55:28.100992 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:28.102953 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:28.503304 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 3774 2024.10.22 00:55:28.607587 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_170_35, 202410_171_171_0, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0] 2024.10.22 00:55:28.616448 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:28.620153 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:28.712565 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_170_35, 202410_171_171_0, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0] 2024.10.22 00:55:28.748125 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_170_35, 202410_171_171_0, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0] 2024.10.22 00:55:28.770981 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_265_265_0 (state Temporary) 2024.10.22 00:55:28.799821 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_265_265_0 to 202410_278_278_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:28.981781 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 432172 2024.10.22 00:55:29.058316 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3774 2024.10.22 00:55:29.296867 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f66b4985fa67d2c6e15032d58c8baf93) : [0, 502790]: Setting download as failed: File segment: [0, 502790], key: f66b4985fa67d2c6e15032d58c8baf93, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:29.299386 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f66b4985fa67d2c6e15032d58c8baf93): No space left in cache to reserve 502791 bytes, will continue without cache download 2024.10.22 00:55:29.573046 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f66b4985fa67d2c6e15032d58c8baf93, offset: 0, size: 0 2024.10.22 00:55:29.575594 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f66b4985fa67d2c6e15032d58c8baf93 for removal 2024.10.22 00:55:29.578484 [ 1476 ] {} CacheMetadata: Key f66b4985fa67d2c6e15032d58c8baf93 is removed from metadata 2024.10.22 00:55:29.778004 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:55:29.780075 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:55:30.050250 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:30.056447 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:30.300690 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_253_253_0 (state Temporary) 2024.10.22 00:55:30.429019 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_253_253_0 to 202410_260_260_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:30.685184 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 432172 2024.10.22 00:55:31.170751 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 17939 2024.10.22 00:55:31.182201 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5a439d222015a32155880e9fa9eb6cfa) : [0, 1946938]: Setting download as failed: File segment: [0, 1946938], key: 5a439d222015a32155880e9fa9eb6cfa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:31.183725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5a439d222015a32155880e9fa9eb6cfa): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:55:31.387893 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:31.391712 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:31.464704 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_263_263_0 (state Temporary) 2024.10.22 00:55:31.515423 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 25984 2024.10.22 00:55:31.519570 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_263_263_0 to 202410_269_269_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:31.523011 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5a439d222015a32155880e9fa9eb6cfa, offset: 0, size: 0 2024.10.22 00:55:31.524994 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5a439d222015a32155880e9fa9eb6cfa for removal 2024.10.22 00:55:31.529756 [ 1476 ] {} CacheMetadata: Key 5a439d222015a32155880e9fa9eb6cfa is removed from metadata 2024.10.22 00:55:31.554490 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_53_53_0 (state Temporary) 2024.10.22 00:55:31.574076 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 17939 2024.10.22 00:55:31.577492 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.88 GiB, free memory in arenas 4.77 MiB, will set to 1.44 GiB (RSS), difference: -10.13 MiB 2024.10.22 00:55:31.859625 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:31.866764 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:32.064280 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_453_453_0 (state Temporary) 2024.10.22 00:55:32.099841 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_453_453_0 to 202410_466_466_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:32.573444 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 25984 2024.10.22 00:55:33.004468 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1e19e90e21843028c3a9629c8f1406c1) : [0, 2235760]: Setting download as failed: File segment: [0, 2235760], key: 1e19e90e21843028c3a9629c8f1406c1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:33.011548 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1e19e90e21843028c3a9629c8f1406c1): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:55:33.370508 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1e19e90e21843028c3a9629c8f1406c1, offset: 0, size: 0 2024.10.22 00:55:33.375280 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1e19e90e21843028c3a9629c8f1406c1 for removal 2024.10.22 00:55:33.380285 [ 1476 ] {} CacheMetadata: Key 1e19e90e21843028c3a9629c8f1406c1 is removed from metadata 2024.10.22 00:55:33.694724 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 4.54 MiB, will set to 1.44 GiB (RSS), difference: 1.31 MiB 2024.10.22 00:55:34.131518 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 232 2024.10.22 00:55:34.441360 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:34.454732 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:34.744222 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_214_214_0 (state Temporary) 2024.10.22 00:55:34.856144 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d75ba0cd54d3680043f932957a7a3f15) : [0, 874102]: Setting download as failed: File segment: [0, 874102], key: d75ba0cd54d3680043f932957a7a3f15, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:34.862573 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d75ba0cd54d3680043f932957a7a3f15): No space left in cache to reserve 874103 bytes, will continue without cache download 2024.10.22 00:55:35.047304 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d75ba0cd54d3680043f932957a7a3f15, offset: 0, size: 0 2024.10.22 00:55:35.055204 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d75ba0cd54d3680043f932957a7a3f15 for removal 2024.10.22 00:55:35.059585 [ 1476 ] {} CacheMetadata: Key d75ba0cd54d3680043f932957a7a3f15 is removed from metadata 2024.10.22 00:55:35.193107 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_214_214_0 to 202410_224_224_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:35.230594 [ 1260 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_219_45 to 202410_224_224_0 2024.10.22 00:55:35.243564 [ 1260 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:35.253722 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:35.350610 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_219_45 to 202410_224_224_0 into Compact with storage Full 2024.10.22 00:55:35.384077 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 232 2024.10.22 00:55:35.469043 [ 1172 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 00:55:36.174850 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(658599d8fc652097115ece6bb4eadc17) : [0, 229249]: Setting download as failed: File segment: [0, 229249], key: 658599d8fc652097115ece6bb4eadc17, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:55:36.176495 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(658599d8fc652097115ece6bb4eadc17): No space left in cache to reserve 229250 bytes, will continue without cache download 2024.10.22 00:55:36.178021 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:36.197123 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 658599d8fc652097115ece6bb4eadc17, offset: 0, size: 0 2024.10.22 00:55:36.200040 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 658599d8fc652097115ece6bb4eadc17 for removal 2024.10.22 00:55:36.208364 [ 1476 ] {} CacheMetadata: Key 658599d8fc652097115ece6bb4eadc17 is removed from metadata 2024.10.22 00:55:36.211203 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ech/gnqlxuxuamziemjkojorbcvelgycp. 2024.10.22 00:55:36.257675 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_219_45, total 263 rows starting from the beginning of the part 2024.10.22 00:55:36.263650 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pnx/eooflzhvcxezkhttkegbnvrnyvwpf. 2024.10.22 00:55:36.416143 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 4.01 MiB, will set to 1.44 GiB (RSS), difference: 3.04 MiB 2024.10.22 00:55:36.561381 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 3779 2024.10.22 00:55:36.716288 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 26030 2024.10.22 00:55:36.741236 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_220_220_0, total 1 rows starting from the beginning of the part 2024.10.22 00:55:36.841401 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:36.847217 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:36.990913 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_266_266_0 (state Temporary) 2024.10.22 00:55:37.034353 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pnx/eooflzhvcxezkhttkegbnvrnyvwpf. 2024.10.22 00:55:37.034702 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_221_221_0, total 1 rows starting from the beginning of the part 2024.10.22 00:55:37.041187 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_16_16_0 to 201403_16_16_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:37.068994 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ech/gnqlxuxuamziemjkojorbcvelgycp. 2024.10.22 00:55:37.178173 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 16 2024.10.22 00:55:37.185288 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:37.188964 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:37.217688 [ 1156 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937) (MergerMutator): Selected 9 parts from 201403_8_8_0 to 201403_16_16_0 2024.10.22 00:55:37.283772 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(c7ff784f3a7590369f297c47ba26d7ab) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c7ff784f3a7590369f297c47ba26d7ab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 00:55:37.287243 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 70074, file segment info: File segment: [0, 33554431], key: c7ff784f3a7590369f297c47ba26d7ab, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 00:55:37.290514 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:55:37.372955 [ 1156 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 485.79 MiB using storage policy from min volume index 0 2024.10.22 00:55:37.374618 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_454_454_0 (state Temporary) 2024.10.22 00:55:37.378733 [ 1156 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 310 ms. 2024.10.22 00:55:37.380182 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_266_266_0 to 202410_279_279_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:37.395998 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTask::PrepareStage: Merging 9 parts: from 201403_8_8_0 to 201403_16_16_0 into Wide with storage Full 2024.10.22 00:55:37.458135 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_222_222_0, total 1 rows starting from the beginning of the part 2024.10.22 00:55:37.465160 [ 1684 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uzm/nyfgaisqshulvgvsgndfmfidwdoav, size 70074 2024.10.22 00:55:37.475815 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3779 2024.10.22 00:55:37.494476 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:37.511634 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:37.516230 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_17_17_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:55:37.622746 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column DontCountHits 2024.10.22 00:55:37.633077 [ 1179 ] {} DatabaseCatalog: Execution took 228 ms. 2024.10.22 00:55:37.634807 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_454_454_0 to 202410_467_467_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:37.665186 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column DontCountHits 2024.10.22 00:55:37.679120 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_16_16_0 (state Temporary) 2024.10.22 00:55:37.682051 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column DontCountHits 2024.10.22 00:55:37.692560 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1207 entries to flush up to offset 433379 2024.10.22 00:55:37.697503 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column DontCountHits 2024.10.22 00:55:37.724366 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26030 2024.10.22 00:55:37.727029 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column DontCountHits 2024.10.22 00:55:37.746869 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column DontCountHits 2024.10.22 00:55:37.773496 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column DontCountHits 2024.10.22 00:55:37.835000 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_223_223_0, total 1 rows starting from the beginning of the part 2024.10.22 00:55:37.941517 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kbg/adtutqqbtvjubnrfiggscpztlqzuz (LogSeriesLimiter: on interval from 2024-10-22 00:55:19 to 2024-10-22 00:55:37 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:55:37.979224 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_16_16_0 to 202410_16_16_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:38.061561 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_224_224_0, total 1 rows starting from the beginning of the part 2024.10.22 00:55:38.071220 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 16 2024.10.22 00:55:38.141540 [ 1423 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Found 5 old parts to remove. Parts: [202410_1_9_2, 202410_10_10_0, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0] 2024.10.22 00:55:38.162980 [ 1423 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from filesystem (serially): Parts: [202410_1_9_2, 202410_10_10_0, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0] 2024.10.22 00:55:38.193810 [ 1423 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from memory: Parts: [202410_1_9_2, 202410_10_10_0, 202410_11_11_0, 202410_12_12_0, 202410_13_13_0] 2024.10.22 00:55:38.313657 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Vertical 2024.10.22 00:55:38.371246 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part 2024.10.22 00:55:38.414125 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part 2024.10.22 00:55:38.470295 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part 2024.10.22 00:55:38.503217 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part 2024.10.22 00:55:38.694486 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:38.698866 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:38.709809 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part 2024.10.22 00:55:38.776162 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part 2024.10.22 00:55:38.862683 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part 2024.10.22 00:55:38.952141 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_254_254_0 (state Temporary) 2024.10.22 00:55:39.032892 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part 2024.10.22 00:55:39.107383 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_53_53_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:55:39.151942 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 80 entries to flush up to offset 18019 2024.10.22 00:55:39.168950 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part 2024.10.22 00:55:39.178089 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 5 old parts to remove. Parts: [202410_1_362_76, 202410_363_363_0, 202410_364_364_0, 202410_365_365_0, 202410_366_366_0] 2024.10.22 00:55:39.180052 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from filesystem (serially): Parts: [202410_1_362_76, 202410_363_363_0, 202410_364_364_0, 202410_365_365_0, 202410_366_366_0] 2024.10.22 00:55:39.215565 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_254_254_0 to 202410_261_261_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:39.213348 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from memory: Parts: [202410_1_362_76, 202410_363_363_0, 202410_364_364_0, 202410_365_365_0, 202410_366_366_0] 2024.10.22 00:55:39.272622 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 433379 2024.10.22 00:55:39.296244 [ 1190 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_256_51 to 202410_261_261_0 2024.10.22 00:55:39.311569 [ 1190 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:39.335921 [ 1190 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:39.472776 [ 1190 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 00:55:39.587077 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:39.588173 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_256_51 to 202410_261_261_0 into Compact with storage Full 2024.10.22 00:55:39.589731 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:39.634984 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:39.746089 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} MergeTreeSequentialSource: Reading 55 marks from part 202410_1_256_51, total 439456 rows starting from the beginning of the part 2024.10.22 00:55:39.770105 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_257_257_0, total 1377 rows starting from the beginning of the part 2024.10.22 00:55:39.777549 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_264_264_0 (state Temporary) 2024.10.22 00:55:39.779432 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:55:39.784047 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:55:39.786571 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_258_258_0, total 1761 rows starting from the beginning of the part 2024.10.22 00:55:39.801125 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_259_259_0, total 1248 rows starting from the beginning of the part 2024.10.22 00:55:39.891618 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 978.08 MiB, peak 1.88 GiB, free memory in arenas 18.08 MiB, will set to 1.19 GiB (RSS), difference: 243.24 MiB 2024.10.22 00:55:39.928766 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_264_264_0 to 202410_270_270_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:39.956962 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_260_260_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:55:39.978950 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_261_261_0, total 1207 rows starting from the beginning of the part 2024.10.22 00:55:39.985627 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 18019 2024.10.22 00:55:40.928950 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_53_53_0 to 202410_56_56_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:41.118697 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} MergeTask::MergeProjectionsStage: Merge sorted 268 rows, containing 32 columns (32 merged, 0 gathered) in 5.833960671 sec., 45.937916813906476 rows/sec., 4.79 KiB/sec. 2024.10.22 00:55:41.145746 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} MergedBlockOutputStream: filled checksums 202410_1_224_46 (state Temporary) 2024.10.22 00:55:41.509030 [ 1266 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:55:41.532858 [ 1266 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:55:41.732362 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 26122 2024.10.22 00:55:41.809317 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_224_46 to 202410_1_224_46 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:41.817160 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_224_46} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_219_45, 202410_224_224_0] -> 202410_1_224_46 2024.10.22 00:55:41.982785 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 1.88 GiB, free memory in arenas 19.30 MiB, will set to 1.23 GiB (RSS), difference: -70.92 MiB 2024.10.22 00:55:42.073130 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:42.076432 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:42.080485 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:55:42.246756 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_455_455_0 (state Temporary) 2024.10.22 00:55:42.720790 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_455_455_0 to 202410_468_468_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:42.886139 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 233 2024.10.22 00:55:43.155684 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26122 2024.10.22 00:55:43.432504 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:43.436868 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:43.439744 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:55:43.599988 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.28 GiB, peak 1.88 GiB, free memory in arenas 18.51 MiB, will set to 1.27 GiB (RSS), difference: -7.06 MiB 2024.10.22 00:55:43.697031 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_215_215_0 (state Temporary) 2024.10.22 00:55:43.697280 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:55:43.755675 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:55:43.782360 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_215_215_0 to 202410_225_225_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:43.894985 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 233 2024.10.22 00:55:43.928078 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:55:43.985191 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:55:44.067174 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 00:55:44.077160 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2131 2024.10.22 00:55:44.164732 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_261_52/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:55:44.979117 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 3795 2024.10.22 00:55:45.199469 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_216_43, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0, 202410_220_220_0, 202410_221_221_0] 2024.10.22 00:55:45.202232 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_216_43, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0, 202410_220_220_0, 202410_221_221_0] 2024.10.22 00:55:45.226227 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_216_43, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0, 202410_220_220_0, 202410_221_221_0] 2024.10.22 00:55:45.339524 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:45.344352 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:45.386235 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_267_267_0 (state Temporary) 2024.10.22 00:55:45.538933 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_267_267_0 to 202410_280_280_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:45.773842 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 1.88 GiB, free memory in arenas 23.47 MiB, will set to 1.31 GiB (RSS), difference: -42.83 MiB 2024.10.22 00:55:45.786052 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3795 2024.10.22 00:55:46.276365 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1865 entries to flush up to offset 435244 2024.10.22 00:55:47.176718 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:47.199420 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 26158 2024.10.22 00:55:47.205113 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:47.339626 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_255_255_0 (state Temporary) 2024.10.22 00:55:47.341720 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:47.346658 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:47.402401 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_456_456_0 (state Temporary) 2024.10.22 00:55:47.498979 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 137 entries to flush up to offset 18156 2024.10.22 00:55:47.571560 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_456_456_0 to 202410_469_469_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:47.575879 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_255_255_0 to 202410_262_262_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:47.584548 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.88 GiB, free memory in arenas 13.19 MiB, will set to 1.35 GiB (RSS), difference: -91.45 MiB 2024.10.22 00:55:47.596241 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 435244 2024.10.22 00:55:47.603663 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 5 parts from 202410_1_464_114 to 202410_468_468_0 2024.10.22 00:55:47.613560 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26158 2024.10.22 00:55:47.618724 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:47.668675 [ 1222 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 210 ms. 2024.10.22 00:55:47.723789 [ 1183 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:47.765805 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:47.768406 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:47.824125 [ 1183 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 00:55:47.907577 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_265_265_0 (state Temporary) 2024.10.22 00:55:47.962882 [ 1214 ] {} DatabaseCatalog: Execution took 332 ms. 2024.10.22 00:55:47.974963 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_468_115} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_464_114 to 202410_468_468_0 into Compact with storage Full 2024.10.22 00:55:48.032053 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_468_115} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:48.105147 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_265_265_0 to 202410_271_271_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:48.162460 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_468_115} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_464_114, total 26697 rows starting from the beginning of the part 2024.10.22 00:55:48.180651 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_468_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_465_465_0, total 69 rows starting from the beginning of the part 2024.10.22 00:55:48.191655 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 18156 2024.10.22 00:55:48.193949 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_468_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_466_466_0, total 49 rows starting from the beginning of the part 2024.10.22 00:55:48.356845 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_468_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_467_467_0, total 46 rows starting from the beginning of the part 2024.10.22 00:55:48.394891 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_468_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_468_468_0, total 92 rows starting from the beginning of the part 2024.10.22 00:55:49.583105 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.88 GiB, free memory in arenas 16.77 MiB, will set to 1.42 GiB (RSS), difference: -27.76 MiB 2024.10.22 00:55:49.779818 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:55:49.782026 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:55:49.979187 [ 1220 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 00:55:51.225534 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_54_11/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:55:51.397509 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 234 2024.10.22 00:55:51.445127 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_468_115} MergeTask::MergeProjectionsStage: Merge sorted 26953 rows, containing 13 columns (13 merged, 0 gathered) in 3.585926657 sec., 7516.327738434278 rows/sec., 1.13 MiB/sec. 2024.10.22 00:55:51.474661 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.88 GiB, free memory in arenas 16.41 MiB, will set to 1.43 GiB (RSS), difference: -57.84 MiB 2024.10.22 00:55:51.517743 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_468_115} MergedBlockOutputStream: filled checksums 202410_1_468_115 (state Temporary) 2024.10.22 00:55:51.620971 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_468_115} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_468_115 to 202410_1_468_115 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:51.622910 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 2170 2024.10.22 00:55:51.633025 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 26198 2024.10.22 00:55:51.645530 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_468_115} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 5 parts: [202410_1_464_114, 202410_468_468_0] -> 202410_1_468_115 2024.10.22 00:55:51.715148 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.79 MiB. 2024.10.22 00:55:51.871909 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:51.877609 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:51.890495 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 00:55:52.044927 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_216_216_0 (state Temporary) 2024.10.22 00:55:52.077407 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:52.079222 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:52.135856 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_457_457_0 (state Temporary) 2024.10.22 00:55:52.209037 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_457_457_0 to 202410_470_470_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:52.329650 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_216_216_0 to 202410_226_226_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:52.375014 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26198 2024.10.22 00:55:52.703279 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 234 2024.10.22 00:55:53.287669 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 3807 2024.10.22 00:55:53.441496 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:53.445359 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:53.612995 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_268_268_0 (state Temporary) 2024.10.22 00:55:53.836332 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(2597e5a36c4b699d5aa7c74e4d9b23e0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2597e5a36c4b699d5aa7c74e4d9b23e0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 00:55:53.837095 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_268_268_0 to 202410_281_281_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:53.841672 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 892195, file segment info: File segment: [0, 33554431], key: 2597e5a36c4b699d5aa7c74e4d9b23e0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 00:55:53.851538 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:55:53.879683 [ 1254 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_276_55 to 202410_281_281_0 2024.10.22 00:55:53.889298 [ 1254 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:53.893063 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:53.896071 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.88 GiB, free memory in arenas 11.55 MiB, will set to 1.44 GiB (RSS), difference: -79.33 MiB 2024.10.22 00:55:53.899157 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3807 2024.10.22 00:55:53.944412 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_276_55 to 202410_281_281_0 into Compact with storage Full 2024.10.22 00:55:53.980229 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:55:54.027239 [ 1726 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kbg/adtutqqbtvjubnrfiggscpztlqzuz, size 892195 2024.10.22 00:55:54.085572 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_276_55, total 3831 rows starting from the beginning of the part 2024.10.22 00:55:54.120414 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_277_277_0, total 26 rows starting from the beginning of the part 2024.10.22 00:55:54.169208 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_278_278_0, total 23 rows starting from the beginning of the part 2024.10.22 00:55:54.214976 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_279_279_0, total 5 rows starting from the beginning of the part 2024.10.22 00:55:54.236851 [ 1267 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 337 ms. 2024.10.22 00:55:54.260474 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_280_280_0, total 16 rows starting from the beginning of the part 2024.10.22 00:55:54.366334 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column WithHash 2024.10.22 00:55:54.381170 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column WithHash 2024.10.22 00:55:54.385155 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_281_281_0, total 12 rows starting from the beginning of the part 2024.10.22 00:55:54.391781 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column WithHash 2024.10.22 00:55:54.420625 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column WithHash 2024.10.22 00:55:54.446091 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column WithHash 2024.10.22 00:55:54.489692 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column WithHash 2024.10.22 00:55:54.532144 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column WithHash 2024.10.22 00:55:54.608017 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1433 entries to flush up to offset 436677 2024.10.22 00:55:54.612014 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xjw/iolrfvqbtqgydmkijfswhgbpijbtx (LogSeriesLimiter: on interval from 2024-10-22 00:55:37 to 2024-10-22 00:55:54 accepted series 1 / 9 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:55:55.006605 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} MergeTask::MergeProjectionsStage: Merge sorted 3913 rows, containing 12 columns (12 merged, 0 gathered) in 1.10468632 sec., 3542.182001493419 rows/sec., 554.13 KiB/sec. 2024.10.22 00:55:55.146860 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} MergedBlockOutputStream: filled checksums 202410_1_281_56 (state Temporary) 2024.10.22 00:55:55.272434 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_261_52/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:55:55.401048 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_281_56 to 202410_1_281_56 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:55.417023 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_281_56} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_276_55, 202410_281_281_0] -> 202410_1_281_56 2024.10.22 00:55:55.463638 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:55:55.494057 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} MergeTask::MergeProjectionsStage: Merge sorted 2110 rows, containing 679 columns (679 merged, 0 gathered) in 94.314915952 sec., 22.371858986481513 rows/sec., 30.57 KiB/sec. 2024.10.22 00:55:55.578775 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.88 GiB, free memory in arenas 10.50 MiB, will set to 1.47 GiB (RSS), difference: -1.10 MiB 2024.10.22 00:55:55.579444 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:55.581578 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:55.714716 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 104 entries to flush up to offset 18260 2024.10.22 00:55:55.729835 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_256_256_0 (state Temporary) 2024.10.22 00:55:55.865029 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:55.869568 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:55.895752 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_256_256_0 to 202410_263_263_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:55.900757 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_266_266_0 (state Temporary) 2024.10.22 00:55:55.943610 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} MergeTask::MergeProjectionsStage: Merge sorted 446969 rows, containing 4 columns (4 merged, 0 gathered) in 16.359427853 sec., 27321.79902722176 rows/sec., 400.67 KiB/sec. 2024.10.22 00:55:55.963397 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 436677 2024.10.22 00:55:55.983942 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} MergedBlockOutputStream: filled checksums 202410_1_261_52 (state Temporary) 2024.10.22 00:55:55.999827 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d9a81ae8138ca5ce745534dda577ab1c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d9a81ae8138ca5ce745534dda577ab1c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 00:55:56.001731 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: d9a81ae8138ca5ce745534dda577ab1c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 00:55:56.005410 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:55:56.035371 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_261_52 to 202410_1_261_52 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:56.038734 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_261_52} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_256_51, 202410_261_261_0] -> 202410_1_261_52 2024.10.22 00:55:56.091582 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:55:56.146500 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_266_266_0 to 202410_272_272_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:56.196959 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 18260 2024.10.22 00:55:56.382937 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 26273 2024.10.22 00:55:56.539101 [ 1169 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:55:56.672005 [ 1169 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:55:56.726564 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} MergedBlockOutputStream: filled checksums 202410_1_54_11 (state Temporary) 2024.10.22 00:55:56.849191 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:55:56.855433 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:55:56.992149 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 5 old parts to remove. Parts: [202410_1_366_77, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0, 202410_370_370_0] 2024.10.22 00:55:56.993406 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from filesystem (serially): Parts: [202410_1_366_77, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0, 202410_370_370_0] 2024.10.22 00:55:57.010116 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_458_458_0 (state Temporary) 2024.10.22 00:55:57.015030 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from memory: Parts: [202410_1_366_77, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0, 202410_370_370_0] 2024.10.22 00:55:57.180224 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_458_458_0 to 202410_471_471_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:55:57.295040 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26273 2024.10.22 00:55:57.595003 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.88 GiB, free memory in arenas 12.05 MiB, will set to 1.50 GiB (RSS), difference: 22.30 MiB 2024.10.22 00:55:59.565234 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_209_44, 202410_210_210_0, 202410_211_211_0, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0] 2024.10.22 00:55:59.569443 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_209_44, 202410_210_210_0, 202410_211_211_0, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0] 2024.10.22 00:55:59.599671 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_209_44, 202410_210_210_0, 202410_211_211_0, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0] 2024.10.22 00:55:59.779954 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:55:59.781317 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:55:59.782166 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.88 GiB, free memory in arenas 8.31 MiB, will set to 1.52 GiB (RSS), difference: -53.50 MiB 2024.10.22 00:56:00.205341 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 235 2024.10.22 00:56:00.385404 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:00.387640 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:00.558393 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_217_217_0 (state Temporary) 2024.10.22 00:56:00.841603 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_217_217_0 to 202410_227_227_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:01.235823 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 235 2024.10.22 00:56:01.323182 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 25 entries to flush up to offset 26298 2024.10.22 00:56:01.401109 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 3826 2024.10.22 00:56:01.452204 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:01.459080 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:01.529721 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.88 GiB, free memory in arenas 7.55 MiB, will set to 1.56 GiB (RSS), difference: -31.13 MiB 2024.10.22 00:56:01.605245 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_459_459_0 (state Temporary) 2024.10.22 00:56:01.625270 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:01.630737 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:01.664438 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_459_459_0 to 202410_472_472_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:01.679062 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_54_11/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:56:01.695340 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_269_269_0 (state Temporary) 2024.10.22 00:56:01.725089 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26298 2024.10.22 00:56:01.882095 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_269_269_0 to 202410_282_282_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:02.103500 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3826 2024.10.22 00:56:02.968059 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1639 entries to flush up to offset 438316 2024.10.22 00:56:03.121750 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_201_40, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0, 202410_205_205_0, 202410_206_206_0] 2024.10.22 00:56:03.124034 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_201_40, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0, 202410_205_205_0, 202410_206_206_0] 2024.10.22 00:56:03.265588 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_201_40, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0, 202410_205_205_0, 202410_206_206_0] 2024.10.22 00:56:03.501804 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 8.23 MiB, will set to 1.59 GiB (RSS), difference: -44.24 MiB 2024.10.22 00:56:03.843140 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:03.845588 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 42 entries to flush up to offset 18302 2024.10.22 00:56:03.848502 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:03.962275 [ 1217 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 00:56:04.106216 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_257_257_0 (state Temporary) 2024.10.22 00:56:04.085772 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:04.189551 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:04.232580 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_267_267_0 (state Temporary) 2024.10.22 00:56:04.386463 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_267_267_0 to 202410_273_273_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:04.487883 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_257_257_0 to 202410_264_264_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:04.525711 [ 1165 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 5 parts from 202410_1_268_73 to 202410_272_272_0 2024.10.22 00:56:04.548265 [ 1165 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 00:56:04.554590 [ 1165 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:04.599101 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_272_74} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_268_73 to 202410_272_272_0 into Compact with storage Full 2024.10.22 00:56:04.654062 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 438316 2024.10.22 00:56:04.660158 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 18302 2024.10.22 00:56:04.713524 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_272_74} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:04.731529 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_272_74} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_268_73, total 17953 rows starting from the beginning of the part 2024.10.22 00:56:04.882058 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_272_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_269_269_0, total 90 rows starting from the beginning of the part 2024.10.22 00:56:04.940771 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_272_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_270_270_0, total 80 rows starting from the beginning of the part 2024.10.22 00:56:05.008151 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_272_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_271_271_0, total 137 rows starting from the beginning of the part 2024.10.22 00:56:05.051808 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_272_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_272_272_0, total 104 rows starting from the beginning of the part 2024.10.22 00:56:05.578918 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 7.66 MiB, will set to 1.65 GiB (RSS), difference: -55.90 MiB 2024.10.22 00:56:05.728546 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 26340 2024.10.22 00:56:06.025415 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:06.027453 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_174_36, 202410_175_175_0, 202410_176_176_0, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0] 2024.10.22 00:56:06.032418 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:06.035127 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_174_36, 202410_175_175_0, 202410_176_176_0, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0] 2024.10.22 00:56:06.128035 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_460_460_0 (state Temporary) 2024.10.22 00:56:06.175713 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_174_36, 202410_175_175_0, 202410_176_176_0, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0] 2024.10.22 00:56:06.200516 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_460_460_0 to 202410_473_473_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:06.202835 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 00:56:06.271410 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1_468_115 to 202410_471_471_0 2024.10.22 00:56:06.281319 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:06.283645 [ 1235 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:06.356127 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_471_116} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_468_115 to 202410_471_471_0 into Compact with storage Full 2024.10.22 00:56:06.396441 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26340 2024.10.22 00:56:06.550267 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_471_116} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:06.588592 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_471_116} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_468_115, total 26953 rows starting from the beginning of the part 2024.10.22 00:56:06.694795 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_471_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_469_469_0, total 36 rows starting from the beginning of the part 2024.10.22 00:56:06.768752 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 240.82 MiB using storage policy from min volume index 0 2024.10.22 00:56:06.831243 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_471_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_470_470_0, total 40 rows starting from the beginning of the part 2024.10.22 00:56:07.206706 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_471_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_471_471_0, total 75 rows starting from the beginning of the part 2024.10.22 00:56:07.337416 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_272_74/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:56:07.751572 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_54_11 to 202410_1_54_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:07.780209 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_54_11} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_49_10, 202410_54_54_0] -> 202410_1_54_11 2024.10.22 00:56:07.807095 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ygi/ykonxchhfcqdpwjncjvykgtbyocno (LogSeriesLimiter: on interval from 2024-10-22 00:55:54 to 2024-10-22 00:56:07 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:56:07.827397 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:07.830292 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:08.047884 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 6.63 MiB, will set to 1.66 GiB (RSS), difference: -25.95 MiB 2024.10.22 00:56:08.184028 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_272_74} MergeTask::MergeProjectionsStage: Merge sorted 18364 rows, containing 13 columns (13 merged, 0 gathered) in 3.595777981 sec., 5107.100632195567 rows/sec., 2.15 MiB/sec. 2024.10.22 00:56:08.368903 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_272_74} MergedBlockOutputStream: filled checksums 202410_1_272_74 (state Temporary) 2024.10.22 00:56:08.548422 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_272_74} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_272_74 to 202410_1_272_74 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:08.555310 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_272_74} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 5 parts: [202410_1_268_73, 202410_272_272_0] -> 202410_1_272_74 2024.10.22 00:56:08.595745 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.28 MiB. 2024.10.22 00:56:08.608306 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 31.68 MiB. 2024.10.22 00:56:08.617222 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(b39493e8a5d709a92f60f0e24b5ca548) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b39493e8a5d709a92f60f0e24b5ca548, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 00:56:08.618226 [ 1244 ] {} DatabaseCatalog: Execution took 533 ms. 2024.10.22 00:56:08.626943 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 55341, file segment info: File segment: [0, 33554431], key: b39493e8a5d709a92f60f0e24b5ca548, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 00:56:08.630352 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:56:08.700006 [ 1711 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xjw/iolrfvqbtqgydmkijfswhgbpijbtx, size 55341 2024.10.22 00:56:09.226045 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column HitColor 2024.10.22 00:56:09.239168 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column HitColor 2024.10.22 00:56:09.272827 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column HitColor 2024.10.22 00:56:09.279500 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column HitColor 2024.10.22 00:56:09.290960 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column HitColor 2024.10.22 00:56:09.306184 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column HitColor 2024.10.22 00:56:09.331014 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column HitColor 2024.10.22 00:56:09.609336 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 3844 2024.10.22 00:56:09.781197 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:56:09.784241 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:56:09.942516 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:09.946256 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:09.979819 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_270_270_0 (state Temporary) 2024.10.22 00:56:10.062296 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_270_270_0 to 202410_283_283_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:10.107415 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_471_116} MergeTask::MergeProjectionsStage: Merge sorted 27104 rows, containing 13 columns (13 merged, 0 gathered) in 3.800942183 sec., 7130.863532001271 rows/sec., 1.08 MiB/sec. 2024.10.22 00:56:10.214220 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3844 2024.10.22 00:56:10.403451 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 26394 2024.10.22 00:56:10.465906 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.83 GiB, peak 1.88 GiB, free memory in arenas 6.52 MiB, will set to 1.65 GiB (RSS), difference: -179.93 MiB 2024.10.22 00:56:10.501624 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_471_116} MergedBlockOutputStream: filled checksums 202410_1_471_116 (state Temporary) 2024.10.22 00:56:10.869353 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_471_116} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_471_116 to 202410_1_471_116 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:10.874207 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_471_116} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1_468_115, 202410_471_471_0] -> 202410_1_471_116 2024.10.22 00:56:10.877324 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_54_54_0 (state Temporary) 2024.10.22 00:56:10.898084 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.72 MiB. 2024.10.22 00:56:10.919731 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.161904206 sec. 2024.10.22 00:56:11.057944 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:11.064160 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:11.111565 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_461_461_0 (state Temporary) 2024.10.22 00:56:11.529710 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_461_461_0 to 202410_474_474_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:11.584855 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26394 2024.10.22 00:56:11.660556 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 439852 2024.10.22 00:56:11.683416 [ 1151 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:56:11.704983 [ 1151 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:56:12.090837 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/czx/ioojhiffcdypmgpsoccuzlejvdomy (LogSeriesLimiter: on interval from 2024-10-22 00:56:07 to 2024-10-22 00:56:12 accepted series 1 / 233 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:56:12.174174 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 111 entries to flush up to offset 18413 2024.10.22 00:56:12.408021 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:12.414856 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:12.467138 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_268_268_0 (state Temporary) 2024.10.22 00:56:12.547930 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.84 GiB, peak 1.88 GiB, free memory in arenas 5.41 MiB, will set to 1.65 GiB (RSS), difference: -187.14 MiB 2024.10.22 00:56:12.681271 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:12.682988 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:12.712537 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_268_268_0 to 202410_274_274_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:12.772594 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 18413 2024.10.22 00:56:12.782698 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column WatchID 2024.10.22 00:56:12.791327 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column WatchID 2024.10.22 00:56:12.800440 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column WatchID 2024.10.22 00:56:12.819902 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column WatchID 2024.10.22 00:56:12.908005 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_258_258_0 (state Temporary) 2024.10.22 00:56:12.942682 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column WatchID 2024.10.22 00:56:12.973068 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column WatchID 2024.10.22 00:56:12.984590 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column WatchID 2024.10.22 00:56:13.003416 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column WatchID 2024.10.22 00:56:13.018455 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column WatchID 2024.10.22 00:56:13.099115 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_258_258_0 to 202410_265_265_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:13.240987 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 439852 2024.10.22 00:56:14.283496 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.82 GiB, peak 1.88 GiB, free memory in arenas 4.71 MiB, will set to 1.66 GiB (RSS), difference: -164.01 MiB 2024.10.22 00:56:15.564835 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c43ea813d2bb6c5426cfa8c637ef1673) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c43ea813d2bb6c5426cfa8c637ef1673, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 00:56:15.568831 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: c43ea813d2bb6c5426cfa8c637ef1673, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 00:56:15.571171 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:56:15.589780 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 26436 2024.10.22 00:56:15.785817 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:15.789082 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:15.831304 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_462_462_0 (state Temporary) 2024.10.22 00:56:16.041019 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_462_462_0 to 202410_475_475_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:16.187819 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 5 old parts to remove. Parts: [202410_1_370_78, 202410_371_371_0, 202410_372_372_0, 202410_373_373_0, 202410_374_374_0] 2024.10.22 00:56:16.189317 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from filesystem (serially): Parts: [202410_1_370_78, 202410_371_371_0, 202410_372_372_0, 202410_373_373_0, 202410_374_374_0] 2024.10.22 00:56:16.239405 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 236 2024.10.22 00:56:16.288810 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 4.28 MiB, will set to 1.67 GiB (RSS), difference: -4.52 MiB 2024.10.22 00:56:16.308737 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from memory: Parts: [202410_1_370_78, 202410_371_371_0, 202410_372_372_0, 202410_373_373_0, 202410_374_374_0] 2024.10.22 00:56:16.311628 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26436 2024.10.22 00:56:16.389111 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:16.392649 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:16.475366 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_218_218_0 (state Temporary) 2024.10.22 00:56:16.647633 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_218_218_0 to 202410_228_228_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:17.193470 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 236 2024.10.22 00:56:17.719611 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 3855 2024.10.22 00:56:17.803179 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:17.810308 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:17.878626 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_271_271_0 (state Temporary) 2024.10.22 00:56:17.928646 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_271_271_0 to 202410_284_284_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:18.081110 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3855 2024.10.22 00:56:19.124563 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_54_54_0 to 202410_57_57_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:19.781063 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:56:19.801838 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:56:20.256667 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 441772 2024.10.22 00:56:20.286788 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 67 entries to flush up to offset 18480 2024.10.22 00:56:20.332275 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 26463 2024.10.22 00:56:20.442075 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:20.450657 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:20.469393 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:20.474333 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:20.603498 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_269_269_0 (state Temporary) 2024.10.22 00:56:20.617035 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_463_463_0 (state Temporary) 2024.10.22 00:56:20.659592 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_269_269_0 to 202410_275_275_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:20.666723 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_463_463_0 to 202410_476_476_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:20.690573 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_471_116 to 202410_472_472_0 2024.10.22 00:56:20.697682 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:20.700868 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 18480 2024.10.22 00:56:20.700911 [ 1161 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:20.715375 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26463 2024.10.22 00:56:20.744779 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_472_117} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_471_116 to 202410_472_472_0 into Compact with storage Full 2024.10.22 00:56:20.789581 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_472_117} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:20.799863 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_472_117} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_471_116, total 27104 rows starting from the beginning of the part 2024.10.22 00:56:20.859304 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_472_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_472_472_0, total 25 rows starting from the beginning of the part 2024.10.22 00:56:21.370161 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:21.372947 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:21.378543 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 6.75 MiB, will set to 1.70 GiB (RSS), difference: 5.25 MiB 2024.10.22 00:56:21.480497 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_259_259_0 (state Temporary) 2024.10.22 00:56:21.573143 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_259_259_0 to 202410_266_266_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:21.585042 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 441772 2024.10.22 00:56:21.675265 [ 1211 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_261_52 to 202410_266_266_0 2024.10.22 00:56:21.682620 [ 1211 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:21.683961 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:21.691456 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_261_52 to 202410_266_266_0 into Compact with storage Full 2024.10.22 00:56:21.720443 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:21.731414 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} MergeTreeSequentialSource: Reading 56 marks from part 202410_1_261_52, total 446969 rows starting from the beginning of the part 2024.10.22 00:56:21.755958 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_262_262_0, total 1865 rows starting from the beginning of the part 2024.10.22 00:56:21.762488 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2170 2024.10.22 00:56:21.779767 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_263_263_0, total 1433 rows starting from the beginning of the part 2024.10.22 00:56:21.893839 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_264_264_0, total 1639 rows starting from the beginning of the part 2024.10.22 00:56:21.914511 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_265_265_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:56:21.936551 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_266_266_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:56:22.491586 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 6.98 MiB, will set to 1.70 GiB (RSS), difference: 2.57 MiB 2024.10.22 00:56:23.354843 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(1be2a7aa86fa40226321e4bc889c3ab7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1be2a7aa86fa40226321e4bc889c3ab7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 00:56:23.360302 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 470794, file segment info: File segment: [0, 33554431], key: 1be2a7aa86fa40226321e4bc889c3ab7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 00:56:23.368129 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:56:23.420916 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_472_117} MergeTask::MergeProjectionsStage: Merge sorted 27129 rows, containing 13 columns (13 merged, 0 gathered) in 2.682829528 sec., 10112.084915147094 rows/sec., 1.53 MiB/sec. 2024.10.22 00:56:23.717318 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_472_117} MergedBlockOutputStream: filled checksums 202410_1_472_117 (state Temporary) 2024.10.22 00:56:23.869721 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 00:56:23.885366 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 00:56:23.894675 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 00:56:23.917772 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 00:56:23.939561 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_472_117} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_472_117 to 202410_1_472_117 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:23.955356 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_472_117} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_471_116, 202410_472_472_0] -> 202410_1_472_117 2024.10.22 00:56:23.979532 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.77 MiB. 2024.10.22 00:56:24.124932 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_266_53/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:56:24.151253 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 00:56:24.169106 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 00:56:24.182280 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 00:56:24.341834 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tjn/bvibxvtodfnvqtbtzxmbmgzvipmyr (LogSeriesLimiter: on interval from 2024-10-22 00:56:12 to 2024-10-22 00:56:24 accepted series 1 / 126 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:56:24.695610 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 237 2024.10.22 00:56:24.727047 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 26518 2024.10.22 00:56:24.988176 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:24.990879 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:25.080957 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:25.084017 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:25.239602 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_464_464_0 (state Temporary) 2024.10.22 00:56:25.283534 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_219_219_0 (state Temporary) 2024.10.22 00:56:25.298549 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_464_464_0 to 202410_477_477_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:25.417245 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_219_219_0 to 202410_229_229_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:25.453323 [ 1189 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_472_117 to 202410_473_473_0 2024.10.22 00:56:25.466594 [ 1189 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:25.467854 [ 1187 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_224_46 to 202410_229_229_0 2024.10.22 00:56:25.476843 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26518 2024.10.22 00:56:25.481012 [ 1189 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:25.481349 [ 1187 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:25.507762 [ 1187 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:25.510982 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_473_118} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_472_117 to 202410_473_473_0 into Compact with storage Full 2024.10.22 00:56:25.535389 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_224_46 to 202410_229_229_0 into Compact with storage Full 2024.10.22 00:56:25.568845 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_473_118} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:25.571136 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_221_44, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0, 202410_225_225_0, 202410_226_226_0] 2024.10.22 00:56:25.573252 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_221_44, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0, 202410_225_225_0, 202410_226_226_0] 2024.10.22 00:56:25.574321 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 237 2024.10.22 00:56:25.578154 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_473_118} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_472_117, total 27129 rows starting from the beginning of the part 2024.10.22 00:56:25.584059 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 3865 2024.10.22 00:56:25.655960 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:25.672790 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.88 GiB, free memory in arenas 12.79 MiB, will set to 1.72 GiB (RSS), difference: -9.94 MiB 2024.10.22 00:56:25.701154 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_473_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_473_473_0, total 42 rows starting from the beginning of the part 2024.10.22 00:56:25.702451 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_224_46, total 268 rows starting from the beginning of the part 2024.10.22 00:56:25.814218 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_221_44, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0, 202410_225_225_0, 202410_226_226_0] 2024.10.22 00:56:25.940239 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:25.946685 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:25.971367 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_225_225_0, total 1 rows starting from the beginning of the part 2024.10.22 00:56:26.127003 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_272_272_0 (state Temporary) 2024.10.22 00:56:26.157520 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_226_226_0, total 1 rows starting from the beginning of the part 2024.10.22 00:56:26.292687 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_272_272_0 to 202410_285_285_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:26.335831 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_227_227_0, total 1 rows starting from the beginning of the part 2024.10.22 00:56:26.401802 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_228_228_0, total 1 rows starting from the beginning of the part 2024.10.22 00:56:26.673387 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_229_229_0, total 1 rows starting from the beginning of the part 2024.10.22 00:56:26.705180 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3865 2024.10.22 00:56:26.712413 [ 1218 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:56:26.755836 [ 1218 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:56:27.171949 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_473_118} MergeTask::MergeProjectionsStage: Merge sorted 27171 rows, containing 13 columns (13 merged, 0 gathered) in 1.669545197 sec., 16274.492028621613 rows/sec., 2.46 MiB/sec. 2024.10.22 00:56:27.288493 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_473_118} MergedBlockOutputStream: filled checksums 202410_1_473_118 (state Temporary) 2024.10.22 00:56:27.535694 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 6 similar messages) 2024.10.22 00:56:27.620821 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_473_118} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_473_118 to 202410_1_473_118 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:27.628329 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_473_118} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_472_117, 202410_473_473_0] -> 202410_1_473_118 2024.10.22 00:56:27.640460 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.88 GiB, free memory in arenas 12.75 MiB, will set to 1.73 GiB (RSS), difference: 3.51 MiB 2024.10.22 00:56:27.659670 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.77 MiB. 2024.10.22 00:56:28.216133 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 148 entries to flush up to offset 18628 2024.10.22 00:56:28.593449 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 442924 2024.10.22 00:56:28.629820 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:28.635535 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:28.689552 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_270_270_0 (state Temporary) 2024.10.22 00:56:28.742338 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_270_270_0 to 202410_276_276_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:28.803013 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} MergeTask::MergeProjectionsStage: Merge sorted 273 rows, containing 32 columns (32 merged, 0 gathered) in 3.281595937 sec., 83.19122928021835 rows/sec., 8.66 KiB/sec. 2024.10.22 00:56:28.874271 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} MergedBlockOutputStream: filled checksums 202410_1_229_47 (state Temporary) 2024.10.22 00:56:28.892532 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 18628 2024.10.22 00:56:29.121061 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(d68ac72127b53702caa0188390ec9b7d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d68ac72127b53702caa0188390ec9b7d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 00:56:29.126463 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: d68ac72127b53702caa0188390ec9b7d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 00:56:29.132836 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:56:29.219589 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:29.224940 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:29.280345 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 2208 2024.10.22 00:56:29.491813 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 26581 2024.10.22 00:56:29.537901 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_260_260_0 (state Temporary) 2024.10.22 00:56:29.610278 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_229_47 to 202410_1_229_47 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:29.613898 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_229_47} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_224_46, 202410_229_229_0] -> 202410_1_229_47 2024.10.22 00:56:29.731238 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_260_260_0 to 202410_267_267_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:29.765701 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:56:29.780442 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:56:29.781747 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:56:29.861508 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:29.865780 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:29.871469 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 442924 2024.10.22 00:56:29.980582 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 00:56:30.068861 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_465_465_0 (state Temporary) 2024.10.22 00:56:30.136072 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_465_465_0 to 202410_478_478_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:30.175809 [ 1224 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_473_118 to 202410_474_474_0 2024.10.22 00:56:30.179961 [ 1224 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:30.182103 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.26 GiB. 2024.10.22 00:56:30.194747 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26581 2024.10.22 00:56:30.225800 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_474_119} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_473_118 to 202410_474_474_0 into Compact with storage Full 2024.10.22 00:56:30.502613 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_474_119} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:30.518403 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_474_119} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_473_118, total 27171 rows starting from the beginning of the part 2024.10.22 00:56:30.581949 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 12.54 MiB, will set to 1.74 GiB (RSS), difference: 39.04 MiB 2024.10.22 00:56:30.678457 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_474_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_474_474_0, total 54 rows starting from the beginning of the part 2024.10.22 00:56:31.888926 [ 1680 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/czx/ioojhiffcdypmgpsoccuzlejvdomy, size 198026 2024.10.22 00:56:32.784628 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.88 GiB, free memory in arenas 14.02 MiB, will set to 1.74 GiB (RSS), difference: 171.87 MiB 2024.10.22 00:56:33.076297 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 238 2024.10.22 00:56:33.531709 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_474_119} MergeTask::MergeProjectionsStage: Merge sorted 27225 rows, containing 13 columns (13 merged, 0 gathered) in 3.315952679 sec., 8210.310168904554 rows/sec., 1.24 MiB/sec. 2024.10.22 00:56:33.632930 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:33.640707 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.23 GiB. 2024.10.22 00:56:33.669530 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_474_119} MergedBlockOutputStream: filled checksums 202410_1_474_119 (state Temporary) 2024.10.22 00:56:33.937547 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_220_220_0 (state Temporary) 2024.10.22 00:56:34.097485 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_220_220_0 to 202410_230_230_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:34.204526 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_474_119} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_474_119 to 202410_1_474_119 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:34.208407 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 26613 2024.10.22 00:56:34.211156 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 3880 2024.10.22 00:56:34.225258 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_474_119} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_473_118, 202410_474_474_0] -> 202410_1_474_119 2024.10.22 00:56:34.298155 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.63 MiB. 2024.10.22 00:56:34.310599 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 238 2024.10.22 00:56:34.491015 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:34.505709 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.23 GiB. 2024.10.22 00:56:34.554448 [ 1237 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 353 ms. 2024.10.22 00:56:34.578267 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_273_273_0 (state Temporary) 2024.10.22 00:56:34.613113 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:34.627135 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.21 GiB. 2024.10.22 00:56:34.670868 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_273_273_0 to 202410_286_286_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:34.696306 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:56:34.730169 [ 1250 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_281_56 to 202410_286_286_0 2024.10.22 00:56:34.777149 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_466_466_0 (state Temporary) 2024.10.22 00:56:34.789107 [ 1250 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:34.709455 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:56:34.838543 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.20 GiB. 2024.10.22 00:56:34.853620 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:56:34.901090 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:56:34.921522 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_281_56 to 202410_286_286_0 into Compact with storage Full 2024.10.22 00:56:34.929055 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.88 GiB, free memory in arenas 13.10 MiB, will set to 1.73 GiB (RSS), difference: 130.54 MiB 2024.10.22 00:56:34.938507 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_466_466_0 to 202410_479_479_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:34.980529 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3880 2024.10.22 00:56:35.026448 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:56:35.088018 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26613 2024.10.22 00:56:35.098643 [ 1251 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_474_119 to 202410_475_475_0 2024.10.22 00:56:35.105396 [ 1251 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:35.171802 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:35.185723 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_266_53/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:56:35.194712 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:35.266330 [ 1251 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 206 ms. 2024.10.22 00:56:35.295797 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:56:35.326924 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_475_120} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_474_119 to 202410_475_475_0 into Compact with storage Full 2024.10.22 00:56:35.357075 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:56:35.364981 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_281_56, total 3913 rows starting from the beginning of the part 2024.10.22 00:56:35.373667 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:56:35.380823 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column JavaEnable 2024.10.22 00:56:35.534936 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_374_79, 202410_375_375_0] 2024.10.22 00:56:35.542417 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_374_79, 202410_375_375_0] 2024.10.22 00:56:35.546575 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_475_120} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:35.572006 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_374_79, 202410_375_375_0] 2024.10.22 00:56:35.645155 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_282_282_0, total 19 rows starting from the beginning of the part 2024.10.22 00:56:35.674457 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_475_120} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_474_119, total 27225 rows starting from the beginning of the part 2024.10.22 00:56:35.735324 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pyo/iucnzirkndatqgnkyapobhofidlaw (LogSeriesLimiter: on interval from 2024-10-22 00:56:24 to 2024-10-22 00:56:35 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:56:35.755236 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_475_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_475_475_0, total 42 rows starting from the beginning of the part 2024.10.22 00:56:35.890452 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_283_283_0, total 18 rows starting from the beginning of the part 2024.10.22 00:56:35.993306 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_18_18_0 (state Temporary) 2024.10.22 00:56:36.031051 [ 1736 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ygi/ykonxchhfcqdpwjncjvykgtbyocno, size 356 2024.10.22 00:56:36.076618 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_284_284_0, total 11 rows starting from the beginning of the part 2024.10.22 00:56:36.419527 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_285_285_0, total 10 rows starting from the beginning of the part 2024.10.22 00:56:36.428833 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 146 entries to flush up to offset 18774 2024.10.22 00:56:36.623635 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_286_286_0, total 15 rows starting from the beginning of the part 2024.10.22 00:56:36.874973 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 444460 2024.10.22 00:56:37.251801 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:37.262272 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:37.549325 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 13.97 MiB, will set to 1.69 GiB (RSS), difference: 15.68 MiB 2024.10.22 00:56:37.566427 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_271_271_0 (state Temporary) 2024.10.22 00:56:37.633148 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_271_271_0 to 202410_277_277_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:37.644663 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:37.654863 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:37.680398 [ 1160 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_272_74 to 202410_277_277_0 2024.10.22 00:56:37.688370 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 18774 2024.10.22 00:56:37.691000 [ 1160 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 00:56:37.694374 [ 1160 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:37.712975 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_272_74 to 202410_277_277_0 into Compact with storage Full 2024.10.22 00:56:37.782015 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:37.793080 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_272_74, total 18364 rows starting from the beginning of the part 2024.10.22 00:56:37.840678 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_261_261_0 (state Temporary) 2024.10.22 00:56:37.862241 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_273_273_0, total 42 rows starting from the beginning of the part 2024.10.22 00:56:37.868895 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_475_120} MergeTask::MergeProjectionsStage: Merge sorted 27267 rows, containing 13 columns (13 merged, 0 gathered) in 2.56900747 sec., 10613.826669799446 rows/sec., 1.60 MiB/sec. 2024.10.22 00:56:37.894956 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_274_274_0, total 111 rows starting from the beginning of the part 2024.10.22 00:56:38.009509 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_275_275_0, total 67 rows starting from the beginning of the part 2024.10.22 00:56:38.047407 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_261_261_0 to 202410_268_268_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:38.057439 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_475_120} MergedBlockOutputStream: filled checksums 202410_1_475_120 (state Temporary) 2024.10.22 00:56:38.081046 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_276_276_0, total 148 rows starting from the beginning of the part 2024.10.22 00:56:38.094851 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 444460 2024.10.22 00:56:38.165829 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_277_277_0, total 146 rows starting from the beginning of the part 2024.10.22 00:56:38.206371 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_475_120} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_475_120 to 202410_1_475_120 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:38.239411 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_475_120} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_474_119, 202410_475_475_0] -> 202410_1_475_120 2024.10.22 00:56:38.479497 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.78 MiB. 2024.10.22 00:56:38.535178 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} MergeTask::MergeProjectionsStage: Merge sorted 3986 rows, containing 12 columns (12 merged, 0 gathered) in 3.636009238 sec., 1096.256840698379 rows/sec., 171.56 KiB/sec. 2024.10.22 00:56:38.697695 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} MergedBlockOutputStream: filled checksums 202410_1_286_57 (state Temporary) 2024.10.22 00:56:38.787703 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_375_80, 202410_376_376_0] 2024.10.22 00:56:38.789722 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_375_80, 202410_376_376_0] 2024.10.22 00:56:38.800497 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_375_80, 202410_376_376_0] 2024.10.22 00:56:39.092691 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 26700 2024.10.22 00:56:39.121415 [ 1163 ] {} DatabaseCatalog: Execution took 282 ms. 2024.10.22 00:56:39.194720 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_286_57 to 202410_1_286_57 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:39.199484 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_286_57} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_281_56, 202410_286_286_0] -> 202410_1_286_57 2024.10.22 00:56:39.289058 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:56:39.560156 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:39.567568 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:39.736432 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_467_467_0 (state Temporary) 2024.10.22 00:56:39.782177 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:56:39.784621 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:56:39.843507 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 12.25 MiB, will set to 1.70 GiB (RSS), difference: 8.10 MiB 2024.10.22 00:56:39.897412 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_467_467_0 to 202410_480_480_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:39.963905 [ 1181 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_475_120 to 202410_476_476_0 2024.10.22 00:56:39.971554 [ 1181 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:39.996996 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:39.998191 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26700 2024.10.22 00:56:40.054415 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_476_121} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_475_120 to 202410_476_476_0 into Compact with storage Full 2024.10.22 00:56:40.095372 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_476_121} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:40.103861 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_476_121} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_475_120, total 27267 rows starting from the beginning of the part 2024.10.22 00:56:40.124151 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_476_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_476_476_0, total 27 rows starting from the beginning of the part 2024.10.22 00:56:40.278950 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} MergeTask::MergeProjectionsStage: Merge sorted 18878 rows, containing 13 columns (13 merged, 0 gathered) in 2.575860331 sec., 7328.813512443506 rows/sec., 3.09 MiB/sec. 2024.10.22 00:56:40.375416 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_214_45, 202410_215_215_0, 202410_216_216_0, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0] 2024.10.22 00:56:40.377125 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_214_45, 202410_215_215_0, 202410_216_216_0, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0] 2024.10.22 00:56:40.390489 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} MergedBlockOutputStream: filled checksums 202410_1_277_75 (state Temporary) 2024.10.22 00:56:40.505401 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_214_45, 202410_215_215_0, 202410_216_216_0, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0] 2024.10.22 00:56:40.687911 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_277_75 to 202410_1_277_75 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:40.690990 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_277_75} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_272_74, 202410_277_277_0] -> 202410_1_277_75 2024.10.22 00:56:40.729134 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.86 MiB. 2024.10.22 00:56:40.970874 [ 1158 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 00:56:41.571884 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 14.82 MiB, will set to 1.69 GiB (RSS), difference: -2.61 MiB 2024.10.22 00:56:41.584423 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/elr/fhkzpojidfabjsxhcrwyplogsiawu. 2024.10.22 00:56:41.767631 [ 1248 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:56:41.772264 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} MergeTask::MergeProjectionsStage: Merge sorted 455362 rows, containing 4 columns (4 merged, 0 gathered) in 20.083440787 sec., 22673.505243919935 rows/sec., 332.54 KiB/sec. 2024.10.22 00:56:41.807223 [ 1248 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:56:41.816777 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 239 2024.10.22 00:56:41.842495 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} MergedBlockOutputStream: filled checksums 202410_1_266_53 (state Temporary) 2024.10.22 00:56:42.234797 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_266_53 to 202410_1_266_53 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:42.235315 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:42.241329 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:42.245403 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_266_53} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_261_52, 202410_266_266_0] -> 202410_1_266_53 2024.10.22 00:56:42.300637 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_221_221_0 (state Temporary) 2024.10.22 00:56:42.402838 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:56:42.423260 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_221_221_0 to 202410_231_231_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:42.482884 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 33 entries to flush up to offset 3913 2024.10.22 00:56:42.573673 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 239 2024.10.22 00:56:42.777781 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:42.881378 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:42.922289 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_274_274_0 (state Temporary) 2024.10.22 00:56:42.984123 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_274_274_0 to 202410_287_287_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:43.053803 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3913 2024.10.22 00:56:43.288092 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_476_121} MergeTask::MergeProjectionsStage: Merge sorted 27294 rows, containing 13 columns (13 merged, 0 gathered) in 3.246235943 sec., 8407.89162563961 rows/sec., 1.27 MiB/sec. 2024.10.22 00:56:43.572351 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 9.71 MiB, will set to 1.69 GiB (RSS), difference: 4.33 MiB 2024.10.22 00:56:43.682552 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_476_121} MergedBlockOutputStream: filled checksums 202410_1_476_121 (state Temporary) 2024.10.22 00:56:43.843109 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_476_121} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_476_121 to 202410_1_476_121 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:43.851856 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_476_121} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_475_120, 202410_476_476_0] -> 202410_1_476_121 2024.10.22 00:56:43.891610 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.77 MiB. 2024.10.22 00:56:44.023803 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 26756 2024.10.22 00:56:44.292067 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:44.293800 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:44.472963 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_468_468_0 (state Temporary) 2024.10.22 00:56:44.544811 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_468_468_0 to 202410_481_481_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:44.591159 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_476_121 to 202410_477_477_0 2024.10.22 00:56:44.598490 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:44.601292 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:44.663261 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_477_122} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_476_121 to 202410_477_477_0 into Compact with storage Full 2024.10.22 00:56:44.672001 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26756 2024.10.22 00:56:44.852923 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_477_122} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:44.870136 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_477_122} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_476_121, total 27294 rows starting from the beginning of the part 2024.10.22 00:56:44.895963 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_477_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_477_477_0, total 55 rows starting from the beginning of the part 2024.10.22 00:56:45.100772 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 445996 2024.10.22 00:56:45.222354 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 120 entries to flush up to offset 18894 2024.10.22 00:56:45.485515 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:45.488611 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:45.546084 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:56:45.694167 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_272_272_0 (state Temporary) 2024.10.22 00:56:45.794480 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_272_272_0 to 202410_278_278_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:45.884524 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_376_81, 202410_377_377_0, 202410_378_378_0, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0] 2024.10.22 00:56:45.887325 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_376_81, 202410_377_377_0, 202410_378_378_0, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0] 2024.10.22 00:56:45.977078 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 18894 2024.10.22 00:56:45.988837 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 7.55 MiB, will set to 1.68 GiB (RSS), difference: -1.68 MiB 2024.10.22 00:56:46.037084 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_376_81, 202410_377_377_0, 202410_378_378_0, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0] 2024.10.22 00:56:46.222399 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_477_122/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:56:46.279547 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_206_41, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0, 202410_210_210_0, 202410_211_211_0] 2024.10.22 00:56:46.281629 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_206_41, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0, 202410_210_210_0, 202410_211_211_0] 2024.10.22 00:56:46.313459 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_206_41, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0, 202410_210_210_0, 202410_211_211_0] 2024.10.22 00:56:46.362923 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:46.366445 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.19 GiB. 2024.10.22 00:56:46.399940 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Age 2024.10.22 00:56:46.406312 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Age 2024.10.22 00:56:46.512229 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_262_262_0 (state Temporary) 2024.10.22 00:56:46.525541 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Age 2024.10.22 00:56:46.540318 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Age 2024.10.22 00:56:46.560139 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Age 2024.10.22 00:56:46.574853 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Age 2024.10.22 00:56:46.595707 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Age 2024.10.22 00:56:46.596594 [ 1709 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tjn/bvibxvtodfnvqtbtzxmbmgzvipmyr, size 10651484 2024.10.22 00:56:46.622797 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_262_262_0 to 202410_269_269_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:46.664910 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mlh/osnlhwafulnnednoboxslorqvliij (LogSeriesLimiter: on interval from 2024-10-22 00:56:35 to 2024-10-22 00:56:46 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:56:46.681828 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 445996 2024.10.22 00:56:47.912554 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 7.95 MiB, will set to 1.68 GiB (RSS), difference: 7.32 MiB 2024.10.22 00:56:48.030424 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_477_122} MergeTask::MergeProjectionsStage: Merge sorted 27349 rows, containing 13 columns (13 merged, 0 gathered) in 3.39517603 sec., 8055.252440033279 rows/sec., 1.22 MiB/sec. 2024.10.22 00:56:48.199170 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_477_122} MergedBlockOutputStream: filled checksums 202410_1_477_122 (state Temporary) 2024.10.22 00:56:48.554483 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_477_122} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_477_122 to 202410_1_477_122 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:48.561344 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_477_122} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_476_121, 202410_477_477_0] -> 202410_1_477_122 2024.10.22 00:56:48.680448 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 26804 2024.10.22 00:56:48.728026 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.62 MiB. 2024.10.22 00:56:48.973620 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:48.976549 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:49.032617 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_469_469_0 (state Temporary) 2024.10.22 00:56:49.084603 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_469_469_0 to 202410_482_482_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:49.174574 [ 1222 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_477_122 to 202410_478_478_0 2024.10.22 00:56:49.180772 [ 1222 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:49.185764 [ 1222 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:49.275030 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26804 2024.10.22 00:56:49.283670 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:49.286829 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:49.378002 [ 1277 ] {} DatabaseCatalog: Execution took 254 ms. 2024.10.22 00:56:49.418647 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_478_123} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_477_122 to 202410_478_478_0 into Compact with storage Full 2024.10.22 00:56:49.568955 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_478_123} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:49.615810 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 7.93 MiB, will set to 1.67 GiB (RSS), difference: 1.62 MiB 2024.10.22 00:56:49.686720 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_478_123} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_477_122, total 27349 rows starting from the beginning of the part 2024.10.22 00:56:49.714775 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_478_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_478_478_0, total 63 rows starting from the beginning of the part 2024.10.22 00:56:49.782244 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:56:49.784367 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:56:50.079998 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 240 2024.10.22 00:56:50.390990 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:50.393818 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:50.557106 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 3924 2024.10.22 00:56:50.574854 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_222_222_0 (state Temporary) 2024.10.22 00:56:50.703324 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:50.710280 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:50.770853 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_275_275_0 (state Temporary) 2024.10.22 00:56:50.815408 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_222_222_0 to 202410_232_232_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:50.936738 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_275_275_0 to 202410_288_288_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:50.967241 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 240 2024.10.22 00:56:51.172238 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3924 2024.10.22 00:56:51.391004 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 6.38 MiB, will set to 1.68 GiB (RSS), difference: -5.34 MiB 2024.10.22 00:56:52.222760 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_478_123} MergeTask::MergeProjectionsStage: Merge sorted 27412 rows, containing 13 columns (13 merged, 0 gathered) in 2.910367214 sec., 9418.742716773884 rows/sec., 1.42 MiB/sec. 2024.10.22 00:56:52.485547 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_478_123} MergedBlockOutputStream: filled checksums 202410_1_478_123 (state Temporary) 2024.10.22 00:56:52.548579 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_478_123} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_478_123 to 202410_1_478_123 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:52.555805 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_478_123} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_477_122, 202410_478_478_0] -> 202410_1_478_123 2024.10.22 00:56:52.586744 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.63 MiB. 2024.10.22 00:56:53.280509 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 26842 2024.10.22 00:56:53.284488 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_55_55_0 (state Temporary) 2024.10.22 00:56:53.392088 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:53.397768 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:53.495325 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 92 entries to flush up to offset 18986 2024.10.22 00:56:53.555653 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_470_470_0 (state Temporary) 2024.10.22 00:56:53.558748 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 6.21 MiB, will set to 1.67 GiB (RSS), difference: -4.67 MiB 2024.10.22 00:56:53.628990 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_470_470_0 to 202410_483_483_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:53.643564 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:53.649319 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:53.667562 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_478_123 to 202410_479_479_0 2024.10.22 00:56:53.687748 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 447532 2024.10.22 00:56:53.718569 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26842 2024.10.22 00:56:53.759533 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:53.767376 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:53.813333 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_479_124} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_478_123 to 202410_479_479_0 into Compact with storage Full 2024.10.22 00:56:53.899137 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_273_273_0 (state Temporary) 2024.10.22 00:56:53.979787 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_273_273_0 to 202410_279_279_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:54.021161 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 18986 2024.10.22 00:56:54.170893 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_479_124} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:54.196096 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_479_124} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_478_123, total 27412 rows starting from the beginning of the part 2024.10.22 00:56:54.431093 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_479_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_479_479_0, total 32 rows starting from the beginning of the part 2024.10.22 00:56:55.098881 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:55.104970 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:55.365083 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_263_263_0 (state Temporary) 2024.10.22 00:56:55.775747 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 4.53 MiB, will set to 1.67 GiB (RSS), difference: -15.85 MiB 2024.10.22 00:56:55.786483 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_263_263_0 to 202410_270_270_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:55.828907 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 447532 2024.10.22 00:56:56.461667 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_179_37, 202410_180_180_0, 202410_181_181_0, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0] 2024.10.22 00:56:56.468357 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_179_37, 202410_180_180_0, 202410_181_181_0, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0] 2024.10.22 00:56:56.499800 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_179_37, 202410_180_180_0, 202410_181_181_0, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0] 2024.10.22 00:56:56.821566 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_479_124} MergeTask::MergeProjectionsStage: Merge sorted 27444 rows, containing 13 columns (13 merged, 0 gathered) in 3.041052202 sec., 9024.508024541961 rows/sec., 1.36 MiB/sec. 2024.10.22 00:56:56.824294 [ 1172 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:56:56.858933 [ 1172 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:56:56.870499 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_479_124} MergedBlockOutputStream: filled checksums 202410_1_479_124 (state Temporary) 2024.10.22 00:56:57.036553 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_479_124/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:56:57.431814 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_479_124} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_479_124 to 202410_1_479_124 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:57.443784 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_479_124} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_478_123, 202410_479_479_0] -> 202410_1_479_124 2024.10.22 00:56:57.479100 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.77 MiB. 2024.10.22 00:56:57.513092 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(77ec80561addcf51b5fd7ccc2d66457c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 77ec80561addcf51b5fd7ccc2d66457c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 00:56:57.517451 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 501908, file segment info: File segment: [0, 33554431], key: 77ec80561addcf51b5fd7ccc2d66457c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 00:56:57.523255 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:56:57.667958 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 4.02 MiB, will set to 1.67 GiB (RSS), difference: 23.17 MiB 2024.10.22 00:56:57.706839 [ 1727 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pyo/iucnzirkndatqgnkyapobhofidlaw, size 501908 2024.10.22 00:56:57.766565 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 26886 2024.10.22 00:56:57.935586 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Title 2024.10.22 00:56:57.992124 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_55_55_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:56:58.080051 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Title 2024.10.22 00:56:58.093176 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Title 2024.10.22 00:56:58.117604 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Title 2024.10.22 00:56:58.136947 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:58.147966 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:58.183528 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Title 2024.10.22 00:56:58.195806 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Title 2024.10.22 00:56:58.207521 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Title 2024.10.22 00:56:58.231615 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_471_471_0 (state Temporary) 2024.10.22 00:56:58.246051 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Title 2024.10.22 00:56:58.269315 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Title 2024.10.22 00:56:58.297162 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_471_471_0 to 202410_484_484_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:58.315493 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/joa/uvwsudilnfximdksetazbxhkjoxaa (LogSeriesLimiter: on interval from 2024-10-22 00:56:46 to 2024-10-22 00:56:58 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:56:58.359877 [ 1178 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_479_124 to 202410_480_480_0 2024.10.22 00:56:58.379325 [ 1178 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:58.382157 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:58.505589 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_480_125} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_479_124 to 202410_480_480_0 into Compact with storage Full 2024.10.22 00:56:58.600769 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_480_125} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:56:58.620954 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_480_125} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_479_124, total 27444 rows starting from the beginning of the part 2024.10.22 00:56:58.624602 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26886 2024.10.22 00:56:58.676013 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 3936 2024.10.22 00:56:58.689592 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_480_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_480_480_0, total 87 rows starting from the beginning of the part 2024.10.22 00:56:59.259980 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:56:59.265604 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:56:59.285077 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_276_276_0 (state Temporary) 2024.10.22 00:56:59.556150 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 4.07 MiB, will set to 1.67 GiB (RSS), difference: -37.70 MiB 2024.10.22 00:56:59.573594 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_276_276_0 to 202410_289_289_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:56:59.735967 [ 1163 ] {} DatabaseCatalog: Execution took 356 ms. 2024.10.22 00:56:59.758810 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3936 2024.10.22 00:56:59.783076 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:56:59.784870 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:57:00.737827 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_480_125} MergeTask::MergeProjectionsStage: Merge sorted 27531 rows, containing 13 columns (13 merged, 0 gathered) in 2.242062316 sec., 12279.319715393673 rows/sec., 1.85 MiB/sec. 2024.10.22 00:57:01.128257 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_480_125} MergedBlockOutputStream: filled checksums 202410_1_480_125 (state Temporary) 2024.10.22 00:57:01.559176 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 19076 2024.10.22 00:57:01.715929 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_480_125} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_480_125 to 202410_1_480_125 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:01.733623 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_480_125} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_479_124, 202410_480_480_0] -> 202410_1_480_125 2024.10.22 00:57:01.771807 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.63 MiB. 2024.10.22 00:57:01.797168 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:01.805605 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:57:01.875569 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_274_274_0 (state Temporary) 2024.10.22 00:57:01.918896 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_274_274_0 to 202410_280_280_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:01.960540 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 3.35 MiB, will set to 1.69 GiB (RSS), difference: 16.31 MiB 2024.10.22 00:57:02.463407 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 19076 2024.10.22 00:57:02.638361 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 26932 2024.10.22 00:57:02.834929 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 449452 2024.10.22 00:57:02.979963 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:02.989398 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:57:03.140914 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_472_472_0 (state Temporary) 2024.10.22 00:57:03.180960 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_472_472_0 to 202410_485_485_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:03.307455 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_55_55_0 to 202410_58_58_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:03.210455 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_480_125 to 202410_481_481_0 2024.10.22 00:57:03.348489 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:03.362282 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:57:03.367999 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26932 2024.10.22 00:57:03.409409 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_481_126} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_480_125 to 202410_481_481_0 into Compact with storage Full 2024.10.22 00:57:03.575389 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_481_126} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:03.587886 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_481_126} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_480_125, total 27531 rows starting from the beginning of the part 2024.10.22 00:57:03.606109 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_481_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_481_481_0, total 56 rows starting from the beginning of the part 2024.10.22 00:57:03.797542 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:03.800658 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:57:03.996085 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_264_264_0 (state Temporary) 2024.10.22 00:57:04.326814 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_264_264_0 to 202410_271_271_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:04.383000 [ 1266 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_266_53 to 202410_271_271_0 2024.10.22 00:57:04.384641 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 449452 2024.10.22 00:57:04.395572 [ 1266 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:04.400879 [ 1266 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.18 GiB. 2024.10.22 00:57:04.439392 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_266_53 to 202410_271_271_0 into Compact with storage Full 2024.10.22 00:57:04.591283 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:04.601048 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} MergeTreeSequentialSource: Reading 57 marks from part 202410_1_266_53, total 455362 rows starting from the beginning of the part 2024.10.22 00:57:04.630248 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_267_267_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:57:04.632764 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ad616a7930721a2d9dac02121ddaba3a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ad616a7930721a2d9dac02121ddaba3a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 00:57:04.636000 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: ad616a7930721a2d9dac02121ddaba3a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 00:57:04.640721 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:57:04.657130 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_268_268_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:57:04.683778 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_269_269_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:57:04.701038 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_270_270_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:57:04.877117 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_271_271_0, total 1920 rows starting from the beginning of the part 2024.10.22 00:57:05.139566 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_481_126} MergeTask::MergeProjectionsStage: Merge sorted 27587 rows, containing 13 columns (13 merged, 0 gathered) in 1.748601756 sec., 15776.605453666261 rows/sec., 2.38 MiB/sec. 2024.10.22 00:57:05.357747 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_481_126} MergedBlockOutputStream: filled checksums 202410_1_481_126 (state Temporary) 2024.10.22 00:57:05.788292 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_481_126} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_481_126 to 202410_1_481_126 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:05.791944 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_481_126} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_480_125, 202410_481_481_0] -> 202410_1_481_126 2024.10.22 00:57:05.821238 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.62 MiB. 2024.10.22 00:57:05.968961 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(654009bd98fb82526d0223a724d294c8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 654009bd98fb82526d0223a724d294c8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 00:57:05.971683 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 241 2024.10.22 00:57:05.972129 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 867453, file segment info: File segment: [0, 33554431], key: 654009bd98fb82526d0223a724d294c8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 00:57:05.976164 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:57:06.081175 [ 1702 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mlh/osnlhwafulnnednoboxslorqvliij, size 867453 2024.10.22 00:57:06.620944 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:06.626786 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:06.691725 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_223_223_0 (state Temporary) 2024.10.22 00:57:06.694209 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Sex 2024.10.22 00:57:06.707475 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Sex 2024.10.22 00:57:06.720248 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Sex 2024.10.22 00:57:06.737071 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Sex 2024.10.22 00:57:06.747660 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2208 2024.10.22 00:57:06.860328 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_223_223_0 to 202410_233_233_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:06.871676 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Sex 2024.10.22 00:57:06.898670 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Sex 2024.10.22 00:57:06.927104 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Sex 2024.10.22 00:57:06.989090 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 241 2024.10.22 00:57:07.109705 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/aek/cossotqgfcgpivfxtixqwtthhjkqu (LogSeriesLimiter: on interval from 2024-10-22 00:56:58 to 2024-10-22 00:57:07 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:57:07.264506 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 3948 2024.10.22 00:57:07.380035 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 26991 2024.10.22 00:57:07.557813 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 7.92 MiB, will set to 1.70 GiB (RSS), difference: -3.63 MiB 2024.10.22 00:57:07.670734 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:07.675306 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:07.736885 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:07.741695 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:07.964623 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_473_473_0 (state Temporary) 2024.10.22 00:57:08.004114 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_277_277_0 (state Temporary) 2024.10.22 00:57:08.264853 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_277_277_0 to 202410_290_290_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:08.275154 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_271_54/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:57:08.337315 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3948 2024.10.22 00:57:08.417446 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_473_473_0 to 202410_486_486_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:08.463666 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_481_126 to 202410_482_482_0 2024.10.22 00:57:08.470904 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:08.474893 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:08.489764 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_482_127} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_481_126 to 202410_482_482_0 into Compact with storage Full 2024.10.22 00:57:08.495822 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_226_45, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0, 202410_230_230_0, 202410_231_231_0] 2024.10.22 00:57:08.498907 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_226_45, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0, 202410_230_230_0, 202410_231_231_0] 2024.10.22 00:57:08.579495 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 26991 2024.10.22 00:57:08.632298 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_226_45, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0, 202410_230_230_0, 202410_231_231_0] 2024.10.22 00:57:08.632771 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_482_127} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:08.650719 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_482_127} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_481_126, total 27587 rows starting from the beginning of the part 2024.10.22 00:57:08.680263 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_482_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_482_482_0, total 48 rows starting from the beginning of the part 2024.10.22 00:57:09.526993 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 10.97 MiB, will set to 1.70 GiB (RSS), difference: -9.82 MiB 2024.10.22 00:57:09.691501 [ 1233 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 00:57:09.783022 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:57:09.785408 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:57:10.090083 [ 1212 ] {} DatabaseCatalog: Execution took 352 ms. 2024.10.22 00:57:10.183763 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 134 entries to flush up to offset 19210 2024.10.22 00:57:10.428187 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/elr/fhkzpojidfabjsxhcrwyplogsiawu. 2024.10.22 00:57:10.540924 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:10.547090 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:10.713616 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_275_275_0 (state Temporary) 2024.10.22 00:57:10.736349 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_482_127} MergeTask::MergeProjectionsStage: Merge sorted 27635 rows, containing 13 columns (13 merged, 0 gathered) in 2.251510371 sec., 12273.982992015273 rows/sec., 1.85 MiB/sec. 2024.10.22 00:57:10.903320 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_275_275_0 to 202410_281_281_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:11.000002 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 19210 2024.10.22 00:57:11.072386 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_482_127} MergedBlockOutputStream: filled checksums 202410_1_482_127 (state Temporary) 2024.10.22 00:57:11.363529 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_381_82, 202410_382_382_0, 202410_383_383_0, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0] 2024.10.22 00:57:11.366478 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_381_82, 202410_382_382_0, 202410_383_383_0, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0] 2024.10.22 00:57:11.394441 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 450604 2024.10.22 00:57:11.430474 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_482_127} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_482_127 to 202410_1_482_127 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:11.469658 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:57:11.568794 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_482_127} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_481_126, 202410_482_482_0] -> 202410_1_482_127 2024.10.22 00:57:11.602203 [ 1239 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 211 ms. 2024.10.22 00:57:11.627545 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_381_82, 202410_382_382_0, 202410_383_383_0, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0] 2024.10.22 00:57:11.695132 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.78 MiB. 2024.10.22 00:57:11.881471 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.88 GiB, free memory in arenas 11.46 MiB, will set to 1.71 GiB (RSS), difference: 100.08 MiB 2024.10.22 00:57:11.886967 [ 1214 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:57:11.954302 [ 1214 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:57:12.481029 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:12.484232 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:12.589126 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 27042 2024.10.22 00:57:12.600724 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_265_265_0 (state Temporary) 2024.10.22 00:57:12.772551 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_265_265_0 to 202410_272_272_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:12.800291 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 450604 2024.10.22 00:57:13.006320 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:13.008959 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:13.158127 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_474_474_0 (state Temporary) 2024.10.22 00:57:13.404502 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_474_474_0 to 202410_487_487_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:13.435923 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_482_127 to 202410_483_483_0 2024.10.22 00:57:13.444023 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:13.446492 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:13.466541 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 27042 2024.10.22 00:57:13.474634 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_483_128} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_482_127 to 202410_483_483_0 into Compact with storage Full 2024.10.22 00:57:13.535643 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_483_128} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:13.551378 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_483_128} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_482_127, total 27635 rows starting from the beginning of the part 2024.10.22 00:57:13.587111 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_483_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_483_483_0, total 38 rows starting from the beginning of the part 2024.10.22 00:57:14.271780 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 45 entries to flush up to offset 2253 2024.10.22 00:57:14.492019 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 242 2024.10.22 00:57:14.519302 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.88 GiB, free memory in arenas 12.91 MiB, will set to 1.70 GiB (RSS), difference: 99.44 MiB 2024.10.22 00:57:14.855593 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:14.860030 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:15.223277 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_224_224_0 (state Temporary) 2024.10.22 00:57:15.294671 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_224_224_0 to 202410_234_234_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:15.337564 [ 1256 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_229_47 to 202410_234_234_0 2024.10.22 00:57:15.455479 [ 1256 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:15.459858 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:15.542315 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_229_47 to 202410_234_234_0 into Compact with storage Full 2024.10.22 00:57:15.723236 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:15.740080 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 242 2024.10.22 00:57:15.745057 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_229_47, total 273 rows starting from the beginning of the part 2024.10.22 00:57:15.814559 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_483_128} MergeTask::MergeProjectionsStage: Merge sorted 27673 rows, containing 13 columns (13 merged, 0 gathered) in 2.345811567 sec., 11796.770204944598 rows/sec., 1.78 MiB/sec. 2024.10.22 00:57:15.820681 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_230_230_0, total 1 rows starting from the beginning of the part 2024.10.22 00:57:15.841090 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 3957 2024.10.22 00:57:15.877459 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_231_231_0, total 1 rows starting from the beginning of the part 2024.10.22 00:57:15.903042 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2c34431b7105e5d7ff59038e2f9f1ff8) : [0, 2369470]: Setting download as failed: File segment: [0, 2369470], key: 2c34431b7105e5d7ff59038e2f9f1ff8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:15.905580 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2c34431b7105e5d7ff59038e2f9f1ff8): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:57:15.939799 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_232_232_0, total 1 rows starting from the beginning of the part 2024.10.22 00:57:15.982458 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:15.985714 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:15.987416 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2c34431b7105e5d7ff59038e2f9f1ff8, offset: 0, size: 0 2024.10.22 00:57:15.990354 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2c34431b7105e5d7ff59038e2f9f1ff8 for removal 2024.10.22 00:57:15.994454 [ 1476 ] {} CacheMetadata: Key 2c34431b7105e5d7ff59038e2f9f1ff8 is removed from metadata 2024.10.22 00:57:16.014564 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_233_233_0, total 1 rows starting from the beginning of the part 2024.10.22 00:57:16.083938 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_483_128} MergedBlockOutputStream: filled checksums 202410_1_483_128 (state Temporary) 2024.10.22 00:57:16.114257 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_278_278_0 (state Temporary) 2024.10.22 00:57:16.174479 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_278_278_0 to 202410_291_291_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:16.174699 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_39_8, 202410_40_40_0, 202410_41_41_0, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0] 2024.10.22 00:57:16.177026 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_39_8, 202410_40_40_0, 202410_41_41_0, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0] 2024.10.22 00:57:16.194691 [ 1249 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_286_57 to 202410_291_291_0 2024.10.22 00:57:16.202490 [ 1249 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:16.206450 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3957 2024.10.22 00:57:16.209043 [ 1249 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:16.254788 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_286_57 to 202410_291_291_0 into Compact with storage Full 2024.10.22 00:57:16.281485 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_483_128} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_483_128 to 202410_1_483_128 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:16.285002 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_483_128} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_482_127, 202410_483_483_0] -> 202410_1_483_128 2024.10.22 00:57:16.296100 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_234_234_0, total 1 rows starting from the beginning of the part 2024.10.22 00:57:16.305231 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_39_8, 202410_40_40_0, 202410_41_41_0, 202410_42_42_0, 202410_43_43_0, 202410_44_44_0] 2024.10.22 00:57:16.332183 [ 1397 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_483_128}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 00:57:16.352580 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:16.366442 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.77 MiB. 2024.10.22 00:57:16.370524 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_286_57, total 3986 rows starting from the beginning of the part 2024.10.22 00:57:16.443481 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 10.48 MiB, will set to 1.70 GiB (RSS), difference: -4.97 MiB 2024.10.22 00:57:16.502219 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_287_287_0, total 33 rows starting from the beginning of the part 2024.10.22 00:57:16.547744 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_288_288_0, total 11 rows starting from the beginning of the part 2024.10.22 00:57:16.694252 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_289_289_0, total 12 rows starting from the beginning of the part 2024.10.22 00:57:16.742181 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_290_290_0, total 12 rows starting from the beginning of the part 2024.10.22 00:57:16.788607 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_291_291_0, total 9 rows starting from the beginning of the part 2024.10.22 00:57:17.526344 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 27109 2024.10.22 00:57:17.845712 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:17.854856 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:17.942915 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(156976bf4f4cd9a39dbc2ccfe1416aa9) : [0, 49549]: Setting download as failed: File segment: [0, 49549], key: 156976bf4f4cd9a39dbc2ccfe1416aa9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:17.947326 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(156976bf4f4cd9a39dbc2ccfe1416aa9): No space left in cache to reserve 49550 bytes, will continue without cache download 2024.10.22 00:57:17.988169 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 156976bf4f4cd9a39dbc2ccfe1416aa9, offset: 0, size: 0 2024.10.22 00:57:17.990616 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 156976bf4f4cd9a39dbc2ccfe1416aa9 for removal 2024.10.22 00:57:17.993628 [ 1476 ] {} CacheMetadata: Key 156976bf4f4cd9a39dbc2ccfe1416aa9 is removed from metadata 2024.10.22 00:57:18.070493 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_475_475_0 (state Temporary) 2024.10.22 00:57:18.086817 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} MergeTask::MergeProjectionsStage: Merge sorted 4063 rows, containing 12 columns (12 merged, 0 gathered) in 1.840237161 sec., 2207.8675977786106 rows/sec., 345.64 KiB/sec. 2024.10.22 00:57:18.110272 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_475_475_0 to 202410_488_488_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:18.173808 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_483_128 to 202410_488_488_0 2024.10.22 00:57:18.182375 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:18.188262 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:18.191734 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 27109 2024.10.22 00:57:18.341604 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} MergedBlockOutputStream: filled checksums 202410_1_291_58 (state Temporary) 2024.10.22 00:57:18.347673 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_483_128 to 202410_488_488_0 into Compact with storage Full 2024.10.22 00:57:18.475728 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:18.490253 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_483_128, total 27673 rows starting from the beginning of the part 2024.10.22 00:57:18.540991 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 106 entries to flush up to offset 19316 2024.10.22 00:57:18.684642 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} MergeTask::MergeProjectionsStage: Merge sorted 278 rows, containing 32 columns (32 merged, 0 gathered) in 3.20820419 sec., 86.6528386399246 rows/sec., 9.01 KiB/sec. 2024.10.22 00:57:18.700702 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_484_484_0, total 44 rows starting from the beginning of the part 2024.10.22 00:57:18.780855 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_485_485_0, total 46 rows starting from the beginning of the part 2024.10.22 00:57:18.794277 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_291_58 to 202410_1_291_58 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:18.798933 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_291_58} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_286_57, 202410_291_291_0] -> 202410_1_291_58 2024.10.22 00:57:18.819796 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} MergedBlockOutputStream: filled checksums 202410_1_234_48 (state Temporary) 2024.10.22 00:57:18.839591 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:57:18.885804 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:18.894823 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:18.985565 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_276_276_0 (state Temporary) 2024.10.22 00:57:19.095379 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_486_486_0, total 59 rows starting from the beginning of the part 2024.10.22 00:57:19.139158 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 164 bytes for e3ed51598dd8aee0222dec0f3ec731a7:0) 2024.10.22 00:57:19.147061 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_276_276_0 to 202410_282_282_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:19.148577 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e8336026cc471072747f3e88c0e6fc5f, offset: 0, size: 202 2024.10.22 00:57:19.171096 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e8336026cc471072747f3e88c0e6fc5f for removal 2024.10.22 00:57:19.175499 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 699233bdf92e166f394ff7485fd6f52f, offset: 0, size: 4 2024.10.22 00:57:19.176156 [ 1476 ] {} CacheMetadata: Key e8336026cc471072747f3e88c0e6fc5f is removed from metadata 2024.10.22 00:57:19.188924 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 699233bdf92e166f394ff7485fd6f52f for removal 2024.10.22 00:57:19.195122 [ 1172 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_277_75 to 202410_278_278_0 2024.10.22 00:57:19.208868 [ 1172 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.05 MiB using storage policy from min volume index 0 2024.10.22 00:57:19.228238 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_487_487_0, total 51 rows starting from the beginning of the part 2024.10.22 00:57:19.232303 [ 1172 ] {} DiskLocal: Reserved 1.05 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:19.297595 [ 1476 ] {} CacheMetadata: Key 699233bdf92e166f394ff7485fd6f52f is removed from metadata 2024.10.22 00:57:19.397001 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_278_76} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_277_75 to 202410_278_278_0 into Compact with storage Full 2024.10.22 00:57:19.440741 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 19316 2024.10.22 00:57:19.449628 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_278_76} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:19.457109 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_278_76} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_277_75, total 18878 rows starting from the beginning of the part 2024.10.22 00:57:19.465156 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_488_488_0, total 67 rows starting from the beginning of the part 2024.10.22 00:57:19.477115 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_278_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_278_278_0, total 120 rows starting from the beginning of the part 2024.10.22 00:57:19.607406 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_234_48 to 202410_1_234_48 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:19.617521 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_234_48} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_229_47, 202410_234_234_0] -> 202410_1_234_48 2024.10.22 00:57:19.744323 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:57:19.770497 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_271_54/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:57:19.783350 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:57:19.786430 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:57:19.795756 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 9.44 MiB, will set to 1.70 GiB (RSS), difference: -7.58 MiB 2024.10.22 00:57:19.804403 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 451756 2024.10.22 00:57:20.419241 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_278_76/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:57:20.444561 [ 1156 ] {} DatabaseCatalog: Execution took 354 ms. 2024.10.22 00:57:20.505387 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:20.511556 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:20.765605 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 4 old parts to remove. Parts: [202410_1_219_46, 202410_220_220_0, 202410_221_221_0, 202410_222_222_0] 2024.10.22 00:57:20.772651 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from filesystem (serially): Parts: [202410_1_219_46, 202410_220_220_0, 202410_221_221_0, 202410_222_222_0] 2024.10.22 00:57:20.794842 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from memory: Parts: [202410_1_219_46, 202410_220_220_0, 202410_221_221_0, 202410_222_222_0] 2024.10.22 00:57:20.970439 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_266_266_0 (state Temporary) 2024.10.22 00:57:20.986854 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_278_76} MergeTask::MergeProjectionsStage: Merge sorted 18998 rows, containing 13 columns (13 merged, 0 gathered) in 1.598657215 sec., 11883.72330337245 rows/sec., 5.02 MiB/sec. 2024.10.22 00:57:21.121113 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f5f6a31f924193a0f4fd99f00c1513c9) : [0, 1111842]: Setting download as failed: File segment: [0, 1111842], key: f5f6a31f924193a0f4fd99f00c1513c9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:21.123349 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f5f6a31f924193a0f4fd99f00c1513c9): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:57:21.150207 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_278_76} MergedBlockOutputStream: filled checksums 202410_1_278_76 (state Temporary) 2024.10.22 00:57:21.307039 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f5f6a31f924193a0f4fd99f00c1513c9, offset: 0, size: 0 2024.10.22 00:57:21.311176 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f5f6a31f924193a0f4fd99f00c1513c9 for removal 2024.10.22 00:57:21.321434 [ 1476 ] {} CacheMetadata: Key f5f6a31f924193a0f4fd99f00c1513c9 is removed from metadata 2024.10.22 00:57:21.388636 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_278_76} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_278_76 to 202410_1_278_76 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:21.392332 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_278_76} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_277_75, 202410_278_278_0] -> 202410_1_278_76 2024.10.22 00:57:21.458431 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.70 MiB. 2024.10.22 00:57:21.467224 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_266_266_0 to 202410_273_273_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:21.497833 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 451756 2024.10.22 00:57:22.200339 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 27186 2024.10.22 00:57:22.540715 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:22.594209 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 10.01 MiB, will set to 1.70 GiB (RSS), difference: 17.19 MiB 2024.10.22 00:57:22.648325 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:22.681915 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} MergeTask::MergeProjectionsStage: Merge sorted 27940 rows, containing 13 columns (13 merged, 0 gathered) in 4.346597494 sec., 6428.016405606477 rows/sec., 996.89 KiB/sec. 2024.10.22 00:57:22.744539 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} MergedBlockOutputStream: filled checksums 202410_1_488_129 (state Temporary) 2024.10.22 00:57:22.924047 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_476_476_0 (state Temporary) 2024.10.22 00:57:22.949309 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_488_129 to 202410_1_488_129 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:22.955544 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_488_129} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_483_128, 202410_488_488_0] -> 202410_1_488_129 2024.10.22 00:57:22.992971 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_476_476_0 to 202410_489_489_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:22.996889 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(51a7e4dacce55a8ce7927fc7724baaf8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 51a7e4dacce55a8ce7927fc7724baaf8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 00:57:23.007821 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 717000, file segment info: File segment: [0, 33554431], key: 51a7e4dacce55a8ce7927fc7724baaf8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 00:57:23.011898 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:57:23.014388 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.97 MiB. 2024.10.22 00:57:23.048437 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 27186 2024.10.22 00:57:23.244209 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 243 2024.10.22 00:57:23.289614 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} MergeTask::MergeProjectionsStage: Merge sorted 463042 rows, containing 4 columns (4 merged, 0 gathered) in 18.854681598 sec., 24558.46297871808 rows/sec., 360.14 KiB/sec. 2024.10.22 00:57:23.345021 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} MergedBlockOutputStream: filled checksums 202410_1_271_54 (state Temporary) 2024.10.22 00:57:23.375077 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_271_54 to 202410_1_271_54 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:23.377262 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_271_54} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_266_53, 202410_271_271_0] -> 202410_1_271_54 2024.10.22 00:57:23.513053 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:57:23.594573 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:23.596612 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:23.654339 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_225_225_0 (state Temporary) 2024.10.22 00:57:23.709627 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 39 entries to flush up to offset 3996 2024.10.22 00:57:23.825762 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_225_225_0 to 202410_235_235_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:23.849402 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:23.859847 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 164 bytes for 083f2ba88d4b2e26f0d42cb5428b011c:0) 2024.10.22 00:57:23.969530 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 77e36c683788087adb28df56bb6c5e9d, offset: 0, size: 200 2024.10.22 00:57:23.973662 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 77e36c683788087adb28df56bb6c5e9d for removal 2024.10.22 00:57:23.976595 [ 1476 ] {} CacheMetadata: Key 77e36c683788087adb28df56bb6c5e9d is removed from metadata 2024.10.22 00:57:24.029536 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 243 2024.10.22 00:57:24.064893 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:24.109268 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_279_279_0 (state Temporary) 2024.10.22 00:57:24.381179 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_279_279_0 to 202410_292_292_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:24.458412 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 3996 2024.10.22 00:57:24.720827 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 526 bytes for db5569dcaee885dc70865b94b0dbb7e8:0) 2024.10.22 00:57:24.728709 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 69f6f211494b0a99f7e02c09e5f36bfd, offset: 0, size: 163 2024.10.22 00:57:24.735737 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 69f6f211494b0a99f7e02c09e5f36bfd for removal 2024.10.22 00:57:24.740544 [ 1476 ] {} CacheMetadata: Key 69f6f211494b0a99f7e02c09e5f36bfd is removed from metadata 2024.10.22 00:57:24.741270 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bd45461717f9f68cd5c7326a6a771306, offset: 0, size: 164 2024.10.22 00:57:24.746909 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bd45461717f9f68cd5c7326a6a771306 for removal 2024.10.22 00:57:24.851650 [ 1211 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 00:57:24.856993 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1d00533f83c0275d7fb8a1a44b71a43f, offset: 0, size: 164 2024.10.22 00:57:24.864145 [ 1476 ] {} CacheMetadata: Key bd45461717f9f68cd5c7326a6a771306 is removed from metadata 2024.10.22 00:57:24.861811 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1d00533f83c0275d7fb8a1a44b71a43f for removal 2024.10.22 00:57:24.967207 [ 1476 ] {} CacheMetadata: Key 1d00533f83c0275d7fb8a1a44b71a43f is removed from metadata 2024.10.22 00:57:25.124973 [ 1742 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/aek/cossotqgfcgpivfxtixqwtthhjkqu, size 717000 2024.10.22 00:57:25.523991 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Income 2024.10.22 00:57:25.535756 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Income 2024.10.22 00:57:25.556396 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Income 2024.10.22 00:57:25.566425 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Income 2024.10.22 00:57:25.577145 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Income 2024.10.22 00:57:25.589768 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Income 2024.10.22 00:57:25.611552 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Income 2024.10.22 00:57:25.752613 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xvm/vgcoqoaplocsukqqozsxcytkkrjur (LogSeriesLimiter: on interval from 2024-10-22 00:57:07 to 2024-10-22 00:57:25 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:57:26.375981 [ 1238 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 00:57:26.557497 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(49d8878574a8d3a957fb0c490de5fb99) : [0, 46329]: Setting download as failed: File segment: [0, 46329], key: 49d8878574a8d3a957fb0c490de5fb99, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:26.560565 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(49d8878574a8d3a957fb0c490de5fb99): No space left in cache to reserve 46330 bytes, will continue without cache download 2024.10.22 00:57:26.618224 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 10.08 MiB, will set to 1.70 GiB (RSS), difference: 5.68 MiB 2024.10.22 00:57:26.680918 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 49d8878574a8d3a957fb0c490de5fb99, offset: 0, size: 0 2024.10.22 00:57:26.684188 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 49d8878574a8d3a957fb0c490de5fb99 for removal 2024.10.22 00:57:26.699278 [ 1476 ] {} CacheMetadata: Key 49d8878574a8d3a957fb0c490de5fb99 is removed from metadata 2024.10.22 00:57:26.963607 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 99 entries to flush up to offset 19415 2024.10.22 00:57:26.968080 [ 1188 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:57:27.065092 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 27251 2024.10.22 00:57:27.105761 [ 1188 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:57:27.350582 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:27.354494 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:27.411622 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:27.417645 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:27.432851 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_277_277_0 (state Temporary) 2024.10.22 00:57:27.508620 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 147 bytes for 362c2654096325764c233fe697ee9a6b:0) 2024.10.22 00:57:27.516638 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 03ab05a65d000562b1e4e76a4eabd5f3, offset: 0, size: 202 2024.10.22 00:57:27.523364 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 03ab05a65d000562b1e4e76a4eabd5f3 for removal 2024.10.22 00:57:27.528602 [ 1476 ] {} CacheMetadata: Key 03ab05a65d000562b1e4e76a4eabd5f3 is removed from metadata 2024.10.22 00:57:27.630037 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_477_477_0 (state Temporary) 2024.10.22 00:57:27.643736 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_277_277_0 to 202410_283_283_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:27.719157 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 19415 2024.10.22 00:57:27.685447 [ 1189 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_278_76 to 202410_279_279_0 2024.10.22 00:57:27.804664 [ 1189 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 00:57:27.808989 [ 1189 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:27.837350 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_477_477_0 to 202410_490_490_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:27.905200 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 27251 2024.10.22 00:57:27.957480 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_279_77} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_278_76 to 202410_279_279_0 into Compact with storage Full 2024.10.22 00:57:28.088823 [ 1187 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 00:57:28.099476 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_279_77} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:28.126707 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_279_77} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_278_76, total 18998 rows starting from the beginning of the part 2024.10.22 00:57:28.191207 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_279_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_279_279_0, total 92 rows starting from the beginning of the part 2024.10.22 00:57:28.427639 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 39 file segments (while reserving 34628 bytes for 62ede895e1a15069b5ec9a68fc2dc903:0) 2024.10.22 00:57:28.431271 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 099c4b024460e0e9ae37dac729d8d22f, offset: 0, size: 245 2024.10.22 00:57:28.434010 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 099c4b024460e0e9ae37dac729d8d22f for removal 2024.10.22 00:57:28.439161 [ 1476 ] {} CacheMetadata: Key 099c4b024460e0e9ae37dac729d8d22f is removed from metadata 2024.10.22 00:57:28.439732 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d5fb66a14eb0101f0aa010417cb79366, offset: 0, size: 201 2024.10.22 00:57:28.443279 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d5fb66a14eb0101f0aa010417cb79366 for removal 2024.10.22 00:57:28.444821 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d9a15360d0eb501bf8d0dbf8a8ab8979, offset: 0, size: 10092 2024.10.22 00:57:28.447871 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d9a15360d0eb501bf8d0dbf8a8ab8979 for removal 2024.10.22 00:57:28.453633 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3acda7c12c2ff3c37d2ff93b82fbdf52, offset: 0, size: 257 2024.10.22 00:57:28.458374 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3acda7c12c2ff3c37d2ff93b82fbdf52 for removal 2024.10.22 00:57:28.464927 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c23fb09acff8bda704e630b6c94a8bba, offset: 0, size: 234 2024.10.22 00:57:28.469636 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c23fb09acff8bda704e630b6c94a8bba for removal 2024.10.22 00:57:28.472693 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4782869955bce228f6abdfc42b046b0e, offset: 0, size: 242 2024.10.22 00:57:28.476382 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4782869955bce228f6abdfc42b046b0e for removal 2024.10.22 00:57:28.478646 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 296efeb0a75360d5a0a899f686f5c316, offset: 0, size: 169 2024.10.22 00:57:28.481171 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 296efeb0a75360d5a0a899f686f5c316 for removal 2024.10.22 00:57:28.484859 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 45922d06ec6970f9798d46c027d615f9, offset: 0, size: 164 2024.10.22 00:57:28.489665 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 45922d06ec6970f9798d46c027d615f9 for removal 2024.10.22 00:57:28.496535 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2e728cc06d7914a918e49b49d284d7ac, offset: 0, size: 168 2024.10.22 00:57:28.500091 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2e728cc06d7914a918e49b49d284d7ac for removal 2024.10.22 00:57:28.512980 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 64b135042af36a68eb8e394f02192bfa, offset: 0, size: 7038 2024.10.22 00:57:28.520059 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 64b135042af36a68eb8e394f02192bfa for removal 2024.10.22 00:57:28.528397 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e5377164163bad3d73162572302e1b32, offset: 0, size: 230 2024.10.22 00:57:28.539809 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e5377164163bad3d73162572302e1b32 for removal 2024.10.22 00:57:28.547016 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2698d9b4fb1f0647e8793ca609fca05e, offset: 0, size: 163 2024.10.22 00:57:28.552491 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2698d9b4fb1f0647e8793ca609fca05e for removal 2024.10.22 00:57:28.564141 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8396463580bcf80f780ba08d08d3040e, offset: 0, size: 244 2024.10.22 00:57:28.566840 [ 1476 ] {} CacheMetadata: Key 2698d9b4fb1f0647e8793ca609fca05e is removed from metadata 2024.10.22 00:57:28.572300 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8396463580bcf80f780ba08d08d3040e for removal 2024.10.22 00:57:28.579132 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fa5335d7c628540fa21ee2b3827bd502, offset: 0, size: 162 2024.10.22 00:57:28.579654 [ 1476 ] {} CacheMetadata: Key 8396463580bcf80f780ba08d08d3040e is removed from metadata 2024.10.22 00:57:28.584519 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fa5335d7c628540fa21ee2b3827bd502 for removal 2024.10.22 00:57:28.590922 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4d77867f2f7a8314b1631820f3f76121, offset: 0, size: 165 2024.10.22 00:57:28.594641 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4d77867f2f7a8314b1631820f3f76121 for removal 2024.10.22 00:57:28.595362 [ 1476 ] {} CacheMetadata: Key fa5335d7c628540fa21ee2b3827bd502 is removed from metadata 2024.10.22 00:57:28.599980 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 75f8daca0d198f8e24591684bb0e70ab, offset: 0, size: 267 2024.10.22 00:57:28.603917 [ 1476 ] {} CacheMetadata: Key 4d77867f2f7a8314b1631820f3f76121 is removed from metadata 2024.10.22 00:57:28.604332 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 75f8daca0d198f8e24591684bb0e70ab for removal 2024.10.22 00:57:28.615657 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: df8ed5fafc50b832b5c177cf6bb1f184, offset: 0, size: 239 2024.10.22 00:57:28.620691 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 452908 2024.10.22 00:57:28.621225 [ 1476 ] {} CacheMetadata: Key 75f8daca0d198f8e24591684bb0e70ab is removed from metadata 2024.10.22 00:57:28.624556 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key df8ed5fafc50b832b5c177cf6bb1f184 for removal 2024.10.22 00:57:28.642384 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0fad5f75fe06cdd8a897c9d472991210, offset: 0, size: 234 2024.10.22 00:57:28.647171 [ 1476 ] {} CacheMetadata: Key df8ed5fafc50b832b5c177cf6bb1f184 is removed from metadata 2024.10.22 00:57:28.652082 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0fad5f75fe06cdd8a897c9d472991210 for removal 2024.10.22 00:57:28.655814 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ac569ad996dcc5434e2097bb507f4b24, offset: 0, size: 269 2024.10.22 00:57:28.658087 [ 1476 ] {} CacheMetadata: Key 0fad5f75fe06cdd8a897c9d472991210 is removed from metadata 2024.10.22 00:57:28.661430 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ac569ad996dcc5434e2097bb507f4b24 for removal 2024.10.22 00:57:28.669274 [ 1476 ] {} CacheMetadata: Key e5377164163bad3d73162572302e1b32 is removed from metadata 2024.10.22 00:57:28.672908 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7b070076d2da669a9b92ada5879ef209, offset: 0, size: 242 2024.10.22 00:57:28.676982 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7b070076d2da669a9b92ada5879ef209 for removal 2024.10.22 00:57:28.681739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0f25658a46abe68afb650f4f5db5e817, offset: 0, size: 162 2024.10.22 00:57:28.686195 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0f25658a46abe68afb650f4f5db5e817 for removal 2024.10.22 00:57:28.691115 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2605ed0f433ca84111d361b71549b9f0, offset: 0, size: 164 2024.10.22 00:57:28.695464 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2605ed0f433ca84111d361b71549b9f0 for removal 2024.10.22 00:57:28.699074 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9a5e947b5f32e8b66b11753f542c45f5, offset: 0, size: 198 2024.10.22 00:57:28.778994 [ 1476 ] {} CacheMetadata: Key 2605ed0f433ca84111d361b71549b9f0 is removed from metadata 2024.10.22 00:57:28.788716 [ 1476 ] {} CacheMetadata: Key 0f25658a46abe68afb650f4f5db5e817 is removed from metadata 2024.10.22 00:57:28.804468 [ 1476 ] {} CacheMetadata: Key 7b070076d2da669a9b92ada5879ef209 is removed from metadata 2024.10.22 00:57:28.811853 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9a5e947b5f32e8b66b11753f542c45f5 for removal 2024.10.22 00:57:28.821777 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f9325e4395be876a9edddff7e13877bd, offset: 0, size: 202 2024.10.22 00:57:28.827811 [ 1476 ] {} CacheMetadata: Key 9a5e947b5f32e8b66b11753f542c45f5 is removed from metadata 2024.10.22 00:57:28.830952 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f9325e4395be876a9edddff7e13877bd for removal 2024.10.22 00:57:28.839381 [ 1476 ] {} CacheMetadata: Key f9325e4395be876a9edddff7e13877bd is removed from metadata 2024.10.22 00:57:28.840309 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f51335d06cf6a4049eea1525a20e5219, offset: 0, size: 197 2024.10.22 00:57:28.848136 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f51335d06cf6a4049eea1525a20e5219 for removal 2024.10.22 00:57:28.852269 [ 1476 ] {} CacheMetadata: Key f51335d06cf6a4049eea1525a20e5219 is removed from metadata 2024.10.22 00:57:28.854099 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 78195e8e0e11dd0336bd02686517e0f4, offset: 0, size: 171 2024.10.22 00:57:28.859778 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 78195e8e0e11dd0336bd02686517e0f4 for removal 2024.10.22 00:57:28.871546 [ 1476 ] {} CacheMetadata: Key 78195e8e0e11dd0336bd02686517e0f4 is removed from metadata 2024.10.22 00:57:28.871747 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aec8dd0982d3439bd2c2785220b3cc6e, offset: 0, size: 171 2024.10.22 00:57:28.876440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aec8dd0982d3439bd2c2785220b3cc6e for removal 2024.10.22 00:57:28.878084 [ 1476 ] {} CacheMetadata: Key ac569ad996dcc5434e2097bb507f4b24 is removed from metadata 2024.10.22 00:57:28.879556 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 925bc1743d7b99e92bcf63b58b42b2ce, offset: 0, size: 247 2024.10.22 00:57:28.885098 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 925bc1743d7b99e92bcf63b58b42b2ce for removal 2024.10.22 00:57:28.887725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cce8ff76ffe71552e1ffe2e4276532d8, offset: 0, size: 196 2024.10.22 00:57:28.894475 [ 1476 ] {} CacheMetadata: Key 925bc1743d7b99e92bcf63b58b42b2ce is removed from metadata 2024.10.22 00:57:28.900721 [ 1476 ] {} CacheMetadata: Key aec8dd0982d3439bd2c2785220b3cc6e is removed from metadata 2024.10.22 00:57:28.917812 [ 1476 ] {} CacheMetadata: Key 64b135042af36a68eb8e394f02192bfa is removed from metadata 2024.10.22 00:57:28.931532 [ 1476 ] {} CacheMetadata: Key 2e728cc06d7914a918e49b49d284d7ac is removed from metadata 2024.10.22 00:57:28.957354 [ 1476 ] {} CacheMetadata: Key 45922d06ec6970f9798d46c027d615f9 is removed from metadata 2024.10.22 00:57:28.993211 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cce8ff76ffe71552e1ffe2e4276532d8 for removal 2024.10.22 00:57:28.995259 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: be634fa8796679a616cb4eab9ca42a9a, offset: 0, size: 243 2024.10.22 00:57:28.998133 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key be634fa8796679a616cb4eab9ca42a9a for removal 2024.10.22 00:57:29.005727 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5cbb30c4cb4af8016e36ae62a5ab5bb2, offset: 0, size: 269 2024.10.22 00:57:29.009761 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5cbb30c4cb4af8016e36ae62a5ab5bb2 for removal 2024.10.22 00:57:29.016004 [ 1476 ] {} CacheMetadata: Key be634fa8796679a616cb4eab9ca42a9a is removed from metadata 2024.10.22 00:57:29.026714 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 279163c322fd7ea52d5caa018ec07c4e, offset: 0, size: 4577 2024.10.22 00:57:29.035785 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 279163c322fd7ea52d5caa018ec07c4e for removal 2024.10.22 00:57:29.034994 [ 1476 ] {} CacheMetadata: Key 5cbb30c4cb4af8016e36ae62a5ab5bb2 is removed from metadata 2024.10.22 00:57:29.042561 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1435fe972a9a0744fd5f3d6b25b96c04, offset: 0, size: 162 2024.10.22 00:57:29.046623 [ 1476 ] {} CacheMetadata: Key 279163c322fd7ea52d5caa018ec07c4e is removed from metadata 2024.10.22 00:57:29.047092 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1435fe972a9a0744fd5f3d6b25b96c04 for removal 2024.10.22 00:57:29.052570 [ 1476 ] {} CacheMetadata: Key 1435fe972a9a0744fd5f3d6b25b96c04 is removed from metadata 2024.10.22 00:57:29.052232 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fa2bb7b06751bd45fcdc2726d134fa88, offset: 0, size: 171 2024.10.22 00:57:29.057690 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fa2bb7b06751bd45fcdc2726d134fa88 for removal 2024.10.22 00:57:29.067596 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cc3f213e6093e966df069adea1f280a7, offset: 0, size: 170 2024.10.22 00:57:29.172707 [ 1205 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 212 ms. 2024.10.22 00:57:29.173039 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cc3f213e6093e966df069adea1f280a7 for removal 2024.10.22 00:57:29.183744 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ad8e3a73ce69e5310a0dd057011fb1a, offset: 0, size: 5712 2024.10.22 00:57:29.190830 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ad8e3a73ce69e5310a0dd057011fb1a for removal 2024.10.22 00:57:29.260563 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_211_42, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0, 202410_215_215_0, 202410_216_216_0] 2024.10.22 00:57:29.267595 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_211_42, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0, 202410_215_215_0, 202410_216_216_0] 2024.10.22 00:57:29.270650 [ 1476 ] {} CacheMetadata: Key 7ad8e3a73ce69e5310a0dd057011fb1a is removed from metadata 2024.10.22 00:57:29.277840 [ 1476 ] {} CacheMetadata: Key cc3f213e6093e966df069adea1f280a7 is removed from metadata 2024.10.22 00:57:29.284086 [ 1476 ] {} CacheMetadata: Key fa2bb7b06751bd45fcdc2726d134fa88 is removed from metadata 2024.10.22 00:57:29.197392 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d107b36fe5b8bb14136c6bc3b47b5939, offset: 0, size: 163 2024.10.22 00:57:29.301036 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_211_42, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0, 202410_215_215_0, 202410_216_216_0] 2024.10.22 00:57:29.302455 [ 1476 ] {} CacheMetadata: Key cce8ff76ffe71552e1ffe2e4276532d8 is removed from metadata 2024.10.22 00:57:29.303307 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d107b36fe5b8bb14136c6bc3b47b5939 for removal 2024.10.22 00:57:29.317760 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9b0b98157f3aaac8ea8df54a5727fbd3, offset: 0, size: 196 2024.10.22 00:57:29.330206 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9b0b98157f3aaac8ea8df54a5727fbd3 for removal 2024.10.22 00:57:29.338348 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d70f4e4bf8d8e151d0911a21546288aa, offset: 0, size: 168 2024.10.22 00:57:29.348015 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d70f4e4bf8d8e151d0911a21546288aa for removal 2024.10.22 00:57:29.424285 [ 1476 ] {} CacheMetadata: Key d70f4e4bf8d8e151d0911a21546288aa is removed from metadata 2024.10.22 00:57:29.428992 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:29.438890 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:29.444139 [ 1476 ] {} CacheMetadata: Key 9b0b98157f3aaac8ea8df54a5727fbd3 is removed from metadata 2024.10.22 00:57:29.478464 [ 1476 ] {} CacheMetadata: Key d107b36fe5b8bb14136c6bc3b47b5939 is removed from metadata 2024.10.22 00:57:29.493683 [ 1476 ] {} CacheMetadata: Key 296efeb0a75360d5a0a899f686f5c316 is removed from metadata 2024.10.22 00:57:29.603583 [ 1476 ] {} CacheMetadata: Key 4782869955bce228f6abdfc42b046b0e is removed from metadata 2024.10.22 00:57:29.613441 [ 1476 ] {} CacheMetadata: Key c23fb09acff8bda704e630b6c94a8bba is removed from metadata 2024.10.22 00:57:29.786677 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:57:29.791148 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:57:29.808336 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_267_267_0 (state Temporary) 2024.10.22 00:57:29.829338 [ 1476 ] {} CacheMetadata: Key 3acda7c12c2ff3c37d2ff93b82fbdf52 is removed from metadata 2024.10.22 00:57:29.855773 [ 1476 ] {} CacheMetadata: Key d9a15360d0eb501bf8d0dbf8a8ab8979 is removed from metadata 2024.10.22 00:57:29.870988 [ 1476 ] {} CacheMetadata: Key d5fb66a14eb0101f0aa010417cb79366 is removed from metadata 2024.10.22 00:57:30.000715 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_267_267_0 to 202410_274_274_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:30.049649 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 452908 2024.10.22 00:57:30.390770 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_279_77} MergeTask::MergeProjectionsStage: Merge sorted 19090 rows, containing 13 columns (13 merged, 0 gathered) in 2.545651085 sec., 7499.063839693491 rows/sec., 3.17 MiB/sec. 2024.10.22 00:57:30.571552 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 6.96 MiB, will set to 1.71 GiB (RSS), difference: -10.64 MiB 2024.10.22 00:57:30.646669 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_279_77} MergedBlockOutputStream: filled checksums 202410_1_279_77 (state Temporary) 2024.10.22 00:57:30.827409 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 153 bytes for 6085d5e624f0fe08bc141e70917a5339:0) 2024.10.22 00:57:30.836446 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f12c24ad57f393f9f9fb14c258169299, offset: 0, size: 257 2024.10.22 00:57:30.843272 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f12c24ad57f393f9f9fb14c258169299 for removal 2024.10.22 00:57:30.847227 [ 1476 ] {} CacheMetadata: Key f12c24ad57f393f9f9fb14c258169299 is removed from metadata 2024.10.22 00:57:30.902620 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_279_77} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_279_77 to 202410_1_279_77 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:30.910404 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_279_77} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_278_76, 202410_279_279_0] -> 202410_1_279_77 2024.10.22 00:57:30.964962 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.66 MiB. 2024.10.22 00:57:31.536993 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 244 2024.10.22 00:57:31.883546 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3a1b446f4e7c31bede93df5ae27ab622) : [0, 278174]: Setting download as failed: File segment: [0, 278174], key: 3a1b446f4e7c31bede93df5ae27ab622, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:31.885468 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3a1b446f4e7c31bede93df5ae27ab622): No space left in cache to reserve 278175 bytes, will continue without cache download 2024.10.22 00:57:31.949746 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3a1b446f4e7c31bede93df5ae27ab622, offset: 0, size: 0 2024.10.22 00:57:31.970946 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 4004 2024.10.22 00:57:32.038531 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3a1b446f4e7c31bede93df5ae27ab622 for removal 2024.10.22 00:57:32.040876 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 167 entries to flush up to offset 27418 2024.10.22 00:57:32.043479 [ 1476 ] {} CacheMetadata: Key 3a1b446f4e7c31bede93df5ae27ab622 is removed from metadata 2024.10.22 00:57:32.273757 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:32.276956 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:32.296487 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:32.319794 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_280_280_0 (state Temporary) 2024.10.22 00:57:32.404816 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:32.495578 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_226_226_0 (state Temporary) 2024.10.22 00:57:32.568076 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 6.16 MiB, will set to 1.70 GiB (RSS), difference: -8.39 MiB 2024.10.22 00:57:32.585431 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_280_280_0 to 202410_293_293_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:32.599347 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_226_226_0 to 202410_236_236_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:32.665004 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4004 2024.10.22 00:57:32.734078 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 244 2024.10.22 00:57:32.787944 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:32.790991 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:32.880815 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:32.884438 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:33.070497 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_478_478_0 (state Temporary) 2024.10.22 00:57:33.276287 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_478_478_0 to 202410_491_491_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:33.300969 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 27418 2024.10.22 00:57:33.704414 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(034170048649e22c2a5cc2562add6abe) : [0, 182173]: Setting download as failed: File segment: [0, 182173], key: 034170048649e22c2a5cc2562add6abe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:33.709330 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(034170048649e22c2a5cc2562add6abe): No space left in cache to reserve 182174 bytes, will continue without cache download 2024.10.22 00:57:33.838562 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 034170048649e22c2a5cc2562add6abe, offset: 0, size: 0 2024.10.22 00:57:33.840584 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 034170048649e22c2a5cc2562add6abe for removal 2024.10.22 00:57:33.844966 [ 1476 ] {} CacheMetadata: Key 034170048649e22c2a5cc2562add6abe is removed from metadata 2024.10.22 00:57:34.731412 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_386_83, 202410_387_387_0, 202410_388_388_0, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0] 2024.10.22 00:57:34.739607 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_386_83, 202410_387_387_0, 202410_388_388_0, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0] 2024.10.22 00:57:34.991543 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_386_83, 202410_387_387_0, 202410_388_388_0, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0] 2024.10.22 00:57:35.060808 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 187 bytes for 424095cbd7579caf62a6e6f42c2177b4:0) 2024.10.22 00:57:35.066771 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c729fd8aabdd1fa51c57b4fb3736db10, offset: 0, size: 248 2024.10.22 00:57:35.072477 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c729fd8aabdd1fa51c57b4fb3736db10 for removal 2024.10.22 00:57:35.075901 [ 1476 ] {} CacheMetadata: Key c729fd8aabdd1fa51c57b4fb3736db10 is removed from metadata 2024.10.22 00:57:35.278057 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 66 entries to flush up to offset 19481 2024.10.22 00:57:35.495573 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 6.59 MiB, will set to 1.71 GiB (RSS), difference: 8.48 MiB 2024.10.22 00:57:35.580260 [ 1166 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 352 ms. 2024.10.22 00:57:35.678424 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:35.681080 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:35.833647 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_278_278_0 (state Temporary) 2024.10.22 00:57:35.879610 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_278_278_0 to 202410_284_284_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:35.897073 [ 1243 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_279_77 to 202410_280_280_0 2024.10.22 00:57:35.907102 [ 1243 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 00:57:35.911443 [ 1228 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 325 ms. 2024.10.22 00:57:35.918873 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 19481 2024.10.22 00:57:36.023189 [ 1243 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:36.065715 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_280_78} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_279_77 to 202410_280_280_0 into Compact with storage Full 2024.10.22 00:57:36.132449 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f4001c83e385b4c2cc7400bde60efc58) : [0, 400245]: Setting download as failed: File segment: [0, 400245], key: f4001c83e385b4c2cc7400bde60efc58, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:36.135454 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f4001c83e385b4c2cc7400bde60efc58): No space left in cache to reserve 400246 bytes, will continue without cache download 2024.10.22 00:57:36.169218 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f4001c83e385b4c2cc7400bde60efc58, offset: 0, size: 0 2024.10.22 00:57:36.174255 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f4001c83e385b4c2cc7400bde60efc58 for removal 2024.10.22 00:57:36.187409 [ 1476 ] {} CacheMetadata: Key f4001c83e385b4c2cc7400bde60efc58 is removed from metadata 2024.10.22 00:57:36.226797 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_280_78} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:36.255328 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_280_78} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_279_77, total 19090 rows starting from the beginning of the part 2024.10.22 00:57:36.290894 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_280_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_280_280_0, total 90 rows starting from the beginning of the part 2024.10.22 00:57:36.894987 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_56_56_0 (state Temporary) 2024.10.22 00:57:37.049626 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_280_78/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:57:37.080651 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1525 entries to flush up to offset 454433 2024.10.22 00:57:37.518156 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 27477 2024.10.22 00:57:37.731746 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 159 bytes for 3948567fdde35c89d3806cb8ec85f373:0) 2024.10.22 00:57:37.740740 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 50bf040860912c8c337f19741cfde4e9, offset: 0, size: 246 2024.10.22 00:57:37.751249 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 50bf040860912c8c337f19741cfde4e9 for removal 2024.10.22 00:57:37.761322 [ 1476 ] {} CacheMetadata: Key 50bf040860912c8c337f19741cfde4e9 is removed from metadata 2024.10.22 00:57:37.780760 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:37.783294 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:37.926290 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_479_479_0 (state Temporary) 2024.10.22 00:57:38.023444 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_280_78} MergeTask::MergeProjectionsStage: Merge sorted 19180 rows, containing 13 columns (13 merged, 0 gathered) in 1.963011156 sec., 9770.703514025266 rows/sec., 4.12 MiB/sec. 2024.10.22 00:57:38.086121 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_280_78} MergedBlockOutputStream: filled checksums 202410_1_280_78 (state Temporary) 2024.10.22 00:57:38.175142 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_479_479_0 to 202410_492_492_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:38.238862 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 27477 2024.10.22 00:57:38.440521 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:38.444637 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:38.506354 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_280_78} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_280_78 to 202410_1_280_78 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:38.514095 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_280_78} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_279_77, 202410_280_280_0] -> 202410_1_280_78 2024.10.22 00:57:38.545107 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.78 MiB. 2024.10.22 00:57:38.668652 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 12 file segments (while reserving 5149 bytes for e28f35a5a60d60f84d4843af9da3e99f:0) 2024.10.22 00:57:38.670412 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 4.86 MiB, will set to 1.71 GiB (RSS), difference: -3.82 MiB 2024.10.22 00:57:38.672180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2ae2dc1d9e795d3d73100bea041ee536, offset: 0, size: 254 2024.10.22 00:57:38.675285 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2ae2dc1d9e795d3d73100bea041ee536 for removal 2024.10.22 00:57:38.677658 [ 1476 ] {} CacheMetadata: Key 2ae2dc1d9e795d3d73100bea041ee536 is removed from metadata 2024.10.22 00:57:38.677738 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_268_268_0 (state Temporary) 2024.10.22 00:57:38.678129 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2eebe21c5443e6ba9df2bf76383739bd, offset: 0, size: 265 2024.10.22 00:57:38.692965 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2eebe21c5443e6ba9df2bf76383739bd for removal 2024.10.22 00:57:38.697160 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5f76ad61992998c31c902a163760a70b, offset: 0, size: 245 2024.10.22 00:57:38.701984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5f76ad61992998c31c902a163760a70b for removal 2024.10.22 00:57:38.705787 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c4568d99cebaa9cc604c74aa1887d6f8, offset: 0, size: 2532 2024.10.22 00:57:38.715494 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c4568d99cebaa9cc604c74aa1887d6f8 for removal 2024.10.22 00:57:38.718801 [ 1476 ] {} CacheMetadata: Key 5f76ad61992998c31c902a163760a70b is removed from metadata 2024.10.22 00:57:38.721709 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a45874a1efc21fcce9a9a3a3658aa134, offset: 0, size: 255 2024.10.22 00:57:38.725110 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a45874a1efc21fcce9a9a3a3658aa134 for removal 2024.10.22 00:57:38.730812 [ 1476 ] {} CacheMetadata: Key a45874a1efc21fcce9a9a3a3658aa134 is removed from metadata 2024.10.22 00:57:38.731127 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bddfc383258868b17374393b2663b48c, offset: 0, size: 245 2024.10.22 00:57:38.738365 [ 1476 ] {} CacheMetadata: Key c4568d99cebaa9cc604c74aa1887d6f8 is removed from metadata 2024.10.22 00:57:38.738536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bddfc383258868b17374393b2663b48c for removal 2024.10.22 00:57:38.738438 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_268_268_0 to 202410_275_275_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:38.742306 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9588679302cd47f658e886f5b0ba5a06, offset: 0, size: 260 2024.10.22 00:57:38.747963 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9588679302cd47f658e886f5b0ba5a06 for removal 2024.10.22 00:57:38.748673 [ 1476 ] {} CacheMetadata: Key bddfc383258868b17374393b2663b48c is removed from metadata 2024.10.22 00:57:38.764098 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 74d7e0666bc4d087876536cd04843118, offset: 0, size: 163 2024.10.22 00:57:38.771880 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 74d7e0666bc4d087876536cd04843118 for removal 2024.10.22 00:57:38.772993 [ 1476 ] {} CacheMetadata: Key 9588679302cd47f658e886f5b0ba5a06 is removed from metadata 2024.10.22 00:57:38.785054 [ 1476 ] {} CacheMetadata: Key 74d7e0666bc4d087876536cd04843118 is removed from metadata 2024.10.22 00:57:38.785668 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6ab16a50b1aa345fd5d2f701f68c3423, offset: 0, size: 256 2024.10.22 00:57:38.793233 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6ab16a50b1aa345fd5d2f701f68c3423 for removal 2024.10.22 00:57:38.795707 [ 1476 ] {} CacheMetadata: Key 2eebe21c5443e6ba9df2bf76383739bd is removed from metadata 2024.10.22 00:57:38.797790 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0c5e54f8d4ab6b930baec6278261eaec, offset: 0, size: 246 2024.10.22 00:57:38.803571 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0c5e54f8d4ab6b930baec6278261eaec for removal 2024.10.22 00:57:38.812448 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9a17e92841b4e041be2357560f56d4b5, offset: 0, size: 246 2024.10.22 00:57:38.821222 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9a17e92841b4e041be2357560f56d4b5 for removal 2024.10.22 00:57:38.826936 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 41a4663551aa5329f04a01692442ae50, offset: 0, size: 249 2024.10.22 00:57:38.833621 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 41a4663551aa5329f04a01692442ae50 for removal 2024.10.22 00:57:38.889291 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 454433 2024.10.22 00:57:38.904511 [ 1476 ] {} CacheMetadata: Key 41a4663551aa5329f04a01692442ae50 is removed from metadata 2024.10.22 00:57:38.930210 [ 1476 ] {} CacheMetadata: Key 9a17e92841b4e041be2357560f56d4b5 is removed from metadata 2024.10.22 00:57:38.944139 [ 1476 ] {} CacheMetadata: Key 0c5e54f8d4ab6b930baec6278261eaec is removed from metadata 2024.10.22 00:57:38.960631 [ 1476 ] {} CacheMetadata: Key 6ab16a50b1aa345fd5d2f701f68c3423 is removed from metadata 2024.10.22 00:57:39.785961 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:57:39.789616 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:57:40.173163 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 4013 2024.10.22 00:57:40.224301 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(103930fe7df83f46293b0aa9f1518a5d) : [0, 73090]: Setting download as failed: File segment: [0, 73090], key: 103930fe7df83f46293b0aa9f1518a5d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:40.226213 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(103930fe7df83f46293b0aa9f1518a5d): No space left in cache to reserve 73091 bytes, will continue without cache download 2024.10.22 00:57:40.238190 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 245 2024.10.22 00:57:40.371745 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:40.375769 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:40.349775 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 103930fe7df83f46293b0aa9f1518a5d, offset: 0, size: 0 2024.10.22 00:57:40.454196 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 103930fe7df83f46293b0aa9f1518a5d for removal 2024.10.22 00:57:40.459432 [ 1476 ] {} CacheMetadata: Key 103930fe7df83f46293b0aa9f1518a5d is removed from metadata 2024.10.22 00:57:40.523842 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_281_281_0 (state Temporary) 2024.10.22 00:57:40.629176 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_281_281_0 to 202410_294_294_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:40.667835 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:40.680104 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 5.57 MiB, will set to 1.71 GiB (RSS), difference: -1.35 MiB 2024.10.22 00:57:40.741953 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:40.800144 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4013 2024.10.22 00:57:40.925750 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_227_227_0 (state Temporary) 2024.10.22 00:57:41.165315 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_227_227_0 to 202410_237_237_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:41.356614 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 245 2024.10.22 00:57:41.502926 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 160 bytes for d64854667af7ae74b2bdb00addf395a0:0) 2024.10.22 00:57:41.504773 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d71b36e2cef3474d135d07f201de17d9, offset: 0, size: 171 2024.10.22 00:57:41.509622 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d71b36e2cef3474d135d07f201de17d9 for removal 2024.10.22 00:57:41.519040 [ 1476 ] {} CacheMetadata: Key d71b36e2cef3474d135d07f201de17d9 is removed from metadata 2024.10.22 00:57:42.117764 [ 1178 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:57:42.144111 [ 1178 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:57:42.253232 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 27561 2024.10.22 00:57:42.260425 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/joa/uvwsudilnfximdksetazbxhkjoxaa, upload id 4b41fd86-bb5b-46f6-a981-76839d59ef91 2024.10.22 00:57:42.287700 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ab2772e10e4e5f8ba44c6003de53ad76) : [0, 82111]: Setting download as failed: File segment: [0, 82111], key: ab2772e10e4e5f8ba44c6003de53ad76, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:42.290596 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ab2772e10e4e5f8ba44c6003de53ad76): No space left in cache to reserve 82112 bytes, will continue without cache download 2024.10.22 00:57:42.323738 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ab2772e10e4e5f8ba44c6003de53ad76, offset: 0, size: 0 2024.10.22 00:57:42.328522 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ab2772e10e4e5f8ba44c6003de53ad76 for removal 2024.10.22 00:57:42.337454 [ 1476 ] {} CacheMetadata: Key ab2772e10e4e5f8ba44c6003de53ad76 is removed from metadata 2024.10.22 00:57:42.824256 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:42.828018 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.17 GiB. 2024.10.22 00:57:42.998585 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_480_480_0 (state Temporary) 2024.10.22 00:57:43.195382 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_480_480_0 to 202410_493_493_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:43.265417 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1_488_129 to 202410_491_491_0 2024.10.22 00:57:43.286357 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:43.290868 [ 1176 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.14 GiB. 2024.10.22 00:57:43.296037 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 27561 2024.10.22 00:57:43.426368 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 67 entries to flush up to offset 19548 2024.10.22 00:57:43.444719 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_491_130} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_488_129 to 202410_491_491_0 into Compact with storage Full 2024.10.22 00:57:43.518977 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(beebf6d1365afb7e78aa3d1fe5b3af95) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: beebf6d1365afb7e78aa3d1fe5b3af95, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 00:57:43.522965 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 695210, file segment info: File segment: [0, 33554431], key: beebf6d1365afb7e78aa3d1fe5b3af95, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 00:57:43.529460 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:57:43.555949 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_491_130} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:43.610182 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 4.90 MiB, will set to 1.71 GiB (RSS), difference: 38.66 MiB 2024.10.22 00:57:43.650126 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_491_130} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_488_129, total 27940 rows starting from the beginning of the part 2024.10.22 00:57:43.700213 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_491_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_489_489_0, total 77 rows starting from the beginning of the part 2024.10.22 00:57:43.733599 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:43.738399 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.08 GiB. 2024.10.22 00:57:43.884171 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_279_279_0 (state Temporary) 2024.10.22 00:57:43.900775 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_491_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_490_490_0, total 65 rows starting from the beginning of the part 2024.10.22 00:57:44.007666 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_491_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_491_491_0, total 167 rows starting from the beginning of the part 2024.10.22 00:57:44.062552 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_279_279_0 to 202410_285_285_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:44.177336 [ 1182 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_280_78 to 202410_281_281_0 2024.10.22 00:57:44.182584 [ 1182 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.07 MiB using storage policy from min volume index 0 2024.10.22 00:57:44.196325 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 19548 2024.10.22 00:57:44.185437 [ 1182 ] {} DiskLocal: Reserved 1.07 MiB on local disk `default`, having unreserved 298.08 GiB. 2024.10.22 00:57:44.229242 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_281_79} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_280_78 to 202410_281_281_0 into Compact with storage Full 2024.10.22 00:57:44.245081 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 165 bytes for a92b4d54a7dc64d5deea9d30a37bd731:0) 2024.10.22 00:57:44.249670 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c5fc14df8bfbb1fa60b1de903394bb52, offset: 0, size: 217 2024.10.22 00:57:44.269362 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c5fc14df8bfbb1fa60b1de903394bb52 for removal 2024.10.22 00:57:44.272236 [ 1476 ] {} CacheMetadata: Key c5fc14df8bfbb1fa60b1de903394bb52 is removed from metadata 2024.10.22 00:57:44.288425 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_281_79} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:44.320316 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_281_79} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_280_78, total 19180 rows starting from the beginning of the part 2024.10.22 00:57:44.455341 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_281_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_281_281_0, total 134 rows starting from the beginning of the part 2024.10.22 00:57:44.612737 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_222_47, 202410_223_223_0] 2024.10.22 00:57:44.617469 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_222_47, 202410_223_223_0] 2024.10.22 00:57:44.629674 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_222_47, 202410_223_223_0] 2024.10.22 00:57:45.277696 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 527 bytes for db4a202d6e86714e47e4c2bc374bd1c5:0) 2024.10.22 00:57:45.280396 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: abb4eafa9b45198a6bc64c796566b018, offset: 0, size: 2637 2024.10.22 00:57:45.287743 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key abb4eafa9b45198a6bc64c796566b018 for removal 2024.10.22 00:57:45.290936 [ 1476 ] {} CacheMetadata: Key abb4eafa9b45198a6bc64c796566b018 is removed from metadata 2024.10.22 00:57:45.292076 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 62ec0818ed558e163f84081c2929bc97, offset: 0, size: 164 2024.10.22 00:57:45.297493 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 62ec0818ed558e163f84081c2929bc97 for removal 2024.10.22 00:57:45.300269 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 61c86fa30d13798bfd8a7288176dc591, offset: 0, size: 159 2024.10.22 00:57:45.303514 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 61c86fa30d13798bfd8a7288176dc591 for removal 2024.10.22 00:57:45.315256 [ 1476 ] {} CacheMetadata: Key 61c86fa30d13798bfd8a7288176dc591 is removed from metadata 2024.10.22 00:57:45.335823 [ 1476 ] {} CacheMetadata: Key 62ec0818ed558e163f84081c2929bc97 is removed from metadata 2024.10.22 00:57:45.402648 [ 1719 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xvm/vgcoqoaplocsukqqozsxcytkkrjur, size 695210 2024.10.22 00:57:45.894070 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 779 entries to flush up to offset 455212 2024.10.22 00:57:46.025785 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Interests 2024.10.22 00:57:46.051352 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Interests 2024.10.22 00:57:45.965504 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_56_56_0 to 202410_59_59_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:46.078549 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Interests 2024.10.22 00:57:46.110017 [ 1274 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_54_11 to 202410_59_59_0 2024.10.22 00:57:46.120729 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Interests 2024.10.22 00:57:46.140850 [ 1274 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.32 MiB using storage policy from min volume index 0 2024.10.22 00:57:46.155818 [ 1274 ] {} DiskLocal: Reserved 1.32 MiB on local disk `default`, having unreserved 298.08 GiB. 2024.10.22 00:57:46.164401 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_281_79} MergeTask::MergeProjectionsStage: Merge sorted 19314 rows, containing 13 columns (13 merged, 0 gathered) in 1.941539554 sec., 9947.77570212716 rows/sec., 4.20 MiB/sec. 2024.10.22 00:57:46.178991 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_54_11 to 202410_59_59_0 into Compact with storage Full 2024.10.22 00:57:46.285032 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Interests 2024.10.22 00:57:46.320523 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Interests 2024.10.22 00:57:46.349267 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Interests 2024.10.22 00:57:46.382697 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_281_79} MergedBlockOutputStream: filled checksums 202410_1_281_79 (state Temporary) 2024.10.22 00:57:46.473631 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:46.480053 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.07 GiB. 2024.10.22 00:57:46.507219 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d2706253995610f701896a055aadfd66) : [0, 127251]: Setting download as failed: File segment: [0, 127251], key: d2706253995610f701896a055aadfd66, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:46.513108 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d2706253995610f701896a055aadfd66): No space left in cache to reserve 127252 bytes, will continue without cache download 2024.10.22 00:57:46.515862 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xhw/headhyumksywuxsilagctcoopnpem (LogSeriesLimiter: on interval from 2024-10-22 00:57:25 to 2024-10-22 00:57:46 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:57:46.537842 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_281_79} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_281_79 to 202410_1_281_79 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:46.543086 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_281_79} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_280_78, 202410_281_281_0] -> 202410_1_281_79 2024.10.22 00:57:46.548232 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_491_130/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:57:46.578221 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.82 MiB. 2024.10.22 00:57:46.642769 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d2706253995610f701896a055aadfd66, offset: 0, size: 0 2024.10.22 00:57:46.670267 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d2706253995610f701896a055aadfd66 for removal 2024.10.22 00:57:46.674818 [ 1476 ] {} CacheMetadata: Key d2706253995610f701896a055aadfd66 is removed from metadata 2024.10.22 00:57:46.743784 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_269_269_0 (state Temporary) 2024.10.22 00:57:46.785320 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_184_38, 202410_185_185_0, 202410_186_186_0, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0] 2024.10.22 00:57:46.796468 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_184_38, 202410_185_185_0, 202410_186_186_0, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0] 2024.10.22 00:57:46.817541 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_269_269_0 to 202410_276_276_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:46.864374 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 455212 2024.10.22 00:57:46.938526 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_184_38, 202410_185_185_0, 202410_186_186_0, 202410_187_187_0, 202410_188_188_0, 202410_189_189_0] 2024.10.22 00:57:46.965745 [ 1204 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_271_54 to 202410_276_276_0 2024.10.22 00:57:46.982326 [ 1204 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:46.990237 [ 1204 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.08 GiB. 2024.10.22 00:57:47.027758 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_271_54 to 202410_276_276_0 into Compact with storage Full 2024.10.22 00:57:47.092756 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:47.099294 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} MergeTreeSequentialSource: Reading 58 marks from part 202410_1_271_54, total 463042 rows starting from the beginning of the part 2024.10.22 00:57:47.250644 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_272_272_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:57:47.313277 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 27657 2024.10.22 00:57:47.396905 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_273_273_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:57:47.429788 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_491_130} MergeTask::MergeProjectionsStage: Merge sorted 28249 rows, containing 13 columns (13 merged, 0 gathered) in 3.999213893 sec., 7063.638193857415 rows/sec., 1.07 MiB/sec. 2024.10.22 00:57:47.493034 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_274_274_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:57:47.513081 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_491_130} MergedBlockOutputStream: filled checksums 202410_1_491_130 (state Temporary) 2024.10.22 00:57:47.555035 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_275_275_0, total 1525 rows starting from the beginning of the part 2024.10.22 00:57:47.600627 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_276_276_0, total 779 rows starting from the beginning of the part 2024.10.22 00:57:47.671179 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:47.677017 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.07 GiB. 2024.10.22 00:57:47.830998 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_481_481_0 (state Temporary) 2024.10.22 00:57:47.849446 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_491_130} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_491_130 to 202410_1_491_130 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:47.854406 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_491_130} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1_488_129, 202410_491_491_0] -> 202410_1_491_130 2024.10.22 00:57:47.974249 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(50214f710ebbacbc77509855e7a27bfb) : [0, 123106]: Setting download as failed: File segment: [0, 123106], key: 50214f710ebbacbc77509855e7a27bfb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:47.979353 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_481_481_0 to 202410_494_494_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:47.977762 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(50214f710ebbacbc77509855e7a27bfb): No space left in cache to reserve 123107 bytes, will continue without cache download 2024.10.22 00:57:47.994639 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.88 MiB. 2024.10.22 00:57:47.997369 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 50214f710ebbacbc77509855e7a27bfb, offset: 0, size: 0 2024.10.22 00:57:48.000150 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 50214f710ebbacbc77509855e7a27bfb for removal 2024.10.22 00:57:48.002499 [ 1476 ] {} CacheMetadata: Key 50214f710ebbacbc77509855e7a27bfb is removed from metadata 2024.10.22 00:57:48.144906 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_276_55/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:57:48.148025 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 27657 2024.10.22 00:57:48.303455 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 4028 2024.10.22 00:57:48.435678 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:48.439899 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.07 GiB. 2024.10.22 00:57:48.481197 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_282_282_0 (state Temporary) 2024.10.22 00:57:48.499649 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 4.52 MiB, will set to 1.67 GiB (RSS), difference: -47.69 MiB 2024.10.22 00:57:48.766535 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_282_282_0 to 202410_295_295_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:48.858716 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4028 2024.10.22 00:57:48.892252 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2253 2024.10.22 00:57:48.991137 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_231_46, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0, 202410_236_236_0] 2024.10.22 00:57:48.992226 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_231_46, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0, 202410_236_236_0] 2024.10.22 00:57:49.020575 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_231_46, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0, 202410_236_236_0] 2024.10.22 00:57:49.385488 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(96e1d455df71e2ff652f8ddf8a3ef966) : [0, 61071]: Setting download as failed: File segment: [0, 61071], key: 96e1d455df71e2ff652f8ddf8a3ef966, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:49.386518 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(96e1d455df71e2ff652f8ddf8a3ef966): No space left in cache to reserve 61072 bytes, will continue without cache download 2024.10.22 00:57:49.395640 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:57:49.398208 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 96e1d455df71e2ff652f8ddf8a3ef966, offset: 0, size: 0 2024.10.22 00:57:49.477064 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 96e1d455df71e2ff652f8ddf8a3ef966 for removal 2024.10.22 00:57:49.479214 [ 1476 ] {} CacheMetadata: Key 96e1d455df71e2ff652f8ddf8a3ef966 is removed from metadata 2024.10.22 00:57:49.787552 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:57:49.890838 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:57:50.297202 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d58aeeba38fa76ad3ad7ea215e3ec831) : [0, 58856]: Setting download as failed: File segment: [0, 58856], key: d58aeeba38fa76ad3ad7ea215e3ec831, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:50.299325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d58aeeba38fa76ad3ad7ea215e3ec831): No space left in cache to reserve 58857 bytes, will continue without cache download 2024.10.22 00:57:50.313625 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d58aeeba38fa76ad3ad7ea215e3ec831, offset: 0, size: 0 2024.10.22 00:57:50.317003 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d58aeeba38fa76ad3ad7ea215e3ec831 for removal 2024.10.22 00:57:50.320205 [ 1476 ] {} CacheMetadata: Key d58aeeba38fa76ad3ad7ea215e3ec831 is removed from metadata 2024.10.22 00:57:51.111873 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 35337 bytes for c054404b7b2afa5f97ebc65cdd2565e0:0) 2024.10.22 00:57:51.118314 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 62ede895e1a15069b5ec9a68fc2dc903, offset: 0, size: 34628 2024.10.22 00:57:51.128897 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 62ede895e1a15069b5ec9a68fc2dc903 for removal 2024.10.22 00:57:51.237825 [ 1476 ] {} CacheMetadata: Key 62ede895e1a15069b5ec9a68fc2dc903 is removed from metadata 2024.10.22 00:57:51.239436 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 362c2654096325764c233fe697ee9a6b, offset: 0, size: 147 2024.10.22 00:57:51.243871 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 362c2654096325764c233fe697ee9a6b for removal 2024.10.22 00:57:51.248476 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 083f2ba88d4b2e26f0d42cb5428b011c, offset: 0, size: 164 2024.10.22 00:57:51.254764 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 083f2ba88d4b2e26f0d42cb5428b011c for removal 2024.10.22 00:57:51.262333 [ 1476 ] {} CacheMetadata: Key 083f2ba88d4b2e26f0d42cb5428b011c is removed from metadata 2024.10.22 00:57:51.258709 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e3ed51598dd8aee0222dec0f3ec731a7, offset: 0, size: 164 2024.10.22 00:57:51.271894 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e3ed51598dd8aee0222dec0f3ec731a7 for removal 2024.10.22 00:57:51.287864 [ 1476 ] {} CacheMetadata: Key e3ed51598dd8aee0222dec0f3ec731a7 is removed from metadata 2024.10.22 00:57:51.308363 [ 1476 ] {} CacheMetadata: Key 362c2654096325764c233fe697ee9a6b is removed from metadata 2024.10.22 00:57:51.406227 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 15.26 MiB, will set to 1.68 GiB (RSS), difference: 21.15 MiB 2024.10.22 00:57:51.572120 [ 1171 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 255 ms. 2024.10.22 00:57:51.716997 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 119 entries to flush up to offset 19667 2024.10.22 00:57:51.949107 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:57:51.961000 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:57:51.983174 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:57:52.001751 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:57:52.035840 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:57:52.091427 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:52.094256 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:57:52.147829 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_280_280_0 (state Temporary) 2024.10.22 00:57:52.165489 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:57:52.172998 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 27726 2024.10.22 00:57:52.181818 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:57:52.204909 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:57:52.229067 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_280_280_0 to 202410_286_286_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:52.236975 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column GoodEvent 2024.10.22 00:57:52.288985 [ 1222 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_281_79 to 202410_282_282_0 2024.10.22 00:57:52.296878 [ 1222 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.08 MiB using storage policy from min volume index 0 2024.10.22 00:57:52.307647 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 19667 2024.10.22 00:57:52.324203 [ 1222 ] {} DiskLocal: Reserved 1.08 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:57:52.383024 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_282_80} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_281_79 to 202410_282_282_0 into Compact with storage Full 2024.10.22 00:57:52.433140 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5a4145c674beb536b184619cb4fa7ff4) : [0, 122448]: Setting download as failed: File segment: [0, 122448], key: 5a4145c674beb536b184619cb4fa7ff4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:52.437414 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5a4145c674beb536b184619cb4fa7ff4): No space left in cache to reserve 122449 bytes, will continue without cache download 2024.10.22 00:57:52.471430 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_282_80} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:52.480206 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5a4145c674beb536b184619cb4fa7ff4, offset: 0, size: 0 2024.10.22 00:57:52.496154 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:52.499111 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5a4145c674beb536b184619cb4fa7ff4 for removal 2024.10.22 00:57:52.501292 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:57:52.508028 [ 1476 ] {} CacheMetadata: Key 5a4145c674beb536b184619cb4fa7ff4 is removed from metadata 2024.10.22 00:57:52.534839 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_282_80} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_281_79, total 19314 rows starting from the beginning of the part 2024.10.22 00:57:52.529314 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_223_48, 202410_224_224_0] 2024.10.22 00:57:52.547126 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_223_48, 202410_224_224_0] 2024.10.22 00:57:52.594534 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gmh/hkapthxcptwdjnqujadtruvnjqsck (LogSeriesLimiter: on interval from 2024-10-22 00:57:46 to 2024-10-22 00:57:52 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:57:52.699576 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_482_482_0 (state Temporary) 2024.10.22 00:57:52.710114 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_282_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_282_282_0, total 106 rows starting from the beginning of the part 2024.10.22 00:57:52.768507 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_482_482_0 to 202410_495_495_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:52.791446 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_223_48, 202410_224_224_0] 2024.10.22 00:57:53.036297 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 27726 2024.10.22 00:57:53.640604 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 159 bytes for 278b4e0cca38453e8d5bbc11ed29f058:0) 2024.10.22 00:57:53.644605 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6085d5e624f0fe08bc141e70917a5339, offset: 0, size: 153 2024.10.22 00:57:53.651992 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6085d5e624f0fe08bc141e70917a5339 for removal 2024.10.22 00:57:53.655826 [ 1476 ] {} CacheMetadata: Key 6085d5e624f0fe08bc141e70917a5339 is removed from metadata 2024.10.22 00:57:53.868407 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 456364 2024.10.22 00:57:54.613627 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:54.618766 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:57:54.697561 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 2368 bytes for f37b97ea016a396799ba979b017ced80:0) 2024.10.22 00:57:54.701373 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e28f35a5a60d60f84d4843af9da3e99f, offset: 0, size: 5149 2024.10.22 00:57:54.704456 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e28f35a5a60d60f84d4843af9da3e99f for removal 2024.10.22 00:57:54.709258 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 424095cbd7579caf62a6e6f42c2177b4, offset: 0, size: 187 2024.10.22 00:57:54.711570 [ 1476 ] {} CacheMetadata: Key e28f35a5a60d60f84d4843af9da3e99f is removed from metadata 2024.10.22 00:57:54.717365 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 424095cbd7579caf62a6e6f42c2177b4 for removal 2024.10.22 00:57:54.725144 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3948567fdde35c89d3806cb8ec85f373, offset: 0, size: 159 2024.10.22 00:57:54.734537 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3948567fdde35c89d3806cb8ec85f373 for removal 2024.10.22 00:57:54.739213 [ 1476 ] {} CacheMetadata: Key 424095cbd7579caf62a6e6f42c2177b4 is removed from metadata 2024.10.22 00:57:54.740023 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3b4241e1206c46b8fdfe471e9f8a1b43, offset: 0, size: 151 2024.10.22 00:57:54.750591 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3b4241e1206c46b8fdfe471e9f8a1b43 for removal 2024.10.22 00:57:54.765595 [ 1476 ] {} CacheMetadata: Key 3b4241e1206c46b8fdfe471e9f8a1b43 is removed from metadata 2024.10.22 00:57:54.783892 [ 1476 ] {} CacheMetadata: Key 3948567fdde35c89d3806cb8ec85f373 is removed from metadata 2024.10.22 00:57:54.927666 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_282_80} MergeTask::MergeProjectionsStage: Merge sorted 19420 rows, containing 13 columns (13 merged, 0 gathered) in 2.552104206 sec., 7609.407152867644 rows/sec., 3.21 MiB/sec. 2024.10.22 00:57:54.999577 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_270_270_0 (state Temporary) 2024.10.22 00:57:55.001402 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_282_80} MergedBlockOutputStream: filled checksums 202410_1_282_80 (state Temporary) 2024.10.22 00:57:55.088766 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_270_270_0 to 202410_277_277_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:55.099870 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_282_80} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_282_80 to 202410_1_282_80 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:55.104098 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_282_80} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_281_79, 202410_282_282_0] -> 202410_1_282_80 2024.10.22 00:57:55.129540 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 456364 2024.10.22 00:57:55.235348 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.81 MiB. 2024.10.22 00:57:55.327396 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 17.21 MiB, will set to 1.66 GiB (RSS), difference: 14.42 MiB 2024.10.22 00:57:55.687412 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 159 bytes for 1b662818c6031024519db6834c0a0784:0) 2024.10.22 00:57:55.688893 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3e9907b2fef90b75db96ee9268143628, offset: 0, size: 160 2024.10.22 00:57:55.694644 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3e9907b2fef90b75db96ee9268143628 for removal 2024.10.22 00:57:55.697194 [ 1476 ] {} CacheMetadata: Key 3e9907b2fef90b75db96ee9268143628 is removed from metadata 2024.10.22 00:57:56.321468 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 7 file segments (while reserving 34011 bytes for 6850fa6d8112967bd598d04e11dce772:0) 2024.10.22 00:57:56.325571 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6967482682fe837e3a2af65e37c0e4c6, offset: 0, size: 159 2024.10.22 00:57:56.345308 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6967482682fe837e3a2af65e37c0e4c6 for removal 2024.10.22 00:57:56.351012 [ 1476 ] {} CacheMetadata: Key 6967482682fe837e3a2af65e37c0e4c6 is removed from metadata 2024.10.22 00:57:56.352115 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 302759f12116b7caee21e87a9a0b8403, offset: 0, size: 159 2024.10.22 00:57:56.357157 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 302759f12116b7caee21e87a9a0b8403 for removal 2024.10.22 00:57:56.361437 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 246 2024.10.22 00:57:56.363573 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 4035 2024.10.22 00:57:56.363817 [ 1476 ] {} CacheMetadata: Key 302759f12116b7caee21e87a9a0b8403 is removed from metadata 2024.10.22 00:57:56.365739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 914e010b8cd9bc314d910e61e8019738, offset: 0, size: 165 2024.10.22 00:57:56.372386 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 914e010b8cd9bc314d910e61e8019738 for removal 2024.10.22 00:57:56.375356 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f1997ac007710fa4f664440588ff4270, offset: 0, size: 159 2024.10.22 00:57:56.375779 [ 1476 ] {} CacheMetadata: Key 914e010b8cd9bc314d910e61e8019738 is removed from metadata 2024.10.22 00:57:56.377699 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f1997ac007710fa4f664440588ff4270 for removal 2024.10.22 00:57:56.382693 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c054404b7b2afa5f97ebc65cdd2565e0, offset: 0, size: 35337 2024.10.22 00:57:56.384940 [ 1476 ] {} CacheMetadata: Key f1997ac007710fa4f664440588ff4270 is removed from metadata 2024.10.22 00:57:56.386503 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c054404b7b2afa5f97ebc65cdd2565e0 for removal 2024.10.22 00:57:56.389466 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a92b4d54a7dc64d5deea9d30a37bd731, offset: 0, size: 165 2024.10.22 00:57:56.391098 [ 1476 ] {} CacheMetadata: Key c054404b7b2afa5f97ebc65cdd2565e0 is removed from metadata 2024.10.22 00:57:56.391573 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a92b4d54a7dc64d5deea9d30a37bd731 for removal 2024.10.22 00:57:56.397587 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d64854667af7ae74b2bdb00addf395a0, offset: 0, size: 160 2024.10.22 00:57:56.401051 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d64854667af7ae74b2bdb00addf395a0 for removal 2024.10.22 00:57:56.410195 [ 1476 ] {} CacheMetadata: Key d64854667af7ae74b2bdb00addf395a0 is removed from metadata 2024.10.22 00:57:56.423397 [ 1476 ] {} CacheMetadata: Key a92b4d54a7dc64d5deea9d30a37bd731 is removed from metadata 2024.10.22 00:57:56.447351 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 2295 2024.10.22 00:57:56.469704 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:56.473700 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:57:56.495281 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_283_283_0 (state Temporary) 2024.10.22 00:57:56.757282 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:56.762857 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_283_283_0 to 202410_296_296_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:56.767832 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:57:56.896628 [ 1227 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_291_58 to 202410_296_296_0 2024.10.22 00:57:56.903673 [ 1227 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:56.906304 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:57:56.916168 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4035 2024.10.22 00:57:56.924981 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_228_228_0 (state Temporary) 2024.10.22 00:57:56.942502 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_291_58 to 202410_296_296_0 into Compact with storage Full 2024.10.22 00:57:57.007940 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:57.017115 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_291_58, total 4063 rows starting from the beginning of the part 2024.10.22 00:57:57.049731 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 100 entries to flush up to offset 27826 2024.10.22 00:57:57.064918 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_292_292_0, total 39 rows starting from the beginning of the part 2024.10.22 00:57:57.156976 [ 1237 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:57:57.195171 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_293_293_0, total 8 rows starting from the beginning of the part 2024.10.22 00:57:57.215482 [ 1237 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:57:57.240272 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_294_294_0, total 9 rows starting from the beginning of the part 2024.10.22 00:57:57.290615 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_295_295_0, total 15 rows starting from the beginning of the part 2024.10.22 00:57:57.319347 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_296_296_0, total 7 rows starting from the beginning of the part 2024.10.22 00:57:57.326372 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_228_228_0 to 202410_238_238_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:57.387160 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 3 old parts to remove. Parts: [202410_1_391_84, 202410_392_392_0, 202410_393_393_0] 2024.10.22 00:57:57.390559 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 3 parts from filesystem (serially): Parts: [202410_1_391_84, 202410_392_392_0, 202410_393_393_0] 2024.10.22 00:57:57.402064 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 3 parts from memory: Parts: [202410_1_391_84, 202410_392_392_0, 202410_393_393_0] 2024.10.22 00:57:57.476954 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 246 2024.10.22 00:57:57.519178 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:57:57.526657 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:57:57.634566 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_483_483_0 (state Temporary) 2024.10.22 00:57:57.691996 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 20.75 MiB, will set to 1.67 GiB (RSS), difference: -5.40 MiB 2024.10.22 00:57:57.717515 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 7 file segments (while reserving 31183 bytes for 7e29992ac45c09760da3dd4612dd8b12:0) 2024.10.22 00:57:57.724808 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6850fa6d8112967bd598d04e11dce772, offset: 0, size: 34011 2024.10.22 00:57:57.740309 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6850fa6d8112967bd598d04e11dce772 for removal 2024.10.22 00:57:57.746201 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f37b97ea016a396799ba979b017ced80, offset: 0, size: 2368 2024.10.22 00:57:57.745563 [ 1476 ] {} CacheMetadata: Key 6850fa6d8112967bd598d04e11dce772 is removed from metadata 2024.10.22 00:57:57.753764 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f37b97ea016a396799ba979b017ced80 for removal 2024.10.22 00:57:57.771973 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0e4a7815f193759e45d5d0f91d7c927d, offset: 0, size: 3252 2024.10.22 00:57:57.878061 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0e4a7815f193759e45d5d0f91d7c927d for removal 2024.10.22 00:57:57.880468 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d219aadfcd7df487962d889db2e8038c, offset: 0, size: 159 2024.10.22 00:57:57.882936 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d219aadfcd7df487962d889db2e8038c for removal 2024.10.22 00:57:57.884091 [ 1476 ] {} CacheMetadata: Key 0e4a7815f193759e45d5d0f91d7c927d is removed from metadata 2024.10.22 00:57:57.885003 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1b662818c6031024519db6834c0a0784, offset: 0, size: 159 2024.10.22 00:57:57.889534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1b662818c6031024519db6834c0a0784 for removal 2024.10.22 00:57:57.892717 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 278b4e0cca38453e8d5bbc11ed29f058, offset: 0, size: 159 2024.10.22 00:57:57.894130 [ 1476 ] {} CacheMetadata: Key 1b662818c6031024519db6834c0a0784 is removed from metadata 2024.10.22 00:57:57.895892 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 278b4e0cca38453e8d5bbc11ed29f058 for removal 2024.10.22 00:57:57.898318 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 56ed6bb85256f49bf7247d053858e088, offset: 0, size: 165 2024.10.22 00:57:57.901805 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 56ed6bb85256f49bf7247d053858e088 for removal 2024.10.22 00:57:57.935601 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_483_483_0 to 202410_496_496_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:57.973794 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_491_130 to 202410_496_496_0 2024.10.22 00:57:57.980799 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.01 MiB using storage policy from min volume index 0 2024.10.22 00:57:57.982180 [ 1268 ] {} DiskLocal: Reserved 1.01 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:57:57.983860 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 27826 2024.10.22 00:57:58.006600 [ 1476 ] {} CacheMetadata: Key 56ed6bb85256f49bf7247d053858e088 is removed from metadata 2024.10.22 00:57:58.007366 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_491_130 to 202410_496_496_0 into Compact with storage Full 2024.10.22 00:57:58.023692 [ 1476 ] {} CacheMetadata: Key 278b4e0cca38453e8d5bbc11ed29f058 is removed from metadata 2024.10.22 00:57:58.058279 [ 1476 ] {} CacheMetadata: Key d219aadfcd7df487962d889db2e8038c is removed from metadata 2024.10.22 00:57:58.069099 [ 1476 ] {} CacheMetadata: Key f37b97ea016a396799ba979b017ced80 is removed from metadata 2024.10.22 00:57:58.076094 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:57:58.090731 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_491_130, total 28249 rows starting from the beginning of the part 2024.10.22 00:57:58.182095 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_492_492_0, total 59 rows starting from the beginning of the part 2024.10.22 00:57:58.247089 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_493_493_0, total 84 rows starting from the beginning of the part 2024.10.22 00:57:58.329571 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_494_494_0, total 96 rows starting from the beginning of the part 2024.10.22 00:57:58.425450 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_495_495_0, total 69 rows starting from the beginning of the part 2024.10.22 00:57:58.500333 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_496_496_0, total 100 rows starting from the beginning of the part 2024.10.22 00:57:58.832725 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} MergeTask::MergeProjectionsStage: Merge sorted 4141 rows, containing 12 columns (12 merged, 0 gathered) in 1.904535327 sec., 2174.283638268265 rows/sec., 340.56 KiB/sec. 2024.10.22 00:57:58.895476 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} MergedBlockOutputStream: filled checksums 202410_1_296_59 (state Temporary) 2024.10.22 00:57:59.083127 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(666da40dbbf719da9f4e0f3252e16502) : [0, 210112]: Setting download as failed: File segment: [0, 210112], key: 666da40dbbf719da9f4e0f3252e16502, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:57:59.084596 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(666da40dbbf719da9f4e0f3252e16502): No space left in cache to reserve 210113 bytes, will continue without cache download 2024.10.22 00:57:59.099271 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 666da40dbbf719da9f4e0f3252e16502, offset: 0, size: 0 2024.10.22 00:57:59.101236 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 666da40dbbf719da9f4e0f3252e16502 for removal 2024.10.22 00:57:59.104990 [ 1476 ] {} CacheMetadata: Key 666da40dbbf719da9f4e0f3252e16502 is removed from metadata 2024.10.22 00:57:59.408925 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 13.99 MiB, will set to 1.67 GiB (RSS), difference: -5.22 MiB 2024.10.22 00:57:59.538273 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_296_59 to 202410_1_296_59 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:57:59.541188 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_296_59} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_291_58, 202410_296_296_0] -> 202410_1_296_59 2024.10.22 00:57:59.581159 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 00:57:59.783029 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_276_55/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:57:59.787671 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:57:59.788993 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:57:59.832709 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 118 entries to flush up to offset 19785 2024.10.22 00:58:00.086937 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(79d8ddb117c0bd4e21fd3ffc6330d71e) : [0, 131923]: Setting download as failed: File segment: [0, 131923], key: 79d8ddb117c0bd4e21fd3ffc6330d71e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:00.090229 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(79d8ddb117c0bd4e21fd3ffc6330d71e): No space left in cache to reserve 131924 bytes, will continue without cache download 2024.10.22 00:58:00.204343 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 79d8ddb117c0bd4e21fd3ffc6330d71e, offset: 0, size: 0 2024.10.22 00:58:00.207326 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 79d8ddb117c0bd4e21fd3ffc6330d71e for removal 2024.10.22 00:58:00.212191 [ 1476 ] {} CacheMetadata: Key 79d8ddb117c0bd4e21fd3ffc6330d71e is removed from metadata 2024.10.22 00:58:00.373561 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:00.376391 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:00.412623 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_281_281_0 (state Temporary) 2024.10.22 00:58:00.570475 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_281_281_0 to 202410_287_287_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:00.587160 [ 1159 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_282_80 to 202410_283_283_0 2024.10.22 00:58:00.592981 [ 1159 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.08 MiB using storage policy from min volume index 0 2024.10.22 00:58:00.618618 [ 1159 ] {} DiskLocal: Reserved 1.08 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:00.634654 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_283_81} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_282_80 to 202410_283_283_0 into Compact with storage Full 2024.10.22 00:58:00.648543 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 19785 2024.10.22 00:58:00.700562 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_224_49, 202410_225_225_0] 2024.10.22 00:58:00.790941 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} MergeTask::MergeProjectionsStage: Merge sorted 28657 rows, containing 13 columns (13 merged, 0 gathered) in 2.804122736 sec., 10219.59546638047 rows/sec., 1.55 MiB/sec. 2024.10.22 00:58:00.795520 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_283_81} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:00.802395 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_224_49, 202410_225_225_0] 2024.10.22 00:58:00.803996 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_283_81} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_282_80, total 19420 rows starting from the beginning of the part 2024.10.22 00:58:00.813991 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_224_49, 202410_225_225_0] 2024.10.22 00:58:00.846663 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_283_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_283_283_0, total 99 rows starting from the beginning of the part 2024.10.22 00:58:00.967823 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} MergedBlockOutputStream: filled checksums 202410_1_496_131 (state Temporary) 2024.10.22 00:58:01.101823 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_496_131 to 202410_1_496_131 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:01.210271 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_496_131} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_491_130, 202410_496_496_0] -> 202410_1_496_131 2024.10.22 00:58:01.276520 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.05 MiB. 2024.10.22 00:58:01.407392 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 15.00 MiB, will set to 1.67 GiB (RSS), difference: -1.38 MiB 2024.10.22 00:58:01.441273 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(306f362c9d6a67ec47e56c76384c12d1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 306f362c9d6a67ec47e56c76384c12d1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 00:58:01.448363 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 306f362c9d6a67ec47e56c76384c12d1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 00:58:01.453003 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:58:01.744933 [ 1170 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 00:58:01.791675 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:02.001513 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 98 entries to flush up to offset 27924 2024.10.22 00:58:02.104499 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6a4c321625218f758b5431ccd4598e33) : [0, 81741]: Setting download as failed: File segment: [0, 81741], key: 6a4c321625218f758b5431ccd4598e33, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:02.113031 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6a4c321625218f758b5431ccd4598e33): No space left in cache to reserve 81742 bytes, will continue without cache download 2024.10.22 00:58:02.135193 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 457900 2024.10.22 00:58:02.264842 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6a4c321625218f758b5431ccd4598e33, offset: 0, size: 0 2024.10.22 00:58:02.267824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6a4c321625218f758b5431ccd4598e33 for removal 2024.10.22 00:58:02.278256 [ 1476 ] {} CacheMetadata: Key 6a4c321625218f758b5431ccd4598e33 is removed from metadata 2024.10.22 00:58:02.312796 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:02.323772 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:02.425456 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_484_484_0 (state Temporary) 2024.10.22 00:58:02.529241 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_484_484_0 to 202410_497_497_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:02.620989 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 27924 2024.10.22 00:58:02.767225 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_283_81} MergeTask::MergeProjectionsStage: Merge sorted 19519 rows, containing 13 columns (13 merged, 0 gathered) in 2.140190375 sec., 9120.216700348445 rows/sec., 3.85 MiB/sec. 2024.10.22 00:58:03.033229 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:03.044026 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:03.269765 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_283_81} MergedBlockOutputStream: filled checksums 202410_1_283_81 (state Temporary) 2024.10.22 00:58:03.283800 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_54_11, total 2110 rows starting from the beginning of the part 2024.10.22 00:58:03.316622 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_271_271_0 (state Temporary) 2024.10.22 00:58:03.379738 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(010e9ff8d7a3e4d798b4f66378c5451c) : [0, 87374]: Setting download as failed: File segment: [0, 87374], key: 010e9ff8d7a3e4d798b4f66378c5451c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:03.380766 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(010e9ff8d7a3e4d798b4f66378c5451c): No space left in cache to reserve 87375 bytes, will continue without cache download 2024.10.22 00:58:03.539782 [ 1685 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xhw/headhyumksywuxsilagctcoopnpem, size 1153322 2024.10.22 00:58:03.495389 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 010e9ff8d7a3e4d798b4f66378c5451c, offset: 0, size: 0 2024.10.22 00:58:03.624700 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 010e9ff8d7a3e4d798b4f66378c5451c for removal 2024.10.22 00:58:03.631495 [ 1476 ] {} CacheMetadata: Key 010e9ff8d7a3e4d798b4f66378c5451c is removed from metadata 2024.10.22 00:58:03.691905 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_271_271_0 to 202410_278_278_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:03.728207 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 457900 2024.10.22 00:58:03.933650 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_283_81} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_283_81 to 202410_1_283_81 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:03.951147 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_283_81} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_282_80, 202410_283_283_0] -> 202410_1_283_81 2024.10.22 00:58:03.981610 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.81 MiB. 2024.10.22 00:58:04.031355 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Robotness 2024.10.22 00:58:04.041605 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Robotness 2024.10.22 00:58:04.050009 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Robotness 2024.10.22 00:58:04.075808 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Robotness 2024.10.22 00:58:04.089813 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Robotness 2024.10.22 00:58:04.102471 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Robotness 2024.10.22 00:58:04.123196 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Robotness 2024.10.22 00:58:04.178452 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xtl/uipmfpvjubuhbaiuppksgzotzyojc (LogSeriesLimiter: on interval from 2024-10-22 00:57:52 to 2024-10-22 00:58:04 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:58:04.426314 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 10.19 MiB, will set to 1.66 GiB (RSS), difference: 8.89 MiB 2024.10.22 00:58:04.427709 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 4058 2024.10.22 00:58:04.533447 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:04.539124 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:04.639941 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7b9819912e5b306d2317daca6dc6b740) : [0, 164222]: Setting download as failed: File segment: [0, 164222], key: 7b9819912e5b306d2317daca6dc6b740, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:04.641838 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7b9819912e5b306d2317daca6dc6b740): No space left in cache to reserve 164223 bytes, will continue without cache download 2024.10.22 00:58:04.721838 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_284_284_0 (state Temporary) 2024.10.22 00:58:04.768110 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_284_284_0 to 202410_297_297_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:04.774777 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} MergeTask::MergeProjectionsStage: Merge sorted 468802 rows, containing 4 columns (4 merged, 0 gathered) in 17.774695353 sec., 26374.68551160715 rows/sec., 386.66 KiB/sec. 2024.10.22 00:58:04.776687 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7b9819912e5b306d2317daca6dc6b740, offset: 0, size: 0 2024.10.22 00:58:04.778855 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7b9819912e5b306d2317daca6dc6b740 for removal 2024.10.22 00:58:04.784190 [ 1476 ] {} CacheMetadata: Key 7b9819912e5b306d2317daca6dc6b740 is removed from metadata 2024.10.22 00:58:04.837136 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4058 2024.10.22 00:58:04.856046 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} MergedBlockOutputStream: filled checksums 202410_1_276_55 (state Temporary) 2024.10.22 00:58:04.900515 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_276_55 to 202410_1_276_55 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:04.903620 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_276_55} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_271_54, 202410_276_276_0] -> 202410_1_276_55 2024.10.22 00:58:04.948592 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:58:04.982444 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 247 2024.10.22 00:58:05.283563 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:05.287467 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:05.491104 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_229_229_0 (state Temporary) 2024.10.22 00:58:05.686790 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_229_229_0 to 202410_239_239_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:05.725314 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_234_48 to 202410_239_239_0 2024.10.22 00:58:05.742707 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:05.757782 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a03799aedc1ffe4401e3010a8bb434f8) : [0, 176244]: Setting download as failed: File segment: [0, 176244], key: a03799aedc1ffe4401e3010a8bb434f8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:05.771768 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a03799aedc1ffe4401e3010a8bb434f8): No space left in cache to reserve 176245 bytes, will continue without cache download 2024.10.22 00:58:05.776261 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 247 2024.10.22 00:58:05.747614 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:05.793142 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a03799aedc1ffe4401e3010a8bb434f8, offset: 0, size: 0 2024.10.22 00:58:05.793469 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_234_48 to 202410_239_239_0 into Compact with storage Full 2024.10.22 00:58:05.805776 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a03799aedc1ffe4401e3010a8bb434f8 for removal 2024.10.22 00:58:05.810994 [ 1476 ] {} CacheMetadata: Key a03799aedc1ffe4401e3010a8bb434f8 is removed from metadata 2024.10.22 00:58:06.042435 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:06.083329 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_234_48, total 278 rows starting from the beginning of the part 2024.10.22 00:58:06.182515 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_235_235_0, total 1 rows starting from the beginning of the part 2024.10.22 00:58:06.299831 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_236_236_0, total 1 rows starting from the beginning of the part 2024.10.22 00:58:06.399881 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 5 old parts to remove. Parts: [202410_1_393_85, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0, 202410_397_397_0] 2024.10.22 00:58:06.406047 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from filesystem (serially): Parts: [202410_1_393_85, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0, 202410_397_397_0] 2024.10.22 00:58:06.433269 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from memory: Parts: [202410_1_393_85, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0, 202410_397_397_0] 2024.10.22 00:58:06.529439 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_237_237_0, total 1 rows starting from the beginning of the part 2024.10.22 00:58:06.591133 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 8.14 MiB, will set to 1.66 GiB (RSS), difference: 2.32 MiB 2024.10.22 00:58:06.622913 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_238_238_0, total 1 rows starting from the beginning of the part 2024.10.22 00:58:06.736838 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 28002 2024.10.22 00:58:06.765063 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(718bfbb2f6a3bfe1f9e9d2c5a66a8bc9) : [0, 92924]: Setting download as failed: File segment: [0, 92924], key: 718bfbb2f6a3bfe1f9e9d2c5a66a8bc9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:06.768420 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(718bfbb2f6a3bfe1f9e9d2c5a66a8bc9): No space left in cache to reserve 92925 bytes, will continue without cache download 2024.10.22 00:58:06.902915 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 718bfbb2f6a3bfe1f9e9d2c5a66a8bc9, offset: 0, size: 0 2024.10.22 00:58:06.905527 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 718bfbb2f6a3bfe1f9e9d2c5a66a8bc9 for removal 2024.10.22 00:58:06.907920 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_239_239_0, total 1 rows starting from the beginning of the part 2024.10.22 00:58:06.909179 [ 1476 ] {} CacheMetadata: Key 718bfbb2f6a3bfe1f9e9d2c5a66a8bc9 is removed from metadata 2024.10.22 00:58:06.970637 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_55_55_0, total 43 rows starting from the beginning of the part 2024.10.22 00:58:07.080513 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:07.084625 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:07.299479 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_485_485_0 (state Temporary) 2024.10.22 00:58:07.364055 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_485_485_0 to 202410_498_498_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:07.503808 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28002 2024.10.22 00:58:07.842605 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(300b44997e5aa59e25b79c8ea1a00886) : [0, 1122315]: Setting download as failed: File segment: [0, 1122315], key: 300b44997e5aa59e25b79c8ea1a00886, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:07.844719 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(300b44997e5aa59e25b79c8ea1a00886): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:58:07.878443 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 300b44997e5aa59e25b79c8ea1a00886, offset: 0, size: 0 2024.10.22 00:58:07.879893 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 300b44997e5aa59e25b79c8ea1a00886 for removal 2024.10.22 00:58:07.882079 [ 1476 ] {} CacheMetadata: Key 300b44997e5aa59e25b79c8ea1a00886 is removed from metadata 2024.10.22 00:58:08.168400 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 19875 2024.10.22 00:58:08.304139 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:08.408180 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:08.460100 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_282_282_0 (state Temporary) 2024.10.22 00:58:08.608101 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_282_282_0 to 202410_288_288_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:08.656603 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_283_81 to 202410_284_284_0 2024.10.22 00:58:08.677205 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.09 MiB using storage policy from min volume index 0 2024.10.22 00:58:08.678212 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 19875 2024.10.22 00:58:08.680392 [ 1200 ] {} DiskLocal: Reserved 1.09 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:08.693705 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 6.40 MiB, will set to 1.66 GiB (RSS), difference: -6.95 MiB 2024.10.22 00:58:08.841389 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_284_82} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_283_81 to 202410_284_284_0 into Compact with storage Full 2024.10.22 00:58:08.892200 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_284_82} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:08.900488 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_284_82} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_283_81, total 19519 rows starting from the beginning of the part 2024.10.22 00:58:08.923867 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_284_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_284_284_0, total 66 rows starting from the beginning of the part 2024.10.22 00:58:08.946292 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_239_49/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:58:09.035902 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7d738a6ce39d6e01a6ad06916f209f1a) : [0, 55430]: Setting download as failed: File segment: [0, 55430], key: 7d738a6ce39d6e01a6ad06916f209f1a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:09.038000 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7d738a6ce39d6e01a6ad06916f209f1a): No space left in cache to reserve 55431 bytes, will continue without cache download 2024.10.22 00:58:09.272393 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7d738a6ce39d6e01a6ad06916f209f1a, offset: 0, size: 0 2024.10.22 00:58:09.274780 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7d738a6ce39d6e01a6ad06916f209f1a for removal 2024.10.22 00:58:09.276905 [ 1476 ] {} CacheMetadata: Key 7d738a6ce39d6e01a6ad06916f209f1a is removed from metadata 2024.10.22 00:58:09.788478 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:58:09.801777 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:58:09.888058 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_56_56_0, total 40 rows starting from the beginning of the part 2024.10.22 00:58:09.970685 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} MergeTask::MergeProjectionsStage: Merge sorted 283 rows, containing 32 columns (32 merged, 0 gathered) in 4.182738731 sec., 67.65901917386579 rows/sec., 7.02 KiB/sec. 2024.10.22 00:58:10.245146 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} MergedBlockOutputStream: filled checksums 202410_1_239_49 (state Temporary) 2024.10.22 00:58:10.291498 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_216_43, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0, 202410_220_220_0, 202410_221_221_0] 2024.10.22 00:58:10.293670 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_216_43, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0, 202410_220_220_0, 202410_221_221_0] 2024.10.22 00:58:10.334271 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_216_43, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0, 202410_220_220_0, 202410_221_221_0] 2024.10.22 00:58:10.392117 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 29886 bytes for 0ea9232faefcdd705877811b32e2dcd9:0) 2024.10.22 00:58:10.395473 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7e29992ac45c09760da3dd4612dd8b12, offset: 0, size: 31183 2024.10.22 00:58:10.405071 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7e29992ac45c09760da3dd4612dd8b12 for removal 2024.10.22 00:58:10.417676 [ 1476 ] {} CacheMetadata: Key 7e29992ac45c09760da3dd4612dd8b12 is removed from metadata 2024.10.22 00:58:10.419494 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 62caedbc5e3a2b48701359b967e6bb5d, offset: 0, size: 159 2024.10.22 00:58:10.427106 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 62caedbc5e3a2b48701359b967e6bb5d for removal 2024.10.22 00:58:10.468988 [ 1476 ] {} CacheMetadata: Key 62caedbc5e3a2b48701359b967e6bb5d is removed from metadata 2024.10.22 00:58:10.552411 [ 1713 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gmh/hkapthxcptwdjnqujadtruvnjqsck, size 29886 2024.10.22 00:58:10.697584 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_239_49 to 202410_1_239_49 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:10.803556 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_239_49} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_234_48, 202410_239_239_0] -> 202410_1_239_49 2024.10.22 00:58:10.836283 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1316 entries to flush up to offset 459216 2024.10.22 00:58:11.053466 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:58:11.074766 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column EventTime 2024.10.22 00:58:11.080687 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column EventTime 2024.10.22 00:58:11.125366 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column EventTime 2024.10.22 00:58:11.148796 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column EventTime 2024.10.22 00:58:11.176766 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column EventTime 2024.10.22 00:58:11.204530 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column EventTime 2024.10.22 00:58:11.213816 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_284_82} MergeTask::MergeProjectionsStage: Merge sorted 19585 rows, containing 13 columns (13 merged, 0 gathered) in 2.488566847 sec., 7869.991526894274 rows/sec., 3.32 MiB/sec. 2024.10.22 00:58:11.237604 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column EventTime 2024.10.22 00:58:11.254463 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_225_50, 202410_226_226_0] 2024.10.22 00:58:11.264470 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_225_50, 202410_226_226_0] 2024.10.22 00:58:11.277702 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_225_50, 202410_226_226_0] 2024.10.22 00:58:11.278870 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column EventTime 2024.10.22 00:58:11.290443 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_284_82/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:58:11.310137 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column EventTime 2024.10.22 00:58:11.338548 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(47f85e072e76862c9d8a243d784ac29f) : [0, 48173]: Setting download as failed: File segment: [0, 48173], key: 47f85e072e76862c9d8a243d784ac29f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:11.340889 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(47f85e072e76862c9d8a243d784ac29f): No space left in cache to reserve 48174 bytes, will continue without cache download 2024.10.22 00:58:11.495995 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_284_82} MergedBlockOutputStream: filled checksums 202410_1_284_82 (state Temporary) 2024.10.22 00:58:11.526882 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 28078 2024.10.22 00:58:11.567602 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 47f85e072e76862c9d8a243d784ac29f, offset: 0, size: 0 2024.10.22 00:58:11.571164 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 47f85e072e76862c9d8a243d784ac29f for removal 2024.10.22 00:58:11.576343 [ 1476 ] {} CacheMetadata: Key 47f85e072e76862c9d8a243d784ac29f is removed from metadata 2024.10.22 00:58:11.594667 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/otn/whrdzloqnvzorfrmtknrdviajrqrn (LogSeriesLimiter: on interval from 2024-10-22 00:58:04 to 2024-10-22 00:58:11 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:58:11.707445 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:11.716568 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:11.724857 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_284_82} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_284_82 to 202410_1_284_82 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:11.729442 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_284_82} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_283_81, 202410_284_284_0] -> 202410_1_284_82 2024.10.22 00:58:11.799906 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:11.802878 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:11.878161 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.78 MiB. 2024.10.22 00:58:11.881354 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_272_272_0 (state Temporary) 2024.10.22 00:58:11.982095 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_486_486_0 (state Temporary) 2024.10.22 00:58:12.000833 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_272_272_0 to 202410_279_279_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:12.024777 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_57_57_0, total 39 rows starting from the beginning of the part 2024.10.22 00:58:12.044599 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 459216 2024.10.22 00:58:12.136116 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_486_486_0 to 202410_499_499_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:12.223023 [ 1239 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:58:12.277697 [ 1239 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:58:12.285283 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28078 2024.10.22 00:58:12.339815 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 4081 2024.10.22 00:58:12.431778 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 4.96 MiB, will set to 1.66 GiB (RSS), difference: -7.48 MiB 2024.10.22 00:58:12.481269 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b0abbf0e0edca485d494310b794f4397) : [0, 89719]: Setting download as failed: File segment: [0, 89719], key: b0abbf0e0edca485d494310b794f4397, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:12.483245 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b0abbf0e0edca485d494310b794f4397): No space left in cache to reserve 89720 bytes, will continue without cache download 2024.10.22 00:58:12.565408 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:12.567472 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:12.578471 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:12.579911 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:12.601549 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b0abbf0e0edca485d494310b794f4397, offset: 0, size: 0 2024.10.22 00:58:12.603565 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b0abbf0e0edca485d494310b794f4397 for removal 2024.10.22 00:58:12.607458 [ 1476 ] {} CacheMetadata: Key b0abbf0e0edca485d494310b794f4397 is removed from metadata 2024.10.22 00:58:12.612393 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_285_285_0 (state Temporary) 2024.10.22 00:58:12.755126 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.186934351 sec. 2024.10.22 00:58:12.773235 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_285_285_0 to 202410_298_298_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:12.905453 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4081 2024.10.22 00:58:13.279328 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 248 2024.10.22 00:58:13.303763 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_58_58_0, total 38 rows starting from the beginning of the part 2024.10.22 00:58:13.699052 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:13.701838 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:13.765362 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_230_230_0 (state Temporary) 2024.10.22 00:58:13.789687 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ef247de4bc8d8a6065832b81ab0ebea2) : [0, 135375]: Setting download as failed: File segment: [0, 135375], key: ef247de4bc8d8a6065832b81ab0ebea2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:13.791893 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ef247de4bc8d8a6065832b81ab0ebea2): No space left in cache to reserve 135376 bytes, will continue without cache download 2024.10.22 00:58:13.909503 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ef247de4bc8d8a6065832b81ab0ebea2, offset: 0, size: 0 2024.10.22 00:58:13.919385 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ef247de4bc8d8a6065832b81ab0ebea2 for removal 2024.10.22 00:58:13.926055 [ 1476 ] {} CacheMetadata: Key ef247de4bc8d8a6065832b81ab0ebea2 is removed from metadata 2024.10.22 00:58:13.981696 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_230_230_0 to 202410_240_240_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:14.149023 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 248 2024.10.22 00:58:14.418740 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 3.36 MiB, will set to 1.67 GiB (RSS), difference: -2.74 MiB 2024.10.22 00:58:14.842033 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_59_59_0, total 45 rows starting from the beginning of the part 2024.10.22 00:58:14.980388 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3acc9c290927bae6f84fada971f7e2ee) : [0, 86350]: Setting download as failed: File segment: [0, 86350], key: 3acc9c290927bae6f84fada971f7e2ee, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:14.982282 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3acc9c290927bae6f84fada971f7e2ee): No space left in cache to reserve 86351 bytes, will continue without cache download 2024.10.22 00:58:14.993173 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3acc9c290927bae6f84fada971f7e2ee, offset: 0, size: 0 2024.10.22 00:58:14.995579 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3acc9c290927bae6f84fada971f7e2ee for removal 2024.10.22 00:58:14.997809 [ 1476 ] {} CacheMetadata: Key 3acc9c290927bae6f84fada971f7e2ee is removed from metadata 2024.10.22 00:58:15.555428 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_57_57_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:58:15.726525 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_57_57_0 (state Temporary) 2024.10.22 00:58:16.123533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9fa2fc76d1759eb1be288ede4e61a496) : [0, 291844]: Setting download as failed: File segment: [0, 291844], key: 9fa2fc76d1759eb1be288ede4e61a496, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:16.125379 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9fa2fc76d1759eb1be288ede4e61a496): No space left in cache to reserve 291845 bytes, will continue without cache download 2024.10.22 00:58:16.144020 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9fa2fc76d1759eb1be288ede4e61a496, offset: 0, size: 0 2024.10.22 00:58:16.147392 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9fa2fc76d1759eb1be288ede4e61a496 for removal 2024.10.22 00:58:16.151602 [ 1476 ] {} CacheMetadata: Key 9fa2fc76d1759eb1be288ede4e61a496 is removed from metadata 2024.10.22 00:58:16.187318 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 19965 2024.10.22 00:58:16.296374 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 28141 2024.10.22 00:58:16.300815 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:16.311891 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:16.374782 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_283_283_0 (state Temporary) 2024.10.22 00:58:16.410487 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_283_283_0 to 202410_289_289_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:16.447658 [ 1246 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_284_82 to 202410_285_285_0 2024.10.22 00:58:16.455519 [ 1246 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.09 MiB using storage policy from min volume index 0 2024.10.22 00:58:16.458345 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 19965 2024.10.22 00:58:16.459933 [ 1246 ] {} DiskLocal: Reserved 1.09 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:16.503578 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_285_83} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_284_82 to 202410_285_285_0 into Compact with storage Full 2024.10.22 00:58:16.535632 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_285_83} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:16.543321 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_285_83} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_284_82, total 19585 rows starting from the beginning of the part 2024.10.22 00:58:16.575911 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_285_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_285_285_0, total 67 rows starting from the beginning of the part 2024.10.22 00:58:16.583116 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:16.588533 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:16.659523 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_487_487_0 (state Temporary) 2024.10.22 00:58:16.711970 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 4.46 MiB, will set to 1.67 GiB (RSS), difference: -13.20 MiB 2024.10.22 00:58:16.875130 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(f1f5f41618307d2fcb4ec8c635c7db0e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f1f5f41618307d2fcb4ec8c635c7db0e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 00:58:16.877048 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: f1f5f41618307d2fcb4ec8c635c7db0e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 00:58:16.881175 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:58:16.899391 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_487_487_0 to 202410_500_500_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:16.956643 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28141 2024.10.22 00:58:17.859476 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(20ed724d02a324328e7ab73149b9d61d) : [0, 990807]: Setting download as failed: File segment: [0, 990807], key: 20ed724d02a324328e7ab73149b9d61d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:17.863775 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(20ed724d02a324328e7ab73149b9d61d): No space left in cache to reserve 990808 bytes, will continue without cache download 2024.10.22 00:58:17.882141 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 20ed724d02a324328e7ab73149b9d61d, offset: 0, size: 0 2024.10.22 00:58:17.883206 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 20ed724d02a324328e7ab73149b9d61d for removal 2024.10.22 00:58:17.884833 [ 1476 ] {} CacheMetadata: Key 20ed724d02a324328e7ab73149b9d61d is removed from metadata 2024.10.22 00:58:18.392503 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_285_83} MergeTask::MergeProjectionsStage: Merge sorted 19652 rows, containing 13 columns (13 merged, 0 gathered) in 1.898480303 sec., 10351.437393870081 rows/sec., 4.37 MiB/sec. 2024.10.22 00:58:18.476308 [ 1254 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 00:58:18.479717 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_285_83} MergedBlockOutputStream: filled checksums 202410_1_285_83 (state Temporary) 2024.10.22 00:58:18.513659 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_226_51, 202410_227_227_0] 2024.10.22 00:58:18.621515 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_226_51, 202410_227_227_0] 2024.10.22 00:58:18.635320 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_226_51, 202410_227_227_0] 2024.10.22 00:58:18.753354 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_285_83} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_285_83 to 202410_1_285_83 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:18.756269 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_285_83} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_284_82, 202410_285_285_0] -> 202410_1_285_83 2024.10.22 00:58:18.887116 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.78 MiB. 2024.10.22 00:58:19.055386 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1372 entries to flush up to offset 460588 2024.10.22 00:58:19.499483 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2fe8bb99e1beafcddc1783c558bde391) : [0, 110275]: Setting download as failed: File segment: [0, 110275], key: 2fe8bb99e1beafcddc1783c558bde391, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:19.500657 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2fe8bb99e1beafcddc1783c558bde391): No space left in cache to reserve 110276 bytes, will continue without cache download 2024.10.22 00:58:19.503840 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 3.55 MiB, will set to 1.67 GiB (RSS), difference: 9.93 MiB 2024.10.22 00:58:19.532402 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2fe8bb99e1beafcddc1783c558bde391, offset: 0, size: 0 2024.10.22 00:58:19.556536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2fe8bb99e1beafcddc1783c558bde391 for removal 2024.10.22 00:58:19.559064 [ 1476 ] {} CacheMetadata: Key 2fe8bb99e1beafcddc1783c558bde391 is removed from metadata 2024.10.22 00:58:19.789701 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:58:19.793429 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:58:20.193065 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:20.198870 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:20.408634 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 4088 2024.10.22 00:58:20.597264 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_273_273_0 (state Temporary) 2024.10.22 00:58:20.665414 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(aba539cefa692bca6942df225bc58847) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: aba539cefa692bca6942df225bc58847, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 00:58:20.669678 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 563367, file segment info: File segment: [0, 33554431], key: aba539cefa692bca6942df225bc58847, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 00:58:20.672270 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:58:20.739182 [ 1679 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xtl/uipmfpvjubuhbaiuppksgzotzyojc, size 563367 2024.10.22 00:58:20.850965 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:20.856061 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:20.887255 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_286_286_0 (state Temporary) 2024.10.22 00:58:20.958394 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_286_286_0 to 202410_299_299_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:20.972934 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 28198 2024.10.22 00:58:20.981097 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_273_273_0 to 202410_280_280_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:20.998738 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4088 2024.10.22 00:58:21.027833 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 460588 2024.10.22 00:58:21.090098 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 00:58:21.221395 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 00:58:21.255072 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 00:58:21.303888 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:21.307856 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:21.376128 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_488_488_0 (state Temporary) 2024.10.22 00:58:21.380219 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 00:58:21.407633 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 00:58:21.411331 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 2.95 MiB, will set to 1.67 GiB (RSS), difference: -9.20 MiB 2024.10.22 00:58:21.458838 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(46e5a5209175ec97bc03dd91598278d7) : [0, 1109143]: Setting download as failed: File segment: [0, 1109143], key: 46e5a5209175ec97bc03dd91598278d7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:21.465323 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(46e5a5209175ec97bc03dd91598278d7): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:58:21.557697 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 00:58:21.581106 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 00:58:21.595333 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 46e5a5209175ec97bc03dd91598278d7, offset: 0, size: 0 2024.10.22 00:58:21.596243 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 46e5a5209175ec97bc03dd91598278d7 for removal 2024.10.22 00:58:21.598382 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_488_488_0 to 202410_501_501_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:21.599363 [ 1476 ] {} CacheMetadata: Key 46e5a5209175ec97bc03dd91598278d7 is removed from metadata 2024.10.22 00:58:21.625271 [ 1178 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_496_131 to 202410_497_497_0 2024.10.22 00:58:21.635882 [ 1178 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:21.638258 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28198 2024.10.22 00:58:21.640921 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:21.653019 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 249 2024.10.22 00:58:21.733087 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_497_132} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_496_131 to 202410_497_497_0 into Compact with storage Full 2024.10.22 00:58:21.790994 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_497_132} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:21.802093 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_497_132} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_496_131, total 28657 rows starting from the beginning of the part 2024.10.22 00:58:21.721712 [ 1159 ] {} DatabaseCatalog: Execution took 415 ms. 2024.10.22 00:58:21.856473 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_497_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_497_497_0, total 98 rows starting from the beginning of the part 2024.10.22 00:58:21.989089 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mjj/osfczchqiomyjtrkihuwaaugjkujc (LogSeriesLimiter: on interval from 2024-10-22 00:58:11 to 2024-10-22 00:58:21 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:58:22.006501 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:22.014673 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:22.090856 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_231_231_0 (state Temporary) 2024.10.22 00:58:22.182364 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_231_231_0 to 202410_241_241_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:22.487164 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 249 2024.10.22 00:58:22.788568 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e506d1a9f938eb2a8be34130c70fa8a6) : [0, 83053]: Setting download as failed: File segment: [0, 83053], key: e506d1a9f938eb2a8be34130c70fa8a6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:22.791025 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e506d1a9f938eb2a8be34130c70fa8a6): No space left in cache to reserve 83054 bytes, will continue without cache download 2024.10.22 00:58:22.903426 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e506d1a9f938eb2a8be34130c70fa8a6, offset: 0, size: 0 2024.10.22 00:58:22.905158 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e506d1a9f938eb2a8be34130c70fa8a6 for removal 2024.10.22 00:58:22.909817 [ 1476 ] {} CacheMetadata: Key e506d1a9f938eb2a8be34130c70fa8a6 is removed from metadata 2024.10.22 00:58:23.482356 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 2.75 MiB, will set to 1.68 GiB (RSS), difference: -15.94 MiB 2024.10.22 00:58:23.964232 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2c1222a05feb669c32ade4927b093f44) : [0, 69590]: Setting download as failed: File segment: [0, 69590], key: 2c1222a05feb669c32ade4927b093f44, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:23.968797 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2c1222a05feb669c32ade4927b093f44): No space left in cache to reserve 69591 bytes, will continue without cache download 2024.10.22 00:58:23.976730 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 133 entries to flush up to offset 20098 2024.10.22 00:58:23.979060 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_57_57_0 to 202410_60_60_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:23.981824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2c1222a05feb669c32ade4927b093f44, offset: 0, size: 0 2024.10.22 00:58:23.983222 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2c1222a05feb669c32ade4927b093f44 for removal 2024.10.22 00:58:23.985244 [ 1476 ] {} CacheMetadata: Key 2c1222a05feb669c32ade4927b093f44 is removed from metadata 2024.10.22 00:58:24.278477 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_397_86, 202410_398_398_0] 2024.10.22 00:58:24.279895 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_397_86, 202410_398_398_0] 2024.10.22 00:58:24.287936 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_397_86, 202410_398_398_0] 2024.10.22 00:58:24.497972 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:24.502245 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:24.556036 [ 1177 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 00:58:24.591211 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_497_132} MergeTask::MergeProjectionsStage: Merge sorted 28755 rows, containing 13 columns (13 merged, 0 gathered) in 2.816071832 sec., 10211.032145290817 rows/sec., 1.55 MiB/sec. 2024.10.22 00:58:24.758622 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_284_284_0 (state Temporary) 2024.10.22 00:58:24.770751 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_497_132} MergedBlockOutputStream: filled checksums 202410_1_497_132 (state Temporary) 2024.10.22 00:58:24.903830 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_497_132} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_497_132 to 202410_1_497_132 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:24.906185 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_284_284_0 to 202410_290_290_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:24.909584 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_497_132} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_496_131, 202410_497_497_0] -> 202410_1_497_132 2024.10.22 00:58:24.962828 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.82 MiB. 2024.10.22 00:58:25.048526 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_285_83 to 202410_286_286_0 2024.10.22 00:58:25.065068 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.10 MiB using storage policy from min volume index 0 2024.10.22 00:58:25.072052 [ 1154 ] {} DiskLocal: Reserved 1.10 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:25.077349 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 20098 2024.10.22 00:58:25.083357 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_286_84} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_285_83 to 202410_286_286_0 into Compact with storage Full 2024.10.22 00:58:25.219958 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_286_84} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:25.225418 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bf7f3e947b5af80955542f1fe34e2c74) : [0, 69837]: Setting download as failed: File segment: [0, 69837], key: bf7f3e947b5af80955542f1fe34e2c74, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:25.227592 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bf7f3e947b5af80955542f1fe34e2c74): No space left in cache to reserve 69838 bytes, will continue without cache download 2024.10.22 00:58:25.231554 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_286_84} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_285_83, total 19652 rows starting from the beginning of the part 2024.10.22 00:58:25.291516 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_286_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_286_286_0, total 119 rows starting from the beginning of the part 2024.10.22 00:58:25.468580 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bf7f3e947b5af80955542f1fe34e2c74, offset: 0, size: 0 2024.10.22 00:58:25.472785 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bf7f3e947b5af80955542f1fe34e2c74 for removal 2024.10.22 00:58:25.476472 [ 1476 ] {} CacheMetadata: Key bf7f3e947b5af80955542f1fe34e2c74 is removed from metadata 2024.10.22 00:58:25.652356 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 28274 2024.10.22 00:58:25.828768 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 3.92 MiB, will set to 1.68 GiB (RSS), difference: 8.00 MiB 2024.10.22 00:58:25.919851 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:25.929526 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:26.002609 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2295 2024.10.22 00:58:26.127999 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_489_489_0 (state Temporary) 2024.10.22 00:58:26.281466 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_489_489_0 to 202410_502_502_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:26.321375 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_497_132 to 202410_498_498_0 2024.10.22 00:58:26.332067 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:26.332072 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28274 2024.10.22 00:58:26.336924 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:26.394591 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_498_133} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_497_132 to 202410_498_498_0 into Compact with storage Full 2024.10.22 00:58:26.453072 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3bb9750409c3854327b29af6487059b3) : [0, 97459]: Setting download as failed: File segment: [0, 97459], key: 3bb9750409c3854327b29af6487059b3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:26.458247 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3bb9750409c3854327b29af6487059b3): No space left in cache to reserve 97460 bytes, will continue without cache download 2024.10.22 00:58:26.542624 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_498_133} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:26.575161 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_286_84} MergeTask::MergeProjectionsStage: Merge sorted 19771 rows, containing 13 columns (13 merged, 0 gathered) in 1.494841661 sec., 13226.149976830222 rows/sec., 5.59 MiB/sec. 2024.10.22 00:58:26.580580 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3bb9750409c3854327b29af6487059b3, offset: 0, size: 0 2024.10.22 00:58:26.581566 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3bb9750409c3854327b29af6487059b3 for removal 2024.10.22 00:58:26.583157 [ 1476 ] {} CacheMetadata: Key 3bb9750409c3854327b29af6487059b3 is removed from metadata 2024.10.22 00:58:26.614229 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_286_84} MergedBlockOutputStream: filled checksums 202410_1_286_84 (state Temporary) 2024.10.22 00:58:26.643487 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_498_133} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_497_132, total 28755 rows starting from the beginning of the part 2024.10.22 00:58:26.665272 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_498_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_498_498_0, total 78 rows starting from the beginning of the part 2024.10.22 00:58:26.896374 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_286_84} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_286_84 to 202410_1_286_84 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:26.900146 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_286_84} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_285_83, 202410_286_286_0] -> 202410_1_286_84 2024.10.22 00:58:26.927046 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.82 MiB. 2024.10.22 00:58:27.282969 [ 1262 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:58:27.308886 [ 1262 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:58:27.759609 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(81c8bd7ffa89da4a24fc1759f6d24b2c) : [0, 44557]: Setting download as failed: File segment: [0, 44557], key: 81c8bd7ffa89da4a24fc1759f6d24b2c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:27.766900 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(81c8bd7ffa89da4a24fc1759f6d24b2c): No space left in cache to reserve 44558 bytes, will continue without cache download 2024.10.22 00:58:27.781916 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 81c8bd7ffa89da4a24fc1759f6d24b2c, offset: 0, size: 0 2024.10.22 00:58:27.783892 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 81c8bd7ffa89da4a24fc1759f6d24b2c for removal 2024.10.22 00:58:27.786787 [ 1476 ] {} CacheMetadata: Key 81c8bd7ffa89da4a24fc1759f6d24b2c is removed from metadata 2024.10.22 00:58:28.034802 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 462124 2024.10.22 00:58:28.171927 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_227_52, 202410_228_228_0] 2024.10.22 00:58:28.173131 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_227_52, 202410_228_228_0] 2024.10.22 00:58:28.181375 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_227_52, 202410_228_228_0] 2024.10.22 00:58:28.501826 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(572add9f4ddb2f5f6167da8f0c3f3c72) : [0, 262470]: Setting download as failed: File segment: [0, 262470], key: 572add9f4ddb2f5f6167da8f0c3f3c72, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:28.503373 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(572add9f4ddb2f5f6167da8f0c3f3c72): No space left in cache to reserve 262471 bytes, will continue without cache download 2024.10.22 00:58:28.540407 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 572add9f4ddb2f5f6167da8f0c3f3c72, offset: 0, size: 0 2024.10.22 00:58:28.544920 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 572add9f4ddb2f5f6167da8f0c3f3c72 for removal 2024.10.22 00:58:28.602796 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 4101 2024.10.22 00:58:28.615951 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 7.61 MiB, will set to 1.70 GiB (RSS), difference: 11.42 MiB 2024.10.22 00:58:28.649562 [ 1476 ] {} CacheMetadata: Key 572add9f4ddb2f5f6167da8f0c3f3c72 is removed from metadata 2024.10.22 00:58:28.796675 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:28.802440 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:29.004930 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_498_133} MergeTask::MergeProjectionsStage: Merge sorted 28833 rows, containing 13 columns (13 merged, 0 gathered) in 2.503846988 sec., 11515.480034597067 rows/sec., 1.75 MiB/sec. 2024.10.22 00:58:29.101543 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_498_133} MergedBlockOutputStream: filled checksums 202410_1_498_133 (state Temporary) 2024.10.22 00:58:29.170160 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_287_287_0 (state Temporary) 2024.10.22 00:58:29.210269 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_287_287_0 to 202410_300_300_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:29.267116 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:29.269462 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:29.277835 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4101 2024.10.22 00:58:29.359496 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_498_133} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_498_133 to 202410_1_498_133 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:29.366493 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_498_133} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_497_132, 202410_498_498_0] -> 202410_1_498_133 2024.10.22 00:58:29.384358 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.83 MiB. 2024.10.22 00:58:29.400363 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_274_274_0 (state Temporary) 2024.10.22 00:58:29.514540 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_274_274_0 to 202410_281_281_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:29.560449 [ 1217 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_276_55 to 202410_281_281_0 2024.10.22 00:58:29.564393 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 462124 2024.10.22 00:58:29.673071 [ 1217 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:29.675629 [ 1217 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:29.675732 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(2dd9b1212cdf63e0259701e2d44b7809) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2dd9b1212cdf63e0259701e2d44b7809, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 00:58:29.678006 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 2dd9b1212cdf63e0259701e2d44b7809, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 00:58:29.682194 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:58:29.711702 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_276_55 to 202410_281_281_0 into Compact with storage Full 2024.10.22 00:58:29.790565 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:58:29.792158 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:58:29.883308 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:29.900659 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} MergeTreeSequentialSource: Reading 59 marks from part 202410_1_276_55, total 468802 rows starting from the beginning of the part 2024.10.22 00:58:29.925563 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_277_277_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:58:29.936527 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_278_278_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:58:29.962435 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_279_279_0, total 1316 rows starting from the beginning of the part 2024.10.22 00:58:29.988832 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 250 2024.10.22 00:58:30.003811 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_280_280_0, total 1372 rows starting from the beginning of the part 2024.10.22 00:58:30.068575 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_59_12/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:58:30.135422 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_281_281_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:58:30.291804 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:30.294194 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:30.346746 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 28352 2024.10.22 00:58:30.486108 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_232_232_0 (state Temporary) 2024.10.22 00:58:30.584121 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_232_232_0 to 202410_242_242_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:30.628045 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_398_87, 202410_399_399_0] 2024.10.22 00:58:30.630862 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_398_87, 202410_399_399_0] 2024.10.22 00:58:30.655786 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_398_87, 202410_399_399_0] 2024.10.22 00:58:30.670796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5165f4c6741fe97cc6f40ca8960b3e16) : [0, 212176]: Setting download as failed: File segment: [0, 212176], key: 5165f4c6741fe97cc6f40ca8960b3e16, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:30.672629 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5165f4c6741fe97cc6f40ca8960b3e16): No space left in cache to reserve 212177 bytes, will continue without cache download 2024.10.22 00:58:30.685689 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5165f4c6741fe97cc6f40ca8960b3e16, offset: 0, size: 0 2024.10.22 00:58:30.690181 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5165f4c6741fe97cc6f40ca8960b3e16 for removal 2024.10.22 00:58:30.692357 [ 1476 ] {} CacheMetadata: Key 5165f4c6741fe97cc6f40ca8960b3e16 is removed from metadata 2024.10.22 00:58:30.702636 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_236_47, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0, 202410_241_241_0] 2024.10.22 00:58:30.705571 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_236_47, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0, 202410_241_241_0] 2024.10.22 00:58:30.716821 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:30.722852 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:30.733310 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_236_47, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0, 202410_241_241_0] 2024.10.22 00:58:30.781655 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 250 2024.10.22 00:58:30.787369 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_490_490_0 (state Temporary) 2024.10.22 00:58:30.833275 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_490_490_0 to 202410_503_503_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:30.941951 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_498_133 to 202410_499_499_0 2024.10.22 00:58:30.952376 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:30.957146 [ 1233 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.05 GiB. 2024.10.22 00:58:31.067079 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_499_134} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_498_133 to 202410_499_499_0 into Compact with storage Full 2024.10.22 00:58:31.083536 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28352 2024.10.22 00:58:31.202445 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_499_134} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:31.215057 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_499_134} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_498_133, total 28833 rows starting from the beginning of the part 2024.10.22 00:58:31.238476 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_499_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_499_499_0, total 76 rows starting from the beginning of the part 2024.10.22 00:58:31.521079 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 8.97 MiB, will set to 1.69 GiB (RSS), difference: -11.04 MiB 2024.10.22 00:58:32.061144 [ 1235 ] {} DatabaseCatalog: Execution took 401 ms. 2024.10.22 00:58:32.330669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(829fc684a30d232a03beee9d08b4d679) : [0, 155961]: Setting download as failed: File segment: [0, 155961], key: 829fc684a30d232a03beee9d08b4d679, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:32.333707 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(829fc684a30d232a03beee9d08b4d679): No space left in cache to reserve 155962 bytes, will continue without cache download 2024.10.22 00:58:32.351301 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_281_56/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:58:32.582276 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 829fc684a30d232a03beee9d08b4d679, offset: 0, size: 0 2024.10.22 00:58:32.589578 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 829fc684a30d232a03beee9d08b4d679 for removal 2024.10.22 00:58:32.592534 [ 1476 ] {} CacheMetadata: Key 829fc684a30d232a03beee9d08b4d679 is removed from metadata 2024.10.22 00:58:32.594500 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 164 entries to flush up to offset 20262 2024.10.22 00:58:32.695448 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:58:32.704038 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:58:32.725441 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:58:32.733713 [ 1693 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/otn/whrdzloqnvzorfrmtknrdviajrqrn, size 12044092 2024.10.22 00:58:32.745445 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:58:32.764413 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:58:32.774516 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:58:32.786182 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:58:32.839395 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_499_134} MergeTask::MergeProjectionsStage: Merge sorted 28909 rows, containing 13 columns (13 merged, 0 gathered) in 1.861298154 sec., 15531.63309052527 rows/sec., 2.35 MiB/sec. 2024.10.22 00:58:32.903808 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:58:32.905273 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_499_134} MergedBlockOutputStream: filled checksums 202410_1_499_134 (state Temporary) 2024.10.22 00:58:32.917825 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ClientIP 2024.10.22 00:58:32.954047 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/art/gvnqnyjohyulrxmyhwswpbctgglet (LogSeriesLimiter: on interval from 2024-10-22 00:58:21 to 2024-10-22 00:58:32 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:58:33.010672 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_499_134} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_499_134 to 202410_1_499_134 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:33.016161 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_499_134} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_498_133, 202410_499_499_0] -> 202410_1_499_134 2024.10.22 00:58:33.046422 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:33.073833 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.87 MiB. 2024.10.22 00:58:33.061815 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:33.199157 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_285_285_0 (state Temporary) 2024.10.22 00:58:33.250159 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_285_285_0 to 202410_291_291_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:33.290433 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_286_84 to 202410_291_291_0 2024.10.22 00:58:33.422759 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 20262 2024.10.22 00:58:33.529100 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.15 MiB using storage policy from min volume index 0 2024.10.22 00:58:33.559961 [ 1219 ] {} DiskLocal: Reserved 1.15 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:33.562779 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 12.85 MiB, will set to 1.69 GiB (RSS), difference: 16.08 MiB 2024.10.22 00:58:33.569272 [ 1219 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 299 ms. 2024.10.22 00:58:33.571675 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 2332 2024.10.22 00:58:33.573775 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_286_84 to 202410_291_291_0 into Compact with storage Full 2024.10.22 00:58:33.672101 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_228_53, 202410_229_229_0] 2024.10.22 00:58:33.673317 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_228_53, 202410_229_229_0] 2024.10.22 00:58:33.680513 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_228_53, 202410_229_229_0] 2024.10.22 00:58:33.722329 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a816eeccb9a4d00dde5c81afc2166ea0) : [0, 203714]: Setting download as failed: File segment: [0, 203714], key: a816eeccb9a4d00dde5c81afc2166ea0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:33.726748 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a816eeccb9a4d00dde5c81afc2166ea0): No space left in cache to reserve 203715 bytes, will continue without cache download 2024.10.22 00:58:33.735288 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:33.745603 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a816eeccb9a4d00dde5c81afc2166ea0, offset: 0, size: 0 2024.10.22 00:58:33.769805 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a816eeccb9a4d00dde5c81afc2166ea0 for removal 2024.10.22 00:58:33.774709 [ 1476 ] {} CacheMetadata: Key a816eeccb9a4d00dde5c81afc2166ea0 is removed from metadata 2024.10.22 00:58:33.794500 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_286_84, total 19771 rows starting from the beginning of the part 2024.10.22 00:58:33.931779 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_287_287_0, total 118 rows starting from the beginning of the part 2024.10.22 00:58:34.081471 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_288_288_0, total 90 rows starting from the beginning of the part 2024.10.22 00:58:34.123193 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_289_289_0, total 90 rows starting from the beginning of the part 2024.10.22 00:58:34.275257 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_290_290_0, total 133 rows starting from the beginning of the part 2024.10.22 00:58:34.331799 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_291_291_0, total 164 rows starting from the beginning of the part 2024.10.22 00:58:34.757554 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 153 bytes for 8aad256204ce9fe01db0ea829aa25a80:0) 2024.10.22 00:58:34.763386 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 70d5420e2039485bc3f8351e9014ee8c, offset: 0, size: 152 2024.10.22 00:58:34.767149 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 70d5420e2039485bc3f8351e9014ee8c for removal 2024.10.22 00:58:34.769316 [ 1476 ] {} CacheMetadata: Key 70d5420e2039485bc3f8351e9014ee8c is removed from metadata 2024.10.22 00:58:34.981487 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_399_88, 202410_400_400_0, 202410_401_401_0, 202410_402_402_0, 202410_403_403_0, 202410_404_404_0] 2024.10.22 00:58:34.982709 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_399_88, 202410_400_400_0, 202410_401_401_0, 202410_402_402_0, 202410_403_403_0, 202410_404_404_0] 2024.10.22 00:58:35.014283 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_399_88, 202410_400_400_0, 202410_401_401_0, 202410_402_402_0, 202410_403_403_0, 202410_404_404_0] 2024.10.22 00:58:35.104705 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 28438 2024.10.22 00:58:35.463656 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:35.570175 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:35.599297 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 14.65 MiB, will set to 1.70 GiB (RSS), difference: -12.30 MiB 2024.10.22 00:58:35.681756 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_491_491_0 (state Temporary) 2024.10.22 00:58:35.776306 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_491_491_0 to 202410_504_504_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:35.801212 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2d15d702117cc5c2182472ee2dfe14bb) : [0, 265630]: Setting download as failed: File segment: [0, 265630], key: 2d15d702117cc5c2182472ee2dfe14bb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:35.803204 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2d15d702117cc5c2182472ee2dfe14bb): No space left in cache to reserve 265631 bytes, will continue without cache download 2024.10.22 00:58:35.805592 [ 1256 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_499_134 to 202410_504_504_0 2024.10.22 00:58:35.825083 [ 1256 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.04 MiB using storage policy from min volume index 0 2024.10.22 00:58:35.829498 [ 1256 ] {} DiskLocal: Reserved 1.04 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:35.880786 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_499_134 to 202410_504_504_0 into Compact with storage Full 2024.10.22 00:58:35.935005 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28438 2024.10.22 00:58:36.075121 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:36.141059 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2d15d702117cc5c2182472ee2dfe14bb, offset: 0, size: 0 2024.10.22 00:58:36.144543 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2d15d702117cc5c2182472ee2dfe14bb for removal 2024.10.22 00:58:36.150309 [ 1476 ] {} CacheMetadata: Key 2d15d702117cc5c2182472ee2dfe14bb is removed from metadata 2024.10.22 00:58:36.196458 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_499_134, total 28909 rows starting from the beginning of the part 2024.10.22 00:58:36.375663 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_500_500_0, total 63 rows starting from the beginning of the part 2024.10.22 00:58:36.448677 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_189_39, 202410_190_190_0, 202410_191_191_0, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0] 2024.10.22 00:58:36.454797 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_189_39, 202410_190_190_0, 202410_191_191_0, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0] 2024.10.22 00:58:36.494159 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_189_39, 202410_190_190_0, 202410_191_191_0, 202410_192_192_0, 202410_193_193_0, 202410_194_194_0] 2024.10.22 00:58:36.572296 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 463660 2024.10.22 00:58:36.577049 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_501_501_0, total 57 rows starting from the beginning of the part 2024.10.22 00:58:36.591742 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} MergeTask::MergeProjectionsStage: Merge sorted 20366 rows, containing 13 columns (13 merged, 0 gathered) in 3.021179619 sec., 6741.075529544673 rows/sec., 2.85 MiB/sec. 2024.10.22 00:58:36.696740 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} MergedBlockOutputStream: filled checksums 202410_1_291_85 (state Temporary) 2024.10.22 00:58:36.735025 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_502_502_0, total 76 rows starting from the beginning of the part 2024.10.22 00:58:36.780852 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 4113 2024.10.22 00:58:36.896802 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_503_503_0, total 78 rows starting from the beginning of the part 2024.10.22 00:58:36.922358 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:36.927373 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:36.971197 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_504_504_0, total 86 rows starting from the beginning of the part 2024.10.22 00:58:37.029442 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for dd8aedd29e4fe8e286a6573dd9be991f:0) 2024.10.22 00:58:37.035144 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: afdea3e2e62037f06af3df488808a743, offset: 0, size: 160 2024.10.22 00:58:37.039402 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key afdea3e2e62037f06af3df488808a743 for removal 2024.10.22 00:58:37.042698 [ 1476 ] {} CacheMetadata: Key afdea3e2e62037f06af3df488808a743 is removed from metadata 2024.10.22 00:58:37.121446 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_288_288_0 (state Temporary) 2024.10.22 00:58:37.202078 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_291_85 to 202410_1_291_85 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:37.205171 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_291_85} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_286_84, 202410_291_291_0] -> 202410_1_291_85 2024.10.22 00:58:37.279385 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.18 MiB. 2024.10.22 00:58:37.289298 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_288_288_0 to 202410_301_301_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:37.336400 [ 1258 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_296_59 to 202410_301_301_0 2024.10.22 00:58:37.346404 [ 1258 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:37.356707 [ 1258 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:37.395834 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_296_59 to 202410_301_301_0 into Compact with storage Full 2024.10.22 00:58:37.478311 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4113 2024.10.22 00:58:37.480692 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:37.490201 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:37.492233 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_296_59, total 4141 rows starting from the beginning of the part 2024.10.22 00:58:37.493393 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:58:37.676019 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_297_297_0, total 23 rows starting from the beginning of the part 2024.10.22 00:58:37.733006 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_298_298_0, total 23 rows starting from the beginning of the part 2024.10.22 00:58:37.734191 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_275_275_0 (state Temporary) 2024.10.22 00:58:37.943531 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_275_275_0 to 202410_282_282_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:37.981260 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 463660 2024.10.22 00:58:38.038547 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2cb2aae2d1e8c02a474a6b31c5a066a7) : [0, 331622]: Setting download as failed: File segment: [0, 331622], key: 2cb2aae2d1e8c02a474a6b31c5a066a7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:38.040055 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2cb2aae2d1e8c02a474a6b31c5a066a7): No space left in cache to reserve 331623 bytes, will continue without cache download 2024.10.22 00:58:38.072333 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_299_299_0, total 7 rows starting from the beginning of the part 2024.10.22 00:58:38.281282 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2cb2aae2d1e8c02a474a6b31c5a066a7, offset: 0, size: 0 2024.10.22 00:58:38.283766 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2cb2aae2d1e8c02a474a6b31c5a066a7 for removal 2024.10.22 00:58:38.286449 [ 1476 ] {} CacheMetadata: Key 2cb2aae2d1e8c02a474a6b31c5a066a7 is removed from metadata 2024.10.22 00:58:38.331753 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_300_300_0, total 13 rows starting from the beginning of the part 2024.10.22 00:58:38.458749 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_301_301_0, total 12 rows starting from the beginning of the part 2024.10.22 00:58:38.964109 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 159 bytes for 53b49ea614a596a87af989666db21f93:0) 2024.10.22 00:58:39.067923 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dc912f4ce149a1ba1d02f718ad4bda7f, offset: 0, size: 165 2024.10.22 00:58:39.073635 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dc912f4ce149a1ba1d02f718ad4bda7f for removal 2024.10.22 00:58:39.080482 [ 1476 ] {} CacheMetadata: Key dc912f4ce149a1ba1d02f718ad4bda7f is removed from metadata 2024.10.22 00:58:39.482713 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} MergeTask::MergeProjectionsStage: Merge sorted 29269 rows, containing 13 columns (13 merged, 0 gathered) in 3.608076711 sec., 8112.07808048181 rows/sec., 1.23 MiB/sec. 2024.10.22 00:58:39.543642 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} MergedBlockOutputStream: filled checksums 202410_1_504_135 (state Temporary) 2024.10.22 00:58:39.585175 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 12.56 MiB, will set to 1.70 GiB (RSS), difference: 4.03 MiB 2024.10.22 00:58:39.741323 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(03d8b3c6058571e4771665b069e12514) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 03d8b3c6058571e4771665b069e12514, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 00:58:39.746012 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 03d8b3c6058571e4771665b069e12514, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 00:58:39.752824 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:58:39.788620 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:58:39.793946 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:58:39.955139 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_504_135 to 202410_1_504_135 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:39.972206 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 28513 2024.10.22 00:58:39.976281 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_504_135} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_499_134, 202410_504_504_0] -> 202410_1_504_135 2024.10.22 00:58:40.112844 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} MergeTask::MergeProjectionsStage: Merge sorted 4219 rows, containing 12 columns (12 merged, 0 gathered) in 2.737438486 sec., 1541.221847203912 rows/sec., 241.49 KiB/sec. 2024.10.22 00:58:40.122957 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.27 MiB. 2024.10.22 00:58:40.268886 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} MergedBlockOutputStream: filled checksums 202410_1_301_60 (state Temporary) 2024.10.22 00:58:40.329117 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:40.355662 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:40.610421 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_492_492_0 (state Temporary) 2024.10.22 00:58:40.685217 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7e17d8c067dedb53c0e17cde7d8a4143) : [0, 538889]: Setting download as failed: File segment: [0, 538889], key: 7e17d8c067dedb53c0e17cde7d8a4143, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:40.687428 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7e17d8c067dedb53c0e17cde7d8a4143): No space left in cache to reserve 538890 bytes, will continue without cache download 2024.10.22 00:58:40.727620 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7e17d8c067dedb53c0e17cde7d8a4143, offset: 0, size: 0 2024.10.22 00:58:40.731772 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7e17d8c067dedb53c0e17cde7d8a4143 for removal 2024.10.22 00:58:40.735751 [ 1476 ] {} CacheMetadata: Key 7e17d8c067dedb53c0e17cde7d8a4143 is removed from metadata 2024.10.22 00:58:40.787952 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_492_492_0 to 202410_505_505_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:40.799773 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_301_60 to 202410_1_301_60 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:40.802545 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_301_60} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_296_59, 202410_301_301_0] -> 202410_1_301_60 2024.10.22 00:58:40.846874 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.44 MiB. 2024.10.22 00:58:40.952562 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28513 2024.10.22 00:58:40.974260 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 129 entries to flush up to offset 20391 2024.10.22 00:58:41.107948 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:41.111604 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:41.175272 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_286_286_0 (state Temporary) 2024.10.22 00:58:41.220685 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_286_286_0 to 202410_292_292_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:41.391474 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 20391 2024.10.22 00:58:42.020618 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 159 bytes for 8cb8959c69cc28f85de7e55ff3be824f:0) 2024.10.22 00:58:42.022869 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4d81b6d0f499a1a020b992fa746ae8cd, offset: 0, size: 160 2024.10.22 00:58:42.026069 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4d81b6d0f499a1a020b992fa746ae8cd for removal 2024.10.22 00:58:42.028902 [ 1476 ] {} CacheMetadata: Key 4d81b6d0f499a1a020b992fa746ae8cd is removed from metadata 2024.10.22 00:58:42.300949 [ 1175 ] {} DatabaseCatalog: Execution took 238 ms. 2024.10.22 00:58:42.325311 [ 1163 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:58:42.352476 [ 1163 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:58:42.601273 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 9.58 MiB, will set to 1.70 GiB (RSS), difference: 6.72 MiB 2024.10.22 00:58:42.833077 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(5a9443235694259bda78a336928c8852) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5a9443235694259bda78a336928c8852, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 00:58:42.849568 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 5a9443235694259bda78a336928c8852, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 00:58:42.856233 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:58:43.197800 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_281_56/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:58:43.491160 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:58:43.729122 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ae4f2af7d8fe628281d885187ab24dc6) : [0, 533335]: Setting download as failed: File segment: [0, 533335], key: ae4f2af7d8fe628281d885187ab24dc6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:43.784793 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_229_54, 202410_230_230_0] 2024.10.22 00:58:43.787671 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_229_54, 202410_230_230_0] 2024.10.22 00:58:43.797528 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_229_54, 202410_230_230_0] 2024.10.22 00:58:43.835385 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ae4f2af7d8fe628281d885187ab24dc6): No space left in cache to reserve 533336 bytes, will continue without cache download 2024.10.22 00:58:43.984207 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ae4f2af7d8fe628281d885187ab24dc6, offset: 0, size: 0 2024.10.22 00:58:43.987484 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ae4f2af7d8fe628281d885187ab24dc6 for removal 2024.10.22 00:58:43.992119 [ 1476 ] {} CacheMetadata: Key ae4f2af7d8fe628281d885187ab24dc6 is removed from metadata 2024.10.22 00:58:44.897321 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for 24959969a3b345222dc980f8570e52b9:0) 2024.10.22 00:58:44.901304 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 742e8bca04fa1b4a26faa8a770404c27, offset: 0, size: 166 2024.10.22 00:58:44.905757 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 742e8bca04fa1b4a26faa8a770404c27 for removal 2024.10.22 00:58:44.981711 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 4138 2024.10.22 00:58:44.986894 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 464812 2024.10.22 00:58:44.912813 [ 1476 ] {} CacheMetadata: Key 742e8bca04fa1b4a26faa8a770404c27 is removed from metadata 2024.10.22 00:58:45.037322 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_59_12/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:58:45.076830 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 28562 2024.10.22 00:58:45.412772 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:45.418668 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:45.479244 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_289_289_0 (state Temporary) 2024.10.22 00:58:45.674374 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_289_289_0 to 202410_302_302_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:45.720084 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:45.724771 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:45.733479 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4138 2024.10.22 00:58:45.783195 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 251 2024.10.22 00:58:45.791929 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_493_493_0 (state Temporary) 2024.10.22 00:58:45.980844 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_493_493_0 to 202410_506_506_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:46.043676 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28562 2024.10.22 00:58:46.102899 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:46.111376 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:46.277163 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_276_276_0 (state Temporary) 2024.10.22 00:58:46.296531 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:46.301063 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:46.346698 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_276_276_0 to 202410_283_283_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:46.381080 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 464812 2024.10.22 00:58:46.387672 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_233_233_0 (state Temporary) 2024.10.22 00:58:46.581056 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0694ce5039c68027720e11fc361d9af2) : [0, 332250]: Setting download as failed: File segment: [0, 332250], key: 0694ce5039c68027720e11fc361d9af2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:46.584472 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0694ce5039c68027720e11fc361d9af2): No space left in cache to reserve 332251 bytes, will continue without cache download 2024.10.22 00:58:46.646464 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0694ce5039c68027720e11fc361d9af2, offset: 0, size: 0 2024.10.22 00:58:46.653175 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0694ce5039c68027720e11fc361d9af2 for removal 2024.10.22 00:58:46.675726 [ 1476 ] {} CacheMetadata: Key 0694ce5039c68027720e11fc361d9af2 is removed from metadata 2024.10.22 00:58:46.684535 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_233_233_0 to 202410_243_243_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:46.980275 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 251 2024.10.22 00:58:47.239644 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} MergeTask::MergeProjectionsStage: Merge sorted 475714 rows, containing 4 columns (4 merged, 0 gathered) in 17.546347338 sec., 27111.853586173434 rows/sec., 397.53 KiB/sec. 2024.10.22 00:58:47.295523 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} MergedBlockOutputStream: filled checksums 202410_1_281_56 (state Temporary) 2024.10.22 00:58:47.371486 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_281_56 to 202410_1_281_56 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:47.396522 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_281_56} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_276_55, 202410_281_281_0] -> 202410_1_281_56 2024.10.22 00:58:47.550530 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:58:47.801092 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 871d0d7069cd88f9000053b0405a78ed:0) 2024.10.22 00:58:47.803375 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cce1ceb7f8e92904b5ad5c0440b92f82, offset: 0, size: 166 2024.10.22 00:58:47.809052 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cce1ceb7f8e92904b5ad5c0440b92f82 for removal 2024.10.22 00:58:47.814323 [ 1476 ] {} CacheMetadata: Key cce1ceb7f8e92904b5ad5c0440b92f82 is removed from metadata 2024.10.22 00:58:47.823161 [ 1238 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 00:58:48.176453 [ 1195 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 00:58:48.523798 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 10.98 MiB, will set to 1.69 GiB (RSS), difference: 3.83 MiB 2024.10.22 00:58:48.824678 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f9d1e1e6fb92648c150290c685ab96c4) : [0, 243042]: Setting download as failed: File segment: [0, 243042], key: f9d1e1e6fb92648c150290c685ab96c4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:48.826088 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f9d1e1e6fb92648c150290c685ab96c4): No space left in cache to reserve 243043 bytes, will continue without cache download 2024.10.22 00:58:48.868019 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f9d1e1e6fb92648c150290c685ab96c4, offset: 0, size: 0 2024.10.22 00:58:48.869550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f9d1e1e6fb92648c150290c685ab96c4 for removal 2024.10.22 00:58:48.875239 [ 1476 ] {} CacheMetadata: Key f9d1e1e6fb92648c150290c685ab96c4 is removed from metadata 2024.10.22 00:58:48.913906 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 67 entries to flush up to offset 20458 2024.10.22 00:58:49.068604 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:49.072729 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.04 GiB. 2024.10.22 00:58:49.217202 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_287_287_0 (state Temporary) 2024.10.22 00:58:49.369123 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_287_287_0 to 202410_293_293_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:49.400294 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 20458 2024.10.22 00:58:49.790570 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:58:49.794200 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:58:49.802823 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 962 bytes for abb7b49c405e096d99562cb1c31482e3:0) 2024.10.22 00:58:49.805586 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2bdd94009f723a04872d5b3b9693c8a2, offset: 0, size: 150 2024.10.22 00:58:49.812087 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2bdd94009f723a04872d5b3b9693c8a2 for removal 2024.10.22 00:58:49.822574 [ 1476 ] {} CacheMetadata: Key 2bdd94009f723a04872d5b3b9693c8a2 is removed from metadata 2024.10.22 00:58:49.824856 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f12ce6d01b32d7eb8549c9d0523b930b, offset: 0, size: 149 2024.10.22 00:58:49.828617 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f12ce6d01b32d7eb8549c9d0523b930b for removal 2024.10.22 00:58:49.832202 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e4c3308ac8ced85cde0c26cc53a07961, offset: 0, size: 159 2024.10.22 00:58:49.836106 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e4c3308ac8ced85cde0c26cc53a07961 for removal 2024.10.22 00:58:49.839332 [ 1476 ] {} CacheMetadata: Key e4c3308ac8ced85cde0c26cc53a07961 is removed from metadata 2024.10.22 00:58:49.840963 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4221a93e7f5018f130dd921741e474d8, offset: 0, size: 159 2024.10.22 00:58:49.844787 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4221a93e7f5018f130dd921741e474d8 for removal 2024.10.22 00:58:49.849429 [ 1476 ] {} CacheMetadata: Key f12ce6d01b32d7eb8549c9d0523b930b is removed from metadata 2024.10.22 00:58:49.851472 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a629070495d3ce3844270180bea8f27c, offset: 0, size: 166 2024.10.22 00:58:49.857003 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a629070495d3ce3844270180bea8f27c for removal 2024.10.22 00:58:49.861348 [ 1476 ] {} CacheMetadata: Key 4221a93e7f5018f130dd921741e474d8 is removed from metadata 2024.10.22 00:58:49.866166 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1984d75b2ada31652c002fadc3a509f9, offset: 0, size: 165 2024.10.22 00:58:49.873048 [ 1476 ] {} CacheMetadata: Key a629070495d3ce3844270180bea8f27c is removed from metadata 2024.10.22 00:58:49.873629 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1984d75b2ada31652c002fadc3a509f9 for removal 2024.10.22 00:58:49.879628 [ 1476 ] {} CacheMetadata: Key 1984d75b2ada31652c002fadc3a509f9 is removed from metadata 2024.10.22 00:58:50.057574 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 28637 2024.10.22 00:58:50.343816 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:50.350405 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:58:50.408181 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_494_494_0 (state Temporary) 2024.10.22 00:58:50.460979 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_494_494_0 to 202410_507_507_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:50.634815 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 156 bytes for 24096cfd3a330a63a71b53acc941065c:0) 2024.10.22 00:58:50.636719 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d75f91f8cea80ecfe58ca2dbce9d04d3, offset: 0, size: 160 2024.10.22 00:58:50.684526 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 9.48 MiB, will set to 1.69 GiB (RSS), difference: 11.13 MiB 2024.10.22 00:58:50.699220 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28637 2024.10.22 00:58:50.741024 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d75f91f8cea80ecfe58ca2dbce9d04d3 for removal 2024.10.22 00:58:50.745072 [ 1476 ] {} CacheMetadata: Key d75f91f8cea80ecfe58ca2dbce9d04d3 is removed from metadata 2024.10.22 00:58:51.343772 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 839 bytes for 6806fd49816baebea28b3e9d42ffb2a7:0) 2024.10.22 00:58:51.345820 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4b40bfdd2b7a6ad7f41e98ea0e24c644, offset: 0, size: 160 2024.10.22 00:58:51.349966 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4b40bfdd2b7a6ad7f41e98ea0e24c644 for removal 2024.10.22 00:58:51.353689 [ 1476 ] {} CacheMetadata: Key 4b40bfdd2b7a6ad7f41e98ea0e24c644 is removed from metadata 2024.10.22 00:58:51.354167 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e4324b96a141839428d2f996c72ba151, offset: 0, size: 166 2024.10.22 00:58:51.460416 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e4324b96a141839428d2f996c72ba151 for removal 2024.10.22 00:58:51.470726 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a22a72f0b206315f2bf8271e81df087c, offset: 0, size: 160 2024.10.22 00:58:51.473324 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a22a72f0b206315f2bf8271e81df087c for removal 2024.10.22 00:58:51.475696 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 22f7d4954c3cd346cc84c479b72efc82, offset: 0, size: 165 2024.10.22 00:58:51.478442 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 22f7d4954c3cd346cc84c479b72efc82 for removal 2024.10.22 00:58:51.480244 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 36ce82d99b81c1ebd23303d96f9671eb, offset: 0, size: 185 2024.10.22 00:58:51.482677 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 36ce82d99b81c1ebd23303d96f9671eb for removal 2024.10.22 00:58:51.486163 [ 1476 ] {} CacheMetadata: Key 36ce82d99b81c1ebd23303d96f9671eb is removed from metadata 2024.10.22 00:58:51.494355 [ 1476 ] {} CacheMetadata: Key 22f7d4954c3cd346cc84c479b72efc82 is removed from metadata 2024.10.22 00:58:51.502088 [ 1476 ] {} CacheMetadata: Key a22a72f0b206315f2bf8271e81df087c is removed from metadata 2024.10.22 00:58:51.521714 [ 1476 ] {} CacheMetadata: Key e4324b96a141839428d2f996c72ba151 is removed from metadata 2024.10.22 00:58:51.607668 [ 1718 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mjj/osfczchqiomyjtrkihuwaaugjkujc, size 1552087 2024.10.22 00:58:51.710400 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RemoteIP 2024.10.22 00:58:51.950437 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RemoteIP 2024.10.22 00:58:51.960386 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RemoteIP 2024.10.22 00:58:51.977645 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RemoteIP 2024.10.22 00:58:51.991161 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RemoteIP 2024.10.22 00:58:52.108533 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RemoteIP 2024.10.22 00:58:52.147903 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RemoteIP 2024.10.22 00:58:52.189241 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tor/spmugwarivyhniwjeplzjzczallfm (LogSeriesLimiter: on interval from 2024-10-22 00:58:32 to 2024-10-22 00:58:52 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:58:52.229206 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e650198c7c2c024cdb14bb5fb9a011f3) : [0, 435134]: Setting download as failed: File segment: [0, 435134], key: e650198c7c2c024cdb14bb5fb9a011f3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:52.230305 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e650198c7c2c024cdb14bb5fb9a011f3): No space left in cache to reserve 435135 bytes, will continue without cache download 2024.10.22 00:58:52.382134 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e650198c7c2c024cdb14bb5fb9a011f3, offset: 0, size: 0 2024.10.22 00:58:52.383313 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e650198c7c2c024cdb14bb5fb9a011f3 for removal 2024.10.22 00:58:52.386093 [ 1476 ] {} CacheMetadata: Key e650198c7c2c024cdb14bb5fb9a011f3 is removed from metadata 2024.10.22 00:58:52.502964 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:52.607332 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:58:52.638234 [ 1230 ] {} DatabaseCatalog: Execution took 336 ms. 2024.10.22 00:58:52.738218 [ 1216 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 204 ms. 2024.10.22 00:58:53.235543 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 4151 2024.10.22 00:58:53.384658 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 465964 2024.10.22 00:58:53.445123 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:53.457038 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 157 bytes for fccc94ea822b304b9e59d97ead17ce1d:0) 2024.10.22 00:58:53.552762 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:58:53.554461 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b8a9fbaf807ee286f172d76d49ae9691, offset: 0, size: 159 2024.10.22 00:58:53.565509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b8a9fbaf807ee286f172d76d49ae9691 for removal 2024.10.22 00:58:53.570346 [ 1476 ] {} CacheMetadata: Key b8a9fbaf807ee286f172d76d49ae9691 is removed from metadata 2024.10.22 00:58:53.596703 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_290_290_0 (state Temporary) 2024.10.22 00:58:53.753075 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_290_290_0 to 202410_303_303_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:53.836699 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_230_55, 202410_231_231_0, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0] 2024.10.22 00:58:53.839807 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_230_55, 202410_231_231_0, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0] 2024.10.22 00:58:53.924635 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4151 2024.10.22 00:58:53.994084 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_230_55, 202410_231_231_0, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0] 2024.10.22 00:58:54.297032 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:54.299003 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:58:54.405646 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 7 file segments (while reserving 1037 bytes for ccbc9be1064a70a167fac274ee3f8748:0) 2024.10.22 00:58:54.419099 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4af22bc6560df4d32d5bb45be6ef3928, offset: 0, size: 5256 2024.10.22 00:58:54.422815 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 4af22bc6560df4d32d5bb45be6ef3928 for removal 2024.10.22 00:58:54.430235 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_277_277_0 (state Temporary) 2024.10.22 00:58:54.430731 [ 1476 ] {} CacheMetadata: Key 4af22bc6560df4d32d5bb45be6ef3928 is removed from metadata 2024.10.22 00:58:54.433806 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ee9d04379ac1b0b465f2a4477785b6cc, offset: 0, size: 147 2024.10.22 00:58:54.440140 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ee9d04379ac1b0b465f2a4477785b6cc for removal 2024.10.22 00:58:54.443246 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1e396f945b89b49efbc771d5bf06359d, offset: 0, size: 159 2024.10.22 00:58:54.446241 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1e396f945b89b49efbc771d5bf06359d for removal 2024.10.22 00:58:54.460264 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bda5e8fc7cca99e488a9b81fb3428165, offset: 0, size: 152 2024.10.22 00:58:54.465774 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bda5e8fc7cca99e488a9b81fb3428165 for removal 2024.10.22 00:58:54.472674 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7ee048356a2007790b4d04a0e0bf6bb8, offset: 0, size: 154 2024.10.22 00:58:54.478253 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7ee048356a2007790b4d04a0e0bf6bb8 for removal 2024.10.22 00:58:54.480730 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 424f887651f0f0ba2ba45848bccef496, offset: 0, size: 165 2024.10.22 00:58:54.481711 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 252 2024.10.22 00:58:54.483219 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 424f887651f0f0ba2ba45848bccef496 for removal 2024.10.22 00:58:54.486370 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0cd0d68136ead26688686b0d3c5adf60, offset: 0, size: 159 2024.10.22 00:58:54.488401 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0cd0d68136ead26688686b0d3c5adf60 for removal 2024.10.22 00:58:54.563298 [ 1723 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/art/gvnqnyjohyulrxmyhwswpbctgglet, size 2928449 2024.10.22 00:58:54.575388 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_277_277_0 to 202410_284_284_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:54.659874 [ 1476 ] {} CacheMetadata: Key 0cd0d68136ead26688686b0d3c5adf60 is removed from metadata 2024.10.22 00:58:54.669358 [ 1476 ] {} CacheMetadata: Key 424f887651f0f0ba2ba45848bccef496 is removed from metadata 2024.10.22 00:58:54.681160 [ 1476 ] {} CacheMetadata: Key 7ee048356a2007790b4d04a0e0bf6bb8 is removed from metadata 2024.10.22 00:58:54.689238 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 465964 2024.10.22 00:58:54.704637 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 28722 2024.10.22 00:58:54.706472 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 7.77 MiB, will set to 1.69 GiB (RSS), difference: -13.28 MiB 2024.10.22 00:58:54.744704 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_221_44, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0, 202410_225_225_0, 202410_226_226_0] 2024.10.22 00:58:54.748353 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_221_44, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0, 202410_225_225_0, 202410_226_226_0] 2024.10.22 00:58:54.788242 [ 1476 ] {} CacheMetadata: Key bda5e8fc7cca99e488a9b81fb3428165 is removed from metadata 2024.10.22 00:58:54.793625 [ 1476 ] {} CacheMetadata: Key 1e396f945b89b49efbc771d5bf06359d is removed from metadata 2024.10.22 00:58:54.800064 [ 1476 ] {} CacheMetadata: Key ee9d04379ac1b0b465f2a4477785b6cc is removed from metadata 2024.10.22 00:58:54.860019 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:54.863847 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:58:54.899281 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_221_44, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0, 202410_225_225_0, 202410_226_226_0] 2024.10.22 00:58:54.932204 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_495_495_0 (state Temporary) 2024.10.22 00:58:54.996548 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:58:55.003876 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_495_495_0 to 202410_508_508_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:55.026991 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:58:55.060341 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:58:55.079961 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:58:55.091927 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:58:55.125662 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:58:55.152889 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:58:55.178581 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:58:55.179946 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28722 2024.10.22 00:58:55.195592 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ClientIP6 2024.10.22 00:58:55.278925 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_58_58_0 (state Temporary) 2024.10.22 00:58:55.331781 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:55.341373 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a20ef71e608eb430dd78b3663b247208) : [0, 397868]: Setting download as failed: File segment: [0, 397868], key: a20ef71e608eb430dd78b3663b247208, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:55.343854 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a20ef71e608eb430dd78b3663b247208): No space left in cache to reserve 397869 bytes, will continue without cache download 2024.10.22 00:58:55.437615 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:58:55.608655 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_234_234_0 (state Temporary) 2024.10.22 00:58:55.780351 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a20ef71e608eb430dd78b3663b247208, offset: 0, size: 0 2024.10.22 00:58:55.782260 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a20ef71e608eb430dd78b3663b247208 for removal 2024.10.22 00:58:55.784687 [ 1476 ] {} CacheMetadata: Key a20ef71e608eb430dd78b3663b247208 is removed from metadata 2024.10.22 00:58:55.788761 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_234_234_0 to 202410_244_244_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:55.974052 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 252 2024.10.22 00:58:56.015161 [ 1220 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_239_49 to 202410_244_244_0 2024.10.22 00:58:56.059361 [ 1220 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:56.070952 [ 1220 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:58:56.110439 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_239_49 to 202410_244_244_0 into Compact with storage Full 2024.10.22 00:58:56.303943 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 7.05 MiB, will set to 1.68 GiB (RSS), difference: -19.03 MiB 2024.10.22 00:58:56.336228 [ 1166 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 248 ms. 2024.10.22 00:58:56.587589 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:58:56.661003 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_239_49, total 283 rows starting from the beginning of the part 2024.10.22 00:58:56.832423 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_240_240_0, total 1 rows starting from the beginning of the part 2024.10.22 00:58:56.911886 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 53 entries to flush up to offset 20511 2024.10.22 00:58:57.035463 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:57.038726 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:58:57.060810 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_241_241_0, total 1 rows starting from the beginning of the part 2024.10.22 00:58:57.107635 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(682ad896b80f6e7d8267bed8077a90da) : [0, 191162]: Setting download as failed: File segment: [0, 191162], key: 682ad896b80f6e7d8267bed8077a90da, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:57.114874 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(682ad896b80f6e7d8267bed8077a90da): No space left in cache to reserve 191163 bytes, will continue without cache download 2024.10.22 00:58:57.159773 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 682ad896b80f6e7d8267bed8077a90da, offset: 0, size: 0 2024.10.22 00:58:57.162193 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 682ad896b80f6e7d8267bed8077a90da for removal 2024.10.22 00:58:57.173055 [ 1476 ] {} CacheMetadata: Key 682ad896b80f6e7d8267bed8077a90da is removed from metadata 2024.10.22 00:58:57.288746 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_288_288_0 (state Temporary) 2024.10.22 00:58:57.343070 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_288_288_0 to 202410_294_294_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:57.364988 [ 1243 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:58:57.496186 [ 1243 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:58:57.577260 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_242_242_0, total 1 rows starting from the beginning of the part 2024.10.22 00:58:57.586043 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 20511 2024.10.22 00:58:57.861296 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_243_243_0, total 1 rows starting from the beginning of the part 2024.10.22 00:58:58.239771 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2b11cc19c918379928123126a21ea193) : [0, 190316]: Setting download as failed: File segment: [0, 190316], key: 2b11cc19c918379928123126a21ea193, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:58:58.242700 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2b11cc19c918379928123126a21ea193): No space left in cache to reserve 190317 bytes, will continue without cache download 2024.10.22 00:58:58.271272 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_244_244_0, total 1 rows starting from the beginning of the part 2024.10.22 00:58:58.523405 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 7.39 MiB, will set to 1.69 GiB (RSS), difference: 7.39 MiB 2024.10.22 00:58:58.578634 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2b11cc19c918379928123126a21ea193, offset: 0, size: 0 2024.10.22 00:58:58.681973 [ 1253 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 00:58:58.691061 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2b11cc19c918379928123126a21ea193 for removal 2024.10.22 00:58:58.692966 [ 1476 ] {} CacheMetadata: Key 2b11cc19c918379928123126a21ea193 is removed from metadata 2024.10.22 00:58:59.026345 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_244_50/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:58:59.188500 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 28789 2024.10.22 00:58:59.250336 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(5e138c155f144795011d6af2c59d0d74) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5e138c155f144795011d6af2c59d0d74, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 00:58:59.372479 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 5e138c155f144795011d6af2c59d0d74, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 00:58:59.375195 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:58:59.572016 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1_404_89, 202410_405_405_0, 202410_406_406_0, 202410_407_407_0] 2024.10.22 00:58:59.574409 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1_404_89, 202410_405_405_0, 202410_406_406_0, 202410_407_407_0] 2024.10.22 00:58:59.591262 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1_404_89, 202410_405_405_0, 202410_406_406_0, 202410_407_407_0] 2024.10.22 00:58:59.641661 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:58:59.652223 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:58:59.790638 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:58:59.792417 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:58:59.824111 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_496_496_0 (state Temporary) 2024.10.22 00:58:59.870423 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_496_496_0 to 202410_509_509_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:58:59.886846 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_504_135 to 202410_505_505_0 2024.10.22 00:58:59.891721 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.01 MiB using storage policy from min volume index 0 2024.10.22 00:58:59.893414 [ 1268 ] {} DiskLocal: Reserved 1.01 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:58:59.907962 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28789 2024.10.22 00:58:59.921441 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_505_136} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_504_135 to 202410_505_505_0 into Compact with storage Full 2024.10.22 00:58:59.992492 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_505_136} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:00.015401 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_505_136} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_504_135, total 29269 rows starting from the beginning of the part 2024.10.22 00:59:00.059295 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} MergeTask::MergeProjectionsStage: Merge sorted 288 rows, containing 32 columns (32 merged, 0 gathered) in 3.973243497 sec., 72.48486034582442 rows/sec., 7.51 KiB/sec. 2024.10.22 00:59:00.084878 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} MergedBlockOutputStream: filled checksums 202410_1_244_50 (state Temporary) 2024.10.22 00:59:00.187114 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_505_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_505_505_0, total 75 rows starting from the beginning of the part 2024.10.22 00:59:00.339680 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(217dd5ecfbaf8623e44bf0fdccee2a58) : [0, 159718]: Setting download as failed: File segment: [0, 159718], key: 217dd5ecfbaf8623e44bf0fdccee2a58, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:59:00.353727 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_59_12/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:59:00.341838 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(217dd5ecfbaf8623e44bf0fdccee2a58): No space left in cache to reserve 159719 bytes, will continue without cache download 2024.10.22 00:59:00.582608 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 217dd5ecfbaf8623e44bf0fdccee2a58, offset: 0, size: 0 2024.10.22 00:59:00.583787 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 217dd5ecfbaf8623e44bf0fdccee2a58 for removal 2024.10.22 00:59:00.585176 [ 1476 ] {} CacheMetadata: Key 217dd5ecfbaf8623e44bf0fdccee2a58 is removed from metadata 2024.10.22 00:59:00.781733 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 6.37 MiB, will set to 1.69 GiB (RSS), difference: -11.35 MiB 2024.10.22 00:59:01.095993 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_244_50 to 202410_1_244_50 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:01.098509 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_244_50} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_239_49, 202410_244_244_0] -> 202410_1_244_50 2024.10.22 00:59:01.162499 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:59:01.428870 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 4164 2024.10.22 00:59:01.566752 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(dadaabb0878d51ce3451fb4f4d230c09) : [0, 424156]: Setting download as failed: File segment: [0, 424156], key: dadaabb0878d51ce3451fb4f4d230c09, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:59:01.571539 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(dadaabb0878d51ce3451fb4f4d230c09): No space left in cache to reserve 424157 bytes, will continue without cache download 2024.10.22 00:59:01.594300 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dadaabb0878d51ce3451fb4f4d230c09, offset: 0, size: 0 2024.10.22 00:59:01.595551 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dadaabb0878d51ce3451fb4f4d230c09 for removal 2024.10.22 00:59:01.597409 [ 1476 ] {} CacheMetadata: Key dadaabb0878d51ce3451fb4f4d230c09 is removed from metadata 2024.10.22 00:59:01.693134 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 467116 2024.10.22 00:59:01.896095 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_505_136} MergeTask::MergeProjectionsStage: Merge sorted 29344 rows, containing 13 columns (13 merged, 0 gathered) in 1.977867333 sec., 14836.182139421582 rows/sec., 2.25 MiB/sec. 2024.10.22 00:59:01.960216 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:01.968205 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:02.021758 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_291_291_0 (state Temporary) 2024.10.22 00:59:02.075378 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_505_136} MergedBlockOutputStream: filled checksums 202410_1_505_136 (state Temporary) 2024.10.22 00:59:02.095266 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_291_291_0 to 202410_304_304_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:02.145074 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4164 2024.10.22 00:59:02.481683 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:02.484547 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:02.495886 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(466e1b77499629b90ac4b9109b7691da) : [0, 270261]: Setting download as failed: File segment: [0, 270261], key: 466e1b77499629b90ac4b9109b7691da, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:59:02.498478 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(466e1b77499629b90ac4b9109b7691da): No space left in cache to reserve 270262 bytes, will continue without cache download 2024.10.22 00:59:02.496212 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_505_136} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_505_136 to 202410_1_505_136 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:02.515224 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_505_136} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_504_135, 202410_505_505_0] -> 202410_1_505_136 2024.10.22 00:59:02.553502 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.03 MiB. 2024.10.22 00:59:02.632891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 466e1b77499629b90ac4b9109b7691da, offset: 0, size: 0 2024.10.22 00:59:02.641389 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 466e1b77499629b90ac4b9109b7691da for removal 2024.10.22 00:59:02.645923 [ 1476 ] {} CacheMetadata: Key 466e1b77499629b90ac4b9109b7691da is removed from metadata 2024.10.22 00:59:02.700455 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_278_278_0 (state Temporary) 2024.10.22 00:59:02.871770 [ 1178 ] {} DatabaseCatalog: Execution took 230 ms. 2024.10.22 00:59:02.976068 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_278_278_0 to 202410_285_285_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:03.001321 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 467116 2024.10.22 00:59:03.009345 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_58_58_0 to 202410_61_61_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:03.394508 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 8.95 MiB, will set to 1.68 GiB (RSS), difference: 16.13 MiB 2024.10.22 00:59:03.477405 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 253 2024.10.22 00:59:03.775786 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:03.779790 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:03.928405 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 28855 2024.10.22 00:59:04.084930 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:04.091656 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:04.134965 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_235_235_0 (state Temporary) 2024.10.22 00:59:04.157407 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_497_497_0 (state Temporary) 2024.10.22 00:59:04.175096 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(85b4b096f2a6d4fa8d5bd6d901723167) : [0, 1149304]: Setting download as failed: File segment: [0, 1149304], key: 85b4b096f2a6d4fa8d5bd6d901723167, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:59:04.178543 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(85b4b096f2a6d4fa8d5bd6d901723167): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:59:04.192656 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_497_497_0 to 202410_510_510_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:04.221671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 85b4b096f2a6d4fa8d5bd6d901723167, offset: 0, size: 0 2024.10.22 00:59:04.224462 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_505_136 to 202410_506_506_0 2024.10.22 00:59:04.227087 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 85b4b096f2a6d4fa8d5bd6d901723167 for removal 2024.10.22 00:59:04.229629 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.02 MiB using storage policy from min volume index 0 2024.10.22 00:59:04.231143 [ 1476 ] {} CacheMetadata: Key 85b4b096f2a6d4fa8d5bd6d901723167 is removed from metadata 2024.10.22 00:59:04.237074 [ 1183 ] {} DiskLocal: Reserved 1.02 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:04.291149 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_506_137} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_505_136 to 202410_506_506_0 into Compact with storage Full 2024.10.22 00:59:04.343983 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_235_235_0 to 202410_245_245_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:04.372397 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28855 2024.10.22 00:59:04.489268 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_506_137} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:04.502688 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_506_137} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_505_136, total 29344 rows starting from the beginning of the part 2024.10.22 00:59:04.525378 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 253 2024.10.22 00:59:04.566100 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_506_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_506_506_0, total 49 rows starting from the beginning of the part 2024.10.22 00:59:05.095797 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 101 entries to flush up to offset 20612 2024.10.22 00:59:05.208046 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5222f02ad72b72ea1dba8fa9b92fa6b2) : [0, 1351170]: Setting download as failed: File segment: [0, 1351170], key: 5222f02ad72b72ea1dba8fa9b92fa6b2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:59:05.214504 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5222f02ad72b72ea1dba8fa9b92fa6b2): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 00:59:05.392349 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 8.26 MiB, will set to 1.69 GiB (RSS), difference: -6.95 MiB 2024.10.22 00:59:05.416628 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:05.420173 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:05.489691 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5222f02ad72b72ea1dba8fa9b92fa6b2, offset: 0, size: 0 2024.10.22 00:59:05.491320 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5222f02ad72b72ea1dba8fa9b92fa6b2 for removal 2024.10.22 00:59:05.493110 [ 1476 ] {} CacheMetadata: Key 5222f02ad72b72ea1dba8fa9b92fa6b2 is removed from metadata 2024.10.22 00:59:05.679244 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_289_289_0 (state Temporary) 2024.10.22 00:59:05.729301 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_289_289_0 to 202410_295_295_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:05.853691 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 20612 2024.10.22 00:59:05.978154 [ 1177 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 00:59:06.327363 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2332 2024.10.22 00:59:06.391623 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_506_137} MergeTask::MergeProjectionsStage: Merge sorted 29393 rows, containing 13 columns (13 merged, 0 gathered) in 2.10471193 sec., 13965.331588157054 rows/sec., 2.12 MiB/sec. 2024.10.22 00:59:06.395312 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2d75085e7bb3c32c1acb043465467344) : [0, 454783]: Setting download as failed: File segment: [0, 454783], key: 2d75085e7bb3c32c1acb043465467344, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:59:06.396306 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2d75085e7bb3c32c1acb043465467344): No space left in cache to reserve 454784 bytes, will continue without cache download 2024.10.22 00:59:06.422290 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2d75085e7bb3c32c1acb043465467344, offset: 0, size: 0 2024.10.22 00:59:06.423753 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2d75085e7bb3c32c1acb043465467344 for removal 2024.10.22 00:59:06.460455 [ 1476 ] {} CacheMetadata: Key 2d75085e7bb3c32c1acb043465467344 is removed from metadata 2024.10.22 00:59:06.557393 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_506_137} MergedBlockOutputStream: filled checksums 202410_1_506_137 (state Temporary) 2024.10.22 00:59:06.722346 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_506_137} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_506_137 to 202410_1_506_137 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:06.724304 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_506_137} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_505_136, 202410_506_506_0] -> 202410_1_506_137 2024.10.22 00:59:06.845699 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.03 MiB. 2024.10.22 00:59:07.308965 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(5da4d73e1aeb7c1b02da9df1d42d7149) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5da4d73e1aeb7c1b02da9df1d42d7149, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 00:59:07.313175 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 5da4d73e1aeb7c1b02da9df1d42d7149, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 00:59:07.318528 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:59:07.376567 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 15.80 MiB, will set to 1.70 GiB (RSS), difference: 19.55 MiB 2024.10.22 00:59:07.687853 [ 1728 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tor/spmugwarivyhniwjeplzjzczallfm, size 2618841 2024.10.22 00:59:07.924033 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 00:59:08.005431 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 00:59:08.022954 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 00:59:08.087193 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 00:59:08.234472 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(61f4bc970c2db69d9edd640134d5f027) : [0, 98028]: Setting download as failed: File segment: [0, 98028], key: 61f4bc970c2db69d9edd640134d5f027, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 00:59:08.237209 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(61f4bc970c2db69d9edd640134d5f027): No space left in cache to reserve 98029 bytes, will continue without cache download 2024.10.22 00:59:08.242172 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 00:59:08.278559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 61f4bc970c2db69d9edd640134d5f027, offset: 0, size: 0 2024.10.22 00:59:08.280522 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 61f4bc970c2db69d9edd640134d5f027 for removal 2024.10.22 00:59:08.279549 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 00:59:08.282998 [ 1476 ] {} CacheMetadata: Key 61f4bc970c2db69d9edd640134d5f027 is removed from metadata 2024.10.22 00:59:08.285746 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yoi/otbfrghsffeslnywenpiqwqccgixe. 2024.10.22 00:59:08.380785 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 28917 2024.10.22 00:59:08.440008 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 00:59:08.615203 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pik/jkejfhtakwpxeenqrpcbvzuitdiii (LogSeriesLimiter: on interval from 2024-10-22 00:58:52 to 2024-10-22 00:59:08 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:59:08.881437 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:08.894363 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:08.968053 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_498_498_0 (state Temporary) 2024.10.22 00:59:09.223491 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_498_498_0 to 202410_511_511_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:09.277334 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28917 2024.10.22 00:59:09.346446 [ 1204 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_506_137 to 202410_507_507_0 2024.10.22 00:59:09.353830 [ 1204 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.02 MiB using storage policy from min volume index 0 2024.10.22 00:59:09.357803 [ 1204 ] {} DiskLocal: Reserved 1.02 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:09.389310 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_17_17_0 to 201403_17_17_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:09.392892 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_507_138} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_506_137 to 202410_507_507_0 into Compact with storage Full 2024.10.22 00:59:09.402841 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yoi/otbfrghsffeslnywenpiqwqccgixe. 2024.10.22 00:59:09.428555 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_507_138} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:09.439135 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_507_138} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_506_137, total 29393 rows starting from the beginning of the part 2024.10.22 00:59:09.454244 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 21.34 MiB, will set to 1.68 GiB (RSS), difference: -4.54 MiB 2024.10.22 00:59:09.484223 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_507_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_507_0, total 75 rows starting from the beginning of the part 2024.10.22 00:59:09.553285 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 00:59:09.653180 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 4178 2024.10.22 00:59:09.793490 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:59:09.795847 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:59:09.954741 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:09.960741 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:09.988413 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_292_292_0 (state Temporary) 2024.10.22 00:59:10.004968 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1827 entries to flush up to offset 468943 2024.10.22 00:59:10.136882 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_292_292_0 to 202410_305_305_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:10.190976 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4178 2024.10.22 00:59:10.777009 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:10.779836 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:11.178190 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_241_48, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 00:59:11.179245 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_241_48, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 00:59:11.194213 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_241_48, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 00:59:11.294528 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_279_279_0 (state Temporary) 2024.10.22 00:59:11.370778 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_279_279_0 to 202410_286_286_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:11.375776 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_507_138} MergeTask::MergeProjectionsStage: Merge sorted 29468 rows, containing 13 columns (13 merged, 0 gathered) in 1.987466432 sec., 14826.91708676869 rows/sec., 2.25 MiB/sec. 2024.10.22 00:59:11.392114 [ 1196 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_281_56 to 202410_286_286_0 2024.10.22 00:59:11.394800 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 468943 2024.10.22 00:59:11.401582 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.23 GiB, peak 1.88 GiB, free memory in arenas 15.55 MiB, will set to 1.45 GiB (RSS), difference: 232.82 MiB 2024.10.22 00:59:11.402075 [ 1196 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:11.404348 [ 1196 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:11.528400 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_281_56 to 202410_286_286_0 into Compact with storage Full 2024.10.22 00:59:11.572779 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:11.575517 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_507_138} MergedBlockOutputStream: filled checksums 202410_1_507_138 (state Temporary) 2024.10.22 00:59:11.579779 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} MergeTreeSequentialSource: Reading 60 marks from part 202410_1_281_56, total 475714 rows starting from the beginning of the part 2024.10.22 00:59:11.599199 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_282_282_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:59:11.619962 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_283_283_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:59:11.629786 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_284_284_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:59:11.750907 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_285_285_0, total 1152 rows starting from the beginning of the part 2024.10.22 00:59:11.775337 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_286_286_0, total 1827 rows starting from the beginning of the part 2024.10.22 00:59:11.828938 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_507_138} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_507_138 to 202410_1_507_138 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:11.831608 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_507_138} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_506_137, 202410_507_507_0] -> 202410_1_507_138 2024.10.22 00:59:11.878522 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.03 MiB. 2024.10.22 00:59:12.027659 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 254 2024.10.22 00:59:12.316462 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:12.457505 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.88 GiB, free memory in arenas 15.82 MiB, will set to 1.48 GiB (RSS), difference: -58.84 MiB 2024.10.22 00:59:12.499981 [ 1202 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:59:12.516598 [ 1202 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:59:12.523531 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:12.702828 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_236_236_0 (state Temporary) 2024.10.22 00:59:12.770518 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_407_90, 202410_408_408_0] 2024.10.22 00:59:12.777949 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_407_90, 202410_408_408_0] 2024.10.22 00:59:12.786355 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_407_90, 202410_408_408_0] 2024.10.22 00:59:13.098133 [ 1199 ] {} DatabaseCatalog: Execution took 224 ms. 2024.10.22 00:59:13.271025 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_236_236_0 to 202410_246_246_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:13.281837 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 28981 2024.10.22 00:59:13.375828 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 20727 2024.10.22 00:59:13.498877 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:13.504456 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.03 GiB. 2024.10.22 00:59:13.504989 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:13.508524 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:13.572884 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_499_499_0 (state Temporary) 2024.10.22 00:59:13.574702 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 254 2024.10.22 00:59:13.577897 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_290_290_0 (state Temporary) 2024.10.22 00:59:13.629192 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_499_499_0 to 202410_512_512_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:13.674965 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_507_138 to 202410_508_508_0 2024.10.22 00:59:13.681651 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.02 MiB using storage policy from min volume index 0 2024.10.22 00:59:13.683489 [ 1213 ] {} DiskLocal: Reserved 1.02 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:13.708716 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_508_139} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_507_138 to 202410_508_508_0 into Compact with storage Full 2024.10.22 00:59:13.736346 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 28981 2024.10.22 00:59:13.932799 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_59_12/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 00:59:13.940158 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_290_290_0 to 202410_296_296_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:13.976208 [ 1235 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_291_85 to 202410_292_292_0 2024.10.22 00:59:13.978522 [ 1235 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.14 MiB using storage policy from min volume index 0 2024.10.22 00:59:13.854255 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 2372 2024.10.22 00:59:13.980014 [ 1235 ] {} DiskLocal: Reserved 1.14 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:13.989975 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_508_139} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:13.992260 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 20727 2024.10.22 00:59:13.998100 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_508_139} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_507_138, total 29468 rows starting from the beginning of the part 2024.10.22 00:59:14.125615 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_292_86} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_291_85 to 202410_292_292_0 into Compact with storage Full 2024.10.22 00:59:14.194123 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_508_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_508_508_0, total 85 rows starting from the beginning of the part 2024.10.22 00:59:14.205341 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_292_86} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:14.323378 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_292_86} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_291_85, total 20366 rows starting from the beginning of the part 2024.10.22 00:59:14.421699 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_292_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_292_292_0, total 129 rows starting from the beginning of the part 2024.10.22 00:59:14.489124 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.88 GiB, free memory in arenas 15.14 MiB, will set to 1.53 GiB (RSS), difference: -20.53 MiB 2024.10.22 00:59:15.408089 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RegionID 2024.10.22 00:59:15.427031 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RegionID 2024.10.22 00:59:15.454833 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RegionID 2024.10.22 00:59:15.587893 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RegionID 2024.10.22 00:59:15.592082 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 17 2024.10.22 00:59:15.731369 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RegionID 2024.10.22 00:59:15.844757 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RegionID 2024.10.22 00:59:15.878715 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RegionID 2024.10.22 00:59:15.889689 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(1949be129636de003c4ddef0ff089aa4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1949be129636de003c4ddef0ff089aa4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 00:59:15.892831 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 1949be129636de003c4ddef0ff089aa4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 00:59:15.897370 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:59:15.915901 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RegionID 2024.10.22 00:59:15.970519 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RegionID 2024.10.22 00:59:16.008051 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:16.014965 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:16.150660 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/btf/amvgwphczqiajftivzttflrplrkod (LogSeriesLimiter: on interval from 2024-10-22 00:59:08 to 2024-10-22 00:59:16 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:59:16.192561 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_286_57/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:59:16.204028 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_292_86} MergeTask::MergeProjectionsStage: Merge sorted 20495 rows, containing 13 columns (13 merged, 0 gathered) in 2.089030435 sec., 9810.771378254238 rows/sec., 4.15 MiB/sec. 2024.10.22 00:59:16.209196 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_17_17_0 (state Temporary) 2024.10.22 00:59:16.289186 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_292_86} MergedBlockOutputStream: filled checksums 202410_1_292_86 (state Temporary) 2024.10.22 00:59:16.296750 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_17_17_0 to 202410_17_17_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:16.404249 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_508_139} MergeTask::MergeProjectionsStage: Merge sorted 29553 rows, containing 13 columns (13 merged, 0 gathered) in 2.705957125 sec., 10921.459075224631 rows/sec., 1.66 MiB/sec. 2024.10.22 00:59:16.503275 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 16.32 MiB, will set to 1.59 GiB (RSS), difference: -46.40 MiB 2024.10.22 00:59:16.567803 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 17 2024.10.22 00:59:16.606738 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_508_139} MergedBlockOutputStream: filled checksums 202410_1_508_139 (state Temporary) 2024.10.22 00:59:16.672657 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_292_86} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_292_86 to 202410_1_292_86 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:16.677080 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_292_86} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_291_85, 202410_292_292_0] -> 202410_1_292_86 2024.10.22 00:59:16.707279 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.82 MiB. 2024.10.22 00:59:17.207404 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_508_139} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_508_139 to 202410_1_508_139 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:17.209665 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_508_139} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_507_138, 202410_508_508_0] -> 202410_1_508_139 2024.10.22 00:59:17.248832 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.05 MiB. 2024.10.22 00:59:17.368756 [ 1228 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 00:59:17.692835 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 4193 2024.10.22 00:59:17.752777 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 29045 2024.10.22 00:59:17.896622 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:17.899775 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:17.947927 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_293_293_0 (state Temporary) 2024.10.22 00:59:17.994493 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_293_293_0 to 202410_306_306_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:18.034726 [ 1249 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_301_60 to 202410_306_306_0 2024.10.22 00:59:18.067534 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4193 2024.10.22 00:59:18.081447 [ 1249 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:18.088199 [ 1249 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:18.105598 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:18.109382 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:18.112993 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_301_60 to 202410_306_306_0 into Compact with storage Full 2024.10.22 00:59:18.172177 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:18.180736 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_500_500_0 (state Temporary) 2024.10.22 00:59:18.280833 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_301_60, total 4219 rows starting from the beginning of the part 2024.10.22 00:59:18.301001 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_302_302_0, total 25 rows starting from the beginning of the part 2024.10.22 00:59:18.327926 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_303_303_0, total 13 rows starting from the beginning of the part 2024.10.22 00:59:18.348989 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_500_500_0 to 202410_513_513_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:18.357132 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_304_304_0, total 13 rows starting from the beginning of the part 2024.10.22 00:59:18.390951 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_305_305_0, total 14 rows starting from the beginning of the part 2024.10.22 00:59:18.399218 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1629 entries to flush up to offset 470572 2024.10.22 00:59:18.425682 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_306_306_0, total 15 rows starting from the beginning of the part 2024.10.22 00:59:18.506006 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29045 2024.10.22 00:59:18.541646 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 13.84 MiB, will set to 1.61 GiB (RSS), difference: -29.31 MiB 2024.10.22 00:59:18.599623 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_408_91, 202410_409_409_0] 2024.10.22 00:59:18.604285 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_508_139 to 202410_509_509_0 2024.10.22 00:59:18.604639 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_408_91, 202410_409_409_0] 2024.10.22 00:59:18.609706 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.02 MiB using storage policy from min volume index 0 2024.10.22 00:59:18.618781 [ 1253 ] {} DiskLocal: Reserved 1.02 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:18.675072 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_509_140} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_508_139 to 202410_509_509_0 into Compact with storage Full 2024.10.22 00:59:18.725653 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_408_91, 202410_409_409_0] 2024.10.22 00:59:18.727853 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_509_140} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:18.740311 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_509_140} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_508_139, total 29553 rows starting from the beginning of the part 2024.10.22 00:59:18.814220 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_509_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_509_509_0, total 67 rows starting from the beginning of the part 2024.10.22 00:59:19.275858 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} MergeTask::MergeProjectionsStage: Merge sorted 4299 rows, containing 12 columns (12 merged, 0 gathered) in 1.176801131 sec., 3653.123613457846 rows/sec., 572.61 KiB/sec. 2024.10.22 00:59:19.331662 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} MergedBlockOutputStream: filled checksums 202410_1_306_61 (state Temporary) 2024.10.22 00:59:19.398555 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:19.404110 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:19.724022 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_280_280_0 (state Temporary) 2024.10.22 00:59:19.792852 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:59:19.796308 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:59:19.798464 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_306_61 to 202410_1_306_61 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:19.802206 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_306_61} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_301_60, 202410_306_306_0] -> 202410_1_306_61 2024.10.22 00:59:19.834447 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_280_280_0 to 202410_287_287_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:19.851720 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.45 MiB. 2024.10.22 00:59:19.987313 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 470572 2024.10.22 00:59:20.839042 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.88 GiB, free memory in arenas 15.02 MiB, will set to 1.67 GiB (RSS), difference: -123.15 MiB 2024.10.22 00:59:21.077807 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 255 2024.10.22 00:59:21.281013 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:21.284574 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:21.289290 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_509_140} MergeTask::MergeProjectionsStage: Merge sorted 29620 rows, containing 13 columns (13 merged, 0 gathered) in 2.620995965 sec., 11301.047539002982 rows/sec., 1.72 MiB/sec. 2024.10.22 00:59:21.513603 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 154 entries to flush up to offset 20881 2024.10.22 00:59:21.561811 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_237_237_0 (state Temporary) 2024.10.22 00:59:21.581165 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_509_140} MergedBlockOutputStream: filled checksums 202410_1_509_140 (state Temporary) 2024.10.22 00:59:21.741147 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_509_140} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_509_140 to 202410_1_509_140 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:21.746394 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_509_140} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_508_139, 202410_509_509_0] -> 202410_1_509_140 2024.10.22 00:59:21.795510 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.04 MiB. 2024.10.22 00:59:21.847429 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:21.863607 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:21.871225 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_237_237_0 to 202410_247_247_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:21.909542 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_291_291_0 (state Temporary) 2024.10.22 00:59:22.036989 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 255 2024.10.22 00:59:22.095430 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_291_291_0 to 202410_297_297_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:22.129299 [ 1164 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_292_86 to 202410_293_293_0 2024.10.22 00:59:22.136318 [ 1164 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.14 MiB using storage policy from min volume index 0 2024.10.22 00:59:22.140389 [ 1164 ] {} DiskLocal: Reserved 1.14 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:22.185321 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 20881 2024.10.22 00:59:22.184556 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_293_87} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_292_86 to 202410_293_293_0 into Compact with storage Full 2024.10.22 00:59:22.331092 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_293_87} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:22.358335 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_293_87} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_292_86, total 20495 rows starting from the beginning of the part 2024.10.22 00:59:22.414743 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_293_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_293_293_0, total 67 rows starting from the beginning of the part 2024.10.22 00:59:22.610774 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 29116 2024.10.22 00:59:22.671725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 268.60 MiB using storage policy from min volume index 0 2024.10.22 00:59:22.968114 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 11.71 MiB, will set to 1.67 GiB (RSS), difference: 20.39 MiB 2024.10.22 00:59:23.254208 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:23.269782 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:23.328863 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yss/kdtvhnottettweeojkbzzirdfltgy (LogSeriesLimiter: on interval from 2024-10-22 00:59:16 to 2024-10-22 00:59:23 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:59:23.463449 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_501_501_0 (state Temporary) 2024.10.22 00:59:23.482988 [ 1241 ] {} DatabaseCatalog: Execution took 373 ms. 2024.10.22 00:59:23.527518 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_501_501_0 to 202410_514_514_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:23.571579 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_509_140 to 202410_510_510_0 2024.10.22 00:59:23.588485 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 00:59:23.590150 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29116 2024.10.22 00:59:23.592576 [ 1168 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:23.727441 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_510_141} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_509_140 to 202410_510_510_0 into Compact with storage Full 2024.10.22 00:59:23.826115 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_510_141} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:23.839069 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_510_141} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_509_140, total 29620 rows starting from the beginning of the part 2024.10.22 00:59:23.900697 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_293_87} MergeTask::MergeProjectionsStage: Merge sorted 20562 rows, containing 13 columns (13 merged, 0 gathered) in 1.704212602 sec., 12065.396052035532 rows/sec., 5.10 MiB/sec. 2024.10.22 00:59:24.126491 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_510_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_510_510_0, total 66 rows starting from the beginning of the part 2024.10.22 00:59:24.190275 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_293_87} MergedBlockOutputStream: filled checksums 202410_1_293_87 (state Temporary) 2024.10.22 00:59:24.533141 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.88 GiB, free memory in arenas 10.49 MiB, will set to 1.68 GiB (RSS), difference: -102.28 MiB 2024.10.22 00:59:24.677128 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_59_12/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:59:24.684875 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_293_87} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_293_87 to 202410_1_293_87 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:24.695187 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_293_87} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_292_86, 202410_293_293_0] -> 202410_1_293_87 2024.10.22 00:59:24.806315 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_409_92, 202410_410_410_0] 2024.10.22 00:59:24.811054 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_409_92, 202410_410_410_0] 2024.10.22 00:59:24.825462 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_409_92, 202410_410_410_0] 2024.10.22 00:59:24.933593 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.79 MiB. 2024.10.22 00:59:25.292846 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 867 bytes for fc962699ae5f0ab2c64f629bc3d688ca:0) 2024.10.22 00:59:25.296915 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8aad256204ce9fe01db0ea829aa25a80, offset: 0, size: 153 2024.10.22 00:59:25.306941 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8aad256204ce9fe01db0ea829aa25a80 for removal 2024.10.22 00:59:25.315598 [ 1476 ] {} CacheMetadata: Key 8aad256204ce9fe01db0ea829aa25a80 is removed from metadata 2024.10.22 00:59:25.315978 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a40b9c4ed1fbe7502b980bb35995acdc, offset: 0, size: 152 2024.10.22 00:59:25.321162 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a40b9c4ed1fbe7502b980bb35995acdc for removal 2024.10.22 00:59:25.326765 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dd8aedd29e4fe8e286a6573dd9be991f, offset: 0, size: 155 2024.10.22 00:59:25.338365 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dd8aedd29e4fe8e286a6573dd9be991f for removal 2024.10.22 00:59:25.345730 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b9142f48402984ed8ee6b21ab8933192, offset: 0, size: 166 2024.10.22 00:59:25.355055 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b9142f48402984ed8ee6b21ab8933192 for removal 2024.10.22 00:59:25.355055 [ 1476 ] {} CacheMetadata: Key dd8aedd29e4fe8e286a6573dd9be991f is removed from metadata 2024.10.22 00:59:25.361233 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d1f5007d50d11b160ce99ade9dcd8979, offset: 0, size: 4 2024.10.22 00:59:25.367143 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d1f5007d50d11b160ce99ade9dcd8979 for removal 2024.10.22 00:59:25.368117 [ 1476 ] {} CacheMetadata: Key b9142f48402984ed8ee6b21ab8933192 is removed from metadata 2024.10.22 00:59:25.377151 [ 1476 ] {} CacheMetadata: Key d1f5007d50d11b160ce99ade9dcd8979 is removed from metadata 2024.10.22 00:59:25.383130 [ 1476 ] {} CacheMetadata: Key a40b9c4ed1fbe7502b980bb35995acdc is removed from metadata 2024.10.22 00:59:25.567477 [ 1673 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pik/jkejfhtakwpxeenqrpcbvzuitdiii, size 2648463 2024.10.22 00:59:25.571576 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 4212 2024.10.22 00:59:25.680817 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 00:59:25.694161 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:25.799130 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:25.884378 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_294_294_0 (state Temporary) 2024.10.22 00:59:25.929718 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_294_294_0 to 202410_307_307_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:25.992667 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4212 2024.10.22 00:59:26.106492 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column WindowName 2024.10.22 00:59:26.127071 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column WindowName 2024.10.22 00:59:26.156642 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column WindowName 2024.10.22 00:59:26.177341 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column WindowName 2024.10.22 00:59:26.191400 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column WindowName 2024.10.22 00:59:26.203877 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column WindowName 2024.10.22 00:59:26.226409 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column WindowName 2024.10.22 00:59:26.267758 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_510_141} MergeTask::MergeProjectionsStage: Merge sorted 29686 rows, containing 13 columns (13 merged, 0 gathered) in 2.666578997 sec., 11132.615997275105 rows/sec., 1.69 MiB/sec. 2024.10.22 00:59:26.561466 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_510_141} MergedBlockOutputStream: filled checksums 202410_1_510_141 (state Temporary) 2024.10.22 00:59:26.630059 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.88 GiB, free memory in arenas 11.49 MiB, will set to 1.68 GiB (RSS), difference: -120.02 MiB 2024.10.22 00:59:26.762746 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_510_141} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_510_141 to 202410_1_510_141 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:26.767008 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_510_141} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_509_140, 202410_510_510_0] -> 202410_1_510_141 2024.10.22 00:59:26.788074 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.05 MiB. 2024.10.22 00:59:26.863893 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_286_57/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:59:27.084698 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1599 entries to flush up to offset 472171 2024.10.22 00:59:27.526393 [ 1150 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:59:27.568468 [ 1150 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:59:27.598358 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 29184 2024.10.22 00:59:27.748523 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:27.752471 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:27.792819 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_502_502_0 (state Temporary) 2024.10.22 00:59:28.018368 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/usw/krbsjxieopnvczcmybiaibkuabzgp (LogSeriesLimiter: on interval from 2024-10-22 00:59:23 to 2024-10-22 00:59:28 accepted series 1 / 238 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:59:28.052297 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_502_502_0 to 202410_515_515_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:28.056544 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:28.063211 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:28.084923 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_510_141 to 202410_511_511_0 2024.10.22 00:59:28.092172 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 00:59:28.096737 [ 1266 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:28.120670 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_511_142} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_510_141 to 202410_511_511_0 into Compact with storage Full 2024.10.22 00:59:28.198838 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29184 2024.10.22 00:59:28.205607 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_511_142} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:28.198767 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_194_40, 202410_195_195_0, 202410_196_196_0, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0] 2024.10.22 00:59:28.213640 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_194_40, 202410_195_195_0, 202410_196_196_0, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0] 2024.10.22 00:59:28.285349 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_281_281_0 (state Temporary) 2024.10.22 00:59:28.286277 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_511_142} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_510_141, total 29686 rows starting from the beginning of the part 2024.10.22 00:59:28.345403 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_194_40, 202410_195_195_0, 202410_196_196_0, 202410_197_197_0, 202410_198_198_0, 202410_199_199_0] 2024.10.22 00:59:28.482267 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_511_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_511_511_0, total 62 rows starting from the beginning of the part 2024.10.22 00:59:28.525154 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_281_281_0 to 202410_288_288_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:28.588476 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.83 GiB, peak 1.88 GiB, free memory in arenas 12.42 MiB, will set to 1.68 GiB (RSS), difference: -158.61 MiB 2024.10.22 00:59:28.665151 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 472171 2024.10.22 00:59:28.791303 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_410_93, 202410_411_411_0, 202410_412_412_0, 202410_413_413_0, 202410_414_414_0, 202410_415_415_0] 2024.10.22 00:59:28.792977 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_410_93, 202410_411_411_0, 202410_412_412_0, 202410_413_413_0, 202410_414_414_0, 202410_415_415_0] 2024.10.22 00:59:28.857073 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_410_93, 202410_411_411_0, 202410_412_412_0, 202410_413_413_0, 202410_414_414_0, 202410_415_415_0] 2024.10.22 00:59:29.495908 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} MergeTask::MergeProjectionsStage: Merge sorted 482533 rows, containing 4 columns (4 merged, 0 gathered) in 18.085382895 sec., 26680.828534374254 rows/sec., 391.20 KiB/sec. 2024.10.22 00:59:29.661102 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} MergedBlockOutputStream: filled checksums 202410_1_286_57 (state Temporary) 2024.10.22 00:59:29.705070 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 135 entries to flush up to offset 21016 2024.10.22 00:59:29.792525 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:59:29.793493 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:59:29.842736 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_286_57 to 202410_1_286_57 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:29.847804 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_286_57} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_281_56, 202410_286_286_0] -> 202410_1_286_57 2024.10.22 00:59:29.894547 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 00:59:30.059872 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:30.065584 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:30.096997 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_292_292_0 (state Temporary) 2024.10.22 00:59:30.105532 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_511_142/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 00:59:30.283530 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_292_292_0 to 202410_298_298_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:30.308698 [ 1203 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_293_87 to 202410_294_294_0 2024.10.22 00:59:30.310196 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:30.318260 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:30.328706 [ 1203 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.14 MiB using storage policy from min volume index 0 2024.10.22 00:59:30.365469 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 21016 2024.10.22 00:59:30.366698 [ 1203 ] {} DiskLocal: Reserved 1.14 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:30.387604 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_294_88} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_293_87 to 202410_294_294_0 into Compact with storage Full 2024.10.22 00:59:30.477063 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_294_88} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:30.495929 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_294_88} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_293_87, total 20562 rows starting from the beginning of the part 2024.10.22 00:59:30.529015 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_294_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_294_294_0, total 53 rows starting from the beginning of the part 2024.10.22 00:59:30.587322 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.83 GiB, peak 1.88 GiB, free memory in arenas 10.80 MiB, will set to 1.68 GiB (RSS), difference: -153.25 MiB 2024.10.22 00:59:30.627689 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d4518825df9b0bc95fa7d8bf9b6edb46) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d4518825df9b0bc95fa7d8bf9b6edb46, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 00:59:30.631703 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: d4518825df9b0bc95fa7d8bf9b6edb46, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 00:59:30.637577 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:59:30.787766 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_511_142} MergeTask::MergeProjectionsStage: Merge sorted 29748 rows, containing 13 columns (13 merged, 0 gathered) in 2.684564972 sec., 11081.124990555825 rows/sec., 1.69 MiB/sec. 2024.10.22 00:59:30.950317 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_511_142} MergedBlockOutputStream: filled checksums 202410_1_511_142 (state Temporary) 2024.10.22 00:59:31.119444 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_511_142} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_511_142 to 202410_1_511_142 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:31.125022 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_511_142} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_510_141, 202410_511_511_0] -> 202410_1_511_142 2024.10.22 00:59:31.171223 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.04 MiB. 2024.10.22 00:59:32.184990 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_294_88} MergeTask::MergeProjectionsStage: Merge sorted 20615 rows, containing 13 columns (13 merged, 0 gathered) in 1.800528745 sec., 11449.41454405939 rows/sec., 4.84 MiB/sec. 2024.10.22 00:59:32.207648 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 29243 2024.10.22 00:59:32.390273 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:32.393593 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:32.471853 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_294_88} MergedBlockOutputStream: filled checksums 202410_1_294_88 (state Temporary) 2024.10.22 00:59:32.488499 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_503_503_0 (state Temporary) 2024.10.22 00:59:32.534445 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_503_503_0 to 202410_516_516_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:32.553400 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_511_142 to 202410_512_512_0 2024.10.22 00:59:32.564304 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 00:59:32.573079 [ 1209 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 298.02 GiB. 2024.10.22 00:59:32.683148 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29243 2024.10.22 00:59:32.683517 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 9.66 MiB, will set to 1.69 GiB (RSS), difference: -1.97 MiB 2024.10.22 00:59:32.699555 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_512_143} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_511_142 to 202410_512_512_0 into Compact with storage Full 2024.10.22 00:59:32.767264 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_512_143} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:32.779938 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_294_88} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_294_88 to 202410_1_294_88 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:32.781106 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_512_143} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_511_142, total 29748 rows starting from the beginning of the part 2024.10.22 00:59:32.782860 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_294_88} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_293_87, 202410_294_294_0] -> 202410_1_294_88 2024.10.22 00:59:32.797593 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.07 MiB. 2024.10.22 00:59:32.961100 [ 1213 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Selected 5 parts from 202410_1_13_3 to 202410_17_17_0 2024.10.22 00:59:32.975489 [ 1213 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:32.979759 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:32.995058 [ 1389 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_17_4} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_13_3 to 202410_17_17_0 into Compact with storage Full 2024.10.22 00:59:33.013539 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_512_143} MergeTreeSequentialSource: Reading 2 marks from part 202410_512_512_0, total 64 rows starting from the beginning of the part 2024.10.22 00:59:33.190915 [ 1389 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_17_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:33.204101 [ 1389 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_17_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_13_3, total 13 rows starting from the beginning of the part 2024.10.22 00:59:33.258980 [ 1222 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 00:59:33.430500 [ 1389 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_17_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_14_14_0, total 1 rows starting from the beginning of the part 2024.10.22 00:59:33.495266 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 4233 2024.10.22 00:59:33.680311 [ 1389 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_17_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_15_15_0, total 1 rows starting from the beginning of the part 2024.10.22 00:59:33.728235 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:33.733074 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:33.778115 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_59_59_0 (state Temporary) 2024.10.22 00:59:33.784989 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_295_295_0 (state Temporary) 2024.10.22 00:59:33.872019 [ 1389 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_17_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_16_16_0, total 1 rows starting from the beginning of the part 2024.10.22 00:59:33.940759 [ 1389 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_17_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_17_17_0, total 1 rows starting from the beginning of the part 2024.10.22 00:59:34.027928 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_295_295_0 to 202410_308_308_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:34.079264 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4233 2024.10.22 00:59:34.303838 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 7.18 MiB, will set to 1.69 GiB (RSS), difference: -9.24 MiB 2024.10.22 00:59:34.384149 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_235_56, 202410_236_236_0, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0] 2024.10.22 00:59:34.386435 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_235_56, 202410_236_236_0, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0] 2024.10.22 00:59:34.518579 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_235_56, 202410_236_236_0, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0] 2024.10.22 00:59:34.927419 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} MergeTask::MergeProjectionsStage: Merge sorted 2315 rows, containing 679 columns (679 merged, 0 gathered) in 108.75242433 sec., 21.286881779989834 rows/sec., 28.30 KiB/sec. 2024.10.22 00:59:35.383492 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_512_143} MergeTask::MergeProjectionsStage: Merge sorted 29812 rows, containing 13 columns (13 merged, 0 gathered) in 2.694370069 sec., 11064.552840384154 rows/sec., 1.68 MiB/sec. 2024.10.22 00:59:35.391336 [ 1391 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_17_4} MergeTask::MergeProjectionsStage: Merge sorted 17 rows, containing 27 columns (27 merged, 0 gathered) in 2.403694042 sec., 7.072447534069313 rows/sec., 22.96 KiB/sec. 2024.10.22 00:59:35.431084 [ 1397 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_17_4} MergedBlockOutputStream: filled checksums 202410_1_17_4 (state Temporary) 2024.10.22 00:59:35.669307 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1857 entries to flush up to offset 474028 2024.10.22 00:59:35.673782 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_512_143} MergedBlockOutputStream: filled checksums 202410_1_512_143 (state Temporary) 2024.10.22 00:59:36.349764 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_512_143} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_512_143 to 202410_1_512_143 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:36.362349 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_512_143} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_511_142, 202410_512_512_0] -> 202410_1_512_143 2024.10.22 00:59:36.415808 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.05 MiB. 2024.10.22 00:59:36.633759 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 5.80 MiB, will set to 1.70 GiB (RSS), difference: -2.05 MiB 2024.10.22 00:59:36.736960 [ 1398 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_17_4} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_merge_202410_1_17_4 to 202410_1_17_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:36.792005 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 29293 2024.10.22 00:59:36.794244 [ 1398 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_17_4} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Merged 5 parts: [202410_1_13_3, 202410_17_17_0] -> 202410_1_17_4 2024.10.22 00:59:36.839463 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.24 MiB. 2024.10.22 00:59:36.885172 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:36.887283 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:36.930274 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:36.935540 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:36.985400 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} MergedBlockOutputStream: filled checksums 202410_1_59_12 (state Temporary) 2024.10.22 00:59:36.992890 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_504_504_0 (state Temporary) 2024.10.22 00:59:37.040858 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 256 2024.10.22 00:59:37.145000 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_504_504_0 to 202410_517_517_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:37.183217 [ 1232 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_512_143 to 202410_513_513_0 2024.10.22 00:59:37.198908 [ 1232 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 00:59:37.204382 [ 1232 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:37.237493 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_513_144} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_512_143 to 202410_513_513_0 into Compact with storage Full 2024.10.22 00:59:37.301347 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29293 2024.10.22 00:59:37.408934 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:37.412532 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_513_144} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:37.419433 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:37.423432 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_513_144} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_512_143, total 29812 rows starting from the beginning of the part 2024.10.22 00:59:37.454377 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_513_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_513_513_0, total 64 rows starting from the beginning of the part 2024.10.22 00:59:37.492763 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_282_282_0 (state Temporary) 2024.10.22 00:59:37.592192 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_238_238_0 (state Temporary) 2024.10.22 00:59:37.708324 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_282_282_0 to 202410_289_289_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:37.765132 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 474028 2024.10.22 00:59:37.989174 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_238_238_0 to 202410_248_248_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:38.084027 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 256 2024.10.22 00:59:38.130875 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 140 entries to flush up to offset 21156 2024.10.22 00:59:38.195018 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1014 bytes for 0a38fc00cb2c2261d2192578b0673ad4:0) 2024.10.22 00:59:38.199743 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 53b49ea614a596a87af989666db21f93, offset: 0, size: 159 2024.10.22 00:59:38.209128 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 53b49ea614a596a87af989666db21f93 for removal 2024.10.22 00:59:38.219210 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 24096cfd3a330a63a71b53acc941065c, offset: 0, size: 156 2024.10.22 00:59:38.220251 [ 1476 ] {} CacheMetadata: Key 53b49ea614a596a87af989666db21f93 is removed from metadata 2024.10.22 00:59:38.224177 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 24096cfd3a330a63a71b53acc941065c for removal 2024.10.22 00:59:38.230530 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: fccc94ea822b304b9e59d97ead17ce1d, offset: 0, size: 157 2024.10.22 00:59:38.236069 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key fccc94ea822b304b9e59d97ead17ce1d for removal 2024.10.22 00:59:38.240520 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 871d0d7069cd88f9000053b0405a78ed, offset: 0, size: 152 2024.10.22 00:59:38.251648 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 871d0d7069cd88f9000053b0405a78ed for removal 2024.10.22 00:59:38.260181 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 24959969a3b345222dc980f8570e52b9, offset: 0, size: 155 2024.10.22 00:59:38.267516 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 24959969a3b345222dc980f8570e52b9 for removal 2024.10.22 00:59:38.272062 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8cb8959c69cc28f85de7e55ff3be824f, offset: 0, size: 159 2024.10.22 00:59:38.276576 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8cb8959c69cc28f85de7e55ff3be824f for removal 2024.10.22 00:59:38.355389 [ 1476 ] {} CacheMetadata: Key 8cb8959c69cc28f85de7e55ff3be824f is removed from metadata 2024.10.22 00:59:38.371368 [ 1476 ] {} CacheMetadata: Key 24959969a3b345222dc980f8570e52b9 is removed from metadata 2024.10.22 00:59:38.439811 [ 1692 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/btf/amvgwphczqiajftivzttflrplrkod, size 1890901 2024.10.22 00:59:38.481425 [ 1476 ] {} CacheMetadata: Key 871d0d7069cd88f9000053b0405a78ed is removed from metadata 2024.10.22 00:59:38.571419 [ 1476 ] {} CacheMetadata: Key fccc94ea822b304b9e59d97ead17ce1d is removed from metadata 2024.10.22 00:59:38.581595 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:38.582674 [ 1476 ] {} CacheMetadata: Key 24096cfd3a330a63a71b53acc941065c is removed from metadata 2024.10.22 00:59:38.586409 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:38.599018 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 7.74 MiB, will set to 1.70 GiB (RSS), difference: 6.85 MiB 2024.10.22 00:59:38.637648 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_293_293_0 (state Temporary) 2024.10.22 00:59:38.834557 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_293_293_0 to 202410_299_299_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:38.835105 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:59:38.845345 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:59:38.864633 [ 1251 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_294_88 to 202410_295_295_0 2024.10.22 00:59:38.873145 [ 1251 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.15 MiB using storage policy from min volume index 0 2024.10.22 00:59:38.876667 [ 1251 ] {} DiskLocal: Reserved 1.15 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:38.880924 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 21156 2024.10.22 00:59:39.070860 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:59:39.075719 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_295_89} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_294_88 to 202410_295_295_0 into Compact with storage Full 2024.10.22 00:59:39.086847 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:59:39.100203 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:59:39.145580 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(5489d8815b525c3c9e94d1f1f58314af) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5489d8815b525c3c9e94d1f1f58314af, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 00:59:39.153357 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 5489d8815b525c3c9e94d1f1f58314af, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 00:59:39.163269 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:59:39.206178 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:59:39.233598 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:59:39.265656 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:59:39.289124 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column CounterClass 2024.10.22 00:59:39.293710 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_295_89} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:39.331125 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bfq/zdrneonesomalqwcqxfogkrsbduny (LogSeriesLimiter: on interval from 2024-10-22 00:59:28 to 2024-10-22 00:59:39 accepted series 1 / 121 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:59:39.346964 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_513_144} MergeTask::MergeProjectionsStage: Merge sorted 29876 rows, containing 13 columns (13 merged, 0 gathered) in 2.117618831 sec., 14108.299171995792 rows/sec., 2.15 MiB/sec. 2024.10.22 00:59:39.359047 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_295_89} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_294_88, total 20615 rows starting from the beginning of the part 2024.10.22 00:59:39.479791 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_295_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_295_295_0, total 101 rows starting from the beginning of the part 2024.10.22 00:59:39.779380 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_513_144} MergedBlockOutputStream: filled checksums 202410_1_513_144 (state Temporary) 2024.10.22 00:59:39.792781 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:59:39.887000 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:59:39.888548 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_513_144} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_513_144 to 202410_1_513_144 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:39.892896 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_513_144} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_512_143, 202410_513_513_0] -> 202410_1_513_144 2024.10.22 00:59:39.931151 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.05 MiB. 2024.10.22 00:59:40.518259 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 7.22 MiB, will set to 1.71 GiB (RSS), difference: -9.77 MiB 2024.10.22 00:59:41.301228 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_295_89} MergeTask::MergeProjectionsStage: Merge sorted 20716 rows, containing 13 columns (13 merged, 0 gathered) in 2.399537447 sec., 8633.33057206504 rows/sec., 3.65 MiB/sec. 2024.10.22 00:59:41.322789 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 29377 2024.10.22 00:59:41.482999 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_295_89} MergedBlockOutputStream: filled checksums 202410_1_295_89 (state Temporary) 2024.10.22 00:59:41.545563 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_226_45, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0, 202410_230_230_0, 202410_231_231_0] 2024.10.22 00:59:41.553076 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_226_45, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0, 202410_230_230_0, 202410_231_231_0] 2024.10.22 00:59:41.584108 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 4250 2024.10.22 00:59:41.584933 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_226_45, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0, 202410_230_230_0, 202410_231_231_0] 2024.10.22 00:59:41.699287 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:41.702417 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:41.708511 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:41.714548 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:41.771033 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_296_296_0 (state Temporary) 2024.10.22 00:59:41.778560 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_295_89} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_295_89 to 202410_1_295_89 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:41.782846 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_295_89} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_294_88, 202410_295_295_0] -> 202410_1_295_89 2024.10.22 00:59:41.789126 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_505_505_0 (state Temporary) 2024.10.22 00:59:41.839849 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_296_296_0 to 202410_309_309_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:41.851243 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.11 MiB. 2024.10.22 00:59:41.886253 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4250 2024.10.22 00:59:41.942661 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_505_505_0 to 202410_518_518_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:41.984642 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_513_144 to 202410_514_514_0 2024.10.22 00:59:41.988257 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 00:59:41.994623 [ 1173 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:42.001321 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29377 2024.10.22 00:59:42.019489 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_514_145} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_513_144 to 202410_514_514_0 into Compact with storage Full 2024.10.22 00:59:42.071814 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_514_145} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:42.076116 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_514_145} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_513_144, total 29876 rows starting from the beginning of the part 2024.10.22 00:59:42.090773 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_514_145} MergeTreeSequentialSource: Reading 2 marks from part 202410_514_514_0, total 71 rows starting from the beginning of the part 2024.10.22 00:59:42.188575 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_59_12/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 00:59:42.556081 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 5.13 MiB, will set to 1.71 GiB (RSS), difference: 7.20 MiB 2024.10.22 00:59:42.575137 [ 1168 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:59:42.607272 [ 1168 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:59:43.325451 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_514_145} MergeTask::MergeProjectionsStage: Merge sorted 29947 rows, containing 13 columns (13 merged, 0 gathered) in 1.313632805 sec., 22797.085978680323 rows/sec., 3.47 MiB/sec. 2024.10.22 00:59:43.501168 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_514_145} MergedBlockOutputStream: filled checksums 202410_1_514_145 (state Temporary) 2024.10.22 00:59:43.592211 [ 1163 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 00:59:43.609337 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_514_145} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_514_145 to 202410_1_514_145 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:43.615062 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_514_145} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_513_144, 202410_514_514_0] -> 202410_1_514_145 2024.10.22 00:59:43.738944 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.04 MiB. 2024.10.22 00:59:44.487168 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 4.23 MiB, will set to 1.71 GiB (RSS), difference: 12.97 MiB 2024.10.22 00:59:44.532721 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_59_59_0 to 202410_62_62_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:44.614558 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 899 bytes for 71541bcee26be8abce10fc6fc69e1a16:0) 2024.10.22 00:59:44.617534 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8cc93950b83cb0c93f57f869ddbe2706, offset: 0, size: 159 2024.10.22 00:59:44.634534 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8cc93950b83cb0c93f57f869ddbe2706 for removal 2024.10.22 00:59:44.639130 [ 1476 ] {} CacheMetadata: Key 8cc93950b83cb0c93f57f869ddbe2706 is removed from metadata 2024.10.22 00:59:44.640351 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: db492294bc19323d864cba7170db7f41, offset: 0, size: 156 2024.10.22 00:59:44.643870 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key db492294bc19323d864cba7170db7f41 for removal 2024.10.22 00:59:44.651027 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3ed086f70a447ff0b2639ae9073286b0, offset: 0, size: 150 2024.10.22 00:59:44.655548 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3ed086f70a447ff0b2639ae9073286b0 for removal 2024.10.22 00:59:44.659589 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b3ff6cb37fb258df46b8a62e080db938, offset: 0, size: 154 2024.10.22 00:59:44.663321 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b3ff6cb37fb258df46b8a62e080db938 for removal 2024.10.22 00:59:44.670675 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 79e79a4629f5aeb08ca0e62fb69a39fb, offset: 0, size: 150 2024.10.22 00:59:44.675719 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 79e79a4629f5aeb08ca0e62fb69a39fb for removal 2024.10.22 00:59:44.678608 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 84644851ae6a2e3bdc9371e4f13bb1fb, offset: 0, size: 150 2024.10.22 00:59:44.680999 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 84644851ae6a2e3bdc9371e4f13bb1fb for removal 2024.10.22 00:59:44.761301 [ 1476 ] {} CacheMetadata: Key 84644851ae6a2e3bdc9371e4f13bb1fb is removed from metadata 2024.10.22 00:59:44.771569 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 475564 2024.10.22 00:59:44.774907 [ 1476 ] {} CacheMetadata: Key b3ff6cb37fb258df46b8a62e080db938 is removed from metadata 2024.10.22 00:59:44.784014 [ 1476 ] {} CacheMetadata: Key 79e79a4629f5aeb08ca0e62fb69a39fb is removed from metadata 2024.10.22 00:59:44.793294 [ 1476 ] {} CacheMetadata: Key 3ed086f70a447ff0b2639ae9073286b0 is removed from metadata 2024.10.22 00:59:44.825972 [ 1476 ] {} CacheMetadata: Key db492294bc19323d864cba7170db7f41 is removed from metadata 2024.10.22 00:59:45.180165 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OpenerName 2024.10.22 00:59:45.182913 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OpenerName 2024.10.22 00:59:45.186005 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OpenerName 2024.10.22 00:59:45.196514 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OpenerName 2024.10.22 00:59:45.222598 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OpenerName 2024.10.22 00:59:45.240929 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OpenerName 2024.10.22 00:59:45.269326 [ 1188 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 252 ms. 2024.10.22 00:59:45.284458 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OpenerName 2024.10.22 00:59:45.326851 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nna/zktxqgzbamiffvxegndvzqogdrojb (LogSeriesLimiter: on interval from 2024-10-22 00:59:39 to 2024-10-22 00:59:45 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:59:45.586310 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 257 2024.10.22 00:59:45.708512 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:45.711224 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:45.927354 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_283_283_0 (state Temporary) 2024.10.22 00:59:45.987112 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:45.989037 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:46.018375 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 29450 2024.10.22 00:59:46.048377 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_239_239_0 (state Temporary) 2024.10.22 00:59:46.111236 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_283_283_0 to 202410_290_290_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:46.221390 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_239_239_0 to 202410_249_249_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:46.365127 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 475564 2024.10.22 00:59:46.395822 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:46.464792 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.01 GiB. 2024.10.22 00:59:46.487130 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 116 entries to flush up to offset 21272 2024.10.22 00:59:46.507366 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_506_506_0 (state Temporary) 2024.10.22 00:59:46.574133 [ 1161 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_244_50 to 202410_249_249_0 2024.10.22 00:59:46.660087 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 257 2024.10.22 00:59:46.677072 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_506_506_0 to 202410_519_519_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:46.729439 [ 1161 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:46.730441 [ 1274 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_514_145 to 202410_515_515_0 2024.10.22 00:59:46.732743 [ 1161 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.00 GiB. 2024.10.22 00:59:46.740399 [ 1274 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.04 MiB using storage policy from min volume index 0 2024.10.22 00:59:46.743560 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29450 2024.10.22 00:59:46.747186 [ 1274 ] {} DiskLocal: Reserved 1.04 MiB on local disk `default`, having unreserved 298.00 GiB. 2024.10.22 00:59:46.753756 [ 1161 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 398 ms. 2024.10.22 00:59:46.913525 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_515_146} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_514_145 to 202410_515_515_0 into Compact with storage Full 2024.10.22 00:59:46.915662 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_244_50 to 202410_249_249_0 into Compact with storage Full 2024.10.22 00:59:46.984776 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:46.993117 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 298.00 GiB. 2024.10.22 00:59:47.009136 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 6.23 MiB, will set to 1.71 GiB (RSS), difference: 76.72 MiB 2024.10.22 00:59:47.075408 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_294_294_0 (state Temporary) 2024.10.22 00:59:47.211927 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_515_146} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:47.232354 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_294_294_0 to 202410_300_300_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:47.234956 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_515_146} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_514_145, total 29947 rows starting from the beginning of the part 2024.10.22 00:59:47.281450 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_515_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_515_515_0, total 68 rows starting from the beginning of the part 2024.10.22 00:59:47.291574 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2372 2024.10.22 00:59:47.356680 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 21272 2024.10.22 00:59:47.383039 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:47.395828 [ 1211 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_295_89 to 202410_296_296_0 2024.10.22 00:59:47.400294 [ 1211 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.16 MiB using storage policy from min volume index 0 2024.10.22 00:59:47.403062 [ 1211 ] {} DiskLocal: Reserved 1.16 MiB on local disk `default`, having unreserved 298.00 GiB. 2024.10.22 00:59:47.532368 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_244_50, total 288 rows starting from the beginning of the part 2024.10.22 00:59:47.582478 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_296_90} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_295_89 to 202410_296_296_0 into Compact with storage Full 2024.10.22 00:59:47.687121 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_245_245_0, total 1 rows starting from the beginning of the part 2024.10.22 00:59:47.730282 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_296_90} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:47.845787 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_296_90} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_295_89, total 20716 rows starting from the beginning of the part 2024.10.22 00:59:47.848143 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_246_246_0, total 1 rows starting from the beginning of the part 2024.10.22 00:59:47.897351 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_296_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_296_296_0, total 115 rows starting from the beginning of the part 2024.10.22 00:59:48.035510 [ 1687 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/usw/krbsjxieopnvczcmybiaibkuabzgp, size 155 2024.10.22 00:59:48.175073 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 7 similar messages) 2024.10.22 00:59:48.280898 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_247_247_0, total 1 rows starting from the beginning of the part 2024.10.22 00:59:48.465539 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 5.93 MiB, will set to 1.71 GiB (RSS), difference: 63.82 MiB 2024.10.22 00:59:48.608867 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_248_248_0, total 1 rows starting from the beginning of the part 2024.10.22 00:59:48.824761 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_249_249_0, total 1 rows starting from the beginning of the part 2024.10.22 00:59:49.387372 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 4263 2024.10.22 00:59:49.482656 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_515_146} MergeTask::MergeProjectionsStage: Merge sorted 30015 rows, containing 13 columns (13 merged, 0 gathered) in 2.576972687 sec., 11647.387708614857 rows/sec., 1.77 MiB/sec. 2024.10.22 00:59:49.590782 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:49.595959 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_515_146} MergedBlockOutputStream: filled checksums 202410_1_515_146 (state Temporary) 2024.10.22 00:59:49.683936 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.98 GiB. 2024.10.22 00:59:49.697056 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_515_146} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_515_146 to 202410_1_515_146 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:49.704051 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_515_146} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_514_145, 202410_515_515_0] -> 202410_1_515_146 2024.10.22 00:59:49.799375 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:59:49.800544 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:59:49.834939 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_297_297_0 (state Temporary) 2024.10.22 00:59:49.874895 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.05 MiB. 2024.10.22 00:59:50.021622 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_297_297_0 to 202410_310_310_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:50.201708 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_296_90} MergeTask::MergeProjectionsStage: Merge sorted 20831 rows, containing 13 columns (13 merged, 0 gathered) in 2.750773831 sec., 7572.778163454907 rows/sec., 3.20 MiB/sec. 2024.10.22 00:59:50.226945 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4263 2024.10.22 00:59:50.282127 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_296_90} MergedBlockOutputStream: filled checksums 202410_1_296_90 (state Temporary) 2024.10.22 00:59:50.570781 [ 1208 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 320 ms. 2024.10.22 00:59:50.633644 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_296_90} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_296_90 to 202410_1_296_90 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:50.713028 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.88 GiB, free memory in arenas 4.75 MiB, will set to 1.69 GiB (RSS), difference: 186.70 MiB 2024.10.22 00:59:50.739051 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_296_90} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_295_89, 202410_296_296_0] -> 202410_1_296_90 2024.10.22 00:59:50.927972 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.12 MiB. 2024.10.22 00:59:50.980753 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 29517 2024.10.22 00:59:50.996409 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_19_19_0 (state Temporary) 2024.10.22 00:59:51.021341 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/luh/goczjcwcykrslndmerqppyqogybmu (LogSeriesLimiter: on interval from 2024-10-22 00:59:45 to 2024-10-22 00:59:51 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:59:51.100414 [ 1690 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/luh/goczjcwcykrslndmerqppyqogybmu, size 4 2024.10.22 00:59:51.135074 [ 1696 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yss/kdtvhnottettweeojkbzzirdfltgy, size 367 2024.10.22 00:59:51.213000 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:51.221025 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.96 GiB. 2024.10.22 00:59:51.265314 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_246_49, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0, 202410_250_250_0, 202410_251_251_0] 2024.10.22 00:59:51.269305 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_246_49, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0, 202410_250_250_0, 202410_251_251_0] 2024.10.22 00:59:51.291804 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_246_49, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0, 202410_250_250_0, 202410_251_251_0] 2024.10.22 00:59:51.492857 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_507_507_0 (state Temporary) 2024.10.22 00:59:51.527689 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} MergeTask::MergeProjectionsStage: Merge sorted 293 rows, containing 32 columns (32 merged, 0 gathered) in 4.624599736 sec., 63.35683447783685 rows/sec., 6.55 KiB/sec. 2024.10.22 00:59:51.569277 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} MergedBlockOutputStream: filled checksums 202410_1_249_51 (state Temporary) 2024.10.22 00:59:51.859248 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_507_507_0 to 202410_520_520_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:51.980825 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_515_146 to 202410_516_516_0 2024.10.22 00:59:51.988333 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.04 MiB using storage policy from min volume index 0 2024.10.22 00:59:51.991483 [ 1233 ] {} DiskLocal: Reserved 1.04 MiB on local disk `default`, having unreserved 297.96 GiB. 2024.10.22 00:59:52.000773 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29517 2024.10.22 00:59:52.025268 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_516_147} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_515_146 to 202410_516_516_0 into Compact with storage Full 2024.10.22 00:59:52.062528 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_249_51 to 202410_1_249_51 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:52.067925 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_249_51} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_244_50, 202410_249_249_0] -> 202410_1_249_51 2024.10.22 00:59:52.124549 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 00:59:52.162933 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_516_147} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:52.175969 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_516_147} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_515_146, total 30015 rows starting from the beginning of the part 2024.10.22 00:59:52.196680 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_516_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_516_516_0, total 59 rows starting from the beginning of the part 2024.10.22 00:59:52.627777 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 3.10 MiB, will set to 1.67 GiB (RSS), difference: 13.18 MiB 2024.10.22 00:59:53.418382 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1870 entries to flush up to offset 477434 2024.10.22 00:59:53.773809 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_516_147} MergeTask::MergeProjectionsStage: Merge sorted 30074 rows, containing 13 columns (13 merged, 0 gathered) in 1.760011882 sec., 17087.384640736193 rows/sec., 2.60 MiB/sec. 2024.10.22 00:59:53.957629 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_516_147} MergedBlockOutputStream: filled checksums 202410_1_516_147 (state Temporary) 2024.10.22 00:59:54.116247 [ 1192 ] {} DatabaseCatalog: Execution took 404 ms. 2024.10.22 00:59:54.166366 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 258 2024.10.22 00:59:54.261126 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 5 old parts to remove. Parts: [202410_1_415_94, 202410_416_416_0, 202410_417_417_0, 202410_418_418_0, 202410_419_419_0] 2024.10.22 00:59:54.269004 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from filesystem (serially): Parts: [202410_1_415_94, 202410_416_416_0, 202410_417_417_0, 202410_418_418_0, 202410_419_419_0] 2024.10.22 00:59:54.397356 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from memory: Parts: [202410_1_415_94, 202410_416_416_0, 202410_417_417_0, 202410_418_418_0, 202410_419_419_0] 2024.10.22 00:59:54.647361 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_516_147} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_516_147 to 202410_1_516_147 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:54.658077 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_516_147} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_515_146, 202410_516_516_0] -> 202410_1_516_147 2024.10.22 00:59:54.688480 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.05 MiB. 2024.10.22 00:59:54.731218 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:54.736279 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.96 GiB. 2024.10.22 00:59:54.783248 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:54.786929 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.96 GiB. 2024.10.22 00:59:54.860910 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_284_284_0 (state Temporary) 2024.10.22 00:59:54.855181 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 2413 2024.10.22 00:59:54.893098 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 184 entries to flush up to offset 21456 2024.10.22 00:59:54.908246 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_284_284_0 to 202410_291_291_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:54.939816 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_240_240_0 (state Temporary) 2024.10.22 00:59:55.040625 [ 1221 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_286_57 to 202410_291_291_0 2024.10.22 00:59:55.053492 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 477434 2024.10.22 00:59:55.061099 [ 1221 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:55.068705 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.96 GiB. 2024.10.22 00:59:55.080261 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_286_57 to 202410_291_291_0 into Compact with storage Full 2024.10.22 00:59:55.093837 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:55.101187 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} MergeTreeSequentialSource: Reading 60 marks from part 202410_1_286_57, total 482533 rows starting from the beginning of the part 2024.10.22 00:59:55.125357 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_287_287_0, total 1629 rows starting from the beginning of the part 2024.10.22 00:59:55.135175 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_288_288_0, total 1599 rows starting from the beginning of the part 2024.10.22 00:59:55.149180 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_289_289_0, total 1857 rows starting from the beginning of the part 2024.10.22 00:59:55.166932 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_290_290_0, total 1536 rows starting from the beginning of the part 2024.10.22 00:59:55.212991 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_291_291_0, total 1870 rows starting from the beginning of the part 2024.10.22 00:59:55.270781 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_240_240_0 to 202410_250_250_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:55.341039 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:55.356364 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.96 GiB. 2024.10.22 00:59:55.371036 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 258 2024.10.22 00:59:55.387195 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_59_12 to 202410_1_59_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:55.391171 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_59_12} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_54_11, 202410_59_59_0] -> 202410_1_59_12 2024.10.22 00:59:55.429134 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_295_295_0 (state Temporary) 2024.10.22 00:59:55.596470 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_295_295_0 to 202410_301_301_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:55.645496 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 21456 2024.10.22 00:59:55.646277 [ 1234 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_296_90 to 202410_301_301_0 2024.10.22 00:59:55.667974 [ 1234 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.21 MiB using storage policy from min volume index 0 2024.10.22 00:59:55.673324 [ 1226 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 00:59:55.684631 [ 1234 ] {} DiskLocal: Reserved 1.21 MiB on local disk `default`, having unreserved 297.96 GiB. 2024.10.22 00:59:55.716322 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_296_90 to 202410_301_301_0 into Compact with storage Full 2024.10.22 00:59:55.768415 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:55.775125 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_296_90, total 20831 rows starting from the beginning of the part 2024.10.22 00:59:55.788814 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_297_297_0, total 154 rows starting from the beginning of the part 2024.10.22 00:59:55.803797 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_298_298_0, total 135 rows starting from the beginning of the part 2024.10.22 00:59:55.968634 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_299_299_0, total 140 rows starting from the beginning of the part 2024.10.22 00:59:56.012060 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 29595 2024.10.22 00:59:56.073429 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_300_300_0, total 116 rows starting from the beginning of the part 2024.10.22 00:59:56.098328 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_301_301_0, total 184 rows starting from the beginning of the part 2024.10.22 00:59:56.356224 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 33.66 MiB. 2024.10.22 00:59:56.484854 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:56.487929 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 00:59:56.705205 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_508_508_0 (state Temporary) 2024.10.22 00:59:56.882312 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_508_508_0 to 202410_521_521_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:56.899135 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 4.68 MiB, will set to 1.67 GiB (RSS), difference: -16.75 MiB 2024.10.22 00:59:56.999502 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_516_147 to 202410_517_517_0 2024.10.22 00:59:57.005457 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.04 MiB using storage policy from min volume index 0 2024.10.22 00:59:57.009630 [ 1243 ] {} DiskLocal: Reserved 1.04 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 00:59:57.041131 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29595 2024.10.22 00:59:57.075236 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_517_148} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_516_147 to 202410_517_517_0 into Compact with storage Full 2024.10.22 00:59:57.296343 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_517_148} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:57.303878 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_517_148} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_516_147, total 30074 rows starting from the beginning of the part 2024.10.22 00:59:57.357804 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_517_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_517_517_0, total 50 rows starting from the beginning of the part 2024.10.22 00:59:57.618459 [ 1253 ] {} DNSResolver: Updating DNS cache 2024.10.22 00:59:57.705173 [ 1253 ] {} DNSResolver: Updated DNS cache 2024.10.22 00:59:57.729516 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 29 entries to flush up to offset 4292 2024.10.22 00:59:57.750459 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(0e7ffea1ab346168c7db85476117ca41) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0e7ffea1ab346168c7db85476117ca41, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 00:59:57.834480 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 56295, file segment info: File segment: [0, 33554431], key: 0e7ffea1ab346168c7db85476117ca41, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 00:59:57.838777 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 00:59:58.096398 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:58.098798 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 00:59:58.216167 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} MergeTask::MergeProjectionsStage: Merge sorted 21560 rows, containing 13 columns (13 merged, 0 gathered) in 2.508394205 sec., 8595.14025228742 rows/sec., 3.64 MiB/sec. 2024.10.22 00:59:58.261320 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_298_298_0 (state Temporary) 2024.10.22 00:59:58.315734 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_298_298_0 to 202410_311_311_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:58.344971 [ 1259 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_306_61 to 202410_311_311_0 2024.10.22 00:59:58.379046 [ 1259 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 00:59:58.381572 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 00:59:58.405053 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_306_61 to 202410_311_311_0 into Compact with storage Full 2024.10.22 00:59:58.416037 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} MergedBlockOutputStream: filled checksums 202410_1_301_91 (state Temporary) 2024.10.22 00:59:58.439657 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 562 bytes for 9fe31d1095e5972eec844c52929b942f:0) 2024.10.22 00:59:58.442920 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4affbed7f682812aec3885cbaf2a3356, offset: 0, size: 1332 2024.10.22 00:59:58.445810 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 4affbed7f682812aec3885cbaf2a3356 for removal 2024.10.22 00:59:58.448801 [ 1476 ] {} CacheMetadata: Key 4affbed7f682812aec3885cbaf2a3356 is removed from metadata 2024.10.22 00:59:58.477549 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 00:59:58.483282 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_306_61, total 4299 rows starting from the beginning of the part 2024.10.22 00:59:58.493495 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_307_307_0, total 19 rows starting from the beginning of the part 2024.10.22 00:59:58.504100 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_308_308_0, total 21 rows starting from the beginning of the part 2024.10.22 00:59:58.530765 [ 1674 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bfq/zdrneonesomalqwcqxfogkrsbduny, size 56295 2024.10.22 00:59:58.546971 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_309_309_0, total 17 rows starting from the beginning of the part 2024.10.22 00:59:58.558930 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4292 2024.10.22 00:59:58.561815 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_310_310_0, total 13 rows starting from the beginning of the part 2024.10.22 00:59:58.579441 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_311_311_0, total 29 rows starting from the beginning of the part 2024.10.22 00:59:58.928057 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column OS 2024.10.22 00:59:58.945105 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_301_91 to 202410_1_301_91 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:58.955112 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_301_91} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_296_90, 202410_301_301_0] -> 202410_1_301_91 2024.10.22 00:59:58.979076 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column OS 2024.10.22 00:59:59.000685 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column OS 2024.10.22 00:59:59.010196 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column OS 2024.10.22 00:59:59.026972 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column OS 2024.10.22 00:59:59.029769 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.55 MiB. 2024.10.22 00:59:59.051540 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column OS 2024.10.22 00:59:59.072814 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column OS 2024.10.22 00:59:59.081083 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column OS 2024.10.22 00:59:59.096676 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column OS 2024.10.22 00:59:59.139895 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ahj/eswlkgbbeibcozlddwibhqnlcvkdr (LogSeriesLimiter: on interval from 2024-10-22 00:59:51 to 2024-10-22 00:59:59 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 00:59:59.266717 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_517_148} MergeTask::MergeProjectionsStage: Merge sorted 30124 rows, containing 13 columns (13 merged, 0 gathered) in 2.240081981 sec., 13447.722117095143 rows/sec., 2.05 MiB/sec. 2024.10.22 00:59:59.291174 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} MergeTask::MergeProjectionsStage: Merge sorted 4398 rows, containing 12 columns (12 merged, 0 gathered) in 0.89318388 sec., 4923.958099199013 rows/sec., 772.44 KiB/sec. 2024.10.22 00:59:59.373037 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 8.54 MiB, will set to 1.68 GiB (RSS), difference: 24.26 MiB 2024.10.22 00:59:59.432589 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_517_148} MergedBlockOutputStream: filled checksums 202410_1_517_148 (state Temporary) 2024.10.22 00:59:59.483767 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} MergedBlockOutputStream: filled checksums 202410_1_311_62 (state Temporary) 2024.10.22 00:59:59.492002 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_517_148} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_517_148 to 202410_1_517_148 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:59.494783 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_517_148} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_516_147, 202410_517_517_0] -> 202410_1_517_148 2024.10.22 00:59:59.527549 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.04 MiB. 2024.10.22 00:59:59.799791 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 00:59:59.805889 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 00:59:59.815947 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_311_62 to 202410_1_311_62 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 00:59:59.826139 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_311_62} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_306_61, 202410_311_311_0] -> 202410_1_311_62 2024.10.22 00:59:59.886227 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.41 MiB. 2024.10.22 01:00:00.146033 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_291_58/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:00:01.163231 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 29668 2024.10.22 01:00:01.305463 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:01.311432 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:01.401222 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_509_509_0 (state Temporary) 2024.10.22 01:00:01.418680 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 10.97 MiB, will set to 1.68 GiB (RSS), difference: 2.42 MiB 2024.10.22 01:00:01.642897 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_509_509_0 to 202410_522_522_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:01.697313 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29668 2024.10.22 01:00:01.797438 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_517_148 to 202410_518_518_0 2024.10.22 01:00:01.800104 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.04 MiB using storage policy from min volume index 0 2024.10.22 01:00:01.802974 [ 1170 ] {} DiskLocal: Reserved 1.04 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:01.809729 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_518_149} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_517_148 to 202410_518_518_0 into Compact with storage Full 2024.10.22 01:00:01.878619 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_518_149} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:01.887104 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_518_149} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_517_148, total 30124 rows starting from the beginning of the part 2024.10.22 01:00:01.959542 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_518_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_518_518_0, total 84 rows starting from the beginning of the part 2024.10.22 01:00:02.064105 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1588 entries to flush up to offset 479022 2024.10.22 01:00:02.303495 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(881ef692f68b82bf415cbff5f4c0d0f3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 881ef692f68b82bf415cbff5f4c0d0f3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:00:02.306163 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 50972, file segment info: File segment: [0, 33554431], key: 881ef692f68b82bf415cbff5f4c0d0f3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:00:02.312435 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:00:02.484467 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 11.20 MiB, will set to 1.68 GiB (RSS), difference: 2.48 MiB 2024.10.22 01:00:02.694107 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:02.699088 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:02.873661 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 259 2024.10.22 01:00:03.019332 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_285_285_0 (state Temporary) 2024.10.22 01:00:03.139791 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:03.143508 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:03.168792 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 827 bytes for ed0569152e5d7bc322fb6ed7ac8e1258:0) 2024.10.22 01:00:03.171640 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 025e2ec3ecda263b9d3ffae49b86c825, offset: 0, size: 198 2024.10.22 01:00:03.177044 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 025e2ec3ecda263b9d3ffae49b86c825 for removal 2024.10.22 01:00:03.180927 [ 1476 ] {} CacheMetadata: Key 025e2ec3ecda263b9d3ffae49b86c825 is removed from metadata 2024.10.22 01:00:03.183227 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_285_285_0 to 202410_292_292_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:03.206465 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_241_241_0 (state Temporary) 2024.10.22 01:00:03.214725 [ 1683 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nna/zktxqgzbamiffvxegndvzqogdrojb, size 50972 2024.10.22 01:00:03.228203 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 479022 2024.10.22 01:00:03.282187 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 156 entries to flush up to offset 21612 2024.10.22 01:00:03.426467 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_241_241_0 to 202410_251_251_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:03.497487 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 259 2024.10.22 01:00:03.635760 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/czx/ioojhiffcdypmgpsoccuzlejvdomy. 2024.10.22 01:00:03.697557 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:03.700478 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:03.809181 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:00:03.821517 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:00:03.837370 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:00:03.840835 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_296_296_0 (state Temporary) 2024.10.22 01:00:03.855959 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:00:03.870173 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:00:03.886611 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:00:03.897693 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:00:03.928158 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_518_149} MergeTask::MergeProjectionsStage: Merge sorted 30208 rows, containing 13 columns (13 merged, 0 gathered) in 2.121973426 sec., 14235.805043488797 rows/sec., 2.17 MiB/sec. 2024.10.22 01:00:04.075326 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mkh/ayumpqdmxkixwzsicvgxikaxkahke (LogSeriesLimiter: on interval from 2024-10-22 00:59:59 to 2024-10-22 01:00:04 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:00:04.099713 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_518_149} MergedBlockOutputStream: filled checksums 202410_1_518_149 (state Temporary) 2024.10.22 01:00:04.124168 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_296_296_0 to 202410_302_302_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:04.224574 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_518_149} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_518_149 to 202410_1_518_149 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:04.229661 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_518_149} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_517_148, 202410_518_518_0] -> 202410_1_518_149 2024.10.22 01:00:04.275521 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.05 MiB. 2024.10.22 01:00:04.278415 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 21612 2024.10.22 01:00:04.435571 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 12.86 MiB, will set to 1.68 GiB (RSS), difference: 6.10 MiB 2024.10.22 01:00:05.709673 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 29724 2024.10.22 01:00:05.890538 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:05.894815 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:06.061631 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 4317 2024.10.22 01:00:06.059685 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_510_510_0 (state Temporary) 2024.10.22 01:00:06.301405 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:06.304785 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:06.368559 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_510_510_0 to 202410_523_523_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:06.440209 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_299_299_0 (state Temporary) 2024.10.22 01:00:06.464025 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29724 2024.10.22 01:00:06.483454 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 13.17 MiB, will set to 1.68 GiB (RSS), difference: -8.32 MiB 2024.10.22 01:00:06.498994 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_299_299_0 to 202410_312_312_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:06.500233 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_518_149 to 202410_519_519_0 2024.10.22 01:00:06.503380 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:00:06.507287 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.05 MiB using storage policy from min volume index 0 2024.10.22 01:00:06.513632 [ 1161 ] {} DiskLocal: Reserved 1.05 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:06.554335 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4317 2024.10.22 01:00:06.595085 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_519_150} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_518_149 to 202410_519_519_0 into Compact with storage Full 2024.10.22 01:00:06.765722 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_519_150} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:06.788507 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_519_150} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_518_149, total 30208 rows starting from the beginning of the part 2024.10.22 01:00:06.834977 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_519_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_519_519_0, total 73 rows starting from the beginning of the part 2024.10.22 01:00:07.280938 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:00:08.815067 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_519_150/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:00:08.856819 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 11.77 MiB, will set to 1.69 GiB (RSS), difference: 2.03 MiB 2024.10.22 01:00:09.524227 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_519_150} MergeTask::MergeProjectionsStage: Merge sorted 30281 rows, containing 13 columns (13 merged, 0 gathered) in 2.966052009 sec., 10209.193873916322 rows/sec., 1.55 MiB/sec. 2024.10.22 01:00:09.800912 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:00:09.803030 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:00:09.837103 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_519_150} MergedBlockOutputStream: filled checksums 202410_1_519_150 (state Temporary) 2024.10.22 01:00:10.075615 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_519_150} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_519_150 to 202410_1_519_150 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:10.080656 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_519_150} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_518_149, 202410_519_519_0] -> 202410_1_519_150 2024.10.22 01:00:10.111383 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.05 MiB. 2024.10.22 01:00:10.246342 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 480558 2024.10.22 01:00:10.419782 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 12.57 MiB, will set to 1.68 GiB (RSS), difference: 5.44 MiB 2024.10.22 01:00:10.475706 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 29757 2024.10.22 01:00:10.815863 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:10.824496 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:10.825751 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:10.930190 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:10.999750 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 260 2024.10.22 01:00:11.091711 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_511_511_0 (state Temporary) 2024.10.22 01:00:11.146850 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_511_511_0 to 202410_524_524_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:11.182724 [ 1217 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_519_150 to 202410_520_520_0 2024.10.22 01:00:11.185384 [ 1217 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.05 MiB using storage policy from min volume index 0 2024.10.22 01:00:11.188547 [ 1217 ] {} DiskLocal: Reserved 1.05 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:11.199011 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_286_286_0 (state Temporary) 2024.10.22 01:00:11.218368 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_520_151} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_519_150 to 202410_520_520_0 into Compact with storage Full 2024.10.22 01:00:11.227983 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29757 2024.10.22 01:00:11.384941 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_520_151} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:11.455355 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_520_151} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_519_150, total 30281 rows starting from the beginning of the part 2024.10.22 01:00:11.502388 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_286_286_0 to 202410_293_293_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:11.521758 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_520_151} MergeTreeSequentialSource: Reading 2 marks from part 202410_520_520_0, total 67 rows starting from the beginning of the part 2024.10.22 01:00:11.533754 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:11.541586 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:11.674239 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 480558 2024.10.22 01:00:11.692449 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:11.694553 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:11.790198 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_291_58/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:00:11.867779 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 76 entries to flush up to offset 21688 2024.10.22 01:00:11.943680 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_242_242_0 (state Temporary) 2024.10.22 01:00:11.944161 [ 1202 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 346 ms. 2024.10.22 01:00:12.205138 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:12.208961 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:12.260628 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_297_297_0 (state Temporary) 2024.10.22 01:00:12.309153 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_242_242_0 to 202410_252_252_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:12.311685 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_297_297_0 to 202410_303_303_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:12.405793 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 260 2024.10.22 01:00:12.475123 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 21688 2024.10.22 01:00:12.614988 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 11.89 MiB, will set to 1.68 GiB (RSS), difference: -11.88 MiB 2024.10.22 01:00:12.715085 [ 1207 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:00:12.750911 [ 1207 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:00:12.885147 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.067450166 sec. 2024.10.22 01:00:13.546936 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 3 old parts to remove. Parts: [202410_1_419_95, 202410_420_420_0, 202410_421_421_0] 2024.10.22 01:00:13.562467 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 3 parts from filesystem (serially): Parts: [202410_1_419_95, 202410_420_420_0, 202410_421_421_0] 2024.10.22 01:00:13.583317 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 3 parts from memory: Parts: [202410_1_419_95, 202410_420_420_0, 202410_421_421_0] 2024.10.22 01:00:13.695720 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_520_151} MergeTask::MergeProjectionsStage: Merge sorted 30348 rows, containing 13 columns (13 merged, 0 gathered) in 2.498254842 sec., 12147.679848267457 rows/sec., 1.85 MiB/sec. 2024.10.22 01:00:13.887428 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_520_151} MergedBlockOutputStream: filled checksums 202410_1_520_151 (state Temporary) 2024.10.22 01:00:14.058844 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 4326 2024.10.22 01:00:14.197833 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_520_151} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_520_151 to 202410_1_520_151 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:14.203915 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_520_151} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_519_150, 202410_520_520_0] -> 202410_1_520_151 2024.10.22 01:00:14.238834 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.04 MiB. 2024.10.22 01:00:14.436230 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:14.440490 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:14.482190 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 217 ms. 2024.10.22 01:00:14.492513 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_300_300_0 (state Temporary) 2024.10.22 01:00:14.510507 [ 1192 ] {} DatabaseCatalog: Execution took 222 ms. 2024.10.22 01:00:14.545780 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_300_300_0 to 202410_313_313_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:14.588781 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4326 2024.10.22 01:00:14.682825 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_240_57, 202410_241_241_0] 2024.10.22 01:00:14.686579 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_240_57, 202410_241_241_0] 2024.10.22 01:00:14.695131 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_240_57, 202410_241_241_0] 2024.10.22 01:00:14.812166 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 10.77 MiB, will set to 1.68 GiB (RSS), difference: 7.66 MiB 2024.10.22 01:00:14.851869 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_60_60_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:00:14.994730 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_60_60_0 (state Temporary) 2024.10.22 01:00:15.241485 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 29817 2024.10.22 01:00:15.392869 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:15.395964 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:15.471358 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_512_512_0 (state Temporary) 2024.10.22 01:00:15.476511 [ 1240 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 01:00:15.588605 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(84269666b4fea5967eabf859348b9299) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 84269666b4fea5967eabf859348b9299, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:00:15.608712 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 760401, file segment info: File segment: [0, 33554431], key: 84269666b4fea5967eabf859348b9299, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:00:15.615441 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:00:15.628172 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_512_512_0 to 202410_525_525_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:15.697243 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29817 2024.10.22 01:00:15.874197 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_520_151 to 202410_521_521_0 2024.10.22 01:00:15.879985 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.05 MiB using storage policy from min volume index 0 2024.10.22 01:00:15.883766 [ 1166 ] {} DiskLocal: Reserved 1.05 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:15.892521 [ 1166 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 246 ms. 2024.10.22 01:00:15.932330 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_521_152} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_520_151 to 202410_521_521_0 into Compact with storage Full 2024.10.22 01:00:16.053478 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} MergeTask::MergeProjectionsStage: Merge sorted 491024 rows, containing 4 columns (4 merged, 0 gathered) in 20.974553194 sec., 23410.46292897733 rows/sec., 343.32 KiB/sec. 2024.10.22 01:00:16.172738 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_521_152} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:16.194928 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_521_152} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_520_151, total 30348 rows starting from the beginning of the part 2024.10.22 01:00:16.268566 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} MergedBlockOutputStream: filled checksums 202410_1_291_58 (state Temporary) 2024.10.22 01:00:16.293935 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_521_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_521_521_0, total 78 rows starting from the beginning of the part 2024.10.22 01:00:16.323219 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_291_58 to 202410_1_291_58 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:16.345252 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_291_58} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_286_57, 202410_291_291_0] -> 202410_1_291_58 2024.10.22 01:00:16.386942 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:00:16.582741 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 585 bytes for 825f4973398a6752cbf9867e7bcf4486:0) 2024.10.22 01:00:16.584646 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6070d50bb0c3e4479b25b6d0bcbff4e5, offset: 0, size: 160 2024.10.22 01:00:16.588397 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6070d50bb0c3e4479b25b6d0bcbff4e5 for removal 2024.10.22 01:00:16.692080 [ 1476 ] {} CacheMetadata: Key 6070d50bb0c3e4479b25b6d0bcbff4e5 is removed from metadata 2024.10.22 01:00:16.693222 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 20013264e3949e7e68ef5f47264abca9, offset: 0, size: 157 2024.10.22 01:00:16.698446 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 6.07 MiB, will set to 1.68 GiB (RSS), difference: -4.72 MiB 2024.10.22 01:00:16.799961 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 20013264e3949e7e68ef5f47264abca9 for removal 2024.10.22 01:00:16.803827 [ 1476 ] {} CacheMetadata: Key 20013264e3949e7e68ef5f47264abca9 is removed from metadata 2024.10.22 01:00:16.804496 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c6411a501a2e277c777ba35377a648f5, offset: 0, size: 201 2024.10.22 01:00:16.808739 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c6411a501a2e277c777ba35377a648f5 for removal 2024.10.22 01:00:16.817105 [ 1476 ] {} CacheMetadata: Key c6411a501a2e277c777ba35377a648f5 is removed from metadata 2024.10.22 01:00:16.977644 [ 1713 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ahj/eswlkgbbeibcozlddwibhqnlcvkdr, size 760401 2024.10.22 01:00:17.492478 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:00:17.609336 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:00:17.730856 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:00:17.729670 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(54c18d418786ac23a2410efb9290fff9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 54c18d418786ac23a2410efb9290fff9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:00:17.740184 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 54c18d418786ac23a2410efb9290fff9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:00:17.743777 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:00:17.747119 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:00:17.767417 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:00:17.786929 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:00:17.809834 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:00:17.846691 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:00:17.970376 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:00:17.994984 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/flz/xqekkwsexcjdtqyuxvishwaxsvegm (LogSeriesLimiter: on interval from 2024-10-22 01:00:04 to 2024-10-22 01:00:17 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:00:18.662800 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_521_152} MergeTask::MergeProjectionsStage: Merge sorted 30426 rows, containing 13 columns (13 merged, 0 gathered) in 2.745117054 sec., 11083.68036826163 rows/sec., 1.69 MiB/sec. 2024.10.22 01:00:18.677699 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 482094 2024.10.22 01:00:18.986367 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_521_152} MergedBlockOutputStream: filled checksums 202410_1_521_152 (state Temporary) 2024.10.22 01:00:19.382644 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_521_152} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_521_152 to 202410_1_521_152 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:19.386336 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_521_152} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_520_151, 202410_521_521_0] -> 202410_1_521_152 2024.10.22 01:00:19.411430 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.04 MiB. 2024.10.22 01:00:19.703873 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 29871 2024.10.22 01:00:19.741419 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 4.88 MiB, will set to 1.68 GiB (RSS), difference: -3.42 MiB 2024.10.22 01:00:19.800380 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:00:19.803511 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:00:19.821391 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:19.832620 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:19.909786 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 261 2024.10.22 01:00:19.945318 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:19.952260 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:19.989483 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 102 entries to flush up to offset 21790 2024.10.22 01:00:20.142761 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_513_513_0 (state Temporary) 2024.10.22 01:00:20.142799 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_287_287_0 (state Temporary) 2024.10.22 01:00:20.202053 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:20.204027 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:20.207410 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:20.310913 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:20.332499 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_513_513_0 to 202410_526_526_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:20.383355 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(3db5a497b55e20c554d8f3947b44e855) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3db5a497b55e20c554d8f3947b44e855, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:00:20.386023 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 682, file segment info: File segment: [0, 33554431], key: 3db5a497b55e20c554d8f3947b44e855, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:00:20.393259 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29871 2024.10.22 01:00:20.395067 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:00:20.419326 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_287_287_0 to 202410_294_294_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:20.453828 [ 1673 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mkh/ayumpqdmxkixwzsicvgxikaxkahke, size 1233357 2024.10.22 01:00:20.455011 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_521_152 to 202410_522_522_0 2024.10.22 01:00:20.464718 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.05 MiB using storage policy from min volume index 0 2024.10.22 01:00:20.469423 [ 1265 ] {} DiskLocal: Reserved 1.05 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:20.476659 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_298_298_0 (state Temporary) 2024.10.22 01:00:20.481197 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 482094 2024.10.22 01:00:20.497187 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_243_243_0 (state Temporary) 2024.10.22 01:00:20.503767 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_199_41, 202410_200_200_0, 202410_201_201_0, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0] 2024.10.22 01:00:20.508281 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_199_41, 202410_200_200_0, 202410_201_201_0, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0] 2024.10.22 01:00:20.537033 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_298_298_0 to 202410_304_304_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:20.580663 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_522_153} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_521_152 to 202410_522_522_0 into Compact with storage Full 2024.10.22 01:00:20.582285 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_243_243_0 to 202410_253_253_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:20.605209 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 21790 2024.10.22 01:00:20.648499 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_522_153} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:20.662595 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_522_153} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_521_152, total 30426 rows starting from the beginning of the part 2024.10.22 01:00:20.690308 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_522_153} MergeTreeSequentialSource: Reading 2 marks from part 202410_522_522_0, total 73 rows starting from the beginning of the part 2024.10.22 01:00:20.708183 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:00:20.719109 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:00:20.735789 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:00:20.743353 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:00:20.763726 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:00:20.770585 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_199_41, 202410_200_200_0, 202410_201_201_0, 202410_202_202_0, 202410_203_203_0, 202410_204_204_0] 2024.10.22 01:00:20.782974 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:00:20.800170 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 261 2024.10.22 01:00:20.804404 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:00:22.091292 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 4345 2024.10.22 01:00:22.178291 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_522_153} MergeTask::MergeProjectionsStage: Merge sorted 30499 rows, containing 13 columns (13 merged, 0 gathered) in 1.593434662 sec., 19140.41455689132 rows/sec., 2.92 MiB/sec. 2024.10.22 01:00:22.200022 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:22.211387 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:22.253775 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_301_301_0 (state Temporary) 2024.10.22 01:00:22.413590 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_301_301_0 to 202410_314_314_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:22.462202 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_522_153} MergedBlockOutputStream: filled checksums 202410_1_522_153 (state Temporary) 2024.10.22 01:00:22.479580 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 4.55 MiB, will set to 1.68 GiB (RSS), difference: -8.94 MiB 2024.10.22 01:00:22.718900 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4345 2024.10.22 01:00:22.729710 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_522_153} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_522_153 to 202410_1_522_153 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:22.734811 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_522_153} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_521_152, 202410_522_522_0] -> 202410_1_522_153 2024.10.22 01:00:22.767378 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.05 MiB. 2024.10.22 01:00:23.373730 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_421_96, 202410_422_422_0, 202410_423_423_0, 202410_424_424_0, 202410_425_425_0, 202410_426_426_0] 2024.10.22 01:00:23.376832 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_421_96, 202410_422_422_0, 202410_423_423_0, 202410_424_424_0, 202410_425_425_0, 202410_426_426_0] 2024.10.22 01:00:23.397833 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_421_96, 202410_422_422_0, 202410_423_423_0, 202410_424_424_0, 202410_425_425_0, 202410_426_426_0] 2024.10.22 01:00:23.533565 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_60_60_0 to 202410_63_63_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:24.281299 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_231_46, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0, 202410_236_236_0] 2024.10.22 01:00:24.283697 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_231_46, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0, 202410_236_236_0] 2024.10.22 01:00:24.317482 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_231_46, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0, 202410_235_235_0, 202410_236_236_0] 2024.10.22 01:00:24.408601 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 29937 2024.10.22 01:00:24.595452 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 4.20 MiB, will set to 1.67 GiB (RSS), difference: 12.80 MiB 2024.10.22 01:00:24.601447 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:24.613343 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:24.662353 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_514_514_0 (state Temporary) 2024.10.22 01:00:24.777122 [ 1176 ] {} DatabaseCatalog: Execution took 230 ms. 2024.10.22 01:00:24.904816 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_514_514_0 to 202410_527_527_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:24.976519 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29937 2024.10.22 01:00:24.976662 [ 1189 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_522_153 to 202410_523_523_0 2024.10.22 01:00:24.984449 [ 1189 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 01:00:24.988279 [ 1189 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:25.020881 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_523_154} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_522_153 to 202410_523_523_0 into Compact with storage Full 2024.10.22 01:00:25.037582 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_523_154} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:25.043659 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_523_154} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_522_153, total 30499 rows starting from the beginning of the part 2024.10.22 01:00:25.076249 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_523_154} MergeTreeSequentialSource: Reading 2 marks from part 202410_523_523_0, total 56 rows starting from the beginning of the part 2024.10.22 01:00:25.458814 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_241_58, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 01:00:25.461323 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_241_58, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 01:00:25.496327 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_241_58, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 01:00:26.420194 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2413 2024.10.22 01:00:26.441510 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/czx/ioojhiffcdypmgpsoccuzlejvdomy. 2024.10.22 01:00:26.592335 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_523_154} MergeTask::MergeProjectionsStage: Merge sorted 30555 rows, containing 13 columns (13 merged, 0 gathered) in 1.584898375 sec., 19278.838619542403 rows/sec., 2.94 MiB/sec. 2024.10.22 01:00:26.674810 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_523_154} MergedBlockOutputStream: filled checksums 202410_1_523_154 (state Temporary) 2024.10.22 01:00:26.748166 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.88 GiB, free memory in arenas 7.09 MiB, will set to 1.68 GiB (RSS), difference: 99.76 MiB 2024.10.22 01:00:26.755181 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_523_154} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_523_154 to 202410_1_523_154 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:26.776045 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_523_154} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_522_153, 202410_523_523_0] -> 202410_1_523_154 2024.10.22 01:00:26.972286 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.84 MiB. 2024.10.22 01:00:27.529475 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1510 entries to flush up to offset 483604 2024.10.22 01:00:27.758225 [ 1266 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:00:27.777612 [ 1266 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:00:28.125638 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 21874 2024.10.22 01:00:28.246279 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:28.248534 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:28.289202 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_299_299_0 (state Temporary) 2024.10.22 01:00:28.467121 [ 1269 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 222 ms. 2024.10.22 01:00:28.495016 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.88 GiB, free memory in arenas 7.41 MiB, will set to 1.66 GiB (RSS), difference: 90.58 MiB 2024.10.22 01:00:28.521769 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_299_299_0 to 202410_305_305_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:28.575854 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:28.578856 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:28.642803 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 21874 2024.10.22 01:00:28.757205 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_288_288_0 (state Temporary) 2024.10.22 01:00:28.929322 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_288_288_0 to 202410_295_295_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:28.993912 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 29978 2024.10.22 01:00:29.076942 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 483604 2024.10.22 01:00:29.200358 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:29.205677 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:29.460156 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_515_515_0 (state Temporary) 2024.10.22 01:00:29.494660 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_515_515_0 to 202410_528_528_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:29.522410 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(12270501856aba1049410ea24ef9ceda) : [0, 203]: Setting download as failed: File segment: [0, 203], key: 12270501856aba1049410ea24ef9ceda, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:29.525220 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(12270501856aba1049410ea24ef9ceda): No space left in cache to reserve 204 bytes, will continue without cache download 2024.10.22 01:00:29.529689 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_523_154 to 202410_524_524_0 2024.10.22 01:00:29.537827 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 01:00:29.543062 [ 1200 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:29.560968 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 29978 2024.10.22 01:00:29.638765 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_524_155} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_523_154 to 202410_524_524_0 into Compact with storage Full 2024.10.22 01:00:29.794472 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_524_155} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:29.801023 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:00:29.804516 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:00:29.808011 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_524_155} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_523_154, total 30555 rows starting from the beginning of the part 2024.10.22 01:00:29.884842 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_524_155} MergeTreeSequentialSource: Reading 2 marks from part 202410_524_524_0, total 33 rows starting from the beginning of the part 2024.10.22 01:00:30.222660 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 4357 2024.10.22 01:00:30.232255 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(966b932bc4988de172761ed965fbcb6d) : [0, 2172026]: Setting download as failed: File segment: [0, 2172026], key: 966b932bc4988de172761ed965fbcb6d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:30.235641 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(966b932bc4988de172761ed965fbcb6d): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:00:30.240176 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_251_50, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0] 2024.10.22 01:00:30.241623 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_251_50, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0] 2024.10.22 01:00:30.281084 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_251_50, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0] 2024.10.22 01:00:30.404231 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 966b932bc4988de172761ed965fbcb6d, offset: 0, size: 0 2024.10.22 01:00:30.456710 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 8.00 MiB, will set to 1.66 GiB (RSS), difference: -3.13 MiB 2024.10.22 01:00:30.405632 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 966b932bc4988de172761ed965fbcb6d for removal 2024.10.22 01:00:30.512667 [ 1476 ] {} CacheMetadata: Key 966b932bc4988de172761ed965fbcb6d is removed from metadata 2024.10.22 01:00:30.516399 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 12270501856aba1049410ea24ef9ceda, offset: 0, size: 0 2024.10.22 01:00:30.520349 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 12270501856aba1049410ea24ef9ceda for removal 2024.10.22 01:00:30.524324 [ 1476 ] {} CacheMetadata: Key 12270501856aba1049410ea24ef9ceda is removed from metadata 2024.10.22 01:00:30.637539 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:30.746747 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:30.783664 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_302_302_0 (state Temporary) 2024.10.22 01:00:30.835768 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_302_302_0 to 202410_315_315_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:30.888010 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4357 2024.10.22 01:00:31.528989 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(eaf5973e1e0a6be16b0e3006ed71665e) : [0, 151]: Setting download as failed: File segment: [0, 151], key: eaf5973e1e0a6be16b0e3006ed71665e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:31.531350 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(eaf5973e1e0a6be16b0e3006ed71665e): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:00:31.884313 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_524_155} MergeTask::MergeProjectionsStage: Merge sorted 30588 rows, containing 13 columns (13 merged, 0 gathered) in 2.24754741 sec., 13609.501567755582 rows/sec., 2.07 MiB/sec. 2024.10.22 01:00:32.276463 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_524_155} MergedBlockOutputStream: filled checksums 202410_1_524_155 (state Temporary) 2024.10.22 01:00:32.333090 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0e74a2f9a7db3a5d573ab4d2c76982ea) : [0, 47208]: Setting download as failed: File segment: [0, 47208], key: 0e74a2f9a7db3a5d573ab4d2c76982ea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:32.336388 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0e74a2f9a7db3a5d573ab4d2c76982ea): No space left in cache to reserve 47209 bytes, will continue without cache download 2024.10.22 01:00:32.401173 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 8.54 MiB, will set to 1.66 GiB (RSS), difference: 8.49 MiB 2024.10.22 01:00:32.357109 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0e74a2f9a7db3a5d573ab4d2c76982ea, offset: 0, size: 0 2024.10.22 01:00:32.464220 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0e74a2f9a7db3a5d573ab4d2c76982ea for removal 2024.10.22 01:00:32.468526 [ 1476 ] {} CacheMetadata: Key 0e74a2f9a7db3a5d573ab4d2c76982ea is removed from metadata 2024.10.22 01:00:32.472611 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eaf5973e1e0a6be16b0e3006ed71665e, offset: 0, size: 0 2024.10.22 01:00:32.474203 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eaf5973e1e0a6be16b0e3006ed71665e for removal 2024.10.22 01:00:32.476825 [ 1476 ] {} CacheMetadata: Key eaf5973e1e0a6be16b0e3006ed71665e is removed from metadata 2024.10.22 01:00:32.587137 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_524_155} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_524_155 to 202410_1_524_155 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:32.591603 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_524_155} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_523_154, 202410_524_524_0] -> 202410_1_524_155 2024.10.22 01:00:32.713061 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.83 MiB. 2024.10.22 01:00:33.354790 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3a7059ad099f7183394229f5bf39ac6a) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 3a7059ad099f7183394229f5bf39ac6a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:33.357167 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3a7059ad099f7183394229f5bf39ac6a): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:00:33.573478 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 30032 2024.10.22 01:00:33.787016 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:33.789444 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:33.928085 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_516_516_0 (state Temporary) 2024.10.22 01:00:33.978912 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 2452 2024.10.22 01:00:34.030407 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fb3eeeb52b98824b25a13ce3b60bf05a) : [0, 2444]: Setting download as failed: File segment: [0, 2444], key: fb3eeeb52b98824b25a13ce3b60bf05a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:34.032198 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fb3eeeb52b98824b25a13ce3b60bf05a): No space left in cache to reserve 2445 bytes, will continue without cache download 2024.10.22 01:00:34.058891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fb3eeeb52b98824b25a13ce3b60bf05a, offset: 0, size: 0 2024.10.22 01:00:34.063438 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fb3eeeb52b98824b25a13ce3b60bf05a for removal 2024.10.22 01:00:34.069228 [ 1476 ] {} CacheMetadata: Key fb3eeeb52b98824b25a13ce3b60bf05a is removed from metadata 2024.10.22 01:00:34.069670 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3a7059ad099f7183394229f5bf39ac6a, offset: 0, size: 0 2024.10.22 01:00:34.071274 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3a7059ad099f7183394229f5bf39ac6a for removal 2024.10.22 01:00:34.075654 [ 1476 ] {} CacheMetadata: Key 3a7059ad099f7183394229f5bf39ac6a is removed from metadata 2024.10.22 01:00:34.374496 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_516_516_0 to 202410_529_529_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:34.394781 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_524_155 to 202410_525_525_0 2024.10.22 01:00:34.400565 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 01:00:34.404883 [ 1223 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:34.448816 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_525_156} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_524_155 to 202410_525_525_0 into Compact with storage Full 2024.10.22 01:00:34.528067 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_525_156} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:34.540349 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_525_156} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_524_155, total 30588 rows starting from the beginning of the part 2024.10.22 01:00:34.584887 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_525_156} MergeTreeSequentialSource: Reading 2 marks from part 202410_525_525_0, total 60 rows starting from the beginning of the part 2024.10.22 01:00:34.671265 [ 1221 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 01:00:34.707163 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30032 2024.10.22 01:00:34.723174 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a90340dc62dd04cbb340967f260a6a87) : [0, 162]: Setting download as failed: File segment: [0, 162], key: a90340dc62dd04cbb340967f260a6a87, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:34.727458 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a90340dc62dd04cbb340967f260a6a87): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:00:35.099567 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:00:35.297069 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_44_9, 202410_45_45_0, 202410_46_46_0, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0] 2024.10.22 01:00:35.399841 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_44_9, 202410_45_45_0, 202410_46_46_0, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0] 2024.10.22 01:00:35.432259 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_44_9, 202410_45_45_0, 202410_46_46_0, 202410_47_47_0, 202410_48_48_0, 202410_49_49_0] 2024.10.22 01:00:35.488572 [ 1166 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_249_51 to 202410_253_253_0 2024.10.22 01:00:35.500343 [ 1166 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:35.502969 [ 1166 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:35.566749 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_253_52} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_249_51 to 202410_253_253_0 into Compact with storage Full 2024.10.22 01:00:35.692035 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(18798fb2b518b2b7e9c467b3d4080c5c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 18798fb2b518b2b7e9c467b3d4080c5c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:00:35.705779 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_525_156/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:00:35.759302 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_253_52} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:35.696770 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 863615, file segment info: File segment: [0, 33554431], key: 18798fb2b518b2b7e9c467b3d4080c5c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:00:35.802147 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:00:35.802386 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 262 2024.10.22 01:00:35.887139 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_253_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_249_51, total 293 rows starting from the beginning of the part 2024.10.22 01:00:36.083250 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1178 entries to flush up to offset 484782 2024.10.22 01:00:36.086177 [ 1247 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:00:36.164160 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:36.172532 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 21964 2024.10.22 01:00:36.172830 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:36.385255 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_253_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_250_250_0, total 1 rows starting from the beginning of the part 2024.10.22 01:00:36.389591 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_244_244_0 (state Temporary) 2024.10.22 01:00:36.580350 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e8968ecdd83f595d835e55e4f9e7a58a) : [0, 1066976]: Setting download as failed: File segment: [0, 1066976], key: e8968ecdd83f595d835e55e4f9e7a58a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:36.581587 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e8968ecdd83f595d835e55e4f9e7a58a): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:00:36.592806 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_253_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_251_251_0, total 1 rows starting from the beginning of the part 2024.10.22 01:00:36.606447 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e8968ecdd83f595d835e55e4f9e7a58a, offset: 0, size: 0 2024.10.22 01:00:36.608764 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e8968ecdd83f595d835e55e4f9e7a58a for removal 2024.10.22 01:00:36.611423 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:36.612858 [ 1476 ] {} CacheMetadata: Key e8968ecdd83f595d835e55e4f9e7a58a is removed from metadata 2024.10.22 01:00:36.620988 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:36.697145 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:36.699013 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:36.719907 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_244_244_0 to 202410_254_254_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:36.720797 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a90340dc62dd04cbb340967f260a6a87, offset: 0, size: 0 2024.10.22 01:00:36.722923 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a90340dc62dd04cbb340967f260a6a87 for removal 2024.10.22 01:00:36.728302 [ 1476 ] {} CacheMetadata: Key a90340dc62dd04cbb340967f260a6a87 is removed from metadata 2024.10.22 01:00:36.755871 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_300_300_0 (state Temporary) 2024.10.22 01:00:36.906956 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_525_156} MergeTask::MergeProjectionsStage: Merge sorted 30648 rows, containing 13 columns (13 merged, 0 gathered) in 2.456331202 sec., 12477.14476575704 rows/sec., 1.90 MiB/sec. 2024.10.22 01:00:36.910669 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_300_300_0 to 202410_306_306_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:36.924322 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 262 2024.10.22 01:00:36.943982 [ 1151 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_301_91 to 202410_302_302_0 2024.10.22 01:00:36.959704 [ 1151 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.20 MiB using storage policy from min volume index 0 2024.10.22 01:00:36.965149 [ 1151 ] {} DiskLocal: Reserved 1.20 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:36.971716 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 21964 2024.10.22 01:00:36.981825 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_302_92} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_301_91 to 202410_302_302_0 into Compact with storage Full 2024.10.22 01:00:36.998532 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_253_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_252_252_0, total 1 rows starting from the beginning of the part 2024.10.22 01:00:37.076630 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_302_92} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:37.089296 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_302_92} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_301_91, total 21560 rows starting from the beginning of the part 2024.10.22 01:00:37.092925 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_289_289_0 (state Temporary) 2024.10.22 01:00:37.131184 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_302_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_302_302_0, total 156 rows starting from the beginning of the part 2024.10.22 01:00:37.144988 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(8253e81d11ffae589550ba7bf1c55ce4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8253e81d11ffae589550ba7bf1c55ce4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:00:37.162638 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_289_289_0 to 202410_296_296_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:37.183124 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 206905, file segment info: File segment: [0, 33554431], key: 8253e81d11ffae589550ba7bf1c55ce4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:00:37.187116 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:00:37.191182 [ 1255 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_291_58 to 202410_296_296_0 2024.10.22 01:00:37.197538 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_253_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_253_253_0, total 1 rows starting from the beginning of the part 2024.10.22 01:00:37.197952 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 484782 2024.10.22 01:00:37.198929 [ 1255 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:37.201762 [ 1255 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:37.243985 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_291_58 to 202410_296_296_0 into Compact with storage Full 2024.10.22 01:00:37.298598 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_525_156} MergedBlockOutputStream: filled checksums 202410_1_525_156 (state Temporary) 2024.10.22 01:00:37.376026 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:37.379814 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} MergeTreeSequentialSource: Reading 61 marks from part 202410_1_291_58, total 491024 rows starting from the beginning of the part 2024.10.22 01:00:37.387168 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_292_292_0, total 1588 rows starting from the beginning of the part 2024.10.22 01:00:37.459975 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_525_156} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_525_156 to 202410_1_525_156 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:37.468182 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_525_156} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_524_155, 202410_525_525_0] -> 202410_1_525_156 2024.10.22 01:00:37.488588 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.83 MiB. 2024.10.22 01:00:37.497821 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_293_293_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:00:37.506652 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_294_294_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:00:37.536826 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_295_295_0, total 1510 rows starting from the beginning of the part 2024.10.22 01:00:37.556195 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_296_296_0, total 1178 rows starting from the beginning of the part 2024.10.22 01:00:38.038612 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(8d4af8f09f4f8c4bcb906efb387bc932) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8d4af8f09f4f8c4bcb906efb387bc932, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:00:38.043270 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 600, file segment info: File segment: [0, 33554431], key: 8d4af8f09f4f8c4bcb906efb387bc932, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:00:38.045342 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:00:38.190468 [ 1706 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/flz/xqekkwsexcjdtqyuxvishwaxsvegm, size 863615 2024.10.22 01:00:38.194736 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_302_92/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:00:38.305832 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column URL 2024.10.22 01:00:38.310593 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column URL 2024.10.22 01:00:38.389350 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 4368 2024.10.22 01:00:38.396558 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 7.83 MiB, will set to 1.67 GiB (RSS), difference: -3.62 MiB 2024.10.22 01:00:38.431370 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column URL 2024.10.22 01:00:38.503210 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:38.505635 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:38.546372 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_303_303_0 (state Temporary) 2024.10.22 01:00:38.571440 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column URL 2024.10.22 01:00:38.583953 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column URL 2024.10.22 01:00:38.603473 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column URL 2024.10.22 01:00:38.606624 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_303_303_0 to 202410_316_316_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:38.620249 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column URL 2024.10.22 01:00:38.642291 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column URL 2024.10.22 01:00:38.645673 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_302_92} MergeTask::MergeProjectionsStage: Merge sorted 21716 rows, containing 13 columns (13 merged, 0 gathered) in 1.672407294 sec., 12984.875202296265 rows/sec., 5.50 MiB/sec. 2024.10.22 01:00:38.646548 [ 1259 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_311_62 to 202410_316_316_0 2024.10.22 01:00:38.666993 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column URL 2024.10.22 01:00:38.667857 [ 1259 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:38.673699 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4368 2024.10.22 01:00:38.674622 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.95 GiB. 2024.10.22 01:00:38.714895 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_311_62 to 202410_316_316_0 into Compact with storage Full 2024.10.22 01:00:38.721071 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 121 entries to flush up to offset 30153 2024.10.22 01:00:38.760348 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/cku/suotwqdzaqkgweepwawbfznftrvtq (LogSeriesLimiter: on interval from 2024-10-22 01:00:17 to 2024-10-22 01:00:38 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:00:38.772621 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_253_52} MergeTask::MergeProjectionsStage: Merge sorted 297 rows, containing 32 columns (32 merged, 0 gathered) in 3.215962367 sec., 92.35182695158696 rows/sec., 9.47 KiB/sec. 2024.10.22 01:00:38.785088 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_253_52} MergedBlockOutputStream: filled checksums 202410_1_253_52 (state Temporary) 2024.10.22 01:00:38.796406 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6f6a0df3578961a6b4a4eb8340e77f3e) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 6f6a0df3578961a6b4a4eb8340e77f3e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:38.801172 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6f6a0df3578961a6b4a4eb8340e77f3e): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:00:38.831548 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:38.845686 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_302_92} MergedBlockOutputStream: filled checksums 202410_1_302_92 (state Temporary) 2024.10.22 01:00:38.947630 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_311_62, total 4398 rows starting from the beginning of the part 2024.10.22 01:00:39.003289 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_312_312_0, total 25 rows starting from the beginning of the part 2024.10.22 01:00:39.044241 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_313_313_0, total 9 rows starting from the beginning of the part 2024.10.22 01:00:39.097421 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_302_92} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_302_92 to 202410_1_302_92 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:39.101419 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_302_92} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_301_91, 202410_302_302_0] -> 202410_1_302_92 2024.10.22 01:00:39.104619 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_314_314_0, total 19 rows starting from the beginning of the part 2024.10.22 01:00:39.132719 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.13 MiB. 2024.10.22 01:00:39.275900 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_315_315_0, total 12 rows starting from the beginning of the part 2024.10.22 01:00:39.326513 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_316_316_0, total 11 rows starting from the beginning of the part 2024.10.22 01:00:39.467223 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_253_52} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_253_52 to 202410_1_253_52 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:39.471871 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_253_52} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_249_51, 202410_253_253_0] -> 202410_1_253_52 2024.10.22 01:00:39.480129 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:39.483069 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:39.528906 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:00:39.565414 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_517_517_0 (state Temporary) 2024.10.22 01:00:39.717688 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_517_517_0 to 202410_530_530_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:39.746077 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_525_156 to 202410_530_530_0 2024.10.22 01:00:39.749538 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(80399e0e55a63a3de06370f8f06dd1db) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 80399e0e55a63a3de06370f8f06dd1db, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:00:39.754977 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 653, file segment info: File segment: [0, 33554431], key: 80399e0e55a63a3de06370f8f06dd1db, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:00:39.757980 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:00:39.766345 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.10 MiB using storage policy from min volume index 0 2024.10.22 01:00:39.770292 [ 1231 ] {} DiskLocal: Reserved 1.10 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:39.811984 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_525_156 to 202410_530_530_0 into Compact with storage Full 2024.10.22 01:00:39.812291 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:00:39.819376 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:00:39.980120 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:39.996056 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_525_156, total 30648 rows starting from the beginning of the part 2024.10.22 01:00:40.070859 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_526_526_0, total 54 rows starting from the beginning of the part 2024.10.22 01:00:40.123013 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_527_527_0, total 66 rows starting from the beginning of the part 2024.10.22 01:00:40.200646 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30153 2024.10.22 01:00:40.211741 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_528_528_0, total 41 rows starting from the beginning of the part 2024.10.22 01:00:40.284790 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_529_529_0, total 54 rows starting from the beginning of the part 2024.10.22 01:00:40.347360 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:00:40.387341 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_296_59/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:00:40.481568 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:00:40.488621 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_530_530_0, total 121 rows starting from the beginning of the part 2024.10.22 01:00:40.497725 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:00:40.532338 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:00:40.628473 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} MergeTask::MergeProjectionsStage: Merge sorted 4474 rows, containing 12 columns (12 merged, 0 gathered) in 1.923841519 sec., 2325.555382714453 rows/sec., 364.93 KiB/sec. 2024.10.22 01:00:40.692873 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} MergedBlockOutputStream: filled checksums 202410_1_316_63 (state Temporary) 2024.10.22 01:00:40.772426 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:00:40.797659 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:00:40.821944 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:00:40.867303 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a52a9dbd4c6641271158307d1e5fd24e) : [0, 40856]: Setting download as failed: File segment: [0, 40856], key: a52a9dbd4c6641271158307d1e5fd24e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:40.868157 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a52a9dbd4c6641271158307d1e5fd24e): No space left in cache to reserve 40857 bytes, will continue without cache download 2024.10.22 01:00:40.887455 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a52a9dbd4c6641271158307d1e5fd24e, offset: 0, size: 0 2024.10.22 01:00:40.890793 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a52a9dbd4c6641271158307d1e5fd24e for removal 2024.10.22 01:00:40.893979 [ 1476 ] {} CacheMetadata: Key a52a9dbd4c6641271158307d1e5fd24e is removed from metadata 2024.10.22 01:00:40.894830 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6f6a0df3578961a6b4a4eb8340e77f3e, offset: 0, size: 0 2024.10.22 01:00:40.896745 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6f6a0df3578961a6b4a4eb8340e77f3e for removal 2024.10.22 01:00:40.898442 [ 1476 ] {} CacheMetadata: Key 6f6a0df3578961a6b4a4eb8340e77f3e is removed from metadata 2024.10.22 01:00:40.935462 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_316_63 to 202410_1_316_63 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:41.044135 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_316_63} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_311_62, 202410_316_316_0] -> 202410_1_316_63 2024.10.22 01:00:41.196618 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.42 MiB. 2024.10.22 01:00:41.703877 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4fdf3086e56d151ea6bee5f59b0b7299) : [0, 145]: Setting download as failed: File segment: [0, 145], key: 4fdf3086e56d151ea6bee5f59b0b7299, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:41.707349 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4fdf3086e56d151ea6bee5f59b0b7299): No space left in cache to reserve 146 bytes, will continue without cache download 2024.10.22 01:00:42.319788 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 13.56 MiB, will set to 1.70 GiB (RSS), difference: -22.30 MiB 2024.10.22 01:00:42.395390 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1676ba861718ece4a1d5e138b47719a5) : [0, 30318]: Setting download as failed: File segment: [0, 30318], key: 1676ba861718ece4a1d5e138b47719a5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:42.396673 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1676ba861718ece4a1d5e138b47719a5): No space left in cache to reserve 30319 bytes, will continue without cache download 2024.10.22 01:00:42.410737 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1676ba861718ece4a1d5e138b47719a5, offset: 0, size: 0 2024.10.22 01:00:42.413772 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1676ba861718ece4a1d5e138b47719a5 for removal 2024.10.22 01:00:42.419241 [ 1476 ] {} CacheMetadata: Key 1676ba861718ece4a1d5e138b47719a5 is removed from metadata 2024.10.22 01:00:42.420922 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4fdf3086e56d151ea6bee5f59b0b7299, offset: 0, size: 0 2024.10.22 01:00:42.425498 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4fdf3086e56d151ea6bee5f59b0b7299 for removal 2024.10.22 01:00:42.430472 [ 1476 ] {} CacheMetadata: Key 4fdf3086e56d151ea6bee5f59b0b7299 is removed from metadata 2024.10.22 01:00:42.781580 [ 1183 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:00:42.802170 [ 1183 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:00:42.839172 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} MergeTask::MergeProjectionsStage: Merge sorted 30984 rows, containing 13 columns (13 merged, 0 gathered) in 3.046819277 sec., 10169.29367419123 rows/sec., 1.55 MiB/sec. 2024.10.22 01:00:43.079440 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(2d023c5b0df17a8362683d072e6260ab) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2d023c5b0df17a8362683d072e6260ab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:00:43.082101 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 2d023c5b0df17a8362683d072e6260ab, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:00:43.084248 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:00:43.122580 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} MergedBlockOutputStream: filled checksums 202410_1_530_157 (state Temporary) 2024.10.22 01:00:43.219571 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_530_157 to 202410_1_530_157 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:43.228269 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_530_157} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_525_156, 202410_530_530_0] -> 202410_1_530_157 2024.10.22 01:00:43.282116 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.08 MiB. 2024.10.22 01:00:43.650250 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(18a9ff8250afb20cf7a6fe0443b16860) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 18a9ff8250afb20cf7a6fe0443b16860, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:43.653247 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(18a9ff8250afb20cf7a6fe0443b16860): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:00:44.203462 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1042 entries to flush up to offset 485824 2024.10.22 01:00:44.219133 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 30239 2024.10.22 01:00:44.426117 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 263 2024.10.22 01:00:44.483834 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 135 entries to flush up to offset 22099 2024.10.22 01:00:44.502169 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:44.504997 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:44.529270 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(cb847a1ba50b924206b9dc98e74c9c81) : [0, 301387]: Setting download as failed: File segment: [0, 301387], key: cb847a1ba50b924206b9dc98e74c9c81, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:44.532785 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(cb847a1ba50b924206b9dc98e74c9c81): No space left in cache to reserve 301388 bytes, will continue without cache download 2024.10.22 01:00:44.554778 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cb847a1ba50b924206b9dc98e74c9c81, offset: 0, size: 0 2024.10.22 01:00:44.557012 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cb847a1ba50b924206b9dc98e74c9c81 for removal 2024.10.22 01:00:44.566919 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 18a9ff8250afb20cf7a6fe0443b16860, offset: 0, size: 0 2024.10.22 01:00:44.569129 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 18a9ff8250afb20cf7a6fe0443b16860 for removal 2024.10.22 01:00:44.573510 [ 1476 ] {} CacheMetadata: Key cb847a1ba50b924206b9dc98e74c9c81 is removed from metadata 2024.10.22 01:00:44.580935 [ 1476 ] {} CacheMetadata: Key 18a9ff8250afb20cf7a6fe0443b16860 is removed from metadata 2024.10.22 01:00:44.581312 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_518_518_0 (state Temporary) 2024.10.22 01:00:44.736839 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_518_518_0 to 202410_531_531_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:44.800149 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:44.807463 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:44.852323 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:44.864610 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:44.916842 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30239 2024.10.22 01:00:44.960726 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_301_301_0 (state Temporary) 2024.10.22 01:00:45.083028 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:45.085499 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:45.128624 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_301_301_0 to 202410_307_307_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:45.158077 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_290_290_0 (state Temporary) 2024.10.22 01:00:45.168197 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_245_245_0 (state Temporary) 2024.10.22 01:00:45.174273 [ 1177 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_302_92 to 202410_303_303_0 2024.10.22 01:00:45.182048 [ 1177 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.20 MiB using storage policy from min volume index 0 2024.10.22 01:00:45.188715 [ 1177 ] {} DiskLocal: Reserved 1.20 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:45.189344 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 22099 2024.10.22 01:00:45.198590 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_303_93} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_302_92 to 202410_303_303_0 into Compact with storage Full 2024.10.22 01:00:45.282405 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_245_245_0 to 202410_255_255_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:45.353651 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 15.35 MiB, will set to 1.70 GiB (RSS), difference: -1.91 MiB 2024.10.22 01:00:45.381459 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 263 2024.10.22 01:00:45.538593 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_290_290_0 to 202410_297_297_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:45.558573 [ 1158 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 248 ms. 2024.10.22 01:00:45.575982 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(407931da53364350af2f5338da5006f3) : [0, 149]: Setting download as failed: File segment: [0, 149], key: 407931da53364350af2f5338da5006f3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:45.580244 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(407931da53364350af2f5338da5006f3): No space left in cache to reserve 150 bytes, will continue without cache download 2024.10.22 01:00:45.589234 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_303_93} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:45.685475 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 485824 2024.10.22 01:00:45.706139 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_303_93} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_302_92, total 21716 rows starting from the beginning of the part 2024.10.22 01:00:45.868555 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_303_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_303_303_0, total 76 rows starting from the beginning of the part 2024.10.22 01:00:46.176049 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 29 entries to flush up to offset 4397 2024.10.22 01:00:46.316237 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(81031899d7af4316a2083d3e1a7cd540) : [0, 199313]: Setting download as failed: File segment: [0, 199313], key: 81031899d7af4316a2083d3e1a7cd540, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:46.318658 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(81031899d7af4316a2083d3e1a7cd540): No space left in cache to reserve 199314 bytes, will continue without cache download 2024.10.22 01:00:46.358596 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 81031899d7af4316a2083d3e1a7cd540, offset: 0, size: 0 2024.10.22 01:00:46.362256 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 81031899d7af4316a2083d3e1a7cd540 for removal 2024.10.22 01:00:46.366406 [ 1476 ] {} CacheMetadata: Key 81031899d7af4316a2083d3e1a7cd540 is removed from metadata 2024.10.22 01:00:46.368458 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 407931da53364350af2f5338da5006f3, offset: 0, size: 0 2024.10.22 01:00:46.372721 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 407931da53364350af2f5338da5006f3 for removal 2024.10.22 01:00:46.376759 [ 1476 ] {} CacheMetadata: Key 407931da53364350af2f5338da5006f3 is removed from metadata 2024.10.22 01:00:46.403979 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:46.406632 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:46.464791 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_304_304_0 (state Temporary) 2024.10.22 01:00:46.830335 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_304_304_0 to 202410_317_317_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:46.879866 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4397 2024.10.22 01:00:47.232489 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e2e529ff0b6f8e941dd60332968889f4) : [0, 176]: Setting download as failed: File segment: [0, 176], key: e2e529ff0b6f8e941dd60332968889f4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:47.236548 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e2e529ff0b6f8e941dd60332968889f4): No space left in cache to reserve 177 bytes, will continue without cache download 2024.10.22 01:00:47.339703 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_303_93} MergeTask::MergeProjectionsStage: Merge sorted 21792 rows, containing 13 columns (13 merged, 0 gathered) in 2.145234272 sec., 10158.33108972445 rows/sec., 4.30 MiB/sec. 2024.10.22 01:00:47.429553 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_303_93} MergedBlockOutputStream: filled checksums 202410_1_303_93 (state Temporary) 2024.10.22 01:00:47.702428 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 12.32 MiB, will set to 1.70 GiB (RSS), difference: 8.27 MiB 2024.10.22 01:00:47.862421 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_303_93} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_303_93 to 202410_1_303_93 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:47.970544 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_303_93} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_302_92, 202410_303_303_0] -> 202410_1_303_93 2024.10.22 01:00:47.942625 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1_426_97, 202410_427_427_0, 202410_428_428_0, 202410_429_429_0] 2024.10.22 01:00:47.974918 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1_426_97, 202410_427_427_0, 202410_428_428_0, 202410_429_429_0] 2024.10.22 01:00:48.005446 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.32 MiB. 2024.10.22 01:00:48.103779 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1_426_97, 202410_427_427_0, 202410_428_428_0, 202410_429_429_0] 2024.10.22 01:00:48.287921 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(543c672f8aab151407b4d1e372f8c8a2) : [0, 426421]: Setting download as failed: File segment: [0, 426421], key: 543c672f8aab151407b4d1e372f8c8a2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:48.289557 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(543c672f8aab151407b4d1e372f8c8a2): No space left in cache to reserve 426422 bytes, will continue without cache download 2024.10.22 01:00:48.331036 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 543c672f8aab151407b4d1e372f8c8a2, offset: 0, size: 0 2024.10.22 01:00:48.337435 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 543c672f8aab151407b4d1e372f8c8a2 for removal 2024.10.22 01:00:48.341116 [ 1476 ] {} CacheMetadata: Key 543c672f8aab151407b4d1e372f8c8a2 is removed from metadata 2024.10.22 01:00:48.444297 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e2e529ff0b6f8e941dd60332968889f4, offset: 0, size: 0 2024.10.22 01:00:48.446053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e2e529ff0b6f8e941dd60332968889f4 for removal 2024.10.22 01:00:48.459940 [ 1476 ] {} CacheMetadata: Key e2e529ff0b6f8e941dd60332968889f4 is removed from metadata 2024.10.22 01:00:48.996003 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 30315 2024.10.22 01:00:49.292820 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b058b65359a99120d3f702a9c1bbf1f0) : [0, 151]: Setting download as failed: File segment: [0, 151], key: b058b65359a99120d3f702a9c1bbf1f0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:49.296131 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b058b65359a99120d3f702a9c1bbf1f0): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:00:49.575645 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:49.580042 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:49.654207 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_519_519_0 (state Temporary) 2024.10.22 01:00:49.763048 [ 1196 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 01:00:49.806374 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_519_519_0 to 202410_532_532_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:49.817242 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:00:49.824600 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:00:49.918338 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30315 2024.10.22 01:00:50.337107 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:50.341484 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:50.443499 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f90e6482757fe6f7e9e81a23b694bdd6) : [0, 4488]: Setting download as failed: File segment: [0, 4488], key: f90e6482757fe6f7e9e81a23b694bdd6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:50.444780 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f90e6482757fe6f7e9e81a23b694bdd6): No space left in cache to reserve 4489 bytes, will continue without cache download 2024.10.22 01:00:50.481273 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f90e6482757fe6f7e9e81a23b694bdd6, offset: 0, size: 0 2024.10.22 01:00:50.483485 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f90e6482757fe6f7e9e81a23b694bdd6 for removal 2024.10.22 01:00:50.487184 [ 1476 ] {} CacheMetadata: Key f90e6482757fe6f7e9e81a23b694bdd6 is removed from metadata 2024.10.22 01:00:50.588042 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b058b65359a99120d3f702a9c1bbf1f0, offset: 0, size: 0 2024.10.22 01:00:50.589946 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b058b65359a99120d3f702a9c1bbf1f0 for removal 2024.10.22 01:00:50.592259 [ 1476 ] {} CacheMetadata: Key b058b65359a99120d3f702a9c1bbf1f0 is removed from metadata 2024.10.22 01:00:50.747244 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 12.04 MiB, will set to 1.70 GiB (RSS), difference: -1.46 MiB 2024.10.22 01:00:50.888668 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_296_59/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:00:51.216667 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7bfff1466fede0be2294de6e17edffb4) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 7bfff1466fede0be2294de6e17edffb4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:51.219765 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7bfff1466fede0be2294de6e17edffb4): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:00:51.980437 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(328d2e9833b338945b663ab9fa01e0c3) : [0, 73510]: Setting download as failed: File segment: [0, 73510], key: 328d2e9833b338945b663ab9fa01e0c3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:51.982492 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(328d2e9833b338945b663ab9fa01e0c3): No space left in cache to reserve 73511 bytes, will continue without cache download 2024.10.22 01:00:52.097702 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 328d2e9833b338945b663ab9fa01e0c3, offset: 0, size: 0 2024.10.22 01:00:52.102162 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 328d2e9833b338945b663ab9fa01e0c3 for removal 2024.10.22 01:00:52.107409 [ 1476 ] {} CacheMetadata: Key 328d2e9833b338945b663ab9fa01e0c3 is removed from metadata 2024.10.22 01:00:52.111299 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7bfff1466fede0be2294de6e17edffb4, offset: 0, size: 0 2024.10.22 01:00:52.113250 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7bfff1466fede0be2294de6e17edffb4 for removal 2024.10.22 01:00:52.116169 [ 1476 ] {} CacheMetadata: Key 7bfff1466fede0be2294de6e17edffb4 is removed from metadata 2024.10.22 01:00:52.456774 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 12.41 MiB, will set to 1.70 GiB (RSS), difference: -2.46 MiB 2024.10.22 01:00:52.688796 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1262 entries to flush up to offset 487086 2024.10.22 01:00:52.708252 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 78 entries to flush up to offset 22177 2024.10.22 01:00:52.873019 [ 1212 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 01:00:52.883761 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 264 2024.10.22 01:00:52.887392 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a7638d6b12d701ffd194df10855aae54) : [0, 152]: Setting download as failed: File segment: [0, 152], key: a7638d6b12d701ffd194df10855aae54, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:52.889202 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a7638d6b12d701ffd194df10855aae54): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:00:52.938152 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:52.947894 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:53.026553 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_302_302_0 (state Temporary) 2024.10.22 01:00:53.180618 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_302_302_0 to 202410_308_308_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:53.203722 [ 1216 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_303_93 to 202410_304_304_0 2024.10.22 01:00:53.211427 [ 1216 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.21 MiB using storage policy from min volume index 0 2024.10.22 01:00:53.214957 [ 1216 ] {} DiskLocal: Reserved 1.21 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:53.236326 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_304_94} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_303_93 to 202410_304_304_0 into Compact with storage Full 2024.10.22 01:00:53.252727 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 22177 2024.10.22 01:00:53.304575 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} MergeTask::MergeProjectionsStage: Merge sorted 498372 rows, containing 4 columns (4 merged, 0 gathered) in 16.067631966 sec., 31017.140612542207 rows/sec., 454.80 KiB/sec. 2024.10.22 01:00:53.310020 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_61_61_0 (state Temporary) 2024.10.22 01:00:53.332031 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_304_94} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:00:53.357106 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_304_94} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_303_93, total 21792 rows starting from the beginning of the part 2024.10.22 01:00:53.408300 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} MergedBlockOutputStream: filled checksums 202410_1_296_59 (state Temporary) 2024.10.22 01:00:53.423748 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:53.429233 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:53.476347 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_304_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_304_304_0, total 102 rows starting from the beginning of the part 2024.10.22 01:00:53.497713 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_296_59 to 202410_1_296_59 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:53.501590 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_296_59} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_291_58, 202410_296_296_0] -> 202410_1_296_59 2024.10.22 01:00:53.566329 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:00:53.581011 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:53.584303 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:53.594987 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4abf7f0c3fc5e926fa34fafc1ff7e036) : [0, 79467]: Setting download as failed: File segment: [0, 79467], key: 4abf7f0c3fc5e926fa34fafc1ff7e036, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:53.596447 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4abf7f0c3fc5e926fa34fafc1ff7e036): No space left in cache to reserve 79468 bytes, will continue without cache download 2024.10.22 01:00:53.611278 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4abf7f0c3fc5e926fa34fafc1ff7e036, offset: 0, size: 0 2024.10.22 01:00:53.613708 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4abf7f0c3fc5e926fa34fafc1ff7e036 for removal 2024.10.22 01:00:53.620699 [ 1476 ] {} CacheMetadata: Key 4abf7f0c3fc5e926fa34fafc1ff7e036 is removed from metadata 2024.10.22 01:00:53.692322 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_291_291_0 (state Temporary) 2024.10.22 01:00:53.698367 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_246_246_0 (state Temporary) 2024.10.22 01:00:53.723935 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a7638d6b12d701ffd194df10855aae54, offset: 0, size: 0 2024.10.22 01:00:53.728156 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a7638d6b12d701ffd194df10855aae54 for removal 2024.10.22 01:00:53.731971 [ 1476 ] {} CacheMetadata: Key a7638d6b12d701ffd194df10855aae54 is removed from metadata 2024.10.22 01:00:53.780032 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_246_246_0 to 202410_256_256_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:53.866496 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_291_291_0 to 202410_298_298_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:53.896242 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 487086 2024.10.22 01:00:53.944589 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 30390 2024.10.22 01:00:54.084204 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 264 2024.10.22 01:00:54.284052 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:54.285699 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:54.348046 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_520_520_0 (state Temporary) 2024.10.22 01:00:54.383392 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 4412 2024.10.22 01:00:54.401224 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_520_520_0 to 202410_533_533_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:54.406553 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8f57687b9fe06d6cec7fe9feb1c5df0f) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 8f57687b9fe06d6cec7fe9feb1c5df0f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:54.408537 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8f57687b9fe06d6cec7fe9feb1c5df0f): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:00:54.486903 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30390 2024.10.22 01:00:54.532784 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:54.540095 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:54.583741 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_305_305_0 (state Temporary) 2024.10.22 01:00:54.590057 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.88 GiB, free memory in arenas 12.54 MiB, will set to 1.71 GiB (RSS), difference: -20.30 MiB 2024.10.22 01:00:54.836064 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_305_305_0 to 202410_318_318_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:54.918410 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_304_94} MergeTask::MergeProjectionsStage: Merge sorted 21894 rows, containing 13 columns (13 merged, 0 gathered) in 1.689263802 sec., 12960.675516801253 rows/sec., 5.49 MiB/sec. 2024.10.22 01:00:55.132587 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4412 2024.10.22 01:00:55.360117 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(ba55df74268f270a2be3a495e6194867) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ba55df74268f270a2be3a495e6194867, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:00:55.369429 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 680932, file segment info: File segment: [0, 33554431], key: ba55df74268f270a2be3a495e6194867, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:00:55.372696 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:00:55.406724 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_304_94} MergedBlockOutputStream: filled checksums 202410_1_304_94 (state Temporary) 2024.10.22 01:00:55.473102 [ 1219 ] {} DatabaseCatalog: Execution took 379 ms. 2024.10.22 01:00:55.499796 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_304_94} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_304_94 to 202410_1_304_94 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:55.503668 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_304_94} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_303_93, 202410_304_304_0] -> 202410_1_304_94 2024.10.22 01:00:55.551733 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.36 MiB. 2024.10.22 01:00:56.262649 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(810a5de6dc8822f9d0d79c4ae3d278c5) : [0, 134535]: Setting download as failed: File segment: [0, 134535], key: 810a5de6dc8822f9d0d79c4ae3d278c5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:56.265363 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(810a5de6dc8822f9d0d79c4ae3d278c5): No space left in cache to reserve 134536 bytes, will continue without cache download 2024.10.22 01:00:56.391136 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 810a5de6dc8822f9d0d79c4ae3d278c5, offset: 0, size: 0 2024.10.22 01:00:56.391969 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 810a5de6dc8822f9d0d79c4ae3d278c5 for removal 2024.10.22 01:00:56.395048 [ 1476 ] {} CacheMetadata: Key 810a5de6dc8822f9d0d79c4ae3d278c5 is removed from metadata 2024.10.22 01:00:56.396606 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8f57687b9fe06d6cec7fe9feb1c5df0f, offset: 0, size: 0 2024.10.22 01:00:56.409470 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8f57687b9fe06d6cec7fe9feb1c5df0f for removal 2024.10.22 01:00:56.412088 [ 1476 ] {} CacheMetadata: Key 8f57687b9fe06d6cec7fe9feb1c5df0f is removed from metadata 2024.10.22 01:00:56.817197 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(edb0e095985172dfe711fe35cd8f636a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: edb0e095985172dfe711fe35cd8f636a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:00:56.821800 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 667, file segment info: File segment: [0, 33554431], key: edb0e095985172dfe711fe35cd8f636a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:00:56.827805 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:00:57.171884 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:00:57.185614 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:00:57.230741 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:00:57.278732 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:00:57.303334 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:00:57.448769 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(41c63f4f0d57371bcfa8fc482ad45949) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 41c63f4f0d57371bcfa8fc482ad45949, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:57.453177 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(41c63f4f0d57371bcfa8fc482ad45949): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:00:57.459978 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:00:57.498351 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:00:57.576698 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tmv/wryvhchwfilyrchvxyktfviavixmx (LogSeriesLimiter: on interval from 2024-10-22 01:00:38 to 2024-10-22 01:00:57 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:00:57.805122 [ 1246 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:00:57.822407 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 9.70 MiB, will set to 1.70 GiB (RSS), difference: 11.51 MiB 2024.10.22 01:00:57.830646 [ 1246 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:00:58.094805 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_61_61_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:00:58.445682 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(19bc29c4d7a355a3d3da43ab4f7703c2) : [0, 131342]: Setting download as failed: File segment: [0, 131342], key: 19bc29c4d7a355a3d3da43ab4f7703c2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:58.448079 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(19bc29c4d7a355a3d3da43ab4f7703c2): No space left in cache to reserve 131343 bytes, will continue without cache download 2024.10.22 01:00:58.483912 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 19bc29c4d7a355a3d3da43ab4f7703c2, offset: 0, size: 0 2024.10.22 01:00:58.485453 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 19bc29c4d7a355a3d3da43ab4f7703c2 for removal 2024.10.22 01:00:58.486991 [ 1476 ] {} CacheMetadata: Key 19bc29c4d7a355a3d3da43ab4f7703c2 is removed from metadata 2024.10.22 01:00:58.490053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 41c63f4f0d57371bcfa8fc482ad45949, offset: 0, size: 0 2024.10.22 01:00:58.491284 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 41c63f4f0d57371bcfa8fc482ad45949 for removal 2024.10.22 01:00:58.492527 [ 1476 ] {} CacheMetadata: Key 41c63f4f0d57371bcfa8fc482ad45949 is removed from metadata 2024.10.22 01:00:58.498108 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 30448 2024.10.22 01:00:58.981211 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:00:58.988381 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:00:59.279955 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_521_521_0 (state Temporary) 2024.10.22 01:00:59.452899 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_521_521_0 to 202410_534_534_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:00:59.499249 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30448 2024.10.22 01:00:59.730250 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(99a4fd8e3a3e996f6cfa98d185e5694e) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 99a4fd8e3a3e996f6cfa98d185e5694e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:00:59.731386 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(99a4fd8e3a3e996f6cfa98d185e5694e): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:00:59.819131 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:00:59.821915 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:01:00.464948 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 7.70 MiB, will set to 1.70 GiB (RSS), difference: 4.28 MiB 2024.10.22 01:01:00.546688 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3ae713a5e630345d7b397ccb691ab91c) : [0, 59408]: Setting download as failed: File segment: [0, 59408], key: 3ae713a5e630345d7b397ccb691ab91c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:00.547615 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3ae713a5e630345d7b397ccb691ab91c): No space left in cache to reserve 59409 bytes, will continue without cache download 2024.10.22 01:01:00.558846 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3ae713a5e630345d7b397ccb691ab91c, offset: 0, size: 0 2024.10.22 01:01:00.560631 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3ae713a5e630345d7b397ccb691ab91c for removal 2024.10.22 01:01:00.563297 [ 1476 ] {} CacheMetadata: Key 3ae713a5e630345d7b397ccb691ab91c is removed from metadata 2024.10.22 01:01:00.563556 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 99a4fd8e3a3e996f6cfa98d185e5694e, offset: 0, size: 0 2024.10.22 01:01:00.565085 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 99a4fd8e3a3e996f6cfa98d185e5694e for removal 2024.10.22 01:01:00.566985 [ 1476 ] {} CacheMetadata: Key 99a4fd8e3a3e996f6cfa98d185e5694e is removed from metadata 2024.10.22 01:01:00.774912 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 83 entries to flush up to offset 22260 2024.10.22 01:01:00.900043 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 488622 2024.10.22 01:01:00.960814 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_61_61_0 to 202410_64_64_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:01.082930 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:01.085041 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:01.113188 [ 1267 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_59_12 to 202410_64_64_0 2024.10.22 01:01:01.131982 [ 1267 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.40 MiB using storage policy from min volume index 0 2024.10.22 01:01:01.135164 [ 1267 ] {} DiskLocal: Reserved 1.40 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:01.197466 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_303_303_0 (state Temporary) 2024.10.22 01:01:01.244944 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_303_303_0 to 202410_309_309_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:01.280479 [ 1157 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_304_94 to 202410_305_305_0 2024.10.22 01:01:01.286846 [ 1157 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.21 MiB using storage policy from min volume index 0 2024.10.22 01:01:01.293680 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 22260 2024.10.22 01:01:01.296825 [ 1157 ] {} DiskLocal: Reserved 1.21 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:01.302501 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_59_12 to 202410_64_64_0 into Compact with storage Full 2024.10.22 01:01:01.322405 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_305_95} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_304_94 to 202410_305_305_0 into Compact with storage Full 2024.10.22 01:01:01.486945 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4b3e6dfaec156516bd6ff0e287d3caa3) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 4b3e6dfaec156516bd6ff0e287d3caa3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:01.488362 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4b3e6dfaec156516bd6ff0e287d3caa3): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:01:01.500701 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 6.93 MiB, will set to 1.70 GiB (RSS), difference: -11.75 MiB 2024.10.22 01:01:01.585831 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 265 2024.10.22 01:01:01.679364 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_305_95} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:01.689629 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_305_95} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_304_94, total 21894 rows starting from the beginning of the part 2024.10.22 01:01:01.735270 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:01.743436 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:01.812806 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_305_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_305_305_0, total 84 rows starting from the beginning of the part 2024.10.22 01:01:01.901321 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_292_292_0 (state Temporary) 2024.10.22 01:01:02.015993 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_292_292_0 to 202410_299_299_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:02.042060 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 488622 2024.10.22 01:01:02.150574 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:02.261452 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:02.326476 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a82287fbe8dabcc4861824c6260f997b) : [0, 61370]: Setting download as failed: File segment: [0, 61370], key: a82287fbe8dabcc4861824c6260f997b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:02.330996 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a82287fbe8dabcc4861824c6260f997b): No space left in cache to reserve 61371 bytes, will continue without cache download 2024.10.22 01:01:02.347829 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_247_247_0 (state Temporary) 2024.10.22 01:01:02.357759 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a82287fbe8dabcc4861824c6260f997b, offset: 0, size: 0 2024.10.22 01:01:02.363563 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a82287fbe8dabcc4861824c6260f997b for removal 2024.10.22 01:01:02.369752 [ 1476 ] {} CacheMetadata: Key a82287fbe8dabcc4861824c6260f997b is removed from metadata 2024.10.22 01:01:02.371489 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4b3e6dfaec156516bd6ff0e287d3caa3, offset: 0, size: 0 2024.10.22 01:01:02.375355 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4b3e6dfaec156516bd6ff0e287d3caa3 for removal 2024.10.22 01:01:02.379087 [ 1476 ] {} CacheMetadata: Key 4b3e6dfaec156516bd6ff0e287d3caa3 is removed from metadata 2024.10.22 01:01:02.417317 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_247_247_0 to 202410_257_257_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:02.604626 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 265 2024.10.22 01:01:02.636630 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 4422 2024.10.22 01:01:02.848930 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:02.861411 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:02.904741 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_306_306_0 (state Temporary) 2024.10.22 01:01:02.994877 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_306_306_0 to 202410_319_319_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:03.171255 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4422 2024.10.22 01:01:03.427147 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1_429_98, 202410_430_430_0, 202410_431_431_0, 202410_432_432_0] 2024.10.22 01:01:03.429024 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1_429_98, 202410_430_430_0, 202410_431_431_0, 202410_432_432_0] 2024.10.22 01:01:03.446616 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(eea6e76ee6f95892ad761941141ae4e9) : [0, 151]: Setting download as failed: File segment: [0, 151], key: eea6e76ee6f95892ad761941141ae4e9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:03.534842 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(eea6e76ee6f95892ad761941141ae4e9): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:01:03.556222 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1_429_98, 202410_430_430_0, 202410_431_431_0, 202410_432_432_0] 2024.10.22 01:01:03.574756 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 30517 2024.10.22 01:01:03.688676 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 5.09 MiB, will set to 1.71 GiB (RSS), difference: 2.84 MiB 2024.10.22 01:01:03.864204 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:03.868730 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:03.904380 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_305_95} MergeTask::MergeProjectionsStage: Merge sorted 21978 rows, containing 13 columns (13 merged, 0 gathered) in 2.592194949 sec., 8478.528981193536 rows/sec., 3.59 MiB/sec. 2024.10.22 01:01:03.918352 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_522_522_0 (state Temporary) 2024.10.22 01:01:03.988328 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_305_95} MergedBlockOutputStream: filled checksums 202410_1_305_95 (state Temporary) 2024.10.22 01:01:04.079069 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_522_522_0 to 202410_535_535_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:04.112444 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30517 2024.10.22 01:01:04.121088 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_305_95} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_305_95 to 202410_1_305_95 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:04.139665 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_305_95} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_304_94, 202410_305_305_0] -> 202410_1_305_95 2024.10.22 01:01:04.178637 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.33 MiB. 2024.10.22 01:01:04.199282 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_530_157 to 202410_531_531_0 2024.10.22 01:01:04.304311 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.08 MiB using storage policy from min volume index 0 2024.10.22 01:01:04.306741 [ 1182 ] {} DiskLocal: Reserved 1.08 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:04.320773 [ 1182 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 01:01:04.329680 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_531_158} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_530_157 to 202410_531_531_0 into Compact with storage Full 2024.10.22 01:01:04.339176 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a2bf7a25bc1d4fe6b50f10032c35692b) : [0, 37113]: Setting download as failed: File segment: [0, 37113], key: a2bf7a25bc1d4fe6b50f10032c35692b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:04.343029 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a2bf7a25bc1d4fe6b50f10032c35692b): No space left in cache to reserve 37114 bytes, will continue without cache download 2024.10.22 01:01:04.383846 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a2bf7a25bc1d4fe6b50f10032c35692b, offset: 0, size: 0 2024.10.22 01:01:04.386123 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a2bf7a25bc1d4fe6b50f10032c35692b for removal 2024.10.22 01:01:04.389599 [ 1476 ] {} CacheMetadata: Key a2bf7a25bc1d4fe6b50f10032c35692b is removed from metadata 2024.10.22 01:01:04.394250 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eea6e76ee6f95892ad761941141ae4e9, offset: 0, size: 0 2024.10.22 01:01:04.395908 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eea6e76ee6f95892ad761941141ae4e9 for removal 2024.10.22 01:01:04.400862 [ 1476 ] {} CacheMetadata: Key eea6e76ee6f95892ad761941141ae4e9 is removed from metadata 2024.10.22 01:01:04.602905 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_531_158} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:04.630675 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_531_158} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_530_157, total 30984 rows starting from the beginning of the part 2024.10.22 01:01:04.514582 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2452 2024.10.22 01:01:04.657148 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_246_59, 202410_247_247_0] 2024.10.22 01:01:04.659459 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_246_59, 202410_247_247_0] 2024.10.22 01:01:04.685582 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_246_59, 202410_247_247_0] 2024.10.22 01:01:04.724783 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_531_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_531_531_0, total 86 rows starting from the beginning of the part 2024.10.22 01:01:06.020357 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(60ee0051d18cad2fde8763aa309808c7) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 60ee0051d18cad2fde8763aa309808c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:06.025146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(60ee0051d18cad2fde8763aa309808c7): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:01:06.490244 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 5.70 MiB, will set to 1.71 GiB (RSS), difference: 3.16 MiB 2024.10.22 01:01:06.728647 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_236_47, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0, 202410_241_241_0] 2024.10.22 01:01:06.738882 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_236_47, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0, 202410_241_241_0] 2024.10.22 01:01:06.776974 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_236_47, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0, 202410_240_240_0, 202410_241_241_0] 2024.10.22 01:01:06.797347 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1fe129330748ba02caba76308af52600) : [0, 9432]: Setting download as failed: File segment: [0, 9432], key: 1fe129330748ba02caba76308af52600, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:06.898824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1fe129330748ba02caba76308af52600): No space left in cache to reserve 9433 bytes, will continue without cache download 2024.10.22 01:01:06.899104 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_531_158} MergeTask::MergeProjectionsStage: Merge sorted 31070 rows, containing 13 columns (13 merged, 0 gathered) in 2.575759051 sec., 12062.463679565655 rows/sec., 1.84 MiB/sec. 2024.10.22 01:01:06.942872 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1fe129330748ba02caba76308af52600, offset: 0, size: 0 2024.10.22 01:01:06.944792 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1fe129330748ba02caba76308af52600 for removal 2024.10.22 01:01:06.947406 [ 1476 ] {} CacheMetadata: Key 1fe129330748ba02caba76308af52600 is removed from metadata 2024.10.22 01:01:06.948929 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 60ee0051d18cad2fde8763aa309808c7, offset: 0, size: 0 2024.10.22 01:01:06.959447 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 60ee0051d18cad2fde8763aa309808c7 for removal 2024.10.22 01:01:06.962605 [ 1476 ] {} CacheMetadata: Key 60ee0051d18cad2fde8763aa309808c7 is removed from metadata 2024.10.22 01:01:07.078490 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_531_158} MergedBlockOutputStream: filled checksums 202410_1_531_158 (state Temporary) 2024.10.22 01:01:07.372857 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_531_158} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_531_158 to 202410_1_531_158 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:07.380170 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_531_158} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_530_157, 202410_531_531_0] -> 202410_1_531_158 2024.10.22 01:01:07.495286 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.85 MiB. 2024.10.22 01:01:07.752199 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6ee55f836d310be052a6de085272fa90) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 6ee55f836d310be052a6de085272fa90, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:07.755117 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6ee55f836d310be052a6de085272fa90): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:01:08.121548 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 30572 2024.10.22 01:01:08.345134 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:08.355778 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:08.391766 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_523_523_0 (state Temporary) 2024.10.22 01:01:08.419021 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6fabc8ddca82a49be897038ce9ad9552) : [0, 127995]: Setting download as failed: File segment: [0, 127995], key: 6fabc8ddca82a49be897038ce9ad9552, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:08.420428 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6fabc8ddca82a49be897038ce9ad9552): No space left in cache to reserve 127996 bytes, will continue without cache download 2024.10.22 01:01:08.439573 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_523_523_0 to 202410_536_536_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:08.496110 [ 1272 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_531_158 to 202410_532_532_0 2024.10.22 01:01:08.498545 [ 1272 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.08 MiB using storage policy from min volume index 0 2024.10.22 01:01:08.500730 [ 1272 ] {} DiskLocal: Reserved 1.08 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:08.532701 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_532_159} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_531_158 to 202410_532_532_0 into Compact with storage Full 2024.10.22 01:01:08.661126 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30572 2024.10.22 01:01:08.687253 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_532_159} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:08.693326 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_532_159} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_531_158, total 31070 rows starting from the beginning of the part 2024.10.22 01:01:08.693657 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 4.62 MiB, will set to 1.71 GiB (RSS), difference: 4.29 MiB 2024.10.22 01:01:08.715671 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_532_159} MergeTreeSequentialSource: Reading 2 marks from part 202410_532_532_0, total 76 rows starting from the beginning of the part 2024.10.22 01:01:08.766609 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6fabc8ddca82a49be897038ce9ad9552, offset: 0, size: 0 2024.10.22 01:01:08.768518 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6fabc8ddca82a49be897038ce9ad9552 for removal 2024.10.22 01:01:08.771998 [ 1476 ] {} CacheMetadata: Key 6fabc8ddca82a49be897038ce9ad9552 is removed from metadata 2024.10.22 01:01:08.775241 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6ee55f836d310be052a6de085272fa90, offset: 0, size: 0 2024.10.22 01:01:08.776293 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6ee55f836d310be052a6de085272fa90 for removal 2024.10.22 01:01:08.779656 [ 1476 ] {} CacheMetadata: Key 6ee55f836d310be052a6de085272fa90 is removed from metadata 2024.10.22 01:01:08.886291 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 109 entries to flush up to offset 22369 2024.10.22 01:01:09.046963 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1369 entries to flush up to offset 489991 2024.10.22 01:01:09.081519 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:09.185299 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:09.233498 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_304_304_0 (state Temporary) 2024.10.22 01:01:09.306139 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_204_42, 202410_205_205_0, 202410_206_206_0, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0] 2024.10.22 01:01:09.307410 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_204_42, 202410_205_205_0, 202410_206_206_0, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0] 2024.10.22 01:01:09.311311 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_304_304_0 to 202410_310_310_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:09.346665 [ 1263 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_305_95 to 202410_306_306_0 2024.10.22 01:01:09.346631 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_204_42, 202410_205_205_0, 202410_206_206_0, 202410_207_207_0, 202410_208_208_0, 202410_209_209_0] 2024.10.22 01:01:09.357226 [ 1263 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.22 MiB using storage policy from min volume index 0 2024.10.22 01:01:09.360561 [ 1263 ] {} DiskLocal: Reserved 1.22 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:09.375674 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 22369 2024.10.22 01:01:09.396792 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_306_96} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_305_95 to 202410_306_306_0 into Compact with storage Full 2024.10.22 01:01:09.491467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(15d20f625be381fba1e800c6491c8203) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 15d20f625be381fba1e800c6491c8203, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:09.492744 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(15d20f625be381fba1e800c6491c8203): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:01:09.570207 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_306_96} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:09.646627 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:09.660094 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:09.681357 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_306_96} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_305_95, total 21978 rows starting from the beginning of the part 2024.10.22 01:01:09.751150 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_306_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_306_306_0, total 90 rows starting from the beginning of the part 2024.10.22 01:01:09.790953 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_293_293_0 (state Temporary) 2024.10.22 01:01:09.823080 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:01:09.825389 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:01:09.847248 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_293_293_0 to 202410_300_300_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:10.087104 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 489991 2024.10.22 01:01:10.105995 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 266 2024.10.22 01:01:10.318789 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_532_159} MergeTask::MergeProjectionsStage: Merge sorted 31146 rows, containing 13 columns (13 merged, 0 gathered) in 1.786245832 sec., 17436.56972743044 rows/sec., 2.66 MiB/sec. 2024.10.22 01:01:10.390359 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_532_159} MergedBlockOutputStream: filled checksums 202410_1_532_159 (state Temporary) 2024.10.22 01:01:10.417141 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:10.420738 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:10.439702 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_532_159} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_532_159 to 202410_1_532_159 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:10.447470 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_532_159} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_531_158, 202410_532_532_0] -> 202410_1_532_159 2024.10.22 01:01:10.481473 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.67 MiB. 2024.10.22 01:01:10.575236 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b2ece9ebbc5d31f3a9395f3e4245fcbe) : [0, 2839]: Setting download as failed: File segment: [0, 2839], key: b2ece9ebbc5d31f3a9395f3e4245fcbe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:10.576696 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b2ece9ebbc5d31f3a9395f3e4245fcbe): No space left in cache to reserve 2840 bytes, will continue without cache download 2024.10.22 01:01:10.586534 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 4.49 MiB, will set to 1.72 GiB (RSS), difference: 3.55 MiB 2024.10.22 01:01:10.586865 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b2ece9ebbc5d31f3a9395f3e4245fcbe, offset: 0, size: 0 2024.10.22 01:01:10.588159 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b2ece9ebbc5d31f3a9395f3e4245fcbe for removal 2024.10.22 01:01:10.590779 [ 1476 ] {} CacheMetadata: Key b2ece9ebbc5d31f3a9395f3e4245fcbe is removed from metadata 2024.10.22 01:01:10.592119 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 15d20f625be381fba1e800c6491c8203, offset: 0, size: 0 2024.10.22 01:01:10.592084 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_248_248_0 (state Temporary) 2024.10.22 01:01:10.593498 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 15d20f625be381fba1e800c6491c8203 for removal 2024.10.22 01:01:10.595548 [ 1476 ] {} CacheMetadata: Key 15d20f625be381fba1e800c6491c8203 is removed from metadata 2024.10.22 01:01:10.672625 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 4436 2024.10.22 01:01:10.775904 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_306_96} MergeTask::MergeProjectionsStage: Merge sorted 22068 rows, containing 13 columns (13 merged, 0 gathered) in 1.38998838 sec., 15876.391714871746 rows/sec., 6.73 MiB/sec. 2024.10.22 01:01:10.796647 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:10.799672 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:10.833784 [ 1208 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 222 ms. 2024.10.22 01:01:10.838443 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_307_307_0 (state Temporary) 2024.10.22 01:01:10.893779 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_307_307_0 to 202410_320_320_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:10.936026 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_306_96} MergedBlockOutputStream: filled checksums 202410_1_306_96 (state Temporary) 2024.10.22 01:01:10.973586 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_248_248_0 to 202410_258_258_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:11.037081 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 266 2024.10.22 01:01:11.036968 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4436 2024.10.22 01:01:11.081751 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_306_96} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_306_96 to 202410_1_306_96 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:11.084361 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_306_96} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_305_95, 202410_306_306_0] -> 202410_1_306_96 2024.10.22 01:01:11.005838 [ 1210 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_253_52 to 202410_258_258_0 2024.10.22 01:01:11.107616 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a27cbc0920f01172867fd8a9c450834d) : [0, 151]: Setting download as failed: File segment: [0, 151], key: a27cbc0920f01172867fd8a9c450834d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:11.109335 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a27cbc0920f01172867fd8a9c450834d): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:01:11.111470 [ 1210 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:11.113329 [ 1210 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:11.116728 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.33 MiB. 2024.10.22 01:01:11.139862 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_253_52 to 202410_258_258_0 into Compact with storage Full 2024.10.22 01:01:11.394218 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:11.433995 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_253_52, total 297 rows starting from the beginning of the part 2024.10.22 01:01:11.704478 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_254_254_0, total 1 rows starting from the beginning of the part 2024.10.22 01:01:11.879462 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_255_255_0, total 1 rows starting from the beginning of the part 2024.10.22 01:01:12.053096 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1d7d9cdf2efd3ada4b7617d8f191e30e) : [0, 3829]: Setting download as failed: File segment: [0, 3829], key: 1d7d9cdf2efd3ada4b7617d8f191e30e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:12.056826 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1d7d9cdf2efd3ada4b7617d8f191e30e): No space left in cache to reserve 3830 bytes, will continue without cache download 2024.10.22 01:01:12.084425 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_256_256_0, total 1 rows starting from the beginning of the part 2024.10.22 01:01:12.157413 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 2491 2024.10.22 01:01:12.183272 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1d7d9cdf2efd3ada4b7617d8f191e30e, offset: 0, size: 0 2024.10.22 01:01:12.185172 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1d7d9cdf2efd3ada4b7617d8f191e30e for removal 2024.10.22 01:01:12.189243 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a27cbc0920f01172867fd8a9c450834d, offset: 0, size: 0 2024.10.22 01:01:12.190823 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a27cbc0920f01172867fd8a9c450834d for removal 2024.10.22 01:01:12.195864 [ 1476 ] {} CacheMetadata: Key 1d7d9cdf2efd3ada4b7617d8f191e30e is removed from metadata 2024.10.22 01:01:12.200420 [ 1476 ] {} CacheMetadata: Key a27cbc0920f01172867fd8a9c450834d is removed from metadata 2024.10.22 01:01:12.473632 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_257_257_0, total 1 rows starting from the beginning of the part 2024.10.22 01:01:12.578913 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/cku/suotwqdzaqkgweepwawbfznftrvtq, upload id 6755a331-9324-40b3-ab88-8585eef3da56 2024.10.22 01:01:12.673704 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 102 entries to flush up to offset 30674 2024.10.22 01:01:12.789216 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_258_258_0, total 1 rows starting from the beginning of the part 2024.10.22 01:01:12.835092 [ 1235 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:01:12.886916 [ 1235 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:01:13.009218 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_247_60, 202410_248_248_0] 2024.10.22 01:01:13.018067 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_247_60, 202410_248_248_0] 2024.10.22 01:01:13.027944 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_247_60, 202410_248_248_0] 2024.10.22 01:01:13.090716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0d0d5387d1dc2af41e4bd4c025dde04d) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 0d0d5387d1dc2af41e4bd4c025dde04d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:13.093484 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0d0d5387d1dc2af41e4bd4c025dde04d): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:01:13.163731 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:13.172981 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.94 GiB. 2024.10.22 01:01:13.274655 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_524_524_0 (state Temporary) 2024.10.22 01:01:13.364382 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_524_524_0 to 202410_537_537_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:13.393391 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_532_159 to 202410_533_533_0 2024.10.22 01:01:13.396416 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.08 MiB using storage policy from min volume index 0 2024.10.22 01:01:13.412494 [ 1223 ] {} DiskLocal: Reserved 1.08 MiB on local disk `default`, having unreserved 297.91 GiB. 2024.10.22 01:01:13.465137 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_533_160} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_532_159 to 202410_533_533_0 into Compact with storage Full 2024.10.22 01:01:13.472205 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30674 2024.10.22 01:01:13.588494 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_533_160} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:13.619400 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_533_160} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_532_159, total 31146 rows starting from the beginning of the part 2024.10.22 01:01:13.694196 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_533_160} MergeTreeSequentialSource: Reading 2 marks from part 202410_533_533_0, total 75 rows starting from the beginning of the part 2024.10.22 01:01:13.887135 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ac49dcef289c73edd9a781a7760949ac) : [0, 31754]: Setting download as failed: File segment: [0, 31754], key: ac49dcef289c73edd9a781a7760949ac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:13.888548 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ac49dcef289c73edd9a781a7760949ac): No space left in cache to reserve 31755 bytes, will continue without cache download 2024.10.22 01:01:13.902567 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ac49dcef289c73edd9a781a7760949ac, offset: 0, size: 0 2024.10.22 01:01:13.904436 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ac49dcef289c73edd9a781a7760949ac for removal 2024.10.22 01:01:13.909394 [ 1476 ] {} CacheMetadata: Key ac49dcef289c73edd9a781a7760949ac is removed from metadata 2024.10.22 01:01:13.909441 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0d0d5387d1dc2af41e4bd4c025dde04d, offset: 0, size: 0 2024.10.22 01:01:13.912536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0d0d5387d1dc2af41e4bd4c025dde04d for removal 2024.10.22 01:01:13.916152 [ 1476 ] {} CacheMetadata: Key 0d0d5387d1dc2af41e4bd4c025dde04d is removed from metadata 2024.10.22 01:01:14.196078 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:14.279664 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_256_51, 202410_257_257_0, 202410_258_258_0, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0] 2024.10.22 01:01:14.281636 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_256_51, 202410_257_257_0, 202410_258_258_0, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0] 2024.10.22 01:01:14.301601 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_256_51, 202410_257_257_0, 202410_258_258_0, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0] 2024.10.22 01:01:14.492986 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_533_160/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:01:14.534716 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} MergeTask::MergeProjectionsStage: Merge sorted 302 rows, containing 32 columns (32 merged, 0 gathered) in 3.405346491 sec., 88.68407394024563 rows/sec., 9.15 KiB/sec. 2024.10.22 01:01:14.572644 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} MergedBlockOutputStream: filled checksums 202410_1_258_53 (state Temporary) 2024.10.22 01:01:14.608611 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 2.60 MiB, will set to 1.66 GiB (RSS), difference: -25.12 MiB 2024.10.22 01:01:14.767687 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7a044e96002acbdd943ff2fae568fd0c) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 7a044e96002acbdd943ff2fae568fd0c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:14.769165 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7a044e96002acbdd943ff2fae568fd0c): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:01:15.271360 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_258_53 to 202410_1_258_53 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:15.275382 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_253_52, 202410_258_258_0] -> 202410_1_258_53 2024.10.22 01:01:15.286024 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_533_160} MergeTask::MergeProjectionsStage: Merge sorted 31221 rows, containing 13 columns (13 merged, 0 gathered) in 1.838939499 sec., 16977.72004841797 rows/sec., 2.59 MiB/sec. 2024.10.22 01:01:15.294587 [ 1387 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_258_53}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 01:01:15.349439 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_533_160} MergedBlockOutputStream: filled checksums 202410_1_533_160 (state Temporary) 2024.10.22 01:01:15.385468 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:01:15.438880 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_59_12, total 2315 rows starting from the beginning of the part 2024.10.22 01:01:15.534751 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(a453ecee9b5ad6cf7519aec91c03e8a6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a453ecee9b5ad6cf7519aec91c03e8a6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:01:15.542599 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 54, file segment info: File segment: [0, 33554431], key: a453ecee9b5ad6cf7519aec91c03e8a6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:01:15.543676 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:01:15.596993 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_533_160} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_533_160 to 202410_1_533_160 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:15.601811 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_533_160} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_532_159, 202410_533_533_0] -> 202410_1_533_160 2024.10.22 01:01:15.632448 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.67 MiB. 2024.10.22 01:01:15.751881 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_432_99, 202410_433_433_0] 2024.10.22 01:01:15.756763 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_432_99, 202410_433_433_0] 2024.10.22 01:01:15.769746 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_432_99, 202410_433_433_0] 2024.10.22 01:01:15.787401 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:01:16.145741 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1ecb492aa63513ea8ca35174c0352e20) : [0, 28772]: Setting download as failed: File segment: [0, 28772], key: 1ecb492aa63513ea8ca35174c0352e20, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:16.150268 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1ecb492aa63513ea8ca35174c0352e20): No space left in cache to reserve 28773 bytes, will continue without cache download 2024.10.22 01:01:16.166418 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1ecb492aa63513ea8ca35174c0352e20, offset: 0, size: 0 2024.10.22 01:01:16.167755 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1ecb492aa63513ea8ca35174c0352e20 for removal 2024.10.22 01:01:16.174143 [ 1476 ] {} CacheMetadata: Key 1ecb492aa63513ea8ca35174c0352e20 is removed from metadata 2024.10.22 01:01:16.173601 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7a044e96002acbdd943ff2fae568fd0c, offset: 0, size: 0 2024.10.22 01:01:16.177123 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7a044e96002acbdd943ff2fae568fd0c for removal 2024.10.22 01:01:16.180523 [ 1476 ] {} CacheMetadata: Key 7a044e96002acbdd943ff2fae568fd0c is removed from metadata 2024.10.22 01:01:16.593706 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 2.66 MiB, will set to 1.67 GiB (RSS), difference: 16.35 MiB 2024.10.22 01:01:16.930699 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(7b42a44984289b7c95f74c321c41c92c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7b42a44984289b7c95f74c321c41c92c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:01:16.936856 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 469, file segment info: File segment: [0, 33554431], key: 7b42a44984289b7c95f74c321c41c92c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:01:16.943519 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:01:16.992241 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 151 entries to flush up to offset 22520 2024.10.22 01:01:17.090690 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1319 entries to flush up to offset 491310 2024.10.22 01:01:17.090689 [ 1738 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tmv/wryvhchwfilyrchvxyktfviavixmx, size 54 2024.10.22 01:01:17.225371 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:17.232189 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.84 GiB. 2024.10.22 01:01:17.387019 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_305_305_0 (state Temporary) 2024.10.22 01:01:17.396316 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:01:17.420074 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:01:17.436089 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:01:17.484504 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:01:17.543988 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_305_305_0 to 202410_311_311_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:17.584789 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 30751 2024.10.22 01:01:17.606445 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:01:17.628065 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_306_96 to 202410_307_307_0 2024.10.22 01:01:17.634262 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 22520 2024.10.22 01:01:17.641952 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.22 MiB using storage policy from min volume index 0 2024.10.22 01:01:17.646139 [ 1265 ] {} DiskLocal: Reserved 1.22 MiB on local disk `default`, having unreserved 297.84 GiB. 2024.10.22 01:01:17.676538 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_307_97} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_306_96 to 202410_307_307_0 into Compact with storage Full 2024.10.22 01:01:17.645593 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:01:17.824327 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_307_97} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:17.831626 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_307_97} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_306_96, total 22068 rows starting from the beginning of the part 2024.10.22 01:01:17.847426 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_307_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_307_307_0, total 135 rows starting from the beginning of the part 2024.10.22 01:01:17.865419 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(735c7350b6d838600aad7aefd5810ef4) : [0, 150]: Setting download as failed: File segment: [0, 150], key: 735c7350b6d838600aad7aefd5810ef4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:17.872528 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(735c7350b6d838600aad7aefd5810ef4): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:01:17.922562 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:17.927711 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.84 GiB. 2024.10.22 01:01:17.929610 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:17.941076 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.84 GiB. 2024.10.22 01:01:17.987663 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:01:18.035246 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_60_60_0, total 42 rows starting from the beginning of the part 2024.10.22 01:01:18.079674 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_294_294_0 (state Temporary) 2024.10.22 01:01:18.100088 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kmq/rhnriyhuphhzzbfztukfuknoowuuw (LogSeriesLimiter: on interval from 2024-10-22 01:00:57 to 2024-10-22 01:01:18 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:01:18.237123 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_294_294_0 to 202410_301_301_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:18.276423 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 491310 2024.10.22 01:01:18.283925 [ 1172 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_296_59 to 202410_301_301_0 2024.10.22 01:01:18.298435 [ 1172 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:18.304227 [ 1172 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.84 GiB. 2024.10.22 01:01:18.307427 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_525_525_0 (state Temporary) 2024.10.22 01:01:18.373364 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_296_59 to 202410_301_301_0 into Compact with storage Full 2024.10.22 01:01:18.488212 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_525_525_0 to 202410_538_538_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:18.525232 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:18.541165 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 4455 2024.10.22 01:01:18.542939 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} MergeTreeSequentialSource: Reading 62 marks from part 202410_1_296_59, total 498372 rows starting from the beginning of the part 2024.10.22 01:01:18.570289 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_307_97/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:01:18.576390 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30751 2024.10.22 01:01:18.578945 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_533_160 to 202410_538_538_0 2024.10.22 01:01:18.591415 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.12 MiB using storage policy from min volume index 0 2024.10.22 01:01:18.596119 [ 1157 ] {} DiskLocal: Reserved 1.12 MiB on local disk `default`, having unreserved 297.84 GiB. 2024.10.22 01:01:18.637588 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_533_160 to 202410_538_538_0 into Compact with storage Full 2024.10.22 01:01:18.638350 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:18.643494 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.84 GiB. 2024.10.22 01:01:18.666576 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_297_297_0, total 1042 rows starting from the beginning of the part 2024.10.22 01:01:18.682224 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_298_298_0, total 1262 rows starting from the beginning of the part 2024.10.22 01:01:18.686839 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_308_308_0 (state Temporary) 2024.10.22 01:01:18.702289 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:18.702986 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_299_299_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:01:18.722318 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_533_160, total 31221 rows starting from the beginning of the part 2024.10.22 01:01:18.743166 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_300_300_0, total 1369 rows starting from the beginning of the part 2024.10.22 01:01:18.762323 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_308_308_0 to 202410_321_321_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:18.774114 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_301_301_0, total 1319 rows starting from the beginning of the part 2024.10.22 01:01:18.810516 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4455 2024.10.22 01:01:18.811240 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_534_534_0, total 58 rows starting from the beginning of the part 2024.10.22 01:01:18.812421 [ 1179 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_316_63 to 202410_321_321_0 2024.10.22 01:01:18.830802 [ 1179 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:18.839180 [ 1179 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.84 GiB. 2024.10.22 01:01:18.865315 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_316_63 to 202410_321_321_0 into Compact with storage Full 2024.10.22 01:01:18.879348 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(698e2ba3c2f852d9ddb9a09096ea93ab) : [0, 219680]: Setting download as failed: File segment: [0, 219680], key: 698e2ba3c2f852d9ddb9a09096ea93ab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:18.880992 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(698e2ba3c2f852d9ddb9a09096ea93ab): No space left in cache to reserve 219681 bytes, will continue without cache download 2024.10.22 01:01:18.900389 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 698e2ba3c2f852d9ddb9a09096ea93ab, offset: 0, size: 0 2024.10.22 01:01:18.928841 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_535_535_0, total 69 rows starting from the beginning of the part 2024.10.22 01:01:18.948216 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:18.973132 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_316_63, total 4474 rows starting from the beginning of the part 2024.10.22 01:01:18.995836 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_536_536_0, total 55 rows starting from the beginning of the part 2024.10.22 01:01:19.000605 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_317_317_0, total 29 rows starting from the beginning of the part 2024.10.22 01:01:19.002984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 698e2ba3c2f852d9ddb9a09096ea93ab for removal 2024.10.22 01:01:19.004740 [ 1476 ] {} CacheMetadata: Key 698e2ba3c2f852d9ddb9a09096ea93ab is removed from metadata 2024.10.22 01:01:19.005687 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 735c7350b6d838600aad7aefd5810ef4, offset: 0, size: 0 2024.10.22 01:01:19.006667 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 735c7350b6d838600aad7aefd5810ef4 for removal 2024.10.22 01:01:19.008697 [ 1476 ] {} CacheMetadata: Key 735c7350b6d838600aad7aefd5810ef4 is removed from metadata 2024.10.22 01:01:19.040644 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_318_318_0, total 15 rows starting from the beginning of the part 2024.10.22 01:01:19.072903 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_319_319_0, total 10 rows starting from the beginning of the part 2024.10.22 01:01:19.076130 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_537_537_0, total 102 rows starting from the beginning of the part 2024.10.22 01:01:19.133636 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_538_538_0, total 77 rows starting from the beginning of the part 2024.10.22 01:01:19.227298 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_320_320_0, total 14 rows starting from the beginning of the part 2024.10.22 01:01:19.292932 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_307_97} MergeTask::MergeProjectionsStage: Merge sorted 22203 rows, containing 13 columns (13 merged, 0 gathered) in 1.620426585 sec., 13701.947502916339 rows/sec., 5.81 MiB/sec. 2024.10.22 01:01:19.379377 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_321_321_0, total 19 rows starting from the beginning of the part 2024.10.22 01:01:19.491335 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_307_97} MergedBlockOutputStream: filled checksums 202410_1_307_97 (state Temporary) 2024.10.22 01:01:19.523536 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 4.09 MiB, will set to 1.68 GiB (RSS), difference: -17.50 MiB 2024.10.22 01:01:19.573688 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_307_97} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_307_97 to 202410_1_307_97 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:19.577244 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_307_97} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_306_96, 202410_307_307_0] -> 202410_1_307_97 2024.10.22 01:01:19.605275 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.37 MiB. 2024.10.22 01:01:19.751160 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(128d19ccddd744e5bee57d1aea6da1b0) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 128d19ccddd744e5bee57d1aea6da1b0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:19.755792 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(128d19ccddd744e5bee57d1aea6da1b0): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:01:19.806651 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_61_61_0, total 37 rows starting from the beginning of the part 2024.10.22 01:01:19.830869 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:01:19.834034 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:01:20.342925 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} MergeTask::MergeProjectionsStage: Merge sorted 4561 rows, containing 12 columns (12 merged, 0 gathered) in 1.494478329 sec., 3051.9010623940603 rows/sec., 479.18 KiB/sec. 2024.10.22 01:01:20.435778 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(504a9f481fda83814f3a37cedc44d855) : [0, 119875]: Setting download as failed: File segment: [0, 119875], key: 504a9f481fda83814f3a37cedc44d855, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:20.438633 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(504a9f481fda83814f3a37cedc44d855): No space left in cache to reserve 119876 bytes, will continue without cache download 2024.10.22 01:01:20.500951 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} MergedBlockOutputStream: filled checksums 202410_1_321_64 (state Temporary) 2024.10.22 01:01:20.584048 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 504a9f481fda83814f3a37cedc44d855, offset: 0, size: 0 2024.10.22 01:01:20.586781 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 504a9f481fda83814f3a37cedc44d855 for removal 2024.10.22 01:01:20.596971 [ 1476 ] {} CacheMetadata: Key 504a9f481fda83814f3a37cedc44d855 is removed from metadata 2024.10.22 01:01:20.597387 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 128d19ccddd744e5bee57d1aea6da1b0, offset: 0, size: 0 2024.10.22 01:01:20.599827 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 128d19ccddd744e5bee57d1aea6da1b0 for removal 2024.10.22 01:01:20.603394 [ 1476 ] {} CacheMetadata: Key 128d19ccddd744e5bee57d1aea6da1b0 is removed from metadata 2024.10.22 01:01:20.771038 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_321_64 to 202410_1_321_64 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:20.775081 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_321_64} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_316_63, 202410_321_321_0] -> 202410_1_321_64 2024.10.22 01:01:20.817002 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.44 MiB. 2024.10.22 01:01:21.077217 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d9c5fe7653874480d13eb250620f5bdd) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d9c5fe7653874480d13eb250620f5bdd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:01:21.081147 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1357, file segment info: File segment: [0, 33554431], key: d9c5fe7653874480d13eb250620f5bdd, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:01:21.083684 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:01:21.186694 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_433_100, 202410_434_434_0] 2024.10.22 01:01:21.189318 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_433_100, 202410_434_434_0] 2024.10.22 01:01:21.199281 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_433_100, 202410_434_434_0] 2024.10.22 01:01:21.199716 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Referer 2024.10.22 01:01:21.211090 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Referer 2024.10.22 01:01:21.228376 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Referer 2024.10.22 01:01:21.263336 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Referer 2024.10.22 01:01:21.277554 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Referer 2024.10.22 01:01:21.288172 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Referer 2024.10.22 01:01:21.399305 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Referer 2024.10.22 01:01:21.427307 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Referer 2024.10.22 01:01:21.443002 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Referer 2024.10.22 01:01:21.456483 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 9.52 MiB, will set to 1.68 GiB (RSS), difference: 22.29 MiB 2024.10.22 01:01:21.581314 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_248_61, 202410_249_249_0] 2024.10.22 01:01:21.583516 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_248_61, 202410_249_249_0] 2024.10.22 01:01:21.595114 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_248_61, 202410_249_249_0] 2024.10.22 01:01:21.619352 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} MergeTask::MergeProjectionsStage: Merge sorted 31582 rows, containing 13 columns (13 merged, 0 gathered) in 2.999386065 sec., 10529.488140433832 rows/sec., 1.61 MiB/sec. 2024.10.22 01:01:21.813102 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} MergedBlockOutputStream: filled checksums 202410_1_538_161 (state Temporary) 2024.10.22 01:01:21.835996 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_62_62_0, total 40 rows starting from the beginning of the part 2024.10.22 01:01:21.884289 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_538_161 to 202410_1_538_161 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:21.887603 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_538_161} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_533_160, 202410_538_538_0] -> 202410_1_538_161 2024.10.22 01:01:21.910852 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.94 MiB. 2024.10.22 01:01:21.929344 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d7f44c529118505217c28a71da1f3651) : [0, 162]: Setting download as failed: File segment: [0, 162], key: d7f44c529118505217c28a71da1f3651, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:21.932925 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d7f44c529118505217c28a71da1f3651): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:01:22.582781 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 127 entries to flush up to offset 30878 2024.10.22 01:01:22.882747 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a270fbc414c4e754878b3cc6361a4c74) : [0, 80364]: Setting download as failed: File segment: [0, 80364], key: a270fbc414c4e754878b3cc6361a4c74, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:22.884593 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a270fbc414c4e754878b3cc6361a4c74): No space left in cache to reserve 80365 bytes, will continue without cache download 2024.10.22 01:01:23.001463 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a270fbc414c4e754878b3cc6361a4c74, offset: 0, size: 0 2024.10.22 01:01:23.003182 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a270fbc414c4e754878b3cc6361a4c74 for removal 2024.10.22 01:01:23.011391 [ 1476 ] {} CacheMetadata: Key a270fbc414c4e754878b3cc6361a4c74 is removed from metadata 2024.10.22 01:01:23.015391 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d7f44c529118505217c28a71da1f3651, offset: 0, size: 0 2024.10.22 01:01:23.020376 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d7f44c529118505217c28a71da1f3651 for removal 2024.10.22 01:01:23.029582 [ 1476 ] {} CacheMetadata: Key d7f44c529118505217c28a71da1f3651 is removed from metadata 2024.10.22 01:01:23.209467 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:23.216080 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:23.402173 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_526_526_0 (state Temporary) 2024.10.22 01:01:23.590078 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_301_60/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:01:23.782570 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_526_526_0 to 202410_539_539_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:23.799450 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 11.46 MiB, will set to 1.69 GiB (RSS), difference: -11.23 MiB 2024.10.22 01:01:23.943352 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30878 2024.10.22 01:01:24.031300 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a6e3e0006a889f854a642b980876c2d5) : [0, 152]: Setting download as failed: File segment: [0, 152], key: a6e3e0006a889f854a642b980876c2d5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:24.033067 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a6e3e0006a889f854a642b980876c2d5): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:01:24.106752 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_63_63_0, total 41 rows starting from the beginning of the part 2024.10.22 01:01:24.692055 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9f4006fcbfe42465bb21c06066a405d0) : [0, 81187]: Setting download as failed: File segment: [0, 81187], key: 9f4006fcbfe42465bb21c06066a405d0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:24.693210 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9f4006fcbfe42465bb21c06066a405d0): No space left in cache to reserve 81188 bytes, will continue without cache download 2024.10.22 01:01:24.704638 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9f4006fcbfe42465bb21c06066a405d0, offset: 0, size: 0 2024.10.22 01:01:24.706620 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9f4006fcbfe42465bb21c06066a405d0 for removal 2024.10.22 01:01:24.710399 [ 1476 ] {} CacheMetadata: Key 9f4006fcbfe42465bb21c06066a405d0 is removed from metadata 2024.10.22 01:01:24.710405 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a6e3e0006a889f854a642b980876c2d5, offset: 0, size: 0 2024.10.22 01:01:24.712042 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a6e3e0006a889f854a642b980876c2d5 for removal 2024.10.22 01:01:24.715657 [ 1476 ] {} CacheMetadata: Key a6e3e0006a889f854a642b980876c2d5 is removed from metadata 2024.10.22 01:01:25.169357 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 166 entries to flush up to offset 22686 2024.10.22 01:01:25.280402 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 492846 2024.10.22 01:01:25.490617 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 16.11 MiB, will set to 1.70 GiB (RSS), difference: 13.70 MiB 2024.10.22 01:01:25.491675 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_64_64_0, total 39 rows starting from the beginning of the part 2024.10.22 01:01:25.501255 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:25.503244 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:25.541682 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_306_306_0 (state Temporary) 2024.10.22 01:01:25.556561 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4d144e9a25b5ba93cacd85c093cc699d) : [0, 163]: Setting download as failed: File segment: [0, 163], key: 4d144e9a25b5ba93cacd85c093cc699d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:25.558477 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4d144e9a25b5ba93cacd85c093cc699d): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 01:01:25.604511 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_306_306_0 to 202410_312_312_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:25.641589 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_307_97 to 202410_312_312_0 2024.10.22 01:01:25.654999 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.27 MiB using storage policy from min volume index 0 2024.10.22 01:01:25.659838 [ 1248 ] {} DiskLocal: Reserved 1.27 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:25.672528 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 22686 2024.10.22 01:01:25.677090 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_307_97 to 202410_312_312_0 into Compact with storage Full 2024.10.22 01:01:25.714602 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:25.724636 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_307_97, total 22203 rows starting from the beginning of the part 2024.10.22 01:01:25.852184 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_308_308_0, total 78 rows starting from the beginning of the part 2024.10.22 01:01:25.886195 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_309_309_0, total 83 rows starting from the beginning of the part 2024.10.22 01:01:25.899118 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:25.903182 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:25.932480 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_310_310_0, total 109 rows starting from the beginning of the part 2024.10.22 01:01:25.999430 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_311_311_0, total 151 rows starting from the beginning of the part 2024.10.22 01:01:26.043402 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_312_312_0, total 166 rows starting from the beginning of the part 2024.10.22 01:01:26.043080 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 267 2024.10.22 01:01:26.055453 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_295_295_0 (state Temporary) 2024.10.22 01:01:26.210978 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_295_295_0 to 202410_302_302_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:26.249972 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 492846 2024.10.22 01:01:26.291413 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d6cdff6104ed493a25c0fbaf0a76f64d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d6cdff6104ed493a25c0fbaf0a76f64d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:01:26.304839 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: d6cdff6104ed493a25c0fbaf0a76f64d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:01:26.308561 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:26.310406 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:01:26.311736 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:26.316801 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 4476 2024.10.22 01:01:26.425231 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_249_249_0 (state Temporary) 2024.10.22 01:01:26.519104 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:26.520281 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:26.576392 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_309_309_0 (state Temporary) 2024.10.22 01:01:26.603486 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_249_249_0 to 202410_259_259_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:26.619853 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_434_101, 202410_435_435_0] 2024.10.22 01:01:26.624447 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_434_101, 202410_435_435_0] 2024.10.22 01:01:26.643480 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_434_101, 202410_435_435_0] 2024.10.22 01:01:26.659502 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_309_309_0 to 202410_322_322_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:26.720428 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4476 2024.10.22 01:01:26.790430 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 267 2024.10.22 01:01:27.384366 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(eb953fe7c93eeaebb659bd088d63f64a) : [0, 155397]: Setting download as failed: File segment: [0, 155397], key: eb953fe7c93eeaebb659bd088d63f64a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:27.385589 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(eb953fe7c93eeaebb659bd088d63f64a): No space left in cache to reserve 155398 bytes, will continue without cache download 2024.10.22 01:01:27.399927 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eb953fe7c93eeaebb659bd088d63f64a, offset: 0, size: 0 2024.10.22 01:01:27.401553 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eb953fe7c93eeaebb659bd088d63f64a for removal 2024.10.22 01:01:27.406076 [ 1476 ] {} CacheMetadata: Key eb953fe7c93eeaebb659bd088d63f64a is removed from metadata 2024.10.22 01:01:27.406355 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4d144e9a25b5ba93cacd85c093cc699d, offset: 0, size: 0 2024.10.22 01:01:27.408180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4d144e9a25b5ba93cacd85c093cc699d for removal 2024.10.22 01:01:27.412890 [ 1476 ] {} CacheMetadata: Key 4d144e9a25b5ba93cacd85c093cc699d is removed from metadata 2024.10.22 01:01:27.894383 [ 1205 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:01:27.935070 [ 1205 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:01:28.180491 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 30960 2024.10.22 01:01:28.278132 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f9b029cf73fe9c8a48096177db54b796) : [0, 163]: Setting download as failed: File segment: [0, 163], key: f9b029cf73fe9c8a48096177db54b796, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:28.280296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f9b029cf73fe9c8a48096177db54b796): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 01:01:28.378401 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 8.61 MiB, will set to 1.71 GiB (RSS), difference: -10.31 MiB 2024.10.22 01:01:28.580400 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} MergeTask::MergeProjectionsStage: Merge sorted 22790 rows, containing 13 columns (13 merged, 0 gathered) in 2.906998448 sec., 7839.701467910794 rows/sec., 3.33 MiB/sec. 2024.10.22 01:01:28.601049 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:28.608362 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:28.658193 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} MergedBlockOutputStream: filled checksums 202410_1_312_98 (state Temporary) 2024.10.22 01:01:28.704639 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_527_527_0 (state Temporary) 2024.10.22 01:01:28.815569 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:28.824466 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:28.890670 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_527_527_0 to 202410_540_540_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:28.932596 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(be01d98338a40c250fb1e01d0e9b03cf) : [0, 173551]: Setting download as failed: File segment: [0, 173551], key: be01d98338a40c250fb1e01d0e9b03cf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:28.934807 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(be01d98338a40c250fb1e01d0e9b03cf): No space left in cache to reserve 173552 bytes, will continue without cache download 2024.10.22 01:01:28.942177 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_312_98 to 202410_1_312_98 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:28.945171 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 30960 2024.10.22 01:01:28.947762 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_312_98} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_307_97, 202410_312_312_0] -> 202410_1_312_98 2024.10.22 01:01:28.959645 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: be01d98338a40c250fb1e01d0e9b03cf, offset: 0, size: 0 2024.10.22 01:01:28.963227 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key be01d98338a40c250fb1e01d0e9b03cf for removal 2024.10.22 01:01:28.973160 [ 1476 ] {} CacheMetadata: Key be01d98338a40c250fb1e01d0e9b03cf is removed from metadata 2024.10.22 01:01:28.973466 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f9b029cf73fe9c8a48096177db54b796, offset: 0, size: 0 2024.10.22 01:01:28.977466 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f9b029cf73fe9c8a48096177db54b796 for removal 2024.10.22 01:01:28.979966 [ 1476 ] {} CacheMetadata: Key f9b029cf73fe9c8a48096177db54b796 is removed from metadata 2024.10.22 01:01:29.004998 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.71 MiB. 2024.10.22 01:01:29.833236 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:01:29.837104 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:01:29.961270 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(428d5c1e425d912b21d81ee86283edd1) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 428d5c1e425d912b21d81ee86283edd1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:29.965290 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(428d5c1e425d912b21d81ee86283edd1): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:01:30.494290 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 11.37 MiB, will set to 1.71 GiB (RSS), difference: -8.60 MiB 2024.10.22 01:01:30.759901 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(581f6692a9c851ee7c38a8416e17d690) : [0, 106799]: Setting download as failed: File segment: [0, 106799], key: 581f6692a9c851ee7c38a8416e17d690, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:30.762259 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(581f6692a9c851ee7c38a8416e17d690): No space left in cache to reserve 106800 bytes, will continue without cache download 2024.10.22 01:01:30.780458 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 581f6692a9c851ee7c38a8416e17d690, offset: 0, size: 0 2024.10.22 01:01:30.782173 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 581f6692a9c851ee7c38a8416e17d690 for removal 2024.10.22 01:01:30.785486 [ 1476 ] {} CacheMetadata: Key 581f6692a9c851ee7c38a8416e17d690 is removed from metadata 2024.10.22 01:01:30.786342 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 428d5c1e425d912b21d81ee86283edd1, offset: 0, size: 0 2024.10.22 01:01:30.787966 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 428d5c1e425d912b21d81ee86283edd1 for removal 2024.10.22 01:01:30.798800 [ 1476 ] {} CacheMetadata: Key 428d5c1e425d912b21d81ee86283edd1 is removed from metadata 2024.10.22 01:01:30.898942 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_249_62, 202410_250_250_0, 202410_251_251_0, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0] 2024.10.22 01:01:30.899994 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_249_62, 202410_250_250_0, 202410_251_251_0, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0] 2024.10.22 01:01:30.938322 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_249_62, 202410_250_250_0, 202410_251_251_0, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0] 2024.10.22 01:01:31.735036 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(45f6e6dc9fc6dc8e54153d6e3daa8de3) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 45f6e6dc9fc6dc8e54153d6e3daa8de3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:31.737118 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(45f6e6dc9fc6dc8e54153d6e3daa8de3): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:01:31.772615 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 01:01:31.805563 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_64_13/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:01:31.959554 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_435_102, 202410_436_436_0, 202410_437_437_0, 202410_438_438_0, 202410_439_439_0, 202410_440_440_0] 2024.10.22 01:01:31.964275 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_435_102, 202410_436_436_0, 202410_437_437_0, 202410_438_438_0, 202410_439_439_0, 202410_440_440_0] 2024.10.22 01:01:31.988590 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_435_102, 202410_436_436_0, 202410_437_437_0, 202410_438_438_0, 202410_439_439_0, 202410_440_440_0] 2024.10.22 01:01:32.136399 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} MergeTask::MergeProjectionsStage: Merge sorted 504900 rows, containing 4 columns (4 merged, 0 gathered) in 13.782330313 sec., 36633.862963200045 rows/sec., 537.09 KiB/sec. 2024.10.22 01:01:32.193328 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} MergedBlockOutputStream: filled checksums 202410_1_301_60 (state Temporary) 2024.10.22 01:01:32.286985 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_62_62_0 (state Temporary) 2024.10.22 01:01:32.383968 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_301_60 to 202410_1_301_60 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:32.386935 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_301_60} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_296_59, 202410_301_301_0] -> 202410_1_301_60 2024.10.22 01:01:32.507120 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f39d6368a6dcd7fabdfc16210a2e5977) : [0, 1071126]: Setting download as failed: File segment: [0, 1071126], key: f39d6368a6dcd7fabdfc16210a2e5977, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:32.511264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f39d6368a6dcd7fabdfc16210a2e5977): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:01:32.519081 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:01:32.545161 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f39d6368a6dcd7fabdfc16210a2e5977, offset: 0, size: 0 2024.10.22 01:01:32.547613 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f39d6368a6dcd7fabdfc16210a2e5977 for removal 2024.10.22 01:01:32.552986 [ 1476 ] {} CacheMetadata: Key f39d6368a6dcd7fabdfc16210a2e5977 is removed from metadata 2024.10.22 01:01:32.557173 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 45f6e6dc9fc6dc8e54153d6e3daa8de3, offset: 0, size: 0 2024.10.22 01:01:32.560500 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 45f6e6dc9fc6dc8e54153d6e3daa8de3 for removal 2024.10.22 01:01:32.566699 [ 1476 ] {} CacheMetadata: Key 45f6e6dc9fc6dc8e54153d6e3daa8de3 is removed from metadata 2024.10.22 01:01:32.611741 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 14.16 MiB, will set to 1.71 GiB (RSS), difference: 12.85 MiB 2024.10.22 01:01:32.951500 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 31022 2024.10.22 01:01:33.183112 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:33.184384 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:33.191261 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 95 entries to flush up to offset 22781 2024.10.22 01:01:33.227608 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_528_528_0 (state Temporary) 2024.10.22 01:01:33.243772 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2b042c2f9c26fdf5ef89f52af9f369e8) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 2b042c2f9c26fdf5ef89f52af9f369e8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:33.245055 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2b042c2f9c26fdf5ef89f52af9f369e8): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:01:33.253568 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 493998 2024.10.22 01:01:33.268893 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_528_528_0 to 202410_541_541_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:33.328241 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:33.330530 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:33.399015 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_307_307_0 (state Temporary) 2024.10.22 01:01:33.407880 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31022 2024.10.22 01:01:33.445593 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_307_307_0 to 202410_313_313_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:33.485353 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 22781 2024.10.22 01:01:33.877663 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:33.882671 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:33.980047 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_296_296_0 (state Temporary) 2024.10.22 01:01:34.041051 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(44eff8dccc90ad81a4522be203814318) : [0, 57105]: Setting download as failed: File segment: [0, 57105], key: 44eff8dccc90ad81a4522be203814318, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:34.043543 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(44eff8dccc90ad81a4522be203814318): No space left in cache to reserve 57106 bytes, will continue without cache download 2024.10.22 01:01:34.045807 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_296_296_0 to 202410_303_303_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:34.049367 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 44eff8dccc90ad81a4522be203814318, offset: 0, size: 0 2024.10.22 01:01:34.050493 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 44eff8dccc90ad81a4522be203814318 for removal 2024.10.22 01:01:34.053618 [ 1476 ] {} CacheMetadata: Key 44eff8dccc90ad81a4522be203814318 is removed from metadata 2024.10.22 01:01:34.061085 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2b042c2f9c26fdf5ef89f52af9f369e8, offset: 0, size: 0 2024.10.22 01:01:34.068469 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2b042c2f9c26fdf5ef89f52af9f369e8 for removal 2024.10.22 01:01:34.072899 [ 1476 ] {} CacheMetadata: Key 2b042c2f9c26fdf5ef89f52af9f369e8 is removed from metadata 2024.10.22 01:01:34.089561 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 493998 2024.10.22 01:01:34.231628 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 4496 2024.10.22 01:01:34.295005 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 268 2024.10.22 01:01:34.376246 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:34.380598 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:34.410264 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_310_310_0 (state Temporary) 2024.10.22 01:01:34.456893 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_310_310_0 to 202410_323_323_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:34.495604 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:34.498308 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:34.646417 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.88 GiB, free memory in arenas 13.15 MiB, will set to 1.71 GiB (RSS), difference: -17.25 MiB 2024.10.22 01:01:34.694738 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4496 2024.10.22 01:01:34.771300 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_250_250_0 (state Temporary) 2024.10.22 01:01:34.954930 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2e8951f448c30da49dbb85d9ebfa7516) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 2e8951f448c30da49dbb85d9ebfa7516, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:34.960174 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2e8951f448c30da49dbb85d9ebfa7516): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:01:35.171114 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_250_250_0 to 202410_260_260_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:35.409466 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 268 2024.10.22 01:01:36.290345 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(179144ba4e05581a34e8e28861e2cdbc) : [0, 49664]: Setting download as failed: File segment: [0, 49664], key: 179144ba4e05581a34e8e28861e2cdbc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:36.292072 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(179144ba4e05581a34e8e28861e2cdbc): No space left in cache to reserve 49665 bytes, will continue without cache download 2024.10.22 01:01:36.302522 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 179144ba4e05581a34e8e28861e2cdbc, offset: 0, size: 0 2024.10.22 01:01:36.304842 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 179144ba4e05581a34e8e28861e2cdbc for removal 2024.10.22 01:01:36.307804 [ 1476 ] {} CacheMetadata: Key 179144ba4e05581a34e8e28861e2cdbc is removed from metadata 2024.10.22 01:01:36.310085 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2e8951f448c30da49dbb85d9ebfa7516, offset: 0, size: 0 2024.10.22 01:01:36.313696 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2e8951f448c30da49dbb85d9ebfa7516 for removal 2024.10.22 01:01:36.317198 [ 1476 ] {} CacheMetadata: Key 2e8951f448c30da49dbb85d9ebfa7516 is removed from metadata 2024.10.22 01:01:36.471555 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 10.92 MiB, will set to 1.72 GiB (RSS), difference: 2.99 MiB 2024.10.22 01:01:36.967466 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(23bda489b4b3b2fd42737590134725c2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 23bda489b4b3b2fd42737590134725c2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:01:36.971786 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 58, file segment info: File segment: [0, 33554431], key: 23bda489b4b3b2fd42737590134725c2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:01:36.978194 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:01:37.417526 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 31077 2024.10.22 01:01:37.594894 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:37.596776 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:37.756709 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_529_529_0 (state Temporary) 2024.10.22 01:01:37.797655 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_529_529_0 to 202410_542_542_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:37.848083 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31077 2024.10.22 01:01:37.857603 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f3c114b76cf2c23b388e658e39acfd7e) : [0, 147]: Setting download as failed: File segment: [0, 147], key: f3c114b76cf2c23b388e658e39acfd7e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:37.860766 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f3c114b76cf2c23b388e658e39acfd7e): No space left in cache to reserve 148 bytes, will continue without cache download 2024.10.22 01:01:38.564904 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(00ed69f37fc99153adfcc47ce3d74340) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 00ed69f37fc99153adfcc47ce3d74340, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:01:38.572583 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 54, file segment info: File segment: [0, 33554431], key: 00ed69f37fc99153adfcc47ce3d74340, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:01:38.575620 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:01:39.205916 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(65e360f0a29a384bcc26e9b691a9b422) : [0, 95293]: Setting download as failed: File segment: [0, 95293], key: 65e360f0a29a384bcc26e9b691a9b422, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:39.209444 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(65e360f0a29a384bcc26e9b691a9b422): No space left in cache to reserve 95294 bytes, will continue without cache download 2024.10.22 01:01:39.334663 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 65e360f0a29a384bcc26e9b691a9b422, offset: 0, size: 0 2024.10.22 01:01:39.341103 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 65e360f0a29a384bcc26e9b691a9b422 for removal 2024.10.22 01:01:39.347165 [ 1476 ] {} CacheMetadata: Key 65e360f0a29a384bcc26e9b691a9b422 is removed from metadata 2024.10.22 01:01:39.348887 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f3c114b76cf2c23b388e658e39acfd7e, offset: 0, size: 0 2024.10.22 01:01:39.452626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f3c114b76cf2c23b388e658e39acfd7e for removal 2024.10.22 01:01:39.457486 [ 1476 ] {} CacheMetadata: Key f3c114b76cf2c23b388e658e39acfd7e is removed from metadata 2024.10.22 01:01:39.835042 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:01:39.842316 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:01:40.401599 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 9.67 MiB, will set to 1.72 GiB (RSS), difference: 9.31 MiB 2024.10.22 01:01:40.742023 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(a02450b68ca206e5d6a00364a004aa7d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a02450b68ca206e5d6a00364a004aa7d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:01:40.745989 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 469, file segment info: File segment: [0, 33554431], key: a02450b68ca206e5d6a00364a004aa7d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:01:40.749691 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:01:40.996094 [ 1705 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kmq/rhnriyhuphhzzbfztukfuknoowuuw, size 54 2024.10.22 01:01:40.998530 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 52 entries to flush up to offset 22833 2024.10.22 01:01:41.060506 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_62_62_0 to 202410_65_65_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:41.092463 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 495150 2024.10.22 01:01:41.194877 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(035fdb5394727391103b45ff42c9bed9) : [0, 149]: Setting download as failed: File segment: [0, 149], key: 035fdb5394727391103b45ff42c9bed9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:41.195602 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(035fdb5394727391103b45ff42c9bed9): No space left in cache to reserve 150 bytes, will continue without cache download 2024.10.22 01:01:41.303392 [ 1169 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:01:41.317524 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:01:41.334253 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:41.339876 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:41.347762 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:01:41.470136 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:01:41.486375 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:01:41.489338 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_308_308_0 (state Temporary) 2024.10.22 01:01:41.522804 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:01:41.555072 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:01:41.555961 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_308_308_0 to 202410_314_314_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:41.780609 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:01:41.856066 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 31120 2024.10.22 01:01:41.859746 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qsk/mutmrfmtdegmnlcarchnyuqtcrsaz (LogSeriesLimiter: on interval from 2024-10-22 01:01:18 to 2024-10-22 01:01:41 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:01:41.884754 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 22833 2024.10.22 01:01:41.999300 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:42.003075 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:42.061109 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(eefed65c5104d285dc059ae609491aa3) : [0, 141324]: Setting download as failed: File segment: [0, 141324], key: eefed65c5104d285dc059ae609491aa3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:42.064953 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(eefed65c5104d285dc059ae609491aa3): No space left in cache to reserve 141325 bytes, will continue without cache download 2024.10.22 01:01:42.069624 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:42.072899 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:42.082044 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eefed65c5104d285dc059ae609491aa3, offset: 0, size: 0 2024.10.22 01:01:42.083668 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eefed65c5104d285dc059ae609491aa3 for removal 2024.10.22 01:01:42.087348 [ 1476 ] {} CacheMetadata: Key eefed65c5104d285dc059ae609491aa3 is removed from metadata 2024.10.22 01:01:42.088153 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 035fdb5394727391103b45ff42c9bed9, offset: 0, size: 0 2024.10.22 01:01:42.089704 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 035fdb5394727391103b45ff42c9bed9 for removal 2024.10.22 01:01:42.092596 [ 1476 ] {} CacheMetadata: Key 035fdb5394727391103b45ff42c9bed9 is removed from metadata 2024.10.22 01:01:42.103090 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_297_297_0 (state Temporary) 2024.10.22 01:01:42.106629 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_530_530_0 (state Temporary) 2024.10.22 01:01:42.185437 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_297_297_0 to 202410_304_304_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:42.196396 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 4502 2024.10.22 01:01:42.211872 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 495150 2024.10.22 01:01:42.218692 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_530_530_0 to 202410_543_543_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:42.260738 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_538_161 to 202410_539_539_0 2024.10.22 01:01:42.268001 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.10 MiB using storage policy from min volume index 0 2024.10.22 01:01:42.273081 [ 1163 ] {} DiskLocal: Reserved 1.10 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:42.277288 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31120 2024.10.22 01:01:42.305407 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_539_162} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_538_161 to 202410_539_539_0 into Compact with storage Full 2024.10.22 01:01:42.418261 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.88 GiB, free memory in arenas 7.90 MiB, will set to 1.73 GiB (RSS), difference: -17.93 MiB 2024.10.22 01:01:42.488150 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_539_162} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:42.491731 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:42.493731 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:42.497634 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_539_162} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_538_161, total 31582 rows starting from the beginning of the part 2024.10.22 01:01:42.510227 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_311_311_0 (state Temporary) 2024.10.22 01:01:42.541621 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_539_162} MergeTreeSequentialSource: Reading 2 marks from part 202410_539_539_0, total 127 rows starting from the beginning of the part 2024.10.22 01:01:42.677430 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_311_311_0 to 202410_324_324_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:42.801125 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e6e48bd8c1c7c64968dc88f7b5a7c500) : [0, 152]: Setting download as failed: File segment: [0, 152], key: e6e48bd8c1c7c64968dc88f7b5a7c500, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:42.802638 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e6e48bd8c1c7c64968dc88f7b5a7c500): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:01:42.820182 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4502 2024.10.22 01:01:42.913816 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 269 2024.10.22 01:01:42.940060 [ 1189 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:01:42.988799 [ 1189 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:01:43.435396 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2491 2024.10.22 01:01:43.437192 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:43.442820 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:43.462229 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(dcd246c8378851eb60ca306eea356ab1) : [0, 79063]: Setting download as failed: File segment: [0, 79063], key: dcd246c8378851eb60ca306eea356ab1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:43.465547 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(dcd246c8378851eb60ca306eea356ab1): No space left in cache to reserve 79064 bytes, will continue without cache download 2024.10.22 01:01:43.480064 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dcd246c8378851eb60ca306eea356ab1, offset: 0, size: 0 2024.10.22 01:01:43.481556 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dcd246c8378851eb60ca306eea356ab1 for removal 2024.10.22 01:01:43.483810 [ 1476 ] {} CacheMetadata: Key dcd246c8378851eb60ca306eea356ab1 is removed from metadata 2024.10.22 01:01:43.484704 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e6e48bd8c1c7c64968dc88f7b5a7c500, offset: 0, size: 0 2024.10.22 01:01:43.487207 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e6e48bd8c1c7c64968dc88f7b5a7c500 for removal 2024.10.22 01:01:43.489240 [ 1476 ] {} CacheMetadata: Key e6e48bd8c1c7c64968dc88f7b5a7c500 is removed from metadata 2024.10.22 01:01:43.803901 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_251_251_0 (state Temporary) 2024.10.22 01:01:44.176159 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(477b932fecdf2611a6f64d89a662bf3d) : [0, 174]: Setting download as failed: File segment: [0, 174], key: 477b932fecdf2611a6f64d89a662bf3d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:44.178885 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(477b932fecdf2611a6f64d89a662bf3d): No space left in cache to reserve 175 bytes, will continue without cache download 2024.10.22 01:01:44.197569 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_251_251_0 to 202410_261_261_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:44.307638 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.88 GiB, free memory in arenas 9.27 MiB, will set to 1.73 GiB (RSS), difference: 8.12 MiB 2024.10.22 01:01:44.347694 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_539_162} MergeTask::MergeProjectionsStage: Merge sorted 31709 rows, containing 13 columns (13 merged, 0 gathered) in 2.049060006 sec., 15474.900640855123 rows/sec., 2.37 MiB/sec. 2024.10.22 01:01:44.489344 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 269 2024.10.22 01:01:44.601320 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_539_162} MergedBlockOutputStream: filled checksums 202410_1_539_162 (state Temporary) 2024.10.22 01:01:44.668858 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_539_162} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_539_162 to 202410_1_539_162 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:44.672721 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_539_162} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_538_161, 202410_539_539_0] -> 202410_1_539_162 2024.10.22 01:01:44.705486 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.69 MiB. 2024.10.22 01:01:45.295685 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d47ecdf894469f73e0f263f6d601d12c) : [0, 298823]: Setting download as failed: File segment: [0, 298823], key: d47ecdf894469f73e0f263f6d601d12c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:45.297095 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d47ecdf894469f73e0f263f6d601d12c): No space left in cache to reserve 298824 bytes, will continue without cache download 2024.10.22 01:01:45.418296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d47ecdf894469f73e0f263f6d601d12c, offset: 0, size: 0 2024.10.22 01:01:45.423180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d47ecdf894469f73e0f263f6d601d12c for removal 2024.10.22 01:01:45.428369 [ 1476 ] {} CacheMetadata: Key d47ecdf894469f73e0f263f6d601d12c is removed from metadata 2024.10.22 01:01:45.429541 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 477b932fecdf2611a6f64d89a662bf3d, offset: 0, size: 0 2024.10.22 01:01:45.536970 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 477b932fecdf2611a6f64d89a662bf3d for removal 2024.10.22 01:01:45.541360 [ 1476 ] {} CacheMetadata: Key 477b932fecdf2611a6f64d89a662bf3d is removed from metadata 2024.10.22 01:01:46.257684 [ 1273 ] {} DatabaseCatalog: Execution took 250 ms. 2024.10.22 01:01:46.290944 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 31191 2024.10.22 01:01:46.377388 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(83b66b99759078854e66c38ca78b5ed5) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 83b66b99759078854e66c38ca78b5ed5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:46.379287 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(83b66b99759078854e66c38ca78b5ed5): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:01:46.881268 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:46.884237 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:46.933193 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_531_531_0 (state Temporary) 2024.10.22 01:01:47.103993 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_531_531_0 to 202410_544_544_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:47.148484 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_539_162 to 202410_540_540_0 2024.10.22 01:01:47.154634 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.10 MiB using storage policy from min volume index 0 2024.10.22 01:01:47.167824 [ 1266 ] {} DiskLocal: Reserved 1.10 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:47.184655 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_540_163} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_539_162 to 202410_540_540_0 into Compact with storage Full 2024.10.22 01:01:47.344241 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_540_163} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:47.376656 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_540_163} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_539_162, total 31709 rows starting from the beginning of the part 2024.10.22 01:01:47.424943 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31191 2024.10.22 01:01:47.453097 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(883ce102f03adc6f209270d50e540248) : [0, 1032014]: Setting download as failed: File segment: [0, 1032014], key: 883ce102f03adc6f209270d50e540248, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:47.457536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(883ce102f03adc6f209270d50e540248): No space left in cache to reserve 1032015 bytes, will continue without cache download 2024.10.22 01:01:47.484168 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_64_13/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:01:47.495190 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 883ce102f03adc6f209270d50e540248, offset: 0, size: 0 2024.10.22 01:01:47.497049 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 883ce102f03adc6f209270d50e540248 for removal 2024.10.22 01:01:47.500909 [ 1476 ] {} CacheMetadata: Key 883ce102f03adc6f209270d50e540248 is removed from metadata 2024.10.22 01:01:47.505032 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 83b66b99759078854e66c38ca78b5ed5, offset: 0, size: 0 2024.10.22 01:01:47.506584 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 83b66b99759078854e66c38ca78b5ed5 for removal 2024.10.22 01:01:47.508400 [ 1476 ] {} CacheMetadata: Key 83b66b99759078854e66c38ca78b5ed5 is removed from metadata 2024.10.22 01:01:47.659476 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_540_163} MergeTreeSequentialSource: Reading 2 marks from part 202410_540_540_0, total 82 rows starting from the beginning of the part 2024.10.22 01:01:47.697226 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.88 GiB, free memory in arenas 5.24 MiB, will set to 1.74 GiB (RSS), difference: 7.79 MiB 2024.10.22 01:01:48.667108 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(62ee1d772a5f1ce30c6448afa98a4079) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 62ee1d772a5f1ce30c6448afa98a4079, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:48.669769 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(62ee1d772a5f1ce30c6448afa98a4079): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:01:49.287550 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 497070 2024.10.22 01:01:49.471893 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 138 entries to flush up to offset 22971 2024.10.22 01:01:49.768792 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(837f46637721a4ebe93a1b8c13f0b029) : [0, 86173]: Setting download as failed: File segment: [0, 86173], key: 837f46637721a4ebe93a1b8c13f0b029, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:49.771922 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(837f46637721a4ebe93a1b8c13f0b029): No space left in cache to reserve 86174 bytes, will continue without cache download 2024.10.22 01:01:49.812925 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:49.831636 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:01:49.832151 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_540_163} MergeTask::MergeProjectionsStage: Merge sorted 31791 rows, containing 13 columns (13 merged, 0 gathered) in 2.652429045 sec., 11985.61750781914 rows/sec., 1.84 MiB/sec. 2024.10.22 01:01:49.836166 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:01:49.921807 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.82 GiB. 2024.10.22 01:01:49.939500 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 837f46637721a4ebe93a1b8c13f0b029, offset: 0, size: 0 2024.10.22 01:01:49.941387 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 837f46637721a4ebe93a1b8c13f0b029 for removal 2024.10.22 01:01:49.943986 [ 1476 ] {} CacheMetadata: Key 837f46637721a4ebe93a1b8c13f0b029 is removed from metadata 2024.10.22 01:01:49.947236 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 62ee1d772a5f1ce30c6448afa98a4079, offset: 0, size: 0 2024.10.22 01:01:49.953480 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 62ee1d772a5f1ce30c6448afa98a4079 for removal 2024.10.22 01:01:49.987214 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_309_309_0 (state Temporary) 2024.10.22 01:01:50.065245 [ 1476 ] {} CacheMetadata: Key 62ee1d772a5f1ce30c6448afa98a4079 is removed from metadata 2024.10.22 01:01:50.186861 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_540_163} MergedBlockOutputStream: filled checksums 202410_1_540_163 (state Temporary) 2024.10.22 01:01:50.287250 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_540_163} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_540_163 to 202410_1_540_163 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:50.291264 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_540_163} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_539_162, 202410_540_540_0] -> 202410_1_540_163 2024.10.22 01:01:50.308488 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.67 MiB. 2024.10.22 01:01:50.322315 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 4512 2024.10.22 01:01:50.463123 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:50.469132 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.73 GiB. 2024.10.22 01:01:50.527807 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 3.99 MiB, will set to 1.74 GiB (RSS), difference: 32.26 MiB 2024.10.22 01:01:50.531104 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_309_309_0 to 202410_315_315_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:50.585580 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 22971 2024.10.22 01:01:50.632898 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:50.637444 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.73 GiB. 2024.10.22 01:01:50.699949 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_312_312_0 (state Temporary) 2024.10.22 01:01:50.936919 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_312_312_0 to 202410_325_325_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:50.986986 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4512 2024.10.22 01:01:50.976567 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 2529 2024.10.22 01:01:50.993362 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_298_298_0 (state Temporary) 2024.10.22 01:01:51.056771 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3f8c6a5937f8ce51a2d52167d10852db) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 3f8c6a5937f8ce51a2d52167d10852db, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:51.059780 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3f8c6a5937f8ce51a2d52167d10852db): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:01:51.219100 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_298_298_0 to 202410_305_305_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:51.324464 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_241_48, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 01:01:51.328134 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_241_48, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 01:01:51.369750 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_241_48, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0, 202410_245_245_0, 202410_246_246_0] 2024.10.22 01:01:51.379269 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 497070 2024.10.22 01:01:51.434993 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 31258 2024.10.22 01:01:51.773805 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4f4736d3d0c8ce9578406c466f10dc7e) : [0, 1070928]: Setting download as failed: File segment: [0, 1070928], key: 4f4736d3d0c8ce9578406c466f10dc7e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:51.776253 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4f4736d3d0c8ce9578406c466f10dc7e): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:01:51.800118 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4f4736d3d0c8ce9578406c466f10dc7e, offset: 0, size: 0 2024.10.22 01:01:51.802412 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4f4736d3d0c8ce9578406c466f10dc7e for removal 2024.10.22 01:01:51.805810 [ 1476 ] {} CacheMetadata: Key 4f4736d3d0c8ce9578406c466f10dc7e is removed from metadata 2024.10.22 01:01:51.887666 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:51.892011 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.73 GiB. 2024.10.22 01:01:51.908630 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3f8c6a5937f8ce51a2d52167d10852db, offset: 0, size: 0 2024.10.22 01:01:51.910420 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3f8c6a5937f8ce51a2d52167d10852db for removal 2024.10.22 01:01:51.914857 [ 1476 ] {} CacheMetadata: Key 3f8c6a5937f8ce51a2d52167d10852db is removed from metadata 2024.10.22 01:01:51.948056 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_532_532_0 (state Temporary) 2024.10.22 01:01:51.992640 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 270 2024.10.22 01:01:51.993480 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_532_532_0 to 202410_545_545_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:52.019654 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_540_163 to 202410_541_541_0 2024.10.22 01:01:52.042358 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 01:01:52.043727 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31258 2024.10.22 01:01:52.046981 [ 1209 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 297.73 GiB. 2024.10.22 01:01:52.089489 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_541_164} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_540_163 to 202410_541_541_0 into Compact with storage Full 2024.10.22 01:01:52.166376 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_541_164} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:52.177255 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_541_164} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_540_163, total 31791 rows starting from the beginning of the part 2024.10.22 01:01:52.311837 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_541_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_541_541_0, total 62 rows starting from the beginning of the part 2024.10.22 01:01:52.486995 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:52.490483 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.73 GiB. 2024.10.22 01:01:52.594168 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1078e1642b23b0b59160d6378d368224) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 1078e1642b23b0b59160d6378d368224, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:52.595273 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1078e1642b23b0b59160d6378d368224): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:01:52.597276 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.88 GiB, free memory in arenas 4.06 MiB, will set to 1.69 GiB (RSS), difference: -54.10 MiB 2024.10.22 01:01:52.653114 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_252_252_0 (state Temporary) 2024.10.22 01:01:52.725765 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_252_252_0 to 202410_262_262_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:53.093634 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 270 2024.10.22 01:01:53.344949 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_541_164/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:01:53.486507 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(02744f98bb2ef285a1d7a53bb303f13a) : [0, 81729]: Setting download as failed: File segment: [0, 81729], key: 02744f98bb2ef285a1d7a53bb303f13a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:53.487876 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(02744f98bb2ef285a1d7a53bb303f13a): No space left in cache to reserve 81730 bytes, will continue without cache download 2024.10.22 01:01:53.500842 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 02744f98bb2ef285a1d7a53bb303f13a, offset: 0, size: 0 2024.10.22 01:01:53.504355 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 02744f98bb2ef285a1d7a53bb303f13a for removal 2024.10.22 01:01:53.507867 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1078e1642b23b0b59160d6378d368224, offset: 0, size: 0 2024.10.22 01:01:53.508930 [ 1476 ] {} CacheMetadata: Key 02744f98bb2ef285a1d7a53bb303f13a is removed from metadata 2024.10.22 01:01:53.509544 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1078e1642b23b0b59160d6378d368224 for removal 2024.10.22 01:01:53.513308 [ 1476 ] {} CacheMetadata: Key 1078e1642b23b0b59160d6378d368224 is removed from metadata 2024.10.22 01:01:54.088900 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_541_164} MergeTask::MergeProjectionsStage: Merge sorted 31853 rows, containing 13 columns (13 merged, 0 gathered) in 2.005572382 sec., 15882.249020718715 rows/sec., 2.44 MiB/sec. 2024.10.22 01:01:54.316372 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4ca678b2a0deaea60a20db94df80f311) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 4ca678b2a0deaea60a20db94df80f311, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:54.319391 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4ca678b2a0deaea60a20db94df80f311): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:01:54.398972 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_541_164} MergedBlockOutputStream: filled checksums 202410_1_541_164 (state Temporary) 2024.10.22 01:01:54.584980 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_541_164} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_541_164 to 202410_1_541_164 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:54.588388 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_541_164} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_540_163, 202410_541_541_0] -> 202410_1_541_164 2024.10.22 01:01:54.784720 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_440_103, 202410_441_441_0] 2024.10.22 01:01:54.787865 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_440_103, 202410_441_441_0] 2024.10.22 01:01:54.797834 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_440_103, 202410_441_441_0] 2024.10.22 01:01:54.807641 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.67 MiB. 2024.10.22 01:01:54.886702 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f01aa9b0eff69d48172f02d38e85221c) : [0, 71808]: Setting download as failed: File segment: [0, 71808], key: f01aa9b0eff69d48172f02d38e85221c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:54.887597 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f01aa9b0eff69d48172f02d38e85221c): No space left in cache to reserve 71809 bytes, will continue without cache download 2024.10.22 01:01:54.898802 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f01aa9b0eff69d48172f02d38e85221c, offset: 0, size: 0 2024.10.22 01:01:54.900730 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f01aa9b0eff69d48172f02d38e85221c for removal 2024.10.22 01:01:54.904229 [ 1476 ] {} CacheMetadata: Key f01aa9b0eff69d48172f02d38e85221c is removed from metadata 2024.10.22 01:01:54.908188 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4ca678b2a0deaea60a20db94df80f311, offset: 0, size: 0 2024.10.22 01:01:54.912785 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4ca678b2a0deaea60a20db94df80f311 for removal 2024.10.22 01:01:54.919808 [ 1476 ] {} CacheMetadata: Key 4ca678b2a0deaea60a20db94df80f311 is removed from metadata 2024.10.22 01:01:55.321723 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 5.89 MiB, will set to 1.70 GiB (RSS), difference: 18.33 MiB 2024.10.22 01:01:56.008328 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(36053d74d8107da09f46e06c04f400fa) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 36053d74d8107da09f46e06c04f400fa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:56.009651 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(36053d74d8107da09f46e06c04f400fa): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:01:56.050835 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 31318 2024.10.22 01:01:56.135329 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_261_52, 202410_262_262_0, 202410_263_263_0, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0] 2024.10.22 01:01:56.138385 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_261_52, 202410_262_262_0, 202410_263_263_0, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0] 2024.10.22 01:01:56.245161 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:56.271326 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.73 GiB. 2024.10.22 01:01:56.271574 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_261_52, 202410_262_262_0, 202410_263_263_0, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0] 2024.10.22 01:01:56.323625 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_533_533_0 (state Temporary) 2024.10.22 01:01:56.479774 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_533_533_0 to 202410_546_546_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:56.514808 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31318 2024.10.22 01:01:56.518630 [ 1236 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_541_164 to 202410_542_542_0 2024.10.22 01:01:56.523864 [ 1236 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 01:01:56.526825 [ 1236 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 297.73 GiB. 2024.10.22 01:01:56.673230 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_542_165} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_541_164 to 202410_542_542_0 into Compact with storage Full 2024.10.22 01:01:56.756276 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_542_165} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:56.774734 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_542_165} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_541_164, total 31853 rows starting from the beginning of the part 2024.10.22 01:01:56.807759 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_542_165} MergeTreeSequentialSource: Reading 2 marks from part 202410_542_542_0, total 55 rows starting from the beginning of the part 2024.10.22 01:01:56.996008 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5d28fb178ac3ae25800d493967ef6366) : [0, 69790]: Setting download as failed: File segment: [0, 69790], key: 5d28fb178ac3ae25800d493967ef6366, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:56.997615 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5d28fb178ac3ae25800d493967ef6366): No space left in cache to reserve 69791 bytes, will continue without cache download 2024.10.22 01:01:57.018417 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5d28fb178ac3ae25800d493967ef6366, offset: 0, size: 0 2024.10.22 01:01:57.023954 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5d28fb178ac3ae25800d493967ef6366 for removal 2024.10.22 01:01:57.029217 [ 1476 ] {} CacheMetadata: Key 5d28fb178ac3ae25800d493967ef6366 is removed from metadata 2024.10.22 01:01:57.032419 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 36053d74d8107da09f46e06c04f400fa, offset: 0, size: 0 2024.10.22 01:01:57.035845 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 36053d74d8107da09f46e06c04f400fa for removal 2024.10.22 01:01:57.046172 [ 1476 ] {} CacheMetadata: Key 36053d74d8107da09f46e06c04f400fa is removed from metadata 2024.10.22 01:01:57.936482 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_209_43, 202410_210_210_0, 202410_211_211_0, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0] 2024.10.22 01:01:57.944205 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_209_43, 202410_210_210_0, 202410_211_211_0, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0] 2024.10.22 01:01:57.993101 [ 1227 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:01:58.000917 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_209_43, 202410_210_210_0, 202410_211_211_0, 202410_212_212_0, 202410_213_213_0, 202410_214_214_0] 2024.10.22 01:01:58.119932 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 114 entries to flush up to offset 23085 2024.10.22 01:01:58.151476 [ 1227 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:01:58.340841 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:58.347716 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.73 GiB. 2024.10.22 01:01:58.386458 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 498222 2024.10.22 01:01:58.441790 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_310_310_0 (state Temporary) 2024.10.22 01:01:58.489451 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 4521 2024.10.22 01:01:58.545528 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9868fb4a68e352c058cc82b998cc187c) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 9868fb4a68e352c058cc82b998cc187c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:01:58.547990 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9868fb4a68e352c058cc82b998cc187c): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:01:58.575875 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:58.578473 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.70 GiB. 2024.10.22 01:01:58.589784 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_542_165} MergeTask::MergeProjectionsStage: Merge sorted 31908 rows, containing 13 columns (13 merged, 0 gathered) in 1.928074265 sec., 16549.15507105739 rows/sec., 2.54 MiB/sec. 2024.10.22 01:01:58.609511 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_313_313_0 (state Temporary) 2024.10.22 01:01:58.744794 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_313_313_0 to 202410_326_326_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:58.786274 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_321_64 to 202410_326_326_0 2024.10.22 01:01:58.793589 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:58.795903 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.70 GiB. 2024.10.22 01:01:58.818580 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_310_310_0 to 202410_316_316_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:58.833504 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_321_64 to 202410_326_326_0 into Compact with storage Full 2024.10.22 01:01:58.882747 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 23085 2024.10.22 01:01:58.888848 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_542_165} MergedBlockOutputStream: filled checksums 202410_1_542_165 (state Temporary) 2024.10.22 01:01:58.888988 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4521 2024.10.22 01:01:58.922167 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:58.922775 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_542_165} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_542_165 to 202410_1_542_165 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:58.929670 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_542_165} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_541_164, 202410_542_542_0] -> 202410_1_542_165 2024.10.22 01:01:58.941130 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_321_64, total 4561 rows starting from the beginning of the part 2024.10.22 01:01:58.969457 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.67 MiB. 2024.10.22 01:01:59.093241 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_322_322_0, total 21 rows starting from the beginning of the part 2024.10.22 01:01:59.110912 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_323_323_0, total 20 rows starting from the beginning of the part 2024.10.22 01:01:59.162881 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_324_324_0, total 6 rows starting from the beginning of the part 2024.10.22 01:01:59.185359 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_325_325_0, total 10 rows starting from the beginning of the part 2024.10.22 01:01:59.198599 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_326_326_0, total 9 rows starting from the beginning of the part 2024.10.22 01:01:59.211275 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:59.219508 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.70 GiB. 2024.10.22 01:01:59.244625 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(67a72c81630238eaadfce21578ebf1d3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 67a72c81630238eaadfce21578ebf1d3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:01:59.273175 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 58, file segment info: File segment: [0, 33554431], key: 67a72c81630238eaadfce21578ebf1d3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:01:59.280380 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:01:59.466902 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_299_299_0 (state Temporary) 2024.10.22 01:01:59.536057 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_441_104, 202410_442_442_0] 2024.10.22 01:01:59.542810 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_441_104, 202410_442_442_0] 2024.10.22 01:01:59.559265 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_441_104, 202410_442_442_0] 2024.10.22 01:01:59.677249 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_299_299_0 to 202410_306_306_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:01:59.793706 [ 1260 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_301_60 to 202410_306_306_0 2024.10.22 01:01:59.804068 [ 1260 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:01:59.823793 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.70 GiB. 2024.10.22 01:01:59.835072 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:01:59.847324 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:01:59.961470 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_301_60 to 202410_306_306_0 into Compact with storage Full 2024.10.22 01:01:59.990525 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:01:59.996457 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} MergeTreeSequentialSource: Reading 63 marks from part 202410_1_301_60, total 504900 rows starting from the beginning of the part 2024.10.22 01:02:00.080762 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_302_302_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:02:00.087734 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_303_303_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:02:00.097548 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_304_304_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:02:00.116781 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_305_305_0, total 1920 rows starting from the beginning of the part 2024.10.22 01:02:00.132149 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_306_306_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:02:00.163322 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 498222 2024.10.22 01:02:00.306364 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2f72379f048ae152459767feae2d5373) : [0, 96331]: Setting download as failed: File segment: [0, 96331], key: 2f72379f048ae152459767feae2d5373, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:00.307692 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2f72379f048ae152459767feae2d5373): No space left in cache to reserve 96332 bytes, will continue without cache download 2024.10.22 01:02:00.337593 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} MergeTask::MergeProjectionsStage: Merge sorted 4627 rows, containing 12 columns (12 merged, 0 gathered) in 1.51101076 sec., 3062.1886504633494 rows/sec., 480.85 KiB/sec. 2024.10.22 01:02:00.366558 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2f72379f048ae152459767feae2d5373, offset: 0, size: 0 2024.10.22 01:02:00.444284 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2f72379f048ae152459767feae2d5373 for removal 2024.10.22 01:02:00.448108 [ 1476 ] {} CacheMetadata: Key 2f72379f048ae152459767feae2d5373 is removed from metadata 2024.10.22 01:02:00.449824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9868fb4a68e352c058cc82b998cc187c, offset: 0, size: 0 2024.10.22 01:02:00.452932 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9868fb4a68e352c058cc82b998cc187c for removal 2024.10.22 01:02:00.458195 [ 1476 ] {} CacheMetadata: Key 9868fb4a68e352c058cc82b998cc187c is removed from metadata 2024.10.22 01:02:00.528182 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 31411 2024.10.22 01:02:00.595036 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 271 2024.10.22 01:02:00.684729 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} MergedBlockOutputStream: filled checksums 202410_1_326_65 (state Temporary) 2024.10.22 01:02:00.764054 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_326_65 to 202410_1_326_65 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:00.767864 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_326_65} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_321_64, 202410_326_326_0] -> 202410_1_326_65 2024.10.22 01:02:00.906342 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.47 MiB. 2024.10.22 01:02:00.965149 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(5bf87b550953995e2ba638525f0975e4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5bf87b550953995e2ba638525f0975e4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:02:00.971138 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 54, file segment info: File segment: [0, 33554431], key: 5bf87b550953995e2ba638525f0975e4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:02:00.976451 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:01.020193 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:01.023466 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.70 GiB. 2024.10.22 01:02:01.189732 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_253_253_0 (state Temporary) 2024.10.22 01:02:01.211502 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:01.218517 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.69 GiB. 2024.10.22 01:02:01.398886 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_534_534_0 (state Temporary) 2024.10.22 01:02:01.442942 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_253_253_0 to 202410_263_263_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:01.475244 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 8.52 MiB, will set to 1.69 GiB (RSS), difference: -10.51 MiB 2024.10.22 01:02:01.479045 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_534_534_0 to 202410_547_547_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:01.483956 [ 1156 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_258_53 to 202410_263_263_0 2024.10.22 01:02:01.493332 [ 1156 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:01.496002 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.70 GiB. 2024.10.22 01:02:01.500705 [ 1178 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_542_165 to 202410_543_543_0 2024.10.22 01:02:01.506520 [ 1178 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 01:02:01.513002 [ 1178 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 297.69 GiB. 2024.10.22 01:02:01.531750 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_258_53 to 202410_263_263_0 into Compact with storage Full 2024.10.22 01:02:01.533495 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(300e9689445d3bfd6e0e28357d0008aa) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 300e9689445d3bfd6e0e28357d0008aa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:01.537004 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(300e9689445d3bfd6e0e28357d0008aa): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:02:01.539374 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_543_166} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_542_165 to 202410_543_543_0 into Compact with storage Full 2024.10.22 01:02:01.540624 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31411 2024.10.22 01:02:01.597806 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_543_166} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:01.605772 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_543_166} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_542_165, total 31908 rows starting from the beginning of the part 2024.10.22 01:02:01.627477 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 271 2024.10.22 01:02:01.682300 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_543_166} MergeTreeSequentialSource: Reading 2 marks from part 202410_543_543_0, total 43 rows starting from the beginning of the part 2024.10.22 01:02:01.788124 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:01.902961 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_258_53, total 302 rows starting from the beginning of the part 2024.10.22 01:02:02.094765 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_259_259_0, total 1 rows starting from the beginning of the part 2024.10.22 01:02:02.097259 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(0d98703a87c615e093030f93ff5b5de2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0d98703a87c615e093030f93ff5b5de2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:02:02.099657 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 469, file segment info: File segment: [0, 33554431], key: 0d98703a87c615e093030f93ff5b5de2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:02:02.101479 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:02.252436 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_260_260_0, total 1 rows starting from the beginning of the part 2024.10.22 01:02:02.297008 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_261_261_0, total 1 rows starting from the beginning of the part 2024.10.22 01:02:02.338802 [ 1730 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qsk/mutmrfmtdegmnlcarchnyuqtcrsaz, size 54 2024.10.22 01:02:02.355818 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_262_262_0, total 1 rows starting from the beginning of the part 2024.10.22 01:02:02.400221 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_263_263_0, total 1 rows starting from the beginning of the part 2024.10.22 01:02:02.760666 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:02:02.768081 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:02:02.777641 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:02:02.782743 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:02:02.792528 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:02:02.825772 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:02:02.876656 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(04f25a9779c4525e1e1124a3d21dd5f3) : [0, 42921]: Setting download as failed: File segment: [0, 42921], key: 04f25a9779c4525e1e1124a3d21dd5f3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:02.878254 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(04f25a9779c4525e1e1124a3d21dd5f3): No space left in cache to reserve 42922 bytes, will continue without cache download 2024.10.22 01:02:02.921537 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 04f25a9779c4525e1e1124a3d21dd5f3, offset: 0, size: 0 2024.10.22 01:02:02.923332 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 04f25a9779c4525e1e1124a3d21dd5f3 for removal 2024.10.22 01:02:02.930684 [ 1476 ] {} CacheMetadata: Key 04f25a9779c4525e1e1124a3d21dd5f3 is removed from metadata 2024.10.22 01:02:02.931154 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 300e9689445d3bfd6e0e28357d0008aa, offset: 0, size: 0 2024.10.22 01:02:02.934625 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 300e9689445d3bfd6e0e28357d0008aa for removal 2024.10.22 01:02:02.943112 [ 1476 ] {} CacheMetadata: Key 300e9689445d3bfd6e0e28357d0008aa is removed from metadata 2024.10.22 01:02:02.964382 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:02:03.133320 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zfh/tnwozxabqisrkirooholjgogtnyug (LogSeriesLimiter: on interval from 2024-10-22 01:01:41 to 2024-10-22 01:02:03 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:02:03.566823 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_543_166} MergeTask::MergeProjectionsStage: Merge sorted 31951 rows, containing 13 columns (13 merged, 0 gathered) in 2.04098223 sec., 15654.717385755976 rows/sec., 2.40 MiB/sec. 2024.10.22 01:02:03.944931 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_543_166} MergedBlockOutputStream: filled checksums 202410_1_543_166 (state Temporary) 2024.10.22 01:02:04.089503 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b634cc28f5a7db0bc0df1b2739720149) : [0, 152]: Setting download as failed: File segment: [0, 152], key: b634cc28f5a7db0bc0df1b2739720149, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:04.092154 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b634cc28f5a7db0bc0df1b2739720149): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:02:04.161126 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} MergeTask::MergeProjectionsStage: Merge sorted 307 rows, containing 32 columns (32 merged, 0 gathered) in 2.644311353 sec., 116.09828005000439 rows/sec., 11.96 KiB/sec. 2024.10.22 01:02:04.217568 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} MergedBlockOutputStream: filled checksums 202410_1_263_54 (state Temporary) 2024.10.22 01:02:04.281630 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_543_166} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_543_166 to 202410_1_543_166 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:04.284364 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_543_166} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_542_165, 202410_543_543_0] -> 202410_1_543_166 2024.10.22 01:02:04.338464 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.67 MiB. 2024.10.22 01:02:04.450231 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_306_61/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:02:04.607982 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 14.73 MiB, will set to 1.72 GiB (RSS), difference: 15.26 MiB 2024.10.22 01:02:04.881216 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_64_13/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:02:05.075090 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(95ac779503608fd8f31dfb9751269c51) : [0, 302289]: Setting download as failed: File segment: [0, 302289], key: 95ac779503608fd8f31dfb9751269c51, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:05.081785 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(95ac779503608fd8f31dfb9751269c51): No space left in cache to reserve 302290 bytes, will continue without cache download 2024.10.22 01:02:05.238630 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_263_54 to 202410_1_263_54 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:05.244253 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_263_54} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_258_53, 202410_263_263_0] -> 202410_1_263_54 2024.10.22 01:02:05.161730 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_442_105, 202410_443_443_0] 2024.10.22 01:02:05.268882 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_442_105, 202410_443_443_0] 2024.10.22 01:02:05.278446 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_442_105, 202410_443_443_0] 2024.10.22 01:02:05.302625 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 95ac779503608fd8f31dfb9751269c51, offset: 0, size: 0 2024.10.22 01:02:05.304755 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 95ac779503608fd8f31dfb9751269c51 for removal 2024.10.22 01:02:05.306882 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:02:05.307704 [ 1476 ] {} CacheMetadata: Key 95ac779503608fd8f31dfb9751269c51 is removed from metadata 2024.10.22 01:02:05.308522 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b634cc28f5a7db0bc0df1b2739720149, offset: 0, size: 0 2024.10.22 01:02:05.310188 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b634cc28f5a7db0bc0df1b2739720149 for removal 2024.10.22 01:02:05.312544 [ 1476 ] {} CacheMetadata: Key b634cc28f5a7db0bc0df1b2739720149 is removed from metadata 2024.10.22 01:02:05.560809 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 31497 2024.10.22 01:02:05.697016 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:05.704148 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.66 GiB. 2024.10.22 01:02:05.772925 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_535_535_0 (state Temporary) 2024.10.22 01:02:05.825806 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_535_535_0 to 202410_548_548_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:05.969225 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_543_166 to 202410_544_544_0 2024.10.22 01:02:06.004449 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b3275f28111050c0adf7ccd5a624e703) : [0, 149]: Setting download as failed: File segment: [0, 149], key: b3275f28111050c0adf7ccd5a624e703, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:06.005986 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b3275f28111050c0adf7ccd5a624e703): No space left in cache to reserve 150 bytes, will continue without cache download 2024.10.22 01:02:05.977836 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 01:02:06.081107 [ 1158 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 297.66 GiB. 2024.10.22 01:02:06.092317 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31497 2024.10.22 01:02:06.098694 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_544_167} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_543_166 to 202410_544_544_0 into Compact with storage Full 2024.10.22 01:02:06.130456 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_544_167} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:06.149919 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_544_167} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_543_166, total 31951 rows starting from the beginning of the part 2024.10.22 01:02:06.172099 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_544_167} MergeTreeSequentialSource: Reading 2 marks from part 202410_544_544_0, total 71 rows starting from the beginning of the part 2024.10.22 01:02:06.391445 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 4547 2024.10.22 01:02:06.398894 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 145 entries to flush up to offset 23230 2024.10.22 01:02:06.464733 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 14.67 MiB, will set to 1.71 GiB (RSS), difference: -5.97 MiB 2024.10.22 01:02:06.531960 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:06.537831 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.66 GiB. 2024.10.22 01:02:06.600863 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:06.606678 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.66 GiB. 2024.10.22 01:02:06.681088 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_314_314_0 (state Temporary) 2024.10.22 01:02:06.733726 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_314_314_0 to 202410_327_327_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:06.789562 [ 1152 ] {} DatabaseCatalog: Execution took 338 ms. 2024.10.22 01:02:06.794330 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c9644693f0af1a9ed00d8bf5a49d6103) : [0, 168236]: Setting download as failed: File segment: [0, 168236], key: c9644693f0af1a9ed00d8bf5a49d6103, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:06.795396 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c9644693f0af1a9ed00d8bf5a49d6103): No space left in cache to reserve 168237 bytes, will continue without cache download 2024.10.22 01:02:06.875890 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_311_311_0 (state Temporary) 2024.10.22 01:02:06.908937 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c9644693f0af1a9ed00d8bf5a49d6103, offset: 0, size: 0 2024.10.22 01:02:06.911040 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c9644693f0af1a9ed00d8bf5a49d6103 for removal 2024.10.22 01:02:06.913816 [ 1476 ] {} CacheMetadata: Key c9644693f0af1a9ed00d8bf5a49d6103 is removed from metadata 2024.10.22 01:02:06.916718 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_311_311_0 to 202410_317_317_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:06.917666 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b3275f28111050c0adf7ccd5a624e703, offset: 0, size: 0 2024.10.22 01:02:06.921676 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b3275f28111050c0adf7ccd5a624e703 for removal 2024.10.22 01:02:06.928823 [ 1476 ] {} CacheMetadata: Key b3275f28111050c0adf7ccd5a624e703 is removed from metadata 2024.10.22 01:02:06.979601 [ 1273 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_312_98 to 202410_313_313_0 2024.10.22 01:02:06.982799 [ 1273 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.26 MiB using storage policy from min volume index 0 2024.10.22 01:02:06.986551 [ 1273 ] {} DiskLocal: Reserved 1.26 MiB on local disk `default`, having unreserved 297.66 GiB. 2024.10.22 01:02:06.998725 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4547 2024.10.22 01:02:07.010830 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_313_99} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_312_98 to 202410_313_313_0 into Compact with storage Full 2024.10.22 01:02:07.170069 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 499374 2024.10.22 01:02:07.069775 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 23230 2024.10.22 01:02:07.226982 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_313_99} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:07.282243 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_313_99} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_312_98, total 22790 rows starting from the beginning of the part 2024.10.22 01:02:07.394984 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_313_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_313_313_0, total 95 rows starting from the beginning of the part 2024.10.22 01:02:07.492243 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(8d27870e158bd4f1963b84844c38f6bd) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8d27870e158bd4f1963b84844c38f6bd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:02:07.495321 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1357, file segment info: File segment: [0, 33554431], key: 8d27870e158bd4f1963b84844c38f6bd, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:02:07.498668 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:07.839413 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:02:07.870870 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:02:07.880226 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:02:07.898078 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:02:07.928775 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:02:07.956146 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:02:07.985882 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:02:08.002695 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:02:08.005017 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:08.011451 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:08.044075 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:02:08.185161 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_544_167} MergeTask::MergeProjectionsStage: Merge sorted 32022 rows, containing 13 columns (13 merged, 0 gathered) in 2.097829844 sec., 15264.345719737983 rows/sec., 2.34 MiB/sec. 2024.10.22 01:02:08.246304 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ocu/rrilnxmbhtaihtbopyrzcjljoedqm (LogSeriesLimiter: on interval from 2024-10-22 01:02:03 to 2024-10-22 01:02:08 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:02:08.309511 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_300_300_0 (state Temporary) 2024.10.22 01:02:08.408251 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_300_300_0 to 202410_307_307_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:08.409546 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:08.415535 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:08.471614 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 499374 2024.10.22 01:02:08.482305 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_544_167} MergedBlockOutputStream: filled checksums 202410_1_544_167 (state Temporary) 2024.10.22 01:02:08.576375 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3b5a95aa02773ff125e28d037b86c6af) : [0, 145]: Setting download as failed: File segment: [0, 145], key: 3b5a95aa02773ff125e28d037b86c6af, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:08.579661 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3b5a95aa02773ff125e28d037b86c6af): No space left in cache to reserve 146 bytes, will continue without cache download 2024.10.22 01:02:08.870473 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_313_99} MergeTask::MergeProjectionsStage: Merge sorted 22885 rows, containing 13 columns (13 merged, 0 gathered) in 1.875233924 sec., 12203.810792407572 rows/sec., 5.18 MiB/sec. 2024.10.22 01:02:08.906721 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_544_167} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_544_167 to 202410_1_544_167 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:08.912641 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_544_167} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_543_166, 202410_544_544_0] -> 202410_1_544_167 2024.10.22 01:02:09.164976 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.67 MiB. 2024.10.22 01:02:09.346218 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_313_99} MergedBlockOutputStream: filled checksums 202410_1_313_99 (state Temporary) 2024.10.22 01:02:09.599162 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_313_99} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_313_99 to 202410_1_313_99 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:09.602869 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_313_99} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_312_98, 202410_313_313_0] -> 202410_1_313_99 2024.10.22 01:02:09.668947 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.36 MiB. 2024.10.22 01:02:09.706739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f76d99b8be89ba5dbdd3846f976dc477) : [0, 4835]: Setting download as failed: File segment: [0, 4835], key: f76d99b8be89ba5dbdd3846f976dc477, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:09.707703 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f76d99b8be89ba5dbdd3846f976dc477): No space left in cache to reserve 4836 bytes, will continue without cache download 2024.10.22 01:02:09.732162 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f76d99b8be89ba5dbdd3846f976dc477, offset: 0, size: 0 2024.10.22 01:02:09.738528 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f76d99b8be89ba5dbdd3846f976dc477 for removal 2024.10.22 01:02:09.745073 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3b5a95aa02773ff125e28d037b86c6af, offset: 0, size: 0 2024.10.22 01:02:09.745476 [ 1476 ] {} CacheMetadata: Key f76d99b8be89ba5dbdd3846f976dc477 is removed from metadata 2024.10.22 01:02:09.749490 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3b5a95aa02773ff125e28d037b86c6af for removal 2024.10.22 01:02:09.758202 [ 1476 ] {} CacheMetadata: Key 3b5a95aa02773ff125e28d037b86c6af is removed from metadata 2024.10.22 01:02:09.841096 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:02:09.845536 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:02:10.105807 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 31584 2024.10.22 01:02:10.378358 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(dd64736b50fe69b1de8f84c60820dc82) : [0, 162]: Setting download as failed: File segment: [0, 162], key: dd64736b50fe69b1de8f84c60820dc82, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:10.380996 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(dd64736b50fe69b1de8f84c60820dc82): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:02:10.398983 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_254_63, 202410_255_255_0] 2024.10.22 01:02:10.400344 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_254_63, 202410_255_255_0] 2024.10.22 01:02:10.410851 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_254_63, 202410_255_255_0] 2024.10.22 01:02:10.495064 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 11.41 MiB, will set to 1.70 GiB (RSS), difference: 7.21 MiB 2024.10.22 01:02:10.561137 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_443_106, 202410_444_444_0] 2024.10.22 01:02:10.566327 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_443_106, 202410_444_444_0] 2024.10.22 01:02:10.589230 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_443_106, 202410_444_444_0] 2024.10.22 01:02:10.840274 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:10.948005 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:11.030907 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_536_536_0 (state Temporary) 2024.10.22 01:02:11.134641 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a026bb01f4fb127d2da1725b0e328696) : [0, 95623]: Setting download as failed: File segment: [0, 95623], key: a026bb01f4fb127d2da1725b0e328696, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:11.138171 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a026bb01f4fb127d2da1725b0e328696): No space left in cache to reserve 95624 bytes, will continue without cache download 2024.10.22 01:02:11.281075 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a026bb01f4fb127d2da1725b0e328696, offset: 0, size: 0 2024.10.22 01:02:11.283036 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a026bb01f4fb127d2da1725b0e328696 for removal 2024.10.22 01:02:11.287382 [ 1476 ] {} CacheMetadata: Key a026bb01f4fb127d2da1725b0e328696 is removed from metadata 2024.10.22 01:02:11.287958 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dd64736b50fe69b1de8f84c60820dc82, offset: 0, size: 0 2024.10.22 01:02:11.289655 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dd64736b50fe69b1de8f84c60820dc82 for removal 2024.10.22 01:02:11.289649 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_536_536_0 to 202410_549_549_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:11.293196 [ 1476 ] {} CacheMetadata: Key dd64736b50fe69b1de8f84c60820dc82 is removed from metadata 2024.10.22 01:02:11.312294 [ 1229 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_544_167 to 202410_545_545_0 2024.10.22 01:02:11.368576 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31584 2024.10.22 01:02:11.430158 [ 1229 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 01:02:11.438556 [ 1229 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:11.487424 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_545_168} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_544_167 to 202410_545_545_0 into Compact with storage Full 2024.10.22 01:02:11.677537 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_63_63_0 (state Temporary) 2024.10.22 01:02:11.752601 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_545_168} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:11.772946 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_545_168} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_544_167, total 32022 rows starting from the beginning of the part 2024.10.22 01:02:11.808761 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_545_168} MergeTreeSequentialSource: Reading 2 marks from part 202410_545_545_0, total 67 rows starting from the beginning of the part 2024.10.22 01:02:11.905197 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(ce9ad911b1b3578a3b7aca47a87b65a6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ce9ad911b1b3578a3b7aca47a87b65a6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:02:11.947901 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: ce9ad911b1b3578a3b7aca47a87b65a6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:02:11.952965 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:12.756035 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(26f5d2e0ca8e8ab192f3411f0c1c137d) : [0, 149]: Setting download as failed: File segment: [0, 149], key: 26f5d2e0ca8e8ab192f3411f0c1c137d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:12.757580 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(26f5d2e0ca8e8ab192f3411f0c1c137d): No space left in cache to reserve 150 bytes, will continue without cache download 2024.10.22 01:02:12.908834 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_545_168} MergeTask::MergeProjectionsStage: Merge sorted 32089 rows, containing 13 columns (13 merged, 0 gathered) in 1.426294828 sec., 22498.153516406077 rows/sec., 3.45 MiB/sec. 2024.10.22 01:02:12.980351 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_545_168} MergedBlockOutputStream: filled checksums 202410_1_545_168 (state Temporary) 2024.10.22 01:02:13.059619 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_545_168} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_545_168 to 202410_1_545_168 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:13.063352 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_545_168} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_544_167, 202410_545_545_0] -> 202410_1_545_168 2024.10.22 01:02:13.086735 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.67 MiB. 2024.10.22 01:02:13.159264 [ 1230 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:02:13.198827 [ 1230 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:02:13.231450 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 11.09 MiB, will set to 1.70 GiB (RSS), difference: -3.95 MiB 2024.10.22 01:02:13.482331 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(aab6b2bbc885256d6b3d459f9f37d16a) : [0, 195643]: Setting download as failed: File segment: [0, 195643], key: aab6b2bbc885256d6b3d459f9f37d16a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:13.483739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(aab6b2bbc885256d6b3d459f9f37d16a): No space left in cache to reserve 195644 bytes, will continue without cache download 2024.10.22 01:02:13.506564 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aab6b2bbc885256d6b3d459f9f37d16a, offset: 0, size: 0 2024.10.22 01:02:13.508299 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aab6b2bbc885256d6b3d459f9f37d16a for removal 2024.10.22 01:02:13.510811 [ 1476 ] {} CacheMetadata: Key aab6b2bbc885256d6b3d459f9f37d16a is removed from metadata 2024.10.22 01:02:13.514308 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 26f5d2e0ca8e8ab192f3411f0c1c137d, offset: 0, size: 0 2024.10.22 01:02:13.520745 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 26f5d2e0ca8e8ab192f3411f0c1c137d for removal 2024.10.22 01:02:13.524289 [ 1476 ] {} CacheMetadata: Key 26f5d2e0ca8e8ab192f3411f0c1c137d is removed from metadata 2024.10.22 01:02:13.698663 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.04174158 sec. 2024.10.22 01:02:14.211645 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9c4d83e475783c8bc4781b011ad72c8f) : [0, 150]: Setting download as failed: File segment: [0, 150], key: 9c4d83e475783c8bc4781b011ad72c8f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:14.213606 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9c4d83e475783c8bc4781b011ad72c8f): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:02:14.422414 [ 1224 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 212 ms. 2024.10.22 01:02:14.500852 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 4560 2024.10.22 01:02:14.599701 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_306_61/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:02:14.635291 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:14.645842 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:14.685101 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_315_315_0 (state Temporary) 2024.10.22 01:02:14.725488 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_315_315_0 to 202410_328_328_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:14.765174 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4560 2024.10.22 01:02:14.791921 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 142 entries to flush up to offset 23372 2024.10.22 01:02:15.142545 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:15.146288 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:15.301217 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_312_312_0 (state Temporary) 2024.10.22 01:02:15.387723 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 31651 2024.10.22 01:02:15.470116 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_312_312_0 to 202410_318_318_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:15.478053 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 500526 2024.10.22 01:02:15.481220 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7c8aea5ae6de5a027c51cb1375c66688) : [0, 259251]: Setting download as failed: File segment: [0, 259251], key: 7c8aea5ae6de5a027c51cb1375c66688, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:15.482464 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7c8aea5ae6de5a027c51cb1375c66688): No space left in cache to reserve 259252 bytes, will continue without cache download 2024.10.22 01:02:15.498183 [ 1181 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_313_99 to 202410_314_314_0 2024.10.22 01:02:15.505452 [ 1181 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.27 MiB using storage policy from min volume index 0 2024.10.22 01:02:15.507359 [ 1181 ] {} DiskLocal: Reserved 1.27 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:15.512807 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7c8aea5ae6de5a027c51cb1375c66688, offset: 0, size: 0 2024.10.22 01:02:15.515019 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7c8aea5ae6de5a027c51cb1375c66688 for removal 2024.10.22 01:02:15.517525 [ 1476 ] {} CacheMetadata: Key 7c8aea5ae6de5a027c51cb1375c66688 is removed from metadata 2024.10.22 01:02:15.521199 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9c4d83e475783c8bc4781b011ad72c8f, offset: 0, size: 0 2024.10.22 01:02:15.524249 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9c4d83e475783c8bc4781b011ad72c8f for removal 2024.10.22 01:02:15.524924 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 23372 2024.10.22 01:02:15.528360 [ 1476 ] {} CacheMetadata: Key 9c4d83e475783c8bc4781b011ad72c8f is removed from metadata 2024.10.22 01:02:15.529603 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_314_100} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_313_99 to 202410_314_314_0 into Compact with storage Full 2024.10.22 01:02:15.534963 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:15.540450 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:15.610484 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_537_537_0 (state Temporary) 2024.10.22 01:02:15.664031 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_314_100} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:15.678066 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_314_100} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_313_99, total 22885 rows starting from the beginning of the part 2024.10.22 01:02:15.697002 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_314_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_314_314_0, total 52 rows starting from the beginning of the part 2024.10.22 01:02:15.757332 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_537_537_0 to 202410_550_550_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:15.641803 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 12.11 MiB, will set to 1.70 GiB (RSS), difference: -3.84 MiB 2024.10.22 01:02:15.774682 [ 1220 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_545_168 to 202410_546_546_0 2024.10.22 01:02:15.777985 [ 1220 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.12 MiB using storage policy from min volume index 0 2024.10.22 01:02:15.779704 [ 1220 ] {} DiskLocal: Reserved 1.12 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:15.788898 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_546_169} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_545_168 to 202410_546_546_0 into Compact with storage Full 2024.10.22 01:02:15.815608 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_546_169} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:15.826668 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_546_169} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_545_168, total 32089 rows starting from the beginning of the part 2024.10.22 01:02:15.848474 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_546_169} MergeTreeSequentialSource: Reading 2 marks from part 202410_546_546_0, total 60 rows starting from the beginning of the part 2024.10.22 01:02:15.881069 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:02:15.887861 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31651 2024.10.22 01:02:16.083080 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_444_107, 202410_445_445_0, 202410_446_446_0, 202410_447_447_0, 202410_448_448_0, 202410_449_449_0] 2024.10.22 01:02:16.086521 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_444_107, 202410_445_445_0, 202410_446_446_0, 202410_447_447_0, 202410_448_448_0, 202410_449_449_0] 2024.10.22 01:02:16.129840 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_444_107, 202410_445_445_0, 202410_446_446_0, 202410_447_447_0, 202410_448_448_0, 202410_449_449_0] 2024.10.22 01:02:16.237889 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(743f27e7b1814459f0ca35bef62fbef0) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 743f27e7b1814459f0ca35bef62fbef0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:16.241536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(743f27e7b1814459f0ca35bef62fbef0): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:02:16.284038 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:16.286573 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:16.628435 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_301_301_0 (state Temporary) 2024.10.22 01:02:16.631952 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 272 2024.10.22 01:02:16.761427 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_301_301_0 to 202410_308_308_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:16.804899 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 500526 2024.10.22 01:02:16.833491 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:16.854935 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:16.931641 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} MergeTask::MergeProjectionsStage: Merge sorted 511812 rows, containing 4 columns (4 merged, 0 gathered) in 16.976844652 sec., 30147.651727478384 rows/sec., 442.01 KiB/sec. 2024.10.22 01:02:17.040036 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_254_254_0 (state Temporary) 2024.10.22 01:02:17.088925 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} MergedBlockOutputStream: filled checksums 202410_1_306_61 (state Temporary) 2024.10.22 01:02:17.190922 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_314_100} MergeTask::MergeProjectionsStage: Merge sorted 22937 rows, containing 13 columns (13 merged, 0 gathered) in 1.665917065 sec., 13768.392485972885 rows/sec., 5.84 MiB/sec. 2024.10.22 01:02:17.194551 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_306_61 to 202410_1_306_61 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:17.209043 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_306_61} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_301_60, 202410_306_306_0] -> 202410_1_306_61 2024.10.22 01:02:17.220519 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_254_254_0 to 202410_264_264_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:17.260723 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ed2eca01b7c835e39fa769e0fb4ca06b) : [0, 323536]: Setting download as failed: File segment: [0, 323536], key: ed2eca01b7c835e39fa769e0fb4ca06b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:17.264281 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ed2eca01b7c835e39fa769e0fb4ca06b): No space left in cache to reserve 323537 bytes, will continue without cache download 2024.10.22 01:02:17.296131 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:02:17.296897 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_314_100} MergedBlockOutputStream: filled checksums 202410_1_314_100 (state Temporary) 2024.10.22 01:02:17.397753 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ed2eca01b7c835e39fa769e0fb4ca06b, offset: 0, size: 0 2024.10.22 01:02:17.399273 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ed2eca01b7c835e39fa769e0fb4ca06b for removal 2024.10.22 01:02:17.403734 [ 1476 ] {} CacheMetadata: Key ed2eca01b7c835e39fa769e0fb4ca06b is removed from metadata 2024.10.22 01:02:17.405034 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 743f27e7b1814459f0ca35bef62fbef0, offset: 0, size: 0 2024.10.22 01:02:17.407269 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 743f27e7b1814459f0ca35bef62fbef0 for removal 2024.10.22 01:02:17.410199 [ 1476 ] {} CacheMetadata: Key 743f27e7b1814459f0ca35bef62fbef0 is removed from metadata 2024.10.22 01:02:17.497454 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_314_100} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_314_100 to 202410_1_314_100 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:17.500779 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_314_100} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_313_99, 202410_314_314_0] -> 202410_1_314_100 2024.10.22 01:02:17.535983 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.32 MiB. 2024.10.22 01:02:17.703647 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 272 2024.10.22 01:02:18.300781 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b8aa748f866a3b2945f67f72a93cc6e9) : [0, 157]: Setting download as failed: File segment: [0, 157], key: b8aa748f866a3b2945f67f72a93cc6e9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:18.302380 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b8aa748f866a3b2945f67f72a93cc6e9): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:02:18.376765 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_546_169} MergeTask::MergeProjectionsStage: Merge sorted 32149 rows, containing 13 columns (13 merged, 0 gathered) in 2.590081778 sec., 12412.34939879956 rows/sec., 1.91 MiB/sec. 2024.10.22 01:02:18.542792 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_546_169} MergedBlockOutputStream: filled checksums 202410_1_546_169 (state Temporary) 2024.10.22 01:02:18.623461 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_546_169} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_546_169 to 202410_1_546_169 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:18.628061 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_546_169} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_545_168, 202410_546_546_0] -> 202410_1_546_169 2024.10.22 01:02:18.668661 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.67 MiB. 2024.10.22 01:02:18.776035 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_255_64, 202410_256_256_0] 2024.10.22 01:02:18.780115 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_255_64, 202410_256_256_0] 2024.10.22 01:02:18.788182 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_255_64, 202410_256_256_0] 2024.10.22 01:02:18.811006 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(a5395ab220e2220f2f6d06777af4a0ea) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a5395ab220e2220f2f6d06777af4a0ea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:02:18.812625 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 899, file segment info: File segment: [0, 33554431], key: a5395ab220e2220f2f6d06777af4a0ea, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:02:18.815635 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:18.972225 [ 1711 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zfh/tnwozxabqisrkirooholjgogtnyug, size 1872138 2024.10.22 01:02:19.473134 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:02:19.498941 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:02:19.510928 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:02:19.545657 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:02:19.578940 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:02:19.594873 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:02:19.609633 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:02:19.746224 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vms/vmffqfwidshjvzdwthayirdpyreiw (LogSeriesLimiter: on interval from 2024-10-22 01:02:08 to 2024-10-22 01:02:19 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:02:19.814721 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 8.09 MiB, will set to 1.69 GiB (RSS), difference: 2.96 MiB 2024.10.22 01:02:19.823175 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6be14359168e0458ee88783d784775f3) : [0, 551085]: Setting download as failed: File segment: [0, 551085], key: 6be14359168e0458ee88783d784775f3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:19.828196 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6be14359168e0458ee88783d784775f3): No space left in cache to reserve 551086 bytes, will continue without cache download 2024.10.22 01:02:19.840394 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:02:19.845540 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:02:19.895239 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 31743 2024.10.22 01:02:20.168138 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6be14359168e0458ee88783d784775f3, offset: 0, size: 0 2024.10.22 01:02:20.205808 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6be14359168e0458ee88783d784775f3 for removal 2024.10.22 01:02:20.220448 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b8aa748f866a3b2945f67f72a93cc6e9, offset: 0, size: 0 2024.10.22 01:02:20.222479 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b8aa748f866a3b2945f67f72a93cc6e9 for removal 2024.10.22 01:02:20.226536 [ 1476 ] {} CacheMetadata: Key 6be14359168e0458ee88783d784775f3 is removed from metadata 2024.10.22 01:02:20.231212 [ 1476 ] {} CacheMetadata: Key b8aa748f866a3b2945f67f72a93cc6e9 is removed from metadata 2024.10.22 01:02:20.414786 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:20.418874 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:20.584608 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_538_538_0 (state Temporary) 2024.10.22 01:02:20.655138 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_538_538_0 to 202410_551_551_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:20.681462 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_546_169 to 202410_547_547_0 2024.10.22 01:02:20.684857 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.12 MiB using storage policy from min volume index 0 2024.10.22 01:02:20.686422 [ 1198 ] {} DiskLocal: Reserved 1.12 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:20.691134 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31743 2024.10.22 01:02:20.702773 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_547_170} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_546_169 to 202410_547_547_0 into Compact with storage Full 2024.10.22 01:02:20.727590 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_547_170} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:20.745255 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_547_170} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_546_169, total 32149 rows starting from the beginning of the part 2024.10.22 01:02:20.776763 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_547_170} MergeTreeSequentialSource: Reading 2 marks from part 202410_547_547_0, total 93 rows starting from the beginning of the part 2024.10.22 01:02:21.209180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6e1746dce8e54c1bafae48abff62abb4) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 6e1746dce8e54c1bafae48abff62abb4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:21.211506 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6e1746dce8e54c1bafae48abff62abb4): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:02:21.288637 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 5.98 MiB, will set to 1.70 GiB (RSS), difference: -6.29 MiB 2024.10.22 01:02:21.787336 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_64_13/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:02:22.268673 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 4579 2024.10.22 01:02:22.273295 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_63_63_0 to 202410_66_66_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:22.280258 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_547_170} MergeTask::MergeProjectionsStage: Merge sorted 32242 rows, containing 13 columns (13 merged, 0 gathered) in 1.580781667 sec., 20396.238565436248 rows/sec., 3.13 MiB/sec. 2024.10.22 01:02:22.364433 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:22.367395 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:22.392076 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_316_316_0 (state Temporary) 2024.10.22 01:02:22.392516 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ec836022deeed923e91d08a8ecbae03d) : [0, 554684]: Setting download as failed: File segment: [0, 554684], key: ec836022deeed923e91d08a8ecbae03d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:22.393131 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ec836022deeed923e91d08a8ecbae03d): No space left in cache to reserve 554685 bytes, will continue without cache download 2024.10.22 01:02:22.433756 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_316_316_0 to 202410_329_329_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:22.463116 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4579 2024.10.22 01:02:22.463333 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_547_170} MergedBlockOutputStream: filled checksums 202410_1_547_170 (state Temporary) 2024.10.22 01:02:22.528062 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ec836022deeed923e91d08a8ecbae03d, offset: 0, size: 0 2024.10.22 01:02:22.529469 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ec836022deeed923e91d08a8ecbae03d for removal 2024.10.22 01:02:22.533716 [ 1476 ] {} CacheMetadata: Key ec836022deeed923e91d08a8ecbae03d is removed from metadata 2024.10.22 01:02:22.538274 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6e1746dce8e54c1bafae48abff62abb4, offset: 0, size: 0 2024.10.22 01:02:22.539963 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6e1746dce8e54c1bafae48abff62abb4 for removal 2024.10.22 01:02:22.541755 [ 1476 ] {} CacheMetadata: Key 6e1746dce8e54c1bafae48abff62abb4 is removed from metadata 2024.10.22 01:02:22.831639 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_547_170} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_547_170 to 202410_1_547_170 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:22.837227 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_547_170} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_546_169, 202410_547_547_0] -> 202410_1_547_170 2024.10.22 01:02:22.853445 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.69 MiB. 2024.10.22 01:02:23.048676 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 161 entries to flush up to offset 23533 2024.10.22 01:02:23.212746 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:23.216527 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:23.349310 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(71b2fe73bb175b10c89d55dd990feb3d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 71b2fe73bb175b10c89d55dd990feb3d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:02:23.358702 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 71b2fe73bb175b10c89d55dd990feb3d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:02:23.363074 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:23.477191 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_313_313_0 (state Temporary) 2024.10.22 01:02:23.607496 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 19.35 MiB, will set to 1.69 GiB (RSS), difference: 20.77 MiB 2024.10.22 01:02:23.624565 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_313_313_0 to 202410_319_319_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:23.691052 [ 1175 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_314_100 to 202410_315_315_0 2024.10.22 01:02:23.799000 [ 1175 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.27 MiB using storage policy from min volume index 0 2024.10.22 01:02:23.800334 [ 1175 ] {} DiskLocal: Reserved 1.27 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:23.803169 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 23533 2024.10.22 01:02:23.811468 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 502062 2024.10.22 01:02:23.812522 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_315_101} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_314_100 to 202410_315_315_0 into Compact with storage Full 2024.10.22 01:02:23.861832 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_315_101} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:23.957263 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_315_101} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_314_100, total 22937 rows starting from the beginning of the part 2024.10.22 01:02:23.976127 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_315_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_315_315_0, total 138 rows starting from the beginning of the part 2024.10.22 01:02:24.231697 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fb7a911ae629173c1017372521c86292) : [0, 153]: Setting download as failed: File segment: [0, 153], key: fb7a911ae629173c1017372521c86292, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:24.234195 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fb7a911ae629173c1017372521c86292): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:02:24.403986 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:24.405796 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:24.685665 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_302_302_0 (state Temporary) 2024.10.22 01:02:24.698198 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 31811 2024.10.22 01:02:24.724105 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_302_302_0 to 202410_309_309_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:24.770748 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 502062 2024.10.22 01:02:24.822907 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4f991649563b2dd0058c59cd58a36fbc) : [0, 342833]: Setting download as failed: File segment: [0, 342833], key: 4f991649563b2dd0058c59cd58a36fbc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:24.824182 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4f991649563b2dd0058c59cd58a36fbc): No space left in cache to reserve 342834 bytes, will continue without cache download 2024.10.22 01:02:24.944092 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2529 2024.10.22 01:02:24.983941 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4f991649563b2dd0058c59cd58a36fbc, offset: 0, size: 0 2024.10.22 01:02:24.995821 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4f991649563b2dd0058c59cd58a36fbc for removal 2024.10.22 01:02:25.072152 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:25.086940 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:25.102382 [ 1476 ] {} CacheMetadata: Key 4f991649563b2dd0058c59cd58a36fbc is removed from metadata 2024.10.22 01:02:25.109671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fb7a911ae629173c1017372521c86292, offset: 0, size: 0 2024.10.22 01:02:25.115385 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fb7a911ae629173c1017372521c86292 for removal 2024.10.22 01:02:25.129173 [ 1476 ] {} CacheMetadata: Key fb7a911ae629173c1017372521c86292 is removed from metadata 2024.10.22 01:02:25.205241 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 273 2024.10.22 01:02:25.396114 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_539_539_0 (state Temporary) 2024.10.22 01:02:25.473095 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_539_539_0 to 202410_552_552_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:25.473465 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 19.19 MiB, will set to 1.70 GiB (RSS), difference: 2.55 MiB 2024.10.22 01:02:25.493769 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_547_170 to 202410_548_548_0 2024.10.22 01:02:25.497993 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.12 MiB using storage policy from min volume index 0 2024.10.22 01:02:25.499518 [ 1176 ] {} DiskLocal: Reserved 1.12 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:25.513539 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31811 2024.10.22 01:02:25.628922 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_548_171} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_547_170 to 202410_548_548_0 into Compact with storage Full 2024.10.22 01:02:25.683003 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_315_101} MergeTask::MergeProjectionsStage: Merge sorted 23075 rows, containing 13 columns (13 merged, 0 gathered) in 1.878421261 sec., 12284.251929578219 rows/sec., 5.21 MiB/sec. 2024.10.22 01:02:25.684374 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:25.685107 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_548_171} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:25.685691 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:25.698088 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_548_171} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_547_170, total 32242 rows starting from the beginning of the part 2024.10.22 01:02:25.739584 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_548_171} MergeTreeSequentialSource: Reading 2 marks from part 202410_548_548_0, total 86 rows starting from the beginning of the part 2024.10.22 01:02:25.775011 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_315_101} MergedBlockOutputStream: filled checksums 202410_1_315_101 (state Temporary) 2024.10.22 01:02:25.798293 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_255_255_0 (state Temporary) 2024.10.22 01:02:25.911636 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2f40c2fdbfeddc7dfa94a77a799605f1) : [0, 150]: Setting download as failed: File segment: [0, 150], key: 2f40c2fdbfeddc7dfa94a77a799605f1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:25.913690 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2f40c2fdbfeddc7dfa94a77a799605f1): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:02:25.932018 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_315_101} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_315_101 to 202410_1_315_101 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:25.939491 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_315_101} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_314_100, 202410_315_315_0] -> 202410_1_315_101 2024.10.22 01:02:26.059898 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.37 MiB. 2024.10.22 01:02:26.265065 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_255_255_0 to 202410_265_265_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:26.464020 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 273 2024.10.22 01:02:26.942704 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e8d01fc04bde5b573c1f3c54bc28a4e3) : [0, 246828]: Setting download as failed: File segment: [0, 246828], key: e8d01fc04bde5b573c1f3c54bc28a4e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:27.047168 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e8d01fc04bde5b573c1f3c54bc28a4e3): No space left in cache to reserve 246829 bytes, will continue without cache download 2024.10.22 01:02:27.185586 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e8d01fc04bde5b573c1f3c54bc28a4e3, offset: 0, size: 0 2024.10.22 01:02:27.189410 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e8d01fc04bde5b573c1f3c54bc28a4e3 for removal 2024.10.22 01:02:27.191691 [ 1476 ] {} CacheMetadata: Key e8d01fc04bde5b573c1f3c54bc28a4e3 is removed from metadata 2024.10.22 01:02:27.193007 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2f40c2fdbfeddc7dfa94a77a799605f1, offset: 0, size: 0 2024.10.22 01:02:27.196280 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2f40c2fdbfeddc7dfa94a77a799605f1 for removal 2024.10.22 01:02:27.200922 [ 1476 ] {} CacheMetadata: Key 2f40c2fdbfeddc7dfa94a77a799605f1 is removed from metadata 2024.10.22 01:02:27.201924 [ 1167 ] {} DatabaseCatalog: Execution took 293 ms. 2024.10.22 01:02:27.641038 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 17.18 MiB, will set to 1.69 GiB (RSS), difference: 3.60 MiB 2024.10.22 01:02:27.759409 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_256_65, 202410_257_257_0] 2024.10.22 01:02:27.762464 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_256_65, 202410_257_257_0] 2024.10.22 01:02:27.857475 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_548_171} MergeTask::MergeProjectionsStage: Merge sorted 32328 rows, containing 13 columns (13 merged, 0 gathered) in 2.340337622 sec., 13813.391579106103 rows/sec., 2.12 MiB/sec. 2024.10.22 01:02:27.877967 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_256_65, 202410_257_257_0] 2024.10.22 01:02:27.905586 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fcff5ea3bfb18d0bd8aa59b9f32b966d) : [0, 153]: Setting download as failed: File segment: [0, 153], key: fcff5ea3bfb18d0bd8aa59b9f32b966d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:27.908424 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fcff5ea3bfb18d0bd8aa59b9f32b966d): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:02:28.043865 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_548_171} MergedBlockOutputStream: filled checksums 202410_1_548_171 (state Temporary) 2024.10.22 01:02:28.215221 [ 1150 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:02:28.250828 [ 1150 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:02:28.307249 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_548_171} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_548_171 to 202410_1_548_171 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:28.310146 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_548_171} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_547_170, 202410_548_548_0] -> 202410_1_548_171 2024.10.22 01:02:28.342479 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.69 MiB. 2024.10.22 01:02:28.678718 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 01:02:28.828239 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(98c5efea24b02947ffb0737489be9d6e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 98c5efea24b02947ffb0737489be9d6e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:02:28.833150 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 98c5efea24b02947ffb0737489be9d6e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:02:28.836278 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:29.387560 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7eca372a0f3044141bacdb1f6bfce30f) : [0, 401794]: Setting download as failed: File segment: [0, 401794], key: 7eca372a0f3044141bacdb1f6bfce30f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:29.389507 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7eca372a0f3044141bacdb1f6bfce30f): No space left in cache to reserve 401795 bytes, will continue without cache download 2024.10.22 01:02:29.413602 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7eca372a0f3044141bacdb1f6bfce30f, offset: 0, size: 0 2024.10.22 01:02:29.415472 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7eca372a0f3044141bacdb1f6bfce30f for removal 2024.10.22 01:02:29.419711 [ 1476 ] {} CacheMetadata: Key 7eca372a0f3044141bacdb1f6bfce30f is removed from metadata 2024.10.22 01:02:29.421811 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fcff5ea3bfb18d0bd8aa59b9f32b966d, offset: 0, size: 0 2024.10.22 01:02:29.426641 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fcff5ea3bfb18d0bd8aa59b9f32b966d for removal 2024.10.22 01:02:29.430903 [ 1476 ] {} CacheMetadata: Key fcff5ea3bfb18d0bd8aa59b9f32b966d is removed from metadata 2024.10.22 01:02:29.635912 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 31883 2024.10.22 01:02:29.840565 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:02:29.883188 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:02:29.883322 [ 1266 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 249 ms. 2024.10.22 01:02:29.970519 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 4594 2024.10.22 01:02:29.986345 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:29.989305 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:30.060493 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_540_540_0 (state Temporary) 2024.10.22 01:02:30.084467 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:30.087120 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:30.119285 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_317_317_0 (state Temporary) 2024.10.22 01:02:30.162894 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(29d162a2e8093c9a8002f58b3609d129) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 29d162a2e8093c9a8002f58b3609d129, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:30.166476 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(29d162a2e8093c9a8002f58b3609d129): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:02:30.171015 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_317_317_0 to 202410_330_330_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:30.197152 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_540_540_0 to 202410_553_553_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:30.219830 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_548_171 to 202410_549_549_0 2024.10.22 01:02:30.248931 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31883 2024.10.22 01:02:30.327304 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.13 MiB using storage policy from min volume index 0 2024.10.22 01:02:30.332479 [ 1269 ] {} DiskLocal: Reserved 1.13 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:30.384986 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_549_172} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_548_171 to 202410_549_549_0 into Compact with storage Full 2024.10.22 01:02:30.429502 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_549_172} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:30.443448 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_549_172} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_548_171, total 32328 rows starting from the beginning of the part 2024.10.22 01:02:30.479933 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_549_172} MergeTreeSequentialSource: Reading 2 marks from part 202410_549_549_0, total 87 rows starting from the beginning of the part 2024.10.22 01:02:30.789739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8f18b8e60c77f29cfecb1ac2fb084def) : [0, 346109]: Setting download as failed: File segment: [0, 346109], key: 8f18b8e60c77f29cfecb1ac2fb084def, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:30.791168 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8f18b8e60c77f29cfecb1ac2fb084def): No space left in cache to reserve 346110 bytes, will continue without cache download 2024.10.22 01:02:30.897608 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4594 2024.10.22 01:02:31.029634 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8f18b8e60c77f29cfecb1ac2fb084def, offset: 0, size: 0 2024.10.22 01:02:31.036333 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8f18b8e60c77f29cfecb1ac2fb084def for removal 2024.10.22 01:02:31.041660 [ 1476 ] {} CacheMetadata: Key 8f18b8e60c77f29cfecb1ac2fb084def is removed from metadata 2024.10.22 01:02:31.042096 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 29d162a2e8093c9a8002f58b3609d129, offset: 0, size: 0 2024.10.22 01:02:31.046739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 29d162a2e8093c9a8002f58b3609d129 for removal 2024.10.22 01:02:31.049622 [ 1476 ] {} CacheMetadata: Key 29d162a2e8093c9a8002f58b3609d129 is removed from metadata 2024.10.22 01:02:31.321353 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 126 entries to flush up to offset 23659 2024.10.22 01:02:31.558655 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_549_172/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:02:31.578900 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:31.581565 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:31.779065 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 503598 2024.10.22 01:02:31.822341 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(342c67f48948f25ad93a6255c92636b3) : [0, 149]: Setting download as failed: File segment: [0, 149], key: 342c67f48948f25ad93a6255c92636b3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:31.824626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(342c67f48948f25ad93a6255c92636b3): No space left in cache to reserve 150 bytes, will continue without cache download 2024.10.22 01:02:31.876046 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_314_314_0 (state Temporary) 2024.10.22 01:02:31.934179 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_314_314_0 to 202410_320_320_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:31.973773 [ 1205 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_315_101 to 202410_316_316_0 2024.10.22 01:02:31.986387 [ 1205 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.28 MiB using storage policy from min volume index 0 2024.10.22 01:02:31.988817 [ 1205 ] {} DiskLocal: Reserved 1.28 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:31.994508 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 23659 2024.10.22 01:02:31.999658 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_316_102} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_315_101 to 202410_316_316_0 into Compact with storage Full 2024.10.22 01:02:32.049115 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_316_102} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:32.066789 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_316_102} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_315_101, total 23075 rows starting from the beginning of the part 2024.10.22 01:02:32.112553 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_316_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_316_316_0, total 114 rows starting from the beginning of the part 2024.10.22 01:02:32.131016 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_549_172} MergeTask::MergeProjectionsStage: Merge sorted 32415 rows, containing 13 columns (13 merged, 0 gathered) in 1.750184925 sec., 18520.90001289435 rows/sec., 2.85 MiB/sec. 2024.10.22 01:02:32.507260 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 12.33 MiB, will set to 1.68 GiB (RSS), difference: -11.20 MiB 2024.10.22 01:02:32.489299 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 2571 2024.10.22 01:02:32.581552 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:32.582659 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_549_172} MergedBlockOutputStream: filled checksums 202410_1_549_172 (state Temporary) 2024.10.22 01:02:32.586816 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:32.797914 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_303_303_0 (state Temporary) 2024.10.22 01:02:32.809568 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_549_172} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_549_172 to 202410_1_549_172 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:32.833705 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_549_172} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_548_171, 202410_549_549_0] -> 202410_1_549_172 2024.10.22 01:02:32.882806 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.71 MiB. 2024.10.22 01:02:32.964643 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_303_303_0 to 202410_310_310_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:33.019006 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(16dcaad49aba145bd41f65edea264c30) : [0, 199278]: Setting download as failed: File segment: [0, 199278], key: 16dcaad49aba145bd41f65edea264c30, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:33.030986 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(16dcaad49aba145bd41f65edea264c30): No space left in cache to reserve 199279 bytes, will continue without cache download 2024.10.22 01:02:33.070643 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 16dcaad49aba145bd41f65edea264c30, offset: 0, size: 0 2024.10.22 01:02:33.073182 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 16dcaad49aba145bd41f65edea264c30 for removal 2024.10.22 01:02:33.078975 [ 1476 ] {} CacheMetadata: Key 16dcaad49aba145bd41f65edea264c30 is removed from metadata 2024.10.22 01:02:33.079984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 342c67f48948f25ad93a6255c92636b3, offset: 0, size: 0 2024.10.22 01:02:33.081204 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 342c67f48948f25ad93a6255c92636b3 for removal 2024.10.22 01:02:33.084058 [ 1476 ] {} CacheMetadata: Key 342c67f48948f25ad93a6255c92636b3 is removed from metadata 2024.10.22 01:02:33.094144 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 503598 2024.10.22 01:02:33.346651 [ 1210 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 01:02:33.969574 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 274 2024.10.22 01:02:34.027126 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c6f843e05fdcdb6c006f52a2e3d8983e) : [0, 149]: Setting download as failed: File segment: [0, 149], key: c6f843e05fdcdb6c006f52a2e3d8983e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:34.028856 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c6f843e05fdcdb6c006f52a2e3d8983e): No space left in cache to reserve 150 bytes, will continue without cache download 2024.10.22 01:02:34.104419 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_316_102} MergeTask::MergeProjectionsStage: Merge sorted 23189 rows, containing 13 columns (13 merged, 0 gathered) in 2.111316203 sec., 10983.196153683855 rows/sec., 4.66 MiB/sec. 2024.10.22 01:02:34.192679 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_316_102} MergedBlockOutputStream: filled checksums 202410_1_316_102 (state Temporary) 2024.10.22 01:02:34.209805 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:34.219033 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:34.256309 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_257_66, 202410_258_258_0] 2024.10.22 01:02:34.259162 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 31960 2024.10.22 01:02:34.260421 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_257_66, 202410_258_258_0] 2024.10.22 01:02:34.272221 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_257_66, 202410_258_258_0] 2024.10.22 01:02:34.418160 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_256_256_0 (state Temporary) 2024.10.22 01:02:34.431756 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_316_102} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_316_102 to 202410_1_316_102 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:34.435714 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_316_102} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_315_101, 202410_316_316_0] -> 202410_1_316_102 2024.10.22 01:02:34.534703 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 8.80 MiB, will set to 1.69 GiB (RSS), difference: -1.08 MiB 2024.10.22 01:02:34.577820 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.37 MiB. 2024.10.22 01:02:34.583806 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_256_256_0 to 202410_266_266_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:34.668280 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 274 2024.10.22 01:02:34.882714 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:34.886570 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:35.060499 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_541_541_0 (state Temporary) 2024.10.22 01:02:35.081186 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(f95524f22e27b3bd3946f87396cbf07f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f95524f22e27b3bd3946f87396cbf07f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:02:35.084415 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 989, file segment info: File segment: [0, 33554431], key: f95524f22e27b3bd3946f87396cbf07f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:02:35.087203 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:35.243108 [ 1712 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ocu/rrilnxmbhtaihtbopyrzcjljoedqm, size 2261708 2024.10.22 01:02:35.389368 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_246_49, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0, 202410_250_250_0, 202410_251_251_0] 2024.10.22 01:02:35.392167 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_246_49, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0, 202410_250_250_0, 202410_251_251_0] 2024.10.22 01:02:35.432680 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_541_541_0 to 202410_554_554_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:35.442546 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:02:35.466475 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:02:35.476563 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:02:35.481717 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:02:35.483937 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_549_172 to 202410_550_550_0 2024.10.22 01:02:35.484285 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 31960 2024.10.22 01:02:35.486540 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.13 MiB using storage policy from min volume index 0 2024.10.22 01:02:35.489302 [ 1239 ] {} DiskLocal: Reserved 1.13 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:35.516537 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_246_49, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0, 202410_250_250_0, 202410_251_251_0] 2024.10.22 01:02:35.526768 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_550_173} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_549_172 to 202410_550_550_0 into Compact with storage Full 2024.10.22 01:02:35.589144 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_64_13/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:02:35.596728 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:02:35.602694 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:02:35.644044 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fe6207adab3729e335851403b0b2579e) : [0, 198025]: Setting download as failed: File segment: [0, 198025], key: fe6207adab3729e335851403b0b2579e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:35.645672 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fe6207adab3729e335851403b0b2579e): No space left in cache to reserve 198026 bytes, will continue without cache download 2024.10.22 01:02:35.673399 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fe6207adab3729e335851403b0b2579e, offset: 0, size: 0 2024.10.22 01:02:35.675413 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fe6207adab3729e335851403b0b2579e for removal 2024.10.22 01:02:35.677584 [ 1476 ] {} CacheMetadata: Key fe6207adab3729e335851403b0b2579e is removed from metadata 2024.10.22 01:02:35.679596 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c6f843e05fdcdb6c006f52a2e3d8983e, offset: 0, size: 0 2024.10.22 01:02:35.680889 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c6f843e05fdcdb6c006f52a2e3d8983e for removal 2024.10.22 01:02:35.682858 [ 1476 ] {} CacheMetadata: Key c6f843e05fdcdb6c006f52a2e3d8983e is removed from metadata 2024.10.22 01:02:35.704330 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:02:35.738105 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:02:35.748238 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_550_173} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:35.748810 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:02:35.766814 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_550_173} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_549_172, total 32415 rows starting from the beginning of the part 2024.10.22 01:02:35.799465 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_550_173} MergeTreeSequentialSource: Reading 2 marks from part 202410_550_550_0, total 67 rows starting from the beginning of the part 2024.10.22 01:02:35.885383 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mwb/wcsthspxdcwhoiavzscyfkpndonfu (LogSeriesLimiter: on interval from 2024-10-22 01:02:19 to 2024-10-22 01:02:35 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:02:35.970798 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_266_53, 202410_267_267_0, 202410_268_268_0, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0] 2024.10.22 01:02:35.991511 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_266_53, 202410_267_267_0, 202410_268_268_0, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0] 2024.10.22 01:02:36.122882 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_266_53, 202410_267_267_0, 202410_268_268_0, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0] 2024.10.22 01:02:36.381340 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(76af5b9227d76618eafd429ba3d70717) : [0, 149]: Setting download as failed: File segment: [0, 149], key: 76af5b9227d76618eafd429ba3d70717, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:36.383130 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(76af5b9227d76618eafd429ba3d70717): No space left in cache to reserve 150 bytes, will continue without cache download 2024.10.22 01:02:36.974820 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(e56424aa857fdb0c92ea9552cbb3409d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e56424aa857fdb0c92ea9552cbb3409d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:02:36.978360 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 910, file segment info: File segment: [0, 33554431], key: e56424aa857fdb0c92ea9552cbb3409d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:02:36.979997 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:37.142482 [ 1732 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vms/vmffqfwidshjvzdwthayirdpyreiw, size 2363042 2024.10.22 01:02:37.602793 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:02:37.604084 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2406514136657c0d7b5df6197d0d92b4) : [0, 168444]: Setting download as failed: File segment: [0, 168444], key: 2406514136657c0d7b5df6197d0d92b4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:37.607133 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2406514136657c0d7b5df6197d0d92b4): No space left in cache to reserve 168445 bytes, will continue without cache download 2024.10.22 01:02:37.612380 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:02:37.628201 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:02:37.653681 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:02:37.688000 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:02:37.792221 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:02:37.801585 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:02:37.804099 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_550_173} MergeTask::MergeProjectionsStage: Merge sorted 32482 rows, containing 13 columns (13 merged, 0 gathered) in 2.29021651 sec., 14182.938537981285 rows/sec., 2.18 MiB/sec. 2024.10.22 01:02:37.888779 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2406514136657c0d7b5df6197d0d92b4, offset: 0, size: 0 2024.10.22 01:02:37.891170 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2406514136657c0d7b5df6197d0d92b4 for removal 2024.10.22 01:02:37.893372 [ 1476 ] {} CacheMetadata: Key 2406514136657c0d7b5df6197d0d92b4 is removed from metadata 2024.10.22 01:02:37.893738 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 76af5b9227d76618eafd429ba3d70717, offset: 0, size: 0 2024.10.22 01:02:37.895034 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 76af5b9227d76618eafd429ba3d70717 for removal 2024.10.22 01:02:37.899568 [ 1476 ] {} CacheMetadata: Key 76af5b9227d76618eafd429ba3d70717 is removed from metadata 2024.10.22 01:02:37.928826 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_550_173} MergedBlockOutputStream: filled checksums 202410_1_550_173 (state Temporary) 2024.10.22 01:02:38.239105 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_550_173} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_550_173 to 202410_1_550_173 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:38.243187 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_550_173} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_549_172, 202410_550_550_0] -> 202410_1_550_173 2024.10.22 01:02:38.275841 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.70 MiB. 2024.10.22 01:02:38.296433 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 8.52 MiB, will set to 1.68 GiB (RSS), difference: -5.78 MiB 2024.10.22 01:02:38.400163 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 4609 2024.10.22 01:02:38.681277 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(346848f928b24bd27e0ff0accd3d1f12) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 346848f928b24bd27e0ff0accd3d1f12, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:38.682607 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(346848f928b24bd27e0ff0accd3d1f12): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:02:38.726487 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 3 old parts to remove. Parts: [202410_1_449_108, 202410_450_450_0, 202410_451_451_0] 2024.10.22 01:02:38.729201 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 3 parts from filesystem (serially): Parts: [202410_1_449_108, 202410_450_450_0, 202410_451_451_0] 2024.10.22 01:02:38.748260 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 3 parts from memory: Parts: [202410_1_449_108, 202410_450_450_0, 202410_451_451_0] 2024.10.22 01:02:38.892361 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:38.895611 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:38.927312 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_318_318_0 (state Temporary) 2024.10.22 01:02:39.208338 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_318_318_0 to 202410_331_331_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:39.266177 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4609 2024.10.22 01:02:39.345538 [ 1246 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_326_65 to 202410_331_331_0 2024.10.22 01:02:39.364985 [ 1246 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:39.370531 [ 1246 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.65 GiB. 2024.10.22 01:02:39.477176 [ 1246 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 252 ms. 2024.10.22 01:02:39.490494 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_326_65 to 202410_331_331_0 into Compact with storage Full 2024.10.22 01:02:39.526231 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 122 entries to flush up to offset 23781 2024.10.22 01:02:39.719747 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4d8444ba7bf204abf1fb1a18210c3739) : [0, 389978]: Setting download as failed: File segment: [0, 389978], key: 4d8444ba7bf204abf1fb1a18210c3739, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:39.843009 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:02:39.845984 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:02:39.890982 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 94 entries to flush up to offset 32054 2024.10.22 01:02:39.892950 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4d8444ba7bf204abf1fb1a18210c3739): No space left in cache to reserve 389979 bytes, will continue without cache download 2024.10.22 01:02:40.013436 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:40.019839 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:40.028793 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4d8444ba7bf204abf1fb1a18210c3739, offset: 0, size: 0 2024.10.22 01:02:40.032705 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4d8444ba7bf204abf1fb1a18210c3739 for removal 2024.10.22 01:02:40.036207 [ 1476 ] {} CacheMetadata: Key 4d8444ba7bf204abf1fb1a18210c3739 is removed from metadata 2024.10.22 01:02:40.040533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 346848f928b24bd27e0ff0accd3d1f12, offset: 0, size: 0 2024.10.22 01:02:40.042949 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 346848f928b24bd27e0ff0accd3d1f12 for removal 2024.10.22 01:02:40.046561 [ 1476 ] {} CacheMetadata: Key 346848f928b24bd27e0ff0accd3d1f12 is removed from metadata 2024.10.22 01:02:40.055389 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:40.063012 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_315_315_0 (state Temporary) 2024.10.22 01:02:40.069491 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_326_65, total 4627 rows starting from the beginning of the part 2024.10.22 01:02:40.107539 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_327_327_0, total 26 rows starting from the beginning of the part 2024.10.22 01:02:40.112387 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 505134 2024.10.22 01:02:40.198367 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_315_315_0 to 202410_321_321_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:40.227157 [ 1253 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_316_102 to 202410_317_317_0 2024.10.22 01:02:40.234769 [ 1253 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.29 MiB using storage policy from min volume index 0 2024.10.22 01:02:40.236919 [ 1253 ] {} DiskLocal: Reserved 1.29 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:40.255917 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 23781 2024.10.22 01:02:40.391578 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_317_103} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_316_102 to 202410_317_317_0 into Compact with storage Full 2024.10.22 01:02:40.416157 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:40.424912 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:40.436141 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_317_103} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:40.512010 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 6.05 MiB, will set to 1.68 GiB (RSS), difference: -3.56 MiB 2024.10.22 01:02:40.544486 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_328_328_0, total 13 rows starting from the beginning of the part 2024.10.22 01:02:40.545146 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_317_103} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_316_102, total 23189 rows starting from the beginning of the part 2024.10.22 01:02:40.623322 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_317_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_317_317_0, total 145 rows starting from the beginning of the part 2024.10.22 01:02:40.685440 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_329_329_0, total 19 rows starting from the beginning of the part 2024.10.22 01:02:40.703075 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_542_542_0 (state Temporary) 2024.10.22 01:02:40.744596 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_330_330_0, total 15 rows starting from the beginning of the part 2024.10.22 01:02:40.759703 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_542_542_0 to 202410_555_555_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:40.767474 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_331_331_0, total 15 rows starting from the beginning of the part 2024.10.22 01:02:40.791731 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_550_173 to 202410_551_551_0 2024.10.22 01:02:40.805009 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.13 MiB using storage policy from min volume index 0 2024.10.22 01:02:40.808701 [ 1259 ] {} DiskLocal: Reserved 1.13 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:40.828693 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_551_174} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_550_173 to 202410_551_551_0 into Compact with storage Full 2024.10.22 01:02:40.922721 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32054 2024.10.22 01:02:40.973356 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:40.976457 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:40.981202 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_551_174} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:40.990328 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_551_174} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_550_173, total 32482 rows starting from the beginning of the part 2024.10.22 01:02:41.013623 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_551_174} MergeTreeSequentialSource: Reading 2 marks from part 202410_551_551_0, total 92 rows starting from the beginning of the part 2024.10.22 01:02:41.091203 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_331_66/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:02:41.105748 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(15b773b59b907c7beba2c8dd129811b8) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 15b773b59b907c7beba2c8dd129811b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:41.107473 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(15b773b59b907c7beba2c8dd129811b8): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:02:41.130800 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_304_304_0 (state Temporary) 2024.10.22 01:02:41.195454 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_304_304_0 to 202410_311_311_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:41.228142 [ 1260 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_306_61 to 202410_311_311_0 2024.10.22 01:02:41.242947 [ 1260 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:41.250402 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:41.302963 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_306_61 to 202410_311_311_0 into Compact with storage Full 2024.10.22 01:02:41.339942 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 505134 2024.10.22 01:02:41.360170 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:41.373699 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} MergeTreeSequentialSource: Reading 64 marks from part 202410_1_306_61, total 511812 rows starting from the beginning of the part 2024.10.22 01:02:41.393287 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_307_307_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:02:41.408684 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_308_308_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:02:41.430775 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_309_309_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:02:41.462019 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_310_310_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:02:41.491315 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_311_311_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:02:41.872370 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(739ad4b7087b017e1684f97e94904e16) : [0, 277432]: Setting download as failed: File segment: [0, 277432], key: 739ad4b7087b017e1684f97e94904e16, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:41.876169 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(739ad4b7087b017e1684f97e94904e16): No space left in cache to reserve 277433 bytes, will continue without cache download 2024.10.22 01:02:41.898538 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} MergeTask::MergeProjectionsStage: Merge sorted 4715 rows, containing 12 columns (12 merged, 0 gathered) in 2.412816787 sec., 1954.1475446473673 rows/sec., 307.06 KiB/sec. 2024.10.22 01:02:41.899235 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 739ad4b7087b017e1684f97e94904e16, offset: 0, size: 0 2024.10.22 01:02:41.903155 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 739ad4b7087b017e1684f97e94904e16 for removal 2024.10.22 01:02:41.906794 [ 1476 ] {} CacheMetadata: Key 739ad4b7087b017e1684f97e94904e16 is removed from metadata 2024.10.22 01:02:42.007771 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 15b773b59b907c7beba2c8dd129811b8, offset: 0, size: 0 2024.10.22 01:02:42.009335 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 15b773b59b907c7beba2c8dd129811b8 for removal 2024.10.22 01:02:42.015986 [ 1476 ] {} CacheMetadata: Key 15b773b59b907c7beba2c8dd129811b8 is removed from metadata 2024.10.22 01:02:42.041918 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_551_174/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:02:42.088355 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_317_103} MergeTask::MergeProjectionsStage: Merge sorted 23334 rows, containing 13 columns (13 merged, 0 gathered) in 1.703203499 sec., 13700.06579583712 rows/sec., 5.81 MiB/sec. 2024.10.22 01:02:42.151180 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} MergedBlockOutputStream: filled checksums 202410_1_331_66 (state Temporary) 2024.10.22 01:02:42.170551 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 275 2024.10.22 01:02:42.299613 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_317_103} MergedBlockOutputStream: filled checksums 202410_1_317_103 (state Temporary) 2024.10.22 01:02:42.453663 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:42.461426 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:42.632692 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_317_103} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_317_103 to 202410_1_317_103 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:42.635873 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_317_103} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_316_102, 202410_317_317_0] -> 202410_1_317_103 2024.10.22 01:02:42.638429 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_257_257_0 (state Temporary) 2024.10.22 01:02:42.701195 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_331_66 to 202410_1_331_66 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:42.703888 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_331_66} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_326_65, 202410_331_331_0] -> 202410_1_331_66 2024.10.22 01:02:42.729181 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.42 MiB. 2024.10.22 01:02:42.759736 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_551_174} MergeTask::MergeProjectionsStage: Merge sorted 32574 rows, containing 13 columns (13 merged, 0 gathered) in 1.939880733 sec., 16791.75397016534 rows/sec., 2.58 MiB/sec. 2024.10.22 01:02:42.784420 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.37 MiB. 2024.10.22 01:02:42.825840 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_257_257_0 to 202410_267_267_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:42.845134 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6530c2af0974aa5f84e2dcb932ef6c66) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 6530c2af0974aa5f84e2dcb932ef6c66, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:42.848396 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6530c2af0974aa5f84e2dcb932ef6c66): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:02:42.907678 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 275 2024.10.22 01:02:42.946949 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_551_174} MergedBlockOutputStream: filled checksums 202410_1_551_174 (state Temporary) 2024.10.22 01:02:43.258858 [ 1178 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:02:43.283531 [ 1178 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:02:43.323203 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_551_174} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_551_174 to 202410_1_551_174 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:43.329627 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_551_174} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_550_173, 202410_551_551_0] -> 202410_1_551_174 2024.10.22 01:02:43.330545 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_311_62/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:02:43.371278 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.71 MiB. 2024.10.22 01:02:43.385026 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 7.15 MiB, will set to 1.68 GiB (RSS), difference: -3.42 MiB 2024.10.22 01:02:43.531306 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_258_67, 202410_259_259_0] 2024.10.22 01:02:43.534807 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_258_67, 202410_259_259_0] 2024.10.22 01:02:43.567014 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_258_67, 202410_259_259_0] 2024.10.22 01:02:43.891738 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b47efeb5fe106b6637948cf885a0e790) : [0, 1235]: Setting download as failed: File segment: [0, 1235], key: b47efeb5fe106b6637948cf885a0e790, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:43.892922 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b47efeb5fe106b6637948cf885a0e790): No space left in cache to reserve 1236 bytes, will continue without cache download 2024.10.22 01:02:43.903325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b47efeb5fe106b6637948cf885a0e790, offset: 0, size: 0 2024.10.22 01:02:43.905974 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b47efeb5fe106b6637948cf885a0e790 for removal 2024.10.22 01:02:43.908471 [ 1476 ] {} CacheMetadata: Key b47efeb5fe106b6637948cf885a0e790 is removed from metadata 2024.10.22 01:02:43.909042 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6530c2af0974aa5f84e2dcb932ef6c66, offset: 0, size: 0 2024.10.22 01:02:43.911930 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6530c2af0974aa5f84e2dcb932ef6c66 for removal 2024.10.22 01:02:43.917405 [ 1476 ] {} CacheMetadata: Key 6530c2af0974aa5f84e2dcb932ef6c66 is removed from metadata 2024.10.22 01:02:44.511838 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(17c54cd28559ba947e530ef27b487d2d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 17c54cd28559ba947e530ef27b487d2d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:02:44.525326 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 17c54cd28559ba947e530ef27b487d2d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:02:44.529540 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:44.837475 [ 1163 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 213 ms. 2024.10.22 01:02:44.950572 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 115 entries to flush up to offset 32169 2024.10.22 01:02:45.248191 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(da927899838a9d36d13e21a9e101cd4b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: da927899838a9d36d13e21a9e101cd4b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:02:45.250465 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: da927899838a9d36d13e21a9e101cd4b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:02:45.262668 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:45.404694 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:45.411908 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:45.601745 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_543_543_0 (state Temporary) 2024.10.22 01:02:45.673071 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_543_543_0 to 202410_556_556_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:45.714787 [ 1225 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_551_174 to 202410_552_552_0 2024.10.22 01:02:45.720137 [ 1225 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.13 MiB using storage policy from min volume index 0 2024.10.22 01:02:45.723415 [ 1225 ] {} DiskLocal: Reserved 1.13 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:45.726268 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32169 2024.10.22 01:02:45.847143 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_552_175} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_551_174 to 202410_552_552_0 into Compact with storage Full 2024.10.22 01:02:45.873335 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_552_175} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:45.878537 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(27f67cb25b19538a01a54d064fbc0713) : [0, 184]: Setting download as failed: File segment: [0, 184], key: 27f67cb25b19538a01a54d064fbc0713, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:45.880080 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(27f67cb25b19538a01a54d064fbc0713): No space left in cache to reserve 185 bytes, will continue without cache download 2024.10.22 01:02:45.884005 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_552_175} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_551_174, total 32574 rows starting from the beginning of the part 2024.10.22 01:02:45.907111 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_552_175} MergeTreeSequentialSource: Reading 2 marks from part 202410_552_552_0, total 68 rows starting from the beginning of the part 2024.10.22 01:02:46.343978 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} MergeTask::MergeProjectionsStage: Merge sorted 2514 rows, containing 679 columns (679 merged, 0 gathered) in 105.16612846 sec., 23.905035174478268 rows/sec., 31.95 KiB/sec. 2024.10.22 01:02:46.399579 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 9.41 MiB, will set to 1.68 GiB (RSS), difference: -6.62 MiB 2024.10.22 01:02:46.508626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(93cd74f096f30be03707d6551f5c3dfb) : [0, 989572]: Setting download as failed: File segment: [0, 989572], key: 93cd74f096f30be03707d6551f5c3dfb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:46.509602 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(93cd74f096f30be03707d6551f5c3dfb): No space left in cache to reserve 989573 bytes, will continue without cache download 2024.10.22 01:02:46.619381 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_64_13/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:02:46.672689 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 93cd74f096f30be03707d6551f5c3dfb, offset: 0, size: 0 2024.10.22 01:02:46.675499 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 93cd74f096f30be03707d6551f5c3dfb for removal 2024.10.22 01:02:46.679292 [ 1476 ] {} CacheMetadata: Key 93cd74f096f30be03707d6551f5c3dfb is removed from metadata 2024.10.22 01:02:46.681623 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 27f67cb25b19538a01a54d064fbc0713, offset: 0, size: 0 2024.10.22 01:02:46.686986 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 27f67cb25b19538a01a54d064fbc0713 for removal 2024.10.22 01:02:46.689777 [ 1476 ] {} CacheMetadata: Key 27f67cb25b19538a01a54d064fbc0713 is removed from metadata 2024.10.22 01:02:46.770868 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 4628 2024.10.22 01:02:46.864420 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:46.869079 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:46.905363 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_319_319_0 (state Temporary) 2024.10.22 01:02:46.961088 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_319_319_0 to 202410_332_332_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:47.017156 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4628 2024.10.22 01:02:47.734990 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(09232792731608f62a7e5f243ecebdaa) : [0, 185]: Setting download as failed: File segment: [0, 185], key: 09232792731608f62a7e5f243ecebdaa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:47.739375 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(09232792731608f62a7e5f243ecebdaa): No space left in cache to reserve 186 bytes, will continue without cache download 2024.10.22 01:02:47.798823 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 158 entries to flush up to offset 23939 2024.10.22 01:02:47.801914 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:47.808284 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:47.838905 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_552_175} MergeTask::MergeProjectionsStage: Merge sorted 32642 rows, containing 13 columns (13 merged, 0 gathered) in 2.004327168 sec., 16285.76438075802 rows/sec., 2.50 MiB/sec. 2024.10.22 01:02:48.147888 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:48.160761 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:48.220328 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_552_175} MergedBlockOutputStream: filled checksums 202410_1_552_175 (state Temporary) 2024.10.22 01:02:48.225163 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_316_316_0 (state Temporary) 2024.10.22 01:02:48.290991 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_316_316_0 to 202410_322_322_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:48.324473 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_317_103 to 202410_318_318_0 2024.10.22 01:02:48.338998 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.29 MiB using storage policy from min volume index 0 2024.10.22 01:02:48.342927 [ 1150 ] {} DiskLocal: Reserved 1.29 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:48.348336 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_552_175} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_552_175 to 202410_1_552_175 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:48.350105 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 506286 2024.10.22 01:02:48.354348 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_552_175} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_551_174, 202410_552_552_0] -> 202410_1_552_175 2024.10.22 01:02:48.357625 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 15.05 MiB, will set to 1.69 GiB (RSS), difference: 16.65 MiB 2024.10.22 01:02:48.369587 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_318_104} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_317_103 to 202410_318_318_0 into Compact with storage Full 2024.10.22 01:02:48.388949 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.74 MiB. 2024.10.22 01:02:48.395568 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_318_104} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:48.406579 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_318_104} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_317_103, total 23334 rows starting from the beginning of the part 2024.10.22 01:02:48.509060 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} MergedBlockOutputStream: filled checksums 202410_1_64_13 (state Temporary) 2024.10.22 01:02:48.544990 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 23939 2024.10.22 01:02:48.557489 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_318_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_318_318_0, total 142 rows starting from the beginning of the part 2024.10.22 01:02:48.577768 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(433839015bc9045b6358a7a0e0ee39bc) : [0, 1088994]: Setting download as failed: File segment: [0, 1088994], key: 433839015bc9045b6358a7a0e0ee39bc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:48.616626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(433839015bc9045b6358a7a0e0ee39bc): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:02:48.659723 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_214_44, 202410_215_215_0, 202410_216_216_0, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0] 2024.10.22 01:02:48.671940 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_214_44, 202410_215_215_0, 202410_216_216_0, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0] 2024.10.22 01:02:48.701604 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 433839015bc9045b6358a7a0e0ee39bc, offset: 0, size: 0 2024.10.22 01:02:48.705443 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 433839015bc9045b6358a7a0e0ee39bc for removal 2024.10.22 01:02:48.708623 [ 1476 ] {} CacheMetadata: Key 433839015bc9045b6358a7a0e0ee39bc is removed from metadata 2024.10.22 01:02:48.711897 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 09232792731608f62a7e5f243ecebdaa, offset: 0, size: 0 2024.10.22 01:02:48.714236 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 09232792731608f62a7e5f243ecebdaa for removal 2024.10.22 01:02:48.720535 [ 1476 ] {} CacheMetadata: Key 09232792731608f62a7e5f243ecebdaa is removed from metadata 2024.10.22 01:02:48.806704 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_214_44, 202410_215_215_0, 202410_216_216_0, 202410_217_217_0, 202410_218_218_0, 202410_219_219_0] 2024.10.22 01:02:49.268618 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:49.273565 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:49.486754 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_305_305_0 (state Temporary) 2024.10.22 01:02:49.503875 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d9c1521b67e2f8ff6be4210ee0f97b8b) : [0, 153]: Setting download as failed: File segment: [0, 153], key: d9c1521b67e2f8ff6be4210ee0f97b8b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:49.507453 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d9c1521b67e2f8ff6be4210ee0f97b8b): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:02:49.576292 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_305_305_0 to 202410_312_312_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:49.698544 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 506286 2024.10.22 01:02:49.795343 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_451_109, 202410_452_452_0, 202410_453_453_0, 202410_454_454_0, 202410_455_455_0, 202410_456_456_0] 2024.10.22 01:02:49.843851 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:02:49.845978 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 32248 2024.10.22 01:02:49.848745 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_451_109, 202410_452_452_0, 202410_453_453_0, 202410_454_454_0, 202410_455_455_0, 202410_456_456_0] 2024.10.22 01:02:49.849726 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:02:49.877502 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_451_109, 202410_452_452_0, 202410_453_453_0, 202410_454_454_0, 202410_455_455_0, 202410_456_456_0] 2024.10.22 01:02:49.908649 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_318_104} MergeTask::MergeProjectionsStage: Merge sorted 23476 rows, containing 13 columns (13 merged, 0 gathered) in 1.539326685 sec., 15250.8237716934 rows/sec., 6.47 MiB/sec. 2024.10.22 01:02:50.052061 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:50.062769 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:50.132260 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_544_544_0 (state Temporary) 2024.10.22 01:02:50.193182 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_544_544_0 to 202410_557_557_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:50.246644 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_552_175 to 202410_553_553_0 2024.10.22 01:02:50.260871 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.14 MiB using storage policy from min volume index 0 2024.10.22 01:02:50.263645 [ 1195 ] {} DiskLocal: Reserved 1.14 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:50.283901 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32248 2024.10.22 01:02:50.284781 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_553_176} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_552_175 to 202410_553_553_0 into Compact with storage Full 2024.10.22 01:02:50.305293 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_318_104} MergedBlockOutputStream: filled checksums 202410_1_318_104 (state Temporary) 2024.10.22 01:02:50.340809 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_553_176} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:50.354782 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_553_176} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_552_175, total 32642 rows starting from the beginning of the part 2024.10.22 01:02:50.390725 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_553_176} MergeTreeSequentialSource: Reading 2 marks from part 202410_553_553_0, total 72 rows starting from the beginning of the part 2024.10.22 01:02:50.397817 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 16.62 MiB, will set to 1.69 GiB (RSS), difference: -2.97 MiB 2024.10.22 01:02:50.418877 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 276 2024.10.22 01:02:50.492009 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5ad2855b75e80082ef1aef9218b20602) : [0, 372376]: Setting download as failed: File segment: [0, 372376], key: 5ad2855b75e80082ef1aef9218b20602, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:50.494119 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5ad2855b75e80082ef1aef9218b20602): No space left in cache to reserve 372377 bytes, will continue without cache download 2024.10.22 01:02:50.532523 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5ad2855b75e80082ef1aef9218b20602, offset: 0, size: 0 2024.10.22 01:02:50.542096 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5ad2855b75e80082ef1aef9218b20602 for removal 2024.10.22 01:02:50.551825 [ 1476 ] {} CacheMetadata: Key 5ad2855b75e80082ef1aef9218b20602 is removed from metadata 2024.10.22 01:02:50.557386 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d9c1521b67e2f8ff6be4210ee0f97b8b, offset: 0, size: 0 2024.10.22 01:02:50.559123 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d9c1521b67e2f8ff6be4210ee0f97b8b for removal 2024.10.22 01:02:50.563123 [ 1476 ] {} CacheMetadata: Key d9c1521b67e2f8ff6be4210ee0f97b8b is removed from metadata 2024.10.22 01:02:50.641631 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:50.647978 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:50.824119 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_318_104} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_318_104 to 202410_1_318_104 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:50.829494 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_318_104} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_317_103, 202410_318_318_0] -> 202410_1_318_104 2024.10.22 01:02:50.931907 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.37 MiB. 2024.10.22 01:02:51.226608 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_258_258_0 (state Temporary) 2024.10.22 01:02:51.486297 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(810b197b46a8a7be9a326105557cea4b) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 810b197b46a8a7be9a326105557cea4b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:51.487358 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(810b197b46a8a7be9a326105557cea4b): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:02:51.876116 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_258_258_0 to 202410_268_268_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:51.900166 [ 1208 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_263_54 to 202410_268_268_0 2024.10.22 01:02:51.911981 [ 1208 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:51.926493 [ 1208 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:51.937621 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 276 2024.10.22 01:02:51.934966 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_553_176} MergeTask::MergeProjectionsStage: Merge sorted 32714 rows, containing 13 columns (13 merged, 0 gathered) in 1.654915976 sec., 19767.77097715322 rows/sec., 3.04 MiB/sec. 2024.10.22 01:02:52.042592 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_553_176} MergedBlockOutputStream: filled checksums 202410_1_553_176 (state Temporary) 2024.10.22 01:02:52.047397 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_263_54 to 202410_268_268_0 into Compact with storage Full 2024.10.22 01:02:52.195976 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6529c101f39b41ea11c8fbfe20cd3b6e) : [0, 83509]: Setting download as failed: File segment: [0, 83509], key: 6529c101f39b41ea11c8fbfe20cd3b6e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:02:52.199119 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6529c101f39b41ea11c8fbfe20cd3b6e): No space left in cache to reserve 83510 bytes, will continue without cache download 2024.10.22 01:02:52.209454 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_259_68, 202410_260_260_0] 2024.10.22 01:02:52.214727 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_259_68, 202410_260_260_0] 2024.10.22 01:02:52.229042 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6529c101f39b41ea11c8fbfe20cd3b6e, offset: 0, size: 0 2024.10.22 01:02:52.231119 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6529c101f39b41ea11c8fbfe20cd3b6e for removal 2024.10.22 01:02:52.233733 [ 1476 ] {} CacheMetadata: Key 6529c101f39b41ea11c8fbfe20cd3b6e is removed from metadata 2024.10.22 01:02:52.235817 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 810b197b46a8a7be9a326105557cea4b, offset: 0, size: 0 2024.10.22 01:02:52.238047 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 810b197b46a8a7be9a326105557cea4b for removal 2024.10.22 01:02:52.239885 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ygi/ykonxchhfcqdpwjncjvykgtbyocno. 2024.10.22 01:02:52.241676 [ 1476 ] {} CacheMetadata: Key 810b197b46a8a7be9a326105557cea4b is removed from metadata 2024.10.22 01:02:52.283619 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_553_176} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_553_176 to 202410_1_553_176 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:52.287368 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_553_176} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_552_175, 202410_553_553_0] -> 202410_1_553_176 2024.10.22 01:02:52.306063 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.75 MiB. 2024.10.22 01:02:52.332375 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_259_68, 202410_260_260_0] 2024.10.22 01:02:52.403348 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_64_64_0 (state Temporary) 2024.10.22 01:02:52.456556 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:52.474671 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_263_54, total 307 rows starting from the beginning of the part 2024.10.22 01:02:52.510486 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_264_264_0, total 1 rows starting from the beginning of the part 2024.10.22 01:02:52.565380 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_265_265_0, total 1 rows starting from the beginning of the part 2024.10.22 01:02:52.596513 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_266_266_0, total 1 rows starting from the beginning of the part 2024.10.22 01:02:52.645431 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 12.10 MiB, will set to 1.69 GiB (RSS), difference: 1.75 MiB 2024.10.22 01:02:52.652108 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_267_267_0, total 1 rows starting from the beginning of the part 2024.10.22 01:02:52.687254 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_268_268_0, total 1 rows starting from the beginning of the part 2024.10.22 01:02:53.090870 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_18_18_0 to 201403_18_18_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:53.099819 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ygi/ykonxchhfcqdpwjncjvykgtbyocno. 2024.10.22 01:02:53.215765 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:02:53.782200 [ 1222 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 01:02:53.790853 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_311_62/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:02:54.093496 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 18 2024.10.22 01:02:54.297125 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 32324 2024.10.22 01:02:54.393669 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:54.398596 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:54.521578 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 4643 2024.10.22 01:02:54.572369 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.88 GiB, free memory in arenas 13.11 MiB, will set to 1.49 GiB (RSS), difference: 347.06 MiB 2024.10.22 01:02:54.624926 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:54.628664 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:54.687901 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:54.690236 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:54.700561 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_545_545_0 (state Temporary) 2024.10.22 01:02:54.787358 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_18_18_0 (state Temporary) 2024.10.22 01:02:54.828191 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_320_320_0 (state Temporary) 2024.10.22 01:02:54.962260 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_545_545_0 to 202410_558_558_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:54.983207 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_553_176 to 202410_554_554_0 2024.10.22 01:02:54.987779 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_320_320_0 to 202410_333_333_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:54.992085 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.14 MiB using storage policy from min volume index 0 2024.10.22 01:02:54.995887 [ 1223 ] {} DiskLocal: Reserved 1.14 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:55.009246 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} MergeTask::MergeProjectionsStage: Merge sorted 312 rows, containing 32 columns (32 merged, 0 gathered) in 2.968584989 sec., 105.10057861105759 rows/sec., 10.81 KiB/sec. 2024.10.22 01:02:55.021030 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32324 2024.10.22 01:02:55.055146 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_18_18_0 to 202410_18_18_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:55.122905 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_554_177} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_553_176 to 202410_554_554_0 into Compact with storage Full 2024.10.22 01:02:55.155770 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} MergedBlockOutputStream: filled checksums 202410_1_268_55 (state Temporary) 2024.10.22 01:02:55.209144 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_554_177} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:55.257373 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4643 2024.10.22 01:02:55.260068 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_554_177} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_553_176, total 32714 rows starting from the beginning of the part 2024.10.22 01:02:55.291344 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 18 2024.10.22 01:02:55.330944 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(b916d2bc80c2eafe1c20d1a0059e1e38) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b916d2bc80c2eafe1c20d1a0059e1e38, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:02:55.334720 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 920, file segment info: File segment: [0, 33554431], key: b916d2bc80c2eafe1c20d1a0059e1e38, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:02:55.340441 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:55.341773 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_554_177} MergeTreeSequentialSource: Reading 2 marks from part 202410_554_554_0, total 77 rows starting from the beginning of the part 2024.10.22 01:02:55.677773 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:02:55.686879 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_268_55 to 202410_1_268_55 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:55.690689 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_268_55} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_263_54, 202410_268_268_0] -> 202410_1_268_55 2024.10.22 01:02:55.699554 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:02:55.707727 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:02:55.735269 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:02:55.759422 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:02:55.789802 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:02:55.833637 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:02:55.858916 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:02:55.917258 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/azg/hkskhgwkpmevwrmpgqujcndzipffi (LogSeriesLimiter: on interval from 2024-10-22 01:02:35 to 2024-10-22 01:02:55 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:02:56.212224 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 145 entries to flush up to offset 24084 2024.10.22 01:02:56.665070 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:56.672646 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:56.705077 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_317_317_0 (state Temporary) 2024.10.22 01:02:56.705309 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 507822 2024.10.22 01:02:56.755898 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_317_317_0 to 202410_323_323_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:56.788404 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_318_104 to 202410_319_319_0 2024.10.22 01:02:56.795333 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.30 MiB using storage policy from min volume index 0 2024.10.22 01:02:56.808327 [ 1256 ] {} DiskLocal: Reserved 1.30 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:56.849180 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_319_105} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_318_104 to 202410_319_319_0 into Compact with storage Full 2024.10.22 01:02:56.913597 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 24084 2024.10.22 01:02:57.104623 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_319_105} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:57.200248 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_319_105} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_318_104, total 23476 rows starting from the beginning of the part 2024.10.22 01:02:57.205663 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_554_177} MergeTask::MergeProjectionsStage: Merge sorted 32791 rows, containing 13 columns (13 merged, 0 gathered) in 2.094092397 sec., 15658.81240339559 rows/sec., 2.41 MiB/sec. 2024.10.22 01:02:57.281582 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_319_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_319_319_0, total 161 rows starting from the beginning of the part 2024.10.22 01:02:57.474627 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:57.481056 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:57.502175 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_554_177} MergedBlockOutputStream: filled checksums 202410_1_554_177 (state Temporary) 2024.10.22 01:02:57.692050 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 7.83 MiB, will set to 1.48 GiB (RSS), difference: -251.02 MiB 2024.10.22 01:02:57.697118 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_554_177} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_554_177 to 202410_1_554_177 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:57.699861 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_554_177} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_553_176, 202410_554_554_0] -> 202410_1_554_177 2024.10.22 01:02:57.714733 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_306_306_0 (state Temporary) 2024.10.22 01:02:57.800520 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_306_306_0 to 202410_313_313_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:57.813614 [ 1237 ] {} DatabaseCatalog: Execution took 268 ms. 2024.10.22 01:02:57.836920 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.75 MiB. 2024.10.22 01:02:57.945156 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 507822 2024.10.22 01:02:58.288009 [ 1253 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:02:58.440718 [ 1253 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:02:59.035001 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 32393 2024.10.22 01:02:59.185806 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(adc28596f83bc528d9d4d301a6b2c896) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: adc28596f83bc528d9d4d301a6b2c896, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:02:59.189001 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1185, file segment info: File segment: [0, 33554431], key: adc28596f83bc528d9d4d301a6b2c896, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:02:59.191099 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:02:59.313466 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:02:59.322542 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.64 GiB. 2024.10.22 01:02:59.386987 [ 1684 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mwb/wcsthspxdcwhoiavzscyfkpndonfu, size 3869948 2024.10.22 01:02:59.408443 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_546_546_0 (state Temporary) 2024.10.22 01:02:59.477303 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_546_546_0 to 202410_559_559_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:59.511610 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_554_177 to 202410_555_555_0 2024.10.22 01:02:59.536254 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.14 MiB using storage policy from min volume index 0 2024.10.22 01:02:59.623621 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_319_105} MergeTask::MergeProjectionsStage: Merge sorted 23637 rows, containing 13 columns (13 merged, 0 gathered) in 2.79140555 sec., 8467.777102470833 rows/sec., 3.60 MiB/sec. 2024.10.22 01:02:59.643079 [ 1264 ] {} DiskLocal: Reserved 1.14 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:02:59.673012 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32393 2024.10.22 01:02:59.678502 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_555_178} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_554_177 to 202410_555_555_0 into Compact with storage Full 2024.10.22 01:02:59.704580 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_555_178} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:02:59.709810 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_555_178} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_554_177, total 32791 rows starting from the beginning of the part 2024.10.22 01:02:59.719897 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_319_105} MergedBlockOutputStream: filled checksums 202410_1_319_105 (state Temporary) 2024.10.22 01:02:59.725443 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_64_64_0 to 202410_67_67_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:02:59.747554 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_555_178} MergeTreeSequentialSource: Reading 2 marks from part 202410_555_555_0, total 94 rows starting from the beginning of the part 2024.10.22 01:02:59.849363 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:02:59.854765 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:02:59.878335 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Refresh 2024.10.22 01:02:59.888717 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Refresh 2024.10.22 01:02:59.931020 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_64_13/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:03:00.022512 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Refresh 2024.10.22 01:03:00.083941 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_319_105} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_319_105 to 202410_1_319_105 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:00.087324 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_319_105} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_318_104, 202410_319_319_0] -> 202410_1_319_105 2024.10.22 01:03:00.157421 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Refresh 2024.10.22 01:03:00.198930 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} MergeTask::MergeProjectionsStage: Merge sorted 518724 rows, containing 4 columns (4 merged, 0 gathered) in 18.912419553 sec., 27427.691023157156 rows/sec., 402.13 KiB/sec. 2024.10.22 01:03:00.285522 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Refresh 2024.10.22 01:03:00.291382 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.38 MiB. 2024.10.22 01:03:00.326389 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Refresh 2024.10.22 01:03:00.361008 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} MergedBlockOutputStream: filled checksums 202410_1_311_62 (state Temporary) 2024.10.22 01:03:00.362118 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Refresh 2024.10.22 01:03:00.387819 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Refresh 2024.10.22 01:03:00.397380 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_311_62 to 202410_1_311_62 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:00.400565 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_311_62} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_306_61, 202410_311_311_0] -> 202410_1_311_62 2024.10.22 01:03:00.420309 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Refresh 2024.10.22 01:03:00.486944 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:03:00.506140 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pqf/vyfqqiicweffbarajxrsvwavaodxc (LogSeriesLimiter: on interval from 2024-10-22 01:02:55 to 2024-10-22 01:03:00 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:03:00.745068 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(907a15295684a5ef4242e6a2fcfa9ac6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 907a15295684a5ef4242e6a2fcfa9ac6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:03:00.751119 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 907a15295684a5ef4242e6a2fcfa9ac6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:03:00.764591 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:00.922994 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_260_69, 202410_261_261_0] 2024.10.22 01:03:00.925261 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_260_69, 202410_261_261_0] 2024.10.22 01:03:00.942605 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_260_69, 202410_261_261_0] 2024.10.22 01:03:01.305469 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.88 GiB, free memory in arenas 7.52 MiB, will set to 1.56 GiB (RSS), difference: -29.34 MiB 2024.10.22 01:03:02.074923 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_555_178} MergeTask::MergeProjectionsStage: Merge sorted 32885 rows, containing 13 columns (13 merged, 0 gathered) in 2.407918493 sec., 13657.023730495515 rows/sec., 2.10 MiB/sec. 2024.10.22 01:03:02.152856 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_555_178} MergedBlockOutputStream: filled checksums 202410_1_555_178 (state Temporary) 2024.10.22 01:03:02.408520 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_555_178} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_555_178 to 202410_1_555_178 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:02.412490 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_555_178} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_554_177, 202410_555_555_0] -> 202410_1_555_178 2024.10.22 01:03:02.632476 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.77 MiB. 2024.10.22 01:03:02.681165 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.88 GiB, free memory in arenas 7.47 MiB, will set to 1.58 GiB (RSS), difference: -25.58 MiB 2024.10.22 01:03:02.668802 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2571 2024.10.22 01:03:02.759820 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 30 entries to flush up to offset 4673 2024.10.22 01:03:02.963195 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:02.969702 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:02.996476 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_321_321_0 (state Temporary) 2024.10.22 01:03:03.062529 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_insert_202410_321_321_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:03:03.199504 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_64_13 to 202410_1_64_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:03.220147 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_64_13} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_59_12, 202410_64_64_0] -> 202410_1_64_13 2024.10.22 01:03:03.352411 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_321_321_0 to 202410_334_334_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:03.394489 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4673 2024.10.22 01:03:03.743916 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 32457 2024.10.22 01:03:03.870646 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 34.64 MiB. 2024.10.22 01:03:04.131904 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:04.135696 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:04.280536 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_547_547_0 (state Temporary) 2024.10.22 01:03:04.432393 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_547_547_0 to 202410_560_560_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:04.482979 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 133 entries to flush up to offset 24217 2024.10.22 01:03:04.591633 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_555_178 to 202410_556_556_0 2024.10.22 01:03:04.597821 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.15 MiB using storage policy from min volume index 0 2024.10.22 01:03:04.598601 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 7.80 MiB, will set to 1.63 GiB (RSS), difference: -66.92 MiB 2024.10.22 01:03:04.599088 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:04.603191 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:04.604448 [ 1185 ] {} DiskLocal: Reserved 1.15 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:04.606171 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32457 2024.10.22 01:03:04.626562 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_556_179} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_555_178 to 202410_556_556_0 into Compact with storage Full 2024.10.22 01:03:04.672328 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_556_179} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:04.676084 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_556_179} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_555_178, total 32885 rows starting from the beginning of the part 2024.10.22 01:03:04.763182 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_318_318_0 (state Temporary) 2024.10.22 01:03:04.784593 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_556_179} MergeTreeSequentialSource: Reading 2 marks from part 202410_556_556_0, total 115 rows starting from the beginning of the part 2024.10.22 01:03:04.823445 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_318_318_0 to 202410_324_324_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:04.856454 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_319_105 to 202410_320_320_0 2024.10.22 01:03:04.864389 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 01:03:04.950481 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1714 entries to flush up to offset 509536 2024.10.22 01:03:04.969231 [ 1176 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:04.981332 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_320_106} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_319_105 to 202410_320_320_0 into Compact with storage Full 2024.10.22 01:03:04.998527 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 24217 2024.10.22 01:03:05.041029 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_320_106} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:05.053164 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_320_106} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_319_105, total 23637 rows starting from the beginning of the part 2024.10.22 01:03:05.190938 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_320_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_320_320_0, total 126 rows starting from the beginning of the part 2024.10.22 01:03:05.500010 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:05.603090 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:05.745387 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_307_307_0 (state Temporary) 2024.10.22 01:03:05.992953 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_307_307_0 to 202410_314_314_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:06.057711 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 509536 2024.10.22 01:03:06.259502 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_556_179} MergeTask::MergeProjectionsStage: Merge sorted 33000 rows, containing 13 columns (13 merged, 0 gathered) in 1.639074672 sec., 20133.310924592213 rows/sec., 3.10 MiB/sec. 2024.10.22 01:03:06.288180 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_556_179} MergedBlockOutputStream: filled checksums 202410_1_556_179 (state Temporary) 2024.10.22 01:03:06.391436 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 7.99 MiB, will set to 1.67 GiB (RSS), difference: -23.34 MiB 2024.10.22 01:03:06.672347 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_320_106} MergeTask::MergeProjectionsStage: Merge sorted 23763 rows, containing 13 columns (13 merged, 0 gathered) in 1.693216453 sec., 14034.23641312798 rows/sec., 5.96 MiB/sec. 2024.10.22 01:03:06.735236 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 237.72 MiB using storage policy from min volume index 0 2024.10.22 01:03:06.826312 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_556_179} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_556_179 to 202410_1_556_179 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:06.832742 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_556_179} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_555_178, 202410_556_556_0] -> 202410_1_556_179 2024.10.22 01:03:06.943359 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 277 2024.10.22 01:03:06.955939 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.85 MiB. 2024.10.22 01:03:07.039383 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_320_106} MergedBlockOutputStream: filled checksums 202410_1_320_106 (state Temporary) 2024.10.22 01:03:07.101973 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:07.107675 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:07.288067 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_259_259_0 (state Temporary) 2024.10.22 01:03:07.382047 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_320_106} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_320_106 to 202410_1_320_106 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:07.388955 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_320_106} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_319_105, 202410_320_320_0] -> 202410_1_320_106 2024.10.22 01:03:07.399204 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yfp/ttuwewfltqvtllcmetnvveelpgxef (LogSeriesLimiter: on interval from 2024-10-22 01:03:00 to 2024-10-22 01:03:07 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:03:07.414833 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.37 MiB. 2024.10.22 01:03:07.563802 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 5.75 MiB, will set to 1.66 GiB (RSS), difference: -10.04 MiB 2024.10.22 01:03:07.644492 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_259_259_0 to 202410_269_269_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:07.708926 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 277 2024.10.22 01:03:08.625047 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 32511 2024.10.22 01:03:08.910683 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_261_70, 202410_262_262_0] 2024.10.22 01:03:08.916549 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_261_70, 202410_262_262_0] 2024.10.22 01:03:08.931713 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_261_70, 202410_262_262_0] 2024.10.22 01:03:08.933436 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:09.035796 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:09.099408 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_548_548_0 (state Temporary) 2024.10.22 01:03:09.273286 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_548_548_0 to 202410_561_561_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:09.364807 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_556_179 to 202410_557_557_0 2024.10.22 01:03:09.373333 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.88 GiB, free memory in arenas 5.77 MiB, will set to 1.65 GiB (RSS), difference: -116.94 MiB 2024.10.22 01:03:09.391185 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32511 2024.10.22 01:03:09.477024 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.15 MiB using storage policy from min volume index 0 2024.10.22 01:03:09.478678 [ 1269 ] {} DiskLocal: Reserved 1.15 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:09.493546 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_557_180} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_556_179 to 202410_557_557_0 into Compact with storage Full 2024.10.22 01:03:09.548479 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 01:03:09.622449 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_557_180} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:09.634087 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_557_180} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_556_179, total 33000 rows starting from the beginning of the part 2024.10.22 01:03:09.802099 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_557_180} MergeTreeSequentialSource: Reading 2 marks from part 202410_557_557_0, total 79 rows starting from the beginning of the part 2024.10.22 01:03:09.846542 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:03:09.848901 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:03:10.204701 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 2609 2024.10.22 01:03:10.898801 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 4692 2024.10.22 01:03:11.303924 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:11.306405 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:11.343417 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_322_322_0 (state Temporary) 2024.10.22 01:03:11.394346 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_322_322_0 to 202410_335_335_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:11.488320 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4692 2024.10.22 01:03:11.533374 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.88 GiB, free memory in arenas 5.88 MiB, will set to 1.66 GiB (RSS), difference: -112.15 MiB 2024.10.22 01:03:12.025531 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/usg/bfhrzuxddlohkdgmsaudowlfmdqzd (LogSeriesLimiter: on interval from 2024-10-22 01:03:07 to 2024-10-22 01:03:12 accepted series 1 / 191 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:03:12.282359 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_557_180} MergeTask::MergeProjectionsStage: Merge sorted 33079 rows, containing 13 columns (13 merged, 0 gathered) in 2.791050855 sec., 11851.80841142359 rows/sec., 1.83 MiB/sec. 2024.10.22 01:03:12.395549 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_456_110, 202410_457_457_0] 2024.10.22 01:03:12.399331 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_456_110, 202410_457_457_0] 2024.10.22 01:03:12.417104 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_456_110, 202410_457_457_0] 2024.10.22 01:03:12.439043 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_557_180} MergedBlockOutputStream: filled checksums 202410_1_557_180 (state Temporary) 2024.10.22 01:03:12.523077 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 124 entries to flush up to offset 24341 2024.10.22 01:03:12.674743 [ 1199 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 205 ms. 2024.10.22 01:03:12.725101 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_557_180} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_557_180 to 202410_1_557_180 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:12.729555 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_557_180} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_556_179, 202410_557_557_0] -> 202410_1_557_180 2024.10.22 01:03:12.764452 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:03:12.856450 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:12.858682 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:12.892020 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_319_319_0 (state Temporary) 2024.10.22 01:03:13.052658 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_319_319_0 to 202410_325_325_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:13.065459 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1742 entries to flush up to offset 511278 2024.10.22 01:03:13.081928 [ 1213 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_320_106 to 202410_321_321_0 2024.10.22 01:03:13.088308 [ 1213 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 01:03:13.088363 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 24341 2024.10.22 01:03:13.106264 [ 1213 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:13.147063 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_321_107} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_320_106 to 202410_321_321_0 into Compact with storage Full 2024.10.22 01:03:13.192796 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_321_107} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:13.200853 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_321_107} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_320_106, total 23763 rows starting from the beginning of the part 2024.10.22 01:03:13.270751 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_321_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_321_321_0, total 122 rows starting from the beginning of the part 2024.10.22 01:03:13.334892 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.85 GiB, peak 1.88 GiB, free memory in arenas 4.75 MiB, will set to 1.66 GiB (RSS), difference: -201.90 MiB 2024.10.22 01:03:13.397037 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 32564 2024.10.22 01:03:13.449459 [ 1222 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:03:13.491970 [ 1222 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:03:13.522024 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:13.527949 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:13.584565 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_549_549_0 (state Temporary) 2024.10.22 01:03:13.618920 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_549_549_0 to 202410_562_562_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:13.646485 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_557_180 to 202410_558_558_0 2024.10.22 01:03:13.651088 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.16 MiB using storage policy from min volume index 0 2024.10.22 01:03:13.675803 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32564 2024.10.22 01:03:13.735037 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:13.743615 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:13.779707 [ 1277 ] {} DiskLocal: Reserved 1.16 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:13.795585 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_558_181} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_557_180 to 202410_558_558_0 into Compact with storage Full 2024.10.22 01:03:13.812671 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_558_181} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:13.826594 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_558_181} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_557_180, total 33079 rows starting from the beginning of the part 2024.10.22 01:03:13.849215 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_558_181} MergeTreeSequentialSource: Reading 2 marks from part 202410_558_558_0, total 76 rows starting from the beginning of the part 2024.10.22 01:03:13.907596 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_308_308_0 (state Temporary) 2024.10.22 01:03:14.055050 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(8d5c74f3a2e53007b24b0cfecd950c39) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8d5c74f3a2e53007b24b0cfecd950c39, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:03:14.094507 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 949, file segment info: File segment: [0, 33554431], key: 8d5c74f3a2e53007b24b0cfecd950c39, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:03:14.099106 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:14.100855 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_308_308_0 to 202410_315_315_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:14.164213 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 511278 2024.10.22 01:03:14.308089 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:03:14.324453 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:03:14.357671 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:03:14.371131 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:03:14.387730 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:03:14.406748 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:03:14.461549 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:03:14.542323 [ 1685 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/azg/hkskhgwkpmevwrmpgqujcndzipffi, size 4309865 2024.10.22 01:03:14.708117 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_321_107/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:03:14.777194 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_321_107} MergeTask::MergeProjectionsStage: Merge sorted 23885 rows, containing 13 columns (13 merged, 0 gathered) in 1.634447365 sec., 14613.50209953075 rows/sec., 6.21 MiB/sec. 2024.10.22 01:03:14.964350 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_321_107} MergedBlockOutputStream: filled checksums 202410_1_321_107 (state Temporary) 2024.10.22 01:03:15.210673 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 278 2024.10.22 01:03:15.229135 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_321_107} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_321_107 to 202410_1_321_107 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:15.260870 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_321_107} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_320_106, 202410_321_321_0] -> 202410_1_321_107 2024.10.22 01:03:15.303624 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_251_50, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0] 2024.10.22 01:03:15.305746 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_251_50, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0] 2024.10.22 01:03:15.306639 [ 1385 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_321_107}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 01:03:15.343336 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.37 MiB. 2024.10.22 01:03:15.451564 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_251_50, 202410_252_252_0, 202410_253_253_0, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0] 2024.10.22 01:03:15.555411 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:15.559903 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:15.605112 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.88 GiB, free memory in arenas 2.98 MiB, will set to 1.67 GiB (RSS), difference: -105.07 MiB 2024.10.22 01:03:15.824331 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_260_260_0 (state Temporary) 2024.10.22 01:03:15.983827 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_558_181} MergeTask::MergeProjectionsStage: Merge sorted 33155 rows, containing 13 columns (13 merged, 0 gathered) in 2.193676387 sec., 15113.89747206136 rows/sec., 2.33 MiB/sec. 2024.10.22 01:03:16.008713 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_558_181} MergedBlockOutputStream: filled checksums 202410_1_558_181 (state Temporary) 2024.10.22 01:03:16.017135 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_260_260_0 to 202410_270_270_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:16.124402 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_558_181} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_558_181 to 202410_1_558_181 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:16.131449 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_558_181} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_557_180, 202410_558_558_0] -> 202410_1_558_181 2024.10.22 01:03:16.148655 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:03:16.197550 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 278 2024.10.22 01:03:16.316916 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_271_54, 202410_272_272_0, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0] 2024.10.22 01:03:16.322795 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_271_54, 202410_272_272_0, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0] 2024.10.22 01:03:16.332938 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_262_71, 202410_263_263_0] 2024.10.22 01:03:16.340257 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_262_71, 202410_263_263_0] 2024.10.22 01:03:16.354948 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_262_71, 202410_263_263_0] 2024.10.22 01:03:16.365467 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_271_54, 202410_272_272_0, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0] 2024.10.22 01:03:17.286543 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_457_111, 202410_458_458_0] 2024.10.22 01:03:17.288183 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_457_111, 202410_458_458_0] 2024.10.22 01:03:17.296768 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_457_111, 202410_458_458_0] 2024.10.22 01:03:17.682842 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 32626 2024.10.22 01:03:17.712056 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 3.28 MiB, will set to 1.67 GiB (RSS), difference: 5.35 MiB 2024.10.22 01:03:17.804845 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:17.809434 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:17.849202 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_550_550_0 (state Temporary) 2024.10.22 01:03:17.886601 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_550_550_0 to 202410_563_563_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:17.906360 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_558_181 to 202410_559_559_0 2024.10.22 01:03:17.909469 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.16 MiB using storage policy from min volume index 0 2024.10.22 01:03:17.914848 [ 1255 ] {} DiskLocal: Reserved 1.16 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:17.939614 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_559_182} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_558_181 to 202410_559_559_0 into Compact with storage Full 2024.10.22 01:03:17.998291 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32626 2024.10.22 01:03:18.001525 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_559_182} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:18.011465 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_559_182} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_558_181, total 33155 rows starting from the beginning of the part 2024.10.22 01:03:18.160789 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_559_182} MergeTreeSequentialSource: Reading 2 marks from part 202410_559_559_0, total 69 rows starting from the beginning of the part 2024.10.22 01:03:18.279511 [ 1190 ] {} DatabaseCatalog: Execution took 359 ms. 2024.10.22 01:03:18.693150 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(010dd14a0dcb66d270cd1fd19b8085ee) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 010dd14a0dcb66d270cd1fd19b8085ee, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:03:18.808380 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 010dd14a0dcb66d270cd1fd19b8085ee, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:03:18.811973 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:18.989686 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 4707 2024.10.22 01:03:19.093390 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:19.095917 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:19.119835 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_323_323_0 (state Temporary) 2024.10.22 01:03:19.177949 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_323_323_0 to 202410_336_336_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:19.203524 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_331_66 to 202410_336_336_0 2024.10.22 01:03:19.222159 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:19.227151 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:19.228419 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4707 2024.10.22 01:03:19.259333 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_331_66 to 202410_336_336_0 into Compact with storage Full 2024.10.22 01:03:19.295057 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d25d8f83156d8e6492d6d906d130570c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d25d8f83156d8e6492d6d906d130570c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:03:19.307293 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 500, file segment info: File segment: [0, 33554431], key: d25d8f83156d8e6492d6d906d130570c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:03:19.308955 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:19.330357 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:19.346547 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_331_66, total 4715 rows starting from the beginning of the part 2024.10.22 01:03:19.374562 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 2.56 MiB, will set to 1.68 GiB (RSS), difference: -1.01 MiB 2024.10.22 01:03:19.380686 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_332_332_0, total 19 rows starting from the beginning of the part 2024.10.22 01:03:19.414491 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_333_333_0, total 15 rows starting from the beginning of the part 2024.10.22 01:03:19.472414 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_334_334_0, total 30 rows starting from the beginning of the part 2024.10.22 01:03:19.493837 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_335_335_0, total 19 rows starting from the beginning of the part 2024.10.22 01:03:19.505010 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_336_336_0, total 15 rows starting from the beginning of the part 2024.10.22 01:03:19.846706 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:03:19.859654 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:03:19.991527 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:03:20.007667 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_559_182} MergeTask::MergeProjectionsStage: Merge sorted 33224 rows, containing 13 columns (13 merged, 0 gathered) in 2.072968377 sec., 16027.25848046065 rows/sec., 2.47 MiB/sec. 2024.10.22 01:03:20.023861 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ccced49eb1b2e1798b8643c4ec57aa54) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ccced49eb1b2e1798b8643c4ec57aa54, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:03:20.134730 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 863, file segment info: File segment: [0, 33554431], key: ccced49eb1b2e1798b8643c4ec57aa54, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:03:20.137470 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:20.175453 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_559_182} MergedBlockOutputStream: filled checksums 202410_1_559_182 (state Temporary) 2024.10.22 01:03:20.226863 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_559_182} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_559_182 to 202410_1_559_182 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:20.230954 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_559_182} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_558_181, 202410_559_559_0] -> 202410_1_559_182 2024.10.22 01:03:20.250334 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:03:20.527851 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} MergeTask::MergeProjectionsStage: Merge sorted 4813 rows, containing 12 columns (12 merged, 0 gathered) in 1.273287103 sec., 3779.9801699554323 rows/sec., 594.34 KiB/sec. 2024.10.22 01:03:20.587561 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} MergedBlockOutputStream: filled checksums 202410_1_336_67 (state Temporary) 2024.10.22 01:03:20.598844 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 139 entries to flush up to offset 24480 2024.10.22 01:03:20.733258 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b530611f0b23313f64ede9cc2e532194) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b530611f0b23313f64ede9cc2e532194, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:03:20.737751 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 102989, file segment info: File segment: [0, 33554431], key: b530611f0b23313f64ede9cc2e532194, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:03:20.742853 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:20.763934 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_336_67 to 202410_1_336_67 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:20.785277 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_336_67} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_331_66, 202410_336_336_0] -> 202410_1_336_67 2024.10.22 01:03:20.826287 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.40 MiB. 2024.10.22 01:03:21.025414 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:21.032408 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:21.094471 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_320_320_0 (state Temporary) 2024.10.22 01:03:21.171706 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 512814 2024.10.22 01:03:21.302057 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_320_320_0 to 202410_326_326_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:21.342636 [ 1179 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_321_107 to 202410_322_322_0 2024.10.22 01:03:21.350207 [ 1179 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.32 MiB using storage policy from min volume index 0 2024.10.22 01:03:21.351936 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 24480 2024.10.22 01:03:21.357682 [ 1179 ] {} DiskLocal: Reserved 1.32 MiB on local disk `default`, having unreserved 297.63 GiB. 2024.10.22 01:03:21.503915 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_322_108} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_321_107 to 202410_322_322_0 into Wide with storage Full 2024.10.22 01:03:21.541068 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_322_108} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:21.551324 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_322_108} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_321_107, total 23885 rows starting from the beginning of the part 2024.10.22 01:03:21.572637 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_322_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_322_322_0, total 158 rows starting from the beginning of the part 2024.10.22 01:03:21.644422 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 3.23 MiB, will set to 1.68 GiB (RSS), difference: 7.25 MiB 2024.10.22 01:03:21.808549 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(5594f1a3deaaf7d955a8a63015de2dae) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5594f1a3deaaf7d955a8a63015de2dae, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:03:21.824778 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 879, file segment info: File segment: [0, 33554431], key: 5594f1a3deaaf7d955a8a63015de2dae, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:03:21.829283 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:21.982955 [ 1723 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pqf/vyfqqiicweffbarajxrsvwavaodxc, size 102989 2024.10.22 01:03:22.017345 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:22.042950 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 32703 2024.10.22 01:03:22.044560 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:22.100018 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_458_112, 202410_459_459_0] 2024.10.22 01:03:22.102686 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_458_112, 202410_459_459_0] 2024.10.22 01:03:22.119459 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_458_112, 202410_459_459_0] 2024.10.22 01:03:22.203972 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_309_309_0 (state Temporary) 2024.10.22 01:03:22.291894 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_309_309_0 to 202410_316_316_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:22.330299 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 512814 2024.10.22 01:03:22.335503 [ 1169 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_311_62 to 202410_316_316_0 2024.10.22 01:03:22.353566 [ 1169 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:22.359861 [ 1169 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:22.396045 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_311_62 to 202410_316_316_0 into Compact with storage Full 2024.10.22 01:03:22.397615 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:03:22.410087 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:22.413467 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:03:22.424521 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} MergeTreeSequentialSource: Reading 65 marks from part 202410_1_311_62, total 518724 rows starting from the beginning of the part 2024.10.22 01:03:22.447075 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_312_312_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:03:22.461289 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:22.462852 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_313_313_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:03:22.468324 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:22.475034 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:03:22.492306 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:03:22.525609 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:03:22.559887 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:03:22.568501 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_551_551_0 (state Temporary) 2024.10.22 01:03:22.579122 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_314_314_0, total 1714 rows starting from the beginning of the part 2024.10.22 01:03:22.585479 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:03:22.605015 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:03:22.606485 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_315_315_0, total 1742 rows starting from the beginning of the part 2024.10.22 01:03:22.616280 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:03:22.636809 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_316_316_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:03:22.686406 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_551_551_0 to 202410_564_564_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:22.693762 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qvk/axjxmtphfcibbqolldznjigcmcxdg (LogSeriesLimiter: on interval from 2024-10-22 01:03:12 to 2024-10-22 01:03:22 accepted series 1 / 164 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:03:22.748844 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32703 2024.10.22 01:03:22.737828 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_559_182 to 202410_560_560_0 2024.10.22 01:03:22.845043 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.17 MiB using storage policy from min volume index 0 2024.10.22 01:03:22.847587 [ 1183 ] {} DiskLocal: Reserved 1.17 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:22.980078 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_560_183} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_559_182 to 202410_560_560_0 into Compact with storage Full 2024.10.22 01:03:23.029458 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_560_183} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:23.059570 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_560_183} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_559_182, total 33224 rows starting from the beginning of the part 2024.10.22 01:03:23.079769 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_322_108} MergeTask::MergeProjectionsStage: Merge sorted 24043 rows, containing 13 columns (13 merged, 0 gathered) in 1.579617294 sec., 15220.775368391225 rows/sec., 6.47 MiB/sec. 2024.10.22 01:03:23.105617 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_560_183} MergeTreeSequentialSource: Reading 2 marks from part 202410_560_560_0, total 64 rows starting from the beginning of the part 2024.10.22 01:03:23.341432 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_322_108} MergedBlockOutputStream: filled checksums 202410_1_322_108 (state Temporary) 2024.10.22 01:03:23.415845 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.88 GiB, free memory in arenas 3.89 MiB, will set to 1.69 GiB (RSS), difference: -45.10 MiB 2024.10.22 01:03:23.700336 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 279 2024.10.22 01:03:23.954033 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_322_108} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_322_108 to 202410_1_322_108 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:23.959152 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_322_108} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_321_107, 202410_322_322_0] -> 202410_1_322_108 2024.10.22 01:03:23.997133 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.08 MiB. 2024.10.22 01:03:24.118384 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:24.123734 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:24.295051 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_261_261_0 (state Temporary) 2024.10.22 01:03:24.588370 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_261_261_0 to 202410_271_271_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:24.696680 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 279 2024.10.22 01:03:25.254624 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_560_183} MergeTask::MergeProjectionsStage: Merge sorted 33288 rows, containing 13 columns (13 merged, 0 gathered) in 2.385148547 sec., 13956.363448251092 rows/sec., 2.15 MiB/sec. 2024.10.22 01:03:25.287238 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_560_183/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:03:25.310485 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_560_183} MergedBlockOutputStream: filled checksums 202410_1_560_183 (state Temporary) 2024.10.22 01:03:25.379068 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_560_183} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_560_183 to 202410_1_560_183 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:25.383640 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_560_183} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_559_182, 202410_560_560_0] -> 202410_1_560_183 2024.10.22 01:03:25.425133 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:03:25.507628 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 7.34 MiB, will set to 1.70 GiB (RSS), difference: 54.98 MiB 2024.10.22 01:03:26.182902 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:26.186316 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:26.732338 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 4728 2024.10.22 01:03:26.773313 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 32767 2024.10.22 01:03:26.852169 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_459_113, 202410_460_460_0, 202410_461_461_0, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0] 2024.10.22 01:03:26.867753 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_459_113, 202410_460_460_0, 202410_461_461_0, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0] 2024.10.22 01:03:26.875187 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:26.883312 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:26.906110 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_459_113, 202410_460_460_0, 202410_461_461_0, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0] 2024.10.22 01:03:27.052826 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_324_324_0 (state Temporary) 2024.10.22 01:03:27.127373 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:27.139398 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_324_324_0 to 202410_337_337_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:27.233215 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:27.300391 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4728 2024.10.22 01:03:27.404768 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_552_552_0 (state Temporary) 2024.10.22 01:03:27.598403 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_552_552_0 to 202410_565_565_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:27.643867 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_560_183 to 202410_561_561_0 2024.10.22 01:03:27.664164 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.17 MiB using storage policy from min volume index 0 2024.10.22 01:03:27.665694 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32767 2024.10.22 01:03:27.673551 [ 1271 ] {} DiskLocal: Reserved 1.17 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:27.695626 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_561_184} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_560_183 to 202410_561_561_0 into Compact with storage Full 2024.10.22 01:03:27.712534 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_263_72, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0, 202410_267_267_0, 202410_268_268_0] 2024.10.22 01:03:27.712594 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_561_184} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:27.716781 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_263_72, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0, 202410_267_267_0, 202410_268_268_0] 2024.10.22 01:03:27.726844 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_561_184} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_560_183, total 33288 rows starting from the beginning of the part 2024.10.22 01:03:27.863134 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_561_184} MergeTreeSequentialSource: Reading 2 marks from part 202410_561_561_0, total 54 rows starting from the beginning of the part 2024.10.22 01:03:27.864203 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_263_72, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0, 202410_267_267_0, 202410_268_268_0] 2024.10.22 01:03:28.031736 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_316_63/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:03:28.156417 [ 1211 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 01:03:28.495649 [ 1272 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:03:28.525692 [ 1270 ] {} DatabaseCatalog: Execution took 248 ms. 2024.10.22 01:03:28.610704 [ 1272 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:03:28.889375 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 202 entries to flush up to offset 24682 2024.10.22 01:03:29.338402 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 514350 2024.10.22 01:03:29.390326 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:29.395467 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:29.517052 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_321_321_0 (state Temporary) 2024.10.22 01:03:29.693114 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 13.16 MiB, will set to 1.72 GiB (RSS), difference: 78.24 MiB 2024.10.22 01:03:29.702824 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_321_321_0 to 202410_327_327_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:29.743440 [ 1196 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_322_108 to 202410_323_323_0 2024.10.22 01:03:29.759116 [ 1196 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 01:03:29.761017 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 24682 2024.10.22 01:03:29.763473 [ 1196 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 297.62 GiB. 2024.10.22 01:03:29.789246 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_323_109} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_322_108 to 202410_323_323_0 into Wide with storage Full 2024.10.22 01:03:29.831004 [ 1721 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/usg/bfhrzuxddlohkdgmsaudowlfmdqzd, size 183865 2024.10.22 01:03:29.848175 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:03:29.867676 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_323_109} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:29.876068 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_323_109} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_322_108, total 24043 rows starting from the beginning of the part 2024.10.22 01:03:29.876706 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:03:29.962778 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_65_65_0 (state Temporary) 2024.10.22 01:03:30.131586 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_323_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_323_323_0, total 145 rows starting from the beginning of the part 2024.10.22 01:03:30.299535 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:30.303572 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.61 GiB. 2024.10.22 01:03:30.412844 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_561_184} MergeTask::MergeProjectionsStage: Merge sorted 33342 rows, containing 13 columns (13 merged, 0 gathered) in 2.703589045 sec., 12332.495599382042 rows/sec., 1.90 MiB/sec. 2024.10.22 01:03:30.499227 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_561_184} MergedBlockOutputStream: filled checksums 202410_1_561_184 (state Temporary) 2024.10.22 01:03:30.532233 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_310_310_0 (state Temporary) 2024.10.22 01:03:30.630917 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(b8d4643403893663813ae51bf758cb93) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b8d4643403893663813ae51bf758cb93, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:03:30.634669 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 952, file segment info: File segment: [0, 33554431], key: b8d4643403893663813ae51bf758cb93, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:03:30.642686 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:30.670550 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_310_310_0 to 202410_317_317_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:30.858873 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 514350 2024.10.22 01:03:31.260769 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_561_184} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_561_184 to 202410_1_561_184 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:31.272258 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_561_184} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_560_183, 202410_561_561_0] -> 202410_1_561_184 2024.10.22 01:03:31.289436 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:03:31.392538 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:03:31.522482 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:03:31.555005 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:03:31.580959 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:03:31.598594 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:03:31.618262 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:03:31.680362 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 32834 2024.10.22 01:03:31.698502 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:03:31.764561 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yny/vuryrykucvnkbtcgpifhusigwjlbq (LogSeriesLimiter: on interval from 2024-10-22 01:03:22 to 2024-10-22 01:03:31 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:03:31.897611 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:31.900195 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.60 GiB. 2024.10.22 01:03:31.952233 [ 1212 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 01:03:32.009971 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_553_553_0 (state Temporary) 2024.10.22 01:03:32.068493 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_553_553_0 to 202410_566_566_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:32.129958 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_561_184 to 202410_562_562_0 2024.10.22 01:03:32.149586 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.17 MiB using storage policy from min volume index 0 2024.10.22 01:03:32.167477 [ 1171 ] {} DiskLocal: Reserved 1.17 MiB on local disk `default`, having unreserved 297.60 GiB. 2024.10.22 01:03:32.188599 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32834 2024.10.22 01:03:32.198488 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 280 2024.10.22 01:03:32.214643 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_562_185} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_561_184 to 202410_562_562_0 into Compact with storage Full 2024.10.22 01:03:32.271812 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_323_109} MergeTask::MergeProjectionsStage: Merge sorted 24188 rows, containing 13 columns (13 merged, 0 gathered) in 2.492545386 sec., 9704.136235936929 rows/sec., 4.12 MiB/sec. 2024.10.22 01:03:32.342074 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_562_185} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:32.382924 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_562_185} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_561_184, total 33342 rows starting from the beginning of the part 2024.10.22 01:03:32.434104 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_323_109} MergedBlockOutputStream: filled checksums 202410_1_323_109 (state Temporary) 2024.10.22 01:03:32.550488 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_562_185} MergeTreeSequentialSource: Reading 2 marks from part 202410_562_562_0, total 53 rows starting from the beginning of the part 2024.10.22 01:03:32.576498 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:32.581716 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.59 GiB. 2024.10.22 01:03:32.684875 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_262_262_0 (state Temporary) 2024.10.22 01:03:32.746501 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.88 GiB, free memory in arenas 10.93 MiB, will set to 1.70 GiB (RSS), difference: 154.12 MiB 2024.10.22 01:03:32.830342 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_262_262_0 to 202410_272_272_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:33.180666 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 280 2024.10.22 01:03:33.427684 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_323_109} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_323_109 to 202410_1_323_109 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:33.438201 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_323_109} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_322_108, 202410_323_323_0] -> 202410_1_323_109 2024.10.22 01:03:33.483576 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.47 MiB. 2024.10.22 01:03:33.554903 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_20_20_0 (state Temporary) 2024.10.22 01:03:33.586646 [ 1682 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yfp/ttuwewfltqvtllcmetnvveelpgxef, size 344 2024.10.22 01:03:34.307590 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.88 GiB, free memory in arenas 11.57 MiB, will set to 1.69 GiB (RSS), difference: 90.55 MiB 2024.10.22 01:03:34.787151 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:03:34.803408 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 4740 2024.10.22 01:03:35.225745 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:35.235931 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.58 GiB. 2024.10.22 01:03:35.293535 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_325_325_0 (state Temporary) 2024.10.22 01:03:35.474355 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_325_325_0 to 202410_338_338_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:35.488301 [ 1256 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 216 ms. 2024.10.22 01:03:35.526934 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4740 2024.10.22 01:03:35.762726 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_562_185} MergeTask::MergeProjectionsStage: Merge sorted 33395 rows, containing 13 columns (13 merged, 0 gathered) in 3.560101623 sec., 9380.350208053598 rows/sec., 1.45 MiB/sec. 2024.10.22 01:03:35.820992 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_562_185} MergedBlockOutputStream: filled checksums 202410_1_562_185 (state Temporary) 2024.10.22 01:03:35.941928 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_562_185} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_562_185 to 202410_1_562_185 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:35.947243 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_562_185} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_561_184, 202410_562_562_0] -> 202410_1_562_185 2024.10.22 01:03:35.988499 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:03:36.210225 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 32878 2024.10.22 01:03:36.918248 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 12.04 MiB, will set to 1.68 GiB (RSS), difference: 4.63 MiB 2024.10.22 01:03:36.994395 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:36.997516 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.58 GiB. 2024.10.22 01:03:37.050467 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_554_554_0 (state Temporary) 2024.10.22 01:03:37.188052 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_554_554_0 to 202410_567_567_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:37.212135 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_562_185 to 202410_563_563_0 2024.10.22 01:03:37.224768 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.17 MiB using storage policy from min volume index 0 2024.10.22 01:03:37.227011 [ 1255 ] {} DiskLocal: Reserved 1.17 MiB on local disk `default`, having unreserved 297.58 GiB. 2024.10.22 01:03:37.244325 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32878 2024.10.22 01:03:37.248268 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_563_186} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_562_185 to 202410_563_563_0 into Compact with storage Full 2024.10.22 01:03:37.303445 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_563_186} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:37.316905 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_563_186} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_562_185, total 33395 rows starting from the beginning of the part 2024.10.22 01:03:37.357322 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_563_186} MergeTreeSequentialSource: Reading 2 marks from part 202410_563_563_0, total 62 rows starting from the beginning of the part 2024.10.22 01:03:37.400436 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 202 entries to flush up to offset 24884 2024.10.22 01:03:37.857796 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:37.867021 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.58 GiB. 2024.10.22 01:03:37.906884 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1528 entries to flush up to offset 515878 2024.10.22 01:03:37.927657 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_322_322_0 (state Temporary) 2024.10.22 01:03:38.092474 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_322_322_0 to 202410_328_328_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:38.133142 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(8ef9bfdedae7b690c581547bd4af761a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8ef9bfdedae7b690c581547bd4af761a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:03:38.138900 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_323_109 to 202410_328_328_0 2024.10.22 01:03:38.141526 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 8ef9bfdedae7b690c581547bd4af761a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:03:38.147408 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:38.164190 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 01:03:38.170930 [ 1268 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 297.58 GiB. 2024.10.22 01:03:38.172661 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 24884 2024.10.22 01:03:38.295863 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_323_109 to 202410_328_328_0 into Wide with storage Full 2024.10.22 01:03:38.325281 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:38.351136 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_323_109, total 24188 rows starting from the beginning of the part 2024.10.22 01:03:38.373315 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 13.09 MiB, will set to 1.69 GiB (RSS), difference: -4.72 MiB 2024.10.22 01:03:38.380434 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_65_65_0 to 202410_68_68_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:38.490393 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_324_324_0, total 133 rows starting from the beginning of the part 2024.10.22 01:03:38.504451 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_325_325_0, total 124 rows starting from the beginning of the part 2024.10.22 01:03:38.523756 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_326_326_0, total 139 rows starting from the beginning of the part 2024.10.22 01:03:38.777706 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_327_327_0, total 202 rows starting from the beginning of the part 2024.10.22 01:03:38.802209 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_328_328_0, total 202 rows starting from the beginning of the part 2024.10.22 01:03:39.007588 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:39.022460 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.58 GiB. 2024.10.22 01:03:39.251646 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_311_311_0 (state Temporary) 2024.10.22 01:03:39.338625 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_311_311_0 to 202410_318_318_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:39.391769 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 515878 2024.10.22 01:03:39.429287 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_316_63/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:03:39.531391 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 12.76 MiB, will set to 1.69 GiB (RSS), difference: -38.44 MiB 2024.10.22 01:03:39.856110 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:03:39.878744 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:03:39.873088 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_563_186} MergeTask::MergeProjectionsStage: Merge sorted 33457 rows, containing 13 columns (13 merged, 0 gathered) in 2.635711077 sec., 12693.728190451431 rows/sec., 1.96 MiB/sec. 2024.10.22 01:03:40.032249 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_563_186} MergedBlockOutputStream: filled checksums 202410_1_563_186 (state Temporary) 2024.10.22 01:03:40.084553 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} MergeTask::MergeProjectionsStage: Merge sorted 526404 rows, containing 4 columns (4 merged, 0 gathered) in 17.702087306 sec., 29736.832210830813 rows/sec., 436.02 KiB/sec. 2024.10.22 01:03:40.202625 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_563_186} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_563_186 to 202410_1_563_186 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:40.217207 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_563_186} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_562_185, 202410_563_563_0] -> 202410_1_563_186 2024.10.22 01:03:40.243446 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} MergedBlockOutputStream: filled checksums 202410_1_316_63 (state Temporary) 2024.10.22 01:03:40.382315 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.24 MiB. 2024.10.22 01:03:40.389359 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_316_63 to 202410_1_316_63 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:40.392959 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_316_63} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_311_62, 202410_316_316_0] -> 202410_1_316_63 2024.10.22 01:03:40.429747 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:03:40.684966 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 281 2024.10.22 01:03:41.054073 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:41.059333 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.58 GiB. 2024.10.22 01:03:41.064653 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2609 2024.10.22 01:03:41.132547 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_263_263_0 (state Temporary) 2024.10.22 01:03:41.193577 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_263_263_0 to 202410_273_273_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:41.230728 [ 1179 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_268_55 to 202410_273_273_0 2024.10.22 01:03:41.245829 [ 1179 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:41.252682 [ 1179 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.58 GiB. 2024.10.22 01:03:41.254726 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 32942 2024.10.22 01:03:41.282920 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 281 2024.10.22 01:03:41.288325 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_268_55 to 202410_273_273_0 into Compact with storage Full 2024.10.22 01:03:41.403815 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:41.409314 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.58 GiB. 2024.10.22 01:03:41.437134 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_219_45, 202410_220_220_0, 202410_221_221_0, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0] 2024.10.22 01:03:41.443256 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_219_45, 202410_220_220_0, 202410_221_221_0, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0] 2024.10.22 01:03:41.483053 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_555_555_0 (state Temporary) 2024.10.22 01:03:41.496670 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:41.515807 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_268_55, total 312 rows starting from the beginning of the part 2024.10.22 01:03:41.531211 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 15.81 MiB, will set to 1.68 GiB (RSS), difference: 24.56 MiB 2024.10.22 01:03:41.585159 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_219_45, 202410_220_220_0, 202410_221_221_0, 202410_222_222_0, 202410_223_223_0, 202410_224_224_0] 2024.10.22 01:03:41.621438 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_328_110/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:03:41.637642 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_269_269_0, total 1 rows starting from the beginning of the part 2024.10.22 01:03:41.745660 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_555_555_0 to 202410_568_568_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:41.771350 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_563_186 to 202410_564_564_0 2024.10.22 01:03:41.781057 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.18 MiB using storage policy from min volume index 0 2024.10.22 01:03:41.764966 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} MergeTask::MergeProjectionsStage: Merge sorted 24988 rows, containing 13 columns (13 merged, 0 gathered) in 3.573514213 sec., 6992.556489378652 rows/sec., 2.97 MiB/sec. 2024.10.22 01:03:41.791410 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_270_270_0, total 1 rows starting from the beginning of the part 2024.10.22 01:03:41.784092 [ 1241 ] {} DiskLocal: Reserved 1.18 MiB on local disk `default`, having unreserved 297.58 GiB. 2024.10.22 01:03:41.832549 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_564_187} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_563_186 to 202410_564_564_0 into Compact with storage Full 2024.10.22 01:03:41.909333 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} MergedBlockOutputStream: filled checksums 202410_1_328_110 (state Temporary) 2024.10.22 01:03:41.935414 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 32942 2024.10.22 01:03:41.957047 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_271_271_0, total 1 rows starting from the beginning of the part 2024.10.22 01:03:42.008220 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_272_272_0, total 1 rows starting from the beginning of the part 2024.10.22 01:03:42.084066 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_564_187} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:42.095571 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_564_187} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_563_186, total 33457 rows starting from the beginning of the part 2024.10.22 01:03:42.124459 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_564_187} MergeTreeSequentialSource: Reading 2 marks from part 202410_564_564_0, total 77 rows starting from the beginning of the part 2024.10.22 01:03:42.168597 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_273_273_0, total 1 rows starting from the beginning of the part 2024.10.22 01:03:42.723358 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_328_110 to 202410_1_328_110 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:42.729218 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_328_110} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_323_109, 202410_328_328_0] -> 202410_1_328_110 2024.10.22 01:03:42.783140 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 56.04 MiB. 2024.10.22 01:03:43.031958 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 4767 2024.10.22 01:03:43.151792 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:43.158545 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:43.199330 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_326_326_0 (state Temporary) 2024.10.22 01:03:43.361153 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_273_56/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:03:43.460625 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_326_326_0 to 202410_339_339_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:43.499537 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 14.32 MiB, will set to 1.68 GiB (RSS), difference: 22.65 MiB 2024.10.22 01:03:43.503116 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4767 2024.10.22 01:03:43.615043 [ 1191 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:03:43.843872 [ 1191 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:03:43.847188 [ 1191 ] {} DNSCacheUpdater: Execution took 232 ms. 2024.10.22 01:03:44.177644 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(8ce6bb802f2a9a49bca2f164ca348145) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8ce6bb802f2a9a49bca2f164ca348145, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:03:44.181173 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 696, file segment info: File segment: [0, 33554431], key: 8ce6bb802f2a9a49bca2f164ca348145, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:03:44.183558 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:44.194677 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} MergeTask::MergeProjectionsStage: Merge sorted 317 rows, containing 32 columns (32 merged, 0 gathered) in 2.913394331 sec., 108.80779049610912 rows/sec., 11.18 KiB/sec. 2024.10.22 01:03:44.238658 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} MergedBlockOutputStream: filled checksums 202410_1_273_56 (state Temporary) 2024.10.22 01:03:44.547774 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_564_187} MergeTask::MergeProjectionsStage: Merge sorted 33534 rows, containing 13 columns (13 merged, 0 gathered) in 2.729277212 sec., 12286.769498004367 rows/sec., 1.90 MiB/sec. 2024.10.22 01:03:44.587275 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_564_187} MergedBlockOutputStream: filled checksums 202410_1_564_187 (state Temporary) 2024.10.22 01:03:44.626370 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_564_187} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_564_187 to 202410_1_564_187 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:44.631262 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_564_187} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_563_186, 202410_564_564_0] -> 202410_1_564_187 2024.10.22 01:03:44.648212 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:03:44.914603 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ed71bb3d1639813d7b8d6c4390183154) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ed71bb3d1639813d7b8d6c4390183154, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:03:44.916545 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 758, file segment info: File segment: [0, 33554431], key: ed71bb3d1639813d7b8d6c4390183154, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:03:44.923242 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:45.305632 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_273_56 to 202410_1_273_56 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:45.310245 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_273_56} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_268_55, 202410_273_273_0] -> 202410_1_273_56 2024.10.22 01:03:45.380259 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:03:45.385243 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 11.16 MiB, will set to 1.68 GiB (RSS), difference: 13.74 MiB 2024.10.22 01:03:45.599468 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(8c63c007df184e84b6da0dc0ace4a078) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8c63c007df184e84b6da0dc0ace4a078, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:03:45.601711 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 41019, file segment info: File segment: [0, 33554431], key: 8c63c007df184e84b6da0dc0ace4a078, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:03:45.604283 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:45.697580 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 201 entries to flush up to offset 25085 2024.10.22 01:03:45.952767 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 33007 2024.10.22 01:03:46.096153 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:46.097792 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:46.285308 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(21839aabc968578f6ecd46ea20129a16) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 21839aabc968578f6ecd46ea20129a16, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:03:46.286826 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:46.287497 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 780, file segment info: File segment: [0, 33554431], key: 21839aabc968578f6ecd46ea20129a16, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:03:46.288673 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:46.289605 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:46.311031 [ 1747 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qvk/axjxmtphfcibbqolldznjigcmcxdg, size 41019 2024.10.22 01:03:46.372060 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_556_556_0 (state Temporary) 2024.10.22 01:03:46.373649 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_323_323_0 (state Temporary) 2024.10.22 01:03:46.395540 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1928 entries to flush up to offset 517806 2024.10.22 01:03:46.419688 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/usw/krbsjxieopnvczcmybiaibkuabzgp. 2024.10.22 01:03:46.485026 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_323_323_0 to 202410_329_329_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:46.583122 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_556_556_0 to 202410_569_569_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:46.622489 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 25085 2024.10.22 01:03:46.622805 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_564_187 to 202410_565_565_0 2024.10.22 01:03:46.637556 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.18 MiB using storage policy from min volume index 0 2024.10.22 01:03:46.645128 [ 1150 ] {} DiskLocal: Reserved 1.18 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:46.688740 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:03:46.730372 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33007 2024.10.22 01:03:46.742962 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:03:46.772851 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:03:46.673496 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_565_188} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_564_187 to 202410_565_565_0 into Compact with storage Full 2024.10.22 01:03:46.780993 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:03:46.800304 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_565_188} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:46.805735 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_565_188} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_564_187, total 33534 rows starting from the beginning of the part 2024.10.22 01:03:46.890675 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_565_188} MergeTreeSequentialSource: Reading 2 marks from part 202410_565_565_0, total 64 rows starting from the beginning of the part 2024.10.22 01:03:46.891730 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:03:46.902461 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:03:46.925494 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:03:47.000604 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:03:47.037255 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:03:47.091562 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xxb/hdhihkzgphhpvkqfplouaswrlwmfo (LogSeriesLimiter: on interval from 2024-10-22 01:03:31 to 2024-10-22 01:03:47 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:03:47.352442 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_565_188/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:03:47.532801 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:47.536114 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:47.605745 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 10.43 MiB, will set to 1.68 GiB (RSS), difference: -24.38 MiB 2024.10.22 01:03:47.675837 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_312_312_0 (state Temporary) 2024.10.22 01:03:47.855523 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_312_312_0 to 202410_319_319_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:47.903283 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 517806 2024.10.22 01:03:48.592564 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_565_188} MergeTask::MergeProjectionsStage: Merge sorted 33598 rows, containing 13 columns (13 merged, 0 gathered) in 1.936354441 sec., 17351.162209047245 rows/sec., 2.68 MiB/sec. 2024.10.22 01:03:48.600732 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 2647 2024.10.22 01:03:48.651756 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_565_188} MergedBlockOutputStream: filled checksums 202410_1_565_188 (state Temporary) 2024.10.22 01:03:48.898197 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_565_188} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_565_188 to 202410_1_565_188 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:48.900043 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_565_188} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_564_187, 202410_565_565_0] -> 202410_1_565_188 2024.10.22 01:03:48.921095 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:03:49.388983 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 10.64 MiB, will set to 1.68 GiB (RSS), difference: 12.21 MiB 2024.10.22 01:03:49.519486 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(176052c0463d29cefa31b0a09c1429cb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 176052c0463d29cefa31b0a09c1429cb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:03:49.523373 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 927, file segment info: File segment: [0, 33554431], key: 176052c0463d29cefa31b0a09c1429cb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:03:49.527408 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:49.594265 [ 1687 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yny/vuryrykucvnkbtcgpifhusigwjlbq, size 3075347 2024.10.22 01:03:49.848734 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:03:49.851491 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:03:49.857061 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:03:49.863142 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:03:49.876378 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:03:49.894323 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:03:49.901895 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:03:49.910994 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:03:49.933007 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:03:50.743353 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 33068 2024.10.22 01:03:50.983495 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:50.995920 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:51.006959 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 4784 2024.10.22 01:03:51.272928 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_557_557_0 (state Temporary) 2024.10.22 01:03:51.327233 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:51.334732 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:51.384461 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_327_327_0 (state Temporary) 2024.10.22 01:03:51.408855 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_557_557_0 to 202410_570_570_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:51.435521 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 11.07 MiB, will set to 1.69 GiB (RSS), difference: -5.23 MiB 2024.10.22 01:03:51.443837 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_565_188 to 202410_566_566_0 2024.10.22 01:03:51.450354 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.18 MiB using storage policy from min volume index 0 2024.10.22 01:03:51.459896 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33068 2024.10.22 01:03:51.459572 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_327_327_0 to 202410_340_340_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:51.463670 [ 1201 ] {} DiskLocal: Reserved 1.18 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:51.497003 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_566_189} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_565_188 to 202410_566_566_0 into Compact with storage Full 2024.10.22 01:03:51.577487 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_566_189} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:51.592381 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_566_189} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_565_188, total 33598 rows starting from the beginning of the part 2024.10.22 01:03:51.741322 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_566_189} MergeTreeSequentialSource: Reading 2 marks from part 202410_566_566_0, total 67 rows starting from the beginning of the part 2024.10.22 01:03:51.760453 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4784 2024.10.22 01:03:51.838744 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 5 old parts to remove. Parts: [202410_1_464_114, 202410_465_465_0, 202410_466_466_0, 202410_467_467_0, 202410_468_468_0] 2024.10.22 01:03:51.842933 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from filesystem (serially): Parts: [202410_1_464_114, 202410_465_465_0, 202410_466_466_0, 202410_467_467_0, 202410_468_468_0] 2024.10.22 01:03:51.867378 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from memory: Parts: [202410_1_464_114, 202410_465_465_0, 202410_466_466_0, 202410_467_467_0, 202410_468_468_0] 2024.10.22 01:03:53.593731 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 10.23 MiB, will set to 1.69 GiB (RSS), difference: 8.26 MiB 2024.10.22 01:03:53.684668 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_566_189} MergeTask::MergeProjectionsStage: Merge sorted 33665 rows, containing 13 columns (13 merged, 0 gathered) in 2.195837699 sec., 15331.27881688673 rows/sec., 2.37 MiB/sec. 2024.10.22 01:03:53.826960 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_566_189} MergedBlockOutputStream: filled checksums 202410_1_566_189 (state Temporary) 2024.10.22 01:03:53.917474 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_566_189} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_566_189 to 202410_1_566_189 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:53.930528 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_566_189} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_565_188, 202410_566_566_0] -> 202410_1_566_189 2024.10.22 01:03:54.088886 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:03:54.408323 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 101 entries to flush up to offset 25186 2024.10.22 01:03:54.500308 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:54.504553 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:54.565497 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_324_324_0 (state Temporary) 2024.10.22 01:03:54.797501 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_324_324_0 to 202410_330_330_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:54.834967 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 25186 2024.10.22 01:03:54.908252 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 519342 2024.10.22 01:03:55.106723 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_276_55, 202410_277_277_0, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0, 202410_281_281_0] 2024.10.22 01:03:55.109352 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_276_55, 202410_277_277_0, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0, 202410_281_281_0] 2024.10.22 01:03:55.285640 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_276_55, 202410_277_277_0, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0, 202410_281_281_0] 2024.10.22 01:03:55.478416 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 33107 2024.10.22 01:03:55.532486 [ 1215 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 214 ms. 2024.10.22 01:03:55.600741 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 8.02 MiB, will set to 1.68 GiB (RSS), difference: -1.05 MiB 2024.10.22 01:03:55.611117 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:55.615240 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:55.695772 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:55.699436 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:55.736038 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_558_558_0 (state Temporary) 2024.10.22 01:03:55.739208 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_313_313_0 (state Temporary) 2024.10.22 01:03:55.897334 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_558_558_0 to 202410_571_571_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:55.910491 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:03:55.914626 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_313_313_0 to 202410_320_320_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:55.931842 [ 1226 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_566_189 to 202410_567_567_0 2024.10.22 01:03:55.940046 [ 1226 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.18 MiB using storage policy from min volume index 0 2024.10.22 01:03:55.952657 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33107 2024.10.22 01:03:55.977344 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 519342 2024.10.22 01:03:56.070277 [ 1226 ] {} DiskLocal: Reserved 1.18 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:56.100349 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_567_190} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_566_189 to 202410_567_567_0 into Compact with storage Full 2024.10.22 01:03:56.155964 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_567_190} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:56.174589 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_567_190} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_566_189, total 33665 rows starting from the beginning of the part 2024.10.22 01:03:56.202326 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_567_190} MergeTreeSequentialSource: Reading 2 marks from part 202410_567_567_0, total 44 rows starting from the beginning of the part 2024.10.22 01:03:56.285182 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 282 2024.10.22 01:03:56.727401 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:56.729589 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:56.901471 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_264_264_0 (state Temporary) 2024.10.22 01:03:57.007891 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_264_264_0 to 202410_274_274_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:57.078203 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 282 2024.10.22 01:03:57.430547 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_256_51, 202410_257_257_0, 202410_258_258_0, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0] 2024.10.22 01:03:57.435622 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_256_51, 202410_257_257_0, 202410_258_258_0, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0] 2024.10.22 01:03:57.479536 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_256_51, 202410_257_257_0, 202410_258_258_0, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0] 2024.10.22 01:03:57.574970 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 6.85 MiB, will set to 1.69 GiB (RSS), difference: -9.79 MiB 2024.10.22 01:03:57.734492 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(5c2ad3b96e90900db108d9065bd616d0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5c2ad3b96e90900db108d9065bd616d0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:03:57.755888 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 5c2ad3b96e90900db108d9065bd616d0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:03:57.769409 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:58.035588 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_567_190/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:03:58.464872 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(02c47cbe32c9ba43344f38d2b77f64e6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 02c47cbe32c9ba43344f38d2b77f64e6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:03:58.467027 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 02c47cbe32c9ba43344f38d2b77f64e6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:03:58.470907 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:03:58.644068 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_567_190} MergeTask::MergeProjectionsStage: Merge sorted 33709 rows, containing 13 columns (13 merged, 0 gathered) in 2.559130491 sec., 13172.05203820144 rows/sec., 2.04 MiB/sec. 2024.10.22 01:03:58.851168 [ 1250 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:03:58.893341 [ 1250 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:03:58.895672 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_567_190} MergedBlockOutputStream: filled checksums 202410_1_567_190 (state Temporary) 2024.10.22 01:03:59.269118 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 4793 2024.10.22 01:03:59.278772 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_567_190} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_567_190 to 202410_1_567_190 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:59.284124 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_567_190} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_566_189, 202410_567_567_0] -> 202410_1_567_190 2024.10.22 01:03:59.444535 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:03:59.603458 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:59.606526 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:59.659682 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_328_328_0 (state Temporary) 2024.10.22 01:03:59.718182 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_328_328_0 to 202410_341_341_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:03:59.760057 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_336_67 to 202410_341_341_0 2024.10.22 01:03:59.777073 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:03:59.780096 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:03:59.849414 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_336_67 to 202410_341_341_0 into Compact with storage Full 2024.10.22 01:03:59.851093 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:03:59.885366 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4793 2024.10.22 01:03:59.908442 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:03:59.915897 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_336_67, total 4813 rows starting from the beginning of the part 2024.10.22 01:03:59.853716 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:03:59.973318 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 33165 2024.10.22 01:03:59.988569 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_337_337_0, total 21 rows starting from the beginning of the part 2024.10.22 01:04:00.050278 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_338_338_0, total 12 rows starting from the beginning of the part 2024.10.22 01:04:00.224125 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:00.229335 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_339_339_0, total 27 rows starting from the beginning of the part 2024.10.22 01:04:00.230752 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:04:00.309988 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_559_559_0 (state Temporary) 2024.10.22 01:04:00.376809 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_559_559_0 to 202410_572_572_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:00.413209 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_340_340_0, total 17 rows starting from the beginning of the part 2024.10.22 01:04:00.414413 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_567_190 to 202410_568_568_0 2024.10.22 01:04:00.428096 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.18 MiB using storage policy from min volume index 0 2024.10.22 01:04:00.438680 [ 1267 ] {} DiskLocal: Reserved 1.18 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:04:00.474696 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33165 2024.10.22 01:04:00.486950 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_568_191} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_567_190 to 202410_568_568_0 into Compact with storage Full 2024.10.22 01:04:00.525775 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_568_191} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:00.528886 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_341_341_0, total 9 rows starting from the beginning of the part 2024.10.22 01:04:00.535038 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_568_191} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_567_190, total 33709 rows starting from the beginning of the part 2024.10.22 01:04:00.656445 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_568_191} MergeTreeSequentialSource: Reading 2 marks from part 202410_568_568_0, total 64 rows starting from the beginning of the part 2024.10.22 01:04:00.698461 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 6.46 MiB, will set to 1.68 GiB (RSS), difference: 4.88 MiB 2024.10.22 01:04:01.386329 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} MergeTask::MergeProjectionsStage: Merge sorted 4899 rows, containing 12 columns (12 merged, 0 gathered) in 1.541865896 sec., 3177.319125294409 rows/sec., 499.85 KiB/sec. 2024.10.22 01:04:01.535258 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} MergedBlockOutputStream: filled checksums 202410_1_341_68 (state Temporary) 2024.10.22 01:04:01.851174 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_341_68 to 202410_1_341_68 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:01.855728 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_341_68} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_336_67, 202410_341_341_0] -> 202410_1_341_68 2024.10.22 01:04:02.011797 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.41 MiB. 2024.10.22 01:04:02.355099 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 25301 2024.10.22 01:04:02.421439 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 2.42 MiB, will set to 1.68 GiB (RSS), difference: 3.03 MiB 2024.10.22 01:04:02.547605 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_568_191} MergeTask::MergeProjectionsStage: Merge sorted 33773 rows, containing 13 columns (13 merged, 0 gathered) in 2.071168753 sec., 16306.252183015627 rows/sec., 2.52 MiB/sec. 2024.10.22 01:04:02.577432 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:02.579938 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:04:02.600774 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_568_191} MergedBlockOutputStream: filled checksums 202410_1_568_191 (state Temporary) 2024.10.22 01:04:02.800183 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_568_191} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_568_191 to 202410_1_568_191 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:02.802775 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_568_191} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_567_190, 202410_568_568_0] -> 202410_1_568_191 2024.10.22 01:04:02.819850 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_325_325_0 (state Temporary) 2024.10.22 01:04:02.850359 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:04:02.893495 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_325_325_0 to 202410_331_331_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:02.954322 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 25301 2024.10.22 01:04:03.087794 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1496 entries to flush up to offset 520838 2024.10.22 01:04:03.599708 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:03.602223 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:04:03.767737 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:03.771477 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:04:03.992432 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_314_314_0 (state Temporary) 2024.10.22 01:04:04.169212 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_314_314_0 to 202410_321_321_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:04.193064 [ 1175 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_316_63 to 202410_321_321_0 2024.10.22 01:04:04.197732 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 520838 2024.10.22 01:04:04.200688 [ 1175 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:04.302403 [ 1175 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:04:04.346736 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_316_63 to 202410_321_321_0 into Compact with storage Full 2024.10.22 01:04:04.364405 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:04.376645 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} MergeTreeSequentialSource: Reading 66 marks from part 202410_1_316_63, total 526404 rows starting from the beginning of the part 2024.10.22 01:04:04.386390 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_317_317_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:04:04.396410 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_318_318_0, total 1528 rows starting from the beginning of the part 2024.10.22 01:04:04.406676 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_319_319_0, total 1928 rows starting from the beginning of the part 2024.10.22 01:04:04.422282 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_320_320_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:04:04.451574 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_321_321_0, total 1496 rows starting from the beginning of the part 2024.10.22 01:04:04.478955 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 33221 2024.10.22 01:04:04.544149 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 3.63 MiB, will set to 1.68 GiB (RSS), difference: -4.49 MiB 2024.10.22 01:04:04.580339 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 283 2024.10.22 01:04:04.680021 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:04.683228 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:04:04.787775 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_560_560_0 (state Temporary) 2024.10.22 01:04:04.796943 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(1a41da72912ad243a87a79ddf1d299a8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1a41da72912ad243a87a79ddf1d299a8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:04:04.802815 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 1a41da72912ad243a87a79ddf1d299a8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:04:04.837007 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_560_560_0 to 202410_573_573_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:04.859537 [ 1225 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_568_191 to 202410_569_569_0 2024.10.22 01:04:04.874686 [ 1225 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.19 MiB using storage policy from min volume index 0 2024.10.22 01:04:04.879609 [ 1225 ] {} DiskLocal: Reserved 1.19 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:04:04.893612 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33221 2024.10.22 01:04:04.901222 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_569_192} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_568_191 to 202410_569_569_0 into Compact with storage Full 2024.10.22 01:04:04.908016 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:04.975904 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_569_192} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:05.065236 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:05.074144 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:05.227953 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_569_192} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_568_191, total 33773 rows starting from the beginning of the part 2024.10.22 01:04:05.353409 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_265_265_0 (state Temporary) 2024.10.22 01:04:05.430763 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_265_265_0 to 202410_275_275_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:05.483753 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_569_192} MergeTreeSequentialSource: Reading 2 marks from part 202410_569_569_0, total 65 rows starting from the beginning of the part 2024.10.22 01:04:05.530396 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 283 2024.10.22 01:04:06.476510 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_321_64/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:04:06.556175 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 9.24 MiB, will set to 1.69 GiB (RSS), difference: -15.63 MiB 2024.10.22 01:04:06.703325 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_66_66_0 (state Temporary) 2024.10.22 01:04:07.279604 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(56cace9a7f4063834ae3deed45aaed35) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 56cace9a7f4063834ae3deed45aaed35, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:04:07.283827 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 907, file segment info: File segment: [0, 33554431], key: 56cace9a7f4063834ae3deed45aaed35, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:04:07.287535 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:07.388353 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 4812 2024.10.22 01:04:07.584113 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:07.587991 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:07.616654 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_329_329_0 (state Temporary) 2024.10.22 01:04:07.705364 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:04:07.709531 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:04:07.740955 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:04:07.773685 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:04:07.864656 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_66_66_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:04:07.872968 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_329_329_0 to 202410_342_342_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:07.878394 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_49_10, 202410_50_50_0, 202410_51_51_0, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0] 2024.10.22 01:04:07.880276 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_49_10, 202410_50_50_0, 202410_51_51_0, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0] 2024.10.22 01:04:07.897406 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:04:07.912304 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_49_10, 202410_50_50_0, 202410_51_51_0, 202410_52_52_0, 202410_53_53_0, 202410_54_54_0] 2024.10.22 01:04:07.913314 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:04:07.915557 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_569_192} MergeTask::MergeProjectionsStage: Merge sorted 33838 rows, containing 13 columns (13 merged, 0 gathered) in 3.018961791 sec., 11208.488991439506 rows/sec., 1.73 MiB/sec. 2024.10.22 01:04:07.934974 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4812 2024.10.22 01:04:07.944423 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:04:07.979496 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_569_192} MergedBlockOutputStream: filled checksums 202410_1_569_192 (state Temporary) 2024.10.22 01:04:08.076789 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/usw/krbsjxieopnvczcmybiaibkuabzgp. 2024.10.22 01:04:08.127685 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qyv/uuqtmwujtktqoasudptzzlhbyidzu (LogSeriesLimiter: on interval from 2024-10-22 01:03:47 to 2024-10-22 01:04:08 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:04:08.242297 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_569_192} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_569_192 to 202410_1_569_192 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:08.338785 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 5 old parts to remove. Parts: [202410_1_268_73, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0, 202410_272_272_0] 2024.10.22 01:04:08.345484 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from filesystem (serially): Parts: [202410_1_268_73, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0, 202410_272_272_0] 2024.10.22 01:04:08.348375 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_569_192} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_568_191, 202410_569_569_0] -> 202410_1_569_192 2024.10.22 01:04:08.375553 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from memory: Parts: [202410_1_268_73, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0, 202410_272_272_0] 2024.10.22 01:04:08.386731 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:04:08.628276 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.88 GiB, free memory in arenas 10.82 MiB, will set to 1.68 GiB (RSS), difference: 100.26 MiB 2024.10.22 01:04:08.899646 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 33278 2024.10.22 01:04:09.138675 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:09.142297 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:09.295390 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_561_561_0 (state Temporary) 2024.10.22 01:04:09.340896 [ 1273 ] {} DatabaseCatalog: Execution took 207 ms. 2024.10.22 01:04:09.363132 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_561_561_0 to 202410_574_574_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:09.386523 [ 1211 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_569_192 to 202410_570_570_0 2024.10.22 01:04:09.391583 [ 1211 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.19 MiB using storage policy from min volume index 0 2024.10.22 01:04:09.395099 [ 1211 ] {} DiskLocal: Reserved 1.19 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:09.403111 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_570_193} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_569_192 to 202410_570_570_0 into Compact with storage Full 2024.10.22 01:04:09.403184 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33278 2024.10.22 01:04:09.438682 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_570_193} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:09.445085 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_570_193} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_569_192, total 33838 rows starting from the beginning of the part 2024.10.22 01:04:09.465774 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_570_193} MergeTreeSequentialSource: Reading 2 marks from part 202410_570_570_0, total 61 rows starting from the beginning of the part 2024.10.22 01:04:09.951850 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:04:09.956230 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:04:10.347510 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.88 GiB, free memory in arenas 12.50 MiB, will set to 1.68 GiB (RSS), difference: 84.31 MiB 2024.10.22 01:04:10.341759 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1_468_115, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0] 2024.10.22 01:04:10.353201 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1_468_115, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0] 2024.10.22 01:04:10.476224 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1_468_115, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0] 2024.10.22 01:04:10.478962 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 114 entries to flush up to offset 25415 2024.10.22 01:04:10.591441 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:10.592540 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:10.649004 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_326_326_0 (state Temporary) 2024.10.22 01:04:10.796942 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_326_326_0 to 202410_332_332_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:10.850938 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 25415 2024.10.22 01:04:11.201143 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1576 entries to flush up to offset 522414 2024.10.22 01:04:11.250056 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_570_193} MergeTask::MergeProjectionsStage: Merge sorted 33899 rows, containing 13 columns (13 merged, 0 gathered) in 1.850467986 sec., 18319.149672660158 rows/sec., 2.83 MiB/sec. 2024.10.22 01:04:11.292444 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_570_193} MergedBlockOutputStream: filled checksums 202410_1_570_193 (state Temporary) 2024.10.22 01:04:11.604807 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_570_193} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_570_193 to 202410_1_570_193 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:11.611796 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_570_193} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_569_192, 202410_570_570_0] -> 202410_1_570_193 2024.10.22 01:04:11.650990 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:04:11.706930 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e8fac1dd9fcbdd0ea08bedd4e4925cbd) : [0, 211]: Setting download as failed: File segment: [0, 211], key: e8fac1dd9fcbdd0ea08bedd4e4925cbd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:11.707798 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e8fac1dd9fcbdd0ea08bedd4e4925cbd): No space left in cache to reserve 212 bytes, will continue without cache download 2024.10.22 01:04:11.774698 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:11.782702 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:12.002823 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_315_315_0 (state Temporary) 2024.10.22 01:04:12.252758 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_315_315_0 to 202410_322_322_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:12.350906 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 15.32 MiB, will set to 1.68 GiB (RSS), difference: 16.78 MiB 2024.10.22 01:04:12.383067 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 522414 2024.10.22 01:04:12.616405 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7ddab3c1eff09103124b89b2cdf9dcb5) : [0, 2303660]: Setting download as failed: File segment: [0, 2303660], key: 7ddab3c1eff09103124b89b2cdf9dcb5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:12.618926 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7ddab3c1eff09103124b89b2cdf9dcb5): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:04:12.773427 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ddab3c1eff09103124b89b2cdf9dcb5, offset: 0, size: 0 2024.10.22 01:04:12.775837 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ddab3c1eff09103124b89b2cdf9dcb5 for removal 2024.10.22 01:04:12.779066 [ 1476 ] {} CacheMetadata: Key 7ddab3c1eff09103124b89b2cdf9dcb5 is removed from metadata 2024.10.22 01:04:12.782105 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e8fac1dd9fcbdd0ea08bedd4e4925cbd, offset: 0, size: 0 2024.10.22 01:04:12.785077 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e8fac1dd9fcbdd0ea08bedd4e4925cbd for removal 2024.10.22 01:04:12.788173 [ 1476 ] {} CacheMetadata: Key e8fac1dd9fcbdd0ea08bedd4e4925cbd is removed from metadata 2024.10.22 01:04:12.884526 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_66_66_0 to 202410_69_69_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:13.011648 [ 1202 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_64_13 to 202410_69_69_0 2024.10.22 01:04:13.027615 [ 1202 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.48 MiB using storage policy from min volume index 0 2024.10.22 01:04:13.031408 [ 1202 ] {} DiskLocal: Reserved 1.48 MiB on local disk `default`, having unreserved 297.57 GiB. 2024.10.22 01:04:13.032112 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 284 2024.10.22 01:04:13.101489 [ 1202 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 205 ms. 2024.10.22 01:04:13.136076 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_64_13 to 202410_69_69_0 into Compact with storage Full 2024.10.22 01:04:13.399658 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:13.404183 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:13.524303 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 33334 2024.10.22 01:04:13.709461 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_266_266_0 (state Temporary) 2024.10.22 01:04:13.779688 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(f0c4e70b1cbfe69855dedc87a6c5c8bf) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f0c4e70b1cbfe69855dedc87a6c5c8bf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 01:04:13.792473 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: f0c4e70b1cbfe69855dedc87a6c5c8bf, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 01:04:13.795439 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:13.951199 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_266_266_0 to 202410_276_276_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:13.988891 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:13.990525 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:14.000728 [ 1222 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:04:14.049087 [ 1222 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:04:14.052511 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_562_562_0 (state Temporary) 2024.10.22 01:04:14.115821 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 284 2024.10.22 01:04:14.199820 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_562_562_0 to 202410_575_575_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:14.308214 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(69f96a54b317f1932938253520300f9d) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 69f96a54b317f1932938253520300f9d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:14.310510 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(69f96a54b317f1932938253520300f9d): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:04:14.330786 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_570_193 to 202410_571_571_0 2024.10.22 01:04:14.345619 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.19 MiB using storage policy from min volume index 0 2024.10.22 01:04:14.386813 [ 1244 ] {} DiskLocal: Reserved 1.19 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:14.419717 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_571_194} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_570_193 to 202410_571_571_0 into Compact with storage Full 2024.10.22 01:04:14.499019 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33334 2024.10.22 01:04:14.544654 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_571_194} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:14.651374 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_571_194} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_570_193, total 33899 rows starting from the beginning of the part 2024.10.22 01:04:14.693321 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_571_194} MergeTreeSequentialSource: Reading 2 marks from part 202410_571_571_0, total 39 rows starting from the beginning of the part 2024.10.22 01:04:15.174436 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(02df69b053a8cf9880d45def16c3a517) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 02df69b053a8cf9880d45def16c3a517, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:04:15.194927 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1264, file segment info: File segment: [0, 33554431], key: 02df69b053a8cf9880d45def16c3a517, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:04:15.200166 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:15.448629 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 4825 2024.10.22 01:04:15.675984 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 01:04:15.732410 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 15.69 MiB, will set to 1.68 GiB (RSS), difference: 9.66 MiB 2024.10.22 01:04:15.784259 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:15.786296 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:15.820099 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_330_330_0 (state Temporary) 2024.10.22 01:04:15.871770 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_330_330_0 to 202410_343_343_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:15.924408 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4825 2024.10.22 01:04:16.010348 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9f11fd39254222bff38d3065b1f28e79) : [0, 50386]: Setting download as failed: File segment: [0, 50386], key: 9f11fd39254222bff38d3065b1f28e79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:16.012615 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9f11fd39254222bff38d3065b1f28e79): No space left in cache to reserve 50387 bytes, will continue without cache download 2024.10.22 01:04:16.154431 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9f11fd39254222bff38d3065b1f28e79, offset: 0, size: 0 2024.10.22 01:04:16.158361 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9f11fd39254222bff38d3065b1f28e79 for removal 2024.10.22 01:04:16.164869 [ 1476 ] {} CacheMetadata: Key 9f11fd39254222bff38d3065b1f28e79 is removed from metadata 2024.10.22 01:04:16.174477 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 69f96a54b317f1932938253520300f9d, offset: 0, size: 0 2024.10.22 01:04:16.179094 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 69f96a54b317f1932938253520300f9d for removal 2024.10.22 01:04:16.185257 [ 1476 ] {} CacheMetadata: Key 69f96a54b317f1932938253520300f9d is removed from metadata 2024.10.22 01:04:16.189030 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2647 2024.10.22 01:04:16.581695 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a0a4f5ed4a694b52cedd35e4ad77bbfb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a0a4f5ed4a694b52cedd35e4ad77bbfb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:04:16.587149 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1007, file segment info: File segment: [0, 33554431], key: a0a4f5ed4a694b52cedd35e4ad77bbfb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:04:16.589302 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:16.740235 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_571_194} MergeTask::MergeProjectionsStage: Merge sorted 33938 rows, containing 13 columns (13 merged, 0 gathered) in 2.330983742 sec., 14559.518107526981 rows/sec., 2.25 MiB/sec. 2024.10.22 01:04:16.784014 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_571_194} MergedBlockOutputStream: filled checksums 202410_1_571_194 (state Temporary) 2024.10.22 01:04:16.865650 [ 1675 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xxb/hdhihkzgphhpvkqfplouaswrlwmfo, size 3318905 2024.10.22 01:04:16.958719 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:04:16.993113 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:04:16.993472 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_571_194} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_571_194 to 202410_1_571_194 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:16.997714 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_571_194} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_570_193, 202410_571_571_0] -> 202410_1_571_194 2024.10.22 01:04:17.006091 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:04:17.034650 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:04:17.070998 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:04:17.077545 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:04:17.101113 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:04:17.146349 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:04:17.251455 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:04:17.301288 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:04:17.388615 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(865f109620f44399b807d9f7e3f581b6) : [0, 166]: Setting download as failed: File segment: [0, 166], key: 865f109620f44399b807d9f7e3f581b6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:17.390340 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(865f109620f44399b807d9f7e3f581b6): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 01:04:17.493611 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nsh/nsufyjdwjbzxsnxzfdbfeetejumbo (LogSeriesLimiter: on interval from 2024-10-22 01:04:08 to 2024-10-22 01:04:17 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:04:17.797153 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_321_64/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:04:18.062956 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.279112494 sec. 2024.10.22 01:04:18.188496 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7f69a69af7b8840db2f590b7e962ef34) : [0, 2572]: Setting download as failed: File segment: [0, 2572], key: 7f69a69af7b8840db2f590b7e962ef34, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:18.190562 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7f69a69af7b8840db2f590b7e962ef34): No space left in cache to reserve 2573 bytes, will continue without cache download 2024.10.22 01:04:18.312461 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7f69a69af7b8840db2f590b7e962ef34, offset: 0, size: 0 2024.10.22 01:04:18.316893 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7f69a69af7b8840db2f590b7e962ef34 for removal 2024.10.22 01:04:18.324059 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 865f109620f44399b807d9f7e3f581b6, offset: 0, size: 0 2024.10.22 01:04:18.327808 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 865f109620f44399b807d9f7e3f581b6 for removal 2024.10.22 01:04:18.466445 [ 1476 ] {} CacheMetadata: Key 7f69a69af7b8840db2f590b7e962ef34 is removed from metadata 2024.10.22 01:04:18.477471 [ 1476 ] {} CacheMetadata: Key 865f109620f44399b807d9f7e3f581b6 is removed from metadata 2024.10.22 01:04:18.487979 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 85 entries to flush up to offset 25500 2024.10.22 01:04:18.507416 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 33410 2024.10.22 01:04:18.634161 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:18.652322 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:18.800489 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:18.806094 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:18.819646 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_563_563_0 (state Temporary) 2024.10.22 01:04:18.872224 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_563_563_0 to 202410_576_576_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:18.915104 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_571_194 to 202410_572_572_0 2024.10.22 01:04:18.922088 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.19 MiB using storage policy from min volume index 0 2024.10.22 01:04:18.925148 [ 1255 ] {} DiskLocal: Reserved 1.19 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:18.965176 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_572_195} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_571_194 to 202410_572_572_0 into Compact with storage Full 2024.10.22 01:04:19.014067 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1ff2f2e10c3d5f473326d52a485127b2) : [0, 168]: Setting download as failed: File segment: [0, 168], key: 1ff2f2e10c3d5f473326d52a485127b2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:19.015863 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1ff2f2e10c3d5f473326d52a485127b2): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:04:19.026334 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33410 2024.10.22 01:04:19.047637 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_572_195} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:19.051862 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_327_327_0 (state Temporary) 2024.10.22 01:04:19.059636 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_572_195} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_571_194, total 33938 rows starting from the beginning of the part 2024.10.22 01:04:19.082425 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_572_195} MergeTreeSequentialSource: Reading 2 marks from part 202410_572_572_0, total 58 rows starting from the beginning of the part 2024.10.22 01:04:19.343068 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_327_327_0 to 202410_333_333_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:19.382407 [ 1258 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_328_110 to 202410_329_329_0 2024.10.22 01:04:19.386519 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 768 entries to flush up to offset 523182 2024.10.22 01:04:19.389021 [ 1258 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.36 MiB using storage policy from min volume index 0 2024.10.22 01:04:19.391292 [ 1258 ] {} DiskLocal: Reserved 1.36 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:19.402423 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_329_111} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_328_110 to 202410_329_329_0 into Wide with storage Full 2024.10.22 01:04:19.452165 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_329_111} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:19.467993 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_329_111} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_328_110, total 24988 rows starting from the beginning of the part 2024.10.22 01:04:19.477730 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 25500 2024.10.22 01:04:19.501181 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 14.72 MiB, will set to 1.67 GiB (RSS), difference: -12.41 MiB 2024.10.22 01:04:19.590113 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(77670110612eae054629d9e1502c9683) : [0, 1121086]: Setting download as failed: File segment: [0, 1121086], key: 77670110612eae054629d9e1502c9683, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:19.696784 [ 1250 ] {} DatabaseCatalog: Execution took 285 ms. 2024.10.22 01:04:19.696734 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(77670110612eae054629d9e1502c9683): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:04:19.802033 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_329_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_329_329_0, total 201 rows starting from the beginning of the part 2024.10.22 01:04:19.832345 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 77670110612eae054629d9e1502c9683, offset: 0, size: 0 2024.10.22 01:04:19.835651 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 77670110612eae054629d9e1502c9683 for removal 2024.10.22 01:04:19.847500 [ 1476 ] {} CacheMetadata: Key 77670110612eae054629d9e1502c9683 is removed from metadata 2024.10.22 01:04:19.855756 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:04:19.856999 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1ff2f2e10c3d5f473326d52a485127b2, offset: 0, size: 0 2024.10.22 01:04:19.864593 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:04:19.865099 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1ff2f2e10c3d5f473326d52a485127b2 for removal 2024.10.22 01:04:19.871094 [ 1476 ] {} CacheMetadata: Key 1ff2f2e10c3d5f473326d52a485127b2 is removed from metadata 2024.10.22 01:04:19.971768 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:20.074736 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:20.199129 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_316_316_0 (state Temporary) 2024.10.22 01:04:20.287740 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_316_316_0 to 202410_323_323_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:20.350389 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 523182 2024.10.22 01:04:20.683496 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3295a658cae9ddde42899eab2bd122d2) : [0, 166]: Setting download as failed: File segment: [0, 166], key: 3295a658cae9ddde42899eab2bd122d2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:20.687374 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3295a658cae9ddde42899eab2bd122d2): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 01:04:21.080159 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_572_195} MergeTask::MergeProjectionsStage: Merge sorted 33996 rows, containing 13 columns (13 merged, 0 gathered) in 2.125722275 sec., 15992.681828579887 rows/sec., 2.47 MiB/sec. 2024.10.22 01:04:21.224280 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_572_195} MergedBlockOutputStream: filled checksums 202410_1_572_195 (state Temporary) 2024.10.22 01:04:21.275639 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:21.501646 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ee12af8da72019ed0bb91d15b01e8e19) : [0, 39221]: Setting download as failed: File segment: [0, 39221], key: ee12af8da72019ed0bb91d15b01e8e19, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:21.503607 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ee12af8da72019ed0bb91d15b01e8e19): No space left in cache to reserve 39222 bytes, will continue without cache download 2024.10.22 01:04:21.587286 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 18.21 MiB, will set to 1.69 GiB (RSS), difference: -33.39 MiB 2024.10.22 01:04:21.620275 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 285 2024.10.22 01:04:21.624341 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ee12af8da72019ed0bb91d15b01e8e19, offset: 0, size: 0 2024.10.22 01:04:21.628217 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ee12af8da72019ed0bb91d15b01e8e19 for removal 2024.10.22 01:04:21.633255 [ 1476 ] {} CacheMetadata: Key ee12af8da72019ed0bb91d15b01e8e19 is removed from metadata 2024.10.22 01:04:21.635566 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3295a658cae9ddde42899eab2bd122d2, offset: 0, size: 0 2024.10.22 01:04:21.641333 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3295a658cae9ddde42899eab2bd122d2 for removal 2024.10.22 01:04:21.646552 [ 1476 ] {} CacheMetadata: Key 3295a658cae9ddde42899eab2bd122d2 is removed from metadata 2024.10.22 01:04:21.785504 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_64_13, total 2514 rows starting from the beginning of the part 2024.10.22 01:04:21.787457 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_572_195} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_572_195 to 202410_1_572_195 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:21.790265 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_572_195} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_571_194, 202410_572_572_0] -> 202410_1_572_195 2024.10.22 01:04:21.816347 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:04:21.951619 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:21.962681 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:22.089078 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_329_111} MergeTask::MergeProjectionsStage: Merge sorted 25189 rows, containing 13 columns (13 merged, 0 gathered) in 2.691158817 sec., 9359.90839369329 rows/sec., 3.98 MiB/sec. 2024.10.22 01:04:22.139259 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_267_267_0 (state Temporary) 2024.10.22 01:04:22.209379 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_267_267_0 to 202410_277_277_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:22.238161 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_329_111} MergedBlockOutputStream: filled checksums 202410_1_329_111 (state Temporary) 2024.10.22 01:04:22.277770 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 285 2024.10.22 01:04:22.621523 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b99f60163bd176ef91c56e36be744b02) : [0, 150]: Setting download as failed: File segment: [0, 150], key: b99f60163bd176ef91c56e36be744b02, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:22.676081 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_329_111} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_329_111 to 202410_1_329_111 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:22.681562 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_329_111} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_328_110, 202410_329_329_0] -> 202410_1_329_111 2024.10.22 01:04:22.701887 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} MergeTask::MergeProjectionsStage: Merge sorted 534428 rows, containing 4 columns (4 merged, 0 gathered) in 18.385840903 sec., 29067.36780871404 rows/sec., 426.22 KiB/sec. 2024.10.22 01:04:22.705695 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 58.65 MiB. 2024.10.22 01:04:22.625725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b99f60163bd176ef91c56e36be744b02): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:04:22.753243 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} MergedBlockOutputStream: filled checksums 202410_1_321_64 (state Temporary) 2024.10.22 01:04:22.899188 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_321_64 to 202410_1_321_64 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:22.904589 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_321_64} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_316_63, 202410_321_321_0] -> 202410_1_321_64 2024.10.22 01:04:22.943817 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:04:23.031857 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 33491 2024.10.22 01:04:23.164458 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:23.166835 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:23.304843 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_564_564_0 (state Temporary) 2024.10.22 01:04:23.357441 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_564_564_0 to 202410_577_577_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:23.385680 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 16.18 MiB, will set to 1.67 GiB (RSS), difference: 51.67 MiB 2024.10.22 01:04:23.393035 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_572_195 to 202410_573_573_0 2024.10.22 01:04:23.400011 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.19 MiB using storage policy from min volume index 0 2024.10.22 01:04:23.402913 [ 1186 ] {} DiskLocal: Reserved 1.19 MiB on local disk `default`, having unreserved 297.56 GiB. 2024.10.22 01:04:23.403779 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33491 2024.10.22 01:04:23.412043 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_573_196} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_572_195 to 202410_573_573_0 into Compact with storage Full 2024.10.22 01:04:23.429162 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 4847 2024.10.22 01:04:23.524518 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:23.528637 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:23.560693 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_573_196} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:23.573384 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_331_331_0 (state Temporary) 2024.10.22 01:04:23.582361 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_573_196} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_572_195, total 33996 rows starting from the beginning of the part 2024.10.22 01:04:23.594286 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(db8fcaf582495fb2eaacf4e777b09ece) : [0, 29920]: Setting download as failed: File segment: [0, 29920], key: db8fcaf582495fb2eaacf4e777b09ece, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:23.597470 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(db8fcaf582495fb2eaacf4e777b09ece): No space left in cache to reserve 29921 bytes, will continue without cache download 2024.10.22 01:04:23.624772 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: db8fcaf582495fb2eaacf4e777b09ece, offset: 0, size: 0 2024.10.22 01:04:23.627851 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key db8fcaf582495fb2eaacf4e777b09ece for removal 2024.10.22 01:04:23.631148 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_331_331_0 to 202410_344_344_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:23.633782 [ 1476 ] {} CacheMetadata: Key db8fcaf582495fb2eaacf4e777b09ece is removed from metadata 2024.10.22 01:04:23.635983 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b99f60163bd176ef91c56e36be744b02, offset: 0, size: 0 2024.10.22 01:04:23.637602 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b99f60163bd176ef91c56e36be744b02 for removal 2024.10.22 01:04:23.642534 [ 1476 ] {} CacheMetadata: Key b99f60163bd176ef91c56e36be744b02 is removed from metadata 2024.10.22 01:04:23.661450 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_573_196} MergeTreeSequentialSource: Reading 2 marks from part 202410_573_573_0, total 56 rows starting from the beginning of the part 2024.10.22 01:04:23.709169 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4847 2024.10.22 01:04:23.737454 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 35 entries to flush up to offset 2682 2024.10.22 01:04:23.869132 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_65_65_0, total 39 rows starting from the beginning of the part 2024.10.22 01:04:24.079815 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(0018447e89d1ea3b4e1de321afe217e6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0018447e89d1ea3b4e1de321afe217e6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:04:24.083161 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 937, file segment info: File segment: [0, 33554431], key: 0018447e89d1ea3b4e1de321afe217e6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:04:24.086825 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:24.164553 [ 1711 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qyv/uuqtmwujtktqoasudptzzlhbyidzu, size 3466948 2024.10.22 01:04:24.309049 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_471_116, 202410_472_472_0] 2024.10.22 01:04:24.312167 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_471_116, 202410_472_472_0] 2024.10.22 01:04:24.328466 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_471_116, 202410_472_472_0] 2024.10.22 01:04:24.331067 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:04:24.346407 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:04:24.353140 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:04:24.475814 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:04:24.485049 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:04:24.496367 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:04:24.508540 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:04:24.571948 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ylc/cwokgazznemmagmcvnfstpztdkyzx (LogSeriesLimiter: on interval from 2024-10-22 01:04:17 to 2024-10-22 01:04:24 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:04:24.720383 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(edf6d155a22f7bc279053d972a7d2e12) : [0, 157]: Setting download as failed: File segment: [0, 157], key: edf6d155a22f7bc279053d972a7d2e12, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:24.722318 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(edf6d155a22f7bc279053d972a7d2e12): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:04:25.083446 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_573_196/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:04:25.403069 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_66_66_0, total 38 rows starting from the beginning of the part 2024.10.22 01:04:25.609525 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e08850a7ab046a9d7b05e24914b9f797) : [0, 285540]: Setting download as failed: File segment: [0, 285540], key: e08850a7ab046a9d7b05e24914b9f797, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:25.611698 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e08850a7ab046a9d7b05e24914b9f797): No space left in cache to reserve 285541 bytes, will continue without cache download 2024.10.22 01:04:25.629373 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e08850a7ab046a9d7b05e24914b9f797, offset: 0, size: 0 2024.10.22 01:04:25.631224 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e08850a7ab046a9d7b05e24914b9f797 for removal 2024.10.22 01:04:25.640147 [ 1476 ] {} CacheMetadata: Key e08850a7ab046a9d7b05e24914b9f797 is removed from metadata 2024.10.22 01:04:25.646592 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: edf6d155a22f7bc279053d972a7d2e12, offset: 0, size: 0 2024.10.22 01:04:25.650372 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key edf6d155a22f7bc279053d972a7d2e12 for removal 2024.10.22 01:04:25.652893 [ 1476 ] {} CacheMetadata: Key edf6d155a22f7bc279053d972a7d2e12 is removed from metadata 2024.10.22 01:04:26.403088 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(80418cbcbed18b98d18709b2dea2791d) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 80418cbcbed18b98d18709b2dea2791d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:26.406467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(80418cbcbed18b98d18709b2dea2791d): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:04:26.479242 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 13.83 MiB, will set to 1.68 GiB (RSS), difference: -5.86 MiB 2024.10.22 01:04:26.479747 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_573_196} MergeTask::MergeProjectionsStage: Merge sorted 34052 rows, containing 13 columns (13 merged, 0 gathered) in 3.071862165 sec., 11085.132786223172 rows/sec., 1.71 MiB/sec. 2024.10.22 01:04:26.505154 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_573_196} MergedBlockOutputStream: filled checksums 202410_1_573_196 (state Temporary) 2024.10.22 01:04:26.582350 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_573_196} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_573_196 to 202410_1_573_196 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:26.585341 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_573_196} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_572_195, 202410_573_573_0] -> 202410_1_573_196 2024.10.22 01:04:26.630834 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:04:26.705611 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:04:26.803214 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_67_67_0, total 42 rows starting from the beginning of the part 2024.10.22 01:04:26.990935 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 215 entries to flush up to offset 25715 2024.10.22 01:04:27.086455 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1b8d7e27b7a49722edf3de802964d92d) : [0, 185836]: Setting download as failed: File segment: [0, 185836], key: 1b8d7e27b7a49722edf3de802964d92d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:27.087430 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1b8d7e27b7a49722edf3de802964d92d): No space left in cache to reserve 185837 bytes, will continue without cache download 2024.10.22 01:04:27.103358 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1b8d7e27b7a49722edf3de802964d92d, offset: 0, size: 0 2024.10.22 01:04:27.104570 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1b8d7e27b7a49722edf3de802964d92d for removal 2024.10.22 01:04:27.107097 [ 1476 ] {} CacheMetadata: Key 1b8d7e27b7a49722edf3de802964d92d is removed from metadata 2024.10.22 01:04:27.107118 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 80418cbcbed18b98d18709b2dea2791d, offset: 0, size: 0 2024.10.22 01:04:27.110679 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 80418cbcbed18b98d18709b2dea2791d for removal 2024.10.22 01:04:27.114247 [ 1476 ] {} CacheMetadata: Key 80418cbcbed18b98d18709b2dea2791d is removed from metadata 2024.10.22 01:04:27.376669 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:27.380445 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:27.407208 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1223 entries to flush up to offset 524405 2024.10.22 01:04:27.411128 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 33569 2024.10.22 01:04:27.444675 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_328_328_0 (state Temporary) 2024.10.22 01:04:27.557090 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:27.563628 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:27.611707 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_565_565_0 (state Temporary) 2024.10.22 01:04:27.698124 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_328_328_0 to 202410_334_334_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:27.718240 [ 1278 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_329_111 to 202410_330_330_0 2024.10.22 01:04:27.723343 [ 1278 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 01:04:27.726221 [ 1278 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:27.735137 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 25715 2024.10.22 01:04:27.741469 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_330_112} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_329_111 to 202410_330_330_0 into Wide with storage Full 2024.10.22 01:04:27.783495 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_330_112} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:27.788500 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_330_112} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_329_111, total 25189 rows starting from the beginning of the part 2024.10.22 01:04:27.794395 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(43866cf0a1ad2e3e233de7f4af4359b0) : [0, 182]: Setting download as failed: File segment: [0, 182], key: 43866cf0a1ad2e3e233de7f4af4359b0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:27.795728 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(43866cf0a1ad2e3e233de7f4af4359b0): No space left in cache to reserve 183 bytes, will continue without cache download 2024.10.22 01:04:27.796445 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_565_565_0 to 202410_578_578_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:27.828037 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_573_196 to 202410_574_574_0 2024.10.22 01:04:27.840391 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.20 MiB using storage policy from min volume index 0 2024.10.22 01:04:27.873302 [ 1161 ] {} DiskLocal: Reserved 1.20 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:27.890216 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33569 2024.10.22 01:04:27.998392 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_574_197} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_573_196 to 202410_574_574_0 into Compact with storage Full 2024.10.22 01:04:28.011730 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_574_197} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:28.022959 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_574_197} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_573_196, total 34052 rows starting from the beginning of the part 2024.10.22 01:04:28.023172 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_330_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_330_330_0, total 101 rows starting from the beginning of the part 2024.10.22 01:04:28.034550 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_574_197} MergeTreeSequentialSource: Reading 2 marks from part 202410_574_574_0, total 57 rows starting from the beginning of the part 2024.10.22 01:04:28.130090 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_472_117, 202410_473_473_0] 2024.10.22 01:04:28.132893 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_472_117, 202410_473_473_0] 2024.10.22 01:04:28.148428 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_472_117, 202410_473_473_0] 2024.10.22 01:04:28.208937 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_68_68_0, total 38 rows starting from the beginning of the part 2024.10.22 01:04:28.453082 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 6.78 MiB, will set to 1.67 GiB (RSS), difference: -48.86 MiB 2024.10.22 01:04:28.485259 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:28.507845 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:28.671206 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2559b76c202a845b1ad93fb80c2d84bd) : [0, 418558]: Setting download as failed: File segment: [0, 418558], key: 2559b76c202a845b1ad93fb80c2d84bd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:28.676025 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2559b76c202a845b1ad93fb80c2d84bd): No space left in cache to reserve 418559 bytes, will continue without cache download 2024.10.22 01:04:28.682315 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_317_317_0 (state Temporary) 2024.10.22 01:04:28.781715 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_317_317_0 to 202410_324_324_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:28.807278 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2559b76c202a845b1ad93fb80c2d84bd, offset: 0, size: 0 2024.10.22 01:04:28.808653 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2559b76c202a845b1ad93fb80c2d84bd for removal 2024.10.22 01:04:28.811642 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 43866cf0a1ad2e3e233de7f4af4359b0, offset: 0, size: 0 2024.10.22 01:04:28.812495 [ 1476 ] {} CacheMetadata: Key 2559b76c202a845b1ad93fb80c2d84bd is removed from metadata 2024.10.22 01:04:28.815690 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 43866cf0a1ad2e3e233de7f4af4359b0 for removal 2024.10.22 01:04:28.818538 [ 1476 ] {} CacheMetadata: Key 43866cf0a1ad2e3e233de7f4af4359b0 is removed from metadata 2024.10.22 01:04:28.921613 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 524405 2024.10.22 01:04:29.052979 [ 1211 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:04:29.094660 [ 1211 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:04:29.625010 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(9930dceb06dc4145541cd692df80172a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9930dceb06dc4145541cd692df80172a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:04:29.633038 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 9930dceb06dc4145541cd692df80172a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:04:29.639653 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:29.702462 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_330_112} MergeTask::MergeProjectionsStage: Merge sorted 25290 rows, containing 13 columns (13 merged, 0 gathered) in 1.968695925 sec., 12846.067124357967 rows/sec., 5.47 MiB/sec. 2024.10.22 01:04:29.739195 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_69_69_0, total 38 rows starting from the beginning of the part 2024.10.22 01:04:29.856293 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:04:29.857225 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_330_112} MergedBlockOutputStream: filled checksums 202410_1_330_112 (state Temporary) 2024.10.22 01:04:29.862204 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:04:30.068747 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_574_197} MergeTask::MergeProjectionsStage: Merge sorted 34109 rows, containing 13 columns (13 merged, 0 gathered) in 2.049282753 sec., 16644.360057228278 rows/sec., 2.58 MiB/sec. 2024.10.22 01:04:30.114784 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_574_197} MergedBlockOutputStream: filled checksums 202410_1_574_197 (state Temporary) 2024.10.22 01:04:30.277311 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(577c270e72ee281a2cc22080a75bd649) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 577c270e72ee281a2cc22080a75bd649, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:30.279920 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(577c270e72ee281a2cc22080a75bd649): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:04:30.284797 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_574_197} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_574_197 to 202410_1_574_197 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:30.287218 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_574_197} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_573_196, 202410_574_574_0] -> 202410_1_574_197 2024.10.22 01:04:30.424950 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:04:30.525410 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 5.38 MiB, will set to 1.68 GiB (RSS), difference: 49.87 MiB 2024.10.22 01:04:30.543092 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_330_112} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_330_112 to 202410_1_330_112 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:30.549299 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_330_112} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_329_111, 202410_330_330_0] -> 202410_1_330_112 2024.10.22 01:04:30.576916 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 58.63 MiB. 2024.10.22 01:04:30.913111 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_224_46, 202410_225_225_0, 202410_226_226_0, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0] 2024.10.22 01:04:30.916432 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_224_46, 202410_225_225_0, 202410_226_226_0, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0] 2024.10.22 01:04:31.035135 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(be61c8a14725ee9825a0396bcb642e41) : [0, 4649]: Setting download as failed: File segment: [0, 4649], key: be61c8a14725ee9825a0396bcb642e41, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:31.038451 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(be61c8a14725ee9825a0396bcb642e41): No space left in cache to reserve 4650 bytes, will continue without cache download 2024.10.22 01:04:31.050709 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_224_46, 202410_225_225_0, 202410_226_226_0, 202410_227_227_0, 202410_228_228_0, 202410_229_229_0] 2024.10.22 01:04:31.062065 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: be61c8a14725ee9825a0396bcb642e41, offset: 0, size: 0 2024.10.22 01:04:31.067516 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key be61c8a14725ee9825a0396bcb642e41 for removal 2024.10.22 01:04:31.072677 [ 1476 ] {} CacheMetadata: Key be61c8a14725ee9825a0396bcb642e41 is removed from metadata 2024.10.22 01:04:31.075423 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 577c270e72ee281a2cc22080a75bd649, offset: 0, size: 0 2024.10.22 01:04:31.078443 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 577c270e72ee281a2cc22080a75bd649 for removal 2024.10.22 01:04:31.082359 [ 1476 ] {} CacheMetadata: Key 577c270e72ee281a2cc22080a75bd649 is removed from metadata 2024.10.22 01:04:31.211487 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 4860 2024.10.22 01:04:31.432132 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:31.435477 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:31.486992 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_332_332_0 (state Temporary) 2024.10.22 01:04:31.632824 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_332_332_0 to 202410_345_345_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:31.677455 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4860 2024.10.22 01:04:31.746953 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d0542e0e3e1d657dff3e90380be6eb89) : [0, 156]: Setting download as failed: File segment: [0, 156], key: d0542e0e3e1d657dff3e90380be6eb89, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:31.748841 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d0542e0e3e1d657dff3e90380be6eb89): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:04:31.899464 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 33653 2024.10.22 01:04:32.128907 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:32.132389 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:32.202143 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_566_566_0 (state Temporary) 2024.10.22 01:04:32.250205 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_566_566_0 to 202410_579_579_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:32.283011 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_574_197 to 202410_575_575_0 2024.10.22 01:04:32.289513 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.20 MiB using storage policy from min volume index 0 2024.10.22 01:04:32.299738 [ 1201 ] {} DiskLocal: Reserved 1.20 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:32.311602 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_575_198} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_574_197 to 202410_575_575_0 into Compact with storage Full 2024.10.22 01:04:32.338330 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_575_198} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:32.352389 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33653 2024.10.22 01:04:32.354000 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_575_198} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_574_197, total 34109 rows starting from the beginning of the part 2024.10.22 01:04:32.393622 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e62958ceba851e31017fe45c601ff5a2) : [0, 74536]: Setting download as failed: File segment: [0, 74536], key: e62958ceba851e31017fe45c601ff5a2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:32.396141 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e62958ceba851e31017fe45c601ff5a2): No space left in cache to reserve 74537 bytes, will continue without cache download 2024.10.22 01:04:32.420123 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 3.64 MiB, will set to 1.67 GiB (RSS), difference: -7.61 MiB 2024.10.22 01:04:32.410822 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e62958ceba851e31017fe45c601ff5a2, offset: 0, size: 0 2024.10.22 01:04:32.433553 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e62958ceba851e31017fe45c601ff5a2 for removal 2024.10.22 01:04:32.436543 [ 1476 ] {} CacheMetadata: Key e62958ceba851e31017fe45c601ff5a2 is removed from metadata 2024.10.22 01:04:32.437522 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d0542e0e3e1d657dff3e90380be6eb89, offset: 0, size: 0 2024.10.22 01:04:32.439913 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d0542e0e3e1d657dff3e90380be6eb89 for removal 2024.10.22 01:04:32.443051 [ 1476 ] {} CacheMetadata: Key d0542e0e3e1d657dff3e90380be6eb89 is removed from metadata 2024.10.22 01:04:32.479940 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_575_198} MergeTreeSequentialSource: Reading 2 marks from part 202410_575_575_0, total 56 rows starting from the beginning of the part 2024.10.22 01:04:33.385429 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(39c96a671ba01217d7144416a1de5cdc) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 39c96a671ba01217d7144416a1de5cdc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:33.388247 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(39c96a671ba01217d7144416a1de5cdc): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:04:34.215177 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e3455a263fc0a983bef61d09d0a680ac) : [0, 83142]: Setting download as failed: File segment: [0, 83142], key: e3455a263fc0a983bef61d09d0a680ac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:34.220964 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e3455a263fc0a983bef61d09d0a680ac): No space left in cache to reserve 83143 bytes, will continue without cache download 2024.10.22 01:04:34.249440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e3455a263fc0a983bef61d09d0a680ac, offset: 0, size: 0 2024.10.22 01:04:34.265676 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e3455a263fc0a983bef61d09d0a680ac for removal 2024.10.22 01:04:34.274322 [ 1476 ] {} CacheMetadata: Key e3455a263fc0a983bef61d09d0a680ac is removed from metadata 2024.10.22 01:04:34.276476 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 39c96a671ba01217d7144416a1de5cdc, offset: 0, size: 0 2024.10.22 01:04:34.277523 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 39c96a671ba01217d7144416a1de5cdc for removal 2024.10.22 01:04:34.281094 [ 1476 ] {} CacheMetadata: Key 39c96a671ba01217d7144416a1de5cdc is removed from metadata 2024.10.22 01:04:34.487043 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_575_198} MergeTask::MergeProjectionsStage: Merge sorted 34165 rows, containing 13 columns (13 merged, 0 gathered) in 2.181500466 sec., 15661.238919029413 rows/sec., 2.42 MiB/sec. 2024.10.22 01:04:34.519164 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_575_198} MergedBlockOutputStream: filled checksums 202410_1_575_198 (state Temporary) 2024.10.22 01:04:34.839150 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_473_118, 202410_474_474_0] 2024.10.22 01:04:34.839261 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_575_198} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_575_198 to 202410_1_575_198 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:34.845269 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_473_118, 202410_474_474_0] 2024.10.22 01:04:34.848383 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_575_198} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_574_197, 202410_575_575_0] -> 202410_1_575_198 2024.10.22 01:04:34.858901 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_473_118, 202410_474_474_0] 2024.10.22 01:04:34.900786 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:04:35.279957 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 201 entries to flush up to offset 25916 2024.10.22 01:04:35.410769 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b96809a3ed62fee395510e62b34b53be) : [0, 167]: Setting download as failed: File segment: [0, 167], key: b96809a3ed62fee395510e62b34b53be, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:35.412066 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b96809a3ed62fee395510e62b34b53be): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:04:35.501969 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:35.503165 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:35.562856 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_329_329_0 (state Temporary) 2024.10.22 01:04:35.627157 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_329_329_0 to 202410_335_335_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:35.673071 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_330_112 to 202410_331_331_0 2024.10.22 01:04:35.676724 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 01:04:35.679134 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 25916 2024.10.22 01:04:35.685823 [ 1224 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:35.708082 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_331_113} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_330_112 to 202410_331_331_0 into Wide with storage Full 2024.10.22 01:04:35.778270 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_331_113} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:35.789082 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_331_113} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_330_112, total 25290 rows starting from the beginning of the part 2024.10.22 01:04:35.927562 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1764 entries to flush up to offset 526169 2024.10.22 01:04:36.144131 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_331_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_331_331_0, total 115 rows starting from the beginning of the part 2024.10.22 01:04:36.185090 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(63305d3db285e31ad417adc837266785) : [0, 132635]: Setting download as failed: File segment: [0, 132635], key: 63305d3db285e31ad417adc837266785, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:36.186005 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(63305d3db285e31ad417adc837266785): No space left in cache to reserve 132636 bytes, will continue without cache download 2024.10.22 01:04:36.198434 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 63305d3db285e31ad417adc837266785, offset: 0, size: 0 2024.10.22 01:04:36.222304 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 63305d3db285e31ad417adc837266785 for removal 2024.10.22 01:04:36.224651 [ 1476 ] {} CacheMetadata: Key 63305d3db285e31ad417adc837266785 is removed from metadata 2024.10.22 01:04:36.227719 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b96809a3ed62fee395510e62b34b53be, offset: 0, size: 0 2024.10.22 01:04:36.231822 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b96809a3ed62fee395510e62b34b53be for removal 2024.10.22 01:04:36.239179 [ 1476 ] {} CacheMetadata: Key b96809a3ed62fee395510e62b34b53be is removed from metadata 2024.10.22 01:04:36.372625 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 33717 2024.10.22 01:04:36.535468 [ 1236 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_273_56 to 202410_277_277_0 2024.10.22 01:04:36.553002 [ 1236 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:36.559477 [ 1236 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:36.591361 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:36.595256 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:36.606556 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_277_57} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_273_56 to 202410_277_277_0 into Compact with storage Full 2024.10.22 01:04:36.687213 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:36.688656 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:36.696736 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_318_318_0 (state Temporary) 2024.10.22 01:04:36.738717 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_567_567_0 (state Temporary) 2024.10.22 01:04:36.915091 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_277_57} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:36.959924 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_277_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_273_56, total 317 rows starting from the beginning of the part 2024.10.22 01:04:36.992108 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_567_567_0 to 202410_580_580_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:37.008520 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_575_198 to 202410_576_576_0 2024.10.22 01:04:37.012966 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.20 MiB using storage policy from min volume index 0 2024.10.22 01:04:37.015683 [ 1234 ] {} DiskLocal: Reserved 1.20 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:37.021368 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(396dab1d57f016134ccc655d96295d24) : [0, 167]: Setting download as failed: File segment: [0, 167], key: 396dab1d57f016134ccc655d96295d24, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:37.025284 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(396dab1d57f016134ccc655d96295d24): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:04:37.055799 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33717 2024.10.22 01:04:37.073089 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_318_318_0 to 202410_325_325_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:37.106482 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_277_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_274_274_0, total 1 rows starting from the beginning of the part 2024.10.22 01:04:37.109059 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 526169 2024.10.22 01:04:37.158128 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_576_199} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_575_198 to 202410_576_576_0 into Compact with storage Full 2024.10.22 01:04:37.203009 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_576_199} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:37.210385 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_277_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_275_275_0, total 1 rows starting from the beginning of the part 2024.10.22 01:04:37.210576 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_576_199} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_575_198, total 34165 rows starting from the beginning of the part 2024.10.22 01:04:37.281919 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 286 2024.10.22 01:04:37.290865 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_576_199} MergeTreeSequentialSource: Reading 2 marks from part 202410_576_576_0, total 76 rows starting from the beginning of the part 2024.10.22 01:04:37.320287 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_277_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_276_276_0, total 1 rows starting from the beginning of the part 2024.10.22 01:04:37.394092 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 3.39 MiB, will set to 1.66 GiB (RSS), difference: -58.89 MiB 2024.10.22 01:04:37.425154 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_277_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_277_277_0, total 1 rows starting from the beginning of the part 2024.10.22 01:04:37.536902 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:37.544156 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:37.714995 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_268_268_0 (state Temporary) 2024.10.22 01:04:37.800056 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_268_268_0 to 202410_278_278_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:37.859606 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 286 2024.10.22 01:04:37.893482 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3bf97af9d1412e73372e544048628754) : [0, 128577]: Setting download as failed: File segment: [0, 128577], key: 3bf97af9d1412e73372e544048628754, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:37.896803 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3bf97af9d1412e73372e544048628754): No space left in cache to reserve 128578 bytes, will continue without cache download 2024.10.22 01:04:38.014495 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3bf97af9d1412e73372e544048628754, offset: 0, size: 0 2024.10.22 01:04:38.017804 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3bf97af9d1412e73372e544048628754 for removal 2024.10.22 01:04:38.032455 [ 1476 ] {} CacheMetadata: Key 3bf97af9d1412e73372e544048628754 is removed from metadata 2024.10.22 01:04:38.034809 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 396dab1d57f016134ccc655d96295d24, offset: 0, size: 0 2024.10.22 01:04:38.036225 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 396dab1d57f016134ccc655d96295d24 for removal 2024.10.22 01:04:38.041652 [ 1476 ] {} CacheMetadata: Key 396dab1d57f016134ccc655d96295d24 is removed from metadata 2024.10.22 01:04:38.301827 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_331_113} MergeTask::MergeProjectionsStage: Merge sorted 25405 rows, containing 13 columns (13 merged, 0 gathered) in 2.60210921 sec., 9763.233573121244 rows/sec., 4.16 MiB/sec. 2024.10.22 01:04:38.341359 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_331_113} MergedBlockOutputStream: filled checksums 202410_1_331_113 (state Temporary) 2024.10.22 01:04:38.409726 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:38.413687 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:38.905759 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(97b7c4a5a8a32948e6c3a46c7623cab1) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 97b7c4a5a8a32948e6c3a46c7623cab1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:38.907877 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(97b7c4a5a8a32948e6c3a46c7623cab1): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:04:38.909337 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_474_119, 202410_475_475_0] 2024.10.22 01:04:38.913381 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_474_119, 202410_475_475_0] 2024.10.22 01:04:38.933840 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_474_119, 202410_475_475_0] 2024.10.22 01:04:38.962446 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_331_113} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_331_113 to 202410_1_331_113 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:38.965202 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_331_113} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_330_112, 202410_331_331_0] -> 202410_1_331_113 2024.10.22 01:04:38.984986 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 58.64 MiB. 2024.10.22 01:04:39.146507 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_281_56, 202410_282_282_0, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0, 202410_286_286_0] 2024.10.22 01:04:39.149027 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_281_56, 202410_282_282_0, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0, 202410_286_286_0] 2024.10.22 01:04:39.182890 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 4872 2024.10.22 01:04:39.291210 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:39.293493 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:39.377178 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_281_56, 202410_282_282_0, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0, 202410_286_286_0] 2024.10.22 01:04:39.432105 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_333_333_0 (state Temporary) 2024.10.22 01:04:39.491405 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_333_333_0 to 202410_346_346_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:39.500895 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_277_57} MergeTask::MergeProjectionsStage: Merge sorted 321 rows, containing 32 columns (32 merged, 0 gathered) in 2.911482976 sec., 110.2530918593975 rows/sec., 11.24 KiB/sec. 2024.10.22 01:04:39.524634 [ 1268 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_341_68 to 202410_346_346_0 2024.10.22 01:04:39.534469 [ 1268 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:39.539253 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:39.553321 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 3.29 MiB, will set to 1.67 GiB (RSS), difference: 51.80 MiB 2024.10.22 01:04:39.562797 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_341_68 to 202410_346_346_0 into Compact with storage Full 2024.10.22 01:04:39.629112 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:39.636585 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4872 2024.10.22 01:04:39.641483 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_341_68, total 4899 rows starting from the beginning of the part 2024.10.22 01:04:39.676932 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_277_57} MergedBlockOutputStream: filled checksums 202410_1_277_57 (state Temporary) 2024.10.22 01:04:39.801677 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(aa276f65de168973023402323d8d93fc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: aa276f65de168973023402323d8d93fc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:04:39.808393 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 923, file segment info: File segment: [0, 33554431], key: aa276f65de168973023402323d8d93fc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:04:39.810542 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:39.872025 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:04:39.875781 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:04:39.887412 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_342_342_0, total 19 rows starting from the beginning of the part 2024.10.22 01:04:39.892165 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_576_199} MergeTask::MergeProjectionsStage: Merge sorted 34241 rows, containing 13 columns (13 merged, 0 gathered) in 2.745097728 sec., 12473.508556996627 rows/sec., 1.93 MiB/sec. 2024.10.22 01:04:39.899466 [ 1674 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ylc/cwokgazznemmagmcvnfstpztdkyzx, size 2927169 2024.10.22 01:04:39.950752 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_576_199} MergedBlockOutputStream: filled checksums 202410_1_576_199 (state Temporary) 2024.10.22 01:04:39.955501 [ 1254 ] {} DatabaseCatalog: Execution took 205 ms. 2024.10.22 01:04:40.039299 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_343_343_0, total 13 rows starting from the beginning of the part 2024.10.22 01:04:40.076745 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_344_344_0, total 22 rows starting from the beginning of the part 2024.10.22 01:04:40.079840 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_576_199} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_576_199 to 202410_1_576_199 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:40.085550 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_576_199} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_575_198, 202410_576_576_0] -> 202410_1_576_199 2024.10.22 01:04:40.112849 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:04:40.126734 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_345_345_0, total 13 rows starting from the beginning of the part 2024.10.22 01:04:40.156515 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_346_346_0, total 12 rows starting from the beginning of the part 2024.10.22 01:04:40.228051 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:04:40.262247 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:04:40.287152 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:04:40.333335 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:04:40.463075 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:04:40.485961 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:04:40.501091 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:04:40.501963 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f921b65249a5a260332ac81e11c5f402) : [0, 62577]: Setting download as failed: File segment: [0, 62577], key: f921b65249a5a260332ac81e11c5f402, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:40.507438 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f921b65249a5a260332ac81e11c5f402): No space left in cache to reserve 62578 bytes, will continue without cache download 2024.10.22 01:04:40.531253 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f921b65249a5a260332ac81e11c5f402, offset: 0, size: 0 2024.10.22 01:04:40.564532 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f921b65249a5a260332ac81e11c5f402 for removal 2024.10.22 01:04:40.570951 [ 1476 ] {} CacheMetadata: Key f921b65249a5a260332ac81e11c5f402 is removed from metadata 2024.10.22 01:04:40.571845 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 97b7c4a5a8a32948e6c3a46c7623cab1, offset: 0, size: 0 2024.10.22 01:04:40.576538 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 97b7c4a5a8a32948e6c3a46c7623cab1 for removal 2024.10.22 01:04:40.577477 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wwb/reaojufrkktpdvfcuxvrurvbmpkfw (LogSeriesLimiter: on interval from 2024-10-22 01:04:24 to 2024-10-22 01:04:40 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:04:40.580242 [ 1476 ] {} CacheMetadata: Key 97b7c4a5a8a32948e6c3a46c7623cab1 is removed from metadata 2024.10.22 01:04:40.788602 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_277_57} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_277_57 to 202410_1_277_57 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:40.793037 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_277_57} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_273_56, 202410_277_277_0] -> 202410_1_277_57 2024.10.22 01:04:40.816368 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:04:40.866663 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_69_14/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:04:41.075368 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 112 entries to flush up to offset 33829 2024.10.22 01:04:41.270616 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} MergeTask::MergeProjectionsStage: Merge sorted 4978 rows, containing 12 columns (12 merged, 0 gathered) in 1.717174648 sec., 2898.947993320246 rows/sec., 456.17 KiB/sec. 2024.10.22 01:04:41.332268 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} MergedBlockOutputStream: filled checksums 202410_1_346_69 (state Temporary) 2024.10.22 01:04:41.446033 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:41.450514 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:41.449842 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_272_74, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0, 202410_277_277_0] 2024.10.22 01:04:41.454545 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_272_74, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0, 202410_277_277_0] 2024.10.22 01:04:41.488388 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_272_74, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0, 202410_277_277_0] 2024.10.22 01:04:41.501101 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_67_67_0 (state Temporary) 2024.10.22 01:04:41.510929 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(374917579b4f98206675bc2f317993c9) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 374917579b4f98206675bc2f317993c9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:41.512674 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(374917579b4f98206675bc2f317993c9): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:04:41.521502 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_346_69 to 202410_1_346_69 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:41.527752 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_346_69} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_341_68, 202410_346_346_0] -> 202410_1_346_69 2024.10.22 01:04:41.678286 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.42 MiB. 2024.10.22 01:04:41.727598 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_568_568_0 (state Temporary) 2024.10.22 01:04:41.776427 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_568_568_0 to 202410_581_581_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:41.794980 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_576_199 to 202410_577_577_0 2024.10.22 01:04:41.799309 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.20 MiB using storage policy from min volume index 0 2024.10.22 01:04:41.800716 [ 1179 ] {} DiskLocal: Reserved 1.20 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:41.850494 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_577_200} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_576_199 to 202410_577_577_0 into Compact with storage Full 2024.10.22 01:04:41.886326 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_577_200} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:41.889735 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_577_200} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_576_199, total 34241 rows starting from the beginning of the part 2024.10.22 01:04:41.896251 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_577_200} MergeTreeSequentialSource: Reading 2 marks from part 202410_577_577_0, total 81 rows starting from the beginning of the part 2024.10.22 01:04:42.017023 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33829 2024.10.22 01:04:42.247747 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(4f352059417ed946cab52c91559f0627) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4f352059417ed946cab52c91559f0627, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:04:42.255926 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 996827, file segment info: File segment: [0, 33554431], key: 4f352059417ed946cab52c91559f0627, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:04:42.260900 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:42.599106 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 4.89 MiB, will set to 1.67 GiB (RSS), difference: -20.42 MiB 2024.10.22 01:04:42.841448 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(53b3696e7536f5cbd93f47a9606db98b) : [0, 60157]: Setting download as failed: File segment: [0, 60157], key: 53b3696e7536f5cbd93f47a9606db98b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:42.847184 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(53b3696e7536f5cbd93f47a9606db98b): No space left in cache to reserve 60158 bytes, will continue without cache download 2024.10.22 01:04:42.967901 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 53b3696e7536f5cbd93f47a9606db98b, offset: 0, size: 0 2024.10.22 01:04:42.971973 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 53b3696e7536f5cbd93f47a9606db98b for removal 2024.10.22 01:04:42.975322 [ 1476 ] {} CacheMetadata: Key 53b3696e7536f5cbd93f47a9606db98b is removed from metadata 2024.10.22 01:04:42.976830 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 374917579b4f98206675bc2f317993c9, offset: 0, size: 0 2024.10.22 01:04:42.978596 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 374917579b4f98206675bc2f317993c9 for removal 2024.10.22 01:04:42.980786 [ 1476 ] {} CacheMetadata: Key 374917579b4f98206675bc2f317993c9 is removed from metadata 2024.10.22 01:04:43.029343 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_261_52, 202410_262_262_0, 202410_263_263_0, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0] 2024.10.22 01:04:43.043952 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_261_52, 202410_262_262_0, 202410_263_263_0, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0] 2024.10.22 01:04:43.276689 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_261_52, 202410_262_262_0, 202410_263_263_0, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0] 2024.10.22 01:04:43.302373 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 249 entries to flush up to offset 26165 2024.10.22 01:04:43.588043 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(2a655a65a8c48b1b186363561eb204ea) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2a655a65a8c48b1b186363561eb204ea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:04:43.590874 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1214, file segment info: File segment: [0, 33554431], key: 2a655a65a8c48b1b186363561eb204ea, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:04:43.593309 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:43.725428 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_475_120, 202410_476_476_0] 2024.10.22 01:04:43.729069 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_475_120, 202410_476_476_0] 2024.10.22 01:04:43.742542 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_475_120, 202410_476_476_0] 2024.10.22 01:04:43.963414 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:43.968351 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:44.101811 [ 1182 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:04:44.117255 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1092 entries to flush up to offset 527261 2024.10.22 01:04:44.146072 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_330_330_0 (state Temporary) 2024.10.22 01:04:44.152617 [ 1182 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:04:44.217017 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_330_330_0 to 202410_336_336_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:44.348567 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_577_200} MergeTask::MergeProjectionsStage: Merge sorted 34322 rows, containing 13 columns (13 merged, 0 gathered) in 2.502376191 sec., 13715.763490494302 rows/sec., 2.12 MiB/sec. 2024.10.22 01:04:44.379460 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_1_331_113 to 202410_336_336_0 2024.10.22 01:04:44.384528 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 26165 2024.10.22 01:04:44.394951 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.43 MiB using storage policy from min volume index 0 2024.10.22 01:04:44.398899 [ 1191 ] {} DiskLocal: Reserved 1.43 MiB on local disk `default`, having unreserved 297.55 GiB. 2024.10.22 01:04:44.435773 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_331_113 to 202410_336_336_0 into Wide with storage Full 2024.10.22 01:04:44.488436 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:44.489301 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_577_200} MergedBlockOutputStream: filled checksums 202410_1_577_200 (state Temporary) 2024.10.22 01:04:44.493623 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_331_113, total 25405 rows starting from the beginning of the part 2024.10.22 01:04:44.643893 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_332_332_0, total 114 rows starting from the beginning of the part 2024.10.22 01:04:44.677419 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_333_333_0, total 85 rows starting from the beginning of the part 2024.10.22 01:04:44.768371 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c4f7a89a57671c03119a830d4af132e3) : [0, 155]: Setting download as failed: File segment: [0, 155], key: c4f7a89a57671c03119a830d4af132e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:44.771660 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c4f7a89a57671c03119a830d4af132e3): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:04:44.800757 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:44.808044 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:44.821228 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_334_334_0, total 215 rows starting from the beginning of the part 2024.10.22 01:04:44.857717 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_335_335_0, total 201 rows starting from the beginning of the part 2024.10.22 01:04:44.892644 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_336_336_0, total 249 rows starting from the beginning of the part 2024.10.22 01:04:44.960054 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_319_319_0 (state Temporary) 2024.10.22 01:04:45.035911 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_577_200} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_577_200 to 202410_1_577_200 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:45.040768 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_577_200} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_576_199, 202410_577_577_0] -> 202410_1_577_200 2024.10.22 01:04:45.067012 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_319_319_0 to 202410_326_326_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:45.091350 [ 1193 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_321_64 to 202410_326_326_0 2024.10.22 01:04:45.097763 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 527261 2024.10.22 01:04:45.108302 [ 1193 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:45.110680 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:45.144517 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_321_64 to 202410_326_326_0 into Compact with storage Full 2024.10.22 01:04:45.163619 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:45.175537 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} MergeTreeSequentialSource: Reading 67 marks from part 202410_1_321_64, total 534428 rows starting from the beginning of the part 2024.10.22 01:04:45.184317 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_322_322_0, total 1576 rows starting from the beginning of the part 2024.10.22 01:04:45.195589 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:04:45.198733 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_323_323_0, total 768 rows starting from the beginning of the part 2024.10.22 01:04:45.309390 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 4.00 MiB, will set to 1.68 GiB (RSS), difference: -16.03 MiB 2024.10.22 01:04:45.311353 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_324_324_0, total 1223 rows starting from the beginning of the part 2024.10.22 01:04:45.328473 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_325_325_0, total 1764 rows starting from the beginning of the part 2024.10.22 01:04:45.343985 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_326_326_0, total 1092 rows starting from the beginning of the part 2024.10.22 01:04:45.365127 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 287 2024.10.22 01:04:45.516537 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:45.523449 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:45.529158 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(9728596eb88f5a798d1bc0698f21cbbe) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9728596eb88f5a798d1bc0698f21cbbe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:04:45.534088 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 963962, file segment info: File segment: [0, 33554431], key: 9728596eb88f5a798d1bc0698f21cbbe, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:04:45.540622 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:45.693805 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_269_269_0 (state Temporary) 2024.10.22 01:04:45.865218 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_269_269_0 to 202410_279_279_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:45.946524 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 287 2024.10.22 01:04:46.143305 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 98 entries to flush up to offset 33927 2024.10.22 01:04:46.210401 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(766c77dabb4e07a445cd624995abd9a9) : [0, 34999]: Setting download as failed: File segment: [0, 34999], key: 766c77dabb4e07a445cd624995abd9a9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:46.211781 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(766c77dabb4e07a445cd624995abd9a9): No space left in cache to reserve 35000 bytes, will continue without cache download 2024.10.22 01:04:46.241740 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 766c77dabb4e07a445cd624995abd9a9, offset: 0, size: 0 2024.10.22 01:04:46.245402 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 766c77dabb4e07a445cd624995abd9a9 for removal 2024.10.22 01:04:46.254408 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c4f7a89a57671c03119a830d4af132e3, offset: 0, size: 0 2024.10.22 01:04:46.257831 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c4f7a89a57671c03119a830d4af132e3 for removal 2024.10.22 01:04:46.265768 [ 1476 ] {} CacheMetadata: Key 766c77dabb4e07a445cd624995abd9a9 is removed from metadata 2024.10.22 01:04:46.272056 [ 1476 ] {} CacheMetadata: Key c4f7a89a57671c03119a830d4af132e3 is removed from metadata 2024.10.22 01:04:46.781831 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:46.784411 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:46.992764 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_569_569_0 (state Temporary) 2024.10.22 01:04:47.141225 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_569_569_0 to 202410_582_582_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:47.145258 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 24 entries to flush up to offset 4896 2024.10.22 01:04:47.177604 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(dd24ee55601a9242a4339a2931da681c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: dd24ee55601a9242a4339a2931da681c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:04:47.181092 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 940, file segment info: File segment: [0, 33554431], key: dd24ee55601a9242a4339a2931da681c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:04:47.183496 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:47.279604 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 33927 2024.10.22 01:04:47.284855 [ 1150 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 201 ms. 2024.10.22 01:04:47.302498 [ 1274 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_577_200 to 202410_578_578_0 2024.10.22 01:04:47.306567 [ 1274 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.21 MiB using storage policy from min volume index 0 2024.10.22 01:04:47.310318 [ 1274 ] {} DiskLocal: Reserved 1.21 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:47.330775 [ 1696 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nsh/nsufyjdwjbzxsnxzfdbfeetejumbo, size 963962 2024.10.22 01:04:47.408102 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:47.411028 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:47.477340 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_578_201} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_577_200 to 202410_578_578_0 into Compact with storage Full 2024.10.22 01:04:47.519797 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_578_201} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:47.534724 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_578_201} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_577_200, total 34322 rows starting from the beginning of the part 2024.10.22 01:04:47.589415 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_578_201} MergeTreeSequentialSource: Reading 2 marks from part 202410_578_578_0, total 78 rows starting from the beginning of the part 2024.10.22 01:04:47.624337 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:04:47.633835 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:04:47.653177 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:04:47.685486 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:04:47.700663 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_334_334_0 (state Temporary) 2024.10.22 01:04:47.734891 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:04:47.789593 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:04:47.780715 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} MergeTask::MergeProjectionsStage: Merge sorted 26269 rows, containing 13 columns (13 merged, 0 gathered) in 3.35369513 sec., 7832.852713717003 rows/sec., 3.34 MiB/sec. 2024.10.22 01:04:47.806100 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:04:47.948021 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:04:47.953658 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} MergedBlockOutputStream: filled checksums 202410_1_336_114 (state Temporary) 2024.10.22 01:04:47.996685 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:04:48.010763 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(40af44377e7fc8f339d0d6d8b71d92a0) : [0, 167]: Setting download as failed: File segment: [0, 167], key: 40af44377e7fc8f339d0d6d8b71d92a0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:48.012152 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(40af44377e7fc8f339d0d6d8b71d92a0): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:04:48.084151 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nxa/juvbfuwgugjeefcfufvwxlojxzfbn (LogSeriesLimiter: on interval from 2024-10-22 01:04:40 to 2024-10-22 01:04:48 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:04:48.134751 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_334_334_0 to 202410_347_347_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:48.290467 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4896 2024.10.22 01:04:48.412009 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 8.78 MiB, will set to 1.69 GiB (RSS), difference: 16.00 MiB 2024.10.22 01:04:48.413463 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_336_114 to 202410_1_336_114 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:48.419220 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_336_114} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_1_331_113, 202410_336_336_0] -> 202410_1_336_114 2024.10.22 01:04:48.471639 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.51 MiB. 2024.10.22 01:04:48.995745 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_476_121, 202410_477_477_0] 2024.10.22 01:04:48.997113 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_476_121, 202410_477_477_0] 2024.10.22 01:04:49.004028 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_476_121, 202410_477_477_0] 2024.10.22 01:04:49.432431 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b8c550a028be73cac5d0cce65bd985bf) : [0, 126645]: Setting download as failed: File segment: [0, 126645], key: b8c550a028be73cac5d0cce65bd985bf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:49.435022 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b8c550a028be73cac5d0cce65bd985bf): No space left in cache to reserve 126646 bytes, will continue without cache download 2024.10.22 01:04:49.464413 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b8c550a028be73cac5d0cce65bd985bf, offset: 0, size: 0 2024.10.22 01:04:49.484824 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b8c550a028be73cac5d0cce65bd985bf for removal 2024.10.22 01:04:49.487110 [ 1476 ] {} CacheMetadata: Key b8c550a028be73cac5d0cce65bd985bf is removed from metadata 2024.10.22 01:04:49.489443 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 40af44377e7fc8f339d0d6d8b71d92a0, offset: 0, size: 0 2024.10.22 01:04:49.491017 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 40af44377e7fc8f339d0d6d8b71d92a0 for removal 2024.10.22 01:04:49.499621 [ 1476 ] {} CacheMetadata: Key 40af44377e7fc8f339d0d6d8b71d92a0 is removed from metadata 2024.10.22 01:04:49.618861 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_578_201} MergeTask::MergeProjectionsStage: Merge sorted 34400 rows, containing 13 columns (13 merged, 0 gathered) in 2.256916127 sec., 15242.037392734708 rows/sec., 2.36 MiB/sec. 2024.10.22 01:04:49.694893 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_578_201} MergedBlockOutputStream: filled checksums 202410_1_578_201 (state Temporary) 2024.10.22 01:04:49.731682 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_578_201} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_578_201 to 202410_1_578_201 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:49.836229 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_578_201} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_577_200, 202410_578_578_0] -> 202410_1_578_201 2024.10.22 01:04:49.865366 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:04:49.869200 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:04:49.878764 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:04:50.209310 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(5018533a4c234aa98556e244acf71b11) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5018533a4c234aa98556e244acf71b11, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:04:50.215394 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 5018533a4c234aa98556e244acf71b11, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:04:50.220834 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:51.010347 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b2f4014f876ea9aec880d08def81ecb6) : [0, 155]: Setting download as failed: File segment: [0, 155], key: b2f4014f876ea9aec880d08def81ecb6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:51.012104 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b2f4014f876ea9aec880d08def81ecb6): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:04:51.207747 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_326_65/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:04:51.288987 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 34001 2024.10.22 01:04:51.745264 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_67_67_0 to 202410_70_70_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:51.765697 [ 1233 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 01:04:51.807519 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:51.811650 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:51.995361 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_570_570_0 (state Temporary) 2024.10.22 01:04:52.006873 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 216 entries to flush up to offset 26381 2024.10.22 01:04:52.019653 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b086a5c454f4dbcbfd1348cabcbbe34a) : [0, 2632]: Setting download as failed: File segment: [0, 2632], key: b086a5c454f4dbcbfd1348cabcbbe34a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:52.022847 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b086a5c454f4dbcbfd1348cabcbbe34a): No space left in cache to reserve 2633 bytes, will continue without cache download 2024.10.22 01:04:52.041326 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b086a5c454f4dbcbfd1348cabcbbe34a, offset: 0, size: 0 2024.10.22 01:04:52.076355 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b086a5c454f4dbcbfd1348cabcbbe34a for removal 2024.10.22 01:04:52.081812 [ 1476 ] {} CacheMetadata: Key b086a5c454f4dbcbfd1348cabcbbe34a is removed from metadata 2024.10.22 01:04:52.082448 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b2f4014f876ea9aec880d08def81ecb6, offset: 0, size: 0 2024.10.22 01:04:52.087685 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b2f4014f876ea9aec880d08def81ecb6 for removal 2024.10.22 01:04:52.092738 [ 1476 ] {} CacheMetadata: Key b2f4014f876ea9aec880d08def81ecb6 is removed from metadata 2024.10.22 01:04:52.101584 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 913 entries to flush up to offset 528174 2024.10.22 01:04:52.161686 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_570_570_0 to 202410_583_583_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:52.187037 [ 1212 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_578_201 to 202410_579_579_0 2024.10.22 01:04:52.201108 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34001 2024.10.22 01:04:52.201570 [ 1212 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.21 MiB using storage policy from min volume index 0 2024.10.22 01:04:52.203363 [ 1212 ] {} DiskLocal: Reserved 1.21 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:52.213213 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_579_202} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_578_201 to 202410_579_579_0 into Compact with storage Full 2024.10.22 01:04:52.275838 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_579_202} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:52.289774 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_579_202} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_578_201, total 34400 rows starting from the beginning of the part 2024.10.22 01:04:52.312204 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_579_202} MergeTreeSequentialSource: Reading 2 marks from part 202410_579_579_0, total 84 rows starting from the beginning of the part 2024.10.22 01:04:52.346667 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:52.351176 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:52.369122 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 12.79 MiB, will set to 1.69 GiB (RSS), difference: 16.28 MiB 2024.10.22 01:04:52.402841 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_477_122, 202410_478_478_0] 2024.10.22 01:04:52.409312 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_477_122, 202410_478_478_0] 2024.10.22 01:04:52.418263 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_477_122, 202410_478_478_0] 2024.10.22 01:04:52.535141 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_331_331_0 (state Temporary) 2024.10.22 01:04:52.622699 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_331_331_0 to 202410_337_337_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:52.663758 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 26381 2024.10.22 01:04:53.083193 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5d2ad4356afcb9bcb0ff792446a6c42d) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 5d2ad4356afcb9bcb0ff792446a6c42d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:53.085358 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5d2ad4356afcb9bcb0ff792446a6c42d): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:04:53.138046 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:53.146427 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:53.347126 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_320_320_0 (state Temporary) 2024.10.22 01:04:53.390880 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_69_14/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:04:53.452398 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 288 2024.10.22 01:04:53.455387 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_320_320_0 to 202410_327_327_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:53.599189 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 528174 2024.10.22 01:04:53.740608 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:53.743977 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:53.816097 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_270_270_0 (state Temporary) 2024.10.22 01:04:54.148542 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_270_270_0 to 202410_280_280_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:54.243121 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a8386155cb308f51a2fb72c9824704fc) : [0, 5597]: Setting download as failed: File segment: [0, 5597], key: a8386155cb308f51a2fb72c9824704fc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:54.244832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a8386155cb308f51a2fb72c9824704fc): No space left in cache to reserve 5598 bytes, will continue without cache download 2024.10.22 01:04:54.256774 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a8386155cb308f51a2fb72c9824704fc, offset: 0, size: 0 2024.10.22 01:04:54.276001 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a8386155cb308f51a2fb72c9824704fc for removal 2024.10.22 01:04:54.278398 [ 1476 ] {} CacheMetadata: Key a8386155cb308f51a2fb72c9824704fc is removed from metadata 2024.10.22 01:04:54.279534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5d2ad4356afcb9bcb0ff792446a6c42d, offset: 0, size: 0 2024.10.22 01:04:54.280928 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5d2ad4356afcb9bcb0ff792446a6c42d for removal 2024.10.22 01:04:54.284216 [ 1476 ] {} CacheMetadata: Key 5d2ad4356afcb9bcb0ff792446a6c42d is removed from metadata 2024.10.22 01:04:54.346433 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 288 2024.10.22 01:04:54.439911 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_579_202} MergeTask::MergeProjectionsStage: Merge sorted 34484 rows, containing 13 columns (13 merged, 0 gathered) in 2.230778146 sec., 15458.283048824524 rows/sec., 2.40 MiB/sec. 2024.10.22 01:04:54.512365 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2682 2024.10.22 01:04:54.594278 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_579_202} MergedBlockOutputStream: filled checksums 202410_1_579_202 (state Temporary) 2024.10.22 01:04:54.646845 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 12.69 MiB, will set to 1.69 GiB (RSS), difference: -2.35 MiB 2024.10.22 01:04:54.953361 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_579_202} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_579_202 to 202410_1_579_202 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:54.960531 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_579_202} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_578_201, 202410_579_579_0] -> 202410_1_579_202 2024.10.22 01:04:55.021599 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3f4608a9e34bbb2dab9ba86d942a3d59) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 3f4608a9e34bbb2dab9ba86d942a3d59, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:55.025996 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3f4608a9e34bbb2dab9ba86d942a3d59): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:04:55.094998 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:04:55.638928 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3d3f1b47f02fac3e7b3e14dc276e5470) : [0, 33569]: Setting download as failed: File segment: [0, 33569], key: 3d3f1b47f02fac3e7b3e14dc276e5470, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:55.639929 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3d3f1b47f02fac3e7b3e14dc276e5470): No space left in cache to reserve 33570 bytes, will continue without cache download 2024.10.22 01:04:55.652465 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3d3f1b47f02fac3e7b3e14dc276e5470, offset: 0, size: 0 2024.10.22 01:04:55.658468 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3d3f1b47f02fac3e7b3e14dc276e5470 for removal 2024.10.22 01:04:55.663757 [ 1476 ] {} CacheMetadata: Key 3d3f1b47f02fac3e7b3e14dc276e5470 is removed from metadata 2024.10.22 01:04:55.665342 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3f4608a9e34bbb2dab9ba86d942a3d59, offset: 0, size: 0 2024.10.22 01:04:55.667828 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3f4608a9e34bbb2dab9ba86d942a3d59 for removal 2024.10.22 01:04:55.672919 [ 1476 ] {} CacheMetadata: Key 3f4608a9e34bbb2dab9ba86d942a3d59 is removed from metadata 2024.10.22 01:04:55.793014 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 4916 2024.10.22 01:04:55.910348 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:55.914664 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:55.962462 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_335_335_0 (state Temporary) 2024.10.22 01:04:56.008118 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_335_335_0 to 202410_348_348_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:56.051565 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4916 2024.10.22 01:04:56.215169 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 34080 2024.10.22 01:04:56.412768 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 14.16 MiB, will set to 1.69 GiB (RSS), difference: 7.53 MiB 2024.10.22 01:04:56.489057 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8b263c24074492d61cae60f9872d417c) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 8b263c24074492d61cae60f9872d417c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:56.490345 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8b263c24074492d61cae60f9872d417c): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:04:56.904163 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:04:56.905093 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:56.957408 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_571_571_0 (state Temporary) 2024.10.22 01:04:57.055916 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bb0f0b562a4ce8f38ef4f1e1b7a9e77c) : [0, 31147]: Setting download as failed: File segment: [0, 31147], key: bb0f0b562a4ce8f38ef4f1e1b7a9e77c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:57.057777 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bb0f0b562a4ce8f38ef4f1e1b7a9e77c): No space left in cache to reserve 31148 bytes, will continue without cache download 2024.10.22 01:04:57.077210 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bb0f0b562a4ce8f38ef4f1e1b7a9e77c, offset: 0, size: 0 2024.10.22 01:04:57.082293 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bb0f0b562a4ce8f38ef4f1e1b7a9e77c for removal 2024.10.22 01:04:57.086670 [ 1476 ] {} CacheMetadata: Key bb0f0b562a4ce8f38ef4f1e1b7a9e77c is removed from metadata 2024.10.22 01:04:57.088768 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8b263c24074492d61cae60f9872d417c, offset: 0, size: 0 2024.10.22 01:04:57.090202 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8b263c24074492d61cae60f9872d417c for removal 2024.10.22 01:04:57.094819 [ 1476 ] {} CacheMetadata: Key 8b263c24074492d61cae60f9872d417c is removed from metadata 2024.10.22 01:04:57.119466 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_571_571_0 to 202410_584_584_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:04:57.249812 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_579_202 to 202410_580_580_0 2024.10.22 01:04:57.256745 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.21 MiB using storage policy from min volume index 0 2024.10.22 01:04:57.263231 [ 1255 ] {} DiskLocal: Reserved 1.21 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:04:57.275543 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34080 2024.10.22 01:04:57.288721 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_580_203} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_579_202 to 202410_580_580_0 into Compact with storage Full 2024.10.22 01:04:57.480157 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_580_203} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:04:57.499412 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_580_203} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_579_202, total 34484 rows starting from the beginning of the part 2024.10.22 01:04:57.683030 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_580_203} MergeTreeSequentialSource: Reading 2 marks from part 202410_580_580_0, total 64 rows starting from the beginning of the part 2024.10.22 01:04:57.706539 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5a284279751e5a1b7c20dd26b841082c) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 5a284279751e5a1b7c20dd26b841082c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:57.708432 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5a284279751e5a1b7c20dd26b841082c): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:04:58.085340 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_478_123, 202410_479_479_0] 2024.10.22 01:04:58.193024 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_478_123, 202410_479_479_0] 2024.10.22 01:04:58.210159 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_478_123, 202410_479_479_0] 2024.10.22 01:04:58.458338 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(839954c081f725bd89f0851fa186aa11) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 839954c081f725bd89f0851fa186aa11, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:04:58.463291 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 903, file segment info: File segment: [0, 33554431], key: 839954c081f725bd89f0851fa186aa11, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:04:58.466448 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:04:58.479459 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 13.83 MiB, will set to 1.69 GiB (RSS), difference: -6.83 MiB 2024.10.22 01:04:58.611679 [ 1694 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wwb/reaojufrkktpdvfcuxvrurvbmpkfw, size 1958583 2024.10.22 01:04:58.691113 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:04:58.700863 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:04:58.709465 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:04:58.932677 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:04:58.948883 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:04:58.977278 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(983743c1f1505bb18a6e3c07ec50b6a9) : [0, 212211]: Setting download as failed: File segment: [0, 212211], key: 983743c1f1505bb18a6e3c07ec50b6a9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:58.978534 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:04:58.981587 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(983743c1f1505bb18a6e3c07ec50b6a9): No space left in cache to reserve 212212 bytes, will continue without cache download 2024.10.22 01:04:58.999070 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:04:59.135310 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 983743c1f1505bb18a6e3c07ec50b6a9, offset: 0, size: 0 2024.10.22 01:04:59.140015 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 983743c1f1505bb18a6e3c07ec50b6a9 for removal 2024.10.22 01:04:59.143245 [ 1476 ] {} CacheMetadata: Key 983743c1f1505bb18a6e3c07ec50b6a9 is removed from metadata 2024.10.22 01:04:59.144160 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5a284279751e5a1b7c20dd26b841082c, offset: 0, size: 0 2024.10.22 01:04:59.147218 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5a284279751e5a1b7c20dd26b841082c for removal 2024.10.22 01:04:59.149801 [ 1476 ] {} CacheMetadata: Key 5a284279751e5a1b7c20dd26b841082c is removed from metadata 2024.10.22 01:04:59.160878 [ 1264 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:04:59.163159 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ney/acwkmcgvicbbuxcqtbjdgtyuzupcr (LogSeriesLimiter: on interval from 2024-10-22 01:04:48 to 2024-10-22 01:04:59 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:04:59.182021 [ 1264 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:04:59.839600 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(474a0b7b4e396614cea3d0ffe77c40ea) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 474a0b7b4e396614cea3d0ffe77c40ea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:04:59.841270 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(474a0b7b4e396614cea3d0ffe77c40ea): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:04:59.871012 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:04:59.968130 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:05:00.195099 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 110 entries to flush up to offset 26491 2024.10.22 01:05:00.340455 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_580_203} MergeTask::MergeProjectionsStage: Merge sorted 34548 rows, containing 13 columns (13 merged, 0 gathered) in 3.055561269 sec., 11306.597040126313 rows/sec., 1.75 MiB/sec. 2024.10.22 01:05:00.411726 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:00.414694 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:00.479713 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_332_332_0 (state Temporary) 2024.10.22 01:05:00.506282 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_580_203} MergedBlockOutputStream: filled checksums 202410_1_580_203 (state Temporary) 2024.10.22 01:05:00.601809 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 529710 2024.10.22 01:05:00.881308 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_332_332_0 to 202410_338_338_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:00.917570 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_580_203} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_580_203 to 202410_1_580_203 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:00.927818 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_580_203} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_579_202, 202410_580_580_0] -> 202410_1_580_203 2024.10.22 01:05:00.932695 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 26491 2024.10.22 01:05:00.955759 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:05:00.992117 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(efef707ee29147c069d0f9544a71f23b) : [0, 141937]: Setting download as failed: File segment: [0, 141937], key: efef707ee29147c069d0f9544a71f23b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:00.993729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(efef707ee29147c069d0f9544a71f23b): No space left in cache to reserve 141938 bytes, will continue without cache download 2024.10.22 01:05:01.013255 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: efef707ee29147c069d0f9544a71f23b, offset: 0, size: 0 2024.10.22 01:05:01.016440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key efef707ee29147c069d0f9544a71f23b for removal 2024.10.22 01:05:01.019965 [ 1476 ] {} CacheMetadata: Key efef707ee29147c069d0f9544a71f23b is removed from metadata 2024.10.22 01:05:01.026468 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 474a0b7b4e396614cea3d0ffe77c40ea, offset: 0, size: 0 2024.10.22 01:05:01.027847 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 474a0b7b4e396614cea3d0ffe77c40ea for removal 2024.10.22 01:05:01.032023 [ 1476 ] {} CacheMetadata: Key 474a0b7b4e396614cea3d0ffe77c40ea is removed from metadata 2024.10.22 01:05:01.063568 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:01.071924 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:01.185493 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_321_321_0 (state Temporary) 2024.10.22 01:05:01.280403 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_321_321_0 to 202410_328_328_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:01.285133 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 34159 2024.10.22 01:05:01.304859 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 529710 2024.10.22 01:05:01.510417 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:01.513529 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:01.614752 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(edd7217d21aedeee8fbb342995fdce4d) : [0, 167]: Setting download as failed: File segment: [0, 167], key: edd7217d21aedeee8fbb342995fdce4d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:01.617484 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(edd7217d21aedeee8fbb342995fdce4d): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:05:01.663722 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_572_572_0 (state Temporary) 2024.10.22 01:05:01.805357 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_572_572_0 to 202410_585_585_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:01.849996 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 289 2024.10.22 01:05:01.887808 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34159 2024.10.22 01:05:01.980813 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_580_203 to 202410_581_581_0 2024.10.22 01:05:01.989630 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.22 MiB using storage policy from min volume index 0 2024.10.22 01:05:01.992439 [ 1168 ] {} DiskLocal: Reserved 1.22 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:02.003966 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_581_204} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_580_203 to 202410_581_581_0 into Compact with storage Full 2024.10.22 01:05:02.028757 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_581_204} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:02.038762 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_581_204} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_580_203, total 34548 rows starting from the beginning of the part 2024.10.22 01:05:02.062652 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_581_204} MergeTreeSequentialSource: Reading 2 marks from part 202410_581_581_0, total 112 rows starting from the beginning of the part 2024.10.22 01:05:02.067600 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 2721 2024.10.22 01:05:02.211678 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_479_124, 202410_480_480_0] 2024.10.22 01:05:02.214980 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_479_124, 202410_480_480_0] 2024.10.22 01:05:02.226793 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_479_124, 202410_480_480_0] 2024.10.22 01:05:02.349090 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:02.353552 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:02.403290 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(95794aaf867ba4490ce35f50b42383da) : [0, 92070]: Setting download as failed: File segment: [0, 92070], key: 95794aaf867ba4490ce35f50b42383da, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:02.406218 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(95794aaf867ba4490ce35f50b42383da): No space left in cache to reserve 92071 bytes, will continue without cache download 2024.10.22 01:05:02.524429 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_271_271_0 (state Temporary) 2024.10.22 01:05:02.544550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 95794aaf867ba4490ce35f50b42383da, offset: 0, size: 0 2024.10.22 01:05:02.563939 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 95794aaf867ba4490ce35f50b42383da for removal 2024.10.22 01:05:02.576401 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: edd7217d21aedeee8fbb342995fdce4d, offset: 0, size: 0 2024.10.22 01:05:02.581062 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key edd7217d21aedeee8fbb342995fdce4d for removal 2024.10.22 01:05:02.670896 [ 1476 ] {} CacheMetadata: Key 95794aaf867ba4490ce35f50b42383da is removed from metadata 2024.10.22 01:05:02.679550 [ 1476 ] {} CacheMetadata: Key edd7217d21aedeee8fbb342995fdce4d is removed from metadata 2024.10.22 01:05:02.699805 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_271_271_0 to 202410_281_281_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:02.690629 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_326_65/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:05:02.703555 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 16.71 MiB, will set to 1.69 GiB (RSS), difference: -14.53 MiB 2024.10.22 01:05:02.802382 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 289 2024.10.22 01:05:03.509288 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} MergeTask::MergeProjectionsStage: Merge sorted 540851 rows, containing 4 columns (4 merged, 0 gathered) in 18.374481153 sec., 29434.899167843727 rows/sec., 431.52 KiB/sec. 2024.10.22 01:05:03.553252 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 4925 2024.10.22 01:05:03.598985 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6dd0e1d97e815544b4dc528a3c98a125) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 6dd0e1d97e815544b4dc528a3c98a125, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:03.600721 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6dd0e1d97e815544b4dc528a3c98a125): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:05:03.604278 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_581_204} MergeTask::MergeProjectionsStage: Merge sorted 34660 rows, containing 13 columns (13 merged, 0 gathered) in 1.603618619 sec., 21613.617844879864 rows/sec., 3.35 MiB/sec. 2024.10.22 01:05:03.665805 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_581_204} MergedBlockOutputStream: filled checksums 202410_1_581_204 (state Temporary) 2024.10.22 01:05:03.682151 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} MergedBlockOutputStream: filled checksums 202410_1_326_65 (state Temporary) 2024.10.22 01:05:03.783452 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:03.786326 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:03.825766 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_581_204} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_581_204 to 202410_1_581_204 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:03.840985 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_581_204} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_580_203, 202410_581_581_0] -> 202410_1_581_204 2024.10.22 01:05:03.861722 [ 1391 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_581_204}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 01:05:03.889643 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:05:03.900305 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_326_65 to 202410_1_326_65 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:03.903131 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_326_65} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_321_64, 202410_326_326_0] -> 202410_1_326_65 2024.10.22 01:05:03.916809 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_336_336_0 (state Temporary) 2024.10.22 01:05:03.945408 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:05:03.979727 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_336_336_0 to 202410_349_349_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:04.112566 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4925 2024.10.22 01:05:04.269739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(cb36391d5086e128b6bcebc36d67a6b2) : [0, 107345]: Setting download as failed: File segment: [0, 107345], key: cb36391d5086e128b6bcebc36d67a6b2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:04.272302 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(cb36391d5086e128b6bcebc36d67a6b2): No space left in cache to reserve 107346 bytes, will continue without cache download 2024.10.22 01:05:04.288550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cb36391d5086e128b6bcebc36d67a6b2, offset: 0, size: 0 2024.10.22 01:05:04.291670 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cb36391d5086e128b6bcebc36d67a6b2 for removal 2024.10.22 01:05:04.293809 [ 1476 ] {} CacheMetadata: Key cb36391d5086e128b6bcebc36d67a6b2 is removed from metadata 2024.10.22 01:05:04.295241 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6dd0e1d97e815544b4dc528a3c98a125, offset: 0, size: 0 2024.10.22 01:05:04.296911 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6dd0e1d97e815544b4dc528a3c98a125 for removal 2024.10.22 01:05:04.299506 [ 1476 ] {} CacheMetadata: Key 6dd0e1d97e815544b4dc528a3c98a125 is removed from metadata 2024.10.22 01:05:05.114326 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(88665aa08ecee03444d54fe20c2680eb) : [0, 168]: Setting download as failed: File segment: [0, 168], key: 88665aa08ecee03444d54fe20c2680eb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:05.128188 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(88665aa08ecee03444d54fe20c2680eb): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:05:05.555526 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 11.48 MiB, will set to 1.68 GiB (RSS), difference: 22.21 MiB 2024.10.22 01:05:05.572387 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_480_125, 202410_481_481_0] 2024.10.22 01:05:05.575767 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_480_125, 202410_481_481_0] 2024.10.22 01:05:05.585451 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_480_125, 202410_481_481_0] 2024.10.22 01:05:05.964973 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5f257ffed1334c0d227899a6f65f4268) : [0, 168755]: Setting download as failed: File segment: [0, 168755], key: 5f257ffed1334c0d227899a6f65f4268, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:06.055836 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 34229 2024.10.22 01:05:06.057586 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5f257ffed1334c0d227899a6f65f4268): No space left in cache to reserve 168756 bytes, will continue without cache download 2024.10.22 01:05:06.183119 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5f257ffed1334c0d227899a6f65f4268, offset: 0, size: 0 2024.10.22 01:05:06.184735 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5f257ffed1334c0d227899a6f65f4268 for removal 2024.10.22 01:05:06.189339 [ 1476 ] {} CacheMetadata: Key 5f257ffed1334c0d227899a6f65f4268 is removed from metadata 2024.10.22 01:05:06.191525 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 88665aa08ecee03444d54fe20c2680eb, offset: 0, size: 0 2024.10.22 01:05:06.194952 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 88665aa08ecee03444d54fe20c2680eb for removal 2024.10.22 01:05:06.197527 [ 1476 ] {} CacheMetadata: Key 88665aa08ecee03444d54fe20c2680eb is removed from metadata 2024.10.22 01:05:06.240480 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:06.259297 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:06.321705 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_573_573_0 (state Temporary) 2024.10.22 01:05:06.376096 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_573_573_0 to 202410_586_586_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:06.396634 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_581_204 to 202410_582_582_0 2024.10.22 01:05:06.401048 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.22 MiB using storage policy from min volume index 0 2024.10.22 01:05:06.409833 [ 1153 ] {} DiskLocal: Reserved 1.22 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:06.426565 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34229 2024.10.22 01:05:06.444700 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_582_205} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_581_204 to 202410_582_582_0 into Compact with storage Full 2024.10.22 01:05:06.475444 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_582_205} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:06.481655 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_582_205} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_581_204, total 34660 rows starting from the beginning of the part 2024.10.22 01:05:06.512390 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_582_205} MergeTreeSequentialSource: Reading 2 marks from part 202410_582_582_0, total 98 rows starting from the beginning of the part 2024.10.22 01:05:07.053731 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a83637fd3a27ae5d5d9ac2078259947c) : [0, 167]: Setting download as failed: File segment: [0, 167], key: a83637fd3a27ae5d5d9ac2078259947c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:07.056509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a83637fd3a27ae5d5d9ac2078259947c): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:05:07.354279 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 8.34 MiB, will set to 1.68 GiB (RSS), difference: -3.91 MiB 2024.10.22 01:05:07.391107 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:05:07.886181 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e811ba7e31b8fab657a7df6e56cbc6f4) : [0, 185576]: Setting download as failed: File segment: [0, 185576], key: e811ba7e31b8fab657a7df6e56cbc6f4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:07.888086 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e811ba7e31b8fab657a7df6e56cbc6f4): No space left in cache to reserve 185577 bytes, will continue without cache download 2024.10.22 01:05:07.888218 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_69_14/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:05:07.906920 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e811ba7e31b8fab657a7df6e56cbc6f4, offset: 0, size: 0 2024.10.22 01:05:07.908160 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e811ba7e31b8fab657a7df6e56cbc6f4 for removal 2024.10.22 01:05:07.911058 [ 1476 ] {} CacheMetadata: Key e811ba7e31b8fab657a7df6e56cbc6f4 is removed from metadata 2024.10.22 01:05:07.911505 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a83637fd3a27ae5d5d9ac2078259947c, offset: 0, size: 0 2024.10.22 01:05:07.913562 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a83637fd3a27ae5d5d9ac2078259947c for removal 2024.10.22 01:05:07.917097 [ 1476 ] {} CacheMetadata: Key a83637fd3a27ae5d5d9ac2078259947c is removed from metadata 2024.10.22 01:05:08.309327 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 530862 2024.10.22 01:05:08.613016 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 123 entries to flush up to offset 26614 2024.10.22 01:05:08.637203 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_582_205/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:05:08.832706 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(82cdb06f10c8d13f71718f827726f576) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 82cdb06f10c8d13f71718f827726f576, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:05:08.840851 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 82cdb06f10c8d13f71718f827726f576, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:05:08.849453 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:08.911041 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:08.917360 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:08.964444 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:08.969001 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:08.977364 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_333_333_0 (state Temporary) 2024.10.22 01:05:09.132292 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_333_333_0 to 202410_339_339_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:09.190376 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 26614 2024.10.22 01:05:09.209484 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_322_322_0 (state Temporary) 2024.10.22 01:05:09.421563 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_322_322_0 to 202410_329_329_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:09.457729 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 530862 2024.10.22 01:05:09.564487 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f8384ed82018cdcadda892635b48cf78) : [0, 167]: Setting download as failed: File segment: [0, 167], key: f8384ed82018cdcadda892635b48cf78, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:09.566582 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f8384ed82018cdcadda892635b48cf78): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:05:09.693230 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_582_205} MergeTask::MergeProjectionsStage: Merge sorted 34758 rows, containing 13 columns (13 merged, 0 gathered) in 3.260300224 sec., 10660.981385743697 rows/sec., 1.65 MiB/sec. 2024.10.22 01:05:09.758096 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_582_205} MergedBlockOutputStream: filled checksums 202410_1_582_205 (state Temporary) 2024.10.22 01:05:09.866920 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:05:09.870766 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:05:09.971265 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_582_205} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_582_205 to 202410_1_582_205 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:09.978083 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_582_205} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_581_204, 202410_582_582_0] -> 202410_1_582_205 2024.10.22 01:05:10.006825 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:05:10.294808 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 7.66 MiB, will set to 1.68 GiB (RSS), difference: 5.35 MiB 2024.10.22 01:05:10.296302 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c9283c963b0165346bdcc8b12ea2690c) : [0, 102589]: Setting download as failed: File segment: [0, 102589], key: c9283c963b0165346bdcc8b12ea2690c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:10.297366 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c9283c963b0165346bdcc8b12ea2690c): No space left in cache to reserve 102590 bytes, will continue without cache download 2024.10.22 01:05:10.304533 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 290 2024.10.22 01:05:10.329615 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c9283c963b0165346bdcc8b12ea2690c, offset: 0, size: 0 2024.10.22 01:05:10.332128 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c9283c963b0165346bdcc8b12ea2690c for removal 2024.10.22 01:05:10.336521 [ 1476 ] {} CacheMetadata: Key c9283c963b0165346bdcc8b12ea2690c is removed from metadata 2024.10.22 01:05:10.340804 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f8384ed82018cdcadda892635b48cf78, offset: 0, size: 0 2024.10.22 01:05:10.344891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f8384ed82018cdcadda892635b48cf78 for removal 2024.10.22 01:05:10.349115 [ 1476 ] {} CacheMetadata: Key f8384ed82018cdcadda892635b48cf78 is removed from metadata 2024.10.22 01:05:10.507853 [ 1263 ] {} DatabaseCatalog: Execution took 231 ms. 2024.10.22 01:05:10.555914 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 34298 2024.10.22 01:05:10.596472 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:10.599487 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:10.712317 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:10.715182 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:10.756262 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_272_272_0 (state Temporary) 2024.10.22 01:05:10.795136 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_574_574_0 (state Temporary) 2024.10.22 01:05:10.832274 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_574_574_0 to 202410_587_587_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:10.874644 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_582_205 to 202410_583_583_0 2024.10.22 01:05:10.879778 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.22 MiB using storage policy from min volume index 0 2024.10.22 01:05:10.882413 [ 1200 ] {} DiskLocal: Reserved 1.22 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:10.889515 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34298 2024.10.22 01:05:10.898973 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_583_206} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_582_205 to 202410_583_583_0 into Compact with storage Full 2024.10.22 01:05:10.929581 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_272_272_0 to 202410_282_282_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:10.959611 [ 1197 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_277_57 to 202410_282_282_0 2024.10.22 01:05:10.976741 [ 1197 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:10.987938 [ 1197 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:10.994795 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b1f673de47f7767a3dd45793cdc95999) : [0, 168]: Setting download as failed: File segment: [0, 168], key: b1f673de47f7767a3dd45793cdc95999, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:10.998641 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b1f673de47f7767a3dd45793cdc95999): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:05:11.008963 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_277_57 to 202410_282_282_0 into Compact with storage Full 2024.10.22 01:05:11.075141 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 290 2024.10.22 01:05:11.176504 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_583_206} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:11.184019 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_583_206} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_582_205, total 34758 rows starting from the beginning of the part 2024.10.22 01:05:11.194803 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:11.211491 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_277_57, total 321 rows starting from the beginning of the part 2024.10.22 01:05:11.215623 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_583_206} MergeTreeSequentialSource: Reading 2 marks from part 202410_583_583_0, total 74 rows starting from the beginning of the part 2024.10.22 01:05:11.334240 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_278_278_0, total 1 rows starting from the beginning of the part 2024.10.22 01:05:11.519332 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_279_279_0, total 1 rows starting from the beginning of the part 2024.10.22 01:05:11.584630 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 8.29 MiB, will set to 1.68 GiB (RSS), difference: -8.79 MiB 2024.10.22 01:05:11.607379 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_280_280_0, total 1 rows starting from the beginning of the part 2024.10.22 01:05:11.616173 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 4944 2024.10.22 01:05:11.698335 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_281_281_0, total 1 rows starting from the beginning of the part 2024.10.22 01:05:11.731305 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:11.739767 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:11.752317 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(68b4974d08899fa686583f5ed8468608) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 68b4974d08899fa686583f5ed8468608, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:05:11.755979 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 647657, file segment info: File segment: [0, 33554431], key: 68b4974d08899fa686583f5ed8468608, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:05:11.760325 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:11.805768 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:05:11.929338 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_337_337_0 (state Temporary) 2024.10.22 01:05:11.996172 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_282_282_0, total 1 rows starting from the beginning of the part 2024.10.22 01:05:12.091423 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_337_337_0 to 202410_350_350_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:12.255792 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4944 2024.10.22 01:05:12.591841 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(eac44029942981ebb754a28d93208481) : [0, 1119800]: Setting download as failed: File segment: [0, 1119800], key: eac44029942981ebb754a28d93208481, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:12.593470 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(eac44029942981ebb754a28d93208481): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:05:12.636625 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eac44029942981ebb754a28d93208481, offset: 0, size: 0 2024.10.22 01:05:12.639308 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eac44029942981ebb754a28d93208481 for removal 2024.10.22 01:05:12.643393 [ 1476 ] {} CacheMetadata: Key eac44029942981ebb754a28d93208481 is removed from metadata 2024.10.22 01:05:12.647087 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b1f673de47f7767a3dd45793cdc95999, offset: 0, size: 0 2024.10.22 01:05:12.649918 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b1f673de47f7767a3dd45793cdc95999 for removal 2024.10.22 01:05:12.704312 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_481_126, 202410_482_482_0] 2024.10.22 01:05:12.707079 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_481_126, 202410_482_482_0] 2024.10.22 01:05:12.721819 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_481_126, 202410_482_482_0] 2024.10.22 01:05:12.749404 [ 1476 ] {} CacheMetadata: Key b1f673de47f7767a3dd45793cdc95999 is removed from metadata 2024.10.22 01:05:12.969714 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_583_206} MergeTask::MergeProjectionsStage: Merge sorted 34832 rows, containing 13 columns (13 merged, 0 gathered) in 2.079442586 sec., 16750.642809043635 rows/sec., 2.60 MiB/sec. 2024.10.22 01:05:13.116575 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_583_206} MergedBlockOutputStream: filled checksums 202410_1_583_206 (state Temporary) 2024.10.22 01:05:13.204250 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_583_206} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_583_206 to 202410_1_583_206 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:13.208041 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_583_206} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_582_205, 202410_583_583_0] -> 202410_1_583_206 2024.10.22 01:05:13.240877 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:05:13.447683 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7c482176003bd02037dba9eac8337b49) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7c482176003bd02037dba9eac8337b49, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:05:13.451685 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1273, file segment info: File segment: [0, 33554431], key: 7c482176003bd02037dba9eac8337b49, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:05:13.455565 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:13.786993 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} MergeTask::MergeProjectionsStage: Merge sorted 326 rows, containing 32 columns (32 merged, 0 gathered) in 2.793771817 sec., 116.68812678841623 rows/sec., 11.96 KiB/sec. 2024.10.22 01:05:13.807908 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} MergedBlockOutputStream: filled checksums 202410_1_282_58 (state Temporary) 2024.10.22 01:05:14.186227 [ 1222 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:05:14.280266 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_282_58/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:05:14.304384 [ 1222 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:05:14.407644 [ 1222 ] {} DNSCacheUpdater: Execution took 221 ms. 2024.10.22 01:05:14.416535 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 8.66 MiB, will set to 1.68 GiB (RSS), difference: 5.69 MiB 2024.10.22 01:05:14.664388 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_282_58 to 202410_1_282_58 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:14.667614 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f1a6433dedf371e4f46fb2f154639caf) : [0, 155]: Setting download as failed: File segment: [0, 155], key: f1a6433dedf371e4f46fb2f154639caf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:14.670144 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f1a6433dedf371e4f46fb2f154639caf): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:05:14.671604 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_282_58} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_277_57, 202410_282_282_0] -> 202410_1_282_58 2024.10.22 01:05:14.712765 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:05:15.096072 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d1d912b816a61728559f458ddbf1688f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d1d912b816a61728559f458ddbf1688f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:05:15.459938 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 34372 2024.10.22 01:05:15.463342 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 807640, file segment info: File segment: [0, 33554431], key: d1d912b816a61728559f458ddbf1688f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:05:15.469536 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:15.486594 [ 1221 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 202 ms. 2024.10.22 01:05:15.981368 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:15.985066 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:16.055310 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_575_575_0 (state Temporary) 2024.10.22 01:05:16.100997 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1427b96e2cdfa11aa3b80798c3757ad4) : [0, 55178]: Setting download as failed: File segment: [0, 55178], key: 1427b96e2cdfa11aa3b80798c3757ad4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:16.103146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1427b96e2cdfa11aa3b80798c3757ad4): No space left in cache to reserve 55179 bytes, will continue without cache download 2024.10.22 01:05:16.121538 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1427b96e2cdfa11aa3b80798c3757ad4, offset: 0, size: 0 2024.10.22 01:05:16.143290 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1427b96e2cdfa11aa3b80798c3757ad4 for removal 2024.10.22 01:05:16.151381 [ 1476 ] {} CacheMetadata: Key 1427b96e2cdfa11aa3b80798c3757ad4 is removed from metadata 2024.10.22 01:05:16.154206 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f1a6433dedf371e4f46fb2f154639caf, offset: 0, size: 0 2024.10.22 01:05:16.161143 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f1a6433dedf371e4f46fb2f154639caf for removal 2024.10.22 01:05:16.167805 [ 1476 ] {} CacheMetadata: Key f1a6433dedf371e4f46fb2f154639caf is removed from metadata 2024.10.22 01:05:16.215286 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_575_575_0 to 202410_588_588_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:16.257010 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_583_206 to 202410_584_584_0 2024.10.22 01:05:16.269044 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.23 MiB using storage policy from min volume index 0 2024.10.22 01:05:16.282733 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34372 2024.10.22 01:05:16.275549 [ 1240 ] {} DiskLocal: Reserved 1.23 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:16.301364 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_584_207} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_583_206 to 202410_584_584_0 into Compact with storage Full 2024.10.22 01:05:16.475412 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1228 entries to flush up to offset 532090 2024.10.22 01:05:16.477643 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_584_207} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:16.487572 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_584_207} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_583_206, total 34832 rows starting from the beginning of the part 2024.10.22 01:05:16.612408 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_584_207} MergeTreeSequentialSource: Reading 2 marks from part 202410_584_584_0, total 79 rows starting from the beginning of the part 2024.10.22 01:05:16.709121 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 97 entries to flush up to offset 26711 2024.10.22 01:05:16.863536 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c169f75f7b37ad446191050ba4de8bbe) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c169f75f7b37ad446191050ba4de8bbe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:05:16.885720 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 936, file segment info: File segment: [0, 33554431], key: c169f75f7b37ad446191050ba4de8bbe, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:05:16.890550 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:16.915481 [ 1686 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nxa/juvbfuwgugjeefcfufvwxlojxzfbn, size 807640 2024.10.22 01:05:17.025703 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:17.033702 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:17.080107 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_334_334_0 (state Temporary) 2024.10.22 01:05:17.092303 [ 1226 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 348 ms. 2024.10.22 01:05:17.099536 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_482_127, 202410_483_483_0] 2024.10.22 01:05:17.102624 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_482_127, 202410_483_483_0] 2024.10.22 01:05:17.110477 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_482_127, 202410_483_483_0] 2024.10.22 01:05:17.114173 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_334_334_0 to 202410_340_340_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:17.177036 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 26711 2024.10.22 01:05:17.222322 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:17.228343 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:17.303809 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:05:17.358442 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:05:17.386711 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:05:17.419973 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:05:17.541703 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_323_323_0 (state Temporary) 2024.10.22 01:05:17.543378 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:05:17.627814 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(26a970fa1bbadb644e6b4014ef06e112) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 26a970fa1bbadb644e6b4014ef06e112, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:17.629280 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(26a970fa1bbadb644e6b4014ef06e112): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:05:17.645229 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_323_323_0 to 202410_330_330_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:17.702577 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 532090 2024.10.22 01:05:17.778821 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:05:17.789474 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:05:17.813292 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:05:17.945114 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 6.18 MiB, will set to 1.69 GiB (RSS), difference: -6.71 MiB 2024.10.22 01:05:17.954464 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:05:18.000698 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hhf/laqpseenvemgijhkptekgpxvmkmal (LogSeriesLimiter: on interval from 2024-10-22 01:04:59 to 2024-10-22 01:05:17 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:05:18.133948 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(fd82e66bf39a439ab2138f27b9eacdff) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fd82e66bf39a439ab2138f27b9eacdff, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:05:18.136928 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 902, file segment info: File segment: [0, 33554431], key: fd82e66bf39a439ab2138f27b9eacdff, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:05:18.140857 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:18.207933 [ 1698 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ney/acwkmcgvicbbuxcqtbjdgtyuzupcr, size 1956807 2024.10.22 01:05:18.663372 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:18.672713 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:18.723183 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_584_207} MergeTask::MergeProjectionsStage: Merge sorted 34911 rows, containing 13 columns (13 merged, 0 gathered) in 2.426436976 sec., 14387.762940190209 rows/sec., 2.23 MiB/sec. 2024.10.22 01:05:18.800708 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:05:18.807831 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:05:18.818156 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:05:18.831157 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_286_57, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:05:18.833506 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_286_57, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:05:18.870749 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_286_57, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:05:18.933085 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:05:18.956877 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:05:18.980844 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:05:18.996669 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:05:19.031487 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d05607e84608931e259b885fece466a9) : [0, 48033]: Setting download as failed: File segment: [0, 48033], key: d05607e84608931e259b885fece466a9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:19.032881 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d05607e84608931e259b885fece466a9): No space left in cache to reserve 48034 bytes, will continue without cache download 2024.10.22 01:05:19.078641 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_584_207} MergedBlockOutputStream: filled checksums 202410_1_584_207 (state Temporary) 2024.10.22 01:05:19.143455 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d05607e84608931e259b885fece466a9, offset: 0, size: 0 2024.10.22 01:05:19.144659 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d05607e84608931e259b885fece466a9 for removal 2024.10.22 01:05:19.189108 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_584_207} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_584_207 to 202410_1_584_207 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:19.254901 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 26a970fa1bbadb644e6b4014ef06e112, offset: 0, size: 0 2024.10.22 01:05:19.254382 [ 1476 ] {} CacheMetadata: Key d05607e84608931e259b885fece466a9 is removed from metadata 2024.10.22 01:05:19.255701 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_584_207} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_583_206, 202410_584_584_0] -> 202410_1_584_207 2024.10.22 01:05:19.257397 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 26a970fa1bbadb644e6b4014ef06e112 for removal 2024.10.22 01:05:19.365618 [ 1476 ] {} CacheMetadata: Key 26a970fa1bbadb644e6b4014ef06e112 is removed from metadata 2024.10.22 01:05:19.385715 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:05:19.759228 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 4961 2024.10.22 01:05:19.879468 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:05:19.886475 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:19.889066 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:19.915679 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_338_338_0 (state Temporary) 2024.10.22 01:05:19.973591 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:05:19.978528 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_338_338_0 to 202410_351_351_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:19.997118 [ 1268 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_346_69 to 202410_351_351_0 2024.10.22 01:05:20.009364 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4961 2024.10.22 01:05:20.014219 [ 1268 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:20.019807 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:20.025452 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(62952fed4ca11201eda2142c147582a8) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 62952fed4ca11201eda2142c147582a8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:20.032464 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(62952fed4ca11201eda2142c147582a8): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:05:20.095441 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_346_69 to 202410_351_351_0 into Compact with storage Full 2024.10.22 01:05:20.171034 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:20.179224 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_346_69, total 4978 rows starting from the beginning of the part 2024.10.22 01:05:20.197620 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_347_347_0, total 24 rows starting from the beginning of the part 2024.10.22 01:05:20.207694 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_229_47, 202410_230_230_0, 202410_231_231_0, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0] 2024.10.22 01:05:20.210517 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_229_47, 202410_230_230_0, 202410_231_231_0, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0] 2024.10.22 01:05:20.233478 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_348_348_0, total 20 rows starting from the beginning of the part 2024.10.22 01:05:20.296332 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 107 entries to flush up to offset 34479 2024.10.22 01:05:20.350997 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_229_47, 202410_230_230_0, 202410_231_231_0, 202410_232_232_0, 202410_233_233_0, 202410_234_234_0] 2024.10.22 01:05:20.386518 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_349_349_0, total 9 rows starting from the beginning of the part 2024.10.22 01:05:20.508805 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_350_350_0, total 19 rows starting from the beginning of the part 2024.10.22 01:05:20.547554 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_351_351_0, total 17 rows starting from the beginning of the part 2024.10.22 01:05:20.591270 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 13.23 MiB, will set to 1.69 GiB (RSS), difference: 17.25 MiB 2024.10.22 01:05:20.678217 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:20.721331 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:20.818519 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_576_576_0 (state Temporary) 2024.10.22 01:05:20.839693 [ 1265 ] {} DatabaseCatalog: Execution took 330 ms. 2024.10.22 01:05:20.859325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a195d60a263491e31e3bde2fe1a14497) : [0, 91878]: Setting download as failed: File segment: [0, 91878], key: a195d60a263491e31e3bde2fe1a14497, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:20.861730 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a195d60a263491e31e3bde2fe1a14497): No space left in cache to reserve 91879 bytes, will continue without cache download 2024.10.22 01:05:20.900222 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a195d60a263491e31e3bde2fe1a14497, offset: 0, size: 0 2024.10.22 01:05:20.901625 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a195d60a263491e31e3bde2fe1a14497 for removal 2024.10.22 01:05:20.903343 [ 1476 ] {} CacheMetadata: Key a195d60a263491e31e3bde2fe1a14497 is removed from metadata 2024.10.22 01:05:20.904157 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 62952fed4ca11201eda2142c147582a8, offset: 0, size: 0 2024.10.22 01:05:20.905129 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 62952fed4ca11201eda2142c147582a8 for removal 2024.10.22 01:05:20.909231 [ 1476 ] {} CacheMetadata: Key 62952fed4ca11201eda2142c147582a8 is removed from metadata 2024.10.22 01:05:20.983284 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_576_576_0 to 202410_589_589_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:20.989967 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_69_14/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:05:21.104106 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34479 2024.10.22 01:05:21.039990 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_584_207 to 202410_585_585_0 2024.10.22 01:05:21.181781 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.23 MiB using storage policy from min volume index 0 2024.10.22 01:05:21.183381 [ 1172 ] {} DiskLocal: Reserved 1.23 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:21.198606 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_585_208} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_584_207 to 202410_585_585_0 into Compact with storage Full 2024.10.22 01:05:21.216074 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_585_208} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:21.225831 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_585_208} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_584_207, total 34911 rows starting from the beginning of the part 2024.10.22 01:05:21.244407 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_585_208} MergeTreeSequentialSource: Reading 2 marks from part 202410_585_585_0, total 79 rows starting from the beginning of the part 2024.10.22 01:05:21.381662 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} MergeTask::MergeProjectionsStage: Merge sorted 5067 rows, containing 12 columns (12 merged, 0 gathered) in 1.291198809 sec., 3924.2601253049947 rows/sec., 617.87 KiB/sec. 2024.10.22 01:05:21.402647 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} MergedBlockOutputStream: filled checksums 202410_1_351_70 (state Temporary) 2024.10.22 01:05:21.723691 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_277_75, 202410_278_278_0] 2024.10.22 01:05:21.727814 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_277_75, 202410_278_278_0] 2024.10.22 01:05:21.737779 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_277_75, 202410_278_278_0] 2024.10.22 01:05:21.777683 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(df83367c20e532b6e00b665cc4f79cce) : [0, 154]: Setting download as failed: File segment: [0, 154], key: df83367c20e532b6e00b665cc4f79cce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:21.781333 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(df83367c20e532b6e00b665cc4f79cce): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:05:21.819052 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_351_70 to 202410_1_351_70 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:21.843365 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_351_70} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_346_69, 202410_351_351_0] -> 202410_1_351_70 2024.10.22 01:05:21.891978 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.39 MiB. 2024.10.22 01:05:22.173175 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_483_128, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0, 202410_487_487_0, 202410_488_488_0] 2024.10.22 01:05:22.176588 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_483_128, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0, 202410_487_487_0, 202410_488_488_0] 2024.10.22 01:05:22.200246 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_483_128, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0, 202410_487_487_0, 202410_488_488_0] 2024.10.22 01:05:22.602914 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(16e1e4418a86b849c36c815c652716dd) : [0, 135971]: Setting download as failed: File segment: [0, 135971], key: 16e1e4418a86b849c36c815c652716dd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:22.604586 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(16e1e4418a86b849c36c815c652716dd): No space left in cache to reserve 135972 bytes, will continue without cache download 2024.10.22 01:05:22.714965 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_68_68_0 (state Temporary) 2024.10.22 01:05:22.733143 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 16e1e4418a86b849c36c815c652716dd, offset: 0, size: 0 2024.10.22 01:05:22.736303 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 16e1e4418a86b849c36c815c652716dd for removal 2024.10.22 01:05:22.740016 [ 1476 ] {} CacheMetadata: Key 16e1e4418a86b849c36c815c652716dd is removed from metadata 2024.10.22 01:05:22.741467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: df83367c20e532b6e00b665cc4f79cce, offset: 0, size: 0 2024.10.22 01:05:22.743551 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key df83367c20e532b6e00b665cc4f79cce for removal 2024.10.22 01:05:22.747086 [ 1476 ] {} CacheMetadata: Key df83367c20e532b6e00b665cc4f79cce is removed from metadata 2024.10.22 01:05:23.197840 [ 1168 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 249 ms. 2024.10.22 01:05:23.467902 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_585_208} MergeTask::MergeProjectionsStage: Merge sorted 34990 rows, containing 13 columns (13 merged, 0 gathered) in 2.282130888 sec., 15332.161789661557 rows/sec., 2.38 MiB/sec. 2024.10.22 01:05:23.517260 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 11.69 MiB, will set to 1.69 GiB (RSS), difference: -14.22 MiB 2024.10.22 01:05:23.629720 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_585_208} MergedBlockOutputStream: filled checksums 202410_1_585_208 (state Temporary) 2024.10.22 01:05:23.737773 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3a0b5c0dea0443226c58021eedbc3559) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 3a0b5c0dea0443226c58021eedbc3559, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:23.745457 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3a0b5c0dea0443226c58021eedbc3559): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:05:23.821608 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_585_208} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_585_208 to 202410_1_585_208 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:23.827987 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_585_208} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_584_207, 202410_585_585_0] -> 202410_1_585_208 2024.10.22 01:05:24.000588 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:05:24.012879 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_266_53, 202410_267_267_0, 202410_268_268_0, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0] 2024.10.22 01:05:24.026200 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_266_53, 202410_267_267_0, 202410_268_268_0, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0] 2024.10.22 01:05:24.065642 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_266_53, 202410_267_267_0, 202410_268_268_0, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0] 2024.10.22 01:05:24.598511 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bb94bfaebcd148202fb9153bd6732bf4) : [0, 99549]: Setting download as failed: File segment: [0, 99549], key: bb94bfaebcd148202fb9153bd6732bf4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:24.600183 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bb94bfaebcd148202fb9153bd6732bf4): No space left in cache to reserve 99550 bytes, will continue without cache download 2024.10.22 01:05:24.611612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bb94bfaebcd148202fb9153bd6732bf4, offset: 0, size: 0 2024.10.22 01:05:24.613341 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bb94bfaebcd148202fb9153bd6732bf4 for removal 2024.10.22 01:05:24.617004 [ 1476 ] {} CacheMetadata: Key bb94bfaebcd148202fb9153bd6732bf4 is removed from metadata 2024.10.22 01:05:24.619627 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3a0b5c0dea0443226c58021eedbc3559, offset: 0, size: 0 2024.10.22 01:05:24.621381 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3a0b5c0dea0443226c58021eedbc3559 for removal 2024.10.22 01:05:24.625312 [ 1476 ] {} CacheMetadata: Key 3a0b5c0dea0443226c58021eedbc3559 is removed from metadata 2024.10.22 01:05:24.689373 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 141 entries to flush up to offset 26852 2024.10.22 01:05:24.706341 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1460 entries to flush up to offset 533550 2024.10.22 01:05:25.093824 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:25.098244 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:25.175350 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_335_335_0 (state Temporary) 2024.10.22 01:05:25.226083 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_335_335_0 to 202410_341_341_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:25.129753 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 34553 2024.10.22 01:05:25.281025 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(49afb72253e6afb2b74cf55744718be3) : [0, 180]: Setting download as failed: File segment: [0, 180], key: 49afb72253e6afb2b74cf55744718be3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:25.283295 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(49afb72253e6afb2b74cf55744718be3): No space left in cache to reserve 181 bytes, will continue without cache download 2024.10.22 01:05:25.391033 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 10.70 MiB, will set to 1.69 GiB (RSS), difference: -1.20 MiB 2024.10.22 01:05:25.399465 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:25.406699 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:25.544953 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:25.554137 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:25.587157 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_577_577_0 (state Temporary) 2024.10.22 01:05:25.642622 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 26852 2024.10.22 01:05:25.744103 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_577_577_0 to 202410_590_590_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:25.769789 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_585_208 to 202410_586_586_0 2024.10.22 01:05:25.786484 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.23 MiB using storage policy from min volume index 0 2024.10.22 01:05:25.789005 [ 1176 ] {} DiskLocal: Reserved 1.23 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:25.888604 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_324_324_0 (state Temporary) 2024.10.22 01:05:25.890596 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34553 2024.10.22 01:05:25.936839 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_586_209} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_585_208 to 202410_586_586_0 into Compact with storage Full 2024.10.22 01:05:25.982437 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_586_209} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:25.989122 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_586_209} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_585_208, total 34990 rows starting from the beginning of the part 2024.10.22 01:05:26.007822 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_586_209} MergeTreeSequentialSource: Reading 2 marks from part 202410_586_586_0, total 70 rows starting from the beginning of the part 2024.10.22 01:05:26.065376 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_324_324_0 to 202410_331_331_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:26.077645 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 291 2024.10.22 01:05:26.099338 [ 1162 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_326_65 to 202410_331_331_0 2024.10.22 01:05:26.111420 [ 1162 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:26.115331 [ 1162 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:26.167829 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_326_65 to 202410_331_331_0 into Compact with storage Full 2024.10.22 01:05:26.193221 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:26.199261 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4d4b715c333f604ba901e50077b6c208) : [0, 325852]: Setting download as failed: File segment: [0, 325852], key: 4d4b715c333f604ba901e50077b6c208, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:26.201327 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4d4b715c333f604ba901e50077b6c208): No space left in cache to reserve 325853 bytes, will continue without cache download 2024.10.22 01:05:26.202935 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} MergeTreeSequentialSource: Reading 68 marks from part 202410_1_326_65, total 540851 rows starting from the beginning of the part 2024.10.22 01:05:26.203622 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 533550 2024.10.22 01:05:26.220718 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_327_327_0, total 913 rows starting from the beginning of the part 2024.10.22 01:05:26.248603 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4d4b715c333f604ba901e50077b6c208, offset: 0, size: 0 2024.10.22 01:05:26.252301 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4d4b715c333f604ba901e50077b6c208 for removal 2024.10.22 01:05:26.256111 [ 1476 ] {} CacheMetadata: Key 4d4b715c333f604ba901e50077b6c208 is removed from metadata 2024.10.22 01:05:26.258501 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 49afb72253e6afb2b74cf55744718be3, offset: 0, size: 0 2024.10.22 01:05:26.260887 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 49afb72253e6afb2b74cf55744718be3 for removal 2024.10.22 01:05:26.266901 [ 1476 ] {} CacheMetadata: Key 49afb72253e6afb2b74cf55744718be3 is removed from metadata 2024.10.22 01:05:26.345225 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_328_328_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:05:26.376796 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_329_329_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:05:26.400266 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_330_330_0, total 1228 rows starting from the beginning of the part 2024.10.22 01:05:26.427654 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_331_331_0, total 1460 rows starting from the beginning of the part 2024.10.22 01:05:26.479896 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:26.482289 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:26.643550 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_273_273_0 (state Temporary) 2024.10.22 01:05:26.822639 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_273_273_0 to 202410_283_283_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:26.911013 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 291 2024.10.22 01:05:27.512027 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 4977 2024.10.22 01:05:27.523594 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5bdf5f05dbc162361e536b253675b4fc) : [0, 168]: Setting download as failed: File segment: [0, 168], key: 5bdf5f05dbc162361e536b253675b4fc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:27.525208 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 14.65 MiB, will set to 1.69 GiB (RSS), difference: -5.83 MiB 2024.10.22 01:05:27.532150 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5bdf5f05dbc162361e536b253675b4fc): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:05:27.648977 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:27.653187 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:27.687997 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_339_339_0 (state Temporary) 2024.10.22 01:05:27.830200 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_339_339_0 to 202410_352_352_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:28.184063 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4977 2024.10.22 01:05:28.423481 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_586_209} MergeTask::MergeProjectionsStage: Merge sorted 35060 rows, containing 13 columns (13 merged, 0 gathered) in 2.514770601 sec., 13941.62950133836 rows/sec., 2.17 MiB/sec. 2024.10.22 01:05:28.503555 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(65d59499cf1f5674c2d48c1cc5b9481f) : [0, 1032588]: Setting download as failed: File segment: [0, 1032588], key: 65d59499cf1f5674c2d48c1cc5b9481f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:28.504802 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(65d59499cf1f5674c2d48c1cc5b9481f): No space left in cache to reserve 1032589 bytes, will continue without cache download 2024.10.22 01:05:28.589726 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_586_209} MergedBlockOutputStream: filled checksums 202410_1_586_209 (state Temporary) 2024.10.22 01:05:28.633722 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 65d59499cf1f5674c2d48c1cc5b9481f, offset: 0, size: 0 2024.10.22 01:05:28.636118 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 65d59499cf1f5674c2d48c1cc5b9481f for removal 2024.10.22 01:05:28.639516 [ 1476 ] {} CacheMetadata: Key 65d59499cf1f5674c2d48c1cc5b9481f is removed from metadata 2024.10.22 01:05:28.745108 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5bdf5f05dbc162361e536b253675b4fc, offset: 0, size: 0 2024.10.22 01:05:28.746621 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5bdf5f05dbc162361e536b253675b4fc for removal 2024.10.22 01:05:28.760075 [ 1476 ] {} CacheMetadata: Key 5bdf5f05dbc162361e536b253675b4fc is removed from metadata 2024.10.22 01:05:28.895059 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_586_209} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_586_209 to 202410_1_586_209 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:28.899835 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_586_209} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_585_208, 202410_586_586_0] -> 202410_1_586_209 2024.10.22 01:05:28.939837 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.24 MiB. 2024.10.22 01:05:29.031407 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_331_66/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:05:29.292838 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(7fa814d82ae1bfa666d362211e9bde76) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7fa814d82ae1bfa666d362211e9bde76, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:05:29.310988 [ 1270 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:05:29.331997 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 7fa814d82ae1bfa666d362211e9bde76, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:05:29.336296 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:29.459091 [ 1270 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:05:29.533659 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 18.22 MiB, will set to 1.69 GiB (RSS), difference: 9.54 MiB 2024.10.22 01:05:29.871151 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:05:29.874941 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:05:29.900240 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 34633 2024.10.22 01:05:29.976850 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(cc26314d5edf6d6b65e81bff64f64518) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cc26314d5edf6d6b65e81bff64f64518, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:05:29.983893 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: cc26314d5edf6d6b65e81bff64f64518, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:05:29.990834 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:30.178358 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:30.187309 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:30.379939 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_578_578_0 (state Temporary) 2024.10.22 01:05:30.454065 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_578_578_0 to 202410_591_591_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:30.485677 [ 1238 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_586_209 to 202410_587_587_0 2024.10.22 01:05:30.490958 [ 1238 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.24 MiB using storage policy from min volume index 0 2024.10.22 01:05:30.495498 [ 1238 ] {} DiskLocal: Reserved 1.24 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:30.513662 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_587_210} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_586_209 to 202410_587_587_0 into Compact with storage Full 2024.10.22 01:05:30.567243 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_587_210} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:30.578902 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_587_210} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_586_209, total 35060 rows starting from the beginning of the part 2024.10.22 01:05:30.600746 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_587_210} MergeTreeSequentialSource: Reading 2 marks from part 202410_587_587_0, total 69 rows starting from the beginning of the part 2024.10.22 01:05:30.614901 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34633 2024.10.22 01:05:30.799893 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fdf58b92ef44ca1c62a741a9585a6251) : [0, 156]: Setting download as failed: File segment: [0, 156], key: fdf58b92ef44ca1c62a741a9585a6251, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:30.805517 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fdf58b92ef44ca1c62a741a9585a6251): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:05:31.071767 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_68_68_0 to 202410_71_71_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:31.136191 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_69_14/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:05:31.177449 [ 1196 ] {} DatabaseCatalog: Execution took 329 ms. 2024.10.22 01:05:31.390860 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_278_76, 202410_279_279_0] 2024.10.22 01:05:31.392116 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_278_76, 202410_279_279_0] 2024.10.22 01:05:31.400522 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_278_76, 202410_279_279_0] 2024.10.22 01:05:31.515501 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_587_210/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:05:31.809051 [ 1208 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:05:32.160390 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c004f268452bd2f4ec753f7c9e1aae11) : [0, 114680]: Setting download as failed: File segment: [0, 114680], key: c004f268452bd2f4ec753f7c9e1aae11, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:32.163296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c004f268452bd2f4ec753f7c9e1aae11): No space left in cache to reserve 114681 bytes, will continue without cache download 2024.10.22 01:05:32.187125 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c004f268452bd2f4ec753f7c9e1aae11, offset: 0, size: 0 2024.10.22 01:05:32.188395 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c004f268452bd2f4ec753f7c9e1aae11 for removal 2024.10.22 01:05:32.190795 [ 1476 ] {} CacheMetadata: Key c004f268452bd2f4ec753f7c9e1aae11 is removed from metadata 2024.10.22 01:05:32.191668 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fdf58b92ef44ca1c62a741a9585a6251, offset: 0, size: 0 2024.10.22 01:05:32.193791 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fdf58b92ef44ca1c62a741a9585a6251 for removal 2024.10.22 01:05:32.197721 [ 1476 ] {} CacheMetadata: Key fdf58b92ef44ca1c62a741a9585a6251 is removed from metadata 2024.10.22 01:05:32.491072 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.88 GiB, free memory in arenas 16.91 MiB, will set to 1.70 GiB (RSS), difference: 2.32 MiB 2024.10.22 01:05:32.783171 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(344efe377f91491e9d950195eb294f10) : [0, 168]: Setting download as failed: File segment: [0, 168], key: 344efe377f91491e9d950195eb294f10, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:32.784252 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(344efe377f91491e9d950195eb294f10): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:05:33.132654 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_587_210} MergeTask::MergeProjectionsStage: Merge sorted 35129 rows, containing 13 columns (13 merged, 0 gathered) in 2.630451746 sec., 13354.740322995456 rows/sec., 2.08 MiB/sec. 2024.10.22 01:05:33.192410 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 123 entries to flush up to offset 26975 2024.10.22 01:05:33.207946 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1305 entries to flush up to offset 534855 2024.10.22 01:05:33.297295 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_587_210} MergedBlockOutputStream: filled checksums 202410_1_587_210 (state Temporary) 2024.10.22 01:05:33.392297 [ 1213 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 215 ms. 2024.10.22 01:05:33.616510 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:33.619676 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:33.671024 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_336_336_0 (state Temporary) 2024.10.22 01:05:33.718240 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_336_336_0 to 202410_342_342_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:33.764810 [ 1171 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_336_114 to 202410_337_337_0 2024.10.22 01:05:33.773029 [ 1171 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.43 MiB using storage policy from min volume index 0 2024.10.22 01:05:33.776817 [ 1171 ] {} DiskLocal: Reserved 1.43 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:33.790616 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_587_210} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_587_210 to 202410_1_587_210 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:33.792376 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_337_115} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_336_114 to 202410_337_337_0 into Wide with storage Full 2024.10.22 01:05:33.794198 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_587_210} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_586_209, 202410_587_587_0] -> 202410_1_587_210 2024.10.22 01:05:33.811435 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:05:33.859415 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_337_115} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:33.874226 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_337_115} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_336_114, total 26269 rows starting from the beginning of the part 2024.10.22 01:05:33.893609 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(47ef3d267e47b0a6bb3e2d0762bc3a45) : [0, 1121004]: Setting download as failed: File segment: [0, 1121004], key: 47ef3d267e47b0a6bb3e2d0762bc3a45, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:33.895180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(47ef3d267e47b0a6bb3e2d0762bc3a45): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:05:33.922908 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 26975 2024.10.22 01:05:34.016592 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2721 2024.10.22 01:05:34.084890 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 47ef3d267e47b0a6bb3e2d0762bc3a45, offset: 0, size: 0 2024.10.22 01:05:34.105189 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 47ef3d267e47b0a6bb3e2d0762bc3a45 for removal 2024.10.22 01:05:34.107196 [ 1476 ] {} CacheMetadata: Key 47ef3d267e47b0a6bb3e2d0762bc3a45 is removed from metadata 2024.10.22 01:05:34.111333 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_337_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_337_337_0, total 216 rows starting from the beginning of the part 2024.10.22 01:05:34.113494 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 344efe377f91491e9d950195eb294f10, offset: 0, size: 0 2024.10.22 01:05:34.118035 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 344efe377f91491e9d950195eb294f10 for removal 2024.10.22 01:05:34.129217 [ 1476 ] {} CacheMetadata: Key 344efe377f91491e9d950195eb294f10 is removed from metadata 2024.10.22 01:05:34.161711 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:34.169181 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.54 GiB. 2024.10.22 01:05:34.355531 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(d8007ac33e1aa0276af262c5d2ed8cb1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d8007ac33e1aa0276af262c5d2ed8cb1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:05:34.366611 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 893, file segment info: File segment: [0, 33554431], key: d8007ac33e1aa0276af262c5d2ed8cb1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:05:34.367445 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 17.26 MiB, will set to 1.69 GiB (RSS), difference: -31.22 MiB 2024.10.22 01:05:34.473376 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:34.413724 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 292 2024.10.22 01:05:34.635577 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 34706 2024.10.22 01:05:34.645675 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:05:34.684046 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:05:34.694389 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:05:34.728721 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:05:34.749816 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:05:34.782868 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:05:34.799259 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:05:34.874790 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_325_325_0 (state Temporary) 2024.10.22 01:05:34.915512 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:34.954978 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:34.960934 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:34.978888 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_325_325_0 to 202410_332_332_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:35.015425 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 534855 2024.10.22 01:05:35.017157 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_579_579_0 (state Temporary) 2024.10.22 01:05:35.020825 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:35.098485 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/eel/notjelerexwituiribwosnnivgqjw (LogSeriesLimiter: on interval from 2024-10-22 01:05:17 to 2024-10-22 01:05:35 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:05:35.289555 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_579_579_0 to 202410_592_592_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:35.295745 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_274_274_0 (state Temporary) 2024.10.22 01:05:35.316075 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_587_210 to 202410_588_588_0 2024.10.22 01:05:35.322453 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.24 MiB using storage policy from min volume index 0 2024.10.22 01:05:35.327830 [ 1221 ] {} DiskLocal: Reserved 1.24 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:35.346836 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_588_211} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_587_210 to 202410_588_588_0 into Compact with storage Full 2024.10.22 01:05:35.425459 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_588_211} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:35.446067 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34706 2024.10.22 01:05:35.447566 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_588_211} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_587_210, total 35129 rows starting from the beginning of the part 2024.10.22 01:05:35.499414 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_588_211} MergeTreeSequentialSource: Reading 2 marks from part 202410_588_588_0, total 74 rows starting from the beginning of the part 2024.10.22 01:05:35.516398 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_274_274_0 to 202410_284_284_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:35.596710 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(04f43b073f873ef55fadcf0eb1f8dcc9) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 04f43b073f873ef55fadcf0eb1f8dcc9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:35.600431 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(04f43b073f873ef55fadcf0eb1f8dcc9): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:05:35.628055 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 292 2024.10.22 01:05:35.687673 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 4990 2024.10.22 01:05:35.919936 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:35.923311 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:35.973233 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_340_340_0 (state Temporary) 2024.10.22 01:05:36.037118 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_340_340_0 to 202410_353_353_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:36.170855 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 4990 2024.10.22 01:05:36.545140 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(de4507d33ceaa692121d87015172bea4) : [0, 83276]: Setting download as failed: File segment: [0, 83276], key: de4507d33ceaa692121d87015172bea4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:36.552099 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(de4507d33ceaa692121d87015172bea4): No space left in cache to reserve 83277 bytes, will continue without cache download 2024.10.22 01:05:36.557789 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_337_115} MergeTask::MergeProjectionsStage: Merge sorted 26485 rows, containing 13 columns (13 merged, 0 gathered) in 2.774635504 sec., 9545.397931302476 rows/sec., 4.07 MiB/sec. 2024.10.22 01:05:36.581016 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: de4507d33ceaa692121d87015172bea4, offset: 0, size: 0 2024.10.22 01:05:36.683644 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key de4507d33ceaa692121d87015172bea4 for removal 2024.10.22 01:05:36.686355 [ 1476 ] {} CacheMetadata: Key de4507d33ceaa692121d87015172bea4 is removed from metadata 2024.10.22 01:05:36.687184 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 04f43b073f873ef55fadcf0eb1f8dcc9, offset: 0, size: 0 2024.10.22 01:05:36.688801 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 04f43b073f873ef55fadcf0eb1f8dcc9 for removal 2024.10.22 01:05:36.695369 [ 1476 ] {} CacheMetadata: Key 04f43b073f873ef55fadcf0eb1f8dcc9 is removed from metadata 2024.10.22 01:05:36.969094 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_337_115} MergedBlockOutputStream: filled checksums 202410_1_337_115 (state Temporary) 2024.10.22 01:05:37.568538 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(14742dc5cbc02695f988a24e1f6e1cac) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 14742dc5cbc02695f988a24e1f6e1cac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:37.572263 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(14742dc5cbc02695f988a24e1f6e1cac): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:05:37.683941 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_337_115} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_337_115 to 202410_1_337_115 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:37.687772 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_337_115} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_336_114, 202410_337_337_0] -> 202410_1_337_115 2024.10.22 01:05:37.808721 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.60 MiB. 2024.10.22 01:05:37.710235 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 13.24 MiB, will set to 1.70 GiB (RSS), difference: 35.21 MiB 2024.10.22 01:05:38.008483 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_588_211} MergeTask::MergeProjectionsStage: Merge sorted 35203 rows, containing 13 columns (13 merged, 0 gathered) in 2.669123187 sec., 13188.975380176038 rows/sec., 2.05 MiB/sec. 2024.10.22 01:05:38.193667 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_279_77, 202410_280_280_0] 2024.10.22 01:05:38.198220 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_279_77, 202410_280_280_0] 2024.10.22 01:05:38.210479 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_279_77, 202410_280_280_0] 2024.10.22 01:05:38.255265 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_588_211} MergedBlockOutputStream: filled checksums 202410_1_588_211 (state Temporary) 2024.10.22 01:05:38.415906 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(de73a7720b55b9f17a389531adfc1a79) : [0, 71709]: Setting download as failed: File segment: [0, 71709], key: de73a7720b55b9f17a389531adfc1a79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:38.417291 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(de73a7720b55b9f17a389531adfc1a79): No space left in cache to reserve 71710 bytes, will continue without cache download 2024.10.22 01:05:38.460965 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: de73a7720b55b9f17a389531adfc1a79, offset: 0, size: 0 2024.10.22 01:05:38.463955 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key de73a7720b55b9f17a389531adfc1a79 for removal 2024.10.22 01:05:38.467375 [ 1476 ] {} CacheMetadata: Key de73a7720b55b9f17a389531adfc1a79 is removed from metadata 2024.10.22 01:05:38.472885 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 14742dc5cbc02695f988a24e1f6e1cac, offset: 0, size: 0 2024.10.22 01:05:38.478331 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 14742dc5cbc02695f988a24e1f6e1cac for removal 2024.10.22 01:05:38.482517 [ 1476 ] {} CacheMetadata: Key 14742dc5cbc02695f988a24e1f6e1cac is removed from metadata 2024.10.22 01:05:38.563807 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_588_211} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_588_211 to 202410_1_588_211 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:38.589608 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_588_211} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_587_210, 202410_588_588_0] -> 202410_1_588_211 2024.10.22 01:05:38.622464 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:05:39.491484 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6b050ab240b5016a40cfe151cd1e3008) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 6b050ab240b5016a40cfe151cd1e3008, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:39.568886 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 34775 2024.10.22 01:05:39.572037 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6b050ab240b5016a40cfe151cd1e3008): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:05:39.877198 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:05:39.884386 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:05:39.965553 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:39.975002 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:40.176514 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_580_580_0 (state Temporary) 2024.10.22 01:05:40.222589 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_580_580_0 to 202410_593_593_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:40.384864 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_588_211 to 202410_589_589_0 2024.10.22 01:05:40.393251 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.24 MiB using storage policy from min volume index 0 2024.10.22 01:05:40.399042 [ 1254 ] {} DiskLocal: Reserved 1.24 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:40.452280 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_589_212} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_588_211 to 202410_589_589_0 into Compact with storage Full 2024.10.22 01:05:40.500246 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_589_212} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:40.514078 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_589_212} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_588_211, total 35203 rows starting from the beginning of the part 2024.10.22 01:05:40.600287 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_331_66/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:05:40.627331 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_589_212} MergeTreeSequentialSource: Reading 2 marks from part 202410_589_589_0, total 107 rows starting from the beginning of the part 2024.10.22 01:05:40.728008 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34775 2024.10.22 01:05:40.810564 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(39f6239bd9edc8d008b1a775562697f5) : [0, 69634]: Setting download as failed: File segment: [0, 69634], key: 39f6239bd9edc8d008b1a775562697f5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:40.812137 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(39f6239bd9edc8d008b1a775562697f5): No space left in cache to reserve 69635 bytes, will continue without cache download 2024.10.22 01:05:40.946935 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 39f6239bd9edc8d008b1a775562697f5, offset: 0, size: 0 2024.10.22 01:05:40.951622 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 39f6239bd9edc8d008b1a775562697f5 for removal 2024.10.22 01:05:40.955794 [ 1476 ] {} CacheMetadata: Key 39f6239bd9edc8d008b1a775562697f5 is removed from metadata 2024.10.22 01:05:40.966499 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6b050ab240b5016a40cfe151cd1e3008, offset: 0, size: 0 2024.10.22 01:05:40.968536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6b050ab240b5016a40cfe151cd1e3008 for removal 2024.10.22 01:05:40.972271 [ 1476 ] {} CacheMetadata: Key 6b050ab240b5016a40cfe151cd1e3008 is removed from metadata 2024.10.22 01:05:41.395507 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 13.19 MiB, will set to 1.68 GiB (RSS), difference: -10.21 MiB 2024.10.22 01:05:41.444617 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 202 entries to flush up to offset 27177 2024.10.22 01:05:41.502492 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(0300fccae38fe675829ced06bb1210bd) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0300fccae38fe675829ced06bb1210bd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:05:41.514035 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 0300fccae38fe675829ced06bb1210bd, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:05:41.518206 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:41.551901 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 2760 2024.10.22 01:05:41.921789 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:41.927624 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:42.005751 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_337_337_0 (state Temporary) 2024.10.22 01:05:42.022223 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1385 entries to flush up to offset 536240 2024.10.22 01:05:42.150813 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e05dfd79d31365e3dfdb9c5f8b2738f7) : [0, 167]: Setting download as failed: File segment: [0, 167], key: e05dfd79d31365e3dfdb9c5f8b2738f7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:42.152287 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e05dfd79d31365e3dfdb9c5f8b2738f7): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:05:42.200975 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_337_337_0 to 202410_343_343_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:42.249156 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 27177 2024.10.22 01:05:42.345762 [ 1179 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_337_115 to 202410_338_338_0 2024.10.22 01:05:42.350589 [ 1179 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.43 MiB using storage policy from min volume index 0 2024.10.22 01:05:42.385279 [ 1179 ] {} DiskLocal: Reserved 1.43 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:42.409780 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_338_116} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_337_115 to 202410_338_338_0 into Wide with storage Full 2024.10.22 01:05:42.458420 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_338_116} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:42.478886 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_338_116} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_337_115, total 26485 rows starting from the beginning of the part 2024.10.22 01:05:42.564054 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_589_212/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:05:42.937243 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_338_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_338_338_0, total 110 rows starting from the beginning of the part 2024.10.22 01:05:42.988359 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2ad48dd65dfcc2331b724de109e02c85) : [0, 112135]: Setting download as failed: File segment: [0, 112135], key: 2ad48dd65dfcc2331b724de109e02c85, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:42.991757 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2ad48dd65dfcc2331b724de109e02c85): No space left in cache to reserve 112136 bytes, will continue without cache download 2024.10.22 01:05:43.018392 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2ad48dd65dfcc2331b724de109e02c85, offset: 0, size: 0 2024.10.22 01:05:43.032423 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2ad48dd65dfcc2331b724de109e02c85 for removal 2024.10.22 01:05:43.037038 [ 1476 ] {} CacheMetadata: Key 2ad48dd65dfcc2331b724de109e02c85 is removed from metadata 2024.10.22 01:05:43.039629 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e05dfd79d31365e3dfdb9c5f8b2738f7, offset: 0, size: 0 2024.10.22 01:05:43.044988 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e05dfd79d31365e3dfdb9c5f8b2738f7 for removal 2024.10.22 01:05:43.049136 [ 1476 ] {} CacheMetadata: Key e05dfd79d31365e3dfdb9c5f8b2738f7 is removed from metadata 2024.10.22 01:05:43.130370 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 293 2024.10.22 01:05:43.228091 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_589_212} MergeTask::MergeProjectionsStage: Merge sorted 35310 rows, containing 13 columns (13 merged, 0 gathered) in 2.794126732 sec., 12637.222068565787 rows/sec., 1.97 MiB/sec. 2024.10.22 01:05:43.304541 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_589_212} MergedBlockOutputStream: filled checksums 202410_1_589_212 (state Temporary) 2024.10.22 01:05:43.323875 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:43.330290 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:43.393996 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_275_275_0 (state Temporary) 2024.10.22 01:05:43.427147 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:43.430482 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:43.581087 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_275_275_0 to 202410_285_285_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:43.653914 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_326_326_0 (state Temporary) 2024.10.22 01:05:43.673837 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 5000 2024.10.22 01:05:43.746902 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(62919f4acad83a67a5097f159ec619b0) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 62919f4acad83a67a5097f159ec619b0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:43.749835 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(62919f4acad83a67a5097f159ec619b0): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:05:43.751953 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_326_326_0 to 202410_333_333_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:43.776444 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 293 2024.10.22 01:05:43.874808 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_589_212} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_589_212 to 202410_1_589_212 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:43.881702 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_589_212} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_588_211, 202410_589_589_0] -> 202410_1_589_212 2024.10.22 01:05:43.902106 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 536240 2024.10.22 01:05:43.910977 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:05:44.012335 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:44.014749 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:44.064143 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_341_341_0 (state Temporary) 2024.10.22 01:05:44.212691 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_341_341_0 to 202410_354_354_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:44.258455 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5000 2024.10.22 01:05:44.471904 [ 1163 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:05:44.486583 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.88 GiB, free memory in arenas 12.18 MiB, will set to 1.70 GiB (RSS), difference: -30.51 MiB 2024.10.22 01:05:44.491668 [ 1163 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:05:44.585716 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(f90074af79542f3d8c17805a08f984b7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f90074af79542f3d8c17805a08f984b7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:05:44.588127 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: f90074af79542f3d8c17805a08f984b7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:05:44.589746 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:44.739407 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 34861 2024.10.22 01:05:44.993167 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_338_116} MergeTask::MergeProjectionsStage: Merge sorted 26595 rows, containing 13 columns (13 merged, 0 gathered) in 2.588066869 sec., 10276.009603367014 rows/sec., 4.38 MiB/sec. 2024.10.22 01:05:45.000304 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:45.002076 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:45.082265 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_581_581_0 (state Temporary) 2024.10.22 01:05:45.118699 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_581_581_0 to 202410_594_594_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:45.168883 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_338_116} MergedBlockOutputStream: filled checksums 202410_1_338_116 (state Temporary) 2024.10.22 01:05:45.185607 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_589_212 to 202410_590_590_0 2024.10.22 01:05:45.190830 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.25 MiB using storage policy from min volume index 0 2024.10.22 01:05:45.192439 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34861 2024.10.22 01:05:45.192965 [ 1185 ] {} DiskLocal: Reserved 1.25 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:45.207876 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_590_213} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_589_212 to 202410_590_590_0 into Compact with storage Full 2024.10.22 01:05:45.364838 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(cfd1ab1c938b59de51fd45d2f7ac85f1) : [0, 47722]: Setting download as failed: File segment: [0, 47722], key: cfd1ab1c938b59de51fd45d2f7ac85f1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:45.367481 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(cfd1ab1c938b59de51fd45d2f7ac85f1): No space left in cache to reserve 47723 bytes, will continue without cache download 2024.10.22 01:05:45.389010 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cfd1ab1c938b59de51fd45d2f7ac85f1, offset: 0, size: 0 2024.10.22 01:05:45.391486 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cfd1ab1c938b59de51fd45d2f7ac85f1 for removal 2024.10.22 01:05:45.394745 [ 1476 ] {} CacheMetadata: Key cfd1ab1c938b59de51fd45d2f7ac85f1 is removed from metadata 2024.10.22 01:05:45.395793 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 62919f4acad83a67a5097f159ec619b0, offset: 0, size: 0 2024.10.22 01:05:45.397264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 62919f4acad83a67a5097f159ec619b0 for removal 2024.10.22 01:05:45.400585 [ 1476 ] {} CacheMetadata: Key 62919f4acad83a67a5097f159ec619b0 is removed from metadata 2024.10.22 01:05:45.456404 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_590_213} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:45.472043 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_590_213} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_589_212, total 35310 rows starting from the beginning of the part 2024.10.22 01:05:45.514827 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_590_213} MergeTreeSequentialSource: Reading 2 marks from part 202410_590_590_0, total 74 rows starting from the beginning of the part 2024.10.22 01:05:45.793353 [ 1193 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 01:05:45.809748 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} MergeTask::MergeProjectionsStage: Merge sorted 547140 rows, containing 4 columns (4 merged, 0 gathered) in 19.648820198 sec., 27845.946702473866 rows/sec., 408.21 KiB/sec. 2024.10.22 01:05:45.948384 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_338_116} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_338_116 to 202410_1_338_116 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:45.955229 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_338_116} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_337_115, 202410_338_338_0] -> 202410_1_338_116 2024.10.22 01:05:46.079378 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} MergedBlockOutputStream: filled checksums 202410_1_331_66 (state Temporary) 2024.10.22 01:05:46.141510 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_331_66 to 202410_1_331_66 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:46.147925 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_331_66} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_326_65, 202410_331_331_0] -> 202410_1_331_66 2024.10.22 01:05:46.183904 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.52 MiB. 2024.10.22 01:05:46.191187 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:05:46.388022 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 11.35 MiB, will set to 1.69 GiB (RSS), difference: 34.87 MiB 2024.10.22 01:05:46.395581 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a468073a1c5ccbe9451d2887a4dba1d3) : [0, 157]: Setting download as failed: File segment: [0, 157], key: a468073a1c5ccbe9451d2887a4dba1d3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:46.396676 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a468073a1c5ccbe9451d2887a4dba1d3): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:05:46.545307 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_280_78, 202410_281_281_0] 2024.10.22 01:05:46.553503 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_280_78, 202410_281_281_0] 2024.10.22 01:05:46.588700 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_69_14/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:05:46.669578 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_280_78, 202410_281_281_0] 2024.10.22 01:05:47.207254 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(cf21a592f69761aebb973eca63aca0fe) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cf21a592f69761aebb973eca63aca0fe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:05:47.223150 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1243, file segment info: File segment: [0, 33554431], key: cf21a592f69761aebb973eca63aca0fe, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:05:47.227605 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:47.260238 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_590_213} MergeTask::MergeProjectionsStage: Merge sorted 35384 rows, containing 13 columns (13 merged, 0 gathered) in 2.056850515 sec., 17203.000287067534 rows/sec., 2.68 MiB/sec. 2024.10.22 01:05:47.423218 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_590_213} MergedBlockOutputStream: filled checksums 202410_1_590_213 (state Temporary) 2024.10.22 01:05:47.493760 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_590_213} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_590_213 to 202410_1_590_213 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:47.497616 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_590_213} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_589_212, 202410_590_590_0] -> 202410_1_590_213 2024.10.22 01:05:47.525407 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:05:47.988918 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(01cdf86dbdbdba844851f7b5b2815d9e) : [0, 283426]: Setting download as failed: File segment: [0, 283426], key: 01cdf86dbdbdba844851f7b5b2815d9e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:47.990512 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(01cdf86dbdbdba844851f7b5b2815d9e): No space left in cache to reserve 283427 bytes, will continue without cache download 2024.10.22 01:05:48.011119 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 01cdf86dbdbdba844851f7b5b2815d9e, offset: 0, size: 0 2024.10.22 01:05:48.012273 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 01cdf86dbdbdba844851f7b5b2815d9e for removal 2024.10.22 01:05:48.018133 [ 1476 ] {} CacheMetadata: Key 01cdf86dbdbdba844851f7b5b2815d9e is removed from metadata 2024.10.22 01:05:48.018981 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a468073a1c5ccbe9451d2887a4dba1d3, offset: 0, size: 0 2024.10.22 01:05:48.020456 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a468073a1c5ccbe9451d2887a4dba1d3 for removal 2024.10.22 01:05:48.023668 [ 1476 ] {} CacheMetadata: Key a468073a1c5ccbe9451d2887a4dba1d3 is removed from metadata 2024.10.22 01:05:48.202847 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1_488_129, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0] 2024.10.22 01:05:48.212128 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1_488_129, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0] 2024.10.22 01:05:48.229359 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1_488_129, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0] 2024.10.22 01:05:48.391805 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(3d09ef420d6bfa021bfeae762e1b5088) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3d09ef420d6bfa021bfeae762e1b5088, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:05:48.395165 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1004, file segment info: File segment: [0, 33554431], key: 3d09ef420d6bfa021bfeae762e1b5088, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:05:48.397399 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:48.571434 [ 1744 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hhf/laqpseenvemgijhkptekgpxvmkmal, size 3088250 2024.10.22 01:05:48.604851 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:05:48.611033 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:05:48.617414 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:05:48.867172 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:05:48.880546 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:05:48.887013 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:05:48.887096 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5837be4cd5c25c395f8c25028455f6eb) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 5837be4cd5c25c395f8c25028455f6eb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:48.889656 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5837be4cd5c25c395f8c25028455f6eb): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:05:48.899196 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:05:48.907298 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:05:48.921777 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:05:49.072506 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wgg/ncjwzhbuhdghjcqtvqdaeccrxfncb (LogSeriesLimiter: on interval from 2024-10-22 01:05:35 to 2024-10-22 01:05:49 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:05:49.202450 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 34935 2024.10.22 01:05:49.381316 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 8.77 MiB, will set to 1.67 GiB (RSS), difference: 5.81 MiB 2024.10.22 01:05:49.489162 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:49.490964 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:49.576105 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(62d55f44142d51e37516b268d6ec5991) : [0, 223374]: Setting download as failed: File segment: [0, 223374], key: 62d55f44142d51e37516b268d6ec5991, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:49.578827 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(62d55f44142d51e37516b268d6ec5991): No space left in cache to reserve 223375 bytes, will continue without cache download 2024.10.22 01:05:49.590147 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 62d55f44142d51e37516b268d6ec5991, offset: 0, size: 0 2024.10.22 01:05:49.591329 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 62d55f44142d51e37516b268d6ec5991 for removal 2024.10.22 01:05:49.596248 [ 1476 ] {} CacheMetadata: Key 62d55f44142d51e37516b268d6ec5991 is removed from metadata 2024.10.22 01:05:49.597022 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5837be4cd5c25c395f8c25028455f6eb, offset: 0, size: 0 2024.10.22 01:05:49.599118 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5837be4cd5c25c395f8c25028455f6eb for removal 2024.10.22 01:05:49.600884 [ 1476 ] {} CacheMetadata: Key 5837be4cd5c25c395f8c25028455f6eb is removed from metadata 2024.10.22 01:05:49.784657 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 187 entries to flush up to offset 27364 2024.10.22 01:05:49.875788 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:05:49.880355 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:05:49.966899 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_582_582_0 (state Temporary) 2024.10.22 01:05:50.008132 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_582_582_0 to 202410_595_595_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:50.040149 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_590_213 to 202410_591_591_0 2024.10.22 01:05:50.049462 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.25 MiB using storage policy from min volume index 0 2024.10.22 01:05:50.065677 [ 1262 ] {} DiskLocal: Reserved 1.25 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:50.084981 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_591_214} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_590_213 to 202410_591_591_0 into Compact with storage Full 2024.10.22 01:05:50.104498 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_591_214} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:50.125519 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_591_214} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_590_213, total 35384 rows starting from the beginning of the part 2024.10.22 01:05:50.209405 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:50.212607 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:50.232433 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} MergeTask::MergeProjectionsStage: Merge sorted 2709 rows, containing 679 columns (679 merged, 0 gathered) in 97.104376023 sec., 27.8978158446572 rows/sec., 37.49 KiB/sec. 2024.10.22 01:05:50.268669 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_338_338_0 (state Temporary) 2024.10.22 01:05:50.286216 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 34935 2024.10.22 01:05:50.289416 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_591_214} MergeTreeSequentialSource: Reading 2 marks from part 202410_591_591_0, total 80 rows starting from the beginning of the part 2024.10.22 01:05:50.363251 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(eb22a7e59a2fe7203961008071cc227c) : [0, 150]: Setting download as failed: File segment: [0, 150], key: eb22a7e59a2fe7203961008071cc227c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:50.364497 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_338_338_0 to 202410_344_344_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:50.365271 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(eb22a7e59a2fe7203961008071cc227c): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:05:50.384479 [ 1269 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_338_116 to 202410_339_339_0 2024.10.22 01:05:50.388123 [ 1269 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.44 MiB using storage policy from min volume index 0 2024.10.22 01:05:50.393672 [ 1269 ] {} DiskLocal: Reserved 1.44 MiB on local disk `default`, having unreserved 297.53 GiB. 2024.10.22 01:05:50.400503 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 27364 2024.10.22 01:05:50.592387 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_339_117} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_338_116 to 202410_339_339_0 into Wide with storage Full 2024.10.22 01:05:50.779084 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_339_117} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:50.793454 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_339_117} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_338_116, total 26595 rows starting from the beginning of the part 2024.10.22 01:05:50.906940 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 537776 2024.10.22 01:05:51.008277 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_339_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_339_339_0, total 123 rows starting from the beginning of the part 2024.10.22 01:05:51.187383 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6af5b3d7752c91bf46891c23e18c88bc) : [0, 5127]: Setting download as failed: File segment: [0, 5127], key: 6af5b3d7752c91bf46891c23e18c88bc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:51.188894 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6af5b3d7752c91bf46891c23e18c88bc): No space left in cache to reserve 5128 bytes, will continue without cache download 2024.10.22 01:05:51.204318 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6af5b3d7752c91bf46891c23e18c88bc, offset: 0, size: 0 2024.10.22 01:05:51.206526 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6af5b3d7752c91bf46891c23e18c88bc for removal 2024.10.22 01:05:51.212936 [ 1476 ] {} CacheMetadata: Key 6af5b3d7752c91bf46891c23e18c88bc is removed from metadata 2024.10.22 01:05:51.213072 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eb22a7e59a2fe7203961008071cc227c, offset: 0, size: 0 2024.10.22 01:05:51.226525 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eb22a7e59a2fe7203961008071cc227c for removal 2024.10.22 01:05:51.230758 [ 1476 ] {} CacheMetadata: Key eb22a7e59a2fe7203961008071cc227c is removed from metadata 2024.10.22 01:05:51.283135 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 294 2024.10.22 01:05:51.509644 [ 1197 ] {} DatabaseCatalog: Execution took 218 ms. 2024.10.22 01:05:51.511090 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} MergedBlockOutputStream: filled checksums 202410_1_69_14 (state Temporary) 2024.10.22 01:05:51.528514 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.88 GiB, free memory in arenas 10.52 MiB, will set to 1.68 GiB (RSS), difference: -14.55 MiB 2024.10.22 01:05:51.637385 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:51.640597 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:05:51.667440 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:51.673231 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:05:51.763464 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 5021 2024.10.22 01:05:51.812102 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_327_327_0 (state Temporary) 2024.10.22 01:05:51.876104 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:51.880212 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:05:51.991854 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(c459df21e16892ff3a6fe091a6201d2d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c459df21e16892ff3a6fe091a6201d2d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:05:52.004408 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_327_327_0 to 202410_334_334_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:52.013002 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_276_276_0 (state Temporary) 2024.10.22 01:05:52.058836 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_342_342_0 (state Temporary) 2024.10.22 01:05:52.095152 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 930, file segment info: File segment: [0, 33554431], key: c459df21e16892ff3a6fe091a6201d2d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:05:52.095974 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_342_342_0 to 202410_355_355_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:52.099361 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:52.186975 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 537776 2024.10.22 01:05:52.193108 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5021 2024.10.22 01:05:52.198405 [ 1734 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/eel/notjelerexwituiribwosnnivgqjw, size 3224825 2024.10.22 01:05:52.232003 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_276_276_0 to 202410_286_286_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:52.319150 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 294 2024.10.22 01:05:52.454471 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:05:52.481116 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:05:52.494613 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:05:52.504660 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:05:52.544909 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:05:52.560484 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:05:52.579318 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:05:52.678526 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_591_214} MergeTask::MergeProjectionsStage: Merge sorted 35464 rows, containing 13 columns (13 merged, 0 gathered) in 2.600779507 sec., 13635.911812035052 rows/sec., 2.12 MiB/sec. 2024.10.22 01:05:52.788983 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1e0630e52603ccd869dd842d81601707) : [0, 168]: Setting download as failed: File segment: [0, 168], key: 1e0630e52603ccd869dd842d81601707, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:52.789947 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1e0630e52603ccd869dd842d81601707): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:05:52.831433 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_591_214} MergedBlockOutputStream: filled checksums 202410_1_591_214 (state Temporary) 2024.10.22 01:05:53.167371 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_591_214} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_591_214 to 202410_1_591_214 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:53.172575 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_591_214} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_590_213, 202410_591_591_0] -> 202410_1_591_214 2024.10.22 01:05:53.199497 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:05:53.511280 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(08af645406db54e36b4d3a2036e72b22) : [0, 161402]: Setting download as failed: File segment: [0, 161402], key: 08af645406db54e36b4d3a2036e72b22, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:53.512772 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(08af645406db54e36b4d3a2036e72b22): No space left in cache to reserve 161403 bytes, will continue without cache download 2024.10.22 01:05:53.753563 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 08af645406db54e36b4d3a2036e72b22, offset: 0, size: 0 2024.10.22 01:05:53.851746 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_339_117} MergeTask::MergeProjectionsStage: Merge sorted 26718 rows, containing 13 columns (13 merged, 0 gathered) in 3.366698031 sec., 7935.965671404167 rows/sec., 3.38 MiB/sec. 2024.10.22 01:05:53.880270 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 08af645406db54e36b4d3a2036e72b22 for removal 2024.10.22 01:05:53.884632 [ 1476 ] {} CacheMetadata: Key 08af645406db54e36b4d3a2036e72b22 is removed from metadata 2024.10.22 01:05:53.887992 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1e0630e52603ccd869dd842d81601707, offset: 0, size: 0 2024.10.22 01:05:53.889802 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1e0630e52603ccd869dd842d81601707 for removal 2024.10.22 01:05:53.894816 [ 1476 ] {} CacheMetadata: Key 1e0630e52603ccd869dd842d81601707 is removed from metadata 2024.10.22 01:05:54.087117 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_339_117} MergedBlockOutputStream: filled checksums 202410_1_339_117 (state Temporary) 2024.10.22 01:05:54.295298 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 97 entries to flush up to offset 35032 2024.10.22 01:05:54.584303 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:54.587749 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:05:54.682434 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_583_583_0 (state Temporary) 2024.10.22 01:05:54.739102 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 9.79 MiB, will set to 1.67 GiB (RSS), difference: 12.91 MiB 2024.10.22 01:05:54.988944 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2559cb0f63887a33423a5837a1652d50) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 2559cb0f63887a33423a5837a1652d50, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:54.991508 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2559cb0f63887a33423a5837a1652d50): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:05:55.002181 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_339_117} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_339_117 to 202410_1_339_117 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:55.006094 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_339_117} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_338_116, 202410_339_339_0] -> 202410_1_339_117 2024.10.22 01:05:55.028301 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.56 MiB. 2024.10.22 01:05:55.288214 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_583_583_0 to 202410_596_596_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:55.328557 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_591_214 to 202410_592_592_0 2024.10.22 01:05:55.335519 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.25 MiB using storage policy from min volume index 0 2024.10.22 01:05:55.377328 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_281_79, 202410_282_282_0] 2024.10.22 01:05:55.381053 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_281_79, 202410_282_282_0] 2024.10.22 01:05:55.395989 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_281_79, 202410_282_282_0] 2024.10.22 01:05:55.445409 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35032 2024.10.22 01:05:55.454891 [ 1216 ] {} DiskLocal: Reserved 1.25 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:05:55.531230 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_592_215} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_591_214 to 202410_592_592_0 into Compact with storage Full 2024.10.22 01:05:55.568976 [ 1216 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 272 ms. 2024.10.22 01:05:55.681808 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_592_215} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:55.689353 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_592_215} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_591_214, total 35464 rows starting from the beginning of the part 2024.10.22 01:05:55.842800 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_592_215} MergeTreeSequentialSource: Reading 2 marks from part 202410_592_592_0, total 73 rows starting from the beginning of the part 2024.10.22 01:05:55.995716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1bcf96091aeb04860eb9467e21102266) : [0, 231967]: Setting download as failed: File segment: [0, 231967], key: 1bcf96091aeb04860eb9467e21102266, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:55.996922 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1bcf96091aeb04860eb9467e21102266): No space left in cache to reserve 231968 bytes, will continue without cache download 2024.10.22 01:05:56.015390 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1bcf96091aeb04860eb9467e21102266, offset: 0, size: 0 2024.10.22 01:05:56.017629 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1bcf96091aeb04860eb9467e21102266 for removal 2024.10.22 01:05:56.028378 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2559cb0f63887a33423a5837a1652d50, offset: 0, size: 0 2024.10.22 01:05:56.030809 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2559cb0f63887a33423a5837a1652d50 for removal 2024.10.22 01:05:56.038379 [ 1476 ] {} CacheMetadata: Key 1bcf96091aeb04860eb9467e21102266 is removed from metadata 2024.10.22 01:05:56.050826 [ 1476 ] {} CacheMetadata: Key 2559cb0f63887a33423a5837a1652d50 is removed from metadata 2024.10.22 01:05:56.402728 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 8.45 MiB, will set to 1.67 GiB (RSS), difference: -5.65 MiB 2024.10.22 01:05:56.452161 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:56.462540 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:05:56.869055 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d12e6f759e8dbad1ea143df8f0bc0d37) : [0, 157]: Setting download as failed: File segment: [0, 157], key: d12e6f759e8dbad1ea143df8f0bc0d37, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:56.873472 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d12e6f759e8dbad1ea143df8f0bc0d37): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:05:57.752394 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(889e214d8a6f3b0dc57a35962fdd4876) : [0, 293305]: Setting download as failed: File segment: [0, 293305], key: 889e214d8a6f3b0dc57a35962fdd4876, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:57.754236 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(889e214d8a6f3b0dc57a35962fdd4876): No space left in cache to reserve 293306 bytes, will continue without cache download 2024.10.22 01:05:57.782378 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 889e214d8a6f3b0dc57a35962fdd4876, offset: 0, size: 0 2024.10.22 01:05:57.787121 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 889e214d8a6f3b0dc57a35962fdd4876 for removal 2024.10.22 01:05:57.789389 [ 1476 ] {} CacheMetadata: Key 889e214d8a6f3b0dc57a35962fdd4876 is removed from metadata 2024.10.22 01:05:57.790453 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d12e6f759e8dbad1ea143df8f0bc0d37, offset: 0, size: 0 2024.10.22 01:05:57.791534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d12e6f759e8dbad1ea143df8f0bc0d37 for removal 2024.10.22 01:05:57.793027 [ 1476 ] {} CacheMetadata: Key d12e6f759e8dbad1ea143df8f0bc0d37 is removed from metadata 2024.10.22 01:05:57.934330 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 202 entries to flush up to offset 27566 2024.10.22 01:05:58.057254 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_592_215} MergeTask::MergeProjectionsStage: Merge sorted 35537 rows, containing 13 columns (13 merged, 0 gathered) in 2.562503525 sec., 13868.07848391155 rows/sec., 2.16 MiB/sec. 2024.10.22 01:05:58.203640 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_592_215} MergedBlockOutputStream: filled checksums 202410_1_592_215 (state Temporary) 2024.10.22 01:05:58.277116 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_592_215} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_592_215 to 202410_1_592_215 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:58.281581 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_592_215} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_591_214, 202410_592_592_0] -> 202410_1_592_215 2024.10.22 01:05:58.309045 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:05:58.310637 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:58.412168 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:05:58.418867 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(f7995ca9d9be970eae390027c4dbcb8e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f7995ca9d9be970eae390027c4dbcb8e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:05:58.423779 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: f7995ca9d9be970eae390027c4dbcb8e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:05:58.427506 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:05:58.576163 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_339_339_0 (state Temporary) 2024.10.22 01:05:58.667201 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 7.08 MiB, will set to 1.67 GiB (RSS), difference: 3.91 MiB 2024.10.22 01:05:58.784055 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_339_339_0 to 202410_345_345_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:05:58.836909 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_339_117 to 202410_340_340_0 2024.10.22 01:05:58.850393 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.44 MiB using storage policy from min volume index 0 2024.10.22 01:05:58.852185 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 27566 2024.10.22 01:05:58.860771 [ 1256 ] {} DiskLocal: Reserved 1.44 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:05:58.885459 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_340_118} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_339_117 to 202410_340_340_0 into Wide with storage Full 2024.10.22 01:05:59.140165 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bc84c306b7d57044f963fbf32ac0c437) : [0, 158]: Setting download as failed: File segment: [0, 158], key: bc84c306b7d57044f963fbf32ac0c437, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:05:59.143026 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bc84c306b7d57044f963fbf32ac0c437): No space left in cache to reserve 159 bytes, will continue without cache download 2024.10.22 01:05:59.174123 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_340_118} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:05:59.191564 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 538928 2024.10.22 01:05:59.196375 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_340_118} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_339_117, total 26718 rows starting from the beginning of the part 2024.10.22 01:05:59.402927 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_69_69_0 (state Temporary) 2024.10.22 01:05:59.499579 [ 1243 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:05:59.578008 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 35101 2024.10.22 01:05:59.695971 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 5035 2024.10.22 01:05:59.704455 [ 1243 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:05:59.711682 [ 1243 ] {} DNSCacheUpdater: Execution took 212 ms. 2024.10.22 01:05:59.736024 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:59.739975 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:05:59.765504 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_69_14/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:05:59.839829 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:05:59.846951 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:05:59.879966 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:05:59.907038 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_584_584_0 (state Temporary) 2024.10.22 01:05:59.907336 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:05:59.908077 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_340_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_340_340_0, total 97 rows starting from the beginning of the part 2024.10.22 01:05:59.949412 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_343_343_0 (state Temporary) 2024.10.22 01:05:59.972554 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_584_584_0 to 202410_597_597_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:00.020203 [ 1246 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_592_215 to 202410_593_593_0 2024.10.22 01:06:00.024533 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35101 2024.10.22 01:06:00.032320 [ 1246 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.25 MiB using storage policy from min volume index 0 2024.10.22 01:06:00.037051 [ 1246 ] {} DiskLocal: Reserved 1.25 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:06:00.063269 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_593_216} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_592_215 to 202410_593_593_0 into Compact with storage Full 2024.10.22 01:06:00.102169 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_343_343_0 to 202410_356_356_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:00.190120 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9d5109871f383daeacf92f21c2fc4646) : [0, 380055]: Setting download as failed: File segment: [0, 380055], key: 9d5109871f383daeacf92f21c2fc4646, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:00.191566 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9d5109871f383daeacf92f21c2fc4646): No space left in cache to reserve 380056 bytes, will continue without cache download 2024.10.22 01:06:00.238665 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_351_70 to 202410_356_356_0 2024.10.22 01:06:00.268536 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:00.272971 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:06:00.306190 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_593_216} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:00.316302 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9d5109871f383daeacf92f21c2fc4646, offset: 0, size: 0 2024.10.22 01:06:00.320861 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9d5109871f383daeacf92f21c2fc4646 for removal 2024.10.22 01:06:00.325615 [ 1476 ] {} CacheMetadata: Key 9d5109871f383daeacf92f21c2fc4646 is removed from metadata 2024.10.22 01:06:00.329579 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bc84c306b7d57044f963fbf32ac0c437, offset: 0, size: 0 2024.10.22 01:06:00.334872 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:00.335295 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bc84c306b7d57044f963fbf32ac0c437 for removal 2024.10.22 01:06:00.338052 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:06:00.338767 [ 1476 ] {} CacheMetadata: Key bc84c306b7d57044f963fbf32ac0c437 is removed from metadata 2024.10.22 01:06:00.408100 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_351_70 to 202410_356_356_0 into Compact with storage Full 2024.10.22 01:06:00.321789 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5035 2024.10.22 01:06:00.467936 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:00.405828 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_593_216} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_592_215, total 35537 rows starting from the beginning of the part 2024.10.22 01:06:00.584400 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_351_70, total 5067 rows starting from the beginning of the part 2024.10.22 01:06:00.586190 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_291_58, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0, 202410_296_296_0] 2024.10.22 01:06:00.588630 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_291_58, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0, 202410_296_296_0] 2024.10.22 01:06:00.600177 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_593_216} MergeTreeSequentialSource: Reading 2 marks from part 202410_593_593_0, total 69 rows starting from the beginning of the part 2024.10.22 01:06:00.611911 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_291_58, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0, 202410_296_296_0] 2024.10.22 01:06:00.618809 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_352_352_0, total 16 rows starting from the beginning of the part 2024.10.22 01:06:00.663223 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_353_353_0, total 13 rows starting from the beginning of the part 2024.10.22 01:06:00.694022 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_354_354_0, total 10 rows starting from the beginning of the part 2024.10.22 01:06:00.766999 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_328_328_0 (state Temporary) 2024.10.22 01:06:00.813458 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_355_355_0, total 21 rows starting from the beginning of the part 2024.10.22 01:06:00.883262 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_328_328_0 to 202410_335_335_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:00.905598 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 538928 2024.10.22 01:06:01.061727 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_356_356_0, total 14 rows starting from the beginning of the part 2024.10.22 01:06:01.299020 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(77b05aeb8753192532554bd09f0a6f25) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 77b05aeb8753192532554bd09f0a6f25, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:01.300459 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(77b05aeb8753192532554bd09f0a6f25): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:06:01.398991 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.88 GiB, free memory in arenas 5.26 MiB, will set to 1.68 GiB (RSS), difference: -56.02 MiB 2024.10.22 01:06:01.975655 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_491_130, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:06:01.988409 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_491_130, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:06:02.009808 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_491_130, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:06:02.100368 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_340_118} MergeTask::MergeProjectionsStage: Merge sorted 26815 rows, containing 13 columns (13 merged, 0 gathered) in 3.22164421 sec., 8323.389627186672 rows/sec., 3.55 MiB/sec. 2024.10.22 01:06:02.287225 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_340_118} MergedBlockOutputStream: filled checksums 202410_1_340_118 (state Temporary) 2024.10.22 01:06:02.356073 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} MergeTask::MergeProjectionsStage: Merge sorted 5141 rows, containing 12 columns (12 merged, 0 gathered) in 1.971626331 sec., 2607.4920582910395 rows/sec., 410.64 KiB/sec. 2024.10.22 01:06:02.471264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d6dc2a8282bd118eba86062e66a49291) : [0, 562955]: Setting download as failed: File segment: [0, 562955], key: d6dc2a8282bd118eba86062e66a49291, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:02.474163 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d6dc2a8282bd118eba86062e66a49291): No space left in cache to reserve 562956 bytes, will continue without cache download 2024.10.22 01:06:02.498926 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} MergedBlockOutputStream: filled checksums 202410_1_356_71 (state Temporary) 2024.10.22 01:06:02.512297 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d6dc2a8282bd118eba86062e66a49291, offset: 0, size: 0 2024.10.22 01:06:02.514215 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d6dc2a8282bd118eba86062e66a49291 for removal 2024.10.22 01:06:02.517485 [ 1476 ] {} CacheMetadata: Key d6dc2a8282bd118eba86062e66a49291 is removed from metadata 2024.10.22 01:06:02.629299 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 77b05aeb8753192532554bd09f0a6f25, offset: 0, size: 0 2024.10.22 01:06:02.631475 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 77b05aeb8753192532554bd09f0a6f25 for removal 2024.10.22 01:06:02.635566 [ 1476 ] {} CacheMetadata: Key 77b05aeb8753192532554bd09f0a6f25 is removed from metadata 2024.10.22 01:06:02.787293 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_356_71 to 202410_1_356_71 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:02.791955 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_356_71} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_351_70, 202410_356_356_0] -> 202410_1_356_71 2024.10.22 01:06:02.924347 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.43 MiB. 2024.10.22 01:06:02.995075 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_340_118} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_340_118 to 202410_1_340_118 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:02.997061 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_340_118} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_339_117, 202410_340_340_0] -> 202410_1_340_118 2024.10.22 01:06:03.016237 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.67 MiB. 2024.10.22 01:06:03.278332 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2de7b5863d659857453ce37da882f86b) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 2de7b5863d659857453ce37da882f86b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:03.279470 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2de7b5863d659857453ce37da882f86b): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:06:03.335889 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 4.38 MiB, will set to 1.67 GiB (RSS), difference: 47.51 MiB 2024.10.22 01:06:03.500462 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_593_216} MergeTask::MergeProjectionsStage: Merge sorted 35606 rows, containing 13 columns (13 merged, 0 gathered) in 3.447582418 sec., 10327.816911380942 rows/sec., 1.61 MiB/sec. 2024.10.22 01:06:03.654690 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_282_80, 202410_283_283_0] 2024.10.22 01:06:03.654452 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_593_216} MergedBlockOutputStream: filled checksums 202410_1_593_216 (state Temporary) 2024.10.22 01:06:03.660024 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_282_80, 202410_283_283_0] 2024.10.22 01:06:03.674063 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_282_80, 202410_283_283_0] 2024.10.22 01:06:03.700381 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_593_216} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_593_216 to 202410_1_593_216 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:03.702711 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_593_216} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_592_215, 202410_593_593_0] -> 202410_1_593_216 2024.10.22 01:06:03.912860 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:06:04.040085 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 35189 2024.10.22 01:06:04.089584 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4c1a768ae7e3ec32e0d613e00b9c8942) : [0, 557792]: Setting download as failed: File segment: [0, 557792], key: 4c1a768ae7e3ec32e0d613e00b9c8942, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:04.091073 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4c1a768ae7e3ec32e0d613e00b9c8942): No space left in cache to reserve 557793 bytes, will continue without cache download 2024.10.22 01:06:04.132245 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4c1a768ae7e3ec32e0d613e00b9c8942, offset: 0, size: 0 2024.10.22 01:06:04.136485 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4c1a768ae7e3ec32e0d613e00b9c8942 for removal 2024.10.22 01:06:04.139671 [ 1476 ] {} CacheMetadata: Key 4c1a768ae7e3ec32e0d613e00b9c8942 is removed from metadata 2024.10.22 01:06:04.141370 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2de7b5863d659857453ce37da882f86b, offset: 0, size: 0 2024.10.22 01:06:04.143590 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2de7b5863d659857453ce37da882f86b for removal 2024.10.22 01:06:04.146886 [ 1476 ] {} CacheMetadata: Key 2de7b5863d659857453ce37da882f86b is removed from metadata 2024.10.22 01:06:04.147148 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_271_54, 202410_272_272_0, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0] 2024.10.22 01:06:04.150415 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_271_54, 202410_272_272_0, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0] 2024.10.22 01:06:04.188018 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_271_54, 202410_272_272_0, 202410_273_273_0, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0] 2024.10.22 01:06:04.196779 [ 1156 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:06:04.329688 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:04.335357 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:06:04.539520 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_585_585_0 (state Temporary) 2024.10.22 01:06:04.609180 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_585_585_0 to 202410_598_598_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:04.650471 [ 1169 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_593_216 to 202410_594_594_0 2024.10.22 01:06:04.656909 [ 1169 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.26 MiB using storage policy from min volume index 0 2024.10.22 01:06:04.676786 [ 1169 ] {} DiskLocal: Reserved 1.26 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:06:04.700863 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_594_217} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_593_216 to 202410_594_594_0 into Compact with storage Full 2024.10.22 01:06:04.737458 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_594_217} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:04.754978 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_594_217} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_593_216, total 35606 rows starting from the beginning of the part 2024.10.22 01:06:04.779577 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_594_217} MergeTreeSequentialSource: Reading 2 marks from part 202410_594_594_0, total 86 rows starting from the beginning of the part 2024.10.22 01:06:04.790128 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35189 2024.10.22 01:06:04.890753 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7c8cb6b0ebccd928fff4f7f84f431530) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7c8cb6b0ebccd928fff4f7f84f431530, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:06:04.904489 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 7c8cb6b0ebccd928fff4f7f84f431530, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:06:04.907943 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:05.571016 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_69_14 to 202410_1_69_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:05.575580 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_69_14} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_64_13, 202410_69_69_0] -> 202410_1_69_14 2024.10.22 01:06:05.607686 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 4.75 MiB, will set to 1.66 GiB (RSS), difference: -10.42 MiB 2024.10.22 01:06:05.860000 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(96fa35ef648ce12cf91e1b78f1d6efc7) : [0, 158]: Setting download as failed: File segment: [0, 158], key: 96fa35ef648ce12cf91e1b78f1d6efc7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:05.863312 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(96fa35ef648ce12cf91e1b78f1d6efc7): No space left in cache to reserve 159 bytes, will continue without cache download 2024.10.22 01:06:06.372896 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 37.65 MiB. 2024.10.22 01:06:06.599219 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_69_69_0 to 202410_72_72_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:06.604669 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 214 entries to flush up to offset 27780 2024.10.22 01:06:06.721564 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e6ccba687a69221c786b5e061b3b8ab2) : [0, 351757]: Setting download as failed: File segment: [0, 351757], key: e6ccba687a69221c786b5e061b3b8ab2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:06.723978 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e6ccba687a69221c786b5e061b3b8ab2): No space left in cache to reserve 351758 bytes, will continue without cache download 2024.10.22 01:06:06.755909 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:06.764390 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:06:06.828505 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_340_340_0 (state Temporary) 2024.10.22 01:06:06.907480 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_340_340_0 to 202410_346_346_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:06.931549 [ 1276 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_340_118 to 202410_341_341_0 2024.10.22 01:06:06.943903 [ 1276 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.45 MiB using storage policy from min volume index 0 2024.10.22 01:06:06.946744 [ 1276 ] {} DiskLocal: Reserved 1.45 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:06:06.964321 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e6ccba687a69221c786b5e061b3b8ab2, offset: 0, size: 0 2024.10.22 01:06:06.977270 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e6ccba687a69221c786b5e061b3b8ab2 for removal 2024.10.22 01:06:06.981821 [ 1476 ] {} CacheMetadata: Key e6ccba687a69221c786b5e061b3b8ab2 is removed from metadata 2024.10.22 01:06:06.984100 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 96fa35ef648ce12cf91e1b78f1d6efc7, offset: 0, size: 0 2024.10.22 01:06:06.987862 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 96fa35ef648ce12cf91e1b78f1d6efc7 for removal 2024.10.22 01:06:06.990822 [ 1476 ] {} CacheMetadata: Key 96fa35ef648ce12cf91e1b78f1d6efc7 is removed from metadata 2024.10.22 01:06:06.990891 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_341_119} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_340_118 to 202410_341_341_0 into Wide with storage Full 2024.10.22 01:06:07.045430 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 27780 2024.10.22 01:06:07.136529 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_341_119} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:07.249165 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_341_119} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_340_118, total 26815 rows starting from the beginning of the part 2024.10.22 01:06:07.294878 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_594_217} MergeTask::MergeProjectionsStage: Merge sorted 35692 rows, containing 13 columns (13 merged, 0 gathered) in 2.60297338 sec., 13712.011146268427 rows/sec., 2.14 MiB/sec. 2024.10.22 01:06:07.323338 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 295 2024.10.22 01:06:07.327440 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_594_217} MergedBlockOutputStream: filled checksums 202410_1_594_217 (state Temporary) 2024.10.22 01:06:07.380273 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 4.81 MiB, will set to 1.67 GiB (RSS), difference: 18.84 MiB 2024.10.22 01:06:07.398672 [ 1193 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_282_58 to 202410_286_286_0 2024.10.22 01:06:07.404026 [ 1193 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:07.408043 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:06:07.439322 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_286_59} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_282_58 to 202410_286_286_0 into Compact with storage Full 2024.10.22 01:06:07.487170 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_341_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_341_341_0, total 141 rows starting from the beginning of the part 2024.10.22 01:06:07.512169 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(8eb29eadc26178cf52acf2d19f68cbc3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8eb29eadc26178cf52acf2d19f68cbc3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 01:06:07.533746 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 914, file segment info: File segment: [0, 33554431], key: 8eb29eadc26178cf52acf2d19f68cbc3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 01:06:07.537561 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:07.640494 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_594_217} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_594_217 to 202410_1_594_217 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:07.645157 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_594_217} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_593_216, 202410_594_594_0] -> 202410_1_594_217 2024.10.22 01:06:07.689376 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:06:07.730477 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:07.733218 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.52 GiB. 2024.10.22 01:06:07.772711 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:06:07.804192 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_277_277_0 (state Temporary) 2024.10.22 01:06:07.807383 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_286_59} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:07.845611 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_286_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_282_58, total 326 rows starting from the beginning of the part 2024.10.22 01:06:07.909216 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:06:07.910667 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 540464 2024.10.22 01:06:07.924993 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 29 entries to flush up to offset 5064 2024.10.22 01:06:07.924890 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:06:07.953312 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:06:08.080550 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_277_277_0 to 202410_287_287_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:08.085808 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:06:08.098201 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:06:08.113648 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:06:08.153337 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:08.158271 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:08.205739 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_344_344_0 (state Temporary) 2024.10.22 01:06:08.253734 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1a47981868501febac6ebccd584fa329) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 1a47981868501febac6ebccd584fa329, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:08.256224 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1a47981868501febac6ebccd584fa329): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:06:08.265329 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_286_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_283_283_0, total 1 rows starting from the beginning of the part 2024.10.22 01:06:08.268153 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_344_344_0 to 202410_357_357_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:08.269452 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kth/qjfooosmbkbrnvplepsdhajcdtbga (LogSeriesLimiter: on interval from 2024-10-22 01:05:49 to 2024-10-22 01:06:08 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:06:08.319271 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5064 2024.10.22 01:06:08.340136 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 295 2024.10.22 01:06:08.488522 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_286_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_284_284_0, total 1 rows starting from the beginning of the part 2024.10.22 01:06:08.511987 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:08.514494 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:08.611169 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_329_329_0 (state Temporary) 2024.10.22 01:06:08.703148 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_329_329_0 to 202410_336_336_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:08.749158 [ 1154 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_331_66 to 202410_336_336_0 2024.10.22 01:06:08.751124 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 540464 2024.10.22 01:06:08.753136 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_286_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_285_285_0, total 1 rows starting from the beginning of the part 2024.10.22 01:06:08.779279 [ 1154 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:08.781343 [ 1154 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:08.801387 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 105 entries to flush up to offset 35294 2024.10.22 01:06:08.802764 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_331_66 to 202410_336_336_0 into Compact with storage Full 2024.10.22 01:06:08.797589 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(5e72ee16f6455ae9550450893e41673a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5e72ee16f6455ae9550450893e41673a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:06:08.809833 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 764, file segment info: File segment: [0, 33554431], key: 5e72ee16f6455ae9550450893e41673a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:06:08.813250 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:08.828929 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:08.838485 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} MergeTreeSequentialSource: Reading 68 marks from part 202410_1_331_66, total 547140 rows starting from the beginning of the part 2024.10.22 01:06:08.851259 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_332_332_0, total 1305 rows starting from the beginning of the part 2024.10.22 01:06:08.863817 [ 1704 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wgg/ncjwzhbuhdghjcqtvqdaeccrxfncb, size 1337556 2024.10.22 01:06:08.871114 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_333_333_0, total 1385 rows starting from the beginning of the part 2024.10.22 01:06:08.891635 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_334_334_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:06:08.895939 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_335_335_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:06:08.898052 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_286_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_286_286_0, total 1 rows starting from the beginning of the part 2024.10.22 01:06:08.917455 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_336_336_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:06:09.174500 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:06:09.187361 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:06:09.209390 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:06:09.233931 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:06:09.225480 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2760 2024.10.22 01:06:09.248972 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:06:09.281818 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:06:09.290465 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:09.293019 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:09.304811 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:06:09.334673 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:06:09.367195 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:06:09.384444 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_586_586_0 (state Temporary) 2024.10.22 01:06:09.479731 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_586_586_0 to 202410_599_599_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:09.502376 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_594_217 to 202410_595_595_0 2024.10.22 01:06:09.520159 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35294 2024.10.22 01:06:09.539125 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.26 MiB using storage policy from min volume index 0 2024.10.22 01:06:09.542262 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_341_119} MergeTask::MergeProjectionsStage: Merge sorted 26956 rows, containing 13 columns (13 merged, 0 gathered) in 2.526464278 sec., 10669.456217817142 rows/sec., 4.55 MiB/sec. 2024.10.22 01:06:09.543149 [ 1248 ] {} DiskLocal: Reserved 1.26 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:09.579676 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_595_218} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_594_217 to 202410_595_595_0 into Compact with storage Full 2024.10.22 01:06:09.588843 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(93338e043ac86db381c03c203ae1e0a6) : [0, 259831]: Setting download as failed: File segment: [0, 259831], key: 93338e043ac86db381c03c203ae1e0a6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:09.589605 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(93338e043ac86db381c03c203ae1e0a6): No space left in cache to reserve 259832 bytes, will continue without cache download 2024.10.22 01:06:09.598719 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_595_218} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:09.610756 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 93338e043ac86db381c03c203ae1e0a6, offset: 0, size: 0 2024.10.22 01:06:09.612962 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 93338e043ac86db381c03c203ae1e0a6 for removal 2024.10.22 01:06:09.614216 [ 1476 ] {} CacheMetadata: Key 93338e043ac86db381c03c203ae1e0a6 is removed from metadata 2024.10.22 01:06:09.616312 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1a47981868501febac6ebccd584fa329, offset: 0, size: 0 2024.10.22 01:06:09.620591 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1a47981868501febac6ebccd584fa329 for removal 2024.10.22 01:06:09.627292 [ 1476 ] {} CacheMetadata: Key 1a47981868501febac6ebccd584fa329 is removed from metadata 2024.10.22 01:06:09.626491 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_595_218} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_594_217, total 35692 rows starting from the beginning of the part 2024.10.22 01:06:09.845629 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_595_218} MergeTreeSequentialSource: Reading 2 marks from part 202410_595_595_0, total 74 rows starting from the beginning of the part 2024.10.22 01:06:09.880226 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:06:09.881578 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:06:09.956518 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_341_119} MergedBlockOutputStream: filled checksums 202410_1_341_119 (state Temporary) 2024.10.22 01:06:10.295784 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(06550ec459a9c3f82a7a6fbe956d62fa) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 06550ec459a9c3f82a7a6fbe956d62fa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:10.297148 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(06550ec459a9c3f82a7a6fbe956d62fa): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:06:10.495303 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 10.64 MiB, will set to 1.66 GiB (RSS), difference: -20.11 MiB 2024.10.22 01:06:10.508193 [ 1266 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 212 ms. 2024.10.22 01:06:10.575882 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_341_119} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_341_119 to 202410_1_341_119 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:10.579993 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_341_119} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_340_118, 202410_341_341_0] -> 202410_1_341_119 2024.10.22 01:06:10.604253 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.83 MiB. 2024.10.22 01:06:10.870921 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_286_59} MergeTask::MergeProjectionsStage: Merge sorted 330 rows, containing 32 columns (32 merged, 0 gathered) in 3.439257618 sec., 95.95093960768833 rows/sec., 9.76 KiB/sec. 2024.10.22 01:06:10.876861 [ 1263 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 01:06:10.887300 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_286_59/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:06:10.890963 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_286_59} MergedBlockOutputStream: filled checksums 202410_1_286_59 (state Temporary) 2024.10.22 01:06:11.004021 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_234_48, 202410_235_235_0, 202410_236_236_0, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0] 2024.10.22 01:06:11.107793 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_234_48, 202410_235_235_0, 202410_236_236_0, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0] 2024.10.22 01:06:11.143991 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_234_48, 202410_235_235_0, 202410_236_236_0, 202410_237_237_0, 202410_238_238_0, 202410_239_239_0] 2024.10.22 01:06:11.200186 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(739407ff4a328295700bdc1f93f72a19) : [0, 530534]: Setting download as failed: File segment: [0, 530534], key: 739407ff4a328295700bdc1f93f72a19, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:11.202158 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(739407ff4a328295700bdc1f93f72a19): No space left in cache to reserve 530535 bytes, will continue without cache download 2024.10.22 01:06:11.336092 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 739407ff4a328295700bdc1f93f72a19, offset: 0, size: 0 2024.10.22 01:06:11.338758 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 739407ff4a328295700bdc1f93f72a19 for removal 2024.10.22 01:06:11.345805 [ 1476 ] {} CacheMetadata: Key 739407ff4a328295700bdc1f93f72a19 is removed from metadata 2024.10.22 01:06:11.353671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 06550ec459a9c3f82a7a6fbe956d62fa, offset: 0, size: 0 2024.10.22 01:06:11.355661 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 06550ec459a9c3f82a7a6fbe956d62fa for removal 2024.10.22 01:06:11.458058 [ 1476 ] {} CacheMetadata: Key 06550ec459a9c3f82a7a6fbe956d62fa is removed from metadata 2024.10.22 01:06:11.944691 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_286_59} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_286_59 to 202410_1_286_59 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:11.951648 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_286_59} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_282_58, 202410_286_286_0] -> 202410_1_286_59 2024.10.22 01:06:11.997666 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:06:12.073161 [ 1165 ] {} DatabaseCatalog: Execution took 428 ms. 2024.10.22 01:06:12.152838 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_283_81, 202410_284_284_0] 2024.10.22 01:06:12.159312 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_283_81, 202410_284_284_0] 2024.10.22 01:06:12.179564 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_283_81, 202410_284_284_0] 2024.10.22 01:06:12.248874 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6225355d5a218f8af2fb3d7e0eda6463) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 6225355d5a218f8af2fb3d7e0eda6463, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:12.251631 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6225355d5a218f8af2fb3d7e0eda6463): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:06:12.310358 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_595_218} MergeTask::MergeProjectionsStage: Merge sorted 35766 rows, containing 13 columns (13 merged, 0 gathered) in 2.736552013 sec., 13069.731483302161 rows/sec., 2.04 MiB/sec. 2024.10.22 01:06:12.467395 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_595_218} MergedBlockOutputStream: filled checksums 202410_1_595_218 (state Temporary) 2024.10.22 01:06:12.496556 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 14.20 MiB, will set to 1.67 GiB (RSS), difference: 18.67 MiB 2024.10.22 01:06:13.091357 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_595_218} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_595_218 to 202410_1_595_218 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:13.096024 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_595_218} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_594_217, 202410_595_595_0] -> 202410_1_595_218 2024.10.22 01:06:13.253006 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:06:13.430289 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c61bd23f4805b274bc02776e25e6affb) : [0, 490048]: Setting download as failed: File segment: [0, 490048], key: c61bd23f4805b274bc02776e25e6affb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:13.431640 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c61bd23f4805b274bc02776e25e6affb): No space left in cache to reserve 490049 bytes, will continue without cache download 2024.10.22 01:06:13.550753 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 35381 2024.10.22 01:06:13.666110 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c61bd23f4805b274bc02776e25e6affb, offset: 0, size: 0 2024.10.22 01:06:13.668337 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c61bd23f4805b274bc02776e25e6affb for removal 2024.10.22 01:06:13.675438 [ 1476 ] {} CacheMetadata: Key c61bd23f4805b274bc02776e25e6affb is removed from metadata 2024.10.22 01:06:13.682201 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6225355d5a218f8af2fb3d7e0eda6463, offset: 0, size: 0 2024.10.22 01:06:13.685146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6225355d5a218f8af2fb3d7e0eda6463 for removal 2024.10.22 01:06:13.688427 [ 1476 ] {} CacheMetadata: Key 6225355d5a218f8af2fb3d7e0eda6463 is removed from metadata 2024.10.22 01:06:13.827615 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:13.833568 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:14.000220 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_336_67/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:06:14.212410 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_587_587_0 (state Temporary) 2024.10.22 01:06:14.540481 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(dc823fa5b85dbd08021465f205224f3e) : [0, 155]: Setting download as failed: File segment: [0, 155], key: dc823fa5b85dbd08021465f205224f3e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:14.544539 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(dc823fa5b85dbd08021465f205224f3e): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:06:14.680671 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_587_587_0 to 202410_600_600_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:14.696086 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 185 entries to flush up to offset 27965 2024.10.22 01:06:14.719146 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_595_218 to 202410_596_596_0 2024.10.22 01:06:14.719425 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35381 2024.10.22 01:06:14.729059 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.26 MiB using storage policy from min volume index 0 2024.10.22 01:06:14.730920 [ 1244 ] {} DiskLocal: Reserved 1.26 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:14.757739 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_596_219} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_595_218 to 202410_596_596_0 into Compact with storage Full 2024.10.22 01:06:14.797604 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_596_219} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:14.803580 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_596_219} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_595_218, total 35766 rows starting from the beginning of the part 2024.10.22 01:06:14.830314 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_596_219} MergeTreeSequentialSource: Reading 2 marks from part 202410_596_596_0, total 97 rows starting from the beginning of the part 2024.10.22 01:06:14.711917 [ 1216 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:06:14.951294 [ 1216 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:06:14.966427 [ 1216 ] {} DNSCacheUpdater: Execution took 254 ms. 2024.10.22 01:06:15.025695 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:15.028031 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:15.094347 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_341_341_0 (state Temporary) 2024.10.22 01:06:15.343711 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_341_341_0 to 202410_347_347_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:15.384424 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 27965 2024.10.22 01:06:15.476591 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_341_119 to 202410_342_342_0 2024.10.22 01:06:15.483012 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.46 MiB using storage policy from min volume index 0 2024.10.22 01:06:15.486572 [ 1221 ] {} DiskLocal: Reserved 1.46 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:15.498625 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_342_120} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_341_119 to 202410_342_342_0 into Wide with storage Full 2024.10.22 01:06:15.505746 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b3cc8cbab12789f1b0efe9af2ccf4db8) : [0, 229787]: Setting download as failed: File segment: [0, 229787], key: b3cc8cbab12789f1b0efe9af2ccf4db8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:15.507294 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b3cc8cbab12789f1b0efe9af2ccf4db8): No space left in cache to reserve 229788 bytes, will continue without cache download 2024.10.22 01:06:15.540857 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_342_120} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:15.555345 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_342_120} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_341_119, total 26956 rows starting from the beginning of the part 2024.10.22 01:06:15.633290 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b3cc8cbab12789f1b0efe9af2ccf4db8, offset: 0, size: 0 2024.10.22 01:06:15.635524 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b3cc8cbab12789f1b0efe9af2ccf4db8 for removal 2024.10.22 01:06:15.637722 [ 1476 ] {} CacheMetadata: Key b3cc8cbab12789f1b0efe9af2ccf4db8 is removed from metadata 2024.10.22 01:06:15.639806 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dc823fa5b85dbd08021465f205224f3e, offset: 0, size: 0 2024.10.22 01:06:15.643624 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dc823fa5b85dbd08021465f205224f3e for removal 2024.10.22 01:06:15.648706 [ 1476 ] {} CacheMetadata: Key dc823fa5b85dbd08021465f205224f3e is removed from metadata 2024.10.22 01:06:15.756201 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 541616 2024.10.22 01:06:15.821512 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 5082 2024.10.22 01:06:15.844463 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 296 2024.10.22 01:06:15.896514 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.065265339 sec. 2024.10.22 01:06:15.914522 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_342_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_342_342_0, total 123 rows starting from the beginning of the part 2024.10.22 01:06:16.215309 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:16.221615 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:16.264211 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:16.268733 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:16.308289 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_345_345_0 (state Temporary) 2024.10.22 01:06:16.308786 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_278_278_0 (state Temporary) 2024.10.22 01:06:16.362470 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_345_345_0 to 202410_358_358_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:16.435085 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5082 2024.10.22 01:06:16.487578 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_278_278_0 to 202410_288_288_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:16.588435 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:16.590970 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:16.634309 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(355aa5088a3bd0aa223ea675f56660f5) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 355aa5088a3bd0aa223ea675f56660f5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:16.636694 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(355aa5088a3bd0aa223ea675f56660f5): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:06:16.639303 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 296 2024.10.22 01:06:16.786935 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_330_330_0 (state Temporary) 2024.10.22 01:06:16.883311 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 35 entries to flush up to offset 2795 2024.10.22 01:06:17.285395 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_330_330_0 to 202410_337_337_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:17.319601 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 541616 2024.10.22 01:06:17.377377 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.88 GiB, free memory in arenas 17.06 MiB, will set to 1.68 GiB (RSS), difference: -42.78 MiB 2024.10.22 01:06:17.539426 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f4eaf649ab24d0d64ab544c29a570e74) : [0, 228546]: Setting download as failed: File segment: [0, 228546], key: f4eaf649ab24d0d64ab544c29a570e74, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:17.541370 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f4eaf649ab24d0d64ab544c29a570e74): No space left in cache to reserve 228547 bytes, will continue without cache download 2024.10.22 01:06:17.579482 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f4eaf649ab24d0d64ab544c29a570e74, offset: 0, size: 0 2024.10.22 01:06:17.582198 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f4eaf649ab24d0d64ab544c29a570e74 for removal 2024.10.22 01:06:17.585540 [ 1476 ] {} CacheMetadata: Key f4eaf649ab24d0d64ab544c29a570e74 is removed from metadata 2024.10.22 01:06:17.587705 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 355aa5088a3bd0aa223ea675f56660f5, offset: 0, size: 0 2024.10.22 01:06:17.589017 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 355aa5088a3bd0aa223ea675f56660f5 for removal 2024.10.22 01:06:17.590888 [ 1476 ] {} CacheMetadata: Key 355aa5088a3bd0aa223ea675f56660f5 is removed from metadata 2024.10.22 01:06:17.635858 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_596_219} MergeTask::MergeProjectionsStage: Merge sorted 35863 rows, containing 13 columns (13 merged, 0 gathered) in 2.896866079 sec., 12379.930249443885 rows/sec., 1.93 MiB/sec. 2024.10.22 01:06:17.701288 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_596_219} MergedBlockOutputStream: filled checksums 202410_1_596_219 (state Temporary) 2024.10.22 01:06:17.928529 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:06:18.012898 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_342_120} MergeTask::MergeProjectionsStage: Merge sorted 27079 rows, containing 13 columns (13 merged, 0 gathered) in 2.519201719 sec., 10749.039981899125 rows/sec., 4.58 MiB/sec. 2024.10.22 01:06:18.063452 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_596_219} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_596_219 to 202410_1_596_219 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:18.068105 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_596_219} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_595_218, 202410_596_596_0] -> 202410_1_596_219 2024.10.22 01:06:18.097047 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:06:18.252279 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_284_82, 202410_285_285_0] 2024.10.22 01:06:18.254265 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_284_82, 202410_285_285_0] 2024.10.22 01:06:18.265658 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_284_82, 202410_285_285_0] 2024.10.22 01:06:18.275594 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_342_120} MergedBlockOutputStream: filled checksums 202410_1_342_120 (state Temporary) 2024.10.22 01:06:18.487834 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ec525b864b85def2b574f1f54c75c7b3) : [0, 154]: Setting download as failed: File segment: [0, 154], key: ec525b864b85def2b574f1f54c75c7b3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:18.490162 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ec525b864b85def2b574f1f54c75c7b3): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:06:18.748223 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 90 entries to flush up to offset 35471 2024.10.22 01:06:18.901455 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_342_120} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_342_120 to 202410_1_342_120 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:18.905207 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_342_120} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_341_119, 202410_342_342_0] -> 202410_1_342_120 2024.10.22 01:06:19.013237 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:19.022895 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:19.034641 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.88 MiB. 2024.10.22 01:06:19.209251 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_588_588_0 (state Temporary) 2024.10.22 01:06:19.260602 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_588_588_0 to 202410_601_601_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:19.287571 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_596_219 to 202410_597_597_0 2024.10.22 01:06:19.291802 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.27 MiB using storage policy from min volume index 0 2024.10.22 01:06:19.294628 [ 1268 ] {} DiskLocal: Reserved 1.27 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:19.299413 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35471 2024.10.22 01:06:19.307945 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_597_220} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_596_219 to 202410_597_597_0 into Compact with storage Full 2024.10.22 01:06:19.336398 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3669f793461dacb482decf60b593561e) : [0, 190451]: Setting download as failed: File segment: [0, 190451], key: 3669f793461dacb482decf60b593561e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:19.337502 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3669f793461dacb482decf60b593561e): No space left in cache to reserve 190452 bytes, will continue without cache download 2024.10.22 01:06:19.374053 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_597_220} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:19.384939 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_597_220} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_596_219, total 35863 rows starting from the beginning of the part 2024.10.22 01:06:19.412116 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_597_220} MergeTreeSequentialSource: Reading 2 marks from part 202410_597_597_0, total 69 rows starting from the beginning of the part 2024.10.22 01:06:19.486262 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3669f793461dacb482decf60b593561e, offset: 0, size: 0 2024.10.22 01:06:19.493627 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3669f793461dacb482decf60b593561e for removal 2024.10.22 01:06:19.498557 [ 1476 ] {} CacheMetadata: Key 3669f793461dacb482decf60b593561e is removed from metadata 2024.10.22 01:06:19.501514 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ec525b864b85def2b574f1f54c75c7b3, offset: 0, size: 0 2024.10.22 01:06:19.505283 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ec525b864b85def2b574f1f54c75c7b3 for removal 2024.10.22 01:06:19.511484 [ 1476 ] {} CacheMetadata: Key ec525b864b85def2b574f1f54c75c7b3 is removed from metadata 2024.10.22 01:06:19.595189 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.88 GiB, free memory in arenas 18.37 MiB, will set to 1.67 GiB (RSS), difference: 50.54 MiB 2024.10.22 01:06:19.880185 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:06:19.884586 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:06:20.377667 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_597_220/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:06:20.428199 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(07ac305cde09a1603ecf558db3162c98) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 07ac305cde09a1603ecf558db3162c98, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:20.430931 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(07ac305cde09a1603ecf558db3162c98): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:06:21.585397 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0f089a1ebfa6e95b1ae274099b43776d) : [0, 518133]: Setting download as failed: File segment: [0, 518133], key: 0f089a1ebfa6e95b1ae274099b43776d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:21.588200 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0f089a1ebfa6e95b1ae274099b43776d): No space left in cache to reserve 518134 bytes, will continue without cache download 2024.10.22 01:06:21.608514 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0f089a1ebfa6e95b1ae274099b43776d, offset: 0, size: 0 2024.10.22 01:06:21.711185 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0f089a1ebfa6e95b1ae274099b43776d for removal 2024.10.22 01:06:21.815174 [ 1476 ] {} CacheMetadata: Key 0f089a1ebfa6e95b1ae274099b43776d is removed from metadata 2024.10.22 01:06:21.819149 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 07ac305cde09a1603ecf558db3162c98, offset: 0, size: 0 2024.10.22 01:06:21.824146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 07ac305cde09a1603ecf558db3162c98 for removal 2024.10.22 01:06:21.829604 [ 1476 ] {} CacheMetadata: Key 07ac305cde09a1603ecf558db3162c98 is removed from metadata 2024.10.22 01:06:22.008039 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_597_220} MergeTask::MergeProjectionsStage: Merge sorted 35932 rows, containing 13 columns (13 merged, 0 gathered) in 2.707562115 sec., 13270.979011316238 rows/sec., 2.07 MiB/sec. 2024.10.22 01:06:22.184361 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_597_220} MergedBlockOutputStream: filled checksums 202410_1_597_220 (state Temporary) 2024.10.22 01:06:22.392226 [ 1179 ] {} DatabaseCatalog: Execution took 318 ms. 2024.10.22 01:06:22.492023 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.88 GiB, free memory in arenas 14.91 MiB, will set to 1.67 GiB (RSS), difference: -1.59 MiB 2024.10.22 01:06:22.547848 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_597_220} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_597_220 to 202410_1_597_220 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:22.553760 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_597_220} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_596_219, 202410_597_597_0] -> 202410_1_597_220 2024.10.22 01:06:22.580608 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:06:22.757696 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(1d8ad350f378f57809f83834c19072c0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1d8ad350f378f57809f83834c19072c0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:22.760745 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 12752, file segment info: File segment: [0, 33554431], key: 1d8ad350f378f57809f83834c19072c0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:22.764933 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:22.916044 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 200 entries to flush up to offset 28165 2024.10.22 01:06:23.105306 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:23.107293 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:23.308340 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 35522 2024.10.22 01:06:23.330105 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9f6932f6dd142b29edd9d0aa3c09ccb8) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 9f6932f6dd142b29edd9d0aa3c09ccb8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:23.332946 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9f6932f6dd142b29edd9d0aa3c09ccb8): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:06:23.432785 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:23.435923 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:23.476346 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_342_342_0 (state Temporary) 2024.10.22 01:06:23.492058 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_589_589_0 (state Temporary) 2024.10.22 01:06:23.543237 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_589_589_0 to 202410_602_602_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:23.633016 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_342_342_0 to 202410_348_348_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:23.767125 [ 1169 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_342_120 to 202410_343_343_0 2024.10.22 01:06:23.775921 [ 1169 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.47 MiB using storage policy from min volume index 0 2024.10.22 01:06:23.777768 [ 1169 ] {} DiskLocal: Reserved 1.47 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:23.802483 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35522 2024.10.22 01:06:23.811721 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_343_121} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_342_120 to 202410_343_343_0 into Wide with storage Full 2024.10.22 01:06:23.842275 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_343_121} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:23.851825 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_343_121} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_342_120, total 27079 rows starting from the beginning of the part 2024.10.22 01:06:23.851301 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_597_220 to 202410_598_598_0 2024.10.22 01:06:23.865052 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.27 MiB using storage policy from min volume index 0 2024.10.22 01:06:23.872835 [ 1186 ] {} DiskLocal: Reserved 1.27 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:23.880476 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 28165 2024.10.22 01:06:23.880773 [ 1186 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 213 ms. 2024.10.22 01:06:23.895966 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_598_221} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_597_220 to 202410_598_598_0 into Compact with storage Full 2024.10.22 01:06:23.928011 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_598_221} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:23.936856 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 5097 2024.10.22 01:06:23.937787 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_598_221} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_597_220, total 35932 rows starting from the beginning of the part 2024.10.22 01:06:23.973690 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_598_221} MergeTreeSequentialSource: Reading 2 marks from part 202410_598_598_0, total 88 rows starting from the beginning of the part 2024.10.22 01:06:24.069312 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(accf6466d755a33413a5344a0147b118) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: accf6466d755a33413a5344a0147b118, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:24.073087 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 504, file segment info: File segment: [0, 33554431], key: accf6466d755a33413a5344a0147b118, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:24.076878 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:24.106614 [ 1712 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kth/qjfooosmbkbrnvplepsdhajcdtbga, size 12752 2024.10.22 01:06:24.141058 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 297 2024.10.22 01:06:24.153723 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:24.157592 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:24.166413 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_343_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_343_343_0, total 202 rows starting from the beginning of the part 2024.10.22 01:06:24.325718 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 542768 2024.10.22 01:06:24.389726 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:24.394915 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:24.402860 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_346_346_0 (state Temporary) 2024.10.22 01:06:24.419212 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:06:24.452580 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:06:24.473125 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:06:24.597704 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:06:24.612037 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_346_346_0 to 202410_359_359_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:24.614278 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:06:24.652281 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_279_279_0 (state Temporary) 2024.10.22 01:06:24.659667 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:06:24.694508 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5097 2024.10.22 01:06:24.705262 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:06:24.721568 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_279_279_0 to 202410_289_289_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:24.788218 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pfx/unmtfchujitfpuwncbaalfbgtqzwv (LogSeriesLimiter: on interval from 2024-10-22 01:06:08 to 2024-10-22 01:06:24 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:06:24.912410 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 297 2024.10.22 01:06:24.920690 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_496_131, 202410_497_497_0] 2024.10.22 01:06:24.924575 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_496_131, 202410_497_497_0] 2024.10.22 01:06:24.938451 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_496_131, 202410_497_497_0] 2024.10.22 01:06:24.988807 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(63176e9bcb40909575b0f70e468936ba) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 63176e9bcb40909575b0f70e468936ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:06:24.990876 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 63176e9bcb40909575b0f70e468936ba, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:06:24.992247 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:25.200713 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:25.205173 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:25.263879 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_336_67/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:06:25.532467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(aeaf53c3ced19314ef8fa7348d1c165f) : [0, 284373]: Setting download as failed: File segment: [0, 284373], key: aeaf53c3ced19314ef8fa7348d1c165f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:25.535230 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(aeaf53c3ced19314ef8fa7348d1c165f): No space left in cache to reserve 284374 bytes, will continue without cache download 2024.10.22 01:06:25.558899 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_331_331_0 (state Temporary) 2024.10.22 01:06:25.582656 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_598_221} MergeTask::MergeProjectionsStage: Merge sorted 36020 rows, containing 13 columns (13 merged, 0 gathered) in 1.690837924 sec., 21303.04713936615 rows/sec., 3.32 MiB/sec. 2024.10.22 01:06:25.611127 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_598_221} MergedBlockOutputStream: filled checksums 202410_1_598_221 (state Temporary) 2024.10.22 01:06:25.637420 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_331_331_0 to 202410_338_338_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:25.746270 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_598_221} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_598_221 to 202410_1_598_221 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:25.752049 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_598_221} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_597_220, 202410_598_598_0] -> 202410_1_598_221 2024.10.22 01:06:25.788327 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 542768 2024.10.22 01:06:25.806418 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:06:25.879440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aeaf53c3ced19314ef8fa7348d1c165f, offset: 0, size: 0 2024.10.22 01:06:25.983306 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aeaf53c3ced19314ef8fa7348d1c165f for removal 2024.10.22 01:06:25.986275 [ 1476 ] {} CacheMetadata: Key aeaf53c3ced19314ef8fa7348d1c165f is removed from metadata 2024.10.22 01:06:25.987440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9f6932f6dd142b29edd9d0aa3c09ccb8, offset: 0, size: 0 2024.10.22 01:06:25.990103 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9f6932f6dd142b29edd9d0aa3c09ccb8 for removal 2024.10.22 01:06:25.992005 [ 1476 ] {} CacheMetadata: Key 9f6932f6dd142b29edd9d0aa3c09ccb8 is removed from metadata 2024.10.22 01:06:26.065015 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_343_121/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:06:26.657130 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.88 GiB, free memory in arenas 8.49 MiB, will set to 1.68 GiB (RSS), difference: -37.62 MiB 2024.10.22 01:06:26.841719 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_343_121} MergeTask::MergeProjectionsStage: Merge sorted 27281 rows, containing 13 columns (13 merged, 0 gathered) in 3.035007979 sec., 8988.77373264395 rows/sec., 3.83 MiB/sec. 2024.10.22 01:06:26.902329 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(35c2c0a8aa81a7806f04aba70a95b53d) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 35c2c0a8aa81a7806f04aba70a95b53d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:26.903343 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(35c2c0a8aa81a7806f04aba70a95b53d): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:06:26.907830 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_343_121} MergedBlockOutputStream: filled checksums 202410_1_343_121 (state Temporary) 2024.10.22 01:06:26.958099 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_285_83, 202410_286_286_0] 2024.10.22 01:06:26.961596 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_285_83, 202410_286_286_0] 2024.10.22 01:06:26.979201 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_285_83, 202410_286_286_0] 2024.10.22 01:06:27.610246 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_343_121} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_343_121 to 202410_1_343_121 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:27.727580 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_343_121} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_342_120, 202410_343_343_0] -> 202410_1_343_121 2024.10.22 01:06:27.813186 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 35609 2024.10.22 01:06:27.834329 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7ec246f0b8cb37520af207fb76fc0659) : [0, 1299]: Setting download as failed: File segment: [0, 1299], key: 7ec246f0b8cb37520af207fb76fc0659, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:27.836142 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7ec246f0b8cb37520af207fb76fc0659): No space left in cache to reserve 1300 bytes, will continue without cache download 2024.10.22 01:06:27.857010 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ec246f0b8cb37520af207fb76fc0659, offset: 0, size: 0 2024.10.22 01:06:27.862472 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ec246f0b8cb37520af207fb76fc0659 for removal 2024.10.22 01:06:27.865900 [ 1476 ] {} CacheMetadata: Key 7ec246f0b8cb37520af207fb76fc0659 is removed from metadata 2024.10.22 01:06:27.867951 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 35c2c0a8aa81a7806f04aba70a95b53d, offset: 0, size: 0 2024.10.22 01:06:27.871840 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 35c2c0a8aa81a7806f04aba70a95b53d for removal 2024.10.22 01:06:27.880430 [ 1476 ] {} CacheMetadata: Key 35c2c0a8aa81a7806f04aba70a95b53d is removed from metadata 2024.10.22 01:06:27.883952 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.00 MiB. 2024.10.22 01:06:28.046157 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:28.049245 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:28.304291 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_590_590_0 (state Temporary) 2024.10.22 01:06:28.485214 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_590_590_0 to 202410_603_603_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:28.513374 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_598_221 to 202410_599_599_0 2024.10.22 01:06:28.521701 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.27 MiB using storage policy from min volume index 0 2024.10.22 01:06:28.523706 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(9fa83dbc4668d5b7e8e1ed099cafde02) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9fa83dbc4668d5b7e8e1ed099cafde02, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:06:28.527971 [ 1248 ] {} DiskLocal: Reserved 1.27 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:28.528771 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 767, file segment info: File segment: [0, 33554431], key: 9fa83dbc4668d5b7e8e1ed099cafde02, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:06:28.535668 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:28.542030 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 12.03 MiB, will set to 1.67 GiB (RSS), difference: 31.19 MiB 2024.10.22 01:06:28.546958 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35609 2024.10.22 01:06:28.547186 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_599_222} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_598_221 to 202410_599_599_0 into Compact with storage Full 2024.10.22 01:06:28.600146 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_599_222} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:28.606815 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_599_222} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_598_221, total 36020 rows starting from the beginning of the part 2024.10.22 01:06:28.622962 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} MergeTask::MergeProjectionsStage: Merge sorted 554054 rows, containing 4 columns (4 merged, 0 gathered) in 19.830672399 sec., 27939.244260216776 rows/sec., 409.61 KiB/sec. 2024.10.22 01:06:28.668768 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_599_222} MergeTreeSequentialSource: Reading 2 marks from part 202410_599_599_0, total 105 rows starting from the beginning of the part 2024.10.22 01:06:28.678057 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} MergedBlockOutputStream: filled checksums 202410_1_336_67 (state Temporary) 2024.10.22 01:06:28.707691 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_336_67 to 202410_1_336_67 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:28.711113 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_336_67} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_331_66, 202410_336_336_0] -> 202410_1_336_67 2024.10.22 01:06:28.843891 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:06:29.098400 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:06:29.103175 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d38851168e106d0077b68d45d17ac889) : [0, 191]: Setting download as failed: File segment: [0, 191], key: d38851168e106d0077b68d45d17ac889, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:29.103821 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:06:29.106177 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d38851168e106d0077b68d45d17ac889): No space left in cache to reserve 192 bytes, will continue without cache download 2024.10.22 01:06:29.118631 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:06:29.233743 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:06:29.265620 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:06:29.281609 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:06:29.294162 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:06:29.423371 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:06:29.464154 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:06:29.536809 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kqy/uuqadegslbwgkjkxxswtczyehjilh (LogSeriesLimiter: on interval from 2024-10-22 01:06:24 to 2024-10-22 01:06:29 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:06:29.880856 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:06:29.883122 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:06:29.922713 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(41238aaf8d931a87b99d83f790313a91) : [0, 1041708]: Setting download as failed: File segment: [0, 1041708], key: 41238aaf8d931a87b99d83f790313a91, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:29.928010 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(41238aaf8d931a87b99d83f790313a91): No space left in cache to reserve 1041709 bytes, will continue without cache download 2024.10.22 01:06:29.964471 [ 1266 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:06:29.980007 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 41238aaf8d931a87b99d83f790313a91, offset: 0, size: 0 2024.10.22 01:06:29.983870 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 41238aaf8d931a87b99d83f790313a91 for removal 2024.10.22 01:06:29.987039 [ 1476 ] {} CacheMetadata: Key 41238aaf8d931a87b99d83f790313a91 is removed from metadata 2024.10.22 01:06:29.988709 [ 1266 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:06:30.093297 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d38851168e106d0077b68d45d17ac889, offset: 0, size: 0 2024.10.22 01:06:30.095643 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d38851168e106d0077b68d45d17ac889 for removal 2024.10.22 01:06:30.098383 [ 1476 ] {} CacheMetadata: Key d38851168e106d0077b68d45d17ac889 is removed from metadata 2024.10.22 01:06:30.440277 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.88 GiB, free memory in arenas 11.61 MiB, will set to 1.67 GiB (RSS), difference: 1.79 MiB 2024.10.22 01:06:30.512273 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_497_132, 202410_498_498_0] 2024.10.22 01:06:30.516769 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_497_132, 202410_498_498_0] 2024.10.22 01:06:30.530759 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_497_132, 202410_498_498_0] 2024.10.22 01:06:30.894960 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(86c641a5c4a30d47183fe2f862d42019) : [0, 196]: Setting download as failed: File segment: [0, 196], key: 86c641a5c4a30d47183fe2f862d42019, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:30.897011 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(86c641a5c4a30d47183fe2f862d42019): No space left in cache to reserve 197 bytes, will continue without cache download 2024.10.22 01:06:30.992580 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_599_222} MergeTask::MergeProjectionsStage: Merge sorted 36125 rows, containing 13 columns (13 merged, 0 gathered) in 2.453008631 sec., 14726.813246178097 rows/sec., 2.30 MiB/sec. 2024.10.22 01:06:31.183843 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_599_222} MergedBlockOutputStream: filled checksums 202410_1_599_222 (state Temporary) 2024.10.22 01:06:31.404357 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 197 entries to flush up to offset 28362 2024.10.22 01:06:31.534979 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_599_222} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_599_222 to 202410_1_599_222 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:31.539841 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_599_222} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_598_221, 202410_599_599_0] -> 202410_1_599_222 2024.10.22 01:06:31.566981 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:06:31.582437 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(475445d1acaf63e40d6704f740c8789f) : [0, 1354771]: Setting download as failed: File segment: [0, 1354771], key: 475445d1acaf63e40d6704f740c8789f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:31.584593 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(475445d1acaf63e40d6704f740c8789f): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:06:31.839595 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 475445d1acaf63e40d6704f740c8789f, offset: 0, size: 0 2024.10.22 01:06:31.842366 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 475445d1acaf63e40d6704f740c8789f for removal 2024.10.22 01:06:31.845129 [ 1476 ] {} CacheMetadata: Key 475445d1acaf63e40d6704f740c8789f is removed from metadata 2024.10.22 01:06:31.852687 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 86c641a5c4a30d47183fe2f862d42019, offset: 0, size: 0 2024.10.22 01:06:31.855819 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 86c641a5c4a30d47183fe2f862d42019 for removal 2024.10.22 01:06:31.864037 [ 1476 ] {} CacheMetadata: Key 86c641a5c4a30d47183fe2f862d42019 is removed from metadata 2024.10.22 01:06:32.010982 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:32.013276 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:32.180230 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:32.185561 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:32.190109 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_343_343_0 (state Temporary) 2024.10.22 01:06:32.198276 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 5117 2024.10.22 01:06:32.351110 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_343_343_0 to 202410_349_349_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:32.394373 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 28362 2024.10.22 01:06:32.416259 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 298 2024.10.22 01:06:32.482265 [ 1208 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_343_121 to 202410_344_344_0 2024.10.22 01:06:32.485375 [ 1208 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.48 MiB using storage policy from min volume index 0 2024.10.22 01:06:32.487292 [ 1208 ] {} DiskLocal: Reserved 1.48 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:32.500218 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_344_122} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_343_121 to 202410_344_344_0 into Wide with storage Full 2024.10.22 01:06:32.508653 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8201f3c8bee1600f1cd2eca8d752b5bd) : [0, 159]: Setting download as failed: File segment: [0, 159], key: 8201f3c8bee1600f1cd2eca8d752b5bd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:32.510193 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8201f3c8bee1600f1cd2eca8d752b5bd): No space left in cache to reserve 160 bytes, will continue without cache download 2024.10.22 01:06:32.520223 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.88 GiB, free memory in arenas 12.45 MiB, will set to 1.67 GiB (RSS), difference: 3.09 MiB 2024.10.22 01:06:32.564699 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 91 entries to flush up to offset 35700 2024.10.22 01:06:32.597353 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:32.600279 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:32.602050 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:32.603314 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:32.638117 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_347_347_0 (state Temporary) 2024.10.22 01:06:32.651294 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_344_122} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:32.663187 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_344_122} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_343_121, total 27281 rows starting from the beginning of the part 2024.10.22 01:06:32.679180 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_347_347_0 to 202410_360_360_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:32.816319 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_344_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_344_344_0, total 187 rows starting from the beginning of the part 2024.10.22 01:06:32.823767 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5117 2024.10.22 01:06:32.846447 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1545 entries to flush up to offset 544313 2024.10.22 01:06:32.862692 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_280_280_0 (state Temporary) 2024.10.22 01:06:32.879627 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:32.884882 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:32.944662 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_591_591_0 (state Temporary) 2024.10.22 01:06:32.988456 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_591_591_0 to 202410_604_604_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:33.029694 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_599_222 to 202410_600_600_0 2024.10.22 01:06:33.038558 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.28 MiB using storage policy from min volume index 0 2024.10.22 01:06:33.045781 [ 1233 ] {} DiskLocal: Reserved 1.28 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:33.088876 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_600_223} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_599_222 to 202410_600_600_0 into Compact with storage Full 2024.10.22 01:06:33.092133 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35700 2024.10.22 01:06:33.238414 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(42102dc1da74142816890077c517909d) : [0, 428160]: Setting download as failed: File segment: [0, 428160], key: 42102dc1da74142816890077c517909d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:33.239897 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(42102dc1da74142816890077c517909d): No space left in cache to reserve 428161 bytes, will continue without cache download 2024.10.22 01:06:33.248066 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_280_280_0 to 202410_290_290_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:33.313829 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_600_223} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:33.327162 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_600_223} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_599_222, total 36125 rows starting from the beginning of the part 2024.10.22 01:06:33.333687 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 298 2024.10.22 01:06:33.374894 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_600_223} MergeTreeSequentialSource: Reading 2 marks from part 202410_600_600_0, total 87 rows starting from the beginning of the part 2024.10.22 01:06:33.391408 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 42102dc1da74142816890077c517909d, offset: 0, size: 0 2024.10.22 01:06:33.394382 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 42102dc1da74142816890077c517909d for removal 2024.10.22 01:06:33.397050 [ 1476 ] {} CacheMetadata: Key 42102dc1da74142816890077c517909d is removed from metadata 2024.10.22 01:06:33.401412 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8201f3c8bee1600f1cd2eca8d752b5bd, offset: 0, size: 0 2024.10.22 01:06:33.403939 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8201f3c8bee1600f1cd2eca8d752b5bd for removal 2024.10.22 01:06:33.406822 [ 1476 ] {} CacheMetadata: Key 8201f3c8bee1600f1cd2eca8d752b5bd is removed from metadata 2024.10.22 01:06:33.598120 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_498_133, 202410_499_499_0] 2024.10.22 01:06:33.601023 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_498_133, 202410_499_499_0] 2024.10.22 01:06:33.636568 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:33.639332 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:33.717614 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_498_133, 202410_499_499_0] 2024.10.22 01:06:33.754609 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_332_332_0 (state Temporary) 2024.10.22 01:06:33.847789 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_332_332_0 to 202410_339_339_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:33.893588 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 544313 2024.10.22 01:06:34.453219 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(139360529b58f86128bcfdb50ccca40b) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 139360529b58f86128bcfdb50ccca40b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:34.455216 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(139360529b58f86128bcfdb50ccca40b): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:06:34.546135 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_70_70_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:06:34.624143 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_70_70_0 (state Temporary) 2024.10.22 01:06:34.712245 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.88 GiB, free memory in arenas 5.10 MiB, will set to 1.67 GiB (RSS), difference: -64.97 MiB 2024.10.22 01:06:35.372189 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(11dc18d955e29ac51864c3ef94d40427) : [0, 116062]: Setting download as failed: File segment: [0, 116062], key: 11dc18d955e29ac51864c3ef94d40427, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:06:35.380058 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(11dc18d955e29ac51864c3ef94d40427): No space left in cache to reserve 116063 bytes, will continue without cache download 2024.10.22 01:06:35.505466 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 11dc18d955e29ac51864c3ef94d40427, offset: 0, size: 0 2024.10.22 01:06:35.507923 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 11dc18d955e29ac51864c3ef94d40427 for removal 2024.10.22 01:06:35.511048 [ 1476 ] {} CacheMetadata: Key 11dc18d955e29ac51864c3ef94d40427 is removed from metadata 2024.10.22 01:06:35.512050 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 139360529b58f86128bcfdb50ccca40b, offset: 0, size: 0 2024.10.22 01:06:35.515609 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 139360529b58f86128bcfdb50ccca40b for removal 2024.10.22 01:06:35.527319 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yss/kdtvhnottettweeojkbzzirdfltgy. 2024.10.22 01:06:35.529230 [ 1476 ] {} CacheMetadata: Key 139360529b58f86128bcfdb50ccca40b is removed from metadata 2024.10.22 01:06:35.575522 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_344_122} MergeTask::MergeProjectionsStage: Merge sorted 27468 rows, containing 13 columns (13 merged, 0 gathered) in 3.079398656 sec., 8919.923357919397 rows/sec., 3.81 MiB/sec. 2024.10.22 01:06:35.680914 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_344_122} MergedBlockOutputStream: filled checksums 202410_1_344_122 (state Temporary) 2024.10.22 01:06:35.686940 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/luh/goczjcwcykrslndmerqppyqogybmu. 2024.10.22 01:06:36.272228 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_600_223} MergeTask::MergeProjectionsStage: Merge sorted 36212 rows, containing 13 columns (13 merged, 0 gathered) in 3.201230673 sec., 11311.899609553691 rows/sec., 1.77 MiB/sec. 2024.10.22 01:06:36.547339 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_600_223} MergedBlockOutputStream: filled checksums 202410_1_600_223 (state Temporary) 2024.10.22 01:06:36.596411 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_344_122/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:06:36.656685 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/luh/goczjcwcykrslndmerqppyqogybmu. 2024.10.22 01:06:36.662530 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_19_19_0 to 201403_19_19_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:36.685172 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yss/kdtvhnottettweeojkbzzirdfltgy. 2024.10.22 01:06:36.695118 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_600_223} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_600_223 to 202410_1_600_223 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:36.698832 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_600_223} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_599_222, 202410_600_600_0] -> 202410_1_600_223 2024.10.22 01:06:36.703851 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:06:36.762873 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_344_122} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_344_122 to 202410_1_344_122 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:36.771087 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_344_122} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_343_121, 202410_344_344_0] -> 202410_1_344_122 2024.10.22 01:06:36.794371 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.01 MiB. 2024.10.22 01:06:36.840093 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:06:37.107508 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 35773 2024.10.22 01:06:37.450846 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:37.453156 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:37.620103 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_592_592_0 (state Temporary) 2024.10.22 01:06:37.722031 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.11 GiB, peak 1.88 GiB, free memory in arenas 5.90 MiB, will set to 1.53 GiB (RSS), difference: 428.87 MiB 2024.10.22 01:06:37.882353 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_592_592_0 to 202410_605_605_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:37.926500 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35773 2024.10.22 01:06:37.935105 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_600_223 to 202410_601_601_0 2024.10.22 01:06:37.946584 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.28 MiB using storage policy from min volume index 0 2024.10.22 01:06:37.950424 [ 1249 ] {} DiskLocal: Reserved 1.28 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:37.953489 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_286_84, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:06:37.957053 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_286_84, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:06:37.967051 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_601_224} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_600_223 to 202410_601_601_0 into Compact with storage Full 2024.10.22 01:06:38.071160 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_601_224} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:38.087412 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_601_224} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_600_223, total 36212 rows starting from the beginning of the part 2024.10.22 01:06:38.106524 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_286_84, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:06:38.110464 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_601_224} MergeTreeSequentialSource: Reading 2 marks from part 202410_601_601_0, total 90 rows starting from the beginning of the part 2024.10.22 01:06:39.308753 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.88 GiB, free memory in arenas 5.23 MiB, will set to 1.47 GiB (RSS), difference: -164.95 MiB 2024.10.22 01:06:39.880589 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:06:39.882661 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:06:39.940806 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 189 entries to flush up to offset 28551 2024.10.22 01:06:40.065090 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_499_134, 202410_500_500_0, 202410_501_501_0, 202410_502_502_0, 202410_503_503_0, 202410_504_504_0] 2024.10.22 01:06:40.069491 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_499_134, 202410_500_500_0, 202410_501_501_0, 202410_502_502_0, 202410_503_503_0, 202410_504_504_0] 2024.10.22 01:06:40.091335 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_499_134, 202410_500_500_0, 202410_501_501_0, 202410_502_502_0, 202410_503_503_0, 202410_504_504_0] 2024.10.22 01:06:40.161091 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_601_224} MergeTask::MergeProjectionsStage: Merge sorted 36302 rows, containing 13 columns (13 merged, 0 gathered) in 2.202352605 sec., 16483.28243060788 rows/sec., 2.57 MiB/sec. 2024.10.22 01:06:40.205610 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:40.208582 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:40.212177 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_601_224} MergedBlockOutputStream: filled checksums 202410_1_601_224 (state Temporary) 2024.10.22 01:06:40.259255 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_344_344_0 (state Temporary) 2024.10.22 01:06:40.318769 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 19 2024.10.22 01:06:40.323773 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_344_344_0 to 202410_350_350_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:40.330385 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 5130 2024.10.22 01:06:40.375591 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 28551 2024.10.22 01:06:40.445629 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:40.449275 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:40.474258 [ 1242 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_344_122 to 202410_345_345_0 2024.10.22 01:06:40.480128 [ 1242 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.49 MiB using storage policy from min volume index 0 2024.10.22 01:06:40.484701 [ 1242 ] {} DiskLocal: Reserved 1.49 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:40.485318 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_348_348_0 (state Temporary) 2024.10.22 01:06:40.500741 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.88 GiB, free memory in arenas 5.90 MiB, will set to 1.48 GiB (RSS), difference: -44.35 MiB 2024.10.22 01:06:40.510600 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_345_123} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_344_122 to 202410_345_345_0 into Wide with storage Full 2024.10.22 01:06:40.527638 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_601_224} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_601_224 to 202410_1_601_224 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:40.531757 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_601_224} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_600_223, 202410_601_601_0] -> 202410_1_601_224 2024.10.22 01:06:40.545238 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_345_123} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:40.554637 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_345_123} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_344_122, total 27468 rows starting from the beginning of the part 2024.10.22 01:06:40.584191 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:40.586927 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:40.637766 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_19_19_0 (state Temporary) 2024.10.22 01:06:40.649180 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_345_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_345_345_0, total 202 rows starting from the beginning of the part 2024.10.22 01:06:40.657059 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_348_348_0 to 202410_361_361_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:40.676939 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:06:40.701203 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_356_71 to 202410_361_361_0 2024.10.22 01:06:40.709322 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:40.709705 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5130 2024.10.22 01:06:40.716885 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:40.808228 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(49e4bfbd6e452e7f07949579ee65a41a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 49e4bfbd6e452e7f07949579ee65a41a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:40.811508 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 6726, file segment info: File segment: [0, 33554431], key: 49e4bfbd6e452e7f07949579ee65a41a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:40.815683 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:40.899921 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1527 entries to flush up to offset 545840 2024.10.22 01:06:40.899633 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_356_71 to 202410_361_361_0 into Compact with storage Full 2024.10.22 01:06:40.910512 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_19_19_0 to 202410_19_19_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:40.934620 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:40.837831 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 299 2024.10.22 01:06:40.946044 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_356_71, total 5141 rows starting from the beginning of the part 2024.10.22 01:06:40.983451 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_357_357_0, total 29 rows starting from the beginning of the part 2024.10.22 01:06:41.005818 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_358_358_0, total 18 rows starting from the beginning of the part 2024.10.22 01:06:41.006521 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_296_59, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:06:41.008418 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_296_59, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:06:41.033375 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_296_59, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:06:41.038885 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_359_359_0, total 15 rows starting from the beginning of the part 2024.10.22 01:06:41.092250 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 19 2024.10.22 01:06:41.195012 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_360_360_0, total 20 rows starting from the beginning of the part 2024.10.22 01:06:41.323470 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:41.331276 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_361_361_0, total 13 rows starting from the beginning of the part 2024.10.22 01:06:41.331389 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:41.391915 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_281_281_0 (state Temporary) 2024.10.22 01:06:41.550784 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_281_281_0 to 202410_291_291_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:41.557770 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(acc197455bae921bdcc7e688ac2f6b23) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: acc197455bae921bdcc7e688ac2f6b23, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:41.593119 [ 1160 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_286_59 to 202410_291_291_0 2024.10.22 01:06:41.589053 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 462, file segment info: File segment: [0, 33554431], key: acc197455bae921bdcc7e688ac2f6b23, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:41.596008 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:41.601128 [ 1160 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:41.603902 [ 1160 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:41.626784 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_286_59 to 202410_291_291_0 into Compact with storage Full 2024.10.22 01:06:41.746287 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 299 2024.10.22 01:06:41.829708 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:41.870057 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_286_59, total 330 rows starting from the beginning of the part 2024.10.22 01:06:41.948477 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 35859 2024.10.22 01:06:42.010784 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:42.013012 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:42.093773 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_287_287_0, total 1 rows starting from the beginning of the part 2024.10.22 01:06:42.105680 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:42.111155 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:42.128444 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_70_70_0 to 202410_73_73_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:42.136106 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} MergeTask::MergeProjectionsStage: Merge sorted 5236 rows, containing 12 columns (12 merged, 0 gathered) in 1.25877908 sec., 4159.586128488884 rows/sec., 655.47 KiB/sec. 2024.10.22 01:06:42.302264 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_333_333_0 (state Temporary) 2024.10.22 01:06:42.330412 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_288_288_0, total 1 rows starting from the beginning of the part 2024.10.22 01:06:42.397432 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} MergedBlockOutputStream: filled checksums 202410_1_361_72 (state Temporary) 2024.10.22 01:06:42.399486 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_593_593_0 (state Temporary) 2024.10.22 01:06:42.401063 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_333_333_0 to 202410_340_340_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:42.419974 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(8f077afe327ae7d10875454e7d164b68) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8f077afe327ae7d10875454e7d164b68, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:42.461161 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 545840 2024.10.22 01:06:42.496594 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.88 GiB, free memory in arenas 5.81 MiB, will set to 1.52 GiB (RSS), difference: -77.39 MiB 2024.10.22 01:06:42.425746 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 19554, file segment info: File segment: [0, 33554431], key: 8f077afe327ae7d10875454e7d164b68, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:42.529138 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:42.559648 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_289_289_0, total 1 rows starting from the beginning of the part 2024.10.22 01:06:42.586574 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_593_593_0 to 202410_606_606_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:42.603547 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_601_224 to 202410_602_602_0 2024.10.22 01:06:42.605659 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.28 MiB using storage policy from min volume index 0 2024.10.22 01:06:42.609042 [ 1168 ] {} DiskLocal: Reserved 1.28 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:42.657164 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_361_72 to 202410_1_361_72 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:42.663676 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_361_72} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_356_71, 202410_361_361_0] -> 202410_1_361_72 2024.10.22 01:06:42.665087 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_602_225} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_601_224 to 202410_602_602_0 into Compact with storage Full 2024.10.22 01:06:42.701279 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_290_290_0, total 1 rows starting from the beginning of the part 2024.10.22 01:06:42.742801 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35859 2024.10.22 01:06:42.744292 [ 1173 ] {} DatabaseCatalog: Execution took 238 ms. 2024.10.22 01:06:42.760075 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.41 MiB. 2024.10.22 01:06:42.901754 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_291_291_0, total 1 rows starting from the beginning of the part 2024.10.22 01:06:42.963153 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_602_225} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:42.973993 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_602_225} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_601_224, total 36302 rows starting from the beginning of the part 2024.10.22 01:06:43.016518 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_602_225} MergeTreeSequentialSource: Reading 2 marks from part 202410_602_602_0, total 51 rows starting from the beginning of the part 2024.10.22 01:06:43.496225 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_345_123} MergeTask::MergeProjectionsStage: Merge sorted 27670 rows, containing 13 columns (13 merged, 0 gathered) in 2.999649628 sec., 9224.410658403736 rows/sec., 3.94 MiB/sec. 2024.10.22 01:06:43.563783 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(bebe322d11535e9bc37bbca35ffe3157) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: bebe322d11535e9bc37bbca35ffe3157, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:43.567695 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 559, file segment info: File segment: [0, 33554431], key: bebe322d11535e9bc37bbca35ffe3157, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:06:43.573007 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:43.662934 [ 1741 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pfx/unmtfchujitfpuwncbaalfbgtqzwv, size 19554 2024.10.22 01:06:43.860222 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_345_123} MergedBlockOutputStream: filled checksums 202410_1_345_123 (state Temporary) 2024.10.22 01:06:43.919540 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:06:43.936562 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:06:43.959093 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:06:44.002270 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:06:44.021668 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:06:44.050889 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:06:44.086209 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:06:44.114993 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rnf/dpurgjbvuhqbyyqnwuzwhdidgeppo (LogSeriesLimiter: on interval from 2024-10-22 01:06:29 to 2024-10-22 01:06:44 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:06:44.484310 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.88 GiB, free memory in arenas 3.41 MiB, will set to 1.57 GiB (RSS), difference: 23.77 MiB 2024.10.22 01:06:44.516856 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} MergeTask::MergeProjectionsStage: Merge sorted 335 rows, containing 32 columns (32 merged, 0 gathered) in 2.905743004 sec., 115.28892938530498 rows/sec., 11.79 KiB/sec. 2024.10.22 01:06:44.540160 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} MergedBlockOutputStream: filled checksums 202410_1_291_60 (state Temporary) 2024.10.22 01:06:44.590965 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_345_123} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_345_123 to 202410_1_345_123 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:44.595682 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_345_123} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_344_122, 202410_345_345_0] -> 202410_1_345_123 2024.10.22 01:06:44.713536 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.13 MiB. 2024.10.22 01:06:44.995865 [ 1176 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:06:45.052724 [ 1176 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:06:45.292418 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_602_225} MergeTask::MergeProjectionsStage: Merge sorted 36353 rows, containing 13 columns (13 merged, 0 gathered) in 2.638552154 sec., 13777.631776157796 rows/sec., 2.15 MiB/sec. 2024.10.22 01:06:45.316468 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_291_60 to 202410_1_291_60 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:45.319758 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_291_60} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_286_59, 202410_291_291_0] -> 202410_1_291_60 2024.10.22 01:06:45.378814 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2795 2024.10.22 01:06:45.382152 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:06:45.403235 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_602_225} MergedBlockOutputStream: filled checksums 202410_1_602_225 (state Temporary) 2024.10.22 01:06:45.419411 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.88 GiB, free memory in arenas 5.80 MiB, will set to 1.59 GiB (RSS), difference: -16.60 MiB 2024.10.22 01:06:45.657163 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_602_225} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_602_225 to 202410_1_602_225 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:45.767271 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_602_225} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_601_224, 202410_602_602_0] -> 202410_1_602_225 2024.10.22 01:06:45.796238 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:06:46.429541 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.88 GiB, free memory in arenas 5.96 MiB, will set to 1.60 GiB (RSS), difference: -35.26 MiB 2024.10.22 01:06:46.767098 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 35927 2024.10.22 01:06:46.947048 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:46.953013 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:47.016369 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_594_594_0 (state Temporary) 2024.10.22 01:06:47.080190 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_276_55, 202410_277_277_0, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0, 202410_281_281_0] 2024.10.22 01:06:47.087013 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_276_55, 202410_277_277_0, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0, 202410_281_281_0] 2024.10.22 01:06:47.125533 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_276_55, 202410_277_277_0, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0, 202410_281_281_0] 2024.10.22 01:06:47.276555 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_594_594_0 to 202410_607_607_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:47.315185 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_602_225 to 202410_603_603_0 2024.10.22 01:06:47.320606 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.29 MiB using storage policy from min volume index 0 2024.10.22 01:06:47.385527 [ 1150 ] {} DiskLocal: Reserved 1.29 MiB on local disk `default`, having unreserved 297.51 GiB. 2024.10.22 01:06:47.445148 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 35927 2024.10.22 01:06:47.497562 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_603_226} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_602_225 to 202410_603_603_0 into Compact with storage Full 2024.10.22 01:06:47.540516 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_603_226} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:47.550378 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_603_226} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_602_225, total 36353 rows starting from the beginning of the part 2024.10.22 01:06:47.587912 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_603_226} MergeTreeSequentialSource: Reading 2 marks from part 202410_603_603_0, total 87 rows starting from the beginning of the part 2024.10.22 01:06:47.604306 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:06:47.907373 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 203 entries to flush up to offset 28754 2024.10.22 01:06:48.079066 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(6d2e7d37c977b7e001ff81a38968f7c7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6d2e7d37c977b7e001ff81a38968f7c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:06:48.083464 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 641310, file segment info: File segment: [0, 33554431], key: 6d2e7d37c977b7e001ff81a38968f7c7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:06:48.085830 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:48.207821 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:48.209697 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:48.216698 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 31 entries to flush up to offset 5161 2024.10.22 01:06:48.252884 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_345_345_0 (state Temporary) 2024.10.22 01:06:48.285779 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_603_226/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:06:48.316506 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_345_345_0 to 202410_351_351_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:48.356615 [ 1273 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_345_123 to 202410_346_346_0 2024.10.22 01:06:48.368605 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 28754 2024.10.22 01:06:48.383037 [ 1273 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.50 MiB using storage policy from min volume index 0 2024.10.22 01:06:48.403831 [ 1273 ] {} DiskLocal: Reserved 1.50 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:48.452338 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_346_124} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_345_123 to 202410_346_346_0 into Wide with storage Full 2024.10.22 01:06:48.453140 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:48.522235 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:48.570908 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_346_124} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:48.572678 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_349_349_0 (state Temporary) 2024.10.22 01:06:48.580524 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_346_124} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_345_123, total 27670 rows starting from the beginning of the part 2024.10.22 01:06:48.613804 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.88 GiB, free memory in arenas 6.25 MiB, will set to 1.65 GiB (RSS), difference: -106.07 MiB 2024.10.22 01:06:48.715054 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_349_349_0 to 202410_362_362_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:48.800489 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_346_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_346_346_0, total 214 rows starting from the beginning of the part 2024.10.22 01:06:48.805751 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5161 2024.10.22 01:06:49.148172 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_603_226} MergeTask::MergeProjectionsStage: Merge sorted 36440 rows, containing 13 columns (13 merged, 0 gathered) in 1.655362673 sec., 22013.302942224796 rows/sec., 3.44 MiB/sec. 2024.10.22 01:06:49.297434 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_603_226} MergedBlockOutputStream: filled checksums 202410_1_603_226 (state Temporary) 2024.10.22 01:06:49.341439 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_603_226} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_603_226 to 202410_1_603_226 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:49.345424 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_603_226} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_602_225, 202410_603_603_0] -> 202410_1_603_226 2024.10.22 01:06:49.439873 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(cee83f6db1d1f0b14199b4490f242333) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cee83f6db1d1f0b14199b4490f242333, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:06:49.450001 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 582, file segment info: File segment: [0, 33554431], key: cee83f6db1d1f0b14199b4490f242333, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:06:49.453155 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:06:49.475252 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 547760 2024.10.22 01:06:49.502283 [ 1708 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kqy/uuqadegslbwgkjkxxswtczyehjilh, size 641310 2024.10.22 01:06:49.504626 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:06:49.592546 [ 1269 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 311 ms. 2024.10.22 01:06:49.883585 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:06:49.886955 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:06:50.047506 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:06:50.073533 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:06:50.102571 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:06:50.119246 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:06:50.147300 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:06:50.192302 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:06:50.200984 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:06:50.234196 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:06:50.382979 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:06:50.393778 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 257.85 MiB using storage policy from min volume index 0 2024.10.22 01:06:50.479676 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:50.501792 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:50.541345 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bds/qezqtbfndhmavcemljodlcgdisyxe (LogSeriesLimiter: on interval from 2024-10-22 01:06:44 to 2024-10-22 01:06:50 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:06:50.804985 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_334_334_0 (state Temporary) 2024.10.22 01:06:50.892907 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_334_334_0 to 202410_341_341_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:50.911235 [ 1218 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_336_67 to 202410_341_341_0 2024.10.22 01:06:50.930650 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 547760 2024.10.22 01:06:50.930548 [ 1218 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:50.940675 [ 1218 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:50.981257 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_336_67 to 202410_341_341_0 into Compact with storage Full 2024.10.22 01:06:51.108005 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:51.115922 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} MergeTreeSequentialSource: Reading 69 marks from part 202410_1_336_67, total 554054 rows starting from the beginning of the part 2024.10.22 01:06:51.134073 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_337_337_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:06:51.156380 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_338_338_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:06:51.274393 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_339_339_0, total 1545 rows starting from the beginning of the part 2024.10.22 01:06:51.479667 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_340_340_0, total 1527 rows starting from the beginning of the part 2024.10.22 01:06:51.495524 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_341_341_0, total 1920 rows starting from the beginning of the part 2024.10.22 01:06:51.500631 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.88 GiB, free memory in arenas 4.71 MiB, will set to 1.66 GiB (RSS), difference: 10.97 MiB 2024.10.22 01:06:51.517118 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 36007 2024.10.22 01:06:51.809703 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_346_124} MergeTask::MergeProjectionsStage: Merge sorted 27884 rows, containing 13 columns (13 merged, 0 gathered) in 3.369483384 sec., 8275.452590865187 rows/sec., 3.53 MiB/sec. 2024.10.22 01:06:52.091723 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:52.097284 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:52.105688 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_346_124} MergedBlockOutputStream: filled checksums 202410_1_346_124 (state Temporary) 2024.10.22 01:06:52.253807 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_595_595_0 (state Temporary) 2024.10.22 01:06:52.308572 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_595_595_0 to 202410_608_608_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:52.332520 [ 1199 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_603_226 to 202410_604_604_0 2024.10.22 01:06:52.339394 [ 1199 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.29 MiB using storage policy from min volume index 0 2024.10.22 01:06:52.370483 [ 1199 ] {} DiskLocal: Reserved 1.29 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:52.376268 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36007 2024.10.22 01:06:52.400917 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_604_227} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_603_226 to 202410_604_604_0 into Compact with storage Full 2024.10.22 01:06:52.451621 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_604_227} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:52.461382 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_604_227} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_603_226, total 36440 rows starting from the beginning of the part 2024.10.22 01:06:52.478541 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_604_227} MergeTreeSequentialSource: Reading 2 marks from part 202410_604_604_0, total 91 rows starting from the beginning of the part 2024.10.22 01:06:52.692775 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_346_124} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_346_124 to 202410_1_346_124 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:52.709609 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_346_124} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_345_123, 202410_346_346_0] -> 202410_1_346_124 2024.10.22 01:06:52.729827 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.87 MiB. 2024.10.22 01:06:52.898027 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 36 entries to flush up to offset 2831 2024.10.22 01:06:53.428303 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.88 GiB, free memory in arenas 9.23 MiB, will set to 1.66 GiB (RSS), difference: -119.16 MiB 2024.10.22 01:06:53.767576 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_341_68/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:06:55.095192 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zln/efxtdlaxkznseiudgrfuvhsesqiqf (LogSeriesLimiter: on interval from 2024-10-22 01:06:50 to 2024-10-22 01:06:55 accepted series 1 / 179 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:06:55.417032 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.83 GiB, peak 1.88 GiB, free memory in arenas 12.52 MiB, will set to 1.67 GiB (RSS), difference: -159.04 MiB 2024.10.22 01:06:55.589285 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_604_227} MergeTask::MergeProjectionsStage: Merge sorted 36531 rows, containing 13 columns (13 merged, 0 gathered) in 3.193322405 sec., 11439.80950460904 rows/sec., 1.79 MiB/sec. 2024.10.22 01:06:55.645574 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_604_227} MergedBlockOutputStream: filled checksums 202410_1_604_227 (state Temporary) 2024.10.22 01:06:55.707555 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_604_227} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_604_227 to 202410_1_604_227 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:55.720377 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_604_227} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_603_226, 202410_604_604_0] -> 202410_1_604_227 2024.10.22 01:06:55.747214 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:06:55.901688 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 205 entries to flush up to offset 28959 2024.10.22 01:06:56.069568 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:56.077407 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:56.128051 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_346_346_0 (state Temporary) 2024.10.22 01:06:56.291417 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_346_346_0 to 202410_352_352_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:56.307954 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 5175 2024.10.22 01:06:56.312107 [ 1228 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_346_124 to 202410_347_347_0 2024.10.22 01:06:56.316881 [ 1228 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.50 MiB using storage policy from min volume index 0 2024.10.22 01:06:56.320096 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 28959 2024.10.22 01:06:56.321749 [ 1228 ] {} DiskLocal: Reserved 1.50 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:56.392393 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 36047 2024.10.22 01:06:56.441179 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_347_125} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_346_124 to 202410_347_347_0 into Wide with storage Full 2024.10.22 01:06:56.498865 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_347_125} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:56.617205 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:56.623163 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:56.629539 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:56.637501 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:56.671710 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_350_350_0 (state Temporary) 2024.10.22 01:06:56.699798 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_596_596_0 (state Temporary) 2024.10.22 01:06:56.722724 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_350_350_0 to 202410_363_363_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:56.741909 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_347_125} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_346_124, total 27884 rows starting from the beginning of the part 2024.10.22 01:06:56.751140 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 300 2024.10.22 01:06:56.753731 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_596_596_0 to 202410_609_609_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:56.789994 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5175 2024.10.22 01:06:56.888555 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36047 2024.10.22 01:06:56.963876 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_604_227 to 202410_605_605_0 2024.10.22 01:06:56.987704 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.29 MiB using storage policy from min volume index 0 2024.10.22 01:06:56.993085 [ 1243 ] {} DiskLocal: Reserved 1.29 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:56.997594 [ 1243 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 216 ms. 2024.10.22 01:06:57.039796 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_605_228} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_604_227 to 202410_605_605_0 into Compact with storage Full 2024.10.22 01:06:57.089479 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_605_228} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:06:57.099804 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_605_228} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_604_227, total 36531 rows starting from the beginning of the part 2024.10.22 01:06:57.206974 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_347_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_347_347_0, total 185 rows starting from the beginning of the part 2024.10.22 01:06:57.207650 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_605_228} MergeTreeSequentialSource: Reading 2 marks from part 202410_605_605_0, total 73 rows starting from the beginning of the part 2024.10.22 01:06:57.428715 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:57.432177 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:57.629791 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_282_282_0 (state Temporary) 2024.10.22 01:06:57.683962 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.89 GiB, peak 1.89 GiB, free memory in arenas 15.35 MiB, will set to 1.67 GiB (RSS), difference: -218.44 MiB 2024.10.22 01:06:57.834905 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_282_282_0 to 202410_292_292_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:57.939194 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 549296 2024.10.22 01:06:58.080619 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 300 2024.10.22 01:06:58.410851 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_347_125/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:06:58.932088 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:06:58.948905 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:06:59.099533 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_605_228/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:06:59.306921 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_335_335_0 (state Temporary) 2024.10.22 01:06:59.489815 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_335_335_0 to 202410_342_342_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:06:59.592476 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 549296 2024.10.22 01:06:59.642323 [ 1265 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 01:06:59.715501 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_605_228} MergeTask::MergeProjectionsStage: Merge sorted 36604 rows, containing 13 columns (13 merged, 0 gathered) in 2.685538999 sec., 13630.038518759191 rows/sec., 2.13 MiB/sec. 2024.10.22 01:06:59.882460 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:06:59.888124 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:06:59.993148 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_605_228} MergedBlockOutputStream: filled checksums 202410_1_605_228 (state Temporary) 2024.10.22 01:06:59.997162 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_347_125} MergeTask::MergeProjectionsStage: Merge sorted 28069 rows, containing 13 columns (13 merged, 0 gathered) in 3.563563775 sec., 7876.66554389082 rows/sec., 3.36 MiB/sec. 2024.10.22 01:07:00.091487 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_347_125} MergedBlockOutputStream: filled checksums 202410_1_347_125 (state Temporary) 2024.10.22 01:07:00.163762 [ 1172 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:07:00.191540 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_605_228} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_605_228 to 202410_1_605_228 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:00.193980 [ 1172 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:07:00.195490 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_605_228} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_604_227, 202410_605_605_0] -> 202410_1_605_228 2024.10.22 01:07:00.400093 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:07:00.452030 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.89 GiB, free memory in arenas 15.73 MiB, will set to 1.69 GiB (RSS), difference: -12.50 MiB 2024.10.22 01:07:00.713192 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_347_125} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_347_125 to 202410_1_347_125 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:00.716577 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_347_125} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_346_124, 202410_347_347_0] -> 202410_1_347_125 2024.10.22 01:07:00.743934 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.81 MiB. 2024.10.22 01:07:00.893601 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 36101 2024.10.22 01:07:01.279592 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:01.286512 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:07:01.352606 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_597_597_0 (state Temporary) 2024.10.22 01:07:01.389143 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_597_597_0 to 202410_610_610_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:01.405211 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_605_228 to 202410_606_606_0 2024.10.22 01:07:01.409129 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.29 MiB using storage policy from min volume index 0 2024.10.22 01:07:01.441665 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36101 2024.10.22 01:07:01.512599 [ 1179 ] {} DiskLocal: Reserved 1.29 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:07:01.533605 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_606_229} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_605_228 to 202410_606_606_0 into Compact with storage Full 2024.10.22 01:07:01.585088 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_606_229} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:01.592327 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_606_229} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_605_228, total 36604 rows starting from the beginning of the part 2024.10.22 01:07:01.754026 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_606_229} MergeTreeSequentialSource: Reading 2 marks from part 202410_606_606_0, total 86 rows starting from the beginning of the part 2024.10.22 01:07:02.368064 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.89 GiB, free memory in arenas 12.87 MiB, will set to 1.70 GiB (RSS), difference: -9.24 MiB 2024.10.22 01:07:02.982952 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_504_135, 202410_505_505_0] 2024.10.22 01:07:02.987765 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_504_135, 202410_505_505_0] 2024.10.22 01:07:02.996514 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_504_135, 202410_505_505_0] 2024.10.22 01:07:03.088716 [ 1186 ] {} DatabaseCatalog: Execution took 234 ms. 2024.10.22 01:07:03.191175 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_239_49, 202410_240_240_0, 202410_241_241_0, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0] 2024.10.22 01:07:03.192312 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_239_49, 202410_240_240_0, 202410_241_241_0, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0] 2024.10.22 01:07:03.234669 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_239_49, 202410_240_240_0, 202410_241_241_0, 202410_242_242_0, 202410_243_243_0, 202410_244_244_0] 2024.10.22 01:07:03.812943 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_606_229} MergeTask::MergeProjectionsStage: Merge sorted 36690 rows, containing 13 columns (13 merged, 0 gathered) in 2.286022304 sec., 16049.712172887008 rows/sec., 2.51 MiB/sec. 2024.10.22 01:07:03.878920 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 206 entries to flush up to offset 29165 2024.10.22 01:07:03.899749 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_606_229} MergedBlockOutputStream: filled checksums 202410_1_606_229 (state Temporary) 2024.10.22 01:07:04.118380 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_606_229} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_606_229 to 202410_1_606_229 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:04.129181 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_606_229} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_605_228, 202410_606_606_0] -> 202410_1_606_229 2024.10.22 01:07:04.187480 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:07:04.228834 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:04.234893 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:07:04.292092 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 5189 2024.10.22 01:07:04.317016 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_347_347_0 (state Temporary) 2024.10.22 01:07:04.513322 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_347_347_0 to 202410_353_353_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:04.586638 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 29165 2024.10.22 01:07:04.587553 [ 1185 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_347_125 to 202410_348_348_0 2024.10.22 01:07:04.596863 [ 1185 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.51 MiB using storage policy from min volume index 0 2024.10.22 01:07:04.601237 [ 1185 ] {} DiskLocal: Reserved 1.51 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:07:04.610496 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:04.615144 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_348_126} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_347_125 to 202410_348_348_0 into Wide with storage Full 2024.10.22 01:07:04.615105 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 13.10 MiB, will set to 1.70 GiB (RSS), difference: 17.37 MiB 2024.10.22 01:07:04.618636 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:07:04.672822 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_351_351_0 (state Temporary) 2024.10.22 01:07:04.735573 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_351_351_0 to 202410_364_364_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:04.776004 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_348_126} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:04.788011 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_348_126} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_347_125, total 28069 rows starting from the beginning of the part 2024.10.22 01:07:04.882977 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5189 2024.10.22 01:07:05.446992 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 6 similar messages) 2024.10.22 01:07:05.456098 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 36146 2024.10.22 01:07:05.506707 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_348_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_348_348_0, total 200 rows starting from the beginning of the part 2024.10.22 01:07:05.584644 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 301 2024.10.22 01:07:05.689982 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:05.695443 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:07:05.796324 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:05.800068 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.49 GiB. 2024.10.22 01:07:05.806747 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_598_598_0 (state Temporary) 2024.10.22 01:07:05.811010 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_341_68/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:07:05.904150 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(591823c1641fb277a6892c53dbd3ed03) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 591823c1641fb277a6892c53dbd3ed03, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:07:05.977701 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_283_283_0 (state Temporary) 2024.10.22 01:07:06.006931 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 42229, file segment info: File segment: [0, 33554431], key: 591823c1641fb277a6892c53dbd3ed03, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:07:06.008720 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:06.076483 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_283_283_0 to 202410_293_293_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:06.184379 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_598_598_0 to 202410_611_611_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:06.191635 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 301 2024.10.22 01:07:06.208992 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_606_229 to 202410_607_607_0 2024.10.22 01:07:06.240866 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.30 MiB using storage policy from min volume index 0 2024.10.22 01:07:06.245622 [ 1153 ] {} DiskLocal: Reserved 1.30 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:07:06.250154 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36146 2024.10.22 01:07:06.289834 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_607_230} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_606_229 to 202410_607_607_0 into Compact with storage Full 2024.10.22 01:07:06.331256 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_607_230} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:06.338266 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_607_230} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_606_229, total 36690 rows starting from the beginning of the part 2024.10.22 01:07:06.375740 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_607_230} MergeTreeSequentialSource: Reading 2 marks from part 202410_607_607_0, total 68 rows starting from the beginning of the part 2024.10.22 01:07:06.599430 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 550832 2024.10.22 01:07:06.719843 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.89 GiB, free memory in arenas 10.83 MiB, will set to 1.70 GiB (RSS), difference: -66.54 MiB 2024.10.22 01:07:06.789122 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d14f1f70c4c8ce2814cc91992ea41234) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d14f1f70c4c8ce2814cc91992ea41234, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 01:07:06.791139 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 601454, file segment info: File segment: [0, 33554431], key: d14f1f70c4c8ce2814cc91992ea41234, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 01:07:06.797287 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:07.297225 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_348_126} MergeTask::MergeProjectionsStage: Merge sorted 28269 rows, containing 13 columns (13 merged, 0 gathered) in 2.688586804 sec., 10514.445714730959 rows/sec., 4.49 MiB/sec. 2024.10.22 01:07:07.390278 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_348_126} MergedBlockOutputStream: filled checksums 202410_1_348_126 (state Temporary) 2024.10.22 01:07:07.576154 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:07.581808 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.49 GiB. 2024.10.22 01:07:07.605605 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_505_136, 202410_506_506_0] 2024.10.22 01:07:07.607092 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_505_136, 202410_506_506_0] 2024.10.22 01:07:07.691860 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_336_336_0 (state Temporary) 2024.10.22 01:07:07.718540 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(9f498d28b71ced5cae463677ca062239) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9f498d28b71ced5cae463677ca062239, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:07:07.721505 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 520, file segment info: File segment: [0, 33554431], key: 9f498d28b71ced5cae463677ca062239, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:07:07.727625 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:07.778511 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_505_136, 202410_506_506_0] 2024.10.22 01:07:07.792608 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_336_336_0 to 202410_343_343_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:07.993836 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 550832 2024.10.22 01:07:08.104659 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_348_126} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_348_126 to 202410_1_348_126 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:08.106835 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_348_126} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_347_125, 202410_348_348_0] -> 202410_1_348_126 2024.10.22 01:07:08.120441 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.87 MiB. 2024.10.22 01:07:08.211316 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_607_230} MergeTask::MergeProjectionsStage: Merge sorted 36758 rows, containing 13 columns (13 merged, 0 gathered) in 1.931142298 sec., 19034.330115428915 rows/sec., 2.98 MiB/sec. 2024.10.22 01:07:08.293467 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_607_230} MergedBlockOutputStream: filled checksums 202410_1_607_230 (state Temporary) 2024.10.22 01:07:08.382559 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ab05bf47e3493b1c04171bd4c6e09037) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ab05bf47e3493b1c04171bd4c6e09037, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 01:07:08.385022 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 583, file segment info: File segment: [0, 33554431], key: ab05bf47e3493b1c04171bd4c6e09037, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 01:07:08.387759 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:08.499230 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_607_230} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_607_230 to 202410_1_607_230 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:08.504453 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_607_230} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_606_229, 202410_607_607_0] -> 202410_1_607_230 2024.10.22 01:07:08.519846 [ 1739 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bds/qezqtbfndhmavcemljodlcgdisyxe, size 601454 2024.10.22 01:07:08.548264 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:07:08.738506 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:07:08.749710 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:07:08.763072 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:07:08.786401 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:07:08.796652 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:07:08.914063 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:07:08.951952 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:07:08.969465 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:07:09.060970 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(b0909abb3a5f79f4ea25f6827c307704) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b0909abb3a5f79f4ea25f6827c307704, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:07:09.066944 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 10342, file segment info: File segment: [0, 33554431], key: b0909abb3a5f79f4ea25f6827c307704, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:07:09.068931 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:08.989745 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:07:09.125967 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lur/hzrighiudyvdrbomqxybujugrgrun (LogSeriesLimiter: on interval from 2024-10-22 01:06:55 to 2024-10-22 01:07:09 accepted series 1 / 180 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:07:09.379734 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:09.394211 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.50 GiB. 2024.10.22 01:07:09.789473 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(2efaecaf5743bbd8a9f7c8246285c905) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2efaecaf5743bbd8a9f7c8246285c905, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:07:09.798844 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 520, file segment info: File segment: [0, 33554431], key: 2efaecaf5743bbd8a9f7c8246285c905, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:07:09.801466 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:09.858976 [ 1732 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rnf/dpurgjbvuhqbyyqnwuzwhdidgeppo, size 10342 2024.10.22 01:07:09.886000 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:07:09.887807 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:07:10.043567 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:07:10.192119 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:07:10.232494 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:07:10.260406 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 36225 2024.10.22 01:07:10.356290 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:07:10.389107 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:07:10.439138 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.89 GiB, free memory in arenas 14.46 MiB, will set to 1.72 GiB (RSS), difference: 83.60 MiB 2024.10.22 01:07:10.451345 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:10.454690 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.49 GiB. 2024.10.22 01:07:10.571926 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:07:10.607919 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:07:10.755538 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_599_599_0 (state Temporary) 2024.10.22 01:07:10.902983 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_599_599_0 to 202410_612_612_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:10.944033 [ 1217 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_607_230 to 202410_608_608_0 2024.10.22 01:07:10.950838 [ 1217 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.30 MiB using storage policy from min volume index 0 2024.10.22 01:07:10.963081 [ 1217 ] {} DiskLocal: Reserved 1.30 MiB on local disk `default`, having unreserved 297.49 GiB. 2024.10.22 01:07:10.975779 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36225 2024.10.22 01:07:10.981617 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_608_231} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_607_230 to 202410_608_608_0 into Compact with storage Full 2024.10.22 01:07:11.129349 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_608_231} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:11.401297 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_608_231} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_607_230, total 36758 rows starting from the beginning of the part 2024.10.22 01:07:11.452986 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_608_231} MergeTreeSequentialSource: Reading 2 marks from part 202410_608_608_0, total 80 rows starting from the beginning of the part 2024.10.22 01:07:11.659180 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} MergeTask::MergeProjectionsStage: Merge sorted 561350 rows, containing 4 columns (4 merged, 0 gathered) in 20.708999803 sec., 27106.572279684904 rows/sec., 397.41 KiB/sec. 2024.10.22 01:07:11.929791 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} MergedBlockOutputStream: filled checksums 202410_1_341_68 (state Temporary) 2024.10.22 01:07:12.057192 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_341_68 to 202410_1_341_68 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:12.069381 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_341_68} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_336_67, 202410_341_341_0] -> 202410_1_341_68 2024.10.22 01:07:12.250916 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 198 entries to flush up to offset 29363 2024.10.22 01:07:12.387878 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 5208 2024.10.22 01:07:12.434768 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:07:12.489375 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:12.495821 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.48 GiB. 2024.10.22 01:07:12.584316 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:12.588784 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.48 GiB. 2024.10.22 01:07:12.588667 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_348_348_0 (state Temporary) 2024.10.22 01:07:12.648145 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_352_352_0 (state Temporary) 2024.10.22 01:07:12.721193 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.89 GiB, free memory in arenas 12.04 MiB, will set to 1.72 GiB (RSS), difference: 62.49 MiB 2024.10.22 01:07:12.753312 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_352_352_0 to 202410_365_365_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:12.788879 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_348_348_0 to 202410_354_354_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:12.827744 [ 1207 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_348_126 to 202410_349_349_0 2024.10.22 01:07:12.846758 [ 1207 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.52 MiB using storage policy from min volume index 0 2024.10.22 01:07:12.856658 [ 1207 ] {} DiskLocal: Reserved 1.52 MiB on local disk `default`, having unreserved 297.48 GiB. 2024.10.22 01:07:12.875251 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 29363 2024.10.22 01:07:12.889413 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5208 2024.10.22 01:07:12.911984 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_349_127} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_348_126 to 202410_349_349_0 into Wide with storage Full 2024.10.22 01:07:12.997598 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_506_137, 202410_507_507_0] 2024.10.22 01:07:13.000222 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_506_137, 202410_507_507_0] 2024.10.22 01:07:13.008921 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_349_127} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:13.007221 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_506_137, 202410_507_507_0] 2024.10.22 01:07:13.020260 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_349_127} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_348_126, total 28269 rows starting from the beginning of the part 2024.10.22 01:07:13.161095 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_349_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_349_349_0, total 197 rows starting from the beginning of the part 2024.10.22 01:07:13.333469 [ 1230 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:07:13.459581 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_71_71_0 (state Temporary) 2024.10.22 01:07:13.696080 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 302 2024.10.22 01:07:14.025300 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:14.029758 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.48 GiB. 2024.10.22 01:07:14.188829 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_284_284_0 (state Temporary) 2024.10.22 01:07:14.253085 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_608_231} MergeTask::MergeProjectionsStage: Merge sorted 36838 rows, containing 13 columns (13 merged, 0 gathered) in 3.276670852 sec., 11242.50852889755 rows/sec., 1.76 MiB/sec. 2024.10.22 01:07:14.331594 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_608_231} MergedBlockOutputStream: filled checksums 202410_1_608_231 (state Temporary) 2024.10.22 01:07:14.398818 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_284_284_0 to 202410_294_294_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:14.658654 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_608_231} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_608_231 to 202410_1_608_231 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:14.665554 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_608_231} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_607_230, 202410_608_608_0] -> 202410_1_608_231 2024.10.22 01:07:14.707077 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:07:14.709107 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 302 2024.10.22 01:07:15.000315 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 551984 2024.10.22 01:07:15.004655 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 36284 2024.10.22 01:07:15.115638 [ 1721 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zln/efxtdlaxkznseiudgrfuvhsesqiqf, size 66550 2024.10.22 01:07:15.200062 [ 1215 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:07:15.264887 [ 1215 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:07:15.404305 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:15.411005 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.47 GiB. 2024.10.22 01:07:15.484475 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_349_127} MergeTask::MergeProjectionsStage: Merge sorted 28466 rows, containing 13 columns (13 merged, 0 gathered) in 2.578551406 sec., 11039.531705190291 rows/sec., 4.72 MiB/sec. 2024.10.22 01:07:15.486269 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_600_600_0 (state Temporary) 2024.10.22 01:07:15.660931 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_349_127} MergedBlockOutputStream: filled checksums 202410_1_349_127 (state Temporary) 2024.10.22 01:07:15.798416 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_600_600_0 to 202410_613_613_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:15.871934 [ 1236 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_608_231 to 202410_609_609_0 2024.10.22 01:07:15.895590 [ 1236 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.30 MiB using storage policy from min volume index 0 2024.10.22 01:07:15.900854 [ 1236 ] {} DiskLocal: Reserved 1.30 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:15.904851 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:15.912239 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:15.951525 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.89 GiB, free memory in arenas 7.99 MiB, will set to 1.71 GiB (RSS), difference: 203.32 MiB 2024.10.22 01:07:15.968357 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_609_232} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_608_231 to 202410_609_609_0 into Compact with storage Full 2024.10.22 01:07:15.991609 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36284 2024.10.22 01:07:16.048239 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_609_232} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:16.066844 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_609_232} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_608_231, total 36838 rows starting from the beginning of the part 2024.10.22 01:07:16.099817 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_337_337_0 (state Temporary) 2024.10.22 01:07:16.136402 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_609_232} MergeTreeSequentialSource: Reading 2 marks from part 202410_609_609_0, total 40 rows starting from the beginning of the part 2024.10.22 01:07:16.190804 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_337_337_0 to 202410_344_344_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:16.210061 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 551984 2024.10.22 01:07:16.456015 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_21_21_0 (state Temporary) 2024.10.22 01:07:16.475233 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/onk/lrbsfruzyjurltghpfxnfzceochko (LogSeriesLimiter: on interval from 2024-10-22 01:07:09 to 2024-10-22 01:07:16 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:07:16.585623 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_349_127} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_349_127 to 202410_1_349_127 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:16.589729 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_349_127} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_348_126, 202410_349_349_0] -> 202410_1_349_127 2024.10.22 01:07:16.603175 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.87 MiB. 2024.10.22 01:07:16.642632 [ 1673 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/onk/lrbsfruzyjurltghpfxnfzceochko, size 4 2024.10.22 01:07:17.095582 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_291_85, 202410_292_292_0] 2024.10.22 01:07:17.098332 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_291_85, 202410_292_292_0] 2024.10.22 01:07:17.107113 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_291_85, 202410_292_292_0] 2024.10.22 01:07:17.586479 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.89 GiB, free memory in arenas 4.63 MiB, will set to 1.69 GiB (RSS), difference: 42.92 MiB 2024.10.22 01:07:17.605716 [ 1151 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 210 ms. 2024.10.22 01:07:17.729163 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_507_138, 202410_508_508_0] 2024.10.22 01:07:17.733744 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_507_138, 202410_508_508_0] 2024.10.22 01:07:17.744462 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_507_138, 202410_508_508_0] 2024.10.22 01:07:17.802429 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_609_232} MergeTask::MergeProjectionsStage: Merge sorted 36878 rows, containing 13 columns (13 merged, 0 gathered) in 1.842973062 sec., 20010.059159508215 rows/sec., 3.13 MiB/sec. 2024.10.22 01:07:18.084918 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_609_232} MergedBlockOutputStream: filled checksums 202410_1_609_232 (state Temporary) 2024.10.22 01:07:18.134386 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_609_232} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_609_232 to 202410_1_609_232 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:18.137645 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_609_232} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_608_231, 202410_609_609_0] -> 202410_1_609_232 2024.10.22 01:07:18.181399 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:07:19.492700 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.89 GiB, free memory in arenas 4.18 MiB, will set to 1.68 GiB (RSS), difference: 5.18 MiB 2024.10.22 01:07:19.887335 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:07:19.891855 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:07:19.942920 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:07:20.008398 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 36332 2024.10.22 01:07:20.198165 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_301_60, 202410_302_302_0, 202410_303_303_0, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0] 2024.10.22 01:07:20.208853 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_301_60, 202410_302_302_0, 202410_303_303_0, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0] 2024.10.22 01:07:20.244611 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_301_60, 202410_302_302_0, 202410_303_303_0, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0] 2024.10.22 01:07:20.342711 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:20.347364 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:20.392770 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 210 entries to flush up to offset 29573 2024.10.22 01:07:20.393033 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_601_601_0 (state Temporary) 2024.10.22 01:07:20.394485 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 5222 2024.10.22 01:07:20.524241 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_601_601_0 to 202410_614_614_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:20.565777 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_609_232 to 202410_610_610_0 2024.10.22 01:07:20.580108 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36332 2024.10.22 01:07:20.596290 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:20.598771 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:20.630113 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_353_353_0 (state Temporary) 2024.10.22 01:07:20.678969 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.30 MiB using storage policy from min volume index 0 2024.10.22 01:07:20.683411 [ 1265 ] {} DiskLocal: Reserved 1.30 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:20.710537 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_610_233} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_609_232 to 202410_610_610_0 into Compact with storage Full 2024.10.22 01:07:20.750323 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_610_233} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:20.759320 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_610_233} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_609_232, total 36878 rows starting from the beginning of the part 2024.10.22 01:07:20.782581 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_610_233} MergeTreeSequentialSource: Reading 2 marks from part 202410_610_610_0, total 54 rows starting from the beginning of the part 2024.10.22 01:07:20.886464 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:20.888236 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:21.000488 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_353_353_0 to 202410_366_366_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:21.042100 [ 1157 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_361_72 to 202410_366_366_0 2024.10.22 01:07:21.061945 [ 1157 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:21.069718 [ 1157 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:21.094516 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_361_72 to 202410_366_366_0 into Compact with storage Full 2024.10.22 01:07:21.147747 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_349_349_0 (state Temporary) 2024.10.22 01:07:21.154083 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:21.160030 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_361_72, total 5236 rows starting from the beginning of the part 2024.10.22 01:07:21.189641 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_362_362_0, total 31 rows starting from the beginning of the part 2024.10.22 01:07:21.201634 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_349_349_0 to 202410_355_355_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:21.205360 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_363_363_0, total 14 rows starting from the beginning of the part 2024.10.22 01:07:21.236901 [ 1252 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_349_127 to 202410_350_350_0 2024.10.22 01:07:21.239251 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 29573 2024.10.22 01:07:21.254636 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5222 2024.10.22 01:07:21.347592 [ 1252 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.53 MiB using storage policy from min volume index 0 2024.10.22 01:07:21.350524 [ 1252 ] {} DiskLocal: Reserved 1.53 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:21.351380 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_364_364_0, total 14 rows starting from the beginning of the part 2024.10.22 01:07:21.388134 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_365_365_0, total 19 rows starting from the beginning of the part 2024.10.22 01:07:21.400532 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_350_128} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_349_127 to 202410_350_350_0 into Wide with storage Full 2024.10.22 01:07:21.408856 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 4.87 MiB, will set to 1.68 GiB (RSS), difference: -4.31 MiB 2024.10.22 01:07:21.416118 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_366_366_0, total 14 rows starting from the beginning of the part 2024.10.22 01:07:21.463007 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_350_128} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:21.478842 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_350_128} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_349_127, total 28466 rows starting from the beginning of the part 2024.10.22 01:07:21.605777 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_350_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_350_350_0, total 189 rows starting from the beginning of the part 2024.10.22 01:07:21.914495 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_71_71_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:07:22.224429 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 303 2024.10.22 01:07:22.226485 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_508_139, 202410_509_509_0] 2024.10.22 01:07:22.229781 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_508_139, 202410_509_509_0] 2024.10.22 01:07:22.242666 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_508_139, 202410_509_509_0] 2024.10.22 01:07:22.596643 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:22.599397 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:22.668362 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_285_285_0 (state Temporary) 2024.10.22 01:07:22.700466 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} MergeTask::MergeProjectionsStage: Merge sorted 5328 rows, containing 12 columns (12 merged, 0 gathered) in 1.608555165 sec., 3312.289261773624 rows/sec., 522.22 KiB/sec. 2024.10.22 01:07:22.772862 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} MergedBlockOutputStream: filled checksums 202410_1_366_73 (state Temporary) 2024.10.22 01:07:22.802129 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_285_285_0 to 202410_295_295_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:22.968176 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_366_73 to 202410_1_366_73 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:22.976839 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_366_73} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_361_72, 202410_366_366_0] -> 202410_1_366_73 2024.10.22 01:07:23.015923 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.42 MiB. 2024.10.22 01:07:23.100700 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 303 2024.10.22 01:07:23.213385 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 553520 2024.10.22 01:07:23.375618 [ 1183 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 01:07:23.605108 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.89 GiB, free memory in arenas 4.30 MiB, will set to 1.70 GiB (RSS), difference: -38.06 MiB 2024.10.22 01:07:23.791185 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_71_71_0 to 202410_74_74_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:23.837177 [ 1189 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_69_14 to 202410_74_74_0 2024.10.22 01:07:23.846013 [ 1189 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.55 MiB using storage policy from min volume index 0 2024.10.22 01:07:23.852695 [ 1189 ] {} DiskLocal: Reserved 1.55 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:23.891637 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_69_14 to 202410_74_74_0 into Compact with storage Full 2024.10.22 01:07:23.986699 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_610_233} MergeTask::MergeProjectionsStage: Merge sorted 36932 rows, containing 13 columns (13 merged, 0 gathered) in 3.282129859 sec., 11252.449350450885 rows/sec., 1.76 MiB/sec. 2024.10.22 01:07:24.001147 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/usg/bfhrzuxddlohkdgmsaudowlfmdqzd. 2024.10.22 01:07:24.008044 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:24.012879 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:24.138208 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_610_233} MergedBlockOutputStream: filled checksums 202410_1_610_233 (state Temporary) 2024.10.22 01:07:24.192622 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_292_86, 202410_293_293_0] 2024.10.22 01:07:24.195697 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_292_86, 202410_293_293_0] 2024.10.22 01:07:24.206566 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_292_86, 202410_293_293_0] 2024.10.22 01:07:24.241537 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_338_338_0 (state Temporary) 2024.10.22 01:07:24.409397 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_338_338_0 to 202410_345_345_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:24.525124 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_350_128} MergeTask::MergeProjectionsStage: Merge sorted 28655 rows, containing 13 columns (13 merged, 0 gathered) in 3.133293304 sec., 9145.329600461815 rows/sec., 3.91 MiB/sec. 2024.10.22 01:07:24.544683 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 553520 2024.10.22 01:07:24.591646 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 36414 2024.10.22 01:07:24.697190 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_610_233} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_610_233 to 202410_1_610_233 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:24.703770 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_610_233} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_609_232, 202410_610_610_0] -> 202410_1_610_233 2024.10.22 01:07:24.759120 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_350_128} MergedBlockOutputStream: filled checksums 202410_1_350_128 (state Temporary) 2024.10.22 01:07:24.761179 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:07:24.988674 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:24.992759 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:25.203460 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_602_602_0 (state Temporary) 2024.10.22 01:07:25.286732 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_602_602_0 to 202410_615_615_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:25.303033 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_350_128} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_350_128 to 202410_1_350_128 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:25.304542 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_610_233 to 202410_611_611_0 2024.10.22 01:07:25.305691 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_350_128} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_349_127, 202410_350_350_0] -> 202410_1_350_128 2024.10.22 01:07:25.309392 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.30 MiB using storage policy from min volume index 0 2024.10.22 01:07:25.312967 [ 1196 ] {} DiskLocal: Reserved 1.30 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:25.327905 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36414 2024.10.22 01:07:25.347604 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_611_234} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_610_233 to 202410_611_611_0 into Compact with storage Full 2024.10.22 01:07:25.350866 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.88 MiB. 2024.10.22 01:07:25.430769 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_611_234} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:25.464619 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_611_234} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_610_233, total 36932 rows starting from the beginning of the part 2024.10.22 01:07:25.499756 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.89 GiB, free memory in arenas 3.64 MiB, will set to 1.68 GiB (RSS), difference: 47.80 MiB 2024.10.22 01:07:25.538919 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_611_234} MergeTreeSequentialSource: Reading 2 marks from part 202410_611_611_0, total 45 rows starting from the beginning of the part 2024.10.22 01:07:26.886896 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(0091f0081388ddb6e4a4f8ed442bcd1b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0091f0081388ddb6e4a4f8ed442bcd1b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:07:26.894328 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 374538, file segment info: File segment: [0, 33554431], key: 0091f0081388ddb6e4a4f8ed442bcd1b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:07:26.896189 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:27.549122 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 6.25 MiB, will set to 1.68 GiB (RSS), difference: -6.05 MiB 2024.10.22 01:07:27.636828 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2831 2024.10.22 01:07:27.689719 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_509_140, 202410_510_510_0] 2024.10.22 01:07:27.693374 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_509_140, 202410_510_510_0] 2024.10.22 01:07:27.712184 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_509_140, 202410_510_510_0] 2024.10.22 01:07:28.043518 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ded603209da8b8d3b2b89a2313cbbf01) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ded603209da8b8d3b2b89a2313cbbf01, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:07:28.049145 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 577, file segment info: File segment: [0, 33554431], key: ded603209da8b8d3b2b89a2313cbbf01, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:07:28.053837 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:28.104208 [ 1739 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lur/hzrighiudyvdrbomqxybujugrgrun, size 374538 2024.10.22 01:07:28.232789 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:07:28.240952 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:07:28.251963 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:07:28.278796 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:07:28.289797 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:07:28.300989 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:07:28.421118 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:07:28.431088 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:07:28.458690 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:07:28.507074 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rvv/fylhznkzzfkehevenqiqkljlraewa (LogSeriesLimiter: on interval from 2024-10-22 01:07:16 to 2024-10-22 01:07:28 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:07:28.757012 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 5242 2024.10.22 01:07:28.789712 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 206 entries to flush up to offset 29779 2024.10.22 01:07:28.987425 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_611_234} MergeTask::MergeProjectionsStage: Merge sorted 36977 rows, containing 13 columns (13 merged, 0 gathered) in 3.650202632 sec., 10130.122551508806 rows/sec., 1.58 MiB/sec. 2024.10.22 01:07:29.144844 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_611_234} MergedBlockOutputStream: filled checksums 202410_1_611_234 (state Temporary) 2024.10.22 01:07:29.221817 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:29.231877 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:29.295460 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:29.297604 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:29.338620 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_354_354_0 (state Temporary) 2024.10.22 01:07:29.360037 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 36465 2024.10.22 01:07:29.406236 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_350_350_0 (state Temporary) 2024.10.22 01:07:29.416048 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_354_354_0 to 202410_367_367_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:29.484241 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_611_234} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_611_234 to 202410_1_611_234 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:29.488168 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_611_234} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_610_233, 202410_611_611_0] -> 202410_1_611_234 2024.10.22 01:07:29.495879 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5242 2024.10.22 01:07:29.497626 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_350_350_0 to 202410_356_356_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:29.506584 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:07:29.523070 [ 1238 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_350_128 to 202410_351_351_0 2024.10.22 01:07:29.528815 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(2c1560d74c092e3e3199832096610092) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2c1560d74c092e3e3199832096610092, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:07:29.530203 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.89 GiB, free memory in arenas 7.68 MiB, will set to 1.68 GiB (RSS), difference: 3.80 MiB 2024.10.22 01:07:29.535182 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 29779 2024.10.22 01:07:29.535681 [ 1238 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.54 MiB using storage policy from min volume index 0 2024.10.22 01:07:29.536430 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1182, file segment info: File segment: [0, 33554431], key: 2c1560d74c092e3e3199832096610092, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:07:29.539426 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:29.540170 [ 1238 ] {} DiskLocal: Reserved 1.54 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:29.572443 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_351_129} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_350_128 to 202410_351_351_0 into Wide with storage Full 2024.10.22 01:07:29.689120 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:29.693499 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:29.693524 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_351_129} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:29.707052 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_351_129} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_350_128, total 28655 rows starting from the beginning of the part 2024.10.22 01:07:29.748748 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_603_603_0 (state Temporary) 2024.10.22 01:07:29.884755 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_351_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_351_351_0, total 203 rows starting from the beginning of the part 2024.10.22 01:07:29.886360 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:07:29.888923 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:07:29.910280 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_603_603_0 to 202410_616_616_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:29.986267 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_611_234 to 202410_612_612_0 2024.10.22 01:07:29.991943 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 01:07:29.993203 [ 1195 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:29.999380 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36465 2024.10.22 01:07:30.014711 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_612_235} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_611_234 to 202410_612_612_0 into Compact with storage Full 2024.10.22 01:07:30.056744 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_612_235} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:30.070663 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_612_235} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_611_234, total 36977 rows starting from the beginning of the part 2024.10.22 01:07:30.097704 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_281_56, 202410_282_282_0, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0, 202410_286_286_0] 2024.10.22 01:07:30.099853 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_281_56, 202410_282_282_0, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0, 202410_286_286_0] 2024.10.22 01:07:30.144425 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_281_56, 202410_282_282_0, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0, 202410_286_286_0] 2024.10.22 01:07:30.203908 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(5e4ee5d233542998094859aa8a3fe862) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5e4ee5d233542998094859aa8a3fe862, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:07:30.219301 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_612_235} MergeTreeSequentialSource: Reading 2 marks from part 202410_612_612_0, total 79 rows starting from the beginning of the part 2024.10.22 01:07:30.219840 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 155, file segment info: File segment: [0, 33554431], key: 5e4ee5d233542998094859aa8a3fe862, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:07:30.224741 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:30.271438 [ 1208 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:07:30.303334 [ 1208 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:07:30.604060 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 304 2024.10.22 01:07:30.818583 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(86a61c55b65b9cd9a0d6ed39b6aff592) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 86a61c55b65b9cd9a0d6ed39b6aff592, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:07:30.836321 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1086, file segment info: File segment: [0, 33554431], key: 86a61c55b65b9cd9a0d6ed39b6aff592, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:07:30.839816 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:30.984269 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:30.990995 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:31.055723 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_286_286_0 (state Temporary) 2024.10.22 01:07:31.340373 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_286_286_0 to 202410_296_296_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:31.382834 [ 1213 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_291_60 to 202410_296_296_0 2024.10.22 01:07:31.411619 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 304 2024.10.22 01:07:31.493155 [ 1213 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:31.495318 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:31.550809 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 555056 2024.10.22 01:07:31.623923 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_291_60 to 202410_296_296_0 into Compact with storage Full 2024.10.22 01:07:31.666840 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_351_129/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:07:31.699678 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_351_129} MergeTask::MergeProjectionsStage: Merge sorted 28858 rows, containing 13 columns (13 merged, 0 gathered) in 2.144011079 sec., 13459.818506842705 rows/sec., 5.75 MiB/sec. 2024.10.22 01:07:31.720012 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(4bb8223b745898646ec0ff9e1f644bb5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4bb8223b745898646ec0ff9e1f644bb5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:07:31.739003 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 469, file segment info: File segment: [0, 33554431], key: 4bb8223b745898646ec0ff9e1f644bb5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:07:31.740952 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:31.863865 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_351_129} MergedBlockOutputStream: filled checksums 202410_1_351_129 (state Temporary) 2024.10.22 01:07:31.876958 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:31.895785 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_291_60, total 335 rows starting from the beginning of the part 2024.10.22 01:07:31.931258 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_292_292_0, total 1 rows starting from the beginning of the part 2024.10.22 01:07:31.982845 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_293_293_0, total 1 rows starting from the beginning of the part 2024.10.22 01:07:32.033806 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_510_141, 202410_511_511_0] 2024.10.22 01:07:32.036162 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_510_141, 202410_511_511_0] 2024.10.22 01:07:32.149035 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_510_141, 202410_511_511_0] 2024.10.22 01:07:32.225430 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_294_294_0, total 1 rows starting from the beginning of the part 2024.10.22 01:07:32.273846 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:32.279941 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:32.343486 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:07:32.343735 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_295_295_0, total 1 rows starting from the beginning of the part 2024.10.22 01:07:32.373662 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:07:32.387805 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:07:32.398429 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_296_296_0, total 1 rows starting from the beginning of the part 2024.10.22 01:07:32.444381 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:07:32.445040 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_351_129} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_351_129 to 202410_1_351_129 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:32.451854 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_351_129} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_350_128, 202410_351_351_0] -> 202410_1_351_129 2024.10.22 01:07:32.487208 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_339_339_0 (state Temporary) 2024.10.22 01:07:32.556388 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 7.38 MiB, will set to 1.69 GiB (RSS), difference: -7.57 MiB 2024.10.22 01:07:32.568514 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_612_235} MergeTask::MergeProjectionsStage: Merge sorted 37056 rows, containing 13 columns (13 merged, 0 gathered) in 2.560769232 sec., 14470.651840446666 rows/sec., 2.27 MiB/sec. 2024.10.22 01:07:32.587366 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.93 MiB. 2024.10.22 01:07:32.613637 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_339_339_0 to 202410_346_346_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:32.652987 [ 1214 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_341_68 to 202410_346_346_0 2024.10.22 01:07:32.678387 [ 1214 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:32.680809 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:07:32.683126 [ 1214 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:32.698796 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_612_235} MergedBlockOutputStream: filled checksums 202410_1_612_235 (state Temporary) 2024.10.22 01:07:32.806560 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:07:32.809642 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_341_68 to 202410_346_346_0 into Compact with storage Full 2024.10.22 01:07:32.838526 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:07:32.848598 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:32.864446 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 555056 2024.10.22 01:07:32.964067 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} MergeTreeSequentialSource: Reading 70 marks from part 202410_1_341_68, total 561350 rows starting from the beginning of the part 2024.10.22 01:07:32.984984 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_342_342_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:07:32.998678 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_343_343_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:07:33.014669 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_344_344_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:07:33.039896 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_345_345_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:07:33.056291 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_612_235} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_612_235 to 202410_1_612_235 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:33.081581 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_612_235} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_611_234, 202410_612_612_0] -> 202410_1_612_235 2024.10.22 01:07:33.098918 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/klq/aooluchlywvibxzlgfnaevfzmckqn (LogSeriesLimiter: on interval from 2024-10-22 01:07:28 to 2024-10-22 01:07:33 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:07:33.108846 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:07:33.270948 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_346_346_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:07:33.339200 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_293_87, 202410_294_294_0] 2024.10.22 01:07:33.344271 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_293_87, 202410_294_294_0] 2024.10.22 01:07:33.359669 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_293_87, 202410_294_294_0] 2024.10.22 01:07:33.515035 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:33.731619 [ 1234 ] {} DatabaseCatalog: Execution took 331 ms. 2024.10.22 01:07:33.823551 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} MergeTask::MergeProjectionsStage: Merge sorted 340 rows, containing 32 columns (32 merged, 0 gathered) in 2.311345528 sec., 147.1004641587279 rows/sec., 15.02 KiB/sec. 2024.10.22 01:07:33.847610 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} MergedBlockOutputStream: filled checksums 202410_1_296_61 (state Temporary) 2024.10.22 01:07:34.115378 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 113 entries to flush up to offset 36578 2024.10.22 01:07:34.148085 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_69_14, total 2709 rows starting from the beginning of the part 2024.10.22 01:07:34.514180 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 7.58 MiB, will set to 1.68 GiB (RSS), difference: -8.62 MiB 2024.10.22 01:07:34.652385 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_296_61 to 202410_1_296_61 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:34.659190 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_296_61} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_291_60, 202410_296_296_0] -> 202410_1_296_61 2024.10.22 01:07:34.699283 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:07:34.896189 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:34.898610 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:35.086461 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_604_604_0 (state Temporary) 2024.10.22 01:07:35.186767 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_604_604_0 to 202410_617_617_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:35.177046 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 2874 2024.10.22 01:07:35.242331 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_612_235 to 202410_613_613_0 2024.10.22 01:07:35.243563 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36578 2024.10.22 01:07:35.251920 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 01:07:35.255340 [ 1257 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:35.287435 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_613_236} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_612_235 to 202410_613_613_0 into Compact with storage Full 2024.10.22 01:07:35.314247 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_613_236} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:35.331107 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_613_236} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_612_235, total 37056 rows starting from the beginning of the part 2024.10.22 01:07:35.354364 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_613_236} MergeTreeSequentialSource: Reading 2 marks from part 202410_613_613_0, total 59 rows starting from the beginning of the part 2024.10.22 01:07:35.854742 [ 1255 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 01:07:36.208478 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_70_70_0, total 35 rows starting from the beginning of the part 2024.10.22 01:07:36.274525 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_613_236/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:07:36.413198 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 10.38 MiB, will set to 1.69 GiB (RSS), difference: -2.75 MiB 2024.10.22 01:07:36.508185 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_511_142, 202410_512_512_0] 2024.10.22 01:07:36.511273 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_511_142, 202410_512_512_0] 2024.10.22 01:07:36.520291 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_511_142, 202410_512_512_0] 2024.10.22 01:07:36.997073 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 5264 2024.10.22 01:07:37.093594 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 249 entries to flush up to offset 30028 2024.10.22 01:07:37.189468 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:37.192735 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:37.229079 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_355_355_0 (state Temporary) 2024.10.22 01:07:37.335400 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_355_355_0 to 202410_368_368_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:37.398782 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5264 2024.10.22 01:07:37.595439 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:37.600855 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:37.660342 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_351_351_0 (state Temporary) 2024.10.22 01:07:37.750419 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_351_351_0 to 202410_357_357_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:37.792104 [ 1157 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_351_129 to 202410_352_352_0 2024.10.22 01:07:37.794590 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 30028 2024.10.22 01:07:37.799423 [ 1157 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.55 MiB using storage policy from min volume index 0 2024.10.22 01:07:37.803163 [ 1157 ] {} DiskLocal: Reserved 1.55 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:37.829753 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_346_69/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:07:37.845830 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_352_130} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_351_129 to 202410_352_352_0 into Wide with storage Full 2024.10.22 01:07:37.918714 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_352_130} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:37.948967 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_352_130} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_351_129, total 28858 rows starting from the beginning of the part 2024.10.22 01:07:38.066337 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_613_236} MergeTask::MergeProjectionsStage: Merge sorted 37115 rows, containing 13 columns (13 merged, 0 gathered) in 2.800302993 sec., 13253.922912191096 rows/sec., 2.07 MiB/sec. 2024.10.22 01:07:38.104316 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_613_236} MergedBlockOutputStream: filled checksums 202410_1_613_236 (state Temporary) 2024.10.22 01:07:38.301744 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_613_236} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_613_236 to 202410_1_613_236 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:38.304249 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_613_236} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_612_235, 202410_613_613_0] -> 202410_1_613_236 2024.10.22 01:07:38.453358 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.89 GiB, free memory in arenas 9.02 MiB, will set to 1.69 GiB (RSS), difference: 21.03 MiB 2024.10.22 01:07:38.466987 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:07:38.503313 [ 1155 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 290 ms. 2024.10.22 01:07:38.664270 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_352_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_352_352_0, total 205 rows starting from the beginning of the part 2024.10.22 01:07:39.265830 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 36631 2024.10.22 01:07:39.268190 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_71_71_0, total 39 rows starting from the beginning of the part 2024.10.22 01:07:39.499457 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:39.502118 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:39.559367 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_605_605_0 (state Temporary) 2024.10.22 01:07:39.668780 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_512_143, 202410_513_513_0] 2024.10.22 01:07:39.669301 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_605_605_0 to 202410_618_618_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:39.678040 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_512_143, 202410_513_513_0] 2024.10.22 01:07:39.697786 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_512_143, 202410_513_513_0] 2024.10.22 01:07:39.699891 [ 1156 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_613_236 to 202410_614_614_0 2024.10.22 01:07:39.702686 [ 1156 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 01:07:39.725252 [ 1156 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:39.724850 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36631 2024.10.22 01:07:39.850747 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_614_237} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_613_236 to 202410_614_614_0 into Compact with storage Full 2024.10.22 01:07:39.872202 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 556592 2024.10.22 01:07:39.897601 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:07:39.901991 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:07:40.007625 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_614_237} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:40.026986 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_614_237} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_613_236, total 37115 rows starting from the beginning of the part 2024.10.22 01:07:40.063087 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_614_237} MergeTreeSequentialSource: Reading 2 marks from part 202410_614_614_0, total 48 rows starting from the beginning of the part 2024.10.22 01:07:40.511283 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:40.514824 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:40.654680 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_340_340_0 (state Temporary) 2024.10.22 01:07:40.701689 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.89 GiB, free memory in arenas 9.07 MiB, will set to 1.70 GiB (RSS), difference: -54.18 MiB 2024.10.22 01:07:40.903489 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_340_340_0 to 202410_347_347_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:40.956765 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 556592 2024.10.22 01:07:41.032882 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_72_72_0, total 39 rows starting from the beginning of the part 2024.10.22 01:07:41.277784 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:07:41.756242 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_352_130} MergeTask::MergeProjectionsStage: Merge sorted 29063 rows, containing 13 columns (13 merged, 0 gathered) in 3.92555209 sec., 7403.544605620046 rows/sec., 3.17 MiB/sec. 2024.10.22 01:07:41.876727 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_294_88, 202410_295_295_0] 2024.10.22 01:07:41.883519 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_294_88, 202410_295_295_0] 2024.10.22 01:07:41.896613 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_294_88, 202410_295_295_0] 2024.10.22 01:07:42.074760 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_352_130} MergedBlockOutputStream: filled checksums 202410_1_352_130 (state Temporary) 2024.10.22 01:07:42.583898 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.89 GiB, free memory in arenas 10.31 MiB, will set to 1.70 GiB (RSS), difference: 54.84 MiB 2024.10.22 01:07:42.720228 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_73_73_0, total 35 rows starting from the beginning of the part 2024.10.22 01:07:42.783747 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_614_237} MergeTask::MergeProjectionsStage: Merge sorted 37163 rows, containing 13 columns (13 merged, 0 gathered) in 2.945851482 sec., 12615.36782389629 rows/sec., 1.97 MiB/sec. 2024.10.22 01:07:42.925457 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_352_130} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_352_130 to 202410_1_352_130 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:43.005609 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_614_237} MergedBlockOutputStream: filled checksums 202410_1_614_237 (state Temporary) 2024.10.22 01:07:43.065135 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_352_130} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_351_129, 202410_352_352_0] -> 202410_1_352_130 2024.10.22 01:07:43.088928 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 61.13 MiB. 2024.10.22 01:07:43.123680 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_614_237} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_614_237 to 202410_1_614_237 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:43.129114 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_614_237} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_613_236, 202410_614_614_0] -> 202410_1_614_237 2024.10.22 01:07:43.167066 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:07:43.743707 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 36674 2024.10.22 01:07:43.748704 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_513_144, 202410_514_514_0] 2024.10.22 01:07:43.751828 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_513_144, 202410_514_514_0] 2024.10.22 01:07:43.876300 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_513_144, 202410_514_514_0] 2024.10.22 01:07:44.083884 [ 1188 ] {} DatabaseCatalog: Execution took 347 ms. 2024.10.22 01:07:44.091562 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:44.093287 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:44.156130 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_606_606_0 (state Temporary) 2024.10.22 01:07:44.349321 [ 1425 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Found 5 old parts to remove. Parts: [202410_1_13_3, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0, 202410_17_17_0] 2024.10.22 01:07:44.355562 [ 1425 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from filesystem (serially): Parts: [202410_1_13_3, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0, 202410_17_17_0] 2024.10.22 01:07:44.373511 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_74_74_0, total 36 rows starting from the beginning of the part 2024.10.22 01:07:44.384191 [ 1425 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from memory: Parts: [202410_1_13_3, 202410_14_14_0, 202410_15_15_0, 202410_16_16_0, 202410_17_17_0] 2024.10.22 01:07:44.548039 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_606_606_0 to 202410_619_619_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:44.609417 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36674 2024.10.22 01:07:44.683896 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 10.89 MiB, will set to 1.69 GiB (RSS), difference: 1.32 MiB 2024.10.22 01:07:44.693463 [ 1245 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_614_237 to 202410_615_615_0 2024.10.22 01:07:44.699578 [ 1245 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 01:07:44.802685 [ 1245 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:44.809261 [ 1245 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 244 ms. 2024.10.22 01:07:44.848195 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_615_238} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_614_237 to 202410_615_615_0 into Compact with storage Full 2024.10.22 01:07:44.900448 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 5278 2024.10.22 01:07:44.962586 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_615_238} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:45.041297 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_615_238} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_614_237, total 37163 rows starting from the beginning of the part 2024.10.22 01:07:45.296183 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:45.307596 [ 1274 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:07:45.331540 [ 1274 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:07:45.352366 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 193 entries to flush up to offset 30221 2024.10.22 01:07:45.384094 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:45.391725 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_615_238} MergeTreeSequentialSource: Reading 2 marks from part 202410_615_615_0, total 82 rows starting from the beginning of the part 2024.10.22 01:07:45.438510 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_356_356_0 (state Temporary) 2024.10.22 01:07:45.648283 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_356_356_0 to 202410_369_369_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:45.800100 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5278 2024.10.22 01:07:46.237010 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:46.240335 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:46.300463 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_352_352_0 (state Temporary) 2024.10.22 01:07:46.370188 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_352_352_0 to 202410_358_358_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:46.393815 [ 1275 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_352_130 to 202410_353_353_0 2024.10.22 01:07:46.403371 [ 1275 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.56 MiB using storage policy from min volume index 0 2024.10.22 01:07:46.412669 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 30221 2024.10.22 01:07:46.415322 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 305 2024.10.22 01:07:46.427033 [ 1275 ] {} DiskLocal: Reserved 1.56 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:46.460233 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_353_131} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_352_130 to 202410_353_353_0 into Wide with storage Full 2024.10.22 01:07:46.599834 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.89 GiB, free memory in arenas 9.50 MiB, will set to 1.69 GiB (RSS), difference: -16.54 MiB 2024.10.22 01:07:46.700293 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_353_131} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:46.711600 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_353_131} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_352_130, total 29063 rows starting from the beginning of the part 2024.10.22 01:07:46.827317 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:46.831493 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:47.102686 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_353_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_353_353_0, total 206 rows starting from the beginning of the part 2024.10.22 01:07:47.130743 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_287_287_0 (state Temporary) 2024.10.22 01:07:47.315895 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_287_287_0 to 202410_297_297_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:47.330940 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_615_238/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:07:47.504693 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 305 2024.10.22 01:07:47.978285 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 558128 2024.10.22 01:07:48.131500 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_615_238} MergeTask::MergeProjectionsStage: Merge sorted 37245 rows, containing 13 columns (13 merged, 0 gathered) in 3.291774242 sec., 11314.566936209716 rows/sec., 1.77 MiB/sec. 2024.10.22 01:07:48.447515 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_615_238} MergedBlockOutputStream: filled checksums 202410_1_615_238 (state Temporary) 2024.10.22 01:07:48.475664 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(394ce66a62df8c3de1e5a6f3aca7285d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 394ce66a62df8c3de1e5a6f3aca7285d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:07:48.482966 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 948310, file segment info: File segment: [0, 33554431], key: 394ce66a62df8c3de1e5a6f3aca7285d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:07:48.489002 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:48.588384 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_615_238} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_615_238 to 202410_1_615_238 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:48.601589 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_615_238} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_614_237, 202410_615_615_0] -> 202410_1_615_238 2024.10.22 01:07:48.615328 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 36734 2024.10.22 01:07:48.633368 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:07:48.672231 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.89 GiB, free memory in arenas 9.40 MiB, will set to 1.70 GiB (RSS), difference: -37.93 MiB 2024.10.22 01:07:48.763335 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:48.771837 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.45 GiB. 2024.10.22 01:07:48.937058 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:48.940509 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:49.029745 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_341_341_0 (state Temporary) 2024.10.22 01:07:49.101228 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_353_131/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:07:49.136172 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_341_341_0 to 202410_348_348_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:49.173383 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 558128 2024.10.22 01:07:49.185009 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_346_69/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:07:49.238011 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_607_607_0 (state Temporary) 2024.10.22 01:07:49.288309 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_514_145, 202410_515_515_0] 2024.10.22 01:07:49.295122 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_514_145, 202410_515_515_0] 2024.10.22 01:07:49.306283 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_514_145, 202410_515_515_0] 2024.10.22 01:07:49.368942 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_353_131} MergeTask::MergeProjectionsStage: Merge sorted 29269 rows, containing 13 columns (13 merged, 0 gathered) in 2.919226117 sec., 10026.287388137944 rows/sec., 4.29 MiB/sec. 2024.10.22 01:07:49.402611 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_607_607_0 to 202410_620_620_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:49.444399 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7f81e4ca157e9fdf27a44597f0b86045) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7f81e4ca157e9fdf27a44597f0b86045, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:07:49.452693 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_615_238 to 202410_616_616_0 2024.10.22 01:07:49.486335 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_353_131} MergedBlockOutputStream: filled checksums 202410_1_353_131 (state Temporary) 2024.10.22 01:07:49.487851 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36734 2024.10.22 01:07:49.497972 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1234, file segment info: File segment: [0, 33554431], key: 7f81e4ca157e9fdf27a44597f0b86045, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:07:49.503886 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:49.557667 [ 1693 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rvv/fylhznkzzfkehevenqiqkljlraewa, size 948310 2024.10.22 01:07:49.467722 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 01:07:49.577828 [ 1209 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:49.624891 [ 1209 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 200 ms. 2024.10.22 01:07:49.756484 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_616_239} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_615_238 to 202410_616_616_0 into Compact with storage Full 2024.10.22 01:07:49.878979 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:07:49.889018 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:07:49.892201 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:07:49.934753 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:07:49.935607 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_616_239} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:49.954601 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_616_239} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_615_238, total 37245 rows starting from the beginning of the part 2024.10.22 01:07:49.958488 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:07:49.981033 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_616_239} MergeTreeSequentialSource: Reading 2 marks from part 202410_616_616_0, total 51 rows starting from the beginning of the part 2024.10.22 01:07:50.113306 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:07:50.143882 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:07:50.187207 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:07:50.383326 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:07:50.409381 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:07:50.456205 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:07:50.582404 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.89 GiB, free memory in arenas 9.07 MiB, will set to 1.69 GiB (RSS), difference: 33.29 MiB 2024.10.22 01:07:50.601904 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_353_131} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_353_131 to 202410_1_353_131 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:50.611131 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_353_131} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_352_130, 202410_353_353_0] -> 202410_1_353_131 2024.10.22 01:07:50.682296 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/juu/xasdrhhybkhansjfomnqsjyfsxhdq (LogSeriesLimiter: on interval from 2024-10-22 01:07:33 to 2024-10-22 01:07:50 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:07:50.740451 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 61.38 MiB. 2024.10.22 01:07:51.461991 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_295_89, 202410_296_296_0] 2024.10.22 01:07:51.464103 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_295_89, 202410_296_296_0] 2024.10.22 01:07:51.479287 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_295_89, 202410_296_296_0] 2024.10.22 01:07:51.654742 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/usg/bfhrzuxddlohkdgmsaudowlfmdqzd. 2024.10.22 01:07:51.860034 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_616_239} MergeTask::MergeProjectionsStage: Merge sorted 37296 rows, containing 13 columns (13 merged, 0 gathered) in 2.211767028 sec., 16862.535487620986 rows/sec., 2.64 MiB/sec. 2024.10.22 01:07:52.014360 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_616_239} MergedBlockOutputStream: filled checksums 202410_1_616_239 (state Temporary) 2024.10.22 01:07:52.349483 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_616_239} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_616_239 to 202410_1_616_239 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:52.352350 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_616_239} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_615_238, 202410_616_616_0] -> 202410_1_616_239 2024.10.22 01:07:52.469094 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:52.474427 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:52.489151 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:07:52.586057 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_244_50, 202410_245_245_0, 202410_246_246_0, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0] 2024.10.22 01:07:52.588323 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_244_50, 202410_245_245_0, 202410_246_246_0, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0] 2024.10.22 01:07:52.617284 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_244_50, 202410_245_245_0, 202410_246_246_0, 202410_247_247_0, 202410_248_248_0, 202410_249_249_0] 2024.10.22 01:07:52.715435 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.89 GiB, free memory in arenas 11.04 MiB, will set to 1.69 GiB (RSS), difference: 115.38 MiB 2024.10.22 01:07:53.183699 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} MergeTask::MergeProjectionsStage: Merge sorted 568646 rows, containing 4 columns (4 merged, 0 gathered) in 20.383396905 sec., 27897.509068300213 rows/sec., 409.00 KiB/sec. 2024.10.22 01:07:53.244330 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} MergedBlockOutputStream: filled checksums 202410_1_346_69 (state Temporary) 2024.10.22 01:07:53.284425 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_346_69 to 202410_1_346_69 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:53.287967 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_346_69} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_341_68, 202410_346_346_0] -> 202410_1_346_69 2024.10.22 01:07:53.301116 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 5292 2024.10.22 01:07:53.413356 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:07:53.496213 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 36803 2024.10.22 01:07:53.708063 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:53.711784 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:53.732014 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:53.736688 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:53.752919 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_357_357_0 (state Temporary) 2024.10.22 01:07:53.803646 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_357_357_0 to 202410_370_370_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:53.814178 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_608_608_0 (state Temporary) 2024.10.22 01:07:53.875640 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5292 2024.10.22 01:07:53.951466 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 242 entries to flush up to offset 30463 2024.10.22 01:07:53.962965 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_608_608_0 to 202410_621_621_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:53.974119 [ 1226 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:07:54.007364 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36803 2024.10.22 01:07:54.082295 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_74_15/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:07:54.411866 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.89 GiB, free memory in arenas 11.20 MiB, will set to 1.68 GiB (RSS), difference: 82.04 MiB 2024.10.22 01:07:54.364043 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_616_239 to 202410_617_617_0 2024.10.22 01:07:54.492798 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.32 MiB using storage policy from min volume index 0 2024.10.22 01:07:54.495698 [ 1194 ] {} DiskLocal: Reserved 1.32 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:54.502520 [ 1194 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 420 ms. 2024.10.22 01:07:54.511545 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_617_240} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_616_239 to 202410_617_617_0 into Compact with storage Full 2024.10.22 01:07:54.573107 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:54.580119 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:54.580408 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_617_240} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:54.589269 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_617_240} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_616_239, total 37296 rows starting from the beginning of the part 2024.10.22 01:07:54.615245 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_353_353_0 (state Temporary) 2024.10.22 01:07:54.625185 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_617_240} MergeTreeSequentialSource: Reading 2 marks from part 202410_617_617_0, total 113 rows starting from the beginning of the part 2024.10.22 01:07:54.795982 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_353_353_0 to 202410_359_359_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:54.867063 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 30463 2024.10.22 01:07:55.001119 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6cf3259303c09fc6194c88deb555bde5) : [0, 203]: Setting download as failed: File segment: [0, 203], key: 6cf3259303c09fc6194c88deb555bde5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:07:55.004523 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6cf3259303c09fc6194c88deb555bde5): No space left in cache to reserve 204 bytes, will continue without cache download 2024.10.22 01:07:55.006591 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 306 2024.10.22 01:07:55.034045 [ 1227 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_353_131 to 202410_354_354_0 2024.10.22 01:07:55.042007 [ 1227 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.57 MiB using storage policy from min volume index 0 2024.10.22 01:07:55.049300 [ 1227 ] {} DiskLocal: Reserved 1.57 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:55.054318 [ 1227 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 246 ms. 2024.10.22 01:07:55.185302 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_354_132} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_353_131 to 202410_354_354_0 into Wide with storage Full 2024.10.22 01:07:55.212405 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_354_132} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:55.217468 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_354_132} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_353_131, total 29269 rows starting from the beginning of the part 2024.10.22 01:07:55.493163 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:55.495670 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:55.502460 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_354_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_354_354_0, total 198 rows starting from the beginning of the part 2024.10.22 01:07:55.539206 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(fe59a3b7eb5ee2aa98770e38ea206b23) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fe59a3b7eb5ee2aa98770e38ea206b23, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:07:55.543768 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4449, file segment info: File segment: [0, 33554431], key: fe59a3b7eb5ee2aa98770e38ea206b23, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:07:55.550085 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:55.687282 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_288_288_0 (state Temporary) 2024.10.22 01:07:55.738810 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_515_146, 202410_516_516_0] 2024.10.22 01:07:55.743019 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_515_146, 202410_516_516_0] 2024.10.22 01:07:55.857448 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_515_146, 202410_516_516_0] 2024.10.22 01:07:55.915480 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_288_288_0 to 202410_298_298_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:56.009241 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 306 2024.10.22 01:07:56.179577 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 559664 2024.10.22 01:07:56.180122 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_72_72_0 (state Temporary) 2024.10.22 01:07:56.681029 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_617_240} MergeTask::MergeProjectionsStage: Merge sorted 37409 rows, containing 13 columns (13 merged, 0 gathered) in 2.15104045 sec., 17391.11879555775 rows/sec., 2.72 MiB/sec. 2024.10.22 01:07:56.768936 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0f6cb019b64c043ba501a6a1c8302fd4) : [0, 2172022]: Setting download as failed: File segment: [0, 2172022], key: 0f6cb019b64c043ba501a6a1c8302fd4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:07:56.777379 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0f6cb019b64c043ba501a6a1c8302fd4): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:07:56.825649 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_617_240} MergedBlockOutputStream: filled checksums 202410_1_617_240 (state Temporary) 2024.10.22 01:07:57.025120 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:57.032138 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:57.140815 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0f6cb019b64c043ba501a6a1c8302fd4, offset: 0, size: 0 2024.10.22 01:07:57.144029 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0f6cb019b64c043ba501a6a1c8302fd4 for removal 2024.10.22 01:07:57.147414 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_617_240} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_617_240 to 202410_1_617_240 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:57.148231 [ 1476 ] {} CacheMetadata: Key 0f6cb019b64c043ba501a6a1c8302fd4 is removed from metadata 2024.10.22 01:07:57.152350 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6cf3259303c09fc6194c88deb555bde5, offset: 0, size: 0 2024.10.22 01:07:57.153172 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_617_240} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_616_239, 202410_617_617_0] -> 202410_1_617_240 2024.10.22 01:07:57.155201 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6cf3259303c09fc6194c88deb555bde5 for removal 2024.10.22 01:07:57.161940 [ 1476 ] {} CacheMetadata: Key 6cf3259303c09fc6194c88deb555bde5 is removed from metadata 2024.10.22 01:07:57.201199 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.24 MiB. 2024.10.22 01:07:57.280976 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_342_342_0 (state Temporary) 2024.10.22 01:07:57.341384 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_342_342_0 to 202410_349_349_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:57.398310 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 559664 2024.10.22 01:07:57.734423 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.89 GiB, free memory in arenas 8.00 MiB, will set to 1.70 GiB (RSS), difference: -29.46 MiB 2024.10.22 01:07:57.738773 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(4b4cdfba41a4a50bbdea90dd18f8d404) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4b4cdfba41a4a50bbdea90dd18f8d404, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:07:57.741355 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 145, file segment info: File segment: [0, 33554431], key: 4b4cdfba41a4a50bbdea90dd18f8d404, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:07:57.745172 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:57.858967 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_354_132} MergeTask::MergeProjectionsStage: Merge sorted 29467 rows, containing 13 columns (13 merged, 0 gathered) in 2.679738963 sec., 10996.220306104493 rows/sec., 4.70 MiB/sec. 2024.10.22 01:07:57.913630 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_354_132} MergedBlockOutputStream: filled checksums 202410_1_354_132 (state Temporary) 2024.10.22 01:07:58.013803 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 36879 2024.10.22 01:07:58.294360 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_296_90, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:07:58.297144 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_296_90, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:07:58.306194 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:07:58.310226 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:58.389411 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_609_609_0 (state Temporary) 2024.10.22 01:07:58.333829 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_296_90, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:07:58.576332 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_354_132} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_354_132 to 202410_1_354_132 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:58.583242 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_354_132} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_353_131, 202410_354_354_0] -> 202410_1_354_132 2024.10.22 01:07:58.610002 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(32703065e0ce43aac9ef9de2b4fdceb2) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 32703065e0ce43aac9ef9de2b4fdceb2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:07:58.611906 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(32703065e0ce43aac9ef9de2b4fdceb2): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:07:58.702635 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 61.39 MiB. 2024.10.22 01:07:58.742303 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_609_609_0 to 202410_622_622_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:07:58.768132 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_617_240 to 202410_618_618_0 2024.10.22 01:07:58.776031 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.32 MiB using storage policy from min volume index 0 2024.10.22 01:07:58.779159 [ 1170 ] {} DiskLocal: Reserved 1.32 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:07:58.797000 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36879 2024.10.22 01:07:58.867911 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_618_241} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_617_240 to 202410_618_618_0 into Compact with storage Full 2024.10.22 01:07:58.989679 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_618_241} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:07:58.999087 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_618_241} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_617_240, total 37409 rows starting from the beginning of the part 2024.10.22 01:07:59.021313 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_618_241} MergeTreeSequentialSource: Reading 2 marks from part 202410_618_618_0, total 53 rows starting from the beginning of the part 2024.10.22 01:07:59.028332 [ 1178 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 01:07:59.279602 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_516_147, 202410_517_517_0] 2024.10.22 01:07:59.281147 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_516_147, 202410_517_517_0] 2024.10.22 01:07:59.389517 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_516_147, 202410_517_517_0] 2024.10.22 01:07:59.482286 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(43630316afa988222e7acac6150524b3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 43630316afa988222e7acac6150524b3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:07:59.494540 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1042, file segment info: File segment: [0, 33554431], key: 43630316afa988222e7acac6150524b3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:07:59.507052 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:07:59.891643 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:07:59.893833 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:08:00.108908 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(add06c41f5b71caa818d5ac73ec3a3c4) : [0, 40587]: Setting download as failed: File segment: [0, 40587], key: add06c41f5b71caa818d5ac73ec3a3c4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:00.111332 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(add06c41f5b71caa818d5ac73ec3a3c4): No space left in cache to reserve 40588 bytes, will continue without cache download 2024.10.22 01:08:00.143308 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: add06c41f5b71caa818d5ac73ec3a3c4, offset: 0, size: 0 2024.10.22 01:08:00.147850 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key add06c41f5b71caa818d5ac73ec3a3c4 for removal 2024.10.22 01:08:00.152035 [ 1476 ] {} CacheMetadata: Key add06c41f5b71caa818d5ac73ec3a3c4 is removed from metadata 2024.10.22 01:08:00.158445 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 32703065e0ce43aac9ef9de2b4fdceb2, offset: 0, size: 0 2024.10.22 01:08:00.163415 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 32703065e0ce43aac9ef9de2b4fdceb2 for removal 2024.10.22 01:08:00.167244 [ 1476 ] {} CacheMetadata: Key 32703065e0ce43aac9ef9de2b4fdceb2 is removed from metadata 2024.10.22 01:08:00.337285 [ 1163 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:08:00.369967 [ 1163 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:08:00.671904 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_306_61, 202410_307_307_0, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0] 2024.10.22 01:08:00.677318 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_306_61, 202410_307_307_0, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0] 2024.10.22 01:08:00.715886 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_306_61, 202410_307_307_0, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0] 2024.10.22 01:08:00.904376 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(31eef16797a42d1c72eff929e924d9ba) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 31eef16797a42d1c72eff929e924d9ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:08:00.906788 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 469, file segment info: File segment: [0, 33554431], key: 31eef16797a42d1c72eff929e924d9ba, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:08:00.909271 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:00.945225 [ 1691 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/klq/aooluchlywvibxzlgfnaevfzmckqn, size 1042 2024.10.22 01:08:01.301559 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:08:01.379644 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 5311 2024.10.22 01:08:01.414320 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:08:01.448732 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:08:01.466973 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:08:01.477142 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:01.482307 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:01.484470 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:08:01.510915 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:08:01.536817 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:08:01.539025 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6c36e000d930d4357b73082d67625731) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 6c36e000d930d4357b73082d67625731, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:01.547476 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6c36e000d930d4357b73082d67625731): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:08:01.597974 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yby/omjtswjittdbhjageemascqenxxjz (LogSeriesLimiter: on interval from 2024-10-22 01:07:50 to 2024-10-22 01:08:01 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:08:01.612667 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_358_358_0 (state Temporary) 2024.10.22 01:08:01.784636 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_358_358_0 to 202410_371_371_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:01.803710 [ 1184 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_366_73 to 202410_371_371_0 2024.10.22 01:08:01.808589 [ 1184 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:01.835772 [ 1184 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:01.836593 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_618_241} MergeTask::MergeProjectionsStage: Merge sorted 37462 rows, containing 13 columns (13 merged, 0 gathered) in 3.035173038 sec., 12342.624137398521 rows/sec., 1.93 MiB/sec. 2024.10.22 01:08:01.847898 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5311 2024.10.22 01:08:01.904594 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.89 GiB, free memory in arenas 6.79 MiB, will set to 1.70 GiB (RSS), difference: 30.14 MiB 2024.10.22 01:08:01.987094 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_366_73 to 202410_371_371_0 into Compact with storage Full 2024.10.22 01:08:02.153458 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:02.195792 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_54_11, 202410_55_55_0, 202410_56_56_0, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0] 2024.10.22 01:08:02.196832 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_54_11, 202410_55_55_0, 202410_56_56_0, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0] 2024.10.22 01:08:02.209901 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_618_241} MergedBlockOutputStream: filled checksums 202410_1_618_241 (state Temporary) 2024.10.22 01:08:02.280871 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_366_73, total 5328 rows starting from the beginning of the part 2024.10.22 01:08:02.297688 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_618_241} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_618_241 to 202410_1_618_241 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:02.300496 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_618_241} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_617_240, 202410_618_618_0] -> 202410_1_618_241 2024.10.22 01:08:02.303540 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_367_367_0, total 20 rows starting from the beginning of the part 2024.10.22 01:08:02.318818 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:08:02.319107 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_54_11, 202410_55_55_0, 202410_56_56_0, 202410_57_57_0, 202410_58_58_0, 202410_59_59_0] 2024.10.22 01:08:02.336546 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_368_368_0, total 22 rows starting from the beginning of the part 2024.10.22 01:08:02.403694 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 231 entries to flush up to offset 30694 2024.10.22 01:08:02.483695 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_369_369_0, total 14 rows starting from the beginning of the part 2024.10.22 01:08:02.525734 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_370_370_0, total 14 rows starting from the beginning of the part 2024.10.22 01:08:02.572619 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_371_371_0, total 19 rows starting from the beginning of the part 2024.10.22 01:08:02.600078 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:02.604569 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:02.751913 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_354_354_0 (state Temporary) 2024.10.22 01:08:02.804407 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 36966 2024.10.22 01:08:02.938887 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_354_354_0 to 202410_360_360_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:02.980473 [ 1197 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_354_132 to 202410_355_355_0 2024.10.22 01:08:02.986482 [ 1197 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.58 MiB using storage policy from min volume index 0 2024.10.22 01:08:02.991069 [ 1197 ] {} DiskLocal: Reserved 1.58 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:03.011846 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_355_133} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_354_132 to 202410_355_355_0 into Wide with storage Full 2024.10.22 01:08:03.054066 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_355_133} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:03.063752 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_355_133} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_354_132, total 29467 rows starting from the beginning of the part 2024.10.22 01:08:03.070206 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0eb4fc2c8f0dcece3d4976e8826b7d82) : [0, 2444]: Setting download as failed: File segment: [0, 2444], key: 0eb4fc2c8f0dcece3d4976e8826b7d82, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:03.072671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0eb4fc2c8f0dcece3d4976e8826b7d82): No space left in cache to reserve 2445 bytes, will continue without cache download 2024.10.22 01:08:03.084274 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 30694 2024.10.22 01:08:03.107794 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0eb4fc2c8f0dcece3d4976e8826b7d82, offset: 0, size: 0 2024.10.22 01:08:03.110962 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0eb4fc2c8f0dcece3d4976e8826b7d82 for removal 2024.10.22 01:08:03.116728 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6c36e000d930d4357b73082d67625731, offset: 0, size: 0 2024.10.22 01:08:03.119224 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6c36e000d930d4357b73082d67625731 for removal 2024.10.22 01:08:03.124984 [ 1476 ] {} CacheMetadata: Key 0eb4fc2c8f0dcece3d4976e8826b7d82 is removed from metadata 2024.10.22 01:08:03.134722 [ 1476 ] {} CacheMetadata: Key 6c36e000d930d4357b73082d67625731 is removed from metadata 2024.10.22 01:08:03.191547 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} MergeTask::MergeProjectionsStage: Merge sorted 5417 rows, containing 12 columns (12 merged, 0 gathered) in 1.324386089 sec., 4090.196993906963 rows/sec., 645.15 KiB/sec. 2024.10.22 01:08:03.204959 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:03.207797 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:03.262486 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} MergedBlockOutputStream: filled checksums 202410_1_371_74 (state Temporary) 2024.10.22 01:08:03.330382 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_355_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_355_355_0, total 210 rows starting from the beginning of the part 2024.10.22 01:08:03.387436 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_610_610_0 (state Temporary) 2024.10.22 01:08:03.403931 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_371_74 to 202410_1_371_74 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:03.461381 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_610_610_0 to 202410_623_623_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:03.489030 [ 1162 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_618_241 to 202410_619_619_0 2024.10.22 01:08:03.496042 [ 1162 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.32 MiB using storage policy from min volume index 0 2024.10.22 01:08:03.498289 [ 1162 ] {} DiskLocal: Reserved 1.32 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:03.507165 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_371_74} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_366_73, 202410_371_371_0] -> 202410_1_371_74 2024.10.22 01:08:03.511527 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 307 2024.10.22 01:08:03.626764 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_619_242} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_618_241 to 202410_619_619_0 into Compact with storage Full 2024.10.22 01:08:03.630607 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 36966 2024.10.22 01:08:03.754326 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.40 MiB. 2024.10.22 01:08:03.758214 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_619_242} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:03.768764 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_619_242} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_618_241, total 37462 rows starting from the beginning of the part 2024.10.22 01:08:03.782144 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bbb4e1ea47ac4839d414bac09889da41) : [0, 162]: Setting download as failed: File segment: [0, 162], key: bbb4e1ea47ac4839d414bac09889da41, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:03.784276 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bbb4e1ea47ac4839d414bac09889da41): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:08:03.794460 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_619_242} MergeTreeSequentialSource: Reading 2 marks from part 202410_619_619_0, total 43 rows starting from the beginning of the part 2024.10.22 01:08:03.865113 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:03.889513 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:03.951601 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_289_289_0 (state Temporary) 2024.10.22 01:08:04.114409 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_72_72_0 to 202410_75_75_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:04.265489 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_289_289_0 to 202410_299_299_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:04.404267 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 768 entries to flush up to offset 560432 2024.10.22 01:08:04.419925 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 307 2024.10.22 01:08:04.512360 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.89 GiB, free memory in arenas 5.82 MiB, will set to 1.69 GiB (RSS), difference: -61.85 MiB 2024.10.22 01:08:04.924326 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:04.930950 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:04.930579 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0dff8dbc2dbcb2b138abb6aa1ffe3832) : [0, 1041557]: Setting download as failed: File segment: [0, 1041557], key: 0dff8dbc2dbcb2b138abb6aa1ffe3832, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:04.912163 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_517_148, 202410_518_518_0] 2024.10.22 01:08:04.938663 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0dff8dbc2dbcb2b138abb6aa1ffe3832): No space left in cache to reserve 1041558 bytes, will continue without cache download 2024.10.22 01:08:04.946754 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_517_148, 202410_518_518_0] 2024.10.22 01:08:04.964260 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_517_148, 202410_518_518_0] 2024.10.22 01:08:05.080058 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_355_133} MergeTask::MergeProjectionsStage: Merge sorted 29677 rows, containing 13 columns (13 merged, 0 gathered) in 2.078462739 sec., 14278.341123535532 rows/sec., 6.11 MiB/sec. 2024.10.22 01:08:05.097823 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0dff8dbc2dbcb2b138abb6aa1ffe3832, offset: 0, size: 0 2024.10.22 01:08:05.099328 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0dff8dbc2dbcb2b138abb6aa1ffe3832 for removal 2024.10.22 01:08:05.102576 [ 1476 ] {} CacheMetadata: Key 0dff8dbc2dbcb2b138abb6aa1ffe3832 is removed from metadata 2024.10.22 01:08:05.104913 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bbb4e1ea47ac4839d414bac09889da41, offset: 0, size: 0 2024.10.22 01:08:05.107394 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bbb4e1ea47ac4839d414bac09889da41 for removal 2024.10.22 01:08:05.110995 [ 1476 ] {} CacheMetadata: Key bbb4e1ea47ac4839d414bac09889da41 is removed from metadata 2024.10.22 01:08:05.236578 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_343_343_0 (state Temporary) 2024.10.22 01:08:05.309069 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_343_343_0 to 202410_350_350_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:05.390420 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_355_133} MergedBlockOutputStream: filled checksums 202410_1_355_133 (state Temporary) 2024.10.22 01:08:05.451407 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 560432 2024.10.22 01:08:05.980828 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(81c55e20d0e78540b9eb51b650b7f546) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 81c55e20d0e78540b9eb51b650b7f546, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:05.984146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(81c55e20d0e78540b9eb51b650b7f546): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:08:06.003940 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_619_242} MergeTask::MergeProjectionsStage: Merge sorted 37505 rows, containing 13 columns (13 merged, 0 gathered) in 2.385595705 sec., 15721.440108813407 rows/sec., 2.46 MiB/sec. 2024.10.22 01:08:06.076547 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_619_242} MergedBlockOutputStream: filled checksums 202410_1_619_242 (state Temporary) 2024.10.22 01:08:06.089021 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_355_133} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_355_133 to 202410_1_355_133 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:06.090913 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_355_133} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_354_132, 202410_355_355_0] -> 202410_1_355_133 2024.10.22 01:08:06.201508 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 61.39 MiB. 2024.10.22 01:08:06.258812 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_619_242} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_619_242 to 202410_1_619_242 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:06.263260 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_619_242} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_618_241, 202410_619_619_0] -> 202410_1_619_242 2024.10.22 01:08:06.279704 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.89 GiB, free memory in arenas 5.52 MiB, will set to 1.69 GiB (RSS), difference: 61.98 MiB 2024.10.22 01:08:06.285769 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:08:06.892590 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c0851e2d1329fde7fc5160aa9aac0129) : [0, 41657]: Setting download as failed: File segment: [0, 41657], key: c0851e2d1329fde7fc5160aa9aac0129, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:06.893804 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c0851e2d1329fde7fc5160aa9aac0129): No space left in cache to reserve 41658 bytes, will continue without cache download 2024.10.22 01:08:06.910030 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c0851e2d1329fde7fc5160aa9aac0129, offset: 0, size: 0 2024.10.22 01:08:06.912735 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c0851e2d1329fde7fc5160aa9aac0129 for removal 2024.10.22 01:08:06.917072 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 81c55e20d0e78540b9eb51b650b7f546, offset: 0, size: 0 2024.10.22 01:08:06.924235 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 81c55e20d0e78540b9eb51b650b7f546 for removal 2024.10.22 01:08:06.929566 [ 1476 ] {} CacheMetadata: Key c0851e2d1329fde7fc5160aa9aac0129 is removed from metadata 2024.10.22 01:08:06.940319 [ 1476 ] {} CacheMetadata: Key 81c55e20d0e78540b9eb51b650b7f546 is removed from metadata 2024.10.22 01:08:06.981173 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2874 2024.10.22 01:08:07.651183 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 37050 2024.10.22 01:08:07.736850 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6406d08b04acd10ac05a155c3a0b7ffd) : [0, 144]: Setting download as failed: File segment: [0, 144], key: 6406d08b04acd10ac05a155c3a0b7ffd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:07.740317 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6406d08b04acd10ac05a155c3a0b7ffd): No space left in cache to reserve 145 bytes, will continue without cache download 2024.10.22 01:08:08.226294 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:08.232061 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:08.339214 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.89 GiB, free memory in arenas 7.11 MiB, will set to 1.68 GiB (RSS), difference: 2.80 MiB 2024.10.22 01:08:08.492780 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2c6e198499bae61302b3fed29183b469) : [0, 30922]: Setting download as failed: File segment: [0, 30922], key: 2c6e198499bae61302b3fed29183b469, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:08.493696 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2c6e198499bae61302b3fed29183b469): No space left in cache to reserve 30923 bytes, will continue without cache download 2024.10.22 01:08:08.500928 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_611_611_0 (state Temporary) 2024.10.22 01:08:08.503958 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2c6e198499bae61302b3fed29183b469, offset: 0, size: 0 2024.10.22 01:08:08.506318 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2c6e198499bae61302b3fed29183b469 for removal 2024.10.22 01:08:08.508309 [ 1476 ] {} CacheMetadata: Key 2c6e198499bae61302b3fed29183b469 is removed from metadata 2024.10.22 01:08:08.510565 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6406d08b04acd10ac05a155c3a0b7ffd, offset: 0, size: 0 2024.10.22 01:08:08.512145 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6406d08b04acd10ac05a155c3a0b7ffd for removal 2024.10.22 01:08:08.519662 [ 1476 ] {} CacheMetadata: Key 6406d08b04acd10ac05a155c3a0b7ffd is removed from metadata 2024.10.22 01:08:08.535039 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_611_611_0 to 202410_624_624_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:08.548635 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_74_15/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:08:08.562983 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_619_242 to 202410_620_620_0 2024.10.22 01:08:08.569230 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.32 MiB using storage policy from min volume index 0 2024.10.22 01:08:08.575381 [ 1233 ] {} DiskLocal: Reserved 1.32 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:08.593646 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_620_243} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_619_242 to 202410_620_620_0 into Compact with storage Full 2024.10.22 01:08:08.631684 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37050 2024.10.22 01:08:08.732410 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_620_243} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:08.741215 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_620_243} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_619_242, total 37505 rows starting from the beginning of the part 2024.10.22 01:08:08.762922 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_620_243} MergeTreeSequentialSource: Reading 2 marks from part 202410_620_620_0, total 60 rows starting from the beginning of the part 2024.10.22 01:08:09.297729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a8e3c1838209505a810db729a4c68d79) : [0, 150]: Setting download as failed: File segment: [0, 150], key: a8e3c1838209505a810db729a4c68d79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:09.299044 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a8e3c1838209505a810db729a4c68d79): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:08:09.352852 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 5334 2024.10.22 01:08:09.702301 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:09.705731 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:09.754536 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_359_359_0 (state Temporary) 2024.10.22 01:08:09.891080 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:08:09.894170 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:08:09.916255 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_359_359_0 to 202410_372_372_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:10.079142 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5334 2024.10.22 01:08:10.491455 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ab9c6ac68f478cc36fe197ff9748f847) : [0, 248557]: Setting download as failed: File segment: [0, 248557], key: ab9c6ac68f478cc36fe197ff9748f847, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:10.492815 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ab9c6ac68f478cc36fe197ff9748f847): No space left in cache to reserve 248558 bytes, will continue without cache download 2024.10.22 01:08:10.512604 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ab9c6ac68f478cc36fe197ff9748f847, offset: 0, size: 0 2024.10.22 01:08:10.515520 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ab9c6ac68f478cc36fe197ff9748f847 for removal 2024.10.22 01:08:10.519860 [ 1476 ] {} CacheMetadata: Key ab9c6ac68f478cc36fe197ff9748f847 is removed from metadata 2024.10.22 01:08:10.521771 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a8e3c1838209505a810db729a4c68d79, offset: 0, size: 0 2024.10.22 01:08:10.526623 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a8e3c1838209505a810db729a4c68d79 for removal 2024.10.22 01:08:10.631523 [ 1476 ] {} CacheMetadata: Key a8e3c1838209505a810db729a4c68d79 is removed from metadata 2024.10.22 01:08:10.736643 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 224 entries to flush up to offset 30918 2024.10.22 01:08:10.818339 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_620_243} MergeTask::MergeProjectionsStage: Merge sorted 37565 rows, containing 13 columns (13 merged, 0 gathered) in 2.230279772 sec., 16843.178363364543 rows/sec., 2.64 MiB/sec. 2024.10.22 01:08:11.130609 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_620_243} MergedBlockOutputStream: filled checksums 202410_1_620_243 (state Temporary) 2024.10.22 01:08:11.257355 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:11.359089 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_518_149, 202410_519_519_0] 2024.10.22 01:08:11.362352 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_518_149, 202410_519_519_0] 2024.10.22 01:08:11.370889 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:11.380218 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_518_149, 202410_519_519_0] 2024.10.22 01:08:11.391297 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_620_243} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_620_243 to 202410_1_620_243 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:11.394426 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_620_243} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_619_242, 202410_620_620_0] -> 202410_1_620_243 2024.10.22 01:08:11.420416 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_355_355_0 (state Temporary) 2024.10.22 01:08:11.496922 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_355_355_0 to 202410_361_361_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:11.508983 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:08:11.511803 [ 1236 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_355_133 to 202410_356_356_0 2024.10.22 01:08:11.516016 [ 1236 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.59 MiB using storage policy from min volume index 0 2024.10.22 01:08:11.518664 [ 1236 ] {} DiskLocal: Reserved 1.59 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:11.530026 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_356_134} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_355_133 to 202410_356_356_0 into Wide with storage Full 2024.10.22 01:08:11.567926 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_356_134} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:11.618251 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 30918 2024.10.22 01:08:11.677235 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_356_134} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_355_133, total 29677 rows starting from the beginning of the part 2024.10.22 01:08:11.617774 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a62821c1061bd61ca7f70ff5941ed1a2) : [0, 149]: Setting download as failed: File segment: [0, 149], key: a62821c1061bd61ca7f70ff5941ed1a2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:11.719517 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a62821c1061bd61ca7f70ff5941ed1a2): No space left in cache to reserve 150 bytes, will continue without cache download 2024.10.22 01:08:11.877496 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_356_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_356_356_0, total 206 rows starting from the beginning of the part 2024.10.22 01:08:11.923513 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 308 2024.10.22 01:08:12.313761 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:12.318216 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:12.357737 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:08:12.455047 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 561968 2024.10.22 01:08:12.495614 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.89 GiB, free memory in arenas 7.37 MiB, will set to 1.68 GiB (RSS), difference: -44.24 MiB 2024.10.22 01:08:12.499584 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(cfe62333ff881be926efc365bc7e15a4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cfe62333ff881be926efc365bc7e15a4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:08:12.503265 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_290_290_0 (state Temporary) 2024.10.22 01:08:12.504109 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 7142, file segment info: File segment: [0, 33554431], key: cfe62333ff881be926efc365bc7e15a4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:08:12.507137 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:12.698954 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 37126 2024.10.22 01:08:12.883680 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_290_290_0 to 202410_300_300_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:12.990566 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 308 2024.10.22 01:08:13.061141 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:13.068243 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:13.118787 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_612_612_0 (state Temporary) 2024.10.22 01:08:13.160254 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:13.183947 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_612_612_0 to 202410_625_625_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:13.210589 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:13.215098 [ 1151 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_620_243 to 202410_621_621_0 2024.10.22 01:08:13.217771 [ 1151 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 01:08:13.221674 [ 1151 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:13.222142 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37126 2024.10.22 01:08:13.311931 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3e2707c2422d36af439ac2e66dee9ac9) : [0, 161301]: Setting download as failed: File segment: [0, 161301], key: 3e2707c2422d36af439ac2e66dee9ac9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:13.313785 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3e2707c2422d36af439ac2e66dee9ac9): No space left in cache to reserve 161302 bytes, will continue without cache download 2024.10.22 01:08:13.379202 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_621_244} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_620_243 to 202410_621_621_0 into Compact with storage Full 2024.10.22 01:08:13.395644 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_621_244} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:13.403438 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_621_244} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_620_243, total 37565 rows starting from the beginning of the part 2024.10.22 01:08:13.434836 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_344_344_0 (state Temporary) 2024.10.22 01:08:13.435814 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_621_244} MergeTreeSequentialSource: Reading 2 marks from part 202410_621_621_0, total 69 rows starting from the beginning of the part 2024.10.22 01:08:13.448445 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3e2707c2422d36af439ac2e66dee9ac9, offset: 0, size: 0 2024.10.22 01:08:13.450955 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3e2707c2422d36af439ac2e66dee9ac9 for removal 2024.10.22 01:08:13.453091 [ 1476 ] {} CacheMetadata: Key 3e2707c2422d36af439ac2e66dee9ac9 is removed from metadata 2024.10.22 01:08:13.454198 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a62821c1061bd61ca7f70ff5941ed1a2, offset: 0, size: 0 2024.10.22 01:08:13.455547 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a62821c1061bd61ca7f70ff5941ed1a2 for removal 2024.10.22 01:08:13.500587 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_344_344_0 to 202410_351_351_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:13.551899 [ 1227 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_346_69 to 202410_351_351_0 2024.10.22 01:08:13.558389 [ 1476 ] {} CacheMetadata: Key a62821c1061bd61ca7f70ff5941ed1a2 is removed from metadata 2024.10.22 01:08:13.569164 [ 1227 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:13.575638 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:13.598024 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_346_69 to 202410_351_351_0 into Compact with storage Full 2024.10.22 01:08:13.626160 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:13.637332 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} MergeTreeSequentialSource: Reading 71 marks from part 202410_1_346_69, total 568646 rows starting from the beginning of the part 2024.10.22 01:08:13.644961 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_347_347_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:08:13.663687 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_348_348_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:08:13.728139 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 561968 2024.10.22 01:08:13.785129 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_349_349_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:08:13.813615 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_350_350_0, total 768 rows starting from the beginning of the part 2024.10.22 01:08:13.837327 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_351_351_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:08:13.917374 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ac079411ace282d173204d4d5fd67062) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ac079411ace282d173204d4d5fd67062, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:08:14.022371 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 880, file segment info: File segment: [0, 33554431], key: ac079411ace282d173204d4d5fd67062, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:08:14.025755 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:14.029607 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_356_134} MergeTask::MergeProjectionsStage: Merge sorted 29883 rows, containing 13 columns (13 merged, 0 gathered) in 2.506409192 sec., 11922.634219257205 rows/sec., 5.10 MiB/sec. 2024.10.22 01:08:14.191727 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_356_134} MergedBlockOutputStream: filled checksums 202410_1_356_134 (state Temporary) 2024.10.22 01:08:14.370138 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_621_244/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:08:14.496531 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 2913 2024.10.22 01:08:14.567730 [ 1268 ] {} DatabaseCatalog: Execution took 201 ms. 2024.10.22 01:08:14.580000 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_519_150, 202410_520_520_0] 2024.10.22 01:08:14.584459 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_519_150, 202410_520_520_0] 2024.10.22 01:08:14.593369 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_519_150, 202410_520_520_0] 2024.10.22 01:08:14.622808 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(53c560048c3e550f38de0806984db006) : [0, 175]: Setting download as failed: File segment: [0, 175], key: 53c560048c3e550f38de0806984db006, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:14.624413 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(53c560048c3e550f38de0806984db006): No space left in cache to reserve 176 bytes, will continue without cache download 2024.10.22 01:08:14.688167 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_356_134} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_356_134 to 202410_1_356_134 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:14.691553 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_356_134} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_355_133, 202410_356_356_0] -> 202410_1_356_134 2024.10.22 01:08:14.716429 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 61.40 MiB. 2024.10.22 01:08:15.389803 [ 1254 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:08:15.414856 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.89 GiB, free memory in arenas 6.05 MiB, will set to 1.69 GiB (RSS), difference: 37.61 MiB 2024.10.22 01:08:15.443431 [ 1254 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:08:15.463046 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a247986bb9524d2f7ad5b04abe81692f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a247986bb9524d2f7ad5b04abe81692f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:08:15.471413 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 42394, file segment info: File segment: [0, 33554431], key: a247986bb9524d2f7ad5b04abe81692f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:08:15.478153 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:15.592397 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_621_244} MergeTask::MergeProjectionsStage: Merge sorted 37634 rows, containing 13 columns (13 merged, 0 gathered) in 2.226364259 sec., 16903.79274095237 rows/sec., 2.65 MiB/sec. 2024.10.22 01:08:15.677658 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_621_244} MergedBlockOutputStream: filled checksums 202410_1_621_244 (state Temporary) 2024.10.22 01:08:15.965718 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_621_244} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_621_244 to 202410_1_621_244 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:15.974435 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_621_244} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_620_243, 202410_621_621_0] -> 202410_1_621_244 2024.10.22 01:08:16.021109 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:08:16.673131 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0783728568f54d98ce8bbd5af7656b25) : [0, 358461]: Setting download as failed: File segment: [0, 358461], key: 0783728568f54d98ce8bbd5af7656b25, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:16.677205 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0783728568f54d98ce8bbd5af7656b25): No space left in cache to reserve 358462 bytes, will continue without cache download 2024.10.22 01:08:16.796867 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0783728568f54d98ce8bbd5af7656b25, offset: 0, size: 0 2024.10.22 01:08:16.797814 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0783728568f54d98ce8bbd5af7656b25 for removal 2024.10.22 01:08:16.800539 [ 1476 ] {} CacheMetadata: Key 0783728568f54d98ce8bbd5af7656b25 is removed from metadata 2024.10.22 01:08:16.801521 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 53c560048c3e550f38de0806984db006, offset: 0, size: 0 2024.10.22 01:08:16.802557 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 53c560048c3e550f38de0806984db006 for removal 2024.10.22 01:08:16.804652 [ 1476 ] {} CacheMetadata: Key 53c560048c3e550f38de0806984db006 is removed from metadata 2024.10.22 01:08:17.331956 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(8de966fa81db70114fa077f83a3cab75) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8de966fa81db70114fa077f83a3cab75, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:08:17.347210 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 37200 2024.10.22 01:08:17.349116 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 899, file segment info: File segment: [0, 33554431], key: 8de966fa81db70114fa077f83a3cab75, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:08:17.353834 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:17.505985 [ 1720 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/juu/xasdrhhybkhansjfomnqsjyfsxhdq, size 42394 2024.10.22 01:08:17.582571 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 5348 2024.10.22 01:08:17.753447 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:08:17.775414 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:17.779520 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:17.792583 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:08:17.831314 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:08:17.838380 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_360_360_0 (state Temporary) 2024.10.22 01:08:17.870229 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:08:17.898215 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_360_360_0 to 202410_373_373_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:17.920196 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:08:17.953394 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5348 2024.10.22 01:08:17.960482 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:08:17.996912 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.07451826 sec. 2024.10.22 01:08:18.004313 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_286_57, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:08:18.015292 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:18.016040 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_286_57, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:08:18.017430 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:18.086719 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:08:18.124614 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:08:18.152637 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:08:18.197603 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(87959b6a65c0fffc28c6b38382fd4463) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 87959b6a65c0fffc28c6b38382fd4463, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:08:18.199819 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 41661, file segment info: File segment: [0, 33554431], key: 87959b6a65c0fffc28c6b38382fd4463, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:08:18.201066 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:18.243354 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_286_57, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:08:18.273244 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_613_613_0 (state Temporary) 2024.10.22 01:08:18.339905 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jkl/lzodacwtthrgoriormwqyhrpbattv (LogSeriesLimiter: on interval from 2024-10-22 01:08:01 to 2024-10-22 01:08:18 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:08:18.353379 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_613_613_0 to 202410_626_626_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:18.389814 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_621_244 to 202410_622_622_0 2024.10.22 01:08:18.399426 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_351_70/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:08:18.404237 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37200 2024.10.22 01:08:18.434248 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 01:08:18.439057 [ 1163 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:18.476333 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_622_245} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_621_244 to 202410_622_622_0 into Compact with storage Full 2024.10.22 01:08:18.496405 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_622_245} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:18.500563 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_622_245} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_621_244, total 37634 rows starting from the beginning of the part 2024.10.22 01:08:18.515143 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_622_245} MergeTreeSequentialSource: Reading 2 marks from part 202410_622_622_0, total 76 rows starting from the beginning of the part 2024.10.22 01:08:18.699697 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5b62d907fe1ba7ff6a72907a93ced560) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 5b62d907fe1ba7ff6a72907a93ced560, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:18.700807 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5b62d907fe1ba7ff6a72907a93ced560): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:08:19.254798 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 226 entries to flush up to offset 31144 2024.10.22 01:08:19.527866 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.89 GiB, free memory in arenas 12.48 MiB, will set to 1.69 GiB (RSS), difference: -1.75 MiB 2024.10.22 01:08:19.611369 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(430b7ed54f3b37946c41263cbae01cbd) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 430b7ed54f3b37946c41263cbae01cbd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:08:19.633088 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 726, file segment info: File segment: [0, 33554431], key: 430b7ed54f3b37946c41263cbae01cbd, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:08:19.648398 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:19.737224 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:19.737414 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:19.823052 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_356_356_0 (state Temporary) 2024.10.22 01:08:19.874842 [ 1741 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yby/omjtswjittdbhjageemascqenxxjz, size 41661 2024.10.22 01:08:19.890268 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:08:19.891597 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:08:19.897458 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_356_356_0 to 202410_362_362_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:19.939990 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 31144 2024.10.22 01:08:19.950598 [ 1182 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_356_134 to 202410_357_357_0 2024.10.22 01:08:19.958304 [ 1182 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.60 MiB using storage policy from min volume index 0 2024.10.22 01:08:19.962792 [ 1182 ] {} DiskLocal: Reserved 1.60 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:20.091742 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_357_135} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_356_134 to 202410_357_357_0 into Wide with storage Full 2024.10.22 01:08:20.102378 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_520_151, 202410_521_521_0] 2024.10.22 01:08:20.104304 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_520_151, 202410_521_521_0] 2024.10.22 01:08:20.120014 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_520_151, 202410_521_521_0] 2024.10.22 01:08:20.131154 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_357_135} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:20.152836 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_357_135} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_356_134, total 29883 rows starting from the beginning of the part 2024.10.22 01:08:20.234792 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_74_15/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:08:20.381743 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:08:20.467994 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_357_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_357_357_0, total 249 rows starting from the beginning of the part 2024.10.22 01:08:20.491951 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 309 2024.10.22 01:08:20.522104 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:08:20.542822 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:08:20.576705 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:08:20.612841 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:08:20.616997 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_622_245} MergeTask::MergeProjectionsStage: Merge sorted 37710 rows, containing 13 columns (13 merged, 0 gathered) in 2.151150406 sec., 17530.154978851813 rows/sec., 2.75 MiB/sec. 2024.10.22 01:08:20.672572 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:08:20.717575 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1c1d764a1bfb306dd0b24876c8003d03) : [0, 65155]: Setting download as failed: File segment: [0, 65155], key: 1c1d764a1bfb306dd0b24876c8003d03, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:20.720269 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1c1d764a1bfb306dd0b24876c8003d03): No space left in cache to reserve 65156 bytes, will continue without cache download 2024.10.22 01:08:20.728937 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:08:20.735260 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 768 entries to flush up to offset 562736 2024.10.22 01:08:20.793441 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_622_245} MergedBlockOutputStream: filled checksums 202410_1_622_245 (state Temporary) 2024.10.22 01:08:20.753742 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1c1d764a1bfb306dd0b24876c8003d03, offset: 0, size: 0 2024.10.22 01:08:20.912880 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1c1d764a1bfb306dd0b24876c8003d03 for removal 2024.10.22 01:08:20.915998 [ 1476 ] {} CacheMetadata: Key 1c1d764a1bfb306dd0b24876c8003d03 is removed from metadata 2024.10.22 01:08:20.916501 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5b62d907fe1ba7ff6a72907a93ced560, offset: 0, size: 0 2024.10.22 01:08:20.919982 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5b62d907fe1ba7ff6a72907a93ced560 for removal 2024.10.22 01:08:21.011005 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_622_245} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_622_245 to 202410_1_622_245 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:21.014203 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_622_245} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_621_244, 202410_622_622_0] -> 202410_1_622_245 2024.10.22 01:08:21.032778 [ 1476 ] {} CacheMetadata: Key 5b62d907fe1ba7ff6a72907a93ced560 is removed from metadata 2024.10.22 01:08:21.058162 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:08:21.173319 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:21.180030 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:21.351107 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:21.366666 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:21.385102 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_345_345_0 (state Temporary) 2024.10.22 01:08:21.458583 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_345_345_0 to 202410_352_352_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:21.503994 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 562736 2024.10.22 01:08:21.637017 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_291_291_0 (state Temporary) 2024.10.22 01:08:21.724075 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_291_291_0 to 202410_301_301_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:21.765326 [ 1162 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_296_61 to 202410_301_301_0 2024.10.22 01:08:21.783934 [ 1162 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:21.786435 [ 1162 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:21.795492 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_296_61 to 202410_301_301_0 into Compact with storage Full 2024.10.22 01:08:21.819680 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b87693f9bf67cf63527aadedc73e4577) : [0, 152]: Setting download as failed: File segment: [0, 152], key: b87693f9bf67cf63527aadedc73e4577, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:21.824984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b87693f9bf67cf63527aadedc73e4577): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:08:21.995132 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:22.002787 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_296_61, total 340 rows starting from the beginning of the part 2024.10.22 01:08:22.005578 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 309 2024.10.22 01:08:22.047336 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_297_297_0, total 1 rows starting from the beginning of the part 2024.10.22 01:08:22.099535 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_298_298_0, total 1 rows starting from the beginning of the part 2024.10.22 01:08:22.152179 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_299_299_0, total 1 rows starting from the beginning of the part 2024.10.22 01:08:22.307522 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_357_135/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:08:22.412725 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 111 entries to flush up to offset 37311 2024.10.22 01:08:22.447733 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_300_300_0, total 1 rows starting from the beginning of the part 2024.10.22 01:08:22.505146 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.89 GiB, free memory in arenas 13.34 MiB, will set to 1.71 GiB (RSS), difference: -21.38 MiB 2024.10.22 01:08:22.510204 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_301_301_0, total 1 rows starting from the beginning of the part 2024.10.22 01:08:22.677346 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_357_135} MergeTask::MergeProjectionsStage: Merge sorted 30132 rows, containing 13 columns (13 merged, 0 gathered) in 2.595431918 sec., 11609.628359359645 rows/sec., 4.97 MiB/sec. 2024.10.22 01:08:22.741661 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_357_135} MergedBlockOutputStream: filled checksums 202410_1_357_135 (state Temporary) 2024.10.22 01:08:23.070896 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:23.079689 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:23.258007 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_614_614_0 (state Temporary) 2024.10.22 01:08:23.266974 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(04483ee7096ea8a386f292545f3b1a6b) : [0, 68720]: Setting download as failed: File segment: [0, 68720], key: 04483ee7096ea8a386f292545f3b1a6b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:23.269138 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(04483ee7096ea8a386f292545f3b1a6b): No space left in cache to reserve 68721 bytes, will continue without cache download 2024.10.22 01:08:23.291710 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 04483ee7096ea8a386f292545f3b1a6b, offset: 0, size: 0 2024.10.22 01:08:23.299413 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 04483ee7096ea8a386f292545f3b1a6b for removal 2024.10.22 01:08:23.302822 [ 1476 ] {} CacheMetadata: Key 04483ee7096ea8a386f292545f3b1a6b is removed from metadata 2024.10.22 01:08:23.303717 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b87693f9bf67cf63527aadedc73e4577, offset: 0, size: 0 2024.10.22 01:08:23.305211 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b87693f9bf67cf63527aadedc73e4577 for removal 2024.10.22 01:08:23.306923 [ 1476 ] {} CacheMetadata: Key b87693f9bf67cf63527aadedc73e4577 is removed from metadata 2024.10.22 01:08:23.355556 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_357_135} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_357_135 to 202410_1_357_135 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:23.357624 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_357_135} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_356_134, 202410_357_357_0] -> 202410_1_357_135 2024.10.22 01:08:23.388230 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 61.49 MiB. 2024.10.22 01:08:23.585432 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_521_152, 202410_522_522_0] 2024.10.22 01:08:23.586145 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_614_614_0 to 202410_627_627_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:23.588330 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_521_152, 202410_522_522_0] 2024.10.22 01:08:23.606494 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_521_152, 202410_522_522_0] 2024.10.22 01:08:23.613228 [ 1211 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_622_245 to 202410_623_623_0 2024.10.22 01:08:23.619007 [ 1211 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 01:08:23.621703 [ 1211 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:23.627865 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37311 2024.10.22 01:08:23.650338 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_623_246} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_622_245 to 202410_623_623_0 into Compact with storage Full 2024.10.22 01:08:23.737708 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_623_246} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:23.756891 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_623_246} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_622_245, total 37710 rows starting from the beginning of the part 2024.10.22 01:08:23.809556 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_623_246} MergeTreeSequentialSource: Reading 2 marks from part 202410_623_623_0, total 87 rows starting from the beginning of the part 2024.10.22 01:08:24.277801 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3fbeb5eab2d4998517921ec92293e38e) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 3fbeb5eab2d4998517921ec92293e38e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:24.279568 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3fbeb5eab2d4998517921ec92293e38e): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:08:24.437729 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.89 GiB, free memory in arenas 13.67 MiB, will set to 1.69 GiB (RSS), difference: 11.67 MiB 2024.10.22 01:08:24.713473 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} MergeTask::MergeProjectionsStage: Merge sorted 345 rows, containing 32 columns (32 merged, 0 gathered) in 2.922569145 sec., 118.04682212232142 rows/sec., 12.04 KiB/sec. 2024.10.22 01:08:24.792410 [ 1206 ] {} DatabaseCatalog: Execution took 223 ms. 2024.10.22 01:08:24.856607 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} MergedBlockOutputStream: filled checksums 202410_1_301_62 (state Temporary) 2024.10.22 01:08:25.002558 [ 1205 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 01:08:25.015541 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_623_246/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:08:25.287102 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ab64c9f6e387e52b8bb2530d0c9d1ae4) : [0, 23814]: Setting download as failed: File segment: [0, 23814], key: ab64c9f6e387e52b8bb2530d0c9d1ae4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:25.288571 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ab64c9f6e387e52b8bb2530d0c9d1ae4): No space left in cache to reserve 23815 bytes, will continue without cache download 2024.10.22 01:08:25.302961 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ab64c9f6e387e52b8bb2530d0c9d1ae4, offset: 0, size: 0 2024.10.22 01:08:25.304314 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ab64c9f6e387e52b8bb2530d0c9d1ae4 for removal 2024.10.22 01:08:25.306142 [ 1476 ] {} CacheMetadata: Key ab64c9f6e387e52b8bb2530d0c9d1ae4 is removed from metadata 2024.10.22 01:08:25.316729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3fbeb5eab2d4998517921ec92293e38e, offset: 0, size: 0 2024.10.22 01:08:25.318664 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3fbeb5eab2d4998517921ec92293e38e for removal 2024.10.22 01:08:25.321164 [ 1476 ] {} CacheMetadata: Key 3fbeb5eab2d4998517921ec92293e38e is removed from metadata 2024.10.22 01:08:25.389486 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_301_62 to 202410_1_301_62 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:25.460430 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 5360 2024.10.22 01:08:25.495294 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_301_62} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_296_61, 202410_301_301_0] -> 202410_1_301_62 2024.10.22 01:08:25.549551 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:08:25.598470 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:25.602704 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:25.657742 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_361_361_0 (state Temporary) 2024.10.22 01:08:25.880951 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_361_361_0 to 202410_374_374_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:25.885629 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_623_246} MergeTask::MergeProjectionsStage: Merge sorted 37797 rows, containing 13 columns (13 merged, 0 gathered) in 2.246397013 sec., 16825.609979565976 rows/sec., 2.64 MiB/sec. 2024.10.22 01:08:25.949082 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5360 2024.10.22 01:08:26.189250 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_623_246} MergedBlockOutputStream: filled checksums 202410_1_623_246 (state Temporary) 2024.10.22 01:08:26.394072 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fcf24f0efb495846199cc291fa1c750b) : [0, 162]: Setting download as failed: File segment: [0, 162], key: fcf24f0efb495846199cc291fa1c750b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:26.395117 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fcf24f0efb495846199cc291fa1c750b): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:08:26.410207 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_522_153, 202410_523_523_0] 2024.10.22 01:08:26.410436 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_623_246} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_623_246 to 202410_1_623_246 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:26.415674 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_522_153, 202410_523_523_0] 2024.10.22 01:08:26.417423 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_623_246} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_622_245, 202410_623_623_0] -> 202410_1_623_246 2024.10.22 01:08:26.464795 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.89 GiB, free memory in arenas 14.70 MiB, will set to 1.70 GiB (RSS), difference: 16.63 MiB 2024.10.22 01:08:26.550324 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_522_153, 202410_523_523_0] 2024.10.22 01:08:26.576173 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:08:27.136582 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d27220025fc15c6755b47a1bc448a046) : [0, 113367]: Setting download as failed: File segment: [0, 113367], key: d27220025fc15c6755b47a1bc448a046, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:27.139469 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d27220025fc15c6755b47a1bc448a046): No space left in cache to reserve 113368 bytes, will continue without cache download 2024.10.22 01:08:27.180009 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d27220025fc15c6755b47a1bc448a046, offset: 0, size: 0 2024.10.22 01:08:27.182482 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d27220025fc15c6755b47a1bc448a046 for removal 2024.10.22 01:08:27.185195 [ 1476 ] {} CacheMetadata: Key d27220025fc15c6755b47a1bc448a046 is removed from metadata 2024.10.22 01:08:27.185897 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fcf24f0efb495846199cc291fa1c750b, offset: 0, size: 0 2024.10.22 01:08:27.187279 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fcf24f0efb495846199cc291fa1c750b for removal 2024.10.22 01:08:27.191012 [ 1476 ] {} CacheMetadata: Key fcf24f0efb495846199cc291fa1c750b is removed from metadata 2024.10.22 01:08:27.476912 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 204 entries to flush up to offset 31348 2024.10.22 01:08:27.645639 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 37388 2024.10.22 01:08:27.668096 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:27.676501 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:27.881674 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f9d99df74b18253e077f3b2ff61fa310) : [0, 162]: Setting download as failed: File segment: [0, 162], key: f9d99df74b18253e077f3b2ff61fa310, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:27.882964 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f9d99df74b18253e077f3b2ff61fa310): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:08:27.933679 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_357_357_0 (state Temporary) 2024.10.22 01:08:28.000918 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_357_357_0 to 202410_363_363_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:28.015495 [ 1244 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_357_135 to 202410_358_358_0 2024.10.22 01:08:28.024958 [ 1244 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.61 MiB using storage policy from min volume index 0 2024.10.22 01:08:28.032268 [ 1244 ] {} DiskLocal: Reserved 1.61 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:28.068711 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:28.076747 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:28.089225 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_358_136} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_357_135 to 202410_358_358_0 into Wide with storage Full 2024.10.22 01:08:28.130612 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_358_136} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:28.145591 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_358_136} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_357_135, total 30132 rows starting from the beginning of the part 2024.10.22 01:08:28.151099 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_615_615_0 (state Temporary) 2024.10.22 01:08:28.226262 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_358_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_358_358_0, total 193 rows starting from the beginning of the part 2024.10.22 01:08:28.420310 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_615_615_0 to 202410_628_628_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:28.507550 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 564272 2024.10.22 01:08:28.579822 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 31348 2024.10.22 01:08:28.594487 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_623_246 to 202410_624_624_0 2024.10.22 01:08:28.599424 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.34 MiB using storage policy from min volume index 0 2024.10.22 01:08:28.612785 [ 1235 ] {} DiskLocal: Reserved 1.34 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:28.615007 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.89 GiB, free memory in arenas 13.40 MiB, will set to 1.69 GiB (RSS), difference: -33.13 MiB 2024.10.22 01:08:28.704910 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(471d415eba616dcba5411344cb3a54f9) : [0, 109599]: Setting download as failed: File segment: [0, 109599], key: 471d415eba616dcba5411344cb3a54f9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:28.706227 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(471d415eba616dcba5411344cb3a54f9): No space left in cache to reserve 109600 bytes, will continue without cache download 2024.10.22 01:08:28.751891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 471d415eba616dcba5411344cb3a54f9, offset: 0, size: 0 2024.10.22 01:08:28.753904 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 471d415eba616dcba5411344cb3a54f9 for removal 2024.10.22 01:08:28.761465 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f9d99df74b18253e077f3b2ff61fa310, offset: 0, size: 0 2024.10.22 01:08:28.764741 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f9d99df74b18253e077f3b2ff61fa310 for removal 2024.10.22 01:08:28.779898 [ 1476 ] {} CacheMetadata: Key 471d415eba616dcba5411344cb3a54f9 is removed from metadata 2024.10.22 01:08:28.787246 [ 1476 ] {} CacheMetadata: Key f9d99df74b18253e077f3b2ff61fa310 is removed from metadata 2024.10.22 01:08:28.808295 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_624_247} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_623_246 to 202410_624_624_0 into Compact with storage Full 2024.10.22 01:08:28.892558 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37388 2024.10.22 01:08:28.972535 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_624_247} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:28.983816 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_624_247} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_623_246, total 37797 rows starting from the beginning of the part 2024.10.22 01:08:29.034816 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_624_247} MergeTreeSequentialSource: Reading 2 marks from part 202410_624_624_0, total 84 rows starting from the beginning of the part 2024.10.22 01:08:29.394985 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_351_70/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:08:29.449266 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:29.451925 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:29.735825 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_346_346_0 (state Temporary) 2024.10.22 01:08:29.891025 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:08:29.893442 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:08:29.910395 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8606913d58edff1f2bf9f5ed0deaeb62) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 8606913d58edff1f2bf9f5ed0deaeb62, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:29.926612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8606913d58edff1f2bf9f5ed0deaeb62): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:08:30.051752 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_346_346_0 to 202410_353_353_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:30.097708 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 564272 2024.10.22 01:08:30.447886 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.89 GiB, free memory in arenas 13.28 MiB, will set to 1.71 GiB (RSS), difference: -14.94 MiB 2024.10.22 01:08:30.450806 [ 1249 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:08:30.606657 [ 1249 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:08:31.307838 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(70d8a42fc84de101ef534113fc60d206) : [0, 52139]: Setting download as failed: File segment: [0, 52139], key: 70d8a42fc84de101ef534113fc60d206, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:31.309919 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(70d8a42fc84de101ef534113fc60d206): No space left in cache to reserve 52140 bytes, will continue without cache download 2024.10.22 01:08:31.418198 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 70d8a42fc84de101ef534113fc60d206, offset: 0, size: 0 2024.10.22 01:08:31.420291 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 70d8a42fc84de101ef534113fc60d206 for removal 2024.10.22 01:08:31.426898 [ 1476 ] {} CacheMetadata: Key 70d8a42fc84de101ef534113fc60d206 is removed from metadata 2024.10.22 01:08:31.431093 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8606913d58edff1f2bf9f5ed0deaeb62, offset: 0, size: 0 2024.10.22 01:08:31.535851 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8606913d58edff1f2bf9f5ed0deaeb62 for removal 2024.10.22 01:08:31.542508 [ 1476 ] {} CacheMetadata: Key 8606913d58edff1f2bf9f5ed0deaeb62 is removed from metadata 2024.10.22 01:08:31.554792 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_358_136} MergeTask::MergeProjectionsStage: Merge sorted 30325 rows, containing 13 columns (13 merged, 0 gathered) in 3.468055077 sec., 8744.094118087733 rows/sec., 3.74 MiB/sec. 2024.10.22 01:08:31.647886 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_358_136} MergedBlockOutputStream: filled checksums 202410_1_358_136 (state Temporary) 2024.10.22 01:08:32.071905 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_624_247} MergeTask::MergeProjectionsStage: Merge sorted 37881 rows, containing 13 columns (13 merged, 0 gathered) in 3.366851489 sec., 11251.164514907417 rows/sec., 1.76 MiB/sec. 2024.10.22 01:08:32.176198 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_624_247} MergedBlockOutputStream: filled checksums 202410_1_624_247 (state Temporary) 2024.10.22 01:08:32.529046 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:32.537231 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:32.589269 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.89 GiB, free memory in arenas 14.86 MiB, will set to 1.70 GiB (RSS), difference: 47.93 MiB 2024.10.22 01:08:32.612484 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_523_154, 202410_524_524_0] 2024.10.22 01:08:32.616726 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_523_154, 202410_524_524_0] 2024.10.22 01:08:32.612444 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_624_247} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_624_247 to 202410_1_624_247 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:32.640424 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_624_247} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_623_246, 202410_624_624_0] -> 202410_1_624_247 2024.10.22 01:08:32.761134 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_523_154, 202410_524_524_0] 2024.10.22 01:08:32.884886 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:08:32.895189 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_358_136} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_358_136 to 202410_1_358_136 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:33.000738 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 37459 2024.10.22 01:08:33.004093 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_358_136} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_357_135, 202410_358_358_0] -> 202410_1_358_136 2024.10.22 01:08:33.014534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4d2dd815e1950f18688f2e4be5d9e52e) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 4d2dd815e1950f18688f2e4be5d9e52e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:33.016608 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4d2dd815e1950f18688f2e4be5d9e52e): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:08:33.021233 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 61.57 MiB. 2024.10.22 01:08:33.182427 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:33.186369 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:33.237263 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_616_616_0 (state Temporary) 2024.10.22 01:08:33.383569 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_616_616_0 to 202410_629_629_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:33.415263 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_624_247 to 202410_625_625_0 2024.10.22 01:08:33.422435 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.34 MiB using storage policy from min volume index 0 2024.10.22 01:08:33.428439 [ 1268 ] {} DiskLocal: Reserved 1.34 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:33.455691 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 5381 2024.10.22 01:08:33.483608 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_625_248} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_624_247 to 202410_625_625_0 into Compact with storage Full 2024.10.22 01:08:33.591263 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37459 2024.10.22 01:08:33.607614 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_625_248} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:33.623356 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_625_248} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_624_247, total 37881 rows starting from the beginning of the part 2024.10.22 01:08:33.651151 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_625_248} MergeTreeSequentialSource: Reading 2 marks from part 202410_625_625_0, total 76 rows starting from the beginning of the part 2024.10.22 01:08:33.678916 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:33.681643 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:33.802942 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5e0a48d41d3f61a2c2bd85ce8f9498a1) : [0, 52576]: Setting download as failed: File segment: [0, 52576], key: 5e0a48d41d3f61a2c2bd85ce8f9498a1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:33.804411 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5e0a48d41d3f61a2c2bd85ce8f9498a1): No space left in cache to reserve 52577 bytes, will continue without cache download 2024.10.22 01:08:33.807693 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_362_362_0 (state Temporary) 2024.10.22 01:08:33.834614 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5e0a48d41d3f61a2c2bd85ce8f9498a1, offset: 0, size: 0 2024.10.22 01:08:33.839834 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5e0a48d41d3f61a2c2bd85ce8f9498a1 for removal 2024.10.22 01:08:33.844091 [ 1476 ] {} CacheMetadata: Key 5e0a48d41d3f61a2c2bd85ce8f9498a1 is removed from metadata 2024.10.22 01:08:33.845165 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4d2dd815e1950f18688f2e4be5d9e52e, offset: 0, size: 0 2024.10.22 01:08:33.847908 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4d2dd815e1950f18688f2e4be5d9e52e for removal 2024.10.22 01:08:33.851299 [ 1476 ] {} CacheMetadata: Key 4d2dd815e1950f18688f2e4be5d9e52e is removed from metadata 2024.10.22 01:08:33.863950 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_362_362_0 to 202410_375_375_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:33.929010 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5381 2024.10.22 01:08:34.161086 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_74_15/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:08:34.389383 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} MergeTask::MergeProjectionsStage: Merge sorted 575558 rows, containing 4 columns (4 merged, 0 gathered) in 20.795982184 sec., 27676.403783555 rows/sec., 405.74 KiB/sec. 2024.10.22 01:08:34.415741 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} MergedBlockOutputStream: filled checksums 202410_1_351_70 (state Temporary) 2024.10.22 01:08:34.450790 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_351_70 to 202410_1_351_70 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:34.453590 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_351_70} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_346_69, 202410_351_351_0] -> 202410_1_351_70 2024.10.22 01:08:34.492304 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:08:34.564772 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.89 GiB, free memory in arenas 16.79 MiB, will set to 1.70 GiB (RSS), difference: -3.60 MiB 2024.10.22 01:08:34.859473 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(17204410ead05b3130dbd59b1b0a6f76) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 17204410ead05b3130dbd59b1b0a6f76, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:34.860833 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(17204410ead05b3130dbd59b1b0a6f76): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:08:35.785117 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b0ba0630e077d56346265ef2441b01f6) : [0, 31073]: Setting download as failed: File segment: [0, 31073], key: b0ba0630e077d56346265ef2441b01f6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:35.786748 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b0ba0630e077d56346265ef2441b01f6): No space left in cache to reserve 31074 bytes, will continue without cache download 2024.10.22 01:08:35.787063 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_625_248/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:08:35.797550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b0ba0630e077d56346265ef2441b01f6, offset: 0, size: 0 2024.10.22 01:08:35.799637 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b0ba0630e077d56346265ef2441b01f6 for removal 2024.10.22 01:08:35.802928 [ 1476 ] {} CacheMetadata: Key b0ba0630e077d56346265ef2441b01f6 is removed from metadata 2024.10.22 01:08:35.803870 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 17204410ead05b3130dbd59b1b0a6f76, offset: 0, size: 0 2024.10.22 01:08:35.805588 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 17204410ead05b3130dbd59b1b0a6f76 for removal 2024.10.22 01:08:35.807311 [ 1476 ] {} CacheMetadata: Key 17204410ead05b3130dbd59b1b0a6f76 is removed from metadata 2024.10.22 01:08:36.035923 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_73_73_0 (state Temporary) 2024.10.22 01:08:36.215999 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 217 entries to flush up to offset 31565 2024.10.22 01:08:36.414958 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 15.01 MiB, will set to 1.70 GiB (RSS), difference: 8.85 MiB 2024.10.22 01:08:36.591358 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_625_248} MergeTask::MergeProjectionsStage: Merge sorted 37957 rows, containing 13 columns (13 merged, 0 gathered) in 3.11708904 sec., 12177.066331092035 rows/sec., 1.91 MiB/sec. 2024.10.22 01:08:36.672922 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_625_248} MergedBlockOutputStream: filled checksums 202410_1_625_248 (state Temporary) 2024.10.22 01:08:36.714201 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:36.716037 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:36.876851 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_358_358_0 (state Temporary) 2024.10.22 01:08:36.891562 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_625_248} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_625_248 to 202410_1_625_248 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:36.894985 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_625_248} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_624_247, 202410_625_625_0] -> 202410_1_625_248 2024.10.22 01:08:36.903106 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(42eea60416da613af4f948b1ccbb12e2) : [0, 150]: Setting download as failed: File segment: [0, 150], key: 42eea60416da613af4f948b1ccbb12e2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:36.905121 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(42eea60416da613af4f948b1ccbb12e2): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:08:36.915182 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:08:37.007846 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 310 2024.10.22 01:08:37.047866 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_358_358_0 to 202410_364_364_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:37.084982 [ 1168 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_358_136 to 202410_359_359_0 2024.10.22 01:08:37.094740 [ 1168 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.62 MiB using storage policy from min volume index 0 2024.10.22 01:08:37.095347 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 31565 2024.10.22 01:08:37.104205 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 565808 2024.10.22 01:08:37.200273 [ 1168 ] {} DiskLocal: Reserved 1.62 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:37.217690 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_359_137} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_358_136 to 202410_359_359_0 into Wide with storage Full 2024.10.22 01:08:37.259631 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:37.264407 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:37.270686 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_359_137} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:37.291600 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_359_137} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_358_136, total 30325 rows starting from the beginning of the part 2024.10.22 01:08:37.430733 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_292_292_0 (state Temporary) 2024.10.22 01:08:37.539141 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(932ff2d36befc626e049f7bc39258e15) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 932ff2d36befc626e049f7bc39258e15, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:08:37.545498 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 48, file segment info: File segment: [0, 33554431], key: 932ff2d36befc626e049f7bc39258e15, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:08:37.551504 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:37.605382 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 37538 2024.10.22 01:08:37.610989 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_292_292_0 to 202410_302_302_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:37.652320 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_524_155, 202410_525_525_0] 2024.10.22 01:08:37.658885 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_524_155, 202410_525_525_0] 2024.10.22 01:08:37.676131 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_524_155, 202410_525_525_0] 2024.10.22 01:08:37.731634 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_359_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_359_359_0, total 242 rows starting from the beginning of the part 2024.10.22 01:08:37.811275 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 310 2024.10.22 01:08:37.876952 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:37.882386 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:38.041030 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:38.047317 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:38.096624 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_617_617_0 (state Temporary) 2024.10.22 01:08:38.204532 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_347_347_0 (state Temporary) 2024.10.22 01:08:38.241469 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_617_617_0 to 202410_630_630_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:38.281766 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(1c324bc4f89757f09559647065814638) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1c324bc4f89757f09559647065814638, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:08:38.285137 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1445, file segment info: File segment: [0, 33554431], key: 1c324bc4f89757f09559647065814638, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:08:38.287807 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:38.348480 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_347_347_0 to 202410_354_354_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:38.388275 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_625_248 to 202410_626_626_0 2024.10.22 01:08:38.392578 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.34 MiB using storage policy from min volume index 0 2024.10.22 01:08:38.396010 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 565808 2024.10.22 01:08:38.396082 [ 1187 ] {} DiskLocal: Reserved 1.34 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:38.421012 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_626_249} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_625_248 to 202410_626_626_0 into Compact with storage Full 2024.10.22 01:08:38.485112 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_626_249} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:38.492615 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_626_249} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_625_248, total 37957 rows starting from the beginning of the part 2024.10.22 01:08:38.493321 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37538 2024.10.22 01:08:38.509758 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_626_249} MergeTreeSequentialSource: Reading 2 marks from part 202410_626_626_0, total 74 rows starting from the beginning of the part 2024.10.22 01:08:38.898264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(55fb3702633bfc03ef4954e9cb14fbad) : [0, 9544]: Setting download as failed: File segment: [0, 9544], key: 55fb3702633bfc03ef4954e9cb14fbad, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:38.899684 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(55fb3702633bfc03ef4954e9cb14fbad): No space left in cache to reserve 9545 bytes, will continue without cache download 2024.10.22 01:08:38.904789 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 55fb3702633bfc03ef4954e9cb14fbad, offset: 0, size: 0 2024.10.22 01:08:38.906620 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 55fb3702633bfc03ef4954e9cb14fbad for removal 2024.10.22 01:08:38.907868 [ 1476 ] {} CacheMetadata: Key 55fb3702633bfc03ef4954e9cb14fbad is removed from metadata 2024.10.22 01:08:38.908610 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 42eea60416da613af4f948b1ccbb12e2, offset: 0, size: 0 2024.10.22 01:08:38.911725 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 42eea60416da613af4f948b1ccbb12e2 for removal 2024.10.22 01:08:38.926699 [ 1476 ] {} CacheMetadata: Key 42eea60416da613af4f948b1ccbb12e2 is removed from metadata 2024.10.22 01:08:39.487003 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.89 GiB, free memory in arenas 7.54 MiB, will set to 1.71 GiB (RSS), difference: -47.76 MiB 2024.10.22 01:08:39.520169 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(1e4499c1856b06df07f6c6ea122fbfbe) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1e4499c1856b06df07f6c6ea122fbfbe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:08:39.536305 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 141, file segment info: File segment: [0, 33554431], key: 1e4499c1856b06df07f6c6ea122fbfbe, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:08:39.539910 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:39.788026 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_301_91, 202410_302_302_0] 2024.10.22 01:08:39.790645 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_301_91, 202410_302_302_0] 2024.10.22 01:08:39.797262 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_301_91, 202410_302_302_0] 2024.10.22 01:08:39.892247 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:08:39.894440 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:08:40.085431 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_359_137} MergeTask::MergeProjectionsStage: Merge sorted 30567 rows, containing 13 columns (13 merged, 0 gathered) in 2.855922866 sec., 10703.020156427432 rows/sec., 4.58 MiB/sec. 2024.10.22 01:08:40.184017 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_359_137} MergedBlockOutputStream: filled checksums 202410_1_359_137 (state Temporary) 2024.10.22 01:08:40.306431 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(0bd680259737ef847b2f5b7212065aba) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0bd680259737ef847b2f5b7212065aba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:08:40.308702 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 876, file segment info: File segment: [0, 33554431], key: 0bd680259737ef847b2f5b7212065aba, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:08:40.315927 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:40.505055 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_249_51, 202410_250_250_0, 202410_251_251_0, 202410_252_252_0, 202410_253_253_0] 2024.10.22 01:08:40.508747 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_249_51, 202410_250_250_0, 202410_251_251_0, 202410_252_252_0, 202410_253_253_0] 2024.10.22 01:08:40.565786 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_249_51, 202410_250_250_0, 202410_251_251_0, 202410_252_252_0, 202410_253_253_0] 2024.10.22 01:08:40.804425 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_359_137} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_359_137 to 202410_1_359_137 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:40.807053 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_359_137} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_358_136, 202410_359_359_0] -> 202410_1_359_137 2024.10.22 01:08:40.828961 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 61.65 MiB. 2024.10.22 01:08:41.306458 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2492823d522eab886323c5e4c8dc7a37) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 2492823d522eab886323c5e4c8dc7a37, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:41.307768 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2492823d522eab886323c5e4c8dc7a37): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:08:41.313162 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_626_249} MergeTask::MergeProjectionsStage: Merge sorted 38031 rows, containing 13 columns (13 merged, 0 gathered) in 2.89781632 sec., 13124.020227755498 rows/sec., 2.06 MiB/sec. 2024.10.22 01:08:41.431230 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 5399 2024.10.22 01:08:41.561607 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:41.569191 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_626_249} MergedBlockOutputStream: filled checksums 202410_1_626_249 (state Temporary) 2024.10.22 01:08:41.569554 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:41.613706 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_363_363_0 (state Temporary) 2024.10.22 01:08:41.724722 [ 1153 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 317 ms. 2024.10.22 01:08:41.804631 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_626_249} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_626_249 to 202410_1_626_249 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:41.807734 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_626_249} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_625_248, 202410_626_626_0] -> 202410_1_626_249 2024.10.22 01:08:41.813002 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_311_62, 202410_312_312_0, 202410_313_313_0, 202410_314_314_0, 202410_315_315_0, 202410_316_316_0] 2024.10.22 01:08:41.827442 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:08:41.827475 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_311_62, 202410_312_312_0, 202410_313_313_0, 202410_314_314_0, 202410_315_315_0, 202410_316_316_0] 2024.10.22 01:08:41.867040 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_311_62, 202410_312_312_0, 202410_313_313_0, 202410_314_314_0, 202410_315_315_0, 202410_316_316_0] 2024.10.22 01:08:41.879284 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_363_363_0 to 202410_376_376_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:42.020200 [ 1278 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_371_74 to 202410_376_376_0 2024.10.22 01:08:42.034351 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5399 2024.10.22 01:08:42.035801 [ 1278 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:42.050927 [ 1278 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:42.081764 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_371_74 to 202410_376_376_0 into Compact with storage Full 2024.10.22 01:08:42.099337 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(0d713e24fba00c6b2320514b48e52acb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0d713e24fba00c6b2320514b48e52acb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:08:42.107650 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1042, file segment info: File segment: [0, 33554431], key: 0d713e24fba00c6b2320514b48e52acb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:08:42.109320 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:42.235735 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:42.247691 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_371_74, total 5417 rows starting from the beginning of the part 2024.10.22 01:08:42.398872 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_372_372_0, total 23 rows starting from the beginning of the part 2024.10.22 01:08:42.501457 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 37620 2024.10.22 01:08:42.553288 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_373_373_0, total 14 rows starting from the beginning of the part 2024.10.22 01:08:42.606584 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_374_374_0, total 12 rows starting from the beginning of the part 2024.10.22 01:08:42.687765 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_375_375_0, total 21 rows starting from the beginning of the part 2024.10.22 01:08:42.795300 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b4d6d0b4caa28472bd422ab1b91c82e9) : [0, 105748]: Setting download as failed: File segment: [0, 105748], key: b4d6d0b4caa28472bd422ab1b91c82e9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:42.797127 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b4d6d0b4caa28472bd422ab1b91c82e9): No space left in cache to reserve 105749 bytes, will continue without cache download 2024.10.22 01:08:42.822986 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_376_376_0, total 18 rows starting from the beginning of the part 2024.10.22 01:08:42.912882 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b4d6d0b4caa28472bd422ab1b91c82e9, offset: 0, size: 0 2024.10.22 01:08:42.915141 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b4d6d0b4caa28472bd422ab1b91c82e9 for removal 2024.10.22 01:08:42.919898 [ 1476 ] {} CacheMetadata: Key b4d6d0b4caa28472bd422ab1b91c82e9 is removed from metadata 2024.10.22 01:08:42.923170 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2492823d522eab886323c5e4c8dc7a37, offset: 0, size: 0 2024.10.22 01:08:42.927045 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2492823d522eab886323c5e4c8dc7a37 for removal 2024.10.22 01:08:42.931484 [ 1476 ] {} CacheMetadata: Key 2492823d522eab886323c5e4c8dc7a37 is removed from metadata 2024.10.22 01:08:43.258401 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.89 GiB, free memory in arenas 6.43 MiB, will set to 1.70 GiB (RSS), difference: 42.92 MiB 2024.10.22 01:08:43.318998 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:43.323377 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:43.383426 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b35e437890688b4fac4a4d6d8137486d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b35e437890688b4fac4a4d6d8137486d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:08:43.387333 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 39426, file segment info: File segment: [0, 33554431], key: b35e437890688b4fac4a4d6d8137486d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:08:43.391091 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:43.401557 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_618_618_0 (state Temporary) 2024.10.22 01:08:43.453777 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_618_618_0 to 202410_631_631_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:43.589808 [ 1273 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_626_249 to 202410_627_627_0 2024.10.22 01:08:43.594479 [ 1273 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.35 MiB using storage policy from min volume index 0 2024.10.22 01:08:43.595898 [ 1273 ] {} DiskLocal: Reserved 1.35 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:43.599582 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37620 2024.10.22 01:08:43.616228 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_627_250} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_626_249 to 202410_627_627_0 into Compact with storage Full 2024.10.22 01:08:43.662201 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_627_250} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:43.676774 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_627_250} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_626_249, total 38031 rows starting from the beginning of the part 2024.10.22 01:08:43.801158 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_627_250} MergeTreeSequentialSource: Reading 2 marks from part 202410_627_627_0, total 111 rows starting from the beginning of the part 2024.10.22 01:08:44.031088 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(46bd9d04e0ebcabac93b868b94baf425) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 46bd9d04e0ebcabac93b868b94baf425, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:44.034673 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(46bd9d04e0ebcabac93b868b94baf425): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:08:44.290930 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_525_156, 202410_526_526_0, 202410_527_527_0, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0] 2024.10.22 01:08:44.299910 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} MergeTask::MergeProjectionsStage: Merge sorted 5505 rows, containing 12 columns (12 merged, 0 gathered) in 2.231538001 sec., 2466.908471884902 rows/sec., 389.28 KiB/sec. 2024.10.22 01:08:44.305244 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_525_156, 202410_526_526_0, 202410_527_527_0, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0] 2024.10.22 01:08:44.445237 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_525_156, 202410_526_526_0, 202410_527_527_0, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0] 2024.10.22 01:08:44.545526 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} MergedBlockOutputStream: filled checksums 202410_1_376_75 (state Temporary) 2024.10.22 01:08:44.734265 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 240 entries to flush up to offset 31805 2024.10.22 01:08:44.794962 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_376_75 to 202410_1_376_75 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:44.798418 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_376_75} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_371_74, 202410_376_376_0] -> 202410_1_376_75 2024.10.22 01:08:44.838937 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.41 MiB. 2024.10.22 01:08:44.896848 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(f324b23042ac0024c5a21f33502a1cab) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f324b23042ac0024c5a21f33502a1cab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:08:44.905126 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 469, file segment info: File segment: [0, 33554431], key: f324b23042ac0024c5a21f33502a1cab, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:08:44.906631 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:45.104003 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_73_73_0 to 202410_76_76_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:45.124418 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:45.133047 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:45.307933 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_359_359_0 (state Temporary) 2024.10.22 01:08:45.314525 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 311 2024.10.22 01:08:45.400187 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1129 entries to flush up to offset 566937 2024.10.22 01:08:45.440943 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:08:45.453665 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:08:45.469220 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:08:45.570734 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d30705bab8facd285ffaad566038dba3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d30705bab8facd285ffaad566038dba3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:08:45.575304 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:08:45.575745 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 890, file segment info: File segment: [0, 33554431], key: d30705bab8facd285ffaad566038dba3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:08:45.581097 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:08:45.587809 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:08:45.610920 [ 1210 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:08:45.617171 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:08:45.648415 [ 1210 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:08:45.709341 [ 1709 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jkl/lzodacwtthrgoriormwqyhrpbattv, size 39426 2024.10.22 01:08:45.748860 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:08:45.780903 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:45.784633 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:45.797165 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:08:45.804259 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:08:45.821742 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:08:45.827500 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_359_359_0 to 202410_365_365_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:45.877267 [ 1233 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_359_137 to 202410_360_360_0 2024.10.22 01:08:45.889051 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 31805 2024.10.22 01:08:45.891838 [ 1233 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.63 MiB using storage policy from min volume index 0 2024.10.22 01:08:45.894599 [ 1233 ] {} DiskLocal: Reserved 1.63 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:45.914942 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_360_138} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_359_137 to 202410_360_360_0 into Wide with storage Full 2024.10.22 01:08:45.926591 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kgd/ivarfoumosmdfhdwuqxvpunefsmwu (LogSeriesLimiter: on interval from 2024-10-22 01:08:18 to 2024-10-22 01:08:45 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:08:45.959491 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:45.966882 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_293_293_0 (state Temporary) 2024.10.22 01:08:45.966633 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:45.971413 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_360_138} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:45.975585 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:08:45.986773 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_360_138} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_359_137, total 30567 rows starting from the beginning of the part 2024.10.22 01:08:45.990559 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:08:46.011099 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:08:46.028721 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:08:46.065202 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:08:46.069425 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_293_293_0 to 202410_303_303_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:46.104406 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_348_348_0 (state Temporary) 2024.10.22 01:08:46.155191 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 311 2024.10.22 01:08:46.202074 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:08:46.250936 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_627_250} MergeTask::MergeProjectionsStage: Merge sorted 38142 rows, containing 13 columns (13 merged, 0 gathered) in 2.641095714 sec., 14441.733329775114 rows/sec., 2.27 MiB/sec. 2024.10.22 01:08:46.255676 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_360_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_360_360_0, total 231 rows starting from the beginning of the part 2024.10.22 01:08:46.306633 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(194c2c82501c88e4152164e14754517c) : [0, 2626]: Setting download as failed: File segment: [0, 2626], key: 194c2c82501c88e4152164e14754517c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:46.307780 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(194c2c82501c88e4152164e14754517c): No space left in cache to reserve 2627 bytes, will continue without cache download 2024.10.22 01:08:46.314228 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_348_348_0 to 202410_355_355_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:46.330575 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 194c2c82501c88e4152164e14754517c, offset: 0, size: 0 2024.10.22 01:08:46.333600 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 194c2c82501c88e4152164e14754517c for removal 2024.10.22 01:08:46.339123 [ 1476 ] {} CacheMetadata: Key 194c2c82501c88e4152164e14754517c is removed from metadata 2024.10.22 01:08:46.341121 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 46bd9d04e0ebcabac93b868b94baf425, offset: 0, size: 0 2024.10.22 01:08:46.346328 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 46bd9d04e0ebcabac93b868b94baf425 for removal 2024.10.22 01:08:46.351119 [ 1476 ] {} CacheMetadata: Key 46bd9d04e0ebcabac93b868b94baf425 is removed from metadata 2024.10.22 01:08:46.428189 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_627_250} MergedBlockOutputStream: filled checksums 202410_1_627_250 (state Temporary) 2024.10.22 01:08:46.559428 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 566937 2024.10.22 01:08:46.563816 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.89 GiB, free memory in arenas 12.10 MiB, will set to 1.69 GiB (RSS), difference: -38.55 MiB 2024.10.22 01:08:46.847984 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_627_250} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_627_250 to 202410_1_627_250 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:46.854588 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_627_250} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_626_249, 202410_627_627_0] -> 202410_1_627_250 2024.10.22 01:08:46.892873 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:08:47.217230 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(64ddc5570f24520d052b86fff48c3d53) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 64ddc5570f24520d052b86fff48c3d53, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:47.220349 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(64ddc5570f24520d052b86fff48c3d53): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:08:47.581565 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_74_15/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:08:47.607749 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 108 entries to flush up to offset 37728 2024.10.22 01:08:47.903430 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_302_92, 202410_303_303_0] 2024.10.22 01:08:47.904833 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_302_92, 202410_303_303_0] 2024.10.22 01:08:47.914800 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_302_92, 202410_303_303_0] 2024.10.22 01:08:48.045073 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:48.049439 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:48.085487 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6e518c8c59734e5a57fec3e2a494bcc8) : [0, 3612]: Setting download as failed: File segment: [0, 3612], key: 6e518c8c59734e5a57fec3e2a494bcc8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:48.087115 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6e518c8c59734e5a57fec3e2a494bcc8): No space left in cache to reserve 3613 bytes, will continue without cache download 2024.10.22 01:08:48.097748 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6e518c8c59734e5a57fec3e2a494bcc8, offset: 0, size: 0 2024.10.22 01:08:48.099648 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6e518c8c59734e5a57fec3e2a494bcc8 for removal 2024.10.22 01:08:48.102812 [ 1476 ] {} CacheMetadata: Key 6e518c8c59734e5a57fec3e2a494bcc8 is removed from metadata 2024.10.22 01:08:48.103581 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 64ddc5570f24520d052b86fff48c3d53, offset: 0, size: 0 2024.10.22 01:08:48.105393 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 64ddc5570f24520d052b86fff48c3d53 for removal 2024.10.22 01:08:48.108746 [ 1476 ] {} CacheMetadata: Key 64ddc5570f24520d052b86fff48c3d53 is removed from metadata 2024.10.22 01:08:48.122916 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_619_619_0 (state Temporary) 2024.10.22 01:08:48.185471 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2913 2024.10.22 01:08:48.386986 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_619_619_0 to 202410_632_632_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:48.407725 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_627_250 to 202410_628_628_0 2024.10.22 01:08:48.432835 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.35 MiB using storage policy from min volume index 0 2024.10.22 01:08:48.435957 [ 1214 ] {} DiskLocal: Reserved 1.35 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:48.457701 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_628_251} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_627_250 to 202410_628_628_0 into Compact with storage Full 2024.10.22 01:08:48.503422 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_628_251} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:48.511363 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_628_251} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_627_250, total 38142 rows starting from the beginning of the part 2024.10.22 01:08:48.541834 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37728 2024.10.22 01:08:48.553367 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_628_251} MergeTreeSequentialSource: Reading 2 marks from part 202410_628_628_0, total 77 rows starting from the beginning of the part 2024.10.22 01:08:48.708990 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_360_138} MergeTask::MergeProjectionsStage: Merge sorted 30798 rows, containing 13 columns (13 merged, 0 gathered) in 2.801638827 sec., 10992.851649253646 rows/sec., 4.71 MiB/sec. 2024.10.22 01:08:48.785160 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_360_138} MergedBlockOutputStream: filled checksums 202410_1_360_138 (state Temporary) 2024.10.22 01:08:48.962632 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c7cb2592abc03e9e4d51b3fb76e1546a) : [0, 151]: Setting download as failed: File segment: [0, 151], key: c7cb2592abc03e9e4d51b3fb76e1546a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:48.968591 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c7cb2592abc03e9e4d51b3fb76e1546a): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:08:49.394981 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_360_138} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_360_138 to 202410_1_360_138 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:49.399186 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_360_138} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_359_137, 202410_360_360_0] -> 202410_1_360_138 2024.10.22 01:08:49.416942 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.41 MiB. 2024.10.22 01:08:49.536592 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 5422 2024.10.22 01:08:49.546921 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.89 GiB, free memory in arenas 14.20 MiB, will set to 1.69 GiB (RSS), difference: 40.88 MiB 2024.10.22 01:08:49.664760 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:49.776375 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:49.894515 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:08:49.896545 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:08:49.905564 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_364_364_0 (state Temporary) 2024.10.22 01:08:49.960721 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b44aabc05d89a767e98f23573e43439e) : [0, 30457]: Setting download as failed: File segment: [0, 30457], key: b44aabc05d89a767e98f23573e43439e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:49.965583 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b44aabc05d89a767e98f23573e43439e): No space left in cache to reserve 30458 bytes, will continue without cache download 2024.10.22 01:08:49.974862 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_364_364_0 to 202410_377_377_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:50.095800 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b44aabc05d89a767e98f23573e43439e, offset: 0, size: 0 2024.10.22 01:08:50.098619 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b44aabc05d89a767e98f23573e43439e for removal 2024.10.22 01:08:50.100939 [ 1476 ] {} CacheMetadata: Key b44aabc05d89a767e98f23573e43439e is removed from metadata 2024.10.22 01:08:50.102609 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c7cb2592abc03e9e4d51b3fb76e1546a, offset: 0, size: 0 2024.10.22 01:08:50.104646 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c7cb2592abc03e9e4d51b3fb76e1546a for removal 2024.10.22 01:08:50.106450 [ 1476 ] {} CacheMetadata: Key c7cb2592abc03e9e4d51b3fb76e1546a is removed from metadata 2024.10.22 01:08:50.149797 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5422 2024.10.22 01:08:50.839495 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1108878a134ed7a06e21011ff75987ec) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 1108878a134ed7a06e21011ff75987ec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:50.842554 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1108878a134ed7a06e21011ff75987ec): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:08:51.448760 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 13.31 MiB, will set to 1.69 GiB (RSS), difference: 5.89 MiB 2024.10.22 01:08:51.736621 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d76481086d3cd6b275e7a96776e246ec) : [0, 29352]: Setting download as failed: File segment: [0, 29352], key: d76481086d3cd6b275e7a96776e246ec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:51.741808 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d76481086d3cd6b275e7a96776e246ec): No space left in cache to reserve 29353 bytes, will continue without cache download 2024.10.22 01:08:51.749553 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_628_251} MergeTask::MergeProjectionsStage: Merge sorted 38219 rows, containing 13 columns (13 merged, 0 gathered) in 3.303666241 sec., 11568.662574228847 rows/sec., 1.81 MiB/sec. 2024.10.22 01:08:51.768895 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d76481086d3cd6b275e7a96776e246ec, offset: 0, size: 0 2024.10.22 01:08:51.771439 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d76481086d3cd6b275e7a96776e246ec for removal 2024.10.22 01:08:51.775619 [ 1476 ] {} CacheMetadata: Key d76481086d3cd6b275e7a96776e246ec is removed from metadata 2024.10.22 01:08:51.777135 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1108878a134ed7a06e21011ff75987ec, offset: 0, size: 0 2024.10.22 01:08:51.778822 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1108878a134ed7a06e21011ff75987ec for removal 2024.10.22 01:08:51.780598 [ 1476 ] {} CacheMetadata: Key 1108878a134ed7a06e21011ff75987ec is removed from metadata 2024.10.22 01:08:51.941037 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_628_251} MergedBlockOutputStream: filled checksums 202410_1_628_251 (state Temporary) 2024.10.22 01:08:52.004758 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_628_251} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_628_251 to 202410_1_628_251 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:52.008043 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_628_251} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_627_250, 202410_628_628_0] -> 202410_1_628_251 2024.10.22 01:08:52.034874 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:08:52.509753 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c0ea6f67f437457a08509eccebd438a0) : [0, 149]: Setting download as failed: File segment: [0, 149], key: c0ea6f67f437457a08509eccebd438a0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:52.510423 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c0ea6f67f437457a08509eccebd438a0): No space left in cache to reserve 150 bytes, will continue without cache download 2024.10.22 01:08:52.666188 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 37795 2024.10.22 01:08:53.050523 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:53.054370 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.44 GiB. 2024.10.22 01:08:53.120923 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(202e1a13327f55ff582787f9207c887e) : [0, 184008]: Setting download as failed: File segment: [0, 184008], key: 202e1a13327f55ff582787f9207c887e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:53.122567 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(202e1a13327f55ff582787f9207c887e): No space left in cache to reserve 184009 bytes, will continue without cache download 2024.10.22 01:08:53.151541 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 202e1a13327f55ff582787f9207c887e, offset: 0, size: 0 2024.10.22 01:08:53.153196 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 202e1a13327f55ff582787f9207c887e for removal 2024.10.22 01:08:53.154694 [ 1476 ] {} CacheMetadata: Key 202e1a13327f55ff582787f9207c887e is removed from metadata 2024.10.22 01:08:53.155440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c0ea6f67f437457a08509eccebd438a0, offset: 0, size: 0 2024.10.22 01:08:53.157181 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c0ea6f67f437457a08509eccebd438a0 for removal 2024.10.22 01:08:53.159581 [ 1476 ] {} CacheMetadata: Key c0ea6f67f437457a08509eccebd438a0 is removed from metadata 2024.10.22 01:08:53.309629 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_620_620_0 (state Temporary) 2024.10.22 01:08:53.387030 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_620_620_0 to 202410_633_633_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:53.408127 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_628_251 to 202410_629_629_0 2024.10.22 01:08:53.412794 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.35 MiB using storage policy from min volume index 0 2024.10.22 01:08:53.419526 [ 1250 ] {} DiskLocal: Reserved 1.35 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:53.437522 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37795 2024.10.22 01:08:53.459452 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_629_252} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_628_251 to 202410_629_629_0 into Compact with storage Full 2024.10.22 01:08:53.503327 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_629_252} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:53.506948 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_629_252} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_628_251, total 38219 rows starting from the beginning of the part 2024.10.22 01:08:53.603109 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 203 entries to flush up to offset 32008 2024.10.22 01:08:53.606578 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1175 entries to flush up to offset 568112 2024.10.22 01:08:53.619554 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_629_252} MergeTreeSequentialSource: Reading 2 marks from part 202410_629_629_0, total 71 rows starting from the beginning of the part 2024.10.22 01:08:53.658401 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 312 2024.10.22 01:08:53.823492 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:53.831891 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:53.952442 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_294_294_0 (state Temporary) 2024.10.22 01:08:54.160093 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7ffd2aa1df65909e827503b9aa0b9402) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 7ffd2aa1df65909e827503b9aa0b9402, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:54.168259 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7ffd2aa1df65909e827503b9aa0b9402): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:08:54.274783 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:54.278890 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:54.339390 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_294_294_0 to 202410_304_304_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:54.532258 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:54.540844 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:54.598855 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_349_349_0 (state Temporary) 2024.10.22 01:08:54.611415 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 312 2024.10.22 01:08:54.711342 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_360_360_0 (state Temporary) 2024.10.22 01:08:54.757127 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_349_349_0 to 202410_356_356_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:54.757581 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_291_58, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0, 202410_296_296_0] 2024.10.22 01:08:54.764227 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_291_58, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0, 202410_296_296_0] 2024.10.22 01:08:54.872387 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_291_58, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0, 202410_296_296_0] 2024.10.22 01:08:54.995141 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 568112 2024.10.22 01:08:55.074783 [ 1252 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_351_70 to 202410_356_356_0 2024.10.22 01:08:55.092706 [ 1252 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:55.095970 [ 1252 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:55.103013 [ 1252 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 01:08:55.211843 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0691f890afff97563153409f5376a7b9) : [0, 126918]: Setting download as failed: File segment: [0, 126918], key: 0691f890afff97563153409f5376a7b9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:55.215130 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0691f890afff97563153409f5376a7b9): No space left in cache to reserve 126919 bytes, will continue without cache download 2024.10.22 01:08:55.279396 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_351_70 to 202410_356_356_0 into Compact with storage Full 2024.10.22 01:08:55.314062 [ 1164 ] {} DatabaseCatalog: Execution took 260 ms. 2024.10.22 01:08:55.320816 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:55.337644 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} MergeTreeSequentialSource: Reading 72 marks from part 202410_1_351_70, total 575558 rows starting from the beginning of the part 2024.10.22 01:08:55.354669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0691f890afff97563153409f5376a7b9, offset: 0, size: 0 2024.10.22 01:08:55.357139 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0691f890afff97563153409f5376a7b9 for removal 2024.10.22 01:08:55.464203 [ 1476 ] {} CacheMetadata: Key 0691f890afff97563153409f5376a7b9 is removed from metadata 2024.10.22 01:08:55.464978 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ffd2aa1df65909e827503b9aa0b9402, offset: 0, size: 0 2024.10.22 01:08:55.469480 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ffd2aa1df65909e827503b9aa0b9402 for removal 2024.10.22 01:08:55.494047 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_352_352_0, total 768 rows starting from the beginning of the part 2024.10.22 01:08:55.576376 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_360_360_0 to 202410_366_366_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:55.581907 [ 1476 ] {} CacheMetadata: Key 7ffd2aa1df65909e827503b9aa0b9402 is removed from metadata 2024.10.22 01:08:55.600237 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_353_353_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:08:55.635572 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_354_354_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:08:55.679728 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_355_355_0, total 1129 rows starting from the beginning of the part 2024.10.22 01:08:55.688200 [ 1159 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_360_138 to 202410_361_361_0 2024.10.22 01:08:55.694601 [ 1159 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.64 MiB using storage policy from min volume index 0 2024.10.22 01:08:55.696646 [ 1159 ] {} DiskLocal: Reserved 1.64 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:55.705730 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 32008 2024.10.22 01:08:55.707672 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_361_139} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_360_138 to 202410_361_361_0 into Wide with storage Full 2024.10.22 01:08:55.710102 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_356_356_0, total 1175 rows starting from the beginning of the part 2024.10.22 01:08:55.746822 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_361_139} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:55.765637 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_361_139} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_360_138, total 30798 rows starting from the beginning of the part 2024.10.22 01:08:55.855061 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 2954 2024.10.22 01:08:55.910158 [ 1178 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 207 ms. 2024.10.22 01:08:56.116042 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_361_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_361_361_0, total 224 rows starting from the beginning of the part 2024.10.22 01:08:56.137048 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_629_252} MergeTask::MergeProjectionsStage: Merge sorted 38290 rows, containing 13 columns (13 merged, 0 gathered) in 2.684704294 sec., 14262.278376644188 rows/sec., 2.24 MiB/sec. 2024.10.22 01:08:56.256377 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_629_252} MergedBlockOutputStream: filled checksums 202410_1_629_252 (state Temporary) 2024.10.22 01:08:56.348919 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_629_252} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_629_252 to 202410_1_629_252 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:56.355824 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_629_252} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_628_251, 202410_629_629_0] -> 202410_1_629_252 2024.10.22 01:08:56.430473 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6acb98ba56839952924ff54bc42e2d98) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 6acb98ba56839952924ff54bc42e2d98, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:56.432678 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6acb98ba56839952924ff54bc42e2d98): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:08:56.475979 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_303_93, 202410_304_304_0] 2024.10.22 01:08:56.478359 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_303_93, 202410_304_304_0] 2024.10.22 01:08:56.485798 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:08:56.488584 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_303_93, 202410_304_304_0] 2024.10.22 01:08:56.612358 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.89 GiB, free memory in arenas 9.58 MiB, will set to 1.68 GiB (RSS), difference: -49.92 MiB 2024.10.22 01:08:57.212067 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a6df164e91e82b75b2abd74226afaec1) : [0, 60099]: Setting download as failed: File segment: [0, 60099], key: a6df164e91e82b75b2abd74226afaec1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:57.214775 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a6df164e91e82b75b2abd74226afaec1): No space left in cache to reserve 60100 bytes, will continue without cache download 2024.10.22 01:08:57.238978 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a6df164e91e82b75b2abd74226afaec1, offset: 0, size: 0 2024.10.22 01:08:57.241400 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a6df164e91e82b75b2abd74226afaec1 for removal 2024.10.22 01:08:57.245634 [ 1476 ] {} CacheMetadata: Key a6df164e91e82b75b2abd74226afaec1 is removed from metadata 2024.10.22 01:08:57.246818 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6acb98ba56839952924ff54bc42e2d98, offset: 0, size: 0 2024.10.22 01:08:57.249762 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6acb98ba56839952924ff54bc42e2d98 for removal 2024.10.22 01:08:57.277320 [ 1476 ] {} CacheMetadata: Key 6acb98ba56839952924ff54bc42e2d98 is removed from metadata 2024.10.22 01:08:57.546262 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:08:57.548941 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 37888 2024.10.22 01:08:57.653839 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 5433 2024.10.22 01:08:57.803829 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:57.808272 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:57.885235 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:08:57.888156 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:57.952786 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_365_365_0 (state Temporary) 2024.10.22 01:08:57.954099 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_621_621_0 (state Temporary) 2024.10.22 01:08:57.997707 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_365_365_0 to 202410_378_378_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:58.098592 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_621_621_0 to 202410_634_634_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:58.127025 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_629_252 to 202410_630_630_0 2024.10.22 01:08:58.138834 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.35 MiB using storage policy from min volume index 0 2024.10.22 01:08:58.142381 [ 1185 ] {} DiskLocal: Reserved 1.35 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:08:58.147732 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5433 2024.10.22 01:08:58.177728 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37888 2024.10.22 01:08:58.231655 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_361_139} MergeTask::MergeProjectionsStage: Merge sorted 31022 rows, containing 13 columns (13 merged, 0 gathered) in 2.5291635 sec., 12265.715522147935 rows/sec., 5.26 MiB/sec. 2024.10.22 01:08:58.244504 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8ae9e1250e408e1996a2686309595db1) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 8ae9e1250e408e1996a2686309595db1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:58.249127 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8ae9e1250e408e1996a2686309595db1): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:08:58.267581 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_630_253} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_629_252 to 202410_630_630_0 into Compact with storage Full 2024.10.22 01:08:58.307520 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_361_139} MergedBlockOutputStream: filled checksums 202410_1_361_139 (state Temporary) 2024.10.22 01:08:58.340311 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 10.72 MiB, will set to 1.69 GiB (RSS), difference: 7.21 MiB 2024.10.22 01:08:58.386063 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_630_253} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:08:58.607421 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_630_253} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_629_252, total 38290 rows starting from the beginning of the part 2024.10.22 01:08:58.629578 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_356_71/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:08:58.797799 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_630_253} MergeTreeSequentialSource: Reading 2 marks from part 202410_630_630_0, total 79 rows starting from the beginning of the part 2024.10.22 01:08:59.116534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(055eeb5be7fc59003c3808b58ffef1d6) : [0, 98836]: Setting download as failed: File segment: [0, 98836], key: 055eeb5be7fc59003c3808b58ffef1d6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:59.123391 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(055eeb5be7fc59003c3808b58ffef1d6): No space left in cache to reserve 98837 bytes, will continue without cache download 2024.10.22 01:08:59.258573 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 055eeb5be7fc59003c3808b58ffef1d6, offset: 0, size: 0 2024.10.22 01:08:59.262014 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 055eeb5be7fc59003c3808b58ffef1d6 for removal 2024.10.22 01:08:59.267768 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_361_139} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_361_139 to 202410_1_361_139 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:08:59.268363 [ 1476 ] {} CacheMetadata: Key 055eeb5be7fc59003c3808b58ffef1d6 is removed from metadata 2024.10.22 01:08:59.268864 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8ae9e1250e408e1996a2686309595db1, offset: 0, size: 0 2024.10.22 01:08:59.270752 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8ae9e1250e408e1996a2686309595db1 for removal 2024.10.22 01:08:59.272017 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_361_139} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_360_138, 202410_361_361_0] -> 202410_1_361_139 2024.10.22 01:08:59.279387 [ 1476 ] {} CacheMetadata: Key 8ae9e1250e408e1996a2686309595db1 is removed from metadata 2024.10.22 01:08:59.348286 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.41 MiB. 2024.10.22 01:08:59.893608 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:08:59.895497 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:08:59.947084 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(95fb3697aad1ed454c853d1855613e01) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 95fb3697aad1ed454c853d1855613e01, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:08:59.952061 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(95fb3697aad1ed454c853d1855613e01): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:09:00.335578 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_74_15/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:09:00.656678 [ 1204 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:09:00.790389 [ 1204 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:09:00.808705 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(87293d205fc13b56834de3aa7e4e2342) : [0, 131681]: Setting download as failed: File segment: [0, 131681], key: 87293d205fc13b56834de3aa7e4e2342, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:00.810034 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(87293d205fc13b56834de3aa7e4e2342): No space left in cache to reserve 131682 bytes, will continue without cache download 2024.10.22 01:09:00.827909 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_630_253} MergeTask::MergeProjectionsStage: Merge sorted 38369 rows, containing 13 columns (13 merged, 0 gathered) in 2.570303661 sec., 14927.808173868527 rows/sec., 2.34 MiB/sec. 2024.10.22 01:09:00.948228 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 87293d205fc13b56834de3aa7e4e2342, offset: 0, size: 0 2024.10.22 01:09:00.952380 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 87293d205fc13b56834de3aa7e4e2342 for removal 2024.10.22 01:09:00.984267 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_630_253} MergedBlockOutputStream: filled checksums 202410_1_630_253 (state Temporary) 2024.10.22 01:09:01.060197 [ 1476 ] {} CacheMetadata: Key 87293d205fc13b56834de3aa7e4e2342 is removed from metadata 2024.10.22 01:09:01.061324 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 95fb3697aad1ed454c853d1855613e01, offset: 0, size: 0 2024.10.22 01:09:01.064174 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 95fb3697aad1ed454c853d1855613e01 for removal 2024.10.22 01:09:01.066995 [ 1476 ] {} CacheMetadata: Key 95fb3697aad1ed454c853d1855613e01 is removed from metadata 2024.10.22 01:09:01.407730 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_630_253} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_630_253 to 202410_1_630_253 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:01.510989 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_630_253} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_629_252, 202410_630_630_0] -> 202410_1_630_253 2024.10.22 01:09:01.546834 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:09:01.526987 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.89 GiB, free memory in arenas 12.12 MiB, will set to 1.69 GiB (RSS), difference: 36.64 MiB 2024.10.22 01:09:01.603637 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a23949621e6307f9093ef31c58d7053e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a23949621e6307f9093ef31c58d7053e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:09:01.607714 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: a23949621e6307f9093ef31c58d7053e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:09:01.610146 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:01.998533 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 569264 2024.10.22 01:09:02.113070 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 313 2024.10.22 01:09:02.196126 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 37951 2024.10.22 01:09:02.318899 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:02.325363 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:02.484985 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_622_622_0 (state Temporary) 2024.10.22 01:09:02.486714 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:02.489785 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:02.550877 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_295_295_0 (state Temporary) 2024.10.22 01:09:02.628315 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7dc5e87c981d544a554d8d50b484bae3) : [0, 163]: Setting download as failed: File segment: [0, 163], key: 7dc5e87c981d544a554d8d50b484bae3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:02.633595 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7dc5e87c981d544a554d8d50b484bae3): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 01:09:02.679845 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:02.683304 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:02.771427 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_622_622_0 to 202410_635_635_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:02.787393 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_295_295_0 to 202410_305_305_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:02.804173 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_350_350_0 (state Temporary) 2024.10.22 01:09:02.899610 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_630_253 to 202410_631_631_0 2024.10.22 01:09:02.916276 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_350_350_0 to 202410_357_357_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:02.934115 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.36 MiB using storage policy from min volume index 0 2024.10.22 01:09:02.943942 [ 1271 ] {} DiskLocal: Reserved 1.36 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:02.986751 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 569264 2024.10.22 01:09:03.003815 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 313 2024.10.22 01:09:03.095516 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_631_254} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_630_253 to 202410_631_631_0 into Compact with storage Full 2024.10.22 01:09:03.169837 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_631_254} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:03.194208 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_631_254} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_630_253, total 38369 rows starting from the beginning of the part 2024.10.22 01:09:03.217163 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 37951 2024.10.22 01:09:03.243854 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_631_254} MergeTreeSequentialSource: Reading 2 marks from part 202410_631_631_0, total 82 rows starting from the beginning of the part 2024.10.22 01:09:03.354869 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 229 entries to flush up to offset 32237 2024.10.22 01:09:03.624649 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1ee47a6119019a2b2988465923639e51) : [0, 142257]: Setting download as failed: File segment: [0, 142257], key: 1ee47a6119019a2b2988465923639e51, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:03.632405 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1ee47a6119019a2b2988465923639e51): No space left in cache to reserve 142258 bytes, will continue without cache download 2024.10.22 01:09:03.754504 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1ee47a6119019a2b2988465923639e51, offset: 0, size: 0 2024.10.22 01:09:03.756471 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1ee47a6119019a2b2988465923639e51 for removal 2024.10.22 01:09:03.758630 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7dc5e87c981d544a554d8d50b484bae3, offset: 0, size: 0 2024.10.22 01:09:03.759451 [ 1476 ] {} CacheMetadata: Key 1ee47a6119019a2b2988465923639e51 is removed from metadata 2024.10.22 01:09:03.759548 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7dc5e87c981d544a554d8d50b484bae3 for removal 2024.10.22 01:09:03.761454 [ 1476 ] {} CacheMetadata: Key 7dc5e87c981d544a554d8d50b484bae3 is removed from metadata 2024.10.22 01:09:03.895116 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:03.898356 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:03.944254 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_361_361_0 (state Temporary) 2024.10.22 01:09:04.146869 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_361_361_0 to 202410_367_367_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:04.299650 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 32237 2024.10.22 01:09:04.395167 [ 1262 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_361_139 to 202410_362_362_0 2024.10.22 01:09:04.399143 [ 1262 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.65 MiB using storage policy from min volume index 0 2024.10.22 01:09:04.403514 [ 1262 ] {} DiskLocal: Reserved 1.65 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:04.407597 [ 1262 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 247 ms. 2024.10.22 01:09:04.414845 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_362_140} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_361_139 to 202410_362_362_0 into Wide with storage Full 2024.10.22 01:09:04.452049 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_362_140} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:04.475076 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_362_140} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_361_139, total 31022 rows starting from the beginning of the part 2024.10.22 01:09:04.507739 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 14.32 MiB, will set to 1.70 GiB (RSS), difference: 1.48 MiB 2024.10.22 01:09:04.521782 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_631_254} MergeTask::MergeProjectionsStage: Merge sorted 38451 rows, containing 13 columns (13 merged, 0 gathered) in 1.531199605 sec., 25111.683593988386 rows/sec., 3.94 MiB/sec. 2024.10.22 01:09:04.529538 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(09fc569b5f4319b60040cde830b470a9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 09fc569b5f4319b60040cde830b470a9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:09:04.579133 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_304_94, 202410_305_305_0] 2024.10.22 01:09:04.581269 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_304_94, 202410_305_305_0] 2024.10.22 01:09:04.588496 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_304_94, 202410_305_305_0] 2024.10.22 01:09:04.640236 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 763, file segment info: File segment: [0, 33554431], key: 09fc569b5f4319b60040cde830b470a9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:09:04.653938 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:04.836303 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_362_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_362_362_0, total 226 rows starting from the beginning of the part 2024.10.22 01:09:04.937810 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_631_254} MergedBlockOutputStream: filled checksums 202410_1_631_254 (state Temporary) 2024.10.22 01:09:05.012407 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:09:05.138295 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:09:05.149660 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:09:05.169476 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:09:05.180567 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6da0cd3f4d788af0c45c0049a8cc0a40) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 6da0cd3f4d788af0c45c0049a8cc0a40, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:05.183180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6da0cd3f4d788af0c45c0049a8cc0a40): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:09:05.314665 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:09:05.338181 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:09:05.348280 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:09:05.381986 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:09:05.412822 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:09:05.508353 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_631_254} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_631_254 to 202410_1_631_254 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:05.511075 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_631_254} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_630_253, 202410_631_631_0] -> 202410_1_631_254 2024.10.22 01:09:05.535665 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:09:05.613100 [ 1217 ] {} DatabaseCatalog: Execution took 298 ms. 2024.10.22 01:09:05.615783 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hkj/ddduvuiyzcbktmsizgpppftnlunzx (LogSeriesLimiter: on interval from 2024-10-22 01:08:45 to 2024-10-22 01:09:05 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:09:05.627743 [ 1208 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 358 ms. 2024.10.22 01:09:05.651129 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 5448 2024.10.22 01:09:05.976178 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:05.983056 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:06.015506 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_366_366_0 (state Temporary) 2024.10.22 01:09:06.343242 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(475986ccd6f6d65bf6afb18f265d4da2) : [0, 76561]: Setting download as failed: File segment: [0, 76561], key: 475986ccd6f6d65bf6afb18f265d4da2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:06.345606 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(475986ccd6f6d65bf6afb18f265d4da2): No space left in cache to reserve 76562 bytes, will continue without cache download 2024.10.22 01:09:06.370120 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 475986ccd6f6d65bf6afb18f265d4da2, offset: 0, size: 0 2024.10.22 01:09:06.372450 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 475986ccd6f6d65bf6afb18f265d4da2 for removal 2024.10.22 01:09:06.378375 [ 1476 ] {} CacheMetadata: Key 475986ccd6f6d65bf6afb18f265d4da2 is removed from metadata 2024.10.22 01:09:06.380984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6da0cd3f4d788af0c45c0049a8cc0a40, offset: 0, size: 0 2024.10.22 01:09:06.384828 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6da0cd3f4d788af0c45c0049a8cc0a40 for removal 2024.10.22 01:09:06.388190 [ 1476 ] {} CacheMetadata: Key 6da0cd3f4d788af0c45c0049a8cc0a40 is removed from metadata 2024.10.22 01:09:06.392331 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_366_366_0 to 202410_379_379_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:06.444989 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5448 2024.10.22 01:09:06.501012 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_362_140/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:09:06.939964 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_362_140} MergeTask::MergeProjectionsStage: Merge sorted 31248 rows, containing 13 columns (13 merged, 0 gathered) in 2.529379684 sec., 12354.017151977727 rows/sec., 5.30 MiB/sec. 2024.10.22 01:09:07.228394 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 38040 2024.10.22 01:09:07.229660 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_362_140} MergedBlockOutputStream: filled checksums 202410_1_362_140 (state Temporary) 2024.10.22 01:09:07.308355 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3ab4ee00d2781e3dcdcc1b3c161e4e6d) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 3ab4ee00d2781e3dcdcc1b3c161e4e6d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:07.309448 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3ab4ee00d2781e3dcdcc1b3c161e4e6d): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:09:07.692071 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:07.696354 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:07.750841 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.89 GiB, free memory in arenas 11.19 MiB, will set to 1.69 GiB (RSS), difference: -44.53 MiB 2024.10.22 01:09:07.823539 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_623_623_0 (state Temporary) 2024.10.22 01:09:07.895404 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_623_623_0 to 202410_636_636_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:07.918071 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_631_254 to 202410_632_632_0 2024.10.22 01:09:07.949225 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38040 2024.10.22 01:09:08.035734 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.36 MiB using storage policy from min volume index 0 2024.10.22 01:09:08.040473 [ 1235 ] {} DiskLocal: Reserved 1.36 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:08.064448 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_632_255} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_631_254 to 202410_632_632_0 into Compact with storage Full 2024.10.22 01:09:08.141326 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_632_255} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:08.149720 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_632_255} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_631_254, total 38451 rows starting from the beginning of the part 2024.10.22 01:09:08.201829 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_632_255} MergeTreeSequentialSource: Reading 2 marks from part 202410_632_632_0, total 108 rows starting from the beginning of the part 2024.10.22 01:09:08.216282 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9f492a99f08c82c4994bf29dc0ef2d96) : [0, 1060915]: Setting download as failed: File segment: [0, 1060915], key: 9f492a99f08c82c4994bf29dc0ef2d96, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:08.222559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9f492a99f08c82c4994bf29dc0ef2d96): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:09:08.448763 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_530_157, 202410_531_531_0] 2024.10.22 01:09:08.451129 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_530_157, 202410_531_531_0] 2024.10.22 01:09:08.497237 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} MergeTask::MergeProjectionsStage: Merge sorted 2893 rows, containing 679 columns (679 merged, 0 gathered) in 104.606488919 sec., 27.656028128810807 rows/sec., 36.77 KiB/sec. 2024.10.22 01:09:08.577054 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9f492a99f08c82c4994bf29dc0ef2d96, offset: 0, size: 0 2024.10.22 01:09:08.581082 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9f492a99f08c82c4994bf29dc0ef2d96 for removal 2024.10.22 01:09:08.585018 [ 1476 ] {} CacheMetadata: Key 9f492a99f08c82c4994bf29dc0ef2d96 is removed from metadata 2024.10.22 01:09:08.587403 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3ab4ee00d2781e3dcdcc1b3c161e4e6d, offset: 0, size: 0 2024.10.22 01:09:08.588171 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_530_157, 202410_531_531_0] 2024.10.22 01:09:08.588518 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3ab4ee00d2781e3dcdcc1b3c161e4e6d for removal 2024.10.22 01:09:08.590297 [ 1476 ] {} CacheMetadata: Key 3ab4ee00d2781e3dcdcc1b3c161e4e6d is removed from metadata 2024.10.22 01:09:08.777228 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_362_140} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_362_140 to 202410_1_362_140 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:08.795322 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_362_140} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_361_139, 202410_362_362_0] -> 202410_1_362_140 2024.10.22 01:09:08.823200 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.42 MiB. 2024.10.22 01:09:08.924666 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(227e4cf47a389049f1db6ad9298c3628) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 227e4cf47a389049f1db6ad9298c3628, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:09:08.930191 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 334564, file segment info: File segment: [0, 33554431], key: 227e4cf47a389049f1db6ad9298c3628, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:09:08.935850 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:09.904017 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:09:09.905199 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:09:09.988752 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0c1b17797aeb3d65502367ad772c1171) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 0c1b17797aeb3d65502367ad772c1171, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:09.990296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0c1b17797aeb3d65502367ad772c1171): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:09:09.990507 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 570416 2024.10.22 01:09:10.104140 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_356_71/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:09:10.287180 [ 1228 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 01:09:10.483042 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(eeb8ec1c385f28e5d1c46fd6f0cdcbe6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: eeb8ec1c385f28e5d1c46fd6f0cdcbe6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:09:10.504400 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} MergedBlockOutputStream: filled checksums 202410_1_74_15 (state Temporary) 2024.10.22 01:09:10.505549 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 644, file segment info: File segment: [0, 33554431], key: eeb8ec1c385f28e5d1c46fd6f0cdcbe6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:09:10.506626 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 314 2024.10.22 01:09:10.508626 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:10.747952 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.89 GiB, free memory in arenas 14.71 MiB, will set to 1.69 GiB (RSS), difference: 51.23 MiB 2024.10.22 01:09:10.810309 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_632_255} MergeTask::MergeProjectionsStage: Merge sorted 38559 rows, containing 13 columns (13 merged, 0 gathered) in 2.759702293 sec., 13972.159278848705 rows/sec., 2.19 MiB/sec. 2024.10.22 01:09:10.814862 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:10.886089 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:10.894542 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:10.919169 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:10.925728 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_531_158, 202410_532_532_0] 2024.10.22 01:09:10.927626 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_531_158, 202410_532_532_0] 2024.10.22 01:09:10.936864 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_531_158, 202410_532_532_0] 2024.10.22 01:09:11.003107 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_632_255} MergedBlockOutputStream: filled checksums 202410_1_632_255 (state Temporary) 2024.10.22 01:09:11.027174 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:11.030362 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:11.060047 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_296_296_0 (state Temporary) 2024.10.22 01:09:11.202941 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_632_255} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_632_255 to 202410_1_632_255 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:11.204951 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_632_255} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_631_254, 202410_632_632_0] -> 202410_1_632_255 2024.10.22 01:09:11.206331 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(cb1a234335f09bae9e2c589cbdbe175a) : [0, 47829]: Setting download as failed: File segment: [0, 47829], key: cb1a234335f09bae9e2c589cbdbe175a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:11.207346 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(cb1a234335f09bae9e2c589cbdbe175a): No space left in cache to reserve 47830 bytes, will continue without cache download 2024.10.22 01:09:11.221740 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cb1a234335f09bae9e2c589cbdbe175a, offset: 0, size: 0 2024.10.22 01:09:11.224995 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cb1a234335f09bae9e2c589cbdbe175a for removal 2024.10.22 01:09:11.230145 [ 1476 ] {} CacheMetadata: Key cb1a234335f09bae9e2c589cbdbe175a is removed from metadata 2024.10.22 01:09:11.326775 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:09:11.333650 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0c1b17797aeb3d65502367ad772c1171, offset: 0, size: 0 2024.10.22 01:09:11.338197 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0c1b17797aeb3d65502367ad772c1171 for removal 2024.10.22 01:09:11.341790 [ 1476 ] {} CacheMetadata: Key 0c1b17797aeb3d65502367ad772c1171 is removed from metadata 2024.10.22 01:09:11.383828 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_296_296_0 to 202410_306_306_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:11.410474 [ 1151 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_301_62 to 202410_306_306_0 2024.10.22 01:09:11.423887 [ 1151 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:11.428230 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:11.451225 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_301_62 to 202410_306_306_0 into Compact with storage Full 2024.10.22 01:09:11.501079 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_351_351_0 (state Temporary) 2024.10.22 01:09:11.578958 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 314 2024.10.22 01:09:11.675442 [ 1246 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 01:09:11.726510 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:11.743265 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_301_62, total 345 rows starting from the beginning of the part 2024.10.22 01:09:11.783000 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_351_351_0 to 202410_358_358_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:11.806325 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 570416 2024.10.22 01:09:11.818270 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(45e0249c7ddd66652b5decdc0e565445) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 45e0249c7ddd66652b5decdc0e565445, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:09:11.820684 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 852893, file segment info: File segment: [0, 33554431], key: 45e0249c7ddd66652b5decdc0e565445, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:09:11.824605 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:11.934665 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_302_302_0, total 1 rows starting from the beginning of the part 2024.10.22 01:09:11.969735 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 38127 2024.10.22 01:09:11.986220 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 223 entries to flush up to offset 32460 2024.10.22 01:09:12.032666 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_303_303_0, total 1 rows starting from the beginning of the part 2024.10.22 01:09:12.092543 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_305_95, 202410_306_306_0] 2024.10.22 01:09:12.094743 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_305_95, 202410_306_306_0] 2024.10.22 01:09:12.107477 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_305_95, 202410_306_306_0] 2024.10.22 01:09:12.293044 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:12.295102 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:12.351126 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_304_304_0, total 1 rows starting from the beginning of the part 2024.10.22 01:09:12.399674 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_624_624_0 (state Temporary) 2024.10.22 01:09:12.411508 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_305_305_0, total 1 rows starting from the beginning of the part 2024.10.22 01:09:12.447242 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_624_624_0 to 202410_637_637_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:12.555532 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:12.560094 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:12.589824 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_632_255 to 202410_633_633_0 2024.10.22 01:09:12.597035 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.36 MiB using storage policy from min volume index 0 2024.10.22 01:09:12.600189 [ 1268 ] {} DiskLocal: Reserved 1.36 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:12.620877 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_362_362_0 (state Temporary) 2024.10.22 01:09:12.624255 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_633_256} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_632_255 to 202410_633_633_0 into Compact with storage Full 2024.10.22 01:09:12.687417 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_306_306_0, total 1 rows starting from the beginning of the part 2024.10.22 01:09:12.688654 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_633_256} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:12.698609 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_633_256} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_632_255, total 38559 rows starting from the beginning of the part 2024.10.22 01:09:12.700406 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38127 2024.10.22 01:09:12.703465 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_362_362_0 to 202410_368_368_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:12.741636 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_633_256} MergeTreeSequentialSource: Reading 2 marks from part 202410_633_633_0, total 67 rows starting from the beginning of the part 2024.10.22 01:09:12.754429 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 32460 2024.10.22 01:09:12.758364 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(aa271c8dbbf0731074dc76f9dbb69c9d) : [0, 151]: Setting download as failed: File segment: [0, 151], key: aa271c8dbbf0731074dc76f9dbb69c9d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:12.759953 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(aa271c8dbbf0731074dc76f9dbb69c9d): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:09:12.843245 [ 1264 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_362_140 to 202410_363_363_0 2024.10.22 01:09:12.854786 [ 1264 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.66 MiB using storage policy from min volume index 0 2024.10.22 01:09:12.859216 [ 1264 ] {} DiskLocal: Reserved 1.66 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:12.884909 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_363_141} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_362_140 to 202410_363_363_0 into Wide with storage Full 2024.10.22 01:09:13.086947 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_363_141} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:13.117378 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_363_141} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_362_140, total 31248 rows starting from the beginning of the part 2024.10.22 01:09:13.442586 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(4fadfb920f723f9a3cfba6f5c5ec8948) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4fadfb920f723f9a3cfba6f5c5ec8948, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:09:13.448802 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 815, file segment info: File segment: [0, 33554431], key: 4fadfb920f723f9a3cfba6f5c5ec8948, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:09:13.453482 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:13.505671 [ 1682 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kgd/ivarfoumosmdfhdwuqxvpunefsmwu, size 852893 2024.10.22 01:09:13.631763 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_363_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_363_363_0, total 204 rows starting from the beginning of the part 2024.10.22 01:09:13.695706 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_74_15/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:09:13.725469 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_74_74_0 (state Temporary) 2024.10.22 01:09:13.948764 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 5460 2024.10.22 01:09:14.021361 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:09:14.164644 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:09:14.200790 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:09:14.238110 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3e7b471cfa8ba91e441393683aad3c6f) : [0, 41993]: Setting download as failed: File segment: [0, 41993], key: 3e7b471cfa8ba91e441393683aad3c6f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:14.240594 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3e7b471cfa8ba91e441393683aad3c6f): No space left in cache to reserve 41994 bytes, will continue without cache download 2024.10.22 01:09:14.260260 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:14.264754 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:14.317702 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_367_367_0 (state Temporary) 2024.10.22 01:09:14.241353 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:09:14.400009 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_367_367_0 to 202410_380_380_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:14.377720 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3e7b471cfa8ba91e441393683aad3c6f, offset: 0, size: 0 2024.10.22 01:09:14.482460 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3e7b471cfa8ba91e441393683aad3c6f for removal 2024.10.22 01:09:14.482630 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:09:14.485391 [ 1476 ] {} CacheMetadata: Key 3e7b471cfa8ba91e441393683aad3c6f is removed from metadata 2024.10.22 01:09:14.485802 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aa271c8dbbf0731074dc76f9dbb69c9d, offset: 0, size: 0 2024.10.22 01:09:14.487489 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aa271c8dbbf0731074dc76f9dbb69c9d for removal 2024.10.22 01:09:14.489313 [ 1476 ] {} CacheMetadata: Key aa271c8dbbf0731074dc76f9dbb69c9d is removed from metadata 2024.10.22 01:09:14.508084 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:09:14.510213 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5460 2024.10.22 01:09:14.554885 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:09:14.638459 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nqn/sfhmcajghpkxahmlbixcjubfxsyyw (LogSeriesLimiter: on interval from 2024-10-22 01:09:05 to 2024-10-22 01:09:14 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:09:14.659030 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} MergeTask::MergeProjectionsStage: Merge sorted 581702 rows, containing 4 columns (4 merged, 0 gathered) in 19.518770482 sec., 29802.184545201722 rows/sec., 436.86 KiB/sec. 2024.10.22 01:09:14.677446 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.89 GiB, free memory in arenas 14.39 MiB, will set to 1.69 GiB (RSS), difference: -68.63 MiB 2024.10.22 01:09:14.815041 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} MergeTask::MergeProjectionsStage: Merge sorted 350 rows, containing 32 columns (32 merged, 0 gathered) in 3.267888694 sec., 107.1027910597496 rows/sec., 10.91 KiB/sec. 2024.10.22 01:09:14.927001 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} MergedBlockOutputStream: filled checksums 202410_1_356_71 (state Temporary) 2024.10.22 01:09:14.963931 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} MergedBlockOutputStream: filled checksums 202410_1_306_63 (state Temporary) 2024.10.22 01:09:15.083306 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_356_71 to 202410_1_356_71 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:15.088342 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_356_71} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_351_70, 202410_356_356_0] -> 202410_1_356_71 2024.10.22 01:09:15.109409 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_306_63/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:09:15.135313 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_253_52, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0, 202410_257_257_0, 202410_258_258_0] 2024.10.22 01:09:15.140120 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_253_52, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0, 202410_257_257_0, 202410_258_258_0] 2024.10.22 01:09:15.195604 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_633_256} MergeTask::MergeProjectionsStage: Merge sorted 38626 rows, containing 13 columns (13 merged, 0 gathered) in 2.578795834 sec., 14978.308670557593 rows/sec., 2.35 MiB/sec. 2024.10.22 01:09:15.248286 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:09:15.260694 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_633_256} MergedBlockOutputStream: filled checksums 202410_1_633_256 (state Temporary) 2024.10.22 01:09:15.266355 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_253_52, 202410_254_254_0, 202410_255_255_0, 202410_256_256_0, 202410_257_257_0, 202410_258_258_0] 2024.10.22 01:09:15.281122 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(acac8ff4495a87898e1838c0996efd4c) : [0, 147]: Setting download as failed: File segment: [0, 147], key: acac8ff4495a87898e1838c0996efd4c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:15.283696 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(acac8ff4495a87898e1838c0996efd4c): No space left in cache to reserve 148 bytes, will continue without cache download 2024.10.22 01:09:15.324061 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_633_256} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_633_256 to 202410_1_633_256 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:15.332015 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_633_256} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_632_255, 202410_633_633_0] -> 202410_1_633_256 2024.10.22 01:09:15.487378 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:09:15.598937 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_306_63 to 202410_1_306_63 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:15.604161 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_306_63} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_301_62, 202410_306_306_0] -> 202410_1_306_63 2024.10.22 01:09:15.707261 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:09:15.838737 [ 1168 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:09:15.899546 [ 1156 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:09:15.916788 [ 1156 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:09:15.938927 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_363_141} MergeTask::MergeProjectionsStage: Merge sorted 31452 rows, containing 13 columns (13 merged, 0 gathered) in 3.060574699 sec., 10276.501341488742 rows/sec., 4.41 MiB/sec. 2024.10.22 01:09:16.051299 [ 1178 ] {} DatabaseCatalog: Execution took 434 ms. 2024.10.22 01:09:16.102480 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_363_141} MergedBlockOutputStream: filled checksums 202410_1_363_141 (state Temporary) 2024.10.22 01:09:16.369670 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(377e788a1af6c2ea50937a3d38c3a469) : [0, 78595]: Setting download as failed: File segment: [0, 78595], key: 377e788a1af6c2ea50937a3d38c3a469, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:16.375611 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(377e788a1af6c2ea50937a3d38c3a469): No space left in cache to reserve 78596 bytes, will continue without cache download 2024.10.22 01:09:16.396453 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 377e788a1af6c2ea50937a3d38c3a469, offset: 0, size: 0 2024.10.22 01:09:16.398134 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 377e788a1af6c2ea50937a3d38c3a469 for removal 2024.10.22 01:09:16.400638 [ 1476 ] {} CacheMetadata: Key 377e788a1af6c2ea50937a3d38c3a469 is removed from metadata 2024.10.22 01:09:16.401741 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: acac8ff4495a87898e1838c0996efd4c, offset: 0, size: 0 2024.10.22 01:09:16.403647 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key acac8ff4495a87898e1838c0996efd4c for removal 2024.10.22 01:09:16.406413 [ 1476 ] {} CacheMetadata: Key acac8ff4495a87898e1838c0996efd4c is removed from metadata 2024.10.22 01:09:16.588919 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_363_141} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_363_141 to 202410_1_363_141 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:16.593730 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_363_141} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_362_140, 202410_363_363_0] -> 202410_1_363_141 2024.10.22 01:09:16.607454 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.42 MiB. 2024.10.22 01:09:16.810696 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 102 entries to flush up to offset 38229 2024.10.22 01:09:16.831229 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_532_159, 202410_533_533_0] 2024.10.22 01:09:16.832636 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_532_159, 202410_533_533_0] 2024.10.22 01:09:16.842246 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_532_159, 202410_533_533_0] 2024.10.22 01:09:16.996926 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:16.999684 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:17.085414 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_625_625_0 (state Temporary) 2024.10.22 01:09:17.135116 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_625_625_0 to 202410_638_638_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:17.176914 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_633_256 to 202410_634_634_0 2024.10.22 01:09:17.199078 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 01:09:17.201579 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38229 2024.10.22 01:09:17.204229 [ 1187 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:17.340805 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_634_257} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_633_256 to 202410_634_634_0 into Compact with storage Full 2024.10.22 01:09:17.361362 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7cfddf6c80a9f0f930c4f8a6b6cc53f7) : [0, 147]: Setting download as failed: File segment: [0, 147], key: 7cfddf6c80a9f0f930c4f8a6b6cc53f7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:17.364564 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7cfddf6c80a9f0f930c4f8a6b6cc53f7): No space left in cache to reserve 148 bytes, will continue without cache download 2024.10.22 01:09:17.384520 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_634_257} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:17.390939 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_634_257} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_633_256, total 38626 rows starting from the beginning of the part 2024.10.22 01:09:17.461061 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_634_257} MergeTreeSequentialSource: Reading 2 marks from part 202410_634_634_0, total 93 rows starting from the beginning of the part 2024.10.22 01:09:17.759489 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.89 GiB, free memory in arenas 6.06 MiB, will set to 1.67 GiB (RSS), difference: 46.00 MiB 2024.10.22 01:09:18.324171 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(52a985efa1a13ac507d9c8cec9b49eb4) : [0, 125155]: Setting download as failed: File segment: [0, 125155], key: 52a985efa1a13ac507d9c8cec9b49eb4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:18.327568 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(52a985efa1a13ac507d9c8cec9b49eb4): No space left in cache to reserve 125156 bytes, will continue without cache download 2024.10.22 01:09:18.358181 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 52a985efa1a13ac507d9c8cec9b49eb4, offset: 0, size: 0 2024.10.22 01:09:18.359933 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 52a985efa1a13ac507d9c8cec9b49eb4 for removal 2024.10.22 01:09:18.367330 [ 1476 ] {} CacheMetadata: Key 52a985efa1a13ac507d9c8cec9b49eb4 is removed from metadata 2024.10.22 01:09:18.372105 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7cfddf6c80a9f0f930c4f8a6b6cc53f7, offset: 0, size: 0 2024.10.22 01:09:18.378218 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7cfddf6c80a9f0f930c4f8a6b6cc53f7 for removal 2024.10.22 01:09:18.383692 [ 1476 ] {} CacheMetadata: Key 7cfddf6c80a9f0f930c4f8a6b6cc53f7 is removed from metadata 2024.10.22 01:09:18.836117 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 878 entries to flush up to offset 571294 2024.10.22 01:09:19.335167 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1a7a3b5ca82a85257e808d8dd807588f) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 1a7a3b5ca82a85257e808d8dd807588f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:19.336863 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1a7a3b5ca82a85257e808d8dd807588f): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:09:19.485634 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:19.487629 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:19.697736 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_352_352_0 (state Temporary) 2024.10.22 01:09:19.770560 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_634_257} MergeTask::MergeProjectionsStage: Merge sorted 38719 rows, containing 13 columns (13 merged, 0 gathered) in 2.540421248 sec., 15241.173104847217 rows/sec., 2.39 MiB/sec. 2024.10.22 01:09:19.904573 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:09:19.906671 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:09:20.007050 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_352_352_0 to 202410_359_359_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:20.072313 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_634_257} MergedBlockOutputStream: filled checksums 202410_1_634_257 (state Temporary) 2024.10.22 01:09:20.155861 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 571294 2024.10.22 01:09:20.208759 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_316_63, 202410_317_317_0, 202410_318_318_0, 202410_319_319_0, 202410_320_320_0, 202410_321_321_0] 2024.10.22 01:09:20.211027 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_316_63, 202410_317_317_0, 202410_318_318_0, 202410_319_319_0, 202410_320_320_0, 202410_321_321_0] 2024.10.22 01:09:20.287609 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 231 entries to flush up to offset 32691 2024.10.22 01:09:20.350250 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_634_257} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_634_257 to 202410_1_634_257 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:20.353198 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_316_63, 202410_317_317_0, 202410_318_318_0, 202410_319_319_0, 202410_320_320_0, 202410_321_321_0] 2024.10.22 01:09:20.356264 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_634_257} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_633_256, 202410_634_634_0] -> 202410_1_634_257 2024.10.22 01:09:20.388151 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a7bc34c3b2936e3b0b950f9b9d53e6cd) : [0, 97613]: Setting download as failed: File segment: [0, 97613], key: a7bc34c3b2936e3b0b950f9b9d53e6cd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:20.390396 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:09:20.390798 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a7bc34c3b2936e3b0b950f9b9d53e6cd): No space left in cache to reserve 97614 bytes, will continue without cache download 2024.10.22 01:09:20.391723 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.89 GiB, free memory in arenas 4.38 MiB, will set to 1.67 GiB (RSS), difference: 1.21 MiB 2024.10.22 01:09:20.414044 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a7bc34c3b2936e3b0b950f9b9d53e6cd, offset: 0, size: 0 2024.10.22 01:09:20.417072 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a7bc34c3b2936e3b0b950f9b9d53e6cd for removal 2024.10.22 01:09:20.428344 [ 1476 ] {} CacheMetadata: Key a7bc34c3b2936e3b0b950f9b9d53e6cd is removed from metadata 2024.10.22 01:09:20.428985 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1a7a3b5ca82a85257e808d8dd807588f, offset: 0, size: 0 2024.10.22 01:09:20.435255 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1a7a3b5ca82a85257e808d8dd807588f for removal 2024.10.22 01:09:20.536233 [ 1476 ] {} CacheMetadata: Key 1a7a3b5ca82a85257e808d8dd807588f is removed from metadata 2024.10.22 01:09:20.768411 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_306_96, 202410_307_307_0] 2024.10.22 01:09:20.774330 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_306_96, 202410_307_307_0] 2024.10.22 01:09:20.792627 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_306_96, 202410_307_307_0] 2024.10.22 01:09:20.992634 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:20.995934 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:21.118618 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1f6121786191049976b57093747870ab) : [0, 173]: Setting download as failed: File segment: [0, 173], key: 1f6121786191049976b57093747870ab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:21.120317 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1f6121786191049976b57093747870ab): No space left in cache to reserve 174 bytes, will continue without cache download 2024.10.22 01:09:21.225272 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 38291 2024.10.22 01:09:21.250242 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_363_363_0 (state Temporary) 2024.10.22 01:09:21.395047 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:21.402466 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:21.424418 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_363_363_0 to 202410_369_369_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:21.447882 [ 1162 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_363_141 to 202410_364_364_0 2024.10.22 01:09:21.464049 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 32691 2024.10.22 01:09:21.510959 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_74_74_0 to 202410_77_77_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:21.556483 [ 1162 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.67 MiB using storage policy from min volume index 0 2024.10.22 01:09:21.563791 [ 1162 ] {} DiskLocal: Reserved 1.67 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:21.677339 [ 1162 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 01:09:21.695500 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_626_626_0 (state Temporary) 2024.10.22 01:09:21.709069 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_533_160, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0, 202410_538_538_0] 2024.10.22 01:09:21.717237 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_364_142} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_363_141 to 202410_364_364_0 into Wide with storage Full 2024.10.22 01:09:21.781059 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(481dc73fd66acdf7586c88bbb7a8cb3f) : [0, 230193]: Setting download as failed: File segment: [0, 230193], key: 481dc73fd66acdf7586c88bbb7a8cb3f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:21.785095 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_533_160, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0, 202410_538_538_0] 2024.10.22 01:09:21.786712 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(481dc73fd66acdf7586c88bbb7a8cb3f): No space left in cache to reserve 230194 bytes, will continue without cache download 2024.10.22 01:09:21.814832 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_364_142} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:21.820117 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_364_142} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_363_141, total 31452 rows starting from the beginning of the part 2024.10.22 01:09:21.832380 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_626_626_0 to 202410_639_639_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:21.864814 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_634_257 to 202410_635_635_0 2024.10.22 01:09:21.877574 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 01:09:21.883766 [ 1158 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:21.908861 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 481dc73fd66acdf7586c88bbb7a8cb3f, offset: 0, size: 0 2024.10.22 01:09:21.911387 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 481dc73fd66acdf7586c88bbb7a8cb3f for removal 2024.10.22 01:09:21.917059 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38291 2024.10.22 01:09:21.916022 [ 1476 ] {} CacheMetadata: Key 481dc73fd66acdf7586c88bbb7a8cb3f is removed from metadata 2024.10.22 01:09:21.919108 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1f6121786191049976b57093747870ab, offset: 0, size: 0 2024.10.22 01:09:21.921612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1f6121786191049976b57093747870ab for removal 2024.10.22 01:09:22.008985 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_533_160, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0, 202410_538_538_0] 2024.10.22 01:09:22.015515 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 29 entries to flush up to offset 5489 2024.10.22 01:09:22.025274 [ 1476 ] {} CacheMetadata: Key 1f6121786191049976b57093747870ab is removed from metadata 2024.10.22 01:09:22.069713 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_364_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_364_364_0, total 217 rows starting from the beginning of the part 2024.10.22 01:09:22.136871 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_635_258} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_634_257 to 202410_635_635_0 into Compact with storage Full 2024.10.22 01:09:22.199901 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_635_258} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:22.225553 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_635_258} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_634_257, total 38719 rows starting from the beginning of the part 2024.10.22 01:09:22.260866 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_635_258} MergeTreeSequentialSource: Reading 2 marks from part 202410_635_635_0, total 63 rows starting from the beginning of the part 2024.10.22 01:09:22.444280 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:22.450918 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:22.491111 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_368_368_0 (state Temporary) 2024.10.22 01:09:22.640484 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_368_368_0 to 202410_381_381_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:22.643032 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.89 GiB, free memory in arenas 4.84 MiB, will set to 1.67 GiB (RSS), difference: -56.69 MiB 2024.10.22 01:09:22.688184 [ 1248 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_376_75 to 202410_381_381_0 2024.10.22 01:09:22.695698 [ 1248 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:22.697372 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:22.706054 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5489 2024.10.22 01:09:22.713436 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(03d074cf9aa4ed018d8b4bd5bd7b4f27) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 03d074cf9aa4ed018d8b4bd5bd7b4f27, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:22.715916 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 589172, file segment info: File segment: [0, 33554431], key: 03d074cf9aa4ed018d8b4bd5bd7b4f27, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:22.717760 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:22.859414 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_376_75 to 202410_381_381_0 into Compact with storage Full 2024.10.22 01:09:22.896595 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:22.905587 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_376_75, total 5505 rows starting from the beginning of the part 2024.10.22 01:09:22.939248 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_377_377_0, total 23 rows starting from the beginning of the part 2024.10.22 01:09:22.984245 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_378_378_0, total 11 rows starting from the beginning of the part 2024.10.22 01:09:23.022608 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_379_379_0, total 15 rows starting from the beginning of the part 2024.10.22 01:09:23.082111 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_380_380_0, total 12 rows starting from the beginning of the part 2024.10.22 01:09:23.210118 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_381_381_0, total 29 rows starting from the beginning of the part 2024.10.22 01:09:23.686729 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_635_258/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:09:23.734250 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(88e5279a8fd0e0ceb693156ddc744935) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 88e5279a8fd0e0ceb693156ddc744935, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:23.739250 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(88e5279a8fd0e0ceb693156ddc744935): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:09:24.144556 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2954 2024.10.22 01:09:24.238845 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_364_142} MergeTask::MergeProjectionsStage: Merge sorted 31669 rows, containing 13 columns (13 merged, 0 gathered) in 2.526416618 sec., 12535.145539483623 rows/sec., 5.38 MiB/sec. 2024.10.22 01:09:24.249443 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} MergeTask::MergeProjectionsStage: Merge sorted 5595 rows, containing 12 columns (12 merged, 0 gathered) in 1.403902731 sec., 3985.3188375911777 rows/sec., 629.12 KiB/sec. 2024.10.22 01:09:24.371121 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_635_258} MergeTask::MergeProjectionsStage: Merge sorted 38782 rows, containing 13 columns (13 merged, 0 gathered) in 2.476252586 sec., 15661.568702345616 rows/sec., 2.46 MiB/sec. 2024.10.22 01:09:24.399050 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} MergedBlockOutputStream: filled checksums 202410_1_381_76 (state Temporary) 2024.10.22 01:09:24.712038 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(4fb972563368ac623396d6b2be25b487) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4fb972563368ac623396d6b2be25b487, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:24.715037 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 743, file segment info: File segment: [0, 33554431], key: 4fb972563368ac623396d6b2be25b487, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:24.742432 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_381_76 to 202410_1_381_76 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:24.744725 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_635_258} MergedBlockOutputStream: filled checksums 202410_1_635_258 (state Temporary) 2024.10.22 01:09:24.745777 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_364_142} MergedBlockOutputStream: filled checksums 202410_1_364_142 (state Temporary) 2024.10.22 01:09:24.747074 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_381_76} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_376_75, 202410_381_381_0] -> 202410_1_381_76 2024.10.22 01:09:24.818821 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:24.892757 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.38 MiB. 2024.10.22 01:09:24.991462 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_635_258} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_635_258 to 202410_1_635_258 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:25.000631 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_635_258} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_634_257, 202410_635_635_0] -> 202410_1_635_258 2024.10.22 01:09:25.017693 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:09:25.075259 [ 1678 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hkj/ddduvuiyzcbktmsizgpppftnlunzx, size 589172 2024.10.22 01:09:25.147438 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:09:25.154203 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:09:25.163802 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:09:25.186295 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:09:25.207999 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:09:25.236173 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:09:25.256342 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:09:25.309755 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:09:25.318095 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:09:25.378588 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gdl/jpcblunzqlcrgesiixeeylwbllaaw (LogSeriesLimiter: on interval from 2024-10-22 01:09:14 to 2024-10-22 01:09:25 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:09:25.393225 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_364_142} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_364_142 to 202410_1_364_142 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:25.396279 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_364_142} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_363_141, 202410_364_364_0] -> 202410_1_364_142 2024.10.22 01:09:25.460917 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(cad854e69a5a5ce4da231fd4496e653b) : [0, 972185]: Setting download as failed: File segment: [0, 972185], key: cad854e69a5a5ce4da231fd4496e653b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:25.463805 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(cad854e69a5a5ce4da231fd4496e653b): No space left in cache to reserve 972186 bytes, will continue without cache download 2024.10.22 01:09:25.488737 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cad854e69a5a5ce4da231fd4496e653b, offset: 0, size: 0 2024.10.22 01:09:25.492492 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cad854e69a5a5ce4da231fd4496e653b for removal 2024.10.22 01:09:25.495289 [ 1476 ] {} CacheMetadata: Key cad854e69a5a5ce4da231fd4496e653b is removed from metadata 2024.10.22 01:09:25.500503 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 88e5279a8fd0e0ceb693156ddc744935, offset: 0, size: 0 2024.10.22 01:09:25.503224 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 88e5279a8fd0e0ceb693156ddc744935 for removal 2024.10.22 01:09:25.506478 [ 1476 ] {} CacheMetadata: Key 88e5279a8fd0e0ceb693156ddc744935 is removed from metadata 2024.10.22 01:09:25.535446 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.42 MiB. 2024.10.22 01:09:25.674581 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_74_15 to 202410_1_74_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:25.679341 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_74_15} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_69_14, 202410_74_74_0] -> 202410_1_74_15 2024.10.22 01:09:25.932287 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 100 entries to flush up to offset 38391 2024.10.22 01:09:26.201467 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:26.207643 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 37.65 MiB. 2024.10.22 01:09:26.207764 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:26.213280 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ab13b547cf85632054acc82c8b82dde0) : [0, 152]: Setting download as failed: File segment: [0, 152], key: ab13b547cf85632054acc82c8b82dde0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:26.215382 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ab13b547cf85632054acc82c8b82dde0): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:09:26.378786 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_627_627_0 (state Temporary) 2024.10.22 01:09:26.481346 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.89 GiB, free memory in arenas 6.08 MiB, will set to 1.66 GiB (RSS), difference: 60.75 MiB 2024.10.22 01:09:26.582701 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 315 2024.10.22 01:09:26.667157 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_627_627_0 to 202410_640_640_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:26.797449 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_635_258 to 202410_636_636_0 2024.10.22 01:09:26.800892 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 01:09:26.803285 [ 1213 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:26.815785 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_636_259} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_635_258 to 202410_636_636_0 into Compact with storage Full 2024.10.22 01:09:26.818744 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38391 2024.10.22 01:09:26.821696 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:26.824810 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:26.856027 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5b97acc0f2a7534848dd0e10c31f5fa2) : [0, 103744]: Setting download as failed: File segment: [0, 103744], key: 5b97acc0f2a7534848dd0e10c31f5fa2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:26.859594 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5b97acc0f2a7534848dd0e10c31f5fa2): No space left in cache to reserve 103745 bytes, will continue without cache download 2024.10.22 01:09:26.889366 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5b97acc0f2a7534848dd0e10c31f5fa2, offset: 0, size: 0 2024.10.22 01:09:26.894722 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5b97acc0f2a7534848dd0e10c31f5fa2 for removal 2024.10.22 01:09:26.898799 [ 1476 ] {} CacheMetadata: Key 5b97acc0f2a7534848dd0e10c31f5fa2 is removed from metadata 2024.10.22 01:09:26.899536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ab13b547cf85632054acc82c8b82dde0, offset: 0, size: 0 2024.10.22 01:09:26.900371 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ab13b547cf85632054acc82c8b82dde0 for removal 2024.10.22 01:09:26.901724 [ 1476 ] {} CacheMetadata: Key ab13b547cf85632054acc82c8b82dde0 is removed from metadata 2024.10.22 01:09:26.978572 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_636_259} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:26.992570 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_297_297_0 (state Temporary) 2024.10.22 01:09:27.090353 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_636_259} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_635_258, total 38782 rows starting from the beginning of the part 2024.10.22 01:09:27.160362 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1051 entries to flush up to offset 572345 2024.10.22 01:09:27.186062 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_297_297_0 to 202410_307_307_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:27.218436 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_636_259} MergeTreeSequentialSource: Reading 2 marks from part 202410_636_636_0, total 89 rows starting from the beginning of the part 2024.10.22 01:09:27.267366 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 315 2024.10.22 01:09:27.925508 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(16e12e70753afeabea7acd6b8e0caebb) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 16e12e70753afeabea7acd6b8e0caebb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:27.928471 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(16e12e70753afeabea7acd6b8e0caebb): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:09:28.003970 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:28.006527 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:28.127898 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_353_353_0 (state Temporary) 2024.10.22 01:09:28.229015 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_353_353_0 to 202410_360_360_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:28.277259 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 572345 2024.10.22 01:09:28.336420 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.89 GiB, free memory in arenas 6.78 MiB, will set to 1.67 GiB (RSS), difference: -6.35 MiB 2024.10.22 01:09:28.398959 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_307_97, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0, 202410_312_312_0] 2024.10.22 01:09:28.400363 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_307_97, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0, 202410_312_312_0] 2024.10.22 01:09:28.418963 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_307_97, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0, 202410_312_312_0] 2024.10.22 01:09:28.823293 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d9296d67589d20e3fd1e458afecd27ae) : [0, 3683]: Setting download as failed: File segment: [0, 3683], key: d9296d67589d20e3fd1e458afecd27ae, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:28.831650 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d9296d67589d20e3fd1e458afecd27ae): No space left in cache to reserve 3684 bytes, will continue without cache download 2024.10.22 01:09:28.854820 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d9296d67589d20e3fd1e458afecd27ae, offset: 0, size: 0 2024.10.22 01:09:28.856586 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d9296d67589d20e3fd1e458afecd27ae for removal 2024.10.22 01:09:28.861031 [ 1476 ] {} CacheMetadata: Key d9296d67589d20e3fd1e458afecd27ae is removed from metadata 2024.10.22 01:09:28.862798 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 16e12e70753afeabea7acd6b8e0caebb, offset: 0, size: 0 2024.10.22 01:09:28.868108 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 16e12e70753afeabea7acd6b8e0caebb for removal 2024.10.22 01:09:28.878639 [ 1476 ] {} CacheMetadata: Key 16e12e70753afeabea7acd6b8e0caebb is removed from metadata 2024.10.22 01:09:29.005232 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 211 entries to flush up to offset 32902 2024.10.22 01:09:29.300150 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:29.314245 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.43 GiB. 2024.10.22 01:09:29.596478 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_364_364_0 (state Temporary) 2024.10.22 01:09:29.736325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8a61cb08d2a93d7d871d3933d566c40d) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 8a61cb08d2a93d7d871d3933d566c40d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:29.840709 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8a61cb08d2a93d7d871d3933d566c40d): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:09:29.889281 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_364_364_0 to 202410_370_370_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:29.905102 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:09:29.906703 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_636_259} MergeTask::MergeProjectionsStage: Merge sorted 38871 rows, containing 13 columns (13 merged, 0 gathered) in 3.097048871 sec., 12550.980504046427 rows/sec., 1.97 MiB/sec. 2024.10.22 01:09:29.907697 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:09:29.913501 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 32902 2024.10.22 01:09:29.929699 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_364_142 to 202410_365_365_0 2024.10.22 01:09:29.959906 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.68 MiB using storage policy from min volume index 0 2024.10.22 01:09:29.967950 [ 1256 ] {} DiskLocal: Reserved 1.68 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:30.008769 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_365_143} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_364_142 to 202410_365_365_0 into Wide with storage Full 2024.10.22 01:09:30.041798 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_365_143} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:30.050724 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_365_143} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_364_142, total 31669 rows starting from the beginning of the part 2024.10.22 01:09:30.208814 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 5514 2024.10.22 01:09:30.233678 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_636_259} MergedBlockOutputStream: filled checksums 202410_1_636_259 (state Temporary) 2024.10.22 01:09:30.299322 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_365_143} MergeTreeSequentialSource: Reading 2 marks from part 202410_365_365_0, total 240 rows starting from the beginning of the part 2024.10.22 01:09:30.336019 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:30.339165 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:30.367208 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_369_369_0 (state Temporary) 2024.10.22 01:09:30.400219 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_636_259} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_636_259 to 202410_1_636_259 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:30.403730 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_636_259} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_635_258, 202410_636_636_0] -> 202410_1_636_259 2024.10.22 01:09:30.411865 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_369_369_0 to 202410_382_382_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:30.424848 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:09:30.460411 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5514 2024.10.22 01:09:30.493821 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.89 GiB, free memory in arenas 6.92 MiB, will set to 1.67 GiB (RSS), difference: 16.60 MiB 2024.10.22 01:09:30.741355 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ab1afab676677506d39a2c1e2a63beba) : [0, 1041537]: Setting download as failed: File segment: [0, 1041537], key: ab1afab676677506d39a2c1e2a63beba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:30.749060 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ab1afab676677506d39a2c1e2a63beba): No space left in cache to reserve 1041538 bytes, will continue without cache download 2024.10.22 01:09:30.836667 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 38473 2024.10.22 01:09:30.889242 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ab1afab676677506d39a2c1e2a63beba, offset: 0, size: 0 2024.10.22 01:09:30.892297 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ab1afab676677506d39a2c1e2a63beba for removal 2024.10.22 01:09:30.895113 [ 1476 ] {} CacheMetadata: Key ab1afab676677506d39a2c1e2a63beba is removed from metadata 2024.10.22 01:09:30.898560 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8a61cb08d2a93d7d871d3933d566c40d, offset: 0, size: 0 2024.10.22 01:09:30.899778 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8a61cb08d2a93d7d871d3933d566c40d for removal 2024.10.22 01:09:30.901743 [ 1476 ] {} CacheMetadata: Key 8a61cb08d2a93d7d871d3933d566c40d is removed from metadata 2024.10.22 01:09:30.919883 [ 1243 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:09:30.959624 [ 1243 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:09:31.024373 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:31.131902 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:31.204959 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_628_628_0 (state Temporary) 2024.10.22 01:09:31.252241 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_628_628_0 to 202410_641_641_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:31.290721 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_636_259 to 202410_637_637_0 2024.10.22 01:09:31.296190 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 01:09:31.322567 [ 1227 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:31.359813 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_637_260} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_636_259 to 202410_637_637_0 into Compact with storage Full 2024.10.22 01:09:31.398849 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38473 2024.10.22 01:09:31.399783 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_637_260} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:31.406235 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_637_260} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_636_259, total 38871 rows starting from the beginning of the part 2024.10.22 01:09:31.440283 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_637_260} MergeTreeSequentialSource: Reading 2 marks from part 202410_637_637_0, total 87 rows starting from the beginning of the part 2024.10.22 01:09:31.649068 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(011dfe9533c3a1710814b0a95941de31) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 011dfe9533c3a1710814b0a95941de31, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:31.652087 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(011dfe9533c3a1710814b0a95941de31): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:09:31.686682 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 36 entries to flush up to offset 2990 2024.10.22 01:09:32.527463 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_296_59, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:09:32.531314 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_296_59, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:09:32.572188 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_296_59, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:09:32.759570 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.89 GiB, free memory in arenas 7.22 MiB, will set to 1.68 GiB (RSS), difference: -36.04 MiB 2024.10.22 01:09:32.844177 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_365_143} MergeTask::MergeProjectionsStage: Merge sorted 31909 rows, containing 13 columns (13 merged, 0 gathered) in 2.847453927 sec., 11206.151466555406 rows/sec., 4.81 MiB/sec. 2024.10.22 01:09:32.912533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9871f6a617302e9fe564f45f2ec27475) : [0, 66225]: Setting download as failed: File segment: [0, 66225], key: 9871f6a617302e9fe564f45f2ec27475, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:32.915826 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9871f6a617302e9fe564f45f2ec27475): No space left in cache to reserve 66226 bytes, will continue without cache download 2024.10.22 01:09:32.941105 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9871f6a617302e9fe564f45f2ec27475, offset: 0, size: 0 2024.10.22 01:09:32.944456 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9871f6a617302e9fe564f45f2ec27475 for removal 2024.10.22 01:09:32.948937 [ 1476 ] {} CacheMetadata: Key 9871f6a617302e9fe564f45f2ec27475 is removed from metadata 2024.10.22 01:09:32.949609 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 011dfe9533c3a1710814b0a95941de31, offset: 0, size: 0 2024.10.22 01:09:32.952415 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 011dfe9533c3a1710814b0a95941de31 for removal 2024.10.22 01:09:32.955573 [ 1476 ] {} CacheMetadata: Key 011dfe9533c3a1710814b0a95941de31 is removed from metadata 2024.10.22 01:09:33.036279 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_365_143} MergedBlockOutputStream: filled checksums 202410_1_365_143 (state Temporary) 2024.10.22 01:09:33.538479 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6c8f0403b2ca7f0ef7bc5237f7e278dc) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 6c8f0403b2ca7f0ef7bc5237f7e278dc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:33.539622 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6c8f0403b2ca7f0ef7bc5237f7e278dc): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:09:33.645437 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_365_143} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_365_143 to 202410_1_365_143 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:33.648911 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_365_143} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_364_142, 202410_365_365_0] -> 202410_1_365_143 2024.10.22 01:09:33.660277 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.42 MiB. 2024.10.22 01:09:33.895235 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_637_260} MergeTask::MergeProjectionsStage: Merge sorted 38958 rows, containing 13 columns (13 merged, 0 gathered) in 2.544042148 sec., 15313.425538419971 rows/sec., 2.41 MiB/sec. 2024.10.22 01:09:34.001157 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_637_260} MergedBlockOutputStream: filled checksums 202410_1_637_260 (state Temporary) 2024.10.22 01:09:34.061081 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_637_260} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_637_260 to 202410_1_637_260 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:34.068235 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_637_260} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_636_259, 202410_637_637_0] -> 202410_1_637_260 2024.10.22 01:09:34.107647 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:09:34.194094 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(54ee93c73fb61ac5a14c8689494bef5a) : [0, 58110]: Setting download as failed: File segment: [0, 58110], key: 54ee93c73fb61ac5a14c8689494bef5a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:34.196398 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(54ee93c73fb61ac5a14c8689494bef5a): No space left in cache to reserve 58111 bytes, will continue without cache download 2024.10.22 01:09:34.208333 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 54ee93c73fb61ac5a14c8689494bef5a, offset: 0, size: 0 2024.10.22 01:09:34.209806 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 54ee93c73fb61ac5a14c8689494bef5a for removal 2024.10.22 01:09:34.213754 [ 1476 ] {} CacheMetadata: Key 54ee93c73fb61ac5a14c8689494bef5a is removed from metadata 2024.10.22 01:09:34.221771 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6c8f0403b2ca7f0ef7bc5237f7e278dc, offset: 0, size: 0 2024.10.22 01:09:34.224464 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6c8f0403b2ca7f0ef7bc5237f7e278dc for removal 2024.10.22 01:09:34.227792 [ 1476 ] {} CacheMetadata: Key 6c8f0403b2ca7f0ef7bc5237f7e278dc is removed from metadata 2024.10.22 01:09:34.577596 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.89 GiB, free memory in arenas 4.47 MiB, will set to 1.66 GiB (RSS), difference: 37.50 MiB 2024.10.22 01:09:34.777202 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 316 2024.10.22 01:09:35.036518 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:35.040987 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:35.191556 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(fdb48e5626baa87516e20bcfb706529a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fdb48e5626baa87516e20bcfb706529a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:35.193204 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 744, file segment info: File segment: [0, 33554431], key: fdb48e5626baa87516e20bcfb706529a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:35.195594 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:35.283402 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1527 entries to flush up to offset 573872 2024.10.22 01:09:35.315889 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_298_298_0 (state Temporary) 2024.10.22 01:09:35.379519 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_298_298_0 to 202410_308_308_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:35.405263 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 38539 2024.10.22 01:09:35.420521 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 316 2024.10.22 01:09:35.606747 [ 1164 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 335 ms. 2024.10.22 01:09:35.826854 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(35611d58d23e0b825a0196258f0e5b11) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 35611d58d23e0b825a0196258f0e5b11, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:35.829275 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(35611d58d23e0b825a0196258f0e5b11): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:09:35.891526 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:35.895216 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:36.204775 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:36.206918 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:36.210490 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_629_629_0 (state Temporary) 2024.10.22 01:09:36.293763 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_629_629_0 to 202410_642_642_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:36.312695 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_637_260 to 202410_638_638_0 2024.10.22 01:09:36.316747 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 01:09:36.318896 [ 1168 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:36.331041 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38539 2024.10.22 01:09:36.439559 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_638_261} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_637_260 to 202410_638_638_0 into Compact with storage Full 2024.10.22 01:09:36.440498 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_354_354_0 (state Temporary) 2024.10.22 01:09:36.486357 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_638_261} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:36.489379 [ 1241 ] {} DatabaseCatalog: Execution took 321 ms. 2024.10.22 01:09:36.492109 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_638_261} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_637_260, total 38958 rows starting from the beginning of the part 2024.10.22 01:09:36.506421 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_638_261} MergeTreeSequentialSource: Reading 2 marks from part 202410_638_638_0, total 102 rows starting from the beginning of the part 2024.10.22 01:09:36.535728 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_354_354_0 to 202410_361_361_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:36.584883 [ 1163 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_356_71 to 202410_361_361_0 2024.10.22 01:09:36.594200 [ 1163 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:36.596694 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 573872 2024.10.22 01:09:36.616000 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:36.629747 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_356_71 to 202410_361_361_0 into Compact with storage Full 2024.10.22 01:09:36.685114 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:36.689265 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} MergeTreeSequentialSource: Reading 73 marks from part 202410_1_356_71, total 581702 rows starting from the beginning of the part 2024.10.22 01:09:36.707564 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_357_357_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:09:36.718098 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_358_358_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:09:36.734518 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_359_359_0, total 878 rows starting from the beginning of the part 2024.10.22 01:09:36.754882 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_360_360_0, total 1051 rows starting from the beginning of the part 2024.10.22 01:09:36.796226 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_361_361_0, total 1527 rows starting from the beginning of the part 2024.10.22 01:09:36.810903 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(f76b51f84b21b21b621b780b4d8f2ec0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f76b51f84b21b21b621b780b4d8f2ec0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:36.924008 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 377, file segment info: File segment: [0, 33554431], key: f76b51f84b21b21b621b780b4d8f2ec0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:36.927895 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:37.340136 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.89 GiB, free memory in arenas 4.08 MiB, will set to 1.67 GiB (RSS), difference: -13.36 MiB 2024.10.22 01:09:37.561420 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 201 entries to flush up to offset 33103 2024.10.22 01:09:37.684333 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c800c23a86bde497c4d8ad1e5bf38c0d) : [0, 58285]: Setting download as failed: File segment: [0, 58285], key: c800c23a86bde497c4d8ad1e5bf38c0d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:37.685638 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c800c23a86bde497c4d8ad1e5bf38c0d): No space left in cache to reserve 58286 bytes, will continue without cache download 2024.10.22 01:09:37.698255 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c800c23a86bde497c4d8ad1e5bf38c0d, offset: 0, size: 0 2024.10.22 01:09:37.707579 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c800c23a86bde497c4d8ad1e5bf38c0d for removal 2024.10.22 01:09:37.723193 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 35611d58d23e0b825a0196258f0e5b11, offset: 0, size: 0 2024.10.22 01:09:37.727844 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 35611d58d23e0b825a0196258f0e5b11 for removal 2024.10.22 01:09:37.731795 [ 1476 ] {} CacheMetadata: Key c800c23a86bde497c4d8ad1e5bf38c0d is removed from metadata 2024.10.22 01:09:37.742215 [ 1476 ] {} CacheMetadata: Key 35611d58d23e0b825a0196258f0e5b11 is removed from metadata 2024.10.22 01:09:37.965415 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 5528 2024.10.22 01:09:38.001461 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:38.003869 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:38.056958 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_365_365_0 (state Temporary) 2024.10.22 01:09:38.196718 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:38.199231 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:38.214420 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_365_365_0 to 202410_371_371_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:38.241171 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_365_143 to 202410_366_366_0 2024.10.22 01:09:38.246448 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_370_370_0 (state Temporary) 2024.10.22 01:09:38.248755 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.69 MiB using storage policy from min volume index 0 2024.10.22 01:09:38.250184 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 33103 2024.10.22 01:09:38.277333 [ 1176 ] {} DiskLocal: Reserved 1.69 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:38.291482 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_366_144} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_365_143 to 202410_366_366_0 into Wide with storage Full 2024.10.22 01:09:38.324557 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_366_144} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:38.337587 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_366_144} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_365_143, total 31909 rows starting from the beginning of the part 2024.10.22 01:09:38.395543 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_370_370_0 to 202410_383_383_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:38.408843 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_361_72/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:09:38.535219 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_366_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_366_366_0, total 203 rows starting from the beginning of the part 2024.10.22 01:09:38.643899 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(039fee4f144ad77f0e7d6279abc1206b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 039fee4f144ad77f0e7d6279abc1206b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:38.646880 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 5149, file segment info: File segment: [0, 33554431], key: 039fee4f144ad77f0e7d6279abc1206b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:38.652582 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:38.739234 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_638_261} MergeTask::MergeProjectionsStage: Merge sorted 39060 rows, containing 13 columns (13 merged, 0 gathered) in 2.39316077 sec., 16321.511070064882 rows/sec., 2.57 MiB/sec. 2024.10.22 01:09:38.811164 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5528 2024.10.22 01:09:39.040850 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_638_261} MergedBlockOutputStream: filled checksums 202410_1_638_261 (state Temporary) 2024.10.22 01:09:39.203995 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_638_261} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_638_261 to 202410_1_638_261 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:39.207740 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_638_261} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_637_260, 202410_638_638_0] -> 202410_1_638_261 2024.10.22 01:09:39.244911 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:09:39.346814 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(db39a9087b022d9f5020a039a6d0e366) : [0, 162]: Setting download as failed: File segment: [0, 162], key: db39a9087b022d9f5020a039a6d0e366, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:39.348822 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(db39a9087b022d9f5020a039a6d0e366): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:09:39.905641 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:09:39.907462 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:09:40.213729 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(12ec9b6b65575195e043693785f1b780) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 12ec9b6b65575195e043693785f1b780, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:40.304196 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 505, file segment info: File segment: [0, 33554431], key: 12ec9b6b65575195e043693785f1b780, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:40.306506 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:40.341623 [ 1740 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nqn/sfhmcajghpkxahmlbixcjubfxsyyw, size 5149 2024.10.22 01:09:40.342749 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 38620 2024.10.22 01:09:40.552188 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:09:40.601320 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:09:40.616483 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 12.78 MiB, will set to 1.68 GiB (RSS), difference: -3.50 MiB 2024.10.22 01:09:40.639818 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:09:40.730696 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:40.735026 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:40.785551 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:09:40.807058 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:09:40.810384 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_630_630_0 (state Temporary) 2024.10.22 01:09:40.840501 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:09:40.857641 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_630_630_0 to 202410_643_643_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:40.893059 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:09:40.894968 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_366_144} MergeTask::MergeProjectionsStage: Merge sorted 32112 rows, containing 13 columns (13 merged, 0 gathered) in 2.608574583 sec., 12310.17131320412 rows/sec., 5.28 MiB/sec. 2024.10.22 01:09:40.900308 [ 1203 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_638_261 to 202410_639_639_0 2024.10.22 01:09:40.904356 [ 1203 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 01:09:40.905998 [ 1203 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:40.939913 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38620 2024.10.22 01:09:41.009258 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_639_262} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_638_261 to 202410_639_639_0 into Compact with storage Full 2024.10.22 01:09:41.055379 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_366_144} MergedBlockOutputStream: filled checksums 202410_1_366_144 (state Temporary) 2024.10.22 01:09:41.066962 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zes/tehrstlwagxfsuqgjfearqlualrfy (LogSeriesLimiter: on interval from 2024-10-22 01:09:25 to 2024-10-22 01:09:41 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:09:41.071572 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d9d5ce2e7dded80e8ff38d19a45d0b99) : [0, 72608]: Setting download as failed: File segment: [0, 72608], key: d9d5ce2e7dded80e8ff38d19a45d0b99, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:41.074548 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d9d5ce2e7dded80e8ff38d19a45d0b99): No space left in cache to reserve 72609 bytes, will continue without cache download 2024.10.22 01:09:41.218656 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_639_262} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:41.238569 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_639_262} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_638_261, total 39060 rows starting from the beginning of the part 2024.10.22 01:09:41.294033 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d9d5ce2e7dded80e8ff38d19a45d0b99, offset: 0, size: 0 2024.10.22 01:09:41.295234 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d9d5ce2e7dded80e8ff38d19a45d0b99 for removal 2024.10.22 01:09:41.297142 [ 1476 ] {} CacheMetadata: Key d9d5ce2e7dded80e8ff38d19a45d0b99 is removed from metadata 2024.10.22 01:09:41.298300 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: db39a9087b022d9f5020a039a6d0e366, offset: 0, size: 0 2024.10.22 01:09:41.300102 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key db39a9087b022d9f5020a039a6d0e366 for removal 2024.10.22 01:09:41.302556 [ 1476 ] {} CacheMetadata: Key db39a9087b022d9f5020a039a6d0e366 is removed from metadata 2024.10.22 01:09:41.303940 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_639_262} MergeTreeSequentialSource: Reading 2 marks from part 202410_639_639_0, total 62 rows starting from the beginning of the part 2024.10.22 01:09:41.812978 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b85ea7799c8c1988c1b95344fd3310d1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b85ea7799c8c1988c1b95344fd3310d1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:41.816197 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 25115, file segment info: File segment: [0, 33554431], key: b85ea7799c8c1988c1b95344fd3310d1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:41.818519 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:42.063168 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_366_144} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_366_144 to 202410_1_366_144 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:42.069119 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_366_144} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_365_143, 202410_366_366_0] -> 202410_1_366_144 2024.10.22 01:09:42.101026 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.42 MiB. 2024.10.22 01:09:42.767438 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d39755061cf30b4386c42059a4e7a04e) : [0, 151]: Setting download as failed: File segment: [0, 151], key: d39755061cf30b4386c42059a4e7a04e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:42.772976 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d39755061cf30b4386c42059a4e7a04e): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:09:42.922954 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 317 2024.10.22 01:09:43.220925 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:43.229016 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:43.238611 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_639_262} MergeTask::MergeProjectionsStage: Merge sorted 39122 rows, containing 13 columns (13 merged, 0 gathered) in 2.232802472 sec., 17521.478272530327 rows/sec., 2.76 MiB/sec. 2024.10.22 01:09:43.346505 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(511d621a95ad2cdda277ba396f6c1f73) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 511d621a95ad2cdda277ba396f6c1f73, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:43.376629 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 567, file segment info: File segment: [0, 33554431], key: 511d621a95ad2cdda277ba396f6c1f73, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:09:43.381034 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:09:43.409447 [ 1717 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gdl/jpcblunzqlcrgesiixeeylwbllaaw, size 25115 2024.10.22 01:09:43.415955 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_639_262} MergedBlockOutputStream: filled checksums 202410_1_639_262 (state Temporary) 2024.10.22 01:09:43.526766 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.89 GiB, free memory in arenas 14.79 MiB, will set to 1.68 GiB (RSS), difference: 25.21 MiB 2024.10.22 01:09:43.560105 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_299_299_0 (state Temporary) 2024.10.22 01:09:43.643619 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 575024 2024.10.22 01:09:43.681524 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:09:43.688907 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:09:43.703807 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:09:43.715858 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:09:43.736033 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:09:43.748570 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:09:43.779327 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:09:43.785822 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:09:43.798088 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:09:43.871450 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_299_299_0 to 202410_309_309_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:43.941611 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_639_262} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_639_262 to 202410_1_639_262 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:43.948912 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_639_262} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_638_261, 202410_639_639_0] -> 202410_1_639_262 2024.10.22 01:09:44.084110 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 317 2024.10.22 01:09:44.113648 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:09:44.132308 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(df7b467921136f5a2cb211a6e4696647) : [0, 27283]: Setting download as failed: File segment: [0, 27283], key: df7b467921136f5a2cb211a6e4696647, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:44.137806 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(df7b467921136f5a2cb211a6e4696647): No space left in cache to reserve 27284 bytes, will continue without cache download 2024.10.22 01:09:44.162887 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: df7b467921136f5a2cb211a6e4696647, offset: 0, size: 0 2024.10.22 01:09:44.168237 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key df7b467921136f5a2cb211a6e4696647 for removal 2024.10.22 01:09:44.182332 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d39755061cf30b4386c42059a4e7a04e, offset: 0, size: 0 2024.10.22 01:09:44.186843 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d39755061cf30b4386c42059a4e7a04e for removal 2024.10.22 01:09:44.195353 [ 1476 ] {} CacheMetadata: Key df7b467921136f5a2cb211a6e4696647 is removed from metadata 2024.10.22 01:09:44.202850 [ 1476 ] {} CacheMetadata: Key d39755061cf30b4386c42059a4e7a04e is removed from metadata 2024.10.22 01:09:44.844308 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:44.856457 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:44.911728 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_538_161, 202410_539_539_0] 2024.10.22 01:09:44.913335 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_538_161, 202410_539_539_0] 2024.10.22 01:09:44.931187 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_538_161, 202410_539_539_0] 2024.10.22 01:09:44.953979 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 38700 2024.10.22 01:09:45.114714 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_355_355_0 (state Temporary) 2024.10.22 01:09:45.228332 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fefe0c492f2665c4ff337adc7a0daec9) : [0, 150]: Setting download as failed: File segment: [0, 150], key: fefe0c492f2665c4ff337adc7a0daec9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:45.232572 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fefe0c492f2665c4ff337adc7a0daec9): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:09:45.311679 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_355_355_0 to 202410_362_362_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:45.364397 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 575024 2024.10.22 01:09:45.387619 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:45.394643 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:45.676043 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_631_631_0 (state Temporary) 2024.10.22 01:09:45.794687 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 183 entries to flush up to offset 33286 2024.10.22 01:09:45.822853 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_631_631_0 to 202410_644_644_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:45.882840 [ 1229 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_639_262 to 202410_640_640_0 2024.10.22 01:09:45.891487 [ 1229 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.39 MiB using storage policy from min volume index 0 2024.10.22 01:09:45.904312 [ 1229 ] {} DiskLocal: Reserved 1.39 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:45.939891 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_640_263} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_639_262 to 202410_640_640_0 into Compact with storage Full 2024.10.22 01:09:45.972164 [ 1208 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:09:45.988709 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_640_263} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:46.018996 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38700 2024.10.22 01:09:46.019494 [ 1208 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:09:46.068842 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(85db761449f24a701020e92d0e00f905) : [0, 229036]: Setting download as failed: File segment: [0, 229036], key: 85db761449f24a701020e92d0e00f905, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:46.070680 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(85db761449f24a701020e92d0e00f905): No space left in cache to reserve 229037 bytes, will continue without cache download 2024.10.22 01:09:46.086262 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:46.090410 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:46.097074 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 85db761449f24a701020e92d0e00f905, offset: 0, size: 0 2024.10.22 01:09:46.099639 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 85db761449f24a701020e92d0e00f905 for removal 2024.10.22 01:09:46.101342 [ 1476 ] {} CacheMetadata: Key 85db761449f24a701020e92d0e00f905 is removed from metadata 2024.10.22 01:09:46.104444 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fefe0c492f2665c4ff337adc7a0daec9, offset: 0, size: 0 2024.10.22 01:09:46.107888 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fefe0c492f2665c4ff337adc7a0daec9 for removal 2024.10.22 01:09:46.110480 [ 1476 ] {} CacheMetadata: Key fefe0c492f2665c4ff337adc7a0daec9 is removed from metadata 2024.10.22 01:09:46.122164 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_640_263} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_639_262, total 39122 rows starting from the beginning of the part 2024.10.22 01:09:46.141709 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_366_366_0 (state Temporary) 2024.10.22 01:09:46.153053 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_640_263} MergeTreeSequentialSource: Reading 2 marks from part 202410_640_640_0, total 100 rows starting from the beginning of the part 2024.10.22 01:09:46.189334 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:46.192810 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:46.312994 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 5543 2024.10.22 01:09:46.412903 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_366_366_0 to 202410_372_372_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:46.555169 [ 1207 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_366_144 to 202410_367_367_0 2024.10.22 01:09:46.618875 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:46.621293 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.42 GiB. 2024.10.22 01:09:46.664012 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 12.80 MiB, will set to 1.68 GiB (RSS), difference: -17.35 MiB 2024.10.22 01:09:46.667960 [ 1207 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.70 MiB using storage policy from min volume index 0 2024.10.22 01:09:46.671684 [ 1207 ] {} DiskLocal: Reserved 1.70 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:09:46.680957 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 33286 2024.10.22 01:09:46.683380 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_371_371_0 (state Temporary) 2024.10.22 01:09:46.685771 [ 1207 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 256 ms. 2024.10.22 01:09:46.703378 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_367_145} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_366_144 to 202410_367_367_0 into Wide with storage Full 2024.10.22 01:09:46.750441 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_367_145} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:46.790658 [ 1213 ] {} DatabaseCatalog: Execution took 300 ms. 2024.10.22 01:09:46.880039 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_367_145} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_366_144, total 32112 rows starting from the beginning of the part 2024.10.22 01:09:46.996733 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_371_371_0 to 202410_384_384_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:47.157144 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_367_145} MergeTreeSequentialSource: Reading 2 marks from part 202410_367_367_0, total 229 rows starting from the beginning of the part 2024.10.22 01:09:47.222507 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ce80d7907e0aaebb7858f909478608c8) : [0, 154]: Setting download as failed: File segment: [0, 154], key: ce80d7907e0aaebb7858f909478608c8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:47.224780 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ce80d7907e0aaebb7858f909478608c8): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:09:47.295867 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5543 2024.10.22 01:09:48.160122 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5f0ac0f5bef9dcd71a3a3e2b70d04811) : [0, 422601]: Setting download as failed: File segment: [0, 422601], key: 5f0ac0f5bef9dcd71a3a3e2b70d04811, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:48.162952 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5f0ac0f5bef9dcd71a3a3e2b70d04811): No space left in cache to reserve 422602 bytes, will continue without cache download 2024.10.22 01:09:48.289777 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5f0ac0f5bef9dcd71a3a3e2b70d04811, offset: 0, size: 0 2024.10.22 01:09:48.291596 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5f0ac0f5bef9dcd71a3a3e2b70d04811 for removal 2024.10.22 01:09:48.292981 [ 1476 ] {} CacheMetadata: Key 5f0ac0f5bef9dcd71a3a3e2b70d04811 is removed from metadata 2024.10.22 01:09:48.295158 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ce80d7907e0aaebb7858f909478608c8, offset: 0, size: 0 2024.10.22 01:09:48.296363 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ce80d7907e0aaebb7858f909478608c8 for removal 2024.10.22 01:09:48.299411 [ 1476 ] {} CacheMetadata: Key ce80d7907e0aaebb7858f909478608c8 is removed from metadata 2024.10.22 01:09:48.697778 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.89 GiB, free memory in arenas 12.62 MiB, will set to 1.68 GiB (RSS), difference: -50.66 MiB 2024.10.22 01:09:48.706814 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_640_263} MergeTask::MergeProjectionsStage: Merge sorted 39222 rows, containing 13 columns (13 merged, 0 gathered) in 2.676086405 sec., 14656.477431639581 rows/sec., 2.31 MiB/sec. 2024.10.22 01:09:48.893976 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_640_263} MergedBlockOutputStream: filled checksums 202410_1_640_263 (state Temporary) 2024.10.22 01:09:48.948417 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(31b55900ffd00e67b8de70a1a2dc68e5) : [0, 145]: Setting download as failed: File segment: [0, 145], key: 31b55900ffd00e67b8de70a1a2dc68e5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:48.950702 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(31b55900ffd00e67b8de70a1a2dc68e5): No space left in cache to reserve 146 bytes, will continue without cache download 2024.10.22 01:09:48.996846 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_640_263} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_640_263 to 202410_1_640_263 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:49.102972 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_640_263} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_639_262, 202410_640_640_0] -> 202410_1_640_263 2024.10.22 01:09:49.235588 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:09:49.476562 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_75_75_0 (state Temporary) 2024.10.22 01:09:49.888616 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_367_145} MergeTask::MergeProjectionsStage: Merge sorted 32341 rows, containing 13 columns (13 merged, 0 gathered) in 3.202503811 sec., 10098.660894302368 rows/sec., 4.33 MiB/sec. 2024.10.22 01:09:49.906799 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:09:49.907601 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:09:49.933017 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(dc769d59602f58a5650687b7ea4d543b) : [0, 4835]: Setting download as failed: File segment: [0, 4835], key: dc769d59602f58a5650687b7ea4d543b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:49.946615 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(dc769d59602f58a5650687b7ea4d543b): No space left in cache to reserve 4836 bytes, will continue without cache download 2024.10.22 01:09:50.071638 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dc769d59602f58a5650687b7ea4d543b, offset: 0, size: 0 2024.10.22 01:09:50.080432 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 38776 2024.10.22 01:09:50.081232 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dc769d59602f58a5650687b7ea4d543b for removal 2024.10.22 01:09:50.084770 [ 1476 ] {} CacheMetadata: Key dc769d59602f58a5650687b7ea4d543b is removed from metadata 2024.10.22 01:09:50.086208 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 31b55900ffd00e67b8de70a1a2dc68e5, offset: 0, size: 0 2024.10.22 01:09:50.090708 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 31b55900ffd00e67b8de70a1a2dc68e5 for removal 2024.10.22 01:09:50.096345 [ 1476 ] {} CacheMetadata: Key 31b55900ffd00e67b8de70a1a2dc68e5 is removed from metadata 2024.10.22 01:09:50.155016 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_367_145} MergedBlockOutputStream: filled checksums 202410_1_367_145 (state Temporary) 2024.10.22 01:09:50.229469 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:50.233892 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:09:50.311411 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_361_72/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:09:50.379401 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_632_632_0 (state Temporary) 2024.10.22 01:09:50.444617 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_632_632_0 to 202410_645_645_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:50.584401 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_640_263 to 202410_641_641_0 2024.10.22 01:09:50.587717 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.39 MiB using storage policy from min volume index 0 2024.10.22 01:09:50.589728 [ 1234 ] {} DiskLocal: Reserved 1.39 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:09:50.594919 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38776 2024.10.22 01:09:50.602641 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_641_264} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_640_263 to 202410_641_641_0 into Compact with storage Full 2024.10.22 01:09:50.653382 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.89 GiB, free memory in arenas 14.26 MiB, will set to 1.68 GiB (RSS), difference: 58.30 MiB 2024.10.22 01:09:50.755485 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_641_264} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:50.770290 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_641_264} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_640_263, total 39222 rows starting from the beginning of the part 2024.10.22 01:09:50.808765 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_367_145} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_367_145 to 202410_1_367_145 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:50.811710 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_367_145} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_366_144, 202410_367_367_0] -> 202410_1_367_145 2024.10.22 01:09:50.836220 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.43 MiB. 2024.10.22 01:09:50.891842 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_539_162, 202410_540_540_0] 2024.10.22 01:09:50.894015 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_539_162, 202410_540_540_0] 2024.10.22 01:09:50.899023 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_641_264} MergeTreeSequentialSource: Reading 2 marks from part 202410_641_641_0, total 82 rows starting from the beginning of the part 2024.10.22 01:09:50.902928 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_539_162, 202410_540_540_0] 2024.10.22 01:09:51.134656 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(224d4dc2bcc9c143316182129538f9bf) : [0, 163]: Setting download as failed: File segment: [0, 163], key: 224d4dc2bcc9c143316182129538f9bf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:51.139042 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(224d4dc2bcc9c143316182129538f9bf): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 01:09:51.404397 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_75_75_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:09:51.587276 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 318 2024.10.22 01:09:51.781206 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:51.784320 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:09:51.942703 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_300_300_0 (state Temporary) 2024.10.22 01:09:52.038522 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(05258655b4c23d01c7a287f79334d0ac) : [0, 183864]: Setting download as failed: File segment: [0, 183864], key: 05258655b4c23d01c7a287f79334d0ac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:52.041480 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(05258655b4c23d01c7a287f79334d0ac): No space left in cache to reserve 183865 bytes, will continue without cache download 2024.10.22 01:09:52.162192 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 05258655b4c23d01c7a287f79334d0ac, offset: 0, size: 0 2024.10.22 01:09:52.166317 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 05258655b4c23d01c7a287f79334d0ac for removal 2024.10.22 01:09:52.169543 [ 1476 ] {} CacheMetadata: Key 05258655b4c23d01c7a287f79334d0ac is removed from metadata 2024.10.22 01:09:52.171820 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 224d4dc2bcc9c143316182129538f9bf, offset: 0, size: 0 2024.10.22 01:09:52.178972 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 224d4dc2bcc9c143316182129538f9bf for removal 2024.10.22 01:09:52.184226 [ 1476 ] {} CacheMetadata: Key 224d4dc2bcc9c143316182129538f9bf is removed from metadata 2024.10.22 01:09:52.240225 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_300_300_0 to 202410_310_310_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:52.301677 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 318 2024.10.22 01:09:52.370878 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 576560 2024.10.22 01:09:52.447582 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 12.40 MiB, will set to 1.68 GiB (RSS), difference: -11.14 MiB 2024.10.22 01:09:52.629290 [ 1258 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 01:09:53.279011 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fd4c05c35e484609194ffdd8929d8eb0) : [0, 152]: Setting download as failed: File segment: [0, 152], key: fd4c05c35e484609194ffdd8929d8eb0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:53.283509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fd4c05c35e484609194ffdd8929d8eb0): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:09:53.398766 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:53.400989 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:09:53.603122 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_356_356_0 (state Temporary) 2024.10.22 01:09:53.660883 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_356_356_0 to 202410_363_363_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:53.718315 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 576560 2024.10.22 01:09:53.683657 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_641_264} MergeTask::MergeProjectionsStage: Merge sorted 39304 rows, containing 13 columns (13 merged, 0 gathered) in 3.086476929 sec., 12734.260097882625 rows/sec., 2.00 MiB/sec. 2024.10.22 01:09:53.893070 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_641_264} MergedBlockOutputStream: filled checksums 202410_1_641_264 (state Temporary) 2024.10.22 01:09:53.950762 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_641_264} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_641_264 to 202410_1_641_264 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:53.955526 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_641_264} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_640_263, 202410_641_641_0] -> 202410_1_641_264 2024.10.22 01:09:54.049504 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:09:54.200889 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 204 entries to flush up to offset 33490 2024.10.22 01:09:54.203823 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b8148c3b9ad4a86eb6455a1ceadc0739) : [0, 280772]: Setting download as failed: File segment: [0, 280772], key: b8148c3b9ad4a86eb6455a1ceadc0739, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:54.205261 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b8148c3b9ad4a86eb6455a1ceadc0739): No space left in cache to reserve 280773 bytes, will continue without cache download 2024.10.22 01:09:54.243955 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b8148c3b9ad4a86eb6455a1ceadc0739, offset: 0, size: 0 2024.10.22 01:09:54.250854 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b8148c3b9ad4a86eb6455a1ceadc0739 for removal 2024.10.22 01:09:54.260186 [ 1476 ] {} CacheMetadata: Key b8148c3b9ad4a86eb6455a1ceadc0739 is removed from metadata 2024.10.22 01:09:54.261172 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fd4c05c35e484609194ffdd8929d8eb0, offset: 0, size: 0 2024.10.22 01:09:54.265395 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fd4c05c35e484609194ffdd8929d8eb0 for removal 2024.10.22 01:09:54.287372 [ 1476 ] {} CacheMetadata: Key fd4c05c35e484609194ffdd8929d8eb0 is removed from metadata 2024.10.22 01:09:54.495950 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.89 GiB, free memory in arenas 12.41 MiB, will set to 1.68 GiB (RSS), difference: 7.04 MiB 2024.10.22 01:09:54.578889 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:54.584946 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:09:54.797507 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 5557 2024.10.22 01:09:54.820349 [ 1242 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 01:09:54.841507 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 38848 2024.10.22 01:09:54.888319 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_367_367_0 (state Temporary) 2024.10.22 01:09:55.059817 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:55.066239 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:09:55.102773 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_372_372_0 (state Temporary) 2024.10.22 01:09:55.152792 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_367_367_0 to 202410_373_373_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:55.186576 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_367_145 to 202410_368_368_0 2024.10.22 01:09:55.188903 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ca51edf49e2c16bc5ddc09cc43b68a14) : [0, 152]: Setting download as failed: File segment: [0, 152], key: ca51edf49e2c16bc5ddc09cc43b68a14, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:55.193398 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ca51edf49e2c16bc5ddc09cc43b68a14): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:09:55.193582 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 33490 2024.10.22 01:09:55.195010 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.71 MiB using storage policy from min volume index 0 2024.10.22 01:09:55.197611 [ 1265 ] {} DiskLocal: Reserved 1.71 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:09:55.213736 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_368_146} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_367_145 to 202410_368_368_0 into Wide with storage Full 2024.10.22 01:09:55.289386 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_368_146} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:55.296165 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_368_146} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_367_145, total 32341 rows starting from the beginning of the part 2024.10.22 01:09:55.467227 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_368_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_368_368_0, total 223 rows starting from the beginning of the part 2024.10.22 01:09:55.470459 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_372_372_0 to 202410_385_385_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:55.554779 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_540_163, 202410_541_541_0] 2024.10.22 01:09:55.558774 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_540_163, 202410_541_541_0] 2024.10.22 01:09:55.579118 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_540_163, 202410_541_541_0] 2024.10.22 01:09:55.616096 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5557 2024.10.22 01:09:55.686740 [ 1157 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 365 ms. 2024.10.22 01:09:55.949824 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:09:55.956707 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:09:56.093107 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(558f0ec57d9ab8368aa4b420d143c40b) : [0, 269318]: Setting download as failed: File segment: [0, 269318], key: 558f0ec57d9ab8368aa4b420d143c40b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:56.094604 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(558f0ec57d9ab8368aa4b420d143c40b): No space left in cache to reserve 269319 bytes, will continue without cache download 2024.10.22 01:09:56.112251 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_633_633_0 (state Temporary) 2024.10.22 01:09:56.170912 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_633_633_0 to 202410_646_646_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:56.201837 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_641_264 to 202410_642_642_0 2024.10.22 01:09:56.211949 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.39 MiB using storage policy from min volume index 0 2024.10.22 01:09:56.219594 [ 1170 ] {} DiskLocal: Reserved 1.39 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:09:56.240413 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38848 2024.10.22 01:09:56.344374 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 558f0ec57d9ab8368aa4b420d143c40b, offset: 0, size: 0 2024.10.22 01:09:56.347412 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 558f0ec57d9ab8368aa4b420d143c40b for removal 2024.10.22 01:09:56.355909 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ca51edf49e2c16bc5ddc09cc43b68a14, offset: 0, size: 0 2024.10.22 01:09:56.359799 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ca51edf49e2c16bc5ddc09cc43b68a14 for removal 2024.10.22 01:09:56.360589 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_642_265} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_641_264 to 202410_642_642_0 into Compact with storage Full 2024.10.22 01:09:56.376192 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_368_146/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:09:56.353685 [ 1476 ] {} CacheMetadata: Key 558f0ec57d9ab8368aa4b420d143c40b is removed from metadata 2024.10.22 01:09:56.461107 [ 1476 ] {} CacheMetadata: Key ca51edf49e2c16bc5ddc09cc43b68a14 is removed from metadata 2024.10.22 01:09:56.578468 [ 1155 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 01:09:56.624195 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_642_265} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:09:56.653596 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_642_265} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_641_264, total 39304 rows starting from the beginning of the part 2024.10.22 01:09:56.698582 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_642_265} MergeTreeSequentialSource: Reading 2 marks from part 202410_642_642_0, total 66 rows starting from the beginning of the part 2024.10.22 01:09:57.141707 [ 1173 ] {} DatabaseCatalog: Execution took 350 ms. 2024.10.22 01:09:57.315731 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fb2888bc3c33ecfb7b4aec6c86bff31b) : [0, 152]: Setting download as failed: File segment: [0, 152], key: fb2888bc3c33ecfb7b4aec6c86bff31b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:57.318375 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fb2888bc3c33ecfb7b4aec6c86bff31b): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:09:57.578323 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.89 GiB, free memory in arenas 13.64 MiB, will set to 1.70 GiB (RSS), difference: -41.96 MiB 2024.10.22 01:09:58.008211 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(65cd65809b31fa8325606a5b9ce5971b) : [0, 312450]: Setting download as failed: File segment: [0, 312450], key: 65cd65809b31fa8325606a5b9ce5971b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:58.009766 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(65cd65809b31fa8325606a5b9ce5971b): No space left in cache to reserve 312451 bytes, will continue without cache download 2024.10.22 01:09:58.052662 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} MergeTask::MergeProjectionsStage: Merge sorted 587462 rows, containing 4 columns (4 merged, 0 gathered) in 21.427963243 sec., 27415.671444737505 rows/sec., 401.86 KiB/sec. 2024.10.22 01:09:58.063479 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_368_146} MergeTask::MergeProjectionsStage: Merge sorted 32564 rows, containing 13 columns (13 merged, 0 gathered) in 2.858961513 sec., 11390.149833052334 rows/sec., 4.89 MiB/sec. 2024.10.22 01:09:58.142308 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 65cd65809b31fa8325606a5b9ce5971b, offset: 0, size: 0 2024.10.22 01:09:58.144794 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 65cd65809b31fa8325606a5b9ce5971b for removal 2024.10.22 01:09:58.151551 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fb2888bc3c33ecfb7b4aec6c86bff31b, offset: 0, size: 0 2024.10.22 01:09:58.155301 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fb2888bc3c33ecfb7b4aec6c86bff31b for removal 2024.10.22 01:09:58.158982 [ 1476 ] {} CacheMetadata: Key 65cd65809b31fa8325606a5b9ce5971b is removed from metadata 2024.10.22 01:09:58.163472 [ 1476 ] {} CacheMetadata: Key fb2888bc3c33ecfb7b4aec6c86bff31b is removed from metadata 2024.10.22 01:09:58.218735 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} MergedBlockOutputStream: filled checksums 202410_1_361_72 (state Temporary) 2024.10.22 01:09:58.277110 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_368_146} MergedBlockOutputStream: filled checksums 202410_1_368_146 (state Temporary) 2024.10.22 01:09:58.396765 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_361_72 to 202410_1_361_72 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:58.399886 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_361_72} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_356_71, 202410_361_361_0] -> 202410_1_361_72 2024.10.22 01:09:58.522157 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:09:58.810515 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0d2ae26dc6b3d24e5e095c013e29ca32) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 0d2ae26dc6b3d24e5e095c013e29ca32, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:58.812196 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0d2ae26dc6b3d24e5e095c013e29ca32): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:09:58.955104 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_642_265} MergeTask::MergeProjectionsStage: Merge sorted 39370 rows, containing 13 columns (13 merged, 0 gathered) in 2.600873995 sec., 15137.21928693435 rows/sec., 2.38 MiB/sec. 2024.10.22 01:09:58.993673 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_368_146} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_368_146 to 202410_1_368_146 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:58.997646 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_368_146} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_367_145, 202410_368_368_0] -> 202410_1_368_146 2024.10.22 01:09:59.007067 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_75_75_0 to 202410_78_78_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:59.010813 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.43 MiB. 2024.10.22 01:09:59.043302 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_642_265} MergedBlockOutputStream: filled checksums 202410_1_642_265 (state Temporary) 2024.10.22 01:09:59.513056 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_541_164, 202410_542_542_0] 2024.10.22 01:09:59.513614 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_642_265} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_642_265 to 202410_1_642_265 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:09:59.515507 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_541_164, 202410_542_542_0] 2024.10.22 01:09:59.520537 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_642_265} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_641_264, 202410_642_642_0] -> 202410_1_642_265 2024.10.22 01:09:59.537363 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_541_164, 202410_542_542_0] 2024.10.22 01:09:59.558284 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:09:59.649758 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.89 GiB, free memory in arenas 9.43 MiB, will set to 1.68 GiB (RSS), difference: 53.53 MiB 2024.10.22 01:09:59.712383 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(987064d85c467c872aa715e0603543e7) : [0, 517461]: Setting download as failed: File segment: [0, 517461], key: 987064d85c467c872aa715e0603543e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:09:59.714114 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(987064d85c467c872aa715e0603543e7): No space left in cache to reserve 517462 bytes, will continue without cache download 2024.10.22 01:09:59.906150 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:09:59.908343 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:09:59.968590 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 987064d85c467c872aa715e0603543e7, offset: 0, size: 0 2024.10.22 01:09:59.992870 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 987064d85c467c872aa715e0603543e7 for removal 2024.10.22 01:10:00.096269 [ 1476 ] {} CacheMetadata: Key 987064d85c467c872aa715e0603543e7 is removed from metadata 2024.10.22 01:10:00.099032 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0d2ae26dc6b3d24e5e095c013e29ca32, offset: 0, size: 0 2024.10.22 01:10:00.100858 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0d2ae26dc6b3d24e5e095c013e29ca32 for removal 2024.10.22 01:10:00.104572 [ 1476 ] {} CacheMetadata: Key 0d2ae26dc6b3d24e5e095c013e29ca32 is removed from metadata 2024.10.22 01:10:00.356235 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 90 entries to flush up to offset 38938 2024.10.22 01:10:00.609008 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:00.610802 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:00.682253 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_634_634_0 (state Temporary) 2024.10.22 01:10:00.723066 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_634_634_0 to 202410_647_647_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:00.726841 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 578096 2024.10.22 01:10:00.758992 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_642_265 to 202410_643_643_0 2024.10.22 01:10:00.777687 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.39 MiB using storage policy from min volume index 0 2024.10.22 01:10:00.780961 [ 1267 ] {} DiskLocal: Reserved 1.39 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:00.788918 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 38938 2024.10.22 01:10:00.789819 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_643_266} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_642_265 to 202410_643_643_0 into Compact with storage Full 2024.10.22 01:10:00.814457 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_643_266} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:00.831254 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_643_266} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_642_265, total 39370 rows starting from the beginning of the part 2024.10.22 01:10:00.872968 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_643_266} MergeTreeSequentialSource: Reading 2 marks from part 202410_643_643_0, total 81 rows starting from the beginning of the part 2024.10.22 01:10:01.027525 [ 1196 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:10:01.059437 [ 1196 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:10:01.280050 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(004125f429690ca452bf02591e1c242b) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 004125f429690ca452bf02591e1c242b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:01.281284 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(004125f429690ca452bf02591e1c242b): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:10:01.384245 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 2990 2024.10.22 01:10:01.427384 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:01.431834 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:01.509800 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 9.64 MiB, will set to 1.67 GiB (RSS), difference: -20.06 MiB 2024.10.22 01:10:01.507230 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_321_64, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0, 202410_325_325_0, 202410_326_326_0] 2024.10.22 01:10:01.527930 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_321_64, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0, 202410_325_325_0, 202410_326_326_0] 2024.10.22 01:10:01.553828 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_321_64, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0, 202410_325_325_0, 202410_326_326_0] 2024.10.22 01:10:01.650368 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_357_357_0 (state Temporary) 2024.10.22 01:10:01.722868 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_357_357_0 to 202410_364_364_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:01.757781 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(19613c091bb646fc15ce08801a6c0d98) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 19613c091bb646fc15ce08801a6c0d98, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:01.851230 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 28948, file segment info: File segment: [0, 33554431], key: 19613c091bb646fc15ce08801a6c0d98, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:01.859659 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:01.964863 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 578096 2024.10.22 01:10:02.399145 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6a5271a98381a591e8e5387deba9932e) : [0, 483395]: Setting download as failed: File segment: [0, 483395], key: 6a5271a98381a591e8e5387deba9932e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:02.400566 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6a5271a98381a591e8e5387deba9932e): No space left in cache to reserve 483396 bytes, will continue without cache download 2024.10.22 01:10:02.522773 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6a5271a98381a591e8e5387deba9932e, offset: 0, size: 0 2024.10.22 01:10:02.525836 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6a5271a98381a591e8e5387deba9932e for removal 2024.10.22 01:10:02.636467 [ 1476 ] {} CacheMetadata: Key 6a5271a98381a591e8e5387deba9932e is removed from metadata 2024.10.22 01:10:02.639102 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 004125f429690ca452bf02591e1c242b, offset: 0, size: 0 2024.10.22 01:10:02.641178 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 004125f429690ca452bf02591e1c242b for removal 2024.10.22 01:10:02.644506 [ 1476 ] {} CacheMetadata: Key 004125f429690ca452bf02591e1c242b is removed from metadata 2024.10.22 01:10:02.715088 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 206 entries to flush up to offset 33696 2024.10.22 01:10:02.857315 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:10:03.000334 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7b5b22cb80bd986bfbb9bfefb0f33ee7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7b5b22cb80bd986bfbb9bfefb0f33ee7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:03.002840 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 564, file segment info: File segment: [0, 33554431], key: 7b5b22cb80bd986bfbb9bfefb0f33ee7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:03.005747 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:03.222854 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 5576 2024.10.22 01:10:03.244195 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:03.249796 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:03.339596 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_368_368_0 (state Temporary) 2024.10.22 01:10:03.368084 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:03.373365 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:03.411607 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_373_373_0 (state Temporary) 2024.10.22 01:10:03.423870 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:10:03.430244 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_643_266} MergeTask::MergeProjectionsStage: Merge sorted 39451 rows, containing 13 columns (13 merged, 0 gathered) in 2.644003405 sec., 14920.933885862376 rows/sec., 2.35 MiB/sec. 2024.10.22 01:10:03.463629 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:10:03.481663 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_373_373_0 to 202410_386_386_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:03.487870 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:10:03.503713 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:10:03.513665 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_643_266} MergedBlockOutputStream: filled checksums 202410_1_643_266 (state Temporary) 2024.10.22 01:10:03.526306 [ 1158 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_381_76 to 202410_386_386_0 2024.10.22 01:10:03.548680 [ 1158 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:03.553747 [ 1158 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:03.572571 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:10:03.567632 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5576 2024.10.22 01:10:03.588773 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_381_76 to 202410_386_386_0 into Compact with storage Full 2024.10.22 01:10:03.595086 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:10:03.600890 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_643_266} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_643_266 to 202410_1_643_266 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:03.607180 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_643_266} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_642_265, 202410_643_643_0] -> 202410_1_643_266 2024.10.22 01:10:03.622737 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:10:03.646406 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_368_368_0 to 202410_374_374_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:03.648110 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:10:03.659342 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:03.678639 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_381_76, total 5595 rows starting from the beginning of the part 2024.10.22 01:10:03.685113 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:10:03.692461 [ 1245 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_368_146 to 202410_369_369_0 2024.10.22 01:10:03.697063 [ 1245 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.72 MiB using storage policy from min volume index 0 2024.10.22 01:10:03.699824 [ 1245 ] {} DiskLocal: Reserved 1.72 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:03.706562 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 33696 2024.10.22 01:10:03.732043 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uec/qskmyvjyodgubladlziwfvvnsroci (LogSeriesLimiter: on interval from 2024-10-22 01:09:41 to 2024-10-22 01:10:03 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:10:03.715664 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_369_147} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_368_146 to 202410_369_369_0 into Wide with storage Full 2024.10.22 01:10:03.743015 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:10:03.784749 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_382_382_0, total 25 rows starting from the beginning of the part 2024.10.22 01:10:03.812715 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_369_147} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:03.823008 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_369_147} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_368_146, total 32564 rows starting from the beginning of the part 2024.10.22 01:10:03.845032 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_383_383_0, total 14 rows starting from the beginning of the part 2024.10.22 01:10:04.011892 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(f0f42d0fb254366fca3355f6f3927552) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f0f42d0fb254366fca3355f6f3927552, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:04.012650 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_384_384_0, total 15 rows starting from the beginning of the part 2024.10.22 01:10:04.014626 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 9955, file segment info: File segment: [0, 33554431], key: f0f42d0fb254366fca3355f6f3927552, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:04.017568 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:04.099790 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_385_385_0, total 14 rows starting from the beginning of the part 2024.10.22 01:10:04.106252 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_369_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_369_369_0, total 231 rows starting from the beginning of the part 2024.10.22 01:10:04.261499 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_386_386_0, total 19 rows starting from the beginning of the part 2024.10.22 01:10:04.467463 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_542_165, 202410_543_543_0] 2024.10.22 01:10:04.472838 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_542_165, 202410_543_543_0] 2024.10.22 01:10:04.485479 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_542_165, 202410_543_543_0] 2024.10.22 01:10:04.528985 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e1fae65d13dc85b438bfca556b898afe) : [0, 152]: Setting download as failed: File segment: [0, 152], key: e1fae65d13dc85b438bfca556b898afe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:04.534487 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e1fae65d13dc85b438bfca556b898afe): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:10:04.814256 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 39034 2024.10.22 01:10:05.051311 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(6e39be6328441068b6b3ac6b4ed6bc47) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6e39be6328441068b6b3ac6b4ed6bc47, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:05.064226 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 360, file segment info: File segment: [0, 33554431], key: 6e39be6328441068b6b3ac6b4ed6bc47, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:05.068346 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:05.229597 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:05.234902 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:05.333506 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_635_635_0 (state Temporary) 2024.10.22 01:10:05.494665 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_635_635_0 to 202410_648_648_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:05.509243 [ 1270 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_643_266 to 202410_644_644_0 2024.10.22 01:10:05.519755 [ 1270 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.40 MiB using storage policy from min volume index 0 2024.10.22 01:10:05.523959 [ 1270 ] {} DiskLocal: Reserved 1.40 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:05.537138 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39034 2024.10.22 01:10:05.545563 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.89 GiB, free memory in arenas 9.62 MiB, will set to 1.67 GiB (RSS), difference: -44.99 MiB 2024.10.22 01:10:05.559516 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_644_267} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_643_266 to 202410_644_644_0 into Compact with storage Full 2024.10.22 01:10:05.611663 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_644_267} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:05.629231 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_644_267} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_643_266, total 39451 rows starting from the beginning of the part 2024.10.22 01:10:05.690184 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_644_267} MergeTreeSequentialSource: Reading 2 marks from part 202410_644_644_0, total 80 rows starting from the beginning of the part 2024.10.22 01:10:05.792951 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} MergeTask::MergeProjectionsStage: Merge sorted 5682 rows, containing 12 columns (12 merged, 0 gathered) in 2.214019492 sec., 2566.373069673047 rows/sec., 405.30 KiB/sec. 2024.10.22 01:10:05.811994 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c252c71e5199763cb9d83394d732c99c) : [0, 300093]: Setting download as failed: File segment: [0, 300093], key: c252c71e5199763cb9d83394d732c99c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:05.813716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c252c71e5199763cb9d83394d732c99c): No space left in cache to reserve 300094 bytes, will continue without cache download 2024.10.22 01:10:05.861267 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c252c71e5199763cb9d83394d732c99c, offset: 0, size: 0 2024.10.22 01:10:05.865514 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c252c71e5199763cb9d83394d732c99c for removal 2024.10.22 01:10:05.870759 [ 1476 ] {} CacheMetadata: Key c252c71e5199763cb9d83394d732c99c is removed from metadata 2024.10.22 01:10:05.877434 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e1fae65d13dc85b438bfca556b898afe, offset: 0, size: 0 2024.10.22 01:10:05.882266 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e1fae65d13dc85b438bfca556b898afe for removal 2024.10.22 01:10:05.995467 [ 1476 ] {} CacheMetadata: Key e1fae65d13dc85b438bfca556b898afe is removed from metadata 2024.10.22 01:10:06.055934 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} MergedBlockOutputStream: filled checksums 202410_1_386_77 (state Temporary) 2024.10.22 01:10:06.275521 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_644_267/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:10:06.311542 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_258_53, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0, 202410_262_262_0, 202410_263_263_0] 2024.10.22 01:10:06.315560 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_258_53, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0, 202410_262_262_0, 202410_263_263_0] 2024.10.22 01:10:06.346802 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_258_53, 202410_259_259_0, 202410_260_260_0, 202410_261_261_0, 202410_262_262_0, 202410_263_263_0] 2024.10.22 01:10:06.458433 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(b010bd28ebb7fcfc822c854c2a58989b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b010bd28ebb7fcfc822c854c2a58989b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:06.463670 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4878, file segment info: File segment: [0, 33554431], key: b010bd28ebb7fcfc822c854c2a58989b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:06.470723 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:06.496323 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_386_77 to 202410_1_386_77 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:06.499490 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_386_77} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_381_76, 202410_386_386_0] -> 202410_1_386_77 2024.10.22 01:10:06.518309 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.39 MiB. 2024.10.22 01:10:06.790641 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_369_147} MergeTask::MergeProjectionsStage: Merge sorted 32795 rows, containing 13 columns (13 merged, 0 gathered) in 3.082720375 sec., 10638.3310876842 rows/sec., 4.57 MiB/sec. 2024.10.22 01:10:06.826663 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_369_147} MergedBlockOutputStream: filled checksums 202410_1_369_147 (state Temporary) 2024.10.22 01:10:07.285198 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(970cd6ad23854094f5794cb248ce7ed6) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 970cd6ad23854094f5794cb248ce7ed6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:07.287702 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(970cd6ad23854094f5794cb248ce7ed6): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:10:07.308077 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 319 2024.10.22 01:10:07.440840 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_369_147} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_369_147 to 202410_1_369_147 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:07.487694 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:07.490959 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:07.527705 [ 1209 ] {} DatabaseCatalog: Execution took 381 ms. 2024.10.22 01:10:07.551295 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_369_147} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_368_146, 202410_369_369_0] -> 202410_1_369_147 2024.10.22 01:10:07.580763 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.44 MiB. 2024.10.22 01:10:07.615438 [ 1212 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_306_63 to 202410_310_310_0 2024.10.22 01:10:07.626891 [ 1212 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:07.643723 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_301_301_0 (state Temporary) 2024.10.22 01:10:07.692646 [ 1212 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:07.718824 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_310_64} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_306_63 to 202410_310_310_0 into Compact with storage Full 2024.10.22 01:10:07.945496 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_310_64} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:07.969221 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_310_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_306_63, total 350 rows starting from the beginning of the part 2024.10.22 01:10:08.050913 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_301_301_0 to 202410_311_311_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:08.107297 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_310_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_307_307_0, total 1 rows starting from the beginning of the part 2024.10.22 01:10:08.117774 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 319 2024.10.22 01:10:08.196291 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(a531f44242ca4968960fae50a82e0fbc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a531f44242ca4968960fae50a82e0fbc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:08.198405 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 498, file segment info: File segment: [0, 33554431], key: a531f44242ca4968960fae50a82e0fbc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:08.200666 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:08.245298 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_310_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_308_308_0, total 1 rows starting from the beginning of the part 2024.10.22 01:10:08.248182 [ 1712 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zes/tehrstlwagxfsuqgjfearqlualrfy, size 4878 2024.10.22 01:10:08.335611 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:10:08.348993 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:10:08.371716 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:10:08.392568 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:10:08.407038 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:10:08.449712 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_644_267} MergeTask::MergeProjectionsStage: Merge sorted 39531 rows, containing 13 columns (13 merged, 0 gathered) in 2.914514221 sec., 13563.495321164191 rows/sec., 2.14 MiB/sec. 2024.10.22 01:10:08.490842 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_543_166, 202410_544_544_0] 2024.10.22 01:10:08.494186 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_543_166, 202410_544_544_0] 2024.10.22 01:10:08.505304 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_543_166, 202410_544_544_0] 2024.10.22 01:10:08.520186 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:10:08.648793 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:10:08.650901 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_644_267} MergedBlockOutputStream: filled checksums 202410_1_644_267 (state Temporary) 2024.10.22 01:10:08.653507 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_310_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_309_309_0, total 1 rows starting from the beginning of the part 2024.10.22 01:10:08.706668 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_310_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_310_310_0, total 1 rows starting from the beginning of the part 2024.10.22 01:10:08.715950 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uxk/zietyegqpshabekprdmhektbiojxp (LogSeriesLimiter: on interval from 2024-10-22 01:10:03 to 2024-10-22 01:10:08 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:10:08.818939 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_644_267} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_644_267 to 202410_1_644_267 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:08.825081 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_644_267} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_643_266, 202410_644_644_0] -> 202410_1_644_267 2024.10.22 01:10:08.934453 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 3027 2024.10.22 01:10:08.976406 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 768 entries to flush up to offset 578864 2024.10.22 01:10:08.978560 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:10:09.003754 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a7d4dba3372040ddb9202ceabd10954f) : [0, 238416]: Setting download as failed: File segment: [0, 238416], key: a7d4dba3372040ddb9202ceabd10954f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:09.005143 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a7d4dba3372040ddb9202ceabd10954f): No space left in cache to reserve 238417 bytes, will continue without cache download 2024.10.22 01:10:09.020196 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a7d4dba3372040ddb9202ceabd10954f, offset: 0, size: 0 2024.10.22 01:10:09.023053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a7d4dba3372040ddb9202ceabd10954f for removal 2024.10.22 01:10:09.027798 [ 1476 ] {} CacheMetadata: Key a7d4dba3372040ddb9202ceabd10954f is removed from metadata 2024.10.22 01:10:09.032165 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 970cd6ad23854094f5794cb248ce7ed6, offset: 0, size: 0 2024.10.22 01:10:09.034990 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 970cd6ad23854094f5794cb248ce7ed6 for removal 2024.10.22 01:10:09.040651 [ 1476 ] {} CacheMetadata: Key 970cd6ad23854094f5794cb248ce7ed6 is removed from metadata 2024.10.22 01:10:09.363279 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_312_98, 202410_313_313_0] 2024.10.22 01:10:09.366527 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_312_98, 202410_313_313_0] 2024.10.22 01:10:09.382212 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_312_98, 202410_313_313_0] 2024.10.22 01:10:09.390935 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.89 GiB, free memory in arenas 7.35 MiB, will set to 1.67 GiB (RSS), difference: 50.03 MiB 2024.10.22 01:10:09.403293 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:10:09.474739 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:09.479696 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:09.671390 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 98 entries to flush up to offset 39132 2024.10.22 01:10:09.696080 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_358_358_0 (state Temporary) 2024.10.22 01:10:09.701113 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_310_64} MergeTask::MergeProjectionsStage: Merge sorted 354 rows, containing 32 columns (32 merged, 0 gathered) in 1.988838675 sec., 177.9933206498008 rows/sec., 18.01 KiB/sec. 2024.10.22 01:10:09.725264 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_310_64} MergedBlockOutputStream: filled checksums 202410_1_310_64 (state Temporary) 2024.10.22 01:10:09.768556 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_358_358_0 to 202410_365_365_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:09.806444 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 578864 2024.10.22 01:10:09.869729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(320ecba8f653cb217ac85fb424e393f0) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 320ecba8f653cb217ac85fb424e393f0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:09.872188 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(320ecba8f653cb217ac85fb424e393f0): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:10:09.906398 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:10:09.907831 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:10:10.103106 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_310_64} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_310_64 to 202410_1_310_64 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:10.206582 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_310_64} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_306_63, 202410_310_310_0] -> 202410_1_310_64 2024.10.22 01:10:10.314220 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:10.315798 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:10.391265 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:10:10.542777 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(32ea5028f4c05d770d41c191c380068e) : [0, 466231]: Setting download as failed: File segment: [0, 466231], key: 32ea5028f4c05d770d41c191c380068e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:10.544504 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(32ea5028f4c05d770d41c191c380068e): No space left in cache to reserve 466232 bytes, will continue without cache download 2024.10.22 01:10:10.571256 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 32ea5028f4c05d770d41c191c380068e, offset: 0, size: 0 2024.10.22 01:10:10.574609 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 32ea5028f4c05d770d41c191c380068e for removal 2024.10.22 01:10:10.579778 [ 1476 ] {} CacheMetadata: Key 32ea5028f4c05d770d41c191c380068e is removed from metadata 2024.10.22 01:10:10.582188 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 320ecba8f653cb217ac85fb424e393f0, offset: 0, size: 0 2024.10.22 01:10:10.583937 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 320ecba8f653cb217ac85fb424e393f0 for removal 2024.10.22 01:10:10.588619 [ 1476 ] {} CacheMetadata: Key 320ecba8f653cb217ac85fb424e393f0 is removed from metadata 2024.10.22 01:10:10.616448 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_636_636_0 (state Temporary) 2024.10.22 01:10:10.677737 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_636_636_0 to 202410_649_649_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:10.841726 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39132 2024.10.22 01:10:11.084315 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 5604 2024.10.22 01:10:11.247137 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 198 entries to flush up to offset 33894 2024.10.22 01:10:11.328231 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6b35ec9d6dc590975e05af310f8cb1e4) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 6b35ec9d6dc590975e05af310f8cb1e4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:11.329647 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6b35ec9d6dc590975e05af310f8cb1e4): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:10:11.395374 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:11.397746 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:11.419671 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.89 GiB, free memory in arenas 5.45 MiB, will set to 1.67 GiB (RSS), difference: 1.52 MiB 2024.10.22 01:10:11.533229 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_374_374_0 (state Temporary) 2024.10.22 01:10:11.673592 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:11.677656 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:11.695724 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_374_374_0 to 202410_387_387_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:11.718272 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_369_369_0 (state Temporary) 2024.10.22 01:10:11.756436 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5604 2024.10.22 01:10:11.806826 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_369_369_0 to 202410_375_375_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:11.834943 [ 1194 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_369_147 to 202410_370_370_0 2024.10.22 01:10:11.845116 [ 1194 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.74 MiB using storage policy from min volume index 0 2024.10.22 01:10:11.867064 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 33894 2024.10.22 01:10:11.851481 [ 1194 ] {} DiskLocal: Reserved 1.74 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:11.993726 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_370_148} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_369_147 to 202410_370_370_0 into Wide with storage Full 2024.10.22 01:10:12.032614 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(edd9b0ac30e6df8469e95e58fa1148c0) : [0, 350357]: Setting download as failed: File segment: [0, 350357], key: edd9b0ac30e6df8469e95e58fa1148c0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:12.051525 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_370_148} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:12.062516 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_370_148} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_369_147, total 32795 rows starting from the beginning of the part 2024.10.22 01:10:12.136594 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(edd9b0ac30e6df8469e95e58fa1148c0): No space left in cache to reserve 350358 bytes, will continue without cache download 2024.10.22 01:10:12.168227 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_370_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_370_370_0, total 211 rows starting from the beginning of the part 2024.10.22 01:10:12.186732 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: edd9b0ac30e6df8469e95e58fa1148c0, offset: 0, size: 0 2024.10.22 01:10:12.191077 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key edd9b0ac30e6df8469e95e58fa1148c0 for removal 2024.10.22 01:10:12.195309 [ 1476 ] {} CacheMetadata: Key edd9b0ac30e6df8469e95e58fa1148c0 is removed from metadata 2024.10.22 01:10:12.198367 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6b35ec9d6dc590975e05af310f8cb1e4, offset: 0, size: 0 2024.10.22 01:10:12.201033 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6b35ec9d6dc590975e05af310f8cb1e4 for removal 2024.10.22 01:10:12.205012 [ 1476 ] {} CacheMetadata: Key 6b35ec9d6dc590975e05af310f8cb1e4 is removed from metadata 2024.10.22 01:10:12.849733 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ca3348acef09ff8d3a208b54f12c694a) : [0, 150]: Setting download as failed: File segment: [0, 150], key: ca3348acef09ff8d3a208b54f12c694a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:12.852275 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ca3348acef09ff8d3a208b54f12c694a): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:10:13.467458 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.89 GiB, free memory in arenas 6.21 MiB, will set to 1.67 GiB (RSS), difference: -47.72 MiB 2024.10.22 01:10:13.524729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(76ef2971055b5e67fe12493e7f0d1d71) : [0, 250296]: Setting download as failed: File segment: [0, 250296], key: 76ef2971055b5e67fe12493e7f0d1d71, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:13.529107 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(76ef2971055b5e67fe12493e7f0d1d71): No space left in cache to reserve 250297 bytes, will continue without cache download 2024.10.22 01:10:13.680842 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 76ef2971055b5e67fe12493e7f0d1d71, offset: 0, size: 0 2024.10.22 01:10:13.684384 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 76ef2971055b5e67fe12493e7f0d1d71 for removal 2024.10.22 01:10:13.688123 [ 1476 ] {} CacheMetadata: Key 76ef2971055b5e67fe12493e7f0d1d71 is removed from metadata 2024.10.22 01:10:13.690433 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ca3348acef09ff8d3a208b54f12c694a, offset: 0, size: 0 2024.10.22 01:10:13.692064 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ca3348acef09ff8d3a208b54f12c694a for removal 2024.10.22 01:10:13.693372 [ 1476 ] {} CacheMetadata: Key ca3348acef09ff8d3a208b54f12c694a is removed from metadata 2024.10.22 01:10:14.025086 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_544_167, 202410_545_545_0] 2024.10.22 01:10:14.029546 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_544_167, 202410_545_545_0] 2024.10.22 01:10:14.159469 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_544_167, 202410_545_545_0] 2024.10.22 01:10:14.634988 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9c2dae497c1e5fbe2b7e345b12d04cfe) : [0, 150]: Setting download as failed: File segment: [0, 150], key: 9c2dae497c1e5fbe2b7e345b12d04cfe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:14.639278 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9c2dae497c1e5fbe2b7e345b12d04cfe): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:10:14.856080 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 39204 2024.10.22 01:10:15.114347 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_370_148} MergeTask::MergeProjectionsStage: Merge sorted 33006 rows, containing 13 columns (13 merged, 0 gathered) in 3.125074162 sec., 10561.669352152801 rows/sec., 4.54 MiB/sec. 2024.10.22 01:10:15.182910 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_370_148} MergedBlockOutputStream: filled checksums 202410_1_370_148 (state Temporary) 2024.10.22 01:10:15.358809 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.89 GiB, free memory in arenas 5.26 MiB, will set to 1.67 GiB (RSS), difference: 11.94 MiB 2024.10.22 01:10:15.366213 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:15.374488 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:15.441140 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_637_637_0 (state Temporary) 2024.10.22 01:10:15.590014 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_637_637_0 to 202410_650_650_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:15.614472 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_644_267 to 202410_645_645_0 2024.10.22 01:10:15.619691 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 320 2024.10.22 01:10:15.621172 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.40 MiB using storage policy from min volume index 0 2024.10.22 01:10:15.628607 [ 1252 ] {} DiskLocal: Reserved 1.40 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:15.649242 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_645_268} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_644_267 to 202410_645_645_0 into Compact with storage Full 2024.10.22 01:10:15.719330 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_645_268} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:15.736314 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_645_268} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_644_267, total 39531 rows starting from the beginning of the part 2024.10.22 01:10:15.751284 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_370_148} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_370_148 to 202410_1_370_148 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:15.756336 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_370_148} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_369_147, 202410_370_370_0] -> 202410_1_370_148 2024.10.22 01:10:15.801644 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39204 2024.10.22 01:10:15.802760 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.92 MiB. 2024.10.22 01:10:15.843110 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(64335eb27c99e22757ccf59b66a8afcf) : [0, 249552]: Setting download as failed: File segment: [0, 249552], key: 64335eb27c99e22757ccf59b66a8afcf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:15.845672 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(64335eb27c99e22757ccf59b66a8afcf): No space left in cache to reserve 249553 bytes, will continue without cache download 2024.10.22 01:10:15.895908 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 64335eb27c99e22757ccf59b66a8afcf, offset: 0, size: 0 2024.10.22 01:10:15.896175 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_645_268} MergeTreeSequentialSource: Reading 2 marks from part 202410_645_645_0, total 76 rows starting from the beginning of the part 2024.10.22 01:10:15.899518 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 64335eb27c99e22757ccf59b66a8afcf for removal 2024.10.22 01:10:15.902304 [ 1476 ] {} CacheMetadata: Key 64335eb27c99e22757ccf59b66a8afcf is removed from metadata 2024.10.22 01:10:15.905707 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:15.906712 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9c2dae497c1e5fbe2b7e345b12d04cfe, offset: 0, size: 0 2024.10.22 01:10:15.907932 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:15.907998 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9c2dae497c1e5fbe2b7e345b12d04cfe for removal 2024.10.22 01:10:15.909387 [ 1476 ] {} CacheMetadata: Key 9c2dae497c1e5fbe2b7e345b12d04cfe is removed from metadata 2024.10.22 01:10:15.960452 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_302_302_0 (state Temporary) 2024.10.22 01:10:16.065589 [ 1159 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:10:16.118173 [ 1159 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:10:16.127876 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_302_302_0 to 202410_312_312_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:16.147721 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.038032975 sec. 2024.10.22 01:10:16.228720 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 320 2024.10.22 01:10:16.601269 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5223ff725ebc2537d886b97d7d98a5b3) : [0, 150]: Setting download as failed: File segment: [0, 150], key: 5223ff725ebc2537d886b97d7d98a5b3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:16.602595 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5223ff725ebc2537d886b97d7d98a5b3): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:10:16.808655 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1538 entries to flush up to offset 580402 2024.10.22 01:10:17.015734 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_645_268/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:10:17.449288 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.89 GiB, free memory in arenas 5.68 MiB, will set to 1.67 GiB (RSS), difference: 31.20 MiB 2024.10.22 01:10:17.456713 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:17.463259 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:17.603266 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_359_359_0 (state Temporary) 2024.10.22 01:10:17.760136 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(251aa9946f5a62e4a52a0420d5a74050) : [0, 209139]: Setting download as failed: File segment: [0, 209139], key: 251aa9946f5a62e4a52a0420d5a74050, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:17.841780 [ 1186 ] {} DatabaseCatalog: Execution took 276 ms. 2024.10.22 01:10:17.865048 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(251aa9946f5a62e4a52a0420d5a74050): No space left in cache to reserve 209140 bytes, will continue without cache download 2024.10.22 01:10:17.896408 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 251aa9946f5a62e4a52a0420d5a74050, offset: 0, size: 0 2024.10.22 01:10:17.898638 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 251aa9946f5a62e4a52a0420d5a74050 for removal 2024.10.22 01:10:17.900758 [ 1476 ] {} CacheMetadata: Key 251aa9946f5a62e4a52a0420d5a74050 is removed from metadata 2024.10.22 01:10:18.003953 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5223ff725ebc2537d886b97d7d98a5b3, offset: 0, size: 0 2024.10.22 01:10:18.006892 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5223ff725ebc2537d886b97d7d98a5b3 for removal 2024.10.22 01:10:18.009707 [ 1476 ] {} CacheMetadata: Key 5223ff725ebc2537d886b97d7d98a5b3 is removed from metadata 2024.10.22 01:10:18.051554 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_359_359_0 to 202410_366_366_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:18.196075 [ 1175 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_361_72 to 202410_366_366_0 2024.10.22 01:10:18.291709 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_313_99, 202410_314_314_0] 2024.10.22 01:10:18.294670 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_313_99, 202410_314_314_0] 2024.10.22 01:10:18.300709 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 580402 2024.10.22 01:10:18.307744 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_313_99, 202410_314_314_0] 2024.10.22 01:10:18.308371 [ 1175 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:18.311617 [ 1175 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:18.350397 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_361_72 to 202410_366_366_0 into Compact with storage Full 2024.10.22 01:10:18.395409 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:18.403447 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} MergeTreeSequentialSource: Reading 73 marks from part 202410_1_361_72, total 587462 rows starting from the beginning of the part 2024.10.22 01:10:18.460860 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_362_362_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:10:18.501484 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_301_60, 202410_302_302_0, 202410_303_303_0, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0] 2024.10.22 01:10:18.505171 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_301_60, 202410_302_302_0, 202410_303_303_0, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0] 2024.10.22 01:10:18.578354 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_363_363_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:10:18.608081 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_645_268} MergeTask::MergeProjectionsStage: Merge sorted 39607 rows, containing 13 columns (13 merged, 0 gathered) in 2.97036667 sec., 13334.044042448133 rows/sec., 2.10 MiB/sec. 2024.10.22 01:10:18.629367 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_364_364_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:10:18.633238 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_301_60, 202410_302_302_0, 202410_303_303_0, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0] 2024.10.22 01:10:18.668992 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_365_365_0, total 768 rows starting from the beginning of the part 2024.10.22 01:10:18.712098 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_366_366_0, total 1538 rows starting from the beginning of the part 2024.10.22 01:10:18.784063 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_645_268} MergedBlockOutputStream: filled checksums 202410_1_645_268 (state Temporary) 2024.10.22 01:10:18.829287 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3a95da655415fd199dbcb18cb7a5c55f) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 3a95da655415fd199dbcb18cb7a5c55f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:18.832494 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3a95da655415fd199dbcb18cb7a5c55f): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:10:19.089176 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_645_268} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_645_268 to 202410_1_645_268 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:19.212074 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_645_268} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_644_267, 202410_645_645_0] -> 202410_1_645_268 2024.10.22 01:10:19.260289 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 5615 2024.10.22 01:10:19.349572 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:10:19.438667 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 172 entries to flush up to offset 34066 2024.10.22 01:10:19.680346 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:19.682858 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_545_168, 202410_546_546_0] 2024.10.22 01:10:19.683060 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:19.684828 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_545_168, 202410_546_546_0] 2024.10.22 01:10:19.694276 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_545_168, 202410_546_546_0] 2024.10.22 01:10:19.698777 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:19.701498 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:19.728434 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_375_375_0 (state Temporary) 2024.10.22 01:10:19.756395 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_370_370_0 (state Temporary) 2024.10.22 01:10:19.801652 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7f9a99ac14956d09afd673df166bbba4) : [0, 463785]: Setting download as failed: File segment: [0, 463785], key: 7f9a99ac14956d09afd673df166bbba4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:19.807702 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_370_370_0 to 202410_376_376_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:19.812048 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 90 entries to flush up to offset 39294 2024.10.22 01:10:19.812971 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7f9a99ac14956d09afd673df166bbba4): No space left in cache to reserve 463786 bytes, will continue without cache download 2024.10.22 01:10:19.855788 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7f9a99ac14956d09afd673df166bbba4, offset: 0, size: 0 2024.10.22 01:10:19.859296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7f9a99ac14956d09afd673df166bbba4 for removal 2024.10.22 01:10:19.865030 [ 1476 ] {} CacheMetadata: Key 7f9a99ac14956d09afd673df166bbba4 is removed from metadata 2024.10.22 01:10:19.879494 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_370_148 to 202410_371_371_0 2024.10.22 01:10:19.880647 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3a95da655415fd199dbcb18cb7a5c55f, offset: 0, size: 0 2024.10.22 01:10:19.883468 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3a95da655415fd199dbcb18cb7a5c55f for removal 2024.10.22 01:10:19.886250 [ 1476 ] {} CacheMetadata: Key 3a95da655415fd199dbcb18cb7a5c55f is removed from metadata 2024.10.22 01:10:19.892075 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.74 MiB using storage policy from min volume index 0 2024.10.22 01:10:19.895065 [ 1193 ] {} DiskLocal: Reserved 1.74 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:19.907712 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:10:19.912715 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:10:19.914604 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_375_375_0 to 202410_388_388_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:19.919149 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_371_149} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_370_148 to 202410_371_371_0 into Wide with storage Full 2024.10.22 01:10:19.941066 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 34066 2024.10.22 01:10:19.987357 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_371_149} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:20.080958 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5615 2024.10.22 01:10:20.108244 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_371_149} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_370_148, total 33006 rows starting from the beginning of the part 2024.10.22 01:10:20.281775 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:20.282520 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_371_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_371_371_0, total 201 rows starting from the beginning of the part 2024.10.22 01:10:20.285415 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:20.340359 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_638_638_0 (state Temporary) 2024.10.22 01:10:20.400178 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_638_638_0 to 202410_651_651_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:20.410410 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_645_268 to 202410_646_646_0 2024.10.22 01:10:20.422628 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.40 MiB using storage policy from min volume index 0 2024.10.22 01:10:20.429028 [ 1201 ] {} DiskLocal: Reserved 1.40 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:20.556885 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39294 2024.10.22 01:10:20.680472 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_646_269} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_645_268 to 202410_646_646_0 into Compact with storage Full 2024.10.22 01:10:20.727941 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7dbe1c2154014dcf0d9f1d166584dbb3) : [0, 149]: Setting download as failed: File segment: [0, 149], key: 7dbe1c2154014dcf0d9f1d166584dbb3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:20.731077 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7dbe1c2154014dcf0d9f1d166584dbb3): No space left in cache to reserve 150 bytes, will continue without cache download 2024.10.22 01:10:20.880420 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_646_269} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:20.898655 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_646_269} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_645_268, total 39607 rows starting from the beginning of the part 2024.10.22 01:10:20.951542 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_646_269} MergeTreeSequentialSource: Reading 2 marks from part 202410_646_646_0, total 72 rows starting from the beginning of the part 2024.10.22 01:10:21.408167 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.89 GiB, free memory in arenas 8.17 MiB, will set to 1.67 GiB (RSS), difference: -64.13 MiB 2024.10.22 01:10:21.409945 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ce11dc38604abceb4e8afa50541d8cc8) : [0, 169787]: Setting download as failed: File segment: [0, 169787], key: ce11dc38604abceb4e8afa50541d8cc8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:21.411058 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ce11dc38604abceb4e8afa50541d8cc8): No space left in cache to reserve 169788 bytes, will continue without cache download 2024.10.22 01:10:21.546683 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ce11dc38604abceb4e8afa50541d8cc8, offset: 0, size: 0 2024.10.22 01:10:21.553684 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ce11dc38604abceb4e8afa50541d8cc8 for removal 2024.10.22 01:10:21.560417 [ 1476 ] {} CacheMetadata: Key ce11dc38604abceb4e8afa50541d8cc8 is removed from metadata 2024.10.22 01:10:21.560168 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7dbe1c2154014dcf0d9f1d166584dbb3, offset: 0, size: 0 2024.10.22 01:10:21.667126 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7dbe1c2154014dcf0d9f1d166584dbb3 for removal 2024.10.22 01:10:21.672051 [ 1476 ] {} CacheMetadata: Key 7dbe1c2154014dcf0d9f1d166584dbb3 is removed from metadata 2024.10.22 01:10:22.007722 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(9bd2df032fdbcae4a61262aed359478d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9bd2df032fdbcae4a61262aed359478d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:10:22.022636 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 313303, file segment info: File segment: [0, 33554431], key: 9bd2df032fdbcae4a61262aed359478d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:10:22.027824 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:22.510066 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_546_169, 202410_547_547_0] 2024.10.22 01:10:22.512536 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_546_169, 202410_547_547_0] 2024.10.22 01:10:22.525141 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_546_169, 202410_547_547_0] 2024.10.22 01:10:22.693319 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_371_149/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:10:22.730663 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6ff81a80f7459e9c3cf6fdcc9b31ae44) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 6ff81a80f7459e9c3cf6fdcc9b31ae44, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:22.732751 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6ff81a80f7459e9c3cf6fdcc9b31ae44): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:10:23.131992 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_371_149} MergeTask::MergeProjectionsStage: Merge sorted 33207 rows, containing 13 columns (13 merged, 0 gathered) in 3.220363459 sec., 10311.568996100697 rows/sec., 4.43 MiB/sec. 2024.10.22 01:10:23.205494 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_371_149} MergedBlockOutputStream: filled checksums 202410_1_371_149 (state Temporary) 2024.10.22 01:10:23.386565 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_646_269} MergeTask::MergeProjectionsStage: Merge sorted 39679 rows, containing 13 columns (13 merged, 0 gathered) in 2.704153784 sec., 14673.351876203798 rows/sec., 2.31 MiB/sec. 2024.10.22 01:10:23.506650 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_371_149} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_371_149 to 202410_1_371_149 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:23.509543 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_371_149} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_370_148, 202410_371_371_0] -> 202410_1_371_149 2024.10.22 01:10:23.531088 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.91 MiB. 2024.10.22 01:10:23.568527 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(6f2eb6646b33798e0767d439f3274699) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6f2eb6646b33798e0767d439f3274699, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:10:23.588522 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 577, file segment info: File segment: [0, 33554431], key: 6f2eb6646b33798e0767d439f3274699, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:10:23.592880 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:23.692178 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_646_269} MergedBlockOutputStream: filled checksums 202410_1_646_269 (state Temporary) 2024.10.22 01:10:23.731788 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 321 2024.10.22 01:10:23.733543 [ 1675 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uec/qskmyvjyodgubladlziwfvvnsroci, size 313303 2024.10.22 01:10:23.822935 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_646_269} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_646_269 to 202410_1_646_269 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:23.852025 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_646_269} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_645_268, 202410_646_646_0] -> 202410_1_646_269 2024.10.22 01:10:23.933001 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_366_73/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:10:23.987696 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:10:24.022142 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:10:24.036879 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:10:24.045240 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:10:24.056778 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:10:24.250822 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:10:24.261788 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:10:24.280777 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:10:24.294345 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:10:24.304089 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:10:24.342314 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:24.352376 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:24.408839 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5b35e06bea22297766e5d647fcc38f85) : [0, 1235]: Setting download as failed: File segment: [0, 1235], key: 5b35e06bea22297766e5d647fcc38f85, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:24.435680 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.89 GiB, free memory in arenas 13.95 MiB, will set to 1.67 GiB (RSS), difference: 72.73 MiB 2024.10.22 01:10:24.436671 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tsi/vxjjsvcdgwlhjubqofltkumyxaroe (LogSeriesLimiter: on interval from 2024-10-22 01:10:08 to 2024-10-22 01:10:24 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:10:24.511191 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5b35e06bea22297766e5d647fcc38f85): No space left in cache to reserve 1236 bytes, will continue without cache download 2024.10.22 01:10:24.528004 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5b35e06bea22297766e5d647fcc38f85, offset: 0, size: 0 2024.10.22 01:10:24.530911 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5b35e06bea22297766e5d647fcc38f85 for removal 2024.10.22 01:10:24.531364 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_303_303_0 (state Temporary) 2024.10.22 01:10:24.537612 [ 1476 ] {} CacheMetadata: Key 5b35e06bea22297766e5d647fcc38f85 is removed from metadata 2024.10.22 01:10:24.583129 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 39383 2024.10.22 01:10:24.637293 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_303_303_0 to 202410_313_313_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:24.541548 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6ff81a80f7459e9c3cf6fdcc9b31ae44, offset: 0, size: 0 2024.10.22 01:10:24.644230 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6ff81a80f7459e9c3cf6fdcc9b31ae44 for removal 2024.10.22 01:10:24.647535 [ 1476 ] {} CacheMetadata: Key 6ff81a80f7459e9c3cf6fdcc9b31ae44 is removed from metadata 2024.10.22 01:10:24.842423 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 321 2024.10.22 01:10:25.287124 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:25.289719 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:25.303756 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 581938 2024.10.22 01:10:25.347947 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(979a3abd1eb782f01fd053e04479f1ae) : [0, 184]: Setting download as failed: File segment: [0, 184], key: 979a3abd1eb782f01fd053e04479f1ae, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:25.354496 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(979a3abd1eb782f01fd053e04479f1ae): No space left in cache to reserve 185 bytes, will continue without cache download 2024.10.22 01:10:25.422680 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_314_100, 202410_315_315_0] 2024.10.22 01:10:25.427047 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_314_100, 202410_315_315_0] 2024.10.22 01:10:25.476400 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_639_639_0 (state Temporary) 2024.10.22 01:10:25.538751 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_639_639_0 to 202410_652_652_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:25.552332 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_314_100, 202410_315_315_0] 2024.10.22 01:10:25.587218 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39383 2024.10.22 01:10:25.680192 [ 1238 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_646_269 to 202410_647_647_0 2024.10.22 01:10:25.683747 [ 1238 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.41 MiB using storage policy from min volume index 0 2024.10.22 01:10:25.686696 [ 1238 ] {} DiskLocal: Reserved 1.41 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:25.697585 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_647_270} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_646_269 to 202410_647_647_0 into Compact with storage Full 2024.10.22 01:10:25.819770 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_647_270} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:25.826472 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_647_270} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_646_269, total 39679 rows starting from the beginning of the part 2024.10.22 01:10:25.891883 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_647_270} MergeTreeSequentialSource: Reading 2 marks from part 202410_647_647_0, total 90 rows starting from the beginning of the part 2024.10.22 01:10:25.992828 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:25.996113 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:26.106442 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_360_360_0 (state Temporary) 2024.10.22 01:10:26.304467 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_360_360_0 to 202410_367_367_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:26.305450 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(683c0de3ff91adb5c73eba0653cb7a16) : [0, 953655]: Setting download as failed: File segment: [0, 953655], key: 683c0de3ff91adb5c73eba0653cb7a16, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:26.307455 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(683c0de3ff91adb5c73eba0653cb7a16): No space left in cache to reserve 953656 bytes, will continue without cache download 2024.10.22 01:10:26.350318 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 581938 2024.10.22 01:10:26.352673 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 683c0de3ff91adb5c73eba0653cb7a16, offset: 0, size: 0 2024.10.22 01:10:26.355653 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 683c0de3ff91adb5c73eba0653cb7a16 for removal 2024.10.22 01:10:26.359246 [ 1476 ] {} CacheMetadata: Key 683c0de3ff91adb5c73eba0653cb7a16 is removed from metadata 2024.10.22 01:10:26.360540 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 979a3abd1eb782f01fd053e04479f1ae, offset: 0, size: 0 2024.10.22 01:10:26.363614 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 979a3abd1eb782f01fd053e04479f1ae for removal 2024.10.22 01:10:26.369374 [ 1476 ] {} CacheMetadata: Key 979a3abd1eb782f01fd053e04479f1ae is removed from metadata 2024.10.22 01:10:26.472162 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:26.479217 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:26.627372 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.89 GiB, free memory in arenas 13.23 MiB, will set to 1.68 GiB (RSS), difference: -15.09 MiB 2024.10.22 01:10:27.184992 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(fc4942846d9f0b18b50b47217299149f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fc4942846d9f0b18b50b47217299149f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:27.191217 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 14241, file segment info: File segment: [0, 33554431], key: fc4942846d9f0b18b50b47217299149f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:27.194510 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:27.481261 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 210 entries to flush up to offset 34276 2024.10.22 01:10:27.583715 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 5627 2024.10.22 01:10:27.688813 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_647_270} MergeTask::MergeProjectionsStage: Merge sorted 39769 rows, containing 13 columns (13 merged, 0 gathered) in 1.994552659 sec., 19938.806739722182 rows/sec., 3.14 MiB/sec. 2024.10.22 01:10:27.732407 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_647_270} MergedBlockOutputStream: filled checksums 202410_1_647_270 (state Temporary) 2024.10.22 01:10:27.788550 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:27.790994 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:27.843393 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_376_376_0 (state Temporary) 2024.10.22 01:10:27.870749 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_647_270} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_647_270 to 202410_1_647_270 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:27.880197 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_647_270} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_646_269, 202410_647_647_0] -> 202410_1_647_270 2024.10.22 01:10:27.907958 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:10:27.916983 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_376_376_0 to 202410_389_389_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:28.008886 [ 1222 ] {} DatabaseCatalog: Execution took 200 ms. 2024.10.22 01:10:28.015511 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:28.018297 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:28.049306 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7ab2f242672a88e7bffd740d4d4105df) : [0, 186]: Setting download as failed: File segment: [0, 186], key: 7ab2f242672a88e7bffd740d4d4105df, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:28.050974 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7ab2f242672a88e7bffd740d4d4105df): No space left in cache to reserve 187 bytes, will continue without cache download 2024.10.22 01:10:28.079576 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5627 2024.10.22 01:10:28.311742 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_371_371_0 (state Temporary) 2024.10.22 01:10:28.384394 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_371_371_0 to 202410_377_377_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:28.402793 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_371_149 to 202410_372_372_0 2024.10.22 01:10:28.405828 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.75 MiB using storage policy from min volume index 0 2024.10.22 01:10:28.407760 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 34276 2024.10.22 01:10:28.416539 [ 1224 ] {} DiskLocal: Reserved 1.75 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:28.518766 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_547_170, 202410_548_548_0] 2024.10.22 01:10:28.521744 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_547_170, 202410_548_548_0] 2024.10.22 01:10:28.540230 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_547_170, 202410_548_548_0] 2024.10.22 01:10:28.546703 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_372_150} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_371_149 to 202410_372_372_0 into Wide with storage Full 2024.10.22 01:10:28.593688 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_372_150} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:28.602921 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_372_150} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_371_149, total 33207 rows starting from the beginning of the part 2024.10.22 01:10:28.706602 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(3b209d8c6973eb93c7dd1309982b3a2b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3b209d8c6973eb93c7dd1309982b3a2b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:28.712882 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 365, file segment info: File segment: [0, 33554431], key: 3b209d8c6973eb93c7dd1309982b3a2b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:28.716563 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:28.814521 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_372_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_372_372_0, total 183 rows starting from the beginning of the part 2024.10.22 01:10:29.060620 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3cfeae9f5515b4a210dd8a6419b609b6) : [0, 1100519]: Setting download as failed: File segment: [0, 1100519], key: 3cfeae9f5515b4a210dd8a6419b609b6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:29.067388 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3cfeae9f5515b4a210dd8a6419b609b6): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:10:29.136236 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3cfeae9f5515b4a210dd8a6419b609b6, offset: 0, size: 0 2024.10.22 01:10:29.138422 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3cfeae9f5515b4a210dd8a6419b609b6 for removal 2024.10.22 01:10:29.143054 [ 1476 ] {} CacheMetadata: Key 3cfeae9f5515b4a210dd8a6419b609b6 is removed from metadata 2024.10.22 01:10:29.251955 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ab2f242672a88e7bffd740d4d4105df, offset: 0, size: 0 2024.10.22 01:10:29.255007 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ab2f242672a88e7bffd740d4d4105df for removal 2024.10.22 01:10:29.259534 [ 1476 ] {} CacheMetadata: Key 7ab2f242672a88e7bffd740d4d4105df is removed from metadata 2024.10.22 01:10:29.409445 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_76_76_0 (state Temporary) 2024.10.22 01:10:29.603831 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 39468 2024.10.22 01:10:29.842731 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:29.846282 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:29.907211 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:10:29.908298 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:10:29.936076 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(eac1d965cd8c29f7b09462d29912b8b8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: eac1d965cd8c29f7b09462d29912b8b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:29.940306 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4887, file segment info: File segment: [0, 33554431], key: eac1d965cd8c29f7b09462d29912b8b8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:29.942871 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:30.026841 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_640_640_0 (state Temporary) 2024.10.22 01:10:30.193772 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_640_640_0 to 202410_653_653_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:30.312267 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_647_270 to 202410_648_648_0 2024.10.22 01:10:30.316978 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.41 MiB using storage policy from min volume index 0 2024.10.22 01:10:30.317420 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.89 GiB, free memory in arenas 14.68 MiB, will set to 1.69 GiB (RSS), difference: -40.08 MiB 2024.10.22 01:10:30.318605 [ 1247 ] {} DiskLocal: Reserved 1.41 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:30.340565 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39468 2024.10.22 01:10:30.561315 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e8a25220818f0603a9f28c3756247058) : [0, 154]: Setting download as failed: File segment: [0, 154], key: e8a25220818f0603a9f28c3756247058, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:30.561638 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_648_271} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_647_270 to 202410_648_648_0 into Compact with storage Full 2024.10.22 01:10:30.567998 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e8a25220818f0603a9f28c3756247058): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:10:30.787992 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_648_271} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:30.796767 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_648_271} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_647_270, total 39769 rows starting from the beginning of the part 2024.10.22 01:10:30.955597 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_648_271} MergeTreeSequentialSource: Reading 2 marks from part 202410_648_648_0, total 96 rows starting from the beginning of the part 2024.10.22 01:10:31.122976 [ 1226 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:10:31.156287 [ 1226 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:10:31.326765 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(1b51ebdc3eefb3e2f4723b6a4005ea22) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1b51ebdc3eefb3e2f4723b6a4005ea22, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:31.340812 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 498, file segment info: File segment: [0, 33554431], key: 1b51ebdc3eefb3e2f4723b6a4005ea22, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:10:31.343289 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:31.398601 [ 1676 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uxk/zietyegqpshabekprdmhektbiojxp, size 4887 2024.10.22 01:10:31.753132 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_372_150} MergeTask::MergeProjectionsStage: Merge sorted 33390 rows, containing 13 columns (13 merged, 0 gathered) in 3.214160518 sec., 10388.40462789855 rows/sec., 4.46 MiB/sec. 2024.10.22 01:10:31.808944 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_372_150} MergedBlockOutputStream: filled checksums 202410_1_372_150 (state Temporary) 2024.10.22 01:10:32.063662 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b95fde71c1fe542713da284cb1fd473f) : [0, 435423]: Setting download as failed: File segment: [0, 435423], key: b95fde71c1fe542713da284cb1fd473f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:32.064988 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b95fde71c1fe542713da284cb1fd473f): No space left in cache to reserve 435424 bytes, will continue without cache download 2024.10.22 01:10:32.199515 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b95fde71c1fe542713da284cb1fd473f, offset: 0, size: 0 2024.10.22 01:10:32.203666 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b95fde71c1fe542713da284cb1fd473f for removal 2024.10.22 01:10:32.208482 [ 1476 ] {} CacheMetadata: Key b95fde71c1fe542713da284cb1fd473f is removed from metadata 2024.10.22 01:10:32.210017 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:10:32.218530 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:10:32.210854 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e8a25220818f0603a9f28c3756247058, offset: 0, size: 0 2024.10.22 01:10:32.229035 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e8a25220818f0603a9f28c3756247058 for removal 2024.10.22 01:10:32.240513 [ 1476 ] {} CacheMetadata: Key e8a25220818f0603a9f28c3756247058 is removed from metadata 2024.10.22 01:10:32.251887 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:10:32.344967 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 322 2024.10.22 01:10:32.368910 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:10:32.391173 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:10:32.412837 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:10:32.497819 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_372_150} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_372_150 to 202410_1_372_150 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:32.502314 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_372_150} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_371_149, 202410_372_372_0] -> 202410_1_372_150 2024.10.22 01:10:32.527694 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.85 MiB. 2024.10.22 01:10:32.555359 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:10:32.588765 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:32.599322 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:32.703959 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hxw/crditzkzsmwxutjgpeivhrqnjpach (LogSeriesLimiter: on interval from 2024-10-22 01:10:24 to 2024-10-22 01:10:32 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:10:32.773262 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_304_304_0 (state Temporary) 2024.10.22 01:10:33.017297 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_648_271/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:10:33.154979 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8c5c34f8e746619943e51d9d7ac20ae0) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 8c5c34f8e746619943e51d9d7ac20ae0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:33.157765 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8c5c34f8e746619943e51d9d7ac20ae0): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:10:33.160972 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_548_171, 202410_549_549_0] 2024.10.22 01:10:33.166192 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_548_171, 202410_549_549_0] 2024.10.22 01:10:33.179123 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_548_171, 202410_549_549_0] 2024.10.22 01:10:33.285137 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_304_304_0 to 202410_314_314_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:33.356104 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 768 entries to flush up to offset 582706 2024.10.22 01:10:33.378572 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 322 2024.10.22 01:10:33.405104 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.89 GiB, free memory in arenas 13.18 MiB, will set to 1.68 GiB (RSS), difference: 40.12 MiB 2024.10.22 01:10:33.850296 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:33.852548 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:34.052596 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_361_361_0 (state Temporary) 2024.10.22 01:10:34.096520 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7c15467112ebe49f2da5f3601199a71f) : [0, 107918]: Setting download as failed: File segment: [0, 107918], key: 7c15467112ebe49f2da5f3601199a71f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:10:34.098161 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7c15467112ebe49f2da5f3601199a71f): No space left in cache to reserve 107919 bytes, will continue without cache download 2024.10.22 01:10:34.143569 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_648_271} MergeTask::MergeProjectionsStage: Merge sorted 39865 rows, containing 13 columns (13 merged, 0 gathered) in 3.69826846 sec., 10779.368894166217 rows/sec., 1.70 MiB/sec. 2024.10.22 01:10:34.155173 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7c15467112ebe49f2da5f3601199a71f, offset: 0, size: 0 2024.10.22 01:10:34.161587 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7c15467112ebe49f2da5f3601199a71f for removal 2024.10.22 01:10:34.218533 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_648_271} MergedBlockOutputStream: filled checksums 202410_1_648_271 (state Temporary) 2024.10.22 01:10:34.238875 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_361_361_0 to 202410_368_368_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:34.270918 [ 1476 ] {} CacheMetadata: Key 7c15467112ebe49f2da5f3601199a71f is removed from metadata 2024.10.22 01:10:34.275456 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8c5c34f8e746619943e51d9d7ac20ae0, offset: 0, size: 0 2024.10.22 01:10:34.277311 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8c5c34f8e746619943e51d9d7ac20ae0 for removal 2024.10.22 01:10:34.281389 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yfp/ttuwewfltqvtllcmetnvveelpgxef. 2024.10.22 01:10:34.281389 [ 1476 ] {} CacheMetadata: Key 8c5c34f8e746619943e51d9d7ac20ae0 is removed from metadata 2024.10.22 01:10:34.291998 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 582706 2024.10.22 01:10:34.357375 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 39546 2024.10.22 01:10:34.467968 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_648_271} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_648_271 to 202410_1_648_271 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:34.477740 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_648_271} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_647_270, 202410_648_648_0] -> 202410_1_648_271 2024.10.22 01:10:34.533368 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:10:34.575917 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:34.580708 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:34.751749 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_641_641_0 (state Temporary) 2024.10.22 01:10:34.994361 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_641_641_0 to 202410_654_654_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:35.009023 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_366_73/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:10:35.057349 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_648_271 to 202410_649_649_0 2024.10.22 01:10:35.065406 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39546 2024.10.22 01:10:35.066449 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.41 MiB using storage policy from min volume index 0 2024.10.22 01:10:35.084243 [ 1157 ] {} DiskLocal: Reserved 1.41 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:35.103515 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_649_272} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_648_271 to 202410_649_649_0 into Compact with storage Full 2024.10.22 01:10:35.119844 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_20_20_0 to 201403_20_20_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:35.194299 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_649_272} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:35.206534 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_649_272} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_648_271, total 39865 rows starting from the beginning of the part 2024.10.22 01:10:35.251168 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yfp/ttuwewfltqvtllcmetnvveelpgxef. 2024.10.22 01:10:35.378319 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.89 GiB, free memory in arenas 14.47 MiB, will set to 1.68 GiB (RSS), difference: 73.61 MiB 2024.10.22 01:10:35.395495 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:10:35.570556 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_649_272} MergeTreeSequentialSource: Reading 2 marks from part 202410_649_649_0, total 98 rows starting from the beginning of the part 2024.10.22 01:10:35.583308 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 5643 2024.10.22 01:10:35.710643 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:35.713916 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:35.856796 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_377_377_0 (state Temporary) 2024.10.22 01:10:35.931167 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_377_377_0 to 202410_390_390_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:35.961252 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_315_101, 202410_316_316_0] 2024.10.22 01:10:35.967076 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_315_101, 202410_316_316_0] 2024.10.22 01:10:35.984200 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_315_101, 202410_316_316_0] 2024.10.22 01:10:36.001515 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5643 2024.10.22 01:10:36.036263 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 184 entries to flush up to offset 34460 2024.10.22 01:10:36.320475 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:36.326742 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:36.496927 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_372_372_0 (state Temporary) 2024.10.22 01:10:36.797074 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_372_372_0 to 202410_378_378_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:36.832102 [ 1156 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_372_150 to 202410_373_373_0 2024.10.22 01:10:36.846484 [ 1156 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.76 MiB using storage policy from min volume index 0 2024.10.22 01:10:36.852105 [ 1156 ] {} DiskLocal: Reserved 1.76 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:36.951767 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 34460 2024.10.22 01:10:36.986503 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_373_151} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_372_150 to 202410_373_373_0 into Wide with storage Full 2024.10.22 01:10:37.013232 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_373_151} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:37.136028 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_373_151} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_372_150, total 33390 rows starting from the beginning of the part 2024.10.22 01:10:37.412427 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_373_151} MergeTreeSequentialSource: Reading 2 marks from part 202410_373_373_0, total 204 rows starting from the beginning of the part 2024.10.22 01:10:37.557281 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_76_76_0 to 202410_79_79_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:37.594604 [ 1186 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_74_15 to 202410_79_79_0 2024.10.22 01:10:37.604423 [ 1186 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.62 MiB using storage policy from min volume index 0 2024.10.22 01:10:37.606464 [ 1186 ] {} DiskLocal: Reserved 1.62 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:37.656176 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_74_15 to 202410_79_79_0 into Compact with storage Full 2024.10.22 01:10:37.813100 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.26 GiB, peak 1.89 GiB, free memory in arenas 12.81 MiB, will set to 1.44 GiB (RSS), difference: 183.43 MiB 2024.10.22 01:10:38.820848 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} MergeTask::MergeProjectionsStage: Merge sorted 593992 rows, containing 4 columns (4 merged, 0 gathered) in 20.493269838 sec., 28984.735217733778 rows/sec., 424.89 KiB/sec. 2024.10.22 01:10:38.910773 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} MergedBlockOutputStream: filled checksums 202410_1_366_73 (state Temporary) 2024.10.22 01:10:38.984085 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_366_73 to 202410_1_366_73 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:38.989593 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_361_72, 202410_366_366_0] -> 202410_1_366_73 2024.10.22 01:10:39.030024 [ 1390 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_366_73}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 01:10:39.059103 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:10:39.079966 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 39601 2024.10.22 01:10:39.095821 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_549_172, 202410_550_550_0] 2024.10.22 01:10:39.098381 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_549_172, 202410_550_550_0] 2024.10.22 01:10:39.211731 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_549_172, 202410_550_550_0] 2024.10.22 01:10:39.235307 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_649_272} MergeTask::MergeProjectionsStage: Merge sorted 39963 rows, containing 13 columns (13 merged, 0 gathered) in 4.135990624 sec., 9662.255946158548 rows/sec., 1.52 MiB/sec. 2024.10.22 01:10:39.308144 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:39.319604 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_373_151/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:10:39.327597 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:39.367196 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_649_272} MergedBlockOutputStream: filled checksums 202410_1_649_272 (state Temporary) 2024.10.22 01:10:39.388281 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_642_642_0 (state Temporary) 2024.10.22 01:10:39.606150 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.89 GiB, free memory in arenas 11.10 MiB, will set to 1.50 GiB (RSS), difference: -123.88 MiB 2024.10.22 01:10:39.681297 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_649_272} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_649_272 to 202410_1_649_272 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:39.688442 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_649_272} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_648_271, 202410_649_649_0] -> 202410_1_649_272 2024.10.22 01:10:39.696587 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_642_642_0 to 202410_655_655_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:39.703510 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:10:39.715181 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_649_272 to 202410_650_650_0 2024.10.22 01:10:39.724443 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.42 MiB using storage policy from min volume index 0 2024.10.22 01:10:39.733761 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39601 2024.10.22 01:10:39.740807 [ 1198 ] {} DiskLocal: Reserved 1.42 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:39.775495 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_650_273} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_649_272 to 202410_650_650_0 into Compact with storage Full 2024.10.22 01:10:39.827710 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_650_273} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:39.835665 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_650_273} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_649_272, total 39963 rows starting from the beginning of the part 2024.10.22 01:10:39.857395 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_650_273} MergeTreeSequentialSource: Reading 2 marks from part 202410_650_650_0, total 72 rows starting from the beginning of the part 2024.10.22 01:10:39.908035 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:10:39.913555 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:10:40.363047 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3027 2024.10.22 01:10:40.689402 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_373_151} MergeTask::MergeProjectionsStage: Merge sorted 33594 rows, containing 13 columns (13 merged, 0 gathered) in 3.823864095 sec., 8785.354072579821 rows/sec., 3.78 MiB/sec. 2024.10.22 01:10:40.749485 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_373_151} MergedBlockOutputStream: filled checksums 202410_1_373_151 (state Temporary) 2024.10.22 01:10:40.882262 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 323 2024.10.22 01:10:41.113204 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 20 2024.10.22 01:10:41.260706 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:41.267388 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:41.276535 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(76ed00f48ff5d936685fbb4a7546980d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 76ed00f48ff5d936685fbb4a7546980d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:41.286152 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 302507, file segment info: File segment: [0, 33554431], key: 76ed00f48ff5d936685fbb4a7546980d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:41.289408 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:41.294412 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 584242 2024.10.22 01:10:41.294679 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:41.299290 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:41.333778 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_20_20_0 (state Temporary) 2024.10.22 01:10:41.387255 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_373_151} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_373_151 to 202410_1_373_151 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:41.389754 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_373_151} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_372_150, 202410_373_373_0] -> 202410_1_373_151 2024.10.22 01:10:41.407340 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.91 MiB. 2024.10.22 01:10:41.500387 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_20_20_0 to 202410_20_20_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:41.556378 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.89 GiB, free memory in arenas 9.45 MiB, will set to 1.51 GiB (RSS), difference: 15.39 MiB 2024.10.22 01:10:41.579193 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_305_305_0 (state Temporary) 2024.10.22 01:10:41.580569 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 20 2024.10.22 01:10:41.625059 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_305_305_0 to 202410_315_315_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:41.672943 [ 1162 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_310_64 to 202410_315_315_0 2024.10.22 01:10:41.689427 [ 1162 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:41.692014 [ 1162 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:41.715208 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 323 2024.10.22 01:10:41.827497 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_310_64 to 202410_315_315_0 into Compact with storage Full 2024.10.22 01:10:41.851061 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_650_273} MergeTask::MergeProjectionsStage: Merge sorted 40035 rows, containing 13 columns (13 merged, 0 gathered) in 2.077728564 sec., 19268.638210818764 rows/sec., 3.04 MiB/sec. 2024.10.22 01:10:42.080227 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:42.102505 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_310_64, total 354 rows starting from the beginning of the part 2024.10.22 01:10:42.136249 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_316_102, 202410_317_317_0] 2024.10.22 01:10:42.138510 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_316_102, 202410_317_317_0] 2024.10.22 01:10:42.149265 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_316_102, 202410_317_317_0] 2024.10.22 01:10:42.164731 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_326_65, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0, 202410_330_330_0, 202410_331_331_0] 2024.10.22 01:10:42.168451 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_311_311_0, total 1 rows starting from the beginning of the part 2024.10.22 01:10:42.168949 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_326_65, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0, 202410_330_330_0, 202410_331_331_0] 2024.10.22 01:10:42.209331 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_326_65, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0, 202410_330_330_0, 202410_331_331_0] 2024.10.22 01:10:42.212915 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:42.216591 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:42.238228 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_650_273} MergedBlockOutputStream: filled checksums 202410_1_650_273 (state Temporary) 2024.10.22 01:10:42.250908 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_312_312_0, total 1 rows starting from the beginning of the part 2024.10.22 01:10:42.397939 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(77bb2587621b9d45be807ad7114f0b0c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 77bb2587621b9d45be807ad7114f0b0c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:42.401060 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 577, file segment info: File segment: [0, 33554431], key: 77bb2587621b9d45be807ad7114f0b0c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:10:42.402992 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_650_273} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_650_273 to 202410_1_650_273 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:42.405409 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_313_313_0, total 1 rows starting from the beginning of the part 2024.10.22 01:10:42.405516 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:42.406674 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_650_273} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_649_272, 202410_650_650_0] -> 202410_1_650_273 2024.10.22 01:10:42.440867 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_314_314_0, total 1 rows starting from the beginning of the part 2024.10.22 01:10:42.450760 [ 1688 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tsi/vxjjsvcdgwlhjubqofltkumyxaroe, size 302507 2024.10.22 01:10:42.486999 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_315_315_0, total 1 rows starting from the beginning of the part 2024.10.22 01:10:42.537682 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:10:42.583806 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_362_362_0 (state Temporary) 2024.10.22 01:10:42.793829 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:10:42.832486 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:10:42.855938 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:10:42.878404 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:10:42.895100 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:10:42.922856 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_362_362_0 to 202410_369_369_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:42.923628 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:10:43.051369 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:10:43.082747 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 584242 2024.10.22 01:10:43.180105 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:10:43.211256 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:10:43.397024 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tda/jazfrqidqdwwseitnbkmehevzngoj (LogSeriesLimiter: on interval from 2024-10-22 01:10:32 to 2024-10-22 01:10:43 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:10:43.492709 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_550_173, 202410_551_551_0] 2024.10.22 01:10:43.493081 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.89 GiB, free memory in arenas 8.87 MiB, will set to 1.55 GiB (RSS), difference: -33.94 MiB 2024.10.22 01:10:43.495480 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_550_173, 202410_551_551_0] 2024.10.22 01:10:43.504361 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_550_173, 202410_551_551_0] 2024.10.22 01:10:43.505124 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 5666 2024.10.22 01:10:43.751377 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 94 entries to flush up to offset 39695 2024.10.22 01:10:43.833318 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:43.838727 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:43.891071 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_378_378_0 (state Temporary) 2024.10.22 01:10:43.931969 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_378_378_0 to 202410_391_391_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:43.970218 [ 1263 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 01:10:43.980614 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5666 2024.10.22 01:10:44.058396 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} MergeTask::MergeProjectionsStage: Merge sorted 359 rows, containing 32 columns (32 merged, 0 gathered) in 2.357625368 sec., 152.27186001334204 rows/sec., 15.48 KiB/sec. 2024.10.22 01:10:44.077950 [ 1195 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_386_77 to 202410_391_391_0 2024.10.22 01:10:44.094564 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} MergedBlockOutputStream: filled checksums 202410_1_315_65 (state Temporary) 2024.10.22 01:10:44.135550 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:44.138317 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:44.186696 [ 1195 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:44.189074 [ 1195 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:44.193178 [ 1195 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 01:10:44.202358 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_386_77 to 202410_391_391_0 into Compact with storage Full 2024.10.22 01:10:44.213978 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_643_643_0 (state Temporary) 2024.10.22 01:10:44.314461 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:44.319944 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_386_77, total 5682 rows starting from the beginning of the part 2024.10.22 01:10:44.340525 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_387_387_0, total 28 rows starting from the beginning of the part 2024.10.22 01:10:44.358193 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_388_388_0, total 11 rows starting from the beginning of the part 2024.10.22 01:10:44.379406 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_643_643_0 to 202410_656_656_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:44.383820 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_389_389_0, total 12 rows starting from the beginning of the part 2024.10.22 01:10:44.396434 [ 1238 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_650_273 to 202410_651_651_0 2024.10.22 01:10:44.402104 [ 1238 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.42 MiB using storage policy from min volume index 0 2024.10.22 01:10:44.404500 [ 1238 ] {} DiskLocal: Reserved 1.42 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:44.410516 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39695 2024.10.22 01:10:44.421080 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_651_274} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_650_273 to 202410_651_651_0 into Compact with storage Full 2024.10.22 01:10:44.484230 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 213 entries to flush up to offset 34673 2024.10.22 01:10:44.487475 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_315_65 to 202410_1_315_65 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:44.490826 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_315_65} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_310_64, 202410_315_315_0] -> 202410_1_315_65 2024.10.22 01:10:44.492677 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_390_390_0, total 16 rows starting from the beginning of the part 2024.10.22 01:10:44.501447 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_391_391_0, total 23 rows starting from the beginning of the part 2024.10.22 01:10:44.554546 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:10:44.580734 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_651_274} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:44.592428 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_651_274} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_650_273, total 40035 rows starting from the beginning of the part 2024.10.22 01:10:44.680519 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:44.690463 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:44.735043 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_651_274} MergeTreeSequentialSource: Reading 2 marks from part 202410_651_651_0, total 90 rows starting from the beginning of the part 2024.10.22 01:10:44.746554 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_373_373_0 (state Temporary) 2024.10.22 01:10:44.813164 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_373_373_0 to 202410_379_379_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:44.851996 [ 1213 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_373_151 to 202410_374_374_0 2024.10.22 01:10:44.859463 [ 1213 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.77 MiB using storage policy from min volume index 0 2024.10.22 01:10:44.863531 [ 1213 ] {} DiskLocal: Reserved 1.77 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:44.875956 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 34673 2024.10.22 01:10:45.018941 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_374_152} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_373_151 to 202410_374_374_0 into Wide with storage Full 2024.10.22 01:10:45.065641 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_374_152} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:45.077349 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_374_152} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_373_151, total 33594 rows starting from the beginning of the part 2024.10.22 01:10:45.343098 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_374_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_374_374_0, total 206 rows starting from the beginning of the part 2024.10.22 01:10:45.407359 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} MergeTask::MergeProjectionsStage: Merge sorted 5772 rows, containing 12 columns (12 merged, 0 gathered) in 1.211214261 sec., 4765.4656866692885 rows/sec., 752.84 KiB/sec. 2024.10.22 01:10:45.529437 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} MergedBlockOutputStream: filled checksums 202410_1_391_78 (state Temporary) 2024.10.22 01:10:45.718612 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.89 GiB, free memory in arenas 7.88 MiB, will set to 1.59 GiB (RSS), difference: -57.25 MiB 2024.10.22 01:10:45.824460 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_391_78 to 202410_1_391_78 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:45.828795 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_391_78} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_386_77, 202410_391_391_0] -> 202410_1_391_78 2024.10.22 01:10:45.955419 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.39 MiB. 2024.10.22 01:10:46.035664 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:10:46.166435 [ 1223 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:10:46.233014 [ 1223 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:10:46.301356 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:46.888410 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_651_274} MergeTask::MergeProjectionsStage: Merge sorted 40125 rows, containing 13 columns (13 merged, 0 gathered) in 2.455007829 sec., 16344.143397841686 rows/sec., 2.58 MiB/sec. 2024.10.22 01:10:47.101325 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_74_15, total 2893 rows starting from the beginning of the part 2024.10.22 01:10:47.306669 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_651_274} MergedBlockOutputStream: filled checksums 202410_1_651_274 (state Temporary) 2024.10.22 01:10:47.678617 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.89 GiB, free memory in arenas 7.64 MiB, will set to 1.65 GiB (RSS), difference: -61.49 MiB 2024.10.22 01:10:47.908325 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 3066 2024.10.22 01:10:48.050741 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_651_274} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_651_274 to 202410_1_651_274 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:48.056549 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_651_274} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_650_273, 202410_651_651_0] -> 202410_1_651_274 2024.10.22 01:10:48.102548 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:10:48.239045 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_551_174, 202410_552_552_0] 2024.10.22 01:10:48.257265 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_551_174, 202410_552_552_0] 2024.10.22 01:10:48.272811 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_551_174, 202410_552_552_0] 2024.10.22 01:10:48.394469 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_374_152} MergeTask::MergeProjectionsStage: Merge sorted 33800 rows, containing 13 columns (13 merged, 0 gathered) in 3.48535751 sec., 9697.713908264177 rows/sec., 4.17 MiB/sec. 2024.10.22 01:10:48.425523 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 39766 2024.10.22 01:10:48.462304 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_374_152} MergedBlockOutputStream: filled checksums 202410_1_374_152 (state Temporary) 2024.10.22 01:10:48.590892 [ 1234 ] {} DatabaseCatalog: Execution took 471 ms. 2024.10.22 01:10:48.868614 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:48.967332 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_75_75_0, total 43 rows starting from the beginning of the part 2024.10.22 01:10:48.977134 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:49.077007 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_644_644_0 (state Temporary) 2024.10.22 01:10:49.104341 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_374_152} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_374_152 to 202410_1_374_152 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:49.108026 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_374_152} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_373_151, 202410_374_374_0] -> 202410_1_374_152 2024.10.22 01:10:49.242158 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.91 MiB. 2024.10.22 01:10:49.345359 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_644_644_0 to 202410_657_657_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:49.469680 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.89 GiB, free memory in arenas 6.66 MiB, will set to 1.66 GiB (RSS), difference: -61.31 MiB 2024.10.22 01:10:49.485038 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_651_274 to 202410_652_652_0 2024.10.22 01:10:49.490736 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.42 MiB using storage policy from min volume index 0 2024.10.22 01:10:49.492806 [ 1227 ] {} DiskLocal: Reserved 1.42 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:49.558845 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39766 2024.10.22 01:10:49.621003 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_652_275} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_651_274 to 202410_652_652_0 into Compact with storage Full 2024.10.22 01:10:49.657021 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_652_275} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:49.666910 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_652_275} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_651_274, total 40125 rows starting from the beginning of the part 2024.10.22 01:10:49.737628 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_652_275} MergeTreeSequentialSource: Reading 2 marks from part 202410_652_652_0, total 89 rows starting from the beginning of the part 2024.10.22 01:10:49.909195 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:10:49.913018 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:10:50.087144 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1759 entries to flush up to offset 586001 2024.10.22 01:10:50.188188 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_317_103, 202410_318_318_0] 2024.10.22 01:10:50.190762 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_317_103, 202410_318_318_0] 2024.10.22 01:10:50.313511 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_317_103, 202410_318_318_0] 2024.10.22 01:10:51.033019 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:51.036830 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:51.248160 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_76_76_0, total 39 rows starting from the beginning of the part 2024.10.22 01:10:51.292559 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_363_363_0 (state Temporary) 2024.10.22 01:10:51.361595 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_363_363_0 to 202410_370_370_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:51.459583 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 586001 2024.10.22 01:10:51.482794 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 5691 2024.10.22 01:10:51.695396 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:51.701237 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:51.756995 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.96 GiB, free memory in arenas 7.55 MiB, will set to 1.91 GiB (RSS), difference: 198.54 MiB 2024.10.22 01:10:51.837670 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_379_379_0 (state Temporary) 2024.10.22 01:10:51.848628 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_652_275} MergeTask::MergeProjectionsStage: Merge sorted 40214 rows, containing 13 columns (13 merged, 0 gathered) in 2.336509473 sec., 17211.143573223595 rows/sec., 2.71 MiB/sec. 2024.10.22 01:10:51.893765 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_379_379_0 to 202410_392_392_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:51.933015 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_652_275} MergedBlockOutputStream: filled checksums 202410_1_652_275 (state Temporary) 2024.10.22 01:10:51.964878 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5691 2024.10.22 01:10:52.092961 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_652_275} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_652_275 to 202410_1_652_275 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:52.120339 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_652_275} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_651_274, 202410_652_652_0] -> 202410_1_652_275 2024.10.22 01:10:52.249073 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:10:52.490567 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 241 entries to flush up to offset 34914 2024.10.22 01:10:52.947603 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_77_77_0, total 41 rows starting from the beginning of the part 2024.10.22 01:10:53.033113 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:53.039981 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:53.101499 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_374_374_0 (state Temporary) 2024.10.22 01:10:53.332530 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_552_175, 202410_553_553_0] 2024.10.22 01:10:53.336210 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_552_175, 202410_553_553_0] 2024.10.22 01:10:53.348837 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_552_175, 202410_553_553_0] 2024.10.22 01:10:53.366082 [ 1168 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 251 ms. 2024.10.22 01:10:53.379191 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_374_374_0 to 202410_380_380_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:53.498144 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.96 GiB, free memory in arenas 7.88 MiB, will set to 1.67 GiB (RSS), difference: 93.43 MiB 2024.10.22 01:10:53.502440 [ 1241 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_374_152 to 202410_375_375_0 2024.10.22 01:10:53.506677 [ 1241 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.78 MiB using storage policy from min volume index 0 2024.10.22 01:10:53.517362 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 34914 2024.10.22 01:10:53.536158 [ 1241 ] {} DiskLocal: Reserved 1.78 MiB on local disk `default`, having unreserved 297.41 GiB. 2024.10.22 01:10:53.579978 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_375_153} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_374_152 to 202410_375_375_0 into Wide with storage Full 2024.10.22 01:10:53.648022 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 495.65 MiB using storage policy from min volume index 0 2024.10.22 01:10:53.690906 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 39825 2024.10.22 01:10:53.733107 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_375_153} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:53.759281 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_375_153} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_374_152, total 33800 rows starting from the beginning of the part 2024.10.22 01:10:54.036558 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_375_153} MergeTreeSequentialSource: Reading 2 marks from part 202410_375_375_0, total 198 rows starting from the beginning of the part 2024.10.22 01:10:54.134785 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:54.156286 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:54.346310 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_645_645_0 (state Temporary) 2024.10.22 01:10:54.364397 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_78_78_0, total 36 rows starting from the beginning of the part 2024.10.22 01:10:54.596124 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_645_645_0 to 202410_658_658_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:54.650794 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_652_275 to 202410_653_653_0 2024.10.22 01:10:54.660071 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.43 MiB using storage policy from min volume index 0 2024.10.22 01:10:54.664055 [ 1180 ] {} DiskLocal: Reserved 1.43 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:54.692991 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_653_276} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_652_275 to 202410_653_653_0 into Compact with storage Full 2024.10.22 01:10:54.756978 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39825 2024.10.22 01:10:54.851666 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_653_276} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:54.856900 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_653_276} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_652_275, total 40214 rows starting from the beginning of the part 2024.10.22 01:10:54.933773 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_653_276} MergeTreeSequentialSource: Reading 2 marks from part 202410_653_653_0, total 85 rows starting from the beginning of the part 2024.10.22 01:10:55.182935 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(44f336792bad7b0ea672d2e80a86c72c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 44f336792bad7b0ea672d2e80a86c72c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:10:55.190207 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 8612, file segment info: File segment: [0, 33554431], key: 44f336792bad7b0ea672d2e80a86c72c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:10:55.194272 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:55.299157 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wpm/puicreeujtmagrdirsahrtverpkun (LogSeriesLimiter: on interval from 2024-10-22 01:10:43 to 2024-10-22 01:10:55 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:10:55.411903 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.96 GiB, free memory in arenas 6.20 MiB, will set to 1.67 GiB (RSS), difference: -67.10 MiB 2024.10.22 01:10:55.494285 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_375_153/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:10:56.002792 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(44cb6b0f4f7a00b9c7eefe8b68282699) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 44cb6b0f4f7a00b9c7eefe8b68282699, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:10:56.005033 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 370, file segment info: File segment: [0, 33554431], key: 44cb6b0f4f7a00b9c7eefe8b68282699, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:10:56.007775 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:56.652634 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(da1e72161092a71196d0b8c76ba7e76a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: da1e72161092a71196d0b8c76ba7e76a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:10:56.653996 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4521, file segment info: File segment: [0, 33554431], key: da1e72161092a71196d0b8c76ba7e76a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:10:56.656424 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:56.681040 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_79_79_0, total 37 rows starting from the beginning of the part 2024.10.22 01:10:56.717438 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 324 2024.10.22 01:10:57.113895 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:57.128036 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:57.223460 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_375_153} MergeTask::MergeProjectionsStage: Merge sorted 33998 rows, containing 13 columns (13 merged, 0 gathered) in 3.657654905 sec., 9295.026699627913 rows/sec., 4.00 MiB/sec. 2024.10.22 01:10:57.323686 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_375_153} MergedBlockOutputStream: filled checksums 202410_1_375_153 (state Temporary) 2024.10.22 01:10:57.417636 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_306_306_0 (state Temporary) 2024.10.22 01:10:57.430979 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_653_276} MergeTask::MergeProjectionsStage: Merge sorted 40299 rows, containing 13 columns (13 merged, 0 gathered) in 2.721465875 sec., 14807.828519988332 rows/sec., 2.34 MiB/sec. 2024.10.22 01:10:57.486059 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_553_176, 202410_554_554_0] 2024.10.22 01:10:57.488957 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_553_176, 202410_554_554_0] 2024.10.22 01:10:57.533482 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_306_306_0 to 202410_316_316_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:57.582919 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.80 GiB, peak 1.96 GiB, free memory in arenas 5.38 MiB, will set to 1.68 GiB (RSS), difference: -121.34 MiB 2024.10.22 01:10:57.686140 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_653_276} MergedBlockOutputStream: filled checksums 202410_1_653_276 (state Temporary) 2024.10.22 01:10:57.698686 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_553_176, 202410_554_554_0] 2024.10.22 01:10:57.708088 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_263_54, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0, 202410_267_267_0, 202410_268_268_0] 2024.10.22 01:10:57.711906 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_263_54, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0, 202410_267_267_0, 202410_268_268_0] 2024.10.22 01:10:57.745226 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_263_54, 202410_264_264_0, 202410_265_265_0, 202410_266_266_0, 202410_267_267_0, 202410_268_268_0] 2024.10.22 01:10:57.755721 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(2d99f3f85fdaa21b5ec01fd7a07da00a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2d99f3f85fdaa21b5ec01fd7a07da00a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:10:57.758978 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 495, file segment info: File segment: [0, 33554431], key: 2d99f3f85fdaa21b5ec01fd7a07da00a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:10:57.763437 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:10:57.807961 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 324 2024.10.22 01:10:57.857232 [ 1695 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hxw/crditzkzsmwxutjgpeivhrqnjpach, size 4521 2024.10.22 01:10:57.915079 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_653_276} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_653_276 to 202410_1_653_276 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:57.920085 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_653_276} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_652_275, 202410_653_653_0] -> 202410_1_653_276 2024.10.22 01:10:58.062078 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:10:58.112681 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:10:58.216007 [ 1203 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 211 ms. 2024.10.22 01:10:58.232579 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:10:58.257357 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:10:58.274826 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_375_153} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_375_153 to 202410_1_375_153 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:58.379449 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_375_153} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_374_152, 202410_375_375_0] -> 202410_1_375_153 2024.10.22 01:10:58.384722 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:10:58.400769 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:10:58.406392 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.91 MiB. 2024.10.22 01:10:58.472655 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1699 entries to flush up to offset 587700 2024.10.22 01:10:58.530490 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:10:58.575078 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:10:58.771217 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 39893 2024.10.22 01:10:59.080105 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:59.085458 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:59.149644 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_646_646_0 (state Temporary) 2024.10.22 01:10:59.213395 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:59.221191 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:59.299662 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_646_646_0 to 202410_659_659_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:59.349996 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_364_364_0 (state Temporary) 2024.10.22 01:10:59.448208 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_653_276 to 202410_654_654_0 2024.10.22 01:10:59.458489 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.43 MiB using storage policy from min volume index 0 2024.10.22 01:10:59.464200 [ 1161 ] {} DiskLocal: Reserved 1.43 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:59.473716 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 5705 2024.10.22 01:10:59.485130 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.84 GiB, peak 1.96 GiB, free memory in arenas 4.55 MiB, will set to 1.67 GiB (RSS), difference: -169.48 MiB 2024.10.22 01:10:59.488127 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39893 2024.10.22 01:10:59.493185 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_654_277} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_653_276 to 202410_654_654_0 into Compact with storage Full 2024.10.22 01:10:59.506229 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_364_364_0 to 202410_371_371_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:59.533228 [ 1152 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_366_73 to 202410_371_371_0 2024.10.22 01:10:59.550839 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 587700 2024.10.22 01:10:59.559806 [ 1152 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:59.569165 [ 1152 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:59.616757 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:10:59.624723 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_654_277} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:59.640307 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_654_277} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_653_276, total 40299 rows starting from the beginning of the part 2024.10.22 01:10:59.680727 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:10:59.689538 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_366_73 to 202410_371_371_0 into Compact with storage Full 2024.10.22 01:10:59.707760 [ 1245 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:10:59.735623 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:10:59.751923 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} MergeTreeSequentialSource: Reading 74 marks from part 202410_1_366_73, total 593992 rows starting from the beginning of the part 2024.10.22 01:10:59.764607 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_380_380_0 (state Temporary) 2024.10.22 01:10:59.791560 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_367_367_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:10:59.805113 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_368_368_0, total 768 rows starting from the beginning of the part 2024.10.22 01:10:59.816896 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_369_369_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:10:59.838305 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_370_370_0, total 1759 rows starting from the beginning of the part 2024.10.22 01:10:59.866507 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_371_371_0, total 1699 rows starting from the beginning of the part 2024.10.22 01:10:59.896329 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_380_380_0 to 202410_393_393_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:10:59.909834 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:10:59.912999 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:11:00.004865 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_654_277} MergeTreeSequentialSource: Reading 2 marks from part 202410_654_654_0, total 78 rows starting from the beginning of the part 2024.10.22 01:11:00.014685 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tqp/adppnoclfyickzhpohontzvygeroz (LogSeriesLimiter: on interval from 2024-10-22 01:10:55 to 2024-10-22 01:11:00 accepted series 1 / 247 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:11:00.051839 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5705 2024.10.22 01:11:01.082155 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 236 entries to flush up to offset 35150 2024.10.22 01:11:01.155020 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_306_61, 202410_307_307_0, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0] 2024.10.22 01:11:01.156995 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_306_61, 202410_307_307_0, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0] 2024.10.22 01:11:01.182440 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_306_61, 202410_307_307_0, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0, 202410_311_311_0] 2024.10.22 01:11:01.239734 [ 1269 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:11:01.400494 [ 1269 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:11:01.425327 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.82 GiB, peak 1.96 GiB, free memory in arenas 5.38 MiB, will set to 1.68 GiB (RSS), difference: -147.74 MiB 2024.10.22 01:11:01.436357 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:01.438383 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:01.511852 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_375_375_0 (state Temporary) 2024.10.22 01:11:01.583479 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_375_375_0 to 202410_381_381_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:01.620902 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 35150 2024.10.22 01:11:01.638450 [ 1165 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_375_153 to 202410_376_376_0 2024.10.22 01:11:01.643502 [ 1165 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.79 MiB using storage policy from min volume index 0 2024.10.22 01:11:01.648058 [ 1165 ] {} DiskLocal: Reserved 1.79 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:01.685460 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_376_154} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_375_153 to 202410_376_376_0 into Wide with storage Full 2024.10.22 01:11:01.729221 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_376_154} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:01.739162 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_376_154} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_375_153, total 33998 rows starting from the beginning of the part 2024.10.22 01:11:01.771909 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_318_104, 202410_319_319_0] 2024.10.22 01:11:01.775532 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_318_104, 202410_319_319_0] 2024.10.22 01:11:01.792099 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_318_104, 202410_319_319_0] 2024.10.22 01:11:01.880908 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_376_154} MergeTreeSequentialSource: Reading 2 marks from part 202410_376_376_0, total 172 rows starting from the beginning of the part 2024.10.22 01:11:02.675738 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_554_177, 202410_555_555_0] 2024.10.22 01:11:02.680380 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_554_177, 202410_555_555_0] 2024.10.22 01:11:02.696476 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_554_177, 202410_555_555_0] 2024.10.22 01:11:02.739484 [ 1213 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 291 ms. 2024.10.22 01:11:02.823230 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_654_277} MergeTask::MergeProjectionsStage: Merge sorted 40377 rows, containing 13 columns (13 merged, 0 gathered) in 3.336977282 sec., 12099.872605605591 rows/sec., 1.91 MiB/sec. 2024.10.22 01:11:02.885535 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_654_277} MergedBlockOutputStream: filled checksums 202410_1_654_277 (state Temporary) 2024.10.22 01:11:02.968852 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_654_277} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_654_277 to 202410_1_654_277 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:02.980402 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_654_277} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_653_276, 202410_654_654_0] -> 202410_1_654_277 2024.10.22 01:11:03.016151 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:11:03.147847 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_371_74/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:11:03.510778 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 39965 2024.10.22 01:11:03.760610 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:03.778582 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:03.804821 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 01:11:03.818956 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:03.823128 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:03.892395 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_647_647_0 (state Temporary) 2024.10.22 01:11:03.904001 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.80 GiB, peak 1.96 GiB, free memory in arenas 9.12 MiB, will set to 1.69 GiB (RSS), difference: -115.78 MiB 2024.10.22 01:11:03.980278 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_647_647_0 to 202410_660_660_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:03.998808 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_654_277 to 202410_655_655_0 2024.10.22 01:11:04.013477 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 39965 2024.10.22 01:11:04.105008 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.43 MiB using storage policy from min volume index 0 2024.10.22 01:11:04.109128 [ 1223 ] {} DiskLocal: Reserved 1.43 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:04.248947 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_655_278} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_654_277 to 202410_655_655_0 into Compact with storage Full 2024.10.22 01:11:04.494601 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_655_278} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:04.511855 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_655_278} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_654_277, total 40377 rows starting from the beginning of the part 2024.10.22 01:11:04.734660 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_655_278} MergeTreeSequentialSource: Reading 2 marks from part 202410_655_655_0, total 55 rows starting from the beginning of the part 2024.10.22 01:11:05.045757 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(6535868ab3c0774ad33d8dca37d7bec0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6535868ab3c0774ad33d8dca37d7bec0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:11:05.064599 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 487824, file segment info: File segment: [0, 33554431], key: 6535868ab3c0774ad33d8dca37d7bec0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:11:05.079122 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:05.211721 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_59_12, 202410_60_60_0, 202410_61_61_0, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0] 2024.10.22 01:11:05.217722 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_59_12, 202410_60_60_0, 202410_61_61_0, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0] 2024.10.22 01:11:05.231386 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_79_16/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:11:05.257380 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_59_12, 202410_60_60_0, 202410_61_61_0, 202410_62_62_0, 202410_63_63_0, 202410_64_64_0] 2024.10.22 01:11:05.316794 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 325 2024.10.22 01:11:05.403875 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_376_154} MergeTask::MergeProjectionsStage: Merge sorted 34170 rows, containing 13 columns (13 merged, 0 gathered) in 3.722591679 sec., 9179.08891076098 rows/sec., 3.95 MiB/sec. 2024.10.22 01:11:05.513790 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_376_154} MergedBlockOutputStream: filled checksums 202410_1_376_154 (state Temporary) 2024.10.22 01:11:05.818997 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.96 GiB, free memory in arenas 10.77 MiB, will set to 1.71 GiB (RSS), difference: 43.74 MiB 2024.10.22 01:11:05.837106 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:05.840328 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:06.101168 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ab159d9b6e5db5bb131b6c0b75147c19) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ab159d9b6e5db5bb131b6c0b75147c19, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:11:06.104055 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1154, file segment info: File segment: [0, 33554431], key: ab159d9b6e5db5bb131b6c0b75147c19, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:11:06.110372 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:06.125337 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_307_307_0 (state Temporary) 2024.10.22 01:11:06.213495 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_376_154} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_376_154 to 202410_1_376_154 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:06.219207 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_376_154} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_375_153, 202410_376_376_0] -> 202410_1_376_154 2024.10.22 01:11:06.270623 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_307_307_0 to 202410_317_317_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:06.272184 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.85 MiB. 2024.10.22 01:11:06.278951 [ 1725 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tda/jazfrqidqdwwseitnbkmehevzngoj, size 487824 2024.10.22 01:11:06.309496 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Params 2024.10.22 01:11:06.331220 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Params 2024.10.22 01:11:06.375310 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Params 2024.10.22 01:11:06.399419 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Params 2024.10.22 01:11:06.522344 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Params 2024.10.22 01:11:06.558357 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1378 entries to flush up to offset 589078 2024.10.22 01:11:06.585777 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Params 2024.10.22 01:11:06.591929 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 325 2024.10.22 01:11:06.620360 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Params 2024.10.22 01:11:06.677988 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Params 2024.10.22 01:11:06.747447 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Params 2024.10.22 01:11:06.784698 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_555_178, 202410_556_556_0] 2024.10.22 01:11:06.787053 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_555_178, 202410_556_556_0] 2024.10.22 01:11:06.803780 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_555_178, 202410_556_556_0] 2024.10.22 01:11:07.010743 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mcw/pwgyuqsqkcchhzehrwlkajtzvgxvh (LogSeriesLimiter: on interval from 2024-10-22 01:11:00 to 2024-10-22 01:11:07 accepted series 1 / 110 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:11:07.302938 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_319_105, 202410_320_320_0] 2024.10.22 01:11:07.305548 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_319_105, 202410_320_320_0] 2024.10.22 01:11:07.312708 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:07.416067 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_319_105, 202410_320_320_0] 2024.10.22 01:11:07.511329 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:07.536714 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_655_278} MergeTask::MergeProjectionsStage: Merge sorted 40432 rows, containing 13 columns (13 merged, 0 gathered) in 3.402868835 sec., 11881.739191396133 rows/sec., 1.88 MiB/sec. 2024.10.22 01:11:07.556599 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 5716 2024.10.22 01:11:07.685452 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_365_365_0 (state Temporary) 2024.10.22 01:11:07.751368 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.96 GiB, free memory in arenas 9.99 MiB, will set to 1.72 GiB (RSS), difference: -4.15 MiB 2024.10.22 01:11:07.810344 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_365_365_0 to 202410_372_372_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:07.856912 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 589078 2024.10.22 01:11:07.863792 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_655_278} MergedBlockOutputStream: filled checksums 202410_1_655_278 (state Temporary) 2024.10.22 01:11:08.005650 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_655_278} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_655_278 to 202410_1_655_278 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:08.011002 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_655_278} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_654_277, 202410_655_655_0] -> 202410_1_655_278 2024.10.22 01:11:08.022697 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 40033 2024.10.22 01:11:08.039706 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:11:08.104075 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:08.107861 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:08.167759 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_381_381_0 (state Temporary) 2024.10.22 01:11:08.318307 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:08.325087 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:08.467205 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_381_381_0 to 202410_394_394_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:08.486105 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_648_648_0 (state Temporary) 2024.10.22 01:11:08.525600 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_648_648_0 to 202410_661_661_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:08.544651 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5716 2024.10.22 01:11:08.557023 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_77_77_0 (state Temporary) 2024.10.22 01:11:08.580504 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_655_278 to 202410_656_656_0 2024.10.22 01:11:08.591058 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.44 MiB using storage policy from min volume index 0 2024.10.22 01:11:08.592953 [ 1264 ] {} DiskLocal: Reserved 1.44 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:08.608165 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40033 2024.10.22 01:11:08.612870 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_656_279} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_655_278 to 202410_656_656_0 into Compact with storage Full 2024.10.22 01:11:08.612549 [ 1259 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 214 ms. 2024.10.22 01:11:08.661413 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_656_279} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:08.687944 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_656_279} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_655_278, total 40432 rows starting from the beginning of the part 2024.10.22 01:11:08.830788 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_656_279} MergeTreeSequentialSource: Reading 2 marks from part 202410_656_656_0, total 94 rows starting from the beginning of the part 2024.10.22 01:11:08.933162 [ 1261 ] {} DatabaseCatalog: Execution took 242 ms. 2024.10.22 01:11:09.299117 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 224 entries to flush up to offset 35374 2024.10.22 01:11:09.634838 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:09.640620 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:09.689178 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.96 GiB, free memory in arenas 12.15 MiB, will set to 1.74 GiB (RSS), difference: -14.03 MiB 2024.10.22 01:11:09.815081 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_376_376_0 (state Temporary) 2024.10.22 01:11:09.898507 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_376_376_0 to 202410_382_382_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:09.909983 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:11:09.912341 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:11:09.947332 [ 1160 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_376_154 to 202410_377_377_0 2024.10.22 01:11:09.974682 [ 1160 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.80 MiB using storage policy from min volume index 0 2024.10.22 01:11:10.080342 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 35374 2024.10.22 01:11:09.984238 [ 1160 ] {} DiskLocal: Reserved 1.80 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:10.242422 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_377_155} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_376_154 to 202410_377_377_0 into Wide with storage Full 2024.10.22 01:11:10.287911 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_377_155} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:10.295563 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_377_155} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_376_154, total 34170 rows starting from the beginning of the part 2024.10.22 01:11:10.205799 [ 1160 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 287 ms. 2024.10.22 01:11:10.455007 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_377_155} MergeTreeSequentialSource: Reading 2 marks from part 202410_377_377_0, total 210 rows starting from the beginning of the part 2024.10.22 01:11:11.033756 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_656_279} MergeTask::MergeProjectionsStage: Merge sorted 40526 rows, containing 13 columns (13 merged, 0 gathered) in 2.42575077 sec., 16706.58028895524 rows/sec., 2.64 MiB/sec. 2024.10.22 01:11:11.208595 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_656_279/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:11:11.231897 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_377_155/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:11:11.310246 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_656_279} MergedBlockOutputStream: filled checksums 202410_1_656_279 (state Temporary) 2024.10.22 01:11:11.487285 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.96 GiB, free memory in arenas 14.06 MiB, will set to 1.76 GiB (RSS), difference: -35.37 MiB 2024.10.22 01:11:11.943610 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_656_279} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_656_279 to 202410_1_656_279 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:11.949588 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_656_279} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_655_278, 202410_656_656_0] -> 202410_1_656_279 2024.10.22 01:11:11.994149 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:11:12.301744 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_377_155} MergeTask::MergeProjectionsStage: Merge sorted 34380 rows, containing 13 columns (13 merged, 0 gathered) in 2.070686397 sec., 16603.19015463161 rows/sec., 7.14 MiB/sec. 2024.10.22 01:11:12.331490 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_556_179, 202410_557_557_0] 2024.10.22 01:11:12.335664 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_556_179, 202410_557_557_0] 2024.10.22 01:11:12.350914 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_556_179, 202410_557_557_0] 2024.10.22 01:11:12.507809 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_377_155} MergedBlockOutputStream: filled checksums 202410_1_377_155 (state Temporary) 2024.10.22 01:11:12.617234 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(e572de6d198553ef2c83bfee1c2b68a1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e572de6d198553ef2c83bfee1c2b68a1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:11:12.622707 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 40085 2024.10.22 01:11:12.625809 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: e572de6d198553ef2c83bfee1c2b68a1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:11:12.730799 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:12.892547 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:12.896890 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:13.076249 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_649_649_0 (state Temporary) 2024.10.22 01:11:13.117356 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_649_649_0 to 202410_662_662_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:13.174742 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_656_279 to 202410_657_657_0 2024.10.22 01:11:13.180232 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.44 MiB using storage policy from min volume index 0 2024.10.22 01:11:13.186814 [ 1191 ] {} DiskLocal: Reserved 1.44 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:13.216571 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40085 2024.10.22 01:11:13.225545 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_657_280} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_656_279 to 202410_657_657_0 into Compact with storage Full 2024.10.22 01:11:13.362847 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_377_155} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_377_155 to 202410_1_377_155 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:13.371936 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.96 GiB, free memory in arenas 13.78 MiB, will set to 1.75 GiB (RSS), difference: 57.65 MiB 2024.10.22 01:11:13.374825 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_377_155} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_376_154, 202410_377_377_0] -> 202410_1_377_155 2024.10.22 01:11:13.399530 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.92 MiB. 2024.10.22 01:11:13.400524 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_657_280} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:13.409625 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_657_280} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_656_279, total 40526 rows starting from the beginning of the part 2024.10.22 01:11:13.557300 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_657_280} MergeTreeSequentialSource: Reading 2 marks from part 202410_657_657_0, total 71 rows starting from the beginning of the part 2024.10.22 01:11:14.094716 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 326 2024.10.22 01:11:14.601398 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_371_74/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:11:14.755360 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:14.765535 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:14.868912 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1694 entries to flush up to offset 590772 2024.10.22 01:11:15.081268 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_308_308_0 (state Temporary) 2024.10.22 01:11:15.189278 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_308_308_0 to 202410_318_318_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:15.386468 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 326 2024.10.22 01:11:15.626520 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_657_280} MergeTask::MergeProjectionsStage: Merge sorted 40597 rows, containing 13 columns (13 merged, 0 gathered) in 2.412080356 sec., 16830.699648548525 rows/sec., 2.66 MiB/sec. 2024.10.22 01:11:15.717767 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_657_280} MergedBlockOutputStream: filled checksums 202410_1_657_280 (state Temporary) 2024.10.22 01:11:15.781713 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.96 GiB, free memory in arenas 13.53 MiB, will set to 1.76 GiB (RSS), difference: 4.65 MiB 2024.10.22 01:11:15.805777 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:15.910442 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:16.046326 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 5731 2024.10.22 01:11:16.178223 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_366_366_0 (state Temporary) 2024.10.22 01:11:16.265744 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_657_280} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_657_280 to 202410_1_657_280 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:16.271534 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_657_280} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_656_279, 202410_657_657_0] -> 202410_1_657_280 2024.10.22 01:11:16.284337 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:16.287806 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:16.331077 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_366_366_0 to 202410_373_373_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:16.403220 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:11:16.405676 [ 1200 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:11:16.414272 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_382_382_0 (state Temporary) 2024.10.22 01:11:16.480280 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 590772 2024.10.22 01:11:16.572657 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_382_382_0 to 202410_395_395_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:16.579850 [ 1200 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:11:16.711667 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5731 2024.10.22 01:11:17.105963 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_79_16/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:11:17.198131 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(0eae7d0c9734ce19fa15b0a46cfb6aca) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0eae7d0c9734ce19fa15b0a46cfb6aca, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:11:17.209618 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 38350, file segment info: File segment: [0, 33554431], key: 0eae7d0c9734ce19fa15b0a46cfb6aca, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:11:17.212866 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:17.229191 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 40136 2024.10.22 01:11:17.339014 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_320_106, 202410_321_321_0] 2024.10.22 01:11:17.344765 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_320_106, 202410_321_321_0] 2024.10.22 01:11:17.357620 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_320_106, 202410_321_321_0] 2024.10.22 01:11:17.370037 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_557_180, 202410_558_558_0] 2024.10.22 01:11:17.374289 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_557_180, 202410_558_558_0] 2024.10.22 01:11:17.485156 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_557_180, 202410_558_558_0] 2024.10.22 01:11:17.431643 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:17.494660 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:17.593612 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_650_650_0 (state Temporary) 2024.10.22 01:11:17.632051 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 212 entries to flush up to offset 35586 2024.10.22 01:11:17.797988 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:17.799724 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:17.851043 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_650_650_0 to 202410_663_663_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:18.005379 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(92c3d1c705e09e5082f1b3d8637dd4f1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 92c3d1c705e09e5082f1b3d8637dd4f1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:11:18.014418 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40136 2024.10.22 01:11:18.025100 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 467, file segment info: File segment: [0, 33554431], key: 92c3d1c705e09e5082f1b3d8637dd4f1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:11:18.029284 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:17.964369 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_377_377_0 (state Temporary) 2024.10.22 01:11:18.096176 [ 1162 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_657_280 to 202410_658_658_0 2024.10.22 01:11:18.101323 [ 1162 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.44 MiB using storage policy from min volume index 0 2024.10.22 01:11:18.103936 [ 1162 ] {} DiskLocal: Reserved 1.44 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:18.110809 [ 1162 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 01:11:18.128075 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_658_281} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_657_280 to 202410_658_658_0 into Compact with storage Full 2024.10.22 01:11:18.166222 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_658_281} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:18.182427 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_658_281} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_657_280, total 40597 rows starting from the beginning of the part 2024.10.22 01:11:18.196110 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_658_281} MergeTreeSequentialSource: Reading 2 marks from part 202410_658_658_0, total 59 rows starting from the beginning of the part 2024.10.22 01:11:18.305329 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:11:18.439069 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_377_377_0 to 202410_383_383_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:18.587253 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 35586 2024.10.22 01:11:18.587373 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_377_155 to 202410_378_378_0 2024.10.22 01:11:18.593364 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.80 MiB using storage policy from min volume index 0 2024.10.22 01:11:18.621681 [ 1154 ] {} DiskLocal: Reserved 1.80 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:18.766307 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_378_156} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_377_155 to 202410_378_378_0 into Wide with storage Full 2024.10.22 01:11:18.847512 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_378_156} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:18.851186 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_77_77_0 to 202410_80_80_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:18.876498 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_378_156} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_377_155, total 34380 rows starting from the beginning of the part 2024.10.22 01:11:18.914777 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(4577a628869b840af25850673170c5fc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4577a628869b840af25850673170c5fc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:11:18.969333 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 19244, file segment info: File segment: [0, 33554431], key: 4577a628869b840af25850673170c5fc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:11:18.974608 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:19.101613 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_378_156} MergeTreeSequentialSource: Reading 2 marks from part 202410_378_378_0, total 184 rows starting from the beginning of the part 2024.10.22 01:11:19.297339 [ 1150 ] {} DatabaseCatalog: Execution took 361 ms. 2024.10.22 01:11:19.484931 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.96 GiB, free memory in arenas 14.67 MiB, will set to 1.78 GiB (RSS), difference: -5.42 MiB 2024.10.22 01:11:19.773768 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(eee49173a22bbae9fd05bbc3edb044ed) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: eee49173a22bbae9fd05bbc3edb044ed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:11:19.797631 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 562, file segment info: File segment: [0, 33554431], key: eee49173a22bbae9fd05bbc3edb044ed, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:11:19.799999 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:19.911039 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:11:19.913246 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:11:20.088702 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:11:20.095522 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:11:20.102187 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:11:20.125838 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:11:20.157774 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:11:20.190431 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:11:20.294075 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:11:20.387154 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vjn/ndpewmjzxiqespwdriyzvcfdgqpou (LogSeriesLimiter: on interval from 2024-10-22 01:11:07 to 2024-10-22 01:11:20 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:11:20.448944 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_658_281} MergeTask::MergeProjectionsStage: Merge sorted 40656 rows, containing 13 columns (13 merged, 0 gathered) in 2.333544323 sec., 17422.42459218976 rows/sec., 2.75 MiB/sec. 2024.10.22 01:11:20.553604 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_558_181, 202410_559_559_0] 2024.10.22 01:11:20.555967 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_558_181, 202410_559_559_0] 2024.10.22 01:11:20.572942 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_558_181, 202410_559_559_0] 2024.10.22 01:11:20.643333 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_658_281} MergedBlockOutputStream: filled checksums 202410_1_658_281 (state Temporary) 2024.10.22 01:11:20.752096 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_331_66, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0, 202410_336_336_0] 2024.10.22 01:11:20.755532 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_331_66, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0, 202410_336_336_0] 2024.10.22 01:11:20.788905 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_331_66, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0, 202410_336_336_0] 2024.10.22 01:11:20.896584 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_658_281} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_658_281 to 202410_1_658_281 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:20.900248 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_658_281} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_657_280, 202410_658_658_0] -> 202410_1_658_281 2024.10.22 01:11:20.944257 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:11:21.590523 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.96 GiB, free memory in arenas 11.36 MiB, will set to 1.78 GiB (RSS), difference: -4.72 MiB 2024.10.22 01:11:21.723481 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3066 2024.10.22 01:11:21.778568 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_378_156} MergeTask::MergeProjectionsStage: Merge sorted 34564 rows, containing 13 columns (13 merged, 0 gathered) in 3.027513286 sec., 11416.630328207913 rows/sec., 4.91 MiB/sec. 2024.10.22 01:11:21.851668 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_378_156} MergedBlockOutputStream: filled checksums 202410_1_378_156 (state Temporary) 2024.10.22 01:11:22.143505 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 40207 2024.10.22 01:11:22.232037 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} MergeTask::MergeProjectionsStage: Merge sorted 601290 rows, containing 4 columns (4 merged, 0 gathered) in 22.549817884 sec., 26664.9603599078 rows/sec., 390.92 KiB/sec. 2024.10.22 01:11:22.294534 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} MergedBlockOutputStream: filled checksums 202410_1_371_74 (state Temporary) 2024.10.22 01:11:22.358942 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_371_74 to 202410_1_371_74 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:22.384046 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_371_74} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_366_73, 202410_371_371_0] -> 202410_1_371_74 2024.10.22 01:11:22.439703 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:11:22.520633 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:22.534785 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:22.600704 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_651_651_0 (state Temporary) 2024.10.22 01:11:22.642582 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_378_156} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_378_156 to 202410_1_378_156 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:22.647352 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_378_156} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_377_155, 202410_378_378_0] -> 202410_1_378_156 2024.10.22 01:11:22.676590 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.86 MiB. 2024.10.22 01:11:22.872796 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_651_651_0 to 202410_664_664_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:22.888515 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 327 2024.10.22 01:11:22.894587 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_658_281 to 202410_659_659_0 2024.10.22 01:11:22.900957 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.44 MiB using storage policy from min volume index 0 2024.10.22 01:11:22.903956 [ 1171 ] {} DiskLocal: Reserved 1.44 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:22.949086 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40207 2024.10.22 01:11:22.954727 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_659_282} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_658_281 to 202410_659_659_0 into Compact with storage Full 2024.10.22 01:11:23.092792 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_659_282} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:23.101809 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_659_282} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_658_281, total 40656 rows starting from the beginning of the part 2024.10.22 01:11:23.199376 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:23.201825 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:23.229286 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_659_282} MergeTreeSequentialSource: Reading 2 marks from part 202410_659_659_0, total 68 rows starting from the beginning of the part 2024.10.22 01:11:23.294810 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_309_309_0 (state Temporary) 2024.10.22 01:11:23.408414 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_309_309_0 to 202410_319_319_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:23.487679 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 592308 2024.10.22 01:11:23.603219 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.96 GiB, free memory in arenas 15.77 MiB, will set to 1.78 GiB (RSS), difference: 35.51 MiB 2024.10.22 01:11:23.684550 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 327 2024.10.22 01:11:23.691652 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_321_107, 202410_322_322_0] 2024.10.22 01:11:23.795594 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_321_107, 202410_322_322_0] 2024.10.22 01:11:23.805673 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_321_107, 202410_322_322_0] 2024.10.22 01:11:24.213696 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 5754 2024.10.22 01:11:24.419447 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:24.423960 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:24.428212 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:24.433728 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:24.604194 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_367_367_0 (state Temporary) 2024.10.22 01:11:24.689365 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_383_383_0 (state Temporary) 2024.10.22 01:11:24.874940 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_383_383_0 to 202410_396_396_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:24.914615 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_391_78 to 202410_396_396_0 2024.10.22 01:11:24.936289 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:24.939197 [ 1228 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:25.084077 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5754 2024.10.22 01:11:25.090814 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_391_78 to 202410_396_396_0 into Compact with storage Full 2024.10.22 01:11:25.150962 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_367_367_0 to 202410_374_374_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:25.198750 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 592308 2024.10.22 01:11:25.329205 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.96 GiB, free memory in arenas 11.34 MiB, will set to 1.78 GiB (RSS), difference: 7.39 MiB 2024.10.22 01:11:25.391126 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:25.397423 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_391_78, total 5772 rows starting from the beginning of the part 2024.10.22 01:11:25.410840 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_559_182, 202410_560_560_0] 2024.10.22 01:11:25.412070 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_559_182, 202410_560_560_0] 2024.10.22 01:11:25.417734 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_392_392_0, total 25 rows starting from the beginning of the part 2024.10.22 01:11:25.423108 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_559_182, 202410_560_560_0] 2024.10.22 01:11:25.428784 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_659_282} MergeTask::MergeProjectionsStage: Merge sorted 40724 rows, containing 13 columns (13 merged, 0 gathered) in 2.480882638 sec., 16415.12555903501 rows/sec., 2.60 MiB/sec. 2024.10.22 01:11:25.481288 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_393_393_0, total 14 rows starting from the beginning of the part 2024.10.22 01:11:25.530920 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_394_394_0, total 11 rows starting from the beginning of the part 2024.10.22 01:11:25.565349 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_395_395_0, total 15 rows starting from the beginning of the part 2024.10.22 01:11:25.607479 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_396_396_0, total 23 rows starting from the beginning of the part 2024.10.22 01:11:25.769782 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_659_282} MergedBlockOutputStream: filled checksums 202410_1_659_282 (state Temporary) 2024.10.22 01:11:26.184372 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_659_282} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_659_282 to 202410_1_659_282 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:26.189602 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_659_282} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_658_281, 202410_659_659_0] -> 202410_1_659_282 2024.10.22 01:11:26.236461 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.23 MiB. 2024.10.22 01:11:26.344844 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 226 entries to flush up to offset 35812 2024.10.22 01:11:26.691904 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} MergeTask::MergeProjectionsStage: Merge sorted 5860 rows, containing 12 columns (12 merged, 0 gathered) in 1.606339941 sec., 3648.0447571713585 rows/sec., 576.54 KiB/sec. 2024.10.22 01:11:26.844796 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} MergedBlockOutputStream: filled checksums 202410_1_396_79 (state Temporary) 2024.10.22 01:11:26.958050 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 40273 2024.10.22 01:11:26.983063 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_396_79 to 202410_1_396_79 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:26.987261 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_396_79} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_391_78, 202410_396_396_0] -> 202410_1_396_79 2024.10.22 01:11:26.991968 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:26.994838 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:27.038923 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.40 MiB. 2024.10.22 01:11:27.115008 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:27.117287 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:27.196459 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_378_378_0 (state Temporary) 2024.10.22 01:11:27.265614 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_378_378_0 to 202410_384_384_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:27.295557 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_652_652_0 (state Temporary) 2024.10.22 01:11:27.309004 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 35812 2024.10.22 01:11:27.315007 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_378_156 to 202410_379_379_0 2024.10.22 01:11:27.326944 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.81 MiB using storage policy from min volume index 0 2024.10.22 01:11:27.347359 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_652_652_0 to 202410_665_665_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:27.347690 [ 1237 ] {} DiskLocal: Reserved 1.81 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:27.455284 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40273 2024.10.22 01:11:27.467224 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_659_282 to 202410_660_660_0 2024.10.22 01:11:27.479469 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.45 MiB using storage policy from min volume index 0 2024.10.22 01:11:27.487030 [ 1255 ] {} DiskLocal: Reserved 1.45 MiB on local disk `default`, having unreserved 297.40 GiB. 2024.10.22 01:11:27.488142 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_379_157} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_378_156 to 202410_379_379_0 into Wide with storage Full 2024.10.22 01:11:27.527662 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_379_157} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:27.533192 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_379_157} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_378_156, total 34564 rows starting from the beginning of the part 2024.10.22 01:11:27.603380 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_660_283} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_659_282 to 202410_660_660_0 into Compact with storage Full 2024.10.22 01:11:27.635894 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_379_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_379_379_0, total 213 rows starting from the beginning of the part 2024.10.22 01:11:27.788190 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_660_283} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:27.801619 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_660_283} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_659_282, total 40724 rows starting from the beginning of the part 2024.10.22 01:11:27.872554 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_660_283} MergeTreeSequentialSource: Reading 2 marks from part 202410_660_660_0, total 72 rows starting from the beginning of the part 2024.10.22 01:11:28.184143 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_79_16/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:11:28.517353 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.84 GiB, peak 1.96 GiB, free memory in arenas 9.24 MiB, will set to 1.78 GiB (RSS), difference: -56.51 MiB 2024.10.22 01:11:29.278729 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 3108 2024.10.22 01:11:29.488229 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_379_157} MergeTask::MergeProjectionsStage: Merge sorted 34777 rows, containing 13 columns (13 merged, 0 gathered) in 2.009256621 sec., 17308.39139038975 rows/sec., 7.45 MiB/sec. 2024.10.22 01:11:29.571651 [ 1206 ] {} DatabaseCatalog: Execution took 271 ms. 2024.10.22 01:11:29.578304 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_379_157} MergedBlockOutputStream: filled checksums 202410_1_379_157 (state Temporary) 2024.10.22 01:11:29.755296 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_660_283} MergeTask::MergeProjectionsStage: Merge sorted 40796 rows, containing 13 columns (13 merged, 0 gathered) in 2.263834921 sec., 18020.748607402544 rows/sec., 2.85 MiB/sec. 2024.10.22 01:11:29.912356 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:11:29.914619 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:11:29.948694 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_660_283} MergedBlockOutputStream: filled checksums 202410_1_660_283 (state Temporary) 2024.10.22 01:11:30.461957 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.96 GiB, free memory in arenas 9.59 MiB, will set to 1.78 GiB (RSS), difference: 57.87 MiB 2024.10.22 01:11:30.502061 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_379_157} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_379_157 to 202410_1_379_157 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:30.511503 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_379_157} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_378_156, 202410_379_379_0] -> 202410_1_379_157 2024.10.22 01:11:30.558703 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.92 MiB. 2024.10.22 01:11:30.460183 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_660_283} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_660_283 to 202410_1_660_283 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:30.584284 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_660_283} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_659_282, 202410_660_660_0] -> 202410_1_660_283 2024.10.22 01:11:30.618590 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:11:31.183065 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(e08d763794e5ebb98d2c34e9b401b042) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e08d763794e5ebb98d2c34e9b401b042, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:11:31.186488 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 328 2024.10.22 01:11:31.287817 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1106, file segment info: File segment: [0, 33554431], key: e08d763794e5ebb98d2c34e9b401b042, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:11:31.290164 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:31.412825 [ 1748 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mcw/pwgyuqsqkcchhzehrwlkajtzvgxvh, size 1657823 2024.10.22 01:11:31.480061 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 40323 2024.10.22 01:11:31.576268 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_560_183, 202410_561_561_0] 2024.10.22 01:11:31.579463 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_560_183, 202410_561_561_0] 2024.10.22 01:11:31.590338 [ 1183 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:11:31.590870 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_560_183, 202410_561_561_0] 2024.10.22 01:11:31.606149 [ 1183 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:11:31.639976 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:31.649652 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.38 GiB. 2024.10.22 01:11:31.718383 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:31.727490 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.38 GiB. 2024.10.22 01:11:32.019852 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_653_653_0 (state Temporary) 2024.10.22 01:11:32.021716 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_310_310_0 (state Temporary) 2024.10.22 01:11:32.033149 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:11:32.175792 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:11:32.207769 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 594228 2024.10.22 01:11:32.246642 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:11:32.262758 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:11:32.285970 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:11:32.296058 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:11:32.358660 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_310_310_0 to 202410_320_320_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:32.385363 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:11:32.406171 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:11:32.412446 [ 1191 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_315_65 to 202410_320_320_0 2024.10.22 01:11:32.442879 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:11:32.449527 [ 1191 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:32.456654 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.38 GiB. 2024.10.22 01:11:32.490180 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_315_65 to 202410_320_320_0 into Compact with storage Full 2024.10.22 01:11:32.553808 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_653_653_0 to 202410_666_666_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:32.586382 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 5776 2024.10.22 01:11:32.588112 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.96 GiB, free memory in arenas 9.34 MiB, will set to 1.76 GiB (RSS), difference: 141.38 MiB 2024.10.22 01:11:32.594426 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_660_283 to 202410_661_661_0 2024.10.22 01:11:32.604025 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.45 MiB using storage policy from min volume index 0 2024.10.22 01:11:32.634627 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qvb/beqxshdapezxrjcaomaazucfcldmm (LogSeriesLimiter: on interval from 2024-10-22 01:11:20 to 2024-10-22 01:11:32 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:11:32.649423 [ 1182 ] {} DiskLocal: Reserved 1.45 MiB on local disk `default`, having unreserved 297.38 GiB. 2024.10.22 01:11:32.689319 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40323 2024.10.22 01:11:32.701753 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_661_284} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_660_283 to 202410_661_661_0 into Compact with storage Full 2024.10.22 01:11:32.734792 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_661_284} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:32.761821 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:32.766555 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.37 GiB. 2024.10.22 01:11:32.769516 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:32.781591 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 328 2024.10.22 01:11:32.821600 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_661_284} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_660_283, total 40796 rows starting from the beginning of the part 2024.10.22 01:11:32.825766 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_315_65, total 359 rows starting from the beginning of the part 2024.10.22 01:11:32.904247 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_661_284} MergeTreeSequentialSource: Reading 2 marks from part 202410_661_661_0, total 68 rows starting from the beginning of the part 2024.10.22 01:11:32.906091 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_384_384_0 (state Temporary) 2024.10.22 01:11:32.933392 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_316_316_0, total 1 rows starting from the beginning of the part 2024.10.22 01:11:32.974599 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_384_384_0 to 202410_397_397_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:33.079991 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:33.082479 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.37 GiB. 2024.10.22 01:11:33.103991 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_317_317_0, total 1 rows starting from the beginning of the part 2024.10.22 01:11:33.122172 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5776 2024.10.22 01:11:33.190735 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_318_318_0, total 1 rows starting from the beginning of the part 2024.10.22 01:11:33.268555 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_368_368_0 (state Temporary) 2024.10.22 01:11:33.506601 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_319_319_0, total 1 rows starting from the beginning of the part 2024.10.22 01:11:33.534468 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_368_368_0 to 202410_375_375_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:33.589464 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 594228 2024.10.22 01:11:33.690645 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_320_320_0, total 1 rows starting from the beginning of the part 2024.10.22 01:11:34.556447 [ 1680 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tqp/adppnoclfyickzhpohontzvygeroz, size 3510 2024.10.22 01:11:34.622999 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.96 GiB, free memory in arenas 6.32 MiB, will set to 1.75 GiB (RSS), difference: 116.58 MiB 2024.10.22 01:11:34.945117 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 243 entries to flush up to offset 36055 2024.10.22 01:11:35.109437 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_322_108, 202410_323_323_0] 2024.10.22 01:11:35.121996 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_322_108, 202410_323_323_0] 2024.10.22 01:11:35.152427 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_322_108, 202410_323_323_0] 2024.10.22 01:11:35.586342 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:35.589757 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:35.599153 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_22_22_0 (state Temporary) 2024.10.22 01:11:35.628889 [ 1733 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wpm/puicreeujtmagrdirsahrtverpkun, size 710 2024.10.22 01:11:35.687424 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_379_379_0 (state Temporary) 2024.10.22 01:11:35.882655 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_379_379_0 to 202410_385_385_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:35.911016 [ 1177 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_379_157 to 202410_380_380_0 2024.10.22 01:11:35.923632 [ 1177 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.82 MiB using storage policy from min volume index 0 2024.10.22 01:11:35.927022 [ 1177 ] {} DiskLocal: Reserved 1.82 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:35.987460 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_661_284} MergeTask::MergeProjectionsStage: Merge sorted 40864 rows, containing 13 columns (13 merged, 0 gathered) in 3.292385247 sec., 12411.670243400285 rows/sec., 1.96 MiB/sec. 2024.10.22 01:11:35.988807 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} MergeTask::MergeProjectionsStage: Merge sorted 364 rows, containing 32 columns (32 merged, 0 gathered) in 3.504847135 sec., 103.85617003521611 rows/sec., 10.55 KiB/sec. 2024.10.22 01:11:36.022862 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 36055 2024.10.22 01:11:36.152767 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} MergedBlockOutputStream: filled checksums 202410_1_320_66 (state Temporary) 2024.10.22 01:11:36.172601 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_380_158} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_379_157 to 202410_380_380_0 into Wide with storage Full 2024.10.22 01:11:36.192521 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_661_284} MergedBlockOutputStream: filled checksums 202410_1_661_284 (state Temporary) 2024.10.22 01:11:36.242987 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_380_158} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:36.252092 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_380_158} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_379_157, total 34777 rows starting from the beginning of the part 2024.10.22 01:11:36.627633 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_380_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_380_380_0, total 241 rows starting from the beginning of the part 2024.10.22 01:11:36.694137 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 40404 2024.10.22 01:11:36.713756 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_561_184, 202410_562_562_0] 2024.10.22 01:11:36.723337 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_561_184, 202410_562_562_0] 2024.10.22 01:11:36.761418 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_661_284} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_661_284 to 202410_1_661_284 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:36.848957 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.96 GiB, free memory in arenas 6.12 MiB, will set to 1.71 GiB (RSS), difference: 57.00 MiB 2024.10.22 01:11:36.877297 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_661_284} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_660_283, 202410_661_661_0] -> 202410_1_661_284 2024.10.22 01:11:36.879563 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_561_184, 202410_562_562_0] 2024.10.22 01:11:36.908561 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:11:36.982229 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_320_66 to 202410_1_320_66 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:36.986193 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_320_66} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_315_65, 202410_320_320_0] -> 202410_1_320_66 2024.10.22 01:11:36.993383 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:36.999463 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:37.085532 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_654_654_0 (state Temporary) 2024.10.22 01:11:37.093105 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:11:37.238603 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_654_654_0 to 202410_667_667_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:37.283162 [ 1274 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_661_284 to 202410_662_662_0 2024.10.22 01:11:37.294946 [ 1274 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.45 MiB using storage policy from min volume index 0 2024.10.22 01:11:37.297706 [ 1274 ] {} DiskLocal: Reserved 1.45 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:37.299953 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40404 2024.10.22 01:11:37.323570 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_662_285} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_661_284 to 202410_662_662_0 into Compact with storage Full 2024.10.22 01:11:37.378715 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_662_285} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:37.387993 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_662_285} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_661_284, total 40864 rows starting from the beginning of the part 2024.10.22 01:11:37.400920 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_662_285} MergeTreeSequentialSource: Reading 2 marks from part 202410_662_662_0, total 52 rows starting from the beginning of the part 2024.10.22 01:11:37.494344 [ 1245 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 217 ms. 2024.10.22 01:11:38.398818 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_380_158/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:11:38.676039 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.96 GiB, free memory in arenas 5.54 MiB, will set to 1.73 GiB (RSS), difference: -10.36 MiB 2024.10.22 01:11:38.822529 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_380_158} MergeTask::MergeProjectionsStage: Merge sorted 35018 rows, containing 13 columns (13 merged, 0 gathered) in 2.770915809 sec., 12637.699018591871 rows/sec., 5.44 MiB/sec. 2024.10.22 01:11:39.038895 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_380_158} MergedBlockOutputStream: filled checksums 202410_1_380_158 (state Temporary) 2024.10.22 01:11:39.202554 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_662_285} MergeTask::MergeProjectionsStage: Merge sorted 40916 rows, containing 13 columns (13 merged, 0 gathered) in 1.892632058 sec., 21618.57072379781 rows/sec., 3.42 MiB/sec. 2024.10.22 01:11:39.500161 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_662_285} MergedBlockOutputStream: filled checksums 202410_1_662_285 (state Temporary) 2024.10.22 01:11:39.599877 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_380_158} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_380_158 to 202410_1_380_158 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:39.602995 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_380_158} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_379_157, 202410_380_380_0] -> 202410_1_380_158 2024.10.22 01:11:39.629087 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.94 MiB. 2024.10.22 01:11:39.663609 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_662_285} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_662_285 to 202410_1_662_285 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:39.667644 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_662_285} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_661_284, 202410_662_662_0] -> 202410_1_662_285 2024.10.22 01:11:39.693452 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:11:39.710347 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:11:39.912266 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:11:39.916771 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:11:40.285385 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 329 2024.10.22 01:11:40.493205 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:40.495277 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:40.594417 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1538 entries to flush up to offset 595766 2024.10.22 01:11:40.665607 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 5796 2024.10.22 01:11:40.733731 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.96 GiB, free memory in arenas 6.77 MiB, will set to 1.71 GiB (RSS), difference: 30.19 MiB 2024.10.22 01:11:40.743601 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_311_62, 202410_312_312_0, 202410_313_313_0, 202410_314_314_0, 202410_315_315_0, 202410_316_316_0] 2024.10.22 01:11:40.751705 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_311_62, 202410_312_312_0, 202410_313_313_0, 202410_314_314_0, 202410_315_315_0, 202410_316_316_0] 2024.10.22 01:11:40.781320 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_311_311_0 (state Temporary) 2024.10.22 01:11:40.794802 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_311_62, 202410_312_312_0, 202410_313_313_0, 202410_314_314_0, 202410_315_315_0, 202410_316_316_0] 2024.10.22 01:11:40.815589 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:40.824733 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:40.866372 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_311_311_0 to 202410_321_321_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:40.930187 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_562_185, 202410_563_563_0] 2024.10.22 01:11:40.939131 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_562_185, 202410_563_563_0] 2024.10.22 01:11:40.966078 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_562_185, 202410_563_563_0] 2024.10.22 01:11:40.981459 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_385_385_0 (state Temporary) 2024.10.22 01:11:41.037420 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_385_385_0 to 202410_398_398_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:41.152841 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 329 2024.10.22 01:11:41.292758 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5796 2024.10.22 01:11:41.309471 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:41.311418 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 40463 2024.10.22 01:11:41.313486 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:41.538328 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_369_369_0 (state Temporary) 2024.10.22 01:11:41.572998 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:41.577270 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:41.627391 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_655_655_0 (state Temporary) 2024.10.22 01:11:41.731152 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_369_369_0 to 202410_376_376_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:41.779702 [ 1224 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_371_74 to 202410_376_376_0 2024.10.22 01:11:41.790498 [ 1224 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:41.794161 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:41.869727 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 595766 2024.10.22 01:11:41.874953 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(1bc57785d81387c43be37462ff11fcf9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1bc57785d81387c43be37462ff11fcf9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:11:41.878524 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 28232, file segment info: File segment: [0, 33554431], key: 1bc57785d81387c43be37462ff11fcf9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:11:41.881025 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:41.881421 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_655_655_0 to 202410_668_668_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:41.901822 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_662_285 to 202410_663_663_0 2024.10.22 01:11:41.915665 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.45 MiB using storage policy from min volume index 0 2024.10.22 01:11:41.927037 [ 1216 ] {} DiskLocal: Reserved 1.45 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:41.945137 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_371_74 to 202410_376_376_0 into Compact with storage Full 2024.10.22 01:11:41.948402 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_663_286} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_662_285 to 202410_663_663_0 into Compact with storage Full 2024.10.22 01:11:42.000476 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:42.007448 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} MergeTreeSequentialSource: Reading 75 marks from part 202410_1_371_74, total 601290 rows starting from the beginning of the part 2024.10.22 01:11:42.030380 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_372_372_0, total 1378 rows starting from the beginning of the part 2024.10.22 01:11:42.045512 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_373_373_0, total 1694 rows starting from the beginning of the part 2024.10.22 01:11:42.087727 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_663_286} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:42.098510 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_663_286} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_662_285, total 40916 rows starting from the beginning of the part 2024.10.22 01:11:42.129100 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_663_286} MergeTreeSequentialSource: Reading 2 marks from part 202410_663_663_0, total 51 rows starting from the beginning of the part 2024.10.22 01:11:42.136210 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40463 2024.10.22 01:11:42.198048 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_374_374_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:11:42.240753 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_375_375_0, total 1920 rows starting from the beginning of the part 2024.10.22 01:11:42.280530 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_376_376_0, total 1538 rows starting from the beginning of the part 2024.10.22 01:11:42.592231 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.96 GiB, free memory in arenas 7.98 MiB, will set to 1.71 GiB (RSS), difference: -6.42 MiB 2024.10.22 01:11:42.691438 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(4e0234cc71967188db9ab05e1b33745c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4e0234cc71967188db9ab05e1b33745c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:11:42.694072 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 458, file segment info: File segment: [0, 33554431], key: 4e0234cc71967188db9ab05e1b33745c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:11:42.695935 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:43.149390 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_376_75/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:11:43.236213 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_323_109, 202410_324_324_0, 202410_325_325_0, 202410_326_326_0, 202410_327_327_0, 202410_328_328_0] 2024.10.22 01:11:43.240651 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_323_109, 202410_324_324_0, 202410_325_325_0, 202410_326_326_0, 202410_327_327_0, 202410_328_328_0] 2024.10.22 01:11:43.407208 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_323_109, 202410_324_324_0, 202410_325_325_0, 202410_326_326_0, 202410_327_327_0, 202410_328_328_0] 2024.10.22 01:11:43.584864 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(d66993edab7dc5024008f7b298b9bb32) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d66993edab7dc5024008f7b298b9bb32, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:11:43.588220 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 18728, file segment info: File segment: [0, 33554431], key: d66993edab7dc5024008f7b298b9bb32, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:11:43.591124 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:43.692917 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 235 entries to flush up to offset 36290 2024.10.22 01:11:43.987403 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:43.989210 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:44.042663 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_380_380_0 (state Temporary) 2024.10.22 01:11:44.193461 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_380_380_0 to 202410_386_386_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:44.223330 [ 1226 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_380_158 to 202410_381_381_0 2024.10.22 01:11:44.235656 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 36290 2024.10.22 01:11:44.245726 [ 1226 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.84 MiB using storage policy from min volume index 0 2024.10.22 01:11:44.252234 [ 1226 ] {} DiskLocal: Reserved 1.84 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:44.290381 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_381_159} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_380_158 to 202410_381_381_0 into Wide with storage Full 2024.10.22 01:11:44.325490 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(ebbcb8bff22adf32d162786ddf2171ce) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ebbcb8bff22adf32d162786ddf2171ce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:11:44.328298 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 564, file segment info: File segment: [0, 33554431], key: ebbcb8bff22adf32d162786ddf2171ce, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:11:44.332524 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:44.392190 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_663_286} MergeTask::MergeProjectionsStage: Merge sorted 40967 rows, containing 13 columns (13 merged, 0 gathered) in 2.452319138 sec., 16705.41136559038 rows/sec., 2.64 MiB/sec. 2024.10.22 01:11:44.415850 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_381_159} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:44.433013 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_381_159} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_380_158, total 35018 rows starting from the beginning of the part 2024.10.22 01:11:44.476010 [ 1740 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vjn/ndpewmjzxiqespwdriyzvcfdgqpou, size 18728 2024.10.22 01:11:44.582760 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_663_286} MergedBlockOutputStream: filled checksums 202410_1_663_286 (state Temporary) 2024.10.22 01:11:44.606167 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.96 GiB, free memory in arenas 9.97 MiB, will set to 1.72 GiB (RSS), difference: 15.33 MiB 2024.10.22 01:11:44.655172 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_663_286} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_663_286 to 202410_1_663_286 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:44.669989 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_663_286} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_662_285, 202410_663_663_0] -> 202410_1_663_286 2024.10.22 01:11:44.701723 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:11:44.709260 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_381_159} MergeTreeSequentialSource: Reading 2 marks from part 202410_381_381_0, total 236 rows starting from the beginning of the part 2024.10.22 01:11:44.825591 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_79_16/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:11:44.853736 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a4286881e5b66bc4fd280e9f7ea9d505) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a4286881e5b66bc4fd280e9f7ea9d505, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:11:44.884835 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: a4286881e5b66bc4fd280e9f7ea9d505, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:11:44.891705 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:45.142779 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:11:45.161155 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:11:45.182188 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:11:45.210806 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:11:45.246336 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:11:45.277002 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:11:45.340016 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:11:45.453019 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bga/rzmwvrkiwjryqsmrkqcxsjtjdrxto (LogSeriesLimiter: on interval from 2024-10-22 01:11:32 to 2024-10-22 01:11:45 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:11:45.508441 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_563_186, 202410_564_564_0] 2024.10.22 01:11:45.511026 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_563_186, 202410_564_564_0] 2024.10.22 01:11:45.527839 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_563_186, 202410_564_564_0] 2024.10.22 01:11:46.148114 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 40544 2024.10.22 01:11:46.179115 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_268_55, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0, 202410_272_272_0, 202410_273_273_0] 2024.10.22 01:11:46.180806 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_268_55, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0, 202410_272_272_0, 202410_273_273_0] 2024.10.22 01:11:46.206169 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_268_55, 202410_269_269_0, 202410_270_270_0, 202410_271_271_0, 202410_272_272_0, 202410_273_273_0] 2024.10.22 01:11:46.453126 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.96 GiB, free memory in arenas 11.46 MiB, will set to 1.74 GiB (RSS), difference: -37.91 MiB 2024.10.22 01:11:46.610024 [ 1250 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:11:46.648727 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:46.654813 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:46.735749 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_656_656_0 (state Temporary) 2024.10.22 01:11:46.781307 [ 1250 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:11:46.784958 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_656_656_0 to 202410_669_669_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:46.820766 [ 1261 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_663_286 to 202410_664_664_0 2024.10.22 01:11:46.831222 [ 1261 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.45 MiB using storage policy from min volume index 0 2024.10.22 01:11:46.835021 [ 1261 ] {} DiskLocal: Reserved 1.45 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:46.850608 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40544 2024.10.22 01:11:46.850976 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_664_287} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_663_286 to 202410_664_664_0 into Compact with storage Full 2024.10.22 01:11:46.987559 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_664_287} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:47.005795 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_664_287} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_663_286, total 40967 rows starting from the beginning of the part 2024.10.22 01:11:47.058071 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_664_287} MergeTreeSequentialSource: Reading 2 marks from part 202410_664_664_0, total 71 rows starting from the beginning of the part 2024.10.22 01:11:47.518254 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_381_159} MergeTask::MergeProjectionsStage: Merge sorted 35254 rows, containing 13 columns (13 merged, 0 gathered) in 3.230868147 sec., 10911.61830071427 rows/sec., 4.70 MiB/sec. 2024.10.22 01:11:47.709987 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:47.715015 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:47.716456 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_381_159} MergedBlockOutputStream: filled checksums 202410_1_381_159 (state Temporary) 2024.10.22 01:11:48.185631 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_381_159} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_381_159 to 202410_1_381_159 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:48.188278 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_381_159} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_380_158, 202410_381_381_0] -> 202410_1_381_159 2024.10.22 01:11:48.207902 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.93 MiB. 2024.10.22 01:11:48.488044 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.96 GiB, free memory in arenas 19.20 MiB, will set to 1.74 GiB (RSS), difference: 44.93 MiB 2024.10.22 01:11:48.800848 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 5808 2024.10.22 01:11:48.895987 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_564_187, 202410_565_565_0] 2024.10.22 01:11:48.897322 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_564_187, 202410_565_565_0] 2024.10.22 01:11:48.903294 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_564_187, 202410_565_565_0] 2024.10.22 01:11:48.927967 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1780 entries to flush up to offset 597546 2024.10.22 01:11:49.064534 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:49.073177 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:49.120348 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_386_386_0 (state Temporary) 2024.10.22 01:11:49.275771 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_386_386_0 to 202410_399_399_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:49.333077 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5808 2024.10.22 01:11:49.739537 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:49.745519 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:49.804709 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_664_287} MergeTask::MergeProjectionsStage: Merge sorted 41038 rows, containing 13 columns (13 merged, 0 gathered) in 2.961311961 sec., 13858.046886131495 rows/sec., 2.19 MiB/sec. 2024.10.22 01:11:49.918106 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:11:49.924521 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:11:49.995582 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_664_287} MergedBlockOutputStream: filled checksums 202410_1_664_287 (state Temporary) 2024.10.22 01:11:50.003474 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_370_370_0 (state Temporary) 2024.10.22 01:11:50.099418 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_370_370_0 to 202410_377_377_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:50.100512 [ 1186 ] {} DatabaseCatalog: Execution took 360 ms. 2024.10.22 01:11:50.144556 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 597546 2024.10.22 01:11:50.200451 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_664_287} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_664_287 to 202410_1_664_287 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:50.234296 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_664_287} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_663_286, 202410_664_664_0] -> 202410_1_664_287 2024.10.22 01:11:50.282322 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:11:50.577514 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.96 GiB, free memory in arenas 25.03 MiB, will set to 1.72 GiB (RSS), difference: 5.52 MiB 2024.10.22 01:11:50.595426 [ 1163 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 01:11:50.883856 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 40596 2024.10.22 01:11:51.045668 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:51.057609 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:51.216579 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_657_657_0 (state Temporary) 2024.10.22 01:11:51.308868 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_657_657_0 to 202410_670_670_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:51.481988 [ 1219 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_664_287 to 202410_665_665_0 2024.10.22 01:11:51.482586 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40596 2024.10.22 01:11:51.486811 [ 1219 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.45 MiB using storage policy from min volume index 0 2024.10.22 01:11:51.488381 [ 1219 ] {} DiskLocal: Reserved 1.45 MiB on local disk `default`, having unreserved 297.32 GiB. 2024.10.22 01:11:51.598086 [ 1219 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 275 ms. 2024.10.22 01:11:51.603147 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_665_288} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_664_287 to 202410_665_665_0 into Compact with storage Full 2024.10.22 01:11:51.639596 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_665_288} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:51.664773 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_665_288} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_664_287, total 41038 rows starting from the beginning of the part 2024.10.22 01:11:51.681828 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_665_288} MergeTreeSequentialSource: Reading 2 marks from part 202410_665_665_0, total 66 rows starting from the beginning of the part 2024.10.22 01:11:51.778493 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 205 entries to flush up to offset 36495 2024.10.22 01:11:51.911902 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_78_78_0 (state Temporary) 2024.10.22 01:11:52.180871 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 01:11:52.388040 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:52.391230 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:11:52.460388 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_381_381_0 (state Temporary) 2024.10.22 01:11:52.503951 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(132ebf21a23d4cfc5e404154f1bda3c9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 132ebf21a23d4cfc5e404154f1bda3c9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:11:52.515308 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1020, file segment info: File segment: [0, 33554431], key: 132ebf21a23d4cfc5e404154f1bda3c9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:11:52.520411 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:11:52.546323 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_381_381_0 to 202410_387_387_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:52.602102 [ 1727 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qvb/beqxshdapezxrjcaomaazucfcldmm, size 2203514 2024.10.22 01:11:52.608138 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 36495 2024.10.22 01:11:52.598844 [ 1192 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_381_159 to 202410_382_382_0 2024.10.22 01:11:52.617448 [ 1192 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.85 MiB using storage policy from min volume index 0 2024.10.22 01:11:52.622656 [ 1192 ] {} DiskLocal: Reserved 1.85 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:11:52.655519 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.96 GiB, free memory in arenas 20.41 MiB, will set to 1.72 GiB (RSS), difference: -16.61 MiB 2024.10.22 01:11:52.744174 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_382_160} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_381_159 to 202410_382_382_0 into Wide with storage Full 2024.10.22 01:11:52.891248 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_382_160} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:52.959877 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_665_288/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:11:53.039080 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_382_160} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_381_159, total 35254 rows starting from the beginning of the part 2024.10.22 01:11:53.284646 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:11:53.307584 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:11:53.324735 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:11:53.349522 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:11:53.377306 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:11:53.401339 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_382_160} MergeTreeSequentialSource: Reading 2 marks from part 202410_382_382_0, total 224 rows starting from the beginning of the part 2024.10.22 01:11:53.409019 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:11:53.439144 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:11:53.454792 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:11:53.615477 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:11:53.656580 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gym/tandbbjzpxfjhbnklivneukgyrbzk (LogSeriesLimiter: on interval from 2024-10-22 01:11:45 to 2024-10-22 01:11:53 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:11:54.391871 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_665_288} MergeTask::MergeProjectionsStage: Merge sorted 41104 rows, containing 13 columns (13 merged, 0 gathered) in 2.792226088 sec., 14720.870984140736 rows/sec., 2.33 MiB/sec. 2024.10.22 01:11:54.501036 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.96 GiB, free memory in arenas 20.12 MiB, will set to 1.73 GiB (RSS), difference: -29.23 MiB 2024.10.22 01:11:54.634462 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_665_288} MergedBlockOutputStream: filled checksums 202410_1_665_288 (state Temporary) 2024.10.22 01:11:54.685819 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_565_188, 202410_566_566_0] 2024.10.22 01:11:54.687303 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_565_188, 202410_566_566_0] 2024.10.22 01:11:54.694679 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_565_188, 202410_566_566_0] 2024.10.22 01:11:54.777559 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_665_288} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_665_288 to 202410_1_665_288 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:54.780055 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_665_288} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_664_287, 202410_665_665_0] -> 202410_1_665_288 2024.10.22 01:11:54.820101 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:11:54.883323 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_376_75/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:11:55.488931 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 40651 2024.10.22 01:11:56.158565 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 330 2024.10.22 01:11:56.310027 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.96 GiB, free memory in arenas 20.25 MiB, will set to 1.73 GiB (RSS), difference: -2.62 MiB 2024.10.22 01:11:56.312231 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:56.318498 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:11:56.327181 [ 1205 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 01:11:56.481967 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_658_658_0 (state Temporary) 2024.10.22 01:11:56.575708 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:56.580085 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:11:56.609779 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_382_160} MergeTask::MergeProjectionsStage: Merge sorted 35478 rows, containing 13 columns (13 merged, 0 gathered) in 3.977183775 sec., 8920.382362768742 rows/sec., 3.84 MiB/sec. 2024.10.22 01:11:56.628671 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_658_658_0 to 202410_671_671_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:56.681827 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_665_288 to 202410_666_666_0 2024.10.22 01:11:56.702510 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.46 MiB using storage policy from min volume index 0 2024.10.22 01:11:56.706293 [ 1158 ] {} DiskLocal: Reserved 1.46 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:11:56.706999 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40651 2024.10.22 01:11:56.711207 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_382_160} MergedBlockOutputStream: filled checksums 202410_1_382_160 (state Temporary) 2024.10.22 01:11:56.734345 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_666_289} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_665_288 to 202410_666_666_0 into Compact with storage Full 2024.10.22 01:11:56.743904 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_312_312_0 (state Temporary) 2024.10.22 01:11:56.791029 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_666_289} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:11:56.804768 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_666_289} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_665_288, total 41104 rows starting from the beginning of the part 2024.10.22 01:11:56.843291 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 5819 2024.10.22 01:11:56.929519 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_312_312_0 to 202410_322_322_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:56.932175 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_666_289} MergeTreeSequentialSource: Reading 2 marks from part 202410_666_666_0, total 50 rows starting from the beginning of the part 2024.10.22 01:11:57.032941 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 330 2024.10.22 01:11:57.152532 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1676 entries to flush up to offset 599222 2024.10.22 01:11:57.193749 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:57.196577 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:11:57.269756 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_387_387_0 (state Temporary) 2024.10.22 01:11:57.332155 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_387_387_0 to 202410_400_400_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:57.399377 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5819 2024.10.22 01:11:57.443301 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_382_160} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_382_160 to 202410_1_382_160 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:57.446016 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_382_160} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_381_159, 202410_382_382_0] -> 202410_1_382_160 2024.10.22 01:11:57.482318 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.93 MiB. 2024.10.22 01:11:57.880132 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:11:57.883941 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:11:58.130484 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_371_371_0 (state Temporary) 2024.10.22 01:11:58.211464 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_371_371_0 to 202410_378_378_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:11:58.270268 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 599222 2024.10.22 01:11:58.908493 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.96 GiB, free memory in arenas 16.25 MiB, will set to 1.72 GiB (RSS), difference: 28.11 MiB 2024.10.22 01:11:59.785599 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zln/efxtdlaxkznseiudgrfuvhsesqiqf. 2024.10.22 01:11:59.795595 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_566_189, 202410_567_567_0] 2024.10.22 01:11:59.799503 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_566_189, 202410_567_567_0] 2024.10.22 01:11:59.818920 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_566_189, 202410_567_567_0] 2024.10.22 01:11:59.921188 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:11:59.926459 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:11:59.935398 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_666_289} MergeTask::MergeProjectionsStage: Merge sorted 41154 rows, containing 13 columns (13 merged, 0 gathered) in 3.208378125 sec., 12827.041700391846 rows/sec., 2.03 MiB/sec. 2024.10.22 01:12:00.231125 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_666_289} MergedBlockOutputStream: filled checksums 202410_1_666_289 (state Temporary) 2024.10.22 01:12:00.344879 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_666_289} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_666_289 to 202410_1_666_289 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:00.382350 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_666_289} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_665_288, 202410_666_666_0] -> 202410_1_666_289 2024.10.22 01:12:00.412866 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.22 MiB. 2024.10.22 01:12:00.516417 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 229 entries to flush up to offset 36724 2024.10.22 01:12:00.556653 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.96 GiB, free memory in arenas 17.01 MiB, will set to 1.72 GiB (RSS), difference: 2.11 MiB 2024.10.22 01:12:00.654930 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_78_78_0 to 202410_81_81_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:00.713650 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 40704 2024.10.22 01:12:00.720780 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_79_16/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:12:00.859110 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:00.868102 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:00.925353 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:00.929455 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:00.985458 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_382_382_0 (state Temporary) 2024.10.22 01:12:01.028965 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_659_659_0 (state Temporary) 2024.10.22 01:12:01.083900 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_659_659_0 to 202410_672_672_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:01.122928 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_666_289 to 202410_667_667_0 2024.10.22 01:12:01.134264 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.46 MiB using storage policy from min volume index 0 2024.10.22 01:12:01.142913 [ 1239 ] {} DiskLocal: Reserved 1.46 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:01.149253 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_382_382_0 to 202410_388_388_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:01.161334 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_667_290} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_666_289 to 202410_667_667_0 into Compact with storage Full 2024.10.22 01:12:01.183644 [ 1277 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_382_160 to 202410_383_383_0 2024.10.22 01:12:01.190485 [ 1277 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.86 MiB using storage policy from min volume index 0 2024.10.22 01:12:01.191806 [ 1277 ] {} DiskLocal: Reserved 1.86 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:01.204996 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_667_290} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:01.204829 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_383_161} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_382_160 to 202410_383_383_0 into Wide with storage Full 2024.10.22 01:12:01.226169 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_667_290} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_666_289, total 41154 rows starting from the beginning of the part 2024.10.22 01:12:01.236520 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 36724 2024.10.22 01:12:01.264342 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_383_161} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:01.277393 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_383_161} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_382_160, total 35478 rows starting from the beginning of the part 2024.10.22 01:12:01.341089 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40704 2024.10.22 01:12:01.378761 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_667_290} MergeTreeSequentialSource: Reading 2 marks from part 202410_667_667_0, total 81 rows starting from the beginning of the part 2024.10.22 01:12:01.508439 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_383_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_383_383_0, total 212 rows starting from the beginning of the part 2024.10.22 01:12:01.785212 [ 1216 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:12:01.796793 [ 1216 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:12:01.829311 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_336_67, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0, 202410_340_340_0, 202410_341_341_0] 2024.10.22 01:12:01.832423 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_336_67, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0, 202410_340_340_0, 202410_341_341_0] 2024.10.22 01:12:01.974365 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_336_67, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0, 202410_340_340_0, 202410_341_341_0] 2024.10.22 01:12:02.628965 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_567_190, 202410_568_568_0] 2024.10.22 01:12:02.632984 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_567_190, 202410_568_568_0] 2024.10.22 01:12:02.738801 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.96 GiB, free memory in arenas 14.64 MiB, will set to 1.72 GiB (RSS), difference: -45.68 MiB 2024.10.22 01:12:02.751395 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_567_190, 202410_568_568_0] 2024.10.22 01:12:02.803347 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} MergeTask::MergeProjectionsStage: Merge sorted 609356 rows, containing 4 columns (4 merged, 0 gathered) in 20.892632203 sec., 29166.071277151077 rows/sec., 427.61 KiB/sec. 2024.10.22 01:12:02.886383 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} MergedBlockOutputStream: filled checksums 202410_1_376_75 (state Temporary) 2024.10.22 01:12:02.950318 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_376_75 to 202410_1_376_75 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:03.056652 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_376_75} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_371_74, 202410_376_376_0] -> 202410_1_376_75 2024.10.22 01:12:03.098368 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:12:03.839268 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3108 2024.10.22 01:12:03.949794 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_667_290} MergeTask::MergeProjectionsStage: Merge sorted 41235 rows, containing 13 columns (13 merged, 0 gathered) in 2.797167874 sec., 14741.696550744813 rows/sec., 2.33 MiB/sec. 2024.10.22 01:12:04.042434 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_667_290} MergedBlockOutputStream: filled checksums 202410_1_667_290 (state Temporary) 2024.10.22 01:12:04.536324 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 331 2024.10.22 01:12:04.567944 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.96 GiB, free memory in arenas 13.61 MiB, will set to 1.73 GiB (RSS), difference: 24.15 MiB 2024.10.22 01:12:04.509455 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_667_290} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_667_290 to 202410_1_667_290 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:04.633453 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_667_290} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_666_289, 202410_667_667_0] -> 202410_1_667_290 2024.10.22 01:12:04.763750 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.21 MiB. 2024.10.22 01:12:04.900826 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 5841 2024.10.22 01:12:05.037127 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_383_161} MergeTask::MergeProjectionsStage: Merge sorted 35690 rows, containing 13 columns (13 merged, 0 gathered) in 3.83817499 sec., 9298.690156907098 rows/sec., 4.00 MiB/sec. 2024.10.22 01:12:05.083980 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:05.091173 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:05.101731 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_383_161} MergedBlockOutputStream: filled checksums 202410_1_383_161 (state Temporary) 2024.10.22 01:12:05.143240 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:05.148763 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:05.149075 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_388_388_0 (state Temporary) 2024.10.22 01:12:05.208207 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_388_388_0 to 202410_401_401_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:05.230649 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_313_313_0 (state Temporary) 2024.10.22 01:12:05.277285 [ 1243 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_396_79 to 202410_401_401_0 2024.10.22 01:12:05.278021 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 600758 2024.10.22 01:12:05.365246 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 40764 2024.10.22 01:12:05.395801 [ 1243 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:05.396280 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5841 2024.10.22 01:12:05.405590 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:05.559260 [ 1243 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 01:12:05.581153 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_396_79 to 202410_401_401_0 into Compact with storage Full 2024.10.22 01:12:05.635062 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_313_313_0 to 202410_323_323_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:05.786971 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_383_161} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_383_161 to 202410_1_383_161 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:05.795846 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_383_161} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_382_160, 202410_383_383_0] -> 202410_1_383_161 2024.10.22 01:12:05.852331 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:05.859700 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:05.887745 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:05.898397 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_396_79, total 5860 rows starting from the beginning of the part 2024.10.22 01:12:05.916338 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.94 MiB. 2024.10.22 01:12:06.010209 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 331 2024.10.22 01:12:06.027995 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_397_397_0, total 22 rows starting from the beginning of the part 2024.10.22 01:12:06.042990 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_660_660_0 (state Temporary) 2024.10.22 01:12:06.099743 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_398_398_0, total 20 rows starting from the beginning of the part 2024.10.22 01:12:06.103629 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_660_660_0 to 202410_673_673_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:06.239320 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_399_399_0, total 12 rows starting from the beginning of the part 2024.10.22 01:12:06.255946 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_667_290 to 202410_668_668_0 2024.10.22 01:12:06.265005 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.46 MiB using storage policy from min volume index 0 2024.10.22 01:12:06.281659 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_400_400_0, total 11 rows starting from the beginning of the part 2024.10.22 01:12:06.301017 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_401_401_0, total 22 rows starting from the beginning of the part 2024.10.22 01:12:06.273735 [ 1257 ] {} DiskLocal: Reserved 1.46 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:06.397578 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.96 GiB, free memory in arenas 9.21 MiB, will set to 1.71 GiB (RSS), difference: 31.10 MiB 2024.10.22 01:12:06.422465 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_668_291} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_667_290 to 202410_668_668_0 into Compact with storage Full 2024.10.22 01:12:06.484012 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40764 2024.10.22 01:12:06.499488 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_668_291} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:06.510738 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_668_291} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_667_290, total 41235 rows starting from the beginning of the part 2024.10.22 01:12:06.645379 [ 1258 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 01:12:06.724002 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:06.731871 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_668_291} MergeTreeSequentialSource: Reading 2 marks from part 202410_668_668_0, total 59 rows starting from the beginning of the part 2024.10.22 01:12:06.732627 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:06.984188 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_372_372_0 (state Temporary) 2024.10.22 01:12:07.041505 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_372_372_0 to 202410_379_379_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:07.087951 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 600758 2024.10.22 01:12:07.147686 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(cb9bc238079c96feea4bee8a243cfa01) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cb9bc238079c96feea4bee8a243cfa01, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:12:07.155048 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 53467, file segment info: File segment: [0, 33554431], key: cb9bc238079c96feea4bee8a243cfa01, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:12:07.188201 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:07.354982 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} MergeTask::MergeProjectionsStage: Merge sorted 5947 rows, containing 12 columns (12 merged, 0 gathered) in 1.887451724 sec., 3150.8090640839087 rows/sec., 498.17 KiB/sec. 2024.10.22 01:12:07.412224 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} MergedBlockOutputStream: filled checksums 202410_1_401_80 (state Temporary) 2024.10.22 01:12:07.640083 [ 1259 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:12:07.771833 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_401_80 to 202410_1_401_80 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:07.775657 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_401_80} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_396_79, 202410_401_401_0] -> 202410_1_401_80 2024.10.22 01:12:07.809283 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.41 MiB. 2024.10.22 01:12:08.023524 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7dab4cd80a02bb858d9d71c71cc8ec31) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7dab4cd80a02bb858d9d71c71cc8ec31, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:12:08.044652 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 7dab4cd80a02bb858d9d71c71cc8ec31, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:12:08.048138 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:08.467064 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_568_191, 202410_569_569_0] 2024.10.22 01:12:08.474473 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_568_191, 202410_569_569_0] 2024.10.22 01:12:08.485095 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_568_191, 202410_569_569_0] 2024.10.22 01:12:08.778282 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 226 entries to flush up to offset 36950 2024.10.22 01:12:08.837789 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(f9f5a108fb42d5927b613e9d4e0eea0b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f9f5a108fb42d5927b613e9d4e0eea0b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:12:08.840828 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 464, file segment info: File segment: [0, 33554431], key: f9f5a108fb42d5927b613e9d4e0eea0b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:12:08.844238 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:09.171913 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:09.177105 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:09.229335 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_383_383_0 (state Temporary) 2024.10.22 01:12:09.368746 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_668_291} MergeTask::MergeProjectionsStage: Merge sorted 41294 rows, containing 13 columns (13 merged, 0 gathered) in 2.957915994 sec., 13960.504653872196 rows/sec., 2.21 MiB/sec. 2024.10.22 01:12:09.394734 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_668_291} MergedBlockOutputStream: filled checksums 202410_1_668_291 (state Temporary) 2024.10.22 01:12:09.400992 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.96 GiB, free memory in arenas 6.76 MiB, will set to 1.70 GiB (RSS), difference: -5.05 MiB 2024.10.22 01:12:09.491949 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_383_383_0 to 202410_389_389_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:09.526736 [ 1155 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_383_161 to 202410_384_384_0 2024.10.22 01:12:09.533044 [ 1155 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.87 MiB using storage policy from min volume index 0 2024.10.22 01:12:09.552733 [ 1155 ] {} DiskLocal: Reserved 1.87 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:09.632073 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 36950 2024.10.22 01:12:09.672495 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_384_162} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_383_161 to 202410_384_384_0 into Wide with storage Full 2024.10.22 01:12:09.675570 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_668_291} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_668_291 to 202410_1_668_291 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:09.678820 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_668_291} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_667_290, 202410_668_668_0] -> 202410_1_668_291 2024.10.22 01:12:09.682929 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_384_162} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:09.686423 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(ffbfa8bdb567049d11a961d85284e0eb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ffbfa8bdb567049d11a961d85284e0eb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:12:09.687037 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_384_162} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_383_161, total 35690 rows starting from the beginning of the part 2024.10.22 01:12:09.689103 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 17914, file segment info: File segment: [0, 33554431], key: ffbfa8bdb567049d11a961d85284e0eb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:12:09.691427 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:09.798849 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:12:09.920381 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:12:09.924434 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:12:10.023358 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_384_162} MergeTreeSequentialSource: Reading 2 marks from part 202410_384_384_0, total 226 rows starting from the beginning of the part 2024.10.22 01:12:10.497445 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 40843 2024.10.22 01:12:10.510911 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} MergeTask::MergeProjectionsStage: Merge sorted 3089 rows, containing 679 columns (679 merged, 0 gathered) in 92.869880572 sec., 33.26159117438689 rows/sec., 44.37 KiB/sec. 2024.10.22 01:12:10.510031 [ 1156 ] {} DatabaseCatalog: Execution took 211 ms. 2024.10.22 01:12:10.540382 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(73e55aebde06b43c03b522f502ef75a2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 73e55aebde06b43c03b522f502ef75a2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:12:10.545326 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 559, file segment info: File segment: [0, 33554431], key: 73e55aebde06b43c03b522f502ef75a2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:12:10.551135 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:10.596719 [ 1341 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bga/rzmwvrkiwjryqsmrkqcxsjtjdrxto, size 17914 2024.10.22 01:12:10.875423 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:12:10.970087 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:10.992039 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:12:11.008829 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:11.009753 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:12:11.063470 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:12:11.096209 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:12:11.097894 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_661_661_0 (state Temporary) 2024.10.22 01:12:11.152031 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_661_661_0 to 202410_674_674_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:11.155371 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:12:11.189257 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:12:11.231704 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:12:11.282493 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/oxl/cwsvnanjxliypwejlmegaivrpeeoc (LogSeriesLimiter: on interval from 2024-10-22 01:11:53 to 2024-10-22 01:12:11 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:12:11.297124 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_668_291 to 202410_669_669_0 2024.10.22 01:12:11.303834 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.47 MiB using storage policy from min volume index 0 2024.10.22 01:12:11.309776 [ 1163 ] {} DiskLocal: Reserved 1.47 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:11.320795 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40843 2024.10.22 01:12:11.362816 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_669_292} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_668_291 to 202410_669_669_0 into Compact with storage Full 2024.10.22 01:12:11.383962 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 3150 2024.10.22 01:12:11.536143 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_669_292} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:11.551173 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_669_292} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_668_291, total 41294 rows starting from the beginning of the part 2024.10.22 01:12:11.603223 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.96 GiB, free memory in arenas 7.35 MiB, will set to 1.70 GiB (RSS), difference: -10.00 MiB 2024.10.22 01:12:11.636667 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_669_292} MergeTreeSequentialSource: Reading 2 marks from part 202410_669_669_0, total 81 rows starting from the beginning of the part 2024.10.22 01:12:11.710737 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_569_192, 202410_570_570_0] 2024.10.22 01:12:11.714525 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_569_192, 202410_570_570_0] 2024.10.22 01:12:11.825220 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_569_192, 202410_570_570_0] 2024.10.22 01:12:12.686364 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_384_162} MergeTask::MergeProjectionsStage: Merge sorted 35916 rows, containing 13 columns (13 merged, 0 gathered) in 3.126309799 sec., 11488.304841538195 rows/sec., 4.95 MiB/sec. 2024.10.22 01:12:12.742996 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_384_162} MergedBlockOutputStream: filled checksums 202410_1_384_162 (state Temporary) 2024.10.22 01:12:12.804215 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} MergedBlockOutputStream: filled checksums 202410_1_79_16 (state Temporary) 2024.10.22 01:12:12.902215 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 5860 2024.10.22 01:12:13.017271 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:13.023579 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:13.059248 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_389_389_0 (state Temporary) 2024.10.22 01:12:13.105369 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_389_389_0 to 202410_402_402_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:13.176691 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5860 2024.10.22 01:12:13.342126 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 215 ms. 2024.10.22 01:12:13.344540 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 01:12:13.388414 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_669_292} MergeTask::MergeProjectionsStage: Merge sorted 41375 rows, containing 13 columns (13 merged, 0 gathered) in 2.042393445 sec., 20258.094786433277 rows/sec., 3.21 MiB/sec. 2024.10.22 01:12:13.420816 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_669_292} MergedBlockOutputStream: filled checksums 202410_1_669_292 (state Temporary) 2024.10.22 01:12:13.429439 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_384_162} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_384_162 to 202410_1_384_162 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:13.437298 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_384_162} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_383_161, 202410_384_384_0] -> 202410_1_384_162 2024.10.22 01:12:13.474696 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.95 MiB. 2024.10.22 01:12:13.475627 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zln/efxtdlaxkznseiudgrfuvhsesqiqf. 2024.10.22 01:12:13.516096 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 332 2024.10.22 01:12:13.544122 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.96 GiB, free memory in arenas 5.75 MiB, will set to 1.69 GiB (RSS), difference: 148.85 MiB 2024.10.22 01:12:13.799733 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_669_292} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_669_292 to 202410_1_669_292 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:13.802991 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_669_292} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_668_291, 202410_669_669_0] -> 202410_1_669_292 2024.10.22 01:12:13.827060 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:12:13.915224 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:13.918290 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:14.085645 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_314_314_0 (state Temporary) 2024.10.22 01:12:14.092423 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 601910 2024.10.22 01:12:14.280172 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_314_314_0 to 202410_324_324_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:14.302773 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ed0b98044ad70a4b9fd3cb28a97fe529) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ed0b98044ad70a4b9fd3cb28a97fe529, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:12:14.310858 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 616, file segment info: File segment: [0, 33554431], key: ed0b98044ad70a4b9fd3cb28a97fe529, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:12:14.312675 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:14.339823 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 332 2024.10.22 01:12:14.368876 [ 1691 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gym/tandbbjzpxfjhbnklivneukgyrbzk, size 1580285 2024.10.22 01:12:14.481647 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:12:14.491167 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:12:14.520858 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:12:14.537744 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:12:14.555129 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:12:14.675988 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:12:14.690492 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:12:14.703854 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:12:14.731817 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:12:14.830399 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:14.836363 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:14.962914 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_373_373_0 (state Temporary) 2024.10.22 01:12:15.077740 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_79_16/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:12:15.242019 [ 1201 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 244 ms. 2024.10.22 01:12:15.281340 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_373_373_0 to 202410_380_380_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:15.335113 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 40923 2024.10.22 01:12:15.410491 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 601910 2024.10.22 01:12:15.796138 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:15.799670 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:15.868437 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_662_662_0 (state Temporary) 2024.10.22 01:12:15.900321 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_662_662_0 to 202410_675_675_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:15.917514 [ 1208 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_669_292 to 202410_670_670_0 2024.10.22 01:12:15.927823 [ 1208 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.47 MiB using storage policy from min volume index 0 2024.10.22 01:12:15.932480 [ 1208 ] {} DiskLocal: Reserved 1.47 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:15.943011 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40923 2024.10.22 01:12:15.960376 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_670_293} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_669_292 to 202410_670_670_0 into Compact with storage Full 2024.10.22 01:12:16.020706 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_670_293} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:16.028859 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_670_293} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_669_292, total 41375 rows starting from the beginning of the part 2024.10.22 01:12:16.092622 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_670_293} MergeTreeSequentialSource: Reading 2 marks from part 202410_670_670_0, total 52 rows starting from the beginning of the part 2024.10.22 01:12:16.279512 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7b94b2a6d26b53ae4b889b25fcec298f) : [0, 211]: Setting download as failed: File segment: [0, 211], key: 7b94b2a6d26b53ae4b889b25fcec298f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:16.280924 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7b94b2a6d26b53ae4b889b25fcec298f): No space left in cache to reserve 212 bytes, will continue without cache download 2024.10.22 01:12:16.471417 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.96 GiB, free memory in arenas 5.88 MiB, will set to 1.67 GiB (RSS), difference: 41.12 MiB 2024.10.22 01:12:16.801039 [ 1177 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:12:16.825512 [ 1177 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:12:17.179232 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 212 entries to flush up to offset 37162 2024.10.22 01:12:17.235139 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(27e1f2d0ac12c9e834bf831a850512b4) : [0, 2303664]: Setting download as failed: File segment: [0, 2303664], key: 27e1f2d0ac12c9e834bf831a850512b4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:17.238219 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(27e1f2d0ac12c9e834bf831a850512b4): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:12:17.383867 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.083152179 sec. 2024.10.22 01:12:17.395297 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 27e1f2d0ac12c9e834bf831a850512b4, offset: 0, size: 0 2024.10.22 01:12:17.397011 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 27e1f2d0ac12c9e834bf831a850512b4 for removal 2024.10.22 01:12:17.399743 [ 1476 ] {} CacheMetadata: Key 27e1f2d0ac12c9e834bf831a850512b4 is removed from metadata 2024.10.22 01:12:17.405046 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7b94b2a6d26b53ae4b889b25fcec298f, offset: 0, size: 0 2024.10.22 01:12:17.406719 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7b94b2a6d26b53ae4b889b25fcec298f for removal 2024.10.22 01:12:17.416543 [ 1476 ] {} CacheMetadata: Key 7b94b2a6d26b53ae4b889b25fcec298f is removed from metadata 2024.10.22 01:12:17.434127 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:17.438769 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:17.506563 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_384_384_0 (state Temporary) 2024.10.22 01:12:17.569474 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_384_384_0 to 202410_390_390_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:17.699988 [ 1188 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_384_162 to 202410_385_385_0 2024.10.22 01:12:17.705062 [ 1188 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.88 MiB using storage policy from min volume index 0 2024.10.22 01:12:17.736740 [ 1188 ] {} DiskLocal: Reserved 1.88 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:17.739979 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 37162 2024.10.22 01:12:17.981078 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_385_163} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_384_162 to 202410_385_385_0 into Wide with storage Full 2024.10.22 01:12:18.017067 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_385_163} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:18.042738 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_385_163} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_384_162, total 35916 rows starting from the beginning of the part 2024.10.22 01:12:18.251671 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_385_163} MergeTreeSequentialSource: Reading 2 marks from part 202410_385_385_0, total 243 rows starting from the beginning of the part 2024.10.22 01:12:18.258400 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6387362bdd5864f2c2f754837516c605) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 6387362bdd5864f2c2f754837516c605, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:18.264552 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6387362bdd5864f2c2f754837516c605): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:12:18.300071 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_570_193, 202410_571_571_0] 2024.10.22 01:12:18.306037 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_570_193, 202410_571_571_0] 2024.10.22 01:12:18.317203 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_570_193, 202410_571_571_0] 2024.10.22 01:12:18.540743 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.96 GiB, free memory in arenas 4.91 MiB, will set to 1.68 GiB (RSS), difference: -37.48 MiB 2024.10.22 01:12:19.300938 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a83fe4ff669f62286450e564a33757f4) : [0, 44173]: Setting download as failed: File segment: [0, 44173], key: a83fe4ff669f62286450e564a33757f4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:19.303680 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a83fe4ff669f62286450e564a33757f4): No space left in cache to reserve 44174 bytes, will continue without cache download 2024.10.22 01:12:19.338916 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a83fe4ff669f62286450e564a33757f4, offset: 0, size: 0 2024.10.22 01:12:19.343550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a83fe4ff669f62286450e564a33757f4 for removal 2024.10.22 01:12:19.346122 [ 1476 ] {} CacheMetadata: Key a83fe4ff669f62286450e564a33757f4 is removed from metadata 2024.10.22 01:12:19.347302 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6387362bdd5864f2c2f754837516c605, offset: 0, size: 0 2024.10.22 01:12:19.352067 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6387362bdd5864f2c2f754837516c605 for removal 2024.10.22 01:12:19.361226 [ 1476 ] {} CacheMetadata: Key 6387362bdd5864f2c2f754837516c605 is removed from metadata 2024.10.22 01:12:19.480620 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_670_293} MergeTask::MergeProjectionsStage: Merge sorted 41427 rows, containing 13 columns (13 merged, 0 gathered) in 3.533694878 sec., 11723.423054411209 rows/sec., 1.85 MiB/sec. 2024.10.22 01:12:19.510665 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_670_293} MergedBlockOutputStream: filled checksums 202410_1_670_293 (state Temporary) 2024.10.22 01:12:19.697353 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_670_293} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_670_293 to 202410_1_670_293 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:19.700292 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_670_293} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_669_292, 202410_670_670_0] -> 202410_1_670_293 2024.10.22 01:12:19.736139 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:12:19.920251 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:12:19.923983 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:12:19.966293 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 40985 2024.10.22 01:12:20.223272 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(851f632d44189eaf707215aacd0e6b55) : [0, 166]: Setting download as failed: File segment: [0, 166], key: 851f632d44189eaf707215aacd0e6b55, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:20.228094 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(851f632d44189eaf707215aacd0e6b55): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 01:12:20.373764 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:20.383939 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:20.546548 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_663_663_0 (state Temporary) 2024.10.22 01:12:20.602346 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_663_663_0 to 202410_676_676_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:20.657455 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_670_293 to 202410_671_671_0 2024.10.22 01:12:20.671894 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.48 MiB using storage policy from min volume index 0 2024.10.22 01:12:20.680055 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 5875 2024.10.22 01:12:20.762678 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_385_163/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:12:20.767927 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 40985 2024.10.22 01:12:20.846583 [ 1218 ] {} DatabaseCatalog: Execution took 250 ms. 2024.10.22 01:12:20.881591 [ 1269 ] {} DiskLocal: Reserved 1.48 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:20.886349 [ 1269 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 260 ms. 2024.10.22 01:12:20.895858 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:20.901237 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:20.901900 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_671_294} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_670_293 to 202410_671_671_0 into Compact with storage Full 2024.10.22 01:12:20.929379 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_390_390_0 (state Temporary) 2024.10.22 01:12:20.950083 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_671_294} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:20.970989 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_671_294} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_670_293, total 41427 rows starting from the beginning of the part 2024.10.22 01:12:20.995122 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_671_294} MergeTreeSequentialSource: Reading 2 marks from part 202410_671_671_0, total 55 rows starting from the beginning of the part 2024.10.22 01:12:21.187913 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_390_390_0 to 202410_403_403_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:21.341175 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_385_163} MergeTask::MergeProjectionsStage: Merge sorted 36159 rows, containing 13 columns (13 merged, 0 gathered) in 3.465318155 sec., 10434.539739973747 rows/sec., 4.49 MiB/sec. 2024.10.22 01:12:21.368836 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5875 2024.10.22 01:12:21.408785 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0adebb9a815e45948bad5d8137dc95b5) : [0, 2572]: Setting download as failed: File segment: [0, 2572], key: 0adebb9a815e45948bad5d8137dc95b5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:21.411544 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0adebb9a815e45948bad5d8137dc95b5): No space left in cache to reserve 2573 bytes, will continue without cache download 2024.10.22 01:12:21.456772 [ 1238 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 250 ms. 2024.10.22 01:12:21.498511 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_571_194, 202410_572_572_0] 2024.10.22 01:12:21.502153 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_571_194, 202410_572_572_0] 2024.10.22 01:12:21.622681 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_571_194, 202410_572_572_0] 2024.10.22 01:12:21.677481 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0adebb9a815e45948bad5d8137dc95b5, offset: 0, size: 0 2024.10.22 01:12:21.677531 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_385_163} MergedBlockOutputStream: filled checksums 202410_1_385_163 (state Temporary) 2024.10.22 01:12:21.679263 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0adebb9a815e45948bad5d8137dc95b5 for removal 2024.10.22 01:12:21.681191 [ 1476 ] {} CacheMetadata: Key 0adebb9a815e45948bad5d8137dc95b5 is removed from metadata 2024.10.22 01:12:21.783753 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 851f632d44189eaf707215aacd0e6b55, offset: 0, size: 0 2024.10.22 01:12:21.785436 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 851f632d44189eaf707215aacd0e6b55 for removal 2024.10.22 01:12:21.789748 [ 1476 ] {} CacheMetadata: Key 851f632d44189eaf707215aacd0e6b55 is removed from metadata 2024.10.22 01:12:21.845148 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 333 2024.10.22 01:12:22.225638 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:22.237441 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:22.326965 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_315_315_0 (state Temporary) 2024.10.22 01:12:22.382223 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_385_163} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_385_163 to 202410_1_385_163 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:22.384569 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_385_163} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_384_162, 202410_385_385_0] -> 202410_1_385_163 2024.10.22 01:12:22.415483 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 603446 2024.10.22 01:12:22.474027 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.96 GiB, free memory in arenas 2.38 MiB, will set to 1.68 GiB (RSS), difference: 29.88 MiB 2024.10.22 01:12:22.499916 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_315_315_0 to 202410_325_325_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:22.501179 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.96 MiB. 2024.10.22 01:12:22.544800 [ 1230 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_320_66 to 202410_325_325_0 2024.10.22 01:12:22.573814 [ 1230 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:22.577069 [ 1230 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:22.590728 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 333 2024.10.22 01:12:22.599281 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_320_66 to 202410_325_325_0 into Compact with storage Full 2024.10.22 01:12:22.608493 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_328_110, 202410_329_329_0] 2024.10.22 01:12:22.611023 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_328_110, 202410_329_329_0] 2024.10.22 01:12:22.753153 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_328_110, 202410_329_329_0] 2024.10.22 01:12:22.774279 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:22.785562 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_320_66, total 364 rows starting from the beginning of the part 2024.10.22 01:12:22.825647 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_321_321_0, total 1 rows starting from the beginning of the part 2024.10.22 01:12:22.875878 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_322_322_0, total 1 rows starting from the beginning of the part 2024.10.22 01:12:22.904018 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:22.906789 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:22.931427 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_323_323_0, total 1 rows starting from the beginning of the part 2024.10.22 01:12:22.978900 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c8be276ab054c2cd558e19f59c4b7f29) : [0, 168]: Setting download as failed: File segment: [0, 168], key: c8be276ab054c2cd558e19f59c4b7f29, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:22.980043 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c8be276ab054c2cd558e19f59c4b7f29): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:12:23.072623 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_324_324_0, total 1 rows starting from the beginning of the part 2024.10.22 01:12:23.106077 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_325_325_0, total 1 rows starting from the beginning of the part 2024.10.22 01:12:23.333039 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_374_374_0 (state Temporary) 2024.10.22 01:12:23.464145 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_671_294} MergeTask::MergeProjectionsStage: Merge sorted 41482 rows, containing 13 columns (13 merged, 0 gathered) in 2.57327955 sec., 16120.285104663424 rows/sec., 2.55 MiB/sec. 2024.10.22 01:12:23.491916 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_374_374_0 to 202410_381_381_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:23.497066 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_671_294} MergedBlockOutputStream: filled checksums 202410_1_671_294 (state Temporary) 2024.10.22 01:12:23.520538 [ 1224 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_376_75 to 202410_381_381_0 2024.10.22 01:12:23.534396 [ 1224 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:23.540119 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:23.574288 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_376_75 to 202410_381_381_0 into Compact with storage Full 2024.10.22 01:12:23.623754 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:23.634837 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 603446 2024.10.22 01:12:23.640388 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} MergeTreeSequentialSource: Reading 76 marks from part 202410_1_376_75, total 609356 rows starting from the beginning of the part 2024.10.22 01:12:23.658546 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_316_63, 202410_317_317_0, 202410_318_318_0, 202410_319_319_0, 202410_320_320_0, 202410_321_321_0] 2024.10.22 01:12:23.672859 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_316_63, 202410_317_317_0, 202410_318_318_0, 202410_319_319_0, 202410_320_320_0, 202410_321_321_0] 2024.10.22 01:12:23.675593 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_377_377_0, total 1780 rows starting from the beginning of the part 2024.10.22 01:12:23.689380 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_378_378_0, total 1676 rows starting from the beginning of the part 2024.10.22 01:12:23.702495 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_379_379_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:12:23.713455 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_671_294} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_671_294 to 202410_1_671_294 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:23.719993 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_671_294} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_670_293, 202410_671_671_0] -> 202410_1_671_294 2024.10.22 01:12:23.724040 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_380_380_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:12:23.744486 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_381_381_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:12:23.796412 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_316_63, 202410_317_317_0, 202410_318_318_0, 202410_319_319_0, 202410_320_320_0, 202410_321_321_0] 2024.10.22 01:12:23.860710 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:12:23.945678 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(861f5fda0e5b182c2a9d906e543e9430) : [0, 1050940]: Setting download as failed: File segment: [0, 1050940], key: 861f5fda0e5b182c2a9d906e543e9430, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:23.947918 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(861f5fda0e5b182c2a9d906e543e9430): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:12:24.084949 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 861f5fda0e5b182c2a9d906e543e9430, offset: 0, size: 0 2024.10.22 01:12:24.187159 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 861f5fda0e5b182c2a9d906e543e9430 for removal 2024.10.22 01:12:24.189469 [ 1476 ] {} CacheMetadata: Key 861f5fda0e5b182c2a9d906e543e9430 is removed from metadata 2024.10.22 01:12:24.191919 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c8be276ab054c2cd558e19f59c4b7f29, offset: 0, size: 0 2024.10.22 01:12:24.193245 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c8be276ab054c2cd558e19f59c4b7f29 for removal 2024.10.22 01:12:24.195943 [ 1476 ] {} CacheMetadata: Key c8be276ab054c2cd558e19f59c4b7f29 is removed from metadata 2024.10.22 01:12:24.564564 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.96 GiB, free memory in arenas 3.94 MiB, will set to 1.67 GiB (RSS), difference: -1.88 MiB 2024.10.22 01:12:24.834969 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 98 entries to flush up to offset 41083 2024.10.22 01:12:25.019287 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:25.026258 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:25.240217 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_664_664_0 (state Temporary) 2024.10.22 01:12:25.249766 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5e067326661cf91ef1a67a51d11cab99) : [0, 166]: Setting download as failed: File segment: [0, 166], key: 5e067326661cf91ef1a67a51d11cab99, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:25.253252 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5e067326661cf91ef1a67a51d11cab99): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 01:12:25.286222 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 209 entries to flush up to offset 37371 2024.10.22 01:12:25.305115 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_664_664_0 to 202410_677_677_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:25.470116 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:25.475738 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:25.479084 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} MergeTask::MergeProjectionsStage: Merge sorted 369 rows, containing 32 columns (32 merged, 0 gathered) in 2.888881547 sec., 127.73109384951879 rows/sec., 12.96 KiB/sec. 2024.10.22 01:12:25.488526 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41083 2024.10.22 01:12:25.539560 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_385_385_0 (state Temporary) 2024.10.22 01:12:25.574334 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_671_294 to 202410_672_672_0 2024.10.22 01:12:25.581588 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.48 MiB using storage policy from min volume index 0 2024.10.22 01:12:25.585948 [ 1247 ] {} DiskLocal: Reserved 1.48 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:25.589614 [ 1247 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 01:12:25.641246 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} MergedBlockOutputStream: filled checksums 202410_1_325_67 (state Temporary) 2024.10.22 01:12:25.690117 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_385_385_0 to 202410_391_391_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:25.801179 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 37371 2024.10.22 01:12:25.862347 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_672_295} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_671_294 to 202410_672_672_0 into Compact with storage Full 2024.10.22 01:12:25.891418 [ 1151 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_385_163 to 202410_386_386_0 2024.10.22 01:12:25.896167 [ 1151 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.89 MiB using storage policy from min volume index 0 2024.10.22 01:12:25.899406 [ 1151 ] {} DiskLocal: Reserved 1.89 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:25.947831 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_386_164} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_385_163 to 202410_386_386_0 into Wide with storage Full 2024.10.22 01:12:25.989693 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_386_164} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:26.008271 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_386_164} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_385_163, total 36159 rows starting from the beginning of the part 2024.10.22 01:12:26.049710 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_672_295} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:26.232136 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(38c3b3dbf16d23c9d69f103779e90d39) : [0, 31313]: Setting download as failed: File segment: [0, 31313], key: 38c3b3dbf16d23c9d69f103779e90d39, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:26.246674 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_672_295} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_671_294, total 41482 rows starting from the beginning of the part 2024.10.22 01:12:26.252897 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_386_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_386_386_0, total 235 rows starting from the beginning of the part 2024.10.22 01:12:26.290385 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_672_295} MergeTreeSequentialSource: Reading 2 marks from part 202410_672_672_0, total 53 rows starting from the beginning of the part 2024.10.22 01:12:26.334729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(38c3b3dbf16d23c9d69f103779e90d39): No space left in cache to reserve 31314 bytes, will continue without cache download 2024.10.22 01:12:26.376932 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 38c3b3dbf16d23c9d69f103779e90d39, offset: 0, size: 0 2024.10.22 01:12:26.378726 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 38c3b3dbf16d23c9d69f103779e90d39 for removal 2024.10.22 01:12:26.379934 [ 1476 ] {} CacheMetadata: Key 38c3b3dbf16d23c9d69f103779e90d39 is removed from metadata 2024.10.22 01:12:26.380433 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5e067326661cf91ef1a67a51d11cab99, offset: 0, size: 0 2024.10.22 01:12:26.381488 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5e067326661cf91ef1a67a51d11cab99 for removal 2024.10.22 01:12:26.383980 [ 1476 ] {} CacheMetadata: Key 5e067326661cf91ef1a67a51d11cab99 is removed from metadata 2024.10.22 01:12:26.591446 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_325_67 to 202410_1_325_67 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:26.596707 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_325_67} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_320_66, 202410_325_325_0] -> 202410_1_325_67 2024.10.22 01:12:26.643802 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:12:26.778665 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_79_16 to 202410_1_79_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:26.787586 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_79_16} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_74_15, 202410_79_79_0] -> 202410_1_79_16 2024.10.22 01:12:26.934341 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_572_195, 202410_573_573_0] 2024.10.22 01:12:27.041786 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_572_195, 202410_573_573_0] 2024.10.22 01:12:27.053457 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_572_195, 202410_573_573_0] 2024.10.22 01:12:27.496997 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0dbd05ba16a452d76f7682a1ed2cd564) : [0, 150]: Setting download as failed: File segment: [0, 150], key: 0dbd05ba16a452d76f7682a1ed2cd564, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:27.498592 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0dbd05ba16a452d76f7682a1ed2cd564): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:12:27.614475 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:27.627883 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:27.649055 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 36.66 MiB. 2024.10.22 01:12:27.687685 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.96 GiB, free memory in arenas 8.64 MiB, will set to 1.68 GiB (RSS), difference: -42.41 MiB 2024.10.22 01:12:28.356691 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5882ed984e19974bad91d8ce303d0eff) : [0, 24112]: Setting download as failed: File segment: [0, 24112], key: 5882ed984e19974bad91d8ce303d0eff, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:28.359451 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5882ed984e19974bad91d8ce303d0eff): No space left in cache to reserve 24113 bytes, will continue without cache download 2024.10.22 01:12:28.483055 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5882ed984e19974bad91d8ce303d0eff, offset: 0, size: 0 2024.10.22 01:12:28.484848 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5882ed984e19974bad91d8ce303d0eff for removal 2024.10.22 01:12:28.490856 [ 1476 ] {} CacheMetadata: Key 5882ed984e19974bad91d8ce303d0eff is removed from metadata 2024.10.22 01:12:28.492396 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0dbd05ba16a452d76f7682a1ed2cd564, offset: 0, size: 0 2024.10.22 01:12:28.495817 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0dbd05ba16a452d76f7682a1ed2cd564 for removal 2024.10.22 01:12:28.498321 [ 1476 ] {} CacheMetadata: Key 0dbd05ba16a452d76f7682a1ed2cd564 is removed from metadata 2024.10.22 01:12:28.872037 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 5900 2024.10.22 01:12:29.182154 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_386_164} MergeTask::MergeProjectionsStage: Merge sorted 36394 rows, containing 13 columns (13 merged, 0 gathered) in 3.243243742 sec., 11221.481607656486 rows/sec., 4.83 MiB/sec. 2024.10.22 01:12:29.203021 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:29.208139 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:29.280881 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_391_391_0 (state Temporary) 2024.10.22 01:12:29.366428 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_391_391_0 to 202410_404_404_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:29.397139 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_386_164} MergedBlockOutputStream: filled checksums 202410_1_386_164 (state Temporary) 2024.10.22 01:12:29.442600 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5900 2024.10.22 01:12:29.452204 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_672_295} MergeTask::MergeProjectionsStage: Merge sorted 41535 rows, containing 13 columns (13 merged, 0 gathered) in 3.677014617 sec., 11295.848487512281 rows/sec., 1.79 MiB/sec. 2024.10.22 01:12:29.502451 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 41153 2024.10.22 01:12:29.697227 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_672_295} MergedBlockOutputStream: filled checksums 202410_1_672_295 (state Temporary) 2024.10.22 01:12:29.919103 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:29.925340 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:29.983707 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_381_76/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:12:29.986247 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_665_665_0 (state Temporary) 2024.10.22 01:12:30.024320 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:12:30.025986 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f20b15348fbceb9d71f84f10ff6d10ec) : [0, 155]: Setting download as failed: File segment: [0, 155], key: f20b15348fbceb9d71f84f10ff6d10ec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:30.034456 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:12:30.036365 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f20b15348fbceb9d71f84f10ff6d10ec): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:12:30.071755 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_665_665_0 to 202410_678_678_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:30.095089 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 334 2024.10.22 01:12:30.099835 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_672_295} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_672_295 to 202410_1_672_295 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:30.105314 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_672_295} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_671_294, 202410_672_672_0] -> 202410_1_672_295 2024.10.22 01:12:30.119113 [ 1160 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_673_673_0 to 202410_678_678_0 2024.10.22 01:12:30.150578 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41153 2024.10.22 01:12:30.157933 [ 1160 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:30.170707 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:12:30.171579 [ 1160 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:30.193457 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_673_673_0 to 202410_678_678_0 into Compact with storage Full 2024.10.22 01:12:30.397185 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:30.402186 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_673_673_0, total 60 rows starting from the beginning of the part 2024.10.22 01:12:30.429804 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_674_674_0, total 79 rows starting from the beginning of the part 2024.10.22 01:12:30.476658 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_675_675_0, total 80 rows starting from the beginning of the part 2024.10.22 01:12:30.492560 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_676_676_0, total 62 rows starting from the beginning of the part 2024.10.22 01:12:30.527312 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_677_677_0, total 98 rows starting from the beginning of the part 2024.10.22 01:12:30.565377 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_678_678_0, total 70 rows starting from the beginning of the part 2024.10.22 01:12:30.604429 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.96 GiB, free memory in arenas 10.91 MiB, will set to 1.68 GiB (RSS), difference: 57.31 MiB 2024.10.22 01:12:30.677785 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_573_196, 202410_574_574_0] 2024.10.22 01:12:30.679930 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 604598 2024.10.22 01:12:30.680020 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_573_196, 202410_574_574_0] 2024.10.22 01:12:30.765738 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:30.773235 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:30.780294 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_386_164} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_386_164 to 202410_1_386_164 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:30.783761 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_386_164} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_385_163, 202410_386_386_0] -> 202410_1_386_164 2024.10.22 01:12:30.787726 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_573_196, 202410_574_574_0] 2024.10.22 01:12:30.840047 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_316_316_0 (state Temporary) 2024.10.22 01:12:30.838450 [ 1396 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_386_164}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 01:12:30.866864 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.95 MiB. 2024.10.22 01:12:31.046893 [ 1179 ] {} DatabaseCatalog: Execution took 281 ms. 2024.10.22 01:12:31.134263 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_316_316_0 to 202410_326_326_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:31.188110 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_329_111, 202410_330_330_0] 2024.10.22 01:12:31.197170 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_79_79_0 (state Temporary) 2024.10.22 01:12:31.190195 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_329_111, 202410_330_330_0] 2024.10.22 01:12:31.216597 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 334 2024.10.22 01:12:31.231097 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(99f90bc82dd46ce25f1ecb6c0525b48f) : [0, 235582]: Setting download as failed: File segment: [0, 235582], key: 99f90bc82dd46ce25f1ecb6c0525b48f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:31.235770 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(99f90bc82dd46ce25f1ecb6c0525b48f): No space left in cache to reserve 235583 bytes, will continue without cache download 2024.10.22 01:12:31.300200 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:31.302904 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:31.311539 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 99f90bc82dd46ce25f1ecb6c0525b48f, offset: 0, size: 0 2024.10.22 01:12:31.314617 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 99f90bc82dd46ce25f1ecb6c0525b48f for removal 2024.10.22 01:12:31.319579 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_329_111, 202410_330_330_0] 2024.10.22 01:12:31.320683 [ 1476 ] {} CacheMetadata: Key 99f90bc82dd46ce25f1ecb6c0525b48f is removed from metadata 2024.10.22 01:12:31.322088 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f20b15348fbceb9d71f84f10ff6d10ec, offset: 0, size: 0 2024.10.22 01:12:31.324957 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f20b15348fbceb9d71f84f10ff6d10ec for removal 2024.10.22 01:12:31.329420 [ 1476 ] {} CacheMetadata: Key f20b15348fbceb9d71f84f10ff6d10ec is removed from metadata 2024.10.22 01:12:31.381355 [ 1241 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 01:12:31.481054 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_375_375_0 (state Temporary) 2024.10.22 01:12:31.546727 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_375_375_0 to 202410_382_382_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:31.582860 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 604598 2024.10.22 01:12:31.831334 [ 1175 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:12:31.867951 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} MergeTask::MergeProjectionsStage: Merge sorted 449 rows, containing 13 columns (13 merged, 0 gathered) in 1.680406513 sec., 267.1972505024443 rows/sec., 45.10 KiB/sec. 2024.10.22 01:12:31.875794 [ 1175 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:12:31.915653 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} MergedBlockOutputStream: filled checksums 202410_673_678_1 (state Temporary) 2024.10.22 01:12:32.067541 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_673_678_1 to 202410_673_678_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:32.072541 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_673_678_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_673_673_0, 202410_678_678_0] -> 202410_673_678_1 2024.10.22 01:12:32.198857 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:12:32.403947 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3469e0c4d8215c9e27f1507a4fa4fb36) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 3469e0c4d8215c9e27f1507a4fa4fb36, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:32.405759 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3469e0c4d8215c9e27f1507a4fa4fb36): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:12:32.494287 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.96 GiB, free memory in arenas 10.97 MiB, will set to 1.68 GiB (RSS), difference: -3.16 MiB 2024.10.22 01:12:33.278506 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(58f84dd602484f7138d655e10c7983d6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 58f84dd602484f7138d655e10c7983d6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:12:33.282311 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 25578, file segment info: File segment: [0, 33554431], key: 58f84dd602484f7138d655e10c7983d6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:12:33.284904 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:33.341083 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 189 entries to flush up to offset 37560 2024.10.22 01:12:33.831482 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:33.837556 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:33.869791 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ab00c2358cf2493acb979a968dc5bf4d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ab00c2358cf2493acb979a968dc5bf4d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:12:33.875212 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 805765, file segment info: File segment: [0, 33554431], key: ab00c2358cf2493acb979a968dc5bf4d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:12:33.879786 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:33.988922 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_386_386_0 (state Temporary) 2024.10.22 01:12:34.040800 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_386_386_0 to 202410_392_392_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:34.064391 [ 1198 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_386_164 to 202410_387_387_0 2024.10.22 01:12:34.075345 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 37560 2024.10.22 01:12:34.075769 [ 1198 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.90 MiB using storage policy from min volume index 0 2024.10.22 01:12:34.079273 [ 1198 ] {} DiskLocal: Reserved 1.90 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:34.101208 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_387_165} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_386_164 to 202410_387_387_0 into Wide with storage Full 2024.10.22 01:12:34.123838 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_387_165} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:34.249219 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_387_165} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_386_164, total 36394 rows starting from the beginning of the part 2024.10.22 01:12:34.293620 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 41240 2024.10.22 01:12:34.440997 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(64000bf7f86c08a9eb120fd2029a2c53) : [0, 154140]: Setting download as failed: File segment: [0, 154140], key: 64000bf7f86c08a9eb120fd2029a2c53, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:34.442018 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(64000bf7f86c08a9eb120fd2029a2c53): No space left in cache to reserve 154141 bytes, will continue without cache download 2024.10.22 01:12:34.516818 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_387_165} MergeTreeSequentialSource: Reading 2 marks from part 202410_387_387_0, total 205 rows starting from the beginning of the part 2024.10.22 01:12:34.567804 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 64000bf7f86c08a9eb120fd2029a2c53, offset: 0, size: 0 2024.10.22 01:12:34.569573 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 64000bf7f86c08a9eb120fd2029a2c53 for removal 2024.10.22 01:12:34.573058 [ 1476 ] {} CacheMetadata: Key 64000bf7f86c08a9eb120fd2029a2c53 is removed from metadata 2024.10.22 01:12:34.576179 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3469e0c4d8215c9e27f1507a4fa4fb36, offset: 0, size: 0 2024.10.22 01:12:34.675459 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:34.577796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3469e0c4d8215c9e27f1507a4fa4fb36 for removal 2024.10.22 01:12:34.679899 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:34.681032 [ 1476 ] {} CacheMetadata: Key 3469e0c4d8215c9e27f1507a4fa4fb36 is removed from metadata 2024.10.22 01:12:34.857665 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_666_666_0 (state Temporary) 2024.10.22 01:12:34.901759 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_666_666_0 to 202410_679_679_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:34.948434 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41240 2024.10.22 01:12:34.950167 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(08349b656a1ad96fd38f7c372d6d0d21) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 08349b656a1ad96fd38f7c372d6d0d21, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:12:34.994298 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 754, file segment info: File segment: [0, 33554431], key: 08349b656a1ad96fd38f7c372d6d0d21, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:12:34.996031 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:35.315547 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_574_197, 202410_575_575_0] 2024.10.22 01:12:35.320262 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_574_197, 202410_575_575_0] 2024.10.22 01:12:35.340231 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_574_197, 202410_575_575_0] 2024.10.22 01:12:35.501584 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:12:35.608163 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:12:35.644979 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:12:35.675351 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:12:35.692939 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:12:35.743738 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:12:35.779916 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:12:35.813279 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:12:35.821302 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(6d27075e331496096b2a322e1eba96ad) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6d27075e331496096b2a322e1eba96ad, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:12:35.829957 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 393, file segment info: File segment: [0, 33554431], key: 6d27075e331496096b2a322e1eba96ad, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:12:35.834301 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:35.867677 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:12:35.932952 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rhq/xfhyoxvsgpdlfbyahngeydsgumcwx (LogSeriesLimiter: on interval from 2024-10-22 01:12:11 to 2024-10-22 01:12:35 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:12:36.275174 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.96 GiB, free memory in arenas 12.15 MiB, will set to 1.69 GiB (RSS), difference: -57.70 MiB 2024.10.22 01:12:36.464329 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_387_165/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:12:36.544474 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7a1d126a6c54657113ed27a02c2abf14) : [0, 181]: Setting download as failed: File segment: [0, 181], key: 7a1d126a6c54657113ed27a02c2abf14, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:36.546807 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7a1d126a6c54657113ed27a02c2abf14): No space left in cache to reserve 182 bytes, will continue without cache download 2024.10.22 01:12:36.946228 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 5919 2024.10.22 01:12:37.096606 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:37.098967 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:37.134603 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_392_392_0 (state Temporary) 2024.10.22 01:12:37.191022 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_387_165} MergeTask::MergeProjectionsStage: Merge sorted 36599 rows, containing 13 columns (13 merged, 0 gathered) in 3.104290431 sec., 11789.811814808252 rows/sec., 5.08 MiB/sec. 2024.10.22 01:12:37.285045 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_392_392_0 to 202410_405_405_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:37.285284 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_387_165} MergedBlockOutputStream: filled checksums 202410_1_387_165 (state Temporary) 2024.10.22 01:12:37.327222 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_79_79_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:12:37.383482 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5919 2024.10.22 01:12:37.431242 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(9c78c1355b35309e09a722975ea45f4f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9c78c1355b35309e09a722975ea45f4f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:12:37.437606 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 8263, file segment info: File segment: [0, 33554431], key: 9c78c1355b35309e09a722975ea45f4f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:12:37.441507 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:38.084652 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ffb04ac61277b459e537d4f2eef0723f) : [0, 347746]: Setting download as failed: File segment: [0, 347746], key: ffb04ac61277b459e537d4f2eef0723f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:38.086053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ffb04ac61277b459e537d4f2eef0723f): No space left in cache to reserve 347747 bytes, will continue without cache download 2024.10.22 01:12:38.093512 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_387_165} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_387_165 to 202410_1_387_165 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:38.096051 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_387_165} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_386_164, 202410_387_387_0] -> 202410_1_387_165 2024.10.22 01:12:38.105133 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ffb04ac61277b459e537d4f2eef0723f, offset: 0, size: 0 2024.10.22 01:12:38.106993 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ffb04ac61277b459e537d4f2eef0723f for removal 2024.10.22 01:12:38.109744 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7a1d126a6c54657113ed27a02c2abf14, offset: 0, size: 0 2024.10.22 01:12:38.111896 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7a1d126a6c54657113ed27a02c2abf14 for removal 2024.10.22 01:12:38.129122 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.95 MiB. 2024.10.22 01:12:38.145280 [ 1476 ] {} CacheMetadata: Key ffb04ac61277b459e537d4f2eef0723f is removed from metadata 2024.10.22 01:12:38.169075 [ 1476 ] {} CacheMetadata: Key 7a1d126a6c54657113ed27a02c2abf14 is removed from metadata 2024.10.22 01:12:38.585377 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 605750 2024.10.22 01:12:38.697113 [ 1154 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 01:12:38.781601 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(2e12083ae0a5f22060606fb7150df6a8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2e12083ae0a5f22060606fb7150df6a8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:12:38.932494 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:38.936793 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:38.969175 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 41307 2024.10.22 01:12:38.995187 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 521, file segment info: File segment: [0, 33554431], key: 2e12083ae0a5f22060606fb7150df6a8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:12:38.997646 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:39.031032 [ 1726 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/oxl/cwsvnanjxliypwejlmegaivrpeeoc, size 8263 2024.10.22 01:12:39.095267 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:39.097361 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:39.161582 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_376_376_0 (state Temporary) 2024.10.22 01:12:39.350270 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_376_376_0 to 202410_383_383_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:39.368259 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_667_667_0 (state Temporary) 2024.10.22 01:12:39.414169 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:12:39.497272 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 605750 2024.10.22 01:12:39.549759 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:12:39.572368 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.96 GiB, free memory in arenas 13.08 MiB, will set to 1.68 GiB (RSS), difference: 46.97 MiB 2024.10.22 01:12:39.575235 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:12:39.590183 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:12:39.604851 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:12:39.647164 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:12:39.680481 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:12:39.681642 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_667_667_0 to 202410_680_680_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:39.744209 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41307 2024.10.22 01:12:39.772833 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_330_112, 202410_331_331_0] 2024.10.22 01:12:39.775106 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_330_112, 202410_331_331_0] 2024.10.22 01:12:39.787233 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_330_112, 202410_331_331_0] 2024.10.22 01:12:39.788590 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6aca793f8d160f4624c41fc279f6cf90) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 6aca793f8d160f4624c41fc279f6cf90, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:39.789806 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6aca793f8d160f4624c41fc279f6cf90): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:12:39.923874 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:12:39.928037 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:12:40.624968 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(aea6719b07a3f9a68bf304a85f67cef8) : [0, 67719]: Setting download as failed: File segment: [0, 67719], key: aea6719b07a3f9a68bf304a85f67cef8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:40.630878 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(aea6719b07a3f9a68bf304a85f67cef8): No space left in cache to reserve 67720 bytes, will continue without cache download 2024.10.22 01:12:40.637803 [ 1270 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 01:12:40.662497 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aea6719b07a3f9a68bf304a85f67cef8, offset: 0, size: 0 2024.10.22 01:12:40.666709 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aea6719b07a3f9a68bf304a85f67cef8 for removal 2024.10.22 01:12:40.671366 [ 1476 ] {} CacheMetadata: Key aea6719b07a3f9a68bf304a85f67cef8 is removed from metadata 2024.10.22 01:12:40.673336 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6aca793f8d160f4624c41fc279f6cf90, offset: 0, size: 0 2024.10.22 01:12:40.675212 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6aca793f8d160f4624c41fc279f6cf90 for removal 2024.10.22 01:12:40.678271 [ 1476 ] {} CacheMetadata: Key 6aca793f8d160f4624c41fc279f6cf90 is removed from metadata 2024.10.22 01:12:40.860079 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_575_198, 202410_576_576_0] 2024.10.22 01:12:40.867008 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_575_198, 202410_576_576_0] 2024.10.22 01:12:40.880806 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_575_198, 202410_576_576_0] 2024.10.22 01:12:40.928364 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_273_56, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0, 202410_277_277_0] 2024.10.22 01:12:40.932780 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_273_56, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0, 202410_277_277_0] 2024.10.22 01:12:40.960815 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_273_56, 202410_274_274_0, 202410_275_275_0, 202410_276_276_0, 202410_277_277_0] 2024.10.22 01:12:41.293468 [ 1195 ] {} DatabaseCatalog: Execution took 247 ms. 2024.10.22 01:12:41.492448 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.96 GiB, free memory in arenas 9.19 MiB, will set to 1.68 GiB (RSS), difference: -4.57 MiB 2024.10.22 01:12:41.605505 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 157 entries to flush up to offset 37717 2024.10.22 01:12:41.773292 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_381_76/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:12:41.779170 [ 1217 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 01:12:41.853210 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_79_79_0 to 202410_82_82_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:41.859343 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(740364729a5b4a089425ac8e030fe541) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 740364729a5b4a089425ac8e030fe541, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:41.863085 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(740364729a5b4a089425ac8e030fe541): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:12:41.877567 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:41.880649 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:41.939529 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_387_387_0 (state Temporary) 2024.10.22 01:12:41.993115 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_387_387_0 to 202410_393_393_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:42.022928 [ 1230 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_387_165 to 202410_388_388_0 2024.10.22 01:12:42.029317 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 37717 2024.10.22 01:12:42.040297 [ 1230 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.91 MiB using storage policy from min volume index 0 2024.10.22 01:12:42.044059 [ 1230 ] {} DiskLocal: Reserved 1.91 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:42.069424 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_388_166} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_387_165 to 202410_388_388_0 into Wide with storage Full 2024.10.22 01:12:42.106358 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_341_68, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0, 202410_345_345_0, 202410_346_346_0] 2024.10.22 01:12:42.108295 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_341_68, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0, 202410_345_345_0, 202410_346_346_0] 2024.10.22 01:12:42.233488 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_388_166} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:42.242911 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_341_68, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0, 202410_345_345_0, 202410_346_346_0] 2024.10.22 01:12:42.245348 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_388_166} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_387_165, total 36599 rows starting from the beginning of the part 2024.10.22 01:12:42.402904 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_388_166} MergeTreeSequentialSource: Reading 2 marks from part 202410_388_388_0, total 229 rows starting from the beginning of the part 2024.10.22 01:12:42.703360 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8fc217cd1d497fc30f6b7d5efdfb2cd2) : [0, 74473]: Setting download as failed: File segment: [0, 74473], key: 8fc217cd1d497fc30f6b7d5efdfb2cd2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:42.704706 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8fc217cd1d497fc30f6b7d5efdfb2cd2): No space left in cache to reserve 74474 bytes, will continue without cache download 2024.10.22 01:12:42.724805 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8fc217cd1d497fc30f6b7d5efdfb2cd2, offset: 0, size: 0 2024.10.22 01:12:42.727495 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8fc217cd1d497fc30f6b7d5efdfb2cd2 for removal 2024.10.22 01:12:42.732077 [ 1476 ] {} CacheMetadata: Key 8fc217cd1d497fc30f6b7d5efdfb2cd2 is removed from metadata 2024.10.22 01:12:42.732840 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 740364729a5b4a089425ac8e030fe541, offset: 0, size: 0 2024.10.22 01:12:42.737467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 740364729a5b4a089425ac8e030fe541 for removal 2024.10.22 01:12:42.743448 [ 1476 ] {} CacheMetadata: Key 740364729a5b4a089425ac8e030fe541 is removed from metadata 2024.10.22 01:12:43.467427 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.96 GiB, free memory in arenas 5.84 MiB, will set to 1.68 GiB (RSS), difference: -39.51 MiB 2024.10.22 01:12:43.491877 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d82bcac318463b76e4bb249b2491c5ed) : [0, 167]: Setting download as failed: File segment: [0, 167], key: d82bcac318463b76e4bb249b2491c5ed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:43.493384 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d82bcac318463b76e4bb249b2491c5ed): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:12:43.871870 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 41383 2024.10.22 01:12:44.179562 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:44.183960 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:44.246859 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_668_668_0 (state Temporary) 2024.10.22 01:12:44.351013 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c33beb3d530ecb0021ae08956070a5f0) : [0, 113283]: Setting download as failed: File segment: [0, 113283], key: c33beb3d530ecb0021ae08956070a5f0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:44.353684 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c33beb3d530ecb0021ae08956070a5f0): No space left in cache to reserve 113284 bytes, will continue without cache download 2024.10.22 01:12:44.436011 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_668_668_0 to 202410_681_681_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:44.480958 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c33beb3d530ecb0021ae08956070a5f0, offset: 0, size: 0 2024.10.22 01:12:44.486045 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c33beb3d530ecb0021ae08956070a5f0 for removal 2024.10.22 01:12:44.488509 [ 1476 ] {} CacheMetadata: Key c33beb3d530ecb0021ae08956070a5f0 is removed from metadata 2024.10.22 01:12:44.490788 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d82bcac318463b76e4bb249b2491c5ed, offset: 0, size: 0 2024.10.22 01:12:44.497832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d82bcac318463b76e4bb249b2491c5ed for removal 2024.10.22 01:12:44.504756 [ 1476 ] {} CacheMetadata: Key d82bcac318463b76e4bb249b2491c5ed is removed from metadata 2024.10.22 01:12:44.506383 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41383 2024.10.22 01:12:44.772953 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_388_166} MergeTask::MergeProjectionsStage: Merge sorted 36828 rows, containing 13 columns (13 merged, 0 gathered) in 2.71535075 sec., 13562.888698633133 rows/sec., 5.84 MiB/sec. 2024.10.22 01:12:44.884934 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 5928 2024.10.22 01:12:44.996647 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:44.998857 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:45.037131 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_393_393_0 (state Temporary) 2024.10.22 01:12:45.089319 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_388_166} MergedBlockOutputStream: filled checksums 202410_1_388_166 (state Temporary) 2024.10.22 01:12:45.198893 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(34c494b6e6bc17546d545a09dbf08be3) : [0, 167]: Setting download as failed: File segment: [0, 167], key: 34c494b6e6bc17546d545a09dbf08be3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:45.199943 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(34c494b6e6bc17546d545a09dbf08be3): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:12:45.284622 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_393_393_0 to 202410_406_406_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:45.290995 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3150 2024.10.22 01:12:45.298138 [ 1232 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_401_80 to 202410_406_406_0 2024.10.22 01:12:45.298251 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} MergeTask::MergeProjectionsStage: Merge sorted 617036 rows, containing 4 columns (4 merged, 0 gathered) in 21.746642549 sec., 28373.851209890505 rows/sec., 415.98 KiB/sec. 2024.10.22 01:12:45.304663 [ 1232 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:45.307099 [ 1232 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:45.311292 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5928 2024.10.22 01:12:45.331459 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_401_80 to 202410_406_406_0 into Compact with storage Full 2024.10.22 01:12:45.434644 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.96 GiB, free memory in arenas 12.59 MiB, will set to 1.68 GiB (RSS), difference: 40.60 MiB 2024.10.22 01:12:45.474739 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} MergedBlockOutputStream: filled checksums 202410_1_381_76 (state Temporary) 2024.10.22 01:12:45.572990 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:45.680145 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_401_80, total 5947 rows starting from the beginning of the part 2024.10.22 01:12:45.702823 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_402_402_0, total 19 rows starting from the beginning of the part 2024.10.22 01:12:45.739545 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_381_76 to 202410_1_381_76 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:45.777087 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_381_76} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_376_75, 202410_381_381_0] -> 202410_1_381_76 2024.10.22 01:12:45.799355 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:12:45.865538 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_403_403_0, total 15 rows starting from the beginning of the part 2024.10.22 01:12:45.899147 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_404_404_0, total 25 rows starting from the beginning of the part 2024.10.22 01:12:45.903453 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_388_166} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_388_166 to 202410_1_388_166 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:45.907332 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_388_166} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_387_165, 202410_388_388_0] -> 202410_1_388_166 2024.10.22 01:12:45.925540 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_576_199, 202410_577_577_0] 2024.10.22 01:12:45.931639 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_576_199, 202410_577_577_0] 2024.10.22 01:12:45.932130 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.95 MiB. 2024.10.22 01:12:45.943158 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_576_199, 202410_577_577_0] 2024.10.22 01:12:46.060994 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_405_405_0, total 19 rows starting from the beginning of the part 2024.10.22 01:12:46.203687 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_406_406_0, total 9 rows starting from the beginning of the part 2024.10.22 01:12:46.221069 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 335 2024.10.22 01:12:46.332334 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(fa4256509c2fb5ea6d422f2d6bee8254) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fa4256509c2fb5ea6d422f2d6bee8254, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:12:46.349805 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: fa4256509c2fb5ea6d422f2d6bee8254, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:12:46.354355 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:46.501693 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 606902 2024.10.22 01:12:46.691494 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:46.819345 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:46.881046 [ 1237 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:12:46.985110 [ 1237 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:12:47.064399 [ 1253 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 353 ms. 2024.10.22 01:12:47.095663 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_insert_202410_317_317_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:12:47.096008 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_317_317_0 (state Temporary) 2024.10.22 01:12:47.135150 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(497855059cd78927c00d4733c79665c1) : [0, 109246]: Setting download as failed: File segment: [0, 109246], key: 497855059cd78927c00d4733c79665c1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:47.138738 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(497855059cd78927c00d4733c79665c1): No space left in cache to reserve 109247 bytes, will continue without cache download 2024.10.22 01:12:47.180156 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 497855059cd78927c00d4733c79665c1, offset: 0, size: 0 2024.10.22 01:12:47.182164 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 497855059cd78927c00d4733c79665c1 for removal 2024.10.22 01:12:47.184955 [ 1476 ] {} CacheMetadata: Key 497855059cd78927c00d4733c79665c1 is removed from metadata 2024.10.22 01:12:47.185433 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 34c494b6e6bc17546d545a09dbf08be3, offset: 0, size: 0 2024.10.22 01:12:47.189819 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 34c494b6e6bc17546d545a09dbf08be3 for removal 2024.10.22 01:12:47.192526 [ 1476 ] {} CacheMetadata: Key 34c494b6e6bc17546d545a09dbf08be3 is removed from metadata 2024.10.22 01:12:47.192544 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:47.197157 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:47.286701 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_317_317_0 to 202410_327_327_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:47.379980 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 335 2024.10.22 01:12:47.442256 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_377_377_0 (state Temporary) 2024.10.22 01:12:47.667121 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_377_377_0 to 202410_384_384_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:47.796746 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 606902 2024.10.22 01:12:47.857279 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} MergeTask::MergeProjectionsStage: Merge sorted 6034 rows, containing 12 columns (12 merged, 0 gathered) in 2.539669318 sec., 2375.8998690238145 rows/sec., 375.72 KiB/sec. 2024.10.22 01:12:47.895797 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} MergedBlockOutputStream: filled checksums 202410_1_406_81 (state Temporary) 2024.10.22 01:12:48.118770 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_406_81 to 202410_1_406_81 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:48.123931 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_406_81} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_401_80, 202410_406_406_0] -> 202410_1_406_81 2024.10.22 01:12:48.174863 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.42 MiB. 2024.10.22 01:12:48.272285 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c6fd5da981787b9f18f8ca062877e5c5) : [0, 155]: Setting download as failed: File segment: [0, 155], key: c6fd5da981787b9f18f8ca062877e5c5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:48.274066 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c6fd5da981787b9f18f8ca062877e5c5): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:12:48.392468 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.96 GiB, free memory in arenas 10.55 MiB, will set to 1.67 GiB (RSS), difference: 12.33 MiB 2024.10.22 01:12:48.532929 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 41466 2024.10.22 01:12:48.984986 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:48.987836 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:49.132940 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_669_669_0 (state Temporary) 2024.10.22 01:12:49.176405 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6a7f5433848de90ad16330ca58a9dfec) : [0, 53874]: Setting download as failed: File segment: [0, 53874], key: 6a7f5433848de90ad16330ca58a9dfec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:49.178154 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6a7f5433848de90ad16330ca58a9dfec): No space left in cache to reserve 53875 bytes, will continue without cache download 2024.10.22 01:12:49.189066 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6a7f5433848de90ad16330ca58a9dfec, offset: 0, size: 0 2024.10.22 01:12:49.191271 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6a7f5433848de90ad16330ca58a9dfec for removal 2024.10.22 01:12:49.193275 [ 1476 ] {} CacheMetadata: Key 6a7f5433848de90ad16330ca58a9dfec is removed from metadata 2024.10.22 01:12:49.195021 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c6fd5da981787b9f18f8ca062877e5c5, offset: 0, size: 0 2024.10.22 01:12:49.196296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c6fd5da981787b9f18f8ca062877e5c5 for removal 2024.10.22 01:12:49.216251 [ 1476 ] {} CacheMetadata: Key c6fd5da981787b9f18f8ca062877e5c5 is removed from metadata 2024.10.22 01:12:49.288438 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_669_669_0 to 202410_682_682_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:49.325651 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_1_331_113, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0, 202410_336_336_0] 2024.10.22 01:12:49.331301 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_1_331_113, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0, 202410_336_336_0] 2024.10.22 01:12:49.440141 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41466 2024.10.22 01:12:49.491967 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_1_331_113, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0, 202410_336_336_0] 2024.10.22 01:12:49.573170 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 151 entries to flush up to offset 37868 2024.10.22 01:12:49.828829 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:49.832530 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:49.917729 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_388_388_0 (state Temporary) 2024.10.22 01:12:49.923672 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:12:49.927775 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:12:50.078576 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_388_388_0 to 202410_394_394_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:50.117277 [ 1169 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_388_166 to 202410_389_389_0 2024.10.22 01:12:50.121065 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 37868 2024.10.22 01:12:50.150863 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5010323296e7fa0b53e3d446c71b0141) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 5010323296e7fa0b53e3d446c71b0141, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:50.153231 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5010323296e7fa0b53e3d446c71b0141): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:12:50.228061 [ 1169 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.92 MiB using storage policy from min volume index 0 2024.10.22 01:12:50.332889 [ 1169 ] {} DiskLocal: Reserved 1.92 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:50.347712 [ 1169 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 255 ms. 2024.10.22 01:12:50.383670 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_389_167} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_388_166 to 202410_389_389_0 into Wide with storage Full 2024.10.22 01:12:50.509580 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_389_167} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:50.527423 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_389_167} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_388_166, total 36828 rows starting from the beginning of the part 2024.10.22 01:12:50.734813 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_389_167} MergeTreeSequentialSource: Reading 2 marks from part 202410_389_389_0, total 226 rows starting from the beginning of the part 2024.10.22 01:12:50.984284 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bf329c8eb2719bee40b03bdf86de4377) : [0, 52365]: Setting download as failed: File segment: [0, 52365], key: bf329c8eb2719bee40b03bdf86de4377, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:50.985427 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bf329c8eb2719bee40b03bdf86de4377): No space left in cache to reserve 52366 bytes, will continue without cache download 2024.10.22 01:12:50.997218 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bf329c8eb2719bee40b03bdf86de4377, offset: 0, size: 0 2024.10.22 01:12:51.006719 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bf329c8eb2719bee40b03bdf86de4377 for removal 2024.10.22 01:12:51.010691 [ 1476 ] {} CacheMetadata: Key bf329c8eb2719bee40b03bdf86de4377 is removed from metadata 2024.10.22 01:12:51.010613 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5010323296e7fa0b53e3d446c71b0141, offset: 0, size: 0 2024.10.22 01:12:51.014402 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5010323296e7fa0b53e3d446c71b0141 for removal 2024.10.22 01:12:51.017890 [ 1476 ] {} CacheMetadata: Key 5010323296e7fa0b53e3d446c71b0141 is removed from metadata 2024.10.22 01:12:51.201286 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_577_200, 202410_578_578_0] 2024.10.22 01:12:51.203678 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_577_200, 202410_578_578_0] 2024.10.22 01:12:51.211929 [ 1241 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 507 ms. 2024.10.22 01:12:51.212176 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_577_200, 202410_578_578_0] 2024.10.22 01:12:51.603391 [ 1175 ] {} DatabaseCatalog: Execution took 308 ms. 2024.10.22 01:12:51.891395 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(536009c654f0debb997f9b503e3136d6) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 536009c654f0debb997f9b503e3136d6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:51.893128 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(536009c654f0debb997f9b503e3136d6): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:12:52.706364 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.96 GiB, free memory in arenas 8.73 MiB, will set to 1.68 GiB (RSS), difference: -54.01 MiB 2024.10.22 01:12:52.835033 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 3191 2024.10.22 01:12:52.835766 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 5950 2024.10.22 01:12:52.972823 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:52.978896 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:53.018681 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_394_394_0 (state Temporary) 2024.10.22 01:12:53.050696 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(95331be6ef817b8dceed30a07d2a1046) : [0, 27872]: Setting download as failed: File segment: [0, 27872], key: 95331be6ef817b8dceed30a07d2a1046, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:53.053290 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(95331be6ef817b8dceed30a07d2a1046): No space left in cache to reserve 27873 bytes, will continue without cache download 2024.10.22 01:12:53.077707 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 95331be6ef817b8dceed30a07d2a1046, offset: 0, size: 0 2024.10.22 01:12:53.079537 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 95331be6ef817b8dceed30a07d2a1046 for removal 2024.10.22 01:12:53.081490 [ 1476 ] {} CacheMetadata: Key 95331be6ef817b8dceed30a07d2a1046 is removed from metadata 2024.10.22 01:12:53.082833 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 536009c654f0debb997f9b503e3136d6, offset: 0, size: 0 2024.10.22 01:12:53.084184 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 536009c654f0debb997f9b503e3136d6 for removal 2024.10.22 01:12:53.085728 [ 1476 ] {} CacheMetadata: Key 536009c654f0debb997f9b503e3136d6 is removed from metadata 2024.10.22 01:12:53.197064 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_389_167/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:12:53.402423 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_394_394_0 to 202410_407_407_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:53.458003 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 41532 2024.10.22 01:12:53.547663 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5950 2024.10.22 01:12:53.917452 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:53.982900 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d5c7cea28d8e48201ebf2cc33b9a9f50) : [0, 167]: Setting download as failed: File segment: [0, 167], key: d5c7cea28d8e48201ebf2cc33b9a9f50, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:54.024836 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d5c7cea28d8e48201ebf2cc33b9a9f50): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:12:54.024928 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:54.096071 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_389_167} MergeTask::MergeProjectionsStage: Merge sorted 37054 rows, containing 13 columns (13 merged, 0 gathered) in 3.609802176 sec., 10264.828429201989 rows/sec., 4.42 MiB/sec. 2024.10.22 01:12:54.200943 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_389_167} MergedBlockOutputStream: filled checksums 202410_1_389_167 (state Temporary) 2024.10.22 01:12:54.210437 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_670_670_0 (state Temporary) 2024.10.22 01:12:54.368057 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.96 GiB, free memory in arenas 8.75 MiB, will set to 1.68 GiB (RSS), difference: 9.37 MiB 2024.10.22 01:12:54.375115 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_670_670_0 to 202410_683_683_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:54.503366 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_672_295 to 202410_673_678_1 2024.10.22 01:12:54.512263 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.50 MiB using storage policy from min volume index 0 2024.10.22 01:12:54.521639 [ 1196 ] {} DiskLocal: Reserved 1.50 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:54.547640 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_678_296} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_672_295 to 202410_673_678_1 into Compact with storage Full 2024.10.22 01:12:54.584405 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_678_296} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:54.596774 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_678_296} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_672_295, total 41535 rows starting from the beginning of the part 2024.10.22 01:12:54.625270 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_678_296} MergeTreeSequentialSource: Reading 2 marks from part 202410_673_678_1, total 449 rows starting from the beginning of the part 2024.10.22 01:12:54.720935 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41532 2024.10.22 01:12:54.801107 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 608438 2024.10.22 01:12:54.881775 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 336 2024.10.22 01:12:55.148609 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:55.154756 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:55.172340 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(67b20a99c0247980d0c4c2ccf5583e53) : [0, 105755]: Setting download as failed: File segment: [0, 105755], key: 67b20a99c0247980d0c4c2ccf5583e53, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:55.176403 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(67b20a99c0247980d0c4c2ccf5583e53): No space left in cache to reserve 105756 bytes, will continue without cache download 2024.10.22 01:12:55.187298 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_389_167} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_389_167 to 202410_1_389_167 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:55.191159 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_389_167} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_388_166, 202410_389_389_0] -> 202410_1_389_167 2024.10.22 01:12:55.209676 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 67b20a99c0247980d0c4c2ccf5583e53, offset: 0, size: 0 2024.10.22 01:12:55.233683 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 67b20a99c0247980d0c4c2ccf5583e53 for removal 2024.10.22 01:12:55.241065 [ 1476 ] {} CacheMetadata: Key 67b20a99c0247980d0c4c2ccf5583e53 is removed from metadata 2024.10.22 01:12:55.243116 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d5c7cea28d8e48201ebf2cc33b9a9f50, offset: 0, size: 0 2024.10.22 01:12:55.247525 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d5c7cea28d8e48201ebf2cc33b9a9f50 for removal 2024.10.22 01:12:55.250804 [ 1476 ] {} CacheMetadata: Key d5c7cea28d8e48201ebf2cc33b9a9f50 is removed from metadata 2024.10.22 01:12:55.316943 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_318_318_0 (state Temporary) 2024.10.22 01:12:55.318613 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.95 MiB. 2024.10.22 01:12:55.414323 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_318_318_0 to 202410_328_328_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:55.500415 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 336 2024.10.22 01:12:55.733337 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_578_201, 202410_579_579_0] 2024.10.22 01:12:55.740759 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_578_201, 202410_579_579_0] 2024.10.22 01:12:55.766050 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_578_201, 202410_579_579_0] 2024.10.22 01:12:55.803980 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:55.807354 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.31 GiB. 2024.10.22 01:12:56.156657 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_378_378_0 (state Temporary) 2024.10.22 01:12:56.431687 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_378_378_0 to 202410_385_385_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:56.483925 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 608438 2024.10.22 01:12:56.511188 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c6706aa58be1240955428c18072e9ead) : [0, 155]: Setting download as failed: File segment: [0, 155], key: c6706aa58be1240955428c18072e9ead, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:56.515262 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c6706aa58be1240955428c18072e9ead): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:12:56.789741 [ 1167 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 01:12:56.901084 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:12:57.069483 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_678_296} MergeTask::MergeProjectionsStage: Merge sorted 41984 rows, containing 13 columns (13 merged, 0 gathered) in 2.532402652 sec., 16578.722173917547 rows/sec., 2.62 MiB/sec. 2024.10.22 01:12:57.307618 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(ed8b858d432e413823d41c5c4167f4b9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ed8b858d432e413823d41c5c4167f4b9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:12:57.342873 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 14679, file segment info: File segment: [0, 33554431], key: ed8b858d432e413823d41c5c4167f4b9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:12:57.349367 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:57.249660 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_678_296} MergedBlockOutputStream: filled checksums 202410_1_678_296 (state Temporary) 2024.10.22 01:12:57.623127 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_678_296} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_678_296 to 202410_1_678_296 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:57.629688 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.96 GiB, free memory in arenas 9.45 MiB, will set to 1.68 GiB (RSS), difference: 44.29 MiB 2024.10.22 01:12:57.727954 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_678_296} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_672_295, 202410_673_678_1] -> 202410_1_678_296 2024.10.22 01:12:57.753704 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.62 MiB. 2024.10.22 01:12:57.771300 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 161 entries to flush up to offset 38029 2024.10.22 01:12:58.095368 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:58.096894 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:58.137284 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_389_389_0 (state Temporary) 2024.10.22 01:12:58.200848 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_389_389_0 to 202410_395_395_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:58.214849 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(140e6be7f8fa1c801ff1ebe052cb1f3b) : [0, 2431]: Setting download as failed: File segment: [0, 2431], key: 140e6be7f8fa1c801ff1ebe052cb1f3b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:58.220025 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(140e6be7f8fa1c801ff1ebe052cb1f3b): No space left in cache to reserve 2432 bytes, will continue without cache download 2024.10.22 01:12:58.249707 [ 1274 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_389_167 to 202410_390_390_0 2024.10.22 01:12:58.262907 [ 1274 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.93 MiB using storage policy from min volume index 0 2024.10.22 01:12:58.270350 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 38029 2024.10.22 01:12:58.271488 [ 1274 ] {} DiskLocal: Reserved 1.93 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:58.291710 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_390_168} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_389_167 to 202410_390_390_0 into Wide with storage Full 2024.10.22 01:12:58.349489 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_390_168} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:58.376580 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_390_168} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_389_167, total 37054 rows starting from the beginning of the part 2024.10.22 01:12:58.358877 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 140e6be7f8fa1c801ff1ebe052cb1f3b, offset: 0, size: 0 2024.10.22 01:12:58.473749 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 140e6be7f8fa1c801ff1ebe052cb1f3b for removal 2024.10.22 01:12:58.475220 [ 1476 ] {} CacheMetadata: Key 140e6be7f8fa1c801ff1ebe052cb1f3b is removed from metadata 2024.10.22 01:12:58.476568 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c6706aa58be1240955428c18072e9ead, offset: 0, size: 0 2024.10.22 01:12:58.556156 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_390_168} MergeTreeSequentialSource: Reading 2 marks from part 202410_390_390_0, total 212 rows starting from the beginning of the part 2024.10.22 01:12:58.579027 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c6706aa58be1240955428c18072e9ead for removal 2024.10.22 01:12:58.581719 [ 1476 ] {} CacheMetadata: Key c6706aa58be1240955428c18072e9ead is removed from metadata 2024.10.22 01:12:58.732227 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 41611 2024.10.22 01:12:58.903872 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(990175d2e6cd6c67cd641c2094595e24) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 990175d2e6cd6c67cd641c2094595e24, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:12:58.913819 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 342, file segment info: File segment: [0, 33554431], key: 990175d2e6cd6c67cd641c2094595e24, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:12:58.918036 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:12:59.002717 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:12:59.007463 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:59.082382 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_671_671_0 (state Temporary) 2024.10.22 01:12:59.328078 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_671_671_0 to 202410_684_684_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:12:59.374821 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_678_296 to 202410_679_679_0 2024.10.22 01:12:59.387492 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41611 2024.10.22 01:12:59.391057 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.50 MiB using storage policy from min volume index 0 2024.10.22 01:12:59.394454 [ 1248 ] {} DiskLocal: Reserved 1.50 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:12:59.517782 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_679_297} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_678_296 to 202410_679_679_0 into Compact with storage Full 2024.10.22 01:12:59.659770 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_679_297} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:12:59.669257 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_679_297} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_678_296, total 41984 rows starting from the beginning of the part 2024.10.22 01:12:59.693172 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_679_297} MergeTreeSequentialSource: Reading 2 marks from part 202410_679_679_0, total 87 rows starting from the beginning of the part 2024.10.22 01:12:59.724010 [ 1275 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 326 ms. 2024.10.22 01:12:59.790447 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9c3dd43cb7b818604e640117b3a8ac92) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 9c3dd43cb7b818604e640117b3a8ac92, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:12:59.791952 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9c3dd43cb7b818604e640117b3a8ac92): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:12:59.924738 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:12:59.926736 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:13:00.372294 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(3096ca39c047a0fde878540ea20b7c55) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3096ca39c047a0fde878540ea20b7c55, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:00.426566 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.96 GiB, free memory in arenas 8.53 MiB, will set to 1.69 GiB (RSS), difference: -43.20 MiB 2024.10.22 01:13:00.476429 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4164, file segment info: File segment: [0, 33554431], key: 3096ca39c047a0fde878540ea20b7c55, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:00.487307 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:01.050883 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 5962 2024.10.22 01:13:01.267087 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:01.275008 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:01.303477 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_395_395_0 (state Temporary) 2024.10.22 01:13:01.348618 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_390_168} MergeTask::MergeProjectionsStage: Merge sorted 37266 rows, containing 13 columns (13 merged, 0 gathered) in 3.060269508 sec., 12177.358857636926 rows/sec., 5.25 MiB/sec. 2024.10.22 01:13:01.363715 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_395_395_0 to 202410_408_408_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:01.403232 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(464603aa377d7de57ad88548b4941a29) : [0, 2606]: Setting download as failed: File segment: [0, 2606], key: 464603aa377d7de57ad88548b4941a29, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:01.405278 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(464603aa377d7de57ad88548b4941a29): No space left in cache to reserve 2607 bytes, will continue without cache download 2024.10.22 01:13:01.412928 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5962 2024.10.22 01:13:01.431039 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 464603aa377d7de57ad88548b4941a29, offset: 0, size: 0 2024.10.22 01:13:01.448626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 464603aa377d7de57ad88548b4941a29 for removal 2024.10.22 01:13:01.453257 [ 1476 ] {} CacheMetadata: Key 464603aa377d7de57ad88548b4941a29 is removed from metadata 2024.10.22 01:13:01.454254 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9c3dd43cb7b818604e640117b3a8ac92, offset: 0, size: 0 2024.10.22 01:13:01.457551 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9c3dd43cb7b818604e640117b3a8ac92 for removal 2024.10.22 01:13:01.463306 [ 1476 ] {} CacheMetadata: Key 9c3dd43cb7b818604e640117b3a8ac92 is removed from metadata 2024.10.22 01:13:01.513656 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_390_168} MergedBlockOutputStream: filled checksums 202410_1_390_168 (state Temporary) 2024.10.22 01:13:01.944356 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(1c1befa5ba2069bd78040446358ebfc3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1c1befa5ba2069bd78040446358ebfc3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:01.951837 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 495, file segment info: File segment: [0, 33554431], key: 1c1befa5ba2069bd78040446358ebfc3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:01.956878 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:01.990936 [ 1230 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:13:02.008907 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_679_297} MergeTask::MergeProjectionsStage: Merge sorted 42071 rows, containing 13 columns (13 merged, 0 gathered) in 2.49973613 sec., 16830.176391457764 rows/sec., 2.67 MiB/sec. 2024.10.22 01:13:02.032350 [ 1230 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:13:02.086625 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_579_202, 202410_580_580_0] 2024.10.22 01:13:02.088486 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_579_202, 202410_580_580_0] 2024.10.22 01:13:02.180585 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_679_297} MergedBlockOutputStream: filled checksums 202410_1_679_297 (state Temporary) 2024.10.22 01:13:02.190662 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FromTag 2024.10.22 01:13:02.201240 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_579_202, 202410_580_580_0] 2024.10.22 01:13:02.319335 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_679_297} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_679_297 to 202410_1_679_297 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:02.323855 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_679_297} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_678_296, 202410_679_679_0] -> 202410_1_679_297 2024.10.22 01:13:02.348250 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.55 MiB. 2024.10.22 01:13:02.392534 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FromTag 2024.10.22 01:13:02.394746 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_390_168} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_390_168 to 202410_1_390_168 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:02.396419 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_390_168} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_389_167, 202410_390_390_0] -> 202410_1_390_168 2024.10.22 01:13:02.405578 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FromTag 2024.10.22 01:13:02.512026 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.95 MiB. 2024.10.22 01:13:02.547119 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(10e632d028135f827fdd8885bdb328da) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 10e632d028135f827fdd8885bdb328da, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:02.550776 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(10e632d028135f827fdd8885bdb328da): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:13:02.548872 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FromTag 2024.10.22 01:13:02.586536 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FromTag 2024.10.22 01:13:02.610135 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FromTag 2024.10.22 01:13:02.654706 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FromTag 2024.10.22 01:13:02.920999 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kvd/yhbsmwmddoplbqkjuyzcgzqpdxlte (LogSeriesLimiter: on interval from 2024-10-22 01:12:35 to 2024-10-22 01:13:02 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:13:03.003912 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 337 2024.10.22 01:13:03.347146 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_321_64, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0, 202410_325_325_0, 202410_326_326_0] 2024.10.22 01:13:03.349352 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_321_64, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0, 202410_325_325_0, 202410_326_326_0] 2024.10.22 01:13:03.351427 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(1a418f828a5ad2535ff48c0b8c3c76ba) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1a418f828a5ad2535ff48c0b8c3c76ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:03.355962 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1263, file segment info: File segment: [0, 33554431], key: 1a418f828a5ad2535ff48c0b8c3c76ba, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:03.366195 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:03.398084 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 41684 2024.10.22 01:13:03.480090 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_321_64, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0, 202410_325_325_0, 202410_326_326_0] 2024.10.22 01:13:03.487788 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 609590 2024.10.22 01:13:03.487999 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:03.490114 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:03.508841 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.96 GiB, free memory in arenas 8.61 MiB, will set to 1.67 GiB (RSS), difference: 44.61 MiB 2024.10.22 01:13:03.616339 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_580_203, 202410_581_581_0] 2024.10.22 01:13:03.623061 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_580_203, 202410_581_581_0] 2024.10.22 01:13:03.637393 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_580_203, 202410_581_581_0] 2024.10.22 01:13:03.666802 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_319_319_0 (state Temporary) 2024.10.22 01:13:03.840157 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_319_319_0 to 202410_329_329_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:04.031575 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:04.036405 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:04.042747 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 337 2024.10.22 01:13:04.066199 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4eb9e6e345b87a5c76128ad15c85a4f9) : [0, 34090]: Setting download as failed: File segment: [0, 34090], key: 4eb9e6e345b87a5c76128ad15c85a4f9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:04.068271 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4eb9e6e345b87a5c76128ad15c85a4f9): No space left in cache to reserve 34091 bytes, will continue without cache download 2024.10.22 01:13:04.083585 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4eb9e6e345b87a5c76128ad15c85a4f9, offset: 0, size: 0 2024.10.22 01:13:04.084795 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4eb9e6e345b87a5c76128ad15c85a4f9 for removal 2024.10.22 01:13:04.087227 [ 1476 ] {} CacheMetadata: Key 4eb9e6e345b87a5c76128ad15c85a4f9 is removed from metadata 2024.10.22 01:13:04.087822 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 10e632d028135f827fdd8885bdb328da, offset: 0, size: 0 2024.10.22 01:13:04.089482 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 10e632d028135f827fdd8885bdb328da for removal 2024.10.22 01:13:04.092775 [ 1476 ] {} CacheMetadata: Key 10e632d028135f827fdd8885bdb328da is removed from metadata 2024.10.22 01:13:04.148268 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_672_672_0 (state Temporary) 2024.10.22 01:13:04.279722 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:04.284338 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:04.309370 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_672_672_0 to 202410_685_685_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:04.350931 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_679_297 to 202410_680_680_0 2024.10.22 01:13:04.360065 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.50 MiB using storage policy from min volume index 0 2024.10.22 01:13:04.469163 [ 1240 ] {} DiskLocal: Reserved 1.50 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:04.472649 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41684 2024.10.22 01:13:04.487485 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_680_298} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_679_297 to 202410_680_680_0 into Compact with storage Full 2024.10.22 01:13:04.488461 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_379_379_0 (state Temporary) 2024.10.22 01:13:04.521539 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_680_298} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:04.535249 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_680_298} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_679_297, total 42071 rows starting from the beginning of the part 2024.10.22 01:13:04.571445 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_680_298} MergeTreeSequentialSource: Reading 2 marks from part 202410_680_680_0, total 67 rows starting from the beginning of the part 2024.10.22 01:13:04.598707 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(0d67991a970ae80aee8a378f99c4de29) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0d67991a970ae80aee8a378f99c4de29, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:04.601562 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 153762, file segment info: File segment: [0, 33554431], key: 0d67991a970ae80aee8a378f99c4de29, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:04.615749 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:04.655950 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_379_379_0 to 202410_386_386_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:04.781836 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 609590 2024.10.22 01:13:04.804602 [ 1256 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_381_76 to 202410_386_386_0 2024.10.22 01:13:04.820786 [ 1256 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:04.824662 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:04.852234 [ 1256 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 01:13:04.887306 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_381_76 to 202410_386_386_0 into Compact with storage Full 2024.10.22 01:13:05.008787 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:05.027680 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} MergeTreeSequentialSource: Reading 77 marks from part 202410_1_381_76, total 617036 rows starting from the beginning of the part 2024.10.22 01:13:05.044240 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_382_382_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:13:05.066314 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_383_383_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:13:05.083233 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_384_384_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:13:05.096967 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_385_385_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:13:05.119543 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_386_386_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:13:05.359189 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8ef86994918d6848ba7cf92643083d84) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 8ef86994918d6848ba7cf92643083d84, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:05.360955 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8ef86994918d6848ba7cf92643083d84): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:13:05.800660 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 221 entries to flush up to offset 38250 2024.10.22 01:13:05.980664 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:05.983489 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:06.176865 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_390_390_0 (state Temporary) 2024.10.22 01:13:06.188372 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b56d2036adb80b3e1189486b023949b6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b56d2036adb80b3e1189486b023949b6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:06.191210 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 914, file segment info: File segment: [0, 33554431], key: b56d2036adb80b3e1189486b023949b6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:06.193181 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:06.266781 [ 1689 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rhq/xfhyoxvsgpdlfbyahngeydsgumcwx, size 153762 2024.10.22 01:13:06.326356 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:13:06.343041 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:13:06.355434 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_390_390_0 to 202410_396_396_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:06.453023 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:13:06.498467 [ 1227 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_390_168 to 202410_391_391_0 2024.10.22 01:13:06.507743 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 38250 2024.10.22 01:13:06.518150 [ 1227 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.94 MiB using storage policy from min volume index 0 2024.10.22 01:13:06.524968 [ 1227 ] {} DiskLocal: Reserved 1.94 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:06.570303 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:13:06.578287 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_391_169} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_390_168 to 202410_391_391_0 into Wide with storage Full 2024.10.22 01:13:06.635189 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:13:06.639209 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_680_298} MergeTask::MergeProjectionsStage: Merge sorted 42138 rows, containing 13 columns (13 merged, 0 gathered) in 2.157665602 sec., 19529.439576244404 rows/sec., 3.10 MiB/sec. 2024.10.22 01:13:06.683423 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_391_169} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:06.699913 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_391_169} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_390_168, total 37266 rows starting from the beginning of the part 2024.10.22 01:13:06.699268 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:13:06.714034 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_680_298} MergedBlockOutputStream: filled checksums 202410_1_680_298 (state Temporary) 2024.10.22 01:13:06.721880 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:13:06.760719 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:13:06.824978 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(541ed42071310a055a8cfb17c27c5ef3) : [0, 32211]: Setting download as failed: File segment: [0, 32211], key: 541ed42071310a055a8cfb17c27c5ef3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:06.828223 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(541ed42071310a055a8cfb17c27c5ef3): No space left in cache to reserve 32212 bytes, will continue without cache download 2024.10.22 01:13:06.851480 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 541ed42071310a055a8cfb17c27c5ef3, offset: 0, size: 0 2024.10.22 01:13:06.853264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 541ed42071310a055a8cfb17c27c5ef3 for removal 2024.10.22 01:13:06.855515 [ 1476 ] {} CacheMetadata: Key 541ed42071310a055a8cfb17c27c5ef3 is removed from metadata 2024.10.22 01:13:06.857124 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8ef86994918d6848ba7cf92643083d84, offset: 0, size: 0 2024.10.22 01:13:06.859806 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8ef86994918d6848ba7cf92643083d84 for removal 2024.10.22 01:13:06.862698 [ 1476 ] {} CacheMetadata: Key 8ef86994918d6848ba7cf92643083d84 is removed from metadata 2024.10.22 01:13:06.901387 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:13:06.904330 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_391_169} MergeTreeSequentialSource: Reading 2 marks from part 202410_391_391_0, total 209 rows starting from the beginning of the part 2024.10.22 01:13:06.925506 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_680_298} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_680_298 to 202410_1_680_298 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:06.932198 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_680_298} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_679_297, 202410_680_680_0] -> 202410_1_680_298 2024.10.22 01:13:06.972187 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:13:07.080373 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/oqw/jjityjqworspfzbpfmtzddrezymnw (LogSeriesLimiter: on interval from 2024-10-22 01:13:02 to 2024-10-22 01:13:07 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:13:07.871021 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(95ac8162bf5ef20300899e93742ee107) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 95ac8162bf5ef20300899e93742ee107, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:07.873817 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(95ac8162bf5ef20300899e93742ee107): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:13:08.482676 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 41780 2024.10.22 01:13:08.611043 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.96 GiB, free memory in arenas 11.92 MiB, will set to 1.68 GiB (RSS), difference: -59.60 MiB 2024.10.22 01:13:08.915362 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 5976 2024.10.22 01:13:08.936542 [ 1254 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:13:09.038879 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:09.042633 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:09.069498 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fc16cdf5e319048af9334c65ad3089e4) : [0, 177781]: Setting download as failed: File segment: [0, 177781], key: fc16cdf5e319048af9334c65ad3089e4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:09.070812 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fc16cdf5e319048af9334c65ad3089e4): No space left in cache to reserve 177782 bytes, will continue without cache download 2024.10.22 01:13:09.137235 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:09.140537 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:09.190208 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_396_396_0 (state Temporary) 2024.10.22 01:13:09.197116 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fc16cdf5e319048af9334c65ad3089e4, offset: 0, size: 0 2024.10.22 01:13:09.202378 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fc16cdf5e319048af9334c65ad3089e4 for removal 2024.10.22 01:13:09.205399 [ 1476 ] {} CacheMetadata: Key fc16cdf5e319048af9334c65ad3089e4 is removed from metadata 2024.10.22 01:13:09.206696 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 95ac8162bf5ef20300899e93742ee107, offset: 0, size: 0 2024.10.22 01:13:09.210384 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 95ac8162bf5ef20300899e93742ee107 for removal 2024.10.22 01:13:09.216139 [ 1476 ] {} CacheMetadata: Key 95ac8162bf5ef20300899e93742ee107 is removed from metadata 2024.10.22 01:13:09.240000 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_673_673_0 (state Temporary) 2024.10.22 01:13:09.361547 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_391_169} MergeTask::MergeProjectionsStage: Merge sorted 37475 rows, containing 13 columns (13 merged, 0 gathered) in 2.821523043 sec., 13281.833757471106 rows/sec., 5.73 MiB/sec. 2024.10.22 01:13:09.400431 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_396_396_0 to 202410_409_409_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:09.404513 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_673_673_0 to 202410_686_686_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:09.435145 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_391_169} MergedBlockOutputStream: filled checksums 202410_1_391_169 (state Temporary) 2024.10.22 01:13:09.455025 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_680_298 to 202410_681_681_0 2024.10.22 01:13:09.482774 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41780 2024.10.22 01:13:09.492596 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.51 MiB using storage policy from min volume index 0 2024.10.22 01:13:09.495997 [ 1157 ] {} DiskLocal: Reserved 1.51 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:09.528271 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_681_299} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_680_298 to 202410_681_681_0 into Compact with storage Full 2024.10.22 01:13:09.568096 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_681_299} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:09.569703 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5976 2024.10.22 01:13:09.575179 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_681_299} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_680_298, total 42138 rows starting from the beginning of the part 2024.10.22 01:13:09.590438 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_681_299} MergeTreeSequentialSource: Reading 2 marks from part 202410_681_681_0, total 76 rows starting from the beginning of the part 2024.10.22 01:13:09.702481 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_581_204, 202410_582_582_0] 2024.10.22 01:13:09.704582 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_581_204, 202410_582_582_0] 2024.10.22 01:13:09.733776 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_581_204, 202410_582_582_0] 2024.10.22 01:13:09.927685 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:13:09.995001 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:13:10.382368 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1c22d39bd26465bb04221900aee2de31) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 1c22d39bd26465bb04221900aee2de31, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:10.383660 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1c22d39bd26465bb04221900aee2de31): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:13:10.523517 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:10.530893 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:10.647433 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.96 GiB, free memory in arenas 16.62 MiB, will set to 1.68 GiB (RSS), difference: 47.62 MiB 2024.10.22 01:13:10.798324 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_386_77/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:13:10.972966 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_391_169} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_391_169 to 202410_1_391_169 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:10.977122 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_391_169} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_390_168, 202410_391_391_0] -> 202410_1_391_169 2024.10.22 01:13:10.999102 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.95 MiB. 2024.10.22 01:13:11.430697 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(dfe211b356504f2008fca2fee9e7b504) : [0, 143122]: Setting download as failed: File segment: [0, 143122], key: dfe211b356504f2008fca2fee9e7b504, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:11.435729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(dfe211b356504f2008fca2fee9e7b504): No space left in cache to reserve 143123 bytes, will continue without cache download 2024.10.22 01:13:11.545084 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 338 2024.10.22 01:13:11.473831 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dfe211b356504f2008fca2fee9e7b504, offset: 0, size: 0 2024.10.22 01:13:11.576086 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dfe211b356504f2008fca2fee9e7b504 for removal 2024.10.22 01:13:11.578796 [ 1476 ] {} CacheMetadata: Key dfe211b356504f2008fca2fee9e7b504 is removed from metadata 2024.10.22 01:13:11.579691 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1c22d39bd26465bb04221900aee2de31, offset: 0, size: 0 2024.10.22 01:13:11.581349 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1c22d39bd26465bb04221900aee2de31 for removal 2024.10.22 01:13:11.584137 [ 1476 ] {} CacheMetadata: Key 1c22d39bd26465bb04221900aee2de31 is removed from metadata 2024.10.22 01:13:11.784066 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1428 entries to flush up to offset 611018 2024.10.22 01:13:11.978617 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:11.982492 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:12.068880 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_320_320_0 (state Temporary) 2024.10.22 01:13:12.117638 [ 1225 ] {} DatabaseCatalog: Execution took 379 ms. 2024.10.22 01:13:12.151817 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_320_320_0 to 202410_330_330_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:12.186346 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_325_67 to 202410_330_330_0 2024.10.22 01:13:12.214558 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:12.222624 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:12.240318 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 338 2024.10.22 01:13:12.295837 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_325_67 to 202410_330_330_0 into Compact with storage Full 2024.10.22 01:13:12.405150 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:12.428637 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_325_67, total 369 rows starting from the beginning of the part 2024.10.22 01:13:12.545378 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:12.549294 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:12.589020 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_326_326_0, total 1 rows starting from the beginning of the part 2024.10.22 01:13:12.657067 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.96 GiB, free memory in arenas 13.88 MiB, will set to 1.68 GiB (RSS), difference: 2.87 MiB 2024.10.22 01:13:12.661292 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_327_327_0, total 1 rows starting from the beginning of the part 2024.10.22 01:13:12.721457 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_380_380_0 (state Temporary) 2024.10.22 01:13:12.728224 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(cac6bf2d77e0a6012f7643bb99bede5f) : [0, 156]: Setting download as failed: File segment: [0, 156], key: cac6bf2d77e0a6012f7643bb99bede5f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:12.730598 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(cac6bf2d77e0a6012f7643bb99bede5f): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:13:12.749541 [ 1175 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 222 ms. 2024.10.22 01:13:12.778797 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_681_299} MergeTask::MergeProjectionsStage: Merge sorted 42214 rows, containing 13 columns (13 merged, 0 gathered) in 3.25702301 sec., 12960.915495650735 rows/sec., 2.06 MiB/sec. 2024.10.22 01:13:12.799331 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_328_328_0, total 1 rows starting from the beginning of the part 2024.10.22 01:13:12.800230 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_380_380_0 to 202410_387_387_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:12.881878 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_329_329_0, total 1 rows starting from the beginning of the part 2024.10.22 01:13:12.913113 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_681_299} MergedBlockOutputStream: filled checksums 202410_1_681_299 (state Temporary) 2024.10.22 01:13:12.974127 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 611018 2024.10.22 01:13:12.997420 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_330_330_0, total 1 rows starting from the beginning of the part 2024.10.22 01:13:13.408725 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_681_299} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_681_299 to 202410_1_681_299 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:13.499877 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 41864 2024.10.22 01:13:13.517234 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_681_299} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_680_298, 202410_681_681_0] -> 202410_1_681_299 2024.10.22 01:13:13.569548 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:13:13.741654 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_80_80_0 (state Temporary) 2024.10.22 01:13:13.799837 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0fa9ea41b8717a4abfca487849010d86) : [0, 111150]: Setting download as failed: File segment: [0, 111150], key: 0fa9ea41b8717a4abfca487849010d86, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:13.801516 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0fa9ea41b8717a4abfca487849010d86): No space left in cache to reserve 111151 bytes, will continue without cache download 2024.10.22 01:13:13.821472 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0fa9ea41b8717a4abfca487849010d86, offset: 0, size: 0 2024.10.22 01:13:13.827369 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0fa9ea41b8717a4abfca487849010d86 for removal 2024.10.22 01:13:13.832442 [ 1476 ] {} CacheMetadata: Key 0fa9ea41b8717a4abfca487849010d86 is removed from metadata 2024.10.22 01:13:13.832610 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cac6bf2d77e0a6012f7643bb99bede5f, offset: 0, size: 0 2024.10.22 01:13:13.835755 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cac6bf2d77e0a6012f7643bb99bede5f for removal 2024.10.22 01:13:13.843355 [ 1476 ] {} CacheMetadata: Key cac6bf2d77e0a6012f7643bb99bede5f is removed from metadata 2024.10.22 01:13:13.893604 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:13.896709 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:13.985264 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_674_674_0 (state Temporary) 2024.10.22 01:13:14.160813 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 178 entries to flush up to offset 38428 2024.10.22 01:13:14.219137 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_582_205, 202410_583_583_0] 2024.10.22 01:13:14.224650 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_674_674_0 to 202410_687_687_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:14.227270 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_582_205, 202410_583_583_0] 2024.10.22 01:13:14.243800 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_582_205, 202410_583_583_0] 2024.10.22 01:13:14.279259 [ 1193 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_681_299 to 202410_682_682_0 2024.10.22 01:13:14.298304 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41864 2024.10.22 01:13:14.307304 [ 1193 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.51 MiB using storage policy from min volume index 0 2024.10.22 01:13:14.353368 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:14.313780 [ 1193 ] {} DiskLocal: Reserved 1.51 MiB on local disk `default`, having unreserved 297.30 GiB. 2024.10.22 01:13:14.420555 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:14.578420 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_682_300} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_681_299 to 202410_682_682_0 into Compact with storage Full 2024.10.22 01:13:14.615911 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_391_391_0 (state Temporary) 2024.10.22 01:13:14.683869 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_682_300} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:14.695525 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_682_300} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_681_299, total 42214 rows starting from the beginning of the part 2024.10.22 01:13:14.777952 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_682_300} MergeTreeSequentialSource: Reading 2 marks from part 202410_682_682_0, total 83 rows starting from the beginning of the part 2024.10.22 01:13:14.782426 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_391_391_0 to 202410_397_397_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:14.831627 [ 1197 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_391_169 to 202410_392_392_0 2024.10.22 01:13:14.846873 [ 1197 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.95 MiB using storage policy from min volume index 0 2024.10.22 01:13:14.848255 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_330_68/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:13:14.852131 [ 1197 ] {} DiskLocal: Reserved 1.95 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:14.863517 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 38428 2024.10.22 01:13:14.875918 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_392_170} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_391_169 to 202410_392_392_0 into Wide with storage Full 2024.10.22 01:13:14.979391 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_80_80_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:13:15.027278 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} MergeTask::MergeProjectionsStage: Merge sorted 374 rows, containing 32 columns (32 merged, 0 gathered) in 2.747554835 sec., 136.12103213947321 rows/sec., 13.79 KiB/sec. 2024.10.22 01:13:15.056221 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(58bfd7c157b2d7bfecde1d5b1dc690cf) : [0, 167]: Setting download as failed: File segment: [0, 167], key: 58bfd7c157b2d7bfecde1d5b1dc690cf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:15.061279 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(58bfd7c157b2d7bfecde1d5b1dc690cf): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:13:15.090191 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} MergedBlockOutputStream: filled checksums 202410_1_330_68 (state Temporary) 2024.10.22 01:13:15.097399 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_392_170} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:15.122559 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_392_170} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_391_169, total 37475 rows starting from the beginning of the part 2024.10.22 01:13:15.468027 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_392_170} MergeTreeSequentialSource: Reading 2 marks from part 202410_392_392_0, total 189 rows starting from the beginning of the part 2024.10.22 01:13:15.604915 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_330_68 to 202410_1_330_68 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:15.608834 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_330_68} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_325_67, 202410_330_330_0] -> 202410_1_330_68 2024.10.22 01:13:15.678308 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:13:16.144585 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2110dbfd5b4f6b1e507b462ae8b51e8e) : [0, 128819]: Setting download as failed: File segment: [0, 128819], key: 2110dbfd5b4f6b1e507b462ae8b51e8e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:16.184755 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_277_57, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0, 202410_281_281_0, 202410_282_282_0] 2024.10.22 01:13:16.188064 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_277_57, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0, 202410_281_281_0, 202410_282_282_0] 2024.10.22 01:13:16.212956 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_277_57, 202410_278_278_0, 202410_279_279_0, 202410_280_280_0, 202410_281_281_0, 202410_282_282_0] 2024.10.22 01:13:16.247073 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2110dbfd5b4f6b1e507b462ae8b51e8e): No space left in cache to reserve 128820 bytes, will continue without cache download 2024.10.22 01:13:16.285832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2110dbfd5b4f6b1e507b462ae8b51e8e, offset: 0, size: 0 2024.10.22 01:13:16.287070 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2110dbfd5b4f6b1e507b462ae8b51e8e for removal 2024.10.22 01:13:16.290539 [ 1476 ] {} CacheMetadata: Key 2110dbfd5b4f6b1e507b462ae8b51e8e is removed from metadata 2024.10.22 01:13:16.291985 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 58bfd7c157b2d7bfecde1d5b1dc690cf, offset: 0, size: 0 2024.10.22 01:13:16.295318 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 58bfd7c157b2d7bfecde1d5b1dc690cf for removal 2024.10.22 01:13:16.297100 [ 1476 ] {} CacheMetadata: Key 58bfd7c157b2d7bfecde1d5b1dc690cf is removed from metadata 2024.10.22 01:13:16.450436 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.96 GiB, free memory in arenas 14.56 MiB, will set to 1.68 GiB (RSS), difference: -48.90 MiB 2024.10.22 01:13:17.040269 [ 1203 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:13:17.065655 [ 1203 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:13:17.073378 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 5995 2024.10.22 01:13:17.388102 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(61f34743f6449d0d6c398b1a7d1a3084) : [0, 168]: Setting download as failed: File segment: [0, 168], key: 61f34743f6449d0d6c398b1a7d1a3084, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:17.391296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(61f34743f6449d0d6c398b1a7d1a3084): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:13:17.464678 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:17.468314 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:17.603260 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_397_397_0 (state Temporary) 2024.10.22 01:13:17.672766 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_397_397_0 to 202410_410_410_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:17.690710 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_682_300} MergeTask::MergeProjectionsStage: Merge sorted 42297 rows, containing 13 columns (13 merged, 0 gathered) in 3.120866693 sec., 13552.96594207973 rows/sec., 2.15 MiB/sec. 2024.10.22 01:13:17.710351 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 5995 2024.10.22 01:13:17.778137 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_682_300} MergedBlockOutputStream: filled checksums 202410_1_682_300 (state Temporary) 2024.10.22 01:13:17.971395 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_682_300} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_682_300 to 202410_1_682_300 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:17.975861 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_682_300} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_681_299, 202410_682_682_0] -> 202410_1_682_300 2024.10.22 01:13:17.994379 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:13:18.003011 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_392_170} MergeTask::MergeProjectionsStage: Merge sorted 37664 rows, containing 13 columns (13 merged, 0 gathered) in 3.135409102 sec., 12012.46752009971 rows/sec., 5.18 MiB/sec. 2024.10.22 01:13:18.166435 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_392_170} MergedBlockOutputStream: filled checksums 202410_1_392_170 (state Temporary) 2024.10.22 01:13:18.315432 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7ac1180e78140c63a67a55016449b037) : [0, 148739]: Setting download as failed: File segment: [0, 148739], key: 7ac1180e78140c63a67a55016449b037, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:18.324681 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 41942 2024.10.22 01:13:18.326627 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7ac1180e78140c63a67a55016449b037): No space left in cache to reserve 148740 bytes, will continue without cache download 2024.10.22 01:13:18.577502 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ac1180e78140c63a67a55016449b037, offset: 0, size: 0 2024.10.22 01:13:18.578911 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ac1180e78140c63a67a55016449b037 for removal 2024.10.22 01:13:18.582350 [ 1476 ] {} CacheMetadata: Key 7ac1180e78140c63a67a55016449b037 is removed from metadata 2024.10.22 01:13:18.582884 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 61f34743f6449d0d6c398b1a7d1a3084, offset: 0, size: 0 2024.10.22 01:13:18.583828 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 61f34743f6449d0d6c398b1a7d1a3084 for removal 2024.10.22 01:13:18.586594 [ 1476 ] {} CacheMetadata: Key 61f34743f6449d0d6c398b1a7d1a3084 is removed from metadata 2024.10.22 01:13:18.617303 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:18.624843 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:18.699937 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_675_675_0 (state Temporary) 2024.10.22 01:13:18.877512 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_675_675_0 to 202410_688_688_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:18.921760 [ 1274 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_682_300 to 202410_683_683_0 2024.10.22 01:13:18.936592 [ 1274 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.51 MiB using storage policy from min volume index 0 2024.10.22 01:13:18.941750 [ 1274 ] {} DiskLocal: Reserved 1.51 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:18.946088 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 41942 2024.10.22 01:13:18.971030 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_683_301} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_682_300 to 202410_683_683_0 into Compact with storage Full 2024.10.22 01:13:19.035464 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_683_301} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:19.057630 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_683_301} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_682_300, total 42297 rows starting from the beginning of the part 2024.10.22 01:13:19.112920 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_683_301} MergeTreeSequentialSource: Reading 2 marks from part 202410_683_683_0, total 66 rows starting from the beginning of the part 2024.10.22 01:13:19.332572 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_392_170} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_392_170 to 202410_1_392_170 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:19.350368 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_392_170} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_391_169, 202410_392_392_0] -> 202410_1_392_170 2024.10.22 01:13:19.379144 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.96 MiB. 2024.10.22 01:13:19.485221 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.96 GiB, free memory in arenas 14.77 MiB, will set to 1.68 GiB (RSS), difference: 46.16 MiB 2024.10.22 01:13:19.777433 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(efe06db628a0a4bf9c2e19b5e596df5b) : [0, 167]: Setting download as failed: File segment: [0, 167], key: efe06db628a0a4bf9c2e19b5e596df5b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:19.779430 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(efe06db628a0a4bf9c2e19b5e596df5b): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:13:19.977623 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1316 entries to flush up to offset 612334 2024.10.22 01:13:20.031795 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:13:20.037421 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:13:20.479083 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:20.482558 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:20.487977 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_583_206, 202410_584_584_0] 2024.10.22 01:13:20.490474 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_583_206, 202410_584_584_0] 2024.10.22 01:13:20.504475 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_583_206, 202410_584_584_0] 2024.10.22 01:13:20.726788 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9293d2497ec519bcc959d18d48252681) : [0, 81797]: Setting download as failed: File segment: [0, 81797], key: 9293d2497ec519bcc959d18d48252681, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:20.728429 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9293d2497ec519bcc959d18d48252681): No space left in cache to reserve 81798 bytes, will continue without cache download 2024.10.22 01:13:20.753458 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9293d2497ec519bcc959d18d48252681, offset: 0, size: 0 2024.10.22 01:13:20.756242 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9293d2497ec519bcc959d18d48252681 for removal 2024.10.22 01:13:20.762203 [ 1476 ] {} CacheMetadata: Key 9293d2497ec519bcc959d18d48252681 is removed from metadata 2024.10.22 01:13:20.763952 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: efe06db628a0a4bf9c2e19b5e596df5b, offset: 0, size: 0 2024.10.22 01:13:20.767288 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key efe06db628a0a4bf9c2e19b5e596df5b for removal 2024.10.22 01:13:20.771068 [ 1476 ] {} CacheMetadata: Key efe06db628a0a4bf9c2e19b5e596df5b is removed from metadata 2024.10.22 01:13:20.815965 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_381_381_0 (state Temporary) 2024.10.22 01:13:20.907926 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_381_381_0 to 202410_388_388_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:20.944686 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 612334 2024.10.22 01:13:20.966390 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_386_77/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:13:21.586705 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fa0444ad57dcb8f7c12985f179f209dc) : [0, 168]: Setting download as failed: File segment: [0, 168], key: fa0444ad57dcb8f7c12985f179f209dc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:21.588486 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fa0444ad57dcb8f7c12985f179f209dc): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:13:21.768014 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.96 GiB, free memory in arenas 14.92 MiB, will set to 1.69 GiB (RSS), difference: 8.36 MiB 2024.10.22 01:13:21.873756 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_683_301} MergeTask::MergeProjectionsStage: Merge sorted 42363 rows, containing 13 columns (13 merged, 0 gathered) in 2.912746336 sec., 14544.005935709465 rows/sec., 2.31 MiB/sec. 2024.10.22 01:13:21.976963 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_80_80_0 to 202410_83_83_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:22.041001 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_683_301} MergedBlockOutputStream: filled checksums 202410_1_683_301 (state Temporary) 2024.10.22 01:13:22.200209 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_683_301} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_683_301 to 202410_1_683_301 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:22.317640 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_683_301} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_682_300, 202410_683_683_0] -> 202410_1_683_301 2024.10.22 01:13:22.321461 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(af4a2ecaa67b5d756b0ce3bf75d2afd1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: af4a2ecaa67b5d756b0ce3bf75d2afd1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:13:22.327129 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 225349, file segment info: File segment: [0, 33554431], key: af4a2ecaa67b5d756b0ce3bf75d2afd1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:13:22.332656 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:22.392932 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 199 entries to flush up to offset 38627 2024.10.22 01:13:22.554291 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:13:22.691884 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:22.695119 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:22.852242 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_392_392_0 (state Temporary) 2024.10.22 01:13:22.915612 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_392_392_0 to 202410_398_398_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:22.948821 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 38627 2024.10.22 01:13:23.000027 [ 1244 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_392_170 to 202410_393_393_0 2024.10.22 01:13:23.055063 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 42006 2024.10.22 01:13:23.055170 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_346_69, 202410_347_347_0, 202410_348_348_0, 202410_349_349_0, 202410_350_350_0, 202410_351_351_0] 2024.10.22 01:13:23.062472 [ 1244 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.96 MiB using storage policy from min volume index 0 2024.10.22 01:13:23.066683 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_346_69, 202410_347_347_0, 202410_348_348_0, 202410_349_349_0, 202410_350_350_0, 202410_351_351_0] 2024.10.22 01:13:23.067702 [ 1244 ] {} DiskLocal: Reserved 1.96 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:23.093305 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_393_171} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_392_170 to 202410_393_393_0 into Wide with storage Full 2024.10.22 01:13:23.173693 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_393_171} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:23.186168 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_346_69, 202410_347_347_0, 202410_348_348_0, 202410_349_349_0, 202410_350_350_0, 202410_351_351_0] 2024.10.22 01:13:23.194145 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_393_171} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_392_170, total 37664 rows starting from the beginning of the part 2024.10.22 01:13:23.304864 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:23.307238 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:23.327048 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bf1ad01033f308b798872b0be43b9aec) : [0, 1049087]: Setting download as failed: File segment: [0, 1049087], key: bf1ad01033f308b798872b0be43b9aec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:23.328920 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bf1ad01033f308b798872b0be43b9aec): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:13:23.374035 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_676_676_0 (state Temporary) 2024.10.22 01:13:23.439029 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_676_676_0 to 202410_689_689_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:23.476552 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_393_171} MergeTreeSequentialSource: Reading 2 marks from part 202410_393_393_0, total 157 rows starting from the beginning of the part 2024.10.22 01:13:23.480911 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bf1ad01033f308b798872b0be43b9aec, offset: 0, size: 0 2024.10.22 01:13:23.481930 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bf1ad01033f308b798872b0be43b9aec for removal 2024.10.22 01:13:23.484632 [ 1476 ] {} CacheMetadata: Key bf1ad01033f308b798872b0be43b9aec is removed from metadata 2024.10.22 01:13:23.485827 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_683_301 to 202410_684_684_0 2024.10.22 01:13:23.488923 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fa0444ad57dcb8f7c12985f179f209dc, offset: 0, size: 0 2024.10.22 01:13:23.491723 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fa0444ad57dcb8f7c12985f179f209dc for removal 2024.10.22 01:13:23.494213 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.51 MiB using storage policy from min volume index 0 2024.10.22 01:13:23.494282 [ 1476 ] {} CacheMetadata: Key fa0444ad57dcb8f7c12985f179f209dc is removed from metadata 2024.10.22 01:13:23.500122 [ 1215 ] {} DiskLocal: Reserved 1.51 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:23.552788 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_684_302} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_683_301 to 202410_684_684_0 into Compact with storage Full 2024.10.22 01:13:23.596383 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_684_302} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:23.605410 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_684_302} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_683_301, total 42363 rows starting from the beginning of the part 2024.10.22 01:13:23.676248 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_684_302} MergeTreeSequentialSource: Reading 2 marks from part 202410_684_684_0, total 79 rows starting from the beginning of the part 2024.10.22 01:13:23.678352 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42006 2024.10.22 01:13:24.022448 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_584_207, 202410_585_585_0] 2024.10.22 01:13:24.024755 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_584_207, 202410_585_585_0] 2024.10.22 01:13:24.039764 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_584_207, 202410_585_585_0] 2024.10.22 01:13:24.078899 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_684_302/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:13:24.088655 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(6f0727a112ca65511cb20faebcedf97f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6f0727a112ca65511cb20faebcedf97f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:13:24.095423 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 649, file segment info: File segment: [0, 33554431], key: 6f0727a112ca65511cb20faebcedf97f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:13:24.100255 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:25.155655 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3191 2024.10.22 01:13:25.205440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e4a389cb3aeea1a43cc7a09a425807fa) : [0, 154]: Setting download as failed: File segment: [0, 154], key: e4a389cb3aeea1a43cc7a09a425807fa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:25.207667 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e4a389cb3aeea1a43cc7a09a425807fa): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:13:25.218937 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 6010 2024.10.22 01:13:25.575110 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.96 GiB, free memory in arenas 13.34 MiB, will set to 1.69 GiB (RSS), difference: -40.23 MiB 2024.10.22 01:13:25.600429 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:25.603034 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:25.658746 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_398_398_0 (state Temporary) 2024.10.22 01:13:25.686864 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_393_171/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:13:25.703905 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_398_398_0 to 202410_411_411_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:25.769116 [ 1194 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_406_81 to 202410_411_411_0 2024.10.22 01:13:25.783541 [ 1194 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:25.786464 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:25.857267 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6010 2024.10.22 01:13:25.883911 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_406_81 to 202410_411_411_0 into Compact with storage Full 2024.10.22 01:13:25.999912 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_393_171} MergeTask::MergeProjectionsStage: Merge sorted 37821 rows, containing 13 columns (13 merged, 0 gathered) in 2.911593004 sec., 12989.796289536625 rows/sec., 5.60 MiB/sec. 2024.10.22 01:13:26.101745 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_393_171} MergedBlockOutputStream: filled checksums 202410_1_393_171 (state Temporary) 2024.10.22 01:13:26.207958 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(900439dadfd3b63f5e2af7136715978e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 900439dadfd3b63f5e2af7136715978e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:13:26.214299 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 46263, file segment info: File segment: [0, 33554431], key: 900439dadfd3b63f5e2af7136715978e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:13:26.217229 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:26.617519 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_684_302} MergeTask::MergeProjectionsStage: Merge sorted 42442 rows, containing 13 columns (13 merged, 0 gathered) in 3.077291965 sec., 13791.996496504029 rows/sec., 2.19 MiB/sec. 2024.10.22 01:13:26.788108 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_393_171} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_393_171 to 202410_1_393_171 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:26.792907 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_393_171} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_392_170, 202410_393_393_0] -> 202410_1_393_171 2024.10.22 01:13:26.805317 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.89 MiB. 2024.10.22 01:13:26.818260 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:26.827673 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_406_81, total 6034 rows starting from the beginning of the part 2024.10.22 01:13:26.886717 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_684_302} MergedBlockOutputStream: filled checksums 202410_1_684_302 (state Temporary) 2024.10.22 01:13:26.893790 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_407_407_0, total 22 rows starting from the beginning of the part 2024.10.22 01:13:26.901282 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e1a5552aa0312f3ffa5eacfa54766e7a) : [0, 50373]: Setting download as failed: File segment: [0, 50373], key: e1a5552aa0312f3ffa5eacfa54766e7a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:26.903805 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e1a5552aa0312f3ffa5eacfa54766e7a): No space left in cache to reserve 50374 bytes, will continue without cache download 2024.10.22 01:13:26.944644 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e1a5552aa0312f3ffa5eacfa54766e7a, offset: 0, size: 0 2024.10.22 01:13:26.958289 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_408_408_0, total 12 rows starting from the beginning of the part 2024.10.22 01:13:27.025620 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_409_409_0, total 14 rows starting from the beginning of the part 2024.10.22 01:13:27.046597 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e1a5552aa0312f3ffa5eacfa54766e7a for removal 2024.10.22 01:13:27.051561 [ 1476 ] {} CacheMetadata: Key e1a5552aa0312f3ffa5eacfa54766e7a is removed from metadata 2024.10.22 01:13:27.052224 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e4a389cb3aeea1a43cc7a09a425807fa, offset: 0, size: 0 2024.10.22 01:13:27.055361 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e4a389cb3aeea1a43cc7a09a425807fa for removal 2024.10.22 01:13:27.059228 [ 1476 ] {} CacheMetadata: Key e4a389cb3aeea1a43cc7a09a425807fa is removed from metadata 2024.10.22 01:13:27.088540 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_410_410_0, total 19 rows starting from the beginning of the part 2024.10.22 01:13:27.089751 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_684_302} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_684_302 to 202410_1_684_302 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:27.093418 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_684_302} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_683_301, 202410_684_684_0] -> 202410_1_684_302 2024.10.22 01:13:27.143948 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:13:27.153388 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_411_411_0, total 15 rows starting from the beginning of the part 2024.10.22 01:13:27.243411 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 339 2024.10.22 01:13:27.639241 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(42386e0b579dc08dd1c72207081e1bb8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 42386e0b579dc08dd1c72207081e1bb8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:13:27.642919 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 621, file segment info: File segment: [0, 33554431], key: 42386e0b579dc08dd1c72207081e1bb8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:13:27.648169 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:27.648428 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:27.651621 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:27.676294 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} MergeTask::MergeProjectionsStage: Merge sorted 623180 rows, containing 4 columns (4 merged, 0 gathered) in 22.795870498 sec., 27337.407450822062 rows/sec., 400.71 KiB/sec. 2024.10.22 01:13:27.688710 [ 1679 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kvd/yhbsmwmddoplbqkjuyzcgzqpdxlte, size 46263 2024.10.22 01:13:27.696427 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 42095 2024.10.22 01:13:27.716362 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_321_321_0 (state Temporary) 2024.10.22 01:13:27.719894 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} MergedBlockOutputStream: filled checksums 202410_1_386_77 (state Temporary) 2024.10.22 01:13:27.771522 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_386_77 to 202410_1_386_77 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:27.775922 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_386_77} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_381_76, 202410_386_386_0] -> 202410_1_386_77 2024.10.22 01:13:27.823916 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:13:27.894243 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_321_321_0 to 202410_331_331_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:27.952608 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:13:27.963966 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 712 entries to flush up to offset 613046 2024.10.22 01:13:27.968248 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:13:27.973340 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 339 2024.10.22 01:13:27.982287 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:13:27.996968 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:13:28.125482 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:13:28.154651 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:13:28.171981 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:28.175600 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:28.183825 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:13:28.228515 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/omh/nqakwcdphbgbovrdgtzygtvkigmzw (LogSeriesLimiter: on interval from 2024-10-22 01:13:07 to 2024-10-22 01:13:28 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:13:28.257083 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} MergeTask::MergeProjectionsStage: Merge sorted 6116 rows, containing 12 columns (12 merged, 0 gathered) in 2.380336673 sec., 2569.384435980582 rows/sec., 406.43 KiB/sec. 2024.10.22 01:13:28.387922 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_677_677_0 (state Temporary) 2024.10.22 01:13:28.439143 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} MergedBlockOutputStream: filled checksums 202410_1_411_82 (state Temporary) 2024.10.22 01:13:28.445757 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_677_677_0 to 202410_690_690_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:28.476594 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_684_302 to 202410_685_685_0 2024.10.22 01:13:28.483230 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.52 MiB using storage policy from min volume index 0 2024.10.22 01:13:28.487982 [ 1254 ] {} DiskLocal: Reserved 1.52 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:28.488829 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42095 2024.10.22 01:13:28.504031 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_685_303} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_684_302 to 202410_685_685_0 into Compact with storage Full 2024.10.22 01:13:28.544635 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_685_303} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:28.566828 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_685_303} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_684_302, total 42442 rows starting from the beginning of the part 2024.10.22 01:13:28.641723 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:28.645403 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:28.646255 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_585_208, 202410_586_586_0] 2024.10.22 01:13:28.651258 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_585_208, 202410_586_586_0] 2024.10.22 01:13:28.669631 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_585_208, 202410_586_586_0] 2024.10.22 01:13:28.723760 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(dd55f2af6afe1571d0e9d486249faa07) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: dd55f2af6afe1571d0e9d486249faa07, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:28.728581 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 2142, file segment info: File segment: [0, 33554431], key: dd55f2af6afe1571d0e9d486249faa07, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:28.733278 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:28.799053 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_411_82 to 202410_1_411_82 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:28.804044 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_411_82} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_406_81, 202410_411_411_0] -> 202410_1_411_82 2024.10.22 01:13:28.811624 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_685_303} MergeTreeSequentialSource: Reading 2 marks from part 202410_685_685_0, total 73 rows starting from the beginning of the part 2024.10.22 01:13:28.948260 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_382_382_0 (state Temporary) 2024.10.22 01:13:29.048247 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.43 MiB. 2024.10.22 01:13:29.125391 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_382_382_0 to 202410_389_389_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:29.267414 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 613046 2024.10.22 01:13:29.327795 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(832c2b70b2c516df3c2d4785b02cded3) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 832c2b70b2c516df3c2d4785b02cded3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:29.329707 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(832c2b70b2c516df3c2d4785b02cded3): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:13:30.036085 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:13:30.038279 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:13:30.295111 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b40d32af54b02850343aafff2cbb445d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b40d32af54b02850343aafff2cbb445d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:30.296716 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 524, file segment info: File segment: [0, 33554431], key: b40d32af54b02850343aafff2cbb445d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:30.298785 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.96 GiB, free memory in arenas 10.58 MiB, will set to 1.68 GiB (RSS), difference: 35.44 MiB 2024.10.22 01:13:30.299039 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:30.588775 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 221 entries to flush up to offset 38848 2024.10.22 01:13:30.806207 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:30.817593 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:31.031070 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_393_393_0 (state Temporary) 2024.10.22 01:13:31.068609 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(180d39c43085a10fac00598e0a0e9ee4) : [0, 43994]: Setting download as failed: File segment: [0, 43994], key: 180d39c43085a10fac00598e0a0e9ee4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:31.136297 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(180d39c43085a10fac00598e0a0e9ee4): No space left in cache to reserve 43995 bytes, will continue without cache download 2024.10.22 01:13:31.169684 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 180d39c43085a10fac00598e0a0e9ee4, offset: 0, size: 0 2024.10.22 01:13:31.172190 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 180d39c43085a10fac00598e0a0e9ee4 for removal 2024.10.22 01:13:31.275533 [ 1476 ] {} CacheMetadata: Key 180d39c43085a10fac00598e0a0e9ee4 is removed from metadata 2024.10.22 01:13:31.278753 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 832c2b70b2c516df3c2d4785b02cded3, offset: 0, size: 0 2024.10.22 01:13:31.280290 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 832c2b70b2c516df3c2d4785b02cded3 for removal 2024.10.22 01:13:31.284716 [ 1476 ] {} CacheMetadata: Key 832c2b70b2c516df3c2d4785b02cded3 is removed from metadata 2024.10.22 01:13:31.311164 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_393_393_0 to 202410_399_399_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:31.355874 [ 1225 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_393_171 to 202410_394_394_0 2024.10.22 01:13:31.367117 [ 1225 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.96 MiB using storage policy from min volume index 0 2024.10.22 01:13:31.368715 [ 1225 ] {} DiskLocal: Reserved 1.96 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:31.377815 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_394_172} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_393_171 to 202410_394_394_0 into Wide with storage Full 2024.10.22 01:13:31.407710 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_394_172} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:31.423614 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_394_172} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_393_171, total 37821 rows starting from the beginning of the part 2024.10.22 01:13:31.464248 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 38848 2024.10.22 01:13:31.846605 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_394_172} MergeTreeSequentialSource: Reading 2 marks from part 202410_394_394_0, total 151 rows starting from the beginning of the part 2024.10.22 01:13:31.951325 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a1b36dc14df2ca3bd48782e749fc2895) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a1b36dc14df2ca3bd48782e749fc2895, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:31.954670 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 14235, file segment info: File segment: [0, 33554431], key: a1b36dc14df2ca3bd48782e749fc2895, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:31.957817 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:32.071566 [ 1191 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:13:32.077585 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_685_303} MergeTask::MergeProjectionsStage: Merge sorted 42515 rows, containing 13 columns (13 merged, 0 gathered) in 3.577871153 sec., 11882.764409878682 rows/sec., 1.89 MiB/sec. 2024.10.22 01:13:32.087699 [ 1191 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:13:32.118969 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_685_303} MergedBlockOutputStream: filled checksums 202410_1_685_303 (state Temporary) 2024.10.22 01:13:32.209332 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_685_303} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_685_303 to 202410_1_685_303 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:32.223749 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_685_303} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_684_302, 202410_685_685_0] -> 202410_1_685_303 2024.10.22 01:13:32.253599 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:13:32.470399 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(05f8999d7b47f38cdb2ed65e14dade21) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 05f8999d7b47f38cdb2ed65e14dade21, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:32.472752 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(05f8999d7b47f38cdb2ed65e14dade21): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:13:32.644004 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 42182 2024.10.22 01:13:32.693592 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 3231 2024.10.22 01:13:32.863797 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:32.867412 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:33.003329 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b446d4fadaea5a88488e4f6595cbc45b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b446d4fadaea5a88488e4f6595cbc45b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:33.026647 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 626, file segment info: File segment: [0, 33554431], key: b446d4fadaea5a88488e4f6595cbc45b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:13:33.028389 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:33.069574 [ 1717 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/oqw/jjityjqworspfzbpfmtzddrezymnw, size 14235 2024.10.22 01:13:33.253182 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_678_678_0 (state Temporary) 2024.10.22 01:13:33.273524 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:13:33.278391 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:13:33.302656 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:13:33.328335 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:13:33.348646 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:13:33.361007 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:13:33.361554 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 6038 2024.10.22 01:13:33.374265 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:13:33.390182 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:13:33.429214 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:13:33.500309 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.96 GiB, free memory in arenas 9.11 MiB, will set to 1.68 GiB (RSS), difference: -48.67 MiB 2024.10.22 01:13:33.578363 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:33.581383 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:33.590261 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/sat/ywqcnpbjsiaxzmvdmdurkzxohxsmf (LogSeriesLimiter: on interval from 2024-10-22 01:13:28 to 2024-10-22 01:13:33 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:13:33.724033 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_399_399_0 (state Temporary) 2024.10.22 01:13:33.737373 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_678_678_0 to 202410_691_691_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:33.769127 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9ad5aae360f84920308e4b2e504902c4) : [0, 76585]: Setting download as failed: File segment: [0, 76585], key: 9ad5aae360f84920308e4b2e504902c4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:33.773024 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9ad5aae360f84920308e4b2e504902c4): No space left in cache to reserve 76586 bytes, will continue without cache download 2024.10.22 01:13:33.736067 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_586_209, 202410_587_587_0] 2024.10.22 01:13:33.780752 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_586_209, 202410_587_587_0] 2024.10.22 01:13:33.791431 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_685_303 to 202410_686_686_0 2024.10.22 01:13:33.792593 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_586_209, 202410_587_587_0] 2024.10.22 01:13:33.801696 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.52 MiB using storage policy from min volume index 0 2024.10.22 01:13:33.804497 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42182 2024.10.22 01:13:33.814716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9ad5aae360f84920308e4b2e504902c4, offset: 0, size: 0 2024.10.22 01:13:33.817538 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9ad5aae360f84920308e4b2e504902c4 for removal 2024.10.22 01:13:33.822649 [ 1476 ] {} CacheMetadata: Key 9ad5aae360f84920308e4b2e504902c4 is removed from metadata 2024.10.22 01:13:33.823323 [ 1196 ] {} DiskLocal: Reserved 1.52 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:33.824187 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 05f8999d7b47f38cdb2ed65e14dade21, offset: 0, size: 0 2024.10.22 01:13:33.827571 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 05f8999d7b47f38cdb2ed65e14dade21 for removal 2024.10.22 01:13:33.830664 [ 1476 ] {} CacheMetadata: Key 05f8999d7b47f38cdb2ed65e14dade21 is removed from metadata 2024.10.22 01:13:33.862535 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_686_304} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_685_303 to 202410_686_686_0 into Compact with storage Full 2024.10.22 01:13:33.785786 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_399_399_0 to 202410_412_412_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:33.924357 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_686_304} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:33.928905 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6038 2024.10.22 01:13:33.957914 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_686_304} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_685_303, total 42515 rows starting from the beginning of the part 2024.10.22 01:13:34.063269 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_686_304} MergeTreeSequentialSource: Reading 2 marks from part 202410_686_686_0, total 96 rows starting from the beginning of the part 2024.10.22 01:13:34.787681 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_394_172} MergeTask::MergeProjectionsStage: Merge sorted 37972 rows, containing 13 columns (13 merged, 0 gathered) in 3.414715259 sec., 11120.107276856843 rows/sec., 4.80 MiB/sec. 2024.10.22 01:13:34.895559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(62e33ef8aeff8f87e5615491cdbb298a) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 62e33ef8aeff8f87e5615491cdbb298a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:34.897345 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(62e33ef8aeff8f87e5615491cdbb298a): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:13:34.952624 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_394_172} MergedBlockOutputStream: filled checksums 202410_1_394_172 (state Temporary) 2024.10.22 01:13:35.363589 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.96 GiB, free memory in arenas 8.60 MiB, will set to 1.68 GiB (RSS), difference: 50.98 MiB 2024.10.22 01:13:35.475239 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 340 2024.10.22 01:13:35.550074 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(11e05e5ec9d826e705378d338a188c04) : [0, 111992]: Setting download as failed: File segment: [0, 111992], key: 11e05e5ec9d826e705378d338a188c04, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:35.554945 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(11e05e5ec9d826e705378d338a188c04): No space left in cache to reserve 111993 bytes, will continue without cache download 2024.10.22 01:13:35.579542 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 11e05e5ec9d826e705378d338a188c04, offset: 0, size: 0 2024.10.22 01:13:35.591844 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 11e05e5ec9d826e705378d338a188c04 for removal 2024.10.22 01:13:35.594748 [ 1476 ] {} CacheMetadata: Key 11e05e5ec9d826e705378d338a188c04 is removed from metadata 2024.10.22 01:13:35.596200 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 62e33ef8aeff8f87e5615491cdbb298a, offset: 0, size: 0 2024.10.22 01:13:35.598438 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 62e33ef8aeff8f87e5615491cdbb298a for removal 2024.10.22 01:13:35.601042 [ 1476 ] {} CacheMetadata: Key 62e33ef8aeff8f87e5615491cdbb298a is removed from metadata 2024.10.22 01:13:35.613077 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:35.616519 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:35.684929 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_394_172} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_394_172 to 202410_1_394_172 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:35.688207 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_394_172} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_393_171, 202410_394_394_0] -> 202410_1_394_172 2024.10.22 01:13:35.688739 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_322_322_0 (state Temporary) 2024.10.22 01:13:35.704609 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.89 MiB. 2024.10.22 01:13:35.867457 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_322_322_0 to 202410_332_332_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:36.060375 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 340 2024.10.22 01:13:36.271121 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1538 entries to flush up to offset 614584 2024.10.22 01:13:36.326310 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_686_304} MergeTask::MergeProjectionsStage: Merge sorted 42611 rows, containing 13 columns (13 merged, 0 gathered) in 2.473877583 sec., 17224.376942826275 rows/sec., 2.73 MiB/sec. 2024.10.22 01:13:36.372819 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_686_304} MergedBlockOutputStream: filled checksums 202410_1_686_304 (state Temporary) 2024.10.22 01:13:36.403173 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_686_304} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_686_304 to 202410_1_686_304 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:36.405048 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_686_304} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_685_303, 202410_686_686_0] -> 202410_1_686_304 2024.10.22 01:13:36.425695 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:13:36.437658 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6ed6fb81ea70ff47c6879ed60c55f4d0) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 6ed6fb81ea70ff47c6879ed60c55f4d0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:36.440019 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6ed6fb81ea70ff47c6879ed60c55f4d0): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:13:37.222703 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:37.226992 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:37.410402 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.96 GiB, free memory in arenas 5.36 MiB, will set to 1.67 GiB (RSS), difference: -2.96 MiB 2024.10.22 01:13:37.540661 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b36e912ba913ad5e661dc0837e4de912) : [0, 110003]: Setting download as failed: File segment: [0, 110003], key: b36e912ba913ad5e661dc0837e4de912, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:37.545064 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b36e912ba913ad5e661dc0837e4de912): No space left in cache to reserve 110004 bytes, will continue without cache download 2024.10.22 01:13:37.561559 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_383_383_0 (state Temporary) 2024.10.22 01:13:37.570582 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b36e912ba913ad5e661dc0837e4de912, offset: 0, size: 0 2024.10.22 01:13:37.572125 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b36e912ba913ad5e661dc0837e4de912 for removal 2024.10.22 01:13:37.574833 [ 1476 ] {} CacheMetadata: Key b36e912ba913ad5e661dc0837e4de912 is removed from metadata 2024.10.22 01:13:37.577412 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6ed6fb81ea70ff47c6879ed60c55f4d0, offset: 0, size: 0 2024.10.22 01:13:37.580205 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6ed6fb81ea70ff47c6879ed60c55f4d0 for removal 2024.10.22 01:13:37.582019 [ 1476 ] {} CacheMetadata: Key 6ed6fb81ea70ff47c6879ed60c55f4d0 is removed from metadata 2024.10.22 01:13:37.640871 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_383_383_0 to 202410_390_390_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:37.823529 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 614584 2024.10.22 01:13:37.929624 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 42271 2024.10.22 01:13:37.950745 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_336_114, 202410_337_337_0] 2024.10.22 01:13:37.954317 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_336_114, 202410_337_337_0] 2024.10.22 01:13:38.006997 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_336_114, 202410_337_337_0] 2024.10.22 01:13:38.200480 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_587_210, 202410_588_588_0] 2024.10.22 01:13:38.205507 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_587_210, 202410_588_588_0] 2024.10.22 01:13:38.217076 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_587_210, 202410_588_588_0] 2024.10.22 01:13:38.709599 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:38.723393 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:38.806752 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_679_679_0 (state Temporary) 2024.10.22 01:13:38.901557 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d884202ef8e31e91ec2ea27334b04e89) : [0, 179]: Setting download as failed: File segment: [0, 179], key: d884202ef8e31e91ec2ea27334b04e89, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:38.904273 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d884202ef8e31e91ec2ea27334b04e89): No space left in cache to reserve 180 bytes, will continue without cache download 2024.10.22 01:13:38.987104 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 173 entries to flush up to offset 39021 2024.10.22 01:13:39.056933 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_679_679_0 to 202410_692_692_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:39.146289 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_686_304 to 202410_687_687_0 2024.10.22 01:13:39.159337 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42271 2024.10.22 01:13:39.160230 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.52 MiB using storage policy from min volume index 0 2024.10.22 01:13:39.165269 [ 1271 ] {} DiskLocal: Reserved 1.52 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:39.239095 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_687_305} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_686_304 to 202410_687_687_0 into Compact with storage Full 2024.10.22 01:13:39.284403 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:39.286196 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:39.362705 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_394_394_0 (state Temporary) 2024.10.22 01:13:39.410157 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_394_394_0 to 202410_400_400_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:39.475439 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 39021 2024.10.22 01:13:39.488563 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_687_305} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:39.504429 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_687_305} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_686_304, total 42611 rows starting from the beginning of the part 2024.10.22 01:13:39.537424 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_687_305} MergeTreeSequentialSource: Reading 2 marks from part 202410_687_687_0, total 84 rows starting from the beginning of the part 2024.10.22 01:13:39.637124 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_394_172 to 202410_395_395_0 2024.10.22 01:13:39.642627 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.97 MiB using storage policy from min volume index 0 2024.10.22 01:13:39.646788 [ 1248 ] {} DiskLocal: Reserved 1.97 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:39.654090 [ 1248 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 209 ms. 2024.10.22 01:13:39.701730 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.96 GiB, free memory in arenas 6.89 MiB, will set to 1.67 GiB (RSS), difference: 2.12 MiB 2024.10.22 01:13:39.770979 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_395_173} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_394_172 to 202410_395_395_0 into Wide with storage Full 2024.10.22 01:13:39.860515 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_395_173} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:39.879146 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_395_173} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_394_172, total 37972 rows starting from the beginning of the part 2024.10.22 01:13:39.943747 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5b88973b4f7a3472a32ec98f9aaf1e79) : [0, 242827]: Setting download as failed: File segment: [0, 242827], key: 5b88973b4f7a3472a32ec98f9aaf1e79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:39.946124 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5b88973b4f7a3472a32ec98f9aaf1e79): No space left in cache to reserve 242828 bytes, will continue without cache download 2024.10.22 01:13:39.982411 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5b88973b4f7a3472a32ec98f9aaf1e79, offset: 0, size: 0 2024.10.22 01:13:39.985396 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5b88973b4f7a3472a32ec98f9aaf1e79 for removal 2024.10.22 01:13:40.034849 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:13:40.037803 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:13:40.093171 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d884202ef8e31e91ec2ea27334b04e89, offset: 0, size: 0 2024.10.22 01:13:40.094586 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d884202ef8e31e91ec2ea27334b04e89 for removal 2024.10.22 01:13:40.097369 [ 1476 ] {} CacheMetadata: Key 5b88973b4f7a3472a32ec98f9aaf1e79 is removed from metadata 2024.10.22 01:13:40.104509 [ 1476 ] {} CacheMetadata: Key d884202ef8e31e91ec2ea27334b04e89 is removed from metadata 2024.10.22 01:13:40.180228 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_395_173} MergeTreeSequentialSource: Reading 2 marks from part 202410_395_395_0, total 161 rows starting from the beginning of the part 2024.10.22 01:13:40.825033 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_687_305/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:13:41.114379 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1001b43e6b6969d5827949c1ffb1d9b1) : [0, 166]: Setting download as failed: File segment: [0, 166], key: 1001b43e6b6969d5827949c1ffb1d9b1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:41.117153 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1001b43e6b6969d5827949c1ffb1d9b1): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 01:13:41.431588 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 6050 2024.10.22 01:13:41.536908 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.96 GiB, free memory in arenas 6.40 MiB, will set to 1.68 GiB (RSS), difference: -43.98 MiB 2024.10.22 01:13:41.539413 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:41.546331 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:41.590721 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_400_400_0 (state Temporary) 2024.10.22 01:13:41.757067 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_400_400_0 to 202410_413_413_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:41.799611 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_395_173/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:13:41.848690 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6050 2024.10.22 01:13:41.947274 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5ce8dc1970ed8513ba42f7e537e97827) : [0, 936602]: Setting download as failed: File segment: [0, 936602], key: 5ce8dc1970ed8513ba42f7e537e97827, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:41.949124 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5ce8dc1970ed8513ba42f7e537e97827): No space left in cache to reserve 936603 bytes, will continue without cache download 2024.10.22 01:13:41.953540 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_687_305} MergeTask::MergeProjectionsStage: Merge sorted 42695 rows, containing 13 columns (13 merged, 0 gathered) in 2.780975583 sec., 15352.526020362402 rows/sec., 2.44 MiB/sec. 2024.10.22 01:13:41.979884 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5ce8dc1970ed8513ba42f7e537e97827, offset: 0, size: 0 2024.10.22 01:13:41.981451 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5ce8dc1970ed8513ba42f7e537e97827 for removal 2024.10.22 01:13:41.985163 [ 1476 ] {} CacheMetadata: Key 5ce8dc1970ed8513ba42f7e537e97827 is removed from metadata 2024.10.22 01:13:41.989524 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1001b43e6b6969d5827949c1ffb1d9b1, offset: 0, size: 0 2024.10.22 01:13:41.992574 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1001b43e6b6969d5827949c1ffb1d9b1 for removal 2024.10.22 01:13:41.994704 [ 1476 ] {} CacheMetadata: Key 1001b43e6b6969d5827949c1ffb1d9b1 is removed from metadata 2024.10.22 01:13:42.000987 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_687_305} MergedBlockOutputStream: filled checksums 202410_1_687_305 (state Temporary) 2024.10.22 01:13:42.045975 [ 1233 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 01:13:42.295400 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_687_305} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_687_305 to 202410_1_687_305 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:42.299726 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_687_305} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_686_304, 202410_687_687_0] -> 202410_1_687_305 2024.10.22 01:13:42.328458 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:13:42.790576 [ 1217 ] {} DatabaseCatalog: Execution took 210 ms. 2024.10.22 01:13:42.976524 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7cb7fbb32dc594281ebb1e79b1c39b6e) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 7cb7fbb32dc594281ebb1e79b1c39b6e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:42.977736 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7cb7fbb32dc594281ebb1e79b1c39b6e): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:13:43.174282 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 42346 2024.10.22 01:13:43.192328 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_395_173} MergeTask::MergeProjectionsStage: Merge sorted 38133 rows, containing 13 columns (13 merged, 0 gathered) in 3.431834067 sec., 11111.551215917223 rows/sec., 4.79 MiB/sec. 2024.10.22 01:13:43.491793 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_395_173} MergedBlockOutputStream: filled checksums 202410_1_395_173 (state Temporary) 2024.10.22 01:13:43.506326 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_588_211, 202410_589_589_0] 2024.10.22 01:13:43.518983 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_588_211, 202410_589_589_0] 2024.10.22 01:13:43.563953 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 341 2024.10.22 01:13:43.632147 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.96 GiB, free memory in arenas 4.24 MiB, will set to 1.67 GiB (RSS), difference: 9.34 MiB 2024.10.22 01:13:43.634165 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_588_211, 202410_589_589_0] 2024.10.22 01:13:43.694916 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:43.697447 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:43.750465 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1ce5d90af89b0e633346bf97dd4439c2) : [0, 122661]: Setting download as failed: File segment: [0, 122661], key: 1ce5d90af89b0e633346bf97dd4439c2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:43.758848 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1ce5d90af89b0e633346bf97dd4439c2): No space left in cache to reserve 122662 bytes, will continue without cache download 2024.10.22 01:13:43.772603 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_680_680_0 (state Temporary) 2024.10.22 01:13:43.781391 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1ce5d90af89b0e633346bf97dd4439c2, offset: 0, size: 0 2024.10.22 01:13:43.783062 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1ce5d90af89b0e633346bf97dd4439c2 for removal 2024.10.22 01:13:43.788559 [ 1476 ] {} CacheMetadata: Key 1ce5d90af89b0e633346bf97dd4439c2 is removed from metadata 2024.10.22 01:13:43.789538 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7cb7fbb32dc594281ebb1e79b1c39b6e, offset: 0, size: 0 2024.10.22 01:13:43.791493 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7cb7fbb32dc594281ebb1e79b1c39b6e for removal 2024.10.22 01:13:43.795371 [ 1476 ] {} CacheMetadata: Key 7cb7fbb32dc594281ebb1e79b1c39b6e is removed from metadata 2024.10.22 01:13:43.934181 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_680_680_0 to 202410_693_693_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:43.973161 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_687_305 to 202410_688_688_0 2024.10.22 01:13:43.980012 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.53 MiB using storage policy from min volume index 0 2024.10.22 01:13:43.991692 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42346 2024.10.22 01:13:43.981645 [ 1223 ] {} DiskLocal: Reserved 1.53 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:44.012087 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_688_306} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_687_305 to 202410_688_688_0 into Compact with storage Full 2024.10.22 01:13:44.025009 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:44.027795 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:44.051474 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_688_306} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:44.066782 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_688_306} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_687_305, total 42695 rows starting from the beginning of the part 2024.10.22 01:13:44.195547 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_688_306} MergeTreeSequentialSource: Reading 2 marks from part 202410_688_688_0, total 78 rows starting from the beginning of the part 2024.10.22 01:13:44.220990 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_323_323_0 (state Temporary) 2024.10.22 01:13:44.282070 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_323_323_0 to 202410_333_333_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:44.336806 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_395_173} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_395_173 to 202410_1_395_173 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:44.340791 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_395_173} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_394_172, 202410_395_395_0] -> 202410_1_395_173 2024.10.22 01:13:44.367805 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.90 MiB. 2024.10.22 01:13:44.396360 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 341 2024.10.22 01:13:44.932565 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 616120 2024.10.22 01:13:44.946967 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f9a20b47dbb191b4351eb4426d5a2662) : [0, 168]: Setting download as failed: File segment: [0, 168], key: f9a20b47dbb191b4351eb4426d5a2662, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:44.948809 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f9a20b47dbb191b4351eb4426d5a2662): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:13:45.392084 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_337_115, 202410_338_338_0] 2024.10.22 01:13:45.393764 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_337_115, 202410_338_338_0] 2024.10.22 01:13:45.405023 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_337_115, 202410_338_338_0] 2024.10.22 01:13:45.435271 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.96 GiB, free memory in arenas 3.28 MiB, will set to 1.67 GiB (RSS), difference: 23.37 MiB 2024.10.22 01:13:45.802932 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(6efd086754eacb7fb48aa8a52583f0bc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6efd086754eacb7fb48aa8a52583f0bc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:13:45.822339 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 47, file segment info: File segment: [0, 33554431], key: 6efd086754eacb7fb48aa8a52583f0bc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:13:45.827985 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:46.099987 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:46.102435 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:46.311805 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_384_384_0 (state Temporary) 2024.10.22 01:13:46.487649 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_384_384_0 to 202410_391_391_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:46.512300 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 616120 2024.10.22 01:13:46.512834 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_386_77 to 202410_391_391_0 2024.10.22 01:13:46.546252 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:46.602151 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0077ca0517fa702ed428532558ab857f) : [0, 1057949]: Setting download as failed: File segment: [0, 1057949], key: 0077ca0517fa702ed428532558ab857f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:46.603716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0077ca0517fa702ed428532558ab857f): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:13:46.656913 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:46.763786 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_688_306} MergeTask::MergeProjectionsStage: Merge sorted 42773 rows, containing 13 columns (13 merged, 0 gathered) in 2.762040788 sec., 15486.013163104672 rows/sec., 2.46 MiB/sec. 2024.10.22 01:13:46.775024 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_386_77 to 202410_391_391_0 into Compact with storage Full 2024.10.22 01:13:46.792724 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:46.796190 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_688_306} MergedBlockOutputStream: filled checksums 202410_1_688_306 (state Temporary) 2024.10.22 01:13:46.808715 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} MergeTreeSequentialSource: Reading 78 marks from part 202410_1_386_77, total 623180 rows starting from the beginning of the part 2024.10.22 01:13:46.843279 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0077ca0517fa702ed428532558ab857f, offset: 0, size: 0 2024.10.22 01:13:46.846456 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0077ca0517fa702ed428532558ab857f for removal 2024.10.22 01:13:46.856727 [ 1476 ] {} CacheMetadata: Key 0077ca0517fa702ed428532558ab857f is removed from metadata 2024.10.22 01:13:46.861209 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f9a20b47dbb191b4351eb4426d5a2662, offset: 0, size: 0 2024.10.22 01:13:46.864228 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f9a20b47dbb191b4351eb4426d5a2662 for removal 2024.10.22 01:13:46.869251 [ 1476 ] {} CacheMetadata: Key f9a20b47dbb191b4351eb4426d5a2662 is removed from metadata 2024.10.22 01:13:46.821801 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_387_387_0, total 1428 rows starting from the beginning of the part 2024.10.22 01:13:46.939366 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_388_388_0, total 1316 rows starting from the beginning of the part 2024.10.22 01:13:46.959247 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_389_389_0, total 712 rows starting from the beginning of the part 2024.10.22 01:13:46.978915 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_390_390_0, total 1538 rows starting from the beginning of the part 2024.10.22 01:13:46.997217 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_326_65, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0, 202410_330_330_0, 202410_331_331_0] 2024.10.22 01:13:46.999422 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_326_65, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0, 202410_330_330_0, 202410_331_331_0] 2024.10.22 01:13:46.999197 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 200 entries to flush up to offset 39221 2024.10.22 01:13:47.009443 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_688_306} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_688_306 to 202410_1_688_306 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:47.016235 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_688_306} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_687_305, 202410_688_688_0] -> 202410_1_688_306 2024.10.22 01:13:47.026796 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_326_65, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0, 202410_330_330_0, 202410_331_331_0] 2024.10.22 01:13:47.059423 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:13:47.088026 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_391_391_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:13:47.094111 [ 1258 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:13:47.225103 [ 1258 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:13:47.320708 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:47.324425 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:47.498906 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_395_395_0 (state Temporary) 2024.10.22 01:13:47.545071 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_395_395_0 to 202410_401_401_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:47.580891 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 39221 2024.10.22 01:13:47.672695 [ 1255 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_395_173 to 202410_396_396_0 2024.10.22 01:13:47.688560 [ 1255 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.98 MiB using storage policy from min volume index 0 2024.10.22 01:13:47.690776 [ 1255 ] {} DiskLocal: Reserved 1.98 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:47.694346 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(e280326fa9831821528e3759708784ce) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e280326fa9831821528e3759708784ce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:13:47.697170 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 273, file segment info: File segment: [0, 33554431], key: e280326fa9831821528e3759708784ce, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:13:47.699935 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:47.711291 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_396_174} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_395_173 to 202410_396_396_0 into Wide with storage Full 2024.10.22 01:13:47.753482 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_396_174} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:47.769882 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_396_174} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_395_173, total 38133 rows starting from the beginning of the part 2024.10.22 01:13:47.999684 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 95 entries to flush up to offset 42441 2024.10.22 01:13:48.135500 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_589_212, 202410_590_590_0] 2024.10.22 01:13:48.137689 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_589_212, 202410_590_590_0] 2024.10.22 01:13:48.172076 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_396_174} MergeTreeSequentialSource: Reading 2 marks from part 202410_396_396_0, total 221 rows starting from the beginning of the part 2024.10.22 01:13:48.250390 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_589_212, 202410_590_590_0] 2024.10.22 01:13:48.277888 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:48.280373 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:48.373508 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_681_681_0 (state Temporary) 2024.10.22 01:13:48.504735 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(345061eadd7de4e03259e06499aa6634) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 345061eadd7de4e03259e06499aa6634, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:48.506535 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(345061eadd7de4e03259e06499aa6634): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:13:48.606636 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_681_681_0 to 202410_694_694_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:48.697305 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:48.743990 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:48.754273 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_688_306 to 202410_689_689_0 2024.10.22 01:13:48.766522 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.53 MiB using storage policy from min volume index 0 2024.10.22 01:13:48.771285 [ 1265 ] {} DiskLocal: Reserved 1.53 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:48.780850 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42441 2024.10.22 01:13:48.790966 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_689_307} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_688_306 to 202410_689_689_0 into Compact with storage Full 2024.10.22 01:13:48.869203 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_689_307} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:48.878038 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_689_307} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_688_306, total 42773 rows starting from the beginning of the part 2024.10.22 01:13:48.900332 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_689_307} MergeTreeSequentialSource: Reading 2 marks from part 202410_689_689_0, total 64 rows starting from the beginning of the part 2024.10.22 01:13:49.354680 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 6065 2024.10.22 01:13:49.395929 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.96 GiB, free memory in arenas 8.14 MiB, will set to 1.67 GiB (RSS), difference: -49.82 MiB 2024.10.22 01:13:49.600086 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(8ca6bd88d0ed832c82b84e4cf351a830) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8ca6bd88d0ed832c82b84e4cf351a830, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:13:49.602970 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4052, file segment info: File segment: [0, 33554431], key: 8ca6bd88d0ed832c82b84e4cf351a830, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:13:49.607929 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:49.792407 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:49.794774 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:49.836837 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_401_401_0 (state Temporary) 2024.10.22 01:13:50.040291 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:13:50.043464 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:13:50.092502 [ 1160 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 01:13:50.168205 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_401_401_0 to 202410_414_414_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:50.176024 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_391_78/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:13:50.322221 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6065 2024.10.22 01:13:50.576045 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b7ed4baccae0c61fd02e6e2d7e5a2cc6) : [0, 76151]: Setting download as failed: File segment: [0, 76151], key: b7ed4baccae0c61fd02e6e2d7e5a2cc6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:50.583858 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b7ed4baccae0c61fd02e6e2d7e5a2cc6): No space left in cache to reserve 76152 bytes, will continue without cache download 2024.10.22 01:13:50.617560 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b7ed4baccae0c61fd02e6e2d7e5a2cc6, offset: 0, size: 0 2024.10.22 01:13:50.623113 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b7ed4baccae0c61fd02e6e2d7e5a2cc6 for removal 2024.10.22 01:13:50.629587 [ 1476 ] {} CacheMetadata: Key b7ed4baccae0c61fd02e6e2d7e5a2cc6 is removed from metadata 2024.10.22 01:13:50.638952 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 345061eadd7de4e03259e06499aa6634, offset: 0, size: 0 2024.10.22 01:13:50.655806 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 345061eadd7de4e03259e06499aa6634 for removal 2024.10.22 01:13:50.663867 [ 1476 ] {} CacheMetadata: Key 345061eadd7de4e03259e06499aa6634 is removed from metadata 2024.10.22 01:13:51.205427 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_689_307} MergeTask::MergeProjectionsStage: Merge sorted 42837 rows, containing 13 columns (13 merged, 0 gathered) in 2.424046234 sec., 17671.692643136277 rows/sec., 2.81 MiB/sec. 2024.10.22 01:13:51.676040 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_689_307} MergedBlockOutputStream: filled checksums 202410_1_689_307 (state Temporary) 2024.10.22 01:13:51.687265 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(dc7f8c5dcd1fb817cb0fae9f68f00d4a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: dc7f8c5dcd1fb817cb0fae9f68f00d4a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:13:51.691291 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_81_81_0 (state Temporary) 2024.10.22 01:13:51.800152 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: dc7f8c5dcd1fb817cb0fae9f68f00d4a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:13:51.804543 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:51.898170 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 342 2024.10.22 01:13:52.204928 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_396_174} MergeTask::MergeProjectionsStage: Merge sorted 38354 rows, containing 13 columns (13 merged, 0 gathered) in 4.499848694 sec., 8523.397698047134 rows/sec., 3.68 MiB/sec. 2024.10.22 01:13:52.214367 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_689_307} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_689_307 to 202410_1_689_307 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:52.220395 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_689_307} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_688_306, 202410_689_689_0] -> 202410_1_689_307 2024.10.22 01:13:52.391870 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:52.394511 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:52.469752 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:13:52.470362 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_324_324_0 (state Temporary) 2024.10.22 01:13:52.669305 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_324_324_0 to 202410_334_334_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:52.705131 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_396_174} MergedBlockOutputStream: filled checksums 202410_1_396_174 (state Temporary) 2024.10.22 01:13:52.753700 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 342 2024.10.22 01:13:52.783248 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b857e811e7cec85f3cfcc3479956d64c) : [0, 155]: Setting download as failed: File segment: [0, 155], key: b857e811e7cec85f3cfcc3479956d64c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:52.786307 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b857e811e7cec85f3cfcc3479956d64c): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:13:52.889492 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 42503 2024.10.22 01:13:52.972226 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_81_81_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:13:53.012967 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:53.029256 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:13:53.108014 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_682_682_0 (state Temporary) 2024.10.22 01:13:53.163324 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_682_682_0 to 202410_695_695_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:53.182335 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_689_307 to 202410_690_690_0 2024.10.22 01:13:53.206469 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.53 MiB using storage policy from min volume index 0 2024.10.22 01:13:53.211758 [ 1187 ] {} DiskLocal: Reserved 1.53 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:13:53.224555 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42503 2024.10.22 01:13:53.228930 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_690_308} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_689_307 to 202410_690_690_0 into Compact with storage Full 2024.10.22 01:13:53.388776 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(1e86f91c9ae4ad6e181cac537505f5eb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1e86f91c9ae4ad6e181cac537505f5eb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:13:53.393037 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 604, file segment info: File segment: [0, 33554431], key: 1e86f91c9ae4ad6e181cac537505f5eb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:13:53.396441 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:53.445370 [ 1678 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/sat/ywqcnpbjsiaxzmvdmdurkzxohxsmf, size 1076178 2024.10.22 01:13:53.484011 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.96 GiB, free memory in arenas 13.61 MiB, will set to 1.68 GiB (RSS), difference: 62.71 MiB 2024.10.22 01:13:53.495103 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_690_308} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:53.502973 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_690_308} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_689_307, total 42837 rows starting from the beginning of the part 2024.10.22 01:13:53.515705 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 768 entries to flush up to offset 616888 2024.10.22 01:13:53.686061 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:13:53.702363 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:13:53.707630 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_590_213, 202410_591_591_0] 2024.10.22 01:13:53.712569 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_590_213, 202410_591_591_0] 2024.10.22 01:13:53.715886 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 249 ms. 2024.10.22 01:13:53.719592 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:13:53.724771 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_396_174} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_396_174 to 202410_1_396_174 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:53.733312 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_590_213, 202410_591_591_0] 2024.10.22 01:13:53.740851 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_396_174} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_395_173, 202410_396_396_0] -> 202410_1_396_174 2024.10.22 01:13:53.745432 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_690_308} MergeTreeSequentialSource: Reading 2 marks from part 202410_690_690_0, total 89 rows starting from the beginning of the part 2024.10.22 01:13:53.761608 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:13:53.777410 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.96 MiB. 2024.10.22 01:13:53.788992 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:13:53.835674 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:13:53.866155 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:13:53.878676 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:13:53.897206 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:13:53.971770 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/veu/tadkuihrcrctknzypdbunepmsvrpi (LogSeriesLimiter: on interval from 2024-10-22 01:13:33 to 2024-10-22 01:13:53 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:13:54.116397 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:54.122527 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:13:54.167198 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(f9252afa461f19db7f00e9f2fe85bd92) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f9252afa461f19db7f00e9f2fe85bd92, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:13:54.170708 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 491, file segment info: File segment: [0, 33554431], key: f9252afa461f19db7f00e9f2fe85bd92, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:13:54.182476 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:54.225512 [ 1728 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/omh/nqakwcdphbgbovrdgtzygtvkigmzw, size 4052 2024.10.22 01:13:54.285228 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_385_385_0 (state Temporary) 2024.10.22 01:13:54.446157 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_385_385_0 to 202410_392_392_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:54.487949 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 616888 2024.10.22 01:13:54.581805 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:13:54.599085 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:13:54.613363 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:13:54.733680 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:13:54.770610 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:13:54.791594 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:13:54.816171 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:13:54.911731 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(eab80fc46702b32954fd579ffb1f6497) : [0, 66549]: Setting download as failed: File segment: [0, 66549], key: eab80fc46702b32954fd579ffb1f6497, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:54.918654 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(eab80fc46702b32954fd579ffb1f6497): No space left in cache to reserve 66550 bytes, will continue without cache download 2024.10.22 01:13:54.950271 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eab80fc46702b32954fd579ffb1f6497, offset: 0, size: 0 2024.10.22 01:13:54.956892 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eab80fc46702b32954fd579ffb1f6497 for removal 2024.10.22 01:13:54.967616 [ 1476 ] {} CacheMetadata: Key eab80fc46702b32954fd579ffb1f6497 is removed from metadata 2024.10.22 01:13:54.970316 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b857e811e7cec85f3cfcc3479956d64c, offset: 0, size: 0 2024.10.22 01:13:54.972687 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b857e811e7cec85f3cfcc3479956d64c for removal 2024.10.22 01:13:54.992775 [ 1476 ] {} CacheMetadata: Key b857e811e7cec85f3cfcc3479956d64c is removed from metadata 2024.10.22 01:13:55.120460 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 234 entries to flush up to offset 39455 2024.10.22 01:13:55.168489 [ 1199 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 200 ms. 2024.10.22 01:13:55.436049 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:55.440306 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:13:55.473758 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_338_116, 202410_339_339_0] 2024.10.22 01:13:55.475172 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_338_116, 202410_339_339_0] 2024.10.22 01:13:55.512682 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_338_116, 202410_339_339_0] 2024.10.22 01:13:55.587273 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_396_396_0 (state Temporary) 2024.10.22 01:13:55.656688 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_396_396_0 to 202410_402_402_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:55.705251 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 39455 2024.10.22 01:13:55.786302 [ 1207 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_396_174 to 202410_397_397_0 2024.10.22 01:13:55.789677 [ 1207 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.99 MiB using storage policy from min volume index 0 2024.10.22 01:13:55.795242 [ 1207 ] {} DiskLocal: Reserved 1.99 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:55.843946 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_397_175} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_396_174 to 202410_397_397_0 into Wide with storage Full 2024.10.22 01:13:55.903764 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_397_175} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:55.926539 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_397_175} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_396_174, total 38354 rows starting from the beginning of the part 2024.10.22 01:13:56.110455 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_690_308/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:13:56.303927 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(897082f34091beff6c63c6383be8c62a) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 897082f34091beff6c63c6383be8c62a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:56.305579 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(897082f34091beff6c63c6383be8c62a): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:13:56.368283 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_397_175} MergeTreeSequentialSource: Reading 2 marks from part 202410_397_397_0, total 178 rows starting from the beginning of the part 2024.10.22 01:13:56.680302 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_690_308} MergeTask::MergeProjectionsStage: Merge sorted 42926 rows, containing 13 columns (13 merged, 0 gathered) in 3.457997958 sec., 12413.541164965603 rows/sec., 1.97 MiB/sec. 2024.10.22 01:13:56.722441 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_690_308} MergedBlockOutputStream: filled checksums 202410_1_690_308 (state Temporary) 2024.10.22 01:13:56.750868 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.96 GiB, free memory in arenas 12.15 MiB, will set to 1.69 GiB (RSS), difference: -53.50 MiB 2024.10.22 01:13:57.237523 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 42587 2024.10.22 01:13:57.249703 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(613ea1eb0ce1532eac3daaa75e8ea923) : [0, 64120]: Setting download as failed: File segment: [0, 64120], key: 613ea1eb0ce1532eac3daaa75e8ea923, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:57.253428 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(613ea1eb0ce1532eac3daaa75e8ea923): No space left in cache to reserve 64121 bytes, will continue without cache download 2024.10.22 01:13:57.279428 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_690_308} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_690_308 to 202410_1_690_308 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:57.283942 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_690_308} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_689_307, 202410_690_690_0] -> 202410_1_690_308 2024.10.22 01:13:57.310386 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:13:57.383346 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 613ea1eb0ce1532eac3daaa75e8ea923, offset: 0, size: 0 2024.10.22 01:13:57.385763 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 613ea1eb0ce1532eac3daaa75e8ea923 for removal 2024.10.22 01:13:57.390311 [ 1476 ] {} CacheMetadata: Key 613ea1eb0ce1532eac3daaa75e8ea923 is removed from metadata 2024.10.22 01:13:57.390677 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 897082f34091beff6c63c6383be8c62a, offset: 0, size: 0 2024.10.22 01:13:57.392992 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 897082f34091beff6c63c6383be8c62a for removal 2024.10.22 01:13:57.399014 [ 1476 ] {} CacheMetadata: Key 897082f34091beff6c63c6383be8c62a is removed from metadata 2024.10.22 01:13:57.618631 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:57.726072 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:13:57.789722 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_397_175/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:13:57.825418 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 6079 2024.10.22 01:13:57.838110 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_683_683_0 (state Temporary) 2024.10.22 01:13:57.879812 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_683_683_0 to 202410_696_696_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:57.911896 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_690_308 to 202410_691_691_0 2024.10.22 01:13:57.919764 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.54 MiB using storage policy from min volume index 0 2024.10.22 01:13:57.926772 [ 1161 ] {} DiskLocal: Reserved 1.54 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:13:57.937190 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42587 2024.10.22 01:13:57.969627 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:13:57.980950 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:13:57.992717 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_691_309} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_690_308 to 202410_691_691_0 into Compact with storage Full 2024.10.22 01:13:58.009135 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_691_309} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:13:58.017804 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_691_309} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_690_308, total 42926 rows starting from the beginning of the part 2024.10.22 01:13:58.033565 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_402_402_0 (state Temporary) 2024.10.22 01:13:58.068715 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_691_309} MergeTreeSequentialSource: Reading 2 marks from part 202410_691_691_0, total 87 rows starting from the beginning of the part 2024.10.22 01:13:58.221010 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_402_402_0 to 202410_415_415_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:58.260631 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_591_214, 202410_592_592_0] 2024.10.22 01:13:58.267430 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_591_214, 202410_592_592_0] 2024.10.22 01:13:58.278610 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_591_214, 202410_592_592_0] 2024.10.22 01:13:58.283265 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6079 2024.10.22 01:13:58.384801 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_81_81_0 to 202410_84_84_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:58.396408 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9a76bf43dcadcf5f1d7b099bd571c630) : [0, 167]: Setting download as failed: File segment: [0, 167], key: 9a76bf43dcadcf5f1d7b099bd571c630, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:13:58.398431 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9a76bf43dcadcf5f1d7b099bd571c630): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:13:58.505445 [ 1154 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_79_16 to 202410_84_84_0 2024.10.22 01:13:58.517009 [ 1154 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.70 MiB using storage policy from min volume index 0 2024.10.22 01:13:58.523265 [ 1154 ] {} DiskLocal: Reserved 1.70 MiB on local disk `default`, having unreserved 297.29 GiB. 2024.10.22 01:13:58.746456 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_397_175} MergeTask::MergeProjectionsStage: Merge sorted 38532 rows, containing 13 columns (13 merged, 0 gathered) in 2.941575624 sec., 13099.10229253382 rows/sec., 5.65 MiB/sec. 2024.10.22 01:13:58.784303 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_79_16 to 202410_84_84_0 into Compact with storage Full 2024.10.22 01:13:58.946367 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_397_175} MergedBlockOutputStream: filled checksums 202410_1_397_175 (state Temporary) 2024.10.22 01:13:59.359786 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(cf1b07f1e944b97ddbefdf9020f867ea) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cf1b07f1e944b97ddbefdf9020f867ea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:13:59.363459 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: cf1b07f1e944b97ddbefdf9020f867ea, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:13:59.366945 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:13:59.623776 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_397_175} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_397_175 to 202410_1_397_175 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:13:59.627450 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_397_175} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_396_174, 202410_397_397_0] -> 202410_1_397_175 2024.10.22 01:13:59.659405 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.90 MiB. 2024.10.22 01:14:00.040903 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:14:00.045343 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:14:00.256474 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 343 2024.10.22 01:14:00.262446 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_691_309} MergeTask::MergeProjectionsStage: Merge sorted 43013 rows, containing 13 columns (13 merged, 0 gathered) in 2.290484852 sec., 18778.993435578504 rows/sec., 2.98 MiB/sec. 2024.10.22 01:14:00.302624 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b12a10038ca86a772f233795f8ccbccc) : [0, 77738]: Setting download as failed: File segment: [0, 77738], key: b12a10038ca86a772f233795f8ccbccc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:00.304680 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b12a10038ca86a772f233795f8ccbccc): No space left in cache to reserve 77739 bytes, will continue without cache download 2024.10.22 01:14:00.326137 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b12a10038ca86a772f233795f8ccbccc, offset: 0, size: 0 2024.10.22 01:14:00.328427 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b12a10038ca86a772f233795f8ccbccc for removal 2024.10.22 01:14:00.346694 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9a76bf43dcadcf5f1d7b099bd571c630, offset: 0, size: 0 2024.10.22 01:14:00.351186 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9a76bf43dcadcf5f1d7b099bd571c630 for removal 2024.10.22 01:14:00.368359 [ 1476 ] {} CacheMetadata: Key b12a10038ca86a772f233795f8ccbccc is removed from metadata 2024.10.22 01:14:00.371503 [ 1476 ] {} CacheMetadata: Key 9a76bf43dcadcf5f1d7b099bd571c630 is removed from metadata 2024.10.22 01:14:00.422868 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_691_309} MergedBlockOutputStream: filled checksums 202410_1_691_309 (state Temporary) 2024.10.22 01:14:00.495326 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:00.507835 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:00.537047 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_691_309} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_691_309 to 202410_1_691_309 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:00.545296 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_691_309} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_690_308, 202410_691_691_0] -> 202410_1_691_309 2024.10.22 01:14:00.591561 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_325_325_0 (state Temporary) 2024.10.22 01:14:00.621078 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:14:00.910926 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_325_325_0 to 202410_335_335_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:00.904903 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3231 2024.10.22 01:14:00.964265 [ 1233 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_330_68 to 202410_335_335_0 2024.10.22 01:14:00.977446 [ 1233 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:00.980458 [ 1233 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:01.077293 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_330_68 to 202410_335_335_0 into Compact with storage Full 2024.10.22 01:14:01.166688 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 343 2024.10.22 01:14:01.183726 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(120efed09fed0d8bade5b53cb9c60887) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 120efed09fed0d8bade5b53cb9c60887, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:01.185190 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(120efed09fed0d8bade5b53cb9c60887): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:14:01.374553 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:01.384644 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_330_68, total 374 rows starting from the beginning of the part 2024.10.22 01:14:01.496997 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 618040 2024.10.22 01:14:01.539480 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_331_331_0, total 1 rows starting from the beginning of the part 2024.10.22 01:14:01.566340 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.96 GiB, free memory in arenas 13.68 MiB, will set to 1.69 GiB (RSS), difference: 47.63 MiB 2024.10.22 01:14:01.591733 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_332_332_0, total 1 rows starting from the beginning of the part 2024.10.22 01:14:01.640888 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_333_333_0, total 1 rows starting from the beginning of the part 2024.10.22 01:14:01.792498 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_334_334_0, total 1 rows starting from the beginning of the part 2024.10.22 01:14:01.951888 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_335_335_0, total 1 rows starting from the beginning of the part 2024.10.22 01:14:01.968657 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 42672 2024.10.22 01:14:02.099927 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_391_78/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:14:02.168590 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:02.173151 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:02.186406 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_339_117, 202410_340_340_0] 2024.10.22 01:14:02.188363 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_339_117, 202410_340_340_0] 2024.10.22 01:14:02.206943 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_339_117, 202410_340_340_0] 2024.10.22 01:14:02.232212 [ 1224 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:14:02.255482 [ 1224 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:14:02.284045 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_351_70, 202410_352_352_0, 202410_353_353_0, 202410_354_354_0, 202410_355_355_0, 202410_356_356_0] 2024.10.22 01:14:02.286146 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_351_70, 202410_352_352_0, 202410_353_353_0, 202410_354_354_0, 202410_355_355_0, 202410_356_356_0] 2024.10.22 01:14:02.294265 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_386_386_0 (state Temporary) 2024.10.22 01:14:02.300511 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(79b7550e926a2033b953833d74361c71) : [0, 32152]: Setting download as failed: File segment: [0, 32152], key: 79b7550e926a2033b953833d74361c71, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:02.304599 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(79b7550e926a2033b953833d74361c71): No space left in cache to reserve 32153 bytes, will continue without cache download 2024.10.22 01:14:02.310663 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_351_70, 202410_352_352_0, 202410_353_353_0, 202410_354_354_0, 202410_355_355_0, 202410_356_356_0] 2024.10.22 01:14:02.319837 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 79b7550e926a2033b953833d74361c71, offset: 0, size: 0 2024.10.22 01:14:02.323671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 79b7550e926a2033b953833d74361c71 for removal 2024.10.22 01:14:02.327972 [ 1476 ] {} CacheMetadata: Key 79b7550e926a2033b953833d74361c71 is removed from metadata 2024.10.22 01:14:02.329334 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 120efed09fed0d8bade5b53cb9c60887, offset: 0, size: 0 2024.10.22 01:14:02.330930 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 120efed09fed0d8bade5b53cb9c60887 for removal 2024.10.22 01:14:02.336401 [ 1476 ] {} CacheMetadata: Key 120efed09fed0d8bade5b53cb9c60887 is removed from metadata 2024.10.22 01:14:02.362730 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:14:02.473230 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:02.476677 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:02.655758 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_684_684_0 (state Temporary) 2024.10.22 01:14:02.693455 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_386_386_0 to 202410_393_393_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:02.710562 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_684_684_0 to 202410_697_697_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:02.734119 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 618040 2024.10.22 01:14:02.741804 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_691_309 to 202410_692_692_0 2024.10.22 01:14:02.759995 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.54 MiB using storage policy from min volume index 0 2024.10.22 01:14:02.774890 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42672 2024.10.22 01:14:02.780080 [ 1166 ] {} DiskLocal: Reserved 1.54 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:02.810638 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_692_310} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_691_309 to 202410_692_692_0 into Compact with storage Full 2024.10.22 01:14:03.041578 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b983d63f46e3dbc1b8a1d11abe1e0902) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b983d63f46e3dbc1b8a1d11abe1e0902, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:14:03.048059 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: b983d63f46e3dbc1b8a1d11abe1e0902, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:14:03.052724 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:03.091849 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_692_310} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:03.102850 [ 1228 ] {} DatabaseCatalog: Execution took 221 ms. 2024.10.22 01:14:03.120479 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_692_310} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_691_309, total 43013 rows starting from the beginning of the part 2024.10.22 01:14:03.267170 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 207 entries to flush up to offset 39662 2024.10.22 01:14:03.276717 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_692_310} MergeTreeSequentialSource: Reading 2 marks from part 202410_692_692_0, total 89 rows starting from the beginning of the part 2024.10.22 01:14:03.419545 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:03.422817 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:03.577935 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} MergeTask::MergeProjectionsStage: Merge sorted 379 rows, containing 32 columns (32 merged, 0 gathered) in 2.506292544 sec., 151.21937816370092 rows/sec., 15.31 KiB/sec. 2024.10.22 01:14:03.676626 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(49c51f9fac29e24755a4f3a611dde22b) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 49c51f9fac29e24755a4f3a611dde22b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:03.678459 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(49c51f9fac29e24755a4f3a611dde22b): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:14:03.699632 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} MergedBlockOutputStream: filled checksums 202410_1_335_69 (state Temporary) 2024.10.22 01:14:03.724044 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_397_397_0 (state Temporary) 2024.10.22 01:14:03.752397 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_592_215, 202410_593_593_0] 2024.10.22 01:14:03.841569 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_397_397_0 to 202410_403_403_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:03.868062 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_592_215, 202410_593_593_0] 2024.10.22 01:14:03.884673 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_592_215, 202410_593_593_0] 2024.10.22 01:14:03.885952 [ 1249 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_397_175 to 202410_398_398_0 2024.10.22 01:14:03.895007 [ 1249 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:03.896881 [ 1249 ] {} DiskLocal: Reserved 2.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:04.000803 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_398_176} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_397_175 to 202410_398_398_0 into Wide with storage Full 2024.10.22 01:14:04.041416 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_398_176} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:04.067571 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 39662 2024.10.22 01:14:04.153193 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_398_176} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_397_175, total 38532 rows starting from the beginning of the part 2024.10.22 01:14:04.282055 [ 1249 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 273 ms. 2024.10.22 01:14:04.357352 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_335_69 to 202410_1_335_69 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:04.367625 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_335_69} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_330_68, 202410_335_335_0] -> 202410_1_335_69 2024.10.22 01:14:04.440697 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_398_176} MergeTreeSequentialSource: Reading 2 marks from part 202410_398_398_0, total 199 rows starting from the beginning of the part 2024.10.22 01:14:04.492876 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.96 GiB, free memory in arenas 14.71 MiB, will set to 1.69 GiB (RSS), difference: -12.36 MiB 2024.10.22 01:14:04.527827 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:14:04.639492 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d6cb2375ee3fb5d08a6189f641032820) : [0, 216398]: Setting download as failed: File segment: [0, 216398], key: d6cb2375ee3fb5d08a6189f641032820, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:04.643337 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d6cb2375ee3fb5d08a6189f641032820): No space left in cache to reserve 216399 bytes, will continue without cache download 2024.10.22 01:14:04.680139 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d6cb2375ee3fb5d08a6189f641032820, offset: 0, size: 0 2024.10.22 01:14:04.681979 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d6cb2375ee3fb5d08a6189f641032820 for removal 2024.10.22 01:14:04.683691 [ 1476 ] {} CacheMetadata: Key d6cb2375ee3fb5d08a6189f641032820 is removed from metadata 2024.10.22 01:14:04.686560 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 49c51f9fac29e24755a4f3a611dde22b, offset: 0, size: 0 2024.10.22 01:14:04.688557 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 49c51f9fac29e24755a4f3a611dde22b for removal 2024.10.22 01:14:04.690769 [ 1476 ] {} CacheMetadata: Key 49c51f9fac29e24755a4f3a611dde22b is removed from metadata 2024.10.22 01:14:05.693347 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f68e5c1b7db86e24d64756872f654d46) : [0, 158]: Setting download as failed: File segment: [0, 158], key: f68e5c1b7db86e24d64756872f654d46, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:05.784894 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 6099 2024.10.22 01:14:05.798608 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f68e5c1b7db86e24d64756872f654d46): No space left in cache to reserve 159 bytes, will continue without cache download 2024.10.22 01:14:05.873495 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_64_13, 202410_65_65_0, 202410_66_66_0, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0] 2024.10.22 01:14:05.875285 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_64_13, 202410_65_65_0, 202410_66_66_0, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0] 2024.10.22 01:14:05.906309 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_64_13, 202410_65_65_0, 202410_66_66_0, 202410_67_67_0, 202410_68_68_0, 202410_69_69_0] 2024.10.22 01:14:06.141180 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:06.143663 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:06.204779 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_403_403_0 (state Temporary) 2024.10.22 01:14:06.271136 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_403_403_0 to 202410_416_416_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:06.331048 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6099 2024.10.22 01:14:06.402415 [ 1259 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_411_82 to 202410_416_416_0 2024.10.22 01:14:06.423051 [ 1259 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:06.426576 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:06.471513 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_411_82 to 202410_416_416_0 into Compact with storage Full 2024.10.22 01:14:06.529101 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:06.578069 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_411_82, total 6116 rows starting from the beginning of the part 2024.10.22 01:14:06.594131 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_412_412_0, total 28 rows starting from the beginning of the part 2024.10.22 01:14:06.649729 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_413_413_0, total 12 rows starting from the beginning of the part 2024.10.22 01:14:06.698187 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_414_414_0, total 15 rows starting from the beginning of the part 2024.10.22 01:14:06.739250 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.96 GiB, free memory in arenas 15.09 MiB, will set to 1.71 GiB (RSS), difference: -5.85 MiB 2024.10.22 01:14:06.780146 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 95 entries to flush up to offset 42767 2024.10.22 01:14:06.864286 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_415_415_0, total 14 rows starting from the beginning of the part 2024.10.22 01:14:06.888491 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_416_416_0, total 20 rows starting from the beginning of the part 2024.10.22 01:14:07.086944 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_692_310} MergeTask::MergeProjectionsStage: Merge sorted 43102 rows, containing 13 columns (13 merged, 0 gathered) in 4.284669111 sec., 10059.586605963234 rows/sec., 1.60 MiB/sec. 2024.10.22 01:14:07.104094 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_398_176} MergeTask::MergeProjectionsStage: Merge sorted 38731 rows, containing 13 columns (13 merged, 0 gathered) in 3.106621622 sec., 12467.240852803156 rows/sec., 5.38 MiB/sec. 2024.10.22 01:14:07.137396 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_692_310} MergedBlockOutputStream: filled checksums 202410_1_692_310 (state Temporary) 2024.10.22 01:14:07.176616 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6185ffbf5b440da257de3a28f6d7ebab) : [0, 346477]: Setting download as failed: File segment: [0, 346477], key: 6185ffbf5b440da257de3a28f6d7ebab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:07.180018 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6185ffbf5b440da257de3a28f6d7ebab): No space left in cache to reserve 346478 bytes, will continue without cache download 2024.10.22 01:14:07.221426 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_398_176} MergedBlockOutputStream: filled checksums 202410_1_398_176 (state Temporary) 2024.10.22 01:14:07.296258 [ 1254 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 217 ms. 2024.10.22 01:14:07.337733 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6185ffbf5b440da257de3a28f6d7ebab, offset: 0, size: 0 2024.10.22 01:14:07.341623 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6185ffbf5b440da257de3a28f6d7ebab for removal 2024.10.22 01:14:07.344462 [ 1476 ] {} CacheMetadata: Key 6185ffbf5b440da257de3a28f6d7ebab is removed from metadata 2024.10.22 01:14:07.352455 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f68e5c1b7db86e24d64756872f654d46, offset: 0, size: 0 2024.10.22 01:14:07.355924 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f68e5c1b7db86e24d64756872f654d46 for removal 2024.10.22 01:14:07.364075 [ 1476 ] {} CacheMetadata: Key f68e5c1b7db86e24d64756872f654d46 is removed from metadata 2024.10.22 01:14:07.408153 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_692_310} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_692_310 to 202410_1_692_310 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:07.412636 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_692_310} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_691_309, 202410_692_692_0] -> 202410_1_692_310 2024.10.22 01:14:07.548799 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:14:07.568993 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:07.571466 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:07.832506 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_685_685_0 (state Temporary) 2024.10.22 01:14:07.891934 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_685_685_0 to 202410_698_698_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:08.019460 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_692_310 to 202410_693_693_0 2024.10.22 01:14:08.025559 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.54 MiB using storage policy from min volume index 0 2024.10.22 01:14:08.038407 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42767 2024.10.22 01:14:08.045401 [ 1170 ] {} DiskLocal: Reserved 1.54 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:08.190762 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_693_311} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_692_310 to 202410_693_693_0 into Compact with storage Full 2024.10.22 01:14:08.269149 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_593_216, 202410_594_594_0] 2024.10.22 01:14:08.272515 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_593_216, 202410_594_594_0] 2024.10.22 01:14:08.280513 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_593_216, 202410_594_594_0] 2024.10.22 01:14:08.302872 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_398_176} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_398_176 to 202410_1_398_176 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:08.306336 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_398_176} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_397_175, 202410_398_398_0] -> 202410_1_398_176 2024.10.22 01:14:08.324757 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.96 MiB. 2024.10.22 01:14:08.351396 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_693_311} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:08.358521 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_693_311} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_692_310, total 43102 rows starting from the beginning of the part 2024.10.22 01:14:08.372250 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_693_311} MergeTreeSequentialSource: Reading 2 marks from part 202410_693_693_0, total 75 rows starting from the beginning of the part 2024.10.22 01:14:08.374531 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} MergeTask::MergeProjectionsStage: Merge sorted 6205 rows, containing 12 columns (12 merged, 0 gathered) in 1.932563994 sec., 3210.760429804427 rows/sec., 508.06 KiB/sec. 2024.10.22 01:14:08.436894 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} MergedBlockOutputStream: filled checksums 202410_1_416_83 (state Temporary) 2024.10.22 01:14:08.465087 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 36 entries to flush up to offset 3267 2024.10.22 01:14:08.479321 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(13855ed9038352add74b500cba19676c) : [0, 150]: Setting download as failed: File segment: [0, 150], key: 13855ed9038352add74b500cba19676c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:08.480826 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(13855ed9038352add74b500cba19676c): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:14:08.506242 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_416_83 to 202410_1_416_83 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:08.512290 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_416_83} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_411_82, 202410_416_416_0] -> 202410_1_416_83 2024.10.22 01:14:08.556798 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.44 MiB. 2024.10.22 01:14:09.280319 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e819cc482f75d9acfcb05926a97cd86d) : [0, 5127]: Setting download as failed: File segment: [0, 5127], key: e819cc482f75d9acfcb05926a97cd86d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:09.282339 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e819cc482f75d9acfcb05926a97cd86d): No space left in cache to reserve 5128 bytes, will continue without cache download 2024.10.22 01:14:09.292619 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e819cc482f75d9acfcb05926a97cd86d, offset: 0, size: 0 2024.10.22 01:14:09.304622 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e819cc482f75d9acfcb05926a97cd86d for removal 2024.10.22 01:14:09.307753 [ 1476 ] {} CacheMetadata: Key e819cc482f75d9acfcb05926a97cd86d is removed from metadata 2024.10.22 01:14:09.310776 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 13855ed9038352add74b500cba19676c, offset: 0, size: 0 2024.10.22 01:14:09.315632 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 13855ed9038352add74b500cba19676c for removal 2024.10.22 01:14:09.327979 [ 1476 ] {} CacheMetadata: Key 13855ed9038352add74b500cba19676c is removed from metadata 2024.10.22 01:14:09.578643 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.96 GiB, free memory in arenas 10.24 MiB, will set to 1.69 GiB (RSS), difference: 13.65 MiB 2024.10.22 01:14:09.753556 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 619192 2024.10.22 01:14:09.918739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b8899c8afd67b44fcbb6624ee4028164) : [0, 168]: Setting download as failed: File segment: [0, 168], key: b8899c8afd67b44fcbb6624ee4028164, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:09.920356 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b8899c8afd67b44fcbb6624ee4028164): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:14:10.040213 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:14:10.073650 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:14:10.281766 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} MergeTask::MergeProjectionsStage: Merge sorted 629710 rows, containing 4 columns (4 merged, 0 gathered) in 23.512149074 sec., 26782.324236636472 rows/sec., 392.59 KiB/sec. 2024.10.22 01:14:10.310916 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} MergedBlockOutputStream: filled checksums 202410_1_391_78 (state Temporary) 2024.10.22 01:14:10.335772 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:10.339152 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:10.354822 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_391_78 to 202410_1_391_78 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:10.359231 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_391_78} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_386_77, 202410_391_391_0] -> 202410_1_391_78 2024.10.22 01:14:10.388013 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:14:10.545396 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_387_387_0 (state Temporary) 2024.10.22 01:14:10.585173 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(34d10cc0ee7bcae63deef79e6f88d1ab) : [0, 163702]: Setting download as failed: File segment: [0, 163702], key: 34d10cc0ee7bcae63deef79e6f88d1ab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:10.586542 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(34d10cc0ee7bcae63deef79e6f88d1ab): No space left in cache to reserve 163703 bytes, will continue without cache download 2024.10.22 01:14:10.602669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 34d10cc0ee7bcae63deef79e6f88d1ab, offset: 0, size: 0 2024.10.22 01:14:10.603878 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 34d10cc0ee7bcae63deef79e6f88d1ab for removal 2024.10.22 01:14:10.615969 [ 1476 ] {} CacheMetadata: Key 34d10cc0ee7bcae63deef79e6f88d1ab is removed from metadata 2024.10.22 01:14:10.617544 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b8899c8afd67b44fcbb6624ee4028164, offset: 0, size: 0 2024.10.22 01:14:10.620809 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b8899c8afd67b44fcbb6624ee4028164 for removal 2024.10.22 01:14:10.624830 [ 1476 ] {} CacheMetadata: Key b8899c8afd67b44fcbb6624ee4028164 is removed from metadata 2024.10.22 01:14:10.637346 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_387_387_0 to 202410_394_394_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:10.850453 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 619192 2024.10.22 01:14:11.371114 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.96 GiB, free memory in arenas 11.04 MiB, will set to 1.69 GiB (RSS), difference: 18.48 MiB 2024.10.22 01:14:11.372457 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_693_311} MergeTask::MergeProjectionsStage: Merge sorted 43177 rows, containing 13 columns (13 merged, 0 gathered) in 3.281402764 sec., 13158.092165244485 rows/sec., 2.09 MiB/sec. 2024.10.22 01:14:11.380100 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b0b74765ca0cf6a31a4b9ccaabedd56a) : [0, 157]: Setting download as failed: File segment: [0, 157], key: b0b74765ca0cf6a31a4b9ccaabedd56a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:11.382935 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b0b74765ca0cf6a31a4b9ccaabedd56a): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:14:11.441566 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_693_311} MergedBlockOutputStream: filled checksums 202410_1_693_311 (state Temporary) 2024.10.22 01:14:11.597053 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_340_118, 202410_341_341_0] 2024.10.22 01:14:11.602860 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 231 entries to flush up to offset 39893 2024.10.22 01:14:11.602675 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_340_118, 202410_341_341_0] 2024.10.22 01:14:11.725175 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_340_118, 202410_341_341_0] 2024.10.22 01:14:11.882623 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_693_311} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_693_311 to 202410_1_693_311 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:11.887272 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_693_311} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_692_310, 202410_693_693_0] -> 202410_1_693_311 2024.10.22 01:14:11.980193 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:11.983917 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:12.016313 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:14:12.023322 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_398_398_0 (state Temporary) 2024.10.22 01:14:12.094554 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_398_398_0 to 202410_404_404_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:12.172066 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 42859 2024.10.22 01:14:12.183894 [ 1196 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_398_176 to 202410_399_399_0 2024.10.22 01:14:12.211721 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 39893 2024.10.22 01:14:12.211804 [ 1196 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.01 MiB using storage policy from min volume index 0 2024.10.22 01:14:12.214354 [ 1196 ] {} DiskLocal: Reserved 2.01 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:12.269803 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(f8e3548932750f4cdc5bc7fcf3152172) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f8e3548932750f4cdc5bc7fcf3152172, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:14:12.274085 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 793, file segment info: File segment: [0, 33554431], key: f8e3548932750f4cdc5bc7fcf3152172, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:14:12.277263 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:12.313346 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:12.318894 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:12.332581 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_399_177} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_398_176 to 202410_399_399_0 into Wide with storage Full 2024.10.22 01:14:12.380190 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_686_686_0 (state Temporary) 2024.10.22 01:14:12.437384 [ 1730 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/veu/tadkuihrcrctknzypdbunepmsvrpi, size 2050586 2024.10.22 01:14:12.451193 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_282_58, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0, 202410_286_286_0] 2024.10.22 01:14:12.454883 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_282_58, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0, 202410_286_286_0] 2024.10.22 01:14:12.464545 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_686_686_0 to 202410_699_699_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:12.471712 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_399_177} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:12.478404 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_399_177} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_398_176, total 38731 rows starting from the beginning of the part 2024.10.22 01:14:12.499472 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_693_311 to 202410_694_694_0 2024.10.22 01:14:12.507070 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.54 MiB using storage policy from min volume index 0 2024.10.22 01:14:12.514327 [ 1201 ] {} DiskLocal: Reserved 1.54 MiB on local disk `default`, having unreserved 297.28 GiB. 2024.10.22 01:14:12.538857 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_694_312} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_693_311 to 202410_694_694_0 into Compact with storage Full 2024.10.22 01:14:12.563996 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42859 2024.10.22 01:14:12.586466 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_282_58, 202410_283_283_0, 202410_284_284_0, 202410_285_285_0, 202410_286_286_0] 2024.10.22 01:14:12.591946 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_694_312} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:12.600903 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_694_312} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_693_311, total 43177 rows starting from the beginning of the part 2024.10.22 01:14:12.676332 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:14:12.676987 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_399_177} MergeTreeSequentialSource: Reading 2 marks from part 202410_399_399_0, total 221 rows starting from the beginning of the part 2024.10.22 01:14:12.682427 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:14:12.687523 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:14:12.722726 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:14:12.736666 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:14:12.761601 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:14:12.769954 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:14:12.777888 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:14:12.795763 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:14:12.916790 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_694_312} MergeTreeSequentialSource: Reading 2 marks from part 202410_694_694_0, total 95 rows starting from the beginning of the part 2024.10.22 01:14:13.063859 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6ea4a7fb7351a141c352ce7a72ac045f) : [0, 293645]: Setting download as failed: File segment: [0, 293645], key: 6ea4a7fb7351a141c352ce7a72ac045f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:13.065401 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6ea4a7fb7351a141c352ce7a72ac045f): No space left in cache to reserve 293646 bytes, will continue without cache download 2024.10.22 01:14:13.096005 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6ea4a7fb7351a141c352ce7a72ac045f, offset: 0, size: 0 2024.10.22 01:14:13.099823 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6ea4a7fb7351a141c352ce7a72ac045f for removal 2024.10.22 01:14:13.102325 [ 1476 ] {} CacheMetadata: Key 6ea4a7fb7351a141c352ce7a72ac045f is removed from metadata 2024.10.22 01:14:13.129326 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/sfr/gnwlsiznjxglqzhuudamjfcgosrpy (LogSeriesLimiter: on interval from 2024-10-22 01:13:53 to 2024-10-22 01:14:13 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:14:13.205718 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b0b74765ca0cf6a31a4b9ccaabedd56a, offset: 0, size: 0 2024.10.22 01:14:13.207581 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b0b74765ca0cf6a31a4b9ccaabedd56a for removal 2024.10.22 01:14:13.210576 [ 1476 ] {} CacheMetadata: Key b0b74765ca0cf6a31a4b9ccaabedd56a is removed from metadata 2024.10.22 01:14:13.246433 [ 1209 ] {} DatabaseCatalog: Execution took 241 ms. 2024.10.22 01:14:13.332304 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:13.379487 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_594_217, 202410_595_595_0] 2024.10.22 01:14:13.383340 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_594_217, 202410_595_595_0] 2024.10.22 01:14:13.393051 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_594_217, 202410_595_595_0] 2024.10.22 01:14:13.833413 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 6127 2024.10.22 01:14:13.874979 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(604ff3cb7e63a51db633051e55318cb3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 604ff3cb7e63a51db633051e55318cb3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 01:14:13.987490 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 990, file segment info: File segment: [0, 33554431], key: 604ff3cb7e63a51db633051e55318cb3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 01:14:13.990804 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:14.055367 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_399_177/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:14:14.259797 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:14.265462 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:14.300429 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_404_404_0 (state Temporary) 2024.10.22 01:14:14.346406 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_404_404_0 to 202410_417_417_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:14.439036 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.96 GiB, free memory in arenas 6.41 MiB, will set to 1.70 GiB (RSS), difference: -43.97 MiB 2024.10.22 01:14:14.494006 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6127 2024.10.22 01:14:14.544664 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column URLHash 2024.10.22 01:14:14.555794 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column URLHash 2024.10.22 01:14:14.567477 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column URLHash 2024.10.22 01:14:14.698975 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column URLHash 2024.10.22 01:14:14.703795 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column URLHash 2024.10.22 01:14:14.715083 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column URLHash 2024.10.22 01:14:14.736716 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column URLHash 2024.10.22 01:14:14.768286 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1ea84d5b476c2d9d89412e1b01468abd) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 1ea84d5b476c2d9d89412e1b01468abd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:14.774856 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1ea84d5b476c2d9d89412e1b01468abd): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:14:14.890389 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_79_16, total 3089 rows starting from the beginning of the part 2024.10.22 01:14:15.265337 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_399_177} MergeTask::MergeProjectionsStage: Merge sorted 38952 rows, containing 13 columns (13 merged, 0 gathered) in 2.940715591 sec., 13245.75559745111 rows/sec., 5.72 MiB/sec. 2024.10.22 01:14:15.463491 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_399_177} MergedBlockOutputStream: filled checksums 202410_1_399_177 (state Temporary) 2024.10.22 01:14:15.696325 [ 1278 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 248 ms. 2024.10.22 01:14:15.804264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9dce59cc0c01d0838c7755ec2ac031be) : [0, 287444]: Setting download as failed: File segment: [0, 287444], key: 9dce59cc0c01d0838c7755ec2ac031be, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:15.806226 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9dce59cc0c01d0838c7755ec2ac031be): No space left in cache to reserve 287445 bytes, will continue without cache download 2024.10.22 01:14:15.954636 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9dce59cc0c01d0838c7755ec2ac031be, offset: 0, size: 0 2024.10.22 01:14:15.961100 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9dce59cc0c01d0838c7755ec2ac031be for removal 2024.10.22 01:14:15.967905 [ 1476 ] {} CacheMetadata: Key 9dce59cc0c01d0838c7755ec2ac031be is removed from metadata 2024.10.22 01:14:15.970730 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1ea84d5b476c2d9d89412e1b01468abd, offset: 0, size: 0 2024.10.22 01:14:15.972338 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1ea84d5b476c2d9d89412e1b01468abd for removal 2024.10.22 01:14:15.975197 [ 1476 ] {} CacheMetadata: Key 1ea84d5b476c2d9d89412e1b01468abd is removed from metadata 2024.10.22 01:14:16.171980 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 344 2024.10.22 01:14:16.390529 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_399_177} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_399_177 to 202410_1_399_177 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:16.407199 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:16.413321 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_399_177} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_398_176, 202410_399_399_0] -> 202410_1_399_177 2024.10.22 01:14:16.413806 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:16.443176 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.96 MiB. 2024.10.22 01:14:16.486169 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_326_326_0 (state Temporary) 2024.10.22 01:14:16.536560 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_694_312} MergeTask::MergeProjectionsStage: Merge sorted 43272 rows, containing 13 columns (13 merged, 0 gathered) in 4.007657575 sec., 10797.329659583005 rows/sec., 1.72 MiB/sec. 2024.10.22 01:14:16.562642 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.96 GiB, free memory in arenas 3.98 MiB, will set to 1.68 GiB (RSS), difference: 18.85 MiB 2024.10.22 01:14:16.578924 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 42952 2024.10.22 01:14:16.624084 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_694_312} MergedBlockOutputStream: filled checksums 202410_1_694_312 (state Temporary) 2024.10.22 01:14:16.784914 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_326_326_0 to 202410_336_336_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:16.860841 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 344 2024.10.22 01:14:16.874616 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_694_312} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_694_312 to 202410_1_694_312 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:16.879437 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_694_312} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_693_311, 202410_694_694_0] -> 202410_1_694_312 2024.10.22 01:14:16.910585 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:14:17.040486 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:17.045134 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:17.050575 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1d81b12e3212f80e99105e2811bd195e) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 1d81b12e3212f80e99105e2811bd195e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:17.054585 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1d81b12e3212f80e99105e2811bd195e): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:14:17.149273 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_687_687_0 (state Temporary) 2024.10.22 01:14:17.268489 [ 1150 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:14:17.292787 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_687_687_0 to 202410_700_700_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:17.300576 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.173769535 sec. 2024.10.22 01:14:17.300645 [ 1150 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:14:17.325102 [ 1275 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_694_312 to 202410_695_695_0 2024.10.22 01:14:17.336947 [ 1275 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.55 MiB using storage policy from min volume index 0 2024.10.22 01:14:17.342618 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 42952 2024.10.22 01:14:17.343321 [ 1275 ] {} DiskLocal: Reserved 1.55 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:17.379670 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_80_80_0, total 39 rows starting from the beginning of the part 2024.10.22 01:14:17.387641 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_695_313} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_694_312 to 202410_695_695_0 into Compact with storage Full 2024.10.22 01:14:17.429149 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_695_313} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:17.462196 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_695_313} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_694_312, total 43272 rows starting from the beginning of the part 2024.10.22 01:14:17.512148 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_695_313} MergeTreeSequentialSource: Reading 2 marks from part 202410_695_695_0, total 62 rows starting from the beginning of the part 2024.10.22 01:14:17.855899 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 620728 2024.10.22 01:14:17.904467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4ebfd076310ee21eec0c45763c58f856) : [0, 326558]: Setting download as failed: File segment: [0, 326558], key: 4ebfd076310ee21eec0c45763c58f856, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:17.907729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4ebfd076310ee21eec0c45763c58f856): No space left in cache to reserve 326559 bytes, will continue without cache download 2024.10.22 01:14:17.977563 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4ebfd076310ee21eec0c45763c58f856, offset: 0, size: 0 2024.10.22 01:14:17.980011 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4ebfd076310ee21eec0c45763c58f856 for removal 2024.10.22 01:14:17.982763 [ 1476 ] {} CacheMetadata: Key 4ebfd076310ee21eec0c45763c58f856 is removed from metadata 2024.10.22 01:14:17.984739 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1d81b12e3212f80e99105e2811bd195e, offset: 0, size: 0 2024.10.22 01:14:17.991061 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1d81b12e3212f80e99105e2811bd195e for removal 2024.10.22 01:14:17.995230 [ 1476 ] {} CacheMetadata: Key 1d81b12e3212f80e99105e2811bd195e is removed from metadata 2024.10.22 01:14:18.703297 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.96 GiB, free memory in arenas 3.50 MiB, will set to 1.68 GiB (RSS), difference: -4.12 MiB 2024.10.22 01:14:18.746021 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:18.762369 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:18.871906 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_341_119, 202410_342_342_0] 2024.10.22 01:14:18.875287 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_341_119, 202410_342_342_0] 2024.10.22 01:14:18.888525 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_341_119, 202410_342_342_0] 2024.10.22 01:14:19.078655 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_388_388_0 (state Temporary) 2024.10.22 01:14:19.126249 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_595_218, 202410_596_596_0] 2024.10.22 01:14:19.130957 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_595_218, 202410_596_596_0] 2024.10.22 01:14:19.159396 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_595_218, 202410_596_596_0] 2024.10.22 01:14:19.175420 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_388_388_0 to 202410_395_395_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:19.179221 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(86cd13c3e4fd8527fd607d8d4350cdce) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 86cd13c3e4fd8527fd607d8d4350cdce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:19.181346 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(86cd13c3e4fd8527fd607d8d4350cdce): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:14:19.214484 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 620728 2024.10.22 01:14:19.750993 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 224 entries to flush up to offset 40117 2024.10.22 01:14:19.855539 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_81_81_0, total 42 rows starting from the beginning of the part 2024.10.22 01:14:19.995698 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:19.999939 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:20.040869 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:14:20.056978 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:14:20.083831 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_399_399_0 (state Temporary) 2024.10.22 01:14:20.323083 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(42ced4d444b695630fece2b8964cdb67) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 42ced4d444b695630fece2b8964cdb67, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:14:20.352857 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 42ced4d444b695630fece2b8964cdb67, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:14:20.355879 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:20.356733 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_399_399_0 to 202410_405_405_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:20.391659 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 40117 2024.10.22 01:14:20.489158 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_695_313} MergeTask::MergeProjectionsStage: Merge sorted 43334 rows, containing 13 columns (13 merged, 0 gathered) in 3.10836409 sec., 13941.095298138 rows/sec., 2.22 MiB/sec. 2024.10.22 01:14:20.540911 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_695_313} MergedBlockOutputStream: filled checksums 202410_1_695_313 (state Temporary) 2024.10.22 01:14:20.662708 [ 1223 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_399_177 to 202410_400_400_0 2024.10.22 01:14:20.668481 [ 1223 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.02 MiB using storage policy from min volume index 0 2024.10.22 01:14:20.672213 [ 1223 ] {} DiskLocal: Reserved 2.02 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:20.677657 [ 1223 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 306 ms. 2024.10.22 01:14:20.683973 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_400_178} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_399_177 to 202410_400_400_0 into Wide with storage Full 2024.10.22 01:14:20.714728 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_400_178} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:20.735374 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_400_178} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_399_177, total 38952 rows starting from the beginning of the part 2024.10.22 01:14:20.901002 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_695_313} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_695_313 to 202410_1_695_313 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:20.904308 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_695_313} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_694_312, 202410_695_695_0] -> 202410_1_695_313 2024.10.22 01:14:20.942366 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_400_178} MergeTreeSequentialSource: Reading 2 marks from part 202410_400_400_0, total 173 rows starting from the beginning of the part 2024.10.22 01:14:21.071585 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:14:21.089605 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9e0f9089c45e2e01d7fab3ea5213973e) : [0, 516717]: Setting download as failed: File segment: [0, 516717], key: 9e0f9089c45e2e01d7fab3ea5213973e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:21.103160 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9e0f9089c45e2e01d7fab3ea5213973e): No space left in cache to reserve 516718 bytes, will continue without cache download 2024.10.22 01:14:21.141005 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9e0f9089c45e2e01d7fab3ea5213973e, offset: 0, size: 0 2024.10.22 01:14:21.143180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9e0f9089c45e2e01d7fab3ea5213973e for removal 2024.10.22 01:14:21.146581 [ 1476 ] {} CacheMetadata: Key 9e0f9089c45e2e01d7fab3ea5213973e is removed from metadata 2024.10.22 01:14:21.153147 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 86cd13c3e4fd8527fd607d8d4350cdce, offset: 0, size: 0 2024.10.22 01:14:21.155288 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 86cd13c3e4fd8527fd607d8d4350cdce for removal 2024.10.22 01:14:21.263953 [ 1476 ] {} CacheMetadata: Key 86cd13c3e4fd8527fd607d8d4350cdce is removed from metadata 2024.10.22 01:14:21.353550 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 43033 2024.10.22 01:14:21.501839 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:21.503190 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:21.544339 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_82_82_0, total 42 rows starting from the beginning of the part 2024.10.22 01:14:21.560600 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_688_688_0 (state Temporary) 2024.10.22 01:14:21.604404 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_688_688_0 to 202410_701_701_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:21.647053 [ 1228 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_695_313 to 202410_696_696_0 2024.10.22 01:14:21.666154 [ 1228 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.55 MiB using storage policy from min volume index 0 2024.10.22 01:14:21.675108 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43033 2024.10.22 01:14:21.780001 [ 1228 ] {} DiskLocal: Reserved 1.55 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:21.884247 [ 1228 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 258 ms. 2024.10.22 01:14:21.905299 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_696_314} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_695_313 to 202410_696_696_0 into Compact with storage Full 2024.10.22 01:14:21.957096 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_696_314} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:21.979010 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_696_314} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_695_313, total 43334 rows starting from the beginning of the part 2024.10.22 01:14:21.979296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(10ae33e3d7fd1007b03b43005b057d26) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 10ae33e3d7fd1007b03b43005b057d26, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:21.980910 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(10ae33e3d7fd1007b03b43005b057d26): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:14:21.996334 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 6142 2024.10.22 01:14:22.025716 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_696_314} MergeTreeSequentialSource: Reading 2 marks from part 202410_696_696_0, total 84 rows starting from the beginning of the part 2024.10.22 01:14:22.101355 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:22.105338 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:22.148798 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_405_405_0 (state Temporary) 2024.10.22 01:14:22.399249 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.96 GiB, free memory in arenas 4.14 MiB, will set to 1.69 GiB (RSS), difference: -55.31 MiB 2024.10.22 01:14:22.455073 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_405_405_0 to 202410_418_418_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:22.491624 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6142 2024.10.22 01:14:22.521714 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:22.526897 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:22.892578 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(43046fa1481d6d8d922ae4adefe38b2f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 43046fa1481d6d8d922ae4adefe38b2f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:14:22.910297 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 43046fa1481d6d8d922ae4adefe38b2f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:14:22.914187 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:23.204786 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_596_219, 202410_597_597_0] 2024.10.22 01:14:23.215372 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_596_219, 202410_597_597_0] 2024.10.22 01:14:23.327927 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_596_219, 202410_597_597_0] 2024.10.22 01:14:23.465102 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_83_83_0, total 41 rows starting from the beginning of the part 2024.10.22 01:14:23.592128 [ 1227 ] {} DatabaseCatalog: Execution took 243 ms. 2024.10.22 01:14:23.725214 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_400_178} MergeTask::MergeProjectionsStage: Merge sorted 39125 rows, containing 13 columns (13 merged, 0 gathered) in 3.046374493 sec., 12843.135369568628 rows/sec., 5.54 MiB/sec. 2024.10.22 01:14:23.731357 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8d7f595967df34e49a697fb73c421b5a) : [0, 485749]: Setting download as failed: File segment: [0, 485749], key: 8d7f595967df34e49a697fb73c421b5a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:23.734507 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8d7f595967df34e49a697fb73c421b5a): No space left in cache to reserve 485750 bytes, will continue without cache download 2024.10.22 01:14:23.784033 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_400_178} MergedBlockOutputStream: filled checksums 202410_1_400_178 (state Temporary) 2024.10.22 01:14:23.879040 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8d7f595967df34e49a697fb73c421b5a, offset: 0, size: 0 2024.10.22 01:14:23.880933 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8d7f595967df34e49a697fb73c421b5a for removal 2024.10.22 01:14:23.885022 [ 1476 ] {} CacheMetadata: Key 8d7f595967df34e49a697fb73c421b5a is removed from metadata 2024.10.22 01:14:23.889429 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 10ae33e3d7fd1007b03b43005b057d26, offset: 0, size: 0 2024.10.22 01:14:23.891271 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 10ae33e3d7fd1007b03b43005b057d26 for removal 2024.10.22 01:14:23.896892 [ 1476 ] {} CacheMetadata: Key 10ae33e3d7fd1007b03b43005b057d26 is removed from metadata 2024.10.22 01:14:24.227332 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_696_314} MergeTask::MergeProjectionsStage: Merge sorted 43418 rows, containing 13 columns (13 merged, 0 gathered) in 2.329137851 sec., 18641.232411966845 rows/sec., 2.96 MiB/sec. 2024.10.22 01:14:24.365063 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 345 2024.10.22 01:14:24.389288 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_696_314} MergedBlockOutputStream: filled checksums 202410_1_696_314 (state Temporary) 2024.10.22 01:14:24.474609 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_696_314} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_696_314 to 202410_1_696_314 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:24.479147 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_696_314} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_695_313, 202410_696_696_0] -> 202410_1_696_314 2024.10.22 01:14:24.482679 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_400_178} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_400_178 to 202410_1_400_178 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:24.487080 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_400_178} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_399_177, 202410_400_400_0] -> 202410_1_400_178 2024.10.22 01:14:24.687170 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.90 MiB. 2024.10.22 01:14:24.695301 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:14:24.873274 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:24.876142 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:24.894350 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_84_84_0, total 40 rows starting from the beginning of the part 2024.10.22 01:14:24.921259 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(41a31dfd5f583f55646c3fdf212fa3bf) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 41a31dfd5f583f55646c3fdf212fa3bf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:24.923763 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(41a31dfd5f583f55646c3fdf212fa3bf): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:14:25.041304 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_327_327_0 (state Temporary) 2024.10.22 01:14:25.206154 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_327_327_0 to 202410_337_337_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:25.286445 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 345 2024.10.22 01:14:25.376973 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_82_82_0 (state Temporary) 2024.10.22 01:14:25.382787 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.96 GiB, free memory in arenas 2.85 MiB, will set to 1.68 GiB (RSS), difference: 58.86 MiB 2024.10.22 01:14:25.687617 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 43096 2024.10.22 01:14:25.851365 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:25.858916 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:25.939933 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c7da7020144fee47f107dcae05842847) : [0, 315326]: Setting download as failed: File segment: [0, 315326], key: c7da7020144fee47f107dcae05842847, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:25.943631 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c7da7020144fee47f107dcae05842847): No space left in cache to reserve 315327 bytes, will continue without cache download 2024.10.22 01:14:25.996794 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c7da7020144fee47f107dcae05842847, offset: 0, size: 0 2024.10.22 01:14:25.999441 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c7da7020144fee47f107dcae05842847 for removal 2024.10.22 01:14:26.004787 [ 1476 ] {} CacheMetadata: Key c7da7020144fee47f107dcae05842847 is removed from metadata 2024.10.22 01:14:26.007094 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 41a31dfd5f583f55646c3fdf212fa3bf, offset: 0, size: 0 2024.10.22 01:14:26.012458 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 41a31dfd5f583f55646c3fdf212fa3bf for removal 2024.10.22 01:14:26.022727 [ 1476 ] {} CacheMetadata: Key 41a31dfd5f583f55646c3fdf212fa3bf is removed from metadata 2024.10.22 01:14:26.129403 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_689_689_0 (state Temporary) 2024.10.22 01:14:26.171361 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_597_220, 202410_598_598_0] 2024.10.22 01:14:26.174349 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_597_220, 202410_598_598_0] 2024.10.22 01:14:26.178507 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_689_689_0 to 202410_702_702_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:26.191669 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_597_220, 202410_598_598_0] 2024.10.22 01:14:26.203271 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_696_314 to 202410_697_697_0 2024.10.22 01:14:26.212569 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.55 MiB using storage policy from min volume index 0 2024.10.22 01:14:26.221696 [ 1172 ] {} DiskLocal: Reserved 1.55 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:26.222815 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 799 entries to flush up to offset 621527 2024.10.22 01:14:26.341600 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43096 2024.10.22 01:14:26.460361 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_697_315} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_696_314 to 202410_697_697_0 into Compact with storage Full 2024.10.22 01:14:26.627149 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_697_315} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:26.673388 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(67c3eeb678c4cfccad1c30de408ef806) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 67c3eeb678c4cfccad1c30de408ef806, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:26.675728 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(67c3eeb678c4cfccad1c30de408ef806): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:14:26.775200 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_697_315} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_696_314, total 43418 rows starting from the beginning of the part 2024.10.22 01:14:26.824936 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_697_315} MergeTreeSequentialSource: Reading 2 marks from part 202410_697_697_0, total 85 rows starting from the beginning of the part 2024.10.22 01:14:27.271581 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:27.275362 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:27.435895 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_389_389_0 (state Temporary) 2024.10.22 01:14:27.477066 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.96 GiB, free memory in arenas 3.51 MiB, will set to 1.68 GiB (RSS), difference: -23.31 MiB 2024.10.22 01:14:27.663096 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_389_389_0 to 202410_396_396_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:27.674573 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_342_120, 202410_343_343_0] 2024.10.22 01:14:27.677566 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_342_120, 202410_343_343_0] 2024.10.22 01:14:27.695883 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(236181d074403d42627778ae540f7d74) : [0, 264559]: Setting download as failed: File segment: [0, 264559], key: 236181d074403d42627778ae540f7d74, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:27.696508 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 621527 2024.10.22 01:14:27.698923 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(236181d074403d42627778ae540f7d74): No space left in cache to reserve 264560 bytes, will continue without cache download 2024.10.22 01:14:27.701924 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_391_78 to 202410_396_396_0 2024.10.22 01:14:27.706407 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_342_120, 202410_343_343_0] 2024.10.22 01:14:27.720421 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:27.724794 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:27.768707 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_391_78 to 202410_396_396_0 into Compact with storage Full 2024.10.22 01:14:27.813445 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:27.824321 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} MergeTreeSequentialSource: Reading 78 marks from part 202410_1_391_78, total 629710 rows starting from the beginning of the part 2024.10.22 01:14:27.839898 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_392_392_0, total 768 rows starting from the beginning of the part 2024.10.22 01:14:27.867704 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_393_393_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:14:27.880823 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_394_394_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:14:27.906108 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_395_395_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:14:27.932185 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 236181d074403d42627778ae540f7d74, offset: 0, size: 0 2024.10.22 01:14:27.934883 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 236181d074403d42627778ae540f7d74 for removal 2024.10.22 01:14:27.938127 [ 1476 ] {} CacheMetadata: Key 236181d074403d42627778ae540f7d74 is removed from metadata 2024.10.22 01:14:27.941391 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 67c3eeb678c4cfccad1c30de408ef806, offset: 0, size: 0 2024.10.22 01:14:28.026080 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_396_396_0, total 799 rows starting from the beginning of the part 2024.10.22 01:14:28.044033 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 67c3eeb678c4cfccad1c30de408ef806 for removal 2024.10.22 01:14:28.049176 [ 1476 ] {} CacheMetadata: Key 67c3eeb678c4cfccad1c30de408ef806 is removed from metadata 2024.10.22 01:14:28.172214 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 220 entries to flush up to offset 40337 2024.10.22 01:14:28.481449 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:28.489019 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:28.564119 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_400_400_0 (state Temporary) 2024.10.22 01:14:28.631411 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_400_400_0 to 202410_406_406_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:28.681829 [ 1241 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_400_178 to 202410_401_401_0 2024.10.22 01:14:28.704265 [ 1241 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.02 MiB using storage policy from min volume index 0 2024.10.22 01:14:28.710478 [ 1241 ] {} DiskLocal: Reserved 2.02 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:28.730575 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_401_179} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_400_178 to 202410_401_401_0 into Wide with storage Full 2024.10.22 01:14:28.748458 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_401_179} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:28.764459 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_401_179} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_400_178, total 39125 rows starting from the beginning of the part 2024.10.22 01:14:28.777695 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 40337 2024.10.22 01:14:28.825682 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_697_315/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:14:29.143781 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_401_179} MergeTreeSequentialSource: Reading 2 marks from part 202410_401_401_0, total 200 rows starting from the beginning of the part 2024.10.22 01:14:29.178747 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_331_66, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0, 202410_336_336_0] 2024.10.22 01:14:29.191160 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_331_66, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0, 202410_336_336_0] 2024.10.22 01:14:29.320832 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8f9d6cebfbda488f84c2fbb692c670ef) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 8f9d6cebfbda488f84c2fbb692c670ef, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:29.322609 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8f9d6cebfbda488f84c2fbb692c670ef): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:14:29.376907 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_331_66, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0, 202410_336_336_0] 2024.10.22 01:14:29.621013 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.96 GiB, free memory in arenas 3.29 MiB, will set to 1.69 GiB (RSS), difference: -27.86 MiB 2024.10.22 01:14:29.916243 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_697_315} MergeTask::MergeProjectionsStage: Merge sorted 43503 rows, containing 13 columns (13 merged, 0 gathered) in 3.46700164 sec., 12547.729859164416 rows/sec., 2.00 MiB/sec. 2024.10.22 01:14:29.994757 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 6153 2024.10.22 01:14:30.042334 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:14:30.052816 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:14:30.087207 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_697_315} MergedBlockOutputStream: filled checksums 202410_1_697_315 (state Temporary) 2024.10.22 01:14:30.171933 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_697_315} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_697_315 to 202410_1_697_315 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:30.176051 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_697_315} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_696_314, 202410_697_697_0] -> 202410_1_697_315 2024.10.22 01:14:30.195633 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:14:30.282687 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:30.284677 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:30.361096 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 43181 2024.10.22 01:14:30.435464 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_406_406_0 (state Temporary) 2024.10.22 01:14:30.501190 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_406_406_0 to 202410_419_419_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:30.560901 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6b3671728b2f9aabfa5ef365856509a7) : [0, 495459]: Setting download as failed: File segment: [0, 495459], key: 6b3671728b2f9aabfa5ef365856509a7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:30.564418 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6b3671728b2f9aabfa5ef365856509a7): No space left in cache to reserve 495460 bytes, will continue without cache download 2024.10.22 01:14:30.570246 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6153 2024.10.22 01:14:30.594653 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_401_179/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:14:30.597833 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6b3671728b2f9aabfa5ef365856509a7, offset: 0, size: 0 2024.10.22 01:14:30.604321 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6b3671728b2f9aabfa5ef365856509a7 for removal 2024.10.22 01:14:30.608815 [ 1476 ] {} CacheMetadata: Key 6b3671728b2f9aabfa5ef365856509a7 is removed from metadata 2024.10.22 01:14:30.618183 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8f9d6cebfbda488f84c2fbb692c670ef, offset: 0, size: 0 2024.10.22 01:14:30.621235 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8f9d6cebfbda488f84c2fbb692c670ef for removal 2024.10.22 01:14:30.624176 [ 1476 ] {} CacheMetadata: Key 8f9d6cebfbda488f84c2fbb692c670ef is removed from metadata 2024.10.22 01:14:30.700123 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:30.706394 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:30.802159 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_690_690_0 (state Temporary) 2024.10.22 01:14:30.903936 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_690_690_0 to 202410_703_703_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:31.038331 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 01:14:31.085090 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43181 2024.10.22 01:14:31.087258 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_697_315 to 202410_698_698_0 2024.10.22 01:14:31.102232 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_84_17/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:14:31.110956 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.56 MiB using storage policy from min volume index 0 2024.10.22 01:14:31.119288 [ 1184 ] {} DiskLocal: Reserved 1.56 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:31.171682 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_698_316} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_697_315 to 202410_698_698_0 into Compact with storage Full 2024.10.22 01:14:31.191326 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_698_316} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:31.203470 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_698_316} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_697_315, total 43503 rows starting from the beginning of the part 2024.10.22 01:14:31.243586 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_698_316} MergeTreeSequentialSource: Reading 2 marks from part 202410_698_698_0, total 95 rows starting from the beginning of the part 2024.10.22 01:14:31.689724 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_401_179} MergeTask::MergeProjectionsStage: Merge sorted 39325 rows, containing 13 columns (13 merged, 0 gathered) in 2.969735156 sec., 13241.921563459446 rows/sec., 5.72 MiB/sec. 2024.10.22 01:14:31.749223 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5369b21eeef97f072d3ab7d2cbb6bc18) : [0, 159]: Setting download as failed: File segment: [0, 159], key: 5369b21eeef97f072d3ab7d2cbb6bc18, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:31.754042 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5369b21eeef97f072d3ab7d2cbb6bc18): No space left in cache to reserve 160 bytes, will continue without cache download 2024.10.22 01:14:31.749294 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_401_179} MergedBlockOutputStream: filled checksums 202410_1_401_179 (state Temporary) 2024.10.22 01:14:32.243979 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_598_221, 202410_599_599_0] 2024.10.22 01:14:32.252341 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_598_221, 202410_599_599_0] 2024.10.22 01:14:32.269696 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_598_221, 202410_599_599_0] 2024.10.22 01:14:32.304847 [ 1199 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:14:32.454126 [ 1199 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:14:32.572540 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.96 GiB, free memory in arenas 10.00 MiB, will set to 1.70 GiB (RSS), difference: 37.60 MiB 2024.10.22 01:14:32.624167 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b305f7b9b5b4bb9878c4ac7266103455) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b305f7b9b5b4bb9878c4ac7266103455, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:14:32.626273 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 795, file segment info: File segment: [0, 33554431], key: b305f7b9b5b4bb9878c4ac7266103455, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:14:32.640618 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:32.704498 [ 1681 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/sfr/gnwlsiznjxglqzhuudamjfcgosrpy, size 2199653 2024.10.22 01:14:32.789390 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 346 2024.10.22 01:14:32.841992 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_401_179} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_401_179 to 202410_1_401_179 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:32.951322 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_401_179} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_400_178, 202410_401_401_0] -> 202410_1_401_179 2024.10.22 01:14:32.982635 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.97 MiB. 2024.10.22 01:14:33.151154 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:14:33.165145 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:14:33.176417 [ 1210 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 211 ms. 2024.10.22 01:14:33.184676 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:14:33.195629 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:14:33.224687 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:14:33.235785 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:14:33.260833 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:14:33.264506 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:33.271160 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:33.276012 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:14:33.283806 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_698_316} MergeTask::MergeProjectionsStage: Merge sorted 43598 rows, containing 13 columns (13 merged, 0 gathered) in 2.14713635 sec., 20305.18462416232 rows/sec., 3.23 MiB/sec. 2024.10.22 01:14:33.308419 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:14:33.336539 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_698_316} MergedBlockOutputStream: filled checksums 202410_1_698_316 (state Temporary) 2024.10.22 01:14:33.474435 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(928cd4ebe35ef38f1e6bb2e7cab6614d) : [0, 416450]: Setting download as failed: File segment: [0, 416450], key: 928cd4ebe35ef38f1e6bb2e7cab6614d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:33.476231 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(928cd4ebe35ef38f1e6bb2e7cab6614d): No space left in cache to reserve 416451 bytes, will continue without cache download 2024.10.22 01:14:33.484052 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wjn/gygddtdkoowzcbsovvmwctbgrdfyu (LogSeriesLimiter: on interval from 2024-10-22 01:14:13 to 2024-10-22 01:14:33 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:14:33.502317 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_698_316} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_698_316 to 202410_1_698_316 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:33.503714 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 928cd4ebe35ef38f1e6bb2e7cab6614d, offset: 0, size: 0 2024.10.22 01:14:33.505699 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 928cd4ebe35ef38f1e6bb2e7cab6614d for removal 2024.10.22 01:14:33.507680 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_698_316} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_697_315, 202410_698_698_0] -> 202410_1_698_316 2024.10.22 01:14:33.510731 [ 1476 ] {} CacheMetadata: Key 928cd4ebe35ef38f1e6bb2e7cab6614d is removed from metadata 2024.10.22 01:14:33.514196 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5369b21eeef97f072d3ab7d2cbb6bc18, offset: 0, size: 0 2024.10.22 01:14:33.518929 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5369b21eeef97f072d3ab7d2cbb6bc18 for removal 2024.10.22 01:14:33.522043 [ 1476 ] {} CacheMetadata: Key 5369b21eeef97f072d3ab7d2cbb6bc18 is removed from metadata 2024.10.22 01:14:33.573262 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_328_328_0 (state Temporary) 2024.10.22 01:14:33.662660 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:14:33.775543 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_328_328_0 to 202410_338_338_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:33.829210 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 346 2024.10.22 01:14:34.597100 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_396_79/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:14:34.612890 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ef84f69900433d6d8c782441c7bfcccc) : [0, 157]: Setting download as failed: File segment: [0, 157], key: ef84f69900433d6d8c782441c7bfcccc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:34.614793 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ef84f69900433d6d8c782441c7bfcccc): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:14:34.700587 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1505 entries to flush up to offset 623032 2024.10.22 01:14:35.098656 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 43259 2024.10.22 01:14:35.201527 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_82_82_0 to 202410_85_85_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:35.392977 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:35.421050 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:35.431852 [ 1188 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 01:14:35.444007 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:35.445680 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.96 GiB, free memory in arenas 10.85 MiB, will set to 1.70 GiB (RSS), difference: 9.82 MiB 2024.10.22 01:14:35.448214 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:35.483434 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_691_691_0 (state Temporary) 2024.10.22 01:14:35.523031 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3b2aacffe8aa91d2578747eb655ceada) : [0, 302508]: Setting download as failed: File segment: [0, 302508], key: 3b2aacffe8aa91d2578747eb655ceada, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:35.525517 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3b2aacffe8aa91d2578747eb655ceada): No space left in cache to reserve 302509 bytes, will continue without cache download 2024.10.22 01:14:35.683913 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3b2aacffe8aa91d2578747eb655ceada, offset: 0, size: 0 2024.10.22 01:14:35.685717 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3b2aacffe8aa91d2578747eb655ceada for removal 2024.10.22 01:14:35.688112 [ 1476 ] {} CacheMetadata: Key 3b2aacffe8aa91d2578747eb655ceada is removed from metadata 2024.10.22 01:14:35.689699 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ef84f69900433d6d8c782441c7bfcccc, offset: 0, size: 0 2024.10.22 01:14:35.691460 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ef84f69900433d6d8c782441c7bfcccc for removal 2024.10.22 01:14:35.692601 [ 1476 ] {} CacheMetadata: Key ef84f69900433d6d8c782441c7bfcccc is removed from metadata 2024.10.22 01:14:35.811027 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_390_390_0 (state Temporary) 2024.10.22 01:14:35.828453 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_691_691_0 to 202410_704_704_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:35.877739 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_698_316 to 202410_699_699_0 2024.10.22 01:14:35.884900 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.56 MiB using storage policy from min volume index 0 2024.10.22 01:14:35.888393 [ 1278 ] {} DiskLocal: Reserved 1.56 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:14:35.897298 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43259 2024.10.22 01:14:35.929178 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_390_390_0 to 202410_397_397_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:36.054505 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_699_317} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_698_316 to 202410_699_699_0 into Compact with storage Full 2024.10.22 01:14:36.083644 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 623032 2024.10.22 01:14:36.161086 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(98500ba549094cfe8bae5e44d5fa94ec) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 98500ba549094cfe8bae5e44d5fa94ec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:14:36.179290 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1060, file segment info: File segment: [0, 33554431], key: 98500ba549094cfe8bae5e44d5fa94ec, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:14:36.184668 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:36.203136 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_699_317} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:36.240872 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_699_317} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_698_316, total 43598 rows starting from the beginning of the part 2024.10.22 01:14:36.356085 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column CLID 2024.10.22 01:14:36.379079 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column CLID 2024.10.22 01:14:36.396994 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 217 entries to flush up to offset 40554 2024.10.22 01:14:36.420470 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column CLID 2024.10.22 01:14:36.486510 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column CLID 2024.10.22 01:14:36.520661 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column CLID 2024.10.22 01:14:36.525735 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_699_317} MergeTreeSequentialSource: Reading 2 marks from part 202410_699_699_0, total 92 rows starting from the beginning of the part 2024.10.22 01:14:36.588664 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column CLID 2024.10.22 01:14:36.606741 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column CLID 2024.10.22 01:14:36.781910 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:36.885797 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:14:36.988321 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_401_401_0 (state Temporary) 2024.10.22 01:14:37.053209 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_401_401_0 to 202410_407_407_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:37.086651 [ 1270 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_401_179 to 202410_402_402_0 2024.10.22 01:14:37.092939 [ 1270 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.03 MiB using storage policy from min volume index 0 2024.10.22 01:14:37.092925 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 40554 2024.10.22 01:14:37.096424 [ 1270 ] {} DiskLocal: Reserved 2.03 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:14:37.111552 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_599_222, 202410_600_600_0] 2024.10.22 01:14:37.114133 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_599_222, 202410_600_600_0] 2024.10.22 01:14:37.125358 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_599_222, 202410_600_600_0] 2024.10.22 01:14:37.126471 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_402_180} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_401_179 to 202410_402_402_0 into Wide with storage Full 2024.10.22 01:14:37.137644 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3fe86918ab8ff93cfc375a1e01be11c2) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 3fe86918ab8ff93cfc375a1e01be11c2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:37.142909 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3fe86918ab8ff93cfc375a1e01be11c2): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:14:37.203456 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_402_180} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:37.214860 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_402_180} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_401_179, total 39325 rows starting from the beginning of the part 2024.10.22 01:14:37.271229 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_343_121, 202410_344_344_0] 2024.10.22 01:14:37.275031 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_343_121, 202410_344_344_0] 2024.10.22 01:14:37.289766 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_343_121, 202410_344_344_0] 2024.10.22 01:14:37.483833 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_402_180} MergeTreeSequentialSource: Reading 2 marks from part 202410_402_402_0, total 234 rows starting from the beginning of the part 2024.10.22 01:14:38.073708 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 6169 2024.10.22 01:14:38.227353 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3b11340ae07ae94a4bc925e8674e7e5b) : [0, 302578]: Setting download as failed: File segment: [0, 302578], key: 3b11340ae07ae94a4bc925e8674e7e5b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:38.233509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3b11340ae07ae94a4bc925e8674e7e5b): No space left in cache to reserve 302579 bytes, will continue without cache download 2024.10.22 01:14:38.273106 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3b11340ae07ae94a4bc925e8674e7e5b, offset: 0, size: 0 2024.10.22 01:14:38.274291 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3b11340ae07ae94a4bc925e8674e7e5b for removal 2024.10.22 01:14:38.277641 [ 1476 ] {} CacheMetadata: Key 3b11340ae07ae94a4bc925e8674e7e5b is removed from metadata 2024.10.22 01:14:38.279296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3fe86918ab8ff93cfc375a1e01be11c2, offset: 0, size: 0 2024.10.22 01:14:38.280825 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3fe86918ab8ff93cfc375a1e01be11c2 for removal 2024.10.22 01:14:38.284381 [ 1476 ] {} CacheMetadata: Key 3fe86918ab8ff93cfc375a1e01be11c2 is removed from metadata 2024.10.22 01:14:38.292534 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:38.295381 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:38.332289 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_407_407_0 (state Temporary) 2024.10.22 01:14:38.369643 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.96 GiB, free memory in arenas 9.66 MiB, will set to 1.69 GiB (RSS), difference: -24.01 MiB 2024.10.22 01:14:38.387418 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_407_407_0 to 202410_420_420_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:38.429399 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6169 2024.10.22 01:14:38.423170 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_699_317} MergeTask::MergeProjectionsStage: Merge sorted 43690 rows, containing 13 columns (13 merged, 0 gathered) in 2.37926805 sec., 18362.790186671067 rows/sec., 2.92 MiB/sec. 2024.10.22 01:14:38.492139 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_699_317} MergedBlockOutputStream: filled checksums 202410_1_699_317 (state Temporary) 2024.10.22 01:14:38.558508 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_699_317} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_699_317 to 202410_1_699_317 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:38.576727 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_699_317} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_698_316, 202410_699_699_0] -> 202410_1_699_317 2024.10.22 01:14:38.601178 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:14:38.958935 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(aae2d289d3af0f60f575334458e30c25) : [0, 155]: Setting download as failed: File segment: [0, 155], key: aae2d289d3af0f60f575334458e30c25, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:38.962572 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(aae2d289d3af0f60f575334458e30c25): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:14:39.274877 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3267 2024.10.22 01:14:39.882422 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e314501c522d36a9d32377fe1002ef4d) : [0, 239201]: Setting download as failed: File segment: [0, 239201], key: e314501c522d36a9d32377fe1002ef4d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:39.883722 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e314501c522d36a9d32377fe1002ef4d): No space left in cache to reserve 239202 bytes, will continue without cache download 2024.10.22 01:14:39.911631 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e314501c522d36a9d32377fe1002ef4d, offset: 0, size: 0 2024.10.22 01:14:39.916060 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e314501c522d36a9d32377fe1002ef4d for removal 2024.10.22 01:14:39.920348 [ 1476 ] {} CacheMetadata: Key e314501c522d36a9d32377fe1002ef4d is removed from metadata 2024.10.22 01:14:39.924186 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aae2d289d3af0f60f575334458e30c25, offset: 0, size: 0 2024.10.22 01:14:39.925467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aae2d289d3af0f60f575334458e30c25 for removal 2024.10.22 01:14:39.940428 [ 1476 ] {} CacheMetadata: Key aae2d289d3af0f60f575334458e30c25 is removed from metadata 2024.10.22 01:14:40.019760 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 91 entries to flush up to offset 43350 2024.10.22 01:14:40.043487 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:14:40.049330 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:14:40.324953 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:40.328822 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:40.440976 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.96 GiB, free memory in arenas 17.54 MiB, will set to 1.71 GiB (RSS), difference: 2.36 MiB 2024.10.22 01:14:40.697235 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_692_692_0 (state Temporary) 2024.10.22 01:14:40.703568 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_600_223, 202410_601_601_0] 2024.10.22 01:14:40.709515 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_600_223, 202410_601_601_0] 2024.10.22 01:14:40.720071 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_600_223, 202410_601_601_0] 2024.10.22 01:14:40.756952 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_692_692_0 to 202410_705_705_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:40.882289 [ 1236 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_699_317 to 202410_700_700_0 2024.10.22 01:14:40.886373 [ 1236 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.56 MiB using storage policy from min volume index 0 2024.10.22 01:14:40.888321 [ 1236 ] {} DiskLocal: Reserved 1.56 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:40.917438 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_700_318} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_699_317 to 202410_700_700_0 into Compact with storage Full 2024.10.22 01:14:40.968501 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43350 2024.10.22 01:14:40.991677 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_402_180} MergeTask::MergeProjectionsStage: Merge sorted 39559 rows, containing 13 columns (13 merged, 0 gathered) in 3.874414812 sec., 10210.316117282075 rows/sec., 4.41 MiB/sec. 2024.10.22 01:14:40.992525 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7069e20cc5ab81a123e9c301d7178bae) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 7069e20cc5ab81a123e9c301d7178bae, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:40.996417 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7069e20cc5ab81a123e9c301d7178bae): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:14:41.002054 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_700_318} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:41.039282 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_700_318} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_699_317, total 43690 rows starting from the beginning of the part 2024.10.22 01:14:41.121075 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_700_318} MergeTreeSequentialSource: Reading 2 marks from part 202410_700_700_0, total 93 rows starting from the beginning of the part 2024.10.22 01:14:41.333269 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_402_180} MergedBlockOutputStream: filled checksums 202410_1_402_180 (state Temporary) 2024.10.22 01:14:41.333291 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 347 2024.10.22 01:14:41.607783 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:41.615606 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:41.790264 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_329_329_0 (state Temporary) 2024.10.22 01:14:41.826648 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_402_180} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_402_180 to 202410_1_402_180 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:41.834797 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_402_180} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_401_179, 202410_402_402_0] -> 202410_1_402_180 2024.10.22 01:14:41.875191 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.97 MiB. 2024.10.22 01:14:42.095920 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_329_329_0 to 202410_339_339_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:42.065555 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c6d3f7413521481813871db69da800e1) : [0, 495095]: Setting download as failed: File segment: [0, 495095], key: c6d3f7413521481813871db69da800e1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:42.170247 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c6d3f7413521481813871db69da800e1): No space left in cache to reserve 495096 bytes, will continue without cache download 2024.10.22 01:14:42.224648 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 347 2024.10.22 01:14:42.297116 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c6d3f7413521481813871db69da800e1, offset: 0, size: 0 2024.10.22 01:14:42.299532 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c6d3f7413521481813871db69da800e1 for removal 2024.10.22 01:14:42.304502 [ 1476 ] {} CacheMetadata: Key c6d3f7413521481813871db69da800e1 is removed from metadata 2024.10.22 01:14:42.311226 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7069e20cc5ab81a123e9c301d7178bae, offset: 0, size: 0 2024.10.22 01:14:42.315354 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7069e20cc5ab81a123e9c301d7178bae for removal 2024.10.22 01:14:42.320633 [ 1476 ] {} CacheMetadata: Key 7069e20cc5ab81a123e9c301d7178bae is removed from metadata 2024.10.22 01:14:42.492088 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.96 GiB, free memory in arenas 13.69 MiB, will set to 1.70 GiB (RSS), difference: 23.91 MiB 2024.10.22 01:14:42.682519 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_356_71, 202410_357_357_0, 202410_358_358_0, 202410_359_359_0, 202410_360_360_0, 202410_361_361_0] 2024.10.22 01:14:42.696032 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_356_71, 202410_357_357_0, 202410_358_358_0, 202410_359_359_0, 202410_360_360_0, 202410_361_361_0] 2024.10.22 01:14:42.825291 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_356_71, 202410_357_357_0, 202410_358_358_0, 202410_359_359_0, 202410_360_360_0, 202410_361_361_0] 2024.10.22 01:14:42.989432 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a815b51095506cc3d9b40f4191e90056) : [0, 154]: Setting download as failed: File segment: [0, 154], key: a815b51095506cc3d9b40f4191e90056, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:42.991071 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a815b51095506cc3d9b40f4191e90056): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:14:43.059055 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_84_17/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:14:43.087725 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 624184 2024.10.22 01:14:43.915679 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:43.924027 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:43.939376 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_700_318} MergeTask::MergeProjectionsStage: Merge sorted 43783 rows, containing 13 columns (13 merged, 0 gathered) in 3.036176621 sec., 14420.439080246775 rows/sec., 2.30 MiB/sec. 2024.10.22 01:14:44.008980 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_700_318} MergedBlockOutputStream: filled checksums 202410_1_700_318 (state Temporary) 2024.10.22 01:14:44.064034 [ 1237 ] {} DatabaseCatalog: Execution took 331 ms. 2024.10.22 01:14:44.095472 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_700_318} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_700_318 to 202410_1_700_318 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:44.099533 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_700_318} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_699_317, 202410_700_700_0] -> 202410_1_700_318 2024.10.22 01:14:44.100374 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_391_391_0 (state Temporary) 2024.10.22 01:14:44.263430 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f186933a4333ea0d671b01d419fababf) : [0, 158650]: Setting download as failed: File segment: [0, 158650], key: f186933a4333ea0d671b01d419fababf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:44.265765 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f186933a4333ea0d671b01d419fababf): No space left in cache to reserve 158651 bytes, will continue without cache download 2024.10.22 01:14:44.298561 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_391_391_0 to 202410_398_398_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:44.325276 [ 1255 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 01:14:44.361518 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:14:44.371094 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 624184 2024.10.22 01:14:44.389565 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f186933a4333ea0d671b01d419fababf, offset: 0, size: 0 2024.10.22 01:14:44.391509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f186933a4333ea0d671b01d419fababf for removal 2024.10.22 01:14:44.395071 [ 1476 ] {} CacheMetadata: Key f186933a4333ea0d671b01d419fababf is removed from metadata 2024.10.22 01:14:44.402076 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a815b51095506cc3d9b40f4191e90056, offset: 0, size: 0 2024.10.22 01:14:44.405786 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a815b51095506cc3d9b40f4191e90056 for removal 2024.10.22 01:14:44.476760 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.96 GiB, free memory in arenas 11.94 MiB, will set to 1.70 GiB (RSS), difference: 6.73 MiB 2024.10.22 01:14:44.516163 [ 1476 ] {} CacheMetadata: Key a815b51095506cc3d9b40f4191e90056 is removed from metadata 2024.10.22 01:14:44.631311 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_344_122, 202410_345_345_0] 2024.10.22 01:14:44.634177 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_344_122, 202410_345_345_0] 2024.10.22 01:14:44.656333 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_344_122, 202410_345_345_0] 2024.10.22 01:14:44.763315 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 248 entries to flush up to offset 40802 2024.10.22 01:14:44.888753 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_396_79/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:14:44.985343 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 43424 2024.10.22 01:14:45.199818 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:45.203500 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:45.275187 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(64d003a9d8c10279926e99aee2050eeb) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 64d003a9d8c10279926e99aee2050eeb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:45.279119 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(64d003a9d8c10279926e99aee2050eeb): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:14:45.283706 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_693_693_0 (state Temporary) 2024.10.22 01:14:45.467403 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:45.472112 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:45.445725 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_693_693_0 to 202410_706_706_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:45.574858 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_700_318 to 202410_701_701_0 2024.10.22 01:14:45.578829 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.57 MiB using storage policy from min volume index 0 2024.10.22 01:14:45.581164 [ 1172 ] {} DiskLocal: Reserved 1.57 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:45.593721 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_701_319} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_700_318 to 202410_701_701_0 into Compact with storage Full 2024.10.22 01:14:45.597404 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43424 2024.10.22 01:14:45.676286 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_286_59, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:14:45.680191 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_286_59, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:14:45.705821 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_286_59, 202410_287_287_0, 202410_288_288_0, 202410_289_289_0, 202410_290_290_0, 202410_291_291_0] 2024.10.22 01:14:45.838283 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_701_319} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:45.843565 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_701_319} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_700_318, total 43783 rows starting from the beginning of the part 2024.10.22 01:14:45.765758 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_402_402_0 (state Temporary) 2024.10.22 01:14:45.869089 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_701_319} MergeTreeSequentialSource: Reading 2 marks from part 202410_701_701_0, total 81 rows starting from the beginning of the part 2024.10.22 01:14:45.896086 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_601_224, 202410_602_602_0] 2024.10.22 01:14:45.897675 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_601_224, 202410_602_602_0] 2024.10.22 01:14:45.910576 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_601_224, 202410_602_602_0] 2024.10.22 01:14:45.934106 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 6183 2024.10.22 01:14:45.960409 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(39a351a94ca83b3e1f6265d887122852) : [0, 1299]: Setting download as failed: File segment: [0, 1299], key: 39a351a94ca83b3e1f6265d887122852, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:45.967163 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(39a351a94ca83b3e1f6265d887122852): No space left in cache to reserve 1300 bytes, will continue without cache download 2024.10.22 01:14:45.977073 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_402_402_0 to 202410_408_408_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:45.998864 [ 1155 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_402_180 to 202410_403_403_0 2024.10.22 01:14:46.003528 [ 1155 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.04 MiB using storage policy from min volume index 0 2024.10.22 01:14:46.004031 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 39a351a94ca83b3e1f6265d887122852, offset: 0, size: 0 2024.10.22 01:14:46.007575 [ 1155 ] {} DiskLocal: Reserved 2.04 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:46.009437 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 39a351a94ca83b3e1f6265d887122852 for removal 2024.10.22 01:14:46.016581 [ 1476 ] {} CacheMetadata: Key 39a351a94ca83b3e1f6265d887122852 is removed from metadata 2024.10.22 01:14:46.017673 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 64d003a9d8c10279926e99aee2050eeb, offset: 0, size: 0 2024.10.22 01:14:46.024823 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 64d003a9d8c10279926e99aee2050eeb for removal 2024.10.22 01:14:46.032506 [ 1476 ] {} CacheMetadata: Key 64d003a9d8c10279926e99aee2050eeb is removed from metadata 2024.10.22 01:14:46.033147 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 40802 2024.10.22 01:14:46.125237 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:46.129794 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:46.159931 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_403_181} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_402_180 to 202410_403_403_0 into Wide with storage Full 2024.10.22 01:14:46.178293 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_408_408_0 (state Temporary) 2024.10.22 01:14:46.215956 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_408_408_0 to 202410_421_421_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:46.241168 [ 1156 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_416_83 to 202410_421_421_0 2024.10.22 01:14:46.266727 [ 1156 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:46.271618 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_403_181} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:46.281391 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:46.281775 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_403_181} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_402_180, total 39559 rows starting from the beginning of the part 2024.10.22 01:14:46.299531 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_416_83 to 202410_421_421_0 into Compact with storage Full 2024.10.22 01:14:46.355501 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:46.374702 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6183 2024.10.22 01:14:46.378543 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_416_83, total 6205 rows starting from the beginning of the part 2024.10.22 01:14:46.471155 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_403_181} MergeTreeSequentialSource: Reading 2 marks from part 202410_403_403_0, total 207 rows starting from the beginning of the part 2024.10.22 01:14:46.616234 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_417_417_0, total 28 rows starting from the beginning of the part 2024.10.22 01:14:46.656340 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_418_418_0, total 15 rows starting from the beginning of the part 2024.10.22 01:14:46.684090 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_419_419_0, total 11 rows starting from the beginning of the part 2024.10.22 01:14:46.704885 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_420_420_0, total 16 rows starting from the beginning of the part 2024.10.22 01:14:46.727518 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_421_421_0, total 14 rows starting from the beginning of the part 2024.10.22 01:14:46.795274 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 3305 2024.10.22 01:14:47.007938 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(da1b8569eead11997d227bba611ed46f) : [0, 155]: Setting download as failed: File segment: [0, 155], key: da1b8569eead11997d227bba611ed46f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:47.009561 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(da1b8569eead11997d227bba611ed46f): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:14:47.463797 [ 1163 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:14:47.508059 [ 1163 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:14:47.517159 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.96 GiB, free memory in arenas 13.22 MiB, will set to 1.71 GiB (RSS), difference: -47.80 MiB 2024.10.22 01:14:47.772265 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} MergeTask::MergeProjectionsStage: Merge sorted 6289 rows, containing 12 columns (12 merged, 0 gathered) in 1.461468813 sec., 4303.205066066641 rows/sec., 681.10 KiB/sec. 2024.10.22 01:14:47.857263 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} MergedBlockOutputStream: filled checksums 202410_1_421_84 (state Temporary) 2024.10.22 01:14:47.874918 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(57c5541fc9e4c209a70048f66c9f0000) : [0, 30230]: Setting download as failed: File segment: [0, 30230], key: 57c5541fc9e4c209a70048f66c9f0000, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:47.876280 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(57c5541fc9e4c209a70048f66c9f0000): No space left in cache to reserve 30231 bytes, will continue without cache download 2024.10.22 01:14:47.896609 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 57c5541fc9e4c209a70048f66c9f0000, offset: 0, size: 0 2024.10.22 01:14:47.899617 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 57c5541fc9e4c209a70048f66c9f0000 for removal 2024.10.22 01:14:47.905075 [ 1476 ] {} CacheMetadata: Key 57c5541fc9e4c209a70048f66c9f0000 is removed from metadata 2024.10.22 01:14:47.911180 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: da1b8569eead11997d227bba611ed46f, offset: 0, size: 0 2024.10.22 01:14:47.914204 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key da1b8569eead11997d227bba611ed46f for removal 2024.10.22 01:14:48.016011 [ 1476 ] {} CacheMetadata: Key da1b8569eead11997d227bba611ed46f is removed from metadata 2024.10.22 01:14:48.086585 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_421_84 to 202410_1_421_84 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:48.089049 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_421_84} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_416_83, 202410_421_421_0] -> 202410_1_421_84 2024.10.22 01:14:48.131052 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.45 MiB. 2024.10.22 01:14:48.297517 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_701_319} MergeTask::MergeProjectionsStage: Merge sorted 43864 rows, containing 13 columns (13 merged, 0 gathered) in 2.706565938 sec., 16206.514455883913 rows/sec., 2.58 MiB/sec. 2024.10.22 01:14:48.447458 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_701_319} MergedBlockOutputStream: filled checksums 202410_1_701_319 (state Temporary) 2024.10.22 01:14:48.606361 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_701_319} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_701_319 to 202410_1_701_319 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:48.612683 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_701_319} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_700_318, 202410_701_701_0] -> 202410_1_701_319 2024.10.22 01:14:48.743846 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:14:48.953385 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0fef204a01c31b4c2796e7505c232a55) : [0, 192]: Setting download as failed: File segment: [0, 192], key: 0fef204a01c31b4c2796e7505c232a55, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:48.959051 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0fef204a01c31b4c2796e7505c232a55): No space left in cache to reserve 193 bytes, will continue without cache download 2024.10.22 01:14:49.496556 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.96 GiB, free memory in arenas 16.07 MiB, will set to 1.71 GiB (RSS), difference: 21.11 MiB 2024.10.22 01:14:49.568629 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_403_181} MergeTask::MergeProjectionsStage: Merge sorted 39766 rows, containing 13 columns (13 merged, 0 gathered) in 3.429586845 sec., 11594.98266036765 rows/sec., 5.01 MiB/sec. 2024.10.22 01:14:49.637902 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 43510 2024.10.22 01:14:49.849358 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(9eaf90f81bdb172b415bf3d2743f70b8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9eaf90f81bdb172b415bf3d2743f70b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:14:49.852059 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 976458, file segment info: File segment: [0, 33554431], key: 9eaf90f81bdb172b415bf3d2743f70b8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:14:49.857020 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:49.965064 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:49.969640 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:50.044102 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:14:50.068111 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_403_181} MergedBlockOutputStream: filled checksums 202410_1_403_181 (state Temporary) 2024.10.22 01:14:50.093834 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_602_225, 202410_603_603_0] 2024.10.22 01:14:50.096291 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_602_225, 202410_603_603_0] 2024.10.22 01:14:50.108098 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_602_225, 202410_603_603_0] 2024.10.22 01:14:50.148798 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:14:50.167322 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_694_694_0 (state Temporary) 2024.10.22 01:14:50.275150 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_694_694_0 to 202410_707_707_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:50.294919 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_701_319 to 202410_702_702_0 2024.10.22 01:14:50.296961 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.57 MiB using storage policy from min volume index 0 2024.10.22 01:14:50.298261 [ 1201 ] {} DiskLocal: Reserved 1.57 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:14:50.300626 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43510 2024.10.22 01:14:50.305241 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_702_320} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_701_319 to 202410_702_702_0 into Compact with storage Full 2024.10.22 01:14:50.352222 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_702_320} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:50.364736 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_702_320} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_701_319, total 43864 rows starting from the beginning of the part 2024.10.22 01:14:50.396183 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_702_320} MergeTreeSequentialSource: Reading 2 marks from part 202410_702_702_0, total 63 rows starting from the beginning of the part 2024.10.22 01:14:50.583349 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(579e30d284815a2756381b23dcf8a1ca) : [0, 1048786]: Setting download as failed: File segment: [0, 1048786], key: 579e30d284815a2756381b23dcf8a1ca, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:50.585372 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(579e30d284815a2756381b23dcf8a1ca): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:14:50.988409 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 579e30d284815a2756381b23dcf8a1ca, offset: 0, size: 0 2024.10.22 01:14:50.990944 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 579e30d284815a2756381b23dcf8a1ca for removal 2024.10.22 01:14:50.993147 [ 1476 ] {} CacheMetadata: Key 579e30d284815a2756381b23dcf8a1ca is removed from metadata 2024.10.22 01:14:50.999109 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0fef204a01c31b4c2796e7505c232a55, offset: 0, size: 0 2024.10.22 01:14:51.000711 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0fef204a01c31b4c2796e7505c232a55 for removal 2024.10.22 01:14:51.012622 [ 1476 ] {} CacheMetadata: Key 0fef204a01c31b4c2796e7505c232a55 is removed from metadata 2024.10.22 01:14:51.361643 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_403_181} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_403_181 to 202410_1_403_181 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:51.370231 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_403_181} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_402_180, 202410_403_403_0] -> 202410_1_403_181 2024.10.22 01:14:51.374908 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 625336 2024.10.22 01:14:51.392563 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.97 MiB. 2024.10.22 01:14:51.493276 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(75adc223457f6adb949ceabee9db99c9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 75adc223457f6adb949ceabee9db99c9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:14:51.499107 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 753, file segment info: File segment: [0, 33554431], key: 75adc223457f6adb949ceabee9db99c9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:14:51.503493 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:51.664659 [ 1684 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wjn/gygddtdkoowzcbsovvmwctbgrdfyu, size 976458 2024.10.22 01:14:51.824087 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} MergeTask::MergeProjectionsStage: Merge sorted 635117 rows, containing 4 columns (4 merged, 0 gathered) in 24.072937078 sec., 26383.029122791446 rows/sec., 386.69 KiB/sec. 2024.10.22 01:14:51.918767 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} MergedBlockOutputStream: filled checksums 202410_1_396_79 (state Temporary) 2024.10.22 01:14:51.972116 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_396_79 to 202410_1_396_79 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:51.976501 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_396_79} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_391_78, 202410_396_396_0] -> 202410_1_396_79 2024.10.22 01:14:52.016326 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:14:52.019173 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:14:52.053319 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:14:52.078849 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:14:52.092680 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:14:52.117786 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:14:52.119722 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:52.125714 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:14:52.154667 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:14:52.173158 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:14:52.192377 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(d6786e6264b5778631c0197241e4a1ae) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d6786e6264b5778631c0197241e4a1ae, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:14:52.278701 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 581928, file segment info: File segment: [0, 33554431], key: d6786e6264b5778631c0197241e4a1ae, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:14:52.280273 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:52.293147 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:14:52.311265 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:14:52.342703 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_392_392_0 (state Temporary) 2024.10.22 01:14:52.463924 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fki/cmcycbnwcockxlgwfmwfckzjsyttm (LogSeriesLimiter: on interval from 2024-10-22 01:14:33 to 2024-10-22 01:14:52 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:14:52.494896 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_345_123, 202410_346_346_0] 2024.10.22 01:14:52.496096 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_345_123, 202410_346_346_0] 2024.10.22 01:14:52.518603 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_345_123, 202410_346_346_0] 2024.10.22 01:14:52.564948 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_392_392_0 to 202410_399_399_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:52.596547 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 625336 2024.10.22 01:14:52.868187 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(9563fd165635de65823175f8e78a6334) : [0, 194]: Setting download as failed: File segment: [0, 194], key: 9563fd165635de65823175f8e78a6334, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:52.869808 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(9563fd165635de65823175f8e78a6334): No space left in cache to reserve 195 bytes, will continue without cache download 2024.10.22 01:14:53.650949 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_702_320} MergeTask::MergeProjectionsStage: Merge sorted 43927 rows, containing 13 columns (13 merged, 0 gathered) in 3.349820685 sec., 13113.2392240273 rows/sec., 2.09 MiB/sec. 2024.10.22 01:14:53.718776 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.96 GiB, free memory in arenas 14.72 MiB, will set to 1.70 GiB (RSS), difference: 24.82 MiB 2024.10.22 01:14:53.732033 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(b7e8463e00a0cf903e425840f62be3cc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b7e8463e00a0cf903e425840f62be3cc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:14:53.735402 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 867, file segment info: File segment: [0, 33554431], key: b7e8463e00a0cf903e425840f62be3cc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:14:53.736866 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:53.786848 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 235 entries to flush up to offset 41037 2024.10.22 01:14:53.804613 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_702_320} MergedBlockOutputStream: filled checksums 202410_1_702_320 (state Temporary) 2024.10.22 01:14:53.877374 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 24 entries to flush up to offset 6207 2024.10.22 01:14:54.081216 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_702_320} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_702_320 to 202410_1_702_320 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:54.084964 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_702_320} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_701_319, 202410_702_702_0] -> 202410_1_702_320 2024.10.22 01:14:54.215235 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:14:54.317792 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column YCLID 2024.10.22 01:14:54.323782 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:54.389775 [ 1188 ] {} DatabaseCatalog: Execution took 325 ms. 2024.10.22 01:14:54.392851 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:54.395859 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6195ed99613c2efe940e42bfaf306ea6) : [0, 1187433]: Setting download as failed: File segment: [0, 1187433], key: 6195ed99613c2efe940e42bfaf306ea6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:54.408563 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 43587 2024.10.22 01:14:54.415444 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:14:54.415940 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6195ed99613c2efe940e42bfaf306ea6): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:14:54.418781 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:14:54.423734 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column YCLID 2024.10.22 01:14:54.478264 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column YCLID 2024.10.22 01:14:54.480165 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_409_409_0 (state Temporary) 2024.10.22 01:14:54.498904 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6195ed99613c2efe940e42bfaf306ea6, offset: 0, size: 0 2024.10.22 01:14:54.499730 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column YCLID 2024.10.22 01:14:54.502256 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6195ed99613c2efe940e42bfaf306ea6 for removal 2024.10.22 01:14:54.507012 [ 1476 ] {} CacheMetadata: Key 6195ed99613c2efe940e42bfaf306ea6 is removed from metadata 2024.10.22 01:14:54.512212 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_403_403_0 (state Temporary) 2024.10.22 01:14:54.515376 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9563fd165635de65823175f8e78a6334, offset: 0, size: 0 2024.10.22 01:14:54.521972 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9563fd165635de65823175f8e78a6334 for removal 2024.10.22 01:14:54.530856 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column YCLID 2024.10.22 01:14:54.532332 [ 1476 ] {} CacheMetadata: Key 9563fd165635de65823175f8e78a6334 is removed from metadata 2024.10.22 01:14:54.539557 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_409_409_0 to 202410_422_422_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:54.569336 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column YCLID 2024.10.22 01:14:54.590471 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column YCLID 2024.10.22 01:14:54.594451 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6207 2024.10.22 01:14:54.680746 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_403_403_0 to 202410_409_409_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:54.700239 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_403_181 to 202410_404_404_0 2024.10.22 01:14:54.721379 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 41037 2024.10.22 01:14:54.729789 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.05 MiB using storage policy from min volume index 0 2024.10.22 01:14:54.734315 [ 1154 ] {} DiskLocal: Reserved 2.05 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:14:54.883214 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:54.884682 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:14:54.887434 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_404_182} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_403_181 to 202410_404_404_0 into Wide with storage Full 2024.10.22 01:14:54.964039 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_695_695_0 (state Temporary) 2024.10.22 01:14:55.060367 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_404_182} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:55.070260 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_695_695_0 to 202410_708_708_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:55.072659 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_404_182} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_403_181, total 39766 rows starting from the beginning of the part 2024.10.22 01:14:55.195048 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_702_320 to 202410_703_703_0 2024.10.22 01:14:55.198179 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.57 MiB using storage policy from min volume index 0 2024.10.22 01:14:55.199869 [ 1150 ] {} DiskLocal: Reserved 1.57 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:14:55.212491 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_703_321} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_702_320 to 202410_703_703_0 into Compact with storage Full 2024.10.22 01:14:55.220053 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43587 2024.10.22 01:14:55.244315 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_703_321} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:55.266986 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_703_321} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_702_320, total 43927 rows starting from the beginning of the part 2024.10.22 01:14:55.283583 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_703_321} MergeTreeSequentialSource: Reading 2 marks from part 202410_703_703_0, total 85 rows starting from the beginning of the part 2024.10.22 01:14:55.304313 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e76b159b81a9a3afe022c62d4c4264d9) : [0, 159]: Setting download as failed: File segment: [0, 159], key: e76b159b81a9a3afe022c62d4c4264d9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:55.307995 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e76b159b81a9a3afe022c62d4c4264d9): No space left in cache to reserve 160 bytes, will continue without cache download 2024.10.22 01:14:55.341411 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_404_182} MergeTreeSequentialSource: Reading 2 marks from part 202410_404_404_0, total 231 rows starting from the beginning of the part 2024.10.22 01:14:55.528160 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_603_226, 202410_604_604_0] 2024.10.22 01:14:55.541460 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_603_226, 202410_604_604_0] 2024.10.22 01:14:55.550507 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_603_226, 202410_604_604_0] 2024.10.22 01:14:56.072952 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_703_321/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:14:56.223622 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ad403efaa78d88f0f9a4028dd5b44a91) : [0, 465073]: Setting download as failed: File segment: [0, 465073], key: ad403efaa78d88f0f9a4028dd5b44a91, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:56.225930 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ad403efaa78d88f0f9a4028dd5b44a91): No space left in cache to reserve 465074 bytes, will continue without cache download 2024.10.22 01:14:56.280899 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ad403efaa78d88f0f9a4028dd5b44a91, offset: 0, size: 0 2024.10.22 01:14:56.282624 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ad403efaa78d88f0f9a4028dd5b44a91 for removal 2024.10.22 01:14:56.286114 [ 1476 ] {} CacheMetadata: Key ad403efaa78d88f0f9a4028dd5b44a91 is removed from metadata 2024.10.22 01:14:56.289102 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e76b159b81a9a3afe022c62d4c4264d9, offset: 0, size: 0 2024.10.22 01:14:56.291530 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e76b159b81a9a3afe022c62d4c4264d9 for removal 2024.10.22 01:14:56.294902 [ 1476 ] {} CacheMetadata: Key e76b159b81a9a3afe022c62d4c4264d9 is removed from metadata 2024.10.22 01:14:56.683617 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.96 GiB, free memory in arenas 9.77 MiB, will set to 1.71 GiB (RSS), difference: -62.67 MiB 2024.10.22 01:14:56.817229 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_404_182/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:14:57.204425 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(37f7f644a8228e1f80db8d7656a66428) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 37f7f644a8228e1f80db8d7656a66428, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:57.207535 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(37f7f644a8228e1f80db8d7656a66428): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:14:57.228215 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 348 2024.10.22 01:14:57.669630 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:57.679669 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_703_321} MergeTask::MergeProjectionsStage: Merge sorted 44012 rows, containing 13 columns (13 merged, 0 gathered) in 2.47477211 sec., 17784.26378015065 rows/sec., 2.83 MiB/sec. 2024.10.22 01:14:57.773526 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:14:57.859097 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_703_321} MergedBlockOutputStream: filled checksums 202410_1_703_321 (state Temporary) 2024.10.22 01:14:57.937392 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_330_330_0 (state Temporary) 2024.10.22 01:14:57.959931 [ 1277 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_335_69 to 202410_339_339_0 2024.10.22 01:14:57.976513 [ 1277 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:57.979230 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:14:58.035245 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_339_70} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_335_69 to 202410_339_339_0 into Compact with storage Full 2024.10.22 01:14:58.110116 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(fd1e5409b8d370a7a458bcd756b57dbf) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fd1e5409b8d370a7a458bcd756b57dbf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:14:58.116572 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: fd1e5409b8d370a7a458bcd756b57dbf, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:14:58.119574 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:14:58.121261 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_703_321} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_703_321 to 202410_1_703_321 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:58.161294 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_703_321} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_702_320, 202410_703_703_0] -> 202410_1_703_321 2024.10.22 01:14:58.195440 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:14:58.215247 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_330_330_0 to 202410_340_340_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:58.270385 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_404_182} MergeTask::MergeProjectionsStage: Merge sorted 39997 rows, containing 13 columns (13 merged, 0 gathered) in 3.387374122 sec., 11807.671240159518 rows/sec., 5.10 MiB/sec. 2024.10.22 01:14:58.291032 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 348 2024.10.22 01:14:58.367072 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_339_70} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:14:58.382462 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_339_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_335_69, total 379 rows starting from the beginning of the part 2024.10.22 01:14:58.438304 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_404_182} MergedBlockOutputStream: filled checksums 202410_1_404_182 (state Temporary) 2024.10.22 01:14:58.464475 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_339_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_336_336_0, total 1 rows starting from the beginning of the part 2024.10.22 01:14:58.520360 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_339_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_337_337_0, total 1 rows starting from the beginning of the part 2024.10.22 01:14:58.583911 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_339_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_338_338_0, total 1 rows starting from the beginning of the part 2024.10.22 01:14:58.641825 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_339_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_339_339_0, total 1 rows starting from the beginning of the part 2024.10.22 01:14:58.864912 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(06c2005baa5b8f25f9287c49dd09d1ad) : [0, 118014]: Setting download as failed: File segment: [0, 118014], key: 06c2005baa5b8f25f9287c49dd09d1ad, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:14:58.866669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(06c2005baa5b8f25f9287c49dd09d1ad): No space left in cache to reserve 118015 bytes, will continue without cache download 2024.10.22 01:14:58.882499 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 06c2005baa5b8f25f9287c49dd09d1ad, offset: 0, size: 0 2024.10.22 01:14:58.885191 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 06c2005baa5b8f25f9287c49dd09d1ad for removal 2024.10.22 01:14:58.888224 [ 1476 ] {} CacheMetadata: Key 06c2005baa5b8f25f9287c49dd09d1ad is removed from metadata 2024.10.22 01:14:58.888857 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 37f7f644a8228e1f80db8d7656a66428, offset: 0, size: 0 2024.10.22 01:14:58.991686 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 37f7f644a8228e1f80db8d7656a66428 for removal 2024.10.22 01:14:58.994886 [ 1476 ] {} CacheMetadata: Key 37f7f644a8228e1f80db8d7656a66428 is removed from metadata 2024.10.22 01:14:59.041255 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/onk/lrbsfruzyjurltghpfxnfzceochko. 2024.10.22 01:14:59.240850 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 99 entries to flush up to offset 43686 2024.10.22 01:14:59.465152 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_404_182} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_404_182 to 202410_1_404_182 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:59.470261 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_404_182} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_403_181, 202410_404_404_0] -> 202410_1_404_182 2024.10.22 01:14:59.495286 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.97 MiB. 2024.10.22 01:14:59.589438 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:14:59.592544 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:14:59.599695 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1036 entries to flush up to offset 626372 2024.10.22 01:14:59.758919 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/onk/lrbsfruzyjurltghpfxnfzceochko. 2024.10.22 01:14:59.767671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_21_21_0 to 201403_21_21_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:14:59.785616 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_696_696_0 (state Temporary) 2024.10.22 01:14:59.803454 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:15:00.042764 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_696_696_0 to 202410_709_709_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:00.048628 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:15:00.109195 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43686 2024.10.22 01:15:00.152420 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:15:00.069834 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_703_321 to 202410_704_704_0 2024.10.22 01:15:00.175476 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.58 MiB using storage policy from min volume index 0 2024.10.22 01:15:00.178558 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_339_70} MergeTask::MergeProjectionsStage: Merge sorted 383 rows, containing 32 columns (32 merged, 0 gathered) in 2.144257942 sec., 178.61657056182656 rows/sec., 17.97 KiB/sec. 2024.10.22 01:15:00.203187 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_339_70} MergedBlockOutputStream: filled checksums 202410_1_339_70 (state Temporary) 2024.10.22 01:15:00.279450 [ 1240 ] {} DiskLocal: Reserved 1.58 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:00.284263 [ 1240 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 265 ms. 2024.10.22 01:15:00.296016 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_704_322} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_703_321 to 202410_704_704_0 into Compact with storage Full 2024.10.22 01:15:00.318260 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_704_322} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:00.325709 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_704_322} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_703_321, total 44012 rows starting from the beginning of the part 2024.10.22 01:15:00.355225 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_704_322} MergeTreeSequentialSource: Reading 2 marks from part 202410_704_704_0, total 78 rows starting from the beginning of the part 2024.10.22 01:15:00.403934 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:00.407196 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:00.491770 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 1.96 GiB, free memory in arenas 6.92 MiB, will set to 1.65 GiB (RSS), difference: 342.37 MiB 2024.10.22 01:15:00.669251 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_346_124, 202410_347_347_0] 2024.10.22 01:15:00.672703 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_346_124, 202410_347_347_0] 2024.10.22 01:15:00.672711 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_393_393_0 (state Temporary) 2024.10.22 01:15:00.688168 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_346_124, 202410_347_347_0] 2024.10.22 01:15:00.764301 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_604_227, 202410_605_605_0] 2024.10.22 01:15:00.767395 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_604_227, 202410_605_605_0] 2024.10.22 01:15:00.776061 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_604_227, 202410_605_605_0] 2024.10.22 01:15:00.945144 [ 1247 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 01:15:00.965376 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_84_17/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:15:01.002410 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_393_393_0 to 202410_400_400_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:01.053235 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 626372 2024.10.22 01:15:01.216475 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_339_70/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:15:01.404691 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_339_70} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_339_70 to 202410_1_339_70 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:01.408025 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_339_70} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_335_69, 202410_339_339_0] -> 202410_1_339_70 2024.10.22 01:15:01.446566 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:15:02.100441 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 6229 2024.10.22 01:15:02.256218 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:02.264635 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:02.288251 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 225 entries to flush up to offset 41262 2024.10.22 01:15:02.300284 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_410_410_0 (state Temporary) 2024.10.22 01:15:02.365841 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_410_410_0 to 202410_423_423_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:02.514693 [ 1237 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:15:02.553721 [ 1237 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:15:02.564614 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:02.567451 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:02.589276 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6229 2024.10.22 01:15:02.661081 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.96 GiB, free memory in arenas 5.21 MiB, will set to 1.23 GiB (RSS), difference: 72.25 MiB 2024.10.22 01:15:02.690909 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_704_322} MergeTask::MergeProjectionsStage: Merge sorted 44090 rows, containing 13 columns (13 merged, 0 gathered) in 2.40114967 sec., 18362.037381867995 rows/sec., 2.92 MiB/sec. 2024.10.22 01:15:02.716276 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:02.720572 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:02.868676 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_704_322} MergedBlockOutputStream: filled checksums 202410_1_704_322 (state Temporary) 2024.10.22 01:15:02.882528 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_404_404_0 (state Temporary) 2024.10.22 01:15:02.964012 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_404_404_0 to 202410_410_410_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:02.986825 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_404_182 to 202410_405_405_0 2024.10.22 01:15:02.992644 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.07 MiB using storage policy from min volume index 0 2024.10.22 01:15:03.000340 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 41262 2024.10.22 01:15:03.014531 [ 1268 ] {} DiskLocal: Reserved 2.07 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:03.035242 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_405_183} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_404_182 to 202410_405_405_0 into Wide with storage Full 2024.10.22 01:15:03.165661 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_704_322} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_704_322 to 202410_1_704_322 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:03.169632 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_704_322} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_703_321, 202410_704_704_0] -> 202410_1_704_322 2024.10.22 01:15:03.184332 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:15:03.204130 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_405_183} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:03.215791 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_405_183} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_404_182, total 39997 rows starting from the beginning of the part 2024.10.22 01:15:03.482511 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_405_183} MergeTreeSequentialSource: Reading 2 marks from part 202410_405_405_0, total 224 rows starting from the beginning of the part 2024.10.22 01:15:04.128020 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 43758 2024.10.22 01:15:04.204642 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 21 2024.10.22 01:15:04.362776 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 1.96 GiB, free memory in arenas 5.15 MiB, will set to 1.27 GiB (RSS), difference: -103.32 MiB 2024.10.22 01:15:04.484677 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:04.487229 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:04.661570 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_21_21_0 (state Temporary) 2024.10.22 01:15:04.672551 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:04.676712 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:04.747087 [ 1206 ] {} DatabaseCatalog: Execution took 353 ms. 2024.10.22 01:15:04.750535 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_21_21_0 to 202410_21_21_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:04.775006 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_697_697_0 (state Temporary) 2024.10.22 01:15:04.880949 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_605_228, 202410_606_606_0] 2024.10.22 01:15:04.882713 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_697_697_0 to 202410_710_710_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:04.883364 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_605_228, 202410_606_606_0] 2024.10.22 01:15:04.904275 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_704_322 to 202410_705_705_0 2024.10.22 01:15:04.926902 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.58 MiB using storage policy from min volume index 0 2024.10.22 01:15:04.929630 [ 1173 ] {} DiskLocal: Reserved 1.58 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:04.958611 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 21 2024.10.22 01:15:04.969696 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_705_323} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_704_322 to 202410_705_705_0 into Compact with storage Full 2024.10.22 01:15:04.999505 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_605_228, 202410_606_606_0] 2024.10.22 01:15:05.033445 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_705_323} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:05.059464 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_705_323} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_704_322, total 44090 rows starting from the beginning of the part 2024.10.22 01:15:05.142356 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_705_323} MergeTreeSequentialSource: Reading 2 marks from part 202410_705_705_0, total 91 rows starting from the beginning of the part 2024.10.22 01:15:05.316816 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43758 2024.10.22 01:15:05.761289 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_405_183} MergeTask::MergeProjectionsStage: Merge sorted 40221 rows, containing 13 columns (13 merged, 0 gathered) in 2.736759623 sec., 14696.577537164285 rows/sec., 6.35 MiB/sec. 2024.10.22 01:15:05.792393 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 349 2024.10.22 01:15:05.800589 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_405_183} MergedBlockOutputStream: filled checksums 202410_1_405_183 (state Temporary) 2024.10.22 01:15:05.972539 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_83_83_0 (state Temporary) 2024.10.22 01:15:06.187531 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:06.190466 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:06.373551 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_331_331_0 (state Temporary) 2024.10.22 01:15:06.474882 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_331_331_0 to 202410_341_341_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:06.601011 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_405_183} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_405_183 to 202410_1_405_183 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:06.609237 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_405_183} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_404_182, 202410_405_405_0] -> 202410_1_405_183 2024.10.22 01:15:06.644413 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.97 MiB. 2024.10.22 01:15:06.648753 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 1.96 GiB, free memory in arenas 5.16 MiB, will set to 1.30 GiB (RSS), difference: 9.26 MiB 2024.10.22 01:15:06.676949 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 349 2024.10.22 01:15:07.146516 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_705_323} MergeTask::MergeProjectionsStage: Merge sorted 44181 rows, containing 13 columns (13 merged, 0 gathered) in 2.18432955 sec., 20226.343593621208 rows/sec., 3.22 MiB/sec. 2024.10.22 01:15:07.196914 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_705_323} MergedBlockOutputStream: filled checksums 202410_1_705_323 (state Temporary) 2024.10.22 01:15:07.278371 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_705_323} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_705_323 to 202410_1_705_323 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:07.280865 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_705_323} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_704_322, 202410_705_705_0] -> 202410_1_705_323 2024.10.22 01:15:07.303301 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:15:08.065249 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1652 entries to flush up to offset 628024 2024.10.22 01:15:08.513382 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 1.96 GiB, free memory in arenas 4.67 MiB, will set to 1.33 GiB (RSS), difference: -38.93 MiB 2024.10.22 01:15:08.731385 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_606_229, 202410_607_607_0] 2024.10.22 01:15:08.738364 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_606_229, 202410_607_607_0] 2024.10.22 01:15:08.854554 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_606_229, 202410_607_607_0] 2024.10.22 01:15:08.965816 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:08.969560 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:09.089107 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_394_394_0 (state Temporary) 2024.10.22 01:15:09.278073 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_394_394_0 to 202410_401_401_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:09.303119 [ 1192 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_396_79 to 202410_401_401_0 2024.10.22 01:15:09.332832 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 43810 2024.10.22 01:15:09.445308 [ 1192 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:09.449825 [ 1192 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:09.470447 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_396_79 to 202410_401_401_0 into Compact with storage Full 2024.10.22 01:15:09.500002 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:09.509682 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 628024 2024.10.22 01:15:09.528481 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} MergeTreeSequentialSource: Reading 79 marks from part 202410_1_396_79, total 635117 rows starting from the beginning of the part 2024.10.22 01:15:09.535818 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_397_397_0, total 1505 rows starting from the beginning of the part 2024.10.22 01:15:09.550341 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_398_398_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:15:09.568963 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_399_399_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:15:09.568849 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_347_125, 202410_348_348_0] 2024.10.22 01:15:09.671920 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_347_125, 202410_348_348_0] 2024.10.22 01:15:09.682242 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_400_400_0, total 1036 rows starting from the beginning of the part 2024.10.22 01:15:09.703243 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_401_401_0, total 1652 rows starting from the beginning of the part 2024.10.22 01:15:09.718071 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_347_125, 202410_348_348_0] 2024.10.22 01:15:09.783208 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:09.786708 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:09.834324 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_698_698_0 (state Temporary) 2024.10.22 01:15:09.887988 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:15:09.972238 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_698_698_0 to 202410_711_711_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:09.989200 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_705_323 to 202410_706_706_0 2024.10.22 01:15:09.992639 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.58 MiB using storage policy from min volume index 0 2024.10.22 01:15:09.995443 [ 1200 ] {} DiskLocal: Reserved 1.58 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:10.023617 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_706_324} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_705_323 to 202410_706_706_0 into Compact with storage Full 2024.10.22 01:15:10.028332 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43810 2024.10.22 01:15:10.052911 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:15:10.059613 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:15:10.095795 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 6245 2024.10.22 01:15:10.095819 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_706_324} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:10.103079 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_706_324} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_705_323, total 44181 rows starting from the beginning of the part 2024.10.22 01:15:10.124557 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_706_324} MergeTreeSequentialSource: Reading 2 marks from part 202410_706_706_0, total 74 rows starting from the beginning of the part 2024.10.22 01:15:10.203218 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:10.207538 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:10.344789 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_411_411_0 (state Temporary) 2024.10.22 01:15:10.388432 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_411_411_0 to 202410_424_424_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:10.544946 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6245 2024.10.22 01:15:10.637079 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 206 entries to flush up to offset 41468 2024.10.22 01:15:10.665712 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.96 GiB, free memory in arenas 5.40 MiB, will set to 1.36 GiB (RSS), difference: -41.61 MiB 2024.10.22 01:15:10.679781 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:15:10.896179 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:10.899717 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:10.975555 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_405_405_0 (state Temporary) 2024.10.22 01:15:11.075502 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_405_405_0 to 202410_411_411_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:11.229323 [ 1168 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_405_183 to 202410_406_406_0 2024.10.22 01:15:11.240772 [ 1168 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.08 MiB using storage policy from min volume index 0 2024.10.22 01:15:11.247454 [ 1168 ] {} DiskLocal: Reserved 2.08 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:15:11.272772 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_406_184} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_405_183 to 202410_406_406_0 into Wide with storage Full 2024.10.22 01:15:11.424854 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_406_184} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:11.567635 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 41468 2024.10.22 01:15:11.588081 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_406_184} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_405_183, total 40221 rows starting from the beginning of the part 2024.10.22 01:15:11.785230 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(8ebd86a27e1c213acc0247bdccc32e8f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8ebd86a27e1c213acc0247bdccc32e8f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:15:11.786452 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 19002, file segment info: File segment: [0, 33554431], key: 8ebd86a27e1c213acc0247bdccc32e8f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:15:11.789337 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:11.942149 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_406_184} MergeTreeSequentialSource: Reading 2 marks from part 202410_406_406_0, total 220 rows starting from the beginning of the part 2024.10.22 01:15:12.296532 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_401_80/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:15:12.330123 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_336_67, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0, 202410_340_340_0, 202410_341_341_0] 2024.10.22 01:15:12.332498 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_336_67, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0, 202410_340_340_0, 202410_341_341_0] 2024.10.22 01:15:12.373242 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_336_67, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0, 202410_340_340_0, 202410_341_341_0] 2024.10.22 01:15:12.404035 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_706_324} MergeTask::MergeProjectionsStage: Merge sorted 44255 rows, containing 13 columns (13 merged, 0 gathered) in 2.390048354 sec., 18516.361782360826 rows/sec., 2.95 MiB/sec. 2024.10.22 01:15:12.449022 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_706_324} MergedBlockOutputStream: filled checksums 202410_1_706_324 (state Temporary) 2024.10.22 01:15:12.505126 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.96 GiB, free memory in arenas 9.20 MiB, will set to 1.40 GiB (RSS), difference: -20.81 MiB 2024.10.22 01:15:12.798860 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_706_324} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_706_324 to 202410_1_706_324 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:12.801704 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_706_324} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_705_323, 202410_706_706_0] -> 202410_1_706_324 2024.10.22 01:15:12.835772 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:15:12.836875 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(9e18c84d264447d6bcf0454caac20222) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9e18c84d264447d6bcf0454caac20222, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:15:12.839080 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 355, file segment info: File segment: [0, 33554431], key: 9e18c84d264447d6bcf0454caac20222, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:15:12.842503 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:13.579224 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(69a64914ed0da6057de6fc86644063f2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 69a64914ed0da6057de6fc86644063f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:15:13.582687 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1168, file segment info: File segment: [0, 33554431], key: 69a64914ed0da6057de6fc86644063f2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:15:13.587125 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:14.041477 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 43880 2024.10.22 01:15:14.074761 [ 1694 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fki/cmcycbnwcockxlgwfmwfckzjsyttm, size 12247217 2024.10.22 01:15:14.168774 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:15:14.174523 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:15:14.178327 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 350 2024.10.22 01:15:14.182178 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:15:14.222497 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:15:14.233177 [ 1274 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 01:15:14.240031 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:15:14.255357 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:15:14.276111 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:15:14.284828 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:14.291065 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:14.292862 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:15:14.338206 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:15:14.388683 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:14.394803 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:14.403281 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_699_699_0 (state Temporary) 2024.10.22 01:15:14.451420 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(879bb8d688a49081b459de19c8a3fa04) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 879bb8d688a49081b459de19c8a3fa04, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:15:14.489761 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 5185, file segment info: File segment: [0, 33554431], key: 879bb8d688a49081b459de19c8a3fa04, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:15:14.496264 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:14.510645 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qif/sftyspystusxrvdclsgrxcwsamhzt (LogSeriesLimiter: on interval from 2024-10-22 01:14:52 to 2024-10-22 01:15:14 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:15:14.522648 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_607_230, 202410_608_608_0] 2024.10.22 01:15:14.526848 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_607_230, 202410_608_608_0] 2024.10.22 01:15:14.537576 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_607_230, 202410_608_608_0] 2024.10.22 01:15:14.568003 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_699_699_0 to 202410_712_712_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:14.571796 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_332_332_0 (state Temporary) 2024.10.22 01:15:14.594703 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_706_324 to 202410_707_707_0 2024.10.22 01:15:14.600403 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.58 MiB using storage policy from min volume index 0 2024.10.22 01:15:14.608195 [ 1150 ] {} DiskLocal: Reserved 1.58 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:14.666431 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_707_325} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_706_324 to 202410_707_707_0 into Compact with storage Full 2024.10.22 01:15:14.667416 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43880 2024.10.22 01:15:14.667497 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_332_332_0 to 202410_342_342_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:14.695325 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_707_325} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:14.739471 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_707_325} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_706_324, total 44255 rows starting from the beginning of the part 2024.10.22 01:15:14.740139 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 350 2024.10.22 01:15:14.799749 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_707_325} MergeTreeSequentialSource: Reading 2 marks from part 202410_707_707_0, total 86 rows starting from the beginning of the part 2024.10.22 01:15:14.971099 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_406_184} MergeTask::MergeProjectionsStage: Merge sorted 40441 rows, containing 13 columns (13 merged, 0 gathered) in 3.709814831 sec., 10901.083165139786 rows/sec., 4.71 MiB/sec. 2024.10.22 01:15:15.032744 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_406_184} MergedBlockOutputStream: filled checksums 202410_1_406_184 (state Temporary) 2024.10.22 01:15:15.378725 [ 1270 ] {} DatabaseCatalog: Execution took 631 ms. 2024.10.22 01:15:15.487514 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(521bac2dd3f3529ee5ad4fb46c019204) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 521bac2dd3f3529ee5ad4fb46c019204, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:15:15.489610 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 504, file segment info: File segment: [0, 33554431], key: 521bac2dd3f3529ee5ad4fb46c019204, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:15:15.492235 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:15.691984 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_83_83_0 to 202410_86_86_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:15.774907 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_84_17/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:15:15.851400 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_406_184} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_406_184 to 202410_1_406_184 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:15.874955 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_406_184} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_405_183, 202410_406_406_0] -> 202410_1_406_184 2024.10.22 01:15:15.898751 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.98 MiB. 2024.10.22 01:15:15.923301 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ShareService 2024.10.22 01:15:15.999565 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ShareService 2024.10.22 01:15:16.131068 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ShareService 2024.10.22 01:15:16.156850 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ShareService 2024.10.22 01:15:16.183885 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ShareService 2024.10.22 01:15:16.205145 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ShareService 2024.10.22 01:15:16.253165 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ShareService 2024.10.22 01:15:16.440625 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.96 GiB, free memory in arenas 14.42 MiB, will set to 1.49 GiB (RSS), difference: -64.36 MiB 2024.10.22 01:15:16.516329 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 629176 2024.10.22 01:15:16.585261 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 259.80 MiB using storage policy from min volume index 0 2024.10.22 01:15:17.494707 [ 1222 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 251 ms. 2024.10.22 01:15:17.530290 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_348_126, 202410_349_349_0] 2024.10.22 01:15:17.534493 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_348_126, 202410_349_349_0] 2024.10.22 01:15:17.555389 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_348_126, 202410_349_349_0] 2024.10.22 01:15:17.562932 [ 1235 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:15:17.607162 [ 1235 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:15:17.714514 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:17.719963 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:17.775205 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_707_325/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:15:17.831922 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_707_325} MergeTask::MergeProjectionsStage: Merge sorted 44341 rows, containing 13 columns (13 merged, 0 gathered) in 3.189807003 sec., 13900.841009596343 rows/sec., 2.21 MiB/sec. 2024.10.22 01:15:18.054152 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 6256 2024.10.22 01:15:18.089960 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_395_395_0 (state Temporary) 2024.10.22 01:15:18.184655 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_395_395_0 to 202410_402_402_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:18.188298 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:18.190186 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:18.223207 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_707_325} MergedBlockOutputStream: filled checksums 202410_1_707_325 (state Temporary) 2024.10.22 01:15:18.321170 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 629176 2024.10.22 01:15:18.389035 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_707_325} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_707_325 to 202410_1_707_325 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:18.392760 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_707_325} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_706_324, 202410_707_707_0] -> 202410_1_707_325 2024.10.22 01:15:18.441180 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_412_412_0 (state Temporary) 2024.10.22 01:15:18.489387 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_412_412_0 to 202410_425_425_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:18.574574 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 15.39 MiB, will set to 1.48 GiB (RSS), difference: -35.23 MiB 2024.10.22 01:15:18.647238 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6256 2024.10.22 01:15:18.668996 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_608_231, 202410_609_609_0] 2024.10.22 01:15:18.677314 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 43961 2024.10.22 01:15:18.678900 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_608_231, 202410_609_609_0] 2024.10.22 01:15:18.688922 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:15:18.687617 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_608_231, 202410_609_609_0] 2024.10.22 01:15:19.015147 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hqq/kbgmotevhfpbqkebcruqcqzludyge (LogSeriesLimiter: on interval from 2024-10-22 01:15:14 to 2024-10-22 01:15:19 accepted series 1 / 50 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:15:19.031419 [ 1228 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 01:15:19.039728 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:19.041823 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:19.106578 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 232 entries to flush up to offset 41700 2024.10.22 01:15:19.129350 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_700_700_0 (state Temporary) 2024.10.22 01:15:19.098164 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3305 2024.10.22 01:15:19.279701 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_700_700_0 to 202410_713_713_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:19.333046 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 43961 2024.10.22 01:15:19.387202 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:19.387954 [ 1232 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_707_325 to 202410_708_708_0 2024.10.22 01:15:19.390428 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:19.391687 [ 1232 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.59 MiB using storage policy from min volume index 0 2024.10.22 01:15:19.397004 [ 1232 ] {} DiskLocal: Reserved 1.59 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:19.487130 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_406_406_0 (state Temporary) 2024.10.22 01:15:19.555443 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_406_406_0 to 202410_412_412_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:19.581143 [ 1234 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_406_184 to 202410_407_407_0 2024.10.22 01:15:19.588062 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 41700 2024.10.22 01:15:19.593665 [ 1234 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.09 MiB using storage policy from min volume index 0 2024.10.22 01:15:19.596125 [ 1234 ] {} DiskLocal: Reserved 2.09 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:19.634240 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_407_185} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_406_184 to 202410_407_407_0 into Wide with storage Full 2024.10.22 01:15:19.631267 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_708_326} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_707_325 to 202410_708_708_0 into Compact with storage Full 2024.10.22 01:15:19.821249 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_708_326} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:19.832862 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_708_326} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_707_325, total 44341 rows starting from the beginning of the part 2024.10.22 01:15:19.876219 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_708_326} MergeTreeSequentialSource: Reading 2 marks from part 202410_708_708_0, total 77 rows starting from the beginning of the part 2024.10.22 01:15:19.882869 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_407_185} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:19.891515 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_407_185} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_406_184, total 40441 rows starting from the beginning of the part 2024.10.22 01:15:20.075299 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_407_185} MergeTreeSequentialSource: Reading 2 marks from part 202410_407_407_0, total 217 rows starting from the beginning of the part 2024.10.22 01:15:20.151873 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:15:20.260773 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:15:20.404693 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.96 GiB, free memory in arenas 11.97 MiB, will set to 1.48 GiB (RSS), difference: -178.91 MiB 2024.10.22 01:15:22.243761 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 351 2024.10.22 01:15:22.260313 [ 1250 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Selected 5 parts from 202410_1_17_4 to 202410_21_21_0 2024.10.22 01:15:22.277471 [ 1250 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:22.280188 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:22.290377 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_361_72, 202410_362_362_0, 202410_363_363_0, 202410_364_364_0, 202410_365_365_0, 202410_366_366_0] 2024.10.22 01:15:22.293829 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_361_72, 202410_362_362_0, 202410_363_363_0, 202410_364_364_0, 202410_365_365_0, 202410_366_366_0] 2024.10.22 01:15:22.304059 [ 1400 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_21_5} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_17_4 to 202410_21_21_0 into Compact with storage Full 2024.10.22 01:15:22.354896 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_407_185} MergeTask::MergeProjectionsStage: Merge sorted 40658 rows, containing 13 columns (13 merged, 0 gathered) in 2.744510096 sec., 14814.301488362971 rows/sec., 6.40 MiB/sec. 2024.10.22 01:15:22.435602 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.96 GiB, free memory in arenas 24.54 MiB, will set to 1.50 GiB (RSS), difference: -124.11 MiB 2024.10.22 01:15:22.461713 [ 1400 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_21_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:22.472037 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:22.473514 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:22.536874 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_333_333_0 (state Temporary) 2024.10.22 01:15:22.537593 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_361_72, 202410_362_362_0, 202410_363_363_0, 202410_364_364_0, 202410_365_365_0, 202410_366_366_0] 2024.10.22 01:15:22.588873 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_401_80/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:15:22.590805 [ 1400 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_21_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_17_4, total 17 rows starting from the beginning of the part 2024.10.22 01:15:22.719907 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_333_333_0 to 202410_343_343_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:22.763860 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_407_185} MergedBlockOutputStream: filled checksums 202410_1_407_185 (state Temporary) 2024.10.22 01:15:22.888146 [ 1400 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_21_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_18_18_0, total 1 rows starting from the beginning of the part 2024.10.22 01:15:22.892969 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 351 2024.10.22 01:15:22.976244 [ 1400 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_21_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_19_19_0, total 1 rows starting from the beginning of the part 2024.10.22 01:15:22.982104 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_708_326} MergeTask::MergeProjectionsStage: Merge sorted 44418 rows, containing 13 columns (13 merged, 0 gathered) in 3.44549278 sec., 12891.624750407982 rows/sec., 2.05 MiB/sec. 2024.10.22 01:15:23.145677 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_708_326} MergedBlockOutputStream: filled checksums 202410_1_708_326 (state Temporary) 2024.10.22 01:15:23.184640 [ 1400 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_21_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_20_20_0, total 1 rows starting from the beginning of the part 2024.10.22 01:15:23.336326 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_708_326} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_708_326 to 202410_1_708_326 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:23.357604 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 44021 2024.10.22 01:15:23.362837 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_708_326} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_707_325, 202410_708_708_0] -> 202410_1_708_326 2024.10.22 01:15:23.435054 [ 1400 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_21_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_21_21_0, total 1 rows starting from the beginning of the part 2024.10.22 01:15:23.486339 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:15:23.549178 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.96 GiB, free memory in arenas 21.22 MiB, will set to 1.48 GiB (RSS), difference: -40.95 MiB 2024.10.22 01:15:23.585572 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:23.589005 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:23.671857 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_701_701_0 (state Temporary) 2024.10.22 01:15:23.803578 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_701_701_0 to 202410_714_714_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:23.836758 [ 1159 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_708_326 to 202410_709_709_0 2024.10.22 01:15:23.854611 [ 1159 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.59 MiB using storage policy from min volume index 0 2024.10.22 01:15:23.864511 [ 1159 ] {} DiskLocal: Reserved 1.59 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:23.959316 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44021 2024.10.22 01:15:23.978741 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_407_185} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_407_185 to 202410_1_407_185 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:23.981190 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_709_327} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_708_326 to 202410_709_709_0 into Compact with storage Full 2024.10.22 01:15:23.982900 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_407_185} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_406_184, 202410_407_407_0] -> 202410_1_407_185 2024.10.22 01:15:24.025508 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_709_327} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:24.034516 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_709_327} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_708_326, total 44418 rows starting from the beginning of the part 2024.10.22 01:15:24.035027 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ghu/ioxvqmpxylmhnveipkhitaopcnrio (LogSeriesLimiter: on interval from 2024-10-22 01:15:19 to 2024-10-22 01:15:24 accepted series 1 / 228 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:15:24.113602 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.98 MiB. 2024.10.22 01:15:24.177250 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_709_327} MergeTreeSequentialSource: Reading 2 marks from part 202410_709_709_0, total 99 rows starting from the beginning of the part 2024.10.22 01:15:24.672823 [ 1388 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_21_5} MergeTask::MergeProjectionsStage: Merge sorted 21 rows, containing 27 columns (27 merged, 0 gathered) in 2.375894815 sec., 8.838775129024388 rows/sec., 28.42 KiB/sec. 2024.10.22 01:15:24.691166 [ 1400 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_21_5} MergedBlockOutputStream: filled checksums 202410_1_21_5 (state Temporary) 2024.10.22 01:15:24.999482 [ 1397 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_21_5} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_merge_202410_1_21_5 to 202410_1_21_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:25.005123 [ 1397 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_21_5} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Merged 5 parts: [202410_1_17_4, 202410_21_21_0] -> 202410_1_21_5 2024.10.22 01:15:25.041293 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.24 MiB. 2024.10.22 01:15:25.277251 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_609_232, 202410_610_610_0] 2024.10.22 01:15:25.279253 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_609_232, 202410_610_610_0] 2024.10.22 01:15:25.288052 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_609_232, 202410_610_610_0] 2024.10.22 01:15:25.324660 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 631096 2024.10.22 01:15:25.572602 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.96 GiB, free memory in arenas 21.68 MiB, will set to 1.48 GiB (RSS), difference: -169.22 MiB 2024.10.22 01:15:25.603267 [ 1186 ] {} DatabaseCatalog: Execution took 223 ms. 2024.10.22 01:15:25.742685 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_349_127, 202410_350_350_0] 2024.10.22 01:15:25.747623 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_349_127, 202410_350_350_0] 2024.10.22 01:15:25.812293 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:25.821296 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:25.864407 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_349_127, 202410_350_350_0] 2024.10.22 01:15:26.151543 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 6277 2024.10.22 01:15:26.290545 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_396_396_0 (state Temporary) 2024.10.22 01:15:26.295834 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_709_327} MergeTask::MergeProjectionsStage: Merge sorted 44517 rows, containing 13 columns (13 merged, 0 gathered) in 2.423027293 sec., 18372.471547723504 rows/sec., 2.93 MiB/sec. 2024.10.22 01:15:26.358914 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_709_327} MergedBlockOutputStream: filled checksums 202410_1_709_327 (state Temporary) 2024.10.22 01:15:26.375559 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:26.381385 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:26.533211 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_413_413_0 (state Temporary) 2024.10.22 01:15:26.580011 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_709_327} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_709_327 to 202410_1_709_327 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:26.585309 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_396_396_0 to 202410_403_403_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:26.585601 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_709_327} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_708_326, 202410_709_709_0] -> 202410_1_709_327 2024.10.22 01:15:26.617397 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 631096 2024.10.22 01:15:26.674566 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 3345 2024.10.22 01:15:26.698730 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_413_413_0 to 202410_426_426_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:26.717174 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.54 MiB. 2024.10.22 01:15:26.736829 [ 1185 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_421_84 to 202410_426_426_0 2024.10.22 01:15:26.749514 [ 1185 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:26.764274 [ 1185 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:26.773736 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6277 2024.10.22 01:15:26.792075 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_421_84 to 202410_426_426_0 into Compact with storage Full 2024.10.22 01:15:26.852351 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:26.865456 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_421_84, total 6289 rows starting from the beginning of the part 2024.10.22 01:15:26.885377 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_422_422_0, total 24 rows starting from the beginning of the part 2024.10.22 01:15:26.903033 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_423_423_0, total 22 rows starting from the beginning of the part 2024.10.22 01:15:26.931407 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_424_424_0, total 16 rows starting from the beginning of the part 2024.10.22 01:15:26.970873 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_425_425_0, total 11 rows starting from the beginning of the part 2024.10.22 01:15:27.004925 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_426_426_0, total 21 rows starting from the beginning of the part 2024.10.22 01:15:27.166588 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 233 entries to flush up to offset 41933 2024.10.22 01:15:27.316368 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 01:15:27.383916 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 19.31 MiB, will set to 1.49 GiB (RSS), difference: 2.64 MiB 2024.10.22 01:15:27.487791 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:27.491804 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:27.788823 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_407_407_0 (state Temporary) 2024.10.22 01:15:27.966159 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 44091 2024.10.22 01:15:27.981933 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_407_407_0 to 202410_413_413_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:28.047823 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 41933 2024.10.22 01:15:28.131863 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} MergeTask::MergeProjectionsStage: Merge sorted 6383 rows, containing 12 columns (12 merged, 0 gathered) in 1.343683844 sec., 4750.373407034876 rows/sec., 752.16 KiB/sec. 2024.10.22 01:15:28.170615 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_407_185 to 202410_408_408_0 2024.10.22 01:15:28.180582 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.10 MiB using storage policy from min volume index 0 2024.10.22 01:15:28.181756 [ 1200 ] {} DiskLocal: Reserved 2.10 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:28.229790 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_408_186} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_407_185 to 202410_408_408_0 into Wide with storage Full 2024.10.22 01:15:28.305362 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:28.309206 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_408_186} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:28.311225 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:28.323387 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_408_186} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_407_185, total 40658 rows starting from the beginning of the part 2024.10.22 01:15:28.472058 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_408_186} MergeTreeSequentialSource: Reading 2 marks from part 202410_408_408_0, total 248 rows starting from the beginning of the part 2024.10.22 01:15:28.475355 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_702_702_0 (state Temporary) 2024.10.22 01:15:28.475886 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} MergedBlockOutputStream: filled checksums 202410_1_426_85 (state Temporary) 2024.10.22 01:15:28.552923 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_702_702_0 to 202410_715_715_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:28.593248 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_709_327 to 202410_710_710_0 2024.10.22 01:15:28.643480 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44091 2024.10.22 01:15:28.645764 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.59 MiB using storage policy from min volume index 0 2024.10.22 01:15:28.650273 [ 1264 ] {} DiskLocal: Reserved 1.59 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:28.777045 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_710_328} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_709_327 to 202410_710_710_0 into Compact with storage Full 2024.10.22 01:15:28.823378 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_710_328} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:28.835085 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_710_328} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_709_327, total 44517 rows starting from the beginning of the part 2024.10.22 01:15:28.860232 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_710_328} MergeTreeSequentialSource: Reading 2 marks from part 202410_710_710_0, total 72 rows starting from the beginning of the part 2024.10.22 01:15:29.295454 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_426_85/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:15:29.422632 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_426_85 to 202410_1_426_85 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:29.432328 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_426_85} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_421_84, 202410_426_426_0] -> 202410_1_426_85 2024.10.22 01:15:29.498929 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.46 MiB. 2024.10.22 01:15:29.766486 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.96 GiB, free memory in arenas 13.52 MiB, will set to 1.50 GiB (RSS), difference: -64.83 MiB 2024.10.22 01:15:30.053404 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:15:30.061370 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:15:30.189164 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_610_233, 202410_611_611_0] 2024.10.22 01:15:30.191685 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_610_233, 202410_611_611_0] 2024.10.22 01:15:30.207377 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_610_233, 202410_611_611_0] 2024.10.22 01:15:30.400279 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 352 2024.10.22 01:15:30.643252 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:30.646673 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:30.808971 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_334_334_0 (state Temporary) 2024.10.22 01:15:30.989177 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_334_334_0 to 202410_344_344_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:31.034593 [ 1153 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_339_70 to 202410_344_344_0 2024.10.22 01:15:31.062457 [ 1153 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:31.080389 [ 1153 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:31.171872 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_339_70 to 202410_344_344_0 into Compact with storage Full 2024.10.22 01:15:31.184535 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 352 2024.10.22 01:15:31.287070 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_408_186} MergeTask::MergeProjectionsStage: Merge sorted 40906 rows, containing 13 columns (13 merged, 0 gathered) in 3.095486883 sec., 13214.722447912889 rows/sec., 5.71 MiB/sec. 2024.10.22 01:15:31.383430 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_408_186} MergedBlockOutputStream: filled checksums 202410_1_408_186 (state Temporary) 2024.10.22 01:15:31.667132 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_84_17/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:15:31.679575 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:31.738037 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_339_70, total 383 rows starting from the beginning of the part 2024.10.22 01:15:31.893555 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_340_340_0, total 1 rows starting from the beginning of the part 2024.10.22 01:15:32.115901 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_341_341_0, total 1 rows starting from the beginning of the part 2024.10.22 01:15:32.181310 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_408_186} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_408_186 to 202410_1_408_186 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:32.185287 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_408_186} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_407_185, 202410_408_408_0] -> 202410_1_408_186 2024.10.22 01:15:32.204297 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.00 MiB. 2024.10.22 01:15:32.231270 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_342_342_0, total 1 rows starting from the beginning of the part 2024.10.22 01:15:32.291928 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(989d035a5c7d674edd359d493cddead3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 989d035a5c7d674edd359d493cddead3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:15:32.293820 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 576373, file segment info: File segment: [0, 33554431], key: 989d035a5c7d674edd359d493cddead3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:15:32.298057 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:32.391296 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.96 GiB, free memory in arenas 16.93 MiB, will set to 1.50 GiB (RSS), difference: 59.02 MiB 2024.10.22 01:15:32.560457 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_710_328} MergeTask::MergeProjectionsStage: Merge sorted 44589 rows, containing 13 columns (13 merged, 0 gathered) in 3.880046292 sec., 11491.873200568505 rows/sec., 1.83 MiB/sec. 2024.10.22 01:15:32.575642 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_343_343_0, total 1 rows starting from the beginning of the part 2024.10.22 01:15:32.617392 [ 1150 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:15:32.636879 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_710_328} MergedBlockOutputStream: filled checksums 202410_1_710_328 (state Temporary) 2024.10.22 01:15:32.652001 [ 1150 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:15:32.662483 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 44154 2024.10.22 01:15:32.688565 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_350_128, 202410_351_351_0] 2024.10.22 01:15:32.694601 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_350_128, 202410_351_351_0] 2024.10.22 01:15:32.725333 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_350_128, 202410_351_351_0] 2024.10.22 01:15:32.774927 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_710_328} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_710_328 to 202410_1_710_328 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:32.778047 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_710_328} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_709_327, 202410_710_710_0] -> 202410_1_710_328 2024.10.22 01:15:32.809052 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:15:32.875719 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_401_80/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:15:32.956838 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_344_344_0, total 1 rows starting from the beginning of the part 2024.10.22 01:15:32.961363 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:32.966032 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:33.028619 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_703_703_0 (state Temporary) 2024.10.22 01:15:33.198373 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} MergeTask::MergeProjectionsStage: Merge sorted 641614 rows, containing 4 columns (4 merged, 0 gathered) in 23.73687588 sec., 27030.263091218556 rows/sec., 396.22 KiB/sec. 2024.10.22 01:15:33.286106 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} MergedBlockOutputStream: filled checksums 202410_1_401_80 (state Temporary) 2024.10.22 01:15:33.322597 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_401_80 to 202410_1_401_80 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:33.323532 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a59b789b9b488cc11bd5fa8d6a6533a7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a59b789b9b488cc11bd5fa8d6a6533a7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:15:33.327526 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_401_80} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_396_79, 202410_401_401_0] -> 202410_1_401_80 2024.10.22 01:15:33.369682 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 580, file segment info: File segment: [0, 33554431], key: a59b789b9b488cc11bd5fa8d6a6533a7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:15:33.371973 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:33.415828 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:15:33.416836 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_611_234, 202410_612_612_0] 2024.10.22 01:15:33.418447 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_703_703_0 to 202410_716_716_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:33.431176 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_611_234, 202410_612_612_0] 2024.10.22 01:15:33.441092 [ 1708 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qif/sftyspystusxrvdclsgrxcwsamhzt, size 576373 2024.10.22 01:15:33.462356 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_611_234, 202410_612_612_0] 2024.10.22 01:15:33.475028 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_710_328 to 202410_711_711_0 2024.10.22 01:15:33.487708 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.59 MiB using storage policy from min volume index 0 2024.10.22 01:15:33.492498 [ 1269 ] {} DiskLocal: Reserved 1.59 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:33.554686 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44154 2024.10.22 01:15:33.633185 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1192 entries to flush up to offset 632288 2024.10.22 01:15:33.777574 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:15:33.794378 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:15:33.814618 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:15:33.822547 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_711_329} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_710_328 to 202410_711_711_0 into Compact with storage Full 2024.10.22 01:15:33.846645 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:15:33.877727 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:15:33.899215 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:15:33.931533 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:15:33.999222 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_711_329} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:34.028896 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_711_329} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_710_328, total 44589 rows starting from the beginning of the part 2024.10.22 01:15:34.063138 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:15:34.083462 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:15:34.084505 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_711_329} MergeTreeSequentialSource: Reading 2 marks from part 202410_711_711_0, total 52 rows starting from the beginning of the part 2024.10.22 01:15:34.282842 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 29 entries to flush up to offset 6306 2024.10.22 01:15:34.373828 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_344_71/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:15:34.375883 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rqe/uhktjlphpxtywbffcraunnjfgetmj (LogSeriesLimiter: on interval from 2024-10-22 01:15:24 to 2024-10-22 01:15:34 accepted series 1 / 83 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:15:34.476299 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:34.484016 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:34.640530 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:34.643318 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:34.695081 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_414_414_0 (state Temporary) 2024.10.22 01:15:34.734865 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_397_397_0 (state Temporary) 2024.10.22 01:15:34.747409 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_414_414_0 to 202410_427_427_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:34.842519 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_397_397_0 to 202410_404_404_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:34.903403 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 632288 2024.10.22 01:15:34.948407 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6306 2024.10.22 01:15:35.475913 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_291_60, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0, 202410_296_296_0] 2024.10.22 01:15:35.480220 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_291_60, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0, 202410_296_296_0] 2024.10.22 01:15:35.503286 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_291_60, 202410_292_292_0, 202410_293_293_0, 202410_294_294_0, 202410_295_295_0, 202410_296_296_0] 2024.10.22 01:15:35.623347 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 13.18 MiB, will set to 1.51 GiB (RSS), difference: -1.70 MiB 2024.10.22 01:15:35.818699 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 240 entries to flush up to offset 42173 2024.10.22 01:15:35.970564 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} MergeTask::MergeProjectionsStage: Merge sorted 388 rows, containing 32 columns (32 merged, 0 gathered) in 4.848154343 sec., 80.03045541654696 rows/sec., 8.09 KiB/sec. 2024.10.22 01:15:36.002233 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} MergedBlockOutputStream: filled checksums 202410_1_344_71 (state Temporary) 2024.10.22 01:15:36.253517 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_344_71 to 202410_1_344_71 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:36.260415 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_344_71} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_339_70, 202410_344_344_0] -> 202410_1_344_71 2024.10.22 01:15:36.411515 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:15:36.431747 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_711_329} MergeTask::MergeProjectionsStage: Merge sorted 44641 rows, containing 13 columns (13 merged, 0 gathered) in 2.663062855 sec., 16763.029049871973 rows/sec., 2.67 MiB/sec. 2024.10.22 01:15:36.499433 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:36.511008 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:36.516324 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_711_329} MergedBlockOutputStream: filled checksums 202410_1_711_329 (state Temporary) 2024.10.22 01:15:36.789629 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_408_408_0 (state Temporary) 2024.10.22 01:15:36.946487 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_711_329} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_711_329 to 202410_1_711_329 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:36.949293 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_711_329} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_710_328, 202410_711_711_0] -> 202410_1_711_329 2024.10.22 01:15:36.971495 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_408_408_0 to 202410_414_414_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:36.972547 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:15:36.989418 [ 1232 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_408_186 to 202410_409_409_0 2024.10.22 01:15:36.995683 [ 1232 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.11 MiB using storage policy from min volume index 0 2024.10.22 01:15:37.012148 [ 1232 ] {} DiskLocal: Reserved 2.11 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:37.022169 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_409_187} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_408_186 to 202410_409_409_0 into Wide with storage Full 2024.10.22 01:15:37.079399 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_409_187} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:37.083951 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_409_187} MergeTreeSequentialSource: Reading 6 marks from part 202410_1_408_186, total 40906 rows starting from the beginning of the part 2024.10.22 01:15:37.233526 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 42173 2024.10.22 01:15:37.406909 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_409_187} MergeTreeSequentialSource: Reading 2 marks from part 202410_409_409_0, total 235 rows starting from the beginning of the part 2024.10.22 01:15:37.423622 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(3c6c279f279f937e4d2a916a54298397) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3c6c279f279f937e4d2a916a54298397, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:15:37.432792 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 141, file segment info: File segment: [0, 33554431], key: 3c6c279f279f937e4d2a916a54298397, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:15:37.436527 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:37.567320 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 44241 2024.10.22 01:15:37.591003 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.96 GiB, free memory in arenas 10.91 MiB, will set to 1.51 GiB (RSS), difference: 11.59 MiB 2024.10.22 01:15:37.957106 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:37.960347 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:38.028611 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(608eb24272c0637d0374933f76aa4815) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 608eb24272c0637d0374933f76aa4815, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:15:38.035708 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 152, file segment info: File segment: [0, 33554431], key: 608eb24272c0637d0374933f76aa4815, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:15:38.039374 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:38.074986 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_704_704_0 (state Temporary) 2024.10.22 01:15:38.328581 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_704_704_0 to 202410_717_717_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:38.370627 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_711_329 to 202410_712_712_0 2024.10.22 01:15:38.375393 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.60 MiB using storage policy from min volume index 0 2024.10.22 01:15:38.377670 [ 1257 ] {} DiskLocal: Reserved 1.60 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:15:38.419489 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_712_330} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_711_329 to 202410_712_712_0 into Compact with storage Full 2024.10.22 01:15:38.493263 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44241 2024.10.22 01:15:38.636844 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_712_330} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:38.642490 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_712_330} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_711_329, total 44641 rows starting from the beginning of the part 2024.10.22 01:15:38.674237 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_712_330} MergeTreeSequentialSource: Reading 2 marks from part 202410_712_712_0, total 70 rows starting from the beginning of the part 2024.10.22 01:15:38.687300 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(b43a6142ba6a4869b501f59394736e83) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b43a6142ba6a4869b501f59394736e83, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:15:38.689753 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 359, file segment info: File segment: [0, 33554431], key: b43a6142ba6a4869b501f59394736e83, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:15:38.691899 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:39.203197 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_712_330/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:15:39.407465 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_612_235, 202410_613_613_0] 2024.10.22 01:15:39.417292 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_612_235, 202410_613_613_0] 2024.10.22 01:15:39.430667 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_612_235, 202410_613_613_0] 2024.10.22 01:15:39.621812 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.96 GiB, free memory in arenas 11.77 MiB, will set to 1.53 GiB (RSS), difference: -23.11 MiB 2024.10.22 01:15:39.668052 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_409_187/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:15:39.746963 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(8c43559654b20dfe5c3459c28e0284a3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8c43559654b20dfe5c3459c28e0284a3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:15:39.753421 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 464, file segment info: File segment: [0, 33554431], key: 8c43559654b20dfe5c3459c28e0284a3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:15:39.761034 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:40.063559 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:15:40.067228 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:15:40.183054 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:15:40.190464 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:15:40.244773 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:15:40.267390 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_409_187} MergeTask::MergeProjectionsStage: Merge sorted 41141 rows, containing 13 columns (13 merged, 0 gathered) in 3.249854438 sec., 12659.336221015079 rows/sec., 5.47 MiB/sec. 2024.10.22 01:15:40.289233 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:15:40.334716 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:15:40.348561 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_409_187} MergedBlockOutputStream: filled checksums 202410_1_409_187 (state Temporary) 2024.10.22 01:15:40.376428 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:15:40.424040 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:15:40.632032 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/flk/flxpahovvevadryxjfanrzkwqgonm (LogSeriesLimiter: on interval from 2024-10-22 01:15:34 to 2024-10-22 01:15:40 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:15:40.939777 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_409_187} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_409_187 to 202410_1_409_187 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:40.944260 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_409_187} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_408_186, 202410_409_409_0] -> 202410_1_409_187 2024.10.22 01:15:40.972580 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 63.98 MiB. 2024.10.22 01:15:41.271996 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_712_330} MergeTask::MergeProjectionsStage: Merge sorted 44711 rows, containing 13 columns (13 merged, 0 gathered) in 2.868911772 sec., 15584.654933055223 rows/sec., 2.48 MiB/sec. 2024.10.22 01:15:41.453734 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_712_330} MergedBlockOutputStream: filled checksums 202410_1_712_330 (state Temporary) 2024.10.22 01:15:41.486764 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.96 GiB, free memory in arenas 7.40 MiB, will set to 1.51 GiB (RSS), difference: 25.99 MiB 2024.10.22 01:15:41.562245 [ 1157 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 01:15:41.659222 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_712_330} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_712_330 to 202410_1_712_330 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:41.663268 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_712_330} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_711_329, 202410_712_712_0] -> 202410_1_712_330 2024.10.22 01:15:41.784461 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:15:41.913265 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1709 entries to flush up to offset 633997 2024.10.22 01:15:42.148706 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:15:42.451551 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 6326 2024.10.22 01:15:42.509283 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 44295 2024.10.22 01:15:42.698261 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:42.702716 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:15:42.880183 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:42.882916 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:15:42.933724 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_415_415_0 (state Temporary) 2024.10.22 01:15:42.971368 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:42.974943 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:15:43.032518 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_398_398_0 (state Temporary) 2024.10.22 01:15:43.079039 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_705_705_0 (state Temporary) 2024.10.22 01:15:43.092450 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_415_415_0 to 202410_428_428_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:43.132559 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_398_398_0 to 202410_405_405_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:43.183453 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6326 2024.10.22 01:15:43.193555 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 633997 2024.10.22 01:15:43.345740 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_705_705_0 to 202410_718_718_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:43.501081 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_712_330 to 202410_713_713_0 2024.10.22 01:15:43.513591 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.60 MiB using storage policy from min volume index 0 2024.10.22 01:15:43.535106 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44295 2024.10.22 01:15:43.559359 [ 1186 ] {} DiskLocal: Reserved 1.60 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:15:43.582114 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_713_331} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_712_330 to 202410_713_713_0 into Compact with storage Full 2024.10.22 01:15:43.582081 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.96 GiB, free memory in arenas 5.82 MiB, will set to 1.51 GiB (RSS), difference: 86.27 MiB 2024.10.22 01:15:43.646953 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_713_331} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:43.669832 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_713_331} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_712_330, total 44711 rows starting from the beginning of the part 2024.10.22 01:15:43.694229 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_713_331} MergeTreeSequentialSource: Reading 2 marks from part 202410_713_713_0, total 81 rows starting from the beginning of the part 2024.10.22 01:15:43.888248 [ 1706 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 01:15:43.900156 [ 1742 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} HTTPSessionAdapter: Created HTTP(S) session with localhost:11111 (::1:11111) 2024.10.22 01:15:43.947576 [ 1701 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hqq/kbgmotevhfpbqkebcruqcqzludyge, size 166 2024.10.22 01:15:44.151083 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:44.164101 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.22 GiB. 2024.10.22 01:15:44.436213 [ 1727 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ghu/ioxvqmpxylmhnveipkhitaopcnrio, size 145 2024.10.22 01:15:44.781608 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 238 entries to flush up to offset 42411 2024.10.22 01:15:44.886034 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_351_129, 202410_352_352_0] 2024.10.22 01:15:44.893203 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_351_129, 202410_352_352_0] 2024.10.22 01:15:44.960099 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_613_236, 202410_614_614_0] 2024.10.22 01:15:45.034190 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_351_129, 202410_352_352_0] 2024.10.22 01:15:45.065493 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_613_236, 202410_614_614_0] 2024.10.22 01:15:45.083583 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_84_17/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:15:45.176113 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_613_236, 202410_614_614_0] 2024.10.22 01:15:45.263728 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} MergeTask::MergeProjectionsStage: Merge sorted 3293 rows, containing 679 columns (679 merged, 0 gathered) in 106.488417737 sec., 30.92355084224177 rows/sec., 41.22 KiB/sec. 2024.10.22 01:15:45.372474 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:45.381346 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.21 GiB. 2024.10.22 01:15:45.594707 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.96 GiB, free memory in arenas 5.88 MiB, will set to 1.49 GiB (RSS), difference: 168.83 MiB 2024.10.22 01:15:45.596128 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_409_409_0 (state Temporary) 2024.10.22 01:15:45.664858 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_409_409_0 to 202410_415_415_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:45.682256 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_409_187 to 202410_410_410_0 2024.10.22 01:15:45.692509 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.12 MiB using storage policy from min volume index 0 2024.10.22 01:15:45.698350 [ 1193 ] {} DiskLocal: Reserved 2.12 MiB on local disk `default`, having unreserved 297.21 GiB. 2024.10.22 01:15:45.743569 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 42411 2024.10.22 01:15:45.883557 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_410_188} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_409_187 to 202410_410_410_0 into Wide with storage Full 2024.10.22 01:15:45.916738 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_410_188} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:45.943277 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_410_188} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_409_187, total 41141 rows starting from the beginning of the part 2024.10.22 01:15:46.186550 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 353 2024.10.22 01:15:46.445294 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_410_188} MergeTreeSequentialSource: Reading 2 marks from part 202410_410_410_0, total 225 rows starting from the beginning of the part 2024.10.22 01:15:46.629751 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:46.634527 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:46.764471 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_23_23_0 (state Temporary) 2024.10.22 01:15:46.775674 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_335_335_0 (state Temporary) 2024.10.22 01:15:46.783703 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gsf/khklciupwcnxwiopunsfqryugdete (LogSeriesLimiter: on interval from 2024-10-22 01:15:40 to 2024-10-22 01:15:46 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:15:46.876133 [ 1732 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gsf/khklciupwcnxwiopunsfqryugdete, size 4 2024.10.22 01:15:46.948445 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_84_84_0 (state Temporary) 2024.10.22 01:15:46.973774 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_335_335_0 to 202410_345_345_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:47.115548 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 353 2024.10.22 01:15:47.557603 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 44357 2024.10.22 01:15:47.637083 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} MergedBlockOutputStream: filled checksums 202410_1_84_17 (state Temporary) 2024.10.22 01:15:47.641689 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_713_331} MergeTask::MergeProjectionsStage: Merge sorted 44792 rows, containing 13 columns (13 merged, 0 gathered) in 4.068252007 sec., 11010.134063211684 rows/sec., 1.75 MiB/sec. 2024.10.22 01:15:47.665446 [ 1264 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:15:47.669133 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 5.54 MiB, will set to 1.47 GiB (RSS), difference: 36.13 MiB 2024.10.22 01:15:47.693354 [ 1264 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:15:47.837392 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:47.847498 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:47.959202 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_706_706_0 (state Temporary) 2024.10.22 01:15:47.965748 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_713_331} MergedBlockOutputStream: filled checksums 202410_1_713_331 (state Temporary) 2024.10.22 01:15:48.052097 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_706_706_0 to 202410_719_719_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:48.075435 [ 1207 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_714_714_0 to 202410_719_719_0 2024.10.22 01:15:48.081517 [ 1207 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:48.083968 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:48.088183 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44357 2024.10.22 01:15:48.137297 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_713_331} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_713_331 to 202410_1_713_331 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:48.140677 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_713_331} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_712_330, 202410_713_713_0] -> 202410_1_713_331 2024.10.22 01:15:48.181233 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.53 MiB. 2024.10.22 01:15:48.317128 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_714_714_0 to 202410_719_719_0 into Compact with storage Full 2024.10.22 01:15:48.388514 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:48.397783 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_714_714_0, total 60 rows starting from the beginning of the part 2024.10.22 01:15:48.444352 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_715_715_0, total 70 rows starting from the beginning of the part 2024.10.22 01:15:48.485549 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_716_716_0, total 63 rows starting from the beginning of the part 2024.10.22 01:15:48.685568 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_717_717_0, total 87 rows starting from the beginning of the part 2024.10.22 01:15:48.740999 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_718_718_0, total 54 rows starting from the beginning of the part 2024.10.22 01:15:48.794999 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_719_719_0, total 62 rows starting from the beginning of the part 2024.10.22 01:15:49.123771 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_410_188} MergeTask::MergeProjectionsStage: Merge sorted 41366 rows, containing 13 columns (13 merged, 0 gathered) in 3.245384857 sec., 12746.100022860865 rows/sec., 5.51 MiB/sec. 2024.10.22 01:15:49.204581 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_410_188} MergedBlockOutputStream: filled checksums 202410_1_410_188 (state Temporary) 2024.10.22 01:15:49.230210 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_614_237, 202410_615_615_0] 2024.10.22 01:15:49.233431 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_614_237, 202410_615_615_0] 2024.10.22 01:15:49.356357 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_614_237, 202410_615_615_0] 2024.10.22 01:15:49.373513 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.96 GiB, free memory in arenas 4.13 MiB, will set to 1.46 GiB (RSS), difference: 7.58 MiB 2024.10.22 01:15:49.522767 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_714_719_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:15:49.881583 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_410_188} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_410_188 to 202410_1_410_188 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:49.885283 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_410_188} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_409_187, 202410_410_410_0] -> 202410_1_410_188 2024.10.22 01:15:49.909086 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.06 MiB. 2024.10.22 01:15:50.060533 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:15:50.064943 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:15:50.198453 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} MergeTask::MergeProjectionsStage: Merge sorted 396 rows, containing 13 columns (13 merged, 0 gathered) in 2.109595286 sec., 187.71373003532585 rows/sec., 31.66 KiB/sec. 2024.10.22 01:15:50.272026 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} MergedBlockOutputStream: filled checksums 202410_714_719_1 (state Temporary) 2024.10.22 01:15:50.295150 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1701 entries to flush up to offset 635698 2024.10.22 01:15:50.382885 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_352_130, 202410_353_353_0] 2024.10.22 01:15:50.384548 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_352_130, 202410_353_353_0] 2024.10.22 01:15:50.429203 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_714_719_1 to 202410_714_719_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:50.439558 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_714_719_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_714_714_0, 202410_719_719_0] -> 202410_714_719_1 2024.10.22 01:15:50.450391 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_352_130, 202410_353_353_0] 2024.10.22 01:15:50.603048 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:15:50.685588 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 6345 2024.10.22 01:15:50.951636 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:50.963872 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:50.989791 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_416_416_0 (state Temporary) 2024.10.22 01:15:51.208015 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_416_416_0 to 202410_429_429_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:51.362322 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:51.365388 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:51.390436 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6345 2024.10.22 01:15:51.577717 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_399_399_0 (state Temporary) 2024.10.22 01:15:51.635648 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 3.91 MiB, will set to 1.46 GiB (RSS), difference: 29.77 MiB 2024.10.22 01:15:51.893420 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_399_399_0 to 202410_406_406_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:51.962234 [ 1278 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_401_80 to 202410_406_406_0 2024.10.22 01:15:51.972886 [ 1278 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:51.974757 [ 1278 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:52.046101 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 635698 2024.10.22 01:15:52.091844 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_401_80 to 202410_406_406_0 into Compact with storage Full 2024.10.22 01:15:52.096458 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 44419 2024.10.22 01:15:52.147286 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:52.164374 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} MergeTreeSequentialSource: Reading 80 marks from part 202410_1_401_80, total 641614 rows starting from the beginning of the part 2024.10.22 01:15:52.190693 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_402_402_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:15:52.318792 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_403_403_0, total 1920 rows starting from the beginning of the part 2024.10.22 01:15:52.337363 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_404_404_0, total 1192 rows starting from the beginning of the part 2024.10.22 01:15:52.359857 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_405_405_0, total 1709 rows starting from the beginning of the part 2024.10.22 01:15:52.379649 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:52.381405 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:52.382831 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_406_406_0, total 1701 rows starting from the beginning of the part 2024.10.22 01:15:52.432577 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_707_707_0 (state Temporary) 2024.10.22 01:15:52.471436 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_707_707_0 to 202410_720_720_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:52.513780 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44419 2024.10.22 01:15:53.297390 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 205 entries to flush up to offset 42616 2024.10.22 01:15:53.288646 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(206908e73cbfb79534f402d50e9a4ae1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 206908e73cbfb79534f402d50e9a4ae1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:15:53.303895 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 500667, file segment info: File segment: [0, 33554431], key: 206908e73cbfb79534f402d50e9a4ae1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:15:53.313165 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:53.462471 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_615_238, 202410_616_616_0] 2024.10.22 01:15:53.465193 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_615_238, 202410_616_616_0] 2024.10.22 01:15:53.475432 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_615_238, 202410_616_616_0] 2024.10.22 01:15:53.599828 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_341_68, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0, 202410_345_345_0, 202410_346_346_0] 2024.10.22 01:15:53.603292 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_341_68, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0, 202410_345_345_0, 202410_346_346_0] 2024.10.22 01:15:53.653346 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_341_68, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0, 202410_345_345_0, 202410_346_346_0] 2024.10.22 01:15:53.771858 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:53.774547 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:54.071800 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_410_410_0 (state Temporary) 2024.10.22 01:15:54.189135 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(8c14f3c4d613ee31fe5e2e7940b77d21) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8c14f3c4d613ee31fe5e2e7940b77d21, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:15:54.190916 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 581, file segment info: File segment: [0, 33554431], key: 8c14f3c4d613ee31fe5e2e7940b77d21, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:15:54.192763 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:15:54.230003 [ 1748 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rqe/uhktjlphpxtywbffcraunnjfgetmj, size 500667 2024.10.22 01:15:54.350516 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_410_410_0 to 202410_416_416_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:54.387580 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 42616 2024.10.22 01:15:54.377407 [ 1165 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_410_188 to 202410_411_411_0 2024.10.22 01:15:54.392978 [ 1165 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.13 MiB using storage policy from min volume index 0 2024.10.22 01:15:54.396116 [ 1165 ] {} DiskLocal: Reserved 2.13 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:54.412008 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_411_189} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_410_188 to 202410_411_411_0 into Wide with storage Full 2024.10.22 01:15:54.466536 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_411_189} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:15:54.473490 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_411_189} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_410_188, total 41366 rows starting from the beginning of the part 2024.10.22 01:15:54.492607 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:15:54.506661 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:15:54.531364 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:15:54.574331 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:15:54.619121 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 354 2024.10.22 01:15:54.686957 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_411_189} MergeTreeSequentialSource: Reading 2 marks from part 202410_411_411_0, total 206 rows starting from the beginning of the part 2024.10.22 01:15:54.690573 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:15:54.699556 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:15:54.726778 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:15:54.777561 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:15:54.886966 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:15:54.920838 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uix/jhrhxjytpeethcbqdkrgpdtswkjns (LogSeriesLimiter: on interval from 2024-10-22 01:15:46 to 2024-10-22 01:15:54 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:15:55.184575 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:55.196666 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:55.460499 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_336_336_0 (state Temporary) 2024.10.22 01:15:55.589678 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_406_81/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:15:55.631966 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.96 GiB, free memory in arenas 12.77 MiB, will set to 1.46 GiB (RSS), difference: -38.81 MiB 2024.10.22 01:15:55.834636 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_336_336_0 to 202410_346_346_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:56.070822 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 354 2024.10.22 01:15:56.273066 [ 1223 ] {} DatabaseCatalog: Execution took 375 ms. 2024.10.22 01:15:56.389570 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_411_189/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:15:56.519846 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 44479 2024.10.22 01:15:56.584901 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_84_84_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:15:56.757790 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_84_84_0 to 202410_87_87_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:56.874538 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:56.876819 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:56.920118 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_708_708_0 (state Temporary) 2024.10.22 01:15:57.177162 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_708_708_0 to 202410_721_721_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:57.434788 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44479 2024.10.22 01:15:57.577236 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.96 GiB, free memory in arenas 11.84 MiB, will set to 1.48 GiB (RSS), difference: 19.47 MiB 2024.10.22 01:15:57.786467 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_616_239, 202410_617_617_0] 2024.10.22 01:15:57.790708 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_616_239, 202410_617_617_0] 2024.10.22 01:15:57.801725 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_616_239, 202410_617_617_0] 2024.10.22 01:15:57.971079 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_411_189} MergeTask::MergeProjectionsStage: Merge sorted 41572 rows, containing 13 columns (13 merged, 0 gathered) in 3.562642838 sec., 11668.865471605268 rows/sec., 5.04 MiB/sec. 2024.10.22 01:15:58.034874 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_411_189} MergedBlockOutputStream: filled checksums 202410_1_411_189 (state Temporary) 2024.10.22 01:15:58.225638 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_353_131, 202410_354_354_0] 2024.10.22 01:15:58.228951 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_353_131, 202410_354_354_0] 2024.10.22 01:15:58.349290 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_353_131, 202410_354_354_0] 2024.10.22 01:15:58.828213 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_411_189} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_411_189 to 202410_1_411_189 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:58.836129 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_411_189} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_410_188, 202410_411_411_0] -> 202410_1_411_189 2024.10.22 01:15:58.861835 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.06 MiB. 2024.10.22 01:15:58.894307 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 6355 2024.10.22 01:15:59.056985 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 637240 2024.10.22 01:15:59.104392 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:59.110410 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:59.160637 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_417_417_0 (state Temporary) 2024.10.22 01:15:59.200325 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_417_417_0 to 202410_430_430_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:59.460646 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 12.68 MiB, will set to 1.46 GiB (RSS), difference: 24.25 MiB 2024.10.22 01:15:59.467590 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6355 2024.10.22 01:15:59.819349 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:15:59.822785 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:15:59.824736 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_84_17 to 202410_1_84_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:15:59.848155 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_84_17} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_79_16, 202410_84_84_0] -> 202410_1_84_17 2024.10.22 01:15:59.918474 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_400_400_0 (state Temporary) 2024.10.22 01:15:59.998465 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_400_400_0 to 202410_407_407_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:00.027245 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 637240 2024.10.22 01:16:00.062209 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:16:00.066421 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:16:00.075171 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3345 2024.10.22 01:16:00.606560 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.96 GiB, free memory in arenas 12.25 MiB, will set to 1.47 GiB (RSS), difference: 8.31 MiB 2024.10.22 01:16:00.670835 [ 1250 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 213 ms. 2024.10.22 01:16:00.722229 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 42.69 MiB. 2024.10.22 01:16:01.458700 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 44520 2024.10.22 01:16:01.559860 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(ba161c6e6eaad78fc7ea4ffb8160e7bf) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ba161c6e6eaad78fc7ea4ffb8160e7bf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:01.565598 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 598, file segment info: File segment: [0, 33554431], key: ba161c6e6eaad78fc7ea4ffb8160e7bf, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:01.568611 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:01.680201 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:01.684151 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:01.748924 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_709_709_0 (state Temporary) 2024.10.22 01:16:01.897324 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_709_709_0 to 202410_722_722_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:02.021474 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 156 entries to flush up to offset 42772 2024.10.22 01:16:02.088473 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44520 2024.10.22 01:16:02.165071 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:02.167075 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:02.199663 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_411_411_0 (state Temporary) 2024.10.22 01:16:02.270261 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_411_411_0 to 202410_417_417_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:02.295354 [ 1170 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_411_189 to 202410_412_412_0 2024.10.22 01:16:02.303247 [ 1170 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.14 MiB using storage policy from min volume index 0 2024.10.22 01:16:02.312294 [ 1170 ] {} DiskLocal: Reserved 2.14 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:02.357591 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.96 GiB, free memory in arenas 12.49 MiB, will set to 1.46 GiB (RSS), difference: -1.03 MiB 2024.10.22 01:16:02.360521 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_412_190} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_411_189 to 202410_412_412_0 into Wide with storage Full 2024.10.22 01:16:02.384239 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_412_190} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:02.393022 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_412_190} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_411_189, total 41572 rows starting from the beginning of the part 2024.10.22 01:16:02.423049 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 42772 2024.10.22 01:16:02.669098 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(a9f8bb3ac2ed3ac8cbd79988c2091b79) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a9f8bb3ac2ed3ac8cbd79988c2091b79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:02.671540 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 156, file segment info: File segment: [0, 33554431], key: a9f8bb3ac2ed3ac8cbd79988c2091b79, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:02.673155 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:02.800773 [ 1183 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:16:02.836599 [ 1183 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:16:02.975618 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_412_190} MergeTreeSequentialSource: Reading 2 marks from part 202410_412_412_0, total 232 rows starting from the beginning of the part 2024.10.22 01:16:03.001956 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:16:03.572744 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 355 2024.10.22 01:16:03.577382 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(823981d07f7ed9fcffa776404e22c05f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 823981d07f7ed9fcffa776404e22c05f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:03.680973 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 359, file segment info: File segment: [0, 33554431], key: 823981d07f7ed9fcffa776404e22c05f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:03.683347 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:03.924752 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:03.927286 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:03.946881 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_617_240, 202410_618_618_0] 2024.10.22 01:16:03.951238 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_617_240, 202410_618_618_0] 2024.10.22 01:16:03.967832 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_617_240, 202410_618_618_0] 2024.10.22 01:16:04.211860 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_337_337_0 (state Temporary) 2024.10.22 01:16:04.331617 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_337_337_0 to 202410_347_347_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:04.367687 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_366_73, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0, 202410_370_370_0, 202410_371_371_0] 2024.10.22 01:16:04.371043 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_366_73, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0, 202410_370_370_0, 202410_371_371_0] 2024.10.22 01:16:04.397193 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_366_73, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0, 202410_370_370_0, 202410_371_371_0] 2024.10.22 01:16:04.559931 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 355 2024.10.22 01:16:04.702623 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.96 GiB, free memory in arenas 15.82 MiB, will set to 1.48 GiB (RSS), difference: -42.82 MiB 2024.10.22 01:16:04.790197 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(8da3f815aa237dae756fe3fea6143346) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8da3f815aa237dae756fe3fea6143346, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:04.793080 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 464, file segment info: File segment: [0, 33554431], key: 8da3f815aa237dae756fe3fea6143346, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:04.802112 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:04.851440 [ 1684 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/flk/flxpahovvevadryxjfanrzkwqgonm, size 359 2024.10.22 01:16:05.657215 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:16:05.671838 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:16:05.683878 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:16:05.701431 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:16:05.747300 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:16:05.775045 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:16:05.791845 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:16:05.912772 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/cxp/qygcigsljdmdjogwvsdutxkgodrbd (LogSeriesLimiter: on interval from 2024-10-22 01:15:54 to 2024-10-22 01:16:05 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:16:06.096357 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 44577 2024.10.22 01:16:06.124089 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_412_190} MergeTask::MergeProjectionsStage: Merge sorted 41804 rows, containing 13 columns (13 merged, 0 gathered) in 3.784545118 sec., 11045.977441561578 rows/sec., 4.78 MiB/sec. 2024.10.22 01:16:06.366910 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:06.371592 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:06.575315 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_710_710_0 (state Temporary) 2024.10.22 01:16:06.617736 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_710_710_0 to 202410_723_723_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:06.632418 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_406_81/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:16:06.663318 [ 1192 ] {} DatabaseCatalog: Execution took 388 ms. 2024.10.22 01:16:06.740754 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_412_190} MergedBlockOutputStream: filled checksums 202410_1_412_190 (state Temporary) 2024.10.22 01:16:06.746467 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.96 GiB, free memory in arenas 16.98 MiB, will set to 1.47 GiB (RSS), difference: 12.93 MiB 2024.10.22 01:16:06.879313 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44577 2024.10.22 01:16:06.968790 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 6368 2024.10.22 01:16:07.008621 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_618_241, 202410_619_619_0] 2024.10.22 01:16:07.012479 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_618_241, 202410_619_619_0] 2024.10.22 01:16:07.032854 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 638776 2024.10.22 01:16:07.100734 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:07.107185 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:07.029560 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_618_241, 202410_619_619_0] 2024.10.22 01:16:07.189677 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_354_132, 202410_355_355_0] 2024.10.22 01:16:07.193610 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_354_132, 202410_355_355_0] 2024.10.22 01:16:07.295158 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_418_418_0 (state Temporary) 2024.10.22 01:16:07.323366 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_354_132, 202410_355_355_0] 2024.10.22 01:16:07.389516 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_418_418_0 to 202410_431_431_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:07.464762 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6368 2024.10.22 01:16:07.436898 [ 1208 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_426_85 to 202410_431_431_0 2024.10.22 01:16:07.616900 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 3386 2024.10.22 01:16:07.650171 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_412_190} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_412_190 to 202410_1_412_190 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:07.656600 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_412_190} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_411_189, 202410_412_412_0] -> 202410_1_412_190 2024.10.22 01:16:07.675218 [ 1208 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:07.679334 [ 1208 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:07.784436 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.06 MiB. 2024.10.22 01:16:07.791410 [ 1208 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 376 ms. 2024.10.22 01:16:07.838636 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_426_85 to 202410_431_431_0 into Compact with storage Full 2024.10.22 01:16:07.879595 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:07.885495 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:07.918760 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:07.965250 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_426_85, total 6383 rows starting from the beginning of the part 2024.10.22 01:16:07.988188 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_427_427_0, total 29 rows starting from the beginning of the part 2024.10.22 01:16:08.024667 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_428_428_0, total 20 rows starting from the beginning of the part 2024.10.22 01:16:08.051256 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_429_429_0, total 19 rows starting from the beginning of the part 2024.10.22 01:16:08.064155 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_401_401_0 (state Temporary) 2024.10.22 01:16:08.078128 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_430_430_0, total 10 rows starting from the beginning of the part 2024.10.22 01:16:08.092886 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_431_431_0, total 13 rows starting from the beginning of the part 2024.10.22 01:16:08.127407 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_401_401_0 to 202410_408_408_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:08.183339 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 638776 2024.10.22 01:16:09.406780 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 16.90 MiB, will set to 1.47 GiB (RSS), difference: 30.37 MiB 2024.10.22 01:16:09.434793 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} MergeTask::MergeProjectionsStage: Merge sorted 6474 rows, containing 12 columns (12 merged, 0 gathered) in 1.641186592 sec., 3944.706855124003 rows/sec., 624.79 KiB/sec. 2024.10.22 01:16:09.568534 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} MergedBlockOutputStream: filled checksums 202410_1_431_86 (state Temporary) 2024.10.22 01:16:09.872224 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_431_86 to 202410_1_431_86 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:09.875412 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_431_86} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_426_85, 202410_431_431_0] -> 202410_1_431_86 2024.10.22 01:16:10.009931 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.47 MiB. 2024.10.22 01:16:10.061523 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:16:10.062702 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 161 entries to flush up to offset 42933 2024.10.22 01:16:10.064759 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:16:10.208753 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:10.218966 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:10.280625 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_412_412_0 (state Temporary) 2024.10.22 01:16:10.345333 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_412_412_0 to 202410_418_418_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:10.483645 [ 1245 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_412_190 to 202410_413_413_0 2024.10.22 01:16:10.491277 [ 1245 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.15 MiB using storage policy from min volume index 0 2024.10.22 01:16:10.492888 [ 1245 ] {} DiskLocal: Reserved 2.15 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:10.495341 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 42933 2024.10.22 01:16:10.512135 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_413_191} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_412_190 to 202410_413_413_0 into Wide with storage Full 2024.10.22 01:16:10.587090 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_413_191} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:10.593181 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_413_191} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_412_190, total 41804 rows starting from the beginning of the part 2024.10.22 01:16:10.776537 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_413_191} MergeTreeSequentialSource: Reading 2 marks from part 202410_413_413_0, total 233 rows starting from the beginning of the part 2024.10.22 01:16:10.893770 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 44643 2024.10.22 01:16:11.114870 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:11.118926 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:11.298659 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_711_711_0 (state Temporary) 2024.10.22 01:16:11.355226 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_711_711_0 to 202410_724_724_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:11.378330 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_713_331 to 202410_714_719_1 2024.10.22 01:16:11.383865 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.62 MiB using storage policy from min volume index 0 2024.10.22 01:16:11.386295 [ 1271 ] {} DiskLocal: Reserved 1.62 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:11.396232 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44643 2024.10.22 01:16:11.522443 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_719_332} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_713_331 to 202410_714_719_1 into Compact with storage Full 2024.10.22 01:16:11.567464 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.96 GiB, free memory in arenas 17.09 MiB, will set to 1.47 GiB (RSS), difference: -57.55 MiB 2024.10.22 01:16:11.600975 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_719_332} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:11.626453 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_719_332} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_713_331, total 44792 rows starting from the beginning of the part 2024.10.22 01:16:11.699051 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_619_242, 202410_620_620_0] 2024.10.22 01:16:11.704174 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_619_242, 202410_620_620_0] 2024.10.22 01:16:11.709600 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_719_332} MergeTreeSequentialSource: Reading 2 marks from part 202410_714_719_1, total 396 rows starting from the beginning of the part 2024.10.22 01:16:11.722446 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_619_242, 202410_620_620_0] 2024.10.22 01:16:12.062294 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 356 2024.10.22 01:16:12.085058 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(e2fd614c651e8c05501f9521f127e3eb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e2fd614c651e8c05501f9521f127e3eb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:12.086472 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 935638, file segment info: File segment: [0, 33554431], key: e2fd614c651e8c05501f9521f127e3eb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:12.088305 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:12.370341 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:12.379001 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:12.470362 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_338_338_0 (state Temporary) 2024.10.22 01:16:12.742958 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_338_338_0 to 202410_348_348_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:12.833089 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 356 2024.10.22 01:16:13.101521 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(08d814b841b118884f2e4bf659eb7e51) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 08d814b841b118884f2e4bf659eb7e51, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:13.229534 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 990, file segment info: File segment: [0, 33554431], key: 08d814b841b118884f2e4bf659eb7e51, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 01:16:13.231615 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:13.281523 [ 1702 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uix/jhrhxjytpeethcbqdkrgpdtswkjns, size 935638 2024.10.22 01:16:13.320136 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_413_191} MergeTask::MergeProjectionsStage: Merge sorted 42037 rows, containing 13 columns (13 merged, 0 gathered) in 2.81608148 sec., 14927.480010272999 rows/sec., 6.46 MiB/sec. 2024.10.22 01:16:13.480908 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.96 GiB, free memory in arenas 16.88 MiB, will set to 1.48 GiB (RSS), difference: 25.94 MiB 2024.10.22 01:16:13.493749 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_413_191} MergedBlockOutputStream: filled checksums 202410_1_413_191 (state Temporary) 2024.10.22 01:16:13.518623 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tqp/adppnoclfyickzhpohontzvygeroz. 2024.10.22 01:16:13.601281 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:16:13.754977 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} MergeTask::MergeProjectionsStage: Merge sorted 649288 rows, containing 4 columns (4 merged, 0 gathered) in 21.770127385 sec., 29824.722130352387 rows/sec., 437.23 KiB/sec. 2024.10.22 01:16:13.755421 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:16:13.770474 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:16:13.797661 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} MergedBlockOutputStream: filled checksums 202410_1_406_81 (state Temporary) 2024.10.22 01:16:13.869566 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_406_81 to 202410_1_406_81 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:13.874723 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_406_81} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_401_80, 202410_406_406_0] -> 202410_1_406_81 2024.10.22 01:16:13.907511 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:16:13.913099 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:16:13.952149 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:16:13.982873 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:16:14.013288 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:16:14.048004 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:16:14.075489 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:16:14.258932 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qwb/dpnbhphpgwfmbszskwcozyedxsvql (LogSeriesLimiter: on interval from 2024-10-22 01:16:05 to 2024-10-22 01:16:14 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:16:14.503747 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_413_191} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_413_191 to 202410_1_413_191 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:14.520938 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_413_191} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_412_190, 202410_413_413_0] -> 202410_1_413_191 2024.10.22 01:16:14.541011 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.06 MiB. 2024.10.22 01:16:14.967735 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 6393 2024.10.22 01:16:15.007722 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_719_332} MergeTask::MergeProjectionsStage: Merge sorted 45188 rows, containing 13 columns (13 merged, 0 gathered) in 3.601235914 sec., 12547.914404699008 rows/sec., 2.00 MiB/sec. 2024.10.22 01:16:15.079446 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:15.088384 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:15.141254 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_719_332} MergedBlockOutputStream: filled checksums 202410_1_719_332 (state Temporary) 2024.10.22 01:16:15.147109 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_419_419_0 (state Temporary) 2024.10.22 01:16:15.189430 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 640312 2024.10.22 01:16:15.192724 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_419_419_0 to 202410_432_432_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:15.296207 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_719_332} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_719_332 to 202410_1_719_332 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:15.301247 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_719_332} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_713_331, 202410_714_719_1] -> 202410_1_719_332 2024.10.22 01:16:15.361038 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6393 2024.10.22 01:16:15.365508 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_355_133, 202410_356_356_0] 2024.10.22 01:16:15.368977 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_355_133, 202410_356_356_0] 2024.10.22 01:16:15.398958 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_355_133, 202410_356_356_0] 2024.10.22 01:16:15.402946 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 44708 2024.10.22 01:16:15.437458 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.62 MiB. 2024.10.22 01:16:15.480525 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 13.59 MiB, will set to 1.47 GiB (RSS), difference: 39.36 MiB 2024.10.22 01:16:15.794912 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_620_243, 202410_621_621_0] 2024.10.22 01:16:15.796316 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_620_243, 202410_621_621_0] 2024.10.22 01:16:15.807745 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_620_243, 202410_621_621_0] 2024.10.22 01:16:15.848004 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:15.852677 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:15.924323 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:15.929110 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:15.957973 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_402_402_0 (state Temporary) 2024.10.22 01:16:16.003559 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_402_402_0 to 202410_409_409_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:16.064050 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 640312 2024.10.22 01:16:16.203272 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_712_712_0 (state Temporary) 2024.10.22 01:16:16.357727 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_712_712_0 to 202410_725_725_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:16.379920 [ 1256 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_719_332 to 202410_720_720_0 2024.10.22 01:16:16.383827 [ 1256 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.61 MiB using storage policy from min volume index 0 2024.10.22 01:16:16.386843 [ 1256 ] {} DiskLocal: Reserved 1.61 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:16.398537 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_720_333} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_719_332 to 202410_720_720_0 into Compact with storage Full 2024.10.22 01:16:16.493091 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_720_333} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:16.503286 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44708 2024.10.22 01:16:16.533017 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_720_333} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_719_332, total 45188 rows starting from the beginning of the part 2024.10.22 01:16:16.588061 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_720_333} MergeTreeSequentialSource: Reading 2 marks from part 202410_720_720_0, total 62 rows starting from the beginning of the part 2024.10.22 01:16:17.272998 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_720_333/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:16:17.710626 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 10.50 MiB, will set to 1.46 GiB (RSS), difference: -18.02 MiB 2024.10.22 01:16:17.845249 [ 1257 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:16:17.874355 [ 1257 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:16:17.876353 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.053008346 sec. 2024.10.22 01:16:18.020417 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 207 entries to flush up to offset 43140 2024.10.22 01:16:18.424045 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:18.436503 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:18.601156 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_413_413_0 (state Temporary) 2024.10.22 01:16:18.678295 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_413_413_0 to 202410_419_419_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:18.739993 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 43140 2024.10.22 01:16:18.741742 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_413_191 to 202410_414_414_0 2024.10.22 01:16:18.752224 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.16 MiB using storage policy from min volume index 0 2024.10.22 01:16:18.754780 [ 1237 ] {} DiskLocal: Reserved 2.16 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:18.788301 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_414_192} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_413_191 to 202410_414_414_0 into Wide with storage Full 2024.10.22 01:16:18.868883 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_414_192} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:18.879656 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_414_192} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_413_191, total 42037 rows starting from the beginning of the part 2024.10.22 01:16:19.163293 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_414_192} MergeTreeSequentialSource: Reading 2 marks from part 202410_414_414_0, total 240 rows starting from the beginning of the part 2024.10.22 01:16:19.328311 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_720_333} MergeTask::MergeProjectionsStage: Merge sorted 45250 rows, containing 13 columns (13 merged, 0 gathered) in 2.935204978 sec., 15416.29982885645 rows/sec., 2.46 MiB/sec. 2024.10.22 01:16:19.391348 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_720_333} MergedBlockOutputStream: filled checksums 202410_1_720_333 (state Temporary) 2024.10.22 01:16:19.621423 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 6.76 MiB, will set to 1.46 GiB (RSS), difference: -16.97 MiB 2024.10.22 01:16:19.895983 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_720_333} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_720_333 to 202410_1_720_333 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:19.902367 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_720_333} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_719_332, 202410_720_720_0] -> 202410_1_720_333 2024.10.22 01:16:19.965085 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:16:20.064658 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:16:20.071381 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:16:20.334900 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 357 2024.10.22 01:16:20.519945 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 44759 2024.10.22 01:16:20.823015 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:20.837302 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:20.854979 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:20.857530 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:20.925521 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_339_339_0 (state Temporary) 2024.10.22 01:16:21.015879 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_339_339_0 to 202410_349_349_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:21.065147 [ 1170 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_344_71 to 202410_349_349_0 2024.10.22 01:16:21.091407 [ 1170 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:21.096136 [ 1170 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:21.110299 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_713_713_0 (state Temporary) 2024.10.22 01:16:21.137034 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_344_71 to 202410_349_349_0 into Compact with storage Full 2024.10.22 01:16:21.198484 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 357 2024.10.22 01:16:21.261462 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_713_713_0 to 202410_726_726_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:21.316671 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:21.426596 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_344_71, total 388 rows starting from the beginning of the part 2024.10.22 01:16:21.434921 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_720_333 to 202410_721_721_0 2024.10.22 01:16:21.449583 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.62 MiB using storage policy from min volume index 0 2024.10.22 01:16:21.458314 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44759 2024.10.22 01:16:21.481161 [ 1173 ] {} DiskLocal: Reserved 1.62 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:21.500699 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 1.96 GiB, free memory in arenas 5.73 MiB, will set to 1.47 GiB (RSS), difference: 87.42 MiB 2024.10.22 01:16:21.536340 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_721_334} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_720_333 to 202410_721_721_0 into Compact with storage Full 2024.10.22 01:16:21.573745 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_721_334} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:21.590436 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_721_334} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_720_333, total 45250 rows starting from the beginning of the part 2024.10.22 01:16:21.596921 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_345_345_0, total 1 rows starting from the beginning of the part 2024.10.22 01:16:21.667587 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_346_346_0, total 1 rows starting from the beginning of the part 2024.10.22 01:16:21.672708 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_721_334} MergeTreeSequentialSource: Reading 2 marks from part 202410_721_721_0, total 60 rows starting from the beginning of the part 2024.10.22 01:16:21.696901 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_347_347_0, total 1 rows starting from the beginning of the part 2024.10.22 01:16:21.830789 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_348_348_0, total 1 rows starting from the beginning of the part 2024.10.22 01:16:21.851947 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_414_192} MergeTask::MergeProjectionsStage: Merge sorted 42277 rows, containing 13 columns (13 merged, 0 gathered) in 3.072199025 sec., 13761.15273000583 rows/sec., 5.95 MiB/sec. 2024.10.22 01:16:21.891140 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_349_349_0, total 1 rows starting from the beginning of the part 2024.10.22 01:16:22.025563 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_414_192} MergedBlockOutputStream: filled checksums 202410_1_414_192 (state Temporary) 2024.10.22 01:16:22.631957 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_414_192} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_414_192 to 202410_1_414_192 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:22.636813 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_414_192} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_413_191, 202410_414_414_0] -> 202410_1_414_192 2024.10.22 01:16:22.671000 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tqp/adppnoclfyickzhpohontzvygeroz. 2024.10.22 01:16:22.671106 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.06 MiB. 2024.10.22 01:16:22.864093 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 6408 2024.10.22 01:16:22.864250 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_621_244, 202410_622_622_0] 2024.10.22 01:16:22.867022 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_621_244, 202410_622_622_0] 2024.10.22 01:16:22.944043 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:22.948741 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:22.978632 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_621_244, 202410_622_622_0] 2024.10.22 01:16:23.068846 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 641848 2024.10.22 01:16:23.184046 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:23.185694 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:23.319572 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_420_420_0 (state Temporary) 2024.10.22 01:16:23.598328 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_420_420_0 to 202410_433_433_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:23.614769 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:23.620382 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:23.668889 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6408 2024.10.22 01:16:23.832089 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.96 GiB, free memory in arenas 4.93 MiB, will set to 1.45 GiB (RSS), difference: 109.68 MiB 2024.10.22 01:16:23.992892 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} MergeTask::MergeProjectionsStage: Merge sorted 393 rows, containing 32 columns (32 merged, 0 gathered) in 2.8665216 sec., 137.09996115152245 rows/sec., 12.92 KiB/sec. 2024.10.22 01:16:24.037662 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} MergedBlockOutputStream: filled checksums 202410_1_349_72 (state Temporary) 2024.10.22 01:16:24.177050 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_356_134, 202410_357_357_0] 2024.10.22 01:16:24.178716 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_356_134, 202410_357_357_0] 2024.10.22 01:16:24.214582 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_356_134, 202410_357_357_0] 2024.10.22 01:16:24.377633 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_403_403_0 (state Temporary) 2024.10.22 01:16:24.507483 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_403_403_0 to 202410_410_410_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:24.431079 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_721_334} MergeTask::MergeProjectionsStage: Merge sorted 45310 rows, containing 13 columns (13 merged, 0 gathered) in 2.813556896 sec., 16104.17051256958 rows/sec., 2.57 MiB/sec. 2024.10.22 01:16:24.599284 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_349_72 to 202410_1_349_72 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:24.611631 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_349_72} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_344_71, 202410_349_349_0] -> 202410_1_349_72 2024.10.22 01:16:24.640273 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:16:24.661783 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 641848 2024.10.22 01:16:24.915012 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_721_334} MergedBlockOutputStream: filled checksums 202410_1_721_334 (state Temporary) 2024.10.22 01:16:24.991810 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_721_334} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_721_334 to 202410_1_721_334 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:24.994855 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_721_334} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_720_333, 202410_721_721_0] -> 202410_1_721_334 2024.10.22 01:16:25.019073 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.52 MiB. 2024.10.22 01:16:25.148613 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4d34cf5f0cbeeeb8ba7f685f10d8b058) : [0, 288]: Setting download as failed: File segment: [0, 288], key: 4d34cf5f0cbeeeb8ba7f685f10d8b058, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:25.152797 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4d34cf5f0cbeeeb8ba7f685f10d8b058): No space left in cache to reserve 289 bytes, will continue without cache download 2024.10.22 01:16:25.376908 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 5.34 MiB, will set to 1.44 GiB (RSS), difference: 5.72 MiB 2024.10.22 01:16:25.465442 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 44829 2024.10.22 01:16:25.875038 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_85_85_0 (state Temporary) 2024.10.22 01:16:25.904895 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:25.908488 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:26.077705 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_714_714_0 (state Temporary) 2024.10.22 01:16:26.234818 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_714_714_0 to 202410_727_727_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:26.265464 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_721_334 to 202410_722_722_0 2024.10.22 01:16:26.268476 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.62 MiB using storage policy from min volume index 0 2024.10.22 01:16:26.273584 [ 1202 ] {} DiskLocal: Reserved 1.62 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:26.282992 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 173 entries to flush up to offset 43313 2024.10.22 01:16:26.292533 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_722_335} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_721_334 to 202410_722_722_0 into Compact with storage Full 2024.10.22 01:16:26.295137 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8eeaac978f898888b17d7f2fa20a9d76) : [0, 4607315]: Setting download as failed: File segment: [0, 4607315], key: 8eeaac978f898888b17d7f2fa20a9d76, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:26.296572 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8eeaac978f898888b17d7f2fa20a9d76): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:16:26.339514 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44829 2024.10.22 01:16:26.380888 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_722_335} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:26.397566 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_722_335} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_721_334, total 45310 rows starting from the beginning of the part 2024.10.22 01:16:26.422757 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:26.428420 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:26.464830 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_722_335} MergeTreeSequentialSource: Reading 2 marks from part 202410_722_722_0, total 41 rows starting from the beginning of the part 2024.10.22 01:16:26.588416 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_414_414_0 (state Temporary) 2024.10.22 01:16:26.835213 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_414_414_0 to 202410_420_420_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:26.846344 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_296_61, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:16:26.848716 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_296_61, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:16:26.874587 [ 1215 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_414_192 to 202410_415_415_0 2024.10.22 01:16:26.880741 [ 1215 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.18 MiB using storage policy from min volume index 0 2024.10.22 01:16:26.883524 [ 1215 ] {} DiskLocal: Reserved 2.18 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:26.887276 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_296_61, 202410_297_297_0, 202410_298_298_0, 202410_299_299_0, 202410_300_300_0, 202410_301_301_0] 2024.10.22 01:16:26.894744 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8eeaac978f898888b17d7f2fa20a9d76, offset: 0, size: 0 2024.10.22 01:16:26.897664 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8eeaac978f898888b17d7f2fa20a9d76 for removal 2024.10.22 01:16:26.901705 [ 1476 ] {} CacheMetadata: Key 8eeaac978f898888b17d7f2fa20a9d76 is removed from metadata 2024.10.22 01:16:26.905097 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_415_193} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_414_192 to 202410_415_415_0 into Wide with storage Full 2024.10.22 01:16:26.909239 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4d34cf5f0cbeeeb8ba7f685f10d8b058, offset: 0, size: 0 2024.10.22 01:16:26.912972 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4d34cf5f0cbeeeb8ba7f685f10d8b058 for removal 2024.10.22 01:16:26.915462 [ 1476 ] {} CacheMetadata: Key 4d34cf5f0cbeeeb8ba7f685f10d8b058 is removed from metadata 2024.10.22 01:16:26.979997 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(77cc6d0c9f4805fd532ff7f0a62e6e79) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 77cc6d0c9f4805fd532ff7f0a62e6e79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:16:26.983314 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 993, file segment info: File segment: [0, 33554431], key: 77cc6d0c9f4805fd532ff7f0a62e6e79, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:16:26.987340 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:26.991552 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 43313 2024.10.22 01:16:27.062778 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_415_193} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:27.083875 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_415_193} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_414_192, total 42277 rows starting from the beginning of the part 2024.10.22 01:16:27.166029 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_85_85_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:16:27.260963 [ 1199 ] {} DatabaseCatalog: Execution took 395 ms. 2024.10.22 01:16:27.472095 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_415_193} MergeTreeSequentialSource: Reading 2 marks from part 202410_415_415_0, total 238 rows starting from the beginning of the part 2024.10.22 01:16:27.690137 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_622_245, 202410_623_623_0] 2024.10.22 01:16:27.694781 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_622_245, 202410_623_623_0] 2024.10.22 01:16:27.713237 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_622_245, 202410_623_623_0] 2024.10.22 01:16:27.826657 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7ea4512320273c9752a9b4858f471e7d) : [0, 163]: Setting download as failed: File segment: [0, 163], key: 7ea4512320273c9752a9b4858f471e7d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:27.828350 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7ea4512320273c9752a9b4858f471e7d): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 01:16:27.995746 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_722_335/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:16:28.268236 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.96 GiB, free memory in arenas 5.64 MiB, will set to 1.45 GiB (RSS), difference: -44.69 MiB 2024.10.22 01:16:28.378327 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_415_193/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:16:28.615639 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(542c4284f224bc7772d43f7b3b6caad6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 542c4284f224bc7772d43f7b3b6caad6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:16:28.619731 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 161, file segment info: File segment: [0, 33554431], key: 542c4284f224bc7772d43f7b3b6caad6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:16:28.626342 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:29.448569 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5864f81f018d8f2d3367cf2a50e9d374) : [0, 102024]: Setting download as failed: File segment: [0, 102024], key: 5864f81f018d8f2d3367cf2a50e9d374, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:29.451048 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5864f81f018d8f2d3367cf2a50e9d374): No space left in cache to reserve 102025 bytes, will continue without cache download 2024.10.22 01:16:29.573424 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5864f81f018d8f2d3367cf2a50e9d374, offset: 0, size: 0 2024.10.22 01:16:29.575549 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5864f81f018d8f2d3367cf2a50e9d374 for removal 2024.10.22 01:16:29.580112 [ 1476 ] {} CacheMetadata: Key 5864f81f018d8f2d3367cf2a50e9d374 is removed from metadata 2024.10.22 01:16:29.581055 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ea4512320273c9752a9b4858f471e7d, offset: 0, size: 0 2024.10.22 01:16:29.582105 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ea4512320273c9752a9b4858f471e7d for removal 2024.10.22 01:16:29.583947 [ 1476 ] {} CacheMetadata: Key 7ea4512320273c9752a9b4858f471e7d is removed from metadata 2024.10.22 01:16:29.659257 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_722_335} MergeTask::MergeProjectionsStage: Merge sorted 45351 rows, containing 13 columns (13 merged, 0 gathered) in 3.371037149 sec., 13453.129703258573 rows/sec., 2.14 MiB/sec. 2024.10.22 01:16:29.791325 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_722_335} MergedBlockOutputStream: filled checksums 202410_1_722_335 (state Temporary) 2024.10.22 01:16:29.854727 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_722_335} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_722_335 to 202410_1_722_335 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:29.871931 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_722_335} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_721_334, 202410_722_722_0] -> 202410_1_722_335 2024.10.22 01:16:30.063722 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:16:30.065964 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:16:30.095844 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(4833a1f6bb4b24df76ae3e892eea043c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4833a1f6bb4b24df76ae3e892eea043c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:16:30.141059 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.51 MiB. 2024.10.22 01:16:30.101737 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 359, file segment info: File segment: [0, 33554431], key: 4833a1f6bb4b24df76ae3e892eea043c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:16:30.207457 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:30.355544 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 44901 2024.10.22 01:16:30.504221 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:30.512123 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:30.770497 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_415_193} MergeTask::MergeProjectionsStage: Merge sorted 42515 rows, containing 13 columns (13 merged, 0 gathered) in 3.871385883 sec., 10981.855409116291 rows/sec., 4.75 MiB/sec. 2024.10.22 01:16:30.777930 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_715_715_0 (state Temporary) 2024.10.22 01:16:30.801055 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d63df878c7607adeabb9ff4be4890afd) : [0, 193]: Setting download as failed: File segment: [0, 193], key: d63df878c7607adeabb9ff4be4890afd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:30.805179 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d63df878c7607adeabb9ff4be4890afd): No space left in cache to reserve 194 bytes, will continue without cache download 2024.10.22 01:16:30.806802 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_415_193} MergedBlockOutputStream: filled checksums 202410_1_415_193 (state Temporary) 2024.10.22 01:16:30.859613 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_715_715_0 to 202410_728_728_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:30.877217 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_722_335 to 202410_723_723_0 2024.10.22 01:16:30.896110 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.62 MiB using storage policy from min volume index 0 2024.10.22 01:16:30.898968 [ 1153 ] {} DiskLocal: Reserved 1.62 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:30.899921 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44901 2024.10.22 01:16:30.935714 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_723_336} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_722_335 to 202410_723_723_0 into Compact with storage Full 2024.10.22 01:16:31.078574 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_723_336} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:31.085093 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_723_336} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_722_335, total 45351 rows starting from the beginning of the part 2024.10.22 01:16:31.117190 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_723_336} MergeTreeSequentialSource: Reading 2 marks from part 202410_723_723_0, total 57 rows starting from the beginning of the part 2024.10.22 01:16:31.171405 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 6426 2024.10.22 01:16:31.264853 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:31.267718 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:31.398358 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.96 GiB, free memory in arenas 5.36 MiB, will set to 1.45 GiB (RSS), difference: 35.70 MiB 2024.10.22 01:16:31.512824 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_421_421_0 (state Temporary) 2024.10.22 01:16:31.578718 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_421_421_0 to 202410_434_434_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:31.770840 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 643000 2024.10.22 01:16:31.786358 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_415_193} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_415_193 to 202410_1_415_193 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:31.787463 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6426 2024.10.22 01:16:31.788185 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_415_193} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_414_192, 202410_415_415_0] -> 202410_1_415_193 2024.10.22 01:16:31.823006 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.07 MiB. 2024.10.22 01:16:31.875929 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(1fb457f097b161b0ca88f57433aa466e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1fb457f097b161b0ca88f57433aa466e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:16:31.881628 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 464, file segment info: File segment: [0, 33554431], key: 1fb457f097b161b0ca88f57433aa466e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:16:31.884228 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:31.916852 [ 1673 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/cxp/qygcigsljdmdjogwvsdutxkgodrbd, size 359 2024.10.22 01:16:32.111485 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:16:32.135452 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:16:32.172632 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:16:32.220839 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:16:32.248844 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:16:32.292037 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:16:32.305001 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:32.313285 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:32.320357 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:16:32.560868 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_623_246, 202410_624_624_0] 2024.10.22 01:16:32.567418 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_623_246, 202410_624_624_0] 2024.10.22 01:16:32.668237 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/aak/yivywkgpczhviiusinnlkosxaanqc (LogSeriesLimiter: on interval from 2024-10-22 01:16:14 to 2024-10-22 01:16:32 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:16:32.680347 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_623_246, 202410_624_624_0] 2024.10.22 01:16:32.721612 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_404_404_0 (state Temporary) 2024.10.22 01:16:32.746811 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(00554d01bccf09e3016b24aa52736585) : [0, 5145]: Setting download as failed: File segment: [0, 5145], key: 00554d01bccf09e3016b24aa52736585, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:32.748698 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(00554d01bccf09e3016b24aa52736585): No space left in cache to reserve 5146 bytes, will continue without cache download 2024.10.22 01:16:32.766585 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 00554d01bccf09e3016b24aa52736585, offset: 0, size: 0 2024.10.22 01:16:32.768186 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 00554d01bccf09e3016b24aa52736585 for removal 2024.10.22 01:16:32.770648 [ 1476 ] {} CacheMetadata: Key 00554d01bccf09e3016b24aa52736585 is removed from metadata 2024.10.22 01:16:32.771711 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d63df878c7607adeabb9ff4be4890afd, offset: 0, size: 0 2024.10.22 01:16:32.773129 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d63df878c7607adeabb9ff4be4890afd for removal 2024.10.22 01:16:32.778948 [ 1476 ] {} CacheMetadata: Key d63df878c7607adeabb9ff4be4890afd is removed from metadata 2024.10.22 01:16:32.789683 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_404_404_0 to 202410_411_411_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:32.815138 [ 1266 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 4 parts from 202410_1_406_81 to 202410_409_409_0 2024.10.22 01:16:32.821436 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 643000 2024.10.22 01:16:32.849685 [ 1266 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:32.878431 [ 1263 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:16:32.960622 [ 1266 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:32.978078 [ 1263 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:16:32.988327 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_409_82} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_406_81 to 202410_409_409_0 into Compact with storage Full 2024.10.22 01:16:33.020834 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_409_82} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:33.027792 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_409_82} MergeTreeSequentialSource: Reading 81 marks from part 202410_1_406_81, total 649288 rows starting from the beginning of the part 2024.10.22 01:16:33.059487 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_409_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_407_407_0, total 1542 rows starting from the beginning of the part 2024.10.22 01:16:33.069151 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_409_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_408_408_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:16:33.077118 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_409_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_409_409_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:16:33.297561 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 37 bytes for f540f3ed2c70f074ee2336fff9dbeb84:0) 2024.10.22 01:16:33.299139 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0510ac76201b5fafd1626b7eb7bbadb6, offset: 0, size: 4 2024.10.22 01:16:33.303805 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0510ac76201b5fafd1626b7eb7bbadb6 for removal 2024.10.22 01:16:33.307323 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 20b3fb98c67bc8642089a842d9a6cec8, offset: 0, size: 4 2024.10.22 01:16:33.307595 [ 1476 ] {} CacheMetadata: Key 0510ac76201b5fafd1626b7eb7bbadb6 is removed from metadata 2024.10.22 01:16:33.311949 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 20b3fb98c67bc8642089a842d9a6cec8 for removal 2024.10.22 01:16:33.317769 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 862bad8bf867b966b9cc90071cfa716e, offset: 0, size: 4 2024.10.22 01:16:33.318757 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_85_85_0 to 202410_88_88_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:33.321369 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 862bad8bf867b966b9cc90071cfa716e for removal 2024.10.22 01:16:33.327657 [ 1476 ] {} CacheMetadata: Key 862bad8bf867b966b9cc90071cfa716e is removed from metadata 2024.10.22 01:16:33.328234 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6a76706c2c4e50ca6b372cfd4000c3fe, offset: 0, size: 4 2024.10.22 01:16:33.331080 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6a76706c2c4e50ca6b372cfd4000c3fe for removal 2024.10.22 01:16:33.438526 [ 1476 ] {} CacheMetadata: Key 6a76706c2c4e50ca6b372cfd4000c3fe is removed from metadata 2024.10.22 01:16:33.463229 [ 1476 ] {} CacheMetadata: Key 20b3fb98c67bc8642089a842d9a6cec8 is removed from metadata 2024.10.22 01:16:33.520590 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:16:33.579200 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_723_336} MergeTask::MergeProjectionsStage: Merge sorted 45408 rows, containing 13 columns (13 merged, 0 gathered) in 2.65643719 sec., 17093.57186043612 rows/sec., 2.73 MiB/sec. 2024.10.22 01:16:33.644937 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_357_135, 202410_358_358_0] 2024.10.22 01:16:33.647120 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_357_135, 202410_358_358_0] 2024.10.22 01:16:33.741169 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_723_336} MergedBlockOutputStream: filled checksums 202410_1_723_336 (state Temporary) 2024.10.22 01:16:33.790893 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_357_135, 202410_358_358_0] 2024.10.22 01:16:33.837769 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_723_336} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_723_336 to 202410_1_723_336 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:33.844266 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_723_336} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_722_335, 202410_723_723_0] -> 202410_1_723_336 2024.10.22 01:16:33.875206 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.67 MiB. 2024.10.22 01:16:34.072133 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(82496f13c7d38b25b9b2251d87298bc2) : [0, 266]: Setting download as failed: File segment: [0, 266], key: 82496f13c7d38b25b9b2251d87298bc2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:34.073392 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(82496f13c7d38b25b9b2251d87298bc2): No space left in cache to reserve 267 bytes, will continue without cache download 2024.10.22 01:16:34.545009 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 225 entries to flush up to offset 43538 2024.10.22 01:16:34.727492 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_346_69, 202410_347_347_0, 202410_348_348_0, 202410_349_349_0, 202410_350_350_0, 202410_351_351_0] 2024.10.22 01:16:34.731599 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_346_69, 202410_347_347_0, 202410_348_348_0, 202410_349_349_0, 202410_350_350_0, 202410_351_351_0] 2024.10.22 01:16:34.768448 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_346_69, 202410_347_347_0, 202410_348_348_0, 202410_349_349_0, 202410_350_350_0, 202410_351_351_0] 2024.10.22 01:16:34.879550 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b68471eaf488b72b03f9ad045a8da931) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b68471eaf488b72b03f9ad045a8da931, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:16:34.899521 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 74, file segment info: File segment: [0, 33554431], key: b68471eaf488b72b03f9ad045a8da931, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:16:34.908236 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:34.917824 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 98 entries to flush up to offset 44999 2024.10.22 01:16:35.070951 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:35.072581 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:35.154287 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_415_415_0 (state Temporary) 2024.10.22 01:16:35.169667 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:35.202661 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:35.278186 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_716_716_0 (state Temporary) 2024.10.22 01:16:35.317324 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_415_415_0 to 202410_421_421_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:35.329743 [ 1224 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 01:16:35.344975 [ 1181 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_415_193 to 202410_416_416_0 2024.10.22 01:16:35.353414 [ 1181 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.19 MiB using storage policy from min volume index 0 2024.10.22 01:16:35.358624 [ 1181 ] {} DiskLocal: Reserved 2.19 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:35.381152 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_416_194} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_415_193 to 202410_416_416_0 into Wide with storage Full 2024.10.22 01:16:35.424606 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_416_194} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:35.426654 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_716_716_0 to 202410_729_729_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:35.452650 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_723_336 to 202410_724_724_0 2024.10.22 01:16:35.456253 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 10.09 MiB, will set to 1.45 GiB (RSS), difference: 7.01 MiB 2024.10.22 01:16:35.458465 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.62 MiB using storage policy from min volume index 0 2024.10.22 01:16:35.484153 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 44999 2024.10.22 01:16:35.549210 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 43538 2024.10.22 01:16:35.557789 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_416_194} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_415_193, total 42515 rows starting from the beginning of the part 2024.10.22 01:16:35.563887 [ 1240 ] {} DiskLocal: Reserved 1.62 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:35.596015 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_724_337} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_723_336 to 202410_724_724_0 into Compact with storage Full 2024.10.22 01:16:35.669803 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_724_337} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:35.672366 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_409_82/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:16:35.685100 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_724_337} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_723_336, total 45408 rows starting from the beginning of the part 2024.10.22 01:16:35.695095 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(28fe7c16626d948cfc0ad348a08efc8d) : [0, 2212832]: Setting download as failed: File segment: [0, 2212832], key: 28fe7c16626d948cfc0ad348a08efc8d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:35.696587 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(28fe7c16626d948cfc0ad348a08efc8d): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:16:35.738390 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_724_337} MergeTreeSequentialSource: Reading 2 marks from part 202410_724_724_0, total 66 rows starting from the beginning of the part 2024.10.22 01:16:35.897326 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_416_194} MergeTreeSequentialSource: Reading 2 marks from part 202410_416_416_0, total 205 rows starting from the beginning of the part 2024.10.22 01:16:35.973205 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 28fe7c16626d948cfc0ad348a08efc8d, offset: 0, size: 0 2024.10.22 01:16:35.974500 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 28fe7c16626d948cfc0ad348a08efc8d for removal 2024.10.22 01:16:35.976466 [ 1476 ] {} CacheMetadata: Key 28fe7c16626d948cfc0ad348a08efc8d is removed from metadata 2024.10.22 01:16:35.979054 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 82496f13c7d38b25b9b2251d87298bc2, offset: 0, size: 0 2024.10.22 01:16:35.982415 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 82496f13c7d38b25b9b2251d87298bc2 for removal 2024.10.22 01:16:35.987569 [ 1476 ] {} CacheMetadata: Key 82496f13c7d38b25b9b2251d87298bc2 is removed from metadata 2024.10.22 01:16:36.201227 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 358 2024.10.22 01:16:36.589342 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:36.596183 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:36.659933 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_340_340_0 (state Temporary) 2024.10.22 01:16:36.872073 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d4b685b24b3b7ce2faa71b44ffc336d4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d4b685b24b3b7ce2faa71b44ffc336d4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:16:36.875366 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 112, file segment info: File segment: [0, 33554431], key: d4b685b24b3b7ce2faa71b44ffc336d4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:16:36.879834 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:36.872920 [ 1232 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 217 ms. 2024.10.22 01:16:36.976070 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_340_340_0 to 202410_350_350_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:37.047646 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_624_247, 202410_625_625_0] 2024.10.22 01:16:37.052326 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_624_247, 202410_625_625_0] 2024.10.22 01:16:37.065234 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_624_247, 202410_625_625_0] 2024.10.22 01:16:37.167733 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 358 2024.10.22 01:16:37.228871 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3386 2024.10.22 01:16:37.737174 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a83af2ee4d35e3ebedf3282121e45dfc) : [0, 195]: Setting download as failed: File segment: [0, 195], key: a83af2ee4d35e3ebedf3282121e45dfc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:37.738663 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a83af2ee4d35e3ebedf3282121e45dfc): No space left in cache to reserve 196 bytes, will continue without cache download 2024.10.22 01:16:38.215799 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_416_194} MergeTask::MergeProjectionsStage: Merge sorted 42720 rows, containing 13 columns (13 merged, 0 gathered) in 2.735480084 sec., 15617.002752047818 rows/sec., 6.76 MiB/sec. 2024.10.22 01:16:38.363982 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.96 GiB, free memory in arenas 14.90 MiB, will set to 1.46 GiB (RSS), difference: -31.81 MiB 2024.10.22 01:16:38.586842 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_416_194} MergedBlockOutputStream: filled checksums 202410_1_416_194 (state Temporary) 2024.10.22 01:16:38.825136 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c2c0e5985303f508088d8cde091b0e2b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c2c0e5985303f508088d8cde091b0e2b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:16:38.852752 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 530, file segment info: File segment: [0, 33554431], key: c2c0e5985303f508088d8cde091b0e2b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:16:38.856468 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:16:38.883491 [ 1690 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qwb/dpnbhphpgwfmbszskwcozyedxsvql, size 112 2024.10.22 01:16:38.976811 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_724_337} MergeTask::MergeProjectionsStage: Merge sorted 45474 rows, containing 13 columns (13 merged, 0 gathered) in 3.386780167 sec., 13426.912216827093 rows/sec., 2.14 MiB/sec. 2024.10.22 01:16:39.148821 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:16:39.174196 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_724_337} MergedBlockOutputStream: filled checksums 202410_1_724_337 (state Temporary) 2024.10.22 01:16:39.196709 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_416_194} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_416_194 to 202410_1_416_194 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:39.206914 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_416_194} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_415_193, 202410_416_416_0] -> 202410_1_416_194 2024.10.22 01:16:39.234198 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.07 MiB. 2024.10.22 01:16:39.266352 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:16:39.279816 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:16:39.291809 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 6441 2024.10.22 01:16:39.292019 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:16:39.316646 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:16:39.337583 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:16:39.366603 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:16:39.373397 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_724_337} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_724_337 to 202410_1_724_337 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:39.376453 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_724_337} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_723_336, 202410_724_724_0] -> 202410_1_724_337 2024.10.22 01:16:39.378121 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:16:39.481139 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.67 MiB. 2024.10.22 01:16:39.481365 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:16:39.560519 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:39.611804 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qjh/nqirnmvrobagvysohkhbkcjzkiewc (LogSeriesLimiter: on interval from 2024-10-22 01:16:32 to 2024-10-22 01:16:39 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:16:39.619427 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:39.620422 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 45077 2024.10.22 01:16:39.685309 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_422_422_0 (state Temporary) 2024.10.22 01:16:39.689691 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3eee27040f3ca4e1a31afa880b761b2a) : [0, 74833]: Setting download as failed: File segment: [0, 74833], key: 3eee27040f3ca4e1a31afa880b761b2a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:39.692921 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3eee27040f3ca4e1a31afa880b761b2a): No space left in cache to reserve 74834 bytes, will continue without cache download 2024.10.22 01:16:39.747790 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3eee27040f3ca4e1a31afa880b761b2a, offset: 0, size: 0 2024.10.22 01:16:39.750298 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3eee27040f3ca4e1a31afa880b761b2a for removal 2024.10.22 01:16:39.760752 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a83af2ee4d35e3ebedf3282121e45dfc, offset: 0, size: 0 2024.10.22 01:16:39.763017 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a83af2ee4d35e3ebedf3282121e45dfc for removal 2024.10.22 01:16:39.767819 [ 1476 ] {} CacheMetadata: Key 3eee27040f3ca4e1a31afa880b761b2a is removed from metadata 2024.10.22 01:16:39.774660 [ 1476 ] {} CacheMetadata: Key a83af2ee4d35e3ebedf3282121e45dfc is removed from metadata 2024.10.22 01:16:39.829324 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 768 entries to flush up to offset 643768 2024.10.22 01:16:40.058300 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_422_422_0 to 202410_435_435_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:40.062780 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:40.065115 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:16:40.066891 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:40.067650 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:16:40.254288 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_717_717_0 (state Temporary) 2024.10.22 01:16:40.260270 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6441 2024.10.22 01:16:40.432497 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_358_136, 202410_359_359_0] 2024.10.22 01:16:40.487747 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_717_717_0 to 202410_730_730_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:40.526110 [ 1160 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_724_337 to 202410_725_725_0 2024.10.22 01:16:40.538315 [ 1160 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.63 MiB using storage policy from min volume index 0 2024.10.22 01:16:40.542672 [ 1160 ] {} DiskLocal: Reserved 1.63 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:40.436068 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_358_136, 202410_359_359_0] 2024.10.22 01:16:40.563620 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_725_338} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_724_337 to 202410_725_725_0 into Compact with storage Full 2024.10.22 01:16:40.571133 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_358_136, 202410_359_359_0] 2024.10.22 01:16:40.627582 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45077 2024.10.22 01:16:40.643318 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:40.648002 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:40.714977 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_725_338} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:40.857006 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_725_338} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_724_337, total 45474 rows starting from the beginning of the part 2024.10.22 01:16:40.886696 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fcab3eff26329d5088db64763cf6185c) : [0, 229]: Setting download as failed: File segment: [0, 229], key: fcab3eff26329d5088db64763cf6185c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:40.890373 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fcab3eff26329d5088db64763cf6185c): No space left in cache to reserve 230 bytes, will continue without cache download 2024.10.22 01:16:41.047634 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_725_338} MergeTreeSequentialSource: Reading 2 marks from part 202410_725_725_0, total 65 rows starting from the beginning of the part 2024.10.22 01:16:41.070450 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_405_405_0 (state Temporary) 2024.10.22 01:16:41.472177 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_405_405_0 to 202410_412_412_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:41.526545 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 643768 2024.10.22 01:16:41.534637 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 13.28 MiB, will set to 1.45 GiB (RSS), difference: 26.10 MiB 2024.10.22 01:16:41.955929 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(49e4b54fed1314769254fed5d0a97d68) : [0, 56143]: Setting download as failed: File segment: [0, 56143], key: 49e4b54fed1314769254fed5d0a97d68, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:41.959502 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(49e4b54fed1314769254fed5d0a97d68): No space left in cache to reserve 56144 bytes, will continue without cache download 2024.10.22 01:16:42.196204 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 49e4b54fed1314769254fed5d0a97d68, offset: 0, size: 0 2024.10.22 01:16:42.198428 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 49e4b54fed1314769254fed5d0a97d68 for removal 2024.10.22 01:16:42.201330 [ 1476 ] {} CacheMetadata: Key 49e4b54fed1314769254fed5d0a97d68 is removed from metadata 2024.10.22 01:16:42.206275 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fcab3eff26329d5088db64763cf6185c, offset: 0, size: 0 2024.10.22 01:16:42.212066 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fcab3eff26329d5088db64763cf6185c for removal 2024.10.22 01:16:42.217544 [ 1476 ] {} CacheMetadata: Key fcab3eff26329d5088db64763cf6185c is removed from metadata 2024.10.22 01:16:42.618782 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_625_248, 202410_626_626_0] 2024.10.22 01:16:42.623960 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_625_248, 202410_626_626_0] 2024.10.22 01:16:42.639877 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_625_248, 202410_626_626_0] 2024.10.22 01:16:43.019343 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_725_338} MergeTask::MergeProjectionsStage: Merge sorted 45539 rows, containing 13 columns (13 merged, 0 gathered) in 2.46582295 sec., 18468.073711456047 rows/sec., 2.95 MiB/sec. 2024.10.22 01:16:43.080098 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 212 entries to flush up to offset 43750 2024.10.22 01:16:43.188936 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_725_338} MergedBlockOutputStream: filled checksums 202410_1_725_338 (state Temporary) 2024.10.22 01:16:43.345538 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:43.354550 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:43.413296 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_416_416_0 (state Temporary) 2024.10.22 01:16:43.415690 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.96 GiB, free memory in arenas 16.35 MiB, will set to 1.46 GiB (RSS), difference: 3.80 MiB 2024.10.22 01:16:43.515396 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_725_338} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_725_338 to 202410_1_725_338 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:43.524278 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_725_338} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_724_337, 202410_725_725_0] -> 202410_1_725_338 2024.10.22 01:16:43.531921 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(34f44270ff2b649b47349a5cac70ed1d) : [0, 244]: Setting download as failed: File segment: [0, 244], key: 34f44270ff2b649b47349a5cac70ed1d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:43.535308 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(34f44270ff2b649b47349a5cac70ed1d): No space left in cache to reserve 245 bytes, will continue without cache download 2024.10.22 01:16:43.668747 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.68 MiB. 2024.10.22 01:16:43.773768 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_416_416_0 to 202410_422_422_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:43.897794 [ 1187 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_416_194 to 202410_417_417_0 2024.10.22 01:16:43.916686 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 43750 2024.10.22 01:16:43.921313 [ 1187 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.19 MiB using storage policy from min volume index 0 2024.10.22 01:16:44.071530 [ 1187 ] {} DiskLocal: Reserved 2.19 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:44.082561 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_417_195} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_416_194 to 202410_417_417_0 into Wide with storage Full 2024.10.22 01:16:44.113533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(62e389974f9d605ed49d6282e6fbd37f) : [0, 618284]: Setting download as failed: File segment: [0, 618284], key: 62e389974f9d605ed49d6282e6fbd37f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:44.116393 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(62e389974f9d605ed49d6282e6fbd37f): No space left in cache to reserve 618285 bytes, will continue without cache download 2024.10.22 01:16:44.200832 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_417_195} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:44.238571 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_417_195} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_416_194, total 42720 rows starting from the beginning of the part 2024.10.22 01:16:44.389321 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 62e389974f9d605ed49d6282e6fbd37f, offset: 0, size: 0 2024.10.22 01:16:44.391099 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 62e389974f9d605ed49d6282e6fbd37f for removal 2024.10.22 01:16:44.394636 [ 1476 ] {} CacheMetadata: Key 62e389974f9d605ed49d6282e6fbd37f is removed from metadata 2024.10.22 01:16:44.397204 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 34f44270ff2b649b47349a5cac70ed1d, offset: 0, size: 0 2024.10.22 01:16:44.401349 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 34f44270ff2b649b47349a5cac70ed1d for removal 2024.10.22 01:16:44.476698 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_417_195} MergeTreeSequentialSource: Reading 2 marks from part 202410_417_417_0, total 156 rows starting from the beginning of the part 2024.10.22 01:16:44.504849 [ 1476 ] {} CacheMetadata: Key 34f44270ff2b649b47349a5cac70ed1d is removed from metadata 2024.10.22 01:16:44.670900 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 359 2024.10.22 01:16:44.764912 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 45158 2024.10.22 01:16:44.876569 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 3423 2024.10.22 01:16:45.001324 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:45.004484 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:45.099289 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:45.102539 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:45.121151 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_371_74, 202410_372_372_0, 202410_373_373_0, 202410_374_374_0, 202410_375_375_0, 202410_376_376_0] 2024.10.22 01:16:45.127408 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_371_74, 202410_372_372_0, 202410_373_373_0, 202410_374_374_0, 202410_375_375_0, 202410_376_376_0] 2024.10.22 01:16:45.173221 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_371_74, 202410_372_372_0, 202410_373_373_0, 202410_374_374_0, 202410_375_375_0, 202410_376_376_0] 2024.10.22 01:16:45.183303 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_718_718_0 (state Temporary) 2024.10.22 01:16:45.255953 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_718_718_0 to 202410_731_731_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:45.281637 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_341_341_0 (state Temporary) 2024.10.22 01:16:45.283542 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_725_338 to 202410_726_726_0 2024.10.22 01:16:45.289250 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.63 MiB using storage policy from min volume index 0 2024.10.22 01:16:45.294942 [ 1198 ] {} DiskLocal: Reserved 1.63 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:45.319072 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45158 2024.10.22 01:16:45.334697 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(dc7b3f3de303e015de651ae6063ea879) : [0, 238]: Setting download as failed: File segment: [0, 238], key: dc7b3f3de303e015de651ae6063ea879, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:45.337188 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_726_339} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_725_338 to 202410_726_726_0 into Compact with storage Full 2024.10.22 01:16:45.345571 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(dc7b3f3de303e015de651ae6063ea879): No space left in cache to reserve 239 bytes, will continue without cache download 2024.10.22 01:16:45.442565 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.96 GiB, free memory in arenas 14.35 MiB, will set to 1.45 GiB (RSS), difference: -39.54 MiB 2024.10.22 01:16:45.495246 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_726_339} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:45.520282 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_726_339} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_725_338, total 45539 rows starting from the beginning of the part 2024.10.22 01:16:45.592079 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_341_341_0 to 202410_351_351_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:45.664172 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_726_339} MergeTreeSequentialSource: Reading 2 marks from part 202410_726_726_0, total 51 rows starting from the beginning of the part 2024.10.22 01:16:45.673695 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 359 2024.10.22 01:16:46.255449 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f96e94e9c3834675c5bdd25e6a5f2ce2) : [0, 389425]: Setting download as failed: File segment: [0, 389425], key: f96e94e9c3834675c5bdd25e6a5f2ce2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:46.258615 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f96e94e9c3834675c5bdd25e6a5f2ce2): No space left in cache to reserve 389426 bytes, will continue without cache download 2024.10.22 01:16:46.383623 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f96e94e9c3834675c5bdd25e6a5f2ce2, offset: 0, size: 0 2024.10.22 01:16:46.385249 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f96e94e9c3834675c5bdd25e6a5f2ce2 for removal 2024.10.22 01:16:46.386830 [ 1476 ] {} CacheMetadata: Key f96e94e9c3834675c5bdd25e6a5f2ce2 is removed from metadata 2024.10.22 01:16:46.387717 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dc7b3f3de303e015de651ae6063ea879, offset: 0, size: 0 2024.10.22 01:16:46.388925 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dc7b3f3de303e015de651ae6063ea879 for removal 2024.10.22 01:16:46.391019 [ 1476 ] {} CacheMetadata: Key dc7b3f3de303e015de651ae6063ea879 is removed from metadata 2024.10.22 01:16:46.625201 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_409_82/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:16:46.765836 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_626_249, 202410_627_627_0] 2024.10.22 01:16:46.771760 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_626_249, 202410_627_627_0] 2024.10.22 01:16:46.779314 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_626_249, 202410_627_627_0] 2024.10.22 01:16:47.468430 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(871221861e535b48fb44e6039563df07) : [0, 239]: Setting download as failed: File segment: [0, 239], key: 871221861e535b48fb44e6039563df07, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:47.469124 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(871221861e535b48fb44e6039563df07): No space left in cache to reserve 240 bytes, will continue without cache download 2024.10.22 01:16:47.580991 [ 1213 ] {} DatabaseCatalog: Execution took 241 ms. 2024.10.22 01:16:47.763455 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 6453 2024.10.22 01:16:47.864986 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:47.869610 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:47.992695 [ 1214 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:16:48.025348 [ 1214 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:16:48.118898 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_423_423_0 (state Temporary) 2024.10.22 01:16:48.178569 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_423_423_0 to 202410_436_436_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:48.215791 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_431_86 to 202410_436_436_0 2024.10.22 01:16:48.217387 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_726_339} MergeTask::MergeProjectionsStage: Merge sorted 45590 rows, containing 13 columns (13 merged, 0 gathered) in 2.884874073 sec., 15803.116131370909 rows/sec., 2.52 MiB/sec. 2024.10.22 01:16:48.230332 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:48.234528 [ 1220 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:48.234596 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6453 2024.10.22 01:16:48.247401 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_417_195} MergeTask::MergeProjectionsStage: Merge sorted 42876 rows, containing 13 columns (13 merged, 0 gathered) in 4.168272746 sec., 10286.275062289313 rows/sec., 4.45 MiB/sec. 2024.10.22 01:16:48.279637 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c8c2347de1255581dcc38f98ffc58108) : [0, 856979]: Setting download as failed: File segment: [0, 856979], key: c8c2347de1255581dcc38f98ffc58108, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:48.282171 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c8c2347de1255581dcc38f98ffc58108): No space left in cache to reserve 856980 bytes, will continue without cache download 2024.10.22 01:16:48.284678 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_431_86 to 202410_436_436_0 into Compact with storage Full 2024.10.22 01:16:48.311747 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_417_195} MergedBlockOutputStream: filled checksums 202410_1_417_195 (state Temporary) 2024.10.22 01:16:48.325076 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:48.352376 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_431_86, total 6474 rows starting from the beginning of the part 2024.10.22 01:16:48.373659 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_432_432_0, total 25 rows starting from the beginning of the part 2024.10.22 01:16:48.399362 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_433_433_0, total 15 rows starting from the beginning of the part 2024.10.22 01:16:48.427843 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_434_434_0, total 18 rows starting from the beginning of the part 2024.10.22 01:16:48.466190 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_435_435_0, total 15 rows starting from the beginning of the part 2024.10.22 01:16:48.491988 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_436_436_0, total 12 rows starting from the beginning of the part 2024.10.22 01:16:48.519970 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_726_339} MergedBlockOutputStream: filled checksums 202410_1_726_339 (state Temporary) 2024.10.22 01:16:48.531472 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1920 entries to flush up to offset 645688 2024.10.22 01:16:48.600953 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_726_339} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_726_339 to 202410_1_726_339 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:48.708956 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_726_339} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_725_338, 202410_726_726_0] -> 202410_1_726_339 2024.10.22 01:16:48.764789 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c8c2347de1255581dcc38f98ffc58108, offset: 0, size: 0 2024.10.22 01:16:48.766548 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c8c2347de1255581dcc38f98ffc58108 for removal 2024.10.22 01:16:48.771147 [ 1476 ] {} CacheMetadata: Key c8c2347de1255581dcc38f98ffc58108 is removed from metadata 2024.10.22 01:16:48.774089 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.67 MiB. 2024.10.22 01:16:48.774699 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 871221861e535b48fb44e6039563df07, offset: 0, size: 0 2024.10.22 01:16:48.778270 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 871221861e535b48fb44e6039563df07 for removal 2024.10.22 01:16:48.781768 [ 1476 ] {} CacheMetadata: Key 871221861e535b48fb44e6039563df07 is removed from metadata 2024.10.22 01:16:49.329657 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 45234 2024.10.22 01:16:49.430110 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_417_195} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_417_195 to 202410_1_417_195 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:49.433661 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_417_195} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_416_194, 202410_417_417_0] -> 202410_1_417_195 2024.10.22 01:16:49.469524 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.01 MiB. 2024.10.22 01:16:49.572071 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} MergeTask::MergeProjectionsStage: Merge sorted 6559 rows, containing 12 columns (12 merged, 0 gathered) in 1.293352428 sec., 5071.3168800731555 rows/sec., 803.46 KiB/sec. 2024.10.22 01:16:49.734115 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:49.737392 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:49.789544 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:49.794317 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:49.814273 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_719_719_0 (state Temporary) 2024.10.22 01:16:49.920034 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} MergedBlockOutputStream: filled checksums 202410_1_436_87 (state Temporary) 2024.10.22 01:16:49.926802 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(20ede69e6dce4e269a50e2b23e4b306c) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 20ede69e6dce4e269a50e2b23e4b306c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:49.937396 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(20ede69e6dce4e269a50e2b23e4b306c): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:16:49.945018 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_406_406_0 (state Temporary) 2024.10.22 01:16:49.966409 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_359_137, 202410_360_360_0] 2024.10.22 01:16:49.971128 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_359_137, 202410_360_360_0] 2024.10.22 01:16:49.979585 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_719_719_0 to 202410_732_732_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:49.981259 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 9.54 MiB, will set to 1.45 GiB (RSS), difference: 22.74 MiB 2024.10.22 01:16:49.984065 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_359_137, 202410_360_360_0] 2024.10.22 01:16:50.015271 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_726_339 to 202410_727_727_0 2024.10.22 01:16:50.027953 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.63 MiB using storage policy from min volume index 0 2024.10.22 01:16:50.051661 [ 1153 ] {} DiskLocal: Reserved 1.63 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:50.066951 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_406_406_0 to 202410_413_413_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:50.068014 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:16:50.069749 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45234 2024.10.22 01:16:50.070391 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:16:50.170600 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_727_340} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_726_339 to 202410_727_727_0 into Compact with storage Full 2024.10.22 01:16:50.185615 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_727_340} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:50.185636 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 645688 2024.10.22 01:16:50.193037 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_727_340} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_726_339, total 45590 rows starting from the beginning of the part 2024.10.22 01:16:50.226759 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_727_340} MergeTreeSequentialSource: Reading 2 marks from part 202410_727_727_0, total 70 rows starting from the beginning of the part 2024.10.22 01:16:50.551377 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_436_87 to 202410_1_436_87 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:50.587390 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_436_87} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_431_86, 202410_436_436_0] -> 202410_1_436_87 2024.10.22 01:16:50.653438 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.48 MiB. 2024.10.22 01:16:51.134451 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2aeea985688a2ffb54bb1068f73ba21b) : [0, 8625]: Setting download as failed: File segment: [0, 8625], key: 2aeea985688a2ffb54bb1068f73ba21b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:51.136770 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2aeea985688a2ffb54bb1068f73ba21b): No space left in cache to reserve 8626 bytes, will continue without cache download 2024.10.22 01:16:51.261320 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2aeea985688a2ffb54bb1068f73ba21b, offset: 0, size: 0 2024.10.22 01:16:51.265103 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2aeea985688a2ffb54bb1068f73ba21b for removal 2024.10.22 01:16:51.268132 [ 1476 ] {} CacheMetadata: Key 2aeea985688a2ffb54bb1068f73ba21b is removed from metadata 2024.10.22 01:16:51.271108 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 20ede69e6dce4e269a50e2b23e4b306c, offset: 0, size: 0 2024.10.22 01:16:51.273352 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 20ede69e6dce4e269a50e2b23e4b306c for removal 2024.10.22 01:16:51.283466 [ 1476 ] {} CacheMetadata: Key 20ede69e6dce4e269a50e2b23e4b306c is removed from metadata 2024.10.22 01:16:51.478690 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 212 entries to flush up to offset 43962 2024.10.22 01:16:52.093784 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:52.098987 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:52.152834 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(61364695485b46404ba59a1174e69af4) : [0, 169]: Setting download as failed: File segment: [0, 169], key: 61364695485b46404ba59a1174e69af4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:52.156305 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(61364695485b46404ba59a1174e69af4): No space left in cache to reserve 170 bytes, will continue without cache download 2024.10.22 01:16:52.168888 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_417_417_0 (state Temporary) 2024.10.22 01:16:52.252349 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_417_417_0 to 202410_423_423_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:52.368684 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_727_340} MergeTask::MergeProjectionsStage: Merge sorted 45660 rows, containing 13 columns (13 merged, 0 gathered) in 2.203906219 sec., 20717.75995110979 rows/sec., 3.30 MiB/sec. 2024.10.22 01:16:52.376350 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_417_195 to 202410_418_418_0 2024.10.22 01:16:52.381275 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.20 MiB using storage policy from min volume index 0 2024.10.22 01:16:52.383846 [ 1248 ] {} DiskLocal: Reserved 2.20 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:52.496909 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 43962 2024.10.22 01:16:52.550713 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_418_196} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_417_195 to 202410_418_418_0 into Wide with storage Full 2024.10.22 01:16:52.517836 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_727_340} MergedBlockOutputStream: filled checksums 202410_1_727_340 (state Temporary) 2024.10.22 01:16:52.673579 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_418_196} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:52.784061 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_418_196} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_417_195, total 42876 rows starting from the beginning of the part 2024.10.22 01:16:52.725753 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_727_340} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_727_340 to 202410_1_727_340 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:52.848100 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_727_340} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_726_339, 202410_727_727_0] -> 202410_1_727_340 2024.10.22 01:16:53.075609 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.67 MiB. 2024.10.22 01:16:53.182897 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_418_196} MergeTreeSequentialSource: Reading 2 marks from part 202410_418_418_0, total 161 rows starting from the beginning of the part 2024.10.22 01:16:53.183534 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 360 2024.10.22 01:16:53.271425 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(18aad6a43fe173b9a5ed841a113b04aa) : [0, 158077]: Setting download as failed: File segment: [0, 158077], key: 18aad6a43fe173b9a5ed841a113b04aa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:53.272352 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(18aad6a43fe173b9a5ed841a113b04aa): No space left in cache to reserve 158078 bytes, will continue without cache download 2024.10.22 01:16:53.286533 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 18aad6a43fe173b9a5ed841a113b04aa, offset: 0, size: 0 2024.10.22 01:16:53.287734 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 18aad6a43fe173b9a5ed841a113b04aa for removal 2024.10.22 01:16:53.290013 [ 1476 ] {} CacheMetadata: Key 18aad6a43fe173b9a5ed841a113b04aa is removed from metadata 2024.10.22 01:16:53.293046 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 61364695485b46404ba59a1174e69af4, offset: 0, size: 0 2024.10.22 01:16:53.295053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 61364695485b46404ba59a1174e69af4 for removal 2024.10.22 01:16:53.297056 [ 1476 ] {} CacheMetadata: Key 61364695485b46404ba59a1174e69af4 is removed from metadata 2024.10.22 01:16:53.444167 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 12.01 MiB, will set to 1.45 GiB (RSS), difference: 23.42 MiB 2024.10.22 01:16:53.474751 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_627_250, 202410_628_628_0] 2024.10.22 01:16:53.479280 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_627_250, 202410_628_628_0] 2024.10.22 01:16:53.490230 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_627_250, 202410_628_628_0] 2024.10.22 01:16:53.769764 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:53.775433 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:53.963555 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_342_342_0 (state Temporary) 2024.10.22 01:16:54.068010 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6db9ce617913f343b271d27ceeb80f94) : [0, 169]: Setting download as failed: File segment: [0, 169], key: 6db9ce617913f343b271d27ceeb80f94, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:54.069799 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6db9ce617913f343b271d27ceeb80f94): No space left in cache to reserve 170 bytes, will continue without cache download 2024.10.22 01:16:54.079445 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 45313 2024.10.22 01:16:54.209730 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_342_342_0 to 202410_352_352_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:54.304721 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 360 2024.10.22 01:16:54.763902 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:54.767536 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:54.880031 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_720_720_0 (state Temporary) 2024.10.22 01:16:55.031918 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(25dc918a2a1746b4d42ce5458d49402c) : [0, 167473]: Setting download as failed: File segment: [0, 167473], key: 25dc918a2a1746b4d42ce5458d49402c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:55.039705 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(25dc918a2a1746b4d42ce5458d49402c): No space left in cache to reserve 167474 bytes, will continue without cache download 2024.10.22 01:16:55.084826 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 25dc918a2a1746b4d42ce5458d49402c, offset: 0, size: 0 2024.10.22 01:16:55.186510 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 25dc918a2a1746b4d42ce5458d49402c for removal 2024.10.22 01:16:55.188343 [ 1476 ] {} CacheMetadata: Key 25dc918a2a1746b4d42ce5458d49402c is removed from metadata 2024.10.22 01:16:55.189103 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6db9ce617913f343b271d27ceeb80f94, offset: 0, size: 0 2024.10.22 01:16:55.193628 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6db9ce617913f343b271d27ceeb80f94 for removal 2024.10.22 01:16:55.196239 [ 1476 ] {} CacheMetadata: Key 6db9ce617913f343b271d27ceeb80f94 is removed from metadata 2024.10.22 01:16:55.336626 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_720_720_0 to 202410_733_733_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:55.384380 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_727_340 to 202410_728_728_0 2024.10.22 01:16:55.394222 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.63 MiB using storage policy from min volume index 0 2024.10.22 01:16:55.418535 [ 1166 ] {} DiskLocal: Reserved 1.63 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:16:55.435266 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45313 2024.10.22 01:16:55.466223 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_409_82} MergeTask::MergeProjectionsStage: Merge sorted 653902 rows, containing 4 columns (4 merged, 0 gathered) in 22.480837749 sec., 29087.083288481415 rows/sec., 426.28 KiB/sec. 2024.10.22 01:16:55.580331 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.96 GiB, free memory in arenas 13.83 MiB, will set to 1.47 GiB (RSS), difference: -30.53 MiB 2024.10.22 01:16:55.641739 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_418_196/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:16:55.659695 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_728_341} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_727_340 to 202410_728_728_0 into Compact with storage Full 2024.10.22 01:16:55.665093 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_409_82} MergedBlockOutputStream: filled checksums 202410_1_409_82 (state Temporary) 2024.10.22 01:16:55.688654 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_728_341} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:55.693430 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_728_341} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_727_340, total 45660 rows starting from the beginning of the part 2024.10.22 01:16:55.737664 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 6475 2024.10.22 01:16:55.750697 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_409_82} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_409_82 to 202410_1_409_82 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:55.760033 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_409_82} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 4 parts: [202410_1_406_81, 202410_409_409_0] -> 202410_1_409_82 2024.10.22 01:16:55.807302 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:16:55.951817 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_728_341} MergeTreeSequentialSource: Reading 2 marks from part 202410_728_728_0, total 72 rows starting from the beginning of the part 2024.10.22 01:16:56.148971 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:56.160917 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:56.317478 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_418_196} MergeTask::MergeProjectionsStage: Merge sorted 43037 rows, containing 13 columns (13 merged, 0 gathered) in 3.771777867 sec., 11410.268981251224 rows/sec., 4.94 MiB/sec. 2024.10.22 01:16:56.320034 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_424_424_0 (state Temporary) 2024.10.22 01:16:56.473550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(19c888a360d84149e70e78e3ca6a3d88) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 19c888a360d84149e70e78e3ca6a3d88, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:56.478439 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(19c888a360d84149e70e78e3ca6a3d88): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:16:56.499204 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_418_196} MergedBlockOutputStream: filled checksums 202410_1_418_196 (state Temporary) 2024.10.22 01:16:56.677273 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_424_424_0 to 202410_437_437_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:56.774373 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6475 2024.10.22 01:16:57.013223 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_628_251, 202410_629_629_0] 2024.10.22 01:16:57.020126 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_628_251, 202410_629_629_0] 2024.10.22 01:16:57.031614 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_628_251, 202410_629_629_0] 2024.10.22 01:16:57.195260 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1152 entries to flush up to offset 646840 2024.10.22 01:16:57.321278 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(472e3dbde1391a3265b9981868afc6c1) : [0, 44163]: Setting download as failed: File segment: [0, 44163], key: 472e3dbde1391a3265b9981868afc6c1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:57.325163 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(472e3dbde1391a3265b9981868afc6c1): No space left in cache to reserve 44164 bytes, will continue without cache download 2024.10.22 01:16:57.374722 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_418_196} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_418_196 to 202410_1_418_196 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:57.378371 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_418_196} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_417_195, 202410_418_418_0] -> 202410_1_418_196 2024.10.22 01:16:57.412501 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.01 MiB. 2024.10.22 01:16:57.459729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 472e3dbde1391a3265b9981868afc6c1, offset: 0, size: 0 2024.10.22 01:16:57.461757 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 472e3dbde1391a3265b9981868afc6c1 for removal 2024.10.22 01:16:57.567163 [ 1476 ] {} CacheMetadata: Key 472e3dbde1391a3265b9981868afc6c1 is removed from metadata 2024.10.22 01:16:57.568328 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 19c888a360d84149e70e78e3ca6a3d88, offset: 0, size: 0 2024.10.22 01:16:57.569643 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 19c888a360d84149e70e78e3ca6a3d88 for removal 2024.10.22 01:16:57.673637 [ 1476 ] {} CacheMetadata: Key 19c888a360d84149e70e78e3ca6a3d88 is removed from metadata 2024.10.22 01:16:58.402802 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:58.409166 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:58.639966 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3192162ff4fb1931bf9cb02d5a502ff1) : [0, 202]: Setting download as failed: File segment: [0, 202], key: 3192162ff4fb1931bf9cb02d5a502ff1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:58.643332 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3192162ff4fb1931bf9cb02d5a502ff1): No space left in cache to reserve 203 bytes, will continue without cache download 2024.10.22 01:16:58.819676 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 9.58 MiB, will set to 1.46 GiB (RSS), difference: 31.21 MiB 2024.10.22 01:16:58.838019 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_407_407_0 (state Temporary) 2024.10.22 01:16:58.934184 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_407_407_0 to 202410_414_414_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:16:58.983643 [ 1255 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 5 parts from 202410_1_409_82 to 202410_413_413_0 2024.10.22 01:16:58.986738 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 646840 2024.10.22 01:16:59.012019 [ 1255 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:59.019235 [ 1255 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:59.141061 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_413_83} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_409_82 to 202410_413_413_0 into Compact with storage Full 2024.10.22 01:16:59.207646 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_413_83} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:16:59.216820 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_413_83} MergeTreeSequentialSource: Reading 81 marks from part 202410_1_409_82, total 653902 rows starting from the beginning of the part 2024.10.22 01:16:59.234419 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_413_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_410_410_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:16:59.269001 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_413_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_411_411_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:16:59.280833 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_413_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_412_412_0, total 768 rows starting from the beginning of the part 2024.10.22 01:16:59.424328 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_413_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_413_413_0, total 1920 rows starting from the beginning of the part 2024.10.22 01:16:59.551839 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 45387 2024.10.22 01:16:59.568105 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_728_341} MergeTask::MergeProjectionsStage: Merge sorted 45732 rows, containing 13 columns (13 merged, 0 gathered) in 3.902976738 sec., 11717.210495964786 rows/sec., 1.87 MiB/sec. 2024.10.22 01:16:59.616350 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(aaa5d87fd262888bd982d976a198aae6) : [0, 278932]: Setting download as failed: File segment: [0, 278932], key: aaa5d87fd262888bd982d976a198aae6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:16:59.619088 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(aaa5d87fd262888bd982d976a198aae6): No space left in cache to reserve 278933 bytes, will continue without cache download 2024.10.22 01:16:59.731892 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_360_138, 202410_361_361_0] 2024.10.22 01:16:59.738802 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_360_138, 202410_361_361_0] 2024.10.22 01:16:59.748172 [ 1242 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 202 ms. 2024.10.22 01:16:59.764253 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_360_138, 202410_361_361_0] 2024.10.22 01:16:59.817225 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:16:59.819830 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:16:59.764919 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aaa5d87fd262888bd982d976a198aae6, offset: 0, size: 0 2024.10.22 01:16:59.863389 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aaa5d87fd262888bd982d976a198aae6 for removal 2024.10.22 01:16:59.865549 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_728_341} MergedBlockOutputStream: filled checksums 202410_1_728_341 (state Temporary) 2024.10.22 01:16:59.868684 [ 1476 ] {} CacheMetadata: Key aaa5d87fd262888bd982d976a198aae6 is removed from metadata 2024.10.22 01:16:59.869013 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3192162ff4fb1931bf9cb02d5a502ff1, offset: 0, size: 0 2024.10.22 01:16:59.871625 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3192162ff4fb1931bf9cb02d5a502ff1 for removal 2024.10.22 01:16:59.874827 [ 1476 ] {} CacheMetadata: Key 3192162ff4fb1931bf9cb02d5a502ff1 is removed from metadata 2024.10.22 01:16:59.932646 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_721_721_0 (state Temporary) 2024.10.22 01:17:00.056182 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 192 entries to flush up to offset 44154 2024.10.22 01:17:00.068938 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:17:00.071109 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:17:00.211472 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_721_721_0 to 202410_734_734_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:00.248022 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_729_729_0 to 202410_734_734_0 2024.10.22 01:17:00.271240 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_728_341} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_728_341 to 202410_1_728_341 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:00.272467 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:00.274272 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_728_341} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_727_340, 202410_728_728_0] -> 202410_1_728_341 2024.10.22 01:17:00.282501 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45387 2024.10.22 01:17:00.297795 [ 1172 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:00.326582 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_729_729_0 to 202410_734_734_0 into Compact with storage Full 2024.10.22 01:17:00.363931 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:00.369195 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:00.418572 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.68 MiB. 2024.10.22 01:17:00.428563 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_418_418_0 (state Temporary) 2024.10.22 01:17:00.507079 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_418_418_0 to 202410_424_424_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:00.515770 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:00.553352 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_729_729_0, total 98 rows starting from the beginning of the part 2024.10.22 01:17:00.647026 [ 1157 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_418_196 to 202410_419_419_0 2024.10.22 01:17:00.656936 [ 1157 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.21 MiB using storage policy from min volume index 0 2024.10.22 01:17:00.660665 [ 1157 ] {} DiskLocal: Reserved 2.21 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:00.666860 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 44154 2024.10.22 01:17:00.671697 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_419_197} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_418_196 to 202410_419_419_0 into Wide with storage Full 2024.10.22 01:17:00.683923 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_730_730_0, total 78 rows starting from the beginning of the part 2024.10.22 01:17:00.729781 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_731_731_0, total 81 rows starting from the beginning of the part 2024.10.22 01:17:00.768921 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_419_197} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:00.782268 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_732_732_0, total 76 rows starting from the beginning of the part 2024.10.22 01:17:00.783223 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_419_197} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_418_196, total 43037 rows starting from the beginning of the part 2024.10.22 01:17:00.862851 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_733_733_0, total 79 rows starting from the beginning of the part 2024.10.22 01:17:00.895084 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_734_734_0, total 74 rows starting from the beginning of the part 2024.10.22 01:17:01.007436 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_419_197} MergeTreeSequentialSource: Reading 2 marks from part 202410_419_419_0, total 207 rows starting from the beginning of the part 2024.10.22 01:17:01.060985 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0c5abd67cb3964160568fca9bfcd93e1) : [0, 200]: Setting download as failed: File segment: [0, 200], key: 0c5abd67cb3964160568fca9bfcd93e1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:01.062501 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0c5abd67cb3964160568fca9bfcd93e1): No space left in cache to reserve 201 bytes, will continue without cache download 2024.10.22 01:17:01.355563 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 9.88 MiB, will set to 1.45 GiB (RSS), difference: -27.08 MiB 2024.10.22 01:17:01.683565 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_629_252, 202410_630_630_0] 2024.10.22 01:17:01.685002 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_629_252, 202410_630_630_0] 2024.10.22 01:17:01.692099 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_629_252, 202410_630_630_0] 2024.10.22 01:17:01.806893 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 361 2024.10.22 01:17:02.003447 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(6a886020cdd87f81d704de8800de957a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6a886020cdd87f81d704de8800de957a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:17:02.008550 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 289760, file segment info: File segment: [0, 33554431], key: 6a886020cdd87f81d704de8800de957a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:17:02.102993 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:02.111005 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:02.118094 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:02.387191 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_343_343_0 (state Temporary) 2024.10.22 01:17:02.439919 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:02.444365 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:02.507153 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} MergeTask::MergeProjectionsStage: Merge sorted 486 rows, containing 13 columns (13 merged, 0 gathered) in 2.190796036 sec., 221.83717334423733 rows/sec., 37.83 KiB/sec. 2024.10.22 01:17:02.572946 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_343_343_0 to 202410_353_353_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:02.577676 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} MergedBlockOutputStream: filled checksums 202410_729_734_1 (state Temporary) 2024.10.22 01:17:02.682552 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 361 2024.10.22 01:17:02.780840 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_729_734_1 to 202410_729_734_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:02.784611 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_729_734_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_729_729_0, 202410_734_734_0] -> 202410_729_734_1 2024.10.22 01:17:02.926880 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:17:03.035344 [ 1182 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:17:03.162655 [ 1182 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:17:03.285599 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_413_83/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:17:03.358627 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a04b1dbea796a4530ba8c59d838b4923) : [0, 272767]: Setting download as failed: File segment: [0, 272767], key: a04b1dbea796a4530ba8c59d838b4923, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:03.360388 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a04b1dbea796a4530ba8c59d838b4923): No space left in cache to reserve 272768 bytes, will continue without cache download 2024.10.22 01:17:03.378494 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a04b1dbea796a4530ba8c59d838b4923, offset: 0, size: 0 2024.10.22 01:17:03.385806 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a04b1dbea796a4530ba8c59d838b4923 for removal 2024.10.22 01:17:03.388794 [ 1476 ] {} CacheMetadata: Key a04b1dbea796a4530ba8c59d838b4923 is removed from metadata 2024.10.22 01:17:03.389325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0c5abd67cb3964160568fca9bfcd93e1, offset: 0, size: 0 2024.10.22 01:17:03.390887 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0c5abd67cb3964160568fca9bfcd93e1 for removal 2024.10.22 01:17:03.394282 [ 1476 ] {} CacheMetadata: Key 0c5abd67cb3964160568fca9bfcd93e1 is removed from metadata 2024.10.22 01:17:03.867059 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_419_197} MergeTask::MergeProjectionsStage: Merge sorted 43244 rows, containing 13 columns (13 merged, 0 gathered) in 3.200889925 sec., 13509.992849879085 rows/sec., 5.85 MiB/sec. 2024.10.22 01:17:03.924769 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_419_197} MergedBlockOutputStream: filled checksums 202410_1_419_197 (state Temporary) 2024.10.22 01:17:03.951914 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(fafc483d554a876f0ed0a9cf867286cd) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fafc483d554a876f0ed0a9cf867286cd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:17:03.971233 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 574, file segment info: File segment: [0, 33554431], key: fafc483d554a876f0ed0a9cf867286cd, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:17:03.972650 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:04.277187 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 6498 2024.10.22 01:17:04.294755 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 45475 2024.10.22 01:17:04.611452 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:04.618381 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:04.675418 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_419_197} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_419_197 to 202410_1_419_197 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:04.678374 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_419_197} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_418_196, 202410_419_419_0] -> 202410_1_419_197 2024.10.22 01:17:04.689803 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.07 MiB. 2024.10.22 01:17:04.732440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(db3eff9c60a1255a4c7bafb29815b32f) : [0, 164]: Setting download as failed: File segment: [0, 164], key: db3eff9c60a1255a4c7bafb29815b32f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:04.734544 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(db3eff9c60a1255a4c7bafb29815b32f): No space left in cache to reserve 165 bytes, will continue without cache download 2024.10.22 01:17:04.762871 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_425_425_0 (state Temporary) 2024.10.22 01:17:04.797590 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_425_425_0 to 202410_438_438_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:04.945448 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6498 2024.10.22 01:17:05.030908 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_86_86_0 (state Temporary) 2024.10.22 01:17:05.138857 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:05.223876 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:05.387507 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_630_253, 202410_631_631_0] 2024.10.22 01:17:05.389614 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_630_253, 202410_631_631_0] 2024.10.22 01:17:05.400575 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_630_253, 202410_631_631_0] 2024.10.22 01:17:05.443085 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_722_722_0 (state Temporary) 2024.10.22 01:17:05.494437 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_722_722_0 to 202410_735_735_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:05.513264 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 14.31 MiB, will set to 1.45 GiB (RSS), difference: 18.87 MiB 2024.10.22 01:17:05.531185 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(255fcf15322de323d6f97814cd234232) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 255fcf15322de323d6f97814cd234232, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:17:05.650450 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 118713, file segment info: File segment: [0, 33554431], key: 255fcf15322de323d6f97814cd234232, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:17:05.655397 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:05.666758 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45475 2024.10.22 01:17:05.990487 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 768 entries to flush up to offset 647608 2024.10.22 01:17:06.428760 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d7520d16f6da66a8c9c5fa78b4e5d5e7) : [0, 130262]: Setting download as failed: File segment: [0, 130262], key: d7520d16f6da66a8c9c5fa78b4e5d5e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:06.432239 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d7520d16f6da66a8c9c5fa78b4e5d5e7): No space left in cache to reserve 130263 bytes, will continue without cache download 2024.10.22 01:17:06.570242 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d7520d16f6da66a8c9c5fa78b4e5d5e7, offset: 0, size: 0 2024.10.22 01:17:06.571628 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d7520d16f6da66a8c9c5fa78b4e5d5e7 for removal 2024.10.22 01:17:06.573084 [ 1476 ] {} CacheMetadata: Key d7520d16f6da66a8c9c5fa78b4e5d5e7 is removed from metadata 2024.10.22 01:17:06.574538 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: db3eff9c60a1255a4c7bafb29815b32f, offset: 0, size: 0 2024.10.22 01:17:06.578409 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key db3eff9c60a1255a4c7bafb29815b32f for removal 2024.10.22 01:17:06.580371 [ 1476 ] {} CacheMetadata: Key db3eff9c60a1255a4c7bafb29815b32f is removed from metadata 2024.10.22 01:17:06.727641 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:06.733561 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:06.944886 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_408_408_0 (state Temporary) 2024.10.22 01:17:07.113147 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(df63f52de8be00cb9c584a4c67c9a598) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: df63f52de8be00cb9c584a4c67c9a598, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:17:07.134572 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 975558, file segment info: File segment: [0, 33554431], key: df63f52de8be00cb9c584a4c67c9a598, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:17:07.138796 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:07.265649 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_408_408_0 to 202410_415_415_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:07.293700 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 647608 2024.10.22 01:17:07.893067 [ 1216 ] {} DatabaseCatalog: Execution took 209 ms. 2024.10.22 01:17:08.006840 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b0badf436c93df2620a4720cf8785284) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b0badf436c93df2620a4720cf8785284, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:17:08.009383 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1267, file segment info: File segment: [0, 33554431], key: b0badf436c93df2620a4720cf8785284, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:17:08.114163 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:08.153519 [ 1731 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qjh/nqirnmvrobagvysohkhbkcjzkiewc, size 118713 2024.10.22 01:17:08.182847 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 156 entries to flush up to offset 44310 2024.10.22 01:17:08.224756 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:17:08.233667 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:17:08.242207 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:17:08.265097 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:17:08.276178 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:17:08.287207 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:17:08.396155 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:17:08.426418 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:17:08.471106 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:17:08.504267 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/scp/cfdcvlkarmybtghpvascanojugkxl (LogSeriesLimiter: on interval from 2024-10-22 01:16:39 to 2024-10-22 01:17:08 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:17:08.585132 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:08.588959 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:08.752910 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_419_419_0 (state Temporary) 2024.10.22 01:17:08.793142 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.96 GiB, free memory in arenas 17.09 MiB, will set to 1.45 GiB (RSS), difference: 2.35 MiB 2024.10.22 01:17:08.983931 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4f7ad6730ac62223a9f7f0b4d8767628) : [0, 168]: Setting download as failed: File segment: [0, 168], key: 4f7ad6730ac62223a9f7f0b4d8767628, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:08.985217 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4f7ad6730ac62223a9f7f0b4d8767628): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 01:17:09.065084 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_361_139, 202410_362_362_0] 2024.10.22 01:17:09.065572 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_419_419_0 to 202410_425_425_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:09.066070 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_361_139, 202410_362_362_0] 2024.10.22 01:17:09.085240 [ 1217 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_419_197 to 202410_420_420_0 2024.10.22 01:17:09.085702 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_361_139, 202410_362_362_0] 2024.10.22 01:17:09.093365 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 44310 2024.10.22 01:17:09.197313 [ 1217 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.22 MiB using storage policy from min volume index 0 2024.10.22 01:17:09.204872 [ 1217 ] {} DiskLocal: Reserved 2.22 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:09.312451 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_420_198} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_419_197 to 202410_420_420_0 into Wide with storage Full 2024.10.22 01:17:09.392469 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_420_198} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:09.412664 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_420_198} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_419_197, total 43244 rows starting from the beginning of the part 2024.10.22 01:17:09.571855 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_420_198} MergeTreeSequentialSource: Reading 2 marks from part 202410_420_420_0, total 173 rows starting from the beginning of the part 2024.10.22 01:17:09.673638 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 45550 2024.10.22 01:17:09.885069 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(f90e11c3223206088a2d45316949929d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f90e11c3223206088a2d45316949929d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:17:09.898345 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 818, file segment info: File segment: [0, 33554431], key: f90e11c3223206088a2d45316949929d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:17:09.995275 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:09.997734 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:10.004150 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:10.065585 [ 1680 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/aak/yivywkgpczhviiusinnlkosxaanqc, size 975558 2024.10.22 01:17:10.066885 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:17:10.068195 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:17:10.183954 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 362 2024.10.22 01:17:10.188927 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_723_723_0 (state Temporary) 2024.10.22 01:17:10.252683 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_723_723_0 to 202410_736_736_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:10.291544 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45550 2024.10.22 01:17:10.368382 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:17:10.505157 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:17:10.573673 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:10.577594 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:10.677448 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:17:10.714463 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_344_344_0 (state Temporary) 2024.10.22 01:17:10.735748 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:17:10.792793 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:17:10.867728 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_344_344_0 to 202410_354_354_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:10.869779 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:17:10.896382 [ 1274 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_349_72 to 202410_354_354_0 2024.10.22 01:17:10.918164 [ 1274 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:10.924977 [ 1274 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:10.945299 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:17:10.977555 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_349_72 to 202410_354_354_0 into Compact with storage Full 2024.10.22 01:17:11.149707 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7f652bfebb9db8c834bda9f886febb65) : [0, 124176]: Setting download as failed: File segment: [0, 124176], key: 7f652bfebb9db8c834bda9f886febb65, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:11.152212 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7f652bfebb9db8c834bda9f886febb65): No space left in cache to reserve 124177 bytes, will continue without cache download 2024.10.22 01:17:11.170297 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 362 2024.10.22 01:17:11.174716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7f652bfebb9db8c834bda9f886febb65, offset: 0, size: 0 2024.10.22 01:17:11.252763 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7f652bfebb9db8c834bda9f886febb65 for removal 2024.10.22 01:17:11.257576 [ 1476 ] {} CacheMetadata: Key 7f652bfebb9db8c834bda9f886febb65 is removed from metadata 2024.10.22 01:17:11.263106 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4f7ad6730ac62223a9f7f0b4d8767628, offset: 0, size: 0 2024.10.22 01:17:11.266041 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4f7ad6730ac62223a9f7f0b4d8767628 for removal 2024.10.22 01:17:11.268028 [ 1476 ] {} CacheMetadata: Key 4f7ad6730ac62223a9f7f0b4d8767628 is removed from metadata 2024.10.22 01:17:11.472932 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_420_198/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:17:11.727969 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:11.772682 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_349_72, total 393 rows starting from the beginning of the part 2024.10.22 01:17:12.086149 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_350_350_0, total 1 rows starting from the beginning of the part 2024.10.22 01:17:12.184049 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_631_254, 202410_632_632_0] 2024.10.22 01:17:12.191591 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_631_254, 202410_632_632_0] 2024.10.22 01:17:12.209639 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_631_254, 202410_632_632_0] 2024.10.22 01:17:12.455053 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 6507 2024.10.22 01:17:12.492588 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_351_351_0, total 1 rows starting from the beginning of the part 2024.10.22 01:17:12.547207 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_420_198} MergeTask::MergeProjectionsStage: Merge sorted 43417 rows, containing 13 columns (13 merged, 0 gathered) in 3.29829581 sec., 13163.464558989936 rows/sec., 5.70 MiB/sec. 2024.10.22 01:17:12.564497 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(af842f4b5210306f10e72a422d665c58) : [0, 163]: Setting download as failed: File segment: [0, 163], key: af842f4b5210306f10e72a422d665c58, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:12.566795 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(af842f4b5210306f10e72a422d665c58): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 01:17:12.587362 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_420_198} MergedBlockOutputStream: filled checksums 202410_1_420_198 (state Temporary) 2024.10.22 01:17:12.652493 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.96 GiB, free memory in arenas 13.08 MiB, will set to 1.46 GiB (RSS), difference: -39.97 MiB 2024.10.22 01:17:12.679344 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_352_352_0, total 1 rows starting from the beginning of the part 2024.10.22 01:17:12.904744 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:12.914784 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:12.915114 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_353_353_0, total 1 rows starting from the beginning of the part 2024.10.22 01:17:12.972202 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_426_426_0 (state Temporary) 2024.10.22 01:17:13.039619 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_354_354_0, total 1 rows starting from the beginning of the part 2024.10.22 01:17:13.061094 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_426_426_0 to 202410_439_439_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:13.213339 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6507 2024.10.22 01:17:13.520195 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_86_86_0 to 202410_89_89_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:13.533257 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_420_198} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_420_198 to 202410_1_420_198 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:13.536902 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_420_198} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_419_197, 202410_420_420_0] -> 202410_1_420_198 2024.10.22 01:17:13.564245 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8a0f106345c107070dc5e0949f7c1898) : [0, 75518]: Setting download as failed: File segment: [0, 75518], key: 8a0f106345c107070dc5e0949f7c1898, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:13.566986 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.01 MiB. 2024.10.22 01:17:13.567400 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8a0f106345c107070dc5e0949f7c1898): No space left in cache to reserve 75519 bytes, will continue without cache download 2024.10.22 01:17:13.567738 [ 1195 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_84_17 to 202410_89_89_0 2024.10.22 01:17:13.583388 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8a0f106345c107070dc5e0949f7c1898, offset: 0, size: 0 2024.10.22 01:17:13.585274 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8a0f106345c107070dc5e0949f7c1898 for removal 2024.10.22 01:17:13.587378 [ 1195 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.78 MiB using storage policy from min volume index 0 2024.10.22 01:17:13.587588 [ 1476 ] {} CacheMetadata: Key 8a0f106345c107070dc5e0949f7c1898 is removed from metadata 2024.10.22 01:17:13.589279 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: af842f4b5210306f10e72a422d665c58, offset: 0, size: 0 2024.10.22 01:17:13.591287 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key af842f4b5210306f10e72a422d665c58 for removal 2024.10.22 01:17:13.591980 [ 1195 ] {} DiskLocal: Reserved 1.78 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:13.647357 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_84_17 to 202410_89_89_0 into Compact with storage Full 2024.10.22 01:17:13.674913 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_413_83/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:17:13.722241 [ 1476 ] {} CacheMetadata: Key af842f4b5210306f10e72a422d665c58 is removed from metadata 2024.10.22 01:17:14.296082 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1109 entries to flush up to offset 648717 2024.10.22 01:17:14.425005 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 45626 2024.10.22 01:17:14.671770 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3bbc3bd396dbae33c48f62288b521554) : [0, 198]: Setting download as failed: File segment: [0, 198], key: 3bbc3bd396dbae33c48f62288b521554, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:14.672849 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3bbc3bd396dbae33c48f62288b521554): No space left in cache to reserve 199 bytes, will continue without cache download 2024.10.22 01:17:14.824448 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:14.827992 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:15.194328 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_724_724_0 (state Temporary) 2024.10.22 01:17:15.368547 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_724_724_0 to 202410_737_737_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:15.380960 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_301_62, 202410_302_302_0, 202410_303_303_0, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0] 2024.10.22 01:17:15.383338 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_301_62, 202410_302_302_0, 202410_303_303_0, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0] 2024.10.22 01:17:15.405507 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_301_62, 202410_302_302_0, 202410_303_303_0, 202410_304_304_0, 202410_305_305_0, 202410_306_306_0] 2024.10.22 01:17:15.414132 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45626 2024.10.22 01:17:15.477606 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:15.515726 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:15.420479 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} MergeTask::MergeProjectionsStage: Merge sorted 398 rows, containing 32 columns (32 merged, 0 gathered) in 4.481651733 sec., 88.80654359404683 rows/sec., 8.36 KiB/sec. 2024.10.22 01:17:15.594790 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} MergedBlockOutputStream: filled checksums 202410_1_354_73 (state Temporary) 2024.10.22 01:17:15.647858 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_632_255, 202410_633_633_0] 2024.10.22 01:17:15.656174 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_632_255, 202410_633_633_0] 2024.10.22 01:17:15.668192 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_632_255, 202410_633_633_0] 2024.10.22 01:17:15.753159 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_409_409_0 (state Temporary) 2024.10.22 01:17:15.757004 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 11.71 MiB, will set to 1.45 GiB (RSS), difference: 22.75 MiB 2024.10.22 01:17:16.051588 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_409_409_0 to 202410_416_416_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:16.086139 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 648717 2024.10.22 01:17:16.167958 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4a2e22df56bf861e1499faef975aa72e) : [0, 259365]: Setting download as failed: File segment: [0, 259365], key: 4a2e22df56bf861e1499faef975aa72e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:16.170553 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4a2e22df56bf861e1499faef975aa72e): No space left in cache to reserve 259366 bytes, will continue without cache download 2024.10.22 01:17:16.287835 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4a2e22df56bf861e1499faef975aa72e, offset: 0, size: 0 2024.10.22 01:17:16.289625 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4a2e22df56bf861e1499faef975aa72e for removal 2024.10.22 01:17:16.292110 [ 1476 ] {} CacheMetadata: Key 4a2e22df56bf861e1499faef975aa72e is removed from metadata 2024.10.22 01:17:16.292652 [ 1256 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:17:16.295109 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3bbc3bd396dbae33c48f62288b521554, offset: 0, size: 0 2024.10.22 01:17:16.301568 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3bbc3bd396dbae33c48f62288b521554 for removal 2024.10.22 01:17:16.307838 [ 1476 ] {} CacheMetadata: Key 3bbc3bd396dbae33c48f62288b521554 is removed from metadata 2024.10.22 01:17:16.382777 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3423 2024.10.22 01:17:16.374414 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_351_70, 202410_352_352_0, 202410_353_353_0, 202410_354_354_0, 202410_355_355_0, 202410_356_356_0] 2024.10.22 01:17:16.388402 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_351_70, 202410_352_352_0, 202410_353_353_0, 202410_354_354_0, 202410_355_355_0, 202410_356_356_0] 2024.10.22 01:17:16.426180 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_351_70, 202410_352_352_0, 202410_353_353_0, 202410_354_354_0, 202410_355_355_0, 202410_356_356_0] 2024.10.22 01:17:16.523256 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_354_73 to 202410_1_354_73 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:16.529750 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_354_73} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_349_72, 202410_354_354_0] -> 202410_1_354_73 2024.10.22 01:17:16.629689 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:17:16.672223 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 175 entries to flush up to offset 44485 2024.10.22 01:17:16.895604 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_362_140, 202410_363_363_0] 2024.10.22 01:17:16.897493 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_362_140, 202410_363_363_0] 2024.10.22 01:17:16.962937 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:16.975244 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:17:17.013103 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_420_420_0 (state Temporary) 2024.10.22 01:17:16.917744 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_362_140, 202410_363_363_0] 2024.10.22 01:17:17.078769 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_420_420_0 to 202410_426_426_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:17.104496 [ 1243 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_420_198 to 202410_421_421_0 2024.10.22 01:17:17.130863 [ 1243 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.23 MiB using storage policy from min volume index 0 2024.10.22 01:17:17.148859 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 44485 2024.10.22 01:17:17.137815 [ 1243 ] {} DiskLocal: Reserved 2.23 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:17:17.292455 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(8ca9e8aa6434c9715479982d294f46f8) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 8ca9e8aa6434c9715479982d294f46f8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:17.294118 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(8ca9e8aa6434c9715479982d294f46f8): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:17:17.307536 [ 1243 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 01:17:17.347501 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_421_199} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_420_198 to 202410_421_421_0 into Wide with storage Full 2024.10.22 01:17:17.404956 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_421_199} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:17.412917 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_421_199} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_420_198, total 43417 rows starting from the beginning of the part 2024.10.22 01:17:17.413047 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 17.28 MiB, will set to 1.45 GiB (RSS), difference: 13.54 MiB 2024.10.22 01:17:17.498759 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_421_199} MergeTreeSequentialSource: Reading 2 marks from part 202410_421_421_0, total 225 rows starting from the beginning of the part 2024.10.22 01:17:18.129804 [ 1253 ] {} DatabaseCatalog: Execution took 236 ms. 2024.10.22 01:17:18.166016 [ 1265 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:17:18.199475 [ 1265 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:17:18.404718 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3ae208e6434666f440b441a9ede12140) : [0, 5679]: Setting download as failed: File segment: [0, 5679], key: 3ae208e6434666f440b441a9ede12140, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:18.412592 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3ae208e6434666f440b441a9ede12140): No space left in cache to reserve 5680 bytes, will continue without cache download 2024.10.22 01:17:18.568743 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3ae208e6434666f440b441a9ede12140, offset: 0, size: 0 2024.10.22 01:17:18.583150 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3ae208e6434666f440b441a9ede12140 for removal 2024.10.22 01:17:18.587183 [ 1476 ] {} CacheMetadata: Key 3ae208e6434666f440b441a9ede12140 is removed from metadata 2024.10.22 01:17:18.587769 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8ca9e8aa6434c9715479982d294f46f8, offset: 0, size: 0 2024.10.22 01:17:18.590919 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8ca9e8aa6434c9715479982d294f46f8 for removal 2024.10.22 01:17:18.593822 [ 1476 ] {} CacheMetadata: Key 8ca9e8aa6434c9715479982d294f46f8 is removed from metadata 2024.10.22 01:17:19.116280 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:17:19.533770 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 45697 2024.10.22 01:17:19.481793 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 14.05 MiB, will set to 1.47 GiB (RSS), difference: -40.00 MiB 2024.10.22 01:17:20.019809 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:20.025502 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:17:20.059098 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6baf1fb00e3ac8bfe12f8c0418943104) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 6baf1fb00e3ac8bfe12f8c0418943104, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:20.060075 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6baf1fb00e3ac8bfe12f8c0418943104): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 01:17:20.066624 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:17:20.069326 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:17:20.072306 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_725_725_0 (state Temporary) 2024.10.22 01:17:20.331810 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_725_725_0 to 202410_738_738_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:20.387780 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45697 2024.10.22 01:17:20.716716 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 6523 2024.10.22 01:17:20.794425 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3d6fe7332e8f12f93c3125cb29d178c5) : [0, 7358]: Setting download as failed: File segment: [0, 7358], key: 3d6fe7332e8f12f93c3125cb29d178c5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:20.798990 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3d6fe7332e8f12f93c3125cb29d178c5): No space left in cache to reserve 7359 bytes, will continue without cache download 2024.10.22 01:17:20.868263 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_633_256, 202410_634_634_0] 2024.10.22 01:17:20.870758 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3d6fe7332e8f12f93c3125cb29d178c5, offset: 0, size: 0 2024.10.22 01:17:20.871846 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_633_256, 202410_634_634_0] 2024.10.22 01:17:20.873186 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3d6fe7332e8f12f93c3125cb29d178c5 for removal 2024.10.22 01:17:20.879469 [ 1476 ] {} CacheMetadata: Key 3d6fe7332e8f12f93c3125cb29d178c5 is removed from metadata 2024.10.22 01:17:20.882568 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6baf1fb00e3ac8bfe12f8c0418943104, offset: 0, size: 0 2024.10.22 01:17:20.885713 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6baf1fb00e3ac8bfe12f8c0418943104 for removal 2024.10.22 01:17:20.886962 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_633_256, 202410_634_634_0] 2024.10.22 01:17:20.889413 [ 1476 ] {} CacheMetadata: Key 6baf1fb00e3ac8bfe12f8c0418943104 is removed from metadata 2024.10.22 01:17:20.935519 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:20.939985 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_421_199} MergeTask::MergeProjectionsStage: Merge sorted 43642 rows, containing 13 columns (13 merged, 0 gathered) in 3.629078532 sec., 12025.642216110633 rows/sec., 5.21 MiB/sec. 2024.10.22 01:17:20.947280 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:17:20.990222 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_427_427_0 (state Temporary) 2024.10.22 01:17:21.037369 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_427_427_0 to 202410_440_440_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:21.144951 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_421_199} MergedBlockOutputStream: filled checksums 202410_1_421_199 (state Temporary) 2024.10.22 01:17:21.194148 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6523 2024.10.22 01:17:21.499287 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 15.85 MiB, will set to 1.46 GiB (RSS), difference: 26.62 MiB 2024.10.22 01:17:21.727961 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d8615b41510e79c93d408814d0c31ff9) : [0, 161]: Setting download as failed: File segment: [0, 161], key: d8615b41510e79c93d408814d0c31ff9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:21.730469 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d8615b41510e79c93d408814d0c31ff9): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:17:22.062970 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_421_199} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_421_199 to 202410_1_421_199 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:22.067862 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_421_199} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_420_198, 202410_421_421_0] -> 202410_1_421_199 2024.10.22 01:17:22.080649 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.08 MiB. 2024.10.22 01:17:22.478715 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fad7157f8f92713b0a7b6a1daf436937) : [0, 69258]: Setting download as failed: File segment: [0, 69258], key: fad7157f8f92713b0a7b6a1daf436937, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:22.479984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fad7157f8f92713b0a7b6a1daf436937): No space left in cache to reserve 69259 bytes, will continue without cache download 2024.10.22 01:17:22.607405 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_413_83} MergeTask::MergeProjectionsStage: Merge sorted 659278 rows, containing 4 columns (4 merged, 0 gathered) in 23.523153489 sec., 28026.769468145267 rows/sec., 410.77 KiB/sec. 2024.10.22 01:17:22.710806 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fad7157f8f92713b0a7b6a1daf436937, offset: 0, size: 0 2024.10.22 01:17:22.716469 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fad7157f8f92713b0a7b6a1daf436937 for removal 2024.10.22 01:17:22.821611 [ 1476 ] {} CacheMetadata: Key fad7157f8f92713b0a7b6a1daf436937 is removed from metadata 2024.10.22 01:17:22.822347 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d8615b41510e79c93d408814d0c31ff9, offset: 0, size: 0 2024.10.22 01:17:22.824561 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d8615b41510e79c93d408814d0c31ff9 for removal 2024.10.22 01:17:22.866237 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_413_83} MergedBlockOutputStream: filled checksums 202410_1_413_83 (state Temporary) 2024.10.22 01:17:22.928625 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_413_83} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_413_83 to 202410_1_413_83 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:22.930919 [ 1476 ] {} CacheMetadata: Key d8615b41510e79c93d408814d0c31ff9 is removed from metadata 2024.10.22 01:17:22.931467 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_413_83} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 5 parts: [202410_1_409_82, 202410_413_413_0] -> 202410_1_413_83 2024.10.22 01:17:23.071249 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:17:23.092568 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1586 entries to flush up to offset 650303 2024.10.22 01:17:23.265233 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 13.64 MiB, will set to 1.45 GiB (RSS), difference: 17.22 MiB 2024.10.22 01:17:23.444006 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(780c5a48f5900df0127c219bb9942e95) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 780c5a48f5900df0127c219bb9942e95, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:17:23.447744 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 780c5a48f5900df0127c219bb9942e95, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:17:23.451087 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:23.920630 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 3462 2024.10.22 01:17:24.277583 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:24.279486 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:24.439135 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 45754 2024.10.22 01:17:24.503939 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_410_410_0 (state Temporary) 2024.10.22 01:17:24.597803 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_410_410_0 to 202410_417_417_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:24.616654 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(78ceae09527780eaedd78b68173ea5be) : [0, 163]: Setting download as failed: File segment: [0, 163], key: 78ceae09527780eaedd78b68173ea5be, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:24.620360 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(78ceae09527780eaedd78b68173ea5be): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 01:17:24.631905 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 650303 2024.10.22 01:17:24.772328 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 149 entries to flush up to offset 44634 2024.10.22 01:17:24.925514 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:24.931236 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:24.931160 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:24.939411 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:24.979806 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_421_421_0 (state Temporary) 2024.10.22 01:17:25.111672 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_726_726_0 (state Temporary) 2024.10.22 01:17:25.125663 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_421_421_0 to 202410_427_427_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:25.270989 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_726_726_0 to 202410_739_739_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:25.295366 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_728_341 to 202410_729_734_1 2024.10.22 01:17:25.303228 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.65 MiB using storage policy from min volume index 0 2024.10.22 01:17:25.311090 [ 1201 ] {} DiskLocal: Reserved 1.65 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:25.330327 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(823203e83c04716e073ec51665212489) : [0, 72562]: Setting download as failed: File segment: [0, 72562], key: 823203e83c04716e073ec51665212489, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:25.332683 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(823203e83c04716e073ec51665212489): No space left in cache to reserve 72563 bytes, will continue without cache download 2024.10.22 01:17:25.345802 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45754 2024.10.22 01:17:25.358236 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 823203e83c04716e073ec51665212489, offset: 0, size: 0 2024.10.22 01:17:25.359828 [ 1215 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_421_199 to 202410_422_422_0 2024.10.22 01:17:25.363201 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 823203e83c04716e073ec51665212489 for removal 2024.10.22 01:17:25.365669 [ 1476 ] {} CacheMetadata: Key 823203e83c04716e073ec51665212489 is removed from metadata 2024.10.22 01:17:25.367188 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 78ceae09527780eaedd78b68173ea5be, offset: 0, size: 0 2024.10.22 01:17:25.367724 [ 1215 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.24 MiB using storage policy from min volume index 0 2024.10.22 01:17:25.368774 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 78ceae09527780eaedd78b68173ea5be for removal 2024.10.22 01:17:25.371100 [ 1215 ] {} DiskLocal: Reserved 2.24 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:25.372180 [ 1476 ] {} CacheMetadata: Key 78ceae09527780eaedd78b68173ea5be is removed from metadata 2024.10.22 01:17:25.464337 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_422_200} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_421_199 to 202410_422_422_0 into Wide with storage Full 2024.10.22 01:17:25.470940 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 44634 2024.10.22 01:17:25.485057 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_363_141, 202410_364_364_0] 2024.10.22 01:17:25.487663 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_363_141, 202410_364_364_0] 2024.10.22 01:17:25.520940 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_363_141, 202410_364_364_0] 2024.10.22 01:17:25.526961 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_422_200} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:25.564323 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_734_342} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_728_341 to 202410_729_734_1 into Compact with storage Full 2024.10.22 01:17:25.620448 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_734_342} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:25.647438 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_734_342} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_728_341, total 45732 rows starting from the beginning of the part 2024.10.22 01:17:25.659777 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_422_200} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_421_199, total 43642 rows starting from the beginning of the part 2024.10.22 01:17:25.795012 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_734_342} MergeTreeSequentialSource: Reading 2 marks from part 202410_729_734_1, total 486 rows starting from the beginning of the part 2024.10.22 01:17:25.795540 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_376_75, 202410_377_377_0, 202410_378_378_0, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0] 2024.10.22 01:17:25.799669 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_376_75, 202410_377_377_0, 202410_378_378_0, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0] 2024.10.22 01:17:25.852987 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_376_75, 202410_377_377_0, 202410_378_378_0, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0] 2024.10.22 01:17:26.172864 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 363 2024.10.22 01:17:26.185108 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_422_200} MergeTreeSequentialSource: Reading 2 marks from part 202410_422_422_0, total 212 rows starting from the beginning of the part 2024.10.22 01:17:26.295493 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(324110be60ed40a546cd15ab503e6542) : [0, 240]: Setting download as failed: File segment: [0, 240], key: 324110be60ed40a546cd15ab503e6542, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:26.300973 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(324110be60ed40a546cd15ab503e6542): No space left in cache to reserve 241 bytes, will continue without cache download 2024.10.22 01:17:26.465799 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:26.468702 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:26.594711 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 8.69 MiB, will set to 1.45 GiB (RSS), difference: -35.32 MiB 2024.10.22 01:17:26.891390 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_69_14, 202410_70_70_0, 202410_71_71_0, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0] 2024.10.22 01:17:26.892789 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_69_14, 202410_70_70_0, 202410_71_71_0, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0] 2024.10.22 01:17:26.921024 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_69_14, 202410_70_70_0, 202410_71_71_0, 202410_72_72_0, 202410_73_73_0, 202410_74_74_0] 2024.10.22 01:17:26.927981 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_634_257, 202410_635_635_0] 2024.10.22 01:17:26.932922 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_634_257, 202410_635_635_0] 2024.10.22 01:17:26.949331 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 204 ms. 2024.10.22 01:17:26.957608 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_634_257, 202410_635_635_0] 2024.10.22 01:17:26.970747 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_345_345_0 (state Temporary) 2024.10.22 01:17:27.160340 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_345_345_0 to 202410_355_355_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:27.261108 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(73f40248388e6754a9f42887d61e9d87) : [0, 424785]: Setting download as failed: File segment: [0, 424785], key: 73f40248388e6754a9f42887d61e9d87, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:27.269311 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(73f40248388e6754a9f42887d61e9d87): No space left in cache to reserve 424786 bytes, will continue without cache download 2024.10.22 01:17:27.422196 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 363 2024.10.22 01:17:27.447941 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 73f40248388e6754a9f42887d61e9d87, offset: 0, size: 0 2024.10.22 01:17:27.467866 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 73f40248388e6754a9f42887d61e9d87 for removal 2024.10.22 01:17:27.473661 [ 1476 ] {} CacheMetadata: Key 73f40248388e6754a9f42887d61e9d87 is removed from metadata 2024.10.22 01:17:27.476272 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 324110be60ed40a546cd15ab503e6542, offset: 0, size: 0 2024.10.22 01:17:27.479708 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 324110be60ed40a546cd15ab503e6542 for removal 2024.10.22 01:17:27.487626 [ 1476 ] {} CacheMetadata: Key 324110be60ed40a546cd15ab503e6542 is removed from metadata 2024.10.22 01:17:27.733821 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_422_200/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:17:28.181621 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:28.277094 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_734_342} MergeTask::MergeProjectionsStage: Merge sorted 46218 rows, containing 13 columns (13 merged, 0 gathered) in 2.835379445 sec., 16300.463799123012 rows/sec., 2.60 MiB/sec. 2024.10.22 01:17:28.352527 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(cc036cf12ee7050eed9deacbab9ccef8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cc036cf12ee7050eed9deacbab9ccef8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:17:28.361726 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1002, file segment info: File segment: [0, 33554431], key: cc036cf12ee7050eed9deacbab9ccef8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:17:28.365623 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:28.423077 [ 1703 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/scp/cfdcvlkarmybtghpvascanojugkxl, size 1483775 2024.10.22 01:17:28.571428 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_734_342} MergedBlockOutputStream: filled checksums 202410_1_734_342 (state Temporary) 2024.10.22 01:17:28.648729 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IsLink 2024.10.22 01:17:28.664001 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IsLink 2024.10.22 01:17:28.675104 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IsLink 2024.10.22 01:17:28.696407 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 6537 2024.10.22 01:17:28.705818 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IsLink 2024.10.22 01:17:28.737704 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IsLink 2024.10.22 01:17:28.785712 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IsLink 2024.10.22 01:17:28.790904 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_734_342} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_734_342 to 202410_1_734_342 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:28.799056 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_734_342} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_728_341, 202410_729_734_1] -> 202410_1_734_342 2024.10.22 01:17:28.827993 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IsLink 2024.10.22 01:17:28.844123 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.78 MiB. 2024.10.22 01:17:28.955541 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IsLink 2024.10.22 01:17:29.074302 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IsLink 2024.10.22 01:17:29.123469 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6f9cfb5799b900563c5f480d1b8c2d7e) : [0, 171]: Setting download as failed: File segment: [0, 171], key: 6f9cfb5799b900563c5f480d1b8c2d7e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:29.124988 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6f9cfb5799b900563c5f480d1b8c2d7e): No space left in cache to reserve 172 bytes, will continue without cache download 2024.10.22 01:17:29.251159 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tej/btsmkwpzcgfqzvdwcgqcegblabsbb (LogSeriesLimiter: on interval from 2024-10-22 01:17:08 to 2024-10-22 01:17:29 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:17:29.276459 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:29.279423 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:17:29.374102 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 95 entries to flush up to offset 45849 2024.10.22 01:17:29.429620 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_428_428_0 (state Temporary) 2024.10.22 01:17:29.481133 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_428_428_0 to 202410_441_441_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:29.530815 [ 1245 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_436_87 to 202410_441_441_0 2024.10.22 01:17:29.552332 [ 1245 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:29.558190 [ 1245 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:17:29.666831 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6537 2024.10.22 01:17:29.672136 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.96 GiB, free memory in arenas 3.82 MiB, will set to 1.46 GiB (RSS), difference: -1.86 MiB 2024.10.22 01:17:29.681268 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_436_87 to 202410_441_441_0 into Compact with storage Full 2024.10.22 01:17:29.717143 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:29.724106 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:29.726563 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_84_17, total 3293 rows starting from the beginning of the part 2024.10.22 01:17:29.786242 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_727_727_0 (state Temporary) 2024.10.22 01:17:29.869506 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_727_727_0 to 202410_740_740_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:29.875227 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:29.884938 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_436_87, total 6559 rows starting from the beginning of the part 2024.10.22 01:17:29.906237 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_734_342 to 202410_735_735_0 2024.10.22 01:17:29.918718 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.65 MiB using storage policy from min volume index 0 2024.10.22 01:17:29.932749 [ 1278 ] {} DiskLocal: Reserved 1.65 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:30.035068 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_735_343} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_734_342 to 202410_735_735_0 into Compact with storage Full 2024.10.22 01:17:30.058531 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45849 2024.10.22 01:17:30.067955 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:17:30.071624 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:17:30.075122 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_437_437_0, total 22 rows starting from the beginning of the part 2024.10.22 01:17:30.131991 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_735_343} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:30.154659 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_735_343} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_734_342, total 46218 rows starting from the beginning of the part 2024.10.22 01:17:30.156886 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_438_438_0, total 23 rows starting from the beginning of the part 2024.10.22 01:17:30.221356 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_735_343} MergeTreeSequentialSource: Reading 2 marks from part 202410_735_735_0, total 88 rows starting from the beginning of the part 2024.10.22 01:17:30.222444 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_439_439_0, total 9 rows starting from the beginning of the part 2024.10.22 01:17:30.381696 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(471ff1e333cbc170fbdcae8b44831638) : [0, 287948]: Setting download as failed: File segment: [0, 287948], key: 471ff1e333cbc170fbdcae8b44831638, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:30.382866 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(471ff1e333cbc170fbdcae8b44831638): No space left in cache to reserve 287949 bytes, will continue without cache download 2024.10.22 01:17:30.359047 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_422_200} MergeTask::MergeProjectionsStage: Merge sorted 43854 rows, containing 13 columns (13 merged, 0 gathered) in 4.89866192 sec., 8952.240574299522 rows/sec., 3.88 MiB/sec. 2024.10.22 01:17:30.541777 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_440_440_0, total 16 rows starting from the beginning of the part 2024.10.22 01:17:30.542758 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 471ff1e333cbc170fbdcae8b44831638, offset: 0, size: 0 2024.10.22 01:17:30.571101 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_422_200} MergedBlockOutputStream: filled checksums 202410_1_422_200 (state Temporary) 2024.10.22 01:17:30.614118 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_441_441_0, total 14 rows starting from the beginning of the part 2024.10.22 01:17:30.652207 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 471ff1e333cbc170fbdcae8b44831638 for removal 2024.10.22 01:17:30.660626 [ 1476 ] {} CacheMetadata: Key 471ff1e333cbc170fbdcae8b44831638 is removed from metadata 2024.10.22 01:17:30.665447 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6f9cfb5799b900563c5f480d1b8c2d7e, offset: 0, size: 0 2024.10.22 01:17:30.670232 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6f9cfb5799b900563c5f480d1b8c2d7e for removal 2024.10.22 01:17:30.675474 [ 1476 ] {} CacheMetadata: Key 6f9cfb5799b900563c5f480d1b8c2d7e is removed from metadata 2024.10.22 01:17:30.774892 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_635_258, 202410_636_636_0] 2024.10.22 01:17:30.777192 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_635_258, 202410_636_636_0] 2024.10.22 01:17:30.784644 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_635_258, 202410_636_636_0] 2024.10.22 01:17:31.327019 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_422_200} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_422_200 to 202410_1_422_200 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:31.332037 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_422_200} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_421_199, 202410_422_422_0] -> 202410_1_422_200 2024.10.22 01:17:31.574339 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.08 MiB. 2024.10.22 01:17:31.639100 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 651458 2024.10.22 01:17:31.728531 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(99024b85f0c1b5d8150542f867b79da1) : [0, 196]: Setting download as failed: File segment: [0, 196], key: 99024b85f0c1b5d8150542f867b79da1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:31.730698 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(99024b85f0c1b5d8150542f867b79da1): No space left in cache to reserve 197 bytes, will continue without cache download 2024.10.22 01:17:31.817752 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_735_343/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:17:32.083517 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} MergeTask::MergeProjectionsStage: Merge sorted 6643 rows, containing 12 columns (12 merged, 0 gathered) in 2.414793523 sec., 2750.9598384822234 rows/sec., 435.98 KiB/sec. 2024.10.22 01:17:32.271099 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} MergedBlockOutputStream: filled checksums 202410_1_441_88 (state Temporary) 2024.10.22 01:17:32.418966 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_441_88 to 202410_1_441_88 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:32.427773 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_441_88} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_436_87, 202410_441_441_0] -> 202410_1_441_88 2024.10.22 01:17:32.471085 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.44 MiB. 2024.10.22 01:17:32.499742 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_85_85_0, total 36 rows starting from the beginning of the part 2024.10.22 01:17:32.509367 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.96 GiB, free memory in arenas 3.15 MiB, will set to 1.45 GiB (RSS), difference: 27.07 MiB 2024.10.22 01:17:32.514148 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:32.518204 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:32.790685 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_411_411_0 (state Temporary) 2024.10.22 01:17:32.845475 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(31defbf7aba028ffac5a9b900cc7c4ed) : [0, 144603]: Setting download as failed: File segment: [0, 144603], key: 31defbf7aba028ffac5a9b900cc7c4ed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:32.850031 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(31defbf7aba028ffac5a9b900cc7c4ed): No space left in cache to reserve 144604 bytes, will continue without cache download 2024.10.22 01:17:32.972028 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 31defbf7aba028ffac5a9b900cc7c4ed, offset: 0, size: 0 2024.10.22 01:17:32.985152 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 31defbf7aba028ffac5a9b900cc7c4ed for removal 2024.10.22 01:17:32.987540 [ 1476 ] {} CacheMetadata: Key 31defbf7aba028ffac5a9b900cc7c4ed is removed from metadata 2024.10.22 01:17:32.989351 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 99024b85f0c1b5d8150542f867b79da1, offset: 0, size: 0 2024.10.22 01:17:32.990559 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 99024b85f0c1b5d8150542f867b79da1 for removal 2024.10.22 01:17:32.992759 [ 1476 ] {} CacheMetadata: Key 99024b85f0c1b5d8150542f867b79da1 is removed from metadata 2024.10.22 01:17:32.994172 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_735_343} MergeTask::MergeProjectionsStage: Merge sorted 46306 rows, containing 13 columns (13 merged, 0 gathered) in 3.048312035 sec., 15190.702089656645 rows/sec., 2.42 MiB/sec. 2024.10.22 01:17:32.994362 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 236 entries to flush up to offset 44870 2024.10.22 01:17:33.070293 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_411_411_0 to 202410_418_418_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:33.101056 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_735_343} MergedBlockOutputStream: filled checksums 202410_1_735_343 (state Temporary) 2024.10.22 01:17:33.110405 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 651458 2024.10.22 01:17:33.110382 [ 1244 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 5 parts from 202410_1_413_83 to 202410_417_417_0 2024.10.22 01:17:33.126938 [ 1244 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:33.130582 [ 1244 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:33.156361 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_417_84} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_413_83 to 202410_417_417_0 into Compact with storage Full 2024.10.22 01:17:33.184811 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_417_84} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:33.193347 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_417_84} MergeTreeSequentialSource: Reading 82 marks from part 202410_1_413_83, total 659278 rows starting from the beginning of the part 2024.10.22 01:17:33.205207 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_417_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_414_414_0, total 1152 rows starting from the beginning of the part 2024.10.22 01:17:33.205719 [ 1224 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:17:33.237345 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_417_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_415_415_0, total 768 rows starting from the beginning of the part 2024.10.22 01:17:33.258707 [ 1224 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:17:33.274216 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:33.277755 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:33.287796 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_417_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_416_416_0, total 1109 rows starting from the beginning of the part 2024.10.22 01:17:33.321737 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_735_343} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_735_343 to 202410_1_735_343 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:33.325267 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_735_343} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_734_342, 202410_735_735_0] -> 202410_1_735_343 2024.10.22 01:17:33.375241 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_364_142, 202410_365_365_0] 2024.10.22 01:17:33.376889 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_364_142, 202410_365_365_0] 2024.10.22 01:17:33.391613 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_364_142, 202410_365_365_0] 2024.10.22 01:17:33.407870 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_417_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_417_417_0, total 1586 rows starting from the beginning of the part 2024.10.22 01:17:33.436294 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.69 MiB. 2024.10.22 01:17:33.548072 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_422_422_0 (state Temporary) 2024.10.22 01:17:33.598459 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_422_422_0 to 202410_428_428_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:33.627743 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 44870 2024.10.22 01:17:33.634770 [ 1240 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_422_200 to 202410_423_423_0 2024.10.22 01:17:33.649098 [ 1240 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.25 MiB using storage policy from min volume index 0 2024.10.22 01:17:33.656444 [ 1240 ] {} DiskLocal: Reserved 2.25 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:33.665242 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_423_201} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_422_200 to 202410_423_423_0 into Wide with storage Full 2024.10.22 01:17:33.696667 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_423_201} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:33.703523 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_423_201} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_422_200, total 43854 rows starting from the beginning of the part 2024.10.22 01:17:33.819469 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_423_201} MergeTreeSequentialSource: Reading 2 marks from part 202410_423_423_0, total 212 rows starting from the beginning of the part 2024.10.22 01:17:33.877299 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a4b33e0097b57ce732ce7e6a1b88dcec) : [0, 170]: Setting download as failed: File segment: [0, 170], key: a4b33e0097b57ce732ce7e6a1b88dcec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:33.880772 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a4b33e0097b57ce732ce7e6a1b88dcec): No space left in cache to reserve 171 bytes, will continue without cache download 2024.10.22 01:17:34.092766 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 106 entries to flush up to offset 45955 2024.10.22 01:17:34.467492 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.96 GiB, free memory in arenas 5.53 MiB, will set to 1.44 GiB (RSS), difference: -56.73 MiB 2024.10.22 01:17:34.584069 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:34.587188 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:34.611670 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_86_86_0, total 38 rows starting from the beginning of the part 2024.10.22 01:17:34.912823 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_728_728_0 (state Temporary) 2024.10.22 01:17:34.928464 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 364 2024.10.22 01:17:35.070143 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c52137046b23abf6c53907442dc68a61) : [0, 205425]: Setting download as failed: File segment: [0, 205425], key: c52137046b23abf6c53907442dc68a61, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:35.072469 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c52137046b23abf6c53907442dc68a61): No space left in cache to reserve 205426 bytes, will continue without cache download 2024.10.22 01:17:35.133513 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c52137046b23abf6c53907442dc68a61, offset: 0, size: 0 2024.10.22 01:17:35.138148 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c52137046b23abf6c53907442dc68a61 for removal 2024.10.22 01:17:35.144193 [ 1476 ] {} CacheMetadata: Key c52137046b23abf6c53907442dc68a61 is removed from metadata 2024.10.22 01:17:35.150469 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a4b33e0097b57ce732ce7e6a1b88dcec, offset: 0, size: 0 2024.10.22 01:17:35.154933 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a4b33e0097b57ce732ce7e6a1b88dcec for removal 2024.10.22 01:17:35.163379 [ 1476 ] {} CacheMetadata: Key a4b33e0097b57ce732ce7e6a1b88dcec is removed from metadata 2024.10.22 01:17:35.167483 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:35.170417 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:35.325107 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_636_259, 202410_637_637_0] 2024.10.22 01:17:35.325917 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_728_728_0 to 202410_741_741_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:35.342879 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_346_346_0 (state Temporary) 2024.10.22 01:17:35.347653 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_636_259, 202410_637_637_0] 2024.10.22 01:17:35.371365 [ 1190 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_735_343 to 202410_736_736_0 2024.10.22 01:17:35.386276 [ 1190 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.66 MiB using storage policy from min volume index 0 2024.10.22 01:17:35.389480 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 45955 2024.10.22 01:17:35.393891 [ 1190 ] {} DiskLocal: Reserved 1.66 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:35.451376 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_736_344} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_735_343 to 202410_736_736_0 into Compact with storage Full 2024.10.22 01:17:35.472240 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_636_259, 202410_637_637_0] 2024.10.22 01:17:35.518145 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_736_344} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:35.529608 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_736_344} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_735_343, total 46306 rows starting from the beginning of the part 2024.10.22 01:17:35.582630 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_736_344} MergeTreeSequentialSource: Reading 2 marks from part 202410_736_736_0, total 75 rows starting from the beginning of the part 2024.10.22 01:17:35.651755 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_346_346_0 to 202410_356_356_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:35.739829 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 364 2024.10.22 01:17:35.893650 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_423_201} MergeTask::MergeProjectionsStage: Merge sorted 44066 rows, containing 13 columns (13 merged, 0 gathered) in 2.231431531 sec., 19747.861132110174 rows/sec., 8.55 MiB/sec. 2024.10.22 01:17:35.981999 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_417_84/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:17:36.015107 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_423_201} MergedBlockOutputStream: filled checksums 202410_1_423_201 (state Temporary) 2024.10.22 01:17:36.114424 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_87_87_0, total 40 rows starting from the beginning of the part 2024.10.22 01:17:36.186952 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(07d93c5052c754b8fb9c9c34870a3ade) : [0, 200]: Setting download as failed: File segment: [0, 200], key: 07d93c5052c754b8fb9c9c34870a3ade, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:36.193750 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(07d93c5052c754b8fb9c9c34870a3ade): No space left in cache to reserve 201 bytes, will continue without cache download 2024.10.22 01:17:36.828858 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_423_201} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_423_201 to 202410_1_423_201 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:36.953149 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_423_201} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_422_200, 202410_423_423_0] -> 202410_1_423_201 2024.10.22 01:17:36.982110 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.08 MiB. 2024.10.22 01:17:37.169528 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 6562 2024.10.22 01:17:37.260529 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7a67b64a3193fcfebac2195001664def) : [0, 358603]: Setting download as failed: File segment: [0, 358603], key: 7a67b64a3193fcfebac2195001664def, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:37.264053 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7a67b64a3193fcfebac2195001664def): No space left in cache to reserve 358604 bytes, will continue without cache download 2024.10.22 01:17:37.312330 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7a67b64a3193fcfebac2195001664def, offset: 0, size: 0 2024.10.22 01:17:37.314265 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7a67b64a3193fcfebac2195001664def for removal 2024.10.22 01:17:37.318340 [ 1476 ] {} CacheMetadata: Key 7a67b64a3193fcfebac2195001664def is removed from metadata 2024.10.22 01:17:37.318958 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 07d93c5052c754b8fb9c9c34870a3ade, offset: 0, size: 0 2024.10.22 01:17:37.322645 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 07d93c5052c754b8fb9c9c34870a3ade for removal 2024.10.22 01:17:37.326240 [ 1476 ] {} CacheMetadata: Key 07d93c5052c754b8fb9c9c34870a3ade is removed from metadata 2024.10.22 01:17:37.423343 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:37.442434 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:37.477226 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.96 GiB, free memory in arenas 8.41 MiB, will set to 1.46 GiB (RSS), difference: 51.68 MiB 2024.10.22 01:17:37.490168 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_429_429_0 (state Temporary) 2024.10.22 01:17:37.543191 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_429_429_0 to 202410_442_442_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:37.683247 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6562 2024.10.22 01:17:37.905699 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(907060928c0191265eafdda044643e68) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 907060928c0191265eafdda044643e68, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:17:37.922583 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 396373, file segment info: File segment: [0, 33554431], key: 907060928c0191265eafdda044643e68, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:17:37.931747 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:38.245339 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_88_88_0, total 41 rows starting from the beginning of the part 2024.10.22 01:17:38.612065 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_736_344} MergeTask::MergeProjectionsStage: Merge sorted 46381 rows, containing 13 columns (13 merged, 0 gathered) in 3.179616635 sec., 14586.978659457165 rows/sec., 2.33 MiB/sec. 2024.10.22 01:17:38.619312 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7044945b075f6a50ab9b8e8e018e1339) : [0, 202]: Setting download as failed: File segment: [0, 202], key: 7044945b075f6a50ab9b8e8e018e1339, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:38.623556 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7044945b075f6a50ab9b8e8e018e1339): No space left in cache to reserve 203 bytes, will continue without cache download 2024.10.22 01:17:38.669773 [ 1170 ] {} DatabaseCatalog: Execution took 422 ms. 2024.10.22 01:17:38.823217 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_736_344} MergedBlockOutputStream: filled checksums 202410_1_736_344 (state Temporary) 2024.10.22 01:17:39.008003 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_736_344} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_736_344 to 202410_1_736_344 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:39.014112 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_736_344} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_735_343, 202410_736_736_0] -> 202410_1_736_344 2024.10.22 01:17:39.051128 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.68 MiB. 2024.10.22 01:17:39.315263 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_637_260, 202410_638_638_0] 2024.10.22 01:17:39.320748 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_637_260, 202410_638_638_0] 2024.10.22 01:17:39.332950 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_637_260, 202410_638_638_0] 2024.10.22 01:17:39.406857 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 46026 2024.10.22 01:17:39.442850 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(ec5a5169b1f0c6d0fcc725360b321559) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ec5a5169b1f0c6d0fcc725360b321559, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:17:39.457751 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 519, file segment info: File segment: [0, 33554431], key: ec5a5169b1f0c6d0fcc725360b321559, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:17:39.459831 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:39.570490 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:39.573422 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:39.744669 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_729_729_0 (state Temporary) 2024.10.22 01:17:39.795246 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_729_729_0 to 202410_742_742_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:39.866322 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_736_344 to 202410_737_737_0 2024.10.22 01:17:39.866332 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46026 2024.10.22 01:17:39.871169 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.66 MiB using storage policy from min volume index 0 2024.10.22 01:17:39.874532 [ 1241 ] {} DiskLocal: Reserved 1.66 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:39.965729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(327bac3eed75c4d2b8b5d6bdf105af25) : [0, 416866]: Setting download as failed: File segment: [0, 416866], key: 327bac3eed75c4d2b8b5d6bdf105af25, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:39.966629 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(327bac3eed75c4d2b8b5d6bdf105af25): No space left in cache to reserve 416867 bytes, will continue without cache download 2024.10.22 01:17:39.988460 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 327bac3eed75c4d2b8b5d6bdf105af25, offset: 0, size: 0 2024.10.22 01:17:39.990033 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 327bac3eed75c4d2b8b5d6bdf105af25 for removal 2024.10.22 01:17:39.992609 [ 1476 ] {} CacheMetadata: Key 327bac3eed75c4d2b8b5d6bdf105af25 is removed from metadata 2024.10.22 01:17:39.994455 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_737_345} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_736_344 to 202410_737_737_0 into Compact with storage Full 2024.10.22 01:17:39.994522 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7044945b075f6a50ab9b8e8e018e1339, offset: 0, size: 0 2024.10.22 01:17:39.996735 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7044945b075f6a50ab9b8e8e018e1339 for removal 2024.10.22 01:17:40.001747 [ 1476 ] {} CacheMetadata: Key 7044945b075f6a50ab9b8e8e018e1339 is removed from metadata 2024.10.22 01:17:40.032424 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_737_345} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:40.040423 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_89_89_0, total 37 rows starting from the beginning of the part 2024.10.22 01:17:40.057165 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_737_345} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_736_344, total 46381 rows starting from the beginning of the part 2024.10.22 01:17:40.068337 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:17:40.072175 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:17:40.104030 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_737_345} MergeTreeSequentialSource: Reading 2 marks from part 202410_737_737_0, total 76 rows starting from the beginning of the part 2024.10.22 01:17:40.114488 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 652613 2024.10.22 01:17:40.560517 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 9.01 MiB, will set to 1.45 GiB (RSS), difference: 4.15 MiB 2024.10.22 01:17:40.576434 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(5ec88d5ec6db37816f526b9c75042b36) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5ec88d5ec6db37816f526b9c75042b36, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:17:40.580211 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 975558, file segment info: File segment: [0, 33554431], key: 5ec88d5ec6db37816f526b9c75042b36, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:17:40.582725 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:40.860096 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:40.864786 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:41.014835 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:41.017648 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:41.182234 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 235 entries to flush up to offset 45105 2024.10.22 01:17:41.270116 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_412_412_0 (state Temporary) 2024.10.22 01:17:41.372956 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(dd0967ccea7fcc28cb9b5b105e0baf29) : [0, 197]: Setting download as failed: File segment: [0, 197], key: dd0967ccea7fcc28cb9b5b105e0baf29, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:41.375480 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(dd0967ccea7fcc28cb9b5b105e0baf29): No space left in cache to reserve 198 bytes, will continue without cache download 2024.10.22 01:17:41.564159 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_412_412_0 to 202410_419_419_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:41.600880 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 652613 2024.10.22 01:17:41.606372 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:41.616082 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:41.794253 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_423_423_0 (state Temporary) 2024.10.22 01:17:42.005662 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_423_423_0 to 202410_429_429_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:42.136660 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_423_201 to 202410_424_424_0 2024.10.22 01:17:42.140361 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(7ef0fd763bc61de9496780c271051e1c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7ef0fd763bc61de9496780c271051e1c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:17:42.149212 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 45105 2024.10.22 01:17:42.150163 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.26 MiB using storage policy from min volume index 0 2024.10.22 01:17:42.151577 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 818, file segment info: File segment: [0, 33554431], key: 7ef0fd763bc61de9496780c271051e1c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:17:42.158798 [ 1176 ] {} DiskLocal: Reserved 2.26 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:42.157330 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:42.177103 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_424_202} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_423_201 to 202410_424_424_0 into Wide with storage Full 2024.10.22 01:17:42.251515 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_424_202} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:42.282918 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_424_202} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_423_201, total 44066 rows starting from the beginning of the part 2024.10.22 01:17:42.594411 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:17:42.629602 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:17:42.662513 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:17:42.763357 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_424_202} MergeTreeSequentialSource: Reading 2 marks from part 202410_424_424_0, total 192 rows starting from the beginning of the part 2024.10.22 01:17:42.785234 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:17:42.844528 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:17:42.885573 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:17:42.967906 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e028894f2dfc74e96d38adf25e8a11aa) : [0, 201842]: Setting download as failed: File segment: [0, 201842], key: e028894f2dfc74e96d38adf25e8a11aa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:42.989711 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e028894f2dfc74e96d38adf25e8a11aa): No space left in cache to reserve 201843 bytes, will continue without cache download 2024.10.22 01:17:43.137202 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:17:43.172060 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_365_143, 202410_366_366_0] 2024.10.22 01:17:43.188575 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_365_143, 202410_366_366_0] 2024.10.22 01:17:43.207845 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xvm/dikafbiabwdvtwvlpcmrngbxbvyqo (LogSeriesLimiter: on interval from 2024-10-22 01:17:29 to 2024-10-22 01:17:43 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:17:43.246409 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 365 2024.10.22 01:17:43.272781 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e028894f2dfc74e96d38adf25e8a11aa, offset: 0, size: 0 2024.10.22 01:17:43.286463 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e028894f2dfc74e96d38adf25e8a11aa for removal 2024.10.22 01:17:43.290000 [ 1476 ] {} CacheMetadata: Key e028894f2dfc74e96d38adf25e8a11aa is removed from metadata 2024.10.22 01:17:43.291968 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: dd0967ccea7fcc28cb9b5b105e0baf29, offset: 0, size: 0 2024.10.22 01:17:43.293751 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key dd0967ccea7fcc28cb9b5b105e0baf29 for removal 2024.10.22 01:17:43.298150 [ 1476 ] {} CacheMetadata: Key dd0967ccea7fcc28cb9b5b105e0baf29 is removed from metadata 2024.10.22 01:17:43.307584 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_365_143, 202410_366_366_0] 2024.10.22 01:17:43.438926 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.96 GiB, free memory in arenas 11.23 MiB, will set to 1.46 GiB (RSS), difference: -33.02 MiB 2024.10.22 01:17:43.448549 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:43.455909 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:43.536291 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_737_345} MergeTask::MergeProjectionsStage: Merge sorted 46457 rows, containing 13 columns (13 merged, 0 gathered) in 3.547397562 sec., 13096.07936185417 rows/sec., 2.09 MiB/sec. 2024.10.22 01:17:43.679605 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_347_347_0 (state Temporary) 2024.10.22 01:17:43.696338 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_87_87_0 (state Temporary) 2024.10.22 01:17:43.758181 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_737_345} MergedBlockOutputStream: filled checksums 202410_1_737_345 (state Temporary) 2024.10.22 01:17:43.860730 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_737_345} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_737_345 to 202410_1_737_345 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:43.864606 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_737_345} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_736_344, 202410_737_737_0] -> 202410_1_737_345 2024.10.22 01:17:43.871486 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 46112 2024.10.22 01:17:43.872951 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_347_347_0 to 202410_357_357_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:43.880849 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.68 MiB. 2024.10.22 01:17:44.143037 [ 1207 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 01:17:44.147961 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6190151ed1dd0c29bdad464af813f89c) : [0, 265]: Setting download as failed: File segment: [0, 265], key: 6190151ed1dd0c29bdad464af813f89c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:44.151476 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6190151ed1dd0c29bdad464af813f89c): No space left in cache to reserve 266 bytes, will continue without cache download 2024.10.22 01:17:44.390547 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 365 2024.10.22 01:17:44.415915 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_638_261, 202410_639_639_0] 2024.10.22 01:17:44.423005 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_638_261, 202410_639_639_0] 2024.10.22 01:17:44.435077 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_638_261, 202410_639_639_0] 2024.10.22 01:17:44.677405 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:44.692301 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:44.845722 [ 1199 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 01:17:44.876654 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_730_730_0 (state Temporary) 2024.10.22 01:17:45.066959 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5981f08e6828c2b151360079ac0c5ae4) : [0, 2249023]: Setting download as failed: File segment: [0, 2249023], key: 5981f08e6828c2b151360079ac0c5ae4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:45.071512 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5981f08e6828c2b151360079ac0c5ae4): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:17:45.142937 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_730_730_0 to 202410_743_743_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:45.170141 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_737_345 to 202410_738_738_0 2024.10.22 01:17:45.176857 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.66 MiB using storage policy from min volume index 0 2024.10.22 01:17:45.178774 [ 1168 ] {} DiskLocal: Reserved 1.66 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:45.189840 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_738_346} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_737_345 to 202410_738_738_0 into Compact with storage Full 2024.10.22 01:17:45.190620 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46112 2024.10.22 01:17:45.197409 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 6574 2024.10.22 01:17:45.234750 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_738_346} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:45.257248 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_738_346} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_737_345, total 46457 rows starting from the beginning of the part 2024.10.22 01:17:45.346820 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5981f08e6828c2b151360079ac0c5ae4, offset: 0, size: 0 2024.10.22 01:17:45.348315 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5981f08e6828c2b151360079ac0c5ae4 for removal 2024.10.22 01:17:45.357498 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_738_346} MergeTreeSequentialSource: Reading 2 marks from part 202410_738_738_0, total 71 rows starting from the beginning of the part 2024.10.22 01:17:45.457046 [ 1476 ] {} CacheMetadata: Key 5981f08e6828c2b151360079ac0c5ae4 is removed from metadata 2024.10.22 01:17:45.457400 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6190151ed1dd0c29bdad464af813f89c, offset: 0, size: 0 2024.10.22 01:17:45.461265 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6190151ed1dd0c29bdad464af813f89c for removal 2024.10.22 01:17:45.464757 [ 1476 ] {} CacheMetadata: Key 6190151ed1dd0c29bdad464af813f89c is removed from metadata 2024.10.22 01:17:45.499782 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 11.86 MiB, will set to 1.47 GiB (RSS), difference: -9.18 MiB 2024.10.22 01:17:45.679802 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:45.682934 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:45.685153 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_424_202} MergeTask::MergeProjectionsStage: Merge sorted 44258 rows, containing 13 columns (13 merged, 0 gathered) in 3.516979466 sec., 12584.093944209568 rows/sec., 5.45 MiB/sec. 2024.10.22 01:17:45.730170 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_430_430_0 (state Temporary) 2024.10.22 01:17:45.795885 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_430_430_0 to 202410_443_443_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:45.974414 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_424_202} MergedBlockOutputStream: filled checksums 202410_1_424_202 (state Temporary) 2024.10.22 01:17:46.066698 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6574 2024.10.22 01:17:46.389564 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c557c78c80093e42b61db2d8da83d441) : [0, 165]: Setting download as failed: File segment: [0, 165], key: c557c78c80093e42b61db2d8da83d441, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:46.392451 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c557c78c80093e42b61db2d8da83d441): No space left in cache to reserve 166 bytes, will continue without cache download 2024.10.22 01:17:46.728841 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_424_202} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_424_202 to 202410_1_424_202 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:46.736235 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_424_202} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_423_201, 202410_424_424_0] -> 202410_1_424_202 2024.10.22 01:17:46.763455 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.08 MiB. 2024.10.22 01:17:47.381998 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(998265ff20fd02101b17d8f11c48d618) : [0, 108239]: Setting download as failed: File segment: [0, 108239], key: 998265ff20fd02101b17d8f11c48d618, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:47.383360 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(998265ff20fd02101b17d8f11c48d618): No space left in cache to reserve 108240 bytes, will continue without cache download 2024.10.22 01:17:47.398630 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_417_84/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:17:47.430268 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 998265ff20fd02101b17d8f11c48d618, offset: 0, size: 0 2024.10.22 01:17:47.433597 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 998265ff20fd02101b17d8f11c48d618 for removal 2024.10.22 01:17:47.448913 [ 1476 ] {} CacheMetadata: Key 998265ff20fd02101b17d8f11c48d618 is removed from metadata 2024.10.22 01:17:47.454953 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c557c78c80093e42b61db2d8da83d441, offset: 0, size: 0 2024.10.22 01:17:47.458778 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c557c78c80093e42b61db2d8da83d441 for removal 2024.10.22 01:17:47.464497 [ 1476 ] {} CacheMetadata: Key c557c78c80093e42b61db2d8da83d441 is removed from metadata 2024.10.22 01:17:47.833992 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 11.58 MiB, will set to 1.46 GiB (RSS), difference: 27.90 MiB 2024.10.22 01:17:48.148764 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_738_346} MergeTask::MergeProjectionsStage: Merge sorted 46528 rows, containing 13 columns (13 merged, 0 gathered) in 2.965417389 sec., 15690.202725792406 rows/sec., 2.51 MiB/sec. 2024.10.22 01:17:48.263557 [ 1161 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:17:48.298708 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_738_346/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:17:48.315007 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0552844252489bbc27dc006a93aedc8e) : [0, 166]: Setting download as failed: File segment: [0, 166], key: 0552844252489bbc27dc006a93aedc8e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:48.317635 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0552844252489bbc27dc006a93aedc8e): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 01:17:48.396915 [ 1161 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:17:48.533493 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_738_346} MergedBlockOutputStream: filled checksums 202410_1_738_346 (state Temporary) 2024.10.22 01:17:48.607864 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 653768 2024.10.22 01:17:48.720371 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_738_346} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_738_346 to 202410_1_738_346 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:48.726192 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_738_346} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_737_345, 202410_738_738_0] -> 202410_1_738_346 2024.10.22 01:17:48.761677 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.18 MiB. 2024.10.22 01:17:49.016050 [ 1162 ] {} DatabaseCatalog: Execution took 342 ms. 2024.10.22 01:17:49.328572 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:49.329783 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 46179 2024.10.22 01:17:49.336119 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:49.344590 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(befecfb1c897b64028e3353ffaf14f7e) : [0, 95010]: Setting download as failed: File segment: [0, 95010], key: befecfb1c897b64028e3353ffaf14f7e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:49.348058 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(befecfb1c897b64028e3353ffaf14f7e): No space left in cache to reserve 95011 bytes, will continue without cache download 2024.10.22 01:17:49.370574 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: befecfb1c897b64028e3353ffaf14f7e, offset: 0, size: 0 2024.10.22 01:17:49.371997 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key befecfb1c897b64028e3353ffaf14f7e for removal 2024.10.22 01:17:49.378762 [ 1476 ] {} CacheMetadata: Key befecfb1c897b64028e3353ffaf14f7e is removed from metadata 2024.10.22 01:17:49.384119 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0552844252489bbc27dc006a93aedc8e, offset: 0, size: 0 2024.10.22 01:17:49.386726 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0552844252489bbc27dc006a93aedc8e for removal 2024.10.22 01:17:49.390823 [ 1476 ] {} CacheMetadata: Key 0552844252489bbc27dc006a93aedc8e is removed from metadata 2024.10.22 01:17:49.574828 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_413_413_0 (state Temporary) 2024.10.22 01:17:49.754844 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_639_262, 202410_640_640_0] 2024.10.22 01:17:49.756843 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_639_262, 202410_640_640_0] 2024.10.22 01:17:49.772835 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_639_262, 202410_640_640_0] 2024.10.22 01:17:49.783498 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:49.785646 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:49.705638 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 230 entries to flush up to offset 45335 2024.10.22 01:17:49.872532 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_413_413_0 to 202410_420_420_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:49.919373 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 653768 2024.10.22 01:17:50.024188 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_731_731_0 (state Temporary) 2024.10.22 01:17:50.068808 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:17:50.071455 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:17:50.169048 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_731_731_0 to 202410_744_744_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:50.265063 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:50.268633 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:50.288002 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_738_346 to 202410_739_739_0 2024.10.22 01:17:50.291439 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.66 MiB using storage policy from min volume index 0 2024.10.22 01:17:50.309738 [ 1263 ] {} DiskLocal: Reserved 1.66 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:50.311228 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46179 2024.10.22 01:17:50.396737 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(43424f77e7d7a752af051891d1688115) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 43424f77e7d7a752af051891d1688115, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:17:50.431420 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_424_424_0 (state Temporary) 2024.10.22 01:17:50.536404 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 576, file segment info: File segment: [0, 33554431], key: 43424f77e7d7a752af051891d1688115, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:17:50.540753 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:50.543660 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_739_347} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_738_346 to 202410_739_739_0 into Compact with storage Full 2024.10.22 01:17:50.584297 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_739_347} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:50.599781 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_739_347} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_738_346, total 46528 rows starting from the beginning of the part 2024.10.22 01:17:50.630334 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_424_424_0 to 202410_430_430_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:50.633684 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.96 GiB, free memory in arenas 10.61 MiB, will set to 1.46 GiB (RSS), difference: -2.12 MiB 2024.10.22 01:17:50.757165 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_424_202 to 202410_425_425_0 2024.10.22 01:17:50.762108 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.26 MiB using storage policy from min volume index 0 2024.10.22 01:17:50.767872 [ 1248 ] {} DiskLocal: Reserved 2.26 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:50.774559 [ 1248 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:17:50.786821 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_739_347} MergeTreeSequentialSource: Reading 2 marks from part 202410_739_739_0, total 57 rows starting from the beginning of the part 2024.10.22 01:17:50.787394 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_425_203} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_424_202 to 202410_425_425_0 into Wide with storage Full 2024.10.22 01:17:50.820835 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_425_203} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:50.835979 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_425_203} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_424_202, total 44258 rows starting from the beginning of the part 2024.10.22 01:17:50.877282 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 45335 2024.10.22 01:17:50.880622 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_366_144, 202410_367_367_0] 2024.10.22 01:17:50.883097 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_366_144, 202410_367_367_0] 2024.10.22 01:17:50.996343 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_366_144, 202410_367_367_0] 2024.10.22 01:17:51.230206 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_425_203} MergeTreeSequentialSource: Reading 2 marks from part 202410_425_425_0, total 156 rows starting from the beginning of the part 2024.10.22 01:17:51.367628 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(87167841e4623268811366e5df1f04b8) : [0, 232]: Setting download as failed: File segment: [0, 232], key: 87167841e4623268811366e5df1f04b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:51.369775 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(87167841e4623268811366e5df1f04b8): No space left in cache to reserve 233 bytes, will continue without cache download 2024.10.22 01:17:51.655845 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_89_18/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:17:51.892631 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 366 2024.10.22 01:17:52.076304 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d62c1799e2aef9238717dd69d14f1f65) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d62c1799e2aef9238717dd69d14f1f65, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:17:52.079070 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 839, file segment info: File segment: [0, 33554431], key: d62c1799e2aef9238717dd69d14f1f65, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:17:52.081273 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:52.284426 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:52.286122 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:52.352521 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_348_348_0 (state Temporary) 2024.10.22 01:17:52.532109 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_348_348_0 to 202410_358_358_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:52.709372 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 366 2024.10.22 01:17:52.789090 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_739_347} MergeTask::MergeProjectionsStage: Merge sorted 46585 rows, containing 13 columns (13 merged, 0 gathered) in 2.358174978 sec., 19754.683360905376 rows/sec., 3.15 MiB/sec. 2024.10.22 01:17:52.827874 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3d373de61e8a7caaf657b58ecbe1d8fb) : [0, 177991]: Setting download as failed: File segment: [0, 177991], key: 3d373de61e8a7caaf657b58ecbe1d8fb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:52.830095 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3d373de61e8a7caaf657b58ecbe1d8fb): No space left in cache to reserve 177992 bytes, will continue without cache download 2024.10.22 01:17:52.862041 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3d373de61e8a7caaf657b58ecbe1d8fb, offset: 0, size: 0 2024.10.22 01:17:52.864065 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3d373de61e8a7caaf657b58ecbe1d8fb for removal 2024.10.22 01:17:52.866745 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 87167841e4623268811366e5df1f04b8, offset: 0, size: 0 2024.10.22 01:17:52.867953 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 87167841e4623268811366e5df1f04b8 for removal 2024.10.22 01:17:52.967509 [ 1476 ] {} CacheMetadata: Key 3d373de61e8a7caaf657b58ecbe1d8fb is removed from metadata 2024.10.22 01:17:52.979079 [ 1476 ] {} CacheMetadata: Key 87167841e4623268811366e5df1f04b8 is removed from metadata 2024.10.22 01:17:53.082384 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_739_347} MergedBlockOutputStream: filled checksums 202410_1_739_347 (state Temporary) 2024.10.22 01:17:53.231301 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_739_347} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_739_347 to 202410_1_739_347 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:53.234698 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_739_347} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_738_346, 202410_739_739_0] -> 202410_1_739_347 2024.10.22 01:17:53.262035 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.17 MiB. 2024.10.22 01:17:53.330404 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c8fc994da8d134980a087b5d6a37b0ef) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c8fc994da8d134980a087b5d6a37b0ef, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:17:53.333834 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 114757, file segment info: File segment: [0, 33554431], key: c8fc994da8d134980a087b5d6a37b0ef, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:17:53.342117 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:53.568323 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 6588 2024.10.22 01:17:53.870096 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:53.874367 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:53.907621 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_431_431_0 (state Temporary) 2024.10.22 01:17:54.141163 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_640_263, 202410_641_641_0] 2024.10.22 01:17:54.146208 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_640_263, 202410_641_641_0] 2024.10.22 01:17:54.152694 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_425_203} MergeTask::MergeProjectionsStage: Merge sorted 44414 rows, containing 13 columns (13 merged, 0 gathered) in 3.36817961 sec., 13186.351425006103 rows/sec., 5.71 MiB/sec. 2024.10.22 01:17:54.161115 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_87_87_0 to 202410_90_90_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:54.172389 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_640_263, 202410_641_641_0] 2024.10.22 01:17:54.172817 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_431_431_0 to 202410_444_444_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:54.274539 [ 1240 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 262 ms. 2024.10.22 01:17:54.318995 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6588 2024.10.22 01:17:54.321415 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f41f46c7d8dfc22f3ba208804b9b83c7) : [0, 237]: Setting download as failed: File segment: [0, 237], key: f41f46c7d8dfc22f3ba208804b9b83c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:54.327715 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 91 entries to flush up to offset 46270 2024.10.22 01:17:54.328771 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f41f46c7d8dfc22f3ba208804b9b83c7): No space left in cache to reserve 238 bytes, will continue without cache download 2024.10.22 01:17:54.370250 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_425_203} MergedBlockOutputStream: filled checksums 202410_1_425_203 (state Temporary) 2024.10.22 01:17:54.520173 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:54.524194 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:54.581069 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.96 GiB, free memory in arenas 10.93 MiB, will set to 1.47 GiB (RSS), difference: -36.06 MiB 2024.10.22 01:17:54.585994 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_732_732_0 (state Temporary) 2024.10.22 01:17:54.736335 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_732_732_0 to 202410_745_745_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:54.772307 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_739_347 to 202410_740_740_0 2024.10.22 01:17:54.790276 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.67 MiB using storage policy from min volume index 0 2024.10.22 01:17:54.793378 [ 1247 ] {} DiskLocal: Reserved 1.67 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:54.814806 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46270 2024.10.22 01:17:54.834731 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_740_348} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_739_347 to 202410_740_740_0 into Compact with storage Full 2024.10.22 01:17:54.887575 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_740_348} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:54.912399 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_740_348} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_739_347, total 46585 rows starting from the beginning of the part 2024.10.22 01:17:54.970410 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(69cc27d6bbce59fc0548a6ca753473ba) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 69cc27d6bbce59fc0548a6ca753473ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:17:54.984735 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 898, file segment info: File segment: [0, 33554431], key: 69cc27d6bbce59fc0548a6ca753473ba, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:17:54.989259 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:17:55.033282 [ 1709 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tej/btsmkwpzcgfqzvdwcgqcegblabsbb, size 114757 2024.10.22 01:17:55.082989 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_740_348} MergeTreeSequentialSource: Reading 2 marks from part 202410_740_740_0, total 95 rows starting from the beginning of the part 2024.10.22 01:17:55.337085 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:17:55.339609 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_425_203} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_425_203 to 202410_1_425_203 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:55.349523 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_425_203} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_424_202, 202410_425_425_0] -> 202410_1_425_203 2024.10.22 01:17:55.382621 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:17:55.418530 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:17:55.472550 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.02 MiB. 2024.10.22 01:17:55.474631 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:17:55.502125 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:17:55.560860 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:17:55.687572 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:17:55.724710 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:17:55.774476 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:17:55.884187 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kph/yrnywuhxpjgyursowlqkzieuxefzf (LogSeriesLimiter: on interval from 2024-10-22 01:17:43 to 2024-10-22 01:17:55 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:17:56.035702 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2e5349f5ff9a9835f7d4cfae06b19701) : [0, 295564]: Setting download as failed: File segment: [0, 295564], key: 2e5349f5ff9a9835f7d4cfae06b19701, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:56.042456 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2e5349f5ff9a9835f7d4cfae06b19701): No space left in cache to reserve 295565 bytes, will continue without cache download 2024.10.22 01:17:56.092313 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2e5349f5ff9a9835f7d4cfae06b19701, offset: 0, size: 0 2024.10.22 01:17:56.093738 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2e5349f5ff9a9835f7d4cfae06b19701 for removal 2024.10.22 01:17:56.100070 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f41f46c7d8dfc22f3ba208804b9b83c7, offset: 0, size: 0 2024.10.22 01:17:56.104088 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f41f46c7d8dfc22f3ba208804b9b83c7 for removal 2024.10.22 01:17:56.107935 [ 1476 ] {} CacheMetadata: Key 2e5349f5ff9a9835f7d4cfae06b19701 is removed from metadata 2024.10.22 01:17:56.115840 [ 1476 ] {} CacheMetadata: Key f41f46c7d8dfc22f3ba208804b9b83c7 is removed from metadata 2024.10.22 01:17:56.925393 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1157 entries to flush up to offset 654925 2024.10.22 01:17:57.021552 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3462 2024.10.22 01:17:57.319042 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(61663cbe91ba412e405f692840dc13f6) : [0, 164]: Setting download as failed: File segment: [0, 164], key: 61663cbe91ba412e405f692840dc13f6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:57.320478 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(61663cbe91ba412e405f692840dc13f6): No space left in cache to reserve 165 bytes, will continue without cache download 2024.10.22 01:17:57.455733 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 12.18 MiB, will set to 1.48 GiB (RSS), difference: 38.35 MiB 2024.10.22 01:17:57.533649 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_417_84} MergeTask::MergeProjectionsStage: Merge sorted 663893 rows, containing 4 columns (4 merged, 0 gathered) in 24.38825404 sec., 27221.83387589479 rows/sec., 398.94 KiB/sec. 2024.10.22 01:17:57.599119 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_417_84} MergedBlockOutputStream: filled checksums 202410_1_417_84 (state Temporary) 2024.10.22 01:17:57.660602 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_417_84} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_417_84 to 202410_1_417_84 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:57.768730 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_417_84} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 5 parts: [202410_1_413_83, 202410_417_417_0] -> 202410_1_417_84 2024.10.22 01:17:57.973873 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:17:58.045334 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_740_348} MergeTask::MergeProjectionsStage: Merge sorted 46680 rows, containing 13 columns (13 merged, 0 gathered) in 3.23409108 sec., 14433.730790290545 rows/sec., 2.31 MiB/sec. 2024.10.22 01:17:58.066264 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:58.069172 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:58.132939 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_740_348} MergedBlockOutputStream: filled checksums 202410_1_740_348 (state Temporary) 2024.10.22 01:17:58.170854 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3f542cf98dd90c661aaabcac396e70b7) : [0, 193958]: Setting download as failed: File segment: [0, 193958], key: 3f542cf98dd90c661aaabcac396e70b7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:58.171716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3f542cf98dd90c661aaabcac396e70b7): No space left in cache to reserve 193959 bytes, will continue without cache download 2024.10.22 01:17:58.192009 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3f542cf98dd90c661aaabcac396e70b7, offset: 0, size: 0 2024.10.22 01:17:58.193740 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3f542cf98dd90c661aaabcac396e70b7 for removal 2024.10.22 01:17:58.197298 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 61663cbe91ba412e405f692840dc13f6, offset: 0, size: 0 2024.10.22 01:17:58.198849 [ 1476 ] {} CacheMetadata: Key 3f542cf98dd90c661aaabcac396e70b7 is removed from metadata 2024.10.22 01:17:58.199939 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 61663cbe91ba412e405f692840dc13f6 for removal 2024.10.22 01:17:58.208146 [ 1476 ] {} CacheMetadata: Key 61663cbe91ba412e405f692840dc13f6 is removed from metadata 2024.10.22 01:17:58.276887 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_740_348} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_740_348 to 202410_1_740_348 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:58.280450 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_740_348} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_739_347, 202410_740_740_0] -> 202410_1_740_348 2024.10.22 01:17:58.304083 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_414_414_0 (state Temporary) 2024.10.22 01:17:58.409269 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.19 MiB. 2024.10.22 01:17:58.416130 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 260 entries to flush up to offset 45595 2024.10.22 01:17:58.473797 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_414_414_0 to 202410_421_421_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:58.621419 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 654925 2024.10.22 01:17:58.798580 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:58.803219 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:58.836536 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 46344 2024.10.22 01:17:58.866112 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_425_425_0 (state Temporary) 2024.10.22 01:17:58.965559 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_425_425_0 to 202410_431_431_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:58.982438 [ 1157 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_425_203 to 202410_426_426_0 2024.10.22 01:17:58.985544 [ 1157 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.27 MiB using storage policy from min volume index 0 2024.10.22 01:17:58.988329 [ 1157 ] {} DiskLocal: Reserved 2.27 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:58.991870 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 45595 2024.10.22 01:17:59.052337 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_426_204} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_425_203 to 202410_426_426_0 into Wide with storage Full 2024.10.22 01:17:59.068462 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_426_204} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:59.073807 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_426_204} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_425_203, total 44414 rows starting from the beginning of the part 2024.10.22 01:17:59.168051 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a5a8cc6e8e894f6551f2d0959cd4038a) : [0, 231]: Setting download as failed: File segment: [0, 231], key: a5a8cc6e8e894f6551f2d0959cd4038a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:17:59.170072 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a5a8cc6e8e894f6551f2d0959cd4038a): No space left in cache to reserve 232 bytes, will continue without cache download 2024.10.22 01:17:59.178829 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_367_145, 202410_368_368_0] 2024.10.22 01:17:59.180641 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_367_145, 202410_368_368_0] 2024.10.22 01:17:59.184120 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:17:59.189995 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:59.207045 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_367_145, 202410_368_368_0] 2024.10.22 01:17:59.280301 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_426_204} MergeTreeSequentialSource: Reading 2 marks from part 202410_426_426_0, total 175 rows starting from the beginning of the part 2024.10.22 01:17:59.294922 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_733_733_0 (state Temporary) 2024.10.22 01:17:59.365618 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_733_733_0 to 202410_746_746_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:17:59.391955 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_740_348 to 202410_741_741_0 2024.10.22 01:17:59.432236 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46344 2024.10.22 01:17:59.467615 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.96 GiB, free memory in arenas 10.81 MiB, will set to 1.46 GiB (RSS), difference: -30.14 MiB 2024.10.22 01:17:59.524496 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.67 MiB using storage policy from min volume index 0 2024.10.22 01:17:59.531231 [ 1173 ] {} DiskLocal: Reserved 1.67 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:17:59.585640 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_741_349} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_740_348 to 202410_741_741_0 into Compact with storage Full 2024.10.22 01:17:59.651531 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_356_71, 202410_357_357_0, 202410_358_358_0, 202410_359_359_0, 202410_360_360_0, 202410_361_361_0] 2024.10.22 01:17:59.655947 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_356_71, 202410_357_357_0, 202410_358_358_0, 202410_359_359_0, 202410_360_360_0, 202410_361_361_0] 2024.10.22 01:17:59.684194 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_356_71, 202410_357_357_0, 202410_358_358_0, 202410_359_359_0, 202410_360_360_0, 202410_361_361_0] 2024.10.22 01:17:59.723047 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_741_349} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:17:59.752296 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_741_349} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_740_348, total 46680 rows starting from the beginning of the part 2024.10.22 01:17:59.794917 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_741_349} MergeTreeSequentialSource: Reading 2 marks from part 202410_741_741_0, total 106 rows starting from the beginning of the part 2024.10.22 01:17:59.814632 [ 1178 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 244 ms. 2024.10.22 01:18:00.070271 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:18:00.113818 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:18:00.143681 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(44553a8691486abece278ccc22bc6363) : [0, 557653]: Setting download as failed: File segment: [0, 557653], key: 44553a8691486abece278ccc22bc6363, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:00.147051 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(44553a8691486abece278ccc22bc6363): No space left in cache to reserve 557654 bytes, will continue without cache download 2024.10.22 01:18:00.216094 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 367 2024.10.22 01:18:00.287048 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 44553a8691486abece278ccc22bc6363, offset: 0, size: 0 2024.10.22 01:18:00.290717 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 44553a8691486abece278ccc22bc6363 for removal 2024.10.22 01:18:00.299359 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a5a8cc6e8e894f6551f2d0959cd4038a, offset: 0, size: 0 2024.10.22 01:18:00.302632 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a5a8cc6e8e894f6551f2d0959cd4038a for removal 2024.10.22 01:18:00.412822 [ 1476 ] {} CacheMetadata: Key 44553a8691486abece278ccc22bc6363 is removed from metadata 2024.10.22 01:18:00.422915 [ 1476 ] {} CacheMetadata: Key a5a8cc6e8e894f6551f2d0959cd4038a is removed from metadata 2024.10.22 01:18:00.466272 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_641_264, 202410_642_642_0] 2024.10.22 01:18:00.468702 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_641_264, 202410_642_642_0] 2024.10.22 01:18:00.480103 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_641_264, 202410_642_642_0] 2024.10.22 01:18:00.591614 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:00.595680 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:00.771134 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_349_349_0 (state Temporary) 2024.10.22 01:18:00.889591 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_349_349_0 to 202410_359_359_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:00.961735 [ 1180 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_354_73 to 202410_359_359_0 2024.10.22 01:18:00.993403 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 367 2024.10.22 01:18:00.993787 [ 1180 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:00.998926 [ 1180 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:01.018691 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_354_73 to 202410_359_359_0 into Compact with storage Full 2024.10.22 01:18:01.311925 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:01.359770 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_354_73, total 398 rows starting from the beginning of the part 2024.10.22 01:18:01.519438 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7cad0e7f305b49d7edeebe428756882a) : [0, 265]: Setting download as failed: File segment: [0, 265], key: 7cad0e7f305b49d7edeebe428756882a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:01.524346 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7cad0e7f305b49d7edeebe428756882a): No space left in cache to reserve 266 bytes, will continue without cache download 2024.10.22 01:18:01.592272 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_426_204} MergeTask::MergeProjectionsStage: Merge sorted 44589 rows, containing 13 columns (13 merged, 0 gathered) in 2.440757469 sec., 18268.50908634872 rows/sec., 7.91 MiB/sec. 2024.10.22 01:18:01.734558 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_355_355_0, total 1 rows starting from the beginning of the part 2024.10.22 01:18:01.803730 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_426_204} MergedBlockOutputStream: filled checksums 202410_1_426_204 (state Temporary) 2024.10.22 01:18:01.816071 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_356_356_0, total 1 rows starting from the beginning of the part 2024.10.22 01:18:01.828833 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 6607 2024.10.22 01:18:01.916627 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_741_349} MergeTask::MergeProjectionsStage: Merge sorted 46786 rows, containing 13 columns (13 merged, 0 gathered) in 2.33703601 sec., 20019.374883316414 rows/sec., 3.20 MiB/sec. 2024.10.22 01:18:01.984526 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_357_357_0, total 1 rows starting from the beginning of the part 2024.10.22 01:18:02.181658 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_358_358_0, total 1 rows starting from the beginning of the part 2024.10.22 01:18:02.206617 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_741_349} MergedBlockOutputStream: filled checksums 202410_1_741_349 (state Temporary) 2024.10.22 01:18:02.259084 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_741_349} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_741_349 to 202410_1_741_349 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:02.264033 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_741_349} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_740_348, 202410_741_741_0] -> 202410_1_741_349 2024.10.22 01:18:02.291818 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.19 MiB. 2024.10.22 01:18:02.352876 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_359_359_0, total 1 rows starting from the beginning of the part 2024.10.22 01:18:02.374492 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:02.376409 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:02.393342 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 6.16 MiB, will set to 1.47 GiB (RSS), difference: 38.30 MiB 2024.10.22 01:18:02.415579 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_432_432_0 (state Temporary) 2024.10.22 01:18:02.462248 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(02f2ae6b922e1a9fcb3dfb407f250f80) : [0, 2121928]: Setting download as failed: File segment: [0, 2121928], key: 02f2ae6b922e1a9fcb3dfb407f250f80, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:02.464205 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(02f2ae6b922e1a9fcb3dfb407f250f80): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:18:02.477044 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_432_432_0 to 202410_445_445_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:02.581623 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_426_204} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_426_204 to 202410_1_426_204 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:02.609589 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_426_204} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_425_203, 202410_426_426_0] -> 202410_1_426_204 2024.10.22 01:18:02.640091 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.02 MiB. 2024.10.22 01:18:02.686188 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 02f2ae6b922e1a9fcb3dfb407f250f80, offset: 0, size: 0 2024.10.22 01:18:02.688594 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6607 2024.10.22 01:18:02.689157 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 02f2ae6b922e1a9fcb3dfb407f250f80 for removal 2024.10.22 01:18:02.692245 [ 1476 ] {} CacheMetadata: Key 02f2ae6b922e1a9fcb3dfb407f250f80 is removed from metadata 2024.10.22 01:18:02.697355 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7cad0e7f305b49d7edeebe428756882a, offset: 0, size: 0 2024.10.22 01:18:02.700790 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7cad0e7f305b49d7edeebe428756882a for removal 2024.10.22 01:18:02.707785 [ 1476 ] {} CacheMetadata: Key 7cad0e7f305b49d7edeebe428756882a is removed from metadata 2024.10.22 01:18:03.409758 [ 1201 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:18:03.443506 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 91 entries to flush up to offset 46435 2024.10.22 01:18:03.462256 [ 1201 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:18:03.494505 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c83716cdb68f3705acee048709b26a7a) : [0, 169]: Setting download as failed: File segment: [0, 169], key: c83716cdb68f3705acee048709b26a7a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:03.497166 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c83716cdb68f3705acee048709b26a7a): No space left in cache to reserve 170 bytes, will continue without cache download 2024.10.22 01:18:03.837695 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_642_265, 202410_643_643_0] 2024.10.22 01:18:03.842280 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_642_265, 202410_643_643_0] 2024.10.22 01:18:03.854874 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_642_265, 202410_643_643_0] 2024.10.22 01:18:03.971781 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:03.975114 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:04.045442 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_734_734_0 (state Temporary) 2024.10.22 01:18:04.103143 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_734_734_0 to 202410_747_747_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:04.145065 [ 1199 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_741_349 to 202410_742_742_0 2024.10.22 01:18:04.163743 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46435 2024.10.22 01:18:04.166980 [ 1199 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.67 MiB using storage policy from min volume index 0 2024.10.22 01:18:04.170046 [ 1199 ] {} DiskLocal: Reserved 1.67 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:04.192016 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_742_350} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_741_349 to 202410_742_742_0 into Compact with storage Full 2024.10.22 01:18:04.228328 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_742_350} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:04.239742 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_742_350} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_741_349, total 46786 rows starting from the beginning of the part 2024.10.22 01:18:04.245428 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} MergeTask::MergeProjectionsStage: Merge sorted 403 rows, containing 32 columns (32 merged, 0 gathered) in 3.233963722 sec., 124.61487964706365 rows/sec., 11.72 KiB/sec. 2024.10.22 01:18:04.286995 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} MergedBlockOutputStream: filled checksums 202410_1_359_74 (state Temporary) 2024.10.22 01:18:04.368385 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_742_350} MergeTreeSequentialSource: Reading 2 marks from part 202410_742_742_0, total 71 rows starting from the beginning of the part 2024.10.22 01:18:04.375405 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_359_74/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:18:04.387274 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(17b3935c210dc4fe7be7a11efea59c9d) : [0, 214343]: Setting download as failed: File segment: [0, 214343], key: 17b3935c210dc4fe7be7a11efea59c9d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:04.389088 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(17b3935c210dc4fe7be7a11efea59c9d): No space left in cache to reserve 214344 bytes, will continue without cache download 2024.10.22 01:18:04.545118 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 3503 2024.10.22 01:18:04.625349 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 4.67 MiB, will set to 1.46 GiB (RSS), difference: -17.82 MiB 2024.10.22 01:18:04.744921 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 17b3935c210dc4fe7be7a11efea59c9d, offset: 0, size: 0 2024.10.22 01:18:04.749225 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 17b3935c210dc4fe7be7a11efea59c9d for removal 2024.10.22 01:18:04.756913 [ 1476 ] {} CacheMetadata: Key 17b3935c210dc4fe7be7a11efea59c9d is removed from metadata 2024.10.22 01:18:04.757809 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c83716cdb68f3705acee048709b26a7a, offset: 0, size: 0 2024.10.22 01:18:04.763245 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c83716cdb68f3705acee048709b26a7a for removal 2024.10.22 01:18:04.766488 [ 1476 ] {} CacheMetadata: Key c83716cdb68f3705acee048709b26a7a is removed from metadata 2024.10.22 01:18:05.235775 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_359_74 to 202410_1_359_74 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:05.239047 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_359_74} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_354_73, 202410_359_359_0] -> 202410_1_359_74 2024.10.22 01:18:05.279782 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:18:05.343499 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_89_18/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:18:05.626931 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 656465 2024.10.22 01:18:05.671316 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(276252446058833a3561dc9085cb81c5) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 276252446058833a3561dc9085cb81c5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:05.673637 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(276252446058833a3561dc9085cb81c5): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:18:06.381401 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_742_350} MergeTask::MergeProjectionsStage: Merge sorted 46857 rows, containing 13 columns (13 merged, 0 gathered) in 2.196305697 sec., 21334.461802837093 rows/sec., 3.41 MiB/sec. 2024.10.22 01:18:06.519542 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:06.527540 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:06.535893 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 225 entries to flush up to offset 45820 2024.10.22 01:18:06.562731 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e05e987270c413dd7499a1283d42edf8) : [0, 6911]: Setting download as failed: File segment: [0, 6911], key: e05e987270c413dd7499a1283d42edf8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:06.565311 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e05e987270c413dd7499a1283d42edf8): No space left in cache to reserve 6912 bytes, will continue without cache download 2024.10.22 01:18:06.572161 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_742_350} MergedBlockOutputStream: filled checksums 202410_1_742_350 (state Temporary) 2024.10.22 01:18:06.604187 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e05e987270c413dd7499a1283d42edf8, offset: 0, size: 0 2024.10.22 01:18:06.608774 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e05e987270c413dd7499a1283d42edf8 for removal 2024.10.22 01:18:06.618669 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 276252446058833a3561dc9085cb81c5, offset: 0, size: 0 2024.10.22 01:18:06.619025 [ 1476 ] {} CacheMetadata: Key e05e987270c413dd7499a1283d42edf8 is removed from metadata 2024.10.22 01:18:06.623568 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 276252446058833a3561dc9085cb81c5 for removal 2024.10.22 01:18:06.627768 [ 1476 ] {} CacheMetadata: Key 276252446058833a3561dc9085cb81c5 is removed from metadata 2024.10.22 01:18:06.768770 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_415_415_0 (state Temporary) 2024.10.22 01:18:06.769811 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_742_350} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_742_350 to 202410_1_742_350 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:06.773975 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_742_350} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_741_349, 202410_742_742_0] -> 202410_1_742_350 2024.10.22 01:18:06.793567 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.18 MiB. 2024.10.22 01:18:06.798535 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:06.801450 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:06.808991 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.96 GiB, free memory in arenas 5.30 MiB, will set to 1.46 GiB (RSS), difference: 8.98 MiB 2024.10.22 01:18:06.876533 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_415_415_0 to 202410_422_422_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:06.919643 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 656465 2024.10.22 01:18:06.987048 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_426_426_0 (state Temporary) 2024.10.22 01:18:07.077347 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_426_426_0 to 202410_432_432_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:07.129277 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 45820 2024.10.22 01:18:06.913739 [ 1177 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_417_84 to 202410_418_418_0 2024.10.22 01:18:07.139121 [ 1177 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:07.144594 [ 1177 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:07.215049 [ 1159 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_426_204 to 202410_427_427_0 2024.10.22 01:18:07.252208 [ 1177 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 360 ms. 2024.10.22 01:18:07.262832 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_418_85} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_417_84 to 202410_418_418_0 into Compact with storage Full 2024.10.22 01:18:07.270734 [ 1159 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.28 MiB using storage policy from min volume index 0 2024.10.22 01:18:07.274680 [ 1159 ] {} DiskLocal: Reserved 2.28 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:07.276559 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_418_85} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:07.288151 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_427_205} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_426_204 to 202410_427_427_0 into Wide with storage Full 2024.10.22 01:18:07.288529 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_418_85} MergeTreeSequentialSource: Reading 83 marks from part 202410_1_417_84, total 663893 rows starting from the beginning of the part 2024.10.22 01:18:07.311710 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_418_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_418_418_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:18:07.480460 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_368_146, 202410_369_369_0] 2024.10.22 01:18:07.483311 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_368_146, 202410_369_369_0] 2024.10.22 01:18:07.517252 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_427_205} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:07.527114 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_427_205} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_426_204, total 44589 rows starting from the beginning of the part 2024.10.22 01:18:07.630245 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_368_146, 202410_369_369_0] 2024.10.22 01:18:07.763000 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_381_76, 202410_382_382_0, 202410_383_383_0, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0] 2024.10.22 01:18:07.766536 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_381_76, 202410_382_382_0, 202410_383_383_0, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0] 2024.10.22 01:18:07.800269 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_381_76, 202410_382_382_0, 202410_383_383_0, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0] 2024.10.22 01:18:07.863676 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f2375380013eb6130f21415aaf22fc23) : [0, 266]: Setting download as failed: File segment: [0, 266], key: f2375380013eb6130f21415aaf22fc23, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:07.865192 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f2375380013eb6130f21415aaf22fc23): No space left in cache to reserve 267 bytes, will continue without cache download 2024.10.22 01:18:07.871192 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_427_205} MergeTreeSequentialSource: Reading 2 marks from part 202410_427_427_0, total 149 rows starting from the beginning of the part 2024.10.22 01:18:08.132798 [ 1274 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 221 ms. 2024.10.22 01:18:08.180933 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 46523 2024.10.22 01:18:08.504868 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:08.514529 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:08.601797 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_735_735_0 (state Temporary) 2024.10.22 01:18:08.764408 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_735_735_0 to 202410_748_748_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:08.792473 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_742_350 to 202410_743_743_0 2024.10.22 01:18:08.798685 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.68 MiB using storage policy from min volume index 0 2024.10.22 01:18:08.799763 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46523 2024.10.22 01:18:08.799915 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ccf70d9260cfd7b397ed8c8f7ecb8dcb) : [0, 2215983]: Setting download as failed: File segment: [0, 2215983], key: ccf70d9260cfd7b397ed8c8f7ecb8dcb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:08.803822 [ 1150 ] {} DiskLocal: Reserved 1.68 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:08.807110 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ccf70d9260cfd7b397ed8c8f7ecb8dcb): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:18:08.857247 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_743_351} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_742_350 to 202410_743_743_0 into Compact with storage Full 2024.10.22 01:18:08.991316 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_743_351} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:09.002979 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_743_351} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_742_350, total 46857 rows starting from the beginning of the part 2024.10.22 01:18:09.066453 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_418_85/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:18:09.082545 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ccf70d9260cfd7b397ed8c8f7ecb8dcb, offset: 0, size: 0 2024.10.22 01:18:09.088473 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ccf70d9260cfd7b397ed8c8f7ecb8dcb for removal 2024.10.22 01:18:09.091345 [ 1476 ] {} CacheMetadata: Key ccf70d9260cfd7b397ed8c8f7ecb8dcb is removed from metadata 2024.10.22 01:18:09.098351 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f2375380013eb6130f21415aaf22fc23, offset: 0, size: 0 2024.10.22 01:18:09.100270 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f2375380013eb6130f21415aaf22fc23 for removal 2024.10.22 01:18:09.104721 [ 1476 ] {} CacheMetadata: Key f2375380013eb6130f21415aaf22fc23 is removed from metadata 2024.10.22 01:18:09.134198 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_743_351} MergeTreeSequentialSource: Reading 2 marks from part 202410_743_743_0, total 86 rows starting from the beginning of the part 2024.10.22 01:18:09.392760 [ 1248 ] {} DatabaseCatalog: Execution took 245 ms. 2024.10.22 01:18:09.557044 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 7.60 MiB, will set to 1.48 GiB (RSS), difference: -27.17 MiB 2024.10.22 01:18:09.944148 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_427_205} MergeTask::MergeProjectionsStage: Merge sorted 44738 rows, containing 13 columns (13 merged, 0 gathered) in 2.660147618 sec., 16817.863676917197 rows/sec., 7.28 MiB/sec. 2024.10.22 01:18:10.069780 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:18:10.110261 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:18:10.161398 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_643_266, 202410_644_644_0] 2024.10.22 01:18:10.167369 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_643_266, 202410_644_644_0] 2024.10.22 01:18:10.180755 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_643_266, 202410_644_644_0] 2024.10.22 01:18:10.191371 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 6628 2024.10.22 01:18:10.198906 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d89edd17d9b0de8ce53cd580fdbc8de0) : [0, 170]: Setting download as failed: File segment: [0, 170], key: d89edd17d9b0de8ce53cd580fdbc8de0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:10.203808 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d89edd17d9b0de8ce53cd580fdbc8de0): No space left in cache to reserve 171 bytes, will continue without cache download 2024.10.22 01:18:10.206598 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_427_205} MergedBlockOutputStream: filled checksums 202410_1_427_205 (state Temporary) 2024.10.22 01:18:10.344478 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:10.349709 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:10.383595 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_433_433_0 (state Temporary) 2024.10.22 01:18:10.533657 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_433_433_0 to 202410_446_446_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:10.564867 [ 1233 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_441_88 to 202410_446_446_0 2024.10.22 01:18:10.572102 [ 1233 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:10.576765 [ 1233 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:10.588206 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_441_88 to 202410_446_446_0 into Compact with storage Full 2024.10.22 01:18:10.646555 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_427_205} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_427_205 to 202410_1_427_205 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:10.655798 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_427_205} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_426_204, 202410_427_427_0] -> 202410_1_427_205 2024.10.22 01:18:10.669285 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.01 MiB. 2024.10.22 01:18:10.683897 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6628 2024.10.22 01:18:10.695875 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_306_63, 202410_307_307_0, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0] 2024.10.22 01:18:10.698382 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_306_63, 202410_307_307_0, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0] 2024.10.22 01:18:10.762717 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:10.805960 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_441_88, total 6643 rows starting from the beginning of the part 2024.10.22 01:18:10.843844 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_306_63, 202410_307_307_0, 202410_308_308_0, 202410_309_309_0, 202410_310_310_0] 2024.10.22 01:18:10.884327 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_442_442_0, total 25 rows starting from the beginning of the part 2024.10.22 01:18:10.964229 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_443_443_0, total 12 rows starting from the beginning of the part 2024.10.22 01:18:11.088097 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(f241ef8fcfaff4cb890021fe30212a26) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f241ef8fcfaff4cb890021fe30212a26, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:18:11.092261 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 92667, file segment info: File segment: [0, 33554431], key: f241ef8fcfaff4cb890021fe30212a26, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:18:11.096359 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:11.142546 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_444_444_0, total 14 rows starting from the beginning of the part 2024.10.22 01:18:11.336522 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_445_445_0, total 19 rows starting from the beginning of the part 2024.10.22 01:18:11.538461 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_446_446_0, total 21 rows starting from the beginning of the part 2024.10.22 01:18:12.119049 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(7b5fdcb3f8c31408e19ee53443ae92c5) : [0, 163551]: Setting download as failed: File segment: [0, 163551], key: 7b5fdcb3f8c31408e19ee53443ae92c5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:12.120348 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(7b5fdcb3f8c31408e19ee53443ae92c5): No space left in cache to reserve 163552 bytes, will continue without cache download 2024.10.22 01:18:12.226882 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_743_351} MergeTask::MergeProjectionsStage: Merge sorted 46943 rows, containing 13 columns (13 merged, 0 gathered) in 3.388169332 sec., 13854.974589563992 rows/sec., 2.21 MiB/sec. 2024.10.22 01:18:12.371170 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7b5fdcb3f8c31408e19ee53443ae92c5, offset: 0, size: 0 2024.10.22 01:18:12.373294 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7b5fdcb3f8c31408e19ee53443ae92c5 for removal 2024.10.22 01:18:12.376217 [ 1476 ] {} CacheMetadata: Key 7b5fdcb3f8c31408e19ee53443ae92c5 is removed from metadata 2024.10.22 01:18:12.379552 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d89edd17d9b0de8ce53cd580fdbc8de0, offset: 0, size: 0 2024.10.22 01:18:12.386467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d89edd17d9b0de8ce53cd580fdbc8de0 for removal 2024.10.22 01:18:12.396344 [ 1476 ] {} CacheMetadata: Key d89edd17d9b0de8ce53cd580fdbc8de0 is removed from metadata 2024.10.22 01:18:12.423649 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_743_351} MergedBlockOutputStream: filled checksums 202410_1_743_351 (state Temporary) 2024.10.22 01:18:12.464158 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} MergeTask::MergeProjectionsStage: Merge sorted 6734 rows, containing 12 columns (12 merged, 0 gathered) in 1.881490856 sec., 3579.0766553690864 rows/sec., 567.39 KiB/sec. 2024.10.22 01:18:12.493562 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} MergedBlockOutputStream: filled checksums 202410_1_446_89 (state Temporary) 2024.10.22 01:18:12.505288 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 12.64 MiB, will set to 1.47 GiB (RSS), difference: 35.75 MiB 2024.10.22 01:18:12.586442 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_743_351} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_743_351 to 202410_1_743_351 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:12.592949 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_743_351} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_742_350, 202410_743_743_0] -> 202410_1_743_351 2024.10.22 01:18:12.634755 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.19 MiB. 2024.10.22 01:18:12.800433 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_446_89 to 202410_1_446_89 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:12.805706 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_446_89} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_441_88, 202410_446_446_0] -> 202410_1_446_89 2024.10.22 01:18:12.856265 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 46599 2024.10.22 01:18:12.995427 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.40 MiB. 2024.10.22 01:18:13.189176 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(d2cbb1399a9b5e56b65023f905146a41) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d2cbb1399a9b5e56b65023f905146a41, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:18:13.191898 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 512, file segment info: File segment: [0, 33554431], key: d2cbb1399a9b5e56b65023f905146a41, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:18:13.193958 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:13.237220 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:13.247592 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:13.310501 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_736_736_0 (state Temporary) 2024.10.22 01:18:13.362864 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_736_736_0 to 202410_749_749_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:13.392563 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_743_351 to 202410_744_744_0 2024.10.22 01:18:13.408275 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46599 2024.10.22 01:18:13.415174 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.68 MiB using storage policy from min volume index 0 2024.10.22 01:18:13.419329 [ 1249 ] {} DiskLocal: Reserved 1.68 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:13.463889 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_744_352} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_743_351 to 202410_744_744_0 into Compact with storage Full 2024.10.22 01:18:13.489471 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_744_352} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:13.508712 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_744_352} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_743_351, total 46943 rows starting from the beginning of the part 2024.10.22 01:18:13.534046 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_744_352} MergeTreeSequentialSource: Reading 2 marks from part 202410_744_744_0, total 67 rows starting from the beginning of the part 2024.10.22 01:18:13.926381 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 657235 2024.10.22 01:18:14.788617 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(710f2b0363919a14876034c50667f74f) : [0, 165]: Setting download as failed: File segment: [0, 165], key: 710f2b0363919a14876034c50667f74f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:14.790495 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(710f2b0363919a14876034c50667f74f): No space left in cache to reserve 166 bytes, will continue without cache download 2024.10.22 01:18:15.010271 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:15.017122 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:15.062428 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 235 entries to flush up to offset 46055 2024.10.22 01:18:15.234566 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_416_416_0 (state Temporary) 2024.10.22 01:18:15.291248 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_416_416_0 to 202410_423_423_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:15.347311 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 657235 2024.10.22 01:18:15.354640 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:15.359836 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:15.376528 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_744_352/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:18:15.425480 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_427_427_0 (state Temporary) 2024.10.22 01:18:15.635788 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_427_427_0 to 202410_433_433_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:15.665099 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(5cb15f6e8404b0b093fa5cfcb9e8cf4e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5cb15f6e8404b0b093fa5cfcb9e8cf4e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:18:15.667467 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 975558, file segment info: File segment: [0, 33554431], key: 5cb15f6e8404b0b093fa5cfcb9e8cf4e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:18:15.671642 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:15.671486 [ 1260 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_427_205 to 202410_428_428_0 2024.10.22 01:18:15.693757 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 46055 2024.10.22 01:18:15.705210 [ 1260 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.29 MiB using storage policy from min volume index 0 2024.10.22 01:18:15.711760 [ 1260 ] {} DiskLocal: Reserved 2.29 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:15.731823 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_428_206} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_427_205 to 202410_428_428_0 into Wide with storage Full 2024.10.22 01:18:15.879587 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_428_206} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:15.951268 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_428_206} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_427_205, total 44738 rows starting from the beginning of the part 2024.10.22 01:18:15.964468 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_744_352} MergeTask::MergeProjectionsStage: Merge sorted 47010 rows, containing 13 columns (13 merged, 0 gathered) in 2.509704853 sec., 18731.28624818418 rows/sec., 2.99 MiB/sec. 2024.10.22 01:18:15.996439 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 368 2024.10.22 01:18:16.134622 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_369_147, 202410_370_370_0] 2024.10.22 01:18:16.142809 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_369_147, 202410_370_370_0] 2024.10.22 01:18:16.201514 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_369_147, 202410_370_370_0] 2024.10.22 01:18:16.247691 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_428_206} MergeTreeSequentialSource: Reading 2 marks from part 202410_428_428_0, total 236 rows starting from the beginning of the part 2024.10.22 01:18:16.261668 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_744_352} MergedBlockOutputStream: filled checksums 202410_1_744_352 (state Temporary) 2024.10.22 01:18:16.307531 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:16.311855 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:16.375358 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_350_350_0 (state Temporary) 2024.10.22 01:18:16.383715 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bbdc3f1803f69b814dd38abcbf69b78a) : [0, 144809]: Setting download as failed: File segment: [0, 144809], key: bbdc3f1803f69b814dd38abcbf69b78a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:16.384997 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bbdc3f1803f69b814dd38abcbf69b78a): No space left in cache to reserve 144810 bytes, will continue without cache download 2024.10.22 01:18:16.416371 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bbdc3f1803f69b814dd38abcbf69b78a, offset: 0, size: 0 2024.10.22 01:18:16.419367 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bbdc3f1803f69b814dd38abcbf69b78a for removal 2024.10.22 01:18:16.423938 [ 1476 ] {} CacheMetadata: Key bbdc3f1803f69b814dd38abcbf69b78a is removed from metadata 2024.10.22 01:18:16.426172 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 710f2b0363919a14876034c50667f74f, offset: 0, size: 0 2024.10.22 01:18:16.430088 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 710f2b0363919a14876034c50667f74f for removal 2024.10.22 01:18:16.434251 [ 1476 ] {} CacheMetadata: Key 710f2b0363919a14876034c50667f74f is removed from metadata 2024.10.22 01:18:16.482806 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_744_352} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_744_352 to 202410_1_744_352 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:16.486422 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_744_352} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_743_351, 202410_744_744_0] -> 202410_1_744_352 2024.10.22 01:18:16.512150 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.21 MiB. 2024.10.22 01:18:16.543934 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_350_350_0 to 202410_360_360_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:16.744325 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 368 2024.10.22 01:18:16.955920 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(590a8acd55ca9ed4df37153b0cd27093) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 590a8acd55ca9ed4df37153b0cd27093, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:16.963049 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 183, file segment info: File segment: [0, 33554431], key: 590a8acd55ca9ed4df37153b0cd27093, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:16.966227 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:17.425285 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 46667 2024.10.22 01:18:17.540844 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.96 GiB, free memory in arenas 10.66 MiB, will set to 1.48 GiB (RSS), difference: -49.13 MiB 2024.10.22 01:18:17.628148 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:17.631327 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:17.788946 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_737_737_0 (state Temporary) 2024.10.22 01:18:17.990164 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(c13ed6c35c5a3301d4de7a26fc92d685) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c13ed6c35c5a3301d4de7a26fc92d685, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:18:17.992963 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 818, file segment info: File segment: [0, 33554431], key: c13ed6c35c5a3301d4de7a26fc92d685, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:18:17.995704 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:18.167947 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:18:18.168853 [ 1692 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xvm/dikafbiabwdvtwvlpcmrngbxbvyqo, size 975558 2024.10.22 01:18:18.185739 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 6649 2024.10.22 01:18:18.291717 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:18:18.299575 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_737_737_0 to 202410_750_750_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:18.314559 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:18:18.369111 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_744_352 to 202410_745_745_0 2024.10.22 01:18:18.465001 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:18:18.467798 [ 1211 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:18:18.479297 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.68 MiB using storage policy from min volume index 0 2024.10.22 01:18:18.482801 [ 1179 ] {} DiskLocal: Reserved 1.68 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:18:18.504833 [ 1211 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:18:18.523530 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_745_353} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_744_352 to 202410_745_745_0 into Compact with storage Full 2024.10.22 01:18:18.540456 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:18:18.568102 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46667 2024.10.22 01:18:18.575008 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:18:18.583353 [ 1186 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 215 ms. 2024.10.22 01:18:18.603014 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:18.620239 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:18.656188 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:18:18.672241 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(15549e577f7f8fd1ea2b292ec154ba39) : [0, 163]: Setting download as failed: File segment: [0, 163], key: 15549e577f7f8fd1ea2b292ec154ba39, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:18.675611 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(15549e577f7f8fd1ea2b292ec154ba39): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 01:18:18.708932 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mil/fcyzlpnrkqysvmfumfxgymvborikd (LogSeriesLimiter: on interval from 2024-10-22 01:17:55 to 2024-10-22 01:18:18 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:18:18.773453 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_434_434_0 (state Temporary) 2024.10.22 01:18:18.805666 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_745_353} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:18.831578 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_434_434_0 to 202410_447_447_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:18.846543 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_745_353} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_744_352, total 47010 rows starting from the beginning of the part 2024.10.22 01:18:18.918407 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6649 2024.10.22 01:18:19.255635 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_745_353} MergeTreeSequentialSource: Reading 2 marks from part 202410_745_745_0, total 91 rows starting from the beginning of the part 2024.10.22 01:18:19.291378 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_418_85/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:18:19.328700 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_428_206} MergeTask::MergeProjectionsStage: Merge sorted 44974 rows, containing 13 columns (13 merged, 0 gathered) in 3.604923229 sec., 12475.716441949227 rows/sec., 5.40 MiB/sec. 2024.10.22 01:18:19.389683 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_89_18/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:18:19.428149 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_428_206} MergedBlockOutputStream: filled checksums 202410_1_428_206 (state Temporary) 2024.10.22 01:18:19.560622 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ed3f7402827d7c5de2677605e00b99cd) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ed3f7402827d7c5de2677605e00b99cd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:19.670568 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 650, file segment info: File segment: [0, 33554431], key: ed3f7402827d7c5de2677605e00b99cd, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:19.678713 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:19.966908 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.306572935 sec. 2024.10.22 01:18:20.069678 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:18:20.071425 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:18:20.274973 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:20.277303 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:20.293667 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_428_206} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_428_206 to 202410_1_428_206 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:20.297527 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_428_206} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_427_205, 202410_428_428_0] -> 202410_1_428_206 2024.10.22 01:18:20.313051 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_644_267, 202410_645_645_0] 2024.10.22 01:18:20.314764 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_644_267, 202410_645_645_0] 2024.10.22 01:18:20.327320 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3e258c9666c209073db2cb03ab1dcc68) : [0, 139907]: Setting download as failed: File segment: [0, 139907], key: 3e258c9666c209073db2cb03ab1dcc68, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:20.331877 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3e258c9666c209073db2cb03ab1dcc68): No space left in cache to reserve 139908 bytes, will continue without cache download 2024.10.22 01:18:20.431288 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_644_267, 202410_645_645_0] 2024.10.22 01:18:20.470913 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3e258c9666c209073db2cb03ab1dcc68, offset: 0, size: 0 2024.10.22 01:18:20.472525 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3e258c9666c209073db2cb03ab1dcc68 for removal 2024.10.22 01:18:20.476993 [ 1476 ] {} CacheMetadata: Key 3e258c9666c209073db2cb03ab1dcc68 is removed from metadata 2024.10.22 01:18:20.479110 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 15549e577f7f8fd1ea2b292ec154ba39, offset: 0, size: 0 2024.10.22 01:18:20.480797 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 15549e577f7f8fd1ea2b292ec154ba39 for removal 2024.10.22 01:18:20.483845 [ 1476 ] {} CacheMetadata: Key 15549e577f7f8fd1ea2b292ec154ba39 is removed from metadata 2024.10.22 01:18:20.530055 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.09 MiB. 2024.10.22 01:18:20.635854 [ 1193 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 282 ms. 2024.10.22 01:18:21.168270 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7562a0cb9668413c597572a08bad6683) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7562a0cb9668413c597572a08bad6683, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:21.196276 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 34299, file segment info: File segment: [0, 33554431], key: 7562a0cb9668413c597572a08bad6683, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:21.204714 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:21.381094 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 10.70 MiB, will set to 1.48 GiB (RSS), difference: 46.24 MiB 2024.10.22 01:18:22.345275 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_745_353} MergeTask::MergeProjectionsStage: Merge sorted 47101 rows, containing 13 columns (13 merged, 0 gathered) in 3.845826757 sec., 12247.301549470185 rows/sec., 1.96 MiB/sec. 2024.10.22 01:18:22.354465 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 658005 2024.10.22 01:18:22.445092 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_745_353} MergedBlockOutputStream: filled checksums 202410_1_745_353 (state Temporary) 2024.10.22 01:18:22.512175 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_745_353} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_745_353 to 202410_1_745_353 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:22.519534 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_745_353} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_744_352, 202410_745_745_0] -> 202410_1_745_353 2024.10.22 01:18:22.556680 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.22 MiB. 2024.10.22 01:18:22.560336 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(aef80f71d4a855b4832a348ed6aeb68b) : [0, 196]: Setting download as failed: File segment: [0, 196], key: aef80f71d4a855b4832a348ed6aeb68b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:22.561666 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(aef80f71d4a855b4832a348ed6aeb68b): No space left in cache to reserve 197 bytes, will continue without cache download 2024.10.22 01:18:22.576088 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 46743 2024.10.22 01:18:23.076490 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:23.082983 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:23.167456 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:23.170697 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:23.244243 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_417_417_0 (state Temporary) 2024.10.22 01:18:23.255172 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 205 entries to flush up to offset 46260 2024.10.22 01:18:23.399832 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_88_88_0 (state Temporary) 2024.10.22 01:18:23.404922 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c0df5e84f8e88f79ff3e63d84a8e8d08) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c0df5e84f8e88f79ff3e63d84a8e8d08, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:23.415935 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_417_417_0 to 202410_424_424_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:23.419637 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 737, file segment info: File segment: [0, 33554431], key: c0df5e84f8e88f79ff3e63d84a8e8d08, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:23.427634 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:23.469512 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 658005 2024.10.22 01:18:23.472188 [ 1723 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kph/yrnywuhxpjgyursowlqkzieuxefzf, size 34299 2024.10.22 01:18:23.462082 [ 1213 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_419_419_0 to 202410_424_424_0 2024.10.22 01:18:23.484462 [ 1213 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:23.487736 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:23.570498 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_738_738_0 (state Temporary) 2024.10.22 01:18:23.574861 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_419_419_0 to 202410_424_424_0 into Compact with storage Full 2024.10.22 01:18:23.587786 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:23.594693 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_419_419_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:18:23.716291 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_420_420_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:18:23.735340 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_421_421_0, total 1157 rows starting from the beginning of the part 2024.10.22 01:18:23.759753 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_422_422_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:18:23.783056 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_423_423_0, total 770 rows starting from the beginning of the part 2024.10.22 01:18:23.810747 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_424_424_0, total 770 rows starting from the beginning of the part 2024.10.22 01:18:23.824840 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:23.828243 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:24.010485 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_645_268, 202410_646_646_0] 2024.10.22 01:18:24.011515 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_738_738_0 to 202410_751_751_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:24.013899 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_645_268, 202410_646_646_0] 2024.10.22 01:18:24.026724 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_645_268, 202410_646_646_0] 2024.10.22 01:18:24.071257 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_745_353 to 202410_746_746_0 2024.10.22 01:18:24.085240 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.68 MiB using storage policy from min volume index 0 2024.10.22 01:18:24.088355 [ 1214 ] {} DiskLocal: Reserved 1.68 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:24.132855 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_746_354} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_745_353 to 202410_746_746_0 into Compact with storage Full 2024.10.22 01:18:24.169479 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:18:24.196371 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46743 2024.10.22 01:18:24.249225 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 369 2024.10.22 01:18:24.277541 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_746_354} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:24.283095 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:18:24.315351 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:18:24.374649 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:18:24.387487 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_746_354} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_745_353, total 47101 rows starting from the beginning of the part 2024.10.22 01:18:24.393026 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:18:24.415289 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_746_354} MergeTreeSequentialSource: Reading 2 marks from part 202410_746_746_0, total 74 rows starting from the beginning of the part 2024.10.22 01:18:24.426151 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:18:24.434272 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_428_428_0 (state Temporary) 2024.10.22 01:18:24.506587 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:18:24.530423 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:24.535185 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:24.553465 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:18:24.594072 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_370_148, 202410_371_371_0] 2024.10.22 01:18:24.600758 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_370_148, 202410_371_371_0] 2024.10.22 01:18:24.727725 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_370_148, 202410_371_371_0] 2024.10.22 01:18:24.752209 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3a0b41544b0bff85bfcaf151742963d9) : [0, 186314]: Setting download as failed: File segment: [0, 186314], key: 3a0b41544b0bff85bfcaf151742963d9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:24.754714 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3a0b41544b0bff85bfcaf151742963d9): No space left in cache to reserve 186315 bytes, will continue without cache download 2024.10.22 01:18:24.790233 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:18:24.791105 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_351_351_0 (state Temporary) 2024.10.22 01:18:24.847317 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_428_428_0 to 202410_434_434_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:24.874078 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zyt/xsrqgzxbpwxdivelzovyjiiyghmod (LogSeriesLimiter: on interval from 2024-10-22 01:18:18 to 2024-10-22 01:18:24 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:18:24.887141 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_428_206 to 202410_429_429_0 2024.10.22 01:18:24.892699 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.96 GiB, free memory in arenas 7.81 MiB, will set to 1.47 GiB (RSS), difference: -21.98 MiB 2024.10.22 01:18:24.913121 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3a0b41544b0bff85bfcaf151742963d9, offset: 0, size: 0 2024.10.22 01:18:24.917562 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3a0b41544b0bff85bfcaf151742963d9 for removal 2024.10.22 01:18:24.925197 [ 1476 ] {} CacheMetadata: Key 3a0b41544b0bff85bfcaf151742963d9 is removed from metadata 2024.10.22 01:18:24.927888 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aef80f71d4a855b4832a348ed6aeb68b, offset: 0, size: 0 2024.10.22 01:18:24.930999 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aef80f71d4a855b4832a348ed6aeb68b for removal 2024.10.22 01:18:24.934855 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 46260 2024.10.22 01:18:24.940812 [ 1476 ] {} CacheMetadata: Key aef80f71d4a855b4832a348ed6aeb68b is removed from metadata 2024.10.22 01:18:24.979636 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} MergeTask::MergeProjectionsStage: Merge sorted 6547 rows, containing 4 columns (4 merged, 0 gathered) in 1.472058179 sec., 4447.514434821805 rows/sec., 69.49 KiB/sec. 2024.10.22 01:18:25.045369 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} MergedBlockOutputStream: filled checksums 202410_419_424_1 (state Temporary) 2024.10.22 01:18:25.095699 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.30 MiB using storage policy from min volume index 0 2024.10.22 01:18:25.105280 [ 1219 ] {} DiskLocal: Reserved 2.30 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:25.112088 [ 1219 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 01:18:25.119442 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_429_207} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_428_206 to 202410_429_429_0 into Wide with storage Full 2024.10.22 01:18:25.162762 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_429_207} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:25.171043 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_429_207} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_428_206, total 44974 rows starting from the beginning of the part 2024.10.22 01:18:25.193202 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_419_424_1 to 202410_419_424_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:25.205007 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_419_424_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_419_419_0, 202410_424_424_0] -> 202410_419_424_1 2024.10.22 01:18:25.230990 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:18:25.318425 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_351_351_0 to 202410_361_361_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:25.369385 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:18:25.405750 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 369 2024.10.22 01:18:25.517603 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_429_207} MergeTreeSequentialSource: Reading 2 marks from part 202410_429_429_0, total 235 rows starting from the beginning of the part 2024.10.22 01:18:25.885722 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_746_354/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:18:26.136009 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a87c1c533fdd8cadaddb6003bbe83565) : [0, 169]: Setting download as failed: File segment: [0, 169], key: a87c1c533fdd8cadaddb6003bbe83565, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:26.138393 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a87c1c533fdd8cadaddb6003bbe83565): No space left in cache to reserve 170 bytes, will continue without cache download 2024.10.22 01:18:26.427678 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 6668 2024.10.22 01:18:26.687886 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:26.695429 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:26.861506 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_435_435_0 (state Temporary) 2024.10.22 01:18:26.920490 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_435_435_0 to 202410_448_448_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:26.978717 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6668 2024.10.22 01:18:27.719647 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(222e83f75fd80cf3e1e908604ffeea03) : [0, 75829]: Setting download as failed: File segment: [0, 75829], key: 222e83f75fd80cf3e1e908604ffeea03, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:27.723687 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(222e83f75fd80cf3e1e908604ffeea03): No space left in cache to reserve 75830 bytes, will continue without cache download 2024.10.22 01:18:27.819791 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 222e83f75fd80cf3e1e908604ffeea03, offset: 0, size: 0 2024.10.22 01:18:27.821570 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 222e83f75fd80cf3e1e908604ffeea03 for removal 2024.10.22 01:18:27.831057 [ 1476 ] {} CacheMetadata: Key 222e83f75fd80cf3e1e908604ffeea03 is removed from metadata 2024.10.22 01:18:27.831842 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a87c1c533fdd8cadaddb6003bbe83565, offset: 0, size: 0 2024.10.22 01:18:27.835275 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a87c1c533fdd8cadaddb6003bbe83565 for removal 2024.10.22 01:18:27.841326 [ 1476 ] {} CacheMetadata: Key a87c1c533fdd8cadaddb6003bbe83565 is removed from metadata 2024.10.22 01:18:27.842968 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_746_354} MergeTask::MergeProjectionsStage: Merge sorted 47175 rows, containing 13 columns (13 merged, 0 gathered) in 3.715928715 sec., 12695.345798634353 rows/sec., 2.03 MiB/sec. 2024.10.22 01:18:28.142458 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_646_269, 202410_647_647_0] 2024.10.22 01:18:28.149176 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_646_269, 202410_647_647_0] 2024.10.22 01:18:28.045813 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_746_354} MergedBlockOutputStream: filled checksums 202410_1_746_354 (state Temporary) 2024.10.22 01:18:28.167022 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_646_269, 202410_647_647_0] 2024.10.22 01:18:28.211027 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 107 entries to flush up to offset 46850 2024.10.22 01:18:28.216919 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_746_354} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_746_354 to 202410_1_746_354 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:28.223862 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_746_354} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_745_353, 202410_746_746_0] -> 202410_1_746_354 2024.10.22 01:18:28.267612 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.33 MiB. 2024.10.22 01:18:28.655460 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:28.659503 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:28.666536 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 12.96 MiB, will set to 1.50 GiB (RSS), difference: -13.85 MiB 2024.10.22 01:18:28.814616 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_739_739_0 (state Temporary) 2024.10.22 01:18:28.861649 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c1397f53fe308c5203b449cffa61b646) : [0, 244]: Setting download as failed: File segment: [0, 244], key: c1397f53fe308c5203b449cffa61b646, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:28.864205 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c1397f53fe308c5203b449cffa61b646): No space left in cache to reserve 245 bytes, will continue without cache download 2024.10.22 01:18:28.975124 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_739_739_0 to 202410_752_752_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:29.000984 [ 1229 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_746_354 to 202410_747_747_0 2024.10.22 01:18:29.014441 [ 1229 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.69 MiB using storage policy from min volume index 0 2024.10.22 01:18:29.020455 [ 1229 ] {} DiskLocal: Reserved 1.69 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:29.072914 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_747_355} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_746_354 to 202410_747_747_0 into Compact with storage Full 2024.10.22 01:18:29.097978 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_747_355} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:29.110647 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_747_355} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_746_354, total 47175 rows starting from the beginning of the part 2024.10.22 01:18:29.206969 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46850 2024.10.22 01:18:29.230864 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_747_355} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_747_0, total 91 rows starting from the beginning of the part 2024.10.22 01:18:29.391002 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_429_207} MergeTask::MergeProjectionsStage: Merge sorted 45209 rows, containing 13 columns (13 merged, 0 gathered) in 4.276694037 sec., 10571.015744608432 rows/sec., 4.58 MiB/sec. 2024.10.22 01:18:29.622948 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_429_207} MergedBlockOutputStream: filled checksums 202410_1_429_207 (state Temporary) 2024.10.22 01:18:29.664746 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(236c0a60b4f18ab1ed71ec8328bd9305) : [0, 620770]: Setting download as failed: File segment: [0, 620770], key: 236c0a60b4f18ab1ed71ec8328bd9305, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:29.668605 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(236c0a60b4f18ab1ed71ec8328bd9305): No space left in cache to reserve 620771 bytes, will continue without cache download 2024.10.22 01:18:29.817389 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 236c0a60b4f18ab1ed71ec8328bd9305, offset: 0, size: 0 2024.10.22 01:18:29.820867 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 236c0a60b4f18ab1ed71ec8328bd9305 for removal 2024.10.22 01:18:29.826235 [ 1476 ] {} CacheMetadata: Key 236c0a60b4f18ab1ed71ec8328bd9305 is removed from metadata 2024.10.22 01:18:29.831645 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c1397f53fe308c5203b449cffa61b646, offset: 0, size: 0 2024.10.22 01:18:29.833645 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c1397f53fe308c5203b449cffa61b646 for removal 2024.10.22 01:18:29.841712 [ 1476 ] {} CacheMetadata: Key c1397f53fe308c5203b449cffa61b646 is removed from metadata 2024.10.22 01:18:30.070410 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:18:30.071016 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:18:30.472356 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 659160 2024.10.22 01:18:30.622532 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_418_85/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:18:30.748266 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(1e27bce95887b19d0c7177847aa4cc7e) : [0, 243]: Setting download as failed: File segment: [0, 243], key: 1e27bce95887b19d0c7177847aa4cc7e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:30.751728 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(1e27bce95887b19d0c7177847aa4cc7e): No space left in cache to reserve 244 bytes, will continue without cache download 2024.10.22 01:18:30.967424 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_429_207} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_429_207 to 202410_1_429_207 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:30.970194 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_429_207} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_428_206, 202410_429_429_0] -> 202410_1_429_207 2024.10.22 01:18:31.007128 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.09 MiB. 2024.10.22 01:18:31.241024 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:31.256315 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:31.390212 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.96 GiB, free memory in arenas 14.44 MiB, will set to 1.49 GiB (RSS), difference: 29.11 MiB 2024.10.22 01:18:31.613492 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_418_418_0 (state Temporary) 2024.10.22 01:18:31.668912 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(28254bab3f75adbc2ce1458ce57b138f) : [0, 564786]: Setting download as failed: File segment: [0, 564786], key: 28254bab3f75adbc2ce1458ce57b138f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:31.669780 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(28254bab3f75adbc2ce1458ce57b138f): No space left in cache to reserve 564787 bytes, will continue without cache download 2024.10.22 01:18:31.720334 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 28254bab3f75adbc2ce1458ce57b138f, offset: 0, size: 0 2024.10.22 01:18:31.725267 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 28254bab3f75adbc2ce1458ce57b138f for removal 2024.10.22 01:18:31.728452 [ 1476 ] {} CacheMetadata: Key 28254bab3f75adbc2ce1458ce57b138f is removed from metadata 2024.10.22 01:18:31.736538 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1e27bce95887b19d0c7177847aa4cc7e, offset: 0, size: 0 2024.10.22 01:18:31.744830 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1e27bce95887b19d0c7177847aa4cc7e for removal 2024.10.22 01:18:31.762304 [ 1476 ] {} CacheMetadata: Key 1e27bce95887b19d0c7177847aa4cc7e is removed from metadata 2024.10.22 01:18:31.866155 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_418_418_0 to 202410_425_425_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:31.961012 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 659160 2024.10.22 01:18:31.992251 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_747_355} MergeTask::MergeProjectionsStage: Merge sorted 47266 rows, containing 13 columns (13 merged, 0 gathered) in 2.94761079 sec., 16035.359946555223 rows/sec., 2.56 MiB/sec. 2024.10.22 01:18:32.091038 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_747_355} MergedBlockOutputStream: filled checksums 202410_1_747_355 (state Temporary) 2024.10.22 01:18:32.474222 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 255 entries to flush up to offset 46515 2024.10.22 01:18:32.536438 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_747_355} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_747_355 to 202410_1_747_355 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:32.555894 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_747_355} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_746_354, 202410_747_747_0] -> 202410_1_747_355 2024.10.22 01:18:32.571368 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_418_85} MergeTask::MergeProjectionsStage: Merge sorted 665048 rows, containing 4 columns (4 merged, 0 gathered) in 25.316437187 sec., 26269.415205924095 rows/sec., 384.85 KiB/sec. 2024.10.22 01:18:32.577075 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.37 MiB. 2024.10.22 01:18:32.634961 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_418_85} MergedBlockOutputStream: filled checksums 202410_1_418_85 (state Temporary) 2024.10.22 01:18:32.665554 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_418_85} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_418_85 to 202410_1_418_85 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:32.669155 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_418_85} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_417_84, 202410_418_418_0] -> 202410_1_418_85 2024.10.22 01:18:32.672989 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(88a356d44e9f41e1ce173de2c734547e) : [0, 229]: Setting download as failed: File segment: [0, 229], key: 88a356d44e9f41e1ce173de2c734547e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:32.674567 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(88a356d44e9f41e1ce173de2c734547e): No space left in cache to reserve 230 bytes, will continue without cache download 2024.10.22 01:18:32.703965 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:18:32.776716 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:32.778785 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:32.821240 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_429_429_0 (state Temporary) 2024.10.22 01:18:32.910940 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 370 2024.10.22 01:18:33.096974 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_88_88_0 to 202410_91_91_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:33.165616 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_89_18/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:18:33.182886 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_371_149, 202410_372_372_0] 2024.10.22 01:18:33.183418 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_429_429_0 to 202410_435_435_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:33.184702 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_371_149, 202410_372_372_0] 2024.10.22 01:18:33.205113 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_371_149, 202410_372_372_0] 2024.10.22 01:18:33.216251 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(591e13b9e285c6261df09277e4f91c99) : [0, 10219]: Setting download as failed: File segment: [0, 10219], key: 591e13b9e285c6261df09277e4f91c99, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:33.232008 [ 1249 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_429_207 to 202410_430_430_0 2024.10.22 01:18:33.234573 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 46515 2024.10.22 01:18:33.249015 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 46925 2024.10.22 01:18:33.251422 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(591e13b9e285c6261df09277e4f91c99): No space left in cache to reserve 10220 bytes, will continue without cache download 2024.10.22 01:18:33.256574 [ 1249 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.31 MiB using storage policy from min volume index 0 2024.10.22 01:18:33.263349 [ 1249 ] {} DiskLocal: Reserved 2.31 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:33.269429 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 591e13b9e285c6261df09277e4f91c99, offset: 0, size: 0 2024.10.22 01:18:33.272137 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 591e13b9e285c6261df09277e4f91c99 for removal 2024.10.22 01:18:33.276055 [ 1476 ] {} CacheMetadata: Key 591e13b9e285c6261df09277e4f91c99 is removed from metadata 2024.10.22 01:18:33.278163 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 88a356d44e9f41e1ce173de2c734547e, offset: 0, size: 0 2024.10.22 01:18:33.280927 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 88a356d44e9f41e1ce173de2c734547e for removal 2024.10.22 01:18:33.297674 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_430_208} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_429_207 to 202410_430_430_0 into Wide with storage Full 2024.10.22 01:18:33.317567 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:33.327223 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:33.375344 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_430_208} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:33.381497 [ 1476 ] {} CacheMetadata: Key 88a356d44e9f41e1ce173de2c734547e is removed from metadata 2024.10.22 01:18:33.470202 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.96 GiB, free memory in arenas 11.87 MiB, will set to 1.47 GiB (RSS), difference: 13.24 MiB 2024.10.22 01:18:33.488088 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_430_208} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_429_207, total 45209 rows starting from the beginning of the part 2024.10.22 01:18:33.516933 [ 1246 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:18:33.556473 [ 1246 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:18:33.584616 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:33.587398 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:33.662477 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_740_740_0 (state Temporary) 2024.10.22 01:18:33.804789 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_352_352_0 (state Temporary) 2024.10.22 01:18:33.879048 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_430_208} MergeTreeSequentialSource: Reading 2 marks from part 202410_430_430_0, total 230 rows starting from the beginning of the part 2024.10.22 01:18:33.905169 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b5c321205240bf45ebf5b6880d240d9a) : [0, 199]: Setting download as failed: File segment: [0, 199], key: b5c321205240bf45ebf5b6880d240d9a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:33.907046 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b5c321205240bf45ebf5b6880d240d9a): No space left in cache to reserve 200 bytes, will continue without cache download 2024.10.22 01:18:33.991232 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_352_352_0 to 202410_362_362_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:34.077145 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_740_740_0 to 202410_753_753_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:34.109071 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 46925 2024.10.22 01:18:34.186992 [ 1237 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_747_355 to 202410_748_748_0 2024.10.22 01:18:34.187906 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 370 2024.10.22 01:18:34.192474 [ 1237 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.69 MiB using storage policy from min volume index 0 2024.10.22 01:18:34.196004 [ 1237 ] {} DiskLocal: Reserved 1.69 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:34.256320 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_748_356} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_747_355 to 202410_748_748_0 into Compact with storage Full 2024.10.22 01:18:34.319328 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_748_356} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:34.357640 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_748_356} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_747_355, total 47266 rows starting from the beginning of the part 2024.10.22 01:18:34.456055 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_748_356} MergeTreeSequentialSource: Reading 2 marks from part 202410_748_748_0, total 88 rows starting from the beginning of the part 2024.10.22 01:18:34.484104 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 6687 2024.10.22 01:18:34.595804 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:34.600284 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:34.735508 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_436_436_0 (state Temporary) 2024.10.22 01:18:34.923836 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_436_436_0 to 202410_449_449_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:34.983162 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6687 2024.10.22 01:18:35.278374 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3417ae1fc12799249aba944e2020c34f) : [0, 323898]: Setting download as failed: File segment: [0, 323898], key: 3417ae1fc12799249aba944e2020c34f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:35.279957 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3417ae1fc12799249aba944e2020c34f): No space left in cache to reserve 323899 bytes, will continue without cache download 2024.10.22 01:18:35.352192 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3417ae1fc12799249aba944e2020c34f, offset: 0, size: 0 2024.10.22 01:18:35.327402 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3503 2024.10.22 01:18:35.354589 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3417ae1fc12799249aba944e2020c34f for removal 2024.10.22 01:18:35.360281 [ 1476 ] {} CacheMetadata: Key 3417ae1fc12799249aba944e2020c34f is removed from metadata 2024.10.22 01:18:35.365118 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b5c321205240bf45ebf5b6880d240d9a, offset: 0, size: 0 2024.10.22 01:18:35.368882 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b5c321205240bf45ebf5b6880d240d9a for removal 2024.10.22 01:18:35.371336 [ 1476 ] {} CacheMetadata: Key b5c321205240bf45ebf5b6880d240d9a is removed from metadata 2024.10.22 01:18:35.467460 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.96 GiB, free memory in arenas 12.55 MiB, will set to 1.49 GiB (RSS), difference: -47.90 MiB 2024.10.22 01:18:35.483240 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_647_270, 202410_648_648_0] 2024.10.22 01:18:35.485066 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_647_270, 202410_648_648_0] 2024.10.22 01:18:35.495545 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_647_270, 202410_648_648_0] 2024.10.22 01:18:36.051176 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f11c37f8a57fe1b1e0617cf6323d8eeb) : [0, 241]: Setting download as failed: File segment: [0, 241], key: f11c37f8a57fe1b1e0617cf6323d8eeb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:36.053057 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f11c37f8a57fe1b1e0617cf6323d8eeb): No space left in cache to reserve 242 bytes, will continue without cache download 2024.10.22 01:18:36.343404 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_748_356/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:18:36.835635 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_430_208} MergeTask::MergeProjectionsStage: Merge sorted 45439 rows, containing 13 columns (13 merged, 0 gathered) in 3.543671375 sec., 12822.577262825336 rows/sec., 5.56 MiB/sec. 2024.10.22 01:18:36.966408 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(b7257dddd7c9057502ce16978ace5183) : [0, 510963]: Setting download as failed: File segment: [0, 510963], key: b7257dddd7c9057502ce16978ace5183, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:36.968488 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(b7257dddd7c9057502ce16978ace5183): No space left in cache to reserve 510964 bytes, will continue without cache download 2024.10.22 01:18:37.136268 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b7257dddd7c9057502ce16978ace5183, offset: 0, size: 0 2024.10.22 01:18:37.217636 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_430_208} MergedBlockOutputStream: filled checksums 202410_1_430_208 (state Temporary) 2024.10.22 01:18:37.240557 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b7257dddd7c9057502ce16978ace5183 for removal 2024.10.22 01:18:37.248542 [ 1476 ] {} CacheMetadata: Key b7257dddd7c9057502ce16978ace5183 is removed from metadata 2024.10.22 01:18:37.260890 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f11c37f8a57fe1b1e0617cf6323d8eeb, offset: 0, size: 0 2024.10.22 01:18:37.262918 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f11c37f8a57fe1b1e0617cf6323d8eeb for removal 2024.10.22 01:18:37.266121 [ 1476 ] {} CacheMetadata: Key f11c37f8a57fe1b1e0617cf6323d8eeb is removed from metadata 2024.10.22 01:18:37.569002 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_748_356} MergeTask::MergeProjectionsStage: Merge sorted 47354 rows, containing 13 columns (13 merged, 0 gathered) in 3.331123607 sec., 14215.623791471033 rows/sec., 2.27 MiB/sec. 2024.10.22 01:18:37.638220 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_748_356} MergedBlockOutputStream: filled checksums 202410_1_748_356 (state Temporary) 2024.10.22 01:18:37.676822 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 10.70 MiB, will set to 1.48 GiB (RSS), difference: 44.56 MiB 2024.10.22 01:18:37.875348 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_430_208} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_430_208 to 202410_1_430_208 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:37.879716 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_430_208} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_429_207, 202410_430_430_0] -> 202410_1_430_208 2024.10.22 01:18:37.891246 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.09 MiB. 2024.10.22 01:18:38.000299 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bca7371a68896739a16c9145d4d0971f) : [0, 244]: Setting download as failed: File segment: [0, 244], key: bca7371a68896739a16c9145d4d0971f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:38.004528 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bca7371a68896739a16c9145d4d0971f): No space left in cache to reserve 245 bytes, will continue without cache download 2024.10.22 01:18:38.012102 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_748_356} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_748_356 to 202410_1_748_356 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:38.017312 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_748_356} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_747_355, 202410_748_748_0] -> 202410_1_748_356 2024.10.22 01:18:38.052257 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.37 MiB. 2024.10.22 01:18:38.134152 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 47007 2024.10.22 01:18:38.488190 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:38.491699 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:38.680085 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_361_72, 202410_362_362_0, 202410_363_363_0, 202410_364_364_0, 202410_365_365_0, 202410_366_366_0] 2024.10.22 01:18:38.681323 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_361_72, 202410_362_362_0, 202410_363_363_0, 202410_364_364_0, 202410_365_365_0, 202410_366_366_0] 2024.10.22 01:18:38.788255 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_741_741_0 (state Temporary) 2024.10.22 01:18:38.805489 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_361_72, 202410_362_362_0, 202410_363_363_0, 202410_364_364_0, 202410_365_365_0, 202410_366_366_0] 2024.10.22 01:18:38.945717 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_741_741_0 to 202410_754_754_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:38.966506 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 660700 2024.10.22 01:18:38.968260 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_748_356 to 202410_749_749_0 2024.10.22 01:18:38.976703 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.69 MiB using storage policy from min volume index 0 2024.10.22 01:18:38.979548 [ 1180 ] {} DiskLocal: Reserved 1.69 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:38.981455 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47007 2024.10.22 01:18:39.033305 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_749_357} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_748_356 to 202410_749_749_0 into Compact with storage Full 2024.10.22 01:18:39.149631 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(916d3b460e02bdf79f7d8c878929a9c1) : [0, 618282]: Setting download as failed: File segment: [0, 618282], key: 916d3b460e02bdf79f7d8c878929a9c1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:39.152397 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(916d3b460e02bdf79f7d8c878929a9c1): No space left in cache to reserve 618283 bytes, will continue without cache download 2024.10.22 01:18:39.193135 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 916d3b460e02bdf79f7d8c878929a9c1, offset: 0, size: 0 2024.10.22 01:18:39.195141 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 916d3b460e02bdf79f7d8c878929a9c1 for removal 2024.10.22 01:18:39.202474 [ 1476 ] {} CacheMetadata: Key 916d3b460e02bdf79f7d8c878929a9c1 is removed from metadata 2024.10.22 01:18:39.209578 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bca7371a68896739a16c9145d4d0971f, offset: 0, size: 0 2024.10.22 01:18:39.212035 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bca7371a68896739a16c9145d4d0971f for removal 2024.10.22 01:18:39.214900 [ 1476 ] {} CacheMetadata: Key bca7371a68896739a16c9145d4d0971f is removed from metadata 2024.10.22 01:18:39.375536 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_749_357} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:39.383152 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_749_357} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_748_356, total 47354 rows starting from the beginning of the part 2024.10.22 01:18:39.408818 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_749_357} MergeTreeSequentialSource: Reading 2 marks from part 202410_749_749_0, total 76 rows starting from the beginning of the part 2024.10.22 01:18:39.856660 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:39.859752 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:39.955852 [ 1187 ] {} DatabaseCatalog: Execution took 316 ms. 2024.10.22 01:18:40.071579 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:18:40.072978 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:18:40.076559 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_419_419_0 (state Temporary) 2024.10.22 01:18:40.278550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(cb0ae42e82ad97117d944a32453c3602) : [0, 247]: Setting download as failed: File segment: [0, 247], key: cb0ae42e82ad97117d944a32453c3602, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:40.279404 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(cb0ae42e82ad97117d944a32453c3602): No space left in cache to reserve 248 bytes, will continue without cache download 2024.10.22 01:18:40.376064 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_419_419_0 to 202410_426_426_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:40.484059 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_648_271, 202410_649_649_0] 2024.10.22 01:18:40.490865 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_648_271, 202410_649_649_0] 2024.10.22 01:18:40.507192 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 660700 2024.10.22 01:18:40.508929 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_648_271, 202410_649_649_0] 2024.10.22 01:18:40.810126 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 232 entries to flush up to offset 46747 2024.10.22 01:18:41.323262 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:41.331980 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:41.446550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5f82098cf2810464fdcc711d1674b7a4) : [0, 722167]: Setting download as failed: File segment: [0, 722167], key: 5f82098cf2810464fdcc711d1674b7a4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:41.457799 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5f82098cf2810464fdcc711d1674b7a4): No space left in cache to reserve 722168 bytes, will continue without cache download 2024.10.22 01:18:41.551868 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_430_430_0 (state Temporary) 2024.10.22 01:18:41.609110 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5f82098cf2810464fdcc711d1674b7a4, offset: 0, size: 0 2024.10.22 01:18:41.615253 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5f82098cf2810464fdcc711d1674b7a4 for removal 2024.10.22 01:18:41.629390 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cb0ae42e82ad97117d944a32453c3602, offset: 0, size: 0 2024.10.22 01:18:41.634127 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cb0ae42e82ad97117d944a32453c3602 for removal 2024.10.22 01:18:41.636638 [ 1476 ] {} CacheMetadata: Key 5f82098cf2810464fdcc711d1674b7a4 is removed from metadata 2024.10.22 01:18:41.665753 [ 1476 ] {} CacheMetadata: Key cb0ae42e82ad97117d944a32453c3602 is removed from metadata 2024.10.22 01:18:41.690092 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 371 2024.10.22 01:18:41.781534 [ 1202 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 329 ms. 2024.10.22 01:18:41.801244 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_372_150, 202410_373_373_0] 2024.10.22 01:18:41.808494 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_372_150, 202410_373_373_0] 2024.10.22 01:18:41.904369 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_372_150, 202410_373_373_0] 2024.10.22 01:18:41.993501 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:42.000010 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:42.080993 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_353_353_0 (state Temporary) 2024.10.22 01:18:42.162628 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_430_430_0 to 202410_436_436_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:42.177477 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_749_357} MergeTask::MergeProjectionsStage: Merge sorted 47430 rows, containing 13 columns (13 merged, 0 gathered) in 3.15861783 sec., 15016.06162971606 rows/sec., 2.40 MiB/sec. 2024.10.22 01:18:42.190452 [ 1264 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_430_208 to 202410_431_431_0 2024.10.22 01:18:42.194417 [ 1264 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.33 MiB using storage policy from min volume index 0 2024.10.22 01:18:42.199241 [ 1264 ] {} DiskLocal: Reserved 2.33 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:42.203313 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 46747 2024.10.22 01:18:42.237767 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_431_209} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_430_208 to 202410_431_431_0 into Wide with storage Full 2024.10.22 01:18:42.274994 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_353_353_0 to 202410_363_363_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:42.380866 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_749_357} MergedBlockOutputStream: filled checksums 202410_1_749_357 (state Temporary) 2024.10.22 01:18:42.391797 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_431_209} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:42.398390 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_431_209} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_430_208, total 45439 rows starting from the beginning of the part 2024.10.22 01:18:42.480406 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 371 2024.10.22 01:18:42.486405 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 6698 2024.10.22 01:18:42.555631 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_431_209} MergeTreeSequentialSource: Reading 2 marks from part 202410_431_431_0, total 260 rows starting from the beginning of the part 2024.10.22 01:18:42.586257 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 12.40 MiB, will set to 1.48 GiB (RSS), difference: 5.79 MiB 2024.10.22 01:18:42.660247 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:42.662351 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:42.696338 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_437_437_0 (state Temporary) 2024.10.22 01:18:42.852921 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(2114c2c0e6f3bf9dbfacc0de7c8780be) : [0, 253]: Setting download as failed: File segment: [0, 253], key: 2114c2c0e6f3bf9dbfacc0de7c8780be, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:42.856240 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(2114c2c0e6f3bf9dbfacc0de7c8780be): No space left in cache to reserve 254 bytes, will continue without cache download 2024.10.22 01:18:42.859921 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_749_357} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_749_357 to 202410_1_749_357 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:42.864093 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_749_357} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_748_356, 202410_749_749_0] -> 202410_1_749_357 2024.10.22 01:18:42.888474 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.37 MiB. 2024.10.22 01:18:42.999181 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 47088 2024.10.22 01:18:43.014675 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 3542 2024.10.22 01:18:43.074625 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_437_437_0 to 202410_450_450_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:43.155055 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6698 2024.10.22 01:18:43.245083 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:43.458955 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:43.631714 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_649_272, 202410_650_650_0] 2024.10.22 01:18:43.639479 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_649_272, 202410_650_650_0] 2024.10.22 01:18:43.662492 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_649_272, 202410_650_650_0] 2024.10.22 01:18:43.846290 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_742_742_0 (state Temporary) 2024.10.22 01:18:43.884445 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_742_742_0 to 202410_755_755_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:43.945268 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(73bf05113ed9f326966091d918719dde) : [0, 1073681]: Setting download as failed: File segment: [0, 1073681], key: 73bf05113ed9f326966091d918719dde, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:43.951809 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(73bf05113ed9f326966091d918719dde): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:18:44.018494 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_749_357 to 202410_750_750_0 2024.10.22 01:18:44.028743 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.70 MiB using storage policy from min volume index 0 2024.10.22 01:18:44.032330 [ 1218 ] {} DiskLocal: Reserved 1.70 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:44.048770 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_750_358} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_749_357 to 202410_750_750_0 into Compact with storage Full 2024.10.22 01:18:44.051496 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47088 2024.10.22 01:18:44.114799 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_750_358} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:44.125450 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 73bf05113ed9f326966091d918719dde, offset: 0, size: 0 2024.10.22 01:18:44.127540 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 73bf05113ed9f326966091d918719dde for removal 2024.10.22 01:18:44.128492 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_750_358} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_749_357, total 47430 rows starting from the beginning of the part 2024.10.22 01:18:44.130567 [ 1476 ] {} CacheMetadata: Key 73bf05113ed9f326966091d918719dde is removed from metadata 2024.10.22 01:18:44.135647 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2114c2c0e6f3bf9dbfacc0de7c8780be, offset: 0, size: 0 2024.10.22 01:18:44.138583 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2114c2c0e6f3bf9dbfacc0de7c8780be for removal 2024.10.22 01:18:44.143709 [ 1476 ] {} CacheMetadata: Key 2114c2c0e6f3bf9dbfacc0de7c8780be is removed from metadata 2024.10.22 01:18:44.184046 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_750_358} MergeTreeSequentialSource: Reading 2 marks from part 202410_750_750_0, total 68 rows starting from the beginning of the part 2024.10.22 01:18:44.344237 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 15.56 MiB, will set to 1.47 GiB (RSS), difference: -40.88 MiB 2024.10.22 01:18:44.635860 [ 1222 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 200 ms. 2024.10.22 01:18:44.944273 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(031dadcd4767bfb1077831e991664d28) : [0, 253]: Setting download as failed: File segment: [0, 253], key: 031dadcd4767bfb1077831e991664d28, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:44.947970 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(031dadcd4767bfb1077831e991664d28): No space left in cache to reserve 254 bytes, will continue without cache download 2024.10.22 01:18:45.283841 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_310_64, 202410_311_311_0, 202410_312_312_0, 202410_313_313_0, 202410_314_314_0, 202410_315_315_0] 2024.10.22 01:18:45.286305 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_310_64, 202410_311_311_0, 202410_312_312_0, 202410_313_313_0, 202410_314_314_0, 202410_315_315_0] 2024.10.22 01:18:45.421347 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_310_64, 202410_311_311_0, 202410_312_312_0, 202410_313_313_0, 202410_314_314_0, 202410_315_315_0] 2024.10.22 01:18:45.807772 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c2a2428a2f79041dff54aae80c712112) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c2a2428a2f79041dff54aae80c712112, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:18:45.812947 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 829654, file segment info: File segment: [0, 33554431], key: c2a2428a2f79041dff54aae80c712112, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:18:45.815063 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:46.372915 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_386_77, 202410_387_387_0, 202410_388_388_0, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0] 2024.10.22 01:18:46.374492 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4ca50656f35bc9c75183604befd71ba6) : [0, 1077603]: Setting download as failed: File segment: [0, 1077603], key: 4ca50656f35bc9c75183604befd71ba6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:46.375103 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_386_77, 202410_387_387_0, 202410_388_388_0, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0] 2024.10.22 01:18:46.375916 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4ca50656f35bc9c75183604befd71ba6): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:18:46.413647 [ 1161 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 221 ms. 2024.10.22 01:18:46.502741 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_386_77, 202410_387_387_0, 202410_388_388_0, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0] 2024.10.22 01:18:46.594486 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_431_209} MergeTask::MergeProjectionsStage: Merge sorted 45699 rows, containing 13 columns (13 merged, 0 gathered) in 4.36726132 sec., 10463.9948589108 rows/sec., 4.53 MiB/sec. 2024.10.22 01:18:46.662494 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_431_209} MergedBlockOutputStream: filled checksums 202410_1_431_209 (state Temporary) 2024.10.22 01:18:46.853013 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4ca50656f35bc9c75183604befd71ba6, offset: 0, size: 0 2024.10.22 01:18:46.856519 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4ca50656f35bc9c75183604befd71ba6 for removal 2024.10.22 01:18:46.862590 [ 1476 ] {} CacheMetadata: Key 4ca50656f35bc9c75183604befd71ba6 is removed from metadata 2024.10.22 01:18:46.865065 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 031dadcd4767bfb1077831e991664d28, offset: 0, size: 0 2024.10.22 01:18:46.866995 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 031dadcd4767bfb1077831e991664d28 for removal 2024.10.22 01:18:46.967777 [ 1476 ] {} CacheMetadata: Key 031dadcd4767bfb1077831e991664d28 is removed from metadata 2024.10.22 01:18:47.068171 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(e9cb2f80cd4198ace31c4e139000a97a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e9cb2f80cd4198ace31c4e139000a97a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:47.070032 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 29936, file segment info: File segment: [0, 33554431], key: e9cb2f80cd4198ace31c4e139000a97a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:47.073833 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:47.270518 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_750_358} MergeTask::MergeProjectionsStage: Merge sorted 47498 rows, containing 13 columns (13 merged, 0 gathered) in 3.231578218 sec., 14698.081493257545 rows/sec., 2.35 MiB/sec. 2024.10.22 01:18:47.331724 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_750_358} MergedBlockOutputStream: filled checksums 202410_1_750_358 (state Temporary) 2024.10.22 01:18:47.373397 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_431_209} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_431_209 to 202410_1_431_209 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:47.376005 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_431_209} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_430_208, 202410_431_431_0] -> 202410_1_431_209 2024.10.22 01:18:47.387191 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.11 MiB. 2024.10.22 01:18:47.416028 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_750_358/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:18:47.530364 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 946 entries to flush up to offset 661646 2024.10.22 01:18:47.589733 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_750_358} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_750_358 to 202410_1_750_358 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:47.593680 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_750_358} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_749_357, 202410_750_750_0] -> 202410_1_750_358 2024.10.22 01:18:47.632640 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:18:47.697287 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(636478c81f03e131df4a42f58ee60777) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 636478c81f03e131df4a42f58ee60777, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:18:47.716462 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 587, file segment info: File segment: [0, 33554431], key: 636478c81f03e131df4a42f58ee60777, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:18:47.823873 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:47.954847 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:18:47.961189 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:18:48.067452 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 47157 2024.10.22 01:18:48.069222 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:18:48.075563 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:18:48.084009 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:18:48.097324 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:18:48.123771 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:18:48.151570 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:18:48.169247 [ 1699 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zyt/xsrqgzxbpwxdivelzovyjiiyghmod, size 829654 2024.10.22 01:18:48.331639 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:18:48.333833 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d9726daea7544760701343fb31491b27) : [0, 246]: Setting download as failed: File segment: [0, 246], key: d9726daea7544760701343fb31491b27, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:48.336112 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d9726daea7544760701343fb31491b27): No space left in cache to reserve 247 bytes, will continue without cache download 2024.10.22 01:18:48.374759 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_89_18/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:18:48.397371 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/iyr/ujfqvgdmvcrfxzxfvlztfemmvpsje (LogSeriesLimiter: on interval from 2024-10-22 01:18:24 to 2024-10-22 01:18:48 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:18:48.425043 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:48.429171 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:48.434539 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.96 GiB, free memory in arenas 10.26 MiB, will set to 1.46 GiB (RSS), difference: 42.65 MiB 2024.10.22 01:18:48.561123 [ 1248 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:18:48.577464 [ 1248 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:18:48.693548 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:48.699597 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:48.711287 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_743_743_0 (state Temporary) 2024.10.22 01:18:48.787758 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_420_420_0 (state Temporary) 2024.10.22 01:18:48.847268 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_420_420_0 to 202410_427_427_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:48.867434 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_650_273, 202410_651_651_0] 2024.10.22 01:18:48.867494 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_743_743_0 to 202410_756_756_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:48.868649 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_650_273, 202410_651_651_0] 2024.10.22 01:18:48.872603 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 661646 2024.10.22 01:18:48.872972 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(12e1d483ab5a22e71e1a0d085d313fb8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 12e1d483ab5a22e71e1a0d085d313fb8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:48.876405 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 474, file segment info: File segment: [0, 33554431], key: 12e1d483ab5a22e71e1a0d085d313fb8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:48.876816 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_650_273, 202410_651_651_0] 2024.10.22 01:18:48.880534 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:48.882803 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_750_358 to 202410_751_751_0 2024.10.22 01:18:48.888022 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.70 MiB using storage policy from min volume index 0 2024.10.22 01:18:48.896438 [ 1269 ] {} DiskLocal: Reserved 1.70 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:48.942857 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_751_359} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_750_358 to 202410_751_751_0 into Compact with storage Full 2024.10.22 01:18:49.116133 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47157 2024.10.22 01:18:49.175109 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_751_359} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:49.181008 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_751_359} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_750_358, total 47498 rows starting from the beginning of the part 2024.10.22 01:18:49.204196 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_751_359} MergeTreeSequentialSource: Reading 2 marks from part 202410_751_751_0, total 76 rows starting from the beginning of the part 2024.10.22 01:18:49.316821 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_373_151, 202410_374_374_0] 2024.10.22 01:18:49.321242 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_373_151, 202410_374_374_0] 2024.10.22 01:18:49.340802 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_373_151, 202410_374_374_0] 2024.10.22 01:18:49.470787 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(4349a7926c2b796f8651a7338b1fb0cf) : [0, 687705]: Setting download as failed: File segment: [0, 687705], key: 4349a7926c2b796f8651a7338b1fb0cf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:49.472612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(4349a7926c2b796f8651a7338b1fb0cf): No space left in cache to reserve 687706 bytes, will continue without cache download 2024.10.22 01:18:49.748116 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4349a7926c2b796f8651a7338b1fb0cf, offset: 0, size: 0 2024.10.22 01:18:49.751899 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4349a7926c2b796f8651a7338b1fb0cf for removal 2024.10.22 01:18:49.754607 [ 1476 ] {} CacheMetadata: Key 4349a7926c2b796f8651a7338b1fb0cf is removed from metadata 2024.10.22 01:18:49.767039 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d9726daea7544760701343fb31491b27, offset: 0, size: 0 2024.10.22 01:18:49.768492 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d9726daea7544760701343fb31491b27 for removal 2024.10.22 01:18:49.772489 [ 1476 ] {} CacheMetadata: Key d9726daea7544760701343fb31491b27 is removed from metadata 2024.10.22 01:18:49.958034 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 207 entries to flush up to offset 46954 2024.10.22 01:18:50.035800 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(f7d71cba3bed99a2348cec1b81a952db) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f7d71cba3bed99a2348cec1b81a952db, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:50.042509 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 975558, file segment info: File segment: [0, 33554431], key: f7d71cba3bed99a2348cec1b81a952db, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:50.046246 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:50.071156 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:18:50.083535 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:18:50.571441 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:50.574691 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:50.663028 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 6711 2024.10.22 01:18:50.787914 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_431_431_0 (state Temporary) 2024.10.22 01:18:50.841095 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:50.853449 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:50.876024 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(39a5ed7f937cc6d7f979cbc2edfa2dc3) : [0, 242]: Setting download as failed: File segment: [0, 242], key: 39a5ed7f937cc6d7f979cbc2edfa2dc3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:50.877226 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(39a5ed7f937cc6d7f979cbc2edfa2dc3): No space left in cache to reserve 243 bytes, will continue without cache download 2024.10.22 01:18:50.891938 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_438_438_0 (state Temporary) 2024.10.22 01:18:50.954088 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_438_438_0 to 202410_451_451_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:50.967318 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_446_89 to 202410_451_451_0 2024.10.22 01:18:50.972851 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:50.977018 [ 1228 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:50.978970 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6711 2024.10.22 01:18:50.984777 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_431_431_0 to 202410_437_437_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:51.024529 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_446_89 to 202410_451_451_0 into Compact with storage Full 2024.10.22 01:18:51.040967 [ 1151 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_431_209 to 202410_432_432_0 2024.10.22 01:18:51.044402 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 46954 2024.10.22 01:18:51.050747 [ 1151 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.34 MiB using storage policy from min volume index 0 2024.10.22 01:18:51.058348 [ 1151 ] {} DiskLocal: Reserved 2.34 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:51.081320 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_432_210} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_431_209 to 202410_432_432_0 into Wide with storage Full 2024.10.22 01:18:51.083233 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:51.091221 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_446_89, total 6734 rows starting from the beginning of the part 2024.10.22 01:18:51.124195 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_432_210} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:51.131064 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_432_210} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_431_209, total 45699 rows starting from the beginning of the part 2024.10.22 01:18:51.193632 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_447_447_0, total 21 rows starting from the beginning of the part 2024.10.22 01:18:51.291091 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_432_210} MergeTreeSequentialSource: Reading 2 marks from part 202410_432_432_0, total 225 rows starting from the beginning of the part 2024.10.22 01:18:51.367790 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_448_448_0, total 19 rows starting from the beginning of the part 2024.10.22 01:18:51.412584 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_449_449_0, total 19 rows starting from the beginning of the part 2024.10.22 01:18:51.462082 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 8.22 MiB, will set to 1.47 GiB (RSS), difference: -1.87 MiB 2024.10.22 01:18:51.469563 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_450_450_0, total 11 rows starting from the beginning of the part 2024.10.22 01:18:51.483644 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(7c9cd814bb99f1dd05de5abb8b7fc17b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7c9cd814bb99f1dd05de5abb8b7fc17b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:51.487470 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 818, file segment info: File segment: [0, 33554431], key: 7c9cd814bb99f1dd05de5abb8b7fc17b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 01:18:51.489481 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:51.500474 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_451_451_0, total 13 rows starting from the beginning of the part 2024.10.22 01:18:51.628874 [ 1718 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mil/fcyzlpnrkqysvmfumfxgymvborikd, size 975558 2024.10.22 01:18:51.833647 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:18:51.847759 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_751_359} MergeTask::MergeProjectionsStage: Merge sorted 47574 rows, containing 13 columns (13 merged, 0 gathered) in 2.91574042 sec., 16316.267275946326 rows/sec., 2.61 MiB/sec. 2024.10.22 01:18:51.868965 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:18:51.886650 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:18:51.915220 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:18:51.970726 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:18:52.055036 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_751_359} MergedBlockOutputStream: filled checksums 202410_1_751_359 (state Temporary) 2024.10.22 01:18:52.096417 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:18:52.186774 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ad4915add7c5fd5d22cb41c64403df45) : [0, 493143]: Setting download as failed: File segment: [0, 493143], key: ad4915add7c5fd5d22cb41c64403df45, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:52.187784 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_751_359} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_751_359 to 202410_1_751_359 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:52.188541 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ad4915add7c5fd5d22cb41c64403df45): No space left in cache to reserve 493144 bytes, will continue without cache download 2024.10.22 01:18:52.191602 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_751_359} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_750_358, 202410_751_751_0] -> 202410_1_751_359 2024.10.22 01:18:52.230631 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.37 MiB. 2024.10.22 01:18:52.267846 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ad4915add7c5fd5d22cb41c64403df45, offset: 0, size: 0 2024.10.22 01:18:52.271359 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ad4915add7c5fd5d22cb41c64403df45 for removal 2024.10.22 01:18:52.274787 [ 1476 ] {} CacheMetadata: Key ad4915add7c5fd5d22cb41c64403df45 is removed from metadata 2024.10.22 01:18:52.282455 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 39a5ed7f937cc6d7f979cbc2edfa2dc3, offset: 0, size: 0 2024.10.22 01:18:52.283801 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 39a5ed7f937cc6d7f979cbc2edfa2dc3 for removal 2024.10.22 01:18:52.290482 [ 1476 ] {} CacheMetadata: Key 39a5ed7f937cc6d7f979cbc2edfa2dc3 is removed from metadata 2024.10.22 01:18:52.292528 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:18:52.823302 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} MergeTask::MergeProjectionsStage: Merge sorted 6817 rows, containing 12 columns (12 merged, 0 gathered) in 1.817605615 sec., 3750.5385897479196 rows/sec., 594.75 KiB/sec. 2024.10.22 01:18:52.980716 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} MergedBlockOutputStream: filled checksums 202410_1_451_90 (state Temporary) 2024.10.22 01:18:53.124699 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 114 entries to flush up to offset 47271 2024.10.22 01:18:53.336458 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:53.343481 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:53.490361 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a7dbc220a868244af97a8bb49a3608a6) : [0, 251]: Setting download as failed: File segment: [0, 251], key: a7dbc220a868244af97a8bb49a3608a6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:53.492065 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_451_90 to 202410_1_451_90 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:53.492793 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a7dbc220a868244af97a8bb49a3608a6): No space left in cache to reserve 252 bytes, will continue without cache download 2024.10.22 01:18:53.500916 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_451_90} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_446_89, 202410_451_451_0] -> 202410_1_451_90 2024.10.22 01:18:53.535617 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_744_744_0 (state Temporary) 2024.10.22 01:18:53.569284 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.40 MiB. 2024.10.22 01:18:53.596460 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_744_744_0 to 202410_757_757_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:53.619974 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.96 GiB, free memory in arenas 14.24 MiB, will set to 1.49 GiB (RSS), difference: -49.97 MiB 2024.10.22 01:18:53.628586 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_751_359 to 202410_752_752_0 2024.10.22 01:18:53.635117 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.70 MiB using storage policy from min volume index 0 2024.10.22 01:18:53.638749 [ 1259 ] {} DiskLocal: Reserved 1.70 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:53.665153 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_752_360} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_751_359 to 202410_752_752_0 into Compact with storage Full 2024.10.22 01:18:53.695975 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_752_360} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:53.715750 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_752_360} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_751_359, total 47574 rows starting from the beginning of the part 2024.10.22 01:18:53.770145 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_752_360} MergeTreeSequentialSource: Reading 2 marks from part 202410_752_752_0, total 107 rows starting from the beginning of the part 2024.10.22 01:18:53.827356 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_651_274, 202410_652_652_0] 2024.10.22 01:18:53.829188 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_651_274, 202410_652_652_0] 2024.10.22 01:18:53.845340 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_651_274, 202410_652_652_0] 2024.10.22 01:18:53.859501 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47271 2024.10.22 01:18:54.388655 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_432_210/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:18:54.539067 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_432_210} MergeTask::MergeProjectionsStage: Merge sorted 45924 rows, containing 13 columns (13 merged, 0 gathered) in 3.351639997 sec., 13701.948908923943 rows/sec., 5.94 MiB/sec. 2024.10.22 01:18:54.566595 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(90864b401cc5723d2aafcee52dec29cb) : [0, 946469]: Setting download as failed: File segment: [0, 946469], key: 90864b401cc5723d2aafcee52dec29cb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:54.567818 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(90864b401cc5723d2aafcee52dec29cb): No space left in cache to reserve 946470 bytes, will continue without cache download 2024.10.22 01:18:54.611453 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_432_210} MergedBlockOutputStream: filled checksums 202410_1_432_210 (state Temporary) 2024.10.22 01:18:54.959895 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 90864b401cc5723d2aafcee52dec29cb, offset: 0, size: 0 2024.10.22 01:18:54.969511 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 90864b401cc5723d2aafcee52dec29cb for removal 2024.10.22 01:18:54.972089 [ 1476 ] {} CacheMetadata: Key 90864b401cc5723d2aafcee52dec29cb is removed from metadata 2024.10.22 01:18:54.974738 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a7dbc220a868244af97a8bb49a3608a6, offset: 0, size: 0 2024.10.22 01:18:54.978229 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a7dbc220a868244af97a8bb49a3608a6 for removal 2024.10.22 01:18:54.982396 [ 1476 ] {} CacheMetadata: Key a7dbc220a868244af97a8bb49a3608a6 is removed from metadata 2024.10.22 01:18:55.533830 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 8.82 MiB, will set to 1.48 GiB (RSS), difference: 35.27 MiB 2024.10.22 01:18:55.576947 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_432_210} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_432_210 to 202410_1_432_210 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:55.580594 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_432_210} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_431_209, 202410_432_432_0] -> 202410_1_432_210 2024.10.22 01:18:55.707751 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.09 MiB. 2024.10.22 01:18:55.876365 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1364 entries to flush up to offset 663010 2024.10.22 01:18:55.922723 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(10ae63ce901d0ddf0b4dfcb85bde0115) : [0, 248]: Setting download as failed: File segment: [0, 248], key: 10ae63ce901d0ddf0b4dfcb85bde0115, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:55.924292 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(10ae63ce901d0ddf0b4dfcb85bde0115): No space left in cache to reserve 249 bytes, will continue without cache download 2024.10.22 01:18:56.292285 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:56.297086 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:56.490730 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_421_421_0 (state Temporary) 2024.10.22 01:18:56.679621 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_752_360} MergeTask::MergeProjectionsStage: Merge sorted 47681 rows, containing 13 columns (13 merged, 0 gathered) in 3.021019519 sec., 15783.082399872437 rows/sec., 2.52 MiB/sec. 2024.10.22 01:18:56.743349 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_421_421_0 to 202410_428_428_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:56.772776 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_418_85 to 202410_428_428_0 2024.10.22 01:18:56.773209 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0dd2f7992dcff06d7560c34397cc71d8) : [0, 823362]: Setting download as failed: File segment: [0, 823362], key: 0dd2f7992dcff06d7560c34397cc71d8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:56.777690 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 663010 2024.10.22 01:18:56.778361 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0dd2f7992dcff06d7560c34397cc71d8): No space left in cache to reserve 823363 bytes, will continue without cache download 2024.10.22 01:18:56.817131 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:56.819090 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:56.844984 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_418_85 to 202410_428_428_0 into Compact with storage Full 2024.10.22 01:18:56.847629 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_752_360} MergedBlockOutputStream: filled checksums 202410_1_752_360 (state Temporary) 2024.10.22 01:18:56.857024 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:56.859386 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} MergeTreeSequentialSource: Reading 83 marks from part 202410_1_418_85, total 665048 rows starting from the beginning of the part 2024.10.22 01:18:56.865438 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_419_424_1, total 6547 rows starting from the beginning of the part 2024.10.22 01:18:56.874238 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_425_425_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:18:56.915424 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_426_426_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:18:56.920379 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_752_360} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_752_360 to 202410_1_752_360 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:56.926963 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_752_360} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_751_359, 202410_752_752_0] -> 202410_1_752_360 2024.10.22 01:18:56.945425 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_427_427_0, total 946 rows starting from the beginning of the part 2024.10.22 01:18:56.970666 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_428_428_0, total 1364 rows starting from the beginning of the part 2024.10.22 01:18:56.974586 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.37 MiB. 2024.10.22 01:18:57.071858 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0dd2f7992dcff06d7560c34397cc71d8, offset: 0, size: 0 2024.10.22 01:18:57.073359 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0dd2f7992dcff06d7560c34397cc71d8 for removal 2024.10.22 01:18:57.075586 [ 1476 ] {} CacheMetadata: Key 0dd2f7992dcff06d7560c34397cc71d8 is removed from metadata 2024.10.22 01:18:57.078955 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 10ae63ce901d0ddf0b4dfcb85bde0115, offset: 0, size: 0 2024.10.22 01:18:57.081283 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 10ae63ce901d0ddf0b4dfcb85bde0115 for removal 2024.10.22 01:18:57.083973 [ 1476 ] {} CacheMetadata: Key 10ae63ce901d0ddf0b4dfcb85bde0115 is removed from metadata 2024.10.22 01:18:57.382809 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_652_275, 202410_653_653_0] 2024.10.22 01:18:57.384987 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_652_275, 202410_653_653_0] 2024.10.22 01:18:57.391911 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(bee28f16fcdbf64249b9fab13d00f177) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: bee28f16fcdbf64249b9fab13d00f177, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:18:57.392784 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_652_275, 202410_653_653_0] 2024.10.22 01:18:57.428598 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: bee28f16fcdbf64249b9fab13d00f177, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:18:57.432006 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:18:57.464427 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.96 GiB, free memory in arenas 9.52 MiB, will set to 1.47 GiB (RSS), difference: 2.12 MiB 2024.10.22 01:18:57.484982 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 372 2024.10.22 01:18:57.870336 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:57.876636 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 47348 2024.10.22 01:18:57.881776 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:58.012939 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(3d103f2e18117a1424d16e1e66d44aa8) : [0, 243]: Setting download as failed: File segment: [0, 243], key: 3d103f2e18117a1424d16e1e66d44aa8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:58.015097 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(3d103f2e18117a1424d16e1e66d44aa8): No space left in cache to reserve 244 bytes, will continue without cache download 2024.10.22 01:18:58.049250 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_354_354_0 (state Temporary) 2024.10.22 01:18:58.126151 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:58.134915 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:58.217547 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_354_354_0 to 202410_364_364_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:58.254935 [ 1163 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_359_74 to 202410_364_364_0 2024.10.22 01:18:58.266955 [ 1163 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:58.273244 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:58.320960 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 372 2024.10.22 01:18:58.321189 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_745_745_0 (state Temporary) 2024.10.22 01:18:58.301810 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_359_74 to 202410_364_364_0 into Compact with storage Full 2024.10.22 01:18:58.466827 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_745_745_0 to 202410_758_758_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:58.480227 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 6733 2024.10.22 01:18:58.498971 [ 1189 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_752_360 to 202410_753_753_0 2024.10.22 01:18:58.514297 [ 1189 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.71 MiB using storage policy from min volume index 0 2024.10.22 01:18:58.521140 [ 1189 ] {} DiskLocal: Reserved 1.71 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:58.524057 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47348 2024.10.22 01:18:58.552175 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_753_361} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_752_360 to 202410_753_753_0 into Compact with storage Full 2024.10.22 01:18:58.567742 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 240 entries to flush up to offset 47194 2024.10.22 01:18:58.649130 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:58.670246 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_753_361} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:58.680752 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_753_361} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_752_360, total 47681 rows starting from the beginning of the part 2024.10.22 01:18:58.692335 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(aff2beefc0a53d3920d9161b35ea6a96) : [0, 526791]: Setting download as failed: File segment: [0, 526791], key: aff2beefc0a53d3920d9161b35ea6a96, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:58.694851 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(aff2beefc0a53d3920d9161b35ea6a96): No space left in cache to reserve 526792 bytes, will continue without cache download 2024.10.22 01:18:58.703832 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:58.708673 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:58.723646 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_428_86/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:18:58.758580 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: aff2beefc0a53d3920d9161b35ea6a96, offset: 0, size: 0 2024.10.22 01:18:58.759838 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key aff2beefc0a53d3920d9161b35ea6a96 for removal 2024.10.22 01:18:58.763088 [ 1476 ] {} CacheMetadata: Key aff2beefc0a53d3920d9161b35ea6a96 is removed from metadata 2024.10.22 01:18:58.765097 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3d103f2e18117a1424d16e1e66d44aa8, offset: 0, size: 0 2024.10.22 01:18:58.766576 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3d103f2e18117a1424d16e1e66d44aa8 for removal 2024.10.22 01:18:58.768694 [ 1476 ] {} CacheMetadata: Key 3d103f2e18117a1424d16e1e66d44aa8 is removed from metadata 2024.10.22 01:18:58.790069 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_359_74, total 403 rows starting from the beginning of the part 2024.10.22 01:18:58.834713 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_753_361} MergeTreeSequentialSource: Reading 2 marks from part 202410_753_753_0, total 75 rows starting from the beginning of the part 2024.10.22 01:18:58.876877 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_439_439_0 (state Temporary) 2024.10.22 01:18:59.073249 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_374_152, 202410_375_375_0] 2024.10.22 01:18:59.074388 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_374_152, 202410_375_375_0] 2024.10.22 01:18:59.087744 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_374_152, 202410_375_375_0] 2024.10.22 01:18:59.135027 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_360_360_0, total 1 rows starting from the beginning of the part 2024.10.22 01:18:59.151496 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_439_439_0 to 202410_452_452_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:59.169941 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:18:59.172908 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:59.220740 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6733 2024.10.22 01:18:59.262876 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_432_432_0 (state Temporary) 2024.10.22 01:18:59.347563 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_361_361_0, total 1 rows starting from the beginning of the part 2024.10.22 01:18:59.378972 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_432_432_0 to 202410_438_438_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:18:59.426528 [ 1197 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_432_210 to 202410_433_433_0 2024.10.22 01:18:59.435736 [ 1197 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.35 MiB using storage policy from min volume index 0 2024.10.22 01:18:59.436820 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 47194 2024.10.22 01:18:59.441181 [ 1197 ] {} DiskLocal: Reserved 2.35 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:18:59.462027 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_433_211} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_432_210 to 202410_433_433_0 into Wide with storage Full 2024.10.22 01:18:59.493593 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.96 GiB, free memory in arenas 12.52 MiB, will set to 1.47 GiB (RSS), difference: 2.24 MiB 2024.10.22 01:18:59.522432 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_433_211} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:18:59.534331 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_362_362_0, total 1 rows starting from the beginning of the part 2024.10.22 01:18:59.543260 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_433_211} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_432_210, total 45924 rows starting from the beginning of the part 2024.10.22 01:18:59.711541 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_433_211} MergeTreeSequentialSource: Reading 2 marks from part 202410_433_433_0, total 235 rows starting from the beginning of the part 2024.10.22 01:18:59.729085 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_363_363_0, total 1 rows starting from the beginning of the part 2024.10.22 01:18:59.857363 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_364_364_0, total 1 rows starting from the beginning of the part 2024.10.22 01:18:59.954953 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f3b7c4c2f9365dce910879b28846afc2) : [0, 242]: Setting download as failed: File segment: [0, 242], key: f3b7c4c2f9365dce910879b28846afc2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:18:59.956099 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f3b7c4c2f9365dce910879b28846afc2): No space left in cache to reserve 243 bytes, will continue without cache download 2024.10.22 01:19:00.172445 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:19:00.173556 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:19:01.137819 [ 1200 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 214 ms. 2024.10.22 01:19:01.291984 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d8e83ef9715a42410f9825d7af000c18) : [0, 528322]: Setting download as failed: File segment: [0, 528322], key: d8e83ef9715a42410f9825d7af000c18, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:01.296307 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d8e83ef9715a42410f9825d7af000c18): No space left in cache to reserve 528323 bytes, will continue without cache download 2024.10.22 01:19:01.378145 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_89_18/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:19:01.396556 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:01.403651 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:19:01.474898 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.96 GiB, free memory in arenas 10.26 MiB, will set to 1.49 GiB (RSS), difference: -59.29 MiB 2024.10.22 01:19:01.677512 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d8e83ef9715a42410f9825d7af000c18, offset: 0, size: 0 2024.10.22 01:19:01.679461 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d8e83ef9715a42410f9825d7af000c18 for removal 2024.10.22 01:19:01.682369 [ 1476 ] {} CacheMetadata: Key d8e83ef9715a42410f9825d7af000c18 is removed from metadata 2024.10.22 01:19:01.684901 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f3b7c4c2f9365dce910879b28846afc2, offset: 0, size: 0 2024.10.22 01:19:01.690241 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f3b7c4c2f9365dce910879b28846afc2 for removal 2024.10.22 01:19:01.693240 [ 1476 ] {} CacheMetadata: Key f3b7c4c2f9365dce910879b28846afc2 is removed from metadata 2024.10.22 01:19:01.987276 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} MergeTask::MergeProjectionsStage: Merge sorted 408 rows, containing 32 columns (32 merged, 0 gathered) in 3.693078258 sec., 110.4769440279757 rows/sec., 10.38 KiB/sec. 2024.10.22 01:19:02.161116 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} MergedBlockOutputStream: filled checksums 202410_1_364_75 (state Temporary) 2024.10.22 01:19:02.451544 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_753_361} MergeTask::MergeProjectionsStage: Merge sorted 47756 rows, containing 13 columns (13 merged, 0 gathered) in 3.916380492 sec., 12193.912235430469 rows/sec., 1.95 MiB/sec. 2024.10.22 01:19:02.544526 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 47431 2024.10.22 01:19:02.604897 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_433_211} MergeTask::MergeProjectionsStage: Merge sorted 46159 rows, containing 13 columns (13 merged, 0 gathered) in 3.145888201 sec., 14672.80368874113 rows/sec., 6.36 MiB/sec. 2024.10.22 01:19:02.677374 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_753_361} MergedBlockOutputStream: filled checksums 202410_1_753_361 (state Temporary) 2024.10.22 01:19:02.758999 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_653_276, 202410_654_654_0] 2024.10.22 01:19:02.761005 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_653_276, 202410_654_654_0] 2024.10.22 01:19:02.787650 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(bb255b8436eaef53378449f42083cb98) : [0, 240]: Setting download as failed: File segment: [0, 240], key: bb255b8436eaef53378449f42083cb98, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:02.789495 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(bb255b8436eaef53378449f42083cb98): No space left in cache to reserve 241 bytes, will continue without cache download 2024.10.22 01:19:02.795158 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_753_361} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_753_361 to 202410_1_753_361 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:02.805003 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_753_361} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_752_360, 202410_753_753_0] -> 202410_1_753_361 2024.10.22 01:19:02.821115 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_364_75 to 202410_1_364_75 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:02.823702 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_364_75} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_359_74, 202410_364_364_0] -> 202410_1_364_75 2024.10.22 01:19:02.871687 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_653_276, 202410_654_654_0] 2024.10.22 01:19:02.887626 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:02.890314 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:19:02.932540 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:19:02.974518 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} MergeTask::MergeProjectionsStage: Merge sorted 3485 rows, containing 679 columns (679 merged, 0 gathered) in 109.348561879 sec., 31.87056089367081 rows/sec., 42.48 KiB/sec. 2024.10.22 01:19:03.036774 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_433_211} MergedBlockOutputStream: filled checksums 202410_1_433_211 (state Temporary) 2024.10.22 01:19:03.092933 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_746_746_0 (state Temporary) 2024.10.22 01:19:03.041608 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:19:03.264855 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_746_746_0 to 202410_759_759_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:03.300249 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_753_361 to 202410_754_754_0 2024.10.22 01:19:03.309491 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.71 MiB using storage policy from min volume index 0 2024.10.22 01:19:03.314451 [ 1218 ] {} DiskLocal: Reserved 1.71 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:19:03.368624 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_754_362} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_753_361 to 202410_754_754_0 into Compact with storage Full 2024.10.22 01:19:03.463686 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47431 2024.10.22 01:19:03.581699 [ 1203 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:19:03.604826 [ 1203 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:19:03.605150 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.96 GiB, free memory in arenas 12.54 MiB, will set to 1.48 GiB (RSS), difference: 64.60 MiB 2024.10.22 01:19:03.704485 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_754_362} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:03.715488 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_754_362} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_753_361, total 47756 rows starting from the beginning of the part 2024.10.22 01:19:03.764641 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_433_211} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_433_211 to 202410_1_433_211 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:03.767782 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_433_211} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_432_210, 202410_433_433_0] -> 202410_1_433_211 2024.10.22 01:19:03.782858 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 664550 2024.10.22 01:19:03.791877 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.09 MiB. 2024.10.22 01:19:03.836606 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_754_362} MergeTreeSequentialSource: Reading 2 marks from part 202410_754_754_0, total 82 rows starting from the beginning of the part 2024.10.22 01:19:03.915088 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(5402b1d9953a982f3e234f3d310e2d61) : [0, 434450]: Setting download as failed: File segment: [0, 434450], key: 5402b1d9953a982f3e234f3d310e2d61, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:03.917696 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(5402b1d9953a982f3e234f3d310e2d61): No space left in cache to reserve 434451 bytes, will continue without cache download 2024.10.22 01:19:04.189146 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5402b1d9953a982f3e234f3d310e2d61, offset: 0, size: 0 2024.10.22 01:19:04.192576 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5402b1d9953a982f3e234f3d310e2d61 for removal 2024.10.22 01:19:04.197807 [ 1476 ] {} CacheMetadata: Key 5402b1d9953a982f3e234f3d310e2d61 is removed from metadata 2024.10.22 01:19:04.199745 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bb255b8436eaef53378449f42083cb98, offset: 0, size: 0 2024.10.22 01:19:04.207886 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bb255b8436eaef53378449f42083cb98 for removal 2024.10.22 01:19:04.215176 [ 1476 ] {} CacheMetadata: Key bb255b8436eaef53378449f42083cb98 is removed from metadata 2024.10.22 01:19:04.931994 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} MergedBlockOutputStream: filled checksums 202410_1_89_18 (state Temporary) 2024.10.22 01:19:05.280612 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(97b0b43b96404eb0630f90120f70902f) : [0, 251]: Setting download as failed: File segment: [0, 251], key: 97b0b43b96404eb0630f90120f70902f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:05.283208 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(97b0b43b96404eb0630f90120f70902f): No space left in cache to reserve 252 bytes, will continue without cache download 2024.10.22 01:19:05.285936 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:05.285784 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_89_89_0 (state Temporary) 2024.10.22 01:19:05.390569 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:19:05.733997 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_422_422_0 (state Temporary) 2024.10.22 01:19:05.735174 [ 1188 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 01:19:05.824611 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 373 2024.10.22 01:19:05.832376 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_422_422_0 to 202410_429_429_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:05.882170 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 664550 2024.10.22 01:19:06.058009 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:06.066771 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:19:06.442246 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_355_355_0 (state Temporary) 2024.10.22 01:19:06.441193 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_375_153, 202410_376_376_0] 2024.10.22 01:19:06.554928 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_375_153, 202410_376_376_0] 2024.10.22 01:19:06.724758 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 6750 2024.10.22 01:19:06.739159 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_375_153, 202410_376_376_0] 2024.10.22 01:19:06.749490 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_355_355_0 to 202410_365_365_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:06.838802 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 13.21 MiB, will set to 1.47 GiB (RSS), difference: -9.53 MiB 2024.10.22 01:19:06.858133 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(c58e5f43596c5a72b290d2c0b10d055b) : [0, 929736]: Setting download as failed: File segment: [0, 929736], key: c58e5f43596c5a72b290d2c0b10d055b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:06.860460 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(c58e5f43596c5a72b290d2c0b10d055b): No space left in cache to reserve 929737 bytes, will continue without cache download 2024.10.22 01:19:06.875611 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:06.876744 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_754_362} MergeTask::MergeProjectionsStage: Merge sorted 47838 rows, containing 13 columns (13 merged, 0 gathered) in 3.517341753 sec., 13600.611870938661 rows/sec., 2.18 MiB/sec. 2024.10.22 01:19:06.886905 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:19:06.966212 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_754_362} MergedBlockOutputStream: filled checksums 202410_1_754_362 (state Temporary) 2024.10.22 01:19:06.987354 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 223 entries to flush up to offset 47417 2024.10.22 01:19:07.033717 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 373 2024.10.22 01:19:07.043440 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c58e5f43596c5a72b290d2c0b10d055b, offset: 0, size: 0 2024.10.22 01:19:07.045244 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_440_440_0 (state Temporary) 2024.10.22 01:19:07.046043 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c58e5f43596c5a72b290d2c0b10d055b for removal 2024.10.22 01:19:07.055810 [ 1476 ] {} CacheMetadata: Key c58e5f43596c5a72b290d2c0b10d055b is removed from metadata 2024.10.22 01:19:07.096184 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_440_440_0 to 202410_453_453_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:07.134911 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6750 2024.10.22 01:19:07.154311 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_754_362} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_754_362 to 202410_1_754_362 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:07.159465 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_754_362} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_753_361, 202410_754_754_0] -> 202410_1_754_362 2024.10.22 01:19:07.160334 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 97b0b43b96404eb0630f90120f70902f, offset: 0, size: 0 2024.10.22 01:19:07.162208 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 97b0b43b96404eb0630f90120f70902f for removal 2024.10.22 01:19:07.165439 [ 1476 ] {} CacheMetadata: Key 97b0b43b96404eb0630f90120f70902f is removed from metadata 2024.10.22 01:19:07.198618 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:19:07.476126 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 47514 2024.10.22 01:19:07.766814 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:07.845151 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:19:07.928084 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_433_433_0 (state Temporary) 2024.10.22 01:19:07.945154 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:07.949544 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:19:08.015236 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_747_747_0 (state Temporary) 2024.10.22 01:19:08.050745 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(0aa8d24e6dc7da5c906a4189419f8b0c) : [0, 240]: Setting download as failed: File segment: [0, 240], key: 0aa8d24e6dc7da5c906a4189419f8b0c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:08.053105 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(0aa8d24e6dc7da5c906a4189419f8b0c): No space left in cache to reserve 241 bytes, will continue without cache download 2024.10.22 01:19:08.175384 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_747_747_0 to 202410_760_760_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:08.197428 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_754_362 to 202410_755_755_0 2024.10.22 01:19:08.217725 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.71 MiB using storage policy from min volume index 0 2024.10.22 01:19:08.236389 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47514 2024.10.22 01:19:08.237011 [ 1248 ] {} DiskLocal: Reserved 1.71 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:19:08.236905 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_433_433_0 to 202410_439_439_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:08.378468 [ 1262 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_433_211 to 202410_434_434_0 2024.10.22 01:19:08.380046 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_755_363} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_754_362 to 202410_755_755_0 into Compact with storage Full 2024.10.22 01:19:08.386739 [ 1262 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.36 MiB using storage policy from min volume index 0 2024.10.22 01:19:08.392938 [ 1262 ] {} DiskLocal: Reserved 2.36 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:19:08.416807 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_755_363} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:08.430658 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_434_212} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_433_211 to 202410_434_434_0 into Wide with storage Full 2024.10.22 01:19:08.433448 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_755_363} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_754_362, total 47838 rows starting from the beginning of the part 2024.10.22 01:19:08.492654 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 47417 2024.10.22 01:19:08.551605 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_755_363} MergeTreeSequentialSource: Reading 2 marks from part 202410_755_755_0, total 81 rows starting from the beginning of the part 2024.10.22 01:19:08.555962 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_654_277, 202410_655_655_0] 2024.10.22 01:19:08.559444 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_654_277, 202410_655_655_0] 2024.10.22 01:19:08.575794 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_654_277, 202410_655_655_0] 2024.10.22 01:19:08.715811 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.96 GiB, free memory in arenas 13.09 MiB, will set to 1.46 GiB (RSS), difference: -6.20 MiB 2024.10.22 01:19:08.783919 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(afd76fe99c4fca97d4865c4d88376c76) : [0, 433889]: Setting download as failed: File segment: [0, 433889], key: afd76fe99c4fca97d4865c4d88376c76, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:08.786086 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(afd76fe99c4fca97d4865c4d88376c76): No space left in cache to reserve 433890 bytes, will continue without cache download 2024.10.22 01:19:08.836543 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_434_212} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:08.860518 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: afd76fe99c4fca97d4865c4d88376c76, offset: 0, size: 0 2024.10.22 01:19:08.863846 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key afd76fe99c4fca97d4865c4d88376c76 for removal 2024.10.22 01:19:08.866638 [ 1476 ] {} CacheMetadata: Key afd76fe99c4fca97d4865c4d88376c76 is removed from metadata 2024.10.22 01:19:08.869325 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0aa8d24e6dc7da5c906a4189419f8b0c, offset: 0, size: 0 2024.10.22 01:19:08.871756 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0aa8d24e6dc7da5c906a4189419f8b0c for removal 2024.10.22 01:19:08.874613 [ 1476 ] {} CacheMetadata: Key 0aa8d24e6dc7da5c906a4189419f8b0c is removed from metadata 2024.10.22 01:19:08.860812 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_434_212} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_433_211, total 46159 rows starting from the beginning of the part 2024.10.22 01:19:09.092002 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 278 ms. 2024.10.22 01:19:09.223157 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_428_86/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:19:09.228783 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_434_212} MergeTreeSequentialSource: Reading 2 marks from part 202410_434_434_0, total 205 rows starting from the beginning of the part 2024.10.22 01:19:10.174265 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:19:10.177713 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:19:10.219621 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_434_212/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:19:10.295807 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(d01a41c47dba59e704ab92a1bc9b8868) : [0, 163]: Setting download as failed: File segment: [0, 163], key: d01a41c47dba59e704ab92a1bc9b8868, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:10.298492 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(d01a41c47dba59e704ab92a1bc9b8868): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 01:19:10.471772 [ 1244 ] {} DatabaseCatalog: Execution took 229 ms. 2024.10.22 01:19:10.560840 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.96 GiB, free memory in arenas 11.23 MiB, will set to 1.47 GiB (RSS), difference: -50.70 MiB 2024.10.22 01:19:11.447145 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(e3ddb9a73e55005e081fac5a8d8e1b6f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e3ddb9a73e55005e081fac5a8d8e1b6f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:19:11.466871 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1011, file segment info: File segment: [0, 33554431], key: e3ddb9a73e55005e081fac5a8d8e1b6f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:19:11.469240 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:11.554557 [ 1674 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/iyr/ujfqvgdmvcrfxzxfvlztfemmvpsje, size 3358153 2024.10.22 01:19:11.596623 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column HID 2024.10.22 01:19:11.607741 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_755_363} MergeTask::MergeProjectionsStage: Merge sorted 47919 rows, containing 13 columns (13 merged, 0 gathered) in 3.23499235 sec., 14812.708907951512 rows/sec., 2.37 MiB/sec. 2024.10.22 01:19:11.608857 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column HID 2024.10.22 01:19:11.679205 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_755_363} MergedBlockOutputStream: filled checksums 202410_1_755_363 (state Temporary) 2024.10.22 01:19:11.722909 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column HID 2024.10.22 01:19:11.732146 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column HID 2024.10.22 01:19:11.774882 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column HID 2024.10.22 01:19:11.784548 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column HID 2024.10.22 01:19:11.815935 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column HID 2024.10.22 01:19:11.839978 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column HID 2024.10.22 01:19:11.861010 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column HID 2024.10.22 01:19:11.939458 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_655_278, 202410_656_656_0] 2024.10.22 01:19:11.944320 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_655_278, 202410_656_656_0] 2024.10.22 01:19:12.022526 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fyd/wgxgemeqbrumiqcxbrmycouzujlvx (LogSeriesLimiter: on interval from 2024-10-22 01:18:48 to 2024-10-22 01:19:12 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:19:12.051471 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_755_363} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_755_363 to 202410_1_755_363 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:12.057665 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_755_363} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_754_362, 202410_755_755_0] -> 202410_1_755_363 2024.10.22 01:19:12.058449 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_655_278, 202410_656_656_0] 2024.10.22 01:19:12.083139 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:19:12.252776 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 47584 2024.10.22 01:19:12.368099 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(03ec91c34d53867546d3c3d771e05363) : [0, 2563]: Setting download as failed: File segment: [0, 2563], key: 03ec91c34d53867546d3c3d771e05363, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:12.369271 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(03ec91c34d53867546d3c3d771e05363): No space left in cache to reserve 2564 bytes, will continue without cache download 2024.10.22 01:19:12.381660 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 03ec91c34d53867546d3c3d771e05363, offset: 0, size: 0 2024.10.22 01:19:12.384378 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 03ec91c34d53867546d3c3d771e05363 for removal 2024.10.22 01:19:12.386400 [ 1476 ] {} CacheMetadata: Key 03ec91c34d53867546d3c3d771e05363 is removed from metadata 2024.10.22 01:19:12.388589 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d01a41c47dba59e704ab92a1bc9b8868, offset: 0, size: 0 2024.10.22 01:19:12.390508 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d01a41c47dba59e704ab92a1bc9b8868 for removal 2024.10.22 01:19:12.494492 [ 1476 ] {} CacheMetadata: Key d01a41c47dba59e704ab92a1bc9b8868 is removed from metadata 2024.10.22 01:19:12.531738 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_434_212} MergeTask::MergeProjectionsStage: Merge sorted 46364 rows, containing 13 columns (13 merged, 0 gathered) in 4.120106388 sec., 11253.107476796544 rows/sec., 4.88 MiB/sec. 2024.10.22 01:19:12.701388 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:12.709645 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:12.718689 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_434_212} MergedBlockOutputStream: filled checksums 202410_1_434_212 (state Temporary) 2024.10.22 01:19:12.786829 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_748_748_0 (state Temporary) 2024.10.22 01:19:12.885088 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1447 entries to flush up to offset 665997 2024.10.22 01:19:12.987603 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_748_748_0 to 202410_761_761_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:13.016394 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_755_363 to 202410_756_756_0 2024.10.22 01:19:13.027341 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.71 MiB using storage policy from min volume index 0 2024.10.22 01:19:13.038771 [ 1257 ] {} DiskLocal: Reserved 1.71 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:13.070647 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_756_364} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_755_363 to 202410_756_756_0 into Compact with storage Full 2024.10.22 01:19:13.144142 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47584 2024.10.22 01:19:13.152006 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_756_364} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:13.163356 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_756_364} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_755_363, total 47919 rows starting from the beginning of the part 2024.10.22 01:19:13.192485 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_756_364} MergeTreeSequentialSource: Reading 2 marks from part 202410_756_756_0, total 69 rows starting from the beginning of the part 2024.10.22 01:19:13.465175 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_89_89_0 to 202410_92_92_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:13.481394 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:13.485007 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:13.516692 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 12.79 MiB, will set to 1.46 GiB (RSS), difference: 33.35 MiB 2024.10.22 01:19:13.524113 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(305309b887b6085190a73a13b19cca60) : [0, 261]: Setting download as failed: File segment: [0, 261], key: 305309b887b6085190a73a13b19cca60, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:13.525315 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(305309b887b6085190a73a13b19cca60): No space left in cache to reserve 262 bytes, will continue without cache download 2024.10.22 01:19:13.645576 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_423_423_0 (state Temporary) 2024.10.22 01:19:13.802562 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_434_212} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_434_212 to 202410_1_434_212 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:13.809034 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_434_212} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_433_211, 202410_434_434_0] -> 202410_1_434_212 2024.10.22 01:19:13.866052 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_423_423_0 to 202410_430_430_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:13.897638 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 665997 2024.10.22 01:19:13.954479 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.09 MiB. 2024.10.22 01:19:14.536596 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 374 2024.10.22 01:19:14.559497 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(fca09a42723c2c5ebfb0cc4e4b9d5598) : [0, 1925700]: Setting download as failed: File segment: [0, 1925700], key: fca09a42723c2c5ebfb0cc4e4b9d5598, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:14.561346 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(fca09a42723c2c5ebfb0cc4e4b9d5598): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:19:14.637219 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 6766 2024.10.22 01:19:14.761607 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fca09a42723c2c5ebfb0cc4e4b9d5598, offset: 0, size: 0 2024.10.22 01:19:14.763263 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fca09a42723c2c5ebfb0cc4e4b9d5598 for removal 2024.10.22 01:19:14.764951 [ 1476 ] {} CacheMetadata: Key fca09a42723c2c5ebfb0cc4e4b9d5598 is removed from metadata 2024.10.22 01:19:14.766005 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 305309b887b6085190a73a13b19cca60, offset: 0, size: 0 2024.10.22 01:19:14.766806 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 305309b887b6085190a73a13b19cca60 for removal 2024.10.22 01:19:14.767839 [ 1476 ] {} CacheMetadata: Key 305309b887b6085190a73a13b19cca60 is removed from metadata 2024.10.22 01:19:14.859416 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:14.863965 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:14.957475 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_376_154, 202410_377_377_0] 2024.10.22 01:19:14.963044 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_376_154, 202410_377_377_0] 2024.10.22 01:19:14.999041 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_441_441_0 (state Temporary) 2024.10.22 01:19:15.030772 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_376_154, 202410_377_377_0] 2024.10.22 01:19:15.057190 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:15.061161 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:15.116428 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_356_356_0 (state Temporary) 2024.10.22 01:19:15.170775 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_441_441_0 to 202410_454_454_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:15.227461 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_356_356_0 to 202410_366_366_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:15.236304 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6766 2024.10.22 01:19:15.285892 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 374 2024.10.22 01:19:15.327779 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.96 GiB, free memory in arenas 14.45 MiB, will set to 1.47 GiB (RSS), difference: 16.05 MiB 2024.10.22 01:19:15.602763 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(f15ee4b6e756ff3c5757d47a517f48b7) : [0, 266]: Setting download as failed: File segment: [0, 266], key: f15ee4b6e756ff3c5757d47a517f48b7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:15.604549 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(f15ee4b6e756ff3c5757d47a517f48b7): No space left in cache to reserve 267 bytes, will continue without cache download 2024.10.22 01:19:16.012483 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_756_364} MergeTask::MergeProjectionsStage: Merge sorted 47988 rows, containing 13 columns (13 merged, 0 gathered) in 2.947574423 sec., 16280.504955379036 rows/sec., 2.61 MiB/sec. 2024.10.22 01:19:16.029808 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 235 entries to flush up to offset 47652 2024.10.22 01:19:16.176885 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_756_364} MergedBlockOutputStream: filled checksums 202410_1_756_364 (state Temporary) 2024.10.22 01:19:16.290117 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3542 2024.10.22 01:19:16.460781 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_756_364} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_756_364 to 202410_1_756_364 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:16.463332 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_756_364} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_755_363, 202410_756_756_0] -> 202410_1_756_364 2024.10.22 01:19:16.482897 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.37 MiB. 2024.10.22 01:19:16.537452 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(ba72e01778bda6caa33d6bed348ce708) : [0, 2175553]: Setting download as failed: File segment: [0, 2175553], key: ba72e01778bda6caa33d6bed348ce708, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:16.543181 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(ba72e01778bda6caa33d6bed348ce708): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:19:16.568610 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:16.571243 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:16.854014 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_434_434_0 (state Temporary) 2024.10.22 01:19:16.957666 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ba72e01778bda6caa33d6bed348ce708, offset: 0, size: 0 2024.10.22 01:19:16.960531 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ba72e01778bda6caa33d6bed348ce708 for removal 2024.10.22 01:19:16.964601 [ 1476 ] {} CacheMetadata: Key ba72e01778bda6caa33d6bed348ce708 is removed from metadata 2024.10.22 01:19:16.969401 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f15ee4b6e756ff3c5757d47a517f48b7, offset: 0, size: 0 2024.10.22 01:19:16.971925 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f15ee4b6e756ff3c5757d47a517f48b7 for removal 2024.10.22 01:19:16.974530 [ 1476 ] {} CacheMetadata: Key f15ee4b6e756ff3c5757d47a517f48b7 is removed from metadata 2024.10.22 01:19:16.981237 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_89_18 to 202410_1_89_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:16.984933 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_89_18} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_84_17, 202410_89_89_0] -> 202410_1_89_18 2024.10.22 01:19:17.092596 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a201f0cb708f13bd91bd5481715086cc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a201f0cb708f13bd91bd5481715086cc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:19:17.108143 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: a201f0cb708f13bd91bd5481715086cc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 01:19:17.111673 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:17.168641 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 47669 2024.10.22 01:19:17.270876 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_434_434_0 to 202410_440_440_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:17.301719 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 47652 2024.10.22 01:19:17.315061 [ 1170 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_434_212 to 202410_435_435_0 2024.10.22 01:19:17.322572 [ 1170 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.37 MiB using storage policy from min volume index 0 2024.10.22 01:19:17.328977 [ 1170 ] {} DiskLocal: Reserved 2.37 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:17.426236 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:17.431945 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:17.433644 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_656_279, 202410_657_657_0] 2024.10.22 01:19:17.440111 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_656_279, 202410_657_657_0] 2024.10.22 01:19:17.453161 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_656_279, 202410_657_657_0] 2024.10.22 01:19:17.484761 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_749_749_0 (state Temporary) 2024.10.22 01:19:17.513563 [ 1170 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:19:17.558737 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_749_749_0 to 202410_762_762_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:17.588064 [ 1156 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_756_364 to 202410_757_757_0 2024.10.22 01:19:17.596633 [ 1156 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.72 MiB using storage policy from min volume index 0 2024.10.22 01:19:17.606959 [ 1156 ] {} DiskLocal: Reserved 1.72 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:17.615229 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47669 2024.10.22 01:19:17.629035 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.96 GiB, free memory in arenas 15.89 MiB, will set to 1.46 GiB (RSS), difference: 10.45 MiB 2024.10.22 01:19:17.641024 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_757_365} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_756_364 to 202410_757_757_0 into Compact with storage Full 2024.10.22 01:19:17.648776 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_435_213} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_434_212 to 202410_435_435_0 into Wide with storage Full 2024.10.22 01:19:17.710139 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_757_365} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:17.716927 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_435_213} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:17.717805 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_757_365} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_756_364, total 47988 rows starting from the beginning of the part 2024.10.22 01:19:17.736118 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_435_213} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_434_212, total 46364 rows starting from the beginning of the part 2024.10.22 01:19:17.753542 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_757_365} MergeTreeSequentialSource: Reading 2 marks from part 202410_757_757_0, total 114 rows starting from the beginning of the part 2024.10.22 01:19:17.788964 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 42.67 MiB. 2024.10.22 01:19:17.899238 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(a63b5e28cecc3810520436fbfc4e58d7) : [0, 250]: Setting download as failed: File segment: [0, 250], key: a63b5e28cecc3810520436fbfc4e58d7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:17.904683 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(a63b5e28cecc3810520436fbfc4e58d7): No space left in cache to reserve 251 bytes, will continue without cache download 2024.10.22 01:19:17.908289 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_435_213} MergeTreeSequentialSource: Reading 2 marks from part 202410_435_435_0, total 255 rows starting from the beginning of the part 2024.10.22 01:19:18.539978 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(851fd806c605ff11dc156623493c9110) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 851fd806c605ff11dc156623493c9110, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:19:18.563042 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 2585, file segment info: File segment: [0, 33554431], key: 851fd806c605ff11dc156623493c9110, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:19:18.572183 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:18.614806 [ 1225 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:19:18.755631 [ 1225 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:19:19.199398 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_757_365/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:19:19.419264 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(6bef52a64ce9d4a134c3c485e4521f80) : [0, 915191]: Setting download as failed: File segment: [0, 915191], key: 6bef52a64ce9d4a134c3c485e4521f80, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:19.421103 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(6bef52a64ce9d4a134c3c485e4521f80): No space left in cache to reserve 915192 bytes, will continue without cache download 2024.10.22 01:19:19.571341 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6bef52a64ce9d4a134c3c485e4521f80, offset: 0, size: 0 2024.10.22 01:19:19.572825 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6bef52a64ce9d4a134c3c485e4521f80 for removal 2024.10.22 01:19:19.580423 [ 1476 ] {} CacheMetadata: Key 6bef52a64ce9d4a134c3c485e4521f80 is removed from metadata 2024.10.22 01:19:19.583507 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a63b5e28cecc3810520436fbfc4e58d7, offset: 0, size: 0 2024.10.22 01:19:19.586161 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a63b5e28cecc3810520436fbfc4e58d7 for removal 2024.10.22 01:19:19.590221 [ 1476 ] {} CacheMetadata: Key a63b5e28cecc3810520436fbfc4e58d7 is removed from metadata 2024.10.22 01:19:20.131137 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_757_365} MergeTask::MergeProjectionsStage: Merge sorted 48102 rows, containing 13 columns (13 merged, 0 gathered) in 2.50008806 sec., 19240.1222859326 rows/sec., 3.08 MiB/sec. 2024.10.22 01:19:20.139527 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(d398f23831b6cc3ae3a3c38f0e0d9e6b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d398f23831b6cc3ae3a3c38f0e0d9e6b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:19:20.147069 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 468, file segment info: File segment: [0, 33554431], key: d398f23831b6cc3ae3a3c38f0e0d9e6b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:19:20.154141 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:20.175931 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:19:20.178979 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:19:20.215480 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_428_86/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:19:20.314393 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_757_365} MergedBlockOutputStream: filled checksums 202410_1_757_365 (state Temporary) 2024.10.22 01:19:20.492064 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_757_365} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_757_365 to 202410_1_757_365 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:20.501095 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_757_365} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_756_364, 202410_757_757_0] -> 202410_1_757_365 2024.10.22 01:19:20.548247 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.38 MiB. 2024.10.22 01:19:20.755068 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(43c1bb5a20f7d212528a6f80cf4e620f) : [0, 170]: Setting download as failed: File segment: [0, 170], key: 43c1bb5a20f7d212528a6f80cf4e620f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:20.756403 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(43c1bb5a20f7d212528a6f80cf4e620f): No space left in cache to reserve 171 bytes, will continue without cache download 2024.10.22 01:19:20.905177 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1248 entries to flush up to offset 667245 2024.10.22 01:19:21.057839 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_657_280, 202410_658_658_0] 2024.10.22 01:19:21.060904 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_657_280, 202410_658_658_0] 2024.10.22 01:19:21.072777 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_657_280, 202410_658_658_0] 2024.10.22 01:19:21.431126 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_435_213} MergeTask::MergeProjectionsStage: Merge sorted 46619 rows, containing 13 columns (13 merged, 0 gathered) in 3.662204075 sec., 12729.765749059192 rows/sec., 5.52 MiB/sec. 2024.10.22 01:19:21.503195 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_435_213} MergedBlockOutputStream: filled checksums 202410_1_435_213 (state Temporary) 2024.10.22 01:19:21.632804 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 47732 2024.10.22 01:19:21.668581 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.96 GiB, free memory in arenas 17.02 MiB, will set to 1.47 GiB (RSS), difference: -30.42 MiB 2024.10.22 01:19:21.879436 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(6d71de787b9da253bc65a614c93fe3c4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6d71de787b9da253bc65a614c93fe3c4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:19:21.882203 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 975558, file segment info: File segment: [0, 33554431], key: 6d71de787b9da253bc65a614c93fe3c4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:19:21.888602 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:22.005210 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:22.009771 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:22.082559 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:22.085641 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:22.153821 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_424_424_0 (state Temporary) 2024.10.22 01:19:22.260890 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_750_750_0 (state Temporary) 2024.10.22 01:19:22.468475 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_424_424_0 to 202410_431_431_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:22.500473 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_750_750_0 to 202410_763_763_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:22.502859 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 667245 2024.10.22 01:19:22.537529 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_377_155, 202410_378_378_0] 2024.10.22 01:19:22.543959 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_435_213} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_435_213 to 202410_1_435_213 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:22.544293 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_377_155, 202410_378_378_0] 2024.10.22 01:19:22.554361 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_435_213} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_434_212, 202410_435_435_0] -> 202410_1_435_213 2024.10.22 01:19:22.569089 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileSegment(e8e078caf77541857547d52fc0d25ebc) : [0, 220593]: Setting download as failed: File segment: [0, 220593], key: e8e078caf77541857547d52fc0d25ebc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, current write offset: 0, caller id: d27f974d-16ec-4d21-bb2c-d65933eaeee0:1448, kind: Regular, unbound: 0 2024.10.22 01:19:22.570882 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CachedOnDiskReadBufferFromFile(e8e078caf77541857547d52fc0d25ebc): No space left in cache to reserve 220594 bytes, will continue without cache download 2024.10.22 01:19:22.571464 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_377_155, 202410_378_378_0] 2024.10.22 01:19:22.575746 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.11 MiB. 2024.10.22 01:19:22.596222 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e8e078caf77541857547d52fc0d25ebc, offset: 0, size: 0 2024.10.22 01:19:22.598221 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e8e078caf77541857547d52fc0d25ebc for removal 2024.10.22 01:19:22.607820 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 43c1bb5a20f7d212528a6f80cf4e620f, offset: 0, size: 0 2024.10.22 01:19:22.610891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 43c1bb5a20f7d212528a6f80cf4e620f for removal 2024.10.22 01:19:22.614932 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/wpm/puicreeujtmagrdirsahrtverpkun. 2024.10.22 01:19:22.617020 [ 1476 ] {} CacheMetadata: Key e8e078caf77541857547d52fc0d25ebc is removed from metadata 2024.10.22 01:19:22.626715 [ 1476 ] {} CacheMetadata: Key 43c1bb5a20f7d212528a6f80cf4e620f is removed from metadata 2024.10.22 01:19:22.632797 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_757_365 to 202410_758_758_0 2024.10.22 01:19:22.638731 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.72 MiB using storage policy from min volume index 0 2024.10.22 01:19:22.681084 [ 1210 ] {} DiskLocal: Reserved 1.72 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:19:22.700078 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_758_366} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_757_365 to 202410_758_758_0 into Compact with storage Full 2024.10.22 01:19:22.744218 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 6788 2024.10.22 01:19:22.765019 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47732 2024.10.22 01:19:22.787691 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 375 2024.10.22 01:19:22.832280 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_758_366} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:22.846898 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_758_366} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_757_365, total 48102 rows starting from the beginning of the part 2024.10.22 01:19:22.858698 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:22.861495 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:22.866974 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_758_366} MergeTreeSequentialSource: Reading 2 marks from part 202410_758_758_0, total 77 rows starting from the beginning of the part 2024.10.22 01:19:23.128242 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_442_442_0 (state Temporary) 2024.10.22 01:19:23.178173 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_442_442_0 to 202410_455_455_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:23.189079 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_366_73, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0, 202410_370_370_0, 202410_371_371_0] 2024.10.22 01:19:23.192325 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_366_73, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0, 202410_370_370_0, 202410_371_371_0] 2024.10.22 01:19:23.215169 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:23.220138 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:23.291341 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} MergeTask::MergeProjectionsStage: Merge sorted 676600 rows, containing 4 columns (4 merged, 0 gathered) in 26.449023683 sec., 25581.284515801686 rows/sec., 375.15 KiB/sec. 2024.10.22 01:19:23.331310 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_366_73, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0, 202410_370_370_0, 202410_371_371_0] 2024.10.22 01:19:23.335617 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6788 2024.10.22 01:19:23.373561 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_357_357_0 (state Temporary) 2024.10.22 01:19:23.399712 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(228e89988eabaf39c5dd300575ce3f7e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 228e89988eabaf39c5dd300575ce3f7e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:19:23.403476 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 818, file segment info: File segment: [0, 33554431], key: 228e89988eabaf39c5dd300575ce3f7e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:19:23.410154 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:23.466998 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} MergedBlockOutputStream: filled checksums 202410_1_428_86 (state Temporary) 2024.10.22 01:19:23.552087 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_357_357_0 to 202410_367_367_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:23.558249 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_428_86 to 202410_1_428_86 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:23.562310 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_428_86} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_418_85, 202410_428_428_0] -> 202410_1_428_86 2024.10.22 01:19:23.634350 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:19:23.636661 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 375 2024.10.22 01:19:23.673796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_22_22_0 to 201403_22_22_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:23.711855 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/wpm/puicreeujtmagrdirsahrtverpkun. 2024.10.22 01:19:23.764368 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:19:23.786266 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:19:23.796545 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:19:23.823032 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:19:23.888958 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 3582 2024.10.22 01:19:23.956311 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:19:23.983021 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:19:24.033698 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:19:24.071486 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:19:24.162390 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nlk/qeopbpckefhxwnkmuxclqocpptldv (LogSeriesLimiter: on interval from 2024-10-22 01:19:12 to 2024-10-22 01:19:24 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:19:24.341428 [ 1224 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 358 ms. 2024.10.22 01:19:24.872305 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 209 entries to flush up to offset 47861 2024.10.22 01:19:25.269696 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:25.272529 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:25.347679 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_435_435_0 (state Temporary) 2024.10.22 01:19:25.570987 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_435_435_0 to 202410_441_441_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:25.587615 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.08 GiB, peak 1.96 GiB, free memory in arenas 17.58 MiB, will set to 1.36 GiB (RSS), difference: 285.52 MiB 2024.10.22 01:19:25.606492 [ 1177 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_435_213 to 202410_436_436_0 2024.10.22 01:19:25.616587 [ 1177 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.38 MiB using storage policy from min volume index 0 2024.10.22 01:19:25.624132 [ 1177 ] {} DiskLocal: Reserved 2.38 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:25.650910 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_436_214} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_435_213 to 202410_436_436_0 into Wide with storage Full 2024.10.22 01:19:25.674285 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 47861 2024.10.22 01:19:25.702489 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_436_214} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:25.710975 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_436_214} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_435_213, total 46619 rows starting from the beginning of the part 2024.10.22 01:19:25.871335 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_436_214} MergeTreeSequentialSource: Reading 2 marks from part 202410_436_436_0, total 232 rows starting from the beginning of the part 2024.10.22 01:19:26.141524 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_758_366} MergeTask::MergeProjectionsStage: Merge sorted 48179 rows, containing 13 columns (13 merged, 0 gathered) in 3.449218265 sec., 13968.092564301669 rows/sec., 2.24 MiB/sec. 2024.10.22 01:19:26.431743 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_758_366} MergedBlockOutputStream: filled checksums 202410_1_758_366 (state Temporary) 2024.10.22 01:19:26.472644 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_758_366} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_758_366 to 202410_1_758_366 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:26.492121 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_758_366} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_757_365, 202410_758_758_0] -> 202410_1_758_366 2024.10.22 01:19:26.715098 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.37 MiB. 2024.10.22 01:19:26.785393 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 47824 2024.10.22 01:19:27.076959 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:27.079435 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:27.134122 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_751_751_0 (state Temporary) 2024.10.22 01:19:27.165608 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_658_281, 202410_659_659_0] 2024.10.22 01:19:27.173303 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_658_281, 202410_659_659_0] 2024.10.22 01:19:27.189784 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_658_281, 202410_659_659_0] 2024.10.22 01:19:27.217068 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_751_751_0 to 202410_764_764_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:27.268971 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_391_78, 202410_392_392_0, 202410_393_393_0, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0] 2024.10.22 01:19:27.273009 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_391_78, 202410_392_392_0, 202410_393_393_0, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0] 2024.10.22 01:19:27.274127 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47824 2024.10.22 01:19:27.265474 [ 1273 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_758_366 to 202410_759_759_0 2024.10.22 01:19:27.384491 [ 1273 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.72 MiB using storage policy from min volume index 0 2024.10.22 01:19:27.388662 [ 1273 ] {} DiskLocal: Reserved 1.72 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:27.407937 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_391_78, 202410_392_392_0, 202410_393_393_0, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0] 2024.10.22 01:19:27.419977 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_759_367} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_758_366 to 202410_759_759_0 into Compact with storage Full 2024.10.22 01:19:27.422846 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.24 GiB, peak 1.96 GiB, free memory in arenas 13.12 MiB, will set to 1.24 GiB (RSS), difference: 3.42 MiB 2024.10.22 01:19:27.487007 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_759_367} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:27.488844 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 22 2024.10.22 01:19:27.493705 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_759_367} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_758_366, total 48179 rows starting from the beginning of the part 2024.10.22 01:19:27.622075 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_759_367} MergeTreeSequentialSource: Reading 2 marks from part 202410_759_759_0, total 83 rows starting from the beginning of the part 2024.10.22 01:19:27.768161 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:27.771486 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:27.933179 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_22_22_0 (state Temporary) 2024.10.22 01:19:28.403585 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_22_22_0 to 202410_22_22_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:28.674524 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 22 2024.10.22 01:19:28.874992 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_436_214} MergeTask::MergeProjectionsStage: Merge sorted 46851 rows, containing 13 columns (13 merged, 0 gathered) in 3.233034218 sec., 14491.340592424254 rows/sec., 6.28 MiB/sec. 2024.10.22 01:19:29.067761 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_436_214} MergedBlockOutputStream: filled checksums 202410_1_436_214 (state Temporary) 2024.10.22 01:19:29.328572 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 1.96 GiB, free memory in arenas 9.76 MiB, will set to 1.28 GiB (RSS), difference: -69.58 MiB 2024.10.22 01:19:29.509560 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 668400 2024.10.22 01:19:29.819170 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_436_214} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_436_214 to 202410_1_436_214 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:29.917206 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_759_367} MergeTask::MergeProjectionsStage: Merge sorted 48262 rows, containing 13 columns (13 merged, 0 gathered) in 2.516290001 sec., 19179.82425746642 rows/sec., 3.07 MiB/sec. 2024.10.22 01:19:29.927245 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_436_214} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_435_213, 202410_436_436_0] -> 202410_1_436_214 2024.10.22 01:19:29.954586 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.10 MiB. 2024.10.22 01:19:29.983129 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_759_367} MergedBlockOutputStream: filled checksums 202410_1_759_367 (state Temporary) 2024.10.22 01:19:30.038414 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:30.048924 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:30.176037 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:19:30.178563 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:19:30.240559 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_759_367} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_759_367 to 202410_1_759_367 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:30.244203 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_759_367} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_758_366, 202410_759_759_0] -> 202410_1_759_367 2024.10.22 01:19:30.269544 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:19:30.339616 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_425_425_0 (state Temporary) 2024.10.22 01:19:30.488520 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_425_425_0 to 202410_432_432_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:30.519035 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 668400 2024.10.22 01:19:30.775844 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_659_282, 202410_660_660_0] 2024.10.22 01:19:30.782215 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_659_282, 202410_660_660_0] 2024.10.22 01:19:30.789500 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_659_282, 202410_660_660_0] 2024.10.22 01:19:30.855659 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 6811 2024.10.22 01:19:31.079014 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:31.083139 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:31.144943 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 376 2024.10.22 01:19:31.216604 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_443_443_0 (state Temporary) 2024.10.22 01:19:31.282707 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_378_156, 202410_379_379_0] 2024.10.22 01:19:31.284239 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_378_156, 202410_379_379_0] 2024.10.22 01:19:31.384092 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 47879 2024.10.22 01:19:31.430620 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_378_156, 202410_379_379_0] 2024.10.22 01:19:31.481550 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.96 GiB, free memory in arenas 8.35 MiB, will set to 1.30 GiB (RSS), difference: -40.63 MiB 2024.10.22 01:19:31.580918 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_443_443_0 to 202410_456_456_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:31.594461 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:31.600325 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:31.626064 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_451_90 to 202410_456_456_0 2024.10.22 01:19:31.633543 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:31.640346 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:31.642587 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:31.650079 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6811 2024.10.22 01:19:31.712784 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_752_752_0 (state Temporary) 2024.10.22 01:19:31.764632 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:31.782304 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_752_752_0 to 202410_765_765_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:31.797286 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_358_358_0 (state Temporary) 2024.10.22 01:19:31.828975 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_759_367 to 202410_760_760_0 2024.10.22 01:19:31.833489 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47879 2024.10.22 01:19:31.853227 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.73 MiB using storage policy from min volume index 0 2024.10.22 01:19:31.861126 [ 1249 ] {} DiskLocal: Reserved 1.73 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:19:31.880029 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_451_90 to 202410_456_456_0 into Compact with storage Full 2024.10.22 01:19:31.890962 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_760_368} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_759_367 to 202410_760_760_0 into Compact with storage Full 2024.10.22 01:19:31.913281 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:31.925212 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_451_90, total 6817 rows starting from the beginning of the part 2024.10.22 01:19:31.958161 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_452_452_0, total 22 rows starting from the beginning of the part 2024.10.22 01:19:31.969452 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_453_453_0, total 17 rows starting from the beginning of the part 2024.10.22 01:19:31.971472 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_358_358_0 to 202410_368_368_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:31.984828 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_454_454_0, total 16 rows starting from the beginning of the part 2024.10.22 01:19:32.000856 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_455_455_0, total 22 rows starting from the beginning of the part 2024.10.22 01:19:32.031154 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_456_456_0, total 23 rows starting from the beginning of the part 2024.10.22 01:19:32.040723 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_760_368} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:32.058198 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_760_368} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_759_367, total 48262 rows starting from the beginning of the part 2024.10.22 01:19:32.080233 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 376 2024.10.22 01:19:32.216934 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_760_368} MergeTreeSequentialSource: Reading 2 marks from part 202410_760_760_0, total 83 rows starting from the beginning of the part 2024.10.22 01:19:32.596455 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a8192f922b61cd8c55b1e1aa89f34c4a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a8192f922b61cd8c55b1e1aa89f34c4a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:19:32.604926 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1164, file segment info: File segment: [0, 33554431], key: a8192f922b61cd8c55b1e1aa89f34c4a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:19:32.608612 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:32.782822 [ 1693 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fyd/wgxgemeqbrumiqcxbrmycouzujlvx, size 10643045 2024.10.22 01:19:32.871955 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:19:32.990954 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:19:33.076156 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} MergeTask::MergeProjectionsStage: Merge sorted 6917 rows, containing 12 columns (12 merged, 0 gathered) in 1.303000218 sec., 5308.5179146148075 rows/sec., 842.13 KiB/sec. 2024.10.22 01:19:33.136964 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:19:33.190171 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:19:33.214635 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 225 entries to flush up to offset 48086 2024.10.22 01:19:33.269125 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:19:33.311515 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:19:33.350785 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:19:33.375736 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:19:33.424274 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:19:33.463041 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} MergedBlockOutputStream: filled checksums 202410_1_456_91 (state Temporary) 2024.10.22 01:19:33.465439 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:19:33.534522 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ioe/zwnvicuysczqxsntvhsdqelgnoloi (LogSeriesLimiter: on interval from 2024-10-22 01:19:24 to 2024-10-22 01:19:33 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:19:33.587756 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.96 GiB, free memory in arenas 5.33 MiB, will set to 1.35 GiB (RSS), difference: -54.62 MiB 2024.10.22 01:19:33.707809 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:33.710804 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:33.764176 [ 1231 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:19:33.778424 [ 1231 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:19:33.867906 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_436_436_0 (state Temporary) 2024.10.22 01:19:33.963505 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_456_91 to 202410_1_456_91 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:33.967361 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_456_91} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_451_90, 202410_456_456_0] -> 202410_1_456_91 2024.10.22 01:19:34.092555 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.42 MiB. 2024.10.22 01:19:34.168993 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_436_436_0 to 202410_442_442_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:34.208652 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 48086 2024.10.22 01:19:34.207007 [ 1242 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_436_214 to 202410_437_437_0 2024.10.22 01:19:34.235332 [ 1242 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.39 MiB using storage policy from min volume index 0 2024.10.22 01:19:34.238783 [ 1242 ] {} DiskLocal: Reserved 2.39 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:34.275016 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_437_215} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_436_214 to 202410_437_437_0 into Wide with storage Full 2024.10.22 01:19:34.416900 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_437_215} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:34.426225 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_437_215} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_436_214, total 46851 rows starting from the beginning of the part 2024.10.22 01:19:34.557792 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_437_215} MergeTreeSequentialSource: Reading 2 marks from part 202410_437_437_0, total 207 rows starting from the beginning of the part 2024.10.22 01:19:35.134818 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_760_368} MergeTask::MergeProjectionsStage: Merge sorted 48345 rows, containing 13 columns (13 merged, 0 gathered) in 3.249794245 sec., 14876.326424167202 rows/sec., 2.38 MiB/sec. 2024.10.22 01:19:35.218471 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_760_368} MergedBlockOutputStream: filled checksums 202410_1_760_368 (state Temporary) 2024.10.22 01:19:35.476352 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_760_368} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_760_368 to 202410_1_760_368 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:35.506984 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_760_368} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_759_367, 202410_760_760_0] -> 202410_1_760_368 2024.10.22 01:19:35.633495 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:19:35.790710 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.96 GiB, free memory in arenas 4.91 MiB, will set to 1.38 GiB (RSS), difference: -74.96 MiB 2024.10.22 01:19:35.857036 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 47954 2024.10.22 01:19:36.128260 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_315_65, 202410_316_316_0, 202410_317_317_0, 202410_318_318_0, 202410_319_319_0, 202410_320_320_0] 2024.10.22 01:19:36.133052 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_315_65, 202410_316_316_0, 202410_317_317_0, 202410_318_318_0, 202410_319_319_0, 202410_320_320_0] 2024.10.22 01:19:36.143458 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:36.149201 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:36.364458 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_315_65, 202410_316_316_0, 202410_317_317_0, 202410_318_318_0, 202410_319_319_0, 202410_320_320_0] 2024.10.22 01:19:36.429371 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_753_753_0 (state Temporary) 2024.10.22 01:19:36.762550 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_437_215/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:19:36.853630 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_660_283, 202410_661_661_0] 2024.10.22 01:19:36.855596 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_753_753_0 to 202410_766_766_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:36.857802 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_660_283, 202410_661_661_0] 2024.10.22 01:19:36.871653 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_660_283, 202410_661_661_0] 2024.10.22 01:19:36.882636 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_760_368 to 202410_761_761_0 2024.10.22 01:19:36.887798 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.73 MiB using storage policy from min volume index 0 2024.10.22 01:19:36.891627 [ 1186 ] {} DiskLocal: Reserved 1.73 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:37.011609 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 47954 2024.10.22 01:19:37.016261 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_761_369} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_760_368 to 202410_761_761_0 into Compact with storage Full 2024.10.22 01:19:37.084255 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_761_369} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:37.116687 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_761_369} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_760_368, total 48345 rows starting from the beginning of the part 2024.10.22 01:19:37.142381 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_761_369} MergeTreeSequentialSource: Reading 2 marks from part 202410_761_761_0, total 70 rows starting from the beginning of the part 2024.10.22 01:19:37.266292 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_437_215} MergeTask::MergeProjectionsStage: Merge sorted 47058 rows, containing 13 columns (13 merged, 0 gathered) in 2.994544665 sec., 15714.576092322204 rows/sec., 6.81 MiB/sec. 2024.10.22 01:19:37.330746 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_437_215} MergedBlockOutputStream: filled checksums 202410_1_437_215 (state Temporary) 2024.10.22 01:19:37.522811 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 669940 2024.10.22 01:19:37.578498 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 5.58 MiB, will set to 1.43 GiB (RSS), difference: -9.17 MiB 2024.10.22 01:19:37.739987 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_437_215} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_437_215 to 202410_1_437_215 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:37.751947 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_437_215} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_436_214, 202410_437_437_0] -> 202410_1_437_215 2024.10.22 01:19:37.781081 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.10 MiB. 2024.10.22 01:19:38.419996 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:38.422813 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:38.671956 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_426_426_0 (state Temporary) 2024.10.22 01:19:38.863829 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_426_426_0 to 202410_433_433_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:38.895215 [ 1176 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_428_86 to 202410_429_429_0 2024.10.22 01:19:38.901265 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 669940 2024.10.22 01:19:39.007942 [ 1176 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:39.015208 [ 1176 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:39.053479 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_429_87} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_428_86 to 202410_429_429_0 into Compact with storage Full 2024.10.22 01:19:39.158207 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 6830 2024.10.22 01:19:39.171925 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_429_87} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:39.176781 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_429_87} MergeTreeSequentialSource: Reading 84 marks from part 202410_1_428_86, total 676600 rows starting from the beginning of the part 2024.10.22 01:19:39.197738 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_429_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_429_429_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:19:39.308192 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:39.312155 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:39.445740 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 5.70 MiB, will set to 1.45 GiB (RSS), difference: -34.54 MiB 2024.10.22 01:19:39.453279 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_761_369} MergeTask::MergeProjectionsStage: Merge sorted 48415 rows, containing 13 columns (13 merged, 0 gathered) in 2.545332426 sec., 19021.091117785494 rows/sec., 3.05 MiB/sec. 2024.10.22 01:19:39.461262 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_444_444_0 (state Temporary) 2024.10.22 01:19:39.511172 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_379_157, 202410_380_380_0] 2024.10.22 01:19:39.519921 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_444_444_0 to 202410_457_457_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:39.519411 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_379_157, 202410_380_380_0] 2024.10.22 01:19:39.571883 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6830 2024.10.22 01:19:39.572669 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_379_157, 202410_380_380_0] 2024.10.22 01:19:39.687057 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_761_369} MergedBlockOutputStream: filled checksums 202410_1_761_369 (state Temporary) 2024.10.22 01:19:40.159911 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_761_369} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_761_369 to 202410_1_761_369 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:40.159718 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_661_284, 202410_662_662_0] 2024.10.22 01:19:40.162164 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_661_284, 202410_662_662_0] 2024.10.22 01:19:40.163130 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_761_369} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_760_368, 202410_761_761_0] -> 202410_1_761_369 2024.10.22 01:19:40.173096 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_661_284, 202410_662_662_0] 2024.10.22 01:19:40.178590 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:19:40.255861 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:19:40.255625 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:19:40.330759 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:40.334295 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:41.033056 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 48015 2024.10.22 01:19:41.113797 [ 1212 ] {} DatabaseCatalog: Execution took 279 ms. 2024.10.22 01:19:41.413824 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:41.463037 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 9.28 MiB, will set to 1.48 GiB (RSS), difference: 1.82 MiB 2024.10.22 01:19:41.417483 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:41.591305 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(1beafe59ee6fa945720dd3aa2499f1f9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1beafe59ee6fa945720dd3aa2499f1f9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:19:41.594431 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 1beafe59ee6fa945720dd3aa2499f1f9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:19:41.600967 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:41.705786 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_754_754_0 (state Temporary) 2024.10.22 01:19:41.878726 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 182 entries to flush up to offset 48268 2024.10.22 01:19:41.912032 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_754_754_0 to 202410_767_767_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:41.980841 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_761_369 to 202410_762_762_0 2024.10.22 01:19:41.990995 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.73 MiB using storage policy from min volume index 0 2024.10.22 01:19:42.011136 [ 1218 ] {} DiskLocal: Reserved 1.73 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:42.033612 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_762_370} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_761_369 to 202410_762_762_0 into Compact with storage Full 2024.10.22 01:19:42.110327 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_762_370} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:42.114525 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48015 2024.10.22 01:19:42.204156 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_762_370} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_761_369, total 48415 rows starting from the beginning of the part 2024.10.22 01:19:42.260045 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_762_370} MergeTreeSequentialSource: Reading 2 marks from part 202410_762_762_0, total 85 rows starting from the beginning of the part 2024.10.22 01:19:42.319999 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:42.326737 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:42.380271 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_437_437_0 (state Temporary) 2024.10.22 01:19:42.475677 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_437_437_0 to 202410_443_443_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:42.501960 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_437_215 to 202410_438_438_0 2024.10.22 01:19:42.513918 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.40 MiB using storage policy from min volume index 0 2024.10.22 01:19:42.520453 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 48268 2024.10.22 01:19:42.523082 [ 1219 ] {} DiskLocal: Reserved 2.40 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:42.590727 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_438_216} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_437_215 to 202410_438_438_0 into Wide with storage Full 2024.10.22 01:19:42.700832 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_438_216} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:42.715119 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_438_216} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_437_215, total 47058 rows starting from the beginning of the part 2024.10.22 01:19:42.826577 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_90_90_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:19:42.952523 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_438_216} MergeTreeSequentialSource: Reading 2 marks from part 202410_438_438_0, total 240 rows starting from the beginning of the part 2024.10.22 01:19:43.277299 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_429_87/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:19:43.421501 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.96 GiB, free memory in arenas 11.77 MiB, will set to 1.52 GiB (RSS), difference: -29.44 MiB 2024.10.22 01:19:44.091116 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_90_90_0 (state Temporary) 2024.10.22 01:19:44.467215 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.96 GiB, free memory in arenas 15.59 MiB, will set to 1.55 GiB (RSS), difference: -40.07 MiB 2024.10.22 01:19:45.596393 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_762_370} MergeTask::MergeProjectionsStage: Merge sorted 48500 rows, containing 13 columns (13 merged, 0 gathered) in 3.571911909 sec., 13578.162405908313 rows/sec., 2.17 MiB/sec. 2024.10.22 01:19:45.848545 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_762_370} MergedBlockOutputStream: filled checksums 202410_1_762_370 (state Temporary) 2024.10.22 01:19:45.907840 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 671865 2024.10.22 01:19:46.119960 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 48054 2024.10.22 01:19:46.123872 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_662_285, 202410_663_663_0] 2024.10.22 01:19:46.132630 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_662_285, 202410_663_663_0] 2024.10.22 01:19:46.197701 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_762_370} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_762_370 to 202410_1_762_370 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:46.202891 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_762_370} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_761_369, 202410_762_762_0] -> 202410_1_762_370 2024.10.22 01:19:46.248318 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_662_285, 202410_663_663_0] 2024.10.22 01:19:46.361071 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.37 MiB. 2024.10.22 01:19:46.385605 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:46.389798 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:46.468270 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_755_755_0 (state Temporary) 2024.10.22 01:19:46.596185 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_755_755_0 to 202410_768_768_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:46.615409 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_762_370 to 202410_763_763_0 2024.10.22 01:19:46.623503 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.74 MiB using storage policy from min volume index 0 2024.10.22 01:19:46.628331 [ 1269 ] {} DiskLocal: Reserved 1.74 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:46.635354 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.96 GiB, free memory in arenas 16.21 MiB, will set to 1.62 GiB (RSS), difference: 82.27 MiB 2024.10.22 01:19:46.671719 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48054 2024.10.22 01:19:46.676937 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_763_371} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_762_370 to 202410_763_763_0 into Compact with storage Full 2024.10.22 01:19:46.720243 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_763_371} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:46.731999 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_763_371} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_762_370, total 48500 rows starting from the beginning of the part 2024.10.22 01:19:46.770592 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_763_371} MergeTreeSequentialSource: Reading 2 marks from part 202410_763_763_0, total 63 rows starting from the beginning of the part 2024.10.22 01:19:46.825633 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_438_216} MergeTask::MergeProjectionsStage: Merge sorted 47298 rows, containing 13 columns (13 merged, 0 gathered) in 4.258687633 sec., 11106.238371063924 rows/sec., 4.82 MiB/sec. 2024.10.22 01:19:46.894274 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_438_216} MergedBlockOutputStream: filled checksums 202410_1_438_216 (state Temporary) 2024.10.22 01:19:47.074613 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 6840 2024.10.22 01:19:47.081812 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 377 2024.10.22 01:19:47.134903 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:47.140354 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:47.336063 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:47.345240 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:47.409585 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_445_445_0 (state Temporary) 2024.10.22 01:19:47.423019 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:47.426262 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:19:47.466355 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_445_445_0 to 202410_458_458_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:47.518212 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_359_359_0 (state Temporary) 2024.10.22 01:19:47.589485 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_438_216} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_438_216 to 202410_1_438_216 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:47.616354 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_438_216} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_437_215, 202410_438_438_0] -> 202410_1_438_216 2024.10.22 01:19:47.625282 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6840 2024.10.22 01:19:47.649021 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_427_427_0 (state Temporary) 2024.10.22 01:19:47.661311 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.10 MiB. 2024.10.22 01:19:47.712515 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_359_359_0 to 202410_369_369_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:47.717504 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_427_427_0 to 202410_434_434_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:47.742841 [ 1277 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_364_75 to 202410_369_369_0 2024.10.22 01:19:47.756624 [ 1277 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:47.759555 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:47.770484 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_364_75 to 202410_369_369_0 into Compact with storage Full 2024.10.22 01:19:47.797431 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 377 2024.10.22 01:19:47.854511 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 671865 2024.10.22 01:19:47.990360 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:48.056833 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_364_75, total 408 rows starting from the beginning of the part 2024.10.22 01:19:48.159631 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_380_158, 202410_381_381_0] 2024.10.22 01:19:48.166642 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_380_158, 202410_381_381_0] 2024.10.22 01:19:48.198691 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_380_158, 202410_381_381_0] 2024.10.22 01:19:48.331963 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_365_365_0, total 1 rows starting from the beginning of the part 2024.10.22 01:19:48.500559 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_366_366_0, total 1 rows starting from the beginning of the part 2024.10.22 01:19:48.756046 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 14.07 MiB, will set to 1.56 GiB (RSS), difference: 114.70 MiB 2024.10.22 01:19:48.782255 [ 1240 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:19:48.808156 [ 1240 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:19:48.941653 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 352.62 MiB using storage policy from min volume index 0 2024.10.22 01:19:48.993361 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_367_367_0, total 1 rows starting from the beginning of the part 2024.10.22 01:19:49.190732 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_763_371} MergeTask::MergeProjectionsStage: Merge sorted 48563 rows, containing 13 columns (13 merged, 0 gathered) in 2.517944502 sec., 19286.763453851534 rows/sec., 3.09 MiB/sec. 2024.10.22 01:19:49.467063 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_763_371} MergedBlockOutputStream: filled checksums 202410_1_763_371 (state Temporary) 2024.10.22 01:19:49.475601 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_368_368_0, total 1 rows starting from the beginning of the part 2024.10.22 01:19:49.689672 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_763_371} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_763_371 to 202410_1_763_371 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:49.695962 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_763_371} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_762_370, 202410_763_763_0] -> 202410_1_763_371 2024.10.22 01:19:49.725199 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:19:49.796910 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_369_369_0, total 1 rows starting from the beginning of the part 2024.10.22 01:19:49.912260 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zeb/wwjqixwulqawwttreuujnfvwidhfb (LogSeriesLimiter: on interval from 2024-10-22 01:19:33 to 2024-10-22 01:19:49 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:19:50.067951 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 207 entries to flush up to offset 48475 2024.10.22 01:19:50.178703 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:19:50.180980 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:19:50.191775 [ 1249 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 01:19:50.336613 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:50.342800 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:50.408567 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_438_438_0 (state Temporary) 2024.10.22 01:19:50.679782 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 48127 2024.10.22 01:19:50.980069 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:50.984364 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:50.989360 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.96 GiB, free memory in arenas 14.27 MiB, will set to 1.55 GiB (RSS), difference: -53.74 MiB 2024.10.22 01:19:51.144068 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_438_438_0 to 202410_444_444_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:51.174934 [ 1227 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_438_216 to 202410_439_439_0 2024.10.22 01:19:51.179685 [ 1227 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.41 MiB using storage policy from min volume index 0 2024.10.22 01:19:51.183231 [ 1227 ] {} DiskLocal: Reserved 2.41 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:51.191261 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_439_217} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_438_216 to 202410_439_439_0 into Wide with storage Full 2024.10.22 01:19:51.191700 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 48475 2024.10.22 01:19:51.260775 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_756_756_0 (state Temporary) 2024.10.22 01:19:51.476721 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_439_217} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:51.508886 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_663_286, 202410_664_664_0] 2024.10.22 01:19:51.510416 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_756_756_0 to 202410_769_769_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:51.517459 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_439_217} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_438_216, total 47298 rows starting from the beginning of the part 2024.10.22 01:19:51.522725 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_663_286, 202410_664_664_0] 2024.10.22 01:19:51.539127 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_663_286, 202410_664_664_0] 2024.10.22 01:19:51.655326 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_763_371 to 202410_764_764_0 2024.10.22 01:19:51.662628 [ 1255 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.74 MiB using storage policy from min volume index 0 2024.10.22 01:19:51.665104 [ 1255 ] {} DiskLocal: Reserved 1.74 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:51.676735 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_764_372} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_763_371 to 202410_764_764_0 into Compact with storage Full 2024.10.22 01:19:51.784567 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_24_24_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 01:19:51.788535 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48127 2024.10.22 01:19:51.907496 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_439_217} MergeTreeSequentialSource: Reading 2 marks from part 202410_439_439_0, total 223 rows starting from the beginning of the part 2024.10.22 01:19:52.036833 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_764_372} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:52.212175 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_764_372} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_763_371, total 48563 rows starting from the beginning of the part 2024.10.22 01:19:52.403463 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_764_372} MergeTreeSequentialSource: Reading 2 marks from part 202410_764_764_0, total 92 rows starting from the beginning of the part 2024.10.22 01:19:53.128311 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(ed6d6d0dc38406b218551d2df5a29a69) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ed6d6d0dc38406b218551d2df5a29a69, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:19:53.139129 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 605, file segment info: File segment: [0, 33554431], key: ed6d6d0dc38406b218551d2df5a29a69, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:19:53.145064 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:53.201529 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_439_217/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:19:53.614411 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} MergeTask::MergeProjectionsStage: Merge sorted 413 rows, containing 32 columns (32 merged, 0 gathered) in 5.84846654 sec., 70.61680137439924 rows/sec., 6.63 KiB/sec. 2024.10.22 01:19:53.660158 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.96 GiB, free memory in arenas 10.71 MiB, will set to 1.56 GiB (RSS), difference: -218.74 MiB 2024.10.22 01:19:53.786112 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} MergedBlockOutputStream: filled checksums 202410_1_369_76 (state Temporary) 2024.10.22 01:19:54.049731 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/dvb/ksxhfxhjhxiragxkioipzobdfvjvn (LogSeriesLimiter: on interval from 2024-10-22 01:19:49 to 2024-10-22 01:19:54 accepted series 1 / 160 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:19:54.161489 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(c5bf59a56e0e9765187b20d247663d97) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c5bf59a56e0e9765187b20d247663d97, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:19:54.167426 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 975558, file segment info: File segment: [0, 33554431], key: c5bf59a56e0e9765187b20d247663d97, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:19:54.173152 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:54.453134 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_90_90_0 to 202410_93_93_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:54.568420 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_429_87/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:19:54.799834 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(3278c808920ea1492a0b63299dbd5429) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3278c808920ea1492a0b63299dbd5429, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:19:54.808177 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 36, file segment info: File segment: [0, 33554431], key: 3278c808920ea1492a0b63299dbd5429, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:19:54.811925 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:54.861605 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 673020 2024.10.22 01:19:54.908112 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_369_76 to 202410_1_369_76 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:55.032858 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_369_76} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_364_75, 202410_369_369_0] -> 202410_1_369_76 2024.10.22 01:19:55.124356 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_439_217} MergeTask::MergeProjectionsStage: Merge sorted 47521 rows, containing 13 columns (13 merged, 0 gathered) in 3.93678371 sec., 12071.021295706389 rows/sec., 5.24 MiB/sec. 2024.10.22 01:19:55.131200 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 6853 2024.10.22 01:19:55.190836 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:19:55.269772 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:55.272505 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:55.293753 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_439_217} MergedBlockOutputStream: filled checksums 202410_1_439_217 (state Temporary) 2024.10.22 01:19:55.300514 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 378 2024.10.22 01:19:55.325259 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_446_446_0 (state Temporary) 2024.10.22 01:19:55.426637 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_764_372} MergeTask::MergeProjectionsStage: Merge sorted 48655 rows, containing 13 columns (13 merged, 0 gathered) in 3.7548042 sec., 12958.065829371342 rows/sec., 2.08 MiB/sec. 2024.10.22 01:19:55.604433 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_446_446_0 to 202410_459_459_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:55.622609 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_764_372} MergedBlockOutputStream: filled checksums 202410_1_764_372 (state Temporary) 2024.10.22 01:19:55.659582 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_664_287, 202410_665_665_0] 2024.10.22 01:19:55.660484 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6853 2024.10.22 01:19:55.661265 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_664_287, 202410_665_665_0] 2024.10.22 01:19:55.670998 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_664_287, 202410_665_665_0] 2024.10.22 01:19:55.686864 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:55.737138 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:55.831872 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_764_372} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_764_372 to 202410_1_764_372 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:55.903735 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_360_360_0 (state Temporary) 2024.10.22 01:19:55.939813 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(c466709fccbc998b25adf024e912c160) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c466709fccbc998b25adf024e912c160, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:19:55.982969 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 48190 2024.10.22 01:19:55.984981 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 818, file segment info: File segment: [0, 33554431], key: c466709fccbc998b25adf024e912c160, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:19:55.985214 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_764_372} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_763_371, 202410_764_764_0] -> 202410_1_764_372 2024.10.22 01:19:55.990669 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:56.111734 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:56.116389 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.15 GiB. 2024.10.22 01:19:56.128890 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_360_360_0 to 202410_370_370_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:56.153706 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.37 MiB. 2024.10.22 01:19:56.160933 [ 1682 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nlk/qeopbpckefhxwnkmuxclqocpptldv, size 975558 2024.10.22 01:19:56.221329 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_439_217} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_439_217 to 202410_1_439_217 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:56.232570 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_439_217} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_438_216, 202410_439_439_0] -> 202410_1_439_217 2024.10.22 01:19:56.238454 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 378 2024.10.22 01:19:56.264531 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.10 MiB. 2024.10.22 01:19:56.288493 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:56.302103 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:19:56.382503 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_757_757_0 (state Temporary) 2024.10.22 01:19:56.427399 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_428_428_0 (state Temporary) 2024.10.22 01:19:56.463186 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_757_757_0 to 202410_770_770_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:56.498498 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_764_372 to 202410_765_765_0 2024.10.22 01:19:56.530824 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.74 MiB using storage policy from min volume index 0 2024.10.22 01:19:56.535417 [ 1180 ] {} DiskLocal: Reserved 1.74 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:19:56.563167 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_765_373} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_764_372 to 202410_765_765_0 into Compact with storage Full 2024.10.22 01:19:56.564646 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_428_428_0 to 202410_435_435_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:56.612618 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 673020 2024.10.22 01:19:56.620614 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_765_373} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:56.601153 [ 1189 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_430_430_0 to 202410_435_435_0 2024.10.22 01:19:56.639383 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48190 2024.10.22 01:19:56.653394 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_765_373} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_764_372, total 48655 rows starting from the beginning of the part 2024.10.22 01:19:56.656263 [ 1189 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:56.659655 [ 1189 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:19:56.691683 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_765_373} MergeTreeSequentialSource: Reading 2 marks from part 202410_765_765_0, total 55 rows starting from the beginning of the part 2024.10.22 01:19:56.704979 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_430_430_0 to 202410_435_435_0 into Compact with storage Full 2024.10.22 01:19:56.742748 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IslandID 2024.10.22 01:19:56.748371 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:56.756769 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IslandID 2024.10.22 01:19:56.760254 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_430_430_0, total 1447 rows starting from the beginning of the part 2024.10.22 01:19:56.766898 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IslandID 2024.10.22 01:19:56.778807 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IslandID 2024.10.22 01:19:56.808457 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_431_431_0, total 1248 rows starting from the beginning of the part 2024.10.22 01:19:56.893377 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IslandID 2024.10.22 01:19:56.928908 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IslandID 2024.10.22 01:19:56.936008 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_432_432_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:19:56.964286 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IslandID 2024.10.22 01:19:56.968200 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_433_433_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:19:57.030315 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b50106425d8a6d97a2648c14e592d227) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b50106425d8a6d97a2648c14e592d227, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:19:57.040896 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 70, file segment info: File segment: [0, 33554431], key: b50106425d8a6d97a2648c14e592d227, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:19:57.055064 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:57.135086 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_434_434_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:19:57.182262 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_435_435_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:19:57.317485 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_765_373/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:19:57.699580 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.96 GiB, free memory in arenas 9.16 MiB, will set to 1.54 GiB (RSS), difference: -218.37 MiB 2024.10.22 01:19:58.172997 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3582 2024.10.22 01:19:58.285204 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(0de67dc98442ed28a259cfef237868e9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0de67dc98442ed28a259cfef237868e9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:19:58.288251 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 106, file segment info: File segment: [0, 33554431], key: 0de67dc98442ed28a259cfef237868e9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:19:58.290888 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:58.571299 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_381_159, 202410_382_382_0] 2024.10.22 01:19:58.572958 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_381_159, 202410_382_382_0] 2024.10.22 01:19:58.588415 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_381_159, 202410_382_382_0] 2024.10.22 01:19:58.751128 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 246 entries to flush up to offset 48721 2024.10.22 01:19:59.059266 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} MergeTask::MergeProjectionsStage: Merge sorted 8470 rows, containing 4 columns (4 merged, 0 gathered) in 2.367005096 sec., 3578.361539784365 rows/sec., 55.91 KiB/sec. 2024.10.22 01:19:59.077425 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} MergedBlockOutputStream: filled checksums 202410_430_435_1 (state Temporary) 2024.10.22 01:19:59.162489 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_430_435_1 to 202410_430_435_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:59.166572 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_430_435_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_430_430_0, 202410_435_435_0] -> 202410_430_435_1 2024.10.22 01:19:59.221787 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:19:59.282509 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:19:59.301140 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:19:59.386101 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_439_439_0 (state Temporary) 2024.10.22 01:19:59.666361 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d5388608342c7727dd2fcdfd61d1d64b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d5388608342c7727dd2fcdfd61d1d64b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:19:59.677055 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 530, file segment info: File segment: [0, 33554431], key: d5388608342c7727dd2fcdfd61d1d64b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:19:59.679784 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:19:59.713559 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_439_439_0 to 202410_445_445_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:19:59.741596 [ 1208 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_439_217 to 202410_440_440_0 2024.10.22 01:19:59.747953 [ 1208 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.42 MiB using storage policy from min volume index 0 2024.10.22 01:19:59.750678 [ 1208 ] {} DiskLocal: Reserved 2.42 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:19:59.772345 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_440_218} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_439_217 to 202410_440_440_0 into Wide with storage Full 2024.10.22 01:19:59.824239 [ 1717 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ioe/zwnvicuysczqxsntvhsdqelgnoloi, size 106 2024.10.22 01:19:59.860791 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_440_218} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:19:59.883635 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_440_218} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_439_217, total 47521 rows starting from the beginning of the part 2024.10.22 01:19:59.885025 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_765_373} MergeTask::MergeProjectionsStage: Merge sorted 48710 rows, containing 13 columns (13 merged, 0 gathered) in 3.333110101 sec., 14613.978693768928 rows/sec., 2.34 MiB/sec. 2024.10.22 01:19:59.932204 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 48721 2024.10.22 01:19:59.970758 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:19:59.996231 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:20:00.036094 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:20:00.060658 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:20:00.085758 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:20:00.120127 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:20:00.173481 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:20:00.177373 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_765_373} MergedBlockOutputStream: filled checksums 202410_1_765_373 (state Temporary) 2024.10.22 01:20:00.178903 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:20:00.181270 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:20:00.331513 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_440_218} MergeTreeSequentialSource: Reading 2 marks from part 202410_440_440_0, total 235 rows starting from the beginning of the part 2024.10.22 01:20:00.363814 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:20:00.389255 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:20:00.480669 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_765_373} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_765_373 to 202410_1_765_373 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:00.491984 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_765_373} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_764_372, 202410_765_765_0] -> 202410_1_765_373 2024.10.22 01:20:00.515634 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bsg/efmeumeqireazpzpywycfzlcqhoit (LogSeriesLimiter: on interval from 2024-10-22 01:19:54 to 2024-10-22 01:20:00 accepted series 1 / 197 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:20:00.523580 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.96 GiB, free memory in arenas 12.50 MiB, will set to 1.55 GiB (RSS), difference: -46.32 MiB 2024.10.22 01:20:00.532936 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:20:00.757409 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 101 entries to flush up to offset 48291 2024.10.22 01:20:00.959598 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_665_288, 202410_666_666_0] 2024.10.22 01:20:00.963734 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_665_288, 202410_666_666_0] 2024.10.22 01:20:00.972978 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_665_288, 202410_666_666_0] 2024.10.22 01:20:01.130681 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:01.134001 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:01.186403 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_758_758_0 (state Temporary) 2024.10.22 01:20:01.265457 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_758_758_0 to 202410_771_771_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:01.310054 [ 1224 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_765_373 to 202410_766_766_0 2024.10.22 01:20:01.378354 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48291 2024.10.22 01:20:01.427083 [ 1224 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.74 MiB using storage policy from min volume index 0 2024.10.22 01:20:01.429768 [ 1224 ] {} DiskLocal: Reserved 1.74 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:01.451267 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_766_374} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_765_373 to 202410_766_766_0 into Compact with storage Full 2024.10.22 01:20:01.480327 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_766_374} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:01.495825 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_766_374} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_765_373, total 48710 rows starting from the beginning of the part 2024.10.22 01:20:01.547124 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_766_374} MergeTreeSequentialSource: Reading 2 marks from part 202410_766_766_0, total 75 rows starting from the beginning of the part 2024.10.22 01:20:01.663151 [ 1223 ] {} DatabaseCatalog: Execution took 424 ms. 2024.10.22 01:20:02.475410 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.96 GiB, free memory in arenas 12.75 MiB, will set to 1.58 GiB (RSS), difference: -75.30 MiB 2024.10.22 01:20:03.163238 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 6881 2024.10.22 01:20:03.249122 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:03.259170 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:03.445427 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_447_447_0 (state Temporary) 2024.10.22 01:20:03.483278 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_447_447_0 to 202410_460_460_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:03.468309 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_429_87} MergeTask::MergeProjectionsStage: Merge sorted 678140 rows, containing 4 columns (4 merged, 0 gathered) in 24.431462641 sec., 27756.831834618446 rows/sec., 406.66 KiB/sec. 2024.10.22 01:20:03.618325 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 674560 2024.10.22 01:20:03.606784 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_371_74, 202410_372_372_0, 202410_373_373_0, 202410_374_374_0, 202410_375_375_0, 202410_376_376_0] 2024.10.22 01:20:03.637230 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_371_74, 202410_372_372_0, 202410_373_373_0, 202410_374_374_0, 202410_375_375_0, 202410_376_376_0] 2024.10.22 01:20:03.665334 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_371_74, 202410_372_372_0, 202410_373_373_0, 202410_374_374_0, 202410_375_375_0, 202410_376_376_0] 2024.10.22 01:20:03.678265 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_429_87} MergedBlockOutputStream: filled checksums 202410_1_429_87 (state Temporary) 2024.10.22 01:20:03.714224 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_440_218} MergeTask::MergeProjectionsStage: Merge sorted 47756 rows, containing 13 columns (13 merged, 0 gathered) in 3.947609299 sec., 12097.448451167002 rows/sec., 5.25 MiB/sec. 2024.10.22 01:20:03.716028 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_429_87} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_429_87 to 202410_1_429_87 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:03.723284 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_429_87} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_428_86, 202410_429_429_0] -> 202410_1_429_87 2024.10.22 01:20:03.741596 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 379 2024.10.22 01:20:03.746584 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6881 2024.10.22 01:20:03.762748 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:20:03.817743 [ 1161 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:20:03.849557 [ 1161 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:20:03.896703 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_766_374} MergeTask::MergeProjectionsStage: Merge sorted 48785 rows, containing 13 columns (13 merged, 0 gathered) in 2.454317726 sec., 19877.21454447092 rows/sec., 3.18 MiB/sec. 2024.10.22 01:20:03.996316 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_766_374} MergedBlockOutputStream: filled checksums 202410_1_766_374 (state Temporary) 2024.10.22 01:20:04.027996 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_440_218} MergedBlockOutputStream: filled checksums 202410_1_440_218 (state Temporary) 2024.10.22 01:20:04.068396 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:04.072943 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:04.223834 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_361_361_0 (state Temporary) 2024.10.22 01:20:04.298344 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_766_374} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_766_374 to 202410_1_766_374 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:04.307244 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_766_374} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_765_373, 202410_766_766_0] -> 202410_1_766_374 2024.10.22 01:20:04.340380 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:20:04.509335 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.96 GiB, free memory in arenas 12.90 MiB, will set to 1.58 GiB (RSS), difference: 61.51 MiB 2024.10.22 01:20:04.606847 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_361_361_0 to 202410_371_371_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:04.759247 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_440_218} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_440_218 to 202410_1_440_218 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:04.762961 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_440_218} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_439_217, 202410_440_440_0] -> 202410_1_440_218 2024.10.22 01:20:04.772285 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 379 2024.10.22 01:20:04.779595 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.11 MiB. 2024.10.22 01:20:04.916274 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:04.919825 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:05.042256 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_429_429_0 (state Temporary) 2024.10.22 01:20:05.310545 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_429_429_0 to 202410_436_436_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:05.348073 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 674560 2024.10.22 01:20:05.408820 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 48348 2024.10.22 01:20:05.659374 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:05.663158 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:05.710389 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_759_759_0 (state Temporary) 2024.10.22 01:20:05.809675 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 3624 2024.10.22 01:20:05.900486 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_666_289, 202410_667_667_0] 2024.10.22 01:20:05.901098 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_759_759_0 to 202410_772_772_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:05.907563 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_666_289, 202410_667_667_0] 2024.10.22 01:20:05.927308 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_666_289, 202410_667_667_0] 2024.10.22 01:20:05.949070 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_766_374 to 202410_767_767_0 2024.10.22 01:20:05.959208 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.74 MiB using storage policy from min volume index 0 2024.10.22 01:20:05.962938 [ 1262 ] {} DiskLocal: Reserved 1.74 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:05.974660 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_767_375} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_766_374 to 202410_767_767_0 into Compact with storage Full 2024.10.22 01:20:06.038136 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_767_375} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:06.060130 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_767_375} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_766_374, total 48785 rows starting from the beginning of the part 2024.10.22 01:20:06.112061 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_24_24_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:20:06.165371 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_382_160, 202410_383_383_0] 2024.10.22 01:20:06.165674 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48348 2024.10.22 01:20:06.167509 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_382_160, 202410_383_383_0] 2024.10.22 01:20:06.182045 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_382_160, 202410_383_383_0] 2024.10.22 01:20:06.192367 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_767_375} MergeTreeSequentialSource: Reading 2 marks from part 202410_767_767_0, total 61 rows starting from the beginning of the part 2024.10.22 01:20:07.478756 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 210 entries to flush up to offset 48931 2024.10.22 01:20:07.936823 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_767_375/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:20:08.008741 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:08.013371 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:08.146811 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_396_79, 202410_397_397_0, 202410_398_398_0, 202410_399_399_0, 202410_400_400_0, 202410_401_401_0] 2024.10.22 01:20:08.186879 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_440_440_0 (state Temporary) 2024.10.22 01:20:08.252810 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_396_79, 202410_397_397_0, 202410_398_398_0, 202410_399_399_0, 202410_400_400_0, 202410_401_401_0] 2024.10.22 01:20:08.280493 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_396_79, 202410_397_397_0, 202410_398_398_0, 202410_399_399_0, 202410_400_400_0, 202410_401_401_0] 2024.10.22 01:20:08.456492 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_440_440_0 to 202410_446_446_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:08.478946 [ 1251 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_440_218 to 202410_441_441_0 2024.10.22 01:20:08.488910 [ 1251 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.43 MiB using storage policy from min volume index 0 2024.10.22 01:20:08.494356 [ 1251 ] {} DiskLocal: Reserved 2.43 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:08.542458 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 48931 2024.10.22 01:20:08.676405 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_441_219} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_440_218 to 202410_441_441_0 into Wide with storage Full 2024.10.22 01:20:08.728243 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.96 GiB, free memory in arenas 8.81 MiB, will set to 1.59 GiB (RSS), difference: 1.39 MiB 2024.10.22 01:20:08.758934 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_441_219} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:08.766734 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_441_219} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_440_218, total 47756 rows starting from the beginning of the part 2024.10.22 01:20:08.926110 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_441_219} MergeTreeSequentialSource: Reading 2 marks from part 202410_441_441_0, total 209 rows starting from the beginning of the part 2024.10.22 01:20:08.988282 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_767_375} MergeTask::MergeProjectionsStage: Merge sorted 48846 rows, containing 13 columns (13 merged, 0 gathered) in 3.017745109 sec., 16186.257697617895 rows/sec., 2.59 MiB/sec. 2024.10.22 01:20:09.289285 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_667_290, 202410_668_668_0] 2024.10.22 01:20:09.291745 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_767_375} MergedBlockOutputStream: filled checksums 202410_1_767_375 (state Temporary) 2024.10.22 01:20:09.298401 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_667_290, 202410_668_668_0] 2024.10.22 01:20:09.316381 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_667_290, 202410_668_668_0] 2024.10.22 01:20:09.560195 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_767_375} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_767_375 to 202410_1_767_375 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:09.563355 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_767_375} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_766_374, 202410_767_767_0] -> 202410_1_767_375 2024.10.22 01:20:09.599258 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:20:10.171436 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 48395 2024.10.22 01:20:10.181347 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:20:10.184873 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:20:10.554885 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.96 GiB, free memory in arenas 7.20 MiB, will set to 1.59 GiB (RSS), difference: -42.71 MiB 2024.10.22 01:20:10.664952 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:10.667463 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:10.717532 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_760_760_0 (state Temporary) 2024.10.22 01:20:10.860678 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_760_760_0 to 202410_773_773_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:10.911981 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48395 2024.10.22 01:20:10.987486 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_767_375 to 202410_768_768_0 2024.10.22 01:20:10.993299 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.75 MiB using storage policy from min volume index 0 2024.10.22 01:20:11.003853 [ 1265 ] {} DiskLocal: Reserved 1.75 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:11.130750 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_768_376} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_767_375 to 202410_768_768_0 into Compact with storage Full 2024.10.22 01:20:11.174338 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_768_376} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:11.182147 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_768_376} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_767_375, total 48846 rows starting from the beginning of the part 2024.10.22 01:20:11.206343 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_768_376} MergeTreeSequentialSource: Reading 2 marks from part 202410_768_768_0, total 39 rows starting from the beginning of the part 2024.10.22 01:20:11.255713 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 6899 2024.10.22 01:20:11.381307 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:11.387372 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:11.426899 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_448_448_0 (state Temporary) 2024.10.22 01:20:11.575489 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_448_448_0 to 202410_461_461_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:11.595705 [ 1242 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_456_91 to 202410_461_461_0 2024.10.22 01:20:11.622993 [ 1242 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:11.629612 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:11.683548 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6899 2024.10.22 01:20:11.684843 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_456_91 to 202410_461_461_0 into Compact with storage Full 2024.10.22 01:20:11.762724 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:11.767882 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_456_91, total 6917 rows starting from the beginning of the part 2024.10.22 01:20:11.816163 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_457_457_0, total 19 rows starting from the beginning of the part 2024.10.22 01:20:11.899282 [ 1250 ] {} DatabaseCatalog: Execution took 238 ms. 2024.10.22 01:20:12.027232 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_458_458_0, total 10 rows starting from the beginning of the part 2024.10.22 01:20:12.273740 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 380 2024.10.22 01:20:12.354717 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 676100 2024.10.22 01:20:12.464937 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_459_459_0, total 13 rows starting from the beginning of the part 2024.10.22 01:20:12.508037 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_441_219} MergeTask::MergeProjectionsStage: Merge sorted 47965 rows, containing 13 columns (13 merged, 0 gathered) in 3.987670839 sec., 12028.324788218559 rows/sec., 5.22 MiB/sec. 2024.10.22 01:20:12.546297 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_460_460_0, total 28 rows starting from the beginning of the part 2024.10.22 01:20:12.563167 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_441_219} MergedBlockOutputStream: filled checksums 202410_1_441_219 (state Temporary) 2024.10.22 01:20:12.653560 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_461_0, total 18 rows starting from the beginning of the part 2024.10.22 01:20:12.867758 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:12.870278 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:12.882096 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.96 GiB, free memory in arenas 5.97 MiB, will set to 1.61 GiB (RSS), difference: 53.14 MiB 2024.10.22 01:20:12.924803 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:20:12.953684 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_362_362_0 (state Temporary) 2024.10.22 01:20:13.067838 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_362_362_0 to 202410_372_372_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:13.140178 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:13.144929 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:13.152700 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 380 2024.10.22 01:20:13.227334 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_441_219} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_441_219 to 202410_1_441_219 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:13.231466 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_441_219} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_440_218, 202410_441_441_0] -> 202410_1_441_219 2024.10.22 01:20:13.256500 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.11 MiB. 2024.10.22 01:20:13.279697 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_430_430_0 (state Temporary) 2024.10.22 01:20:13.498891 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_430_430_0 to 202410_437_437_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:13.559637 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 676100 2024.10.22 01:20:14.166696 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_768_376} MergeTask::MergeProjectionsStage: Merge sorted 48885 rows, containing 13 columns (13 merged, 0 gathered) in 3.150876494 sec., 15514.730613239963 rows/sec., 2.49 MiB/sec. 2024.10.22 01:20:14.268501 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_383_161, 202410_384_384_0] 2024.10.22 01:20:14.272157 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_383_161, 202410_384_384_0] 2024.10.22 01:20:14.284999 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_383_161, 202410_384_384_0] 2024.10.22 01:20:14.308062 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} MergeTask::MergeProjectionsStage: Merge sorted 7005 rows, containing 12 columns (12 merged, 0 gathered) in 2.628693197 sec., 2664.82220443012 rows/sec., 422.88 KiB/sec. 2024.10.22 01:20:14.395921 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(0c2822c3edc8ff367ae0c238aad340da) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0c2822c3edc8ff367ae0c238aad340da, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:20:14.397548 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_461_92/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:20:14.401597 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 197185, file segment info: File segment: [0, 33554431], key: 0c2822c3edc8ff367ae0c238aad340da, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:20:14.440576 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:20:14.551410 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} MergedBlockOutputStream: filled checksums 202410_1_461_92 (state Temporary) 2024.10.22 01:20:14.564012 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_768_376} MergedBlockOutputStream: filled checksums 202410_1_768_376 (state Temporary) 2024.10.22 01:20:14.626973 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_668_291, 202410_669_669_0] 2024.10.22 01:20:14.628875 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_668_291, 202410_669_669_0] 2024.10.22 01:20:14.665822 [ 1186 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 01:20:14.742506 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_668_291, 202410_669_669_0] 2024.10.22 01:20:14.774993 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_768_376} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_768_376 to 202410_1_768_376 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:14.778271 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_768_376} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_767_375, 202410_768_768_0] -> 202410_1_768_376 2024.10.22 01:20:14.821003 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.35 MiB. 2024.10.22 01:20:14.923108 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 48466 2024.10.22 01:20:15.127607 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_461_92 to 202410_1_461_92 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:15.130200 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_461_92} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_456_91, 202410_461_461_0] -> 202410_1_461_92 2024.10.22 01:20:15.171019 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.42 MiB. 2024.10.22 01:20:15.254976 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:15.259507 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:15.329532 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_761_761_0 (state Temporary) 2024.10.22 01:20:15.379031 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_761_761_0 to 202410_774_774_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:15.399742 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_768_376 to 202410_769_769_0 2024.10.22 01:20:15.411503 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.75 MiB using storage policy from min volume index 0 2024.10.22 01:20:15.415507 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(56722647c3793782e190ecfe2bb1b2fb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 56722647c3793782e190ecfe2bb1b2fb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:20:15.415955 [ 1185 ] {} DiskLocal: Reserved 1.75 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:15.433566 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_769_377} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_768_376 to 202410_769_769_0 into Compact with storage Full 2024.10.22 01:20:15.470813 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_769_377} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:15.480995 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_769_377} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_768_376, total 48885 rows starting from the beginning of the part 2024.10.22 01:20:15.518793 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48466 2024.10.22 01:20:15.521489 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 799, file segment info: File segment: [0, 33554431], key: 56722647c3793782e190ecfe2bb1b2fb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 01:20:15.626360 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:20:15.676759 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_769_377} MergeTreeSequentialSource: Reading 2 marks from part 202410_769_769_0, total 73 rows starting from the beginning of the part 2024.10.22 01:20:15.903617 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:20:15.909803 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:20:15.927489 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:20:16.041581 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:20:16.072333 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 190 entries to flush up to offset 49121 2024.10.22 01:20:16.159899 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:20:16.271222 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:20:16.383368 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:20:16.527139 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ima/imazjzemuurnbwzohwwezedhkboni (LogSeriesLimiter: on interval from 2024-10-22 01:20:00 to 2024-10-22 01:20:16 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:20:16.555114 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:16.557620 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:16.604986 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_441_441_0 (state Temporary) 2024.10.22 01:20:16.675141 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.96 GiB, free memory in arenas 6.49 MiB, will set to 1.60 GiB (RSS), difference: -8.04 MiB 2024.10.22 01:20:16.690675 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_441_441_0 to 202410_447_447_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:16.868787 [ 1267 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_441_219 to 202410_442_442_0 2024.10.22 01:20:16.876283 [ 1267 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.44 MiB using storage policy from min volume index 0 2024.10.22 01:20:16.881447 [ 1267 ] {} DiskLocal: Reserved 2.44 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:20:16.887890 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 49121 2024.10.22 01:20:16.985464 [ 1267 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 269 ms. 2024.10.22 01:20:16.996956 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_442_220} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_441_219 to 202410_442_442_0 into Wide with storage Full 2024.10.22 01:20:17.087261 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_442_220} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:17.100687 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_442_220} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_441_219, total 47965 rows starting from the beginning of the part 2024.10.22 01:20:17.497412 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_442_220} MergeTreeSequentialSource: Reading 2 marks from part 202410_442_442_0, total 225 rows starting from the beginning of the part 2024.10.22 01:20:18.561367 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_769_377/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:20:18.755735 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.96 GiB, free memory in arenas 4.56 MiB, will set to 1.61 GiB (RSS), difference: 14.84 MiB 2024.10.22 01:20:18.868832 [ 1212 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:20:18.871820 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_769_377} MergeTask::MergeProjectionsStage: Merge sorted 48958 rows, containing 13 columns (13 merged, 0 gathered) in 3.435628533 sec., 14250.085400603464 rows/sec., 2.28 MiB/sec. 2024.10.22 01:20:19.023515 [ 1212 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:20:19.051359 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.105788745 sec. 2024.10.22 01:20:19.172774 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_769_377} MergedBlockOutputStream: filled checksums 202410_1_769_377 (state Temporary) 2024.10.22 01:20:19.187636 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 6917 2024.10.22 01:20:19.188822 [ 1724 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/dvb/ksxhfxhjhxiragxkioipzobdfvjvn, size 56 2024.10.22 01:20:19.441013 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:19.453669 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.13 GiB. 2024.10.22 01:20:19.553587 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 48519 2024.10.22 01:20:19.606232 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_449_449_0 (state Temporary) 2024.10.22 01:20:19.697513 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_449_449_0 to 202410_462_462_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:19.869362 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:19.872212 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6917 2024.10.22 01:20:19.874290 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.13 GiB. 2024.10.22 01:20:19.994754 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_762_762_0 (state Temporary) 2024.10.22 01:20:20.182799 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_762_762_0 to 202410_775_775_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:20.183107 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:20:20.200614 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_769_377} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_769_377 to 202410_1_769_377 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:20.205831 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_769_377} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_768_376, 202410_769_769_0] -> 202410_1_769_377 2024.10.22 01:20:20.210045 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:20:20.226402 [ 1224 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_769_377 to 202410_770_770_0 2024.10.22 01:20:20.232350 [ 1224 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.75 MiB using storage policy from min volume index 0 2024.10.22 01:20:20.241405 [ 1224 ] {} DiskLocal: Reserved 1.75 MiB on local disk `default`, having unreserved 297.13 GiB. 2024.10.22 01:20:20.253314 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_442_220/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:20:20.265361 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48519 2024.10.22 01:20:20.287561 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_770_378} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_769_377 to 202410_770_770_0 into Compact with storage Full 2024.10.22 01:20:20.358589 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:20:20.505158 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_770_378} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:20.566022 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 677640 2024.10.22 01:20:20.622267 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_770_378} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_769_377, total 48958 rows starting from the beginning of the part 2024.10.22 01:20:20.655472 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 381 2024.10.22 01:20:20.778932 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_770_378} MergeTreeSequentialSource: Reading 2 marks from part 202410_770_770_0, total 63 rows starting from the beginning of the part 2024.10.22 01:20:20.713138 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.96 GiB, free memory in arenas 4.84 MiB, will set to 1.61 GiB (RSS), difference: 208.67 MiB 2024.10.22 01:20:20.914405 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_669_292, 202410_670_670_0] 2024.10.22 01:20:20.920604 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_669_292, 202410_670_670_0] 2024.10.22 01:20:20.945723 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_669_292, 202410_670_670_0] 2024.10.22 01:20:20.979063 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_442_220} MergeTask::MergeProjectionsStage: Merge sorted 48190 rows, containing 13 columns (13 merged, 0 gathered) in 3.980710928 sec., 12105.877787064472 rows/sec., 5.25 MiB/sec. 2024.10.22 01:20:21.080403 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_442_220} MergedBlockOutputStream: filled checksums 202410_1_442_220 (state Temporary) 2024.10.22 01:20:21.405547 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:21.422511 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.10 GiB. 2024.10.22 01:20:21.649187 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:21.653153 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_363_363_0 (state Temporary) 2024.10.22 01:20:21.659820 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:20:21.724215 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_363_363_0 to 202410_373_373_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:21.784721 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(e2bee378f08b3d84fcfc4ff8553f4cd7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e2bee378f08b3d84fcfc4ff8553f4cd7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:20:21.812221 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 58, file segment info: File segment: [0, 33554431], key: e2bee378f08b3d84fcfc4ff8553f4cd7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:20:21.828090 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:20:21.824296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MergedBlockOutputStream: filled checksums 201403_24_24_0 (state Temporary) 2024.10.22 01:20:21.863362 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zzy/lepnxgxrppvddohquojbkuwytxzjj (LogSeriesLimiter: on interval from 2024-10-22 01:20:16 to 2024-10-22 01:20:21 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:20:21.875244 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_431_431_0 (state Temporary) 2024.10.22 01:20:21.964789 [ 1687 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zeb/wwjqixwulqawwttreuujnfvwidhfb, size 478 2024.10.22 01:20:21.970838 [ 1217 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 204 ms. 2024.10.22 01:20:21.973983 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_431_431_0 to 202410_438_438_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:22.014301 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 677640 2024.10.22 01:20:22.187348 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 381 2024.10.22 01:20:22.187775 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_384_162, 202410_385_385_0] 2024.10.22 01:20:22.190420 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_384_162, 202410_385_385_0] 2024.10.22 01:20:22.197670 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_442_220} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_442_220 to 202410_1_442_220 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:22.203119 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_442_220} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_441_219, 202410_442_442_0] -> 202410_1_442_220 2024.10.22 01:20:22.222397 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_384_162, 202410_385_385_0] 2024.10.22 01:20:22.234869 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.11 MiB. 2024.10.22 01:20:22.460974 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(54ee2164aafaa71b23e28eb15017738c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 54ee2164aafaa71b23e28eb15017738c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:20:22.465887 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 56, file segment info: File segment: [0, 33554431], key: 54ee2164aafaa71b23e28eb15017738c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:20:22.467964 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:20:23.055194 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:23.058554 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:23.078519 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 4 bytes for b1abbf154939d9a831e81d40f5c3be75:0) 2024.10.22 01:20:23.215691 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2856b615b9582e8cfb1fc35312da9ac4, offset: 0, size: 4 2024.10.22 01:20:23.224483 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2856b615b9582e8cfb1fc35312da9ac4 for removal 2024.10.22 01:20:23.231501 [ 1476 ] {} CacheMetadata: Key 2856b615b9582e8cfb1fc35312da9ac4 is removed from metadata 2024.10.22 01:20:23.320190 [ 1712 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zzy/lepnxgxrppvddohquojbkuwytxzjj, size 4 2024.10.22 01:20:23.339838 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.96 GiB, free memory in arenas 4.61 MiB, will set to 1.56 GiB (RSS), difference: 116.55 MiB 2024.10.22 01:20:23.727602 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_770_378} MergeTask::MergeProjectionsStage: Merge sorted 49021 rows, containing 13 columns (13 merged, 0 gathered) in 3.450793279 sec., 14205.719101842495 rows/sec., 2.28 MiB/sec. 2024.10.22 01:20:23.804801 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_770_378} MergedBlockOutputStream: filled checksums 202410_1_770_378 (state Temporary) 2024.10.22 01:20:24.158956 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_670_293, 202410_671_671_0] 2024.10.22 01:20:24.160735 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_670_293, 202410_671_671_0] 2024.10.22 01:20:24.167269 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_670_293, 202410_671_671_0] 2024.10.22 01:20:24.269840 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 48588 2024.10.22 01:20:24.305720 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_770_378} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_770_378 to 202410_1_770_378 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:24.314349 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_770_378} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_769_377, 202410_770_770_0] -> 202410_1_770_378 2024.10.22 01:20:24.379051 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(d22efe560d38f58638ab9b37b86aee88) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d22efe560d38f58638ab9b37b86aee88, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:20:24.380853 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: d22efe560d38f58638ab9b37b86aee88, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:20:24.391688 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:20:24.448965 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:20:24.519755 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:24.523490 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:24.531052 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 250 entries to flush up to offset 49371 2024.10.22 01:20:24.586413 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_763_763_0 (state Temporary) 2024.10.22 01:20:24.740742 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_763_763_0 to 202410_776_776_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:24.791595 [ 1272 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_770_378 to 202410_771_771_0 2024.10.22 01:20:24.883288 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48588 2024.10.22 01:20:24.901361 [ 1272 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.76 MiB using storage policy from min volume index 0 2024.10.22 01:20:24.929185 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:24.931837 [ 1272 ] {} DiskLocal: Reserved 1.76 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:24.947466 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:24.961816 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_771_379} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_770_378 to 202410_771_771_0 into Compact with storage Full 2024.10.22 01:20:24.995159 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_771_379} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:25.016121 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_771_379} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_770_378, total 49021 rows starting from the beginning of the part 2024.10.22 01:20:25.119677 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(42b27cc33d4887e6f16ad5016136d19b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 42b27cc33d4887e6f16ad5016136d19b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:20:25.127374 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 535, file segment info: File segment: [0, 33554431], key: 42b27cc33d4887e6f16ad5016136d19b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:20:25.130323 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:20:25.139093 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_442_442_0 (state Temporary) 2024.10.22 01:20:25.239707 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_771_379} MergeTreeSequentialSource: Reading 2 marks from part 202410_771_771_0, total 101 rows starting from the beginning of the part 2024.10.22 01:20:25.270196 [ 1708 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bsg/efmeumeqireazpzpywycfzlcqhoit, size 56 2024.10.22 01:20:25.572722 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_442_442_0 to 202410_448_448_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:25.588694 [ 1269 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_442_220 to 202410_443_443_0 2024.10.22 01:20:25.605000 [ 1269 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.45 MiB using storage policy from min volume index 0 2024.10.22 01:20:25.607191 [ 1269 ] {} DiskLocal: Reserved 2.45 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:25.607584 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 49371 2024.10.22 01:20:25.623698 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_443_221} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_442_220 to 202410_443_443_0 into Wide with storage Full 2024.10.22 01:20:25.660131 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_443_221} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:25.670694 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_443_221} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_442_220, total 48190 rows starting from the beginning of the part 2024.10.22 01:20:25.767257 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:20:25.842741 [ 1229 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:20:25.817751 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:20:25.937540 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:20:25.980513 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:20:25.989427 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_443_221} MergeTreeSequentialSource: Reading 2 marks from part 202410_443_443_0, total 182 rows starting from the beginning of the part 2024.10.22 01:20:26.007601 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:20:26.031592 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:20:26.054046 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:20:26.101554 [ 1270 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 208 ms. 2024.10.22 01:20:26.183689 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_320_66, 202410_321_321_0, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0, 202410_325_325_0] 2024.10.22 01:20:26.188100 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:20:26.187166 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_320_66, 202410_321_321_0, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0, 202410_325_325_0] 2024.10.22 01:20:26.321609 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:20:26.444665 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.96 GiB, free memory in arenas 3.75 MiB, will set to 1.56 GiB (RSS), difference: -42.13 MiB 2024.10.22 01:20:26.448187 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_320_66, 202410_321_321_0, 202410_322_322_0, 202410_323_323_0, 202410_324_324_0, 202410_325_325_0] 2024.10.22 01:20:26.466129 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bgm/zigmiypixbeeomiocjcwyzlamfcns (LogSeriesLimiter: on interval from 2024-10-22 01:20:21 to 2024-10-22 01:20:26 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:20:26.572621 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_91_91_0 (state Temporary) 2024.10.22 01:20:27.375736 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 6932 2024.10.22 01:20:27.616000 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_24_24_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:20:27.731451 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:27.733829 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:27.774761 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_450_450_0 (state Temporary) 2024.10.22 01:20:27.788558 [ 1181 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 01:20:27.928716 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_450_450_0 to 202410_463_463_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:28.074317 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6932 2024.10.22 01:20:28.558525 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.96 GiB, free memory in arenas 2.48 MiB, will set to 1.58 GiB (RSS), difference: -6.88 MiB 2024.10.22 01:20:28.629330 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_771_379} MergeTask::MergeProjectionsStage: Merge sorted 49122 rows, containing 13 columns (13 merged, 0 gathered) in 3.675309179 sec., 13365.406176077247 rows/sec., 2.14 MiB/sec. 2024.10.22 01:20:28.712980 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_771_379} MergedBlockOutputStream: filled checksums 202410_1_771_379 (state Temporary) 2024.10.22 01:20:28.897572 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 48652 2024.10.22 01:20:29.020375 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1385 entries to flush up to offset 679025 2024.10.22 01:20:29.096967 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_771_379} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_771_379 to 202410_1_771_379 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:29.111658 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_771_379} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_770_378, 202410_771_771_0] -> 202410_1_771_379 2024.10.22 01:20:29.253145 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.37 MiB. 2024.10.22 01:20:29.280297 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:29.282284 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_443_221} MergeTask::MergeProjectionsStage: Merge sorted 48372 rows, containing 13 columns (13 merged, 0 gathered) in 3.667468616 sec., 13189.478919865418 rows/sec., 5.72 MiB/sec. 2024.10.22 01:20:29.287038 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:29.575474 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_764_764_0 (state Temporary) 2024.10.22 01:20:29.630934 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_764_764_0 to 202410_777_777_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:29.664011 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_771_379 to 202410_772_772_0 2024.10.22 01:20:29.667672 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.76 MiB using storage policy from min volume index 0 2024.10.22 01:20:29.669770 [ 1249 ] {} DiskLocal: Reserved 1.76 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:29.711980 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_772_380} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_771_379 to 202410_772_772_0 into Compact with storage Full 2024.10.22 01:20:29.769776 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_772_380} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:29.778285 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48652 2024.10.22 01:20:29.782686 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_772_380} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_771_379, total 49122 rows starting from the beginning of the part 2024.10.22 01:20:29.800163 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_443_221} MergedBlockOutputStream: filled checksums 202410_1_443_221 (state Temporary) 2024.10.22 01:20:29.910251 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:29.923254 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:29.957720 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_772_380} MergeTreeSequentialSource: Reading 2 marks from part 202410_772_772_0, total 57 rows starting from the beginning of the part 2024.10.22 01:20:30.181759 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:20:30.183077 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:20:30.369079 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_432_432_0 (state Temporary) 2024.10.22 01:20:30.462110 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_432_432_0 to 202410_439_439_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:30.546387 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.96 GiB, free memory in arenas 2.54 MiB, will set to 1.56 GiB (RSS), difference: 30.05 MiB 2024.10.22 01:20:30.598427 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 679025 2024.10.22 01:20:30.616387 [ 1259 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_429_87 to 202410_439_439_0 2024.10.22 01:20:30.632432 [ 1259 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:30.635809 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:30.745038 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_443_221} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_443_221 to 202410_1_443_221 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:30.763699 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_443_221} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_442_220, 202410_443_443_0] -> 202410_1_443_221 2024.10.22 01:20:30.845329 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.05 MiB. 2024.10.22 01:20:30.847770 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_429_87 to 202410_439_439_0 into Compact with storage Full 2024.10.22 01:20:30.975045 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:30.978143 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} MergeTreeSequentialSource: Reading 84 marks from part 202410_1_429_87, total 678140 rows starting from the beginning of the part 2024.10.22 01:20:31.000357 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} MergeTreeSequentialSource: Reading 3 marks from part 202410_430_435_1, total 8470 rows starting from the beginning of the part 2024.10.22 01:20:31.019319 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_436_436_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:20:31.037764 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_437_437_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:20:31.063922 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_438_438_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:20:31.087050 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_439_439_0, total 1385 rows starting from the beginning of the part 2024.10.22 01:20:31.727659 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_671_294, 202410_672_672_0] 2024.10.22 01:20:31.730233 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_671_294, 202410_672_672_0] 2024.10.22 01:20:31.755204 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_671_294, 202410_672_672_0] 2024.10.22 01:20:31.908363 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_385_163, 202410_386_386_0] 2024.10.22 01:20:31.911058 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_385_163, 202410_386_386_0] 2024.10.22 01:20:32.071735 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_385_163, 202410_386_386_0] 2024.10.22 01:20:32.289592 [ 1252 ] {} DatabaseCatalog: Execution took 203 ms. 2024.10.22 01:20:32.534745 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.96 GiB, free memory in arenas 2.63 MiB, will set to 1.56 GiB (RSS), difference: 4.24 MiB 2024.10.22 01:20:32.761687 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_772_380} MergeTask::MergeProjectionsStage: Merge sorted 49179 rows, containing 13 columns (13 merged, 0 gathered) in 3.060172545 sec., 16070.662446911143 rows/sec., 2.58 MiB/sec. 2024.10.22 01:20:32.872430 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_772_380} MergedBlockOutputStream: filled checksums 202410_1_772_380 (state Temporary) 2024.10.22 01:20:32.929811 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_772_380} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_772_380 to 202410_1_772_380 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:32.933277 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_772_380} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_771_379, 202410_772_772_0] -> 202410_1_772_380 2024.10.22 01:20:32.973341 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:20:33.131558 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_673_673_0, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 01:20:33.138583 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_673_673_0, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 01:20:33.163642 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 220 entries to flush up to offset 49591 2024.10.22 01:20:33.269494 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_673_673_0, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 01:20:33.472448 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:33.475936 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:33.584596 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_91_91_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:20:33.590130 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_74_15, 202410_75_75_0, 202410_76_76_0, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0] 2024.10.22 01:20:33.592984 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_74_15, 202410_75_75_0, 202410_76_76_0, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0] 2024.10.22 01:20:33.669530 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_443_443_0 (state Temporary) 2024.10.22 01:20:33.772552 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_443_443_0 to 202410_449_449_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:33.787438 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 48717 2024.10.22 01:20:33.806423 [ 1183 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_443_221 to 202410_444_444_0 2024.10.22 01:20:33.812100 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 49591 2024.10.22 01:20:33.814625 [ 1183 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.46 MiB using storage policy from min volume index 0 2024.10.22 01:20:33.822856 [ 1183 ] {} DiskLocal: Reserved 2.46 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:33.830717 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_74_15, 202410_75_75_0, 202410_76_76_0, 202410_77_77_0, 202410_78_78_0, 202410_79_79_0] 2024.10.22 01:20:33.860707 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_444_222} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_443_221 to 202410_444_444_0 into Wide with storage Full 2024.10.22 01:20:33.870983 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_439_88/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:20:33.899887 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_444_222} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:33.916215 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_444_222} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_443_221, total 48372 rows starting from the beginning of the part 2024.10.22 01:20:34.029450 [ 1241 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:20:34.161136 [ 1164 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 328 ms. 2024.10.22 01:20:34.263224 [ 1241 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:20:34.265130 [ 1241 ] {} DNSCacheUpdater: Execution took 236 ms. 2024.10.22 01:20:34.272103 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:34.275056 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:34.473612 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_765_765_0 (state Temporary) 2024.10.22 01:20:34.561149 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_444_222} MergeTreeSequentialSource: Reading 2 marks from part 202410_444_444_0, total 207 rows starting from the beginning of the part 2024.10.22 01:20:34.655284 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.96 GiB, free memory in arenas 9.50 MiB, will set to 1.56 GiB (RSS), difference: 2.73 MiB 2024.10.22 01:20:34.682231 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_765_765_0 to 202410_778_778_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:34.761776 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_772_380 to 202410_773_773_0 2024.10.22 01:20:34.766579 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48717 2024.10.22 01:20:34.767387 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.76 MiB using storage policy from min volume index 0 2024.10.22 01:20:34.769135 [ 1182 ] {} DiskLocal: Reserved 1.76 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:34.923403 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_773_381} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_772_380 to 202410_773_773_0 into Compact with storage Full 2024.10.22 01:20:35.068940 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_773_381} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:35.076024 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_773_381} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_772_380, total 49179 rows starting from the beginning of the part 2024.10.22 01:20:35.203581 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_773_381} MergeTreeSequentialSource: Reading 2 marks from part 202410_773_773_0, total 47 rows starting from the beginning of the part 2024.10.22 01:20:35.321069 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_91_91_0 to 202410_94_94_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:35.385740 [ 1185 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_89_18 to 202410_94_94_0 2024.10.22 01:20:35.399732 [ 1185 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.85 MiB using storage policy from min volume index 0 2024.10.22 01:20:35.406836 [ 1185 ] {} DiskLocal: Reserved 1.85 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:35.488355 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_89_18 to 202410_94_94_0 into Compact with storage Full 2024.10.22 01:20:35.579712 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 6947 2024.10.22 01:20:35.892594 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hqq/kbgmotevhfpbqkebcruqcqzludyge. 2024.10.22 01:20:35.961029 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:35.962772 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:36.006777 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_451_451_0 (state Temporary) 2024.10.22 01:20:36.176112 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_451_451_0 to 202410_464_464_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:36.275258 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_444_222/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:20:36.550367 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.96 GiB, free memory in arenas 11.01 MiB, will set to 1.59 GiB (RSS), difference: -34.50 MiB 2024.10.22 01:20:36.563400 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6947 2024.10.22 01:20:37.190621 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 382 2024.10.22 01:20:37.604643 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1312 entries to flush up to offset 680337 2024.10.22 01:20:37.632210 [ 1210 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 5 parts from 202410_1_369_76 to 202410_373_373_0 2024.10.22 01:20:37.649838 [ 1210 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:37.654697 [ 1210 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:37.769534 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(7bf9e2cb5252c8b4f781cbb50acc7995) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7bf9e2cb5252c8b4f781cbb50acc7995, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:20:37.772124 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 954, file segment info: File segment: [0, 33554431], key: 7bf9e2cb5252c8b4f781cbb50acc7995, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:20:37.778191 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:20:37.875826 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:37.880893 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:20:37.885253 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_373_77} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_369_76 to 202410_373_373_0 into Compact with storage Full 2024.10.22 01:20:37.983323 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_773_381} MergeTask::MergeProjectionsStage: Merge sorted 49226 rows, containing 13 columns (13 merged, 0 gathered) in 3.174898418 sec., 15504.748032540043 rows/sec., 2.48 MiB/sec. 2024.10.22 01:20:37.987716 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_364_364_0 (state Temporary) 2024.10.22 01:20:37.998402 [ 1727 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ima/imazjzemuurnbwzohwwezedhkboni, size 4495090 2024.10.22 01:20:38.055321 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_373_77} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:38.067209 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_373_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_369_76, total 413 rows starting from the beginning of the part 2024.10.22 01:20:38.154248 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_373_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_370_370_0, total 1 rows starting from the beginning of the part 2024.10.22 01:20:38.232189 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_373_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_371_371_0, total 1 rows starting from the beginning of the part 2024.10.22 01:20:38.292619 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_373_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_372_372_0, total 1 rows starting from the beginning of the part 2024.10.22 01:20:38.323230 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_444_222} MergeTask::MergeProjectionsStage: Merge sorted 48579 rows, containing 13 columns (13 merged, 0 gathered) in 4.470240427 sec., 10867.20072293776 rows/sec., 4.72 MiB/sec. 2024.10.22 01:20:38.366874 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_373_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_373_373_0, total 1 rows starting from the beginning of the part 2024.10.22 01:20:38.378970 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_773_381} MergedBlockOutputStream: filled checksums 202410_1_773_381 (state Temporary) 2024.10.22 01:20:38.437407 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_364_364_0 to 202410_374_374_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:38.517177 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:20:38.531482 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 382 2024.10.22 01:20:38.539370 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:20:38.555169 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:20:38.563628 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:20:38.582992 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_773_381} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_773_381 to 202410_1_773_381 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:38.583318 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:20:38.589378 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_773_381} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_772_380, 202410_773_773_0] -> 202410_1_773_381 2024.10.22 01:20:38.599231 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_444_222} MergedBlockOutputStream: filled checksums 202410_1_444_222 (state Temporary) 2024.10.22 01:20:38.641312 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3624 2024.10.22 01:20:38.708225 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:20:38.708526 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:20:38.725729 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:20:38.792131 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:38.829074 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 48795 2024.10.22 01:20:38.832909 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:38.850152 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xse/ugozpadjtjlwelxmztbqzkufixsvt (LogSeriesLimiter: on interval from 2024-10-22 01:20:26 to 2024-10-22 01:20:38 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:20:39.005753 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_433_433_0 (state Temporary) 2024.10.22 01:20:39.077572 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ghu/ioxvqmpxylmhnveipkhitaopcnrio. 2024.10.22 01:20:39.181608 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_386_164, 202410_387_387_0] 2024.10.22 01:20:39.184897 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_386_164, 202410_387_387_0] 2024.10.22 01:20:39.191427 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:39.197645 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:39.310332 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_386_164, 202410_387_387_0] 2024.10.22 01:20:39.357134 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_433_433_0 to 202410_440_440_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:39.387732 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 680337 2024.10.22 01:20:39.436085 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_766_766_0 (state Temporary) 2024.10.22 01:20:39.496157 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_766_766_0 to 202410_779_779_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:39.571061 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_444_222} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_444_222 to 202410_1_444_222 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:39.571278 [ 1222 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_773_381 to 202410_774_774_0 2024.10.22 01:20:39.573578 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_444_222} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_443_221, 202410_444_444_0] -> 202410_1_444_222 2024.10.22 01:20:39.576209 [ 1222 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.76 MiB using storage policy from min volume index 0 2024.10.22 01:20:39.578553 [ 1222 ] {} DiskLocal: Reserved 1.76 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:39.588052 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.11 MiB. 2024.10.22 01:20:39.662457 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_774_382} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_773_381 to 202410_774_774_0 into Compact with storage Full 2024.10.22 01:20:39.677534 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48795 2024.10.22 01:20:39.859414 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 18.07 MiB, will set to 1.56 GiB (RSS), difference: 53.74 MiB 2024.10.22 01:20:39.861319 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_774_382} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:39.879603 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_774_382} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_773_381, total 49226 rows starting from the beginning of the part 2024.10.22 01:20:39.949778 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_774_382} MergeTreeSequentialSource: Reading 2 marks from part 202410_774_774_0, total 71 rows starting from the beginning of the part 2024.10.22 01:20:40.183286 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:20:40.186808 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:20:40.447142 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_373_77} MergeTask::MergeProjectionsStage: Merge sorted 417 rows, containing 32 columns (32 merged, 0 gathered) in 2.570593625 sec., 162.2193395115107 rows/sec., 15.13 KiB/sec. 2024.10.22 01:20:40.477751 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_373_77} MergedBlockOutputStream: filled checksums 202410_1_373_77 (state Temporary) 2024.10.22 01:20:40.690020 [ 1159 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:20:40.866475 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_373_77} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_373_77 to 202410_1_373_77 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:40.880419 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_373_77} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 5 parts: [202410_1_369_76, 202410_373_373_0] -> 202410_1_373_77 2024.10.22 01:20:41.039480 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:20:41.568025 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 225 entries to flush up to offset 49816 2024.10.22 01:20:41.749229 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.96 GiB, free memory in arenas 12.80 MiB, will set to 1.57 GiB (RSS), difference: -3.13 MiB 2024.10.22 01:20:42.102972 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:42.115430 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:42.300377 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_444_444_0 (state Temporary) 2024.10.22 01:20:42.376225 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_444_444_0 to 202410_450_450_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:42.506065 [ 1150 ] {} DatabaseCatalog: Execution took 216 ms. 2024.10.22 01:20:42.523078 [ 1266 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_444_222 to 202410_445_445_0 2024.10.22 01:20:42.538631 [ 1266 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.47 MiB using storage policy from min volume index 0 2024.10.22 01:20:42.542749 [ 1266 ] {} DiskLocal: Reserved 2.47 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:42.569148 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_445_223} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_444_222 to 202410_445_445_0 into Wide with storage Full 2024.10.22 01:20:42.604994 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_445_223} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:42.619833 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_445_223} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_444_222, total 48579 rows starting from the beginning of the part 2024.10.22 01:20:42.771475 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_445_223} MergeTreeSequentialSource: Reading 2 marks from part 202410_445_445_0, total 246 rows starting from the beginning of the part 2024.10.22 01:20:43.079341 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_774_382} MergeTask::MergeProjectionsStage: Merge sorted 49297 rows, containing 13 columns (13 merged, 0 gathered) in 3.424049051 sec., 14397.282067440803 rows/sec., 2.31 MiB/sec. 2024.10.22 01:20:43.175397 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_774_382} MergedBlockOutputStream: filled checksums 202410_1_774_382 (state Temporary) 2024.10.22 01:20:43.245663 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 49816 2024.10.22 01:20:43.428815 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_774_382} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_774_382 to 202410_1_774_382 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:43.432880 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_774_382} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_773_381, 202410_774_774_0] -> 202410_1_774_382 2024.10.22 01:20:43.576218 [ 1269 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 248 ms. 2024.10.22 01:20:43.674138 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.96 GiB, free memory in arenas 10.04 MiB, will set to 1.56 GiB (RSS), difference: -28.29 MiB 2024.10.22 01:20:43.695191 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:20:43.696284 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 48851 2024.10.22 01:20:44.063449 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:44.065168 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 6967 2024.10.22 01:20:44.065830 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:44.216338 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:44.227921 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:44.289710 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_767_767_0 (state Temporary) 2024.10.22 01:20:44.424342 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_452_452_0 (state Temporary) 2024.10.22 01:20:44.471225 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_452_452_0 to 202410_465_465_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:44.554637 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6967 2024.10.22 01:20:44.635249 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_767_767_0 to 202410_780_780_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:44.673503 [ 1238 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_774_382 to 202410_775_775_0 2024.10.22 01:20:44.683733 [ 1238 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.76 MiB using storage policy from min volume index 0 2024.10.22 01:20:44.683925 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_439_88/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:20:44.689056 [ 1238 ] {} DiskLocal: Reserved 1.76 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:44.725809 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48851 2024.10.22 01:20:44.826252 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_775_383} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_774_382 to 202410_775_775_0 into Compact with storage Full 2024.10.22 01:20:44.907877 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_775_383} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:45.022822 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_775_383} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_774_382, total 49297 rows starting from the beginning of the part 2024.10.22 01:20:45.088261 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_775_383} MergeTreeSequentialSource: Reading 2 marks from part 202410_775_775_0, total 53 rows starting from the beginning of the part 2024.10.22 01:20:45.529604 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.96 GiB, free memory in arenas 10.90 MiB, will set to 1.57 GiB (RSS), difference: -14.16 MiB 2024.10.22 01:20:45.566528 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_387_165, 202410_388_388_0] 2024.10.22 01:20:45.570432 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_387_165, 202410_388_388_0] 2024.10.22 01:20:45.584587 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_387_165, 202410_388_388_0] 2024.10.22 01:20:45.997747 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:46.036919 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 383 2024.10.22 01:20:46.192195 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 3665 2024.10.22 01:20:46.382823 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:46.385245 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:46.397698 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1850 entries to flush up to offset 682187 2024.10.22 01:20:46.414472 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_445_223} MergeTask::MergeProjectionsStage: Merge sorted 48825 rows, containing 13 columns (13 merged, 0 gathered) in 3.856497556 sec., 12660.451430608919 rows/sec., 5.49 MiB/sec. 2024.10.22 01:20:46.587904 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_89_18, total 3485 rows starting from the beginning of the part 2024.10.22 01:20:46.596766 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_365_365_0 (state Temporary) 2024.10.22 01:20:46.686733 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_365_365_0 to 202410_375_375_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:46.825196 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_445_223} MergedBlockOutputStream: filled checksums 202410_1_445_223 (state Temporary) 2024.10.22 01:20:47.150856 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 383 2024.10.22 01:20:47.331186 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_376_75, 202410_377_377_0, 202410_378_378_0, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0] 2024.10.22 01:20:47.336677 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_376_75, 202410_377_377_0, 202410_378_378_0, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0] 2024.10.22 01:20:47.363983 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:47.368796 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:47.402167 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_376_75, 202410_377_377_0, 202410_378_378_0, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0] 2024.10.22 01:20:47.431315 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_445_223} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_445_223 to 202410_1_445_223 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:47.437317 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_445_223} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_444_222, 202410_445_445_0] -> 202410_1_445_223 2024.10.22 01:20:47.464738 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.13 MiB. 2024.10.22 01:20:47.572592 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(92e4200dd6b8f06508a432f2bf4a73b0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 92e4200dd6b8f06508a432f2bf4a73b0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:20:47.589075 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 225737, file segment info: File segment: [0, 33554431], key: 92e4200dd6b8f06508a432f2bf4a73b0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:20:47.595163 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:20:47.624582 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_434_434_0 (state Temporary) 2024.10.22 01:20:47.655459 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.96 GiB, free memory in arenas 6.74 MiB, will set to 1.57 GiB (RSS), difference: 35.07 MiB 2024.10.22 01:20:47.712002 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_434_434_0 to 202410_441_441_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:47.964094 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 682187 2024.10.22 01:20:48.125722 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_775_383} MergeTask::MergeProjectionsStage: Merge sorted 49350 rows, containing 13 columns (13 merged, 0 gathered) in 3.419742004 sec., 14430.913192362566 rows/sec., 2.31 MiB/sec. 2024.10.22 01:20:48.135064 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_90_90_0, total 39 rows starting from the beginning of the part 2024.10.22 01:20:48.374402 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_775_383} MergedBlockOutputStream: filled checksums 202410_1_775_383 (state Temporary) 2024.10.22 01:20:48.472214 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b60c9d604ba2e6cad8532d4ed7bb2972) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b60c9d604ba2e6cad8532d4ed7bb2972, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:20:48.475351 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 576, file segment info: File segment: [0, 33554431], key: b60c9d604ba2e6cad8532d4ed7bb2972, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:20:48.480484 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:20:48.481550 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_401_80, 202410_402_402_0, 202410_403_403_0, 202410_404_404_0, 202410_405_405_0, 202410_406_406_0] 2024.10.22 01:20:48.529048 [ 1685 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bgm/zigmiypixbeeomiocjcwyzlamfcns, size 225737 2024.10.22 01:20:48.610433 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_401_80, 202410_402_402_0, 202410_403_403_0, 202410_404_404_0, 202410_405_405_0, 202410_406_406_0] 2024.10.22 01:20:48.696870 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:20:48.743575 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:20:48.750553 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 48913 2024.10.22 01:20:48.758508 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:20:48.761991 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_401_80, 202410_402_402_0, 202410_403_403_0, 202410_404_404_0, 202410_405_405_0, 202410_406_406_0] 2024.10.22 01:20:48.776562 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:20:48.786231 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:20:48.813191 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:20:48.831824 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:20:48.872803 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_775_383} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_775_383 to 202410_1_775_383 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:48.899716 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_775_383} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_774_382, 202410_775_775_0] -> 202410_1_775_383 2024.10.22 01:20:48.954977 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.36 MiB. 2024.10.22 01:20:48.961758 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:20:49.034648 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:20:49.081999 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ymh/xhtqlwnawfmikgivdyayaryghzbof (LogSeriesLimiter: on interval from 2024-10-22 01:20:38 to 2024-10-22 01:20:49 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:20:49.173767 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:49.179746 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:49.262267 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_768_768_0 (state Temporary) 2024.10.22 01:20:49.266013 [ 1254 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:20:49.319023 [ 1254 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:20:49.512688 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_768_768_0 to 202410_781_781_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:49.554940 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_775_383 to 202410_776_776_0 2024.10.22 01:20:49.561317 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.77 MiB using storage policy from min volume index 0 2024.10.22 01:20:49.564102 [ 1252 ] {} DiskLocal: Reserved 1.77 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:49.575597 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48913 2024.10.22 01:20:49.604483 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_776_384} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_775_383 to 202410_776_776_0 into Compact with storage Full 2024.10.22 01:20:49.633239 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_776_384} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:49.663034 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_776_384} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_775_383, total 49350 rows starting from the beginning of the part 2024.10.22 01:20:49.731811 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_776_384} MergeTreeSequentialSource: Reading 2 marks from part 202410_776_776_0, total 69 rows starting from the beginning of the part 2024.10.22 01:20:50.181695 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:20:50.183024 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:20:50.258251 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_91_91_0, total 41 rows starting from the beginning of the part 2024.10.22 01:20:50.471703 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.96 GiB, free memory in arenas 11.66 MiB, will set to 1.56 GiB (RSS), difference: -4.33 MiB 2024.10.22 01:20:50.786934 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_776_384/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:20:50.886212 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 228 entries to flush up to offset 50044 2024.10.22 01:20:51.703145 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:51.779540 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:51.845494 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_445_445_0 (state Temporary) 2024.10.22 01:20:51.851945 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_92_92_0, total 39 rows starting from the beginning of the part 2024.10.22 01:20:52.044341 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_445_445_0 to 202410_451_451_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:52.061758 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 6979 2024.10.22 01:20:52.081311 [ 1182 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_445_223 to 202410_446_446_0 2024.10.22 01:20:52.102060 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 50044 2024.10.22 01:20:52.112492 [ 1182 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.48 MiB using storage policy from min volume index 0 2024.10.22 01:20:52.116513 [ 1182 ] {} DiskLocal: Reserved 2.48 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:52.151796 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_446_224} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_445_223 to 202410_446_446_0 into Wide with storage Full 2024.10.22 01:20:52.225322 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_446_224} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:52.240339 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_446_224} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_445_223, total 48825 rows starting from the beginning of the part 2024.10.22 01:20:52.291703 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:52.296709 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:52.605035 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_453_453_0 (state Temporary) 2024.10.22 01:20:52.609585 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_446_224} MergeTreeSequentialSource: Reading 2 marks from part 202410_446_446_0, total 210 rows starting from the beginning of the part 2024.10.22 01:20:52.717360 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_453_453_0 to 202410_466_466_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:52.765084 [ 1191 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_461_92 to 202410_466_466_0 2024.10.22 01:20:52.776626 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 6979 2024.10.22 01:20:52.778851 [ 1191 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:52.794955 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.96 GiB, free memory in arenas 10.88 MiB, will set to 1.57 GiB (RSS), difference: 7.47 MiB 2024.10.22 01:20:52.816097 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:52.873282 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_461_92 to 202410_466_466_0 into Compact with storage Full 2024.10.22 01:20:53.182872 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:53.204056 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_461_92, total 7005 rows starting from the beginning of the part 2024.10.22 01:20:53.251548 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_462_462_0, total 18 rows starting from the beginning of the part 2024.10.22 01:20:53.346885 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_463_463_0, total 15 rows starting from the beginning of the part 2024.10.22 01:20:53.419602 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_464_464_0, total 15 rows starting from the beginning of the part 2024.10.22 01:20:53.487573 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_776_384} MergeTask::MergeProjectionsStage: Merge sorted 49419 rows, containing 13 columns (13 merged, 0 gathered) in 3.890958092 sec., 12700.984906932788 rows/sec., 2.04 MiB/sec. 2024.10.22 01:20:53.583181 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 48977 2024.10.22 01:20:53.621392 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_776_384} MergedBlockOutputStream: filled checksums 202410_1_776_384 (state Temporary) 2024.10.22 01:20:53.645572 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_465_465_0, total 20 rows starting from the beginning of the part 2024.10.22 01:20:53.734077 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_466_466_0, total 12 rows starting from the beginning of the part 2024.10.22 01:20:53.770345 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_776_384} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_776_384 to 202410_1_776_384 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:53.780616 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_776_384} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_775_383, 202410_776_776_0] -> 202410_1_776_384 2024.10.22 01:20:53.823424 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.39 MiB. 2024.10.22 01:20:53.859273 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_93_93_0, total 40 rows starting from the beginning of the part 2024.10.22 01:20:53.963429 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:53.966971 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:54.247519 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_769_769_0 (state Temporary) 2024.10.22 01:20:54.286754 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(2f7a42b0023312c9083db016e740cfc7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2f7a42b0023312c9083db016e740cfc7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:20:54.290833 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 911350, file segment info: File segment: [0, 33554431], key: 2f7a42b0023312c9083db016e740cfc7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:20:54.297727 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:20:54.557832 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.96 GiB, free memory in arenas 9.84 MiB, will set to 1.57 GiB (RSS), difference: -55.62 MiB 2024.10.22 01:20:54.642124 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_769_769_0 to 202410_782_782_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:54.660881 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 384 2024.10.22 01:20:54.678129 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_776_384 to 202410_777_777_0 2024.10.22 01:20:54.684929 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.77 MiB using storage policy from min volume index 0 2024.10.22 01:20:54.688374 [ 1267 ] {} DiskLocal: Reserved 1.77 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:54.707532 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 48977 2024.10.22 01:20:54.821004 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_777_385} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_776_384 to 202410_777_777_0 into Compact with storage Full 2024.10.22 01:20:54.890438 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:54.901609 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:54.963577 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} MergeTask::MergeProjectionsStage: Merge sorted 7085 rows, containing 12 columns (12 merged, 0 gathered) in 2.099444943 sec., 3374.701500805206 rows/sec., 535.57 KiB/sec. 2024.10.22 01:20:54.971701 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1230 entries to flush up to offset 683417 2024.10.22 01:20:55.083130 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_777_385} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:55.128002 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} MergedBlockOutputStream: filled checksums 202410_1_466_93 (state Temporary) 2024.10.22 01:20:55.188239 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_366_366_0 (state Temporary) 2024.10.22 01:20:55.220718 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_777_385} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_776_384, total 49419 rows starting from the beginning of the part 2024.10.22 01:20:55.300531 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_777_385} MergeTreeSequentialSource: Reading 2 marks from part 202410_777_777_0, total 64 rows starting from the beginning of the part 2024.10.22 01:20:55.323654 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_466_93 to 202410_1_466_93 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:55.328576 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_466_93} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_461_92, 202410_466_466_0] -> 202410_1_466_93 2024.10.22 01:20:55.331538 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegment(f8ad94cc8be0018294283e3109c97928) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f8ad94cc8be0018294283e3109c97928, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:20:55.337098 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 530, file segment info: File segment: [0, 33554431], key: f8ad94cc8be0018294283e3109c97928, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:20:55.338950 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_94_94_0, total 42 rows starting from the beginning of the part 2024.10.22 01:20:55.343125 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:20:55.376319 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_366_366_0 to 202410_376_376_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:55.408269 [ 1723 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xse/ugozpadjtjlwelxmztbqzkufixsvt, size 911350 2024.10.22 01:20:55.409326 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.43 MiB. 2024.10.22 01:20:55.595167 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 384 2024.10.22 01:20:55.777206 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:55.780360 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:55.865752 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lfq/racxrqmhsbfxlzedeshjmogapfweh. 2024.10.22 01:20:56.008968 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_435_435_0 (state Temporary) 2024.10.22 01:20:56.094774 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_388_166, 202410_389_389_0] 2024.10.22 01:20:56.105131 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_388_166, 202410_389_389_0] 2024.10.22 01:20:56.139312 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_435_435_0 to 202410_442_442_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:56.185736 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 683417 2024.10.22 01:20:56.231855 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lfq/racxrqmhsbfxlzedeshjmogapfweh. 2024.10.22 01:20:56.285624 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_446_224} MergeTask::MergeProjectionsStage: Merge sorted 49035 rows, containing 13 columns (13 merged, 0 gathered) in 4.150613273 sec., 11813.916829827474 rows/sec., 5.13 MiB/sec. 2024.10.22 01:20:56.307917 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_439_88/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:20:56.330862 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_388_166, 202410_389_389_0] 2024.10.22 01:20:56.480836 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_446_224} MergedBlockOutputStream: filled checksums 202410_1_446_224 (state Temporary) 2024.10.22 01:20:56.551614 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.96 GiB, free memory in arenas 9.35 MiB, will set to 1.57 GiB (RSS), difference: 15.57 MiB 2024.10.22 01:20:56.777675 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} MergeTask::MergeProjectionsStage: Merge sorted 692615 rows, containing 4 columns (4 merged, 0 gathered) in 26.115956548 sec., 26520.759395774134 rows/sec., 389.02 KiB/sec. 2024.10.22 01:20:56.932835 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} MergedBlockOutputStream: filled checksums 202410_1_439_88 (state Temporary) 2024.10.22 01:20:57.100442 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_439_88 to 202410_1_439_88 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:57.109016 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_439_88} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_429_87, 202410_439_439_0] -> 202410_1_439_88 2024.10.22 01:20:57.194367 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.21 MiB. 2024.10.22 01:20:57.536662 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_446_224} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_446_224 to 202410_1_446_224 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:57.544539 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_446_224} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_445_223, 202410_446_446_0] -> 202410_1_446_224 2024.10.22 01:20:57.575218 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.11 MiB. 2024.10.22 01:20:57.911918 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_672_295, 202410_673_678_1] 2024.10.22 01:20:57.915041 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_672_295, 202410_673_678_1] 2024.10.22 01:20:58.031012 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_672_295, 202410_673_678_1] 2024.10.22 01:20:58.481346 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 7.29 MiB, will set to 1.57 GiB (RSS), difference: 53.15 MiB 2024.10.22 01:20:58.541477 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_24_24_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:20:58.671744 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_777_385} MergeTask::MergeProjectionsStage: Merge sorted 49483 rows, containing 13 columns (13 merged, 0 gathered) in 3.85654266 sec., 12830.922503006877 rows/sec., 2.06 MiB/sec. 2024.10.22 01:20:58.722564 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 49045 2024.10.22 01:20:58.820786 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_777_385} MergedBlockOutputStream: filled checksums 202410_1_777_385 (state Temporary) 2024.10.22 01:20:58.941418 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_777_385} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_777_385 to 202410_1_777_385 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:58.953110 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_777_385} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_776_384, 202410_777_777_0] -> 202410_1_777_385 2024.10.22 01:20:59.067663 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:20:59.069678 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:59.076268 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:20:59.139917 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_770_770_0 (state Temporary) 2024.10.22 01:20:59.195712 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_770_770_0 to 202410_783_783_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:20:59.248798 [ 1219 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_777_385 to 202410_778_778_0 2024.10.22 01:20:59.258628 [ 1219 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.77 MiB using storage policy from min volume index 0 2024.10.22 01:20:59.267231 [ 1219 ] {} DiskLocal: Reserved 1.77 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:20:59.283531 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_778_386} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_777_385 to 202410_778_778_0 into Compact with storage Full 2024.10.22 01:20:59.378061 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 49045 2024.10.22 01:20:59.455802 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_778_386} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:20:59.464531 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_778_386} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_777_385, total 49483 rows starting from the beginning of the part 2024.10.22 01:20:59.529205 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lax/fvyfahspymghmuwtwwqniejiqmxsx. 2024.10.22 01:20:59.577598 [ 1167 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 257 ms. 2024.10.22 01:20:59.583785 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_778_386} MergeTreeSequentialSource: Reading 2 marks from part 202410_778_778_0, total 65 rows starting from the beginning of the part 2024.10.22 01:20:59.666534 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 227 entries to flush up to offset 50271 2024.10.22 01:20:59.711272 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hqq/kbgmotevhfpbqkebcruqcqzludyge. 2024.10.22 01:21:00.040603 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lax/fvyfahspymghmuwtwwqniejiqmxsx. 2024.10.22 01:21:00.043948 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:00.052288 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.07 GiB. 2024.10.22 01:21:00.129791 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_446_446_0 (state Temporary) 2024.10.22 01:21:00.182292 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:21:00.184045 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:21:00.196304 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ghu/ioxvqmpxylmhnveipkhitaopcnrio. 2024.10.22 01:21:00.279752 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 7007 2024.10.22 01:21:00.338671 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gat/lfrxnnjxbadvnbtyugoavuyywbgcb, upload id 4d6cbce2-f572-4168-98ed-b2afcb69cafc. 2024.10.22 01:21:00.367109 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.96 GiB, free memory in arenas 5.16 MiB, will set to 1.55 GiB (RSS), difference: 67.16 MiB 2024.10.22 01:21:00.413802 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} AWSClient: Got dangerous response with successful code 200, checking its body: ' http://localhost:11111/test/common/gat/lfrxnnjxbadvnbtyugoavuyywbgcbtestcommon/gat/lfrxnnjxbadvnbtyugoavuyywbgcb"d09cf80c10' 2024.10.22 01:21:00.426860 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Multipart upload has completed. Details: bucket test, key common/gat/lfrxnnjxbadvnbtyugoavuyywbgcb, upload id 4d6cbce2-f572-4168-98ed-b2afcb69cafc, Parts: 3 2024.10.22 01:21:00.534581 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gat/lfrxnnjxbadvnbtyugoavuyywbgcb, upload id 4d6cbce2-f572-4168-98ed-b2afcb69cafc. 2024.10.22 01:21:00.555982 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_446_446_0 to 202410_452_452_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:00.580299 [ 1161 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_446_224 to 202410_447_447_0 2024.10.22 01:21:00.606657 [ 1161 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.49 MiB using storage policy from min volume index 0 2024.10.22 01:21:00.615214 [ 1161 ] {} DiskLocal: Reserved 2.49 MiB on local disk `default`, having unreserved 297.12 GiB. 2024.10.22 01:21:00.642959 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_447_225} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_446_224 to 202410_447_447_0 into Wide with storage Full 2024.10.22 01:21:00.710771 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:00.717144 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.12 GiB. 2024.10.22 01:21:00.733128 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 50271 2024.10.22 01:21:00.767150 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_454_454_0 (state Temporary) 2024.10.22 01:21:00.772784 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_447_225} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:00.777746 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_447_225} MergeTreeSequentialSource: Reading 7 marks from part 202410_1_446_224, total 49035 rows starting from the beginning of the part 2024.10.22 01:21:00.828683 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_454_454_0 to 202410_467_467_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:00.990414 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7007 2024.10.22 01:21:01.029568 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/amw/ncxhweuvwcqggkwegaukcfczjbzks. 2024.10.22 01:21:01.078514 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_447_225} MergeTreeSequentialSource: Reading 2 marks from part 202410_447_447_0, total 190 rows starting from the beginning of the part 2024.10.22 01:21:01.157350 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/amw/ncxhweuvwcqggkwegaukcfczjbzks. 2024.10.22 01:21:02.159797 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_778_386} MergeTask::MergeProjectionsStage: Merge sorted 49548 rows, containing 13 columns (13 merged, 0 gathered) in 2.882825992 sec., 17187.30167464093 rows/sec., 2.75 MiB/sec. 2024.10.22 01:21:02.185989 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_778_386} MergedBlockOutputStream: filled checksums 202410_1_778_386 (state Temporary) 2024.10.22 01:21:02.458492 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_778_386} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_778_386 to 202410_1_778_386 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:02.460988 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_778_386} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_777_385, 202410_778_778_0] -> 202410_1_778_386 2024.10.22 01:21:02.491620 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:21:02.493099 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_389_167, 202410_390_390_0] 2024.10.22 01:21:02.494995 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_389_167, 202410_390_390_0] 2024.10.22 01:21:02.532448 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_389_167, 202410_390_390_0] 2024.10.22 01:21:02.619204 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.96 GiB, free memory in arenas 3.97 MiB, will set to 1.55 GiB (RSS), difference: 51.66 MiB 2024.10.22 01:21:02.922842 [ 1165 ] {} DatabaseCatalog: Execution took 234 ms. 2024.10.22 01:21:03.078015 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_447_225/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:21:03.097586 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 385 2024.10.22 01:21:03.213371 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1719 entries to flush up to offset 685136 2024.10.22 01:21:03.318281 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 217 bytes for 8aff2f585ce94e4a3045e86f7b81f756:0) 2024.10.22 01:21:03.325056 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 854f10257937f71db2f581a95fb90b19, offset: 33554432, size: 24094991 2024.10.22 01:21:03.339906 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 854f10257937f71db2f581a95fb90b19 for removal 2024.10.22 01:21:03.348537 [ 1476 ] {} CacheMetadata: Key 854f10257937f71db2f581a95fb90b19 is removed from metadata 2024.10.22 01:21:03.381435 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:03.400880 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/dlb/dcmwipuqozbpxptksqwgxblqttniq. 2024.10.22 01:21:03.414035 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 49113 2024.10.22 01:21:03.420793 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:21:03.532494 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_367_367_0 (state Temporary) 2024.10.22 01:21:03.542236 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/dlb/dcmwipuqozbpxptksqwgxblqttniq. 2024.10.22 01:21:03.623383 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_367_367_0 to 202410_377_377_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:03.706999 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_678_296, 202410_679_679_0] 2024.10.22 01:21:03.716757 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_678_296, 202410_679_679_0] 2024.10.22 01:21:03.721692 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 385 2024.10.22 01:21:03.740544 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_678_296, 202410_679_679_0] 2024.10.22 01:21:03.881092 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:03.985243 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:21:04.055648 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_447_225} MergeTask::MergeProjectionsStage: Merge sorted 49225 rows, containing 13 columns (13 merged, 0 gathered) in 3.419725804 sec., 14394.428916617317 rows/sec., 6.25 MiB/sec. 2024.10.22 01:21:04.178484 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:04.184486 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:21:04.245316 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_447_225} MergedBlockOutputStream: filled checksums 202410_1_447_225 (state Temporary) 2024.10.22 01:21:04.265280 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_771_771_0 (state Temporary) 2024.10.22 01:21:04.276997 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ffk/axsehijcqcdwmxptpzhyswvuiudsw. 2024.10.22 01:21:04.327160 [ 1235 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:21:04.332298 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_436_436_0 (state Temporary) 2024.10.22 01:21:04.360239 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_771_771_0 to 202410_784_784_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:04.364756 [ 1235 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:21:04.398212 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_778_386 to 202410_779_779_0 2024.10.22 01:21:04.413369 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.78 MiB using storage policy from min volume index 0 2024.10.22 01:21:04.418809 [ 1244 ] {} DiskLocal: Reserved 1.78 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:21:04.463258 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_779_387} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_778_386 to 202410_779_779_0 into Compact with storage Full 2024.10.22 01:21:04.487994 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_436_436_0 to 202410_443_443_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:04.513060 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ffk/axsehijcqcdwmxptpzhyswvuiudsw. 2024.10.22 01:21:04.516479 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/jwc/towfughomidftgvoqqxuvwouuvnia. 2024.10.22 01:21:04.526890 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 49113 2024.10.22 01:21:04.657439 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 685136 2024.10.22 01:21:04.759600 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/jwc/towfughomidftgvoqqxuvwouuvnia. 2024.10.22 01:21:04.768199 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_779_387} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:04.777224 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_779_387} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_778_386, total 49548 rows starting from the beginning of the part 2024.10.22 01:21:04.792092 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_779_387} MergeTreeSequentialSource: Reading 2 marks from part 202410_779_779_0, total 78 rows starting from the beginning of the part 2024.10.22 01:21:04.827187 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 4.97 MiB, will set to 1.54 GiB (RSS), difference: 58.16 MiB 2024.10.22 01:21:04.735186 [ 1151 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 213 ms. 2024.10.22 01:21:05.740846 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_447_225} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_447_225 to 202410_1_447_225 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:05.763111 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_447_225} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_446_224, 202410_447_447_0] -> 202410_1_447_225 2024.10.22 01:21:05.784099 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.11 MiB. 2024.10.22 01:21:05.873693 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yek/kglxnvmsdximxkjzxumezotfodekj. 2024.10.22 01:21:05.964650 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yek/kglxnvmsdximxkjzxumezotfodekj. 2024.10.22 01:21:06.102692 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:06.107228 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.14 GiB. 2024.10.22 01:21:06.446911 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_679_297, 202410_680_680_0] 2024.10.22 01:21:06.450094 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_679_297, 202410_680_680_0] 2024.10.22 01:21:06.458624 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lxg/fzsnpecoohsliawklraeucgpslnsg. 2024.10.22 01:21:06.461923 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_679_297, 202410_680_680_0] 2024.10.22 01:21:06.551990 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lxg/fzsnpecoohsliawklraeucgpslnsg. 2024.10.22 01:21:06.882503 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.96 GiB, free memory in arenas 5.00 MiB, will set to 1.54 GiB (RSS), difference: 15.94 MiB 2024.10.22 01:21:07.040569 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/cqv/nupspueycgrtlvvjjrkworsphwcyb. 2024.10.22 01:21:07.171129 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/cqv/nupspueycgrtlvvjjrkworsphwcyb. 2024.10.22 01:21:07.529086 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nha/fqnkkorkipggbolblpoywpaztgkzw, upload id eaf0db76-1ffc-45ab-b8b1-239f2c196ee2. 2024.10.22 01:21:07.612051 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} AWSClient: Got dangerous response with successful code 200, checking its body: ' http://localhost:11111/test/common/nha/fqnkkorkipggbolblpoywpaztgkzwtestcommon/nha/fqnkkorkipggbolblpoywpaztgkzw"c579778d5c' 2024.10.22 01:21:07.624021 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Multipart upload has completed. Details: bucket test, key common/nha/fqnkkorkipggbolblpoywpaztgkzw, upload id eaf0db76-1ffc-45ab-b8b1-239f2c196ee2, Parts: 4 2024.10.22 01:21:07.817156 [ 1697 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ymh/xhtqlwnawfmikgivdyayaryghzbof, size 1136283 2024.10.22 01:21:07.897194 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column WithHash 2024.10.22 01:21:07.912462 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column WithHash 2024.10.22 01:21:07.975392 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column WithHash 2024.10.22 01:21:07.975693 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nha/fqnkkorkipggbolblpoywpaztgkzw, upload id eaf0db76-1ffc-45ab-b8b1-239f2c196ee2. 2024.10.22 01:21:07.977695 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ald/bmlzonhgrdbacakfdxkosgvvurtwk, upload id 8921de21-1d62-418e-a7cf-780809c64759. 2024.10.22 01:21:08.001315 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column WithHash 2024.10.22 01:21:08.023423 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column WithHash 2024.10.22 01:21:08.052382 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} AWSClient: Got dangerous response with successful code 200, checking its body: ' http://localhost:11111/test/common/ald/bmlzonhgrdbacakfdxkosgvvurtwktestcommon/ald/bmlzonhgrdbacakfdxkosgvvurtwk"242ef4637b' 2024.10.22 01:21:08.056715 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Multipart upload has completed. Details: bucket test, key common/ald/bmlzonhgrdbacakfdxkosgvvurtwk, upload id 8921de21-1d62-418e-a7cf-780809c64759, Parts: 3 2024.10.22 01:21:08.074566 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column WithHash 2024.10.22 01:21:08.083883 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column WithHash 2024.10.22 01:21:08.101174 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column WithHash 2024.10.22 01:21:08.122861 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column WithHash 2024.10.22 01:21:08.204969 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ald/bmlzonhgrdbacakfdxkosgvvurtwk, upload id 8921de21-1d62-418e-a7cf-780809c64759. 2024.10.22 01:21:08.281997 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 233 entries to flush up to offset 50504 2024.10.22 01:21:08.289601 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_779_387} MergeTask::MergeProjectionsStage: Merge sorted 49626 rows, containing 13 columns (13 merged, 0 gathered) in 3.835703745 sec., 12937.912648934258 rows/sec., 2.07 MiB/sec. 2024.10.22 01:21:08.361392 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_779_387} MergedBlockOutputStream: filled checksums 202410_1_779_387 (state Temporary) 2024.10.22 01:21:08.383669 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gym/mcrlshuecevpqhtkjjwtnvixvctpw (LogSeriesLimiter: on interval from 2024-10-22 01:20:49 to 2024-10-22 01:21:08 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:21:08.493821 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 7018 2024.10.22 01:21:08.520782 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_779_387} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_779_387 to 202410_1_779_387 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:08.524020 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.96 GiB, free memory in arenas 5.07 MiB, will set to 1.53 GiB (RSS), difference: 9.71 MiB 2024.10.22 01:21:08.524445 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_779_387} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_778_386, 202410_779_779_0] -> 202410_1_779_387 2024.10.22 01:21:08.591129 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/wie/picmyhdsnaaiuybivsjkeyshtsuqw. 2024.10.22 01:21:08.614409 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 49192 2024.10.22 01:21:08.621031 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:21:08.772092 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:08.774372 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:08.824412 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_455_455_0 (state Temporary) 2024.10.22 01:21:08.948771 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/wie/picmyhdsnaaiuybivsjkeyshtsuqw. 2024.10.22 01:21:09.013680 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:09.018892 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:09.059524 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fdg/veasgpcdltcourddxamuvkcbzldjz. 2024.10.22 01:21:09.101664 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_455_455_0 to 202410_468_468_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:09.121405 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:09.125397 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:09.167281 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fdg/veasgpcdltcourddxamuvkcbzldjz. 2024.10.22 01:21:09.179989 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_447_447_0 (state Temporary) 2024.10.22 01:21:09.308582 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7018 2024.10.22 01:21:09.321692 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_772_772_0 (state Temporary) 2024.10.22 01:21:09.387022 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_772_772_0 to 202410_785_785_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:09.388794 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rpg/tcfjgywptvvrystypquzggxsekbow. 2024.10.22 01:21:09.389445 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_447_447_0 to 202410_453_453_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:09.444716 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_447_225 to 202410_448_448_0 2024.10.22 01:21:09.460207 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_779_387 to 202410_780_780_0 2024.10.22 01:21:09.460785 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.50 MiB using storage policy from min volume index 0 2024.10.22 01:21:09.464263 [ 1250 ] {} DiskLocal: Reserved 2.50 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:09.466227 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.78 MiB using storage policy from min volume index 0 2024.10.22 01:21:09.471671 [ 1172 ] {} DiskLocal: Reserved 1.78 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:09.511480 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_780_388} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_779_387 to 202410_780_780_0 into Compact with storage Full 2024.10.22 01:21:09.521529 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_448_226} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_447_225 to 202410_448_448_0 into Wide with storage Full 2024.10.22 01:21:09.556967 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rpg/tcfjgywptvvrystypquzggxsekbow. 2024.10.22 01:21:09.559984 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 49192 2024.10.22 01:21:09.561275 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 50504 2024.10.22 01:21:09.572566 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_448_226} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:09.581591 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_448_226} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_447_225, total 49225 rows starting from the beginning of the part 2024.10.22 01:21:09.663610 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_780_388} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:09.671023 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/sps/hsoindaragffpvwnrpqruyvucvpme. 2024.10.22 01:21:09.810831 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_780_388} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_779_387, total 49626 rows starting from the beginning of the part 2024.10.22 01:21:09.843215 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_448_226} MergeTreeSequentialSource: Reading 2 marks from part 202410_448_448_0, total 250 rows starting from the beginning of the part 2024.10.22 01:21:10.026356 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_780_388} MergeTreeSequentialSource: Reading 2 marks from part 202410_780_780_0, total 56 rows starting from the beginning of the part 2024.10.22 01:21:10.054669 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_92_92_0 (state Temporary) 2024.10.22 01:21:10.066327 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/sps/hsoindaragffpvwnrpqruyvucvpme. 2024.10.22 01:21:10.124189 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_94_19/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:21:10.186155 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:21:10.190465 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:21:10.277681 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/eyz/utljejbsdarftyeqidczugpccclul. 2024.10.22 01:21:10.369821 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_390_168, 202410_391_391_0] 2024.10.22 01:21:10.373712 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_390_168, 202410_391_391_0] 2024.10.22 01:21:10.396921 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_390_168, 202410_391_391_0] 2024.10.22 01:21:10.484792 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.96 GiB, free memory in arenas 5.36 MiB, will set to 1.53 GiB (RSS), difference: -26.49 MiB 2024.10.22 01:21:10.628314 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/eyz/utljejbsdarftyeqidczugpccclul. 2024.10.22 01:21:10.854243 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kcc/ildtfbiqlmhpzdffgnligrvwlcyqd. 2024.10.22 01:21:11.179172 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kcc/ildtfbiqlmhpzdffgnligrvwlcyqd. 2024.10.22 01:21:11.231736 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 386 2024.10.22 01:21:11.608783 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:11.611719 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:11.613743 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pfz/kzxmviuodqttakisytcqnkiruljxn. 2024.10.22 01:21:11.660493 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1748 entries to flush up to offset 686884 2024.10.22 01:21:11.809716 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_368_368_0 (state Temporary) 2024.10.22 01:21:11.975868 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pfz/kzxmviuodqttakisytcqnkiruljxn. 2024.10.22 01:21:11.994468 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_368_368_0 to 202410_378_378_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:12.048811 [ 1241 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_373_77 to 202410_378_378_0 2024.10.22 01:21:12.069059 [ 1241 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:12.074431 [ 1241 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:12.087517 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_373_77 to 202410_378_378_0 into Compact with storage Full 2024.10.22 01:21:12.140454 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 386 2024.10.22 01:21:12.325440 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:12.474020 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_373_77, total 417 rows starting from the beginning of the part 2024.10.22 01:21:12.488915 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:12.502773 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:12.560394 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.96 GiB, free memory in arenas 4.87 MiB, will set to 1.55 GiB (RSS), difference: 18.69 MiB 2024.10.22 01:21:12.599654 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_780_388/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:21:12.761522 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_437_437_0 (state Temporary) 2024.10.22 01:21:12.815035 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/iij/hcwetkcbspnnvrvhttntongbnirtx. 2024.10.22 01:21:12.827600 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_374_374_0, total 1 rows starting from the beginning of the part 2024.10.22 01:21:12.878090 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_437_437_0 to 202410_444_444_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:12.916617 [ 1191 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_439_88 to 202410_444_444_0 2024.10.22 01:21:12.927557 [ 1191 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:12.930163 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:13.019244 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 686884 2024.10.22 01:21:13.065630 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_375_375_0, total 1 rows starting from the beginning of the part 2024.10.22 01:21:13.081261 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_439_88 to 202410_444_444_0 into Wide with storage Full 2024.10.22 01:21:13.096600 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_448_226} MergeTask::MergeProjectionsStage: Merge sorted 49475 rows, containing 13 columns (13 merged, 0 gathered) in 3.61452916 sec., 13687.813214377276 rows/sec., 5.94 MiB/sec. 2024.10.22 01:21:13.115460 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:13.124632 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} MergeTreeSequentialSource: Reading 86 marks from part 202410_1_439_88, total 692615 rows starting from the beginning of the part 2024.10.22 01:21:13.141005 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_440_440_0, total 1312 rows starting from the beginning of the part 2024.10.22 01:21:13.166292 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_441_441_0, total 1850 rows starting from the beginning of the part 2024.10.22 01:21:13.187039 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_442_442_0, total 1230 rows starting from the beginning of the part 2024.10.22 01:21:13.220875 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_443_443_0, total 1719 rows starting from the beginning of the part 2024.10.22 01:21:13.246865 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_376_376_0, total 1 rows starting from the beginning of the part 2024.10.22 01:21:13.261965 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_448_226} MergedBlockOutputStream: filled checksums 202410_1_448_226 (state Temporary) 2024.10.22 01:21:13.261887 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_444_444_0, total 1748 rows starting from the beginning of the part 2024.10.22 01:21:13.286422 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/iij/hcwetkcbspnnvrvhttntongbnirtx. 2024.10.22 01:21:13.461480 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_780_388} MergeTask::MergeProjectionsStage: Merge sorted 49682 rows, containing 13 columns (13 merged, 0 gathered) in 3.952213452 sec., 12570.677318771495 rows/sec., 2.01 MiB/sec. 2024.10.22 01:21:13.500586 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_780_388} MergedBlockOutputStream: filled checksums 202410_1_780_388 (state Temporary) 2024.10.22 01:21:13.570892 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 49285 2024.10.22 01:21:13.654572 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_377_377_0, total 1 rows starting from the beginning of the part 2024.10.22 01:21:13.752718 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hzs/bodvccjbatlljdbphsquxvmuwxnss. 2024.10.22 01:21:13.825068 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_680_298, 202410_681_681_0] 2024.10.22 01:21:13.825704 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_780_388} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_780_388 to 202410_1_780_388 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:13.831134 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_780_388} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_779_387, 202410_780_780_0] -> 202410_1_780_388 2024.10.22 01:21:13.858786 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:21:13.886548 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_448_226} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_448_226 to 202410_1_448_226 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:13.897084 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_448_226} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_447_225, 202410_448_448_0] -> 202410_1_448_226 2024.10.22 01:21:13.929153 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_680_298, 202410_681_681_0] 2024.10.22 01:21:13.958790 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_378_378_0, total 1 rows starting from the beginning of the part 2024.10.22 01:21:13.989730 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:13.994681 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:14.006412 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hzs/bodvccjbatlljdbphsquxvmuwxnss. 2024.10.22 01:21:14.024411 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.14 MiB. 2024.10.22 01:21:14.044897 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_680_298, 202410_681_681_0] 2024.10.22 01:21:14.084711 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_773_773_0 (state Temporary) 2024.10.22 01:21:14.106492 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_444_89/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:21:14.153279 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_773_773_0 to 202410_786_786_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:14.179781 [ 1192 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_780_388 to 202410_781_781_0 2024.10.22 01:21:14.183904 [ 1192 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.78 MiB using storage policy from min volume index 0 2024.10.22 01:21:14.186491 [ 1192 ] {} DiskLocal: Reserved 1.78 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:14.211603 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_781_389} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_780_388 to 202410_781_781_0 into Compact with storage Full 2024.10.22 01:21:14.213830 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 49285 2024.10.22 01:21:14.264680 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_781_389} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:14.269719 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_781_389} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_780_388, total 49682 rows starting from the beginning of the part 2024.10.22 01:21:14.281110 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_781_389} MergeTreeSequentialSource: Reading 2 marks from part 202410_781_781_0, total 62 rows starting from the beginning of the part 2024.10.22 01:21:14.519506 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/iak/zkpczqbryvkqmuoanokgzsjmqbure. 2024.10.22 01:21:14.568592 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 4.27 MiB, will set to 1.53 GiB (RSS), difference: 21.47 MiB 2024.10.22 01:21:15.024374 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/iak/zkpczqbryvkqmuoanokgzsjmqbure. 2024.10.22 01:21:15.389449 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/srn/kmehfddtofizdujocydrgespvqvsv. 2024.10.22 01:21:15.511669 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/srn/kmehfddtofizdujocydrgespvqvsv. 2024.10.22 01:21:15.653115 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_325_67, 202410_326_326_0, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0, 202410_330_330_0] 2024.10.22 01:21:15.656176 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_325_67, 202410_326_326_0, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0, 202410_330_330_0] 2024.10.22 01:21:15.886157 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_325_67, 202410_326_326_0, 202410_327_327_0, 202410_328_328_0, 202410_329_329_0, 202410_330_330_0] 2024.10.22 01:21:16.162692 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/htf/qekbpubdtnbirivsslpxpghtydzxe. 2024.10.22 01:21:16.376392 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/htf/qekbpubdtnbirivsslpxpghtydzxe. 2024.10.22 01:21:16.710512 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.96 GiB, free memory in arenas 7.99 MiB, will set to 1.54 GiB (RSS), difference: 15.12 MiB 2024.10.22 01:21:16.814343 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 7033 2024.10.22 01:21:17.098656 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} MergeTask::MergeProjectionsStage: Merge sorted 422 rows, containing 32 columns (32 merged, 0 gathered) in 5.017893542 sec., 84.09903408030492 rows/sec., 7.88 KiB/sec. 2024.10.22 01:21:17.106445 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 285 entries to flush up to offset 50789 2024.10.22 01:21:17.157714 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:17.162578 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:17.162915 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} MergedBlockOutputStream: filled checksums 202410_1_378_78 (state Temporary) 2024.10.22 01:21:17.184662 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/goa/hnxerplqhzmhdlislgkzgtvaxaosw. 2024.10.22 01:21:17.431795 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_456_456_0 (state Temporary) 2024.10.22 01:21:17.484269 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_681_299, 202410_682_682_0] 2024.10.22 01:21:17.487000 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_681_299, 202410_682_682_0] 2024.10.22 01:21:17.502361 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_681_299, 202410_682_682_0] 2024.10.22 01:21:17.519971 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_456_456_0 to 202410_469_469_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:17.534873 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/goa/hnxerplqhzmhdlislgkzgtvaxaosw. 2024.10.22 01:21:17.540718 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ncj/puwvecbvdyowysebtutqtongjaedg. 2024.10.22 01:21:17.574002 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7033 2024.10.22 01:21:17.606487 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:17.617257 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:17.692625 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ncj/puwvecbvdyowysebtutqtongjaedg. 2024.10.22 01:21:17.726760 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_448_448_0 (state Temporary) 2024.10.22 01:21:17.763555 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_781_389} MergeTask::MergeProjectionsStage: Merge sorted 49744 rows, containing 13 columns (13 merged, 0 gathered) in 3.563513791 sec., 13959.255644143514 rows/sec., 2.24 MiB/sec. 2024.10.22 01:21:17.800213 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_781_389} MergedBlockOutputStream: filled checksums 202410_1_781_389 (state Temporary) 2024.10.22 01:21:17.834719 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_448_448_0 to 202410_454_454_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:17.885282 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 50789 2024.10.22 01:21:17.888153 [ 1198 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_448_226 to 202410_449_449_0 2024.10.22 01:21:17.896151 [ 1198 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.51 MiB using storage policy from min volume index 0 2024.10.22 01:21:17.904511 [ 1198 ] {} DiskLocal: Reserved 2.51 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:17.921398 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_378_78 to 202410_1_378_78 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:17.927423 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_378_78} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_373_77, 202410_378_378_0] -> 202410_1_378_78 2024.10.22 01:21:17.928947 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_449_227} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_448_226 to 202410_449_449_0 into Wide with storage Full 2024.10.22 01:21:17.978692 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:21:18.062911 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_449_227} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:18.067414 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_449_227} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_448_226, total 49475 rows starting from the beginning of the part 2024.10.22 01:21:18.101620 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_781_389} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_781_389 to 202410_1_781_389 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:18.110309 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_781_389} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_780_388, 202410_781_781_0] -> 202410_1_781_389 2024.10.22 01:21:18.169697 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:21:18.170976 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/abm/ftpigiyfpnnrylmdglzhjewlkdvwd. 2024.10.22 01:21:18.178833 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_449_227} MergeTreeSequentialSource: Reading 2 marks from part 202410_449_449_0, total 220 rows starting from the beginning of the part 2024.10.22 01:21:18.233300 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 49360 2024.10.22 01:21:18.277559 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_92_92_0 to 202410_95_95_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:18.317470 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ed8f4dca1a53d9bc1a751cbf568b0a0a, offset: 0, size: 0 2024.10.22 01:21:18.322153 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ed8f4dca1a53d9bc1a751cbf568b0a0a for removal 2024.10.22 01:21:18.326978 [ 1476 ] {} CacheMetadata: Key ed8f4dca1a53d9bc1a751cbf568b0a0a is removed from metadata 2024.10.22 01:21:18.387850 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/abm/ftpigiyfpnnrylmdglzhjewlkdvwd. 2024.10.22 01:21:18.491788 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uam/irzfmglctjozblifrtfptmzuevyiu. 2024.10.22 01:21:18.562838 [ 1204 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 271 ms. 2024.10.22 01:21:18.700621 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.96 GiB, free memory in arenas 9.37 MiB, will set to 1.53 GiB (RSS), difference: -10.59 MiB 2024.10.22 01:21:18.715996 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uam/irzfmglctjozblifrtfptmzuevyiu. 2024.10.22 01:21:18.728076 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:18.732425 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:18.933667 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_774_774_0 (state Temporary) 2024.10.22 01:21:19.075973 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_774_774_0 to 202410_787_787_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:19.137656 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mcd/ojjnnhpojycmdwjlesokiplijqefh. 2024.10.22 01:21:19.168491 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 49360 2024.10.22 01:21:19.211996 [ 1219 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_781_389 to 202410_782_782_0 2024.10.22 01:21:19.246523 [ 1219 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.79 MiB using storage policy from min volume index 0 2024.10.22 01:21:19.252800 [ 1219 ] {} DiskLocal: Reserved 1.79 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:19.271596 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_782_390} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_781_389 to 202410_782_782_0 into Compact with storage Full 2024.10.22 01:21:19.359849 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mcd/ojjnnhpojycmdwjlesokiplijqefh. 2024.10.22 01:21:19.369418 [ 1205 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:21:19.376279 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_782_390} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:19.386333 [ 1205 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:21:19.387953 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_782_390} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_781_389, total 49744 rows starting from the beginning of the part 2024.10.22 01:21:19.440538 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_782_390} MergeTreeSequentialSource: Reading 2 marks from part 202410_782_782_0, total 64 rows starting from the beginning of the part 2024.10.22 01:21:19.578740 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_449_227/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:21:19.758877 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_391_169, 202410_392_392_0] 2024.10.22 01:21:19.763070 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_391_169, 202410_392_392_0] 2024.10.22 01:21:19.781166 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_391_169, 202410_392_392_0] 2024.10.22 01:21:19.860043 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mye/vnuaqwvmcmrltvzzpbgubgdcxleio. 2024.10.22 01:21:19.995075 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0d6673e58e8ced9bc6e813d7994ee8e3, offset: 0, size: 0 2024.10.22 01:21:20.000616 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0d6673e58e8ced9bc6e813d7994ee8e3 for removal 2024.10.22 01:21:20.008201 [ 1476 ] {} CacheMetadata: Key 0d6673e58e8ced9bc6e813d7994ee8e3 is removed from metadata 2024.10.22 01:21:20.027561 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 688426 2024.10.22 01:21:20.185095 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:21:20.188537 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:21:20.244307 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mye/vnuaqwvmcmrltvzzpbgubgdcxleio. 2024.10.22 01:21:20.725683 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:20.731290 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:20.760828 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.96 GiB, free memory in arenas 6.46 MiB, will set to 1.55 GiB (RSS), difference: 1.58 MiB 2024.10.22 01:21:20.769503 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nrn/dnbatncuxxncwzshvlqavhhjbnjnl. 2024.10.22 01:21:20.848788 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b1515df9bd43a94426139b71a8410581, offset: 0, size: 0 2024.10.22 01:21:20.854130 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b1515df9bd43a94426139b71a8410581 for removal 2024.10.22 01:21:20.856646 [ 1476 ] {} CacheMetadata: Key b1515df9bd43a94426139b71a8410581 is removed from metadata 2024.10.22 01:21:20.932059 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_438_438_0 (state Temporary) 2024.10.22 01:21:20.949927 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nrn/dnbatncuxxncwzshvlqavhhjbnjnl. 2024.10.22 01:21:21.039070 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_438_438_0 to 202410_445_445_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:21.072888 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 688426 2024.10.22 01:21:21.363110 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vbo/twhlrjmqorxwehyeldpfjgccguzyz. 2024.10.22 01:21:21.412969 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 76ea6eece23bcbb309d8d41e310572f2, offset: 0, size: 0 2024.10.22 01:21:21.415729 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 76ea6eece23bcbb309d8d41e310572f2 for removal 2024.10.22 01:21:21.420907 [ 1476 ] {} CacheMetadata: Key 76ea6eece23bcbb309d8d41e310572f2 is removed from metadata 2024.10.22 01:21:21.479824 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vbo/twhlrjmqorxwehyeldpfjgccguzyz. 2024.10.22 01:21:21.481428 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mle/rgcyjafqgecqzwhmzkampwynkvauv. 2024.10.22 01:21:21.523868 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 77b288dbc3877447fa79e7cde8fbb95c, offset: 0, size: 0 2024.10.22 01:21:21.528057 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 77b288dbc3877447fa79e7cde8fbb95c for removal 2024.10.22 01:21:21.542237 [ 1476 ] {} CacheMetadata: Key 77b288dbc3877447fa79e7cde8fbb95c is removed from metadata 2024.10.22 01:21:21.591851 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mle/rgcyjafqgecqzwhmzkampwynkvauv. 2024.10.22 01:21:21.596388 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pey/mharanpaqmoajmttyfkvfxwiminre. 2024.10.22 01:21:21.659643 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 535d5511e222b0a3092f01fea7bce0d3, offset: 0, size: 0 2024.10.22 01:21:21.661943 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 535d5511e222b0a3092f01fea7bce0d3 for removal 2024.10.22 01:21:21.664574 [ 1476 ] {} CacheMetadata: Key 535d5511e222b0a3092f01fea7bce0d3 is removed from metadata 2024.10.22 01:21:21.720435 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pey/mharanpaqmoajmttyfkvfxwiminre. 2024.10.22 01:21:21.735780 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_449_227} MergeTask::MergeProjectionsStage: Merge sorted 49695 rows, containing 13 columns (13 merged, 0 gathered) in 3.817052753 sec., 13019.20702063716 rows/sec., 5.65 MiB/sec. 2024.10.22 01:21:21.776611 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_449_227} MergedBlockOutputStream: filled checksums 202410_1_449_227 (state Temporary) 2024.10.22 01:21:22.375854 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_782_390} MergeTask::MergeProjectionsStage: Merge sorted 49808 rows, containing 13 columns (13 merged, 0 gathered) in 3.110984853 sec., 16010.364033746067 rows/sec., 2.57 MiB/sec. 2024.10.22 01:21:22.495905 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_782_390} MergedBlockOutputStream: filled checksums 202410_1_782_390 (state Temporary) 2024.10.22 01:21:22.500711 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.96 GiB, free memory in arenas 13.45 MiB, will set to 1.54 GiB (RSS), difference: 55.36 MiB 2024.10.22 01:21:22.510789 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/sdx/qcrhvrabqfaqzwpmsviahefjllxgi. 2024.10.22 01:21:22.551741 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_449_227} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_449_227 to 202410_1_449_227 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:22.556231 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_449_227} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_448_226, 202410_449_449_0] -> 202410_1_449_227 2024.10.22 01:21:22.567353 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3665 2024.10.22 01:21:22.584615 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.12 MiB. 2024.10.22 01:21:22.614724 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_782_390} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_782_390 to 202410_1_782_390 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:22.620606 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_782_390} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_781_389, 202410_782_782_0] -> 202410_1_782_390 2024.10.22 01:21:22.657004 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 927ceebccecc3160f7e9ec9243418846, offset: 0, size: 0 2024.10.22 01:21:22.659471 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 927ceebccecc3160f7e9ec9243418846 for removal 2024.10.22 01:21:22.662288 [ 1476 ] {} CacheMetadata: Key 927ceebccecc3160f7e9ec9243418846 is removed from metadata 2024.10.22 01:21:22.665173 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:21:22.721757 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/sdx/qcrhvrabqfaqzwpmsviahefjllxgi. 2024.10.22 01:21:23.179216 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 49441 2024.10.22 01:21:23.341117 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lwv/xhsqvgwyoqsxwpqcfaoigetpxquud. 2024.10.22 01:21:23.356402 [ 1195 ] {} DatabaseCatalog: Execution took 266 ms. 2024.10.22 01:21:23.517064 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:23.521701 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:21:23.580785 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 26e836a289e24534b2ce9c925e2b3cce, offset: 0, size: 0 2024.10.22 01:21:23.583636 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 26e836a289e24534b2ce9c925e2b3cce for removal 2024.10.22 01:21:23.587930 [ 1476 ] {} CacheMetadata: Key 26e836a289e24534b2ce9c925e2b3cce is removed from metadata 2024.10.22 01:21:23.608505 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_775_775_0 (state Temporary) 2024.10.22 01:21:23.635960 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lwv/xhsqvgwyoqsxwpqcfaoigetpxquud. 2024.10.22 01:21:23.942068 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_682_300, 202410_683_683_0] 2024.10.22 01:21:23.945726 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_775_775_0 to 202410_788_788_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:23.953249 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_682_300, 202410_683_683_0] 2024.10.22 01:21:23.964816 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_682_300, 202410_683_683_0] 2024.10.22 01:21:23.985419 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_782_390 to 202410_783_783_0 2024.10.22 01:21:23.991020 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.79 MiB using storage policy from min volume index 0 2024.10.22 01:21:24.022551 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 49441 2024.10.22 01:21:24.038249 [ 1277 ] {} DiskLocal: Reserved 1.79 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:21:24.065222 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_783_391} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_782_390 to 202410_783_783_0 into Compact with storage Full 2024.10.22 01:21:24.154435 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_783_391} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:24.171014 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_783_391} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_782_390, total 49808 rows starting from the beginning of the part 2024.10.22 01:21:24.237241 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_783_391} MergeTreeSequentialSource: Reading 2 marks from part 202410_783_783_0, total 68 rows starting from the beginning of the part 2024.10.22 01:21:24.471586 [ 1166 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 01:21:24.477811 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pjo/vunlpdxneexaxeytfeuhdhxsplhpt. 2024.10.22 01:21:24.564987 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d92f793922a845cb0c146612349d02b3, offset: 0, size: 0 2024.10.22 01:21:24.566906 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d92f793922a845cb0c146612349d02b3 for removal 2024.10.22 01:21:24.571153 [ 1476 ] {} CacheMetadata: Key d92f793922a845cb0c146612349d02b3 is removed from metadata 2024.10.22 01:21:24.831815 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pjo/vunlpdxneexaxeytfeuhdhxsplhpt. 2024.10.22 01:21:24.961544 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uoj/wiijhvsvrwqejvyfuacfdztoidqxp. 2024.10.22 01:21:25.078237 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 7055 2024.10.22 01:21:25.194739 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uoj/wiijhvsvrwqejvyfuacfdztoidqxp. 2024.10.22 01:21:25.198812 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fqn/azxsbsrwhgejnvehfmsrwobeawppt. 2024.10.22 01:21:25.233011 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 09fff66073688a8c95ab8b1df6521cc0, offset: 0, size: 0 2024.10.22 01:21:25.239344 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 09fff66073688a8c95ab8b1df6521cc0 for removal 2024.10.22 01:21:25.243273 [ 1476 ] {} CacheMetadata: Key 09fff66073688a8c95ab8b1df6521cc0 is removed from metadata 2024.10.22 01:21:25.399413 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fqn/azxsbsrwhgejnvehfmsrwobeawppt. 2024.10.22 01:21:25.567637 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_444_89/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:21:25.654022 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 227 entries to flush up to offset 51016 2024.10.22 01:21:25.654069 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 999883 bytes for 6be48307c7a379820e97892041419370:0) 2024.10.22 01:21:25.656849 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4899080f045eb5a9089d4d61d4ffe08d, offset: 0, size: 2753427 2024.10.22 01:21:25.662224 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4899080f045eb5a9089d4d61d4ffe08d for removal 2024.10.22 01:21:25.664536 [ 1476 ] {} CacheMetadata: Key 4899080f045eb5a9089d4d61d4ffe08d is removed from metadata 2024.10.22 01:21:25.666738 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 5f50fe7ec1776a14c50f25f30853210c, offset: 0, size: 1273 2024.10.22 01:21:25.689671 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 5f50fe7ec1776a14c50f25f30853210c for removal 2024.10.22 01:21:25.716061 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:25.725595 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:21:25.776798 [ 1476 ] {} CacheMetadata: Key 5f50fe7ec1776a14c50f25f30853210c is removed from metadata 2024.10.22 01:21:25.796013 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_457_457_0 (state Temporary) 2024.10.22 01:21:25.980398 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:25.982333 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:21:26.063331 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_449_449_0 (state Temporary) 2024.10.22 01:21:26.094121 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_457_457_0 to 202410_470_470_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:26.165014 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7055 2024.10.22 01:21:26.208261 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_449_449_0 to 202410_455_455_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:26.252424 [ 1247 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_449_227 to 202410_450_450_0 2024.10.22 01:21:26.268570 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 51016 2024.10.22 01:21:26.276813 [ 1247 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.53 MiB using storage policy from min volume index 0 2024.10.22 01:21:26.280098 [ 1247 ] {} DiskLocal: Reserved 2.53 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:21:26.279968 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/wbu/uhizqurvulkkgcemctabgydvlsbzz. 2024.10.22 01:21:26.296944 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_450_228} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_449_227 to 202410_450_450_0 into Wide with storage Full 2024.10.22 01:21:26.415210 [ 1725 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gym/mcrlshuecevpqhtkjjwtnvixvctpw, size 45420 2024.10.22 01:21:26.439903 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6a2c5d5778e4af3f611d8881ba1a400b, offset: 0, size: 0 2024.10.22 01:21:26.443213 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6a2c5d5778e4af3f611d8881ba1a400b for removal 2024.10.22 01:21:26.532380 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.96 GiB, free memory in arenas 11.60 MiB, will set to 1.54 GiB (RSS), difference: 14.83 MiB 2024.10.22 01:21:26.552093 [ 1476 ] {} CacheMetadata: Key 6a2c5d5778e4af3f611d8881ba1a400b is removed from metadata 2024.10.22 01:21:26.575601 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_450_228} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:26.772520 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_94_19/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:21:26.782536 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_450_228} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_449_227, total 49695 rows starting from the beginning of the part 2024.10.22 01:21:26.792077 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column HitColor 2024.10.22 01:21:26.797685 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/wbu/uhizqurvulkkgcemctabgydvlsbzz. 2024.10.22 01:21:26.806682 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column HitColor 2024.10.22 01:21:26.835775 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column HitColor 2024.10.22 01:21:26.864769 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column HitColor 2024.10.22 01:21:26.871510 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_392_170, 202410_393_393_0] 2024.10.22 01:21:26.888636 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_392_170, 202410_393_393_0] 2024.10.22 01:21:26.890610 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column HitColor 2024.10.22 01:21:26.914510 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_392_170, 202410_393_393_0] 2024.10.22 01:21:26.920134 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column HitColor 2024.10.22 01:21:26.961050 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column HitColor 2024.10.22 01:21:26.977328 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column HitColor 2024.10.22 01:21:26.992246 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column HitColor 2024.10.22 01:21:27.065801 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 6a2c5d5778e4af3f611d8881ba1a400b:0) 2024.10.22 01:21:27.071736 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7a2eb8cea9c17e12aec9d150ec4c29b7, offset: 67108864, size: 13025587 2024.10.22 01:21:27.117642 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qxp/zcuyismhxdwwmnvaehxtampbiubzn (LogSeriesLimiter: on interval from 2024-10-22 01:21:08 to 2024-10-22 01:21:27 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:21:27.126891 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gsf/khklciupwcnxwiopunsfqryugdete. 2024.10.22 01:21:27.185392 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7a2eb8cea9c17e12aec9d150ec4c29b7 for removal 2024.10.22 01:21:27.188186 [ 1476 ] {} CacheMetadata: Key 7a2eb8cea9c17e12aec9d150ec4c29b7 is removed from metadata 2024.10.22 01:21:27.188690 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bfc3b8db0be8124f5099f8af3039960e, offset: 0, size: 874 2024.10.22 01:21:27.192674 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bfc3b8db0be8124f5099f8af3039960e for removal 2024.10.22 01:21:27.218180 [ 1476 ] {} CacheMetadata: Key bfc3b8db0be8124f5099f8af3039960e is removed from metadata 2024.10.22 01:21:27.257185 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 387 2024.10.22 01:21:27.257383 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_450_228} MergeTreeSequentialSource: Reading 2 marks from part 202410_450_450_0, total 225 rows starting from the beginning of the part 2024.10.22 01:21:27.285679 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_783_391} MergeTask::MergeProjectionsStage: Merge sorted 49876 rows, containing 13 columns (13 merged, 0 gathered) in 3.228054922 sec., 15450.790400151685 rows/sec., 2.48 MiB/sec. 2024.10.22 01:21:27.365840 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_783_391} MergedBlockOutputStream: filled checksums 202410_1_783_391 (state Temporary) 2024.10.22 01:21:27.352429 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_381_76, 202410_382_382_0, 202410_383_383_0, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0] 2024.10.22 01:21:27.374427 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_381_76, 202410_382_382_0, 202410_383_383_0, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0] 2024.10.22 01:21:27.499273 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_381_76, 202410_382_382_0, 202410_383_383_0, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0] 2024.10.22 01:21:27.575702 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rfq/fglcubdxqgsfqomsavhnupwneemnh. 2024.10.22 01:21:27.624955 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:27.631809 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:27.721529 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_783_391} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_783_391 to 202410_1_783_391 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:27.723299 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c30984cb930c9d84b49870a9566d0beb, offset: 0, size: 0 2024.10.22 01:21:27.724356 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_369_369_0 (state Temporary) 2024.10.22 01:21:27.727777 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c30984cb930c9d84b49870a9566d0beb for removal 2024.10.22 01:21:27.731548 [ 1476 ] {} CacheMetadata: Key c30984cb930c9d84b49870a9566d0beb is removed from metadata 2024.10.22 01:21:27.736322 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_783_391} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_782_390, 202410_783_783_0] -> 202410_1_783_391 2024.10.22 01:21:27.778349 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:21:27.790306 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rfq/fglcubdxqgsfqomsavhnupwneemnh. 2024.10.22 01:21:27.977565 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_369_369_0 to 202410_379_379_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:28.033268 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 109 entries to flush up to offset 49550 2024.10.22 01:21:28.080297 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 689966 2024.10.22 01:21:28.165055 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 387 2024.10.22 01:21:28.219808 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gsf/khklciupwcnxwiopunsfqryugdete. 2024.10.22 01:21:28.229467 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_23_23_0 to 201403_23_23_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:28.317778 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_24_24_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:21:28.604567 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_683_301, 202410_684_684_0] 2024.10.22 01:21:28.611235 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_683_301, 202410_684_684_0] 2024.10.22 01:21:28.628963 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_683_301, 202410_684_684_0] 2024.10.22 01:21:28.797654 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 23 2024.10.22 01:21:28.826311 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 1.96 GiB, free memory in arenas 11.21 MiB, will set to 1.53 GiB (RSS), difference: 163.55 MiB 2024.10.22 01:21:28.857764 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:28.861757 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:29.059594 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_776_776_0 (state Temporary) 2024.10.22 01:21:29.153756 [ 1254 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 367 ms. 2024.10.22 01:21:29.157670 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:29.162146 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:29.228890 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:29.234954 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:29.249792 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_776_776_0 to 202410_789_789_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:29.321441 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_783_391 to 202410_784_784_0 2024.10.22 01:21:29.323881 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_23_23_0 (state Temporary) 2024.10.22 01:21:29.344409 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 49550 2024.10.22 01:21:29.393804 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mbo/teorykxdgumrjrrtwrramjgmmovic. 2024.10.22 01:21:29.439226 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.79 MiB using storage policy from min volume index 0 2024.10.22 01:21:29.446623 [ 1252 ] {} DiskLocal: Reserved 1.79 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:29.552699 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d27d7e01cc034014964b0ad12436d011, offset: 0, size: 0 2024.10.22 01:21:29.554649 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d27d7e01cc034014964b0ad12436d011 for removal 2024.10.22 01:21:29.558932 [ 1476 ] {} CacheMetadata: Key d27d7e01cc034014964b0ad12436d011 is removed from metadata 2024.10.22 01:21:29.589704 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_439_439_0 (state Temporary) 2024.10.22 01:21:29.609544 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_23_23_0 to 202410_23_23_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:29.629546 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_784_392} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_783_391 to 202410_784_784_0 into Compact with storage Full 2024.10.22 01:21:29.649001 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_784_392} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:29.687308 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_439_439_0 to 202410_446_446_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:29.763020 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_784_392} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_783_391, total 49876 rows starting from the beginning of the part 2024.10.22 01:21:29.767827 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mbo/teorykxdgumrjrrtwrramjgmmovic. 2024.10.22 01:21:29.780476 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 689966 2024.10.22 01:21:29.801105 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_784_392} MergeTreeSequentialSource: Reading 2 marks from part 202410_784_784_0, total 68 rows starting from the beginning of the part 2024.10.22 01:21:29.836912 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 23 2024.10.22 01:21:30.122470 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_450_228/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:21:30.188275 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:21:30.193782 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:21:30.240297 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 44 entries to flush up to offset 3709 2024.10.22 01:21:30.447563 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_406_81, 202410_407_407_0, 202410_408_408_0, 202410_409_409_0, 202410_410_410_0, 202410_411_411_0] 2024.10.22 01:21:30.452394 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_406_81, 202410_407_407_0, 202410_408_408_0, 202410_409_409_0, 202410_410_410_0, 202410_411_411_0] 2024.10.22 01:21:30.494384 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_406_81, 202410_407_407_0, 202410_408_408_0, 202410_409_409_0, 202410_410_410_0, 202410_411_411_0] 2024.10.22 01:21:30.608967 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cdd6d67d2ea9bee81a92e4b0a93e3247, offset: 0, size: 0 2024.10.22 01:21:30.612006 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cdd6d67d2ea9bee81a92e4b0a93e3247 for removal 2024.10.22 01:21:30.625337 [ 1476 ] {} CacheMetadata: Key cdd6d67d2ea9bee81a92e4b0a93e3247 is removed from metadata 2024.10.22 01:21:30.668465 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.20 GiB, peak 1.96 GiB, free memory in arenas 12.04 MiB, will set to 1.20 GiB (RSS), difference: 5.00 MiB 2024.10.22 01:21:31.488348 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_450_228} MergeTask::MergeProjectionsStage: Merge sorted 49920 rows, containing 13 columns (13 merged, 0 gathered) in 5.200287439 sec., 9599.469372715956 rows/sec., 4.17 MiB/sec. 2024.10.22 01:21:31.844284 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for cdd6d67d2ea9bee81a92e4b0a93e3247:0) 2024.10.22 01:21:31.855647 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a3a42c9839a9416ca2091653b4a3c6fc, offset: 33554432, size: 29260580 2024.10.22 01:21:31.860424 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_450_228} MergedBlockOutputStream: filled checksums 202410_1_450_228 (state Temporary) 2024.10.22 01:21:31.879038 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a3a42c9839a9416ca2091653b4a3c6fc for removal 2024.10.22 01:21:31.883691 [ 1476 ] {} CacheMetadata: Key a3a42c9839a9416ca2091653b4a3c6fc is removed from metadata 2024.10.22 01:21:31.885773 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4981523896110d23fe506c098eb1a6bf, offset: 0, size: 1223 2024.10.22 01:21:31.891866 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4981523896110d23fe506c098eb1a6bf for removal 2024.10.22 01:21:31.927954 [ 1476 ] {} CacheMetadata: Key 4981523896110d23fe506c098eb1a6bf is removed from metadata 2024.10.22 01:21:31.988455 [ 1189 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 259 ms. 2024.10.22 01:21:32.093104 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qcg/gbkhkcchmtswqcnxawgzxzhavbsdv. 2024.10.22 01:21:32.156054 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e52189212a55d7efadf77f2c6b2d8024, offset: 0, size: 0 2024.10.22 01:21:32.159127 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e52189212a55d7efadf77f2c6b2d8024 for removal 2024.10.22 01:21:32.163271 [ 1476 ] {} CacheMetadata: Key e52189212a55d7efadf77f2c6b2d8024 is removed from metadata 2024.10.22 01:21:32.206745 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qcg/gbkhkcchmtswqcnxawgzxzhavbsdv. 2024.10.22 01:21:32.531910 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_784_392} MergeTask::MergeProjectionsStage: Merge sorted 49944 rows, containing 13 columns (13 merged, 0 gathered) in 2.946120155 sec., 16952.46540275612 rows/sec., 2.72 MiB/sec. 2024.10.22 01:21:32.578367 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_784_392} MergedBlockOutputStream: filled checksums 202410_1_784_392 (state Temporary) 2024.10.22 01:21:32.758531 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_450_228} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_450_228 to 202410_1_450_228 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:32.824521 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.96 GiB, free memory in arenas 12.30 MiB, will set to 1.20 GiB (RSS), difference: 50.98 MiB 2024.10.22 01:21:32.863530 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_450_228} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_449_227, 202410_450_450_0] -> 202410_1_450_228 2024.10.22 01:21:32.888238 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_784_392} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_784_392 to 202410_1_784_392 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:32.891907 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_784_392} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_783_391, 202410_784_784_0] -> 202410_1_784_392 2024.10.22 01:21:32.929669 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:21:33.042579 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.13 MiB. 2024.10.22 01:21:33.057936 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 335 ms. 2024.10.22 01:21:33.063330 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hfe/yxmrdaljbjhvgofbltweptehwbifu. 2024.10.22 01:21:33.109108 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9fe04f7c0cf72ff745a95c4749293eb8, offset: 0, size: 0 2024.10.22 01:21:33.111514 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9fe04f7c0cf72ff745a95c4749293eb8 for removal 2024.10.22 01:21:33.115627 [ 1476 ] {} CacheMetadata: Key 9fe04f7c0cf72ff745a95c4749293eb8 is removed from metadata 2024.10.22 01:21:33.165384 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hfe/yxmrdaljbjhvgofbltweptehwbifu. 2024.10.22 01:21:33.319224 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_684_302, 202410_685_685_0] 2024.10.22 01:21:33.328270 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_684_302, 202410_685_685_0] 2024.10.22 01:21:33.342787 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_684_302, 202410_685_685_0] 2024.10.22 01:21:33.461684 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zsg/uwrzinqzglxowihrtfhnyeeuoaydx. 2024.10.22 01:21:33.512309 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 49632 2024.10.22 01:21:33.555874 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2d64afe9d4bc84dcec8d5ac421197cce, offset: 0, size: 0 2024.10.22 01:21:33.558940 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2d64afe9d4bc84dcec8d5ac421197cce for removal 2024.10.22 01:21:33.563173 [ 1476 ] {} CacheMetadata: Key 2d64afe9d4bc84dcec8d5ac421197cce is removed from metadata 2024.10.22 01:21:33.666600 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 7071 2024.10.22 01:21:33.707846 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zsg/uwrzinqzglxowihrtfhnyeeuoaydx. 2024.10.22 01:21:33.669825 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:33.772379 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:33.804801 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 202 entries to flush up to offset 51218 2024.10.22 01:21:33.826784 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_777_777_0 (state Temporary) 2024.10.22 01:21:33.882018 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:33.888505 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:33.918292 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_777_777_0 to 202410_790_790_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:33.957831 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_784_392 to 202410_785_785_0 2024.10.22 01:21:33.963534 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.79 MiB using storage policy from min volume index 0 2024.10.22 01:21:33.968987 [ 1264 ] {} DiskLocal: Reserved 1.79 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:33.984971 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 49632 2024.10.22 01:21:34.004288 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_785_393} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_784_392 to 202410_785_785_0 into Compact with storage Full 2024.10.22 01:21:34.047656 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_458_458_0 (state Temporary) 2024.10.22 01:21:34.059567 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_785_393} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:34.064292 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_785_393} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_784_392, total 49944 rows starting from the beginning of the part 2024.10.22 01:21:34.220971 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_458_458_0 to 202410_471_471_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:34.239565 [ 1215 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 262 ms. 2024.10.22 01:21:34.246311 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:34.252786 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:34.267843 [ 1200 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_466_93 to 202410_471_471_0 2024.10.22 01:21:34.302491 [ 1200 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:34.317617 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:34.356587 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_450_450_0 (state Temporary) 2024.10.22 01:21:34.357031 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_785_393} MergeTreeSequentialSource: Reading 2 marks from part 202410_785_785_0, total 79 rows starting from the beginning of the part 2024.10.22 01:21:34.361623 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_466_93 to 202410_471_471_0 into Compact with storage Full 2024.10.22 01:21:34.435480 [ 1199 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:21:34.462304 [ 1199 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:21:34.494615 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7071 2024.10.22 01:21:34.500072 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:34.515795 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_466_93, total 7085 rows starting from the beginning of the part 2024.10.22 01:21:34.563175 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_450_450_0 to 202410_456_456_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:34.580500 [ 1213 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_450_228 to 202410_451_451_0 2024.10.22 01:21:34.586735 [ 1213 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.54 MiB using storage policy from min volume index 0 2024.10.22 01:21:34.592210 [ 1213 ] {} DiskLocal: Reserved 2.54 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:34.628381 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_451_229} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_450_228 to 202410_451_451_0 into Wide with storage Full 2024.10.22 01:21:34.668023 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_451_229} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:34.679619 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_451_229} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_450_228, total 49920 rows starting from the beginning of the part 2024.10.22 01:21:34.710770 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 51218 2024.10.22 01:21:34.768542 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_467_467_0, total 28 rows starting from the beginning of the part 2024.10.22 01:21:34.837192 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_451_229} MergeTreeSequentialSource: Reading 2 marks from part 202410_451_451_0, total 228 rows starting from the beginning of the part 2024.10.22 01:21:34.884328 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.19 GiB, peak 1.96 GiB, free memory in arenas 7.08 MiB, will set to 1.19 GiB (RSS), difference: -6.93 MiB 2024.10.22 01:21:34.907326 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_468_468_0, total 11 rows starting from the beginning of the part 2024.10.22 01:21:35.086726 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_469_469_0, total 15 rows starting from the beginning of the part 2024.10.22 01:21:35.125430 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/jwa/aeadmywzhbibkxbpudwvdxmiouvxh. 2024.10.22 01:21:35.273241 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_470_470_0, total 22 rows starting from the beginning of the part 2024.10.22 01:21:35.329453 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_471_471_0, total 16 rows starting from the beginning of the part 2024.10.22 01:21:35.365688 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/jwa/aeadmywzhbibkxbpudwvdxmiouvxh. 2024.10.22 01:21:35.369026 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mgf/kvtrmpbyoiivlqxhifcbzfovpslrp. 2024.10.22 01:21:35.614587 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mgf/kvtrmpbyoiivlqxhifcbzfovpslrp. 2024.10.22 01:21:35.628328 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ark/tbxbxekhrwtvddorcdyaqrxjqoysp. 2024.10.22 01:21:35.668000 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 388 2024.10.22 01:21:35.685150 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c840e110eff0a55c1a489e1390710c8e, offset: 0, size: 0 2024.10.22 01:21:35.692297 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c840e110eff0a55c1a489e1390710c8e for removal 2024.10.22 01:21:35.700504 [ 1476 ] {} CacheMetadata: Key c840e110eff0a55c1a489e1390710c8e is removed from metadata 2024.10.22 01:21:35.772487 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ark/tbxbxekhrwtvddorcdyaqrxjqoysp. 2024.10.22 01:21:36.092260 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_393_171, 202410_394_394_0] 2024.10.22 01:21:36.096449 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_393_171, 202410_394_394_0] 2024.10.22 01:21:36.230958 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:36.232589 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:36.364731 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_370_370_0 (state Temporary) 2024.10.22 01:21:36.464049 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} MergeTask::MergeProjectionsStage: Merge sorted 7177 rows, containing 12 columns (12 merged, 0 gathered) in 2.111680236 sec., 3398.7153346639548 rows/sec., 539.55 KiB/sec. 2024.10.22 01:21:36.484283 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_393_171, 202410_394_394_0] 2024.10.22 01:21:36.486428 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_370_370_0 to 202410_380_380_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:36.534213 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} MergedBlockOutputStream: filled checksums 202410_1_471_94 (state Temporary) 2024.10.22 01:21:36.624738 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_685_303, 202410_686_686_0] 2024.10.22 01:21:36.627253 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_685_303, 202410_686_686_0] 2024.10.22 01:21:36.627921 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 1.96 GiB, free memory in arenas 7.31 MiB, will set to 1.21 GiB (RSS), difference: -39.51 MiB 2024.10.22 01:21:36.637957 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_685_303, 202410_686_686_0] 2024.10.22 01:21:36.651433 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gzz/uszgjvzmzmpsnbwmsivgxvlnpqenh. 2024.10.22 01:21:36.695018 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0fa096cd89a6503ffa2847e3a956777e, offset: 0, size: 0 2024.10.22 01:21:36.701139 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0fa096cd89a6503ffa2847e3a956777e for removal 2024.10.22 01:21:36.720233 [ 1476 ] {} CacheMetadata: Key 0fa096cd89a6503ffa2847e3a956777e is removed from metadata 2024.10.22 01:21:36.744961 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 388 2024.10.22 01:21:36.778074 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gzz/uszgjvzmzmpsnbwmsivgxvlnpqenh. 2024.10.22 01:21:36.786994 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 691506 2024.10.22 01:21:36.842739 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} MergeTask::MergeProjectionsStage: Merge sorted 700474 rows, containing 4 columns (4 merged, 0 gathered) in 23.86850128 sec., 29347.213374764517 rows/sec., 430.21 KiB/sec. 2024.10.22 01:21:36.869694 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_444_89/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:21:36.970764 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_471_94 to 202410_1_471_94 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:36.977504 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_471_94} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_466_93, 202410_471_471_0] -> 202410_1_471_94 2024.10.22 01:21:36.983479 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} MergedBlockOutputStream: filled checksums 202410_1_444_89 (state Temporary) 2024.10.22 01:21:37.165392 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.44 MiB. 2024.10.22 01:21:37.554967 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fxb/whoiefrlualzpxzyyqhcbbimsfjjj. 2024.10.22 01:21:37.831189 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fxb/whoiefrlualzpxzyyqhcbbimsfjjj. 2024.10.22 01:21:37.833118 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gqq/nxnupxruawrwcslgrvuzmlkxjntzo. 2024.10.22 01:21:37.881813 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a51bbb62503d58ba797de11bf9336705, offset: 0, size: 0 2024.10.22 01:21:37.886211 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a51bbb62503d58ba797de11bf9336705 for removal 2024.10.22 01:21:37.891560 [ 1476 ] {} CacheMetadata: Key a51bbb62503d58ba797de11bf9336705 is removed from metadata 2024.10.22 01:21:37.953029 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_785_393} MergeTask::MergeProjectionsStage: Merge sorted 50023 rows, containing 13 columns (13 merged, 0 gathered) in 3.964970333 sec., 12616.235633256629 rows/sec., 2.02 MiB/sec. 2024.10.22 01:21:37.980783 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_451_229} MergeTask::MergeProjectionsStage: Merge sorted 50148 rows, containing 13 columns (13 merged, 0 gathered) in 3.358341593 sec., 14932.370222411737 rows/sec., 6.48 MiB/sec. 2024.10.22 01:21:38.002831 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 97 entries to flush up to offset 49729 2024.10.22 01:21:38.057310 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:38.059365 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:38.104590 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_785_393} MergedBlockOutputStream: filled checksums 202410_1_785_393 (state Temporary) 2024.10.22 01:21:38.139341 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gqq/nxnupxruawrwcslgrvuzmlkxjntzo. 2024.10.22 01:21:38.164069 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_785_393} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_785_393 to 202410_1_785_393 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:38.212582 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_785_393} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_784_392, 202410_785_785_0] -> 202410_1_785_393 2024.10.22 01:21:38.268044 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_440_440_0 (state Temporary) 2024.10.22 01:21:38.268355 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_451_229} MergedBlockOutputStream: filled checksums 202410_1_451_229 (state Temporary) 2024.10.22 01:21:38.268691 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:21:38.354871 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:38.357333 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:38.358836 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_440_440_0 to 202410_447_447_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:38.484124 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_778_778_0 (state Temporary) 2024.10.22 01:21:38.504598 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 691506 2024.10.22 01:21:38.555101 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_778_778_0 to 202410_791_791_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:38.582007 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_785_393 to 202410_786_786_0 2024.10.22 01:21:38.587227 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.80 MiB using storage policy from min volume index 0 2024.10.22 01:21:38.591298 [ 1152 ] {} DiskLocal: Reserved 1.80 MiB on local disk `default`, having unreserved 297.27 GiB. 2024.10.22 01:21:38.658914 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/bgo/ukpnjcifiwygwyujttniikuumxvur. 2024.10.22 01:21:38.686923 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 49729 2024.10.22 01:21:38.687863 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.11 GiB, peak 1.96 GiB, free memory in arenas 9.38 MiB, will set to 1.18 GiB (RSS), difference: 76.25 MiB 2024.10.22 01:21:38.705519 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2f989c75d76c902f630aebb13ca5ec17, offset: 0, size: 0 2024.10.22 01:21:38.714498 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2f989c75d76c902f630aebb13ca5ec17 for removal 2024.10.22 01:21:38.720497 [ 1476 ] {} CacheMetadata: Key 2f989c75d76c902f630aebb13ca5ec17 is removed from metadata 2024.10.22 01:21:38.770040 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_786_394} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_785_393 to 202410_786_786_0 into Compact with storage Full 2024.10.22 01:21:38.792784 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/bgo/ukpnjcifiwygwyujttniikuumxvur. 2024.10.22 01:21:38.912809 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_786_394} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:38.931048 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_786_394} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_785_393, total 50023 rows starting from the beginning of the part 2024.10.22 01:21:38.979906 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_786_394} MergeTreeSequentialSource: Reading 2 marks from part 202410_786_786_0, total 93 rows starting from the beginning of the part 2024.10.22 01:21:39.058460 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_451_229} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_451_229 to 202410_1_451_229 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:39.063690 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_451_229} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_450_228, 202410_451_451_0] -> 202410_1_451_229 2024.10.22 01:21:39.090870 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.13 MiB. 2024.10.22 01:21:39.178191 [ 1274 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 01:21:39.366108 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_444_89 to 202410_1_444_89 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:39.369370 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_444_89} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_439_88, 202410_444_444_0] -> 202410_1_444_89 2024.10.22 01:21:39.393377 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:21:39.561634 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/dvb/ksxhfxhjhxiragxkioipzobdfvjvn. 2024.10.22 01:21:39.884114 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_94_19/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:21:39.954597 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lfy/adieuqaykmzqiujpwuoxrrupvsqpr. 2024.10.22 01:21:39.990161 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b4988f305950f6bb511fe3f66cdc6d67, offset: 0, size: 0 2024.10.22 01:21:39.992127 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b4988f305950f6bb511fe3f66cdc6d67 for removal 2024.10.22 01:21:40.000074 [ 1476 ] {} CacheMetadata: Key b4988f305950f6bb511fe3f66cdc6d67 is removed from metadata 2024.10.22 01:21:40.070370 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lfy/adieuqaykmzqiujpwuoxrrupvsqpr. 2024.10.22 01:21:40.186824 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:21:40.288585 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:21:40.475691 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.19 GiB, peak 1.96 GiB, free memory in arenas 7.27 MiB, will set to 1.18 GiB (RSS), difference: -7.88 MiB 2024.10.22 01:21:41.608706 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zed/vyfgpsanlrpdgpbmkdiokllmguthg. 2024.10.22 01:21:41.925129 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_786_394} MergeTask::MergeProjectionsStage: Merge sorted 50116 rows, containing 13 columns (13 merged, 0 gathered) in 3.165006729 sec., 15834.405513518263 rows/sec., 2.54 MiB/sec. 2024.10.22 01:21:41.983141 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_786_394} MergedBlockOutputStream: filled checksums 202410_1_786_394 (state Temporary) 2024.10.22 01:21:42.061381 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_786_394} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_786_394 to 202410_1_786_394 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:42.065095 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_786_394} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_785_393, 202410_786_786_0] -> 202410_1_786_394 2024.10.22 01:21:42.099689 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:21:42.102103 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 7097 2024.10.22 01:21:42.122519 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zed/vyfgpsanlrpdgpbmkdiokllmguthg. 2024.10.22 01:21:42.361007 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ekj/lnabsvhcfbtyoelkwqzpihausxrjl. 2024.10.22 01:21:42.465229 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 232 entries to flush up to offset 51450 2024.10.22 01:21:42.470070 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.96 GiB, free memory in arenas 7.29 MiB, will set to 1.18 GiB (RSS), difference: 23.80 MiB 2024.10.22 01:21:42.500298 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ekj/lnabsvhcfbtyoelkwqzpihausxrjl. 2024.10.22 01:21:42.582687 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:42.587357 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:42.595186 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_686_304, 202410_687_687_0] 2024.10.22 01:21:42.604320 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_686_304, 202410_687_687_0] 2024.10.22 01:21:42.616004 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_686_304, 202410_687_687_0] 2024.10.22 01:21:42.646333 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_459_459_0 (state Temporary) 2024.10.22 01:21:42.703023 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 49793 2024.10.22 01:21:42.807337 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_459_459_0 to 202410_472_472_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:42.849513 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uzm/nyfgaisqshulvgvsgndfmfidwdoav. 2024.10.22 01:21:42.882871 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c7ff784f3a7590369f297c47ba26d7ab, offset: 0, size: 0 2024.10.22 01:21:42.884867 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c7ff784f3a7590369f297c47ba26d7ab for removal 2024.10.22 01:21:42.892392 [ 1476 ] {} CacheMetadata: Key c7ff784f3a7590369f297c47ba26d7ab is removed from metadata 2024.10.22 01:21:42.968160 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:42.971181 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:42.984084 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7097 2024.10.22 01:21:43.047034 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_451_451_0 (state Temporary) 2024.10.22 01:21:43.061066 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:43.159980 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uzm/nyfgaisqshulvgvsgndfmfidwdoav. 2024.10.22 01:21:43.164237 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:43.164673 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kbg/adtutqqbtvjubnrfiggscpztlqzuz. 2024.10.22 01:21:43.178334 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_451_451_0 to 202410_457_457_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:43.202211 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2597e5a36c4b699d5aa7c74e4d9b23e0, offset: 0, size: 0 2024.10.22 01:21:43.216289 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2597e5a36c4b699d5aa7c74e4d9b23e0 for removal 2024.10.22 01:21:43.220403 [ 1476 ] {} CacheMetadata: Key 2597e5a36c4b699d5aa7c74e4d9b23e0 is removed from metadata 2024.10.22 01:21:43.229756 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_451_229 to 202410_452_452_0 2024.10.22 01:21:43.248751 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 51450 2024.10.22 01:21:43.252581 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.55 MiB using storage policy from min volume index 0 2024.10.22 01:21:43.258631 [ 1256 ] {} DiskLocal: Reserved 2.55 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:21:43.281642 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kbg/adtutqqbtvjubnrfiggscpztlqzuz. 2024.10.22 01:21:43.332842 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_779_779_0 (state Temporary) 2024.10.22 01:21:43.376187 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_452_230} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_451_229 to 202410_452_452_0 into Wide with storage Full 2024.10.22 01:21:43.405790 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_452_230} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:43.419683 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_452_230} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_451_229, total 50148 rows starting from the beginning of the part 2024.10.22 01:21:43.472605 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_779_779_0 to 202410_792_792_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:43.507242 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_786_394 to 202410_787_787_0 2024.10.22 01:21:43.514616 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.80 MiB using storage policy from min volume index 0 2024.10.22 01:21:43.516537 [ 1234 ] {} DiskLocal: Reserved 1.80 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:21:43.560786 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_787_395} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_786_394 to 202410_787_787_0 into Compact with storage Full 2024.10.22 01:21:43.574861 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 49793 2024.10.22 01:21:43.612378 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_787_395} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:43.636692 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_787_395} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_786_394, total 50116 rows starting from the beginning of the part 2024.10.22 01:21:43.681576 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_787_395} MergeTreeSequentialSource: Reading 2 marks from part 202410_787_787_0, total 75 rows starting from the beginning of the part 2024.10.22 01:21:43.769208 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_452_230} MergeTreeSequentialSource: Reading 2 marks from part 202410_452_452_0, total 227 rows starting from the beginning of the part 2024.10.22 01:21:43.971105 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xjw/iolrfvqbtqgydmkijfswhgbpijbtx. 2024.10.22 01:21:43.971943 [ 1741 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qxp/zcuyismhxdwwmnvaehxtampbiubzn, size 692289 2024.10.22 01:21:44.048111 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b39493e8a5d709a92f60f0e24b5ca548, offset: 0, size: 0 2024.10.22 01:21:44.052200 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b39493e8a5d709a92f60f0e24b5ca548 for removal 2024.10.22 01:21:44.056261 [ 1476 ] {} CacheMetadata: Key b39493e8a5d709a92f60f0e24b5ca548 is removed from metadata 2024.10.22 01:21:44.225525 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xjw/iolrfvqbtqgydmkijfswhgbpijbtx. 2024.10.22 01:21:44.261348 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 389 2024.10.22 01:21:44.359314 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:21:44.366182 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:21:44.386265 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:21:44.415850 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:21:44.436499 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:21:44.455894 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:21:44.463593 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:21:44.476556 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:21:44.487504 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:21:44.553410 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.21 GiB, peak 1.96 GiB, free memory in arenas 5.59 MiB, will set to 1.18 GiB (RSS), difference: -33.75 MiB 2024.10.22 01:21:44.619469 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qaa/wrpsnnrqwkmayizzhhvgstkkaxnkj (LogSeriesLimiter: on interval from 2024-10-22 01:21:27 to 2024-10-22 01:21:44 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:21:44.631988 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1be2a7aa86fa40226321e4bc889c3ab7, offset: 0, size: 0 2024.10.22 01:21:44.637124 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1be2a7aa86fa40226321e4bc889c3ab7 for removal 2024.10.22 01:21:44.644986 [ 1476 ] {} CacheMetadata: Key 1be2a7aa86fa40226321e4bc889c3ab7 is removed from metadata 2024.10.22 01:21:44.723478 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tjn/bvibxvtodfnvqtbtzxmbmgzvipmyr. 2024.10.22 01:21:44.867753 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d68ac72127b53702caa0188390ec9b7d, offset: 0, size: 0 2024.10.22 01:21:44.871163 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d68ac72127b53702caa0188390ec9b7d for removal 2024.10.22 01:21:44.873197 [ 1476 ] {} CacheMetadata: Key d68ac72127b53702caa0188390ec9b7d is removed from metadata 2024.10.22 01:21:44.925350 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:44.931723 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:21:44.993589 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_371_371_0 (state Temporary) 2024.10.22 01:21:45.025689 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tjn/bvibxvtodfnvqtbtzxmbmgzvipmyr. 2024.10.22 01:21:45.175961 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_371_371_0 to 202410_381_381_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:45.319279 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 389 2024.10.22 01:21:45.613235 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 693431 2024.10.22 01:21:45.614310 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_394_172, 202410_395_395_0] 2024.10.22 01:21:45.618788 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_394_172, 202410_395_395_0] 2024.10.22 01:21:45.657002 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_394_172, 202410_395_395_0] 2024.10.22 01:21:45.989312 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_452_230/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:21:46.165696 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for d68ac72127b53702caa0188390ec9b7d:0) 2024.10.22 01:21:46.168636 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 682a633ce222967da864bc4a009f2f5c, offset: 0, size: 1980640 2024.10.22 01:21:46.174442 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 682a633ce222967da864bc4a009f2f5c for removal 2024.10.22 01:21:46.178451 [ 1476 ] {} CacheMetadata: Key 682a633ce222967da864bc4a009f2f5c is removed from metadata 2024.10.22 01:21:46.179298 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 59324b1b96400975d71733487298f062, offset: 0, size: 1250 2024.10.22 01:21:46.182327 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 59324b1b96400975d71733487298f062 for removal 2024.10.22 01:21:46.203100 [ 1476 ] {} CacheMetadata: Key 59324b1b96400975d71733487298f062 is removed from metadata 2024.10.22 01:21:46.547128 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.23 GiB, peak 1.96 GiB, free memory in arenas 5.19 MiB, will set to 1.21 GiB (RSS), difference: -12.45 MiB 2024.10.22 01:21:46.556889 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:46.561601 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:21:46.571324 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mlh/osnlhwafulnnednoboxslorqvliij. 2024.10.22 01:21:46.728185 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 654009bd98fb82526d0223a724d294c8, offset: 0, size: 0 2024.10.22 01:21:46.831289 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 654009bd98fb82526d0223a724d294c8 for removal 2024.10.22 01:21:46.836527 [ 1476 ] {} CacheMetadata: Key 654009bd98fb82526d0223a724d294c8 is removed from metadata 2024.10.22 01:21:46.943498 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_441_441_0 (state Temporary) 2024.10.22 01:21:46.995620 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mlh/osnlhwafulnnednoboxslorqvliij. 2024.10.22 01:21:47.143047 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_441_441_0 to 202410_448_448_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:47.231397 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 693431 2024.10.22 01:21:47.288856 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/aek/cossotqgfcgpivfxtixqwtthhjkqu. 2024.10.22 01:21:47.340611 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 51a7e4dacce55a8ce7927fc7724baaf8, offset: 0, size: 0 2024.10.22 01:21:47.345612 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 51a7e4dacce55a8ce7927fc7724baaf8 for removal 2024.10.22 01:21:47.353003 [ 1476 ] {} CacheMetadata: Key 51a7e4dacce55a8ce7927fc7724baaf8 is removed from metadata 2024.10.22 01:21:47.508754 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/aek/cossotqgfcgpivfxtixqwtthhjkqu. 2024.10.22 01:21:47.585399 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 97 entries to flush up to offset 49890 2024.10.22 01:21:47.607905 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 717000 bytes for 51a7e4dacce55a8ce7927fc7724baaf8:0) 2024.10.22 01:21:47.610429 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0d49b1a88bb9bd17e4f441e4a7f9ee5e, offset: 0, size: 2833854 2024.10.22 01:21:47.616025 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0d49b1a88bb9bd17e4f441e4a7f9ee5e for removal 2024.10.22 01:21:47.628320 [ 1476 ] {} CacheMetadata: Key 0d49b1a88bb9bd17e4f441e4a7f9ee5e is removed from metadata 2024.10.22 01:21:47.628628 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7dcaad95a50721a4d04e6d4d3779d911, offset: 0, size: 849 2024.10.22 01:21:47.632127 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7dcaad95a50721a4d04e6d4d3779d911 for removal 2024.10.22 01:21:47.675990 [ 1476 ] {} CacheMetadata: Key 7dcaad95a50721a4d04e6d4d3779d911 is removed from metadata 2024.10.22 01:21:47.760794 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_787_395} MergeTask::MergeProjectionsStage: Merge sorted 50191 rows, containing 13 columns (13 merged, 0 gathered) in 4.205069323 sec., 11935.831765121176 rows/sec., 1.91 MiB/sec. 2024.10.22 01:21:47.953533 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_787_395} MergedBlockOutputStream: filled checksums 202410_1_787_395 (state Temporary) 2024.10.22 01:21:47.958225 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_452_230} MergeTask::MergeProjectionsStage: Merge sorted 50375 rows, containing 13 columns (13 merged, 0 gathered) in 4.688442917 sec., 10744.505348959974 rows/sec., 4.67 MiB/sec. 2024.10.22 01:21:48.019210 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_452_230} MergedBlockOutputStream: filled checksums 202410_1_452_230 (state Temporary) 2024.10.22 01:21:48.076253 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_687_305, 202410_688_688_0] 2024.10.22 01:21:48.078894 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_687_305, 202410_688_688_0] 2024.10.22 01:21:48.106289 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_687_305, 202410_688_688_0] 2024.10.22 01:21:48.107466 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 01:21:48.157635 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:48.162848 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:21:48.177592 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_787_395} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_787_395 to 202410_1_787_395 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:48.180243 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_787_395} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_786_394, 202410_787_787_0] -> 202410_1_787_395 2024.10.22 01:21:48.204851 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:21:48.372456 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xvm/vgcoqoaplocsukqqozsxcytkkrjur. 2024.10.22 01:21:48.468081 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_780_780_0 (state Temporary) 2024.10.22 01:21:48.509692 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: beebf6d1365afb7e78aa3d1fe5b3af95, offset: 0, size: 0 2024.10.22 01:21:48.512743 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key beebf6d1365afb7e78aa3d1fe5b3af95 for removal 2024.10.22 01:21:48.522768 [ 1476 ] {} CacheMetadata: Key beebf6d1365afb7e78aa3d1fe5b3af95 is removed from metadata 2024.10.22 01:21:48.549005 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_780_780_0 to 202410_793_793_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:48.577263 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xvm/vgcoqoaplocsukqqozsxcytkkrjur. 2024.10.22 01:21:48.616659 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.14 GiB, peak 1.96 GiB, free memory in arenas 3.30 MiB, will set to 1.19 GiB (RSS), difference: 50.21 MiB 2024.10.22 01:21:48.679126 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 49890 2024.10.22 01:21:48.764877 [ 1211 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_787_395 to 202410_788_788_0 2024.10.22 01:21:48.772044 [ 1211 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.80 MiB using storage policy from min volume index 0 2024.10.22 01:21:48.777303 [ 1211 ] {} DiskLocal: Reserved 1.80 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:21:48.806910 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_788_396} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_787_395 to 202410_788_788_0 into Compact with storage Full 2024.10.22 01:21:48.927091 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_788_396} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:48.972007 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_452_230} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_452_230 to 202410_1_452_230 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:48.979173 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_452_230} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_451_229, 202410_452_452_0] -> 202410_1_452_230 2024.10.22 01:21:48.990833 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.13 MiB. 2024.10.22 01:21:49.014043 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:49.018510 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:49.043319 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_788_396} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_787_395, total 50191 rows starting from the beginning of the part 2024.10.22 01:21:49.220875 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xhw/headhyumksywuxsilagctcoopnpem. 2024.10.22 01:21:49.185337 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_788_396} MergeTreeSequentialSource: Reading 2 marks from part 202410_788_788_0, total 81 rows starting from the beginning of the part 2024.10.22 01:21:49.381192 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 306f362c9d6a67ec47e56c76384c12d1, offset: 0, size: 0 2024.10.22 01:21:49.385390 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 306f362c9d6a67ec47e56c76384c12d1 for removal 2024.10.22 01:21:49.389157 [ 1476 ] {} CacheMetadata: Key 306f362c9d6a67ec47e56c76384c12d1 is removed from metadata 2024.10.22 01:21:49.470126 [ 1163 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:21:49.488678 [ 1163 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:21:49.576915 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xhw/headhyumksywuxsilagctcoopnpem. 2024.10.22 01:21:49.867698 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 10 file segments (while reserving 1048576 bytes for e034d8189fdaaf64ec6ac6172158daef:0) 2024.10.22 01:21:49.871030 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e9795e817555046a87eede6ee4854392, offset: 0, size: 537 2024.10.22 01:21:49.877070 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e9795e817555046a87eede6ee4854392 for removal 2024.10.22 01:21:49.880153 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 974ebd537a661daf4b160b81d3edd651, offset: 0, size: 577 2024.10.22 01:21:49.880183 [ 1476 ] {} CacheMetadata: Key e9795e817555046a87eede6ee4854392 is removed from metadata 2024.10.22 01:21:49.884999 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 974ebd537a661daf4b160b81d3edd651 for removal 2024.10.22 01:21:49.888304 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 862756fa66a17b938feb4e4f021055d0, offset: 0, size: 15581 2024.10.22 01:21:49.889139 [ 1476 ] {} CacheMetadata: Key 974ebd537a661daf4b160b81d3edd651 is removed from metadata 2024.10.22 01:21:49.891624 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 862756fa66a17b938feb4e4f021055d0 for removal 2024.10.22 01:21:49.894286 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8a0592534c2e1d4e7dbb03fac0e7d848, offset: 0, size: 349 2024.10.22 01:21:49.898739 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8a0592534c2e1d4e7dbb03fac0e7d848 for removal 2024.10.22 01:21:49.898878 [ 1476 ] {} CacheMetadata: Key 862756fa66a17b938feb4e4f021055d0 is removed from metadata 2024.10.22 01:21:49.903593 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 021361a750276539e3ae5b46cdfd5f3b, offset: 0, size: 384 2024.10.22 01:21:49.906751 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 021361a750276539e3ae5b46cdfd5f3b for removal 2024.10.22 01:21:49.912992 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cbaedcedf0d889c3c90bf60a6f369cbf, offset: 0, size: 76659 2024.10.22 01:21:49.919641 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cbaedcedf0d889c3c90bf60a6f369cbf for removal 2024.10.22 01:21:49.923881 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3cf9719091c93a7561e633c2c796c1ed, offset: 0, size: 739 2024.10.22 01:21:49.923956 [ 1476 ] {} CacheMetadata: Key cbaedcedf0d889c3c90bf60a6f369cbf is removed from metadata 2024.10.22 01:21:49.967043 [ 1476 ] {} CacheMetadata: Key 021361a750276539e3ae5b46cdfd5f3b is removed from metadata 2024.10.22 01:21:50.030945 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3cf9719091c93a7561e633c2c796c1ed for removal 2024.10.22 01:21:50.035115 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 258b4ab77e04857fb74fc7d5ffd4b711, offset: 0, size: 2695496 2024.10.22 01:21:50.047299 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 258b4ab77e04857fb74fc7d5ffd4b711 for removal 2024.10.22 01:21:50.054480 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ef4de0a717082ae25cdf913d12ce2621, offset: 0, size: 760 2024.10.22 01:21:50.057504 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ef4de0a717082ae25cdf913d12ce2621 for removal 2024.10.22 01:21:50.060515 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0386b33d8d59ea03ba2327169a56f4ea, offset: 0, size: 1026 2024.10.22 01:21:50.064706 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0386b33d8d59ea03ba2327169a56f4ea for removal 2024.10.22 01:21:50.106475 [ 1476 ] {} CacheMetadata: Key 0386b33d8d59ea03ba2327169a56f4ea is removed from metadata 2024.10.22 01:21:50.113693 [ 1476 ] {} CacheMetadata: Key ef4de0a717082ae25cdf913d12ce2621 is removed from metadata 2024.10.22 01:21:50.130343 [ 1476 ] {} CacheMetadata: Key 258b4ab77e04857fb74fc7d5ffd4b711 is removed from metadata 2024.10.22 01:21:50.158244 [ 1476 ] {} CacheMetadata: Key 3cf9719091c93a7561e633c2c796c1ed is removed from metadata 2024.10.22 01:21:50.175992 [ 1476 ] {} CacheMetadata: Key 8a0592534c2e1d4e7dbb03fac0e7d848 is removed from metadata 2024.10.22 01:21:50.287815 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:21:50.291972 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:21:50.388158 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.20 GiB, peak 1.96 GiB, free memory in arenas 3.46 MiB, will set to 1.19 GiB (RSS), difference: -11.17 MiB 2024.10.22 01:21:50.486312 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 7112 2024.10.22 01:21:50.601404 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xtl/uipmfpvjubuhbaiuppksgzotzyojc. 2024.10.22 01:21:50.617769 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:50.626410 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:50.650968 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: aba539cefa692bca6942df225bc58847, offset: 0, size: 0 2024.10.22 01:21:50.655767 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key aba539cefa692bca6942df225bc58847 for removal 2024.10.22 01:21:50.659157 [ 1476 ] {} CacheMetadata: Key aba539cefa692bca6942df225bc58847 is removed from metadata 2024.10.22 01:21:50.670644 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_460_460_0 (state Temporary) 2024.10.22 01:21:50.720344 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xtl/uipmfpvjubuhbaiuppksgzotzyojc. 2024.10.22 01:21:50.727960 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_460_460_0 to 202410_473_473_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:50.777641 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 233 entries to flush up to offset 51683 2024.10.22 01:21:50.801719 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7112 2024.10.22 01:21:51.327115 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:51.331137 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:51.413734 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_452_452_0 (state Temporary) 2024.10.22 01:21:51.429554 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2dd9b1212cdf63e0259701e2d44b7809, offset: 0, size: 0 2024.10.22 01:21:51.535791 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2dd9b1212cdf63e0259701e2d44b7809 for removal 2024.10.22 01:21:51.651295 [ 1476 ] {} CacheMetadata: Key 2dd9b1212cdf63e0259701e2d44b7809 is removed from metadata 2024.10.22 01:21:51.715800 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mjj/osfczchqiomyjtrkihuwaaugjkujc. 2024.10.22 01:21:51.830354 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_452_452_0 to 202410_458_458_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:51.880527 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_452_230 to 202410_453_453_0 2024.10.22 01:21:51.885163 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.56 MiB using storage policy from min volume index 0 2024.10.22 01:21:51.888328 [ 1193 ] {} DiskLocal: Reserved 2.56 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:51.900476 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 51683 2024.10.22 01:21:51.970511 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 03d8b3c6058571e4771665b069e12514, offset: 0, size: 0 2024.10.22 01:21:51.972069 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 03d8b3c6058571e4771665b069e12514 for removal 2024.10.22 01:21:51.974431 [ 1476 ] {} CacheMetadata: Key 03d8b3c6058571e4771665b069e12514 is removed from metadata 2024.10.22 01:21:52.051320 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mjj/osfczchqiomyjtrkihuwaaugjkujc. 2024.10.22 01:21:52.054523 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_453_231} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_452_230 to 202410_453_453_0 into Wide with storage Full 2024.10.22 01:21:52.054920 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tor/spmugwarivyhniwjeplzjzczallfm. 2024.10.22 01:21:52.085480 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_453_231} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:52.092596 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5e138c155f144795011d6af2c59d0d74, offset: 0, size: 0 2024.10.22 01:21:52.100865 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_453_231} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_452_230, total 50375 rows starting from the beginning of the part 2024.10.22 01:21:52.101109 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5e138c155f144795011d6af2c59d0d74 for removal 2024.10.22 01:21:52.106758 [ 1476 ] {} CacheMetadata: Key 5e138c155f144795011d6af2c59d0d74 is removed from metadata 2024.10.22 01:21:52.126021 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_93_93_0 (state Temporary) 2024.10.22 01:21:52.243910 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_453_231} MergeTreeSequentialSource: Reading 2 marks from part 202410_453_453_0, total 233 rows starting from the beginning of the part 2024.10.22 01:21:52.376467 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tor/spmugwarivyhniwjeplzjzczallfm. 2024.10.22 01:21:52.434958 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_788_396} MergeTask::MergeProjectionsStage: Merge sorted 50272 rows, containing 13 columns (13 merged, 0 gathered) in 3.641693226 sec., 13804.567513013244 rows/sec., 2.21 MiB/sec. 2024.10.22 01:21:52.500654 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_788_396} MergedBlockOutputStream: filled checksums 202410_1_788_396 (state Temporary) 2024.10.22 01:21:52.506172 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 1048576 bytes for 5e138c155f144795011d6af2c59d0d74:0) 2024.10.22 01:21:52.511426 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e720c25457e0ec7d4cadf23d1d570803, offset: 0, size: 1498903 2024.10.22 01:21:52.518140 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e720c25457e0ec7d4cadf23d1d570803 for removal 2024.10.22 01:21:52.522694 [ 1476 ] {} CacheMetadata: Key e720c25457e0ec7d4cadf23d1d570803 is removed from metadata 2024.10.22 01:21:52.616356 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.22 GiB, peak 1.96 GiB, free memory in arenas 7.23 MiB, will set to 1.19 GiB (RSS), difference: -28.08 MiB 2024.10.22 01:21:52.735902 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_688_306, 202410_689_689_0] 2024.10.22 01:21:52.827042 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 390 2024.10.22 01:21:52.828661 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_788_396} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_788_396 to 202410_1_788_396 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:52.883222 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 2097152 bytes for e034d8189fdaaf64ec6ac6172158daef:0) 2024.10.22 01:21:53.156422 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 128 entries to flush up to offset 50018 2024.10.22 01:21:53.158466 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bf4de651f62646b449a06eb48638cf68, offset: 0, size: 1281835 2024.10.22 01:21:53.159208 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_688_306, 202410_689_689_0] 2024.10.22 01:21:53.160089 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_788_396} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_787_395, 202410_788_788_0] -> 202410_1_788_396 2024.10.22 01:21:53.164942 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bf4de651f62646b449a06eb48638cf68 for removal 2024.10.22 01:21:53.169753 [ 1476 ] {} CacheMetadata: Key bf4de651f62646b449a06eb48638cf68 is removed from metadata 2024.10.22 01:21:53.169763 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bfdc9d116427fb88714af166132d9fdd, offset: 0, size: 872 2024.10.22 01:21:53.169727 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_688_306, 202410_689_689_0] 2024.10.22 01:21:53.176553 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bfdc9d116427fb88714af166132d9fdd for removal 2024.10.22 01:21:53.179090 [ 1476 ] {} CacheMetadata: Key bfdc9d116427fb88714af166132d9fdd is removed from metadata 2024.10.22 01:21:53.179544 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3263a6b67df1cef40031e3a33b0d63f7, offset: 0, size: 1012 2024.10.22 01:21:53.183775 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3263a6b67df1cef40031e3a33b0d63f7 for removal 2024.10.22 01:21:53.186311 [ 1476 ] {} CacheMetadata: Key 3263a6b67df1cef40031e3a33b0d63f7 is removed from metadata 2024.10.22 01:21:53.203503 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 1048576 bytes for 5e138c155f144795011d6af2c59d0d74:0) 2024.10.22 01:21:53.209216 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 943c26921857b165baf6c892b7ec2804, offset: 0, size: 1148739 2024.10.22 01:21:53.243504 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:21:53.248189 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 943c26921857b165baf6c892b7ec2804 for removal 2024.10.22 01:21:53.252412 [ 1476 ] {} CacheMetadata: Key 943c26921857b165baf6c892b7ec2804 is removed from metadata 2024.10.22 01:21:53.259689 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_395_173, 202410_396_396_0] 2024.10.22 01:21:53.266532 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_395_173, 202410_396_396_0] 2024.10.22 01:21:53.289018 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_395_173, 202410_396_396_0] 2024.10.22 01:21:53.459070 [ 1207 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 01:21:53.522937 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pik/jkejfhtakwpxeenqrpcbvzuitdiii. 2024.10.22 01:21:53.531163 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:53.542720 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:53.595527 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1949be129636de003c4ddef0ff089aa4, offset: 0, size: 0 2024.10.22 01:21:53.598353 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1949be129636de003c4ddef0ff089aa4 for removal 2024.10.22 01:21:53.606447 [ 1476 ] {} CacheMetadata: Key 1949be129636de003c4ddef0ff089aa4 is removed from metadata 2024.10.22 01:21:53.667024 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pik/jkejfhtakwpxeenqrpcbvzuitdiii. 2024.10.22 01:21:53.721923 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5489d8815b525c3c9e94d1f1f58314af, offset: 0, size: 0 2024.10.22 01:21:53.724471 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5489d8815b525c3c9e94d1f1f58314af for removal 2024.10.22 01:21:53.728115 [ 1476 ] {} CacheMetadata: Key 5489d8815b525c3c9e94d1f1f58314af is removed from metadata 2024.10.22 01:21:53.760223 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_372_372_0 (state Temporary) 2024.10.22 01:21:53.820818 [ 1208 ] {} DatabaseCatalog: Execution took 245 ms. 2024.10.22 01:21:53.962739 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:53.965506 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:54.043487 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_372_372_0 to 202410_382_382_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:54.135539 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 390 2024.10.22 01:21:54.239379 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 694971 2024.10.22 01:21:54.268417 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_781_781_0 (state Temporary) 2024.10.22 01:21:54.302342 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 1048576 bytes for 5489d8815b525c3c9e94d1f1f58314af:0) 2024.10.22 01:21:54.316918 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3a885f4b6d48fa8efa56ba9fc5f37fb4, offset: 0, size: 907024 2024.10.22 01:21:54.320410 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3a885f4b6d48fa8efa56ba9fc5f37fb4 for removal 2024.10.22 01:21:54.323706 [ 1476 ] {} CacheMetadata: Key 3a885f4b6d48fa8efa56ba9fc5f37fb4 is removed from metadata 2024.10.22 01:21:54.324685 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9fdbd484e2b5325c2e24c31143bd019f, offset: 0, size: 824 2024.10.22 01:21:54.392994 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.21 GiB, peak 1.96 GiB, free memory in arenas 6.60 MiB, will set to 1.21 GiB (RSS), difference: 3.74 MiB 2024.10.22 01:21:54.428897 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_781_781_0 to 202410_794_794_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:54.430231 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9fdbd484e2b5325c2e24c31143bd019f for removal 2024.10.22 01:21:54.432796 [ 1476 ] {} CacheMetadata: Key 9fdbd484e2b5325c2e24c31143bd019f is removed from metadata 2024.10.22 01:21:54.433030 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 57ea488def117d130794d557978ac76d, offset: 0, size: 1086 2024.10.22 01:21:54.437332 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 57ea488def117d130794d557978ac76d for removal 2024.10.22 01:21:54.441234 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: daedfdfd044c677a7992dc95673d365c, offset: 0, size: 764813 2024.10.22 01:21:54.452055 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key daedfdfd044c677a7992dc95673d365c for removal 2024.10.22 01:21:54.455425 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2896ab96d35c222fab36528c8da7948a, offset: 0, size: 980 2024.10.22 01:21:54.460973 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2896ab96d35c222fab36528c8da7948a for removal 2024.10.22 01:21:54.462634 [ 1476 ] {} CacheMetadata: Key daedfdfd044c677a7992dc95673d365c is removed from metadata 2024.10.22 01:21:54.472139 [ 1476 ] {} CacheMetadata: Key 2896ab96d35c222fab36528c8da7948a is removed from metadata 2024.10.22 01:21:54.480139 [ 1476 ] {} CacheMetadata: Key 57ea488def117d130794d557978ac76d is removed from metadata 2024.10.22 01:21:54.582751 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 50018 2024.10.22 01:21:54.674691 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_788_396 to 202410_789_789_0 2024.10.22 01:21:54.678609 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.81 MiB using storage policy from min volume index 0 2024.10.22 01:21:54.680857 [ 1216 ] {} DiskLocal: Reserved 1.81 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:54.685836 [ 1216 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 01:21:54.700785 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_789_397} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_788_396 to 202410_789_789_0 into Compact with storage Full 2024.10.22 01:21:54.767518 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_789_397} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:54.783629 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_789_397} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_788_396, total 50272 rows starting from the beginning of the part 2024.10.22 01:21:54.850955 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_789_397} MergeTreeSequentialSource: Reading 2 marks from part 202410_789_789_0, total 109 rows starting from the beginning of the part 2024.10.22 01:21:55.223379 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nna/zktxqgzbamiffvxegndvzqogdrojb. 2024.10.22 01:21:55.299852 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:55.305372 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:55.340713 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_789_397/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:21:55.571389 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 881ef692f68b82bf415cbff5f4c0d0f3, offset: 0, size: 0 2024.10.22 01:21:55.571873 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_442_442_0 (state Temporary) 2024.10.22 01:21:55.573037 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 881ef692f68b82bf415cbff5f4c0d0f3 for removal 2024.10.22 01:21:55.577107 [ 1476 ] {} CacheMetadata: Key 881ef692f68b82bf415cbff5f4c0d0f3 is removed from metadata 2024.10.22 01:21:55.584175 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_453_231} MergeTask::MergeProjectionsStage: Merge sorted 50608 rows, containing 13 columns (13 merged, 0 gathered) in 3.546343786 sec., 14270.472084456846 rows/sec., 6.20 MiB/sec. 2024.10.22 01:21:55.688003 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_442_442_0 to 202410_449_449_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:55.716471 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nna/zktxqgzbamiffvxegndvzqogdrojb. 2024.10.22 01:21:55.732675 [ 1229 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_444_89 to 202410_449_449_0 2024.10.22 01:21:55.752601 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 694971 2024.10.22 01:21:55.754842 [ 1229 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:55.759961 [ 1229 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:55.784306 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_444_89 to 202410_449_449_0 into Wide with storage Full 2024.10.22 01:21:55.915761 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:21:55.930172 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_453_231} MergedBlockOutputStream: filled checksums 202410_1_453_231 (state Temporary) 2024.10.22 01:21:55.965454 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 50972 bytes for 881ef692f68b82bf415cbff5f4c0d0f3:0) 2024.10.22 01:21:55.966598 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 13edbad82cf5b37bc07f1066b7c65045, offset: 0, size: 2514155 2024.10.22 01:21:55.970134 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 13edbad82cf5b37bc07f1066b7c65045 for removal 2024.10.22 01:21:55.973961 [ 1476 ] {} CacheMetadata: Key 13edbad82cf5b37bc07f1066b7c65045 is removed from metadata 2024.10.22 01:21:55.976071 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 59adb4c8ea06a098dbe0a1cf1a9ecd98, offset: 0, size: 815 2024.10.22 01:21:55.980577 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 59adb4c8ea06a098dbe0a1cf1a9ecd98 for removal 2024.10.22 01:21:55.992712 [ 1476 ] {} CacheMetadata: Key 59adb4c8ea06a098dbe0a1cf1a9ecd98 is removed from metadata 2024.10.22 01:21:55.929800 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} MergeTreeSequentialSource: Reading 87 marks from part 202410_1_444_89, total 700474 rows starting from the beginning of the part 2024.10.22 01:21:56.106661 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_445_445_0, total 1542 rows starting from the beginning of the part 2024.10.22 01:21:56.242325 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_446_446_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:21:56.269519 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_447_447_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:21:56.293285 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_448_448_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:21:56.362590 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mkh/ayumpqdmxkixwzsicvgxikaxkahke. 2024.10.22 01:21:56.456566 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_449_449_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:21:56.490256 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.96 GiB, free memory in arenas 6.25 MiB, will set to 1.21 GiB (RSS), difference: 48.32 MiB 2024.10.22 01:21:56.500432 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 54c18d418786ac23a2410efb9290fff9, offset: 0, size: 0 2024.10.22 01:21:56.505307 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 54c18d418786ac23a2410efb9290fff9 for removal 2024.10.22 01:21:56.511659 [ 1476 ] {} CacheMetadata: Key 54c18d418786ac23a2410efb9290fff9 is removed from metadata 2024.10.22 01:21:56.560863 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3db5a497b55e20c554d8f3947b44e855, offset: 0, size: 0 2024.10.22 01:21:56.562575 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3db5a497b55e20c554d8f3947b44e855 for removal 2024.10.22 01:21:56.567725 [ 1476 ] {} CacheMetadata: Key 3db5a497b55e20c554d8f3947b44e855 is removed from metadata 2024.10.22 01:21:56.572070 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mkh/ayumpqdmxkixwzsicvgxikaxkahke. 2024.10.22 01:21:56.768939 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_453_231} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_453_231 to 202410_1_453_231 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:56.872892 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_453_231} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_452_230, 202410_453_453_0] -> 202410_1_453_231 2024.10.22 01:21:56.888809 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.13 MiB. 2024.10.22 01:21:57.491706 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8253e81d11ffae589550ba7bf1c55ce4, offset: 0, size: 0 2024.10.22 01:21:57.493982 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8253e81d11ffae589550ba7bf1c55ce4 for removal 2024.10.22 01:21:57.497843 [ 1476 ] {} CacheMetadata: Key 8253e81d11ffae589550ba7bf1c55ce4 is removed from metadata 2024.10.22 01:21:57.568427 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 80399e0e55a63a3de06370f8f06dd1db, offset: 0, size: 0 2024.10.22 01:21:57.571321 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 80399e0e55a63a3de06370f8f06dd1db for removal 2024.10.22 01:21:57.574861 [ 1476 ] {} CacheMetadata: Key 80399e0e55a63a3de06370f8f06dd1db is removed from metadata 2024.10.22 01:21:57.731047 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ba55df74268f270a2be3a495e6194867, offset: 0, size: 0 2024.10.22 01:21:57.733286 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ba55df74268f270a2be3a495e6194867 for removal 2024.10.22 01:21:57.832986 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: edb0e095985172dfe711fe35cd8f636a, offset: 0, size: 0 2024.10.22 01:21:57.836524 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key edb0e095985172dfe711fe35cd8f636a for removal 2024.10.22 01:21:57.845218 [ 1476 ] {} CacheMetadata: Key ba55df74268f270a2be3a495e6194867 is removed from metadata 2024.10.22 01:21:57.855229 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_689_307, 202410_690_690_0] 2024.10.22 01:21:57.867824 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_689_307, 202410_690_690_0] 2024.10.22 01:21:57.882117 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_689_307, 202410_690_690_0] 2024.10.22 01:21:57.941699 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tmv/wryvhchwfilyrchvxyktfviavixmx. 2024.10.22 01:21:57.954700 [ 1476 ] {} CacheMetadata: Key edb0e095985172dfe711fe35cd8f636a is removed from metadata 2024.10.22 01:21:58.071128 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a453ecee9b5ad6cf7519aec91c03e8a6, offset: 0, size: 0 2024.10.22 01:21:58.074400 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a453ecee9b5ad6cf7519aec91c03e8a6 for removal 2024.10.22 01:21:58.079832 [ 1476 ] {} CacheMetadata: Key a453ecee9b5ad6cf7519aec91c03e8a6 is removed from metadata 2024.10.22 01:21:58.147111 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7b42a44984289b7c95f74c321c41c92c, offset: 0, size: 0 2024.10.22 01:21:58.153265 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7b42a44984289b7c95f74c321c41c92c for removal 2024.10.22 01:21:58.157257 [ 1476 ] {} CacheMetadata: Key 7b42a44984289b7c95f74c321c41c92c is removed from metadata 2024.10.22 01:21:58.168438 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tmv/wryvhchwfilyrchvxyktfviavixmx. 2024.10.22 01:21:58.304617 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 7123 2024.10.22 01:21:58.373324 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 23bda489b4b3b2fd42737590134725c2, offset: 0, size: 0 2024.10.22 01:21:58.375652 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 23bda489b4b3b2fd42737590134725c2 for removal 2024.10.22 01:21:58.378806 [ 1476 ] {} CacheMetadata: Key 23bda489b4b3b2fd42737590134725c2 is removed from metadata 2024.10.22 01:21:58.479104 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kmq/rhnriyhuphhzzbfztukfuknoowuuw. 2024.10.22 01:21:58.540540 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 00ed69f37fc99153adfcc47ce3d74340, offset: 0, size: 0 2024.10.22 01:21:58.542417 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 00ed69f37fc99153adfcc47ce3d74340 for removal 2024.10.22 01:21:58.553041 [ 1476 ] {} CacheMetadata: Key 00ed69f37fc99153adfcc47ce3d74340 is removed from metadata 2024.10.22 01:21:58.522577 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_789_397} MergeTask::MergeProjectionsStage: Merge sorted 50381 rows, containing 13 columns (13 merged, 0 gathered) in 3.833264812 sec., 13143.104499924646 rows/sec., 2.11 MiB/sec. 2024.10.22 01:21:58.598379 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a02450b68ca206e5d6a00364a004aa7d, offset: 0, size: 0 2024.10.22 01:21:58.608187 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 146 entries to flush up to offset 50164 2024.10.22 01:21:58.612067 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a02450b68ca206e5d6a00364a004aa7d for removal 2024.10.22 01:21:58.619009 [ 1476 ] {} CacheMetadata: Key a02450b68ca206e5d6a00364a004aa7d is removed from metadata 2024.10.22 01:21:58.626445 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:58.632042 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:58.757576 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kmq/rhnriyhuphhzzbfztukfuknoowuuw. 2024.10.22 01:21:58.787555 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_461_461_0 (state Temporary) 2024.10.22 01:21:58.884941 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.20 GiB, peak 1.96 GiB, free memory in arenas 12.61 MiB, will set to 1.21 GiB (RSS), difference: 7.95 MiB 2024.10.22 01:21:58.913789 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_789_397} MergedBlockOutputStream: filled checksums 202410_1_789_397 (state Temporary) 2024.10.22 01:21:58.975050 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_461_461_0 to 202410_474_474_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:58.988039 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 67a72c81630238eaadfce21578ebf1d3, offset: 0, size: 0 2024.10.22 01:21:59.008635 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 67a72c81630238eaadfce21578ebf1d3 for removal 2024.10.22 01:21:59.077681 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7123 2024.10.22 01:21:59.099208 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_94_19/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:21:59.118936 [ 1476 ] {} CacheMetadata: Key 67a72c81630238eaadfce21578ebf1d3 is removed from metadata 2024.10.22 01:21:59.155507 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_789_397} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_789_397 to 202410_1_789_397 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:21:59.161567 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_789_397} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_788_396, 202410_789_789_0] -> 202410_1_789_397 2024.10.22 01:21:59.219760 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qsk/mutmrfmtdegmnlcarchnyuqtcrsaz. 2024.10.22 01:21:59.260369 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5bf87b550953995e2ba638525f0975e4, offset: 0, size: 0 2024.10.22 01:21:59.262955 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5bf87b550953995e2ba638525f0975e4 for removal 2024.10.22 01:21:59.265204 [ 1476 ] {} CacheMetadata: Key 5bf87b550953995e2ba638525f0975e4 is removed from metadata 2024.10.22 01:21:59.282507 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:21:59.438631 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0d98703a87c615e093030f93ff5b5de2, offset: 0, size: 0 2024.10.22 01:21:59.441536 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0d98703a87c615e093030f93ff5b5de2 for removal 2024.10.22 01:21:59.449453 [ 1476 ] {} CacheMetadata: Key 0d98703a87c615e093030f93ff5b5de2 is removed from metadata 2024.10.22 01:21:59.464649 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 251 entries to flush up to offset 51934 2024.10.22 01:21:59.464639 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qsk/mutmrfmtdegmnlcarchnyuqtcrsaz. 2024.10.22 01:21:59.557077 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zfh/tnwozxabqisrkirooholjgogtnyug. 2024.10.22 01:21:59.580761 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_449_90/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:21:59.607729 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ce9ad911b1b3578a3b7aca47a87b65a6, offset: 0, size: 0 2024.10.22 01:21:59.610562 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ce9ad911b1b3578a3b7aca47a87b65a6 for removal 2024.10.22 01:21:59.613543 [ 1476 ] {} CacheMetadata: Key ce9ad911b1b3578a3b7aca47a87b65a6 is removed from metadata 2024.10.22 01:21:59.664715 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a5395ab220e2220f2f6d06777af4a0ea, offset: 0, size: 0 2024.10.22 01:21:59.666572 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a5395ab220e2220f2f6d06777af4a0ea for removal 2024.10.22 01:21:59.667918 [ 1476 ] {} CacheMetadata: Key a5395ab220e2220f2f6d06777af4a0ea is removed from metadata 2024.10.22 01:21:59.681776 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zfh/tnwozxabqisrkirooholjgogtnyug. 2024.10.22 01:21:59.785141 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:59.797248 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:59.814836 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:21:59.822453 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:21:59.869475 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_453_453_0 (state Temporary) 2024.10.22 01:21:59.921603 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_782_782_0 (state Temporary) 2024.10.22 01:21:59.976261 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_453_453_0 to 202410_459_459_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:00.002401 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_453_231 to 202410_454_454_0 2024.10.22 01:22:00.039817 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.57 MiB using storage policy from min volume index 0 2024.10.22 01:22:00.061228 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 51934 2024.10.22 01:22:00.148218 [ 1248 ] {} DiskLocal: Reserved 2.57 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:00.157453 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_782_782_0 to 202410_795_795_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:00.170995 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_454_232} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_453_231 to 202410_454_454_0 into Wide with storage Full 2024.10.22 01:22:00.184661 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_789_397 to 202410_790_790_0 2024.10.22 01:22:00.188232 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:22:00.188794 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.81 MiB using storage policy from min volume index 0 2024.10.22 01:22:00.190682 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_454_232} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:00.190691 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:22:00.190509 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 50164 2024.10.22 01:22:00.194998 [ 1263 ] {} DiskLocal: Reserved 1.81 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:00.204076 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_454_232} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_453_231, total 50608 rows starting from the beginning of the part 2024.10.22 01:22:00.231701 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_790_398} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_789_397 to 202410_790_790_0 into Compact with storage Full 2024.10.22 01:22:00.270563 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 48 file segments (while reserving 4194304 bytes for e034d8189fdaaf64ec6ac6172158daef:0) 2024.10.22 01:22:00.270464 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_396_174, 202410_397_397_0] 2024.10.22 01:22:00.274198 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cbd3c930dd72e3d362befb72c68430d5, offset: 0, size: 22970616 2024.10.22 01:22:00.275438 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_396_174, 202410_397_397_0] 2024.10.22 01:22:00.286340 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_790_398} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:00.293365 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cbd3c930dd72e3d362befb72c68430d5 for removal 2024.10.22 01:22:00.300466 [ 1476 ] {} CacheMetadata: Key cbd3c930dd72e3d362befb72c68430d5 is removed from metadata 2024.10.22 01:22:00.304538 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5a840533b39e1c2c0bebb9d1404804d5, offset: 0, size: 4 2024.10.22 01:22:00.311038 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5a840533b39e1c2c0bebb9d1404804d5 for removal 2024.10.22 01:22:00.318837 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 812709cfcf7bca5aad2b6d9e2e88def7, offset: 0, size: 867630 2024.10.22 01:22:00.324771 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_790_398} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_789_397, total 50381 rows starting from the beginning of the part 2024.10.22 01:22:00.335150 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 812709cfcf7bca5aad2b6d9e2e88def7 for removal 2024.10.22 01:22:00.347476 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6806fd49816baebea28b3e9d42ffb2a7, offset: 0, size: 839 2024.10.22 01:22:00.356996 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6806fd49816baebea28b3e9d42ffb2a7 for removal 2024.10.22 01:22:00.363085 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_790_398} MergeTreeSequentialSource: Reading 2 marks from part 202410_790_790_0, total 82 rows starting from the beginning of the part 2024.10.22 01:22:00.364560 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_454_232} MergeTreeSequentialSource: Reading 2 marks from part 202410_454_454_0, total 285 rows starting from the beginning of the part 2024.10.22 01:22:00.365112 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_396_174, 202410_397_397_0] 2024.10.22 01:22:00.404735 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.18 GiB, peak 1.96 GiB, free memory in arenas 16.39 MiB, will set to 1.20 GiB (RSS), difference: 13.81 MiB 2024.10.22 01:22:00.434662 [ 1476 ] {} CacheMetadata: Key 812709cfcf7bca5aad2b6d9e2e88def7 is removed from metadata 2024.10.22 01:22:00.471664 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6c266b00bc3d41b799f20dc76d59ebfa, offset: 0, size: 770 2024.10.22 01:22:00.476248 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6c266b00bc3d41b799f20dc76d59ebfa for removal 2024.10.22 01:22:00.479216 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 59fd00f483e52754c06330734b2eda70, offset: 0, size: 922 2024.10.22 01:22:00.482289 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 59fd00f483e52754c06330734b2eda70 for removal 2024.10.22 01:22:00.485268 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: fc962699ae5f0ab2c64f629bc3d688ca, offset: 0, size: 867 2024.10.22 01:22:00.488027 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key fc962699ae5f0ab2c64f629bc3d688ca for removal 2024.10.22 01:22:00.491330 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9d3ce133ba8daf034e448cacb89cd188, offset: 0, size: 545 2024.10.22 01:22:00.494264 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9d3ce133ba8daf034e448cacb89cd188 for removal 2024.10.22 01:22:00.502601 [ 1476 ] {} CacheMetadata: Key 9d3ce133ba8daf034e448cacb89cd188 is removed from metadata 2024.10.22 01:22:00.514577 [ 1476 ] {} CacheMetadata: Key fc962699ae5f0ab2c64f629bc3d688ca is removed from metadata 2024.10.22 01:22:00.527418 [ 1476 ] {} CacheMetadata: Key 59fd00f483e52754c06330734b2eda70 is removed from metadata 2024.10.22 01:22:00.553134 [ 1476 ] {} CacheMetadata: Key 6c266b00bc3d41b799f20dc76d59ebfa is removed from metadata 2024.10.22 01:22:00.561636 [ 1476 ] {} CacheMetadata: Key 6806fd49816baebea28b3e9d42ffb2a7 is removed from metadata 2024.10.22 01:22:00.576325 [ 1476 ] {} CacheMetadata: Key 5a840533b39e1c2c0bebb9d1404804d5 is removed from metadata 2024.10.22 01:22:00.497793 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 72579ff28fefca1f009b2109eccc9e99, offset: 0, size: 685 2024.10.22 01:22:00.603764 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 72579ff28fefca1f009b2109eccc9e99 for removal 2024.10.22 01:22:00.610358 [ 1476 ] {} CacheMetadata: Key 72579ff28fefca1f009b2109eccc9e99 is removed from metadata 2024.10.22 01:22:00.611156 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5af2653727bdf2322eca07f36104f14b, offset: 0, size: 58 2024.10.22 01:22:00.617487 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5af2653727bdf2322eca07f36104f14b for removal 2024.10.22 01:22:00.627598 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 91423d229f4ef19f2a267c48130bd9b4, offset: 0, size: 54 2024.10.22 01:22:00.627654 [ 1476 ] {} CacheMetadata: Key 5af2653727bdf2322eca07f36104f14b is removed from metadata 2024.10.22 01:22:00.641550 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 91423d229f4ef19f2a267c48130bd9b4 for removal 2024.10.22 01:22:00.653649 [ 1476 ] {} CacheMetadata: Key 91423d229f4ef19f2a267c48130bd9b4 is removed from metadata 2024.10.22 01:22:00.655367 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9e3a31b0d91e05c29340f05dae0fd255, offset: 0, size: 58 2024.10.22 01:22:00.663954 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9e3a31b0d91e05c29340f05dae0fd255 for removal 2024.10.22 01:22:00.667971 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e852923e06de0a3cadc45d0290a4a719, offset: 0, size: 54 2024.10.22 01:22:00.672663 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e852923e06de0a3cadc45d0290a4a719 for removal 2024.10.22 01:22:00.677758 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4cb4468ca30005cdec39d9b8cdc32fe4, offset: 0, size: 58 2024.10.22 01:22:00.684337 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 4cb4468ca30005cdec39d9b8cdc32fe4 for removal 2024.10.22 01:22:00.689475 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 354f3d3599f64171df269afdf06572f6, offset: 0, size: 535 2024.10.22 01:22:00.692475 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 354f3d3599f64171df269afdf06572f6 for removal 2024.10.22 01:22:00.765360 [ 1476 ] {} CacheMetadata: Key 4cb4468ca30005cdec39d9b8cdc32fe4 is removed from metadata 2024.10.22 01:22:00.774455 [ 1476 ] {} CacheMetadata: Key e852923e06de0a3cadc45d0290a4a719 is removed from metadata 2024.10.22 01:22:00.782915 [ 1476 ] {} CacheMetadata: Key 9e3a31b0d91e05c29340f05dae0fd255 is removed from metadata 2024.10.22 01:22:00.798566 [ 1476 ] {} CacheMetadata: Key 354f3d3599f64171df269afdf06572f6 is removed from metadata 2024.10.22 01:22:00.798861 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 01185e30b92e9cf36578048bad48d3c3, offset: 0, size: 513 2024.10.22 01:22:00.862238 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_93_93_0 to 202410_96_96_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:00.805601 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 01185e30b92e9cf36578048bad48d3c3 for removal 2024.10.22 01:22:00.912828 [ 1476 ] {} CacheMetadata: Key 01185e30b92e9cf36578048bad48d3c3 is removed from metadata 2024.10.22 01:22:00.913317 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1008f8b2b23608f026a0eca804d7e9e6, offset: 0, size: 412 2024.10.22 01:22:00.921674 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1008f8b2b23608f026a0eca804d7e9e6 for removal 2024.10.22 01:22:00.926255 [ 1476 ] {} CacheMetadata: Key 1008f8b2b23608f026a0eca804d7e9e6 is removed from metadata 2024.10.22 01:22:00.927822 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 21a13775ad9443503e5fc707464017dd, offset: 0, size: 469 2024.10.22 01:22:00.936354 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 21a13775ad9443503e5fc707464017dd for removal 2024.10.22 01:22:00.947358 [ 1476 ] {} CacheMetadata: Key 21a13775ad9443503e5fc707464017dd is removed from metadata 2024.10.22 01:22:00.955036 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 390954f1d9c6d5eca8e4c37ba3c8dab8, offset: 0, size: 1140 2024.10.22 01:22:00.962549 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 390954f1d9c6d5eca8e4c37ba3c8dab8 for removal 2024.10.22 01:22:00.967805 [ 1476 ] {} CacheMetadata: Key 390954f1d9c6d5eca8e4c37ba3c8dab8 is removed from metadata 2024.10.22 01:22:00.969175 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 78783b34e135d26d5cf6105c45721b5a, offset: 0, size: 75 2024.10.22 01:22:00.973755 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 78783b34e135d26d5cf6105c45721b5a for removal 2024.10.22 01:22:01.102835 [ 1476 ] {} CacheMetadata: Key 78783b34e135d26d5cf6105c45721b5a is removed from metadata 2024.10.22 01:22:01.106379 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0b56f165cb7155f2f0d6ed143e4f7bfb, offset: 0, size: 1093 2024.10.22 01:22:01.110996 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0b56f165cb7155f2f0d6ed143e4f7bfb for removal 2024.10.22 01:22:01.116997 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 51d52325bcb201a6f27d7214c9a4d327, offset: 0, size: 783 2024.10.22 01:22:01.120595 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 51d52325bcb201a6f27d7214c9a4d327 for removal 2024.10.22 01:22:01.121114 [ 1476 ] {} CacheMetadata: Key 0b56f165cb7155f2f0d6ed143e4f7bfb is removed from metadata 2024.10.22 01:22:01.127035 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3b5191fef1d3c47a60f394896e6c41d2, offset: 0, size: 97679 2024.10.22 01:22:01.129534 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3b5191fef1d3c47a60f394896e6c41d2 for removal 2024.10.22 01:22:01.134233 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: abb7b49c405e096d99562cb1c31482e3, offset: 0, size: 962 2024.10.22 01:22:01.139387 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key abb7b49c405e096d99562cb1c31482e3 for removal 2024.10.22 01:22:01.149084 [ 1476 ] {} CacheMetadata: Key 3b5191fef1d3c47a60f394896e6c41d2 is removed from metadata 2024.10.22 01:22:01.157322 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1f7481e5d60290612f8139d331e94724, offset: 0, size: 136 2024.10.22 01:22:01.161488 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1f7481e5d60290612f8139d331e94724 for removal 2024.10.22 01:22:01.167564 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: eaaab5839409bdd5ed917e53ed2ae08f, offset: 0, size: 41 2024.10.22 01:22:01.170597 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key eaaab5839409bdd5ed917e53ed2ae08f for removal 2024.10.22 01:22:01.176347 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 110f18899bf08f205927b0440dddd4aa, offset: 0, size: 257 2024.10.22 01:22:01.183684 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 110f18899bf08f205927b0440dddd4aa for removal 2024.10.22 01:22:01.188936 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 17ef5f9bc572ad40f7cf938cd5464c48, offset: 0, size: 1046 2024.10.22 01:22:01.192556 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 17ef5f9bc572ad40f7cf938cd5464c48 for removal 2024.10.22 01:22:01.200712 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3edac63d2cfb4c8933f7f87817b8a3d7, offset: 0, size: 37171 2024.10.22 01:22:01.212967 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3edac63d2cfb4c8933f7f87817b8a3d7 for removal 2024.10.22 01:22:01.218817 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 679fe56b51dfb2f59eaed924b43f9fd7, offset: 0, size: 313 2024.10.22 01:22:01.222922 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 679fe56b51dfb2f59eaed924b43f9fd7 for removal 2024.10.22 01:22:01.229179 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d29b8bbb00104c5e3c709b09d1066596, offset: 0, size: 522 2024.10.22 01:22:01.235075 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d29b8bbb00104c5e3c709b09d1066596 for removal 2024.10.22 01:22:01.242411 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e312273067fb804bdaee1930b1e09459, offset: 0, size: 808 2024.10.22 01:22:01.248468 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e312273067fb804bdaee1930b1e09459 for removal 2024.10.22 01:22:01.252141 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e1e05f5a1555e6629ed85b3d83586e95, offset: 0, size: 469 2024.10.22 01:22:01.255255 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e1e05f5a1555e6629ed85b3d83586e95 for removal 2024.10.22 01:22:01.258576 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5e3c37d3ecdacfd5a8e46071eee93d6b, offset: 0, size: 988 2024.10.22 01:22:01.263429 [ 1476 ] {} CacheMetadata: Key e1e05f5a1555e6629ed85b3d83586e95 is removed from metadata 2024.10.22 01:22:01.266281 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5e3c37d3ecdacfd5a8e46071eee93d6b for removal 2024.10.22 01:22:01.270853 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 427d55515d048261831ef5f547f84f1c, offset: 0, size: 771 2024.10.22 01:22:01.273569 [ 1476 ] {} CacheMetadata: Key 5e3c37d3ecdacfd5a8e46071eee93d6b is removed from metadata 2024.10.22 01:22:01.276121 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 427d55515d048261831ef5f547f84f1c for removal 2024.10.22 01:22:01.283463 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7814db155c256574b66e8df529c20c8a, offset: 0, size: 465 2024.10.22 01:22:01.292462 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7814db155c256574b66e8df529c20c8a for removal 2024.10.22 01:22:01.299097 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 55f411564d87929df943bcebba7daa51, offset: 0, size: 40929 2024.10.22 01:22:01.303459 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 55f411564d87929df943bcebba7daa51 for removal 2024.10.22 01:22:01.305224 [ 1476 ] {} CacheMetadata: Key 7814db155c256574b66e8df529c20c8a is removed from metadata 2024.10.22 01:22:01.310645 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ff312c99498639429db64b93a13a12e0, offset: 0, size: 1437 2024.10.22 01:22:01.318995 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ff312c99498639429db64b93a13a12e0 for removal 2024.10.22 01:22:01.320786 [ 1476 ] {} CacheMetadata: Key 55f411564d87929df943bcebba7daa51 is removed from metadata 2024.10.22 01:22:01.325448 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 49151f345662994afc3bb0be0dd6d261, offset: 0, size: 662 2024.10.22 01:22:01.329070 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 49151f345662994afc3bb0be0dd6d261 for removal 2024.10.22 01:22:01.331622 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 788460c8e2a7e0607585016c3123f301, offset: 0, size: 1073 2024.10.22 01:22:01.334393 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 788460c8e2a7e0607585016c3123f301 for removal 2024.10.22 01:22:01.335496 [ 1476 ] {} CacheMetadata: Key 49151f345662994afc3bb0be0dd6d261 is removed from metadata 2024.10.22 01:22:01.338771 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c2515162714eab78be3f98606a43883e, offset: 0, size: 766 2024.10.22 01:22:01.341276 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c2515162714eab78be3f98606a43883e for removal 2024.10.22 01:22:01.346779 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f893c77035da93f6c338e8ce73081b09, offset: 0, size: 7427 2024.10.22 01:22:01.347463 [ 1476 ] {} CacheMetadata: Key c2515162714eab78be3f98606a43883e is removed from metadata 2024.10.22 01:22:01.355073 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f893c77035da93f6c338e8ce73081b09 for removal 2024.10.22 01:22:01.361935 [ 1476 ] {} CacheMetadata: Key f893c77035da93f6c338e8ce73081b09 is removed from metadata 2024.10.22 01:22:01.365126 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7789bb6ee3c51d2b7bd7826d4a123b56, offset: 0, size: 8600 2024.10.22 01:22:01.368693 [ 1476 ] {} CacheMetadata: Key 788460c8e2a7e0607585016c3123f301 is removed from metadata 2024.10.22 01:22:01.375091 [ 1476 ] {} CacheMetadata: Key ff312c99498639429db64b93a13a12e0 is removed from metadata 2024.10.22 01:22:01.385209 [ 1476 ] {} CacheMetadata: Key 427d55515d048261831ef5f547f84f1c is removed from metadata 2024.10.22 01:22:01.399055 [ 1476 ] {} CacheMetadata: Key e312273067fb804bdaee1930b1e09459 is removed from metadata 2024.10.22 01:22:01.399983 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_690_308, 202410_691_691_0] 2024.10.22 01:22:01.406258 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_690_308, 202410_691_691_0] 2024.10.22 01:22:01.419279 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_690_308, 202410_691_691_0] 2024.10.22 01:22:01.468087 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7789bb6ee3c51d2b7bd7826d4a123b56 for removal 2024.10.22 01:22:01.471205 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1ae86c9a3a0cb3fa1ecd6d2344992e7e, offset: 0, size: 1086 2024.10.22 01:22:01.474124 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1ae86c9a3a0cb3fa1ecd6d2344992e7e for removal 2024.10.22 01:22:01.479149 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9c0273c279de3f91ee0862bfb70a385c, offset: 0, size: 86 2024.10.22 01:22:01.484486 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9c0273c279de3f91ee0862bfb70a385c for removal 2024.10.22 01:22:01.499167 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9178328ae6b30bce9d31f59e2b3ffdf1, offset: 0, size: 866 2024.10.22 01:22:01.506707 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9178328ae6b30bce9d31f59e2b3ffdf1 for removal 2024.10.22 01:22:01.513151 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3da7b91cfb35a6c48495808d00625004, offset: 0, size: 11122 2024.10.22 01:22:01.514965 [ 1476 ] {} CacheMetadata: Key 9178328ae6b30bce9d31f59e2b3ffdf1 is removed from metadata 2024.10.22 01:22:01.519221 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3da7b91cfb35a6c48495808d00625004 for removal 2024.10.22 01:22:01.522523 [ 1476 ] {} CacheMetadata: Key 3da7b91cfb35a6c48495808d00625004 is removed from metadata 2024.10.22 01:22:01.522882 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: eab7361f011deb475088b433a4812d2f, offset: 0, size: 872160 2024.10.22 01:22:01.527141 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key eab7361f011deb475088b433a4812d2f for removal 2024.10.22 01:22:01.539570 [ 1476 ] {} CacheMetadata: Key 9c0273c279de3f91ee0862bfb70a385c is removed from metadata 2024.10.22 01:22:01.562394 [ 1476 ] {} CacheMetadata: Key eab7361f011deb475088b433a4812d2f is removed from metadata 2024.10.22 01:22:01.569655 [ 1476 ] {} CacheMetadata: Key 1ae86c9a3a0cb3fa1ecd6d2344992e7e is removed from metadata 2024.10.22 01:22:01.578359 [ 1476 ] {} CacheMetadata: Key 7789bb6ee3c51d2b7bd7826d4a123b56 is removed from metadata 2024.10.22 01:22:01.606358 [ 1476 ] {} CacheMetadata: Key d29b8bbb00104c5e3c709b09d1066596 is removed from metadata 2024.10.22 01:22:01.629236 [ 1476 ] {} CacheMetadata: Key 679fe56b51dfb2f59eaed924b43f9fd7 is removed from metadata 2024.10.22 01:22:01.641824 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 391 2024.10.22 01:22:01.647351 [ 1476 ] {} CacheMetadata: Key 3edac63d2cfb4c8933f7f87817b8a3d7 is removed from metadata 2024.10.22 01:22:01.667887 [ 1476 ] {} CacheMetadata: Key 17ef5f9bc572ad40f7cf938cd5464c48 is removed from metadata 2024.10.22 01:22:01.682844 [ 1476 ] {} CacheMetadata: Key 110f18899bf08f205927b0440dddd4aa is removed from metadata 2024.10.22 01:22:01.810156 [ 1476 ] {} CacheMetadata: Key eaaab5839409bdd5ed917e53ed2ae08f is removed from metadata 2024.10.22 01:22:01.819621 [ 1476 ] {} CacheMetadata: Key 1f7481e5d60290612f8139d331e94724 is removed from metadata 2024.10.22 01:22:01.830777 [ 1476 ] {} CacheMetadata: Key abb7b49c405e096d99562cb1c31482e3 is removed from metadata 2024.10.22 01:22:01.843941 [ 1476 ] {} CacheMetadata: Key 51d52325bcb201a6f27d7214c9a4d327 is removed from metadata 2024.10.22 01:22:01.947151 [ 1238 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:22:01.977170 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:02.055637 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:22:02.118708 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_373_373_0 (state Temporary) 2024.10.22 01:22:02.247858 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_373_373_0 to 202410_383_383_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:02.274637 [ 1166 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_378_78 to 202410_383_383_0 2024.10.22 01:22:02.311802 [ 1166 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:02.316304 [ 1166 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:22:02.336558 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 391 2024.10.22 01:22:02.375426 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_378_78 to 202410_383_383_0 into Compact with storage Full 2024.10.22 01:22:02.569289 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_454_232/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:22:02.728181 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:02.768268 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 696511 2024.10.22 01:22:02.778411 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_378_78, total 422 rows starting from the beginning of the part 2024.10.22 01:22:02.847863 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_379_379_0, total 1 rows starting from the beginning of the part 2024.10.22 01:22:02.922620 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_380_380_0, total 1 rows starting from the beginning of the part 2024.10.22 01:22:03.025085 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vms/vmffqfwidshjvzdwthayirdpyreiw. 2024.10.22 01:22:03.029701 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_381_381_0, total 1 rows starting from the beginning of the part 2024.10.22 01:22:03.130845 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_382_382_0, total 1 rows starting from the beginning of the part 2024.10.22 01:22:03.195736 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 98c5efea24b02947ffb0737489be9d6e, offset: 0, size: 0 2024.10.22 01:22:03.197662 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 98c5efea24b02947ffb0737489be9d6e for removal 2024.10.22 01:22:03.204517 [ 1476 ] {} CacheMetadata: Key 98c5efea24b02947ffb0737489be9d6e is removed from metadata 2024.10.22 01:22:03.231191 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_383_383_0, total 1 rows starting from the beginning of the part 2024.10.22 01:22:03.282249 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e56424aa857fdb0c92ea9552cbb3409d, offset: 0, size: 0 2024.10.22 01:22:03.286291 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e56424aa857fdb0c92ea9552cbb3409d for removal 2024.10.22 01:22:03.293158 [ 1476 ] {} CacheMetadata: Key e56424aa857fdb0c92ea9552cbb3409d is removed from metadata 2024.10.22 01:22:03.306768 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vms/vmffqfwidshjvzdwthayirdpyreiw. 2024.10.22 01:22:03.465351 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.24 GiB, peak 1.96 GiB, free memory in arenas 11.99 MiB, will set to 1.23 GiB (RSS), difference: -16.74 MiB 2024.10.22 01:22:03.543087 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_454_232} MergeTask::MergeProjectionsStage: Merge sorted 50893 rows, containing 13 columns (13 merged, 0 gathered) in 3.376870767 sec., 15071.053502356312 rows/sec., 6.55 MiB/sec. 2024.10.22 01:22:03.646308 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_454_232} MergedBlockOutputStream: filled checksums 202410_1_454_232 (state Temporary) 2024.10.22 01:22:03.796302 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:03.805694 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:22:04.082439 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_443_443_0 (state Temporary) 2024.10.22 01:22:04.215241 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_443_443_0 to 202410_450_450_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:04.219486 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 250 entries to flush up to offset 50414 2024.10.22 01:22:04.220239 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3709 2024.10.22 01:22:04.274603 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 696511 2024.10.22 01:22:04.441724 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_790_398} MergeTask::MergeProjectionsStage: Merge sorted 50463 rows, containing 13 columns (13 merged, 0 gathered) in 4.221335524 sec., 11954.27364470259 rows/sec., 1.92 MiB/sec. 2024.10.22 01:22:04.503486 [ 1255 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:22:04.535190 [ 1255 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:22:04.571301 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_454_232} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_454_232 to 202410_1_454_232 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:04.576140 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_454_232} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_453_231, 202410_454_454_0] -> 202410_1_454_232 2024.10.22 01:22:04.604450 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_790_398} MergedBlockOutputStream: filled checksums 202410_1_790_398 (state Temporary) 2024.10.22 01:22:04.657549 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: da927899838a9d36d13e21a9e101cd4b, offset: 0, size: 0 2024.10.22 01:22:04.659144 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key da927899838a9d36d13e21a9e101cd4b for removal 2024.10.22 01:22:04.661489 [ 1476 ] {} CacheMetadata: Key da927899838a9d36d13e21a9e101cd4b is removed from metadata 2024.10.22 01:22:04.698905 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.15 MiB. 2024.10.22 01:22:04.712209 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b916d2bc80c2eafe1c20d1a0059e1e38, offset: 0, size: 0 2024.10.22 01:22:04.814859 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b916d2bc80c2eafe1c20d1a0059e1e38 for removal 2024.10.22 01:22:04.819644 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/dvb/ksxhfxhjhxiragxkioipzobdfvjvn. 2024.10.22 01:22:04.821425 [ 1476 ] {} CacheMetadata: Key b916d2bc80c2eafe1c20d1a0059e1e38 is removed from metadata 2024.10.22 01:22:04.826376 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_790_398} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_790_398 to 202410_1_790_398 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:04.829203 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_790_398} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_789_397, 202410_790_790_0] -> 202410_1_790_398 2024.10.22 01:22:05.051268 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:05.055515 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:22:05.062295 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:22:05.071683 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_330_68, 202410_331_331_0, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0] 2024.10.22 01:22:05.074299 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_330_68, 202410_331_331_0, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0] 2024.10.22 01:22:05.112540 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_330_68, 202410_331_331_0, 202410_332_332_0, 202410_333_333_0, 202410_334_334_0, 202410_335_335_0] 2024.10.22 01:22:05.132604 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_783_783_0 (state Temporary) 2024.10.22 01:22:05.290693 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_783_783_0 to 202410_796_796_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:05.328889 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_790_398 to 202410_791_791_0 2024.10.22 01:22:05.332125 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.81 MiB using storage policy from min volume index 0 2024.10.22 01:22:05.333474 [ 1250 ] {} DiskLocal: Reserved 1.81 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:22:05.335505 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 50414 2024.10.22 01:22:05.352523 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_791_399} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_790_398 to 202410_791_791_0 into Compact with storage Full 2024.10.22 01:22:05.392297 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_791_399} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:05.424467 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_791_399} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_790_398, total 50463 rows starting from the beginning of the part 2024.10.22 01:22:05.469763 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_791_399} MergeTreeSequentialSource: Reading 2 marks from part 202410_791_791_0, total 97 rows starting from the beginning of the part 2024.10.22 01:22:05.528204 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} MergeTask::MergeProjectionsStage: Merge sorted 427 rows, containing 32 columns (32 merged, 0 gathered) in 3.172613614 sec., 134.58934870472382 rows/sec., 12.60 KiB/sec. 2024.10.22 01:22:05.580946 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} MergedBlockOutputStream: filled checksums 202410_1_383_79 (state Temporary) 2024.10.22 01:22:05.751973 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.08 GiB, peak 1.96 GiB, free memory in arenas 26.14 MiB, will set to 1.19 GiB (RSS), difference: 114.50 MiB 2024.10.22 01:22:05.854280 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/azg/hkskhgwkpmevwrmpgqujcndzipffi. 2024.10.22 01:22:05.902658 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_383_79 to 202410_1_383_79 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:05.916972 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_383_79} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_378_78, 202410_383_383_0] -> 202410_1_383_79 2024.10.22 01:22:05.970553 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:22:06.037414 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 907a15295684a5ef4242e6a2fcfa9ac6, offset: 0, size: 0 2024.10.22 01:22:06.041436 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 907a15295684a5ef4242e6a2fcfa9ac6 for removal 2024.10.22 01:22:06.054641 [ 1476 ] {} CacheMetadata: Key 907a15295684a5ef4242e6a2fcfa9ac6 is removed from metadata 2024.10.22 01:22:06.202470 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8d5c74f3a2e53007b24b0cfecd950c39, offset: 0, size: 0 2024.10.22 01:22:06.204160 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8d5c74f3a2e53007b24b0cfecd950c39 for removal 2024.10.22 01:22:06.207822 [ 1476 ] {} CacheMetadata: Key 8d5c74f3a2e53007b24b0cfecd950c39 is removed from metadata 2024.10.22 01:22:06.228893 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/azg/hkskhgwkpmevwrmpgqujcndzipffi. 2024.10.22 01:22:06.377070 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Age 2024.10.22 01:22:06.401333 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Age 2024.10.22 01:22:06.519290 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Age 2024.10.22 01:22:06.530922 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Age 2024.10.22 01:22:06.550720 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Age 2024.10.22 01:22:06.572048 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Age 2024.10.22 01:22:06.580202 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 7146 2024.10.22 01:22:06.660665 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Age 2024.10.22 01:22:06.669351 [ 1702 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qaa/wrpsnnrqwkmayizzhhvgstkkaxnkj, size 12051897 2024.10.22 01:22:06.685112 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Age 2024.10.22 01:22:06.715039 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Age 2024.10.22 01:22:06.859010 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/djf/yjhtsvwkjelmwclbxhsbjhbwrzglj (LogSeriesLimiter: on interval from 2024-10-22 01:21:44 to 2024-10-22 01:22:06 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:22:06.923403 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:06.929393 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:06.978685 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_462_462_0 (state Temporary) 2024.10.22 01:22:07.229300 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_462_462_0 to 202410_475_475_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:07.304477 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7146 2024.10.22 01:22:07.491212 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.11 GiB, peak 1.96 GiB, free memory in arenas 22.74 MiB, will set to 1.19 GiB (RSS), difference: 78.69 MiB 2024.10.22 01:22:07.684783 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 234 entries to flush up to offset 52168 2024.10.22 01:22:07.925082 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 010dd14a0dcb66d270cd1fd19b8085ee, offset: 0, size: 0 2024.10.22 01:22:07.926456 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 010dd14a0dcb66d270cd1fd19b8085ee for removal 2024.10.22 01:22:07.928664 [ 1476 ] {} CacheMetadata: Key 010dd14a0dcb66d270cd1fd19b8085ee is removed from metadata 2024.10.22 01:22:07.972505 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:07.975139 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:08.064673 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b8d4643403893663813ae51bf758cb93, offset: 0, size: 0 2024.10.22 01:22:08.066790 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b8d4643403893663813ae51bf758cb93 for removal 2024.10.22 01:22:08.071535 [ 1476 ] {} CacheMetadata: Key b8d4643403893663813ae51bf758cb93 is removed from metadata 2024.10.22 01:22:08.093239 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_691_309, 202410_692_692_0] 2024.10.22 01:22:08.099621 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_691_309, 202410_692_692_0] 2024.10.22 01:22:08.116618 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_691_309, 202410_692_692_0] 2024.10.22 01:22:08.192511 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_454_454_0 (state Temporary) 2024.10.22 01:22:08.702860 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_24_24_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:22:08.975206 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_454_454_0 to 202410_460_460_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:09.000540 [ 1187 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_455_455_0 to 202410_460_460_0 2024.10.22 01:22:09.011890 [ 1187 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:09.015394 [ 1187 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:09.069466 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_455_455_0 to 202410_460_460_0 into Compact with storage Full 2024.10.22 01:22:09.114658 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 52168 2024.10.22 01:22:09.119824 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:09.136279 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_455_455_0, total 227 rows starting from the beginning of the part 2024.10.22 01:22:09.155332 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 010dd14a0dcb66d270cd1fd19b8085ee:0) 2024.10.22 01:22:09.156868 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: aa398d336148aad0340593613a964984, offset: 0, size: 466500 2024.10.22 01:22:09.159893 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_397_175, 202410_398_398_0] 2024.10.22 01:22:09.161239 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key aa398d336148aad0340593613a964984 for removal 2024.10.22 01:22:09.161272 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_397_175, 202410_398_398_0] 2024.10.22 01:22:09.163678 [ 1476 ] {} CacheMetadata: Key aa398d336148aad0340593613a964984 is removed from metadata 2024.10.22 01:22:09.164043 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 93d588ea9d5ad5dccc6ac8925bcd6121, offset: 0, size: 522 2024.10.22 01:22:09.167147 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 93d588ea9d5ad5dccc6ac8925bcd6121 for removal 2024.10.22 01:22:09.170322 [ 1476 ] {} CacheMetadata: Key 93d588ea9d5ad5dccc6ac8925bcd6121 is removed from metadata 2024.10.22 01:22:09.208282 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_397_175, 202410_398_398_0] 2024.10.22 01:22:09.248962 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_456_456_0, total 202 rows starting from the beginning of the part 2024.10.22 01:22:09.286656 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 52888 bytes for 4a822ef1349ef63a32c539da265d1882:0) 2024.10.22 01:22:09.291521 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1ccf6191ac7f5aa9a6c26d32a0a37ae9, offset: 0, size: 10648784 2024.10.22 01:22:09.371300 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 103 entries to flush up to offset 50517 2024.10.22 01:22:09.403466 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1ccf6191ac7f5aa9a6c26d32a0a37ae9 for removal 2024.10.22 01:22:09.413632 [ 1476 ] {} CacheMetadata: Key 1ccf6191ac7f5aa9a6c26d32a0a37ae9 is removed from metadata 2024.10.22 01:22:09.415774 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0b11d7a4644c6821981d4f302e336589, offset: 0, size: 1026 2024.10.22 01:22:09.424197 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0b11d7a4644c6821981d4f302e336589 for removal 2024.10.22 01:22:09.429065 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 3daea68137f87ac49d90e680cc354a43, offset: 0, size: 25592 2024.10.22 01:22:09.431224 [ 1476 ] {} CacheMetadata: Key 0b11d7a4644c6821981d4f302e336589 is removed from metadata 2024.10.22 01:22:09.432080 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 3daea68137f87ac49d90e680cc354a43 for removal 2024.10.22 01:22:09.436688 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.96 GiB, free memory in arenas 20.95 MiB, will set to 1.18 GiB (RSS), difference: 32.83 MiB 2024.10.22 01:22:09.454820 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_457_457_0, total 232 rows starting from the beginning of the part 2024.10.22 01:22:09.513511 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_411_82, 202410_412_412_0, 202410_413_413_0, 202410_414_414_0, 202410_415_415_0, 202410_416_416_0] 2024.10.22 01:22:09.539882 [ 1476 ] {} CacheMetadata: Key 3daea68137f87ac49d90e680cc354a43 is removed from metadata 2024.10.22 01:22:09.540479 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f3761a610dbea7b273dd909b66319b31, offset: 0, size: 648 2024.10.22 01:22:09.552930 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_411_82, 202410_412_412_0, 202410_413_413_0, 202410_414_414_0, 202410_415_415_0, 202410_416_416_0] 2024.10.22 01:22:09.554843 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f3761a610dbea7b273dd909b66319b31 for removal 2024.10.22 01:22:09.559578 [ 1476 ] {} CacheMetadata: Key f3761a610dbea7b273dd909b66319b31 is removed from metadata 2024.10.22 01:22:09.580438 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_791_399} MergeTask::MergeProjectionsStage: Merge sorted 50560 rows, containing 13 columns (13 merged, 0 gathered) in 4.237474186 sec., 11931.636106962706 rows/sec., 1.91 MiB/sec. 2024.10.22 01:22:09.583131 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yny/vuryrykucvnkbtcgpifhusigwjlbq. 2024.10.22 01:22:09.584954 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_411_82, 202410_412_412_0, 202410_413_413_0, 202410_414_414_0, 202410_415_415_0, 202410_416_416_0] 2024.10.22 01:22:09.632343 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8ef9bfdedae7b690c581547bd4af761a, offset: 0, size: 0 2024.10.22 01:22:09.635762 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8ef9bfdedae7b690c581547bd4af761a for removal 2024.10.22 01:22:09.650316 [ 1476 ] {} CacheMetadata: Key 8ef9bfdedae7b690c581547bd4af761a is removed from metadata 2024.10.22 01:22:09.652761 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_791_399} MergedBlockOutputStream: filled checksums 202410_1_791_399 (state Temporary) 2024.10.22 01:22:09.664900 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:09.666745 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:09.692790 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 176052c0463d29cefa31b0a09c1429cb, offset: 0, size: 0 2024.10.22 01:22:09.701719 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 176052c0463d29cefa31b0a09c1429cb for removal 2024.10.22 01:22:09.711982 [ 1476 ] {} CacheMetadata: Key 176052c0463d29cefa31b0a09c1429cb is removed from metadata 2024.10.22 01:22:09.727243 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yny/vuryrykucvnkbtcgpifhusigwjlbq. 2024.10.22 01:22:09.738451 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_784_784_0 (state Temporary) 2024.10.22 01:22:09.792274 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_791_399} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_791_399 to 202410_1_791_399 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:09.799632 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_791_399} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_790_398, 202410_791_791_0] -> 202410_1_791_399 2024.10.22 01:22:09.799688 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_784_784_0 to 202410_797_797_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:09.884376 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_458_458_0, total 233 rows starting from the beginning of the part 2024.10.22 01:22:09.922765 [ 1192 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_791_399 to 202410_792_792_0 2024.10.22 01:22:09.928456 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:22:09.930194 [ 1192 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.82 MiB using storage policy from min volume index 0 2024.10.22 01:22:09.938769 [ 1192 ] {} DiskLocal: Reserved 1.82 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:10.078453 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_792_400} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_791_399 to 202410_792_792_0 into Compact with storage Full 2024.10.22 01:22:10.092328 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_792_400} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:10.112808 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_459_459_0, total 251 rows starting from the beginning of the part 2024.10.22 01:22:10.113598 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_792_400} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_791_399, total 50560 rows starting from the beginning of the part 2024.10.22 01:22:10.177702 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 50517 2024.10.22 01:22:10.189576 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:22:10.193808 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:22:10.215908 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_460_460_0, total 234 rows starting from the beginning of the part 2024.10.22 01:22:10.133625 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_792_400} MergeTreeSequentialSource: Reading 2 marks from part 202410_792_792_0, total 64 rows starting from the beginning of the part 2024.10.22 01:22:10.626761 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_449_90/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:22:10.679065 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 02c47cbe32c9ba43344f38d2b77f64e6, offset: 0, size: 0 2024.10.22 01:22:10.683051 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 02c47cbe32c9ba43344f38d2b77f64e6 for removal 2024.10.22 01:22:10.684781 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_386_77, 202410_387_387_0, 202410_388_388_0, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0] 2024.10.22 01:22:10.688839 [ 1476 ] {} CacheMetadata: Key 02c47cbe32c9ba43344f38d2b77f64e6 is removed from metadata 2024.10.22 01:22:10.690534 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_386_77, 202410_387_387_0, 202410_388_388_0, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0] 2024.10.22 01:22:10.724558 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_386_77, 202410_387_387_0, 202410_388_388_0, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0] 2024.10.22 01:22:10.742131 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 56cace9a7f4063834ae3deed45aaed35, offset: 0, size: 0 2024.10.22 01:22:10.748398 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 56cace9a7f4063834ae3deed45aaed35 for removal 2024.10.22 01:22:10.757769 [ 1476 ] {} CacheMetadata: Key 56cace9a7f4063834ae3deed45aaed35 is removed from metadata 2024.10.22 01:22:11.279730 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 698051 2024.10.22 01:22:11.421093 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_94_19/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:22:11.480308 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.19 GiB, peak 1.96 GiB, free memory in arenas 16.74 MiB, will set to 1.18 GiB (RSS), difference: -8.12 MiB 2024.10.22 01:22:11.514558 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_792_400/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:22:11.706512 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qyv/uuqtmwujtktqoasudptzzlhbyidzu. 2024.10.22 01:22:11.805398 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_692_310, 202410_693_693_0] 2024.10.22 01:22:11.813718 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_692_310, 202410_693_693_0] 2024.10.22 01:22:11.830648 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_692_310, 202410_693_693_0] 2024.10.22 01:22:11.862413 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f0c4e70b1cbfe69855dedc87a6c5c8bf, offset: 0, size: 0 2024.10.22 01:22:11.863635 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f0c4e70b1cbfe69855dedc87a6c5c8bf for removal 2024.10.22 01:22:11.868892 [ 1476 ] {} CacheMetadata: Key f0c4e70b1cbfe69855dedc87a6c5c8bf is removed from metadata 2024.10.22 01:22:11.923526 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0018447e89d1ea3b4e1de321afe217e6, offset: 0, size: 0 2024.10.22 01:22:11.930757 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0018447e89d1ea3b4e1de321afe217e6 for removal 2024.10.22 01:22:11.934947 [ 1476 ] {} CacheMetadata: Key 0018447e89d1ea3b4e1de321afe217e6 is removed from metadata 2024.10.22 01:22:11.939600 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qyv/uuqtmwujtktqoasudptzzlhbyidzu. 2024.10.22 01:22:11.977406 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 3750 2024.10.22 01:22:12.004651 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:12.010512 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:12.559547 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_444_444_0 (state Temporary) 2024.10.22 01:22:12.743139 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} MergeTask::MergeProjectionsStage: Merge sorted 1379 rows, containing 13 columns (13 merged, 0 gathered) in 3.678286132 sec., 374.9028625052054 rows/sec., 170.40 KiB/sec. 2024.10.22 01:22:12.871997 [ 1227 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 01:22:12.888368 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} MergedBlockOutputStream: filled checksums 202410_455_460_1 (state Temporary) 2024.10.22 01:22:13.008337 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_444_444_0 to 202410_451_451_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:13.014359 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for f0c4e70b1cbfe69855dedc87a6c5c8bf:0) 2024.10.22 01:22:13.019052 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 876aef09706ca3e48e26ebf1dc7c9f81, offset: 0, size: 2369459 2024.10.22 01:22:13.024756 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 876aef09706ca3e48e26ebf1dc7c9f81 for removal 2024.10.22 01:22:13.031210 [ 1476 ] {} CacheMetadata: Key 876aef09706ca3e48e26ebf1dc7c9f81 is removed from metadata 2024.10.22 01:22:13.034500 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8aff2f585ce94e4a3045e86f7b81f756, offset: 0, size: 217 2024.10.22 01:22:13.044737 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8aff2f585ce94e4a3045e86f7b81f756 for removal 2024.10.22 01:22:13.067223 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 698051 2024.10.22 01:22:13.156623 [ 1476 ] {} CacheMetadata: Key 8aff2f585ce94e4a3045e86f7b81f756 is removed from metadata 2024.10.22 01:22:13.364588 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_455_460_1 to 202410_455_460_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:13.367444 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_455_460_1} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_455_455_0, 202410_460_460_0] -> 202410_455_460_1 2024.10.22 01:22:13.432155 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:22:13.635456 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.96 GiB, free memory in arenas 12.98 MiB, will set to 1.18 GiB (RSS), difference: 14.48 MiB 2024.10.22 01:22:13.664885 [ 1224 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 01:22:13.728537 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ylc/cwokgazznemmagmcvnfstpztdkyzx. 2024.10.22 01:22:13.783761 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9930dceb06dc4145541cd692df80172a, offset: 0, size: 0 2024.10.22 01:22:13.785442 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9930dceb06dc4145541cd692df80172a for removal 2024.10.22 01:22:13.884024 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_792_400} MergeTask::MergeProjectionsStage: Merge sorted 50624 rows, containing 13 columns (13 merged, 0 gathered) in 3.810904554 sec., 13283.985280309385 rows/sec., 2.13 MiB/sec. 2024.10.22 01:22:13.890621 [ 1476 ] {} CacheMetadata: Key 9930dceb06dc4145541cd692df80172a is removed from metadata 2024.10.22 01:22:14.078094 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: aa276f65de168973023402323d8d93fc, offset: 0, size: 0 2024.10.22 01:22:14.080270 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key aa276f65de168973023402323d8d93fc for removal 2024.10.22 01:22:14.082885 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_792_400} MergedBlockOutputStream: filled checksums 202410_1_792_400 (state Temporary) 2024.10.22 01:22:14.092840 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ylc/cwokgazznemmagmcvnfstpztdkyzx. 2024.10.22 01:22:14.124877 [ 1476 ] {} CacheMetadata: Key aa276f65de168973023402323d8d93fc is removed from metadata 2024.10.22 01:22:14.319001 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 103 entries to flush up to offset 50620 2024.10.22 01:22:14.436053 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_792_400} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_792_400 to 202410_1_792_400 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:14.444573 [ 1223 ] {} DatabaseCatalog: Execution took 471 ms. 2024.10.22 01:22:14.445494 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_792_400} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_791_399, 202410_792_792_0] -> 202410_1_792_400 2024.10.22 01:22:14.587811 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:22:14.707034 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 9930dceb06dc4145541cd692df80172a:0) 2024.10.22 01:22:14.711735 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0b8ed379d7c7c95912b944418ca13ab5, offset: 0, size: 2746789 2024.10.22 01:22:14.718691 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0b8ed379d7c7c95912b944418ca13ab5 for removal 2024.10.22 01:22:14.724116 [ 1476 ] {} CacheMetadata: Key 0b8ed379d7c7c95912b944418ca13ab5 is removed from metadata 2024.10.22 01:22:14.725641 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2d4f22691c0b2602ff41f7e9b6128139, offset: 0, size: 918 2024.10.22 01:22:14.729814 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2d4f22691c0b2602ff41f7e9b6128139 for removal 2024.10.22 01:22:14.767744 [ 1476 ] {} CacheMetadata: Key 2d4f22691c0b2602ff41f7e9b6128139 is removed from metadata 2024.10.22 01:22:14.807402 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 7163 2024.10.22 01:22:14.905318 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:14.911688 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:15.055150 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:15.060470 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:15.021689 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_785_785_0 (state Temporary) 2024.10.22 01:22:15.181002 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_785_785_0 to 202410_798_798_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:15.217809 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_463_463_0 (state Temporary) 2024.10.22 01:22:15.265294 [ 1177 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_792_400 to 202410_793_793_0 2024.10.22 01:22:15.273313 [ 1177 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.82 MiB using storage policy from min volume index 0 2024.10.22 01:22:15.284397 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 50620 2024.10.22 01:22:15.288176 [ 1177 ] {} DiskLocal: Reserved 1.82 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:15.288538 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_463_463_0 to 202410_476_476_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:15.340526 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_793_401} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_792_400 to 202410_793_793_0 into Compact with storage Full 2024.10.22 01:22:15.365452 [ 1159 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_471_94 to 202410_476_476_0 2024.10.22 01:22:15.383550 [ 1159 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:15.388141 [ 1159 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:15.439094 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_471_94 to 202410_476_476_0 into Compact with storage Full 2024.10.22 01:22:15.481712 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:15.486232 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_471_94, total 7177 rows starting from the beginning of the part 2024.10.22 01:22:15.510417 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_793_401} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:15.531534 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_472_472_0, total 26 rows starting from the beginning of the part 2024.10.22 01:22:15.540028 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/wwb/reaojufrkktpdvfcuxvrurvbmpkfw. 2024.10.22 01:22:15.548508 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_793_401} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_792_400, total 50624 rows starting from the beginning of the part 2024.10.22 01:22:15.562840 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_473_473_0, total 15 rows starting from the beginning of the part 2024.10.22 01:22:15.579209 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_793_401} MergeTreeSequentialSource: Reading 2 marks from part 202410_793_793_0, total 97 rows starting from the beginning of the part 2024.10.22 01:22:15.583211 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7163 2024.10.22 01:22:15.587385 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5018533a4c234aa98556e244acf71b11, offset: 0, size: 0 2024.10.22 01:22:15.591593 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5018533a4c234aa98556e244acf71b11 for removal 2024.10.22 01:22:15.593584 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_474_474_0, total 11 rows starting from the beginning of the part 2024.10.22 01:22:15.698730 [ 1476 ] {} CacheMetadata: Key 5018533a4c234aa98556e244acf71b11 is removed from metadata 2024.10.22 01:22:15.741086 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_475_475_0, total 23 rows starting from the beginning of the part 2024.10.22 01:22:15.756989 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 839954c081f725bd89f0851fa186aa11, offset: 0, size: 0 2024.10.22 01:22:15.758655 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 839954c081f725bd89f0851fa186aa11 for removal 2024.10.22 01:22:15.760702 [ 1476 ] {} CacheMetadata: Key 839954c081f725bd89f0851fa186aa11 is removed from metadata 2024.10.22 01:22:15.764672 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_476_476_0, total 17 rows starting from the beginning of the part 2024.10.22 01:22:15.768431 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/wwb/reaojufrkktpdvfcuxvrurvbmpkfw. 2024.10.22 01:22:15.880442 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.17 GiB, peak 1.96 GiB, free memory in arenas 14.87 MiB, will set to 1.17 GiB (RSS), difference: -8.67 MiB 2024.10.22 01:22:16.225467 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 5018533a4c234aa98556e244acf71b11:0) 2024.10.22 01:22:16.229611 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3557cb4832468202d70aa2fa09d1c92a, offset: 0, size: 1144452 2024.10.22 01:22:16.233808 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3557cb4832468202d70aa2fa09d1c92a for removal 2024.10.22 01:22:16.239619 [ 1476 ] {} CacheMetadata: Key 3557cb4832468202d70aa2fa09d1c92a is removed from metadata 2024.10.22 01:22:16.242856 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 31476bf0b6912edff1387c37ee65c647, offset: 0, size: 2637 2024.10.22 01:22:16.248343 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 31476bf0b6912edff1387c37ee65c647 for removal 2024.10.22 01:22:16.252352 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 46ff713f210fff79d3f8c5ab2e210529, offset: 0, size: 166 2024.10.22 01:22:16.256250 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 46ff713f210fff79d3f8c5ab2e210529 for removal 2024.10.22 01:22:16.259069 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c69105ca11ca1693ed4918cd085e88e6, offset: 0, size: 164 2024.10.22 01:22:16.261574 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c69105ca11ca1693ed4918cd085e88e6 for removal 2024.10.22 01:22:16.265411 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fee4cd154098a354279154be2894eebd, offset: 0, size: 52360 2024.10.22 01:22:16.268693 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fee4cd154098a354279154be2894eebd for removal 2024.10.22 01:22:16.273043 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 88f67177e774abeff3bb10193d51eff4, offset: 0, size: 159 2024.10.22 01:22:16.276324 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 88f67177e774abeff3bb10193d51eff4 for removal 2024.10.22 01:22:16.354840 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_398_176, 202410_399_399_0] 2024.10.22 01:22:16.358236 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_398_176, 202410_399_399_0] 2024.10.22 01:22:16.383069 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_398_176, 202410_399_399_0] 2024.10.22 01:22:16.475762 [ 1476 ] {} CacheMetadata: Key 88f67177e774abeff3bb10193d51eff4 is removed from metadata 2024.10.22 01:22:16.485488 [ 1476 ] {} CacheMetadata: Key fee4cd154098a354279154be2894eebd is removed from metadata 2024.10.22 01:22:16.507050 [ 1476 ] {} CacheMetadata: Key c69105ca11ca1693ed4918cd085e88e6 is removed from metadata 2024.10.22 01:22:16.521738 [ 1476 ] {} CacheMetadata: Key 46ff713f210fff79d3f8c5ab2e210529 is removed from metadata 2024.10.22 01:22:16.633689 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_693_311, 202410_694_694_0] 2024.10.22 01:22:16.636789 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_693_311, 202410_694_694_0] 2024.10.22 01:22:16.659074 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_693_311, 202410_694_694_0] 2024.10.22 01:22:16.670887 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 910007 bytes for 5018533a4c234aa98556e244acf71b11:0) 2024.10.22 01:22:16.673694 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 91d10ca0919c1e97aea31d4fe7e98383, offset: 0, size: 1717842 2024.10.22 01:22:16.679574 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 91d10ca0919c1e97aea31d4fe7e98383 for removal 2024.10.22 01:22:16.686306 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b3ece03bfdd4bad020e8cf6329176d33, offset: 0, size: 164 2024.10.22 01:22:16.691148 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b3ece03bfdd4bad020e8cf6329176d33 for removal 2024.10.22 01:22:16.699510 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 17c21bbd1cafa714360b0c178ca4f23d, offset: 0, size: 895 2024.10.22 01:22:16.704792 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 17c21bbd1cafa714360b0c178ca4f23d for removal 2024.10.22 01:22:16.756132 [ 1476 ] {} CacheMetadata: Key 17c21bbd1cafa714360b0c178ca4f23d is removed from metadata 2024.10.22 01:22:16.786103 [ 1476 ] {} CacheMetadata: Key b3ece03bfdd4bad020e8cf6329176d33 is removed from metadata 2024.10.22 01:22:16.807371 [ 1476 ] {} CacheMetadata: Key 91d10ca0919c1e97aea31d4fe7e98383 is removed from metadata 2024.10.22 01:22:16.826470 [ 1476 ] {} CacheMetadata: Key 31476bf0b6912edff1387c37ee65c647 is removed from metadata 2024.10.22 01:22:16.865398 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 164 entries to flush up to offset 52332 2024.10.22 01:22:17.066949 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:17.075418 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:17.111039 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} MergeTask::MergeProjectionsStage: Merge sorted 7269 rows, containing 12 columns (12 merged, 0 gathered) in 1.680691357 sec., 4325.005879113353 rows/sec., 686.82 KiB/sec. 2024.10.22 01:22:17.126319 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_455_455_0 (state Temporary) 2024.10.22 01:22:17.135734 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ney/acwkmcgvicbbuxcqtbjdgtyuzupcr. 2024.10.22 01:22:17.265792 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} MergedBlockOutputStream: filled checksums 202410_1_476_95 (state Temporary) 2024.10.22 01:22:17.346542 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 392 2024.10.22 01:22:17.400345 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 82cdb06f10c8d13f71718f827726f576, offset: 0, size: 0 2024.10.22 01:22:17.405818 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 82cdb06f10c8d13f71718f827726f576 for removal 2024.10.22 01:22:17.413016 [ 1476 ] {} CacheMetadata: Key 82cdb06f10c8d13f71718f827726f576 is removed from metadata 2024.10.22 01:22:17.445809 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_455_455_0 to 202410_461_461_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:17.472583 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fd82e66bf39a439ab2138f27b9eacdff, offset: 0, size: 0 2024.10.22 01:22:17.568743 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 52332 2024.10.22 01:22:17.572498 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_476_95 to 202410_1_476_95 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:17.575645 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fd82e66bf39a439ab2138f27b9eacdff for removal 2024.10.22 01:22:17.577978 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_476_95} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_471_94, 202410_476_476_0] -> 202410_1_476_95 2024.10.22 01:22:17.578560 [ 1476 ] {} CacheMetadata: Key fd82e66bf39a439ab2138f27b9eacdff is removed from metadata 2024.10.22 01:22:17.582561 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ney/acwkmcgvicbbuxcqtbjdgtyuzupcr. 2024.10.22 01:22:17.642555 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.45 MiB. 2024.10.22 01:22:17.729261 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.17 GiB, peak 1.96 GiB, free memory in arenas 16.14 MiB, will set to 1.17 GiB (RSS), difference: 5.97 MiB 2024.10.22 01:22:17.888094 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 5 file segments (while reserving 195166 bytes for 73e952f2b22209202092d45bccb64ad2:0) 2024.10.22 01:22:17.891955 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 262c91b1e98040bb638f941ffe79f7f8, offset: 0, size: 154 2024.10.22 01:22:17.898085 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 262c91b1e98040bb638f941ffe79f7f8 for removal 2024.10.22 01:22:17.905804 [ 1476 ] {} CacheMetadata: Key 262c91b1e98040bb638f941ffe79f7f8 is removed from metadata 2024.10.22 01:22:17.907356 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 317323c6203a0838975dc655f6775e98, offset: 0, size: 322889 2024.10.22 01:22:17.916017 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 317323c6203a0838975dc655f6775e98 for removal 2024.10.22 01:22:17.920700 [ 1476 ] {} CacheMetadata: Key 317323c6203a0838975dc655f6775e98 is removed from metadata 2024.10.22 01:22:17.922014 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a1d9aea701116c8cbd2da76f4fdf713c, offset: 0, size: 36927 2024.10.22 01:22:17.927183 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a1d9aea701116c8cbd2da76f4fdf713c for removal 2024.10.22 01:22:17.930422 [ 1476 ] {} CacheMetadata: Key a1d9aea701116c8cbd2da76f4fdf713c is removed from metadata 2024.10.22 01:22:17.930501 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7eec79984219e1e357c91aff35b4e155, offset: 0, size: 147 2024.10.22 01:22:17.935716 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7eec79984219e1e357c91aff35b4e155 for removal 2024.10.22 01:22:17.949672 [ 1476 ] {} CacheMetadata: Key 7eec79984219e1e357c91aff35b4e155 is removed from metadata 2024.10.22 01:22:17.954342 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e13f53d60a9ba43c259b815b68463b94, offset: 0, size: 51007 2024.10.22 01:22:17.963226 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e13f53d60a9ba43c259b815b68463b94 for removal 2024.10.22 01:22:17.969039 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:17.970753 [ 1476 ] {} CacheMetadata: Key e13f53d60a9ba43c259b815b68463b94 is removed from metadata 2024.10.22 01:22:17.978421 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:18.087989 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_374_374_0 (state Temporary) 2024.10.22 01:22:18.107228 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 908231 bytes for 82cdb06f10c8d13f71718f827726f576:0) 2024.10.22 01:22:18.110065 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a3ad4d65211b4d6841b79a1c472e88a8, offset: 0, size: 452562 2024.10.22 01:22:18.115413 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a3ad4d65211b4d6841b79a1c472e88a8 for removal 2024.10.22 01:22:18.122223 [ 1476 ] {} CacheMetadata: Key a3ad4d65211b4d6841b79a1c472e88a8 is removed from metadata 2024.10.22 01:22:18.122668 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2d472ad8ed20e3233467cc8ff0358527, offset: 0, size: 152 2024.10.22 01:22:18.128292 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2d472ad8ed20e3233467cc8ff0358527 for removal 2024.10.22 01:22:18.131616 [ 1476 ] {} CacheMetadata: Key 2d472ad8ed20e3233467cc8ff0358527 is removed from metadata 2024.10.22 01:22:18.131896 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f49469e1450a79a57c0dc699f7d90d94, offset: 0, size: 187 2024.10.22 01:22:18.137411 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f49469e1450a79a57c0dc699f7d90d94 for removal 2024.10.22 01:22:18.141414 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 905376c1f3659c3f8de2deb1d3b441d0, offset: 0, size: 504 2024.10.22 01:22:18.148077 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 905376c1f3659c3f8de2deb1d3b441d0 for removal 2024.10.22 01:22:18.155707 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c52deeacdc8bb94fde61555a0425dc9c, offset: 0, size: 209734 2024.10.22 01:22:18.173206 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c52deeacdc8bb94fde61555a0425dc9c for removal 2024.10.22 01:22:18.180903 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 82aa81cfbd44dc88b4179d009f9ed94e, offset: 0, size: 30642 2024.10.22 01:22:18.184484 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 82aa81cfbd44dc88b4179d009f9ed94e for removal 2024.10.22 01:22:18.190077 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_374_374_0 to 202410_384_384_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:18.230352 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} MergeTask::MergeProjectionsStage: Merge sorted 3686 rows, containing 679 columns (679 merged, 0 gathered) in 102.7648563 sec., 35.86829323479529 rows/sec., 47.54 KiB/sec. 2024.10.22 01:22:18.141760 [ 1476 ] {} CacheMetadata: Key f49469e1450a79a57c0dc699f7d90d94 is removed from metadata 2024.10.22 01:22:18.257062 [ 1476 ] {} CacheMetadata: Key 82aa81cfbd44dc88b4179d009f9ed94e is removed from metadata 2024.10.22 01:22:18.264350 [ 1476 ] {} CacheMetadata: Key c52deeacdc8bb94fde61555a0425dc9c is removed from metadata 2024.10.22 01:22:18.270135 [ 1476 ] {} CacheMetadata: Key 905376c1f3659c3f8de2deb1d3b441d0 is removed from metadata 2024.10.22 01:22:18.371851 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 204219 bytes for fbb1057b687d172cf4fcdef3817fbabd:0) 2024.10.22 01:22:18.378763 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 11092f76ce2a13549fee4ceefa8d58c4, offset: 0, size: 679840 2024.10.22 01:22:18.387775 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 11092f76ce2a13549fee4ceefa8d58c4 for removal 2024.10.22 01:22:18.394203 [ 1476 ] {} CacheMetadata: Key 11092f76ce2a13549fee4ceefa8d58c4 is removed from metadata 2024.10.22 01:22:18.394997 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 51deb444e83e796e9e208bccd74ec52f, offset: 0, size: 526 2024.10.22 01:22:18.395929 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 392 2024.10.22 01:22:18.399451 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 51deb444e83e796e9e208bccd74ec52f for removal 2024.10.22 01:22:18.411670 [ 1476 ] {} CacheMetadata: Key 51deb444e83e796e9e208bccd74ec52f is removed from metadata 2024.10.22 01:22:18.510908 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ab2437966d41cb1530b840c519b8bf8, offset: 0, size: 159 2024.10.22 01:22:18.514817 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ab2437966d41cb1530b840c519b8bf8 for removal 2024.10.22 01:22:18.517163 [ 1476 ] {} CacheMetadata: Key 7ab2437966d41cb1530b840c519b8bf8 is removed from metadata 2024.10.22 01:22:18.720350 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_793_401} MergeTask::MergeProjectionsStage: Merge sorted 50721 rows, containing 13 columns (13 merged, 0 gathered) in 3.388375349 sec., 14969.120825108446 rows/sec., 2.40 MiB/sec. 2024.10.22 01:22:18.840273 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7fa814d82ae1bfa666d362211e9bde76, offset: 0, size: 0 2024.10.22 01:22:18.848565 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7fa814d82ae1bfa666d362211e9bde76 for removal 2024.10.22 01:22:18.856449 [ 1476 ] {} CacheMetadata: Key 7fa814d82ae1bfa666d362211e9bde76 is removed from metadata 2024.10.22 01:22:19.040046 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d8007ac33e1aa0276af262c5d2ed8cb1, offset: 0, size: 0 2024.10.22 01:22:19.050787 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d8007ac33e1aa0276af262c5d2ed8cb1 for removal 2024.10.22 01:22:19.053574 [ 1476 ] {} CacheMetadata: Key d8007ac33e1aa0276af262c5d2ed8cb1 is removed from metadata 2024.10.22 01:22:19.074387 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_793_401} MergedBlockOutputStream: filled checksums 202410_1_793_401 (state Temporary) 2024.10.22 01:22:19.249566 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_793_401} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_793_401 to 202410_1_793_401 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:19.260093 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_793_401} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_792_400, 202410_793_793_0] -> 202410_1_793_401 2024.10.22 01:22:19.280459 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:22:19.287904 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 5 file segments (while reserving 1048576 bytes for f845becd4778eb395f73094b8cbde0b4:0) 2024.10.22 01:22:19.407929 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} MergeTask::MergeProjectionsStage: Merge sorted 708561 rows, containing 4 columns (4 merged, 0 gathered) in 23.630918799 sec., 29984.487951013758 rows/sec., 449.89 KiB/sec. 2024.10.22 01:22:19.481246 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.96 GiB, free memory in arenas 14.54 MiB, will set to 1.17 GiB (RSS), difference: 6.99 MiB 2024.10.22 01:22:19.538364 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 168 entries to flush up to offset 50788 2024.10.22 01:22:19.540403 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8341176692b88560aa7af504ac528791, offset: 0, size: 81188 2024.10.22 01:22:19.541141 [ 1244 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:22:19.648124 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8341176692b88560aa7af504ac528791 for removal 2024.10.22 01:22:19.651911 [ 1476 ] {} CacheMetadata: Key 8341176692b88560aa7af504ac528791 is removed from metadata 2024.10.22 01:22:19.652497 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7d678d5bdc4dc14e3373470828f01b87, offset: 0, size: 809752 2024.10.22 01:22:19.655114 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7d678d5bdc4dc14e3373470828f01b87 for removal 2024.10.22 01:22:19.668033 [ 1476 ] {} CacheMetadata: Key 7d678d5bdc4dc14e3373470828f01b87 is removed from metadata 2024.10.22 01:22:19.669351 [ 1244 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:22:19.661490 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d0df4c0028d46dab8e4961ed67884322, offset: 0, size: 534 2024.10.22 01:22:19.774909 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d0df4c0028d46dab8e4961ed67884322 for removal 2024.10.22 01:22:19.778723 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a7398c19af51c392cc13049962654629, offset: 0, size: 160 2024.10.22 01:22:19.779163 [ 1476 ] {} CacheMetadata: Key d0df4c0028d46dab8e4961ed67884322 is removed from metadata 2024.10.22 01:22:19.783700 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a7398c19af51c392cc13049962654629 for removal 2024.10.22 01:22:19.788528 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2c9c66b41e5252bc844b27b9da86812f, offset: 0, size: 5632 2024.10.22 01:22:19.799940 [ 1476 ] {} CacheMetadata: Key a7398c19af51c392cc13049962654629 is removed from metadata 2024.10.22 01:22:19.801803 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2c9c66b41e5252bc844b27b9da86812f for removal 2024.10.22 01:22:19.812850 [ 1476 ] {} CacheMetadata: Key 2c9c66b41e5252bc844b27b9da86812f is removed from metadata 2024.10.22 01:22:19.855051 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} MergedBlockOutputStream: filled checksums 202410_1_449_90 (state Temporary) 2024.10.22 01:22:19.884318 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} MergedBlockOutputStream: filled checksums 202410_1_94_19 (state Temporary) 2024.10.22 01:22:20.077084 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 699591 2024.10.22 01:22:20.124345 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 22 file segments (while reserving 1048576 bytes for 7fa814d82ae1bfa666d362211e9bde76:0) 2024.10.22 01:22:20.128818 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 67f5b55745248ac9f15071006d2dc524, offset: 0, size: 3417 2024.10.22 01:22:20.133663 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 67f5b55745248ac9f15071006d2dc524 for removal 2024.10.22 01:22:20.137203 [ 1476 ] {} CacheMetadata: Key 67f5b55745248ac9f15071006d2dc524 is removed from metadata 2024.10.22 01:22:20.142661 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2e203ac6b7f901893cd3b719abcb5d23, offset: 0, size: 2487 2024.10.22 01:22:20.149262 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2e203ac6b7f901893cd3b719abcb5d23 for removal 2024.10.22 01:22:20.155743 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6649f4edec2d8379dc911bd23cbcf342, offset: 0, size: 139337 2024.10.22 01:22:20.158855 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6649f4edec2d8379dc911bd23cbcf342 for removal 2024.10.22 01:22:20.162789 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b0dc3bd58d3ad90214c15cca9470f64a, offset: 0, size: 34200 2024.10.22 01:22:20.162962 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.07148442 sec. 2024.10.22 01:22:20.164932 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b0dc3bd58d3ad90214c15cca9470f64a for removal 2024.10.22 01:22:20.166961 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a842e424defd33878b7b8c0699f5cd74, offset: 0, size: 39092 2024.10.22 01:22:20.170649 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a842e424defd33878b7b8c0699f5cd74 for removal 2024.10.22 01:22:20.175611 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5e8edda6674e014b43393ddcb41e5563, offset: 0, size: 159 2024.10.22 01:22:20.178627 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5e8edda6674e014b43393ddcb41e5563 for removal 2024.10.22 01:22:20.184567 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ab6fd8ff3f47a39d371f28399dd2d2eb, offset: 0, size: 63382 2024.10.22 01:22:20.191697 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:22:20.195107 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:22:20.194783 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ab6fd8ff3f47a39d371f28399dd2d2eb for removal 2024.10.22 01:22:20.206992 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5bc975d482b10e42d105c66c3a41adfe, offset: 0, size: 159 2024.10.22 01:22:20.216214 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5bc975d482b10e42d105c66c3a41adfe for removal 2024.10.22 01:22:20.252386 [ 1476 ] {} CacheMetadata: Key ab6fd8ff3f47a39d371f28399dd2d2eb is removed from metadata 2024.10.22 01:22:20.261309 [ 1476 ] {} CacheMetadata: Key 5e8edda6674e014b43393ddcb41e5563 is removed from metadata 2024.10.22 01:22:20.269280 [ 1476 ] {} CacheMetadata: Key 5bc975d482b10e42d105c66c3a41adfe is removed from metadata 2024.10.22 01:22:20.276134 [ 1476 ] {} CacheMetadata: Key b0dc3bd58d3ad90214c15cca9470f64a is removed from metadata 2024.10.22 01:22:20.283579 [ 1476 ] {} CacheMetadata: Key 6649f4edec2d8379dc911bd23cbcf342 is removed from metadata 2024.10.22 01:22:20.291599 [ 1476 ] {} CacheMetadata: Key a842e424defd33878b7b8c0699f5cd74 is removed from metadata 2024.10.22 01:22:20.307349 [ 1476 ] {} CacheMetadata: Key 2e203ac6b7f901893cd3b719abcb5d23 is removed from metadata 2024.10.22 01:22:20.221803 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 221c490ef44fe7075380c442cec0a5ec, offset: 0, size: 166 2024.10.22 01:22:20.326975 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 221c490ef44fe7075380c442cec0a5ec for removal 2024.10.22 01:22:20.329272 [ 1476 ] {} CacheMetadata: Key 221c490ef44fe7075380c442cec0a5ec is removed from metadata 2024.10.22 01:22:20.331226 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e3da527e2f925b47905ef2c2b740159e, offset: 0, size: 67450 2024.10.22 01:22:20.350361 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e3da527e2f925b47905ef2c2b740159e for removal 2024.10.22 01:22:20.355173 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 093b5d9ab76bccdab679cef4792f5c56, offset: 0, size: 159 2024.10.22 01:22:20.356575 [ 1476 ] {} CacheMetadata: Key e3da527e2f925b47905ef2c2b740159e is removed from metadata 2024.10.22 01:22:20.361932 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 093b5d9ab76bccdab679cef4792f5c56 for removal 2024.10.22 01:22:20.364753 [ 1476 ] {} CacheMetadata: Key 093b5d9ab76bccdab679cef4792f5c56 is removed from metadata 2024.10.22 01:22:20.365556 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: da24d9050861dd3c4feec26b8b2c131c, offset: 0, size: 166 2024.10.22 01:22:20.373192 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key da24d9050861dd3c4feec26b8b2c131c for removal 2024.10.22 01:22:20.377534 [ 1476 ] {} CacheMetadata: Key da24d9050861dd3c4feec26b8b2c131c is removed from metadata 2024.10.22 01:22:20.378563 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4cb7ffffe722bff09d4423dc3198f441, offset: 0, size: 159 2024.10.22 01:22:20.385105 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4cb7ffffe722bff09d4423dc3198f441 for removal 2024.10.22 01:22:20.389664 [ 1476 ] {} CacheMetadata: Key 4cb7ffffe722bff09d4423dc3198f441 is removed from metadata 2024.10.22 01:22:20.389664 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dcc156c3657bb9f72d92b3254abe949b, offset: 0, size: 32814 2024.10.22 01:22:20.412202 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dcc156c3657bb9f72d92b3254abe949b for removal 2024.10.22 01:22:20.421225 [ 1476 ] {} CacheMetadata: Key dcc156c3657bb9f72d92b3254abe949b is removed from metadata 2024.10.22 01:22:20.423317 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 61ba4dfabf1bc0004e0a02e347555bb5, offset: 0, size: 142164 2024.10.22 01:22:20.431994 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 61ba4dfabf1bc0004e0a02e347555bb5 for removal 2024.10.22 01:22:20.435054 [ 1476 ] {} CacheMetadata: Key 61ba4dfabf1bc0004e0a02e347555bb5 is removed from metadata 2024.10.22 01:22:20.439083 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1b039fc2fd22f40197a825e8717055af, offset: 0, size: 159 2024.10.22 01:22:20.446345 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1b039fc2fd22f40197a825e8717055af for removal 2024.10.22 01:22:20.451912 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:20.460346 [ 1476 ] {} CacheMetadata: Key 1b039fc2fd22f40197a825e8717055af is removed from metadata 2024.10.22 01:22:20.556602 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 358ba65976830f0d3372fc37fd243042, offset: 0, size: 159 2024.10.22 01:22:20.557357 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:20.561129 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 358ba65976830f0d3372fc37fd243042 for removal 2024.10.22 01:22:20.563789 [ 1476 ] {} CacheMetadata: Key 358ba65976830f0d3372fc37fd243042 is removed from metadata 2024.10.22 01:22:20.564302 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 772e56a9bf0464b966e0ea38529e5a39, offset: 0, size: 145449 2024.10.22 01:22:20.569541 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 772e56a9bf0464b966e0ea38529e5a39 for removal 2024.10.22 01:22:20.575065 [ 1476 ] {} CacheMetadata: Key 772e56a9bf0464b966e0ea38529e5a39 is removed from metadata 2024.10.22 01:22:20.586190 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3fc22bad72a103a114b6e5c3062d9bed, offset: 0, size: 166 2024.10.22 01:22:20.589387 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3fc22bad72a103a114b6e5c3062d9bed for removal 2024.10.22 01:22:20.592927 [ 1476 ] {} CacheMetadata: Key 3fc22bad72a103a114b6e5c3062d9bed is removed from metadata 2024.10.22 01:22:20.593817 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fe82f7b48136e2bc8036c9d82a495595, offset: 0, size: 159 2024.10.22 01:22:20.605600 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fe82f7b48136e2bc8036c9d82a495595 for removal 2024.10.22 01:22:20.611986 [ 1476 ] {} CacheMetadata: Key fe82f7b48136e2bc8036c9d82a495595 is removed from metadata 2024.10.22 01:22:20.612064 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3ad545033b66debb9db267ef67ad0b5f, offset: 0, size: 91487 2024.10.22 01:22:20.649127 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_786_786_0 (state Temporary) 2024.10.22 01:22:20.718882 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3ad545033b66debb9db267ef67ad0b5f for removal 2024.10.22 01:22:20.723909 [ 1476 ] {} CacheMetadata: Key 3ad545033b66debb9db267ef67ad0b5f is removed from metadata 2024.10.22 01:22:20.725350 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c8b614032841d709fb2c2d4dda5b0dca, offset: 0, size: 160 2024.10.22 01:22:20.732516 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c8b614032841d709fb2c2d4dda5b0dca for removal 2024.10.22 01:22:20.740106 [ 1476 ] {} CacheMetadata: Key c8b614032841d709fb2c2d4dda5b0dca is removed from metadata 2024.10.22 01:22:20.860943 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_786_786_0 to 202410_799_799_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:20.879793 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 6 file segments (while reserving 436755 bytes for f845becd4778eb395f73094b8cbde0b4:0) 2024.10.22 01:22:20.884377 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 32771fb5bccee395ce3fddaea62faf38, offset: 0, size: 85264 2024.10.22 01:22:20.902067 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 32771fb5bccee395ce3fddaea62faf38 for removal 2024.10.22 01:22:20.908563 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9a70d66e03ce234b0b27a1330b0eeb13, offset: 0, size: 165 2024.10.22 01:22:20.908781 [ 1476 ] {} CacheMetadata: Key 32771fb5bccee395ce3fddaea62faf38 is removed from metadata 2024.10.22 01:22:20.919223 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9a70d66e03ce234b0b27a1330b0eeb13 for removal 2024.10.22 01:22:20.919986 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_793_401 to 202410_794_794_0 2024.10.22 01:22:20.929081 [ 1476 ] {} CacheMetadata: Key 9a70d66e03ce234b0b27a1330b0eeb13 is removed from metadata 2024.10.22 01:22:20.940845 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.82 MiB using storage policy from min volume index 0 2024.10.22 01:22:20.944687 [ 1249 ] {} DiskLocal: Reserved 1.82 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:21.030841 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8ccd1f0389ef9e43b33f6a99991b0bc7, offset: 0, size: 147129 2024.10.22 01:22:21.035092 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8ccd1f0389ef9e43b33f6a99991b0bc7 for removal 2024.10.22 01:22:21.041410 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e8ffe2eb68e6286999fc91a9d2bd3eb1, offset: 0, size: 240995 2024.10.22 01:22:21.042239 [ 1476 ] {} CacheMetadata: Key 8ccd1f0389ef9e43b33f6a99991b0bc7 is removed from metadata 2024.10.22 01:22:21.047927 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e8ffe2eb68e6286999fc91a9d2bd3eb1 for removal 2024.10.22 01:22:21.054600 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bddf1a5725655ec078aa8a5ce0e936e7, offset: 0, size: 160 2024.10.22 01:22:21.056399 [ 1476 ] {} CacheMetadata: Key e8ffe2eb68e6286999fc91a9d2bd3eb1 is removed from metadata 2024.10.22 01:22:21.061229 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bddf1a5725655ec078aa8a5ce0e936e7 for removal 2024.10.22 01:22:21.066744 [ 1476 ] {} CacheMetadata: Key bddf1a5725655ec078aa8a5ce0e936e7 is removed from metadata 2024.10.22 01:22:21.066959 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b1f7a7f137d2064834678815f904de90, offset: 0, size: 152 2024.10.22 01:22:21.074237 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b1f7a7f137d2064834678815f904de90 for removal 2024.10.22 01:22:21.079679 [ 1476 ] {} CacheMetadata: Key b1f7a7f137d2064834678815f904de90 is removed from metadata 2024.10.22 01:22:21.095715 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_794_402} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_793_401 to 202410_794_794_0 into Compact with storage Full 2024.10.22 01:22:21.144291 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 50788 2024.10.22 01:22:21.195363 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:21.200839 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:21.288093 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_794_402} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:21.298832 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_794_402} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_793_401, total 50721 rows starting from the beginning of the part 2024.10.22 01:22:21.305494 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_445_445_0 (state Temporary) 2024.10.22 01:22:21.417909 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_445_445_0 to 202410_452_452_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:21.458367 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 559093 bytes for 7fa814d82ae1bfa666d362211e9bde76:0) 2024.10.22 01:22:21.461087 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 17bc03913bdfe997e86579287443c676, offset: 0, size: 1198636 2024.10.22 01:22:21.465634 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 17bc03913bdfe997e86579287443c676 for removal 2024.10.22 01:22:21.465810 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_794_402} MergeTreeSequentialSource: Reading 2 marks from part 202410_794_794_0, total 128 rows starting from the beginning of the part 2024.10.22 01:22:21.468907 [ 1476 ] {} CacheMetadata: Key 17bc03913bdfe997e86579287443c676 is removed from metadata 2024.10.22 01:22:21.471930 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e708bd13b611145aedfe50721738ef61, offset: 0, size: 673 2024.10.22 01:22:21.476702 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e708bd13b611145aedfe50721738ef61 for removal 2024.10.22 01:22:21.480629 [ 1476 ] {} CacheMetadata: Key e708bd13b611145aedfe50721738ef61 is removed from metadata 2024.10.22 01:22:21.481561 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0b17d24987fe0c2876beeccf9a61bd80, offset: 0, size: 160 2024.10.22 01:22:21.488146 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0b17d24987fe0c2876beeccf9a61bd80 for removal 2024.10.22 01:22:21.565789 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 699591 2024.10.22 01:22:21.596138 [ 1476 ] {} CacheMetadata: Key 0b17d24987fe0c2876beeccf9a61bd80 is removed from metadata 2024.10.22 01:22:21.643553 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/eel/notjelerexwituiribwosnnivgqjw. 2024.10.22 01:22:21.667480 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.96 GiB, free memory in arenas 13.83 MiB, will set to 1.16 GiB (RSS), difference: 9.37 MiB 2024.10.22 01:22:21.677549 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0300fccae38fe675829ced06bb1210bd, offset: 0, size: 0 2024.10.22 01:22:21.678886 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0300fccae38fe675829ced06bb1210bd for removal 2024.10.22 01:22:21.682266 [ 1476 ] {} CacheMetadata: Key 0300fccae38fe675829ced06bb1210bd is removed from metadata 2024.10.22 01:22:21.744565 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c459df21e16892ff3a6fe091a6201d2d, offset: 0, size: 0 2024.10.22 01:22:21.753192 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c459df21e16892ff3a6fe091a6201d2d for removal 2024.10.22 01:22:21.756577 [ 1476 ] {} CacheMetadata: Key c459df21e16892ff3a6fe091a6201d2d is removed from metadata 2024.10.22 01:22:21.761008 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/eel/notjelerexwituiribwosnnivgqjw. 2024.10.22 01:22:21.906398 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_694_312, 202410_695_695_0] 2024.10.22 01:22:21.914148 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_694_312, 202410_695_695_0] 2024.10.22 01:22:21.935315 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_694_312, 202410_695_695_0] 2024.10.22 01:22:22.135675 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_794_402/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:22:22.241177 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_449_90 to 202410_1_449_90 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:22.248028 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_449_90} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_444_89, 202410_449_449_0] -> 202410_1_449_90 2024.10.22 01:22:22.311770 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 1048576 bytes for 0300fccae38fe675829ced06bb1210bd:0) 2024.10.22 01:22:22.314639 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ab28130a1d99fd7ee6d9e5ec1cb70c38, offset: 0, size: 111245 2024.10.22 01:22:22.324418 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ab28130a1d99fd7ee6d9e5ec1cb70c38 for removal 2024.10.22 01:22:22.330092 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fd818998b12198cdfe9329c9c7a39620, offset: 0, size: 679 2024.10.22 01:22:22.338707 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fd818998b12198cdfe9329c9c7a39620 for removal 2024.10.22 01:22:22.351264 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 21f5bef2d4024f410bb006c173e72d8e, offset: 0, size: 184476 2024.10.22 01:22:22.354429 [ 1476 ] {} CacheMetadata: Key ab28130a1d99fd7ee6d9e5ec1cb70c38 is removed from metadata 2024.10.22 01:22:22.357037 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 21f5bef2d4024f410bb006c173e72d8e for removal 2024.10.22 01:22:22.365243 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8176289a17bf1e3dba108c90b0219ca8, offset: 0, size: 167 2024.10.22 01:22:22.365453 [ 1476 ] {} CacheMetadata: Key 21f5bef2d4024f410bb006c173e72d8e is removed from metadata 2024.10.22 01:22:22.371460 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8176289a17bf1e3dba108c90b0219ca8 for removal 2024.10.22 01:22:22.374537 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:22:22.375183 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c6070f569e150449609f67fda60179bb, offset: 0, size: 166 2024.10.22 01:22:22.377186 [ 1476 ] {} CacheMetadata: Key 8176289a17bf1e3dba108c90b0219ca8 is removed from metadata 2024.10.22 01:22:22.379608 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c6070f569e150449609f67fda60179bb for removal 2024.10.22 01:22:22.383820 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2a3a5dd252c890753b1fd6c408984740, offset: 0, size: 165 2024.10.22 01:22:22.386479 [ 1476 ] {} CacheMetadata: Key c6070f569e150449609f67fda60179bb is removed from metadata 2024.10.22 01:22:22.388171 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2a3a5dd252c890753b1fd6c408984740 for removal 2024.10.22 01:22:22.391782 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3001a7cc615b57c67abe1c9e7e8140c5, offset: 0, size: 201879 2024.10.22 01:22:22.399083 [ 1476 ] {} CacheMetadata: Key 2a3a5dd252c890753b1fd6c408984740 is removed from metadata 2024.10.22 01:22:22.402819 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3001a7cc615b57c67abe1c9e7e8140c5 for removal 2024.10.22 01:22:22.411606 [ 1476 ] {} CacheMetadata: Key fd818998b12198cdfe9329c9c7a39620 is removed from metadata 2024.10.22 01:22:22.512820 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9c00ed3aabb8aefc0bffd9a6a129d828, offset: 0, size: 105816 2024.10.22 01:22:22.516468 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9c00ed3aabb8aefc0bffd9a6a129d828 for removal 2024.10.22 01:22:22.559938 [ 1476 ] {} CacheMetadata: Key 9c00ed3aabb8aefc0bffd9a6a129d828 is removed from metadata 2024.10.22 01:22:22.564537 [ 1476 ] {} CacheMetadata: Key 3001a7cc615b57c67abe1c9e7e8140c5 is removed from metadata 2024.10.22 01:22:22.752079 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 1048576 bytes for 0300fccae38fe675829ced06bb1210bd:0) 2024.10.22 01:22:22.755305 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 42a8a818f9e58af5d639a1565d52b6d8, offset: 0, size: 1173061 2024.10.22 01:22:22.761118 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 42a8a818f9e58af5d639a1565d52b6d8 for removal 2024.10.22 01:22:22.763258 [ 1476 ] {} CacheMetadata: Key 42a8a818f9e58af5d639a1565d52b6d8 is removed from metadata 2024.10.22 01:22:23.077330 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 102764 bytes for 184ae13a45820a0b5d5802581572bcda:0) 2024.10.22 01:22:23.080100 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f96817875ad4db19da6b333cbde9f64b, offset: 0, size: 1171991 2024.10.22 01:22:23.084480 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f96817875ad4db19da6b333cbde9f64b for removal 2024.10.22 01:22:23.084739 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 7186 2024.10.22 01:22:23.088965 [ 1476 ] {} CacheMetadata: Key f96817875ad4db19da6b333cbde9f64b is removed from metadata 2024.10.22 01:22:23.091432 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 467352101b92065a45cebf4cc3bc3631, offset: 0, size: 166 2024.10.22 01:22:23.096616 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 467352101b92065a45cebf4cc3bc3631 for removal 2024.10.22 01:22:23.139910 [ 1476 ] {} CacheMetadata: Key 467352101b92065a45cebf4cc3bc3631 is removed from metadata 2024.10.22 01:22:23.177245 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_94_19/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:22:23.392476 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 330464 bytes for 51d45afb53009471dff40b491b8ccea9:0) 2024.10.22 01:22:23.399079 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d4ec29e3bc6fa874c24c81651e5a3d39, offset: 0, size: 598170 2024.10.22 01:22:23.403896 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d4ec29e3bc6fa874c24c81651e5a3d39 for removal 2024.10.22 01:22:23.411629 [ 1476 ] {} CacheMetadata: Key d4ec29e3bc6fa874c24c81651e5a3d39 is removed from metadata 2024.10.22 01:22:23.414287 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a659b505469b4bd69e1d2289789ba084, offset: 0, size: 521 2024.10.22 01:22:23.417300 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a659b505469b4bd69e1d2289789ba084 for removal 2024.10.22 01:22:23.427114 [ 1476 ] {} CacheMetadata: Key a659b505469b4bd69e1d2289789ba084 is removed from metadata 2024.10.22 01:22:23.575232 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:23.580151 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:23.726668 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_464_464_0 (state Temporary) 2024.10.22 01:22:23.740150 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f7995ca9d9be970eae390027c4dbcb8e, offset: 0, size: 0 2024.10.22 01:22:23.746885 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f7995ca9d9be970eae390027c4dbcb8e for removal 2024.10.22 01:22:23.751236 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 10 file segments (while reserving 1048576 bytes for f1614cda94a0cd410670097da0a3ae55:0) 2024.10.22 01:22:23.752159 [ 1476 ] {} CacheMetadata: Key f7995ca9d9be970eae390027c4dbcb8e is removed from metadata 2024.10.22 01:22:23.753640 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 658ec3ad041b77c547707b67da823cc7, offset: 0, size: 526 2024.10.22 01:22:23.757313 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 658ec3ad041b77c547707b67da823cc7 for removal 2024.10.22 01:22:23.760404 [ 1476 ] {} CacheMetadata: Key 658ec3ad041b77c547707b67da823cc7 is removed from metadata 2024.10.22 01:22:23.760729 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: eb1970e7e72c72e95e444a4bb1e7a3a5, offset: 0, size: 102996 2024.10.22 01:22:23.767384 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key eb1970e7e72c72e95e444a4bb1e7a3a5 for removal 2024.10.22 01:22:23.769618 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fd9bc8ad76ef81fad8e647cd2421d269, offset: 0, size: 149 2024.10.22 01:22:23.772039 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fd9bc8ad76ef81fad8e647cd2421d269 for removal 2024.10.22 01:22:23.774681 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a9948198f2dccff86987c15bbd7a73fb, offset: 0, size: 164397 2024.10.22 01:22:23.776974 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a9948198f2dccff86987c15bbd7a73fb for removal 2024.10.22 01:22:23.779654 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b297d94557f145d92f0bcbdf391abfd4, offset: 0, size: 159 2024.10.22 01:22:23.785639 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b297d94557f145d92f0bcbdf391abfd4 for removal 2024.10.22 01:22:23.789783 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.96 GiB, free memory in arenas 10.21 MiB, will set to 1.15 GiB (RSS), difference: -9.31 MiB 2024.10.22 01:22:23.791556 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8d2a43fee02ca1b204056c772b9f9766, offset: 0, size: 551240 2024.10.22 01:22:23.799791 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8d2a43fee02ca1b204056c772b9f9766 for removal 2024.10.22 01:22:23.807231 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ce29f32670d7d63b4277393242c39506, offset: 0, size: 150 2024.10.22 01:22:23.811276 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ce29f32670d7d63b4277393242c39506 for removal 2024.10.22 01:22:23.814296 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 09818b0334f49902c820b132c600bc4c, offset: 0, size: 61449 2024.10.22 01:22:23.823457 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 09818b0334f49902c820b132c600bc4c for removal 2024.10.22 01:22:23.831144 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 12736e452560abc5aba4f14406832e7c, offset: 0, size: 53694 2024.10.22 01:22:23.834391 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 12736e452560abc5aba4f14406832e7c for removal 2024.10.22 01:22:23.839546 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: bd0aace1038fb34c83868df92fb9818a, offset: 0, size: 159 2024.10.22 01:22:23.874680 [ 1476 ] {} CacheMetadata: Key 12736e452560abc5aba4f14406832e7c is removed from metadata 2024.10.22 01:22:23.875404 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_464_464_0 to 202410_477_477_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:23.878914 [ 1476 ] {} CacheMetadata: Key 09818b0334f49902c820b132c600bc4c is removed from metadata 2024.10.22 01:22:23.887739 [ 1476 ] {} CacheMetadata: Key ce29f32670d7d63b4277393242c39506 is removed from metadata 2024.10.22 01:22:23.898818 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8eb29eadc26178cf52acf2d19f68cbc3, offset: 0, size: 0 2024.10.22 01:22:23.903901 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8eb29eadc26178cf52acf2d19f68cbc3 for removal 2024.10.22 01:22:23.906891 [ 1476 ] {} CacheMetadata: Key 8d2a43fee02ca1b204056c772b9f9766 is removed from metadata 2024.10.22 01:22:23.920998 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7186 2024.10.22 01:22:23.927894 [ 1476 ] {} CacheMetadata: Key 8eb29eadc26178cf52acf2d19f68cbc3 is removed from metadata 2024.10.22 01:22:23.934497 [ 1476 ] {} CacheMetadata: Key a9948198f2dccff86987c15bbd7a73fb is removed from metadata 2024.10.22 01:22:23.946798 [ 1476 ] {} CacheMetadata: Key fd9bc8ad76ef81fad8e647cd2421d269 is removed from metadata 2024.10.22 01:22:23.947578 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key bd0aace1038fb34c83868df92fb9818a for removal 2024.10.22 01:22:23.954275 [ 1476 ] {} CacheMetadata: Key bd0aace1038fb34c83868df92fb9818a is removed from metadata 2024.10.22 01:22:23.962954 [ 1476 ] {} CacheMetadata: Key b297d94557f145d92f0bcbdf391abfd4 is removed from metadata 2024.10.22 01:22:23.968851 [ 1476 ] {} CacheMetadata: Key eb1970e7e72c72e95e444a4bb1e7a3a5 is removed from metadata 2024.10.22 01:22:24.047090 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 417023 bytes for f1614cda94a0cd410670097da0a3ae55:0) 2024.10.22 01:22:24.051536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fa4aae25ca2d41247b0717ffb653fc96, offset: 0, size: 356072 2024.10.22 01:22:24.055858 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fa4aae25ca2d41247b0717ffb653fc96 for removal 2024.10.22 01:22:24.058136 [ 1476 ] {} CacheMetadata: Key fa4aae25ca2d41247b0717ffb653fc96 is removed from metadata 2024.10.22 01:22:24.058690 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 41e0c41f7b957e907b7bacc4dc65c852, offset: 0, size: 519 2024.10.22 01:22:24.061269 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 41e0c41f7b957e907b7bacc4dc65c852 for removal 2024.10.22 01:22:24.064641 [ 1476 ] {} CacheMetadata: Key 41e0c41f7b957e907b7bacc4dc65c852 is removed from metadata 2024.10.22 01:22:24.065333 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 759dfc773aa164fb8e95e642d254e343, offset: 0, size: 100660 2024.10.22 01:22:24.070591 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 759dfc773aa164fb8e95e642d254e343 for removal 2024.10.22 01:22:24.076619 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: deafcaa67a3b042c3d837fca86c3c5eb, offset: 0, size: 160 2024.10.22 01:22:24.076708 [ 1476 ] {} CacheMetadata: Key 759dfc773aa164fb8e95e642d254e343 is removed from metadata 2024.10.22 01:22:24.079402 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key deafcaa67a3b042c3d837fca86c3c5eb for removal 2024.10.22 01:22:24.179131 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_794_402} MergeTask::MergeProjectionsStage: Merge sorted 50849 rows, containing 13 columns (13 merged, 0 gathered) in 3.104602226 sec., 16378.587754062894 rows/sec., 2.62 MiB/sec. 2024.10.22 01:22:24.189716 [ 1476 ] {} CacheMetadata: Key deafcaa67a3b042c3d837fca86c3c5eb is removed from metadata 2024.10.22 01:22:24.233149 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_794_402} MergedBlockOutputStream: filled checksums 202410_1_794_402 (state Temporary) 2024.10.22 01:22:24.347567 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for f7995ca9d9be970eae390027c4dbcb8e:0) 2024.10.22 01:22:24.349620 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 15e6662f31d843291b970ed6ddd5f92d, offset: 0, size: 1019953 2024.10.22 01:22:24.353315 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 15e6662f31d843291b970ed6ddd5f92d for removal 2024.10.22 01:22:24.355978 [ 1476 ] {} CacheMetadata: Key 15e6662f31d843291b970ed6ddd5f92d is removed from metadata 2024.10.22 01:22:24.356508 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 41d42a1d50517c2ed4d0004075875c15, offset: 0, size: 164 2024.10.22 01:22:24.359232 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 41d42a1d50517c2ed4d0004075875c15 for removal 2024.10.22 01:22:24.360957 [ 1476 ] {} CacheMetadata: Key 41d42a1d50517c2ed4d0004075875c15 is removed from metadata 2024.10.22 01:22:24.361731 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8ab89060ead1003c4ce54d264aab64b1, offset: 0, size: 343586 2024.10.22 01:22:24.364231 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8ab89060ead1003c4ce54d264aab64b1 for removal 2024.10.22 01:22:24.369342 [ 1476 ] {} CacheMetadata: Key 8ab89060ead1003c4ce54d264aab64b1 is removed from metadata 2024.10.22 01:22:24.370477 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f0fc23e23be711ca12ac668e27458312, offset: 0, size: 184 2024.10.22 01:22:24.374953 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f0fc23e23be711ca12ac668e27458312 for removal 2024.10.22 01:22:24.381059 [ 1476 ] {} CacheMetadata: Key f0fc23e23be711ca12ac668e27458312 is removed from metadata 2024.10.22 01:22:24.464240 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_399_177, 202410_400_400_0] 2024.10.22 01:22:24.465452 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_399_177, 202410_400_400_0] 2024.10.22 01:22:24.490938 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_399_177, 202410_400_400_0] 2024.10.22 01:22:24.613806 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_794_402} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_794_402 to 202410_1_794_402 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:24.626361 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_794_402} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_793_401, 202410_794_794_0] -> 202410_1_794_402 2024.10.22 01:22:24.840175 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for f7995ca9d9be970eae390027c4dbcb8e:0) 2024.10.22 01:22:24.847318 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 926630dbcb2f6258061185bc7c72ee76, offset: 0, size: 1142368 2024.10.22 01:22:24.860982 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.82 MiB. 2024.10.22 01:22:24.955405 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 926630dbcb2f6258061185bc7c72ee76 for removal 2024.10.22 01:22:24.960068 [ 1476 ] {} CacheMetadata: Key 926630dbcb2f6258061185bc7c72ee76 is removed from metadata 2024.10.22 01:22:24.961338 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c73e7e185e692ba251db69764f812e59, offset: 0, size: 3985 2024.10.22 01:22:25.000766 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_695_313, 202410_696_696_0] 2024.10.22 01:22:25.064120 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_695_313, 202410_696_696_0] 2024.10.22 01:22:25.064683 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c73e7e185e692ba251db69764f812e59 for removal 2024.10.22 01:22:25.066837 [ 1476 ] {} CacheMetadata: Key c73e7e185e692ba251db69764f812e59 is removed from metadata 2024.10.22 01:22:25.067041 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 345ce5c09877fd5faebfcb4501961525, offset: 0, size: 159 2024.10.22 01:22:25.070818 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 345ce5c09877fd5faebfcb4501961525 for removal 2024.10.22 01:22:25.073025 [ 1476 ] {} CacheMetadata: Key 345ce5c09877fd5faebfcb4501961525 is removed from metadata 2024.10.22 01:22:25.074254 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d87309099fa33d3258a63b8905aa407a, offset: 0, size: 106069 2024.10.22 01:22:25.077608 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d87309099fa33d3258a63b8905aa407a for removal 2024.10.22 01:22:25.080031 [ 1476 ] {} CacheMetadata: Key d87309099fa33d3258a63b8905aa407a is removed from metadata 2024.10.22 01:22:25.080569 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7667f3bb76206830410787c6b6c10906, offset: 0, size: 166 2024.10.22 01:22:25.085522 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7667f3bb76206830410787c6b6c10906 for removal 2024.10.22 01:22:25.088167 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9a760d2a6a82cbe83ec64798b6b232f7, offset: 0, size: 160 2024.10.22 01:22:25.088984 [ 1476 ] {} CacheMetadata: Key 7667f3bb76206830410787c6b6c10906 is removed from metadata 2024.10.22 01:22:25.103407 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9a760d2a6a82cbe83ec64798b6b232f7 for removal 2024.10.22 01:22:25.106497 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 96 entries to flush up to offset 52428 2024.10.22 01:22:25.108420 [ 1476 ] {} CacheMetadata: Key 9a760d2a6a82cbe83ec64798b6b232f7 is removed from metadata 2024.10.22 01:22:25.164201 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 300 entries to flush up to offset 51088 2024.10.22 01:22:25.173298 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_695_313, 202410_696_696_0] 2024.10.22 01:22:25.335040 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 5888e49dbd6c93ad411a6074c1e1c534:0) 2024.10.22 01:22:25.336894 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: fdcf8b4491799d98da6fb0108d8ff2f8, offset: 0, size: 1202203 2024.10.22 01:22:25.348767 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key fdcf8b4491799d98da6fb0108d8ff2f8 for removal 2024.10.22 01:22:25.355417 [ 1476 ] {} CacheMetadata: Key fdcf8b4491799d98da6fb0108d8ff2f8 is removed from metadata 2024.10.22 01:22:25.355266 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f9a7b62c0b4a4215496f88044a88294a, offset: 0, size: 681 2024.10.22 01:22:25.360775 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f9a7b62c0b4a4215496f88044a88294a for removal 2024.10.22 01:22:25.364702 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1fd7b2969399772d0b612110968937d6, offset: 0, size: 91768 2024.10.22 01:22:25.368306 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1fd7b2969399772d0b612110968937d6 for removal 2024.10.22 01:22:25.370250 [ 1476 ] {} CacheMetadata: Key f9a7b62c0b4a4215496f88044a88294a is removed from metadata 2024.10.22 01:22:25.372300 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 932edf6070f0aeecbc06e62360f015be, offset: 0, size: 160 2024.10.22 01:22:25.377196 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 932edf6070f0aeecbc06e62360f015be for removal 2024.10.22 01:22:25.378444 [ 1476 ] {} CacheMetadata: Key 1fd7b2969399772d0b612110968937d6 is removed from metadata 2024.10.22 01:22:25.384316 [ 1476 ] {} CacheMetadata: Key 932edf6070f0aeecbc06e62360f015be is removed from metadata 2024.10.22 01:22:25.455030 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:25.457791 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:25.489240 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_456_456_0 (state Temporary) 2024.10.22 01:22:25.568198 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.14 GiB, peak 1.96 GiB, free memory in arenas 8.74 MiB, will set to 1.15 GiB (RSS), difference: 5.66 MiB 2024.10.22 01:22:25.576730 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kth/qjfooosmbkbrnvplepsdhajcdtbga. 2024.10.22 01:22:25.717059 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1d8ad350f378f57809f83834c19072c0, offset: 0, size: 0 2024.10.22 01:22:25.719969 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1d8ad350f378f57809f83834c19072c0 for removal 2024.10.22 01:22:25.729221 [ 1476 ] {} CacheMetadata: Key 1d8ad350f378f57809f83834c19072c0 is removed from metadata 2024.10.22 01:22:25.855939 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_456_456_0 to 202410_462_462_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:25.883720 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: accf6466d755a33413a5344a0147b118, offset: 0, size: 0 2024.10.22 01:22:25.889176 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key accf6466d755a33413a5344a0147b118 for removal 2024.10.22 01:22:25.896387 [ 1476 ] {} CacheMetadata: Key accf6466d755a33413a5344a0147b118 is removed from metadata 2024.10.22 01:22:25.899476 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 393 2024.10.22 01:22:25.903573 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kth/qjfooosmbkbrnvplepsdhajcdtbga. 2024.10.22 01:22:25.905583 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 52428 2024.10.22 01:22:26.064862 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 83140 bytes for 03f493a6ab6acd1ec24e88bc702db212:0) 2024.10.22 01:22:26.068478 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0b8595bbad43243c7e86ec875ea3afd1, offset: 0, size: 77642 2024.10.22 01:22:26.073182 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0b8595bbad43243c7e86ec875ea3afd1 for removal 2024.10.22 01:22:26.075027 [ 1476 ] {} CacheMetadata: Key 0b8595bbad43243c7e86ec875ea3afd1 is removed from metadata 2024.10.22 01:22:26.075610 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 798c852e84a46e5ebdaa144c0187de8d, offset: 0, size: 160 2024.10.22 01:22:26.087500 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 798c852e84a46e5ebdaa144c0187de8d for removal 2024.10.22 01:22:26.090522 [ 1476 ] {} CacheMetadata: Key 798c852e84a46e5ebdaa144c0187de8d is removed from metadata 2024.10.22 01:22:26.269359 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:26.271288 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:26.272517 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 12 file segments (while reserving 906181 bytes for 164d3e407138c1a648892c2e6fbae707:0) 2024.10.22 01:22:26.274192 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b1c3e938ff4ba63e8ebaebd8b799d79f, offset: 0, size: 511 2024.10.22 01:22:26.276558 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b1c3e938ff4ba63e8ebaebd8b799d79f for removal 2024.10.22 01:22:26.280691 [ 1476 ] {} CacheMetadata: Key b1c3e938ff4ba63e8ebaebd8b799d79f is removed from metadata 2024.10.22 01:22:26.283545 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: fed276ab02bd097b50a76410d84b404a, offset: 0, size: 50400 2024.10.22 01:22:26.286167 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key fed276ab02bd097b50a76410d84b404a for removal 2024.10.22 01:22:26.288114 [ 1476 ] {} CacheMetadata: Key fed276ab02bd097b50a76410d84b404a is removed from metadata 2024.10.22 01:22:26.288960 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a71ecd419acadf0de9c13ce5de13d83a, offset: 0, size: 18211 2024.10.22 01:22:26.297267 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a71ecd419acadf0de9c13ce5de13d83a for removal 2024.10.22 01:22:26.301619 [ 1476 ] {} CacheMetadata: Key a71ecd419acadf0de9c13ce5de13d83a is removed from metadata 2024.10.22 01:22:26.302540 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ce0cfb670adb80de44cd3f66162e320c, offset: 0, size: 159 2024.10.22 01:22:26.307486 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ce0cfb670adb80de44cd3f66162e320c for removal 2024.10.22 01:22:26.310254 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2243325e8dd3020bfa5c08fea739a2bd, offset: 0, size: 154 2024.10.22 01:22:26.312927 [ 1476 ] {} CacheMetadata: Key ce0cfb670adb80de44cd3f66162e320c is removed from metadata 2024.10.22 01:22:26.317485 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 2243325e8dd3020bfa5c08fea739a2bd for removal 2024.10.22 01:22:26.321007 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_375_375_0 (state Temporary) 2024.10.22 01:22:26.321817 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 42e7cc1fb4cf4b487d497130fcd3b50c, offset: 0, size: 77116 2024.10.22 01:22:26.326678 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 42e7cc1fb4cf4b487d497130fcd3b50c for removal 2024.10.22 01:22:26.330886 [ 1476 ] {} CacheMetadata: Key 42e7cc1fb4cf4b487d497130fcd3b50c is removed from metadata 2024.10.22 01:22:26.331335 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d758da5abaf60f7fc50a3c4f43e17e5b, offset: 0, size: 304766 2024.10.22 01:22:26.343194 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d758da5abaf60f7fc50a3c4f43e17e5b for removal 2024.10.22 01:22:26.346951 [ 1476 ] {} CacheMetadata: Key 2243325e8dd3020bfa5c08fea739a2bd is removed from metadata 2024.10.22 01:22:26.348408 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3769392d923fda742fe77df604983563, offset: 0, size: 318580 2024.10.22 01:22:26.352878 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3769392d923fda742fe77df604983563 for removal 2024.10.22 01:22:26.359894 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e459b42764fd15b6abfc749a9122763d, offset: 0, size: 119524 2024.10.22 01:22:26.360356 [ 1476 ] {} CacheMetadata: Key 3769392d923fda742fe77df604983563 is removed from metadata 2024.10.22 01:22:26.362737 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e459b42764fd15b6abfc749a9122763d for removal 2024.10.22 01:22:26.365524 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e9bda8b97a5825338aa5f6d70b246bff, offset: 0, size: 165 2024.10.22 01:22:26.366302 [ 1476 ] {} CacheMetadata: Key e459b42764fd15b6abfc749a9122763d is removed from metadata 2024.10.22 01:22:26.370851 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e9bda8b97a5825338aa5f6d70b246bff for removal 2024.10.22 01:22:26.374560 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0dec4745b5134483ca27990b1804e254, offset: 0, size: 154 2024.10.22 01:22:26.375430 [ 1476 ] {} CacheMetadata: Key e9bda8b97a5825338aa5f6d70b246bff is removed from metadata 2024.10.22 01:22:26.375841 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_375_375_0 to 202410_385_385_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:26.377542 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0dec4745b5134483ca27990b1804e254 for removal 2024.10.22 01:22:26.380367 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 434d8f1c5be3b6d8ae305f90c9b47138, offset: 0, size: 160 2024.10.22 01:22:26.397109 [ 1476 ] {} CacheMetadata: Key 0dec4745b5134483ca27990b1804e254 is removed from metadata 2024.10.22 01:22:26.408226 [ 1476 ] {} CacheMetadata: Key d758da5abaf60f7fc50a3c4f43e17e5b is removed from metadata 2024.10.22 01:22:26.455014 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 393 2024.10.22 01:22:26.462409 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:26.464886 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:26.484615 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 434d8f1c5be3b6d8ae305f90c9b47138 for removal 2024.10.22 01:22:26.487023 [ 1476 ] {} CacheMetadata: Key 434d8f1c5be3b6d8ae305f90c9b47138 is removed from metadata 2024.10.22 01:22:26.536159 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 12752 bytes for 1d8ad350f378f57809f83834c19072c0:0) 2024.10.22 01:22:26.548831 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_787_787_0 (state Temporary) 2024.10.22 01:22:26.642633 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6fb93527b5edd518211fc0fb2b2782a9, offset: 0, size: 5256 2024.10.22 01:22:26.648666 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6fb93527b5edd518211fc0fb2b2782a9 for removal 2024.10.22 01:22:26.652497 [ 1476 ] {} CacheMetadata: Key 6fb93527b5edd518211fc0fb2b2782a9 is removed from metadata 2024.10.22 01:22:26.652578 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5ace3296f8691abc23688ddd005e54da, offset: 0, size: 147 2024.10.22 01:22:26.656860 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5ace3296f8691abc23688ddd005e54da for removal 2024.10.22 01:22:26.659209 [ 1476 ] {} CacheMetadata: Key 5ace3296f8691abc23688ddd005e54da is removed from metadata 2024.10.22 01:22:26.785873 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_insert_201403_24_24_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:22:26.823233 [ 1691 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/djf/yjhtsvwkjelmwclbxhsbjhbwrzglj, size 906181 2024.10.22 01:22:26.823764 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_787_787_0 to 202410_800_800_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:26.857128 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_794_402 to 202410_795_795_0 2024.10.22 01:22:26.865566 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.83 MiB using storage policy from min volume index 0 2024.10.22 01:22:26.870012 [ 1163 ] {} DiskLocal: Reserved 1.83 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:26.882827 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_795_403} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_794_402 to 202410_795_795_0 into Compact with storage Full 2024.10.22 01:22:26.926279 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_795_403} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:26.928254 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 49e4bfbd6e452e7f07949579ee65a41a, offset: 0, size: 0 2024.10.22 01:22:26.934593 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_795_403} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_794_402, total 50849 rows starting from the beginning of the part 2024.10.22 01:22:26.934957 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 49e4bfbd6e452e7f07949579ee65a41a for removal 2024.10.22 01:22:26.939042 [ 1476 ] {} CacheMetadata: Key 49e4bfbd6e452e7f07949579ee65a41a is removed from metadata 2024.10.22 01:22:26.971410 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 51088 2024.10.22 01:22:26.971495 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_795_403} MergeTreeSequentialSource: Reading 2 marks from part 202410_795_795_0, total 146 rows starting from the beginning of the part 2024.10.22 01:22:26.982779 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Sex 2024.10.22 01:22:26.986687 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Sex 2024.10.22 01:22:26.992236 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Sex 2024.10.22 01:22:27.009117 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 81784 bytes for 133f79a46d749d9e7972cc05437f9395:0) 2024.10.22 01:22:27.015316 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0fd04cd3a74aba6988c87429f2aa8b15, offset: 0, size: 166 2024.10.22 01:22:27.021249 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0fd04cd3a74aba6988c87429f2aa8b15 for removal 2024.10.22 01:22:27.025271 [ 1476 ] {} CacheMetadata: Key 0fd04cd3a74aba6988c87429f2aa8b15 is removed from metadata 2024.10.22 01:22:27.027457 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: e22dd758594f709ce984828b0c341c67, offset: 0, size: 186175 2024.10.22 01:22:27.036003 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key e22dd758594f709ce984828b0c341c67 for removal 2024.10.22 01:22:27.042893 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9de6878b1162a6d90da458b15ad1a4f0, offset: 0, size: 509 2024.10.22 01:22:27.047240 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9de6878b1162a6d90da458b15ad1a4f0 for removal 2024.10.22 01:22:27.128895 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Sex 2024.10.22 01:22:27.136956 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Sex 2024.10.22 01:22:27.172511 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Sex 2024.10.22 01:22:27.179732 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: acc197455bae921bdcc7e688ac2f6b23, offset: 0, size: 0 2024.10.22 01:22:27.181921 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key acc197455bae921bdcc7e688ac2f6b23 for removal 2024.10.22 01:22:27.189317 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pfx/unmtfchujitfpuwncbaalfbgtqzwv. 2024.10.22 01:22:27.189608 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Sex 2024.10.22 01:22:27.206180 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Sex 2024.10.22 01:22:27.227348 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Sex 2024.10.22 01:22:27.244044 [ 1476 ] {} CacheMetadata: Key 9de6878b1162a6d90da458b15ad1a4f0 is removed from metadata 2024.10.22 01:22:27.266174 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/loo/mqhulmrkodkpoqdwopopewriwlxpd (LogSeriesLimiter: on interval from 2024-10-22 01:22:06 to 2024-10-22 01:22:27 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:22:27.271430 [ 1476 ] {} CacheMetadata: Key acc197455bae921bdcc7e688ac2f6b23 is removed from metadata 2024.10.22 01:22:27.284760 [ 1476 ] {} CacheMetadata: Key e22dd758594f709ce984828b0c341c67 is removed from metadata 2024.10.22 01:22:27.329303 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8f077afe327ae7d10875454e7d164b68, offset: 0, size: 0 2024.10.22 01:22:27.331538 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8f077afe327ae7d10875454e7d164b68 for removal 2024.10.22 01:22:27.334031 [ 1476 ] {} CacheMetadata: Key 8f077afe327ae7d10875454e7d164b68 is removed from metadata 2024.10.22 01:22:27.403362 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: bebe322d11535e9bc37bbca35ffe3157, offset: 0, size: 0 2024.10.22 01:22:27.405315 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key bebe322d11535e9bc37bbca35ffe3157 for removal 2024.10.22 01:22:27.413026 [ 1476 ] {} CacheMetadata: Key bebe322d11535e9bc37bbca35ffe3157 is removed from metadata 2024.10.22 01:22:27.428810 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pfx/unmtfchujitfpuwncbaalfbgtqzwv. 2024.10.22 01:22:27.563236 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 113367 bytes for 7b28975773b4dd4cfecbcc8ff772401a:0) 2024.10.22 01:22:27.566291 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 4b1fbf802cc14b0fe3c29ad58b5118f6, offset: 0, size: 24042 2024.10.22 01:22:27.569683 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 4b1fbf802cc14b0fe3c29ad58b5118f6 for removal 2024.10.22 01:22:27.571733 [ 1476 ] {} CacheMetadata: Key 4b1fbf802cc14b0fe3c29ad58b5118f6 is removed from metadata 2024.10.22 01:22:27.585131 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 591823c1641fb277a6892c53dbd3ed03, offset: 0, size: 0 2024.10.22 01:22:27.586500 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 591823c1641fb277a6892c53dbd3ed03 for removal 2024.10.22 01:22:27.589051 [ 1476 ] {} CacheMetadata: Key 591823c1641fb277a6892c53dbd3ed03 is removed from metadata 2024.10.22 01:22:27.594574 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.96 GiB, free memory in arenas 6.13 MiB, will set to 1.14 GiB (RSS), difference: -18.10 MiB 2024.10.22 01:22:27.651421 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9f498d28b71ced5cae463677ca062239, offset: 0, size: 0 2024.10.22 01:22:27.653948 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9f498d28b71ced5cae463677ca062239 for removal 2024.10.22 01:22:27.657445 [ 1476 ] {} CacheMetadata: Key 9f498d28b71ced5cae463677ca062239 is removed from metadata 2024.10.22 01:22:27.731421 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rnf/dpurgjbvuhqbyyqnwuzwhdidgeppo. 2024.10.22 01:22:27.877106 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b0909abb3a5f79f4ea25f6827c307704, offset: 0, size: 0 2024.10.22 01:22:27.878396 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b0909abb3a5f79f4ea25f6827c307704 for removal 2024.10.22 01:22:27.881126 [ 1476 ] {} CacheMetadata: Key b0909abb3a5f79f4ea25f6827c307704 is removed from metadata 2024.10.22 01:22:27.996930 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_94_19 to 202410_1_94_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:28.003751 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_94_19} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_89_18, 202410_94_94_0] -> 202410_1_94_19 2024.10.22 01:22:28.007056 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 52258 bytes for 27f2f3e24e3e0e3436124869d109c988:0) 2024.10.22 01:22:28.011047 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 131faf4b7869a474dd543682ffb3c2e1, offset: 0, size: 251458 2024.10.22 01:22:28.023439 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 131faf4b7869a474dd543682ffb3c2e1 for removal 2024.10.22 01:22:28.029725 [ 1476 ] {} CacheMetadata: Key 131faf4b7869a474dd543682ffb3c2e1 is removed from metadata 2024.10.22 01:22:28.031269 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1af11dec421308c2668951b77adbdac4, offset: 0, size: 152 2024.10.22 01:22:28.044536 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1af11dec421308c2668951b77adbdac4 for removal 2024.10.22 01:22:28.054567 [ 1476 ] {} CacheMetadata: Key 1af11dec421308c2668951b77adbdac4 is removed from metadata 2024.10.22 01:22:28.131591 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2efaecaf5743bbd8a9f7c8246285c905, offset: 0, size: 0 2024.10.22 01:22:28.135636 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2efaecaf5743bbd8a9f7c8246285c905 for removal 2024.10.22 01:22:28.141059 [ 1476 ] {} CacheMetadata: Key 2efaecaf5743bbd8a9f7c8246285c905 is removed from metadata 2024.10.22 01:22:28.159298 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rnf/dpurgjbvuhqbyyqnwuzwhdidgeppo. 2024.10.22 01:22:28.307613 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2c1560d74c092e3e3199832096610092, offset: 0, size: 0 2024.10.22 01:22:28.311798 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2c1560d74c092e3e3199832096610092 for removal 2024.10.22 01:22:28.329754 [ 1476 ] {} CacheMetadata: Key 2c1560d74c092e3e3199832096610092 is removed from metadata 2024.10.22 01:22:28.464756 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 319804 bytes for 20564cb8304b74f99f8e177222d120de:0) 2024.10.22 01:22:28.466864 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0d6673e58e8ced9bc6e813d7994ee8e3, offset: 0, size: 246447 2024.10.22 01:22:28.478770 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5e4ee5d233542998094859aa8a3fe862, offset: 0, size: 0 2024.10.22 01:22:28.483018 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5e4ee5d233542998094859aa8a3fe862 for removal 2024.10.22 01:22:28.485221 [ 1476 ] {} CacheMetadata: Key 5e4ee5d233542998094859aa8a3fe862 is removed from metadata 2024.10.22 01:22:28.532244 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 86a61c55b65b9cd9a0d6ed39b6aff592, offset: 0, size: 0 2024.10.22 01:22:28.536391 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 86a61c55b65b9cd9a0d6ed39b6aff592 for removal 2024.10.22 01:22:28.569385 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 701516 2024.10.22 01:22:28.571170 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0d6673e58e8ced9bc6e813d7994ee8e3 for removal 2024.10.22 01:22:28.591744 [ 1476 ] {} CacheMetadata: Key 0d6673e58e8ced9bc6e813d7994ee8e3 is removed from metadata 2024.10.22 01:22:28.595440 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4bb8223b745898646ec0ff9e1f644bb5, offset: 0, size: 0 2024.10.22 01:22:28.597587 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4bb8223b745898646ec0ff9e1f644bb5 for removal 2024.10.22 01:22:28.674942 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1a51f4fb33e90abcdcd729a834f37f89, offset: 0, size: 154 2024.10.22 01:22:28.678308 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1a51f4fb33e90abcdcd729a834f37f89 for removal 2024.10.22 01:22:28.681388 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 2a41ea983675972ea23010f3341d9660, offset: 0, size: 509 2024.10.22 01:22:28.683884 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 2a41ea983675972ea23010f3341d9660 for removal 2024.10.22 01:22:28.717665 [ 1476 ] {} CacheMetadata: Key 2a41ea983675972ea23010f3341d9660 is removed from metadata 2024.10.22 01:22:28.724307 [ 1476 ] {} CacheMetadata: Key 1a51f4fb33e90abcdcd729a834f37f89 is removed from metadata 2024.10.22 01:22:28.747449 [ 1476 ] {} CacheMetadata: Key 4bb8223b745898646ec0ff9e1f644bb5 is removed from metadata 2024.10.22 01:22:28.759484 [ 1476 ] {} CacheMetadata: Key 86a61c55b65b9cd9a0d6ed39b6aff592 is removed from metadata 2024.10.22 01:22:28.775156 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fe59a3b7eb5ee2aa98770e38ea206b23, offset: 0, size: 0 2024.10.22 01:22:28.776327 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fe59a3b7eb5ee2aa98770e38ea206b23 for removal 2024.10.22 01:22:28.778220 [ 1476 ] {} CacheMetadata: Key fe59a3b7eb5ee2aa98770e38ea206b23 is removed from metadata 2024.10.22 01:22:28.830336 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4b4cdfba41a4a50bbdea90dd18f8d404, offset: 0, size: 0 2024.10.22 01:22:28.835010 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4b4cdfba41a4a50bbdea90dd18f8d404 for removal 2024.10.22 01:22:28.840889 [ 1476 ] {} CacheMetadata: Key 4b4cdfba41a4a50bbdea90dd18f8d404 is removed from metadata 2024.10.22 01:22:28.843952 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/klq/aooluchlywvibxzlgfnaevfzmckqn. 2024.10.22 01:22:28.928211 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 43630316afa988222e7acac6150524b3, offset: 0, size: 0 2024.10.22 01:22:28.930410 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 43630316afa988222e7acac6150524b3 for removal 2024.10.22 01:22:28.934809 [ 1476 ] {} CacheMetadata: Key 43630316afa988222e7acac6150524b3 is removed from metadata 2024.10.22 01:22:28.958579 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 42.67 MiB. 2024.10.22 01:22:29.062124 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 31eef16797a42d1c72eff929e924d9ba, offset: 0, size: 0 2024.10.22 01:22:29.064104 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 31eef16797a42d1c72eff929e924d9ba for removal 2024.10.22 01:22:29.067133 [ 1476 ] {} CacheMetadata: Key 31eef16797a42d1c72eff929e924d9ba is removed from metadata 2024.10.22 01:22:29.079578 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/klq/aooluchlywvibxzlgfnaevfzmckqn. 2024.10.22 01:22:29.082867 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yby/omjtswjittdbhjageemascqenxxjz. 2024.10.22 01:22:29.231064 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 87959b6a65c0fffc28c6b38382fd4463, offset: 0, size: 0 2024.10.22 01:22:29.232925 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 87959b6a65c0fffc28c6b38382fd4463 for removal 2024.10.22 01:22:29.236291 [ 1476 ] {} CacheMetadata: Key 87959b6a65c0fffc28c6b38382fd4463 is removed from metadata 2024.10.22 01:22:29.273449 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 430b7ed54f3b37946c41263cbae01cbd, offset: 0, size: 0 2024.10.22 01:22:29.276130 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 430b7ed54f3b37946c41263cbae01cbd for removal 2024.10.22 01:22:29.279879 [ 1476 ] {} CacheMetadata: Key 430b7ed54f3b37946c41263cbae01cbd is removed from metadata 2024.10.22 01:22:29.286455 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yby/omjtswjittdbhjageemascqenxxjz. 2024.10.22 01:22:29.344116 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 333945 bytes for f307af763f7352c69973aff47a714c67:0) 2024.10.22 01:22:29.349784 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7dc45f4ccacc4cbf1dbeee2fb34c37d6, offset: 0, size: 288790 2024.10.22 01:22:29.355661 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 932ff2d36befc626e049f7bc39258e15, offset: 0, size: 0 2024.10.22 01:22:29.357738 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 932ff2d36befc626e049f7bc39258e15 for removal 2024.10.22 01:22:29.360131 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7dc45f4ccacc4cbf1dbeee2fb34c37d6 for removal 2024.10.22 01:22:29.361767 [ 1476 ] {} CacheMetadata: Key 932ff2d36befc626e049f7bc39258e15 is removed from metadata 2024.10.22 01:22:29.368245 [ 1476 ] {} CacheMetadata: Key 7dc45f4ccacc4cbf1dbeee2fb34c37d6 is removed from metadata 2024.10.22 01:22:29.523574 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1e4499c1856b06df07f6c6ea122fbfbe, offset: 0, size: 0 2024.10.22 01:22:29.526204 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1e4499c1856b06df07f6c6ea122fbfbe for removal 2024.10.22 01:22:29.530284 [ 1476 ] {} CacheMetadata: Key 1e4499c1856b06df07f6c6ea122fbfbe is removed from metadata 2024.10.22 01:22:29.570683 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0d713e24fba00c6b2320514b48e52acb, offset: 0, size: 0 2024.10.22 01:22:29.574441 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0d713e24fba00c6b2320514b48e52acb for removal 2024.10.22 01:22:29.578324 [ 1476 ] {} CacheMetadata: Key 0d713e24fba00c6b2320514b48e52acb is removed from metadata 2024.10.22 01:22:29.624088 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f324b23042ac0024c5a21f33502a1cab, offset: 0, size: 0 2024.10.22 01:22:29.627751 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f324b23042ac0024c5a21f33502a1cab for removal 2024.10.22 01:22:29.632893 [ 1476 ] {} CacheMetadata: Key f324b23042ac0024c5a21f33502a1cab is removed from metadata 2024.10.22 01:22:29.716961 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 227e4cf47a389049f1db6ad9298c3628, offset: 0, size: 0 2024.10.22 01:22:29.720309 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 227e4cf47a389049f1db6ad9298c3628 for removal 2024.10.22 01:22:29.724959 [ 1476 ] {} CacheMetadata: Key 227e4cf47a389049f1db6ad9298c3628 is removed from metadata 2024.10.22 01:22:29.861769 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.12 GiB, peak 1.96 GiB, free memory in arenas 4.93 MiB, will set to 1.15 GiB (RSS), difference: 23.34 MiB 2024.10.22 01:22:29.887838 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:29.891875 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: eeb8ec1c385f28e5d1c46fd6f0cdcbe6, offset: 0, size: 0 2024.10.22 01:22:29.896346 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key eeb8ec1c385f28e5d1c46fd6f0cdcbe6 for removal 2024.10.22 01:22:29.904816 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kgd/ivarfoumosmdfhdwuqxvpunefsmwu. 2024.10.22 01:22:29.905614 [ 1476 ] {} CacheMetadata: Key eeb8ec1c385f28e5d1c46fd6f0cdcbe6 is removed from metadata 2024.10.22 01:22:29.996812 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:30.064798 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 45e0249c7ddd66652b5decdc0e565445, offset: 0, size: 0 2024.10.22 01:22:30.066808 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 45e0249c7ddd66652b5decdc0e565445 for removal 2024.10.22 01:22:30.074597 [ 1476 ] {} CacheMetadata: Key 45e0249c7ddd66652b5decdc0e565445 is removed from metadata 2024.10.22 01:22:30.199493 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:22:30.204411 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:22:30.238181 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4fadfb920f723f9a3cfba6f5c5ec8948, offset: 0, size: 0 2024.10.22 01:22:30.242212 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4fadfb920f723f9a3cfba6f5c5ec8948 for removal 2024.10.22 01:22:30.248986 [ 1476 ] {} CacheMetadata: Key 4fadfb920f723f9a3cfba6f5c5ec8948 is removed from metadata 2024.10.22 01:22:30.398765 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 159030 bytes for 0d3e5b9ce612b39ff858433bf16cd51a:0) 2024.10.22 01:22:30.403523 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 27d16cb7f0bcd0e87e60aa10664fd33a, offset: 0, size: 353640 2024.10.22 01:22:30.409058 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 27d16cb7f0bcd0e87e60aa10664fd33a for removal 2024.10.22 01:22:30.413187 [ 1476 ] {} CacheMetadata: Key 27d16cb7f0bcd0e87e60aa10664fd33a is removed from metadata 2024.10.22 01:22:30.414949 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: c504a02030d7a2eac86235781bd437f8, offset: 0, size: 155 2024.10.22 01:22:30.420419 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key c504a02030d7a2eac86235781bd437f8 for removal 2024.10.22 01:22:30.460150 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kgd/ivarfoumosmdfhdwuqxvpunefsmwu. 2024.10.22 01:22:30.504276 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fdb48e5626baa87516e20bcfb706529a, offset: 0, size: 0 2024.10.22 01:22:30.505673 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fdb48e5626baa87516e20bcfb706529a for removal 2024.10.22 01:22:30.532375 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_446_446_0 (state Temporary) 2024.10.22 01:22:30.548925 [ 1476 ] {} CacheMetadata: Key c504a02030d7a2eac86235781bd437f8 is removed from metadata 2024.10.22 01:22:30.559056 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f76b51f84b21b21b621b780b4d8f2ec0, offset: 0, size: 0 2024.10.22 01:22:30.560424 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f76b51f84b21b21b621b780b4d8f2ec0 for removal 2024.10.22 01:22:30.565133 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nqn/sfhmcajghpkxahmlbixcjubfxsyyw. 2024.10.22 01:22:30.602359 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_446_446_0 to 202410_453_453_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:30.661291 [ 1476 ] {} CacheMetadata: Key f76b51f84b21b21b621b780b4d8f2ec0 is removed from metadata 2024.10.22 01:22:30.665243 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 039fee4f144ad77f0e7d6279abc1206b, offset: 0, size: 0 2024.10.22 01:22:30.667133 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 039fee4f144ad77f0e7d6279abc1206b for removal 2024.10.22 01:22:30.667829 [ 1476 ] {} CacheMetadata: Key fdb48e5626baa87516e20bcfb706529a is removed from metadata 2024.10.22 01:22:30.676200 [ 1476 ] {} CacheMetadata: Key 039fee4f144ad77f0e7d6279abc1206b is removed from metadata 2024.10.22 01:22:30.712991 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 12ec9b6b65575195e043693785f1b780, offset: 0, size: 0 2024.10.22 01:22:30.715117 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 12ec9b6b65575195e043693785f1b780 for removal 2024.10.22 01:22:30.721177 [ 1476 ] {} CacheMetadata: Key 12ec9b6b65575195e043693785f1b780 is removed from metadata 2024.10.22 01:22:30.736950 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:30.741129 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nqn/sfhmcajghpkxahmlbixcjubfxsyyw. 2024.10.22 01:22:30.752097 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 701516 2024.10.22 01:22:30.843857 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:30.739624 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_795_403} MergeTask::MergeProjectionsStage: Merge sorted 50995 rows, containing 13 columns (13 merged, 0 gathered) in 3.860895923 sec., 13208.07424417071 rows/sec., 2.12 MiB/sec. 2024.10.22 01:22:30.878317 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f0f42d0fb254366fca3355f6f3927552, offset: 0, size: 0 2024.10.22 01:22:30.886288 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f0f42d0fb254366fca3355f6f3927552 for removal 2024.10.22 01:22:30.889190 [ 1476 ] {} CacheMetadata: Key f0f42d0fb254366fca3355f6f3927552 is removed from metadata 2024.10.22 01:22:30.959744 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 328862 bytes for 9b246279a57be6f0a62aa34aedcfcc58:0) 2024.10.22 01:22:30.964141 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cb72abc6c6fbe6729fca5d7edc853a48, offset: 0, size: 539976 2024.10.22 01:22:30.967312 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cb72abc6c6fbe6729fca5d7edc853a48 for removal 2024.10.22 01:22:30.971445 [ 1476 ] {} CacheMetadata: Key cb72abc6c6fbe6729fca5d7edc853a48 is removed from metadata 2024.10.22 01:22:30.972368 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 9c32a7822800f428ed8b519f8d25b4ac, offset: 0, size: 158 2024.10.22 01:22:31.051360 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6e39be6328441068b6b3ac6b4ed6bc47, offset: 0, size: 0 2024.10.22 01:22:31.126867 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 277 entries to flush up to offset 51365 2024.10.22 01:22:31.130531 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6e39be6328441068b6b3ac6b4ed6bc47 for removal 2024.10.22 01:22:31.131192 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 9c32a7822800f428ed8b519f8d25b4ac for removal 2024.10.22 01:22:31.135017 [ 1476 ] {} CacheMetadata: Key 6e39be6328441068b6b3ac6b4ed6bc47 is removed from metadata 2024.10.22 01:22:31.136612 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zes/tehrstlwagxfsuqgjfearqlualrfy. 2024.10.22 01:22:31.139320 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 039a32c53f82487e7c90ab13bb19fdd9, offset: 0, size: 513 2024.10.22 01:22:31.144396 [ 1476 ] {} CacheMetadata: Key 9c32a7822800f428ed8b519f8d25b4ac is removed from metadata 2024.10.22 01:22:31.151798 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 039a32c53f82487e7c90ab13bb19fdd9 for removal 2024.10.22 01:22:31.163184 [ 1476 ] {} CacheMetadata: Key 039a32c53f82487e7c90ab13bb19fdd9 is removed from metadata 2024.10.22 01:22:31.166140 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_795_403} MergedBlockOutputStream: filled checksums 202410_1_795_403 (state Temporary) 2024.10.22 01:22:31.310956 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_696_314, 202410_697_697_0] 2024.10.22 01:22:31.312970 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_696_314, 202410_697_697_0] 2024.10.22 01:22:31.422537 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b010bd28ebb7fcfc822c854c2a58989b, offset: 0, size: 0 2024.10.22 01:22:31.422537 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_795_403} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_795_403 to 202410_1_795_403 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:31.424509 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 7201 2024.10.22 01:22:31.425473 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b010bd28ebb7fcfc822c854c2a58989b for removal 2024.10.22 01:22:31.427882 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_795_403} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_794_402, 202410_795_795_0] -> 202410_1_795_403 2024.10.22 01:22:31.428407 [ 1476 ] {} CacheMetadata: Key b010bd28ebb7fcfc822c854c2a58989b is removed from metadata 2024.10.22 01:22:31.472293 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.82 MiB. 2024.10.22 01:22:31.528788 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_696_314, 202410_697_697_0] 2024.10.22 01:22:31.655348 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:31.660460 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:31.753244 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.13 GiB, peak 1.96 GiB, free memory in arenas 5.58 MiB, will set to 1.14 GiB (RSS), difference: 8.07 MiB 2024.10.22 01:22:31.781223 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a531f44242ca4968960fae50a82e0fbc, offset: 0, size: 0 2024.10.22 01:22:31.783311 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a531f44242ca4968960fae50a82e0fbc for removal 2024.10.22 01:22:31.788833 [ 1476 ] {} CacheMetadata: Key a531f44242ca4968960fae50a82e0fbc is removed from metadata 2024.10.22 01:22:31.791796 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_465_465_0 (state Temporary) 2024.10.22 01:22:31.824828 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zes/tehrstlwagxfsuqgjfearqlualrfy. 2024.10.22 01:22:31.865403 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_465_465_0 to 202410_478_478_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:31.873483 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:31.876201 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:32.007652 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7201 2024.10.22 01:22:32.083826 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fc4942846d9f0b18b50b47217299149f, offset: 0, size: 0 2024.10.22 01:22:32.086549 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fc4942846d9f0b18b50b47217299149f for removal 2024.10.22 01:22:32.088221 [ 1476 ] {} CacheMetadata: Key fc4942846d9f0b18b50b47217299149f is removed from metadata 2024.10.22 01:22:32.138568 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3b209d8c6973eb93c7dd1309982b3a2b, offset: 0, size: 0 2024.10.22 01:22:32.142287 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3b209d8c6973eb93c7dd1309982b3a2b for removal 2024.10.22 01:22:32.152137 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uxk/zietyegqpshabekprdmhektbiojxp. 2024.10.22 01:22:32.154179 [ 1476 ] {} CacheMetadata: Key 3b209d8c6973eb93c7dd1309982b3a2b is removed from metadata 2024.10.22 01:22:32.187560 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: eac1d965cd8c29f7b09462d29912b8b8, offset: 0, size: 0 2024.10.22 01:22:32.190282 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key eac1d965cd8c29f7b09462d29912b8b8 for removal 2024.10.22 01:22:32.199261 [ 1476 ] {} CacheMetadata: Key eac1d965cd8c29f7b09462d29912b8b8 is removed from metadata 2024.10.22 01:22:32.237781 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 471311 bytes for 8591fc3e15994a3da92bce5853ece44d:0) 2024.10.22 01:22:32.249724 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f45f57cce203d6facf3df26e278bfd32, offset: 0, size: 537122 2024.10.22 01:22:32.255069 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f45f57cce203d6facf3df26e278bfd32 for removal 2024.10.22 01:22:32.260097 [ 1476 ] {} CacheMetadata: Key f45f57cce203d6facf3df26e278bfd32 is removed from metadata 2024.10.22 01:22:32.260431 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 0d2a7d5b88f77cc524d1f494fe78747d, offset: 0, size: 159 2024.10.22 01:22:32.264215 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 0d2a7d5b88f77cc524d1f494fe78747d for removal 2024.10.22 01:22:32.266333 [ 1476 ] {} CacheMetadata: Key 0d2a7d5b88f77cc524d1f494fe78747d is removed from metadata 2024.10.22 01:22:32.266756 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b1515df9bd43a94426139b71a8410581, offset: 0, size: 230914 2024.10.22 01:22:32.271064 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b1515df9bd43a94426139b71a8410581 for removal 2024.10.22 01:22:32.273782 [ 1476 ] {} CacheMetadata: Key b1515df9bd43a94426139b71a8410581 is removed from metadata 2024.10.22 01:22:32.379682 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_788_788_0 (state Temporary) 2024.10.22 01:22:32.408551 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1b51ebdc3eefb3e2f4723b6a4005ea22, offset: 0, size: 0 2024.10.22 01:22:32.421301 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1b51ebdc3eefb3e2f4723b6a4005ea22 for removal 2024.10.22 01:22:32.429629 [ 1476 ] {} CacheMetadata: Key 1b51ebdc3eefb3e2f4723b6a4005ea22 is removed from metadata 2024.10.22 01:22:32.479642 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_788_788_0 to 202410_801_801_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:32.526757 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 51365 2024.10.22 01:22:32.541841 [ 1225 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_795_403 to 202410_796_796_0 2024.10.22 01:22:32.558351 [ 1225 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.84 MiB using storage policy from min volume index 0 2024.10.22 01:22:32.560938 [ 1225 ] {} DiskLocal: Reserved 1.84 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:32.595358 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_796_404} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_795_403 to 202410_796_796_0 into Compact with storage Full 2024.10.22 01:22:32.637285 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_796_404} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:32.655743 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_796_404} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_795_403, total 50995 rows starting from the beginning of the part 2024.10.22 01:22:32.674158 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_796_404} MergeTreeSequentialSource: Reading 2 marks from part 202410_796_796_0, total 250 rows starting from the beginning of the part 2024.10.22 01:22:32.744510 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uxk/zietyegqpshabekprdmhektbiojxp. 2024.10.22 01:22:32.783023 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 44f336792bad7b0ea672d2e80a86c72c, offset: 0, size: 0 2024.10.22 01:22:32.784111 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 44f336792bad7b0ea672d2e80a86c72c for removal 2024.10.22 01:22:32.788338 [ 1476 ] {} CacheMetadata: Key 44f336792bad7b0ea672d2e80a86c72c is removed from metadata 2024.10.22 01:22:32.953067 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 44cb6b0f4f7a00b9c7eefe8b68282699, offset: 0, size: 0 2024.10.22 01:22:32.958768 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 44cb6b0f4f7a00b9c7eefe8b68282699 for removal 2024.10.22 01:22:32.962027 [ 1476 ] {} CacheMetadata: Key 44cb6b0f4f7a00b9c7eefe8b68282699 is removed from metadata 2024.10.22 01:22:32.964071 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hxw/crditzkzsmwxutjgpeivhrqnjpach. 2024.10.22 01:22:33.078784 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_400_178, 202410_401_401_0] 2024.10.22 01:22:33.079921 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_400_178, 202410_401_401_0] 2024.10.22 01:22:33.103845 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_400_178, 202410_401_401_0] 2024.10.22 01:22:33.113090 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: da1e72161092a71196d0b8c76ba7e76a, offset: 0, size: 0 2024.10.22 01:22:33.115131 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key da1e72161092a71196d0b8c76ba7e76a for removal 2024.10.22 01:22:33.117558 [ 1476 ] {} CacheMetadata: Key da1e72161092a71196d0b8c76ba7e76a is removed from metadata 2024.10.22 01:22:33.183575 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 809069 bytes for 7f70add76970701a3f9562ce9721214e:0) 2024.10.22 01:22:33.185668 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 535d5511e222b0a3092f01fea7bce0d3, offset: 0, size: 2027602 2024.10.22 01:22:33.192910 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 535d5511e222b0a3092f01fea7bce0d3 for removal 2024.10.22 01:22:33.200734 [ 1476 ] {} CacheMetadata: Key 535d5511e222b0a3092f01fea7bce0d3 is removed from metadata 2024.10.22 01:22:33.204959 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 249770be3ebb62e8fbbb19bda7efd583, offset: 0, size: 363170 2024.10.22 01:22:33.210702 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 249770be3ebb62e8fbbb19bda7efd583 for removal 2024.10.22 01:22:33.215703 [ 1476 ] {} CacheMetadata: Key 249770be3ebb62e8fbbb19bda7efd583 is removed from metadata 2024.10.22 01:22:33.216698 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 635c7460dbe598ac6986daa956dfc152, offset: 0, size: 902 2024.10.22 01:22:33.223865 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 635c7460dbe598ac6986daa956dfc152 for removal 2024.10.22 01:22:33.226890 [ 1476 ] {} CacheMetadata: Key 635c7460dbe598ac6986daa956dfc152 is removed from metadata 2024.10.22 01:22:33.227020 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 025938a97a91ebb067b5b161b0cdb6d6, offset: 0, size: 155 2024.10.22 01:22:33.230958 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 025938a97a91ebb067b5b161b0cdb6d6 for removal 2024.10.22 01:22:33.234680 [ 1476 ] {} CacheMetadata: Key 025938a97a91ebb067b5b161b0cdb6d6 is removed from metadata 2024.10.22 01:22:33.519631 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 96 entries to flush up to offset 52524 2024.10.22 01:22:33.532012 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.96 GiB, free memory in arenas 4.86 MiB, will set to 1.14 GiB (RSS), difference: -9.28 MiB 2024.10.22 01:22:33.538570 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2d99f3f85fdaa21b5ec01fd7a07da00a, offset: 0, size: 0 2024.10.22 01:22:33.541423 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2d99f3f85fdaa21b5ec01fd7a07da00a for removal 2024.10.22 01:22:33.548254 [ 1476 ] {} CacheMetadata: Key 2d99f3f85fdaa21b5ec01fd7a07da00a is removed from metadata 2024.10.22 01:22:33.566616 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hxw/crditzkzsmwxutjgpeivhrqnjpach. 2024.10.22 01:22:33.717686 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0eae7d0c9734ce19fa15b0a46cfb6aca, offset: 0, size: 0 2024.10.22 01:22:33.719803 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0eae7d0c9734ce19fa15b0a46cfb6aca for removal 2024.10.22 01:22:33.723979 [ 1476 ] {} CacheMetadata: Key 0eae7d0c9734ce19fa15b0a46cfb6aca is removed from metadata 2024.10.22 01:22:33.855163 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:33.865142 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 92c3d1c705e09e5082f1b3d8637dd4f1, offset: 0, size: 0 2024.10.22 01:22:33.866693 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 92c3d1c705e09e5082f1b3d8637dd4f1 for removal 2024.10.22 01:22:33.870338 [ 1476 ] {} CacheMetadata: Key 92c3d1c705e09e5082f1b3d8637dd4f1 is removed from metadata 2024.10.22 01:22:33.911325 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4577a628869b840af25850673170c5fc, offset: 0, size: 0 2024.10.22 01:22:33.913622 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4577a628869b840af25850673170c5fc for removal 2024.10.22 01:22:33.919068 [ 1476 ] {} CacheMetadata: Key 4577a628869b840af25850673170c5fc is removed from metadata 2024.10.22 01:22:33.957597 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 394 2024.10.22 01:22:33.857831 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:33.966649 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: eee49173a22bbae9fd05bbc3edb044ed, offset: 0, size: 0 2024.10.22 01:22:33.968556 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key eee49173a22bbae9fd05bbc3edb044ed for removal 2024.10.22 01:22:33.970665 [ 1476 ] {} CacheMetadata: Key eee49173a22bbae9fd05bbc3edb044ed is removed from metadata 2024.10.22 01:22:34.055232 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1bc57785d81387c43be37462ff11fcf9, offset: 0, size: 0 2024.10.22 01:22:34.060035 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1bc57785d81387c43be37462ff11fcf9 for removal 2024.10.22 01:22:34.063834 [ 1476 ] {} CacheMetadata: Key 1bc57785d81387c43be37462ff11fcf9 is removed from metadata 2024.10.22 01:22:34.112750 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_457_457_0 (state Temporary) 2024.10.22 01:22:34.154556 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_697_315, 202410_698_698_0] 2024.10.22 01:22:34.161260 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_697_315, 202410_698_698_0] 2024.10.22 01:22:34.203590 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4e0234cc71967188db9ab05e1b33745c, offset: 0, size: 0 2024.10.22 01:22:34.206223 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4e0234cc71967188db9ab05e1b33745c for removal 2024.10.22 01:22:34.209080 [ 1476 ] {} CacheMetadata: Key 4e0234cc71967188db9ab05e1b33745c is removed from metadata 2024.10.22 01:22:34.209582 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vjn/ndpewmjzxiqespwdriyzvcfdgqpou. 2024.10.22 01:22:34.253156 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:34.254098 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d66993edab7dc5024008f7b298b9bb32, offset: 0, size: 0 2024.10.22 01:22:34.255741 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d66993edab7dc5024008f7b298b9bb32 for removal 2024.10.22 01:22:34.255683 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:34.258212 [ 1476 ] {} CacheMetadata: Key d66993edab7dc5024008f7b298b9bb32 is removed from metadata 2024.10.22 01:22:34.273605 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_697_315, 202410_698_698_0] 2024.10.22 01:22:34.284492 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_457_457_0 to 202410_463_463_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:34.308802 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ebbcb8bff22adf32d162786ddf2171ce, offset: 0, size: 0 2024.10.22 01:22:34.311017 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ebbcb8bff22adf32d162786ddf2171ce for removal 2024.10.22 01:22:34.312638 [ 1476 ] {} CacheMetadata: Key ebbcb8bff22adf32d162786ddf2171ce is removed from metadata 2024.10.22 01:22:34.334640 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_376_376_0 (state Temporary) 2024.10.22 01:22:34.354694 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 52524 2024.10.22 01:22:34.365069 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vjn/ndpewmjzxiqespwdriyzvcfdgqpou. 2024.10.22 01:22:34.413148 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cb9bc238079c96feea4bee8a243cfa01, offset: 0, size: 0 2024.10.22 01:22:34.415069 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cb9bc238079c96feea4bee8a243cfa01 for removal 2024.10.22 01:22:34.419464 [ 1476 ] {} CacheMetadata: Key cb9bc238079c96feea4bee8a243cfa01 is removed from metadata 2024.10.22 01:22:34.613599 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 586600 bytes for 89594ca5ca4a3a1ba0f0f7c65cad868b:0) 2024.10.22 01:22:34.616550 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 375815e6bc16deace3ea4552b4a2868c, offset: 0, size: 249662 2024.10.22 01:22:34.674839 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f9f5a108fb42d5927b613e9d4e0eea0b, offset: 0, size: 0 2024.10.22 01:22:34.676212 [ 1222 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:22:34.676158 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_94_94_0 (state Temporary) 2024.10.22 01:22:34.676333 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f9f5a108fb42d5927b613e9d4e0eea0b for removal 2024.10.22 01:22:34.682615 [ 1476 ] {} CacheMetadata: Key f9f5a108fb42d5927b613e9d4e0eea0b is removed from metadata 2024.10.22 01:22:34.683151 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/bga/rzmwvrkiwjryqsmrkqcxsjtjdrxto. 2024.10.22 01:22:34.712152 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_376_376_0 to 202410_386_386_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:34.722201 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 375815e6bc16deace3ea4552b4a2868c for removal 2024.10.22 01:22:34.726421 [ 1222 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:22:34.727891 [ 1476 ] {} CacheMetadata: Key 375815e6bc16deace3ea4552b4a2868c is removed from metadata 2024.10.22 01:22:34.728344 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: ca1229bb53badf5145147ebd27ed74f2, offset: 0, size: 152 2024.10.22 01:22:34.731670 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ffbfa8bdb567049d11a961d85284e0eb, offset: 0, size: 0 2024.10.22 01:22:34.733594 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key ca1229bb53badf5145147ebd27ed74f2 for removal 2024.10.22 01:22:34.733451 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ffbfa8bdb567049d11a961d85284e0eb for removal 2024.10.22 01:22:34.777065 [ 1203 ] {} DatabaseCatalog: Execution took 205 ms. 2024.10.22 01:22:34.843017 [ 1476 ] {} CacheMetadata: Key ca1229bb53badf5145147ebd27ed74f2 is removed from metadata 2024.10.22 01:22:34.851769 [ 1476 ] {} CacheMetadata: Key ffbfa8bdb567049d11a961d85284e0eb is removed from metadata 2024.10.22 01:22:34.879044 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 73e55aebde06b43c03b522f502ef75a2, offset: 0, size: 0 2024.10.22 01:22:34.883779 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 73e55aebde06b43c03b522f502ef75a2 for removal 2024.10.22 01:22:34.899151 [ 1476 ] {} CacheMetadata: Key 73e55aebde06b43c03b522f502ef75a2 is removed from metadata 2024.10.22 01:22:35.015015 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/bga/rzmwvrkiwjryqsmrkqcxsjtjdrxto. 2024.10.22 01:22:35.055357 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 58f84dd602484f7138d655e10c7983d6, offset: 0, size: 0 2024.10.22 01:22:35.056643 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 58f84dd602484f7138d655e10c7983d6 for removal 2024.10.22 01:22:35.058894 [ 1476 ] {} CacheMetadata: Key 58f84dd602484f7138d655e10c7983d6 is removed from metadata 2024.10.22 01:22:35.079148 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 394 2024.10.22 01:22:35.088248 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6d27075e331496096b2a322e1eba96ad, offset: 0, size: 0 2024.10.22 01:22:35.091164 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6d27075e331496096b2a322e1eba96ad for removal 2024.10.22 01:22:35.095543 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/oxl/cwsvnanjxliypwejlmegaivrpeeoc. 2024.10.22 01:22:35.199050 [ 1476 ] {} CacheMetadata: Key 6d27075e331496096b2a322e1eba96ad is removed from metadata 2024.10.22 01:22:35.240261 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9c78c1355b35309e09a722975ea45f4f, offset: 0, size: 0 2024.10.22 01:22:35.241885 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9c78c1355b35309e09a722975ea45f4f for removal 2024.10.22 01:22:35.247306 [ 1476 ] {} CacheMetadata: Key 9c78c1355b35309e09a722975ea45f4f is removed from metadata 2024.10.22 01:22:35.273315 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 3 file segments (while reserving 523744 bytes for a86ee83034b72d15f2b508e6983cfbd2:0) 2024.10.22 01:22:35.275366 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 927ceebccecc3160f7e9ec9243418846, offset: 0, size: 1217502 2024.10.22 01:22:35.278506 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 927ceebccecc3160f7e9ec9243418846 for removal 2024.10.22 01:22:35.280790 [ 1476 ] {} CacheMetadata: Key 927ceebccecc3160f7e9ec9243418846 is removed from metadata 2024.10.22 01:22:35.281258 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 7ca21974889b6daf87f179e277d34f3e, offset: 0, size: 537 2024.10.22 01:22:35.284847 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 7ca21974889b6daf87f179e277d34f3e for removal 2024.10.22 01:22:35.287359 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cd6c39ce47813cb8f0912519d31eef7b, offset: 0, size: 158 2024.10.22 01:22:35.287684 [ 1476 ] {} CacheMetadata: Key 7ca21974889b6daf87f179e277d34f3e is removed from metadata 2024.10.22 01:22:35.290582 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cd6c39ce47813cb8f0912519d31eef7b for removal 2024.10.22 01:22:35.301382 [ 1476 ] {} CacheMetadata: Key cd6c39ce47813cb8f0912519d31eef7b is removed from metadata 2024.10.22 01:22:35.305836 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.14 GiB, peak 1.96 GiB, free memory in arenas 5.58 MiB, will set to 1.14 GiB (RSS), difference: 8.38 MiB 2024.10.22 01:22:35.478980 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2e12083ae0a5f22060606fb7150df6a8, offset: 0, size: 0 2024.10.22 01:22:35.480521 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2e12083ae0a5f22060606fb7150df6a8 for removal 2024.10.22 01:22:35.482590 [ 1476 ] {} CacheMetadata: Key 2e12083ae0a5f22060606fb7150df6a8 is removed from metadata 2024.10.22 01:22:35.491168 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/oxl/cwsvnanjxliypwejlmegaivrpeeoc. 2024.10.22 01:22:35.552329 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ed8b858d432e413823d41c5c4167f4b9, offset: 0, size: 0 2024.10.22 01:22:35.558072 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ed8b858d432e413823d41c5c4167f4b9 for removal 2024.10.22 01:22:35.560701 [ 1476 ] {} CacheMetadata: Key ed8b858d432e413823d41c5c4167f4b9 is removed from metadata 2024.10.22 01:22:35.611638 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_796_404} MergeTask::MergeProjectionsStage: Merge sorted 51245 rows, containing 13 columns (13 merged, 0 gathered) in 3.032241952 sec., 16900.03661027113 rows/sec., 2.71 MiB/sec. 2024.10.22 01:22:35.691892 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 990175d2e6cd6c67cd641c2094595e24, offset: 0, size: 0 2024.10.22 01:22:35.693151 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 990175d2e6cd6c67cd641c2094595e24 for removal 2024.10.22 01:22:35.696579 [ 1476 ] {} CacheMetadata: Key 990175d2e6cd6c67cd641c2094595e24 is removed from metadata 2024.10.22 01:22:35.762920 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_796_404} MergedBlockOutputStream: filled checksums 202410_1_796_404 (state Temporary) 2024.10.22 01:22:35.809326 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_796_404} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_796_404 to 202410_1_796_404 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:35.814161 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_796_404} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_795_403, 202410_796_796_0] -> 202410_1_796_404 2024.10.22 01:22:35.844061 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:22:35.956346 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3096ca39c047a0fde878540ea20b7c55, offset: 0, size: 0 2024.10.22 01:22:35.959333 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3096ca39c047a0fde878540ea20b7c55 for removal 2024.10.22 01:22:35.963591 [ 1476 ] {} CacheMetadata: Key 3096ca39c047a0fde878540ea20b7c55 is removed from metadata 2024.10.22 01:22:36.095199 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1c1befa5ba2069bd78040446358ebfc3, offset: 0, size: 0 2024.10.22 01:22:36.096998 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1c1befa5ba2069bd78040446358ebfc3 for removal 2024.10.22 01:22:36.102268 [ 1476 ] {} CacheMetadata: Key 1c1befa5ba2069bd78040446358ebfc3 is removed from metadata 2024.10.22 01:22:36.164495 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: af4a2ecaa67b5d756b0ce3bf75d2afd1, offset: 0, size: 0 2024.10.22 01:22:36.167167 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key af4a2ecaa67b5d756b0ce3bf75d2afd1 for removal 2024.10.22 01:22:36.210912 [ 1476 ] {} CacheMetadata: Key af4a2ecaa67b5d756b0ce3bf75d2afd1 is removed from metadata 2024.10.22 01:22:36.299564 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 1 file segments (while reserving 261779 bytes for 722dd7ffc90957a7ddcd0911d0bc1957:0) 2024.10.22 01:22:36.304592 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 1bd8eaca9a246e07ff4d4625150b60cb, offset: 0, size: 470696 2024.10.22 01:22:36.307291 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6f0727a112ca65511cb20faebcedf97f, offset: 0, size: 0 2024.10.22 01:22:36.308871 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 1bd8eaca9a246e07ff4d4625150b60cb for removal 2024.10.22 01:22:36.309633 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6f0727a112ca65511cb20faebcedf97f for removal 2024.10.22 01:22:36.312318 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kvd/yhbsmwmddoplbqkjuyzcgzqpdxlte. 2024.10.22 01:22:36.312672 [ 1476 ] {} CacheMetadata: Key 6f0727a112ca65511cb20faebcedf97f is removed from metadata 2024.10.22 01:22:36.330728 [ 1476 ] {} CacheMetadata: Key 1bd8eaca9a246e07ff4d4625150b60cb is removed from metadata 2024.10.22 01:22:36.365110 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 900439dadfd3b63f5e2af7136715978e, offset: 0, size: 0 2024.10.22 01:22:36.366500 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 900439dadfd3b63f5e2af7136715978e for removal 2024.10.22 01:22:36.372091 [ 1476 ] {} CacheMetadata: Key 900439dadfd3b63f5e2af7136715978e is removed from metadata 2024.10.22 01:22:36.410396 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.13 GiB, peak 1.96 GiB, free memory in arenas 4.58 MiB, will set to 1.14 GiB (RSS), difference: 5.31 MiB 2024.10.22 01:22:36.422397 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 42386e0b579dc08dd1c72207081e1bb8, offset: 0, size: 0 2024.10.22 01:22:36.424207 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 42386e0b579dc08dd1c72207081e1bb8 for removal 2024.10.22 01:22:36.431090 [ 1476 ] {} CacheMetadata: Key 42386e0b579dc08dd1c72207081e1bb8 is removed from metadata 2024.10.22 01:22:36.545631 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kvd/yhbsmwmddoplbqkjuyzcgzqpdxlte. 2024.10.22 01:22:36.567057 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 225 entries to flush up to offset 51590 2024.10.22 01:22:36.601798 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6efd086754eacb7fb48aa8a52583f0bc, offset: 0, size: 0 2024.10.22 01:22:36.605667 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6efd086754eacb7fb48aa8a52583f0bc for removal 2024.10.22 01:22:36.611126 [ 1476 ] {} CacheMetadata: Key 6efd086754eacb7fb48aa8a52583f0bc is removed from metadata 2024.10.22 01:22:36.640192 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 581122 bytes for 4fe2b3c0b25fcd8478127e8538977c02:0) 2024.10.22 01:22:36.644250 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 82d8a59cc84109950d3d8f45800f4954, offset: 0, size: 416232 2024.10.22 01:22:36.651796 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 82d8a59cc84109950d3d8f45800f4954 for removal 2024.10.22 01:22:36.654849 [ 1476 ] {} CacheMetadata: Key 82d8a59cc84109950d3d8f45800f4954 is removed from metadata 2024.10.22 01:22:36.755949 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d1e8df9a37ee5b310b4b590787e0f3d1, offset: 0, size: 156 2024.10.22 01:22:36.757534 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d1e8df9a37ee5b310b4b590787e0f3d1 for removal 2024.10.22 01:22:36.759589 [ 1476 ] {} CacheMetadata: Key d1e8df9a37ee5b310b4b590787e0f3d1 is removed from metadata 2024.10.22 01:22:36.759588 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e280326fa9831821528e3759708784ce, offset: 0, size: 0 2024.10.22 01:22:36.762154 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e280326fa9831821528e3759708784ce for removal 2024.10.22 01:22:36.764919 [ 1476 ] {} CacheMetadata: Key e280326fa9831821528e3759708784ce is removed from metadata 2024.10.22 01:22:36.765330 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/omh/nqakwcdphbgbovrdgtzygtvkigmzw. 2024.10.22 01:22:36.823794 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8ca6bd88d0ed832c82b84e4cf351a830, offset: 0, size: 0 2024.10.22 01:22:36.826935 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8ca6bd88d0ed832c82b84e4cf351a830 for removal 2024.10.22 01:22:36.830444 [ 1476 ] {} CacheMetadata: Key 8ca6bd88d0ed832c82b84e4cf351a830 is removed from metadata 2024.10.22 01:22:36.988633 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f9252afa461f19db7f00e9f2fe85bd92, offset: 0, size: 0 2024.10.22 01:22:36.989782 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f9252afa461f19db7f00e9f2fe85bd92 for removal 2024.10.22 01:22:36.998658 [ 1476 ] {} CacheMetadata: Key f9252afa461f19db7f00e9f2fe85bd92 is removed from metadata 2024.10.22 01:22:37.019336 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/omh/nqakwcdphbgbovrdgtzygtvkigmzw. 2024.10.22 01:22:37.067146 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cf1b07f1e944b97ddbefdf9020f867ea, offset: 0, size: 0 2024.10.22 01:22:37.069603 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cf1b07f1e944b97ddbefdf9020f867ea for removal 2024.10.22 01:22:37.072427 [ 1476 ] {} CacheMetadata: Key cf1b07f1e944b97ddbefdf9020f867ea is removed from metadata 2024.10.22 01:22:37.112811 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 604ff3cb7e63a51db633051e55318cb3, offset: 0, size: 0 2024.10.22 01:22:37.215671 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 604ff3cb7e63a51db633051e55318cb3 for removal 2024.10.22 01:22:37.221527 [ 1476 ] {} CacheMetadata: Key 604ff3cb7e63a51db633051e55318cb3 is removed from metadata 2024.10.22 01:22:37.290762 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 378532 bytes for b54a655133b412db38a699bba80223ea:0) 2024.10.22 01:22:37.293529 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: a2baf90974a9250621c070862e0a1d36, offset: 0, size: 238389 2024.10.22 01:22:37.301632 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key a2baf90974a9250621c070862e0a1d36 for removal 2024.10.22 01:22:37.307759 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cc01d1e77a2c672462b9c3b61386f4c5, offset: 0, size: 152 2024.10.22 01:22:37.308166 [ 1476 ] {} CacheMetadata: Key a2baf90974a9250621c070862e0a1d36 is removed from metadata 2024.10.22 01:22:37.312643 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cc01d1e77a2c672462b9c3b61386f4c5 for removal 2024.10.22 01:22:37.319144 [ 1476 ] {} CacheMetadata: Key cc01d1e77a2c672462b9c3b61386f4c5 is removed from metadata 2024.10.22 01:22:37.412852 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:37.420617 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:37.530331 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 1048576 bytes for cf1b07f1e944b97ddbefdf9020f867ea:0) 2024.10.22 01:22:37.530522 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_789_789_0 (state Temporary) 2024.10.22 01:22:37.531586 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cbc46e596bbd58650361c4d75bbbec8d, offset: 0, size: 458566 2024.10.22 01:22:37.534216 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cbc46e596bbd58650361c4d75bbbec8d for removal 2024.10.22 01:22:37.538949 [ 1476 ] {} CacheMetadata: Key cbc46e596bbd58650361c4d75bbbec8d is removed from metadata 2024.10.22 01:22:37.539201 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3a74075a84649e9b590ae21f64f1581a, offset: 0, size: 152 2024.10.22 01:22:37.544586 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3a74075a84649e9b590ae21f64f1581a for removal 2024.10.22 01:22:37.549708 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 48881e5b9926c6a1915f4232f81cbc42, offset: 0, size: 239746 2024.10.22 01:22:37.553301 [ 1476 ] {} CacheMetadata: Key 3a74075a84649e9b590ae21f64f1581a is removed from metadata 2024.10.22 01:22:37.558466 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 48881e5b9926c6a1915f4232f81cbc42 for removal 2024.10.22 01:22:37.664742 [ 1476 ] {} CacheMetadata: Key 48881e5b9926c6a1915f4232f81cbc42 is removed from metadata 2024.10.22 01:22:37.665303 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 060cf912d889d2cfb339a84b70f1bdcf, offset: 0, size: 199326 2024.10.22 01:22:37.669317 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 060cf912d889d2cfb339a84b70f1bdcf for removal 2024.10.22 01:22:37.671353 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9c09eb3f7bc180ba072b93719d282747, offset: 0, size: 152 2024.10.22 01:22:37.674926 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9c09eb3f7bc180ba072b93719d282747 for removal 2024.10.22 01:22:37.679043 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 26e836a289e24534b2ce9c925e2b3cce, offset: 0, size: 459400 2024.10.22 01:22:37.683220 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 26e836a289e24534b2ce9c925e2b3cce for removal 2024.10.22 01:22:37.686526 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3642fed9c86db633b0e3fe00e145fd80, offset: 0, size: 157 2024.10.22 01:22:37.690167 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3642fed9c86db633b0e3fe00e145fd80 for removal 2024.10.22 01:22:37.696529 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b81513ac77c7222fd36dd7ce09c17e3e, offset: 0, size: 661 2024.10.22 01:22:37.703415 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b81513ac77c7222fd36dd7ce09c17e3e for removal 2024.10.22 01:22:37.758813 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 703441 2024.10.22 01:22:37.769949 [ 1476 ] {} CacheMetadata: Key 3642fed9c86db633b0e3fe00e145fd80 is removed from metadata 2024.10.22 01:22:37.774195 [ 1476 ] {} CacheMetadata: Key 26e836a289e24534b2ce9c925e2b3cce is removed from metadata 2024.10.22 01:22:37.835159 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_789_789_0 to 202410_802_802_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:37.870646 [ 1272 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_796_404 to 202410_797_797_0 2024.10.22 01:22:37.875382 [ 1272 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.84 MiB using storage policy from min volume index 0 2024.10.22 01:22:37.877559 [ 1272 ] {} DiskLocal: Reserved 1.84 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:37.877829 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 51590 2024.10.22 01:22:37.880313 [ 1476 ] {} CacheMetadata: Key 9c09eb3f7bc180ba072b93719d282747 is removed from metadata 2024.10.22 01:22:37.886206 [ 1476 ] {} CacheMetadata: Key b81513ac77c7222fd36dd7ce09c17e3e is removed from metadata 2024.10.22 01:22:37.899492 [ 1476 ] {} CacheMetadata: Key 060cf912d889d2cfb339a84b70f1bdcf is removed from metadata 2024.10.22 01:22:37.915697 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_797_405} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_796_404 to 202410_797_797_0 into Compact with storage Full 2024.10.22 01:22:37.948904 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 7 file segments (while reserving 1048576 bytes for cf1b07f1e944b97ddbefdf9020f867ea:0) 2024.10.22 01:22:37.952540 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 09fff66073688a8c95ab8b1df6521cc0, offset: 0, size: 915037 2024.10.22 01:22:37.957362 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 09fff66073688a8c95ab8b1df6521cc0 for removal 2024.10.22 01:22:37.960402 [ 1476 ] {} CacheMetadata: Key 09fff66073688a8c95ab8b1df6521cc0 is removed from metadata 2024.10.22 01:22:37.960910 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c9c0e7be8de351b86b530ef57b938026, offset: 0, size: 195 2024.10.22 01:22:37.963904 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c9c0e7be8de351b86b530ef57b938026 for removal 2024.10.22 01:22:37.966318 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fda03349e07eec4f430125b25c313a00, offset: 0, size: 1332 2024.10.22 01:22:37.966427 [ 1476 ] {} CacheMetadata: Key c9c0e7be8de351b86b530ef57b938026 is removed from metadata 2024.10.22 01:22:37.968329 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fda03349e07eec4f430125b25c313a00 for removal 2024.10.22 01:22:37.969633 [ 1476 ] {} CacheMetadata: Key fda03349e07eec4f430125b25c313a00 is removed from metadata 2024.10.22 01:22:37.969779 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0e37d5d6032f072b4fec3583830e584f, offset: 0, size: 159 2024.10.22 01:22:37.971450 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0e37d5d6032f072b4fec3583830e584f for removal 2024.10.22 01:22:37.974991 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 71cafe3473a24f7f54cd69925f09ba1c, offset: 0, size: 530 2024.10.22 01:22:37.975638 [ 1476 ] {} CacheMetadata: Key 0e37d5d6032f072b4fec3583830e584f is removed from metadata 2024.10.22 01:22:37.978764 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 71cafe3473a24f7f54cd69925f09ba1c for removal 2024.10.22 01:22:37.981269 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2c66bebc3429312b7a661c4101f672b9, offset: 0, size: 266086 2024.10.22 01:22:37.982117 [ 1476 ] {} CacheMetadata: Key 71cafe3473a24f7f54cd69925f09ba1c is removed from metadata 2024.10.22 01:22:37.984187 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2c66bebc3429312b7a661c4101f672b9 for removal 2024.10.22 01:22:37.987226 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b451ade25045301bb5800095a61a0e34, offset: 0, size: 154 2024.10.22 01:22:37.989124 [ 1476 ] {} CacheMetadata: Key 2c66bebc3429312b7a661c4101f672b9 is removed from metadata 2024.10.22 01:22:37.989742 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b451ade25045301bb5800095a61a0e34 for removal 2024.10.22 01:22:37.997291 [ 1476 ] {} CacheMetadata: Key b451ade25045301bb5800095a61a0e34 is removed from metadata 2024.10.22 01:22:38.103709 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 1048576 bytes for cf1b07f1e944b97ddbefdf9020f867ea:0) 2024.10.22 01:22:38.107609 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6be48307c7a379820e97892041419370, offset: 0, size: 999883 2024.10.22 01:22:38.112158 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6be48307c7a379820e97892041419370 for removal 2024.10.22 01:22:38.115447 [ 1476 ] {} CacheMetadata: Key 6be48307c7a379820e97892041419370 is removed from metadata 2024.10.22 01:22:38.137787 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_797_405} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:38.151618 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_797_405} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_796_404, total 51245 rows starting from the beginning of the part 2024.10.22 01:22:38.268893 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_797_405} MergeTreeSequentialSource: Reading 2 marks from part 202410_797_797_0, total 103 rows starting from the beginning of the part 2024.10.22 01:22:38.525807 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 210ff3222c5ca5050765a2bd51288b84:0) 2024.10.22 01:22:38.528358 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6bbce8a171e312eaa7927062c2f09a0b, offset: 0, size: 1208871 2024.10.22 01:22:38.538607 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6bbce8a171e312eaa7927062c2f09a0b for removal 2024.10.22 01:22:38.544872 [ 1476 ] {} CacheMetadata: Key 6bbce8a171e312eaa7927062c2f09a0b is removed from metadata 2024.10.22 01:22:38.545470 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d76f894d2121bc836425ce504435f473, offset: 0, size: 198 2024.10.22 01:22:38.551245 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d76f894d2121bc836425ce504435f473 for removal 2024.10.22 01:22:38.559741 [ 1476 ] {} CacheMetadata: Key d76f894d2121bc836425ce504435f473 is removed from metadata 2024.10.22 01:22:38.605442 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for cf1b07f1e944b97ddbefdf9020f867ea:0) 2024.10.22 01:22:38.608980 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a8c3bb6554d52522266c7b81c1ceec82, offset: 0, size: 469264 2024.10.22 01:22:38.613726 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a8c3bb6554d52522266c7b81c1ceec82 for removal 2024.10.22 01:22:38.616126 [ 1476 ] {} CacheMetadata: Key a8c3bb6554d52522266c7b81c1ceec82 is removed from metadata 2024.10.22 01:22:38.627206 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7b13bc02a4fda1e4beaf55555391129e, offset: 0, size: 158 2024.10.22 01:22:38.630887 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7b13bc02a4fda1e4beaf55555391129e for removal 2024.10.22 01:22:38.639933 [ 1476 ] {} CacheMetadata: Key 7b13bc02a4fda1e4beaf55555391129e is removed from metadata 2024.10.22 01:22:38.733143 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 4 file segments (while reserving 298294 bytes for 210ff3222c5ca5050765a2bd51288b84:0) 2024.10.22 01:22:38.741514 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: b1661c92e75c7982eb1cf8c95c4da5a2, offset: 0, size: 114541 2024.10.22 01:22:38.746907 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key b1661c92e75c7982eb1cf8c95c4da5a2 for removal 2024.10.22 01:22:38.752537 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 8789b975250e6fa3003321e8fb695a8d, offset: 0, size: 693 2024.10.22 01:22:38.757118 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 8789b975250e6fa3003321e8fb695a8d for removal 2024.10.22 01:22:38.762297 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: 6a2c5d5778e4af3f611d8881ba1a400b, offset: 0, size: 1740305 2024.10.22 01:22:38.765743 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key 6a2c5d5778e4af3f611d8881ba1a400b for removal 2024.10.22 01:22:38.769088 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: cd2c9111548e3b2bdd1e086d052ff065, offset: 0, size: 160 2024.10.22 01:22:38.852409 [ 1476 ] {} CacheMetadata: Key b1661c92e75c7982eb1cf8c95c4da5a2 is removed from metadata 2024.10.22 01:22:38.860148 [ 1476 ] {} CacheMetadata: Key 6a2c5d5778e4af3f611d8881ba1a400b is removed from metadata 2024.10.22 01:22:38.872162 [ 1476 ] {} CacheMetadata: Key 8789b975250e6fa3003321e8fb695a8d is removed from metadata 2024.10.22 01:22:38.887291 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key cd2c9111548e3b2bdd1e086d052ff065 for removal 2024.10.22 01:22:38.891120 [ 1476 ] {} CacheMetadata: Key cd2c9111548e3b2bdd1e086d052ff065 is removed from metadata 2024.10.22 01:22:38.947056 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:38.950710 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:22:38.957148 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_698_316, 202410_699_699_0] 2024.10.22 01:22:38.960344 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_698_316, 202410_699_699_0] 2024.10.22 01:22:38.971280 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_698_316, 202410_699_699_0] 2024.10.22 01:22:39.082023 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_447_447_0 (state Temporary) 2024.10.22 01:22:39.268319 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for cf1b07f1e944b97ddbefdf9020f867ea:0) 2024.10.22 01:22:39.271462 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c30984cb930c9d84b49870a9566d0beb, offset: 0, size: 1863307 2024.10.22 01:22:39.277505 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_447_447_0 to 202410_454_454_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:39.278892 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c30984cb930c9d84b49870a9566d0beb for removal 2024.10.22 01:22:39.282388 [ 1476 ] {} CacheMetadata: Key c30984cb930c9d84b49870a9566d0beb is removed from metadata 2024.10.22 01:22:39.282427 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a6dd7bc863e2fb205132b22b67dbb770, offset: 0, size: 697 2024.10.22 01:22:39.288129 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a6dd7bc863e2fb205132b22b67dbb770 for removal 2024.10.22 01:22:39.292458 [ 1476 ] {} CacheMetadata: Key a6dd7bc863e2fb205132b22b67dbb770 is removed from metadata 2024.10.22 01:22:39.317164 [ 1226 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_449_90 to 202410_454_454_0 2024.10.22 01:22:39.325077 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 703441 2024.10.22 01:22:39.341062 [ 1226 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:39.350806 [ 1226 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:39.371120 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_449_90 to 202410_454_454_0 into Wide with storage Full 2024.10.22 01:22:39.409651 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:39.415741 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} MergeTreeSequentialSource: Reading 88 marks from part 202410_1_449_90, total 708561 rows starting from the beginning of the part 2024.10.22 01:22:39.512176 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 7213 2024.10.22 01:22:39.551256 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_450_450_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:22:39.573526 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_451_451_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:22:39.612075 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_452_452_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:22:39.639351 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_453_453_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:22:39.649013 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} FileCache(s3_cache): Will evict 2 file segments (while reserving 745494 bytes for 9159519cbcc401b228c41ce1f782db00:0) 2024.10.22 01:22:39.649728 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:39.653424 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: d27d7e01cc034014964b0ad12436d011, offset: 0, size: 958171 2024.10.22 01:22:39.656629 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key d27d7e01cc034014964b0ad12436d011 for removal 2024.10.22 01:22:39.660256 [ 1476 ] {} CacheMetadata: Key d27d7e01cc034014964b0ad12436d011 is removed from metadata 2024.10.22 01:22:39.660509 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Remove from cache. Key: f59af70537803bcb53ad747903fa6d30, offset: 0, size: 677 2024.10.22 01:22:39.669041 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_454_454_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:22:39.669715 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.96 GiB, free memory in arenas 5.01 MiB, will set to 1.14 GiB (RSS), difference: -3.56 MiB 2024.10.22 01:22:39.755145 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:39.763492 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} CacheMetadata: Submitting key f59af70537803bcb53ad747903fa6d30 for removal 2024.10.22 01:22:39.788485 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_466_466_0 (state Temporary) 2024.10.22 01:22:39.825739 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for cf1b07f1e944b97ddbefdf9020f867ea:0) 2024.10.22 01:22:39.849402 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: cdd6d67d2ea9bee81a92e4b0a93e3247, offset: 0, size: 10745530 2024.10.22 01:22:39.859924 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key cdd6d67d2ea9bee81a92e4b0a93e3247 for removal 2024.10.22 01:22:39.862882 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 08d4eb2e4e417a9c874a3ff1744fc113, offset: 0, size: 1028 2024.10.22 01:22:39.865398 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 08d4eb2e4e417a9c874a3ff1744fc113 for removal 2024.10.22 01:22:39.765810 [ 1476 ] {} CacheMetadata: Key f59af70537803bcb53ad747903fa6d30 is removed from metadata 2024.10.22 01:22:39.872138 [ 1476 ] {} CacheMetadata: Key 08d4eb2e4e417a9c874a3ff1744fc113 is removed from metadata 2024.10.22 01:22:39.882307 [ 1476 ] {} CacheMetadata: Key cdd6d67d2ea9bee81a92e4b0a93e3247 is removed from metadata 2024.10.22 01:22:40.058947 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_466_466_0 to 202410_479_479_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:40.100269 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7213 2024.10.22 01:22:40.197729 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:22:40.201295 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:22:40.475781 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 42ced4d444b695630fece2b8964cdb67, offset: 0, size: 0 2024.10.22 01:22:40.477422 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 42ced4d444b695630fece2b8964cdb67 for removal 2024.10.22 01:22:40.479198 [ 1476 ] {} CacheMetadata: Key 42ced4d444b695630fece2b8964cdb67 is removed from metadata 2024.10.22 01:22:40.601805 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zeb/wwjqixwulqawwttreuujnfvwidhfb. 2024.10.22 01:22:40.746904 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 98500ba549094cfe8bae5e44d5fa94ec, offset: 0, size: 0 2024.10.22 01:22:40.749159 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 98500ba549094cfe8bae5e44d5fa94ec for removal 2024.10.22 01:22:40.753254 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zzy/lepnxgxrppvddohquojbkuwytxzjj. 2024.10.22 01:22:40.852566 [ 1476 ] {} CacheMetadata: Key 98500ba549094cfe8bae5e44d5fa94ec is removed from metadata 2024.10.22 01:22:41.574084 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zzy/lepnxgxrppvddohquojbkuwytxzjj. 2024.10.22 01:22:41.588740 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_insert_201403_24_24_0 to 201403_24_24_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:41.620906 [ 1448 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zeb/wwjqixwulqawwttreuujnfvwidhfb. 2024.10.22 01:22:41.650795 [ 1247 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937) (MergerMutator): Selected 8 parts from 201403_17_17_0 to 201403_24_24_0 2024.10.22 01:22:41.667896 [ 1247 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Trying to reserve 403.92 MiB using storage policy from min volume index 0 2024.10.22 01:22:41.677905 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.96 GiB, free memory in arenas 5.81 MiB, will set to 1.15 GiB (RSS), difference: -1.16 MiB 2024.10.22 01:22:41.725674 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:22:41.727407 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTask::PrepareStage: Merging 8 parts: from 201403_17_17_0 to 201403_24_24_0 into Wide with storage Full 2024.10.22 01:22:41.866534 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 52639 2024.10.22 01:22:41.915619 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 193 entries to flush up to offset 51783 2024.10.22 01:22:41.960088 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} executeQuery: Read 8873898 rows, 7.88 GiB in 3750.240882 sec., 2366.220805333325 rows/sec., 2.15 MiB/sec. 2024.10.22 01:22:42.102537 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_401_179, 202410_402_402_0] 2024.10.22 01:22:42.174805 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_797_405} MergeTask::MergeProjectionsStage: Merge sorted 51348 rows, containing 13 columns (13 merged, 0 gathered) in 4.27064801 sec., 12023.468073174216 rows/sec., 1.93 MiB/sec. 2024.10.22 01:22:42.206060 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:42.210887 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_401_179, 202410_402_402_0] 2024.10.22 01:22:42.214648 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:42.236514 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_401_179, 202410_402_402_0] 2024.10.22 01:22:42.257335 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_797_405} MergedBlockOutputStream: filled checksums 202410_1_797_405 (state Temporary) 2024.10.22 01:22:42.270865 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_458_458_0 (state Temporary) 2024.10.22 01:22:42.424717 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_458_458_0 to 202410_464_464_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:42.485069 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 52639 2024.10.22 01:22:42.580047 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 395 2024.10.22 01:22:42.629201 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_797_405} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_797_405 to 202410_1_797_405 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:42.639587 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_797_405} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_796_404, 202410_797_797_0] -> 202410_1_797_405 2024.10.22 01:22:42.646639 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 1048576 bytes for 42ced4d444b695630fece2b8964cdb67:0) 2024.10.22 01:22:42.650819 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b5bd22f0438a36b3472fc54e08b3eb22, offset: 0, size: 521 2024.10.22 01:22:42.655487 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b5bd22f0438a36b3472fc54e08b3eb22 for removal 2024.10.22 01:22:42.659795 [ 1476 ] {} CacheMetadata: Key b5bd22f0438a36b3472fc54e08b3eb22 is removed from metadata 2024.10.22 01:22:42.661741 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e52189212a55d7efadf77f2c6b2d8024, offset: 0, size: 540553 2024.10.22 01:22:42.664592 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e52189212a55d7efadf77f2c6b2d8024 for removal 2024.10.22 01:22:42.668933 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:22:42.667165 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9fe04f7c0cf72ff745a95c4749293eb8, offset: 0, size: 463523 2024.10.22 01:22:42.684682 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_94_94_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:22:42.691439 [ 1476 ] {} CacheMetadata: Key e52189212a55d7efadf77f2c6b2d8024 is removed from metadata 2024.10.22 01:22:42.678929 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9fe04f7c0cf72ff745a95c4749293eb8 for removal 2024.10.22 01:22:42.703854 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 47d08f88db69e0c216f876088fcef6b1, offset: 0, size: 522 2024.10.22 01:22:42.705831 [ 1476 ] {} CacheMetadata: Key 9fe04f7c0cf72ff745a95c4749293eb8 is removed from metadata 2024.10.22 01:22:42.711436 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 47d08f88db69e0c216f876088fcef6b1 for removal 2024.10.22 01:22:42.721461 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b1abbf154939d9a831e81d40f5c3be75, offset: 0, size: 4 2024.10.22 01:22:42.724660 [ 1476 ] {} CacheMetadata: Key 47d08f88db69e0c216f876088fcef6b1 is removed from metadata 2024.10.22 01:22:42.727938 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b1abbf154939d9a831e81d40f5c3be75 for removal 2024.10.22 01:22:42.742508 [ 1476 ] {} CacheMetadata: Key b1abbf154939d9a831e81d40f5c3be75 is removed from metadata 2024.10.22 01:22:42.924750 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:42.927749 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:42.960763 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:42.964825 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:42.978507 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 42ced4d444b695630fece2b8964cdb67:0) 2024.10.22 01:22:42.980998 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c840e110eff0a55c1a489e1390710c8e, offset: 0, size: 1321100 2024.10.22 01:22:42.988202 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c840e110eff0a55c1a489e1390710c8e for removal 2024.10.22 01:22:42.993378 [ 1476 ] {} CacheMetadata: Key c840e110eff0a55c1a489e1390710c8e is removed from metadata 2024.10.22 01:22:43.032100 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Vertical 2024.10.22 01:22:43.088191 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_377_377_0 (state Temporary) 2024.10.22 01:22:42.993396 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: dcf834c53af57100324664f3152b7442, offset: 0, size: 883 2024.10.22 01:22:43.127270 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key dcf834c53af57100324664f3152b7442 for removal 2024.10.22 01:22:43.131347 [ 1476 ] {} CacheMetadata: Key dcf834c53af57100324664f3152b7442 is removed from metadata 2024.10.22 01:22:43.132387 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2d64afe9d4bc84dcec8d5ac421197cce, offset: 0, size: 852584 2024.10.22 01:22:43.141034 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2d64afe9d4bc84dcec8d5ac421197cce for removal 2024.10.22 01:22:43.150336 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2cbf60db6114701d15b2c205003e415b, offset: 0, size: 875 2024.10.22 01:22:43.155166 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2cbf60db6114701d15b2c205003e415b for removal 2024.10.22 01:22:43.158752 [ 1476 ] {} CacheMetadata: Key 2cbf60db6114701d15b2c205003e415b is removed from metadata 2024.10.22 01:22:43.169024 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part 2024.10.22 01:22:43.172523 [ 1476 ] {} CacheMetadata: Key 2d64afe9d4bc84dcec8d5ac421197cce is removed from metadata 2024.10.22 01:22:43.089840 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_790_790_0 (state Temporary) 2024.10.22 01:22:43.215627 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part 2024.10.22 01:22:43.273405 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part 2024.10.22 01:22:43.282341 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_790_790_0 to 202410_803_803_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:43.322854 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_797_405 to 202410_798_798_0 2024.10.22 01:22:43.339377 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.84 MiB using storage policy from min volume index 0 2024.10.22 01:22:43.347157 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part 2024.10.22 01:22:43.355098 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_377_377_0 to 202410_387_387_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:43.370991 [ 1253 ] {} DiskLocal: Reserved 1.84 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:43.415142 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_798_406} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_797_405 to 202410_798_798_0 into Compact with storage Full 2024.10.22 01:22:43.448485 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 51783 2024.10.22 01:22:43.452200 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part 2024.10.22 01:22:43.464084 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 395 2024.10.22 01:22:43.474259 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_798_406} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:43.499453 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_798_406} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_797_405, total 51348 rows starting from the beginning of the part 2024.10.22 01:22:43.499968 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part 2024.10.22 01:22:43.523577 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 42ced4d444b695630fece2b8964cdb67:0) 2024.10.22 01:22:43.553321 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_798_406} MergeTreeSequentialSource: Reading 2 marks from part 202410_798_798_0, total 103 rows starting from the beginning of the part 2024.10.22 01:22:43.555409 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 10b8e844f8c4120ac53eb6742d593379, offset: 0, size: 524 2024.10.22 01:22:43.558481 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part 2024.10.22 01:22:43.558937 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 10b8e844f8c4120ac53eb6742d593379 for removal 2024.10.22 01:22:43.567663 [ 1476 ] {} CacheMetadata: Key 10b8e844f8c4120ac53eb6742d593379 is removed from metadata 2024.10.22 01:22:43.568722 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0fa096cd89a6503ffa2847e3a956777e, offset: 0, size: 374482 2024.10.22 01:22:43.573408 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0fa096cd89a6503ffa2847e3a956777e for removal 2024.10.22 01:22:43.576373 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a51bbb62503d58ba797de11bf9336705, offset: 0, size: 717780 2024.10.22 01:22:43.579758 [ 1476 ] {} CacheMetadata: Key 0fa096cd89a6503ffa2847e3a956777e is removed from metadata 2024.10.22 01:22:43.580361 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a51bbb62503d58ba797de11bf9336705 for removal 2024.10.22 01:22:43.587018 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c38c4000ea3ae9313145d9a85489cb69, offset: 0, size: 516 2024.10.22 01:22:43.591034 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c38c4000ea3ae9313145d9a85489cb69 for removal 2024.10.22 01:22:43.595181 [ 1476 ] {} CacheMetadata: Key a51bbb62503d58ba797de11bf9336705 is removed from metadata 2024.10.22 01:22:43.609613 [ 1476 ] {} CacheMetadata: Key c38c4000ea3ae9313145d9a85489cb69 is removed from metadata 2024.10.22 01:22:43.671586 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.11 GiB, peak 1.96 GiB, free memory in arenas 12.79 MiB, will set to 1.13 GiB (RSS), difference: 26.70 MiB 2024.10.22 01:22:43.705653 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part 2024.10.22 01:22:43.893748 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pms/facyzhdlagmrxdkcroaqoessnrueu (LogSeriesLimiter: on interval from 2024-10-22 01:22:27 to 2024-10-22 01:22:43 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:22:43.959426 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} MemoryTracker: Peak memory usage (for query): 1.96 GiB. 2024.10.22 01:22:43.960797 [ 1039 ] {d27f974d-16ec-4d21-bb2c-d65933eaeee0} TCPHandler: Processed in 3752.342170657 sec. 2024.10.22 01:22:43.997481 [ 1497 ] {} SystemLog (system.query_log): Flushing system log, 1 entries to flush up to offset 16 2024.10.22 01:22:44.057069 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d6786e6264b5778631c0197241e4a1ae, offset: 0, size: 0 2024.10.22 01:22:44.060471 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d6786e6264b5778631c0197241e4a1ae for removal 2024.10.22 01:22:44.063149 [ 1476 ] {} CacheMetadata: Key d6786e6264b5778631c0197241e4a1ae is removed from metadata 2024.10.22 01:22:44.149839 [ 1039 ] {} TCPHandler: Done processing connection. 2024.10.22 01:22:44.174907 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b7e8463e00a0cf903e425840f62be3cc, offset: 0, size: 0 2024.10.22 01:22:44.182834 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b7e8463e00a0cf903e425840f62be3cc for removal 2024.10.22 01:22:44.187684 [ 1476 ] {} CacheMetadata: Key b7e8463e00a0cf903e425840f62be3cc is removed from metadata 2024.10.22 01:22:44.187004 [ 1039 ] {} TCP-Session: 026b32ed-8060-46ad-829a-7911ff45a875 Logout, user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 01:22:44.492505 [ 1039 ] {} TCPHandlerFactory: TCP Request. Address: [::1]:54050 2024.10.22 01:22:44.505687 [ 1039 ] {} TCPHandler: Connected ClickHouse client version 23.8.0, revision: 54465, user: default. 2024.10.22 01:22:44.515455 [ 1039 ] {} TCP-Session: e1573d2d-c521-41dc-a78e-4f5ec5e2148b Authenticating user 'default' from [::1]:54050 2024.10.22 01:22:44.523463 [ 1039 ] {} TCP-Session: e1573d2d-c521-41dc-a78e-4f5ec5e2148b Authenticated with global context as user 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 01:22:44.525402 [ 1039 ] {} TCP-Session: e1573d2d-c521-41dc-a78e-4f5ec5e2148b Creating session context with user_id: 94309d50-4f52-5250-31bd-74fecac179db 2024.10.22 01:22:44.567589 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 761257 bytes for b0cafada75e4e52ee202182c84a45b4c:0) 2024.10.22 01:22:44.570081 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2f989c75d76c902f630aebb13ca5ec17, offset: 0, size: 3500979 2024.10.22 01:22:44.576324 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 2f989c75d76c902f630aebb13ca5ec17 for removal 2024.10.22 01:22:44.576932 [ 1039 ] {} ContextAccess (default): Settings: readonly = 0, allow_ddl = true, allow_introspection_functions = false 2024.10.22 01:22:44.578579 [ 1039 ] {} ContextAccess (default): List of all grants: GRANT ALL ON *.* WITH GRANT OPTION 2024.10.22 01:22:44.578750 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b8cfd903c08567a61d5135c1978e670c, offset: 0, size: 889 2024.10.22 01:22:44.578741 [ 1476 ] {} CacheMetadata: Key 2f989c75d76c902f630aebb13ca5ec17 is removed from metadata 2024.10.22 01:22:44.579623 [ 1039 ] {} ContextAccess (default): List of all grants including implicit: GRANT ALL ON *.* WITH GRANT OPTION 2024.10.22 01:22:44.582780 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b8cfd903c08567a61d5135c1978e670c for removal 2024.10.22 01:22:44.584316 [ 1476 ] {} CacheMetadata: Key b8cfd903c08567a61d5135c1978e670c is removed from metadata 2024.10.22 01:22:44.676591 [ 1684 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/loo/mqhulmrkodkpoqdwopopewriwlxpd, size 761257 2024.10.22 01:22:44.713395 [ 1039 ] {} TCP-Session: e1573d2d-c521-41dc-a78e-4f5ec5e2148b Creating query context from session context, user_id: 94309d50-4f52-5250-31bd-74fecac179db, parent context user: default 2024.10.22 01:22:44.761634 [ 1039 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} executeQuery: (from [::1]:54050) INSERT INTO test.hits SELECT * FROM datasets.hits_v1 SETTINGS enable_filesystem_cache_on_write_operations=0 (stage: Complete) 2024.10.22 01:22:44.788737 [ 1039 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} ContextAccess (default): Access granted: INSERT(WatchID, JavaEnable, Title, GoodEvent, EventTime, EventDate, CounterID, ClientIP, ClientIP6, RegionID, UserID, CounterClass, OS, UserAgent, URL, Referer, URLDomain, RefererDomain, Refresh, IsRobot, RefererCategories, URLCategories, URLRegions, RefererRegions, ResolutionWidth, ResolutionHeight, ResolutionDepth, FlashMajor, FlashMinor, FlashMinor2, NetMajor, NetMinor, UserAgentMajor, UserAgentMinor, CookieEnable, JavascriptEnable, IsMobile, MobilePhone, MobilePhoneModel, Params, IPNetworkID, TraficSourceID, SearchEngineID, SearchPhrase, AdvEngineID, IsArtifical, WindowClientWidth, WindowClientHeight, ClientTimeZone, ClientEventTime, SilverlightVersion1, SilverlightVersion2, SilverlightVersion3, SilverlightVersion4, PageCharset, CodeVersion, IsLink, IsDownload, IsNotBounce, FUniqID, HID, IsOldCounter, IsEvent, IsParameter, DontCountHits, WithHash, HitColor, UTCEventTime, Age, Sex, Income, Interests, Robotness, GeneralInterests, RemoteIP, RemoteIP6, WindowName, OpenerName, HistoryLength, BrowserLanguage, BrowserCountry, SocialNetwork, SocialAction, HTTPError, SendTiming, DNSTiming, ConnectTiming, ResponseStartTiming, ResponseEndTiming, FetchTiming, RedirectTiming, DOMInteractiveTiming, DOMContentLoadedTiming, DOMCompleteTiming, LoadEventStartTiming, LoadEventEndTiming, NSToDOMContentLoadedTiming, FirstPaintTiming, RedirectCount, SocialSourceNetworkID, SocialSourcePage, ParamPrice, ParamOrderID, ParamCurrency, ParamCurrencyID, GoalsReached, OpenstatServiceName, OpenstatCampaignID, OpenstatAdID, OpenstatSourceID, UTMSource, UTMMedium, UTMCampaign, UTMContent, UTMTerm, FromTag, HasGCLID, RefererHash, URLHash, CLID, YCLID, ShareService, ShareURL, ShareTitle, `ParsedParams.Key1`, `ParsedParams.Key2`, `ParsedParams.Key3`, `ParsedParams.Key4`, `ParsedParams.Key5`, `ParsedParams.ValueDouble`, IslandID, RequestNum, RequestTry) ON test.hits 2024.10.22 01:22:44.813498 [ 1497 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:44.819981 [ 1497 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:44.841046 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Income 2024.10.22 01:22:44.840398 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8ebd86a27e1c213acc0247bdccc32e8f, offset: 0, size: 0 2024.10.22 01:22:44.843575 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8ebd86a27e1c213acc0247bdccc32e8f for removal 2024.10.22 01:22:44.848553 [ 1476 ] {} CacheMetadata: Key 8ebd86a27e1c213acc0247bdccc32e8f is removed from metadata 2024.10.22 01:22:44.864139 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Income 2024.10.22 01:22:44.882067 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Income 2024.10.22 01:22:44.886180 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9e18c84d264447d6bcf0454caac20222, offset: 0, size: 0 2024.10.22 01:22:44.888267 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 9e18c84d264447d6bcf0454caac20222 for removal 2024.10.22 01:22:44.894368 [ 1476 ] {} CacheMetadata: Key 9e18c84d264447d6bcf0454caac20222 is removed from metadata 2024.10.22 01:22:44.935229 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Income 2024.10.22 01:22:44.952915 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 24 2024.10.22 01:22:44.975000 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Income 2024.10.22 01:22:45.011660 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Income 2024.10.22 01:22:45.044592 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 879bb8d688a49081b459de19c8a3fa04, offset: 0, size: 0 2024.10.22 01:22:45.046480 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 879bb8d688a49081b459de19c8a3fa04 for removal 2024.10.22 01:22:45.047842 [ 1476 ] {} CacheMetadata: Key 879bb8d688a49081b459de19c8a3fa04 is removed from metadata 2024.10.22 01:22:45.058679 [ 1157 ] {} DatabaseCatalog: Execution took 280 ms. 2024.10.22 01:22:45.128441 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_94_94_0 to 202410_97_97_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:45.155485 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Income 2024.10.22 01:22:45.179359 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Income 2024.10.22 01:22:45.221955 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 521bac2dd3f3529ee5ad4fb46c019204, offset: 0, size: 0 2024.10.22 01:22:45.223811 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 521bac2dd3f3529ee5ad4fb46c019204 for removal 2024.10.22 01:22:45.227581 [ 1476 ] {} CacheMetadata: Key 521bac2dd3f3529ee5ad4fb46c019204 is removed from metadata 2024.10.22 01:22:45.231985 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Income 2024.10.22 01:22:45.299475 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_699_317, 202410_700_700_0] 2024.10.22 01:22:45.303881 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_699_317, 202410_700_700_0] 2024.10.22 01:22:45.318125 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_699_317, 202410_700_700_0] 2024.10.22 01:22:45.337249 [ 1497 ] {} MergedBlockOutputStream: filled checksums 202410_4_4_0 (state Temporary) 2024.10.22 01:22:45.387449 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3c6c279f279f937e4d2a916a54298397, offset: 0, size: 0 2024.10.22 01:22:45.390102 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 3c6c279f279f937e4d2a916a54298397 for removal 2024.10.22 01:22:45.399468 [ 1476 ] {} CacheMetadata: Key 3c6c279f279f937e4d2a916a54298397 is removed from metadata 2024.10.22 01:22:45.551708 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.96 GiB, free memory in arenas 11.88 MiB, will set to 1.12 GiB (RSS), difference: -29.64 MiB 2024.10.22 01:22:45.559784 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 608eb24272c0637d0374933f76aa4815, offset: 0, size: 0 2024.10.22 01:22:45.562361 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 608eb24272c0637d0374933f76aa4815 for removal 2024.10.22 01:22:45.570193 [ 1476 ] {} CacheMetadata: Key 608eb24272c0637d0374933f76aa4815 is removed from metadata 2024.10.22 01:22:45.611421 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_454_91/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:22:45.616219 [ 1497 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Renaming temporary part tmp_insert_202410_4_4_0 to 202410_8_8_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:45.716476 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b43a6142ba6a4869b501f59394736e83, offset: 0, size: 0 2024.10.22 01:22:45.718668 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b43a6142ba6a4869b501f59394736e83 for removal 2024.10.22 01:22:45.724402 [ 1476 ] {} CacheMetadata: Key b43a6142ba6a4869b501f59394736e83 is removed from metadata 2024.10.22 01:22:45.739107 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:45.745368 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:45.785553 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_24_24_0 (state Temporary) 2024.10.22 01:22:45.872945 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8c43559654b20dfe5c3459c28e0284a3, offset: 0, size: 0 2024.10.22 01:22:45.895734 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8c43559654b20dfe5c3459c28e0284a3 for removal 2024.10.22 01:22:45.903964 [ 1476 ] {} CacheMetadata: Key 8c43559654b20dfe5c3459c28e0284a3 is removed from metadata 2024.10.22 01:22:45.963139 [ 1497 ] {} SystemLog (system.query_log): Flushed system log up to offset 16 2024.10.22 01:22:45.978701 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ba161c6e6eaad78fc7ea4ffb8160e7bf, offset: 0, size: 0 2024.10.22 01:22:45.983696 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ba161c6e6eaad78fc7ea4ffb8160e7bf for removal 2024.10.22 01:22:45.992167 [ 1476 ] {} CacheMetadata: Key ba161c6e6eaad78fc7ea4ffb8160e7bf is removed from metadata 2024.10.22 01:22:46.027055 [ 1039 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} ContextAccess (default): Access granted: SELECT(WatchID, JavaEnable, Title, GoodEvent, EventTime, EventDate, CounterID, ClientIP, ClientIP6, RegionID, UserID, CounterClass, OS, UserAgent, URL, Referer, URLDomain, RefererDomain, Refresh, IsRobot, RefererCategories, URLCategories, URLRegions, RefererRegions, ResolutionWidth, ResolutionHeight, ResolutionDepth, FlashMajor, FlashMinor, FlashMinor2, NetMajor, NetMinor, UserAgentMajor, UserAgentMinor, CookieEnable, JavascriptEnable, IsMobile, MobilePhone, MobilePhoneModel, Params, IPNetworkID, TraficSourceID, SearchEngineID, SearchPhrase, AdvEngineID, IsArtifical, WindowClientWidth, WindowClientHeight, ClientTimeZone, ClientEventTime, SilverlightVersion1, SilverlightVersion2, SilverlightVersion3, SilverlightVersion4, PageCharset, CodeVersion, IsLink, IsDownload, IsNotBounce, FUniqID, HID, IsOldCounter, IsEvent, IsParameter, DontCountHits, WithHash, HitColor, UTCEventTime, Age, Sex, Income, Interests, Robotness, GeneralInterests, RemoteIP, RemoteIP6, WindowName, OpenerName, HistoryLength, BrowserLanguage, BrowserCountry, SocialNetwork, SocialAction, HTTPError, SendTiming, DNSTiming, ConnectTiming, ResponseStartTiming, ResponseEndTiming, FetchTiming, RedirectTiming, DOMInteractiveTiming, DOMContentLoadedTiming, DOMCompleteTiming, LoadEventStartTiming, LoadEventEndTiming, NSToDOMContentLoadedTiming, FirstPaintTiming, RedirectCount, SocialSourceNetworkID, SocialSourcePage, ParamPrice, ParamOrderID, ParamCurrency, ParamCurrencyID, GoalsReached, OpenstatServiceName, OpenstatCampaignID, OpenstatAdID, OpenstatSourceID, UTMSource, UTMMedium, UTMCampaign, UTMContent, UTMTerm, FromTag, HasGCLID, RefererHash, URLHash, CLID, YCLID, ShareService, ShareURL, ShareTitle, `ParsedParams.Key1`, `ParsedParams.Key2`, `ParsedParams.Key3`, `ParsedParams.Key4`, `ParsedParams.Key5`, `ParsedParams.ValueDouble`, IslandID, RequestNum, RequestTry) ON datasets.hits_v1 2024.10.22 01:22:46.072485 [ 1039 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} InterpreterSelectQuery: FetchColumns -> Complete 2024.10.22 01:22:46.166734 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_24_24_0 to 202410_24_24_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:46.183844 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: a9f8bb3ac2ed3ac8cbd79988c2091b79, offset: 0, size: 0 2024.10.22 01:22:46.190405 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key a9f8bb3ac2ed3ac8cbd79988c2091b79 for removal 2024.10.22 01:22:46.198666 [ 1476 ] {} CacheMetadata: Key a9f8bb3ac2ed3ac8cbd79988c2091b79 is removed from metadata 2024.10.22 01:22:46.201540 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/flk/flxpahovvevadryxjfanrzkwqgonm. 2024.10.22 01:22:46.285225 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 24 2024.10.22 01:22:46.353104 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 705366 2024.10.22 01:22:46.362217 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 823981d07f7ed9fcffa776404e22c05f, offset: 0, size: 0 2024.10.22 01:22:46.364013 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 823981d07f7ed9fcffa776404e22c05f for removal 2024.10.22 01:22:46.365370 [ 1476 ] {} CacheMetadata: Key 823981d07f7ed9fcffa776404e22c05f is removed from metadata 2024.10.22 01:22:46.429376 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8da3f815aa237dae756fe3fea6143346, offset: 0, size: 0 2024.10.22 01:22:46.531793 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8da3f815aa237dae756fe3fea6143346 for removal 2024.10.22 01:22:46.535506 [ 1476 ] {} CacheMetadata: Key 8da3f815aa237dae756fe3fea6143346 is removed from metadata 2024.10.22 01:22:46.556850 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/flk/flxpahovvevadryxjfanrzkwqgonm. 2024.10.22 01:22:46.738651 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 77cc6d0c9f4805fd532ff7f0a62e6e79, offset: 0, size: 0 2024.10.22 01:22:46.745443 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 77cc6d0c9f4805fd532ff7f0a62e6e79 for removal 2024.10.22 01:22:46.750202 [ 1476 ] {} CacheMetadata: Key 77cc6d0c9f4805fd532ff7f0a62e6e79 is removed from metadata 2024.10.22 01:22:46.787162 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 542c4284f224bc7772d43f7b3b6caad6, offset: 0, size: 0 2024.10.22 01:22:46.788720 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 542c4284f224bc7772d43f7b3b6caad6 for removal 2024.10.22 01:22:46.791671 [ 1476 ] {} CacheMetadata: Key 542c4284f224bc7772d43f7b3b6caad6 is removed from metadata 2024.10.22 01:22:46.792630 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/cxp/qygcigsljdmdjogwvsdutxkgodrbd. 2024.10.22 01:22:46.819563 [ 1039 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} datasets.hits_v1 (SelectExecutor): Key condition: unknown 2024.10.22 01:22:46.821084 [ 1039 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} datasets.hits_v1 (SelectExecutor): MinMax index condition: unknown 2024.10.22 01:22:46.836631 [ 1039 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} datasets.hits_v1 (SelectExecutor): Selected 1/1 parts by partition key, 1 parts by primary key, 1084/1084 marks by primary key, 1084 marks to read from 1 ranges 2024.10.22 01:22:46.848698 [ 1039 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} datasets.hits_v1 (SelectExecutor): Spreading mark ranges among streams (default reading) 2024.10.22 01:22:46.871090 [ 1039 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergeTreeInOrderSelectProcessor: Reading 1 ranges in order from part 201403_10_18_2, approx. 8880128 rows starting from 0 2024.10.22 01:22:46.935198 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4833a1f6bb4b24df76ae3e892eea043c, offset: 0, size: 0 2024.10.22 01:22:46.944570 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 4833a1f6bb4b24df76ae3e892eea043c for removal 2024.10.22 01:22:46.951567 [ 1476 ] {} CacheMetadata: Key 4833a1f6bb4b24df76ae3e892eea043c is removed from metadata 2024.10.22 01:22:47.059836 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:47.064006 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:47.103248 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1fb457f097b161b0ca88f57433aa466e, offset: 0, size: 0 2024.10.22 01:22:47.111550 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1fb457f097b161b0ca88f57433aa466e for removal 2024.10.22 01:22:47.119231 [ 1476 ] {} CacheMetadata: Key 1fb457f097b161b0ca88f57433aa466e is removed from metadata 2024.10.22 01:22:47.129762 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_798_406} MergeTask::MergeProjectionsStage: Merge sorted 51451 rows, containing 13 columns (13 merged, 0 gathered) in 3.746902194 sec., 13731.61009710626 rows/sec., 2.20 MiB/sec. 2024.10.22 01:22:47.147363 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/cxp/qygcigsljdmdjogwvsdutxkgodrbd. 2024.10.22 01:22:47.205049 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6a886020cdd87f81d704de8800de957a, offset: 0, size: 0 2024.10.22 01:22:47.209349 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6a886020cdd87f81d704de8800de957a for removal 2024.10.22 01:22:47.222296 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_798_406} MergedBlockOutputStream: filled checksums 202410_1_798_406 (state Temporary) 2024.10.22 01:22:47.226608 [ 1476 ] {} CacheMetadata: Key 6a886020cdd87f81d704de8800de957a is removed from metadata 2024.10.22 01:22:47.325948 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_448_448_0 (state Temporary) 2024.10.22 01:22:47.464903 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: fafc483d554a876f0ed0a9cf867286cd, offset: 0, size: 0 2024.10.22 01:22:47.473406 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_798_406} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_798_406 to 202410_1_798_406 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:47.476061 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 212 entries to flush up to offset 51995 2024.10.22 01:22:47.477464 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key fafc483d554a876f0ed0a9cf867286cd for removal 2024.10.22 01:22:47.480477 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_798_406} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_797_405, 202410_798_798_0] -> 202410_1_798_406 2024.10.22 01:22:47.484046 [ 1476 ] {} CacheMetadata: Key fafc483d554a876f0ed0a9cf867286cd is removed from metadata 2024.10.22 01:22:47.484700 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/aak/yivywkgpczhviiusinnlkosxaanqc. 2024.10.22 01:22:47.555017 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_448_448_0 to 202410_455_455_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:47.594647 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 705366 2024.10.22 01:22:47.604162 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 7228 2024.10.22 01:22:47.619755 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:22:47.639982 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: df63f52de8be00cb9c584a4c67c9a598, offset: 0, size: 0 2024.10.22 01:22:47.643772 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key df63f52de8be00cb9c584a4c67c9a598 for removal 2024.10.22 01:22:47.646707 [ 1476 ] {} CacheMetadata: Key df63f52de8be00cb9c584a4c67c9a598 is removed from metadata 2024.10.22 01:22:47.684352 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f90e11c3223206088a2d45316949929d, offset: 0, size: 0 2024.10.22 01:22:47.690871 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f90e11c3223206088a2d45316949929d for removal 2024.10.22 01:22:47.721122 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/aak/yivywkgpczhviiusinnlkosxaanqc. 2024.10.22 01:22:47.752651 [ 1476 ] {} CacheMetadata: Key f90e11c3223206088a2d45316949929d is removed from metadata 2024.10.22 01:22:47.771686 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:47.776377 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:47.816573 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_467_467_0 (state Temporary) 2024.10.22 01:22:47.868435 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.11 GiB, peak 1.96 GiB, free memory in arenas 11.38 MiB, will set to 1.12 GiB (RSS), difference: 8.31 MiB 2024.10.22 01:22:47.881136 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_467_467_0 to 202410_480_480_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:47.961817 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 907060928c0191265eafdda044643e68, offset: 0, size: 0 2024.10.22 01:22:47.963090 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 907060928c0191265eafdda044643e68 for removal 2024.10.22 01:22:47.964742 [ 1476 ] {} CacheMetadata: Key 907060928c0191265eafdda044643e68 is removed from metadata 2024.10.22 01:22:48.025793 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ec5a5169b1f0c6d0fcc725360b321559, offset: 0, size: 0 2024.10.22 01:22:48.031737 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ec5a5169b1f0c6d0fcc725360b321559 for removal 2024.10.22 01:22:48.037743 [ 1476 ] {} CacheMetadata: Key ec5a5169b1f0c6d0fcc725360b321559 is removed from metadata 2024.10.22 01:22:48.066928 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7228 2024.10.22 01:22:48.091784 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5ec88d5ec6db37816f526b9c75042b36, offset: 0, size: 0 2024.10.22 01:22:48.098767 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5ec88d5ec6db37816f526b9c75042b36 for removal 2024.10.22 01:22:48.103523 [ 1476 ] {} CacheMetadata: Key 5ec88d5ec6db37816f526b9c75042b36 is removed from metadata 2024.10.22 01:22:48.228408 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:48.236007 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:48.264784 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} S3ObjectStorage: Objects with paths [common/aak/yivywkgpczhviiusinnlkosxaanqc] were removed from S3 2024.10.22 01:22:48.428022 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7ef0fd763bc61de9496780c271051e1c, offset: 0, size: 0 2024.10.22 01:22:48.467093 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_791_791_0 (state Temporary) 2024.10.22 01:22:48.429741 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7ef0fd763bc61de9496780c271051e1c for removal 2024.10.22 01:22:48.547172 [ 1476 ] {} CacheMetadata: Key 7ef0fd763bc61de9496780c271051e1c is removed from metadata 2024.10.22 01:22:48.573621 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} S3ObjectStorage: Objects with paths [common/sui/dnkzxwuxqnmyuuokhyhgxxhqnowqs] were removed from S3 2024.10.22 01:22:48.631182 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_791_791_0 to 202410_804_804_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:48.673156 [ 1197 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_798_406 to 202410_799_799_0 2024.10.22 01:22:48.685127 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 51995 2024.10.22 01:22:48.684353 [ 1197 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.85 MiB using storage policy from min volume index 0 2024.10.22 01:22:48.695453 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f241ef8fcfaff4cb890021fe30212a26, offset: 0, size: 0 2024.10.22 01:22:48.700188 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f241ef8fcfaff4cb890021fe30212a26 for removal 2024.10.22 01:22:48.705271 [ 1476 ] {} CacheMetadata: Key f241ef8fcfaff4cb890021fe30212a26 is removed from metadata 2024.10.22 01:22:48.798864 [ 1197 ] {} DiskLocal: Reserved 1.85 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:48.840211 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_799_407} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_798_406 to 202410_799_799_0 into Compact with storage Full 2024.10.22 01:22:48.864606 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d2cbb1399a9b5e56b65023f905146a41, offset: 0, size: 0 2024.10.22 01:22:48.867386 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d2cbb1399a9b5e56b65023f905146a41 for removal 2024.10.22 01:22:48.871404 [ 1476 ] {} CacheMetadata: Key d2cbb1399a9b5e56b65023f905146a41 is removed from metadata 2024.10.22 01:22:48.872285 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xvm/dikafbiabwdvtwvlpcmrngbxbvyqo. 2024.10.22 01:22:48.909508 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_799_407} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:48.924182 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5cb15f6e8404b0b093fa5cfcb9e8cf4e, offset: 0, size: 0 2024.10.22 01:22:48.926446 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 5cb15f6e8404b0b093fa5cfcb9e8cf4e for removal 2024.10.22 01:22:48.931357 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_799_407} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_798_406, total 51451 rows starting from the beginning of the part 2024.10.22 01:22:48.975933 [ 1476 ] {} CacheMetadata: Key 5cb15f6e8404b0b093fa5cfcb9e8cf4e is removed from metadata 2024.10.22 01:22:48.991827 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_799_407} MergeTreeSequentialSource: Reading 2 marks from part 202410_799_799_0, total 168 rows starting from the beginning of the part 2024.10.22 01:22:49.122802 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} S3ObjectStorage: Objects with paths [common/inz/fmvijkgktebihljiyvkuzjtencsgf] were removed from S3 2024.10.22 01:22:49.122978 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3750 2024.10.22 01:22:49.154728 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_700_318, 202410_701_701_0] 2024.10.22 01:22:49.158710 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_700_318, 202410_701_701_0] 2024.10.22 01:22:49.168618 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_700_318, 202410_701_701_0] 2024.10.22 01:22:49.207739 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c13ed6c35c5a3301d4de7a26fc92d685, offset: 0, size: 0 2024.10.22 01:22:49.210897 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c13ed6c35c5a3301d4de7a26fc92d685 for removal 2024.10.22 01:22:49.317431 [ 1476 ] {} CacheMetadata: Key c13ed6c35c5a3301d4de7a26fc92d685 is removed from metadata 2024.10.22 01:22:49.388725 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} S3ObjectStorage: Objects with paths [common/hav/ignpmfbrbpmwqohilcyfsftyrhsrc] were removed from S3 2024.10.22 01:22:49.372355 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_416_83, 202410_417_417_0, 202410_418_418_0, 202410_419_419_0, 202410_420_420_0, 202410_421_421_0] 2024.10.22 01:22:49.414474 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_416_83, 202410_417_417_0, 202410_418_418_0, 202410_419_419_0, 202410_420_420_0, 202410_421_421_0] 2024.10.22 01:22:49.458910 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_416_83, 202410_417_417_0, 202410_418_418_0, 202410_419_419_0, 202410_420_420_0, 202410_421_421_0] 2024.10.22 01:22:49.548410 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xvm/dikafbiabwdvtwvlpcmrngbxbvyqo. 2024.10.22 01:22:49.599211 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: e9cb2f80cd4198ace31c4e139000a97a, offset: 0, size: 0 2024.10.22 01:22:49.601594 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key e9cb2f80cd4198ace31c4e139000a97a for removal 2024.10.22 01:22:49.607530 [ 1476 ] {} CacheMetadata: Key e9cb2f80cd4198ace31c4e139000a97a is removed from metadata 2024.10.22 01:22:49.671664 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 12e1d483ab5a22e71e1a0d085d313fb8, offset: 0, size: 0 2024.10.22 01:22:49.673303 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 12e1d483ab5a22e71e1a0d085d313fb8 for removal 2024.10.22 01:22:49.676927 [ 1476 ] {} CacheMetadata: Key 12e1d483ab5a22e71e1a0d085d313fb8 is removed from metadata 2024.10.22 01:22:49.677634 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mil/fcyzlpnrkqysvmfumfxgymvborikd. 2024.10.22 01:22:49.735372 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f7d71cba3bed99a2348cec1b81a952db, offset: 0, size: 0 2024.10.22 01:22:49.740411 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f7d71cba3bed99a2348cec1b81a952db for removal 2024.10.22 01:22:49.746288 [ 1476 ] {} CacheMetadata: Key f7d71cba3bed99a2348cec1b81a952db is removed from metadata 2024.10.22 01:22:49.752282 [ 1213 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:22:49.781137 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} S3ObjectStorage: Objects with paths [common/xvm/dikafbiabwdvtwvlpcmrngbxbvyqo] were removed from S3 2024.10.22 01:22:49.795589 [ 1213 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:22:50.018132 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 167 entries to flush up to offset 52806 2024.10.22 01:22:50.064113 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7c9cd814bb99f1dd05de5abb8b7fc17b, offset: 0, size: 0 2024.10.22 01:22:50.065544 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7c9cd814bb99f1dd05de5abb8b7fc17b for removal 2024.10.22 01:22:50.070219 [ 1476 ] {} CacheMetadata: Key 7c9cd814bb99f1dd05de5abb8b7fc17b is removed from metadata 2024.10.22 01:22:50.194267 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} S3ObjectStorage: Objects with paths [common/gol/cbszuksbhbsfhgmyphuotanxehjvr] were removed from S3 2024.10.22 01:22:50.197789 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:22:50.213323 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:22:50.240362 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mil/fcyzlpnrkqysvmfumfxgymvborikd. 2024.10.22 01:22:50.327337 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 851fd806c605ff11dc156623493c9110, offset: 0, size: 0 2024.10.22 01:22:50.336204 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 851fd806c605ff11dc156623493c9110 for removal 2024.10.22 01:22:50.346571 [ 1476 ] {} CacheMetadata: Key 851fd806c605ff11dc156623493c9110 is removed from metadata 2024.10.22 01:22:50.473828 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d398f23831b6cc3ae3a3c38f0e0d9e6b, offset: 0, size: 0 2024.10.22 01:22:50.483454 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d398f23831b6cc3ae3a3c38f0e0d9e6b for removal 2024.10.22 01:22:50.486634 [ 1476 ] {} CacheMetadata: Key d398f23831b6cc3ae3a3c38f0e0d9e6b is removed from metadata 2024.10.22 01:22:50.493205 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:50.508641 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:50.581447 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_459_459_0 (state Temporary) 2024.10.22 01:22:50.613190 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.12 GiB, peak 1.96 GiB, free memory in arenas 13.94 MiB, will set to 1.12 GiB (RSS), difference: 3.42 MiB 2024.10.22 01:22:50.644008 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6d71de787b9da253bc65a614c93fe3c4, offset: 0, size: 0 2024.10.22 01:22:50.646611 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 6d71de787b9da253bc65a614c93fe3c4 for removal 2024.10.22 01:22:50.652370 [ 1476 ] {} CacheMetadata: Key 6d71de787b9da253bc65a614c93fe3c4 is removed from metadata 2024.10.22 01:22:50.691502 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} S3ObjectStorage: Objects with paths [common/mil/fcyzlpnrkqysvmfumfxgymvborikd] were removed from S3 2024.10.22 01:22:50.960262 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 228e89988eabaf39c5dd300575ce3f7e, offset: 0, size: 0 2024.10.22 01:22:50.963198 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 228e89988eabaf39c5dd300575ce3f7e for removal 2024.10.22 01:22:50.965507 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_459_459_0 to 202410_465_465_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:50.966076 [ 1476 ] {} CacheMetadata: Key 228e89988eabaf39c5dd300575ce3f7e is removed from metadata 2024.10.22 01:22:50.967133 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 396 2024.10.22 01:22:51.001560 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} S3ObjectStorage: Objects with paths [common/blc/wadfoclmdehsyifdcpusdspqxjoyf] were removed from S3 2024.10.22 01:22:51.012555 [ 1229 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_454_232 to 202410_455_460_1 2024.10.22 01:22:51.031351 [ 1229 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 2.64 MiB using storage policy from min volume index 0 2024.10.22 01:22:51.042423 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 52806 2024.10.22 01:22:51.054377 [ 1229 ] {} DiskLocal: Reserved 2.64 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:51.074770 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1beafe59ee6fa945720dd3aa2499f1f9, offset: 0, size: 0 2024.10.22 01:22:51.075998 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 1beafe59ee6fa945720dd3aa2499f1f9 for removal 2024.10.22 01:22:51.077470 [ 1476 ] {} CacheMetadata: Key 1beafe59ee6fa945720dd3aa2499f1f9 is removed from metadata 2024.10.22 01:22:51.086316 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_460_233} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_454_232 to 202410_455_460_1 into Wide with storage Full 2024.10.22 01:22:51.212230 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_402_180, 202410_403_403_0] 2024.10.22 01:22:51.220258 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_402_180, 202410_403_403_0] 2024.10.22 01:22:51.223593 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: ed6d6d0dc38406b218551d2df5a29a69, offset: 0, size: 0 2024.10.22 01:22:51.225015 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key ed6d6d0dc38406b218551d2df5a29a69 for removal 2024.10.22 01:22:51.226701 [ 1476 ] {} CacheMetadata: Key ed6d6d0dc38406b218551d2df5a29a69 is removed from metadata 2024.10.22 01:22:51.227407 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nlk/qeopbpckefhxwnkmuxclqocpptldv. 2024.10.22 01:22:51.267608 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_402_180, 202410_403_403_0] 2024.10.22 01:22:51.367400 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c5bf59a56e0e9765187b20d247663d97, offset: 0, size: 0 2024.10.22 01:22:51.371574 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c5bf59a56e0e9765187b20d247663d97 for removal 2024.10.22 01:22:51.375839 [ 1476 ] {} CacheMetadata: Key c5bf59a56e0e9765187b20d247663d97 is removed from metadata 2024.10.22 01:22:51.379665 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:51.382893 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:51.424142 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} S3ObjectStorage: Objects with paths [common/obk/juhevhvigeqziipmvoiscitdbyuiv] were removed from S3 2024.10.22 01:22:51.531107 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_460_233} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:51.633268 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_460_233} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_454_232, total 50893 rows starting from the beginning of the part 2024.10.22 01:22:51.634754 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_378_378_0 (state Temporary) 2024.10.22 01:22:51.793029 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: c466709fccbc998b25adf024e912c160, offset: 0, size: 0 2024.10.22 01:22:51.802250 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key c466709fccbc998b25adf024e912c160 for removal 2024.10.22 01:22:51.828726 [ 1476 ] {} CacheMetadata: Key c466709fccbc998b25adf024e912c160 is removed from metadata 2024.10.22 01:22:51.884570 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_460_233} MergeTreeSequentialSource: Reading 2 marks from part 202410_455_460_1, total 1379 rows starting from the beginning of the part 2024.10.22 01:22:51.970570 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} S3ObjectStorage: Objects with paths [common/pfh/szuoezjxffqostzwlzcatkdavsjfq] were removed from S3 2024.10.22 01:22:52.063563 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nlk/qeopbpckefhxwnkmuxclqocpptldv. 2024.10.22 01:22:52.096910 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_378_378_0 to 202410_388_388_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:52.111506 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0c2822c3edc8ff367ae0c238aad340da, offset: 0, size: 0 2024.10.22 01:22:52.113649 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 0c2822c3edc8ff367ae0c238aad340da for removal 2024.10.22 01:22:52.120633 [ 1476 ] {} CacheMetadata: Key 0c2822c3edc8ff367ae0c238aad340da is removed from metadata 2024.10.22 01:22:52.254468 [ 1224 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_383_79 to 202410_388_388_0 2024.10.22 01:22:52.267603 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 56722647c3793782e190ecfe2bb1b2fb, offset: 0, size: 0 2024.10.22 01:22:52.269764 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 56722647c3793782e190ecfe2bb1b2fb for removal 2024.10.22 01:22:52.275229 [ 1476 ] {} CacheMetadata: Key 56722647c3793782e190ecfe2bb1b2fb is removed from metadata 2024.10.22 01:22:52.280655 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ima/imazjzemuurnbwzohwwezedhkboni. 2024.10.22 01:22:52.292862 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 396 2024.10.22 01:22:52.320477 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: d22efe560d38f58638ab9b37b86aee88, offset: 0, size: 0 2024.10.22 01:22:52.322761 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key d22efe560d38f58638ab9b37b86aee88 for removal 2024.10.22 01:22:52.326094 [ 1476 ] {} CacheMetadata: Key d22efe560d38f58638ab9b37b86aee88 is removed from metadata 2024.10.22 01:22:52.366767 [ 1224 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:52.368626 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:52.377622 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7bf9e2cb5252c8b4f781cbb50acc7995, offset: 0, size: 0 2024.10.22 01:22:52.382332 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 7bf9e2cb5252c8b4f781cbb50acc7995 for removal 2024.10.22 01:22:52.386942 [ 1476 ] {} CacheMetadata: Key 7bf9e2cb5252c8b4f781cbb50acc7995 is removed from metadata 2024.10.22 01:22:52.398559 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ima/imazjzemuurnbwzohwwezedhkboni. 2024.10.22 01:22:52.406730 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_383_79 to 202410_388_388_0 into Compact with storage Full 2024.10.22 01:22:52.471250 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_799_407} MergeTask::MergeProjectionsStage: Merge sorted 51619 rows, containing 13 columns (13 merged, 0 gathered) in 3.65969469 sec., 14104.728501272875 rows/sec., 2.26 MiB/sec. 2024.10.22 01:22:52.676996 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.96 GiB, free memory in arenas 13.17 MiB, will set to 1.12 GiB (RSS), difference: -35.35 MiB 2024.10.22 01:22:52.738092 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 166 entries to flush up to offset 52161 2024.10.22 01:22:52.737452 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:52.844711 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_799_407} MergedBlockOutputStream: filled checksums 202410_1_799_407 (state Temporary) 2024.10.22 01:22:52.856966 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_383_79, total 427 rows starting from the beginning of the part 2024.10.22 01:22:52.970340 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for d22efe560d38f58638ab9b37b86aee88:0) 2024.10.22 01:22:52.977384 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: b4988f305950f6bb511fe3f66cdc6d67, offset: 0, size: 8293612 2024.10.22 01:22:52.985958 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key b4988f305950f6bb511fe3f66cdc6d67 for removal 2024.10.22 01:22:52.987839 [ 1476 ] {} CacheMetadata: Key b4988f305950f6bb511fe3f66cdc6d67 is removed from metadata 2024.10.22 01:22:52.988368 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8b346c3fd1df5f8025a0297551a3e774, offset: 0, size: 1009 2024.10.22 01:22:52.998696 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 8b346c3fd1df5f8025a0297551a3e774 for removal 2024.10.22 01:22:53.013084 [ 1476 ] {} CacheMetadata: Key 8b346c3fd1df5f8025a0297551a3e774 is removed from metadata 2024.10.22 01:22:53.028844 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_384_384_0, total 1 rows starting from the beginning of the part 2024.10.22 01:22:53.049773 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_391_78, 202410_392_392_0, 202410_393_393_0, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0] 2024.10.22 01:22:53.052640 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_391_78, 202410_392_392_0, 202410_393_393_0, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0] 2024.10.22 01:22:53.081662 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_391_78, 202410_392_392_0, 202410_393_393_0, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0] 2024.10.22 01:22:53.090896 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_799_407} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_799_407 to 202410_1_799_407 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:53.094929 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_799_407} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_798_406, 202410_799_799_0] -> 202410_1_799_407 2024.10.22 01:22:53.107826 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_385_385_0, total 1 rows starting from the beginning of the part 2024.10.22 01:22:53.179484 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_386_386_0, total 1 rows starting from the beginning of the part 2024.10.22 01:22:53.231862 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.82 MiB. 2024.10.22 01:22:53.262064 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_387_387_0, total 1 rows starting from the beginning of the part 2024.10.22 01:22:53.273234 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:53.276522 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:22:53.375752 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_792_792_0 (state Temporary) 2024.10.22 01:22:53.460475 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_388_388_0, total 1 rows starting from the beginning of the part 2024.10.22 01:22:53.470862 [ 1497 ] {} SystemLog (system.query_log): Flushing system log, 1 entries to flush up to offset 17 2024.10.22 01:22:53.623092 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xse/ugozpadjtjlwelxmztbqzkufixsvt. 2024.10.22 01:22:53.633149 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_792_792_0 to 202410_805_805_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:53.691834 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_799_407 to 202410_800_800_0 2024.10.22 01:22:53.694529 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2f7a42b0023312c9083db016e740cfc7, offset: 0, size: 0 2024.10.22 01:22:53.696278 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key 2f7a42b0023312c9083db016e740cfc7 for removal 2024.10.22 01:22:53.701144 [ 1476 ] {} CacheMetadata: Key 2f7a42b0023312c9083db016e740cfc7 is removed from metadata 2024.10.22 01:22:53.701527 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.86 MiB using storage policy from min volume index 0 2024.10.22 01:22:53.712461 [ 1152 ] {} DiskLocal: Reserved 1.86 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:22:53.712630 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52161 2024.10.22 01:22:53.722683 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:22:53.763624 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Remove from cache. Key: f8ad94cc8be0018294283e3109c97928, offset: 0, size: 0 2024.10.22 01:22:53.768354 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} CacheMetadata: Submitting key f8ad94cc8be0018294283e3109c97928 for removal 2024.10.22 01:22:53.775250 [ 1476 ] {} CacheMetadata: Key f8ad94cc8be0018294283e3109c97928 is removed from metadata 2024.10.22 01:22:53.782390 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xse/ugozpadjtjlwelxmztbqzkufixsvt. 2024.10.22 01:22:53.857445 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_800_408} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_799_407 to 202410_800_800_0 into Compact with storage Full 2024.10.22 01:22:54.018846 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_800_408} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:54.047501 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_800_408} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_799_407, total 51619 rows starting from the beginning of the part 2024.10.22 01:22:54.080826 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_800_408} MergeTreeSequentialSource: Reading 2 marks from part 202410_800_800_0, total 300 rows starting from the beginning of the part 2024.10.22 01:22:54.663055 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 706521 2024.10.22 01:22:54.663669 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.96 GiB, free memory in arenas 14.65 MiB, will set to 1.15 GiB (RSS), difference: -6.62 MiB 2024.10.22 01:22:54.805682 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergeTask::MergeProjectionsStage: Merge sorted 2859008 rows, containing 133 columns (3 merged, 130 gathered) in 2890.741236141 sec., 989.0224570278858 rows/sec., 747.94 KiB/sec. 2024.10.22 01:22:55.100424 [ 1497 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:55.109677 [ 1497 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:55.120843 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_701_319, 202410_702_702_0] 2024.10.22 01:22:55.127012 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_701_319, 202410_702_702_0] 2024.10.22 01:22:55.256427 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_701_319, 202410_702_702_0] 2024.10.22 01:22:55.324331 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:55.327295 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:55.320695 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} MergedBlockOutputStream: filled checksums 201403_1_7_1 (state Temporary) 2024.10.22 01:22:55.321587 [ 1272 ] {} DatabaseCatalog: Execution took 260 ms. 2024.10.22 01:22:55.352383 [ 1717 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/equ/aviniscxwlanrpdldqzjxjlgorshd, size 2367 2024.10.22 01:22:55.524376 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ycd/vcakgmhpzztaschcomecopfrisdfi (LogSeriesLimiter: on interval from 2024-10-22 01:22:43 to 2024-10-22 01:22:55 accepted series 1 / 9 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:22:55.555261 [ 1497 ] {} MergedBlockOutputStream: filled checksums 202410_5_5_0 (state Temporary) 2024.10.22 01:22:55.573930 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 7236 2024.10.22 01:22:55.600489 [ 1709 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ycd/vcakgmhpzztaschcomecopfrisdfi, size 4 2024.10.22 01:22:55.867391 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_449_449_0 (state Temporary) 2024.10.22 01:22:55.933615 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:55.945038 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:55.955128 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_460_233} MergeTask::MergeProjectionsStage: Merge sorted 52272 rows, containing 13 columns (13 merged, 0 gathered) in 4.873050847 sec., 10726.750374907386 rows/sec., 4.66 MiB/sec. 2024.10.22 01:22:56.017236 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_468_468_0 (state Temporary) 2024.10.22 01:22:56.050474 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_449_449_0 to 202410_456_456_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:56.056294 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_460_233} MergedBlockOutputStream: filled checksums 202410_1_460_233 (state Temporary) 2024.10.22 01:22:56.074528 [ 1497 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Renaming temporary part tmp_insert_202410_5_5_0 to 202410_9_9_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:56.096802 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 706521 2024.10.22 01:22:56.113008 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_454_91/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:22:56.195440 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_468_468_0 to 202410_481_481_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:56.221069 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} MergeTask::MergeProjectionsStage: Merge sorted 432 rows, containing 32 columns (32 merged, 0 gathered) in 3.827758627 sec., 112.8597809048841 rows/sec., 10.55 KiB/sec. 2024.10.22 01:22:56.256196 [ 1226 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_476_95 to 202410_481_481_0 2024.10.22 01:22:56.274706 [ 1226 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:56.276085 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7236 2024.10.22 01:22:56.278568 [ 1226 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:56.307936 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_476_95 to 202410_481_481_0 into Compact with storage Full 2024.10.22 01:22:56.377656 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:56.443275 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} MergedBlockOutputStream: filled checksums 202410_1_388_80 (state Temporary) 2024.10.22 01:22:56.448201 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_476_95, total 7269 rows starting from the beginning of the part 2024.10.22 01:22:56.478834 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_477_477_0, total 23 rows starting from the beginning of the part 2024.10.22 01:22:56.597555 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_478_478_0, total 15 rows starting from the beginning of the part 2024.10.22 01:22:56.682858 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 45 entries to flush up to offset 3795 2024.10.22 01:22:56.755614 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_479_479_0, total 12 rows starting from the beginning of the part 2024.10.22 01:22:56.784227 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_480_480_0, total 15 rows starting from the beginning of the part 2024.10.22 01:22:56.816425 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_481_481_0, total 8 rows starting from the beginning of the part 2024.10.22 01:22:56.937260 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 1.96 GiB, free memory in arenas 9.95 MiB, will set to 1.16 GiB (RSS), difference: 4.85 MiB 2024.10.22 01:22:56.962576 [ 1497 ] {} SystemLog (system.query_log): Flushed system log up to offset 17 2024.10.22 01:22:57.054269 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_460_233} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_460_233 to 202410_1_460_233 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:57.056906 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_460_233} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_454_232, 202410_455_460_1] -> 202410_1_460_233 2024.10.22 01:22:57.074341 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 64.75 MiB. 2024.10.22 01:22:57.849234 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 52253 2024.10.22 01:22:57.888854 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} MergeTask::MergeProjectionsStage: Merge sorted 7342 rows, containing 12 columns (12 merged, 0 gathered) in 1.59932102 sec., 4590.698120131004 rows/sec., 728.93 KiB/sec. 2024.10.22 01:22:58.007057 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_388_80 to 202410_1_388_80 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:58.120985 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_388_80} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_383_79, 202410_388_388_0] -> 202410_1_388_80 2024.10.22 01:22:58.166334 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} MergedBlockOutputStream: filled checksums 202410_1_481_96 (state Temporary) 2024.10.22 01:22:58.184616 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:22:58.324155 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_800_408} MergeTask::MergeProjectionsStage: Merge sorted 51919 rows, containing 13 columns (13 merged, 0 gathered) in 4.580558935 sec., 11334.642941342267 rows/sec., 1.81 MiB/sec. 2024.10.22 01:22:58.357037 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:58.365037 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:58.450260 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_793_793_0 (state Temporary) 2024.10.22 01:22:58.527199 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_800_408} MergedBlockOutputStream: filled checksums 202410_1_800_408 (state Temporary) 2024.10.22 01:22:58.630735 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_793_793_0 to 202410_806_806_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:58.676569 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 263 entries to flush up to offset 53069 2024.10.22 01:22:58.678839 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_801_801_0 to 202410_806_806_0 2024.10.22 01:22:58.694810 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:58.700661 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:58.732161 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_801_801_0 to 202410_806_806_0 into Compact with storage Full 2024.10.22 01:22:58.776866 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52253 2024.10.22 01:22:58.782480 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_800_408} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_800_408 to 202410_1_800_408 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:58.789459 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_800_408} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_799_407, 202410_800_800_0] -> 202410_1_800_408 2024.10.22 01:22:58.811579 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:58.838122 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_801_801_0, total 277 rows starting from the beginning of the part 2024.10.22 01:22:58.841137 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.86 MiB. 2024.10.22 01:22:58.883818 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_802_802_0, total 225 rows starting from the beginning of the part 2024.10.22 01:22:58.998325 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_481_96 to 202410_1_481_96 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:59.009171 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_481_96} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_476_95, 202410_481_481_0] -> 202410_1_481_96 2024.10.22 01:22:59.060292 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.45 MiB. 2024.10.22 01:22:59.076828 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:59.079627 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:59.106746 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_803_803_0, total 193 rows starting from the beginning of the part 2024.10.22 01:22:59.151539 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_460_460_0 (state Temporary) 2024.10.22 01:22:59.167638 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_804_804_0, total 212 rows starting from the beginning of the part 2024.10.22 01:22:59.218456 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_460_460_0 to 202410_466_466_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:22:59.264475 [ 1260 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_461_461_0 to 202410_466_466_0 2024.10.22 01:22:59.278629 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_805_805_0, total 166 rows starting from the beginning of the part 2024.10.22 01:22:59.304858 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 53069 2024.10.22 01:22:59.344974 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_806_806_0, total 92 rows starting from the beginning of the part 2024.10.22 01:22:59.402964 [ 1260 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:22:59.420097 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:22:59.504457 [ 1260 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 256 ms. 2024.10.22 01:22:59.526761 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_461_461_0 to 202410_466_466_0 into Compact with storage Full 2024.10.22 01:22:59.587659 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:22:59.626625 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_461_0, total 164 rows starting from the beginning of the part 2024.10.22 01:22:59.699756 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_462_462_0, total 96 rows starting from the beginning of the part 2024.10.22 01:22:59.759805 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_463_463_0, total 96 rows starting from the beginning of the part 2024.10.22 01:22:59.838442 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_464_464_0, total 115 rows starting from the beginning of the part 2024.10.22 01:23:00.001703 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_465_465_0, total 167 rows starting from the beginning of the part 2024.10.22 01:23:00.066150 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_466_466_0, total 263 rows starting from the beginning of the part 2024.10.22 01:23:00.070931 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_702_320, 202410_703_703_0] 2024.10.22 01:23:00.072953 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_702_320, 202410_703_703_0] 2024.10.22 01:23:00.082828 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_702_320, 202410_703_703_0] 2024.10.22 01:23:00.201091 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:23:00.205292 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:23:00.318981 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_801_806_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:23:00.382390 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_403_181, 202410_404_404_0] 2024.10.22 01:23:00.383990 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_403_181, 202410_404_404_0] 2024.10.22 01:23:00.461107 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_403_181, 202410_404_404_0] 2024.10.22 01:23:00.527207 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.19 GiB, peak 1.96 GiB, free memory in arenas 9.68 MiB, will set to 1.18 GiB (RSS), difference: -7.71 MiB 2024.10.22 01:23:00.767744 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} MergeTask::MergeProjectionsStage: Merge sorted 1165 rows, containing 13 columns (13 merged, 0 gathered) in 2.049837163 sec., 568.3378275252785 rows/sec., 89.60 KiB/sec. 2024.10.22 01:23:00.814539 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} MergedBlockOutputStream: filled checksums 202410_801_806_1 (state Temporary) 2024.10.22 01:23:00.862625 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/equ/aviniscxwlanrpdldqzjxjlgorshd. 2024.10.22 01:23:00.909227 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ycd/vcakgmhpzztaschcomecopfrisdfi. 2024.10.22 01:23:01.216710 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_801_806_1 to 202410_801_806_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:01.223343 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_801_806_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_801_801_0, 202410_806_806_0] -> 202410_801_806_1 2024.10.22 01:23:01.237776 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} MergeTask::MergeProjectionsStage: Merge sorted 901 rows, containing 13 columns (13 merged, 0 gathered) in 1.728079314 sec., 521.3881056850612 rows/sec., 235.41 KiB/sec. 2024.10.22 01:23:01.270728 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:23:01.304946 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} MergedBlockOutputStream: filled checksums 202410_461_466_1 (state Temporary) 2024.10.22 01:23:01.478799 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_461_466_1 to 202410_461_466_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:01.482225 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_466_1} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_461_461_0, 202410_466_466_0] -> 202410_461_466_1 2024.10.22 01:23:01.536305 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:23:01.595735 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ycd/vcakgmhpzztaschcomecopfrisdfi. 2024.10.22 01:23:01.702021 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_merge_201403_1_7_1 to 201403_1_7_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:01.709024 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937) (MergerMutator): Merged 7 parts: [201403_1_1_0, 201403_7_7_0] -> 201403_1_7_1 2024.10.22 01:23:01.752900 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_1_7_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/equ/aviniscxwlanrpdldqzjxjlgorshd. 2024.10.22 01:23:01.780145 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 387.04 MiB. 2024.10.22 01:23:02.130375 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_335_69, 202410_336_336_0, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0] 2024.10.22 01:23:02.158671 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_335_69, 202410_336_336_0, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0] 2024.10.22 01:23:02.180531 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_335_69, 202410_336_336_0, 202410_337_337_0, 202410_338_338_0, 202410_339_339_0] 2024.10.22 01:23:02.795212 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 52331 2024.10.22 01:23:02.979122 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.19 GiB, peak 1.96 GiB, free memory in arenas 13.55 MiB, will set to 1.21 GiB (RSS), difference: 26.53 MiB 2024.10.22 01:23:03.112334 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 708063 2024.10.22 01:23:03.332526 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:03.338467 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:23:03.418364 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_794_794_0 (state Temporary) 2024.10.22 01:23:03.472464 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_794_794_0 to 202410_807_807_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:03.739112 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} MergeTask::MergeProjectionsStage: Merge sorted 717031 rows, containing 4 columns (4 merged, 0 gathered) in 24.382602392 sec., 29407.484421566907 rows/sec., 441.12 KiB/sec. 2024.10.22 01:23:03.761156 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52331 2024.10.22 01:23:03.779936 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 48 entries to flush up to offset 7284 2024.10.22 01:23:03.854384 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:03.861294 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:23:04.027931 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} MergedBlockOutputStream: filled checksums 202410_1_454_91 (state Temporary) 2024.10.22 01:23:04.053324 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:04.059440 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:23:04.104661 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_469_469_0 (state Temporary) 2024.10.22 01:23:04.157586 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_450_450_0 (state Temporary) 2024.10.22 01:23:04.252140 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_703_321, 202410_704_704_0] 2024.10.22 01:23:04.254569 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_703_321, 202410_704_704_0] 2024.10.22 01:23:04.262935 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_703_321, 202410_704_704_0] 2024.10.22 01:23:04.264820 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_469_469_0 to 202410_482_482_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:04.330563 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7284 2024.10.22 01:23:04.496630 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_450_450_0 to 202410_457_457_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:04.590743 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 708063 2024.10.22 01:23:04.805649 [ 1267 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:23:04.863858 [ 1267 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:23:04.902174 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.27 GiB, peak 1.96 GiB, free memory in arenas 13.09 MiB, will set to 1.24 GiB (RSS), difference: -29.24 MiB 2024.10.22 01:23:05.536991 [ 1207 ] {} DatabaseCatalog: Execution took 214 ms. 2024.10.22 01:23:06.280995 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_454_91/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:23:06.333164 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_454_91 to 202410_1_454_91 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:06.340168 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_454_91} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_449_90, 202410_454_454_0] -> 202410_1_454_91 2024.10.22 01:23:06.375523 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:23:06.481717 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.28 GiB, peak 1.96 GiB, free memory in arenas 12.28 MiB, will set to 1.28 GiB (RSS), difference: -3.00 MiB 2024.10.22 01:23:06.721317 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:23:06.988384 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 146 entries to flush up to offset 53215 2024.10.22 01:23:07.223663 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_404_182, 202410_405_405_0] 2024.10.22 01:23:07.227235 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_404_182, 202410_405_405_0] 2024.10.22 01:23:07.301027 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 397 2024.10.22 01:23:07.364223 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_404_182, 202410_405_405_0] 2024.10.22 01:23:07.428676 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:07.431777 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:23:07.488327 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_461_461_0 (state Temporary) 2024.10.22 01:23:07.620984 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:07.624286 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:23:07.685427 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_379_379_0 (state Temporary) 2024.10.22 01:23:07.759017 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_461_461_0 to 202410_467_467_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:07.770172 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 52377 2024.10.22 01:23:07.810863 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_379_379_0 to 202410_389_389_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:07.819412 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 1.96 GiB, free memory in arenas 10.74 MiB, will set to 1.29 GiB (RSS), difference: -8.06 MiB 2024.10.22 01:23:07.834217 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 53215 2024.10.22 01:23:07.928028 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Interests 2024.10.22 01:23:07.937569 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Interests 2024.10.22 01:23:07.983263 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Interests 2024.10.22 01:23:08.004926 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Interests 2024.10.22 01:23:08.025003 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Interests 2024.10.22 01:23:08.040194 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:08.049796 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Interests 2024.10.22 01:23:08.051965 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:23:08.084870 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 397 2024.10.22 01:23:08.125831 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_795_795_0 (state Temporary) 2024.10.22 01:23:08.175597 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Interests 2024.10.22 01:23:08.185207 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_795_795_0 to 202410_808_808_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:08.207023 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Interests 2024.10.22 01:23:08.241308 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Interests 2024.10.22 01:23:08.291796 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mvy/rwideehywmuyubpkarbzcxnofbsie (LogSeriesLimiter: on interval from 2024-10-22 01:22:55 to 2024-10-22 01:23:08 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:23:08.456962 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52377 2024.10.22 01:23:08.683512 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_704_322, 202410_705_705_0] 2024.10.22 01:23:08.685709 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_704_322, 202410_705_705_0] 2024.10.22 01:23:08.697420 [ 1270 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 390 ms. 2024.10.22 01:23:08.699104 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_704_322, 202410_705_705_0] 2024.10.22 01:23:08.789709 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 25 2024.10.22 01:23:08.989647 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:08.996554 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:23:09.261443 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_25_25_0 (state Temporary) 2024.10.22 01:23:09.371252 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.96 GiB, free memory in arenas 9.29 MiB, will set to 1.32 GiB (RSS), difference: -7.23 MiB 2024.10.22 01:23:09.577814 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_25_25_0 to 202410_25_25_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:09.672688 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 25 2024.10.22 01:23:09.863569 [ 1223 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 262 ms. 2024.10.22 01:23:10.204041 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:23:10.206523 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:23:11.604342 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1923 entries to flush up to offset 709986 2024.10.22 01:23:11.834583 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 7297 2024.10.22 01:23:11.909722 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column WatchID 2024.10.22 01:23:11.925218 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column WatchID 2024.10.22 01:23:11.941835 [ 1161 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:23:12.104001 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column WatchID 2024.10.22 01:23:12.224389 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:12.226767 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column WatchID 2024.10.22 01:23:12.229704 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:23:12.236275 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column WatchID 2024.10.22 01:23:12.263587 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column WatchID 2024.10.22 01:23:12.276194 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_470_470_0 (state Temporary) 2024.10.22 01:23:12.277219 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column WatchID 2024.10.22 01:23:12.295910 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column WatchID 2024.10.22 01:23:12.358179 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_470_470_0 to 202410_483_483_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:12.413555 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7297 2024.10.22 01:23:12.466860 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 52427 2024.10.22 01:23:12.522566 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:12.525054 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:23:12.867000 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:12.869279 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:23:12.885662 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_451_451_0 (state Temporary) 2024.10.22 01:23:13.153684 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_451_451_0 to 202410_458_458_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:13.155036 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_796_796_0 (state Temporary) 2024.10.22 01:23:13.215252 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_796_796_0 to 202410_809_809_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:13.280410 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52427 2024.10.22 01:23:13.289396 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 709986 2024.10.22 01:23:13.459174 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.96 GiB, free memory in arenas 8.72 MiB, will set to 1.37 GiB (RSS), difference: -36.11 MiB 2024.10.22 01:23:13.502062 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_705_323, 202410_706_706_0] 2024.10.22 01:23:13.505645 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_705_323, 202410_706_706_0] 2024.10.22 01:23:13.518290 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_705_323, 202410_706_706_0] 2024.10.22 01:23:14.259264 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 0da1bb5a82e7fa189fa7dc9a2bff1627:0) 2024.10.22 01:23:14.262713 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2597e5a36c4b699d5aa7c74e4d9b23e0, offset: 0, size: 892195 2024.10.22 01:23:14.267129 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2597e5a36c4b699d5aa7c74e4d9b23e0 for removal 2024.10.22 01:23:14.270304 [ 1476 ] {} CacheMetadata: Key 2597e5a36c4b699d5aa7c74e4d9b23e0 is removed from metadata 2024.10.22 01:23:14.271247 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ee7f4c848221fcd5e017669cf18c3025, offset: 0, size: 527 2024.10.22 01:23:14.274668 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ee7f4c848221fcd5e017669cf18c3025 for removal 2024.10.22 01:23:14.294716 [ 1476 ] {} CacheMetadata: Key ee7f4c848221fcd5e017669cf18c3025 is removed from metadata 2024.10.22 01:23:15.118553 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:15.137777 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:15.322720 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 0da1bb5a82e7fa189fa7dc9a2bff1627:0) 2024.10.22 01:23:15.328333 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d68ac72127b53702caa0188390ec9b7d, offset: 0, size: 10651484 2024.10.22 01:23:15.330778 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_405_183, 202410_406_406_0] 2024.10.22 01:23:15.349034 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_405_183, 202410_406_406_0] 2024.10.22 01:23:15.352397 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d68ac72127b53702caa0188390ec9b7d for removal 2024.10.22 01:23:15.359114 [ 1476 ] {} CacheMetadata: Key d68ac72127b53702caa0188390ec9b7d is removed from metadata 2024.10.22 01:23:15.359399 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8e439248ccf6a6c37b3c4d1461420ca1, offset: 0, size: 1027 2024.10.22 01:23:15.365051 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8e439248ccf6a6c37b3c4d1461420ca1 for removal 2024.10.22 01:23:15.373412 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b39493e8a5d709a92f60f0e24b5ca548, offset: 0, size: 55341 2024.10.22 01:23:15.378250 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b39493e8a5d709a92f60f0e24b5ca548 for removal 2024.10.22 01:23:15.382051 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b8d8d9981c130f535f3c2f9e9e4e1c20, offset: 0, size: 504 2024.10.22 01:23:15.382399 [ 1476 ] {} CacheMetadata: Key b39493e8a5d709a92f60f0e24b5ca548 is removed from metadata 2024.10.22 01:23:15.385571 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 205 entries to flush up to offset 53420 2024.10.22 01:23:15.387789 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b8d8d9981c130f535f3c2f9e9e4e1c20 for removal 2024.10.22 01:23:15.400802 [ 1476 ] {} CacheMetadata: Key b8d8d9981c130f535f3c2f9e9e4e1c20 is removed from metadata 2024.10.22 01:23:15.415386 [ 1476 ] {} CacheMetadata: Key 8e439248ccf6a6c37b3c4d1461420ca1 is removed from metadata 2024.10.22 01:23:15.470166 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_405_183, 202410_406_406_0] 2024.10.22 01:23:15.587678 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 398 2024.10.22 01:23:15.685535 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.96 GiB, free memory in arenas 8.27 MiB, will set to 1.41 GiB (RSS), difference: -7.83 MiB 2024.10.22 01:23:16.087475 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:16.095102 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.26 GiB. 2024.10.22 01:23:16.101083 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:16.104444 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:16.203766 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_462_462_0 (state Temporary) 2024.10.22 01:23:16.286897 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_380_380_0 (state Temporary) 2024.10.22 01:23:16.634362 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_380_380_0 to 202410_390_390_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:16.650178 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_462_462_0 to 202410_468_468_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:16.687264 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 53420 2024.10.22 01:23:16.924576 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 398 2024.10.22 01:23:17.307584 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 52480 2024.10.22 01:23:17.688705 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:17.692834 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:17.727393 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.96 GiB, free memory in arenas 7.63 MiB, will set to 1.46 GiB (RSS), difference: -28.58 MiB 2024.10.22 01:23:17.873543 [ 1258 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 01:23:17.967165 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_797_797_0 (state Temporary) 2024.10.22 01:23:18.076174 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_797_797_0 to 202410_810_810_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:18.253741 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52480 2024.10.22 01:23:18.669641 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_95_95_0 (state Temporary) 2024.10.22 01:23:19.529175 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.96 GiB, free memory in arenas 8.56 MiB, will set to 1.49 GiB (RSS), difference: -11.66 MiB 2024.10.22 01:23:19.739574 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_706_324, 202410_707_707_0] 2024.10.22 01:23:19.751481 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_706_324, 202410_707_707_0] 2024.10.22 01:23:19.764184 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_706_324, 202410_707_707_0] 2024.10.22 01:23:19.869519 [ 1252 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:23:19.910810 [ 1252 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:23:19.919004 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 7303 2024.10.22 01:23:20.153163 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:20.168378 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:20.203134 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:23:20.208944 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_471_471_0 (state Temporary) 2024.10.22 01:23:20.209211 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:23:20.302433 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1771 entries to flush up to offset 711757 2024.10.22 01:23:20.347760 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_471_471_0 to 202410_484_484_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:20.388077 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7303 2024.10.22 01:23:21.063884 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:21.065521 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:21.409259 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_452_452_0 (state Temporary) 2024.10.22 01:23:21.480630 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_452_452_0 to 202410_459_459_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:21.531851 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_454_91 to 202410_459_459_0 2024.10.22 01:23:21.537359 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 8.77 MiB, will set to 1.51 GiB (RSS), difference: 9.74 MiB 2024.10.22 01:23:21.548418 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 711757 2024.10.22 01:23:21.554901 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:21.557698 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:21.593451 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_454_91 to 202410_459_459_0 into Wide with storage Full 2024.10.22 01:23:21.639890 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:21.655102 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} MergeTreeSequentialSource: Reading 89 marks from part 202410_1_454_91, total 717031 rows starting from the beginning of the part 2024.10.22 01:23:21.727309 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_455_455_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:23:21.849363 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_456_456_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:23:21.867217 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_457_457_0, total 1542 rows starting from the beginning of the part 2024.10.22 01:23:21.890170 [ 1555 ] {} SystemLog (system.session_log): Flushing system log, 2 entries to flush up to offset 17 2024.10.22 01:23:21.910538 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_458_458_0, total 1923 rows starting from the beginning of the part 2024.10.22 01:23:21.952298 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_459_459_0, total 1771 rows starting from the beginning of the part 2024.10.22 01:23:22.259976 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 52522 2024.10.22 01:23:22.301706 [ 1555 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:22.304997 [ 1555 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:22.375474 [ 1555 ] {} MergedBlockOutputStream: filled checksums 202410_3_3_0 (state Temporary) 2024.10.22 01:23:22.556013 [ 1555 ] {} system.session_log (ed97c24a-3021-4d92-aee2-1ef2de3fa803): Renaming temporary part tmp_insert_202410_3_3_0 to 202410_5_5_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:22.641676 [ 1555 ] {} SystemLog (system.session_log): Flushed system log up to offset 17 2024.10.22 01:23:22.756938 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:22.761274 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:22.808312 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_798_798_0 (state Temporary) 2024.10.22 01:23:22.933131 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 378.30 MiB using storage policy from min volume index 0 2024.10.22 01:23:22.966261 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_798_798_0 to 202410_811_811_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:23.020678 [ 1175 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_800_408 to 202410_801_806_1 2024.10.22 01:23:23.023785 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52522 2024.10.22 01:23:23.042346 [ 1175 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.91 MiB using storage policy from min volume index 0 2024.10.22 01:23:23.052576 [ 1175 ] {} DiskLocal: Reserved 1.91 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:23.109752 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_806_409} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_800_408 to 202410_801_806_1 into Compact with storage Full 2024.10.22 01:23:23.158872 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_806_409} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:23.174660 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_459_92/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:23:23.186224 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_806_409} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_800_408, total 51919 rows starting from the beginning of the part 2024.10.22 01:23:23.229391 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_806_409} MergeTreeSequentialSource: Reading 2 marks from part 202410_801_806_1, total 1165 rows starting from the beginning of the part 2024.10.22 01:23:23.299062 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_406_184, 202410_407_407_0] 2024.10.22 01:23:23.303449 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_406_184, 202410_407_407_0] 2024.10.22 01:23:23.321366 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_406_184, 202410_407_407_0] 2024.10.22 01:23:23.480093 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.96 GiB, free memory in arenas 7.99 MiB, will set to 1.52 GiB (RSS), difference: -11.08 MiB 2024.10.22 01:23:23.724763 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_707_325, 202410_708_708_0] 2024.10.22 01:23:23.736162 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_707_325, 202410_708_708_0] 2024.10.22 01:23:23.759713 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_707_325, 202410_708_708_0] 2024.10.22 01:23:24.173287 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yiu/oslbfhwlrpbcttcnpcfygmfpsxnvz (LogSeriesLimiter: on interval from 2024-10-22 01:23:08 to 2024-10-22 01:23:24 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:23:24.427424 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 399 2024.10.22 01:23:24.466841 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 168 entries to flush up to offset 53588 2024.10.22 01:23:24.931883 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:24.951162 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:25.023070 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_381_381_0 (state Temporary) 2024.10.22 01:23:25.040563 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:25.055203 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:25.118409 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_381_381_0 to 202410_391_391_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:25.211727 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_463_463_0 (state Temporary) 2024.10.22 01:23:25.327926 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 399 2024.10.22 01:23:25.370841 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:23:25.426177 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_463_463_0 to 202410_469_469_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:25.472566 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 53588 2024.10.22 01:23:25.485204 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 12 file segments (while reserving 8388608 bytes for 0da1bb5a82e7fa189fa7dc9a2bff1627:0) 2024.10.22 01:23:25.597648 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5e138c155f144795011d6af2c59d0d74, offset: 0, size: 2618841 2024.10.22 01:23:25.708172 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5e138c155f144795011d6af2c59d0d74 for removal 2024.10.22 01:23:25.713074 [ 1476 ] {} CacheMetadata: Key 5e138c155f144795011d6af2c59d0d74 is removed from metadata 2024.10.22 01:23:25.713840 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 306f362c9d6a67ec47e56c76384c12d1, offset: 0, size: 1153322 2024.10.22 01:23:25.723684 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 306f362c9d6a67ec47e56c76384c12d1 for removal 2024.10.22 01:23:25.728026 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5cc25574a8422260595dcaddfb48e57b, offset: 0, size: 915 2024.10.22 01:23:25.735208 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5cc25574a8422260595dcaddfb48e57b for removal 2024.10.22 01:23:25.738139 [ 1476 ] {} CacheMetadata: Key 306f362c9d6a67ec47e56c76384c12d1 is removed from metadata 2024.10.22 01:23:25.740467 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: acf5173e143d7d63a317efd61c5329c2, offset: 0, size: 680 2024.10.22 01:23:25.747309 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key acf5173e143d7d63a317efd61c5329c2 for removal 2024.10.22 01:23:25.753205 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dc7cf04e68171296a07bc8becec5025d, offset: 0, size: 516 2024.10.22 01:23:25.755998 [ 1476 ] {} CacheMetadata: Key acf5173e143d7d63a317efd61c5329c2 is removed from metadata 2024.10.22 01:23:25.756964 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dc7cf04e68171296a07bc8becec5025d for removal 2024.10.22 01:23:25.763380 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 654009bd98fb82526d0223a724d294c8, offset: 0, size: 867453 2024.10.22 01:23:25.771015 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 654009bd98fb82526d0223a724d294c8 for removal 2024.10.22 01:23:25.780088 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: db4a202d6e86714e47e4c2bc374bd1c5, offset: 0, size: 527 2024.10.22 01:23:25.782945 [ 1476 ] {} CacheMetadata: Key dc7cf04e68171296a07bc8becec5025d is removed from metadata 2024.10.22 01:23:25.787225 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key db4a202d6e86714e47e4c2bc374bd1c5 for removal 2024.10.22 01:23:25.799676 [ 1476 ] {} CacheMetadata: Key 654009bd98fb82526d0223a724d294c8 is removed from metadata 2024.10.22 01:23:25.814295 [ 1476 ] {} CacheMetadata: Key 5cc25574a8422260595dcaddfb48e57b is removed from metadata 2024.10.22 01:23:25.825327 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 51a7e4dacce55a8ce7927fc7724baaf8, offset: 0, size: 717000 2024.10.22 01:23:25.832323 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 51a7e4dacce55a8ce7927fc7724baaf8 for removal 2024.10.22 01:23:25.859776 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.96 GiB, free memory in arenas 9.09 MiB, will set to 1.53 GiB (RSS), difference: -75.90 MiB 2024.10.22 01:23:25.829836 [ 1476 ] {} CacheMetadata: Key db4a202d6e86714e47e4c2bc374bd1c5 is removed from metadata 2024.10.22 01:23:25.940377 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: beebf6d1365afb7e78aa3d1fe5b3af95, offset: 0, size: 695210 2024.10.22 01:23:25.948814 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key beebf6d1365afb7e78aa3d1fe5b3af95 for removal 2024.10.22 01:23:25.953633 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: db5569dcaee885dc70865b94b0dbb7e8, offset: 0, size: 526 2024.10.22 01:23:25.944938 [ 1476 ] {} CacheMetadata: Key 51a7e4dacce55a8ce7927fc7724baaf8 is removed from metadata 2024.10.22 01:23:25.958707 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key db5569dcaee885dc70865b94b0dbb7e8 for removal 2024.10.22 01:23:25.962388 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: aba539cefa692bca6942df225bc58847, offset: 0, size: 563367 2024.10.22 01:23:25.963695 [ 1476 ] {} CacheMetadata: Key db5569dcaee885dc70865b94b0dbb7e8 is removed from metadata 2024.10.22 01:23:25.968214 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key aba539cefa692bca6942df225bc58847 for removal 2024.10.22 01:23:25.971304 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c11ed3621440e917db0c70c04cefc2ae, offset: 0, size: 531 2024.10.22 01:23:25.973361 [ 1476 ] {} CacheMetadata: Key aba539cefa692bca6942df225bc58847 is removed from metadata 2024.10.22 01:23:25.980383 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c11ed3621440e917db0c70c04cefc2ae for removal 2024.10.22 01:23:26.084623 [ 1476 ] {} CacheMetadata: Key c11ed3621440e917db0c70c04cefc2ae is removed from metadata 2024.10.22 01:23:26.104889 [ 1476 ] {} CacheMetadata: Key beebf6d1365afb7e78aa3d1fe5b3af95 is removed from metadata 2024.10.22 01:23:26.368511 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_708_326, 202410_709_709_0] 2024.10.22 01:23:26.372051 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_708_326, 202410_709_709_0] 2024.10.22 01:23:26.380127 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_708_326, 202410_709_709_0] 2024.10.22 01:23:26.531231 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_806_409} MergeTask::MergeProjectionsStage: Merge sorted 53084 rows, containing 13 columns (13 merged, 0 gathered) in 3.431562129 sec., 15469.339619816046 rows/sec., 2.47 MiB/sec. 2024.10.22 01:23:26.713605 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_806_409} MergedBlockOutputStream: filled checksums 202410_1_806_409 (state Temporary) 2024.10.22 01:23:26.820883 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_806_409} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_806_409 to 202410_1_806_409 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:26.851296 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_806_409} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_800_408, 202410_801_806_1] -> 202410_1_806_409 2024.10.22 01:23:26.893325 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.09 MiB. 2024.10.22 01:23:27.047425 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 52609 2024.10.22 01:23:27.425213 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.96 GiB, free memory in arenas 10.10 MiB, will set to 1.53 GiB (RSS), difference: -103.40 MiB 2024.10.22 01:23:27.484954 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:27.487714 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:27.684145 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_799_799_0 (state Temporary) 2024.10.22 01:23:27.761916 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_799_799_0 to 202410_812_812_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:27.908095 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52609 2024.10.22 01:23:27.952900 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_806_409 to 202410_807_807_0 2024.10.22 01:23:27.961028 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.91 MiB using storage policy from min volume index 0 2024.10.22 01:23:27.964230 [ 1198 ] {} DiskLocal: Reserved 1.91 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:27.992456 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 7313 2024.10.22 01:23:27.979765 [ 1198 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 263 ms. 2024.10.22 01:23:28.055374 [ 1238 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Selected 5 parts from 202410_1_21_5 to 202410_25_25_0 2024.10.22 01:23:28.064140 [ 1238 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:28.055720 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_95_95_0 to 202410_98_98_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:28.067048 [ 1238 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:28.083841 [ 1423 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Found 5 old parts to remove. Parts: [202410_1_17_4, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 01:23:28.092777 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_807_410} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_806_409 to 202410_807_807_0 into Compact with storage Full 2024.10.22 01:23:28.099789 [ 1423 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from filesystem (serially): Parts: [202410_1_17_4, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 01:23:28.130828 [ 1423 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from memory: Parts: [202410_1_17_4, 202410_18_18_0, 202410_19_19_0, 202410_20_20_0, 202410_21_21_0] 2024.10.22 01:23:28.201222 [ 1390 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_25_6} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_21_5 to 202410_25_25_0 into Compact with storage Full 2024.10.22 01:23:28.223786 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_807_410} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:28.253543 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_807_410} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_806_409, total 53084 rows starting from the beginning of the part 2024.10.22 01:23:28.271497 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:28.274628 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:28.324446 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_472_472_0 (state Temporary) 2024.10.22 01:23:28.340650 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_807_410} MergeTreeSequentialSource: Reading 2 marks from part 202410_807_807_0, total 78 rows starting from the beginning of the part 2024.10.22 01:23:28.380284 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_472_472_0 to 202410_485_485_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:28.404437 [ 1390 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_25_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:28.427709 [ 1390 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_25_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_21_5, total 21 rows starting from the beginning of the part 2024.10.22 01:23:28.435104 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7313 2024.10.22 01:23:28.490850 [ 1390 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_25_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_22_22_0, total 1 rows starting from the beginning of the part 2024.10.22 01:23:28.555313 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1694 entries to flush up to offset 713451 2024.10.22 01:23:28.600885 [ 1390 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_25_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_23_23_0, total 1 rows starting from the beginning of the part 2024.10.22 01:23:28.652610 [ 1390 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_25_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_24_24_0, total 1 rows starting from the beginning of the part 2024.10.22 01:23:28.690189 [ 1390 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_25_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_25_25_0, total 1 rows starting from the beginning of the part 2024.10.22 01:23:29.007486 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vhe/zswtobyhtxvftssjaxiytaecqkrud (LogSeriesLimiter: on interval from 2024-10-22 01:23:24 to 2024-10-22 01:23:29 accepted series 1 / 195 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:23:29.393373 [ 1721 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mvy/rwideehywmuyubpkarbzcxnofbsie, size 1166208 2024.10.22 01:23:29.516544 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:29.520643 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.25 GiB. 2024.10.22 01:23:29.578268 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.96 GiB, free memory in arenas 9.84 MiB, will set to 1.54 GiB (RSS), difference: -178.60 MiB 2024.10.22 01:23:29.643364 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_453_453_0 (state Temporary) 2024.10.22 01:23:29.769323 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_453_453_0 to 202410_460_460_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:29.813039 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 713451 2024.10.22 01:23:29.947575 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_421_84, 202410_422_422_0, 202410_423_423_0, 202410_424_424_0, 202410_425_425_0, 202410_426_426_0] 2024.10.22 01:23:30.057135 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_421_84, 202410_422_422_0, 202410_423_423_0, 202410_424_424_0, 202410_425_425_0, 202410_426_426_0] 2024.10.22 01:23:30.179187 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Robotness 2024.10.22 01:23:30.183350 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_421_84, 202410_422_422_0, 202410_423_423_0, 202410_424_424_0, 202410_425_425_0, 202410_426_426_0] 2024.10.22 01:23:30.191081 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Robotness 2024.10.22 01:23:30.205814 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:23:30.209197 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:23:30.221066 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Robotness 2024.10.22 01:23:30.244803 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Robotness 2024.10.22 01:23:30.261744 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Robotness 2024.10.22 01:23:30.265757 [ 1389 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_25_6} MergeTask::MergeProjectionsStage: Merge sorted 25 rows, containing 27 columns (27 merged, 0 gathered) in 2.177587287 sec., 11.480596047399683 rows/sec., 37.28 KiB/sec. 2024.10.22 01:23:30.275582 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Robotness 2024.10.22 01:23:30.301091 [ 1398 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_25_6} MergedBlockOutputStream: filled checksums 202410_1_25_6 (state Temporary) 2024.10.22 01:23:30.321420 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Robotness 2024.10.22 01:23:30.452036 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Robotness 2024.10.22 01:23:30.479498 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Robotness 2024.10.22 01:23:30.862645 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 4695489 bytes for 0da1bb5a82e7fa189fa7dc9a2bff1627:0) 2024.10.22 01:23:30.865191 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a5395ab220e2220f2f6d06777af4a0ea, offset: 0, size: 899 2024.10.22 01:23:30.870438 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a5395ab220e2220f2f6d06777af4a0ea for removal 2024.10.22 01:23:30.878677 [ 1476 ] {} CacheMetadata: Key a5395ab220e2220f2f6d06777af4a0ea is removed from metadata 2024.10.22 01:23:30.879322 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 54c18d418786ac23a2410efb9290fff9, offset: 0, size: 1233357 2024.10.22 01:23:30.885689 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 54c18d418786ac23a2410efb9290fff9 for removal 2024.10.22 01:23:30.889239 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ce9ad911b1b3578a3b7aca47a87b65a6, offset: 0, size: 1872138 2024.10.22 01:23:30.895567 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ce9ad911b1b3578a3b7aca47a87b65a6 for removal 2024.10.22 01:23:30.907821 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3db5a497b55e20c554d8f3947b44e855, offset: 0, size: 682 2024.10.22 01:23:30.915047 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3db5a497b55e20c554d8f3947b44e855 for removal 2024.10.22 01:23:30.918506 [ 1476 ] {} CacheMetadata: Key ce9ad911b1b3578a3b7aca47a87b65a6 is removed from metadata 2024.10.22 01:23:30.921583 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5489d8815b525c3c9e94d1f1f58314af, offset: 0, size: 1839533 2024.10.22 01:23:30.931609 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5489d8815b525c3c9e94d1f1f58314af for removal 2024.10.22 01:23:30.935846 [ 1476 ] {} CacheMetadata: Key 3db5a497b55e20c554d8f3947b44e855 is removed from metadata 2024.10.22 01:23:30.941577 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 881ef692f68b82bf415cbff5f4c0d0f3, offset: 0, size: 50972 2024.10.22 01:23:30.948751 [ 1476 ] {} CacheMetadata: Key 5489d8815b525c3c9e94d1f1f58314af is removed from metadata 2024.10.22 01:23:30.953529 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 881ef692f68b82bf415cbff5f4c0d0f3 for removal 2024.10.22 01:23:30.956565 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ed0569152e5d7bc322fb6ed7ac8e1258, offset: 0, size: 827 2024.10.22 01:23:30.956687 [ 1476 ] {} CacheMetadata: Key 881ef692f68b82bf415cbff5f4c0d0f3 is removed from metadata 2024.10.22 01:23:30.960432 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ed0569152e5d7bc322fb6ed7ac8e1258 for removal 2024.10.22 01:23:30.965745 [ 1476 ] {} CacheMetadata: Key ed0569152e5d7bc322fb6ed7ac8e1258 is removed from metadata 2024.10.22 01:23:30.966483 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 71541bcee26be8abce10fc6fc69e1a16, offset: 0, size: 899 2024.10.22 01:23:30.971496 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 71541bcee26be8abce10fc6fc69e1a16 for removal 2024.10.22 01:23:30.974663 [ 1476 ] {} CacheMetadata: Key 54c18d418786ac23a2410efb9290fff9 is removed from metadata 2024.10.22 01:23:31.082532 [ 1476 ] {} CacheMetadata: Key 71541bcee26be8abce10fc6fc69e1a16 is removed from metadata 2024.10.22 01:23:31.278685 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3795 2024.10.22 01:23:31.310976 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_25_6} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_merge_202410_1_25_6 to 202410_1_25_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:31.315081 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_25_6} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Merged 5 parts: [202410_1_21_5, 202410_25_25_0] -> 202410_1_25_6 2024.10.22 01:23:31.469502 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.24 MiB. 2024.10.22 01:23:31.510265 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MemoryTracker: Current memory usage (for query): 1.00 GiB. 2024.10.22 01:23:31.769083 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.96 GiB, free memory in arenas 16.62 MiB, will set to 1.54 GiB (RSS), difference: -92.02 MiB 2024.10.22 01:23:31.923790 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:23:31.929564 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 91 entries to flush up to offset 52700 2024.10.22 01:23:32.053736 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_807_410} MergeTask::MergeProjectionsStage: Merge sorted 53162 rows, containing 13 columns (13 merged, 0 gathered) in 3.966358476 sec., 13403.226239301724 rows/sec., 2.14 MiB/sec. 2024.10.22 01:23:32.067568 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:23:32.084788 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:23:32.272010 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:32.277274 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:32.325508 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:23:32.369201 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:23:32.462261 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_807_410} MergedBlockOutputStream: filled checksums 202410_1_807_410 (state Temporary) 2024.10.22 01:23:32.496175 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:23:32.579977 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_800_800_0 (state Temporary) 2024.10.22 01:23:32.599526 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_807_410} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_807_410 to 202410_1_807_410 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:32.605210 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_807_410} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_806_409, 202410_807_807_0] -> 202410_1_807_410 2024.10.22 01:23:32.606066 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_407_185, 202410_408_408_0] 2024.10.22 01:23:32.628596 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:23:32.611103 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_407_185, 202410_408_408_0] 2024.10.22 01:23:32.662956 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:23:32.713744 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_407_185, 202410_408_408_0] 2024.10.22 01:23:32.768442 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_800_800_0 to 202410_813_813_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:32.812324 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_807_410 to 202410_808_808_0 2024.10.22 01:23:32.821487 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.91 MiB using storage policy from min volume index 0 2024.10.22 01:23:32.832027 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 400 2024.10.22 01:23:32.833688 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:23:32.835439 [ 1269 ] {} DiskLocal: Reserved 1.91 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:32.852407 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52700 2024.10.22 01:23:32.870475 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_808_411} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_807_410 to 202410_808_808_0 into Compact with storage Full 2024.10.22 01:23:32.905600 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_808_411} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:32.987039 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 1083 entries to flush up to offset 54671 2024.10.22 01:23:33.032269 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_808_411} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_807_410, total 53162 rows starting from the beginning of the part 2024.10.22 01:23:33.074785 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_709_327, 202410_710_710_0] 2024.10.22 01:23:33.077655 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_709_327, 202410_710_710_0] 2024.10.22 01:23:33.078989 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_808_411} MergeTreeSequentialSource: Reading 2 marks from part 202410_808_808_0, total 46 rows starting from the beginning of the part 2024.10.22 01:23:33.194424 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_709_327, 202410_710_710_0] 2024.10.22 01:23:33.510091 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:33.524614 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:33.628472 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_396_79, 202410_397_397_0, 202410_398_398_0, 202410_399_399_0, 202410_400_400_0, 202410_401_401_0] 2024.10.22 01:23:33.630795 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_396_79, 202410_397_397_0, 202410_398_398_0, 202410_399_399_0, 202410_400_400_0, 202410_401_401_0] 2024.10.22 01:23:33.653226 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.96 GiB, free memory in arenas 16.67 MiB, will set to 1.52 GiB (RSS), difference: -85.71 MiB 2024.10.22 01:23:33.662107 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_396_79, 202410_397_397_0, 202410_398_398_0, 202410_399_399_0, 202410_400_400_0, 202410_401_401_0] 2024.10.22 01:23:33.822659 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_382_382_0 (state Temporary) 2024.10.22 01:23:33.978111 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_459_92/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:23:34.162239 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_382_382_0 to 202410_392_392_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:34.374873 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:34.378699 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:34.574694 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 400 2024.10.22 01:23:34.606515 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_464_464_0 (state Temporary) 2024.10.22 01:23:34.917652 [ 1190 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:23:34.976346 [ 1190 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:23:35.025830 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_464_464_0 to 202410_470_470_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:35.076228 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 54671 2024.10.22 01:23:35.942248 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 7328 2024.10.22 01:23:36.079516 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.96 GiB, free memory in arenas 12.42 MiB, will set to 1.53 GiB (RSS), difference: 11.21 MiB 2024.10.22 01:23:36.272324 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:36.278479 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:36.344975 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_473_473_0 (state Temporary) 2024.10.22 01:23:36.504152 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_473_473_0 to 202410_486_486_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:36.530531 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_808_411} MergeTask::MergeProjectionsStage: Merge sorted 53208 rows, containing 13 columns (13 merged, 0 gathered) in 3.665367597 sec., 14516.415773290855 rows/sec., 2.32 MiB/sec. 2024.10.22 01:23:36.536443 [ 1173 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_481_96 to 202410_486_486_0 2024.10.22 01:23:36.563217 [ 1173 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:36.567257 [ 1173 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:36.605080 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_481_96 to 202410_486_486_0 into Compact with storage Full 2024.10.22 01:23:36.678614 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_808_411} MergedBlockOutputStream: filled checksums 202410_1_808_411 (state Temporary) 2024.10.22 01:23:36.710729 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:36.723230 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_481_96, total 7342 rows starting from the beginning of the part 2024.10.22 01:23:36.773385 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_482_482_0, total 48 rows starting from the beginning of the part 2024.10.22 01:23:36.811659 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_483_483_0, total 13 rows starting from the beginning of the part 2024.10.22 01:23:36.821993 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1410 entries to flush up to offset 714861 2024.10.22 01:23:36.848974 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7328 2024.10.22 01:23:36.869164 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 52769 2024.10.22 01:23:36.876355 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_484_484_0, total 6 rows starting from the beginning of the part 2024.10.22 01:23:36.958230 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_485_485_0, total 10 rows starting from the beginning of the part 2024.10.22 01:23:37.217544 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_486_486_0, total 15 rows starting from the beginning of the part 2024.10.22 01:23:37.366101 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_710_328, 202410_711_711_0] 2024.10.22 01:23:37.366771 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_808_411} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_808_411 to 202410_1_808_411 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:37.368857 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_710_328, 202410_711_711_0] 2024.10.22 01:23:37.371045 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_808_411} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_807_410, 202410_808_808_0] -> 202410_1_808_411 2024.10.22 01:23:37.524437 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_710_328, 202410_711_711_0] 2024.10.22 01:23:37.564544 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:23:37.584541 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:37.591332 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:37.629081 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:37.633367 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:37.768353 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_454_454_0 (state Temporary) 2024.10.22 01:23:37.855327 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_801_801_0 (state Temporary) 2024.10.22 01:23:37.861631 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_454_454_0 to 202410_461_461_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:37.899121 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 714861 2024.10.22 01:23:38.015782 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_801_801_0 to 202410_814_814_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:38.050648 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_808_411 to 202410_809_809_0 2024.10.22 01:23:38.061245 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.91 MiB using storage policy from min volume index 0 2024.10.22 01:23:38.064923 [ 1182 ] {} DiskLocal: Reserved 1.91 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:38.070210 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52769 2024.10.22 01:23:38.097422 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_809_412} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_808_411 to 202410_809_809_0 into Compact with storage Full 2024.10.22 01:23:38.286784 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_809_412} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:38.365353 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_809_412} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_808_411, total 53208 rows starting from the beginning of the part 2024.10.22 01:23:38.559288 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} MergeTask::MergeProjectionsStage: Merge sorted 7434 rows, containing 12 columns (12 merged, 0 gathered) in 1.968064262 sec., 3777.3156819815267 rows/sec., 599.81 KiB/sec. 2024.10.22 01:23:38.591540 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_339_70, 202410_340_340_0, 202410_341_341_0, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0] 2024.10.22 01:23:38.613667 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_339_70, 202410_340_340_0, 202410_341_341_0, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0] 2024.10.22 01:23:38.653315 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_809_412} MergeTreeSequentialSource: Reading 2 marks from part 202410_809_809_0, total 50 rows starting from the beginning of the part 2024.10.22 01:23:38.697254 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} MergedBlockOutputStream: filled checksums 202410_1_486_97 (state Temporary) 2024.10.22 01:23:38.751772 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_339_70, 202410_340_340_0, 202410_341_341_0, 202410_342_342_0, 202410_343_343_0, 202410_344_344_0] 2024.10.22 01:23:38.760805 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.96 GiB, free memory in arenas 10.89 MiB, will set to 1.54 GiB (RSS), difference: -7.86 MiB 2024.10.22 01:23:38.855771 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 3837 2024.10.22 01:23:38.856708 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_486_97 to 202410_1_486_97 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:38.860340 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_486_97} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_481_96, 202410_486_486_0] -> 202410_1_486_97 2024.10.22 01:23:38.986610 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.46 MiB. 2024.10.22 01:23:40.211297 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:23:40.213627 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:23:40.475688 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.96 GiB, free memory in arenas 9.73 MiB, will set to 1.56 GiB (RSS), difference: -1.90 MiB 2024.10.22 01:23:41.071501 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:23:41.616448 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_809_412} MergeTask::MergeProjectionsStage: Merge sorted 53258 rows, containing 13 columns (13 merged, 0 gathered) in 3.529658619 sec., 15088.7113312643 rows/sec., 2.41 MiB/sec. 2024.10.22 01:23:41.664631 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_408_186, 202410_409_409_0] 2024.10.22 01:23:41.667450 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_408_186, 202410_409_409_0] 2024.10.22 01:23:41.672232 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_711_329, 202410_712_712_0] 2024.10.22 01:23:41.674337 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_711_329, 202410_712_712_0] 2024.10.22 01:23:41.680235 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_809_412} MergedBlockOutputStream: filled checksums 202410_1_809_412 (state Temporary) 2024.10.22 01:23:41.682088 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_408_186, 202410_409_409_0] 2024.10.22 01:23:41.685342 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_711_329, 202410_712_712_0] 2024.10.22 01:23:42.075930 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 401 2024.10.22 01:23:42.076817 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_809_412} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_809_412 to 202410_1_809_412 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:42.077142 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 52818 2024.10.22 01:23:42.079846 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_809_412} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_808_411, 202410_809_809_0] -> 202410_1_809_412 2024.10.22 01:23:42.208094 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:23:42.444455 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:42.448388 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.24 GiB. 2024.10.22 01:23:42.477635 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.96 GiB, free memory in arenas 11.63 MiB, will set to 1.56 GiB (RSS), difference: 12.84 MiB 2024.10.22 01:23:42.512590 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_383_383_0 (state Temporary) 2024.10.22 01:23:42.536078 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:42.640411 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:42.752012 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_383_383_0 to 202410_393_393_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:42.774383 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_802_802_0 (state Temporary) 2024.10.22 01:23:42.801049 [ 1200 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_388_80 to 202410_393_393_0 2024.10.22 01:23:42.819396 [ 1200 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:42.824317 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:42.851027 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 401 2024.10.22 01:23:42.916796 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_388_80 to 202410_393_393_0 into Compact with storage Full 2024.10.22 01:23:42.954390 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_802_802_0 to 202410_815_815_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:42.974613 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 206 entries to flush up to offset 54877 2024.10.22 01:23:43.036643 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_809_412 to 202410_810_810_0 2024.10.22 01:23:43.053064 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.91 MiB using storage policy from min volume index 0 2024.10.22 01:23:43.059638 [ 1209 ] {} DiskLocal: Reserved 1.91 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:43.074440 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_810_413} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_809_412 to 202410_810_810_0 into Compact with storage Full 2024.10.22 01:23:43.082358 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:43.101629 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_388_80, total 432 rows starting from the beginning of the part 2024.10.22 01:23:43.167752 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52818 2024.10.22 01:23:43.183958 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_389_389_0, total 1 rows starting from the beginning of the part 2024.10.22 01:23:43.312909 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_810_413} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:43.460925 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_390_390_0, total 1 rows starting from the beginning of the part 2024.10.22 01:23:43.460978 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_810_413} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_809_412, total 53258 rows starting from the beginning of the part 2024.10.22 01:23:43.537741 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_391_391_0, total 1 rows starting from the beginning of the part 2024.10.22 01:23:43.632201 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_810_413} MergeTreeSequentialSource: Reading 2 marks from part 202410_810_810_0, total 53 rows starting from the beginning of the part 2024.10.22 01:23:43.689535 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_392_392_0, total 1 rows starting from the beginning of the part 2024.10.22 01:23:43.848916 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_393_393_0, total 1 rows starting from the beginning of the part 2024.10.22 01:23:44.143633 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:44.152486 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:44.354279 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 7346 2024.10.22 01:23:44.365619 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_465_465_0 (state Temporary) 2024.10.22 01:23:44.659658 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.96 GiB, free memory in arenas 14.23 MiB, will set to 1.56 GiB (RSS), difference: -10.46 MiB 2024.10.22 01:23:44.713639 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:44.718916 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:44.724677 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_465_465_0 to 202410_471_471_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:44.762304 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_461_466_1 to 202410_471_471_0 2024.10.22 01:23:44.764298 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_474_474_0 (state Temporary) 2024.10.22 01:23:44.776591 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:44.781637 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:44.770579 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} MergeTask::MergeProjectionsStage: Merge sorted 725347 rows, containing 4 columns (4 merged, 0 gathered) in 23.185854648 sec., 31284.03119108521 rows/sec., 469.15 KiB/sec. 2024.10.22 01:23:44.845263 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_474_474_0 to 202410_487_487_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:44.898139 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 54877 2024.10.22 01:23:44.902945 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1342 entries to flush up to offset 716203 2024.10.22 01:23:44.919182 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} MergeTask::PrepareStage: Merging 6 parts: from 202410_461_466_1 to 202410_471_471_0 into Compact with storage Full 2024.10.22 01:23:44.929628 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7346 2024.10.22 01:23:44.940474 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:44.955712 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_466_1, total 901 rows starting from the beginning of the part 2024.10.22 01:23:44.968247 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} MergedBlockOutputStream: filled checksums 202410_1_459_92 (state Temporary) 2024.10.22 01:23:44.988928 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_467_467_0, total 146 rows starting from the beginning of the part 2024.10.22 01:23:45.131948 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_468_468_0, total 205 rows starting from the beginning of the part 2024.10.22 01:23:45.161655 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_469_469_0, total 168 rows starting from the beginning of the part 2024.10.22 01:23:45.187359 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_470_470_0, total 1083 rows starting from the beginning of the part 2024.10.22 01:23:45.213017 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_471_471_0, total 206 rows starting from the beginning of the part 2024.10.22 01:23:45.783503 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:45.788108 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:46.150591 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_455_455_0 (state Temporary) 2024.10.22 01:23:46.160213 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} MergeTask::MergeProjectionsStage: Merge sorted 437 rows, containing 32 columns (32 merged, 0 gathered) in 3.310383575 sec., 132.00887150970112 rows/sec., 12.33 KiB/sec. 2024.10.22 01:23:46.187700 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} MergedBlockOutputStream: filled checksums 202410_1_393_81 (state Temporary) 2024.10.22 01:23:46.253216 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_455_455_0 to 202410_462_462_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:46.311062 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 716203 2024.10.22 01:23:46.560961 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.96 GiB, free memory in arenas 6.81 MiB, will set to 1.56 GiB (RSS), difference: 8.62 MiB 2024.10.22 01:23:47.058021 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} MergeTask::MergeProjectionsStage: Merge sorted 2709 rows, containing 13 columns (13 merged, 0 gathered) in 2.204239433 sec., 1228.995343901009 rows/sec., 549.37 KiB/sec. 2024.10.22 01:23:47.094469 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} MergedBlockOutputStream: filled checksums 202410_461_471_2 (state Temporary) 2024.10.22 01:23:47.174539 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 52889 2024.10.22 01:23:47.272562 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_810_413} MergeTask::MergeProjectionsStage: Merge sorted 53311 rows, containing 13 columns (13 merged, 0 gathered) in 4.204304329 sec., 12680.100161226934 rows/sec., 2.03 MiB/sec. 2024.10.22 01:23:47.368457 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_461_471_2 to 202410_461_471_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:47.368454 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_810_413} MergedBlockOutputStream: filled checksums 202410_1_810_413 (state Temporary) 2024.10.22 01:23:47.371477 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:47.375959 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:47.381715 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_471_2} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_461_466_1, 202410_471_471_0] -> 202410_461_471_2 2024.10.22 01:23:47.434743 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_810_413} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_810_413 to 202410_1_810_413 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:47.439505 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_393_81 to 202410_1_393_81 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:47.451779 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_393_81} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_388_80, 202410_393_393_0] -> 202410_1_393_81 2024.10.22 01:23:47.483677 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_810_413} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_809_412, 202410_810_810_0] -> 202410_1_810_413 2024.10.22 01:23:47.558764 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:23:47.601089 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:23:47.603595 [ 1271 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:23:47.633793 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_803_803_0 (state Temporary) 2024.10.22 01:23:47.647091 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_459_92 to 202410_1_459_92 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:47.652178 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_459_92} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_454_91, 202410_459_459_0] -> 202410_1_459_92 2024.10.22 01:23:47.649839 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.05 MiB. 2024.10.22 01:23:47.879658 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:23:47.988044 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_803_803_0 to 202410_816_816_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:48.055431 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 01:23:48.059366 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52889 2024.10.22 01:23:48.060328 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_810_413 to 202410_811_811_0 2024.10.22 01:23:48.067570 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.91 MiB using storage policy from min volume index 0 2024.10.22 01:23:48.082605 [ 1266 ] {} DiskLocal: Reserved 1.91 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:48.131845 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_811_414} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_810_413 to 202410_811_811_0 into Compact with storage Full 2024.10.22 01:23:48.169025 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_811_414} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:48.175919 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_811_414} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_810_413, total 53311 rows starting from the beginning of the part 2024.10.22 01:23:48.288956 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_811_414} MergeTreeSequentialSource: Reading 2 marks from part 202410_811_811_0, total 42 rows starting from the beginning of the part 2024.10.22 01:23:48.394095 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column Title 2024.10.22 01:23:48.429548 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column Title 2024.10.22 01:23:48.564025 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column Title 2024.10.22 01:23:48.579753 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column Title 2024.10.22 01:23:48.609515 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column Title 2024.10.22 01:23:48.731933 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column Title 2024.10.22 01:23:48.751789 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column Title 2024.10.22 01:23:48.770164 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column Title 2024.10.22 01:23:48.864114 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/izl/afuseuleisjxiesaxxmamsfdzqfoj (LogSeriesLimiter: on interval from 2024-10-22 01:23:29 to 2024-10-22 01:23:48 accepted series 1 / 166 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:23:49.324411 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_811_414/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:23:49.494919 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_712_330, 202410_713_713_0] 2024.10.22 01:23:49.600593 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_712_330, 202410_713_713_0] 2024.10.22 01:23:49.614923 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_712_330, 202410_713_713_0] 2024.10.22 01:23:49.770549 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_409_187, 202410_410_410_0] 2024.10.22 01:23:49.773215 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_409_187, 202410_410_410_0] 2024.10.22 01:23:49.798995 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_409_187, 202410_410_410_0] 2024.10.22 01:23:49.981055 [ 1181 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:23:50.057467 [ 1181 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:23:50.209792 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:23:50.213640 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:23:50.353910 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 402 2024.10.22 01:23:50.859485 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:50.864440 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:50.961641 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_384_384_0 (state Temporary) 2024.10.22 01:23:51.120685 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_714_714_0, 202410_715_715_0, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0, 202410_719_719_0] 2024.10.22 01:23:51.127388 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_714_714_0, 202410_715_715_0, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0, 202410_719_719_0] 2024.10.22 01:23:51.150881 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_384_384_0 to 202410_394_394_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:51.333558 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 402 2024.10.22 01:23:51.375314 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_714_714_0, 202410_715_715_0, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0, 202410_719_719_0] 2024.10.22 01:23:51.771523 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 531486 bytes for a3bf8db61fe9ad20a39381a00b92533b:0) 2024.10.22 01:23:51.776892 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 98c5efea24b02947ffb0737489be9d6e, offset: 0, size: 2363042 2024.10.22 01:23:51.784104 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 98c5efea24b02947ffb0737489be9d6e for removal 2024.10.22 01:23:51.788106 [ 1476 ] {} CacheMetadata: Key 98c5efea24b02947ffb0737489be9d6e is removed from metadata 2024.10.22 01:23:51.788200 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e56424aa857fdb0c92ea9552cbb3409d, offset: 0, size: 910 2024.10.22 01:23:51.796450 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e56424aa857fdb0c92ea9552cbb3409d for removal 2024.10.22 01:23:51.813916 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.96 GiB, free memory in arenas 5.30 MiB, will set to 1.60 GiB (RSS), difference: 3.95 MiB 2024.10.22 01:23:51.833054 [ 1476 ] {} CacheMetadata: Key e56424aa857fdb0c92ea9552cbb3409d is removed from metadata 2024.10.22 01:23:51.933366 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_811_414} MergeTask::MergeProjectionsStage: Merge sorted 53353 rows, containing 13 columns (13 merged, 0 gathered) in 3.820980838 sec., 13963.168689410737 rows/sec., 2.23 MiB/sec. 2024.10.22 01:23:52.035530 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_811_414} MergedBlockOutputStream: filled checksums 202410_1_811_414 (state Temporary) 2024.10.22 01:23:52.143438 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:23:52.199434 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 52957 2024.10.22 01:23:52.217578 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:23:52.249012 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:23:52.312632 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_811_414} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_811_414 to 202410_1_811_414 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:52.320147 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_811_414} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_810_413, 202410_811_811_0] -> 202410_1_811_414 2024.10.22 01:23:52.363721 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:23:52.368296 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:23:52.385695 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:23:52.433666 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:23:52.434601 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 32 entries to flush up to offset 7378 2024.10.22 01:23:52.458423 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:23:52.484709 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:23:52.524752 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:23:52.563579 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:52.565783 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:52.678382 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 166 entries to flush up to offset 55043 2024.10.22 01:23:52.718011 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_804_804_0 (state Temporary) 2024.10.22 01:23:52.736183 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:52.744443 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:52.786200 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_804_804_0 to 202410_817_817_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:52.847349 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_811_414 to 202410_812_812_0 2024.10.22 01:23:52.866706 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.92 MiB using storage policy from min volume index 0 2024.10.22 01:23:52.872454 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:52.873008 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 52957 2024.10.22 01:23:52.875369 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:52.882765 [ 1265 ] {} DiskLocal: Reserved 1.92 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:52.921725 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_475_475_0 (state Temporary) 2024.10.22 01:23:53.053077 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_812_415} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_811_414 to 202410_812_812_0 into Compact with storage Full 2024.10.22 01:23:53.077985 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_466_466_0 (state Temporary) 2024.10.22 01:23:53.079414 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_475_475_0 to 202410_488_488_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:53.161303 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7378 2024.10.22 01:23:53.239072 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_812_415} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:53.270670 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_466_466_0 to 202410_472_472_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:53.275594 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_812_415} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_811_414, total 53353 rows starting from the beginning of the part 2024.10.22 01:23:53.321672 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 55043 2024.10.22 01:23:53.455537 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_812_415} MergeTreeSequentialSource: Reading 2 marks from part 202410_812_812_0, total 87 rows starting from the beginning of the part 2024.10.22 01:23:53.523429 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1483 entries to flush up to offset 717686 2024.10.22 01:23:54.574314 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.96 GiB, free memory in arenas 5.38 MiB, will set to 1.60 GiB (RSS), difference: -16.84 MiB 2024.10.22 01:23:54.948370 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:54.953921 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:55.115210 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_456_456_0 (state Temporary) 2024.10.22 01:23:55.492024 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_456_456_0 to 202410_463_463_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:55.570042 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 717686 2024.10.22 01:23:56.110379 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 2097152 bytes for 205ad1f9404ca48beb078c3585fdafb0:0) 2024.10.22 01:23:56.114235 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: da927899838a9d36d13e21a9e101cd4b, offset: 0, size: 2860177 2024.10.22 01:23:56.126172 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key da927899838a9d36d13e21a9e101cd4b for removal 2024.10.22 01:23:56.133917 [ 1476 ] {} CacheMetadata: Key da927899838a9d36d13e21a9e101cd4b is removed from metadata 2024.10.22 01:23:56.236525 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b916d2bc80c2eafe1c20d1a0059e1e38, offset: 0, size: 920 2024.10.22 01:23:56.246688 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b916d2bc80c2eafe1c20d1a0059e1e38 for removal 2024.10.22 01:23:56.257430 [ 1476 ] {} CacheMetadata: Key b916d2bc80c2eafe1c20d1a0059e1e38 is removed from metadata 2024.10.22 01:23:56.301462 [ 1169 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 01:23:56.507319 [ 1178 ] {} DatabaseCatalog: Execution took 329 ms. 2024.10.22 01:23:56.646433 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:56.652751 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:56.866850 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.96 GiB, free memory in arenas 5.59 MiB, will set to 1.62 GiB (RSS), difference: 1.71 MiB 2024.10.22 01:23:56.880325 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 53013 2024.10.22 01:23:57.056837 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_812_415} MergeTask::MergeProjectionsStage: Merge sorted 53440 rows, containing 13 columns (13 merged, 0 gathered) in 4.026134171 sec., 13273.278467698638 rows/sec., 2.12 MiB/sec. 2024.10.22 01:23:57.223815 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_812_415} MergedBlockOutputStream: filled checksums 202410_1_812_415 (state Temporary) 2024.10.22 01:23:57.303110 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_812_415} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_812_415 to 202410_1_812_415 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:57.311493 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_812_415} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_811_414, 202410_812_812_0] -> 202410_1_812_415 2024.10.22 01:23:57.346935 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:23:57.363017 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:23:57.366478 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:57.421279 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_805_805_0 (state Temporary) 2024.10.22 01:23:57.469765 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_805_805_0 to 202410_818_818_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:23:57.627050 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 53013 2024.10.22 01:23:57.708408 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_812_415 to 202410_813_813_0 2024.10.22 01:23:57.818887 [ 1182 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.92 MiB using storage policy from min volume index 0 2024.10.22 01:23:57.824630 [ 1182 ] {} DiskLocal: Reserved 1.92 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:23:57.831205 [ 1182 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 246 ms. 2024.10.22 01:23:57.968455 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_813_416} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_812_415 to 202410_813_813_0 into Compact with storage Full 2024.10.22 01:23:58.084847 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_813_416} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:23:58.305755 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_813_416} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_812_415, total 53440 rows starting from the beginning of the part 2024.10.22 01:23:58.329663 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_813_416} MergeTreeSequentialSource: Reading 2 marks from part 202410_813_813_0, total 91 rows starting from the beginning of the part 2024.10.22 01:23:58.682397 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_410_188, 202410_411_411_0] 2024.10.22 01:23:58.685322 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_410_188, 202410_411_411_0] 2024.10.22 01:23:58.702304 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_410_188, 202410_411_411_0] 2024.10.22 01:23:58.861615 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.96 GiB, free memory in arenas 5.66 MiB, will set to 1.61 GiB (RSS), difference: -7.11 MiB 2024.10.22 01:24:00.179635 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 4194304 bytes for 205ad1f9404ca48beb078c3585fdafb0:0) 2024.10.22 01:24:00.183489 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 907a15295684a5ef4242e6a2fcfa9ac6, offset: 0, size: 4309865 2024.10.22 01:24:00.192739 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 907a15295684a5ef4242e6a2fcfa9ac6 for removal 2024.10.22 01:24:00.199653 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8d5c74f3a2e53007b24b0cfecd950c39, offset: 0, size: 949 2024.10.22 01:24:00.199463 [ 1476 ] {} CacheMetadata: Key 907a15295684a5ef4242e6a2fcfa9ac6 is removed from metadata 2024.10.22 01:24:00.208331 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8d5c74f3a2e53007b24b0cfecd950c39 for removal 2024.10.22 01:24:00.214299 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:24:00.323580 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:24:00.326433 [ 1476 ] {} CacheMetadata: Key 8d5c74f3a2e53007b24b0cfecd950c39 is removed from metadata 2024.10.22 01:24:00.331932 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:24:00.663786 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 7386 2024.10.22 01:24:00.742217 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_96_96_0 (state Temporary) 2024.10.22 01:24:00.759136 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.96 GiB, free memory in arenas 4.99 MiB, will set to 1.63 GiB (RSS), difference: 44.98 MiB 2024.10.22 01:24:00.805378 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:00.808839 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.23 GiB. 2024.10.22 01:24:00.875738 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 158 entries to flush up to offset 55201 2024.10.22 01:24:00.890191 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_476_476_0 (state Temporary) 2024.10.22 01:24:00.964484 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_476_476_0 to 202410_489_489_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:01.038601 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7386 2024.10.22 01:24:01.385930 [ 1740 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vhe/zswtobyhtxvftssjaxiytaecqkrud, size 91400 2024.10.22 01:24:01.526840 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:01.529759 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.22 GiB. 2024.10.22 01:24:01.663064 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 53058 2024.10.22 01:24:01.688211 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_467_467_0 (state Temporary) 2024.10.22 01:24:01.804079 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_467_467_0 to 202410_473_473_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:01.850726 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 55201 2024.10.22 01:24:01.879128 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_813_416} MergeTask::MergeProjectionsStage: Merge sorted 53531 rows, containing 13 columns (13 merged, 0 gathered) in 4.016917209 sec., 13326.388674395008 rows/sec., 2.13 MiB/sec. 2024.10.22 01:24:02.076300 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_813_416} MergedBlockOutputStream: filled checksums 202410_1_813_416 (state Temporary) 2024.10.22 01:24:02.201619 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:02.213678 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.22 GiB. 2024.10.22 01:24:02.304680 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_806_806_0 (state Temporary) 2024.10.22 01:24:02.414862 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_813_416} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_813_416 to 202410_1_813_416 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:02.461990 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_813_416} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_812_415, 202410_813_813_0] -> 202410_1_813_416 2024.10.22 01:24:02.478911 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_806_806_0 to 202410_819_819_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:02.498105 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:24:02.521439 [ 1208 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_813_416 to 202410_814_814_0 2024.10.22 01:24:02.534787 [ 1208 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.92 MiB using storage policy from min volume index 0 2024.10.22 01:24:02.540160 [ 1208 ] {} DiskLocal: Reserved 1.92 MiB on local disk `default`, having unreserved 297.22 GiB. 2024.10.22 01:24:02.552293 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 53058 2024.10.22 01:24:02.574865 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 719226 2024.10.22 01:24:02.624735 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_814_417} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_813_416 to 202410_814_814_0 into Compact with storage Full 2024.10.22 01:24:02.682099 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_814_417} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:02.719975 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_814_417} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_813_416, total 53531 rows starting from the beginning of the part 2024.10.22 01:24:02.776435 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.96 GiB, free memory in arenas 5.59 MiB, will set to 1.61 GiB (RSS), difference: 136.22 MiB 2024.10.22 01:24:02.781758 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_814_417} MergeTreeSequentialSource: Reading 2 marks from part 202410_814_814_0, total 69 rows starting from the beginning of the part 2024.10.22 01:24:04.297723 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:04.306686 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.19 GiB. 2024.10.22 01:24:04.348631 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:24:04.565249 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 4.28 MiB, will set to 1.59 GiB (RSS), difference: 87.98 MiB 2024.10.22 01:24:04.645136 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_457_457_0 (state Temporary) 2024.10.22 01:24:04.973551 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_457_457_0 to 202410_464_464_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:05.065011 [ 1230 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:24:05.092705 [ 1230 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:24:05.114612 [ 1221 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_459_92 to 202410_464_464_0 2024.10.22 01:24:05.123615 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 719226 2024.10.22 01:24:05.139544 [ 1221 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:05.150682 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:05.167519 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_459_92 to 202410_464_464_0 into Wide with storage Full 2024.10.22 01:24:05.214767 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:05.232332 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} MergeTreeSequentialSource: Reading 90 marks from part 202410_1_459_92, total 725347 rows starting from the beginning of the part 2024.10.22 01:24:05.278470 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_460_460_0, total 1694 rows starting from the beginning of the part 2024.10.22 01:24:05.346511 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_461_0, total 1410 rows starting from the beginning of the part 2024.10.22 01:24:05.371040 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_462_462_0, total 1342 rows starting from the beginning of the part 2024.10.22 01:24:05.458982 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_1_1_0 (state Temporary) 2024.10.22 01:24:05.481193 [ 1678 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yiu/oslbfhwlrpbcttcnpcfygmfpsxnvz, size 439 2024.10.22 01:24:05.481735 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/sau/fufhqmymowhfjavopbxfcohiaejox (LogSeriesLimiter: on interval from 2024-10-22 01:23:48 to 2024-10-22 01:24:05 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:24:05.535052 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_463_463_0, total 1483 rows starting from the beginning of the part 2024.10.22 01:24:05.558151 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_464_464_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:24:05.565056 [ 1680 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/sau/fufhqmymowhfjavopbxfcohiaejox, size 4 2024.10.22 01:24:06.018770 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_814_417} MergeTask::MergeProjectionsStage: Merge sorted 53600 rows, containing 13 columns (13 merged, 0 gathered) in 3.420018303 sec., 15672.430744883064 rows/sec., 2.50 MiB/sec. 2024.10.22 01:24:06.192413 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_814_417} MergedBlockOutputStream: filled checksums 202410_1_814_417 (state Temporary) 2024.10.22 01:24:06.339459 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 403 2024.10.22 01:24:06.521987 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:06.522950 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_814_417} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_814_417 to 202410_1_814_417 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:06.529780 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_814_417} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_813_416, 202410_814_814_0] -> 202410_1_814_417 2024.10.22 01:24:06.559764 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.96 GiB, free memory in arenas 2.84 MiB, will set to 1.57 GiB (RSS), difference: 137.57 MiB 2024.10.22 01:24:06.561207 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 53111 2024.10.22 01:24:06.574523 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:24:06.633214 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.16 GiB. 2024.10.22 01:24:06.806281 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_385_385_0 (state Temporary) 2024.10.22 01:24:06.867302 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_79_16, 202410_80_80_0, 202410_81_81_0, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0] 2024.10.22 01:24:06.870199 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_79_16, 202410_80_80_0, 202410_81_81_0, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0] 2024.10.22 01:24:06.875461 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_96_96_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:24:06.883895 [ 1224 ] {} DatabaseCatalog: Execution took 377 ms. 2024.10.22 01:24:06.978702 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:06.980927 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:07.033047 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_79_16, 202410_80_80_0, 202410_81_81_0, 202410_82_82_0, 202410_83_83_0, 202410_84_84_0] 2024.10.22 01:24:07.075432 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_807_807_0 (state Temporary) 2024.10.22 01:24:07.084987 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_385_385_0 to 202410_395_395_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:07.138333 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 8388608 bytes for 205ad1f9404ca48beb078c3585fdafb0:0) 2024.10.22 01:24:07.144143 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8ef9bfdedae7b690c581547bd4af761a, offset: 0, size: 3075347 2024.10.22 01:24:07.144419 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_411_189, 202410_412_412_0] 2024.10.22 01:24:07.154563 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8ef9bfdedae7b690c581547bd4af761a for removal 2024.10.22 01:24:07.158351 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 54b33faff7187086df0297a44a52372c, offset: 0, size: 1490845 2024.10.22 01:24:07.157754 [ 1476 ] {} CacheMetadata: Key 8ef9bfdedae7b690c581547bd4af761a is removed from metadata 2024.10.22 01:24:07.163612 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 54b33faff7187086df0297a44a52372c for removal 2024.10.22 01:24:07.151467 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_411_189, 202410_412_412_0] 2024.10.22 01:24:07.169887 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 176052c0463d29cefa31b0a09c1429cb, offset: 0, size: 927 2024.10.22 01:24:07.175287 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 176052c0463d29cefa31b0a09c1429cb for removal 2024.10.22 01:24:07.177180 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 403 2024.10.22 01:24:07.178870 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 97363d5052316d5512b81ceb14646027, offset: 0, size: 167 2024.10.22 01:24:07.185462 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 97363d5052316d5512b81ceb14646027 for removal 2024.10.22 01:24:07.188351 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 010dd14a0dcb66d270cd1fd19b8085ee, offset: 0, size: 4320586 2024.10.22 01:24:07.199758 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_411_189, 202410_412_412_0] 2024.10.22 01:24:07.201927 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 010dd14a0dcb66d270cd1fd19b8085ee for removal 2024.10.22 01:24:07.208046 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6197a1529728c85c8dda444b844ef260, offset: 0, size: 218 2024.10.22 01:24:07.212450 [ 1476 ] {} CacheMetadata: Key 010dd14a0dcb66d270cd1fd19b8085ee is removed from metadata 2024.10.22 01:24:07.214952 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6197a1529728c85c8dda444b844ef260 for removal 2024.10.22 01:24:07.220101 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d7718663ab7a6c5ff06605dfeb26b1b5, offset: 0, size: 3568 2024.10.22 01:24:07.224609 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d7718663ab7a6c5ff06605dfeb26b1b5 for removal 2024.10.22 01:24:07.225271 [ 1476 ] {} CacheMetadata: Key 6197a1529728c85c8dda444b844ef260 is removed from metadata 2024.10.22 01:24:07.228582 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b8d4643403893663813ae51bf758cb93, offset: 0, size: 952 2024.10.22 01:24:07.237362 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b8d4643403893663813ae51bf758cb93 for removal 2024.10.22 01:24:07.238381 [ 1476 ] {} CacheMetadata: Key d7718663ab7a6c5ff06605dfeb26b1b5 is removed from metadata 2024.10.22 01:24:07.261083 [ 1476 ] {} CacheMetadata: Key b8d4643403893663813ae51bf758cb93 is removed from metadata 2024.10.22 01:24:07.272771 [ 1476 ] {} CacheMetadata: Key 97363d5052316d5512b81ceb14646027 is removed from metadata 2024.10.22 01:24:07.280761 [ 1476 ] {} CacheMetadata: Key 176052c0463d29cefa31b0a09c1429cb is removed from metadata 2024.10.22 01:24:07.281402 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_807_807_0 to 202410_820_820_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:07.286959 [ 1476 ] {} CacheMetadata: Key 54b33faff7187086df0297a44a52372c is removed from metadata 2024.10.22 01:24:07.422803 [ 1177 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_814_417 to 202410_815_815_0 2024.10.22 01:24:07.473803 [ 1177 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.93 MiB using storage policy from min volume index 0 2024.10.22 01:24:07.478050 [ 1177 ] {} DiskLocal: Reserved 1.93 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:07.511430 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_815_418} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_814_417 to 202410_815_815_0 into Compact with storage Full 2024.10.22 01:24:07.554697 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 53111 2024.10.22 01:24:07.757062 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_464_93/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:24:07.783422 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_815_418} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:07.927439 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_815_418} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_814_417, total 53600 rows starting from the beginning of the part 2024.10.22 01:24:08.085601 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_815_418} MergeTreeSequentialSource: Reading 2 marks from part 202410_815_815_0, total 49 rows starting from the beginning of the part 2024.10.22 01:24:08.542531 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 7398 2024.10.22 01:24:08.547359 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.21 GiB, peak 1.96 GiB, free memory in arenas 5.66 MiB, will set to 1.20 GiB (RSS), difference: -8.63 MiB 2024.10.22 01:24:08.778557 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:08.783111 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:08.835379 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_477_477_0 (state Temporary) 2024.10.22 01:24:09.121029 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_477_477_0 to 202410_490_490_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:09.208024 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7398 2024.10.22 01:24:09.386831 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 165 entries to flush up to offset 55366 2024.10.22 01:24:09.613324 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:09.619722 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:09.836018 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_468_468_0 (state Temporary) 2024.10.22 01:24:10.048771 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_468_468_0 to 202410_474_474_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:10.214988 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:24:10.222938 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:24:10.300943 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 55366 2024.10.22 01:24:10.394931 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_96_96_0 to 202410_99_99_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:10.465210 [ 1248 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_94_19 to 202410_99_99_0 2024.10.22 01:24:10.477594 [ 1248 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.94 MiB using storage policy from min volume index 0 2024.10.22 01:24:10.481764 [ 1248 ] {} DiskLocal: Reserved 1.94 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:10.507444 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_426_85, 202410_427_427_0, 202410_428_428_0, 202410_429_429_0, 202410_430_430_0, 202410_431_431_0] 2024.10.22 01:24:10.511766 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_426_85, 202410_427_427_0, 202410_428_428_0, 202410_429_429_0, 202410_430_430_0, 202410_431_431_0] 2024.10.22 01:24:10.557051 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 1.96 GiB, free memory in arenas 7.17 MiB, will set to 1.25 GiB (RSS), difference: -7.92 MiB 2024.10.22 01:24:10.563148 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_426_85, 202410_427_427_0, 202410_428_428_0, 202410_429_429_0, 202410_430_430_0, 202410_431_431_0] 2024.10.22 01:24:10.653500 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_94_19 to 202410_99_99_0 into Compact with storage Full 2024.10.22 01:24:10.778151 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 0cfd7a53e55b3d0e1c8aa1209ba77ae7:0) 2024.10.22 01:24:10.780712 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 56cace9a7f4063834ae3deed45aaed35, offset: 0, size: 907 2024.10.22 01:24:10.785813 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 56cace9a7f4063834ae3deed45aaed35 for removal 2024.10.22 01:24:10.789533 [ 1476 ] {} CacheMetadata: Key 56cace9a7f4063834ae3deed45aaed35 is removed from metadata 2024.10.22 01:24:10.891930 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 709bd5f653ca1872dec7f8ba3f325e7d, offset: 0, size: 318109 2024.10.22 01:24:10.896436 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 709bd5f653ca1872dec7f8ba3f325e7d for removal 2024.10.22 01:24:10.903015 [ 1476 ] {} CacheMetadata: Key 709bd5f653ca1872dec7f8ba3f325e7d is removed from metadata 2024.10.22 01:24:10.904925 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 02c47cbe32c9ba43344f38d2b77f64e6, offset: 0, size: 2096705 2024.10.22 01:24:11.028482 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 02c47cbe32c9ba43344f38d2b77f64e6 for removal 2024.10.22 01:24:11.030976 [ 1476 ] {} CacheMetadata: Key 02c47cbe32c9ba43344f38d2b77f64e6 is removed from metadata 2024.10.22 01:24:11.031082 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4607f3cbad17d5dbdc3854cfb79e0b92, offset: 0, size: 193 2024.10.22 01:24:11.036985 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 4607f3cbad17d5dbdc3854cfb79e0b92 for removal 2024.10.22 01:24:11.043173 [ 1476 ] {} CacheMetadata: Key 4607f3cbad17d5dbdc3854cfb79e0b92 is removed from metadata 2024.10.22 01:24:11.328838 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_815_418} MergeTask::MergeProjectionsStage: Merge sorted 53649 rows, containing 13 columns (13 merged, 0 gathered) in 3.832625523 sec., 13997.975977054517 rows/sec., 2.24 MiB/sec. 2024.10.22 01:24:11.508256 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_815_418} MergedBlockOutputStream: filled checksums 202410_1_815_418 (state Temporary) 2024.10.22 01:24:11.561548 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 53203 2024.10.22 01:24:11.955003 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:11.957663 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:12.021368 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_808_808_0 (state Temporary) 2024.10.22 01:24:12.019002 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_815_418} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_815_418 to 202410_1_815_418 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:12.128333 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 721151 2024.10.22 01:24:12.153463 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_815_418} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_814_417, 202410_815_815_0] -> 202410_1_815_418 2024.10.22 01:24:12.155495 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_808_808_0 to 202410_821_821_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:12.192936 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_815_418 to 202410_816_816_0 2024.10.22 01:24:12.195342 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:24:12.211514 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.93 MiB using storage policy from min volume index 0 2024.10.22 01:24:12.214912 [ 1240 ] {} DiskLocal: Reserved 1.93 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:12.273170 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_816_419} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_815_418 to 202410_816_816_0 into Compact with storage Full 2024.10.22 01:24:12.324873 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 53203 2024.10.22 01:24:12.467182 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_816_419} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:12.482433 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_816_419} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_815_418, total 53649 rows starting from the beginning of the part 2024.10.22 01:24:12.564526 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_816_419} MergeTreeSequentialSource: Reading 2 marks from part 202410_816_816_0, total 71 rows starting from the beginning of the part 2024.10.22 01:24:12.656318 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.27 GiB, peak 1.96 GiB, free memory in arenas 5.93 MiB, will set to 1.26 GiB (RSS), difference: -5.89 MiB 2024.10.22 01:24:12.892640 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:12.899241 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:13.076315 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_458_458_0 (state Temporary) 2024.10.22 01:24:13.194935 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_458_458_0 to 202410_465_465_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:13.246181 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 721151 2024.10.22 01:24:14.151260 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3837 2024.10.22 01:24:14.652040 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.96 GiB, free memory in arenas 10.92 MiB, will set to 1.31 GiB (RSS), difference: -18.51 MiB 2024.10.22 01:24:14.628111 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_401_80, 202410_402_402_0, 202410_403_403_0, 202410_404_404_0, 202410_405_405_0, 202410_406_406_0] 2024.10.22 01:24:14.657633 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_401_80, 202410_402_402_0, 202410_403_403_0, 202410_404_404_0, 202410_405_405_0, 202410_406_406_0] 2024.10.22 01:24:14.681126 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 404 2024.10.22 01:24:14.778460 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_412_190, 202410_413_413_0] 2024.10.22 01:24:14.780443 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_412_190, 202410_413_413_0] 2024.10.22 01:24:14.805221 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_401_80, 202410_402_402_0, 202410_403_403_0, 202410_404_404_0, 202410_405_405_0, 202410_406_406_0] 2024.10.22 01:24:14.848903 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_412_190, 202410_413_413_0] 2024.10.22 01:24:15.122802 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:15.127468 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:15.305059 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_386_386_0 (state Temporary) 2024.10.22 01:24:15.482201 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_386_386_0 to 202410_396_396_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:15.563323 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:15.711226 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 404 2024.10.22 01:24:16.239323 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_816_419} MergeTask::MergeProjectionsStage: Merge sorted 53720 rows, containing 13 columns (13 merged, 0 gathered) in 3.973669464 sec., 13518.990566951696 rows/sec., 2.16 MiB/sec. 2024.10.22 01:24:16.352879 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 53242 2024.10.22 01:24:16.396292 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_816_419} MergedBlockOutputStream: filled checksums 202410_1_816_419 (state Temporary) 2024.10.22 01:24:16.471487 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_94_19, total 3686 rows starting from the beginning of the part 2024.10.22 01:24:16.496080 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_816_419} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_816_419 to 202410_1_816_419 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:16.507678 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.96 GiB, free memory in arenas 15.71 MiB, will set to 1.34 GiB (RSS), difference: 3.06 MiB 2024.10.22 01:24:16.512823 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_816_419} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_815_418, 202410_816_816_0] -> 202410_1_816_419 2024.10.22 01:24:16.576408 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:16.611405 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:16.665386 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:24:16.693597 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_809_809_0 (state Temporary) 2024.10.22 01:24:16.712701 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 7410 2024.10.22 01:24:16.819972 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_713_331, 202410_714_719_1] 2024.10.22 01:24:16.832453 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_713_331, 202410_714_719_1] 2024.10.22 01:24:16.844715 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_713_331, 202410_714_719_1] 2024.10.22 01:24:16.965095 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:16.967963 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:17.014882 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_478_478_0 (state Temporary) 2024.10.22 01:24:17.053287 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_809_809_0 to 202410_822_822_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:17.064196 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_478_478_0 to 202410_491_491_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:17.083433 [ 1180 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_486_97 to 202410_491_491_0 2024.10.22 01:24:17.094465 [ 1180 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:17.175113 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7410 2024.10.22 01:24:17.204794 [ 1178 ] {} DatabaseCatalog: Execution took 320 ms. 2024.10.22 01:24:17.207365 [ 1180 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:17.218658 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_816_419 to 202410_817_817_0 2024.10.22 01:24:17.231310 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 53242 2024.10.22 01:24:17.262864 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_486_97 to 202410_491_491_0 into Compact with storage Full 2024.10.22 01:24:17.266324 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.93 MiB using storage policy from min volume index 0 2024.10.22 01:24:17.270893 [ 1191 ] {} DiskLocal: Reserved 1.93 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:17.303611 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:17.306625 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_817_420} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_816_419 to 202410_817_817_0 into Compact with storage Full 2024.10.22 01:24:17.416295 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_486_97, total 7434 rows starting from the beginning of the part 2024.10.22 01:24:17.531321 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_817_420} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:17.563202 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_487_487_0, total 18 rows starting from the beginning of the part 2024.10.22 01:24:17.607401 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_488_488_0, total 32 rows starting from the beginning of the part 2024.10.22 01:24:17.678949 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_489_489_0, total 8 rows starting from the beginning of the part 2024.10.22 01:24:17.701230 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_817_420} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_816_419, total 53720 rows starting from the beginning of the part 2024.10.22 01:24:17.767844 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_490_490_0, total 12 rows starting from the beginning of the part 2024.10.22 01:24:17.796239 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_95_95_0, total 41 rows starting from the beginning of the part 2024.10.22 01:24:17.948820 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_491_491_0, total 12 rows starting from the beginning of the part 2024.10.22 01:24:17.968151 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_817_420} MergeTreeSequentialSource: Reading 2 marks from part 202410_817_817_0, total 68 rows starting from the beginning of the part 2024.10.22 01:24:18.004667 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 99 entries to flush up to offset 55465 2024.10.22 01:24:18.049336 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_464_93/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:24:18.453156 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:18.458651 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:18.531195 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 1.96 GiB, free memory in arenas 15.32 MiB, will set to 1.37 GiB (RSS), difference: -49.73 MiB 2024.10.22 01:24:18.634236 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_469_469_0 (state Temporary) 2024.10.22 01:24:18.702773 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_469_469_0 to 202410_475_475_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:18.856344 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 55465 2024.10.22 01:24:18.953761 [ 1193 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 01:24:19.155039 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_96_96_0, total 44 rows starting from the beginning of the part 2024.10.22 01:24:19.400125 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} MergeTask::MergeProjectionsStage: Merge sorted 7516 rows, containing 12 columns (12 merged, 0 gathered) in 2.146037239 sec., 3502.269142124612 rows/sec., 556.27 KiB/sec. 2024.10.22 01:24:19.442494 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} MergedBlockOutputStream: filled checksums 202410_1_491_98 (state Temporary) 2024.10.22 01:24:19.659078 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_719_332, 202410_720_720_0] 2024.10.22 01:24:19.663494 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_719_332, 202410_720_720_0] 2024.10.22 01:24:19.679737 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_719_332, 202410_720_720_0] 2024.10.22 01:24:20.099389 [ 1168 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:24:20.170486 [ 1168 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:24:20.215021 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:24:20.220906 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:24:20.254819 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 722691 2024.10.22 01:24:20.277586 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_491_98 to 202410_1_491_98 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:20.281787 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_491_98} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_486_97, 202410_491_491_0] -> 202410_1_491_98 2024.10.22 01:24:20.368866 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.47 MiB. 2024.10.22 01:24:20.833783 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.96 GiB, free memory in arenas 15.36 MiB, will set to 1.41 GiB (RSS), difference: -29.03 MiB 2024.10.22 01:24:21.237838 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 53307 2024.10.22 01:24:21.333651 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_97_97_0, total 41 rows starting from the beginning of the part 2024.10.22 01:24:21.355215 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.19241629 sec. 2024.10.22 01:24:21.472613 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:21.474374 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:21.632421 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:21.642383 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:21.658963 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_459_459_0 (state Temporary) 2024.10.22 01:24:21.670063 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 2097152 bytes for 0cfd7a53e55b3d0e1c8aa1209ba77ae7:0) 2024.10.22 01:24:21.676037 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f0c4e70b1cbfe69855dedc87a6c5c8bf, offset: 0, size: 3466948 2024.10.22 01:24:21.682616 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f0c4e70b1cbfe69855dedc87a6c5c8bf for removal 2024.10.22 01:24:21.687705 [ 1476 ] {} CacheMetadata: Key f0c4e70b1cbfe69855dedc87a6c5c8bf is removed from metadata 2024.10.22 01:24:21.687822 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: de2607ddb7f16241c685eab486ed5e60, offset: 0, size: 151 2024.10.22 01:24:21.692938 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key de2607ddb7f16241c685eab486ed5e60 for removal 2024.10.22 01:24:21.701708 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_810_810_0 (state Temporary) 2024.10.22 01:24:21.702401 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0018447e89d1ea3b4e1de321afe217e6, offset: 0, size: 937 2024.10.22 01:24:21.711193 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0018447e89d1ea3b4e1de321afe217e6 for removal 2024.10.22 01:24:21.718941 [ 1476 ] {} CacheMetadata: Key 0018447e89d1ea3b4e1de321afe217e6 is removed from metadata 2024.10.22 01:24:21.728343 [ 1476 ] {} CacheMetadata: Key de2607ddb7f16241c685eab486ed5e60 is removed from metadata 2024.10.22 01:24:21.741481 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_459_459_0 to 202410_466_466_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:21.744603 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 3880 2024.10.22 01:24:21.746181 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_810_810_0 to 202410_823_823_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:21.819144 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a8ac47066c9b9fedacb362ee26077374, offset: 0, size: 202716 2024.10.22 01:24:21.855886 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a8ac47066c9b9fedacb362ee26077374 for removal 2024.10.22 01:24:21.859719 [ 1476 ] {} CacheMetadata: Key a8ac47066c9b9fedacb362ee26077374 is removed from metadata 2024.10.22 01:24:21.860893 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c6df5365c49319bedc8f917b92149ec6, offset: 0, size: 5126 2024.10.22 01:24:21.865529 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c6df5365c49319bedc8f917b92149ec6 for removal 2024.10.22 01:24:21.868963 [ 1476 ] {} CacheMetadata: Key c6df5365c49319bedc8f917b92149ec6 is removed from metadata 2024.10.22 01:24:21.869702 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b62c3167f8c3eb21f0f4db540d74dfb0, offset: 0, size: 191 2024.10.22 01:24:21.872521 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b62c3167f8c3eb21f0f4db540d74dfb0 for removal 2024.10.22 01:24:21.879320 [ 1476 ] {} CacheMetadata: Key b62c3167f8c3eb21f0f4db540d74dfb0 is removed from metadata 2024.10.22 01:24:21.884628 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 722691 2024.10.22 01:24:21.889562 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_818_818_0 to 202410_823_823_0 2024.10.22 01:24:21.931955 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:21.934360 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:22.068498 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 53307 2024.10.22 01:24:22.130879 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_818_818_0 to 202410_823_823_0 into Compact with storage Full 2024.10.22 01:24:22.190563 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:22.215099 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_818_818_0, total 56 rows starting from the beginning of the part 2024.10.22 01:24:22.292991 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_819_819_0, total 45 rows starting from the beginning of the part 2024.10.22 01:24:22.411796 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_817_420} MergeTask::MergeProjectionsStage: Merge sorted 53788 rows, containing 13 columns (13 merged, 0 gathered) in 5.135597428 sec., 10473.562375964335 rows/sec., 1.67 MiB/sec. 2024.10.22 01:24:22.438975 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_820_820_0, total 53 rows starting from the beginning of the part 2024.10.22 01:24:22.471427 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_821_821_0, total 92 rows starting from the beginning of the part 2024.10.22 01:24:22.561238 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.96 GiB, free memory in arenas 14.04 MiB, will set to 1.45 GiB (RSS), difference: -61.05 MiB 2024.10.22 01:24:22.567157 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 54 file segments (while reserving 16777216 bytes for 205ad1f9404ca48beb078c3585fdafb0:0) 2024.10.22 01:24:22.569663 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0300fccae38fe675829ced06bb1210bd, offset: 0, size: 3224825 2024.10.22 01:24:22.576430 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0300fccae38fe675829ced06bb1210bd for removal 2024.10.22 01:24:22.577463 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_817_420} MergedBlockOutputStream: filled checksums 202410_1_817_420 (state Temporary) 2024.10.22 01:24:22.579163 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0a564cf1a66e81805fe9174c6889c530, offset: 0, size: 52565 2024.10.22 01:24:22.579579 [ 1476 ] {} CacheMetadata: Key 0300fccae38fe675829ced06bb1210bd is removed from metadata 2024.10.22 01:24:22.588193 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0a564cf1a66e81805fe9174c6889c530 for removal 2024.10.22 01:24:22.596293 [ 1476 ] {} CacheMetadata: Key 0a564cf1a66e81805fe9174c6889c530 is removed from metadata 2024.10.22 01:24:22.598437 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 39ce864718af811b4eee226b58e668ae, offset: 0, size: 188 2024.10.22 01:24:22.601287 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 39ce864718af811b4eee226b58e668ae for removal 2024.10.22 01:24:22.604903 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: df5a60c8830a9a9a11bbc328536cb414, offset: 0, size: 60681 2024.10.22 01:24:22.605156 [ 1476 ] {} CacheMetadata: Key 39ce864718af811b4eee226b58e668ae is removed from metadata 2024.10.22 01:24:22.609563 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_822_822_0, total 39 rows starting from the beginning of the part 2024.10.22 01:24:22.658286 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_823_823_0, total 65 rows starting from the beginning of the part 2024.10.22 01:24:22.712128 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key df5a60c8830a9a9a11bbc328536cb414 for removal 2024.10.22 01:24:22.740960 [ 1476 ] {} CacheMetadata: Key df5a60c8830a9a9a11bbc328536cb414 is removed from metadata 2024.10.22 01:24:22.744089 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6edd7b7e036d88dd2717d3149efe115c, offset: 0, size: 151 2024.10.22 01:24:22.752005 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6edd7b7e036d88dd2717d3149efe115c for removal 2024.10.22 01:24:22.758575 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: db0bf952b74ba3b705b726e28871a814, offset: 0, size: 218 2024.10.22 01:24:22.760917 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key db0bf952b74ba3b705b726e28871a814 for removal 2024.10.22 01:24:22.762326 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_413_191, 202410_414_414_0] 2024.10.22 01:24:22.763465 [ 1476 ] {} CacheMetadata: Key 6edd7b7e036d88dd2717d3149efe115c is removed from metadata 2024.10.22 01:24:22.765941 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_413_191, 202410_414_414_0] 2024.10.22 01:24:22.766073 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e2d84b375d3745afeceb6d4c0045d0c9, offset: 0, size: 103407 2024.10.22 01:24:22.770785 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e2d84b375d3745afeceb6d4c0045d0c9 for removal 2024.10.22 01:24:22.774211 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 167decdad23e534ad47f73c5fc7eb299, offset: 0, size: 168 2024.10.22 01:24:22.774183 [ 1476 ] {} CacheMetadata: Key db0bf952b74ba3b705b726e28871a814 is removed from metadata 2024.10.22 01:24:22.776900 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 167decdad23e534ad47f73c5fc7eb299 for removal 2024.10.22 01:24:22.780519 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 219ec65f582a8b986faea17e913c14cd, offset: 0, size: 106749 2024.10.22 01:24:22.780988 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_413_191, 202410_414_414_0] 2024.10.22 01:24:22.780519 [ 1476 ] {} CacheMetadata: Key 167decdad23e534ad47f73c5fc7eb299 is removed from metadata 2024.10.22 01:24:22.783934 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 219ec65f582a8b986faea17e913c14cd for removal 2024.10.22 01:24:22.786768 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7499e6f2fbce2553a650dd5944c59674, offset: 0, size: 150 2024.10.22 01:24:22.791214 [ 1476 ] {} CacheMetadata: Key 219ec65f582a8b986faea17e913c14cd is removed from metadata 2024.10.22 01:24:22.792418 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7499e6f2fbce2553a650dd5944c59674 for removal 2024.10.22 01:24:22.803118 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5f886357d1f88387acb4b6ac1426cc42, offset: 0, size: 155993 2024.10.22 01:24:22.808748 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5f886357d1f88387acb4b6ac1426cc42 for removal 2024.10.22 01:24:22.808967 [ 1476 ] {} CacheMetadata: Key 7499e6f2fbce2553a650dd5944c59674 is removed from metadata 2024.10.22 01:24:22.818614 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_817_420} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_817_420 to 202410_1_817_420 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:22.818854 [ 1476 ] {} CacheMetadata: Key e2d84b375d3745afeceb6d4c0045d0c9 is removed from metadata 2024.10.22 01:24:22.822932 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_817_420} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_816_419, 202410_817_817_0] -> 202410_1_817_420 2024.10.22 01:24:22.844997 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fbb1057b687d172cf4fcdef3817fbabd, offset: 0, size: 204219 2024.10.22 01:24:22.847455 [ 1476 ] {} CacheMetadata: Key 5f886357d1f88387acb4b6ac1426cc42 is removed from metadata 2024.10.22 01:24:22.847897 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:24:22.852940 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fbb1057b687d172cf4fcdef3817fbabd for removal 2024.10.22 01:24:22.866104 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 82cdb06f10c8d13f71718f827726f576, offset: 0, size: 1956807 2024.10.22 01:24:22.869478 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 82cdb06f10c8d13f71718f827726f576 for removal 2024.10.22 01:24:22.874656 [ 1476 ] {} CacheMetadata: Key fbb1057b687d172cf4fcdef3817fbabd is removed from metadata 2024.10.22 01:24:22.876989 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fd82e66bf39a439ab2138f27b9eacdff, offset: 0, size: 902 2024.10.22 01:24:22.879178 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fd82e66bf39a439ab2138f27b9eacdff for removal 2024.10.22 01:24:22.882379 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fb9568880f78b9e42ddc74aaa0e409eb, offset: 0, size: 152 2024.10.22 01:24:22.885321 [ 1476 ] {} CacheMetadata: Key fd82e66bf39a439ab2138f27b9eacdff is removed from metadata 2024.10.22 01:24:22.896395 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fb9568880f78b9e42ddc74aaa0e409eb for removal 2024.10.22 01:24:22.896580 [ 1476 ] {} CacheMetadata: Key 82cdb06f10c8d13f71718f827726f576 is removed from metadata 2024.10.22 01:24:22.930945 [ 1476 ] {} CacheMetadata: Key fb9568880f78b9e42ddc74aaa0e409eb is removed from metadata 2024.10.22 01:24:22.932390 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7fa814d82ae1bfa666d362211e9bde76, offset: 0, size: 1607669 2024.10.22 01:24:23.039828 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7fa814d82ae1bfa666d362211e9bde76 for removal 2024.10.22 01:24:23.047881 [ 1476 ] {} CacheMetadata: Key 7fa814d82ae1bfa666d362211e9bde76 is removed from metadata 2024.10.22 01:24:23.050651 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 264d34dede0498ddab8f508ac323011e, offset: 0, size: 3155 2024.10.22 01:24:23.055757 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 264d34dede0498ddab8f508ac323011e for removal 2024.10.22 01:24:23.061246 [ 1476 ] {} CacheMetadata: Key 264d34dede0498ddab8f508ac323011e is removed from metadata 2024.10.22 01:24:23.066193 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9d69cead7062dc1a8b614902d4271d16, offset: 0, size: 152 2024.10.22 01:24:23.068260 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9d69cead7062dc1a8b614902d4271d16 for removal 2024.10.22 01:24:23.070190 [ 1476 ] {} CacheMetadata: Key 9d69cead7062dc1a8b614902d4271d16 is removed from metadata 2024.10.22 01:24:23.071968 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: aa276f65de168973023402323d8d93fc, offset: 0, size: 923 2024.10.22 01:24:23.075851 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key aa276f65de168973023402323d8d93fc for removal 2024.10.22 01:24:23.083635 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 29e28889e92cf2f44100fed58de3f0de, offset: 0, size: 231249 2024.10.22 01:24:23.087828 [ 1476 ] {} CacheMetadata: Key aa276f65de168973023402323d8d93fc is removed from metadata 2024.10.22 01:24:23.087545 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 29e28889e92cf2f44100fed58de3f0de for removal 2024.10.22 01:24:23.093390 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f845becd4778eb395f73094b8cbde0b4, offset: 0, size: 1485331 2024.10.22 01:24:23.094159 [ 1476 ] {} CacheMetadata: Key 29e28889e92cf2f44100fed58de3f0de is removed from metadata 2024.10.22 01:24:23.105235 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f845becd4778eb395f73094b8cbde0b4 for removal 2024.10.22 01:24:23.113619 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 16a1b749a8a512f95edc0c273edff3f2, offset: 0, size: 153 2024.10.22 01:24:23.118385 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 16a1b749a8a512f95edc0c273edff3f2 for removal 2024.10.22 01:24:23.118725 [ 1476 ] {} CacheMetadata: Key f845becd4778eb395f73094b8cbde0b4 is removed from metadata 2024.10.22 01:24:23.126729 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fd225ac8f9621ffab70653eae2545bf2, offset: 0, size: 33486 2024.10.22 01:24:23.131527 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fd225ac8f9621ffab70653eae2545bf2 for removal 2024.10.22 01:24:23.141052 [ 1476 ] {} CacheMetadata: Key 16a1b749a8a512f95edc0c273edff3f2 is removed from metadata 2024.10.22 01:24:23.142472 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f557e904e9e8c7d56a264de69868b580, offset: 0, size: 151 2024.10.22 01:24:23.147287 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f557e904e9e8c7d56a264de69868b580 for removal 2024.10.22 01:24:23.151841 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ce27a59c40a32fb7f188faccabccba63, offset: 0, size: 34326 2024.10.22 01:24:23.154807 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ce27a59c40a32fb7f188faccabccba63 for removal 2024.10.22 01:24:23.157788 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0efb0d7ae18507aa1a7d39acb5ac0f0d, offset: 0, size: 138203 2024.10.22 01:24:23.160878 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0efb0d7ae18507aa1a7d39acb5ac0f0d for removal 2024.10.22 01:24:23.162594 [ 1476 ] {} CacheMetadata: Key ce27a59c40a32fb7f188faccabccba63 is removed from metadata 2024.10.22 01:24:23.163010 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7c915853c245b27d28dd59e420424939, offset: 0, size: 98748 2024.10.22 01:24:23.164777 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7c915853c245b27d28dd59e420424939 for removal 2024.10.22 01:24:23.168429 [ 1476 ] {} CacheMetadata: Key 0efb0d7ae18507aa1a7d39acb5ac0f0d is removed from metadata 2024.10.22 01:24:23.169271 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 38c2c20eff916916f47dd4c44ff10d62, offset: 0, size: 3814 2024.10.22 01:24:23.172644 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 38c2c20eff916916f47dd4c44ff10d62 for removal 2024.10.22 01:24:23.178622 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 386571327ea751c27e89387049161e2d, offset: 0, size: 92790 2024.10.22 01:24:23.182684 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 386571327ea751c27e89387049161e2d for removal 2024.10.22 01:24:23.183563 [ 1476 ] {} CacheMetadata: Key 38c2c20eff916916f47dd4c44ff10d62 is removed from metadata 2024.10.22 01:24:23.186731 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5018533a4c234aa98556e244acf71b11, offset: 0, size: 1958583 2024.10.22 01:24:23.192625 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5018533a4c234aa98556e244acf71b11 for removal 2024.10.22 01:24:23.198780 [ 1476 ] {} CacheMetadata: Key 386571327ea751c27e89387049161e2d is removed from metadata 2024.10.22 01:24:23.203047 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ddc1adc1498dab6b768e3640389c2670, offset: 0, size: 170 2024.10.22 01:24:23.209659 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ddc1adc1498dab6b768e3640389c2670 for removal 2024.10.22 01:24:23.211121 [ 1476 ] {} CacheMetadata: Key 5018533a4c234aa98556e244acf71b11 is removed from metadata 2024.10.22 01:24:23.214401 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 412c5da17e256e49104ac2d8784d9b0b, offset: 0, size: 170 2024.10.22 01:24:23.216368 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 405 2024.10.22 01:24:23.218800 [ 1476 ] {} CacheMetadata: Key ddc1adc1498dab6b768e3640389c2670 is removed from metadata 2024.10.22 01:24:23.220654 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 412c5da17e256e49104ac2d8784d9b0b for removal 2024.10.22 01:24:23.225641 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 23212ae46d201fd7bbb1426a4dc33f80, offset: 0, size: 152 2024.10.22 01:24:23.228913 [ 1476 ] {} CacheMetadata: Key 412c5da17e256e49104ac2d8784d9b0b is removed from metadata 2024.10.22 01:24:23.229457 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 23212ae46d201fd7bbb1426a4dc33f80 for removal 2024.10.22 01:24:23.247543 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a0dbe44889e4fd98da093fb7365f57ab, offset: 0, size: 152 2024.10.22 01:24:23.247788 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_98_98_0, total 45 rows starting from the beginning of the part 2024.10.22 01:24:23.255306 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a0dbe44889e4fd98da093fb7365f57ab for removal 2024.10.22 01:24:23.287333 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 67b56165fd24d1953c01a1c2f3df67ed, offset: 0, size: 191 2024.10.22 01:24:23.294287 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 67b56165fd24d1953c01a1c2f3df67ed for removal 2024.10.22 01:24:23.309426 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a82749c4814db9449bbae635ea9caf18, offset: 0, size: 62474 2024.10.22 01:24:23.322851 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a82749c4814db9449bbae635ea9caf18 for removal 2024.10.22 01:24:23.328879 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cb3ce35a54391c0dad2330f63fe614d9, offset: 0, size: 152 2024.10.22 01:24:23.337573 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cb3ce35a54391c0dad2330f63fe614d9 for removal 2024.10.22 01:24:23.345317 [ 1476 ] {} CacheMetadata: Key a82749c4814db9449bbae635ea9caf18 is removed from metadata 2024.10.22 01:24:23.469222 [ 1476 ] {} CacheMetadata: Key 67b56165fd24d1953c01a1c2f3df67ed is removed from metadata 2024.10.22 01:24:23.474324 [ 1476 ] {} CacheMetadata: Key a0dbe44889e4fd98da093fb7365f57ab is removed from metadata 2024.10.22 01:24:23.484029 [ 1476 ] {} CacheMetadata: Key 23212ae46d201fd7bbb1426a4dc33f80 is removed from metadata 2024.10.22 01:24:23.491045 [ 1476 ] {} CacheMetadata: Key 7c915853c245b27d28dd59e420424939 is removed from metadata 2024.10.22 01:24:23.506754 [ 1476 ] {} CacheMetadata: Key f557e904e9e8c7d56a264de69868b580 is removed from metadata 2024.10.22 01:24:23.509441 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:23.513294 [ 1476 ] {} CacheMetadata: Key fd225ac8f9621ffab70653eae2545bf2 is removed from metadata 2024.10.22 01:24:23.541399 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.18 GiB. 2024.10.22 01:24:23.561725 [ 1476 ] {} CacheMetadata: Key cb3ce35a54391c0dad2330f63fe614d9 is removed from metadata 2024.10.22 01:24:23.563104 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 962198199316466a2cdb719db6d18bd8, offset: 0, size: 38646 2024.10.22 01:24:23.578461 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 962198199316466a2cdb719db6d18bd8 for removal 2024.10.22 01:24:23.587281 [ 1476 ] {} CacheMetadata: Key 962198199316466a2cdb719db6d18bd8 is removed from metadata 2024.10.22 01:24:23.588107 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9930dceb06dc4145541cd692df80172a, offset: 0, size: 2927169 2024.10.22 01:24:23.600141 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9930dceb06dc4145541cd692df80172a for removal 2024.10.22 01:24:23.609593 [ 1476 ] {} CacheMetadata: Key 9930dceb06dc4145541cd692df80172a is removed from metadata 2024.10.22 01:24:23.611420 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1d95a95448aa21beefe2a5f93058957d, offset: 0, size: 150 2024.10.22 01:24:23.616455 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1d95a95448aa21beefe2a5f93058957d for removal 2024.10.22 01:24:23.721349 [ 1476 ] {} CacheMetadata: Key 1d95a95448aa21beefe2a5f93058957d is removed from metadata 2024.10.22 01:24:23.723024 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 249a5aeb119e2bca15cbfb13eddee527, offset: 0, size: 154 2024.10.22 01:24:23.724497 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_387_387_0 (state Temporary) 2024.10.22 01:24:23.725916 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 249a5aeb119e2bca15cbfb13eddee527 for removal 2024.10.22 01:24:23.733819 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0fafce5daf2a9d154ec49b5f5d918423, offset: 0, size: 67599 2024.10.22 01:24:23.739937 [ 1476 ] {} CacheMetadata: Key 249a5aeb119e2bca15cbfb13eddee527 is removed from metadata 2024.10.22 01:24:23.748612 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0fafce5daf2a9d154ec49b5f5d918423 for removal 2024.10.22 01:24:23.754247 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3072cd0a91cbd01cf71f16ac25810d4d, offset: 0, size: 136901 2024.10.22 01:24:23.757297 [ 1476 ] {} CacheMetadata: Key 0fafce5daf2a9d154ec49b5f5d918423 is removed from metadata 2024.10.22 01:24:23.759500 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3072cd0a91cbd01cf71f16ac25810d4d for removal 2024.10.22 01:24:23.763567 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 969eb50c5846ac3349d7d710971d2293, offset: 0, size: 167 2024.10.22 01:24:23.763662 [ 1476 ] {} CacheMetadata: Key 3072cd0a91cbd01cf71f16ac25810d4d is removed from metadata 2024.10.22 01:24:23.765575 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 969eb50c5846ac3349d7d710971d2293 for removal 2024.10.22 01:24:23.769603 [ 1476 ] {} CacheMetadata: Key 969eb50c5846ac3349d7d710971d2293 is removed from metadata 2024.10.22 01:24:23.770162 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 839954c081f725bd89f0851fa186aa11, offset: 0, size: 903 2024.10.22 01:24:23.785164 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 839954c081f725bd89f0851fa186aa11 for removal 2024.10.22 01:24:23.790705 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 93ec60c13080c47d1d4e6009cc20d49a, offset: 0, size: 170 2024.10.22 01:24:23.791530 [ 1476 ] {} CacheMetadata: Key 839954c081f725bd89f0851fa186aa11 is removed from metadata 2024.10.22 01:24:23.795169 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 93ec60c13080c47d1d4e6009cc20d49a for removal 2024.10.22 01:24:23.809609 [ 1476 ] {} CacheMetadata: Key 93ec60c13080c47d1d4e6009cc20d49a is removed from metadata 2024.10.22 01:24:23.811485 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c459df21e16892ff3a6fe091a6201d2d, offset: 0, size: 930 2024.10.22 01:24:23.814809 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c459df21e16892ff3a6fe091a6201d2d for removal 2024.10.22 01:24:23.820267 [ 1476 ] {} CacheMetadata: Key c459df21e16892ff3a6fe091a6201d2d is removed from metadata 2024.10.22 01:24:23.820309 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d8007ac33e1aa0276af262c5d2ed8cb1, offset: 0, size: 893 2024.10.22 01:24:23.827074 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d8007ac33e1aa0276af262c5d2ed8cb1 for removal 2024.10.22 01:24:23.834412 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 47332c9a4e11e70e259579c4c80b6cbd, offset: 0, size: 141393 2024.10.22 01:24:23.844804 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 47332c9a4e11e70e259579c4c80b6cbd for removal 2024.10.22 01:24:23.851630 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: da06ce4fc04a7e4b43c4a14a7643bf9f, offset: 0, size: 153 2024.10.22 01:24:23.856736 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key da06ce4fc04a7e4b43c4a14a7643bf9f for removal 2024.10.22 01:24:23.861039 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 73e952f2b22209202092d45bccb64ad2, offset: 0, size: 195166 2024.10.22 01:24:23.863284 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 73e952f2b22209202092d45bccb64ad2 for removal 2024.10.22 01:24:23.869268 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9c018e12356f28618fd956f1a29079b6, offset: 0, size: 167 2024.10.22 01:24:23.872587 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9c018e12356f28618fd956f1a29079b6 for removal 2024.10.22 01:24:23.874697 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 47861e3edf37173ee56b2dc5ab7e6147, offset: 0, size: 149 2024.10.22 01:24:23.888626 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 47861e3edf37173ee56b2dc5ab7e6147 for removal 2024.10.22 01:24:23.905649 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8ef026081664de82a307756d23627483, offset: 0, size: 81267 2024.10.22 01:24:23.911634 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8ef026081664de82a307756d23627483 for removal 2024.10.22 01:24:23.937808 [ 1476 ] {} CacheMetadata: Key 8ef026081664de82a307756d23627483 is removed from metadata 2024.10.22 01:24:23.940615 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} MergeTask::MergeProjectionsStage: Merge sorted 350 rows, containing 13 columns (13 merged, 0 gathered) in 1.848992079 sec., 189.2923198401652 rows/sec., 28.55 KiB/sec. 2024.10.22 01:24:23.953183 [ 1476 ] {} CacheMetadata: Key 47861e3edf37173ee56b2dc5ab7e6147 is removed from metadata 2024.10.22 01:24:23.962711 [ 1476 ] {} CacheMetadata: Key 9c018e12356f28618fd956f1a29079b6 is removed from metadata 2024.10.22 01:24:23.968850 [ 1476 ] {} CacheMetadata: Key 73e952f2b22209202092d45bccb64ad2 is removed from metadata 2024.10.22 01:24:23.975101 [ 1476 ] {} CacheMetadata: Key da06ce4fc04a7e4b43c4a14a7643bf9f is removed from metadata 2024.10.22 01:24:23.980885 [ 1476 ] {} CacheMetadata: Key 47332c9a4e11e70e259579c4c80b6cbd is removed from metadata 2024.10.22 01:24:23.991106 [ 1476 ] {} CacheMetadata: Key d8007ac33e1aa0276af262c5d2ed8cb1 is removed from metadata 2024.10.22 01:24:24.028350 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_insert_202410_387_387_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:24:24.070319 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_344_71, 202410_345_345_0, 202410_346_346_0, 202410_347_347_0, 202410_348_348_0, 202410_349_349_0] 2024.10.22 01:24:24.074895 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} MergedBlockOutputStream: filled checksums 202410_818_823_1 (state Temporary) 2024.10.22 01:24:24.074884 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_344_71, 202410_345_345_0, 202410_346_346_0, 202410_347_347_0, 202410_348_348_0, 202410_349_349_0] 2024.10.22 01:24:24.112745 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_387_387_0 to 202410_397_397_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:24.127040 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_344_71, 202410_345_345_0, 202410_346_346_0, 202410_347_347_0, 202410_348_348_0, 202410_349_349_0] 2024.10.22 01:24:24.197317 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 541485 bytes for 820753cede6aa3e7956e92e665fe3023:0) 2024.10.22 01:24:24.211987 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 405 2024.10.22 01:24:24.205824 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5e6930c4aeaaa94196d8fccb93209892, offset: 0, size: 146451 2024.10.22 01:24:24.312316 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5e6930c4aeaaa94196d8fccb93209892 for removal 2024.10.22 01:24:24.313145 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_818_823_1 to 202410_818_823_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:24.316263 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1af700cd85ca114b00fc664334d1fda3, offset: 0, size: 190 2024.10.22 01:24:24.316303 [ 1476 ] {} CacheMetadata: Key 5e6930c4aeaaa94196d8fccb93209892 is removed from metadata 2024.10.22 01:24:24.321545 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_818_823_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_818_818_0, 202410_823_823_0] -> 202410_818_823_1 2024.10.22 01:24:24.322050 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1af700cd85ca114b00fc664334d1fda3 for removal 2024.10.22 01:24:24.327489 [ 1476 ] {} CacheMetadata: Key 1af700cd85ca114b00fc664334d1fda3 is removed from metadata 2024.10.22 01:24:24.400101 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:24:24.530138 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_99_99_0, total 42 rows starting from the beginning of the part 2024.10.22 01:24:24.557957 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:24:24.579570 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:24:24.596949 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:24:24.631371 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:24:24.663576 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:24:24.677094 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 7433 2024.10.22 01:24:24.695219 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:24:24.702901 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_720_333, 202410_721_721_0] 2024.10.22 01:24:24.705928 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_720_333, 202410_721_721_0] 2024.10.22 01:24:24.725907 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_720_333, 202410_721_721_0] 2024.10.22 01:24:24.746864 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:24:24.782394 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:24:24.815161 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:24:24.883669 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:24.888477 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:24.919362 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yci/zliaujazuovlbfwlrksqbqsfbgssw (LogSeriesLimiter: on interval from 2024-10-22 01:24:05 to 2024-10-22 01:24:24 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:24:24.974561 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_479_479_0 (state Temporary) 2024.10.22 01:24:25.032192 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_479_479_0 to 202410_492_492_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:25.126451 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7433 2024.10.22 01:24:25.708896 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.96 GiB, free memory in arenas 11.41 MiB, will set to 1.48 GiB (RSS), difference: -74.95 MiB 2024.10.22 01:24:26.076721 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 260 entries to flush up to offset 53567 2024.10.22 01:24:26.366526 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 157 entries to flush up to offset 55622 2024.10.22 01:24:26.552503 [ 1245 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:24:27.011650 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:27.015501 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:27.083673 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_470_470_0 (state Temporary) 2024.10.22 01:24:27.175791 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_470_470_0 to 202410_476_476_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:27.231911 [ 1162 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_461_471_2 to 202410_476_476_0 2024.10.22 01:24:27.258538 [ 1162 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:27.263627 [ 1162 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:27.264541 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 55622 2024.10.22 01:24:27.274963 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:27.285143 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:27.356104 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_461_471_2 to 202410_476_476_0 into Compact with storage Full 2024.10.22 01:24:27.474734 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:27.507906 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_811_811_0 (state Temporary) 2024.10.22 01:24:27.512325 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_471_2, total 2709 rows starting from the beginning of the part 2024.10.22 01:24:27.667171 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.96 GiB, free memory in arenas 15.00 MiB, will set to 1.52 GiB (RSS), difference: -7.37 MiB 2024.10.22 01:24:27.698947 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_811_811_0 to 202410_824_824_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:27.752843 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 53567 2024.10.22 01:24:27.756661 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_472_472_0, total 166 rows starting from the beginning of the part 2024.10.22 01:24:27.957106 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_473_473_0, total 158 rows starting from the beginning of the part 2024.10.22 01:24:28.263172 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_474_474_0, total 165 rows starting from the beginning of the part 2024.10.22 01:24:28.778797 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_475_475_0, total 99 rows starting from the beginning of the part 2024.10.22 01:24:28.884451 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_476_476_0, total 157 rows starting from the beginning of the part 2024.10.22 01:24:28.888078 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 724231 2024.10.22 01:24:28.951562 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_464_93/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:24:29.165021 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} MergeTask::MergeProjectionsStage: Merge sorted 732816 rows, containing 4 columns (4 merged, 0 gathered) in 24.003982141 sec., 30528.934561583166 rows/sec., 458.02 KiB/sec. 2024.10.22 01:24:29.519825 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} MergedBlockOutputStream: filled checksums 202410_1_464_93 (state Temporary) 2024.10.22 01:24:29.577186 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.96 GiB, free memory in arenas 11.03 MiB, will set to 1.56 GiB (RSS), difference: -24.51 MiB 2024.10.22 01:24:29.910912 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:29.919441 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:30.113475 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_460_460_0 (state Temporary) 2024.10.22 01:24:30.214998 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:24:30.220285 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:24:30.292459 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_460_460_0 to 202410_467_467_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:30.362500 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 724231 2024.10.22 01:24:30.630856 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} MergeTask::MergeProjectionsStage: Merge sorted 3454 rows, containing 13 columns (13 merged, 0 gathered) in 3.30858347 sec., 1043.9512955675862 rows/sec., 466.84 KiB/sec. 2024.10.22 01:24:30.898455 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} MergedBlockOutputStream: filled checksums 202410_461_476_3 (state Temporary) 2024.10.22 01:24:31.212739 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_721_334, 202410_722_722_0] 2024.10.22 01:24:31.215746 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_721_334, 202410_722_722_0] 2024.10.22 01:24:31.216329 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_414_192, 202410_415_415_0] 2024.10.22 01:24:31.220190 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_414_192, 202410_415_415_0] 2024.10.22 01:24:31.230006 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_721_334, 202410_722_722_0] 2024.10.22 01:24:31.389714 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_461_476_3 to 202410_461_476_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:31.404725 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_476_3} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_461_471_2, 202410_476_476_0] -> 202410_461_476_3 2024.10.22 01:24:31.407449 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_414_192, 202410_415_415_0] 2024.10.22 01:24:31.473119 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.74 MiB. 2024.10.22 01:24:31.572656 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.96 GiB, free memory in arenas 9.82 MiB, will set to 1.59 GiB (RSS), difference: -19.20 MiB 2024.10.22 01:24:31.727568 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 406 2024.10.22 01:24:31.899959 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 53617 2024.10.22 01:24:32.090601 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:32.093338 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:32.164993 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_464_93 to 202410_1_464_93 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:32.171060 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_464_93} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_459_92, 202410_464_464_0] -> 202410_1_464_93 2024.10.22 01:24:32.203125 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:24:32.276020 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_388_388_0 (state Temporary) 2024.10.22 01:24:32.404421 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_388_388_0 to 202410_398_398_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:32.424669 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:32.428247 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:32.487078 [ 1261 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_393_81 to 202410_398_398_0 2024.10.22 01:24:32.521323 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_812_812_0 (state Temporary) 2024.10.22 01:24:32.526335 [ 1261 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:32.538333 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:32.546251 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 406 2024.10.22 01:24:32.578929 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_393_81 to 202410_398_398_0 into Compact with storage Full 2024.10.22 01:24:32.583195 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_812_812_0 to 202410_825_825_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:32.630219 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 7453 2024.10.22 01:24:32.651822 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 53617 2024.10.22 01:24:32.829117 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:32.834429 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:32.862919 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:32.873464 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_393_81, total 437 rows starting from the beginning of the part 2024.10.22 01:24:32.882931 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_480_480_0 (state Temporary) 2024.10.22 01:24:32.897357 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_394_394_0, total 1 rows starting from the beginning of the part 2024.10.22 01:24:32.929783 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_395_395_0, total 1 rows starting from the beginning of the part 2024.10.22 01:24:32.961752 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_396_396_0, total 1 rows starting from the beginning of the part 2024.10.22 01:24:33.004788 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_397_397_0, total 1 rows starting from the beginning of the part 2024.10.22 01:24:33.032092 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_480_480_0 to 202410_493_493_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:33.073152 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7453 2024.10.22 01:24:33.146276 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_398_398_0, total 1 rows starting from the beginning of the part 2024.10.22 01:24:33.487255 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.96 GiB, free memory in arenas 7.71 MiB, will set to 1.63 GiB (RSS), difference: -22.18 MiB 2024.10.22 01:24:34.016190 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_722_335, 202410_723_723_0] 2024.10.22 01:24:34.018385 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_722_335, 202410_723_723_0] 2024.10.22 01:24:34.033974 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_722_335, 202410_723_723_0] 2024.10.22 01:24:34.463891 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_398_82/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:24:34.812739 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 93 entries to flush up to offset 55715 2024.10.22 01:24:35.126437 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 58 file segments (while reserving 12827132 bytes for 205ad1f9404ca48beb078c3585fdafb0:33554432) 2024.10.22 01:24:35.130919 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: edd9462692602b26ea5cc5106415a53a, offset: 0, size: 206 2024.10.22 01:24:35.138914 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key edd9462692602b26ea5cc5106415a53a for removal 2024.10.22 01:24:35.150380 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fdbd7500185d99b52b770a8689ca0e9f, offset: 0, size: 191 2024.10.22 01:24:35.149086 [ 1476 ] {} CacheMetadata: Key edd9462692602b26ea5cc5106415a53a is removed from metadata 2024.10.22 01:24:35.155838 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fdbd7500185d99b52b770a8689ca0e9f for removal 2024.10.22 01:24:35.163375 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f47f8533bfe26c97c12d023e3acf3136, offset: 0, size: 309707 2024.10.22 01:24:35.162663 [ 1476 ] {} CacheMetadata: Key fdbd7500185d99b52b770a8689ca0e9f is removed from metadata 2024.10.22 01:24:35.171809 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f47f8533bfe26c97c12d023e3acf3136 for removal 2024.10.22 01:24:35.176308 [ 1476 ] {} CacheMetadata: Key f47f8533bfe26c97c12d023e3acf3136 is removed from metadata 2024.10.22 01:24:35.177233 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:35.177531 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a7c30470e44bc9d658f9b80bd220ed30, offset: 0, size: 193 2024.10.22 01:24:35.178377 [ 1179 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:24:35.181041 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.17 GiB. 2024.10.22 01:24:35.181543 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a7c30470e44bc9d658f9b80bd220ed30 for removal 2024.10.22 01:24:35.184545 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a86ee83034b72d15f2b508e6983cfbd2, offset: 0, size: 523744 2024.10.22 01:24:35.184686 [ 1476 ] {} CacheMetadata: Key a7c30470e44bc9d658f9b80bd220ed30 is removed from metadata 2024.10.22 01:24:35.191888 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a86ee83034b72d15f2b508e6983cfbd2 for removal 2024.10.22 01:24:35.198209 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9ddc90b8f98945a8aacf5a3d434ae9aa, offset: 0, size: 204 2024.10.22 01:24:35.198636 [ 1476 ] {} CacheMetadata: Key a86ee83034b72d15f2b508e6983cfbd2 is removed from metadata 2024.10.22 01:24:35.207806 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9ddc90b8f98945a8aacf5a3d434ae9aa for removal 2024.10.22 01:24:35.216069 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 89594ca5ca4a3a1ba0f0f7c65cad868b, offset: 0, size: 586600 2024.10.22 01:24:35.222294 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 89594ca5ca4a3a1ba0f0f7c65cad868b for removal 2024.10.22 01:24:35.223984 [ 1179 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:24:35.231093 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9e75d976a6d31c2edb14e8a1d9cc3559, offset: 0, size: 198 2024.10.22 01:24:35.231728 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_471_471_0 (state Temporary) 2024.10.22 01:24:35.236732 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9e75d976a6d31c2edb14e8a1d9cc3559 for removal 2024.10.22 01:24:35.245144 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5888e49dbd6c93ad411a6074c1e1c534, offset: 0, size: 1490394 2024.10.22 01:24:35.259237 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5888e49dbd6c93ad411a6074c1e1c534 for removal 2024.10.22 01:24:35.274868 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 184ae13a45820a0b5d5802581572bcda, offset: 0, size: 102764 2024.10.22 01:24:35.277891 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 184ae13a45820a0b5d5802581572bcda for removal 2024.10.22 01:24:35.282144 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 501a5f6d0acabbc23ae19e2998da2026, offset: 0, size: 324883 2024.10.22 01:24:35.285416 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 501a5f6d0acabbc23ae19e2998da2026 for removal 2024.10.22 01:24:35.289132 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 041637f8a85a31aa727b73f5b3782da8, offset: 0, size: 193 2024.10.22 01:24:35.295719 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 041637f8a85a31aa727b73f5b3782da8 for removal 2024.10.22 01:24:35.300216 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 48ad5a980d33a62487a85eb184591f9a, offset: 0, size: 456213 2024.10.22 01:24:35.306787 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 48ad5a980d33a62487a85eb184591f9a for removal 2024.10.22 01:24:35.312107 [ 1476 ] {} CacheMetadata: Key 48ad5a980d33a62487a85eb184591f9a is removed from metadata 2024.10.22 01:24:35.312332 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 83709228ec7719eebedcda5fe7c2ea10, offset: 0, size: 168 2024.10.22 01:24:35.312637 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_471_471_0 to 202410_477_477_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:35.316414 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 83709228ec7719eebedcda5fe7c2ea10 for removal 2024.10.22 01:24:35.320289 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7f70add76970701a3f9562ce9721214e, offset: 0, size: 809069 2024.10.22 01:24:35.322905 [ 1476 ] {} CacheMetadata: Key 83709228ec7719eebedcda5fe7c2ea10 is removed from metadata 2024.10.22 01:24:35.325311 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7f70add76970701a3f9562ce9721214e for removal 2024.10.22 01:24:35.330146 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8591fc3e15994a3da92bce5853ece44d, offset: 0, size: 471311 2024.10.22 01:24:35.332362 [ 1476 ] {} CacheMetadata: Key 7f70add76970701a3f9562ce9721214e is removed from metadata 2024.10.22 01:24:35.336442 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8591fc3e15994a3da92bce5853ece44d for removal 2024.10.22 01:24:35.341730 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4fe2b3c0b25fcd8478127e8538977c02, offset: 0, size: 581122 2024.10.22 01:24:35.352250 [ 1476 ] {} CacheMetadata: Key 8591fc3e15994a3da92bce5853ece44d is removed from metadata 2024.10.22 01:24:35.353241 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4fe2b3c0b25fcd8478127e8538977c02 for removal 2024.10.22 01:24:35.368058 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 55715 2024.10.22 01:24:35.461324 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e79fc69a5af74332458713eb1f7334c0, offset: 0, size: 358855 2024.10.22 01:24:35.461322 [ 1476 ] {} CacheMetadata: Key 4fe2b3c0b25fcd8478127e8538977c02 is removed from metadata 2024.10.22 01:24:35.466793 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e79fc69a5af74332458713eb1f7334c0 for removal 2024.10.22 01:24:35.469142 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f5ec91dba13e50a7d63db0e4bc16a9e0, offset: 0, size: 194 2024.10.22 01:24:35.471072 [ 1476 ] {} CacheMetadata: Key e79fc69a5af74332458713eb1f7334c0 is removed from metadata 2024.10.22 01:24:35.472118 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f5ec91dba13e50a7d63db0e4bc16a9e0 for removal 2024.10.22 01:24:35.474656 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0baa7bc8de30388791be05d9ec62206c, offset: 0, size: 193 2024.10.22 01:24:35.476551 [ 1476 ] {} CacheMetadata: Key f5ec91dba13e50a7d63db0e4bc16a9e0 is removed from metadata 2024.10.22 01:24:35.477197 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0baa7bc8de30388791be05d9ec62206c for removal 2024.10.22 01:24:35.482444 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9b5ff2a104bcd7dd71f1f5eec1ccee7d, offset: 0, size: 218 2024.10.22 01:24:35.485481 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9b5ff2a104bcd7dd71f1f5eec1ccee7d for removal 2024.10.22 01:24:35.485469 [ 1476 ] {} CacheMetadata: Key 0baa7bc8de30388791be05d9ec62206c is removed from metadata 2024.10.22 01:24:35.488864 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b18663db716bc6d8e2dc1bf522713e69, offset: 0, size: 109182 2024.10.22 01:24:35.493318 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b18663db716bc6d8e2dc1bf522713e69 for removal 2024.10.22 01:24:35.494632 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.96 GiB, free memory in arenas 6.07 MiB, will set to 1.66 GiB (RSS), difference: -38.08 MiB 2024.10.22 01:24:35.499023 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6cf11dab7b6ddb3695db7d8c92656783, offset: 0, size: 202 2024.10.22 01:24:35.501687 [ 1476 ] {} CacheMetadata: Key b18663db716bc6d8e2dc1bf522713e69 is removed from metadata 2024.10.22 01:24:35.510164 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6cf11dab7b6ddb3695db7d8c92656783 for removal 2024.10.22 01:24:35.514338 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6b326795c61c3f4ffcf5f45bdb237838, offset: 0, size: 193 2024.10.22 01:24:35.516170 [ 1476 ] {} CacheMetadata: Key 6cf11dab7b6ddb3695db7d8c92656783 is removed from metadata 2024.10.22 01:24:35.520532 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6b326795c61c3f4ffcf5f45bdb237838 for removal 2024.10.22 01:24:35.532360 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1947014fbd310543054648382e420f99, offset: 0, size: 805896 2024.10.22 01:24:35.536199 [ 1476 ] {} CacheMetadata: Key 6b326795c61c3f4ffcf5f45bdb237838 is removed from metadata 2024.10.22 01:24:35.553375 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1947014fbd310543054648382e420f99 for removal 2024.10.22 01:24:35.557190 [ 1476 ] {} CacheMetadata: Key 1947014fbd310543054648382e420f99 is removed from metadata 2024.10.22 01:24:35.562134 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0d3e5b9ce612b39ff858433bf16cd51a, offset: 0, size: 159030 2024.10.22 01:24:35.568763 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0d3e5b9ce612b39ff858433bf16cd51a for removal 2024.10.22 01:24:35.576134 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5b95cc29665d4714fe7f68d21a06de63, offset: 0, size: 7074 2024.10.22 01:24:35.580546 [ 1476 ] {} CacheMetadata: Key 0d3e5b9ce612b39ff858433bf16cd51a is removed from metadata 2024.10.22 01:24:35.583427 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5b95cc29665d4714fe7f68d21a06de63 for removal 2024.10.22 01:24:35.600758 [ 1476 ] {} CacheMetadata: Key 5b95cc29665d4714fe7f68d21a06de63 is removed from metadata 2024.10.22 01:24:35.609726 [ 1476 ] {} CacheMetadata: Key 9b5ff2a104bcd7dd71f1f5eec1ccee7d is removed from metadata 2024.10.22 01:24:35.619292 [ 1476 ] {} CacheMetadata: Key 041637f8a85a31aa727b73f5b3782da8 is removed from metadata 2024.10.22 01:24:35.630684 [ 1476 ] {} CacheMetadata: Key 501a5f6d0acabbc23ae19e2998da2026 is removed from metadata 2024.10.22 01:24:35.641747 [ 1476 ] {} CacheMetadata: Key 184ae13a45820a0b5d5802581572bcda is removed from metadata 2024.10.22 01:24:35.659220 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} MergeTask::MergeProjectionsStage: Merge sorted 442 rows, containing 32 columns (32 merged, 0 gathered) in 2.949115341 sec., 149.87545378612575 rows/sec., 13.99 KiB/sec. 2024.10.22 01:24:35.659654 [ 1476 ] {} CacheMetadata: Key 5888e49dbd6c93ad411a6074c1e1c534 is removed from metadata 2024.10.22 01:24:35.671490 [ 1476 ] {} CacheMetadata: Key 9e75d976a6d31c2edb14e8a1d9cc3559 is removed from metadata 2024.10.22 01:24:35.682456 [ 1476 ] {} CacheMetadata: Key 89594ca5ca4a3a1ba0f0f7c65cad868b is removed from metadata 2024.10.22 01:24:35.695367 [ 1476 ] {} CacheMetadata: Key 9ddc90b8f98945a8aacf5a3d434ae9aa is removed from metadata 2024.10.22 01:24:35.699963 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ba5f298445414e92715fc66f73e96031, offset: 0, size: 193 2024.10.22 01:24:35.705411 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ba5f298445414e92715fc66f73e96031 for removal 2024.10.22 01:24:35.708518 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: accf6466d755a33413a5344a0147b118, offset: 0, size: 504 2024.10.22 01:24:35.715542 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key accf6466d755a33413a5344a0147b118 for removal 2024.10.22 01:24:35.727759 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 27f2f3e24e3e0e3436124869d109c988, offset: 0, size: 52258 2024.10.22 01:24:35.737320 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 27f2f3e24e3e0e3436124869d109c988 for removal 2024.10.22 01:24:35.756626 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: bda19bd3f49a9c05ba0f5515560469ad, offset: 0, size: 170 2024.10.22 01:24:35.760208 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key bda19bd3f49a9c05ba0f5515560469ad for removal 2024.10.22 01:24:35.762396 [ 1476 ] {} CacheMetadata: Key 27f2f3e24e3e0e3436124869d109c988 is removed from metadata 2024.10.22 01:24:35.763628 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: daa52b2a7e5f817a9fcac870a57f45e0, offset: 0, size: 151 2024.10.22 01:24:35.766636 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key daa52b2a7e5f817a9fcac870a57f45e0 for removal 2024.10.22 01:24:35.770132 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 03f493a6ab6acd1ec24e88bc702db212, offset: 0, size: 83140 2024.10.22 01:24:35.781141 [ 1476 ] {} CacheMetadata: Key daa52b2a7e5f817a9fcac870a57f45e0 is removed from metadata 2024.10.22 01:24:35.785201 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 03f493a6ab6acd1ec24e88bc702db212 for removal 2024.10.22 01:24:35.791015 [ 1476 ] {} CacheMetadata: Key 03f493a6ab6acd1ec24e88bc702db212 is removed from metadata 2024.10.22 01:24:35.789809 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f7995ca9d9be970eae390027c4dbcb8e, offset: 0, size: 2344833 2024.10.22 01:24:35.799108 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f7995ca9d9be970eae390027c4dbcb8e for removal 2024.10.22 01:24:35.804694 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0b21742babadea191d6fb6fa96b74762, offset: 0, size: 151 2024.10.22 01:24:35.805780 [ 1476 ] {} CacheMetadata: Key f7995ca9d9be970eae390027c4dbcb8e is removed from metadata 2024.10.22 01:24:35.809331 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0b21742babadea191d6fb6fa96b74762 for removal 2024.10.22 01:24:35.812181 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4e2a17f3831152da38de98bdb2767313, offset: 0, size: 96646 2024.10.22 01:24:35.816060 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4e2a17f3831152da38de98bdb2767313 for removal 2024.10.22 01:24:35.816080 [ 1476 ] {} CacheMetadata: Key 0b21742babadea191d6fb6fa96b74762 is removed from metadata 2024.10.22 01:24:35.828653 [ 1476 ] {} CacheMetadata: Key 4e2a17f3831152da38de98bdb2767313 is removed from metadata 2024.10.22 01:24:35.829174 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1d8ad350f378f57809f83834c19072c0, offset: 0, size: 12752 2024.10.22 01:24:35.832466 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1d8ad350f378f57809f83834c19072c0 for removal 2024.10.22 01:24:35.839365 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8a65e5652e5e015ac87de1f802e9db27, offset: 0, size: 210 2024.10.22 01:24:35.845626 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8a65e5652e5e015ac87de1f802e9db27 for removal 2024.10.22 01:24:35.852367 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8eb29eadc26178cf52acf2d19f68cbc3, offset: 0, size: 914 2024.10.22 01:24:35.855675 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8eb29eadc26178cf52acf2d19f68cbc3 for removal 2024.10.22 01:24:35.859176 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f0348c7244de988fab14af1cad884d57, offset: 0, size: 218 2024.10.22 01:24:35.863357 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f0348c7244de988fab14af1cad884d57 for removal 2024.10.22 01:24:35.865127 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 722dd7ffc90957a7ddcd0911d0bc1957, offset: 0, size: 261779 2024.10.22 01:24:35.868245 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 722dd7ffc90957a7ddcd0911d0bc1957 for removal 2024.10.22 01:24:35.871884 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: edf70d7005aa8d232d7010ff77326824, offset: 0, size: 194 2024.10.22 01:24:35.875518 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key edf70d7005aa8d232d7010ff77326824 for removal 2024.10.22 01:24:35.879038 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e9bec0b1af0ebeb4767750158c870f9b, offset: 0, size: 153 2024.10.22 01:24:35.883546 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e9bec0b1af0ebeb4767750158c870f9b for removal 2024.10.22 01:24:35.888104 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 714fa62e5a11372e2bddc24048ad1a9e, offset: 0, size: 193 2024.10.22 01:24:35.892853 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 714fa62e5a11372e2bddc24048ad1a9e for removal 2024.10.22 01:24:35.897804 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7b28975773b4dd4cfecbcc8ff772401a, offset: 0, size: 113367 2024.10.22 01:24:35.905452 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7b28975773b4dd4cfecbcc8ff772401a for removal 2024.10.22 01:24:35.907407 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} MergedBlockOutputStream: filled checksums 202410_1_398_82 (state Temporary) 2024.10.22 01:24:35.909424 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 87a4345bd398410e8d7e9d4eca26b320, offset: 0, size: 309958 2024.10.22 01:24:35.915331 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 87a4345bd398410e8d7e9d4eca26b320 for removal 2024.10.22 01:24:35.920064 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 133f79a46d749d9e7972cc05437f9395, offset: 0, size: 81784 2024.10.22 01:24:35.941124 [ 1476 ] {} CacheMetadata: Key 87a4345bd398410e8d7e9d4eca26b320 is removed from metadata 2024.10.22 01:24:35.965646 [ 1476 ] {} CacheMetadata: Key 714fa62e5a11372e2bddc24048ad1a9e is removed from metadata 2024.10.22 01:24:35.972595 [ 1476 ] {} CacheMetadata: Key 7b28975773b4dd4cfecbcc8ff772401a is removed from metadata 2024.10.22 01:24:35.982723 [ 1476 ] {} CacheMetadata: Key e9bec0b1af0ebeb4767750158c870f9b is removed from metadata 2024.10.22 01:24:36.024693 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 133f79a46d749d9e7972cc05437f9395 for removal 2024.10.22 01:24:36.028788 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3e8ff06e7d3dbb9191dbd54fdfc1020a, offset: 0, size: 195 2024.10.22 01:24:36.034679 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3e8ff06e7d3dbb9191dbd54fdfc1020a for removal 2024.10.22 01:24:36.044752 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 51d45afb53009471dff40b491b8ccea9, offset: 0, size: 330464 2024.10.22 01:24:36.055331 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 51d45afb53009471dff40b491b8ccea9 for removal 2024.10.22 01:24:36.059415 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cab870427ac220e71cd6b4b3bd9c557a, offset: 0, size: 154 2024.10.22 01:24:36.062586 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cab870427ac220e71cd6b4b3bd9c557a for removal 2024.10.22 01:24:36.068654 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f307af763f7352c69973aff47a714c67, offset: 0, size: 333945 2024.10.22 01:24:36.074418 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f307af763f7352c69973aff47a714c67 for removal 2024.10.22 01:24:36.081145 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f1614cda94a0cd410670097da0a3ae55, offset: 0, size: 1465599 2024.10.22 01:24:36.091301 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f1614cda94a0cd410670097da0a3ae55 for removal 2024.10.22 01:24:36.099602 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f880b82da98581d1cb51956867d5ec47, offset: 0, size: 154 2024.10.22 01:24:36.102693 [ 1476 ] {} CacheMetadata: Key edf70d7005aa8d232d7010ff77326824 is removed from metadata 2024.10.22 01:24:36.114904 [ 1476 ] {} CacheMetadata: Key f1614cda94a0cd410670097da0a3ae55 is removed from metadata 2024.10.22 01:24:36.127389 [ 1476 ] {} CacheMetadata: Key cab870427ac220e71cd6b4b3bd9c557a is removed from metadata 2024.10.22 01:24:36.134905 [ 1476 ] {} CacheMetadata: Key 51d45afb53009471dff40b491b8ccea9 is removed from metadata 2024.10.22 01:24:36.144251 [ 1476 ] {} CacheMetadata: Key 3e8ff06e7d3dbb9191dbd54fdfc1020a is removed from metadata 2024.10.22 01:24:36.207600 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f880b82da98581d1cb51956867d5ec47 for removal 2024.10.22 01:24:36.212412 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 52d567ded30b56716a0264c02a8df7f2, offset: 0, size: 225 2024.10.22 01:24:36.220175 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 52d567ded30b56716a0264c02a8df7f2 for removal 2024.10.22 01:24:36.224588 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9b246279a57be6f0a62aa34aedcfcc58, offset: 0, size: 328862 2024.10.22 01:24:36.229720 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9b246279a57be6f0a62aa34aedcfcc58 for removal 2024.10.22 01:24:36.244711 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b187ea61e9afa1b31f3a76ffb6537b28, offset: 0, size: 210 2024.10.22 01:24:36.255198 [ 1476 ] {} CacheMetadata: Key 52d567ded30b56716a0264c02a8df7f2 is removed from metadata 2024.10.22 01:24:36.255088 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b187ea61e9afa1b31f3a76ffb6537b28 for removal 2024.10.22 01:24:36.260105 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 20564cb8304b74f99f8e177222d120de, offset: 0, size: 319804 2024.10.22 01:24:36.262975 [ 1476 ] {} CacheMetadata: Key b187ea61e9afa1b31f3a76ffb6537b28 is removed from metadata 2024.10.22 01:24:36.264494 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 20564cb8304b74f99f8e177222d120de for removal 2024.10.22 01:24:36.269313 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2fa152d55b7b3f8cdd0d80a54fcc63a0, offset: 0, size: 153 2024.10.22 01:24:36.269755 [ 1476 ] {} CacheMetadata: Key 20564cb8304b74f99f8e177222d120de is removed from metadata 2024.10.22 01:24:36.273459 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2fa152d55b7b3f8cdd0d80a54fcc63a0 for removal 2024.10.22 01:24:36.276491 [ 1476 ] {} CacheMetadata: Key 2fa152d55b7b3f8cdd0d80a54fcc63a0 is removed from metadata 2024.10.22 01:24:36.296196 [ 1476 ] {} CacheMetadata: Key f880b82da98581d1cb51956867d5ec47 is removed from metadata 2024.10.22 01:24:36.309118 [ 1476 ] {} CacheMetadata: Key 133f79a46d749d9e7972cc05437f9395 is removed from metadata 2024.10.22 01:24:36.327107 [ 1476 ] {} CacheMetadata: Key 722dd7ffc90957a7ddcd0911d0bc1957 is removed from metadata 2024.10.22 01:24:36.332585 [ 1476 ] {} CacheMetadata: Key f0348c7244de988fab14af1cad884d57 is removed from metadata 2024.10.22 01:24:36.338796 [ 1476 ] {} CacheMetadata: Key 8eb29eadc26178cf52acf2d19f68cbc3 is removed from metadata 2024.10.22 01:24:36.349531 [ 1476 ] {} CacheMetadata: Key 8a65e5652e5e015ac87de1f802e9db27 is removed from metadata 2024.10.22 01:24:36.457160 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 6bcf41483bfa0ab63a436581c4a1c4f0:0) 2024.10.22 01:24:36.458171 [ 1476 ] {} CacheMetadata: Key f307af763f7352c69973aff47a714c67 is removed from metadata 2024.10.22 01:24:36.459995 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cf1b07f1e944b97ddbefdf9020f867ea, offset: 0, size: 8209791 2024.10.22 01:24:36.464456 [ 1476 ] {} CacheMetadata: Key 1d8ad350f378f57809f83834c19072c0 is removed from metadata 2024.10.22 01:24:36.467294 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cf1b07f1e944b97ddbefdf9020f867ea for removal 2024.10.22 01:24:36.469842 [ 1476 ] {} CacheMetadata: Key cf1b07f1e944b97ddbefdf9020f867ea is removed from metadata 2024.10.22 01:24:36.470859 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 604ff3cb7e63a51db633051e55318cb3, offset: 0, size: 990 2024.10.22 01:24:36.474573 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 604ff3cb7e63a51db633051e55318cb3 for removal 2024.10.22 01:24:36.579797 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b54a655133b412db38a699bba80223ea, offset: 0, size: 378532 2024.10.22 01:24:36.579744 [ 1476 ] {} CacheMetadata: Key 604ff3cb7e63a51db633051e55318cb3 is removed from metadata 2024.10.22 01:24:36.583620 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b54a655133b412db38a699bba80223ea for removal 2024.10.22 01:24:36.599925 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 835ae268390b0d5e122732e2bb31e7d1, offset: 0, size: 194 2024.10.22 01:24:36.604955 [ 1476 ] {} CacheMetadata: Key b54a655133b412db38a699bba80223ea is removed from metadata 2024.10.22 01:24:36.608038 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 835ae268390b0d5e122732e2bb31e7d1 for removal 2024.10.22 01:24:36.661210 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 230 entries to flush up to offset 53847 2024.10.22 01:24:36.720022 [ 1476 ] {} CacheMetadata: Key 835ae268390b0d5e122732e2bb31e7d1 is removed from metadata 2024.10.22 01:24:36.729112 [ 1476 ] {} CacheMetadata: Key bda19bd3f49a9c05ba0f5515560469ad is removed from metadata 2024.10.22 01:24:36.740031 [ 1476 ] {} CacheMetadata: Key 9b246279a57be6f0a62aa34aedcfcc58 is removed from metadata 2024.10.22 01:24:36.747260 [ 1476 ] {} CacheMetadata: Key accf6466d755a33413a5344a0147b118 is removed from metadata 2024.10.22 01:24:36.755565 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/izl/afuseuleisjxiesaxxmamsfdzqfoj, upload id b7174a93-d837-4ec5-919a-707e569e6e1a 2024.10.22 01:24:36.756761 [ 1476 ] {} CacheMetadata: Key ba5f298445414e92715fc66f73e96031 is removed from metadata 2024.10.22 01:24:37.374219 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 725771 2024.10.22 01:24:37.510115 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:24:37.523928 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:24:37.534212 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:24:37.551600 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:24:37.573804 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:24:37.595183 [ 1196 ] {} DatabaseCatalog: Execution took 261 ms. 2024.10.22 01:24:37.596883 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_398_82 to 202410_1_398_82 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:37.606879 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_398_82} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_393_81, 202410_398_398_0] -> 202410_1_398_82 2024.10.22 01:24:37.709160 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:24:37.741655 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:24:37.766983 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:24:37.833022 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/dou/paybvzcggfmainboorlqeamwmbcfw (LogSeriesLimiter: on interval from 2024-10-22 01:24:24 to 2024-10-22 01:24:37 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:24:37.867153 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:24:38.193671 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:24:38.476065 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.82 GiB, peak 1.96 GiB, free memory in arenas 3.75 MiB, will set to 1.70 GiB (RSS), difference: -117.77 MiB 2024.10.22 01:24:38.494620 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:38.528367 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:38.615145 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:38.619420 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:38.775921 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_461_461_0 (state Temporary) 2024.10.22 01:24:38.860292 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_813_813_0 (state Temporary) 2024.10.22 01:24:38.914190 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_813_813_0 to 202410_826_826_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:38.985776 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_461_461_0 to 202410_468_468_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:39.098446 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 53847 2024.10.22 01:24:39.152380 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 725771 2024.10.22 01:24:39.330407 [ 1235 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 217 ms. 2024.10.22 01:24:39.462613 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_99_20/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:24:39.889541 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:39.906847 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:39.920034 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_415_193, 202410_416_416_0] 2024.10.22 01:24:39.923366 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_415_193, 202410_416_416_0] 2024.10.22 01:24:39.957416 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_415_193, 202410_416_416_0] 2024.10.22 01:24:40.218655 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:24:40.223250 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:24:40.545253 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.92 GiB, peak 1.97 GiB, free memory in arenas 5.87 MiB, will set to 1.89 GiB (RSS), difference: -29.34 MiB 2024.10.22 01:24:40.576706 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 7464 2024.10.22 01:24:40.600685 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_723_336, 202410_724_724_0] 2024.10.22 01:24:40.607327 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_723_336, 202410_724_724_0] 2024.10.22 01:24:40.619954 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_723_336, 202410_724_724_0] 2024.10.22 01:24:40.899519 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:40.906798 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:40.947429 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_481_481_0 (state Temporary) 2024.10.22 01:24:41.108333 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_481_481_0 to 202410_494_494_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:41.256326 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7464 2024.10.22 01:24:42.759989 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 1.97 GiB, free memory in arenas 6.27 MiB, will set to 1.66 GiB (RSS), difference: 200.61 MiB 2024.10.22 01:24:42.882920 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 333 entries to flush up to offset 56048 2024.10.22 01:24:43.132510 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 53901 2024.10.22 01:24:43.482799 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_97_97_0 (state Temporary) 2024.10.22 01:24:43.498954 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:43.504673 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:43.509224 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:43.515874 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:43.518672 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 499.68 MiB using storage policy from min volume index 0 2024.10.22 01:24:43.607169 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_472_472_0 (state Temporary) 2024.10.22 01:24:43.672491 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_814_814_0 (state Temporary) 2024.10.22 01:24:43.684497 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_472_472_0 to 202410_478_478_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:43.738123 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 56048 2024.10.22 01:24:43.923184 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_814_814_0 to 202410_827_827_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:43.973374 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 53901 2024.10.22 01:24:44.119373 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_724_337, 202410_725_725_0] 2024.10.22 01:24:44.121126 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_724_337, 202410_725_725_0] 2024.10.22 01:24:44.139442 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_724_337, 202410_725_725_0] 2024.10.22 01:24:45.413133 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/izg/alfgfifaeehuqeflfycbsxppckzmi (LogSeriesLimiter: on interval from 2024-10-22 01:24:37 to 2024-10-22 01:24:45 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:24:46.135393 [ 1676 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yci/zliaujazuovlbfwlrksqbqsfbgssw, size 2799559 2024.10.22 01:24:46.182112 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 727311 2024.10.22 01:24:46.624668 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 4.73 MiB, will set to 1.67 GiB (RSS), difference: -100.50 MiB 2024.10.22 01:24:46.970444 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:24:46.984891 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:24:47.003161 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:24:47.121411 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:24:47.157517 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:24:47.275580 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:24:47.306825 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:24:47.332461 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:24:47.471726 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:24:47.553414 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 407 2024.10.22 01:24:47.688010 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:47.697263 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:47.856564 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_462_462_0 (state Temporary) 2024.10.22 01:24:47.965042 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_462_462_0 to 202410_469_469_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:47.990001 [ 1274 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_464_93 to 202410_469_469_0 2024.10.22 01:24:47.991344 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 727311 2024.10.22 01:24:47.997682 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 53935 2024.10.22 01:24:48.015415 [ 1274 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:48.020323 [ 1274 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:48.055036 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:48.059056 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:48.059667 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_464_93 to 202410_469_469_0 into Wide with storage Full 2024.10.22 01:24:48.089571 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:48.100796 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} MergeTreeSequentialSource: Reading 91 marks from part 202410_1_464_93, total 732816 rows starting from the beginning of the part 2024.10.22 01:24:48.162484 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_465_465_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:24:48.184430 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_466_466_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:24:48.221331 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_389_389_0 (state Temporary) 2024.10.22 01:24:48.304484 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_467_467_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:24:48.361465 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:48.364021 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:48.412455 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_389_389_0 to 202410_399_399_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:48.468425 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_468_468_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:24:48.483881 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_469_469_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:24:48.557428 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 407 2024.10.22 01:24:48.630394 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.81 GiB, peak 1.97 GiB, free memory in arenas 5.05 MiB, will set to 1.67 GiB (RSS), difference: -146.85 MiB 2024.10.22 01:24:48.643505 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_815_815_0 (state Temporary) 2024.10.22 01:24:48.678296 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_815_815_0 to 202410_828_828_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:48.691806 [ 1272 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_817_420 to 202410_818_823_1 2024.10.22 01:24:48.703949 [ 1272 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.94 MiB using storage policy from min volume index 0 2024.10.22 01:24:48.711701 [ 1272 ] {} DiskLocal: Reserved 1.94 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:48.724982 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 53935 2024.10.22 01:24:48.758240 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 7470 2024.10.22 01:24:48.781958 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_823_421} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_817_420 to 202410_818_823_1 into Compact with storage Full 2024.10.22 01:24:48.874450 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:48.883072 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:48.936463 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_823_421} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:48.944446 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_482_482_0 (state Temporary) 2024.10.22 01:24:48.977602 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_823_421} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_817_420, total 53788 rows starting from the beginning of the part 2024.10.22 01:24:49.014878 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_482_482_0 to 202410_495_495_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:49.055789 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_823_421} MergeTreeSequentialSource: Reading 2 marks from part 202410_818_823_1, total 350 rows starting from the beginning of the part 2024.10.22 01:24:49.118626 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_725_338, 202410_726_726_0] 2024.10.22 01:24:49.124481 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_725_338, 202410_726_726_0] 2024.10.22 01:24:49.141277 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_725_338, 202410_726_726_0] 2024.10.22 01:24:49.172576 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7470 2024.10.22 01:24:50.015434 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/aht/jeocexuhoybagwdncmlgomzzhrcmn (LogSeriesLimiter: on interval from 2024-10-22 01:24:45 to 2024-10-22 01:24:50 accepted series 1 / 229 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:24:50.229643 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:24:50.231246 [ 1277 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:24:50.259854 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:24:50.286452 [ 1277 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:24:50.564163 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.85 GiB, peak 1.97 GiB, free memory in arenas 4.36 MiB, will set to 1.68 GiB (RSS), difference: -179.61 MiB 2024.10.22 01:24:50.916874 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_431_86, 202410_432_432_0, 202410_433_433_0, 202410_434_434_0, 202410_435_435_0, 202410_436_436_0] 2024.10.22 01:24:50.946465 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_431_86, 202410_432_432_0, 202410_433_433_0, 202410_434_434_0, 202410_435_435_0, 202410_436_436_0] 2024.10.22 01:24:50.980494 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_431_86, 202410_432_432_0, 202410_433_433_0, 202410_434_434_0, 202410_435_435_0, 202410_436_436_0] 2024.10.22 01:24:51.260913 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_469_94/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:24:51.414244 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_416_194, 202410_417_417_0] 2024.10.22 01:24:51.417709 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_416_194, 202410_417_417_0] 2024.10.22 01:24:51.436920 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_416_194, 202410_417_417_0] 2024.10.22 01:24:51.459611 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 111 entries to flush up to offset 56159 2024.10.22 01:24:51.685814 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:51.700875 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:51.977668 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_473_473_0 (state Temporary) 2024.10.22 01:24:52.072878 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_473_473_0 to 202410_479_479_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:52.133762 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_97_97_0 to 202410_100_100_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:52.142341 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 56159 2024.10.22 01:24:52.392209 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_99_20/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:24:52.395690 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.97 GiB, free memory in arenas 7.56 MiB, will set to 1.68 GiB (RSS), difference: -101.41 MiB 2024.10.22 01:24:52.478556 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_726_339, 202410_727_727_0] 2024.10.22 01:24:52.481099 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_726_339, 202410_727_727_0] 2024.10.22 01:24:52.503710 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_726_339, 202410_727_727_0] 2024.10.22 01:24:52.941220 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 54000 2024.10.22 01:24:53.028544 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_823_421} MergeTask::MergeProjectionsStage: Merge sorted 54138 rows, containing 13 columns (13 merged, 0 gathered) in 4.256711678 sec., 12718.268018903393 rows/sec., 2.03 MiB/sec. 2024.10.22 01:24:53.236173 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_823_421} MergedBlockOutputStream: filled checksums 202410_1_823_421 (state Temporary) 2024.10.22 01:24:53.341234 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:53.347548 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.10 GiB. 2024.10.22 01:24:53.420398 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_816_816_0 (state Temporary) 2024.10.22 01:24:53.472715 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_816_816_0 to 202410_829_829_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:53.579774 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_823_421} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_823_421 to 202410_1_823_421 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:53.584380 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_823_421} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_817_420, 202410_818_823_1] -> 202410_1_823_421 2024.10.22 01:24:53.707290 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_823_421 to 202410_824_824_0 2024.10.22 01:24:53.719668 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.95 MiB using storage policy from min volume index 0 2024.10.22 01:24:53.724435 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:24:53.724687 [ 1172 ] {} DiskLocal: Reserved 1.95 MiB on local disk `default`, having unreserved 297.10 GiB. 2024.10.22 01:24:53.733729 [ 1704 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/dou/paybvzcggfmainboorlqeamwmbcfw, size 23913 2024.10.22 01:24:53.755383 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54000 2024.10.22 01:24:53.780510 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_824_422} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_823_421 to 202410_824_824_0 into Compact with storage Full 2024.10.22 01:24:53.863073 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 6.37 MiB, will set to 1.68 GiB (RSS), difference: -27.74 MiB 2024.10.22 01:24:53.889365 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column EventTime 2024.10.22 01:24:53.896910 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_824_422} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:53.975877 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_824_422} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_823_421, total 54138 rows starting from the beginning of the part 2024.10.22 01:24:54.016012 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column EventTime 2024.10.22 01:24:54.030961 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column EventTime 2024.10.22 01:24:54.059146 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column EventTime 2024.10.22 01:24:54.091223 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_824_422} MergeTreeSequentialSource: Reading 2 marks from part 202410_824_824_0, total 260 rows starting from the beginning of the part 2024.10.22 01:24:54.178625 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column EventTime 2024.10.22 01:24:54.203291 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column EventTime 2024.10.22 01:24:54.214696 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column EventTime 2024.10.22 01:24:54.227231 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column EventTime 2024.10.22 01:24:55.004669 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 729236 2024.10.22 01:24:55.691410 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 10.30 MiB, will set to 1.70 GiB (RSS), difference: -17.40 MiB 2024.10.22 01:24:55.826339 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 4 old parts to remove. Parts: [202410_1_406_81, 202410_407_407_0, 202410_408_408_0, 202410_409_409_0] 2024.10.22 01:24:55.836491 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:55.841530 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:55.858300 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 4 parts from filesystem (serially): Parts: [202410_1_406_81, 202410_407_407_0, 202410_408_408_0, 202410_409_409_0] 2024.10.22 01:24:55.890420 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 4 parts from memory: Parts: [202410_1_406_81, 202410_407_407_0, 202410_408_408_0, 202410_409_409_0] 2024.10.22 01:24:56.060133 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 408 2024.10.22 01:24:56.177310 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_463_463_0 (state Temporary) 2024.10.22 01:24:56.345406 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_463_463_0 to 202410_470_470_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:56.391200 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 729236 2024.10.22 01:24:56.676985 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 7478 2024.10.22 01:24:56.783294 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:56.786501 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:56.817644 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:56.821713 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:56.871459 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_390_390_0 (state Temporary) 2024.10.22 01:24:56.889337 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_483_483_0 (state Temporary) 2024.10.22 01:24:56.935950 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3880 2024.10.22 01:24:56.958118 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_483_483_0 to 202410_496_496_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:56.983054 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_390_390_0 to 202410_400_400_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:57.002771 [ 1180 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_491_98 to 202410_496_496_0 2024.10.22 01:24:57.046607 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7478 2024.10.22 01:24:57.054110 [ 1180 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:57.061373 [ 1180 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:57.122695 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_491_98 to 202410_496_496_0 into Compact with storage Full 2024.10.22 01:24:57.204633 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 408 2024.10.22 01:24:57.287707 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:57.297472 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_491_98, total 7516 rows starting from the beginning of the part 2024.10.22 01:24:57.343128 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_492_492_0, total 23 rows starting from the beginning of the part 2024.10.22 01:24:57.405525 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_493_493_0, total 20 rows starting from the beginning of the part 2024.10.22 01:24:57.445641 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_494_494_0, total 11 rows starting from the beginning of the part 2024.10.22 01:24:57.473369 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_495_495_0, total 6 rows starting from the beginning of the part 2024.10.22 01:24:57.476212 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_824_422} MergeTask::MergeProjectionsStage: Merge sorted 54398 rows, containing 13 columns (13 merged, 0 gathered) in 3.701627279 sec., 14695.698918313487 rows/sec., 2.35 MiB/sec. 2024.10.22 01:24:57.533787 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_496_496_0, total 8 rows starting from the beginning of the part 2024.10.22 01:24:57.558706 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_824_422} MergedBlockOutputStream: filled checksums 202410_1_824_422 (state Temporary) 2024.10.22 01:24:57.718501 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 16.45 MiB, will set to 1.71 GiB (RSS), difference: 21.50 MiB 2024.10.22 01:24:57.763214 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 54065 2024.10.22 01:24:57.817342 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_824_422} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_824_422 to 202410_1_824_422 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:57.824099 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_824_422} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_823_421, 202410_824_824_0] -> 202410_1_824_422 2024.10.22 01:24:57.868921 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.86 MiB. 2024.10.22 01:24:57.966570 [ 1176 ] {} DatabaseCatalog: Execution took 259 ms. 2024.10.22 01:24:58.147543 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:24:58.150355 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:58.220554 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_817_817_0 (state Temporary) 2024.10.22 01:24:58.268619 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_817_817_0 to 202410_830_830_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:24:58.318969 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_824_422 to 202410_825_825_0 2024.10.22 01:24:58.335228 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.95 MiB using storage policy from min volume index 0 2024.10.22 01:24:58.342576 [ 1196 ] {} DiskLocal: Reserved 1.95 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:24:58.383962 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_825_423} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_824_422 to 202410_825_825_0 into Compact with storage Full 2024.10.22 01:24:58.441731 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54065 2024.10.22 01:24:58.496280 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_496_99/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:24:58.635092 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_825_423} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:24:58.660512 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_825_423} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_824_422, total 54398 rows starting from the beginning of the part 2024.10.22 01:24:58.711092 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_825_423} MergeTreeSequentialSource: Reading 2 marks from part 202410_825_825_0, total 50 rows starting from the beginning of the part 2024.10.22 01:24:58.711050 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_417_195, 202410_418_418_0] 2024.10.22 01:24:58.712980 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_417_195, 202410_418_418_0] 2024.10.22 01:24:58.854876 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_417_195, 202410_418_418_0] 2024.10.22 01:24:59.397484 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} MergeTask::MergeProjectionsStage: Merge sorted 7584 rows, containing 12 columns (12 merged, 0 gathered) in 2.284398229 sec., 3319.9115214337703 rows/sec., 527.30 KiB/sec. 2024.10.22 01:24:59.455300 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_825_423/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:24:59.502366 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 13.69 MiB, will set to 1.72 GiB (RSS), difference: -11.42 MiB 2024.10.22 01:24:59.516866 [ 1264 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 01:24:59.683014 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 262 entries to flush up to offset 56421 2024.10.22 01:24:59.867718 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} MergedBlockOutputStream: filled checksums 202410_1_496_99 (state Temporary) 2024.10.22 01:25:00.227241 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:25:00.261214 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:00.230211 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:25:00.270920 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:00.364662 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_496_99 to 202410_1_496_99 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:00.369255 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_496_99} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_491_98, 202410_496_496_0] -> 202410_1_496_99 2024.10.22 01:25:00.385191 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_474_474_0 (state Temporary) 2024.10.22 01:25:00.495657 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.47 MiB. 2024.10.22 01:25:00.563935 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_474_474_0 to 202410_480_480_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:00.617715 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 56421 2024.10.22 01:25:00.971964 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_727_340, 202410_728_728_0] 2024.10.22 01:25:00.974583 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_727_340, 202410_728_728_0] 2024.10.22 01:25:00.989214 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_727_340, 202410_728_728_0] 2024.10.22 01:25:01.503791 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 15.34 MiB, will set to 1.74 GiB (RSS), difference: 14.63 MiB 2024.10.22 01:25:01.623631 [ 1220 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 01:25:01.635656 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 212 ms. 2024.10.22 01:25:01.856855 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_469_94/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:25:01.867436 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_825_423} MergeTask::MergeProjectionsStage: Merge sorted 54448 rows, containing 13 columns (13 merged, 0 gathered) in 3.492523657 sec., 15589.872924946661 rows/sec., 2.49 MiB/sec. 2024.10.22 01:25:02.075074 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_825_423} MergedBlockOutputStream: filled checksums 202410_1_825_423 (state Temporary) 2024.10.22 01:25:02.358745 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_825_423} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_825_423 to 202410_1_825_423 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:02.362822 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_825_423} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_824_422, 202410_825_825_0] -> 202410_1_825_423 2024.10.22 01:25:02.390950 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:25:02.458809 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 54114 2024.10.22 01:25:02.807999 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:02.812161 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:02.877196 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_818_818_0 (state Temporary) 2024.10.22 01:25:02.918040 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_818_818_0 to 202410_831_831_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:02.978501 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_825_423 to 202410_826_826_0 2024.10.22 01:25:02.986315 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.96 MiB using storage policy from min volume index 0 2024.10.22 01:25:02.991049 [ 1198 ] {} DiskLocal: Reserved 1.96 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:03.017813 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54114 2024.10.22 01:25:03.019293 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_826_424} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_825_423 to 202410_826_826_0 into Compact with storage Full 2024.10.22 01:25:03.170695 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_729_729_0, 202410_730_730_0, 202410_731_731_0, 202410_732_732_0, 202410_733_733_0, 202410_734_734_0] 2024.10.22 01:25:03.176649 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_729_729_0, 202410_730_730_0, 202410_731_731_0, 202410_732_732_0, 202410_733_733_0, 202410_734_734_0] 2024.10.22 01:25:03.309149 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_729_729_0, 202410_730_730_0, 202410_731_731_0, 202410_732_732_0, 202410_733_733_0, 202410_734_734_0] 2024.10.22 01:25:03.334713 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_826_424} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:03.404837 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 730776 2024.10.22 01:25:03.477532 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_826_424} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_825_423, total 54448 rows starting from the beginning of the part 2024.10.22 01:25:03.646441 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 15.64 MiB, will set to 1.74 GiB (RSS), difference: 9.80 MiB 2024.10.22 01:25:03.676729 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_826_424} MergeTreeSequentialSource: Reading 2 marks from part 202410_826_826_0, total 230 rows starting from the beginning of the part 2024.10.22 01:25:04.207655 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_418_196, 202410_419_419_0] 2024.10.22 01:25:04.212321 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_418_196, 202410_419_419_0] 2024.10.22 01:25:04.293371 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_418_196, 202410_419_419_0] 2024.10.22 01:25:04.464301 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 3923 2024.10.22 01:25:04.555404 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:04.559165 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 7496 2024.10.22 01:25:04.560427 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:04.709750 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 409 2024.10.22 01:25:04.870169 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:04.874840 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:05.007703 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_464_464_0 (state Temporary) 2024.10.22 01:25:05.157665 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_484_484_0 (state Temporary) 2024.10.22 01:25:05.175286 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:05.177902 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:05.236216 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_464_464_0 to 202410_471_471_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:05.259508 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_391_391_0 (state Temporary) 2024.10.22 01:25:05.281257 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 730776 2024.10.22 01:25:05.296460 [ 1217 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:25:05.358271 [ 1217 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:25:05.454382 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_391_391_0 to 202410_401_401_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:05.519241 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_484_484_0 to 202410_497_497_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:05.565755 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for a200e7576e2e88de8f2ae31d69302bed:0) 2024.10.22 01:25:05.567391 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 210ff3222c5ca5050765a2bd51288b84, offset: 0, size: 1346870 2024.10.22 01:25:05.570040 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 210ff3222c5ca5050765a2bd51288b84 for removal 2024.10.22 01:25:05.572899 [ 1476 ] {} CacheMetadata: Key 210ff3222c5ca5050765a2bd51288b84 is removed from metadata 2024.10.22 01:25:05.574019 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 42b975b56c753212262f0a150fc3d215, offset: 0, size: 242 2024.10.22 01:25:05.588042 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 42b975b56c753212262f0a150fc3d215 for removal 2024.10.22 01:25:05.589556 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7496 2024.10.22 01:25:05.595779 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e331d615f96502829eca32cafb2e4928, offset: 0, size: 26979 2024.10.22 01:25:05.602670 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e331d615f96502829eca32cafb2e4928 for removal 2024.10.22 01:25:05.609920 [ 1476 ] {} CacheMetadata: Key 42b975b56c753212262f0a150fc3d215 is removed from metadata 2024.10.22 01:25:05.620213 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.97 GiB, free memory in arenas 17.52 MiB, will set to 1.75 GiB (RSS), difference: -4.15 MiB 2024.10.22 01:25:05.624565 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c446d8c9bf439ae5db8cf40e370a73a8, offset: 0, size: 1795 2024.10.22 01:25:05.635817 [ 1476 ] {} CacheMetadata: Key e331d615f96502829eca32cafb2e4928 is removed from metadata 2024.10.22 01:25:05.637356 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c446d8c9bf439ae5db8cf40e370a73a8 for removal 2024.10.22 01:25:05.647105 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 501f4c6df26ec0d00582739f12bfc6da, offset: 0, size: 152 2024.10.22 01:25:05.652572 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 501f4c6df26ec0d00582739f12bfc6da for removal 2024.10.22 01:25:05.657507 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 64c2d0b0ddd8fc3be79b574367c353e7, offset: 0, size: 151 2024.10.22 01:25:05.659382 [ 1476 ] {} CacheMetadata: Key 501f4c6df26ec0d00582739f12bfc6da is removed from metadata 2024.10.22 01:25:05.662638 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 64c2d0b0ddd8fc3be79b574367c353e7 for removal 2024.10.22 01:25:05.766552 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 409 2024.10.22 01:25:05.767210 [ 1476 ] {} CacheMetadata: Key 64c2d0b0ddd8fc3be79b574367c353e7 is removed from metadata 2024.10.22 01:25:05.773445 [ 1476 ] {} CacheMetadata: Key c446d8c9bf439ae5db8cf40e370a73a8 is removed from metadata 2024.10.22 01:25:05.863039 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_99_20/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:25:06.956551 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_826_424} MergeTask::MergeProjectionsStage: Merge sorted 54678 rows, containing 13 columns (13 merged, 0 gathered) in 3.946068311 sec., 13856.323735597896 rows/sec., 2.21 MiB/sec. 2024.10.22 01:25:07.022331 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_826_424} MergedBlockOutputStream: filled checksums 202410_1_826_424 (state Temporary) 2024.10.22 01:25:07.024362 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 54177 2024.10.22 01:25:07.335456 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_826_424} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_826_424 to 202410_1_826_424 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:07.342353 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_826_424} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_825_423, 202410_826_826_0] -> 202410_1_826_424 2024.10.22 01:25:07.372369 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.82 MiB. 2024.10.22 01:25:07.533189 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:07.539597 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:07.611851 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_819_819_0 (state Temporary) 2024.10.22 01:25:07.763224 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 15.84 MiB, will set to 1.75 GiB (RSS), difference: 11.24 MiB 2024.10.22 01:25:07.879314 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_819_819_0 to 202410_832_832_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:08.066772 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54177 2024.10.22 01:25:08.153488 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 109 entries to flush up to offset 56530 2024.10.22 01:25:08.212275 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 4194304 bytes for fe2a71089fb2b02b63c123fb480a7765:0) 2024.10.22 01:25:08.217529 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 42ced4d444b695630fece2b8964cdb67, offset: 0, size: 12656903 2024.10.22 01:25:08.262779 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_826_424 to 202410_827_827_0 2024.10.22 01:25:08.279537 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.96 MiB using storage policy from min volume index 0 2024.10.22 01:25:08.283468 [ 1150 ] {} DiskLocal: Reserved 1.96 MiB on local disk `default`, having unreserved 297.10 GiB. 2024.10.22 01:25:08.312586 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_827_425} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_826_424 to 202410_827_827_0 into Compact with storage Full 2024.10.22 01:25:08.329652 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 42ced4d444b695630fece2b8964cdb67 for removal 2024.10.22 01:25:08.335385 [ 1476 ] {} CacheMetadata: Key 42ced4d444b695630fece2b8964cdb67 is removed from metadata 2024.10.22 01:25:08.335860 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 96ac6400d57df25a1b62614393e02e16, offset: 0, size: 110135 2024.10.22 01:25:08.367138 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:08.370215 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.10 GiB. 2024.10.22 01:25:08.400940 [ 1150 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 485 ms. 2024.10.22 01:25:08.444864 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 96ac6400d57df25a1b62614393e02e16 for removal 2024.10.22 01:25:08.451805 [ 1476 ] {} CacheMetadata: Key 96ac6400d57df25a1b62614393e02e16 is removed from metadata 2024.10.22 01:25:08.454460 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6cb9e4b2837f02520b78842c0be49c39, offset: 0, size: 249 2024.10.22 01:25:08.460126 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6cb9e4b2837f02520b78842c0be49c39 for removal 2024.10.22 01:25:08.463506 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5aca421cc1dfd2d9c5eb6e52b33e93e8, offset: 0, size: 688065 2024.10.22 01:25:08.469615 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5aca421cc1dfd2d9c5eb6e52b33e93e8 for removal 2024.10.22 01:25:08.473454 [ 1476 ] {} CacheMetadata: Key 6cb9e4b2837f02520b78842c0be49c39 is removed from metadata 2024.10.22 01:25:08.482736 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_827_425} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:08.493120 [ 1476 ] {} CacheMetadata: Key 5aca421cc1dfd2d9c5eb6e52b33e93e8 is removed from metadata 2024.10.22 01:25:08.494464 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 17e9de29970708fb794112396ad0ff3e, offset: 0, size: 208 2024.10.22 01:25:08.504692 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 17e9de29970708fb794112396ad0ff3e for removal 2024.10.22 01:25:08.516451 [ 1476 ] {} CacheMetadata: Key 17e9de29970708fb794112396ad0ff3e is removed from metadata 2024.10.22 01:25:08.517674 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 98500ba549094cfe8bae5e44d5fa94ec, offset: 0, size: 1060 2024.10.22 01:25:08.529566 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 98500ba549094cfe8bae5e44d5fa94ec for removal 2024.10.22 01:25:08.533207 [ 1476 ] {} CacheMetadata: Key 98500ba549094cfe8bae5e44d5fa94ec is removed from metadata 2024.10.22 01:25:08.535501 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9159519cbcc401b228c41ce1f782db00, offset: 0, size: 1794070 2024.10.22 01:25:08.535531 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_475_475_0 (state Temporary) 2024.10.22 01:25:08.542345 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9159519cbcc401b228c41ce1f782db00 for removal 2024.10.22 01:25:08.624644 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_475_475_0 to 202410_481_481_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:08.643052 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_827_425} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_826_424, total 54678 rows starting from the beginning of the part 2024.10.22 01:25:08.650550 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0c062a19c4c6b0ba27c6d19d7fbefe47, offset: 0, size: 153 2024.10.22 01:25:08.655079 [ 1476 ] {} CacheMetadata: Key 9159519cbcc401b228c41ce1f782db00 is removed from metadata 2024.10.22 01:25:08.655269 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0c062a19c4c6b0ba27c6d19d7fbefe47 for removal 2024.10.22 01:25:08.660889 [ 1476 ] {} CacheMetadata: Key 0c062a19c4c6b0ba27c6d19d7fbefe47 is removed from metadata 2024.10.22 01:25:08.662208 [ 1269 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_461_476_3 to 202410_481_481_0 2024.10.22 01:25:08.681698 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_827_425} MergeTreeSequentialSource: Reading 2 marks from part 202410_827_827_0, total 54 rows starting from the beginning of the part 2024.10.22 01:25:08.682116 [ 1269 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:08.690638 [ 1269 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.10 GiB. 2024.10.22 01:25:08.809725 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_461_476_3 to 202410_481_481_0 into Compact with storage Full 2024.10.22 01:25:08.836534 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 56530 2024.10.22 01:25:08.970268 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:08.981024 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_476_3, total 3454 rows starting from the beginning of the part 2024.10.22 01:25:09.023764 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_477_477_0, total 93 rows starting from the beginning of the part 2024.10.22 01:25:09.083434 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_478_478_0, total 333 rows starting from the beginning of the part 2024.10.22 01:25:09.162513 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_479_479_0, total 111 rows starting from the beginning of the part 2024.10.22 01:25:09.174097 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column WindowName 2024.10.22 01:25:09.197351 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column WindowName 2024.10.22 01:25:09.207497 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_480_480_0, total 262 rows starting from the beginning of the part 2024.10.22 01:25:09.213321 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column WindowName 2024.10.22 01:25:09.241779 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column WindowName 2024.10.22 01:25:09.243862 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_481_481_0, total 109 rows starting from the beginning of the part 2024.10.22 01:25:09.269463 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column WindowName 2024.10.22 01:25:09.304831 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column WindowName 2024.10.22 01:25:09.328561 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column WindowName 2024.10.22 01:25:09.362156 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column WindowName 2024.10.22 01:25:09.382852 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column WindowName 2024.10.22 01:25:09.471355 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.97 GiB, free memory in arenas 11.68 MiB, will set to 1.76 GiB (RSS), difference: -8.01 MiB 2024.10.22 01:25:09.550067 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tzw/kqsxcgcwjjrwjxnerchephocnllgv (LogSeriesLimiter: on interval from 2024-10-22 01:24:50 to 2024-10-22 01:25:09 accepted series 1 / 130 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:25:10.229155 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:25:10.335326 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:25:10.734782 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} MergeTask::MergeProjectionsStage: Merge sorted 4362 rows, containing 13 columns (13 merged, 0 gathered) in 2.00854621 sec., 2171.720012356599 rows/sec., 970.49 KiB/sec. 2024.10.22 01:25:10.995918 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} MergedBlockOutputStream: filled checksums 202410_461_481_4 (state Temporary) 2024.10.22 01:25:11.038819 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_827_425} MergeTask::MergeProjectionsStage: Merge sorted 54732 rows, containing 13 columns (13 merged, 0 gathered) in 2.73388307 sec., 20019.87597809002 rows/sec., 3.20 MiB/sec. 2024.10.22 01:25:11.222819 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_827_425} MergedBlockOutputStream: filled checksums 202410_1_827_425 (state Temporary) 2024.10.22 01:25:11.369274 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_827_425} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_827_425 to 202410_1_827_425 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:11.372258 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_827_425} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_826_424, 202410_827_827_0] -> 202410_1_827_425 2024.10.22 01:25:11.399133 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_461_481_4 to 202410_461_481_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:11.400651 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:25:11.508310 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_481_4} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_461_476_3, 202410_481_481_0] -> 202410_461_481_4 2024.10.22 01:25:11.556783 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.04 MiB. 2024.10.22 01:25:11.797826 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 9.26 MiB, will set to 1.76 GiB (RSS), difference: 14.85 MiB 2024.10.22 01:25:11.853992 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:25:11.877622 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:25:11.899889 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:25:11.934942 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:25:11.973531 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:25:12.019422 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:25:12.040880 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:25:12.074214 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 54269 2024.10.22 01:25:12.084213 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:25:12.286641 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1651 entries to flush up to offset 732427 2024.10.22 01:25:12.597549 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:12.607134 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:12.889178 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_820_820_0 (state Temporary) 2024.10.22 01:25:12.948454 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_820_820_0 to 202410_833_833_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:12.983765 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_827_425 to 202410_828_828_0 2024.10.22 01:25:12.992613 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54269 2024.10.22 01:25:12.992304 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.96 MiB using storage policy from min volume index 0 2024.10.22 01:25:12.997112 [ 1265 ] {} DiskLocal: Reserved 1.96 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:13.009020 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_828_426} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_827_425 to 202410_828_828_0 into Compact with storage Full 2024.10.22 01:25:13.079167 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_828_426} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:13.095099 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 7515 2024.10.22 01:25:13.102577 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_828_426} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_827_425, total 54732 rows starting from the beginning of the part 2024.10.22 01:25:13.178144 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_828_426} MergeTreeSequentialSource: Reading 2 marks from part 202410_828_828_0, total 34 rows starting from the beginning of the part 2024.10.22 01:25:13.184913 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_469_94/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:25:13.193559 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:13.206281 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:25:13.268484 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 410 2024.10.22 01:25:13.358461 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_419_197, 202410_420_420_0] 2024.10.22 01:25:13.362205 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_419_197, 202410_420_420_0] 2024.10.22 01:25:13.378711 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_419_197, 202410_420_420_0] 2024.10.22 01:25:13.466284 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:13.468856 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:13.586174 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_465_465_0 (state Temporary) 2024.10.22 01:25:13.628621 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_485_485_0 (state Temporary) 2024.10.22 01:25:13.689429 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 11.72 MiB, will set to 1.76 GiB (RSS), difference: -11.85 MiB 2024.10.22 01:25:13.695586 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_485_485_0 to 202410_498_498_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:13.726802 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:25:13.756580 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_465_465_0 to 202410_472_472_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:13.772861 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} MergeTask::MergeProjectionsStage: Merge sorted 740901 rows, containing 4 columns (4 merged, 0 gathered) in 25.724251381 sec., 28801.6544787473 rows/sec., 431.99 KiB/sec. 2024.10.22 01:25:13.799762 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7515 2024.10.22 01:25:13.800820 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 732427 2024.10.22 01:25:13.805224 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:13.810135 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:13.867384 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} MergedBlockOutputStream: filled checksums 202410_1_469_94 (state Temporary) 2024.10.22 01:25:13.986671 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_392_392_0 (state Temporary) 2024.10.22 01:25:14.082520 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_392_392_0 to 202410_402_402_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:14.322349 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 410 2024.10.22 01:25:15.626879 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 11.43 MiB, will set to 1.78 GiB (RSS), difference: 28.67 MiB 2024.10.22 01:25:16.360129 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 156 entries to flush up to offset 56686 2024.10.22 01:25:16.571237 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_828_426} MergeTask::MergeProjectionsStage: Merge sorted 54766 rows, containing 13 columns (13 merged, 0 gathered) in 3.56836593 sec., 15347.641210104255 rows/sec., 2.45 MiB/sec. 2024.10.22 01:25:16.629175 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:16.634720 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:16.667912 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_828_426} MergedBlockOutputStream: filled checksums 202410_1_828_426 (state Temporary) 2024.10.22 01:25:16.716113 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_469_94 to 202410_1_469_94 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:16.719992 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_469_94} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_464_93, 202410_469_469_0] -> 202410_1_469_94 2024.10.22 01:25:16.753640 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:25:16.787565 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_476_476_0 (state Temporary) 2024.10.22 01:25:16.839800 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_476_476_0 to 202410_482_482_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:16.909467 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 56686 2024.10.22 01:25:17.001725 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_99_20/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:25:17.074753 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_828_426} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_828_426 to 202410_1_828_426 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:17.177539 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 54321 2024.10.22 01:25:17.183297 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_828_426} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_827_425, 202410_828_828_0] -> 202410_1_828_426 2024.10.22 01:25:17.335790 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_349_72, 202410_350_350_0, 202410_351_351_0, 202410_352_352_0, 202410_353_353_0, 202410_354_354_0] 2024.10.22 01:25:17.381101 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:25:17.340057 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_349_72, 202410_350_350_0, 202410_351_351_0, 202410_352_352_0, 202410_353_353_0, 202410_354_354_0] 2024.10.22 01:25:17.490778 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_349_72, 202410_350_350_0, 202410_351_351_0, 202410_352_352_0, 202410_353_353_0, 202410_354_354_0] 2024.10.22 01:25:17.613040 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.97 GiB, free memory in arenas 12.85 MiB, will set to 1.77 GiB (RSS), difference: 11.58 MiB 2024.10.22 01:25:17.747090 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:17.750507 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:17.815507 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_821_821_0 (state Temporary) 2024.10.22 01:25:17.959914 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_821_821_0 to 202410_834_834_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:18.083183 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_828_426 to 202410_829_829_0 2024.10.22 01:25:18.086281 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.97 MiB using storage policy from min volume index 0 2024.10.22 01:25:18.088498 [ 1185 ] {} DiskLocal: Reserved 1.97 MiB on local disk `default`, having unreserved 297.09 GiB. 2024.10.22 01:25:18.217813 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_829_427} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_828_426 to 202410_829_829_0 into Compact with storage Full 2024.10.22 01:25:18.279212 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_829_427} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:18.280732 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54321 2024.10.22 01:25:18.288113 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_829_427} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_828_426, total 54766 rows starting from the beginning of the part 2024.10.22 01:25:18.346201 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_829_427} MergeTreeSequentialSource: Reading 2 marks from part 202410_829_829_0, total 65 rows starting from the beginning of the part 2024.10.22 01:25:19.500040 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.97 GiB, free memory in arenas 9.37 MiB, will set to 1.78 GiB (RSS), difference: -16.04 MiB 2024.10.22 01:25:20.238334 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:25:20.244575 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:25:20.364967 [ 1207 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:25:20.385082 [ 1207 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:25:20.810848 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1814 entries to flush up to offset 734241 2024.10.22 01:25:21.308951 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 7530 2024.10.22 01:25:21.376268 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_829_427/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:25:21.448449 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:21.453456 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:25:21.477945 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_829_427} MergeTask::MergeProjectionsStage: Merge sorted 54831 rows, containing 13 columns (13 merged, 0 gathered) in 3.371587881 sec., 16262.663746358388 rows/sec., 2.60 MiB/sec. 2024.10.22 01:25:21.498493 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_486_486_0 (state Temporary) 2024.10.22 01:25:21.535726 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_829_427} MergedBlockOutputStream: filled checksums 202410_1_829_427 (state Temporary) 2024.10.22 01:25:21.662443 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_829_427} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_829_427 to 202410_1_829_427 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:21.670510 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_829_427} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_828_426, 202410_829_829_0] -> 202410_1_829_427 2024.10.22 01:25:21.684783 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.97 GiB, free memory in arenas 7.31 MiB, will set to 1.79 GiB (RSS), difference: 9.14 MiB 2024.10.22 01:25:21.701514 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_486_486_0 to 202410_499_499_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:21.714949 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:25:21.825520 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 411 2024.10.22 01:25:21.833311 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:21.845355 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:25:21.875021 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7530 2024.10.22 01:25:21.986530 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_466_466_0 (state Temporary) 2024.10.22 01:25:22.059420 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_466_466_0 to 202410_473_473_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:22.134181 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 734241 2024.10.22 01:25:22.296163 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 54367 2024.10.22 01:25:22.313019 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 5 old parts to remove. Parts: [202410_1_409_82, 202410_410_410_0, 202410_411_411_0, 202410_412_412_0, 202410_413_413_0] 2024.10.22 01:25:22.319857 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 5 parts from filesystem (serially): Parts: [202410_1_409_82, 202410_410_410_0, 202410_411_411_0, 202410_412_412_0, 202410_413_413_0] 2024.10.22 01:25:22.363737 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 5 parts from memory: Parts: [202410_1_409_82, 202410_410_410_0, 202410_411_411_0, 202410_412_412_0, 202410_413_413_0] 2024.10.22 01:25:22.396686 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:22.406434 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:25:22.504201 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_393_393_0 (state Temporary) 2024.10.22 01:25:22.529142 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:22.545207 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:25:22.681108 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_822_822_0 (state Temporary) 2024.10.22 01:25:22.854381 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_393_393_0 to 202410_403_403_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:22.883328 [ 1225 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_398_82 to 202410_403_403_0 2024.10.22 01:25:22.912020 [ 1225 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:22.938285 [ 1225 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:25:22.963269 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_822_822_0 to 202410_835_835_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:22.981005 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_398_82 to 202410_403_403_0 into Compact with storage Full 2024.10.22 01:25:23.017599 [ 1230 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_829_427 to 202410_830_830_0 2024.10.22 01:25:23.025316 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54367 2024.10.22 01:25:23.027223 [ 1230 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.97 MiB using storage policy from min volume index 0 2024.10.22 01:25:23.031470 [ 1230 ] {} DiskLocal: Reserved 1.97 MiB on local disk `default`, having unreserved 297.08 GiB. 2024.10.22 01:25:23.070746 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 411 2024.10.22 01:25:23.146527 [ 1690 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/aht/jeocexuhoybagwdncmlgomzzhrcmn, size 543667 2024.10.22 01:25:23.190807 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_830_428} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_829_427 to 202410_830_830_0 into Compact with storage Full 2024.10.22 01:25:23.256287 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:23.289255 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_398_82, total 442 rows starting from the beginning of the part 2024.10.22 01:25:23.319657 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_830_428} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:23.375835 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_830_428} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_829_427, total 54831 rows starting from the beginning of the part 2024.10.22 01:25:23.475137 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_399_399_0, total 1 rows starting from the beginning of the part 2024.10.22 01:25:23.664885 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_830_428} MergeTreeSequentialSource: Reading 2 marks from part 202410_830_830_0, total 65 rows starting from the beginning of the part 2024.10.22 01:25:23.669723 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_400_400_0, total 1 rows starting from the beginning of the part 2024.10.22 01:25:23.776954 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_401_401_0, total 1 rows starting from the beginning of the part 2024.10.22 01:25:23.872144 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_402_402_0, total 1 rows starting from the beginning of the part 2024.10.22 01:25:24.115084 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_403_403_0, total 1 rows starting from the beginning of the part 2024.10.22 01:25:24.260088 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:24.263347 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.06 GiB. 2024.10.22 01:25:24.350169 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_420_198, 202410_421_421_0] 2024.10.22 01:25:24.355558 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_420_198, 202410_421_421_0] 2024.10.22 01:25:24.375950 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_420_198, 202410_421_421_0] 2024.10.22 01:25:24.470633 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 136 entries to flush up to offset 56822 2024.10.22 01:25:24.787328 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:24.798570 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.06 GiB. 2024.10.22 01:25:24.869163 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 15.41 MiB, will set to 1.78 GiB (RSS), difference: 124.68 MiB 2024.10.22 01:25:24.921205 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_477_477_0 (state Temporary) 2024.10.22 01:25:25.167236 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_477_477_0 to 202410_483_483_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:25.432868 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 56822 2024.10.22 01:25:26.764883 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 14.48 MiB, will set to 1.77 GiB (RSS), difference: 94.78 MiB 2024.10.22 01:25:27.146051 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 54413 2024.10.22 01:25:27.393658 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:27.399864 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:25:27.704101 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_823_823_0 (state Temporary) 2024.10.22 01:25:27.882227 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:25:28.023059 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_823_823_0 to 202410_836_836_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:28.066132 [ 1274 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_831_831_0 to 202410_836_836_0 2024.10.22 01:25:28.079681 [ 1274 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:28.082525 [ 1274 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:25:28.100071 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54413 2024.10.22 01:25:28.126418 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} MergeTask::MergeProjectionsStage: Merge sorted 447 rows, containing 32 columns (32 merged, 0 gathered) in 5.035132266 sec., 88.77621805853869 rows/sec., 8.28 KiB/sec. 2024.10.22 01:25:28.131355 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_2_2_0 (state Temporary) 2024.10.22 01:25:28.175067 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lzw/plfdapeixwwvsouasclinviihgegv (LogSeriesLimiter: on interval from 2024-10-22 01:25:09 to 2024-10-22 01:25:28 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:25:28.214351 [ 1722 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/izg/alfgfifaeehuqeflfycbsxppckzmi, size 600 2024.10.22 01:25:28.304016 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} MergedBlockOutputStream: filled checksums 202410_1_403_83 (state Temporary) 2024.10.22 01:25:28.310871 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_831_831_0 to 202410_836_836_0 into Compact with storage Full 2024.10.22 01:25:28.367983 [ 1680 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lzw/plfdapeixwwvsouasclinviihgegv, size 4 2024.10.22 01:25:28.377350 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:28.405578 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_831_831_0, total 49 rows starting from the beginning of the part 2024.10.22 01:25:28.416111 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_830_428} MergeTask::MergeProjectionsStage: Merge sorted 54896 rows, containing 13 columns (13 merged, 0 gathered) in 5.238936169 sec., 10478.463227865299 rows/sec., 1.67 MiB/sec. 2024.10.22 01:25:28.468927 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_832_832_0, total 63 rows starting from the beginning of the part 2024.10.22 01:25:28.491966 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.97 GiB, free memory in arenas 11.82 MiB, will set to 1.71 GiB (RSS), difference: 109.09 MiB 2024.10.22 01:25:28.516886 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_830_428} MergedBlockOutputStream: filled checksums 202410_1_830_428 (state Temporary) 2024.10.22 01:25:28.669796 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_98_98_0 (state Temporary) 2024.10.22 01:25:28.775017 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_833_833_0, total 92 rows starting from the beginning of the part 2024.10.22 01:25:28.855999 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_834_834_0, total 52 rows starting from the beginning of the part 2024.10.22 01:25:28.947998 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_835_835_0, total 46 rows starting from the beginning of the part 2024.10.22 01:25:28.949390 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_830_428} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_830_428 to 202410_1_830_428 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:28.953350 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_830_428} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_829_427, 202410_830_830_0] -> 202410_1_830_428 2024.10.22 01:25:28.983734 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:25:29.129503 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_403_83 to 202410_1_403_83 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:29.140840 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1477 entries to flush up to offset 735718 2024.10.22 01:25:29.142965 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_403_83} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_398_82, 202410_403_403_0] -> 202410_1_403_83 2024.10.22 01:25:29.165456 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_836_836_0, total 46 rows starting from the beginning of the part 2024.10.22 01:25:29.186759 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:25:29.380205 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 7549 2024.10.22 01:25:29.411119 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_728_341, 202410_729_734_1] 2024.10.22 01:25:29.415360 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_728_341, 202410_729_734_1] 2024.10.22 01:25:29.529137 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_728_341, 202410_729_734_1] 2024.10.22 01:25:29.579799 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:29.583533 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:29.625517 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_487_487_0 (state Temporary) 2024.10.22 01:25:29.883620 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_487_487_0 to 202410_500_500_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:30.056045 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7549 2024.10.22 01:25:30.230457 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:25:30.234434 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:25:30.379563 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:30.384863 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:30.588886 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 412 2024.10.22 01:25:30.643895 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:25:30.691164 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} MergeTask::MergeProjectionsStage: Merge sorted 348 rows, containing 13 columns (13 merged, 0 gathered) in 2.464235952 sec., 141.22024301997521 rows/sec., 21.73 KiB/sec. 2024.10.22 01:25:30.712174 [ 1712 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tzw/kqsxcgcwjjrwjxnerchephocnllgv, size 2176921 2024.10.22 01:25:30.745466 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} MergedBlockOutputStream: filled checksums 202410_831_836_1 (state Temporary) 2024.10.22 01:25:30.796345 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_831_836_1 to 202410_831_836_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:30.801102 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:30.804391 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_831_836_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_831_831_0, 202410_836_836_0] -> 202410_831_836_1 2024.10.22 01:25:30.805518 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.99 GiB. 2024.10.22 01:25:30.830151 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 12.85 MiB, will set to 1.69 GiB (RSS), difference: -8.72 MiB 2024.10.22 01:25:30.860245 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:25:30.876231 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_467_467_0 (state Temporary) 2024.10.22 01:25:31.066157 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_467_467_0 to 202410_474_474_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:31.095165 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:25:31.115058 [ 1171 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_469_94 to 202410_474_474_0 2024.10.22 01:25:31.117182 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:25:31.126924 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 735718 2024.10.22 01:25:31.135204 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:25:31.144986 [ 1171 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:31.151364 [ 1171 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.99 GiB. 2024.10.22 01:25:31.157539 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:25:31.171329 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_469_94 to 202410_474_474_0 into Wide with storage Full 2024.10.22 01:25:31.178043 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:25:31.185042 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:31.202934 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_394_394_0 (state Temporary) 2024.10.22 01:25:31.202915 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} MergeTreeSequentialSource: Reading 92 marks from part 202410_1_469_94, total 740901 rows starting from the beginning of the part 2024.10.22 01:25:31.217271 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:25:31.219002 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:25:31.241537 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:25:31.252297 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:25:31.267769 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_470_470_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:25:31.287152 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_471_471_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:25:31.288491 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:25:31.309076 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_472_472_0, total 1651 rows starting from the beginning of the part 2024.10.22 01:25:31.312138 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:25:31.351018 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:25:31.361199 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:25:31.364789 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_473_473_0, total 1814 rows starting from the beginning of the part 2024.10.22 01:25:31.373494 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:25:31.386072 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_474_474_0, total 1477 rows starting from the beginning of the part 2024.10.22 01:25:31.424378 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:25:31.452385 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:25:31.488425 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_394_394_0 to 202410_404_404_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:31.692461 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 412 2024.10.22 01:25:31.754558 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_421_199, 202410_422_422_0] 2024.10.22 01:25:31.757545 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_421_199, 202410_422_422_0] 2024.10.22 01:25:31.987028 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_421_199, 202410_422_422_0] 2024.10.22 01:25:32.109111 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 95 entries to flush up to offset 54508 2024.10.22 01:25:32.404107 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 11.98 MiB, will set to 1.70 GiB (RSS), difference: -12.84 MiB 2024.10.22 01:25:32.780962 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:32.791163 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:32.883390 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_824_824_0 (state Temporary) 2024.10.22 01:25:33.011274 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 174 entries to flush up to offset 56996 2024.10.22 01:25:33.058132 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_824_824_0 to 202410_837_837_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:33.107706 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_436_87, 202410_437_437_0, 202410_438_438_0, 202410_439_439_0, 202410_440_440_0, 202410_441_441_0] 2024.10.22 01:25:33.118913 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_436_87, 202410_437_437_0, 202410_438_438_0, 202410_439_439_0, 202410_440_440_0, 202410_441_441_0] 2024.10.22 01:25:33.152450 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_436_87, 202410_437_437_0, 202410_438_438_0, 202410_439_439_0, 202410_440_440_0, 202410_441_441_0] 2024.10.22 01:25:33.211438 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54508 2024.10.22 01:25:33.652158 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:33.657150 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:33.815176 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_478_478_0 (state Temporary) 2024.10.22 01:25:33.975814 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_478_478_0 to 202410_484_484_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:34.121068 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 56996 2024.10.22 01:25:34.413295 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_734_342, 202410_735_735_0] 2024.10.22 01:25:34.420067 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_734_342, 202410_735_735_0] 2024.10.22 01:25:34.429701 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_734_342, 202410_735_735_0] 2024.10.22 01:25:34.484222 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 13.92 MiB, will set to 1.70 GiB (RSS), difference: -1.28 MiB 2024.10.22 01:25:35.392235 [ 1228 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:25:35.451990 [ 1228 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:25:35.700559 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_474_95/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:25:36.581421 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 12.06 MiB, will set to 1.70 GiB (RSS), difference: -2.73 MiB 2024.10.22 01:25:36.780589 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_422_200, 202410_423_423_0] 2024.10.22 01:25:36.783776 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_422_200, 202410_423_423_0] 2024.10.22 01:25:36.808911 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_422_200, 202410_423_423_0] 2024.10.22 01:25:37.238631 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 54535 2024.10.22 01:25:37.512302 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:37.517191 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:37.559660 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 7561 2024.10.22 01:25:37.683657 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:37.692145 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:37.755459 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_488_488_0 (state Temporary) 2024.10.22 01:25:37.778786 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_825_825_0 (state Temporary) 2024.10.22 01:25:37.929718 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_488_488_0 to 202410_501_501_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:37.975708 [ 1175 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_496_99 to 202410_501_501_0 2024.10.22 01:25:37.986819 [ 1175 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:37.992160 [ 1175 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:38.022676 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_496_99 to 202410_501_501_0 into Compact with storage Full 2024.10.22 01:25:38.075427 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_825_825_0 to 202410_838_838_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:38.076375 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:38.089740 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_496_99, total 7584 rows starting from the beginning of the part 2024.10.22 01:25:38.120239 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_497_497_0, total 18 rows starting from the beginning of the part 2024.10.22 01:25:38.133530 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7561 2024.10.22 01:25:38.141271 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1603 entries to flush up to offset 737321 2024.10.22 01:25:38.176675 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54535 2024.10.22 01:25:38.187072 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_498_498_0, total 19 rows starting from the beginning of the part 2024.10.22 01:25:38.328234 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_499_499_0, total 15 rows starting from the beginning of the part 2024.10.22 01:25:38.385742 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_98_98_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:25:38.414898 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_500_500_0, total 19 rows starting from the beginning of the part 2024.10.22 01:25:38.512498 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vhe/zswtobyhtxvftssjaxiytaecqkrud. 2024.10.22 01:25:38.571078 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_501_501_0, total 12 rows starting from the beginning of the part 2024.10.22 01:25:38.725822 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:38.729630 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:38.929273 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_98_98_0 to 202410_101_101_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:39.075401 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_468_468_0 (state Temporary) 2024.10.22 01:25:39.196202 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_468_468_0 to 202410_475_475_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:39.495961 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 737321 2024.10.22 01:25:39.981349 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} MergeTask::MergeProjectionsStage: Merge sorted 7667 rows, containing 12 columns (12 merged, 0 gathered) in 1.970078288 sec., 3891.7235151012433 rows/sec., 618.27 KiB/sec. 2024.10.22 01:25:40.079870 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_735_343, 202410_736_736_0] 2024.10.22 01:25:40.086780 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_735_343, 202410_736_736_0] 2024.10.22 01:25:40.118725 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_735_343, 202410_736_736_0] 2024.10.22 01:25:40.230521 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:25:40.238718 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:25:40.267426 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} MergedBlockOutputStream: filled checksums 202410_1_501_100 (state Temporary) 2024.10.22 01:25:40.861026 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_501_100 to 202410_1_501_100 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:40.864232 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_501_100} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_496_99, 202410_501_501_0] -> 202410_1_501_100 2024.10.22 01:25:40.939044 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.48 MiB. 2024.10.22 01:25:41.667587 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 57075 2024.10.22 01:25:41.927277 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:41.931503 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:42.072499 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_479_479_0 (state Temporary) 2024.10.22 01:25:42.197026 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 54581 2024.10.22 01:25:42.358233 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_479_479_0 to 202410_485_485_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:42.475472 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:42.481144 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:42.620253 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 15.54 MiB, will set to 1.70 GiB (RSS), difference: 7.78 MiB 2024.10.22 01:25:42.634792 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_826_826_0 (state Temporary) 2024.10.22 01:25:42.698634 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57075 2024.10.22 01:25:42.704444 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_826_826_0 to 202410_839_839_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:42.710417 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3923 2024.10.22 01:25:42.767978 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54581 2024.10.22 01:25:44.526853 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 15.13 MiB, will set to 1.68 GiB (RSS), difference: -3.32 MiB 2024.10.22 01:25:44.574154 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_736_344, 202410_737_737_0] 2024.10.22 01:25:44.575533 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_736_344, 202410_737_737_0] 2024.10.22 01:25:44.584228 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_736_344, 202410_737_737_0] 2024.10.22 01:25:45.637547 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 7574 2024.10.22 01:25:46.058813 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:46.060206 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:46.120129 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_489_489_0 (state Temporary) 2024.10.22 01:25:46.172315 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_489_489_0 to 202410_502_502_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:46.331050 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7574 2024.10.22 01:25:46.505179 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 738861 2024.10.22 01:25:46.543050 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_423_201, 202410_424_424_0] 2024.10.22 01:25:46.547438 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_423_201, 202410_424_424_0] 2024.10.22 01:25:46.566451 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 13.43 MiB, will set to 1.68 GiB (RSS), difference: 3.81 MiB 2024.10.22 01:25:46.572760 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_423_201, 202410_424_424_0] 2024.10.22 01:25:46.699698 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 413 2024.10.22 01:25:46.773553 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 54607 2024.10.22 01:25:46.934595 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:46.943864 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:46.981530 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:46.985450 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:47.082376 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_827_827_0 (state Temporary) 2024.10.22 01:25:47.083571 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_474_95/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:25:47.221939 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_827_827_0 to 202410_840_840_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:47.373712 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_395_395_0 (state Temporary) 2024.10.22 01:25:47.382717 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54607 2024.10.22 01:25:47.445545 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_395_395_0 to 202410_405_405_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:47.559399 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} MergeTask::MergeProjectionsStage: Merge sorted 3899 rows, containing 679 columns (679 merged, 0 gathered) in 97.023949695 sec., 40.18595421292079 rows/sec., 54.84 KiB/sec. 2024.10.22 01:25:47.590605 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} MergedBlockOutputStream: filled checksums 202410_1_99_20 (state Temporary) 2024.10.22 01:25:47.664319 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:47.666850 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:47.710777 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 413 2024.10.22 01:25:47.836037 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_469_469_0 (state Temporary) 2024.10.22 01:25:47.928010 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_469_469_0 to 202410_476_476_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:47.968658 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 738861 2024.10.22 01:25:48.620659 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 14.47 MiB, will set to 1.68 GiB (RSS), difference: 2.94 MiB 2024.10.22 01:25:48.906953 [ 1274 ] {} DatabaseCatalog: Execution took 364 ms. 2024.10.22 01:25:49.549828 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_737_345, 202410_738_738_0] 2024.10.22 01:25:49.553152 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_737_345, 202410_738_738_0] 2024.10.22 01:25:49.603001 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column RegionID 2024.10.22 01:25:49.628564 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column RegionID 2024.10.22 01:25:49.653892 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column RegionID 2024.10.22 01:25:49.666770 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_737_345, 202410_738_738_0] 2024.10.22 01:25:49.676370 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column RegionID 2024.10.22 01:25:49.708886 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column RegionID 2024.10.22 01:25:49.846598 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column RegionID 2024.10.22 01:25:49.875507 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column RegionID 2024.10.22 01:25:49.912398 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column RegionID 2024.10.22 01:25:49.985469 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vfy/qaxpxrrkahsptutwfluembgciztss (LogSeriesLimiter: on interval from 2024-10-22 01:25:28 to 2024-10-22 01:25:49 accepted series 1 / 12 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:25:50.147844 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:25:50.181443 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:25:50.212788 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:25:50.230681 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:25:50.235812 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:25:50.253064 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:25:50.269453 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 60 entries to flush up to offset 57135 2024.10.22 01:25:50.276609 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 46 entries to flush up to offset 3969 2024.10.22 01:25:50.284662 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:25:50.404622 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:25:50.448216 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:25:50.459866 [ 1263 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:25:50.569384 [ 1263 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:25:50.569631 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:25:50.595307 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:25:50.658653 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:50.661188 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:50.798202 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_480_480_0 (state Temporary) 2024.10.22 01:25:50.869235 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 15.71 MiB, will set to 1.68 GiB (RSS), difference: -7.25 MiB 2024.10.22 01:25:51.117398 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_480_480_0 to 202410_486_486_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:51.157393 [ 1195 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_461_481_4 to 202410_486_486_0 2024.10.22 01:25:51.167177 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57135 2024.10.22 01:25:51.181548 [ 1195 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:51.325584 [ 1195 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:51.385482 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 54664 2024.10.22 01:25:51.413607 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_461_481_4 to 202410_486_486_0 into Compact with storage Full 2024.10.22 01:25:51.489561 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:51.515730 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_481_4, total 4362 rows starting from the beginning of the part 2024.10.22 01:25:51.683995 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_482_482_0, total 156 rows starting from the beginning of the part 2024.10.22 01:25:51.722683 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_483_483_0, total 136 rows starting from the beginning of the part 2024.10.22 01:25:51.766615 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_484_484_0, total 174 rows starting from the beginning of the part 2024.10.22 01:25:51.808799 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_485_485_0, total 79 rows starting from the beginning of the part 2024.10.22 01:25:51.825399 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:51.828840 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:51.958595 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_486_486_0, total 60 rows starting from the beginning of the part 2024.10.22 01:25:52.271392 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_828_828_0 (state Temporary) 2024.10.22 01:25:52.441076 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_828_828_0 to 202410_841_841_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:52.455453 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 16.60 MiB, will set to 1.68 GiB (RSS), difference: -12.48 MiB 2024.10.22 01:25:52.588687 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_830_428 to 202410_831_836_1 2024.10.22 01:25:52.616332 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.98 MiB using storage policy from min volume index 0 2024.10.22 01:25:52.620697 [ 1247 ] {} DiskLocal: Reserved 1.98 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:52.685599 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_836_429} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_830_428 to 202410_831_836_1 into Compact with storage Full 2024.10.22 01:25:52.810325 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54664 2024.10.22 01:25:52.852292 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_836_429} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:52.877049 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_836_429} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_830_428, total 54896 rows starting from the beginning of the part 2024.10.22 01:25:52.971473 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_99_20/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:25:52.985274 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_836_429} MergeTreeSequentialSource: Reading 2 marks from part 202410_831_836_1, total 348 rows starting from the beginning of the part 2024.10.22 01:25:53.282652 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} MergeTask::MergeProjectionsStage: Merge sorted 4967 rows, containing 13 columns (13 merged, 0 gathered) in 1.916805886 sec., 2591.290039475599 rows/sec., 1.13 MiB/sec. 2024.10.22 01:25:53.473415 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} MergedBlockOutputStream: filled checksums 202410_461_486_5 (state Temporary) 2024.10.22 01:25:53.666650 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_461_486_5 to 202410_461_486_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:53.669732 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_486_5} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_461_481_4, 202410_486_486_0] -> 202410_461_486_5 2024.10.22 01:25:53.747734 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.40 MiB. 2024.10.22 01:25:53.836283 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 7587 2024.10.22 01:25:54.051346 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:54.055457 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:54.204859 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_490_490_0 (state Temporary) 2024.10.22 01:25:54.263834 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_490_490_0 to 202410_503_503_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:54.321717 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7587 2024.10.22 01:25:54.563744 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 16.61 MiB, will set to 1.69 GiB (RSS), difference: 7.12 MiB 2024.10.22 01:25:54.828763 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_738_346, 202410_739_739_0] 2024.10.22 01:25:54.832968 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_738_346, 202410_739_739_0] 2024.10.22 01:25:54.953300 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_738_346, 202410_739_739_0] 2024.10.22 01:25:54.973480 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 740401 2024.10.22 01:25:55.212179 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 414 2024.10.22 01:25:55.578319 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:55.581473 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:55.623058 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_424_202, 202410_425_425_0] 2024.10.22 01:25:55.625570 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_424_202, 202410_425_425_0] 2024.10.22 01:25:55.644584 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_424_202, 202410_425_425_0] 2024.10.22 01:25:55.657984 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_396_396_0 (state Temporary) 2024.10.22 01:25:55.762840 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:55.768567 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:55.948739 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_396_396_0 to 202410_406_406_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:55.981806 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_836_429} MergeTask::MergeProjectionsStage: Merge sorted 55244 rows, containing 13 columns (13 merged, 0 gathered) in 3.320081261 sec., 16639.351767962646 rows/sec., 2.65 MiB/sec. 2024.10.22 01:25:56.036706 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 414 2024.10.22 01:25:56.083169 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_470_470_0 (state Temporary) 2024.10.22 01:25:56.207941 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_470_470_0 to 202410_477_477_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:56.273423 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_836_429} MergedBlockOutputStream: filled checksums 202410_1_836_429 (state Temporary) 2024.10.22 01:25:56.367292 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 740401 2024.10.22 01:25:56.516530 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 15.14 MiB, will set to 1.69 GiB (RSS), difference: 10.98 MiB 2024.10.22 01:25:56.564677 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_836_429} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_836_429 to 202410_1_836_429 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:56.567969 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_836_429} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_830_428, 202410_831_836_1] -> 202410_1_836_429 2024.10.22 01:25:56.679260 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} MergeTask::MergeProjectionsStage: Merge sorted 749308 rows, containing 4 columns (4 merged, 0 gathered) in 25.515733849 sec., 29366.507913679565 rows/sec., 440.35 KiB/sec. 2024.10.22 01:25:56.711613 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:25:56.847909 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 54724 2024.10.22 01:25:57.110799 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} MergedBlockOutputStream: filled checksums 202410_1_474_95 (state Temporary) 2024.10.22 01:25:57.270916 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 5 old parts to remove. Parts: [202410_1_413_83, 202410_414_414_0, 202410_415_415_0, 202410_416_416_0, 202410_417_417_0] 2024.10.22 01:25:57.273727 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 5 parts from filesystem (serially): Parts: [202410_1_413_83, 202410_414_414_0, 202410_415_415_0, 202410_416_416_0, 202410_417_417_0] 2024.10.22 01:25:57.300820 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 5 parts from memory: Parts: [202410_1_413_83, 202410_414_414_0, 202410_415_415_0, 202410_416_416_0, 202410_417_417_0] 2024.10.22 01:25:57.307837 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:57.312449 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:57.493676 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_829_829_0 (state Temporary) 2024.10.22 01:25:57.564433 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_829_829_0 to 202410_842_842_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:57.595895 [ 1156 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_836_429 to 202410_837_837_0 2024.10.22 01:25:57.607381 [ 1156 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.98 MiB using storage policy from min volume index 0 2024.10.22 01:25:57.630270 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54724 2024.10.22 01:25:57.676698 [ 1169 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 263 ms. 2024.10.22 01:25:57.712021 [ 1156 ] {} DiskLocal: Reserved 1.98 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:57.818878 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_837_430} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_836_429 to 202410_837_837_0 into Compact with storage Full 2024.10.22 01:25:58.193837 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_837_430} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:25:58.212805 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_837_430} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_836_429, total 55244 rows starting from the beginning of the part 2024.10.22 01:25:58.297344 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_837_430} MergeTreeSequentialSource: Reading 2 marks from part 202410_837_837_0, total 95 rows starting from the beginning of the part 2024.10.22 01:25:58.685313 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 57250 2024.10.22 01:25:58.923643 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 12.11 MiB, will set to 1.68 GiB (RSS), difference: 3.74 MiB 2024.10.22 01:25:58.966539 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:25:58.970704 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:25:59.050719 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_481_481_0 (state Temporary) 2024.10.22 01:25:59.096944 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_739_347, 202410_740_740_0] 2024.10.22 01:25:59.106085 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_739_347, 202410_740_740_0] 2024.10.22 01:25:59.130876 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_739_347, 202410_740_740_0] 2024.10.22 01:25:59.133089 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_481_481_0 to 202410_487_487_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:59.184526 [ 1176 ] {} DatabaseCatalog: Execution took 273 ms. 2024.10.22 01:25:59.321266 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57250 2024.10.22 01:25:59.539117 [ 1185 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:25:59.571641 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_474_95 to 202410_1_474_95 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:25:59.575723 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_474_95} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_469_94, 202410_474_474_0] -> 202410_1_474_95 2024.10.22 01:25:59.625393 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:26:00.231537 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:26:00.234781 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:26:00.379218 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 8.63 MiB, will set to 1.68 GiB (RSS), difference: -1.10 MiB 2024.10.22 01:26:00.834927 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_99_20 to 202410_1_99_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:00.844295 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_99_20} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_94_19, 202410_99_99_0] -> 202410_1_99_20 2024.10.22 01:26:01.374551 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_2_2_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:26:01.643069 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 54763 2024.10.22 01:26:01.769491 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:01.772758 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:01.824757 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 7609 2024.10.22 01:26:01.841796 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_837_430} MergeTask::MergeProjectionsStage: Merge sorted 55339 rows, containing 13 columns (13 merged, 0 gathered) in 4.118260047 sec., 13437.471011650276 rows/sec., 2.14 MiB/sec. 2024.10.22 01:26:01.919920 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_830_830_0 (state Temporary) 2024.10.22 01:26:01.921019 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_837_430} MergedBlockOutputStream: filled checksums 202410_1_837_430 (state Temporary) 2024.10.22 01:26:02.037281 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:02.042986 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:02.056593 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 42.69 MiB. 2024.10.22 01:26:02.091552 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_491_491_0 (state Temporary) 2024.10.22 01:26:02.176806 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_830_830_0 to 202410_843_843_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:02.180141 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_491_491_0 to 202410_504_504_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:02.363925 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54763 2024.10.22 01:26:02.437310 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_740_348, 202410_741_741_0] 2024.10.22 01:26:02.451290 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_838_838_0 to 202410_843_843_0 2024.10.22 01:26:02.454854 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_740_348, 202410_741_741_0] 2024.10.22 01:26:02.456954 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7609 2024.10.22 01:26:02.462445 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:02.466524 [ 1210 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:02.466633 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_740_348, 202410_741_741_0] 2024.10.22 01:26:02.489189 [ 1210 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 280 ms. 2024.10.22 01:26:02.572678 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_837_430} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_837_430 to 202410_1_837_430 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:02.607498 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_838_838_0 to 202410_843_843_0 into Compact with storage Full 2024.10.22 01:26:02.611822 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_837_430} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_836_429, 202410_837_837_0] -> 202410_1_837_430 2024.10.22 01:26:02.655424 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:26:02.666175 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_838_838_0, total 27 rows starting from the beginning of the part 2024.10.22 01:26:02.760338 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:26:02.828633 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 4.45 MiB, will set to 1.67 GiB (RSS), difference: 6.00 MiB 2024.10.22 01:26:02.838578 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_839_839_0, total 46 rows starting from the beginning of the part 2024.10.22 01:26:02.862873 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_425_203, 202410_426_426_0] 2024.10.22 01:26:02.866894 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_425_203, 202410_426_426_0] 2024.10.22 01:26:02.877423 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_840_840_0, total 26 rows starting from the beginning of the part 2024.10.22 01:26:02.891411 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_425_203, 202410_426_426_0] 2024.10.22 01:26:02.932831 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_841_841_0, total 57 rows starting from the beginning of the part 2024.10.22 01:26:02.964876 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_842_842_0, total 60 rows starting from the beginning of the part 2024.10.22 01:26:03.000161 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_843_843_0, total 39 rows starting from the beginning of the part 2024.10.22 01:26:03.372250 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1884 entries to flush up to offset 742285 2024.10.22 01:26:03.540187 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 415 2024.10.22 01:26:04.027082 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} MergeTask::MergeProjectionsStage: Merge sorted 255 rows, containing 13 columns (13 merged, 0 gathered) in 1.513969888 sec., 168.4313552212473 rows/sec., 24.90 KiB/sec. 2024.10.22 01:26:04.084085 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} MergedBlockOutputStream: filled checksums 202410_838_843_1 (state Temporary) 2024.10.22 01:26:04.112619 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:04.118133 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:04.206705 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_397_397_0 (state Temporary) 2024.10.22 01:26:04.316037 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_838_843_1 to 202410_838_843_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:04.320939 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_838_843_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_838_838_0, 202410_843_843_0] -> 202410_838_843_1 2024.10.22 01:26:04.367179 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:26:04.412074 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:04.417106 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:04.513113 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_397_397_0 to 202410_407_407_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:04.519362 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_471_471_0 (state Temporary) 2024.10.22 01:26:04.586166 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 415 2024.10.22 01:26:04.601208 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_471_471_0 to 202410_478_478_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:04.666561 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 742285 2024.10.22 01:26:04.670110 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 3.32 MiB, will set to 1.67 GiB (RSS), difference: -1.02 MiB 2024.10.22 01:26:04.759994 [ 1232 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:26:05.575452 [ 1244 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:26:05.585440 [ 1244 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:26:05.970096 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_354_73, 202410_355_355_0, 202410_356_356_0, 202410_357_357_0, 202410_358_358_0, 202410_359_359_0] 2024.10.22 01:26:05.972056 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_354_73, 202410_355_355_0, 202410_356_356_0, 202410_357_357_0, 202410_358_358_0, 202410_359_359_0] 2024.10.22 01:26:06.205684 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_354_73, 202410_355_355_0, 202410_356_356_0, 202410_357_357_0, 202410_358_358_0, 202410_359_359_0] 2024.10.22 01:26:06.366832 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 54824 2024.10.22 01:26:06.695553 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 3.23 MiB, will set to 1.67 GiB (RSS), difference: 4.26 MiB 2024.10.22 01:26:06.784404 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:06.786083 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:06.849036 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 72 entries to flush up to offset 57322 2024.10.22 01:26:07.063232 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:07.068208 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:07.210258 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_482_482_0 (state Temporary) 2024.10.22 01:26:07.258371 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_482_482_0 to 202410_488_488_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:07.262577 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_831_831_0 (state Temporary) 2024.10.22 01:26:07.320234 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57322 2024.10.22 01:26:07.353069 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 636170 bytes for 20d9ebcd19cf85380021461fdb43be3c:0) 2024.10.22 01:26:07.456216 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b7e8463e00a0cf903e425840f62be3cc, offset: 0, size: 867 2024.10.22 01:26:07.460984 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b7e8463e00a0cf903e425840f62be3cc for removal 2024.10.22 01:26:07.464372 [ 1476 ] {} CacheMetadata: Key b7e8463e00a0cf903e425840f62be3cc is removed from metadata 2024.10.22 01:26:07.464757 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 19a2361d257e40dd6f8a3def9306f4ee, offset: 0, size: 228 2024.10.22 01:26:07.568635 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 19a2361d257e40dd6f8a3def9306f4ee for removal 2024.10.22 01:26:07.571188 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 52f5d59c0b907e01732dd4bd72a8eac4, offset: 0, size: 167 2024.10.22 01:26:07.571486 [ 1476 ] {} CacheMetadata: Key 19a2361d257e40dd6f8a3def9306f4ee is removed from metadata 2024.10.22 01:26:07.573824 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 52f5d59c0b907e01732dd4bd72a8eac4 for removal 2024.10.22 01:26:07.576880 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d6786e6264b5778631c0197241e4a1ae, offset: 0, size: 581928 2024.10.22 01:26:07.577048 [ 1476 ] {} CacheMetadata: Key 52f5d59c0b907e01732dd4bd72a8eac4 is removed from metadata 2024.10.22 01:26:07.579332 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d6786e6264b5778631c0197241e4a1ae for removal 2024.10.22 01:26:07.582378 [ 1476 ] {} CacheMetadata: Key d6786e6264b5778631c0197241e4a1ae is removed from metadata 2024.10.22 01:26:07.583284 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8af1dbe1bd33663e20d4b26cc1dbeead, offset: 0, size: 186 2024.10.22 01:26:07.587929 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8af1dbe1bd33663e20d4b26cc1dbeead for removal 2024.10.22 01:26:07.590741 [ 1476 ] {} CacheMetadata: Key 8af1dbe1bd33663e20d4b26cc1dbeead is removed from metadata 2024.10.22 01:26:07.627034 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_831_831_0 to 202410_844_844_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:07.720458 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54824 2024.10.22 01:26:07.789062 [ 1688 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vfy/qaxpxrrkahsptutwfluembgciztss, size 1684746 2024.10.22 01:26:07.892870 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_741_349, 202410_742_742_0] 2024.10.22 01:26:07.896320 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_741_349, 202410_742_742_0] 2024.10.22 01:26:07.915073 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_741_349, 202410_742_742_0] 2024.10.22 01:26:08.004024 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vhe/zswtobyhtxvftssjaxiytaecqkrud. 2024.10.22 01:26:08.101786 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:26:08.109690 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:26:08.126384 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:26:08.141911 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:26:08.165311 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:26:08.223427 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:26:08.224976 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:08.229240 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:08.251001 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:26:08.264032 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:26:08.303822 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/thv/eqzkchuvdxtsakojyomcelmwdvite (LogSeriesLimiter: on interval from 2024-10-22 01:25:49 to 2024-10-22 01:26:08 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:26:08.457776 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.97 GiB, free memory in arenas 3.08 MiB, will set to 1.67 GiB (RSS), difference: 64.76 MiB 2024.10.22 01:26:09.426432 [ 1154 ] {} DatabaseCatalog: Execution took 240 ms. 2024.10.22 01:26:09.962705 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 7625 2024.10.22 01:26:10.234640 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:26:10.238487 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:26:10.250824 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 304880 bytes for 4d65c76ffb9deeacc0eff90ae45d6b51:0) 2024.10.22 01:26:10.253690 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ab1fae0b4fb5b654440ac8de3eeef185, offset: 0, size: 33180 2024.10.22 01:26:10.259336 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ab1fae0b4fb5b654440ac8de3eeef185 for removal 2024.10.22 01:26:10.261795 [ 1476 ] {} CacheMetadata: Key ab1fae0b4fb5b654440ac8de3eeef185 is removed from metadata 2024.10.22 01:26:10.270880 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:10.272798 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:10.294850 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_492_492_0 (state Temporary) 2024.10.22 01:26:10.362989 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6a0c60a5e20510783f500a8ee750f802, offset: 0, size: 43267 2024.10.22 01:26:10.366784 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6a0c60a5e20510783f500a8ee750f802 for removal 2024.10.22 01:26:10.368776 [ 1476 ] {} CacheMetadata: Key 6a0c60a5e20510783f500a8ee750f802 is removed from metadata 2024.10.22 01:26:10.398936 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_492_492_0 to 202410_505_505_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:10.465333 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.97 GiB, free memory in arenas 3.38 MiB, will set to 1.66 GiB (RSS), difference: 72.63 MiB 2024.10.22 01:26:10.564252 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7625 2024.10.22 01:26:10.987458 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:26:10.999045 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:26:11.009705 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:26:11.051639 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:26:11.061416 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:26:11.073254 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_426_204, 202410_427_427_0] 2024.10.22 01:26:11.075177 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_426_204, 202410_427_427_0] 2024.10.22 01:26:11.078318 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:26:11.091329 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_426_204, 202410_427_427_0] 2024.10.22 01:26:11.106123 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:26:11.135853 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:26:11.157220 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:26:11.671923 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1581 entries to flush up to offset 743866 2024.10.22 01:26:11.730849 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 54900 2024.10.22 01:26:11.896644 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:11.900646 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:11.954348 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_832_832_0 (state Temporary) 2024.10.22 01:26:12.065585 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_99_99_0 (state Temporary) 2024.10.22 01:26:12.090138 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 416 2024.10.22 01:26:12.133722 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 1048576 bytes for 3610caca34cd55b683faa9e6bbce9dec:0) 2024.10.22 01:26:12.136808 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d22efe560d38f58638ab9b37b86aee88, offset: 0, size: 4495090 2024.10.22 01:26:12.146579 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d22efe560d38f58638ab9b37b86aee88 for removal 2024.10.22 01:26:12.150642 [ 1476 ] {} CacheMetadata: Key d22efe560d38f58638ab9b37b86aee88 is removed from metadata 2024.10.22 01:26:12.151252 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7bf9e2cb5252c8b4f781cbb50acc7995, offset: 0, size: 954 2024.10.22 01:26:12.154594 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7bf9e2cb5252c8b4f781cbb50acc7995 for removal 2024.10.22 01:26:12.159256 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2399ce8d0e7a5a008eb121db9640c950, offset: 0, size: 441160 2024.10.22 01:26:12.162403 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2399ce8d0e7a5a008eb121db9640c950 for removal 2024.10.22 01:26:12.263910 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_742_350, 202410_743_743_0] 2024.10.22 01:26:12.265001 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_832_832_0 to 202410_845_845_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:12.276669 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_742_350, 202410_743_743_0] 2024.10.22 01:26:12.293664 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_742_350, 202410_743_743_0] 2024.10.22 01:26:12.354762 [ 1476 ] {} CacheMetadata: Key 2399ce8d0e7a5a008eb121db9640c950 is removed from metadata 2024.10.22 01:26:12.363659 [ 1476 ] {} CacheMetadata: Key 7bf9e2cb5252c8b4f781cbb50acc7995 is removed from metadata 2024.10.22 01:26:12.364792 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:12.368085 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:26:12.423857 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 54900 2024.10.22 01:26:12.477931 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_472_472_0 (state Temporary) 2024.10.22 01:26:12.497352 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:12.501138 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:12.651750 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.97 GiB, free memory in arenas 6.43 MiB, will set to 1.66 GiB (RSS), difference: 33.60 MiB 2024.10.22 01:26:12.654139 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_472_472_0 to 202410_479_479_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:12.671740 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_398_398_0 (state Temporary) 2024.10.22 01:26:12.676808 [ 1151 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 5 parts from 202410_1_474_95 to 202410_478_478_0 2024.10.22 01:26:12.685911 [ 1151 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:12.703896 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:12.784702 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 743866 2024.10.22 01:26:12.813196 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_478_96} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_474_95 to 202410_478_478_0 into Wide with storage Full 2024.10.22 01:26:12.851751 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_478_96} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:26:12.859534 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_478_96} MergeTreeSequentialSource: Reading 93 marks from part 202410_1_474_95, total 749308 rows starting from the beginning of the part 2024.10.22 01:26:12.877354 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_478_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_475_475_0, total 1603 rows starting from the beginning of the part 2024.10.22 01:26:12.903034 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_478_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_476_476_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:26:12.936726 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_478_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_477_477_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:26:12.968031 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_478_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_478_478_0, total 1884 rows starting from the beginning of the part 2024.10.22 01:26:13.056336 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_398_398_0 to 202410_408_408_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:13.080544 [ 1247 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_403_83 to 202410_408_408_0 2024.10.22 01:26:13.118239 [ 1247 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:13.123347 [ 1247 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:13.139639 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 416 2024.10.22 01:26:13.288117 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_403_83 to 202410_408_408_0 into Compact with storage Full 2024.10.22 01:26:13.371184 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_441_88, 202410_442_442_0, 202410_443_443_0, 202410_444_444_0, 202410_445_445_0, 202410_446_446_0] 2024.10.22 01:26:13.373434 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_441_88, 202410_442_442_0, 202410_443_443_0, 202410_444_444_0, 202410_445_445_0, 202410_446_446_0] 2024.10.22 01:26:13.402441 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_441_88, 202410_442_442_0, 202410_443_443_0, 202410_444_444_0, 202410_445_445_0, 202410_446_446_0] 2024.10.22 01:26:13.669500 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:26:13.805711 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_403_83, total 447 rows starting from the beginning of the part 2024.10.22 01:26:14.023919 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_404_404_0, total 1 rows starting from the beginning of the part 2024.10.22 01:26:14.097325 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_405_405_0, total 1 rows starting from the beginning of the part 2024.10.22 01:26:14.176763 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_478_96/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:26:14.206400 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_406_406_0, total 1 rows starting from the beginning of the part 2024.10.22 01:26:14.298048 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_99_99_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:26:14.459374 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 439560 bytes for dd12706bf2f54188549572bd589dc627:0) 2024.10.22 01:26:14.461405 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2f7a42b0023312c9083db016e740cfc7, offset: 0, size: 911350 2024.10.22 01:26:14.464909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2f7a42b0023312c9083db016e740cfc7 for removal 2024.10.22 01:26:14.528036 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_407_407_0, total 1 rows starting from the beginning of the part 2024.10.22 01:26:14.568927 [ 1476 ] {} CacheMetadata: Key 2f7a42b0023312c9083db016e740cfc7 is removed from metadata 2024.10.22 01:26:14.571864 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f8ad94cc8be0018294283e3109c97928, offset: 0, size: 530 2024.10.22 01:26:14.574758 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f8ad94cc8be0018294283e3109c97928 for removal 2024.10.22 01:26:14.588541 [ 1476 ] {} CacheMetadata: Key f8ad94cc8be0018294283e3109c97928 is removed from metadata 2024.10.22 01:26:14.629339 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 5.06 MiB, will set to 1.66 GiB (RSS), difference: -4.60 MiB 2024.10.22 01:26:14.649578 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_408_408_0, total 1 rows starting from the beginning of the part 2024.10.22 01:26:14.885274 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 57393 2024.10.22 01:26:15.321731 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:15.329384 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:26:15.382579 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_483_483_0 (state Temporary) 2024.10.22 01:26:15.650709 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_483_483_0 to 202410_489_489_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:15.678770 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57393 2024.10.22 01:26:15.727506 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 14 file segments (while reserving 648126 bytes for 8cd79d7dcf57006f184f54f0e03884b8:0) 2024.10.22 01:26:15.731023 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 61a9122515b45b414944f5441b3a27e9, offset: 0, size: 895 2024.10.22 01:26:15.736936 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 61a9122515b45b414944f5441b3a27e9 for removal 2024.10.22 01:26:15.744484 [ 1476 ] {} CacheMetadata: Key 61a9122515b45b414944f5441b3a27e9 is removed from metadata 2024.10.22 01:26:15.741800 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a55d1fd35893ea10476be244c2ea8973, offset: 0, size: 3792643 2024.10.22 01:26:15.950562 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a55d1fd35893ea10476be244c2ea8973 for removal 2024.10.22 01:26:15.953109 [ 1476 ] {} CacheMetadata: Key a55d1fd35893ea10476be244c2ea8973 is removed from metadata 2024.10.22 01:26:15.954998 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 230ac59f58640efbefce7497f7730fc2, offset: 0, size: 118542 2024.10.22 01:26:15.958727 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 230ac59f58640efbefce7497f7730fc2 for removal 2024.10.22 01:26:15.961264 [ 1476 ] {} CacheMetadata: Key 230ac59f58640efbefce7497f7730fc2 is removed from metadata 2024.10.22 01:26:15.962390 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d6a6c73410298ed2f4df651407674dfc, offset: 0, size: 7 2024.10.22 01:26:15.965085 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d6a6c73410298ed2f4df651407674dfc for removal 2024.10.22 01:26:15.967254 [ 1476 ] {} CacheMetadata: Key d6a6c73410298ed2f4df651407674dfc is removed from metadata 2024.10.22 01:26:15.967482 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 12b0e58ad3afdb5880b4738c9b08779f, offset: 0, size: 1 2024.10.22 01:26:15.970075 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 12b0e58ad3afdb5880b4738c9b08779f for removal 2024.10.22 01:26:15.973346 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ad1115b9de68eb7bed5c879f340ae843, offset: 0, size: 851 2024.10.22 01:26:15.973344 [ 1476 ] {} CacheMetadata: Key 12b0e58ad3afdb5880b4738c9b08779f is removed from metadata 2024.10.22 01:26:15.977761 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ad1115b9de68eb7bed5c879f340ae843 for removal 2024.10.22 01:26:15.982077 [ 1476 ] {} CacheMetadata: Key ad1115b9de68eb7bed5c879f340ae843 is removed from metadata 2024.10.22 01:26:15.982998 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1d8aba27700aa8eaabddbe3c7dfe954d, offset: 0, size: 155382 2024.10.22 01:26:15.987462 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1d8aba27700aa8eaabddbe3c7dfe954d for removal 2024.10.22 01:26:15.995756 [ 1476 ] {} CacheMetadata: Key 1d8aba27700aa8eaabddbe3c7dfe954d is removed from metadata 2024.10.22 01:26:15.991658 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1d9ad5da324ddc6d7fd718b0291e825d, offset: 0, size: 3034 2024.10.22 01:26:16.124803 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1d9ad5da324ddc6d7fd718b0291e825d for removal 2024.10.22 01:26:16.129756 [ 1476 ] {} CacheMetadata: Key 1d9ad5da324ddc6d7fd718b0291e825d is removed from metadata 2024.10.22 01:26:16.132181 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9c298b9c4f1664a1001f29fa1a05ae4c, offset: 0, size: 16033 2024.10.22 01:26:16.137034 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9c298b9c4f1664a1001f29fa1a05ae4c for removal 2024.10.22 01:26:16.139951 [ 1476 ] {} CacheMetadata: Key 9c298b9c4f1664a1001f29fa1a05ae4c is removed from metadata 2024.10.22 01:26:16.142377 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 930543b88420e3bea1a90f4494c8c333, offset: 0, size: 10 2024.10.22 01:26:16.248755 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 930543b88420e3bea1a90f4494c8c333 for removal 2024.10.22 01:26:16.251925 [ 1476 ] {} CacheMetadata: Key 930543b88420e3bea1a90f4494c8c333 is removed from metadata 2024.10.22 01:26:16.251715 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d45c627baf69b6bf49ab436a7e0afbdf, offset: 0, size: 650 2024.10.22 01:26:16.256382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d45c627baf69b6bf49ab436a7e0afbdf for removal 2024.10.22 01:26:16.261116 [ 1476 ] {} CacheMetadata: Key d45c627baf69b6bf49ab436a7e0afbdf is removed from metadata 2024.10.22 01:26:16.261622 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 06de704384713446cce1a8b0b5cac927, offset: 0, size: 2367 2024.10.22 01:26:16.272199 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 06de704384713446cce1a8b0b5cac927 for removal 2024.10.22 01:26:16.283438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 22484af6e01cfb0842c76a9b6c3484ff, offset: 0, size: 9721 2024.10.22 01:26:16.287928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 22484af6e01cfb0842c76a9b6c3484ff for removal 2024.10.22 01:26:16.298368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 045558310934db33b40b20104cc727c3, offset: 0, size: 4 2024.10.22 01:26:16.304416 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 045558310934db33b40b20104cc727c3 for removal 2024.10.22 01:26:16.378879 [ 1476 ] {} CacheMetadata: Key 06de704384713446cce1a8b0b5cac927 is removed from metadata 2024.10.22 01:26:16.384675 [ 1476 ] {} CacheMetadata: Key 045558310934db33b40b20104cc727c3 is removed from metadata 2024.10.22 01:26:16.392703 [ 1476 ] {} CacheMetadata: Key 22484af6e01cfb0842c76a9b6c3484ff is removed from metadata 2024.10.22 01:26:16.539589 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 115 entries to flush up to offset 55015 2024.10.22 01:26:16.848471 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} MergeTask::MergeProjectionsStage: Merge sorted 452 rows, containing 32 columns (32 merged, 0 gathered) in 3.572556587 sec., 126.52003935914145 rows/sec., 11.79 KiB/sec. 2024.10.22 01:26:16.864977 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:16.869770 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:26:16.898256 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} MergedBlockOutputStream: filled checksums 202410_1_408_84 (state Temporary) 2024.10.22 01:26:17.037470 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_833_833_0 (state Temporary) 2024.10.22 01:26:17.404739 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_408_84 to 202410_1_408_84 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:17.413335 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_408_84} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_403_83, 202410_408_408_0] -> 202410_1_408_84 2024.10.22 01:26:17.456161 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_743_351, 202410_744_744_0] 2024.10.22 01:26:17.457432 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_833_833_0 to 202410_846_846_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:17.459046 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_743_351, 202410_744_744_0] 2024.10.22 01:26:17.573435 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_743_351, 202410_744_744_0] 2024.10.22 01:26:17.590181 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55015 2024.10.22 01:26:17.662295 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:26:17.806808 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 11.26 MiB, will set to 1.66 GiB (RSS), difference: 6.86 MiB 2024.10.22 01:26:18.067461 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 7638 2024.10.22 01:26:18.203412 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:18.208106 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:26:18.462816 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_493_493_0 (state Temporary) 2024.10.22 01:26:18.751623 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_493_493_0 to 202410_506_506_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:18.773759 [ 1276 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_501_100 to 202410_506_506_0 2024.10.22 01:26:18.784484 [ 1276 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:18.792468 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:26:18.907655 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7638 2024.10.22 01:26:18.940258 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_501_100 to 202410_506_506_0 into Compact with storage Full 2024.10.22 01:26:18.973757 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:26:18.982114 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_501_100, total 7667 rows starting from the beginning of the part 2024.10.22 01:26:19.006793 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_502_502_0, total 13 rows starting from the beginning of the part 2024.10.22 01:26:19.026690 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_503_503_0, total 13 rows starting from the beginning of the part 2024.10.22 01:26:19.047315 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_504_504_0, total 22 rows starting from the beginning of the part 2024.10.22 01:26:19.061050 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_505_505_0, total 16 rows starting from the beginning of the part 2024.10.22 01:26:19.073299 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_506_506_0, total 13 rows starting from the beginning of the part 2024.10.22 01:26:19.672169 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 11.68 MiB, will set to 1.66 GiB (RSS), difference: -4.57 MiB 2024.10.22 01:26:19.704716 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} MergeTask::MergeProjectionsStage: Merge sorted 7744 rows, containing 12 columns (12 merged, 0 gathered) in 0.775280159 sec., 9988.64721365841 rows/sec., 1.55 MiB/sec. 2024.10.22 01:26:19.756698 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} MergedBlockOutputStream: filled checksums 202410_1_506_101 (state Temporary) 2024.10.22 01:26:19.789417 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 745406 2024.10.22 01:26:20.072794 [ 1187 ] {} DatabaseCatalog: Execution took 539 ms. 2024.10.22 01:26:20.241461 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:26:20.246978 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:26:20.466459 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_427_205, 202410_428_428_0] 2024.10.22 01:26:20.469415 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_427_205, 202410_428_428_0] 2024.10.22 01:26:20.482865 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_427_205, 202410_428_428_0] 2024.10.22 01:26:20.512804 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:20.518197 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:26:20.570484 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_506_101 to 202410_1_506_101 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:20.593156 [ 1201 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:26:20.644583 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 417 2024.10.22 01:26:20.653576 [ 1201 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:26:20.660062 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_506_101} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_501_100, 202410_506_506_0] -> 202410_1_506_101 2024.10.22 01:26:20.676080 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_473_473_0 (state Temporary) 2024.10.22 01:26:20.736105 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.48 MiB. 2024.10.22 01:26:20.749031 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_473_473_0 to 202410_480_480_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:20.777538 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 745406 2024.10.22 01:26:21.033840 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_99_99_0 to 202410_102_102_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:21.062805 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:21.064893 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:26:21.131773 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_399_399_0 (state Temporary) 2024.10.22 01:26:21.291971 [ 1709 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/thv/eqzkchuvdxtsakojyomcelmwdvite, size 41938 2024.10.22 01:26:21.479872 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 9.93 MiB, will set to 1.66 GiB (RSS), difference: 4.67 MiB 2024.10.22 01:26:21.532711 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_399_399_0 to 202410_409_409_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:21.547362 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column OS 2024.10.22 01:26:21.559863 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column OS 2024.10.22 01:26:21.572050 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column OS 2024.10.22 01:26:21.607238 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column OS 2024.10.22 01:26:21.624414 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 55078 2024.10.22 01:26:21.627477 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 417 2024.10.22 01:26:21.660996 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column OS 2024.10.22 01:26:21.694929 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column OS 2024.10.22 01:26:21.848734 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column OS 2024.10.22 01:26:21.966629 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.093107146 sec. 2024.10.22 01:26:21.971912 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column OS 2024.10.22 01:26:21.984409 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:21.987311 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:26:22.065819 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_834_834_0 (state Temporary) 2024.10.22 01:26:22.078385 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/iwu/tdlhrfljubpzccqxdmrlclztopijj (LogSeriesLimiter: on interval from 2024-10-22 01:26:08 to 2024-10-22 01:26:22 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:26:22.229183 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_834_834_0 to 202410_847_847_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:22.329468 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55078 2024.10.22 01:26:23.201799 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 75 entries to flush up to offset 57468 2024.10.22 01:26:23.454604 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:23.536180 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 504571 bytes for d594cb1c06c661156724c94eb767dabd:0) 2024.10.22 01:26:23.539183 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 35e30567b58015c19b9bd0d225f34344, offset: 0, size: 3175083 2024.10.22 01:26:23.553484 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 35e30567b58015c19b9bd0d225f34344 for removal 2024.10.22 01:26:23.557575 [ 1476 ] {} CacheMetadata: Key 35e30567b58015c19b9bd0d225f34344 is removed from metadata 2024.10.22 01:26:23.558690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8ac6f4b8de395ac494924f6307a82a60, offset: 0, size: 270 2024.10.22 01:26:23.558662 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:26:23.562487 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8ac6f4b8de395ac494924f6307a82a60 for removal 2024.10.22 01:26:23.587317 [ 1476 ] {} CacheMetadata: Key 8ac6f4b8de395ac494924f6307a82a60 is removed from metadata 2024.10.22 01:26:23.628948 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_744_352, 202410_745_745_0] 2024.10.22 01:26:23.632098 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_744_352, 202410_745_745_0] 2024.10.22 01:26:23.643677 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_744_352, 202410_745_745_0] 2024.10.22 01:26:23.679227 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 10.31 MiB, will set to 1.66 GiB (RSS), difference: 4.18 MiB 2024.10.22 01:26:23.815257 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_484_484_0 (state Temporary) 2024.10.22 01:26:23.970448 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_484_484_0 to 202410_490_490_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:24.008512 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57468 2024.10.22 01:26:24.342726 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 3969 2024.10.22 01:26:24.530086 [ 1232 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 247 ms. 2024.10.22 01:26:25.811598 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_478_96/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:26:25.960688 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_419_419_0, 202410_420_420_0, 202410_421_421_0, 202410_422_422_0, 202410_423_423_0, 202410_424_424_0] 2024.10.22 01:26:25.964252 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_419_419_0, 202410_420_420_0, 202410_421_421_0, 202410_422_422_0, 202410_423_423_0, 202410_424_424_0] 2024.10.22 01:26:26.007501 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_419_419_0, 202410_420_420_0, 202410_421_421_0, 202410_422_422_0, 202410_423_423_0, 202410_424_424_0] 2024.10.22 01:26:26.409394 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 7651 2024.10.22 01:26:26.498531 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 37 entries to flush up to offset 55115 2024.10.22 01:26:26.608535 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:26.623860 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:26:26.651219 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 1048576 bytes for a8d37e465bb7c2efc70d6593c8a8f4e3:0) 2024.10.22 01:26:26.653668 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 205ad1f9404ca48beb078c3585fdafb0, offset: 0, size: 33554432 2024.10.22 01:26:26.677427 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4a822ef1349ef63a32c539da265d1882, offset: 0, size: 52888 2024.10.22 01:26:26.684399 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4a822ef1349ef63a32c539da265d1882 for removal 2024.10.22 01:26:26.691043 [ 1476 ] {} CacheMetadata: Key 4a822ef1349ef63a32c539da265d1882 is removed from metadata 2024.10.22 01:26:26.692823 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 363ba83271336bc96f90af0b62225ce6, offset: 0, size: 151 2024.10.22 01:26:26.697317 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 363ba83271336bc96f90af0b62225ce6 for removal 2024.10.22 01:26:26.756719 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_494_494_0 (state Temporary) 2024.10.22 01:26:26.801585 [ 1476 ] {} CacheMetadata: Key 363ba83271336bc96f90af0b62225ce6 is removed from metadata 2024.10.22 01:26:26.851833 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_494_494_0 to 202410_507_507_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:26.917112 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:26.920630 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:26.975095 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_835_835_0 (state Temporary) 2024.10.22 01:26:27.007108 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7651 2024.10.22 01:26:27.134941 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_835_835_0 to 202410_848_848_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:27.183642 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55115 2024.10.22 01:26:27.220182 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_837_430 to 202410_838_843_1 2024.10.22 01:26:27.228773 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.99 MiB using storage policy from min volume index 0 2024.10.22 01:26:27.232545 [ 1158 ] {} DiskLocal: Reserved 1.99 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:27.287384 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_843_431} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_837_430 to 202410_838_843_1 into Compact with storage Full 2024.10.22 01:26:27.451102 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 16.09 MiB, will set to 1.67 GiB (RSS), difference: 5.18 MiB 2024.10.22 01:26:27.490137 [ 1237 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 01:26:27.780565 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 746946 2024.10.22 01:26:27.859036 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_843_431} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:26:27.906222 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_843_431} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_837_430, total 55339 rows starting from the beginning of the part 2024.10.22 01:26:28.097709 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_843_431} MergeTreeSequentialSource: Reading 2 marks from part 202410_838_843_1, total 255 rows starting from the beginning of the part 2024.10.22 01:26:28.705724 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:28.717152 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:28.966794 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_474_474_0 (state Temporary) 2024.10.22 01:26:29.132723 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_474_474_0 to 202410_481_481_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:29.173630 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_745_353, 202410_746_746_0] 2024.10.22 01:26:29.175425 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_745_353, 202410_746_746_0] 2024.10.22 01:26:29.266758 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 746946 2024.10.22 01:26:29.278631 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:26:29.292463 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:26:29.189544 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_745_353, 202410_746_746_0] 2024.10.22 01:26:29.324474 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:26:29.364869 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:26:29.407807 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:26:29.522911 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:26:29.561349 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:26:29.734592 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:26:29.760160 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:26:29.870636 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 12.51 MiB, will set to 1.67 GiB (RSS), difference: -9.51 MiB 2024.10.22 01:26:29.922757 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bxa/otkxhtneavuakbhuyjkhelceulttf (LogSeriesLimiter: on interval from 2024-10-22 01:26:22 to 2024-10-22 01:26:29 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:26:30.251862 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:26:30.255687 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:26:31.010506 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_428_206, 202410_429_429_0] 2024.10.22 01:26:31.012489 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_428_206, 202410_429_429_0] 2024.10.22 01:26:31.130600 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_428_206, 202410_429_429_0] 2024.10.22 01:26:31.172616 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_843_431/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:26:31.188783 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 55170 2024.10.22 01:26:31.438810 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:31.446379 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:31.544025 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_836_836_0 (state Temporary) 2024.10.22 01:26:31.624822 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 57558 2024.10.22 01:26:31.727295 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_836_836_0 to 202410_849_849_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:31.858942 [ 1251 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_844_844_0 to 202410_849_849_0 2024.10.22 01:26:31.874054 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 4010 2024.10.22 01:26:31.882922 [ 1251 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:31.884955 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55170 2024.10.22 01:26:31.884958 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:31.936093 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_844_844_0 to 202410_849_849_0 into Compact with storage Full 2024.10.22 01:26:31.940636 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:31.957166 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:32.012693 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:26:32.035872 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_485_485_0 (state Temporary) 2024.10.22 01:26:32.140237 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_844_844_0, total 61 rows starting from the beginning of the part 2024.10.22 01:26:32.045782 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_843_431} MergeTask::MergeProjectionsStage: Merge sorted 55594 rows, containing 13 columns (13 merged, 0 gathered) in 4.762999099 sec., 11672.057635213927 rows/sec., 1.86 MiB/sec. 2024.10.22 01:26:32.172843 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_485_485_0 to 202410_491_491_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:32.249485 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_417_84, 202410_418_418_0] 2024.10.22 01:26:32.253292 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_417_84, 202410_418_418_0] 2024.10.22 01:26:32.275252 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_417_84, 202410_418_418_0] 2024.10.22 01:26:32.323178 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_461_486_5 to 202410_491_491_0 2024.10.22 01:26:32.339446 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:32.347599 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:32.373441 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_461_486_5 to 202410_491_491_0 into Compact with storage Full 2024.10.22 01:26:32.410437 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_845_845_0, total 76 rows starting from the beginning of the part 2024.10.22 01:26:32.428121 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_843_431} MergedBlockOutputStream: filled checksums 202410_1_843_431 (state Temporary) 2024.10.22 01:26:32.559913 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57558 2024.10.22 01:26:32.566505 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:26:32.573930 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_846_846_0, total 115 rows starting from the beginning of the part 2024.10.22 01:26:32.676119 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_486_5, total 4967 rows starting from the beginning of the part 2024.10.22 01:26:32.726148 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_487_487_0, total 115 rows starting from the beginning of the part 2024.10.22 01:26:32.769349 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_843_431} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_843_431 to 202410_1_843_431 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:32.771811 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_843_431} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_837_430, 202410_838_843_1] -> 202410_1_843_431 2024.10.22 01:26:32.776154 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_488_488_0, total 72 rows starting from the beginning of the part 2024.10.22 01:26:32.810258 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:26:32.832976 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_489_489_0, total 71 rows starting from the beginning of the part 2024.10.22 01:26:32.880956 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_490_490_0, total 75 rows starting from the beginning of the part 2024.10.22 01:26:32.921645 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_847_847_0, total 63 rows starting from the beginning of the part 2024.10.22 01:26:33.030705 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_491_491_0, total 90 rows starting from the beginning of the part 2024.10.22 01:26:33.162665 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_848_848_0, total 37 rows starting from the beginning of the part 2024.10.22 01:26:33.260070 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_746_354, 202410_747_747_0] 2024.10.22 01:26:33.262989 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_746_354, 202410_747_747_0] 2024.10.22 01:26:33.277081 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_746_354, 202410_747_747_0] 2024.10.22 01:26:33.328701 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_849_849_0, total 55 rows starting from the beginning of the part 2024.10.22 01:26:34.511689 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 7659 2024.10.22 01:26:34.723965 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} MergeTask::MergeProjectionsStage: Merge sorted 5390 rows, containing 13 columns (13 merged, 0 gathered) in 2.355699637 sec., 2288.0675937379706 rows/sec., 1021.08 KiB/sec. 2024.10.22 01:26:34.944183 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:34.950678 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:35.010901 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} MergedBlockOutputStream: filled checksums 202410_461_491_6 (state Temporary) 2024.10.22 01:26:35.101285 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_495_495_0 (state Temporary) 2024.10.22 01:26:35.159948 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_495_495_0 to 202410_508_508_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:35.278207 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 13.52 MiB, will set to 1.67 GiB (RSS), difference: -4.51 MiB 2024.10.22 01:26:35.292230 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7659 2024.10.22 01:26:35.392887 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_461_491_6 to 202410_461_491_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:35.409504 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} MergeTask::MergeProjectionsStage: Merge sorted 407 rows, containing 13 columns (13 merged, 0 gathered) in 3.485662261 sec., 116.76403779958761 rows/sec., 17.91 KiB/sec. 2024.10.22 01:26:35.464142 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} MergedBlockOutputStream: filled checksums 202410_844_849_1 (state Temporary) 2024.10.22 01:26:35.506661 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_491_6} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_461_486_5, 202410_491_491_0] -> 202410_461_491_6 2024.10.22 01:26:35.552170 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.56 MiB. 2024.10.22 01:26:35.669819 [ 1170 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:26:35.715026 [ 1170 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:26:35.815131 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_844_849_1 to 202410_844_849_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:35.825438 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_844_849_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_844_844_0, 202410_849_849_0] -> 202410_844_849_1 2024.10.22 01:26:35.895041 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 55234 2024.10.22 01:26:35.979947 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:26:36.205782 [ 1725 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/iwu/tdlhrfljubpzccqxdmrlclztopijj, size 673748 2024.10.22 01:26:36.242806 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:36.248058 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:36.266406 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_478_96} MergeTask::MergeProjectionsStage: Merge sorted 755875 rows, containing 4 columns (4 merged, 0 gathered) in 23.529850105 sec., 32124.089045487777 rows/sec., 481.88 KiB/sec. 2024.10.22 01:26:36.271185 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 748871 2024.10.22 01:26:36.321491 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_837_837_0 (state Temporary) 2024.10.22 01:26:36.432219 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 12.18 MiB, will set to 1.66 GiB (RSS), difference: 4.58 MiB 2024.10.22 01:26:36.453825 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_478_96} MergedBlockOutputStream: filled checksums 202410_1_478_96 (state Temporary) 2024.10.22 01:26:36.544391 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:26:36.555879 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:26:36.563193 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_837_837_0 to 202410_850_850_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:36.563823 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:26:36.571832 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:26:36.607362 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:26:36.633340 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 418 2024.10.22 01:26:36.655351 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:26:36.657618 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55234 2024.10.22 01:26:36.699061 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:26:36.734077 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:26:36.794884 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/cmm/wwzwchshbpxxslxqvxvyafkxetnlm (LogSeriesLimiter: on interval from 2024-10-22 01:26:29 to 2024-10-22 01:26:36 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:26:37.032618 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:37.039765 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:37.196078 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_400_400_0 (state Temporary) 2024.10.22 01:26:37.234978 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:37.245397 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:37.401509 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_400_400_0 to 202410_410_410_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:37.626236 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_475_475_0 (state Temporary) 2024.10.22 01:26:37.639114 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 418 2024.10.22 01:26:37.812807 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_475_475_0 to 202410_482_482_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:37.862476 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 748871 2024.10.22 01:26:38.285434 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yiu/oslbfhwlrpbcttcnpcfygmfpsxnvz. 2024.10.22 01:26:38.329125 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/sau/fufhqmymowhfjavopbxfcohiaejox. 2024.10.22 01:26:38.331596 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_429_207, 202410_430_430_0] 2024.10.22 01:26:38.333715 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_429_207, 202410_430_430_0] 2024.10.22 01:26:38.357232 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_429_207, 202410_430_430_0] 2024.10.22 01:26:38.461825 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 8.35 MiB, will set to 1.66 GiB (RSS), difference: 5.62 MiB 2024.10.22 01:26:38.474240 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_478_96} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_478_96 to 202410_1_478_96 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:38.482764 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_478_96} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 5 parts: [202410_1_474_95, 202410_478_478_0] -> 202410_1_478_96 2024.10.22 01:26:38.535014 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:26:39.082837 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/sau/fufhqmymowhfjavopbxfcohiaejox. 2024.10.22 01:26:39.087787 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_1_1_0 to 201403_1_1_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:39.111004 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yiu/oslbfhwlrpbcttcnpcfygmfpsxnvz. 2024.10.22 01:26:39.129565 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_2_2_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:26:39.174481 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_747_355, 202410_748_748_0] 2024.10.22 01:26:39.176898 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_747_355, 202410_748_748_0] 2024.10.22 01:26:39.290609 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_747_355, 202410_748_748_0] 2024.10.22 01:26:39.710591 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 26 2024.10.22 01:26:40.073501 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 78 entries to flush up to offset 57636 2024.10.22 01:26:40.246864 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:26:40.253118 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:40.253466 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:26:40.256540 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:40.307018 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_26_26_0 (state Temporary) 2024.10.22 01:26:40.368239 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_26_26_0 to 202410_26_26_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:40.385819 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:26:40.394987 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:40.401816 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:40.437114 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 26 2024.10.22 01:26:40.568203 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 857.14 MiB, peak 1.97 GiB, free memory in arenas 6.12 MiB, will set to 1.38 GiB (RSS), difference: 558.18 MiB 2024.10.22 01:26:40.670487 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 55295 2024.10.22 01:26:40.679942 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_486_486_0 (state Temporary) 2024.10.22 01:26:40.858927 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_486_486_0 to 202410_492_492_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:40.896843 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57636 2024.10.22 01:26:40.952934 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:41.056060 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:41.097741 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_838_838_0 (state Temporary) 2024.10.22 01:26:41.152071 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_838_838_0 to 202410_851_851_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:41.211929 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55295 2024.10.22 01:26:41.379319 [ 1209 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 216 ms. 2024.10.22 01:26:41.481575 [ 1212 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 309 ms. 2024.10.22 01:26:42.407815 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 1.97 GiB, free memory in arenas 5.14 MiB, will set to 1.19 GiB (RSS), difference: -254.95 MiB 2024.10.22 01:26:42.685459 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_748_356, 202410_749_749_0] 2024.10.22 01:26:42.687421 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_748_356, 202410_749_749_0] 2024.10.22 01:26:42.699383 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_748_356, 202410_749_749_0] 2024.10.22 01:26:42.794666 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 7687 2024.10.22 01:26:42.956298 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:42.959198 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:42.984440 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_496_496_0 (state Temporary) 2024.10.22 01:26:43.049371 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_496_496_0 to 202410_509_509_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:43.198376 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7687 2024.10.22 01:26:44.359048 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.24 GiB, peak 1.97 GiB, free memory in arenas 5.43 MiB, will set to 1.23 GiB (RSS), difference: -18.38 MiB 2024.10.22 01:26:44.867556 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1744 entries to flush up to offset 750615 2024.10.22 01:26:45.144214 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 419 2024.10.22 01:26:45.234061 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 24 entries to flush up to offset 55319 2024.10.22 01:26:45.605491 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:45.613167 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:45.663861 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_839_839_0 (state Temporary) 2024.10.22 01:26:45.677507 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:45.680656 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:45.702539 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_839_839_0 to 202410_852_852_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:45.759553 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_401_401_0 (state Temporary) 2024.10.22 01:26:45.861495 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55319 2024.10.22 01:26:45.865824 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_401_401_0 to 202410_411_411_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:45.885216 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:45.891321 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:45.979218 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 419 2024.10.22 01:26:46.128263 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_476_476_0 (state Temporary) 2024.10.22 01:26:46.185366 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_476_476_0 to 202410_483_483_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:46.328798 [ 1224 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_1_478_96 to 202410_483_483_0 2024.10.22 01:26:46.332874 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 750615 2024.10.22 01:26:46.349497 [ 1224 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:46.352958 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:46.386824 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_478_96 to 202410_483_483_0 into Wide with storage Full 2024.10.22 01:26:46.440141 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 1.97 GiB, free memory in arenas 5.95 MiB, will set to 1.28 GiB (RSS), difference: -41.45 MiB 2024.10.22 01:26:46.446954 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:26:46.629202 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} MergeTreeSequentialSource: Reading 94 marks from part 202410_1_478_96, total 755875 rows starting from the beginning of the part 2024.10.22 01:26:46.781826 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_479_479_0, total 1581 rows starting from the beginning of the part 2024.10.22 01:26:47.129434 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_480_480_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:26:47.163486 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_481_481_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:26:47.296971 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_482_482_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:26:47.329676 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_483_483_0, total 1744 rows starting from the beginning of the part 2024.10.22 01:26:47.388546 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_749_357, 202410_750_750_0] 2024.10.22 01:26:47.393189 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_749_357, 202410_750_750_0] 2024.10.22 01:26:47.416536 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_749_357, 202410_750_750_0] 2024.10.22 01:26:47.585047 [ 1675 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bxa/otkxhtneavuakbhuyjkhelceulttf, size 772347 2024.10.22 01:26:47.593745 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:26:47.716150 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:26:47.738582 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:26:47.765205 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:26:47.777969 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:26:47.826126 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:26:47.856199 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:26:47.870160 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:26:47.910493 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:26:48.060825 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/prb/kcrmxsvzxnmnsnnjvmeycmxzlswmz (LogSeriesLimiter: on interval from 2024-10-22 01:26:36 to 2024-10-22 01:26:48 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:26:48.411295 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 75 entries to flush up to offset 57711 2024.10.22 01:26:48.451492 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_430_208, 202410_431_431_0] 2024.10.22 01:26:48.460713 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 1.97 GiB, free memory in arenas 4.59 MiB, will set to 1.32 GiB (RSS), difference: -45.37 MiB 2024.10.22 01:26:48.456484 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_430_208, 202410_431_431_0] 2024.10.22 01:26:48.480765 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_430_208, 202410_431_431_0] 2024.10.22 01:26:48.645619 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:48.654346 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:48.806241 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_487_487_0 (state Temporary) 2024.10.22 01:26:48.866103 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_487_487_0 to 202410_493_493_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:48.927993 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57711 2024.10.22 01:26:49.400981 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:49.405570 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:49.863882 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 55373 2024.10.22 01:26:50.250854 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:26:50.255598 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:26:50.449197 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:50.452744 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:50.529982 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_840_840_0 (state Temporary) 2024.10.22 01:26:50.558600 [ 1263 ] {} DatabaseCatalog: Execution took 240 ms. 2024.10.22 01:26:50.622509 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.97 GiB, free memory in arenas 7.14 MiB, will set to 1.36 GiB (RSS), difference: -68.73 MiB 2024.10.22 01:26:50.700827 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 7692 2024.10.22 01:26:50.735015 [ 1269 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:26:50.821055 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_840_840_0 to 202410_853_853_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:50.827965 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:50.830872 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:50.876277 [ 1269 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:26:50.882734 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_497_497_0 (state Temporary) 2024.10.22 01:26:50.889021 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55373 2024.10.22 01:26:50.987535 [ 1727 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/cmm/wwzwchshbpxxslxqvxvyafkxetnlm, size 738189 2024.10.22 01:26:51.052690 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_497_497_0 to 202410_510_510_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:51.210431 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7692 2024.10.22 01:26:51.474788 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column URL 2024.10.22 01:26:51.494996 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column URL 2024.10.22 01:26:51.517465 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column URL 2024.10.22 01:26:51.737361 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column URL 2024.10.22 01:26:51.851191 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column URL 2024.10.22 01:26:51.867371 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column URL 2024.10.22 01:26:51.907704 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column URL 2024.10.22 01:26:51.934892 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column URL 2024.10.22 01:26:52.232972 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_100_100_0 (state Temporary) 2024.10.22 01:26:52.371579 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_483_97/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:26:52.534668 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.97 GiB, free memory in arenas 13.51 MiB, will set to 1.40 GiB (RSS), difference: -80.77 MiB 2024.10.22 01:26:52.786561 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_750_358, 202410_751_751_0] 2024.10.22 01:26:52.891455 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_750_358, 202410_751_751_0] 2024.10.22 01:26:52.903070 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_750_358, 202410_751_751_0] 2024.10.22 01:26:53.441586 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1721 entries to flush up to offset 752336 2024.10.22 01:26:53.480981 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 420 2024.10.22 01:26:53.761639 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:53.763627 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:53.824036 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_402_402_0 (state Temporary) 2024.10.22 01:26:53.852089 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_446_89, 202410_447_447_0, 202410_448_448_0, 202410_449_449_0, 202410_450_450_0, 202410_451_451_0] 2024.10.22 01:26:53.857213 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_446_89, 202410_447_447_0, 202410_448_448_0, 202410_449_449_0, 202410_450_450_0, 202410_451_451_0] 2024.10.22 01:26:53.878436 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_446_89, 202410_447_447_0, 202410_448_448_0, 202410_449_449_0, 202410_450_450_0, 202410_451_451_0] 2024.10.22 01:26:54.018507 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_402_402_0 to 202410_412_412_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:54.319719 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 420 2024.10.22 01:26:54.379946 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 12.51 MiB, will set to 1.46 GiB (RSS), difference: -86.87 MiB 2024.10.22 01:26:54.627405 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:54.631331 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:54.918347 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 55419 2024.10.22 01:26:54.990897 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_477_477_0 (state Temporary) 2024.10.22 01:26:55.144092 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:55.151910 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:55.254604 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_841_841_0 (state Temporary) 2024.10.22 01:26:55.265555 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_431_209, 202410_432_432_0] 2024.10.22 01:26:55.268032 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_431_209, 202410_432_432_0] 2024.10.22 01:26:55.283312 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_431_209, 202410_432_432_0] 2024.10.22 01:26:55.322868 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_477_477_0 to 202410_484_484_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:55.362970 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 752336 2024.10.22 01:26:55.489577 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_841_841_0 to 202410_854_854_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:55.538173 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_843_431 to 202410_844_849_1 2024.10.22 01:26:55.629146 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55419 2024.10.22 01:26:55.657626 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:55.660315 [ 1252 ] {} DiskLocal: Reserved 2.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:26:55.881501 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_849_432} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_843_431 to 202410_844_849_1 into Compact with storage Full 2024.10.22 01:26:55.949410 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_849_432} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:26:55.959451 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_849_432} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_843_431, total 55594 rows starting from the beginning of the part 2024.10.22 01:26:55.974737 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_849_432} MergeTreeSequentialSource: Reading 2 marks from part 202410_844_849_1, total 407 rows starting from the beginning of the part 2024.10.22 01:26:56.266907 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_100_100_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:26:56.446426 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 76 entries to flush up to offset 57787 2024.10.22 01:26:56.478461 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.97 GiB, free memory in arenas 11.78 MiB, will set to 1.52 GiB (RSS), difference: -42.28 MiB 2024.10.22 01:26:56.605328 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:56.610827 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:56.772198 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_488_488_0 (state Temporary) 2024.10.22 01:26:56.828162 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_488_488_0 to 202410_494_494_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:56.866989 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_751_359, 202410_752_752_0] 2024.10.22 01:26:56.870350 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_751_359, 202410_752_752_0] 2024.10.22 01:26:56.872155 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57787 2024.10.22 01:26:56.884643 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_751_359, 202410_752_752_0] 2024.10.22 01:26:58.704735 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.97 GiB, free memory in arenas 12.95 MiB, will set to 1.58 GiB (RSS), difference: -24.54 MiB 2024.10.22 01:26:58.713133 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 7698 2024.10.22 01:26:58.972329 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:58.979865 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:59.036240 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_498_498_0 (state Temporary) 2024.10.22 01:26:59.096088 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_498_498_0 to 202410_511_511_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:26:59.162252 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7698 2024.10.22 01:26:59.268035 [ 1187 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_506_101 to 202410_511_511_0 2024.10.22 01:26:59.380144 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:26:59.384741 [ 1187 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:26:59.395048 [ 1187 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:26:59.412027 [ 1187 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 291 ms. 2024.10.22 01:26:59.454749 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_506_101 to 202410_511_511_0 into Compact with storage Full 2024.10.22 01:26:59.544577 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:26:59.560545 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_506_101, total 7744 rows starting from the beginning of the part 2024.10.22 01:26:59.622454 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_507_0, total 13 rows starting from the beginning of the part 2024.10.22 01:26:59.638185 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 55465 2024.10.22 01:26:59.667687 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_508_508_0, total 8 rows starting from the beginning of the part 2024.10.22 01:26:59.734942 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_509_509_0, total 28 rows starting from the beginning of the part 2024.10.22 01:26:59.782517 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_510_510_0, total 5 rows starting from the beginning of the part 2024.10.22 01:26:59.843014 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_511_511_0, total 6 rows starting from the beginning of the part 2024.10.22 01:26:59.847737 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_849_432} MergeTask::MergeProjectionsStage: Merge sorted 56001 rows, containing 13 columns (13 merged, 0 gathered) in 3.971137167 sec., 14102.00596075255 rows/sec., 2.25 MiB/sec. 2024.10.22 01:27:00.038723 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:00.044741 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:00.077174 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_842_842_0 (state Temporary) 2024.10.22 01:27:00.154754 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_849_432} MergedBlockOutputStream: filled checksums 202410_1_849_432 (state Temporary) 2024.10.22 01:27:00.255142 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:27:00.258276 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_842_842_0 to 202410_855_855_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:00.259625 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:27:00.400343 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55465 2024.10.22 01:27:00.275529 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_850_850_0 to 202410_855_855_0 2024.10.22 01:27:00.431053 [ 1234 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:00.437424 [ 1234 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:00.466231 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_100_100_0 to 202410_103_103_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:00.480241 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_850_850_0 to 202410_855_855_0 into Compact with storage Full 2024.10.22 01:27:00.499132 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 11.36 MiB, will set to 1.61 GiB (RSS), difference: -51.49 MiB 2024.10.22 01:27:00.732028 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_849_432} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_849_432 to 202410_1_849_432 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:00.739060 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:00.740855 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_849_432} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_843_431, 202410_844_849_1] -> 202410_1_849_432 2024.10.22 01:27:00.766327 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_850_850_0, total 64 rows starting from the beginning of the part 2024.10.22 01:27:00.776595 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.88 MiB. 2024.10.22 01:27:00.831919 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} MergeTask::MergeProjectionsStage: Merge sorted 7804 rows, containing 12 columns (12 merged, 0 gathered) in 1.401588721 sec., 5567.9671811514245 rows/sec., 884.38 KiB/sec. 2024.10.22 01:27:00.889485 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_851_851_0, total 61 rows starting from the beginning of the part 2024.10.22 01:27:00.914787 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} MergedBlockOutputStream: filled checksums 202410_1_511_102 (state Temporary) 2024.10.22 01:27:01.073269 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_852_852_0, total 24 rows starting from the beginning of the part 2024.10.22 01:27:01.138455 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_853_853_0, total 54 rows starting from the beginning of the part 2024.10.22 01:27:01.193722 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_854_854_0, total 46 rows starting from the beginning of the part 2024.10.22 01:27:01.286049 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_855_855_0, total 46 rows starting from the beginning of the part 2024.10.22 01:27:01.317989 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_511_102 to 202410_1_511_102 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:01.329609 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_511_102} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_506_101, 202410_511_511_0] -> 202410_1_511_102 2024.10.22 01:27:01.482982 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.47 MiB. 2024.10.22 01:27:01.824937 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 421 2024.10.22 01:27:02.236586 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:02.247049 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:02.261595 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} MergeTask::MergeProjectionsStage: Merge sorted 295 rows, containing 13 columns (13 merged, 0 gathered) in 1.800394263 sec., 163.85299934717688 rows/sec., 24.62 KiB/sec. 2024.10.22 01:27:02.284026 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} MergedBlockOutputStream: filled checksums 202410_850_855_1 (state Temporary) 2024.10.22 01:27:02.307002 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_403_403_0 (state Temporary) 2024.10.22 01:27:02.367949 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 753876 2024.10.22 01:27:02.374815 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_2_2_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:27:02.411427 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_403_403_0 to 202410_413_413_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:02.452626 [ 1227 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_408_84 to 202410_413_413_0 2024.10.22 01:27:02.461632 [ 1227 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:02.478985 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_359_74, 202410_360_360_0, 202410_361_361_0, 202410_362_362_0, 202410_363_363_0, 202410_364_364_0] 2024.10.22 01:27:02.480513 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_359_74, 202410_360_360_0, 202410_361_361_0, 202410_362_362_0, 202410_363_363_0, 202410_364_364_0] 2024.10.22 01:27:02.492378 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:02.523223 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_850_855_1 to 202410_850_855_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:02.526658 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_359_74, 202410_360_360_0, 202410_361_361_0, 202410_362_362_0, 202410_363_363_0, 202410_364_364_0] 2024.10.22 01:27:02.535145 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_850_855_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_850_850_0, 202410_855_855_0] -> 202410_850_855_1 2024.10.22 01:27:02.547578 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_408_84 to 202410_413_413_0 into Compact with storage Full 2024.10.22 01:27:02.584059 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 421 2024.10.22 01:27:02.762730 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 9.11 MiB, will set to 1.65 GiB (RSS), difference: -74.02 MiB 2024.10.22 01:27:02.861312 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:02.901992 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_752_360, 202410_753_753_0] 2024.10.22 01:27:02.906647 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_752_360, 202410_753_753_0] 2024.10.22 01:27:02.921741 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_752_360, 202410_753_753_0] 2024.10.22 01:27:02.935443 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_408_84, total 452 rows starting from the beginning of the part 2024.10.22 01:27:02.943920 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:27:03.066475 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:03.069282 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:03.208984 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_409_409_0, total 1 rows starting from the beginning of the part 2024.10.22 01:27:03.221195 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_478_478_0 (state Temporary) 2024.10.22 01:27:03.425600 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_478_478_0 to 202410_485_485_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:03.438049 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_410_410_0, total 1 rows starting from the beginning of the part 2024.10.22 01:27:03.472595 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 753876 2024.10.22 01:27:03.475685 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_483_97/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:27:03.756439 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_411_411_0, total 1 rows starting from the beginning of the part 2024.10.22 01:27:04.092065 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4010 2024.10.22 01:27:04.343277 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_412_412_0, total 1 rows starting from the beginning of the part 2024.10.22 01:27:04.392561 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 93 entries to flush up to offset 57880 2024.10.22 01:27:04.412745 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 55538 2024.10.22 01:27:04.460837 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_432_210, 202410_433_433_0] 2024.10.22 01:27:04.461216 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.84 GiB, peak 1.97 GiB, free memory in arenas 14.13 MiB, will set to 1.81 GiB (RSS), difference: -30.39 MiB 2024.10.22 01:27:04.461737 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_432_210, 202410_433_433_0] 2024.10.22 01:27:04.475136 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_432_210, 202410_433_433_0] 2024.10.22 01:27:04.553148 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:04.562597 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:04.820701 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_489_489_0 (state Temporary) 2024.10.22 01:27:04.870610 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:04.873112 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:04.900792 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_413_413_0, total 1 rows starting from the beginning of the part 2024.10.22 01:27:04.965542 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_843_843_0 (state Temporary) 2024.10.22 01:27:04.966395 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_489_489_0 to 202410_495_495_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:05.012074 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57880 2024.10.22 01:27:05.141210 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_843_843_0 to 202410_856_856_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:05.218632 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55538 2024.10.22 01:27:05.883251 [ 1158 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:27:05.915629 [ 1158 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:27:06.372402 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.97 GiB, free memory in arenas 14.03 MiB, will set to 1.76 GiB (RSS), difference: 364.32 MiB 2024.10.22 01:27:06.664527 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 24 entries to flush up to offset 7722 2024.10.22 01:27:06.872891 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:06.875630 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:07.019021 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_499_499_0 (state Temporary) 2024.10.22 01:27:07.085580 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_499_499_0 to 202410_512_512_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:07.196438 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7722 2024.10.22 01:27:07.612538 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} MergeTask::MergeProjectionsStage: Merge sorted 457 rows, containing 32 columns (32 merged, 0 gathered) in 5.081189727 sec., 89.93956623418954 rows/sec., 8.37 KiB/sec. 2024.10.22 01:27:07.666001 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} MergedBlockOutputStream: filled checksums 202410_1_413_85 (state Temporary) 2024.10.22 01:27:07.838408 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_753_361, 202410_754_754_0] 2024.10.22 01:27:07.842389 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_753_361, 202410_754_754_0] 2024.10.22 01:27:07.853831 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_753_361, 202410_754_754_0] 2024.10.22 01:27:07.936497 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 451.34 MiB using storage policy from min volume index 0 2024.10.22 01:27:08.106844 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_413_85/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:27:08.260701 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_413_85 to 202410_1_413_85 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:08.266084 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_413_85} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_408_84, 202410_413_413_0] -> 202410_1_413_85 2024.10.22 01:27:08.376532 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:27:08.534183 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.97 GiB, free memory in arenas 13.91 MiB, will set to 1.66 GiB (RSS), difference: -100.86 MiB 2024.10.22 01:27:09.120399 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fml/zjvoglossxyluvskmevegjjdbfuwb (LogSeriesLimiter: on interval from 2024-10-22 01:26:48 to 2024-10-22 01:27:09 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:27:09.407139 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 37 entries to flush up to offset 55575 2024.10.22 01:27:09.958515 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:09.961235 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:10.018733 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_844_844_0 (state Temporary) 2024.10.22 01:27:10.174545 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_844_844_0 to 202410_857_857_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:10.254766 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:27:10.255307 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55575 2024.10.22 01:27:10.258762 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:27:10.467787 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 16.09 MiB, will set to 1.66 GiB (RSS), difference: -56.28 MiB 2024.10.22 01:27:10.476519 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 755416 2024.10.22 01:27:11.350357 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} MergeTask::MergeProjectionsStage: Merge sorted 764205 rows, containing 4 columns (4 merged, 0 gathered) in 24.977566229 sec., 30595.655036747576 rows/sec., 458.84 KiB/sec. 2024.10.22 01:27:11.505670 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:11.512310 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:11.547305 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} MergedBlockOutputStream: filled checksums 202410_1_483_97 (state Temporary) 2024.10.22 01:27:11.609146 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_479_479_0 (state Temporary) 2024.10.22 01:27:11.682386 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_479_479_0 to 202410_486_486_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:11.723369 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 755416 2024.10.22 01:27:11.965949 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 4050 2024.10.22 01:27:12.264994 [ 1718 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/prb/kcrmxsvzxnmnsnnjvmeycmxzlswmz, size 56 2024.10.22 01:27:12.404258 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 14.33 MiB, will set to 1.66 GiB (RSS), difference: -116.98 MiB 2024.10.22 01:27:12.538950 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 59 entries to flush up to offset 57939 2024.10.22 01:27:12.671861 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:27:12.706649 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:27:12.728448 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:27:12.744611 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:12.748724 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:12.884355 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_754_362, 202410_755_755_0] 2024.10.22 01:27:12.887991 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_754_362, 202410_755_755_0] 2024.10.22 01:27:12.902329 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_490_490_0 (state Temporary) 2024.10.22 01:27:12.958388 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:27:12.980288 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:27:13.005221 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_754_362, 202410_755_755_0] 2024.10.22 01:27:13.034933 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:27:13.051302 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_490_490_0 to 202410_496_496_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:13.025154 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_483_97 to 202410_1_483_97 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:13.056086 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_483_97} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_1_478_96, 202410_483_483_0] -> 202410_1_483_97 2024.10.22 01:27:13.066065 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:27:13.070379 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_461_491_6 to 202410_496_496_0 2024.10.22 01:27:13.076802 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:13.081669 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:13.084334 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:27:13.086494 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:27:13.110411 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_461_491_6 to 202410_496_496_0 into Compact with storage Full 2024.10.22 01:27:13.142236 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:27:13.144639 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 57939 2024.10.22 01:27:13.264132 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:13.277296 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_491_6, total 5390 rows starting from the beginning of the part 2024.10.22 01:27:13.310229 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_492_492_0, total 78 rows starting from the beginning of the part 2024.10.22 01:27:13.538998 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_493_493_0, total 75 rows starting from the beginning of the part 2024.10.22 01:27:13.564158 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_494_494_0, total 76 rows starting from the beginning of the part 2024.10.22 01:27:13.584332 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_495_495_0, total 93 rows starting from the beginning of the part 2024.10.22 01:27:13.624465 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_496_496_0, total 59 rows starting from the beginning of the part 2024.10.22 01:27:14.100892 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qcw/llbtnekhbnqrevmcmtebevgjrebam (LogSeriesLimiter: on interval from 2024-10-22 01:27:09 to 2024-10-22 01:27:14 accepted series 1 / 194 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:27:14.162741 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_461_496_7/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:27:14.262240 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 55629 2024.10.22 01:27:14.297079 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_433_211, 202410_434_434_0] 2024.10.22 01:27:14.300700 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_433_211, 202410_434_434_0] 2024.10.22 01:27:14.324541 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_433_211, 202410_434_434_0] 2024.10.22 01:27:14.400430 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.97 GiB, free memory in arenas 11.04 MiB, will set to 1.65 GiB (RSS), difference: -138.04 MiB 2024.10.22 01:27:14.672481 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:14.674798 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:14.699854 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 7740 2024.10.22 01:27:14.848524 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} MergeTask::MergeProjectionsStage: Merge sorted 5771 rows, containing 13 columns (13 merged, 0 gathered) in 1.749837073 sec., 3298.021335269767 rows/sec., 1.44 MiB/sec. 2024.10.22 01:27:14.956169 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} MergedBlockOutputStream: filled checksums 202410_461_496_7 (state Temporary) 2024.10.22 01:27:15.043147 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:15.048796 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:15.077385 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_500_500_0 (state Temporary) 2024.10.22 01:27:15.093702 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_845_845_0 (state Temporary) 2024.10.22 01:27:15.255353 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_500_500_0 to 202410_513_513_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:15.256338 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_845_845_0 to 202410_858_858_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:15.287428 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7740 2024.10.22 01:27:15.299351 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_461_496_7 to 202410_461_496_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:15.306042 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_496_7} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_461_491_6, 202410_496_496_0] -> 202410_461_496_7 2024.10.22 01:27:15.412764 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55629 2024.10.22 01:27:15.546387 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.69 MiB. 2024.10.22 01:27:16.236364 [ 1169 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:27:16.363482 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.81 GiB, peak 1.97 GiB, free memory in arenas 6.34 MiB, will set to 1.65 GiB (RSS), difference: -162.02 MiB 2024.10.22 01:27:16.765211 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_755_363, 202410_756_756_0] 2024.10.22 01:27:16.767057 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_755_363, 202410_756_756_0] 2024.10.22 01:27:16.774200 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_755_363, 202410_756_756_0] 2024.10.22 01:27:17.583575 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 6.49 MiB, will set to 1.66 GiB (RSS), difference: -75.16 MiB 2024.10.22 01:27:17.586699 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 422 2024.10.22 01:27:18.001317 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:18.008814 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:27:18.083805 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_404_404_0 (state Temporary) 2024.10.22 01:27:18.362366 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_404_404_0 to 202410_414_414_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:18.420968 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 422 2024.10.22 01:27:18.786722 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1927 entries to flush up to offset 757343 2024.10.22 01:27:19.015369 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 87 file segments (while reserving 16777216 bytes for 1df1b5cc5d849f9e7fc7257b12796a66:0) 2024.10.22 01:27:19.020031 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 37d1c3f4b5d23cb240b04083d26e9e52, offset: 0, size: 189046 2024.10.22 01:27:19.024743 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 37d1c3f4b5d23cb240b04083d26e9e52 for removal 2024.10.22 01:27:19.026950 [ 1476 ] {} CacheMetadata: Key 37d1c3f4b5d23cb240b04083d26e9e52 is removed from metadata 2024.10.22 01:27:19.030881 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 449053ba4368a98b332e755963035754, offset: 0, size: 145 2024.10.22 01:27:19.049604 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 449053ba4368a98b332e755963035754 for removal 2024.10.22 01:27:19.053784 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cc6f3c45afcbb4f4a034eca4f4218607, offset: 0, size: 105849 2024.10.22 01:27:19.057378 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cc6f3c45afcbb4f4a034eca4f4218607 for removal 2024.10.22 01:27:19.059598 [ 1476 ] {} CacheMetadata: Key 449053ba4368a98b332e755963035754 is removed from metadata 2024.10.22 01:27:19.063212 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a8d37e465bb7c2efc70d6593c8a8f4e3, offset: 0, size: 1545574 2024.10.22 01:27:19.068887 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a8d37e465bb7c2efc70d6593c8a8f4e3 for removal 2024.10.22 01:27:19.073478 [ 1476 ] {} CacheMetadata: Key a8d37e465bb7c2efc70d6593c8a8f4e3 is removed from metadata 2024.10.22 01:27:19.074417 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 01104460f6f079106091ace00195e799, offset: 0, size: 201 2024.10.22 01:27:19.081502 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 01104460f6f079106091ace00195e799 for removal 2024.10.22 01:27:19.085125 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dae0259f28de7d813507fde12ff6c0fe, offset: 0, size: 5485 2024.10.22 01:27:19.095033 [ 1476 ] {} CacheMetadata: Key 01104460f6f079106091ace00195e799 is removed from metadata 2024.10.22 01:27:19.103349 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dae0259f28de7d813507fde12ff6c0fe for removal 2024.10.22 01:27:19.113647 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 92e3380ce9178787b32def84d32a9915, offset: 0, size: 214418 2024.10.22 01:27:19.114796 [ 1476 ] {} CacheMetadata: Key dae0259f28de7d813507fde12ff6c0fe is removed from metadata 2024.10.22 01:27:19.121735 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 92e3380ce9178787b32def84d32a9915 for removal 2024.10.22 01:27:19.125911 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a5adf53b4f487d75cd21d621c9d5e8da, offset: 0, size: 181 2024.10.22 01:27:19.127969 [ 1476 ] {} CacheMetadata: Key 92e3380ce9178787b32def84d32a9915 is removed from metadata 2024.10.22 01:27:19.129183 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a5adf53b4f487d75cd21d621c9d5e8da for removal 2024.10.22 01:27:19.150998 [ 1476 ] {} CacheMetadata: Key a5adf53b4f487d75cd21d621c9d5e8da is removed from metadata 2024.10.22 01:27:19.163577 [ 1476 ] {} CacheMetadata: Key cc6f3c45afcbb4f4a034eca4f4218607 is removed from metadata 2024.10.22 01:27:19.235486 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8d84f2b79e35912223e8fc3bb51a8cab, offset: 0, size: 144 2024.10.22 01:27:19.239616 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8d84f2b79e35912223e8fc3bb51a8cab for removal 2024.10.22 01:27:19.244104 [ 1476 ] {} CacheMetadata: Key 8d84f2b79e35912223e8fc3bb51a8cab is removed from metadata 2024.10.22 01:27:19.245574 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a38d9ae6848bee0298f1b9be404f75ca, offset: 0, size: 180 2024.10.22 01:27:19.252049 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a38d9ae6848bee0298f1b9be404f75ca for removal 2024.10.22 01:27:19.254930 [ 1476 ] {} CacheMetadata: Key a38d9ae6848bee0298f1b9be404f75ca is removed from metadata 2024.10.22 01:27:19.255848 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f29b664b3f418632bc14c96cde7f793b, offset: 0, size: 146 2024.10.22 01:27:19.258508 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f29b664b3f418632bc14c96cde7f793b for removal 2024.10.22 01:27:19.262737 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d80fd7346fd429c1530fc21a51e7875d, offset: 0, size: 84103 2024.10.22 01:27:19.268991 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d80fd7346fd429c1530fc21a51e7875d for removal 2024.10.22 01:27:19.275303 [ 1476 ] {} CacheMetadata: Key d80fd7346fd429c1530fc21a51e7875d is removed from metadata 2024.10.22 01:27:19.276566 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2affefac9848beff6b0b0643e29170fd, offset: 0, size: 157967 2024.10.22 01:27:19.282205 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2affefac9848beff6b0b0643e29170fd for removal 2024.10.22 01:27:19.288461 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b9b16b8183c91a8b02a0c29f0b780cfe, offset: 0, size: 46043 2024.10.22 01:27:19.300489 [ 1476 ] {} CacheMetadata: Key 2affefac9848beff6b0b0643e29170fd is removed from metadata 2024.10.22 01:27:19.300710 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b9b16b8183c91a8b02a0c29f0b780cfe for removal 2024.10.22 01:27:19.308737 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1b8565663cddc708d0cb21f1777bddb1, offset: 0, size: 158 2024.10.22 01:27:19.315335 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1b8565663cddc708d0cb21f1777bddb1 for removal 2024.10.22 01:27:19.319326 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 050073676a84b1a40eb3f39c37bdc635, offset: 0, size: 158 2024.10.22 01:27:19.322870 [ 1476 ] {} CacheMetadata: Key b9b16b8183c91a8b02a0c29f0b780cfe is removed from metadata 2024.10.22 01:27:19.329034 [ 1476 ] {} CacheMetadata: Key 1b8565663cddc708d0cb21f1777bddb1 is removed from metadata 2024.10.22 01:27:19.340754 [ 1476 ] {} CacheMetadata: Key f29b664b3f418632bc14c96cde7f793b is removed from metadata 2024.10.22 01:27:19.362332 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:19.366960 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:19.424860 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 050073676a84b1a40eb3f39c37bdc635 for removal 2024.10.22 01:27:19.447248 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 55712 2024.10.22 01:27:19.450012 [ 1476 ] {} CacheMetadata: Key 050073676a84b1a40eb3f39c37bdc635 is removed from metadata 2024.10.22 01:27:19.449401 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8838703661cbff88e23bc183a17f0e0b, offset: 0, size: 287647 2024.10.22 01:27:19.455306 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8838703661cbff88e23bc183a17f0e0b for removal 2024.10.22 01:27:19.460451 [ 1476 ] {} CacheMetadata: Key 8838703661cbff88e23bc183a17f0e0b is removed from metadata 2024.10.22 01:27:19.461337 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d7c07c168a4baeaf96291f3e36738367, offset: 0, size: 106290 2024.10.22 01:27:19.464739 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d7c07c168a4baeaf96291f3e36738367 for removal 2024.10.22 01:27:19.468006 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c3e448e8e879072ccf39e051c43e12d4, offset: 0, size: 271692 2024.10.22 01:27:19.469397 [ 1476 ] {} CacheMetadata: Key d7c07c168a4baeaf96291f3e36738367 is removed from metadata 2024.10.22 01:27:19.473541 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c3e448e8e879072ccf39e051c43e12d4 for removal 2024.10.22 01:27:19.476584 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fc6781512680c7eb5347295ee91c55f6, offset: 0, size: 164 2024.10.22 01:27:19.477996 [ 1476 ] {} CacheMetadata: Key c3e448e8e879072ccf39e051c43e12d4 is removed from metadata 2024.10.22 01:27:19.480719 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fc6781512680c7eb5347295ee91c55f6 for removal 2024.10.22 01:27:19.486186 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9c67113e4ad06913d9247cbd626aa8b2, offset: 0, size: 83811 2024.10.22 01:27:19.496134 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9c67113e4ad06913d9247cbd626aa8b2 for removal 2024.10.22 01:27:19.503306 [ 1476 ] {} CacheMetadata: Key 9c67113e4ad06913d9247cbd626aa8b2 is removed from metadata 2024.10.22 01:27:19.504661 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2dc9a151503c4503e261a840191432e0, offset: 0, size: 156 2024.10.22 01:27:19.514862 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2dc9a151503c4503e261a840191432e0 for removal 2024.10.22 01:27:19.517212 [ 1476 ] {} CacheMetadata: Key fc6781512680c7eb5347295ee91c55f6 is removed from metadata 2024.10.22 01:27:19.526624 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7634ba5f4aa205cebe45f85f1e0152da, offset: 0, size: 162160 2024.10.22 01:27:19.529292 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7634ba5f4aa205cebe45f85f1e0152da for removal 2024.10.22 01:27:19.530845 [ 1476 ] {} CacheMetadata: Key 2dc9a151503c4503e261a840191432e0 is removed from metadata 2024.10.22 01:27:19.532592 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a9f6ab96b0a2226b983065832f935881, offset: 0, size: 1526761 2024.10.22 01:27:19.542323 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a9f6ab96b0a2226b983065832f935881 for removal 2024.10.22 01:27:19.550629 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6e1cecc2856ee91424eca91c9f14cd15, offset: 0, size: 147 2024.10.22 01:27:19.554414 [ 1476 ] {} CacheMetadata: Key a9f6ab96b0a2226b983065832f935881 is removed from metadata 2024.10.22 01:27:19.564080 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6e1cecc2856ee91424eca91c9f14cd15 for removal 2024.10.22 01:27:19.567349 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 5.84 MiB, will set to 1.67 GiB (RSS), difference: -3.32 MiB 2024.10.22 01:27:19.569301 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: daad17713816c59e977b50bf6b3014a9, offset: 0, size: 37952 2024.10.22 01:27:19.572849 [ 1476 ] {} CacheMetadata: Key 6e1cecc2856ee91424eca91c9f14cd15 is removed from metadata 2024.10.22 01:27:19.585191 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key daad17713816c59e977b50bf6b3014a9 for removal 2024.10.22 01:27:19.585733 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_480_480_0 (state Temporary) 2024.10.22 01:27:19.590597 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c85a6564b4de877819150563f36f9179, offset: 0, size: 144 2024.10.22 01:27:19.605499 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c85a6564b4de877819150563f36f9179 for removal 2024.10.22 01:27:19.607622 [ 1476 ] {} CacheMetadata: Key daad17713816c59e977b50bf6b3014a9 is removed from metadata 2024.10.22 01:27:19.611977 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e9bead3fa4df6972993f773065e52d11, offset: 0, size: 42662 2024.10.22 01:27:19.616764 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:19.619246 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e9bead3fa4df6972993f773065e52d11 for removal 2024.10.22 01:27:19.619892 [ 1476 ] {} CacheMetadata: Key c85a6564b4de877819150563f36f9179 is removed from metadata 2024.10.22 01:27:19.622545 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:19.625438 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b724a8b5f5f535e6086caeac211596cb, offset: 0, size: 144 2024.10.22 01:27:19.630788 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b724a8b5f5f535e6086caeac211596cb for removal 2024.10.22 01:27:19.632077 [ 1476 ] {} CacheMetadata: Key e9bead3fa4df6972993f773065e52d11 is removed from metadata 2024.10.22 01:27:19.636785 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9e6cadcbed34706035d717e5b0cc9894, offset: 0, size: 165 2024.10.22 01:27:19.650138 [ 1476 ] {} CacheMetadata: Key b724a8b5f5f535e6086caeac211596cb is removed from metadata 2024.10.22 01:27:19.661324 [ 1476 ] {} CacheMetadata: Key 7634ba5f4aa205cebe45f85f1e0152da is removed from metadata 2024.10.22 01:27:19.674701 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_480_480_0 to 202410_487_487_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:19.711538 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 757343 2024.10.22 01:27:19.746864 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9e6cadcbed34706035d717e5b0cc9894 for removal 2024.10.22 01:27:19.750495 [ 1476 ] {} CacheMetadata: Key 9e6cadcbed34706035d717e5b0cc9894 is removed from metadata 2024.10.22 01:27:19.750299 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b56736ddb1eb4c7ce1d57ba034d76e99, offset: 0, size: 201 2024.10.22 01:27:19.754737 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b56736ddb1eb4c7ce1d57ba034d76e99 for removal 2024.10.22 01:27:19.761394 [ 1476 ] {} CacheMetadata: Key b56736ddb1eb4c7ce1d57ba034d76e99 is removed from metadata 2024.10.22 01:27:19.762618 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dd12706bf2f54188549572bd589dc627, offset: 0, size: 439560 2024.10.22 01:27:19.766160 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dd12706bf2f54188549572bd589dc627 for removal 2024.10.22 01:27:19.770902 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: bad20757697f8cee5b0bee905a329294, offset: 0, size: 335436 2024.10.22 01:27:19.775270 [ 1476 ] {} CacheMetadata: Key dd12706bf2f54188549572bd589dc627 is removed from metadata 2024.10.22 01:27:19.776387 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key bad20757697f8cee5b0bee905a329294 for removal 2024.10.22 01:27:19.782192 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_846_846_0 (state Temporary) 2024.10.22 01:27:19.784874 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 332f9e692cf93529e538586786788b22, offset: 0, size: 201 2024.10.22 01:27:19.798421 [ 1476 ] {} CacheMetadata: Key bad20757697f8cee5b0bee905a329294 is removed from metadata 2024.10.22 01:27:19.802610 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 332f9e692cf93529e538586786788b22 for removal 2024.10.22 01:27:19.917760 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f07075db36fbf0578720294949cdde11, offset: 0, size: 2411 2024.10.22 01:27:19.931695 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f07075db36fbf0578720294949cdde11 for removal 2024.10.22 01:27:19.932981 [ 1476 ] {} CacheMetadata: Key 332f9e692cf93529e538586786788b22 is removed from metadata 2024.10.22 01:27:20.038604 [ 1476 ] {} CacheMetadata: Key f07075db36fbf0578720294949cdde11 is removed from metadata 2024.10.22 01:27:20.038869 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5f7d504c950276a60987826667885b3d, offset: 0, size: 148994 2024.10.22 01:27:20.048350 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_846_846_0 to 202410_859_859_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:20.048337 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5f7d504c950276a60987826667885b3d for removal 2024.10.22 01:27:20.159797 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d9d184d52d1f47b3f0993a644a2a0b7e, offset: 0, size: 145 2024.10.22 01:27:20.165141 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d9d184d52d1f47b3f0993a644a2a0b7e for removal 2024.10.22 01:27:20.167724 [ 1476 ] {} CacheMetadata: Key 5f7d504c950276a60987826667885b3d is removed from metadata 2024.10.22 01:27:20.170125 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4e676426d82b80b0f3a0f310e67ef389, offset: 0, size: 185307 2024.10.22 01:27:20.175785 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4e676426d82b80b0f3a0f310e67ef389 for removal 2024.10.22 01:27:20.176910 [ 1476 ] {} CacheMetadata: Key d9d184d52d1f47b3f0993a644a2a0b7e is removed from metadata 2024.10.22 01:27:20.179418 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8aa696bd3707e87f15b95277c573330f, offset: 0, size: 86419 2024.10.22 01:27:20.191136 [ 1476 ] {} CacheMetadata: Key 4e676426d82b80b0f3a0f310e67ef389 is removed from metadata 2024.10.22 01:27:20.208762 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55712 2024.10.22 01:27:20.287331 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8aa696bd3707e87f15b95277c573330f for removal 2024.10.22 01:27:20.295128 [ 1476 ] {} CacheMetadata: Key 8aa696bd3707e87f15b95277c573330f is removed from metadata 2024.10.22 01:27:20.294453 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d4576c3112846c9220cf7e516cec288c, offset: 0, size: 144 2024.10.22 01:27:20.300675 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d4576c3112846c9220cf7e516cec288c for removal 2024.10.22 01:27:20.316631 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_756_364, 202410_757_757_0] 2024.10.22 01:27:20.316761 [ 1476 ] {} CacheMetadata: Key d4576c3112846c9220cf7e516cec288c is removed from metadata 2024.10.22 01:27:20.320827 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_756_364, 202410_757_757_0] 2024.10.22 01:27:20.357230 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:27:20.361620 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:27:20.420377 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 09844f8e92b2d08fe6e2929ebec9143c, offset: 0, size: 145 2024.10.22 01:27:20.430484 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 09844f8e92b2d08fe6e2929ebec9143c for removal 2024.10.22 01:27:20.434334 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_756_364, 202410_757_757_0] 2024.10.22 01:27:20.448898 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1c08c79688e599a2976a62bfb6eef9c9, offset: 0, size: 56367 2024.10.22 01:27:20.448242 [ 1476 ] {} CacheMetadata: Key 09844f8e92b2d08fe6e2929ebec9143c is removed from metadata 2024.10.22 01:27:20.453648 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1c08c79688e599a2976a62bfb6eef9c9 for removal 2024.10.22 01:27:20.459769 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d30120e37cf0dfd35e492cf4f63a4123, offset: 0, size: 189182 2024.10.22 01:27:20.463271 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d30120e37cf0dfd35e492cf4f63a4123 for removal 2024.10.22 01:27:20.467704 [ 1476 ] {} CacheMetadata: Key 1c08c79688e599a2976a62bfb6eef9c9 is removed from metadata 2024.10.22 01:27:20.468892 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3610caca34cd55b683faa9e6bbce9dec, offset: 0, size: 3159312 2024.10.22 01:27:20.472953 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3610caca34cd55b683faa9e6bbce9dec for removal 2024.10.22 01:27:20.487730 [ 1476 ] {} CacheMetadata: Key 3610caca34cd55b683faa9e6bbce9dec is removed from metadata 2024.10.22 01:27:20.516923 [ 1476 ] {} CacheMetadata: Key d30120e37cf0dfd35e492cf4f63a4123 is removed from metadata 2024.10.22 01:27:20.484182 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6a936f91e6fc1e11add30b7b7aef2c65, offset: 0, size: 161 2024.10.22 01:27:20.578479 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6a936f91e6fc1e11add30b7b7aef2c65 for removal 2024.10.22 01:27:20.580914 [ 1476 ] {} CacheMetadata: Key 6a936f91e6fc1e11add30b7b7aef2c65 is removed from metadata 2024.10.22 01:27:20.582493 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1a9b4a96239ce762574194931d66cff0, offset: 0, size: 122248 2024.10.22 01:27:20.589241 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1a9b4a96239ce762574194931d66cff0 for removal 2024.10.22 01:27:20.605532 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8bd2ab4132301d2c17d1a50011bd884d, offset: 0, size: 145 2024.10.22 01:27:20.610810 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8bd2ab4132301d2c17d1a50011bd884d for removal 2024.10.22 01:27:20.617820 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3482104025f90299156d2f23d8986d3f, offset: 0, size: 144 2024.10.22 01:27:20.626510 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3482104025f90299156d2f23d8986d3f for removal 2024.10.22 01:27:20.632122 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0db02f2302c4ba4f72c9fb295c911df3, offset: 0, size: 155 2024.10.22 01:27:20.636571 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0db02f2302c4ba4f72c9fb295c911df3 for removal 2024.10.22 01:27:20.644261 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: adb1269df9dd3ad86745016649b074b5, offset: 0, size: 144 2024.10.22 01:27:20.647924 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key adb1269df9dd3ad86745016649b074b5 for removal 2024.10.22 01:27:20.652604 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c980bed21581bc8573da458910b4a564, offset: 0, size: 574754 2024.10.22 01:27:20.655972 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 68 entries to flush up to offset 58007 2024.10.22 01:27:20.656904 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c980bed21581bc8573da458910b4a564 for removal 2024.10.22 01:27:20.660236 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e2ecc1f88540895666b2cfaf0c7f2102, offset: 0, size: 229 2024.10.22 01:27:20.663101 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e2ecc1f88540895666b2cfaf0c7f2102 for removal 2024.10.22 01:27:20.667035 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4cfbd5983c44cd2a761868a7e6308f5f, offset: 0, size: 94103 2024.10.22 01:27:20.669534 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4cfbd5983c44cd2a761868a7e6308f5f for removal 2024.10.22 01:27:20.674447 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d22e431f3e725e49a07da315b4bdb30b, offset: 0, size: 145 2024.10.22 01:27:20.676776 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d22e431f3e725e49a07da315b4bdb30b for removal 2024.10.22 01:27:20.680373 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0dfcf7c4e208ff40a1c9a623f04ec1c6, offset: 0, size: 145 2024.10.22 01:27:20.684543 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0dfcf7c4e208ff40a1c9a623f04ec1c6 for removal 2024.10.22 01:27:20.689688 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 72607875e25366f1643853e4c9b62df6, offset: 0, size: 230 2024.10.22 01:27:20.703756 [ 1476 ] {} CacheMetadata: Key 0dfcf7c4e208ff40a1c9a623f04ec1c6 is removed from metadata 2024.10.22 01:27:20.704960 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 72607875e25366f1643853e4c9b62df6 for removal 2024.10.22 01:27:20.714799 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5c5a792a3fdfe701622b5bea190a7f23, offset: 0, size: 183 2024.10.22 01:27:20.720505 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5c5a792a3fdfe701622b5bea190a7f23 for removal 2024.10.22 01:27:20.721487 [ 1476 ] {} CacheMetadata: Key 72607875e25366f1643853e4c9b62df6 is removed from metadata 2024.10.22 01:27:20.729170 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4ce0bd67d91370ae876bc217ec100088, offset: 0, size: 158 2024.10.22 01:27:20.731182 [ 1476 ] {} CacheMetadata: Key 5c5a792a3fdfe701622b5bea190a7f23 is removed from metadata 2024.10.22 01:27:20.738552 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4ce0bd67d91370ae876bc217ec100088 for removal 2024.10.22 01:27:20.750893 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: bcc245faf9444049d761e979e787fc5b, offset: 0, size: 5380 2024.10.22 01:27:20.756609 [ 1476 ] {} CacheMetadata: Key 4ce0bd67d91370ae876bc217ec100088 is removed from metadata 2024.10.22 01:27:20.773023 [ 1476 ] {} CacheMetadata: Key 4cfbd5983c44cd2a761868a7e6308f5f is removed from metadata 2024.10.22 01:27:20.782841 [ 1476 ] {} CacheMetadata: Key e2ecc1f88540895666b2cfaf0c7f2102 is removed from metadata 2024.10.22 01:27:20.806444 [ 1476 ] {} CacheMetadata: Key c980bed21581bc8573da458910b4a564 is removed from metadata 2024.10.22 01:27:20.821503 [ 1476 ] {} CacheMetadata: Key adb1269df9dd3ad86745016649b074b5 is removed from metadata 2024.10.22 01:27:20.836968 [ 1476 ] {} CacheMetadata: Key d22e431f3e725e49a07da315b4bdb30b is removed from metadata 2024.10.22 01:27:20.854013 [ 1264 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:27:20.855701 [ 1476 ] {} CacheMetadata: Key 0db02f2302c4ba4f72c9fb295c911df3 is removed from metadata 2024.10.22 01:27:20.857533 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key bcc245faf9444049d761e979e787fc5b for removal 2024.10.22 01:27:20.860327 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dd4c6a382abab10acf9108f688f95024, offset: 0, size: 145 2024.10.22 01:27:20.865014 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dd4c6a382abab10acf9108f688f95024 for removal 2024.10.22 01:27:20.866099 [ 1476 ] {} CacheMetadata: Key bcc245faf9444049d761e979e787fc5b is removed from metadata 2024.10.22 01:27:20.871103 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1e5f0feb3b36f17126012d7b6849845d, offset: 0, size: 17280 2024.10.22 01:27:20.874949 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1e5f0feb3b36f17126012d7b6849845d for removal 2024.10.22 01:27:20.875124 [ 1476 ] {} CacheMetadata: Key dd4c6a382abab10acf9108f688f95024 is removed from metadata 2024.10.22 01:27:20.879893 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a16bde0e120389aff5cdd5692d9a3c64, offset: 0, size: 176 2024.10.22 01:27:20.883297 [ 1476 ] {} CacheMetadata: Key 1e5f0feb3b36f17126012d7b6849845d is removed from metadata 2024.10.22 01:27:20.885363 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a16bde0e120389aff5cdd5692d9a3c64 for removal 2024.10.22 01:27:20.891969 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 86b03bc864fe0f026372a65e26543853, offset: 0, size: 186 2024.10.22 01:27:20.901415 [ 1476 ] {} CacheMetadata: Key a16bde0e120389aff5cdd5692d9a3c64 is removed from metadata 2024.10.22 01:27:20.905364 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 86b03bc864fe0f026372a65e26543853 for removal 2024.10.22 01:27:20.909258 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fde266177cb2dbae680b3a9a5fbedb67, offset: 0, size: 1549222 2024.10.22 01:27:20.913509 [ 1476 ] {} CacheMetadata: Key 86b03bc864fe0f026372a65e26543853 is removed from metadata 2024.10.22 01:27:20.914404 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fde266177cb2dbae680b3a9a5fbedb67 for removal 2024.10.22 01:27:20.920121 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1bf47ec00f2758cafccf1ab7032a53f9, offset: 0, size: 158 2024.10.22 01:27:20.923768 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1bf47ec00f2758cafccf1ab7032a53f9 for removal 2024.10.22 01:27:20.929194 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dc3f10874b8939e79dd8ffce9182db83, offset: 0, size: 27711 2024.10.22 01:27:20.936430 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dc3f10874b8939e79dd8ffce9182db83 for removal 2024.10.22 01:27:20.947859 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 569ba616224fadf8c75449ff105ebf6a, offset: 0, size: 146 2024.10.22 01:27:20.952525 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 569ba616224fadf8c75449ff105ebf6a for removal 2024.10.22 01:27:20.956156 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c225e930d77085d8d3a7f999777abb3f, offset: 0, size: 189067 2024.10.22 01:27:21.025781 [ 1476 ] {} CacheMetadata: Key 569ba616224fadf8c75449ff105ebf6a is removed from metadata 2024.10.22 01:27:21.061033 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c225e930d77085d8d3a7f999777abb3f for removal 2024.10.22 01:27:21.063645 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 542efc818372cb3a9c28b16a4dc2d245, offset: 0, size: 294061 2024.10.22 01:27:21.065239 [ 1476 ] {} CacheMetadata: Key c225e930d77085d8d3a7f999777abb3f is removed from metadata 2024.10.22 01:27:21.073373 [ 1476 ] {} CacheMetadata: Key dc3f10874b8939e79dd8ffce9182db83 is removed from metadata 2024.10.22 01:27:21.090935 [ 1476 ] {} CacheMetadata: Key 1bf47ec00f2758cafccf1ab7032a53f9 is removed from metadata 2024.10.22 01:27:21.104880 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:21.107918 [ 1476 ] {} CacheMetadata: Key fde266177cb2dbae680b3a9a5fbedb67 is removed from metadata 2024.10.22 01:27:21.109078 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:27:21.121187 [ 1476 ] {} CacheMetadata: Key 3482104025f90299156d2f23d8986d3f is removed from metadata 2024.10.22 01:27:21.140481 [ 1476 ] {} CacheMetadata: Key 8bd2ab4132301d2c17d1a50011bd884d is removed from metadata 2024.10.22 01:27:21.144535 [ 1168 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:27:21.152044 [ 1231 ] {} DatabaseCatalog: Execution took 295 ms. 2024.10.22 01:27:21.154928 [ 1476 ] {} CacheMetadata: Key 1a9b4a96239ce762574194931d66cff0 is removed from metadata 2024.10.22 01:27:21.170787 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 542efc818372cb3a9c28b16a4dc2d245 for removal 2024.10.22 01:27:21.174437 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: eedbad51e3a47c0fe8bdf62bac6975f3, offset: 0, size: 185 2024.10.22 01:27:21.174933 [ 1476 ] {} CacheMetadata: Key 542efc818372cb3a9c28b16a4dc2d245 is removed from metadata 2024.10.22 01:27:21.177581 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key eedbad51e3a47c0fe8bdf62bac6975f3 for removal 2024.10.22 01:27:21.181210 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8fcc839e8aa15b53e476ff2a11ead960, offset: 0, size: 3516 2024.10.22 01:27:21.182821 [ 1168 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:27:21.183352 [ 1476 ] {} CacheMetadata: Key eedbad51e3a47c0fe8bdf62bac6975f3 is removed from metadata 2024.10.22 01:27:21.185024 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8fcc839e8aa15b53e476ff2a11ead960 for removal 2024.10.22 01:27:21.189540 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: bd9a5911767f54efe48c15adb4b2bce0, offset: 0, size: 104358 2024.10.22 01:27:21.191463 [ 1476 ] {} CacheMetadata: Key 8fcc839e8aa15b53e476ff2a11ead960 is removed from metadata 2024.10.22 01:27:21.193554 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key bd9a5911767f54efe48c15adb4b2bce0 for removal 2024.10.22 01:27:21.204427 [ 1476 ] {} CacheMetadata: Key bd9a5911767f54efe48c15adb4b2bce0 is removed from metadata 2024.10.22 01:27:21.206642 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d34ab56c6829f94b99b3ab7ef3e92a8d, offset: 0, size: 129673 2024.10.22 01:27:21.214254 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d34ab56c6829f94b99b3ab7ef3e92a8d for removal 2024.10.22 01:27:21.222936 [ 1476 ] {} CacheMetadata: Key d34ab56c6829f94b99b3ab7ef3e92a8d is removed from metadata 2024.10.22 01:27:21.220175 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8cd79d7dcf57006f184f54f0e03884b8, offset: 0, size: 648126 2024.10.22 01:27:21.230480 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8cd79d7dcf57006f184f54f0e03884b8 for removal 2024.10.22 01:27:21.247312 [ 1476 ] {} CacheMetadata: Key 8cd79d7dcf57006f184f54f0e03884b8 is removed from metadata 2024.10.22 01:27:21.252708 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d993bec17f0ca8c7309916f0751db2f2, offset: 0, size: 144 2024.10.22 01:27:21.267061 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d993bec17f0ca8c7309916f0751db2f2 for removal 2024.10.22 01:27:21.273131 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 777103940279af66326a58a7427dfef8, offset: 0, size: 72751 2024.10.22 01:27:21.274932 [ 1476 ] {} CacheMetadata: Key d993bec17f0ca8c7309916f0751db2f2 is removed from metadata 2024.10.22 01:27:21.275237 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_491_491_0 (state Temporary) 2024.10.22 01:27:21.278844 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 777103940279af66326a58a7427dfef8 for removal 2024.10.22 01:27:21.283081 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dc943f07e1198a1289dd488af4350054, offset: 0, size: 154 2024.10.22 01:27:21.284783 [ 1476 ] {} CacheMetadata: Key 777103940279af66326a58a7427dfef8 is removed from metadata 2024.10.22 01:27:21.289270 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dc943f07e1198a1289dd488af4350054 for removal 2024.10.22 01:27:21.298544 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 18e4154a85352815e569df13a19bf073, offset: 0, size: 144 2024.10.22 01:27:21.304866 [ 1476 ] {} CacheMetadata: Key dc943f07e1198a1289dd488af4350054 is removed from metadata 2024.10.22 01:27:21.312389 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 18e4154a85352815e569df13a19bf073 for removal 2024.10.22 01:27:21.318281 [ 1476 ] {} CacheMetadata: Key 18e4154a85352815e569df13a19bf073 is removed from metadata 2024.10.22 01:27:21.320252 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2a0df2ee3aae324cacb00ca40e5be037, offset: 0, size: 73677 2024.10.22 01:27:21.321244 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_491_491_0 to 202410_497_497_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:21.330397 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2a0df2ee3aae324cacb00ca40e5be037 for removal 2024.10.22 01:27:21.341771 [ 1476 ] {} CacheMetadata: Key 2a0df2ee3aae324cacb00ca40e5be037 is removed from metadata 2024.10.22 01:27:21.353612 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d594cb1c06c661156724c94eb767dabd, offset: 0, size: 1553147 2024.10.22 01:27:21.362578 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d594cb1c06c661156724c94eb767dabd for removal 2024.10.22 01:27:21.370495 [ 1476 ] {} CacheMetadata: Key d594cb1c06c661156724c94eb767dabd is removed from metadata 2024.10.22 01:27:21.373491 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: bd438da1be0c0a8eedcbd13bc4457db0, offset: 0, size: 269 2024.10.22 01:27:21.382663 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key bd438da1be0c0a8eedcbd13bc4457db0 for removal 2024.10.22 01:27:21.386738 [ 1476 ] {} CacheMetadata: Key bd438da1be0c0a8eedcbd13bc4457db0 is removed from metadata 2024.10.22 01:27:21.390446 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 26ce3df3b0bf7b70e154cefecd894450, offset: 0, size: 201 2024.10.22 01:27:21.398376 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 26ce3df3b0bf7b70e154cefecd894450 for removal 2024.10.22 01:27:21.405602 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f7f490aca5d4a629de1ea64c00cfc41a, offset: 0, size: 144 2024.10.22 01:27:21.408724 [ 1476 ] {} CacheMetadata: Key 26ce3df3b0bf7b70e154cefecd894450 is removed from metadata 2024.10.22 01:27:21.419778 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f7f490aca5d4a629de1ea64c00cfc41a for removal 2024.10.22 01:27:21.424446 [ 1476 ] {} CacheMetadata: Key f7f490aca5d4a629de1ea64c00cfc41a is removed from metadata 2024.10.22 01:27:21.425130 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 58007 2024.10.22 01:27:21.425015 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ecbe2c18588fd227370d8853868da5eb, offset: 0, size: 145 2024.10.22 01:27:21.430587 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ecbe2c18588fd227370d8853868da5eb for removal 2024.10.22 01:27:21.439844 [ 1476 ] {} CacheMetadata: Key ecbe2c18588fd227370d8853868da5eb is removed from metadata 2024.10.22 01:27:21.443282 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 872d4996d3151007226542e29ab29657, offset: 0, size: 42325 2024.10.22 01:27:21.448305 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 872d4996d3151007226542e29ab29657 for removal 2024.10.22 01:27:21.451881 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d870037eb6b059153f2d2abfa4ab2832, offset: 0, size: 2805 2024.10.22 01:27:21.453356 [ 1476 ] {} CacheMetadata: Key 872d4996d3151007226542e29ab29657 is removed from metadata 2024.10.22 01:27:21.458517 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d870037eb6b059153f2d2abfa4ab2832 for removal 2024.10.22 01:27:21.461403 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4d573b4d40d37069d1eaed2f9dd401c2, offset: 0, size: 4 2024.10.22 01:27:21.463181 [ 1476 ] {} CacheMetadata: Key d870037eb6b059153f2d2abfa4ab2832 is removed from metadata 2024.10.22 01:27:21.557241 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_84_17, 202410_85_85_0, 202410_86_86_0, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0] 2024.10.22 01:27:21.562265 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_84_17, 202410_85_85_0, 202410_86_86_0, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0] 2024.10.22 01:27:21.567136 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4d573b4d40d37069d1eaed2f9dd401c2 for removal 2024.10.22 01:27:21.570398 [ 1476 ] {} CacheMetadata: Key 4d573b4d40d37069d1eaed2f9dd401c2 is removed from metadata 2024.10.22 01:27:21.596816 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_84_17, 202410_85_85_0, 202410_86_86_0, 202410_87_87_0, 202410_88_88_0, 202410_89_89_0] 2024.10.22 01:27:22.801414 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 7753 2024.10.22 01:27:22.841791 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_434_212, 202410_435_435_0] 2024.10.22 01:27:22.849154 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_434_212, 202410_435_435_0] 2024.10.22 01:27:22.952367 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:22.955484 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:23.006413 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_434_212, 202410_435_435_0] 2024.10.22 01:27:23.012398 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_501_501_0 (state Temporary) 2024.10.22 01:27:23.200189 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_501_501_0 to 202410_514_514_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:23.469500 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7753 2024.10.22 01:27:23.684788 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 3.79 MiB, will set to 1.68 GiB (RSS), difference: -10.58 MiB 2024.10.22 01:27:24.328545 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 253 entries to flush up to offset 55965 2024.10.22 01:27:24.568554 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_418_85, 202410_419_424_1, 202410_425_425_0, 202410_426_426_0, 202410_427_427_0, 202410_428_428_0] 2024.10.22 01:27:24.589333 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_418_85, 202410_419_424_1, 202410_425_425_0, 202410_426_426_0, 202410_427_427_0, 202410_428_428_0] 2024.10.22 01:27:24.728656 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_418_85, 202410_419_424_1, 202410_425_425_0, 202410_426_426_0, 202410_427_427_0, 202410_428_428_0] 2024.10.22 01:27:24.991832 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:25.003658 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:25.076588 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_847_847_0 (state Temporary) 2024.10.22 01:27:25.460115 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_847_847_0 to 202410_860_860_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:25.495511 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 4.77 MiB, will set to 1.69 GiB (RSS), difference: 13.78 MiB 2024.10.22 01:27:25.506724 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 55965 2024.10.22 01:27:25.599076 [ 1232 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_849_432 to 202410_850_855_1 2024.10.22 01:27:25.608134 [ 1232 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.01 MiB using storage policy from min volume index 0 2024.10.22 01:27:25.613415 [ 1232 ] {} DiskLocal: Reserved 2.01 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:25.672439 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_855_433} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_849_432 to 202410_850_855_1 into Compact with storage Full 2024.10.22 01:27:25.845526 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_855_433} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:25.927301 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 423 2024.10.22 01:27:25.957588 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_855_433} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_849_432, total 56001 rows starting from the beginning of the part 2024.10.22 01:27:26.024255 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_855_433} MergeTreeSequentialSource: Reading 2 marks from part 202410_850_855_1, total 295 rows starting from the beginning of the part 2024.10.22 01:27:26.340412 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:26.346609 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:26.537624 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_405_405_0 (state Temporary) 2024.10.22 01:27:26.718124 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 758498 2024.10.22 01:27:26.781374 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_405_405_0 to 202410_415_415_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:26.889261 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 423 2024.10.22 01:27:27.377351 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_757_365, 202410_758_758_0] 2024.10.22 01:27:27.381831 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_757_365, 202410_758_758_0] 2024.10.22 01:27:27.396918 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_757_365, 202410_758_758_0] 2024.10.22 01:27:27.581327 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 4.60 MiB, will set to 1.70 GiB (RSS), difference: 7.77 MiB 2024.10.22 01:27:27.729718 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:27.734869 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:27.983105 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_481_481_0 (state Temporary) 2024.10.22 01:27:28.045297 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_481_481_0 to 202410_488_488_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:28.064465 [ 1159 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_483_97 to 202410_484_484_0 2024.10.22 01:27:28.069582 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 758498 2024.10.22 01:27:28.080768 [ 1159 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:28.172677 [ 1159 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:28.256219 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_484_98} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_483_97 to 202410_484_484_0 into Wide with storage Full 2024.10.22 01:27:28.290280 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_484_98} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:28.310970 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_484_98} MergeTreeSequentialSource: Reading 95 marks from part 202410_1_483_97, total 764205 rows starting from the beginning of the part 2024.10.22 01:27:28.397322 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_484_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_484_484_0, total 1721 rows starting from the beginning of the part 2024.10.22 01:27:28.713458 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:28.715818 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:28.942245 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 99 entries to flush up to offset 58106 2024.10.22 01:27:29.168504 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:29.173128 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:29.367962 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_492_492_0 (state Temporary) 2024.10.22 01:27:29.529436 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_492_492_0 to 202410_498_498_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:29.555619 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 5.30 MiB, will set to 1.70 GiB (RSS), difference: -7.76 MiB 2024.10.22 01:27:29.624577 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_855_433} MergeTask::MergeProjectionsStage: Merge sorted 56296 rows, containing 13 columns (13 merged, 0 gathered) in 3.960511582 sec., 14214.32530480604 rows/sec., 2.27 MiB/sec. 2024.10.22 01:27:29.659775 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 56011 2024.10.22 01:27:29.720492 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_855_433} MergedBlockOutputStream: filled checksums 202410_1_855_433 (state Temporary) 2024.10.22 01:27:29.828686 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_435_213, 202410_436_436_0] 2024.10.22 01:27:29.834626 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_435_213, 202410_436_436_0] 2024.10.22 01:27:29.856177 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_435_213, 202410_436_436_0] 2024.10.22 01:27:29.906749 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_855_433} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_855_433 to 202410_1_855_433 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:29.911239 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_855_433} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_849_432, 202410_850_855_1] -> 202410_1_855_433 2024.10.22 01:27:29.930403 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 58106 2024.10.22 01:27:29.941238 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:27:30.000946 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:30.010790 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:30.061745 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_848_848_0 (state Temporary) 2024.10.22 01:27:30.238695 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_848_848_0 to 202410_861_861_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:30.255581 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:27:30.260070 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:27:30.276565 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_855_433 to 202410_856_856_0 2024.10.22 01:27:30.293682 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.01 MiB using storage policy from min volume index 0 2024.10.22 01:27:30.302528 [ 1248 ] {} DiskLocal: Reserved 2.01 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:30.334435 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_856_434} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_855_433 to 202410_856_856_0 into Compact with storage Full 2024.10.22 01:27:30.384109 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_856_434} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:30.414303 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_856_434} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_855_433, total 56296 rows starting from the beginning of the part 2024.10.22 01:27:30.423978 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56011 2024.10.22 01:27:30.464154 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_856_434} MergeTreeSequentialSource: Reading 2 marks from part 202410_856_856_0, total 73 rows starting from the beginning of the part 2024.10.22 01:27:30.500376 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_758_366, 202410_759_759_0] 2024.10.22 01:27:30.506482 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_758_366, 202410_759_759_0] 2024.10.22 01:27:30.622370 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_758_366, 202410_759_759_0] 2024.10.22 01:27:30.974815 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 7762 2024.10.22 01:27:31.203579 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:31.207563 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:27:31.266951 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_502_502_0 (state Temporary) 2024.10.22 01:27:31.274435 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_484_98/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:27:31.350517 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_502_502_0 to 202410_515_515_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:31.504990 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7762 2024.10.22 01:27:31.644369 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 9.13 MiB, will set to 1.72 GiB (RSS), difference: 2.36 MiB 2024.10.22 01:27:32.390712 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_101_101_0 (state Temporary) 2024.10.22 01:27:32.411845 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 49 file segments (while reserving 16777216 bytes for 1df1b5cc5d849f9e7fc7257b12796a66:33554432) 2024.10.22 01:27:32.418574 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1df1b5cc5d849f9e7fc7257b12796a66, offset: 0, size: 33554432 2024.10.22 01:27:32.454971 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 772f2b5b553f2d03e57dda1c7e8d761a, offset: 0, size: 144 2024.10.22 01:27:32.458733 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 772f2b5b553f2d03e57dda1c7e8d761a for removal 2024.10.22 01:27:32.462346 [ 1476 ] {} CacheMetadata: Key 772f2b5b553f2d03e57dda1c7e8d761a is removed from metadata 2024.10.22 01:27:32.462857 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 76fddf1bfafb7a6ce0311be0fd7a8fca, offset: 0, size: 1465076 2024.10.22 01:27:32.465455 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 76fddf1bfafb7a6ce0311be0fd7a8fca for removal 2024.10.22 01:27:32.472503 [ 1476 ] {} CacheMetadata: Key 76fddf1bfafb7a6ce0311be0fd7a8fca is removed from metadata 2024.10.22 01:27:32.477825 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f263482935fa739a57d1490e77adf525, offset: 0, size: 655060 2024.10.22 01:27:32.481152 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f263482935fa739a57d1490e77adf525 for removal 2024.10.22 01:27:32.484435 [ 1476 ] {} CacheMetadata: Key f263482935fa739a57d1490e77adf525 is removed from metadata 2024.10.22 01:27:32.484520 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ba667c1045f5c072704ada9a1d75dd6f, offset: 0, size: 481210 2024.10.22 01:27:32.501036 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ba667c1045f5c072704ada9a1d75dd6f for removal 2024.10.22 01:27:32.514285 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2e26fa8cafa6982f7208dd91e853ab4a, offset: 0, size: 187 2024.10.22 01:27:32.515799 [ 1476 ] {} CacheMetadata: Key ba667c1045f5c072704ada9a1d75dd6f is removed from metadata 2024.10.22 01:27:32.521840 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2e26fa8cafa6982f7208dd91e853ab4a for removal 2024.10.22 01:27:32.524801 [ 1476 ] {} CacheMetadata: Key 2e26fa8cafa6982f7208dd91e853ab4a is removed from metadata 2024.10.22 01:27:32.526182 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3cb8c8b62d8d273bfaafa379ea43323d, offset: 0, size: 191 2024.10.22 01:27:32.529722 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3cb8c8b62d8d273bfaafa379ea43323d for removal 2024.10.22 01:27:32.535931 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 43b9225f1c419e78a8ae629bdfdc958b, offset: 0, size: 190 2024.10.22 01:27:32.545459 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 43b9225f1c419e78a8ae629bdfdc958b for removal 2024.10.22 01:27:32.554813 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 911136aba0ba4eb722ed5eb8e1697cff, offset: 0, size: 243 2024.10.22 01:27:32.559064 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 911136aba0ba4eb722ed5eb8e1697cff for removal 2024.10.22 01:27:32.562113 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2dae1904505b209f8d2a991446bf6df5, offset: 0, size: 178 2024.10.22 01:27:32.565167 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2dae1904505b209f8d2a991446bf6df5 for removal 2024.10.22 01:27:32.571929 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4b2739da6acd927053da94650770f754, offset: 0, size: 141046 2024.10.22 01:27:32.574854 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4b2739da6acd927053da94650770f754 for removal 2024.10.22 01:27:32.651899 [ 1476 ] {} CacheMetadata: Key 4b2739da6acd927053da94650770f754 is removed from metadata 2024.10.22 01:27:32.662449 [ 1476 ] {} CacheMetadata: Key 2dae1904505b209f8d2a991446bf6df5 is removed from metadata 2024.10.22 01:27:32.674859 [ 1476 ] {} CacheMetadata: Key 911136aba0ba4eb722ed5eb8e1697cff is removed from metadata 2024.10.22 01:27:32.678329 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 76fdd78a550c706077acee96cd2c8a53, offset: 0, size: 180 2024.10.22 01:27:32.683889 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 76fdd78a550c706077acee96cd2c8a53 for removal 2024.10.22 01:27:32.686658 [ 1476 ] {} CacheMetadata: Key 43b9225f1c419e78a8ae629bdfdc958b is removed from metadata 2024.10.22 01:27:32.716596 [ 1476 ] {} CacheMetadata: Key 3cb8c8b62d8d273bfaafa379ea43323d is removed from metadata 2024.10.22 01:27:32.819880 [ 1476 ] {} CacheMetadata: Key 76fdd78a550c706077acee96cd2c8a53 is removed from metadata 2024.10.22 01:27:32.828985 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 05291b34ebe4652090a0e2415d1feb8c, offset: 0, size: 180 2024.10.22 01:27:32.847948 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 05291b34ebe4652090a0e2415d1feb8c for removal 2024.10.22 01:27:32.859586 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6bfb26b3be4ddaac3e203b23eb616318, offset: 0, size: 578452 2024.10.22 01:27:32.863794 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6bfb26b3be4ddaac3e203b23eb616318 for removal 2024.10.22 01:27:32.869203 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b4b9f2d50a8a24c7128ad75b67462a5d, offset: 0, size: 189 2024.10.22 01:27:32.873202 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b4b9f2d50a8a24c7128ad75b67462a5d for removal 2024.10.22 01:27:32.876491 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dba56d4c441674c93911bfbee3060681, offset: 0, size: 241709 2024.10.22 01:27:32.881338 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dba56d4c441674c93911bfbee3060681 for removal 2024.10.22 01:27:32.889486 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 826edd1b978084de7a4378a08e4b018e, offset: 0, size: 82741 2024.10.22 01:27:32.900164 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 826edd1b978084de7a4378a08e4b018e for removal 2024.10.22 01:27:32.909136 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ccb0f4a3aa5c136f7da64bfa1b41e7ba, offset: 0, size: 1752 2024.10.22 01:27:32.914345 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ccb0f4a3aa5c136f7da64bfa1b41e7ba for removal 2024.10.22 01:27:32.924888 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 88174f7281608ce0b444fae52167708b, offset: 0, size: 181 2024.10.22 01:27:32.932995 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 88174f7281608ce0b444fae52167708b for removal 2024.10.22 01:27:32.966822 [ 1476 ] {} CacheMetadata: Key 88174f7281608ce0b444fae52167708b is removed from metadata 2024.10.22 01:27:32.966872 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c95b96e1b315751671724dce5be7e81f, offset: 0, size: 144 2024.10.22 01:27:32.975969 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c95b96e1b315751671724dce5be7e81f for removal 2024.10.22 01:27:32.984303 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 80b1b80d455f0b1149aa923c9498371f, offset: 0, size: 473939 2024.10.22 01:27:33.006266 [ 1476 ] {} CacheMetadata: Key c95b96e1b315751671724dce5be7e81f is removed from metadata 2024.10.22 01:27:33.091618 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 80b1b80d455f0b1149aa923c9498371f for removal 2024.10.22 01:27:33.111233 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 04b5210cb8a81739c4348ed8f6ee9afe, offset: 0, size: 7008 2024.10.22 01:27:33.132122 [ 1476 ] {} CacheMetadata: Key 80b1b80d455f0b1149aa923c9498371f is removed from metadata 2024.10.22 01:27:33.146301 [ 1476 ] {} CacheMetadata: Key ccb0f4a3aa5c136f7da64bfa1b41e7ba is removed from metadata 2024.10.22 01:27:33.156065 [ 1476 ] {} CacheMetadata: Key 826edd1b978084de7a4378a08e4b018e is removed from metadata 2024.10.22 01:27:33.166273 [ 1476 ] {} CacheMetadata: Key dba56d4c441674c93911bfbee3060681 is removed from metadata 2024.10.22 01:27:33.175377 [ 1476 ] {} CacheMetadata: Key b4b9f2d50a8a24c7128ad75b67462a5d is removed from metadata 2024.10.22 01:27:33.182526 [ 1476 ] {} CacheMetadata: Key 6bfb26b3be4ddaac3e203b23eb616318 is removed from metadata 2024.10.22 01:27:33.198242 [ 1476 ] {} CacheMetadata: Key 05291b34ebe4652090a0e2415d1feb8c is removed from metadata 2024.10.22 01:27:33.219956 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 04b5210cb8a81739c4348ed8f6ee9afe for removal 2024.10.22 01:27:33.223934 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cb821066b2c1ccf86d147dfb7d7b7b79, offset: 0, size: 193 2024.10.22 01:27:33.226524 [ 1476 ] {} CacheMetadata: Key 04b5210cb8a81739c4348ed8f6ee9afe is removed from metadata 2024.10.22 01:27:33.225811 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cb821066b2c1ccf86d147dfb7d7b7b79 for removal 2024.10.22 01:27:33.229675 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4dd760d5b45aee2cb42ca90b992b3e7b, offset: 0, size: 830243 2024.10.22 01:27:33.238510 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4dd760d5b45aee2cb42ca90b992b3e7b for removal 2024.10.22 01:27:33.243133 [ 1476 ] {} CacheMetadata: Key cb821066b2c1ccf86d147dfb7d7b7b79 is removed from metadata 2024.10.22 01:27:33.248733 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d07e5f87b76063675d1a202dd165f882, offset: 0, size: 187 2024.10.22 01:27:33.253220 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d07e5f87b76063675d1a202dd165f882 for removal 2024.10.22 01:27:33.255061 [ 1476 ] {} CacheMetadata: Key 4dd760d5b45aee2cb42ca90b992b3e7b is removed from metadata 2024.10.22 01:27:33.259462 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2062f7273b176ba5557d5f1c7eb58068, offset: 0, size: 141915 2024.10.22 01:27:33.263343 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2062f7273b176ba5557d5f1c7eb58068 for removal 2024.10.22 01:27:33.267611 [ 1476 ] {} CacheMetadata: Key d07e5f87b76063675d1a202dd165f882 is removed from metadata 2024.10.22 01:27:33.271950 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 664c96166d3c70b68fd49eedb6008c72, offset: 0, size: 183 2024.10.22 01:27:33.278614 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 664c96166d3c70b68fd49eedb6008c72 for removal 2024.10.22 01:27:33.286958 [ 1476 ] {} CacheMetadata: Key 2062f7273b176ba5557d5f1c7eb58068 is removed from metadata 2024.10.22 01:27:33.298939 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e9930353083862dbdaa7a8f10ae648b7, offset: 0, size: 437620 2024.10.22 01:27:33.305226 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e9930353083862dbdaa7a8f10ae648b7 for removal 2024.10.22 01:27:33.307902 [ 1476 ] {} CacheMetadata: Key 664c96166d3c70b68fd49eedb6008c72 is removed from metadata 2024.10.22 01:27:33.312428 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 906e0757db98db05a1ada4d06c15bf0f, offset: 0, size: 193 2024.10.22 01:27:33.319862 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 906e0757db98db05a1ada4d06c15bf0f for removal 2024.10.22 01:27:33.320459 [ 1476 ] {} CacheMetadata: Key e9930353083862dbdaa7a8f10ae648b7 is removed from metadata 2024.10.22 01:27:33.324926 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 62d214f5657f5556ce72f60d6e57a9a3, offset: 0, size: 186 2024.10.22 01:27:33.328145 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 62d214f5657f5556ce72f60d6e57a9a3 for removal 2024.10.22 01:27:33.331911 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b47a24986888735ede989bf81cc738b5, offset: 0, size: 178 2024.10.22 01:27:33.336562 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b47a24986888735ede989bf81cc738b5 for removal 2024.10.22 01:27:33.347632 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e0d74d11b537dcf9f2c36c9889a625d8, offset: 0, size: 834544 2024.10.22 01:27:33.356994 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e0d74d11b537dcf9f2c36c9889a625d8 for removal 2024.10.22 01:27:33.363830 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 223cd40d7d867585116d1cfa6d5ed44e, offset: 0, size: 91400 2024.10.22 01:27:33.368376 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 223cd40d7d867585116d1cfa6d5ed44e for removal 2024.10.22 01:27:33.373527 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a2af1fea912e8f5ef8fe797d3b12c04a, offset: 0, size: 176 2024.10.22 01:27:33.376701 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a2af1fea912e8f5ef8fe797d3b12c04a for removal 2024.10.22 01:27:33.380452 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fbfda3fd90fac27e047330ab965acbad, offset: 0, size: 47329 2024.10.22 01:27:33.384827 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fbfda3fd90fac27e047330ab965acbad for removal 2024.10.22 01:27:33.392090 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 93849e0cdd32fe20e88ab237bb094a24, offset: 0, size: 1711687 2024.10.22 01:27:33.403737 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 93849e0cdd32fe20e88ab237bb094a24 for removal 2024.10.22 01:27:33.415571 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 66d3f960ba8b9b576db50abd45629821, offset: 0, size: 120371 2024.10.22 01:27:33.422756 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 66d3f960ba8b9b576db50abd45629821 for removal 2024.10.22 01:27:33.428027 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 768a77359aff1a1c2b8b74a06bf7379c, offset: 0, size: 178 2024.10.22 01:27:33.430459 [ 1476 ] {} CacheMetadata: Key 66d3f960ba8b9b576db50abd45629821 is removed from metadata 2024.10.22 01:27:33.439567 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 768a77359aff1a1c2b8b74a06bf7379c for removal 2024.10.22 01:27:33.449992 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 03db0679ed7d71d2269b634904b59277, offset: 0, size: 326257 2024.10.22 01:27:33.459552 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 03db0679ed7d71d2269b634904b59277 for removal 2024.10.22 01:27:33.465678 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 61150faad25462f65eddfbfd3adaa3d4, offset: 0, size: 190 2024.10.22 01:27:33.468646 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 61150faad25462f65eddfbfd3adaa3d4 for removal 2024.10.22 01:27:33.473615 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b91435210b7cb0b0c1dbe58771e7fbfc, offset: 0, size: 155 2024.10.22 01:27:33.480420 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b91435210b7cb0b0c1dbe58771e7fbfc for removal 2024.10.22 01:27:33.484205 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fb402b09e31d3f437afe8fc0b2036476, offset: 0, size: 92298 2024.10.22 01:27:33.489214 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fb402b09e31d3f437afe8fc0b2036476 for removal 2024.10.22 01:27:33.501719 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8cc0f173c83b62dea3d1c8f9ba1b66dd, offset: 0, size: 682432 2024.10.22 01:27:33.506843 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8cc0f173c83b62dea3d1c8f9ba1b66dd for removal 2024.10.22 01:27:33.511888 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9b2bec5e861d49c288ee51715a8f2092, offset: 0, size: 628635 2024.10.22 01:27:33.520615 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9b2bec5e861d49c288ee51715a8f2092 for removal 2024.10.22 01:27:33.556619 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 51159aee2a65a3f130c6bd1c00a80678, offset: 0, size: 280410 2024.10.22 01:27:33.557822 [ 1476 ] {} CacheMetadata: Key 9b2bec5e861d49c288ee51715a8f2092 is removed from metadata 2024.10.22 01:27:33.562475 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 51159aee2a65a3f130c6bd1c00a80678 for removal 2024.10.22 01:27:33.562815 [ 1476 ] {} CacheMetadata: Key 8cc0f173c83b62dea3d1c8f9ba1b66dd is removed from metadata 2024.10.22 01:27:33.564622 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: be96416c80e5ffa6b79cd0c3c4783ab4, offset: 0, size: 247 2024.10.22 01:27:33.567962 [ 1476 ] {} CacheMetadata: Key 51159aee2a65a3f130c6bd1c00a80678 is removed from metadata 2024.10.22 01:27:33.568461 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key be96416c80e5ffa6b79cd0c3c4783ab4 for removal 2024.10.22 01:27:33.572464 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 28bd737fb46a9e9eaa947c5a2f5f9a41, offset: 0, size: 183 2024.10.22 01:27:33.576245 [ 1476 ] {} CacheMetadata: Key be96416c80e5ffa6b79cd0c3c4783ab4 is removed from metadata 2024.10.22 01:27:33.576246 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 28bd737fb46a9e9eaa947c5a2f5f9a41 for removal 2024.10.22 01:27:33.594869 [ 1476 ] {} CacheMetadata: Key 28bd737fb46a9e9eaa947c5a2f5f9a41 is removed from metadata 2024.10.22 01:27:33.598693 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fd2f9f5ddd2d3633f671869e814b9e52, offset: 0, size: 118836 2024.10.22 01:27:33.604611 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fd2f9f5ddd2d3633f671869e814b9e52 for removal 2024.10.22 01:27:33.611717 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: afba5b0cc46fbcfa148184c2bc0896d9, offset: 0, size: 145 2024.10.22 01:27:33.612062 [ 1476 ] {} CacheMetadata: Key fd2f9f5ddd2d3633f671869e814b9e52 is removed from metadata 2024.10.22 01:27:33.620909 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key afba5b0cc46fbcfa148184c2bc0896d9 for removal 2024.10.22 01:27:33.630264 [ 1476 ] {} CacheMetadata: Key afba5b0cc46fbcfa148184c2bc0896d9 is removed from metadata 2024.10.22 01:27:33.663893 [ 1476 ] {} CacheMetadata: Key fb402b09e31d3f437afe8fc0b2036476 is removed from metadata 2024.10.22 01:27:33.675301 [ 1476 ] {} CacheMetadata: Key b91435210b7cb0b0c1dbe58771e7fbfc is removed from metadata 2024.10.22 01:27:33.680842 [ 1476 ] {} CacheMetadata: Key 03db0679ed7d71d2269b634904b59277 is removed from metadata 2024.10.22 01:27:33.703099 [ 1476 ] {} CacheMetadata: Key 768a77359aff1a1c2b8b74a06bf7379c is removed from metadata 2024.10.22 01:27:33.711739 [ 1476 ] {} CacheMetadata: Key 93849e0cdd32fe20e88ab237bb094a24 is removed from metadata 2024.10.22 01:27:33.747965 [ 1476 ] {} CacheMetadata: Key fbfda3fd90fac27e047330ab965acbad is removed from metadata 2024.10.22 01:27:33.763425 [ 1476 ] {} CacheMetadata: Key a2af1fea912e8f5ef8fe797d3b12c04a is removed from metadata 2024.10.22 01:27:33.770448 [ 1476 ] {} CacheMetadata: Key 223cd40d7d867585116d1cfa6d5ed44e is removed from metadata 2024.10.22 01:27:33.783983 [ 1476 ] {} CacheMetadata: Key e0d74d11b537dcf9f2c36c9889a625d8 is removed from metadata 2024.10.22 01:27:33.803333 [ 1476 ] {} CacheMetadata: Key 61150faad25462f65eddfbfd3adaa3d4 is removed from metadata 2024.10.22 01:27:33.813751 [ 1476 ] {} CacheMetadata: Key b47a24986888735ede989bf81cc738b5 is removed from metadata 2024.10.22 01:27:33.817773 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_856_434} MergeTask::MergeProjectionsStage: Merge sorted 56369 rows, containing 13 columns (13 merged, 0 gathered) in 3.507161324 sec., 16072.542661285346 rows/sec., 2.56 MiB/sec. 2024.10.22 01:27:33.825642 [ 1476 ] {} CacheMetadata: Key 62d214f5657f5556ce72f60d6e57a9a3 is removed from metadata 2024.10.22 01:27:33.840915 [ 1476 ] {} CacheMetadata: Key 906e0757db98db05a1ada4d06c15bf0f is removed from metadata 2024.10.22 01:27:33.914008 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_856_434} MergedBlockOutputStream: filled checksums 202410_1_856_434 (state Temporary) 2024.10.22 01:27:34.362417 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_856_434} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_856_434 to 202410_1_856_434 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:34.365131 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_856_434} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_855_433, 202410_856_856_0] -> 202410_1_856_434 2024.10.22 01:27:34.393546 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 424 2024.10.22 01:27:34.411533 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:27:34.433227 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 190 entries to flush up to offset 56201 2024.10.22 01:27:34.646948 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_101_101_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:27:34.763008 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:34.768888 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.97 GiB. 2024.10.22 01:27:34.833428 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_406_406_0 (state Temporary) 2024.10.22 01:27:35.049656 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_406_406_0 to 202410_416_416_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:35.065657 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_451_90, 202410_452_452_0, 202410_453_453_0, 202410_454_454_0, 202410_455_455_0, 202410_456_456_0] 2024.10.22 01:27:35.068430 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_451_90, 202410_452_452_0, 202410_453_453_0, 202410_454_454_0, 202410_455_455_0, 202410_456_456_0] 2024.10.22 01:27:35.118500 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_451_90, 202410_452_452_0, 202410_453_453_0, 202410_454_454_0, 202410_455_455_0, 202410_456_456_0] 2024.10.22 01:27:35.159646 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 424 2024.10.22 01:27:35.182983 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column Referer 2024.10.22 01:27:35.205633 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column Referer 2024.10.22 01:27:35.219443 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column Referer 2024.10.22 01:27:35.242942 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column Referer 2024.10.22 01:27:35.256476 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 759653 2024.10.22 01:27:35.349349 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:35.354315 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.96 GiB. 2024.10.22 01:27:35.372273 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column Referer 2024.10.22 01:27:35.391875 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column Referer 2024.10.22 01:27:35.433220 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column Referer 2024.10.22 01:27:35.504586 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 12.82 MiB, will set to 1.68 GiB (RSS), difference: 22.29 MiB 2024.10.22 01:27:35.560279 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column Referer 2024.10.22 01:27:35.704900 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:27:35.722472 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:27:35.751347 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_849_849_0 (state Temporary) 2024.10.22 01:27:35.768025 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:27:35.793178 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:27:35.854767 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:27:35.861440 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kqt/cieukdiolxqgnecddjyxkttjkbzsh (LogSeriesLimiter: on interval from 2024-10-22 01:27:14 to 2024-10-22 01:27:35 accepted series 1 / 171 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:27:35.892689 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:27:35.937589 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:35.945597 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.96 GiB. 2024.10.22 01:27:36.017608 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:27:36.029596 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_849_849_0 to 202410_862_862_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:36.085341 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:27:36.087857 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56201 2024.10.22 01:27:36.111362 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_856_434 to 202410_857_857_0 2024.10.22 01:27:36.120684 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.02 MiB using storage policy from min volume index 0 2024.10.22 01:27:36.124379 [ 1170 ] {} DiskLocal: Reserved 2.02 MiB on local disk `default`, having unreserved 296.96 GiB. 2024.10.22 01:27:36.189021 [ 1183 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:27:36.224738 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_482_482_0 (state Temporary) 2024.10.22 01:27:36.225309 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:27:36.232437 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_857_435} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_856_434 to 202410_857_857_0 into Compact with storage Full 2024.10.22 01:27:36.270601 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:27:36.330235 [ 1183 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:27:36.358505 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_857_435} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:36.370102 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_482_482_0 to 202410_489_489_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:36.391283 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_759_367, 202410_760_760_0] 2024.10.22 01:27:36.401838 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_857_435} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_856_434, total 56369 rows starting from the beginning of the part 2024.10.22 01:27:36.434332 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 759653 2024.10.22 01:27:36.498216 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_759_367, 202410_760_760_0] 2024.10.22 01:27:36.510859 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_759_367, 202410_760_760_0] 2024.10.22 01:27:36.591657 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_857_435} MergeTreeSequentialSource: Reading 2 marks from part 202410_857_857_0, total 37 rows starting from the beginning of the part 2024.10.22 01:27:37.479017 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 119 entries to flush up to offset 58225 2024.10.22 01:27:37.483204 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_436_214, 202410_437_437_0] 2024.10.22 01:27:37.486458 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_436_214, 202410_437_437_0] 2024.10.22 01:27:37.564331 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 11.80 MiB, will set to 1.69 GiB (RSS), difference: -24.32 MiB 2024.10.22 01:27:37.611868 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_436_214, 202410_437_437_0] 2024.10.22 01:27:37.636389 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:37.642800 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.97 GiB. 2024.10.22 01:27:37.703327 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_493_493_0 (state Temporary) 2024.10.22 01:27:37.873276 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_493_493_0 to 202410_499_499_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:38.031341 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 58225 2024.10.22 01:27:39.008406 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 7770 2024.10.22 01:27:39.056827 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_857_435/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:27:39.360655 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:39.362911 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.97 GiB. 2024.10.22 01:27:39.409792 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_503_503_0 (state Temporary) 2024.10.22 01:27:39.420648 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 11.08 MiB, will set to 1.72 GiB (RSS), difference: 7.00 MiB 2024.10.22 01:27:39.438641 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_857_435} MergeTask::MergeProjectionsStage: Merge sorted 56406 rows, containing 13 columns (13 merged, 0 gathered) in 3.278648861 sec., 17204.038124044782 rows/sec., 2.74 MiB/sec. 2024.10.22 01:27:39.523244 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_857_435} MergedBlockOutputStream: filled checksums 202410_1_857_435 (state Temporary) 2024.10.22 01:27:39.563220 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_503_503_0 to 202410_516_516_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:39.608039 [ 1197 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_511_102 to 202410_516_516_0 2024.10.22 01:27:39.633176 [ 1197 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:39.645608 [ 1197 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.96 GiB. 2024.10.22 01:27:39.674655 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_511_102 to 202410_516_516_0 into Compact with storage Full 2024.10.22 01:27:39.721771 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:39.747253 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_511_102, total 7804 rows starting from the beginning of the part 2024.10.22 01:27:39.752445 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7770 2024.10.22 01:27:39.766641 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_512_512_0, total 24 rows starting from the beginning of the part 2024.10.22 01:27:39.783402 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_857_435} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_857_435 to 202410_1_857_435 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:39.788005 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_857_435} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_856_434, 202410_857_857_0] -> 202410_1_857_435 2024.10.22 01:27:39.803921 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_513_513_0, total 18 rows starting from the beginning of the part 2024.10.22 01:27:39.822544 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:27:39.829325 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_514_514_0, total 13 rows starting from the beginning of the part 2024.10.22 01:27:39.857782 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_515_515_0, total 9 rows starting from the beginning of the part 2024.10.22 01:27:39.987841 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_516_516_0, total 8 rows starting from the beginning of the part 2024.10.22 01:27:40.122258 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 56288 2024.10.22 01:27:40.257246 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:27:40.259288 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:27:40.499851 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_760_368, 202410_761_761_0] 2024.10.22 01:27:40.510585 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_760_368, 202410_761_761_0] 2024.10.22 01:27:40.522229 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_760_368, 202410_761_761_0] 2024.10.22 01:27:40.678922 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:40.686833 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.96 GiB. 2024.10.22 01:27:40.926449 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_850_850_0 (state Temporary) 2024.10.22 01:27:40.964274 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_101_101_0 to 202410_104_104_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:41.085665 [ 1199 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_1_99_20 to 202410_104_104_0 2024.10.22 01:27:41.139782 [ 1199 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 2.02 MiB using storage policy from min volume index 0 2024.10.22 01:27:41.145301 [ 1199 ] {} DiskLocal: Reserved 2.02 MiB on local disk `default`, having unreserved 296.96 GiB. 2024.10.22 01:27:41.160749 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_99_20 to 202410_104_104_0 into Compact with storage Full 2024.10.22 01:27:41.183654 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_850_850_0 to 202410_863_863_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:41.219414 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} MergeTask::MergeProjectionsStage: Merge sorted 7876 rows, containing 12 columns (12 merged, 0 gathered) in 1.551295979 sec., 5077.045326370951 rows/sec., 806.48 KiB/sec. 2024.10.22 01:27:41.343955 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_857_435 to 202410_858_858_0 2024.10.22 01:27:41.353405 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.02 MiB using storage policy from min volume index 0 2024.10.22 01:27:41.360313 [ 1213 ] {} DiskLocal: Reserved 2.02 MiB on local disk `default`, having unreserved 296.96 GiB. 2024.10.22 01:27:41.375126 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} MergedBlockOutputStream: filled checksums 202410_1_516_103 (state Temporary) 2024.10.22 01:27:41.387337 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56288 2024.10.22 01:27:41.487578 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_858_436} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_857_435 to 202410_858_858_0 into Compact with storage Full 2024.10.22 01:27:41.532371 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_858_436} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:41.550457 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_858_436} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_857_435, total 56406 rows starting from the beginning of the part 2024.10.22 01:27:41.571379 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_858_436} MergeTreeSequentialSource: Reading 2 marks from part 202410_858_858_0, total 54 rows starting from the beginning of the part 2024.10.22 01:27:41.572232 [ 1209 ] {} DatabaseCatalog: Execution took 218 ms. 2024.10.22 01:27:41.606466 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 9.54 MiB, will set to 1.72 GiB (RSS), difference: 2.48 MiB 2024.10.22 01:27:41.854384 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_516_103 to 202410_1_516_103 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:41.858205 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_516_103} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_511_102, 202410_516_516_0] -> 202410_1_516_103 2024.10.22 01:27:42.000942 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.48 MiB. 2024.10.22 01:27:42.570245 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_484_98/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:27:42.661440 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 425 2024.10.22 01:27:43.060759 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:43.066323 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.96 GiB. 2024.10.22 01:27:43.144709 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_407_407_0 (state Temporary) 2024.10.22 01:27:43.443453 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 761193 2024.10.22 01:27:43.459611 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_407_407_0 to 202410_417_417_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:43.563544 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 15.57 MiB, will set to 1.73 GiB (RSS), difference: 6.61 MiB 2024.10.22 01:27:43.568822 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 425 2024.10.22 01:27:43.888870 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4050 2024.10.22 01:27:44.122135 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_858_436} MergeTask::MergeProjectionsStage: Merge sorted 56460 rows, containing 13 columns (13 merged, 0 gathered) in 2.753885977 sec., 20501.93815994728 rows/sec., 3.27 MiB/sec. 2024.10.22 01:27:44.386865 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:44.393472 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.96 GiB. 2024.10.22 01:27:44.759685 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_858_436} MergedBlockOutputStream: filled checksums 202410_1_858_436 (state Temporary) 2024.10.22 01:27:44.785747 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_483_483_0 (state Temporary) 2024.10.22 01:27:44.908199 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_483_483_0 to 202410_490_490_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:44.990263 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 761193 2024.10.22 01:27:45.064573 [ 1241 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_485_485_0 to 202410_490_490_0 2024.10.22 01:27:45.151121 [ 1241 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:45.156212 [ 1241 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:27:45.168113 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_485_485_0 to 202410_490_490_0 into Compact with storage Full 2024.10.22 01:27:45.190491 [ 1698 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qcw/llbtnekhbnqrevmcmtebevgjrebam, size 251 2024.10.22 01:27:45.193722 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_858_436} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_858_436 to 202410_1_858_436 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:45.210678 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_858_436} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_857_435, 202410_858_858_0] -> 202410_1_858_436 2024.10.22 01:27:45.234193 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:45.248263 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_485_485_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:27:45.264653 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_486_486_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:27:45.276933 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:27:45.278898 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_487_487_0, total 1927 rows starting from the beginning of the part 2024.10.22 01:27:45.298884 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_488_488_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:27:45.313698 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_489_489_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:27:45.333050 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_490_490_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:27:45.509822 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 56350 2024.10.22 01:27:45.808530 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 136 entries to flush up to offset 58361 2024.10.22 01:27:45.949411 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:45.955802 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:27:45.873783 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.97 GiB, free memory in arenas 15.99 MiB, will set to 1.72 GiB (RSS), difference: 100.66 MiB 2024.10.22 01:27:46.084719 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:46.089645 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:27:46.137839 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_851_851_0 (state Temporary) 2024.10.22 01:27:46.212352 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_851_851_0 to 202410_864_864_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:46.256657 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_494_494_0 (state Temporary) 2024.10.22 01:27:46.274525 [ 1203 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_858_436 to 202410_859_859_0 2024.10.22 01:27:46.279389 [ 1203 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.02 MiB using storage policy from min volume index 0 2024.10.22 01:27:46.282295 [ 1203 ] {} DiskLocal: Reserved 2.02 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:27:46.402611 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_494_494_0 to 202410_500_500_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:46.412848 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_859_437} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_858_436 to 202410_859_859_0 into Compact with storage Full 2024.10.22 01:27:46.442479 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56350 2024.10.22 01:27:46.479283 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 58361 2024.10.22 01:27:46.682566 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_859_437} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:46.707156 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_761_369, 202410_762_762_0] 2024.10.22 01:27:46.718098 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_761_369, 202410_762_762_0] 2024.10.22 01:27:46.711862 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_859_437} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_858_436, total 56460 rows starting from the beginning of the part 2024.10.22 01:27:46.728921 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_761_369, 202410_762_762_0] 2024.10.22 01:27:46.757195 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_859_437} MergeTreeSequentialSource: Reading 2 marks from part 202410_859_859_0, total 83 rows starting from the beginning of the part 2024.10.22 01:27:47.155208 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} MergeTask::MergeProjectionsStage: Merge sorted 8857 rows, containing 4 columns (4 merged, 0 gathered) in 1.990379134 sec., 4449.905974546858 rows/sec., 69.53 KiB/sec. 2024.10.22 01:27:47.224646 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} MergedBlockOutputStream: filled checksums 202410_485_490_1 (state Temporary) 2024.10.22 01:27:47.254976 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 7790 2024.10.22 01:27:47.337050 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_485_490_1 to 202410_485_490_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:47.351016 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_485_490_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_485_485_0, 202410_490_490_0] -> 202410_485_490_1 2024.10.22 01:27:47.484393 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:47.486362 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:27:47.507244 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:27:47.555604 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_504_504_0 (state Temporary) 2024.10.22 01:27:47.711154 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_504_504_0 to 202410_517_517_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:47.765591 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7790 2024.10.22 01:27:48.559631 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.97 GiB, free memory in arenas 16.65 MiB, will set to 1.71 GiB (RSS), difference: 162.78 MiB 2024.10.22 01:27:48.566972 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_437_215, 202410_438_438_0] 2024.10.22 01:27:48.568273 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_437_215, 202410_438_438_0] 2024.10.22 01:27:48.581767 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_437_215, 202410_438_438_0] 2024.10.22 01:27:48.966448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_3_3_0 (state Temporary) 2024.10.22 01:27:48.978946 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/onc/gugejquaqjcorxuioswwopeqwkdxf (LogSeriesLimiter: on interval from 2024-10-22 01:27:35 to 2024-10-22 01:27:48 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:27:49.071488 [ 1681 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fml/zjvoglossxyluvskmevegjjdbfuwb, size 549 2024.10.22 01:27:49.116461 [ 1679 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/onc/gugejquaqjcorxuioswwopeqwkdxf, size 4 2024.10.22 01:27:50.256898 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:27:50.259216 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:27:50.260263 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_859_437} MergeTask::MergeProjectionsStage: Merge sorted 56543 rows, containing 13 columns (13 merged, 0 gathered) in 3.969622647 sec., 14243.923170564278 rows/sec., 2.27 MiB/sec. 2024.10.22 01:27:50.262784 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:50.334507 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_859_437} MergedBlockOutputStream: filled checksums 202410_1_859_437 (state Temporary) 2024.10.22 01:27:50.457664 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 15.54 MiB, will set to 1.69 GiB (RSS), difference: 62.11 MiB 2024.10.22 01:27:50.458964 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_762_370, 202410_763_763_0] 2024.10.22 01:27:50.459052 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_859_437} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_859_437 to 202410_1_859_437 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:50.461282 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 56397 2024.10.22 01:27:50.463948 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_762_370, 202410_763_763_0] 2024.10.22 01:27:50.473583 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_859_437} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_858_436, 202410_859_859_0] -> 202410_1_859_437 2024.10.22 01:27:50.473821 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_762_370, 202410_763_763_0] 2024.10.22 01:27:50.514435 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:27:50.720769 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:50.723883 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:27:50.806943 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_852_852_0 (state Temporary) 2024.10.22 01:27:50.855725 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_852_852_0 to 202410_865_865_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:50.884856 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_859_437 to 202410_860_860_0 2024.10.22 01:27:50.890652 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.03 MiB using storage policy from min volume index 0 2024.10.22 01:27:50.894263 [ 1171 ] {} DiskLocal: Reserved 2.03 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:27:51.014246 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56397 2024.10.22 01:27:51.059346 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_860_438} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_859_437 to 202410_860_860_0 into Compact with storage Full 2024.10.22 01:27:51.070841 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 426 2024.10.22 01:27:51.262621 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_860_438} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:51.270901 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_860_438} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_859_437, total 56543 rows starting from the beginning of the part 2024.10.22 01:27:51.289597 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_860_438} MergeTreeSequentialSource: Reading 2 marks from part 202410_860_860_0, total 253 rows starting from the beginning of the part 2024.10.22 01:27:51.337014 [ 1247 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:27:51.372547 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_99_20, total 3899 rows starting from the beginning of the part 2024.10.22 01:27:51.376627 [ 1247 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:27:51.439307 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:51.452691 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:27:51.522114 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_408_408_0 (state Temporary) 2024.10.22 01:27:51.534278 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 4090 2024.10.22 01:27:51.669049 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_408_408_0 to 202410_418_418_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:51.696461 [ 1255 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_413_85 to 202410_418_418_0 2024.10.22 01:27:51.707494 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_484_98} MergeTask::MergeProjectionsStage: Merge sorted 765926 rows, containing 4 columns (4 merged, 0 gathered) in 23.482758344 sec., 32616.526081813518 rows/sec., 489.19 KiB/sec. 2024.10.22 01:27:51.711530 [ 1255 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:51.727266 [ 1255 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:27:51.760241 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 426 2024.10.22 01:27:51.760694 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_413_85 to 202410_418_418_0 into Compact with storage Full 2024.10.22 01:27:51.824977 [ 1246 ] {} DatabaseCatalog: Execution took 251 ms. 2024.10.22 01:27:51.825169 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_484_98} MergedBlockOutputStream: filled checksums 202410_1_484_98 (state Temporary) 2024.10.22 01:27:51.966723 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:51.986080 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_413_85, total 457 rows starting from the beginning of the part 2024.10.22 01:27:51.999791 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 762733 2024.10.22 01:27:52.121683 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_414_414_0, total 1 rows starting from the beginning of the part 2024.10.22 01:27:52.209457 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_415_415_0, total 1 rows starting from the beginning of the part 2024.10.22 01:27:52.351447 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_416_416_0, total 1 rows starting from the beginning of the part 2024.10.22 01:27:52.560680 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_417_417_0, total 1 rows starting from the beginning of the part 2024.10.22 01:27:52.662921 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 15.72 MiB, will set to 1.68 GiB (RSS), difference: 6.49 MiB 2024.10.22 01:27:52.804681 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_418_418_0, total 1 rows starting from the beginning of the part 2024.10.22 01:27:53.172744 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:53.186267 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:27:53.194263 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/aht/jeocexuhoybagwdncmlgomzzhrcmn. 2024.10.22 01:27:53.440898 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_484_484_0 (state Temporary) 2024.10.22 01:27:53.481608 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_484_484_0 to 202410_491_491_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:53.531482 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_484_98/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:27:53.534233 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 762733 2024.10.22 01:27:53.983520 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_100_100_0, total 43 rows starting from the beginning of the part 2024.10.22 01:27:54.054256 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 193 entries to flush up to offset 58554 2024.10.22 01:27:54.459246 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:54.462926 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:27:54.730307 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_495_495_0 (state Temporary) 2024.10.22 01:27:54.869616 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 10.41 MiB, will set to 1.69 GiB (RSS), difference: 5.34 MiB 2024.10.22 01:27:54.878299 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} MergeTask::MergeProjectionsStage: Merge sorted 462 rows, containing 32 columns (32 merged, 0 gathered) in 3.124052216 sec., 147.88485212694025 rows/sec., 13.76 KiB/sec. 2024.10.22 01:27:54.909141 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_495_495_0 to 202410_501_501_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:54.955033 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 58554 2024.10.22 01:27:54.964102 [ 1252 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_461_496_7 to 202410_501_501_0 2024.10.22 01:27:54.984815 [ 1252 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:55.029531 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 56461 2024.10.22 01:27:55.097391 [ 1252 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:27:55.160117 [ 1252 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 270 ms. 2024.10.22 01:27:55.161070 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} MergedBlockOutputStream: filled checksums 202410_1_418_86 (state Temporary) 2024.10.22 01:27:55.162138 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_484_98} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_484_98 to 202410_1_484_98 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:55.175333 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_461_496_7 to 202410_501_501_0 into Compact with storage Full 2024.10.22 01:27:55.178974 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_860_438} MergeTask::MergeProjectionsStage: Merge sorted 56796 rows, containing 13 columns (13 merged, 0 gathered) in 4.126810029 sec., 13762.68827517672 rows/sec., 2.19 MiB/sec. 2024.10.22 01:27:55.182083 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_484_98} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_483_97, 202410_484_484_0] -> 202410_1_484_98 2024.10.22 01:27:55.178781 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_364_75, 202410_365_365_0, 202410_366_366_0, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0] 2024.10.22 01:27:55.188978 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_364_75, 202410_365_365_0, 202410_366_366_0, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0] 2024.10.22 01:27:55.258308 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:27:55.268902 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 7808 2024.10.22 01:27:55.330149 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_364_75, 202410_365_365_0, 202410_366_366_0, 202410_367_367_0, 202410_368_368_0, 202410_369_369_0] 2024.10.22 01:27:55.329779 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:55.337315 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:27:55.377006 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_860_438} MergedBlockOutputStream: filled checksums 202410_1_860_438 (state Temporary) 2024.10.22 01:27:55.412820 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_853_853_0 (state Temporary) 2024.10.22 01:27:55.444563 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:55.467492 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_496_7, total 5771 rows starting from the beginning of the part 2024.10.22 01:27:55.478135 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_853_853_0 to 202410_866_866_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:55.502348 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_497_497_0, total 68 rows starting from the beginning of the part 2024.10.22 01:27:55.529358 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_861_861_0 to 202410_866_866_0 2024.10.22 01:27:55.536127 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_498_498_0, total 99 rows starting from the beginning of the part 2024.10.22 01:27:55.554266 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_860_438} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_860_438 to 202410_1_860_438 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:55.556218 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:55.558413 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_860_438} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_859_437, 202410_860_860_0] -> 202410_1_860_438 2024.10.22 01:27:55.563423 [ 1157 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:27:55.582583 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_861_861_0 to 202410_866_866_0 into Compact with storage Full 2024.10.22 01:27:55.608322 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:27:55.613775 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:27:55.656222 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:27:55.665636 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_499_499_0, total 119 rows starting from the beginning of the part 2024.10.22 01:27:55.674058 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_861_861_0, total 46 rows starting from the beginning of the part 2024.10.22 01:27:55.682355 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56461 2024.10.22 01:27:55.683398 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.84 MiB. 2024.10.22 01:27:55.802042 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_500_500_0, total 136 rows starting from the beginning of the part 2024.10.22 01:27:55.806068 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_862_862_0, total 190 rows starting from the beginning of the part 2024.10.22 01:27:55.867490 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_505_505_0 (state Temporary) 2024.10.22 01:27:55.883461 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_418_86 to 202410_1_418_86 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:55.886846 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_418_86} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_413_85, 202410_418_418_0] -> 202410_1_418_86 2024.10.22 01:27:55.944207 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_505_505_0 to 202410_518_518_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:55.954386 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(af6cd34767f5df4976ea7ce243d474ec) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: af6cd34767f5df4976ea7ce243d474ec, state: DOWNLOADING, downloaded size: 16777216, reserved size: 16777216, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 16777216, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:27:55.956965 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 16777216, file segment info: File segment: [0, 33554431], key: af6cd34767f5df4976ea7ce243d474ec, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 16777216, reserved size: 16777216, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 16777216, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:27:55.961529 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:27:55.981130 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7808 2024.10.22 01:27:56.004141 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_501_501_0, total 193 rows starting from the beginning of the part 2024.10.22 01:27:56.007047 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:27:56.065444 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_863_863_0, total 87 rows starting from the beginning of the part 2024.10.22 01:27:56.158098 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_864_864_0, total 62 rows starting from the beginning of the part 2024.10.22 01:27:56.226318 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_865_865_0, total 47 rows starting from the beginning of the part 2024.10.22 01:27:56.381552 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_866_866_0, total 64 rows starting from the beginning of the part 2024.10.22 01:27:56.440001 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_101_101_0, total 43 rows starting from the beginning of the part 2024.10.22 01:27:56.611193 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 6.14 MiB, will set to 1.68 GiB (RSS), difference: -20.50 MiB 2024.10.22 01:27:56.637413 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_438_216, 202410_439_439_0] 2024.10.22 01:27:56.747245 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_438_216, 202410_439_439_0] 2024.10.22 01:27:56.901615 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_763_371, 202410_764_764_0] 2024.10.22 01:27:56.906270 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_763_371, 202410_764_764_0] 2024.10.22 01:27:56.921238 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_763_371, 202410_764_764_0] 2024.10.22 01:27:56.973920 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_438_216, 202410_439_439_0] 2024.10.22 01:27:57.164844 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} MergeTask::MergeProjectionsStage: Merge sorted 6386 rows, containing 13 columns (13 merged, 0 gathered) in 2.039324522 sec., 3131.429025203474 rows/sec., 1.36 MiB/sec. 2024.10.22 01:27:57.366360 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} MergedBlockOutputStream: filled checksums 202410_461_501_8 (state Temporary) 2024.10.22 01:27:57.680408 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:27:57.698362 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:27:57.723573 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:27:57.755485 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:27:57.782632 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:27:57.847842 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:27:57.868417 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} MergeTask::MergeProjectionsStage: Merge sorted 496 rows, containing 13 columns (13 merged, 0 gathered) in 2.29389127 sec., 216.2264648228074 rows/sec., 33.20 KiB/sec. 2024.10.22 01:27:57.869486 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:27:57.888986 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_461_501_8 to 202410_461_501_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:57.905343 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_501_8} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_461_496_7, 202410_501_501_0] -> 202410_461_501_8 2024.10.22 01:27:57.910014 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:27:57.920656 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} MergedBlockOutputStream: filled checksums 202410_861_866_1 (state Temporary) 2024.10.22 01:27:57.950220 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:27:57.953579 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.29 MiB. 2024.10.22 01:27:58.019724 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/psr/ndmijurdyagcxvfkanqwxykgepnvn (LogSeriesLimiter: on interval from 2024-10-22 01:27:48 to 2024-10-22 01:27:58 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:27:58.085451 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_861_866_1 to 202410_861_866_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:27:58.093567 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_861_866_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_861_861_0, 202410_866_866_0] -> 202410_861_866_1 2024.10.22 01:27:58.144470 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:27:58.413276 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_102_102_0, total 46 rows starting from the beginning of the part 2024.10.22 01:27:58.783145 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 5.45 MiB, will set to 1.68 GiB (RSS), difference: 7.35 MiB 2024.10.22 01:27:59.223269 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_430_430_0, 202410_431_431_0, 202410_432_432_0, 202410_433_433_0, 202410_434_434_0, 202410_435_435_0] 2024.10.22 01:27:59.227105 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_430_430_0, 202410_431_431_0, 202410_432_432_0, 202410_433_433_0, 202410_434_434_0, 202410_435_435_0] 2024.10.22 01:27:59.261002 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_430_430_0, 202410_431_431_0, 202410_432_432_0, 202410_433_433_0, 202410_434_434_0, 202410_435_435_0] 2024.10.22 01:27:59.714630 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 56545 2024.10.22 01:28:00.257654 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:28:00.259469 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:28:00.268557 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:00.271574 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:28:00.332896 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_854_854_0 (state Temporary) 2024.10.22 01:28:00.511941 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_854_854_0 to 202410_867_867_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:00.550635 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 764273 2024.10.22 01:28:00.563228 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 3.38 MiB, will set to 1.69 GiB (RSS), difference: -8.23 MiB 2024.10.22 01:28:00.580430 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56545 2024.10.22 01:28:00.617670 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_103_103_0, total 41 rows starting from the beginning of the part 2024.10.22 01:28:01.252452 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_764_372, 202410_765_765_0] 2024.10.22 01:28:01.256312 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_764_372, 202410_765_765_0] 2024.10.22 01:28:01.267657 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_764_372, 202410_765_765_0] 2024.10.22 01:28:01.381534 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:01.385159 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:28:01.658749 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_485_485_0 (state Temporary) 2024.10.22 01:28:01.759556 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_485_485_0 to 202410_492_492_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:01.784932 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 764273 2024.10.22 01:28:01.803965 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_104_104_0, total 40 rows starting from the beginning of the part 2024.10.22 01:28:02.498960 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 4.04 MiB, will set to 1.68 GiB (RSS), difference: 11.06 MiB 2024.10.22 01:28:02.533726 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 58669 2024.10.22 01:28:02.851580 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:02.855536 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:28:02.900907 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_496_496_0 (state Temporary) 2024.10.22 01:28:03.161163 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_496_496_0 to 202410_502_502_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:03.199343 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 58669 2024.10.22 01:28:03.482943 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 29 entries to flush up to offset 7837 2024.10.22 01:28:03.574919 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_428_86, 202410_429_429_0] 2024.10.22 01:28:03.578278 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_428_86, 202410_429_429_0] 2024.10.22 01:28:03.591166 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_428_86, 202410_429_429_0] 2024.10.22 01:28:03.712027 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:03.716243 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:28:03.759533 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_506_506_0 (state Temporary) 2024.10.22 01:28:04.029405 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_506_506_0 to 202410_519_519_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:04.084103 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7837 2024.10.22 01:28:04.611581 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 56581 2024.10.22 01:28:04.681347 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_439_217, 202410_440_440_0] 2024.10.22 01:28:04.686344 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_439_217, 202410_440_440_0] 2024.10.22 01:28:04.708654 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_439_217, 202410_440_440_0] 2024.10.22 01:28:04.881521 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:04.885991 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:28:05.060699 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_765_373, 202410_766_766_0] 2024.10.22 01:28:05.062422 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_765_373, 202410_766_766_0] 2024.10.22 01:28:05.156144 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_855_855_0 (state Temporary) 2024.10.22 01:28:05.169619 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_765_373, 202410_766_766_0] 2024.10.22 01:28:05.411411 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_855_855_0 to 202410_868_868_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:05.655015 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56581 2024.10.22 01:28:06.218972 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/kqt/cieukdiolxqgnecddjyxkttjkbzsh, upload id 3eb7eec4-f9e7-425a-b396-a3c05fc93ab5 2024.10.22 01:28:06.384169 [ 1152 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:28:06.411977 [ 1152 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:28:06.506598 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 2.63 MiB, will set to 1.69 GiB (RSS), difference: -9.32 MiB 2024.10.22 01:28:06.762978 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 427 2024.10.22 01:28:07.089958 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:07.095107 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:28:07.276072 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_409_409_0 (state Temporary) 2024.10.22 01:28:07.452550 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_409_409_0 to 202410_419_419_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:07.542373 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 427 2024.10.22 01:28:07.777520 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_104_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:28:08.503044 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:08.510256 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.80 GiB. 2024.10.22 01:28:08.597374 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 2.01 MiB, will set to 1.65 GiB (RSS), difference: -2.82 MiB 2024.10.22 01:28:08.792652 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 765813 2024.10.22 01:28:09.659531 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 56608 2024.10.22 01:28:09.830527 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:09.903945 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.80 GiB. 2024.10.22 01:28:09.907135 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:09.911860 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.80 GiB. 2024.10.22 01:28:09.976315 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_856_856_0 (state Temporary) 2024.10.22 01:28:10.063568 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_766_374, 202410_767_767_0] 2024.10.22 01:28:10.066986 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_766_374, 202410_767_767_0] 2024.10.22 01:28:10.078069 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_766_374, 202410_767_767_0] 2024.10.22 01:28:10.116657 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_856_856_0 to 202410_869_869_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:10.174988 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56608 2024.10.22 01:28:10.258311 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:28:10.263933 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:28:10.351563 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_486_486_0 (state Temporary) 2024.10.22 01:28:10.360994 [ 1171 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 01:28:10.525789 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_486_486_0 to 202410_493_493_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:10.577553 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 765813 2024.10.22 01:28:10.608234 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 2.16 MiB, will set to 1.65 GiB (RSS), difference: 1.90 MiB 2024.10.22 01:28:10.717381 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 72 entries to flush up to offset 58741 2024.10.22 01:28:11.017330 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:11.022377 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.80 GiB. 2024.10.22 01:28:11.224344 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_497_497_0 (state Temporary) 2024.10.22 01:28:11.308576 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_497_497_0 to 202410_503_503_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:11.586717 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 7843 2024.10.22 01:28:11.587418 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 58741 2024.10.22 01:28:11.623881 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_102_102_0 (state Temporary) 2024.10.22 01:28:11.735569 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:11.757128 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.80 GiB. 2024.10.22 01:28:11.796744 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_507_507_0 (state Temporary) 2024.10.22 01:28:11.962153 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_507_507_0 to 202410_520_520_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:12.027280 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7843 2024.10.22 01:28:12.580018 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 3.57 MiB, will set to 1.66 GiB (RSS), difference: 1.90 MiB 2024.10.22 01:28:14.176926 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 56640 2024.10.22 01:28:14.486846 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_440_218, 202410_441_441_0] 2024.10.22 01:28:14.490639 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_440_218, 202410_441_441_0] 2024.10.22 01:28:14.509770 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:14.516913 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.80 GiB. 2024.10.22 01:28:14.559187 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 3.77 MiB, will set to 1.66 GiB (RSS), difference: 8.56 MiB 2024.10.22 01:28:14.561547 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_857_857_0 (state Temporary) 2024.10.22 01:28:14.624658 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_857_857_0 to 202410_870_870_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:14.650918 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_440_218, 202410_441_441_0] 2024.10.22 01:28:14.676582 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56640 2024.10.22 01:28:15.044752 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 428 2024.10.22 01:28:15.308093 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:15.313431 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.80 GiB. 2024.10.22 01:28:15.373059 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_410_410_0 (state Temporary) 2024.10.22 01:28:15.541071 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_456_91, 202410_457_457_0, 202410_458_458_0, 202410_459_459_0, 202410_460_460_0, 202410_461_461_0] 2024.10.22 01:28:15.548169 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_456_91, 202410_457_457_0, 202410_458_458_0, 202410_459_459_0, 202410_460_460_0, 202410_461_461_0] 2024.10.22 01:28:15.575298 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_456_91, 202410_457_457_0, 202410_458_458_0, 202410_459_459_0, 202410_460_460_0, 202410_461_461_0] 2024.10.22 01:28:15.576127 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_410_410_0 to 202410_420_420_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:15.659028 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 428 2024.10.22 01:28:15.709228 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_767_375, 202410_768_768_0] 2024.10.22 01:28:15.716313 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_767_375, 202410_768_768_0] 2024.10.22 01:28:15.732628 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_767_375, 202410_768_768_0] 2024.10.22 01:28:16.022917 [ 1701 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/psr/ndmijurdyagcxvfkanqwxykgepnvn, size 2018645 2024.10.22 01:28:16.231582 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:28:16.246150 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:28:16.274294 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:28:16.293578 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:28:16.316100 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:28:16.342717 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:28:16.361742 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:28:16.373126 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:28:16.419088 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:28:16.471016 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 5.88 MiB, will set to 1.67 GiB (RSS), difference: -12.37 MiB 2024.10.22 01:28:16.476133 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ikt/kqdmqhwiixhppikmzqlbkeaiqeuvl (LogSeriesLimiter: on interval from 2024-10-22 01:27:58 to 2024-10-22 01:28:16 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:28:17.582708 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 767738 2024.10.22 01:28:18.431729 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 5.14 MiB, will set to 1.66 GiB (RSS), difference: -5.04 MiB 2024.10.22 01:28:18.677777 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:18.680949 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 56676 2024.10.22 01:28:18.682487 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 01:28:18.836673 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:18.840397 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_487_487_0 (state Temporary) 2024.10.22 01:28:18.844800 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 01:28:18.892950 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_858_858_0 (state Temporary) 2024.10.22 01:28:18.934471 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_858_858_0 to 202410_871_871_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:18.970305 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_860_438 to 202410_861_866_1 2024.10.22 01:28:18.975228 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.05 MiB using storage policy from min volume index 0 2024.10.22 01:28:18.977071 [ 1187 ] {} DiskLocal: Reserved 2.05 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 01:28:19.011469 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_866_439} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_860_438 to 202410_861_866_1 into Compact with storage Full 2024.10.22 01:28:19.058225 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_866_439} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:19.084172 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_866_439} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_860_438, total 56796 rows starting from the beginning of the part 2024.10.22 01:28:19.084454 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56676 2024.10.22 01:28:19.108384 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 87 entries to flush up to offset 58828 2024.10.22 01:28:19.120327 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_487_487_0 to 202410_494_494_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:19.160307 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_866_439} MergeTreeSequentialSource: Reading 2 marks from part 202410_861_866_1, total 496 rows starting from the beginning of the part 2024.10.22 01:28:19.161632 [ 1201 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_484_98 to 202410_485_490_1 2024.10.22 01:28:19.167663 [ 1201 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:19.167813 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 767738 2024.10.22 01:28:19.171739 [ 1201 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 01:28:19.408120 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_490_99} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_484_98 to 202410_485_490_1 into Wide with storage Full 2024.10.22 01:28:19.475803 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_490_99} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:19.484218 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_490_99} MergeTreeSequentialSource: Reading 95 marks from part 202410_1_484_98, total 765926 rows starting from the beginning of the part 2024.10.22 01:28:19.500419 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:19.510490 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 01:28:19.529565 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 7848 2024.10.22 01:28:19.551381 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_490_99} MergeTreeSequentialSource: Reading 3 marks from part 202410_485_490_1, total 8857 rows starting from the beginning of the part 2024.10.22 01:28:19.572470 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_498_498_0 (state Temporary) 2024.10.22 01:28:19.629093 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_498_498_0 to 202410_504_504_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:19.745188 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:19.781682 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 58828 2024.10.22 01:28:19.848796 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 01:28:19.973335 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_508_508_0 (state Temporary) 2024.10.22 01:28:20.204681 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_508_508_0 to 202410_521_521_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:20.237573 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_516_103 to 202410_521_521_0 2024.10.22 01:28:20.258699 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:28:20.262698 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:28:20.263073 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:20.268026 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 01:28:20.269544 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7848 2024.10.22 01:28:20.300372 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_516_103 to 202410_521_521_0 into Compact with storage Full 2024.10.22 01:28:20.306297 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:28:20.329184 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:28:20.336802 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_768_376, 202410_769_769_0] 2024.10.22 01:28:20.339096 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_768_376, 202410_769_769_0] 2024.10.22 01:28:20.353366 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_768_376, 202410_769_769_0] 2024.10.22 01:28:20.376065 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:28:20.452210 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:28:20.490800 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:28:20.495780 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:20.523499 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:28:20.659227 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:28:20.718727 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:28:20.728195 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_516_103, total 7876 rows starting from the beginning of the part 2024.10.22 01:28:20.746866 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 5.98 MiB, will set to 1.67 GiB (RSS), difference: 3.00 MiB 2024.10.22 01:28:20.918832 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_517_517_0, total 20 rows starting from the beginning of the part 2024.10.22 01:28:20.964534 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_518_518_0, total 18 rows starting from the beginning of the part 2024.10.22 01:28:21.020573 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_519_519_0, total 29 rows starting from the beginning of the part 2024.10.22 01:28:21.120956 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_520_520_0, total 6 rows starting from the beginning of the part 2024.10.22 01:28:21.195853 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_521_521_0, total 5 rows starting from the beginning of the part 2024.10.22 01:28:21.417480 [ 1227 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:28:21.439548 [ 1216 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 297 ms. 2024.10.22 01:28:21.460156 [ 1227 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:28:21.778867 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_490_99/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:28:22.668266 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} MergeTask::MergeProjectionsStage: Merge sorted 7954 rows, containing 12 columns (12 merged, 0 gathered) in 2.383447517 sec., 3337.182775482948 rows/sec., 530.16 KiB/sec. 2024.10.22 01:28:22.757045 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_441_219, 202410_442_442_0] 2024.10.22 01:28:22.761168 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_441_219, 202410_442_442_0] 2024.10.22 01:28:22.775373 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_441_219, 202410_442_442_0] 2024.10.22 01:28:22.784081 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_102_102_0 to 202410_105_105_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:22.790361 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} MergedBlockOutputStream: filled checksums 202410_1_521_104 (state Temporary) 2024.10.22 01:28:22.865977 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_104_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:28:22.866750 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_866_439} MergeTask::MergeProjectionsStage: Merge sorted 57292 rows, containing 13 columns (13 merged, 0 gathered) in 3.86097467 sec., 14838.740187850028 rows/sec., 2.36 MiB/sec. 2024.10.22 01:28:22.959017 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.185038475 sec. 2024.10.22 01:28:23.064900 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_866_439} MergedBlockOutputStream: filled checksums 202410_1_866_439 (state Temporary) 2024.10.22 01:28:23.092619 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 56752 2024.10.22 01:28:23.154961 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_866_439} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_866_439 to 202410_1_866_439 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:23.159470 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_866_439} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_860_438, 202410_861_866_1] -> 202410_1_866_439 2024.10.22 01:28:23.160887 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 429 2024.10.22 01:28:23.185423 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.90 MiB. 2024.10.22 01:28:23.293560 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_521_104 to 202410_1_521_104 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:23.299425 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_521_104} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_516_103, 202410_521_521_0] -> 202410_1_521_104 2024.10.22 01:28:23.348985 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.48 MiB. 2024.10.22 01:28:23.354749 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:23.360532 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:23.522724 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_859_859_0 (state Temporary) 2024.10.22 01:28:23.550661 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:23.555766 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:23.574517 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_859_859_0 to 202410_872_872_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:23.711062 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_866_439 to 202410_867_867_0 2024.10.22 01:28:23.719623 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.05 MiB using storage policy from min volume index 0 2024.10.22 01:28:23.722380 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_411_411_0 (state Temporary) 2024.10.22 01:28:23.722754 [ 1242 ] {} DiskLocal: Reserved 2.05 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:23.734539 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56752 2024.10.22 01:28:23.742850 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_867_440} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_866_439 to 202410_867_867_0 into Compact with storage Full 2024.10.22 01:28:23.798503 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_411_411_0 to 202410_421_421_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:23.829761 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_867_440} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:23.865831 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_867_440} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_866_439, total 57292 rows starting from the beginning of the part 2024.10.22 01:28:23.955646 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_867_440} MergeTreeSequentialSource: Reading 2 marks from part 202410_867_867_0, total 84 rows starting from the beginning of the part 2024.10.22 01:28:23.983146 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 429 2024.10.22 01:28:24.278216 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_769_377, 202410_770_770_0] 2024.10.22 01:28:24.281574 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_769_377, 202410_770_770_0] 2024.10.22 01:28:24.295182 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_769_377, 202410_770_770_0] 2024.10.22 01:28:24.425282 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 8.63 MiB, will set to 1.67 GiB (RSS), difference: 7.80 MiB 2024.10.22 01:28:25.233674 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:28:25.444003 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/aht/jeocexuhoybagwdncmlgomzzhrcmn. 2024.10.22 01:28:25.982996 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4090 2024.10.22 01:28:26.178060 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:28:26.274719 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 769278 2024.10.22 01:28:26.766697 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.97 GiB, free memory in arenas 14.80 MiB, will set to 1.67 GiB (RSS), difference: 94.96 MiB 2024.10.22 01:28:26.812403 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_867_440} MergeTask::MergeProjectionsStage: Merge sorted 57376 rows, containing 13 columns (13 merged, 0 gathered) in 3.082747175 sec., 18611.97066866179 rows/sec., 2.96 MiB/sec. 2024.10.22 01:28:26.861544 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:26.901440 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_867_440} MergedBlockOutputStream: filled checksums 202410_1_867_440 (state Temporary) 2024.10.22 01:28:26.924081 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:27.153568 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_488_488_0 (state Temporary) 2024.10.22 01:28:27.322846 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_867_440} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_867_440 to 202410_1_867_440 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:27.326590 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_867_440} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_866_439, 202410_867_867_0] -> 202410_1_867_440 2024.10.22 01:28:27.365089 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 169 entries to flush up to offset 58997 2024.10.22 01:28:27.365563 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:28:27.477536 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_488_488_0 to 202410_495_495_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:27.617260 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 769278 2024.10.22 01:28:27.760714 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 56798 2024.10.22 01:28:27.772708 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 7867 2024.10.22 01:28:27.935171 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:27.940977 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:27.949524 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:27.954311 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:27.981365 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_509_509_0 (state Temporary) 2024.10.22 01:28:28.043167 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_499_499_0 (state Temporary) 2024.10.22 01:28:28.086885 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_499_499_0 to 202410_505_505_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:28.158059 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 58997 2024.10.22 01:28:28.208361 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_509_509_0 to 202410_522_522_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:28.301122 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:28.310393 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:28.377119 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7867 2024.10.22 01:28:28.577575 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_860_860_0 (state Temporary) 2024.10.22 01:28:28.630370 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_860_860_0 to 202410_873_873_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:28.710210 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.97 GiB, free memory in arenas 11.04 MiB, will set to 1.66 GiB (RSS), difference: 42.72 MiB 2024.10.22 01:28:28.776542 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_867_440 to 202410_868_868_0 2024.10.22 01:28:28.790195 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.05 MiB using storage policy from min volume index 0 2024.10.22 01:28:28.796377 [ 1166 ] {} DiskLocal: Reserved 2.05 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:28.820904 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_868_441} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_867_440 to 202410_868_868_0 into Compact with storage Full 2024.10.22 01:28:28.849760 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_868_441} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:28.861130 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_868_441} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_867_440, total 57376 rows starting from the beginning of the part 2024.10.22 01:28:28.900724 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_868_441} MergeTreeSequentialSource: Reading 2 marks from part 202410_868_868_0, total 36 rows starting from the beginning of the part 2024.10.22 01:28:28.915875 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56798 2024.10.22 01:28:29.038679 [ 1263 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:28:29.197415 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_770_378, 202410_771_771_0] 2024.10.22 01:28:29.201113 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_770_378, 202410_771_771_0] 2024.10.22 01:28:29.315531 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_770_378, 202410_771_771_0] 2024.10.22 01:28:30.258928 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:28:30.260825 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:28:30.627453 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.97 GiB, free memory in arenas 17.44 MiB, will set to 1.66 GiB (RSS), difference: 58.61 MiB 2024.10.22 01:28:31.378094 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_442_220, 202410_443_443_0] 2024.10.22 01:28:31.382619 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_442_220, 202410_443_443_0] 2024.10.22 01:28:31.427102 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_442_220, 202410_443_443_0] 2024.10.22 01:28:31.484614 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 430 2024.10.22 01:28:31.850779 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:31.856384 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:31.918477 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_412_412_0 (state Temporary) 2024.10.22 01:28:31.950057 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for fcc0799ccbebe353ead076f68434238b:0) 2024.10.22 01:28:31.954185 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3fd66ca296697c7f10206a0dc5af357e, offset: 0, size: 4278216 2024.10.22 01:28:31.960842 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3fd66ca296697c7f10206a0dc5af357e for removal 2024.10.22 01:28:31.964436 [ 1476 ] {} CacheMetadata: Key 3fd66ca296697c7f10206a0dc5af357e is removed from metadata 2024.10.22 01:28:31.964814 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: de1888c1e6e994b6d838556132070027, offset: 0, size: 274 2024.10.22 01:28:31.971174 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key de1888c1e6e994b6d838556132070027 for removal 2024.10.22 01:28:31.989443 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_412_412_0 to 202410_422_422_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:32.084723 [ 1476 ] {} CacheMetadata: Key de1888c1e6e994b6d838556132070027 is removed from metadata 2024.10.22 01:28:32.105077 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 430 2024.10.22 01:28:32.469065 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_771_379, 202410_772_772_0] 2024.10.22 01:28:32.475053 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_771_379, 202410_772_772_0] 2024.10.22 01:28:32.480449 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_771_379, 202410_772_772_0] 2024.10.22 01:28:32.491080 [ 1268 ] {} DatabaseCatalog: Execution took 319 ms. 2024.10.22 01:28:32.531412 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_868_441} MergeTask::MergeProjectionsStage: Merge sorted 57412 rows, containing 13 columns (13 merged, 0 gathered) in 3.721200119 sec., 15428.35595077546 rows/sec., 2.46 MiB/sec. 2024.10.22 01:28:32.638759 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 18.61 MiB, will set to 1.66 GiB (RSS), difference: 4.07 MiB 2024.10.22 01:28:32.723903 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_868_441} MergedBlockOutputStream: filled checksums 202410_1_868_441 (state Temporary) 2024.10.22 01:28:32.923841 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 56853 2024.10.22 01:28:32.998660 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_868_441} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_868_441 to 202410_1_868_441 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:33.004043 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_868_441} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_867_440, 202410_868_868_0] -> 202410_1_868_441 2024.10.22 01:28:33.031894 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:28:33.167658 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:33.170833 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:33.219773 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_861_861_0 (state Temporary) 2024.10.22 01:28:33.466374 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_861_861_0 to 202410_874_874_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:33.527309 [ 1206 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_868_441 to 202410_869_869_0 2024.10.22 01:28:33.527496 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56853 2024.10.22 01:28:33.536652 [ 1206 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.05 MiB using storage policy from min volume index 0 2024.10.22 01:28:33.540483 [ 1206 ] {} DiskLocal: Reserved 2.05 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:33.533625 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 4132 2024.10.22 01:28:33.582614 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_869_442} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_868_441 to 202410_869_869_0 into Compact with storage Full 2024.10.22 01:28:33.608377 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_869_442} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:33.618819 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_869_442} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_868_441, total 57412 rows starting from the beginning of the part 2024.10.22 01:28:33.635329 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_869_442} MergeTreeSequentialSource: Reading 2 marks from part 202410_869_869_0, total 27 rows starting from the beginning of the part 2024.10.22 01:28:33.767961 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_490_99/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:28:34.404937 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 389715 bytes for 013b595828b56d7d8059ecdfdf0f3582:0) 2024.10.22 01:28:34.408248 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 79f1fa5314ef4baff3cc61f48382242a, offset: 0, size: 103207 2024.10.22 01:28:34.413442 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 79f1fa5314ef4baff3cc61f48382242a for removal 2024.10.22 01:28:34.416944 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bdf5fa86995737e5514903668a9000d8, offset: 0, size: 164 2024.10.22 01:28:34.422786 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bdf5fa86995737e5514903668a9000d8 for removal 2024.10.22 01:28:34.437014 [ 1476 ] {} CacheMetadata: Key 79f1fa5314ef4baff3cc61f48382242a is removed from metadata 2024.10.22 01:28:34.552933 [ 1476 ] {} CacheMetadata: Key bdf5fa86995737e5514903668a9000d8 is removed from metadata 2024.10.22 01:28:34.592728 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 19.95 MiB, will set to 1.66 GiB (RSS), difference: -6.79 MiB 2024.10.22 01:28:34.621250 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 770818 2024.10.22 01:28:35.548053 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 835871 bytes for 6b8a4f98007b450fcaaade103260d798:0) 2024.10.22 01:28:35.551466 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1e5e791e25fb580f53a4c79ac82474bc, offset: 0, size: 1998216 2024.10.22 01:28:35.556912 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1e5e791e25fb580f53a4c79ac82474bc for removal 2024.10.22 01:28:35.559665 [ 1476 ] {} CacheMetadata: Key 1e5e791e25fb580f53a4c79ac82474bc is removed from metadata 2024.10.22 01:28:35.560787 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4344facfa8e056138bf1633f2396bf87, offset: 0, size: 258 2024.10.22 01:28:35.592173 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:35.613458 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:28:35.667858 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4344facfa8e056138bf1633f2396bf87 for removal 2024.10.22 01:28:35.672037 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dbacc3ee9524bb5068021b6ca62b9521, offset: 0, size: 4789 2024.10.22 01:28:35.672761 [ 1476 ] {} CacheMetadata: Key 4344facfa8e056138bf1633f2396bf87 is removed from metadata 2024.10.22 01:28:35.677536 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dbacc3ee9524bb5068021b6ca62b9521 for removal 2024.10.22 01:28:35.682626 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 128 entries to flush up to offset 59125 2024.10.22 01:28:35.686722 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 32fa6dc76117d6d72f1c666c708e9e67, offset: 0, size: 186 2024.10.22 01:28:35.688388 [ 1476 ] {} CacheMetadata: Key dbacc3ee9524bb5068021b6ca62b9521 is removed from metadata 2024.10.22 01:28:35.698856 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 32fa6dc76117d6d72f1c666c708e9e67 for removal 2024.10.22 01:28:35.705402 [ 1476 ] {} CacheMetadata: Key 32fa6dc76117d6d72f1c666c708e9e67 is removed from metadata 2024.10.22 01:28:35.879248 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 7875 2024.10.22 01:28:36.063385 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:36.067088 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:36.164093 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_500_500_0 (state Temporary) 2024.10.22 01:28:36.170530 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:36.174928 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:36.229142 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_510_510_0 (state Temporary) 2024.10.22 01:28:36.229256 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_489_489_0 (state Temporary) 2024.10.22 01:28:36.260454 [ 1703 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ikt/kqdmqhwiixhppikmzqlbkeaiqeuvl, size 2826516 2024.10.22 01:28:36.288893 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_500_500_0 to 202410_506_506_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:36.350250 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_489_489_0 to 202410_496_496_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:36.354869 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 59125 2024.10.22 01:28:36.363131 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_869_442} MergeTask::MergeProjectionsStage: Merge sorted 57439 rows, containing 13 columns (13 merged, 0 gathered) in 2.783413714 sec., 20636.170509289946 rows/sec., 3.28 MiB/sec. 2024.10.22 01:28:36.386337 [ 1180 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_461_501_8 to 202410_506_506_0 2024.10.22 01:28:36.398168 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 770818 2024.10.22 01:28:36.398292 [ 1191 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_491_491_0 to 202410_496_496_0 2024.10.22 01:28:36.439540 [ 1180 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:36.445261 [ 1180 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:36.454068 [ 1191 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:36.457414 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:36.464331 [ 1186 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:28:36.472378 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_461_501_8 to 202410_506_506_0 into Compact with storage Full 2024.10.22 01:28:36.474850 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_491_491_0 to 202410_496_496_0 into Compact with storage Full 2024.10.22 01:28:36.493237 [ 1186 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:28:36.502882 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:36.503729 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_510_510_0 to 202410_523_523_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:36.518519 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_501_8, total 6386 rows starting from the beginning of the part 2024.10.22 01:28:36.573102 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7875 2024.10.22 01:28:36.580717 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_502_502_0, total 115 rows starting from the beginning of the part 2024.10.22 01:28:36.585674 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:28:36.622878 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:28:36.623667 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_503_503_0, total 72 rows starting from the beginning of the part 2024.10.22 01:28:36.628244 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:36.635197 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 14.92 MiB, will set to 1.66 GiB (RSS), difference: 12.32 MiB 2024.10.22 01:28:36.639103 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:28:36.639180 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_491_491_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:28:36.675697 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_492_492_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:28:36.700612 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:28:36.737028 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:28:36.745350 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_493_493_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:28:36.768280 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_504_504_0, total 87 rows starting from the beginning of the part 2024.10.22 01:28:36.780886 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_494_494_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:28:36.782443 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_505_505_0, total 169 rows starting from the beginning of the part 2024.10.22 01:28:36.824574 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_506_506_0, total 128 rows starting from the beginning of the part 2024.10.22 01:28:36.841309 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_495_495_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:28:36.858074 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:28:36.869217 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_869_442} MergedBlockOutputStream: filled checksums 202410_1_869_442 (state Temporary) 2024.10.22 01:28:36.877181 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:28:36.927674 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:28:36.967245 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:28:37.030040 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_496_496_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:28:37.155004 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pkc/cylnveqnofeswefoxymhfnixbrkmc (LogSeriesLimiter: on interval from 2024-10-22 01:28:16 to 2024-10-22 01:28:37 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:28:37.347436 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_869_442} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_869_442 to 202410_1_869_442 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:37.353838 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_869_442} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_868_441, 202410_869_869_0] -> 202410_1_869_442 2024.10.22 01:28:37.382298 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:28:37.536779 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 98 entries to flush up to offset 56951 2024.10.22 01:28:37.775020 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 259788 bytes for 9453f4d943305209cc5f1c2d3b85b65d:0) 2024.10.22 01:28:37.777062 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1180f7610a3c7e2e63e93a7f6adf3c20, offset: 0, size: 221 2024.10.22 01:28:37.783560 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1180f7610a3c7e2e63e93a7f6adf3c20 for removal 2024.10.22 01:28:37.786182 [ 1476 ] {} CacheMetadata: Key 1180f7610a3c7e2e63e93a7f6adf3c20 is removed from metadata 2024.10.22 01:28:37.787445 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a4cb63ac9e8053d435084ba7d573e00a, offset: 0, size: 39536 2024.10.22 01:28:37.795465 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a4cb63ac9e8053d435084ba7d573e00a for removal 2024.10.22 01:28:37.804490 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6c8236d76aa1c080b9361b1b2360fd42, offset: 0, size: 26157 2024.10.22 01:28:37.811213 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6c8236d76aa1c080b9361b1b2360fd42 for removal 2024.10.22 01:28:37.815676 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8d5a14afa3e1113c24a7e60861f61795, offset: 0, size: 181 2024.10.22 01:28:37.818886 [ 1476 ] {} CacheMetadata: Key 6c8236d76aa1c080b9361b1b2360fd42 is removed from metadata 2024.10.22 01:28:37.825457 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8d5a14afa3e1113c24a7e60861f61795 for removal 2024.10.22 01:28:37.850690 [ 1476 ] {} CacheMetadata: Key 8d5a14afa3e1113c24a7e60861f61795 is removed from metadata 2024.10.22 01:28:37.864373 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:37.867953 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:37.882692 [ 1476 ] {} CacheMetadata: Key a4cb63ac9e8053d435084ba7d573e00a is removed from metadata 2024.10.22 01:28:37.956964 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_862_862_0 (state Temporary) 2024.10.22 01:28:38.023105 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_862_862_0 to 202410_875_875_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:38.121421 [ 1187 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 01:28:38.175135 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 56951 2024.10.22 01:28:38.182557 [ 1197 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_869_442 to 202410_870_870_0 2024.10.22 01:28:38.184497 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_104_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:28:38.191049 [ 1197 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.05 MiB using storage policy from min volume index 0 2024.10.22 01:28:38.198676 [ 1197 ] {} DiskLocal: Reserved 2.05 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:38.211443 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_870_443} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_869_442 to 202410_870_870_0 into Compact with storage Full 2024.10.22 01:28:38.281841 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} MergeTask::MergeProjectionsStage: Merge sorted 6957 rows, containing 13 columns (13 merged, 0 gathered) in 1.826515883 sec., 3808.8910503057477 rows/sec., 1.66 MiB/sec. 2024.10.22 01:28:38.327971 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_772_380, 202410_773_773_0] 2024.10.22 01:28:38.340845 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_772_380, 202410_773_773_0] 2024.10.22 01:28:38.357248 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} MergeTask::MergeProjectionsStage: Merge sorted 9625 rows, containing 4 columns (4 merged, 0 gathered) in 1.888544312 sec., 5096.517957689308 rows/sec., 79.63 KiB/sec. 2024.10.22 01:28:38.358836 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_772_380, 202410_773_773_0] 2024.10.22 01:28:38.376468 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_870_443} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:38.394777 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_870_443} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_869_442, total 57439 rows starting from the beginning of the part 2024.10.22 01:28:38.476770 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_870_443} MergeTreeSequentialSource: Reading 2 marks from part 202410_870_870_0, total 32 rows starting from the beginning of the part 2024.10.22 01:28:38.477841 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} MergedBlockOutputStream: filled checksums 202410_491_496_1 (state Temporary) 2024.10.22 01:28:38.511080 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 13.44 MiB, will set to 1.66 GiB (RSS), difference: -27.05 MiB 2024.10.22 01:28:38.590191 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 254418 bytes for efe67899e3c31f92c9f5f7f599be7443:0) 2024.10.22 01:28:38.612205 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8b4b4c87669da7b96e9599a23a4d3871, offset: 0, size: 579934 2024.10.22 01:28:38.618750 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8b4b4c87669da7b96e9599a23a4d3871 for removal 2024.10.22 01:28:38.624161 [ 1476 ] {} CacheMetadata: Key 8b4b4c87669da7b96e9599a23a4d3871 is removed from metadata 2024.10.22 01:28:38.625138 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7b1cf6956d342f1a9844e13289e0a53b, offset: 0, size: 238 2024.10.22 01:28:38.628774 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7b1cf6956d342f1a9844e13289e0a53b for removal 2024.10.22 01:28:38.647817 [ 1476 ] {} CacheMetadata: Key 7b1cf6956d342f1a9844e13289e0a53b is removed from metadata 2024.10.22 01:28:38.659367 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_491_496_1 to 202410_491_496_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:38.662062 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_491_496_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_491_491_0, 202410_496_496_0] -> 202410_491_496_1 2024.10.22 01:28:38.683996 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} MergedBlockOutputStream: filled checksums 202410_461_506_9 (state Temporary) 2024.10.22 01:28:38.704435 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:28:38.802933 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_461_506_9 to 202410_461_506_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:38.807995 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_461_506_9} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_461_501_8, 202410_506_506_0] -> 202410_461_506_9 2024.10.22 01:28:38.952173 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.51 MiB. 2024.10.22 01:28:39.465080 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_443_221, 202410_444_444_0] 2024.10.22 01:28:39.467186 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_443_221, 202410_444_444_0] 2024.10.22 01:28:39.507958 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_443_221, 202410_444_444_0] 2024.10.22 01:28:40.354314 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 928371 bytes for fcc0799ccbebe353ead076f68434238b:0) 2024.10.22 01:28:40.359986 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6b8a4f98007b450fcaaade103260d798, offset: 0, size: 835871 2024.10.22 01:28:40.360068 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:28:40.362686 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:28:40.369836 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6b8a4f98007b450fcaaade103260d798 for removal 2024.10.22 01:28:40.373513 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 89d193dbea52ea1523797bd040e2c0c4, offset: 0, size: 229 2024.10.22 01:28:40.374722 [ 1476 ] {} CacheMetadata: Key 6b8a4f98007b450fcaaade103260d798 is removed from metadata 2024.10.22 01:28:40.376070 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 89d193dbea52ea1523797bd040e2c0c4 for removal 2024.10.22 01:28:40.380430 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 013b595828b56d7d8059ecdfdf0f3582, offset: 0, size: 389715 2024.10.22 01:28:40.385113 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 013b595828b56d7d8059ecdfdf0f3582 for removal 2024.10.22 01:28:40.392698 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 31fb217a8cb794eaf90e8d14a175d6ca, offset: 0, size: 233 2024.10.22 01:28:40.396430 [ 1476 ] {} CacheMetadata: Key 013b595828b56d7d8059ecdfdf0f3582 is removed from metadata 2024.10.22 01:28:40.397583 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 31fb217a8cb794eaf90e8d14a175d6ca for removal 2024.10.22 01:28:40.412368 [ 1476 ] {} CacheMetadata: Key 31fb217a8cb794eaf90e8d14a175d6ca is removed from metadata 2024.10.22 01:28:40.434273 [ 1476 ] {} CacheMetadata: Key 89d193dbea52ea1523797bd040e2c0c4 is removed from metadata 2024.10.22 01:28:40.852217 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 14.95 MiB, will set to 1.67 GiB (RSS), difference: 19.61 MiB 2024.10.22 01:28:41.185240 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:28:41.199108 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:28:41.221552 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:28:41.244741 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:28:41.271132 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:28:41.281808 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:28:41.303848 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:28:41.305152 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 5 old parts to remove. Parts: [202410_1_369_76, 202410_370_370_0, 202410_371_371_0, 202410_372_372_0, 202410_373_373_0] 2024.10.22 01:28:41.311395 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from filesystem (serially): Parts: [202410_1_369_76, 202410_370_370_0, 202410_371_371_0, 202410_372_372_0, 202410_373_373_0] 2024.10.22 01:28:41.346239 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:28:41.373559 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 5 parts from memory: Parts: [202410_1_369_76, 202410_370_370_0, 202410_371_371_0, 202410_372_372_0, 202410_373_373_0] 2024.10.22 01:28:41.508396 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_870_443} MergeTask::MergeProjectionsStage: Merge sorted 57471 rows, containing 13 columns (13 merged, 0 gathered) in 3.300785439 sec., 17411.310447797936 rows/sec., 2.77 MiB/sec. 2024.10.22 01:28:41.763622 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_870_443} MergedBlockOutputStream: filled checksums 202410_1_870_443 (state Temporary) 2024.10.22 01:28:41.836797 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 63986 bytes for 70caecf26e4193c0c69ce14f619bba3f:0) 2024.10.22 01:28:41.840362 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3e4669c0a88d5e54120bbdf4ef1f666b, offset: 0, size: 167 2024.10.22 01:28:41.846947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3e4669c0a88d5e54120bbdf4ef1f666b for removal 2024.10.22 01:28:41.851674 [ 1476 ] {} CacheMetadata: Key 3e4669c0a88d5e54120bbdf4ef1f666b is removed from metadata 2024.10.22 01:28:41.851469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 48d720d80d20c7ab0c28ab9abd6a8e4b, offset: 0, size: 145894 2024.10.22 01:28:41.863302 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 48d720d80d20c7ab0c28ab9abd6a8e4b for removal 2024.10.22 01:28:41.864797 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 855ddef7b3738cb642deafa6d108f560, offset: 0, size: 8518 2024.10.22 01:28:41.909666 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_870_443} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_870_443 to 202410_1_870_443 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:41.918795 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_870_443} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_869_442, 202410_870_870_0] -> 202410_1_870_443 2024.10.22 01:28:41.953359 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:28:41.967371 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 855ddef7b3738cb642deafa6d108f560 for removal 2024.10.22 01:28:41.968301 [ 1476 ] {} CacheMetadata: Key 48d720d80d20c7ab0c28ab9abd6a8e4b is removed from metadata 2024.10.22 01:28:41.970098 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a5adaa98a9ecc699445a7ec920a7f6b9, offset: 0, size: 160 2024.10.22 01:28:41.970931 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_490_99} MergeTask::MergeProjectionsStage: Merge sorted 774783 rows, containing 4 columns (4 merged, 0 gathered) in 22.680751439 sec., 34160.37612703366 rows/sec., 512.21 KiB/sec. 2024.10.22 01:28:41.971508 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a5adaa98a9ecc699445a7ec920a7f6b9 for removal 2024.10.22 01:28:41.972798 [ 1476 ] {} CacheMetadata: Key a5adaa98a9ecc699445a7ec920a7f6b9 is removed from metadata 2024.10.22 01:28:41.983287 [ 1476 ] {} CacheMetadata: Key 855ddef7b3738cb642deafa6d108f560 is removed from metadata 2024.10.22 01:28:42.130332 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_490_99} MergedBlockOutputStream: filled checksums 202410_1_490_99 (state Temporary) 2024.10.22 01:28:42.199833 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 99 entries to flush up to offset 57050 2024.10.22 01:28:42.429817 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:42.445507 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:42.537629 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 13.53 MiB, will set to 1.65 GiB (RSS), difference: -1.82 MiB 2024.10.22 01:28:42.538814 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_863_863_0 (state Temporary) 2024.10.22 01:28:42.826371 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_863_863_0 to 202410_876_876_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:42.858322 [ 1151 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_870_443 to 202410_871_871_0 2024.10.22 01:28:42.862837 [ 1151 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.05 MiB using storage policy from min volume index 0 2024.10.22 01:28:42.866637 [ 1151 ] {} DiskLocal: Reserved 2.05 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:42.880691 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 57050 2024.10.22 01:28:42.905341 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_871_444} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_870_443 to 202410_871_871_0 into Compact with storage Full 2024.10.22 01:28:42.936455 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 451077 bytes for 0ef4d68e5bb6b81504c34457f8a7972b:0) 2024.10.22 01:28:42.939859 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 068082546965675e1bd0a9a61058c29a, offset: 0, size: 193 2024.10.22 01:28:42.944950 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 068082546965675e1bd0a9a61058c29a for removal 2024.10.22 01:28:42.947927 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_871_444} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:42.949805 [ 1476 ] {} CacheMetadata: Key 068082546965675e1bd0a9a61058c29a is removed from metadata 2024.10.22 01:28:42.950212 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9453f4d943305209cc5f1c2d3b85b65d, offset: 0, size: 259788 2024.10.22 01:28:42.953650 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9453f4d943305209cc5f1c2d3b85b65d for removal 2024.10.22 01:28:42.957265 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2eec95828db2208de3586826a5bcd68a, offset: 0, size: 168358 2024.10.22 01:28:42.959769 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2eec95828db2208de3586826a5bcd68a for removal 2024.10.22 01:28:42.961490 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ec4f0cc3b29437ac3b6539a647aa32ff, offset: 0, size: 167 2024.10.22 01:28:42.964853 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ec4f0cc3b29437ac3b6539a647aa32ff for removal 2024.10.22 01:28:43.059543 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_871_444} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_870_443, total 57471 rows starting from the beginning of the part 2024.10.22 01:28:43.063515 [ 1476 ] {} CacheMetadata: Key ec4f0cc3b29437ac3b6539a647aa32ff is removed from metadata 2024.10.22 01:28:43.068753 [ 1476 ] {} CacheMetadata: Key 2eec95828db2208de3586826a5bcd68a is removed from metadata 2024.10.22 01:28:43.081213 [ 1476 ] {} CacheMetadata: Key 9453f4d943305209cc5f1c2d3b85b65d is removed from metadata 2024.10.22 01:28:43.084355 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_871_444} MergeTreeSequentialSource: Reading 2 marks from part 202410_871_871_0, total 36 rows starting from the beginning of the part 2024.10.22 01:28:43.490816 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1917 entries to flush up to offset 772735 2024.10.22 01:28:43.519809 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_773_381, 202410_774_774_0] 2024.10.22 01:28:43.526169 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_773_381, 202410_774_774_0] 2024.10.22 01:28:43.553251 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_773_381, 202410_774_774_0] 2024.10.22 01:28:43.823413 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 258931 bytes for 9ba06abae2e75d495c2417cd4f81f442:0) 2024.10.22 01:28:43.829175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: efe67899e3c31f92c9f5f7f599be7443, offset: 0, size: 254418 2024.10.22 01:28:43.835685 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key efe67899e3c31f92c9f5f7f599be7443 for removal 2024.10.22 01:28:43.869314 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 161 entries to flush up to offset 59286 2024.10.22 01:28:43.941479 [ 1476 ] {} CacheMetadata: Key efe67899e3c31f92c9f5f7f599be7443 is removed from metadata 2024.10.22 01:28:43.943693 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 919e0abfb500167b59eb4039394fd227, offset: 0, size: 194 2024.10.22 01:28:43.951253 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 919e0abfb500167b59eb4039394fd227 for removal 2024.10.22 01:28:43.971943 [ 1476 ] {} CacheMetadata: Key 919e0abfb500167b59eb4039394fd227 is removed from metadata 2024.10.22 01:28:44.053395 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_490_99} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_490_99 to 202410_1_490_99 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:44.060359 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_490_99} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_484_98, 202410_485_490_1] -> 202410_1_490_99 2024.10.22 01:28:44.076801 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 7903 2024.10.22 01:28:44.080630 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:28:44.163136 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:44.166761 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:44.171354 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:44.177692 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:44.231923 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:44.240019 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:44.251130 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_501_501_0 (state Temporary) 2024.10.22 01:28:44.295849 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_511_511_0 (state Temporary) 2024.10.22 01:28:44.334242 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_490_490_0 (state Temporary) 2024.10.22 01:28:44.350998 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_511_511_0 to 202410_524_524_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:44.391855 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_490_490_0 to 202410_497_497_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:44.431331 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_501_501_0 to 202410_507_507_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:44.463401 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 10.79 MiB, will set to 1.66 GiB (RSS), difference: -2.32 MiB 2024.10.22 01:28:44.468053 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 59286 2024.10.22 01:28:44.517723 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 142921 bytes for 2328b78085cb4beb4ba48b636dfb1dbd:0) 2024.10.22 01:28:44.521573 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6134810e828b4d3b8f99e3bd3134c0bd, offset: 0, size: 127879 2024.10.22 01:28:44.525126 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6134810e828b4d3b8f99e3bd3134c0bd for removal 2024.10.22 01:28:44.528622 [ 1476 ] {} CacheMetadata: Key 6134810e828b4d3b8f99e3bd3134c0bd is removed from metadata 2024.10.22 01:28:44.529494 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3ed8b6c7f72bade815d8e584a45ab734, offset: 0, size: 165 2024.10.22 01:28:44.535812 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3ed8b6c7f72bade815d8e584a45ab734 for removal 2024.10.22 01:28:44.541463 [ 1476 ] {} CacheMetadata: Key 3ed8b6c7f72bade815d8e584a45ab734 is removed from metadata 2024.10.22 01:28:44.552766 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 772735 2024.10.22 01:28:44.656562 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7903 2024.10.22 01:28:45.306812 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 16 file segments (while reserving 1048576 bytes for f8ec25a0412ed1dbc7f4b238871dba96:0) 2024.10.22 01:28:45.311312 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 835db135e887f2bf0411a01f259e0825, offset: 0, size: 159 2024.10.22 01:28:45.317354 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 835db135e887f2bf0411a01f259e0825 for removal 2024.10.22 01:28:45.323061 [ 1476 ] {} CacheMetadata: Key 835db135e887f2bf0411a01f259e0825 is removed from metadata 2024.10.22 01:28:45.324425 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 70caecf26e4193c0c69ce14f619bba3f, offset: 0, size: 63986 2024.10.22 01:28:45.327106 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 70caecf26e4193c0c69ce14f619bba3f for removal 2024.10.22 01:28:45.329233 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bd2a4a2a0c3572f071c3a7d3b72aabd8, offset: 0, size: 159 2024.10.22 01:28:45.334805 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bd2a4a2a0c3572f071c3a7d3b72aabd8 for removal 2024.10.22 01:28:45.339253 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f6784960cb95f173f0e5fe2cf338c764, offset: 0, size: 161 2024.10.22 01:28:45.348631 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f6784960cb95f173f0e5fe2cf338c764 for removal 2024.10.22 01:28:45.351301 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 01f7d3b4606c2884166506432baa835a, offset: 0, size: 3795 2024.10.22 01:28:45.353811 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 01f7d3b4606c2884166506432baa835a for removal 2024.10.22 01:28:45.367262 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 737ccf5c90aa857664235bd660d69d7c, offset: 0, size: 161 2024.10.22 01:28:45.370436 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 737ccf5c90aa857664235bd660d69d7c for removal 2024.10.22 01:28:45.377285 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 472220affd53ad9583c73bde4aa4c46b, offset: 0, size: 4313 2024.10.22 01:28:45.381028 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 472220affd53ad9583c73bde4aa4c46b for removal 2024.10.22 01:28:45.383479 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2ca297d2d61793375e0ac6ed95b8a1d7, offset: 0, size: 161 2024.10.22 01:28:45.387099 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 2ca297d2d61793375e0ac6ed95b8a1d7 for removal 2024.10.22 01:28:45.392814 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a094cb6a4c5554faf0514ab3b37174bd, offset: 0, size: 160 2024.10.22 01:28:45.397255 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a094cb6a4c5554faf0514ab3b37174bd for removal 2024.10.22 01:28:45.414527 [ 1476 ] {} CacheMetadata: Key 2ca297d2d61793375e0ac6ed95b8a1d7 is removed from metadata 2024.10.22 01:28:45.502580 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3396ca52969ca4cadb776bf912846f61, offset: 0, size: 257505 2024.10.22 01:28:45.509179 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3396ca52969ca4cadb776bf912846f61 for removal 2024.10.22 01:28:45.512744 [ 1476 ] {} CacheMetadata: Key a094cb6a4c5554faf0514ab3b37174bd is removed from metadata 2024.10.22 01:28:45.522668 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: de48b8ce7f022ae5e4bc3b05ef51193a, offset: 0, size: 74999 2024.10.22 01:28:45.529209 [ 1476 ] {} CacheMetadata: Key 3396ca52969ca4cadb776bf912846f61 is removed from metadata 2024.10.22 01:28:45.529546 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key de48b8ce7f022ae5e4bc3b05ef51193a for removal 2024.10.22 01:28:45.532611 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d19ddbe2c5f7dfbc61f8059038584dbc, offset: 0, size: 235 2024.10.22 01:28:45.535970 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d19ddbe2c5f7dfbc61f8059038584dbc for removal 2024.10.22 01:28:45.538432 [ 1476 ] {} CacheMetadata: Key d19ddbe2c5f7dfbc61f8059038584dbc is removed from metadata 2024.10.22 01:28:45.538169 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 054aea9a55c8bd5aef48feb826b6779c, offset: 0, size: 120074 2024.10.22 01:28:45.544806 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 054aea9a55c8bd5aef48feb826b6779c for removal 2024.10.22 01:28:45.550614 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0ef4d68e5bb6b81504c34457f8a7972b, offset: 0, size: 451077 2024.10.22 01:28:45.554868 [ 1476 ] {} CacheMetadata: Key 054aea9a55c8bd5aef48feb826b6779c is removed from metadata 2024.10.22 01:28:45.556283 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0ef4d68e5bb6b81504c34457f8a7972b for removal 2024.10.22 01:28:45.559477 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2bed7d93f79739fb50cba0773aac6080, offset: 0, size: 56453 2024.10.22 01:28:45.560592 [ 1476 ] {} CacheMetadata: Key 0ef4d68e5bb6b81504c34457f8a7972b is removed from metadata 2024.10.22 01:28:45.561156 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 2bed7d93f79739fb50cba0773aac6080 for removal 2024.10.22 01:28:45.563567 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 35372cd62a801c0015d8f8725122e333, offset: 0, size: 192 2024.10.22 01:28:45.565666 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 35372cd62a801c0015d8f8725122e333 for removal 2024.10.22 01:28:45.576441 [ 1476 ] {} CacheMetadata: Key 2bed7d93f79739fb50cba0773aac6080 is removed from metadata 2024.10.22 01:28:45.591800 [ 1476 ] {} CacheMetadata: Key 35372cd62a801c0015d8f8725122e333 is removed from metadata 2024.10.22 01:28:45.610145 [ 1476 ] {} CacheMetadata: Key de48b8ce7f022ae5e4bc3b05ef51193a is removed from metadata 2024.10.22 01:28:45.618655 [ 1476 ] {} CacheMetadata: Key 472220affd53ad9583c73bde4aa4c46b is removed from metadata 2024.10.22 01:28:45.643059 [ 1476 ] {} CacheMetadata: Key 737ccf5c90aa857664235bd660d69d7c is removed from metadata 2024.10.22 01:28:45.655562 [ 1476 ] {} CacheMetadata: Key 01f7d3b4606c2884166506432baa835a is removed from metadata 2024.10.22 01:28:45.664680 [ 1476 ] {} CacheMetadata: Key f6784960cb95f173f0e5fe2cf338c764 is removed from metadata 2024.10.22 01:28:45.670565 [ 1476 ] {} CacheMetadata: Key bd2a4a2a0c3572f071c3a7d3b72aabd8 is removed from metadata 2024.10.22 01:28:45.679424 [ 1476 ] {} CacheMetadata: Key 70caecf26e4193c0c69ce14f619bba3f is removed from metadata 2024.10.22 01:28:46.175220 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 122157 bytes for dbb3569a30f8513b2189901dad4a0ca5:0) 2024.10.22 01:28:46.177772 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9ba06abae2e75d495c2417cd4f81f442, offset: 0, size: 258931 2024.10.22 01:28:46.182801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9ba06abae2e75d495c2417cd4f81f442 for removal 2024.10.22 01:28:46.186365 [ 1476 ] {} CacheMetadata: Key 9ba06abae2e75d495c2417cd4f81f442 is removed from metadata 2024.10.22 01:28:46.188060 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 24f1d742f7961dad755501416c291d15, offset: 0, size: 166 2024.10.22 01:28:46.195286 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 24f1d742f7961dad755501416c291d15 for removal 2024.10.22 01:28:46.201170 [ 1476 ] {} CacheMetadata: Key 24f1d742f7961dad755501416c291d15 is removed from metadata 2024.10.22 01:28:46.529091 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_871_444} MergeTask::MergeProjectionsStage: Merge sorted 57507 rows, containing 13 columns (13 merged, 0 gathered) in 3.633486828 sec., 15826.946049961021 rows/sec., 2.52 MiB/sec. 2024.10.22 01:28:46.718153 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_871_444} MergedBlockOutputStream: filled checksums 202410_1_871_444 (state Temporary) 2024.10.22 01:28:46.913585 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 124 entries to flush up to offset 57174 2024.10.22 01:28:46.930076 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 385742 bytes for a99357919ab4dcf7dfde097b9e9f230d:0) 2024.10.22 01:28:46.935641 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dbb3569a30f8513b2189901dad4a0ca5, offset: 0, size: 122157 2024.10.22 01:28:46.947494 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dbb3569a30f8513b2189901dad4a0ca5 for removal 2024.10.22 01:28:46.951150 [ 1476 ] {} CacheMetadata: Key dbb3569a30f8513b2189901dad4a0ca5 is removed from metadata 2024.10.22 01:28:46.954836 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2328b78085cb4beb4ba48b636dfb1dbd, offset: 0, size: 142921 2024.10.22 01:28:46.958529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2328b78085cb4beb4ba48b636dfb1dbd for removal 2024.10.22 01:28:46.961753 [ 1476 ] {} CacheMetadata: Key 2328b78085cb4beb4ba48b636dfb1dbd is removed from metadata 2024.10.22 01:28:46.962665 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: edd27611c4a40698fd32f198d322f1bf, offset: 0, size: 164 2024.10.22 01:28:46.965466 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key edd27611c4a40698fd32f198d322f1bf for removal 2024.10.22 01:28:46.968658 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 56f1ee3032d5c8cee2447eb43f6d2cef, offset: 0, size: 190 2024.10.22 01:28:46.970725 [ 1476 ] {} CacheMetadata: Key edd27611c4a40698fd32f198d322f1bf is removed from metadata 2024.10.22 01:28:46.973367 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 56f1ee3032d5c8cee2447eb43f6d2cef for removal 2024.10.22 01:28:46.979860 [ 1476 ] {} CacheMetadata: Key 56f1ee3032d5c8cee2447eb43f6d2cef is removed from metadata 2024.10.22 01:28:47.108803 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 431 2024.10.22 01:28:47.201145 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_871_444} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_871_444 to 202410_1_871_444 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:47.207934 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_871_444} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_870_443, 202410_871_871_0] -> 202410_1_871_444 2024.10.22 01:28:47.247726 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:28:47.581394 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:47.586514 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:47.624472 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 8.13 MiB, will set to 1.65 GiB (RSS), difference: 4.60 MiB 2024.10.22 01:28:47.707384 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 411936 bytes for 2cba3840092a58944ed6ea7384d272ed:0) 2024.10.22 01:28:47.711446 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a99357919ab4dcf7dfde097b9e9f230d, offset: 0, size: 385742 2024.10.22 01:28:47.716273 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a99357919ab4dcf7dfde097b9e9f230d for removal 2024.10.22 01:28:47.722708 [ 1476 ] {} CacheMetadata: Key a99357919ab4dcf7dfde097b9e9f230d is removed from metadata 2024.10.22 01:28:47.724047 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 636cd98a7163205dfcf3c3e59eb545bd, offset: 0, size: 193 2024.10.22 01:28:47.726160 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 636cd98a7163205dfcf3c3e59eb545bd for removal 2024.10.22 01:28:47.747039 [ 1278 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 01:28:47.747347 [ 1476 ] {} CacheMetadata: Key 636cd98a7163205dfcf3c3e59eb545bd is removed from metadata 2024.10.22 01:28:47.763618 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_413_413_0 (state Temporary) 2024.10.22 01:28:47.909726 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_444_222, 202410_445_445_0] 2024.10.22 01:28:47.918801 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_444_222, 202410_445_445_0] 2024.10.22 01:28:47.943116 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_444_222, 202410_445_445_0] 2024.10.22 01:28:48.070972 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_413_413_0 to 202410_423_423_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:48.160738 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 431 2024.10.22 01:28:48.199787 [ 1162 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_418_86 to 202410_423_423_0 2024.10.22 01:28:48.257397 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:28:48.258459 [ 1162 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:48.260592 [ 1162 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:48.280259 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_418_86 to 202410_423_423_0 into Compact with storage Full 2024.10.22 01:28:48.365212 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:48.418987 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:48.469098 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_774_382, 202410_775_775_0] 2024.10.22 01:28:48.472179 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_774_382, 202410_775_775_0] 2024.10.22 01:28:48.480675 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_774_382, 202410_775_775_0] 2024.10.22 01:28:48.574524 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_864_864_0 (state Temporary) 2024.10.22 01:28:48.666257 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:48.728955 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_418_86, total 462 rows starting from the beginning of the part 2024.10.22 01:28:48.732927 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 198752 bytes for 107e78d3a51fd4e353c4d24f11e0665a:0) 2024.10.22 01:28:48.738191 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2cba3840092a58944ed6ea7384d272ed, offset: 0, size: 411936 2024.10.22 01:28:48.745194 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2cba3840092a58944ed6ea7384d272ed for removal 2024.10.22 01:28:48.748842 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_864_864_0 to 202410_877_877_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:48.748910 [ 1476 ] {} CacheMetadata: Key 2cba3840092a58944ed6ea7384d272ed is removed from metadata 2024.10.22 01:28:48.749488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3652a1b94cbd5ab5efbb9725dd7b8618, offset: 0, size: 196 2024.10.22 01:28:48.769696 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3652a1b94cbd5ab5efbb9725dd7b8618 for removal 2024.10.22 01:28:48.774353 [ 1476 ] {} CacheMetadata: Key 3652a1b94cbd5ab5efbb9725dd7b8618 is removed from metadata 2024.10.22 01:28:48.775422 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_871_444 to 202410_872_872_0 2024.10.22 01:28:48.783319 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.05 MiB using storage policy from min volume index 0 2024.10.22 01:28:48.790935 [ 1262 ] {} DiskLocal: Reserved 2.05 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:48.835758 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_872_445} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_871_444 to 202410_872_872_0 into Compact with storage Full 2024.10.22 01:28:48.890047 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 57174 2024.10.22 01:28:48.996490 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_872_445} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:49.067762 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:49.070586 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:49.114728 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_419_419_0, total 1 rows starting from the beginning of the part 2024.10.22 01:28:49.115101 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_872_445} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_871_444, total 57507 rows starting from the beginning of the part 2024.10.22 01:28:49.168518 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_872_445} MergeTreeSequentialSource: Reading 2 marks from part 202410_872_872_0, total 76 rows starting from the beginning of the part 2024.10.22 01:28:49.394268 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_420_420_0, total 1 rows starting from the beginning of the part 2024.10.22 01:28:49.428158 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 5.38 MiB, will set to 1.65 GiB (RSS), difference: -4.25 MiB 2024.10.22 01:28:49.512018 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(5fe338a91331f5aef30ae3d4a87f7527) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5fe338a91331f5aef30ae3d4a87f7527, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:28:49.518741 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 5fe338a91331f5aef30ae3d4a87f7527, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:28:49.521787 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:28:49.586099 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_421_421_0, total 1 rows starting from the beginning of the part 2024.10.22 01:28:49.989586 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_422_422_0, total 1 rows starting from the beginning of the part 2024.10.22 01:28:50.009562 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(04a8140a648c444abd52dd9134ead888) : [0, 2020072]: Setting download as failed: File segment: [0, 2020072], key: 04a8140a648c444abd52dd9134ead888, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:28:50.021205 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(04a8140a648c444abd52dd9134ead888): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:28:50.176672 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 04a8140a648c444abd52dd9134ead888, offset: 0, size: 0 2024.10.22 01:28:50.178822 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 04a8140a648c444abd52dd9134ead888 for removal 2024.10.22 01:28:50.182157 [ 1476 ] {} CacheMetadata: Key 04a8140a648c444abd52dd9134ead888 is removed from metadata 2024.10.22 01:28:50.283111 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_423_423_0, total 1 rows starting from the beginning of the part 2024.10.22 01:28:50.361356 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:28:50.363297 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:28:50.673966 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(f8ec25a0412ed1dbc7f4b238871dba96) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f8ec25a0412ed1dbc7f4b238871dba96, state: DOWNLOADING, downloaded size: 1048576, reserved size: 1048576, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, current write offset: 1048576, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 01:28:50.704875 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: f8ec25a0412ed1dbc7f4b238871dba96, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 1048576, reserved size: 1048576, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, current write offset: 1048576, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 01:28:50.709304 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:28:50.765694 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_104_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:28:51.506083 [ 1238 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:28:51.554839 [ 1238 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:28:51.558967 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1163 entries to flush up to offset 773898 2024.10.22 01:28:51.883586 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 201229 bytes for 7da8fd1e121368527f9006d27b96fedd:0) 2024.10.22 01:28:51.986714 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 107e78d3a51fd4e353c4d24f11e0665a, offset: 0, size: 198752 2024.10.22 01:28:51.989180 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 107e78d3a51fd4e353c4d24f11e0665a for removal 2024.10.22 01:28:51.992260 [ 1476 ] {} CacheMetadata: Key 107e78d3a51fd4e353c4d24f11e0665a is removed from metadata 2024.10.22 01:28:51.992487 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f6c88ac645d886aab0671e52ca4edaee, offset: 0, size: 192 2024.10.22 01:28:51.997544 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f6c88ac645d886aab0671e52ca4edaee for removal 2024.10.22 01:28:52.002701 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 85 entries to flush up to offset 59371 2024.10.22 01:28:52.125809 [ 1476 ] {} CacheMetadata: Key f6c88ac645d886aab0671e52ca4edaee is removed from metadata 2024.10.22 01:28:52.161440 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 7911 2024.10.22 01:28:52.325773 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:52.333547 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:52.470375 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:52.473310 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_512_512_0 (state Temporary) 2024.10.22 01:28:52.574560 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:52.618847 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_502_502_0 (state Temporary) 2024.10.22 01:28:52.661009 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_502_502_0 to 202410_508_508_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:52.675801 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 5.67 MiB, will set to 1.65 GiB (RSS), difference: -11.57 MiB 2024.10.22 01:28:52.686329 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_512_512_0 to 202410_525_525_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:52.710368 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:52.718371 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:52.735352 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_103_103_0 (state Temporary) 2024.10.22 01:28:52.763294 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7911 2024.10.22 01:28:52.785376 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_872_445} MergeTask::MergeProjectionsStage: Merge sorted 57583 rows, containing 13 columns (13 merged, 0 gathered) in 3.960768905 sec., 14538.338737033686 rows/sec., 2.31 MiB/sec. 2024.10.22 01:28:52.789150 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 7 file segments (while reserving 271939 bytes for cf1c00746390769722ead9a90b542b9d:0) 2024.10.22 01:28:52.794483 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 26b7712a107a9af807a7a5f5281e84a1, offset: 0, size: 116382 2024.10.22 01:28:52.802872 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 26b7712a107a9af807a7a5f5281e84a1 for removal 2024.10.22 01:28:52.811196 [ 1476 ] {} CacheMetadata: Key 26b7712a107a9af807a7a5f5281e84a1 is removed from metadata 2024.10.22 01:28:52.812787 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0086cc7bee1b84c0cb96f1130957639b, offset: 0, size: 101110 2024.10.22 01:28:52.815639 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 59371 2024.10.22 01:28:52.879082 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_872_445} MergedBlockOutputStream: filled checksums 202410_1_872_445 (state Temporary) 2024.10.22 01:28:52.912307 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 119 entries to flush up to offset 57293 2024.10.22 01:28:52.920295 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0086cc7bee1b84c0cb96f1130957639b for removal 2024.10.22 01:28:52.922905 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7da8fd1e121368527f9006d27b96fedd, offset: 0, size: 201229 2024.10.22 01:28:52.923001 [ 1476 ] {} CacheMetadata: Key 0086cc7bee1b84c0cb96f1130957639b is removed from metadata 2024.10.22 01:28:52.925093 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7da8fd1e121368527f9006d27b96fedd for removal 2024.10.22 01:28:52.929155 [ 1476 ] {} CacheMetadata: Key 7da8fd1e121368527f9006d27b96fedd is removed from metadata 2024.10.22 01:28:52.929392 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 369abc693238ecff5255f0e9442da918, offset: 0, size: 163 2024.10.22 01:28:52.933978 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 369abc693238ecff5255f0e9442da918 for removal 2024.10.22 01:28:52.943577 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 55696f67feecfefd29e4fade0f50ee69, offset: 0, size: 164 2024.10.22 01:28:52.947237 [ 1476 ] {} CacheMetadata: Key 369abc693238ecff5255f0e9442da918 is removed from metadata 2024.10.22 01:28:52.952855 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 55696f67feecfefd29e4fade0f50ee69 for removal 2024.10.22 01:28:52.957537 [ 1476 ] {} CacheMetadata: Key 55696f67feecfefd29e4fade0f50ee69 is removed from metadata 2024.10.22 01:28:52.986680 [ 1265 ] {} DatabaseCatalog: Execution took 348 ms. 2024.10.22 01:28:52.957657 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e3f7f4d39c0828535954b8c1bbf6902e, offset: 0, size: 230 2024.10.22 01:28:53.062715 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e3f7f4d39c0828535954b8c1bbf6902e for removal 2024.10.22 01:28:53.063545 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_491_491_0 (state Temporary) 2024.10.22 01:28:53.076090 [ 1476 ] {} CacheMetadata: Key e3f7f4d39c0828535954b8c1bbf6902e is removed from metadata 2024.10.22 01:28:53.076471 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6d56c73b6d1826a60511afbe52797fb4, offset: 0, size: 258 2024.10.22 01:28:53.078646 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6d56c73b6d1826a60511afbe52797fb4 for removal 2024.10.22 01:28:53.081178 [ 1476 ] {} CacheMetadata: Key 6d56c73b6d1826a60511afbe52797fb4 is removed from metadata 2024.10.22 01:28:53.173629 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_491_491_0 to 202410_498_498_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:53.175565 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:53.180380 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:53.185106 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_872_445} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_872_445 to 202410_1_872_445 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:53.200538 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_872_445} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_871_444, 202410_872_872_0] -> 202410_1_872_445 2024.10.22 01:28:53.222260 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 773898 2024.10.22 01:28:53.251361 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:28:53.255412 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_865_865_0 (state Temporary) 2024.10.22 01:28:53.300969 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_865_865_0 to 202410_878_878_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:53.329711 [ 1160 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_872_445 to 202410_873_873_0 2024.10.22 01:28:53.338801 [ 1160 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.06 MiB using storage policy from min volume index 0 2024.10.22 01:28:53.346047 [ 1160 ] {} DiskLocal: Reserved 2.06 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:53.375017 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_873_446} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_872_445 to 202410_873_873_0 into Compact with storage Full 2024.10.22 01:28:53.455083 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} MergeTask::MergeProjectionsStage: Merge sorted 467 rows, containing 32 columns (32 merged, 0 gathered) in 5.18119071 sec., 90.13372140474635 rows/sec., 8.38 KiB/sec. 2024.10.22 01:28:53.457819 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_873_446} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:53.467666 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_873_446} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_872_445, total 57583 rows starting from the beginning of the part 2024.10.22 01:28:53.476691 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} MergedBlockOutputStream: filled checksums 202410_1_423_87 (state Temporary) 2024.10.22 01:28:53.480870 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_873_446} MergeTreeSequentialSource: Reading 2 marks from part 202410_873_873_0, total 46 rows starting from the beginning of the part 2024.10.22 01:28:53.482155 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 57293 2024.10.22 01:28:54.167438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(596d3a91e8d8e166ca408de45f488fe7) : [0, 746772]: Setting download as failed: File segment: [0, 746772], key: 596d3a91e8d8e166ca408de45f488fe7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:28:54.170661 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(596d3a91e8d8e166ca408de45f488fe7): No space left in cache to reserve 746773 bytes, will continue without cache download 2024.10.22 01:28:54.251856 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_775_383, 202410_776_776_0] 2024.10.22 01:28:54.253716 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_775_383, 202410_776_776_0] 2024.10.22 01:28:54.326456 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 5.57 MiB, will set to 1.65 GiB (RSS), difference: -11.97 MiB 2024.10.22 01:28:54.336175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 596d3a91e8d8e166ca408de45f488fe7, offset: 0, size: 0 2024.10.22 01:28:54.339117 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 596d3a91e8d8e166ca408de45f488fe7 for removal 2024.10.22 01:28:54.345046 [ 1476 ] {} CacheMetadata: Key 596d3a91e8d8e166ca408de45f488fe7 is removed from metadata 2024.10.22 01:28:54.366478 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_775_383, 202410_776_776_0] 2024.10.22 01:28:54.460214 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_423_87 to 202410_1_423_87 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:54.464843 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_423_87} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_418_86, 202410_423_423_0] -> 202410_1_423_87 2024.10.22 01:28:54.537383 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:28:54.929548 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_873_446/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:28:54.982310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e17d1336f7a88871cf7ca3f449823f64) : [0, 1901757]: Setting download as failed: File segment: [0, 1901757], key: e17d1336f7a88871cf7ca3f449823f64, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:28:54.983357 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e17d1336f7a88871cf7ca3f449823f64): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:28:55.154671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e17d1336f7a88871cf7ca3f449823f64, offset: 0, size: 0 2024.10.22 01:28:55.156156 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e17d1336f7a88871cf7ca3f449823f64 for removal 2024.10.22 01:28:55.158452 [ 1476 ] {} CacheMetadata: Key e17d1336f7a88871cf7ca3f449823f64 is removed from metadata 2024.10.22 01:28:55.167175 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_461_92, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0, 202410_465_465_0, 202410_466_466_0] 2024.10.22 01:28:55.168714 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_461_92, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0, 202410_465_465_0, 202410_466_466_0] 2024.10.22 01:28:55.210249 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_461_92, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0, 202410_465_465_0, 202410_466_466_0] 2024.10.22 01:28:55.663887 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 432 2024.10.22 01:28:55.766470 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 4.47 MiB, will set to 1.66 GiB (RSS), difference: 13.54 MiB 2024.10.22 01:28:55.930351 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 128980 bytes for dd6827d415687440e897fc3933ff3664:0) 2024.10.22 01:28:55.934478 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cf1c00746390769722ead9a90b542b9d, offset: 0, size: 271939 2024.10.22 01:28:55.946158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cf1c00746390769722ead9a90b542b9d for removal 2024.10.22 01:28:55.950181 [ 1476 ] {} CacheMetadata: Key cf1c00746390769722ead9a90b542b9d is removed from metadata 2024.10.22 01:28:56.052100 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0cbdc1ed77061f78911672597b918c6e, offset: 0, size: 232 2024.10.22 01:28:56.055056 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0cbdc1ed77061f78911672597b918c6e for removal 2024.10.22 01:28:56.058569 [ 1476 ] {} CacheMetadata: Key 0cbdc1ed77061f78911672597b918c6e is removed from metadata 2024.10.22 01:28:56.128305 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:56.134704 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:56.247057 [ 1717 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pkc/cylnveqnofeswefoxymhfnixbrkmc, size 3519606 2024.10.22 01:28:56.302925 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_414_414_0 (state Temporary) 2024.10.22 01:28:56.473737 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:28:56.479181 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:28:56.489168 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:28:56.492322 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_414_414_0 to 202410_424_424_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:56.579285 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 432 2024.10.22 01:28:56.597876 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:28:56.639873 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:28:56.658552 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:28:56.677221 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:28:56.679220 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_873_446} MergeTask::MergeProjectionsStage: Merge sorted 57629 rows, containing 13 columns (13 merged, 0 gathered) in 3.209507603 sec., 17955.713812963975 rows/sec., 2.86 MiB/sec. 2024.10.22 01:28:56.699741 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:28:56.736090 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:28:56.773109 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_873_446} MergedBlockOutputStream: filled checksums 202410_1_873_446 (state Temporary) 2024.10.22 01:28:56.850529 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_873_446} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_873_446 to 202410_1_873_446 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:56.852969 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_873_446} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_872_445, 202410_873_873_0] -> 202410_1_873_446 2024.10.22 01:28:56.874379 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:28:56.966829 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kqn/xtkixutlmufgqmnmxqqcphcaxkrjw (LogSeriesLimiter: on interval from 2024-10-22 01:28:37 to 2024-10-22 01:28:56 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:28:57.165775 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_445_223, 202410_446_446_0] 2024.10.22 01:28:57.169297 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_445_223, 202410_446_446_0] 2024.10.22 01:28:57.292576 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_445_223, 202410_446_446_0] 2024.10.22 01:28:57.320883 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 6.06 MiB, will set to 1.64 GiB (RSS), difference: -3.54 MiB 2024.10.22 01:28:57.452786 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(45d81d8220c17e0cadecfffbb0af1127) : [0, 2001596]: Setting download as failed: File segment: [0, 2001596], key: 45d81d8220c17e0cadecfffbb0af1127, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:28:57.456627 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(45d81d8220c17e0cadecfffbb0af1127): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:28:57.509752 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 57389 2024.10.22 01:28:57.714397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 45d81d8220c17e0cadecfffbb0af1127, offset: 0, size: 0 2024.10.22 01:28:57.716014 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 45d81d8220c17e0cadecfffbb0af1127 for removal 2024.10.22 01:28:57.718362 [ 1476 ] {} CacheMetadata: Key 45d81d8220c17e0cadecfffbb0af1127 is removed from metadata 2024.10.22 01:28:57.902523 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:28:57.906758 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:57.965914 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_866_866_0 (state Temporary) 2024.10.22 01:28:58.114542 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_866_866_0 to 202410_879_879_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:28:58.149803 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_873_446 to 202410_874_874_0 2024.10.22 01:28:58.153670 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 57389 2024.10.22 01:28:58.157011 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.06 MiB using storage policy from min volume index 0 2024.10.22 01:28:58.159803 [ 1184 ] {} DiskLocal: Reserved 2.06 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:28:58.176253 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_874_447} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_873_446 to 202410_874_874_0 into Compact with storage Full 2024.10.22 01:28:58.205085 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_874_447} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:28:58.211284 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_874_447} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_873_446, total 57629 rows starting from the beginning of the part 2024.10.22 01:28:58.225089 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_874_447} MergeTreeSequentialSource: Reading 2 marks from part 202410_874_874_0, total 55 rows starting from the beginning of the part 2024.10.22 01:28:58.324854 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 7.78 MiB, will set to 1.65 GiB (RSS), difference: -2.81 MiB 2024.10.22 01:28:58.337759 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 6 file segments (while reserving 145348 bytes for 3870c5f39c03d99863dbb22a952e1416:0) 2024.10.22 01:28:58.343298 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dd6827d415687440e897fc3933ff3664, offset: 0, size: 128980 2024.10.22 01:28:58.352859 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dd6827d415687440e897fc3933ff3664 for removal 2024.10.22 01:28:58.357575 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2a7a677c597c0072d331451b464e80db, offset: 0, size: 164 2024.10.22 01:28:58.363577 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2a7a677c597c0072d331451b464e80db for removal 2024.10.22 01:28:58.370322 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 952629c2c79946d63224a252df85aeb6, offset: 0, size: 257 2024.10.22 01:28:58.372988 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 952629c2c79946d63224a252df85aeb6 for removal 2024.10.22 01:28:58.376488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6492143357ef5cbb0ea2d1ea47791691, offset: 0, size: 227 2024.10.22 01:28:58.380831 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6492143357ef5cbb0ea2d1ea47791691 for removal 2024.10.22 01:28:58.385323 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 860b7dc66e2eb32708b2e1f1383fa248, offset: 0, size: 97639 2024.10.22 01:28:58.394450 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 860b7dc66e2eb32708b2e1f1383fa248 for removal 2024.10.22 01:28:58.400204 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 998d1176bf6d0e15eedd474488f5eed0, offset: 0, size: 161 2024.10.22 01:28:58.409704 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 998d1176bf6d0e15eedd474488f5eed0 for removal 2024.10.22 01:28:58.455721 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_429_87, 202410_430_435_1, 202410_436_436_0, 202410_437_437_0, 202410_438_438_0, 202410_439_439_0] 2024.10.22 01:28:58.459162 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_429_87, 202410_430_435_1, 202410_436_436_0, 202410_437_437_0, 202410_438_438_0, 202410_439_439_0] 2024.10.22 01:28:58.460406 [ 1476 ] {} CacheMetadata: Key dd6827d415687440e897fc3933ff3664 is removed from metadata 2024.10.22 01:28:58.470792 [ 1476 ] {} CacheMetadata: Key 998d1176bf6d0e15eedd474488f5eed0 is removed from metadata 2024.10.22 01:28:58.485625 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_429_87, 202410_430_435_1, 202410_436_436_0, 202410_437_437_0, 202410_438_438_0, 202410_439_439_0] 2024.10.22 01:28:58.494540 [ 1476 ] {} CacheMetadata: Key 860b7dc66e2eb32708b2e1f1383fa248 is removed from metadata 2024.10.22 01:28:58.508210 [ 1476 ] {} CacheMetadata: Key 6492143357ef5cbb0ea2d1ea47791691 is removed from metadata 2024.10.22 01:28:58.575279 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_776_384, 202410_777_777_0] 2024.10.22 01:28:58.612497 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_776_384, 202410_777_777_0] 2024.10.22 01:28:58.627817 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_776_384, 202410_777_777_0] 2024.10.22 01:28:58.731796 [ 1476 ] {} CacheMetadata: Key 952629c2c79946d63224a252df85aeb6 is removed from metadata 2024.10.22 01:28:58.764286 [ 1476 ] {} CacheMetadata: Key 2a7a677c597c0072d331451b464e80db is removed from metadata 2024.10.22 01:28:59.444815 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 5 file segments (while reserving 144760 bytes for b803aa50d39d40835b49285c2223850c:0) 2024.10.22 01:28:59.459776 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3336f71fad89f7fc0b9bef7c9e03ef43, offset: 0, size: 162 2024.10.22 01:28:59.464268 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3336f71fad89f7fc0b9bef7c9e03ef43 for removal 2024.10.22 01:28:59.467267 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c0299feae60017f7f6506f620162afa3, offset: 0, size: 171552 2024.10.22 01:28:59.467136 [ 1476 ] {} CacheMetadata: Key 3336f71fad89f7fc0b9bef7c9e03ef43 is removed from metadata 2024.10.22 01:28:59.469396 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c0299feae60017f7f6506f620162afa3 for removal 2024.10.22 01:28:59.472879 [ 1476 ] {} CacheMetadata: Key c0299feae60017f7f6506f620162afa3 is removed from metadata 2024.10.22 01:28:59.475057 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 402b5cc5bfb4cdea1007343cb0b68376, offset: 0, size: 258 2024.10.22 01:28:59.480535 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 402b5cc5bfb4cdea1007343cb0b68376 for removal 2024.10.22 01:28:59.484512 [ 1476 ] {} CacheMetadata: Key 402b5cc5bfb4cdea1007343cb0b68376 is removed from metadata 2024.10.22 01:28:59.486669 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 01e74a7704d696adba76f5f13680a689, offset: 0, size: 5035 2024.10.22 01:28:59.495559 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 01e74a7704d696adba76f5f13680a689 for removal 2024.10.22 01:28:59.503059 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 831837ea0b9ab38ef4ca6755bd97757f, offset: 0, size: 159 2024.10.22 01:28:59.503589 [ 1476 ] {} CacheMetadata: Key 01e74a7704d696adba76f5f13680a689 is removed from metadata 2024.10.22 01:28:59.507741 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 831837ea0b9ab38ef4ca6755bd97757f for removal 2024.10.22 01:28:59.512652 [ 1476 ] {} CacheMetadata: Key 831837ea0b9ab38ef4ca6755bd97757f is removed from metadata 2024.10.22 01:29:00.230326 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 775823 2024.10.22 01:29:00.265428 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 7930 2024.10.22 01:29:00.339250 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 153 entries to flush up to offset 59524 2024.10.22 01:29:00.360633 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:29:00.363842 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:29:00.379397 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 6.86 MiB, will set to 1.67 GiB (RSS), difference: 2.12 MiB 2024.10.22 01:29:00.584116 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:00.587592 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:00.764867 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_513_513_0 (state Temporary) 2024.10.22 01:29:00.809602 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 257283 bytes for 7438dca0c0afb81a77f08a480c741704:0) 2024.10.22 01:29:00.819962 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3870c5f39c03d99863dbb22a952e1416, offset: 0, size: 145348 2024.10.22 01:29:00.822107 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_874_447} MergeTask::MergeProjectionsStage: Merge sorted 57684 rows, containing 13 columns (13 merged, 0 gathered) in 2.648429525 sec., 21780.454966042566 rows/sec., 3.47 MiB/sec. 2024.10.22 01:29:00.826931 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3870c5f39c03d99863dbb22a952e1416 for removal 2024.10.22 01:29:00.828585 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_513_513_0 to 202410_526_526_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:00.829661 [ 1476 ] {} CacheMetadata: Key 3870c5f39c03d99863dbb22a952e1416 is removed from metadata 2024.10.22 01:29:00.831116 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e7f307347daec4c31514b4e550198df8, offset: 0, size: 168 2024.10.22 01:29:00.835176 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e7f307347daec4c31514b4e550198df8 for removal 2024.10.22 01:29:00.880227 [ 1476 ] {} CacheMetadata: Key e7f307347daec4c31514b4e550198df8 is removed from metadata 2024.10.22 01:29:00.873525 [ 1212 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_521_104 to 202410_526_526_0 2024.10.22 01:29:00.917128 [ 1212 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:00.921684 [ 1212 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:00.985352 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:00.987357 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_874_447} MergedBlockOutputStream: filled checksums 202410_1_874_447 (state Temporary) 2024.10.22 01:29:00.994546 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_521_104 to 202410_526_526_0 into Compact with storage Full 2024.10.22 01:29:01.001992 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:01.152222 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7930 2024.10.22 01:29:01.171100 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:01.171081 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_503_503_0 (state Temporary) 2024.10.22 01:29:01.181569 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_521_104, total 7954 rows starting from the beginning of the part 2024.10.22 01:29:01.211966 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_522_522_0, total 19 rows starting from the beginning of the part 2024.10.22 01:29:01.248422 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_523_523_0, total 8 rows starting from the beginning of the part 2024.10.22 01:29:01.257028 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_104_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:29:01.268743 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_524_524_0, total 28 rows starting from the beginning of the part 2024.10.22 01:29:01.285162 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_525_525_0, total 8 rows starting from the beginning of the part 2024.10.22 01:29:01.289552 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_874_447} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_874_447 to 202410_1_874_447 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:01.311654 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_874_447} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_873_446, 202410_874_874_0] -> 202410_1_874_447 2024.10.22 01:29:01.345543 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:29:01.405221 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column Refresh 2024.10.22 01:29:01.419378 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column Refresh 2024.10.22 01:29:01.454451 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_526_526_0, total 19 rows starting from the beginning of the part 2024.10.22 01:29:01.475908 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_503_503_0 to 202410_509_509_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:01.538417 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 59524 2024.10.22 01:29:01.557340 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column Refresh 2024.10.22 01:29:01.568041 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column Refresh 2024.10.22 01:29:01.588156 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column Refresh 2024.10.22 01:29:01.663671 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column Refresh 2024.10.22 01:29:01.729668 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 113044 bytes for 6b8c11b8e1119d9b2387b5ca71244cad:0) 2024.10.22 01:29:01.730074 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_103_103_0 to 202410_106_106_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:01.733466 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b803aa50d39d40835b49285c2223850c, offset: 0, size: 144760 2024.10.22 01:29:01.737279 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b803aa50d39d40835b49285c2223850c for removal 2024.10.22 01:29:01.740477 [ 1476 ] {} CacheMetadata: Key b803aa50d39d40835b49285c2223850c is removed from metadata 2024.10.22 01:29:01.741390 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 61390a56822dec998a518a1329158ff5, offset: 0, size: 167 2024.10.22 01:29:01.770994 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:01.773403 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:01.808863 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column Refresh 2024.10.22 01:29:01.838611 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column Refresh 2024.10.22 01:29:01.852086 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 61390a56822dec998a518a1329158ff5 for removal 2024.10.22 01:29:01.856185 [ 1476 ] {} CacheMetadata: Key 61390a56822dec998a518a1329158ff5 is removed from metadata 2024.10.22 01:29:02.012224 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_492_492_0 (state Temporary) 2024.10.22 01:29:02.109501 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jud/qisashfzuxqifyrvhfepwvrdkiyuc (LogSeriesLimiter: on interval from 2024-10-22 01:28:56 to 2024-10-22 01:29:02 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:29:02.161371 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 118 entries to flush up to offset 57507 2024.10.22 01:29:02.308245 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 6.61 MiB, will set to 1.66 GiB (RSS), difference: -2.77 MiB 2024.10.22 01:29:02.310113 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_492_492_0 to 202410_499_499_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:02.354057 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 775823 2024.10.22 01:29:02.682469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c481c66a9fd41d8fa0c8eda9522e103b) : [0, 572143]: Setting download as failed: File segment: [0, 572143], key: c481c66a9fd41d8fa0c8eda9522e103b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:02.685643 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c481c66a9fd41d8fa0c8eda9522e103b): No space left in cache to reserve 572144 bytes, will continue without cache download 2024.10.22 01:29:02.769805 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c481c66a9fd41d8fa0c8eda9522e103b, offset: 0, size: 0 2024.10.22 01:29:02.772229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c481c66a9fd41d8fa0c8eda9522e103b for removal 2024.10.22 01:29:02.774358 [ 1476 ] {} CacheMetadata: Key c481c66a9fd41d8fa0c8eda9522e103b is removed from metadata 2024.10.22 01:29:02.776787 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_777_385, 202410_778_778_0] 2024.10.22 01:29:02.778689 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_777_385, 202410_778_778_0] 2024.10.22 01:29:02.791590 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_777_385, 202410_778_778_0] 2024.10.22 01:29:02.852412 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:02.855466 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:02.932741 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_867_867_0 (state Temporary) 2024.10.22 01:29:03.110222 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_867_867_0 to 202410_880_880_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:03.162886 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 57507 2024.10.22 01:29:03.152899 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_874_447 to 202410_875_875_0 2024.10.22 01:29:03.174410 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.06 MiB using storage policy from min volume index 0 2024.10.22 01:29:03.179301 [ 1244 ] {} DiskLocal: Reserved 2.06 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:03.212498 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(76d9b3500f21e8cdb0626ffa5cbaa840) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 76d9b3500f21e8cdb0626ffa5cbaa840, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:29:03.217122 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 76d9b3500f21e8cdb0626ffa5cbaa840, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:29:03.237209 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:29:03.241834 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_875_448} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_874_447 to 202410_875_875_0 into Compact with storage Full 2024.10.22 01:29:03.248727 [ 1245 ] {} DatabaseCatalog: Execution took 262 ms. 2024.10.22 01:29:03.284984 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_875_448} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:03.296657 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_526_105/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:29:03.297031 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_875_448} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_874_447, total 57684 rows starting from the beginning of the part 2024.10.22 01:29:03.324053 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_875_448} MergeTreeSequentialSource: Reading 2 marks from part 202410_875_875_0, total 98 rows starting from the beginning of the part 2024.10.22 01:29:03.360337 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} MergeTask::MergeProjectionsStage: Merge sorted 8036 rows, containing 12 columns (12 merged, 0 gathered) in 2.377162 sec., 3380.501623364331 rows/sec., 537.19 KiB/sec. 2024.10.22 01:29:03.519255 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} MergedBlockOutputStream: filled checksums 202410_1_526_105 (state Temporary) 2024.10.22 01:29:04.025203 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 355250 bytes for de7803771ffd535aeae19731e06f700e:0) 2024.10.22 01:29:04.028177 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6b8c11b8e1119d9b2387b5ca71244cad, offset: 0, size: 113044 2024.10.22 01:29:04.033787 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6b8c11b8e1119d9b2387b5ca71244cad for removal 2024.10.22 01:29:04.044719 [ 1476 ] {} CacheMetadata: Key 6b8c11b8e1119d9b2387b5ca71244cad is removed from metadata 2024.10.22 01:29:04.044774 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7438dca0c0afb81a77f08a480c741704, offset: 0, size: 257283 2024.10.22 01:29:04.050795 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7438dca0c0afb81a77f08a480c741704 for removal 2024.10.22 01:29:04.054681 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 643cd1b0cdb3e06bdcc92839a0796010, offset: 0, size: 165 2024.10.22 01:29:04.057224 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 643cd1b0cdb3e06bdcc92839a0796010 for removal 2024.10.22 01:29:04.060131 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6c6c9b3f93fd7ca7fbd003b08221495a, offset: 0, size: 190 2024.10.22 01:29:04.063495 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6c6c9b3f93fd7ca7fbd003b08221495a for removal 2024.10.22 01:29:04.071744 [ 1476 ] {} CacheMetadata: Key 6c6c9b3f93fd7ca7fbd003b08221495a is removed from metadata 2024.10.22 01:29:04.080912 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 433 2024.10.22 01:29:04.095504 [ 1476 ] {} CacheMetadata: Key 643cd1b0cdb3e06bdcc92839a0796010 is removed from metadata 2024.10.22 01:29:04.120658 [ 1476 ] {} CacheMetadata: Key 7438dca0c0afb81a77f08a480c741704 is removed from metadata 2024.10.22 01:29:04.359004 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:04.361106 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:04.388624 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_526_105 to 202410_1_526_105 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:04.398646 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_526_105} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_521_104, 202410_526_526_0] -> 202410_1_526_105 2024.10.22 01:29:04.427246 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_415_415_0 (state Temporary) 2024.10.22 01:29:04.458897 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.48 MiB. 2024.10.22 01:29:04.473194 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 6.50 MiB, will set to 1.67 GiB (RSS), difference: 5.80 MiB 2024.10.22 01:29:04.629499 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_415_415_0 to 202410_425_425_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:04.702712 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 433 2024.10.22 01:29:05.534703 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4132 2024.10.22 01:29:05.559112 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 246671 bytes for 57043da8bc65e73a99c5fcdc9602ab2f:0) 2024.10.22 01:29:05.563204 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: de7803771ffd535aeae19731e06f700e, offset: 0, size: 355250 2024.10.22 01:29:05.567392 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key de7803771ffd535aeae19731e06f700e for removal 2024.10.22 01:29:05.569590 [ 1476 ] {} CacheMetadata: Key de7803771ffd535aeae19731e06f700e is removed from metadata 2024.10.22 01:29:05.569663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 165dd4a7ba806e7b9b1b0c81c6087ff2, offset: 0, size: 233 2024.10.22 01:29:05.573716 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 165dd4a7ba806e7b9b1b0c81c6087ff2 for removal 2024.10.22 01:29:05.576068 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 11e285612c49e89d0046a681ea2709c4, offset: 0, size: 238 2024.10.22 01:29:05.579433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 11e285612c49e89d0046a681ea2709c4 for removal 2024.10.22 01:29:05.581510 [ 1476 ] {} CacheMetadata: Key 11e285612c49e89d0046a681ea2709c4 is removed from metadata 2024.10.22 01:29:05.702226 [ 1476 ] {} CacheMetadata: Key 165dd4a7ba806e7b9b1b0c81c6087ff2 is removed from metadata 2024.10.22 01:29:06.525811 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 15.37 MiB, will set to 1.65 GiB (RSS), difference: 18.51 MiB 2024.10.22 01:29:06.531115 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 238176 bytes for efaed17604e5af3d7ea2cd8bae87a42f:0) 2024.10.22 01:29:06.556891 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_446_224, 202410_447_447_0] 2024.10.22 01:29:06.558623 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_446_224, 202410_447_447_0] 2024.10.22 01:29:06.573257 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_446_224, 202410_447_447_0] 2024.10.22 01:29:06.584532 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_875_448} MergeTask::MergeProjectionsStage: Merge sorted 57782 rows, containing 13 columns (13 merged, 0 gathered) in 3.362725002 sec., 17183.088110277775 rows/sec., 2.73 MiB/sec. 2024.10.22 01:29:06.650450 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 16ac759e6b895837467961aa2ee4177d, offset: 0, size: 192 2024.10.22 01:29:06.650912 [ 1274 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:29:06.654989 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 16ac759e6b895837467961aa2ee4177d for removal 2024.10.22 01:29:06.660002 [ 1476 ] {} CacheMetadata: Key 16ac759e6b895837467961aa2ee4177d is removed from metadata 2024.10.22 01:29:06.661318 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a786cf5ee9803871385effd6a7162a6c, offset: 0, size: 9508 2024.10.22 01:29:06.665503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a786cf5ee9803871385effd6a7162a6c for removal 2024.10.22 01:29:06.669293 [ 1476 ] {} CacheMetadata: Key a786cf5ee9803871385effd6a7162a6c is removed from metadata 2024.10.22 01:29:06.670440 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 57043da8bc65e73a99c5fcdc9602ab2f, offset: 0, size: 246671 2024.10.22 01:29:06.675235 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 57043da8bc65e73a99c5fcdc9602ab2f for removal 2024.10.22 01:29:06.680542 [ 1476 ] {} CacheMetadata: Key 57043da8bc65e73a99c5fcdc9602ab2f is removed from metadata 2024.10.22 01:29:06.681064 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f9ad161cd15bef4f83539e749bbe7345, offset: 0, size: 224 2024.10.22 01:29:06.685310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f9ad161cd15bef4f83539e749bbe7345 for removal 2024.10.22 01:29:06.685514 [ 1274 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:29:06.738346 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_875_448} MergedBlockOutputStream: filled checksums 202410_1_875_448 (state Temporary) 2024.10.22 01:29:06.807507 [ 1476 ] {} CacheMetadata: Key f9ad161cd15bef4f83539e749bbe7345 is removed from metadata 2024.10.22 01:29:06.907066 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_875_448} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_875_448 to 202410_1_875_448 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:06.927768 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_875_448} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_874_447, 202410_875_875_0] -> 202410_1_875_448 2024.10.22 01:29:07.056996 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:29:07.174905 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 57596 2024.10.22 01:29:07.239529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0ef75c1d6757d042365e601020728328) : [0, 465693]: Setting download as failed: File segment: [0, 465693], key: 0ef75c1d6757d042365e601020728328, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:07.242564 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0ef75c1d6757d042365e601020728328): No space left in cache to reserve 465694 bytes, will continue without cache download 2024.10.22 01:29:07.478040 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0ef75c1d6757d042365e601020728328, offset: 0, size: 0 2024.10.22 01:29:07.479658 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0ef75c1d6757d042365e601020728328 for removal 2024.10.22 01:29:07.481895 [ 1476 ] {} CacheMetadata: Key 0ef75c1d6757d042365e601020728328 is removed from metadata 2024.10.22 01:29:07.588820 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:07.598640 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:07.776016 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_868_868_0 (state Temporary) 2024.10.22 01:29:07.837151 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_868_868_0 to 202410_881_881_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:07.865321 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_875_448 to 202410_876_876_0 2024.10.22 01:29:07.871125 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.07 MiB using storage policy from min volume index 0 2024.10.22 01:29:07.874556 [ 1248 ] {} DiskLocal: Reserved 2.07 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:07.898413 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_876_449} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_875_448 to 202410_876_876_0 into Compact with storage Full 2024.10.22 01:29:07.921034 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_876_449} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:07.982792 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 57596 2024.10.22 01:29:08.097177 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_876_449} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_875_448, total 57782 rows starting from the beginning of the part 2024.10.22 01:29:08.325673 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_876_449} MergeTreeSequentialSource: Reading 2 marks from part 202410_876_876_0, total 99 rows starting from the beginning of the part 2024.10.22 01:29:08.379468 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ab63bc49e2eb5ef7292bf2fecfc2a9af) : [0, 605805]: Setting download as failed: File segment: [0, 605805], key: ab63bc49e2eb5ef7292bf2fecfc2a9af, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:08.382359 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ab63bc49e2eb5ef7292bf2fecfc2a9af): No space left in cache to reserve 605806 bytes, will continue without cache download 2024.10.22 01:29:08.455244 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 14.06 MiB, will set to 1.65 GiB (RSS), difference: -5.82 MiB 2024.10.22 01:29:08.652596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ab63bc49e2eb5ef7292bf2fecfc2a9af, offset: 0, size: 0 2024.10.22 01:29:08.654973 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ab63bc49e2eb5ef7292bf2fecfc2a9af for removal 2024.10.22 01:29:08.655133 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 7950 2024.10.22 01:29:08.657353 [ 1476 ] {} CacheMetadata: Key ab63bc49e2eb5ef7292bf2fecfc2a9af is removed from metadata 2024.10.22 01:29:08.853295 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:08.857160 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:08.877757 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_514_514_0 (state Temporary) 2024.10.22 01:29:09.108464 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 136 entries to flush up to offset 59660 2024.10.22 01:29:09.161239 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_514_514_0 to 202410_527_527_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:09.181713 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_778_386, 202410_779_779_0] 2024.10.22 01:29:09.184287 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_778_386, 202410_779_779_0] 2024.10.22 01:29:09.198882 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_778_386, 202410_779_779_0] 2024.10.22 01:29:09.215017 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7950 2024.10.22 01:29:09.353437 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(72846828d6e854f68f952f7b0fe6e86f) : [0, 1024723]: Setting download as failed: File segment: [0, 1024723], key: 72846828d6e854f68f952f7b0fe6e86f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:09.354876 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(72846828d6e854f68f952f7b0fe6e86f): No space left in cache to reserve 1024724 bytes, will continue without cache download 2024.10.22 01:29:09.459534 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 777748 2024.10.22 01:29:09.608877 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 72846828d6e854f68f952f7b0fe6e86f, offset: 0, size: 0 2024.10.22 01:29:09.612906 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 72846828d6e854f68f952f7b0fe6e86f for removal 2024.10.22 01:29:09.615517 [ 1476 ] {} CacheMetadata: Key 72846828d6e854f68f952f7b0fe6e86f is removed from metadata 2024.10.22 01:29:09.621378 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:09.624565 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:09.765802 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_504_504_0 (state Temporary) 2024.10.22 01:29:09.831928 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_504_504_0 to 202410_510_510_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:09.969382 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 59660 2024.10.22 01:29:10.361178 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:29:10.364460 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:29:10.460661 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f88ec58da4b65ec8826f9c96b727238a) : [0, 1023074]: Setting download as failed: File segment: [0, 1023074], key: f88ec58da4b65ec8826f9c96b727238a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:10.462287 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f88ec58da4b65ec8826f9c96b727238a): No space left in cache to reserve 1023075 bytes, will continue without cache download 2024.10.22 01:29:10.464957 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:10.578133 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:10.615624 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f88ec58da4b65ec8826f9c96b727238a, offset: 0, size: 0 2024.10.22 01:29:10.620125 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f88ec58da4b65ec8826f9c96b727238a for removal 2024.10.22 01:29:10.622970 [ 1476 ] {} CacheMetadata: Key f88ec58da4b65ec8826f9c96b727238a is removed from metadata 2024.10.22 01:29:10.701081 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 6.90 MiB, will set to 1.67 GiB (RSS), difference: 1.18 MiB 2024.10.22 01:29:10.985131 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_493_493_0 (state Temporary) 2024.10.22 01:29:11.052192 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_493_493_0 to 202410_500_500_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:11.079344 [ 1238 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_490_99 to 202410_491_496_1 2024.10.22 01:29:11.091743 [ 1238 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:11.095803 [ 1238 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:11.122462 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_496_100} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_490_99 to 202410_491_496_1 into Wide with storage Full 2024.10.22 01:29:11.182396 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 777748 2024.10.22 01:29:11.251225 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_496_100} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:11.251319 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_876_449/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:29:11.256855 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_496_100} MergeTreeSequentialSource: Reading 96 marks from part 202410_1_490_99, total 774783 rows starting from the beginning of the part 2024.10.22 01:29:11.285552 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_496_100} MergeTreeSequentialSource: Reading 3 marks from part 202410_491_496_1, total 9625 rows starting from the beginning of the part 2024.10.22 01:29:11.316586 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_876_449} MergeTask::MergeProjectionsStage: Merge sorted 57881 rows, containing 13 columns (13 merged, 0 gathered) in 3.435509791 sec., 16847.86349659977 rows/sec., 2.68 MiB/sec. 2024.10.22 01:29:11.474228 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_876_449} MergedBlockOutputStream: filled checksums 202410_1_876_449 (state Temporary) 2024.10.22 01:29:11.590426 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_876_449} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_876_449 to 202410_1_876_449 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:11.634518 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(442db8194bbbcf0530f52a2f68bedf77) : [0, 656232]: Setting download as failed: File segment: [0, 656232], key: 442db8194bbbcf0530f52a2f68bedf77, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:11.636707 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(442db8194bbbcf0530f52a2f68bedf77): No space left in cache to reserve 656233 bytes, will continue without cache download 2024.10.22 01:29:11.670756 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 442db8194bbbcf0530f52a2f68bedf77, offset: 0, size: 0 2024.10.22 01:29:11.678193 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 442db8194bbbcf0530f52a2f68bedf77 for removal 2024.10.22 01:29:11.687881 [ 1476 ] {} CacheMetadata: Key 442db8194bbbcf0530f52a2f68bedf77 is removed from metadata 2024.10.22 01:29:11.697709 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_876_449} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_875_448, 202410_876_876_0] -> 202410_1_876_449 2024.10.22 01:29:11.720531 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:29:11.998544 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 57672 2024.10.22 01:29:12.208262 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 434 2024.10.22 01:29:12.256985 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:12.259230 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:12.263878 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 413871 bytes for 536c00bb996fa5d04d85336ac29f5bc1:0) 2024.10.22 01:29:12.265463 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: efaed17604e5af3d7ea2cd8bae87a42f, offset: 0, size: 238176 2024.10.22 01:29:12.267886 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key efaed17604e5af3d7ea2cd8bae87a42f for removal 2024.10.22 01:29:12.268631 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_104_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:29:12.269479 [ 1476 ] {} CacheMetadata: Key efaed17604e5af3d7ea2cd8bae87a42f is removed from metadata 2024.10.22 01:29:12.270842 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 278354ee5e4430a5c4695e407189e5f8, offset: 0, size: 230 2024.10.22 01:29:12.272782 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 278354ee5e4430a5c4695e407189e5f8 for removal 2024.10.22 01:29:12.277442 [ 1476 ] {} CacheMetadata: Key 278354ee5e4430a5c4695e407189e5f8 is removed from metadata 2024.10.22 01:29:12.306450 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_869_869_0 (state Temporary) 2024.10.22 01:29:12.374512 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_869_869_0 to 202410_882_882_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:12.383639 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:12.385630 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:12.411063 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 8.97 MiB, will set to 1.66 GiB (RSS), difference: -13.79 MiB 2024.10.22 01:29:12.426079 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_876_449 to 202410_877_877_0 2024.10.22 01:29:12.431987 [ 1265 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.08 MiB using storage policy from min volume index 0 2024.10.22 01:29:12.440663 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_416_416_0 (state Temporary) 2024.10.22 01:29:12.442423 [ 1265 ] {} DiskLocal: Reserved 2.08 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:12.453326 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 57672 2024.10.22 01:29:12.505724 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_416_416_0 to 202410_426_426_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:12.671587 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_877_450} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_876_449 to 202410_877_877_0 into Compact with storage Full 2024.10.22 01:29:12.849039 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_877_450} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:12.881244 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_877_450} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_876_449, total 57881 rows starting from the beginning of the part 2024.10.22 01:29:12.918572 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 434 2024.10.22 01:29:12.948670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1a758d21006cceab50f0c2ed5d9fbf6f) : [0, 714720]: Setting download as failed: File segment: [0, 714720], key: 1a758d21006cceab50f0c2ed5d9fbf6f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:12.950788 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1a758d21006cceab50f0c2ed5d9fbf6f): No space left in cache to reserve 714721 bytes, will continue without cache download 2024.10.22 01:29:12.992134 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1a758d21006cceab50f0c2ed5d9fbf6f, offset: 0, size: 0 2024.10.22 01:29:13.092952 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 4172 2024.10.22 01:29:13.096614 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1a758d21006cceab50f0c2ed5d9fbf6f for removal 2024.10.22 01:29:13.100721 [ 1476 ] {} CacheMetadata: Key 1a758d21006cceab50f0c2ed5d9fbf6f is removed from metadata 2024.10.22 01:29:13.166950 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_877_450} MergeTreeSequentialSource: Reading 2 marks from part 202410_877_877_0, total 124 rows starting from the beginning of the part 2024.10.22 01:29:13.252878 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_447_225, 202410_448_448_0] 2024.10.22 01:29:13.255130 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_447_225, 202410_448_448_0] 2024.10.22 01:29:13.407354 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_447_225, 202410_448_448_0] 2024.10.22 01:29:13.463209 [ 1173 ] {} DatabaseCatalog: Execution took 212 ms. 2024.10.22 01:29:13.547735 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_779_387, 202410_780_780_0] 2024.10.22 01:29:13.650469 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_779_387, 202410_780_780_0] 2024.10.22 01:29:13.665669 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_779_387, 202410_780_780_0] 2024.10.22 01:29:13.753748 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2c66868eb2f13a751636deb187a279c9) : [0, 543666]: Setting download as failed: File segment: [0, 543666], key: 2c66868eb2f13a751636deb187a279c9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:13.765729 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2c66868eb2f13a751636deb187a279c9): No space left in cache to reserve 543667 bytes, will continue without cache download 2024.10.22 01:29:13.826947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2c66868eb2f13a751636deb187a279c9, offset: 0, size: 0 2024.10.22 01:29:13.830653 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2c66868eb2f13a751636deb187a279c9 for removal 2024.10.22 01:29:13.838390 [ 1476 ] {} CacheMetadata: Key 2c66868eb2f13a751636deb187a279c9 is removed from metadata 2024.10.22 01:29:14.080368 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_496_100/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:29:14.426989 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 19.59 MiB, will set to 1.65 GiB (RSS), difference: -2.27 MiB 2024.10.22 01:29:14.432988 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 7 file segments (while reserving 307683 bytes for fe746f4d3a922e39b6c272147cd85ad9:0) 2024.10.22 01:29:14.441780 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 536c00bb996fa5d04d85336ac29f5bc1, offset: 0, size: 413871 2024.10.22 01:29:14.449792 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 536c00bb996fa5d04d85336ac29f5bc1 for removal 2024.10.22 01:29:14.453651 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ca2f0ddb12fb81ca23d03f0acaffdd18, offset: 0, size: 239 2024.10.22 01:29:14.458624 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ca2f0ddb12fb81ca23d03f0acaffdd18 for removal 2024.10.22 01:29:14.460272 [ 1476 ] {} CacheMetadata: Key 536c00bb996fa5d04d85336ac29f5bc1 is removed from metadata 2024.10.22 01:29:14.462019 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 356aa4c5b3b1c1d433c4f65ffaba16a9, offset: 0, size: 238 2024.10.22 01:29:14.463967 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 356aa4c5b3b1c1d433c4f65ffaba16a9 for removal 2024.10.22 01:29:14.465438 [ 1476 ] {} CacheMetadata: Key 356aa4c5b3b1c1d433c4f65ffaba16a9 is removed from metadata 2024.10.22 01:29:14.465438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5b1eb4b271f6232240fe4a3c9f333192, offset: 0, size: 235 2024.10.22 01:29:14.467359 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5b1eb4b271f6232240fe4a3c9f333192 for removal 2024.10.22 01:29:14.478778 [ 1476 ] {} CacheMetadata: Key 5b1eb4b271f6232240fe4a3c9f333192 is removed from metadata 2024.10.22 01:29:14.480000 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a8752b91629c1cd6d526fc2ac5796c79, offset: 0, size: 246 2024.10.22 01:29:14.485097 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a8752b91629c1cd6d526fc2ac5796c79 for removal 2024.10.22 01:29:14.486064 [ 1476 ] {} CacheMetadata: Key ca2f0ddb12fb81ca23d03f0acaffdd18 is removed from metadata 2024.10.22 01:29:14.498884 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 05ccb35e069002b193b612322f046d7e, offset: 0, size: 246 2024.10.22 01:29:14.499449 [ 1476 ] {} CacheMetadata: Key a8752b91629c1cd6d526fc2ac5796c79 is removed from metadata 2024.10.22 01:29:14.505266 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 05ccb35e069002b193b612322f046d7e for removal 2024.10.22 01:29:14.510472 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e00926c741bc70b745963176650bef2b, offset: 0, size: 232 2024.10.22 01:29:14.511370 [ 1476 ] {} CacheMetadata: Key 05ccb35e069002b193b612322f046d7e is removed from metadata 2024.10.22 01:29:14.516008 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e00926c741bc70b745963176650bef2b for removal 2024.10.22 01:29:14.525612 [ 1476 ] {} CacheMetadata: Key e00926c741bc70b745963176650bef2b is removed from metadata 2024.10.22 01:29:15.452890 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 308676 bytes for c851ec41d21af1a13ab9c63a47697e7e:0) 2024.10.22 01:29:15.454754 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fe746f4d3a922e39b6c272147cd85ad9, offset: 0, size: 307683 2024.10.22 01:29:15.458375 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fe746f4d3a922e39b6c272147cd85ad9 for removal 2024.10.22 01:29:15.462596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5871b523a96273c44197a7443c3d0b92, offset: 0, size: 232 2024.10.22 01:29:15.462648 [ 1476 ] {} CacheMetadata: Key fe746f4d3a922e39b6c272147cd85ad9 is removed from metadata 2024.10.22 01:29:15.465392 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5871b523a96273c44197a7443c3d0b92 for removal 2024.10.22 01:29:15.470734 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f60fc8986ec2aee66e03363baf97f355, offset: 0, size: 238 2024.10.22 01:29:15.474441 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f60fc8986ec2aee66e03363baf97f355 for removal 2024.10.22 01:29:15.476756 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cec32582f515186e206c9b7d7572c7f5, offset: 0, size: 241 2024.10.22 01:29:15.476807 [ 1476 ] {} CacheMetadata: Key f60fc8986ec2aee66e03363baf97f355 is removed from metadata 2024.10.22 01:29:15.479910 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cec32582f515186e206c9b7d7572c7f5 for removal 2024.10.22 01:29:15.486441 [ 1476 ] {} CacheMetadata: Key cec32582f515186e206c9b7d7572c7f5 is removed from metadata 2024.10.22 01:29:15.500204 [ 1476 ] {} CacheMetadata: Key 5871b523a96273c44197a7443c3d0b92 is removed from metadata 2024.10.22 01:29:16.466536 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 57755 2024.10.22 01:29:16.540065 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 18.91 MiB, will set to 1.65 GiB (RSS), difference: 7.88 MiB 2024.10.22 01:29:16.554741 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 252134 bytes for 01e64c8c7392acaab5f520f0944aa3d8:0) 2024.10.22 01:29:16.556916 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c851ec41d21af1a13ab9c63a47697e7e, offset: 0, size: 308676 2024.10.22 01:29:16.561622 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c851ec41d21af1a13ab9c63a47697e7e for removal 2024.10.22 01:29:16.664275 [ 1476 ] {} CacheMetadata: Key c851ec41d21af1a13ab9c63a47697e7e is removed from metadata 2024.10.22 01:29:16.664856 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8a6ea7bcdf6443571c5f8bb072267352, offset: 0, size: 229 2024.10.22 01:29:16.680890 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8a6ea7bcdf6443571c5f8bb072267352 for removal 2024.10.22 01:29:16.684859 [ 1476 ] {} CacheMetadata: Key 8a6ea7bcdf6443571c5f8bb072267352 is removed from metadata 2024.10.22 01:29:16.718335 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 7958 2024.10.22 01:29:16.788419 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:16.801676 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:16.891731 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:16.896510 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:16.993702 [ 1747 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kqn/xtkixutlmufgqmnmxqqcphcaxkrjw, size 5743662 2024.10.22 01:29:17.057258 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_515_515_0 (state Temporary) 2024.10.22 01:29:17.077571 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_870_870_0 (state Temporary) 2024.10.22 01:29:17.112088 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_515_515_0 to 202410_528_528_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:17.210046 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7958 2024.10.22 01:29:17.277275 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_870_870_0 to 202410_883_883_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:17.316298 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_877_450} MergeTask::MergeProjectionsStage: Merge sorted 58005 rows, containing 13 columns (13 merged, 0 gathered) in 4.650872289 sec., 12471.85396537127 rows/sec., 1.98 MiB/sec. 2024.10.22 01:29:17.348273 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 57755 2024.10.22 01:29:17.372868 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:29:17.382106 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:29:17.400794 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:29:17.419381 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:29:17.424338 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_878_878_0 to 202410_883_883_0 2024.10.22 01:29:17.438117 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:17.449795 [ 1202 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:17.512182 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_878_878_0 to 202410_883_883_0 into Compact with storage Full 2024.10.22 01:29:17.519143 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_877_450} MergedBlockOutputStream: filled checksums 202410_1_877_450 (state Temporary) 2024.10.22 01:29:17.547829 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:29:17.562187 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:29:17.581455 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:29:17.586247 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:17.595195 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:29:17.599187 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 116 entries to flush up to offset 59776 2024.10.22 01:29:17.649396 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_877_450} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_877_450 to 202410_1_877_450 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:17.654731 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_877_450} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_876_449, 202410_877_877_0] -> 202410_1_877_450 2024.10.22 01:29:17.704419 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_878_878_0, total 119 rows starting from the beginning of the part 2024.10.22 01:29:17.741550 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:29:17.783347 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_879_879_0, total 96 rows starting from the beginning of the part 2024.10.22 01:29:17.796844 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:29:17.806453 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/oic/hsmqmahlqedwywttleiuvajaqwous (LogSeriesLimiter: on interval from 2024-10-22 01:29:02 to 2024-10-22 01:29:17 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:29:17.835850 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(84d59bbbd825a6f7c1c73249055e5252) : [0, 620436]: Setting download as failed: File segment: [0, 620436], key: 84d59bbbd825a6f7c1c73249055e5252, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:17.837622 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(84d59bbbd825a6f7c1c73249055e5252): No space left in cache to reserve 620437 bytes, will continue without cache download 2024.10.22 01:29:17.854831 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_880_880_0, total 118 rows starting from the beginning of the part 2024.10.22 01:29:17.897098 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_881_881_0, total 89 rows starting from the beginning of the part 2024.10.22 01:29:17.935644 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_882_882_0, total 76 rows starting from the beginning of the part 2024.10.22 01:29:17.985735 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 84d59bbbd825a6f7c1c73249055e5252, offset: 0, size: 0 2024.10.22 01:29:17.989637 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 84d59bbbd825a6f7c1c73249055e5252 for removal 2024.10.22 01:29:17.994755 [ 1476 ] {} CacheMetadata: Key 84d59bbbd825a6f7c1c73249055e5252 is removed from metadata 2024.10.22 01:29:18.012207 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:18.022560 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:18.051204 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_883_883_0, total 83 rows starting from the beginning of the part 2024.10.22 01:29:18.072732 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_505_505_0 (state Temporary) 2024.10.22 01:29:18.164764 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_505_505_0 to 202410_511_511_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:18.196830 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 779288 2024.10.22 01:29:18.204233 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_460_233 to 202410_461_506_9 2024.10.22 01:29:18.216888 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 3.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:18.221369 [ 1193 ] {} DiskLocal: Reserved 3.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:18.232125 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 59776 2024.10.22 01:29:18.351260 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_506_234} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_460_233 to 202410_461_506_9 into Wide with storage Full 2024.10.22 01:29:18.388058 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_506_234} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:18.535522 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_506_234} MergeTreeSequentialSource: Reading 8 marks from part 202410_1_460_233, total 52272 rows starting from the beginning of the part 2024.10.22 01:29:18.576365 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_780_388, 202410_781_781_0] 2024.10.22 01:29:18.577546 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_780_388, 202410_781_781_0] 2024.10.22 01:29:18.594396 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_780_388, 202410_781_781_0] 2024.10.22 01:29:18.659102 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 17.69 MiB, will set to 1.64 GiB (RSS), difference: -2.87 MiB 2024.10.22 01:29:18.699818 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(89c2d0a03fcf5bc1086921a3f2c83ff4) : [0, 535028]: Setting download as failed: File segment: [0, 535028], key: 89c2d0a03fcf5bc1086921a3f2c83ff4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:18.702547 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(89c2d0a03fcf5bc1086921a3f2c83ff4): No space left in cache to reserve 535029 bytes, will continue without cache download 2024.10.22 01:29:18.831118 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:18.841311 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:18.880319 [ 1687 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jud/qisashfzuxqifyrvhfepwvrdkiyuc, size 84341 2024.10.22 01:29:18.939352 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_506_234} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_506_9, total 6957 rows starting from the beginning of the part 2024.10.22 01:29:18.968458 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 89c2d0a03fcf5bc1086921a3f2c83ff4, offset: 0, size: 0 2024.10.22 01:29:18.970650 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 89c2d0a03fcf5bc1086921a3f2c83ff4 for removal 2024.10.22 01:29:18.972813 [ 1476 ] {} CacheMetadata: Key 89c2d0a03fcf5bc1086921a3f2c83ff4 is removed from metadata 2024.10.22 01:29:19.066620 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_373_77, 202410_374_374_0, 202410_375_375_0, 202410_376_376_0, 202410_377_377_0, 202410_378_378_0] 2024.10.22 01:29:19.068210 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_373_77, 202410_374_374_0, 202410_375_375_0, 202410_376_376_0, 202410_377_377_0, 202410_378_378_0] 2024.10.22 01:29:19.085412 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:29:19.096563 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_373_77, 202410_374_374_0, 202410_375_375_0, 202410_376_376_0, 202410_377_377_0, 202410_378_378_0] 2024.10.22 01:29:19.106717 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:29:19.124497 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:29:19.153205 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:29:19.153610 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} MergeTask::MergeProjectionsStage: Merge sorted 581 rows, containing 13 columns (13 merged, 0 gathered) in 1.678997982 sec., 346.0397250197529 rows/sec., 55.15 KiB/sec. 2024.10.22 01:29:19.170687 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:29:19.171791 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} MergedBlockOutputStream: filled checksums 202410_878_883_1 (state Temporary) 2024.10.22 01:29:19.205053 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:29:19.223093 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_494_494_0 (state Temporary) 2024.10.22 01:29:19.228820 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:29:19.261424 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:29:19.328095 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_878_883_1 to 202410_878_883_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:19.331854 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_878_883_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_878_878_0, 202410_883_883_0] -> 202410_878_883_1 2024.10.22 01:29:19.482883 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_494_494_0 to 202410_501_501_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:19.485720 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:29:19.634599 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 779288 2024.10.22 01:29:20.023913 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ddf5a9dc95936e87d050ae382e948692) : [0, 1986805]: Setting download as failed: File segment: [0, 1986805], key: ddf5a9dc95936e87d050ae382e948692, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:20.027682 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ddf5a9dc95936e87d050ae382e948692): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:29:20.263747 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ddf5a9dc95936e87d050ae382e948692, offset: 0, size: 0 2024.10.22 01:29:20.267060 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ddf5a9dc95936e87d050ae382e948692 for removal 2024.10.22 01:29:20.268647 [ 1476 ] {} CacheMetadata: Key ddf5a9dc95936e87d050ae382e948692 is removed from metadata 2024.10.22 01:29:20.361269 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:29:20.363153 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:29:20.421353 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 435 2024.10.22 01:29:20.584280 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 11.74 MiB, will set to 1.64 GiB (RSS), difference: -72.06 MiB 2024.10.22 01:29:20.680978 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:20.684310 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:20.766763 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_417_417_0 (state Temporary) 2024.10.22 01:29:20.873304 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_506_234/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:29:20.951868 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_417_417_0 to 202410_427_427_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:21.172617 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 435 2024.10.22 01:29:21.249729 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f084a0814d1cbfa1f6c1b51752987ba1) : [0, 2516968]: Setting download as failed: File segment: [0, 2516968], key: f084a0814d1cbfa1f6c1b51752987ba1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:21.253200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f084a0814d1cbfa1f6c1b51752987ba1): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:29:21.364633 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 114 entries to flush up to offset 57869 2024.10.22 01:29:21.656471 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f084a0814d1cbfa1f6c1b51752987ba1, offset: 0, size: 0 2024.10.22 01:29:21.658269 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f084a0814d1cbfa1f6c1b51752987ba1 for removal 2024.10.22 01:29:21.660433 [ 1476 ] {} CacheMetadata: Key f084a0814d1cbfa1f6c1b51752987ba1 is removed from metadata 2024.10.22 01:29:21.691460 [ 1226 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:29:21.738790 [ 1226 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:29:21.760570 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:21.768141 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:21.953603 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_871_871_0 (state Temporary) 2024.10.22 01:29:22.057666 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_871_871_0 to 202410_884_884_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:22.202342 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 57869 2024.10.22 01:29:22.604729 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 13.25 MiB, will set to 1.66 GiB (RSS), difference: 34.35 MiB 2024.10.22 01:29:22.707170 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_448_226, 202410_449_449_0] 2024.10.22 01:29:22.725390 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_448_226, 202410_449_449_0] 2024.10.22 01:29:22.758790 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a0e1c04fa4b646cb948f728f2d621cc6) : [0, 930284]: Setting download as failed: File segment: [0, 930284], key: a0e1c04fa4b646cb948f728f2d621cc6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:29:22.760370 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a0e1c04fa4b646cb948f728f2d621cc6): No space left in cache to reserve 930285 bytes, will continue without cache download 2024.10.22 01:29:22.798690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a0e1c04fa4b646cb948f728f2d621cc6, offset: 0, size: 0 2024.10.22 01:29:22.800197 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a0e1c04fa4b646cb948f728f2d621cc6 for removal 2024.10.22 01:29:22.810748 [ 1476 ] {} CacheMetadata: Key a0e1c04fa4b646cb948f728f2d621cc6 is removed from metadata 2024.10.22 01:29:22.848637 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_448_226, 202410_449_449_0] 2024.10.22 01:29:23.186531 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_506_234} MergeTask::MergeProjectionsStage: Merge sorted 59229 rows, containing 13 columns (13 merged, 0 gathered) in 4.857238417 sec., 12193.965977190368 rows/sec., 5.30 MiB/sec. 2024.10.22 01:29:23.225156 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d3f9f35ada37427e56d6b835a1211363) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d3f9f35ada37427e56d6b835a1211363, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:29:23.230079 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: d3f9f35ada37427e56d6b835a1211363, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:29:23.239815 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:29:23.348710 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_506_234} MergedBlockOutputStream: filled checksums 202410_1_506_234 (state Temporary) 2024.10.22 01:29:23.525768 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_781_389, 202410_782_782_0] 2024.10.22 01:29:23.528989 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_781_389, 202410_782_782_0] 2024.10.22 01:29:23.647170 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_781_389, 202410_782_782_0] 2024.10.22 01:29:23.775124 [ 1236 ] {} DatabaseCatalog: Execution took 311 ms. 2024.10.22 01:29:23.979714 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 137258 bytes for dfb95f6f6dc88e8d12b998eb6b56506a:0) 2024.10.22 01:29:23.984253 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 01e64c8c7392acaab5f520f0944aa3d8, offset: 0, size: 252134 2024.10.22 01:29:23.990930 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 01e64c8c7392acaab5f520f0944aa3d8 for removal 2024.10.22 01:29:23.993984 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 321ade17dcdaaecfced5bd7ab0bc169a, offset: 0, size: 226 2024.10.22 01:29:23.993818 [ 1476 ] {} CacheMetadata: Key 01e64c8c7392acaab5f520f0944aa3d8 is removed from metadata 2024.10.22 01:29:23.999202 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 321ade17dcdaaecfced5bd7ab0bc169a for removal 2024.10.22 01:29:24.031344 [ 1476 ] {} CacheMetadata: Key 321ade17dcdaaecfced5bd7ab0bc169a is removed from metadata 2024.10.22 01:29:24.255017 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/izg/alfgfifaeehuqeflfycbsxppckzmi. 2024.10.22 01:29:24.279568 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_506_234} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_506_234 to 202410_1_506_234 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:24.283530 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_506_234} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_460_233, 202410_461_506_9] -> 202410_1_506_234 2024.10.22 01:29:24.391036 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lzw/plfdapeixwwvsouasclinviihgegv. 2024.10.22 01:29:24.417765 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 69.08 MiB. 2024.10.22 01:29:24.711275 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 7977 2024.10.22 01:29:24.929479 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:24.933609 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:24.973195 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_516_516_0 (state Temporary) 2024.10.22 01:29:25.150854 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_516_516_0 to 202410_529_529_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:25.207279 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7977 2024.10.22 01:29:25.384996 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lzw/plfdapeixwwvsouasclinviihgegv. 2024.10.22 01:29:25.391957 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_2_2_0 to 201403_2_2_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:25.412779 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/izg/alfgfifaeehuqeflfycbsxppckzmi. 2024.10.22 01:29:25.454366 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.97 GiB, free memory in arenas 11.89 MiB, will set to 1.64 GiB (RSS), difference: 32.48 MiB 2024.10.22 01:29:25.573476 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 27 2024.10.22 01:29:25.576179 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:29:25.872497 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 175 entries to flush up to offset 59951 2024.10.22 01:29:25.997684 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:26.005356 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:26.066597 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:26.068914 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:26.105804 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_506_506_0 (state Temporary) 2024.10.22 01:29:26.171966 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_506_506_0 to 202410_512_512_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:26.177254 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_27_27_0 (state Temporary) 2024.10.22 01:29:26.178934 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_496_100/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:29:26.220393 [ 1188 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_507_0 to 202410_512_512_0 2024.10.22 01:29:26.235286 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 57931 2024.10.22 01:29:26.256626 [ 1188 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:26.256918 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 59951 2024.10.22 01:29:26.259753 [ 1188 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:26.310578 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_507_0 to 202410_512_512_0 into Compact with storage Full 2024.10.22 01:29:26.467514 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_27_27_0 to 202410_27_27_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:26.564080 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:26.569525 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_507_0, total 161 rows starting from the beginning of the part 2024.10.22 01:29:26.586719 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_508_508_0, total 85 rows starting from the beginning of the part 2024.10.22 01:29:26.597494 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:26.605791 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:26.641511 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 780828 2024.10.22 01:29:26.650923 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 27 2024.10.22 01:29:26.708505 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_509_509_0, total 153 rows starting from the beginning of the part 2024.10.22 01:29:26.751678 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_510_510_0, total 136 rows starting from the beginning of the part 2024.10.22 01:29:26.767428 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_511_511_0, total 116 rows starting from the beginning of the part 2024.10.22 01:29:26.768893 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_872_872_0 (state Temporary) 2024.10.22 01:29:26.799894 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_512_512_0, total 175 rows starting from the beginning of the part 2024.10.22 01:29:26.824814 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_872_872_0 to 202410_885_885_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:26.892204 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 57931 2024.10.22 01:29:27.503486 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:27.506715 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 912.00 MiB, peak 1.97 GiB, free memory in arenas 12.89 MiB, will set to 1.20 GiB (RSS), difference: 314.99 MiB 2024.10.22 01:29:27.507619 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:27.858355 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_495_495_0 (state Temporary) 2024.10.22 01:29:27.898220 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_782_390, 202410_783_783_0] 2024.10.22 01:29:27.902770 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_782_390, 202410_783_783_0] 2024.10.22 01:29:27.911414 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} MergeTask::MergeProjectionsStage: Merge sorted 826 rows, containing 13 columns (13 merged, 0 gathered) in 1.611747821 sec., 512.4871206511158 rows/sec., 230.06 KiB/sec. 2024.10.22 01:29:28.022206 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_782_390, 202410_783_783_0] 2024.10.22 01:29:28.063149 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} MergedBlockOutputStream: filled checksums 202410_507_512_1 (state Temporary) 2024.10.22 01:29:28.068761 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_104_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:29:28.267148 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_512_1 to 202410_507_512_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:28.269766 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_512_1} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_507_0, 202410_512_512_0] -> 202410_507_512_1 2024.10.22 01:29:28.286919 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_495_495_0 to 202410_502_502_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:28.309524 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:29:28.348073 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 780828 2024.10.22 01:29:28.337814 [ 1275 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_497_497_0 to 202410_502_502_0 2024.10.22 01:29:28.369494 [ 1275 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:28.373037 [ 1275 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:28.509633 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_497_497_0 to 202410_502_502_0 into Compact with storage Full 2024.10.22 01:29:28.557471 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:28.566653 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_497_497_0, total 1917 rows starting from the beginning of the part 2024.10.22 01:29:28.573649 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_498_498_0, total 1163 rows starting from the beginning of the part 2024.10.22 01:29:28.809232 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_499_499_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:29:28.915898 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_500_500_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:29:28.932725 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_501_501_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:29:28.948049 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_502_502_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:29:29.764675 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.26 GiB, peak 1.97 GiB, free memory in arenas 9.17 MiB, will set to 1.22 GiB (RSS), difference: -37.32 MiB 2024.10.22 01:29:30.363150 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:29:30.363718 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:29:30.377033 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} MergeTask::MergeProjectionsStage: Merge sorted 10010 rows, containing 4 columns (4 merged, 0 gathered) in 1.984938713 sec., 5042.976860918325 rows/sec., 78.80 KiB/sec. 2024.10.22 01:29:30.409671 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} MergedBlockOutputStream: filled checksums 202410_497_502_1 (state Temporary) 2024.10.22 01:29:30.668635 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_497_502_1 to 202410_497_502_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:30.674063 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_497_502_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_497_497_0, 202410_502_502_0] -> 202410_497_502_1 2024.10.22 01:29:30.857203 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:29:30.900393 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 57986 2024.10.22 01:29:31.065234 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:31.143563 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:31.148515 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:31.168751 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:31.176969 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_873_873_0 (state Temporary) 2024.10.22 01:29:31.340264 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_873_873_0 to 202410_886_886_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:31.363259 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.21 GiB, peak 1.97 GiB, free memory in arenas 10.76 MiB, will set to 1.26 GiB (RSS), difference: 47.09 MiB 2024.10.22 01:29:31.404776 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 57986 2024.10.22 01:29:32.712510 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 7998 2024.10.22 01:29:33.021695 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_449_227, 202410_450_450_0] 2024.10.22 01:29:33.026384 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_449_227, 202410_450_450_0] 2024.10.22 01:29:33.026320 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:33.034442 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:33.070130 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_449_227, 202410_450_450_0] 2024.10.22 01:29:33.191984 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_517_517_0 (state Temporary) 2024.10.22 01:29:33.194993 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_783_391, 202410_784_784_0] 2024.10.22 01:29:33.197684 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_783_391, 202410_784_784_0] 2024.10.22 01:29:33.212349 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_783_391, 202410_784_784_0] 2024.10.22 01:29:33.366819 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_517_517_0 to 202410_530_530_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:33.421702 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 7998 2024.10.22 01:29:33.423687 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 1.97 GiB, free memory in arenas 15.15 MiB, will set to 1.29 GiB (RSS), difference: -34.33 MiB 2024.10.22 01:29:33.767732 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_496_100} MergeTask::MergeProjectionsStage: Merge sorted 784408 rows, containing 4 columns (4 merged, 0 gathered) in 22.657551026 sec., 34620.15815830563 rows/sec., 519.34 KiB/sec. 2024.10.22 01:29:33.781446 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 88 entries to flush up to offset 60039 2024.10.22 01:29:33.798232 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} MergeTask::MergeProjectionsStage: Merge sorted 4112 rows, containing 679 columns (679 merged, 0 gathered) in 112.640448892 sec., 36.505536336619166 rows/sec., 49.35 KiB/sec. 2024.10.22 01:29:33.864317 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_496_100} MergedBlockOutputStream: filled checksums 202410_1_496_100 (state Temporary) 2024.10.22 01:29:34.158398 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:34.160072 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:34.322383 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_507_507_0 (state Temporary) 2024.10.22 01:29:34.374805 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_507_507_0 to 202410_513_513_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:34.456586 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60039 2024.10.22 01:29:34.553838 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 1.97 GiB, free memory in arenas 13.29 MiB, will set to 1.31 GiB (RSS), difference: -24.84 MiB 2024.10.22 01:29:34.651185 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_104_104_0 (state Temporary) 2024.10.22 01:29:34.970690 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} MergedBlockOutputStream: filled checksums 202410_1_104_21 (state Temporary) 2024.10.22 01:29:35.355470 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 782753 2024.10.22 01:29:35.414584 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 58021 2024.10.22 01:29:35.567472 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:35.570219 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:35.717474 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_874_874_0 (state Temporary) 2024.10.22 01:29:35.772437 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_874_874_0 to 202410_887_887_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:35.841518 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58021 2024.10.22 01:29:35.983582 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_496_100} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_496_100 to 202410_1_496_100 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:35.987662 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_496_100} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_490_99, 202410_491_496_1] -> 202410_1_496_100 2024.10.22 01:29:36.022500 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:29:36.090800 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:36.093718 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:36.175139 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 436 2024.10.22 01:29:36.326342 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_496_496_0 (state Temporary) 2024.10.22 01:29:36.382083 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 1.97 GiB, free memory in arenas 8.61 MiB, will set to 1.37 GiB (RSS), difference: -28.61 MiB 2024.10.22 01:29:36.453486 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:36.456623 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:36.505519 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_418_418_0 (state Temporary) 2024.10.22 01:29:36.568458 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_496_496_0 to 202410_503_503_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:36.712306 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 782753 2024.10.22 01:29:36.750438 [ 1189 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:29:36.774804 [ 1189 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:29:36.840366 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_418_418_0 to 202410_428_428_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:36.875627 [ 1186 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_423_87 to 202410_428_428_0 2024.10.22 01:29:36.885532 [ 1186 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:36.889750 [ 1186 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:36.931723 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 436 2024.10.22 01:29:37.051267 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_423_87 to 202410_428_428_0 into Compact with storage Full 2024.10.22 01:29:37.273653 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 214 ms. 2024.10.22 01:29:37.452903 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:37.461157 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_423_87, total 467 rows starting from the beginning of the part 2024.10.22 01:29:37.560714 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_424_424_0, total 1 rows starting from the beginning of the part 2024.10.22 01:29:37.637621 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_425_425_0, total 1 rows starting from the beginning of the part 2024.10.22 01:29:37.823102 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_466_93, 202410_467_467_0, 202410_468_468_0, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0] 2024.10.22 01:29:37.825520 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_466_93, 202410_467_467_0, 202410_468_468_0, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0] 2024.10.22 01:29:37.862506 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_426_426_0, total 1 rows starting from the beginning of the part 2024.10.22 01:29:37.867452 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_466_93, 202410_467_467_0, 202410_468_468_0, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0] 2024.10.22 01:29:37.950342 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_427_427_0, total 1 rows starting from the beginning of the part 2024.10.22 01:29:37.953287 [ 1683 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/oic/hsmqmahlqedwywttleiuvajaqwous, size 5713578 2024.10.22 01:29:37.988485 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_428_428_0, total 1 rows starting from the beginning of the part 2024.10.22 01:29:38.048591 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:29:38.176753 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:29:38.222804 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_784_392, 202410_785_785_0] 2024.10.22 01:29:38.225805 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_784_392, 202410_785_785_0] 2024.10.22 01:29:38.227148 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:29:38.237763 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_784_392, 202410_785_785_0] 2024.10.22 01:29:38.238555 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:29:38.255065 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:29:38.264506 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:29:38.267106 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:29:38.275149 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:29:38.292852 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:29:38.392643 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:29:38.431824 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:29:38.435039 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:29:38.467697 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:29:38.466982 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:29:38.502121 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:29:38.529744 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:29:38.544621 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:29:38.565182 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.97 GiB, free memory in arenas 8.26 MiB, will set to 1.39 GiB (RSS), difference: -14.97 MiB 2024.10.22 01:29:38.630328 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/twt/kggbxpjqkqdvlmyppvfpceyosfzny (LogSeriesLimiter: on interval from 2024-10-22 01:29:17 to 2024-10-22 01:29:38 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:29:39.670924 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_450_228, 202410_451_451_0] 2024.10.22 01:29:39.672579 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_450_228, 202410_451_451_0] 2024.10.22 01:29:39.690992 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_450_228, 202410_451_451_0] 2024.10.22 01:29:39.847012 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_428_88/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:29:39.854613 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 58085 2024.10.22 01:29:39.895340 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} MergeTask::MergeProjectionsStage: Merge sorted 472 rows, containing 32 columns (32 merged, 0 gathered) in 2.986914519 sec., 158.0226005791497 rows/sec., 14.67 KiB/sec. 2024.10.22 01:29:39.947448 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} MergedBlockOutputStream: filled checksums 202410_1_428_88 (state Temporary) 2024.10.22 01:29:40.342840 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_439_88, 202410_440_440_0, 202410_441_441_0, 202410_442_442_0, 202410_443_443_0, 202410_444_444_0] 2024.10.22 01:29:40.351655 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_439_88, 202410_440_440_0, 202410_441_441_0, 202410_442_442_0, 202410_443_443_0, 202410_444_444_0] 2024.10.22 01:29:40.366581 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:29:40.371503 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:29:40.378527 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_439_88, 202410_440_440_0, 202410_441_441_0, 202410_442_442_0, 202410_443_443_0, 202410_444_444_0] 2024.10.22 01:29:40.379398 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:40.385249 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:40.613533 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_428_88 to 202410_1_428_88 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:40.623288 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_428_88} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_423_87, 202410_428_428_0] -> 202410_1_428_88 2024.10.22 01:29:40.635860 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.97 GiB, free memory in arenas 4.49 MiB, will set to 1.44 GiB (RSS), difference: -22.70 MiB 2024.10.22 01:29:40.697023 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_875_875_0 (state Temporary) 2024.10.22 01:29:40.725436 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:29:40.748755 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_875_875_0 to 202410_888_888_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:40.778608 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:29:40.815582 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58085 2024.10.22 01:29:40.887095 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_877_450 to 202410_878_883_1 2024.10.22 01:29:40.927012 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 8014 2024.10.22 01:29:41.001602 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.10 MiB using storage policy from min volume index 0 2024.10.22 01:29:41.025930 [ 1214 ] {} DiskLocal: Reserved 2.10 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:41.033412 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 258 ms. 2024.10.22 01:29:41.072734 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_883_451} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_877_450 to 202410_878_883_1 into Compact with storage Full 2024.10.22 01:29:41.095139 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_883_451} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:41.115533 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_883_451} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_877_450, total 58005 rows starting from the beginning of the part 2024.10.22 01:29:41.155416 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_883_451} MergeTreeSequentialSource: Reading 2 marks from part 202410_878_883_1, total 581 rows starting from the beginning of the part 2024.10.22 01:29:41.181400 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:41.196900 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:41.258386 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_518_518_0 (state Temporary) 2024.10.22 01:29:41.449949 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_518_518_0 to 202410_531_531_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:41.478058 [ 1226 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_526_105 to 202410_531_531_0 2024.10.22 01:29:41.489709 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8014 2024.10.22 01:29:41.491254 [ 1226 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:41.521663 [ 1226 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:41.552365 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_526_105 to 202410_531_531_0 into Compact with storage Full 2024.10.22 01:29:41.593825 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:41.616584 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_526_105, total 8036 rows starting from the beginning of the part 2024.10.22 01:29:41.766012 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_527_527_0, total 20 rows starting from the beginning of the part 2024.10.22 01:29:41.781964 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_528_528_0, total 8 rows starting from the beginning of the part 2024.10.22 01:29:41.860078 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_529_529_0, total 19 rows starting from the beginning of the part 2024.10.22 01:29:41.894468 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_530_530_0, total 21 rows starting from the beginning of the part 2024.10.22 01:29:41.971008 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 86 entries to flush up to offset 60125 2024.10.22 01:29:42.072074 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_531_531_0, total 16 rows starting from the beginning of the part 2024.10.22 01:29:42.331987 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:42.339221 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:42.387138 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_508_508_0 (state Temporary) 2024.10.22 01:29:42.472324 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_104_104_0 to 202410_107_107_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:42.566484 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_104_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:29:42.611473 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 4.49 MiB, will set to 1.49 GiB (RSS), difference: -39.13 MiB 2024.10.22 01:29:42.766227 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_508_508_0 to 202410_514_514_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:42.803849 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60125 2024.10.22 01:29:43.192200 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_785_393, 202410_786_786_0] 2024.10.22 01:29:43.199252 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_785_393, 202410_786_786_0] 2024.10.22 01:29:43.218869 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_785_393, 202410_786_786_0] 2024.10.22 01:29:43.254331 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} MergeTask::MergeProjectionsStage: Merge sorted 8120 rows, containing 12 columns (12 merged, 0 gathered) in 1.711182862 sec., 4745.2555657958665 rows/sec., 754.16 KiB/sec. 2024.10.22 01:29:43.420678 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} MergedBlockOutputStream: filled checksums 202410_1_531_106 (state Temporary) 2024.10.22 01:29:43.718886 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 784293 2024.10.22 01:29:43.745555 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_531_106 to 202410_1_531_106 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:43.752625 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_531_106} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_526_105, 202410_531_531_0] -> 202410_1_531_106 2024.10.22 01:29:43.774295 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.62 MiB. 2024.10.22 01:29:44.434399 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 437 2024.10.22 01:29:44.513346 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:44.521793 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:44.602069 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 3.04 MiB, will set to 1.52 GiB (RSS), difference: -20.32 MiB 2024.10.22 01:29:44.620944 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_883_451} MergeTask::MergeProjectionsStage: Merge sorted 58586 rows, containing 13 columns (13 merged, 0 gathered) in 3.555390991 sec., 16478.07516762648 rows/sec., 2.62 MiB/sec. 2024.10.22 01:29:44.659054 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_497_497_0 (state Temporary) 2024.10.22 01:29:44.725401 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_497_497_0 to 202410_504_504_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:44.735336 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:44.743566 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:44.773361 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 784293 2024.10.22 01:29:44.804354 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_883_451} MergedBlockOutputStream: filled checksums 202410_1_883_451 (state Temporary) 2024.10.22 01:29:44.822505 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_419_419_0 (state Temporary) 2024.10.22 01:29:44.830282 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 58158 2024.10.22 01:29:44.982623 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_883_451} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_883_451 to 202410_1_883_451 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:44.990311 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_883_451} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_877_450, 202410_878_883_1] -> 202410_1_883_451 2024.10.22 01:29:45.025963 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.92 MiB. 2024.10.22 01:29:45.112836 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:45.114947 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:45.144315 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_419_419_0 to 202410_429_429_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:45.226970 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 437 2024.10.22 01:29:45.257193 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_876_876_0 (state Temporary) 2024.10.22 01:29:45.283494 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_876_876_0 to 202410_889_889_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:45.298453 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4172 2024.10.22 01:29:45.321223 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_883_451 to 202410_884_884_0 2024.10.22 01:29:45.324913 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.10 MiB using storage policy from min volume index 0 2024.10.22 01:29:45.327778 [ 1152 ] {} DiskLocal: Reserved 2.10 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:45.329197 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58158 2024.10.22 01:29:45.362509 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_884_452} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_883_451 to 202410_884_884_0 into Compact with storage Full 2024.10.22 01:29:45.388980 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_884_452} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:45.402972 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_884_452} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_883_451, total 58586 rows starting from the beginning of the part 2024.10.22 01:29:45.520750 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(23409d7e68f7cf704ed3c02293b23e58) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 23409d7e68f7cf704ed3c02293b23e58, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:29:45.523706 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 23409d7e68f7cf704ed3c02293b23e58, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:29:45.527597 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:29:45.536896 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_884_452} MergeTreeSequentialSource: Reading 2 marks from part 202410_884_884_0, total 114 rows starting from the beginning of the part 2024.10.22 01:29:46.075242 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 345.34 MiB using storage policy from min volume index 0 2024.10.22 01:29:46.363510 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.97 GiB, free memory in arenas 6.41 MiB, will set to 1.52 GiB (RSS), difference: 4.68 MiB 2024.10.22 01:29:46.953148 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kgw/ijmrqbyxdoxpjjyiiynjhhpqpafca (LogSeriesLimiter: on interval from 2024-10-22 01:29:38 to 2024-10-22 01:29:46 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:29:48.354758 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_104_21 to 202410_1_104_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:48.361004 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_104_21} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_1_99_20, 202410_104_104_0] -> 202410_1_104_21 2024.10.22 01:29:48.448653 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_884_452/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:29:48.568366 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_786_394, 202410_787_787_0] 2024.10.22 01:29:48.571419 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_786_394, 202410_787_787_0] 2024.10.22 01:29:48.580122 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_786_394, 202410_787_787_0] 2024.10.22 01:29:48.890773 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 5.65 MiB, will set to 1.53 GiB (RSS), difference: -104.41 MiB 2024.10.22 01:29:48.968540 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_884_452} MergeTask::MergeProjectionsStage: Merge sorted 58700 rows, containing 13 columns (13 merged, 0 gathered) in 3.609668836 sec., 16261.879598087318 rows/sec., 2.59 MiB/sec. 2024.10.22 01:29:48.992853 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 8037 2024.10.22 01:29:49.025382 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_451_229, 202410_452_452_0] 2024.10.22 01:29:49.031860 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_451_229, 202410_452_452_0] 2024.10.22 01:29:49.061812 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_884_452} MergedBlockOutputStream: filled checksums 202410_1_884_452 (state Temporary) 2024.10.22 01:29:49.103760 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_451_229, 202410_452_452_0] 2024.10.22 01:29:49.163791 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_884_452} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_884_452 to 202410_1_884_452 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:49.166773 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_884_452} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_883_451, 202410_884_884_0] -> 202410_1_884_452 2024.10.22 01:29:49.196515 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.82 MiB. 2024.10.22 01:29:49.227359 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:49.234715 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:29:49.301419 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_519_519_0 (state Temporary) 2024.10.22 01:29:49.354999 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_519_519_0 to 202410_532_532_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:49.355804 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 58203 2024.10.22 01:29:49.406761 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8037 2024.10.22 01:29:49.507951 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 42.69 MiB. 2024.10.22 01:29:49.896569 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:49.901778 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:29:49.964538 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_877_877_0 (state Temporary) 2024.10.22 01:29:50.113017 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_877_877_0 to 202410_890_890_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:50.193390 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(26563b59a80c026e95df5d49bf4cf6aa) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 26563b59a80c026e95df5d49bf4cf6aa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:29:50.244430 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_884_452 to 202410_885_885_0 2024.10.22 01:29:50.297820 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 26563b59a80c026e95df5d49bf4cf6aa, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:29:50.300465 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.10 MiB using storage policy from min volume index 0 2024.10.22 01:29:50.302776 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:29:50.330704 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 102 entries to flush up to offset 60227 2024.10.22 01:29:50.363565 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 4.19 MiB, will set to 1.52 GiB (RSS), difference: -119.24 MiB 2024.10.22 01:29:50.365219 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:29:50.367044 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:29:50.411247 [ 1240 ] {} DiskLocal: Reserved 2.10 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:29:50.461447 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58203 2024.10.22 01:29:50.478503 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:50.564259 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:50.637447 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_885_453} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_884_452 to 202410_885_885_0 into Compact with storage Full 2024.10.22 01:29:50.647522 [ 1240 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 419 ms. 2024.10.22 01:29:50.685415 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_509_509_0 (state Temporary) 2024.10.22 01:29:50.699904 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_885_453} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:50.719170 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_885_453} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_884_452, total 58700 rows starting from the beginning of the part 2024.10.22 01:29:50.740035 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_509_509_0 to 202410_515_515_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:50.750038 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_885_453} MergeTreeSequentialSource: Reading 2 marks from part 202410_885_885_0, total 62 rows starting from the beginning of the part 2024.10.22 01:29:50.880607 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60227 2024.10.22 01:29:51.007253 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rgq/serwynnthjdrcjnwtgkzdviepxkpe (LogSeriesLimiter: on interval from 2024-10-22 01:29:46 to 2024-10-22 01:29:51 accepted series 1 / 176 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:29:51.777119 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 785833 2024.10.22 01:29:51.778686 [ 1260 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:29:51.961659 [ 1260 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:29:52.499332 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 4.05 MiB, will set to 1.53 GiB (RSS), difference: -130.11 MiB 2024.10.22 01:29:52.666416 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:52.671696 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:52.728996 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 438 2024.10.22 01:29:52.846418 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 4211 2024.10.22 01:29:53.045799 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:53.050457 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:53.105139 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_420_420_0 (state Temporary) 2024.10.22 01:29:53.127480 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_498_498_0 (state Temporary) 2024.10.22 01:29:53.184450 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_498_498_0 to 202410_505_505_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:53.275538 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_420_420_0 to 202410_430_430_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:53.339367 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 785833 2024.10.22 01:29:53.483466 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 438 2024.10.22 01:29:54.161780 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_787_395, 202410_788_788_0] 2024.10.22 01:29:54.164834 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_787_395, 202410_788_788_0] 2024.10.22 01:29:54.178141 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_787_395, 202410_788_788_0] 2024.10.22 01:29:54.292100 [ 1156 ] {} DatabaseCatalog: Execution took 225 ms. 2024.10.22 01:29:54.355140 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 4.81 MiB, will set to 1.53 GiB (RSS), difference: -123.74 MiB 2024.10.22 01:29:54.473319 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 58254 2024.10.22 01:29:54.748440 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:54.751974 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:29:54.799797 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_878_878_0 (state Temporary) 2024.10.22 01:29:54.966702 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_878_878_0 to 202410_891_891_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:54.987420 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_886_886_0 to 202410_891_891_0 2024.10.22 01:29:55.007190 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:55.024978 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58254 2024.10.22 01:29:55.027122 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:29:55.151228 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_886_886_0 to 202410_891_891_0 into Compact with storage Full 2024.10.22 01:29:55.518484 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_885_453} MergeTask::MergeProjectionsStage: Merge sorted 58762 rows, containing 13 columns (13 merged, 0 gathered) in 4.969526669 sec., 11824.466174325706 rows/sec., 1.88 MiB/sec. 2024.10.22 01:29:55.535782 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:29:55.554449 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_886_886_0, total 55 rows starting from the beginning of the part 2024.10.22 01:29:55.614386 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_887_887_0, total 35 rows starting from the beginning of the part 2024.10.22 01:29:55.758450 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_888_888_0, total 64 rows starting from the beginning of the part 2024.10.22 01:29:55.805032 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_885_453} MergedBlockOutputStream: filled checksums 202410_1_885_453 (state Temporary) 2024.10.22 01:29:56.249614 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_889_889_0, total 73 rows starting from the beginning of the part 2024.10.22 01:29:56.421333 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_890_890_0, total 45 rows starting from the beginning of the part 2024.10.22 01:29:56.426679 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_885_453} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_885_453 to 202410_1_885_453 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:56.432677 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_885_453} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_884_452, 202410_885_885_0] -> 202410_1_885_453 2024.10.22 01:29:56.475333 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.97 GiB, free memory in arenas 3.22 MiB, will set to 1.53 GiB (RSS), difference: -54.17 MiB 2024.10.22 01:29:56.565637 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:29:56.575043 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_891_891_0, total 51 rows starting from the beginning of the part 2024.10.22 01:29:56.910555 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 8049 2024.10.22 01:29:56.947862 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_452_230, 202410_453_453_0] 2024.10.22 01:29:56.950367 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_452_230, 202410_453_453_0] 2024.10.22 01:29:56.989505 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_452_230, 202410_453_453_0] 2024.10.22 01:29:57.226592 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:57.233517 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:29:57.273073 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_520_520_0 (state Temporary) 2024.10.22 01:29:57.420013 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_520_520_0 to 202410_533_533_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:57.562274 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8049 2024.10.22 01:29:57.605773 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(7224c683c7e4d907ede53e5c5bc41069) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7224c683c7e4d907ede53e5c5bc41069, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:29:57.613206 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 7224c683c7e4d907ede53e5c5bc41069, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:29:57.629341 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:29:57.953632 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:29:57.974639 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:29:57.998841 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} MergeTask::MergeProjectionsStage: Merge sorted 323 rows, containing 13 columns (13 merged, 0 gathered) in 2.96254313 sec., 109.02794856525853 rows/sec., 17.26 KiB/sec. 2024.10.22 01:29:58.083986 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:29:58.106769 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:29:58.163414 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} MergedBlockOutputStream: filled checksums 202410_886_891_1 (state Temporary) 2024.10.22 01:29:58.229753 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:29:58.260665 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_886_891_1 to 202410_886_891_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:58.264164 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_886_891_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_886_886_0, 202410_891_891_0] -> 202410_886_891_1 2024.10.22 01:29:58.267452 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:29:58.303856 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:29:58.318673 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:29:58.335112 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:29:58.374168 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:29:58.400386 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 80 entries to flush up to offset 60307 2024.10.22 01:29:58.475005 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/efn/yjdyrfqvgkbcxetnnmievckqyuwwg (LogSeriesLimiter: on interval from 2024-10-22 01:29:51 to 2024-10-22 01:29:58 accepted series 1 / 179 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:29:58.660532 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:58.662603 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:58.808902 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_510_510_0 (state Temporary) 2024.10.22 01:29:59.034096 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 58308 2024.10.22 01:29:59.077684 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_510_510_0 to 202410_516_516_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:59.292782 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:29:59.301043 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:29:59.370497 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_879_879_0 (state Temporary) 2024.10.22 01:29:59.410221 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60307 2024.10.22 01:29:59.425520 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_879_879_0 to 202410_892_892_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:29:59.586542 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_788_396, 202410_789_789_0] 2024.10.22 01:29:59.590057 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_788_396, 202410_789_789_0] 2024.10.22 01:29:59.609162 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_788_396, 202410_789_789_0] 2024.10.22 01:29:59.611970 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58308 2024.10.22 01:30:00.355600 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 787373 2024.10.22 01:30:00.367678 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:30:00.369381 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:30:00.639080 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.97 GiB, free memory in arenas 3.02 MiB, will set to 1.54 GiB (RSS), difference: -9.26 MiB 2024.10.22 01:30:00.986260 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 439 2024.10.22 01:30:01.291383 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:01.296294 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:30:01.404347 [ 1696 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/twt/kggbxpjqkqdvlmyppvfpceyosfzny, size 2283941 2024.10.22 01:30:01.411947 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_421_421_0 (state Temporary) 2024.10.22 01:30:01.573466 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_421_421_0 to 202410_431_431_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:01.575585 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:01.577584 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:30:01.759492 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 439 2024.10.22 01:30:01.824520 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_499_499_0 (state Temporary) 2024.10.22 01:30:01.852886 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:30:01.861944 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:30:01.886639 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:30:01.907146 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:30:01.921526 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_499_499_0 to 202410_506_506_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:01.933371 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:30:01.975972 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 787373 2024.10.22 01:30:01.963442 [ 1270 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_496_100 to 202410_497_502_1 2024.10.22 01:30:01.997679 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:30:02.007150 [ 1270 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:02.012901 [ 1270 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:30:02.073500 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_502_101} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_496_100 to 202410_497_502_1 into Wide with storage Full 2024.10.22 01:30:02.187119 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:30:02.206735 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:30:02.214877 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_502_101} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:02.323671 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_502_101} MergeTreeSequentialSource: Reading 97 marks from part 202410_1_496_100, total 784408 rows starting from the beginning of the part 2024.10.22 01:30:02.435147 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_502_101} MergeTreeSequentialSource: Reading 3 marks from part 202410_497_502_1, total 10010 rows starting from the beginning of the part 2024.10.22 01:30:02.481155 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.97 GiB, free memory in arenas 4.37 MiB, will set to 1.55 GiB (RSS), difference: 1.32 MiB 2024.10.22 01:30:02.647766 [ 1151 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 01:30:03.295619 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:30:03.625389 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 58353 2024.10.22 01:30:04.017413 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:04.026846 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:30:04.260499 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_880_880_0 (state Temporary) 2024.10.22 01:30:04.320568 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_880_880_0 to 202410_893_893_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:04.382400 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58353 2024.10.22 01:30:04.550367 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.97 GiB, free memory in arenas 9.18 MiB, will set to 1.56 GiB (RSS), difference: -27.87 MiB 2024.10.22 01:30:04.650547 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_789_397, 202410_790_790_0] 2024.10.22 01:30:04.655319 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_789_397, 202410_790_790_0] 2024.10.22 01:30:04.665757 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_789_397, 202410_790_790_0] 2024.10.22 01:30:05.071219 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 8062 2024.10.22 01:30:05.218638 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:05.226343 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:30:05.279348 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_521_521_0 (state Temporary) 2024.10.22 01:30:05.531660 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_453_231, 202410_454_454_0] 2024.10.22 01:30:05.538734 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_453_231, 202410_454_454_0] 2024.10.22 01:30:05.572706 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_521_521_0 to 202410_534_534_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:05.626652 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_453_231, 202410_454_454_0] 2024.10.22 01:30:05.665231 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8062 2024.10.22 01:30:06.447754 [ 1161 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 01:30:06.476430 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_378_78, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0, 202410_382_382_0, 202410_383_383_0] 2024.10.22 01:30:06.557536 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_378_78, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0, 202410_382_382_0, 202410_383_383_0] 2024.10.22 01:30:06.620454 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_378_78, 202410_379_379_0, 202410_380_380_0, 202410_381_381_0, 202410_382_382_0, 202410_383_383_0] 2024.10.22 01:30:06.787040 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_502_101/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:30:06.885473 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.97 GiB, free memory in arenas 12.57 MiB, will set to 1.58 GiB (RSS), difference: 11.36 MiB 2024.10.22 01:30:06.924842 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 74 entries to flush up to offset 60381 2024.10.22 01:30:06.969703 [ 1272 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:30:07.020315 [ 1272 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:30:07.275702 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:07.279254 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:30:07.339616 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_511_511_0 (state Temporary) 2024.10.22 01:30:07.386814 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_511_511_0 to 202410_517_517_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:07.445438 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_512_1 to 202410_517_517_0 2024.10.22 01:30:07.467605 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:07.469215 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60381 2024.10.22 01:30:07.481158 [ 1150 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:30:07.520325 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_512_1 to 202410_517_517_0 into Compact with storage Full 2024.10.22 01:30:07.687333 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:07.716174 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_512_1, total 826 rows starting from the beginning of the part 2024.10.22 01:30:07.804839 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_513_513_0, total 88 rows starting from the beginning of the part 2024.10.22 01:30:07.894890 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_514_514_0, total 86 rows starting from the beginning of the part 2024.10.22 01:30:08.223630 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_515_515_0, total 102 rows starting from the beginning of the part 2024.10.22 01:30:08.311560 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_516_516_0, total 80 rows starting from the beginning of the part 2024.10.22 01:30:08.391201 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_517_517_0, total 74 rows starting from the beginning of the part 2024.10.22 01:30:08.580904 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 58396 2024.10.22 01:30:08.651236 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.97 GiB, free memory in arenas 13.20 MiB, will set to 1.58 GiB (RSS), difference: 6.58 MiB 2024.10.22 01:30:08.984599 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 788913 2024.10.22 01:30:09.034181 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:09.044434 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:30:09.233560 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_881_881_0 (state Temporary) 2024.10.22 01:30:09.283743 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_881_881_0 to 202410_894_894_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:09.377459 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(98252f8c71e856785e9602b1eb18feee) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 98252f8c71e856785e9602b1eb18feee, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:30:09.381571 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 98252f8c71e856785e9602b1eb18feee, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:30:09.384569 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:30:09.446886 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58396 2024.10.22 01:30:09.454406 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_790_398, 202410_791_791_0] 2024.10.22 01:30:09.457521 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_790_398, 202410_791_791_0] 2024.10.22 01:30:09.472439 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_790_398, 202410_791_791_0] 2024.10.22 01:30:09.773468 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:09.776462 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:30:10.060045 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} MergeTask::MergeProjectionsStage: Merge sorted 1256 rows, containing 13 columns (13 merged, 0 gathered) in 2.55710908 sec., 491.17967231964934 rows/sec., 219.00 KiB/sec. 2024.10.22 01:30:10.060618 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_500_500_0 (state Temporary) 2024.10.22 01:30:10.199593 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} MergedBlockOutputStream: filled checksums 202410_507_517_2 (state Temporary) 2024.10.22 01:30:10.282528 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_500_500_0 to 202410_507_507_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:10.367577 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:30:10.370366 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:30:10.453705 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 788913 2024.10.22 01:30:10.536865 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_517_2 to 202410_507_517_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:10.547859 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_517_2} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_512_1, 202410_517_517_0] -> 202410_507_517_2 2024.10.22 01:30:10.595136 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:30:11.525811 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.97 GiB, free memory in arenas 12.03 MiB, will set to 1.59 GiB (RSS), difference: 4.11 MiB 2024.10.22 01:30:11.671079 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:11.674875 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:30:13.175224 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 8073 2024.10.22 01:30:13.384713 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:13.392361 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:30:13.450439 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_522_522_0 (state Temporary) 2024.10.22 01:30:13.455592 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 58429 2024.10.22 01:30:13.529435 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.97 GiB, free memory in arenas 10.18 MiB, will set to 1.59 GiB (RSS), difference: -4.84 MiB 2024.10.22 01:30:13.638869 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_522_522_0 to 202410_535_535_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:13.684261 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:13.690325 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:30:13.764866 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8073 2024.10.22 01:30:13.737805 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_882_882_0 (state Temporary) 2024.10.22 01:30:13.988334 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_882_882_0 to 202410_895_895_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:14.081233 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58429 2024.10.22 01:30:14.274903 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_791_399, 202410_792_792_0] 2024.10.22 01:30:14.277479 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_791_399, 202410_792_792_0] 2024.10.22 01:30:14.326487 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_455_455_0, 202410_456_456_0, 202410_457_457_0, 202410_458_458_0, 202410_459_459_0, 202410_460_460_0] 2024.10.22 01:30:14.328394 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_455_455_0, 202410_456_456_0, 202410_457_457_0, 202410_458_458_0, 202410_459_459_0, 202410_460_460_0] 2024.10.22 01:30:14.361715 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_455_455_0, 202410_456_456_0, 202410_457_457_0, 202410_458_458_0, 202410_459_459_0, 202410_460_460_0] 2024.10.22 01:30:14.387719 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_791_399, 202410_792_792_0] 2024.10.22 01:30:14.769651 [ 1164 ] {} DatabaseCatalog: Execution took 306 ms. 2024.10.22 01:30:14.973147 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_105_105_0 (state Temporary) 2024.10.22 01:30:14.982662 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 58 entries to flush up to offset 60439 2024.10.22 01:30:15.143503 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:15.151679 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:30:15.298397 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_512_512_0 (state Temporary) 2024.10.22 01:30:15.370387 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_512_512_0 to 202410_518_518_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:15.524419 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.97 GiB, free memory in arenas 11.43 MiB, will set to 1.59 GiB (RSS), difference: 4.74 MiB 2024.10.22 01:30:15.529056 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60439 2024.10.22 01:30:15.540674 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:30:16.761708 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 440 2024.10.22 01:30:16.947843 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:16.954173 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:30:17.001404 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_422_422_0 (state Temporary) 2024.10.22 01:30:17.175580 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_422_422_0 to 202410_432_432_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:17.380658 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 440 2024.10.22 01:30:17.457284 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 790453 2024.10.22 01:30:17.471618 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_502_101/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:30:17.591053 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.97 GiB, free memory in arenas 10.33 MiB, will set to 1.60 GiB (RSS), difference: 2.49 MiB 2024.10.22 01:30:17.930830 [ 1735 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/efn/yjdyrfqvgkbcxetnnmievckqyuwwg, size 2544326 2024.10.22 01:30:18.010624 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:30:18.021597 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:30:18.061602 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:30:18.086571 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:30:18.138262 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_471_94, 202410_472_472_0, 202410_473_473_0, 202410_474_474_0, 202410_475_475_0, 202410_476_476_0] 2024.10.22 01:30:18.293479 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 58468 2024.10.22 01:30:18.299764 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_471_94, 202410_472_472_0, 202410_473_473_0, 202410_474_474_0, 202410_475_475_0, 202410_476_476_0] 2024.10.22 01:30:18.354518 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_471_94, 202410_472_472_0, 202410_473_473_0, 202410_474_474_0, 202410_475_475_0, 202410_476_476_0] 2024.10.22 01:30:18.355841 [ 1721 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rgq/serwynnthjdrcjnwtgkzdviepxkpe, size 177 2024.10.22 01:30:18.356206 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:30:18.425597 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:30:18.461542 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:30:18.484591 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:30:18.553984 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:30:18.609607 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vgr/uydwxzksooxgoefebqhbpkrehfcjh (LogSeriesLimiter: on interval from 2024-10-22 01:29:58 to 2024-10-22 01:30:18 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:30:18.780822 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:18.786217 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:30:18.869755 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:18.875219 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:30:18.945187 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_883_883_0 (state Temporary) 2024.10.22 01:30:18.977798 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_501_501_0 (state Temporary) 2024.10.22 01:30:19.062626 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_883_883_0 to 202410_896_896_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:19.217152 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58468 2024.10.22 01:30:19.224407 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_501_501_0 to 202410_508_508_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:19.227177 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_792_400, 202410_793_793_0] 2024.10.22 01:30:19.230725 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_792_400, 202410_793_793_0] 2024.10.22 01:30:19.247414 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_885_453 to 202410_886_891_1 2024.10.22 01:30:19.248058 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_792_400, 202410_793_793_0] 2024.10.22 01:30:19.253731 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.12 MiB using storage policy from min volume index 0 2024.10.22 01:30:19.255778 [ 1231 ] {} DiskLocal: Reserved 2.12 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:30:19.250857 [ 1168 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_503_503_0 to 202410_508_508_0 2024.10.22 01:30:19.271530 [ 1168 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:19.273713 [ 1168 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 01:30:19.279862 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_891_454} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_885_453 to 202410_886_891_1 into Compact with storage Full 2024.10.22 01:30:19.328385 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 790453 2024.10.22 01:30:19.461516 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_503_503_0 to 202410_508_508_0 into Compact with storage Full 2024.10.22 01:30:19.482213 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_891_454} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:19.527275 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_891_454} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_885_453, total 58762 rows starting from the beginning of the part 2024.10.22 01:30:19.550111 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:19.559197 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_503_503_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:30:19.566694 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.97 GiB, free memory in arenas 14.09 MiB, will set to 1.60 GiB (RSS), difference: 90.61 MiB 2024.10.22 01:30:19.571267 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_891_454} MergeTreeSequentialSource: Reading 2 marks from part 202410_886_891_1, total 323 rows starting from the beginning of the part 2024.10.22 01:30:19.611048 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_504_504_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:30:19.653833 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_505_505_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:30:19.699629 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_506_506_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:30:19.766925 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_507_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:30:19.851259 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_508_508_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:30:20.371202 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:30:20.373000 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:30:21.186514 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 274 ms. 2024.10.22 01:30:21.191412 [ 1216 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 529 ms. 2024.10.22 01:30:21.220880 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_891_454/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:30:21.266841 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 8079 2024.10.22 01:30:21.310348 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_105_105_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:30:21.383229 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:21.389562 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.73 GiB. 2024.10.22 01:30:21.429701 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_523_523_0 (state Temporary) 2024.10.22 01:30:21.463438 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} MergeTask::MergeProjectionsStage: Merge sorted 9625 rows, containing 4 columns (4 merged, 0 gathered) in 2.038352749 sec., 4721.950116201403 rows/sec., 73.78 KiB/sec. 2024.10.22 01:30:21.574817 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_523_523_0 to 202410_536_536_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:21.608299 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} MergedBlockOutputStream: filled checksums 202410_503_508_1 (state Temporary) 2024.10.22 01:30:21.615216 [ 1230 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_531_106 to 202410_536_536_0 2024.10.22 01:30:21.630929 [ 1230 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:21.642324 [ 1230 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.73 GiB. 2024.10.22 01:30:21.702243 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_531_106 to 202410_536_536_0 into Compact with storage Full 2024.10.22 01:30:21.719957 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_503_508_1 to 202410_503_508_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:21.720086 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 1.97 GiB, free memory in arenas 13.41 MiB, will set to 1.60 GiB (RSS), difference: 126.50 MiB 2024.10.22 01:30:21.730273 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_503_508_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_503_503_0, 202410_508_508_0] -> 202410_503_508_1 2024.10.22 01:30:21.772700 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8079 2024.10.22 01:30:21.902706 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:30:21.964429 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:21.972606 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_531_106, total 8120 rows starting from the beginning of the part 2024.10.22 01:30:22.020894 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_532_532_0, total 23 rows starting from the beginning of the part 2024.10.22 01:30:22.027267 [ 1221 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:30:22.076161 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_533_533_0, total 12 rows starting from the beginning of the part 2024.10.22 01:30:22.143439 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_534_534_0, total 13 rows starting from the beginning of the part 2024.10.22 01:30:22.180582 [ 1221 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:30:22.180640 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_535_535_0, total 11 rows starting from the beginning of the part 2024.10.22 01:30:22.257024 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_536_536_0, total 6 rows starting from the beginning of the part 2024.10.22 01:30:22.637044 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:30:23.051936 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 160 entries to flush up to offset 60599 2024.10.22 01:30:23.151931 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_4_4_0 (state Temporary) 2024.10.22 01:30:23.175881 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uke/rvkmqhpxayyckyvcoqxyufrblrfjj (LogSeriesLimiter: on interval from 2024-10-22 01:30:18 to 2024-10-22 01:30:23 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:30:23.179166 [ 1696 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kgw/ijmrqbyxdoxpjjyiiynjhhpqpafca, size 305 2024.10.22 01:30:23.192188 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_444_89, 202410_445_445_0, 202410_446_446_0, 202410_447_447_0, 202410_448_448_0, 202410_449_449_0] 2024.10.22 01:30:23.204162 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_444_89, 202410_445_445_0, 202410_446_446_0, 202410_447_447_0, 202410_448_448_0, 202410_449_449_0] 2024.10.22 01:30:23.225537 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 58550 2024.10.22 01:30:23.277689 [ 1689 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uke/rvkmqhpxayyckyvcoqxyufrblrfjj, size 4 2024.10.22 01:30:23.327790 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:23.345196 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_444_89, 202410_445_445_0, 202410_446_446_0, 202410_447_447_0, 202410_448_448_0, 202410_449_449_0] 2024.10.22 01:30:23.347004 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:23.423765 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_513_513_0 (state Temporary) 2024.10.22 01:30:23.501694 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_513_513_0 to 202410_519_519_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:23.510714 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_891_454} MergeTask::MergeProjectionsStage: Merge sorted 59085 rows, containing 13 columns (13 merged, 0 gathered) in 4.235007611 sec., 13951.568787393142 rows/sec., 2.22 MiB/sec. 2024.10.22 01:30:23.559134 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60599 2024.10.22 01:30:23.575054 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:23.577408 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:23.652609 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 1.97 GiB, free memory in arenas 10.23 MiB, will set to 1.55 GiB (RSS), difference: 122.57 MiB 2024.10.22 01:30:23.716153 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_891_454} MergedBlockOutputStream: filled checksums 202410_1_891_454 (state Temporary) 2024.10.22 01:30:23.768441 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_884_884_0 (state Temporary) 2024.10.22 01:30:23.796686 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} MergeTask::MergeProjectionsStage: Merge sorted 8185 rows, containing 12 columns (12 merged, 0 gathered) in 2.113174367 sec., 3873.319744844321 rows/sec., 615.48 KiB/sec. 2024.10.22 01:30:23.888898 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_891_454} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_891_454 to 202410_1_891_454 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:23.894305 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_891_454} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_885_453, 202410_886_891_1] -> 202410_1_891_454 2024.10.22 01:30:23.957303 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.193116353 sec. 2024.10.22 01:30:24.041805 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:30:24.059890 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} MergedBlockOutputStream: filled checksums 202410_1_536_107 (state Temporary) 2024.10.22 01:30:24.186107 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_884_884_0 to 202410_897_897_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:24.326701 [ 1222 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_891_454 to 202410_892_892_0 2024.10.22 01:30:24.356012 [ 1222 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.12 MiB using storage policy from min volume index 0 2024.10.22 01:30:24.357603 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58550 2024.10.22 01:30:24.358664 [ 1222 ] {} DiskLocal: Reserved 2.12 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:24.378301 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_892_455} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_891_454 to 202410_892_892_0 into Compact with storage Full 2024.10.22 01:30:24.425650 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_892_455} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:24.450527 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_892_455} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_891_454, total 59085 rows starting from the beginning of the part 2024.10.22 01:30:24.511983 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_536_107 to 202410_1_536_107 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:24.521211 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_536_107} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_531_106, 202410_536_536_0] -> 202410_1_536_107 2024.10.22 01:30:24.567668 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.63 MiB. 2024.10.22 01:30:24.573557 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_892_455} MergeTreeSequentialSource: Reading 2 marks from part 202410_892_892_0, total 54 rows starting from the beginning of the part 2024.10.22 01:30:24.883138 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 441 2024.10.22 01:30:24.996702 [ 1217 ] {} DatabaseCatalog: Execution took 225 ms. 2024.10.22 01:30:25.155213 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_793_401, 202410_794_794_0] 2024.10.22 01:30:25.160941 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_793_401, 202410_794_794_0] 2024.10.22 01:30:25.172623 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_793_401, 202410_794_794_0] 2024.10.22 01:30:25.425387 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:25.434515 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:25.487641 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(78e8fe78031bac7e24081e0017dec75a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 78e8fe78031bac7e24081e0017dec75a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:30:25.493558 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 589878, file segment info: File segment: [0, 33554431], key: 78e8fe78031bac7e24081e0017dec75a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:30:25.493102 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_423_423_0 (state Temporary) 2024.10.22 01:30:25.502090 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:30:25.564364 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_105_105_0 to 202410_108_108_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:25.688884 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_423_423_0 to 202410_433_433_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:25.767195 [ 1167 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_428_88 to 202410_433_433_0 2024.10.22 01:30:25.773198 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 441 2024.10.22 01:30:25.786790 [ 1167 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:25.796634 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:25.813655 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_428_88 to 202410_433_433_0 into Compact with storage Full 2024.10.22 01:30:25.968292 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_502_101} MergeTask::MergeProjectionsStage: Merge sorted 794418 rows, containing 4 columns (4 merged, 0 gathered) in 23.906361792 sec., 33230.40146852639 rows/sec., 498.40 KiB/sec. 2024.10.22 01:30:26.057471 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_502_101} MergedBlockOutputStream: filled checksums 202410_1_502_101 (state Temporary) 2024.10.22 01:30:26.091435 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:26.160009 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_428_88, total 472 rows starting from the beginning of the part 2024.10.22 01:30:26.341289 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_429_429_0, total 1 rows starting from the beginning of the part 2024.10.22 01:30:26.365268 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(f8936e9a207c2cb6582aa6f6296cb315) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f8936e9a207c2cb6582aa6f6296cb315, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:30:26.367811 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 665245, file segment info: File segment: [0, 33554431], key: f8936e9a207c2cb6582aa6f6296cb315, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:30:26.372165 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:30:26.379795 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 791993 2024.10.22 01:30:26.705296 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_430_430_0, total 1 rows starting from the beginning of the part 2024.10.22 01:30:26.765564 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:30:26.791459 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:30:26.828638 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:30:26.868748 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:30:26.895481 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:30:27.040920 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_431_431_0, total 1 rows starting from the beginning of the part 2024.10.22 01:30:27.050195 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:30:27.184303 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:30:27.344645 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:30:27.445240 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_432_432_0, total 1 rows starting from the beginning of the part 2024.10.22 01:30:27.501086 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:27.507636 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:27.557353 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(096af66b26a92fdf7f85204b2c2959fd) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 096af66b26a92fdf7f85204b2c2959fd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 01:30:27.572508 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 096af66b26a92fdf7f85204b2c2959fd, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 01:30:27.574960 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:30:27.644893 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_502_502_0 (state Temporary) 2024.10.22 01:30:27.761381 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_433_433_0, total 1 rows starting from the beginning of the part 2024.10.22 01:30:27.829022 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_502_502_0 to 202410_509_509_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:27.868227 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 791993 2024.10.22 01:30:27.910547 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_502_101/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:30:28.167444 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4211 2024.10.22 01:30:28.367539 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 58631 2024.10.22 01:30:28.497749 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 9.95 MiB, will set to 1.54 GiB (RSS), difference: 6.89 MiB 2024.10.22 01:30:28.547125 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_502_101} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_502_101 to 202410_1_502_101 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:28.555011 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_502_101} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_496_100, 202410_497_502_1] -> 202410_1_502_101 2024.10.22 01:30:28.585157 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:30:28.958755 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:28.960624 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:29.013410 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_892_455} MergeTask::MergeProjectionsStage: Merge sorted 59139 rows, containing 13 columns (13 merged, 0 gathered) in 4.640369495 sec., 12744.459264229346 rows/sec., 2.03 MiB/sec. 2024.10.22 01:30:29.080880 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_885_885_0 (state Temporary) 2024.10.22 01:30:29.244082 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_885_885_0 to 202410_898_898_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:29.265126 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_892_455} MergedBlockOutputStream: filled checksums 202410_1_892_455 (state Temporary) 2024.10.22 01:30:29.273966 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_893_893_0 to 202410_898_898_0 2024.10.22 01:30:29.275668 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 8106 2024.10.22 01:30:29.309826 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58631 2024.10.22 01:30:29.412654 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:29.422905 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:29.433396 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:29.445836 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:29.460845 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_893_893_0 to 202410_898_898_0 into Compact with storage Full 2024.10.22 01:30:29.465234 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_524_524_0 (state Temporary) 2024.10.22 01:30:29.504373 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_524_524_0 to 202410_537_537_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:29.591761 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:29.605100 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_893_893_0, total 45 rows starting from the beginning of the part 2024.10.22 01:30:29.623316 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_894_894_0, total 43 rows starting from the beginning of the part 2024.10.22 01:30:29.651010 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_895_895_0, total 33 rows starting from the beginning of the part 2024.10.22 01:30:29.662126 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8106 2024.10.22 01:30:29.684400 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_892_455} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_892_455 to 202410_1_892_455 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:29.687847 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_892_455} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_891_454, 202410_892_892_0] -> 202410_1_892_455 2024.10.22 01:30:29.727785 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:30:29.773632 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_896_896_0, total 39 rows starting from the beginning of the part 2024.10.22 01:30:29.791250 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_897_897_0, total 82 rows starting from the beginning of the part 2024.10.22 01:30:29.923639 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_898_898_0, total 81 rows starting from the beginning of the part 2024.10.22 01:30:30.000311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qcw/llbtnekhbnqrevmcmtebevgjrebam. 2024.10.22 01:30:30.264050 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} MergeTask::MergeProjectionsStage: Merge sorted 477 rows, containing 32 columns (32 merged, 0 gathered) in 4.45600877 sec., 107.04646795387703 rows/sec., 9.93 KiB/sec. 2024.10.22 01:30:30.296183 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} MergedBlockOutputStream: filled checksums 202410_1_433_89 (state Temporary) 2024.10.22 01:30:30.369145 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:30:30.370745 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:30:30.772300 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_89_18, 202410_90_90_0, 202410_91_91_0, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0] 2024.10.22 01:30:30.774290 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_89_18, 202410_90_90_0, 202410_91_91_0, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0] 2024.10.22 01:30:30.813111 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_89_18, 202410_90_90_0, 202410_91_91_0, 202410_92_92_0, 202410_93_93_0, 202410_94_94_0] 2024.10.22 01:30:30.825428 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} MergeTask::MergeProjectionsStage: Merge sorted 323 rows, containing 13 columns (13 merged, 0 gathered) in 1.369516613 sec., 235.84963989042168 rows/sec., 38.72 KiB/sec. 2024.10.22 01:30:30.877641 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_433_89 to 202410_1_433_89 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:30.881616 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_433_89} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_428_88, 202410_433_433_0] -> 202410_1_433_89 2024.10.22 01:30:30.939003 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:30:30.951977 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} MergedBlockOutputStream: filled checksums 202410_893_898_1 (state Temporary) 2024.10.22 01:30:31.008906 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_893_898_1 to 202410_893_898_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:31.016004 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_893_898_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_893_893_0, 202410_898_898_0] -> 202410_893_898_1 2024.10.22 01:30:31.063403 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:30:31.076211 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 107 entries to flush up to offset 60706 2024.10.22 01:30:31.517236 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:31.522707 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:31.573592 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_514_514_0 (state Temporary) 2024.10.22 01:30:31.633511 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_514_514_0 to 202410_520_520_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:31.770842 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_794_402, 202410_795_795_0] 2024.10.22 01:30:31.772099 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_794_402, 202410_795_795_0] 2024.10.22 01:30:31.772542 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60706 2024.10.22 01:30:31.780606 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_794_402, 202410_795_795_0] 2024.10.22 01:30:32.637729 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 8.04 MiB, will set to 1.54 GiB (RSS), difference: 9.97 MiB 2024.10.22 01:30:33.280352 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 442 2024.10.22 01:30:33.324065 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 58690 2024.10.22 01:30:33.439918 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:33.456140 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:33.616643 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_424_424_0 (state Temporary) 2024.10.22 01:30:33.659465 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:33.661221 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:33.706870 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_424_424_0 to 202410_434_434_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:33.766987 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 442 2024.10.22 01:30:33.805524 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_886_886_0 (state Temporary) 2024.10.22 01:30:33.967169 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_886_886_0 to 202410_899_899_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:34.108391 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58690 2024.10.22 01:30:34.872633 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1562 entries to flush up to offset 793555 2024.10.22 01:30:35.728615 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 4254 2024.10.22 01:30:35.856973 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_795_403, 202410_796_796_0] 2024.10.22 01:30:35.863219 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_795_403, 202410_796_796_0] 2024.10.22 01:30:35.873153 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_795_403, 202410_796_796_0] 2024.10.22 01:30:36.178688 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 01:30:36.282302 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:36.285384 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.70 GiB. 2024.10.22 01:30:36.546552 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 8.16 MiB, will set to 1.54 GiB (RSS), difference: -3.05 MiB 2024.10.22 01:30:36.626723 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_503_503_0 (state Temporary) 2024.10.22 01:30:36.708989 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_503_503_0 to 202410_510_510_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:36.759637 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 793555 2024.10.22 01:30:37.164483 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 8128 2024.10.22 01:30:37.183901 [ 1247 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:30:37.312468 [ 1247 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:30:37.348934 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:37.351922 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.70 GiB. 2024.10.22 01:30:37.395720 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_525_525_0 (state Temporary) 2024.10.22 01:30:37.418449 [ 1247 ] {} DNSCacheUpdater: Execution took 233 ms. 2024.10.22 01:30:37.454449 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_525_525_0 to 202410_538_538_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:37.620777 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8128 2024.10.22 01:30:38.125436 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 58722 2024.10.22 01:30:38.385523 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:38.389508 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.70 GiB. 2024.10.22 01:30:38.456254 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_887_887_0 (state Temporary) 2024.10.22 01:30:38.466340 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 8.44 MiB, will set to 1.54 GiB (RSS), difference: -2.08 MiB 2024.10.22 01:30:38.503164 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_887_887_0 to 202410_900_900_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:38.573382 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58722 2024.10.22 01:30:39.014523 [ 1220 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 222 ms. 2024.10.22 01:30:39.291820 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 31 entries to flush up to offset 60737 2024.10.22 01:30:39.739385 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:39.742423 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.70 GiB. 2024.10.22 01:30:39.790406 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_515_515_0 (state Temporary) 2024.10.22 01:30:39.831689 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_515_515_0 to 202410_521_521_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:40.072760 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60737 2024.10.22 01:30:40.282557 [ 1708 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vgr/uydwxzksooxgoefebqhbpkrehfcjh, size 5042029 2024.10.22 01:30:40.366020 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.97 GiB, free memory in arenas 6.88 MiB, will set to 1.54 GiB (RSS), difference: 13.69 MiB 2024.10.22 01:30:40.369647 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:30:40.373292 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:30:40.593301 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:30:40.603751 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:30:40.621533 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:30:40.639393 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:30:40.657737 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:30:40.672486 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:30:40.689304 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:30:40.721954 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:30:40.739782 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:30:41.049535 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/plz/lhxfwmybrfgokkeydmbywohnyimgz (LogSeriesLimiter: on interval from 2024-10-22 01:30:23 to 2024-10-22 01:30:40 accepted series 1 / 12 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:30:41.269478 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 443 2024.10.22 01:30:41.653118 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:41.657035 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:42.041530 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_425_425_0 (state Temporary) 2024.10.22 01:30:42.250691 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_425_425_0 to 202410_435_435_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:42.578640 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 58755 2024.10.22 01:30:42.682910 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.97 GiB, free memory in arenas 7.39 MiB, will set to 1.53 GiB (RSS), difference: -12.67 MiB 2024.10.22 01:30:42.700493 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_796_404, 202410_797_797_0] 2024.10.22 01:30:42.706473 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_796_404, 202410_797_797_0] 2024.10.22 01:30:42.722572 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_796_404, 202410_797_797_0] 2024.10.22 01:30:42.752610 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 443 2024.10.22 01:30:43.137599 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:43.149953 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:43.210309 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_888_888_0 (state Temporary) 2024.10.22 01:30:43.380670 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_888_888_0 to 202410_901_901_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:43.451536 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58755 2024.10.22 01:30:43.764456 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1903 entries to flush up to offset 795458 2024.10.22 01:30:44.920873 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:44.924165 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:45.123898 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 8133 2024.10.22 01:30:45.208016 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_504_504_0 (state Temporary) 2024.10.22 01:30:45.210255 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:45.216036 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:45.261472 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_526_526_0 (state Temporary) 2024.10.22 01:30:45.274356 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_504_504_0 to 202410_511_511_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:45.321265 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 795458 2024.10.22 01:30:45.354995 [ 1222 ] {} DatabaseCatalog: Execution took 220 ms. 2024.10.22 01:30:45.428699 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_526_526_0 to 202410_539_539_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:45.497218 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8133 2024.10.22 01:30:47.388250 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_797_405, 202410_798_798_0] 2024.10.22 01:30:47.391501 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_797_405, 202410_798_798_0] 2024.10.22 01:30:47.457330 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 58781 2024.10.22 01:30:47.508965 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_797_405, 202410_798_798_0] 2024.10.22 01:30:47.575427 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 39 entries to flush up to offset 60776 2024.10.22 01:30:47.696155 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:47.701063 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:47.753218 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_516_516_0 (state Temporary) 2024.10.22 01:30:47.873778 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:47.877059 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:47.924247 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_889_889_0 (state Temporary) 2024.10.22 01:30:47.968986 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_889_889_0 to 202410_902_902_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:48.008497 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_516_516_0 to 202410_522_522_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:48.056544 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60776 2024.10.22 01:30:48.062973 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58781 2024.10.22 01:30:48.105366 [ 1274 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_517_2 to 202410_522_522_0 2024.10.22 01:30:48.120880 [ 1274 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:48.124275 [ 1274 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:48.167684 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_517_2 to 202410_522_522_0 into Compact with storage Full 2024.10.22 01:30:48.264571 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:48.299432 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_517_2, total 1256 rows starting from the beginning of the part 2024.10.22 01:30:48.471626 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_518_518_0, total 58 rows starting from the beginning of the part 2024.10.22 01:30:48.560915 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_519_519_0, total 160 rows starting from the beginning of the part 2024.10.22 01:30:48.714603 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_520_520_0, total 107 rows starting from the beginning of the part 2024.10.22 01:30:48.800469 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_521_521_0, total 31 rows starting from the beginning of the part 2024.10.22 01:30:48.867219 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_522_522_0, total 39 rows starting from the beginning of the part 2024.10.22 01:30:49.509585 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 4.91 MiB, will set to 1.53 GiB (RSS), difference: -3.39 MiB 2024.10.22 01:30:49.896610 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(72cd5490af0710e6e47b150babe9aefe) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 72cd5490af0710e6e47b150babe9aefe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:30:49.901518 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 421121, file segment info: File segment: [0, 33554431], key: 72cd5490af0710e6e47b150babe9aefe, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:30:49.903607 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:30:50.357182 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} MergeTask::MergeProjectionsStage: Merge sorted 1651 rows, containing 13 columns (13 merged, 0 gathered) in 2.200599157 sec., 750.250219240632 rows/sec., 332.64 KiB/sec. 2024.10.22 01:30:50.371616 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:30:50.373682 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:30:50.534335 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_4_4_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:30:50.575757 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 4.90 MiB, will set to 1.53 GiB (RSS), difference: 1.05 MiB 2024.10.22 01:30:50.580746 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(cd8664dde187b099f8377096eb650839) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cd8664dde187b099f8377096eb650839, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:30:50.591350 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: cd8664dde187b099f8377096eb650839, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:30:50.604848 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:30:50.731997 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} MergedBlockOutputStream: filled checksums 202410_507_522_3 (state Temporary) 2024.10.22 01:30:51.396665 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_522_3 to 202410_507_522_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:51.400813 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(5293158adbe19cdfd587aadf03f0f407) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5293158adbe19cdfd587aadf03f0f407, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:30:51.401206 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_522_3} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_517_2, 202410_522_522_0] -> 202410_507_522_3 2024.10.22 01:30:51.403491 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 584044, file segment info: File segment: [0, 33554431], key: 5293158adbe19cdfd587aadf03f0f407, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:30:51.405263 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:30:51.565252 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:30:52.065472 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:30:52.067695 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 58824 2024.10.22 01:30:52.076750 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:30:52.090263 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:30:52.120727 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:30:52.160588 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:30:52.194514 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:30:52.226869 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:30:52.260185 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:30:52.318518 [ 1258 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:30:52.349336 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lji/qeahogkxpotwmzbrlfsauzscurdyx (LogSeriesLimiter: on interval from 2024-10-22 01:30:40 to 2024-10-22 01:30:52 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:30:52.357240 [ 1258 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:30:52.425189 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:52.428566 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 797383 2024.10.22 01:30:52.429071 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:52.576115 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_890_890_0 (state Temporary) 2024.10.22 01:30:52.620188 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_890_890_0 to 202410_903_903_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:52.697366 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58824 2024.10.22 01:30:52.763435 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_892_455 to 202410_893_898_1 2024.10.22 01:30:52.767276 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.13 MiB using storage policy from min volume index 0 2024.10.22 01:30:52.771509 [ 1259 ] {} DiskLocal: Reserved 2.13 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:52.888569 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_898_456} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_892_455 to 202410_893_898_1 into Compact with storage Full 2024.10.22 01:30:53.001737 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 8145 2024.10.22 01:30:53.055253 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_898_456} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:53.075133 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_898_456} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_892_455, total 59139 rows starting from the beginning of the part 2024.10.22 01:30:53.117223 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_898_456} MergeTreeSequentialSource: Reading 2 marks from part 202410_893_898_1, total 323 rows starting from the beginning of the part 2024.10.22 01:30:53.149053 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:53.153759 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:53.187753 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_527_527_0 (state Temporary) 2024.10.22 01:30:53.353777 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_527_527_0 to 202410_540_540_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:53.369491 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:53.374869 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:53.570210 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8145 2024.10.22 01:30:53.711387 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_505_505_0 (state Temporary) 2024.10.22 01:30:53.776135 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_505_505_0 to 202410_512_512_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:53.928149 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 797383 2024.10.22 01:30:53.815355 [ 1160 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_502_101 to 202410_503_508_1 2024.10.22 01:30:53.953812 [ 1160 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:53.959891 [ 1160 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:53.973219 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_508_102} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_502_101 to 202410_503_508_1 into Wide with storage Full 2024.10.22 01:30:54.033129 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_508_102} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:54.046227 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_508_102} MergeTreeSequentialSource: Reading 98 marks from part 202410_1_502_101, total 794418 rows starting from the beginning of the part 2024.10.22 01:30:54.122906 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_508_102} MergeTreeSequentialSource: Reading 3 marks from part 202410_503_508_1, total 9625 rows starting from the beginning of the part 2024.10.22 01:30:54.559412 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_798_406, 202410_799_799_0] 2024.10.22 01:30:54.563107 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_798_406, 202410_799_799_0] 2024.10.22 01:30:54.605732 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_798_406, 202410_799_799_0] 2024.10.22 01:30:54.674826 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:54.678155 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:54.737715 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.97 GiB, free memory in arenas 4.58 MiB, will set to 1.54 GiB (RSS), difference: -30.91 MiB 2024.10.22 01:30:55.163897 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_508_102/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:30:55.563789 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 113 entries to flush up to offset 60889 2024.10.22 01:30:55.827480 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:55.834871 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:55.983915 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_517_517_0 (state Temporary) 2024.10.22 01:30:56.053056 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_517_517_0 to 202410_523_523_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:56.317225 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60889 2024.10.22 01:30:56.586933 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 7.62 MiB, will set to 1.55 GiB (RSS), difference: 6.57 MiB 2024.10.22 01:30:56.707165 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 58881 2024.10.22 01:30:57.181418 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:57.185795 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:57.275388 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_891_891_0 (state Temporary) 2024.10.22 01:30:57.324099 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_891_891_0 to 202410_904_904_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:57.565512 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_899_899_0 to 202410_904_904_0 2024.10.22 01:30:57.586657 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_454_232, 202410_455_460_1] 2024.10.22 01:30:57.587009 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:57.590402 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_454_232, 202410_455_460_1] 2024.10.22 01:30:57.597168 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:57.600099 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58881 2024.10.22 01:30:57.653783 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_899_899_0 to 202410_904_904_0 into Compact with storage Full 2024.10.22 01:30:57.721679 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_454_232, 202410_455_460_1] 2024.10.22 01:30:57.723909 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:30:57.744070 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_899_899_0, total 59 rows starting from the beginning of the part 2024.10.22 01:30:57.755797 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 444 2024.10.22 01:30:57.876608 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_900_900_0, total 32 rows starting from the beginning of the part 2024.10.22 01:30:57.913697 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_901_901_0, total 33 rows starting from the beginning of the part 2024.10.22 01:30:57.954779 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qcw/llbtnekhbnqrevmcmtebevgjrebam. 2024.10.22 01:30:58.044042 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_898_456} MergeTask::MergeProjectionsStage: Merge sorted 59462 rows, containing 13 columns (13 merged, 0 gathered) in 5.263416242 sec., 11297.22546461679 rows/sec., 1.80 MiB/sec. 2024.10.22 01:30:58.068677 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_902_902_0, total 26 rows starting from the beginning of the part 2024.10.22 01:30:58.120252 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_903_903_0, total 43 rows starting from the beginning of the part 2024.10.22 01:30:58.119903 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_106_106_0 (state Temporary) 2024.10.22 01:30:58.123550 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_898_456} MergedBlockOutputStream: filled checksums 202410_1_898_456 (state Temporary) 2024.10.22 01:30:58.169237 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_904_904_0, total 57 rows starting from the beginning of the part 2024.10.22 01:30:58.293454 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:30:58.300066 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:30:58.382796 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.97 GiB, free memory in arenas 10.70 MiB, will set to 1.54 GiB (RSS), difference: 45.62 MiB 2024.10.22 01:30:58.412834 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_799_407, 202410_800_800_0] 2024.10.22 01:30:58.416186 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_799_407, 202410_800_800_0] 2024.10.22 01:30:58.425519 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_898_456} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_898_456 to 202410_1_898_456 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:58.469197 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_898_456} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_892_455, 202410_893_898_1] -> 202410_1_898_456 2024.10.22 01:30:58.484730 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:30:58.573062 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_799_407, 202410_800_800_0] 2024.10.22 01:30:58.593528 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_426_426_0 (state Temporary) 2024.10.22 01:30:58.765696 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_426_426_0 to 202410_436_436_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:30:58.981300 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 444 2024.10.22 01:30:58.983594 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_383_79, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0, 202410_387_387_0, 202410_388_388_0] 2024.10.22 01:30:58.986193 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_383_79, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0, 202410_387_387_0, 202410_388_388_0] 2024.10.22 01:30:59.024351 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_383_79, 202410_384_384_0, 202410_385_385_0, 202410_386_386_0, 202410_387_387_0, 202410_388_388_0] 2024.10.22 01:30:59.025453 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 01:30:59.592055 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} MergeTask::MergeProjectionsStage: Merge sorted 250 rows, containing 13 columns (13 merged, 0 gathered) in 1.9553496 sec., 127.85437448116696 rows/sec., 19.82 KiB/sec. 2024.10.22 01:30:59.724538 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_106_106_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:30:59.847275 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} MergedBlockOutputStream: filled checksums 202410_899_904_1 (state Temporary) 2024.10.22 01:31:00.076305 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_899_904_1 to 202410_899_904_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:00.080662 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_899_904_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_899_899_0, 202410_904_904_0] -> 202410_899_904_1 2024.10.22 01:31:00.207112 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:31:00.313562 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:31:00.369827 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.97 GiB, free memory in arenas 13.80 MiB, will set to 1.54 GiB (RSS), difference: 95.84 MiB 2024.10.22 01:31:00.370854 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:31:00.372191 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:31:00.410834 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_476_95, 202410_477_477_0, 202410_478_478_0, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0] 2024.10.22 01:31:00.415047 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_476_95, 202410_477_477_0, 202410_478_478_0, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0] 2024.10.22 01:31:00.465718 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_476_95, 202410_477_477_0, 202410_478_478_0, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0] 2024.10.22 01:31:00.937006 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1833 entries to flush up to offset 799216 2024.10.22 01:31:01.074454 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 8160 2024.10.22 01:31:01.385658 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:01.398816 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:01.440070 [ 1732 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/plz/lhxfwmybrfgokkeydmbywohnyimgz, size 3897085 2024.10.22 01:31:01.455962 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_528_528_0 (state Temporary) 2024.10.22 01:31:01.518992 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_528_528_0 to 202410_541_541_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:01.567296 [ 1210 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_536_107 to 202410_541_541_0 2024.10.22 01:31:01.576166 [ 1210 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:01.578626 [ 1210 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:01.612608 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 58945 2024.10.22 01:31:01.650273 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:31:01.658675 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_536_107 to 202410_541_541_0 into Compact with storage Full 2024.10.22 01:31:01.661229 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:31:01.669122 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:31:01.702091 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:01.716478 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8160 2024.10.22 01:31:01.716920 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_536_107, total 8185 rows starting from the beginning of the part 2024.10.22 01:31:01.728522 [ 1200 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:31:01.754287 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_537_537_0, total 27 rows starting from the beginning of the part 2024.10.22 01:31:01.776451 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:31:01.816499 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:31:01.817298 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_538_538_0, total 22 rows starting from the beginning of the part 2024.10.22 01:31:01.834810 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:31:01.863921 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:31:01.880274 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_539_539_0, total 5 rows starting from the beginning of the part 2024.10.22 01:31:01.892592 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:31:01.926068 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:31:01.951691 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_540_540_0, total 12 rows starting from the beginning of the part 2024.10.22 01:31:01.977259 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qrd/rweqosuxphxypxxbanjkdnvhmvgwh (LogSeriesLimiter: on interval from 2024-10-22 01:30:52 to 2024-10-22 01:31:01 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:31:01.995905 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_541_541_0, total 15 rows starting from the beginning of the part 2024.10.22 01:31:02.007769 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_801_801_0, 202410_802_802_0, 202410_803_803_0, 202410_804_804_0, 202410_805_805_0, 202410_806_806_0] 2024.10.22 01:31:02.013510 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_801_801_0, 202410_802_802_0, 202410_803_803_0, 202410_804_804_0, 202410_805_805_0, 202410_806_806_0] 2024.10.22 01:31:02.081401 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_801_801_0, 202410_802_802_0, 202410_803_803_0, 202410_804_804_0, 202410_805_805_0, 202410_806_806_0] 2024.10.22 01:31:02.178066 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:02.183077 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:02.204448 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:02.209837 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:02.281808 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_892_892_0 (state Temporary) 2024.10.22 01:31:02.375525 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_506_506_0 (state Temporary) 2024.10.22 01:31:02.464958 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_506_506_0 to 202410_513_513_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:02.468600 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_461_461_0, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0, 202410_465_465_0, 202410_466_466_0] 2024.10.22 01:31:02.471909 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_461_461_0, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0, 202410_465_465_0, 202410_466_466_0] 2024.10.22 01:31:02.510489 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 799216 2024.10.22 01:31:02.523883 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_461_461_0, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0, 202410_465_465_0, 202410_466_466_0] 2024.10.22 01:31:02.553764 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_892_892_0 to 202410_905_905_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:02.591694 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 58945 2024.10.22 01:31:02.637537 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.97 GiB, free memory in arenas 15.14 MiB, will set to 1.53 GiB (RSS), difference: 45.01 MiB 2024.10.22 01:31:02.871173 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4b24e97f4917e3ceda599ff47a43bee2) : [0, 4541380]: Setting download as failed: File segment: [0, 4541380], key: 4b24e97f4917e3ceda599ff47a43bee2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:02.871915 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4b24e97f4917e3ceda599ff47a43bee2): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:31:02.968192 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4b24e97f4917e3ceda599ff47a43bee2, offset: 0, size: 0 2024.10.22 01:31:02.969804 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4b24e97f4917e3ceda599ff47a43bee2 for removal 2024.10.22 01:31:02.973073 [ 1476 ] {} CacheMetadata: Key 4b24e97f4917e3ceda599ff47a43bee2 is removed from metadata 2024.10.22 01:31:03.340031 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} MergeTask::MergeProjectionsStage: Merge sorted 8266 rows, containing 12 columns (12 merged, 0 gathered) in 1.754226884 sec., 4712.047270163715 rows/sec., 748.92 KiB/sec. 2024.10.22 01:31:03.375922 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} MergedBlockOutputStream: filled checksums 202410_1_541_108 (state Temporary) 2024.10.22 01:31:03.768728 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 109739 bytes for 3134a50cabaaa2e335b7e5d9cc62dcd1:0) 2024.10.22 01:31:03.773292 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b0d2321374576dcc1e7dcad4e622d775, offset: 0, size: 161 2024.10.22 01:31:03.778395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b0d2321374576dcc1e7dcad4e622d775 for removal 2024.10.22 01:31:03.781748 [ 1476 ] {} CacheMetadata: Key b0d2321374576dcc1e7dcad4e622d775 is removed from metadata 2024.10.22 01:31:03.782850 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 594a3ed4ef93be968949514e898fe7fa, offset: 0, size: 237 2024.10.22 01:31:03.786352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 594a3ed4ef93be968949514e898fe7fa for removal 2024.10.22 01:31:03.792416 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 65cdec648c12be4793f219ee5b7d0102, offset: 0, size: 239 2024.10.22 01:31:03.799617 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 65cdec648c12be4793f219ee5b7d0102 for removal 2024.10.22 01:31:03.804258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0d190bb1b0e2c398b8d1ab4ff1c4233f, offset: 0, size: 4 2024.10.22 01:31:03.810554 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0d190bb1b0e2c398b8d1ab4ff1c4233f for removal 2024.10.22 01:31:03.814433 [ 1476 ] {} CacheMetadata: Key 65cdec648c12be4793f219ee5b7d0102 is removed from metadata 2024.10.22 01:31:03.835715 [ 1476 ] {} CacheMetadata: Key 0d190bb1b0e2c398b8d1ab4ff1c4233f is removed from metadata 2024.10.22 01:31:03.816128 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_541_108 to 202410_1_541_108 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:03.841462 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_541_108} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_536_107, 202410_541_541_0] -> 202410_1_541_108 2024.10.22 01:31:03.850407 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 60960 2024.10.22 01:31:03.850810 [ 1476 ] {} CacheMetadata: Key 594a3ed4ef93be968949514e898fe7fa is removed from metadata 2024.10.22 01:31:03.875492 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.54 MiB. 2024.10.22 01:31:04.148183 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:04.162244 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:04.208103 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_518_518_0 (state Temporary) 2024.10.22 01:31:04.263954 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_518_518_0 to 202410_524_524_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:04.372278 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(3dc363cdc529598442857f5e3b75381e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3dc363cdc529598442857f5e3b75381e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:31:04.374285 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 3dc363cdc529598442857f5e3b75381e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:31:04.375752 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:31:04.562220 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 60960 2024.10.22 01:31:04.581125 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 17.48 MiB, will set to 1.53 GiB (RSS), difference: 7.88 MiB 2024.10.22 01:31:04.906351 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 192 bytes for 74b9203768ad8b4dd845a15cfc217562:0) 2024.10.22 01:31:04.908770 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: eb68d6167fa22a3242253cf33c6deb3b, offset: 0, size: 2404 2024.10.22 01:31:04.914722 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key eb68d6167fa22a3242253cf33c6deb3b for removal 2024.10.22 01:31:04.925426 [ 1476 ] {} CacheMetadata: Key eb68d6167fa22a3242253cf33c6deb3b is removed from metadata 2024.10.22 01:31:05.384100 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_508_102/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:31:05.567612 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 5 file segments (while reserving 5082 bytes for 753e8ad7a713717cc7390e0f34e9c7ec:0) 2024.10.22 01:31:05.570385 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8e212c240e6badd88c4d70b0aa0d1981, offset: 0, size: 245 2024.10.22 01:31:05.674956 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8e212c240e6badd88c4d70b0aa0d1981 for removal 2024.10.22 01:31:05.678568 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dfb95f6f6dc88e8d12b998eb6b56506a, offset: 0, size: 137258 2024.10.22 01:31:05.681756 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dfb95f6f6dc88e8d12b998eb6b56506a for removal 2024.10.22 01:31:05.687155 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 98484d880fb462eea96419e41432b48c, offset: 0, size: 260 2024.10.22 01:31:05.693510 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 98484d880fb462eea96419e41432b48c for removal 2024.10.22 01:31:05.700375 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ee1efbc84c970a447e2adb7521052f11, offset: 0, size: 166 2024.10.22 01:31:05.705298 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ee1efbc84c970a447e2adb7521052f11 for removal 2024.10.22 01:31:05.711734 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2d9fcc77ab9c5093c11df580f65752f3, offset: 0, size: 252 2024.10.22 01:31:05.718890 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2d9fcc77ab9c5093c11df580f65752f3 for removal 2024.10.22 01:31:05.778891 [ 1476 ] {} CacheMetadata: Key 8e212c240e6badd88c4d70b0aa0d1981 is removed from metadata 2024.10.22 01:31:05.886964 [ 1476 ] {} CacheMetadata: Key 2d9fcc77ab9c5093c11df580f65752f3 is removed from metadata 2024.10.22 01:31:05.979869 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_106_106_0 to 202410_109_109_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:05.998486 [ 1476 ] {} CacheMetadata: Key ee1efbc84c970a447e2adb7521052f11 is removed from metadata 2024.10.22 01:31:06.012017 [ 1476 ] {} CacheMetadata: Key 98484d880fb462eea96419e41432b48c is removed from metadata 2024.10.22 01:31:06.124540 [ 1476 ] {} CacheMetadata: Key dfb95f6f6dc88e8d12b998eb6b56506a is removed from metadata 2024.10.22 01:31:06.484571 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 445 2024.10.22 01:31:06.572302 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 17.59 MiB, will set to 1.53 GiB (RSS), difference: 3.47 MiB 2024.10.22 01:31:06.604238 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 95 entries to flush up to offset 59040 2024.10.22 01:31:06.744145 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(43b9524e66038d9835e18177eda605fc) : [0, 2181440]: Setting download as failed: File segment: [0, 2181440], key: 43b9524e66038d9835e18177eda605fc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:06.747685 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(43b9524e66038d9835e18177eda605fc): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:31:06.821755 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 43b9524e66038d9835e18177eda605fc, offset: 0, size: 0 2024.10.22 01:31:06.824636 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 43b9524e66038d9835e18177eda605fc for removal 2024.10.22 01:31:06.828698 [ 1476 ] {} CacheMetadata: Key 43b9524e66038d9835e18177eda605fc is removed from metadata 2024.10.22 01:31:07.041524 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:07.058966 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:07.122924 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:07.125105 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:07.224309 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_427_427_0 (state Temporary) 2024.10.22 01:31:07.300484 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_449_90, 202410_450_450_0, 202410_451_451_0, 202410_452_452_0, 202410_453_453_0, 202410_454_454_0] 2024.10.22 01:31:07.305135 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_449_90, 202410_450_450_0, 202410_451_451_0, 202410_452_452_0, 202410_453_453_0, 202410_454_454_0] 2024.10.22 01:31:07.360447 [ 1217 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:31:07.403380 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_893_893_0 (state Temporary) 2024.10.22 01:31:07.415503 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_427_427_0 to 202410_437_437_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:07.422943 [ 1217 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:31:07.483424 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_893_893_0 to 202410_906_906_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:07.514336 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 445 2024.10.22 01:31:07.533747 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_449_90, 202410_450_450_0, 202410_451_451_0, 202410_452_452_0, 202410_453_453_0, 202410_454_454_0] 2024.10.22 01:31:07.674964 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59040 2024.10.22 01:31:07.756751 [ 1237 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 01:31:08.605011 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1321234afb3c0ab71ccc140f2c8b2491) : [0, 705988]: Setting download as failed: File segment: [0, 705988], key: 1321234afb3c0ab71ccc140f2c8b2491, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:08.609524 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1321234afb3c0ab71ccc140f2c8b2491): No space left in cache to reserve 705989 bytes, will continue without cache download 2024.10.22 01:31:08.672645 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1321234afb3c0ab71ccc140f2c8b2491, offset: 0, size: 0 2024.10.22 01:31:08.675020 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1321234afb3c0ab71ccc140f2c8b2491 for removal 2024.10.22 01:31:08.705104 [ 1476 ] {} CacheMetadata: Key 1321234afb3c0ab71ccc140f2c8b2491 is removed from metadata 2024.10.22 01:31:09.220844 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 8174 2024.10.22 01:31:09.466811 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:09.469009 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:09.510582 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_529_529_0 (state Temporary) 2024.10.22 01:31:09.520948 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1634 entries to flush up to offset 800850 2024.10.22 01:31:09.571247 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_529_529_0 to 202410_542_542_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:09.660477 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(42c3b5e83c4c1c3891329ccba95bf42e) : [0, 412442]: Setting download as failed: File segment: [0, 412442], key: 42c3b5e83c4c1c3891329ccba95bf42e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:09.661562 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(42c3b5e83c4c1c3891329ccba95bf42e): No space left in cache to reserve 412443 bytes, will continue without cache download 2024.10.22 01:31:09.724226 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8174 2024.10.22 01:31:09.815233 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 42c3b5e83c4c1c3891329ccba95bf42e, offset: 0, size: 0 2024.10.22 01:31:09.817683 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 42c3b5e83c4c1c3891329ccba95bf42e for removal 2024.10.22 01:31:09.820684 [ 1476 ] {} CacheMetadata: Key 42c3b5e83c4c1c3891329ccba95bf42e is removed from metadata 2024.10.22 01:31:10.063533 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4254 2024.10.22 01:31:10.372835 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:31:10.374794 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:31:10.472632 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(be4afbaf4527158d9f9facac386ce870) : [0, 923937]: Setting download as failed: File segment: [0, 923937], key: be4afbaf4527158d9f9facac386ce870, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:10.473682 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(be4afbaf4527158d9f9facac386ce870): No space left in cache to reserve 923938 bytes, will continue without cache download 2024.10.22 01:31:10.699488 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:10.703929 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:10.782939 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 16.49 MiB, will set to 1.53 GiB (RSS), difference: 7.20 MiB 2024.10.22 01:31:10.863761 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_507_507_0 (state Temporary) 2024.10.22 01:31:10.945410 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: be4afbaf4527158d9f9facac386ce870, offset: 0, size: 0 2024.10.22 01:31:10.950388 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key be4afbaf4527158d9f9facac386ce870 for removal 2024.10.22 01:31:10.956568 [ 1476 ] {} CacheMetadata: Key be4afbaf4527158d9f9facac386ce870 is removed from metadata 2024.10.22 01:31:11.163379 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_507_507_0 to 202410_514_514_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:11.248308 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 800850 2024.10.22 01:31:11.354259 [ 1277 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_509_509_0 to 202410_514_514_0 2024.10.22 01:31:11.372169 [ 1277 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:11.374794 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:11.392718 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_509_509_0 to 202410_514_514_0 into Compact with storage Full 2024.10.22 01:31:11.447808 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:11.458128 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_509_509_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:31:11.657549 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_510_510_0, total 1562 rows starting from the beginning of the part 2024.10.22 01:31:11.672809 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_511_511_0, total 1903 rows starting from the beginning of the part 2024.10.22 01:31:11.692391 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 59101 2024.10.22 01:31:11.711809 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_512_512_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:31:11.943631 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 167623 bytes for 0899fab9161a2065801752f9186adf11:0) 2024.10.22 01:31:11.942121 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_513_513_0, total 1833 rows starting from the beginning of the part 2024.10.22 01:31:11.951195 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3134a50cabaaa2e335b7e5d9cc62dcd1, offset: 0, size: 109739 2024.10.22 01:31:11.957185 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3134a50cabaaa2e335b7e5d9cc62dcd1 for removal 2024.10.22 01:31:11.959866 [ 1476 ] {} CacheMetadata: Key 3134a50cabaaa2e335b7e5d9cc62dcd1 is removed from metadata 2024.10.22 01:31:11.960619 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1c17b78aedab2ce22049c7b8227bc82f, offset: 0, size: 164 2024.10.22 01:31:11.963080 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1c17b78aedab2ce22049c7b8227bc82f for removal 2024.10.22 01:31:11.970595 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d0719916e51ddf122a7f893d71e65939, offset: 0, size: 286 2024.10.22 01:31:11.978295 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d0719916e51ddf122a7f893d71e65939 for removal 2024.10.22 01:31:11.981764 [ 1476 ] {} CacheMetadata: Key d0719916e51ddf122a7f893d71e65939 is removed from metadata 2024.10.22 01:31:11.986054 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_514_514_0, total 1634 rows starting from the beginning of the part 2024.10.22 01:31:11.998766 [ 1476 ] {} CacheMetadata: Key 1c17b78aedab2ce22049c7b8227bc82f is removed from metadata 2024.10.22 01:31:12.033401 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:12.041817 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:12.068205 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 42 entries to flush up to offset 61002 2024.10.22 01:31:12.083072 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_894_894_0 (state Temporary) 2024.10.22 01:31:12.258607 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_894_894_0 to 202410_907_907_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:12.307430 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59101 2024.10.22 01:31:12.335708 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:12.339073 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:12.377426 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_519_519_0 (state Temporary) 2024.10.22 01:31:12.566961 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 19.85 MiB, will set to 1.53 GiB (RSS), difference: 1.25 MiB 2024.10.22 01:31:12.626942 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_519_519_0 to 202410_525_525_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:12.673649 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 61002 2024.10.22 01:31:12.884571 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 12 file segments (while reserving 195732 bytes for 106c375ac5c492d5d3afdc569453d31c:0) 2024.10.22 01:31:12.886746 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0899fab9161a2065801752f9186adf11, offset: 0, size: 167623 2024.10.22 01:31:12.900707 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0899fab9161a2065801752f9186adf11 for removal 2024.10.22 01:31:12.905678 [ 1476 ] {} CacheMetadata: Key 0899fab9161a2065801752f9186adf11 is removed from metadata 2024.10.22 01:31:12.906709 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 753e8ad7a713717cc7390e0f34e9c7ec, offset: 0, size: 5082 2024.10.22 01:31:12.912215 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 753e8ad7a713717cc7390e0f34e9c7ec for removal 2024.10.22 01:31:12.916505 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5e16f8d7477d43602901dfcfad508295, offset: 0, size: 250 2024.10.22 01:31:12.920825 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5e16f8d7477d43602901dfcfad508295 for removal 2024.10.22 01:31:12.931073 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 38ecf86f8e0d345b107c69456601fee7, offset: 0, size: 193 2024.10.22 01:31:12.942708 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 38ecf86f8e0d345b107c69456601fee7 for removal 2024.10.22 01:31:12.950672 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: df9d48ab0ade343903eacec1b848f575, offset: 0, size: 25310 2024.10.22 01:31:12.954471 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key df9d48ab0ade343903eacec1b848f575 for removal 2024.10.22 01:31:12.960130 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d7ae21d7bdc4487b881f7d45d23afb86, offset: 0, size: 234 2024.10.22 01:31:12.963907 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d7ae21d7bdc4487b881f7d45d23afb86 for removal 2024.10.22 01:31:12.967312 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5d1a70983ef52a036341482b11993239, offset: 0, size: 241 2024.10.22 01:31:12.973518 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5d1a70983ef52a036341482b11993239 for removal 2024.10.22 01:31:12.977374 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7df635e072844f9d2c1f44466fa0685d, offset: 0, size: 239 2024.10.22 01:31:12.981119 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7df635e072844f9d2c1f44466fa0685d for removal 2024.10.22 01:31:12.987742 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4707fc1528935404c2843953ce198a0b, offset: 0, size: 32556 2024.10.22 01:31:12.995619 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4707fc1528935404c2843953ce198a0b for removal 2024.10.22 01:31:12.999901 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d587e40f44dea65f07d4ee01522e3dac, offset: 0, size: 170 2024.10.22 01:31:13.004550 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d587e40f44dea65f07d4ee01522e3dac for removal 2024.10.22 01:31:13.022993 [ 1476 ] {} CacheMetadata: Key d587e40f44dea65f07d4ee01522e3dac is removed from metadata 2024.10.22 01:31:13.024532 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1660412ba5524efb5f31a615026c503d, offset: 0, size: 269 2024.10.22 01:31:13.029424 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1660412ba5524efb5f31a615026c503d for removal 2024.10.22 01:31:13.042925 [ 1476 ] {} CacheMetadata: Key 1660412ba5524efb5f31a615026c503d is removed from metadata 2024.10.22 01:31:13.138845 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 74b9203768ad8b4dd845a15cfc217562, offset: 0, size: 192 2024.10.22 01:31:13.148926 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 74b9203768ad8b4dd845a15cfc217562 for removal 2024.10.22 01:31:13.452182 [ 1476 ] {} CacheMetadata: Key 74b9203768ad8b4dd845a15cfc217562 is removed from metadata 2024.10.22 01:31:13.461273 [ 1476 ] {} CacheMetadata: Key 4707fc1528935404c2843953ce198a0b is removed from metadata 2024.10.22 01:31:13.467837 [ 1476 ] {} CacheMetadata: Key 7df635e072844f9d2c1f44466fa0685d is removed from metadata 2024.10.22 01:31:13.474615 [ 1476 ] {} CacheMetadata: Key 5d1a70983ef52a036341482b11993239 is removed from metadata 2024.10.22 01:31:13.483710 [ 1476 ] {} CacheMetadata: Key d7ae21d7bdc4487b881f7d45d23afb86 is removed from metadata 2024.10.22 01:31:13.490890 [ 1476 ] {} CacheMetadata: Key df9d48ab0ade343903eacec1b848f575 is removed from metadata 2024.10.22 01:31:13.504117 [ 1476 ] {} CacheMetadata: Key 38ecf86f8e0d345b107c69456601fee7 is removed from metadata 2024.10.22 01:31:13.516753 [ 1476 ] {} CacheMetadata: Key 5e16f8d7477d43602901dfcfad508295 is removed from metadata 2024.10.22 01:31:13.529258 [ 1476 ] {} CacheMetadata: Key 753e8ad7a713717cc7390e0f34e9c7ec is removed from metadata 2024.10.22 01:31:13.560206 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(5bec27c39a4f955b239f9aacfff68355) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5bec27c39a4f955b239f9aacfff68355, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:31:13.563417 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 5bec27c39a4f955b239f9aacfff68355, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:31:13.566409 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:31:13.642453 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} MergeTask::MergeProjectionsStage: Merge sorted 10397 rows, containing 4 columns (4 merged, 0 gathered) in 2.261083252 sec., 4598.238472998958 rows/sec., 71.85 KiB/sec. 2024.10.22 01:31:13.923279 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} MergedBlockOutputStream: filled checksums 202410_509_514_1 (state Temporary) 2024.10.22 01:31:14.115311 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_509_514_1 to 202410_509_514_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:14.121826 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_509_514_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_509_509_0, 202410_514_514_0] -> 202410_509_514_1 2024.10.22 01:31:14.161307 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:31:14.455975 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7c05a43f62e8a2f82932aa105c186ebe) : [0, 291706]: Setting download as failed: File segment: [0, 291706], key: 7c05a43f62e8a2f82932aa105c186ebe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:14.457228 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7c05a43f62e8a2f82932aa105c186ebe): No space left in cache to reserve 291707 bytes, will continue without cache download 2024.10.22 01:31:14.683148 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7c05a43f62e8a2f82932aa105c186ebe, offset: 0, size: 0 2024.10.22 01:31:14.791108 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7c05a43f62e8a2f82932aa105c186ebe for removal 2024.10.22 01:31:14.796588 [ 1476 ] {} CacheMetadata: Key 7c05a43f62e8a2f82932aa105c186ebe is removed from metadata 2024.10.22 01:31:15.016662 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 446 2024.10.22 01:31:15.321628 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:15.327394 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:15.489902 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_428_428_0 (state Temporary) 2024.10.22 01:31:15.512529 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 18.92 MiB, will set to 1.53 GiB (RSS), difference: 2.47 MiB 2024.10.22 01:31:15.722891 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_428_428_0 to 202410_438_438_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:15.755471 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(03dc20f4384b44c450e2bdaad70c5329) : [0, 286895]: Setting download as failed: File segment: [0, 286895], key: 03dc20f4384b44c450e2bdaad70c5329, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:15.756346 [ 1179 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_433_89 to 202410_438_438_0 2024.10.22 01:31:15.756971 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(03dc20f4384b44c450e2bdaad70c5329): No space left in cache to reserve 286896 bytes, will continue without cache download 2024.10.22 01:31:15.779192 [ 1179 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:15.791928 [ 1179 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:15.803588 [ 1252 ] {} DatabaseCatalog: Execution took 228 ms. 2024.10.22 01:31:15.814858 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_433_89 to 202410_438_438_0 into Compact with storage Full 2024.10.22 01:31:15.882652 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 03dc20f4384b44c450e2bdaad70c5329, offset: 0, size: 0 2024.10.22 01:31:15.900582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 03dc20f4384b44c450e2bdaad70c5329 for removal 2024.10.22 01:31:15.905005 [ 1476 ] {} CacheMetadata: Key 03dc20f4384b44c450e2bdaad70c5329 is removed from metadata 2024.10.22 01:31:15.908350 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 446 2024.10.22 01:31:16.186262 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_508_102/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:31:16.208627 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:16.261672 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_433_89, total 477 rows starting from the beginning of the part 2024.10.22 01:31:16.250494 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(2d429385fa389a92f1f9be112574fa0a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2d429385fa389a92f1f9be112574fa0a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:31:16.268410 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 837720, file segment info: File segment: [0, 33554431], key: 2d429385fa389a92f1f9be112574fa0a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:31:16.270056 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:31:16.406696 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 99 entries to flush up to offset 59200 2024.10.22 01:31:16.667013 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_434_434_0, total 1 rows starting from the beginning of the part 2024.10.22 01:31:16.707441 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:16.713001 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:16.869770 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 142502 bytes for 66988942498f4967adfb7cca1a01788e:0) 2024.10.22 01:31:16.871218 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 106c375ac5c492d5d3afdc569453d31c, offset: 0, size: 195732 2024.10.22 01:31:16.873240 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 106c375ac5c492d5d3afdc569453d31c for removal 2024.10.22 01:31:16.874477 [ 1476 ] {} CacheMetadata: Key 106c375ac5c492d5d3afdc569453d31c is removed from metadata 2024.10.22 01:31:16.875013 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c25b8a8ee9280d3927da40dddd18ce58, offset: 0, size: 169 2024.10.22 01:31:16.876918 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c25b8a8ee9280d3927da40dddd18ce58 for removal 2024.10.22 01:31:16.877257 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_895_895_0 (state Temporary) 2024.10.22 01:31:16.884920 [ 1476 ] {} CacheMetadata: Key c25b8a8ee9280d3927da40dddd18ce58 is removed from metadata 2024.10.22 01:31:17.032311 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_895_895_0 to 202410_908_908_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:17.090914 [ 1712 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lji/qeahogkxpotwmzbrlfsauzscurdyx, size 2257048 2024.10.22 01:31:17.115051 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_435_435_0, total 1 rows starting from the beginning of the part 2024.10.22 01:31:17.226406 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 8187 2024.10.22 01:31:17.293154 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59200 2024.10.22 01:31:17.422327 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:31:17.432003 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:31:17.455532 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:31:17.469034 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:31:17.472955 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:17.477428 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:17.480659 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.97 GiB, free memory in arenas 18.17 MiB, will set to 1.53 GiB (RSS), difference: 6.68 MiB 2024.10.22 01:31:17.485703 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:31:17.534380 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:31:17.667252 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_530_530_0 (state Temporary) 2024.10.22 01:31:17.677644 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:31:17.702974 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:31:17.723799 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_436_436_0, total 1 rows starting from the beginning of the part 2024.10.22 01:31:17.820932 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_530_530_0 to 202410_543_543_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:17.827497 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 4296 2024.10.22 01:31:17.890220 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qvy/hlkywmoyhijywhrfirejnycgvzqjv (LogSeriesLimiter: on interval from 2024-10-22 01:31:01 to 2024-10-22 01:31:17 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:31:17.967781 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8187 2024.10.22 01:31:18.055767 [ 1189 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 256 ms. 2024.10.22 01:31:18.058147 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 130648 bytes for 1117f533e039401a70aca403d247b6a9:0) 2024.10.22 01:31:18.069633 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 66988942498f4967adfb7cca1a01788e, offset: 0, size: 142502 2024.10.22 01:31:18.072025 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 66988942498f4967adfb7cca1a01788e for removal 2024.10.22 01:31:18.075279 [ 1476 ] {} CacheMetadata: Key 66988942498f4967adfb7cca1a01788e is removed from metadata 2024.10.22 01:31:18.084959 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3b185f6c15db5b4b7640bcd89915f33e, offset: 0, size: 170 2024.10.22 01:31:18.091472 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3b185f6c15db5b4b7640bcd89915f33e for removal 2024.10.22 01:31:18.105003 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 026df7151edac5f23ee446f476000dd9, offset: 0, size: 202 2024.10.22 01:31:18.113609 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 026df7151edac5f23ee446f476000dd9 for removal 2024.10.22 01:31:18.120516 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 37e51cf11c750e61ccea0f370e39fd24, offset: 0, size: 202 2024.10.22 01:31:18.126784 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 37e51cf11c750e61ccea0f370e39fd24 for removal 2024.10.22 01:31:18.128417 [ 1476 ] {} CacheMetadata: Key 026df7151edac5f23ee446f476000dd9 is removed from metadata 2024.10.22 01:31:18.152802 [ 1476 ] {} CacheMetadata: Key 37e51cf11c750e61ccea0f370e39fd24 is removed from metadata 2024.10.22 01:31:18.162240 [ 1476 ] {} CacheMetadata: Key 3b185f6c15db5b4b7640bcd89915f33e is removed from metadata 2024.10.22 01:31:18.188355 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_437_437_0, total 1 rows starting from the beginning of the part 2024.10.22 01:31:18.253678 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 802005 2024.10.22 01:31:19.012067 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_438_438_0, total 1 rows starting from the beginning of the part 2024.10.22 01:31:19.532491 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:19.541519 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:19.622434 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(85ce56cb37c51968f0d73e663697a975) : [0, 294283]: Setting download as failed: File segment: [0, 294283], key: 85ce56cb37c51968f0d73e663697a975, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:19.624831 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(85ce56cb37c51968f0d73e663697a975): No space left in cache to reserve 294284 bytes, will continue without cache download 2024.10.22 01:31:19.842334 [ 1234 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 5 parts from 202410_105_105_0 to 202410_109_109_0 2024.10.22 01:31:19.858534 [ 1234 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:19.862108 [ 1234 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:19.884023 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 85ce56cb37c51968f0d73e663697a975, offset: 0, size: 0 2024.10.22 01:31:19.885619 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_105_109_1} MergeTask::PrepareStage: Merging 5 parts: from 202410_105_105_0 to 202410_109_109_0 into Compact with storage Full 2024.10.22 01:31:19.885813 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 85ce56cb37c51968f0d73e663697a975 for removal 2024.10.22 01:31:19.893452 [ 1476 ] {} CacheMetadata: Key 85ce56cb37c51968f0d73e663697a975 is removed from metadata 2024.10.22 01:31:19.902655 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_508_508_0 (state Temporary) 2024.10.22 01:31:20.021401 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_508_508_0 to 202410_515_515_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:20.061280 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 802005 2024.10.22 01:31:20.278066 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 73 entries to flush up to offset 61075 2024.10.22 01:31:20.374741 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:31:20.381172 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:31:20.436287 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:20.447865 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:20.701739 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_520_520_0 (state Temporary) 2024.10.22 01:31:20.776137 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 15.29 MiB, will set to 1.53 GiB (RSS), difference: -11.56 MiB 2024.10.22 01:31:20.875416 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_520_520_0 to 202410_526_526_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:20.955289 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 53956 bytes for fc2376e62f7920e4e6127f2e9ad3f3c2:0) 2024.10.22 01:31:20.958098 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1117f533e039401a70aca403d247b6a9, offset: 0, size: 130648 2024.10.22 01:31:20.960297 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1117f533e039401a70aca403d247b6a9 for removal 2024.10.22 01:31:20.961956 [ 1476 ] {} CacheMetadata: Key 1117f533e039401a70aca403d247b6a9 is removed from metadata 2024.10.22 01:31:20.962677 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5238a28644a8e99229fb1a70375ee572, offset: 0, size: 169 2024.10.22 01:31:20.968456 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5238a28644a8e99229fb1a70375ee572 for removal 2024.10.22 01:31:20.978105 [ 1476 ] {} CacheMetadata: Key 5238a28644a8e99229fb1a70375ee572 is removed from metadata 2024.10.22 01:31:21.052800 [ 1709 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qrd/rweqosuxphxypxxbanjkdnvhmvgwh, size 2113279 2024.10.22 01:31:21.256838 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 61075 2024.10.22 01:31:21.313344 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_508_102} MergeTask::MergeProjectionsStage: Merge sorted 804043 rows, containing 4 columns (4 merged, 0 gathered) in 27.346074284 sec., 29402.50186003627 rows/sec., 440.93 KiB/sec. 2024.10.22 01:31:21.313445 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 59282 2024.10.22 01:31:21.380874 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_508_102} MergedBlockOutputStream: filled checksums 202410_1_508_102 (state Temporary) 2024.10.22 01:31:21.426792 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} MergeTask::MergeProjectionsStage: Merge sorted 482 rows, containing 32 columns (32 merged, 0 gathered) in 5.62189241 sec., 85.73625477830872 rows/sec., 7.95 KiB/sec. 2024.10.22 01:31:21.564033 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} MergedBlockOutputStream: filled checksums 202410_1_438_90 (state Temporary) 2024.10.22 01:31:21.564154 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:31:21.590213 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:31:21.599489 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:31:21.613543 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:31:21.670599 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:21.674270 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:21.724611 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:31:21.748945 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:31:21.859903 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:31:21.937987 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_896_896_0 (state Temporary) 2024.10.22 01:31:21.954146 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:31:21.961497 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:31:22.062602 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_896_896_0 to 202410_909_909_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:22.084607 [ 1212 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_898_456 to 202410_899_904_1 2024.10.22 01:31:22.096421 [ 1212 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.14 MiB using storage policy from min volume index 0 2024.10.22 01:31:22.101120 [ 1212 ] {} DiskLocal: Reserved 2.14 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:22.130703 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59282 2024.10.22 01:31:22.162862 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/dad/ebvcohxfzeigtqjaearyccukivdhj (LogSeriesLimiter: on interval from 2024-10-22 01:31:17 to 2024-10-22 01:31:22 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:31:22.164665 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_904_457} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_898_456 to 202410_899_904_1 into Compact with storage Full 2024.10.22 01:31:22.324875 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_438_90 to 202410_1_438_90 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:22.353397 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_438_90} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_433_89, 202410_438_438_0] -> 202410_1_438_90 2024.10.22 01:31:22.372257 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_904_457} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:22.381614 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:31:22.391458 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b7e4ec9f97cf8d84f8206ca91b4dff27) : [0, 495593]: Setting download as failed: File segment: [0, 495593], key: b7e4ec9f97cf8d84f8206ca91b4dff27, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:22.394240 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b7e4ec9f97cf8d84f8206ca91b4dff27): No space left in cache to reserve 495594 bytes, will continue without cache download 2024.10.22 01:31:22.417270 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_904_457} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_898_456, total 59462 rows starting from the beginning of the part 2024.10.22 01:31:22.428545 [ 1227 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:31:22.458477 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.97 GiB, free memory in arenas 11.44 MiB, will set to 1.52 GiB (RSS), difference: 5.86 MiB 2024.10.22 01:31:22.529571 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b7e4ec9f97cf8d84f8206ca91b4dff27, offset: 0, size: 0 2024.10.22 01:31:22.535371 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b7e4ec9f97cf8d84f8206ca91b4dff27 for removal 2024.10.22 01:31:22.547490 [ 1476 ] {} CacheMetadata: Key b7e4ec9f97cf8d84f8206ca91b4dff27 is removed from metadata 2024.10.22 01:31:22.457633 [ 1227 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:31:22.727983 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_904_457} MergeTreeSequentialSource: Reading 2 marks from part 202410_899_904_1, total 250 rows starting from the beginning of the part 2024.10.22 01:31:23.411031 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 447 2024.10.22 01:31:23.716544 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:23.720899 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:23.803312 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_429_429_0 (state Temporary) 2024.10.22 01:31:23.975845 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 9 file segments (while reserving 174433 bytes for 5db5c16829d02d73fc04bff4e56ba870:0) 2024.10.22 01:31:23.978355 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fc2376e62f7920e4e6127f2e9ad3f3c2, offset: 0, size: 53956 2024.10.22 01:31:23.982335 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fc2376e62f7920e4e6127f2e9ad3f3c2 for removal 2024.10.22 01:31:23.989115 [ 1476 ] {} CacheMetadata: Key fc2376e62f7920e4e6127f2e9ad3f3c2 is removed from metadata 2024.10.22 01:31:23.989483 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9e3a79414aece7d4015368a7c601393b, offset: 0, size: 164 2024.10.22 01:31:24.006652 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9e3a79414aece7d4015368a7c601393b for removal 2024.10.22 01:31:24.015442 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 064562079449b05e105a7ee3b8879684, offset: 0, size: 6175 2024.10.22 01:31:24.026738 [ 1476 ] {} CacheMetadata: Key 9e3a79414aece7d4015368a7c601393b is removed from metadata 2024.10.22 01:31:24.026986 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 064562079449b05e105a7ee3b8879684 for removal 2024.10.22 01:31:24.036353 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dc27b9ed28d4178a4ee4690b6804772b, offset: 0, size: 162 2024.10.22 01:31:24.046371 [ 1476 ] {} CacheMetadata: Key 064562079449b05e105a7ee3b8879684 is removed from metadata 2024.10.22 01:31:24.087024 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_429_429_0 to 202410_439_439_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:24.146419 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dc27b9ed28d4178a4ee4690b6804772b for removal 2024.10.22 01:31:24.150891 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: df0c973caac0d1e72c10f5c30bed716f, offset: 0, size: 3988 2024.10.22 01:31:24.173487 [ 1476 ] {} CacheMetadata: Key dc27b9ed28d4178a4ee4690b6804772b is removed from metadata 2024.10.22 01:31:24.255414 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key df0c973caac0d1e72c10f5c30bed716f for removal 2024.10.22 01:31:24.258063 [ 1476 ] {} CacheMetadata: Key df0c973caac0d1e72c10f5c30bed716f is removed from metadata 2024.10.22 01:31:24.258942 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cf5199b659c4d825a160c67ed26dcc27, offset: 0, size: 162 2024.10.22 01:31:24.261599 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cf5199b659c4d825a160c67ed26dcc27 for removal 2024.10.22 01:31:24.263709 [ 1476 ] {} CacheMetadata: Key cf5199b659c4d825a160c67ed26dcc27 is removed from metadata 2024.10.22 01:31:24.264844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2c1a5ff7972320211d401a8cd0543c01, offset: 0, size: 83184 2024.10.22 01:31:24.268169 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2c1a5ff7972320211d401a8cd0543c01 for removal 2024.10.22 01:31:24.272369 [ 1476 ] {} CacheMetadata: Key 2c1a5ff7972320211d401a8cd0543c01 is removed from metadata 2024.10.22 01:31:24.271158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bb61bd6282cb0de1b62c52a7016c6525, offset: 0, size: 200 2024.10.22 01:31:24.275414 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bb61bd6282cb0de1b62c52a7016c6525 for removal 2024.10.22 01:31:24.277750 [ 1476 ] {} CacheMetadata: Key bb61bd6282cb0de1b62c52a7016c6525 is removed from metadata 2024.10.22 01:31:24.278226 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 74687d618cb03475b4645a14983b5a58, offset: 0, size: 168 2024.10.22 01:31:24.284962 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 74687d618cb03475b4645a14983b5a58 for removal 2024.10.22 01:31:24.358813 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 447 2024.10.22 01:31:24.386907 [ 1476 ] {} CacheMetadata: Key 74687d618cb03475b4645a14983b5a58 is removed from metadata 2024.10.22 01:31:24.404165 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 8.23 MiB, will set to 1.53 GiB (RSS), difference: -11.15 MiB 2024.10.22 01:31:24.728925 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_508_102} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_508_102 to 202410_1_508_102 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:24.732307 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_508_102} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_502_101, 202410_503_508_1] -> 202410_1_508_102 2024.10.22 01:31:24.755706 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:31:25.219595 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 5 file segments (while reserving 196401 bytes for 5cb97437aa5253434d0f33fe43995d22:0) 2024.10.22 01:31:25.222432 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5db5c16829d02d73fc04bff4e56ba870, offset: 0, size: 174433 2024.10.22 01:31:25.226469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5db5c16829d02d73fc04bff4e56ba870 for removal 2024.10.22 01:31:25.233626 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1e3dc12ae90694ac871e57199db42d82, offset: 0, size: 170 2024.10.22 01:31:25.239396 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1e3dc12ae90694ac871e57199db42d82 for removal 2024.10.22 01:31:25.246773 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0fc757e3bdc4bf0dfeafcf9203b3d0a8, offset: 0, size: 246 2024.10.22 01:31:25.252890 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0fc757e3bdc4bf0dfeafcf9203b3d0a8 for removal 2024.10.22 01:31:25.256111 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8335d265f69849826d999ae93d7eb68f, offset: 0, size: 58250 2024.10.22 01:31:25.258620 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8335d265f69849826d999ae93d7eb68f for removal 2024.10.22 01:31:25.261006 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 211e63ab546c205c453eb8dc224dbcc6, offset: 0, size: 162 2024.10.22 01:31:25.267057 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 211e63ab546c205c453eb8dc224dbcc6 for removal 2024.10.22 01:31:25.332057 [ 1476 ] {} CacheMetadata: Key 5db5c16829d02d73fc04bff4e56ba870 is removed from metadata 2024.10.22 01:31:25.367915 [ 1476 ] {} CacheMetadata: Key 211e63ab546c205c453eb8dc224dbcc6 is removed from metadata 2024.10.22 01:31:25.395862 [ 1476 ] {} CacheMetadata: Key 8335d265f69849826d999ae93d7eb68f is removed from metadata 2024.10.22 01:31:25.417175 [ 1476 ] {} CacheMetadata: Key 0fc757e3bdc4bf0dfeafcf9203b3d0a8 is removed from metadata 2024.10.22 01:31:25.432807 [ 1476 ] {} CacheMetadata: Key 1e3dc12ae90694ac871e57199db42d82 is removed from metadata 2024.10.22 01:31:25.468978 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 8202 2024.10.22 01:31:25.476315 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.97 GiB, free memory in arenas 7.30 MiB, will set to 1.52 GiB (RSS), difference: 5.71 MiB 2024.10.22 01:31:25.670415 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:25.673684 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:25.726910 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_531_531_0 (state Temporary) 2024.10.22 01:31:25.768737 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_904_457} MergeTask::MergeProjectionsStage: Merge sorted 59712 rows, containing 13 columns (13 merged, 0 gathered) in 3.609240163 sec., 16544.202464589496 rows/sec., 2.63 MiB/sec. 2024.10.22 01:31:25.877140 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_531_531_0 to 202410_544_544_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:25.944474 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8202 2024.10.22 01:31:25.986898 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(55a2c6f32810c79be7ffaa2f1c2b8f9d) : [0, 392630]: Setting download as failed: File segment: [0, 392630], key: 55a2c6f32810c79be7ffaa2f1c2b8f9d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:25.991064 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(55a2c6f32810c79be7ffaa2f1c2b8f9d): No space left in cache to reserve 392631 bytes, will continue without cache download 2024.10.22 01:31:26.056143 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_904_457} MergedBlockOutputStream: filled checksums 202410_1_904_457 (state Temporary) 2024.10.22 01:31:26.100159 [ 1249 ] {} DatabaseCatalog: Execution took 295 ms. 2024.10.22 01:31:26.127877 [ 1232 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 01:31:26.154491 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 105 entries to flush up to offset 59387 2024.10.22 01:31:26.159747 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_904_457} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_904_457 to 202410_1_904_457 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:26.162455 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 55a2c6f32810c79be7ffaa2f1c2b8f9d, offset: 0, size: 0 2024.10.22 01:31:26.164726 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_904_457} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_898_456, 202410_899_904_1] -> 202410_1_904_457 2024.10.22 01:31:26.166837 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 55a2c6f32810c79be7ffaa2f1c2b8f9d for removal 2024.10.22 01:31:26.170475 [ 1476 ] {} CacheMetadata: Key 55a2c6f32810c79be7ffaa2f1c2b8f9d is removed from metadata 2024.10.22 01:31:26.183287 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:31:26.705211 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:26.709080 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:26.867462 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5ea14423fd6ccf433167199584fca545) : [0, 422089]: Setting download as failed: File segment: [0, 422089], key: 5ea14423fd6ccf433167199584fca545, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:26.868997 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5ea14423fd6ccf433167199584fca545): No space left in cache to reserve 422090 bytes, will continue without cache download 2024.10.22 01:31:26.892544 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5ea14423fd6ccf433167199584fca545, offset: 0, size: 0 2024.10.22 01:31:26.896751 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5ea14423fd6ccf433167199584fca545 for removal 2024.10.22 01:31:26.900627 [ 1476 ] {} CacheMetadata: Key 5ea14423fd6ccf433167199584fca545 is removed from metadata 2024.10.22 01:31:26.979605 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_897_897_0 (state Temporary) 2024.10.22 01:31:27.060968 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_insert_202410_897_897_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:31:27.070032 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 803930 2024.10.22 01:31:27.273964 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_897_897_0 to 202410_910_910_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:27.273628 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_800_408, 202410_801_806_1] 2024.10.22 01:31:27.275963 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_800_408, 202410_801_806_1] 2024.10.22 01:31:27.308359 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_800_408, 202410_801_806_1] 2024.10.22 01:31:27.345191 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59387 2024.10.22 01:31:27.415766 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_904_457 to 202410_905_905_0 2024.10.22 01:31:27.431985 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.14 MiB using storage policy from min volume index 0 2024.10.22 01:31:27.437789 [ 1218 ] {} DiskLocal: Reserved 2.14 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 01:31:27.480260 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_905_458} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_904_457 to 202410_905_905_0 into Compact with storage Full 2024.10.22 01:31:27.652733 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 231944 bytes for ec1abe583cdf2dd0d027765754e6f271:0) 2024.10.22 01:31:27.656360 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5cb97437aa5253434d0f33fe43995d22, offset: 0, size: 196401 2024.10.22 01:31:27.658737 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5cb97437aa5253434d0f33fe43995d22 for removal 2024.10.22 01:31:27.661977 [ 1476 ] {} CacheMetadata: Key 5cb97437aa5253434d0f33fe43995d22 is removed from metadata 2024.10.22 01:31:27.662675 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a50ffca3c9f559d5d7e9fbc56c5e9e8f, offset: 0, size: 170 2024.10.22 01:31:27.664659 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a50ffca3c9f559d5d7e9fbc56c5e9e8f for removal 2024.10.22 01:31:27.668320 [ 1476 ] {} CacheMetadata: Key a50ffca3c9f559d5d7e9fbc56c5e9e8f is removed from metadata 2024.10.22 01:31:27.668599 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_905_458} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:27.704989 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_905_458} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_904_457, total 59712 rows starting from the beginning of the part 2024.10.22 01:31:27.784003 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:27.789821 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:28.030933 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_905_458} MergeTreeSequentialSource: Reading 2 marks from part 202410_905_905_0, total 64 rows starting from the beginning of the part 2024.10.22 01:31:28.134315 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_509_509_0 (state Temporary) 2024.10.22 01:31:28.213756 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_509_509_0 to 202410_516_516_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:28.251606 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 803930 2024.10.22 01:31:28.604496 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6e319ab014952ce986ad964c7bbdf582) : [0, 2206017]: Setting download as failed: File segment: [0, 2206017], key: 6e319ab014952ce986ad964c7bbdf582, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:28.608741 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6e319ab014952ce986ad964c7bbdf582): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:31:28.663066 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6e319ab014952ce986ad964c7bbdf582, offset: 0, size: 0 2024.10.22 01:31:28.664746 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6e319ab014952ce986ad964c7bbdf582 for removal 2024.10.22 01:31:28.666647 [ 1476 ] {} CacheMetadata: Key 6e319ab014952ce986ad964c7bbdf582 is removed from metadata 2024.10.22 01:31:28.765140 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 114 entries to flush up to offset 61189 2024.10.22 01:31:28.875752 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:28.879513 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:28.928636 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_521_521_0 (state Temporary) 2024.10.22 01:31:28.967782 [ 1188 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 01:31:28.987866 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_521_521_0 to 202410_527_527_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:29.028916 [ 1152 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_522_3 to 202410_527_527_0 2024.10.22 01:31:29.053486 [ 1152 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:29.056288 [ 1152 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:29.056824 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 61189 2024.10.22 01:31:29.099069 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_522_3 to 202410_527_527_0 into Compact with storage Full 2024.10.22 01:31:29.247040 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:29.274010 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_522_3, total 1651 rows starting from the beginning of the part 2024.10.22 01:31:29.482571 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_523_523_0, total 113 rows starting from the beginning of the part 2024.10.22 01:31:29.504775 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 5.11 MiB, will set to 1.52 GiB (RSS), difference: -7.51 MiB 2024.10.22 01:31:29.565444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 123640 bytes for 04e8c88af58c6bd516c5cd6f023c590c:0) 2024.10.22 01:31:29.568445 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 105910fdb931c7310368ef830e56a1d5, offset: 0, size: 202 2024.10.22 01:31:29.571035 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 105910fdb931c7310368ef830e56a1d5 for removal 2024.10.22 01:31:29.573695 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ec1abe583cdf2dd0d027765754e6f271, offset: 0, size: 231944 2024.10.22 01:31:29.574001 [ 1476 ] {} CacheMetadata: Key 105910fdb931c7310368ef830e56a1d5 is removed from metadata 2024.10.22 01:31:29.576712 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_524_524_0, total 71 rows starting from the beginning of the part 2024.10.22 01:31:29.576518 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ec1abe583cdf2dd0d027765754e6f271 for removal 2024.10.22 01:31:29.582716 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 97946b22d21d6d34fc34fc703e761eb4, offset: 0, size: 202 2024.10.22 01:31:29.585670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 97946b22d21d6d34fc34fc703e761eb4 for removal 2024.10.22 01:31:29.591894 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fc81c09a83cc166aca66634409b64a77, offset: 0, size: 199 2024.10.22 01:31:29.591865 [ 1476 ] {} CacheMetadata: Key 97946b22d21d6d34fc34fc703e761eb4 is removed from metadata 2024.10.22 01:31:29.699068 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fc81c09a83cc166aca66634409b64a77 for removal 2024.10.22 01:31:29.732857 [ 1476 ] {} CacheMetadata: Key fc81c09a83cc166aca66634409b64a77 is removed from metadata 2024.10.22 01:31:29.757722 [ 1476 ] {} CacheMetadata: Key ec1abe583cdf2dd0d027765754e6f271 is removed from metadata 2024.10.22 01:31:29.763540 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_525_525_0, total 42 rows starting from the beginning of the part 2024.10.22 01:31:29.776748 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_105_109_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:29.860932 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_526_526_0, total 73 rows starting from the beginning of the part 2024.10.22 01:31:30.039834 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_527_527_0, total 114 rows starting from the beginning of the part 2024.10.22 01:31:30.211649 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(44715bb5e051bfb6b9605d1c58ee8fab) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 44715bb5e051bfb6b9605d1c58ee8fab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:31:30.217132 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 44715bb5e051bfb6b9605d1c58ee8fab, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:31:30.222753 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:31:30.376229 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:31:30.389723 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:31:30.822963 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_105_109_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_105_105_0, total 40 rows starting from the beginning of the part 2024.10.22 01:31:31.267805 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 5 file segments (while reserving 187848 bytes for 2f1cf81c600b5c0edfd35984c940cebc:0) 2024.10.22 01:31:31.270085 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fdca2c3d4a7561a6b9c7af265ceec427, offset: 0, size: 105997 2024.10.22 01:31:31.275194 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fdca2c3d4a7561a6b9c7af265ceec427 for removal 2024.10.22 01:31:31.277784 [ 1476 ] {} CacheMetadata: Key fdca2c3d4a7561a6b9c7af265ceec427 is removed from metadata 2024.10.22 01:31:31.278177 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c7ada87ecaa9c1fa012b77de4409f687, offset: 0, size: 164 2024.10.22 01:31:31.281754 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c7ada87ecaa9c1fa012b77de4409f687 for removal 2024.10.22 01:31:31.301367 [ 1476 ] {} CacheMetadata: Key c7ada87ecaa9c1fa012b77de4409f687 is removed from metadata 2024.10.22 01:31:31.368090 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 59479 2024.10.22 01:31:31.386874 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 04e8c88af58c6bd516c5cd6f023c590c, offset: 0, size: 123640 2024.10.22 01:31:31.390744 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 04e8c88af58c6bd516c5cd6f023c590c for removal 2024.10.22 01:31:31.408172 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2d8fb7b1bc4dfa63ba15a72db3cfa8c4, offset: 0, size: 164 2024.10.22 01:31:31.418248 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2d8fb7b1bc4dfa63ba15a72db3cfa8c4 for removal 2024.10.22 01:31:31.423451 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1974e1783f4b99cd902c533f719f8086, offset: 0, size: 269 2024.10.22 01:31:31.428839 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1974e1783f4b99cd902c533f719f8086 for removal 2024.10.22 01:31:31.469181 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} MergeTask::MergeProjectionsStage: Merge sorted 2064 rows, containing 13 columns (13 merged, 0 gathered) in 2.381562281 sec., 866.6579986030607 rows/sec., 383.61 KiB/sec. 2024.10.22 01:31:31.507093 [ 1476 ] {} CacheMetadata: Key 04e8c88af58c6bd516c5cd6f023c590c is removed from metadata 2024.10.22 01:31:31.549417 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_3_3_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:31:31.579848 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_905_458} MergeTask::MergeProjectionsStage: Merge sorted 59776 rows, containing 13 columns (13 merged, 0 gathered) in 4.107832714 sec., 14551.712341224615 rows/sec., 2.31 MiB/sec. 2024.10.22 01:31:31.616579 [ 1476 ] {} CacheMetadata: Key 1974e1783f4b99cd902c533f719f8086 is removed from metadata 2024.10.22 01:31:31.626941 [ 1476 ] {} CacheMetadata: Key 2d8fb7b1bc4dfa63ba15a72db3cfa8c4 is removed from metadata 2024.10.22 01:31:31.700943 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_905_458} MergedBlockOutputStream: filled checksums 202410_1_905_458 (state Temporary) 2024.10.22 01:31:31.729426 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} MergedBlockOutputStream: filled checksums 202410_507_527_4 (state Temporary) 2024.10.22 01:31:31.761494 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:31.764127 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:31.861788 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 448 2024.10.22 01:31:31.958147 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_898_898_0 (state Temporary) 2024.10.22 01:31:32.012236 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_527_4 to 202410_507_527_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:32.016375 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_527_4} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_522_3, 202410_527_527_0] -> 202410_507_527_4 2024.10.22 01:31:32.089041 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.30 MiB. 2024.10.22 01:31:32.236830 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:32.261336 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:32.332485 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_430_430_0 (state Temporary) 2024.10.22 01:31:32.337716 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_898_898_0 to 202410_911_911_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:32.357498 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_905_458} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_905_458 to 202410_1_905_458 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:32.360739 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_905_458} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_904_457, 202410_905_905_0] -> 202410_1_905_458 2024.10.22 01:31:32.384304 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_905_458 to 202410_906_906_0 2024.10.22 01:31:32.390832 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:31:32.396463 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.14 MiB using storage policy from min volume index 0 2024.10.22 01:31:32.409024 [ 1195 ] {} DiskLocal: Reserved 2.14 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:32.467625 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59479 2024.10.22 01:31:32.561281 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_906_459} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_905_458 to 202410_906_906_0 into Compact with storage Full 2024.10.22 01:31:32.591969 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fca37eb0a50abf36ee18aac9eef4581f) : [0, 338787]: Setting download as failed: File segment: [0, 338787], key: fca37eb0a50abf36ee18aac9eef4581f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:32.595421 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fca37eb0a50abf36ee18aac9eef4581f): No space left in cache to reserve 338788 bytes, will continue without cache download 2024.10.22 01:31:32.639069 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fca37eb0a50abf36ee18aac9eef4581f, offset: 0, size: 0 2024.10.22 01:31:32.647369 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fca37eb0a50abf36ee18aac9eef4581f for removal 2024.10.22 01:31:32.650995 [ 1476 ] {} CacheMetadata: Key fca37eb0a50abf36ee18aac9eef4581f is removed from metadata 2024.10.22 01:31:32.661380 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_430_430_0 to 202410_440_440_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:32.681022 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.97 GiB, free memory in arenas 2.09 MiB, will set to 1.52 GiB (RSS), difference: 7.65 MiB 2024.10.22 01:31:32.707667 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_906_459} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:32.756295 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_906_459} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_905_458, total 59776 rows starting from the beginning of the part 2024.10.22 01:31:32.760975 [ 1724 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qvy/hlkywmoyhijywhrfirejnycgvzqjv, size 1186335 2024.10.22 01:31:32.776278 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 448 2024.10.22 01:31:32.955022 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_906_459} MergeTreeSequentialSource: Reading 2 marks from part 202410_906_906_0, total 95 rows starting from the beginning of the part 2024.10.22 01:31:32.960850 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:31:32.976745 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:31:32.983385 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:31:33.008641 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:31:33.123901 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:31:33.156261 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:31:33.169090 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:31:33.197040 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:31:33.251885 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/iad/iqrgjmbhylitrlnczxayrktdmgnrx (LogSeriesLimiter: on interval from 2024-10-22 01:31:22 to 2024-10-22 01:31:33 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:31:33.292542 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_105_109_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_106_106_0, total 42 rows starting from the beginning of the part 2024.10.22 01:31:33.376934 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 193205 bytes for acb7bb0224ab7b8965bdbe051d360661:0) 2024.10.22 01:31:33.380260 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2f1cf81c600b5c0edfd35984c940cebc, offset: 0, size: 187848 2024.10.22 01:31:33.385357 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2f1cf81c600b5c0edfd35984c940cebc for removal 2024.10.22 01:31:33.388814 [ 1476 ] {} CacheMetadata: Key 2f1cf81c600b5c0edfd35984c940cebc is removed from metadata 2024.10.22 01:31:33.389125 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c2f0c05b4a204bc275a36ada0f7eb2b9, offset: 0, size: 235 2024.10.22 01:31:33.406211 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c2f0c05b4a204bc275a36ada0f7eb2b9 for removal 2024.10.22 01:31:33.426222 [ 1476 ] {} CacheMetadata: Key c2f0c05b4a204bc275a36ada0f7eb2b9 is removed from metadata 2024.10.22 01:31:33.551199 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 8221 2024.10.22 01:31:33.770513 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:33.772963 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:33.828944 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_532_532_0 (state Temporary) 2024.10.22 01:31:33.889024 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_532_532_0 to 202410_545_545_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:34.057096 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8221 2024.10.22 01:31:34.381838 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_806_409, 202410_807_807_0] 2024.10.22 01:31:34.388056 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_806_409, 202410_807_807_0] 2024.10.22 01:31:34.419098 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_806_409, 202410_807_807_0] 2024.10.22 01:31:34.487445 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cc3284fc53bafa56369ed4990b3bcb2c) : [0, 873704]: Setting download as failed: File segment: [0, 873704], key: cc3284fc53bafa56369ed4990b3bcb2c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:34.491390 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cc3284fc53bafa56369ed4990b3bcb2c): No space left in cache to reserve 873705 bytes, will continue without cache download 2024.10.22 01:31:34.555861 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 2.25 MiB, will set to 1.52 GiB (RSS), difference: -13.32 MiB 2024.10.22 01:31:34.636902 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:34.650328 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:34.668085 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cc3284fc53bafa56369ed4990b3bcb2c, offset: 0, size: 0 2024.10.22 01:31:34.669668 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cc3284fc53bafa56369ed4990b3bcb2c for removal 2024.10.22 01:31:34.672579 [ 1476 ] {} CacheMetadata: Key cc3284fc53bafa56369ed4990b3bcb2c is removed from metadata 2024.10.22 01:31:35.147181 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(9872bbfcdab5b61b9be1b446b980ecf5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9872bbfcdab5b61b9be1b446b980ecf5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:31:35.154507 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 9872bbfcdab5b61b9be1b446b980ecf5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:31:35.153394 [ 1172 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 01:31:35.163434 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:31:35.260273 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1163 entries to flush up to offset 805093 2024.10.22 01:31:35.878768 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b5425482dcad9c187ab283dbba4054fe) : [0, 2085004]: Setting download as failed: File segment: [0, 2085004], key: b5425482dcad9c187ab283dbba4054fe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:35.880162 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b5425482dcad9c187ab283dbba4054fe): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:31:36.065597 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b5425482dcad9c187ab283dbba4054fe, offset: 0, size: 0 2024.10.22 01:31:36.068085 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b5425482dcad9c187ab283dbba4054fe for removal 2024.10.22 01:31:36.072036 [ 1476 ] {} CacheMetadata: Key b5425482dcad9c187ab283dbba4054fe is removed from metadata 2024.10.22 01:31:36.083632 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:36.087544 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:36.095664 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_105_109_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_107_107_0, total 40 rows starting from the beginning of the part 2024.10.22 01:31:36.372597 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_510_510_0 (state Temporary) 2024.10.22 01:31:36.457677 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_510_510_0 to 202410_517_517_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:36.474334 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 95 entries to flush up to offset 59574 2024.10.22 01:31:36.509623 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 805093 2024.10.22 01:31:36.574913 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 111 entries to flush up to offset 61300 2024.10.22 01:31:36.649577 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_906_459} MergeTask::MergeProjectionsStage: Merge sorted 59871 rows, containing 13 columns (13 merged, 0 gathered) in 4.174400837 sec., 14342.417591844689 rows/sec., 2.28 MiB/sec. 2024.10.22 01:31:36.735375 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_906_459} MergedBlockOutputStream: filled checksums 202410_1_906_459 (state Temporary) 2024.10.22 01:31:36.843896 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 1.97 GiB, free memory in arenas 2.36 MiB, will set to 1.52 GiB (RSS), difference: 9.68 MiB 2024.10.22 01:31:36.849658 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_906_459} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_906_459 to 202410_1_906_459 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:36.854291 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_906_459} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_905_458, 202410_906_906_0] -> 202410_1_906_459 2024.10.22 01:31:36.929494 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:36.937711 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:36.881838 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:31:36.988237 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:36.997186 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:37.094547 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_522_522_0 (state Temporary) 2024.10.22 01:31:37.168274 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_522_522_0 to 202410_528_528_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:37.191464 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_899_899_0 (state Temporary) 2024.10.22 01:31:37.256385 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_899_899_0 to 202410_912_912_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:37.309674 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_906_459 to 202410_907_907_0 2024.10.22 01:31:37.322504 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 201693 bytes for e9ee3d9159d19895ebaf1d0dc56e4640:0) 2024.10.22 01:31:37.323848 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 61300 2024.10.22 01:31:37.323775 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.14 MiB using storage policy from min volume index 0 2024.10.22 01:31:37.325048 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: acb7bb0224ab7b8965bdbe051d360661, offset: 0, size: 193205 2024.10.22 01:31:37.333310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key acb7bb0224ab7b8965bdbe051d360661 for removal 2024.10.22 01:31:37.338906 [ 1476 ] {} CacheMetadata: Key acb7bb0224ab7b8965bdbe051d360661 is removed from metadata 2024.10.22 01:31:37.339047 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 667a4b18e7cfa18dc9853a93b64a32fa, offset: 0, size: 170 2024.10.22 01:31:37.346467 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 667a4b18e7cfa18dc9853a93b64a32fa for removal 2024.10.22 01:31:37.349815 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f374fe398c3f26780be01462086708dd, offset: 0, size: 243 2024.10.22 01:31:37.353599 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f374fe398c3f26780be01462086708dd for removal 2024.10.22 01:31:37.361781 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59574 2024.10.22 01:31:37.476009 [ 1476 ] {} CacheMetadata: Key f374fe398c3f26780be01462086708dd is removed from metadata 2024.10.22 01:31:37.556354 [ 1170 ] {} DiskLocal: Reserved 2.14 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:37.561000 [ 1476 ] {} CacheMetadata: Key 667a4b18e7cfa18dc9853a93b64a32fa is removed from metadata 2024.10.22 01:31:37.562550 [ 1170 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 284 ms. 2024.10.22 01:31:37.562620 [ 1183 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:31:37.570415 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_907_460} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_906_459 to 202410_907_907_0 into Compact with storage Full 2024.10.22 01:31:37.587025 [ 1183 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:31:37.651447 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_907_460} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:37.662030 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_907_460} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_906_459, total 59871 rows starting from the beginning of the part 2024.10.22 01:31:37.760927 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_907_460} MergeTreeSequentialSource: Reading 2 marks from part 202410_907_907_0, total 61 rows starting from the beginning of the part 2024.10.22 01:31:38.013567 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_807_410, 202410_808_808_0] 2024.10.22 01:31:38.018924 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_807_410, 202410_808_808_0] 2024.10.22 01:31:38.033797 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_807_410, 202410_808_808_0] 2024.10.22 01:31:38.063373 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_107_107_0 (state Temporary) 2024.10.22 01:31:38.184737 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(519464e2c6c2dfb85b52cc6c5546e086) : [0, 2180931]: Setting download as failed: File segment: [0, 2180931], key: 519464e2c6c2dfb85b52cc6c5546e086, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:38.186527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(519464e2c6c2dfb85b52cc6c5546e086): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:31:38.219553 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_481_96, 202410_482_482_0, 202410_483_483_0, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0] 2024.10.22 01:31:38.224144 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_481_96, 202410_482_482_0, 202410_483_483_0, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0] 2024.10.22 01:31:38.257359 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_481_96, 202410_482_482_0, 202410_483_483_0, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0] 2024.10.22 01:31:38.277052 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 519464e2c6c2dfb85b52cc6c5546e086, offset: 0, size: 0 2024.10.22 01:31:38.278739 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 519464e2c6c2dfb85b52cc6c5546e086 for removal 2024.10.22 01:31:38.285327 [ 1476 ] {} CacheMetadata: Key 519464e2c6c2dfb85b52cc6c5546e086 is removed from metadata 2024.10.22 01:31:38.402180 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_105_109_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_108_108_0, total 39 rows starting from the beginning of the part 2024.10.22 01:31:39.119535 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 209710 bytes for e5c864f7a4c28b53d55eb227d58e384e:0) 2024.10.22 01:31:39.127168 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e9ee3d9159d19895ebaf1d0dc56e4640, offset: 0, size: 201693 2024.10.22 01:31:39.135146 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e9ee3d9159d19895ebaf1d0dc56e4640 for removal 2024.10.22 01:31:39.144372 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e99166b7b48eaf72dd05f5c66422bef3, offset: 0, size: 171 2024.10.22 01:31:39.144834 [ 1476 ] {} CacheMetadata: Key e9ee3d9159d19895ebaf1d0dc56e4640 is removed from metadata 2024.10.22 01:31:39.151025 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e99166b7b48eaf72dd05f5c66422bef3 for removal 2024.10.22 01:31:39.155978 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6697e0096350791e0917c18b2df19b95, offset: 0, size: 269 2024.10.22 01:31:39.162627 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6697e0096350791e0917c18b2df19b95 for removal 2024.10.22 01:31:39.166424 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9b5762d2639539e19e48fcebeed106d5, offset: 0, size: 238 2024.10.22 01:31:39.171799 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9b5762d2639539e19e48fcebeed106d5 for removal 2024.10.22 01:31:39.172210 [ 1476 ] {} CacheMetadata: Key 6697e0096350791e0917c18b2df19b95 is removed from metadata 2024.10.22 01:31:39.185066 [ 1476 ] {} CacheMetadata: Key 9b5762d2639539e19e48fcebeed106d5 is removed from metadata 2024.10.22 01:31:39.208541 [ 1476 ] {} CacheMetadata: Key e99166b7b48eaf72dd05f5c66422bef3 is removed from metadata 2024.10.22 01:31:39.717142 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 3.69 MiB, will set to 1.52 GiB (RSS), difference: -9.91 MiB 2024.10.22 01:31:39.889453 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_105_109_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_109_109_0, total 43 rows starting from the beginning of the part 2024.10.22 01:31:40.377379 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:31:40.381606 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:31:40.400012 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 175184 bytes for a748aa5f73426ebedfd1e81618fc0c66:0) 2024.10.22 01:31:40.437598 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f98c1ffe449c9de9374132386f19516f, offset: 0, size: 169 2024.10.22 01:31:40.524022 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_107_107_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:31:40.547833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f98c1ffe449c9de9374132386f19516f for removal 2024.10.22 01:31:40.551264 [ 1476 ] {} CacheMetadata: Key f98c1ffe449c9de9374132386f19516f is removed from metadata 2024.10.22 01:31:40.551509 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e5c864f7a4c28b53d55eb227d58e384e, offset: 0, size: 209710 2024.10.22 01:31:40.557628 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e5c864f7a4c28b53d55eb227d58e384e for removal 2024.10.22 01:31:40.562064 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 18e8d6f2458d84b4b7e3b9c0d4997f32, offset: 0, size: 269 2024.10.22 01:31:40.562166 [ 1476 ] {} CacheMetadata: Key e5c864f7a4c28b53d55eb227d58e384e is removed from metadata 2024.10.22 01:31:40.566751 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 18e8d6f2458d84b4b7e3b9c0d4997f32 for removal 2024.10.22 01:31:40.572765 [ 1476 ] {} CacheMetadata: Key 18e8d6f2458d84b4b7e3b9c0d4997f32 is removed from metadata 2024.10.22 01:31:41.381412 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 59654 2024.10.22 01:31:41.472895 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_907_460} MergeTask::MergeProjectionsStage: Merge sorted 59932 rows, containing 13 columns (13 merged, 0 gathered) in 3.907930708 sec., 15335.993516290362 rows/sec., 2.44 MiB/sec. 2024.10.22 01:31:41.563911 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 8228 2024.10.22 01:31:41.660630 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_907_460} MergedBlockOutputStream: filled checksums 202410_1_907_460 (state Temporary) 2024.10.22 01:31:41.711543 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:41.715602 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:41.766588 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_533_533_0 (state Temporary) 2024.10.22 01:31:41.847103 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_533_533_0 to 202410_546_546_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:41.870319 [ 1235 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_541_108 to 202410_546_546_0 2024.10.22 01:31:41.876338 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 170663 bytes for 86b8b0523edc71126169c0d4d7d242c4:0) 2024.10.22 01:31:41.877659 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a748aa5f73426ebedfd1e81618fc0c66, offset: 0, size: 175184 2024.10.22 01:31:41.880061 [ 1235 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:41.883696 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a748aa5f73426ebedfd1e81618fc0c66 for removal 2024.10.22 01:31:41.884271 [ 1235 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:41.886611 [ 1476 ] {} CacheMetadata: Key a748aa5f73426ebedfd1e81618fc0c66 is removed from metadata 2024.10.22 01:31:41.888035 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e7c761bb7e2618a4582f52db380a3ff5, offset: 0, size: 166 2024.10.22 01:31:41.903319 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e7c761bb7e2618a4582f52db380a3ff5 for removal 2024.10.22 01:31:41.935938 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_541_108 to 202410_546_546_0 into Compact with storage Full 2024.10.22 01:31:41.990464 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8228 2024.10.22 01:31:42.034518 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_907_460} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_907_460 to 202410_1_907_460 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:42.041331 [ 1476 ] {} CacheMetadata: Key e7c761bb7e2618a4582f52db380a3ff5 is removed from metadata 2024.10.22 01:31:42.045044 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_907_460} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_906_459, 202410_907_907_0] -> 202410_1_907_460 2024.10.22 01:31:42.082329 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:42.088710 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:42.094623 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:42.108956 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_541_108, total 8266 rows starting from the beginning of the part 2024.10.22 01:31:42.155249 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:31:42.195199 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_542_542_0, total 14 rows starting from the beginning of the part 2024.10.22 01:31:42.268750 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_543_543_0, total 13 rows starting from the beginning of the part 2024.10.22 01:31:42.274373 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_900_900_0 (state Temporary) 2024.10.22 01:31:42.327433 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_900_900_0 to 202410_913_913_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:42.329551 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_544_544_0, total 15 rows starting from the beginning of the part 2024.10.22 01:31:42.395646 [ 1207 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_907_460 to 202410_908_908_0 2024.10.22 01:31:42.404994 [ 1207 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.15 MiB using storage policy from min volume index 0 2024.10.22 01:31:42.409368 [ 1207 ] {} DiskLocal: Reserved 2.15 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:42.436046 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_545_545_0, total 19 rows starting from the beginning of the part 2024.10.22 01:31:42.436955 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_908_461} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_907_460 to 202410_908_908_0 into Compact with storage Full 2024.10.22 01:31:42.488337 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_908_461} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:42.488801 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59654 2024.10.22 01:31:42.516343 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_908_461} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_907_460, total 59932 rows starting from the beginning of the part 2024.10.22 01:31:42.581826 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.97 GiB, free memory in arenas 3.16 MiB, will set to 1.52 GiB (RSS), difference: 6.50 MiB 2024.10.22 01:31:42.606992 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_546_546_0, total 7 rows starting from the beginning of the part 2024.10.22 01:31:42.710828 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_908_461} MergeTreeSequentialSource: Reading 2 marks from part 202410_908_908_0, total 99 rows starting from the beginning of the part 2024.10.22 01:31:42.860174 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_808_411, 202410_809_809_0] 2024.10.22 01:31:42.861489 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_808_411, 202410_809_809_0] 2024.10.22 01:31:42.870846 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_808_411, 202410_809_809_0] 2024.10.22 01:31:42.871942 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a9813668789990d86d61c2de21eab708) : [0, 267817]: Setting download as failed: File segment: [0, 267817], key: a9813668789990d86d61c2de21eab708, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:42.874584 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a9813668789990d86d61c2de21eab708): No space left in cache to reserve 267818 bytes, will continue without cache download 2024.10.22 01:31:42.951906 [ 1698 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/dad/ebvcohxfzeigtqjaearyccukivdhj, size 2104455 2024.10.22 01:31:43.026334 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a9813668789990d86d61c2de21eab708, offset: 0, size: 0 2024.10.22 01:31:43.027968 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a9813668789990d86d61c2de21eab708 for removal 2024.10.22 01:31:43.029733 [ 1476 ] {} CacheMetadata: Key a9813668789990d86d61c2de21eab708 is removed from metadata 2024.10.22 01:31:43.110191 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:31:43.120238 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:31:43.259459 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:31:43.272676 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:31:43.294295 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:31:43.313368 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:31:43.349372 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:31:43.368523 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:31:43.482388 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:31:43.517057 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1532 entries to flush up to offset 806625 2024.10.22 01:31:43.551308 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tvu/huefavmxcoccggbzkqqjsechxlgfz (LogSeriesLimiter: on interval from 2024-10-22 01:31:33 to 2024-10-22 01:31:43 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:31:43.759577 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 128605 bytes for b6a7aaee93aa9cf614a69527d4cfa911:0) 2024.10.22 01:31:43.763939 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 86b8b0523edc71126169c0d4d7d242c4, offset: 0, size: 170663 2024.10.22 01:31:43.767616 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 86b8b0523edc71126169c0d4d7d242c4 for removal 2024.10.22 01:31:43.772741 [ 1476 ] {} CacheMetadata: Key 86b8b0523edc71126169c0d4d7d242c4 is removed from metadata 2024.10.22 01:31:43.775081 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 60808ef1d0840c1e69c3564bf48fa54b, offset: 0, size: 165 2024.10.22 01:31:43.784655 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_546_109/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:31:43.791893 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 60808ef1d0840c1e69c3564bf48fa54b for removal 2024.10.22 01:31:43.797541 [ 1476 ] {} CacheMetadata: Key 60808ef1d0840c1e69c3564bf48fa54b is removed from metadata 2024.10.22 01:31:43.949640 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} MergeTask::MergeProjectionsStage: Merge sorted 8334 rows, containing 12 columns (12 merged, 0 gathered) in 2.024737449 sec., 4116.089226341958 rows/sec., 654.25 KiB/sec. 2024.10.22 01:31:43.975016 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} MergedBlockOutputStream: filled checksums 202410_1_546_109 (state Temporary) 2024.10.22 01:31:44.423510 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_546_109 to 202410_1_546_109 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:44.428910 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_546_109} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_541_108, 202410_546_546_0] -> 202410_1_546_109 2024.10.22 01:31:44.429713 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:44.438287 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:44.496653 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.56 MiB. 2024.10.22 01:31:44.608787 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 3.14 MiB, will set to 1.53 GiB (RSS), difference: -17.27 MiB 2024.10.22 01:31:44.667117 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_511_511_0 (state Temporary) 2024.10.22 01:31:44.740985 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_511_511_0 to 202410_518_518_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:44.778559 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 806625 2024.10.22 01:31:44.775671 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_508_102 to 202410_509_514_1 2024.10.22 01:31:44.810024 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:44.813038 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:44.838771 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_514_103} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_508_102 to 202410_509_514_1 into Wide with storage Full 2024.10.22 01:31:44.852311 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 135 entries to flush up to offset 61435 2024.10.22 01:31:44.910347 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_514_103} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:44.923863 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_514_103} MergeTreeSequentialSource: Reading 100 marks from part 202410_1_508_102, total 804043 rows starting from the beginning of the part 2024.10.22 01:31:44.970650 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_514_103} MergeTreeSequentialSource: Reading 3 marks from part 202410_509_514_1, total 10397 rows starting from the beginning of the part 2024.10.22 01:31:45.044292 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2abef6a41d0d9f937a1334c3b9ac1a69) : [0, 709525]: Setting download as failed: File segment: [0, 709525], key: 2abef6a41d0d9f937a1334c3b9ac1a69, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:45.048264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2abef6a41d0d9f937a1334c3b9ac1a69): No space left in cache to reserve 709526 bytes, will continue without cache download 2024.10.22 01:31:45.118804 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:45.123502 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:45.199051 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2abef6a41d0d9f937a1334c3b9ac1a69, offset: 0, size: 0 2024.10.22 01:31:45.202524 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2abef6a41d0d9f937a1334c3b9ac1a69 for removal 2024.10.22 01:31:45.209335 [ 1476 ] {} CacheMetadata: Key 2abef6a41d0d9f937a1334c3b9ac1a69 is removed from metadata 2024.10.22 01:31:45.216289 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_523_523_0 (state Temporary) 2024.10.22 01:31:45.278166 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_523_523_0 to 202410_529_529_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:45.358545 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 61435 2024.10.22 01:31:45.924816 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 150297 bytes for 81b6b6d50ef5120f66caefadfe9bc98f:0) 2024.10.22 01:31:45.928928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b6a7aaee93aa9cf614a69527d4cfa911, offset: 0, size: 128605 2024.10.22 01:31:45.937685 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b6a7aaee93aa9cf614a69527d4cfa911 for removal 2024.10.22 01:31:45.946610 [ 1476 ] {} CacheMetadata: Key b6a7aaee93aa9cf614a69527d4cfa911 is removed from metadata 2024.10.22 01:31:45.948866 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 49cebdbf68f242d47ed98fb305344659, offset: 0, size: 164 2024.10.22 01:31:45.955645 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 49cebdbf68f242d47ed98fb305344659 for removal 2024.10.22 01:31:45.959653 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: aec9ec70156f30d1ddbe3c1a490a825e, offset: 0, size: 200 2024.10.22 01:31:45.963526 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key aec9ec70156f30d1ddbe3c1a490a825e for removal 2024.10.22 01:31:45.967869 [ 1476 ] {} CacheMetadata: Key aec9ec70156f30d1ddbe3c1a490a825e is removed from metadata 2024.10.22 01:31:45.971815 [ 1476 ] {} CacheMetadata: Key 49cebdbf68f242d47ed98fb305344659 is removed from metadata 2024.10.22 01:31:46.234619 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_107_107_0 to 202410_110_110_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:46.456198 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 4.63 MiB, will set to 1.53 GiB (RSS), difference: -1.60 MiB 2024.10.22 01:31:46.511984 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 112 entries to flush up to offset 59766 2024.10.22 01:31:46.512881 [ 1239 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:31:46.541799 [ 1267 ] {} DatabaseCatalog: Execution took 243 ms. 2024.10.22 01:31:46.729378 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(094c5b448040b68d5ed10e2821a5a8f5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 094c5b448040b68d5ed10e2821a5a8f5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, kind: Regular, unbound: 0 2024.10.22 01:31:46.735020 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 094c5b448040b68d5ed10e2821a5a8f5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, kind: Regular, unbound: 0 2024.10.22 01:31:46.739294 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:31:46.775391 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_908_461} MergeTask::MergeProjectionsStage: Merge sorted 60031 rows, containing 13 columns (13 merged, 0 gathered) in 4.348698302 sec., 13804.36071465139 rows/sec., 2.20 MiB/sec. 2024.10.22 01:31:46.911899 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_908_461} MergedBlockOutputStream: filled checksums 202410_1_908_461 (state Temporary) 2024.10.22 01:31:46.980922 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_908_461} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_908_461 to 202410_1_908_461 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:46.984342 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_908_461} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_907_460, 202410_908_908_0] -> 202410_1_908_461 2024.10.22 01:31:47.033758 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:31:47.091891 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_454_91, 202410_455_455_0, 202410_456_456_0, 202410_457_457_0, 202410_458_458_0, 202410_459_459_0] 2024.10.22 01:31:47.099985 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_454_91, 202410_455_455_0, 202410_456_456_0, 202410_457_457_0, 202410_458_458_0, 202410_459_459_0] 2024.10.22 01:31:47.234004 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_454_91, 202410_455_455_0, 202410_456_456_0, 202410_457_457_0, 202410_458_458_0, 202410_459_459_0] 2024.10.22 01:31:47.544728 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:47.551074 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:47.781196 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 449 2024.10.22 01:31:47.851882 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_901_901_0 (state Temporary) 2024.10.22 01:31:47.867627 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 102605 bytes for 8d25ca6f45970fc9f4a90518a355b474:0) 2024.10.22 01:31:47.884052 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_809_412, 202410_810_810_0] 2024.10.22 01:31:47.888929 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_809_412, 202410_810_810_0] 2024.10.22 01:31:47.906292 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_901_901_0 to 202410_914_914_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:47.939865 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_908_461 to 202410_909_909_0 2024.10.22 01:31:47.956267 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.15 MiB using storage policy from min volume index 0 2024.10.22 01:31:47.957712 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59766 2024.10.22 01:31:47.958170 [ 1242 ] {} DiskLocal: Reserved 2.15 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:47.970986 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 81b6b6d50ef5120f66caefadfe9bc98f, offset: 0, size: 150297 2024.10.22 01:31:47.974346 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 81b6b6d50ef5120f66caefadfe9bc98f for removal 2024.10.22 01:31:47.976621 [ 1476 ] {} CacheMetadata: Key 81b6b6d50ef5120f66caefadfe9bc98f is removed from metadata 2024.10.22 01:31:47.977108 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 93821d2fb12ba83d48d98d54abdfa1e6, offset: 0, size: 238 2024.10.22 01:31:47.982407 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 93821d2fb12ba83d48d98d54abdfa1e6 for removal 2024.10.22 01:31:47.987192 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a3f56c34f349777260f3d6f4fca1dcab, offset: 0, size: 251 2024.10.22 01:31:48.003298 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a3f56c34f349777260f3d6f4fca1dcab for removal 2024.10.22 01:31:48.003531 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_809_412, 202410_810_810_0] 2024.10.22 01:31:48.013761 [ 1476 ] {} CacheMetadata: Key a3f56c34f349777260f3d6f4fca1dcab is removed from metadata 2024.10.22 01:31:48.026070 [ 1476 ] {} CacheMetadata: Key 93821d2fb12ba83d48d98d54abdfa1e6 is removed from metadata 2024.10.22 01:31:48.061717 [ 1731 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/iad/iqrgjmbhylitrlnczxayrktdmgnrx, size 1154817 2024.10.22 01:31:48.082328 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_388_80, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0, 202410_392_392_0, 202410_393_393_0] 2024.10.22 01:31:48.084243 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_388_80, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0, 202410_392_392_0, 202410_393_393_0] 2024.10.22 01:31:48.136078 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_909_462} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_908_461 to 202410_909_909_0 into Compact with storage Full 2024.10.22 01:31:48.146766 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_388_80, 202410_389_389_0, 202410_390_390_0, 202410_391_391_0, 202410_392_392_0, 202410_393_393_0] 2024.10.22 01:31:48.226370 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:31:48.241449 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:31:48.260493 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:31:48.276571 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:31:48.285034 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:31:48.312013 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:31:48.329168 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:48.330107 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:31:48.332714 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_909_462} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:48.342505 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:48.360212 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_909_462} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_908_461, total 60031 rows starting from the beginning of the part 2024.10.22 01:31:48.365657 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 01:31:48.423944 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_431_431_0 (state Temporary) 2024.10.22 01:31:48.512135 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_909_462} MergeTreeSequentialSource: Reading 2 marks from part 202410_909_909_0, total 82 rows starting from the beginning of the part 2024.10.22 01:31:48.513353 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rxm/jqlyjlpikrdbdjozqyeujmritaaop (LogSeriesLimiter: on interval from 2024-10-22 01:31:43 to 2024-10-22 01:31:48 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:31:48.526332 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_431_431_0 to 202410_441_441_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:48.634961 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.97 GiB, free memory in arenas 9.92 MiB, will set to 1.53 GiB (RSS), difference: 9.21 MiB 2024.10.22 01:31:48.810436 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 449 2024.10.22 01:31:48.821970 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_461_466_1, 202410_467_467_0, 202410_468_468_0, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0] 2024.10.22 01:31:48.956662 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2737fe0647946c8bc92b3b909991032e) : [0, 681153]: Setting download as failed: File segment: [0, 681153], key: 2737fe0647946c8bc92b3b909991032e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:48.957766 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2737fe0647946c8bc92b3b909991032e): No space left in cache to reserve 681154 bytes, will continue without cache download 2024.10.22 01:31:48.823642 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_461_466_1, 202410_467_467_0, 202410_468_468_0, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0] 2024.10.22 01:31:49.166396 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_461_466_1, 202410_467_467_0, 202410_468_468_0, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0] 2024.10.22 01:31:49.229678 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2737fe0647946c8bc92b3b909991032e, offset: 0, size: 0 2024.10.22 01:31:49.233655 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2737fe0647946c8bc92b3b909991032e for removal 2024.10.22 01:31:49.239342 [ 1476 ] {} CacheMetadata: Key 2737fe0647946c8bc92b3b909991032e is removed from metadata 2024.10.22 01:31:49.429042 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4296 2024.10.22 01:31:49.598808 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 8248 2024.10.22 01:31:49.835687 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:49.841314 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:49.868938 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_514_103/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:31:50.000968 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_534_534_0 (state Temporary) 2024.10.22 01:31:50.159709 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_534_534_0 to 202410_547_547_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:50.312426 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8248 2024.10.22 01:31:50.376459 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:31:50.377970 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:31:50.507188 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(56a49d443cf334ac356a2471a3469248) : [0, 605629]: Setting download as failed: File segment: [0, 605629], key: 56a49d443cf334ac356a2471a3469248, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:50.509368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(56a49d443cf334ac356a2471a3469248): No space left in cache to reserve 605630 bytes, will continue without cache download 2024.10.22 01:31:50.764709 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 56a49d443cf334ac356a2471a3469248, offset: 0, size: 0 2024.10.22 01:31:50.768676 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 56a49d443cf334ac356a2471a3469248 for removal 2024.10.22 01:31:50.778538 [ 1476 ] {} CacheMetadata: Key 56a49d443cf334ac356a2471a3469248 is removed from metadata 2024.10.22 01:31:51.652922 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_909_462} MergeTask::MergeProjectionsStage: Merge sorted 60113 rows, containing 13 columns (13 merged, 0 gathered) in 3.595071135 sec., 16720.948694106995 rows/sec., 2.66 MiB/sec. 2024.10.22 01:31:51.784505 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 808165 2024.10.22 01:31:51.827660 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_909_462} MergedBlockOutputStream: filled checksums 202410_1_909_462 (state Temporary) 2024.10.22 01:31:51.971300 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 59851 2024.10.22 01:31:52.027475 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_909_462} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_909_462 to 202410_1_909_462 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:52.032120 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ef9921767659e3cdf76f11d1c453e259) : [0, 690635]: Setting download as failed: File segment: [0, 690635], key: ef9921767659e3cdf76f11d1c453e259, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:52.032375 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_909_462} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_908_461, 202410_909_909_0] -> 202410_1_909_462 2024.10.22 01:31:52.035271 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ef9921767659e3cdf76f11d1c453e259): No space left in cache to reserve 690636 bytes, will continue without cache download 2024.10.22 01:31:52.089119 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ef9921767659e3cdf76f11d1c453e259, offset: 0, size: 0 2024.10.22 01:31:52.091192 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ef9921767659e3cdf76f11d1c453e259 for removal 2024.10.22 01:31:52.095595 [ 1476 ] {} CacheMetadata: Key ef9921767659e3cdf76f11d1c453e259 is removed from metadata 2024.10.22 01:31:52.166759 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:31:52.240620 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:52.247815 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:52.421814 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_902_902_0 (state Temporary) 2024.10.22 01:31:52.417796 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 16.33 MiB, will set to 1.53 GiB (RSS), difference: 7.16 MiB 2024.10.22 01:31:52.595155 [ 1251 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:31:52.623801 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_810_413, 202410_811_811_0] 2024.10.22 01:31:52.625715 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_902_902_0 to 202410_915_915_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:52.627065 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_810_413, 202410_811_811_0] 2024.10.22 01:31:52.647200 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_810_413, 202410_811_811_0] 2024.10.22 01:31:52.659747 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_909_462 to 202410_910_910_0 2024.10.22 01:31:52.663049 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.15 MiB using storage policy from min volume index 0 2024.10.22 01:31:52.664381 [ 1194 ] {} DiskLocal: Reserved 2.15 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:52.715250 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_910_463} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_909_462 to 202410_910_910_0 into Compact with storage Full 2024.10.22 01:31:52.758765 [ 1251 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:31:52.859598 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:52.866205 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:52.873178 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 137 entries to flush up to offset 61572 2024.10.22 01:31:52.875383 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59851 2024.10.22 01:31:52.880394 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_910_463} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:52.915376 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_910_463} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_909_462, total 60113 rows starting from the beginning of the part 2024.10.22 01:31:52.961053 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_910_463} MergeTreeSequentialSource: Reading 2 marks from part 202410_910_910_0, total 105 rows starting from the beginning of the part 2024.10.22 01:31:53.063864 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_105_109_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:31:53.070014 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8daba9b766d22e3971e9226d10ec9f1f) : [0, 981886]: Setting download as failed: File segment: [0, 981886], key: 8daba9b766d22e3971e9226d10ec9f1f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:53.072498 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8daba9b766d22e3971e9226d10ec9f1f): No space left in cache to reserve 981887 bytes, will continue without cache download 2024.10.22 01:31:53.080699 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_512_512_0 (state Temporary) 2024.10.22 01:31:53.175834 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:53.181554 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:53.239601 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_524_524_0 (state Temporary) 2024.10.22 01:31:53.245192 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8daba9b766d22e3971e9226d10ec9f1f, offset: 0, size: 0 2024.10.22 01:31:53.249798 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8daba9b766d22e3971e9226d10ec9f1f for removal 2024.10.22 01:31:53.257571 [ 1476 ] {} CacheMetadata: Key 8daba9b766d22e3971e9226d10ec9f1f is removed from metadata 2024.10.22 01:31:53.283442 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_524_524_0 to 202410_530_530_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:53.408449 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 61572 2024.10.22 01:31:53.594484 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_512_512_0 to 202410_519_519_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:53.657279 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 808165 2024.10.22 01:31:54.539315 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3f5e6933fdefd529eeeb70925ae24055) : [0, 998052]: Setting download as failed: File segment: [0, 998052], key: 3f5e6933fdefd529eeeb70925ae24055, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:54.544528 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3f5e6933fdefd529eeeb70925ae24055): No space left in cache to reserve 998053 bytes, will continue without cache download 2024.10.22 01:31:54.774275 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.97 GiB, free memory in arenas 17.19 MiB, will set to 1.54 GiB (RSS), difference: -5.02 MiB 2024.10.22 01:31:54.819046 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3f5e6933fdefd529eeeb70925ae24055, offset: 0, size: 0 2024.10.22 01:31:54.821691 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3f5e6933fdefd529eeeb70925ae24055 for removal 2024.10.22 01:31:54.825814 [ 1476 ] {} CacheMetadata: Key 3f5e6933fdefd529eeeb70925ae24055 is removed from metadata 2024.10.22 01:31:55.826388 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(15c2c83dcdce5f0995bdcbab74baee9e) : [0, 764360]: Setting download as failed: File segment: [0, 764360], key: 15c2c83dcdce5f0995bdcbab74baee9e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:55.828145 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(15c2c83dcdce5f0995bdcbab74baee9e): No space left in cache to reserve 764361 bytes, will continue without cache download 2024.10.22 01:31:55.974607 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 15c2c83dcdce5f0995bdcbab74baee9e, offset: 0, size: 0 2024.10.22 01:31:55.975862 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 15c2c83dcdce5f0995bdcbab74baee9e for removal 2024.10.22 01:31:55.977506 [ 1476 ] {} CacheMetadata: Key 15c2c83dcdce5f0995bdcbab74baee9e is removed from metadata 2024.10.22 01:31:56.312237 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 450 2024.10.22 01:31:56.402474 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 17.75 MiB, will set to 1.54 GiB (RSS), difference: 8.37 MiB 2024.10.22 01:31:56.452323 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:56.459946 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:56.515878 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_432_432_0 (state Temporary) 2024.10.22 01:31:56.770216 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_432_432_0 to 202410_442_442_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:56.878273 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 450 2024.10.22 01:31:56.885460 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 59912 2024.10.22 01:31:56.977341 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 4336 2024.10.22 01:31:57.055839 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(bb1126d74011b3caa96a5fcb91af0ac9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: bb1126d74011b3caa96a5fcb91af0ac9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:31:57.058585 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: bb1126d74011b3caa96a5fcb91af0ac9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:31:57.061493 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:31:57.257263 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:57.260217 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:57.378838 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_910_463} MergeTask::MergeProjectionsStage: Merge sorted 60218 rows, containing 13 columns (13 merged, 0 gathered) in 4.675054353 sec., 12880.705859891848 rows/sec., 2.05 MiB/sec. 2024.10.22 01:31:57.444265 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_910_463} MergedBlockOutputStream: filled checksums 202410_1_910_463 (state Temporary) 2024.10.22 01:31:57.499942 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_903_903_0 (state Temporary) 2024.10.22 01:31:57.569709 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9aa49f1d5bbba531318fab150551b968) : [0, 492053]: Setting download as failed: File segment: [0, 492053], key: 9aa49f1d5bbba531318fab150551b968, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:57.572446 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9aa49f1d5bbba531318fab150551b968): No space left in cache to reserve 492054 bytes, will continue without cache download 2024.10.22 01:31:57.662510 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_903_903_0 to 202410_916_916_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:57.680885 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_911_911_0 to 202410_916_916_0 2024.10.22 01:31:57.710751 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:57.713759 [ 1180 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:57.764821 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59912 2024.10.22 01:31:57.815492 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 8257 2024.10.22 01:31:57.856796 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9aa49f1d5bbba531318fab150551b968, offset: 0, size: 0 2024.10.22 01:31:57.858920 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9aa49f1d5bbba531318fab150551b968 for removal 2024.10.22 01:31:57.861641 [ 1476 ] {} CacheMetadata: Key 9aa49f1d5bbba531318fab150551b968 is removed from metadata 2024.10.22 01:31:57.887240 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_811_414, 202410_812_812_0] 2024.10.22 01:31:57.887607 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_910_463} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_910_463 to 202410_1_910_463 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:57.891649 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_811_414, 202410_812_812_0] 2024.10.22 01:31:57.893371 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_910_463} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_909_462, 202410_910_910_0] -> 202410_1_910_463 2024.10.22 01:31:57.950647 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_911_911_0 to 202410_916_916_0 into Compact with storage Full 2024.10.22 01:31:57.977890 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_811_414, 202410_812_812_0] 2024.10.22 01:31:58.002270 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:31:58.039463 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:31:58.044964 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:31:58.048927 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:31:58.058372 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_911_911_0, total 92 rows starting from the beginning of the part 2024.10.22 01:31:58.073510 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_535_535_0 (state Temporary) 2024.10.22 01:31:58.122501 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_535_535_0 to 202410_548_548_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:31:58.245072 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_912_912_0, total 95 rows starting from the beginning of the part 2024.10.22 01:31:58.269016 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8257 2024.10.22 01:31:58.335196 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_913_913_0, total 80 rows starting from the beginning of the part 2024.10.22 01:31:58.528398 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_914_914_0, total 112 rows starting from the beginning of the part 2024.10.22 01:31:58.612861 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_915_915_0, total 85 rows starting from the beginning of the part 2024.10.22 01:31:58.675041 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_916_916_0, total 61 rows starting from the beginning of the part 2024.10.22 01:31:58.851879 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9a9b7f0aa32f5d522e844ee2bd46fd28) : [0, 780414]: Setting download as failed: File segment: [0, 780414], key: 9a9b7f0aa32f5d522e844ee2bd46fd28, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:31:58.853422 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9a9b7f0aa32f5d522e844ee2bd46fd28): No space left in cache to reserve 780415 bytes, will continue without cache download 2024.10.22 01:31:59.015212 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9a9b7f0aa32f5d522e844ee2bd46fd28, offset: 0, size: 0 2024.10.22 01:31:59.017172 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9a9b7f0aa32f5d522e844ee2bd46fd28 for removal 2024.10.22 01:31:59.019695 [ 1476 ] {} CacheMetadata: Key 9a9b7f0aa32f5d522e844ee2bd46fd28 is removed from metadata 2024.10.22 01:32:00.124723 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(79dfc762fc05752426e7cb9ce6dc7439) : [0, 652349]: Setting download as failed: File segment: [0, 652349], key: 79dfc762fc05752426e7cb9ce6dc7439, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:32:00.127149 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(79dfc762fc05752426e7cb9ce6dc7439): No space left in cache to reserve 652350 bytes, will continue without cache download 2024.10.22 01:32:00.162276 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 15.42 MiB, will set to 1.53 GiB (RSS), difference: -1.69 MiB 2024.10.22 01:32:00.218291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 79dfc762fc05752426e7cb9ce6dc7439, offset: 0, size: 0 2024.10.22 01:32:00.222640 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 79dfc762fc05752426e7cb9ce6dc7439 for removal 2024.10.22 01:32:00.233524 [ 1476 ] {} CacheMetadata: Key 79dfc762fc05752426e7cb9ce6dc7439 is removed from metadata 2024.10.22 01:32:00.352886 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_514_103/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:32:00.377951 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:32:00.380614 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:32:00.630914 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} MergeTask::MergeProjectionsStage: Merge sorted 525 rows, containing 13 columns (13 merged, 0 gathered) in 2.796256025 sec., 187.7510482968025 rows/sec., 30.92 KiB/sec. 2024.10.22 01:32:00.661654 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1419 entries to flush up to offset 809584 2024.10.22 01:32:00.669792 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} MergedBlockOutputStream: filled checksums 202410_911_916_1 (state Temporary) 2024.10.22 01:32:00.928085 [ 1247 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:32:01.046855 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 96 entries to flush up to offset 61668 2024.10.22 01:32:01.137010 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(32d0efacfa1a96cc4a2e7bccb98ddea3) : [0, 550057]: Setting download as failed: File segment: [0, 550057], key: 32d0efacfa1a96cc4a2e7bccb98ddea3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:32:01.139126 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(32d0efacfa1a96cc4a2e7bccb98ddea3): No space left in cache to reserve 550058 bytes, will continue without cache download 2024.10.22 01:32:01.223622 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_911_916_1 to 202410_911_916_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:01.256456 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:01.257468 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_911_916_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_911_911_0, 202410_916_916_0] -> 202410_911_916_1 2024.10.22 01:32:01.258275 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:01.295534 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 32d0efacfa1a96cc4a2e7bccb98ddea3, offset: 0, size: 0 2024.10.22 01:32:01.298303 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_525_525_0 (state Temporary) 2024.10.22 01:32:01.298665 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 32d0efacfa1a96cc4a2e7bccb98ddea3 for removal 2024.10.22 01:32:01.299477 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:32:01.303257 [ 1476 ] {} CacheMetadata: Key 32d0efacfa1a96cc4a2e7bccb98ddea3 is removed from metadata 2024.10.22 01:32:01.549131 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_525_525_0 to 202410_531_531_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:01.561011 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:01.565470 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:01.597781 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 61668 2024.10.22 01:32:01.676219 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_513_513_0 (state Temporary) 2024.10.22 01:32:01.774036 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 59988 2024.10.22 01:32:01.774750 [ 1424 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Found 5 old parts to remove. Parts: [202410_1_21_5, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0] 2024.10.22 01:32:01.779417 [ 1424 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from filesystem (serially): Parts: [202410_1_21_5, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0] 2024.10.22 01:32:01.781730 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_513_513_0 to 202410_520_520_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:01.877253 [ 1208 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_515_515_0 to 202410_520_520_0 2024.10.22 01:32:01.912492 [ 1208 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:01.919335 [ 1208 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:02.017276 [ 1424 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from memory: Parts: [202410_1_21_5, 202410_22_22_0, 202410_23_23_0, 202410_24_24_0, 202410_25_25_0] 2024.10.22 01:32:02.049742 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:02.054604 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:02.139596 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_515_515_0 to 202410_520_520_0 into Compact with storage Full 2024.10.22 01:32:02.137567 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 809584 2024.10.22 01:32:02.386032 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:02.401015 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_515_515_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:32:02.401845 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_904_904_0 (state Temporary) 2024.10.22 01:32:02.546921 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_516_516_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:32:02.566548 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_904_904_0 to 202410_917_917_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:02.616465 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 59988 2024.10.22 01:32:02.803033 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_517_517_0, total 1163 rows starting from the beginning of the part 2024.10.22 01:32:02.841712 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_518_518_0, total 1532 rows starting from the beginning of the part 2024.10.22 01:32:02.890801 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_519_519_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:32:02.957092 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_520_520_0, total 1419 rows starting from the beginning of the part 2024.10.22 01:32:03.062549 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_812_415, 202410_813_813_0] 2024.10.22 01:32:03.069397 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_812_415, 202410_813_813_0] 2024.10.22 01:32:03.084173 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_812_415, 202410_813_813_0] 2024.10.22 01:32:03.257945 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ec0e0a09b56e80aed3db7917d64702cb) : [0, 547838]: Setting download as failed: File segment: [0, 547838], key: ec0e0a09b56e80aed3db7917d64702cb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:32:03.259272 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ec0e0a09b56e80aed3db7917d64702cb): No space left in cache to reserve 547839 bytes, will continue without cache download 2024.10.22 01:32:03.386909 [ 1690 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tvu/huefavmxcoccggbzkqqjsechxlgfz, size 1763794 2024.10.22 01:32:03.408639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ec0e0a09b56e80aed3db7917d64702cb, offset: 0, size: 0 2024.10.22 01:32:03.411016 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ec0e0a09b56e80aed3db7917d64702cb for removal 2024.10.22 01:32:03.414378 [ 1476 ] {} CacheMetadata: Key ec0e0a09b56e80aed3db7917d64702cb is removed from metadata 2024.10.22 01:32:03.946393 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:32:03.957029 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} MergeTask::MergeProjectionsStage: Merge sorted 8734 rows, containing 4 columns (4 merged, 0 gathered) in 1.89408907 sec., 4611.187582640979 rows/sec., 72.05 KiB/sec. 2024.10.22 01:32:03.958583 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:32:03.976853 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:32:03.989099 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} MergedBlockOutputStream: filled checksums 202410_515_520_1 (state Temporary) 2024.10.22 01:32:04.004892 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:32:04.029680 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:32:04.061556 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:32:04.077137 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:32:04.158300 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(34a3ead0c48ac4d6a8dbf109712760a9) : [0, 426296]: Setting download as failed: File segment: [0, 426296], key: 34a3ead0c48ac4d6a8dbf109712760a9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:32:04.159736 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(34a3ead0c48ac4d6a8dbf109712760a9): No space left in cache to reserve 426297 bytes, will continue without cache download 2024.10.22 01:32:04.215160 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 34a3ead0c48ac4d6a8dbf109712760a9, offset: 0, size: 0 2024.10.22 01:32:04.220084 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 34a3ead0c48ac4d6a8dbf109712760a9 for removal 2024.10.22 01:32:04.224981 [ 1476 ] {} CacheMetadata: Key 34a3ead0c48ac4d6a8dbf109712760a9 is removed from metadata 2024.10.22 01:32:04.258620 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_515_520_1 to 202410_515_520_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:04.263640 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_515_520_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_515_515_0, 202410_520_520_0] -> 202410_515_520_1 2024.10.22 01:32:04.308128 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:32:04.310587 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:32:04.328206 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:32:04.369223 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 14.09 MiB, will set to 1.53 GiB (RSS), difference: 6.18 MiB 2024.10.22 01:32:04.387598 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 451 2024.10.22 01:32:04.491803 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pvp/hayqaxtysacyiceddyilajnkijuql (LogSeriesLimiter: on interval from 2024-10-22 01:31:48 to 2024-10-22 01:32:04 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:32:04.872503 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:04.874932 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:05.032378 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_433_433_0 (state Temporary) 2024.10.22 01:32:05.049839 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(cc8c2033ab645069f7b62ee049cfeab2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cc8c2033ab645069f7b62ee049cfeab2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:32:05.054296 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 554567, file segment info: File segment: [0, 33554431], key: cc8c2033ab645069f7b62ee049cfeab2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:32:05.057903 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:32:05.113675 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_433_433_0 to 202410_443_443_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:05.128486 [ 1701 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rxm/jqlyjlpikrdbdjozqyeujmritaaop, size 554567 2024.10.22 01:32:05.173443 [ 1239 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_438_90 to 202410_443_443_0 2024.10.22 01:32:05.201377 [ 1239 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:05.231206 [ 1239 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:05.267540 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 451 2024.10.22 01:32:05.273226 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_438_90 to 202410_443_443_0 into Compact with storage Full 2024.10.22 01:32:05.510255 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:05.538242 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:32:05.546864 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:32:05.557925 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_438_90, total 482 rows starting from the beginning of the part 2024.10.22 01:32:05.561634 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:32:05.570273 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:32:05.578379 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:32:05.599957 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:32:05.620008 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:32:05.641716 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column FlashMajor 2024.10.22 01:32:05.771034 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 8279 2024.10.22 01:32:05.880074 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_439_439_0, total 1 rows starting from the beginning of the part 2024.10.22 01:32:06.081775 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:06.185826 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:06.391070 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_440_440_0, total 1 rows starting from the beginning of the part 2024.10.22 01:32:06.459479 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_536_536_0 (state Temporary) 2024.10.22 01:32:06.561213 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_536_536_0 to 202410_549_549_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:06.575873 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 1.97 GiB, free memory in arenas 13.20 MiB, will set to 1.54 GiB (RSS), difference: -13.87 MiB 2024.10.22 01:32:06.641619 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8279 2024.10.22 01:32:06.648064 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 60072 2024.10.22 01:32:06.657421 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4f9ebdb3a69b92304c184eab0bd93a3f) : [0, 731593]: Setting download as failed: File segment: [0, 731593], key: 4f9ebdb3a69b92304c184eab0bd93a3f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:32:06.660383 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4f9ebdb3a69b92304c184eab0bd93a3f): No space left in cache to reserve 731594 bytes, will continue without cache download 2024.10.22 01:32:06.720014 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_441_441_0, total 1 rows starting from the beginning of the part 2024.10.22 01:32:06.960481 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4f9ebdb3a69b92304c184eab0bd93a3f, offset: 0, size: 0 2024.10.22 01:32:06.962487 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4f9ebdb3a69b92304c184eab0bd93a3f for removal 2024.10.22 01:32:06.966500 [ 1476 ] {} CacheMetadata: Key 4f9ebdb3a69b92304c184eab0bd93a3f is removed from metadata 2024.10.22 01:32:06.984023 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:06.987957 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:07.023546 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_442_442_0, total 1 rows starting from the beginning of the part 2024.10.22 01:32:07.075882 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_905_905_0 (state Temporary) 2024.10.22 01:32:07.090583 [ 1242 ] {} DatabaseCatalog: Execution took 408 ms. 2024.10.22 01:32:07.107724 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_905_905_0 to 202410_918_918_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:07.203187 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60072 2024.10.22 01:32:07.449917 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_443_443_0, total 1 rows starting from the beginning of the part 2024.10.22 01:32:07.464986 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_813_416, 202410_814_814_0] 2024.10.22 01:32:07.467820 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_813_416, 202410_814_814_0] 2024.10.22 01:32:07.581697 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_813_416, 202410_814_814_0] 2024.10.22 01:32:07.762157 [ 1205 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:32:07.784313 [ 1205 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:32:07.873033 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7113d239028a5fdd24c9f070fa74be04) : [0, 598826]: Setting download as failed: File segment: [0, 598826], key: 7113d239028a5fdd24c9f070fa74be04, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:32:07.875002 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7113d239028a5fdd24c9f070fa74be04): No space left in cache to reserve 598827 bytes, will continue without cache download 2024.10.22 01:32:07.950599 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7113d239028a5fdd24c9f070fa74be04, offset: 0, size: 0 2024.10.22 01:32:07.952315 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7113d239028a5fdd24c9f070fa74be04 for removal 2024.10.22 01:32:07.955803 [ 1476 ] {} CacheMetadata: Key 7113d239028a5fdd24c9f070fa74be04 is removed from metadata 2024.10.22 01:32:08.446540 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_443_91/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:32:08.643765 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 1.97 GiB, free memory in arenas 11.05 MiB, will set to 1.54 GiB (RSS), difference: -3.76 MiB 2024.10.22 01:32:08.745789 [ 1177 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 270 ms. 2024.10.22 01:32:08.758311 [ 1153 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 213 ms. 2024.10.22 01:32:09.125358 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 93 entries to flush up to offset 61761 2024.10.22 01:32:09.249784 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1276 entries to flush up to offset 810860 2024.10.22 01:32:09.295544 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ce2b8bc6b2cddfd17633d7f8bca68d9f) : [0, 1054475]: Setting download as failed: File segment: [0, 1054475], key: ce2b8bc6b2cddfd17633d7f8bca68d9f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:32:09.302709 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ce2b8bc6b2cddfd17633d7f8bca68d9f): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:32:09.467654 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ce2b8bc6b2cddfd17633d7f8bca68d9f, offset: 0, size: 0 2024.10.22 01:32:09.475093 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ce2b8bc6b2cddfd17633d7f8bca68d9f for removal 2024.10.22 01:32:09.477087 [ 1476 ] {} CacheMetadata: Key ce2b8bc6b2cddfd17633d7f8bca68d9f is removed from metadata 2024.10.22 01:32:09.560721 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:09.564138 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:09.612367 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_526_526_0 (state Temporary) 2024.10.22 01:32:09.777087 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_526_526_0 to 202410_532_532_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:09.784244 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_105_109_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:32:09.834462 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_527_4 to 202410_532_532_0 2024.10.22 01:32:09.855639 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:09.856348 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 61761 2024.10.22 01:32:09.860383 [ 1154 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:09.908189 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_527_4 to 202410_532_532_0 into Compact with storage Full 2024.10.22 01:32:10.001656 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:10.028563 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_527_4, total 2064 rows starting from the beginning of the part 2024.10.22 01:32:10.074943 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_528_528_0, total 111 rows starting from the beginning of the part 2024.10.22 01:32:10.090764 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} MergeTask::MergeProjectionsStage: Merge sorted 487 rows, containing 32 columns (32 merged, 0 gathered) in 4.826756189 sec., 100.89591869377101 rows/sec., 9.35 KiB/sec. 2024.10.22 01:32:10.128687 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_529_529_0, total 135 rows starting from the beginning of the part 2024.10.22 01:32:10.195597 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_530_530_0, total 137 rows starting from the beginning of the part 2024.10.22 01:32:10.257499 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_531_531_0, total 96 rows starting from the beginning of the part 2024.10.22 01:32:10.268694 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} MergedBlockOutputStream: filled checksums 202410_1_443_91 (state Temporary) 2024.10.22 01:32:10.304703 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_532_532_0, total 93 rows starting from the beginning of the part 2024.10.22 01:32:10.381131 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:32:10.399890 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:32:10.429672 [ 1271 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 01:32:10.632514 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:10.738237 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:10.806648 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(14f56f99a4b93924877a8bc06b6d6c76) : [0, 1154911]: Setting download as failed: File segment: [0, 1154911], key: 14f56f99a4b93924877a8bc06b6d6c76, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:32:10.812082 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(14f56f99a4b93924877a8bc06b6d6c76): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:32:10.883341 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_514_514_0 (state Temporary) 2024.10.22 01:32:10.893710 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 14f56f99a4b93924877a8bc06b6d6c76, offset: 0, size: 0 2024.10.22 01:32:10.906471 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_514_103} MergeTask::MergeProjectionsStage: Merge sorted 814440 rows, containing 4 columns (4 merged, 0 gathered) in 26.078632411 sec., 31230.16526190492 rows/sec., 468.26 KiB/sec. 2024.10.22 01:32:10.966187 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_514_514_0 to 202410_521_521_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:10.969617 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_443_91 to 202410_1_443_91 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:10.974999 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_443_91} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_438_90, 202410_443_443_0] -> 202410_1_443_91 2024.10.22 01:32:10.993658 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_514_103} MergedBlockOutputStream: filled checksums 202410_1_514_103 (state Temporary) 2024.10.22 01:32:10.897222 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 14f56f99a4b93924877a8bc06b6d6c76 for removal 2024.10.22 01:32:11.023685 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 810860 2024.10.22 01:32:11.052991 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:32:11.106616 [ 1476 ] {} CacheMetadata: Key 14f56f99a4b93924877a8bc06b6d6c76 is removed from metadata 2024.10.22 01:32:11.227648 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 60146 2024.10.22 01:32:11.477379 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:11.479509 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:11.551609 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 8.90 MiB, will set to 1.53 GiB (RSS), difference: 7.73 MiB 2024.10.22 01:32:11.667305 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_906_906_0 (state Temporary) 2024.10.22 01:32:11.728337 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_906_906_0 to 202410_919_919_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:11.953788 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60146 2024.10.22 01:32:11.975657 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} MergeTask::MergeProjectionsStage: Merge sorted 2636 rows, containing 13 columns (13 merged, 0 gathered) in 2.084160062 sec., 1264.7780984107544 rows/sec., 560.95 KiB/sec. 2024.10.22 01:32:12.228320 [ 1195 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 01:32:12.237184 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} MergedBlockOutputStream: filled checksums 202410_507_532_5 (state Temporary) 2024.10.22 01:32:12.242349 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0b846d55d0db2c490dfa73335b58ee27) : [0, 836493]: Setting download as failed: File segment: [0, 836493], key: 0b846d55d0db2c490dfa73335b58ee27, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:32:12.244428 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0b846d55d0db2c490dfa73335b58ee27): No space left in cache to reserve 836494 bytes, will continue without cache download 2024.10.22 01:32:12.286529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0b846d55d0db2c490dfa73335b58ee27, offset: 0, size: 0 2024.10.22 01:32:12.291039 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0b846d55d0db2c490dfa73335b58ee27 for removal 2024.10.22 01:32:12.293430 [ 1476 ] {} CacheMetadata: Key 0b846d55d0db2c490dfa73335b58ee27 is removed from metadata 2024.10.22 01:32:12.462831 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 01:32:12.722918 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_532_5 to 202410_507_532_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:12.728826 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_532_5} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_527_4, 202410_532_532_0] -> 202410_507_532_5 2024.10.22 01:32:12.766567 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.65 MiB. 2024.10.22 01:32:12.769340 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 452 2024.10.22 01:32:12.974860 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(f661ce116c3ef0f653e2199ac45c8023) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f661ce116c3ef0f653e2199ac45c8023, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:32:12.976261 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: f661ce116c3ef0f653e2199ac45c8023, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:32:12.978802 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:32:13.127183 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:13.133590 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:13.193262 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_434_434_0 (state Temporary) 2024.10.22 01:32:13.277298 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_434_434_0 to 202410_444_444_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:13.293428 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_814_417, 202410_815_815_0] 2024.10.22 01:32:13.297699 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_814_417, 202410_815_815_0] 2024.10.22 01:32:13.311067 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_814_417, 202410_815_815_0] 2024.10.22 01:32:13.468116 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 452 2024.10.22 01:32:13.686557 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_514_103} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_514_103 to 202410_1_514_103 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:13.694685 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_514_103} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_508_102, 202410_509_514_1] -> 202410_1_514_103 2024.10.22 01:32:13.722546 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:32:14.079838 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 212934 bytes for 4542d21fb45c8278b5b4ec1f3d4ca073:0) 2024.10.22 01:32:14.081519 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c2a75bcc38b429d47fc4205ab163d412, offset: 0, size: 197 2024.10.22 01:32:14.149517 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 8302 2024.10.22 01:32:14.185460 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c2a75bcc38b429d47fc4205ab163d412 for removal 2024.10.22 01:32:14.190766 [ 1476 ] {} CacheMetadata: Key c2a75bcc38b429d47fc4205ab163d412 is removed from metadata 2024.10.22 01:32:14.190472 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8d25ca6f45970fc9f4a90518a355b474, offset: 0, size: 102605 2024.10.22 01:32:14.199924 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8d25ca6f45970fc9f4a90518a355b474 for removal 2024.10.22 01:32:14.210527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fafe1693bf5c42f26c184256b13b3ddd, offset: 0, size: 10092 2024.10.22 01:32:14.217669 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fafe1693bf5c42f26c184256b13b3ddd for removal 2024.10.22 01:32:14.212838 [ 1476 ] {} CacheMetadata: Key 8d25ca6f45970fc9f4a90518a355b474 is removed from metadata 2024.10.22 01:32:14.225031 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8bd4a743356a21507452b29859272041, offset: 0, size: 234 2024.10.22 01:32:14.230331 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8bd4a743356a21507452b29859272041 for removal 2024.10.22 01:32:14.235244 [ 1476 ] {} CacheMetadata: Key fafe1693bf5c42f26c184256b13b3ddd is removed from metadata 2024.10.22 01:32:14.258554 [ 1476 ] {} CacheMetadata: Key 8bd4a743356a21507452b29859272041 is removed from metadata 2024.10.22 01:32:14.276063 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fml/zjvoglossxyluvskmevegjjdbfuwb. 2024.10.22 01:32:14.325478 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/onc/gugejquaqjcorxuioswwopeqwkdxf. 2024.10.22 01:32:14.524132 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:14.528230 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:14.572119 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_537_537_0 (state Temporary) 2024.10.22 01:32:14.725035 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_537_537_0 to 202410_550_550_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:14.774867 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8302 2024.10.22 01:32:15.259962 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:15.262741 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:15.468215 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.97 GiB, free memory in arenas 4.67 MiB, will set to 1.53 GiB (RSS), difference: 5.69 MiB 2024.10.22 01:32:15.784092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/onc/gugejquaqjcorxuioswwopeqwkdxf. 2024.10.22 01:32:15.787584 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_3_3_0 to 201403_3_3_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:15.812062 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fml/zjvoglossxyluvskmevegjjdbfuwb. 2024.10.22 01:32:15.864076 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_4_4_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:32:15.964642 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 60208 2024.10.22 01:32:16.481349 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:16.486957 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:16.667191 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_907_907_0 (state Temporary) 2024.10.22 01:32:16.830735 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_907_907_0 to 202410_920_920_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:17.000056 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60208 2024.10.22 01:32:17.044767 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_815_418, 202410_816_816_0] 2024.10.22 01:32:17.055151 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_815_418, 202410_816_816_0] 2024.10.22 01:32:17.167569 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_815_418, 202410_816_816_0] 2024.10.22 01:32:17.371469 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 75 entries to flush up to offset 61836 2024.10.22 01:32:17.450858 [ 1211 ] {} DatabaseCatalog: Execution took 257 ms. 2024.10.22 01:32:17.536618 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.00 GiB, peak 1.97 GiB, free memory in arenas 5.08 MiB, will set to 1.19 GiB (RSS), difference: 190.61 MiB 2024.10.22 01:32:17.734517 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:17.740716 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:17.802952 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_527_527_0 (state Temporary) 2024.10.22 01:32:17.965444 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_527_527_0 to 202410_533_533_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:18.125789 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1759 entries to flush up to offset 812619 2024.10.22 01:32:18.157058 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 61836 2024.10.22 01:32:19.027504 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_108_108_0 (state Temporary) 2024.10.22 01:32:19.065007 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:19.068315 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:19.167881 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 28 2024.10.22 01:32:19.217908 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_515_515_0 (state Temporary) 2024.10.22 01:32:19.289315 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_515_515_0 to 202410_522_522_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:19.338480 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 812619 2024.10.22 01:32:19.556663 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.19 GiB, peak 1.97 GiB, free memory in arenas 5.59 MiB, will set to 1.20 GiB (RSS), difference: 12.62 MiB 2024.10.22 01:32:19.752481 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:19.758660 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:19.814502 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_28_28_0 (state Temporary) 2024.10.22 01:32:20.031100 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_28_28_0 to 202410_28_28_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:20.306098 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 28 2024.10.22 01:32:20.357296 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_486_97, 202410_487_487_0, 202410_488_488_0, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0] 2024.10.22 01:32:20.358605 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_486_97, 202410_487_487_0, 202410_488_488_0, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0] 2024.10.22 01:32:20.379044 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:32:20.380946 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:32:20.481268 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_486_97, 202410_487_487_0, 202410_488_488_0, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0] 2024.10.22 01:32:20.849622 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(6799854bd0156bf3de78b2ed3279b084) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6799854bd0156bf3de78b2ed3279b084, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:32:20.853612 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 535047, file segment info: File segment: [0, 33554431], key: 6799854bd0156bf3de78b2ed3279b084, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:32:20.859862 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:32:20.970199 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 453 2024.10.22 01:32:21.016799 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 60248 2024.10.22 01:32:21.160295 [ 1201 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 215 ms. 2024.10.22 01:32:21.317096 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:32:21.352253 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:32:21.366889 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:32:21.371666 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:21.373824 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:21.377775 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:21.384879 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:32:21.391267 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:21.408909 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:32:21.460836 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_908_908_0 (state Temporary) 2024.10.22 01:32:21.570913 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_435_435_0 (state Temporary) 2024.10.22 01:32:21.650757 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:32:21.661622 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.28 GiB, peak 1.97 GiB, free memory in arenas 5.61 MiB, will set to 1.23 GiB (RSS), difference: -56.52 MiB 2024.10.22 01:32:21.667292 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:32:21.695679 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column FlashMinor 2024.10.22 01:32:21.772433 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_435_435_0 to 202410_445_445_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:21.786815 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_908_908_0 to 202410_921_921_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:21.826667 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_910_463 to 202410_911_916_1 2024.10.22 01:32:21.835956 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.17 MiB using storage policy from min volume index 0 2024.10.22 01:32:21.841621 [ 1231 ] {} DiskLocal: Reserved 2.17 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:21.857454 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/asy/oqeiodzqtntfufunzpqutsdtvjusn (LogSeriesLimiter: on interval from 2024-10-22 01:32:04 to 2024-10-22 01:32:21 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:32:21.872092 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_916_464} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_910_463 to 202410_911_916_1 into Compact with storage Full 2024.10.22 01:32:21.937173 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_916_464} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:21.954039 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_916_464} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_910_463, total 60218 rows starting from the beginning of the part 2024.10.22 01:32:21.981491 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60248 2024.10.22 01:32:21.982558 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_916_464} MergeTreeSequentialSource: Reading 2 marks from part 202410_911_916_1, total 525 rows starting from the beginning of the part 2024.10.22 01:32:21.986315 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 453 2024.10.22 01:32:22.277165 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 8310 2024.10.22 01:32:22.386280 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:22.391755 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:22.451234 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_538_538_0 (state Temporary) 2024.10.22 01:32:22.498703 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_538_538_0 to 202410_551_551_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:22.597130 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_546_109 to 202410_551_551_0 2024.10.22 01:32:22.611614 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:22.615219 [ 1220 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:22.674363 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8310 2024.10.22 01:32:22.764330 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_546_109 to 202410_551_551_0 into Compact with storage Full 2024.10.22 01:32:22.798243 [ 1214 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:32:22.829071 [ 1214 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:32:22.879536 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:22.885130 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_816_419, 202410_817_817_0] 2024.10.22 01:32:22.888000 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_816_419, 202410_817_817_0] 2024.10.22 01:32:22.898316 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_546_109, total 8334 rows starting from the beginning of the part 2024.10.22 01:32:22.915293 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_816_419, 202410_817_817_0] 2024.10.22 01:32:22.935617 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_547_547_0, total 20 rows starting from the beginning of the part 2024.10.22 01:32:22.968800 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_548_548_0, total 9 rows starting from the beginning of the part 2024.10.22 01:32:23.016767 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_549_549_0, total 22 rows starting from the beginning of the part 2024.10.22 01:32:23.058767 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_550_550_0, total 23 rows starting from the beginning of the part 2024.10.22 01:32:23.095463 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_551_551_0, total 8 rows starting from the beginning of the part 2024.10.22 01:32:23.590468 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.24 GiB, peak 1.97 GiB, free memory in arenas 5.18 MiB, will set to 1.25 GiB (RSS), difference: 10.78 MiB 2024.10.22 01:32:23.777524 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.087934866 sec. 2024.10.22 01:32:24.417356 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_818_818_0, 202410_819_819_0, 202410_820_820_0, 202410_821_821_0, 202410_822_822_0, 202410_823_823_0] 2024.10.22 01:32:24.432832 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_818_818_0, 202410_819_819_0, 202410_820_820_0, 202410_821_821_0, 202410_822_822_0, 202410_823_823_0] 2024.10.22 01:32:24.462023 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_818_818_0, 202410_819_819_0, 202410_820_820_0, 202410_821_821_0, 202410_822_822_0, 202410_823_823_0] 2024.10.22 01:32:24.739402 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} MergeTask::MergeProjectionsStage: Merge sorted 8416 rows, containing 12 columns (12 merged, 0 gathered) in 1.982592412 sec., 4244.947145495279 rows/sec., 674.77 KiB/sec. 2024.10.22 01:32:24.766290 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} MergedBlockOutputStream: filled checksums 202410_1_551_110 (state Temporary) 2024.10.22 01:32:25.117324 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_108_108_0 to 202410_111_111_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:25.175940 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_551_110 to 202410_1_551_110 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:25.201407 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_551_110} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_546_109, 202410_551_551_0] -> 202410_1_551_110 2024.10.22 01:32:25.252773 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.59 MiB. 2024.10.22 01:32:25.263802 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_916_464} MergeTask::MergeProjectionsStage: Merge sorted 60743 rows, containing 13 columns (13 merged, 0 gathered) in 3.395739557 sec., 17888.003181746957 rows/sec., 2.85 MiB/sec. 2024.10.22 01:32:25.280383 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_105_109_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:32:25.321576 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_916_464} MergedBlockOutputStream: filled checksums 202410_1_916_464 (state Temporary) 2024.10.22 01:32:25.436744 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 1.97 GiB, free memory in arenas 5.44 MiB, will set to 1.28 GiB (RSS), difference: -3.54 MiB 2024.10.22 01:32:25.467206 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_916_464} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_916_464 to 202410_1_916_464 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:25.470050 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_916_464} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_910_463, 202410_911_916_1] -> 202410_1_916_464 2024.10.22 01:32:25.684247 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:32:25.699450 [ 1679 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pvp/hayqaxtysacyiceddyilajnkijuql, size 4286889 2024.10.22 01:32:25.705495 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.92 MiB. 2024.10.22 01:32:25.709068 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:32:25.719292 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:32:25.736862 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:32:25.771047 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 61927 2024.10.22 01:32:25.865075 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:32:25.875938 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:32:25.886445 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:32:25.941072 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:32:25.967944 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:32:26.087208 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 60328 2024.10.22 01:32:26.123819 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/cmu/avugrslubmlwktkrgwstsfbyddaxh (LogSeriesLimiter: on interval from 2024-10-22 01:32:21 to 2024-10-22 01:32:26 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:32:26.153984 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:26.158311 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:26.353390 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1708 entries to flush up to offset 814327 2024.10.22 01:32:26.378740 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:26.385659 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:26.388434 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_528_528_0 (state Temporary) 2024.10.22 01:32:26.560495 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_528_528_0 to 202410_534_534_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:26.701625 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_909_909_0 (state Temporary) 2024.10.22 01:32:26.728620 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 61927 2024.10.22 01:32:26.772277 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_909_909_0 to 202410_922_922_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:26.828742 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_916_464 to 202410_917_917_0 2024.10.22 01:32:26.850582 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60328 2024.10.22 01:32:26.854671 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.18 MiB using storage policy from min volume index 0 2024.10.22 01:32:26.859710 [ 1241 ] {} DiskLocal: Reserved 2.18 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:26.888381 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_917_465} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_916_464 to 202410_917_917_0 into Compact with storage Full 2024.10.22 01:32:27.071275 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_917_465} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:27.082567 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_917_465} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_916_464, total 60743 rows starting from the beginning of the part 2024.10.22 01:32:27.181758 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_917_465} MergeTreeSequentialSource: Reading 2 marks from part 202410_917_917_0, total 76 rows starting from the beginning of the part 2024.10.22 01:32:27.426909 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 1.97 GiB, free memory in arenas 5.55 MiB, will set to 1.32 GiB (RSS), difference: -5.00 MiB 2024.10.22 01:32:27.440090 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:27.445309 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:27.583676 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_516_516_0 (state Temporary) 2024.10.22 01:32:27.771933 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_516_516_0 to 202410_523_523_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:27.797064 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 814327 2024.10.22 01:32:28.260045 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4336 2024.10.22 01:32:29.684482 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 1.97 GiB, free memory in arenas 8.23 MiB, will set to 1.36 GiB (RSS), difference: -6.66 MiB 2024.10.22 01:32:30.176749 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 8325 2024.10.22 01:32:30.380329 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:32:30.384199 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:32:30.552157 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_4_4_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:32:30.718048 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_917_465} MergeTask::MergeProjectionsStage: Merge sorted 60819 rows, containing 13 columns (13 merged, 0 gathered) in 3.837765241 sec., 15847.503997965361 rows/sec., 2.52 MiB/sec. 2024.10.22 01:32:30.723768 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:30.735747 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:30.784518 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_539_539_0 (state Temporary) 2024.10.22 01:32:30.792212 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_917_465/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:32:30.852285 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_539_539_0 to 202410_552_552_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:30.877242 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 60366 2024.10.22 01:32:30.937130 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_917_465} MergedBlockOutputStream: filled checksums 202410_1_917_465 (state Temporary) 2024.10.22 01:32:31.023158 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_917_465} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_917_465 to 202410_1_917_465 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:31.028020 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_917_465} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_916_464, 202410_917_917_0] -> 202410_1_917_465 2024.10.22 01:32:31.094239 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8325 2024.10.22 01:32:31.162611 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:32:31.262249 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:31.263894 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:31.315887 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_461_471_2, 202410_472_472_0, 202410_473_473_0, 202410_474_474_0, 202410_475_475_0, 202410_476_476_0] 2024.10.22 01:32:31.323336 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_461_471_2, 202410_472_472_0, 202410_473_473_0, 202410_474_474_0, 202410_475_475_0, 202410_476_476_0] 2024.10.22 01:32:31.361023 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_461_471_2, 202410_472_472_0, 202410_473_473_0, 202410_474_474_0, 202410_475_475_0, 202410_476_476_0] 2024.10.22 01:32:31.398547 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_910_910_0 (state Temporary) 2024.10.22 01:32:31.404696 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 1.97 GiB, free memory in arenas 7.37 MiB, will set to 1.40 GiB (RSS), difference: -2.53 MiB 2024.10.22 01:32:31.663291 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_910_910_0 to 202410_923_923_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:31.802675 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_917_465 to 202410_918_918_0 2024.10.22 01:32:31.826878 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.18 MiB using storage policy from min volume index 0 2024.10.22 01:32:31.828928 [ 1150 ] {} DiskLocal: Reserved 2.18 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:31.862856 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_918_466} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_917_465 to 202410_918_918_0 into Compact with storage Full 2024.10.22 01:32:31.884837 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_918_466} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:31.896005 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_918_466} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_917_465, total 60819 rows starting from the beginning of the part 2024.10.22 01:32:31.940049 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_918_466} MergeTreeSequentialSource: Reading 2 marks from part 202410_918_918_0, total 84 rows starting from the beginning of the part 2024.10.22 01:32:32.259133 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60366 2024.10.22 01:32:33.250993 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_459_92, 202410_460_460_0, 202410_461_461_0, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0] 2024.10.22 01:32:33.256522 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_459_92, 202410_460_460_0, 202410_461_461_0, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0] 2024.10.22 01:32:33.297410 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.97 GiB, free memory in arenas 6.40 MiB, will set to 1.45 GiB (RSS), difference: -5.32 MiB 2024.10.22 01:32:33.492403 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_459_92, 202410_460_460_0, 202410_461_461_0, 202410_462_462_0, 202410_463_463_0, 202410_464_464_0] 2024.10.22 01:32:34.261281 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 113 entries to flush up to offset 62040 2024.10.22 01:32:34.712767 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:34.719357 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:34.801270 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 815869 2024.10.22 01:32:35.067255 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_529_529_0 (state Temporary) 2024.10.22 01:32:35.122422 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_529_529_0 to 202410_535_535_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:35.128954 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_918_466} MergeTask::MergeProjectionsStage: Merge sorted 60903 rows, containing 13 columns (13 merged, 0 gathered) in 3.270074402 sec., 18624.34688420279 rows/sec., 2.96 MiB/sec. 2024.10.22 01:32:35.176433 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 62040 2024.10.22 01:32:35.402542 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_918_466} MergedBlockOutputStream: filled checksums 202410_1_918_466 (state Temporary) 2024.10.22 01:32:35.524841 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.97 GiB, free memory in arenas 6.86 MiB, will set to 1.46 GiB (RSS), difference: -140.54 MiB 2024.10.22 01:32:35.533531 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_918_466} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_918_466 to 202410_1_918_466 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:35.548802 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_918_466} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_917_465, 202410_918_918_0] -> 202410_1_918_466 2024.10.22 01:32:35.703508 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:32:35.775777 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 4374 2024.10.22 01:32:35.827147 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(911feed3b8162db7c90be37a15c082ef) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 911feed3b8162db7c90be37a15c082ef, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:32:35.829825 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 911feed3b8162db7c90be37a15c082ef, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:32:35.837480 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:32:36.166565 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:36.169724 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:36.267665 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 60411 2024.10.22 01:32:36.392419 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_517_517_0 (state Temporary) 2024.10.22 01:32:36.544587 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_517_517_0 to 202410_524_524_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:36.557586 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:36.560572 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:36.566913 [ 1265 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_514_103 to 202410_515_520_1 2024.10.22 01:32:36.577065 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 815869 2024.10.22 01:32:36.587961 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(1ca58f6febc896266271404bca92c4dc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1ca58f6febc896266271404bca92c4dc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:32:36.591831 [ 1265 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:36.596161 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 319372, file segment info: File segment: [0, 33554431], key: 1ca58f6febc896266271404bca92c4dc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:32:36.597797 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:36.601657 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:32:36.610812 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_911_911_0 (state Temporary) 2024.10.22 01:32:36.630213 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_105_109_1} MergeTask::MergeProjectionsStage: Merge sorted 204 rows, containing 679 columns (679 merged, 0 gathered) in 76.749509585 sec., 2.6579974400236424 rows/sec., 4.04 KiB/sec. 2024.10.22 01:32:36.658987 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_520_104} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_514_103 to 202410_515_520_1 into Wide with storage Full 2024.10.22 01:32:36.661652 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_911_911_0 to 202410_924_924_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:36.681220 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_520_104} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:36.691934 [ 1155 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_918_466 to 202410_919_919_0 2024.10.22 01:32:36.699100 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_520_104} MergeTreeSequentialSource: Reading 101 marks from part 202410_1_514_103, total 814440 rows starting from the beginning of the part 2024.10.22 01:32:36.710779 [ 1155 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.18 MiB using storage policy from min volume index 0 2024.10.22 01:32:36.714545 [ 1155 ] {} DiskLocal: Reserved 2.18 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:32:36.721146 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60411 2024.10.22 01:32:36.759874 [ 1688 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/asy/oqeiodzqtntfufunzpqutsdtvjusn, size 319372 2024.10.22 01:32:36.763333 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_919_467} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_918_466 to 202410_919_919_0 into Compact with storage Full 2024.10.22 01:32:36.878923 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_520_104} MergeTreeSequentialSource: Reading 3 marks from part 202410_515_520_1, total 8734 rows starting from the beginning of the part 2024.10.22 01:32:36.905181 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_919_467} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:36.926365 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_919_467} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_918_466, total 60903 rows starting from the beginning of the part 2024.10.22 01:32:36.958814 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_919_467} MergeTreeSequentialSource: Reading 2 marks from part 202410_919_919_0, total 74 rows starting from the beginning of the part 2024.10.22 01:32:36.992222 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 454 2024.10.22 01:32:37.254081 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:32:37.269186 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:32:37.279229 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:32:37.292846 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:32:37.425180 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:32:37.459322 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:32:37.472464 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:32:37.488147 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:37.496701 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:37.500010 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 01:32:37.594515 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_436_436_0 (state Temporary) 2024.10.22 01:32:37.683416 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tgp/wzkdrovmvejmxepgesbrcbyeryhbk (LogSeriesLimiter: on interval from 2024-10-22 01:32:26 to 2024-10-22 01:32:37 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:32:37.837166 [ 1252 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:32:37.886722 [ 1252 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:32:37.893725 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_436_436_0 to 202410_446_446_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:37.972462 [ 1157 ] {} DatabaseCatalog: Execution took 432 ms. 2024.10.22 01:32:38.001298 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_393_81, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0, 202410_397_397_0, 202410_398_398_0] 2024.10.22 01:32:38.005483 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_393_81, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0, 202410_397_397_0, 202410_398_398_0] 2024.10.22 01:32:38.075128 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_393_81, 202410_394_394_0, 202410_395_395_0, 202410_396_396_0, 202410_397_397_0, 202410_398_398_0] 2024.10.22 01:32:38.096194 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 454 2024.10.22 01:32:38.649491 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_520_104/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:32:38.601284 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 8337 2024.10.22 01:32:38.868314 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 7.35 MiB, will set to 1.51 GiB (RSS), difference: -23.64 MiB 2024.10.22 01:32:39.109535 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:39.115470 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:39.172480 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_540_540_0 (state Temporary) 2024.10.22 01:32:39.270738 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_105_109_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:32:39.340598 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_105_109_1} MergedBlockOutputStream: filled checksums 202410_105_109_1 (state Temporary) 2024.10.22 01:32:39.549641 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_540_540_0 to 202410_553_553_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:39.635699 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8337 2024.10.22 01:32:40.340743 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_919_467} MergeTask::MergeProjectionsStage: Merge sorted 60977 rows, containing 13 columns (13 merged, 0 gathered) in 3.582070547 sec., 17022.836150189312 rows/sec., 2.71 MiB/sec. 2024.10.22 01:32:40.383711 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:32:40.385649 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:32:40.530581 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_919_467} MergedBlockOutputStream: filled checksums 202410_1_919_467 (state Temporary) 2024.10.22 01:32:40.691071 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_919_467} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_919_467 to 202410_1_919_467 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:40.698322 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_919_467} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_918_466, 202410_919_919_0] -> 202410_1_919_467 2024.10.22 01:32:40.733184 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:32:40.743650 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 60476 2024.10.22 01:32:41.168377 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:41.170542 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:41.426805 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_912_912_0 (state Temporary) 2024.10.22 01:32:41.666451 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 13.00 MiB, will set to 1.62 GiB (RSS), difference: -50.36 MiB 2024.10.22 01:32:41.678647 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_912_912_0 to 202410_925_925_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:41.717093 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_919_467 to 202410_920_920_0 2024.10.22 01:32:41.724119 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.18 MiB using storage policy from min volume index 0 2024.10.22 01:32:41.727494 [ 1235 ] {} DiskLocal: Reserved 2.18 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:41.834530 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_920_468} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_919_467 to 202410_920_920_0 into Compact with storage Full 2024.10.22 01:32:41.958481 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60476 2024.10.22 01:32:42.158901 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_920_468} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:42.210193 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_920_468} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_919_467, total 60977 rows starting from the beginning of the part 2024.10.22 01:32:42.354520 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_920_468} MergeTreeSequentialSource: Reading 2 marks from part 202410_920_920_0, total 62 rows starting from the beginning of the part 2024.10.22 01:32:42.692109 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 113 entries to flush up to offset 62153 2024.10.22 01:32:43.066197 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:43.071277 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:43.152775 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_530_530_0 (state Temporary) 2024.10.22 01:32:43.197301 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_530_530_0 to 202410_536_536_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:43.260726 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 62153 2024.10.22 01:32:43.556353 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 11.08 MiB, will set to 1.69 GiB (RSS), difference: -53.58 MiB 2024.10.22 01:32:43.580456 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1159 entries to flush up to offset 817028 2024.10.22 01:32:44.376883 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:44.379560 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:44.502485 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_518_518_0 (state Temporary) 2024.10.22 01:32:44.673738 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_518_518_0 to 202410_525_525_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:44.936052 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 817028 2024.10.22 01:32:45.187618 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_920_468} MergeTask::MergeProjectionsStage: Merge sorted 61039 rows, containing 13 columns (13 merged, 0 gathered) in 3.36330528 sec., 18148.516093073777 rows/sec., 2.89 MiB/sec. 2024.10.22 01:32:45.500323 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_920_468} MergedBlockOutputStream: filled checksums 202410_1_920_468 (state Temporary) 2024.10.22 01:32:45.553912 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 246.38 MiB using storage policy from min volume index 0 2024.10.22 01:32:45.603335 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 455 2024.10.22 01:32:45.628103 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 10.64 MiB, will set to 1.72 GiB (RSS), difference: 3.64 MiB 2024.10.22 01:32:45.974667 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 60509 2024.10.22 01:32:45.972078 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_920_468} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_920_468 to 202410_1_920_468 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:46.079827 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bdx/hfaizxyftgwyickmtqpdickafcmwi (LogSeriesLimiter: on interval from 2024-10-22 01:32:37 to 2024-10-22 01:32:46 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:32:46.082872 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_920_468} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_919_467, 202410_920_920_0] -> 202410_1_920_468 2024.10.22 01:32:46.108846 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:46.114344 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:46.138831 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:32:46.197567 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_437_437_0 (state Temporary) 2024.10.22 01:32:46.227018 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:46.236599 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:46.375071 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_913_913_0 (state Temporary) 2024.10.22 01:32:46.421807 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_913_913_0 to 202410_926_926_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:46.460892 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_920_468 to 202410_921_921_0 2024.10.22 01:32:46.467090 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.18 MiB using storage policy from min volume index 0 2024.10.22 01:32:46.470580 [ 1267 ] {} DiskLocal: Reserved 2.18 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:46.480290 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60509 2024.10.22 01:32:46.490981 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_437_437_0 to 202410_447_447_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:46.497619 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_921_469} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_920_468 to 202410_921_921_0 into Compact with storage Full 2024.10.22 01:32:46.561619 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_921_469} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:46.672455 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_921_469} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_920_468, total 61039 rows starting from the beginning of the part 2024.10.22 01:32:46.691059 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 455 2024.10.22 01:32:46.835740 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_921_469} MergeTreeSequentialSource: Reading 2 marks from part 202410_921_921_0, total 40 rows starting from the beginning of the part 2024.10.22 01:32:46.956188 [ 1677 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/cmu/avugrslubmlwktkrgwstsfbyddaxh, size 2823753 2024.10.22 01:32:47.139463 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 8349 2024.10.22 01:32:47.211968 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:32:47.222387 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:32:47.358794 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:32:47.458795 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:47.460542 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:47.496101 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:32:47.524504 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:32:47.546958 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:32:47.573464 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:32:47.598419 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:32:47.602368 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_541_541_0 (state Temporary) 2024.10.22 01:32:47.628784 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:32:47.665884 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_541_541_0 to 202410_554_554_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:47.735771 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8349 2024.10.22 01:32:47.782283 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.81 GiB, peak 1.97 GiB, free memory in arenas 10.15 MiB, will set to 1.71 GiB (RSS), difference: -102.13 MiB 2024.10.22 01:32:49.983434 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.87 GiB, peak 1.97 GiB, free memory in arenas 12.86 MiB, will set to 1.72 GiB (RSS), difference: -150.34 MiB 2024.10.22 01:32:50.383949 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:32:50.403570 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:32:50.487046 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 60551 2024.10.22 01:32:50.718557 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:50.724631 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:50.787018 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_914_914_0 (state Temporary) 2024.10.22 01:32:50.913387 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 125 entries to flush up to offset 62278 2024.10.22 01:32:50.963812 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_921_469} MergeTask::MergeProjectionsStage: Merge sorted 61079 rows, containing 13 columns (13 merged, 0 gathered) in 4.474610161 sec., 13650.127676452124 rows/sec., 2.17 MiB/sec. 2024.10.22 01:32:50.972030 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_914_914_0 to 202410_927_927_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:51.051805 [ 1188 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_922_922_0 to 202410_927_927_0 2024.10.22 01:32:51.059308 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60551 2024.10.22 01:32:51.063458 [ 1188 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:51.072344 [ 1188 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:51.081787 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fwj/surpufytgzkubqwceeneuhoapuuhl (LogSeriesLimiter: on interval from 2024-10-22 01:32:46 to 2024-10-22 01:32:51 accepted series 1 / 194 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:32:51.127185 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_921_469} MergedBlockOutputStream: filled checksums 202410_1_921_469 (state Temporary) 2024.10.22 01:32:51.228171 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_520_104/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:32:51.232345 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_922_922_0 to 202410_927_927_0 into Compact with storage Full 2024.10.22 01:32:51.256948 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:32:51.285403 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:51.404159 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_922_922_0, total 80 rows starting from the beginning of the part 2024.10.22 01:32:51.435897 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_923_923_0, total 38 rows starting from the beginning of the part 2024.10.22 01:32:51.441238 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:51.451000 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:51.449520 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_921_469} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_921_469 to 202410_1_921_469 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:51.478194 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_921_469} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_920_468, 202410_921_921_0] -> 202410_1_921_469 2024.10.22 01:32:51.479113 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_924_924_0, total 45 rows starting from the beginning of the part 2024.10.22 01:32:51.515983 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_925_925_0, total 65 rows starting from the beginning of the part 2024.10.22 01:32:51.530836 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:32:51.547242 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_926_926_0, total 33 rows starting from the beginning of the part 2024.10.22 01:32:51.587256 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_927_927_0, total 42 rows starting from the beginning of the part 2024.10.22 01:32:51.603448 [ 1159 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 329 ms. 2024.10.22 01:32:51.659825 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_531_531_0 (state Temporary) 2024.10.22 01:32:51.859198 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_105_109_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:32:51.770967 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.80 GiB, peak 1.97 GiB, free memory in arenas 13.63 MiB, will set to 1.72 GiB (RSS), difference: -90.16 MiB 2024.10.22 01:32:51.940135 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_531_531_0 to 202410_537_537_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:51.942144 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 818568 2024.10.22 01:32:51.967805 [ 1274 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_532_5 to 202410_537_537_0 2024.10.22 01:32:51.989376 [ 1274 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:51.994642 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 62278 2024.10.22 01:32:52.023325 [ 1274 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:52.165434 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_532_5 to 202410_537_537_0 into Compact with storage Full 2024.10.22 01:32:52.323269 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_922_927_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:32:52.340296 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:52.362585 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_532_5, total 2636 rows starting from the beginning of the part 2024.10.22 01:32:52.424602 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_533_533_0, total 75 rows starting from the beginning of the part 2024.10.22 01:32:52.489218 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_534_534_0, total 91 rows starting from the beginning of the part 2024.10.22 01:32:52.600932 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:52.611186 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:52.693566 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_535_535_0, total 113 rows starting from the beginning of the part 2024.10.22 01:32:52.760898 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_536_536_0, total 113 rows starting from the beginning of the part 2024.10.22 01:32:52.840542 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_537_537_0, total 125 rows starting from the beginning of the part 2024.10.22 01:32:52.893435 [ 1266 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:32:52.925605 [ 1266 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:32:53.175741 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} MergeTask::MergeProjectionsStage: Merge sorted 303 rows, containing 13 columns (13 merged, 0 gathered) in 1.966691977 sec., 154.06581383537113 rows/sec., 25.44 KiB/sec. 2024.10.22 01:32:53.097759 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_519_519_0 (state Temporary) 2024.10.22 01:32:53.260769 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} MergedBlockOutputStream: filled checksums 202410_922_927_1 (state Temporary) 2024.10.22 01:32:53.303607 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_519_519_0 to 202410_526_526_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:53.357132 [ 1275 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_521_521_0 to 202410_526_526_0 2024.10.22 01:32:53.364166 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 818568 2024.10.22 01:32:53.376659 [ 1275 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:53.381323 [ 1275 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:53.518695 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_521_521_0 to 202410_526_526_0 into Compact with storage Full 2024.10.22 01:32:53.561228 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:53.570281 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_521_521_0, total 1276 rows starting from the beginning of the part 2024.10.22 01:32:53.589091 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_522_522_0, total 1759 rows starting from the beginning of the part 2024.10.22 01:32:53.620297 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_523_523_0, total 1708 rows starting from the beginning of the part 2024.10.22 01:32:53.769199 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_524_524_0, total 1542 rows starting from the beginning of the part 2024.10.22 01:32:53.772565 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_817_420, 202410_818_823_1] 2024.10.22 01:32:53.774825 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_817_420, 202410_818_823_1] 2024.10.22 01:32:53.785992 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_817_420, 202410_818_823_1] 2024.10.22 01:32:53.838573 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_525_525_0, total 1159 rows starting from the beginning of the part 2024.10.22 01:32:53.897151 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.86 GiB, peak 1.97 GiB, free memory in arenas 13.09 MiB, will set to 1.71 GiB (RSS), difference: -145.52 MiB 2024.10.22 01:32:53.965760 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_922_927_1 to 202410_922_927_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:53.969156 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_922_927_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_922_922_0, 202410_927_927_0] -> 202410_922_927_1 2024.10.22 01:32:54.014969 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_526_526_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:32:54.130471 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:32:54.197376 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 456 2024.10.22 01:32:54.315003 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_105_109_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_105_109_1 to 202410_105_109_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:54.320889 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_105_109_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 5 parts: [202410_105_105_0, 202410_109_109_0] -> 202410_105_109_1 2024.10.22 01:32:54.475907 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:54.479499 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:54.567311 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_438_438_0 (state Temporary) 2024.10.22 01:32:54.836638 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_438_438_0 to 202410_448_448_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:54.880381 [ 1190 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_443_91 to 202410_448_448_0 2024.10.22 01:32:54.924740 [ 1190 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:54.932599 [ 1190 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:54.940336 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 456 2024.10.22 01:32:55.021177 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.75 MiB. 2024.10.22 01:32:55.039823 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_443_91 to 202410_448_448_0 into Compact with storage Full 2024.10.22 01:32:55.065047 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 60638 2024.10.22 01:32:55.204523 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:32:55.234304 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_443_91, total 487 rows starting from the beginning of the part 2024.10.22 01:32:55.242957 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 8370 2024.10.22 01:32:55.309048 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_444_444_0, total 1 rows starting from the beginning of the part 2024.10.22 01:32:55.472234 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_445_445_0, total 1 rows starting from the beginning of the part 2024.10.22 01:32:55.477659 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:55.481510 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:55.560108 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_915_915_0 (state Temporary) 2024.10.22 01:32:55.583702 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:55.596414 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:55.593452 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} MergeTask::MergeProjectionsStage: Merge sorted 3153 rows, containing 13 columns (13 merged, 0 gathered) in 3.430806982 sec., 919.0257617354937 rows/sec., 407.86 KiB/sec. 2024.10.22 01:32:55.633286 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:55.640004 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:55.655655 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_446_446_0, total 1 rows starting from the beginning of the part 2024.10.22 01:32:55.687355 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_542_542_0 (state Temporary) 2024.10.22 01:32:55.798915 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} MergedBlockOutputStream: filled checksums 202410_507_537_6 (state Temporary) 2024.10.22 01:32:55.821026 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_447_447_0, total 1 rows starting from the beginning of the part 2024.10.22 01:32:55.831788 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} MergeTask::MergeProjectionsStage: Merge sorted 8984 rows, containing 4 columns (4 merged, 0 gathered) in 2.322280953 sec., 3868.610293855345 rows/sec., 60.45 KiB/sec. 2024.10.22 01:32:55.856292 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_542_542_0 to 202410_555_555_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:55.870451 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_448_448_0, total 1 rows starting from the beginning of the part 2024.10.22 01:32:55.875328 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_915_915_0 to 202410_928_928_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:55.893089 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8370 2024.10.22 01:32:55.900248 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} MergedBlockOutputStream: filled checksums 202410_521_526_1 (state Temporary) 2024.10.22 01:32:55.963528 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60638 2024.10.22 01:32:55.975204 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_537_6 to 202410_507_537_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:55.978113 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_537_6} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_532_5, 202410_537_537_0] -> 202410_507_537_6 2024.10.22 01:32:56.056493 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_521_526_1 to 202410_521_526_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:56.059112 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.74 MiB. 2024.10.22 01:32:56.060133 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_521_526_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_521_521_0, 202410_526_526_0] -> 202410_521_526_1 2024.10.22 01:32:56.234951 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:32:56.824930 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.97 GiB, free memory in arenas 11.33 MiB, will set to 1.72 GiB (RSS), difference: -60.27 MiB 2024.10.22 01:32:57.223246 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a0b3152a1f704865954fabbbe48809de) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a0b3152a1f704865954fabbbe48809de, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 01:32:57.226527 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 833028, file segment info: File segment: [0, 33554431], key: a0b3152a1f704865954fabbbe48809de, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 01:32:57.235053 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:32:57.403233 [ 1738 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tgp/wzkdrovmvejmxepgesbrcbyeryhbk, size 833028 2024.10.22 01:32:57.648863 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:32:57.668205 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:32:57.700335 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} MergeTask::MergeProjectionsStage: Merge sorted 492 rows, containing 32 columns (32 merged, 0 gathered) in 2.671451613 sec., 184.16953449794713 rows/sec., 17.05 KiB/sec. 2024.10.22 01:32:57.775834 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:32:57.788905 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:32:57.809293 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:32:57.876063 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} MergedBlockOutputStream: filled checksums 202410_1_448_92 (state Temporary) 2024.10.22 01:32:57.947250 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:32:57.962063 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:32:57.985548 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column NetMajor 2024.10.22 01:32:58.150156 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/dou/huaohdngpzsqnoriwwknuohapeaer (LogSeriesLimiter: on interval from 2024-10-22 01:32:51 to 2024-10-22 01:32:58 accepted series 1 / 167 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:32:58.234440 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_823_421, 202410_824_824_0] 2024.10.22 01:32:58.238543 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_823_421, 202410_824_824_0] 2024.10.22 01:32:58.254709 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_823_421, 202410_824_824_0] 2024.10.22 01:32:58.422663 [ 1173 ] {} DatabaseCatalog: Execution took 313 ms. 2024.10.22 01:32:58.721412 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 12.27 MiB, will set to 1.72 GiB (RSS), difference: -6.63 MiB 2024.10.22 01:32:59.163183 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_448_92 to 202410_1_448_92 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:59.180656 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_448_92} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_443_91, 202410_448_448_0] -> 202410_1_448_92 2024.10.22 01:32:59.359240 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:32:59.514671 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 120 entries to flush up to offset 62398 2024.10.22 01:32:59.681111 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_109_109_0 (state Temporary) 2024.10.22 01:32:59.797651 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:32:59.804349 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:32:59.865755 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_532_532_0 (state Temporary) 2024.10.22 01:32:59.925468 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_532_532_0 to 202410_538_538_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:32:59.976111 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 60699 2024.10.22 01:33:00.075953 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 62398 2024.10.22 01:33:00.177989 [ 1180 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 221 ms. 2024.10.22 01:33:00.370354 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 820108 2024.10.22 01:33:00.386962 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:33:00.389407 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:33:00.427429 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 13.43 MiB, will set to 1.73 GiB (RSS), difference: 5.98 MiB 2024.10.22 01:33:00.664015 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:00.666866 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:01.035482 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_916_916_0 (state Temporary) 2024.10.22 01:33:01.170744 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_916_916_0 to 202410_929_929_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:01.236568 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60699 2024.10.22 01:33:01.539381 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_109_109_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:33:01.752136 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_491_98, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:33:01.756258 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_491_98, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:33:01.796998 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_491_98, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:33:02.007075 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:02.112457 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:02.383303 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_520_520_0 (state Temporary) 2024.10.22 01:33:02.384335 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_824_422, 202410_825_825_0] 2024.10.22 01:33:02.407832 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_824_422, 202410_825_825_0] 2024.10.22 01:33:02.429487 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_824_422, 202410_825_825_0] 2024.10.22 01:33:02.452719 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 457 2024.10.22 01:33:02.625675 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 14.18 MiB, will set to 1.74 GiB (RSS), difference: -9.62 MiB 2024.10.22 01:33:02.655026 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_520_520_0 to 202410_527_527_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:02.699799 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 820108 2024.10.22 01:33:02.724004 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_520_104/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:33:02.792986 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:02.796370 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:02.903851 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_439_439_0 (state Temporary) 2024.10.22 01:33:03.213165 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_520_104} MergeTask::MergeProjectionsStage: Merge sorted 823174 rows, containing 4 columns (4 merged, 0 gathered) in 26.563073698 sec., 30989.410689395438 rows/sec., 464.61 KiB/sec. 2024.10.22 01:33:03.370940 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_439_439_0 to 202410_449_449_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:03.400432 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 8397 2024.10.22 01:33:03.560637 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_520_104} MergedBlockOutputStream: filled checksums 202410_1_520_104 (state Temporary) 2024.10.22 01:33:03.571672 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 457 2024.10.22 01:33:03.613407 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:03.620410 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:03.766247 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_543_543_0 (state Temporary) 2024.10.22 01:33:03.826475 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_543_543_0 to 202410_556_556_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:03.873767 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8397 2024.10.22 01:33:03.967274 [ 1168 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_551_110 to 202410_556_556_0 2024.10.22 01:33:03.975868 [ 1168 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:03.977231 [ 1168 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:03.997233 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_551_110 to 202410_556_556_0 into Compact with storage Full 2024.10.22 01:33:04.052519 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:33:04.063264 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_551_110, total 8416 rows starting from the beginning of the part 2024.10.22 01:33:04.084620 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_552_552_0, total 15 rows starting from the beginning of the part 2024.10.22 01:33:04.116126 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_553_553_0, total 12 rows starting from the beginning of the part 2024.10.22 01:33:04.156563 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_554_554_0, total 12 rows starting from the beginning of the part 2024.10.22 01:33:04.221240 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_555_555_0, total 21 rows starting from the beginning of the part 2024.10.22 01:33:04.266565 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_556_556_0, total 27 rows starting from the beginning of the part 2024.10.22 01:33:04.394084 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 13.22 MiB, will set to 1.75 GiB (RSS), difference: 18.21 MiB 2024.10.22 01:33:05.266724 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 60751 2024.10.22 01:33:05.679773 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:05.683518 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:05.801734 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_917_917_0 (state Temporary) 2024.10.22 01:33:05.854620 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_917_917_0 to 202410_930_930_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:05.913691 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60751 2024.10.22 01:33:05.910709 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_520_104} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_520_104 to 202410_1_520_104 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:05.940487 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_520_104} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_514_103, 202410_515_520_1] -> 202410_1_520_104 2024.10.22 01:33:05.963895 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:33:06.262598 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} MergeTask::MergeProjectionsStage: Merge sorted 8503 rows, containing 12 columns (12 merged, 0 gathered) in 2.282051439 sec., 3726.033451606171 rows/sec., 592.41 KiB/sec. 2024.10.22 01:33:06.288266 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} MergedBlockOutputStream: filled checksums 202410_1_556_111 (state Temporary) 2024.10.22 01:33:06.536127 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 7.96 MiB, will set to 1.75 GiB (RSS), difference: 1.74 MiB 2024.10.22 01:33:06.641021 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_556_111 to 202410_1_556_111 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:06.747076 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_556_111} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_551_110, 202410_556_556_0] -> 202410_1_556_111 2024.10.22 01:33:07.002707 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:33:07.017391 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.59 MiB. 2024.10.22 01:33:07.039782 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:33:07.059730 [ 1229 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 258 ms. 2024.10.22 01:33:07.061709 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:33:07.227029 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:33:07.251972 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:33:07.277533 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:33:07.321350 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_825_423, 202410_826_826_0] 2024.10.22 01:33:07.327146 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_825_423, 202410_826_826_0] 2024.10.22 01:33:07.349552 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_825_423, 202410_826_826_0] 2024.10.22 01:33:07.412459 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:33:07.429648 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:33:07.486255 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:33:07.584258 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nrj/nktopxjmlnmbrtmweacjtyfmfdvnc (LogSeriesLimiter: on interval from 2024-10-22 01:32:58 to 2024-10-22 01:33:07 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:33:07.606854 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 65 entries to flush up to offset 62463 2024.10.22 01:33:07.613332 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_109_109_0 to 202410_112_112_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:07.848537 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:07.854662 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:07.889036 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_533_533_0 (state Temporary) 2024.10.22 01:33:07.933673 [ 1248 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:33:07.964650 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_533_533_0 to 202410_539_539_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:08.087944 [ 1248 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:33:08.103658 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 62463 2024.10.22 01:33:08.509477 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 4.72 MiB, will set to 1.75 GiB (RSS), difference: 6.41 MiB 2024.10.22 01:33:08.683020 [ 1244 ] {} DatabaseCatalog: Execution took 258 ms. 2024.10.22 01:33:09.288466 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:33:09.707351 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 822033 2024.10.22 01:33:09.927638 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 60793 2024.10.22 01:33:10.197247 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:10.200590 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:10.256767 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_918_918_0 (state Temporary) 2024.10.22 01:33:10.386923 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:33:10.388424 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:33:10.517029 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4374 2024.10.22 01:33:10.576613 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 7.08 MiB, will set to 1.75 GiB (RSS), difference: 2.46 MiB 2024.10.22 01:33:10.616207 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_918_918_0 to 202410_931_931_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:10.664432 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60793 2024.10.22 01:33:10.973189 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:10.975542 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:11.074411 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 458 2024.10.22 01:33:11.107096 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_521_521_0 (state Temporary) 2024.10.22 01:33:11.379905 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 8412 2024.10.22 01:33:11.385374 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_521_521_0 to 202410_528_528_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:11.561716 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 822033 2024.10.22 01:33:11.564974 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:11.568555 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:11.668294 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_826_424, 202410_827_827_0] 2024.10.22 01:33:11.672552 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_826_424, 202410_827_827_0] 2024.10.22 01:33:11.684272 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_826_424, 202410_827_827_0] 2024.10.22 01:33:11.704583 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:11.710201 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:11.750855 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_440_440_0 (state Temporary) 2024.10.22 01:33:11.755374 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_544_544_0 (state Temporary) 2024.10.22 01:33:11.796071 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_544_544_0 to 202410_557_557_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:11.892843 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_461_476_3, 202410_477_477_0, 202410_478_478_0, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0] 2024.10.22 01:33:11.901227 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_461_476_3, 202410_477_477_0, 202410_478_478_0, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0] 2024.10.22 01:33:11.940949 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8412 2024.10.22 01:33:12.067538 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_440_440_0 to 202410_450_450_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:12.098492 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_461_476_3, 202410_477_477_0, 202410_478_478_0, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0] 2024.10.22 01:33:12.155634 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 458 2024.10.22 01:33:12.545683 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 7.01 MiB, will set to 1.75 GiB (RSS), difference: 8.25 MiB 2024.10.22 01:33:14.661173 [ 1674 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fwj/surpufytgzkubqwceeneuhoapuuhl, size 171 2024.10.22 01:33:14.669430 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 60828 2024.10.22 01:33:14.833597 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 6.82 MiB, will set to 1.76 GiB (RSS), difference: 62.28 MiB 2024.10.22 01:33:15.171785 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:15.176146 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:15.330482 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_919_919_0 (state Temporary) 2024.10.22 01:33:15.339498 [ 1275 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 01:33:15.470988 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_919_919_0 to 202410_932_932_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:15.563657 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_921_469 to 202410_922_927_1 2024.10.22 01:33:15.628167 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 56 entries to flush up to offset 62519 2024.10.22 01:33:15.684579 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60828 2024.10.22 01:33:15.712072 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.20 MiB using storage policy from min volume index 0 2024.10.22 01:33:15.726727 [ 1195 ] {} DiskLocal: Reserved 2.20 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:33:15.730191 [ 1195 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:33:15.768711 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_927_470} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_921_469 to 202410_922_927_1 into Compact with storage Full 2024.10.22 01:33:15.925417 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_927_470} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:33:15.967391 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_927_470} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_921_469, total 61079 rows starting from the beginning of the part 2024.10.22 01:33:16.083022 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:16.086264 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 01:33:16.252122 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_534_534_0 (state Temporary) 2024.10.22 01:33:16.271655 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_927_470} MergeTreeSequentialSource: Reading 2 marks from part 202410_922_927_1, total 303 rows starting from the beginning of the part 2024.10.22 01:33:16.390439 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_534_534_0 to 202410_540_540_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:16.571239 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 62519 2024.10.22 01:33:17.014032 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_464_93, 202410_465_465_0, 202410_466_466_0, 202410_467_467_0, 202410_468_468_0, 202410_469_469_0] 2024.10.22 01:33:17.032478 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_464_93, 202410_465_465_0, 202410_466_466_0, 202410_467_467_0, 202410_468_468_0, 202410_469_469_0] 2024.10.22 01:33:17.182172 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_464_93, 202410_465_465_0, 202410_466_466_0, 202410_467_467_0, 202410_468_468_0, 202410_469_469_0] 2024.10.22 01:33:17.726627 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.97 GiB, free memory in arenas 6.86 MiB, will set to 1.75 GiB (RSS), difference: 187.90 MiB 2024.10.22 01:33:17.886268 [ 1253 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 268 ms. 2024.10.22 01:33:18.127983 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_827_425, 202410_828_828_0] 2024.10.22 01:33:18.150344 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_827_425, 202410_828_828_0] 2024.10.22 01:33:18.168610 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_827_425, 202410_828_828_0] 2024.10.22 01:33:18.602771 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1419 entries to flush up to offset 823452 2024.10.22 01:33:18.649685 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_5_5_0 (state Temporary) 2024.10.22 01:33:18.656208 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 4417 2024.10.22 01:33:18.667732 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ypg/gvedqenyzbwiaczbbxpmsdmomfrsh (LogSeriesLimiter: on interval from 2024-10-22 01:33:07 to 2024-10-22 01:33:18 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:33:18.738580 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 10 file segments (while reserving 6788 bytes for af494e564cb714c49057c3683e59dd37:0) 2024.10.22 01:33:18.741642 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 935dd725c9f38485e2b04652a0c4a998, offset: 0, size: 248 2024.10.22 01:33:18.752117 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 935dd725c9f38485e2b04652a0c4a998 for removal 2024.10.22 01:33:18.755089 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 391b0e4beb5a7f4102b2231100305f0a, offset: 0, size: 252 2024.10.22 01:33:18.755420 [ 1476 ] {} CacheMetadata: Key 935dd725c9f38485e2b04652a0c4a998 is removed from metadata 2024.10.22 01:33:18.758219 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 391b0e4beb5a7f4102b2231100305f0a for removal 2024.10.22 01:33:18.760858 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0fa5a55fc4b09815b26dac3ac967ef6a, offset: 0, size: 246 2024.10.22 01:33:18.766693 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0fa5a55fc4b09815b26dac3ac967ef6a for removal 2024.10.22 01:33:18.781054 [ 1690 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bdx/hfaizxyftgwyickmtqpdickafcmwi, size 249 2024.10.22 01:33:18.782023 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2e105430eddf0ea465974d2a298e7bd9, offset: 0, size: 250 2024.10.22 01:33:18.785458 [ 1476 ] {} CacheMetadata: Key 0fa5a55fc4b09815b26dac3ac967ef6a is removed from metadata 2024.10.22 01:33:18.785755 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2e105430eddf0ea465974d2a298e7bd9 for removal 2024.10.22 01:33:18.790470 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 832ae378b5d8d2c344d23d0d01959304, offset: 0, size: 254 2024.10.22 01:33:18.796023 [ 1476 ] {} CacheMetadata: Key 2e105430eddf0ea465974d2a298e7bd9 is removed from metadata 2024.10.22 01:33:18.796037 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 832ae378b5d8d2c344d23d0d01959304 for removal 2024.10.22 01:33:18.808024 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7f24e87e9b8357a6a3f6441245b7867c, offset: 0, size: 248 2024.10.22 01:33:18.815443 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7f24e87e9b8357a6a3f6441245b7867c for removal 2024.10.22 01:33:18.815040 [ 1476 ] {} CacheMetadata: Key 832ae378b5d8d2c344d23d0d01959304 is removed from metadata 2024.10.22 01:33:18.823248 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e0bc876552658e808b8967979292471f, offset: 0, size: 4 2024.10.22 01:33:18.826523 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e0bc876552658e808b8967979292471f for removal 2024.10.22 01:33:18.828709 [ 1476 ] {} CacheMetadata: Key 7f24e87e9b8357a6a3f6441245b7867c is removed from metadata 2024.10.22 01:33:18.835417 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 18984a1df0ab10136932d1df2edb6fe5, offset: 0, size: 249 2024.10.22 01:33:18.843717 [ 1476 ] {} CacheMetadata: Key e0bc876552658e808b8967979292471f is removed from metadata 2024.10.22 01:33:18.844132 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 18984a1df0ab10136932d1df2edb6fe5 for removal 2024.10.22 01:33:18.850813 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d5e3be4523e0f27214b7e4b04dcb36f1, offset: 0, size: 249 2024.10.22 01:33:18.855062 [ 1476 ] {} CacheMetadata: Key 18984a1df0ab10136932d1df2edb6fe5 is removed from metadata 2024.10.22 01:33:18.857333 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d5e3be4523e0f27214b7e4b04dcb36f1 for removal 2024.10.22 01:33:18.862272 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 230a794dc7fb4e99c7d96b49c1661985, offset: 0, size: 250 2024.10.22 01:33:18.864284 [ 1476 ] {} CacheMetadata: Key d5e3be4523e0f27214b7e4b04dcb36f1 is removed from metadata 2024.10.22 01:33:18.867442 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 230a794dc7fb4e99c7d96b49c1661985 for removal 2024.10.22 01:33:18.873428 [ 1476 ] {} CacheMetadata: Key 230a794dc7fb4e99c7d96b49c1661985 is removed from metadata 2024.10.22 01:33:18.902494 [ 1675 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ypg/gvedqenyzbwiaczbbxpmsdmomfrsh, size 4 2024.10.22 01:33:18.986469 [ 1476 ] {} CacheMetadata: Key 391b0e4beb5a7f4102b2231100305f0a is removed from metadata 2024.10.22 01:33:19.310700 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 3.80 MiB, will set to 1.72 GiB (RSS), difference: 73.68 MiB 2024.10.22 01:33:19.443314 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 8417 2024.10.22 01:33:19.470220 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:19.472497 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:19.476945 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_927_470/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:33:19.548442 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:19.551290 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:19.687483 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_522_522_0 (state Temporary) 2024.10.22 01:33:19.707448 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_545_545_0 (state Temporary) 2024.10.22 01:33:19.710121 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 60903 2024.10.22 01:33:19.759509 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_545_545_0 to 202410_558_558_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:19.769456 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_522_522_0 to 202410_529_529_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:19.786864 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8417 2024.10.22 01:33:19.851790 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 702 bytes for dc6289cb7b6670de21c509fff217dd53:0) 2024.10.22 01:33:19.857367 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e842e3ae10bdc2b58cbfb094a961b2c2, offset: 0, size: 240 2024.10.22 01:33:19.863249 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e842e3ae10bdc2b58cbfb094a961b2c2 for removal 2024.10.22 01:33:19.866176 [ 1476 ] {} CacheMetadata: Key e842e3ae10bdc2b58cbfb094a961b2c2 is removed from metadata 2024.10.22 01:33:19.866987 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: af2075fc45ce6320e295114cbfa82195, offset: 0, size: 247 2024.10.22 01:33:19.869218 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key af2075fc45ce6320e295114cbfa82195 for removal 2024.10.22 01:33:19.871956 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 29dec290f40e7537d33249949097050b, offset: 0, size: 247 2024.10.22 01:33:19.872995 [ 1476 ] {} CacheMetadata: Key af2075fc45ce6320e295114cbfa82195 is removed from metadata 2024.10.22 01:33:19.875239 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 29dec290f40e7537d33249949097050b for removal 2024.10.22 01:33:19.876650 [ 1476 ] {} CacheMetadata: Key 29dec290f40e7537d33249949097050b is removed from metadata 2024.10.22 01:33:19.888179 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 823452 2024.10.22 01:33:20.063673 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:20.071743 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:20.138349 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_920_920_0 (state Temporary) 2024.10.22 01:33:20.292103 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rgq/serwynnthjdrcjnwtgkzdviepxkpe. 2024.10.22 01:33:20.311658 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_920_920_0 to 202410_933_933_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:20.338032 [ 1169 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_928_928_0 to 202410_933_933_0 2024.10.22 01:33:20.367751 [ 1169 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:20.369252 [ 1169 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:20.386776 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:33:20.390542 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:33:20.395870 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_928_928_0 to 202410_933_933_0 into Compact with storage Full 2024.10.22 01:33:20.549340 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60903 2024.10.22 01:33:20.727482 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:33:20.762486 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_928_928_0, total 87 rows starting from the beginning of the part 2024.10.22 01:33:20.821004 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_927_470} MergeTask::MergeProjectionsStage: Merge sorted 61382 rows, containing 13 columns (13 merged, 0 gathered) in 5.056902357 sec., 12138.260869330841 rows/sec., 1.93 MiB/sec. 2024.10.22 01:33:20.889287 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_927_470} MergedBlockOutputStream: filled checksums 202410_1_927_470 (state Temporary) 2024.10.22 01:33:20.890933 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_929_929_0, total 61 rows starting from the beginning of the part 2024.10.22 01:33:20.960665 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_930_930_0, total 52 rows starting from the beginning of the part 2024.10.22 01:33:21.011852 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 9 file segments (while reserving 35590 bytes for 5771fd256ae06936f0c21c5de6a91af9:0) 2024.10.22 01:33:21.015336 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4542d21fb45c8278b5b4ec1f3d4ca073, offset: 0, size: 212934 2024.10.22 01:33:21.020115 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4542d21fb45c8278b5b4ec1f3d4ca073 for removal 2024.10.22 01:33:21.024237 [ 1476 ] {} CacheMetadata: Key 4542d21fb45c8278b5b4ec1f3d4ca073 is removed from metadata 2024.10.22 01:33:21.028011 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_931_931_0, total 42 rows starting from the beginning of the part 2024.10.22 01:33:21.026945 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d849c39f96ba59df40c5ae209acd67a5, offset: 0, size: 167 2024.10.22 01:33:21.048061 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d849c39f96ba59df40c5ae209acd67a5 for removal 2024.10.22 01:33:21.053001 [ 1476 ] {} CacheMetadata: Key d849c39f96ba59df40c5ae209acd67a5 is removed from metadata 2024.10.22 01:33:21.053753 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cc5f01ad205fd13e620bca4c61b76b11, offset: 0, size: 253 2024.10.22 01:33:21.057045 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cc5f01ad205fd13e620bca4c61b76b11 for removal 2024.10.22 01:33:21.060523 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 655f51ff2acf943301b2666480432d99, offset: 0, size: 245 2024.10.22 01:33:21.060595 [ 1476 ] {} CacheMetadata: Key cc5f01ad205fd13e620bca4c61b76b11 is removed from metadata 2024.10.22 01:33:21.064174 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 655f51ff2acf943301b2666480432d99 for removal 2024.10.22 01:33:21.069007 [ 1476 ] {} CacheMetadata: Key 655f51ff2acf943301b2666480432d99 is removed from metadata 2024.10.22 01:33:21.070371 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3cc79c87dbe01b2a0e8a1149619cf676, offset: 0, size: 2532 2024.10.22 01:33:21.073802 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3cc79c87dbe01b2a0e8a1149619cf676 for removal 2024.10.22 01:33:21.076357 [ 1476 ] {} CacheMetadata: Key 3cc79c87dbe01b2a0e8a1149619cf676 is removed from metadata 2024.10.22 01:33:21.076860 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: acfa992237f676a0ee1b5cd4f31289ee, offset: 0, size: 163 2024.10.22 01:33:21.077685 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_932_932_0, total 35 rows starting from the beginning of the part 2024.10.22 01:33:21.079840 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key acfa992237f676a0ee1b5cd4f31289ee for removal 2024.10.22 01:33:21.083149 [ 1476 ] {} CacheMetadata: Key acfa992237f676a0ee1b5cd4f31289ee is removed from metadata 2024.10.22 01:33:21.083769 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 30123df5bd41c7de4d553593e2bc75a5, offset: 0, size: 249 2024.10.22 01:33:21.092083 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 30123df5bd41c7de4d553593e2bc75a5 for removal 2024.10.22 01:33:21.101362 [ 1476 ] {} CacheMetadata: Key 30123df5bd41c7de4d553593e2bc75a5 is removed from metadata 2024.10.22 01:33:21.104935 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 72862123516d2e1a079bbb1708743b59, offset: 0, size: 243 2024.10.22 01:33:21.108880 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 72862123516d2e1a079bbb1708743b59 for removal 2024.10.22 01:33:21.114562 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d5b0d70413a19b86dfd2f0345d0f0e56, offset: 0, size: 249 2024.10.22 01:33:21.120742 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d5b0d70413a19b86dfd2f0345d0f0e56 for removal 2024.10.22 01:33:21.123228 [ 1476 ] {} CacheMetadata: Key 72862123516d2e1a079bbb1708743b59 is removed from metadata 2024.10.22 01:33:21.129061 [ 1476 ] {} CacheMetadata: Key d5b0d70413a19b86dfd2f0345d0f0e56 is removed from metadata 2024.10.22 01:33:21.167282 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_927_470} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_927_470 to 202410_1_927_470 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:21.175343 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_927_470} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_921_469, 202410_922_927_1] -> 202410_1_927_470 2024.10.22 01:33:21.180600 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_933_933_0, total 75 rows starting from the beginning of the part 2024.10.22 01:33:21.187193 [ 1717 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/dou/huaohdngpzsqnoriwwknuohapeaer, size 35590 2024.10.22 01:33:21.209708 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:33:21.562473 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:33:21.579916 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:33:21.594569 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:33:21.627543 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:33:21.662152 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:33:21.688371 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 3.38 MiB, will set to 1.72 GiB (RSS), difference: 10.34 MiB 2024.10.22 01:33:21.690360 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:33:21.750292 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:33:21.782779 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column NetMinor 2024.10.22 01:33:22.026883 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_828_426, 202410_829_829_0] 2024.10.22 01:33:22.029249 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_828_426, 202410_829_829_0] 2024.10.22 01:33:22.149159 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_828_426, 202410_829_829_0] 2024.10.22 01:33:23.095416 [ 1187 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:33:23.107445 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} MergeTask::MergeProjectionsStage: Merge sorted 352 rows, containing 13 columns (13 merged, 0 gathered) in 2.722187628 sec., 129.30776570262188 rows/sec., 20.35 KiB/sec. 2024.10.22 01:33:23.147916 [ 1187 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:33:23.153324 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} MergedBlockOutputStream: filled checksums 202410_928_933_1 (state Temporary) 2024.10.22 01:33:23.398852 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_928_933_1 to 202410_928_933_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:23.407397 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_928_933_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_928_928_0, 202410_933_933_0] -> 202410_928_933_1 2024.10.22 01:33:23.449813 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 3.16 MiB, will set to 1.71 GiB (RSS), difference: -3.76 MiB 2024.10.22 01:33:23.459238 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:33:24.080797 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 110 entries to flush up to offset 62629 2024.10.22 01:33:24.212194 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:24.217120 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:33:24.261453 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_535_535_0 (state Temporary) 2024.10.22 01:33:24.578690 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 60996 2024.10.22 01:33:24.713273 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_535_535_0 to 202410_541_541_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:24.767737 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:24.770528 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:33:24.784362 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 62629 2024.10.22 01:33:25.058025 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_921_921_0 (state Temporary) 2024.10.22 01:33:25.138563 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_921_921_0 to 202410_934_934_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:25.201392 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 60996 2024.10.22 01:33:25.635432 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 2.88 MiB, will set to 1.71 GiB (RSS), difference: -2.01 MiB 2024.10.22 01:33:26.897750 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1663 entries to flush up to offset 825115 2024.10.22 01:33:27.159285 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 459 2024.10.22 01:33:27.292483 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 8432 2024.10.22 01:33:27.460765 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:27.462992 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:33:27.525159 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:27.530763 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:27.573748 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_546_546_0 (state Temporary) 2024.10.22 01:33:27.654917 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 2.98 MiB, will set to 1.71 GiB (RSS), difference: -4.02 MiB 2024.10.22 01:33:27.722494 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_441_441_0 (state Temporary) 2024.10.22 01:33:27.773470 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:27.775796 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:27.934675 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_523_523_0 (state Temporary) 2024.10.22 01:33:27.953173 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_546_546_0 to 202410_559_559_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:27.985841 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_441_441_0 to 202410_451_451_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:28.048738 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_523_523_0 to 202410_530_530_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:28.086539 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 459 2024.10.22 01:33:28.091653 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 825115 2024.10.22 01:33:28.092484 [ 1230 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_520_104 to 202410_521_526_1 2024.10.22 01:33:28.109826 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8432 2024.10.22 01:33:28.111917 [ 1230 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:28.114268 [ 1230 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:33:28.181621 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_526_105} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_520_104 to 202410_521_526_1 into Wide with storage Full 2024.10.22 01:33:28.226689 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_526_105} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:33:28.241003 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_526_105} MergeTreeSequentialSource: Reading 102 marks from part 202410_1_520_104, total 823174 rows starting from the beginning of the part 2024.10.22 01:33:28.273380 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_526_105} MergeTreeSequentialSource: Reading 3 marks from part 202410_521_526_1, total 8984 rows starting from the beginning of the part 2024.10.22 01:33:29.026137 [ 1151 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:33:29.123404 [ 1221 ] {} DatabaseCatalog: Execution took 272 ms. 2024.10.22 01:33:29.195882 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_829_427, 202410_830_830_0] 2024.10.22 01:33:29.200809 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_829_427, 202410_830_830_0] 2024.10.22 01:33:29.213642 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_829_427, 202410_830_830_0] 2024.10.22 01:33:29.219483 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 61035 2024.10.22 01:33:29.494096 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_398_82, 202410_399_399_0, 202410_400_400_0, 202410_401_401_0, 202410_402_402_0, 202410_403_403_0] 2024.10.22 01:33:29.500664 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_398_82, 202410_399_399_0, 202410_400_400_0, 202410_401_401_0, 202410_402_402_0, 202410_403_403_0] 2024.10.22 01:33:29.513264 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 5.36 MiB, will set to 1.72 GiB (RSS), difference: -6.63 MiB 2024.10.22 01:33:29.666875 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:29.669930 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:33:29.688390 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_398_82, 202410_399_399_0, 202410_400_400_0, 202410_401_401_0, 202410_402_402_0, 202410_403_403_0] 2024.10.22 01:33:29.711556 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_922_922_0 (state Temporary) 2024.10.22 01:33:29.965112 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_922_922_0 to 202410_935_935_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:30.050448 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61035 2024.10.22 01:33:30.174583 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_526_105/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:33:30.386563 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:33:30.390865 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:33:31.195975 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_831_831_0, 202410_832_832_0, 202410_833_833_0, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0] 2024.10.22 01:33:31.200136 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_831_831_0, 202410_832_832_0, 202410_833_833_0, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0] 2024.10.22 01:33:31.268549 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_831_831_0, 202410_832_832_0, 202410_833_833_0, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0] 2024.10.22 01:33:31.427828 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 8.38 MiB, will set to 1.72 GiB (RSS), difference: 5.58 MiB 2024.10.22 01:33:32.410658 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 72 entries to flush up to offset 62701 2024.10.22 01:33:32.579952 [ 1739 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nrj/nktopxjmlnmbrtmweacjtyfmfdvnc, size 27651 2024.10.22 01:33:32.756882 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:32.759794 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:33:32.805100 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_536_536_0 (state Temporary) 2024.10.22 01:33:32.850588 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_536_536_0 to 202410_542_542_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:32.878087 [ 1158 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_537_6 to 202410_542_542_0 2024.10.22 01:33:32.892137 [ 1158 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:32.907725 [ 1158 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:33:32.908328 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 62701 2024.10.22 01:33:32.939258 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_537_6 to 202410_542_542_0 into Compact with storage Full 2024.10.22 01:33:33.055884 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:33:33.072043 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:33:33.106199 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:33:33.136542 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:33:33.137250 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:33:33.160655 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_537_6, total 3153 rows starting from the beginning of the part 2024.10.22 01:33:33.222825 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_538_538_0, total 120 rows starting from the beginning of the part 2024.10.22 01:33:33.276117 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_539_539_0, total 65 rows starting from the beginning of the part 2024.10.22 01:33:33.285251 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:33:33.307235 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:33:33.320092 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_540_540_0, total 56 rows starting from the beginning of the part 2024.10.22 01:33:33.328441 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:33:33.369084 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_541_541_0, total 110 rows starting from the beginning of the part 2024.10.22 01:33:33.422389 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_542_542_0, total 72 rows starting from the beginning of the part 2024.10.22 01:33:33.465130 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:33:33.494457 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:33:33.519255 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 11.38 MiB, will set to 1.72 GiB (RSS), difference: 4.96 MiB 2024.10.22 01:33:33.562997 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/msf/mwkvghgxfzrtyeajlfqafhcjqhilv (LogSeriesLimiter: on interval from 2024-10-22 01:33:18 to 2024-10-22 01:33:33 accepted series 1 / 12 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:33:34.061318 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 61082 2024.10.22 01:33:34.223916 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:34.229833 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:33:34.715009 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_923_923_0 (state Temporary) 2024.10.22 01:33:34.859362 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_923_923_0 to 202410_936_936_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:34.900093 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} MergeTask::MergeProjectionsStage: Merge sorted 3576 rows, containing 13 columns (13 merged, 0 gathered) in 1.972029727 sec., 1813.3600883593576 rows/sec., 803.30 KiB/sec. 2024.10.22 01:33:34.905285 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61082 2024.10.22 01:33:34.853452 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:33:34.974989 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} MergedBlockOutputStream: filled checksums 202410_507_542_7 (state Temporary) 2024.10.22 01:33:35.101703 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 826655 2024.10.22 01:33:35.268523 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_542_7 to 202410_507_542_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:35.279241 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_542_7} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_537_6, 202410_542_542_0] -> 202410_507_542_7 2024.10.22 01:33:35.326626 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.74 MiB. 2024.10.22 01:33:35.430559 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 10.27 MiB, will set to 1.73 GiB (RSS), difference: -4.22 MiB 2024.10.22 01:33:35.591972 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 460 2024.10.22 01:33:35.613217 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 8445 2024.10.22 01:33:35.893952 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:35.902476 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:33:35.962819 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:35.964006 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:35.980279 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_442_442_0 (state Temporary) 2024.10.22 01:33:36.022245 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_547_547_0 (state Temporary) 2024.10.22 01:33:36.106633 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:36.109307 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:36.195306 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_547_547_0 to 202410_560_560_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:36.263132 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_524_524_0 (state Temporary) 2024.10.22 01:33:36.263366 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8445 2024.10.22 01:33:36.264430 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_442_442_0 to 202410_452_452_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:36.318861 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_524_524_0 to 202410_531_531_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:36.437651 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 460 2024.10.22 01:33:36.462707 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 826655 2024.10.22 01:33:36.603945 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:36.610593 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:33:37.443459 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 7.75 MiB, will set to 1.72 GiB (RSS), difference: -4.90 MiB 2024.10.22 01:33:38.152901 [ 1259 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:33:38.181462 [ 1259 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:33:39.037034 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 61118 2024.10.22 01:33:39.352938 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:39.356728 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:39.486203 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 9.20 MiB, will set to 1.73 GiB (RSS), difference: -3.40 MiB 2024.10.22 01:33:39.515884 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_924_924_0 (state Temporary) 2024.10.22 01:33:39.664266 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_924_924_0 to 202410_937_937_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:39.728650 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61118 2024.10.22 01:33:40.010924 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:33:40.025596 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:33:40.045192 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:33:40.095148 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:33:40.131729 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:33:40.168437 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:33:40.182447 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:33:40.215475 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 01:33:40.390944 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:33:40.429186 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 77 entries to flush up to offset 62778 2024.10.22 01:33:40.453691 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:33:40.473601 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jbs/lblfzbrtclwkmjygreywcbbozksli (LogSeriesLimiter: on interval from 2024-10-22 01:33:33 to 2024-10-22 01:33:40 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:33:40.574973 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_110_110_0 (state Temporary) 2024.10.22 01:33:40.878689 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:40.882404 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:40.937431 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_537_537_0 (state Temporary) 2024.10.22 01:33:40.976536 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_526_105/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:33:41.043959 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_496_99, 202410_497_497_0, 202410_498_498_0, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0] 2024.10.22 01:33:41.073730 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_496_99, 202410_497_497_0, 202410_498_498_0, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0] 2024.10.22 01:33:41.202758 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_537_537_0 to 202410_543_543_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:41.221212 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_496_99, 202410_497_497_0, 202410_498_498_0, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0] 2024.10.22 01:33:41.288172 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 62778 2024.10.22 01:33:41.650499 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 9.71 MiB, will set to 1.73 GiB (RSS), difference: -5.62 MiB 2024.10.22 01:33:42.546309 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_110_110_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:33:43.465556 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 828195 2024.10.22 01:33:43.595575 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 11.81 MiB, will set to 1.73 GiB (RSS), difference: 2.69 MiB 2024.10.22 01:33:43.742262 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 61151 2024.10.22 01:33:43.765222 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 8450 2024.10.22 01:33:43.941816 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 461 2024.10.22 01:33:44.078791 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:44.087988 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:44.133370 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:44.141012 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:44.225169 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_925_925_0 (state Temporary) 2024.10.22 01:33:44.264081 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_548_548_0 (state Temporary) 2024.10.22 01:33:44.341501 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:44.352105 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:44.352482 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_548_548_0 to 202410_561_561_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:44.381570 [ 1185 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_556_111 to 202410_561_561_0 2024.10.22 01:33:44.403829 [ 1185 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:44.414982 [ 1185 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:44.416587 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8450 2024.10.22 01:33:44.529572 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_443_443_0 (state Temporary) 2024.10.22 01:33:44.538727 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_556_111 to 202410_561_561_0 into Compact with storage Full 2024.10.22 01:33:44.566455 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_925_925_0 to 202410_938_938_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:44.603700 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_927_470 to 202410_928_933_1 2024.10.22 01:33:44.621582 [ 1176 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.21 MiB using storage policy from min volume index 0 2024.10.22 01:33:44.625467 [ 1176 ] {} DiskLocal: Reserved 2.21 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:44.658360 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:33:44.707596 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_556_111, total 8503 rows starting from the beginning of the part 2024.10.22 01:33:44.726624 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_443_443_0 to 202410_453_453_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:44.788004 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_933_471} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_927_470 to 202410_928_933_1 into Compact with storage Full 2024.10.22 01:33:44.801530 [ 1201 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_448_92 to 202410_453_453_0 2024.10.22 01:33:44.863217 [ 1201 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:44.874938 [ 1201 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:44.883953 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_557_557_0, total 15 rows starting from the beginning of the part 2024.10.22 01:33:44.886631 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61151 2024.10.22 01:33:44.892420 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:44.901689 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:44.942949 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 461 2024.10.22 01:33:44.990798 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_933_471} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:33:45.026997 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_558_558_0, total 5 rows starting from the beginning of the part 2024.10.22 01:33:45.041664 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_448_92 to 202410_453_453_0 into Compact with storage Full 2024.10.22 01:33:45.053636 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_933_471} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_927_470, total 61382 rows starting from the beginning of the part 2024.10.22 01:33:45.166739 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_525_525_0 (state Temporary) 2024.10.22 01:33:45.223518 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:33:45.238390 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_448_92, total 492 rows starting from the beginning of the part 2024.10.22 01:33:45.257065 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_525_525_0 to 202410_532_532_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:45.290932 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_449_449_0, total 1 rows starting from the beginning of the part 2024.10.22 01:33:45.324558 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_933_471} MergeTreeSequentialSource: Reading 2 marks from part 202410_928_933_1, total 352 rows starting from the beginning of the part 2024.10.22 01:33:45.403908 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 828195 2024.10.22 01:33:45.392887 [ 1199 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_527_527_0 to 202410_532_532_0 2024.10.22 01:33:45.432492 [ 1199 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:45.437565 [ 1199 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:45.493215 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_559_559_0, total 15 rows starting from the beginning of the part 2024.10.22 01:33:45.515038 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_527_527_0 to 202410_532_532_0 into Compact with storage Full 2024.10.22 01:33:45.491698 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_450_450_0, total 1 rows starting from the beginning of the part 2024.10.22 01:33:45.573281 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_451_451_0, total 1 rows starting from the beginning of the part 2024.10.22 01:33:45.667714 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:33:45.683752 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_527_527_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:33:45.707769 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_560_560_0, total 13 rows starting from the beginning of the part 2024.10.22 01:33:45.752774 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_528_528_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:33:45.762513 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 10.10 MiB, will set to 1.73 GiB (RSS), difference: -5.98 MiB 2024.10.22 01:33:45.767947 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_452_452_0, total 1 rows starting from the beginning of the part 2024.10.22 01:33:45.773211 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_561_561_0, total 5 rows starting from the beginning of the part 2024.10.22 01:33:45.790833 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_529_529_0, total 1419 rows starting from the beginning of the part 2024.10.22 01:33:45.817512 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_453_453_0, total 1 rows starting from the beginning of the part 2024.10.22 01:33:45.832672 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_530_530_0, total 1663 rows starting from the beginning of the part 2024.10.22 01:33:45.877464 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_531_531_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:33:46.055184 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_532_532_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:33:46.072529 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_453_93/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:33:46.370915 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_933_471/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:33:47.328317 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} MergeTask::MergeProjectionsStage: Merge sorted 8556 rows, containing 12 columns (12 merged, 0 gathered) in 2.899042369 sec., 2951.319405156303 rows/sec., 469.13 KiB/sec. 2024.10.22 01:33:47.351504 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} MergeTask::MergeProjectionsStage: Merge sorted 9627 rows, containing 4 columns (4 merged, 0 gathered) in 1.846104013 sec., 5214.765761954931 rows/sec., 81.48 KiB/sec. 2024.10.22 01:33:47.382107 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} MergedBlockOutputStream: filled checksums 202410_1_561_112 (state Temporary) 2024.10.22 01:33:47.536408 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 11.02 MiB, will set to 1.74 GiB (RSS), difference: -17.69 MiB 2024.10.22 01:33:47.633285 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} MergedBlockOutputStream: filled checksums 202410_527_532_1 (state Temporary) 2024.10.22 01:33:47.705602 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_527_532_1 to 202410_527_532_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:47.709503 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_527_532_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_527_527_0, 202410_532_532_0] -> 202410_527_532_1 2024.10.22 01:33:47.719995 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} MergeTask::MergeProjectionsStage: Merge sorted 497 rows, containing 32 columns (32 merged, 0 gathered) in 2.69328861 sec., 184.53276717343707 rows/sec., 17.07 KiB/sec. 2024.10.22 01:33:47.763656 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} MergedBlockOutputStream: filled checksums 202410_1_453_93 (state Temporary) 2024.10.22 01:33:47.779362 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:33:47.877648 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_561_112 to 202410_1_561_112 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:47.881292 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_561_112} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_556_111, 202410_561_561_0] -> 202410_1_561_112 2024.10.22 01:33:47.931771 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.63 MiB. 2024.10.22 01:33:47.996647 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_453_93 to 202410_1_453_93 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:48.001777 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_453_93} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_448_92, 202410_453_453_0] -> 202410_1_453_93 2024.10.22 01:33:48.258510 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:33:48.806438 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 132 entries to flush up to offset 62910 2024.10.22 01:33:48.895385 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 61234 2024.10.22 01:33:48.963103 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:48.964901 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:49.019685 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_538_538_0 (state Temporary) 2024.10.22 01:33:49.058676 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_933_471} MergeTask::MergeProjectionsStage: Merge sorted 61734 rows, containing 13 columns (13 merged, 0 gathered) in 4.293370958 sec., 14378.911257357977 rows/sec., 2.29 MiB/sec. 2024.10.22 01:33:49.064145 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_538_538_0 to 202410_544_544_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:49.113352 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_933_471} MergedBlockOutputStream: filled checksums 202410_1_933_471 (state Temporary) 2024.10.22 01:33:49.238293 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 62910 2024.10.22 01:33:49.239694 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:49.247361 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:49.416739 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_926_926_0 (state Temporary) 2024.10.22 01:33:49.445785 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 13.06 MiB, will set to 1.73 GiB (RSS), difference: 18.41 MiB 2024.10.22 01:33:49.533736 [ 1239 ] {} DatabaseCatalog: Execution took 274 ms. 2024.10.22 01:33:49.569373 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_926_926_0 to 202410_939_939_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:49.683805 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_933_471} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_933_471 to 202410_1_933_471 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:49.689723 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_933_471} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_927_470, 202410_928_933_1] -> 202410_1_933_471 2024.10.22 01:33:49.719096 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_933_471 to 202410_934_934_0 2024.10.22 01:33:49.740027 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61234 2024.10.22 01:33:49.741748 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:33:49.855414 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.21 MiB using storage policy from min volume index 0 2024.10.22 01:33:49.859581 [ 1248 ] {} DiskLocal: Reserved 2.21 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:49.882656 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_934_472} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_933_471 to 202410_934_934_0 into Compact with storage Full 2024.10.22 01:33:50.077315 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_934_472} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:33:50.108454 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_934_472} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_933_471, total 61734 rows starting from the beginning of the part 2024.10.22 01:33:50.163892 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_934_472} MergeTreeSequentialSource: Reading 2 marks from part 202410_934_934_0, total 93 rows starting from the beginning of the part 2024.10.22 01:33:50.386611 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:33:50.390265 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:33:50.953948 [ 1241 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 01:33:50.986850 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_110_110_0 to 202410_113_113_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:51.446613 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 7.77 MiB, will set to 1.74 GiB (RSS), difference: 3.63 MiB 2024.10.22 01:33:51.919870 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 31 entries to flush up to offset 8481 2024.10.22 01:33:52.068745 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:52.071684 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:52.109115 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_549_549_0 (state Temporary) 2024.10.22 01:33:52.159910 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_549_549_0 to 202410_562_562_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:52.302852 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8481 2024.10.22 01:33:52.409396 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1927 entries to flush up to offset 830122 2024.10.22 01:33:52.455503 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 462 2024.10.22 01:33:52.669087 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_526_105/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:33:52.692713 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:52.697382 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:52.768096 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_444_444_0 (state Temporary) 2024.10.22 01:33:52.859924 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_444_444_0 to 202410_454_454_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:53.027576 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 462 2024.10.22 01:33:53.196746 [ 1177 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:33:53.256145 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rgq/serwynnthjdrcjnwtgkzdviepxkpe. 2024.10.22 01:33:53.341656 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:53.355598 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:53.434192 [ 1177 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:33:53.438005 [ 1177 ] {} DNSCacheUpdater: Execution took 241 ms. 2024.10.22 01:33:53.571903 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 9.19 MiB, will set to 1.73 GiB (RSS), difference: 34.43 MiB 2024.10.22 01:33:53.669223 [ 1237 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 01:33:53.745676 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_934_472} MergeTask::MergeProjectionsStage: Merge sorted 61827 rows, containing 13 columns (13 merged, 0 gathered) in 3.865601938 sec., 15994.145541014574 rows/sec., 2.54 MiB/sec. 2024.10.22 01:33:53.759699 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 61286 2024.10.22 01:33:53.911387 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_526_526_0 (state Temporary) 2024.10.22 01:33:53.912751 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:53.918882 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:53.942654 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_934_472} MergedBlockOutputStream: filled checksums 202410_1_934_472 (state Temporary) 2024.10.22 01:33:54.004074 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_927_927_0 (state Temporary) 2024.10.22 01:33:54.054421 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_927_927_0 to 202410_940_940_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:54.081430 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_526_526_0 to 202410_533_533_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:54.084286 [ 1188 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_935_935_0 to 202410_940_940_0 2024.10.22 01:33:54.129370 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 830122 2024.10.22 01:33:54.132115 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61286 2024.10.22 01:33:54.135871 [ 1188 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:54.141152 [ 1188 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:54.203438 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_935_935_0 to 202410_940_940_0 into Compact with storage Full 2024.10.22 01:33:54.219781 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_526_105} MergeTask::MergeProjectionsStage: Merge sorted 832158 rows, containing 4 columns (4 merged, 0 gathered) in 26.042952127 sec., 31953.28993202968 rows/sec., 479.27 KiB/sec. 2024.10.22 01:33:54.192721 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_934_472} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_934_472 to 202410_1_934_472 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:54.228182 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_934_472} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_933_471, 202410_934_934_0] -> 202410_1_934_472 2024.10.22 01:33:54.263298 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:33:54.273692 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_461_481_4, 202410_482_482_0, 202410_483_483_0, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0] 2024.10.22 01:33:54.276406 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_461_481_4, 202410_482_482_0, 202410_483_483_0, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0] 2024.10.22 01:33:54.385292 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:33:54.406551 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_461_481_4, 202410_482_482_0, 202410_483_483_0, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0] 2024.10.22 01:33:54.416766 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_935_935_0, total 39 rows starting from the beginning of the part 2024.10.22 01:33:54.426314 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4417 2024.10.22 01:33:54.471893 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_526_105} MergedBlockOutputStream: filled checksums 202410_1_526_105 (state Temporary) 2024.10.22 01:33:54.488013 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_936_936_0, total 47 rows starting from the beginning of the part 2024.10.22 01:33:54.530852 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_937_937_0, total 36 rows starting from the beginning of the part 2024.10.22 01:33:54.578121 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_938_938_0, total 33 rows starting from the beginning of the part 2024.10.22 01:33:54.951439 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_939_939_0, total 83 rows starting from the beginning of the part 2024.10.22 01:33:55.015661 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_940_940_0, total 52 rows starting from the beginning of the part 2024.10.22 01:33:55.744075 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.97 GiB, free memory in arenas 9.95 MiB, will set to 1.72 GiB (RSS), difference: 100.22 MiB 2024.10.22 01:33:56.312886 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(723dc7a7b2d3e0ad7b20e55b9094b531) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 723dc7a7b2d3e0ad7b20e55b9094b531, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:33:56.331386 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 723dc7a7b2d3e0ad7b20e55b9094b531, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:33:56.339063 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:33:56.531309 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} MergeTask::MergeProjectionsStage: Merge sorted 290 rows, containing 13 columns (13 merged, 0 gathered) in 2.375492691 sec., 122.0799378161505 rows/sec., 18.78 KiB/sec. 2024.10.22 01:33:56.770250 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 63000 2024.10.22 01:33:56.779341 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} MergedBlockOutputStream: filled checksums 202410_935_940_1 (state Temporary) 2024.10.22 01:33:56.860934 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_830_428, 202410_831_836_1] 2024.10.22 01:33:56.968907 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_830_428, 202410_831_836_1] 2024.10.22 01:33:56.978861 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_830_428, 202410_831_836_1] 2024.10.22 01:33:57.013203 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_935_940_1 to 202410_935_940_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:57.017241 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_935_940_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_935_935_0, 202410_940_940_0] -> 202410_935_940_1 2024.10.22 01:33:57.023710 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:57.028645 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:57.167789 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:33:57.304636 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_539_539_0 (state Temporary) 2024.10.22 01:33:57.431104 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 10.77 MiB, will set to 1.71 GiB (RSS), difference: 54.81 MiB 2024.10.22 01:33:57.469708 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_539_539_0 to 202410_545_545_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:57.476970 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_526_105} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_526_105 to 202410_1_526_105 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:57.480465 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_526_105} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_520_104, 202410_521_526_1] -> 202410_1_526_105 2024.10.22 01:33:57.511075 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:33:57.661282 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 63000 2024.10.22 01:33:57.827910 [ 1708 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jbs/lblfzbrtclwkmjygreywcbbozksli, size 1124033 2024.10.22 01:33:58.147178 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:33:58.151287 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 61340 2024.10.22 01:33:58.155920 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:33:58.162828 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:33:58.173435 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:33:58.200515 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:33:58.223619 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:33:58.263010 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:33:58.302955 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 01:33:58.347055 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:33:58.348433 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ttu/kjjxikrxnklanuegcihzhyuecpthz (LogSeriesLimiter: on interval from 2024-10-22 01:33:40 to 2024-10-22 01:33:58 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:33:58.417460 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:33:58.483816 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_928_928_0 (state Temporary) 2024.10.22 01:33:58.569494 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_928_928_0 to 202410_941_941_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:33:58.627561 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61340 2024.10.22 01:33:59.050423 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fae072e0b4ba29229ccaa8cffa7ea53b) : [0, 2764373]: Setting download as failed: File segment: [0, 2764373], key: fae072e0b4ba29229ccaa8cffa7ea53b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:33:59.052068 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fae072e0b4ba29229ccaa8cffa7ea53b): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:33:59.140382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fae072e0b4ba29229ccaa8cffa7ea53b, offset: 0, size: 0 2024.10.22 01:33:59.145243 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fae072e0b4ba29229ccaa8cffa7ea53b for removal 2024.10.22 01:33:59.150235 [ 1476 ] {} CacheMetadata: Key fae072e0b4ba29229ccaa8cffa7ea53b is removed from metadata 2024.10.22 01:33:59.524326 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 8.36 MiB, will set to 1.70 GiB (RSS), difference: 9.51 MiB 2024.10.22 01:33:59.776211 [ 1240 ] {} DatabaseCatalog: Execution took 239 ms. 2024.10.22 01:33:59.809256 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 8500 2024.10.22 01:34:00.097480 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9c4c03e1ccb2e77b5b127ea75833b37a) : [0, 55241]: Setting download as failed: File segment: [0, 55241], key: 9c4c03e1ccb2e77b5b127ea75833b37a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:00.117207 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:00.172940 [ 1743 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/msf/mwkvghgxfzrtyeajlfqafhcjqhilv, size 13242 2024.10.22 01:34:00.200645 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9c4c03e1ccb2e77b5b127ea75833b37a): No space left in cache to reserve 55242 bytes, will continue without cache download 2024.10.22 01:34:00.203369 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:00.257770 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_550_550_0 (state Temporary) 2024.10.22 01:34:00.319798 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9c4c03e1ccb2e77b5b127ea75833b37a, offset: 0, size: 0 2024.10.22 01:34:00.323300 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9c4c03e1ccb2e77b5b127ea75833b37a for removal 2024.10.22 01:34:00.328078 [ 1476 ] {} CacheMetadata: Key 9c4c03e1ccb2e77b5b127ea75833b37a is removed from metadata 2024.10.22 01:34:00.392152 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:34:00.401072 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:34:00.529293 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 463 2024.10.22 01:34:00.667327 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_469_94, 202410_470_470_0, 202410_471_471_0, 202410_472_472_0, 202410_473_473_0, 202410_474_474_0] 2024.10.22 01:34:00.673144 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_469_94, 202410_470_470_0, 202410_471_471_0, 202410_472_472_0, 202410_473_473_0, 202410_474_474_0] 2024.10.22 01:34:00.722059 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_469_94, 202410_470_470_0, 202410_471_471_0, 202410_472_472_0, 202410_473_473_0, 202410_474_474_0] 2024.10.22 01:34:00.802237 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:34:00.826289 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:34:00.862148 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:34:00.881592 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_550_550_0 to 202410_563_563_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:00.891364 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:34:00.926541 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:34:00.949133 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8500 2024.10.22 01:34:00.959652 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:34:01.021664 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:01.024707 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:01.141469 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 831662 2024.10.22 01:34:01.186825 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_445_445_0 (state Temporary) 2024.10.22 01:34:01.214835 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:34:01.243867 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:34:01.377172 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:34:01.575660 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 7.48 MiB, will set to 1.71 GiB (RSS), difference: -4.64 MiB 2024.10.22 01:34:01.652529 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_445_445_0 to 202410_455_455_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:01.715663 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 463 2024.10.22 01:34:01.806686 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_94_19, 202410_95_95_0, 202410_96_96_0, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0] 2024.10.22 01:34:01.811258 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_94_19, 202410_95_95_0, 202410_96_96_0, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0] 2024.10.22 01:34:01.928986 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(39eaeefed842c429645e7e6a8889c089) : [0, 1337948]: Setting download as failed: File segment: [0, 1337948], key: 39eaeefed842c429645e7e6a8889c089, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:01.930769 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(39eaeefed842c429645e7e6a8889c089): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:34:01.972109 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 39eaeefed842c429645e7e6a8889c089, offset: 0, size: 0 2024.10.22 01:34:01.973129 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 39eaeefed842c429645e7e6a8889c089 for removal 2024.10.22 01:34:01.982738 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_94_19, 202410_95_95_0, 202410_96_96_0, 202410_97_97_0, 202410_98_98_0, 202410_99_99_0] 2024.10.22 01:34:01.997597 [ 1476 ] {} CacheMetadata: Key 39eaeefed842c429645e7e6a8889c089 is removed from metadata 2024.10.22 01:34:02.002163 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 44 entries to flush up to offset 4461 2024.10.22 01:34:02.244305 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:02.257279 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:02.481729 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_527_527_0 (state Temporary) 2024.10.22 01:34:02.641805 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 61408 2024.10.22 01:34:02.777687 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:02.780452 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:02.877418 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_929_929_0 (state Temporary) 2024.10.22 01:34:02.878293 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_527_527_0 to 202410_534_534_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:03.038951 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 831662 2024.10.22 01:34:03.259221 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_929_929_0 to 202410_942_942_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:03.311427 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61408 2024.10.22 01:34:03.427399 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_836_429, 202410_837_837_0] 2024.10.22 01:34:03.434861 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_836_429, 202410_837_837_0] 2024.10.22 01:34:03.458622 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_836_429, 202410_837_837_0] 2024.10.22 01:34:03.664434 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 6.50 MiB, will set to 1.70 GiB (RSS), difference: 1.10 MiB 2024.10.22 01:34:04.368764 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a6fc4662a015070e93591b267ea74f49) : [0, 315233]: Setting download as failed: File segment: [0, 315233], key: a6fc4662a015070e93591b267ea74f49, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:04.370242 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a6fc4662a015070e93591b267ea74f49): No space left in cache to reserve 315234 bytes, will continue without cache download 2024.10.22 01:34:04.405770 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a6fc4662a015070e93591b267ea74f49, offset: 0, size: 0 2024.10.22 01:34:04.408927 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a6fc4662a015070e93591b267ea74f49 for removal 2024.10.22 01:34:04.414352 [ 1476 ] {} CacheMetadata: Key a6fc4662a015070e93591b267ea74f49 is removed from metadata 2024.10.22 01:34:04.973014 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_838_838_0, 202410_839_839_0, 202410_840_840_0, 202410_841_841_0, 202410_842_842_0, 202410_843_843_0] 2024.10.22 01:34:04.976393 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_838_838_0, 202410_839_839_0, 202410_840_840_0, 202410_841_841_0, 202410_842_842_0, 202410_843_843_0] 2024.10.22 01:34:05.140970 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_838_838_0, 202410_839_839_0, 202410_840_840_0, 202410_841_841_0, 202410_842_842_0, 202410_843_843_0] 2024.10.22 01:34:05.170424 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 52 entries to flush up to offset 63052 2024.10.22 01:34:05.467069 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:05.468810 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:34:05.510661 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_540_540_0 (state Temporary) 2024.10.22 01:34:05.659868 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_540_540_0 to 202410_546_546_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:05.726083 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 63052 2024.10.22 01:34:05.734370 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8de647cbfe74957144845e5b794d69ba) : [0, 181566]: Setting download as failed: File segment: [0, 181566], key: 8de647cbfe74957144845e5b794d69ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:05.741242 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8de647cbfe74957144845e5b794d69ba): No space left in cache to reserve 181567 bytes, will continue without cache download 2024.10.22 01:34:05.865405 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8de647cbfe74957144845e5b794d69ba, offset: 0, size: 0 2024.10.22 01:34:05.866693 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8de647cbfe74957144845e5b794d69ba for removal 2024.10.22 01:34:05.869295 [ 1476 ] {} CacheMetadata: Key 8de647cbfe74957144845e5b794d69ba is removed from metadata 2024.10.22 01:34:06.720098 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e9d0729adcbb94bfeebc996afcc2c87b) : [0, 409197]: Setting download as failed: File segment: [0, 409197], key: e9d0729adcbb94bfeebc996afcc2c87b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:06.724149 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e9d0729adcbb94bfeebc996afcc2c87b): No space left in cache to reserve 409198 bytes, will continue without cache download 2024.10.22 01:34:06.858670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e9d0729adcbb94bfeebc996afcc2c87b, offset: 0, size: 0 2024.10.22 01:34:06.861279 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e9d0729adcbb94bfeebc996afcc2c87b for removal 2024.10.22 01:34:06.864582 [ 1476 ] {} CacheMetadata: Key e9d0729adcbb94bfeebc996afcc2c87b is removed from metadata 2024.10.22 01:34:07.331534 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 61444 2024.10.22 01:34:07.561328 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:07.564508 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:34:07.601350 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_930_930_0 (state Temporary) 2024.10.22 01:34:07.644044 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_930_930_0 to 202410_943_943_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:07.678948 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61444 2024.10.22 01:34:07.681293 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(49b0bcb2f219d269a7be79a378ca6530) : [0, 80082]: Setting download as failed: File segment: [0, 80082], key: 49b0bcb2f219d269a7be79a378ca6530, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:07.682789 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(49b0bcb2f219d269a7be79a378ca6530): No space left in cache to reserve 80083 bytes, will continue without cache download 2024.10.22 01:34:07.704840 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 49b0bcb2f219d269a7be79a378ca6530, offset: 0, size: 0 2024.10.22 01:34:07.709482 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 49b0bcb2f219d269a7be79a378ca6530 for removal 2024.10.22 01:34:07.711348 [ 1476 ] {} CacheMetadata: Key 49b0bcb2f219d269a7be79a378ca6530 is removed from metadata 2024.10.22 01:34:08.427275 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c2b1c5e7d4b8619488b3935b34fd8fcf) : [0, 100546]: Setting download as failed: File segment: [0, 100546], key: c2b1c5e7d4b8619488b3935b34fd8fcf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:08.433735 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c2b1c5e7d4b8619488b3935b34fd8fcf): No space left in cache to reserve 100547 bytes, will continue without cache download 2024.10.22 01:34:08.450082 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c2b1c5e7d4b8619488b3935b34fd8fcf, offset: 0, size: 0 2024.10.22 01:34:08.452653 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c2b1c5e7d4b8619488b3935b34fd8fcf for removal 2024.10.22 01:34:08.454218 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 8506 2024.10.22 01:34:08.457633 [ 1476 ] {} CacheMetadata: Key c2b1c5e7d4b8619488b3935b34fd8fcf is removed from metadata 2024.10.22 01:34:08.547847 [ 1225 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:34:08.567668 [ 1225 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:34:08.744514 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:08.751923 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:34:08.782959 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_551_551_0 (state Temporary) 2024.10.22 01:34:09.165176 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_551_551_0 to 202410_564_564_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:09.224169 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8506 2024.10.22 01:34:09.558469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b6fbec3596573893bd7d2bd16f5ed724) : [0, 139705]: Setting download as failed: File segment: [0, 139705], key: b6fbec3596573893bd7d2bd16f5ed724, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:09.559947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b6fbec3596573893bd7d2bd16f5ed724): No space left in cache to reserve 139706 bytes, will continue without cache download 2024.10.22 01:34:09.674968 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b6fbec3596573893bd7d2bd16f5ed724, offset: 0, size: 0 2024.10.22 01:34:09.676465 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b6fbec3596573893bd7d2bd16f5ed724 for removal 2024.10.22 01:34:09.678360 [ 1476 ] {} CacheMetadata: Key b6fbec3596573893bd7d2bd16f5ed724 is removed from metadata 2024.10.22 01:34:10.048123 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 833202 2024.10.22 01:34:10.075503 [ 1198 ] {} DatabaseCatalog: Execution took 297 ms. 2024.10.22 01:34:10.392174 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:34:10.396129 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:34:10.631253 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(036ecd7403369e706acb6ca91e13013c) : [0, 134295]: Setting download as failed: File segment: [0, 134295], key: 036ecd7403369e706acb6ca91e13013c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:10.635015 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(036ecd7403369e706acb6ca91e13013c): No space left in cache to reserve 134296 bytes, will continue without cache download 2024.10.22 01:34:10.652255 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 036ecd7403369e706acb6ca91e13013c, offset: 0, size: 0 2024.10.22 01:34:10.654552 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 036ecd7403369e706acb6ca91e13013c for removal 2024.10.22 01:34:10.657286 [ 1476 ] {} CacheMetadata: Key 036ecd7403369e706acb6ca91e13013c is removed from metadata 2024.10.22 01:34:10.984672 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:11.000462 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:34:11.291783 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_528_528_0 (state Temporary) 2024.10.22 01:34:11.586489 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_528_528_0 to 202410_535_535_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:11.685703 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 61486 2024.10.22 01:34:11.724534 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 833202 2024.10.22 01:34:11.743401 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(848f34f6281bafd480eac00c7a970c8d) : [0, 71598]: Setting download as failed: File segment: [0, 71598], key: 848f34f6281bafd480eac00c7a970c8d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:11.745269 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(848f34f6281bafd480eac00c7a970c8d): No space left in cache to reserve 71599 bytes, will continue without cache download 2024.10.22 01:34:11.762433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 848f34f6281bafd480eac00c7a970c8d, offset: 0, size: 0 2024.10.22 01:34:11.764482 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 848f34f6281bafd480eac00c7a970c8d for removal 2024.10.22 01:34:11.776751 [ 1476 ] {} CacheMetadata: Key 848f34f6281bafd480eac00c7a970c8d is removed from metadata 2024.10.22 01:34:11.820307 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:34:11.956033 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:11.961330 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:34:12.014143 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_931_931_0 (state Temporary) 2024.10.22 01:34:12.060030 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_931_931_0 to 202410_944_944_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:12.096810 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61486 2024.10.22 01:34:12.570375 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3f802ea684a870ddc4762ac6bb34131b) : [0, 66153]: Setting download as failed: File segment: [0, 66153], key: 3f802ea684a870ddc4762ac6bb34131b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:12.571534 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3f802ea684a870ddc4762ac6bb34131b): No space left in cache to reserve 66154 bytes, will continue without cache download 2024.10.22 01:34:12.581386 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3f802ea684a870ddc4762ac6bb34131b, offset: 0, size: 0 2024.10.22 01:34:12.583179 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3f802ea684a870ddc4762ac6bb34131b for removal 2024.10.22 01:34:12.585608 [ 1476 ] {} CacheMetadata: Key 3f802ea684a870ddc4762ac6bb34131b is removed from metadata 2024.10.22 01:34:13.367425 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 29 entries to flush up to offset 63081 2024.10.22 01:34:13.499009 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:13.512750 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:34:13.567898 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_541_541_0 (state Temporary) 2024.10.22 01:34:13.605488 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_541_541_0 to 202410_547_547_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:13.651954 [ 1218 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_542_7 to 202410_547_547_0 2024.10.22 01:34:13.662589 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 63081 2024.10.22 01:34:13.664040 [ 1218 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:13.774629 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(2de86453f2a5277468887d680076d836) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2de86453f2a5277468887d680076d836, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:34:13.777009 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 475033, file segment info: File segment: [0, 33554431], key: 2de86453f2a5277468887d680076d836, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:34:13.779292 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:34:13.668665 [ 1218 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.63 GiB. 2024.10.22 01:34:13.813139 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_542_7 to 202410_547_547_0 into Compact with storage Full 2024.10.22 01:34:13.881506 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:13.888948 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_542_7, total 3576 rows starting from the beginning of the part 2024.10.22 01:34:13.909559 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_543_543_0, total 77 rows starting from the beginning of the part 2024.10.22 01:34:13.937169 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_544_544_0, total 132 rows starting from the beginning of the part 2024.10.22 01:34:13.967337 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_545_545_0, total 90 rows starting from the beginning of the part 2024.10.22 01:34:14.000463 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_546_546_0, total 52 rows starting from the beginning of the part 2024.10.22 01:34:14.141533 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_547_547_0, total 29 rows starting from the beginning of the part 2024.10.22 01:34:14.764861 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 5.55 MiB, will set to 1.70 GiB (RSS), difference: -5.32 MiB 2024.10.22 01:34:14.816938 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(64eb388b3d16248a7cf3466fae1ae888) : [0, 45338]: Setting download as failed: File segment: [0, 45338], key: 64eb388b3d16248a7cf3466fae1ae888, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:14.820475 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(64eb388b3d16248a7cf3466fae1ae888): No space left in cache to reserve 45339 bytes, will continue without cache download 2024.10.22 01:34:15.066829 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 64eb388b3d16248a7cf3466fae1ae888, offset: 0, size: 0 2024.10.22 01:34:15.068553 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 64eb388b3d16248a7cf3466fae1ae888 for removal 2024.10.22 01:34:15.074364 [ 1476 ] {} CacheMetadata: Key 64eb388b3d16248a7cf3466fae1ae888 is removed from metadata 2024.10.22 01:34:15.091029 [ 1727 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ttu/kjjxikrxnklanuegcihzhyuecpthz, size 475033 2024.10.22 01:34:15.413401 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:34:15.427678 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:34:15.451008 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:34:15.582606 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:34:15.627698 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:34:15.655886 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:34:15.665471 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:34:15.681724 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} MergeTask::MergeProjectionsStage: Merge sorted 3956 rows, containing 13 columns (13 merged, 0 gathered) in 1.865325455 sec., 2120.809529187495 rows/sec., 938.56 KiB/sec. 2024.10.22 01:34:15.694783 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column CookieEnable 2024.10.22 01:34:15.760384 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_547_8/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:34:15.770567 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/htc/gejaewyruozfpktlzpwmcjsifkijw (LogSeriesLimiter: on interval from 2024-10-22 01:33:58 to 2024-10-22 01:34:15 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:34:15.885402 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} MergedBlockOutputStream: filled checksums 202410_507_547_8 (state Temporary) 2024.10.22 01:34:16.077962 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(08bb53218b7d05f355237228a3d840cd) : [0, 143703]: Setting download as failed: File segment: [0, 143703], key: 08bb53218b7d05f355237228a3d840cd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:16.079708 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(08bb53218b7d05f355237228a3d840cd): No space left in cache to reserve 143704 bytes, will continue without cache download 2024.10.22 01:34:16.107857 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 61545 2024.10.22 01:34:16.126845 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 08bb53218b7d05f355237228a3d840cd, offset: 0, size: 0 2024.10.22 01:34:16.128294 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 08bb53218b7d05f355237228a3d840cd for removal 2024.10.22 01:34:16.133247 [ 1476 ] {} CacheMetadata: Key 08bb53218b7d05f355237228a3d840cd is removed from metadata 2024.10.22 01:34:16.223102 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_547_8 to 202410_507_547_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:16.228219 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_547_8} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_542_7, 202410_547_547_0] -> 202410_507_547_8 2024.10.22 01:34:16.368508 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.74 MiB. 2024.10.22 01:34:16.398237 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 5.69 MiB, will set to 1.70 GiB (RSS), difference: 4.76 MiB 2024.10.22 01:34:16.631649 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:16.641781 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:16.691646 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_932_932_0 (state Temporary) 2024.10.22 01:34:16.718675 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 464 2024.10.22 01:34:16.733423 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 8517 2024.10.22 01:34:16.756217 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_932_932_0 to 202410_945_945_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:16.780184 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_934_472 to 202410_935_940_1 2024.10.22 01:34:16.788471 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.22 MiB using storage policy from min volume index 0 2024.10.22 01:34:16.792063 [ 1278 ] {} DiskLocal: Reserved 2.22 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:16.793667 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61545 2024.10.22 01:34:16.856647 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_940_473} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_934_472 to 202410_935_940_1 into Compact with storage Full 2024.10.22 01:34:16.965591 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_940_473} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:16.995272 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_940_473} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_934_472, total 61827 rows starting from the beginning of the part 2024.10.22 01:34:17.060425 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_940_473} MergeTreeSequentialSource: Reading 2 marks from part 202410_935_940_1, total 290 rows starting from the beginning of the part 2024.10.22 01:34:17.074281 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:17.078417 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:17.140876 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_552_552_0 (state Temporary) 2024.10.22 01:34:17.347868 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:17.352282 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:17.473504 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:17.478440 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:17.561496 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_552_552_0 to 202410_565_565_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:17.660293 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_446_446_0 (state Temporary) 2024.10.22 01:34:17.705052 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8517 2024.10.22 01:34:17.786321 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_403_83, 202410_404_404_0, 202410_405_405_0, 202410_406_406_0, 202410_407_407_0, 202410_408_408_0] 2024.10.22 01:34:17.789314 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_403_83, 202410_404_404_0, 202410_405_405_0, 202410_406_406_0, 202410_407_407_0, 202410_408_408_0] 2024.10.22 01:34:17.856946 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_403_83, 202410_404_404_0, 202410_405_405_0, 202410_406_406_0, 202410_407_407_0, 202410_408_408_0] 2024.10.22 01:34:17.874696 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_446_446_0 to 202410_456_456_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:18.008782 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 464 2024.10.22 01:34:18.275322 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 7 file segments (while reserving 20500 bytes for a45516b7baa25a5414ef3398f6925dd6:0) 2024.10.22 01:34:18.277713 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 509d4484916cff7b788128d53ba29e16, offset: 0, size: 184 2024.10.22 01:34:18.284142 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 509d4484916cff7b788128d53ba29e16 for removal 2024.10.22 01:34:18.288382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4ffc311f05919b4badc7bf50c4d898d1, offset: 0, size: 4818 2024.10.22 01:34:18.289067 [ 1476 ] {} CacheMetadata: Key 509d4484916cff7b788128d53ba29e16 is removed from metadata 2024.10.22 01:34:18.296328 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4ffc311f05919b4badc7bf50c4d898d1 for removal 2024.10.22 01:34:18.310306 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8f32089c4b75dca0add3b900169de96a, offset: 0, size: 176 2024.10.22 01:34:18.348022 [ 1476 ] {} CacheMetadata: Key 4ffc311f05919b4badc7bf50c4d898d1 is removed from metadata 2024.10.22 01:34:18.426541 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8f32089c4b75dca0add3b900169de96a for removal 2024.10.22 01:34:18.434145 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f3a23ae55133f5ac3ca85fad0e4fc2b8, offset: 0, size: 3095 2024.10.22 01:34:18.437567 [ 1476 ] {} CacheMetadata: Key 8f32089c4b75dca0add3b900169de96a is removed from metadata 2024.10.22 01:34:18.452888 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f3a23ae55133f5ac3ca85fad0e4fc2b8 for removal 2024.10.22 01:34:18.463985 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8acfa2dc629d3cc50b74b7d268409eac, offset: 0, size: 145 2024.10.22 01:34:18.464985 [ 1476 ] {} CacheMetadata: Key f3a23ae55133f5ac3ca85fad0e4fc2b8 is removed from metadata 2024.10.22 01:34:18.468559 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8acfa2dc629d3cc50b74b7d268409eac for removal 2024.10.22 01:34:18.471074 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f3651f91e45990d9def68e2216198df7, offset: 0, size: 144 2024.10.22 01:34:18.472339 [ 1476 ] {} CacheMetadata: Key 8acfa2dc629d3cc50b74b7d268409eac is removed from metadata 2024.10.22 01:34:18.476799 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f3651f91e45990d9def68e2216198df7 for removal 2024.10.22 01:34:18.481246 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 502986d467490b0776b200ef9902f295, offset: 0, size: 243 2024.10.22 01:34:18.482833 [ 1476 ] {} CacheMetadata: Key f3651f91e45990d9def68e2216198df7 is removed from metadata 2024.10.22 01:34:18.486164 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 502986d467490b0776b200ef9902f295 for removal 2024.10.22 01:34:18.502644 [ 1476 ] {} CacheMetadata: Key 502986d467490b0776b200ef9902f295 is removed from metadata 2024.10.22 01:34:18.524709 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 4.84 MiB, will set to 1.70 GiB (RSS), difference: -10.92 MiB 2024.10.22 01:34:18.729114 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 834744 2024.10.22 01:34:18.880977 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_940_473/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:34:19.608566 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8888cb8a9649ff7108ca3c0a04645c5f) : [0, 223011]: Setting download as failed: File segment: [0, 223011], key: 8888cb8a9649ff7108ca3c0a04645c5f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:19.611902 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8888cb8a9649ff7108ca3c0a04645c5f): No space left in cache to reserve 223012 bytes, will continue without cache download 2024.10.22 01:34:19.658397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8888cb8a9649ff7108ca3c0a04645c5f, offset: 0, size: 0 2024.10.22 01:34:19.659904 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8888cb8a9649ff7108ca3c0a04645c5f for removal 2024.10.22 01:34:19.662307 [ 1476 ] {} CacheMetadata: Key 8888cb8a9649ff7108ca3c0a04645c5f is removed from metadata 2024.10.22 01:34:20.138668 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:20.154434 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:20.283041 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_111_111_0 (state Temporary) 2024.10.22 01:34:20.370684 [ 1259 ] {} DatabaseCatalog: Execution took 293 ms. 2024.10.22 01:34:20.396470 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:34:20.411588 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:34:20.457672 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_529_529_0 (state Temporary) 2024.10.22 01:34:20.492221 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 4.59 MiB, will set to 1.71 GiB (RSS), difference: -6.77 MiB 2024.10.22 01:34:20.629228 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_529_529_0 to 202410_536_536_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:20.673074 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 834744 2024.10.22 01:34:20.770883 [ 1257 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_526_105 to 202410_527_532_1 2024.10.22 01:34:20.795912 [ 1257 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:20.803152 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:20.812594 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 61624 2024.10.22 01:34:20.855823 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_532_106} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_526_105 to 202410_527_532_1 into Wide with storage Full 2024.10.22 01:34:20.885119 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_532_106} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:20.899748 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_532_106} MergeTreeSequentialSource: Reading 103 marks from part 202410_1_526_105, total 832158 rows starting from the beginning of the part 2024.10.22 01:34:20.972606 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_532_106} MergeTreeSequentialSource: Reading 3 marks from part 202410_527_532_1, total 9627 rows starting from the beginning of the part 2024.10.22 01:34:21.079631 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:21.082952 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:21.156442 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_933_933_0 (state Temporary) 2024.10.22 01:34:21.162372 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(565a62f6ba1b67421d857d642f2f81a8) : [0, 204309]: Setting download as failed: File segment: [0, 204309], key: 565a62f6ba1b67421d857d642f2f81a8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:21.164370 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(565a62f6ba1b67421d857d642f2f81a8): No space left in cache to reserve 204310 bytes, will continue without cache download 2024.10.22 01:34:21.175411 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 76 entries to flush up to offset 63157 2024.10.22 01:34:21.213233 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 565a62f6ba1b67421d857d642f2f81a8, offset: 0, size: 0 2024.10.22 01:34:21.355337 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 565a62f6ba1b67421d857d642f2f81a8 for removal 2024.10.22 01:34:21.361699 [ 1476 ] {} CacheMetadata: Key 565a62f6ba1b67421d857d642f2f81a8 is removed from metadata 2024.10.22 01:34:21.427709 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_933_933_0 to 202410_946_946_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:21.461477 [ 1160 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_941_941_0 to 202410_946_946_0 2024.10.22 01:34:21.480071 [ 1160 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:21.483355 [ 1160 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:21.558566 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_941_941_0 to 202410_946_946_0 into Compact with storage Full 2024.10.22 01:34:21.563534 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_501_100, 202410_502_502_0, 202410_503_503_0, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0] 2024.10.22 01:34:21.564401 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:21.565661 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_501_100, 202410_502_502_0, 202410_503_503_0, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0] 2024.10.22 01:34:21.572264 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:21.590761 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_501_100, 202410_502_502_0, 202410_503_503_0, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0] 2024.10.22 01:34:21.590990 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61624 2024.10.22 01:34:21.747318 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_542_542_0 (state Temporary) 2024.10.22 01:34:21.837064 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_542_542_0 to 202410_548_548_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:21.937555 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:21.984229 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 63157 2024.10.22 01:34:22.111252 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_941_941_0, total 54 rows starting from the beginning of the part 2024.10.22 01:34:22.282706 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_942_942_0, total 68 rows starting from the beginning of the part 2024.10.22 01:34:22.333835 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(57d046d73a4f7a6e3c8208bed9f676e7) : [0, 186353]: Setting download as failed: File segment: [0, 186353], key: 57d046d73a4f7a6e3c8208bed9f676e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:22.338143 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(57d046d73a4f7a6e3c8208bed9f676e7): No space left in cache to reserve 186354 bytes, will continue without cache download 2024.10.22 01:34:22.369123 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 57d046d73a4f7a6e3c8208bed9f676e7, offset: 0, size: 0 2024.10.22 01:34:22.370445 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 57d046d73a4f7a6e3c8208bed9f676e7 for removal 2024.10.22 01:34:22.373759 [ 1476 ] {} CacheMetadata: Key 57d046d73a4f7a6e3c8208bed9f676e7 is removed from metadata 2024.10.22 01:34:22.472570 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_943_943_0, total 36 rows starting from the beginning of the part 2024.10.22 01:34:22.516150 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_940_473} MergeTask::MergeProjectionsStage: Merge sorted 62117 rows, containing 13 columns (13 merged, 0 gathered) in 5.677370567 sec., 10941.156520777095 rows/sec., 1.74 MiB/sec. 2024.10.22 01:34:22.573412 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_944_944_0, total 42 rows starting from the beginning of the part 2024.10.22 01:34:22.676252 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_940_473} MergedBlockOutputStream: filled checksums 202410_1_940_473 (state Temporary) 2024.10.22 01:34:22.777251 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_940_473} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_940_473 to 202410_1_940_473 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:22.780796 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_940_473} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_934_472, 202410_935_940_1] -> 202410_1_940_473 2024.10.22 01:34:22.799970 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:34:22.860127 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_945_945_0, total 59 rows starting from the beginning of the part 2024.10.22 01:34:22.925516 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_946_946_0, total 79 rows starting from the beginning of the part 2024.10.22 01:34:23.139641 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c1597057bbfcaf6ac6d7f1031e859ca8) : [0, 183528]: Setting download as failed: File segment: [0, 183528], key: c1597057bbfcaf6ac6d7f1031e859ca8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:23.141943 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c1597057bbfcaf6ac6d7f1031e859ca8): No space left in cache to reserve 183529 bytes, will continue without cache download 2024.10.22 01:34:23.169183 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c1597057bbfcaf6ac6d7f1031e859ca8, offset: 0, size: 0 2024.10.22 01:34:23.170640 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c1597057bbfcaf6ac6d7f1031e859ca8 for removal 2024.10.22 01:34:23.174587 [ 1476 ] {} CacheMetadata: Key c1597057bbfcaf6ac6d7f1031e859ca8 is removed from metadata 2024.10.22 01:34:23.495419 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:34:23.532458 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:34:23.557653 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:34:23.573993 [ 1169 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:34:23.579570 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:34:23.593026 [ 1169 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:34:23.600786 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:34:23.703023 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:34:23.733237 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:34:23.765043 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:34:23.834832 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:34:23.974340 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ayl/bifobpyjvkgszdqoqehehmajxtxtd (LogSeriesLimiter: on interval from 2024-10-22 01:34:15 to 2024-10-22 01:34:23 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:34:24.624926 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(42f4fdb506e123d9efea68f023de68a1) : [0, 200887]: Setting download as failed: File segment: [0, 200887], key: 42f4fdb506e123d9efea68f023de68a1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:24.627347 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(42f4fdb506e123d9efea68f023de68a1): No space left in cache to reserve 200888 bytes, will continue without cache download 2024.10.22 01:34:24.758551 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 42f4fdb506e123d9efea68f023de68a1, offset: 0, size: 0 2024.10.22 01:34:24.760036 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 42f4fdb506e123d9efea68f023de68a1 for removal 2024.10.22 01:34:24.762220 [ 1476 ] {} CacheMetadata: Key 42f4fdb506e123d9efea68f023de68a1 is removed from metadata 2024.10.22 01:34:24.869111 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} MergeTask::MergeProjectionsStage: Merge sorted 338 rows, containing 13 columns (13 merged, 0 gathered) in 3.209582434 sec., 105.309649136745 rows/sec., 17.75 KiB/sec. 2024.10.22 01:34:25.015697 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} MergedBlockOutputStream: filled checksums 202410_941_946_1 (state Temporary) 2024.10.22 01:34:25.116638 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_941_946_1 to 202410_941_946_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:25.122461 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_941_946_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_941_941_0, 202410_946_946_0] -> 202410_941_946_1 2024.10.22 01:34:25.164762 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:34:25.208797 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 8533 2024.10.22 01:34:25.351728 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 14.41 MiB, will set to 1.71 GiB (RSS), difference: 5.67 MiB 2024.10.22 01:34:25.399761 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:25.439800 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:25.478505 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_553_553_0 (state Temporary) 2024.10.22 01:34:25.512256 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 465 2024.10.22 01:34:25.613536 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 61703 2024.10.22 01:34:25.772343 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(091b847c22afe0a1757545f14cf10c17) : [0, 113606]: Setting download as failed: File segment: [0, 113606], key: 091b847c22afe0a1757545f14cf10c17, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:25.773834 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(091b847c22afe0a1757545f14cf10c17): No space left in cache to reserve 113607 bytes, will continue without cache download 2024.10.22 01:34:25.827138 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_553_553_0 to 202410_566_566_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:25.841444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 091b847c22afe0a1757545f14cf10c17, offset: 0, size: 0 2024.10.22 01:34:25.847359 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 091b847c22afe0a1757545f14cf10c17 for removal 2024.10.22 01:34:25.852421 [ 1476 ] {} CacheMetadata: Key 091b847c22afe0a1757545f14cf10c17 is removed from metadata 2024.10.22 01:34:25.861430 [ 1202 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_561_112 to 202410_566_566_0 2024.10.22 01:34:25.878772 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:25.898689 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8533 2024.10.22 01:34:25.928763 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.06718003 sec. 2024.10.22 01:34:25.965579 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:25.975786 [ 1202 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:25.980264 [ 1202 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:26.009618 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_561_112 to 202410_566_566_0 into Compact with storage Full 2024.10.22 01:34:26.060643 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_532_106/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:34:26.156531 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:26.166653 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_561_112, total 8556 rows starting from the beginning of the part 2024.10.22 01:34:26.176321 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_447_447_0 (state Temporary) 2024.10.22 01:34:26.180532 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_562_562_0, total 31 rows starting from the beginning of the part 2024.10.22 01:34:26.211994 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:26.251334 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:26.255235 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_563_563_0, total 19 rows starting from the beginning of the part 2024.10.22 01:34:26.358527 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_447_447_0 to 202410_457_457_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:26.382651 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_934_934_0 (state Temporary) 2024.10.22 01:34:26.457669 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_934_934_0 to 202410_947_947_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:26.473411 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_564_564_0, total 6 rows starting from the beginning of the part 2024.10.22 01:34:26.488524 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_565_565_0, total 11 rows starting from the beginning of the part 2024.10.22 01:34:26.513045 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61703 2024.10.22 01:34:26.552141 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 465 2024.10.22 01:34:26.619893 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_566_566_0, total 16 rows starting from the beginning of the part 2024.10.22 01:34:27.307640 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(098f9c2892375c95be18b886fb609432) : [0, 1346528]: Setting download as failed: File segment: [0, 1346528], key: 098f9c2892375c95be18b886fb609432, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:27.311687 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(098f9c2892375c95be18b886fb609432): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:34:27.381975 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 098f9c2892375c95be18b886fb609432, offset: 0, size: 0 2024.10.22 01:34:27.386494 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 098f9c2892375c95be18b886fb609432 for removal 2024.10.22 01:34:27.390792 [ 1476 ] {} CacheMetadata: Key 098f9c2892375c95be18b886fb609432 is removed from metadata 2024.10.22 01:34:27.682875 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 836284 2024.10.22 01:34:27.732225 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 9.54 MiB, will set to 1.71 GiB (RSS), difference: -11.30 MiB 2024.10.22 01:34:28.075011 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_111_111_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:34:28.470271 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} MergeTask::MergeProjectionsStage: Merge sorted 8639 rows, containing 12 columns (12 merged, 0 gathered) in 2.466548189 sec., 3502.465525922875 rows/sec., 556.84 KiB/sec. 2024.10.22 01:34:28.474715 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:28.478587 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:28.491261 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} MergedBlockOutputStream: filled checksums 202410_1_566_113 (state Temporary) 2024.10.22 01:34:28.687352 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_530_530_0 (state Temporary) 2024.10.22 01:34:28.702982 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_566_113 to 202410_1_566_113 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:28.710631 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_566_113} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_561_112, 202410_566_566_0] -> 202410_1_566_113 2024.10.22 01:34:28.759683 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.64 MiB. 2024.10.22 01:34:28.766672 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_530_530_0 to 202410_537_537_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:28.802656 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 836284 2024.10.22 01:34:28.804383 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b4395a3e2928276d26c0817654a90722) : [0, 66779]: Setting download as failed: File segment: [0, 66779], key: b4395a3e2928276d26c0817654a90722, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:28.807084 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b4395a3e2928276d26c0817654a90722): No space left in cache to reserve 66780 bytes, will continue without cache download 2024.10.22 01:34:28.929687 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b4395a3e2928276d26c0817654a90722, offset: 0, size: 0 2024.10.22 01:34:28.931739 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b4395a3e2928276d26c0817654a90722 for removal 2024.10.22 01:34:28.941579 [ 1476 ] {} CacheMetadata: Key b4395a3e2928276d26c0817654a90722 is removed from metadata 2024.10.22 01:34:29.500702 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 113 entries to flush up to offset 63270 2024.10.22 01:34:29.555137 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_111_111_0 to 202410_114_114_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:29.700052 [ 1227 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 7 parts from 202410_1_104_21 to 202410_114_114_0 2024.10.22 01:34:29.742016 [ 1227 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 2.30 MiB using storage policy from min volume index 0 2024.10.22 01:34:29.748423 [ 1227 ] {} DiskLocal: Reserved 2.30 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:29.762656 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} MergeTask::PrepareStage: Merging 7 parts: from 202410_1_104_21 to 202410_114_114_0 into Compact with storage Full 2024.10.22 01:34:29.873235 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:29.878935 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:30.049286 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 19 file segments (while reserving 23999 bytes for c9ec7d2502e4828e70695b34ecd60494:0) 2024.10.22 01:34:30.157743 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fe08deaa10243f209d26c883747546a6, offset: 0, size: 176 2024.10.22 01:34:30.161325 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fe08deaa10243f209d26c883747546a6 for removal 2024.10.22 01:34:30.163646 [ 1476 ] {} CacheMetadata: Key fe08deaa10243f209d26c883747546a6 is removed from metadata 2024.10.22 01:34:30.166689 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 25caf1a0c80e6cef187bf7d3d89da67d, offset: 0, size: 3092 2024.10.22 01:34:30.169839 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 25caf1a0c80e6cef187bf7d3d89da67d for removal 2024.10.22 01:34:30.173785 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 24788cf759be05ff6059445d3592deab, offset: 0, size: 176 2024.10.22 01:34:30.179654 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 24788cf759be05ff6059445d3592deab for removal 2024.10.22 01:34:30.181124 [ 1476 ] {} CacheMetadata: Key 25caf1a0c80e6cef187bf7d3d89da67d is removed from metadata 2024.10.22 01:34:30.181739 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_543_543_0 (state Temporary) 2024.10.22 01:34:30.185402 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f162ed42e0c818f11810b0d8af0835e8, offset: 0, size: 1995 2024.10.22 01:34:30.190517 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f162ed42e0c818f11810b0d8af0835e8 for removal 2024.10.22 01:34:30.190622 [ 1476 ] {} CacheMetadata: Key 24788cf759be05ff6059445d3592deab is removed from metadata 2024.10.22 01:34:30.197498 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2030d4542d05b93d7658317fb52392d9, offset: 0, size: 176 2024.10.22 01:34:30.202542 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2030d4542d05b93d7658317fb52392d9 for removal 2024.10.22 01:34:30.207864 [ 1476 ] {} CacheMetadata: Key f162ed42e0c818f11810b0d8af0835e8 is removed from metadata 2024.10.22 01:34:30.210281 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 606fe65e99f93b45a3e620f83c1dd611, offset: 0, size: 149 2024.10.22 01:34:30.222547 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 606fe65e99f93b45a3e620f83c1dd611 for removal 2024.10.22 01:34:30.232648 [ 1476 ] {} CacheMetadata: Key 606fe65e99f93b45a3e620f83c1dd611 is removed from metadata 2024.10.22 01:34:30.234399 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: af12abfb1e720588dc6d5c7d8ec7f432, offset: 0, size: 177 2024.10.22 01:34:30.247998 [ 1476 ] {} CacheMetadata: Key 2030d4542d05b93d7658317fb52392d9 is removed from metadata 2024.10.22 01:34:30.252561 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key af12abfb1e720588dc6d5c7d8ec7f432 for removal 2024.10.22 01:34:30.263755 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5aa4d857afb0abd1d479056eac646eb5, offset: 0, size: 177 2024.10.22 01:34:30.263665 [ 1476 ] {} CacheMetadata: Key af12abfb1e720588dc6d5c7d8ec7f432 is removed from metadata 2024.10.22 01:34:30.267338 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5aa4d857afb0abd1d479056eac646eb5 for removal 2024.10.22 01:34:30.273387 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 53e8f90f9a23142a7be2e148b68366a6, offset: 0, size: 176 2024.10.22 01:34:30.278361 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 53e8f90f9a23142a7be2e148b68366a6 for removal 2024.10.22 01:34:30.283601 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7e05a2afc79eff99d25019ba38906d83, offset: 0, size: 176 2024.10.22 01:34:30.286780 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7e05a2afc79eff99d25019ba38906d83 for removal 2024.10.22 01:34:30.357564 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_543_543_0 to 202410_549_549_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:30.375084 [ 1476 ] {} CacheMetadata: Key 53e8f90f9a23142a7be2e148b68366a6 is removed from metadata 2024.10.22 01:34:30.397153 [ 1476 ] {} CacheMetadata: Key 5aa4d857afb0abd1d479056eac646eb5 is removed from metadata 2024.10.22 01:34:30.400263 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:34:30.407709 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4109b1948a7d0d13b3967da0003f953d, offset: 0, size: 147 2024.10.22 01:34:30.415567 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:34:30.419695 [ 1476 ] {} CacheMetadata: Key 7e05a2afc79eff99d25019ba38906d83 is removed from metadata 2024.10.22 01:34:30.422207 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4109b1948a7d0d13b3967da0003f953d for removal 2024.10.22 01:34:30.432372 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e8338d4daa48cb16543c668fc26bd9ad, offset: 0, size: 168 2024.10.22 01:34:30.447639 [ 1476 ] {} CacheMetadata: Key 4109b1948a7d0d13b3967da0003f953d is removed from metadata 2024.10.22 01:34:30.447270 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 63270 2024.10.22 01:34:30.448696 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e8338d4daa48cb16543c668fc26bd9ad for removal 2024.10.22 01:34:30.462976 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e1b67430688da02cc83e1db3d507d9e8, offset: 0, size: 161 2024.10.22 01:34:30.469116 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e1b67430688da02cc83e1db3d507d9e8 for removal 2024.10.22 01:34:30.473791 [ 1476 ] {} CacheMetadata: Key e8338d4daa48cb16543c668fc26bd9ad is removed from metadata 2024.10.22 01:34:30.566610 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 105 entries to flush up to offset 61808 2024.10.22 01:34:30.573341 [ 1476 ] {} CacheMetadata: Key e1b67430688da02cc83e1db3d507d9e8 is removed from metadata 2024.10.22 01:34:30.575254 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dd06c3346d8610d4b2785a969bdc3b3d, offset: 0, size: 7422 2024.10.22 01:34:30.578827 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dd06c3346d8610d4b2785a969bdc3b3d for removal 2024.10.22 01:34:30.585349 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: afbf07f93956de391ce213f1335a0783, offset: 0, size: 147 2024.10.22 01:34:30.590040 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key afbf07f93956de391ce213f1335a0783 for removal 2024.10.22 01:34:30.601365 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 591031ebba18819b7d419e3c943fe0d5, offset: 0, size: 207 2024.10.22 01:34:30.706946 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 591031ebba18819b7d419e3c943fe0d5 for removal 2024.10.22 01:34:30.712513 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a45516b7baa25a5414ef3398f6925dd6, offset: 0, size: 20500 2024.10.22 01:34:30.719007 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a45516b7baa25a5414ef3398f6925dd6 for removal 2024.10.22 01:34:30.720816 [ 1476 ] {} CacheMetadata: Key 591031ebba18819b7d419e3c943fe0d5 is removed from metadata 2024.10.22 01:34:30.725767 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 672347461f499c145aa0ec19b92e6003, offset: 0, size: 165 2024.10.22 01:34:30.728712 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 672347461f499c145aa0ec19b92e6003 for removal 2024.10.22 01:34:30.736792 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0c1b733ecee4941adfc6a7d01581279b, offset: 0, size: 177 2024.10.22 01:34:30.741327 [ 1476 ] {} CacheMetadata: Key 672347461f499c145aa0ec19b92e6003 is removed from metadata 2024.10.22 01:34:30.746975 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0c1b733ecee4941adfc6a7d01581279b for removal 2024.10.22 01:34:30.856287 [ 1476 ] {} CacheMetadata: Key 0c1b733ecee4941adfc6a7d01581279b is removed from metadata 2024.10.22 01:34:30.962184 [ 1476 ] {} CacheMetadata: Key a45516b7baa25a5414ef3398f6925dd6 is removed from metadata 2024.10.22 01:34:30.975634 [ 1476 ] {} CacheMetadata: Key afbf07f93956de391ce213f1335a0783 is removed from metadata 2024.10.22 01:34:30.992753 [ 1476 ] {} CacheMetadata: Key dd06c3346d8610d4b2785a969bdc3b3d is removed from metadata 2024.10.22 01:34:31.007758 [ 1698 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/htc/gejaewyruozfpktlzpwmcjsifkijw, size 23999 2024.10.22 01:34:31.092692 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:31.098858 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:31.201437 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_935_935_0 (state Temporary) 2024.10.22 01:34:31.270475 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_935_935_0 to 202410_948_948_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:31.297332 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:34:31.313675 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:34:31.328387 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61808 2024.10.22 01:34:31.425258 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:34:31.443531 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:34:31.462840 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:34:31.579550 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:34:31.602567 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:34:31.624240 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 01:34:31.777182 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tuw/cxgwqanxofsohocnwoatwvvqcqfpg (LogSeriesLimiter: on interval from 2024-10-22 01:34:23 to 2024-10-22 01:34:31 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:34:31.928591 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dbd52a5691e2b92d7cfb6bd0506d935b) : [0, 56559]: Setting download as failed: File segment: [0, 56559], key: dbd52a5691e2b92d7cfb6bd0506d935b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:31.931035 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dbd52a5691e2b92d7cfb6bd0506d935b): No space left in cache to reserve 56560 bytes, will continue without cache download 2024.10.22 01:34:31.954852 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dbd52a5691e2b92d7cfb6bd0506d935b, offset: 0, size: 0 2024.10.22 01:34:32.060760 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dbd52a5691e2b92d7cfb6bd0506d935b for removal 2024.10.22 01:34:32.063714 [ 1476 ] {} CacheMetadata: Key dbd52a5691e2b92d7cfb6bd0506d935b is removed from metadata 2024.10.22 01:34:32.120480 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4461 2024.10.22 01:34:32.933838 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 15.05 MiB, will set to 1.71 GiB (RSS), difference: 13.25 MiB 2024.10.22 01:34:33.139899 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(336cda1c75471ac09545594d4ac1303b) : [0, 90975]: Setting download as failed: File segment: [0, 90975], key: 336cda1c75471ac09545594d4ac1303b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:33.141756 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(336cda1c75471ac09545594d4ac1303b): No space left in cache to reserve 90976 bytes, will continue without cache download 2024.10.22 01:34:33.158259 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 336cda1c75471ac09545594d4ac1303b, offset: 0, size: 0 2024.10.22 01:34:33.162716 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 336cda1c75471ac09545594d4ac1303b for removal 2024.10.22 01:34:33.165327 [ 1476 ] {} CacheMetadata: Key 336cda1c75471ac09545594d4ac1303b is removed from metadata 2024.10.22 01:34:33.465928 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 8547 2024.10.22 01:34:33.684698 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:33.688085 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:33.736065 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_554_554_0 (state Temporary) 2024.10.22 01:34:33.893046 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_554_554_0 to 202410_567_567_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:33.984963 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8547 2024.10.22 01:34:34.057172 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 466 2024.10.22 01:34:34.156729 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_837_430, 202410_838_843_1] 2024.10.22 01:34:34.166282 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_837_430, 202410_838_843_1] 2024.10.22 01:34:34.178225 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_837_430, 202410_838_843_1] 2024.10.22 01:34:34.455489 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6e6cbcca9be6acd567c51580492e6af4) : [0, 168370]: Setting download as failed: File segment: [0, 168370], key: 6e6cbcca9be6acd567c51580492e6af4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:34.459561 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6e6cbcca9be6acd567c51580492e6af4): No space left in cache to reserve 168371 bytes, will continue without cache download 2024.10.22 01:34:34.478282 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6e6cbcca9be6acd567c51580492e6af4, offset: 0, size: 0 2024.10.22 01:34:34.480548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6e6cbcca9be6acd567c51580492e6af4 for removal 2024.10.22 01:34:34.485410 [ 1476 ] {} CacheMetadata: Key 6e6cbcca9be6acd567c51580492e6af4 is removed from metadata 2024.10.22 01:34:34.756296 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:34.758933 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:34.834401 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_448_448_0 (state Temporary) 2024.10.22 01:34:35.106880 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_448_448_0 to 202410_458_458_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:35.163310 [ 1219 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_453_93 to 202410_458_458_0 2024.10.22 01:34:35.174994 [ 1219 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:35.279314 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:35.332675 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2986d8decd56d7a80bf5a24eb4351c5e) : [0, 136550]: Setting download as failed: File segment: [0, 136550], key: 2986d8decd56d7a80bf5a24eb4351c5e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:35.363913 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 61878 2024.10.22 01:34:35.428027 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_453_93 to 202410_458_458_0 into Compact with storage Full 2024.10.22 01:34:35.440400 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2986d8decd56d7a80bf5a24eb4351c5e): No space left in cache to reserve 136551 bytes, will continue without cache download 2024.10.22 01:34:35.480633 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_844_844_0, 202410_845_845_0, 202410_846_846_0, 202410_847_847_0, 202410_848_848_0, 202410_849_849_0] 2024.10.22 01:34:35.482847 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_844_844_0, 202410_845_845_0, 202410_846_846_0, 202410_847_847_0, 202410_848_848_0, 202410_849_849_0] 2024.10.22 01:34:35.523549 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_844_844_0, 202410_845_845_0, 202410_846_846_0, 202410_847_847_0, 202410_848_848_0, 202410_849_849_0] 2024.10.22 01:34:35.563857 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2986d8decd56d7a80bf5a24eb4351c5e, offset: 0, size: 0 2024.10.22 01:34:35.565061 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2986d8decd56d7a80bf5a24eb4351c5e for removal 2024.10.22 01:34:35.568252 [ 1476 ] {} CacheMetadata: Key 2986d8decd56d7a80bf5a24eb4351c5e is removed from metadata 2024.10.22 01:34:35.611820 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 466 2024.10.22 01:34:35.615331 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_461_486_5, 202410_487_487_0, 202410_488_488_0, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0] 2024.10.22 01:34:35.621836 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_461_486_5, 202410_487_487_0, 202410_488_488_0, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0] 2024.10.22 01:34:35.655675 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:35.662337 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_461_486_5, 202410_487_487_0, 202410_488_488_0, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0] 2024.10.22 01:34:35.668993 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_453_93, total 497 rows starting from the beginning of the part 2024.10.22 01:34:35.702852 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:35.706476 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:35.756576 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_936_936_0 (state Temporary) 2024.10.22 01:34:35.807947 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 816 entries to flush up to offset 837100 2024.10.22 01:34:35.820088 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_936_936_0 to 202410_949_949_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:35.840697 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_454_454_0, total 1 rows starting from the beginning of the part 2024.10.22 01:34:35.871811 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61878 2024.10.22 01:34:35.915131 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_455_455_0, total 1 rows starting from the beginning of the part 2024.10.22 01:34:36.111158 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_456_456_0, total 1 rows starting from the beginning of the part 2024.10.22 01:34:36.192422 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_457_457_0, total 1 rows starting from the beginning of the part 2024.10.22 01:34:36.406836 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_458_458_0, total 1 rows starting from the beginning of the part 2024.10.22 01:34:36.558301 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(68953fea7ae377db882c1f6cb0e9ebed) : [0, 161534]: Setting download as failed: File segment: [0, 161534], key: 68953fea7ae377db882c1f6cb0e9ebed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:36.561531 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(68953fea7ae377db882c1f6cb0e9ebed): No space left in cache to reserve 161535 bytes, will continue without cache download 2024.10.22 01:34:36.694912 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:36.703653 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 68953fea7ae377db882c1f6cb0e9ebed, offset: 0, size: 0 2024.10.22 01:34:36.831527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 68953fea7ae377db882c1f6cb0e9ebed for removal 2024.10.22 01:34:36.839730 [ 1476 ] {} CacheMetadata: Key 68953fea7ae377db882c1f6cb0e9ebed is removed from metadata 2024.10.22 01:34:36.984023 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:36.986855 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:37.115297 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_531_531_0 (state Temporary) 2024.10.22 01:34:37.333381 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_531_531_0 to 202410_538_538_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:37.375006 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 837100 2024.10.22 01:34:37.402594 [ 1266 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_533_533_0 to 202410_538_538_0 2024.10.22 01:34:37.443022 [ 1266 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:37.450661 [ 1266 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:37.488205 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_532_106/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:34:37.604710 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 12.93 MiB, will set to 1.71 GiB (RSS), difference: -7.71 MiB 2024.10.22 01:34:37.663148 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_533_533_0 to 202410_538_538_0 into Compact with storage Full 2024.10.22 01:34:37.667329 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_104_21, total 4112 rows starting from the beginning of the part 2024.10.22 01:34:37.684317 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:37.697266 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_533_533_0, total 1927 rows starting from the beginning of the part 2024.10.22 01:34:37.777596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(02244d1c0aeeb577394834cca1ce9e29) : [0, 401202]: Setting download as failed: File segment: [0, 401202], key: 02244d1c0aeeb577394834cca1ce9e29, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:37.779422 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(02244d1c0aeeb577394834cca1ce9e29): No space left in cache to reserve 401203 bytes, will continue without cache download 2024.10.22 01:34:37.864863 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_534_534_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:34:37.949212 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_535_535_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:34:37.974287 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 63 entries to flush up to offset 63333 2024.10.22 01:34:37.996725 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_536_536_0, total 1542 rows starting from the beginning of the part 2024.10.22 01:34:38.054913 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 02244d1c0aeeb577394834cca1ce9e29, offset: 0, size: 0 2024.10.22 01:34:38.055469 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_537_537_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:34:38.056727 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 02244d1c0aeeb577394834cca1ce9e29 for removal 2024.10.22 01:34:38.059986 [ 1476 ] {} CacheMetadata: Key 02244d1c0aeeb577394834cca1ce9e29 is removed from metadata 2024.10.22 01:34:38.093489 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_538_538_0, total 816 rows starting from the beginning of the part 2024.10.22 01:34:38.316544 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:38.322670 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:38.374428 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_544_544_0 (state Temporary) 2024.10.22 01:34:38.434460 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_544_544_0 to 202410_550_550_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:38.499893 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 63333 2024.10.22 01:34:38.708674 [ 1171 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:34:38.845139 [ 1171 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:34:38.859680 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 5 old parts to remove. Parts: [202410_1_474_95, 202410_475_475_0, 202410_476_476_0, 202410_477_477_0, 202410_478_478_0] 2024.10.22 01:34:38.862946 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 5 parts from filesystem (serially): Parts: [202410_1_474_95, 202410_475_475_0, 202410_476_476_0, 202410_477_477_0, 202410_478_478_0] 2024.10.22 01:34:38.902705 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 5 parts from memory: Parts: [202410_1_474_95, 202410_475_475_0, 202410_476_476_0, 202410_477_477_0, 202410_478_478_0] 2024.10.22 01:34:38.926450 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} MergeTask::MergeProjectionsStage: Merge sorted 502 rows, containing 32 columns (32 merged, 0 gathered) in 3.607890373 sec., 139.13948266188075 rows/sec., 12.86 KiB/sec. 2024.10.22 01:34:38.962450 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} MergedBlockOutputStream: filled checksums 202410_1_458_94 (state Temporary) 2024.10.22 01:34:39.091501 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7ab919df622e0cc734d4de75974abf7a) : [0, 854085]: Setting download as failed: File segment: [0, 854085], key: 7ab919df622e0cc734d4de75974abf7a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:39.101633 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7ab919df622e0cc734d4de75974abf7a): No space left in cache to reserve 854086 bytes, will continue without cache download 2024.10.22 01:34:39.258582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7ab919df622e0cc734d4de75974abf7a, offset: 0, size: 0 2024.10.22 01:34:39.261021 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7ab919df622e0cc734d4de75974abf7a for removal 2024.10.22 01:34:39.265548 [ 1476 ] {} CacheMetadata: Key 7ab919df622e0cc734d4de75974abf7a is removed from metadata 2024.10.22 01:34:39.483498 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 13.84 MiB, will set to 1.71 GiB (RSS), difference: -8.24 MiB 2024.10.22 01:34:39.684091 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 4498 2024.10.22 01:34:39.859491 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} MergeTask::MergeProjectionsStage: Merge sorted 8905 rows, containing 4 columns (4 merged, 0 gathered) in 2.394604096 sec., 3718.7775694842876 rows/sec., 58.11 KiB/sec. 2024.10.22 01:34:39.879826 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 61956 2024.10.22 01:34:39.897086 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} MergedBlockOutputStream: filled checksums 202410_533_538_1 (state Temporary) 2024.10.22 01:34:40.060704 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_458_94 to 202410_1_458_94 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:40.064574 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_458_94} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_453_93, 202410_458_458_0] -> 202410_1_458_94 2024.10.22 01:34:40.108355 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_533_538_1 to 202410_533_538_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:40.112336 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_533_538_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_533_533_0, 202410_538_538_0] -> 202410_533_538_1 2024.10.22 01:34:40.258203 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:34:40.258339 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_105_109_1, total 204 rows starting from the beginning of the part 2024.10.22 01:34:40.271911 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:34:40.402773 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:40.406437 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:34:40.411523 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:40.413225 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:34:40.488411 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_937_937_0 (state Temporary) 2024.10.22 01:34:40.667322 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_937_937_0 to 202410_950_950_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:40.752050 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 61956 2024.10.22 01:34:40.856836 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f7ca6e2cacdc5f45b3530dffc81f49e7) : [0, 183075]: Setting download as failed: File segment: [0, 183075], key: f7ca6e2cacdc5f45b3530dffc81f49e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:40.858233 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f7ca6e2cacdc5f45b3530dffc81f49e7): No space left in cache to reserve 183076 bytes, will continue without cache download 2024.10.22 01:34:40.883858 [ 1240 ] {} DatabaseCatalog: Execution took 296 ms. 2024.10.22 01:34:40.978243 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f7ca6e2cacdc5f45b3530dffc81f49e7, offset: 0, size: 0 2024.10.22 01:34:40.981421 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f7ca6e2cacdc5f45b3530dffc81f49e7 for removal 2024.10.22 01:34:40.984457 [ 1476 ] {} CacheMetadata: Key f7ca6e2cacdc5f45b3530dffc81f49e7 is removed from metadata 2024.10.22 01:34:41.490058 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 8567 2024.10.22 01:34:41.495628 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 15.30 MiB, will set to 1.71 GiB (RSS), difference: 13.27 MiB 2024.10.22 01:34:41.703327 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:41.706534 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:41.748622 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_555_555_0 (state Temporary) 2024.10.22 01:34:41.780576 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_555_555_0 to 202410_568_568_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:41.851762 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b21e1e94e9edd6e55c8dff31edd599b8) : [0, 32456]: Setting download as failed: File segment: [0, 32456], key: b21e1e94e9edd6e55c8dff31edd599b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:41.854534 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b21e1e94e9edd6e55c8dff31edd599b8): No space left in cache to reserve 32457 bytes, will continue without cache download 2024.10.22 01:34:41.870091 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b21e1e94e9edd6e55c8dff31edd599b8, offset: 0, size: 0 2024.10.22 01:34:41.871988 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b21e1e94e9edd6e55c8dff31edd599b8 for removal 2024.10.22 01:34:41.876440 [ 1476 ] {} CacheMetadata: Key b21e1e94e9edd6e55c8dff31edd599b8 is removed from metadata 2024.10.22 01:34:41.931673 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8567 2024.10.22 01:34:42.658486 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_110_110_0, total 42 rows starting from the beginning of the part 2024.10.22 01:34:43.113816 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 467 2024.10.22 01:34:43.119401 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5bd6127affa172731627558d38be6466) : [0, 1337873]: Setting download as failed: File segment: [0, 1337873], key: 5bd6127affa172731627558d38be6466, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:43.124102 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5bd6127affa172731627558d38be6466): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:34:43.372100 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5bd6127affa172731627558d38be6466, offset: 0, size: 0 2024.10.22 01:34:43.374340 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5bd6127affa172731627558d38be6466 for removal 2024.10.22 01:34:43.376356 [ 1476 ] {} CacheMetadata: Key 5bd6127affa172731627558d38be6466 is removed from metadata 2024.10.22 01:34:43.567140 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 13.30 MiB, will set to 1.71 GiB (RSS), difference: 2.15 MiB 2024.10.22 01:34:43.607499 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:43.610596 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:43.668317 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_449_449_0 (state Temporary) 2024.10.22 01:34:43.755167 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_449_449_0 to 202410_459_459_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:43.919486 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 467 2024.10.22 01:34:44.176759 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e384128a2536a91dc9c3a8b9d77ace48) : [0, 109920]: Setting download as failed: File segment: [0, 109920], key: e384128a2536a91dc9c3a8b9d77ace48, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:44.178839 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e384128a2536a91dc9c3a8b9d77ace48): No space left in cache to reserve 109921 bytes, will continue without cache download 2024.10.22 01:34:44.215118 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e384128a2536a91dc9c3a8b9d77ace48, offset: 0, size: 0 2024.10.22 01:34:44.219504 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e384128a2536a91dc9c3a8b9d77ace48 for removal 2024.10.22 01:34:44.223460 [ 1476 ] {} CacheMetadata: Key e384128a2536a91dc9c3a8b9d77ace48 is removed from metadata 2024.10.22 01:34:44.387072 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1543 entries to flush up to offset 838643 2024.10.22 01:34:44.596024 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_532_106} MergeTask::MergeProjectionsStage: Merge sorted 841785 rows, containing 4 columns (4 merged, 0 gathered) in 23.749504689 sec., 35444.31814571222 rows/sec., 531.53 KiB/sec. 2024.10.22 01:34:44.761558 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 62010 2024.10.22 01:34:44.939873 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_532_106} MergedBlockOutputStream: filled checksums 202410_1_532_106 (state Temporary) 2024.10.22 01:34:45.052998 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:45.056800 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:45.080507 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_111_111_0, total 40 rows starting from the beginning of the part 2024.10.22 01:34:45.204257 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_938_938_0 (state Temporary) 2024.10.22 01:34:45.377351 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f9522920eea8db37f4dcb8503d7382e2) : [0, 90285]: Setting download as failed: File segment: [0, 90285], key: f9522920eea8db37f4dcb8503d7382e2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:45.379708 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f9522920eea8db37f4dcb8503d7382e2): No space left in cache to reserve 90286 bytes, will continue without cache download 2024.10.22 01:34:45.422498 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f9522920eea8db37f4dcb8503d7382e2, offset: 0, size: 0 2024.10.22 01:34:45.425023 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f9522920eea8db37f4dcb8503d7382e2 for removal 2024.10.22 01:34:45.428654 [ 1476 ] {} CacheMetadata: Key f9522920eea8db37f4dcb8503d7382e2 is removed from metadata 2024.10.22 01:34:45.564853 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_938_938_0 to 202410_951_951_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:45.620199 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:45.625023 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:45.633297 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62010 2024.10.22 01:34:45.670343 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_940_473 to 202410_941_946_1 2024.10.22 01:34:45.677465 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.23 MiB using storage policy from min volume index 0 2024.10.22 01:34:45.684322 [ 1187 ] {} DiskLocal: Reserved 2.23 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:45.757566 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_532_532_0 (state Temporary) 2024.10.22 01:34:45.777575 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:34:45.811751 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_946_474} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_940_473 to 202410_941_946_1 into Compact with storage Full 2024.10.22 01:34:45.825778 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_532_532_0 to 202410_539_539_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:45.869680 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 838643 2024.10.22 01:34:45.884270 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Found 7 old parts to remove. Parts: [201403_1_1_0, 201403_2_2_0, 201403_3_3_0, 201403_4_4_0, 201403_5_5_0, 201403_6_6_0, 201403_7_7_0] 2024.10.22 01:34:45.886018 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Removing 7 parts from filesystem (serially): Parts: [201403_1_1_0, 201403_2_2_0, 201403_3_3_0, 201403_4_4_0, 201403_5_5_0, 201403_6_6_0, 201403_7_7_0] 2024.10.22 01:34:45.932956 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_946_474} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:46.012034 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 95 entries to flush up to offset 63428 2024.10.22 01:34:46.050048 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_946_474} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_940_473, total 62117 rows starting from the beginning of the part 2024.10.22 01:34:46.075935 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_946_474} MergeTreeSequentialSource: Reading 2 marks from part 202410_941_946_1, total 338 rows starting from the beginning of the part 2024.10.22 01:34:46.144981 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:46.150693 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:46.188028 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_545_545_0 (state Temporary) 2024.10.22 01:34:46.248493 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_545_545_0 to 202410_551_551_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:46.281144 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 63428 2024.10.22 01:34:46.475635 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 12.88 MiB, will set to 1.71 GiB (RSS), difference: 3.00 MiB 2024.10.22 01:34:46.756642 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_112_112_0, total 38 rows starting from the beginning of the part 2024.10.22 01:34:46.826681 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(48ce9cd8e752f4056dac2b97f7c1dd68) : [0, 86244]: Setting download as failed: File segment: [0, 86244], key: 48ce9cd8e752f4056dac2b97f7c1dd68, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:46.828380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(48ce9cd8e752f4056dac2b97f7c1dd68): No space left in cache to reserve 86245 bytes, will continue without cache download 2024.10.22 01:34:46.864522 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 48ce9cd8e752f4056dac2b97f7c1dd68, offset: 0, size: 0 2024.10.22 01:34:46.866565 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 48ce9cd8e752f4056dac2b97f7c1dd68 for removal 2024.10.22 01:34:46.868464 [ 1476 ] {} CacheMetadata: Key 48ce9cd8e752f4056dac2b97f7c1dd68 is removed from metadata 2024.10.22 01:34:46.949840 [ 1264 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 250 ms. 2024.10.22 01:34:47.955749 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(905e3e1d29f3083d31db4f49c1a1f53c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 905e3e1d29f3083d31db4f49c1a1f53c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:34:47.961340 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 33231, file segment info: File segment: [0, 33554431], key: 905e3e1d29f3083d31db4f49c1a1f53c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:34:47.963543 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:34:48.072436 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_532_106} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_532_106 to 202410_1_532_106 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:48.075890 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_532_106} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_526_105, 202410_527_532_1] -> 202410_1_532_106 2024.10.22 01:34:48.090645 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:34:48.103477 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_113_113_0, total 43 rows starting from the beginning of the part 2024.10.22 01:34:48.990697 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9eca872c1deab0b0aab2ad0c1571c674) : [0, 130933]: Setting download as failed: File segment: [0, 130933], key: 9eca872c1deab0b0aab2ad0c1571c674, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:48.995761 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9eca872c1deab0b0aab2ad0c1571c674): No space left in cache to reserve 130934 bytes, will continue without cache download 2024.10.22 01:34:49.027841 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9eca872c1deab0b0aab2ad0c1571c674, offset: 0, size: 0 2024.10.22 01:34:49.037200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9eca872c1deab0b0aab2ad0c1571c674 for removal 2024.10.22 01:34:49.047824 [ 1476 ] {} CacheMetadata: Key 9eca872c1deab0b0aab2ad0c1571c674 is removed from metadata 2024.10.22 01:34:49.070627 [ 1747 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tuw/cxgwqanxofsohocnwoatwvvqcqfpg, size 33231 2024.10.22 01:34:49.182764 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_114_114_0, total 44 rows starting from the beginning of the part 2024.10.22 01:34:49.439196 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 8575 2024.10.22 01:34:49.646813 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 62067 2024.10.22 01:34:49.684834 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:49.693052 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:49.774690 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:34:49.806028 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:34:49.843993 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_556_556_0 (state Temporary) 2024.10.22 01:34:49.852933 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:34:49.877202 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:34:49.914736 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:34:49.926870 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_556_556_0 to 202410_569_569_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:49.961702 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:34:49.983499 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:34:49.999774 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8575 2024.10.22 01:34:50.017386 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column IsMobile 2024.10.22 01:34:50.092333 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:50.098105 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:50.213051 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_939_939_0 (state Temporary) 2024.10.22 01:34:50.271904 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_946_474} MergeTask::MergeProjectionsStage: Merge sorted 62455 rows, containing 13 columns (13 merged, 0 gathered) in 4.572711539 sec., 13658.198088230642 rows/sec., 2.17 MiB/sec. 2024.10.22 01:34:50.325072 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_946_474} MergedBlockOutputStream: filled checksums 202410_1_946_474 (state Temporary) 2024.10.22 01:34:50.402573 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:34:50.408903 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:34:50.414082 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b414b7d42c6b5fd27ae3c9a64e4e0870) : [0, 77093]: Setting download as failed: File segment: [0, 77093], key: b414b7d42c6b5fd27ae3c9a64e4e0870, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:50.417835 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b414b7d42c6b5fd27ae3c9a64e4e0870): No space left in cache to reserve 77094 bytes, will continue without cache download 2024.10.22 01:34:50.440963 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yfu/uevrmbejvrcwpcqlzqikesgucqtje (LogSeriesLimiter: on interval from 2024-10-22 01:34:31 to 2024-10-22 01:34:50 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:34:50.466595 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b414b7d42c6b5fd27ae3c9a64e4e0870, offset: 0, size: 0 2024.10.22 01:34:50.468282 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b414b7d42c6b5fd27ae3c9a64e4e0870 for removal 2024.10.22 01:34:50.471191 [ 1476 ] {} CacheMetadata: Key b414b7d42c6b5fd27ae3c9a64e4e0870 is removed from metadata 2024.10.22 01:34:50.429743 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_946_474} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_946_474 to 202410_1_946_474 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:50.545578 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_946_474} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_940_473, 202410_941_946_1] -> 202410_1_946_474 2024.10.22 01:34:50.547112 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_939_939_0 to 202410_952_952_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:50.568886 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_946_474 to 202410_947_947_0 2024.10.22 01:34:50.573205 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.23 MiB using storage policy from min volume index 0 2024.10.22 01:34:50.576882 [ 1216 ] {} DiskLocal: Reserved 2.23 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:50.578344 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.88 MiB. 2024.10.22 01:34:50.612554 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62067 2024.10.22 01:34:50.620874 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_947_475} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_946_474 to 202410_947_947_0 into Compact with storage Full 2024.10.22 01:34:50.667283 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_947_475} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:50.678481 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_947_475} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_946_474, total 62455 rows starting from the beginning of the part 2024.10.22 01:34:50.713172 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_947_475} MergeTreeSequentialSource: Reading 2 marks from part 202410_947_947_0, total 79 rows starting from the beginning of the part 2024.10.22 01:34:50.776150 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 6.93 MiB, will set to 1.71 GiB (RSS), difference: 2.47 MiB 2024.10.22 01:34:51.428293 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 468 2024.10.22 01:34:51.664333 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_947_475/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:34:51.767285 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:51.773155 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:51.836381 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_450_450_0 (state Temporary) 2024.10.22 01:34:51.912367 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_450_450_0 to 202410_460_460_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:51.950706 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(733c1f9423c51327730acb97e5e22f26) : [0, 317337]: Setting download as failed: File segment: [0, 317337], key: 733c1f9423c51327730acb97e5e22f26, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:51.955182 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(733c1f9423c51327730acb97e5e22f26): No space left in cache to reserve 317338 bytes, will continue without cache download 2024.10.22 01:34:51.990661 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 468 2024.10.22 01:34:52.074613 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 733c1f9423c51327730acb97e5e22f26, offset: 0, size: 0 2024.10.22 01:34:52.077092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 733c1f9423c51327730acb97e5e22f26 for removal 2024.10.22 01:34:52.079908 [ 1476 ] {} CacheMetadata: Key 733c1f9423c51327730acb97e5e22f26 is removed from metadata 2024.10.22 01:34:52.179240 [ 1733 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ayl/bifobpyjvkgszdqoqehehmajxtxtd, size 508 2024.10.22 01:34:52.459618 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:34:52.470668 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:34:52.486460 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:34:52.510996 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:34:52.533561 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:34:52.556755 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:34:52.579732 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:34:52.611043 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:34:52.650673 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:34:52.873559 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1491 entries to flush up to offset 840134 2024.10.22 01:34:53.583680 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 3.72 MiB, will set to 1.72 GiB (RSS), difference: -10.04 MiB 2024.10.22 01:34:53.624968 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:53.629398 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:53.763529 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_533_533_0 (state Temporary) 2024.10.22 01:34:53.795784 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 145 entries to flush up to offset 63573 2024.10.22 01:34:53.852344 [ 1236 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:34:53.884344 [ 1236 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:34:53.905634 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:53.911478 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:53.930378 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_533_533_0 to 202410_540_540_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:53.976003 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 840134 2024.10.22 01:34:54.200446 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_546_546_0 (state Temporary) 2024.10.22 01:34:54.377146 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_546_546_0 to 202410_552_552_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:54.488733 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_947_475} MergeTask::MergeProjectionsStage: Merge sorted 62534 rows, containing 13 columns (13 merged, 0 gathered) in 3.888282767 sec., 16082.678073397432 rows/sec., 2.56 MiB/sec. 2024.10.22 01:34:54.555904 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_947_475} MergedBlockOutputStream: filled checksums 202410_1_947_475 (state Temporary) 2024.10.22 01:34:54.569247 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 63573 2024.10.22 01:34:54.632790 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 62146 2024.10.22 01:34:54.576322 [ 1222 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_547_8 to 202410_552_552_0 2024.10.22 01:34:54.716701 [ 1222 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:54.720290 [ 1222 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:54.728991 [ 1222 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 325 ms. 2024.10.22 01:34:54.762679 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_547_8 to 202410_552_552_0 into Compact with storage Full 2024.10.22 01:34:54.864519 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_947_475} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_947_475 to 202410_1_947_475 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:54.868588 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_947_475} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_946_474, 202410_947_947_0] -> 202410_1_947_475 2024.10.22 01:34:54.889260 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:34:54.911306 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:54.928688 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_547_8, total 3956 rows starting from the beginning of the part 2024.10.22 01:34:54.970686 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:54.974364 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:55.077431 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_548_548_0, total 76 rows starting from the beginning of the part 2024.10.22 01:34:55.127832 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_549_549_0, total 113 rows starting from the beginning of the part 2024.10.22 01:34:55.158171 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_940_940_0 (state Temporary) 2024.10.22 01:34:55.236012 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_940_940_0 to 202410_953_953_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:55.273046 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_550_550_0, total 63 rows starting from the beginning of the part 2024.10.22 01:34:55.274774 [ 1217 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_947_475 to 202410_948_948_0 2024.10.22 01:34:55.278388 [ 1217 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.24 MiB using storage policy from min volume index 0 2024.10.22 01:34:55.280648 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62146 2024.10.22 01:34:55.281171 [ 1217 ] {} DiskLocal: Reserved 2.24 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:55.290502 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 3.23 MiB, will set to 1.71 GiB (RSS), difference: -1.04 MiB 2024.10.22 01:34:55.291500 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_948_476} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_947_475 to 202410_948_948_0 into Compact with storage Full 2024.10.22 01:34:55.360983 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_114_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:34:55.426942 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_551_551_0, total 95 rows starting from the beginning of the part 2024.10.22 01:34:55.467271 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_552_552_0, total 145 rows starting from the beginning of the part 2024.10.22 01:34:55.532512 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_948_476} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:34:55.539889 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_948_476} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_947_475, total 62534 rows starting from the beginning of the part 2024.10.22 01:34:55.565577 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_948_476} MergeTreeSequentialSource: Reading 2 marks from part 202410_948_948_0, total 105 rows starting from the beginning of the part 2024.10.22 01:34:56.418683 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 2.48 MiB, will set to 1.71 GiB (RSS), difference: -17.06 MiB 2024.10.22 01:34:56.732785 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:56.735796 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:56.865097 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} MergeTask::MergeProjectionsStage: Merge sorted 4448 rows, containing 13 columns (13 merged, 0 gathered) in 2.096321616 sec., 2121.8118279423397 rows/sec., 939.40 KiB/sec. 2024.10.22 01:34:57.029809 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} MergedBlockOutputStream: filled checksums 202410_507_552_9 (state Temporary) 2024.10.22 01:34:57.331591 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_552_9 to 202410_507_552_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:57.336659 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_552_9} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_547_8, 202410_552_552_0] -> 202410_507_552_9 2024.10.22 01:34:57.477659 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.04 MiB. 2024.10.22 01:34:57.504194 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 8594 2024.10.22 01:34:57.649278 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:57.653219 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:34:57.682300 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_557_557_0 (state Temporary) 2024.10.22 01:34:57.842685 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_557_557_0 to 202410_570_570_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:57.999749 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8594 2024.10.22 01:34:58.179648 [ 1272 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 215 ms. 2024.10.22 01:34:58.208630 [ 1150 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:34:58.746825 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_948_476} MergeTask::MergeProjectionsStage: Merge sorted 62639 rows, containing 13 columns (13 merged, 0 gathered) in 3.460984578 sec., 18098.60708370946 rows/sec., 2.88 MiB/sec. 2024.10.22 01:34:58.817197 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_948_476} MergedBlockOutputStream: filled checksums 202410_1_948_476 (state Temporary) 2024.10.22 01:34:58.870872 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 2.08 MiB, will set to 1.71 GiB (RSS), difference: 7.07 MiB 2024.10.22 01:34:58.947798 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_948_476} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_948_476 to 202410_1_948_476 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:34:58.951301 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_948_476} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_947_475, 202410_948_948_0] -> 202410_1_948_476 2024.10.22 01:34:59.000552 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:34:59.235271 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_112_112_0 (state Temporary) 2024.10.22 01:34:59.385379 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 62199 2024.10.22 01:34:59.471006 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bffb2567b00073882d01a1fe42ee0130) : [0, 505385]: Setting download as failed: File segment: [0, 505385], key: bffb2567b00073882d01a1fe42ee0130, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:34:59.472122 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bffb2567b00073882d01a1fe42ee0130): No space left in cache to reserve 505386 bytes, will continue without cache download 2024.10.22 01:34:59.615324 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bffb2567b00073882d01a1fe42ee0130, offset: 0, size: 0 2024.10.22 01:34:59.616812 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bffb2567b00073882d01a1fe42ee0130 for removal 2024.10.22 01:34:59.621003 [ 1476 ] {} CacheMetadata: Key bffb2567b00073882d01a1fe42ee0130 is removed from metadata 2024.10.22 01:34:59.803993 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:34:59.811427 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 01:35:00.011289 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_941_941_0 (state Temporary) 2024.10.22 01:35:00.157310 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_941_941_0 to 202410_954_954_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:00.175620 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_948_476 to 202410_949_949_0 2024.10.22 01:35:00.175227 [ 1418 ] {} S3ObjectStorage: Objects with paths [common/crq/sdiclgltunegmohkjazshopdzbzxv, common/xep/bhtzhjbhxwlisrwynhpayyoklburi, common/hfy/apbqyxshqttczerhaggfmchbyifsb, common/mju/zpwslpodyzgeywhwnpkgwzeviosfg, common/yvc/qepdawunmmnotkeqxberruztfgizt, common/ojs/cjkdubskwrmqzkwpaxulfetbhzuhv, common/tnz/cntkajkdnrnftzljlomkpvkonyvii, common/rgj/reycjihdxpvwpstrqdeytvmkwgywh, common/lvw/tsrpmdnjzxoiaasqlbiqtqirgeleh, common/btm/vpfocdszwwsusarccdfipdtgtqnhv, common/qkm/lpfvtpnnvgedrcdkwflpvjbddpypo, common/hsj/fqnzeaudshmroiecmahcdhdoaugzb, common/beh/jeqxbyxtzqwxodgtpqwmenfpdxdan, common/qnh/bftpfppbytwfczindmguspivuaewe, common/uqs/bikjmqsciroancrgdpbnkqlxteaua, common/plz/rytcwxnwosssxjycbyqnkyvsiklzp, common/pjn/bvfzfsebtbyojnoiubdyepxfswqdp, common/jqq/pkbdaitfzshhkashjoauqivyfwkta, common/hfk/rnfnzmgdeyvamerfjiunvpwclebre, common/eys/iyxptzyurbhvlpxjwussdmxsosjjy, common/imy/yymrgsbdrwawyzxldmbykvmtbjvoz, common/rfd/rbgxbmorxsjxvjkmrlydxmjjmjabp, common/sqk/ydatqrfzlaettbippdtcxwwyhgvry, common/fct/lpjmfcieawljqvnoaualskxgfkakd, common/pvr/qqldxdupezycrhtiskpiopafyvqfn, common/hqu/bztjrihembuxtuaemdkhnbefyisuy, common/hha/govkabbuefikfxccvndfbtcwqqpyf, common/usz/rxrtwwadzgrrhbowqepeeoesrssyz, common/gfu/qhercuygwwandjdiwbyhfwzosliwu, common/pwy/kmbfagdneqstiqhjhzwhzmombrzar, common/bgr/bveelryqibyeyvpwdkjpqjknezyxp, common/ynn/kivbulebaqqsnmisvrrixijnzbynw, common/zka/zhelcitcoqytnnpekrwgxxgofcswx, common/juy/rknqmznvfieansqpmquedoiithjoc, common/qfk/hfgklhlzpnostabjmpknolnpjmxfi, common/hqb/hdlsdjugyjgaraqicqebmvquwrbih, common/ypr/clrogcahasrnzjcpfgwitbpguyqsm, common/ezw/imfoletlcyzqmzxijrhqxmzqojzbx, common/aph/nccygtmvzzlkswsofkqzuytnszgxo, common/fav/dndidkpmsmaqvrdwdptiqdstkbfkj, common/mmg/bokynrbzjtzmevkisbbfmmvlegntv, common/zqe/nsilivfmxvtszoosptlaacavdrrnz, common/xlz/fjezbzktzqwmnogrlhecojhkkmzse, common/ujf/dxqjhkmcpbqzjkbrhhjotuofmefxm, common/ikw/gbeupkcitowvpyueekqzyfmlnefen, common/jgp/oqrwaykudjkfbtuvquokqibrnbaud, common/zkp/pybzytjegmrkusaooxxcofohebllu, common/pbo/ceymfnlzofmiryypzxcmubpeipkjg, common/vqq/fwxnxsymzfjehmjkoqtuqosgvgitd, common/utv/uijpytgmwjtpbaxjjoplbiqauxnfn, common/jfd/zzywkbaxvaemzfvmifdyuyepvfnjv, common/hta/gzlyzfzsmqrhwjkomxxlpqelmvavy, common/stl/ugxmitnjcmzpsaedhhkmmqnxsiqem, common/coh/ooennorcakkhhzxzakacqckfpvosp, common/mdz/jwzfdldpgdkxbrlrhpaaekbmuoope, common/vgk/dzjqqslqtnptmkcfqmgumubyxggle, common/efg/uedqnxmmvzpeuceemokiknmoevxgc, common/oij/hjwpbwfnvadymrwpcrguhwuwcjhzn, common/aag/ghbjznawetygtmtupqaeijowrdmtt, common/kxp/gcawakfdahmlgbjftgaitufakejuc, common/mzf/nkjhsynblgtwjbmkcpkkiosdbnbur, common/vnw/kvlqzhvuqxftjotdxvnqwvwcgdysf, common/gel/dopasbcdreiwihxbxuzlaqboqzges, common/izx/ykhjfpmpjuqrbsaiosxuaqbzwffji, common/xzk/xehojxrzjiayagiicwcikynkaimrt, common/msh/phkpppnaqfrcqbpjqsxfcquvrlifl, common/psr/eawbfmydlqdshhsbroatvibobqedj, common/sqq/auodggzszjvatzhrffcyprwbxlcze, common/gnt/isekbkvvexgrzmftpbxiiqxtbcmes, common/gtb/egzcrgerwucwkvivnyuclggjqrbgk, common/gus/fxisrbkkcxwnldqhzbsdokgqhprrn, common/xxz/kasehcuzqinbnlyexdipntkibiqou, common/ydi/rbsfalfkgfizfyhxzkwyxsmocoktm, common/nzc/vobylhtcvrcuocxhdmxfspyqzojwy, common/nvz/spvfvusdcwzzdjtggygrwkxotuzas, common/jya/vkunrstazdglohzogfxdtsrfbgcfn, common/qpr/gjkqfgexqwvqjppihzpuvqqnsfclo, common/arp/ffhhhmczddbmvkfkrwjjromjnbvha, common/jtz/kbpgnxckrtwecchjyuccfsenkrqnn, common/rkc/mwuknxnnsewpxsnxvphlxwugleibr, common/vir/cqwgselvihcnsjimchxtkjbzmeaih, common/hae/wbosojoptacrhhcfrxrdehpgameam, common/tiz/wohqqutalvscigcmmnfcfccpbhqva, common/jhc/xqcvlqhxxdlysntjxeizaoyavgcjk, common/byn/rxuyevxbyfhrwedtnymncqzydhkda, common/yjb/wmrmwbawbnghdisrwvhgfckagnovn, common/svf/tzwthcpcbeyeanhwvxeaaebsqnwai, common/dev/jgvhchlucnpzqijmfpcuhiwmvufiz, common/otn/hovaknammiyxghgjeswmolsbtjsnq, common/chk/naobptpsckzpldvewxjxrwqqcxbut, common/ddd/ryhmtnuwvrjgddlqojcuwdnpahodf, common/gqc/hdneqauuprmuvsgtrciyovjafjffg, common/xqo/rrfqzakekiocrtjpamxsnmlkoqxwh, common/cyg/vhqmmbhmsurdcavtndfeqrpzrmdjs, common/kfl/rwqiazqlbafkskoxkcfxfzqgxmumd, common/jou/qdmbdpxvwjoobekymtfsremyasozj, common/nqw/gbgbcjmjoxkgqbvievgsrmtyhhnmn, common/ebr/ecgxxwdkpvqssarmlikgkhjsxkrvk, common/wfj/zyzznxklldufyirxwnbgmrmfkkuwm, common/jvg/hcqxinsewqyvbxcremodlggfupmtx, common/xzj/scycvdiysncausvrbzdbywlnizjiq, common/ebs/szpweugkjhylejgkafpmqjuprndfo, common/bhz/wmremkptxuoippdtytnlqmefvrmbh, common/aal/psgafooxnwjpyibqozpyyitftzxgm, common/orw/aqqogfykyblxqbrjjkvxpzkljytfq, common/pwu/mzpddoddfzqftxzxruejlysihopti, common/ipp/ikvpldtgnbcpaovzuitavfoxmhavs, common/aid/mujcyptaciswqhixaabjnpdzwyiqo, common/fif/uzwbaqwexpikcvumwymlbowyjppoa, common/pxe/mtjzukhhkjqjppvqzyktjehewuevj, common/qav/wlvypqgwtbdeegokkbhwszumiwbtj, common/scg/geotppbyftlhnfrzgrbjzpvoitisi, common/yoh/oohkjagkclpmjhwfcmbclysiwdwol, common/zcu/jgfatobbuzroggdfgqcsbuxvpfgpt, common/tnx/vxjtmouigcdhxwmkoahyjtobhlfnx, common/zon/iobcvdjkbyqyqgniumkclbrphyjhn, common/txk/yxngekpvtwksstmvkmtpeolmqbufy, common/tcs/goecyowcmoifaekucqsttthjgyube, common/bzv/aeteoqcjknpyejwytlilhlyrwiolk, common/ywj/dytbawtmjbjpmudmvqzsvdjrjywtf, common/fxx/mbdksjutnjdschuaeaqviqodbrvzt, common/arh/swsaontrbfmsiajgzmpmqntrjtrgs, common/zpk/rhcaunpsxbsqmkxyxiupuwazlumwa, common/igv/dsugnlomcaqzmheuiohnnvjhuqcgj, common/cww/mmbnkgdiralgcphjzhohelyvznsmx, common/kmc/zzpuofhkudsasinxznjrgjenwmyyj, common/qxp/wsvcgvcjaunglhacttnmvbxdyqhib, common/ufn/qotyngjqkxvdrrkxbgyeofeswwjsx, common/hiy/ukpjdgspcajbxpchwbaqwzupfiuvc, common/xik/hsomlniiuhvgldsnczhadfuwjygbz, common/ihm/dgnqbujebcvjpcqgbasplbuquljoo, common/oir/mqcbtrepvslborhyajgljppaawwho, common/cep/pnjxbovggncpsmuzmmhmmdupjpwup, common/mex/beuqnaphiaolmzflxsxhfrtxfokke, common/fms/xpkhrmqtunbdmsmpoiojgpnlesmmo, common/jai/wlylhaaioooajjusvkunxneworeph, common/hmo/qmolxgfjzjouudoskyiojcdximpdi, common/hzp/byxxuwswpqzmqiplhnadpngcfysby, common/uwx/ytduhixlidoydjsxfhmlozgfwvdgl, common/tgt/rbvxnohvruzexixssitfetjpcawla, common/lyl/imiwytmbtdwdfefyvnbktkknzavbp, common/vis/psdzpxvqixwvpxmbbfzybbcmopioz, common/rjh/rrxvzgezjivaqpxdwlhxqonweviqo, common/zav/nzjzzubcxeiuaypwnpzzxbtrssawj, common/gql/kzgtybtfeyfqoydfvxpnsrspkzzsp, common/fnf/aftanhuemxckxweajjpahuvqleraa, common/xvn/mxjkxwpxravnfppnzmfskzpxdsqey, common/dzc/ttcliuduikwbheyogerdatimjdckh, common/orx/thiqiasygvkzwwgotfeumpqxibogv, common/snx/qoreobtkdgtnmcwckojxivosatdha, common/mdb/mxpkrobzwgywcmqmshszcohhgtqkg, common/hnc/sugakpqrhbdnqbesdkgpbdngkxuax, common/byl/eujqugenfeuadelxszkqgslouwtwh, common/gtz/qhxrhogjqfcswxngklvndrroqbcqe, common/lfp/catmorbvmcexqrevfjguvpvyyqnta, common/eke/dkguobqwkojjxtpjcyvstoodtouki, common/gdl/pcbphxvzsfjvppmqoocvhzozvrbpt, common/xyo/cnvmpxofqodktwqgkdkzzkccezkry, common/bdi/sqxuakjbodmjvnxskwmveqojokouc, common/rlt/xuncjmvsicwxmvktijsnbqsrksmat, common/bru/lzxtasaaprolxyuooklmnxpxncrbo, common/oau/balxlqedtkfzshghjjklkmsdkzkxe, common/nhk/rvbkrhltjbvchbkdqcjqpbahgeyjx, common/ttm/rrwzlsxkjahlqckmpujplkiefqlok, common/orb/mitziibptdxyhuklhxxddztanricn, common/dqn/xxlaeuuyrxeimqyflugnqvpuqandd, common/awh/twiwfyrbakxifactldribpirmpmjd, common/mlv/yvvwzqhizrapcgjsqzjgjscpwuugt, common/rlx/bgdroyheemimkgnywjelvtnpnuxvs, common/fxr/uxlmeukrltbycksnzlaxvgoyvolrk, common/epg/vscmqmijpqundgicktmwngdypacsg, common/mtj/wofnpzbhypxksohiohqharvqfcnhm, common/yqs/cxdkkbbjrnsyamkythpsiqhaghwmz, common/gzc/swozailoyggvoxklkwmhvzkacxqdl, common/okx/uhwxotevtziichvfrsltbygmuvuhf, common/juc/mclxhpnuhyuomfcqkyqgvhonhvgyz, common/wax/hpjrcfdaavshzwcjjyswloehydtnb, common/mym/qxjepixkccupelznanbqcyxxuoxgg, common/qqt/aqrkirdwatfzejmcoioxkjlnbdgxd, common/wkd/ocmlghtoaskciyaehmothhpaxihsg, common/axa/tsnbmkosqdearclmivefgmavmechg, common/mef/uuxazgjdjragqagacflijmaugwrja, common/had/hyxuknxwbqpiwsdxtvlouncyqhbzj, common/kbw/uyfzenufubdlenabvopneydkrqyrf, common/ifa/eemnrrvyhqtycspkjacdlitjttctx, common/nke/pikuxprakpomjidqajxfjsolqijhf, common/zfm/tzhquwwwwcihqpbfrmqhjocmwqsdw, common/wfu/qqxcdblgufremhbrfkvgzholwcldd, common/tqn/iigdctfxpfpltsfcioexsirsscjrz, common/eey/tdurjynmkhpydaiqwfgsbikyyuyyn, common/pmf/nrpkmbudstuuxatzxqhbojgdeetlo, common/fgr/ctwjinzduhmdxhcjdchixqzseqxnl, common/wqc/hsnsjwccwvhhpppmqjuudgckrblzy, common/bju/uaooscmleprqhonbiuaravyiumljd, common/cmi/atbzklxfctyeumyjvbnpdcvkscjcz, common/hyc/ezdqhgrkrhsoqvafdzbhvczxulmzv, common/uau/hswedpucunonqgatqcdqwdnxantss, common/rdh/kqrxravfztxyricazfwsltkhsnrpo, common/lfy/shhanotfkszibkgamdtfwrngtakfh, common/gnp/muagunlubudpcxftwxjrxlduqpumu, common/vzl/kmvyhyrbnchiczlxhulesctufamje, common/nua/oyoirnbkymonasoouamkklqafkeei, common/qia/wgberdflgtqhpgtepluvjfqojmtjv, common/nev/zjjjfdkeglnrkpuksskujbcpderbq, common/qpn/jrhnwajpipmqvapeludadxupyfxke, common/laq/pgkbouoxfbwspvtopspuvutxqrrtq, common/ibz/zfohkgkezkzhqipbyhkysvzgzixop, common/njy/ytwwxiziutpcznyoalvsnwcvinxhf, common/hhy/rnqtsueymoqrujonhfsmrdeunmgxw, common/kzh/mrzzrxeyousiaatflbuoiagisqlze, common/tsr/lglhxbyjhyyyctymlvswbkncfujyv, common/ymc/fgpwzittuwvcxunkcbnmyyrvnucde, common/lxx/pulcpvylwtalmgeyiswgzkogiljjp, common/vjd/xihirorsdvqlkaqprmvrrqhawxvyk, common/btr/ihjbzxoxsfzkhcwamfnwtlolybrea, common/wno/wvxjqeuisqdhrkieencjmtapjbjkn, common/tac/hiisyjregnpniwafqhtlteeodfgvy, common/bia/gjswoutctrjxraiduqzfzlnirxjsr, common/ytt/mwewwtflzfffysupjaiqocxtusyah, common/apv/mkmmnqohxttsbowzswxhmtbgbxkkk, common/xdy/hygezafjybwtthzbsujgwcjffukyg, common/owd/ujewucqafeizxdatdpbphnwwllnlt, common/tzi/gzdhcluzfoslnvmfswnpnbstaoxzr, common/cex/nrrklvlkviaeprabzyytcxicnpvwr, common/klh/kzdytvkipbfjqjnqbcusounxxcmay, common/mhd/fnmhfbqtnfvlsyuavnzeszqmaxvnz, common/ekt/qkbxbugdpnwmtkwedbawznievawym, common/qtu/sxsmuhmfbxmfdejzuuwzxhpofhgqg, common/wph/ymjiyhsmtoaysbaanfguqtsbzwhjj, common/xcs/tvfplnlkywxtidqslxkcfrljrvifh, common/ivc/jxojoxzfbqpomzvtnacspwarmopev, common/fst/dfvgpkjekcvmaaxnklbmomjsblgbj, common/vpa/mjfwofuxubafylnrxahjbzqxhhnve, common/lux/kkqguszkgrtixiwvviwbfbisflrsk, common/bfb/fbpebjsbssuffeklxjzdabrprkfmi, common/eqi/pftwxrqmxarbmmzbmjwjnyfxuxvxd, common/dac/ndfhrmjagbepmiviltnxpokqzawci, common/htl/klluijafpmgbbuoxhwycqufhfmfwl, common/uek/foxrozbpvfpazvgtsyqmfkqrdkskf, common/ukn/jcabnoxzzesagnvftxxwpczecswkq, common/ets/hibbafqlpwogjphrpfdpnxdmfgicy, common/qxd/loitbipblzzdsadqvdgwazixqlzus, common/bos/cmsomugpoldftvbebasrqdbyaokfx, common/qjq/cyaoihmvfxwulnleddsgvbgvvfvmb, common/ihv/qdmtjihfrcbvzuhaqifomsrunudtv, common/nrm/bdrgqwvgbfbqycsdkayoxeqcpxzvp, common/hic/itlhhxavtywrijqywthsslsuuegfg, common/lqw/leteacpcfkxjhbxqnwiptnbgtigxx, common/iva/kjqhyunjpixczuthsuznnzcrdvoic, common/khv/gxuryqeawkmrjcnjgysioqcgoqasn, common/zds/ryoympoygzkgxvuzlfjgcbwwhemyc, common/dqz/tvexlmlyxgkfmholjpixemzdthywd, common/ymi/ughyxduwugveqqgrnomhhqplojljd, common/ppz/xxiyetbjpsvuobzmshmjkyelfgpmr, common/glz/rwbjcqklkdymhabmziswsqllkfqvf, common/vqs/gqdfblqqaynuabnhigvusyoqmufpu, common/bcr/zxizuktfnztoppvdiqmfmabqzylap, common/hbb/osdijgmeiaufzlfznssvthozvpgzd, common/iyz/nlwetukqkkzobmyyustdqoffvujth, common/pyr/pgtcqvntjrftaypqzfioiiedvgvkz, common/hmr/supwysbsajdkjfmuqalbalitzwyay, common/vdq/mvdrhculpzyoyxjhhxcezstnoiqsl, common/kpq/caeyfupmrdywbcmpiubirywfcdtcu, common/uzf/pnvcliqwraqimyfghdvnftyhjkkbg, common/qjd/mwmczubksyvlctmjnarjjxssnkeqe, common/svw/timtmkahaviiirijrbjaqvvewjxxs, common/fqy/ykborqegqpbxdlfyyvuoibufstkxy, common/zdl/faobnihqruusraenqqevvabcsmdjk, common/lsm/pffloogqimzquatqgzvbmmzwiaqvt, common/lbk/wbymfhjsngslanwpopatwjrlvarsc, common/ueg/oyqbqaxxatkdorffcmuuxtvuoehaj, common/ilz/yoeookdphzojmxctgmckrcwaiajke, common/fxo/wrptjteoncnaihedlljafwwdkdiyb, common/wye/aajigbesxujsyqvwznhlpylhwnngx, common/ppv/dwpopqnbugcqscobrbulgpynkabvk, common/jod/wgynabfbpjroglyisajroojngmnwf, common/jkl/bgzovztlqqmsxpfbyrrnffcuykdxo, common/lrq/ybpiactifbjywmvixrmscwtqfwzzz, common/dvc/urqouqpvzxareuhhuackptqytycoc, common/ccr/ocnusnwngbiswwrrigptspcitofnn, common/kgu/ssoqekmrdbiglahcuygakofsuifom, common/zvo/dzxqhnodumqdfbapfdcheoujfmxiq, common/cxq/ozoghouhsbvtnruykgoodywuuuvir, common/wcn/forboudklyvksbqamxunujtwkjgky, common/sfx/ifopnshnzrkogemjwabraavhsnxgg, common/cdu/jjptqmyrrfxesqbjoqgdocyjnnbrb, common/bji/ijgejedwhhtvbyzueejtkeaeimlho, common/bsi/uzypigzcvntptfpsaojcjblqcdouc, common/kpf/dbusqbukzxhehnxjflkiuynembhhg, common/ncy/zruiqwprgevkhdzoioacbligwtxqj, common/yjv/uqovrcohnpzbpqrpeegmsuklryedu, common/dfk/diixcfybsntkdzhafhufetpuqzanz, common/byy/brwwqhyalrmyqqnjjdacwuhiultpd, common/wbd/xuuixwztaphmpadvehpyswszofszb, common/yeh/prywojfokbfmkaidaiuogulllszcv, common/pgt/cnkdswjejixiraufbihfzwbirrkqr, common/suy/cmctlcqxaaxnceawwwhrdhzocgele, common/hoq/lypzyerebbqtbgkrxcqqbjwpvjgrt, common/ekq/ggmeowkjkusqexsnihodaoxanzpmy, common/zpo/nronnkiwuhlxknpggorqhklqglpqk, common/jhc/vpjoctvopdkbmvfetvsqcmcurzykf, common/kkn/ldlixqhswlkiypioummyqgwkqimzf, common/ldv/jybthmtmrxoqzfcaqzausrgqtrwhd, common/wiq/opeqtinxetjfxtjkjmdyzlcljzcrg, common/iyi/ybcddccjnleyycwavxbcbzatohyup, common/srp/fqgxlptnaxqcemyxsolcihfvhpjfi, common/dak/otvxoqfancvofmmeanchiqoexigun, common/cop/orfukapycrcwukrxurscwkgwhbbyv, common/nnk/zaftydcqubycxcoomtghlovvwhcih, common/kgr/rgimkksvvspcpkyarwigqudlpsjjw, common/gnx/dptanjwrzchxlgcdqmogjbtfmdaym, common/kjy/yshcsglromdyaydnqbhiwhstbzujo, common/wou/cquumlwljxmnrffcfqoomstdhxgar, common/ybp/gkvcmbeprjbefssvrfdmnjmtwosoi, common/fbf/thlaeliqoxrpihdpqonrwnjpyyusf, common/kfz/fzdhqnxyqwtfokutkrtwcnmbmphnk, common/mzy/qkkguvmpyilpenrxiituoityzpjlw, common/nnt/rqjkbvxezuykxxslgndduietojbub, common/hwy/cwgklxncpgqtceiedexlsulcraycf, common/flw/udrinujatscuxrgwmkeroixzsmqcr, common/qde/ecfipbtsalbrkmsofdwzvuueldgjk, common/zhw/ceiwqbxdyicipuyedwprwryoacbpm, common/jyw/iarwvdedtbhwovtrgcdxnwybeodnm, common/sqq/xucuaabtyfzdruaxhsttgezovedqy, common/pyd/ejzruodsaqoxfoztnyeqzijbhbwlz, common/wuu/xydwljtwhivjufcttuakvoluznvxk, common/imv/eeepxjylppzvwdakomvxpgsuabznj, common/fja/mdhxueslfzxjfjlnjssqxfzljvpev, common/asj/ybpucmmanecfaesbiqiaykddhzhbg, common/khw/lrxwyuniqiujfubebcmhxmqgxibxv, common/psk/jnglsffkseewfrcgnoxlfzijwmley, common/hxx/xbrdrvbwrromnsgqzsrjxymvgribs, common/iyi/amqwwvsuvwkdkssslxrbycwmpnbxs, common/odh/jyoivdrsejapugwkvjdiujtyyqnti, common/fet/vpvspcqmvqfrdhvvjuzazmomhgwbk, common/pau/vafdilieltpancnzfkanwmgbamafx, common/wry/qrpljaallzbdmsnccykmovykvlvna, common/qxp/jftzaieilcfikelbhtsusdwvtdrto, common/nwo/zhvdhdjyyctambmpbczotpmiturmq, common/dzz/gbcvnoiynzrvhlpaczogwaosuyshc, common/tav/qwowkordbdzotkmiptohxgarashjt, common/phw/sooeaakgsrufbhmuywpwedmguwrtb, common/vnc/xicgqdzebboniflydsakyblzuinfg, common/fwo/lxrwcjjioalvwxtclkiwrthavxqwc, common/hfy/llgbmxzhyzvdxkygctialypuexfmt, common/meh/xetyudlbcqcfccdhyaobamdaenqcb, common/gak/euyjegnjczvmtjwljcxhqjifyedpg, common/nlc/qcidzolbaplnsdkbbsnhmqfzlbllu, common/xii/pchrgjelqqqozeqiwwgicrwjyuvmd, common/wye/zxyhnwogfytesyqimsgfoxdotylmr, common/snc/qiebqjevqvjyzpbwafnlgppegnhgj, common/ena/xaeaqjedflqiudpqzjazhuuapvzxl, common/dni/fpxavtffovsunqdmxfdlxxwpvothv, common/znt/bccwrbtbanzpgvoxudgbreibbdoio, common/wzv/vrdrhmkossoktbjopexshzzufahcy, common/gik/fzveqjwoobqdizvzlkmxkxcblvbur, common/fwz/oujlmewuybdeoqczngphztaatclsu, common/zml/nkaxiejbxujvzagmtfpzksajatcax, common/pgo/tbzatwxokkiwaqgooaayprztsuzzj, common/kzn/mslfdzmnvruocdibdbvashjctjafg, common/lnq/rofdveaxrjaetjymnbcwbcdaojmgv, common/hmd/tyrnontcbulbgbgdvntkenagwvjyf, common/dtk/vewhwxapjzyddawakquplylxfzvcv, common/zxe/jnpzwsxwieecucbqqtbrusgjkielw, common/xug/oqwsusljtyigxjiutlsbijulwbpnp] were removed from S3 2024.10.22 01:35:00.182176 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.24 MiB using storage policy from min volume index 0 2024.10.22 01:35:00.186977 [ 1263 ] {} DiskLocal: Reserved 2.24 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:00.210922 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62199 2024.10.22 01:35:00.338002 [ 1418 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsNotBounce.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/IsNotBounce.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_1_1_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 01:35:00.357777 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_949_477} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_948_476 to 202410_949_949_0 into Compact with storage Full 2024.10.22 01:35:00.382778 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_949_477} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:35:00.404884 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:35:00.405046 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_949_477} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_948_476, total 62639 rows starting from the beginning of the part 2024.10.22 01:35:00.412861 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:35:00.544296 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_949_477} MergeTreeSequentialSource: Reading 2 marks from part 202410_949_949_0, total 70 rows starting from the beginning of the part 2024.10.22 01:35:00.979533 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 841674 2024.10.22 01:35:01.024098 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d836f8b6785cb560b3848ee26fd8c582) : [0, 332538]: Setting download as failed: File segment: [0, 332538], key: d836f8b6785cb560b3848ee26fd8c582, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:01.025540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d836f8b6785cb560b3848ee26fd8c582): No space left in cache to reserve 332539 bytes, will continue without cache download 2024.10.22 01:35:01.268045 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d836f8b6785cb560b3848ee26fd8c582, offset: 0, size: 0 2024.10.22 01:35:01.269200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d836f8b6785cb560b3848ee26fd8c582 for removal 2024.10.22 01:35:01.271235 [ 1476 ] {} CacheMetadata: Key d836f8b6785cb560b3848ee26fd8c582 is removed from metadata 2024.10.22 01:35:01.281206 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_843_431, 202410_844_849_1] 2024.10.22 01:35:01.283635 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_843_431, 202410_844_849_1] 2024.10.22 01:35:01.299423 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_843_431, 202410_844_849_1] 2024.10.22 01:35:01.356841 [ 1256 ] {} DatabaseCatalog: Execution took 386 ms. 2024.10.22 01:35:01.403445 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 4.46 MiB, will set to 1.73 GiB (RSS), difference: -12.78 MiB 2024.10.22 01:35:02.093376 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 198 entries to flush up to offset 63771 2024.10.22 01:35:02.280551 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:02.283944 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:02.359634 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fc5e6d1003e9e618016f3f927e834760) : [0, 350350]: Setting download as failed: File segment: [0, 350350], key: fc5e6d1003e9e618016f3f927e834760, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:02.361136 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fc5e6d1003e9e618016f3f927e834760): No space left in cache to reserve 350351 bytes, will continue without cache download 2024.10.22 01:35:02.424140 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_506_101, 202410_507_507_0, 202410_508_508_0, 202410_509_509_0, 202410_510_510_0, 202410_511_511_0] 2024.10.22 01:35:02.427871 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_506_101, 202410_507_507_0, 202410_508_508_0, 202410_509_509_0, 202410_510_510_0, 202410_511_511_0] 2024.10.22 01:35:02.482285 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fc5e6d1003e9e618016f3f927e834760, offset: 0, size: 0 2024.10.22 01:35:02.483898 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fc5e6d1003e9e618016f3f927e834760 for removal 2024.10.22 01:35:02.515680 [ 1476 ] {} CacheMetadata: Key fc5e6d1003e9e618016f3f927e834760 is removed from metadata 2024.10.22 01:35:02.542546 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_534_534_0 (state Temporary) 2024.10.22 01:35:02.565050 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_506_101, 202410_507_507_0, 202410_508_508_0, 202410_509_509_0, 202410_510_510_0, 202410_511_511_0] 2024.10.22 01:35:02.657691 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:02.645446 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 4.09 MiB, will set to 1.73 GiB (RSS), difference: -5.61 MiB 2024.10.22 01:35:02.660369 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:02.684711 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_850_850_0, 202410_851_851_0, 202410_852_852_0, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0] 2024.10.22 01:35:02.687060 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_850_850_0, 202410_851_851_0, 202410_852_852_0, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0] 2024.10.22 01:35:02.722426 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_534_534_0 to 202410_541_541_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:02.735053 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_850_850_0, 202410_851_851_0, 202410_852_852_0, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0] 2024.10.22 01:35:02.763878 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 841674 2024.10.22 01:35:02.808592 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_547_547_0 (state Temporary) 2024.10.22 01:35:02.973126 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_547_547_0 to 202410_553_553_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:03.004712 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 63771 2024.10.22 01:35:03.286498 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 170 bytes for 15cda58dd867807c94ef505613ce4599:0) 2024.10.22 01:35:03.289821 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cdbe63532f690959ea85ad3f9b0ca713, offset: 0, size: 167 2024.10.22 01:35:03.303956 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cdbe63532f690959ea85ad3f9b0ca713 for removal 2024.10.22 01:35:03.307979 [ 1476 ] {} CacheMetadata: Key cdbe63532f690959ea85ad3f9b0ca713 is removed from metadata 2024.10.22 01:35:03.483822 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_949_477} MergeTask::MergeProjectionsStage: Merge sorted 62709 rows, containing 13 columns (13 merged, 0 gathered) in 3.269757358 sec., 19178.487310861798 rows/sec., 3.05 MiB/sec. 2024.10.22 01:35:03.565947 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_949_477} MergedBlockOutputStream: filled checksums 202410_1_949_477 (state Temporary) 2024.10.22 01:35:03.668965 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_949_477} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_949_477 to 202410_1_949_477 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:03.670640 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_949_477} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_948_476, 202410_949_949_0] -> 202410_1_949_477 2024.10.22 01:35:03.681040 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:35:04.224292 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 62264 2024.10.22 01:35:04.365148 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0f38046acf0342203c5232b9f5352ea9) : [0, 340101]: Setting download as failed: File segment: [0, 340101], key: 0f38046acf0342203c5232b9f5352ea9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:04.366770 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0f38046acf0342203c5232b9f5352ea9): No space left in cache to reserve 340102 bytes, will continue without cache download 2024.10.22 01:35:04.410786 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0f38046acf0342203c5232b9f5352ea9, offset: 0, size: 0 2024.10.22 01:35:04.412703 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0f38046acf0342203c5232b9f5352ea9 for removal 2024.10.22 01:35:04.417262 [ 1476 ] {} CacheMetadata: Key 0f38046acf0342203c5232b9f5352ea9 is removed from metadata 2024.10.22 01:35:04.508243 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 4.07 MiB, will set to 1.73 GiB (RSS), difference: 15.99 MiB 2024.10.22 01:35:04.627561 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:04.630948 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:04.709036 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_942_942_0 (state Temporary) 2024.10.22 01:35:04.891296 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_942_942_0 to 202410_955_955_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:04.931821 [ 1155 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 7 parts from 202410_1_949_477 to 202410_955_955_0 2024.10.22 01:35:04.967467 [ 1155 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.30 MiB using storage policy from min volume index 0 2024.10.22 01:35:04.970589 [ 1155 ] {} DiskLocal: Reserved 2.30 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:04.987650 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} MergeTask::PrepareStage: Merging 7 parts: from 202410_1_949_477 to 202410_955_955_0 into Compact with storage Full 2024.10.22 01:35:05.021795 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:35:05.031629 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_949_477, total 62709 rows starting from the beginning of the part 2024.10.22 01:35:05.072161 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62264 2024.10.22 01:35:05.081517 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} MergeTreeSequentialSource: Reading 2 marks from part 202410_950_950_0, total 78 rows starting from the beginning of the part 2024.10.22 01:35:05.232132 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} MergeTreeSequentialSource: Reading 2 marks from part 202410_951_951_0, total 54 rows starting from the beginning of the part 2024.10.22 01:35:05.243896 [ 1173 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:35:05.309592 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} MergeTreeSequentialSource: Reading 2 marks from part 202410_952_952_0, total 57 rows starting from the beginning of the part 2024.10.22 01:35:05.371772 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} MergeTreeSequentialSource: Reading 2 marks from part 202410_953_953_0, total 79 rows starting from the beginning of the part 2024.10.22 01:35:05.444789 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} MergeTreeSequentialSource: Reading 2 marks from part 202410_954_954_0, total 53 rows starting from the beginning of the part 2024.10.22 01:35:05.595555 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} MergeTreeSequentialSource: Reading 2 marks from part 202410_955_955_0, total 65 rows starting from the beginning of the part 2024.10.22 01:35:05.608707 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 8605 2024.10.22 01:35:05.968287 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:05.970862 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:06.000421 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_558_558_0 (state Temporary) 2024.10.22 01:35:06.055185 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_558_558_0 to 202410_571_571_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:06.100725 [ 1178 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_566_113 to 202410_571_571_0 2024.10.22 01:35:06.112928 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8605 2024.10.22 01:35:06.121456 [ 1178 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:06.127651 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:06.177435 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_566_113 to 202410_571_571_0 into Compact with storage Full 2024.10.22 01:35:06.257729 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:35:06.262932 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_566_113, total 8639 rows starting from the beginning of the part 2024.10.22 01:35:06.284755 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_567_567_0, total 14 rows starting from the beginning of the part 2024.10.22 01:35:06.333122 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_568_568_0, total 20 rows starting from the beginning of the part 2024.10.22 01:35:06.393623 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_569_569_0, total 8 rows starting from the beginning of the part 2024.10.22 01:35:06.575040 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_570_570_0, total 19 rows starting from the beginning of the part 2024.10.22 01:35:06.737572 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_571_571_0, total 11 rows starting from the beginning of the part 2024.10.22 01:35:06.744263 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 3.70 MiB, will set to 1.73 GiB (RSS), difference: -11.18 MiB 2024.10.22 01:35:06.784398 [ 1419 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:35:06.996967 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 469 2024.10.22 01:35:07.305254 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:07.311851 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:07.480010 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_451_451_0 (state Temporary) 2024.10.22 01:35:07.568371 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_451_451_0 to 202410_461_461_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:07.679471 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_955_478/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:35:07.757249 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 469 2024.10.22 01:35:07.750545 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_112_112_0 to 202410_115_115_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:07.910587 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_114_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:35:08.387330 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} MergeTask::MergeProjectionsStage: Merge sorted 8711 rows, containing 12 columns (12 merged, 0 gathered) in 2.223426721 sec., 3917.826442277429 rows/sec., 622.91 KiB/sec. 2024.10.22 01:35:08.422495 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} MergedBlockOutputStream: filled checksums 202410_1_571_114 (state Temporary) 2024.10.22 01:35:08.758717 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_408_84, 202410_409_409_0, 202410_410_410_0, 202410_411_411_0, 202410_412_412_0, 202410_413_413_0] 2024.10.22 01:35:08.771077 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_408_84, 202410_409_409_0, 202410_410_410_0, 202410_411_411_0, 202410_412_412_0, 202410_413_413_0] 2024.10.22 01:35:08.800391 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_408_84, 202410_409_409_0, 202410_410_410_0, 202410_411_411_0, 202410_412_412_0, 202410_413_413_0] 2024.10.22 01:35:08.858878 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 3.09 MiB, will set to 1.73 GiB (RSS), difference: 1.20 MiB 2024.10.22 01:35:08.891635 [ 1187 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:35:09.032498 [ 1187 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:35:09.071671 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_571_114 to 202410_1_571_114 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:09.081053 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 62325 2024.10.22 01:35:09.084666 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_571_114} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_566_113, 202410_571_571_0] -> 202410_1_571_114 2024.10.22 01:35:09.158918 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.63 MiB. 2024.10.22 01:35:09.361610 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:09.368493 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:09.540921 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_943_943_0 (state Temporary) 2024.10.22 01:35:09.683157 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_943_943_0 to 202410_956_956_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:09.761690 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62325 2024.10.22 01:35:09.768243 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 843599 2024.10.22 01:35:09.860085 [ 1207 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 01:35:10.405369 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:35:10.413427 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:35:10.550310 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:10.553650 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:10.571347 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 105 entries to flush up to offset 63876 2024.10.22 01:35:10.637603 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 4.41 MiB, will set to 1.73 GiB (RSS), difference: 8.36 MiB 2024.10.22 01:35:10.703881 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:10.711744 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} MergeTask::MergeProjectionsStage: Merge sorted 63095 rows, containing 13 columns (13 merged, 0 gathered) in 5.731625922 sec., 11008.220155788456 rows/sec., 1.76 MiB/sec. 2024.10.22 01:35:10.711241 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:10.720876 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 170 bytes for 4cb34bc55aaf6d1deb3ac3a8e1c3ff5c:0) 2024.10.22 01:35:10.721347 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_535_535_0 (state Temporary) 2024.10.22 01:35:10.724098 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 688bc612978ce37f24282bfd9ff8cb93, offset: 0, size: 178 2024.10.22 01:35:10.729707 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 688bc612978ce37f24282bfd9ff8cb93 for removal 2024.10.22 01:35:10.737026 [ 1476 ] {} CacheMetadata: Key 688bc612978ce37f24282bfd9ff8cb93 is removed from metadata 2024.10.22 01:35:10.764415 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_548_548_0 (state Temporary) 2024.10.22 01:35:10.793745 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_535_535_0 to 202410_542_542_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:10.867564 [ 1168 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_532_106 to 202410_533_538_1 2024.10.22 01:35:10.873491 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 843599 2024.10.22 01:35:10.875198 [ 1168 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:10.879276 [ 1168 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:35:10.908595 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} MergedBlockOutputStream: filled checksums 202410_1_955_478 (state Temporary) 2024.10.22 01:35:10.944310 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_548_548_0 to 202410_554_554_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:11.014734 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_955_478 to 202410_1_955_478 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:11.020472 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_955_478} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 7 parts: [202410_1_949_477, 202410_955_955_0] -> 202410_1_955_478 2024.10.22 01:35:11.159616 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_538_107} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_532_106 to 202410_533_538_1 into Wide with storage Full 2024.10.22 01:35:11.170223 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.15 MiB. 2024.10.22 01:35:11.177055 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 63876 2024.10.22 01:35:11.201474 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_538_107} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:35:11.209437 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_538_107} MergeTreeSequentialSource: Reading 104 marks from part 202410_1_532_106, total 841785 rows starting from the beginning of the part 2024.10.22 01:35:11.272852 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_538_107} MergeTreeSequentialSource: Reading 3 marks from part 202410_533_538_1, total 8905 rows starting from the beginning of the part 2024.10.22 01:35:11.268530 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4498 2024.10.22 01:35:11.671998 [ 1210 ] {} DatabaseCatalog: Execution took 314 ms. 2024.10.22 01:35:11.772987 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b67a52556013d24b304a0dea05ee56a5) : [0, 361825]: Setting download as failed: File segment: [0, 361825], key: b67a52556013d24b304a0dea05ee56a5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:11.774395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b67a52556013d24b304a0dea05ee56a5): No space left in cache to reserve 361826 bytes, will continue without cache download 2024.10.22 01:35:11.914607 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b67a52556013d24b304a0dea05ee56a5, offset: 0, size: 0 2024.10.22 01:35:11.916820 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b67a52556013d24b304a0dea05ee56a5 for removal 2024.10.22 01:35:11.922642 [ 1476 ] {} CacheMetadata: Key b67a52556013d24b304a0dea05ee56a5 is removed from metadata 2024.10.22 01:35:12.416242 [ 1418 ] {} S3ObjectStorage: Objects with paths [common/lfh/vzcymnttvtefxrrbnmhnfnnvulpsc, common/zml/phjppjdxakfvudrlbuqvcrvhwrnot, common/yvc/uiwtbeccaacdjeixkzriapghtupfs, common/mql/zsxootwxqmifxkzzojewxopbtdpcg, common/tyc/dlhyjrobjogpkzeeirkugexociobn, common/apz/uyqdemekvpnxlaulylwuexfzevuay, common/qhz/hiictrzqkrjykoqtvhacwyrfxabbx, common/dwk/hsefethewxotmikdnmzjvbseskkol, common/ike/ptgbemkpjqmdghecwtazyhddqxxuw, common/qbc/lqqtekdphipwclbtopkarhbmonjkz, common/dqg/udlfjkpenvrqehnboyktchrpzbwpt, common/rch/dwllcdwobwxxteyquympacqharzua, common/ivc/mqcomrvrdhrppweazsdhwawsdowrj, common/fpz/vhyohdygxnfafyxjzwfxegluvsnyd, common/vyo/kwtivnboafcuqtghqezubchcvaorc, common/zhd/teokqoxejseohgunrzrsrwcnfualr, common/boo/cfrcjgzlwumusexjjaumguwatsmlu, common/jin/qxssemuipihladlexgbkafkroskfy, common/ebt/gfxtjrbsspjgmwbnadfwhtbebexje, common/wor/vpehmvzgsepllggrnqsbqqzgivobo, common/pjv/eoayojvtfhwvxrhxgedpkryeizqxw, common/odt/pqzgppzbwiwdmvrthsmrisnrdyuno, common/nsh/yqavmtjssuhfpqejusmpzzdhpkuag, common/ykw/fndrxmpskihxlpvdmeequbuxwceyp, common/aly/ieeystriffwkllnpxunvuyvsrwkxo, common/bli/nbewaldifxcussrofhcbofatzudjr, common/hsp/mmpxnepkrfqciirbnjuzolwgwbqby, common/ygr/uloqiduxhfbgwxmfxduxeudxyeapf, common/wfw/ofxtfjtsjtjsjxecurhfratcdhqrv, common/rvq/fmvgzbnnupnouukphgttmqlcytubr, common/lbc/xrbrsgjdcmlhslbjhmqkkecgjalcl, common/kcr/wyrbseoxxrbmrputfeyorawzrliwg, common/ceb/svcixeauhbluiwziepaxpwrfjmhex, common/kyb/kneokxpipxxmxnnumjaropbtiaqby, common/son/dpyvgvfgnitugosirroqahklfkyad, common/clv/tvywujfdfuvuwvszaulmyoumumeyi, common/pof/qmdsmgdolmdgitdtqhlsjcmbzrrvy, common/ttb/xmsryfjdmxnnpwazqcnicyozgkmle, common/xxj/mngedyotmgnuxeyabbijbmazgdngb, common/awl/gkzsgldogolrghkpdldcxotexfdgk, common/ayp/ymfdmfcfbawgzszycxcewxoigkshz, common/yex/viyprkspwgjbpaozuwfimyzigzoow, common/igw/yswuetkaolfpgmheopezmjtqvoosv, common/ijt/malqrehnchvsmslmmehxdwaxdilbd, common/ocb/cgnysnetqnzycygnmvqgifyywcduj, common/vht/gwewbwactkffgfzawymcxjqgofssz, common/aoe/iyxmvmnatpydiigoxkxzozhobsiyp, common/vlm/uyqqyflcqpwigehpbstoakfzggrme, common/xlm/vxdpyxcgkhtaopmxcztcqfkaiomgw, common/wyb/sifnybxmimbibcvahzshqqpkpxwwi, common/asw/efaccsffiihminvbjsgpicpynzjpx, common/zfg/dhzzwqfpckdxkwyxunxpiaaoufbmb, common/dhi/jhtfjcfirxbmhkeftnnptanwtcyjs, common/eui/sdynjmhrryqgrwmzbphbrgvcwbmwq, common/mtw/heptbhoarkybsorhssmbvtyutordr, common/liw/xawtepouskgsykdstujwboanzwmdw, common/ifj/nlewcdbvzncodzmdmmevynhrifzzw, common/lfl/hehdecmbqidfnegflogzepxlxiwrf, common/huv/pkeepjqlshldcrwyafmrgsuzufbzu, common/yus/hcdklwzgtihahjwbgoosedrktgkdy, common/wcu/ijphgmrveiywiuebhzmrxkbbazxqo, common/bji/hzyygyqtuowxssxypvqcszsycozsd, common/sya/zmwsfnrxxknjtgnqrmctrdxxmguru, common/hul/ulrisxusjrtwhhwlpwvegwkitfexn, common/rjf/smdsinelzblsciupyrdfjdxurzmgk, common/pmt/rfvqyxeyfgoljupqqckbfljroomzx, common/lez/rnpkqskzhogryhlgdrwsjwdkgnysa, common/wkr/pwuohudrgmkapyxthmmbqocfeshiw, common/pfm/vtxdgmnttdfmykzcrytuqjorhrsqv, common/pst/imbtwnqpvmdlcgmubjiiqpcpeanpv, common/bcl/jxtvdtdjnxilabdesisldmfcarazf, common/nsk/ryudfdilyiocqtfdqwbihupwpqfoy, common/mzn/pphrlvghfferbjurqbjicazzinfkn, common/ogc/gynxbopqvnbhuthdnzljxbkaueypy, common/ikp/rbjjahofcstcluolohvdghgkmbtge, common/xkc/aqzuoafxkpjinexakyenydrwfyetp, common/cmm/oxlegujrugwqqyyyrlxihzagvfacz, common/xqd/jwnaaqgqvevdyxstzggiqikyuzhtj, common/yks/fwzszzgeuabzvbjtqhmddqjiyqilg, common/qxe/stuqzwsaoqljykdfxpmttwuyegwku, common/tmm/ksthukbfasysyzvvdltxbpzrudbyj, common/vsa/aspaibvzzstfvgujmxevnodhakuvr, common/cyr/wwzmuxiebiurwqmqfobnmiqupuzgk, common/xja/yffuamoblllzsueyfhnoyjxcajkzf, common/gql/byxvtffczuxbjounhfvrgjdhofpkg, common/iui/ekbcnqdabtroodthubxynaaaziois, common/dhr/yvndcsfgfislticbjyznkjekulbcg, common/hey/rmgiebamaqefjkwjbwveyoszytjdt, common/guq/nvlzflbreahudkqnwkccbsfwmitsk, common/icq/mfabnztdxhnpnenyllrfeiyynrsmn, common/uzr/wqkqxxdxaqrwgnruelbiixnghesqf, common/awe/llqijvmjheqcteefdhfzkryusdrus, common/vej/uqjedlruonajhemwymlcralhblbgl, common/aia/njppsjiiylphfbsodetsvbpupswif, common/bgi/aezuanwrpoizxumosdiqiochpgpuf, common/bcu/uloapdvnddybmrmgkofafuycbxqzp, common/wfk/zhwgvfnlutzzjbpfvcsulntjjbykq, common/gvd/bmwiyafuwruznpdbzsvifdcysceut, common/bzn/zfrsgchklgmexkkhowhhecevzskkm, common/rms/uzfsrtqafxasidkccstxhsinrqclv, common/vmt/zktqwykxvbuzhgcduonvthqjgwmti, common/efi/sgtzozxzfocmlqrfvmerbgexzamqw, common/kjp/msortuesenhcokaldysgqupojmgig, common/gap/hkojhdccirilrkbvobvbjwoduwizw, common/usz/lgeyugiqlosvkoyfhbfrowwavklrn, common/chw/czchirotnksrzmlrnkvpsxypfeyje, common/vmg/bkkqpshjbxvtqcqapmikbiprvjifj, common/mzm/xpjkjmbxtllwsgmmtemlsodfmijyw, common/lro/uykosilgcpsxjyzahzvspxiehaskp, common/tsf/adqsrxtpzludvwrxeafftvkdfloke, common/ecc/rmdfgusxjypqbilsatbwvobnyvnum, common/kfz/aotwlcufaymnghubmetumehoucjgi, common/gwz/pfglqayczdfhpgwocarfpfcbtdops, common/kyx/gvclqmvtgkgucxhbfwzlhfqwucntn, common/lhj/zdlokypaglogvkhnzaaadimwpeoso, common/gpg/elsqcciadsykdcanoqxhhghymtcfz, common/llf/lwchwbjrdkkrlgcljnvfwhsqmcwoq, common/thn/vuxaarafqlufjxfstrogoqhrletym, common/bqz/ytqvnmhwzljekcraietgkcdgcdrxa, common/mra/qlmokyntvmeokbqmbojwraeeppalc, common/gwg/ttsofcmdwzhjuhelsjxcrvgwswsuy, common/voh/ruvhgtpcpzucmefqoyyhvtibtxcoo, common/kyt/gzbtxxxnhulgzlmyezwqtohwhnddc, common/aak/ibhnxaurifgbgrmanlfwqkwyfdbcq, common/njk/aqkushqyeaxthkdzqsbnctxueykkp, common/rei/joojqbvphcxphdjbxzyyrlbnhphno, common/mfl/uoifvyywsallzpxscnbamfnrftdka, common/xbq/ubugokyworsqdwipkyuhcbubraitx, common/kat/jtwmjumngsmfrbiwwksxncwkmjagq, common/ptk/weefedfosbjpoelnvabmmtzwyicyw, common/buw/lkpmgohwpxctpfutiosjfyytgthfk, common/mcj/lzhetsbsszaxhnnofvluxumiomise, common/smu/nqeoufppceqlyypndnvpgwapfoydm, common/btj/rgibstganqexvuhwfxbomvndnlnie, common/qzq/wvzzhydadbpqvljpnldvfveaekjzo, common/aoi/yhmcoawdjpbgtwaohpdqglouzudfv, common/arp/mmvybpfqdjswyfoxdzutzebfoytob, common/ruz/qguzoyzqofhifcavycjspkvkyalgi, common/zyb/bkrbuginuxpybdlrfnnpzuliyqerv, common/prj/iojcjczfkfsvqhjrvzhcdlweqeknf, common/ngl/tsdmvsywgdsllnbtdygeealnjdheu, common/gba/pnlgyjavuxlzuafkchodmgowufvqn, common/lrj/wydqrtkwsdvtunnpevmqebpcnytiw, common/paz/fnwjzszyfxjdbhpaylsxtkffjmqlf, common/mfe/ntlklrfxiylppgzouaifszinmubyh, common/sik/uryeonbciynzybsuytxgdxwowzrby, common/cnh/ankjxdqaqupunjheytyvxlyryjxay, common/jma/pelanhziolgpfqkbgzqvmbcerksec, common/mep/qxjjmtlsjbwzovrsmhanvbsshuzdk, common/zhw/zdnfkabdujhwwulffpokrudqttlmi, common/rne/yfnaueshjeksiaghemgwqakuqutro, common/jot/emzrbensuxsnzeopfkfmerlsqnavo, common/xnt/nwhquisoyhclsrjxivclzfhxztyro, common/jfo/nifcaopnrwrjaqblsfikgazxithyw, common/asi/qwkpccfgvadxstchkjxuwupaoamiw, common/cvk/rrwtkrtwvbunboeclltqtfeezrypj, common/flv/xmksyrrsolbxynqtnifqyzulcrtat, common/uai/jqqrnmdfzfluyinokcpxqlnijybga, common/luq/egpeheusrcojemhpxemzbzqbbcbqq, common/dgm/lbhqonzvuyoexfthholmicthxdmvd, common/reh/cxaepimxfleuiesbycjzdjarhdekn, common/dbl/zbkirmtdujgeevsmzdyynomeatuti, common/amt/raitomfutglnwemqdhsdvmnnzkkgu, common/pah/dliswsgnfisqyiedlbsktqbgdcqdv, common/jat/zmsxlvvfjkpchxokzgynadjszvowr, common/rzw/mvzstdsqtbaepeveqteaavsipflzm, common/nbf/gxaqsmmdarjfpsgcbdmxhcaxvgvds, common/kng/fwpapgnncgusdzusxwvnkcrgrigtg, common/oll/qnantqqlayrfhrepfcjkvygqtmwpk, common/cve/mxopxunbpickfyrytykleggcxrczu, common/wtb/mvelaookwlynjscuvmzjqdkflyqwr, common/bvx/slykbheoxumhodhfuhsavpuopellw, common/rqz/oevpwjirevmffltvhqdsunuvdfywa, common/wat/hzolrjiwmyqzqhkzhoeiproaaxwxh, common/ilq/xkrugazsluvsfmamhalwfdmoxzstj, common/ueq/ttoinzutzsplkhysgdumoirletbnn, common/igs/uztgdifyczlqsuclprnwclbdbthgp, common/stz/hhwoxpmiskhovntfqpdspvwvamdas, common/wgg/dbfwuxbjwdoktlprsgxgvbjejygne, common/pms/ovzamntetpjnrujmpfwvccihrwgom, common/tgs/fhczkgswzybrwdyshzdqbnshhzgba, common/wvv/hloofdlepovfhotyoyiabcgnfurtw, common/rxu/trgoyechusfvknggaztauqezsviqa, common/wwk/pincfthrenwfdtouqefqzrwxsenav, common/nbq/vbnqttnnekroyzatbfidipfyytauv, common/dil/ycgyliinqldpttfanhldxdyyepvie, common/bqp/uxsjzymrwvcdcjkindovsijuyndux, common/ajb/gmpzkjgztxucxwhevhmhnxpqcdhns, common/bby/jozaxrnvwhhrjifurqjlhhjmzlvnf, common/vvu/hrmwjbbzdckjdthhzbufbovedbemy, common/ofb/rvlzsfihrqvvsvqbtkdizbvqywfbw, common/zea/kncbsdagewcagscwjvfirqldtfwof, common/qdu/afqcokeowyiqxliurlvslevouzfsx, common/wcy/eqokxbxbbukiboxvgeimnkbfagoud, common/suk/dnxcppkaavelfrgawlogkurqqpsej, common/awi/eblcehrhygupevbvgwcyhltpftgyd, common/zku/xycjzpvtjyrpqwpwuadxqieyzpwpc, common/iqd/szadlnwgkqtzmsojqguhvnfatetca, common/edl/yuvezrzitnxtgknldtxvkmunocptc, common/uju/zxuhxhehgdxwqibgjoadgbepnjjea, common/hrb/ktbuhjmqpwbyqnsvnhiypoxduokul, common/ghg/mkptebejjuryrxfrzynodefzgyigc, common/fui/cbtcxgemzmkjgijrlyumelcbmilyy, common/yyx/qyrxufrnkbtgophfzbpjietlbljti, common/beu/rfrnbrrhlcwypeirlbymajrbbbthi, common/svy/iwsekbhncbhzysqukumbpnbscsols, common/wlu/xajajyuopvwanpbzsnpzjfpjaoclb, common/xgq/rkwefvuqqpanduwzlmukilryzjwcg, common/hgj/gnadmwnvfidufakefpjevteyohtil, common/nct/yswsehicmwjrwmhxxhnxnhengbqyz, common/kip/uhwtiekugtcrzjstelhypldufgofh, common/hgh/qgiceapdiajkopxofhtpghmidnozd, common/iey/seeowjlcgogtjqssqdgnymjyyyxhl, common/qbr/nbrngsriyjbbiuwwlidbrkbsyeojv, common/zbr/hixqllhzfdvzgwvftgjokdutfltmk, common/kyx/cofpalejfugkvyeipwgidsocqcfwh, common/iki/mudsaysgkairfirrxcbtunrslvlkt, common/bbu/kmsrmvubxckqugwxdhtxtbogpcnwj, common/jcs/djtuotgjvbvxjzwagymorbyliryty, common/hwb/pnaqcqjxlxthctewvzmdvumdbxqew, common/oet/ciortvbjyiifwepssczpkcvjuckgp, common/hpd/otuwwfegirxfddsibbunavzrdrrox, common/blq/eawwampzawuuruviltivjduqyoozz, common/jqw/nsmjblevjxdmzyqzartlbuxqycrnl, common/ppb/eohvlmxpkbpsqkbohcvqmxwvmstfa, common/iar/ajqozaclyrleezawuqndzrolcjfmo, common/pvd/xmlqfuezbswqemzpwbketcvzyknnr, common/rie/adqrgrpklmpofvwqocuittvymccyk, common/coy/njxbjnbgoecpwnntrkepjufphfyiz, common/kis/rvmcpudyxxkmsdkpjceppvxqbxqin, common/rea/egsaiucyepwuxwupzycthbihjywzb, common/ngk/agbfjlxgkmkfkqvvuovvcferxgqba, common/bmt/wfmftsvwzwtfdpdgisbrejtnyvjrq, common/xjy/fdabodcghsbvjvofnmeywlgaqyqse, common/zcf/ysnxqcckmxbgndgufjyuwphvtjrxq, common/mbm/nfdialefngtbbwfjbbbpoycqibmkk, common/ncl/irndessdwydpyeepjeprpmkenklaj, common/jfz/atwadmbapmdzcvcwawskvgfocdmjz, common/rrv/rhgrdyejtrstjvenedvmzuipsybto, common/gpg/cshevbnmyefcwlrxwiitkobddpljg, common/dgo/buyvcwgaoaqbdsbqvtrlksscxmdfv, common/ocd/igfvcmjqbkojuyfzzffbvxylerbbl, common/hon/raowdqacfrkfopzawlwzvsgabakeo, common/nwj/pxfxdzomdelwgngxfssynwseymqqi, common/gqw/fvoyqqytvxlyshkcdamiqljhnjihj, common/sao/eoemvufumxyubzyclzajaepmwtmga, common/hmd/bwdaijojbogoppbiwuxqotgzzjvoo, common/tmw/pkaekmuwntocrwcuelujggbuwccoi, common/szp/cuqyewiataauocuwvtxnsakswadjp, common/kqh/hkkbhudyseualvcrcvitsruzjdipm, common/jbg/jxjkxwnpwltfkarqtlwhodrgnblcx, common/kzs/vxwkrvcvqvgqlwigmxhiivedgkejm, common/zbn/wjozfjgysuzzwmkrdbqmfkvifyibt, common/ivl/ewgymadroltcxpgvfxfzqyvxhnlxb, common/xld/mpvowtvqzkavgtrwemibitsjwvvoj, common/wko/nznhhoomoyoinykjnlidozvnauewp, common/rxr/bxqozirqvckbfrpjfidaffdeugtgn, common/apc/dmaymmpmkneovlbeojhlyhldptbcf, common/eyu/vpvzxyeyqlupefjwezobfkfnfdivf, common/wuk/vjtmopkhpzrldhfmloljalnyrvzfx, common/djt/osqcuxisbbgoooysgjbbpmjhchjvh, common/bgw/vrnuyjjlmklitblbpcvtusdfjbrqx, common/zfv/grcizbocwvpehcswazyozvqjubztj, common/yyl/tgmibojpzkrvofakngzsgdgdaqwmq, common/rlc/emnanttozkoapwtgkaggymkintvsh, common/riz/dgzogsawkhvnaqwdhoubggnfarmog, common/sne/ogurucbporiffbmpgurbqmdsbodxm, common/ncr/hqzviastahoqqpilkjnpvmyrmifrr, common/ano/anhbxgkiyfnhvekboabvgooxxovel, common/vef/ufgiddqsyqtmaphtmrjacvlytfvua, common/ggy/lsiqdvddqqwpugovkxvfdmlxcidjh, common/ggn/wxzgzxiovkzqafolrpskhuyjrbzsx, common/dsd/pdtzpmziznqsghvzjungogynizash, common/qev/sjdpccbvzfxkjjitqywumakymhbsa, common/lni/ckbpwukvmoxqkmscbgpizrhidzigx, common/bxk/cvbwitrzoerxvybnscgoukgootbhp, common/ugq/eiphrwzcrxgsiktiofbhuyidghuwz, common/yak/uudmixanckzrxaxstobxfutyqdyiy, common/foe/crebmcnvslqkupceibnyqsgkpmrni, common/kti/sqileffmhemtnvqhlessirmhjzled, common/ypd/ughqgrnsehmwsdowbnltdljtrsekd, common/apg/hxiylgtnfixabdkxkgieqesoiomdf, common/tvu/sckdmvxfzuwnkcjyyeslqauwqfwff, common/yox/whdnptidojmaypdzmgjwrxgbpbgbl, common/qmb/kxuchbphcowsuvyorcwknsirvjpnk, common/aqz/cqesvnyqkdkqqqwlvpfjntcgxiwmm, common/vip/rrmcpfxorwwogjtusvbhvwjdebmxk, common/psi/lwvcvickqjdlchuoxsezysluynvkp, common/xyl/azljsbrxpcdlapcwfciicldsqxpkj, common/otz/eeygibptkezjgjlxghurvedkfjxsu, common/lkg/veieywzbmvlkrqcfmfufnpsadgbmf, common/cev/qinqnayozxfyrrvcvvswhbxeezpes, common/iqf/estchuwzdhargjmgtppyqpxtkkmug, common/gry/cobytneujuegzirujfemsucranyfl, common/dvp/awmlkfivyetytvjhwsranlhuwtxzk, common/zyt/iyirsdclfkxkboqkynkdrgtigzroj, common/sau/zzqcfczumabwifhgigfwltejxwvxd, common/qfn/frnbirqxodihrfaznhbufkaurusuv, common/fll/rvyffjrufaiigqrhpgetrekgvrfbg, common/drl/bbmcuzuvjjlllscxvfybiwcujewle, common/qtr/rslnqdwbmgodksanurimazkydqfyw, common/lxa/nqawgzbxxsjermdvgvoeoqxbdgnwz, common/qfi/qazrjamvwyxwakcszloztybslrrue, common/jua/ftvrxatcjdyybtaruhvclmbhzessv, common/zlk/wfxynzoaqnlrsphrrumzfxmwpvchy, common/lkd/dlseelejwzukuliyjnoutgxrfnydt, common/zhx/wbrgjpntfqjwlrtkocujulqoilbnq, common/lwj/ekkldyfrdleezzntbpyqwtdfxzguq, common/fdy/slaplgqdydqndtfcbzafggswwcprg, common/qxr/uovgguuredqxxuiicsmgtzrkaqfti, common/zsy/wcvivgxekramlfaiflwgypclapxpm, common/pga/ztjvahyrecflyejbpzodhgyhdtzmj, common/dzw/veazvsesdrpvoktxntbtborecbhta, common/zls/pvcmarvsfneqpqwilvmjjrffbhmpt, common/lyg/cuciuofhzkslxlpbnglgtcouenvtu, common/xll/sqkmmeizzqebzotkqsudkrtbdyzam, common/rmb/wmfcurpqxcnhngvisyzpmajmuhgiv, common/wcu/hrcbwekshgfrnfovijwdlcdrkzhdm, common/vwb/uvezbpnbuvqmlsgwjrbpbcxxyawgs, common/fdk/ybfbddrfkmngmabcvkbgjbgnusufl, common/tmy/aiizblvuvzjgnuxosmfpemwwzvgct, common/uyc/ierozbxytzvyavsrsmzfcceeqvzoi, common/cvf/mtrvsxldaxbvchhhgcktoekyvwnyb, common/vfl/dcmgagryckolaclvxxgsqncerzcsc, common/tkv/ffqdeploysdosnatapaijlyjmabin, common/nmh/xscxpbgihahiizutkrhjahmgfiwgz, common/ats/ewyoumpenmlpwsfgomrgvunlmxsvo, common/zwm/oaklswotjjecthejsdpddjgavrmaj, common/fmo/twcsucdjzqitafvvfntclknabdkyz, common/xht/uujotaytnqpknsyjxmlhsgctyatzx, common/rxy/yhwhkpwxsktuxgmgnnzgqniufrsla, common/dye/wnfiiccqmpfolmzgcbtzcmdsixnqr, common/vxk/rtrdhbvkdllvxobfhuadwadlegikl, common/cva/btpgbhqltfwaeqstdkntmqzdxpzzf, common/tra/qaagbhdnnkkpywhyqedbuaqcuehay, common/msx/cmiwuepxnmmdbhjituywsyvmaoqqo, common/ywm/oysiulwvhgcxfexwrqdmdbqkkhzmy, common/nvf/hyemgobtyaituogipnnugcphowcyz, common/kmn/svbapwutrnuxadfmoovfsodfevfbc, common/bvu/csoawionkbkyplxmorebqydbcgbyj, common/ewi/kjgqifagfqqmhnjypkrxxqhgalynz, common/fov/hyjgbotbqylxiwilxautaggmwmjqb, common/xjs/zrrylrxxpzfcsknvqmqgpntjdrkdc, common/nei/lamisxkujgrkujvhfoloainggidyi, common/bhl/zxqxqdeumnearlpgzirfsyovonnlv, common/yxf/nnhfettqkxgsahrzzgcffetjmyqvy, common/dms/xwogtljtdygeyvejhqrodoinbwnrn, common/zgt/kiamxssjeqpblfhsrgrclkqwbyoqq, common/ydo/dzzluuttvxcakgnhxflpzdwuroojr, common/ppo/kxqxcorjeefpmldpesihwzvirxcer, common/gpx/okzuwslzgqxlbiqsitswsqveqcwyl, common/sue/tdildyrdeuxdoxghyhuwcpnlgewxz, common/htj/spjypnepfeszemrwhrayecsysotyr, common/eun/grtvxcvwlwbsylqyjsnyqwkqasslh, common/ocp/gvzglkrvnyryxgclwjlbfmnxfemhe, common/vpg/mscibhjlhuokrsfykcpvgcczpsnud, common/owb/uvyuszvitbmrxxiooqmlgwyocwlzl, common/itl/hmnnxofyfqjxvlixsccxzekqqvaga, common/efm/uzigtnliaxznsaqkkqvfkztyukcok, common/yiw/qddafdpisykehdkzqaqhgxvkuetjx, common/xwt/bcknadarbepogjzwjebzgpzikglee, common/lpv/vikhnkvgrnfsfxurfbxnsfhjbmnbr, common/kvy/dbvklwjvljcvsggsixnhlksbvfkqz] were removed from S3 2024.10.22 01:35:12.620919 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 8.27 MiB, will set to 1.73 GiB (RSS), difference: -3.91 MiB 2024.10.22 01:35:12.842790 [ 1418 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_2_2_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 01:35:13.217204 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 171 bytes for f1286829d9f683f9d2589fb5edea1f4f:0) 2024.10.22 01:35:13.222634 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cbf0accf14f417b7ce0071fee67d3d32, offset: 0, size: 178 2024.10.22 01:35:13.227232 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cbf0accf14f417b7ce0071fee67d3d32 for removal 2024.10.22 01:35:13.230494 [ 1476 ] {} CacheMetadata: Key cbf0accf14f417b7ce0071fee67d3d32 is removed from metadata 2024.10.22 01:35:13.620813 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 8626 2024.10.22 01:35:13.728863 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_1_478_96, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0, 202410_482_482_0, 202410_483_483_0] 2024.10.22 01:35:13.733303 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_1_478_96, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0, 202410_482_482_0, 202410_483_483_0] 2024.10.22 01:35:13.760677 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_1_478_96, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0, 202410_482_482_0, 202410_483_483_0] 2024.10.22 01:35:13.766856 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:13.768657 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:35:13.776902 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 62386 2024.10.22 01:35:13.909500 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_559_559_0 (state Temporary) 2024.10.22 01:35:14.056013 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:14.058625 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:35:14.084650 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_559_559_0 to 202410_572_572_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:14.151574 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_944_944_0 (state Temporary) 2024.10.22 01:35:14.163688 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8626 2024.10.22 01:35:14.292945 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_944_944_0 to 202410_957_957_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:14.315590 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(ed62a9761553b77e99d8e597c4bd2b1d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ed62a9761553b77e99d8e597c4bd2b1d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 01:35:14.322493 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 301258, file segment info: File segment: [0, 33554431], key: ed62a9761553b77e99d8e597c4bd2b1d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 01:35:14.330740 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:35:14.368803 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62386 2024.10.22 01:35:14.986782 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ff1970c8ef1abd0907af95a418219e1d) : [0, 432496]: Setting download as failed: File segment: [0, 432496], key: ff1970c8ef1abd0907af95a418219e1d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:14.989540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ff1970c8ef1abd0907af95a418219e1d): No space left in cache to reserve 432497 bytes, will continue without cache download 2024.10.22 01:35:15.246804 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ff1970c8ef1abd0907af95a418219e1d, offset: 0, size: 0 2024.10.22 01:35:15.248705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ff1970c8ef1abd0907af95a418219e1d for removal 2024.10.22 01:35:15.252877 [ 1476 ] {} CacheMetadata: Key ff1970c8ef1abd0907af95a418219e1d is removed from metadata 2024.10.22 01:35:15.262056 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 470 2024.10.22 01:35:15.511993 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:15.516295 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:35:15.678319 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_452_452_0 (state Temporary) 2024.10.22 01:35:15.865710 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_452_452_0 to 202410_462_462_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:15.913722 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 470 2024.10.22 01:35:15.980905 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 491 bytes for 3c3798746caa58c8b5ed6be7060ee8f5:0) 2024.10.22 01:35:15.983632 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a07a8f881f30e538441485ebf322c372, offset: 0, size: 184 2024.10.22 01:35:15.992136 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a07a8f881f30e538441485ebf322c372 for removal 2024.10.22 01:35:16.003019 [ 1476 ] {} CacheMetadata: Key a07a8f881f30e538441485ebf322c372 is removed from metadata 2024.10.22 01:35:16.005080 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e9adff5993bec69caf593012dc2bc7c2, offset: 0, size: 146 2024.10.22 01:35:16.010259 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e9adff5993bec69caf593012dc2bc7c2 for removal 2024.10.22 01:35:16.014276 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b85ad1f520e152f6c0a1a9fd8b186a3a, offset: 0, size: 149 2024.10.22 01:35:16.027817 [ 1476 ] {} CacheMetadata: Key e9adff5993bec69caf593012dc2bc7c2 is removed from metadata 2024.10.22 01:35:16.121130 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b85ad1f520e152f6c0a1a9fd8b186a3a for removal 2024.10.22 01:35:16.125154 [ 1476 ] {} CacheMetadata: Key b85ad1f520e152f6c0a1a9fd8b186a3a is removed from metadata 2024.10.22 01:35:16.126742 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 96d8d03e3cc35a0c47f459f17e84a013, offset: 0, size: 146 2024.10.22 01:35:16.137029 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 96d8d03e3cc35a0c47f459f17e84a013 for removal 2024.10.22 01:35:16.149080 [ 1476 ] {} CacheMetadata: Key 96d8d03e3cc35a0c47f459f17e84a013 is removed from metadata 2024.10.22 01:35:16.200563 [ 1703 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yfu/uevrmbejvrcwpcqlzqikesgucqtje, size 301258 2024.10.22 01:35:16.223075 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_461_491_6, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:35:16.225422 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_461_491_6, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:35:16.261759 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_461_491_6, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:35:16.318622 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:35:16.328872 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:35:16.369237 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 14.38 MiB, will set to 1.73 GiB (RSS), difference: 6.68 MiB 2024.10.22 01:35:16.394619 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:35:16.466383 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:35:16.472425 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:35:16.482983 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:35:16.500777 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:35:16.512593 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:35:16.528633 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column MobilePhone 2024.10.22 01:35:16.674603 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zec/imipawnyssrcebykreacgzkeipuwg (LogSeriesLimiter: on interval from 2024-10-22 01:34:50 to 2024-10-22 01:35:16 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:35:16.706322 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 171 bytes for 41c4a61005a279913e07a910a1945639:0) 2024.10.22 01:35:16.710538 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3bdf983c96b99664038e04742ebcfe29, offset: 0, size: 176 2024.10.22 01:35:16.713778 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3bdf983c96b99664038e04742ebcfe29 for removal 2024.10.22 01:35:16.718136 [ 1476 ] {} CacheMetadata: Key 3bdf983c96b99664038e04742ebcfe29 is removed from metadata 2024.10.22 01:35:17.307688 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_538_107/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:35:17.667718 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(352459d1c4cfdc6748c41e77b6bef95d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 352459d1c4cfdc6748c41e77b6bef95d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 01:35:17.672341 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 45083, file segment info: File segment: [0, 33554431], key: 352459d1c4cfdc6748c41e77b6bef95d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 01:35:17.675863 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:35:17.892807 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 844754 2024.10.22 01:35:18.381257 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 62446 2024.10.22 01:35:18.507358 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ad96ba66a9ae069232ae024e101a85fa) : [0, 435025]: Setting download as failed: File segment: [0, 435025], key: ad96ba66a9ae069232ae024e101a85fa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:18.509495 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ad96ba66a9ae069232ae024e101a85fa): No space left in cache to reserve 435026 bytes, will continue without cache download 2024.10.22 01:35:18.616168 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:18.622864 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:35:18.655378 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:18.657135 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:35:18.669816 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_945_945_0 (state Temporary) 2024.10.22 01:35:18.696144 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_945_945_0 to 202410_958_958_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:18.754132 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ad96ba66a9ae069232ae024e101a85fa, offset: 0, size: 0 2024.10.22 01:35:18.756369 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ad96ba66a9ae069232ae024e101a85fa for removal 2024.10.22 01:35:18.759955 [ 1476 ] {} CacheMetadata: Key ad96ba66a9ae069232ae024e101a85fa is removed from metadata 2024.10.22 01:35:18.799889 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 4537 2024.10.22 01:35:18.824394 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_536_536_0 (state Temporary) 2024.10.22 01:35:18.852725 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62446 2024.10.22 01:35:18.856291 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 102 entries to flush up to offset 63978 2024.10.22 01:35:19.019016 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_536_536_0 to 202410_543_543_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:19.163589 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 844754 2024.10.22 01:35:19.170792 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:19.172881 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:35:19.247612 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_549_549_0 (state Temporary) 2024.10.22 01:35:19.274142 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 819 bytes for 0d518ff4d75744ff81a47989b7ba41e5:0) 2024.10.22 01:35:19.276640 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cc9b8cb121a39d4479550b17f77e5244, offset: 0, size: 177 2024.10.22 01:35:19.278940 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_549_549_0 to 202410_555_555_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:19.280181 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cc9b8cb121a39d4479550b17f77e5244 for removal 2024.10.22 01:35:19.283613 [ 1476 ] {} CacheMetadata: Key cc9b8cb121a39d4479550b17f77e5244 is removed from metadata 2024.10.22 01:35:19.284127 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b764ecc1908da6ae239c39385c689864, offset: 0, size: 165 2024.10.22 01:35:19.288349 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b764ecc1908da6ae239c39385c689864 for removal 2024.10.22 01:35:19.302188 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8c1d83c4e2de84c0364b1d5df6427aa5, offset: 0, size: 164 2024.10.22 01:35:19.308280 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8c1d83c4e2de84c0364b1d5df6427aa5 for removal 2024.10.22 01:35:19.312820 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 86f9e558e78a698540ce73a9720b88fd, offset: 0, size: 176 2024.10.22 01:35:19.314770 [ 1476 ] {} CacheMetadata: Key 8c1d83c4e2de84c0364b1d5df6427aa5 is removed from metadata 2024.10.22 01:35:19.316831 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 86f9e558e78a698540ce73a9720b88fd for removal 2024.10.22 01:35:19.326566 [ 1476 ] {} CacheMetadata: Key 86f9e558e78a698540ce73a9720b88fd is removed from metadata 2024.10.22 01:35:19.348256 [ 1476 ] {} CacheMetadata: Key b764ecc1908da6ae239c39385c689864 is removed from metadata 2024.10.22 01:35:19.412852 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 63978 2024.10.22 01:35:19.615908 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:35:19.653784 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:35:19.771958 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:35:19.801238 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:35:19.855211 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:35:19.870482 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:35:19.891539 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:35:20.032193 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:35:20.066853 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:35:20.407525 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:35:20.413762 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:35:20.417298 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 15.32 MiB, will set to 1.73 GiB (RSS), difference: -5.03 MiB 2024.10.22 01:35:21.168678 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_114_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:35:21.666514 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 8632 2024.10.22 01:35:21.953325 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:21.957185 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:35:21.979958 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_560_560_0 (state Temporary) 2024.10.22 01:35:22.023553 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_560_560_0 to 202410_573_573_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:22.069527 [ 1275 ] {} DatabaseCatalog: Execution took 396 ms. 2024.10.22 01:35:22.087570 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8632 2024.10.22 01:35:22.386593 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 17.11 MiB, will set to 1.74 GiB (RSS), difference: 5.74 MiB 2024.10.22 01:35:22.579711 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 171 bytes for e1158a9a2e6f92c3a617c5d030dd9e26:0) 2024.10.22 01:35:22.581348 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 996fdfb55cefc5ed88d886a5d30798da, offset: 0, size: 178 2024.10.22 01:35:22.583730 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 996fdfb55cefc5ed88d886a5d30798da for removal 2024.10.22 01:35:22.587011 [ 1476 ] {} CacheMetadata: Key 996fdfb55cefc5ed88d886a5d30798da is removed from metadata 2024.10.22 01:35:22.856479 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 62507 2024.10.22 01:35:23.083766 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:23.088031 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:35:23.163024 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_946_946_0 (state Temporary) 2024.10.22 01:35:23.211537 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_946_946_0 to 202410_959_959_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:23.366125 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62507 2024.10.22 01:35:23.426952 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 471 2024.10.22 01:35:23.901757 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:23.914778 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:35:23.965181 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c70b88d8c521e6734aa6734c006138f7) : [0, 395243]: Setting download as failed: File segment: [0, 395243], key: c70b88d8c521e6734aa6734c006138f7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:23.967138 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c70b88d8c521e6734aa6734c006138f7): No space left in cache to reserve 395244 bytes, will continue without cache download 2024.10.22 01:35:24.003762 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c70b88d8c521e6734aa6734c006138f7, offset: 0, size: 0 2024.10.22 01:35:24.006140 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c70b88d8c521e6734aa6734c006138f7 for removal 2024.10.22 01:35:24.011262 [ 1476 ] {} CacheMetadata: Key c70b88d8c521e6734aa6734c006138f7 is removed from metadata 2024.10.22 01:35:24.039380 [ 1259 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:35:24.063754 [ 1259 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:35:24.075314 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_453_453_0 (state Temporary) 2024.10.22 01:35:24.186323 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_453_453_0 to 202410_463_463_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:24.338791 [ 1257 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_458_94 to 202410_463_463_0 2024.10.22 01:35:24.363601 [ 1257 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:24.366857 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 01:35:24.497158 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_458_94 to 202410_463_463_0 into Compact with storage Full 2024.10.22 01:35:24.519688 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 14.67 MiB, will set to 1.74 GiB (RSS), difference: -2.30 MiB 2024.10.22 01:35:24.619444 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 471 2024.10.22 01:35:24.907271 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 168 bytes for 3d8ef2593c2a587f499062385a534f22:0) 2024.10.22 01:35:24.911437 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f6e9aa33672ef8395f69e0ffb47d5c72, offset: 0, size: 207 2024.10.22 01:35:24.921548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f6e9aa33672ef8395f69e0ffb47d5c72 for removal 2024.10.22 01:35:24.926567 [ 1476 ] {} CacheMetadata: Key f6e9aa33672ef8395f69e0ffb47d5c72 is removed from metadata 2024.10.22 01:35:24.927936 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:35:24.981542 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_458_94, total 502 rows starting from the beginning of the part 2024.10.22 01:35:25.072788 [ 1418 ] {} S3ObjectStorage: Objects with paths [common/ipz/ekepkylvhkpkjpxyckfejjlkgckwl, common/ybb/dprrbhybezwvgrcbjgaddevzfofhz, common/hyp/ypolgkqesqxobsqwunmlkvruhdfok, common/pit/btdurmbceeorvxagqzxwmsmxgyhtp, common/zrf/cayjtfkagallakdvkcrwnoommucpg, common/ivw/uafhxqbzvpywdsbihgebpolemyklr, common/zgn/kujusfcfyubmdwfzznjkmpldksrzp, common/xot/ksievewivsioxepjfhmatywaaewih, common/odx/pqjoscuarqpodylwfbbzstlaigmgn, common/rop/cfnmrdsyagihtacpuwojgcxrtthxl, common/jvw/eegzbtpbccnqzeuakxpetuievqcsa, common/olb/rxaqyqbkphiybzoxotmkqeeurdnjr, common/bae/ydrfepxfsznzqhidgondtcnjhutvg, common/jiv/pwhxjpafmrdwrsrgvuhpxrzjavxyq, common/xvd/oikawbicvnsrxvnchhliujwhzwfiv, common/atw/azkfpicpfykqacykayhijyoxxrzau, common/vot/bhrrvtortvydlqgvrxgxrpmstvyff, common/epw/gtqwliplqinudpsgxtqcqoyjhcavm, common/oue/zemlqthmmkllpyiypwtgutaddqkcy, common/mbh/yvppfflmfaepcdoptxyktoxxplbin, common/uby/fmalqjxfwqzyvrzgsyzmocgjkfaei, common/mrk/oraosmkaorndfhzcevwutdgpehbmm, common/opu/gisbjnhtrnbduhhdpwwtqghafvcgi, common/yjo/xmqxhixrfkshjtehrfphdcleraoxe, common/hle/cpkrericgotpxsjkzislicxzoqpep, common/jlq/levrsuzelenyzgkhxwfoinccwwjvd, common/qew/kturwlzizasdhtpjncsvyqagdryhp, common/esc/zdxvikhnyinyhzsutesjdlxfqzvcb, common/fvi/bxdlgtafqjmwrbnizsvsjvqnbcjqc, common/hvm/pdacjlxcpffdsbazkcnalzpbkqoho, common/hnn/dlasedvgzivtgisaxdypzdroedslo, common/ypi/zbpjnillyddeecesugvvmkbzlsvjr, common/sec/tufvkbuyobjqgpyiixyzsasfukapt, common/xhj/lqqxepgdusgrcypysxkvzzrhykgln, common/zws/yluiwlvnkldeshizquvvrjtgdllmk, common/ycj/kydkshdpqlbyqoogfbwosdfifzlaf, common/sue/vbrngwkkzvnquhcyqinqpcayxbngk, common/utr/txmphrsmtpxzihfbowloogzvigxna, common/ohr/wkqscybmlmgueiccytnsaefnmcsvg, common/nab/zwwepjnatazmrnrzfhfccbqkditfd, common/nmd/zpdvdthspdcmxmgdhkhmppulkkipw, common/ber/dhlrxarjhmlhejcwttclnwtqdrgei, common/rdb/dvrgxkebazawqjxdmmamoozbaezrk, common/hdu/njdknqxabxmedvcuwyrszbtwppoxh, common/hqq/rkfhbqvummupogdllntivujxhkhqn, common/zyz/kurpvjqkxalhracpbmcnhvcxykkbw, common/xiu/smtdlpvtlymombmytqkubdibbwala, common/mao/yddpcvrazabwjhjwdqkegihmxvnyj, common/nuo/ayzmhzhrerdinzqpjlarwqitqvhnu, common/aus/aluydpotnathqugpitmcjpzbzpyvd, common/kgl/gncyjndshcweqfgiwdvssfooeqqun, common/agc/kwryhcapqbfycevinwduvexjyqzia, common/cai/viufinyfwczqaxbeugwtqpvevuxwm, common/mpq/zudulrjczvcphqmoqlaceichnfssg, common/xcz/pkglbmvkfldytwnqzlidcsvrfyxdr, common/yig/ihjvtqooacrswxbeltbydmniglmod, common/tvf/mkasbwqyvkqzeibqrnkrvqsphyrhz, common/zjc/iibjuxguxiufnxiaktlpqwfkqduju, common/pzo/foslaalybyefzmvcbyxlnxbfgfbac, common/rqj/woofuftpzsbefcjrsnnciarkkwqsv, common/znm/pnkqilsjcqsimpgcllhosaafrzwwn, common/nhf/uysalsjqmxvdghzjpfhyihtzdvlpc, common/ekh/yjatixyroyijmchciduencfkbpdhb, common/fno/qbojebzgkjcmtqhtgtykwbkvqdpcl, common/vzc/jvunjjxrmzgazmgwtswuesvuzvyjb, common/yii/unsdnjrugkspddjajnmboxdqolqrt, common/ftk/ncrbahxljtqmhammfffuplykpiicv, common/uld/ukmvszxpplqjyjbiezxogldhkgpee, common/ppo/ctnzsxiyjzonevgmroodnmdmsttql, common/uxf/ehvqopfetecgofyxiqnwgqetyboyh, common/syn/rvbqebvtlnhezwsqymhakchyfrczi, common/cub/hgfcnlledjgtfwbgffmyoulutvbmc, common/mja/udcagpnthrclztqufmnrwiusqvcfr, common/alt/xykhgwknxhbeyvjtergnjynoncdii, common/hnu/vzlrpnyjuinpkybbaqnnauqhafkny, common/qrg/mxptglbijvwgvzpoyscivcaqoehjs, common/dlv/ejpbkgalpbaxmlhzttdpvlgwgfnbe, common/tme/gpbuqncysuoaundfznlilnvdyhpls, common/nix/ixnhxfrxidinuwbfapmrlpmwoujfw, common/kak/zmczgyneloolkxebcgaggydjapyjz, common/rtp/oslgccghmkfstvfbwarpswwdcjzjf, common/lec/jxzcqwaoenlfdjifffnejopllimqw, common/sqd/vgakyitskhpfrngeaciwgvzcwrysh, common/tuf/kcpwaapnbynsciipmcgmtyxxbspai, common/fze/iykeumufcsmvdzjduefahrsujlnww, common/pwa/patizdmrxajmqmflathesdcagijuz, common/fas/myurdnzhiatflwkjtjdwrucxoxvss, common/mia/xntzzyqqpkbjlxitbnqrcggsepknm, common/fci/xnpmyfdjklwwefknyzoyeqqesixdd, common/bhj/nqusdgapqziuhdjfudwvuvbtvtkkg, common/cga/mcncqkmmvexesytxvgwhwwcznxrra, common/ckk/lmovucpjkwmnlddfetfupaytgyhea, common/sfh/starqzrglgcmrkmeuckyucqitjeub, common/umb/viskwsgzlgjhzzsoklhkjlekbwafj, common/tfy/vmsqpdywttsacfayuqdebvuqqentn, common/ffw/eskhotssrovegczirpptqskhbndym, common/fup/noxzhecddvutdjjcklljokhkkpnqn, common/ije/uxuxyvvqosxckzaiynwyvxtjhmylg, common/jsa/ufmxplxodboxltqonwrsympjvqmjn, common/yfb/gamrlvakwyroxptvzoxjfapcifwrw, common/hmf/iiwryosbnvicgzofdgoxohboqitxv, common/tor/zhepjfcbchgvlgujtxaaktdvvyfer, common/srf/uzxqqaswqtdserhsghmusmjxzjsom, common/ntq/cezeuqrrmzgfektqkpcrqnilkkiwc, common/xwf/fmxuscpdfvvyzpdqnpxpwocluowng, common/jxh/kqygoyjfeqepqzdpxndrmhsuzlecs, common/rma/vfwdxxxotvqiqwhlmojzznvkzqmgp, common/vof/gikfydzearzctvijwujinwzpqzkde, common/fgm/yxbnkydjdatxseoyrhbzidevsadbn, common/cpb/rybranjkxbwhxhicvevbxqedcslpx, common/wuj/ojnvmhqvmasaxnmewhdthoqlsmlvo, common/jyg/qvungrdmidaukvkxoesnndgdmefsu, common/krg/asxszbdxkfhfxttgrhhrebwxxiapa, common/ufi/cmiwcorpliszpckqtdciwrfvbwaab, common/dwh/reqxmpzazfwkrwpcxmbbbglurvmdz, common/ezr/yybvfddfgzesqmbkyzofkzljhjxly, common/cdw/aclzeowueuigzxzyjbxhzppptmeut, common/glx/tmxpddvxobyqsheotolpduqjnhrqt, common/lxf/crixyqbqxvulwgbcqykmkwegwlmyp, common/mxt/qyqjhqregpsspgwrpxhmbbctrsrhj, common/tyk/niuppjmamqyslauayarjfjlqnyadk, common/vna/yxhqdbhwvcocydnhrgzukchkmepfy, common/ace/zhafzpmvxlnbpnnkhrahfdmkadurq, common/vxs/epjqcdyzcgitravvspjnuuqglzhdo, common/wws/jfhxlupexrlbpoaxfcqpbxzeqpwht, common/xtq/atbdxpalmytigdybhwuduzigfjnzz, common/oea/tbwzymqbxrhugxgsccjdowztkhvnq, common/qkq/gcwwtleclkubniidxserekjcoqcwu, common/ztr/rggkpunocpsndmeiuhckghsctoahx, common/knm/tyjpkskjtitkivzeuezdzvpoqoyyb, common/vth/wyfsfbzgoxetdhqraznxvlgufbfbl, common/mzv/yisozobyqjldovyyylbqrciuhpmwd, common/bfz/otscbawnssbqmujebgxbjjvdvjxsb, common/zgb/efehcggknwbvvtntzohnzydujeoxu, common/qae/yaotodwlynvngifxontktwdnweizq, common/mvk/drlehjxxhwqdttlnjfngwduqxgrdg, common/olz/euvajmxubfpswfagybcwfyihufrhg, common/sse/aekzjxdkhgifwxtxgxmatwqkwvvcq, common/doh/cnufparclbitwrgrdzwctrkumzddj, common/zce/emsgakdwicbptecxqqxpusdcsefps, common/rez/gtgnvstxktwvrrjdwduhvukrzhcfb, common/kom/lbeoglpyjkcsebewdsxzqgibpbjph, common/pkt/nqruhivbjbmclvqgzariijncmfbif, common/sdf/oejotofwuvhqbhlonkaddqupvtjbc, common/pwv/nculjdbzgapeasmedpfneuvfmfqyx, common/uwb/aifjgqaaojtxmbezqnhymovkiaquu, common/hvk/prdbaovzwzitxwiqmdraewsgwwblx, common/oqy/biqwqjmpwdpsrkmsxscibfubdoxdd, common/zos/qehzyqxorgxweliqmzelxdlqkuxqy, common/ciw/fiygdsqfudmxctllzaoabfohyvqkq, common/nci/cnxfeytikgzoeoypfkicfquapyfua, common/kgq/scamqkavwcnflgxyzmkmvvowookbj, common/gdq/ktqbbroyrqekoanrerajqryouytmf, common/mpq/cyajrbgzheeamkeaiqitmergeflif, common/jur/orndhjmmrdnrjuhzewwkuxdosttmx, common/sdi/kbvzsrcxkjjgrquohysrkwklyftzm, common/qju/demzvldxwneuqiemjlskizvvzbmef, common/vqs/nwyfzkoxaehwbqupukqxvqtckyhpv, common/kse/nirqkjxhwwmtzntvkmijmaulxoqxq, common/twr/bquspgpgrgfjwpffvhuekqnhcyzmf, common/ozp/yabevvenmzbvqjpepyjmzasoetdxi, common/zpo/qxzzfhqvoftolroskxsupccoteowm, common/hve/yjtizbpecxxnadkynzbuhcdqukhcf, common/olz/kewwkofpiwoyikumuvsmmtghjwljm, common/nnt/eahhhzjvpdqfuzjaatgnnddjcpxzk, common/jip/sfsrtffgdcvokboekhvrsnsijwogi, common/vkc/jvycbcejykfnthhosaveuzbydfpph, common/rzm/wpmvqffeptggdolqtpjvdlmhcxvpz, common/fxp/yvybxfkvprvcskdcebedwlejxdrdd, common/dtv/izeoxglwxkdnuvjruueotuhdgvnxa, common/ssb/blptjouvdsndyuvsgatxojgzhzclb, common/cot/yeqnzuqhsvvnlifomadhjvkbcktax, common/nud/ijoahgflatfvenbrnxdhjeddqnehf, common/khc/unujjaeodagfrvlwonpejohoezium, common/dws/fnnjibeszuzlmhgkugdodskkxhzab, common/dui/avahibrhifkglrhondvybqkwrmawd, common/zwv/npfcbdyfpzzjcjozxbfltebpibasi, common/ioe/yalpwgsstdngxacodqrnnqwybptkm, common/ebt/ozxgyyssczcvzduzfocaicxdxlovh, common/xxd/fuydiarfqvxoxfxzktkwthzlxymdx, common/cwp/zqhpuxdussyajkryohfylqawyggco, common/cjw/pshydpxbluwylirfsyczdotuafrep, common/ndc/lmewpfplomwoqvqhusnlpihvjhlca, common/ahl/xmpoivmgbzfsuiamqnppkztvoeifb, common/zfy/kqfiitsqddthusnqolpqpjdawrytf, common/cxk/gxgjovameoupfwhkzdpubigjyxcqe, common/dcd/bdtbnztmsmmdrukihxkgagzpgxfsm, common/hne/hpwbqhybbaxrtvcpucwtuwvdavceq, common/duf/khtclqfqpwdjlhwtbqzssmqhtfsyc, common/zlf/jpvrmvjpzakcpfuvdrpcdwlxnvhos, common/dli/hsjgrmdtsoduntzvnbqqmlvcwnsgt, common/chv/bnaxmzrqshwbneqvrycgrfycythuy, common/fzu/yjmknkvcyqqojvccdfbvjsrmiqcym, common/fuh/iuplcwyhmccewakeevxlacyvornii, common/nib/lshvgqwficlslqogkvhhibfwmjhsw, common/jix/lqfvutlbcbipapldzujrdmpalhpki, common/bgj/kmxoykvmpkjyzsrfvrhopwkwoamab, common/evp/jrhbmpmtndmuxungicofeogixlxvd, common/usi/fhgkqdyuugdvhicyacvkqdtwzxtuk, common/fld/ykbijlfvlaqttjwnnsfntyuagpmrw, common/vdh/slltqsntruojysysrwvwyoxfoxwou, common/sin/gofhmildlctrgubseshgbfbwortaq, common/nrn/yypvigrlwlnzojjsehcfagfegjbfm, common/wpg/kjkvkpxcbrolzzyoykprpvasqzlxf, common/xdk/xavvvqksxidujzangeveupigzhtud, common/vct/yidppiuujuajuolwlkjbnnxguellp, common/hkl/orsvnuzojgqlcxvoubjmupjvaaggt, common/xzi/ffeaegojovqsuuuiofudyvfaqmptv, common/arv/bnsfcvdatwwusmracldutmcqnaaht, common/xvx/bgcpodsnrlebuzqcssntrdsnhvlqh, common/pcx/xbhrfvcrvdjstiamphtnizmxqicsa, common/bkr/czuzztbchudfwmvbzknvpdlvgzucz, common/lro/lgflazxjdskvmoaizlofpvqhbrsic, common/fok/gnthcpjrwsfpkvdepjhgpqdeoetxc, common/nzh/weartzridpisrclblhkxfqnofvutx, common/qnm/rxvtmtvozzpcynnhazgpraynurjkf, common/fzc/qphnngcfypfebwnwfgytcmnhkbgqc, common/qhg/wdwwskprgvszatwkleqmeblmwsnqp, common/yro/fcgtghqgqqnoukwegljkzbadiwkgv, common/gwg/xokktkdtpwqlfsgdxdhlfmkhprkns, common/xso/hlekswhdnoliykebrfewkizvvmaag, common/nmq/bpzxsoadioiwsjivnivzvyoyjaoox, common/scx/zmpqyvikmaixuthfdcxtqubphghnx, common/xge/smykwnplslnbyxxbfcaazzkabiufl, common/joo/fpjukvwzslwycsghcndtkmcupoiwu, common/wds/vgugqovbcgycttcspkkpsiizubyax, common/ohv/jzoouewqjwdjvwrpgpkcjluwclgss, common/ccc/acoaaykskogwwuvkgveqrbbnvxkpz, common/pqu/ayopvmtvhzfqtzzgpudelyfaitupq, common/dyj/ybbdsgvlyzcbqhzwxrlwwbjsjxrxq, common/vhi/tzbcljpdsvgyxzjijmwapaxoycqwl, common/rcw/qzfragtuekiwmxjolbdokffdjtlqj, common/xja/wwqoxybrnwjjjqmrqhonuqrbbjbxr, common/jwg/xvoufxexwpcwgghjmnsfupvactfsj, common/evj/xojfikuxbgqqliaswtctyeclyntxw, common/ock/cxsiuxmzaacqmlblffnarvkozemib, common/wbo/krykqygzbpbcaijprjynrdfikthhh, common/rrb/mkuxouuqdxbjsmjucdbtrbmmnwflc, common/pzt/gcvdmleofcctfriozgldfsdwxbqvx, common/kua/omxuldkxqoeosfqkkoayhqgubhfqg, common/kqg/fiohkjitcgqkyxgcahqomadrufbbs, common/usa/kwbcoukhrrtlhtywgmlbdimrqedok, common/hok/uztpsklaauldxoweddzhwcjdirvvu, common/obh/hlbgwptpaagfwjzncgnailojhneum, common/ggc/cijgpreayciaaogsktzsnxzlsnznr, common/wbg/ofattytvtxghboavytwbulmikgftd, common/jmn/eiiskepyuvtiegwajygevxybowggm, common/bbh/rpjnqwilwkofadboqvvfofewgkfvn, common/rpg/hagdiyfintuugnpundfhhdklhwabx, common/ulf/ymcrxoheffvicoecgwthrrushblya, common/xlc/xzrexllvwphidvtdtppyaqirrswfu, common/xop/vtzfwyixealbflfbbwhqkesnpcxql, common/hcv/lbjqyqyjocwbimzwbvvsgqtxbbnna, common/emw/nvqlmwcirsvgyuohjupylboatmtsf, common/bik/dchgkggpnaphreyydyqqcsuapzbox, common/jop/nvhkzeougxqwkkwrpsjhgidthirpj, common/blh/uldpbhahxbariejylrsuzjbphdmtu, common/ltn/elyqmtuezbsbjfumzfptrgytzduau, common/tef/qxntayooxcpzyyoiinnlfjhrmxzxr, common/xzq/ayylezskdnpcxzypapsmhdmroczbh, common/lnc/jzbqqzozlsprbqpfbqcbomepdtovd, common/dqo/rvloyqhnbdetdhyvcprpfwvoaasjk, common/oks/ircqikzdnkgpuigktplpkrgfcrmfq, common/idw/kkujndztzssmktgqxfmfjmgnbgcte, common/yno/whjmpwxjaizfenhzgxzposusxubio, common/hfz/dtwyrzldpdmopwekvsgkincjlmkqt, common/rma/ybytsgeyszopfkxjxayrrwegtphve, common/zpi/jadidmutshkxwdzfvrtbwhslwpdai, common/csd/yupkkgsorgbbvrsvumsravjevflmk, common/avj/yyufdmrmdcfokubczlcsbbfpwolei, common/fun/wwkumfalpyzcabygtudjcbantzctc, common/omz/holgvgxsbktrcsjumjtftagdrcmpi, common/rjh/fmbrickjqeobwzkgqrbfmjhzkumks, common/mhc/flggekpvhtmiivzhfapwjwwtfrodh, common/vjl/dvhpsrghaqvqqsyjtcgtfwuhjzqdn, common/qif/nheifnbpfcsayyptgzmznzhbnbiwp, common/dwa/ypifycebjkvddmsldalevdrkogwcp, common/ini/doisujwrbopudqbddzffesrwxfvem, common/nbm/iozqxnvqlhycynrtgodjqtikrercn, common/zpn/dokdruifefrvcozstkkukcmhmgdvi, common/ydc/tamlzbvbihnchszcfwbfncbiyjfsc, common/lex/eqtxuwrjahcxdqbgpqgrnjwokdvoh, common/eyz/dcbvngmbboiflpblrwisqnlhwgijj, common/crd/clvsacipugbszqyrchhvabooslyzy, common/fpx/yaoqpamckutzyhhrqnqsryqolcmky, common/ebr/eyoxhktwnlqdfbvkpoirgqwyuwubw, common/cwg/akadqfmcmfedfmiqbfkuhwdzkymyx, common/asd/cmsbditqecsiozsuislacjuohywib, common/oee/qaqvjapcnqhjicvmpsgoyrbkuxkdt, common/rca/glukomxzjkoikhtlbydbocukvgtqe, common/zod/glmzvrqsoaojmqrqjwiecmebzlddw, common/evd/uituujsgtesrcsrbteyzqijijigpx, common/xfk/lkfkrxjtmjwxkurtmotuduadepcnd, common/qaf/gkfyecfutoxbyjnaaqgbsufqvvxdv, common/bre/hzkyvhmjblgauwbufjxgavrddecit, common/xuy/xqitnwjgroxyemqhoqtkfyjutoiah, common/lot/ntmeazizzfwnbzpwgfbilurkalfsb, common/mxp/gqfnqsgcxmqlqgqnwntcqhwamljpy, common/hbs/awnynfczqclzdcmlnxgsehgvjwmlq, common/nes/hsidxkxgsvewwvfqehxzlsabhlezt, common/stm/mnajbhfuttpirshiwkiiqzhqtzjjv, common/kyj/igcdngahublaxjgjtqgyhrpimbzhm, common/tvc/ttzdleixyntxyebrfdwvtzuzkyocu, common/czs/whrvtsiwponktbordpeeyskulpcxq, common/ycb/bsyzocthbwqlomfnqwcgkrrnjxbcw, common/yog/bwbyogmtshwpdvwzfwqgvsdjzlhdn, common/bur/vvxvpiruxqqzqxujtlazsjidbdkvc, common/orx/dukgwqdqkdhuwijbntkecuqbpzuuy, common/xyu/xupecdkmqrnprzugmwwceyfzbfzrm, common/txu/jligrisipjtlmqutbdjpcsvhyvuhc, common/pxy/idtcgexkwtdbsrmuvivrnmatcgbmw, common/tjt/ckaymerelsdhyrbyewecabkleaauv, common/fkz/lmmxomighayskbnmszxzdpbbluihi, common/ufm/fjakcqgmlykwpdxehzkbqqzcnqdjs, common/xaz/zjecmohcbjftcllruylmmoxgdesfa, common/buf/oiouaploftrskpdqsjojqfuxladba, common/ahu/baruintqqybcvnzpvfmkysffyahzc, common/lev/qnokuzopeznzctfvzzkuagwgkybhb, common/eoy/jbiclsebnsvjodyxjuxwmdwmbljbp, common/ufg/idgfnneuporzjjijegncagzsixnxh, common/xqf/ereqhftinkdzswsieofkvrgxlzbng, common/biv/iqaljiylqmxgaxnulyydhmagiwyuu, common/pag/jafwrqiwkqocfggfydsvadfcxhpkv, common/duw/cjobtixvcqybyqzepzgxdjheenrze, common/zpb/blmillsognusdbdooopcayufogpaw, common/gti/akzrxwhugdarwvuvpgzlvijrqjabl, common/byx/vbaguzolywxvllyaokgeebbkndoqg, common/gmf/dfdfkzsczlujueyobvjxdzshhiyrp, common/abg/xqjkmxcjlecswwzzcxagdttqspbdi, common/fos/bxmkbibomoggyonlqgauxsvzfywdw, common/kot/aytomzijddzpkwhprovawxfyjtmgz, common/jcp/pvbhhhulquptxmajsvtfaiimlyzrj, common/jnu/fuzasdbkxfneqkqmhllwhbsgvycxk, common/wfx/ouipnerbbtwvzbfnvvbfqpzfnpkdt, common/hbz/xarztxqcduofyakuybaqacjlzszhk, common/iji/rstskcrmnymdyjwjlmhlfopwtgypj, common/mwt/fddypvnjztnxtccwxgarwggtksnon, common/oby/uigajttgzolaacnplxqbrucwfktxs, common/gos/cyzxwgvtpbobyohrlxtnprjhlvuqz, common/vdn/jarcwxbkpztlzibssigwwespxxths, common/bjr/vxwndztwaypdhadiczzlmwwvsnnjl, common/wph/sojhycpyxheksfzzjlxndaxrtmtka, common/hxt/vfpnrlxoozqreaqzqhhatnmrbhdmv, common/ipy/isirrqbnbmgtypmptsaavzmcbyzfl, common/ykh/xkzudkeizjwyojkawhxzoidrarudt, common/ykq/iltmsggdjmayisuazezggomwheyjx, common/dyz/vilgysluyymyqczairlbbtrtiszbx, common/hyc/zsslmxdqlexjhqpiwbuoxsoxppstp, common/hnd/lmwbvmlaxqklhzhebdrxdqclqzvbd, common/jhv/awbeggwdhvaezopgwvjfhbboyxkmm, common/qpn/dfkajmzzdburglnnrbgsjxbwifsdm, common/pwh/cviqqmxqceiuqqybvqvngxcwvzzii, common/fbu/sdzbuohswewwsfbcmkeqsiulwbkpj, common/bgf/daptskijscuehimhsjajsqhfddelo, common/ryc/ekflzssiiruaztxjnrfankbpmyydc, common/ydn/obzpbtrnimubmchfilqmjhmqgcwho, common/iha/itzmvsiusdyftebngmtskgtxxkpns, common/zwq/lxpspajbpkkcodtgtqzodewxmqunr, common/hkj/cryrxwbjtmpnlthblodvtmyjwljew, common/ttk/djijflfqxtluobikhkthrgqkbzpjn, common/okl/yqlhpwyrozrmtfdpnqmjglomtnpjv, common/ywe/mxmpkqwejgkrjtofhphcbfkitssrs, common/pew/ebnsjyiwzzzyvywnhffammnlznuxp, common/zcy/nvexumeaxkrrobhllopnrvnnyekzv, common/sss/ngygqydtgbluijkkgmydyngfcgwgr, common/ntb/bxjaxbtvedbkcfcvsqexgzlnsmebt, common/iuy/uiiwdzcszunzyehwpmrjuhumvuisf, common/fhe/hacvxiyibpwhlhwlzarmegqinjrim, common/qfc/qegmaqejbaznhykaeczylfnliepcu] were removed from S3 2024.10.22 01:35:25.122784 [ 1418 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CounterClass.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SearchEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CounterClass.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SearchEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WithHash.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/WithHash.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_3_3_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 01:35:25.180560 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_459_459_0, total 1 rows starting from the beginning of the part 2024.10.22 01:35:25.552281 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_460_460_0, total 1 rows starting from the beginning of the part 2024.10.22 01:35:25.641749 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_461_461_0, total 1 rows starting from the beginning of the part 2024.10.22 01:35:25.704340 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_462_462_0, total 1 rows starting from the beginning of the part 2024.10.22 01:35:25.785208 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_463_463_0, total 1 rows starting from the beginning of the part 2024.10.22 01:35:25.880066 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(79d7a29f1dd59709e78f2b802fb62264) : [0, 326461]: Setting download as failed: File segment: [0, 326461], key: 79d7a29f1dd59709e78f2b802fb62264, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:25.881476 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(79d7a29f1dd59709e78f2b802fb62264): No space left in cache to reserve 326462 bytes, will continue without cache download 2024.10.22 01:35:25.928899 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 79d7a29f1dd59709e78f2b802fb62264, offset: 0, size: 0 2024.10.22 01:35:25.933387 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 79d7a29f1dd59709e78f2b802fb62264 for removal 2024.10.22 01:35:25.936917 [ 1476 ] {} CacheMetadata: Key 79d7a29f1dd59709e78f2b802fb62264 is removed from metadata 2024.10.22 01:35:26.167036 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 846294 2024.10.22 01:35:26.553001 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 12.87 MiB, will set to 1.74 GiB (RSS), difference: -4.92 MiB 2024.10.22 01:35:26.727995 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 171 bytes for 1169ebcca4b3822a78c114a75c373a94:0) 2024.10.22 01:35:26.730416 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5f49e7615826441b6212c28a104de8dd, offset: 0, size: 179 2024.10.22 01:35:26.734478 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5f49e7615826441b6212c28a104de8dd for removal 2024.10.22 01:35:26.737586 [ 1476 ] {} CacheMetadata: Key 5f49e7615826441b6212c28a104de8dd is removed from metadata 2024.10.22 01:35:26.935551 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 131 entries to flush up to offset 64109 2024.10.22 01:35:26.956751 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:26.961438 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:27.161776 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:27.164176 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:27.174125 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_537_537_0 (state Temporary) 2024.10.22 01:35:27.196385 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_550_550_0 (state Temporary) 2024.10.22 01:35:27.363231 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_550_550_0 to 202410_556_556_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:27.379322 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 62563 2024.10.22 01:35:27.401041 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 64109 2024.10.22 01:35:27.445542 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_537_537_0 to 202410_544_544_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:27.475782 [ 1157 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_539_539_0 to 202410_544_544_0 2024.10.22 01:35:27.490577 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 846294 2024.10.22 01:35:27.498248 [ 1157 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:27.502120 [ 1157 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:27.530379 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_539_539_0 to 202410_544_544_0 into Compact with storage Full 2024.10.22 01:35:27.555596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0f441857077ad5fc427f52357e24c31b) : [0, 438110]: Setting download as failed: File segment: [0, 438110], key: 0f441857077ad5fc427f52357e24c31b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:27.558041 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0f441857077ad5fc427f52357e24c31b): No space left in cache to reserve 438111 bytes, will continue without cache download 2024.10.22 01:35:27.666857 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:27.671079 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:27.680895 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0f441857077ad5fc427f52357e24c31b, offset: 0, size: 0 2024.10.22 01:35:27.683216 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0f441857077ad5fc427f52357e24c31b for removal 2024.10.22 01:35:27.686423 [ 1476 ] {} CacheMetadata: Key 0f441857077ad5fc427f52357e24c31b is removed from metadata 2024.10.22 01:35:27.731927 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_947_947_0 (state Temporary) 2024.10.22 01:35:27.768610 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:35:27.776526 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_539_539_0, total 1543 rows starting from the beginning of the part 2024.10.22 01:35:27.786953 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_540_540_0, total 1491 rows starting from the beginning of the part 2024.10.22 01:35:27.813965 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_541_541_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:35:27.847485 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_542_542_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:35:27.868317 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_543_543_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:35:27.883541 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_947_947_0 to 202410_960_960_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:27.897423 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_544_544_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:35:27.936234 [ 1169 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 01:35:27.966364 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62563 2024.10.22 01:35:27.978876 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_538_107/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:35:28.058815 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} MergeTask::MergeProjectionsStage: Merge sorted 507 rows, containing 32 columns (32 merged, 0 gathered) in 3.572296319 sec., 141.9255164537766 rows/sec., 13.11 KiB/sec. 2024.10.22 01:35:28.090769 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} MergedBlockOutputStream: filled checksums 202410_1_463_95 (state Temporary) 2024.10.22 01:35:28.559262 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 12.45 MiB, will set to 1.74 GiB (RSS), difference: -8.26 MiB 2024.10.22 01:35:28.587866 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 170 bytes for 0931f15e7d68a6e453808aa007ac2464:0) 2024.10.22 01:35:28.591369 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 127a462d2e9b05df1804835f5612e786, offset: 0, size: 178 2024.10.22 01:35:28.596262 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 127a462d2e9b05df1804835f5612e786 for removal 2024.10.22 01:35:28.598915 [ 1476 ] {} CacheMetadata: Key 127a462d2e9b05df1804835f5612e786 is removed from metadata 2024.10.22 01:35:28.907872 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_463_95 to 202410_1_463_95 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:28.913504 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_463_95} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_458_94, 202410_463_463_0] -> 202410_1_463_95 2024.10.22 01:35:28.969404 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:35:29.010258 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} MergeTask::MergeProjectionsStage: Merge sorted 9194 rows, containing 4 columns (4 merged, 0 gathered) in 1.486182934 sec., 6186.317841273233 rows/sec., 96.66 KiB/sec. 2024.10.22 01:35:29.065041 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} MergedBlockOutputStream: filled checksums 202410_539_544_1 (state Temporary) 2024.10.22 01:35:29.206372 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_539_544_1 to 202410_539_544_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:29.225680 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_539_544_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_539_539_0, 202410_544_544_0] -> 202410_539_544_1 2024.10.22 01:35:29.268258 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:35:29.556920 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(046dccf51261510d49dc53b23c4f4d08) : [0, 390580]: Setting download as failed: File segment: [0, 390580], key: 046dccf51261510d49dc53b23c4f4d08, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:29.558686 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(046dccf51261510d49dc53b23c4f4d08): No space left in cache to reserve 390581 bytes, will continue without cache download 2024.10.22 01:35:29.586796 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 046dccf51261510d49dc53b23c4f4d08, offset: 0, size: 0 2024.10.22 01:35:29.590884 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 8652 2024.10.22 01:35:29.589552 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 046dccf51261510d49dc53b23c4f4d08 for removal 2024.10.22 01:35:29.600886 [ 1476 ] {} CacheMetadata: Key 046dccf51261510d49dc53b23c4f4d08 is removed from metadata 2024.10.22 01:35:29.936920 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:29.940615 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:30.031386 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_849_432, 202410_850_855_1] 2024.10.22 01:35:30.035200 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_849_432, 202410_850_855_1] 2024.10.22 01:35:30.059651 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_849_432, 202410_850_855_1] 2024.10.22 01:35:30.072389 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_561_561_0 (state Temporary) 2024.10.22 01:35:30.136645 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_561_561_0 to 202410_574_574_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:30.176425 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8652 2024.10.22 01:35:30.406960 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:35:30.429776 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:35:30.539852 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 12.68 MiB, will set to 1.74 GiB (RSS), difference: 11.48 MiB 2024.10.22 01:35:31.773900 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_114_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:35:31.971023 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 62623 2024.10.22 01:35:32.123980 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 472 2024.10.22 01:35:32.441043 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:32.450840 [ 1199 ] {} DatabaseCatalog: Execution took 379 ms. 2024.10.22 01:35:32.458805 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:32.489211 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:32.507350 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:32.516961 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_948_948_0 (state Temporary) 2024.10.22 01:35:32.569623 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_948_948_0 to 202410_961_961_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:32.578259 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 15.84 MiB, will set to 1.74 GiB (RSS), difference: -2.80 MiB 2024.10.22 01:35:32.591192 [ 1197 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_955_478 to 202410_960_960_0 2024.10.22 01:35:32.617746 [ 1197 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.34 MiB using storage policy from min volume index 0 2024.10.22 01:35:32.623172 [ 1197 ] {} DiskLocal: Reserved 2.34 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:32.667807 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_454_454_0 (state Temporary) 2024.10.22 01:35:32.675123 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_955_478 to 202410_960_960_0 into Compact with storage Full 2024.10.22 01:35:32.770803 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:35:32.796674 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_955_478, total 63095 rows starting from the beginning of the part 2024.10.22 01:35:32.803221 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62623 2024.10.22 01:35:32.875466 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_454_454_0 to 202410_464_464_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:32.953793 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} MergeTreeSequentialSource: Reading 2 marks from part 202410_956_956_0, total 61 rows starting from the beginning of the part 2024.10.22 01:35:32.986160 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} MergeTreeSequentialSource: Reading 2 marks from part 202410_957_957_0, total 61 rows starting from the beginning of the part 2024.10.22 01:35:33.040163 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} MergeTreeSequentialSource: Reading 2 marks from part 202410_958_958_0, total 60 rows starting from the beginning of the part 2024.10.22 01:35:33.087647 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} MergeTreeSequentialSource: Reading 2 marks from part 202410_959_959_0, total 61 rows starting from the beginning of the part 2024.10.22 01:35:33.105078 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 472 2024.10.22 01:35:33.340305 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} MergeTreeSequentialSource: Reading 2 marks from part 202410_960_960_0, total 56 rows starting from the beginning of the part 2024.10.22 01:35:34.340153 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_960_479/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:35:34.358233 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_855_433, 202410_856_856_0] 2024.10.22 01:35:34.359783 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_855_433, 202410_856_856_0] 2024.10.22 01:35:34.372103 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_855_433, 202410_856_856_0] 2024.10.22 01:35:34.501155 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 847834 2024.10.22 01:35:34.871672 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 11.27 MiB, will set to 1.74 GiB (RSS), difference: -7.40 MiB 2024.10.22 01:35:34.941539 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 64200 2024.10.22 01:35:35.076469 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:35.078875 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:35.218501 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_551_551_0 (state Temporary) 2024.10.22 01:35:35.373593 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_551_551_0 to 202410_557_557_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:35.487216 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:35.496564 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:35.527303 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 64200 2024.10.22 01:35:35.607039 [ 1151 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_552_9 to 202410_557_557_0 2024.10.22 01:35:35.635255 [ 1151 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:35.645108 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:35.658299 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_538_538_0 (state Temporary) 2024.10.22 01:35:35.660540 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_538_107} MergeTask::MergeProjectionsStage: Merge sorted 850690 rows, containing 4 columns (4 merged, 0 gathered) in 24.630018723 sec., 34538.74759768692 rows/sec., 517.88 KiB/sec. 2024.10.22 01:35:35.724613 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_538_538_0 to 202410_545_545_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:35.789935 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_552_9 to 202410_557_557_0 into Compact with storage Full 2024.10.22 01:35:35.828938 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_538_107} MergedBlockOutputStream: filled checksums 202410_1_538_107 (state Temporary) 2024.10.22 01:35:35.923280 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(40178c05a655880f8eec986d5941facc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 40178c05a655880f8eec986d5941facc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:35:35.952025 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 291330, file segment info: File segment: [0, 33554431], key: 40178c05a655880f8eec986d5941facc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:35:35.955591 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:35:35.966750 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 847834 2024.10.22 01:35:36.037000 [ 1198 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 01:35:36.155131 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:35:36.182145 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_552_9, total 4448 rows starting from the beginning of the part 2024.10.22 01:35:36.353331 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_553_553_0, total 198 rows starting from the beginning of the part 2024.10.22 01:35:36.429760 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_554_554_0, total 105 rows starting from the beginning of the part 2024.10.22 01:35:36.641371 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_555_555_0, total 102 rows starting from the beginning of the part 2024.10.22 01:35:36.736445 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_556_556_0, total 131 rows starting from the beginning of the part 2024.10.22 01:35:36.764880 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 170 bytes for c4d1dd41aa005e9f4b2a00949b8ed705:0) 2024.10.22 01:35:36.770931 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3270086aaa9f3831fa6e1d9f21b6d969, offset: 0, size: 176 2024.10.22 01:35:36.776245 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3270086aaa9f3831fa6e1d9f21b6d969 for removal 2024.10.22 01:35:36.780542 [ 1476 ] {} CacheMetadata: Key 3270086aaa9f3831fa6e1d9f21b6d969 is removed from metadata 2024.10.22 01:35:36.970536 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_557_557_0, total 91 rows starting from the beginning of the part 2024.10.22 01:35:37.015242 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 62685 2024.10.22 01:35:37.376361 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:37.379440 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:37.571155 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:37.573409 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} MergeTask::MergeProjectionsStage: Merge sorted 63394 rows, containing 13 columns (13 merged, 0 gathered) in 4.898233457 sec., 12942.216935251317 rows/sec., 2.09 MiB/sec. 2024.10.22 01:35:37.576618 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:35:37.655710 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_949_949_0 (state Temporary) 2024.10.22 01:35:37.679527 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 8657 2024.10.22 01:35:37.744735 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_949_949_0 to 202410_962_962_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:37.767535 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 492 bytes for d15fca726044c7d679ec41fbe683d151:0) 2024.10.22 01:35:37.771884 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} MergedBlockOutputStream: filled checksums 202410_1_960_479 (state Temporary) 2024.10.22 01:35:37.777657 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 93c82491c26e0e0fc7f9efe1b335e654, offset: 0, size: 177 2024.10.22 01:35:37.786345 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 93c82491c26e0e0fc7f9efe1b335e654 for removal 2024.10.22 01:35:37.788362 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62685 2024.10.22 01:35:37.810247 [ 1476 ] {} CacheMetadata: Key 93c82491c26e0e0fc7f9efe1b335e654 is removed from metadata 2024.10.22 01:35:37.810758 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7c8355560d7b1bb364ed755a69e831a4, offset: 0, size: 177 2024.10.22 01:35:37.819780 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7c8355560d7b1bb364ed755a69e831a4 for removal 2024.10.22 01:35:37.841220 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 513103d37dc1a07c80d07a2e73e8d1f2, offset: 0, size: 184 2024.10.22 01:35:37.855858 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 513103d37dc1a07c80d07a2e73e8d1f2 for removal 2024.10.22 01:35:37.857811 [ 1476 ] {} CacheMetadata: Key 7c8355560d7b1bb364ed755a69e831a4 is removed from metadata 2024.10.22 01:35:37.867782 [ 1476 ] {} CacheMetadata: Key 513103d37dc1a07c80d07a2e73e8d1f2 is removed from metadata 2024.10.22 01:35:37.905090 [ 1697 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zec/imipawnyssrcebykreacgzkeipuwg, size 291330 2024.10.22 01:35:38.028956 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:38.034950 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:35:38.068554 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_562_562_0 (state Temporary) 2024.10.22 01:35:38.112714 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_562_562_0 to 202410_575_575_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:38.199583 [ 1418 ] {} S3ObjectStorage: Objects with paths [common/trw/xixavynjdlhuaomasemnsmivfzqqo, common/hxw/jeosmvctevrheobvbxrnjiuuolqvx, common/wzd/ydumrnuuxblqmpswrxigdxaaaxphv, common/ejg/jeajplyvesgqguiwsflermyvfnjsy, common/ksi/odzxgjxaelzljehdilcimleektafd, common/qhh/kvighpveeagmhfbrnlrupxnrfhoif, common/xuy/tmryrcpacnkbroqwpgjyhkuaaopyv, common/plm/cadfcgvqmnkeaochpyfpraeooznvi, common/pxs/froybchkpqbnkgzzgkowgvlpsasht, common/yol/mlyermslpastkyohbdqrrfudjdboe, common/ocr/dzqwlzaxdzaqxopkygowjitxezgwn, common/lsx/cefhhutxofsfibydrhihjzowrvpql, common/ubv/zpldbuitmhzvqfykkdxqyjputhbzp, common/xzp/lkvrwauiwqnvzkhrinryebyoyrjrh, common/mtm/tjgigljncxxnipdkuxvnlrucbfvnt, common/rsu/shsusmdnjhpzivmvhzyrbjqjzwpls, common/grb/mmggsbdbbpvgqmlwnglyvhcdlclbw, common/mjb/mqmyjxlsbehvkcretoyffdutemtfs, common/ltw/dzjzbfbwtykoyaybrzhamlyqjtpib, common/vnz/eelkjcvstviyzpfbsicudyzwcvugd, common/gzk/vrikrboyseonoxtcwfpppvjnpwjxs, common/mni/mbjjvgwcnrvepmtvkiyuqpvyoqpqb, common/wwr/jzgsmkvwcnmdsnufwllwinjmiihje, common/kfn/dehykeagdgoudwrcamuropcqjsxzi, common/fzn/hhntilfwfisasocpvcojobgifflxs, common/ebr/uifdjrgwuzghtfdyaacafmkrqjbfh, common/kxe/uwpxhkwmbzzsgbmijwaqswtlgeisw, common/ipf/hxzvdnrzvomxakwlxlpboxkaifcsy, common/uah/pxkafxonowbkaplsypmhrdkhkdjsn, common/opx/vmedomkmlnknjcgcdzbmbndydvwhj, common/qgy/vyscybahugvhlolfewkybvvnobyjn, common/kom/obnfdqoixnperbyaeuqbzhbvrezvi, common/nxn/ibuugwtfhnonqzbhsrveprizbwlbv, common/ppn/nrpfuwnvshxaxitryhabwwcfvsrvz, common/flc/oduycsqffhqvioymhtpffpgfjlsnc, common/xtt/dvyjroejmgztgfjuruzirawywhyvx, common/kne/drsiaszpsdwidhyreyiejorddcseh, common/lxl/qjbmalnjiyaklcmylrkmupfvpilvn, common/xhe/pstwjvyzvuqgsdamuannigcayfmoy, common/rgf/whcbwkcbbntndwofppwxccvikofad, common/zah/tbwmgxrbrqmjudoamdovalpuewgtu, common/oyw/ibxuvkkkrgvfpfrpuqyvyggcclmla, common/gef/nvqtfmfuobevufxerpczimdwvwmbt, common/jgp/fmlncpxfipczgbllasvvfavloultj, common/ksb/kdxrriqkirnkhzsoqtsnkeptqfkip, common/hta/iirmhaqfcyniwqbjgzxluljlunulm, common/xrh/ounnjunvgbdvcadvifvkpmrsyexqk, common/cbe/fdpopstcloqiuzappktmhdgshesgp, common/xjo/pbnokvaugwmrcrcywirnmibtkbfyi, common/ohy/yhpqvpdwhkdaafffbwsqtuknfqpcw, common/dab/orwsfecdtaxemojcjwlkzwcdzmrru, common/qeu/jfuldncdunkumcbqrvnfawplemnep, common/dfj/tkhqkozpuuhekgpzjpbrrgmrwtulm, common/jlh/dtusmbcskfbuubzwjinfthvcyvdkz, common/ntn/yaxkrenagknobikqaxpdymnndgqnp, common/zml/hwiuxmxzkgnwomfmmlwvmwkbsdqxu, common/mnl/jmhwjuslckxguxkgrqvmcrrkemrty, common/xtu/llkfundowuvrcgpycuighxeiwdksk, common/dfg/tjznjqcpqdfggbdgdxubqdflbijje, common/yni/rvumabzhetnxsfudmzycrzavuxuiw, common/apb/hjvwovaedkmyrjgmiajxenejuepbc, common/eix/llnmqvdegonumsnepjwnglwjyvnes, common/iew/fyavxdrtrjomxomohcnhizxvwhqvf, common/ipl/bgjbfyfvbfofzoslsaqzdfmcskvrk, common/bdz/bsayyfoabobhjtbyggrgcrhdqfpgj, common/vxs/hhrdxmldllkkcfbynmwldbgopmibe, common/gwr/kbpqctpslhwftaytdgwmrrkfwwztg, common/tnb/awsceifptcmyrmaqfzfrfgtdzuywn, common/mzh/kwgxvjgujqvgyfrjxtmiqhbgjdasg, common/upf/sbwhpivbqaczaqbjccbprrayylftj, common/jef/awadesgnqukruivjdrpchtwjqkpza, common/euj/mbnnslijghnrvgstuglvwpriojosm, common/xjg/qowezowxbclexodsiomqqjjywxjbn, common/jss/qmzfzefbsvxiytlcnqonpsgsfwtkz, common/zgr/wzucugqgomstzixubmioerfzkzjur, common/cnh/qlbpsijxmlrydghsxlbjnltylbnbp, common/ntk/dwulklsastmbljkgvnzarntdsqkpe, common/mbn/tijzipxryytwqbgjcpzbiykmasejl, common/qzv/soovuxirzbfsaulrnuivfqmgfkuzk, common/tsf/gmnfmudjxnkrlixwrlhuqmhufktfx, common/cpt/kbccbvfuzboveopxytoovihtrqlwd, common/guo/kythquqtghshwebwpzhwwrqleszay, common/bhb/mbpajjdptxgigcorhoohlibcdekvx, common/ssh/bxtrrxeudmbqsyidetfyianowfkzh, common/udl/tubihmitmiwtihulhhmxjzbbptnqc, common/iwp/ruxkvefkktjvacptpztvodwvfnpcq, common/qel/ccgqkakgzjzsjemdflxfbgchxjyre, common/tqb/tolcsqwxfkqfggqtftibakuritpbj, common/jfg/hxmsndobtfebtubtemlogpkxboobm, common/hig/phcaryfidscmomdekajlehcrsbbth, common/fli/csseadwdqrffezjvreuctznrqccrw, common/yjl/smyeibupuptzddmvspwwxvndqoeqs, common/bmj/gmfnpnvdjjlxjbyczbrvpbvmgtarr, common/srm/ujufhpwsnfkqyxcvbneqwsuadpuca, common/ptb/yjvqedpxaoisisrvkgchfwppgasjm, common/wmi/stpsoptjqmxskafnhwywvffltgjpg, common/xqo/nfmmtjtppnncqaxktcqtcjzmaxeio, common/awr/kiuzmewvvyamgkiwjapdrgxkiigai, common/ytl/nptkvivvcmqtyltptpexngqilhige, common/lui/iigvihppcoaxoacgoqbelfhfhyzlt, common/yok/wrgpkipcruejndooiysnbxcsdzgul, common/gyz/uquphfcgnbhmcnxkjftiurzrivcza, common/faf/wjmqwhrcfbzyitkzdndnqvzbsqonq, common/wnh/vqfrrchmimvswmnwcxmoujbqotyqt, common/hqg/bnmkjamsswnbvzlbjjhahfjzusivg, common/kco/epvptmsqxheslfdvwjzdjrfvyvpse, common/wwm/dehrvnrusxohftpekfnhkngrrsezj, common/zes/apbdvzfcbdicxfoitkhzkcfebwidb, common/jxx/weqgqbklokzxdfdoegfzfihhkchuz, common/ser/mbomkhwqqonoqorsebjmbveuyaoku, common/edy/ewzalkzhrzrizswqmdpgxyhvazzwz, common/kwu/yugplplbshrwywmjnjhnppiuaxekr, common/nnz/cfmcdkowrtythngjssnvreeqkctqi, common/eou/tmklfdhgiplaltrzwuqohcpnfvakl, common/uzu/urjgdiazalxnhirqcmqwtdnnvxmps, common/xzc/oprlyozdfsbvpxuvkxvrujyybgwhy, common/tje/zrgvkbzseggzopehlyirapcgfgwgi, common/qre/mbvqikstjprygsiumvqushxeenoak, common/hbl/pyupxruwngtsincjphmbterhvzacy, common/pik/avbqvyszbrxukidxrcwwlafllgfmv, common/qkt/iwudnxxeehtqunwxyovkoetiblqet, common/dme/yuqkejjeblolhvddcnutwfjcqfhob, common/lag/auajpwklznxgzphgexwafrjkdqlhe, common/hlh/ldmeyrnidrxmalpuqmsdpegeohwsi, common/dhq/pycbxaypjxpwltqjxlbpuqoiwulwh, common/jhz/vuaoxrxuwjongyqstydkrmbhjrtct, common/kjh/bzivnfiqgcbdiwvewjlbnnguynseq, common/rti/avkwderqolwcpoahfltihfukrbhij, common/rjw/kimfbknkdnjscvfaktipgdnrorhyp, common/xfd/eluwsylmlprtbsenigcrtgmhoifji, common/iui/hakhdzuugohwssckvynrtmtozslzs, common/ebu/rosccqvxqzxnuhaqcalwcryikdeth, common/wvn/bypdboiaijgpkubggxmxzzcvrlugi, common/vdh/dggjjtpcqnpossqiwnmsnhigcccog, common/kxy/axxdwpldcwtlpzqvetnfbiyitdsnw, common/nsc/jkqsaqhozjsfhizcwcupeevmkidop, common/xmo/jscgaxgrxgbbhqplnxmaidswazrsw, common/lek/ioczpkprddtdeodotbckbqkxzwtkp, common/gis/tcimvszqlobswhboqofgtnzeqguwl, common/udr/uncludpqofbvfxwvndpkziqnswnrb, common/cpk/mfgbtltxkfywjbhswsqdtkmuctnoi, common/umw/hwqvhiqywqkyianmauosqrlyhylgu, common/uhw/zdvwsfwwwzbytoqtipxihawqnisqa, common/ini/kemoggkwccbxjxzsxzssjwhwnvklf, common/swm/tizxugwqlwqpxsjvbicmyftfjldal, common/rtu/egxrfxyugeqgkqpsjvctmlljpqbxn, common/cxg/nosvvqgvsrzktmodxicurelregihs, common/xsm/spwuvermupxpsxadmegudnfgapsae, common/bpw/lexsxwjxonouadqcilgzaiaqiznps, common/mmh/inskgnkvbtuwjublcbfumwrplqqjh, common/jdo/tcrjiyigcfbcsfemkemkvvukmnhbm, common/gtl/zrupnqzeeocruzzkxlzslkqmqoiqe, common/nig/roxutxbgdkkjijykrqhcmovouttej, common/etp/fxsnmecmusyvnzkocrszvjhrreals, common/xdb/vvykpwbcimlewtsnjviabmbtewcvr, common/woc/mnudfmmypgafwlupzyhggkxjdazmn, common/qhu/mbvnmctqbyqecbabrehweklccohai, common/aro/wtfivmimzpazjaehmqyankipllbte, common/hpf/tismiwxmbjnkvoubsbuvpfhmdyusf, common/jzu/acioiblfxysgrcboizeaydytnbdca, common/smb/rmzemeyolvzicezksqvttglkpjilo, common/zbn/dcwzafnpiovrjgipjycfozmvgalmb, common/gva/jnthsntmqssmvbgfqvdgtlorqzptv, common/imd/byhiflpcbpyszaurmgondxumqqdcn, common/ivl/zwvrifzxuetxwhvovcuzkzessrtkj, common/iiu/ssncecyyjuzjbrhxwpkbkggcfxvus, common/owt/apjxobyprjwjftqsmaqrfiukzfshz, common/pqx/kwmxiepvliqaiduzpdidjchrbbghl, common/ivn/huxrwlskajxppvxntnhpfycvtlnfk, common/hgc/ottpekdukqgjwesuyiepetpciuvms, common/tig/ficmqsduanzwezzbgdkjvodsonyfp, common/nwk/adyoltnxzzzvgihufwvehjwgfninb, common/ohp/afpgxvrdnxzttlqtxeiucewerszhd, common/pck/tofuezwxazxanxfckbkbdjnwlywtv, common/ovr/mdgebhyhgihwjnflydntdzyqhzlas, common/mro/lbhjptzteefntntihdmtjojpojwem, common/als/gqmjlgshwcnqsubrqemgeyciiwxzx, common/vbr/pdtepczjiqdxdcsajovhnnwzjxbfl, common/efb/xagcajlgiojzheareiaeqtfkuyznz, common/oyv/szivomtxvonrxoqoffukqdfqobjgb, common/wqf/fkvlprrhnwknmnnkyxsxoaldzxzhh, common/jvh/ewfrbtvwrlbduvfcryypqihvvdpqk, common/qcq/tqcyejrkltaclraibqocjzvjdfpfj, common/hur/ijprstdeipcqlqrqphfntynotlxmb, common/kxr/bacrxeyvwvnwflpvwtqaxdyioyzwl, common/kmg/gxptphgirbesvdkywzvaxfirvwgvc, common/fvm/aubekidqlyjcyoklfyqnjdlechulg, common/kpr/lciawabxvbsbkrttmnziqlydqxvoj, common/cro/fvoxxrotepmqvoexwsznqccelwquk, common/vnb/nbboczierrmibzbvgedxpuoqbqbvh, common/nxl/dwnbidxckdlqemwkfcufaqeltrtxe, common/buo/mtlhnhbqevfesmcqbaomqzrxxaqzu, common/vng/yspvoiltuiakaqhajwdzhutvlgejo, common/jyu/qoguxkrlmjtnydhexramltusicdiz, common/pys/fxasiperlxrxxgwtpckzvitjohgcz, common/zkr/ldvnahapgijajiwfavbabevpymtxr, common/dya/rwaopomyahddqtcviwssgfkurfcit, common/sln/ytpnbweefpyilrokvbwzxjsxsprkc, common/dah/edkatmxkpwkednajuqkgejstjcrpy, common/ijq/naswbicbyswofxgmzmzltjdciklwr, common/lhq/hpkfymzgubkxfbcboatfbuhzxnryz, common/yed/paccqkbmccvgbzdqamwvuhhakubwd, common/dlh/dbvtqzoqzgywuziwwkeaxpeubgjou, common/fwy/nujphvrvcgdomhcqjeeqotdnmrgks, common/ynk/cyiqjiklvzqziaahxsjrqjrmzwgof, common/dzr/lcobinesvqjyxlcdexjpagbgbgnjo, common/gmf/imcsswwthvbfikhzfffulewbcpsjr, common/hoa/nugxfqgurphtbvgepxvamjliovgfx, common/lyp/gmkoubsafefhxjhbtkealquxslixw, common/bjy/gsjpaxsyrikvswoalniunjdgbfuhn, common/vtm/ebdjeifqosrhapepvwfhlyeellqns, common/urg/cbmnpifusdtfrpejuxbcukryrcwek, common/zvj/xptfhgxazguhushikqavkdzdstxgt, common/ylt/xbdueommibavtfyyskapdouimzsgj, common/zpe/nyzqmrasxzjrjugtsoqusmahfidjx, common/jnz/urnioqcrljcqtvwfsrupxoqygfpnu, common/uxl/tkyjwxkwxbmzubxraqvlvfifrxxhn, common/ilc/ujjqijymipjdcgeuotqphyxotzxmf, common/vhc/gkibgpuqnttmoyvyyadguavapsbpo, common/vdl/thwpdkipkftvuulsfsanjuoyijwic, common/ooe/lpqxecbatxgmrdgdloxaqjqjnpoyk, common/cqh/tswxxyzkhfprydcovepsveeklbuem, common/tmi/holulvyywwtuvqtdboofjuonnhtdf, common/ijh/vkvsuugpddiamcnuhischkmzdiqqw, common/lwv/grbkfuetgfwjgsyeyotnkcxeknldk, common/kuh/ojbhuaqkugeryqvdkfjlrjfilsqov, common/lud/pbhnlianjhwthmnifndmndgubpxyj, common/anu/mmexcxuomyujwxgrjrhpxulpzwgmu, common/hwv/pwhbwvgrwyulfgfwavneogitgvazk, common/cki/oowpqthkafznntcxrkefjgtlpoatb, common/fjw/sbvmtccwhdpzorqbxtaxcxmngcytp, common/yiv/qhqigykpgilbqgbcmiyrodrxpardf, common/oyd/cnqukdgkmpvgamixxvcnchmrrjayi, common/iyl/lxjdwqbzvqfjzkjgltujngwyjextg, common/qlo/ogkuhuemivcwnogilegpglwwqsqkf, common/fpb/ujqqhknxxlkccvseinczdzkbmapvk, common/aun/vmuffktgmeelzpahswjkvknxqnacp, common/wlx/ouqhwocyuieevmijsenqtoeofkghp, common/fod/pzsfoobqvjesgrkmnfgllyywvrskr, common/dat/tpsksoikorkcvuvdmelmvyozctfzp, common/cyy/rcfdolcewanlvevjiywktldxqusgj, common/fgk/ehlhdiixoyzdfhmlffxbqvwzyghnb, common/uzi/iqcsdnqsmnypruurhwddwgmzzwzjy, common/lqm/juzzibbwmqxhowcnwcairadxdvxnj, common/yia/uhkqqfrrsiqixicpxorwmiddgxjsb, common/ksz/zhfnrfvhgmymfljuijphhdtfofowm, common/dvb/nophnyiakflaopgeqwnifnzgzkxlx, common/jco/grwprqkvlerdepckpwyazvzvwogjl, common/yqv/opcrwoexlnzvcahijnhiylkmfzvoq, common/avw/mmdobgvxfddnzrlbyiideryasqyby, common/unh/fwuyupqokgumpeuxvuovbvidchlcx, common/wut/fhekgdcacpbuetglamdlutiqlqbrf, common/wba/tsxormamwyfmxkwxykgrhnwoxlksr, common/dwz/mxgwvllduuzwlrzzilykttohxmpbx, common/lqu/nqifbjyplzduemcvrjzqqgiaocyhs, common/sjo/hsipabahgdalbnvaeiajvexkqbwlu, common/ywk/hwtgwpxzflsvvcnqiubmsezegdfwq, common/dkh/kbetmiljjexyvhfcaaqtriixwdgqu, common/iff/dprhijijplgzkalpdgjxxdqookzsh, common/mhx/ryubrrcoxtejnfnmurnlvpbsmvzyh, common/hzs/bhyonimwxoatcrhhpwpvndkmmydes, common/sxr/lschfggkdajniwoqrwtyneizjwefx, common/uns/yoxeiugtqjyomjqzhxkwxpbxqwmlm, common/uew/ahqfbkywbwclulplbwtpqowbculwy, common/cul/wkmojnfckespfmwoogstumvogzomg, common/vsc/zotthntcwcbdamzhvcxgfkqcryuwf, common/ojh/eevsuxranksyeaenkmmsomkwtnime, common/tfk/iqjfkddjonnsrqyvocheaxbukaodu, common/txl/fmdipeopdfbajgzhvbhhwhogbviib, common/glc/ikjkdxsrbhxcypxvkmghbvlodzphy, common/mgg/jnohhqfodqwwycozdjzxngmwnstad, common/leu/xvdehbqqzbagoqkmscxyztxlakvjv, common/ail/zollslsfmhdoywotsdvwculteazsa, common/kqa/pnknfsbkcyzmablhzvwmhmwqlcbyr, common/swd/qrlbucqbwgrparyixnmzhgftpumhp, common/kpu/igdxbheozmsvsgcnpnmwutmcyktek, common/tpx/muocwrvtoevqqmumsxuynsnpkiowf, common/gkx/ziywflghzxpqrpvfxyoaxfyeouuqq, common/kye/evnndlhzkimspbgajwpusqujndecn, common/sgv/sdidtjzxswbxxhsndjbfjezuayzrr, common/okf/xcoselpywwlmuhzlvvbpnuehpfgpg, common/hpw/rilhmbrbknjpyypuouwggepckxcby, common/lcn/lwyjjhkckpqhfuxridivbnrhizamj, common/akh/jlcqzfiubrzuqfggefisrowqivyse, common/upy/illifykmwszmssqqcslhhaubisjra, common/owd/vefkpnfskkelfwdwenquworsialfq, common/vcj/udkmulbdemqegqmdcibbjwakxqxhm, common/pye/rceibkafmagezuhevqcooixycqegz, common/bgu/qerntrojxvnfvfnaleeeghqacghsa, common/ttd/eujbjzxfgimzusnwsvzwywbedoqtt, common/cxw/ewfhemppcntumglsarfgmnvtxfafd, common/nqh/lzsdykluynahudozhhncfnxqalzjb, common/xhh/heyqkxomlyvtybqkdrqkfhedwdhnc, common/tvt/jfurbotijnbtklyyfdjafrzbawxmy, common/ouc/uwypezobnqjtpnljkfxwtztepqekm, common/uis/zugfrsbfovipdynbedwreiitpiecf, common/vcw/bivqscxqzngtaztceuqxheasvtdlu, common/psm/tulmbhcghgazgtsrrgtikvnzngxhh, common/qbh/lypnmrzgyyrvdoeomomewmiulriit, common/ray/foslpwzrsnizmcheflfgryzlddviu, common/opy/nmoafklwqcqnpsgthkbpsepuqhlvz, common/tye/iwsywlwyzjxkqwvusihqphlzmgflq, common/dhd/suekyfmxydxyskugrozkzsmwwsjna, common/zgq/wnpujfubtqyxbfabiwjvnnwtzpujh, common/mxi/tdibivpqztjuxbdtkbswbcdlvylpx, common/don/hwartdgxzzqoputqawlzyzwfozpyq, common/tzp/hevcwyzxmoblpumutyszhuajsuwih, common/oxe/yyxlpbommyeqkbsrcanfguoxljvzs, common/nnh/cqootsbamfphsstnnnuyjtyawmqfl, common/zvd/pebomztsqstgyeiimobsfvgemydon, common/oca/gucvnijlinfduehkmtmsshatieptc, common/xgl/dlduutfurpwhreyisdzrfdksjfvbi, common/mxk/ktaffphcudajyfrosjacdgfbnlqnm, common/mdd/juyqpdlknkykelqcbphbtelzbofjw, common/uhu/dqkxiuipvcrrrxmlhpdlctmmrcfjd, common/sen/jfcmoybpnrbnsyglykmzdkjkeoesc, common/saw/lwqvxgznuvxgxaomuxfstqqrqxkyf, common/dgg/hhedqjoqqfcsfghtydbacdvzvuwak, common/nly/icgyftwkdjnugwtmublfwxulevcjk, common/viz/iuyhnvqpsiuggfextrrtztcdyyumh, common/isb/cjgyiybnssuqkftjsoqizcawihhto, common/niu/ngqjauuwqtibjtbknwgbgaxvzxddv, common/ybs/jbeywgdttvrhjkbwtamntxwldlidk, common/jgk/bcijuabugkbuylmuarlopynkzooma, common/jya/xktnnhescdrddbgezcynovvozdykj, common/ret/qzcmeauxgvplrnckkioldaeqarraz, common/wmk/egxtizvmzeotpxjpbwlpdinowames, common/dcn/iwextelktmxpwbvsvrexaxfdtlmtg, common/rlz/irbvltdbzztoqbrxomyekhzqtfexn, common/yli/nuhoamnvqiiqoqrcjjpflmdeiqxdb, common/fiv/ykvgsrrwcnvmpbegvxjxomddkmicr, common/uoe/fuvyskofmgctklvuzzdvoqtwnksgy, common/kyp/pulbkzwweimoanvpqgtydchrlxayl, common/tbn/jlsowexunvysqdpbxojdmdiyufcqh, common/alg/fwxmxqkqkhjnoxrrwlgkevixtwkuo, common/vbw/yqpggnnkqrlzedflkigxnmndobhfn, common/vfb/qstanldokvwqhqqpswjfonjmuciip, common/ych/qpqbmhaegkpchjukewkbodvplubsm, common/cir/ggwcovktnresirpfccuijujgovflf, common/sxb/ygqdyyuporqckswrdyjygzfkvfckf, common/jwx/bvvehcxwyrengsnftwnqspbrtzgvf, common/hta/mptiiytzcfuvpbnlinobwjsesmikx, common/xjg/czhgenbbrjmlibmovnimvxgiwdsbr, common/ejk/kvcxbrlixhmklqirydeotuftrsxlc, common/esm/lymxzkkwfwzjnzgvfnmjixahlybhi, common/efl/qkjvcqtteqbylcrsuhuzkfhmpfjzd, common/det/jqkomuvlafwbglociltrfoceegxpo, common/els/uznjohrbhlpubjfiqbjxvxqskimsa, common/laa/djvzhjiupvjglmobiohurkavjebjg, common/iuy/atwjcmkcvpoywbahxglzajolelqek, common/qqj/ndttxkfugqeamxunjpkxvykkdwldt, common/xnz/krbptrboenfttivckuowgwxthrmdr, common/tls/trjnoirhvxxlthaffpqedttnbsgjg, common/uwr/elbybmlvfkqgublpxymchpmojaypk, common/vhc/khwwinbjwlrmrgrdlthvsptfynrra, common/clo/axmabhiuqdatkcraqaaiwdslaipcs, common/ztd/fvkceewleuebdhprwpwblniueytej, common/zsq/sxmcnvnvzjomlhoifwelasdjkzzdu, common/pde/rukqtnvoxaixhplhrgkvnqvcbniox, common/zpc/obwdaacwakhhpjihuklhalldnaiqq, common/nsa/fojritcfdtiltujxtpttnwsmonlya, common/fht/qsmesvjcmbwiwxpkmptnsnekhhjvk, common/nun/tegxzhihyrzehkyjxmtregdwcmkex] were removed from S3 2024.10.22 01:35:38.218413 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_960_479 to 202410_1_960_479 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:38.226223 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_960_479} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_955_478, 202410_960_960_0] -> 202410_1_960_479 2024.10.22 01:35:38.273694 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8657 2024.10.22 01:35:38.276041 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.14 MiB. 2024.10.22 01:35:38.312185 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:35:38.337274 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:35:38.356573 [ 1418 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CounterClass.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/MobilePhoneModel.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SearchEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CounterClass.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/MobilePhoneModel.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/MobilePhone.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/MobilePhone.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SearchEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_4_4_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 01:35:38.424577 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:35:38.448400 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:35:38.468045 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:35:38.477624 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 6.29 MiB, will set to 1.74 GiB (RSS), difference: 14.90 MiB 2024.10.22 01:35:38.493084 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:35:38.514187 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:35:38.559765 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 01:35:38.587150 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/luq/tsfkenobwaznbjfmnjjoggvifgdwx (LogSeriesLimiter: on interval from 2024-10-22 01:35:16 to 2024-10-22 01:35:38 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:35:39.024513 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(349137ab361ac01a19a746bebc8a24c8) : [0, 373068]: Setting download as failed: File segment: [0, 373068], key: 349137ab361ac01a19a746bebc8a24c8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:39.036307 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(349137ab361ac01a19a746bebc8a24c8): No space left in cache to reserve 373069 bytes, will continue without cache download 2024.10.22 01:35:39.173758 [ 1224 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:35:39.205803 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 349137ab361ac01a19a746bebc8a24c8, offset: 0, size: 0 2024.10.22 01:35:39.211519 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 349137ab361ac01a19a746bebc8a24c8 for removal 2024.10.22 01:35:39.216746 [ 1476 ] {} CacheMetadata: Key 349137ab361ac01a19a746bebc8a24c8 is removed from metadata 2024.10.22 01:35:39.259866 [ 1224 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:35:39.322680 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_538_107} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_538_107 to 202410_1_538_107 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:39.328337 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_538_107} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_532_106, 202410_533_538_1] -> 202410_1_538_107 2024.10.22 01:35:39.365210 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} MergeTask::MergeProjectionsStage: Merge sorted 5075 rows, containing 13 columns (13 merged, 0 gathered) in 3.58999189 sec., 1413.6522185848169 rows/sec., 626.74 KiB/sec. 2024.10.22 01:35:39.368071 [ 1392 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_538_107}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 01:35:39.386527 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:35:39.644297 [ 1218 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 264 ms. 2024.10.22 01:35:39.799182 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} MergedBlockOutputStream: filled checksums 202410_507_557_10 (state Temporary) 2024.10.22 01:35:40.119666 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_856_434, 202410_857_857_0] 2024.10.22 01:35:40.123639 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_856_434, 202410_857_857_0] 2024.10.22 01:35:40.138915 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_856_434, 202410_857_857_0] 2024.10.22 01:35:40.274110 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_557_10 to 202410_507_557_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:40.276695 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_557_10} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_552_9, 202410_557_557_0] -> 202410_507_557_10 2024.10.22 01:35:40.388499 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 170 bytes for d5b439b43b288f2f5f698f08f60d3bde:0) 2024.10.22 01:35:40.398780 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 09e841488b0becca5ffb7dc2e774a1c2, offset: 0, size: 177 2024.10.22 01:35:40.407166 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:35:40.411034 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 09e841488b0becca5ffb7dc2e774a1c2 for removal 2024.10.22 01:35:40.411172 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:35:40.417623 [ 1476 ] {} CacheMetadata: Key 09e841488b0becca5ffb7dc2e774a1c2 is removed from metadata 2024.10.22 01:35:40.426745 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.51 MiB. 2024.10.22 01:35:40.607792 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 473 2024.10.22 01:35:40.847921 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:40.847179 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_113_113_0 (state Temporary) 2024.10.22 01:35:40.854427 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:35:41.011268 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_455_455_0 (state Temporary) 2024.10.22 01:35:41.122471 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_455_455_0 to 202410_465_465_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:41.235163 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 473 2024.10.22 01:35:41.566095 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 8.27 MiB, will set to 1.74 GiB (RSS), difference: -10.30 MiB 2024.10.22 01:35:41.804947 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 62762 2024.10.22 01:35:41.871970 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_511_102, 202410_512_512_0, 202410_513_513_0, 202410_514_514_0, 202410_515_515_0, 202410_516_516_0] 2024.10.22 01:35:41.874428 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_511_102, 202410_512_512_0, 202410_513_513_0, 202410_514_514_0, 202410_515_515_0, 202410_516_516_0] 2024.10.22 01:35:41.899130 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_511_102, 202410_512_512_0, 202410_513_513_0, 202410_514_514_0, 202410_515_515_0, 202410_516_516_0] 2024.10.22 01:35:41.922882 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(71b9fe1c707772b33382a0c0a4f640e3) : [0, 373310]: Setting download as failed: File segment: [0, 373310], key: 71b9fe1c707772b33382a0c0a4f640e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:41.925831 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(71b9fe1c707772b33382a0c0a4f640e3): No space left in cache to reserve 373311 bytes, will continue without cache download 2024.10.22 01:35:41.974928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 71b9fe1c707772b33382a0c0a4f640e3, offset: 0, size: 0 2024.10.22 01:35:41.976863 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 71b9fe1c707772b33382a0c0a4f640e3 for removal 2024.10.22 01:35:41.978167 [ 1476 ] {} CacheMetadata: Key 71b9fe1c707772b33382a0c0a4f640e3 is removed from metadata 2024.10.22 01:35:42.080835 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:42.084842 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:35:42.177522 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_950_950_0 (state Temporary) 2024.10.22 01:35:42.325700 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_950_950_0 to 202410_963_963_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:42.377561 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62762 2024.10.22 01:35:42.659586 [ 1162 ] {} DatabaseCatalog: Execution took 208 ms. 2024.10.22 01:35:42.762170 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 162 bytes for e09735ac397494c446e75d6502bcb913:0) 2024.10.22 01:35:42.764290 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 17e35e8fc00b55c631f73172f377df17, offset: 0, size: 147 2024.10.22 01:35:42.768758 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 17e35e8fc00b55c631f73172f377df17 for removal 2024.10.22 01:35:42.771630 [ 1476 ] {} CacheMetadata: Key 17e35e8fc00b55c631f73172f377df17 is removed from metadata 2024.10.22 01:35:43.045181 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 64284 2024.10.22 01:35:43.074945 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 848989 2024.10.22 01:35:43.364737 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:43.367397 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:35:43.384536 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 7.74 MiB, will set to 1.73 GiB (RSS), difference: -1.41 MiB 2024.10.22 01:35:43.391335 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 168 bytes for 3bae7ba094c43c5fbe6f9e28f2c5060a:0) 2024.10.22 01:35:43.395683 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b409cfaf966853021900fe987f3acadb, offset: 0, size: 177 2024.10.22 01:35:43.403261 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b409cfaf966853021900fe987f3acadb for removal 2024.10.22 01:35:43.409911 [ 1476 ] {} CacheMetadata: Key b409cfaf966853021900fe987f3acadb is removed from metadata 2024.10.22 01:35:43.617930 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_552_552_0 (state Temporary) 2024.10.22 01:35:43.716711 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:43.726475 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:35:43.926085 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_552_552_0 to 202410_558_558_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:43.962593 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_539_539_0 (state Temporary) 2024.10.22 01:35:44.051082 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_539_539_0 to 202410_546_546_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:44.068548 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 64284 2024.10.22 01:35:44.076913 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 848989 2024.10.22 01:35:44.404513 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 483 bytes for 43d024c81c23252f8694752ee9fa14c5:0) 2024.10.22 01:35:44.413277 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c6d46d7ad86e8edc36a161f51fc2bdfc, offset: 0, size: 161 2024.10.22 01:35:44.418769 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c6d46d7ad86e8edc36a161f51fc2bdfc for removal 2024.10.22 01:35:44.424696 [ 1476 ] {} CacheMetadata: Key c6d46d7ad86e8edc36a161f51fc2bdfc is removed from metadata 2024.10.22 01:35:44.426255 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9765862751548021f23f562112d68023, offset: 0, size: 173 2024.10.22 01:35:44.431764 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9765862751548021f23f562112d68023 for removal 2024.10.22 01:35:44.440185 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d0edb9c113dec498ed48e765050d46e8, offset: 0, size: 169 2024.10.22 01:35:44.440851 [ 1476 ] {} CacheMetadata: Key 9765862751548021f23f562112d68023 is removed from metadata 2024.10.22 01:35:44.444867 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d0edb9c113dec498ed48e765050d46e8 for removal 2024.10.22 01:35:44.449535 [ 1476 ] {} CacheMetadata: Key d0edb9c113dec498ed48e765050d46e8 is removed from metadata 2024.10.22 01:35:45.544768 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e308e6498e253637bda4be7796f9dd81) : [0, 313716]: Setting download as failed: File segment: [0, 313716], key: e308e6498e253637bda4be7796f9dd81, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:45.549049 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e308e6498e253637bda4be7796f9dd81): No space left in cache to reserve 313717 bytes, will continue without cache download 2024.10.22 01:35:45.674330 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e308e6498e253637bda4be7796f9dd81, offset: 0, size: 0 2024.10.22 01:35:45.676395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e308e6498e253637bda4be7796f9dd81 for removal 2024.10.22 01:35:45.679274 [ 1476 ] {} CacheMetadata: Key e308e6498e253637bda4be7796f9dd81 is removed from metadata 2024.10.22 01:35:45.775348 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 8680 2024.10.22 01:35:45.927823 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_857_435, 202410_858_858_0] 2024.10.22 01:35:45.930432 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_857_435, 202410_858_858_0] 2024.10.22 01:35:45.948671 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_857_435, 202410_858_858_0] 2024.10.22 01:35:45.972912 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:45.975637 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:35:46.012496 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_563_563_0 (state Temporary) 2024.10.22 01:35:46.160119 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_563_563_0 to 202410_576_576_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:46.180842 [ 1172 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_571_114 to 202410_576_576_0 2024.10.22 01:35:46.267430 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8680 2024.10.22 01:35:46.308853 [ 1172 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:46.326455 [ 1172 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:35:46.386346 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 62825 2024.10.22 01:35:46.387758 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_571_114 to 202410_576_576_0 into Compact with storage Full 2024.10.22 01:35:46.712463 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 537 bytes for 88193f83256faf63b5cbc93d82512275:0) 2024.10.22 01:35:46.715999 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 989bcac9a1b27372f935b58d1d297e5e, offset: 0, size: 6132 2024.10.22 01:35:46.724080 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 989bcac9a1b27372f935b58d1d297e5e for removal 2024.10.22 01:35:46.729660 [ 1476 ] {} CacheMetadata: Key 989bcac9a1b27372f935b58d1d297e5e is removed from metadata 2024.10.22 01:35:46.731520 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:46.739522 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:35:46.773081 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:35:46.787920 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_571_114, total 8711 rows starting from the beginning of the part 2024.10.22 01:35:46.833179 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_572_572_0, total 21 rows starting from the beginning of the part 2024.10.22 01:35:46.876945 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_573_573_0, total 6 rows starting from the beginning of the part 2024.10.22 01:35:46.921308 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_574_574_0, total 20 rows starting from the beginning of the part 2024.10.22 01:35:46.963500 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_575_575_0, total 5 rows starting from the beginning of the part 2024.10.22 01:35:47.001599 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_951_951_0 (state Temporary) 2024.10.22 01:35:47.061510 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_951_951_0 to 202410_964_964_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:47.110769 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_576_576_0, total 23 rows starting from the beginning of the part 2024.10.22 01:35:47.133032 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62825 2024.10.22 01:35:47.338356 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:35:47.374959 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:35:47.399470 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:35:47.453343 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 9.01 MiB, will set to 1.74 GiB (RSS), difference: -3.85 MiB 2024.10.22 01:35:47.526645 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:35:47.559947 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:35:47.682661 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:35:47.723614 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:35:47.750475 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:35:47.767875 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:35:47.849663 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ojq/hfybufjjavwwigwkpnvhjhphetysk (LogSeriesLimiter: on interval from 2024-10-22 01:35:38 to 2024-10-22 01:35:47 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:35:47.908776 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_485_485_0, 202410_486_486_0, 202410_487_487_0, 202410_488_488_0, 202410_489_489_0, 202410_490_490_0] 2024.10.22 01:35:47.911495 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_485_485_0, 202410_486_486_0, 202410_487_487_0, 202410_488_488_0, 202410_489_489_0, 202410_490_490_0] 2024.10.22 01:35:47.944771 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_113_113_0 to 202410_116_116_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:48.002754 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_114_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:35:48.055336 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_485_485_0, 202410_486_486_0, 202410_487_487_0, 202410_488_488_0, 202410_489_489_0, 202410_490_490_0] 2024.10.22 01:35:48.698182 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_576_115/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:35:49.236952 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} MergeTask::MergeProjectionsStage: Merge sorted 8786 rows, containing 12 columns (12 merged, 0 gathered) in 2.854309351 sec., 3078.152687592131 rows/sec., 489.42 KiB/sec. 2024.10.22 01:35:49.285730 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} MergedBlockOutputStream: filled checksums 202410_1_576_115 (state Temporary) 2024.10.22 01:35:49.591207 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_576_115 to 202410_1_576_115 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:49.601468 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_576_115} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_571_114, 202410_576_576_0] -> 202410_1_576_115 2024.10.22 01:35:49.790318 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 6.93 MiB, will set to 1.74 GiB (RSS), difference: 1.14 MiB 2024.10.22 01:35:49.860371 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.57 MiB. 2024.10.22 01:35:50.409699 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:35:50.412661 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:35:50.614696 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(48c11213c8e4d91bce02ba7b70d02efc) : [0, 421057]: Setting download as failed: File segment: [0, 421057], key: 48c11213c8e4d91bce02ba7b70d02efc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:50.616571 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(48c11213c8e4d91bce02ba7b70d02efc): No space left in cache to reserve 421058 bytes, will continue without cache download 2024.10.22 01:35:50.719977 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_858_436, 202410_859_859_0] 2024.10.22 01:35:50.726880 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_858_436, 202410_859_859_0] 2024.10.22 01:35:50.750342 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_858_436, 202410_859_859_0] 2024.10.22 01:35:50.763943 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 48c11213c8e4d91bce02ba7b70d02efc, offset: 0, size: 0 2024.10.22 01:35:50.765433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 48c11213c8e4d91bce02ba7b70d02efc for removal 2024.10.22 01:35:50.779103 [ 1476 ] {} CacheMetadata: Key 48c11213c8e4d91bce02ba7b70d02efc is removed from metadata 2024.10.22 01:35:51.079963 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 999 entries to flush up to offset 849988 2024.10.22 01:35:51.143455 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 62877 2024.10.22 01:35:51.284425 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:51.287568 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:35:51.463423 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_952_952_0 (state Temporary) 2024.10.22 01:35:51.579291 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 126 entries to flush up to offset 64410 2024.10.22 01:35:51.681405 [ 1418 ] {} S3ObjectStorage: Objects with paths [common/xdg/bwkzmzlmpuyvprfkctagodisgngao, common/jte/ghzelizodzxfcrmkaszoxlspoborc, common/eht/ytujiqkanwivsjonbupqudvncmzub, common/vcf/xjltmjfuetbwzeueajxvqqfqefzzu, common/qez/cinbkptbmrirdnaeppqdykufwvmpz, common/ixj/hyxjiaojivqpqyarjwkfdgwdoxsih, common/wfi/ixcbexlqcobpgoxrhuxqbbdzljcte, common/qus/fjvslyvsggfmaimselvyifffxqdww, common/xri/regundfcyknocmmketzhudvnjwrlp, common/pqh/lyaludpjvbzscibptzrwpckeutcnw, common/gkd/mvmsvwmrlozzwkbnzcarclvpogvqh, common/fas/nffxrrceazwtihriihtdmnagyoxfg, common/azl/zbngmtwfpwxowtakaexrxnyoycrwn, common/hby/cglosjhttrktjjpxlolazjregfrnh, common/snq/ojoxjhuslnpvacvmztfrxripqqizs, common/tvb/xqayvsjukghghihututxwochtposz, common/nfd/xapckmuysorqlpadlrnbehszecrgp, common/poq/sismitjcauykdbrwshyytgbqwwofe, common/yic/tiozijgqykugzbceapptlmlepvoll, common/laq/mrzfdvcmwkkuurkkijbeqbwqepblh, common/svb/uwmrcordvoqzsqohuyubfgrwapjcd, common/tml/hfusbuylbjkrjwtzjyteyiwtxcpdu, common/wxh/ssvhqeytxlktvvfszrnogdeqhdfke, common/xmu/uvbvjriuzamgbgsfjoipgydfvgfpc, common/oix/jmzlmvghfuakibdzmdtmcwggnssen, common/ggh/xbaeydsqbvvskuotbfibhlvkiabxp, common/wmz/npkvrwtmdxyhkbsgptodrpfthhcnh, common/ehe/qkjvtpoloumeajisakcsqegvatqjq, common/rkr/jgxssjgpkjgwtithlgysnwqvziprf, common/qer/nccrmohmhirwbfboertcnqehivqug, common/xdt/ohyhddmauyjpvafuiybcwlijnnxmv, common/njv/ymrmrgfpvzcbpshhydkqmupufzjsx, common/rfl/zaouxbejnbanhwyyyfschcoyyjful, common/uhe/ofkokhexwpnfjjekqhfloobunomqc, common/xpq/gkiurdhwphrjpmtiradjutpoyqtst, common/gdw/intkpyfdswxhhvhborxlyhhaetebf, common/wye/wgzcofrmgbxhtfmflnjvgrflknstv, common/tzy/mbhntjewjhmiiljuzlqbndazebgto, common/fjm/exazwcdmhgminuukxdabgpbxipcmu, common/iyq/mjcnxkvvadriopznuxibymiwjmsjz, common/rmi/dvsjtwmyzvbeyctqskzarptuinzcd, common/vdc/hivetxqjfvlojjoeitghtxvmysyzc, common/ggc/acoodvjjjxkmvfzfuhusmokgirzgl, common/pwm/dvbxkbnnchoietdshevcsryyxtjai, common/qot/zvqgfpwdqghkfzzgqkspfqeqjzwvr, common/gdv/oyxnsqzsjeabraldxizbgprwssurc, common/sim/lxtpysordclohljdficzxjtmqiheq, common/jtz/ruzlhvrtqcktkliimvaayinsxdbyw, common/ayb/puucuhegrjguyozalogsdyiatefzn, common/vpm/qtazugwyvrgdjdyrvkptcvgusnnlc, common/fpd/myruxqchsccimhxyjbikmyzdguugt, common/wuh/xswrsyxnudixwxftzgdjawwpwmpft, common/puv/gfgenbndtbrpksamnjzxzatkfeghc, common/avo/dsfdmppqfgajllqpkrcfnqpxakcbm, common/bsb/xtcwicecjdgcxvpxgirfphsrwqeyc, common/owd/fahevmwvdavhawqcbggxyjaiewrfd, common/eic/ifvjrnexntihjubvnklybniijdrqm, common/swt/qrhttbmqcpzowxkzvgogfjulviaco, common/yru/hdtxouzmllifsmixgqmtxiqgvsosi, common/ggj/okpsgoryvawukztatlluctprpwndi, common/pis/koaocsidagdmxfamwhfcizmccgplk, common/drq/estrvdxsvfdiyhckpxolzsqqnwinr, common/crt/fhkewxrkutnessixtxgjxisslmsjd, common/xkk/wgdshfbjzqsqtxbzxkrhaaxdgylal, common/fgj/elphtjjhvdkdccewvouzffyxjiapc, common/uba/juzbkttqomtfayesnsonemawojogt, common/xeg/kjdetvttkfmtqyexmidhfxvujophp, common/gjf/jravhdalurfrqdhmduwaoyjftsrcc, common/wih/dgrtnjezzfvsbhgjrhwwxxsblmsho, common/qne/atmyxyyqtobhbukamjslxzexdtsgq, common/wxz/xnkwuipytmjbqwcwpjbexomnxutal, common/zrw/bwtcvgeuaclvdiwfputcxfcwvxphw, common/dlu/cyqhzsbpehewfkksrqcfzovpimwuu, common/njz/vjdayweotmfchjftsgoprlddfhfhg, common/moc/nmyibcukowryclczmhgsrnjvvxdyf, common/rkt/bnraybczekxukibwfomunpxqfrxgm, common/jwu/pobpxlfsvzwcbcihgyukttgmltenz, common/dok/jmbmdfvvflnfdimuhiylvfhrmhgmf, common/nxy/esbwlzwcjqgpuxwepwopfdrprxtpf, common/owx/gxvnqflelfzhrrgxficnixcbnkhxf, common/utn/qlgarugbkyxlcocswqqnwcpemjvvt, common/feh/uyynsnzvguyfclmbvoiyeqwtvgzmx, common/ads/dxfnmlzfdttecbvckvnxdbekryrdl, common/zqa/zhucremgapczpptzzpbhagdxncgoo, common/gxl/gcojzfxthcpjklozdxjhirgqksbla, common/igx/aejsnxnbjcnlhcboyzejsuvtrcpco, common/goj/wvmlsskwipbwmkwgocbddtumjohgs, common/ifk/mkyibdjhnobrsnmrwdredrkmmngez, common/qoj/lstmysxzglllnvwvczwvhykzpdyhl, common/lhb/bogsweqqbgjagfmvqkeqrubjinrtq, common/dwy/oocmudkplyxspfnvioimhdctnmkbh, common/nad/pjwvyijaspkdqfqrrrqvuoyxlvpgq, common/wdq/sliqkadixfilxzowsipybpjcqsoac, common/ucj/cfuolxonjobkfdgjstaureunrgqti, common/drv/ngxpdglefqegbdqmvssekomkzkidv, common/qnr/gcteboqljimiuiqrzjzizbwjacjpc, common/vhe/dkdxnrvjlqstkocswmvaxibgjstrl, common/qoj/hjhscowmltffasqciflyuohnrhlah, common/uey/rdooxsvpmupwrmdqcrlhglkgdpvjv, common/nip/wrlhekrngxkllkescwkcbllaqbawr, common/hip/ojcvobzmxbyctqcobzwavfubrftkp, common/alw/nkjskrfzuqyaitpoaftczeddzsgrm, common/nxu/axqhapnxodxhkytoduwyxulrmbpan, common/eym/bjmbrbihjjlkrevsvnvkadaerzdwq, common/lgm/sagotodzzdrhrlrhljdwdnpnkoepf, common/dxk/urdcdhaaiskgeltwehiribuavsryu, common/ffa/cgfjmlwsxwirvdbdhoipniaiatucl, common/jus/gpqhtrdrhwfedsatsfnqjzxosfugo, common/tbo/mgnczgfvmjcytjmaufzepddbhzltf, common/fuh/oteukjpqomrmjycvltvmswdgrcapq, common/qln/tujxuubzatprjmioquaugjhougngc, common/udk/radufcqmpjzbrbobtzwnitouyasro, common/oqe/bahohsfgxxgbvravbquckybzddpph, common/cnj/bwjoizzpebhowqjojjnesnprrxpsx, common/jyl/knmfvmramzkighdyhpvbtgdwxwowm, common/mat/bvjwqpuoynoatvxdfdbjeuwquqprh, common/oyb/gknrtijswnmjpqigqujjgrbbvzrnd, common/nmh/nyprjzfkupnzwnjppfhmtxkofkzdl, common/sab/fgbsonyflxfhwadewmknhkztxnmpu, common/qpf/ilusuoxikuazoyoggnfeegdwcyvbs, common/upa/fcxvyfsrimldpvjswzvzxqsmylznz, common/yqa/dnwqcfiktkkfevvsfcjllavhprzlz, common/yjc/fxkeippyrupxomwczshakbbhjqcur, common/ruu/vdmxoyzxuyaqvgvqokktopdijtjuc, common/kak/njxuhhwbunridsgwgxepjpvrzakxl, common/mvm/aevztrwkhtydqwvktudvmguudbdra, common/rfd/qceqowsbjhysswctrpuhpyuzcbelh, common/asa/djcqrtierfduqecccqjxjadnidvzu, common/qrr/virdvublmmpbojrfpwpaebuapjypd, common/bzw/nnkmqmrumnqqxgqxmpvdwpxehliis, common/ldc/bjfqmexcczytcegsbepchdzvaochk, common/uri/ifzvxbpfspjhwlhkzxslknyziwqvy, common/jye/djpyswjobmfjshivsywcefzzzsaiz, common/wnw/vokiqwvxdcetsgxbqzsrvntqebgud, common/ekr/zftbgxngnwjoalenrjffwxufwzzys, common/yaz/tjamyzdiuoqcmyfxapygllivoqmbm, common/drv/tixsfkjbgemsbhrksrgwbhjwnxxbh, common/woj/upomruxjvnetastgwaeiodglmzqmj, common/tle/bzvdigrzjtlkvkpppljzhrofkyitu, common/sol/fftwgbqnjattqmjmlsnqqawvvbuyt, common/blh/ankaywxdrjnxxkbtofsnzpciirsot, common/owa/eayrhgyaqauatjilngletfajgrhqb, common/zwr/pyvekkwvwztsquyuvqtpsjzjspurc, common/ehh/gpslyfgbxkbavxpfkejaokolwgrad, common/qbd/qnzfsyiaizsxhxguofofaunlrtnke, common/rgn/vhjoycmgmnnlruoscesazukbluqmc, common/gcq/fwrvncsnjrutnoxjykmjzidtlblhr, common/bzc/ukuwyawqbqjatxcumxqlitagwlslk, common/ktr/vuafdrifcqhivlzyhknyvymiaqiwa, common/jom/dhiwyjhnndgbjjttohmkhlfkgnkzt, common/tuk/epaymiobgxofvcoolhnhlwixzzmpy, common/cgt/alvzvppziwydpgyhwgvxvyumaowte, common/sin/jhbpnkfgwriqhleilypbskcsmjtdw, common/jrd/dlfmfqsbreocfqwzpmrxnuiotphob, common/qob/vnoctwfuswpjrrecleimoemwfkkgp, common/zta/qtowafjpwvjopyrlvxdusjzkcgdej, common/ndh/wxpbqgawcobmcbvbrypyigpnykpwd, common/sjc/xabvwgzkoxmabwfjyhtndumyzoglf, common/yxd/wvtnikxnhdceydfwkfzuydmctnydj, common/eqc/hnzfnlrvpejsjdphuncliepzewqlz, common/wcy/afveqsaqycpsfqjzwdxiktgtixubm, common/tnm/cjukpajvcifrxossyvlqvmvccorvw, common/eri/wvszmalzznihwtukpflguxprtdfwy, common/aiq/noidgsakjiduvexxyirzaqywzssaj, common/csa/vzrbbzfbbindtrkbrleugppledmyi, common/scj/ftrosipntoaezhyjgtyldxxnymzin, common/ukk/rzbtnvuanvzciwxcxzhvsoyvkiurs, common/fwh/voxsyqxslioxjnntzmjvmrkayhujq, common/eqw/thxxfpuijjlhssaxregelggflgwkr, common/bqy/plcsbikkcyywrfmggitrwnxqyoqgb, common/vfb/gyuxmfjuwikkiuqulwopldqqanirz, common/kiu/fgizernetxbtbnlvlnzihncmykzis, common/mlb/bozdznfqtfntukpxdbgzkumzihsdi, common/htn/mqncuscorizaojifqgewarjismolx, common/lgx/dvpsyubwzvrpsbvcwzhgzrmkuoall, common/kbk/fezemlmihinzudirswwhhmeqlubeb, common/frc/azegsasujyrwwofqongkycstbwxvp, common/dsr/wbqwzmgwmlbvkojsaghlvlzxzyvtu, common/lqm/qsisjbxaisvtgitfuehqezveupssy, common/etn/paxihtfdiakxusuakaihroocwpahc, common/smj/oovzkmwdvodhyvcqngnigmnzusnbn, common/and/dbsddbizkdskbtfhwncwkcawnbhma, common/jpg/qxywuuplcjdpjojbdnwguehfzfgex, common/ivc/gttmtonbasmlllvnlbywovaqsunqf, common/lwg/mvrivfovbmxdpehaliugoltqdtarc, common/gzm/smohalynqzywjnachwqbeyzomavqh, common/poa/owmpfttfqbrktkhhpmifaxotjkhdb, common/mef/kovvqrdavrawyoyddefskvcafvhpt, common/sjv/efuqjlowauetcvbklkblxmkskapqn, common/brf/gligzgnbpmtzqelvyzewvrmsutkhr, common/rax/yiqhcmismkmlispmeqoaxryypjpmx, common/ypn/whbxbzfkgcrsfylsyacrjbbhtvqpu, common/ywg/bygzzrravopznnlijgligoknwrfrc, common/hqu/mmcgldpjncwjtaegywivttcblllpy, common/uig/uqhjefrbzkdobrhnrcckomuazachu, common/pxq/eckkzblhfqrxhjvunccrrcnabsqyk, common/dda/jymkjzwrfwigvewendfyaewoqxdol, common/dzh/kpidqhoanfomnjantpphsueyyhgtj, common/bqj/psorlzkzmrbcjdcszijehwwgjgwft, common/ikr/xghxqxtmxhcwwcylfmicuyobngxgf, common/adc/kruypuoxjnfwuzgladtaodtsqxqjz, common/cgh/zjqrpzsjotyikskgnpblujirblohw, common/jyx/qdxbpboxkfbbrifktffcottmlhcuu, common/gtw/ttqeodrldqhtxyystviiclaubgqmt, common/apn/enrlzzbqmcbvrzbrcifgwhhwfufqi, common/bny/tatdottegiqcckphrinbrejpjkrsp, common/src/oypkswzuytwajxmevurthovxxszsl, common/ukr/jezbwwklpaufmbozkucxcdrpfutbz, common/hja/yirfirixmcxdqbccdpkgecihjsjos, common/sdg/oqhkfxovubojcthgaosdleguhqptn, common/war/wcfcroctvknsfdcieagoepkjpesrn, common/gkt/zyuidbgvbdimvscocajehfbitqqll, common/lgk/txqangbwljdgwggxfkxnrvqbesnvt, common/wnz/bfpoiazpqcymqaacdqxnkcaoqxpyl, common/wha/biiiqcitxhmadbwwcvodwetksdpbj, common/ovv/wtysowxumkccyimgummpiuzhjzwpx, common/aqm/dhlpknqqlagcffufiqzvnbpfchoar, common/xxz/qdgojxncbbtwhjlktnhlejgozdlcg, common/wwx/tglxpbdqdxubzhrkkqftdhdzlbvpn, common/fbi/delrtqvegmotyljhbusbvqwunnobq, common/zcv/lvbxlaenpqdfqedbdfnanitunrgsc, common/wbc/mofcalupxsylabnzgflhozmlnfmjr, common/psc/zkglbommdkusoorpogwukzmjmriit, common/qye/ysmmjjhaulvjwmbtlxwqndcmpphce, common/mhl/anqzgrfrxnpjvfmvmlugjnsdieakm, common/may/kqbvrkzyjatfjuoaqgabxoqwwejan, common/kir/jxgzoivsgzcxbrtiayasbmplqkqwz, common/edd/qlafazcvohpydbpsadgzeoqoxhkpv, common/qlt/bguwulqpxkdmkguyoceagrhqkrrtm, common/qmq/gvcfozmnyjvzmsbgdofssllvtaleq, common/vtv/rennntnuxorpsrcdvvcetfmirisdm, common/jpr/yicuekvfdauzdtuahlbscipwqasvb, common/kdn/dqbgtpeullwdsuwdmchygncsjxjzq, common/jpe/yjgxfcxrjwzldxwskcptzwgpmjmvx, common/kfb/riiimldqjhwdadflrpkenisgflubb, common/qbq/ydiuomwjhxiokxfnqpxgosgrugurz, common/tvh/fzqubnncstabhhsqlguyjzfhazdrn, common/lca/dfaseehheneasrltvgffjmypxsujt, common/ybr/ftnxgbmhwdvdlwzfujqjjsmdddcep, common/hmw/posnxoyxjhmvtterxyudztvevsdpc, common/efl/irdcpdzwayoikcnpbktaumrsoixzo, common/rwr/xsljdvjbsnkfjpydnreuzgbeivjrz, common/dec/gdugxkkegvbbztedqnzvdjrhagfde, common/iqi/ehrdgkkgdzbzleyyvunuvwmvlvulr, common/hgk/aeggrhnwvdmencfbrgwedulhcpjdf, common/szq/tjrhrwklmqfbnrpmirjkbtlklbzum, common/pnw/yedeovdxfkwywuevcijhmnxlperfy, common/rmu/sqessqixpagqxoyrzybnubjbvngyz, common/kxg/bmiogxnwwabkughrdsbithmvpnejt, common/xnx/sjwinenjherhktrgdrxnvhypzxgbs, common/qkm/yrrwnqsqayllhgkvkeqykfupwntwe, common/kjk/gtxrvpzukmcchzwbxsjjjqxeurqtb, common/rwd/tvfhzvwufstoolmjlbiaskycpoovl, common/wch/xjoabnyhitgwdijckoinbjxrttpyi, common/lsx/qvtbylkhehgbansdcyuospkttwcgq, common/pmm/acvbqchkcrwycwgqkccehrzuvgxwl, common/vhv/lxabhikaswibmpmvinzhvgnopyvsz, common/khz/lyzndlxvszdwgxyhqokmfwkujxawk, common/gnx/nxpfjknprbiegyyogodpxmvokquqa, common/uym/nfzsyitiunklnijfwlrzxhsgorfjo, common/ipk/nvyagjajnkefoqhyszmeydccbkchh, common/bzh/agxviqwyxtcapbbkryzlbqaqyutuf, common/vsl/tdfoxiiygbodkmdcxowrfzuyolzlq, common/vkw/eiqkuhhadqebjzcvteejcntsaalxm, common/cqn/qfohwaguidsdnmgpmuxtnysicsoic, common/eew/vcdcqsxlvasavsntnqoxkjyyoorxb, common/wfl/qaesjmocdjfpyenblyobtxhbrbcie, common/uox/amvjihyivhwxpphxmuexmizmcfoee, common/ixw/atraaoqxkimlgtnxnfanrjfbxyxqs, common/ltv/vzanqnhawfqnbqlazkwqqnxlaoawo, common/bga/nsbjmdhjtkmyjjiaweeelopetebpx, common/bfl/tqinjtosynymnlcwkgsqcexppztvu, common/vem/zoqrgqrclijilyytajmvdhlhhxkwr, common/mdh/wdqeqfnziboooybplssjfjcahmmak, common/uhk/xsflzpyyirbzitaccaapwcgqxjbwc, common/zzd/yohqhtznqtwoffocramxzmatmfvnj, common/xst/cfqotyszmjvkwrommlwkffklwmcoq, common/gmk/yazpvwzpgrcvrhypesnufgazyhaib, common/bpm/weiwcgbjtcpuqeflevsulmxpzzjda, common/vfr/unnhbpmtfojduvzpmumantmxsfhoe, common/zpg/mhpajtbwymejbxzwkqtqninnfmqbv, common/ypd/cvkcqbmvsiwvnuwayqbldphulcudi, common/anl/ezrucfqdjibucwtkeecvhjvthobtb, common/ktm/ojvdiinvuprwxbbmdxeidjfiktaqe, common/ytq/hzajnsipxaegideooqphfzjyqxdgs, common/ivs/fwfccddvqylpelyvvtpinxcrcrezl, common/fmu/rkxpegyecuzwpsfqlqpoxefscygaa, common/qwi/pkkfyfswisnweexfdcmvclgjabuxd, common/enc/cmqrpcfvsdkuacbzcnjeztpsjyuio, common/yhh/rvqamiwdcnibrodfrsrfrvolyjzea, common/xmy/hrbkpfxofnyaiykenfmsgqphnibrx, common/yqx/dtupgxncdhjlezlvjzkzchxmlseyi, common/nhi/sjvowanrzvbdzvobjzoqyxwktyyhd, common/okj/mzpkchgrjoywbotucbxsbbcbzrmkn, common/hud/qtbruqixskzdokafrxpcznvtqoftb, common/rha/hkakmcnojznkpwzarjogdtitpwotd, common/zyh/euboukitwkgkpuzjubzbcrvdrjqvq, common/etf/yjcxsburkhdgjutiyioixdjypqcjm, common/cnf/litrigfaumwrzvsyomyfwknauqxqs, common/wws/bujtebuepmblgzejvwfgwldierruc, common/nql/wlxgvsocnhkwuqholwfsrpfcgvsly, common/fyk/lbjtdwzzojxvtdmjyudvgiemdeuda, common/hrb/skxudathlaigioovjhgamlqsbwiew, common/nzg/iamcafcyaqnmzwmqpaqdzepmxxkoh, common/wnt/lwnqdaexlniwmctwpdfiibwcwobkw, common/jxo/pogyemwsbycbzdumkesyoyihsqgvr, common/oqp/zkdmkkgzzoquzczpbxygranyuhlzy, common/iad/bqohwnjhezwjlawiuaocllgldpejt, common/irm/qamvlbpqavzfzrflecdxkekfcvtkn, common/qrd/mqkibtomvcijozyflknfpabkbhwek, common/ysn/apvcxeilbzwbkwmixdkzlfhpcetto, common/rlf/ecqvntnbpturljtbeizjzzihbncmg, common/bdb/olmnnzdilileelyczpdfciobdlrmv, common/ekj/iufujifwkpxqonconwcegnwuektne, common/pig/kaihwsnlxplimivehugocxjvqniim, common/nhb/hzatqcrtnmlkmzgtyjeariihwomqj, common/lgb/aahdhludmwtqhetjsqqjfhzqwxard, common/vyc/wmudfgsnrcseypjkxxusojngeiuis, common/ttk/ohzkhuqogvncjqufbnngthtnyekmm, common/zqt/tzmxksroxvklcyrqyrfxmifpxanee, common/bgt/fpfksnddrqiyjdrnuaxlmfaokwkre, common/ztw/hwdtyobqfvcfvldkrlwsitxayemmb, common/dqg/kljcxnrgeyqcxpnhjdbwrsumuuexk, common/yny/gnlhtgotqvgzwayxjhiyxymahflqb, common/dpa/aawkwzawgxpvaqzocefnunuqfcehh, common/fve/wmnbqlprjibxrqbapmgsmhrpaduab, common/tom/dmgvtwbuefqpkrxqlclbgrxiltpdd, common/kkn/qelbmyjvtvyhqhnksjdimlsnhffle, common/plr/etywfmzeutinlxgsiewwtohhulple, common/qtl/mkprwhrymcgiveootferfnclpscef, common/cae/qyniiovqlnqpyvowahsmeygtoltna, common/arl/snwvqoddldiuuxqlxisedgaejonxk, common/gbu/ddtzdxwwcuyrcgoqzgnvmmvzcesad, common/ava/pthotawjrduloaocwwakcdceydzsj, common/hqz/bzbuofyuqtbcuwhbtgmovnqltnezm, common/mtm/itnplbakwmrgafzmaluuhevsbmrww, common/gao/bekyxwjdkmleowtmuwntnskftatyw, common/mtv/zrzsxnxhqgkmccsubnjxidjgcjxwf, common/svq/ghuxexxxqdnziazvqskwiuqfdensc, common/lwv/bammojagfkutvecgcclzbqfuibacy, common/yim/oscokjlnwajwdipmjboipzyozpwix, common/wal/wnoccsdxdaerjntygaitqykvjphkt, common/bys/ucbcjxncnhityvnlbrusoxqckkked, common/xkh/sqrqmenwntgtgrlcxthtxcvhzshfl, common/kmf/upxinyfczhetaxquwnjviewljepux, common/iwc/gcgdixfmpdbbawsvwuajwuvmwgjfr, common/vks/xbkummmecxrzgtvryofmkifwyohdq, common/akp/nhsogapwutummbjqyubwztpmmdrxl, common/fzj/cxnmvkholjokfbcipjgsdofmtnbja, common/dcf/wsakgfizxsivgkyxdcekemjzrfkmn, common/enl/syslygpzefhcpwuorcrfljdvmvoln, common/tlu/magfxylmlludkqfcbjzhjdjxncnmg, common/qiw/fndfopklxjutiykzsjvwakzisxlcq, common/jrm/tjlntagifdygmhpskicdywczdcgsc, common/jxp/ekrbdoyqpbdvrpiwltmonnkohfpqb, common/zmn/flsduglboubudbsqnqjsydidashrt, common/zee/jnipobnmczyhbenusaefqqmhuyxlx, common/efm/rnexybshuklnffrzxairtfrwtadif, common/cog/rayftbocnvhwdxohbodqxzvxloskh, common/wkp/zimjlohtxrkpgpsoubpkplgjnkndy, common/qhz/mvgluqbmvyplixzazjwjqkvaaxcya, common/wbt/dtttkreesowkiwitpjqurehcmolwa, common/hso/yahegkascxusjzhvihivvjorqcfkf, common/fqd/pomcijlwqluvxjwtfdoletcbymlpl, common/chw/gxwtbxqdkurgcdxykaghsmsvlwazd, common/opw/yrqzxxcetrcwyparmpnkrwagksyzn, common/byb/reeomwgtazplhlaiprtwdivhlkzke] were removed from S3 2024.10.22 01:35:51.726268 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_952_952_0 to 202410_965_965_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:51.729599 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:51.735464 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:35:51.727923 [ 1418 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CounterClass.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/MobilePhoneModel.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SearchEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CounterClass.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/MobilePhoneModel.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsMobile.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/MobilePhone.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/MobilePhone.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/IsMobile.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SearchEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WithHash.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/WithHash.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_5_5_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 01:35:51.825018 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:51.900927 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62877 2024.10.22 01:35:51.940000 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:35:51.985129 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_540_540_0 (state Temporary) 2024.10.22 01:35:52.135297 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5125c1626e6dc1d916f6169d08f0f217) : [0, 235560]: Setting download as failed: File segment: [0, 235560], key: 5125c1626e6dc1d916f6169d08f0f217, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:52.136854 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5125c1626e6dc1d916f6169d08f0f217): No space left in cache to reserve 235561 bytes, will continue without cache download 2024.10.22 01:35:52.153241 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5125c1626e6dc1d916f6169d08f0f217, offset: 0, size: 0 2024.10.22 01:35:52.155123 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5125c1626e6dc1d916f6169d08f0f217 for removal 2024.10.22 01:35:52.157914 [ 1476 ] {} CacheMetadata: Key 5125c1626e6dc1d916f6169d08f0f217 is removed from metadata 2024.10.22 01:35:52.183835 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_540_540_0 to 202410_547_547_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:52.212937 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_553_553_0 (state Temporary) 2024.10.22 01:35:52.257393 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 849988 2024.10.22 01:35:52.312945 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_553_553_0 to 202410_559_559_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:52.462140 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 64410 2024.10.22 01:35:52.466875 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4537 2024.10.22 01:35:52.886915 [ 1193 ] {} DatabaseCatalog: Execution took 227 ms. 2024.10.22 01:35:53.629099 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(930f3240d05e306a6295ea1797e7ab46) : [0, 1077817]: Setting download as failed: File segment: [0, 1077817], key: 930f3240d05e306a6295ea1797e7ab46, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:53.630647 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(930f3240d05e306a6295ea1797e7ab46): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:35:53.654786 [ 1234 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 260 ms. 2024.10.22 01:35:53.769082 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 8693 2024.10.22 01:35:53.897178 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 930f3240d05e306a6295ea1797e7ab46, offset: 0, size: 0 2024.10.22 01:35:53.899734 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 930f3240d05e306a6295ea1797e7ab46 for removal 2024.10.22 01:35:53.904059 [ 1476 ] {} CacheMetadata: Key 930f3240d05e306a6295ea1797e7ab46 is removed from metadata 2024.10.22 01:35:53.965058 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:53.968855 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:35:54.111562 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_564_564_0 (state Temporary) 2024.10.22 01:35:54.159328 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_564_564_0 to 202410_577_577_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:54.262791 [ 1199 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:35:54.312387 [ 1199 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:35:54.359211 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8693 2024.10.22 01:35:54.807564 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f13fe9303d4f5aeeca9497fb255740c2) : [0, 1964476]: Setting download as failed: File segment: [0, 1964476], key: f13fe9303d4f5aeeca9497fb255740c2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:35:54.810231 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f13fe9303d4f5aeeca9497fb255740c2): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:35:55.069572 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f13fe9303d4f5aeeca9497fb255740c2, offset: 0, size: 0 2024.10.22 01:35:55.071355 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f13fe9303d4f5aeeca9497fb255740c2 for removal 2024.10.22 01:35:55.075847 [ 1476 ] {} CacheMetadata: Key f13fe9303d4f5aeeca9497fb255740c2 is removed from metadata 2024.10.22 01:35:55.318550 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_483_97, 202410_484_484_0] 2024.10.22 01:35:55.320777 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_483_97, 202410_484_484_0] 2024.10.22 01:35:55.337342 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_483_97, 202410_484_484_0] 2024.10.22 01:35:55.607913 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 10.71 MiB, will set to 1.75 GiB (RSS), difference: 8.94 MiB 2024.10.22 01:35:55.923047 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 62926 2024.10.22 01:35:55.959340 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_859_437, 202410_860_860_0] 2024.10.22 01:35:55.961530 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_859_437, 202410_860_860_0] 2024.10.22 01:35:55.969482 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_859_437, 202410_860_860_0] 2024.10.22 01:35:55.999535 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(2025a2164ba53a82c7324e2a93262791) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2025a2164ba53a82c7324e2a93262791, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:35:56.006874 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 469412, file segment info: File segment: [0, 33554431], key: 2025a2164ba53a82c7324e2a93262791, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:35:56.009834 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:35:56.053613 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:56.056233 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:35:56.075520 [ 1702 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/luq/tsfkenobwaznbjfmnjjoggvifgdwx, size 469412 2024.10.22 01:35:56.244620 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 474 2024.10.22 01:35:56.282019 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_953_953_0 (state Temporary) 2024.10.22 01:35:56.483989 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:35:56.485799 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:35:56.666064 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_456_456_0 (state Temporary) 2024.10.22 01:35:56.672323 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_953_953_0 to 202410_966_966_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:56.683038 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 7 parts from 202410_1_960_479 to 202410_966_966_0 2024.10.22 01:35:56.699942 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.40 MiB using storage policy from min volume index 0 2024.10.22 01:35:56.710681 [ 1216 ] {} DiskLocal: Reserved 2.40 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:35:56.743263 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} MergeTask::PrepareStage: Merging 7 parts: from 202410_1_960_479 to 202410_966_966_0 into Compact with storage Full 2024.10.22 01:35:56.774218 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column Params 2024.10.22 01:35:56.789326 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:35:56.808146 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_960_479, total 63394 rows starting from the beginning of the part 2024.10.22 01:35:56.815879 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62926 2024.10.22 01:35:56.826545 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column Params 2024.10.22 01:35:56.859530 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column Params 2024.10.22 01:35:56.865558 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_456_456_0 to 202410_466_466_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:35:56.876192 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} MergeTreeSequentialSource: Reading 2 marks from part 202410_961_961_0, total 60 rows starting from the beginning of the part 2024.10.22 01:35:56.880413 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column Params 2024.10.22 01:35:56.907154 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column Params 2024.10.22 01:35:56.949500 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column Params 2024.10.22 01:35:56.963445 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 474 2024.10.22 01:35:56.970764 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_413_85, 202410_414_414_0, 202410_415_415_0, 202410_416_416_0, 202410_417_417_0, 202410_418_418_0] 2024.10.22 01:35:56.973728 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_413_85, 202410_414_414_0, 202410_415_415_0, 202410_416_416_0, 202410_417_417_0, 202410_418_418_0] 2024.10.22 01:35:56.988017 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column Params 2024.10.22 01:35:57.010970 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_413_85, 202410_414_414_0, 202410_415_415_0, 202410_416_416_0, 202410_417_417_0, 202410_418_418_0] 2024.10.22 01:35:57.050685 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column Params 2024.10.22 01:35:57.110320 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_461_496_7, 202410_497_497_0, 202410_498_498_0, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0] 2024.10.22 01:35:57.108982 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lsq/bzmkkcycmhsmnsnnzehapptpogapa (LogSeriesLimiter: on interval from 2024-10-22 01:35:47 to 2024-10-22 01:35:57 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:35:57.114117 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_461_496_7, 202410_497_497_0, 202410_498_498_0, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0] 2024.10.22 01:35:57.207296 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_461_496_7, 202410_497_497_0, 202410_498_498_0, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0] 2024.10.22 01:35:57.612163 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 7.52 MiB, will set to 1.75 GiB (RSS), difference: -2.95 MiB 2024.10.22 01:35:57.756644 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} MergeTreeSequentialSource: Reading 2 marks from part 202410_962_962_0, total 62 rows starting from the beginning of the part 2024.10.22 01:35:57.888683 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} MergeTreeSequentialSource: Reading 2 marks from part 202410_963_963_0, total 77 rows starting from the beginning of the part 2024.10.22 01:35:58.074892 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} MergeTreeSequentialSource: Reading 2 marks from part 202410_964_964_0, total 63 rows starting from the beginning of the part 2024.10.22 01:35:58.216414 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} MergeTreeSequentialSource: Reading 2 marks from part 202410_965_965_0, total 52 rows starting from the beginning of the part 2024.10.22 01:35:58.256354 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} MergeTreeSequentialSource: Reading 2 marks from part 202410_966_966_0, total 49 rows starting from the beginning of the part 2024.10.22 01:35:59.261803 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1311 entries to flush up to offset 851299 2024.10.22 01:35:59.272784 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_861_861_0, 202410_862_862_0, 202410_863_863_0, 202410_864_864_0, 202410_865_865_0, 202410_866_866_0] 2024.10.22 01:35:59.276661 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_861_861_0, 202410_862_862_0, 202410_863_863_0, 202410_864_864_0, 202410_865_865_0, 202410_866_866_0] 2024.10.22 01:35:59.351662 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 18.59 MiB, will set to 1.74 GiB (RSS), difference: 2.13 MiB 2024.10.22 01:35:59.435090 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_861_861_0, 202410_862_862_0, 202410_863_863_0, 202410_864_864_0, 202410_865_865_0, 202410_866_866_0] 2024.10.22 01:36:00.082751 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 144 entries to flush up to offset 64554 2024.10.22 01:36:00.107639 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 4579 2024.10.22 01:36:00.388967 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:00.393697 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.96 GiB. 2024.10.22 01:36:00.397393 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:00.411569 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:36:00.412221 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:36:00.459695 [ 1246 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 301 ms. 2024.10.22 01:36:00.463534 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:36:00.686460 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_554_554_0 (state Temporary) 2024.10.22 01:36:00.729948 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_541_541_0 (state Temporary) 2024.10.22 01:36:00.822263 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 62989 2024.10.22 01:36:00.857009 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_554_554_0 to 202410_560_560_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:00.909658 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 64554 2024.10.22 01:36:00.944966 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_541_541_0 to 202410_548_548_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:00.985033 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:00.986125 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 851299 2024.10.22 01:36:00.999426 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.96 GiB. 2024.10.22 01:36:01.194367 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_966_480/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:36:00.975515 [ 1222 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_538_107 to 202410_539_544_1 2024.10.22 01:36:01.218981 [ 1222 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:01.222927 [ 1222 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:36:01.229283 [ 1222 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 265 ms. 2024.10.22 01:36:01.262907 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_954_954_0 (state Temporary) 2024.10.22 01:36:01.269356 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_544_108} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_538_107 to 202410_539_544_1 into Wide with storage Full 2024.10.22 01:36:01.326831 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_544_108} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:36:01.338519 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_544_108} MergeTreeSequentialSource: Reading 105 marks from part 202410_1_538_107, total 850690 rows starting from the beginning of the part 2024.10.22 01:36:01.397830 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_544_108} MergeTreeSequentialSource: Reading 3 marks from part 202410_539_544_1, total 9194 rows starting from the beginning of the part 2024.10.22 01:36:01.414961 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_954_954_0 to 202410_967_967_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:01.487829 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 62989 2024.10.22 01:36:01.667541 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d2046e449a87febcad3bad7c7ac512e5) : [0, 241003]: Setting download as failed: File segment: [0, 241003], key: d2046e449a87febcad3bad7c7ac512e5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:36:01.668395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d2046e449a87febcad3bad7c7ac512e5): No space left in cache to reserve 241004 bytes, will continue without cache download 2024.10.22 01:36:01.692832 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.97 GiB, free memory in arenas 16.91 MiB, will set to 1.75 GiB (RSS), difference: -8.29 MiB 2024.10.22 01:36:01.701146 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d2046e449a87febcad3bad7c7ac512e5, offset: 0, size: 0 2024.10.22 01:36:01.703133 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d2046e449a87febcad3bad7c7ac512e5 for removal 2024.10.22 01:36:01.710108 [ 1476 ] {} CacheMetadata: Key d2046e449a87febcad3bad7c7ac512e5 is removed from metadata 2024.10.22 01:36:01.861488 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 8699 2024.10.22 01:36:01.977526 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:01.980586 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:36:02.142285 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_565_565_0 (state Temporary) 2024.10.22 01:36:02.270748 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_544_108/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:36:02.382215 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} MergeTask::MergeProjectionsStage: Merge sorted 63757 rows, containing 13 columns (13 merged, 0 gathered) in 5.645151323 sec., 11294.117084202031 rows/sec., 1.84 MiB/sec. 2024.10.22 01:36:02.382685 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_565_565_0 to 202410_578_578_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:02.549143 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8699 2024.10.22 01:36:02.572595 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} MergedBlockOutputStream: filled checksums 202410_1_966_480 (state Temporary) 2024.10.22 01:36:02.870189 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_966_480 to 202410_1_966_480 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:02.876931 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_966_480} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 7 parts: [202410_1_960_479, 202410_966_966_0] -> 202410_1_966_480 2024.10.22 01:36:02.941715 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.26 MiB. 2024.10.22 01:36:03.032782 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(19f4f527ede8afb80441b000baeb05a8) : [0, 422950]: Setting download as failed: File segment: [0, 422950], key: 19f4f527ede8afb80441b000baeb05a8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:36:03.035086 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(19f4f527ede8afb80441b000baeb05a8): No space left in cache to reserve 422951 bytes, will continue without cache download 2024.10.22 01:36:03.112538 [ 1418 ] {} S3ObjectStorage: Objects with paths [common/ssl/rbeeqilcumvxphczjshcixovfsila, common/udx/lygsbnejtvsimffzmnamsvxshpyez, common/keh/eozaaadrscmbreqjxrivhextqxwqo, common/qxq/mqbpzucesdgjpfgfghnbpwznpfwgt, common/elb/vygnrhedytpnuchqspkwsqcodovet, common/giq/wjirvfrfwkmxtofilhtvhnlizfiua, common/xrp/eqvrtyetflizyaicqwplmzwtrespd, common/wrq/wmwdmanhneakauqijdqduxxioaxqz, common/sdk/peeylqalblxofqfrnbckbaowoeulq, common/wnz/gbhyvbucmevldggeycjgrfhokfkbp, common/jls/eoectzvscosydeeltgeiqsfxrhhcv, common/nfv/tlvlzqwhwtzgtrjirtzskotraykyb, common/wpl/cqxdmiiyngehqwdztckqljoehuizu, common/hqu/waktabfyxmddogyiarsgeklclwjil, common/sno/ytjupwxxcasgtzoyhduecepeueipv, common/sij/vtfsvqwdbjegufbqzrkntlkjjegqd, common/mle/hizqlwotrtxupkdllxkogudoopidy, common/mes/hjvkdjzzjynudsmhivcrlsboiykhu, common/mvl/kclhmflabnklkifeisghpgrervrvw, common/tqh/nfmyjoruzldkqtkttxamgthwrdvqv, common/mei/hlwotmxertztubzpzxtehdrigqeef, common/bxh/nuomrpztxsvkrddbjqeeqezegfkai, common/lca/indqbapbpxgizdlenyqopdfzdwklj, common/brn/vwugrscvxhntgkugfadjbifjtusfx, common/dko/gbmtgemehvronbonvlsbjpvfcghvf, common/yuc/fxyghebkvqnhibskvjuxssejiebzu, common/wlq/gadwusudkacgvikyxllrqqmowowgv, common/woe/vfodoqeetdptxdntevqflqeujvmad, common/tcb/yiajvbevmdxobfvlqbkbnuuyflfop, common/qiw/bbukicuwftgvnjxmaufkqsbojwkbh, common/hya/rxtrqimkbzkftmfgcifqokazhyelq, common/czu/fytqmsriiycgtiyuryzulwtfqbdbw, common/bzm/nvgkefxjhdnbgpaazonyunxcaaswb, common/pzd/bqpgjrcdnygmeriskotwqkxuhhbxn, common/xuq/lrxdnqaomldazweyzspkjiybvjdbm, common/bpv/kgbjopgylfxsxsqyzwmchfdzlbpra, common/izk/oukdtanfdlogphtphreufydvyinsl, common/kjy/scjjmmyxygibcfbaeauavmpclwzwx, common/xjx/wcaylkmrnoteddxpigfcscyzjefqy, common/ild/xvnmboisqdgbnkhepqdwnmldlvqod, common/xya/zghcybqgljemihqhhzqxkfcwjnfyj, common/otw/kzxbhxuxizoglcvsbzzjdgcyofrgq, common/frw/mzcfxeltqqeaqudwieyxntgorztkl, common/cfa/ckksplcanifzunusdjsxfybzmjozv, common/klv/ksbjdmxdqqtisguhdfiqntwhciflc, common/mag/vhfbsiuackrxgdmfquwlrbpqnxhor, common/iev/bekdoarjnryvrcqsubidrwzktxbol, common/qsu/knetizsvawtmciukmblwqvyxzxgmu, common/uyf/wetexugxauozrtgttbeyltyueapsp, common/rtk/tkthgagdzejvdtuzzvdakflrclopy, common/fju/oqxbjbnhlqonlabyevezmeezszcjy, common/kbg/yvqetqsxficmubagpgmzvmxnjmurs, common/dwr/ihlxfpipautfsdhtivxngqjhnrljy, common/nyi/yythrfcxnquyowyijpxovxicqgiub, common/tny/sasisgybrwcaqzvalnjnkmzpurtex, common/wtl/fsccotnkkttgupfzkysgkbudtybnq, common/ddm/zjzvbrwwvcrapezndolsbdsgykexb, common/xxv/ldboffqhcnomchrnpdxhuepcsryjw, common/igj/qxlpmpoibbmwllrhvtkyaqgzexrjs, common/png/htpxrwqwzdjvpcqbinfxwovbcoiwo, common/aqh/qmbdadgtssxglrjadnqrgmaefujtl, common/aim/llbdsckkbocuiuldrlgtmbgtnpoay, common/ozl/upvpdezterxgevrxjbxaalgiobvvk, common/whp/gpojkhlietkxlnpadmpvwlczbtryj, common/yno/lodiqedelefjzjipwgqdoaqicuold, common/hdr/ujehrckdnrawenwxozhffbsvibsgc, common/tcr/qcsliwxighfvhxqtzrbuwzqelwlgp, common/fik/ajncynjtrvlupjtmreqagxehcrsqz, common/nxo/orijtbopfdzvdocjkorsbsroyorvp, common/afn/qwqzktckexpociadbgtfwcgagocmo, common/xqh/mvtxknimqmemamdyfinfrvcihkiht, common/fct/npdfffwlsjuvcgwxyhvdrenrngrgq, common/vzi/hwwxxwegfamoskiggmygrrqieyapk, common/duu/eukbkdmgkqpiaetawbdzoxrxeniqr, common/tbw/jzmxukvihctwgvgzkesxuqmsiqrkr, common/mlv/qaxvtyyppihjzhquowddbhaifhahr, common/uab/sqedzzmbzfpmndwpjnzcdzehfcsuj, common/kjo/bbvvogwtezvnkwnoszlpnptjykfof, common/yrn/bwfbyujhopowkzhdngixbplklrjqn, common/bfq/zynblfbdreqbbjwegzpryojummamh, common/caq/ougotiebygdpkpmylhhmszqdmjyna, common/zba/tawhxerwwoohaakbbsaobgrviirua, common/tkn/dwofganaahfudrszzimfiqxgohhvj, common/nwc/xwjrnbckqhacjhdehppadtbzqsjsx, common/fun/eqejmokbtfzezkjdozpuflkpgmdli, common/ksi/bouzrcxedtqsdxtcngmydghlnfgap, common/dfn/bcwvbehdgourjegddckdlqcgcakff, common/skf/hjxpvolnsfiwnxgjekosqfnsbvouo, common/pbe/bqziebjogqoyvqigldsrbwlyeorpx, common/ffb/nedtlqvppsdvxoushylqsdsaehjoy, common/zjd/oqjxqnyvlqtjcyumdtyxxuhhaskxy, common/zzs/ilkswhiyklabnuzowwjseurbhwkhz, common/zyy/wwwcfrworaonahaeuvwjsmowdrvlc, common/ece/ysbehdohvhbkyjuafxjmyeypmscrn, common/fwr/olbrulqhqdbhztyyqydbffzlhekle, common/hvj/ypfnpebbrgyawvhbpnjyzzitazkkm, common/nqf/ildcppnrhhwomnkgbveenihvndcxh, common/rua/doowgmhcmnsygcotfdcfqrbtvmduf, common/mur/eqqwvfpeoraphjlflwutkjjdszonc, common/njo/mzgrcyjmxtvywquyttbkddgypmivq, common/yzd/ppdjeqjzapoudxthgytgrhsfdkvah, common/ozm/jimsacyypppuzocqukpvlzrzegagr, common/wei/sqdnpstkecvivmnbwdyldhswglufb, common/ody/fbgqtlogppttlbldsbtsgblauxlgx, common/drj/nrmnpnbutzssvtyblkvhsvnmehtsk, common/mry/qydbochujtfxpopmutldryayplcjz, common/fgq/gcirjyomhqdottgtauyuzlawfxsuf, common/hge/fcbaoxikshqtplvebmtsfxrodqpir, common/fwv/htpjdtptpycxuhkkeypogjiaesgzj, common/icy/ykicaoxovzmzkprkabarzlzkieapa, common/zoj/shitlpxaczciscosknnldxohzdtfo, common/kji/qephxzqukyvqmdhmrnbkyzhkormwq, common/nwz/udglywwmcfebsnxxeeadeebuttwax, common/pjl/ahknywyyuooopvjdytvqfydsopyng, common/pcm/auxweohnoxzfqeyfuqgvbixjewqty, common/fzy/ebiqxqssqzwfphnuiezzdklxkvwuy, common/nrm/cruhojdikeibbbzgzdofmmvuinngl, common/ijb/vikfdiuhgolfdlgdqrkkskgsxmvbj, common/mdh/xontpryqrmiwmerbsezzkcthhdmub, common/kvw/ldmsgfagqlufdxqjadkskebxmqnzj, common/hyc/dcjdydadrbrijscwkwcpvlnnypmgp, common/wfc/fvsovfhlxusytkoqvzngrqryrxntx, common/sxk/kmnikskndkzwbmkokisefjnpkheah, common/wpr/llvscrbomoxkttdiolthtmwlwisis, common/tlq/ksusoacpmlhsmmwjlzobpesjjzxpo, common/shd/qpnmobpdepruodvojpxcsdudiefyb, common/jcw/ursswjxvtfdqazetgwtndwklmcutx, common/qqj/hxxxntagollxfkrwqdptgbudwsssb, common/mdv/nycxonszarojvtfozmwytbnjvhydy, common/hse/qlirmltevufejebwccqkndwvpkjze, common/gvq/epmnhjjukaramovoxuniwbvnoeiud, common/yxc/czqvqkggwztgjxxjbomozcqbpusgo, common/fcx/pzkliigabfovdrwselmqtufuxsmwo, common/sjq/nxhtrlaydwavynvxfldhtifgfqunb, common/jjm/ttxfcwzanwmxsfzoijhhijfzkddei, common/zgd/arcjirihbxcrlvugofqurcfegqiev, common/qug/zscbxviqafzbvcspyzcfiiytwkzlo, common/mqj/ebusavbyowkwqdbhyuyhrvdcugvjb, common/sel/vcziyyulhtwcahlvnayjrqchoouwx, common/uzg/iqddhamkqwizinabdhacsokkqleks, common/vjr/wuqsyluqlyueqqiihxpvhvvgylwqj, common/bpj/eiwhxrtdpjgljsyvbqpzlgorxppav, common/hwv/rkmexctpqkskhuibdpispewbwgfpj, common/ptw/qnvrbdqnxoqszminplksofnsmrhwi, common/zvu/qdwjibzqjobnedngchivzlkmtnqnx, common/kkn/kodgagxezodihkffnplhuzvgtyxhc, common/ndx/hxmgpvsjiytstigjsmesolvsttsiw, common/ass/nibpfjleghqjeoxjswbuenrrpouqi, common/jqu/pejgkynrybjvljlpazrtkezorkkjr, common/nsr/harskyydsnuikvyzazhcwmyuijsgr, common/bix/wyfwemhklhsgizbxckitxqojaeivv, common/jty/palupkauyecttxehlxuhknuxbguiy, common/kbr/kjxdrvnmhhwfgoixqsigupgwpymzb, common/xic/bfbxxlkuwzrbjakmgdjelpdaibpey, common/djh/kzdiflvpbwdyszhxjsjybuwdqstup, common/dyx/ybqgmjzhjszpcloqaxuopcveynfph, common/dzc/ldnohrpzytwlfieocigspxzafimcr, common/ynh/jpuzjyfxptligmylnzsfnlcfzqatc, common/nnq/iveqbxzmxlfuskzcfvwxxlmhbtzzo, common/wtf/tonhhnrzwwohpguzspkxvtxwoktsx, common/bni/basmcxmqoejatixggmutviuqrihha, common/wbh/dxyuqgxfqqfukiqymbwyvuzersfvr, common/rbg/xcliqieuxedbukttxmiwvwjlpscgb, common/lzx/vjhttpunbcnhnfpdovkmqbdgqrozd, common/xbb/ygahwdrffawapukpxthlubzecgjdv, common/iqy/fnoqicivaoyvjvadxizrtlfkqpwsx, common/cgh/lusrbzevogtyybmvairyhsezmojat, common/xwc/gepvbbyrsmcrhtgdspoztxmqmijtx, common/vvn/bzixiiygjldzjpesweydnbtsozksk, common/sxz/cyfovsnbhoseniudfsgacmgmdteff, common/zxs/zqepdfscqpqdorfgvtdftjjftkerb, common/bbc/osuclzrnnlfcyjfnbkhohytmlciyl, common/whc/byfiyqvtxxeieyfzwmkoghxyllcax, common/utm/ibawvcxhygxazdbzoyijfphkylzia, common/lnf/fxmfsjmowdfwrndsqbvpxmhloxeoh, common/uyj/xannvgbchnrxirlkgdnogcoayensh, common/nxo/whinjnfwubzghbmwjfntoxtfnogee, common/rmz/rlbvjtklfqxwkmceozdfbypyrmuff, common/ywu/tsasxkopjwwcltfzgzfxhhwazcplw, common/xao/wscqtrpfhikozfpwdxefmsoiwlyrh, common/qul/pempasjuqoihhpstvrhrhiyyyypwn, common/jnt/znsdpbytpvzhsnuuxsloynzawjfds, common/fcp/rrsujasshglrmlponmjfdvdxcsnrd, common/uem/dephgfnvcjsbdsapvlyiijrckwxof, common/pph/zgbboxjwdjsdrbxgpwdpxsjrnubre, common/uox/wbcsuienylgvzseadsusnfgswrmiz, common/muw/lwjsklzhnxtxzriydnpetkousrkdq, common/mkz/wbeswpltrqvewjmobfrfvxgqafemp, common/wqs/sdyzkrgeruqcppzmnjjadeltkqmij, common/shp/vuxaxcwqfgrxjanzgwdstuonxvngl, common/wsd/jbwzfzpsshzqozebzaemyjhgdlrxy, common/cbp/oozqsmqhietxssqdbhyelwbohiqoc, common/udf/yaegxuuugkeprhjufvhkgsrqspcby, common/lki/oftdglijfifguijehgvlxejwtbbmb, common/dhz/ugelkgbstgjsxshdzoxvbiuodoapq, common/jdx/vgyjyyixikykedufzafclymqrhqfb, common/ziz/ytlklxabilbwctdzeggutwzgxraao, common/blv/njkhlibpagrktrbxsmafvldzxirmt, common/cxf/xzkawduteaxmytlcponnjfsnzuzug, common/rtb/wgwevnbipczvqxokqvtyxrpbfpcau, common/zde/bcelbgvopqukbfnglspxquilpvbay, common/zhp/fdoqsaswzwssrsniroxhzvxrvciza, common/obx/nizqpwjlxlpzabqpgskeiecqvphnq, common/smh/ikfhjwmbvmtqfsxzbevxlapoxxxvf, common/fcv/hzlaygwfhejmgjkvlsotlzeqtpytg, common/hin/tyxyymavwqwbbmzkjemwkvwvnewxq, common/jbs/dhnxklcsflxmpjpjhvwirvyegzrns, common/xvp/vwgyydpzptbguxayzhgskelypckqf, common/juh/uvoiwbffztakqhqkooiswvxyzmhri, common/byh/quumxrtljlpybvnyuaaueoaayxxgj, common/rkh/sghtiatspubgejcomezicsjvzcvfu, common/mox/lltepwllcfvjlpzryyibsvnbkzhki, common/ctl/rxddswviyzdehfepihqlzgpksukej, common/uoe/txujugpnjnouczlxhdskvyuwkuimi, common/amz/slwgybwuseyanpbtwfoowvocnjjdd, common/zqd/emvpwerjiuuwcheucctgnplciskbg, common/fjm/nmtsmwvtenwfldsrjublnecydgrqt, common/uzs/gcrdcfvwdacgnmqhkgefqhgnkfmdi, common/vzl/rtrczzhsmiphbmpczltzbwgdrtxkd, common/rba/wnicwmpbapmasyltsuklucliixgsb, common/vdv/hcpfojqnxxbymrbzgqnbcqvjqfjsk, common/fef/hsuxlgnfegncvmhzjrumennrlvhqo, common/axd/uctinfpaqjxfbhdzhzwidcuoaxzqu, common/uek/vtqqeuaamyaijrlobjlyyjudsnhvd, common/daz/qwuxeushubpprnuykewnvxxofnjmr, common/wum/bwgsysutgktnsgmeanwlzxbkrjrud, common/xlm/fkdhlhlndpjfaepdptzexclibzngu, common/tlf/gvlxktmuvahdhofibodbvsbflklmk, common/gvn/lrokmpwpvnbabjyfbozpenpxubtdx, common/mex/viyujovnupvzaixuzpdmluipjdpsu, common/kgs/oscasmadczjgmgzfnzvrpxlvqsxcc, common/mlk/sskmmvwqdhnettgqkdowvvwdomwiq, common/ieg/gnygcfsewsrbyhqhdxqdhpfcfdbfb, common/ioe/soxaxblfmipsrkuawzvqziixithku, common/afw/qabdfjugquaqoshesmpmooatatxkq, common/sfn/ugtktiqzgtwjyzighusirabgaekmp, common/hfj/pwsjxlbqfhpwldiyqupiacnctlfvk, common/qlj/qryicdmkhcgexpfadgczmbqnabnci, common/nnt/psvhgolwjkosxwtpmakwhrouofqyq, common/iig/otobcpueqxmwmaopxskxetrwyyegx, common/sgx/cmcwuunuvutrgasirwwdtbbqwxlds, common/hvk/btllphlglfiijbyithxowwbmbulqz, common/psc/zragmrssztkwgrsjodkszpefnhmyj, common/nap/ubyoynmtjvwqijuaeztwipgdsncbj, common/rhv/aeqktgogrujmsrbvfuazfwhnwbmht, common/fcm/dtrprnahqtxdauohybccznskadspt, common/pnt/grvtckhbadbfgkcfnnidousrxuhom, common/ofx/gythmpocqvrsrqfpsfhejmuuplhct, common/xrc/ceqzlvmunawsjvkvqefbukvrwseot, common/rhb/pckksbhkizahawqfjfiyjtavlgkqb, common/cnk/geybydckbhaqbmtofyylavdomeonj, common/aos/rivgbgiozznbpvnphaxtuubckbwrt, common/yfx/detzedymwngyvszjwbszewwphfjlc, common/wen/njswsjcvcfyhtnozszgtpwxysnyuk, common/ybk/wluzqxswmglhekwpkcrspsjozfbov, common/xtn/drrbgtuddpzlpjzntgkgzmfakjxyb, common/mzd/znsvvnpfzscjahfgvedzmsyfyyvdd, common/fzb/ihposrxlkazlrrzllwlaoxbzptqsz, common/rnw/knqnjmtijuocrycdtenthysxtgfck, common/lis/tsbnmomibfaiuidzwugbiqnflyjqo, common/asd/flcfnuwwpyfqskdjdhznrkkvsjdpc, common/vbm/sspudtvkzqxekmiscnngongreiayc, common/rli/dyusgoisyvmutajbmyzskfglqfbii, common/sjn/jmxusmmquruqzahxynoyagckuerhm, common/fqq/ttcqzuyjgytpwxarjwxrhyntsieat, common/ubq/fclsquggxlbamjqtdnafjilgiihrg, common/gho/gpeqpfltzrnasnfjboqtzwewmmoqq, common/gdj/imnjzikacbzfipaunzwvpwqmovskn, common/nvt/mfktugvadtglhjitmiwweirmodraz, common/dgo/mnrqrrinjotehrjqwwkvhmqyvvjaq, common/imu/rafgsdhdmhocnmyttpccnotjqsian, common/bfa/ixfevobzpoggbsvreqlsmxdectukw, common/vpn/jmknnwzogjmunwdvyciplpdvkllii, common/rpn/zjhzmiyesrpnbfpfngubukjacsqcr, common/dhm/ivkgwjskckmjlvwqxwhrrqutccgrb, common/pth/zotfxwhnooequicpjtuhfwmyoeuky, common/qiy/mmsmryuinusaqpllzwolncpiriwan, common/pme/xxbrcugxkgxanjdctehwhixplnwab, common/jwy/khfhrpofkhfylcamedqcxofydmpkf, common/sjl/fotdzbennhwyqvxmnqhbtahlahnuv, common/snx/djzdchlhqubgkxvrdpjdlobnwblpw, common/ytq/lejzpvqrwecblcqrlmrrkcigdpuwj, common/xfj/ixhzechqzorgzuibjjonxusrdwhdr, common/gcm/oneabwsetcgthhefjwikjdytoeycg, common/dfq/uxuyrijjiqwtkjlquzhehmcswsqyv, common/ahk/cxswprdbotcplmwgmyotovhhmpzyq, common/wxz/qebxrbbldhzqwmhnmiocdazvylprq, common/bav/qeywhuobipaiucegmtdmiuzbllecr, common/boi/szhkjckgaljeceilsezhkqafuhbqu, common/iet/vambztaxgjhzcgpplxdvmxewhmini, common/xqq/uiiqwsfezlbrnevudoopdzisftgjo, common/lxk/heloeyhybwsxqwdsuhbjyoxpdegyh, common/ilx/orpbqcacomwaqentnlwpiapoylfit, common/lon/enagdsqifhophtcogagvgqkpadflm, common/vfi/plvuxlcgfhnxoqacflzxibdgfixfa, common/nhh/zuqtsegmiwpcnwzyykteogomnecyt, common/tqr/lliuetsmqbjafsgbzmkadfkwhldjt, common/jfl/klnnzhlzuwxgetkcwyqrmnqjqfbhi, common/ymn/qjdpyuohcnazakhwtdhddilrllmar, common/nyd/slzknfflrtflokruysjkdzeyqdjyr, common/lxw/smrrkwysqrmsajffbqgarfazuexwi, common/ymj/jlbpmapuidnjxujpaliesndisexxq, common/wdk/mldxyihzhrxelwisipixmvnkwudpb, common/tum/dybsqrbvondzpjpcdmrjyztqtozgp, common/nio/kkbqehhwurgmbgjbrmvxexjpqxupe, common/fke/bkspyzxhbixfkxiufbpswvgtnsmks, common/vxm/jpheipvdlizkylidvfiaxkzphctbi, common/bvj/hmlmdidawotzysrfqfvqhxltqqltj, common/vjq/rzngcilffyllmzpmcaqneninpwgcg, common/xow/rgkvvgpzusvpeafdrjjwfrfifhaui, common/yzd/yofbqupjigonzcomwgwmsiwadhpqr, common/ebs/bjmtqchwybftwbyixwlzrphqkbktg, common/gjj/lyrnlprbxrfzempbddzzcwuankqfi, common/jhq/qkkuywqajrfwuxtkdpdjgwuhtjgby, common/krw/xcbbypjnnwtwdyzwvvmxtjglcypek, common/mlt/fpomuxgbiidfojnhlviflhcmdkmcs, common/fwf/irmwvemzsutbsypobuwsfoxsmohxn, common/zdn/cjwtvppjgswnqaqmijwxpxvunejpw, common/kji/gwlbehfzdtsecgbdhnrpdnutyfsht, common/ovm/rezefalcguhcqofthtemyblbbfzdq, common/hzx/xlxdvbqgnjfudcvnjkwycnuzdyqqt, common/irq/qscxsrofipawicibvdelrgmvhgdps, common/tmn/rotrvgapjzecypfvjpbqjwklwuacm, common/nrb/neffsrrwpxdudsulsxayexyzicmdb, common/gxj/sotmewwupsqoaiesyetwnjaiafebs, common/ovm/oazuwshlhuiadtawqqkuecsqaqjna, common/faf/ipggeyqumogzoudmwhcesyxnhrcjn, common/fvy/npmgevbvtldhezrqmhvbinczfknpa, common/jyv/vkrfrvdfrfxlrylmbjowbxofhdxlo, common/gpe/msrqawiamemirsejbomqtgnlqjhka, common/pyj/pzqbqtuyglbbipjeuxxrbzqbrwwgc, common/jvh/vkqooewqqobqztsumxfvuykgdvuau, common/rru/vjtbcqcmlokpnoqsvembtzijmibfk, common/tte/qwytoyqeroeyhblwoqwuavkzjdbtq, common/gob/ajylxngqaajmofpinytdaqekngunz, common/rhw/weqezqmmdlmctnwfwqptwnnupnhhw, common/bli/iieqgcozdisjxwdyxlwlxqkkshxoc, common/ova/bfiuohdemyniqizyvqdwpxswgytqa, common/osh/diqwhzrpadexrqxoyjqqrqlehhgwp, common/dkf/uyaefbtdszxhhdmbidmalqludqwsn, common/pon/nwseagscnsuqfukovukaeamlvnhbe, common/xgc/zlkkwqpkreufhafnvuhhptevzofhg, common/yfn/sbpyybzzvtvrdohrpwdfitnowjsue, common/isz/ysbjxixmngccjsquiagsdealkjbcd, common/uyo/ycibnuzsrdafitxtpoiehldejqykt, common/ytg/mtghjxteajrkdqmntxefxlazirvez, common/qvt/fjbiaywnjpiouvgvmctisjijqdamx, common/qqd/njjojfzvxvaodnluumjzymzguchtj, common/ksi/eljjlfayxyxawydoztzbyhrvweagm, common/mny/sqxodrtehrxxvbbolpbscncgfusfg, common/qwz/quglcvdfsuvwmhikzbxuiosftmxln, common/ruc/gwjzwxiifigkocljnuketiqwbthby, common/hmr/rfmywqxkbqrzlalpzfmzibxhyaxfl, common/wmz/ynmjfflmhfuvuapyjkrjiwixtnaeg, common/ngq/tjmacxjywkhpnebrkumsxpixahbet, common/udf/xditvyqtatkrtcrjumdxrxboaugzy, common/tfl/nreccoylhrvzcmctcbfssdxiuuuao, common/dky/xitvbinmpimaoolhnvucybmhtyano, common/dre/vkalbrrwrwavgcyjvtegihniwxqpo, common/hua/gvifauaxjqbmmhfyckdyuxogdudlh, common/rlt/uszlfzjorfcixodhvbzywhqkfxedu, common/enx/ickpyrxgrqivqdtbpxupmcopoasas, common/nzr/fhkkzfbakrkqrxxktktshcbtuvyes, common/gtd/cpnnzpqvzgwwctjkccahezjvaayuw, common/cak/akomrzqxdivbevigvitnibdlkibky, common/lby/mvgnpzhkmffspdiiwyjzmbayyevtw, common/ppp/nmghwpphapfkmedzpqtjoiuosbadd, common/yqq/icedvnxhfadchndmgclqzczbwzzsy, common/rvl/hdbulppjvoqyxolpvekenhernidtg, common/tgm/avoesdoghfdgmyojquotrmkbxoypn, common/hde/upadgrtdnmusnojhoaymlmzyknodv, common/mow/zeaxjecsosukrdahmatflrqfmeuja, common/knk/lbyagflhmtnlrmpvxndwmghgdadxl, common/lcb/nhsokszjnmrzapsdnmbudggwgazbx, common/xfr/uhdxtmuflwmirdlzhwzajdcngezne] were removed from S3 2024.10.22 01:36:03.163295 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 19f4f527ede8afb80441b000baeb05a8, offset: 0, size: 0 2024.10.22 01:36:03.163785 [ 1188 ] {} DatabaseCatalog: Execution took 274 ms. 2024.10.22 01:36:03.165485 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 19f4f527ede8afb80441b000baeb05a8 for removal 2024.10.22 01:36:03.169423 [ 1476 ] {} CacheMetadata: Key 19f4f527ede8afb80441b000baeb05a8 is removed from metadata 2024.10.22 01:36:03.276961 [ 1418 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DontCountHits.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CounterClass.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsNotBounce.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/MobilePhoneModel.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CounterClass.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/MobilePhoneModel.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsMobile.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DontCountHits.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsNotBounce.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/MobilePhone.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/MobilePhone.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/IsMobile.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WithHash.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/WithHash.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_6_6_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 01:36:03.364961 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_114_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:36:04.061089 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d0de2ddb5c7bf7a3adde6a9b919f8205) : [0, 181183]: Setting download as failed: File segment: [0, 181183], key: d0de2ddb5c7bf7a3adde6a9b919f8205, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:36:04.063027 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d0de2ddb5c7bf7a3adde6a9b919f8205): No space left in cache to reserve 181184 bytes, will continue without cache download 2024.10.22 01:36:04.085037 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d0de2ddb5c7bf7a3adde6a9b919f8205, offset: 0, size: 0 2024.10.22 01:36:04.087936 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d0de2ddb5c7bf7a3adde6a9b919f8205 for removal 2024.10.22 01:36:04.093279 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kgw/ijmrqbyxdoxpjjyiiynjhhpqpafca. 2024.10.22 01:36:04.093305 [ 1476 ] {} CacheMetadata: Key d0de2ddb5c7bf7a3adde6a9b919f8205 is removed from metadata 2024.10.22 01:36:04.239547 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uke/rvkmqhpxayyckyvcoqxyufrblrfjj. 2024.10.22 01:36:04.468455 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 475 2024.10.22 01:36:04.864105 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:04.866307 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.98 GiB. 2024.10.22 01:36:05.048109 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_457_457_0 (state Temporary) 2024.10.22 01:36:05.194853 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uke/rvkmqhpxayyckyvcoqxyufrblrfjj. 2024.10.22 01:36:05.204280 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_4_4_0 to 201403_4_4_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:05.236076 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kgw/ijmrqbyxdoxpjjyiiynjhhpqpafca. 2024.10.22 01:36:05.257474 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_457_457_0 to 202410_467_467_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:05.278706 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:36:05.352100 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 475 2024.10.22 01:36:05.457823 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 29 2024.10.22 01:36:05.509351 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 63053 2024.10.22 01:36:05.747642 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 7.88 MiB, will set to 1.75 GiB (RSS), difference: 121.26 MiB 2024.10.22 01:36:05.766136 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:05.770308 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.98 GiB. 2024.10.22 01:36:05.812372 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:05.815636 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.98 GiB. 2024.10.22 01:36:05.947814 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_29_29_0 (state Temporary) 2024.10.22 01:36:05.959226 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_955_955_0 (state Temporary) 2024.10.22 01:36:06.133789 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_955_955_0 to 202410_968_968_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:06.171710 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_29_29_0 to 202410_29_29_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:06.191738 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63053 2024.10.22 01:36:06.477139 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 29 2024.10.22 01:36:07.112282 [ 1258 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:36:07.467127 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.97 GiB, free memory in arenas 11.82 MiB, will set to 1.52 GiB (RSS), difference: -71.18 MiB 2024.10.22 01:36:07.996597 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1751 entries to flush up to offset 853050 2024.10.22 01:36:08.455306 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 103 entries to flush up to offset 64657 2024.10.22 01:36:08.634748 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:08.640746 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.98 GiB. 2024.10.22 01:36:08.683101 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_555_555_0 (state Temporary) 2024.10.22 01:36:08.730654 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_555_555_0 to 202410_561_561_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:08.778197 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 64657 2024.10.22 01:36:08.855538 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:08.858719 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.98 GiB. 2024.10.22 01:36:08.987852 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_542_542_0 (state Temporary) 2024.10.22 01:36:09.155685 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_542_542_0 to 202410_549_549_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:09.307754 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 853050 2024.10.22 01:36:09.320649 [ 1179 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:36:09.361616 [ 1179 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:36:09.452275 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.97 GiB, free memory in arenas 23.83 MiB, will set to 1.55 GiB (RSS), difference: -23.73 MiB 2024.10.22 01:36:10.055613 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 8714 2024.10.22 01:36:10.211507 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 63083 2024.10.22 01:36:10.265407 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:10.268443 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.99 GiB. 2024.10.22 01:36:10.300218 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_566_566_0 (state Temporary) 2024.10.22 01:36:10.410822 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:36:10.416370 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:36:10.464661 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_566_566_0 to 202410_579_579_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:10.492840 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8714 2024.10.22 01:36:10.667478 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:10.670265 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.99 GiB. 2024.10.22 01:36:10.712665 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_956_956_0 (state Temporary) 2024.10.22 01:36:10.766467 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_956_956_0 to 202410_969_969_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:11.050914 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63083 2024.10.22 01:36:11.337371 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.97 GiB, free memory in arenas 20.96 MiB, will set to 1.56 GiB (RSS), difference: -21.35 MiB 2024.10.22 01:36:12.855606 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 476 2024.10.22 01:36:13.368024 [ 1202 ] {} DatabaseCatalog: Execution took 201 ms. 2024.10.22 01:36:13.370391 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:13.372580 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.99 GiB. 2024.10.22 01:36:13.476014 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.97 GiB, free memory in arenas 19.77 MiB, will set to 1.60 GiB (RSS), difference: -11.56 MiB 2024.10.22 01:36:13.557459 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_458_458_0 (state Temporary) 2024.10.22 01:36:13.675560 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_458_458_0 to 202410_468_468_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:13.733075 [ 1201 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_463_95 to 202410_468_468_0 2024.10.22 01:36:13.754732 [ 1201 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:13.757063 [ 1201 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.99 GiB. 2024.10.22 01:36:13.782885 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 476 2024.10.22 01:36:13.802451 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_463_95 to 202410_468_468_0 into Compact with storage Full 2024.10.22 01:36:13.870640 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_544_108/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:36:14.272353 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:36:14.312599 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_463_95, total 507 rows starting from the beginning of the part 2024.10.22 01:36:14.964750 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_464_464_0, total 1 rows starting from the beginning of the part 2024.10.22 01:36:15.054045 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 63113 2024.10.22 01:36:15.226571 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_465_465_0, total 1 rows starting from the beginning of the part 2024.10.22 01:36:15.312741 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:15.316637 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.99 GiB. 2024.10.22 01:36:15.375741 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_957_957_0 (state Temporary) 2024.10.22 01:36:15.571000 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_957_957_0 to 202410_970_970_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:15.618260 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63113 2024.10.22 01:36:15.649664 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_466_466_0, total 1 rows starting from the beginning of the part 2024.10.22 01:36:15.669677 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 10.16 MiB, will set to 1.62 GiB (RSS), difference: -29.51 MiB 2024.10.22 01:36:15.824611 [ 1418 ] {} S3ObjectStorage: Objects with paths [common/zwb/lrqkhipaczgofqghlujkagueyemqs, common/kmm/tojvevcyftidlhtzoccsztyyjxmyk, common/jhz/erhlrxgosegxaasoqdmocabnujhxi, common/ofd/danhoyudtoglvgcycnkbmuymerraz, common/uyt/ahpsytpuahmjeerdvketjwdmmzhtp, common/vai/ytqwgiklsqfusofriyfxanfhtxfkf, common/ygy/kkraqrhdjxlazayjehdiutihpdrus, common/nmv/yqjjzcjwwlpwkwedebfibfkjghajb, common/ggl/bgliyasdbslgfxsqoyakmhvspjcbz, common/hhj/vrdlesowcygvueyanztjpiwsfhxyn, common/ejy/vabzpbycnnthraaaudexonsaazlaz, common/ocm/rmorflyqcwlwzelgaraljqpgqeosw, common/kck/dukehjlzmbymkvqcwifzpotjtldhd, common/ety/fbwcuzrzswxotvpiomrjfmsijdmjm, common/iro/macfeiffvnqpfuyxjhulhitxekfzf, common/zzl/famyxvphyurrjnqbpdsbmtvwgukos, common/rjw/crloawltntcxkisszcbpbpyjjuszq, common/jlw/jxtfwlxatcotjhybyijqasmzcevxk, common/okf/vsvjhodthnbgrdqwtngeldwzwocyd, common/xei/oharjckzekuyrzqyarotitutzdspl, common/gcr/yoaooihjmmvmlzkdrqnqrsheqqaiy, common/jhx/shxjgumekwerqzuovgrppptkwkwrn, common/oks/holiefyskvvimnehfccbslztmkoen, common/chx/dgqlwaegmsxovsrzonrwvlqjuuefy, common/yxe/tugmbrivqnijmarnwwjdcuulbncfj, common/frn/natfeysxdvbwizmkewknvtflbgfqm, common/bdu/xuvtowdwcuwbpdnmyatpjxoxnekwx, common/ovj/kndoocxatbnboxiiluruckbssunit, common/mdi/lrnzicksorwjcpsmynozuszlibsnp, common/oib/idcxydlswtinkswujkjnkqvkdeyxn, common/smg/gkagdwgcrilhsuegidzukjdylgxqo, common/hjt/jivslhnpdaqaysrtalcnrqagwrvfw, common/yqm/dbalyufztdckhgnrxeltfihaovqgq, common/pyb/hgmakgmwwgfkauiycxkspgugwfanw, common/iyl/zurapicxqcraowsjumqxlvygclfqm, common/iwo/ffmvrmznkagewlpuvklxdnjrtlobx, common/usj/zzjvdhqctypiugjbxwgiilbutdhqc, common/tvj/xossiuptpyvkgcohdntjdbvyeukdi, common/wsr/qnuzoilymwbowoaonbgevwlrkgkjy, common/tqe/lwcddbqbfoelinnfcbcpoceixyouz, common/epv/gxjxtbdrlaaosxpdrghjwpoabwzws, common/wfy/jeemqgacijvnvkryyiqynmhchixvk, common/ivy/iyutyhzukttkukqwpsgvuosxcknvr, common/szg/nweesvmwmkgybinoajgtsrqmwpygw, common/wtf/xmkgpxrbawkgwqobbgqwercfhczaj, common/epf/ltpwnlmgpcugcqgdihvoehwimlzzt, common/vfy/brkryzbgtwfduqozgcuntlochxwtv, common/xxc/gtvmggbtmqgfnkpkqvkrpablqofxg, common/ztb/zeohwcmclrahproswpucrmqokqwxg, common/xbw/gpjtulzxpynbmfzdcxgqqkywqhcuo, common/cwp/qmirpdswptsamdmgvuvqzorhioqsl, common/aus/tofdgdvdvuxppjlbqtyddamtzvqkb, common/kye/crosyvaokhqpmocrqosjzzzkmursr, common/tts/aeaktfpvhzxnvwmngisqjmkkshyaq, common/ntq/uisrjgobbxxhwentakcwsjszogxhp, common/ykc/smvzubdogezkslmassiahdqkcxyfs, common/ndt/qahcfeskketouiigwdakwurhaqnyu, common/fdd/cqzwmtetgjwztuvhkrlztosyjajgs, common/bav/lsdxhbftjeamuvnqvbwhiqonkrmwm, common/all/rkumoykhwirzjeigtkwoxlaiyocsa, common/tny/sgtvxdpofyuposnbedmijzzzxfdqp, common/fft/nsdncllvdbzgrtwqfsybfblcrpdgh, common/apm/ncpcmfvxlncjfzjffvvsqgmegaxky, common/lig/dfpycvwxtxucownxmltqicxihkvnv, common/uep/eefofkpbvdhjrpwkapldvakjhhhrz, common/ldy/xmujcpdmjgxyrwyvyhwwvfpolmxmh, common/ssj/wktskhatrmsvgofugmkiqhxflujiq, common/lqf/qfkezswgtbjhzfgloihsuvfksqtfj, common/qzw/ghieywxwuyvoqtsmrdwnhpwxnlelt, common/wsg/edlfxrkggcjcvnlrgoosxfrdvycba, common/dsm/hfqtihxacmexkzlaegyrmojpmixhk, common/gll/chmygeyqiwldxhaecjbstaeeankzn, common/gdr/kjzllvrwdlzycuqsuodtasefqewmy, common/spf/mqnbsyezircpsobxzpllzvzqugatb, common/enh/qyeuqimrqasxdrpujfsdohufgqppe, common/xjv/tjefomebeljenrsmozzjtjmofjlkv, common/twb/ybyhdgpowskqacqxavijvvqhgcsry, common/lgf/cuvnqiaducwrwkppusdbrptlysmzu, common/nlh/fpuctqzsmdusxpvwkvjrqjjydbxkz, common/jci/udaicvexdmalxcousihlatafdoaef, common/asq/begdtwyisvylyyvjmxouunwceavyd, common/hmc/hrhilndefqukxcmwuazmkvxlgnrpe, common/rwd/kzajhnmafidwroaclhfodtibduwid, common/chy/uleowlitwalduzxvwglvlanvjwynb, common/rab/kjreiaoagpvlfscfooghosnjjvxfg, common/yst/jqotfuhrsdtbsinojjeprhbrpeidl, common/icf/wclaeqwhzditglwzyqvkrfhcdbetl, common/zeo/rlhxsaezqywckyptvjwnkcepqbpfu, common/jxh/qeipzgeksndwvgwsirqcuvfhwekgs, common/doa/prkozkigcgqcxspgmfjsqxtpvyezu, common/gtn/suahnqcltbgcmquhbeuxpavhekvtk, common/kcr/valtabajgavwzmvtwynlbcmblzxgk, common/gcx/tyknrebwxlvgdshghrqbkayhprsez, common/pdd/eqaczfgzhuuyfbguljachcejnxzvd, common/xru/mjkietaterhalcxmynulojftqmbsq, common/vfv/dpzrgvonmchcxzutzcrsglnvqzfan, common/uve/uclnoifkhedxgbfehlyxcdnevhtjo, common/lvr/uqkrddsoasvvrvgpejistrgrzekja, common/yow/vqbdqsmryvlnbzizcxaoswddaakps, common/nwl/ikcohxjwpdjgzmmpakzvewpnhmpah, common/kgg/fldyzydnmkovzanrdoybfumbaxurq, common/woj/natsgkekezycvesjpjtcszdmpdmwa, common/wqp/qvyxecsyyzfpijljkycefqvsebnhe, common/egv/qkigqpakhvjfyanqzijprkexzrajg, common/cpe/nqvwugvidjufelzkwxfxgpavmscfv, common/wpi/xdeqxwilpvnkhzxdxjkssgoypwbcr, common/zlb/negvxbmucsietarmsofjymxnarbqt, common/noo/yxeoxtcvofdpuiokazdcpwghbxhwy, common/oxq/jnxwbjejfwhkehtkpcwnwzxgnhiwx, common/wbk/rycveqplwgifjwtclsnpccmtbbcaa, common/yvj/cpwdljywgbbehqqtlwejdnuefiaog, common/gqz/ajsgysxxkgrpobvdcxuesjwtdykqz, common/xwr/wsvcnqblorpyqomikegbcpqjmrxbh, common/mbb/lxytfkbfhnablwfocutxqlycwzrla, common/hnj/aoxauteclwesrupyacyoooaujrcrx, common/lap/iwdjhfbnvtepauyokovfyajyxdquy, common/nzy/sziiicdwunplucnngvihiavpwuvik, common/swr/gkuhmwyugtfvbgisifenbmdpmvqkx, common/eld/ncqjxsxeldtdaxngmbeqahxfbfbae, common/gje/zyqveyfkoiaadpgxorvwovrjxvvxg, common/dwp/whhkxznygjhkhrxtrlpnznjoschpm, common/kdt/lngplwseoxerpgnyadiajcbjfepfe, common/ivf/qcrlqdhkuqeegsspdgsfidznmsjqt, common/zkj/ndishozgazdvrttbretsmwdcxdebn, common/dpr/ddkzedcnzkcubdcyjbxbdtejvkins, common/zex/cvyowrshcbwrtkgezaemgtcvkahsy, common/nuq/xrocdrftcsjfhkghjxhiprzonfjar, common/wlm/hhdurwqsltylyyqjkooiebuesbfvk, common/mho/ohoctuxuyekmhcxvxbxxugacmlbha, common/qfa/qbfqcvkmkcecfkmywndsagdeptbua, common/fcx/vjpzbbqwwjltztwsobkgvhirfmiob, common/gqj/knfmhlguddvelkciqyipdmzqbfxyh, common/gyk/rluxiexqugxuuavwzqsfuaowdcjap, common/vva/lbrzisyflybzfpuwnmkvjtybhzniy, common/keh/icqyknzmtfslygocztvftvipimdpk, common/cwy/ehnqilbrhprzbpddamplpjouibpza, common/slu/xrcpzacociqhvicsvjotqwytcloda, common/qnd/cpmwwsrklqibmjzlfvnghybhxkmwn, common/qth/strwjipkeoyoipiquxdvwkcfctamk, common/zpk/cvqdtjzfmtactdhywnrvxhzowzqlp, common/clp/yomdhyryzfcssbzztthxsqbsbcfah, common/ptl/kkviixcccmtsqpottmubgaprojczo, common/syn/dbwwmtsmxjbjogrmrkrizbtwzywhl, common/dak/rwxdbmyjqnvxqpugxysmqkshkkrix, common/yyl/gdrixnfospcyuimkipuefurapagpw, common/lxj/ucjaptdfwwmfkjbtfqbuluvgjxime, common/gpp/ntbrxuetkekmbvdsszmlqdxxmoiql, common/log/uiyyeaodxnumgzkjxoakwrnjwnekt, common/ivy/psmohzitbemjneijvcorrzoaaexvs, common/hrd/czcpowtbruyaodjyeyaeprvnwougi, common/jvw/oplfquzqwwuqxtreihgpfvbpfvogt, common/njw/sgewxmxbxtbpvesoryetknzpxnoua, common/njk/eplxeffmravarkwmdxvsrwkssroum, common/hju/staoahlfefuyyfhflkumugkapgbmq, common/ped/fdcqdbrgdflukuttdkwfsewaqjvca, common/gsl/qfehljgyovyypqkhzazolcluejwjm, common/zaf/oattstyjzhgnhyxuohjqgwcuodhpw, common/lzo/jfijqxedhaiyhhntshdwpsayxcoii, common/uyx/motjgdnpsjvdivjyvortqsxlfwiyh, common/szx/egeraszppahigkqckftrplfvozxpu, common/bpi/ylhousduxtuosskbttcyypnjblfju, common/ket/ruypaettjdpbnnhwictdiqfktmbnv, common/vkf/cpirrnuxpwgjrczggjdmzpexbitgi, common/npx/votbkmhytfwonzycxtjkexqgciwca, common/yog/sglkfjtezhimjavmxwuwgbnqubbuq, common/bpg/jeaigvshmkijnszxbabnmqgtcuqzu, common/mhh/yibkvgypwxzsojhdqszlkqnyoppkw, common/pvt/lkndbaceinsqbyugbvjrqfotadpzk, common/nor/ngqmlhvacestmwikjjleasjcwnrjd, common/jrd/fvmvwnecnkrzjfgvkehhhawmjcoos, common/hzc/oqnljhtbolrwlovyfruyhpgiqwuwx, common/wsh/uodlpkfmrldyfpdrtjamxgldqvacm, common/spi/gaglbrdrtivhemkxqtvxocgvnutyu, common/ura/icgixpusiybkdfsqzmlhgsmjoxgap, common/pjx/iihdcdhaxctmiawfzqqaaexdehhbu, common/vgh/cxtgjzxdkxgynrvkgiuhrxrdyqjxt, common/eqh/zomojeslglyanfzggtwltkelobmfg, common/eos/obvsdrxpibyhvmacdpvlkzhlkrvwd, common/svf/gmbvosfuupahabpkmvqhjbrmiipyx, common/cqw/bwsthiwbtbtgoyxcyvvithbuvisbs, common/vza/pgufcxquxlacakvusezolohfdekmg, common/mhj/fxanxplgtfwnupfnyljiacectoshl, common/gvk/dphwniaxdvdhtlpfnlalrrydaooty, common/bpm/yydgypopfwhnbjzqqnoepiznvuwbz, common/ynb/nmtnqkotoypmtlplbgwjcqbzttzll, common/afl/lzjmauxtpdpnqxoqpnrnrzrermult, common/qsd/xznazuutwfxvjvefvgpiugjnnnbuu, common/rvc/juwgtwktnbymkrnqutadsyqtilgzm, common/rig/njrjknqhbkngvylogkllkvgujadwd, common/nwz/jbtdulutktgwbnmmpgvutldmjryqj, common/oud/thcgfirsyktqplvmidgmeroyycfli, common/uus/iyzipvgfkvhixobnepdwwmtmurhgw, common/njp/dlshihwaxlrmqiryulhklxfzsbayn, common/jkx/lhuxvtnzusbnjgdufzlxmqmwimcqp, common/yad/orqtsacrusewttftvhfzcqlnkfequ, common/bfd/pczyhzjtfwlkoprooahsvwolkoqki, common/fke/qgkdatzbduedxzpfiqzfhdejxcocf, common/vnm/mxurabzncqyfdyugiddyqotjhiqvg, common/mmt/upkukolbwacwuxzcdlkerjzsojotr, common/vcc/bmxyvplvhyjuepceorbfqogulqstj, common/nvf/tjgeuvpiveeikkigudhvpfftzulwo, common/zcv/wqmpixxzoumitegeoyxrudqdeajcs, common/tyq/wgpcwwwijphytftbwszrxcsanffbq, common/wey/bfutsqnrbnsbdieqobyraggcbejwn, common/kim/dyhswglqddhiszygwrxgucrdpuspr, common/ocf/rttluqqvjjvmfxtdsqoxoxxcgoyxi, common/dhy/wwbczzmdoesbqcxtfeqblhvvitwqg, common/cng/guzjkznlrgenppvgkztkbndxsrwsl, common/unj/fschzpvuxmdndkcetrsybcovexajs, common/ixk/ssflfymzyndriuywjgytowkxiypco, common/yll/nhznvqcautykbuvlahubmnfuzztka, common/npy/goafisqwuprdpreqqdjmtqgjqnvdz, common/ptd/xygxleieyiefpogabhmvgbddvqzzj, common/gce/lqsxvktqyghfnpuwibewftetkedhz, common/ojs/memwkclwuqgzbnanmtpqfpyitvgcd, common/drd/zoudkhljpepeobjsrmdatehkacfae, common/haw/wbbcwuimkhgypbdjmdsulwyglkkyi, common/fse/tfwrydabgqdzoslndcrrgdcrujxay, common/cjv/lpebvokrjfqoxevitpwovvlpayrjy, common/fni/mccyswxwpukkazpxezwwmomvkuhoq, common/mee/hvyweuqjlihlgddjqxguvsrexrsmp, common/zoz/bdlqqftaoevlllmmgrjkorzqzlbsk, common/kkh/vtjiildqycmtkwfyzcuachftfgmfc, common/vzf/dgnpvpvekydpwdswwkemibcvqpsib, common/pxx/rijdrtjtwqzahpwycoqaarobzomnp, common/wkw/ziwmmivfiffhnykqyxwvmhignmgfe, common/dqs/jcobmfkufjvlvqcmsxjgfhjyalqcf, common/but/ngfqzkowuiftzadkvuyrjqukipsvp, common/guc/logkrygyimpuykmdqktfxsuycheud, common/guf/iojljglrayklkpcadkitjikqjnbpq, common/emv/ysbushwwxuqbsibbsimgdgkuhotct, common/qtc/nzpxknznclrmfigwjryyajmovjydi, common/sea/ddxtgfcbpuvcrqplvwizlograrmai, common/meg/yhwbezeduwsnvpnkzgoulgwclxqxi, common/eqp/iuctbndzjjwvhttjafsaexiiwjsea, common/qxj/mznowbduxapzrcbxcnwqqhbgjhqmm, common/tpn/jdglfjzhcsmjsddbldelbyjokodqt, common/wwr/wdlciuxrbifgfcdyfurzhutfbkqel, common/udh/coingsryoiflmtsqesxtlbreoykvs, common/kyq/pqprmizcucvqcnkzctoauytmeszyy, common/pfx/qdnuaznmkskimyekvjuumfsdsuabh, common/nhm/jfkwhiyyzeyiepvytrncjjnxununo, common/kae/urhvdovhodcbbmxpymyndsrhseice, common/weq/gisuvpcycxjyjzhswujxsgrsgumkc, common/ixg/kfkcesogvtudeptpsrxwrizkmvsqs, common/kqs/cqymdnckpkkfrsipzzfgzpabymohs, common/gff/wtufyxfjxfnxjnmvvrvovunskgmls, common/fdj/rgatxhylxcslguqxsvjqaxdaldkqk, common/uzi/saerlqipeiqxwaxhbdbuccifdmjnr, common/hqj/unshdfpfthntrijibvkffxprpqwef, common/xsv/ggunoomawtbzbjfvfxicrjdvhzsqx, common/wst/lpepyysosmpmokwkjuomiqyeugsxo, common/esm/qefxvdjbprpfhvewsqhzadhfoqxwi, common/xov/ladaucmgxlubeyfwxathtgkxwxywy, common/krv/qijlowllqdjrhzuthzmzwyfoqpjna, common/gdg/vfplzrroycodpeffjlyqnphfxjnyv, common/vdz/bnbeirlcvrcksrkklhrwkewmbhcfl, common/uwl/sraypedcuckqegwuwrcygbotkepsz, common/xwl/nkrfebtviagqemayafqtobadtfyhg, common/mcu/bnobqcsvjkwwepewsiefmymwozlig, common/yux/hptnelrggtzotgtkuifqrnhltttoz, common/dal/ujjvvoxwtabqvgaumklsmpxaugsmm, common/vzx/skwgmhbnvzbeonmhgrcuhjidmisay, common/vlt/mnvareeeaiommqzbacgbfkdiysoue, common/yxy/hhanyurzskmojmjuqgtfgifavkovp, common/vww/zyausxenvhluakidxsyhpbjklvomr, common/btb/sktsbdqxsdwvahobhachgjqnxdqko, common/kfb/pnfhqukadperllzmnsglirrbuffyv, common/dcw/aqboskhuaiombfkeqfnjwbtlhurmj, common/ydb/zvrdbtxmgleewfwcelicdygmgdhyc, common/sny/taomcqbfpgbhihyomajqjenmslmvz, common/jxc/dmrfzehupzpbisjckybquyhjwcuer, common/ria/diantlckzcwqiuegdgpaalkwtnbmn, common/art/pfoprmidzmqsoxyaklegvwtrfmygr, common/hfl/ympcqrppkuioujabanhcvxmvtkeoh, common/nrr/wvpfkpvxuzgryxcfkdgovthsvasip, common/zuj/bhsdqxiovcfgtvvgepkpmwslbuwoc, common/ush/nrocgmwloandcwbgifbshhzlyhfxt, common/gtb/dwmoiglybxpmznavkpqfefcguitaa, common/bld/rgzrceocdzgydcozfwomskkqebkzt, common/icf/mxxnpizdjomptqwbauwdvnivknpoq, common/krg/ryadqiszaloiukipxndzdengnakbp, common/hfz/jrvjeublluvqnfoxnblorcuvfakad, common/knp/cbzxijhothlwknyzvnyxykcrmfori, common/huh/nicaclfqscmbqiahstoidlrlzebrk, common/xwn/hmmwdwzjxpxxbmxovfrqstijxohqd, common/wbk/wptkiukytyqnhwftdxkeawgxxcpxj, common/cnq/teumwmmrgpwqjznxzbazmynbsrppp, common/rsn/jblsnmsdhljiknkxbspmdqavumvuk, common/xhe/klyccdjagbjiioebwnzwwarmvpihr, common/vcd/hotqjhjrlrkcnqagsygzxgwznwlwb, common/ybo/vqkgvvfxdntzeozhzjdhxrirbqcho, common/int/okrndnqqztquwgfzqxzwoqtvycfcl, common/zhb/alqxgzmxtjiyxdysjogjjkddafevm, common/jcr/jjftuqgrowieepmgwdyhbpyvgpypq, common/pow/fdwnqmohoymqxlvgulnrhlvcsxhjk, common/eto/bkulcpughggnuzzknrgtrdtwxzzqv, common/lir/kmtsdlqxyoilukdyfiqhitdgehlzs, common/ubp/aoldyaajwwbixyaoduqikushdwdcd, common/miu/ozigbpzrnitrwelrhfvoyrfktjbtz, common/xcm/wrpwdfvkxfyilugemfxwvcnakiggi, common/twk/tivgnqhaxcaydlxwgalnuqneyxxow, common/cel/xpwjbqjtconcgkaunprzeqlxfygni, common/pky/vwmajxxgtjycybywqgngosruvfrgy, common/fcu/fvnyimaggtchhtgeoztkdhgaxdbck, common/cyt/sqzzdkefaxqlhhdpkytccfhsxekbn, common/saz/rnrlmjlbgvawkbtazbjxobxbkvcdm, common/rbp/hohsgzobrdnkgxbollagrmpeiqmel, common/pvs/wvsdhkqfhdocydptfnxeqtcikrmic, common/zdh/ksxtvedwpidfmopxldexsikvvgnlq, common/oog/gzfspfmkjhxibmziqkhwlpcsdccjg, common/kry/pwghgngioblnyljvguxkyfkgimxoy, common/blq/dmtlnkhfawkopxfieyisnuswhenoh, common/shb/wmvtyykmufmhaawjiacqfjdgfvtcw, common/mlo/fmxdwyegkjjmmenxfsmrsonthfmjn, common/xms/fyfhdnybptyfkpqdjeviwszwspyxx, common/uyo/xthasrrgxopyavwohjrtrujophuok, common/ook/gnfotyyuhetdhddsbousqebetmmyt, common/vgg/zfjzrpjdpyvgeylojrzlnkaeyjhqg, common/jaj/ccbtlgdawdxpcgesqpxgcttgrcmtd, common/ytq/nggcwkqitstaptjsvwqoycvfimwtb, common/btb/tujphgfwrbprzzstgslrpxgjslysi, common/nmo/tqvlftpddcjikqvhowgzrtbmnlnis, common/hzn/wohsqyigluloyhnldvqfuezigggkz, common/viw/smuhfmkrhhbcftjzncvfpvklvailf, common/myk/lvgnctwfkrggygywireyvqlexsiby, common/ohg/dykzxerkfzmfliorhirgjfgkffiay, common/mdt/wlumdhytepwqwbceoxuozxzqcokqt, common/gad/sxqmnplqrgefaogxdhuzllzwzmgfz, common/gzs/ircvmfzgaqysuyxovmjiytifwvezf, common/dxz/ejcmfurzbeelkzjbddqnijnffcwsw, common/gwk/kulbrsgrdukrwlrruwhkxabyhgolp, common/mjk/scijgxyfgvgaupnnwajapubsvaaxz, common/ihk/pmvidlluqzyvvwuwnibtnjhzhbgza, common/rqf/saupttbtkameiktfakszxqhkexvmg, common/hlz/ysmvhykikxgxhzjqqeeatrgxgnyyl, common/nwz/nrzcezsnhvexqjtpuhztxahqahdpv, common/sme/zxzjpmcpuchcynknsoouhxckbgtoq, common/nis/fklazmbxyifwepwqgnhmjsmigsool, common/zmg/sibbslhvhaqplxjngeqycfkkqjfki, common/aah/juvxxfyofyihihrkqoacdwzzwiwqk, common/vrg/zaqdxyuzdjqeesfldftewmycatpfx, common/ktl/lwgrxecnwynifnhvrmsjrbbnlkvib, common/gxe/edrlinebswsbajchtlpvmjdovofrn, common/atu/ypvuthjwyjxwevwkvwkfuofwadcvn, common/ghu/kriufxyijiahwirtycjcbpbdqeusn, common/hgj/eheddjvwlqllyvtkvbhvrxrelewms, common/zhn/whxwgzxsjgkcfeqejtqgrgakwzasc, common/pwx/fmfadtvvjejtwlcxrkewqtxnkcusa, common/zzd/flvbezavdfhqqxcunkdkyudbkmwxv, common/gbh/lkcpypncpolfhtdrzwmavgmriqasb, common/gwv/cidbvfmwmnidicsoktvryovymwcxk, common/tuy/xgofymlinzdgswikxkcedlzbvfqfq, common/yhz/ueiaizunikhmvbuxmfbadcmphsxgh, common/gal/wwhsrxdefeajrshuqbsbrrixphprl, common/zok/psyrmlwmyvkdqpzcmjeaqiucpbawx, common/nvg/kaqkvthbrvtdtxwjlgsebjenisghx] were removed from S3 2024.10.22 01:36:15.865271 [ 1418 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_7_7_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 01:36:15.885384 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_467_467_0, total 1 rows starting from the beginning of the part 2024.10.22 01:36:16.010677 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Removing 7 parts from memory: Parts: [201403_1_1_0, 201403_2_2_0, 201403_3_3_0, 201403_4_4_0, 201403_5_5_0, 201403_6_6_0, 201403_7_7_0] 2024.10.22 01:36:16.185683 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_468_468_0, total 1 rows starting from the beginning of the part 2024.10.22 01:36:16.299393 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 65 entries to flush up to offset 64722 2024.10.22 01:36:16.313672 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1329 entries to flush up to offset 854379 2024.10.22 01:36:16.439480 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:16.444732 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:16.500744 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_556_556_0 (state Temporary) 2024.10.22 01:36:16.559141 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_556_556_0 to 202410_562_562_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:16.580406 [ 1214 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_557_10 to 202410_562_562_0 2024.10.22 01:36:16.586357 [ 1214 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:16.592354 [ 1214 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:16.616500 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 64722 2024.10.22 01:36:16.663601 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_557_10 to 202410_562_562_0 into Compact with storage Full 2024.10.22 01:36:16.683711 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a44d6eef407ee51c19666ece288c4ded) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a44d6eef407ee51c19666ece288c4ded, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:36:16.688115 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 246107, file segment info: File segment: [0, 33554431], key: a44d6eef407ee51c19666ece288c4ded, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:36:16.691549 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:36:16.867883 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_114_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:36:16.981035 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:36:17.023655 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_557_10, total 5075 rows starting from the beginning of the part 2024.10.22 01:36:17.077936 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_558_558_0, total 84 rows starting from the beginning of the part 2024.10.22 01:36:17.080729 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:17.090669 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:17.098438 [ 1423 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:36:17.192635 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_559_559_0, total 126 rows starting from the beginning of the part 2024.10.22 01:36:17.270364 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_560_560_0, total 144 rows starting from the beginning of the part 2024.10.22 01:36:17.340211 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_561_561_0, total 103 rows starting from the beginning of the part 2024.10.22 01:36:17.478595 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:17.482587 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:17.494672 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_562_562_0, total 65 rows starting from the beginning of the part 2024.10.22 01:36:17.654209 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_543_543_0 (state Temporary) 2024.10.22 01:36:17.706745 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_543_543_0 to 202410_550_550_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:17.762595 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 854379 2024.10.22 01:36:17.754220 [ 1151 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_545_545_0 to 202410_550_550_0 2024.10.22 01:36:17.798674 [ 1151 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:17.802158 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:17.938866 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_545_545_0 to 202410_550_550_0 into Compact with storage Full 2024.10.22 01:36:17.996045 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 8720 2024.10.22 01:36:18.061248 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(698c991925e310ea37acbce74e6d9224) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 698c991925e310ea37acbce74e6d9224, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:36:18.065287 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 746040, file segment info: File segment: [0, 33554431], key: 698c991925e310ea37acbce74e6d9224, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:36:18.066001 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:36:18.066819 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:36:18.071213 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_545_545_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:36:18.087077 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_546_546_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:36:18.137206 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_547_547_0, total 999 rows starting from the beginning of the part 2024.10.22 01:36:18.191011 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_548_548_0, total 1311 rows starting from the beginning of the part 2024.10.22 01:36:18.297109 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_549_549_0, total 1751 rows starting from the beginning of the part 2024.10.22 01:36:18.346289 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_550_550_0, total 1329 rows starting from the beginning of the part 2024.10.22 01:36:18.386965 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:18.403056 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:18.446465 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_567_567_0 (state Temporary) 2024.10.22 01:36:18.454192 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 10.49 MiB, will set to 1.68 GiB (RSS), difference: -28.30 MiB 2024.10.22 01:36:18.542978 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_567_567_0 to 202410_580_580_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:18.697309 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8720 2024.10.22 01:36:19.034238 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 931 bytes for 957d5a54e3867b9d53cdacf68b37bcd9:0) 2024.10.22 01:36:19.037724 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 86ca81e9de55269de5fd8026c596e716, offset: 0, size: 170 2024.10.22 01:36:19.047656 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 86ca81e9de55269de5fd8026c596e716 for removal 2024.10.22 01:36:19.051924 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a6f6056c72336024d7218ff6aae77be4, offset: 0, size: 150 2024.10.22 01:36:19.051924 [ 1476 ] {} CacheMetadata: Key 86ca81e9de55269de5fd8026c596e716 is removed from metadata 2024.10.22 01:36:19.055636 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a6f6056c72336024d7218ff6aae77be4 for removal 2024.10.22 01:36:19.061941 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 37bc0a3c1fb22fb428228ce7d1280787, offset: 0, size: 4 2024.10.22 01:36:19.062225 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} MergeTask::MergeProjectionsStage: Merge sorted 512 rows, containing 32 columns (32 merged, 0 gathered) in 5.277406815 sec., 97.01734544032873 rows/sec., 8.96 KiB/sec. 2024.10.22 01:36:19.064959 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 37bc0a3c1fb22fb428228ce7d1280787 for removal 2024.10.22 01:36:19.070283 [ 1476 ] {} CacheMetadata: Key 37bc0a3c1fb22fb428228ce7d1280787 is removed from metadata 2024.10.22 01:36:19.078631 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} MergedBlockOutputStream: filled checksums 202410_1_468_96 (state Temporary) 2024.10.22 01:36:19.079254 [ 1476 ] {} CacheMetadata: Key a6f6056c72336024d7218ff6aae77be4 is removed from metadata 2024.10.22 01:36:19.327121 [ 1713 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ojq/hfybufjjavwwigwkpnvhjhphetysk, size 746040 2024.10.22 01:36:19.471524 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_562_11/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:36:19.552603 [ 1255 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 01:36:19.623677 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 63193 2024.10.22 01:36:19.788821 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:36:19.818764 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:36:19.792830 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} MergeTask::MergeProjectionsStage: Merge sorted 8085 rows, containing 4 columns (4 merged, 0 gathered) in 1.976593464 sec., 4090.3707045749898 rows/sec., 63.91 KiB/sec. 2024.10.22 01:36:19.826570 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} MergeTask::MergeProjectionsStage: Merge sorted 5597 rows, containing 13 columns (13 merged, 0 gathered) in 3.168213062 sec., 1766.6109855840245 rows/sec., 783.26 KiB/sec. 2024.10.22 01:36:19.856354 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:36:19.911850 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:36:19.972864 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} MergedBlockOutputStream: filled checksums 202410_545_550_1 (state Temporary) 2024.10.22 01:36:20.029587 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} MergedBlockOutputStream: filled checksums 202410_507_562_11 (state Temporary) 2024.10.22 01:36:20.079158 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:36:20.117299 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:36:20.143736 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:36:20.184977 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:36:20.206348 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_545_550_1 to 202410_545_550_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:20.214822 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_545_550_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_545_545_0, 202410_550_550_0] -> 202410_545_550_1 2024.10.22 01:36:20.245732 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:20.251172 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:20.273074 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:36:20.416354 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:36:20.430985 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:36:20.433733 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fwl/xavtbxrnjphzigvcvahyzelmsnrms (LogSeriesLimiter: on interval from 2024-10-22 01:35:57 to 2024-10-22 01:36:20 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:36:20.434325 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:36:20.443640 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_958_958_0 (state Temporary) 2024.10.22 01:36:20.456589 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_468_96 to 202410_1_468_96 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:20.460128 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_562_11 to 202410_507_562_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:20.462996 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_557_10, 202410_562_562_0] -> 202410_507_562_11 2024.10.22 01:36:20.463389 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_468_96} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_463_95, 202410_468_468_0] -> 202410_1_468_96 2024.10.22 01:36:20.490351 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_958_958_0 to 202410_971_971_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:20.584129 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63193 2024.10.22 01:36:20.524853 [ 1385 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_562_11}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 01:36:20.755434 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.67 MiB. 2024.10.22 01:36:20.763755 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:36:20.842267 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_114_114_0 (state Temporary) 2024.10.22 01:36:20.937132 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 288.12 MiB using storage policy from min volume index 0 2024.10.22 01:36:21.287915 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 477 2024.10.22 01:36:21.358983 [ 1205 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 01:36:21.448088 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 9.84 MiB, will set to 1.69 GiB (RSS), difference: -13.45 MiB 2024.10.22 01:36:21.479745 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 7 entries to flush up to offset 36 2024.10.22 01:36:21.642400 [ 1224 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 01:36:21.704831 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:21.708111 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:21.730993 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:21.734972 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:21.760959 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_5_5_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:36:21.776061 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_30_30_0 (state Temporary) 2024.10.22 01:36:21.952934 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_459_459_0 (state Temporary) 2024.10.22 01:36:22.108696 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(4fea9ec1a2efc15b1457520400adb037) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4fea9ec1a2efc15b1457520400adb037, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:36:22.113356 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 629592, file segment info: File segment: [0, 33554431], key: 4fea9ec1a2efc15b1457520400adb037, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:36:22.118366 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:36:22.165350 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_459_459_0 to 202410_469_469_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:22.175745 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_30_30_0 to 202410_30_30_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:22.222362 [ 1177 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Selected 6 parts from 202410_1_25_6 to 202410_30_30_0 2024.10.22 01:36:22.253394 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 477 2024.10.22 01:36:22.343594 [ 1177 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:22.352375 [ 1177 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:22.368280 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_25_6 to 202410_30_30_0 into Compact with storage Full 2024.10.22 01:36:22.375977 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 36 2024.10.22 01:36:22.473459 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:36:22.605295 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_25_6, total 25 rows starting from the beginning of the part 2024.10.22 01:36:22.766031 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_26_26_0, total 1 rows starting from the beginning of the part 2024.10.22 01:36:22.850897 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_27_27_0, total 1 rows starting from the beginning of the part 2024.10.22 01:36:22.855267 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 571 bytes for caee001a89d03b7cce4a5968828cac21:0) 2024.10.22 01:36:22.859618 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f1286829d9f683f9d2589fb5edea1f4f, offset: 0, size: 171 2024.10.22 01:36:22.865217 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f1286829d9f683f9d2589fb5edea1f4f for removal 2024.10.22 01:36:22.870928 [ 1476 ] {} CacheMetadata: Key f1286829d9f683f9d2589fb5edea1f4f is removed from metadata 2024.10.22 01:36:22.870990 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4cb34bc55aaf6d1deb3ac3a8e1c3ff5c, offset: 0, size: 170 2024.10.22 01:36:22.876168 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4cb34bc55aaf6d1deb3ac3a8e1c3ff5c for removal 2024.10.22 01:36:22.881035 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 15cda58dd867807c94ef505613ce4599, offset: 0, size: 170 2024.10.22 01:36:22.884174 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 15cda58dd867807c94ef505613ce4599 for removal 2024.10.22 01:36:22.913066 [ 1476 ] {} CacheMetadata: Key 15cda58dd867807c94ef505613ce4599 is removed from metadata 2024.10.22 01:36:23.046292 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_28_28_0, total 1 rows starting from the beginning of the part 2024.10.22 01:36:23.059524 [ 1476 ] {} CacheMetadata: Key 4cb34bc55aaf6d1deb3ac3a8e1c3ff5c is removed from metadata 2024.10.22 01:36:23.127276 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_29_29_0, total 1 rows starting from the beginning of the part 2024.10.22 01:36:23.205063 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_30_30_0, total 7 rows starting from the beginning of the part 2024.10.22 01:36:23.608367 [ 1154 ] {} DatabaseCatalog: Execution took 240 ms. 2024.10.22 01:36:23.811923 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_860_438, 202410_861_866_1] 2024.10.22 01:36:23.814625 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_860_438, 202410_861_866_1] 2024.10.22 01:36:23.939763 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_860_438, 202410_861_866_1] 2024.10.22 01:36:24.179207 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(76a95a32353445c6f9853ea0755a2189) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 76a95a32353445c6f9853ea0755a2189, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:36:24.187120 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 37737, file segment info: File segment: [0, 33554431], key: 76a95a32353445c6f9853ea0755a2189, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:36:24.194095 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:36:24.229559 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 114 entries to flush up to offset 64836 2024.10.22 01:36:24.370022 [ 1253 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:36:24.431796 [ 1253 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:36:24.470761 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:24.473288 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:24.608145 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 63279 2024.10.22 01:36:24.750935 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.90 GiB, peak 1.97 GiB, free memory in arenas 11.39 MiB, will set to 1.70 GiB (RSS), difference: -206.68 MiB 2024.10.22 01:36:24.767572 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 855534 2024.10.22 01:36:24.854555 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_557_557_0 (state Temporary) 2024.10.22 01:36:24.869068 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_516_103, 202410_517_517_0, 202410_518_518_0, 202410_519_519_0, 202410_520_520_0, 202410_521_521_0] 2024.10.22 01:36:24.872276 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_516_103, 202410_517_517_0, 202410_518_518_0, 202410_519_519_0, 202410_520_520_0, 202410_521_521_0] 2024.10.22 01:36:24.993002 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_557_557_0 to 202410_563_563_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:25.002897 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_516_103, 202410_517_517_0, 202410_518_518_0, 202410_519_519_0, 202410_520_520_0, 202410_521_521_0] 2024.10.22 01:36:25.075728 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_544_108/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:36:25.107119 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 64836 2024.10.22 01:36:25.111235 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vlb/afnzdpmooofkvcpmgwqaruqjxtzpy (LogSeriesLimiter: on interval from 2024-10-22 01:36:20 to 2024-10-22 01:36:25 accepted series 1 / 149 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:36:25.155515 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:25.157985 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:25.234016 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_959_959_0 (state Temporary) 2024.10.22 01:36:25.314404 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 537 bytes for 9d286bcb27bfb4a2d5d5401c5b38208b:0) 2024.10.22 01:36:25.341382 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3d8ef2593c2a587f499062385a534f22, offset: 0, size: 168 2024.10.22 01:36:25.350858 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3d8ef2593c2a587f499062385a534f22 for removal 2024.10.22 01:36:25.358862 [ 1476 ] {} CacheMetadata: Key 3d8ef2593c2a587f499062385a534f22 is removed from metadata 2024.10.22 01:36:25.360908 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e1158a9a2e6f92c3a617c5d030dd9e26, offset: 0, size: 171 2024.10.22 01:36:25.365240 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e1158a9a2e6f92c3a617c5d030dd9e26 for removal 2024.10.22 01:36:25.375353 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 41c4a61005a279913e07a910a1945639, offset: 0, size: 171 2024.10.22 01:36:25.475337 [ 1476 ] {} CacheMetadata: Key e1158a9a2e6f92c3a617c5d030dd9e26 is removed from metadata 2024.10.22 01:36:25.377820 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 41c4a61005a279913e07a910a1945639 for removal 2024.10.22 01:36:25.480387 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:25.485705 [ 1476 ] {} CacheMetadata: Key 41c4a61005a279913e07a910a1945639 is removed from metadata 2024.10.22 01:36:25.486513 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:25.502154 [ 1385 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} MergeTask::MergeProjectionsStage: Merge sorted 36 rows, containing 27 columns (27 merged, 0 gathered) in 3.136205365 sec., 11.478840130100982 rows/sec., 30.51 KiB/sec. 2024.10.22 01:36:25.522678 [ 1678 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lsq/bzmkkcycmhsmnsnnzehapptpogapa, size 37737 2024.10.22 01:36:25.554287 [ 1390 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} MergedBlockOutputStream: filled checksums 202410_1_30_7 (state Temporary) 2024.10.22 01:36:25.580207 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_959_959_0 to 202410_972_972_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:25.622803 [ 1275 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1_966_480 to 202410_971_971_0 2024.10.22 01:36:25.654260 [ 1275 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.44 MiB using storage policy from min volume index 0 2024.10.22 01:36:25.656389 [ 1275 ] {} DiskLocal: Reserved 2.44 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:25.665699 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_966_480 to 202410_971_971_0 into Compact with storage Full 2024.10.22 01:36:25.693344 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:36:25.712559 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_966_480, total 63757 rows starting from the beginning of the part 2024.10.22 01:36:25.824765 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63279 2024.10.22 01:36:25.828746 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} MergeTreeSequentialSource: Reading 2 marks from part 202410_967_967_0, total 63 rows starting from the beginning of the part 2024.10.22 01:36:25.865667 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} MergeTreeSequentialSource: Reading 2 marks from part 202410_968_968_0, total 64 rows starting from the beginning of the part 2024.10.22 01:36:25.867910 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_544_544_0 (state Temporary) 2024.10.22 01:36:25.989132 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} MergeTreeSequentialSource: Reading 2 marks from part 202410_969_969_0, total 30 rows starting from the beginning of the part 2024.10.22 01:36:26.047633 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} MergeTreeSequentialSource: Reading 2 marks from part 202410_970_970_0, total 30 rows starting from the beginning of the part 2024.10.22 01:36:26.056713 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:36:26.062323 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:36:26.062516 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_merge_202410_1_30_7 to 202410_1_30_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:26.068471 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_544_544_0 to 202410_551_551_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:26.070095 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_30_7} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Merged 6 parts: [202410_1_25_6, 202410_30_30_0] -> 202410_1_30_7 2024.10.22 01:36:26.072161 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:36:26.088406 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:36:26.090565 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} MergeTreeSequentialSource: Reading 2 marks from part 202410_971_971_0, total 80 rows starting from the beginning of the part 2024.10.22 01:36:26.112258 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:36:26.128697 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.24 MiB. 2024.10.22 01:36:26.144581 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:36:26.161701 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:36:26.178321 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 01:36:26.204353 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 35 entries to flush up to offset 8755 2024.10.22 01:36:26.213091 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 855534 2024.10.22 01:36:26.463220 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:26.465638 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:26.506309 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_568_568_0 (state Temporary) 2024.10.22 01:36:26.564128 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_568_568_0 to 202410_581_581_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:26.637588 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8755 2024.10.22 01:36:26.691913 [ 1259 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_576_115 to 202410_581_581_0 2024.10.22 01:36:26.709925 [ 1259 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:26.739347 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:26.760837 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_576_115 to 202410_581_581_0 into Compact with storage Full 2024.10.22 01:36:26.826082 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:36:26.838333 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_576_115, total 8786 rows starting from the beginning of the part 2024.10.22 01:36:26.960420 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_577_577_0, total 13 rows starting from the beginning of the part 2024.10.22 01:36:26.982788 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_578_578_0, total 6 rows starting from the beginning of the part 2024.10.22 01:36:27.015458 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_579_579_0, total 15 rows starting from the beginning of the part 2024.10.22 01:36:27.143576 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_580_580_0, total 6 rows starting from the beginning of the part 2024.10.22 01:36:27.280287 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_581_581_0, total 35 rows starting from the beginning of the part 2024.10.22 01:36:27.309236 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_866_439, 202410_867_867_0] 2024.10.22 01:36:27.314887 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.89 GiB, peak 1.97 GiB, free memory in arenas 9.72 MiB, will set to 1.70 GiB (RSS), difference: -189.72 MiB 2024.10.22 01:36:27.422776 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_866_439, 202410_867_867_0] 2024.10.22 01:36:27.432421 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_866_439, 202410_867_867_0] 2024.10.22 01:36:27.651665 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.101764459 sec. 2024.10.22 01:36:28.152249 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_114_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:36:28.335488 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_544_108} MergeTask::MergeProjectionsStage: Merge sorted 859884 rows, containing 4 columns (4 merged, 0 gathered) in 27.081214816 sec., 31752.046791193698 rows/sec., 476.30 KiB/sec. 2024.10.22 01:36:28.388980 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} MergeTask::MergeProjectionsStage: Merge sorted 8861 rows, containing 12 columns (12 merged, 0 gathered) in 1.635111 sec., 5419.203956184015 rows/sec., 861.56 KiB/sec. 2024.10.22 01:36:28.427689 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} MergedBlockOutputStream: filled checksums 202410_1_581_116 (state Temporary) 2024.10.22 01:36:28.490906 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_544_108} MergedBlockOutputStream: filled checksums 202410_1_544_108 (state Temporary) 2024.10.22 01:36:28.780503 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_581_116 to 202410_1_581_116 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:28.786391 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_581_116} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_576_115, 202410_581_581_0] -> 202410_1_581_116 2024.10.22 01:36:28.833206 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.53 MiB. 2024.10.22 01:36:29.059587 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_114_114_0 to 202410_117_117_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:29.340081 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_971_481/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:36:29.436800 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.86 GiB, peak 1.97 GiB, free memory in arenas 8.33 MiB, will set to 1.70 GiB (RSS), difference: -164.01 MiB 2024.10.22 01:36:29.845223 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 63368 2024.10.22 01:36:30.071236 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:30.176745 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.06 GiB. 2024.10.22 01:36:30.241596 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_960_960_0 (state Temporary) 2024.10.22 01:36:30.328293 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_960_960_0 to 202410_973_973_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:30.405539 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63368 2024.10.22 01:36:30.423929 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:36:30.425431 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:36:30.508618 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} MergeTask::MergeProjectionsStage: Merge sorted 64024 rows, containing 13 columns (13 merged, 0 gathered) in 4.846092978 sec., 13211.467524591932 rows/sec., 2.17 MiB/sec. 2024.10.22 01:36:30.712259 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} MergedBlockOutputStream: filled checksums 202410_1_971_481 (state Temporary) 2024.10.22 01:36:30.866390 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_971_481 to 202410_1_971_481 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:30.869948 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_971_481} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1_966_480, 202410_971_971_0] -> 202410_1_971_481 2024.10.22 01:36:31.037709 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.12 MiB. 2024.10.22 01:36:31.162327 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_544_108} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_544_108 to 202410_1_544_108 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:31.164603 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_544_108} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_538_107, 202410_539_544_1] -> 202410_1_544_108 2024.10.22 01:36:31.180820 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:36:31.611269 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 6.80 MiB, will set to 1.70 GiB (RSS), difference: 7.84 MiB 2024.10.22 01:36:32.563753 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4579 2024.10.22 01:36:32.723295 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 107 entries to flush up to offset 64943 2024.10.22 01:36:33.154277 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:33.156752 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:33.186631 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_558_558_0 (state Temporary) 2024.10.22 01:36:33.260875 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_867_440, 202410_868_868_0] 2024.10.22 01:36:33.263021 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_558_558_0 to 202410_564_564_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:33.263220 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_867_440, 202410_868_868_0] 2024.10.22 01:36:33.275097 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 857074 2024.10.22 01:36:33.291172 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 64943 2024.10.22 01:36:33.373262 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_867_440, 202410_868_868_0] 2024.10.22 01:36:34.128955 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:34.132915 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.06 GiB. 2024.10.22 01:36:34.252856 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 8776 2024.10.22 01:36:34.340752 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:34.422505 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 63399 2024.10.22 01:36:34.444970 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.06 GiB. 2024.10.22 01:36:34.498144 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_569_569_0 (state Temporary) 2024.10.22 01:36:34.507831 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_545_545_0 (state Temporary) 2024.10.22 01:36:34.761448 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_569_569_0 to 202410_582_582_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:34.807421 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_545_545_0 to 202410_552_552_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:34.852738 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 857074 2024.10.22 01:36:34.882384 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:36:34.896090 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:34.905044 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.06 GiB. 2024.10.22 01:36:34.964051 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_961_961_0 (state Temporary) 2024.10.22 01:36:34.991937 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_961_961_0 to 202410_974_974_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:35.078051 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8776 2024.10.22 01:36:35.099913 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63399 2024.10.22 01:36:35.474637 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(228c9e1375ccb068f726f0c52a9ce0fa) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 228c9e1375ccb068f726f0c52a9ce0fa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:36:35.478346 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 228c9e1375ccb068f726f0c52a9ce0fa, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:36:35.483351 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:36:35.651922 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 7.06 MiB, will set to 1.71 GiB (RSS), difference: -1.28 MiB 2024.10.22 01:36:37.257557 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 478 2024.10.22 01:36:37.465559 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 7.59 MiB, will set to 1.71 GiB (RSS), difference: 3.42 MiB 2024.10.22 01:36:37.804390 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:37.810703 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.06 GiB. 2024.10.22 01:36:37.969496 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_460_460_0 (state Temporary) 2024.10.22 01:36:38.159802 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_460_460_0 to 202410_470_470_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:38.183420 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_868_441, 202410_869_869_0] 2024.10.22 01:36:38.189407 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_868_441, 202410_869_869_0] 2024.10.22 01:36:38.216645 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_868_441, 202410_869_869_0] 2024.10.22 01:36:38.372172 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 478 2024.10.22 01:36:38.696760 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_461_501_8, 202410_502_502_0, 202410_503_503_0, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0] 2024.10.22 01:36:38.701934 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_461_501_8, 202410_502_502_0, 202410_503_503_0, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0] 2024.10.22 01:36:38.737526 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_461_501_8, 202410_502_502_0, 202410_503_503_0, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0] 2024.10.22 01:36:39.105652 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 63431 2024.10.22 01:36:39.438890 [ 1224 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:36:39.448816 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 7.53 MiB, will set to 1.72 GiB (RSS), difference: 4.48 MiB 2024.10.22 01:36:39.459730 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_491_491_0, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:36:39.460865 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_491_491_0, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:36:39.468810 [ 1224 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:36:39.480880 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_491_491_0, 202410_492_492_0, 202410_493_493_0, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0] 2024.10.22 01:36:39.533091 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:39.537736 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.06 GiB. 2024.10.22 01:36:39.599690 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_962_962_0 (state Temporary) 2024.10.22 01:36:39.756084 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_962_962_0 to 202410_975_975_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:39.908233 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63431 2024.10.22 01:36:40.087398 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 4619 2024.10.22 01:36:40.420178 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:36:40.424123 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:36:40.806465 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 64993 2024.10.22 01:36:41.238972 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:41.245493 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.06 GiB. 2024.10.22 01:36:41.486084 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_559_559_0 (state Temporary) 2024.10.22 01:36:41.546176 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_559_559_0 to 202410_565_565_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:41.651791 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 64993 2024.10.22 01:36:41.860756 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1813 entries to flush up to offset 858887 2024.10.22 01:36:42.165924 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_869_442, 202410_870_870_0] 2024.10.22 01:36:42.167249 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_869_442, 202410_870_870_0] 2024.10.22 01:36:42.175965 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_869_442, 202410_870_870_0] 2024.10.22 01:36:42.579644 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 8782 2024.10.22 01:36:42.586760 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_114_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:36:42.701211 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:42.704477 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:42.710181 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.06 GiB. 2024.10.22 01:36:42.712364 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.06 GiB. 2024.10.22 01:36:42.756261 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_570_570_0 (state Temporary) 2024.10.22 01:36:42.926760 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_546_546_0 (state Temporary) 2024.10.22 01:36:43.013124 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_570_570_0 to 202410_583_583_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:43.072091 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_546_546_0 to 202410_553_553_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:43.104465 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8782 2024.10.22 01:36:43.146996 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 858887 2024.10.22 01:36:43.474802 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 6.50 MiB, will set to 1.72 GiB (RSS), difference: 72.16 MiB 2024.10.22 01:36:43.712694 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 867 bytes for db7b1102a76f161e10292323ab2faf55:0) 2024.10.22 01:36:43.716791 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3bae7ba094c43c5fbe6f9e28f2c5060a, offset: 0, size: 168 2024.10.22 01:36:43.722694 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3bae7ba094c43c5fbe6f9e28f2c5060a for removal 2024.10.22 01:36:43.727308 [ 1476 ] {} CacheMetadata: Key 3bae7ba094c43c5fbe6f9e28f2c5060a is removed from metadata 2024.10.22 01:36:43.727915 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d5b439b43b288f2f5f698f08f60d3bde, offset: 0, size: 170 2024.10.22 01:36:43.735615 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d5b439b43b288f2f5f698f08f60d3bde for removal 2024.10.22 01:36:43.744830 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c4d1dd41aa005e9f4b2a00949b8ed705, offset: 0, size: 170 2024.10.22 01:36:43.748751 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c4d1dd41aa005e9f4b2a00949b8ed705 for removal 2024.10.22 01:36:43.751775 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0931f15e7d68a6e453808aa007ac2464, offset: 0, size: 170 2024.10.22 01:36:43.754555 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0931f15e7d68a6e453808aa007ac2464 for removal 2024.10.22 01:36:43.755003 [ 1476 ] {} CacheMetadata: Key c4d1dd41aa005e9f4b2a00949b8ed705 is removed from metadata 2024.10.22 01:36:43.757253 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1169ebcca4b3822a78c114a75c373a94, offset: 0, size: 171 2024.10.22 01:36:43.760535 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1169ebcca4b3822a78c114a75c373a94 for removal 2024.10.22 01:36:43.903452 [ 1476 ] {} CacheMetadata: Key 1169ebcca4b3822a78c114a75c373a94 is removed from metadata 2024.10.22 01:36:43.911280 [ 1476 ] {} CacheMetadata: Key 0931f15e7d68a6e453808aa007ac2464 is removed from metadata 2024.10.22 01:36:43.914378 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 63483 2024.10.22 01:36:43.931492 [ 1476 ] {} CacheMetadata: Key d5b439b43b288f2f5f698f08f60d3bde is removed from metadata 2024.10.22 01:36:44.105167 [ 1728 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vlb/afnzdpmooofkvcpmgwqaruqjxtzpy, size 60 2024.10.22 01:36:44.293692 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:36:44.336763 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_484_98, 202410_485_490_1] 2024.10.22 01:36:44.346785 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_484_98, 202410_485_490_1] 2024.10.22 01:36:44.365470 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_484_98, 202410_485_490_1] 2024.10.22 01:36:44.368406 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:44.376557 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.05 GiB. 2024.10.22 01:36:44.490707 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_963_963_0 (state Temporary) 2024.10.22 01:36:44.578540 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_963_963_0 to 202410_976_976_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:44.695207 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:36:44.744002 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63483 2024.10.22 01:36:44.769007 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:36:44.800792 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:36:44.860678 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:36:44.884237 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:36:44.922145 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:36:44.962765 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:36:44.977984 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 01:36:45.154559 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nad/utlnvzdcymrpoyttiuxcgmeubgvwv (LogSeriesLimiter: on interval from 2024-10-22 01:36:25 to 2024-10-22 01:36:45 accepted series 1 / 224 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:36:45.676521 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.97 GiB, free memory in arenas 6.02 MiB, will set to 1.72 GiB (RSS), difference: 162.13 MiB 2024.10.22 01:36:45.873631 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 479 2024.10.22 01:36:46.200356 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:46.216494 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.04 GiB. 2024.10.22 01:36:46.309458 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_461_461_0 (state Temporary) 2024.10.22 01:36:46.497296 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_461_461_0 to 202410_471_471_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:46.624068 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_6_6_0 (state Temporary) 2024.10.22 01:36:46.663601 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1192 bytes for 4997ac6aac5125f34d488a822e401427:0) 2024.10.22 01:36:46.665357 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a2e2d5414588be758eefd7287d8f3798, offset: 0, size: 176 2024.10.22 01:36:46.668946 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a2e2d5414588be758eefd7287d8f3798 for removal 2024.10.22 01:36:46.678413 [ 1476 ] {} CacheMetadata: Key a2e2d5414588be758eefd7287d8f3798 is removed from metadata 2024.10.22 01:36:46.682656 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d8abb53ec755ef205bd85a3ddc073441, offset: 0, size: 167 2024.10.22 01:36:46.687510 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d8abb53ec755ef205bd85a3ddc073441 for removal 2024.10.22 01:36:46.705384 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 479 2024.10.22 01:36:46.713331 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d01d7a6204235d01e2a12d1717d496b3, offset: 0, size: 1560 2024.10.22 01:36:46.719811 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d01d7a6204235d01e2a12d1717d496b3 for removal 2024.10.22 01:36:46.726545 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f43fc8c2c82680715b2e7af3efe67aeb, offset: 0, size: 171 2024.10.22 01:36:46.738741 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f43fc8c2c82680715b2e7af3efe67aeb for removal 2024.10.22 01:36:46.823787 [ 1476 ] {} CacheMetadata: Key f43fc8c2c82680715b2e7af3efe67aeb is removed from metadata 2024.10.22 01:36:46.847773 [ 1476 ] {} CacheMetadata: Key d01d7a6204235d01e2a12d1717d496b3 is removed from metadata 2024.10.22 01:36:46.863334 [ 1476 ] {} CacheMetadata: Key d8abb53ec755ef205bd85a3ddc073441 is removed from metadata 2024.10.22 01:36:47.478927 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_870_443, 202410_871_871_0] 2024.10.22 01:36:47.480559 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_870_443, 202410_871_871_0] 2024.10.22 01:36:47.493083 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_870_443, 202410_871_871_0] 2024.10.22 01:36:47.984045 [ 1183 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 341 ms. 2024.10.22 01:36:48.054999 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b162f2da5992839e249c59b049b0d4e7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b162f2da5992839e249c59b049b0d4e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:36:48.063087 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 6443, file segment info: File segment: [0, 33554431], key: b162f2da5992839e249c59b049b0d4e7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:36:48.065242 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:36:48.394267 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.97 GiB, free memory in arenas 4.69 MiB, will set to 1.70 GiB (RSS), difference: 106.06 MiB 2024.10.22 01:36:48.760238 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 63534 2024.10.22 01:36:49.081560 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 572 bytes for b16410573e98950a9f29d0bffe7f6c71:0) 2024.10.22 01:36:49.119815 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 426eaf6f69db3be9663641d859ee1a11, offset: 0, size: 230 2024.10.22 01:36:49.126730 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 426eaf6f69db3be9663641d859ee1a11 for removal 2024.10.22 01:36:49.133324 [ 1476 ] {} CacheMetadata: Key 426eaf6f69db3be9663641d859ee1a11 is removed from metadata 2024.10.22 01:36:49.134839 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a2fe5f947e4f42052376b655e106b605, offset: 0, size: 217 2024.10.22 01:36:49.143021 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a2fe5f947e4f42052376b655e106b605 for removal 2024.10.22 01:36:49.150376 [ 1476 ] {} CacheMetadata: Key a2fe5f947e4f42052376b655e106b605 is removed from metadata 2024.10.22 01:36:49.156937 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 94 entries to flush up to offset 65087 2024.10.22 01:36:49.229587 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:49.244653 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:49.270402 [ 1685 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fwl/xavtbxrnjphzigvcvahyzelmsnrms, size 6443 2024.10.22 01:36:49.300561 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:49.348245 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:49.376511 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_964_964_0 (state Temporary) 2024.10.22 01:36:49.385189 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_560_560_0 (state Temporary) 2024.10.22 01:36:49.432478 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_560_560_0 to 202410_566_566_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:49.465987 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} MergeTask::MergeProjectionsStage: Merge sorted 4523 rows, containing 679 columns (679 merged, 0 gathered) in 139.699819417 sec., 32.37656296819521 rows/sec., 43.18 KiB/sec. 2024.10.22 01:36:49.472919 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 65087 2024.10.22 01:36:49.552137 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_964_964_0 to 202410_977_977_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:49.582250 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_971_481 to 202410_972_972_0 2024.10.22 01:36:49.618715 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.43 MiB using storage policy from min volume index 0 2024.10.22 01:36:49.637694 [ 1180 ] {} DiskLocal: Reserved 2.43 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:49.670030 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_972_482} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_971_481 to 202410_972_972_0 into Compact with storage Full 2024.10.22 01:36:49.683684 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:36:49.709379 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:36:49.711775 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63534 2024.10.22 01:36:49.940445 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_972_482} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:36:49.942673 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:36:49.963516 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_972_482} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_971_481, total 64024 rows starting from the beginning of the part 2024.10.22 01:36:49.971641 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:36:49.997003 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:36:50.036524 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:36:50.069458 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:36:50.089677 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:36:50.158853 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1267 entries to flush up to offset 860154 2024.10.22 01:36:50.180973 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_972_482} MergeTreeSequentialSource: Reading 2 marks from part 202410_972_972_0, total 86 rows starting from the beginning of the part 2024.10.22 01:36:50.211815 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:36:50.252592 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:36:50.350764 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rpx/joqzudphqrxprngyxsttlkyajtyul (LogSeriesLimiter: on interval from 2024-10-22 01:36:45 to 2024-10-22 01:36:50 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:36:50.421375 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:36:50.423492 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:36:50.608472 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 8788 2024.10.22 01:36:50.673256 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 4.35 MiB, will set to 1.69 GiB (RSS), difference: -6.20 MiB 2024.10.22 01:36:50.753047 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:50.756672 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:50.886425 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_571_571_0 (state Temporary) 2024.10.22 01:36:51.141815 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_571_571_0 to 202410_584_584_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:51.195150 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8788 2024.10.22 01:36:51.486711 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:51.495244 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:51.673021 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_547_547_0 (state Temporary) 2024.10.22 01:36:51.867508 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_547_547_0 to 202410_554_554_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:51.902812 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 860154 2024.10.22 01:36:51.891719 [ 1193 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_544_108 to 202410_545_550_1 2024.10.22 01:36:52.034285 [ 1193 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:52.038728 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:52.283446 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_550_109} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_544_108 to 202410_545_550_1 into Wide with storage Full 2024.10.22 01:36:52.316023 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_550_109} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:36:52.338393 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_550_109} MergeTreeSequentialSource: Reading 106 marks from part 202410_1_544_108, total 859884 rows starting from the beginning of the part 2024.10.22 01:36:52.374939 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} MergedBlockOutputStream: filled checksums 202410_1_114_22 (state Temporary) 2024.10.22 01:36:52.386724 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fwj/surpufytgzkubqwceeneuhoapuuhl. 2024.10.22 01:36:52.618638 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_550_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_545_550_1, total 8085 rows starting from the beginning of the part 2024.10.22 01:36:52.833024 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 7.50 MiB, will set to 1.70 GiB (RSS), difference: -9.58 MiB 2024.10.22 01:36:53.035964 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_550_109/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:36:53.215606 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_871_444, 202410_872_872_0] 2024.10.22 01:36:53.218793 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_871_444, 202410_872_872_0] 2024.10.22 01:36:53.232739 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_871_444, 202410_872_872_0] 2024.10.22 01:36:53.821289 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 63602 2024.10.22 01:36:54.203723 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:54.210410 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:54.213008 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 480 2024.10.22 01:36:54.328808 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_114_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:36:54.400513 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_965_965_0 (state Temporary) 2024.10.22 01:36:54.452007 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_965_965_0 to 202410_978_978_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:54.466157 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_973_973_0 to 202410_978_978_0 2024.10.22 01:36:54.475508 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:54.482508 [ 1243 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:36:54.499511 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63602 2024.10.22 01:36:54.509692 [ 1210 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:54.520814 [ 1243 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:36:54.555923 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:54.560272 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:54.568383 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_973_973_0 to 202410_978_978_0 into Compact with storage Full 2024.10.22 01:36:54.629529 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 10.73 MiB, will set to 1.70 GiB (RSS), difference: 5.18 MiB 2024.10.22 01:36:54.637482 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:36:54.654468 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_973_973_0, total 89 rows starting from the beginning of the part 2024.10.22 01:36:54.673298 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_418_86, 202410_419_419_0, 202410_420_420_0, 202410_421_421_0, 202410_422_422_0, 202410_423_423_0] 2024.10.22 01:36:54.675369 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_418_86, 202410_419_419_0, 202410_420_420_0, 202410_421_421_0, 202410_422_422_0, 202410_423_423_0] 2024.10.22 01:36:54.704655 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_974_974_0, total 31 rows starting from the beginning of the part 2024.10.22 01:36:54.756003 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_462_462_0 (state Temporary) 2024.10.22 01:36:54.775993 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_975_975_0, total 32 rows starting from the beginning of the part 2024.10.22 01:36:54.839256 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_462_462_0 to 202410_472_472_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:54.848044 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_976_976_0, total 52 rows starting from the beginning of the part 2024.10.22 01:36:54.880366 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_972_482} MergeTask::MergeProjectionsStage: Merge sorted 64110 rows, containing 13 columns (13 merged, 0 gathered) in 5.215468715 sec., 12292.279659470645 rows/sec., 2.02 MiB/sec. 2024.10.22 01:36:54.925740 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_418_86, 202410_419_419_0, 202410_420_420_0, 202410_421_421_0, 202410_422_422_0, 202410_423_423_0] 2024.10.22 01:36:54.991692 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 480 2024.10.22 01:36:55.015644 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_977_977_0, total 51 rows starting from the beginning of the part 2024.10.22 01:36:55.107783 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_972_482} MergedBlockOutputStream: filled checksums 202410_1_972_482 (state Temporary) 2024.10.22 01:36:55.162756 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_978_978_0, total 68 rows starting from the beginning of the part 2024.10.22 01:36:55.452103 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_972_482} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_972_482 to 202410_1_972_482 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:55.455848 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_972_482} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_971_481, 202410_972_972_0] -> 202410_1_972_482 2024.10.22 01:36:55.470737 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.83 MiB. 2024.10.22 01:36:56.241094 [ 1215 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 01:36:56.404056 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} MergeTask::MergeProjectionsStage: Merge sorted 323 rows, containing 13 columns (13 merged, 0 gathered) in 1.848494306 sec., 174.73681090148838 rows/sec., 28.15 KiB/sec. 2024.10.22 01:36:56.471355 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} MergedBlockOutputStream: filled checksums 202410_973_978_1 (state Temporary) 2024.10.22 01:36:56.508995 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 12.66 MiB, will set to 1.70 GiB (RSS), difference: 1.54 MiB 2024.10.22 01:36:56.680819 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_973_978_1 to 202410_973_978_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:56.687260 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_973_978_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_973_973_0, 202410_978_978_0] -> 202410_973_978_1 2024.10.22 01:36:56.726675 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:36:56.796415 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a31f15e82f7c611c1971940927d7f5c3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a31f15e82f7c611c1971940927d7f5c3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:36:56.803200 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: a31f15e82f7c611c1971940927d7f5c3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:36:56.808252 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:36:56.983987 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 129 entries to flush up to offset 65216 2024.10.22 01:36:57.011354 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_872_445, 202410_873_873_0] 2024.10.22 01:36:57.014403 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_872_445, 202410_873_873_0] 2024.10.22 01:36:57.126594 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_872_445, 202410_873_873_0] 2024.10.22 01:36:57.298935 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:57.306587 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:57.349453 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_561_561_0 (state Temporary) 2024.10.22 01:36:57.547905 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_561_561_0 to 202410_567_567_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:57.578233 [ 1244 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_562_11 to 202410_567_567_0 2024.10.22 01:36:57.588145 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 65216 2024.10.22 01:36:57.605377 [ 1244 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:57.609743 [ 1244 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:57.864029 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_562_11 to 202410_567_567_0 into Compact with storage Full 2024.10.22 01:36:58.208385 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:36:58.221077 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:58.226812 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:58.227650 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_562_11, total 5597 rows starting from the beginning of the part 2024.10.22 01:36:58.261981 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_563_563_0, total 114 rows starting from the beginning of the part 2024.10.22 01:36:58.282163 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_564_564_0, total 107 rows starting from the beginning of the part 2024.10.22 01:36:58.312649 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_565_565_0, total 50 rows starting from the beginning of the part 2024.10.22 01:36:58.361971 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_566_566_0, total 94 rows starting from the beginning of the part 2024.10.22 01:36:58.390637 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_567_567_0, total 129 rows starting from the beginning of the part 2024.10.22 01:36:58.518643 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 63669 2024.10.22 01:36:58.538485 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 16.28 MiB, will set to 1.70 GiB (RSS), difference: 6.78 MiB 2024.10.22 01:36:58.699908 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 8803 2024.10.22 01:36:58.787938 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:58.792078 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:58.844725 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:36:58.859513 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:36:58.896689 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_572_572_0 (state Temporary) 2024.10.22 01:36:58.910165 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 861694 2024.10.22 01:36:58.961345 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_572_572_0 to 202410_585_585_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:58.967768 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_966_966_0 (state Temporary) 2024.10.22 01:36:59.076320 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8803 2024.10.22 01:36:59.233779 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_966_966_0 to 202410_979_979_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:36:59.420536 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63669 2024.10.22 01:36:59.474373 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} MergeTask::MergeProjectionsStage: Merge sorted 6091 rows, containing 13 columns (13 merged, 0 gathered) in 1.719743744 sec., 3541.8067495525656 rows/sec., 1.53 MiB/sec. 2024.10.22 01:36:59.528951 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 511 bytes for e42bf36701b53a7e30bb4fbe712be3f7:0) 2024.10.22 01:36:59.532119 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7788d5e8380dc1c246adc1b32e1fd688, offset: 0, size: 171 2024.10.22 01:36:59.538686 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7788d5e8380dc1c246adc1b32e1fd688 for removal 2024.10.22 01:36:59.547477 [ 1476 ] {} CacheMetadata: Key 7788d5e8380dc1c246adc1b32e1fd688 is removed from metadata 2024.10.22 01:36:59.558472 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 83d4205fa1f6b17246cb0567e0b6e7ed, offset: 0, size: 167 2024.10.22 01:36:59.575402 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 83d4205fa1f6b17246cb0567e0b6e7ed for removal 2024.10.22 01:36:59.585137 [ 1476 ] {} CacheMetadata: Key 83d4205fa1f6b17246cb0567e0b6e7ed is removed from metadata 2024.10.22 01:36:59.733134 [ 1748 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nad/utlnvzdcymrpoyttiuxcgmeubgvwv, size 1255277 2024.10.22 01:36:59.861088 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} MergedBlockOutputStream: filled checksums 202410_507_567_12 (state Temporary) 2024.10.22 01:36:59.966281 [ 1217 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:37:00.176399 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:00.179066 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:00.265494 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:37:00.300827 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:37:00.374875 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:37:00.404665 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:37:00.423306 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:37:00.440969 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_548_548_0 (state Temporary) 2024.10.22 01:37:00.450385 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:37:00.459809 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:37:00.467815 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_567_12 to 202410_507_567_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:00.475578 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_567_12} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_562_11, 202410_567_567_0] -> 202410_507_567_12 2024.10.22 01:37:00.482809 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:37:00.506068 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:37:00.552470 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_548_548_0 to 202410_555_555_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:00.556674 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 01:37:00.557430 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.13 MiB. 2024.10.22 01:37:00.771329 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ict/mvnvryfdkufeiwdxfzvojtjncocwe (LogSeriesLimiter: on interval from 2024-10-22 01:36:50 to 2024-10-22 01:37:00 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:37:00.808591 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 861694 2024.10.22 01:37:01.726323 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_115_115_0 (state Temporary) 2024.10.22 01:37:02.277503 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_873_446, 202410_874_874_0] 2024.10.22 01:37:02.278624 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_873_446, 202410_874_874_0] 2024.10.22 01:37:02.390245 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_873_446, 202410_874_874_0] 2024.10.22 01:37:02.498317 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 481 2024.10.22 01:37:02.621548 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 15.46 MiB, will set to 1.70 GiB (RSS), difference: -16.86 MiB 2024.10.22 01:37:02.783122 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:02.785160 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:02.970485 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_463_463_0 (state Temporary) 2024.10.22 01:37:03.050320 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_463_463_0 to 202410_473_473_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:03.137404 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 481 2024.10.22 01:37:03.270903 [ 1273 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_468_96 to 202410_473_473_0 2024.10.22 01:37:03.281604 [ 1273 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:03.287295 [ 1273 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:03.292066 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_550_109/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:37:03.299167 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 01:37:03.311195 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_468_96 to 202410_473_473_0 into Compact with storage Full 2024.10.22 01:37:03.437045 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 63730 2024.10.22 01:37:03.438522 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:03.469490 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_468_96, total 512 rows starting from the beginning of the part 2024.10.22 01:37:03.683201 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_469_469_0, total 1 rows starting from the beginning of the part 2024.10.22 01:37:03.757645 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:37:03.787604 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_470_470_0, total 1 rows starting from the beginning of the part 2024.10.22 01:37:03.861044 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:03.863525 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:03.973317 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_471_471_0, total 1 rows starting from the beginning of the part 2024.10.22 01:37:04.021301 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_967_967_0 (state Temporary) 2024.10.22 01:37:04.078227 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_472_472_0, total 1 rows starting from the beginning of the part 2024.10.22 01:37:04.148470 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_521_104, 202410_522_522_0, 202410_523_523_0, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0] 2024.10.22 01:37:04.155860 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_521_104, 202410_522_522_0, 202410_523_523_0, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0] 2024.10.22 01:37:04.210783 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_521_104, 202410_522_522_0, 202410_523_523_0, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0] 2024.10.22 01:37:04.266487 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_473_473_0, total 1 rows starting from the beginning of the part 2024.10.22 01:37:04.284708 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_967_967_0 to 202410_980_980_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:04.352466 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63730 2024.10.22 01:37:04.457017 [ 1251 ] {} DatabaseCatalog: Execution took 429 ms. 2024.10.22 01:37:04.559370 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 17.60 MiB, will set to 1.70 GiB (RSS), difference: 2.76 MiB 2024.10.22 01:37:05.119954 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 86 entries to flush up to offset 65302 2024.10.22 01:37:05.442732 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:05.449151 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:05.484939 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_562_562_0 (state Temporary) 2024.10.22 01:37:05.658506 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_562_562_0 to 202410_568_568_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:05.707022 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 65302 2024.10.22 01:37:06.007450 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} MergeTask::MergeProjectionsStage: Merge sorted 517 rows, containing 32 columns (32 merged, 0 gathered) in 2.703494247 sec., 191.2339930346447 rows/sec., 17.64 KiB/sec. 2024.10.22 01:37:06.058437 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} MergedBlockOutputStream: filled checksums 202410_1_473_97 (state Temporary) 2024.10.22 01:37:06.578466 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 8816 2024.10.22 01:37:06.652978 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 15.75 MiB, will set to 1.71 GiB (RSS), difference: -4.16 MiB 2024.10.22 01:37:06.796350 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_874_447, 202410_875_875_0] 2024.10.22 01:37:06.803798 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_874_447, 202410_875_875_0] 2024.10.22 01:37:06.817459 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_874_447, 202410_875_875_0] 2024.10.22 01:37:06.967126 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_473_97 to 202410_1_473_97 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:06.971683 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_473_97} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_468_96, 202410_473_473_0] -> 202410_1_473_97 2024.10.22 01:37:07.022622 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:07.025289 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:07.033091 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:37:07.166535 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_573_573_0 (state Temporary) 2024.10.22 01:37:07.214338 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_573_573_0 to 202410_586_586_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:07.263719 [ 1179 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_581_116 to 202410_586_586_0 2024.10.22 01:37:07.272908 [ 1179 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:07.276779 [ 1179 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:07.302778 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8816 2024.10.22 01:37:07.418531 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_581_116 to 202410_586_586_0 into Compact with storage Full 2024.10.22 01:37:07.481577 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:07.508589 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_581_116, total 8861 rows starting from the beginning of the part 2024.10.22 01:37:07.816122 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 863234 2024.10.22 01:37:07.980402 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_582_582_0, total 21 rows starting from the beginning of the part 2024.10.22 01:37:08.164619 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_583_583_0, total 6 rows starting from the beginning of the part 2024.10.22 01:37:08.209806 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_584_584_0, total 6 rows starting from the beginning of the part 2024.10.22 01:37:08.279743 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_585_585_0, total 15 rows starting from the beginning of the part 2024.10.22 01:37:08.340831 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_586_586_0, total 13 rows starting from the beginning of the part 2024.10.22 01:37:08.342771 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 12.14 MiB, will set to 1.70 GiB (RSS), difference: 2.74 MiB 2024.10.22 01:37:08.366853 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 63783 2024.10.22 01:37:08.951820 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:08.955902 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:09.018793 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_968_968_0 (state Temporary) 2024.10.22 01:37:09.070650 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_968_968_0 to 202410_981_981_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:09.106483 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:09.111073 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:09.151544 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63783 2024.10.22 01:37:09.293023 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_586_117/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:37:09.462673 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_549_549_0 (state Temporary) 2024.10.22 01:37:09.528635 [ 1196 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:37:09.557179 [ 1196 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:37:09.570372 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_549_549_0 to 202410_556_556_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:09.575890 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} MergeTask::MergeProjectionsStage: Merge sorted 8922 rows, containing 12 columns (12 merged, 0 gathered) in 2.267575385 sec., 3934.5990695696314 rows/sec., 625.46 KiB/sec. 2024.10.22 01:37:09.617630 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} MergedBlockOutputStream: filled checksums 202410_1_586_117 (state Temporary) 2024.10.22 01:37:09.623753 [ 1186 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_551_551_0 to 202410_556_556_0 2024.10.22 01:37:09.759321 [ 1186 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:09.764008 [ 1186 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:09.778880 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_551_551_0 to 202410_556_556_0 into Compact with storage Full 2024.10.22 01:37:09.823960 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 863234 2024.10.22 01:37:09.920550 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:09.971530 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_586_117 to 202410_1_586_117 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:09.984772 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_586_117} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_581_116, 202410_586_586_0] -> 202410_1_586_117 2024.10.22 01:37:10.029490 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.53 MiB. 2024.10.22 01:37:10.033418 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_551_551_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:37:10.060245 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_552_552_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:37:10.096248 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_553_553_0, total 1813 rows starting from the beginning of the part 2024.10.22 01:37:10.148417 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_554_554_0, total 1267 rows starting from the beginning of the part 2024.10.22 01:37:10.180211 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_555_555_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:37:10.224004 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_556_556_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:37:10.424152 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:37:10.427339 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:37:10.542462 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 9.07 MiB, will set to 1.70 GiB (RSS), difference: -9.81 MiB 2024.10.22 01:37:10.641347 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 482 2024.10.22 01:37:10.719688 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_115_115_0 to 202410_118_118_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:10.919106 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_114_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:37:11.125650 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:11.132120 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:11.213549 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_464_464_0 (state Temporary) 2024.10.22 01:37:11.322918 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} MergeTask::MergeProjectionsStage: Merge sorted 8855 rows, containing 4 columns (4 merged, 0 gathered) in 1.553264689 sec., 5700.895708703 rows/sec., 89.08 KiB/sec. 2024.10.22 01:37:11.414892 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_464_464_0 to 202410_474_474_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:11.495399 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} MergedBlockOutputStream: filled checksums 202410_551_556_1 (state Temporary) 2024.10.22 01:37:11.512875 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 482 2024.10.22 01:37:11.664085 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_551_556_1 to 202410_551_556_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:11.764597 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:37:11.772182 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_551_556_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_551_551_0, 202410_556_556_0] -> 202410_551_556_1 2024.10.22 01:37:11.942780 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:37:12.137026 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_875_448, 202410_876_876_0] 2024.10.22 01:37:12.148203 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_875_448, 202410_876_876_0] 2024.10.22 01:37:12.265160 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_875_448, 202410_876_876_0] 2024.10.22 01:37:12.509635 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 7.04 MiB, will set to 1.70 GiB (RSS), difference: 11.40 MiB 2024.10.22 01:37:13.107173 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4619 2024.10.22 01:37:13.160428 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 63835 2024.10.22 01:37:13.241072 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_114_22 to 202410_1_114_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:13.244028 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 97 entries to flush up to offset 65399 2024.10.22 01:37:13.247627 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_114_22} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 7 parts: [202410_1_104_21, 202410_114_114_0] -> 202410_1_114_22 2024.10.22 01:37:13.444391 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:13.453372 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:13.496795 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:13.501769 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:13.644546 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_969_969_0 (state Temporary) 2024.10.22 01:37:13.669147 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_563_563_0 (state Temporary) 2024.10.22 01:37:13.701623 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_969_969_0 to 202410_982_982_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:13.803509 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63835 2024.10.22 01:37:13.823149 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_563_563_0 to 202410_569_569_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:13.970597 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 65399 2024.10.22 01:37:14.114199 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 45.53 MiB. 2024.10.22 01:37:14.206068 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_550_109/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:37:14.653396 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 14.07 MiB, will set to 1.70 GiB (RSS), difference: 7.32 MiB 2024.10.22 01:37:14.807848 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 36 entries to flush up to offset 8852 2024.10.22 01:37:15.109496 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:15.115675 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:15.171726 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_574_574_0 (state Temporary) 2024.10.22 01:37:15.229187 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_574_574_0 to 202410_587_587_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:15.265166 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8852 2024.10.22 01:37:15.288741 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(26335d5b4dcead96aa0d86894252461f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 26335d5b4dcead96aa0d86894252461f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:37:15.295188 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 11395, file segment info: File segment: [0, 33554431], key: 26335d5b4dcead96aa0d86894252461f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:37:15.303389 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:16.136747 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(e83736d078e3073fa5d4371d90162ff4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e83736d078e3073fa5d4371d90162ff4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:37:16.164125 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 670068, file segment info: File segment: [0, 33554431], key: e83736d078e3073fa5d4371d90162ff4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:37:16.166479 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:16.767285 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_550_109} MergeTask::MergeProjectionsStage: Merge sorted 867969 rows, containing 4 columns (4 merged, 0 gathered) in 24.601630383 sec., 35280.95441185785 rows/sec., 529.15 KiB/sec. 2024.10.22 01:37:16.827747 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_550_109} MergedBlockOutputStream: filled checksums 202410_1_550_109 (state Temporary) 2024.10.22 01:37:16.831258 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1321 entries to flush up to offset 864555 2024.10.22 01:37:17.066418 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(50d122ca54e36f39a94b0127142bc915) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 50d122ca54e36f39a94b0127142bc915, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:37:17.068820 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 555, file segment info: File segment: [0, 33554431], key: 50d122ca54e36f39a94b0127142bc915, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:37:17.070411 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:17.198511 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_876_449, 202410_877_877_0] 2024.10.22 01:37:17.203535 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_876_449, 202410_877_877_0] 2024.10.22 01:37:17.215995 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_876_449, 202410_877_877_0] 2024.10.22 01:37:17.810618 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 63873 2024.10.22 01:37:17.880862 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(257321d5e7af4e1f6c3b8aa0195f5918) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 257321d5e7af4e1f6c3b8aa0195f5918, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:37:17.891118 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 633, file segment info: File segment: [0, 33554431], key: 257321d5e7af4e1f6c3b8aa0195f5918, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:37:17.894965 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:17.922392 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:17.931918 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:17.939756 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:17.941087 [ 1732 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ict/mvnvryfdkufeiwdxfzvojtjncocwe, size 670068 2024.10.22 01:37:17.947939 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:18.086994 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_970_970_0 (state Temporary) 2024.10.22 01:37:18.150319 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_970_970_0 to 202410_983_983_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:18.167955 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_972_482 to 202410_973_978_1 2024.10.22 01:37:18.171002 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.44 MiB using storage policy from min volume index 0 2024.10.22 01:37:18.172331 [ 1241 ] {} DiskLocal: Reserved 2.44 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:18.172843 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:37:18.179868 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63873 2024.10.22 01:37:18.188960 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_978_483} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_972_482 to 202410_973_978_1 into Compact with storage Full 2024.10.22 01:37:18.190475 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:37:18.226842 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:37:18.272112 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_550_550_0 (state Temporary) 2024.10.22 01:37:18.282939 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:37:18.412947 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:37:18.436448 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:37:18.455932 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_550_550_0 to 202410_557_557_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:18.457326 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:37:18.470353 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 01:37:18.486558 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 864555 2024.10.22 01:37:18.492610 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/upa/hcgstehqtpbqdvioznvkinnrruoat (LogSeriesLimiter: on interval from 2024-10-22 01:37:00 to 2024-10-22 01:37:18 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:37:18.516558 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 14.21 MiB, will set to 1.69 GiB (RSS), difference: 10.24 MiB 2024.10.22 01:37:18.560168 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_978_483} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:18.682310 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_978_483} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_972_482, total 64110 rows starting from the beginning of the part 2024.10.22 01:37:18.758232 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_978_483} MergeTreeSequentialSource: Reading 2 marks from part 202410_973_978_1, total 323 rows starting from the beginning of the part 2024.10.22 01:37:18.999950 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ef41ada3f15ab6a65c05ae70181a09a7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ef41ada3f15ab6a65c05ae70181a09a7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:37:19.004460 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 6006, file segment info: File segment: [0, 33554431], key: ef41ada3f15ab6a65c05ae70181a09a7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:37:19.007262 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:19.552716 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_550_109} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_550_109 to 202410_1_550_109 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:19.564014 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_550_109} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_544_108, 202410_545_550_1] -> 202410_1_550_109 2024.10.22 01:37:19.584298 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_878_878_0, 202410_879_879_0, 202410_880_880_0, 202410_881_881_0, 202410_882_882_0, 202410_883_883_0] 2024.10.22 01:37:19.584775 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:37:19.586514 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_878_878_0, 202410_879_879_0, 202410_880_880_0, 202410_881_881_0, 202410_882_882_0, 202410_883_883_0] 2024.10.22 01:37:19.717067 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_878_878_0, 202410_879_879_0, 202410_880_880_0, 202410_881_881_0, 202410_882_882_0, 202410_883_883_0] 2024.10.22 01:37:19.937101 [ 1204 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 247 ms. 2024.10.22 01:37:20.018605 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(db95e82f6610c50c8010750e16083aa3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: db95e82f6610c50c8010750e16083aa3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:37:20.022993 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 573, file segment info: File segment: [0, 33554431], key: db95e82f6610c50c8010750e16083aa3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:37:20.027997 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:20.071171 [ 1708 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rpx/joqzudphqrxprngyxsttlkyajtyul, size 6006 2024.10.22 01:37:20.176015 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:37:20.255524 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:37:20.268625 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:37:20.427873 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:37:20.430238 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:37:20.443906 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:37:20.564141 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:37:20.649097 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:37:20.647245 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 4659 2024.10.22 01:37:20.692952 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:37:20.754399 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:37:20.753523 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 14.34 MiB, will set to 1.70 GiB (RSS), difference: -13.32 MiB 2024.10.22 01:37:20.790350 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:37:21.481804 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 101 entries to flush up to offset 65500 2024.10.22 01:37:21.747208 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:21.752479 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:21.759652 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_978_483} MergeTask::MergeProjectionsStage: Merge sorted 64433 rows, containing 13 columns (13 merged, 0 gathered) in 3.583819687 sec., 17978.862115670945 rows/sec., 2.96 MiB/sec. 2024.10.22 01:37:21.791260 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_564_564_0 (state Temporary) 2024.10.22 01:37:21.858469 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_564_564_0 to 202410_570_570_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:21.908404 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 65500 2024.10.22 01:37:21.929087 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_978_483} MergedBlockOutputStream: filled checksums 202410_1_978_483 (state Temporary) 2024.10.22 01:37:21.981756 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_978_483} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_978_483 to 202410_1_978_483 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:21.986415 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_978_483} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_972_482, 202410_973_978_1] -> 202410_1_978_483 2024.10.22 01:37:22.013094 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fwj/surpufytgzkubqwceeneuhoapuuhl. 2024.10.22 01:37:22.030858 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 01:37:22.288095 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 63944 2024.10.22 01:37:22.458979 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 11.94 MiB, will set to 1.70 GiB (RSS), difference: 43.19 MiB 2024.10.22 01:37:22.546880 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:22.552432 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:22.688470 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_971_971_0 (state Temporary) 2024.10.22 01:37:22.751195 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_971_971_0 to 202410_984_984_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:22.769772 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 8863 2024.10.22 01:37:22.802304 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63944 2024.10.22 01:37:22.867616 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:22.875820 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:22.893567 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_978_483 to 202410_979_979_0 2024.10.22 01:37:22.902461 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.44 MiB using storage policy from min volume index 0 2024.10.22 01:37:22.910547 [ 1278 ] {} DiskLocal: Reserved 2.44 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:22.915560 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_575_575_0 (state Temporary) 2024.10.22 01:37:22.963566 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_979_484} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_978_483 to 202410_979_979_0 into Compact with storage Full 2024.10.22 01:37:22.980166 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_979_484} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:23.062796 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_575_575_0 to 202410_588_588_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:23.104621 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8863 2024.10.22 01:37:23.139116 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_979_484} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_978_483, total 64433 rows starting from the beginning of the part 2024.10.22 01:37:23.193504 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_979_484} MergeTreeSequentialSource: Reading 2 marks from part 202410_979_979_0, total 67 rows starting from the beginning of the part 2024.10.22 01:37:23.498595 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 5.81 MiB, will set to 1.69 GiB (RSS), difference: 38.25 MiB 2024.10.22 01:37:24.533161 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 5.79 MiB, will set to 1.69 GiB (RSS), difference: 8.67 MiB 2024.10.22 01:37:24.563168 [ 1166 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:37:24.612537 [ 1166 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:37:25.494841 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2120 entries to flush up to offset 866675 2024.10.22 01:37:25.627407 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_460_233, 202410_461_506_9] 2024.10.22 01:37:25.631379 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_460_233, 202410_461_506_9] 2024.10.22 01:37:25.653453 [ 1277 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 262 ms. 2024.10.22 01:37:25.782916 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_460_233, 202410_461_506_9] 2024.10.22 01:37:26.235653 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_979_484} MergeTask::MergeProjectionsStage: Merge sorted 64500 rows, containing 13 columns (13 merged, 0 gathered) in 3.278881116 sec., 19671.34449774909 rows/sec., 3.24 MiB/sec. 2024.10.22 01:37:26.399076 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:26.503832 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:26.515512 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 483 2024.10.22 01:37:26.515551 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.97 GiB, free memory in arenas 6.81 MiB, will set to 1.68 GiB (RSS), difference: 77.13 MiB 2024.10.22 01:37:26.631678 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_979_484} MergedBlockOutputStream: filled checksums 202410_1_979_484 (state Temporary) 2024.10.22 01:37:26.725178 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_551_551_0 (state Temporary) 2024.10.22 01:37:26.828502 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_551_551_0 to 202410_558_558_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:26.931982 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 63980 2024.10.22 01:37:26.939174 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:26.945364 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:26.974359 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 866675 2024.10.22 01:37:26.986683 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_979_484} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_979_484 to 202410_1_979_484 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:26.992416 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_979_484} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_978_483, 202410_979_979_0] -> 202410_1_979_484 2024.10.22 01:37:27.029724 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:37:27.069439 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(387c01fe4621a9131a769eea429b3cb2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 387c01fe4621a9131a769eea429b3cb2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:37:27.074909 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 387c01fe4621a9131a769eea429b3cb2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:37:27.079912 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:27.110826 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_465_465_0 (state Temporary) 2024.10.22 01:37:27.283041 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:27.287022 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:27.441608 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_972_972_0 (state Temporary) 2024.10.22 01:37:27.588213 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_972_972_0 to 202410_985_985_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:27.732216 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_465_465_0 to 202410_475_475_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:27.738809 [ 1260 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_979_484 to 202410_980_980_0 2024.10.22 01:37:27.749322 [ 1260 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.44 MiB using storage policy from min volume index 0 2024.10.22 01:37:27.752467 [ 1260 ] {} DiskLocal: Reserved 2.44 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:27.762495 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 63980 2024.10.22 01:37:27.785718 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 483 2024.10.22 01:37:27.875702 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_980_485} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_979_484 to 202410_980_980_0 into Compact with storage Full 2024.10.22 01:37:28.029255 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(933f52495b74a93ae56ea443cf6c6d3c) : [0, 194]: Setting download as failed: File segment: [0, 194], key: 933f52495b74a93ae56ea443cf6c6d3c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:28.032515 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(933f52495b74a93ae56ea443cf6c6d3c): No space left in cache to reserve 195 bytes, will continue without cache download 2024.10.22 01:37:28.123714 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_980_485} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:28.144908 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_980_485} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_979_484, total 64500 rows starting from the beginning of the part 2024.10.22 01:37:28.180085 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_980_485} MergeTreeSequentialSource: Reading 2 marks from part 202410_980_980_0, total 61 rows starting from the beginning of the part 2024.10.22 01:37:28.728013 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e47d546a0857939f27b1856be0076454) : [0, 2040359]: Setting download as failed: File segment: [0, 2040359], key: e47d546a0857939f27b1856be0076454, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:28.732673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e47d546a0857939f27b1856be0076454): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:37:28.776868 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e47d546a0857939f27b1856be0076454, offset: 0, size: 0 2024.10.22 01:37:28.779677 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e47d546a0857939f27b1856be0076454 for removal 2024.10.22 01:37:28.781203 [ 1476 ] {} CacheMetadata: Key e47d546a0857939f27b1856be0076454 is removed from metadata 2024.10.22 01:37:28.783845 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 933f52495b74a93ae56ea443cf6c6d3c, offset: 0, size: 0 2024.10.22 01:37:28.785048 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 933f52495b74a93ae56ea443cf6c6d3c for removal 2024.10.22 01:37:28.786439 [ 1476 ] {} CacheMetadata: Key 933f52495b74a93ae56ea443cf6c6d3c is removed from metadata 2024.10.22 01:37:29.426927 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 98 entries to flush up to offset 65598 2024.10.22 01:37:29.715067 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_507_0, 202410_508_508_0, 202410_509_509_0, 202410_510_510_0, 202410_511_511_0, 202410_512_512_0] 2024.10.22 01:37:29.723373 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_507_0, 202410_508_508_0, 202410_509_509_0, 202410_510_510_0, 202410_511_511_0, 202410_512_512_0] 2024.10.22 01:37:29.767382 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_507_0, 202410_508_508_0, 202410_509_509_0, 202410_510_510_0, 202410_511_511_0, 202410_512_512_0] 2024.10.22 01:37:29.815149 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 6.08 MiB, will set to 1.69 GiB (RSS), difference: 10.42 MiB 2024.10.22 01:37:29.889413 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:29.899733 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:29.967482 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_565_565_0 (state Temporary) 2024.10.22 01:37:30.009126 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_565_565_0 to 202410_571_571_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:30.074225 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 65598 2024.10.22 01:37:30.149548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for c565adbd9eaffe1631b261c039170374:0) 2024.10.22 01:37:30.152686 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 593d02781897d58b08cebddab852294e, offset: 0, size: 178 2024.10.22 01:37:30.156017 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 593d02781897d58b08cebddab852294e for removal 2024.10.22 01:37:30.159217 [ 1476 ] {} CacheMetadata: Key 593d02781897d58b08cebddab852294e is removed from metadata 2024.10.22 01:37:30.291573 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_497_497_0, 202410_498_498_0, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0, 202410_502_502_0] 2024.10.22 01:37:30.306315 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_497_497_0, 202410_498_498_0, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0, 202410_502_502_0] 2024.10.22 01:37:30.330140 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_497_497_0, 202410_498_498_0, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0, 202410_502_502_0] 2024.10.22 01:37:30.428217 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:37:30.432615 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:37:30.607097 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 8871 2024.10.22 01:37:30.820511 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:30.826601 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:30.863616 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_576_576_0 (state Temporary) 2024.10.22 01:37:31.019137 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_980_485} MergeTask::MergeProjectionsStage: Merge sorted 64561 rows, containing 13 columns (13 merged, 0 gathered) in 3.147618686 sec., 20511.06135795764 rows/sec., 3.38 MiB/sec. 2024.10.22 01:37:31.229192 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_576_576_0 to 202410_589_589_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:31.349072 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6edcfa83b7bcfee960eee889dc04faf7) : [0, 39250]: Setting download as failed: File segment: [0, 39250], key: 6edcfa83b7bcfee960eee889dc04faf7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:31.352506 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6edcfa83b7bcfee960eee889dc04faf7): No space left in cache to reserve 39251 bytes, will continue without cache download 2024.10.22 01:37:31.355296 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_980_485} MergedBlockOutputStream: filled checksums 202410_1_980_485 (state Temporary) 2024.10.22 01:37:31.363075 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8871 2024.10.22 01:37:31.372346 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6edcfa83b7bcfee960eee889dc04faf7, offset: 0, size: 0 2024.10.22 01:37:31.373321 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6edcfa83b7bcfee960eee889dc04faf7 for removal 2024.10.22 01:37:31.375367 [ 1476 ] {} CacheMetadata: Key 6edcfa83b7bcfee960eee889dc04faf7 is removed from metadata 2024.10.22 01:37:31.483744 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 3.80 MiB, will set to 1.68 GiB (RSS), difference: 1.80 MiB 2024.10.22 01:37:31.574557 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_980_485} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_980_485 to 202410_1_980_485 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:31.577043 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_980_485} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_979_484, 202410_980_980_0] -> 202410_1_980_485 2024.10.22 01:37:31.599837 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:37:31.767250 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 64052 2024.10.22 01:37:31.988210 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:37:32.489378 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:32.496592 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:32.553746 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_973_973_0 (state Temporary) 2024.10.22 01:37:32.567608 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 154 bytes for b2bd781635d1a78c12a65c408004577e:0) 2024.10.22 01:37:32.569253 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c565adbd9eaffe1631b261c039170374, offset: 0, size: 144 2024.10.22 01:37:32.571540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c565adbd9eaffe1631b261c039170374 for removal 2024.10.22 01:37:32.575413 [ 1476 ] {} CacheMetadata: Key c565adbd9eaffe1631b261c039170374 is removed from metadata 2024.10.22 01:37:32.906754 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_973_973_0 to 202410_986_986_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:32.961500 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_980_485 to 202410_981_981_0 2024.10.22 01:37:32.968432 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.44 MiB using storage policy from min volume index 0 2024.10.22 01:37:32.970555 [ 1235 ] {} DiskLocal: Reserved 2.44 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:32.974305 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64052 2024.10.22 01:37:33.022620 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_981_486} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_980_485 to 202410_981_981_0 into Compact with storage Full 2024.10.22 01:37:33.079509 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_981_486} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:33.087476 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_981_486} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_980_485, total 64561 rows starting from the beginning of the part 2024.10.22 01:37:33.121458 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_981_486} MergeTreeSequentialSource: Reading 2 marks from part 202410_981_981_0, total 53 rows starting from the beginning of the part 2024.10.22 01:37:33.466284 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(133e75ced2dff0c6e62859667d348bb0) : [0, 2279]: Setting download as failed: File segment: [0, 2279], key: 133e75ced2dff0c6e62859667d348bb0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:33.467719 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(133e75ced2dff0c6e62859667d348bb0): No space left in cache to reserve 2280 bytes, will continue without cache download 2024.10.22 01:37:33.611152 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 133e75ced2dff0c6e62859667d348bb0, offset: 0, size: 0 2024.10.22 01:37:33.612574 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 133e75ced2dff0c6e62859667d348bb0 for removal 2024.10.22 01:37:33.617783 [ 1476 ] {} CacheMetadata: Key 133e75ced2dff0c6e62859667d348bb0 is removed from metadata 2024.10.22 01:37:34.003461 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1564 entries to flush up to offset 868239 2024.10.22 01:37:34.171636 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_981_486/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:37:34.452292 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 3.96 MiB, will set to 1.68 GiB (RSS), difference: -6.58 MiB 2024.10.22 01:37:34.654807 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:34.662536 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:34.671785 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 157 bytes for 2d3bcd9b34b96793e28bee25093160d4:0) 2024.10.22 01:37:34.678166 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b2bd781635d1a78c12a65c408004577e, offset: 0, size: 154 2024.10.22 01:37:34.680668 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b2bd781635d1a78c12a65c408004577e for removal 2024.10.22 01:37:34.685654 [ 1476 ] {} CacheMetadata: Key b2bd781635d1a78c12a65c408004577e is removed from metadata 2024.10.22 01:37:34.786343 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_552_552_0 (state Temporary) 2024.10.22 01:37:34.979330 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_552_552_0 to 202410_559_559_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:35.026208 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 868239 2024.10.22 01:37:35.092619 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_490_99, 202410_491_496_1] 2024.10.22 01:37:35.099999 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_490_99, 202410_491_496_1] 2024.10.22 01:37:35.113751 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_490_99, 202410_491_496_1] 2024.10.22 01:37:35.287413 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 484 2024.10.22 01:37:35.800378 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:35.805298 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:35.828696 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e259cf32b6cf578de8f91e3ecc664a6c) : [0, 988860]: Setting download as failed: File segment: [0, 988860], key: e259cf32b6cf578de8f91e3ecc664a6c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:35.835663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e259cf32b6cf578de8f91e3ecc664a6c): No space left in cache to reserve 988861 bytes, will continue without cache download 2024.10.22 01:37:35.869689 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_466_466_0 (state Temporary) 2024.10.22 01:37:35.965403 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e259cf32b6cf578de8f91e3ecc664a6c, offset: 0, size: 0 2024.10.22 01:37:35.967341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e259cf32b6cf578de8f91e3ecc664a6c for removal 2024.10.22 01:37:35.970408 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_466_466_0 to 202410_476_476_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:35.984681 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_981_486} MergeTask::MergeProjectionsStage: Merge sorted 64614 rows, containing 13 columns (13 merged, 0 gathered) in 2.977857716 sec., 21698.148858096753 rows/sec., 3.57 MiB/sec. 2024.10.22 01:37:36.071247 [ 1476 ] {} CacheMetadata: Key e259cf32b6cf578de8f91e3ecc664a6c is removed from metadata 2024.10.22 01:37:36.079821 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_981_486} MergedBlockOutputStream: filled checksums 202410_1_981_486 (state Temporary) 2024.10.22 01:37:36.168821 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 484 2024.10.22 01:37:36.371581 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_981_486} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_981_486 to 202410_1_981_486 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:36.418994 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 3.99 MiB, will set to 1.68 GiB (RSS), difference: 9.61 MiB 2024.10.22 01:37:36.474822 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_981_486} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_980_485, 202410_981_981_0] -> 202410_1_981_486 2024.10.22 01:37:36.597711 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:37:36.979246 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 64107 2024.10.22 01:37:37.177635 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 154 bytes for ed0cf3c4282e83499e81d054f2654814:0) 2024.10.22 01:37:37.181078 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2d3bcd9b34b96793e28bee25093160d4, offset: 0, size: 157 2024.10.22 01:37:37.183375 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2d3bcd9b34b96793e28bee25093160d4 for removal 2024.10.22 01:37:37.184894 [ 1476 ] {} CacheMetadata: Key 2d3bcd9b34b96793e28bee25093160d4 is removed from metadata 2024.10.22 01:37:37.380313 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:37.382158 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:37.427125 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_974_974_0 (state Temporary) 2024.10.22 01:37:37.581216 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 73 entries to flush up to offset 65671 2024.10.22 01:37:37.620447 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:37.635336 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:37.770204 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_974_974_0 to 202410_987_987_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:37.810943 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_981_486 to 202410_982_982_0 2024.10.22 01:37:37.820493 [ 1198 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.44 MiB using storage policy from min volume index 0 2024.10.22 01:37:37.823268 [ 1198 ] {} DiskLocal: Reserved 2.44 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:37.863212 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9e0fd7bc42751898699816fb0fb95c31) : [0, 5311]: Setting download as failed: File segment: [0, 5311], key: 9e0fd7bc42751898699816fb0fb95c31, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:37.864795 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9e0fd7bc42751898699816fb0fb95c31): No space left in cache to reserve 5312 bytes, will continue without cache download 2024.10.22 01:37:37.870673 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64107 2024.10.22 01:37:37.873573 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9e0fd7bc42751898699816fb0fb95c31, offset: 0, size: 0 2024.10.22 01:37:37.874822 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9e0fd7bc42751898699816fb0fb95c31 for removal 2024.10.22 01:37:37.876889 [ 1476 ] {} CacheMetadata: Key 9e0fd7bc42751898699816fb0fb95c31 is removed from metadata 2024.10.22 01:37:37.884626 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_982_487} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_981_486 to 202410_982_982_0 into Compact with storage Full 2024.10.22 01:37:37.887902 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:37.892464 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:37.950445 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_566_566_0 (state Temporary) 2024.10.22 01:37:38.036948 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_982_487} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:38.060219 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_982_487} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_981_486, total 64614 rows starting from the beginning of the part 2024.10.22 01:37:38.079832 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_566_566_0 to 202410_572_572_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:38.219133 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_567_12 to 202410_572_572_0 2024.10.22 01:37:38.230495 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:38.256529 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 65671 2024.10.22 01:37:38.260529 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:38.304416 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_982_487} MergeTreeSequentialSource: Reading 2 marks from part 202410_982_982_0, total 52 rows starting from the beginning of the part 2024.10.22 01:37:38.308585 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_567_12 to 202410_572_572_0 into Compact with storage Full 2024.10.22 01:37:38.463856 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:38.467970 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_567_12, total 6091 rows starting from the beginning of the part 2024.10.22 01:37:38.476460 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_568_568_0, total 86 rows starting from the beginning of the part 2024.10.22 01:37:38.489734 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_569_569_0, total 97 rows starting from the beginning of the part 2024.10.22 01:37:38.510598 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_570_570_0, total 101 rows starting from the beginning of the part 2024.10.22 01:37:38.534575 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_571_571_0, total 98 rows starting from the beginning of the part 2024.10.22 01:37:38.574603 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_572_572_0, total 73 rows starting from the beginning of the part 2024.10.22 01:37:38.864905 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 8883 2024.10.22 01:37:38.975855 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 141 bytes for fc8b03d26e158b0bafb59fd267803b91:0) 2024.10.22 01:37:38.976994 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ed0cf3c4282e83499e81d054f2654814, offset: 0, size: 154 2024.10.22 01:37:38.979537 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ed0cf3c4282e83499e81d054f2654814 for removal 2024.10.22 01:37:38.982950 [ 1476 ] {} CacheMetadata: Key ed0cf3c4282e83499e81d054f2654814 is removed from metadata 2024.10.22 01:37:39.162633 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:39.166984 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:39.209595 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_577_577_0 (state Temporary) 2024.10.22 01:37:39.271354 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_577_577_0 to 202410_590_590_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:39.371925 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8883 2024.10.22 01:37:39.625514 [ 1242 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:37:39.661525 [ 1242 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:37:39.765338 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} MergeTask::MergeProjectionsStage: Merge sorted 6546 rows, containing 13 columns (13 merged, 0 gathered) in 1.494625024 sec., 4379.6938328258575 rows/sec., 1.90 MiB/sec. 2024.10.22 01:37:39.975313 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c1905bb91beaf3738fb0a13acf6c4fbf) : [0, 6280]: Setting download as failed: File segment: [0, 6280], key: c1905bb91beaf3738fb0a13acf6c4fbf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:39.976865 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c1905bb91beaf3738fb0a13acf6c4fbf): No space left in cache to reserve 6281 bytes, will continue without cache download 2024.10.22 01:37:39.987692 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c1905bb91beaf3738fb0a13acf6c4fbf, offset: 0, size: 0 2024.10.22 01:37:39.989753 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c1905bb91beaf3738fb0a13acf6c4fbf for removal 2024.10.22 01:37:39.998782 [ 1476 ] {} CacheMetadata: Key c1905bb91beaf3738fb0a13acf6c4fbf is removed from metadata 2024.10.22 01:37:40.165078 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} MergedBlockOutputStream: filled checksums 202410_507_572_13 (state Temporary) 2024.10.22 01:37:40.166154 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_116_116_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:37:40.451557 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 4.08 MiB, will set to 1.69 GiB (RSS), difference: -10.19 MiB 2024.10.22 01:37:40.529610 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:37:40.536393 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:37:40.574434 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_572_13 to 202410_507_572_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:40.576769 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_572_13} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_567_12, 202410_572_572_0] -> 202410_507_572_13 2024.10.22 01:37:40.590486 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_423_87, 202410_424_424_0, 202410_425_425_0, 202410_426_426_0, 202410_427_427_0, 202410_428_428_0] 2024.10.22 01:37:40.592443 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_423_87, 202410_424_424_0, 202410_425_425_0, 202410_426_426_0, 202410_427_427_0, 202410_428_428_0] 2024.10.22 01:37:40.634802 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_423_87, 202410_424_424_0, 202410_425_425_0, 202410_426_426_0, 202410_427_427_0, 202410_428_428_0] 2024.10.22 01:37:40.678183 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_116_116_0 (state Temporary) 2024.10.22 01:37:40.727181 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.29 MiB. 2024.10.22 01:37:40.982329 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(8589f389096ec58cc8d3c15efcf44db3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8589f389096ec58cc8d3c15efcf44db3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:37:40.999254 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 972, file segment info: File segment: [0, 33554431], key: 8589f389096ec58cc8d3c15efcf44db3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:37:41.002565 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:41.203329 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_982_487} MergeTask::MergeProjectionsStage: Merge sorted 64666 rows, containing 13 columns (13 merged, 0 gathered) in 3.319482768 sec., 19480.745802744896 rows/sec., 3.21 MiB/sec. 2024.10.22 01:37:41.575591 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_982_487} MergedBlockOutputStream: filled checksums 202410_1_982_487 (state Temporary) 2024.10.22 01:37:41.784042 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 146 bytes for 80cf4438de99dff58f1227d8dab68115:0) 2024.10.22 01:37:41.786435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fc8b03d26e158b0bafb59fd267803b91, offset: 0, size: 141 2024.10.22 01:37:41.791264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fc8b03d26e158b0bafb59fd267803b91 for removal 2024.10.22 01:37:41.804767 [ 1476 ] {} CacheMetadata: Key fc8b03d26e158b0bafb59fd267803b91 is removed from metadata 2024.10.22 01:37:41.976976 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 64187 2024.10.22 01:37:41.987769 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_982_487} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_982_487 to 202410_1_982_487 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:41.991660 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_982_487} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_981_486, 202410_982_982_0] -> 202410_1_982_487 2024.10.22 01:37:42.027689 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.80 MiB. 2024.10.22 01:37:42.032890 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 869394 2024.10.22 01:37:42.319251 [ 1425 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:37:42.676341 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:42.679020 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(4c5d1d1ff48f86cee636e42c003926d1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4c5d1d1ff48f86cee636e42c003926d1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:37:42.680974 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 117343, file segment info: File segment: [0, 33554431], key: 4c5d1d1ff48f86cee636e42c003926d1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:37:42.679974 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:42.683902 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:42.768794 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_975_975_0 (state Temporary) 2024.10.22 01:37:42.815988 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:42.821475 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:42.927266 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_975_975_0 to 202410_988_988_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:42.979046 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64187 2024.10.22 01:37:43.065018 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_553_553_0 (state Temporary) 2024.10.22 01:37:43.156026 [ 1154 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_982_487 to 202410_983_983_0 2024.10.22 01:37:43.161278 [ 1154 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.44 MiB using storage policy from min volume index 0 2024.10.22 01:37:43.165091 [ 1154 ] {} DiskLocal: Reserved 2.44 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:43.185017 [ 1154 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 01:37:43.191427 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_553_553_0 to 202410_560_560_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:43.195489 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_983_488} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_982_487 to 202410_983_983_0 into Compact with storage Full 2024.10.22 01:37:43.222169 [ 1271 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_550_109 to 202410_551_556_1 2024.10.22 01:37:43.252079 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_983_488} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:43.256407 [ 1271 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:43.259359 [ 1271 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:43.281560 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_556_110} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_550_109 to 202410_551_556_1 into Wide with storage Full 2024.10.22 01:37:43.297705 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_556_110} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:43.305815 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_556_110} MergeTreeSequentialSource: Reading 107 marks from part 202410_1_550_109, total 867969 rows starting from the beginning of the part 2024.10.22 01:37:43.338270 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 869394 2024.10.22 01:37:43.362160 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_556_110} MergeTreeSequentialSource: Reading 3 marks from part 202410_551_556_1, total 8855 rows starting from the beginning of the part 2024.10.22 01:37:43.374922 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_983_488} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_982_487, total 64666 rows starting from the beginning of the part 2024.10.22 01:37:43.377498 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4ee88f5f3efbd57d7bef6465d626265e) : [0, 240497]: Setting download as failed: File segment: [0, 240497], key: 4ee88f5f3efbd57d7bef6465d626265e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:43.382848 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4ee88f5f3efbd57d7bef6465d626265e): No space left in cache to reserve 240498 bytes, will continue without cache download 2024.10.22 01:37:43.416588 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_983_488} MergeTreeSequentialSource: Reading 2 marks from part 202410_983_983_0, total 38 rows starting from the beginning of the part 2024.10.22 01:37:43.540456 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4ee88f5f3efbd57d7bef6465d626265e, offset: 0, size: 0 2024.10.22 01:37:43.544466 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4ee88f5f3efbd57d7bef6465d626265e for removal 2024.10.22 01:37:43.552796 [ 1476 ] {} CacheMetadata: Key 4ee88f5f3efbd57d7bef6465d626265e is removed from metadata 2024.10.22 01:37:43.672344 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 485 2024.10.22 01:37:43.720015 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_526_105, 202410_527_527_0, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0] 2024.10.22 01:37:43.725165 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_526_105, 202410_527_527_0, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0] 2024.10.22 01:37:43.761401 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_526_105, 202410_527_527_0, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0] 2024.10.22 01:37:44.104864 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(9f91f974cc5502b4802ffc8954836e8f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9f91f974cc5502b4802ffc8954836e8f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:37:44.112168 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 751, file segment info: File segment: [0, 33554431], key: 9f91f974cc5502b4802ffc8954836e8f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:37:44.116249 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:44.158312 [ 1745 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/upa/hcgstehqtpbqdvioznvkinnrruoat, size 117343 2024.10.22 01:37:44.309247 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:44.312206 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:44.359053 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:37:44.472352 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_467_467_0 (state Temporary) 2024.10.22 01:37:44.478770 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:37:44.500990 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:37:44.576079 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_983_488/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:37:44.593382 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_467_467_0 to 202410_477_477_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:44.643411 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:37:44.661925 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:37:44.804186 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 485 2024.10.22 01:37:44.860131 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 4.25 MiB, will set to 1.68 GiB (RSS), difference: -15.03 MiB 2024.10.22 01:37:44.909720 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:37:44.921758 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:37:45.058973 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 01:37:45.127082 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gtg/ypvlyktxmahrequroabolwcyoiwxo (LogSeriesLimiter: on interval from 2024-10-22 01:37:18 to 2024-10-22 01:37:45 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:37:45.176322 [ 1269 ] {} DatabaseCatalog: Execution took 238 ms. 2024.10.22 01:37:45.290148 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_877_450, 202410_878_883_1] 2024.10.22 01:37:45.295902 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_877_450, 202410_878_883_1] 2024.10.22 01:37:45.305321 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_877_450, 202410_878_883_1] 2024.10.22 01:37:45.524144 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 145 bytes for 3e99340b5d688bcbfee94b53dda22c36:0) 2024.10.22 01:37:45.526828 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 80cf4438de99dff58f1227d8dab68115, offset: 0, size: 146 2024.10.22 01:37:45.539531 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 80cf4438de99dff58f1227d8dab68115 for removal 2024.10.22 01:37:45.545142 [ 1476 ] {} CacheMetadata: Key 80cf4438de99dff58f1227d8dab68115 is removed from metadata 2024.10.22 01:37:45.710620 [ 1277 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 01:37:45.764354 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 155 entries to flush up to offset 65826 2024.10.22 01:37:45.919667 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:45.926247 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:45.991730 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_567_567_0 (state Temporary) 2024.10.22 01:37:46.257824 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_567_567_0 to 202410_573_573_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:46.279391 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(dc2232227d1c9a176ea0151b5a695ad0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: dc2232227d1c9a176ea0151b5a695ad0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:37:46.305623 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 15517, file segment info: File segment: [0, 33554431], key: dc2232227d1c9a176ea0151b5a695ad0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:37:46.309361 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:46.326126 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 65826 2024.10.22 01:37:46.951051 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_983_488} MergeTask::MergeProjectionsStage: Merge sorted 64704 rows, containing 13 columns (13 merged, 0 gathered) in 3.765022962 sec., 17185.552559187818 rows/sec., 2.83 MiB/sec. 2024.10.22 01:37:46.976147 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 8898 2024.10.22 01:37:46.984942 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 64270 2024.10.22 01:37:47.035323 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_983_488} MergedBlockOutputStream: filled checksums 202410_1_983_488 (state Temporary) 2024.10.22 01:37:47.167655 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(228a509cb188a7ac11d4a9f9452d8ce3) : [0, 135290]: Setting download as failed: File segment: [0, 135290], key: 228a509cb188a7ac11d4a9f9452d8ce3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:47.170667 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(228a509cb188a7ac11d4a9f9452d8ce3): No space left in cache to reserve 135291 bytes, will continue without cache download 2024.10.22 01:37:47.199923 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:47.203187 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 228a509cb188a7ac11d4a9f9452d8ce3, offset: 0, size: 0 2024.10.22 01:37:47.204527 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:47.205220 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 228a509cb188a7ac11d4a9f9452d8ce3 for removal 2024.10.22 01:37:47.222199 [ 1476 ] {} CacheMetadata: Key 228a509cb188a7ac11d4a9f9452d8ce3 is removed from metadata 2024.10.22 01:37:47.386911 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_578_578_0 (state Temporary) 2024.10.22 01:37:47.393272 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_983_488} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_983_488 to 202410_1_983_488 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:47.422441 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_983_488} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_982_487, 202410_983_983_0] -> 202410_1_983_488 2024.10.22 01:37:47.459871 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.79 MiB. 2024.10.22 01:37:47.553683 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_578_578_0 to 202410_591_591_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:47.576567 [ 1160 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_586_117 to 202410_591_591_0 2024.10.22 01:37:47.584508 [ 1160 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:47.589961 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8898 2024.10.22 01:37:47.614778 [ 1160 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:47.666039 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_586_117 to 202410_591_591_0 into Compact with storage Full 2024.10.22 01:37:47.684077 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:47.686342 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:47.711201 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:47.720158 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_586_117, total 8922 rows starting from the beginning of the part 2024.10.22 01:37:47.749556 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_587_587_0, total 36 rows starting from the beginning of the part 2024.10.22 01:37:47.769532 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_588_588_0, total 11 rows starting from the beginning of the part 2024.10.22 01:37:47.802200 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_589_589_0, total 8 rows starting from the beginning of the part 2024.10.22 01:37:47.829590 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_590_590_0, total 12 rows starting from the beginning of the part 2024.10.22 01:37:47.867618 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_591_591_0, total 15 rows starting from the beginning of the part 2024.10.22 01:37:47.868649 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(60705ae02c8ab04c5516a7357e755577) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 60705ae02c8ab04c5516a7357e755577, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:37:47.871507 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 567, file segment info: File segment: [0, 33554431], key: 60705ae02c8ab04c5516a7357e755577, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:37:47.957824 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_976_976_0 (state Temporary) 2024.10.22 01:37:47.976610 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:48.128875 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_976_976_0 to 202410_989_989_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:48.159191 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_983_488 to 202410_984_984_0 2024.10.22 01:37:48.163249 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.45 MiB using storage policy from min volume index 0 2024.10.22 01:37:48.162503 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_556_110/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:37:48.164834 [ 1183 ] {} DiskLocal: Reserved 2.45 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:48.171567 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_984_489} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_983_488 to 202410_984_984_0 into Compact with storage Full 2024.10.22 01:37:48.227987 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_984_489} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:48.248598 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_984_489} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_983_488, total 64704 rows starting from the beginning of the part 2024.10.22 01:37:48.402816 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_984_489} MergeTreeSequentialSource: Reading 2 marks from part 202410_984_984_0, total 71 rows starting from the beginning of the part 2024.10.22 01:37:48.475214 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64270 2024.10.22 01:37:48.563163 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 168 bytes for 26d01fc05e6c205dfd1870527fef7455:0) 2024.10.22 01:37:48.564540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3e99340b5d688bcbfee94b53dda22c36, offset: 0, size: 145 2024.10.22 01:37:48.566766 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3e99340b5d688bcbfee94b53dda22c36 for removal 2024.10.22 01:37:48.568626 [ 1476 ] {} CacheMetadata: Key 3e99340b5d688bcbfee94b53dda22c36 is removed from metadata 2024.10.22 01:37:48.644772 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 12.77 MiB, will set to 1.69 GiB (RSS), difference: 11.65 MiB 2024.10.22 01:37:49.209944 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_883_451, 202410_884_884_0] 2024.10.22 01:37:49.213558 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_883_451, 202410_884_884_0] 2024.10.22 01:37:49.224842 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_883_451, 202410_884_884_0] 2024.10.22 01:37:49.470986 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} MergeTask::MergeProjectionsStage: Merge sorted 9004 rows, containing 12 columns (12 merged, 0 gathered) in 1.818356133 sec., 4951.72526250115 rows/sec., 787.33 KiB/sec. 2024.10.22 01:37:49.517276 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} MergedBlockOutputStream: filled checksums 202410_1_591_118 (state Temporary) 2024.10.22 01:37:49.662975 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b68012a0c4dfd82d8772743aa0809b28) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b68012a0c4dfd82d8772743aa0809b28, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:37:49.666537 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 5998, file segment info: File segment: [0, 33554431], key: b68012a0c4dfd82d8772743aa0809b28, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:37:49.667765 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:49.990787 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_591_118 to 202410_1_591_118 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:49.996025 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_591_118} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_586_117, 202410_591_591_0] -> 202410_1_591_118 2024.10.22 01:37:50.163723 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.53 MiB. 2024.10.22 01:37:50.342997 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 870164 2024.10.22 01:37:50.476400 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(27a252f53c9ff6ed01f5a4cbf6c7ebb6) : [0, 301379]: Setting download as failed: File segment: [0, 301379], key: 27a252f53c9ff6ed01f5a4cbf6c7ebb6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:50.480864 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(27a252f53c9ff6ed01f5a4cbf6c7ebb6): No space left in cache to reserve 301380 bytes, will continue without cache download 2024.10.22 01:37:50.534564 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:37:50.539231 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:37:50.645200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 27a252f53c9ff6ed01f5a4cbf6c7ebb6, offset: 0, size: 0 2024.10.22 01:37:50.683063 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_116_116_0 to 202410_119_119_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:50.649776 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 27a252f53c9ff6ed01f5a4cbf6c7ebb6 for removal 2024.10.22 01:37:50.754768 [ 1476 ] {} CacheMetadata: Key 27a252f53c9ff6ed01f5a4cbf6c7ebb6 is removed from metadata 2024.10.22 01:37:50.875877 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 6 old parts to remove. Parts: [202410_1_99_20, 202410_100_100_0, 202410_101_101_0, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0] 2024.10.22 01:37:50.878045 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from filesystem (serially): Parts: [202410_1_99_20, 202410_100_100_0, 202410_101_101_0, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0] 2024.10.22 01:37:51.018556 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 6 parts from memory: Parts: [202410_1_99_20, 202410_100_100_0, 202410_101_101_0, 202410_102_102_0, 202410_103_103_0, 202410_104_104_0] 2024.10.22 01:37:51.144188 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:51.199541 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:51.472224 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(56e2db7f3f6594c06f12b6fafca1c01e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 56e2db7f3f6594c06f12b6fafca1c01e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:37:51.478873 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 573, file segment info: File segment: [0, 33554431], key: 56e2db7f3f6594c06f12b6fafca1c01e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:37:51.483349 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:37:51.559054 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_554_554_0 (state Temporary) 2024.10.22 01:37:51.746763 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_554_554_0 to 202410_561_561_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:51.881655 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 870164 2024.10.22 01:37:52.087999 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:37:52.111717 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:37:52.152729 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:37:52.173113 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:37:52.205823 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:37:52.238515 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:37:52.306658 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 486 2024.10.22 01:37:52.352621 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:37:52.380077 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:37:52.507353 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 64357 2024.10.22 01:37:52.530148 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:37:52.571598 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for 870944269316f2e0d50110de7f19307f:0) 2024.10.22 01:37:52.575709 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 26d01fc05e6c205dfd1870527fef7455, offset: 0, size: 168 2024.10.22 01:37:52.580913 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 26d01fc05e6c205dfd1870527fef7455 for removal 2024.10.22 01:37:52.599799 [ 1476 ] {} CacheMetadata: Key 26d01fc05e6c205dfd1870527fef7455 is removed from metadata 2024.10.22 01:37:52.614435 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 12.06 MiB, will set to 1.69 GiB (RSS), difference: 13.14 MiB 2024.10.22 01:37:52.676301 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:52.681372 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:52.745793 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_468_468_0 (state Temporary) 2024.10.22 01:37:52.771555 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:52.784587 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:52.830230 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_468_468_0 to 202410_478_478_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:52.867892 [ 1207 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_473_97 to 202410_478_478_0 2024.10.22 01:37:52.872710 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gln/lebwdiracosjqaleofgyhgzptmhxt (LogSeriesLimiter: on interval from 2024-10-22 01:37:45 to 2024-10-22 01:37:52 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:37:52.883863 [ 1207 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:52.886869 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_984_489} MergeTask::MergeProjectionsStage: Merge sorted 64775 rows, containing 13 columns (13 merged, 0 gathered) in 4.718509085 sec., 13727.853191152646 rows/sec., 2.26 MiB/sec. 2024.10.22 01:37:52.895422 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:52.954891 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_473_97 to 202410_478_478_0 into Compact with storage Full 2024.10.22 01:37:53.015931 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 486 2024.10.22 01:37:53.046889 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_977_977_0 (state Temporary) 2024.10.22 01:37:53.089045 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_984_489} MergedBlockOutputStream: filled checksums 202410_1_984_489 (state Temporary) 2024.10.22 01:37:53.090368 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:53.104849 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_977_977_0 to 202410_990_990_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:53.142651 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_985_985_0 to 202410_990_990_0 2024.10.22 01:37:53.157491 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:53.256599 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_473_97, total 517 rows starting from the beginning of the part 2024.10.22 01:37:53.259842 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64357 2024.10.22 01:37:53.260850 [ 1231 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:53.286353 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_985_985_0 to 202410_990_990_0 into Compact with storage Full 2024.10.22 01:37:53.384740 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4659 2024.10.22 01:37:53.408488 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_474_474_0, total 1 rows starting from the beginning of the part 2024.10.22 01:37:53.440036 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:37:53.412503 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_984_489} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_984_489 to 202410_1_984_489 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:53.453063 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_984_489} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_983_488, 202410_984_984_0] -> 202410_1_984_489 2024.10.22 01:37:53.454168 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_985_985_0, total 36 rows starting from the beginning of the part 2024.10.22 01:37:53.488277 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.81 MiB. 2024.10.22 01:37:53.516772 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_986_986_0, total 72 rows starting from the beginning of the part 2024.10.22 01:37:53.575206 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_987_987_0, total 55 rows starting from the beginning of the part 2024.10.22 01:37:53.643163 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_988_988_0, total 80 rows starting from the beginning of the part 2024.10.22 01:37:53.715817 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_989_989_0, total 83 rows starting from the beginning of the part 2024.10.22 01:37:53.717641 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_475_475_0, total 1 rows starting from the beginning of the part 2024.10.22 01:37:53.857768 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(60a1c8da4fa6e041ee5953a3702e0fec) : [0, 57408]: Setting download as failed: File segment: [0, 57408], key: 60a1c8da4fa6e041ee5953a3702e0fec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:53.859365 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(60a1c8da4fa6e041ee5953a3702e0fec): No space left in cache to reserve 57409 bytes, will continue without cache download 2024.10.22 01:37:53.859413 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_476_476_0, total 1 rows starting from the beginning of the part 2024.10.22 01:37:53.863430 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 65926 2024.10.22 01:37:53.870909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 60a1c8da4fa6e041ee5953a3702e0fec, offset: 0, size: 0 2024.10.22 01:37:53.873928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 60a1c8da4fa6e041ee5953a3702e0fec for removal 2024.10.22 01:37:53.876847 [ 1476 ] {} CacheMetadata: Key 60a1c8da4fa6e041ee5953a3702e0fec is removed from metadata 2024.10.22 01:37:53.915361 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_990_990_0, total 87 rows starting from the beginning of the part 2024.10.22 01:37:53.965734 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_477_477_0, total 1 rows starting from the beginning of the part 2024.10.22 01:37:54.060259 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_478_478_0, total 1 rows starting from the beginning of the part 2024.10.22 01:37:54.408270 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:54.411391 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:54.462751 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_568_568_0 (state Temporary) 2024.10.22 01:37:54.500169 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_568_568_0 to 202410_574_574_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:54.561251 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 65926 2024.10.22 01:37:54.665257 [ 1216 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:37:54.912328 [ 1216 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:37:54.922836 [ 1216 ] {} DNSCacheUpdater: Execution took 254 ms. 2024.10.22 01:37:55.094116 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 8918 2024.10.22 01:37:55.289169 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for 15728ea97ae5b2fc5a6e91c25b8d1308:0) 2024.10.22 01:37:55.296693 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 870944269316f2e0d50110de7f19307f, offset: 0, size: 144 2024.10.22 01:37:55.302464 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 870944269316f2e0d50110de7f19307f for removal 2024.10.22 01:37:55.306450 [ 1476 ] {} CacheMetadata: Key 870944269316f2e0d50110de7f19307f is removed from metadata 2024.10.22 01:37:55.429561 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:55.442647 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:37:55.468918 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 9.52 MiB, will set to 1.68 GiB (RSS), difference: -14.01 MiB 2024.10.22 01:37:55.511029 [ 1230 ] {} DatabaseCatalog: Execution took 332 ms. 2024.10.22 01:37:55.617621 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_579_579_0 (state Temporary) 2024.10.22 01:37:55.763711 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_579_579_0 to 202410_592_592_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:55.778762 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_985_990_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:37:55.810940 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8918 2024.10.22 01:37:55.909182 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} MergeTask::MergeProjectionsStage: Merge sorted 413 rows, containing 13 columns (13 merged, 0 gathered) in 2.62902504 sec., 157.09245584058795 rows/sec., 27.03 KiB/sec. 2024.10.22 01:37:55.955610 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} MergedBlockOutputStream: filled checksums 202410_985_990_1 (state Temporary) 2024.10.22 01:37:56.112440 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_985_990_1 to 202410_985_990_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:56.116314 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_985_990_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_985_985_0, 202410_990_990_0] -> 202410_985_990_1 2024.10.22 01:37:56.377210 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:37:56.464051 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f21dc47a21b66275090d010570a30e5d) : [0, 74073]: Setting download as failed: File segment: [0, 74073], key: f21dc47a21b66275090d010570a30e5d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:56.468501 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f21dc47a21b66275090d010570a30e5d): No space left in cache to reserve 74074 bytes, will continue without cache download 2024.10.22 01:37:56.507158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f21dc47a21b66275090d010570a30e5d, offset: 0, size: 0 2024.10.22 01:37:56.509521 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f21dc47a21b66275090d010570a30e5d for removal 2024.10.22 01:37:56.515584 [ 1476 ] {} CacheMetadata: Key f21dc47a21b66275090d010570a30e5d is removed from metadata 2024.10.22 01:37:56.619013 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} MergeTask::MergeProjectionsStage: Merge sorted 522 rows, containing 32 columns (32 merged, 0 gathered) in 3.675634499 sec., 142.0162968167853 rows/sec., 13.09 KiB/sec. 2024.10.22 01:37:56.668151 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} MergedBlockOutputStream: filled checksums 202410_1_478_98 (state Temporary) 2024.10.22 01:37:57.225234 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_884_452, 202410_885_885_0] 2024.10.22 01:37:57.229372 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_884_452, 202410_885_885_0] 2024.10.22 01:37:57.251930 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_884_452, 202410_885_885_0] 2024.10.22 01:37:57.265238 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_478_98 to 202410_1_478_98 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:57.271298 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 64445 2024.10.22 01:37:57.273220 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_478_98} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_473_97, 202410_478_478_0] -> 202410_1_478_98 2024.10.22 01:37:57.439443 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:37:57.544152 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:57.551307 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:57.619945 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_978_978_0 (state Temporary) 2024.10.22 01:37:57.767476 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_978_978_0 to 202410_991_991_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:37:57.923842 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64445 2024.10.22 01:37:58.177275 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for 92323c9af466e6316295c43c50eb3299:0) 2024.10.22 01:37:58.178734 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 15728ea97ae5b2fc5a6e91c25b8d1308, offset: 0, size: 144 2024.10.22 01:37:58.179979 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 15728ea97ae5b2fc5a6e91c25b8d1308 for removal 2024.10.22 01:37:58.183145 [ 1476 ] {} CacheMetadata: Key 15728ea97ae5b2fc5a6e91c25b8d1308 is removed from metadata 2024.10.22 01:37:58.263045 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 13.49 MiB, will set to 1.68 GiB (RSS), difference: 10.42 MiB 2024.10.22 01:37:58.464123 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_886_886_0, 202410_887_887_0, 202410_888_888_0, 202410_889_889_0, 202410_890_890_0, 202410_891_891_0] 2024.10.22 01:37:58.470467 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_886_886_0, 202410_887_887_0, 202410_888_888_0, 202410_889_889_0, 202410_890_890_0, 202410_891_891_0] 2024.10.22 01:37:58.491473 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_886_886_0, 202410_887_887_0, 202410_888_888_0, 202410_889_889_0, 202410_890_890_0, 202410_891_891_0] 2024.10.22 01:37:58.883856 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 871704 2024.10.22 01:37:58.958484 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_556_110/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:37:59.316240 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 12.91 MiB, will set to 1.68 GiB (RSS), difference: 2.73 MiB 2024.10.22 01:37:59.624808 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:37:59.628239 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:37:59.856172 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0f82a5af094ab0deb35f659e2c50b6ed) : [0, 102008]: Setting download as failed: File segment: [0, 102008], key: 0f82a5af094ab0deb35f659e2c50b6ed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:37:59.859547 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0f82a5af094ab0deb35f659e2c50b6ed): No space left in cache to reserve 102009 bytes, will continue without cache download 2024.10.22 01:37:59.874130 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0f82a5af094ab0deb35f659e2c50b6ed, offset: 0, size: 0 2024.10.22 01:37:59.878568 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0f82a5af094ab0deb35f659e2c50b6ed for removal 2024.10.22 01:37:59.880850 [ 1476 ] {} CacheMetadata: Key 0f82a5af094ab0deb35f659e2c50b6ed is removed from metadata 2024.10.22 01:37:59.887136 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_555_555_0 (state Temporary) 2024.10.22 01:37:59.964028 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_555_555_0 to 202410_562_562_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:00.087298 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 871704 2024.10.22 01:38:00.083050 [ 1219 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_557_557_0 to 202410_562_562_0 2024.10.22 01:38:00.122510 [ 1219 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:00.125241 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:38:00.144148 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_557_557_0 to 202410_562_562_0 into Compact with storage Full 2024.10.22 01:38:00.276381 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:38:00.284311 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_557_557_0, total 1321 rows starting from the beginning of the part 2024.10.22 01:38:00.299678 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_558_558_0, total 2120 rows starting from the beginning of the part 2024.10.22 01:38:00.321034 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_559_559_0, total 1564 rows starting from the beginning of the part 2024.10.22 01:38:00.340717 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_560_560_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:38:00.357756 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_561_561_0, total 770 rows starting from the beginning of the part 2024.10.22 01:38:00.371104 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_562_562_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:38:00.538587 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:38:00.544540 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:38:00.880068 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for 8c9ae364df066893f0868cac817a6ae7:0) 2024.10.22 01:38:00.883123 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 92323c9af466e6316295c43c50eb3299, offset: 0, size: 155 2024.10.22 01:38:00.886916 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 92323c9af466e6316295c43c50eb3299 for removal 2024.10.22 01:38:00.890454 [ 1476 ] {} CacheMetadata: Key 92323c9af466e6316295c43c50eb3299 is removed from metadata 2024.10.22 01:38:00.944461 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 4699 2024.10.22 01:38:01.929223 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(866e05e9aa17e3127e49e774b5b88238) : [0, 97322]: Setting download as failed: File segment: [0, 97322], key: 866e05e9aa17e3127e49e774b5b88238, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:01.938631 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 64493 2024.10.22 01:38:01.940342 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(866e05e9aa17e3127e49e774b5b88238): No space left in cache to reserve 97323 bytes, will continue without cache download 2024.10.22 01:38:01.959614 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 866e05e9aa17e3127e49e774b5b88238, offset: 0, size: 0 2024.10.22 01:38:01.963611 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 866e05e9aa17e3127e49e774b5b88238 for removal 2024.10.22 01:38:02.068973 [ 1476 ] {} CacheMetadata: Key 866e05e9aa17e3127e49e774b5b88238 is removed from metadata 2024.10.22 01:38:02.084457 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:02.088932 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:02.095275 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 87 entries to flush up to offset 66013 2024.10.22 01:38:02.253666 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_979_979_0 (state Temporary) 2024.10.22 01:38:02.352198 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} MergeTask::MergeProjectionsStage: Merge sorted 8470 rows, containing 4 columns (4 merged, 0 gathered) in 2.212997796 sec., 3827.3874539367143 rows/sec., 59.80 KiB/sec. 2024.10.22 01:38:02.396438 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} MergedBlockOutputStream: filled checksums 202410_557_562_1 (state Temporary) 2024.10.22 01:38:02.417135 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:02.423332 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:02.500630 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_979_979_0 to 202410_992_992_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:02.555072 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_557_562_1 to 202410_557_562_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:02.559996 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_557_562_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_557_557_0, 202410_562_562_0] -> 202410_557_562_1 2024.10.22 01:38:02.567164 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_569_569_0 (state Temporary) 2024.10.22 01:38:02.604892 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64493 2024.10.22 01:38:02.621002 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_569_569_0 to 202410_575_575_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:02.715688 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:38:02.715997 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 11.08 MiB, will set to 1.68 GiB (RSS), difference: -1.19 MiB 2024.10.22 01:38:02.751855 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 66013 2024.10.22 01:38:02.772233 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(8a088ac1aebe4bbe77961f3dc7ff002e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8a088ac1aebe4bbe77961f3dc7ff002e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:38:02.775291 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 2018, file segment info: File segment: [0, 33554431], key: 8a088ac1aebe4bbe77961f3dc7ff002e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:38:02.778591 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:03.114961 [ 1166 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 5 parts from 202410_115_115_0 to 202410_119_119_0 2024.10.22 01:38:03.125753 [ 1166 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:03.129401 [ 1166 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:38:03.172520 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_115_119_1} MergeTask::PrepareStage: Merging 5 parts: from 202410_115_115_0 to 202410_119_119_0 into Compact with storage Full 2024.10.22 01:38:03.312301 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 8944 2024.10.22 01:38:03.520569 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:03.528317 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:03.632712 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for 1ff701f99b5504a2e99539eb2663eef3:0) 2024.10.22 01:38:03.637381 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8c9ae364df066893f0868cac817a6ae7, offset: 0, size: 155 2024.10.22 01:38:03.641142 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8c9ae364df066893f0868cac817a6ae7 for removal 2024.10.22 01:38:03.644835 [ 1476 ] {} CacheMetadata: Key 8c9ae364df066893f0868cac817a6ae7 is removed from metadata 2024.10.22 01:38:03.666866 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_580_580_0 (state Temporary) 2024.10.22 01:38:03.711885 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_580_580_0 to 202410_593_593_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:03.757749 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8944 2024.10.22 01:38:04.572172 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(b603ee0238a597110dda86687f3e1468) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b603ee0238a597110dda86687f3e1468, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:38:04.577138 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 461, file segment info: File segment: [0, 33554431], key: b603ee0238a597110dda86687f3e1468, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:38:04.581328 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:05.526630 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(339e947a8f50d54cefcceb9fc798cef5) : [0, 52746]: Setting download as failed: File segment: [0, 52746], key: 339e947a8f50d54cefcceb9fc798cef5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:05.529198 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(339e947a8f50d54cefcceb9fc798cef5): No space left in cache to reserve 52747 bytes, will continue without cache download 2024.10.22 01:38:05.559253 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 339e947a8f50d54cefcceb9fc798cef5, offset: 0, size: 0 2024.10.22 01:38:05.571434 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 339e947a8f50d54cefcceb9fc798cef5 for removal 2024.10.22 01:38:05.574916 [ 1476 ] {} CacheMetadata: Key 339e947a8f50d54cefcceb9fc798cef5 is removed from metadata 2024.10.22 01:38:06.266707 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 11.71 MiB, will set to 1.68 GiB (RSS), difference: 1.90 MiB 2024.10.22 01:38:06.618798 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 64542 2024.10.22 01:38:06.880927 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(f320dfd6c342db9bf50a3d166d51c3b8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f320dfd6c342db9bf50a3d166d51c3b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:38:06.891623 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 13639, file segment info: File segment: [0, 33554431], key: f320dfd6c342db9bf50a3d166d51c3b8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:38:06.899161 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:07.094667 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 872474 2024.10.22 01:38:07.230925 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:07.237446 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:07.279830 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_980_980_0 (state Temporary) 2024.10.22 01:38:07.326026 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_980_980_0 to 202410_993_993_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:07.485207 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64542 2024.10.22 01:38:07.578530 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:07.581297 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:07.719572 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for eae5b85a6c79a6fef1394d0abbdfe910:0) 2024.10.22 01:38:07.724655 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1ff701f99b5504a2e99539eb2663eef3, offset: 0, size: 144 2024.10.22 01:38:07.728846 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1ff701f99b5504a2e99539eb2663eef3 for removal 2024.10.22 01:38:07.734665 [ 1476 ] {} CacheMetadata: Key 1ff701f99b5504a2e99539eb2663eef3 is removed from metadata 2024.10.22 01:38:07.852450 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_556_556_0 (state Temporary) 2024.10.22 01:38:07.912220 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_556_556_0 to 202410_563_563_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:08.019606 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 487 2024.10.22 01:38:08.070818 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 872474 2024.10.22 01:38:08.293443 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:08.298882 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:08.365037 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_469_469_0 (state Temporary) 2024.10.22 01:38:08.557203 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_469_469_0 to 202410_479_479_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:08.673773 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(8eb0295d74d355f1bb4b016b95b3e31f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8eb0295d74d355f1bb4b016b95b3e31f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:38:08.685108 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 532, file segment info: File segment: [0, 33554431], key: 8eb0295d74d355f1bb4b016b95b3e31f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:38:08.687780 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:08.752421 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 487 2024.10.22 01:38:08.924882 [ 1693 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gtg/ypvlyktxmahrequroabolwcyoiwxo, size 13639 2024.10.22 01:38:09.278745 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:38:09.296865 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:38:09.324644 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:38:09.354506 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:38:09.378942 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:38:09.515152 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:38:09.540854 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:38:09.548142 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 16.70 MiB, will set to 1.68 GiB (RSS), difference: 7.39 MiB 2024.10.22 01:38:09.567007 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column IsArtifical 2024.10.22 01:38:09.630653 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5a9bd44f41e5376dd8fc7b0aec294600) : [0, 47061]: Setting download as failed: File segment: [0, 47061], key: 5a9bd44f41e5376dd8fc7b0aec294600, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:09.644113 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5a9bd44f41e5376dd8fc7b0aec294600): No space left in cache to reserve 47062 bytes, will continue without cache download 2024.10.22 01:38:09.662446 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5a9bd44f41e5376dd8fc7b0aec294600, offset: 0, size: 0 2024.10.22 01:38:09.673175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5a9bd44f41e5376dd8fc7b0aec294600 for removal 2024.10.22 01:38:09.676228 [ 1476 ] {} CacheMetadata: Key 5a9bd44f41e5376dd8fc7b0aec294600 is removed from metadata 2024.10.22 01:38:09.713272 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/otn/tjojijlebekgqazhenqkofvpnrbsl (LogSeriesLimiter: on interval from 2024-10-22 01:37:52 to 2024-10-22 01:38:09 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:38:09.922537 [ 1191 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:38:09.955983 [ 1191 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:38:10.070997 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_556_110/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:38:10.276792 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 55 entries to flush up to offset 66068 2024.10.22 01:38:10.502546 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:10.517044 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:10.539318 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:38:10.545684 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:38:10.568383 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_570_570_0 (state Temporary) 2024.10.22 01:38:10.611520 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_512_1, 202410_513_513_0, 202410_514_514_0, 202410_515_515_0, 202410_516_516_0, 202410_517_517_0] 2024.10.22 01:38:10.615460 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_512_1, 202410_513_513_0, 202410_514_514_0, 202410_515_515_0, 202410_516_516_0, 202410_517_517_0] 2024.10.22 01:38:10.616621 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_570_570_0 to 202410_576_576_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:10.728594 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_512_1, 202410_513_513_0, 202410_514_514_0, 202410_515_515_0, 202410_516_516_0, 202410_517_517_0] 2024.10.22 01:38:10.765502 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 66068 2024.10.22 01:38:10.909313 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for 1778035bf022bc19d18c46e55b369b69:0) 2024.10.22 01:38:10.914355 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: eae5b85a6c79a6fef1394d0abbdfe910, offset: 0, size: 144 2024.10.22 01:38:10.919766 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key eae5b85a6c79a6fef1394d0abbdfe910 for removal 2024.10.22 01:38:10.924009 [ 1476 ] {} CacheMetadata: Key eae5b85a6c79a6fef1394d0abbdfe910 is removed from metadata 2024.10.22 01:38:11.262654 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 8949 2024.10.22 01:38:11.500629 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 64605 2024.10.22 01:38:11.519821 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_556_110} MergeTask::MergeProjectionsStage: Merge sorted 876824 rows, containing 4 columns (4 merged, 0 gathered) in 28.242695781 sec., 31046.044853475883 rows/sec., 465.53 KiB/sec. 2024.10.22 01:38:11.558701 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 13.32 MiB, will set to 1.68 GiB (RSS), difference: -4.03 MiB 2024.10.22 01:38:11.741649 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:11.744503 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:38:11.803463 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_556_110} MergedBlockOutputStream: filled checksums 202410_1_556_110 (state Temporary) 2024.10.22 01:38:11.813108 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_981_981_0 (state Temporary) 2024.10.22 01:38:11.821284 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:11.826956 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:11.865563 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_581_581_0 (state Temporary) 2024.10.22 01:38:11.977708 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_981_981_0 to 202410_994_994_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:12.031157 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_581_581_0 to 202410_594_594_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:12.067261 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(63af46a370c2534248a4268ea9b72ecf) : [0, 32574]: Setting download as failed: File segment: [0, 32574], key: 63af46a370c2534248a4268ea9b72ecf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:12.069491 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(63af46a370c2534248a4268ea9b72ecf): No space left in cache to reserve 32575 bytes, will continue without cache download 2024.10.22 01:38:12.088512 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64605 2024.10.22 01:38:12.100579 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8949 2024.10.22 01:38:12.198705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 63af46a370c2534248a4268ea9b72ecf, offset: 0, size: 0 2024.10.22 01:38:12.204727 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 63af46a370c2534248a4268ea9b72ecf for removal 2024.10.22 01:38:12.208923 [ 1476 ] {} CacheMetadata: Key 63af46a370c2534248a4268ea9b72ecf is removed from metadata 2024.10.22 01:38:12.769191 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:38:13.283477 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for 5699587a52a6ae9ff4b95bd0395e6911:0) 2024.10.22 01:38:13.286733 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1778035bf022bc19d18c46e55b369b69, offset: 0, size: 144 2024.10.22 01:38:13.293102 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1778035bf022bc19d18c46e55b369b69 for removal 2024.10.22 01:38:13.304811 [ 1476 ] {} CacheMetadata: Key 1778035bf022bc19d18c46e55b369b69 is removed from metadata 2024.10.22 01:38:13.987463 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(798b953c3f02654d3569dee0ec3a772a) : [0, 105398]: Setting download as failed: File segment: [0, 105398], key: 798b953c3f02654d3569dee0ec3a772a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:13.988753 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(798b953c3f02654d3569dee0ec3a772a): No space left in cache to reserve 105399 bytes, will continue without cache download 2024.10.22 01:38:14.013239 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 798b953c3f02654d3569dee0ec3a772a, offset: 0, size: 0 2024.10.22 01:38:14.015421 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 798b953c3f02654d3569dee0ec3a772a for removal 2024.10.22 01:38:14.018552 [ 1476 ] {} CacheMetadata: Key 798b953c3f02654d3569dee0ec3a772a is removed from metadata 2024.10.22 01:38:14.109144 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_556_110} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_556_110 to 202410_1_556_110 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:14.113397 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_556_110} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_550_109, 202410_551_556_1] -> 202410_1_556_110 2024.10.22 01:38:14.137355 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:38:14.383863 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 9.22 MiB, will set to 1.68 GiB (RSS), difference: 5.58 MiB 2024.10.22 01:38:14.864114 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:14.866835 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:15.073319 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 873629 2024.10.22 01:38:15.458194 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for b9b82f9d67e9af891f4553b6faf3815d:0) 2024.10.22 01:38:15.460771 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5699587a52a6ae9ff4b95bd0395e6911, offset: 0, size: 155 2024.10.22 01:38:15.464421 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5699587a52a6ae9ff4b95bd0395e6911 for removal 2024.10.22 01:38:15.532150 [ 1476 ] {} CacheMetadata: Key 5699587a52a6ae9ff4b95bd0395e6911 is removed from metadata 2024.10.22 01:38:15.828867 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:15.836393 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:16.102619 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 64647 2024.10.22 01:38:16.168741 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_557_557_0 (state Temporary) 2024.10.22 01:38:16.271185 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_557_557_0 to 202410_564_564_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:16.325465 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:16.325614 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_115_119_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:38:16.329363 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:16.358980 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 488 2024.10.22 01:38:16.383163 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b4aeb7b897a859cbc6ffdbcf33dd8452) : [0, 1397]: Setting download as failed: File segment: [0, 1397], key: b4aeb7b897a859cbc6ffdbcf33dd8452, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:16.387756 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b4aeb7b897a859cbc6ffdbcf33dd8452): No space left in cache to reserve 1398 bytes, will continue without cache download 2024.10.22 01:38:16.396771 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 873629 2024.10.22 01:38:16.401508 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b4aeb7b897a859cbc6ffdbcf33dd8452, offset: 0, size: 0 2024.10.22 01:38:16.403282 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b4aeb7b897a859cbc6ffdbcf33dd8452 for removal 2024.10.22 01:38:16.406162 [ 1476 ] {} CacheMetadata: Key b4aeb7b897a859cbc6ffdbcf33dd8452 is removed from metadata 2024.10.22 01:38:16.490274 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_982_982_0 (state Temporary) 2024.10.22 01:38:16.531441 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_982_982_0 to 202410_995_995_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:16.563643 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_984_489 to 202410_985_990_1 2024.10.22 01:38:16.565821 [ 1216 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.47 MiB using storage policy from min volume index 0 2024.10.22 01:38:16.567683 [ 1216 ] {} DiskLocal: Reserved 2.47 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:16.627830 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_990_490} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_984_489 to 202410_985_990_1 into Wide with storage Full 2024.10.22 01:38:16.715296 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64647 2024.10.22 01:38:16.775461 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_990_490} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:38:16.784337 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_990_490} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_984_489, total 64775 rows starting from the beginning of the part 2024.10.22 01:38:16.812598 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_990_490} MergeTreeSequentialSource: Reading 2 marks from part 202410_985_990_1, total 413 rows starting from the beginning of the part 2024.10.22 01:38:16.836470 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:16.843358 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:16.876175 [ 1233 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 204 ms. 2024.10.22 01:38:17.002870 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_470_470_0 (state Temporary) 2024.10.22 01:38:17.297309 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_470_470_0 to 202410_480_480_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:17.375433 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 488 2024.10.22 01:38:17.471106 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 9.03 MiB, will set to 1.68 GiB (RSS), difference: -41.63 MiB 2024.10.22 01:38:17.554466 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(f5fb348d856804aa3b60d23d624ca4a6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f5fb348d856804aa3b60d23d624ca4a6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:38:17.557501 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 8170, file segment info: File segment: [0, 33554431], key: f5fb348d856804aa3b60d23d624ca4a6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:38:17.559634 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:17.646087 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_115_119_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_115_115_0, total 37 rows starting from the beginning of the part 2024.10.22 01:38:18.205085 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for fa1ac795754eef0176dc333019f534a3:0) 2024.10.22 01:38:18.209171 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b9b82f9d67e9af891f4553b6faf3815d, offset: 0, size: 144 2024.10.22 01:38:18.211982 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b9b82f9d67e9af891f4553b6faf3815d for removal 2024.10.22 01:38:18.215787 [ 1476 ] {} CacheMetadata: Key b9b82f9d67e9af891f4553b6faf3815d is removed from metadata 2024.10.22 01:38:18.271152 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 133 entries to flush up to offset 66201 2024.10.22 01:38:18.317403 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_117_117_0 (state Temporary) 2024.10.22 01:38:18.543041 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:18.546647 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:18.603510 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_571_571_0 (state Temporary) 2024.10.22 01:38:18.656747 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_571_571_0 to 202410_577_577_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:18.686554 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 66201 2024.10.22 01:38:18.761806 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_990_490/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:38:18.764965 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_572_13 to 202410_577_577_0 2024.10.22 01:38:18.776228 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:18.778441 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:18.948764 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(baad079fcc9e94d1ab9ecad23685f5a3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: baad079fcc9e94d1ab9ecad23685f5a3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:38:18.953738 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 608, file segment info: File segment: [0, 33554431], key: baad079fcc9e94d1ab9ecad23685f5a3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:38:18.957014 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:19.039841 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_572_13 to 202410_577_577_0 into Compact with storage Full 2024.10.22 01:38:19.116476 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:38:19.128104 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_572_13, total 6546 rows starting from the beginning of the part 2024.10.22 01:38:19.165131 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_573_573_0, total 155 rows starting from the beginning of the part 2024.10.22 01:38:19.415621 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_574_574_0, total 100 rows starting from the beginning of the part 2024.10.22 01:38:19.571484 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_575_575_0, total 87 rows starting from the beginning of the part 2024.10.22 01:38:19.606233 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 8958 2024.10.22 01:38:19.608194 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_576_576_0, total 55 rows starting from the beginning of the part 2024.10.22 01:38:19.816689 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ac7e1e514c4f20fb8b554ed47cf8cdc5) : [0, 2022]: Setting download as failed: File segment: [0, 2022], key: ac7e1e514c4f20fb8b554ed47cf8cdc5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:19.819075 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ac7e1e514c4f20fb8b554ed47cf8cdc5): No space left in cache to reserve 2023 bytes, will continue without cache download 2024.10.22 01:38:19.860337 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_577_577_0, total 133 rows starting from the beginning of the part 2024.10.22 01:38:19.863720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ac7e1e514c4f20fb8b554ed47cf8cdc5, offset: 0, size: 0 2024.10.22 01:38:19.866828 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ac7e1e514c4f20fb8b554ed47cf8cdc5 for removal 2024.10.22 01:38:19.867163 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_117_117_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:38:19.870972 [ 1476 ] {} CacheMetadata: Key ac7e1e514c4f20fb8b554ed47cf8cdc5 is removed from metadata 2024.10.22 01:38:19.935221 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:20.041471 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:20.079940 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_582_582_0 (state Temporary) 2024.10.22 01:38:20.136122 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_582_582_0 to 202410_595_595_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:20.176044 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_115_119_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_116_116_0, total 39 rows starting from the beginning of the part 2024.10.22 01:38:20.201809 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8958 2024.10.22 01:38:20.540395 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:38:20.604127 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:38:20.718180 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 64722 2024.10.22 01:38:20.730443 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(47dd784321d777431785b288055435ce) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 47dd784321d777431785b288055435ce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:38:20.736048 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 5536, file segment info: File segment: [0, 33554431], key: 47dd784321d777431785b288055435ce, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:38:20.743107 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:20.864916 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_990_490} MergeTask::MergeProjectionsStage: Merge sorted 65188 rows, containing 13 columns (13 merged, 0 gathered) in 4.278675612 sec., 15235.555557699521 rows/sec., 2.51 MiB/sec. 2024.10.22 01:38:20.877621 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:20.880670 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:21.055484 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_983_983_0 (state Temporary) 2024.10.22 01:38:21.184357 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_990_490} MergedBlockOutputStream: filled checksums 202410_1_990_490 (state Temporary) 2024.10.22 01:38:21.294686 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 6.09 MiB, will set to 1.68 GiB (RSS), difference: -12.20 MiB 2024.10.22 01:38:21.317451 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_503_503_0, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0, 202410_507_507_0, 202410_508_508_0] 2024.10.22 01:38:21.321357 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_503_503_0, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0, 202410_507_507_0, 202410_508_508_0] 2024.10.22 01:38:21.360609 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_503_503_0, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0, 202410_507_507_0, 202410_508_508_0] 2024.10.22 01:38:21.393260 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} MergeTask::MergeProjectionsStage: Merge sorted 7076 rows, containing 13 columns (13 merged, 0 gathered) in 2.368679996 sec., 2987.3178360729485 rows/sec., 1.29 MiB/sec. 2024.10.22 01:38:21.476298 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_983_983_0 to 202410_996_996_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:21.508218 [ 1222 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_991_991_0 to 202410_996_996_0 2024.10.22 01:38:21.529658 [ 1222 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:21.537438 [ 1222 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:21.541747 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64722 2024.10.22 01:38:21.595027 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} MergedBlockOutputStream: filled checksums 202410_507_577_14 (state Temporary) 2024.10.22 01:38:21.671286 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_991_991_0 to 202410_996_996_0 into Compact with storage Full 2024.10.22 01:38:21.761154 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 145 bytes for 9f21064ae67e20f50020c7af39966447:0) 2024.10.22 01:38:21.764875 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_577_14 to 202410_507_577_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:21.765453 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fa1ac795754eef0176dc333019f534a3, offset: 0, size: 144 2024.10.22 01:38:21.768067 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fa1ac795754eef0176dc333019f534a3 for removal 2024.10.22 01:38:21.770241 [ 1476 ] {} CacheMetadata: Key fa1ac795754eef0176dc333019f534a3 is removed from metadata 2024.10.22 01:38:21.779491 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_577_14} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_572_13, 202410_577_577_0] -> 202410_507_577_14 2024.10.22 01:38:21.870582 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_990_490} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_990_490 to 202410_1_990_490 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:21.909433 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.55 MiB. 2024.10.22 01:38:21.867946 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:38:21.981081 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_990_490} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_984_489, 202410_985_990_1] -> 202410_1_990_490 2024.10.22 01:38:21.997496 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_991_991_0, total 88 rows starting from the beginning of the part 2024.10.22 01:38:22.001461 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 48.07 MiB. 2024.10.22 01:38:22.068214 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_992_992_0, total 48 rows starting from the beginning of the part 2024.10.22 01:38:22.141474 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_993_993_0, total 49 rows starting from the beginning of the part 2024.10.22 01:38:22.306866 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_994_994_0, total 63 rows starting from the beginning of the part 2024.10.22 01:38:22.333841 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_115_119_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_117_117_0, total 42 rows starting from the beginning of the part 2024.10.22 01:38:22.400069 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_995_995_0, total 42 rows starting from the beginning of the part 2024.10.22 01:38:22.485613 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_996_996_0, total 75 rows starting from the beginning of the part 2024.10.22 01:38:22.657046 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(bd2839eab57038b9cf6ac68d870fc7d1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: bd2839eab57038b9cf6ac68d870fc7d1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:38:22.664280 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 564, file segment info: File segment: [0, 33554431], key: bd2839eab57038b9cf6ac68d870fc7d1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:38:22.666503 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:22.694548 [ 1731 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gln/lebwdiracosjqaleofgyhgzptmhxt, size 5536 2024.10.22 01:38:23.155164 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:38:23.267914 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:38:23.299851 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:38:23.341838 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:38:23.388253 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:38:23.412577 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 874784 2024.10.22 01:38:23.425800 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:38:23.459361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(37b9562be805575afbaa0606fba5cfbe) : [0, 165761]: Setting download as failed: File segment: [0, 165761], key: 37b9562be805575afbaa0606fba5cfbe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:23.462550 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(37b9562be805575afbaa0606fba5cfbe): No space left in cache to reserve 165762 bytes, will continue without cache download 2024.10.22 01:38:23.465785 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:38:23.488205 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 37b9562be805575afbaa0606fba5cfbe, offset: 0, size: 0 2024.10.22 01:38:23.492325 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 37b9562be805575afbaa0606fba5cfbe for removal 2024.10.22 01:38:23.503754 [ 1476 ] {} CacheMetadata: Key 37b9562be805575afbaa0606fba5cfbe is removed from metadata 2024.10.22 01:38:23.607978 [ 1188 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:38:23.627163 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:38:23.792692 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UTMSource 2024.10.22 01:38:23.897165 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zij/rujpvsfkukkcvcvtdhvdeccvhotnz (LogSeriesLimiter: on interval from 2024-10-22 01:38:09 to 2024-10-22 01:38:23 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:38:23.986516 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} MergeTask::MergeProjectionsStage: Merge sorted 365 rows, containing 13 columns (13 merged, 0 gathered) in 2.320237181 sec., 157.31150375009872 rows/sec., 26.62 KiB/sec. 2024.10.22 01:38:24.011255 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:24.013678 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:24.039051 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} MergedBlockOutputStream: filled checksums 202410_991_996_1 (state Temporary) 2024.10.22 01:38:24.215999 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_558_558_0 (state Temporary) 2024.10.22 01:38:24.265051 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_558_558_0 to 202410_565_565_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:24.292811 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 874784 2024.10.22 01:38:24.328441 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_115_119_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_118_118_0, total 40 rows starting from the beginning of the part 2024.10.22 01:38:24.387120 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 3.88 MiB, will set to 1.67 GiB (RSS), difference: 32.83 MiB 2024.10.22 01:38:24.570355 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_991_996_1 to 202410_991_996_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:24.574263 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_991_996_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_991_991_0, 202410_996_996_0] -> 202410_991_996_1 2024.10.22 01:38:24.612600 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:38:24.584487 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(e92a51f7cbd183b911faf451db733549) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e92a51f7cbd183b911faf451db733549, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:38:24.689733 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 900459, file segment info: File segment: [0, 33554431], key: e92a51f7cbd183b911faf451db733549, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:38:24.692013 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:24.876421 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 489 2024.10.22 01:38:24.960675 [ 1266 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:38:25.087528 [ 1266 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:38:25.180405 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:25.183626 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:25.085775 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_885_453, 202410_886_891_1] 2024.10.22 01:38:25.188491 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_885_453, 202410_886_891_1] 2024.10.22 01:38:25.202173 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_885_453, 202410_886_891_1] 2024.10.22 01:38:25.341185 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_471_471_0 (state Temporary) 2024.10.22 01:38:25.424166 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_471_471_0 to 202410_481_481_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:25.462462 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_117_117_0 to 202410_120_120_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:25.502913 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_531_106, 202410_532_532_0, 202410_533_533_0, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0] 2024.10.22 01:38:25.523235 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_531_106, 202410_532_532_0, 202410_533_533_0, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0] 2024.10.22 01:38:25.576933 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 64810 2024.10.22 01:38:25.660710 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_531_106, 202410_532_532_0, 202410_533_533_0, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0] 2024.10.22 01:38:25.721398 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 145 bytes for ec07a63b53fa68a71d83bb0e5f853b01:0) 2024.10.22 01:38:25.724729 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9f21064ae67e20f50020c7af39966447, offset: 0, size: 145 2024.10.22 01:38:25.728864 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9f21064ae67e20f50020c7af39966447 for removal 2024.10.22 01:38:25.735787 [ 1476 ] {} CacheMetadata: Key 9f21064ae67e20f50020c7af39966447 is removed from metadata 2024.10.22 01:38:25.779935 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:25.784888 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:25.835856 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 489 2024.10.22 01:38:26.079044 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_984_984_0 (state Temporary) 2024.10.22 01:38:26.143177 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_984_984_0 to 202410_997_997_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:26.144421 [ 1251 ] {} DatabaseCatalog: Execution took 331 ms. 2024.10.22 01:38:26.185535 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64810 2024.10.22 01:38:26.220027 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 66301 2024.10.22 01:38:26.337814 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(df157de9f7babe97767710714308be65) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: df157de9f7babe97767710714308be65, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:38:26.448009 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 505, file segment info: File segment: [0, 33554431], key: df157de9f7babe97767710714308be65, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:38:26.451584 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:26.461771 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:26.469415 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:26.491626 [ 1681 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/otn/tjojijlebekgqazhenqkofvpnrbsl, size 900459 2024.10.22 01:38:26.835664 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_572_572_0 (state Temporary) 2024.10.22 01:38:26.914982 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_115_119_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_119_119_0, total 40 rows starting from the beginning of the part 2024.10.22 01:38:26.994256 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_572_572_0 to 202410_578_578_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:27.029118 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:38:27.075541 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 66301 2024.10.22 01:38:27.181060 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4e8d4f3e5ad43d5d298a35d859a8a88c) : [0, 142462]: Setting download as failed: File segment: [0, 142462], key: 4e8d4f3e5ad43d5d298a35d859a8a88c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:27.183472 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4e8d4f3e5ad43d5d298a35d859a8a88c): No space left in cache to reserve 142463 bytes, will continue without cache download 2024.10.22 01:38:27.225514 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:38:27.257722 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:38:27.274884 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:38:27.395517 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:38:27.414622 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4e8d4f3e5ad43d5d298a35d859a8a88c, offset: 0, size: 0 2024.10.22 01:38:27.417608 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:38:27.418957 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4e8d4f3e5ad43d5d298a35d859a8a88c for removal 2024.10.22 01:38:27.424486 [ 1476 ] {} CacheMetadata: Key 4e8d4f3e5ad43d5d298a35d859a8a88c is removed from metadata 2024.10.22 01:38:27.458583 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:38:27.470928 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:38:27.491890 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 01:38:27.707582 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 24 entries to flush up to offset 8982 2024.10.22 01:38:27.780882 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4699 2024.10.22 01:38:27.851495 [ 1268 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:38:27.908778 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:27.912353 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:28.066152 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_583_583_0 (state Temporary) 2024.10.22 01:38:28.370893 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_583_583_0 to 202410_596_596_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:28.433664 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8982 2024.10.22 01:38:28.463121 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 145 bytes for e0d373602b1c09475fff7689173d6c35:0) 2024.10.22 01:38:28.464915 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ec07a63b53fa68a71d83bb0e5f853b01, offset: 0, size: 145 2024.10.22 01:38:28.466430 [ 1240 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_591_118 to 202410_596_596_0 2024.10.22 01:38:28.467500 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ec07a63b53fa68a71d83bb0e5f853b01 for removal 2024.10.22 01:38:28.472104 [ 1476 ] {} CacheMetadata: Key ec07a63b53fa68a71d83bb0e5f853b01 is removed from metadata 2024.10.22 01:38:28.473375 [ 1240 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:28.476478 [ 1240 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:28.492085 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_591_118 to 202410_596_596_0 into Compact with storage Full 2024.10.22 01:38:28.519839 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:38:28.530906 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_591_118, total 9004 rows starting from the beginning of the part 2024.10.22 01:38:28.551893 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_592_592_0, total 20 rows starting from the beginning of the part 2024.10.22 01:38:28.565073 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_593_593_0, total 26 rows starting from the beginning of the part 2024.10.22 01:38:28.577122 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_594_594_0, total 5 rows starting from the beginning of the part 2024.10.22 01:38:28.592437 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_595_595_0, total 9 rows starting from the beginning of the part 2024.10.22 01:38:28.613758 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_596_596_0, total 24 rows starting from the beginning of the part 2024.10.22 01:38:28.671188 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 6.78 MiB, will set to 1.67 GiB (RSS), difference: 4.32 MiB 2024.10.22 01:38:29.092300 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_496_100, 202410_497_502_1] 2024.10.22 01:38:29.097405 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_496_100, 202410_497_502_1] 2024.10.22 01:38:29.125701 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_496_100, 202410_497_502_1] 2024.10.22 01:38:29.175180 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(931e5118a1d2d2c47fb3d0d0b2d10336) : [0, 129498]: Setting download as failed: File segment: [0, 129498], key: 931e5118a1d2d2c47fb3d0d0b2d10336, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:29.176401 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(931e5118a1d2d2c47fb3d0d0b2d10336): No space left in cache to reserve 129499 bytes, will continue without cache download 2024.10.22 01:38:29.295609 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 931e5118a1d2d2c47fb3d0d0b2d10336, offset: 0, size: 0 2024.10.22 01:38:29.301199 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 931e5118a1d2d2c47fb3d0d0b2d10336 for removal 2024.10.22 01:38:29.304989 [ 1476 ] {} CacheMetadata: Key 931e5118a1d2d2c47fb3d0d0b2d10336 is removed from metadata 2024.10.22 01:38:29.506565 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.047505949 sec. 2024.10.22 01:38:29.630971 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} MergeTask::MergeProjectionsStage: Merge sorted 9088 rows, containing 12 columns (12 merged, 0 gathered) in 1.146314637 sec., 7928.015316792994 rows/sec., 1.23 MiB/sec. 2024.10.22 01:38:29.762342 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} MergedBlockOutputStream: filled checksums 202410_1_596_119 (state Temporary) 2024.10.22 01:38:30.060882 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_596_119 to 202410_1_596_119 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:30.064426 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_596_119} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_591_118, 202410_596_596_0] -> 202410_1_596_119 2024.10.22 01:38:30.086291 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.54 MiB. 2024.10.22 01:38:30.121479 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for 547e2860d841e87244e8e96a6b61b01d:0) 2024.10.22 01:38:30.124086 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e0d373602b1c09475fff7689173d6c35, offset: 0, size: 145 2024.10.22 01:38:30.127775 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e0d373602b1c09475fff7689173d6c35 for removal 2024.10.22 01:38:30.132358 [ 1476 ] {} CacheMetadata: Key e0d373602b1c09475fff7689173d6c35 is removed from metadata 2024.10.22 01:38:30.270713 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_891_454, 202410_892_892_0] 2024.10.22 01:38:30.332152 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 64890 2024.10.22 01:38:30.334413 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_891_454, 202410_892_892_0] 2024.10.22 01:38:30.349744 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_891_454, 202410_892_892_0] 2024.10.22 01:38:30.543879 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:38:30.550715 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:38:30.573756 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 6.23 MiB, will set to 1.68 GiB (RSS), difference: 4.05 MiB 2024.10.22 01:38:30.772090 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:30.796728 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_428_88, 202410_429_429_0, 202410_430_430_0, 202410_431_431_0, 202410_432_432_0, 202410_433_433_0] 2024.10.22 01:38:30.799049 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_428_88, 202410_429_429_0, 202410_430_430_0, 202410_431_431_0, 202410_432_432_0, 202410_433_433_0] 2024.10.22 01:38:30.845311 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_428_88, 202410_429_429_0, 202410_430_430_0, 202410_431_431_0, 202410_432_432_0, 202410_433_433_0] 2024.10.22 01:38:30.877360 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:38:30.993404 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_985_985_0 (state Temporary) 2024.10.22 01:38:31.062013 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_985_985_0 to 202410_998_998_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:31.111597 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64890 2024.10.22 01:38:31.316118 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c0ab3923e12d29ee2e2bc5f9d4248f74) : [0, 133935]: Setting download as failed: File segment: [0, 133935], key: c0ab3923e12d29ee2e2bc5f9d4248f74, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:31.319058 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c0ab3923e12d29ee2e2bc5f9d4248f74): No space left in cache to reserve 133936 bytes, will continue without cache download 2024.10.22 01:38:31.503304 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 825 entries to flush up to offset 875609 2024.10.22 01:38:31.542992 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c0ab3923e12d29ee2e2bc5f9d4248f74, offset: 0, size: 0 2024.10.22 01:38:31.545448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c0ab3923e12d29ee2e2bc5f9d4248f74 for removal 2024.10.22 01:38:31.550815 [ 1476 ] {} CacheMetadata: Key c0ab3923e12d29ee2e2bc5f9d4248f74 is removed from metadata 2024.10.22 01:38:32.164699 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_893_893_0, 202410_894_894_0, 202410_895_895_0, 202410_896_896_0, 202410_897_897_0, 202410_898_898_0] 2024.10.22 01:38:32.166587 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_893_893_0, 202410_894_894_0, 202410_895_895_0, 202410_896_896_0, 202410_897_897_0, 202410_898_898_0] 2024.10.22 01:38:32.204061 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_893_893_0, 202410_894_894_0, 202410_895_895_0, 202410_896_896_0, 202410_897_897_0, 202410_898_898_0] 2024.10.22 01:38:32.602526 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 156 bytes for 3074fb71d321d40f1ad1693fd4ab2bec:0) 2024.10.22 01:38:32.607400 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 547e2860d841e87244e8e96a6b61b01d, offset: 0, size: 155 2024.10.22 01:38:32.612468 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 547e2860d841e87244e8e96a6b61b01d for removal 2024.10.22 01:38:32.615384 [ 1476 ] {} CacheMetadata: Key 547e2860d841e87244e8e96a6b61b01d is removed from metadata 2024.10.22 01:38:32.716544 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:32.724049 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:38:32.930795 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_559_559_0 (state Temporary) 2024.10.22 01:38:33.073325 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_insert_202410_559_559_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:38:33.191107 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_559_559_0 to 202410_566_566_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:33.234489 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 875609 2024.10.22 01:38:33.238668 [ 1168 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_556_110 to 202410_557_562_1 2024.10.22 01:38:33.257780 [ 1168 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:33.260265 [ 1168 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.03 GiB. 2024.10.22 01:38:33.271711 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_562_111} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_556_110 to 202410_557_562_1 into Wide with storage Full 2024.10.22 01:38:33.311769 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_562_111} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:38:33.320577 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_562_111} MergeTreeSequentialSource: Reading 109 marks from part 202410_1_556_110, total 876824 rows starting from the beginning of the part 2024.10.22 01:38:33.339049 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 490 2024.10.22 01:38:33.366575 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_562_111} MergeTreeSequentialSource: Reading 3 marks from part 202410_557_562_1, total 8470 rows starting from the beginning of the part 2024.10.22 01:38:33.456447 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d05e72d1c9248e3decd9758233326619) : [0, 148436]: Setting download as failed: File segment: [0, 148436], key: d05e72d1c9248e3decd9758233326619, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:33.457958 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d05e72d1c9248e3decd9758233326619): No space left in cache to reserve 148437 bytes, will continue without cache download 2024.10.22 01:38:33.469695 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d05e72d1c9248e3decd9758233326619, offset: 0, size: 0 2024.10.22 01:38:33.471821 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d05e72d1c9248e3decd9758233326619 for removal 2024.10.22 01:38:33.475995 [ 1476 ] {} CacheMetadata: Key d05e72d1c9248e3decd9758233326619 is removed from metadata 2024.10.22 01:38:33.585453 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:33.588127 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:33.745621 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_472_472_0 (state Temporary) 2024.10.22 01:38:33.815169 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_472_472_0 to 202410_482_482_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:34.015396 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 490 2024.10.22 01:38:34.364657 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 3.26 MiB, will set to 1.68 GiB (RSS), difference: -13.72 MiB 2024.10.22 01:38:34.410926 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for 2062596c9d64b024b5062d6856045351:0) 2024.10.22 01:38:34.413350 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3074fb71d321d40f1ad1693fd4ab2bec, offset: 0, size: 156 2024.10.22 01:38:34.416838 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3074fb71d321d40f1ad1693fd4ab2bec for removal 2024.10.22 01:38:34.424331 [ 1476 ] {} CacheMetadata: Key 3074fb71d321d40f1ad1693fd4ab2bec is removed from metadata 2024.10.22 01:38:34.683259 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 66416 2024.10.22 01:38:34.926142 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:34.929046 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:35.138347 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 64950 2024.10.22 01:38:35.184898 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_573_573_0 (state Temporary) 2024.10.22 01:38:35.377898 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:35.380970 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:35.488706 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(046fc442a62c483a765e4253d2033611) : [0, 84854]: Setting download as failed: File segment: [0, 84854], key: 046fc442a62c483a765e4253d2033611, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:35.493824 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(046fc442a62c483a765e4253d2033611): No space left in cache to reserve 84855 bytes, will continue without cache download 2024.10.22 01:38:35.533587 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 046fc442a62c483a765e4253d2033611, offset: 0, size: 0 2024.10.22 01:38:35.537367 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 046fc442a62c483a765e4253d2033611 for removal 2024.10.22 01:38:35.547511 [ 1476 ] {} CacheMetadata: Key 046fc442a62c483a765e4253d2033611 is removed from metadata 2024.10.22 01:38:35.559774 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_986_986_0 (state Temporary) 2024.10.22 01:38:35.544968 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 35 entries to flush up to offset 4734 2024.10.22 01:38:35.582272 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_573_573_0 to 202410_579_579_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:35.598982 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_986_986_0 to 202410_999_999_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:35.662654 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 66416 2024.10.22 01:38:35.769349 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64950 2024.10.22 01:38:35.818855 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_115_119_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:38:35.940314 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 8995 2024.10.22 01:38:36.037232 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:36.044218 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:36.077840 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_584_584_0 (state Temporary) 2024.10.22 01:38:36.271657 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_584_584_0 to 202410_597_597_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:36.431502 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8995 2024.10.22 01:38:36.557632 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 157 bytes for ce2ebab61052ffc1168486143da401c5:0) 2024.10.22 01:38:36.561158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2062596c9d64b024b5062d6856045351, offset: 0, size: 155 2024.10.22 01:38:36.564844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2062596c9d64b024b5062d6856045351 for removal 2024.10.22 01:38:36.567707 [ 1476 ] {} CacheMetadata: Key 2062596c9d64b024b5062d6856045351 is removed from metadata 2024.10.22 01:38:36.553833 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 9.27 MiB, will set to 1.68 GiB (RSS), difference: 8.74 MiB 2024.10.22 01:38:36.679900 [ 1270 ] {} DatabaseCatalog: Execution took 530 ms. 2024.10.22 01:38:37.871571 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(3a009de3d5821f9639ed3193caf89aa4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3a009de3d5821f9639ed3193caf89aa4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:38:37.874454 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 3a009de3d5821f9639ed3193caf89aa4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:38:37.876210 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:38.419094 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 12.01 MiB, will set to 1.69 GiB (RSS), difference: 6.40 MiB 2024.10.22 01:38:38.553444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6f2c3e52a42e5d626aa2b763e39baa02) : [0, 991870]: Setting download as failed: File segment: [0, 991870], key: 6f2c3e52a42e5d626aa2b763e39baa02, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:38.656620 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6f2c3e52a42e5d626aa2b763e39baa02): No space left in cache to reserve 991871 bytes, will continue without cache download 2024.10.22 01:38:38.682592 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6f2c3e52a42e5d626aa2b763e39baa02, offset: 0, size: 0 2024.10.22 01:38:38.684842 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6f2c3e52a42e5d626aa2b763e39baa02 for removal 2024.10.22 01:38:38.687441 [ 1476 ] {} CacheMetadata: Key 6f2c3e52a42e5d626aa2b763e39baa02 is removed from metadata 2024.10.22 01:38:39.739901 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for 7a2796ec15c32c4be48926b600e32986:0) 2024.10.22 01:38:39.744015 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ce2ebab61052ffc1168486143da401c5, offset: 0, size: 157 2024.10.22 01:38:39.751954 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ce2ebab61052ffc1168486143da401c5 for removal 2024.10.22 01:38:39.755402 [ 1476 ] {} CacheMetadata: Key ce2ebab61052ffc1168486143da401c5 is removed from metadata 2024.10.22 01:38:39.776847 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 64991 2024.10.22 01:38:40.095266 [ 1246 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:38:40.237895 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1870 entries to flush up to offset 877479 2024.10.22 01:38:40.258832 [ 1246 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:38:40.354241 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:40.358940 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:40.509714 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_987_987_0 (state Temporary) 2024.10.22 01:38:40.545646 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:38:40.551678 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:38:40.557500 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_987_987_0 to 202410_1000_1000_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:40.570417 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 11.13 MiB, will set to 1.69 GiB (RSS), difference: 2.04 MiB 2024.10.22 01:38:40.685446 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(eb3da07d825dffed4b6e63741f7f84c7) : [0, 49587]: Setting download as failed: File segment: [0, 49587], key: eb3da07d825dffed4b6e63741f7f84c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:40.687202 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(eb3da07d825dffed4b6e63741f7f84c7): No space left in cache to reserve 49588 bytes, will continue without cache download 2024.10.22 01:38:40.712492 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: eb3da07d825dffed4b6e63741f7f84c7, offset: 0, size: 0 2024.10.22 01:38:40.713372 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key eb3da07d825dffed4b6e63741f7f84c7 for removal 2024.10.22 01:38:40.715739 [ 1476 ] {} CacheMetadata: Key eb3da07d825dffed4b6e63741f7f84c7 is removed from metadata 2024.10.22 01:38:40.716311 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 64991 2024.10.22 01:38:41.420025 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:41.423773 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:41.519060 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 491 2024.10.22 01:38:41.644449 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_560_560_0 (state Temporary) 2024.10.22 01:38:41.831464 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_560_560_0 to 202410_567_567_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:41.883344 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for d3fb92d3646e3edc45678a871f905739:0) 2024.10.22 01:38:41.886435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7a2796ec15c32c4be48926b600e32986, offset: 0, size: 144 2024.10.22 01:38:41.897071 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7a2796ec15c32c4be48926b600e32986 for removal 2024.10.22 01:38:41.903150 [ 1476 ] {} CacheMetadata: Key 7a2796ec15c32c4be48926b600e32986 is removed from metadata 2024.10.22 01:38:41.973603 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 877479 2024.10.22 01:38:42.072945 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:42.077160 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:42.143559 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_473_473_0 (state Temporary) 2024.10.22 01:38:42.201081 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_473_473_0 to 202410_483_483_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:42.235993 [ 1219 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_478_98 to 202410_483_483_0 2024.10.22 01:38:42.269748 [ 1219 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:42.273551 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:42.384399 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 491 2024.10.22 01:38:42.422875 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_478_98 to 202410_483_483_0 into Compact with storage Full 2024.10.22 01:38:42.639426 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 14.78 MiB, will set to 1.69 GiB (RSS), difference: 4.10 MiB 2024.10.22 01:38:42.754998 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ef02adbcec8d11b965f07a5d24d026fe) : [0, 41176]: Setting download as failed: File segment: [0, 41176], key: ef02adbcec8d11b965f07a5d24d026fe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:42.756176 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ef02adbcec8d11b965f07a5d24d026fe): No space left in cache to reserve 41177 bytes, will continue without cache download 2024.10.22 01:38:42.766394 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ef02adbcec8d11b965f07a5d24d026fe, offset: 0, size: 0 2024.10.22 01:38:42.769311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ef02adbcec8d11b965f07a5d24d026fe for removal 2024.10.22 01:38:42.771854 [ 1476 ] {} CacheMetadata: Key ef02adbcec8d11b965f07a5d24d026fe is removed from metadata 2024.10.22 01:38:42.992535 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:38:43.028238 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_478_98, total 522 rows starting from the beginning of the part 2024.10.22 01:38:43.175324 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 54 entries to flush up to offset 66470 2024.10.22 01:38:43.281282 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_479_479_0, total 1 rows starting from the beginning of the part 2024.10.22 01:38:43.488182 [ 1154 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 202 ms. 2024.10.22 01:38:43.560298 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_480_480_0, total 1 rows starting from the beginning of the part 2024.10.22 01:38:43.575372 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:43.578456 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:43.608223 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_562_111/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:38:43.711224 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(46f5ab324c71650c46104464f092d870) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 46f5ab324c71650c46104464f092d870, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:38:43.716536 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 648, file segment info: File segment: [0, 33554431], key: 46f5ab324c71650c46104464f092d870, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:38:43.719532 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:43.826735 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_574_574_0 (state Temporary) 2024.10.22 01:38:43.838642 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:38:43.859783 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:38:43.871350 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:38:43.886971 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:38:43.937211 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 4 entries to flush up to offset 8999 2024.10.22 01:38:43.971160 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_574_574_0 to 202410_580_580_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:43.975657 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_481_481_0, total 1 rows starting from the beginning of the part 2024.10.22 01:38:44.005544 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 66470 2024.10.22 01:38:44.007337 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:38:44.029949 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:38:44.059594 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:38:44.118119 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 01:38:44.169442 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tdg/vfoipsuwzibnysqtxhgzeqmgfwziz (LogSeriesLimiter: on interval from 2024-10-22 01:38:23 to 2024-10-22 01:38:44 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:38:44.326908 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_482_482_0, total 1 rows starting from the beginning of the part 2024.10.22 01:38:44.363038 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:44.366588 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:44.400829 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_585_585_0 (state Temporary) 2024.10.22 01:38:44.565617 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_585_585_0 to 202410_598_598_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:44.649812 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 8999 2024.10.22 01:38:44.668277 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_483_483_0, total 1 rows starting from the beginning of the part 2024.10.22 01:38:44.722283 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 65066 2024.10.22 01:38:44.846720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 143 bytes for ce4d5c61aed9da9987b8457957028058:0) 2024.10.22 01:38:44.849624 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d3fb92d3646e3edc45678a871f905739, offset: 0, size: 144 2024.10.22 01:38:44.856622 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d3fb92d3646e3edc45678a871f905739 for removal 2024.10.22 01:38:44.859542 [ 1476 ] {} CacheMetadata: Key d3fb92d3646e3edc45678a871f905739 is removed from metadata 2024.10.22 01:38:45.010901 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_115_119_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:38:45.172474 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:45.270924 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:45.323606 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 13.17 MiB, will set to 1.69 GiB (RSS), difference: -11.87 MiB 2024.10.22 01:38:45.352668 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_988_988_0 (state Temporary) 2024.10.22 01:38:45.408296 [ 1262 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 244 ms. 2024.10.22 01:38:45.526922 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_988_988_0 to 202410_1001_1001_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:45.613142 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65066 2024.10.22 01:38:45.664001 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_990_490 to 202410_991_996_1 2024.10.22 01:38:45.673316 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.43 MiB using storage policy from min volume index 0 2024.10.22 01:38:45.675537 [ 1195 ] {} DiskLocal: Reserved 2.43 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:45.695540 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_996_491} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_990_490 to 202410_991_996_1 into Wide with storage Full 2024.10.22 01:38:45.769482 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9fae9d7d1354d382bde9c9ed14acdfe2) : [0, 67614]: Setting download as failed: File segment: [0, 67614], key: 9fae9d7d1354d382bde9c9ed14acdfe2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:45.772194 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9fae9d7d1354d382bde9c9ed14acdfe2): No space left in cache to reserve 67615 bytes, will continue without cache download 2024.10.22 01:38:45.798851 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9fae9d7d1354d382bde9c9ed14acdfe2, offset: 0, size: 0 2024.10.22 01:38:45.800962 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9fae9d7d1354d382bde9c9ed14acdfe2 for removal 2024.10.22 01:38:45.879322 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_996_491} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:38:45.897095 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_996_491} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_990_490, total 65188 rows starting from the beginning of the part 2024.10.22 01:38:45.907334 [ 1476 ] {} CacheMetadata: Key 9fae9d7d1354d382bde9c9ed14acdfe2 is removed from metadata 2024.10.22 01:38:46.512468 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_996_491} MergeTreeSequentialSource: Reading 2 marks from part 202410_991_996_1, total 365 rows starting from the beginning of the part 2024.10.22 01:38:46.865837 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 143 bytes for d0792a346be78059cc1d013971016a90:0) 2024.10.22 01:38:46.868883 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ce4d5c61aed9da9987b8457957028058, offset: 0, size: 143 2024.10.22 01:38:46.870705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ce4d5c61aed9da9987b8457957028058 for removal 2024.10.22 01:38:46.872525 [ 1476 ] {} CacheMetadata: Key ce4d5c61aed9da9987b8457957028058 is removed from metadata 2024.10.22 01:38:47.056408 [ 1171 ] {} DatabaseCatalog: Execution took 373 ms. 2024.10.22 01:38:47.663344 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 14.32 MiB, will set to 1.69 GiB (RSS), difference: -35.50 MiB 2024.10.22 01:38:47.739537 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} MergeTask::MergeProjectionsStage: Merge sorted 527 rows, containing 32 columns (32 merged, 0 gathered) in 5.327919672 sec., 98.91290267936307 rows/sec., 9.11 KiB/sec. 2024.10.22 01:38:47.784336 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} MergedBlockOutputStream: filled checksums 202410_1_483_99 (state Temporary) 2024.10.22 01:38:48.166884 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dc72ade38da193ffb24e407b98478c07) : [0, 101902]: Setting download as failed: File segment: [0, 101902], key: dc72ade38da193ffb24e407b98478c07, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:48.170540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dc72ade38da193ffb24e407b98478c07): No space left in cache to reserve 101903 bytes, will continue without cache download 2024.10.22 01:38:48.224907 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dc72ade38da193ffb24e407b98478c07, offset: 0, size: 0 2024.10.22 01:38:48.227278 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dc72ade38da193ffb24e407b98478c07 for removal 2024.10.22 01:38:48.230323 [ 1476 ] {} CacheMetadata: Key dc72ade38da193ffb24e407b98478c07 is removed from metadata 2024.10.22 01:38:48.437581 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_483_99 to 202410_1_483_99 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:48.461494 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_483_99} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_478_98, 202410_483_483_0] -> 202410_1_483_99 2024.10.22 01:38:48.608998 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:38:48.787739 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(45f406d7b82d80d8ed5acc1ac2907f2d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 45f406d7b82d80d8ed5acc1ac2907f2d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:38:48.792315 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 78839, file segment info: File segment: [0, 33554431], key: 45f406d7b82d80d8ed5acc1ac2907f2d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:38:48.799006 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:48.977472 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 878634 2024.10.22 01:38:49.725478 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 145 bytes for 3ceff10092ccdee3da690da2de3246a4:0) 2024.10.22 01:38:49.747582 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 65111 2024.10.22 01:38:49.748896 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d0792a346be78059cc1d013971016a90, offset: 0, size: 143 2024.10.22 01:38:49.753494 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d0792a346be78059cc1d013971016a90 for removal 2024.10.22 01:38:49.755926 [ 1476 ] {} CacheMetadata: Key d0792a346be78059cc1d013971016a90 is removed from metadata 2024.10.22 01:38:49.914816 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:49.919423 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:50.126028 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_561_561_0 (state Temporary) 2024.10.22 01:38:50.180044 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:50.184112 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:50.190354 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_561_561_0 to 202410_568_568_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:50.231321 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_989_989_0 (state Temporary) 2024.10.22 01:38:50.213362 [ 1183 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_563_563_0 to 202410_568_568_0 2024.10.22 01:38:50.246833 [ 1183 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:50.252034 [ 1183 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:50.265970 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_563_563_0 to 202410_568_568_0 into Compact with storage Full 2024.10.22 01:38:50.270372 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_989_989_0 to 202410_1002_1002_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:50.292125 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_997_997_0 to 202410_1002_1002_0 2024.10.22 01:38:50.306710 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:50.309507 [ 1173 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:50.331138 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 878634 2024.10.22 01:38:50.347936 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(6e5213e822c6083487f5663f01400153) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6e5213e822c6083487f5663f01400153, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:38:50.353113 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 868, file segment info: File segment: [0, 33554431], key: 6e5213e822c6083487f5663f01400153, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:38:50.359270 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:50.418832 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:38:50.422734 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65111 2024.10.22 01:38:50.435835 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_563_563_0, total 770 rows starting from the beginning of the part 2024.10.22 01:38:50.457617 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_564_564_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:38:50.466378 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_565_565_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:38:50.483307 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_566_566_0, total 825 rows starting from the beginning of the part 2024.10.22 01:38:50.515091 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_567_567_0, total 1870 rows starting from the beginning of the part 2024.10.22 01:38:50.542150 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_568_568_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:38:50.548493 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:38:50.556360 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:38:50.612006 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_997_997_0 to 202410_1002_1002_0 into Compact with storage Full 2024.10.22 01:38:50.669321 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:38:50.679955 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_997_997_0, total 88 rows starting from the beginning of the part 2024.10.22 01:38:50.708621 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_998_998_0, total 80 rows starting from the beginning of the part 2024.10.22 01:38:50.770731 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_999_999_0, total 60 rows starting from the beginning of the part 2024.10.22 01:38:50.809771 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:50.814507 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:50.815869 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1000_1000_0, total 41 rows starting from the beginning of the part 2024.10.22 01:38:50.815404 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_996_491} MergeTask::MergeProjectionsStage: Merge sorted 65553 rows, containing 13 columns (13 merged, 0 gathered) in 5.132546217 sec., 12772.023324967948 rows/sec., 2.10 MiB/sec. 2024.10.22 01:38:50.900423 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_996_491} MergedBlockOutputStream: filled checksums 202410_1_996_491 (state Temporary) 2024.10.22 01:38:50.986145 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1001_1001_0, total 75 rows starting from the beginning of the part 2024.10.22 01:38:51.153409 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1002_1002_0, total 45 rows starting from the beginning of the part 2024.10.22 01:38:51.406514 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e45eca31ebc7361b15d6342fa4dd31c3) : [0, 85894]: Setting download as failed: File segment: [0, 85894], key: e45eca31ebc7361b15d6342fa4dd31c3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:51.410433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e45eca31ebc7361b15d6342fa4dd31c3): No space left in cache to reserve 85895 bytes, will continue without cache download 2024.10.22 01:38:51.456450 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 13.20 MiB, will set to 1.69 GiB (RSS), difference: 22.59 MiB 2024.10.22 01:38:51.458375 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e45eca31ebc7361b15d6342fa4dd31c3, offset: 0, size: 0 2024.10.22 01:38:51.469773 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e45eca31ebc7361b15d6342fa4dd31c3 for removal 2024.10.22 01:38:51.471026 [ 1476 ] {} CacheMetadata: Key e45eca31ebc7361b15d6342fa4dd31c3 is removed from metadata 2024.10.22 01:38:51.515430 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 195 entries to flush up to offset 66665 2024.10.22 01:38:51.601571 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} MergeTask::MergeProjectionsStage: Merge sorted 6930 rows, containing 4 columns (4 merged, 0 gathered) in 1.341395073 sec., 5166.263198284462 rows/sec., 80.72 KiB/sec. 2024.10.22 01:38:51.663773 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} MergedBlockOutputStream: filled checksums 202410_563_568_1 (state Temporary) 2024.10.22 01:38:51.715224 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_996_491} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_996_491 to 202410_1_996_491 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:51.723106 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_996_491} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_990_490, 202410_991_996_1] -> 202410_1_996_491 2024.10.22 01:38:51.758664 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.35 MiB. 2024.10.22 01:38:51.868193 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_563_568_1 to 202410_563_568_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:51.881792 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_563_568_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_563_563_0, 202410_568_568_0] -> 202410_563_568_1 2024.10.22 01:38:51.898429 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:51.906416 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:51.918969 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:38:51.958704 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_575_575_0 (state Temporary) 2024.10.22 01:38:52.033647 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_575_575_0 to 202410_581_581_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:52.103678 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 66665 2024.10.22 01:38:52.152074 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 9022 2024.10.22 01:38:52.251255 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_517_2, 202410_518_518_0, 202410_519_519_0, 202410_520_520_0, 202410_521_521_0, 202410_522_522_0] 2024.10.22 01:38:52.256594 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_517_2, 202410_518_518_0, 202410_519_519_0, 202410_520_520_0, 202410_521_521_0, 202410_522_522_0] 2024.10.22 01:38:52.264599 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:52.266874 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:52.316077 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_586_586_0 (state Temporary) 2024.10.22 01:38:52.371092 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_997_1002_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:38:52.391165 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_586_586_0 to 202410_599_599_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:52.466658 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9022 2024.10.22 01:38:52.494347 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_517_2, 202410_518_518_0, 202410_519_519_0, 202410_520_520_0, 202410_521_521_0, 202410_522_522_0] 2024.10.22 01:38:52.760449 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} MergeTask::MergeProjectionsStage: Merge sorted 389 rows, containing 13 columns (13 merged, 0 gathered) in 2.266585269 sec., 171.62381019604163 rows/sec., 29.92 KiB/sec. 2024.10.22 01:38:52.771746 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(6488f33ca90ffaa5af6c1bcd86c5cfd0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6488f33ca90ffaa5af6c1bcd86c5cfd0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:38:52.775342 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 46055, file segment info: File segment: [0, 33554431], key: 6488f33ca90ffaa5af6c1bcd86c5cfd0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:38:52.782564 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:52.789132 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} MergedBlockOutputStream: filled checksums 202410_997_1002_1 (state Temporary) 2024.10.22 01:38:52.852848 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_997_1002_1 to 202410_997_1002_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:52.857376 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_997_1002_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_997_997_0, 202410_1002_1002_0] -> 202410_997_1002_1 2024.10.22 01:38:52.878784 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:38:53.688963 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 145 bytes for 6a5b80229e3ce8e7f423453f1d6a52e4:0) 2024.10.22 01:38:53.692805 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3ceff10092ccdee3da690da2de3246a4, offset: 0, size: 145 2024.10.22 01:38:53.799003 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3ceff10092ccdee3da690da2de3246a4 for removal 2024.10.22 01:38:53.804379 [ 1476 ] {} CacheMetadata: Key 3ceff10092ccdee3da690da2de3246a4 is removed from metadata 2024.10.22 01:38:54.419973 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_118_118_0 (state Temporary) 2024.10.22 01:38:54.452083 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 65200 2024.10.22 01:38:54.622489 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(94025f4a68c64eca351c2ef17bb93212) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 94025f4a68c64eca351c2ef17bb93212, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:38:54.625205 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 94025f4a68c64eca351c2ef17bb93212, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:38:54.628114 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:54.772972 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_562_111/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:38:54.848636 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:54.852090 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:55.015384 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_990_990_0 (state Temporary) 2024.10.22 01:38:55.183264 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_990_990_0 to 202410_1003_1003_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:55.255112 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65200 2024.10.22 01:38:55.263125 [ 1231 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:38:55.279711 [ 1231 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:38:55.311437 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 14.86 MiB, will set to 1.69 GiB (RSS), difference: 14.12 MiB 2024.10.22 01:38:55.881172 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(93e458e593c8d13af148507e000ea30c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 93e458e593c8d13af148507e000ea30c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:38:55.883703 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 753, file segment info: File segment: [0, 33554431], key: 93e458e593c8d13af148507e000ea30c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:38:55.886586 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:38:55.935446 [ 1684 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zij/rujpvsfkukkcvcvtdhvdeccvhotnz, size 46055 2024.10.22 01:38:56.292085 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:38:56.316537 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:38:56.351692 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:38:56.376506 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:38:56.584828 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f3d64f1a0173fc0b5c35c467079de494) : [0, 113590]: Setting download as failed: File segment: [0, 113590], key: f3d64f1a0173fc0b5c35c467079de494, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:56.586872 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f3d64f1a0173fc0b5c35c467079de494): No space left in cache to reserve 113591 bytes, will continue without cache download 2024.10.22 01:38:56.616086 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:38:56.626225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f3d64f1a0173fc0b5c35c467079de494, offset: 0, size: 0 2024.10.22 01:38:56.629128 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f3d64f1a0173fc0b5c35c467079de494 for removal 2024.10.22 01:38:56.646830 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:38:56.651590 [ 1476 ] {} CacheMetadata: Key f3d64f1a0173fc0b5c35c467079de494 is removed from metadata 2024.10.22 01:38:56.677739 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:38:56.712897 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:38:56.866116 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UTMMedium 2024.10.22 01:38:57.021576 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ysn/zwtdxtrhoxjzxdbwynspkvuymhlqz (LogSeriesLimiter: on interval from 2024-10-22 01:38:44 to 2024-10-22 01:38:57 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:38:57.341027 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 879789 2024.10.22 01:38:57.374188 [ 1210 ] {} DatabaseCatalog: Execution took 317 ms. 2024.10.22 01:38:57.387717 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 492 2024.10.22 01:38:57.568874 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 168 bytes for 2b9a591291efdbb19df3d5eabf030bb9:0) 2024.10.22 01:38:57.571549 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6a5b80229e3ce8e7f423453f1d6a52e4, offset: 0, size: 145 2024.10.22 01:38:57.576040 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6a5b80229e3ce8e7f423453f1d6a52e4 for removal 2024.10.22 01:38:57.578010 [ 1476 ] {} CacheMetadata: Key 6a5b80229e3ce8e7f423453f1d6a52e4 is removed from metadata 2024.10.22 01:38:57.785010 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:57.791028 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:57.861520 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_474_474_0 (state Temporary) 2024.10.22 01:38:58.060937 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_474_474_0 to 202410_484_484_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:58.090274 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:58.093377 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:58.328445 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_562_562_0 (state Temporary) 2024.10.22 01:38:58.343397 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_892_455, 202410_893_898_1] 2024.10.22 01:38:58.352177 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_892_455, 202410_893_898_1] 2024.10.22 01:38:58.354604 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 492 2024.10.22 01:38:58.364426 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_892_455, 202410_893_898_1] 2024.10.22 01:38:58.378063 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5ab739ae4526bab4c7e872523319b03c) : [0, 294991]: Setting download as failed: File segment: [0, 294991], key: 5ab739ae4526bab4c7e872523319b03c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:38:58.380325 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5ab739ae4526bab4c7e872523319b03c): No space left in cache to reserve 294992 bytes, will continue without cache download 2024.10.22 01:38:58.468002 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_562_562_0 to 202410_569_569_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:38:58.505298 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 879789 2024.10.22 01:38:58.550365 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5ab739ae4526bab4c7e872523319b03c, offset: 0, size: 0 2024.10.22 01:38:58.552839 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5ab739ae4526bab4c7e872523319b03c for removal 2024.10.22 01:38:58.556275 [ 1476 ] {} CacheMetadata: Key 5ab739ae4526bab4c7e872523319b03c is removed from metadata 2024.10.22 01:38:58.650759 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 11.63 MiB, will set to 1.69 GiB (RSS), difference: -5.26 MiB 2024.10.22 01:38:58.726255 [ 1239 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 244 ms. 2024.10.22 01:38:59.345297 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 65259 2024.10.22 01:38:59.422860 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_118_118_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:38:59.571666 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_562_111} MergeTask::MergeProjectionsStage: Merge sorted 885294 rows, containing 4 columns (4 merged, 0 gathered) in 26.304728023 sec., 33655.31851254754 rows/sec., 504.87 KiB/sec. 2024.10.22 01:38:59.658794 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 66736 2024.10.22 01:38:59.679642 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 153 bytes for d485237b741633fe0f5ec30b453f9531:0) 2024.10.22 01:38:59.681814 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2b9a591291efdbb19df3d5eabf030bb9, offset: 0, size: 168 2024.10.22 01:38:59.685789 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2b9a591291efdbb19df3d5eabf030bb9 for removal 2024.10.22 01:38:59.691131 [ 1476 ] {} CacheMetadata: Key 2b9a591291efdbb19df3d5eabf030bb9 is removed from metadata 2024.10.22 01:38:59.716266 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:59.725759 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:59.763020 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_562_111} MergedBlockOutputStream: filled checksums 202410_1_562_111 (state Temporary) 2024.10.22 01:38:59.871047 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:38:59.873216 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:38:59.939905 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_991_991_0 (state Temporary) 2024.10.22 01:38:59.968771 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 9033 2024.10.22 01:38:59.970981 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_576_576_0 (state Temporary) 2024.10.22 01:39:00.125459 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_991_991_0 to 202410_1004_1004_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:00.214679 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65259 2024.10.22 01:39:00.228420 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:00.234125 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:00.269050 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_576_576_0 to 202410_582_582_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:00.282080 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_587_587_0 (state Temporary) 2024.10.22 01:39:00.305391 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_577_14 to 202410_582_582_0 2024.10.22 01:39:00.325271 [ 1256 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:00.357749 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:00.360052 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 66736 2024.10.22 01:39:00.397150 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_577_14 to 202410_582_582_0 into Compact with storage Full 2024.10.22 01:39:00.462976 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_587_587_0 to 202410_600_600_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:00.470461 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:00.485245 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_577_14, total 7076 rows starting from the beginning of the part 2024.10.22 01:39:00.525413 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_578_578_0, total 100 rows starting from the beginning of the part 2024.10.22 01:39:00.549642 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:39:00.555753 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:39:00.656380 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9033 2024.10.22 01:39:00.663796 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_579_579_0, total 115 rows starting from the beginning of the part 2024.10.22 01:39:00.765117 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(8c4ea26bbfaed64da73da3792b48a04b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8c4ea26bbfaed64da73da3792b48a04b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:39:00.784063 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 650, file segment info: File segment: [0, 33554431], key: 8c4ea26bbfaed64da73da3792b48a04b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:39:00.788847 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:00.820007 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_580_580_0, total 54 rows starting from the beginning of the part 2024.10.22 01:39:00.860051 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_581_581_0, total 195 rows starting from the beginning of the part 2024.10.22 01:39:00.883731 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_582_582_0, total 71 rows starting from the beginning of the part 2024.10.22 01:39:00.965554 [ 1727 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tdg/vfoipsuwzibnysqtxhgzeqmgfwziz, size 1648659 2024.10.22 01:39:01.211725 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:39:01.227452 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:39:01.344635 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:39:01.361577 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:39:01.385436 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:39:01.408173 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:39:01.436706 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:39:01.461159 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 01:39:01.487957 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_899_899_0, 202410_900_900_0, 202410_901_901_0, 202410_902_902_0, 202410_903_903_0, 202410_904_904_0] 2024.10.22 01:39:01.491822 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_899_899_0, 202410_900_900_0, 202410_901_901_0, 202410_902_902_0, 202410_903_903_0, 202410_904_904_0] 2024.10.22 01:39:01.664677 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_899_899_0, 202410_900_900_0, 202410_901_901_0, 202410_902_902_0, 202410_903_903_0, 202410_904_904_0] 2024.10.22 01:39:01.711848 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 8.08 MiB, will set to 1.69 GiB (RSS), difference: -3.46 MiB 2024.10.22 01:39:01.854743 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c4728eb328beb7994658555e49a17e85) : [0, 630853]: Setting download as failed: File segment: [0, 630853], key: c4728eb328beb7994658555e49a17e85, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:01.858732 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c4728eb328beb7994658555e49a17e85): No space left in cache to reserve 630854 bytes, will continue without cache download 2024.10.22 01:39:01.884746 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c4728eb328beb7994658555e49a17e85, offset: 0, size: 0 2024.10.22 01:39:01.886544 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c4728eb328beb7994658555e49a17e85 for removal 2024.10.22 01:39:01.892659 [ 1476 ] {} CacheMetadata: Key c4728eb328beb7994658555e49a17e85 is removed from metadata 2024.10.22 01:39:02.238670 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} MergeTask::MergeProjectionsStage: Merge sorted 7611 rows, containing 13 columns (13 merged, 0 gathered) in 1.854215823 sec., 4104.700167904888 rows/sec., 1.78 MiB/sec. 2024.10.22 01:39:02.435936 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_562_111} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_562_111 to 202410_1_562_111 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:02.441707 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_562_111} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_556_110, 202410_557_562_1] -> 202410_1_562_111 2024.10.22 01:39:02.479843 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:39:02.524783 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} MergedBlockOutputStream: filled checksums 202410_507_582_15 (state Temporary) 2024.10.22 01:39:02.662388 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_582_15 to 202410_507_582_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:02.666462 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_582_15} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_577_14, 202410_582_582_0] -> 202410_507_582_15 2024.10.22 01:39:02.697750 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.84 MiB. 2024.10.22 01:39:03.069101 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 145 bytes for 363ed99b2759d422eb8a6b297df8447b:0) 2024.10.22 01:39:03.072030 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d485237b741633fe0f5ec30b453f9531, offset: 0, size: 153 2024.10.22 01:39:03.075273 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d485237b741633fe0f5ec30b453f9531 for removal 2024.10.22 01:39:03.077656 [ 1476 ] {} CacheMetadata: Key d485237b741633fe0f5ec30b453f9531 is removed from metadata 2024.10.22 01:39:03.193724 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_536_107, 202410_537_537_0, 202410_538_538_0, 202410_539_539_0, 202410_540_540_0, 202410_541_541_0] 2024.10.22 01:39:03.196305 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_536_107, 202410_537_537_0, 202410_538_538_0, 202410_539_539_0, 202410_540_540_0, 202410_541_541_0] 2024.10.22 01:39:03.223622 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_536_107, 202410_537_537_0, 202410_538_538_0, 202410_539_539_0, 202410_540_540_0, 202410_541_541_0] 2024.10.22 01:39:03.486120 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 6.64 MiB, will set to 1.68 GiB (RSS), difference: 6.94 MiB 2024.10.22 01:39:03.540591 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_118_118_0 to 202410_121_121_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:04.063789 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(313cb9323b630599c4f1b0906cc5108d) : [0, 128519]: Setting download as failed: File segment: [0, 128519], key: 313cb9323b630599c4f1b0906cc5108d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:04.065252 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(313cb9323b630599c4f1b0906cc5108d): No space left in cache to reserve 128520 bytes, will continue without cache download 2024.10.22 01:39:04.087413 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 313cb9323b630599c4f1b0906cc5108d, offset: 0, size: 0 2024.10.22 01:39:04.093410 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 313cb9323b630599c4f1b0906cc5108d for removal 2024.10.22 01:39:04.097606 [ 1476 ] {} CacheMetadata: Key 313cb9323b630599c4f1b0906cc5108d is removed from metadata 2024.10.22 01:39:04.335964 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 65340 2024.10.22 01:39:04.697769 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:04.812282 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:04.874238 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_992_992_0 (state Temporary) 2024.10.22 01:39:04.918351 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_992_992_0 to 202410_1005_1005_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:04.962791 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 157 bytes for 440cc878f263447f206c656882bd903f:0) 2024.10.22 01:39:04.964170 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 363ed99b2759d422eb8a6b297df8447b, offset: 0, size: 145 2024.10.22 01:39:04.966036 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 363ed99b2759d422eb8a6b297df8447b for removal 2024.10.22 01:39:04.968951 [ 1476 ] {} CacheMetadata: Key 363ed99b2759d422eb8a6b297df8447b is removed from metadata 2024.10.22 01:39:05.108109 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65340 2024.10.22 01:39:05.509453 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 880944 2024.10.22 01:39:05.817798 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(60f5a9fbb2dabaa7ce130ed6cebf1e4f) : [0, 988824]: Setting download as failed: File segment: [0, 988824], key: 60f5a9fbb2dabaa7ce130ed6cebf1e4f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:05.821397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(60f5a9fbb2dabaa7ce130ed6cebf1e4f): No space left in cache to reserve 988825 bytes, will continue without cache download 2024.10.22 01:39:05.859716 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 493 2024.10.22 01:39:05.862729 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 60f5a9fbb2dabaa7ce130ed6cebf1e4f, offset: 0, size: 0 2024.10.22 01:39:05.864618 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 60f5a9fbb2dabaa7ce130ed6cebf1e4f for removal 2024.10.22 01:39:05.865953 [ 1476 ] {} CacheMetadata: Key 60f5a9fbb2dabaa7ce130ed6cebf1e4f is removed from metadata 2024.10.22 01:39:06.121425 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:06.125715 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:06.165485 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4734 2024.10.22 01:39:06.260076 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:06.263180 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:06.309356 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_475_475_0 (state Temporary) 2024.10.22 01:39:06.386300 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_475_475_0 to 202410_485_485_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:06.481634 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_563_563_0 (state Temporary) 2024.10.22 01:39:06.590448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for f0405c27781a3bc4cbb21744400275ca:0) 2024.10.22 01:39:06.592248 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 440cc878f263447f206c656882bd903f, offset: 0, size: 157 2024.10.22 01:39:06.595548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 440cc878f263447f206c656882bd903f for removal 2024.10.22 01:39:06.599794 [ 1476 ] {} CacheMetadata: Key 440cc878f263447f206c656882bd903f is removed from metadata 2024.10.22 01:39:06.656830 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_563_563_0 to 202410_570_570_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:06.676088 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 493 2024.10.22 01:39:06.680022 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 880944 2024.10.22 01:39:07.308022 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b280b88773fd12aeff88013c57c03130) : [0, 81895]: Setting download as failed: File segment: [0, 81895], key: b280b88773fd12aeff88013c57c03130, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:07.309643 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b280b88773fd12aeff88013c57c03130): No space left in cache to reserve 81896 bytes, will continue without cache download 2024.10.22 01:39:07.424124 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b280b88773fd12aeff88013c57c03130, offset: 0, size: 0 2024.10.22 01:39:07.426425 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b280b88773fd12aeff88013c57c03130 for removal 2024.10.22 01:39:07.429361 [ 1476 ] {} CacheMetadata: Key b280b88773fd12aeff88013c57c03130 is removed from metadata 2024.10.22 01:39:07.488840 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 7.19 MiB, will set to 1.68 GiB (RSS), difference: 5.39 MiB 2024.10.22 01:39:07.579879 [ 1166 ] {} DatabaseCatalog: Execution took 205 ms. 2024.10.22 01:39:07.888919 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 63 entries to flush up to offset 66799 2024.10.22 01:39:08.009519 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:08.012795 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:08.057034 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_577_577_0 (state Temporary) 2024.10.22 01:39:08.161629 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 9050 2024.10.22 01:39:08.201267 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_577_577_0 to 202410_583_583_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:08.343836 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 66799 2024.10.22 01:39:08.355592 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:08.358453 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:08.481377 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_588_588_0 (state Temporary) 2024.10.22 01:39:08.538814 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_588_588_0 to 202410_601_601_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:08.570803 [ 1165 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_596_119 to 202410_601_601_0 2024.10.22 01:39:08.582955 [ 1165 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:08.586838 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:08.588258 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9050 2024.10.22 01:39:08.640229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for 0eca07697f8edf24e37b4b6cbf380a3c:0) 2024.10.22 01:39:08.643758 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f0405c27781a3bc4cbb21744400275ca, offset: 0, size: 144 2024.10.22 01:39:08.646928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f0405c27781a3bc4cbb21744400275ca for removal 2024.10.22 01:39:08.651320 [ 1476 ] {} CacheMetadata: Key f0405c27781a3bc4cbb21744400275ca is removed from metadata 2024.10.22 01:39:08.732662 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_596_119 to 202410_601_601_0 into Compact with storage Full 2024.10.22 01:39:08.775377 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:08.779480 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_596_119, total 9088 rows starting from the beginning of the part 2024.10.22 01:39:08.824298 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_597_597_0, total 13 rows starting from the beginning of the part 2024.10.22 01:39:08.873194 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_598_598_0, total 4 rows starting from the beginning of the part 2024.10.22 01:39:08.925064 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_599_599_0, total 23 rows starting from the beginning of the part 2024.10.22 01:39:08.979359 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_600_600_0, total 11 rows starting from the beginning of the part 2024.10.22 01:39:09.123672 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_601_601_0, total 17 rows starting from the beginning of the part 2024.10.22 01:39:09.167802 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 65405 2024.10.22 01:39:09.396871 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:09.407472 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:09.652996 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7786de38c26e79e2db77ed53e19a8a0f) : [0, 67104]: Setting download as failed: File segment: [0, 67104], key: 7786de38c26e79e2db77ed53e19a8a0f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:09.654838 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7786de38c26e79e2db77ed53e19a8a0f): No space left in cache to reserve 67105 bytes, will continue without cache download 2024.10.22 01:39:09.670850 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7786de38c26e79e2db77ed53e19a8a0f, offset: 0, size: 0 2024.10.22 01:39:09.671978 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7786de38c26e79e2db77ed53e19a8a0f for removal 2024.10.22 01:39:09.676626 [ 1476 ] {} CacheMetadata: Key 7786de38c26e79e2db77ed53e19a8a0f is removed from metadata 2024.10.22 01:39:09.679104 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_993_993_0 (state Temporary) 2024.10.22 01:39:09.730440 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_993_993_0 to 202410_1006_1006_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:09.874943 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65405 2024.10.22 01:39:10.284558 [ 1206 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:39:10.324272 [ 1206 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:39:10.553302 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:39:10.558431 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:39:10.593661 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 6.20 MiB, will set to 1.68 GiB (RSS), difference: -1.36 MiB 2024.10.22 01:39:10.735958 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for 09ae8a0d7f0f447d42460bbdb8f2c6d5:0) 2024.10.22 01:39:10.741360 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0eca07697f8edf24e37b4b6cbf380a3c, offset: 0, size: 144 2024.10.22 01:39:10.749253 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0eca07697f8edf24e37b4b6cbf380a3c for removal 2024.10.22 01:39:10.753120 [ 1476 ] {} CacheMetadata: Key 0eca07697f8edf24e37b4b6cbf380a3c is removed from metadata 2024.10.22 01:39:11.059692 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} MergeTask::MergeProjectionsStage: Merge sorted 9156 rows, containing 12 columns (12 merged, 0 gathered) in 2.331332185 sec., 3927.3682484677747 rows/sec., 624.58 KiB/sec. 2024.10.22 01:39:11.074830 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} MergedBlockOutputStream: filled checksums 202410_1_601_120 (state Temporary) 2024.10.22 01:39:11.283805 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_601_120 to 202410_1_601_120 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:11.287318 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_601_120} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_596_119, 202410_601_601_0] -> 202410_1_601_120 2024.10.22 01:39:11.528744 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.53 MiB. 2024.10.22 01:39:11.870437 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(81e7dc0ba06b3f920ff5065f6f1e8464) : [0, 61978]: Setting download as failed: File segment: [0, 61978], key: 81e7dc0ba06b3f920ff5065f6f1e8464, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:11.871433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(81e7dc0ba06b3f920ff5065f6f1e8464): No space left in cache to reserve 61979 bytes, will continue without cache download 2024.10.22 01:39:11.879817 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 81e7dc0ba06b3f920ff5065f6f1e8464, offset: 0, size: 0 2024.10.22 01:39:11.881759 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 81e7dc0ba06b3f920ff5065f6f1e8464 for removal 2024.10.22 01:39:11.886612 [ 1476 ] {} CacheMetadata: Key 81e7dc0ba06b3f920ff5065f6f1e8464 is removed from metadata 2024.10.22 01:39:12.184172 [ 1419 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:39:12.458011 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 5.59 MiB, will set to 1.68 GiB (RSS), difference: 1.44 MiB 2024.10.22 01:39:12.941546 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for 75feae95cf70f4a7e189667611b902df:0) 2024.10.22 01:39:12.947585 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 09ae8a0d7f0f447d42460bbdb8f2c6d5, offset: 0, size: 144 2024.10.22 01:39:12.963926 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 09ae8a0d7f0f447d42460bbdb8f2c6d5 for removal 2024.10.22 01:39:12.966424 [ 1476 ] {} CacheMetadata: Key 09ae8a0d7f0f447d42460bbdb8f2c6d5 is removed from metadata 2024.10.22 01:39:13.559002 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_115_119_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:39:13.678833 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 4772 2024.10.22 01:39:13.683023 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 882099 2024.10.22 01:39:13.799665 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(41baa0e522ccd28c615f56e41060325e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 41baa0e522ccd28c615f56e41060325e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:13.838664 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 880218, file segment info: File segment: [0, 33554431], key: 41baa0e522ccd28c615f56e41060325e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:13.844896 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:14.011562 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 65448 2024.10.22 01:39:14.177920 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 494 2024.10.22 01:39:14.202852 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_509_509_0, 202410_510_510_0, 202410_511_511_0, 202410_512_512_0, 202410_513_513_0, 202410_514_514_0] 2024.10.22 01:39:14.208025 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_509_509_0, 202410_510_510_0, 202410_511_511_0, 202410_512_512_0, 202410_513_513_0, 202410_514_514_0] 2024.10.22 01:39:14.309644 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:14.313524 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:14.337494 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_509_509_0, 202410_510_510_0, 202410_511_511_0, 202410_512_512_0, 202410_513_513_0, 202410_514_514_0] 2024.10.22 01:39:14.349940 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:14.356006 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:14.459050 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:14.487341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(53921190168704b078b83c9a3baaf8b6) : [0, 96925]: Setting download as failed: File segment: [0, 96925], key: 53921190168704b078b83c9a3baaf8b6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:14.488936 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(53921190168704b078b83c9a3baaf8b6): No space left in cache to reserve 96926 bytes, will continue without cache download 2024.10.22 01:39:14.515004 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 53921190168704b078b83c9a3baaf8b6, offset: 0, size: 0 2024.10.22 01:39:14.519433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 53921190168704b078b83c9a3baaf8b6 for removal 2024.10.22 01:39:14.525190 [ 1476 ] {} CacheMetadata: Key 53921190168704b078b83c9a3baaf8b6 is removed from metadata 2024.10.22 01:39:14.559134 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:14.571575 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_476_476_0 (state Temporary) 2024.10.22 01:39:14.745409 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_476_476_0 to 202410_486_486_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:14.749837 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_564_564_0 (state Temporary) 2024.10.22 01:39:14.824692 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_994_994_0 (state Temporary) 2024.10.22 01:39:15.044517 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_564_564_0 to 202410_571_571_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:15.075820 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 882099 2024.10.22 01:39:15.127351 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 494 2024.10.22 01:39:15.291312 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_994_994_0 to 202410_1007_1007_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:15.336654 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_996_491 to 202410_997_1002_1 2024.10.22 01:39:15.349790 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.45 MiB using storage policy from min volume index 0 2024.10.22 01:39:15.358626 [ 1187 ] {} DiskLocal: Reserved 2.45 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:15.366428 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65448 2024.10.22 01:39:15.392719 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1002_492} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_996_491 to 202410_997_1002_1 into Wide with storage Full 2024.10.22 01:39:15.430719 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:39:15.463418 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1002_492} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:15.469470 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1002_492} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_996_491, total 65553 rows starting from the beginning of the part 2024.10.22 01:39:15.481002 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 4.96 MiB, will set to 1.68 GiB (RSS), difference: 1.52 MiB 2024.10.22 01:39:15.544577 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(2cf5b4cceb65005feb8446a0b08b7e4b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2cf5b4cceb65005feb8446a0b08b7e4b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:15.562579 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 638, file segment info: File segment: [0, 33554431], key: 2cf5b4cceb65005feb8446a0b08b7e4b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:15.564455 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:15.855599 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 66 entries to flush up to offset 66865 2024.10.22 01:39:15.962952 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1002_492} MergeTreeSequentialSource: Reading 2 marks from part 202410_997_1002_1, total 389 rows starting from the beginning of the part 2024.10.22 01:39:16.047838 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:16.054769 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:16.092536 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 9063 2024.10.22 01:39:16.098263 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_578_578_0 (state Temporary) 2024.10.22 01:39:16.159945 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_578_578_0 to 202410_584_584_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:16.203741 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 66865 2024.10.22 01:39:16.402700 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:39:16.438546 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_115_119_1} MergeTask::MergeProjectionsStage: Merge sorted 198 rows, containing 679 columns (679 merged, 0 gathered) in 73.281691494 sec., 2.701902698523429 rows/sec., 4.14 KiB/sec. 2024.10.22 01:39:16.454561 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for 73cf30584dead8cce987fdab0d8e9178:0) 2024.10.22 01:39:16.457225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 75feae95cf70f4a7e189667611b902df, offset: 0, size: 155 2024.10.22 01:39:16.462547 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 75feae95cf70f4a7e189667611b902df for removal 2024.10.22 01:39:16.464463 [ 1476 ] {} CacheMetadata: Key 75feae95cf70f4a7e189667611b902df is removed from metadata 2024.10.22 01:39:16.517770 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:39:16.530605 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:39:16.564958 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:39:16.578253 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:39:16.604636 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:16.605193 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:39:16.611576 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:16.629149 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:39:16.650617 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 01:39:16.684303 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zrs/lbarnitbdfbmatfyjardwhmcydfkm (LogSeriesLimiter: on interval from 2024-10-22 01:38:57 to 2024-10-22 01:39:16 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:39:16.856928 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_589_589_0 (state Temporary) 2024.10.22 01:39:17.026861 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_589_589_0 to 202410_602_602_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:17.088566 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9063 2024.10.22 01:39:17.685657 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_115_119_1} MergedBlockOutputStream: filled checksums 202410_115_119_1 (state Temporary) 2024.10.22 01:39:17.762686 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6b146726d2b959fce2df4156459f2938) : [0, 52864]: Setting download as failed: File segment: [0, 52864], key: 6b146726d2b959fce2df4156459f2938, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:17.763956 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6b146726d2b959fce2df4156459f2938): No space left in cache to reserve 52865 bytes, will continue without cache download 2024.10.22 01:39:17.778931 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6b146726d2b959fce2df4156459f2938, offset: 0, size: 0 2024.10.22 01:39:17.780228 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6b146726d2b959fce2df4156459f2938 for removal 2024.10.22 01:39:17.782442 [ 1476 ] {} CacheMetadata: Key 6b146726d2b959fce2df4156459f2938 is removed from metadata 2024.10.22 01:39:17.895937 [ 1197 ] {} DatabaseCatalog: Execution took 314 ms. 2024.10.22 01:39:18.860827 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 5.07 MiB, will set to 1.69 GiB (RSS), difference: -43.52 MiB 2024.10.22 01:39:18.984432 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 146 bytes for 101171145e9b08f8e14f1abcc22d19aa:0) 2024.10.22 01:39:18.987502 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 73cf30584dead8cce987fdab0d8e9178, offset: 0, size: 144 2024.10.22 01:39:18.996229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 73cf30584dead8cce987fdab0d8e9178 for removal 2024.10.22 01:39:19.022239 [ 1476 ] {} CacheMetadata: Key 73cf30584dead8cce987fdab0d8e9178 is removed from metadata 2024.10.22 01:39:19.471785 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 65523 2024.10.22 01:39:19.709537 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1002_492} MergeTask::MergeProjectionsStage: Merge sorted 65942 rows, containing 13 columns (13 merged, 0 gathered) in 4.336048713 sec., 15207.854976882036 rows/sec., 2.51 MiB/sec. 2024.10.22 01:39:19.786506 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(eb57a587309c9d5d6b6f68ee27aa181a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: eb57a587309c9d5d6b6f68ee27aa181a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:39:19.812079 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: eb57a587309c9d5d6b6f68ee27aa181a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:39:19.818102 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:19.886694 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1002_492} MergedBlockOutputStream: filled checksums 202410_1_1002_492 (state Temporary) 2024.10.22 01:39:19.948870 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:19.955325 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:20.117641 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_995_995_0 (state Temporary) 2024.10.22 01:39:20.180586 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_995_995_0 to 202410_1008_1008_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:20.232395 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65523 2024.10.22 01:39:20.217687 [ 1270 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1003_1003_0 to 202410_1008_1008_0 2024.10.22 01:39:20.327143 [ 1270 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:20.331976 [ 1270 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:20.555422 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:39:20.558194 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:39:20.570157 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1003_1003_0 to 202410_1008_1008_0 into Compact with storage Full 2024.10.22 01:39:20.624363 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1002_492} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1002_492 to 202410_1_1002_492 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:20.628886 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1002_492} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_996_491, 202410_997_1002_1] -> 202410_1_1002_492 2024.10.22 01:39:20.749682 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.78 MiB. 2024.10.22 01:39:20.772283 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:20.809747 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1003_1003_0, total 89 rows starting from the beginning of the part 2024.10.22 01:39:20.812334 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(86a8ec17f562242d8972354b0a90a423) : [0, 241970]: Setting download as failed: File segment: [0, 241970], key: 86a8ec17f562242d8972354b0a90a423, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:20.814729 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(86a8ec17f562242d8972354b0a90a423): No space left in cache to reserve 241971 bytes, will continue without cache download 2024.10.22 01:39:20.870527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 86a8ec17f562242d8972354b0a90a423, offset: 0, size: 0 2024.10.22 01:39:20.873283 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 86a8ec17f562242d8972354b0a90a423 for removal 2024.10.22 01:39:20.876145 [ 1476 ] {} CacheMetadata: Key 86a8ec17f562242d8972354b0a90a423 is removed from metadata 2024.10.22 01:39:20.896454 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1004_1004_0, total 59 rows starting from the beginning of the part 2024.10.22 01:39:20.967597 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1005_1005_0, total 81 rows starting from the beginning of the part 2024.10.22 01:39:21.055076 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1006_1006_0, total 65 rows starting from the beginning of the part 2024.10.22 01:39:21.114966 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1007_1007_0, total 43 rows starting from the beginning of the part 2024.10.22 01:39:21.179317 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1008_1008_0, total 75 rows starting from the beginning of the part 2024.10.22 01:39:21.869462 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(5c82085508709e9daa49f735d7a862c1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5c82085508709e9daa49f735d7a862c1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:21.872181 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 60022, file segment info: File segment: [0, 33554431], key: 5c82085508709e9daa49f735d7a862c1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:21.874093 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:22.078865 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 883254 2024.10.22 01:39:22.387803 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 4.55 MiB, will set to 1.68 GiB (RSS), difference: 32.22 MiB 2024.10.22 01:39:22.727234 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 149 bytes for 9d367a2a38a07219c74557757ba6de4e:0) 2024.10.22 01:39:22.729707 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 101171145e9b08f8e14f1abcc22d19aa, offset: 0, size: 146 2024.10.22 01:39:22.629741 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 495 2024.10.22 01:39:22.736141 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} MergeTask::MergeProjectionsStage: Merge sorted 412 rows, containing 13 columns (13 merged, 0 gathered) in 2.282255772 sec., 180.52314953242671 rows/sec., 31.24 KiB/sec. 2024.10.22 01:39:22.736369 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 101171145e9b08f8e14f1abcc22d19aa for removal 2024.10.22 01:39:22.740817 [ 1476 ] {} CacheMetadata: Key 101171145e9b08f8e14f1abcc22d19aa is removed from metadata 2024.10.22 01:39:22.777375 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} MergedBlockOutputStream: filled checksums 202410_1003_1008_1 (state Temporary) 2024.10.22 01:39:22.991561 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1003_1008_1 to 202410_1003_1008_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:23.001747 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1003_1008_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1003_1003_0, 202410_1008_1008_0] -> 202410_1003_1008_1 2024.10.22 01:39:23.041175 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:23.045460 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:23.160337 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:23.163519 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:23.163810 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:39:23.164105 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_565_565_0 (state Temporary) 2024.10.22 01:39:23.224881 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_565_565_0 to 202410_572_572_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:23.263003 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 883254 2024.10.22 01:39:23.259318 [ 1182 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_562_111 to 202410_563_568_1 2024.10.22 01:39:23.280805 [ 1182 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:23.283431 [ 1182 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:23.328168 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_568_112} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_562_111 to 202410_563_568_1 into Wide with storage Full 2024.10.22 01:39:23.328368 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_477_477_0 (state Temporary) 2024.10.22 01:39:23.379866 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_568_112} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:23.388136 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_568_112} MergeTreeSequentialSource: Reading 110 marks from part 202410_1_562_111, total 885294 rows starting from the beginning of the part 2024.10.22 01:39:23.456715 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_568_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_563_568_1, total 6930 rows starting from the beginning of the part 2024.10.22 01:39:23.464137 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_433_89, 202410_434_434_0, 202410_435_435_0, 202410_436_436_0, 202410_437_437_0, 202410_438_438_0] 2024.10.22 01:39:23.467426 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_433_89, 202410_434_434_0, 202410_435_435_0, 202410_436_436_0, 202410_437_437_0, 202410_438_438_0] 2024.10.22 01:39:23.508614 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_433_89, 202410_434_434_0, 202410_435_435_0, 202410_436_436_0, 202410_437_437_0, 202410_438_438_0] 2024.10.22 01:39:23.509792 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_477_477_0 to 202410_487_487_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:23.688676 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 495 2024.10.22 01:39:23.715060 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 203 entries to flush up to offset 67068 2024.10.22 01:39:23.748478 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d402c8ef6477953e0e19f80a18d39f94) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d402c8ef6477953e0e19f80a18d39f94, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:23.755143 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 834, file segment info: File segment: [0, 33554431], key: d402c8ef6477953e0e19f80a18d39f94, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:23.759203 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:24.020579 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:24.124047 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:24.180956 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_579_579_0 (state Temporary) 2024.10.22 01:39:24.251554 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 65597 2024.10.22 01:39:24.258251 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_579_579_0 to 202410_585_585_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:24.284743 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 67068 2024.10.22 01:39:24.409691 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4ebc58db9abe56ffe6e332e0b3aac2f7) : [0, 374858]: Setting download as failed: File segment: [0, 374858], key: 4ebc58db9abe56ffe6e332e0b3aac2f7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:24.411659 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4ebc58db9abe56ffe6e332e0b3aac2f7): No space left in cache to reserve 374859 bytes, will continue without cache download 2024.10.22 01:39:24.592138 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 9079 2024.10.22 01:39:24.631053 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:24.638676 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:24.662947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4ebc58db9abe56ffe6e332e0b3aac2f7, offset: 0, size: 0 2024.10.22 01:39:24.664069 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4ebc58db9abe56ffe6e332e0b3aac2f7 for removal 2024.10.22 01:39:24.665647 [ 1476 ] {} CacheMetadata: Key 4ebc58db9abe56ffe6e332e0b3aac2f7 is removed from metadata 2024.10.22 01:39:24.782488 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_996_996_0 (state Temporary) 2024.10.22 01:39:24.832406 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:24.840542 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:24.856732 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_996_996_0 to 202410_1009_1009_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:24.881470 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_590_590_0 (state Temporary) 2024.10.22 01:39:24.891811 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_502_101, 202410_503_508_1] 2024.10.22 01:39:24.904471 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_502_101, 202410_503_508_1] 2024.10.22 01:39:24.915503 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_502_101, 202410_503_508_1] 2024.10.22 01:39:24.929395 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65597 2024.10.22 01:39:25.040518 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_590_590_0 to 202410_603_603_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:25.106068 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9079 2024.10.22 01:39:25.328471 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(947856338a3480e5faeebe424f8a1708) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 947856338a3480e5faeebe424f8a1708, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:25.334672 [ 1237 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:39:25.336408 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 45587, file segment info: File segment: [0, 33554431], key: 947856338a3480e5faeebe424f8a1708, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:25.351472 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:25.367551 [ 1237 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:39:25.695463 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 9.74 MiB, will set to 1.69 GiB (RSS), difference: -3.85 MiB 2024.10.22 01:39:25.824599 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_115_119_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_115_119_1 to 202410_115_119_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:25.830270 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_115_119_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 5 parts: [202410_115_115_0, 202410_119_119_0] -> 202410_115_119_1 2024.10.22 01:39:26.207598 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 141 bytes for 6d5b26d65ea08e5d6aa5e503ca90cf6f:0) 2024.10.22 01:39:26.312861 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9d367a2a38a07219c74557757ba6de4e, offset: 0, size: 149 2024.10.22 01:39:26.315801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9d367a2a38a07219c74557757ba6de4e for removal 2024.10.22 01:39:26.326382 [ 1476 ] {} CacheMetadata: Key 9d367a2a38a07219c74557757ba6de4e is removed from metadata 2024.10.22 01:39:26.902040 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.75 MiB. 2024.10.22 01:39:27.069193 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_568_112/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:39:27.126893 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(e2630261314480e51cc5023763a1a645) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e2630261314480e51cc5023763a1a645, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:27.129993 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 755, file segment info: File segment: [0, 33554431], key: e2630261314480e51cc5023763a1a645, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:39:27.133330 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:27.179712 [ 1697 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ysn/zwtdxtrhoxjzxdbwynspkvuymhlqz, size 45587 2024.10.22 01:39:27.570349 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:39:27.579774 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:39:27.595708 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:39:27.628063 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:39:27.654207 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_898_456, 202410_899_904_1] 2024.10.22 01:39:27.664211 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_898_456, 202410_899_904_1] 2024.10.22 01:39:27.672192 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_898_456, 202410_899_904_1] 2024.10.22 01:39:27.756288 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:39:27.771846 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:39:27.800385 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:39:27.864765 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:39:27.889334 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 01:39:28.075686 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bbc1497f967e68a462600b313a4c33ab) : [0, 4543]: Setting download as failed: File segment: [0, 4543], key: bbc1497f967e68a462600b313a4c33ab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:28.077452 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bbc1497f967e68a462600b313a4c33ab): No space left in cache to reserve 4544 bytes, will continue without cache download 2024.10.22 01:39:28.076556 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hhe/wlxuzypemqdxqqkatpvvjauyggfxk (LogSeriesLimiter: on interval from 2024-10-22 01:39:16 to 2024-10-22 01:39:28 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:39:28.159589 [ 1253 ] {} DatabaseCatalog: Execution took 262 ms. 2024.10.22 01:39:28.305569 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bbc1497f967e68a462600b313a4c33ab, offset: 0, size: 0 2024.10.22 01:39:28.308119 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bbc1497f967e68a462600b313a4c33ab for removal 2024.10.22 01:39:28.311493 [ 1476 ] {} CacheMetadata: Key bbc1497f967e68a462600b313a4c33ab is removed from metadata 2024.10.22 01:39:28.725082 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 11.57 MiB, will set to 1.69 GiB (RSS), difference: 6.51 MiB 2024.10.22 01:39:28.952984 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 65657 2024.10.22 01:39:29.202507 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 157 bytes for e012f1b8b61bcb11956b327c244900da:0) 2024.10.22 01:39:29.208141 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6d5b26d65ea08e5d6aa5e503ca90cf6f, offset: 0, size: 141 2024.10.22 01:39:29.296861 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:29.303565 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:29.313131 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6d5b26d65ea08e5d6aa5e503ca90cf6f for removal 2024.10.22 01:39:29.316874 [ 1476 ] {} CacheMetadata: Key 6d5b26d65ea08e5d6aa5e503ca90cf6f is removed from metadata 2024.10.22 01:39:29.562791 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_997_997_0 (state Temporary) 2024.10.22 01:39:29.612773 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_997_997_0 to 202410_1010_1010_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:29.790287 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65657 2024.10.22 01:39:30.133070 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b68d0eef86dc83e75ddfa9afe2d704c1) : [0, 236000]: Setting download as failed: File segment: [0, 236000], key: b68d0eef86dc83e75ddfa9afe2d704c1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:30.135572 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b68d0eef86dc83e75ddfa9afe2d704c1): No space left in cache to reserve 236001 bytes, will continue without cache download 2024.10.22 01:39:30.160270 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b68d0eef86dc83e75ddfa9afe2d704c1, offset: 0, size: 0 2024.10.22 01:39:30.161993 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b68d0eef86dc83e75ddfa9afe2d704c1 for removal 2024.10.22 01:39:30.163804 [ 1476 ] {} CacheMetadata: Key b68d0eef86dc83e75ddfa9afe2d704c1 is removed from metadata 2024.10.22 01:39:30.266678 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 884409 2024.10.22 01:39:30.515858 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:30.521584 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:30.555825 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:39:30.558085 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:39:30.935512 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(50cce7bddc13e44fa300d0ec9181d60e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 50cce7bddc13e44fa300d0ec9181d60e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:39:30.938821 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 981, file segment info: File segment: [0, 33554431], key: 50cce7bddc13e44fa300d0ec9181d60e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:39:30.944237 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:31.110412 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:31.114617 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:31.142861 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:39:31.157457 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:39:31.180123 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:39:31.190875 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 496 2024.10.22 01:39:31.194226 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:39:31.208320 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:39:31.229736 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:39:31.351693 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_566_566_0 (state Temporary) 2024.10.22 01:39:31.358556 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:39:31.380683 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 01:39:31.493364 [ 1733 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zrs/lbarnitbdfbmatfyjardwhmcydfkm, size 10329294 2024.10.22 01:39:31.501264 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 11.41 MiB, will set to 1.69 GiB (RSS), difference: -9.06 MiB 2024.10.22 01:39:31.745396 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:31.753537 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:31.782359 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_566_566_0 to 202410_573_573_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:31.821722 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 884409 2024.10.22 01:39:31.979293 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 63 entries to flush up to offset 67131 2024.10.22 01:39:31.997000 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_478_478_0 (state Temporary) 2024.10.22 01:39:32.227663 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:32.236750 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:32.283130 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_580_580_0 (state Temporary) 2024.10.22 01:39:32.302351 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_478_478_0 to 202410_488_488_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:32.327685 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_580_580_0 to 202410_586_586_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:32.366659 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 146 bytes for ffdbec8c957e3a48effd68958fd137af:0) 2024.10.22 01:39:32.368241 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e012f1b8b61bcb11956b327c244900da, offset: 0, size: 157 2024.10.22 01:39:32.371874 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e012f1b8b61bcb11956b327c244900da for removal 2024.10.22 01:39:32.376212 [ 1476 ] {} CacheMetadata: Key e012f1b8b61bcb11956b327c244900da is removed from metadata 2024.10.22 01:39:32.420110 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 67131 2024.10.22 01:39:32.451178 [ 1422 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:39:32.455678 [ 1265 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_483_99 to 202410_488_488_0 2024.10.22 01:39:32.465594 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_522_3, 202410_523_523_0, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0, 202410_527_527_0] 2024.10.22 01:39:32.465589 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 496 2024.10.22 01:39:32.467599 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_522_3, 202410_523_523_0, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0, 202410_527_527_0] 2024.10.22 01:39:32.498139 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_522_3, 202410_523_523_0, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0, 202410_527_527_0] 2024.10.22 01:39:32.503890 [ 1265 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:32.515601 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:32.569310 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_483_99 to 202410_488_488_0 into Compact with storage Full 2024.10.22 01:39:32.611090 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 9091 2024.10.22 01:39:32.644553 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:32.665135 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_483_99, total 527 rows starting from the beginning of the part 2024.10.22 01:39:32.748432 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_484_484_0, total 1 rows starting from the beginning of the part 2024.10.22 01:39:32.797240 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:32.802851 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:32.840743 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_591_591_0 (state Temporary) 2024.10.22 01:39:32.915229 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_591_591_0 to 202410_604_604_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:32.915810 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_485_485_0, total 1 rows starting from the beginning of the part 2024.10.22 01:39:32.966745 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9091 2024.10.22 01:39:33.078341 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_486_486_0, total 1 rows starting from the beginning of the part 2024.10.22 01:39:33.172557 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_487_487_0, total 1 rows starting from the beginning of the part 2024.10.22 01:39:33.279361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(32bb38de496bd04a38b028f860094dc8) : [0, 261505]: Setting download as failed: File segment: [0, 261505], key: 32bb38de496bd04a38b028f860094dc8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:33.280417 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(32bb38de496bd04a38b028f860094dc8): No space left in cache to reserve 261506 bytes, will continue without cache download 2024.10.22 01:39:33.314395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 32bb38de496bd04a38b028f860094dc8, offset: 0, size: 0 2024.10.22 01:39:33.316829 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 32bb38de496bd04a38b028f860094dc8 for removal 2024.10.22 01:39:33.324619 [ 1476 ] {} CacheMetadata: Key 32bb38de496bd04a38b028f860094dc8 is removed from metadata 2024.10.22 01:39:33.351749 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_488_488_0, total 1 rows starting from the beginning of the part 2024.10.22 01:39:33.570801 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_119_119_0 (state Temporary) 2024.10.22 01:39:33.928965 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 65738 2024.10.22 01:39:34.052010 [ 1163 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 01:39:34.212915 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_904_457, 202410_905_905_0] 2024.10.22 01:39:34.215858 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_904_457, 202410_905_905_0] 2024.10.22 01:39:34.233234 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_904_457, 202410_905_905_0] 2024.10.22 01:39:34.366915 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:34.370671 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:34.462487 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_998_998_0 (state Temporary) 2024.10.22 01:39:34.517276 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_998_998_0 to 202410_1011_1011_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:34.665624 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 15.30 MiB, will set to 1.69 GiB (RSS), difference: -2.23 MiB 2024.10.22 01:39:34.683694 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 146 bytes for 022141dc51b13ce41da45acef972786f:0) 2024.10.22 01:39:34.685428 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ffdbec8c957e3a48effd68958fd137af, offset: 0, size: 146 2024.10.22 01:39:34.687529 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65738 2024.10.22 01:39:34.694144 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ffdbec8c957e3a48effd68958fd137af for removal 2024.10.22 01:39:34.703423 [ 1476 ] {} CacheMetadata: Key ffdbec8c957e3a48effd68958fd137af is removed from metadata 2024.10.22 01:39:35.769061 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} MergeTask::MergeProjectionsStage: Merge sorted 532 rows, containing 32 columns (32 merged, 0 gathered) in 3.218169166 sec., 165.31138437984774 rows/sec., 15.22 KiB/sec. 2024.10.22 01:39:35.927603 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} MergedBlockOutputStream: filled checksums 202410_1_488_100 (state Temporary) 2024.10.22 01:39:35.961419 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f13f789e54acf0e7a91aedd772a9c9a5) : [0, 241888]: Setting download as failed: File segment: [0, 241888], key: f13f789e54acf0e7a91aedd772a9c9a5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:35.962600 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f13f789e54acf0e7a91aedd772a9c9a5): No space left in cache to reserve 241889 bytes, will continue without cache download 2024.10.22 01:39:35.989127 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f13f789e54acf0e7a91aedd772a9c9a5, offset: 0, size: 0 2024.10.22 01:39:35.992520 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f13f789e54acf0e7a91aedd772a9c9a5 for removal 2024.10.22 01:39:35.998799 [ 1476 ] {} CacheMetadata: Key f13f789e54acf0e7a91aedd772a9c9a5 is removed from metadata 2024.10.22 01:39:36.472288 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_905_458, 202410_906_906_0] 2024.10.22 01:39:36.474077 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_905_458, 202410_906_906_0] 2024.10.22 01:39:36.482118 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_905_458, 202410_906_906_0] 2024.10.22 01:39:36.750728 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_488_100 to 202410_1_488_100 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:36.755937 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_488_100} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_483_99, 202410_488_488_0] -> 202410_1_488_100 2024.10.22 01:39:36.879234 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:39:36.933690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 146 bytes for db2a4774baa9decf2ed075df5d26ede0:0) 2024.10.22 01:39:36.937833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 022141dc51b13ce41da45acef972786f, offset: 0, size: 146 2024.10.22 01:39:37.044630 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 022141dc51b13ce41da45acef972786f for removal 2024.10.22 01:39:37.054935 [ 1476 ] {} CacheMetadata: Key 022141dc51b13ce41da45acef972786f is removed from metadata 2024.10.22 01:39:37.637147 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 13.83 MiB, will set to 1.69 GiB (RSS), difference: 3.71 MiB 2024.10.22 01:39:38.098899 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cdb42b37e06d7eeb4667456972fb50eb) : [0, 257110]: Setting download as failed: File segment: [0, 257110], key: cdb42b37e06d7eeb4667456972fb50eb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:38.102571 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cdb42b37e06d7eeb4667456972fb50eb): No space left in cache to reserve 257111 bytes, will continue without cache download 2024.10.22 01:39:38.251264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cdb42b37e06d7eeb4667456972fb50eb, offset: 0, size: 0 2024.10.22 01:39:38.253437 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cdb42b37e06d7eeb4667456972fb50eb for removal 2024.10.22 01:39:38.258603 [ 1476 ] {} CacheMetadata: Key cdb42b37e06d7eeb4667456972fb50eb is removed from metadata 2024.10.22 01:39:38.754640 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 65776 2024.10.22 01:39:38.827400 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 885564 2024.10.22 01:39:38.827825 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_119_119_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:39:38.995183 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:39.000294 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:39.013566 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_568_112/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:39:39.161633 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 148 bytes for 97a38444e02712ac4589d085eda8412d:0) 2024.10.22 01:39:39.165133 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: db2a4774baa9decf2ed075df5d26ede0, offset: 0, size: 146 2024.10.22 01:39:39.170719 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key db2a4774baa9decf2ed075df5d26ede0 for removal 2024.10.22 01:39:39.172721 [ 1476 ] {} CacheMetadata: Key db2a4774baa9decf2ed075df5d26ede0 is removed from metadata 2024.10.22 01:39:39.274689 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_999_999_0 (state Temporary) 2024.10.22 01:39:39.323017 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_999_999_0 to 202410_1012_1012_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:39.381500 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:39.382334 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65776 2024.10.22 01:39:39.385079 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:39.574753 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_567_567_0 (state Temporary) 2024.10.22 01:39:39.650648 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_567_567_0 to 202410_574_574_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:39.669534 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 12.61 MiB, will set to 1.69 GiB (RSS), difference: -3.23 MiB 2024.10.22 01:39:39.670747 [ 1227 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_569_569_0 to 202410_574_574_0 2024.10.22 01:39:39.675571 [ 1227 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:39.677492 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:39.735813 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_569_569_0 to 202410_574_574_0 into Compact with storage Full 2024.10.22 01:39:39.760169 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 885564 2024.10.22 01:39:39.765175 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:39.769556 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_569_569_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:39:39.779725 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_570_570_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:39:39.792539 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_571_571_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:39:39.818006 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_572_572_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:39:39.939268 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 60 entries to flush up to offset 67191 2024.10.22 01:39:39.949071 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_573_573_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:39:39.960437 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_574_574_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:39:40.091731 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e3f65cfb9b99be368146f42c77fad4d6) : [0, 331840]: Setting download as failed: File segment: [0, 331840], key: e3f65cfb9b99be368146f42c77fad4d6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:40.094164 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e3f65cfb9b99be368146f42c77fad4d6): No space left in cache to reserve 331841 bytes, will continue without cache download 2024.10.22 01:39:40.137688 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e3f65cfb9b99be368146f42c77fad4d6, offset: 0, size: 0 2024.10.22 01:39:40.140396 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e3f65cfb9b99be368146f42c77fad4d6 for removal 2024.10.22 01:39:40.144869 [ 1476 ] {} CacheMetadata: Key e3f65cfb9b99be368146f42c77fad4d6 is removed from metadata 2024.10.22 01:39:40.176877 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:40.182837 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:40.354620 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_581_581_0 (state Temporary) 2024.10.22 01:39:40.370914 [ 1216 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:39:40.390558 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_581_581_0 to 202410_587_587_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:40.393522 [ 1216 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:39:40.452284 [ 1233 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_582_15 to 202410_587_587_0 2024.10.22 01:39:40.462567 [ 1233 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:40.464091 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 67191 2024.10.22 01:39:40.468575 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 9103 2024.10.22 01:39:40.469452 [ 1233 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:40.556108 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:39:40.557824 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:39:40.652956 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_582_15 to 202410_587_587_0 into Compact with storage Full 2024.10.22 01:39:40.689089 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:40.696057 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:40.707536 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} MergeTask::MergeProjectionsStage: Merge sorted 6930 rows, containing 4 columns (4 merged, 0 gathered) in 0.981862171 sec., 7058.0171073726 rows/sec., 110.28 KiB/sec. 2024.10.22 01:39:40.765087 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} MergedBlockOutputStream: filled checksums 202410_569_574_1 (state Temporary) 2024.10.22 01:39:40.833314 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_592_592_0 (state Temporary) 2024.10.22 01:39:40.838693 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:40.856247 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_582_15, total 7611 rows starting from the beginning of the part 2024.10.22 01:39:40.895173 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_569_574_1 to 202410_569_574_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:40.919049 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_569_574_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_569_569_0, 202410_574_574_0] -> 202410_569_574_1 2024.10.22 01:39:40.962586 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:39:40.977644 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_592_592_0 to 202410_605_605_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:41.071076 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9103 2024.10.22 01:39:41.099408 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_583_583_0, total 63 rows starting from the beginning of the part 2024.10.22 01:39:41.143149 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_584_584_0, total 66 rows starting from the beginning of the part 2024.10.22 01:39:41.185756 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_585_585_0, total 203 rows starting from the beginning of the part 2024.10.22 01:39:41.206663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 148 bytes for 578e365534ebff427e1cbd15c01feade:0) 2024.10.22 01:39:41.208978 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 97a38444e02712ac4589d085eda8412d, offset: 0, size: 148 2024.10.22 01:39:41.213124 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 97a38444e02712ac4589d085eda8412d for removal 2024.10.22 01:39:41.215621 [ 1476 ] {} CacheMetadata: Key 97a38444e02712ac4589d085eda8412d is removed from metadata 2024.10.22 01:39:41.239102 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_586_586_0, total 63 rows starting from the beginning of the part 2024.10.22 01:39:41.413067 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_587_587_0, total 60 rows starting from the beginning of the part 2024.10.22 01:39:41.765243 [ 1248 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 01:39:42.372391 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9ec02af1f0b987acaa47f96d9e2c5616) : [0, 335205]: Setting download as failed: File segment: [0, 335205], key: 9ec02af1f0b987acaa47f96d9e2c5616, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:42.373600 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9ec02af1f0b987acaa47f96d9e2c5616): No space left in cache to reserve 335206 bytes, will continue without cache download 2024.10.22 01:39:42.616829 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9ec02af1f0b987acaa47f96d9e2c5616, offset: 0, size: 0 2024.10.22 01:39:42.670267 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9ec02af1f0b987acaa47f96d9e2c5616 for removal 2024.10.22 01:39:42.671877 [ 1476 ] {} CacheMetadata: Key 9ec02af1f0b987acaa47f96d9e2c5616 is removed from metadata 2024.10.22 01:39:42.930977 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} MergeTask::MergeProjectionsStage: Merge sorted 8066 rows, containing 13 columns (13 merged, 0 gathered) in 2.284162438 sec., 3531.272498755625 rows/sec., 1.53 MiB/sec. 2024.10.22 01:39:43.057438 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} MergedBlockOutputStream: filled checksums 202410_507_587_16 (state Temporary) 2024.10.22 01:39:43.109789 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_587_16 to 202410_507_587_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:43.115449 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_587_16} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_582_15, 202410_587_587_0] -> 202410_507_587_16 2024.10.22 01:39:43.259136 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.98 MiB. 2024.10.22 01:39:43.388948 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 65858 2024.10.22 01:39:43.424986 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_119_119_0 to 202410_122_122_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:43.464913 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 9.31 MiB, will set to 1.68 GiB (RSS), difference: 3.65 MiB 2024.10.22 01:39:43.619135 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 148 bytes for e1eaf91d946e0dd9577431b725589e5c:0) 2024.10.22 01:39:43.622469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 578e365534ebff427e1cbd15c01feade, offset: 0, size: 148 2024.10.22 01:39:43.629148 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 578e365534ebff427e1cbd15c01feade for removal 2024.10.22 01:39:43.636353 [ 1476 ] {} CacheMetadata: Key 578e365534ebff427e1cbd15c01feade is removed from metadata 2024.10.22 01:39:43.667023 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:43.743587 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:43.845039 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_906_459, 202410_907_907_0] 2024.10.22 01:39:43.850934 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_906_459, 202410_907_907_0] 2024.10.22 01:39:43.862041 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_906_459, 202410_907_907_0] 2024.10.22 01:39:43.926962 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1000_1000_0 (state Temporary) 2024.10.22 01:39:43.985452 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1000_1000_0 to 202410_1013_1013_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:44.022217 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1002_492 to 202410_1003_1008_1 2024.10.22 01:39:44.040645 [ 1242 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.46 MiB using storage policy from min volume index 0 2024.10.22 01:39:44.045255 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65858 2024.10.22 01:39:44.046756 [ 1242 ] {} DiskLocal: Reserved 2.46 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:44.063906 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1008_493} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1002_492 to 202410_1003_1008_1 into Wide with storage Full 2024.10.22 01:39:44.112685 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1008_493} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:44.121342 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1008_493} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1002_492, total 65942 rows starting from the beginning of the part 2024.10.22 01:39:44.270357 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1008_493} MergeTreeSequentialSource: Reading 2 marks from part 202410_1003_1008_1, total 412 rows starting from the beginning of the part 2024.10.22 01:39:44.957949 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6e291229ed91bac2804f79ea06ef69f6) : [0, 286122]: Setting download as failed: File segment: [0, 286122], key: 6e291229ed91bac2804f79ea06ef69f6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:44.960395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6e291229ed91bac2804f79ea06ef69f6): No space left in cache to reserve 286123 bytes, will continue without cache download 2024.10.22 01:39:45.087207 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6e291229ed91bac2804f79ea06ef69f6, offset: 0, size: 0 2024.10.22 01:39:45.088754 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6e291229ed91bac2804f79ea06ef69f6 for removal 2024.10.22 01:39:45.094737 [ 1476 ] {} CacheMetadata: Key 6e291229ed91bac2804f79ea06ef69f6 is removed from metadata 2024.10.22 01:39:45.804532 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_541_108, 202410_542_542_0, 202410_543_543_0, 202410_544_544_0, 202410_545_545_0, 202410_546_546_0] 2024.10.22 01:39:45.806789 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_541_108, 202410_542_542_0, 202410_543_543_0, 202410_544_544_0, 202410_545_545_0, 202410_546_546_0] 2024.10.22 01:39:45.843977 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_541_108, 202410_542_542_0, 202410_543_543_0, 202410_544_544_0, 202410_545_545_0, 202410_546_546_0] 2024.10.22 01:39:46.174425 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 146 bytes for b2c9782b9ab293c6961f551f22c33e78:0) 2024.10.22 01:39:46.176431 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e1eaf91d946e0dd9577431b725589e5c, offset: 0, size: 148 2024.10.22 01:39:46.179540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e1eaf91d946e0dd9577431b725589e5c for removal 2024.10.22 01:39:46.183382 [ 1476 ] {} CacheMetadata: Key e1eaf91d946e0dd9577431b725589e5c is removed from metadata 2024.10.22 01:39:46.268357 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_907_460, 202410_908_908_0] 2024.10.22 01:39:46.272083 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_907_460, 202410_908_908_0] 2024.10.22 01:39:46.284085 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_907_460, 202410_908_908_0] 2024.10.22 01:39:46.557726 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 12.74 MiB, will set to 1.69 GiB (RSS), difference: -34.58 MiB 2024.10.22 01:39:46.542585 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4772 2024.10.22 01:39:46.765183 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 886719 2024.10.22 01:39:47.128437 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f84ced39dc315d6656c1609e934f02d5) : [0, 229720]: Setting download as failed: File segment: [0, 229720], key: f84ced39dc315d6656c1609e934f02d5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:47.132586 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f84ced39dc315d6656c1609e934f02d5): No space left in cache to reserve 229721 bytes, will continue without cache download 2024.10.22 01:39:47.271717 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f84ced39dc315d6656c1609e934f02d5, offset: 0, size: 0 2024.10.22 01:39:47.272654 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f84ced39dc315d6656c1609e934f02d5 for removal 2024.10.22 01:39:47.275163 [ 1476 ] {} CacheMetadata: Key f84ced39dc315d6656c1609e934f02d5 is removed from metadata 2024.10.22 01:39:47.478282 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 497 2024.10.22 01:39:47.591812 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:47.596165 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:47.676271 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1008_493} MergeTask::MergeProjectionsStage: Merge sorted 66354 rows, containing 13 columns (13 merged, 0 gathered) in 3.619315836 sec., 18333.299166654986 rows/sec., 3.02 MiB/sec. 2024.10.22 01:39:47.745696 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1008_493} MergedBlockOutputStream: filled checksums 202410_1_1008_493 (state Temporary) 2024.10.22 01:39:47.764517 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_568_112} MergeTask::MergeProjectionsStage: Merge sorted 892224 rows, containing 4 columns (4 merged, 0 gathered) in 24.450996278 sec., 36490.29225049559 rows/sec., 547.26 KiB/sec. 2024.10.22 01:39:47.775534 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_568_568_0 (state Temporary) 2024.10.22 01:39:47.851951 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:47.855687 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:47.871121 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_568_568_0 to 202410_575_575_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:47.907695 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 886719 2024.10.22 01:39:47.916293 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_479_479_0 (state Temporary) 2024.10.22 01:39:47.962480 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(6d1f6e259eeb129818313861baf4cd20) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6d1f6e259eeb129818313861baf4cd20, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:39:47.970208 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 495866, file segment info: File segment: [0, 33554431], key: 6d1f6e259eeb129818313861baf4cd20, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:39:47.972755 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 177 entries to flush up to offset 67368 2024.10.22 01:39:47.974487 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:48.071063 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_568_112} MergedBlockOutputStream: filled checksums 202410_1_568_112 (state Temporary) 2024.10.22 01:39:48.078369 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_479_479_0 to 202410_489_489_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:48.132105 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:48.263565 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1008_493} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1008_493 to 202410_1_1008_493 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:48.374005 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 65919 2024.10.22 01:39:48.376172 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:48.377158 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1008_493} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1002_492, 202410_1003_1008_1] -> 202410_1_1008_493 2024.10.22 01:39:48.408917 [ 1399 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1008_493}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 01:39:48.425291 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.53 MiB. 2024.10.22 01:39:48.440159 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_582_582_0 (state Temporary) 2024.10.22 01:39:48.487917 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_582_582_0 to 202410_588_588_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:48.542572 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 67368 2024.10.22 01:39:48.573680 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 497 2024.10.22 01:39:48.574485 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 9126 2024.10.22 01:39:48.634717 [ 1272 ] {} DatabaseCatalog: Execution took 354 ms. 2024.10.22 01:39:48.749745 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 148 bytes for 8f26e1bef7472fe76b32daec72ef9957:0) 2024.10.22 01:39:48.754715 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b2c9782b9ab293c6961f551f22c33e78, offset: 0, size: 146 2024.10.22 01:39:48.759856 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b2c9782b9ab293c6961f551f22c33e78 for removal 2024.10.22 01:39:48.762267 [ 1476 ] {} CacheMetadata: Key b2c9782b9ab293c6961f551f22c33e78 is removed from metadata 2024.10.22 01:39:48.868909 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:48.871051 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:48.880419 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:48.883961 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:49.011263 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_593_593_0 (state Temporary) 2024.10.22 01:39:49.056739 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1001_1001_0 (state Temporary) 2024.10.22 01:39:49.097726 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1001_1001_0 to 202410_1014_1014_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:49.152039 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1008_493 to 202410_1009_1009_0 2024.10.22 01:39:49.172765 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 65919 2024.10.22 01:39:49.175068 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.46 MiB using storage policy from min volume index 0 2024.10.22 01:39:49.178562 [ 1195 ] {} DiskLocal: Reserved 2.46 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:49.201793 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1009_494} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1008_493 to 202410_1009_1009_0 into Wide with storage Full 2024.10.22 01:39:49.280716 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1009_494} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:49.285089 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_593_593_0 to 202410_606_606_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:49.350918 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9126 2024.10.22 01:39:49.399620 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1009_494} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1008_493, total 66354 rows starting from the beginning of the part 2024.10.22 01:39:49.427977 [ 1269 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_601_120 to 202410_606_606_0 2024.10.22 01:39:49.459457 [ 1269 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:49.463612 [ 1269 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:49.487204 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_601_120 to 202410_606_606_0 into Compact with storage Full 2024.10.22 01:39:49.573651 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:49.581002 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_601_120, total 9156 rows starting from the beginning of the part 2024.10.22 01:39:49.588361 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1009_494} MergeTreeSequentialSource: Reading 2 marks from part 202410_1009_1009_0, total 74 rows starting from the beginning of the part 2024.10.22 01:39:49.619184 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_602_602_0, total 13 rows starting from the beginning of the part 2024.10.22 01:39:49.658261 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_603_603_0, total 16 rows starting from the beginning of the part 2024.10.22 01:39:49.657832 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(765e7a6dc95b33e280db7f02a2cc45e7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 765e7a6dc95b33e280db7f02a2cc45e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:39:49.681113 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_568_112/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:39:49.684650 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_604_604_0, total 12 rows starting from the beginning of the part 2024.10.22 01:39:49.737827 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_605_605_0, total 12 rows starting from the beginning of the part 2024.10.22 01:39:49.777134 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_606_606_0, total 23 rows starting from the beginning of the part 2024.10.22 01:39:49.779039 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 496, file segment info: File segment: [0, 33554431], key: 765e7a6dc95b33e280db7f02a2cc45e7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:39:49.782621 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:49.839452 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 13.96 MiB, will set to 1.68 GiB (RSS), difference: 45.60 MiB 2024.10.22 01:39:50.451297 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:39:50.464010 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:39:50.472086 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:39:50.485468 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:39:50.541300 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:39:50.557482 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:39:50.560733 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:39:50.571067 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:39:50.610338 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:39:50.652255 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 01:39:50.678522 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f6025cbc952cf8aa48e71635cb4cc43b) : [0, 322312]: Setting download as failed: File segment: [0, 322312], key: f6025cbc952cf8aa48e71635cb4cc43b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:50.684090 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f6025cbc952cf8aa48e71635cb4cc43b): No space left in cache to reserve 322313 bytes, will continue without cache download 2024.10.22 01:39:50.734230 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hxg/zujqsmnqxswfgrsrekztxvrkixjtc (LogSeriesLimiter: on interval from 2024-10-22 01:39:28 to 2024-10-22 01:39:50 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:39:50.756162 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f6025cbc952cf8aa48e71635cb4cc43b, offset: 0, size: 0 2024.10.22 01:39:50.759509 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f6025cbc952cf8aa48e71635cb4cc43b for removal 2024.10.22 01:39:50.761453 [ 1476 ] {} CacheMetadata: Key f6025cbc952cf8aa48e71635cb4cc43b is removed from metadata 2024.10.22 01:39:50.848214 [ 1258 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:39:51.225002 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_568_112} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_568_112 to 202410_1_568_112 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:51.291316 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} MergeTask::MergeProjectionsStage: Merge sorted 9232 rows, containing 12 columns (12 merged, 0 gathered) in 1.809681136 sec., 5101.4511984170895 rows/sec., 811.39 KiB/sec. 2024.10.22 01:39:51.340974 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_568_112} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_562_111, 202410_563_568_1] -> 202410_1_568_112 2024.10.22 01:39:51.363648 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:39:51.647892 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} MergedBlockOutputStream: filled checksums 202410_1_606_121 (state Temporary) 2024.10.22 01:39:51.981222 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 148 bytes for 694914b639430bc71967cf4918e6764a:0) 2024.10.22 01:39:51.983161 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8f26e1bef7472fe76b32daec72ef9957, offset: 0, size: 148 2024.10.22 01:39:51.985825 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8f26e1bef7472fe76b32daec72ef9957 for removal 2024.10.22 01:39:51.989007 [ 1476 ] {} CacheMetadata: Key 8f26e1bef7472fe76b32daec72ef9957 is removed from metadata 2024.10.22 01:39:52.208425 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_606_121 to 202410_1_606_121 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:52.212705 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_606_121} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_601_120, 202410_606_606_0] -> 202410_1_606_121 2024.10.22 01:39:52.262815 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.53 MiB. 2024.10.22 01:39:52.396615 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 8.40 MiB, will set to 1.69 GiB (RSS), difference: -31.94 MiB 2024.10.22 01:39:52.463848 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_1009_494/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:39:52.889643 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(0051ca12b77879d9bac8677f4f8c496c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0051ca12b77879d9bac8677f4f8c496c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:39:52.908615 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 123635, file segment info: File segment: [0, 33554431], key: 0051ca12b77879d9bac8677f4f8c496c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:39:52.911277 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:53.089509 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_908_461, 202410_909_909_0] 2024.10.22 01:39:53.091387 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_908_461, 202410_909_909_0] 2024.10.22 01:39:53.180790 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 66007 2024.10.22 01:39:53.206198 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_908_461, 202410_909_909_0] 2024.10.22 01:39:53.220147 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1009_494} MergeTask::MergeProjectionsStage: Merge sorted 66428 rows, containing 13 columns (13 merged, 0 gathered) in 4.033179642 sec., 16470.379674697364 rows/sec., 2.71 MiB/sec. 2024.10.22 01:39:53.281058 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1009_494} MergedBlockOutputStream: filled checksums 202410_1_1009_494 (state Temporary) 2024.10.22 01:39:53.341551 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:53.346375 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:53.387725 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1002_1002_0 (state Temporary) 2024.10.22 01:39:53.547187 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1002_1002_0 to 202410_1015_1015_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:53.573831 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1010_1010_0 to 202410_1015_1015_0 2024.10.22 01:39:53.600509 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:53.615243 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f9cd2e9374ba92d9c3860ed8094c20a7) : [0, 297350]: Setting download as failed: File segment: [0, 297350], key: f9cd2e9374ba92d9c3860ed8094c20a7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:53.608281 [ 1157 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:53.617336 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f9cd2e9374ba92d9c3860ed8094c20a7): No space left in cache to reserve 297351 bytes, will continue without cache download 2024.10.22 01:39:53.711776 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66007 2024.10.22 01:39:53.761715 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f9cd2e9374ba92d9c3860ed8094c20a7, offset: 0, size: 0 2024.10.22 01:39:53.765656 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f9cd2e9374ba92d9c3860ed8094c20a7 for removal 2024.10.22 01:39:53.768705 [ 1476 ] {} CacheMetadata: Key f9cd2e9374ba92d9c3860ed8094c20a7 is removed from metadata 2024.10.22 01:39:53.847139 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1010_1010_0 to 202410_1015_1015_0 into Compact with storage Full 2024.10.22 01:39:53.881459 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:39:53.889486 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1010_1010_0, total 60 rows starting from the beginning of the part 2024.10.22 01:39:54.028918 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1011_1011_0, total 81 rows starting from the beginning of the part 2024.10.22 01:39:54.078990 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 4813 2024.10.22 01:39:54.109599 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1009_494} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1009_494 to 202410_1_1009_494 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:54.114448 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1009_494} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1008_493, 202410_1009_1009_0] -> 202410_1_1009_494 2024.10.22 01:39:54.164534 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1012_1012_0, total 38 rows starting from the beginning of the part 2024.10.22 01:39:54.248411 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.44 MiB. 2024.10.22 01:39:54.480766 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1013_1013_0, total 82 rows starting from the beginning of the part 2024.10.22 01:39:54.621224 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1014_1014_0, total 61 rows starting from the beginning of the part 2024.10.22 01:39:54.668168 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(2606b332f81b04157634f9c923884b91) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2606b332f81b04157634f9c923884b91, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:39:54.671568 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 871, file segment info: File segment: [0, 33554431], key: 2606b332f81b04157634f9c923884b91, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:39:54.675209 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:54.764877 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1015_1015_0, total 88 rows starting from the beginning of the part 2024.10.22 01:39:54.912283 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 887489 2024.10.22 01:39:55.407173 [ 1175 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:39:55.517799 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:55.522779 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:55.565544 [ 1175 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:39:55.660839 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 148 bytes for 51517926535ecf9b6559d7839251c3a7:0) 2024.10.22 01:39:55.662399 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 694914b639430bc71967cf4918e6764a, offset: 0, size: 148 2024.10.22 01:39:55.665267 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 694914b639430bc71967cf4918e6764a for removal 2024.10.22 01:39:55.667529 [ 1476 ] {} CacheMetadata: Key 694914b639430bc71967cf4918e6764a is removed from metadata 2024.10.22 01:39:55.964920 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_569_569_0 (state Temporary) 2024.10.22 01:39:56.045594 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_569_569_0 to 202410_576_576_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:56.075772 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 498 2024.10.22 01:39:56.080717 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 887489 2024.10.22 01:39:56.169688 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 158 entries to flush up to offset 67526 2024.10.22 01:39:56.391379 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:56.404068 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:56.423555 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 7.00 MiB, will set to 1.67 GiB (RSS), difference: 29.56 MiB 2024.10.22 01:39:56.424100 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d7b242d66415fc9debc74ffc3547a528) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d7b242d66415fc9debc74ffc3547a528, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:39:56.455946 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 43760, file segment info: File segment: [0, 33554431], key: d7b242d66415fc9debc74ffc3547a528, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:39:56.459507 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:56.464821 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_583_583_0 (state Temporary) 2024.10.22 01:39:56.517552 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_583_583_0 to 202410_589_589_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:56.582713 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} MergeTask::MergeProjectionsStage: Merge sorted 410 rows, containing 13 columns (13 merged, 0 gathered) in 2.845243381 sec., 144.10015070693174 rows/sec., 28.27 KiB/sec. 2024.10.22 01:39:56.616494 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} MergedBlockOutputStream: filled checksums 202410_1010_1015_1 (state Temporary) 2024.10.22 01:39:56.680305 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 67526 2024.10.22 01:39:56.768986 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:56.782038 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:56.854777 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 9144 2024.10.22 01:39:56.961225 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_480_480_0 (state Temporary) 2024.10.22 01:39:57.014635 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1010_1015_1 to 202410_1010_1015_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:57.021245 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1010_1015_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1010_1010_0, 202410_1015_1015_0] -> 202410_1010_1015_1 2024.10.22 01:39:57.030728 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_480_480_0 to 202410_490_490_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:57.043853 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:57.049473 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:39:57.072989 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:39:57.084335 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_594_594_0 (state Temporary) 2024.10.22 01:39:57.143909 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_594_594_0 to 202410_607_607_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:57.158401 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 498 2024.10.22 01:39:57.187982 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9144 2024.10.22 01:39:57.240766 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dc624cbf61f229ba0ecc0f7128ecff8c) : [0, 265638]: Setting download as failed: File segment: [0, 265638], key: dc624cbf61f229ba0ecc0f7128ecff8c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:57.243571 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dc624cbf61f229ba0ecc0f7128ecff8c): No space left in cache to reserve 265639 bytes, will continue without cache download 2024.10.22 01:39:57.271630 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dc624cbf61f229ba0ecc0f7128ecff8c, offset: 0, size: 0 2024.10.22 01:39:57.273168 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dc624cbf61f229ba0ecc0f7128ecff8c for removal 2024.10.22 01:39:57.274863 [ 1476 ] {} CacheMetadata: Key dc624cbf61f229ba0ecc0f7128ecff8c is removed from metadata 2024.10.22 01:39:57.723051 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 66083 2024.10.22 01:39:57.726703 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_909_462, 202410_910_910_0] 2024.10.22 01:39:57.729414 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_909_462, 202410_910_910_0] 2024.10.22 01:39:57.934860 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:39:57.944675 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:39:57.953316 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_909_462, 202410_910_910_0] 2024.10.22 01:39:57.970862 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(3035d6814fd3fbd30ba35847217f1205) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3035d6814fd3fbd30ba35847217f1205, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:39:57.973649 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 744, file segment info: File segment: [0, 33554431], key: 3035d6814fd3fbd30ba35847217f1205, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:39:57.976076 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:39:58.000413 [ 1744 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hhe/wlxuzypemqdxqqkatpvvjauyggfxk, size 43760 2024.10.22 01:39:58.109122 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1003_1003_0 (state Temporary) 2024.10.22 01:39:58.156617 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1003_1003_0 to 202410_1016_1016_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:39:58.180615 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66083 2024.10.22 01:39:58.467201 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:39:58.474043 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:39:58.516283 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:39:58.543974 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:39:58.661562 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:39:58.684551 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 148 bytes for f49402d89e1318cc10ca6a46f46958b8:0) 2024.10.22 01:39:58.687528 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 51517926535ecf9b6559d7839251c3a7, offset: 0, size: 148 2024.10.22 01:39:58.690586 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 51517926535ecf9b6559d7839251c3a7 for removal 2024.10.22 01:39:58.693288 [ 1476 ] {} CacheMetadata: Key 51517926535ecf9b6559d7839251c3a7 is removed from metadata 2024.10.22 01:39:58.772147 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:39:58.783481 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:39:58.910597 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:39:58.976090 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UTMContent 2024.10.22 01:39:59.020761 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tmx/epqubnnvscnkcummwiirokrhersxw (LogSeriesLimiter: on interval from 2024-10-22 01:39:50 to 2024-10-22 01:39:59 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:39:59.719394 [ 1227 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 216 ms. 2024.10.22 01:39:59.750154 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7df4fe38efe4aa68b2f587513ae81dc3) : [0, 265758]: Setting download as failed: File segment: [0, 265758], key: 7df4fe38efe4aa68b2f587513ae81dc3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:39:59.752258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7df4fe38efe4aa68b2f587513ae81dc3): No space left in cache to reserve 265759 bytes, will continue without cache download 2024.10.22 01:39:59.770052 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7df4fe38efe4aa68b2f587513ae81dc3, offset: 0, size: 0 2024.10.22 01:39:59.771258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7df4fe38efe4aa68b2f587513ae81dc3 for removal 2024.10.22 01:39:59.783052 [ 1476 ] {} CacheMetadata: Key 7df4fe38efe4aa68b2f587513ae81dc3 is removed from metadata 2024.10.22 01:40:00.158391 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 01:40:00.464606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 146 bytes for d41eb4f45332dc2eeccd015878f7e119:0) 2024.10.22 01:40:00.466399 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f49402d89e1318cc10ca6a46f46958b8, offset: 0, size: 148 2024.10.22 01:40:00.469475 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f49402d89e1318cc10ca6a46f46958b8 for removal 2024.10.22 01:40:00.471940 [ 1476 ] {} CacheMetadata: Key f49402d89e1318cc10ca6a46f46958b8 is removed from metadata 2024.10.22 01:40:00.557837 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:40:00.561388 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:40:01.262673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(28fcdcdc01ff2fdd844ab89e8cdd5508) : [0, 225610]: Setting download as failed: File segment: [0, 225610], key: 28fcdcdc01ff2fdd844ab89e8cdd5508, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:40:01.264504 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(28fcdcdc01ff2fdd844ab89e8cdd5508): No space left in cache to reserve 225611 bytes, will continue without cache download 2024.10.22 01:40:01.383438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 28fcdcdc01ff2fdd844ab89e8cdd5508, offset: 0, size: 0 2024.10.22 01:40:01.386694 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 28fcdcdc01ff2fdd844ab89e8cdd5508 for removal 2024.10.22 01:40:01.391221 [ 1476 ] {} CacheMetadata: Key 28fcdcdc01ff2fdd844ab89e8cdd5508 is removed from metadata 2024.10.22 01:40:01.574554 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 3.33 MiB, will set to 1.67 GiB (RSS), difference: -3.64 MiB 2024.10.22 01:40:01.825262 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_911_911_0, 202410_912_912_0, 202410_913_913_0, 202410_914_914_0, 202410_915_915_0, 202410_916_916_0] 2024.10.22 01:40:01.828072 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_911_911_0, 202410_912_912_0, 202410_913_913_0, 202410_914_914_0, 202410_915_915_0, 202410_916_916_0] 2024.10.22 01:40:01.853207 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_911_911_0, 202410_912_912_0, 202410_913_913_0, 202410_914_914_0, 202410_915_915_0, 202410_916_916_0] 2024.10.22 01:40:02.188695 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 66132 2024.10.22 01:40:02.461675 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 148 bytes for 1978de49dde5480dbac201f131a64284:0) 2024.10.22 01:40:02.466538 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d41eb4f45332dc2eeccd015878f7e119, offset: 0, size: 146 2024.10.22 01:40:02.471096 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d41eb4f45332dc2eeccd015878f7e119 for removal 2024.10.22 01:40:02.473538 [ 1476 ] {} CacheMetadata: Key d41eb4f45332dc2eeccd015878f7e119 is removed from metadata 2024.10.22 01:40:02.517649 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:02.525467 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:02.588917 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1004_1004_0 (state Temporary) 2024.10.22 01:40:02.762403 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1004_1004_0 to 202410_1017_1017_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:02.913630 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66132 2024.10.22 01:40:03.082939 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 889029 2024.10.22 01:40:03.543689 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4dbf292577fcb0c4cb5e09acbb0fc69f) : [0, 320560]: Setting download as failed: File segment: [0, 320560], key: 4dbf292577fcb0c4cb5e09acbb0fc69f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:40:03.545599 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4dbf292577fcb0c4cb5e09acbb0fc69f): No space left in cache to reserve 320561 bytes, will continue without cache download 2024.10.22 01:40:03.675940 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4dbf292577fcb0c4cb5e09acbb0fc69f, offset: 0, size: 0 2024.10.22 01:40:03.678757 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4dbf292577fcb0c4cb5e09acbb0fc69f for removal 2024.10.22 01:40:03.680769 [ 1476 ] {} CacheMetadata: Key 4dbf292577fcb0c4cb5e09acbb0fc69f is removed from metadata 2024.10.22 01:40:04.035352 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:04.037498 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:04.161559 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_570_570_0 (state Temporary) 2024.10.22 01:40:04.254568 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_515_515_0, 202410_516_516_0, 202410_517_517_0, 202410_518_518_0, 202410_519_519_0, 202410_520_520_0] 2024.10.22 01:40:04.256563 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_515_515_0, 202410_516_516_0, 202410_517_517_0, 202410_518_518_0, 202410_519_519_0, 202410_520_520_0] 2024.10.22 01:40:04.260661 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_570_570_0 to 202410_577_577_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:04.275146 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_515_515_0, 202410_516_516_0, 202410_517_517_0, 202410_518_518_0, 202410_519_519_0, 202410_520_520_0] 2024.10.22 01:40:04.339177 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 45 entries to flush up to offset 67571 2024.10.22 01:40:04.407951 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 889029 2024.10.22 01:40:04.572510 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:04.579860 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:04.661421 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 499 2024.10.22 01:40:04.691045 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 9156 2024.10.22 01:40:04.728362 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_584_584_0 (state Temporary) 2024.10.22 01:40:04.843888 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 145 bytes for 1ff5997d583be31dbdfe8cfee5689b76:0) 2024.10.22 01:40:04.850909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1978de49dde5480dbac201f131a64284, offset: 0, size: 148 2024.10.22 01:40:04.857341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1978de49dde5480dbac201f131a64284 for removal 2024.10.22 01:40:04.860406 [ 1476 ] {} CacheMetadata: Key 1978de49dde5480dbac201f131a64284 is removed from metadata 2024.10.22 01:40:04.903168 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:04.907181 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:04.945424 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_595_595_0 (state Temporary) 2024.10.22 01:40:04.970786 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_584_584_0 to 202410_590_590_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:04.980153 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_595_595_0 to 202410_608_608_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:05.020267 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 67571 2024.10.22 01:40:04.997321 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:05.102817 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:05.126066 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9156 2024.10.22 01:40:05.367194 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_481_481_0 (state Temporary) 2024.10.22 01:40:05.469365 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 2.78 MiB, will set to 1.67 GiB (RSS), difference: -4.18 MiB 2024.10.22 01:40:05.694043 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_481_481_0 to 202410_491_491_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:05.819875 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(43e36fe9063f4a160b7186b6548005ba) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 43e36fe9063f4a160b7186b6548005ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:40:05.823709 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 433028, file segment info: File segment: [0, 33554431], key: 43e36fe9063f4a160b7186b6548005ba, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:40:05.827687 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:05.868101 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 499 2024.10.22 01:40:06.931796 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 66182 2024.10.22 01:40:07.077977 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:07.080678 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:07.151228 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6e82db062f47d507e37c81fd7ddba625) : [0, 167084]: Setting download as failed: File segment: [0, 167084], key: 6e82db062f47d507e37c81fd7ddba625, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:40:07.154855 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6e82db062f47d507e37c81fd7ddba625): No space left in cache to reserve 167085 bytes, will continue without cache download 2024.10.22 01:40:07.177311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6e82db062f47d507e37c81fd7ddba625, offset: 0, size: 0 2024.10.22 01:40:07.178736 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6e82db062f47d507e37c81fd7ddba625 for removal 2024.10.22 01:40:07.192067 [ 1476 ] {} CacheMetadata: Key 6e82db062f47d507e37c81fd7ddba625 is removed from metadata 2024.10.22 01:40:07.263935 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1005_1005_0 (state Temporary) 2024.10.22 01:40:07.544334 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1005_1005_0 to 202410_1018_1018_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:07.585941 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66182 2024.10.22 01:40:07.775299 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(841c6ba4504b8760a79901a4639eada8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 841c6ba4504b8760a79901a4639eada8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:40:07.986738 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 495, file segment info: File segment: [0, 33554431], key: 841c6ba4504b8760a79901a4639eada8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:40:07.989129 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:08.035008 [ 1702 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hxg/zujqsmnqxswfgrsrekztxvrkixjtc, size 433028 2024.10.22 01:40:08.465608 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:40:08.572360 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:40:08.576266 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:40:08.598189 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:40:08.616587 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:40:08.655753 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:40:08.667320 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:40:08.685628 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 01:40:08.735564 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qre/zblkpezrhijnvpdxzjndypkvdvboh (LogSeriesLimiter: on interval from 2024-10-22 01:39:59 to 2024-10-22 01:40:08 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:40:08.892271 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 144 bytes for 7b1892690f1dc7b25b7a1962682a4843:0) 2024.10.22 01:40:08.897397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1ff5997d583be31dbdfe8cfee5689b76, offset: 0, size: 145 2024.10.22 01:40:08.902873 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1ff5997d583be31dbdfe8cfee5689b76 for removal 2024.10.22 01:40:08.906561 [ 1476 ] {} CacheMetadata: Key 1ff5997d583be31dbdfe8cfee5689b76 is removed from metadata 2024.10.22 01:40:09.460477 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 3.64 MiB, will set to 1.67 GiB (RSS), difference: 1.06 MiB 2024.10.22 01:40:09.654821 [ 1150 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 01:40:09.776860 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(76ef99b24253e006d470e1abcca634a8) : [0, 1135]: Setting download as failed: File segment: [0, 1135], key: 76ef99b24253e006d470e1abcca634a8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:40:09.779947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(76ef99b24253e006d470e1abcca634a8): No space left in cache to reserve 1136 bytes, will continue without cache download 2024.10.22 01:40:09.789096 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 76ef99b24253e006d470e1abcca634a8, offset: 0, size: 0 2024.10.22 01:40:09.793397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 76ef99b24253e006d470e1abcca634a8 for removal 2024.10.22 01:40:09.799495 [ 1476 ] {} CacheMetadata: Key 76ef99b24253e006d470e1abcca634a8 is removed from metadata 2024.10.22 01:40:09.883764 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:09.886552 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:10.558375 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:40:10.560774 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:40:10.571381 [ 1253 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:40:10.583753 [ 1253 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:40:10.814503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 175 bytes for d9fd0eab967acb47d37f71802979d273:0) 2024.10.22 01:40:10.816319 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7b1892690f1dc7b25b7a1962682a4843, offset: 0, size: 144 2024.10.22 01:40:10.819393 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7b1892690f1dc7b25b7a1962682a4843 for removal 2024.10.22 01:40:10.822660 [ 1476 ] {} CacheMetadata: Key 7b1892690f1dc7b25b7a1962682a4843 is removed from metadata 2024.10.22 01:40:11.390591 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_438_90, 202410_439_439_0, 202410_440_440_0, 202410_441_441_0, 202410_442_442_0, 202410_443_443_0] 2024.10.22 01:40:11.413576 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 890184 2024.10.22 01:40:11.397221 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_438_90, 202410_439_439_0, 202410_440_440_0, 202410_441_441_0, 202410_442_442_0, 202410_443_443_0] 2024.10.22 01:40:11.457251 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 4.20 MiB, will set to 1.67 GiB (RSS), difference: -3.98 MiB 2024.10.22 01:40:11.657259 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_438_90, 202410_439_439_0, 202410_440_440_0, 202410_441_441_0, 202410_442_442_0, 202410_443_443_0] 2024.10.22 01:40:11.702659 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 66231 2024.10.22 01:40:11.759313 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ab2ce4ac512e8928007f2ab21a79f18a) : [0, 763781]: Setting download as failed: File segment: [0, 763781], key: ab2ce4ac512e8928007f2ab21a79f18a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:40:11.761004 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ab2ce4ac512e8928007f2ab21a79f18a): No space left in cache to reserve 763782 bytes, will continue without cache download 2024.10.22 01:40:11.788149 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ab2ce4ac512e8928007f2ab21a79f18a, offset: 0, size: 0 2024.10.22 01:40:11.790288 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ab2ce4ac512e8928007f2ab21a79f18a for removal 2024.10.22 01:40:11.799280 [ 1476 ] {} CacheMetadata: Key ab2ce4ac512e8928007f2ab21a79f18a is removed from metadata 2024.10.22 01:40:12.109446 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:12.112602 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:12.121550 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:12.125272 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:12.170647 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_527_4, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0, 202410_532_532_0] 2024.10.22 01:40:12.172519 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_527_4, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0, 202410_532_532_0] 2024.10.22 01:40:12.211763 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_527_4, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0, 202410_532_532_0] 2024.10.22 01:40:12.310860 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1006_1006_0 (state Temporary) 2024.10.22 01:40:12.452855 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_571_571_0 (state Temporary) 2024.10.22 01:40:12.454439 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1006_1006_0 to 202410_1019_1019_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:12.512281 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_571_571_0 to 202410_578_578_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:12.534964 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 37 entries to flush up to offset 67608 2024.10.22 01:40:12.620230 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66231 2024.10.22 01:40:12.629424 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 9162 2024.10.22 01:40:12.662712 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 890184 2024.10.22 01:40:12.663347 [ 1194 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_568_112 to 202410_569_574_1 2024.10.22 01:40:12.685154 [ 1194 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:12.688984 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:12.717111 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 185 bytes for a48564e34d50fd6f364e55d9a38b61b0:0) 2024.10.22 01:40:12.719845 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d9fd0eab967acb47d37f71802979d273, offset: 0, size: 175 2024.10.22 01:40:12.722813 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d9fd0eab967acb47d37f71802979d273 for removal 2024.10.22 01:40:12.725036 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_574_113} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_568_112 to 202410_569_574_1 into Wide with storage Full 2024.10.22 01:40:12.726127 [ 1476 ] {} CacheMetadata: Key d9fd0eab967acb47d37f71802979d273 is removed from metadata 2024.10.22 01:40:12.729700 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:12.734335 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:12.736049 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:12.739781 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:12.756458 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_574_113} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:40:12.763181 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_574_113} MergeTreeSequentialSource: Reading 110 marks from part 202410_1_568_112, total 892224 rows starting from the beginning of the part 2024.10.22 01:40:12.765065 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_596_596_0 (state Temporary) 2024.10.22 01:40:12.791469 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_574_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_569_574_1, total 6930 rows starting from the beginning of the part 2024.10.22 01:40:12.881304 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_585_585_0 (state Temporary) 2024.10.22 01:40:13.045794 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_585_585_0 to 202410_591_591_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:13.105019 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_120_120_0 (state Temporary) 2024.10.22 01:40:13.156555 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_596_596_0 to 202410_609_609_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:13.194555 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 67608 2024.10.22 01:40:13.280606 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_508_102, 202410_509_514_1] 2024.10.22 01:40:13.284484 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_508_102, 202410_509_514_1] 2024.10.22 01:40:13.291900 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9162 2024.10.22 01:40:13.305292 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_508_102, 202410_509_514_1] 2024.10.22 01:40:13.307839 [ 1238 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 01:40:13.369573 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 500 2024.10.22 01:40:13.471219 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 4.06 MiB, will set to 1.67 GiB (RSS), difference: -17.00 MiB 2024.10.22 01:40:13.646369 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:13.650242 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:13.709008 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_482_482_0 (state Temporary) 2024.10.22 01:40:13.737960 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e4af3f0ce6d5719793dcda939efe7859) : [0, 1498841]: Setting download as failed: File segment: [0, 1498841], key: e4af3f0ce6d5719793dcda939efe7859, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:40:13.742880 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e4af3f0ce6d5719793dcda939efe7859): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:40:13.788860 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_482_482_0 to 202410_492_492_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:13.880350 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 500 2024.10.22 01:40:13.935372 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e4af3f0ce6d5719793dcda939efe7859, offset: 0, size: 0 2024.10.22 01:40:13.942140 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e4af3f0ce6d5719793dcda939efe7859 for removal 2024.10.22 01:40:13.952944 [ 1476 ] {} CacheMetadata: Key e4af3f0ce6d5719793dcda939efe7859 is removed from metadata 2024.10.22 01:40:14.363435 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_120_120_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:40:14.820113 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 146 bytes for 56a91380a91a2fd2e7cda75a996ba310:0) 2024.10.22 01:40:14.823554 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a48564e34d50fd6f364e55d9a38b61b0, offset: 0, size: 185 2024.10.22 01:40:14.827703 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a48564e34d50fd6f364e55d9a38b61b0 for removal 2024.10.22 01:40:14.833675 [ 1476 ] {} CacheMetadata: Key a48564e34d50fd6f364e55d9a38b61b0 is removed from metadata 2024.10.22 01:40:15.232715 [ 1183 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 252 ms. 2024.10.22 01:40:15.416873 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 8.67 MiB, will set to 1.68 GiB (RSS), difference: 6.03 MiB 2024.10.22 01:40:15.914358 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3e8e81a5d551922673b41283c9ca6309) : [0, 236129]: Setting download as failed: File segment: [0, 236129], key: 3e8e81a5d551922673b41283c9ca6309, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:40:15.916992 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3e8e81a5d551922673b41283c9ca6309): No space left in cache to reserve 236130 bytes, will continue without cache download 2024.10.22 01:40:15.960722 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3e8e81a5d551922673b41283c9ca6309, offset: 0, size: 0 2024.10.22 01:40:15.962641 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3e8e81a5d551922673b41283c9ca6309 for removal 2024.10.22 01:40:15.966775 [ 1476 ] {} CacheMetadata: Key 3e8e81a5d551922673b41283c9ca6309 is removed from metadata 2024.10.22 01:40:16.628348 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 66303 2024.10.22 01:40:16.789659 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 148 bytes for 190784be8e54912744a6aad2bd266cab:0) 2024.10.22 01:40:16.794030 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 56a91380a91a2fd2e7cda75a996ba310, offset: 0, size: 146 2024.10.22 01:40:16.799046 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 56a91380a91a2fd2e7cda75a996ba310 for removal 2024.10.22 01:40:16.801460 [ 1476 ] {} CacheMetadata: Key 56a91380a91a2fd2e7cda75a996ba310 is removed from metadata 2024.10.22 01:40:16.879013 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:16.929697 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:17.090218 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1007_1007_0 (state Temporary) 2024.10.22 01:40:17.198719 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_574_113/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:40:17.240778 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1007_1007_0 to 202410_1020_1020_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:17.275595 [ 1175 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1009_494 to 202410_1010_1015_1 2024.10.22 01:40:17.325447 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66303 2024.10.22 01:40:17.419678 [ 1175 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.48 MiB using storage policy from min volume index 0 2024.10.22 01:40:17.422672 [ 1175 ] {} DiskLocal: Reserved 2.48 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:17.575146 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1015_495} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1009_494 to 202410_1010_1015_1 into Wide with storage Full 2024.10.22 01:40:17.714671 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 14.16 MiB, will set to 1.68 GiB (RSS), difference: 4.29 MiB 2024.10.22 01:40:17.725017 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1015_495} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:40:17.753191 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d5bdc312cf2834e6c68f1493c3eb5131) : [0, 323965]: Setting download as failed: File segment: [0, 323965], key: d5bdc312cf2834e6c68f1493c3eb5131, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:40:17.755238 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d5bdc312cf2834e6c68f1493c3eb5131): No space left in cache to reserve 323966 bytes, will continue without cache download 2024.10.22 01:40:17.768276 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1015_495} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1009_494, total 66428 rows starting from the beginning of the part 2024.10.22 01:40:17.784698 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d5bdc312cf2834e6c68f1493c3eb5131, offset: 0, size: 0 2024.10.22 01:40:17.788918 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d5bdc312cf2834e6c68f1493c3eb5131 for removal 2024.10.22 01:40:17.794676 [ 1476 ] {} CacheMetadata: Key d5bdc312cf2834e6c68f1493c3eb5131 is removed from metadata 2024.10.22 01:40:17.844483 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:40:17.998823 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1015_495} MergeTreeSequentialSource: Reading 2 marks from part 202410_1010_1015_1, total 410 rows starting from the beginning of the part 2024.10.22 01:40:18.569432 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 145 bytes for 9e3c8246b38f1fd20d2d87e4a393fd54:0) 2024.10.22 01:40:18.571598 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 190784be8e54912744a6aad2bd266cab, offset: 0, size: 148 2024.10.22 01:40:18.573629 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 190784be8e54912744a6aad2bd266cab for removal 2024.10.22 01:40:18.574914 [ 1476 ] {} CacheMetadata: Key 190784be8e54912744a6aad2bd266cab is removed from metadata 2024.10.22 01:40:19.064885 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_1015_495/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:40:19.666299 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 891724 2024.10.22 01:40:19.707642 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f4bde2af147ecec731a5bed0f0d2323d) : [0, 126611]: Setting download as failed: File segment: [0, 126611], key: f4bde2af147ecec731a5bed0f0d2323d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:40:19.725045 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f4bde2af147ecec731a5bed0f0d2323d): No space left in cache to reserve 126612 bytes, will continue without cache download 2024.10.22 01:40:19.764256 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f4bde2af147ecec731a5bed0f0d2323d, offset: 0, size: 0 2024.10.22 01:40:19.765976 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f4bde2af147ecec731a5bed0f0d2323d for removal 2024.10.22 01:40:19.769320 [ 1476 ] {} CacheMetadata: Key f4bde2af147ecec731a5bed0f0d2323d is removed from metadata 2024.10.22 01:40:19.769730 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/bdx/hfaizxyftgwyickmtqpdickafcmwi. 2024.10.22 01:40:19.912656 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ypg/gvedqenyzbwiaczbbxpmsdmomfrsh. 2024.10.22 01:40:20.580415 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 13.26 MiB, will set to 1.69 GiB (RSS), difference: -34.60 MiB 2024.10.22 01:40:20.660316 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:40:20.661660 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:40:20.705304 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 164 entries to flush up to offset 67772 2024.10.22 01:40:20.781314 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:20.784038 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:20.793780 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 4 entries to flush up to offset 9166 2024.10.22 01:40:20.978300 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ypg/gvedqenyzbwiaczbbxpmsdmomfrsh. 2024.10.22 01:40:20.980157 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_5_5_0 to 201403_5_5_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:20.998154 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:20.999239 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/bdx/hfaizxyftgwyickmtqpdickafcmwi. 2024.10.22 01:40:21.003089 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:21.077140 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_572_572_0 (state Temporary) 2024.10.22 01:40:21.123008 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:40:21.144332 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:21.151769 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:21.155011 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_597_597_0 (state Temporary) 2024.10.22 01:40:21.165627 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_572_572_0 to 202410_579_579_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:21.206702 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_586_586_0 (state Temporary) 2024.10.22 01:40:21.217278 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_597_597_0 to 202410_610_610_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:21.275403 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_586_586_0 to 202410_592_592_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:21.316271 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9166 2024.10.22 01:40:21.316698 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 891724 2024.10.22 01:40:21.336162 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 66364 2024.10.22 01:40:21.384543 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 501 2024.10.22 01:40:21.440952 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_587_16 to 202410_592_592_0 2024.10.22 01:40:21.461327 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:21.462881 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 67772 2024.10.22 01:40:21.463130 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:21.463710 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:21.587593 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(4c064fbeef40627e0209c72dfe7cb47a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4c064fbeef40627e0209c72dfe7cb47a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:40:21.596921 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 52367, file segment info: File segment: [0, 33554431], key: 4c064fbeef40627e0209c72dfe7cb47a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:40:21.600923 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:21.614428 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1008_1008_0 (state Temporary) 2024.10.22 01:40:21.633159 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_120_120_0 to 202410_123_123_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:21.466509 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:21.668370 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:21.671803 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:21.691777 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1008_1008_0 to 202410_1021_1021_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:21.731483 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1016_1016_0 to 202410_1021_1021_0 2024.10.22 01:40:21.763424 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:21.767889 [ 1210 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:21.771629 [ 1200 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 465 ms. 2024.10.22 01:40:21.778849 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_483_483_0 (state Temporary) 2024.10.22 01:40:21.798161 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66364 2024.10.22 01:40:21.798405 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_587_16 to 202410_592_592_0 into Compact with storage Full 2024.10.22 01:40:21.918807 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1016_1016_0 to 202410_1021_1021_0 into Compact with storage Full 2024.10.22 01:40:21.963769 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:40:21.969721 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_587_16, total 8066 rows starting from the beginning of the part 2024.10.22 01:40:21.973396 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_483_483_0 to 202410_493_493_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:21.985910 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_588_588_0, total 177 rows starting from the beginning of the part 2024.10.22 01:40:21.996843 [ 1209 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_488_100 to 202410_493_493_0 2024.10.22 01:40:22.016100 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_589_589_0, total 158 rows starting from the beginning of the part 2024.10.22 01:40:22.029695 [ 1209 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:22.038221 [ 1209 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:22.062480 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 501 2024.10.22 01:40:22.075716 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_488_100 to 202410_493_493_0 into Compact with storage Full 2024.10.22 01:40:22.080128 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_590_590_0, total 45 rows starting from the beginning of the part 2024.10.22 01:40:22.103695 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_591_591_0, total 37 rows starting from the beginning of the part 2024.10.22 01:40:22.141108 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_592_592_0, total 164 rows starting from the beginning of the part 2024.10.22 01:40:22.166364 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:40:22.170059 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1015_495} MergeTask::MergeProjectionsStage: Merge sorted 66838 rows, containing 13 columns (13 merged, 0 gathered) in 4.602301664 sec., 14522.733379869122 rows/sec., 2.40 MiB/sec. 2024.10.22 01:40:22.186024 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1016_1016_0, total 76 rows starting from the beginning of the part 2024.10.22 01:40:22.360804 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1017_1017_0, total 49 rows starting from the beginning of the part 2024.10.22 01:40:22.408975 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1018_1018_0, total 50 rows starting from the beginning of the part 2024.10.22 01:40:22.417449 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 37 2024.10.22 01:40:22.474728 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:40:22.485774 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1019_1019_0, total 49 rows starting from the beginning of the part 2024.10.22 01:40:22.548263 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_488_100, total 532 rows starting from the beginning of the part 2024.10.22 01:40:22.568085 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1015_495} MergedBlockOutputStream: filled checksums 202410_1_1015_495 (state Temporary) 2024.10.22 01:40:22.588670 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:22.590889 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:22.561560 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1020_1020_0, total 72 rows starting from the beginning of the part 2024.10.22 01:40:22.705697 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1021_1021_0, total 61 rows starting from the beginning of the part 2024.10.22 01:40:22.754381 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_31_31_0 (state Temporary) 2024.10.22 01:40:22.789164 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(445d7591deea600b4eb19e1bc67ec1d8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 445d7591deea600b4eb19e1bc67ec1d8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:40:22.794174 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 704, file segment info: File segment: [0, 33554431], key: 445d7591deea600b4eb19e1bc67ec1d8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:40:22.803882 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:22.806850 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_489_489_0, total 1 rows starting from the beginning of the part 2024.10.22 01:40:22.843648 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_31_31_0 to 202410_31_31_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:23.193577 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_490_490_0, total 1 rows starting from the beginning of the part 2024.10.22 01:40:23.261078 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 37 2024.10.22 01:40:23.368488 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_491_491_0, total 1 rows starting from the beginning of the part 2024.10.22 01:40:23.516043 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.20 GiB, peak 1.97 GiB, free memory in arenas 8.59 MiB, will set to 1.42 GiB (RSS), difference: 231.94 MiB 2024.10.22 01:40:23.583813 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(0a587d3c9177ec3e605c58c6e72f7ae1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0a587d3c9177ec3e605c58c6e72f7ae1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:40:23.591534 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 811946, file segment info: File segment: [0, 33554431], key: 0a587d3c9177ec3e605c58c6e72f7ae1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:40:23.617487 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_492_492_0, total 1 rows starting from the beginning of the part 2024.10.22 01:40:23.655357 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} MergeTask::MergeProjectionsStage: Merge sorted 8647 rows, containing 13 columns (13 merged, 0 gathered) in 1.879016186 sec., 4601.8762714372915 rows/sec., 1.99 MiB/sec. 2024.10.22 01:40:23.661497 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4813 2024.10.22 01:40:23.679323 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} MergedBlockOutputStream: filled checksums 202410_507_592_17 (state Temporary) 2024.10.22 01:40:23.699022 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:23.700905 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1015_495} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1015_495 to 202410_1_1015_495 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:23.705784 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1015_495} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1009_494, 202410_1010_1015_1] -> 202410_1_1015_495 2024.10.22 01:40:23.738400 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.54 MiB. 2024.10.22 01:40:23.758538 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} MergeTask::MergeProjectionsStage: Merge sorted 357 rows, containing 13 columns (13 merged, 0 gathered) in 1.84600141 sec., 193.39096821166567 rows/sec., 32.31 KiB/sec. 2024.10.22 01:40:23.811839 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} MergedBlockOutputStream: filled checksums 202410_1016_1021_1 (state Temporary) 2024.10.22 01:40:23.954683 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_493_493_0, total 1 rows starting from the beginning of the part 2024.10.22 01:40:23.976456 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_592_17 to 202410_507_592_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:23.979629 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_592_17} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_587_16, 202410_592_592_0] -> 202410_507_592_17 2024.10.22 01:40:24.009726 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.19 MiB. 2024.10.22 01:40:24.152871 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1016_1021_1 to 202410_1016_1021_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:24.156921 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1016_1021_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1016_1016_0, 202410_1021_1021_0] -> 202410_1016_1021_1 2024.10.22 01:40:24.179936 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:40:24.376337 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(251f9b604f97d7027d387853e0788706) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 251f9b604f97d7027d387853e0788706, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:40:24.380681 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 19441, file segment info: File segment: [0, 33554431], key: 251f9b604f97d7027d387853e0788706, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:40:24.384362 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:25.302772 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(c3738ae5e1f45a9dacd27a5d3c8d85f7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c3738ae5e1f45a9dacd27a5d3c8d85f7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:40:25.319314 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 833, file segment info: File segment: [0, 33554431], key: c3738ae5e1f45a9dacd27a5d3c8d85f7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:40:25.324239 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:25.324305 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_546_109, 202410_547_547_0, 202410_548_548_0, 202410_549_549_0, 202410_550_550_0, 202410_551_551_0] 2024.10.22 01:40:25.328110 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_546_109, 202410_547_547_0, 202410_548_548_0, 202410_549_549_0, 202410_550_550_0, 202410_551_551_0] 2024.10.22 01:40:25.467042 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_546_109, 202410_547_547_0, 202410_548_548_0, 202410_549_549_0, 202410_550_550_0, 202410_551_551_0] 2024.10.22 01:40:25.468584 [ 1722 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qre/zblkpezrhijnvpdxzjndypkvdvboh, size 811946 2024.10.22 01:40:25.587834 [ 1241 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:40:25.647579 [ 1241 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:40:25.790904 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:40:25.807511 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 66457 2024.10.22 01:40:25.809406 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:40:25.851374 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:40:25.959588 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:40:26.112190 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:40:26.159761 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:40:26.181096 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(690ac04fd9f9b4c56fe023b8a967b1f9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 690ac04fd9f9b4c56fe023b8a967b1f9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:40:26.185685 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 632, file segment info: File segment: [0, 33554431], key: 690ac04fd9f9b4c56fe023b8a967b1f9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:40:26.189014 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:26.237824 [ 1740 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tmx/epqubnnvscnkcummwiirokrhersxw, size 19441 2024.10.22 01:40:26.281132 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:26.284428 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:26.293661 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:40:26.350450 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 01:40:26.376628 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1009_1009_0 (state Temporary) 2024.10.22 01:40:26.561628 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/avv/serunvytomfqarnpaekhdupvbgaux (LogSeriesLimiter: on interval from 2024-10-22 01:40:08 to 2024-10-22 01:40:26 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:40:26.721060 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_910_463, 202410_911_916_1] 2024.10.22 01:40:26.722643 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1009_1009_0 to 202410_1022_1022_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:26.725376 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_910_463, 202410_911_916_1] 2024.10.22 01:40:26.765010 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_910_463, 202410_911_916_1] 2024.10.22 01:40:26.779327 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} MergeTask::MergeProjectionsStage: Merge sorted 537 rows, containing 32 columns (32 merged, 0 gathered) in 4.709283867 sec., 114.0300765819178 rows/sec., 10.49 KiB/sec. 2024.10.22 01:40:26.793288 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66457 2024.10.22 01:40:26.797010 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:40:26.839096 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:40:26.884092 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:40:26.923676 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:40:26.939769 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} MergedBlockOutputStream: filled checksums 202410_1_493_101 (state Temporary) 2024.10.22 01:40:26.979528 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:40:27.007763 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:40:27.043978 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:40:27.171169 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:40:27.202496 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UTMTerm 2024.10.22 01:40:27.359971 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_493_101 to 202410_1_493_101 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:27.367717 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_493_101} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_488_100, 202410_493_493_0] -> 202410_1_493_101 2024.10.22 01:40:27.465582 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.97 GiB, free memory in arenas 15.96 MiB, will set to 1.48 GiB (RSS), difference: -131.47 MiB 2024.10.22 01:40:27.518264 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:40:28.324508 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 892879 2024.10.22 01:40:28.815107 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_574_113/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:40:28.818112 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 33 entries to flush up to offset 9199 2024.10.22 01:40:28.969545 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:28.972277 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:29.019549 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 67872 2024.10.22 01:40:29.030643 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_598_598_0 (state Temporary) 2024.10.22 01:40:29.112181 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:29.115024 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:29.289149 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_598_598_0 to 202410_611_611_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:29.363457 [ 1278 ] {} DatabaseCatalog: Execution took 311 ms. 2024.10.22 01:40:29.368456 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:29.371680 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:29.424841 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_587_587_0 (state Temporary) 2024.10.22 01:40:29.416409 [ 1274 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_606_121 to 202410_611_611_0 2024.10.22 01:40:29.467174 [ 1274 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:29.473117 [ 1274 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:29.489211 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_606_121 to 202410_611_611_0 into Compact with storage Full 2024.10.22 01:40:29.556818 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9199 2024.10.22 01:40:29.569588 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:40:29.601235 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_573_573_0 (state Temporary) 2024.10.22 01:40:29.603551 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_606_121, total 9232 rows starting from the beginning of the part 2024.10.22 01:40:29.672200 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_587_587_0 to 202410_593_593_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:29.801483 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 1.97 GiB, free memory in arenas 15.53 MiB, will set to 1.52 GiB (RSS), difference: -75.65 MiB 2024.10.22 01:40:29.876973 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_607_607_0, total 18 rows starting from the beginning of the part 2024.10.22 01:40:30.031355 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_608_608_0, total 12 rows starting from the beginning of the part 2024.10.22 01:40:30.036068 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_573_573_0 to 202410_580_580_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:30.074215 [ 1253 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_575_575_0 to 202410_580_580_0 2024.10.22 01:40:30.111004 [ 1253 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:30.117652 [ 1253 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:30.145565 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_575_575_0 to 202410_580_580_0 into Compact with storage Full 2024.10.22 01:40:30.195376 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.177991201 sec. 2024.10.22 01:40:30.208133 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_609_609_0, total 6 rows starting from the beginning of the part 2024.10.22 01:40:30.276787 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 67872 2024.10.22 01:40:30.275295 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_610_610_0, total 4 rows starting from the beginning of the part 2024.10.22 01:40:30.277226 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:40:30.280752 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 892879 2024.10.22 01:40:30.287034 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_575_575_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:40:30.311053 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_576_576_0, total 770 rows starting from the beginning of the part 2024.10.22 01:40:30.328477 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_577_577_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:40:30.428424 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_578_578_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:40:30.429084 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_611_611_0, total 33 rows starting from the beginning of the part 2024.10.22 01:40:30.454213 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_579_579_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:40:30.469620 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_580_580_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:40:30.660873 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:40:30.663457 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:40:30.824538 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 66539 2024.10.22 01:40:31.103847 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:31.107800 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:31.190834 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 4850 2024.10.22 01:40:31.296487 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1010_1010_0 (state Temporary) 2024.10.22 01:40:31.441295 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1010_1010_0 to 202410_1023_1023_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:31.441299 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.97 GiB, free memory in arenas 16.52 MiB, will set to 1.57 GiB (RSS), difference: -21.50 MiB 2024.10.22 01:40:31.487418 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66539 2024.10.22 01:40:31.613680 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} MergeTask::MergeProjectionsStage: Merge sorted 7315 rows, containing 4 columns (4 merged, 0 gathered) in 1.474310813 sec., 4961.640337640256 rows/sec., 77.53 KiB/sec. 2024.10.22 01:40:31.671461 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} MergedBlockOutputStream: filled checksums 202410_575_580_1 (state Temporary) 2024.10.22 01:40:31.713630 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_575_580_1 to 202410_575_580_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:31.717486 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_575_580_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_575_575_0, 202410_580_580_0] -> 202410_575_580_1 2024.10.22 01:40:31.771718 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:40:31.997176 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} MergeTask::MergeProjectionsStage: Merge sorted 9305 rows, containing 12 columns (12 merged, 0 gathered) in 2.515113253 sec., 3699.6345945460293 rows/sec., 588.43 KiB/sec. 2024.10.22 01:40:32.237651 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} MergedBlockOutputStream: filled checksums 202410_1_611_122 (state Temporary) 2024.10.22 01:40:32.258342 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_916_464, 202410_917_917_0] 2024.10.22 01:40:32.260410 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_916_464, 202410_917_917_0] 2024.10.22 01:40:32.267862 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_916_464, 202410_917_917_0] 2024.10.22 01:40:32.704360 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_611_122 to 202410_1_611_122 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:32.710767 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_611_122} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_606_121, 202410_611_611_0] -> 202410_1_611_122 2024.10.22 01:40:32.760964 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.57 MiB. 2024.10.22 01:40:33.315030 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 13.33 MiB, will set to 1.62 GiB (RSS), difference: -38.67 MiB 2024.10.22 01:40:34.381225 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 16.54 MiB, will set to 1.65 GiB (RSS), difference: -45.26 MiB 2024.10.22 01:40:35.595959 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 23 entries to flush up to offset 66562 2024.10.22 01:40:35.794668 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_917_465, 202410_918_918_0] 2024.10.22 01:40:35.798482 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_917_465, 202410_918_918_0] 2024.10.22 01:40:35.811502 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_917_465, 202410_918_918_0] 2024.10.22 01:40:35.964026 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:35.967338 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:36.002846 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1011_1011_0 (state Temporary) 2024.10.22 01:40:36.064277 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1011_1011_0 to 202410_1024_1024_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:36.144110 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66562 2024.10.22 01:40:36.313002 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.97 GiB, free memory in arenas 14.86 MiB, will set to 1.80 GiB (RSS), difference: 12.31 MiB 2024.10.22 01:40:37.064760 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 502 2024.10.22 01:40:37.065665 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 9218 2024.10.22 01:40:37.209337 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:37.213815 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:37.256323 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_599_599_0 (state Temporary) 2024.10.22 01:40:37.287399 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 894804 2024.10.22 01:40:37.319813 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_599_599_0 to 202410_612_612_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:37.365397 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:37.467988 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:37.467930 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.80 GiB, peak 1.97 GiB, free memory in arenas 14.95 MiB, will set to 1.87 GiB (RSS), difference: 79.41 MiB 2024.10.22 01:40:37.487012 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9218 2024.10.22 01:40:37.516330 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_484_484_0 (state Temporary) 2024.10.22 01:40:37.594823 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_484_484_0 to 202410_494_494_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:37.607110 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_6_6_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:40:37.673627 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 502 2024.10.22 01:40:37.817667 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 80 entries to flush up to offset 67952 2024.10.22 01:40:37.958078 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:38.042367 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:37.961668 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:38.065415 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:38.210607 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_574_574_0 (state Temporary) 2024.10.22 01:40:38.228477 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_588_588_0 (state Temporary) 2024.10.22 01:40:38.367677 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_588_588_0 to 202410_594_594_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:38.420415 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 67952 2024.10.22 01:40:38.592690 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_574_113} MergeTask::MergeProjectionsStage: Merge sorted 899154 rows, containing 4 columns (4 merged, 0 gathered) in 25.875393647 sec., 34749.38438682452 rows/sec., 521.01 KiB/sec. 2024.10.22 01:40:38.606818 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_574_574_0 to 202410_581_581_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:38.658170 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 894804 2024.10.22 01:40:39.071166 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_574_113} MergedBlockOutputStream: filled checksums 202410_1_574_113 (state Temporary) 2024.10.22 01:40:39.768788 [ 1264 ] {} DatabaseCatalog: Execution took 351 ms. 2024.10.22 01:40:39.795980 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 1.97 GiB, free memory in arenas 17.20 MiB, will set to 1.66 GiB (RSS), difference: 312.60 MiB 2024.10.22 01:40:40.148021 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 66602 2024.10.22 01:40:40.215317 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_574_113/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:40:40.307559 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:40.309648 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:40.486596 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1012_1012_0 (state Temporary) 2024.10.22 01:40:40.512456 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_918_466, 202410_919_919_0] 2024.10.22 01:40:40.515109 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_918_466, 202410_919_919_0] 2024.10.22 01:40:40.523289 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1012_1012_0 to 202410_1025_1025_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:40.535073 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_918_466, 202410_919_919_0] 2024.10.22 01:40:40.578439 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66602 2024.10.22 01:40:40.656116 [ 1243 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:40:40.662793 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:40:40.667683 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:40:40.675880 [ 1243 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:40:40.862101 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 465.91 MiB using storage policy from min volume index 0 2024.10.22 01:40:41.437501 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 11.43 MiB, will set to 1.65 GiB (RSS), difference: -5.38 MiB 2024.10.22 01:40:41.895934 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_574_113} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_574_113 to 202410_1_574_113 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:41.905736 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_574_113} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_568_112, 202410_569_574_1] -> 202410_1_574_113 2024.10.22 01:40:41.942845 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:40:42.460851 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nit/cxccobazsvtnymxnhrzeojpgqpwxb (LogSeriesLimiter: on interval from 2024-10-22 01:40:26 to 2024-10-22 01:40:42 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:40:43.763365 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 69 bytes for 01a14955519cf3b2bba7903e2c592808:0) 2024.10.22 01:40:43.765066 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9e3c8246b38f1fd20d2d87e4a393fd54, offset: 0, size: 145 2024.10.22 01:40:43.769637 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9e3c8246b38f1fd20d2d87e4a393fd54 for removal 2024.10.22 01:40:43.771443 [ 1476 ] {} CacheMetadata: Key 9e3c8246b38f1fd20d2d87e4a393fd54 is removed from metadata 2024.10.22 01:40:43.771874 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: bf91bb027b9592e231868bfc9f592095, offset: 0, size: 4 2024.10.22 01:40:43.773674 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key bf91bb027b9592e231868bfc9f592095 for removal 2024.10.22 01:40:43.784509 [ 1476 ] {} CacheMetadata: Key bf91bb027b9592e231868bfc9f592095 is removed from metadata 2024.10.22 01:40:44.524111 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(331786e2d1d0325ec29a316d2fba5717) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 331786e2d1d0325ec29a316d2fba5717, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:40:44.530233 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 98, file segment info: File segment: [0, 33554431], key: 331786e2d1d0325ec29a316d2fba5717, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:40:44.533661 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:44.584102 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 66632 2024.10.22 01:40:44.799715 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:44.803379 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:44.958486 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1013_1013_0 (state Temporary) 2024.10.22 01:40:44.988990 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 9226 2024.10.22 01:40:45.107510 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:45.111982 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:45.148136 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_600_600_0 (state Temporary) 2024.10.22 01:40:45.172041 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1013_1013_0 to 202410_1026_1026_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:45.175957 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 503 2024.10.22 01:40:45.195725 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1015_495 to 202410_1016_1021_1 2024.10.22 01:40:45.207680 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.49 MiB using storage policy from min volume index 0 2024.10.22 01:40:45.211595 [ 1215 ] {} DiskLocal: Reserved 2.49 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:45.228712 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1021_496} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1015_495 to 202410_1016_1021_1 into Wide with storage Full 2024.10.22 01:40:45.290010 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1021_496} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:40:45.310954 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1021_496} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1015_495, total 66838 rows starting from the beginning of the part 2024.10.22 01:40:45.329270 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_600_600_0 to 202410_613_613_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:45.374510 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9226 2024.10.22 01:40:45.394831 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.97 GiB, peak 1.97 GiB, free memory in arenas 10.09 MiB, will set to 1.65 GiB (RSS), difference: -327.10 MiB 2024.10.22 01:40:45.431263 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66632 2024.10.22 01:40:45.466877 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:45.471374 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:45.493292 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(cb514d3392bea46c9cb138f56e180337) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cb514d3392bea46c9cb138f56e180337, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:40:45.499019 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 438, file segment info: File segment: [0, 33554431], key: cb514d3392bea46c9cb138f56e180337, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:40:45.505303 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:45.581410 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1021_496} MergeTreeSequentialSource: Reading 2 marks from part 202410_1016_1021_1, total 357 rows starting from the beginning of the part 2024.10.22 01:40:45.651395 [ 1737 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/avv/serunvytomfqarnpaekhdupvbgaux, size 98 2024.10.22 01:40:45.671364 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1546 entries to flush up to offset 896350 2024.10.22 01:40:45.748191 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_485_485_0 (state Temporary) 2024.10.22 01:40:45.928281 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_485_485_0 to 202410_495_495_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:45.940138 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 68002 2024.10.22 01:40:46.054858 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:46.057827 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:46.059055 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 503 2024.10.22 01:40:46.115004 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:40:46.163119 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:46.165017 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:46.210204 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:40:46.218867 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:40:46.240276 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:40:46.261374 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:40:46.283151 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:40:46.311187 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:40:46.330616 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:46.336065 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:40:46.353128 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column PageCharset 2024.10.22 01:40:46.455363 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_589_589_0 (state Temporary) 2024.10.22 01:40:46.485388 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_919_467, 202410_920_920_0] 2024.10.22 01:40:46.486335 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_589_589_0 to 202410_595_595_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:46.487365 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_919_467, 202410_920_920_0] 2024.10.22 01:40:46.496597 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_919_467, 202410_920_920_0] 2024.10.22 01:40:46.548691 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 68002 2024.10.22 01:40:46.580074 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_575_575_0 (state Temporary) 2024.10.22 01:40:46.681583 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_575_575_0 to 202410_582_582_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:46.724293 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 896350 2024.10.22 01:40:47.104342 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xxv/mlplsnnfcqoezetnxfwdbcjadwubd (LogSeriesLimiter: on interval from 2024-10-22 01:40:42 to 2024-10-22 01:40:47 accepted series 1 / 303 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:40:47.607344 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.88 GiB, peak 1.97 GiB, free memory in arenas 5.15 MiB, will set to 1.65 GiB (RSS), difference: -234.09 MiB 2024.10.22 01:40:48.141012 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:40:49.436831 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 66690 2024.10.22 01:40:49.718204 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_121_121_0 (state Temporary) 2024.10.22 01:40:49.860819 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:49.862949 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:49.876518 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 5.00 MiB, will set to 1.66 GiB (RSS), difference: -23.24 MiB 2024.10.22 01:40:49.917061 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1021_496} MergeTask::MergeProjectionsStage: Merge sorted 67195 rows, containing 13 columns (13 merged, 0 gathered) in 4.699554163 sec., 14298.164819342246 rows/sec., 2.36 MiB/sec. 2024.10.22 01:40:49.982297 [ 1152 ] {} DatabaseCatalog: Execution took 211 ms. 2024.10.22 01:40:50.012190 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1014_1014_0 (state Temporary) 2024.10.22 01:40:50.066956 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1021_496} MergedBlockOutputStream: filled checksums 202410_1_1021_496 (state Temporary) 2024.10.22 01:40:50.110085 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c8d65f8435a267d1ee23a161727ab924) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c8d65f8435a267d1ee23a161727ab924, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:40:50.116345 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 35877, file segment info: File segment: [0, 33554431], key: c8d65f8435a267d1ee23a161727ab924, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:40:50.119915 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:50.164647 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1014_1014_0 to 202410_1027_1027_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:50.192225 [ 1167 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1022_1022_0 to 202410_1027_1027_0 2024.10.22 01:40:50.210839 [ 1167 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:50.215618 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:50.247985 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1022_1022_0 to 202410_1027_1027_0 into Compact with storage Full 2024.10.22 01:40:50.252715 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66690 2024.10.22 01:40:50.400052 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:40:50.471032 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1022_1022_0, total 93 rows starting from the beginning of the part 2024.10.22 01:40:50.655443 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1023_1023_0, total 82 rows starting from the beginning of the part 2024.10.22 01:40:50.661202 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:40:50.663248 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:40:50.749078 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1024_1024_0, total 23 rows starting from the beginning of the part 2024.10.22 01:40:50.832003 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1025_1025_0, total 40 rows starting from the beginning of the part 2024.10.22 01:40:50.921215 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1026_1026_0, total 30 rows starting from the beginning of the part 2024.10.22 01:40:50.970502 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1021_496} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1021_496 to 202410_1_1021_496 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:50.973047 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1021_496} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1015_495, 202410_1016_1021_1] -> 202410_1_1021_496 2024.10.22 01:40:51.002218 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1027_1027_0, total 58 rows starting from the beginning of the part 2024.10.22 01:40:51.002682 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.53 MiB. 2024.10.22 01:40:51.476747 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ab63d1745910e27630448f8b5fa5c85b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ab63d1745910e27630448f8b5fa5c85b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:40:51.479284 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 630, file segment info: File segment: [0, 33554431], key: ab63d1745910e27630448f8b5fa5c85b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:40:51.483981 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:51.643141 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 5.93 MiB, will set to 1.66 GiB (RSS), difference: 28.24 MiB 2024.10.22 01:40:51.885688 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_920_468, 202410_921_921_0] 2024.10.22 01:40:51.888882 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_920_468, 202410_921_921_0] 2024.10.22 01:40:52.009416 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_920_468, 202410_921_921_0] 2024.10.22 01:40:52.448118 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(154613fad93885bd3aadb530b8f0e9f2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 154613fad93885bd3aadb530b8f0e9f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:40:52.452064 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 10540, file segment info: File segment: [0, 33554431], key: 154613fad93885bd3aadb530b8f0e9f2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:40:52.454883 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:52.504336 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} MergeTask::MergeProjectionsStage: Merge sorted 326 rows, containing 13 columns (13 merged, 0 gathered) in 2.268033995 sec., 143.73682260437192 rows/sec., 24.89 KiB/sec. 2024.10.22 01:40:52.561473 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} MergedBlockOutputStream: filled checksums 202410_1022_1027_1 (state Temporary) 2024.10.22 01:40:52.565941 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_121_121_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:40:52.667047 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1022_1027_1 to 202410_1022_1027_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:52.670412 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1022_1027_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1022_1022_0, 202410_1027_1027_0] -> 202410_1022_1027_1 2024.10.22 01:40:52.694536 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:40:52.877421 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 9242 2024.10.22 01:40:53.028924 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(6c81ce8932f8daea67dc6eabbeb1f6ae) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6c81ce8932f8daea67dc6eabbeb1f6ae, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:40:53.037665 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 591, file segment info: File segment: [0, 33554431], key: 6c81ce8932f8daea67dc6eabbeb1f6ae, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 01:40:53.046023 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:40:53.280999 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:53.284634 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:53.306633 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column FromTag 2024.10.22 01:40:53.321792 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_601_601_0 (state Temporary) 2024.10.22 01:40:53.358326 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column FromTag 2024.10.22 01:40:53.379622 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column FromTag 2024.10.22 01:40:53.474431 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_601_601_0 to 202410_614_614_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:53.546099 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column FromTag 2024.10.22 01:40:53.561603 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 504 2024.10.22 01:40:53.610485 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column FromTag 2024.10.22 01:40:53.652634 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column FromTag 2024.10.22 01:40:53.671729 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9242 2024.10.22 01:40:53.773689 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column FromTag 2024.10.22 01:40:53.832041 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1534 entries to flush up to offset 897884 2024.10.22 01:40:53.831991 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column FromTag 2024.10.22 01:40:53.957531 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:53.960506 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:53.992948 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column FromTag 2024.10.22 01:40:54.014406 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_486_486_0 (state Temporary) 2024.10.22 01:40:54.059207 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 140 entries to flush up to offset 68142 2024.10.22 01:40:54.181389 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_486_486_0 to 202410_496_496_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:54.213346 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/biu/gqqykazxexqodovzxjgdggxztsehi (LogSeriesLimiter: on interval from 2024-10-22 01:40:47 to 2024-10-22 01:40:54 accepted series 1 / 48 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:40:54.256858 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:54.266431 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 66762 2024.10.22 01:40:54.267843 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:54.281250 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 504 2024.10.22 01:40:54.393290 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_922_922_0, 202410_923_923_0, 202410_924_924_0, 202410_925_925_0, 202410_926_926_0, 202410_927_927_0] 2024.10.22 01:40:54.401222 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_922_922_0, 202410_923_923_0, 202410_924_924_0, 202410_925_925_0, 202410_926_926_0, 202410_927_927_0] 2024.10.22 01:40:54.508472 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 4.97 MiB, will set to 1.67 GiB (RSS), difference: -6.41 MiB 2024.10.22 01:40:54.551558 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_922_922_0, 202410_923_923_0, 202410_924_924_0, 202410_925_925_0, 202410_926_926_0, 202410_927_927_0] 2024.10.22 01:40:54.554855 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_590_590_0 (state Temporary) 2024.10.22 01:40:54.617439 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:54.635594 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:54.644104 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_590_590_0 to 202410_596_596_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:54.714546 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1015_1015_0 (state Temporary) 2024.10.22 01:40:54.777031 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1015_1015_0 to 202410_1028_1028_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:54.797287 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 68142 2024.10.22 01:40:54.827596 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66762 2024.10.22 01:40:54.983709 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:54.985695 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:55.212484 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_576_576_0 (state Temporary) 2024.10.22 01:40:55.281589 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_576_576_0 to 202410_583_583_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:55.430315 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 897884 2024.10.22 01:40:55.679759 [ 1268 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:40:55.722483 [ 1268 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:40:56.430827 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_521_521_0, 202410_522_522_0, 202410_523_523_0, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0] 2024.10.22 01:40:56.435758 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_521_521_0, 202410_522_522_0, 202410_523_523_0, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0] 2024.10.22 01:40:56.461203 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_521_521_0, 202410_522_522_0, 202410_523_523_0, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0] 2024.10.22 01:40:56.657613 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 4.77 MiB, will set to 1.68 GiB (RSS), difference: -6.59 MiB 2024.10.22 01:40:56.719897 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_532_5, 202410_533_533_0, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0] 2024.10.22 01:40:56.722610 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_532_5, 202410_533_533_0, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0] 2024.10.22 01:40:56.764022 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_532_5, 202410_533_533_0, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0] 2024.10.22 01:40:58.534869 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 4.50 MiB, will set to 1.69 GiB (RSS), difference: 10.26 MiB 2024.10.22 01:40:58.852481 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 66793 2024.10.22 01:40:59.074386 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_121_121_0 to 202410_124_124_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:59.112214 [ 1189 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 7 parts from 202410_1_114_22 to 202410_124_124_0 2024.10.22 01:40:59.153818 [ 1189 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 2.46 MiB using storage policy from min volume index 0 2024.10.22 01:40:59.157751 [ 1189 ] {} DiskLocal: Reserved 2.46 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:59.175565 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:40:59.176908 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} MergeTask::PrepareStage: Merging 7 parts: from 202410_1_114_22 to 202410_124_124_0 into Compact with storage Full 2024.10.22 01:40:59.180680 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:40:59.185697 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 5 old parts to remove. Parts: [202410_105_105_0, 202410_106_106_0, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0] 2024.10.22 01:40:59.188806 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from filesystem (serially): Parts: [202410_105_105_0, 202410_106_106_0, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0] 2024.10.22 01:40:59.228555 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from memory: Parts: [202410_105_105_0, 202410_106_106_0, 202410_107_107_0, 202410_108_108_0, 202410_109_109_0] 2024.10.22 01:40:59.246621 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1016_1016_0 (state Temporary) 2024.10.22 01:40:59.301253 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1016_1016_0 to 202410_1029_1029_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:40:59.546861 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66793 2024.10.22 01:41:00.190412 [ 1187 ] {} DatabaseCatalog: Execution took 205 ms. 2024.10.22 01:41:00.467152 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 2.66 MiB, will set to 1.69 GiB (RSS), difference: 6.24 MiB 2024.10.22 01:41:00.661788 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:41:00.664103 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:41:01.180296 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 9249 2024.10.22 01:41:01.259304 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_443_91, 202410_444_444_0, 202410_445_445_0, 202410_446_446_0, 202410_447_447_0, 202410_448_448_0] 2024.10.22 01:41:01.262529 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_443_91, 202410_444_444_0, 202410_445_445_0, 202410_446_446_0, 202410_447_447_0, 202410_448_448_0] 2024.10.22 01:41:01.291676 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:01.295872 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:01.345223 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_602_602_0 (state Temporary) 2024.10.22 01:41:01.400551 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_443_91, 202410_444_444_0, 202410_445_445_0, 202410_446_446_0, 202410_447_447_0, 202410_448_448_0] 2024.10.22 01:41:01.527592 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_602_602_0 to 202410_615_615_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:01.584799 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9249 2024.10.22 01:41:01.783172 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 505 2024.10.22 01:41:02.291072 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4850 2024.10.22 01:41:02.306572 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:02.312041 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:02.327920 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 44 entries to flush up to offset 68186 2024.10.22 01:41:02.376581 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_487_487_0 (state Temporary) 2024.10.22 01:41:02.437307 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 899424 2024.10.22 01:41:02.485211 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_487_487_0 to 202410_497_497_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:02.581594 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:02.619058 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 505 2024.10.22 01:41:02.672275 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:02.740886 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_591_591_0 (state Temporary) 2024.10.22 01:41:02.754055 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:41:02.837805 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_591_591_0 to 202410_597_597_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:02.843874 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 6.19 MiB, will set to 1.69 GiB (RSS), difference: 3.01 MiB 2024.10.22 01:41:02.872983 [ 1229 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 5 parts from 202410_507_592_17 to 202410_596_596_0 2024.10.22 01:41:02.891238 [ 1229 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:02.896332 [ 1229 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:02.913484 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 68186 2024.10.22 01:41:02.975390 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_596_18} MergeTask::PrepareStage: Merging 5 parts: from 202410_507_592_17 to 202410_596_596_0 into Compact with storage Full 2024.10.22 01:41:03.076685 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_596_18} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:03.115017 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_596_18} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_592_17, total 8647 rows starting from the beginning of the part 2024.10.22 01:41:03.166804 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:03.170642 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:03.383118 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_596_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_593_593_0, total 100 rows starting from the beginning of the part 2024.10.22 01:41:03.475361 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_577_577_0 (state Temporary) 2024.10.22 01:41:03.553616 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 66846 2024.10.22 01:41:03.554325 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_596_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_594_594_0, total 80 rows starting from the beginning of the part 2024.10.22 01:41:03.617172 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_596_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_595_595_0, total 50 rows starting from the beginning of the part 2024.10.22 01:41:03.678975 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_596_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_596_596_0, total 140 rows starting from the beginning of the part 2024.10.22 01:41:03.801305 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:03.808160 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:03.878865 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_577_577_0 to 202410_584_584_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:03.940947 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 899424 2024.10.22 01:41:03.960296 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1017_1017_0 (state Temporary) 2024.10.22 01:41:03.938546 [ 1239 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_574_113 to 202410_575_580_1 2024.10.22 01:41:03.969503 [ 1239 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:04.007178 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1017_1017_0 to 202410_1030_1030_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:04.073617 [ 1239 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:04.103882 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_580_114} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_574_113 to 202410_575_580_1 into Wide with storage Full 2024.10.22 01:41:04.163406 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66846 2024.10.22 01:41:04.267306 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_580_114} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:04.272567 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_580_114} MergeTreeSequentialSource: Reading 111 marks from part 202410_1_574_113, total 899154 rows starting from the beginning of the part 2024.10.22 01:41:04.321781 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_580_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_575_580_1, total 7315 rows starting from the beginning of the part 2024.10.22 01:41:04.367475 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(f8426d6bddcfec811503fc87d85776bc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f8426d6bddcfec811503fc87d85776bc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:41:04.511746 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 106946, file segment info: File segment: [0, 33554431], key: f8426d6bddcfec811503fc87d85776bc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:41:04.514935 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:04.535627 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 7.37 MiB, will set to 1.70 GiB (RSS), difference: -4.96 MiB 2024.10.22 01:41:05.208264 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_514_103, 202410_515_520_1] 2024.10.22 01:41:05.211559 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_514_103, 202410_515_520_1] 2024.10.22 01:41:05.228251 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_514_103, 202410_515_520_1] 2024.10.22 01:41:05.672519 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(66c1740e566adcc1b4c289ffb9675021) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 66c1740e566adcc1b4c289ffb9675021, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:41:05.676076 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1035, file segment info: File segment: [0, 33554431], key: 66c1740e566adcc1b4c289ffb9675021, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:41:05.678534 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:05.723328 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_596_18} MergeTask::MergeProjectionsStage: Merge sorted 9017 rows, containing 13 columns (13 merged, 0 gathered) in 2.754983916 sec., 3272.977365723394 rows/sec., 1.42 MiB/sec. 2024.10.22 01:41:05.777748 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_596_18} MergedBlockOutputStream: filled checksums 202410_507_596_18 (state Temporary) 2024.10.22 01:41:06.019422 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_596_18} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_596_18 to 202410_507_596_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:06.056993 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_596_18} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 5 parts: [202410_507_592_17, 202410_596_596_0] -> 202410_507_596_18 2024.10.22 01:41:06.079757 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.28 MiB. 2024.10.22 01:41:06.269389 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:41:06.285337 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:41:06.451613 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:41:06.485805 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:41:06.529356 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:41:06.556637 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:41:06.676533 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:41:06.702306 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column CodeVersion 2024.10.22 01:41:06.828771 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/urb/llltzypemipddhnlafsaynqwetptw (LogSeriesLimiter: on interval from 2024-10-22 01:40:54 to 2024-10-22 01:41:06 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:41:07.045764 [ 1223 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 211 ms. 2024.10.22 01:41:07.417325 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_551_110, 202410_552_552_0, 202410_553_553_0, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0] 2024.10.22 01:41:07.422199 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_551_110, 202410_552_552_0, 202410_553_553_0, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0] 2024.10.22 01:41:07.450549 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_551_110, 202410_552_552_0, 202410_553_553_0, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0] 2024.10.22 01:41:07.660772 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 10.87 MiB, will set to 1.71 GiB (RSS), difference: 8.65 MiB 2024.10.22 01:41:08.168269 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 66893 2024.10.22 01:41:08.430890 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:08.439124 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:08.792448 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1018_1018_0 (state Temporary) 2024.10.22 01:41:08.810598 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:41:08.864322 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1018_1018_0 to 202410_1031_1031_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:08.920285 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66893 2024.10.22 01:41:09.099523 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 9261 2024.10.22 01:41:09.354834 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:09.362798 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:09.533508 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_603_603_0 (state Temporary) 2024.10.22 01:41:09.553397 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 15.65 MiB, will set to 1.72 GiB (RSS), difference: 7.64 MiB 2024.10.22 01:41:09.609725 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_603_603_0 to 202410_616_616_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:09.646252 [ 1271 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_611_122 to 202410_616_616_0 2024.10.22 01:41:09.655799 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_580_114/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:41:09.661392 [ 1271 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:09.665554 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9261 2024.10.22 01:41:09.765707 [ 1271 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:09.777634 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_611_122 to 202410_616_616_0 into Compact with storage Full 2024.10.22 01:41:09.837705 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 4888 2024.10.22 01:41:10.065337 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:10.077521 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_611_122, total 9305 rows starting from the beginning of the part 2024.10.22 01:41:10.113929 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_612_612_0, total 19 rows starting from the beginning of the part 2024.10.22 01:41:10.262760 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_613_613_0, total 8 rows starting from the beginning of the part 2024.10.22 01:41:10.313662 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_614_614_0, total 16 rows starting from the beginning of the part 2024.10.22 01:41:10.369495 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_615_615_0, total 7 rows starting from the beginning of the part 2024.10.22 01:41:10.415771 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_616_616_0, total 12 rows starting from the beginning of the part 2024.10.22 01:41:10.428689 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 108 entries to flush up to offset 68294 2024.10.22 01:41:10.473017 [ 1162 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 260 ms. 2024.10.22 01:41:10.663976 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:41:10.665418 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:41:10.727573 [ 1150 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:41:10.758229 [ 1150 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:41:10.787030 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:10.791414 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:10.861293 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:10.950865 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 900964 2024.10.22 01:41:10.941834 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_592_592_0 (state Temporary) 2024.10.22 01:41:11.106691 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_592_592_0 to 202410_598_598_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:11.163518 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 68294 2024.10.22 01:41:11.410822 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 17.52 MiB, will set to 1.73 GiB (RSS), difference: 2.84 MiB 2024.10.22 01:41:11.777442 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:11.781198 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:12.018997 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_578_578_0 (state Temporary) 2024.10.22 01:41:12.106419 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_578_578_0 to 202410_585_585_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:12.180876 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_114_22, total 4523 rows starting from the beginning of the part 2024.10.22 01:41:12.230989 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} MergeTask::MergeProjectionsStage: Merge sorted 9367 rows, containing 12 columns (12 merged, 0 gathered) in 2.458187241 sec., 3810.531534688736 rows/sec., 606.04 KiB/sec. 2024.10.22 01:41:12.260807 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 900964 2024.10.22 01:41:12.273560 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} MergedBlockOutputStream: filled checksums 202410_1_616_123 (state Temporary) 2024.10.22 01:41:12.654605 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 16.52 MiB, will set to 1.73 GiB (RSS), difference: 4.43 MiB 2024.10.22 01:41:12.629153 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_616_123 to 202410_1_616_123 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:12.658327 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_616_123} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_611_122, 202410_616_616_0] -> 202410_1_616_123 2024.10.22 01:41:12.691578 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.56 MiB. 2024.10.22 01:41:12.927573 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 66946 2024.10.22 01:41:13.100343 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:13.104032 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:13.174964 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1019_1019_0 (state Temporary) 2024.10.22 01:41:13.217158 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1019_1019_0 to 202410_1032_1032_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:13.255845 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1021_496 to 202410_1022_1027_1 2024.10.22 01:41:13.263591 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.50 MiB using storage policy from min volume index 0 2024.10.22 01:41:13.272678 [ 1194 ] {} DiskLocal: Reserved 2.50 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:41:13.294953 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1027_497} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1021_496 to 202410_1022_1027_1 into Wide with storage Full 2024.10.22 01:41:13.365282 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1027_497} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:13.383130 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66946 2024.10.22 01:41:13.478688 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1027_497} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1021_496, total 67195 rows starting from the beginning of the part 2024.10.22 01:41:13.678869 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1027_497} MergeTreeSequentialSource: Reading 2 marks from part 202410_1022_1027_1, total 326 rows starting from the beginning of the part 2024.10.22 01:41:14.566006 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_115_119_1, total 198 rows starting from the beginning of the part 2024.10.22 01:41:14.777564 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 11.87 MiB, will set to 1.73 GiB (RSS), difference: -44.58 MiB 2024.10.22 01:41:16.593297 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_120_120_0, total 40 rows starting from the beginning of the part 2024.10.22 01:41:16.675219 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 13.52 MiB, will set to 1.74 GiB (RSS), difference: 27.17 MiB 2024.10.22 01:41:17.168783 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 9272 2024.10.22 01:41:17.414526 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 18 entries to flush up to offset 66964 2024.10.22 01:41:17.529913 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:17.535765 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:41:17.591140 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_604_604_0 (state Temporary) 2024.10.22 01:41:17.624569 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 506 2024.10.22 01:41:17.681360 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_604_604_0 to 202410_617_617_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:17.811735 [ 1696 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xxv/mlplsnnfcqoezetnxfwdbcjadwubd, size 1195 2024.10.22 01:41:17.837076 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:17.846287 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:41:17.858612 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9272 2024.10.22 01:41:17.916072 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:17.926735 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:41:18.041635 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_488_488_0 (state Temporary) 2024.10.22 01:41:18.113246 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_488_488_0 to 202410_498_498_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:18.129084 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1027_497} MergeTask::MergeProjectionsStage: Merge sorted 67521 rows, containing 13 columns (13 merged, 0 gathered) in 4.837621192 sec., 13957.479786069202 rows/sec., 2.30 MiB/sec. 2024.10.22 01:41:18.158513 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1020_1020_0 (state Temporary) 2024.10.22 01:41:18.178983 [ 1156 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_493_101 to 202410_498_498_0 2024.10.22 01:41:18.218890 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1020_1020_0 to 202410_1033_1033_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:18.253680 [ 1156 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:18.259828 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:41:18.267317 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1027_497} MergedBlockOutputStream: filled checksums 202410_1_1027_497 (state Temporary) 2024.10.22 01:41:18.272754 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1028_1028_0 to 202410_1033_1033_0 2024.10.22 01:41:18.289640 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:18.303880 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:41:18.317176 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_493_101 to 202410_498_498_0 into Compact with storage Full 2024.10.22 01:41:18.316718 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 66964 2024.10.22 01:41:18.361914 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1028_1028_0 to 202410_1033_1033_0 into Compact with storage Full 2024.10.22 01:41:18.388527 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 506 2024.10.22 01:41:18.497473 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_121_121_0, total 35 rows starting from the beginning of the part 2024.10.22 01:41:18.553631 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:18.611794 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1028_1028_0, total 72 rows starting from the beginning of the part 2024.10.22 01:41:18.673241 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 196 entries to flush up to offset 68490 2024.10.22 01:41:18.771554 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1029_1029_0, total 31 rows starting from the beginning of the part 2024.10.22 01:41:18.809064 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1030_1030_0, total 53 rows starting from the beginning of the part 2024.10.22 01:41:18.868729 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1031_1031_0, total 47 rows starting from the beginning of the part 2024.10.22 01:41:18.955611 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:18.969512 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1032_1032_0, total 53 rows starting from the beginning of the part 2024.10.22 01:41:19.034194 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_493_101, total 537 rows starting from the beginning of the part 2024.10.22 01:41:19.040233 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1033_1033_0, total 18 rows starting from the beginning of the part 2024.10.22 01:41:19.205004 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(3202df41fd9cdf29e925397d2ef19251) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3202df41fd9cdf29e925397d2ef19251, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:41:19.230668 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 30284, file segment info: File segment: [0, 33554431], key: 3202df41fd9cdf29e925397d2ef19251, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:41:19.243738 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:19.275211 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:19.275831 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 902504 2024.10.22 01:41:19.279379 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.00 GiB. 2024.10.22 01:41:19.448641 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_593_593_0 (state Temporary) 2024.10.22 01:41:19.459574 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_494_494_0, total 1 rows starting from the beginning of the part 2024.10.22 01:41:19.708231 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_593_593_0 to 202410_599_599_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:19.770532 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.97 GiB, free memory in arenas 10.37 MiB, will set to 1.72 GiB (RSS), difference: 239.58 MiB 2024.10.22 01:41:19.778442 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_495_495_0, total 1 rows starting from the beginning of the part 2024.10.22 01:41:19.914288 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 68490 2024.10.22 01:41:19.973550 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_122_122_0, total 38 rows starting from the beginning of the part 2024.10.22 01:41:20.166099 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1027_497} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1027_497 to 202410_1_1027_497 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:20.206309 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1027_497} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1021_496, 202410_1022_1027_1] -> 202410_1_1027_497 2024.10.22 01:41:20.216815 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_496_496_0, total 1 rows starting from the beginning of the part 2024.10.22 01:41:20.275690 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.53 MiB. 2024.10.22 01:41:20.293467 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1028_1033_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:41:20.299946 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:20.308675 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.97 GiB. 2024.10.22 01:41:20.382338 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(cf89f1f402501bfbc1f3539e1e95d5d2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cf89f1f402501bfbc1f3539e1e95d5d2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:41:20.396318 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: cf89f1f402501bfbc1f3539e1e95d5d2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:41:20.401297 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:20.487060 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_497_497_0, total 1 rows starting from the beginning of the part 2024.10.22 01:41:20.617096 [ 1234 ] {} DatabaseCatalog: Execution took 287 ms. 2024.10.22 01:41:20.663193 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:41:20.664700 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:41:20.720272 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_580_114/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:41:20.758091 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_498_498_0, total 1 rows starting from the beginning of the part 2024.10.22 01:41:20.838377 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_579_579_0 (state Temporary) 2024.10.22 01:41:21.166367 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_579_579_0 to 202410_586_586_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:21.190647 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} MergeTask::MergeProjectionsStage: Merge sorted 274 rows, containing 13 columns (13 merged, 0 gathered) in 2.838354163 sec., 96.53481710344263 rows/sec., 16.30 KiB/sec. 2024.10.22 01:41:21.244548 [ 1207 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_581_581_0 to 202410_586_586_0 2024.10.22 01:41:21.285275 [ 1207 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:21.290208 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} MergedBlockOutputStream: filled checksums 202410_1028_1033_1 (state Temporary) 2024.10.22 01:41:21.291379 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:21.335642 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_581_581_0 to 202410_586_586_0 into Compact with storage Full 2024.10.22 01:41:21.347716 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 902504 2024.10.22 01:41:21.354765 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_7_7_0 (state Temporary) 2024.10.22 01:41:21.395334 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:21.463951 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_581_581_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:41:21.465033 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/cql/reswptjtwlshzznxfwxflwioyinhl (LogSeriesLimiter: on interval from 2024-10-22 01:41:06 to 2024-10-22 01:41:21 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:41:21.471496 [ 1677 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nit/cxccobazsvtnymxnhrzeojpgqpwxb, size 575 2024.10.22 01:41:21.477637 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_582_582_0, total 1546 rows starting from the beginning of the part 2024.10.22 01:41:21.489067 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_583_583_0, total 1534 rows starting from the beginning of the part 2024.10.22 01:41:21.527840 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_584_584_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:41:21.559701 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_585_585_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:41:21.579503 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_586_586_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:41:21.682263 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_123_123_0, total 41 rows starting from the beginning of the part 2024.10.22 01:41:21.756532 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_921_469, 202410_922_927_1] 2024.10.22 01:41:21.756530 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1028_1033_1 to 202410_1028_1033_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:21.761726 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1028_1033_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1028_1028_0, 202410_1033_1033_0] -> 202410_1028_1033_1 2024.10.22 01:41:21.762678 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_921_469, 202410_922_927_1] 2024.10.22 01:41:21.790089 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_921_469, 202410_922_927_1] 2024.10.22 01:41:21.797706 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:41:21.939956 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(5c53b297ae4cb53f23d4af85e493d145) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5c53b297ae4cb53f23d4af85e493d145, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:41:21.944957 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 642, file segment info: File segment: [0, 33554431], key: 5c53b297ae4cb53f23d4af85e493d145, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:41:21.954397 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:22.107568 [ 1712 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/cql/reswptjtwlshzznxfwxflwioyinhl, size 4 2024.10.22 01:41:22.337658 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 98 entries to flush up to offset 67062 2024.10.22 01:41:22.686965 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} MergeTask::MergeProjectionsStage: Merge sorted 9625 rows, containing 4 columns (4 merged, 0 gathered) in 1.369684637 sec., 7027.165042225702 rows/sec., 109.80 KiB/sec. 2024.10.22 01:41:22.721551 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:22.731407 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:22.758961 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} MergedBlockOutputStream: filled checksums 202410_581_586_1 (state Temporary) 2024.10.22 01:41:22.818231 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1021_1021_0 (state Temporary) 2024.10.22 01:41:22.877695 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_581_586_1 to 202410_581_586_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:22.879969 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_581_586_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_581_581_0, 202410_586_586_0] -> 202410_581_586_1 2024.10.22 01:41:22.940856 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:41:22.946691 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(2162dbf687a74f0b1568f3cd43a171ff) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2162dbf687a74f0b1568f3cd43a171ff, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:41:22.955296 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 19435, file segment info: File segment: [0, 33554431], key: 2162dbf687a74f0b1568f3cd43a171ff, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:41:22.975054 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1021_1021_0 to 202410_1034_1034_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:22.978357 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:23.111777 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} MergeTask::MergeProjectionsStage: Merge sorted 542 rows, containing 32 columns (32 merged, 0 gathered) in 4.80604319 sec., 112.77468357499302 rows/sec., 10.37 KiB/sec. 2024.10.22 01:41:23.128890 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67062 2024.10.22 01:41:23.150976 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} MergedBlockOutputStream: filled checksums 202410_1_498_102 (state Temporary) 2024.10.22 01:41:23.172155 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_928_928_0, 202410_929_929_0, 202410_930_930_0, 202410_931_931_0, 202410_932_932_0, 202410_933_933_0] 2024.10.22 01:41:23.174000 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_928_928_0, 202410_929_929_0, 202410_930_930_0, 202410_931_931_0, 202410_932_932_0, 202410_933_933_0] 2024.10.22 01:41:23.271192 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_124_124_0, total 37 rows starting from the beginning of the part 2024.10.22 01:41:23.293332 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_928_928_0, 202410_929_929_0, 202410_930_930_0, 202410_931_931_0, 202410_932_932_0, 202410_933_933_0] 2024.10.22 01:41:23.382918 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.97 GiB, free memory in arenas 12.58 MiB, will set to 1.67 GiB (RSS), difference: 111.58 MiB 2024.10.22 01:41:23.763013 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(e2a2970d43f3a71749170bb380a16fd9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e2a2970d43f3a71749170bb380a16fd9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:41:23.767287 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 847, file segment info: File segment: [0, 33554431], key: e2a2970d43f3a71749170bb380a16fd9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:41:23.768732 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:23.813269 [ 1739 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/urb/llltzypemipddhnlafsaynqwetptw, size 1256199 2024.10.22 01:41:23.964151 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_498_102 to 202410_1_498_102 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:23.979252 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_498_102} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_493_101, 202410_498_498_0] -> 202410_1_498_102 2024.10.22 01:41:24.015482 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:41:24.059209 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:41:24.176912 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column IsLink 2024.10.22 01:41:24.193143 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column IsLink 2024.10.22 01:41:24.214043 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column IsLink 2024.10.22 01:41:24.244080 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column IsLink 2024.10.22 01:41:24.366785 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column IsLink 2024.10.22 01:41:24.388771 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column IsLink 2024.10.22 01:41:24.408064 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column IsLink 2024.10.22 01:41:24.452263 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column IsLink 2024.10.22 01:41:24.678863 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(f402c0cb8bc34619dcdd603577452d54) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f402c0cb8bc34619dcdd603577452d54, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:41:24.713373 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 635, file segment info: File segment: [0, 33554431], key: f402c0cb8bc34619dcdd603577452d54, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:41:24.717066 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:24.758503 [ 1732 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/biu/gqqykazxexqodovzxjgdggxztsehi, size 19435 2024.10.22 01:41:24.802699 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:41:24.858098 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:41:24.882020 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:41:25.029751 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:41:25.059931 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:41:25.073584 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:41:25.105012 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:41:25.154181 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:41:25.174624 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column HasGCLID 2024.10.22 01:41:25.360222 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 30 entries to flush up to offset 9302 2024.10.22 01:41:25.583924 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:25.591955 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:25.672174 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_605_605_0 (state Temporary) 2024.10.22 01:41:25.763153 [ 1245 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:41:25.852489 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_605_605_0 to 202410_618_618_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:25.888133 [ 1245 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:41:25.890462 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 507 2024.10.22 01:41:25.998524 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9302 2024.10.22 01:41:26.387098 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:26.395489 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:26.477703 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:26.480584 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:26.523987 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 8.99 MiB, will set to 1.67 GiB (RSS), difference: -2.85 MiB 2024.10.22 01:41:26.566581 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_489_489_0 (state Temporary) 2024.10.22 01:41:26.750909 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_489_489_0 to 202410_499_499_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:26.962306 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 507 2024.10.22 01:41:27.152473 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 67129 2024.10.22 01:41:27.474596 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 143 entries to flush up to offset 68633 2024.10.22 01:41:27.556501 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:27.564352 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:27.647434 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:27.656440 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:27.726798 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_594_594_0 (state Temporary) 2024.10.22 01:41:27.741573 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1022_1022_0 (state Temporary) 2024.10.22 01:41:27.781095 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_594_594_0 to 202410_600_600_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:27.833630 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 68633 2024.10.22 01:41:27.961415 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1022_1022_0 to 202410_1035_1035_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:28.022126 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67129 2024.10.22 01:41:28.399174 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 10.32 MiB, will set to 1.67 GiB (RSS), difference: -6.01 MiB 2024.10.22 01:41:28.463128 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 903659 2024.10.22 01:41:29.164633 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:29.267702 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:29.579634 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_580_580_0 (state Temporary) 2024.10.22 01:41:29.643273 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 9.67 MiB, will set to 1.67 GiB (RSS), difference: -12.23 MiB 2024.10.22 01:41:29.765022 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_580_580_0 to 202410_587_587_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:29.798835 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 903659 2024.10.22 01:41:29.988060 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_122_122_0 (state Temporary) 2024.10.22 01:41:30.664383 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:41:30.664955 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:41:30.862404 [ 1153 ] {} DatabaseCatalog: Execution took 243 ms. 2024.10.22 01:41:31.539080 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vlb/afnzdpmooofkvcpmgwqaruqjxtzpy. 2024.10.22 01:41:31.549548 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 8.96 MiB, will set to 1.67 GiB (RSS), difference: 5.07 MiB 2024.10.22 01:41:31.587973 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_580_114/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:41:31.684404 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_580_114} MergeTask::MergeProjectionsStage: Merge sorted 906469 rows, containing 4 columns (4 merged, 0 gathered) in 27.590989234 sec., 32853.80572302825 rows/sec., 492.75 KiB/sec. 2024.10.22 01:41:31.764371 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_580_114} MergedBlockOutputStream: filled checksums 202410_1_580_114 (state Temporary) 2024.10.22 01:41:32.040474 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 29 entries to flush up to offset 67158 2024.10.22 01:41:32.293155 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:32.299731 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:32.358942 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1023_1023_0 (state Temporary) 2024.10.22 01:41:32.429264 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1023_1023_0 to 202410_1036_1036_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:32.583106 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67158 2024.10.22 01:41:33.444913 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 9.68 MiB, will set to 1.67 GiB (RSS), difference: 6.88 MiB 2024.10.22 01:41:33.508349 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 9308 2024.10.22 01:41:33.610727 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:33.623267 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:33.674177 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_606_606_0 (state Temporary) 2024.10.22 01:41:33.932355 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_606_606_0 to 202410_619_619_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:33.981357 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9308 2024.10.22 01:41:34.468398 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 508 2024.10.22 01:41:34.517834 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_580_114} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_580_114 to 202410_1_580_114 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:34.523127 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_580_114} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_574_113, 202410_575_580_1] -> 202410_1_580_114 2024.10.22 01:41:34.554021 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:41:34.924443 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:34.928773 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:35.046094 [ 1275 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 211 ms. 2024.10.22 01:41:35.186094 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_490_490_0 (state Temporary) 2024.10.22 01:41:35.279621 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_490_490_0 to 202410_500_500_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:35.364469 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 508 2024.10.22 01:41:35.468549 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 61 entries to flush up to offset 68694 2024.10.22 01:41:35.575394 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 7.88 MiB, will set to 1.67 GiB (RSS), difference: 2.83 MiB 2024.10.22 01:41:35.576659 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:35.580194 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:35.640309 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_595_595_0 (state Temporary) 2024.10.22 01:41:35.689234 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_595_595_0 to 202410_601_601_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:35.739711 [ 1165 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_596_18 to 202410_601_601_0 2024.10.22 01:41:35.771950 [ 1165 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:35.779030 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:35.852587 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 68694 2024.10.22 01:41:35.804732 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_596_18 to 202410_601_601_0 into Compact with storage Full 2024.10.22 01:41:35.940398 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_537_6, 202410_538_538_0, 202410_539_539_0, 202410_540_540_0, 202410_541_541_0, 202410_542_542_0] 2024.10.22 01:41:35.969030 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_537_6, 202410_538_538_0, 202410_539_539_0, 202410_540_540_0, 202410_541_541_0, 202410_542_542_0] 2024.10.22 01:41:36.004348 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_537_6, 202410_538_538_0, 202410_539_539_0, 202410_540_540_0, 202410_541_541_0, 202410_542_542_0] 2024.10.22 01:41:36.189716 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:36.232885 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_596_18, total 9017 rows starting from the beginning of the part 2024.10.22 01:41:36.419867 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_597_597_0, total 44 rows starting from the beginning of the part 2024.10.22 01:41:36.587079 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 67198 2024.10.22 01:41:36.726772 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_598_598_0, total 108 rows starting from the beginning of the part 2024.10.22 01:41:36.791451 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:36.794697 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:41:36.804950 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 905584 2024.10.22 01:41:36.880122 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_599_599_0, total 196 rows starting from the beginning of the part 2024.10.22 01:41:36.971095 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_600_600_0, total 143 rows starting from the beginning of the part 2024.10.22 01:41:36.975298 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1024_1024_0 (state Temporary) 2024.10.22 01:41:37.019435 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1024_1024_0 to 202410_1037_1037_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:37.076512 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_601_601_0, total 61 rows starting from the beginning of the part 2024.10.22 01:41:37.134120 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67198 2024.10.22 01:41:37.364677 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 5.23 MiB, will set to 1.67 GiB (RSS), difference: -6.46 MiB 2024.10.22 01:41:37.460634 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_124_23/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:41:37.701806 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:37.707854 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:41:37.863022 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_581_581_0 (state Temporary) 2024.10.22 01:41:37.927984 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_581_581_0 to 202410_588_588_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:37.967963 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 905584 2024.10.22 01:41:38.443194 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 6.19 MiB, will set to 1.68 GiB (RSS), difference: -8.01 MiB 2024.10.22 01:41:39.089974 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} MergeTask::MergeProjectionsStage: Merge sorted 9569 rows, containing 13 columns (13 merged, 0 gathered) in 3.297832792 sec., 2901.602538252643 rows/sec., 1.26 MiB/sec. 2024.10.22 01:41:39.096094 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_122_122_0 to 202410_125_125_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:39.237777 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} MergedBlockOutputStream: filled checksums 202410_507_601_19 (state Temporary) 2024.10.22 01:41:39.677224 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_601_19 to 202410_507_601_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:39.679691 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_601_19} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_596_18, 202410_601_601_0] -> 202410_507_601_19 2024.10.22 01:41:39.709743 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.75 MiB. 2024.10.22 01:41:40.407982 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 6.27 MiB, will set to 1.68 GiB (RSS), difference: 15.92 MiB 2024.10.22 01:41:40.664062 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:41:40.665242 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:41:40.893480 [ 1191 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:41:40.917508 [ 1191 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:41:41.148095 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 67228 2024.10.22 01:41:41.220429 [ 1189 ] {} DatabaseCatalog: Execution took 360 ms. 2024.10.22 01:41:41.327567 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:41.330999 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:41.381289 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1025_1025_0 (state Temporary) 2024.10.22 01:41:41.483906 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 9324 2024.10.22 01:41:41.759559 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1025_1025_0 to 202410_1038_1038_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:41.761786 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:41.766887 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:41.779674 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1027_497 to 202410_1028_1033_1 2024.10.22 01:41:41.787018 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.51 MiB using storage policy from min volume index 0 2024.10.22 01:41:41.797423 [ 1185 ] {} DiskLocal: Reserved 2.51 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:41:41.808352 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_607_607_0 (state Temporary) 2024.10.22 01:41:41.816304 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67228 2024.10.22 01:41:41.934975 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1033_498} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1027_497 to 202410_1028_1033_1 into Wide with storage Full 2024.10.22 01:41:41.961643 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_607_607_0 to 202410_620_620_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:41.974690 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(edf018e753e77ac5144e81cbc9a16bec) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: edf018e753e77ac5144e81cbc9a16bec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:41:41.980446 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 423, file segment info: File segment: [0, 33554431], key: edf018e753e77ac5144e81cbc9a16bec, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:41:41.985262 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:42.016053 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1033_498} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:42.032757 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9324 2024.10.22 01:41:42.060045 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1033_498} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1027_497, total 67521 rows starting from the beginning of the part 2024.10.22 01:41:42.273213 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1033_498} MergeTreeSequentialSource: Reading 2 marks from part 202410_1028_1033_1, total 274 rows starting from the beginning of the part 2024.10.22 01:41:42.688763 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 7.59 MiB, will set to 1.68 GiB (RSS), difference: -9.30 MiB 2024.10.22 01:41:42.766211 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4888 2024.10.22 01:41:42.866678 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 509 2024.10.22 01:41:42.948647 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(2c2563c21f8410f0c9c64c1fa4bf14d3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2c2563c21f8410f0c9c64c1fa4bf14d3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:41:42.954698 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 639, file segment info: File segment: [0, 33554431], key: 2c2563c21f8410f0c9c64c1fa4bf14d3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:41:42.959640 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:43.037048 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:43.041337 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:41:43.224985 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_491_491_0 (state Temporary) 2024.10.22 01:41:43.365500 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 171 entries to flush up to offset 68865 2024.10.22 01:41:43.407520 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_491_491_0 to 202410_501_501_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:43.652819 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 509 2024.10.22 01:41:43.872299 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:43.877252 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:41:43.945542 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(df38d8eb26dace05c1476e15b1103dc3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: df38d8eb26dace05c1476e15b1103dc3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:41:43.955188 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 82478, file segment info: File segment: [0, 33554431], key: df38d8eb26dace05c1476e15b1103dc3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:41:43.958200 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:44.162633 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_596_596_0 (state Temporary) 2024.10.22 01:41:44.213215 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_596_596_0 to 202410_602_602_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:44.277490 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 68865 2024.10.22 01:41:44.654130 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 6.32 MiB, will set to 1.69 GiB (RSS), difference: -31.15 MiB 2024.10.22 01:41:44.767634 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(b2e04141c1cb063520148eb5c9cf4f0f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b2e04141c1cb063520148eb5c9cf4f0f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:41:44.770507 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 716, file segment info: File segment: [0, 33554431], key: b2e04141c1cb063520148eb5c9cf4f0f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:41:44.775455 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:45.052318 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1550 entries to flush up to offset 907134 2024.10.22 01:41:45.795192 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:41:45.829817 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 67277 2024.10.22 01:41:45.839543 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:41:45.872729 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:41:45.913228 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:41:46.018414 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a346e1ff991ba59852720a0f2e03d763) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a346e1ff991ba59852720a0f2e03d763, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:41:46.057247 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:41:46.080708 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:41:46.111813 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 471, file segment info: File segment: [0, 33554431], key: a346e1ff991ba59852720a0f2e03d763, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:41:46.115288 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:46.117263 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:41:46.152119 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:46.161616 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:41:46.167975 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column IsDownload 2024.10.22 01:41:46.296079 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:46.299819 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:41:46.321150 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xcp/lbvulsefsskcrndudsokexwnmbrnb (LogSeriesLimiter: on interval from 2024-10-22 01:41:21 to 2024-10-22 01:41:46 accepted series 1 / 16 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:41:46.331817 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1026_1026_0 (state Temporary) 2024.10.22 01:41:46.403916 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1026_1026_0 to 202410_1039_1039_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:46.506816 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67277 2024.10.22 01:41:46.516766 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 6.33 MiB, will set to 1.69 GiB (RSS), difference: -1.12 MiB 2024.10.22 01:41:46.560722 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_582_582_0 (state Temporary) 2024.10.22 01:41:46.624660 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_582_582_0 to 202410_589_589_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:46.660498 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1034_1034_0 to 202410_1039_1039_0 2024.10.22 01:41:46.666071 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1033_498} MergeTask::MergeProjectionsStage: Merge sorted 67795 rows, containing 13 columns (13 merged, 0 gathered) in 4.838413338 sec., 14011.824799578544 rows/sec., 2.31 MiB/sec. 2024.10.22 01:41:46.668830 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:46.671538 [ 1209 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:41:46.672683 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 907134 2024.10.22 01:41:46.682127 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1034_1034_0 to 202410_1039_1039_0 into Compact with storage Full 2024.10.22 01:41:46.739707 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:46.752755 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1033_498} MergedBlockOutputStream: filled checksums 202410_1_1033_498 (state Temporary) 2024.10.22 01:41:46.753480 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1034_1034_0, total 98 rows starting from the beginning of the part 2024.10.22 01:41:46.771385 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1035_1035_0, total 67 rows starting from the beginning of the part 2024.10.22 01:41:46.673752 [ 1209 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 249 ms. 2024.10.22 01:41:46.785527 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1036_1036_0, total 29 rows starting from the beginning of the part 2024.10.22 01:41:46.816727 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1037_1037_0, total 40 rows starting from the beginning of the part 2024.10.22 01:41:46.855034 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1038_1038_0, total 30 rows starting from the beginning of the part 2024.10.22 01:41:46.875924 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1039_1039_0, total 49 rows starting from the beginning of the part 2024.10.22 01:41:46.984483 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(8cece9b16c216b34f65fcca240bb5ff7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8cece9b16c216b34f65fcca240bb5ff7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:41:46.985339 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 8576, file segment info: File segment: [0, 33554431], key: 8cece9b16c216b34f65fcca240bb5ff7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:41:46.986385 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:47.281905 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1033_498} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1033_498 to 202410_1_1033_498 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:47.304942 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1033_498} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1027_497, 202410_1028_1033_1] -> 202410_1_1033_498 2024.10.22 01:41:47.334224 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.50 MiB. 2024.10.22 01:41:47.777118 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(64ac586e033c77eaae46c70872936e0c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 64ac586e033c77eaae46c70872936e0c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:41:47.779273 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 590, file segment info: File segment: [0, 33554431], key: 64ac586e033c77eaae46c70872936e0c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 01:41:47.781684 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:47.787346 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_556_111, 202410_557_557_0, 202410_558_558_0, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0] 2024.10.22 01:41:47.789454 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_556_111, 202410_557_557_0, 202410_558_558_0, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0] 2024.10.22 01:41:47.817400 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_556_111, 202410_557_557_0, 202410_558_558_0, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0] 2024.10.22 01:41:48.082848 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_448_92, 202410_449_449_0, 202410_450_450_0, 202410_451_451_0, 202410_452_452_0, 202410_453_453_0] 2024.10.22 01:41:48.084706 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_448_92, 202410_449_449_0, 202410_450_450_0, 202410_451_451_0, 202410_452_452_0, 202410_453_453_0] 2024.10.22 01:41:48.115006 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_448_92, 202410_449_449_0, 202410_450_450_0, 202410_451_451_0, 202410_452_452_0, 202410_453_453_0] 2024.10.22 01:41:48.211823 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} MergeTask::MergeProjectionsStage: Merge sorted 313 rows, containing 13 columns (13 merged, 0 gathered) in 1.536048978 sec., 203.76954412452335 rows/sec., 35.78 KiB/sec. 2024.10.22 01:41:48.278896 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} MergedBlockOutputStream: filled checksums 202410_1034_1039_1 (state Temporary) 2024.10.22 01:41:48.360447 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1034_1039_1 to 202410_1034_1039_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:48.364969 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1034_1039_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1034_1034_0, 202410_1039_1039_0] -> 202410_1034_1039_1 2024.10.22 01:41:48.388246 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:41:48.430166 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_527_527_0, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0, 202410_532_532_0] 2024.10.22 01:41:48.437377 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_527_527_0, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0, 202410_532_532_0] 2024.10.22 01:41:48.575361 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_527_527_0, 202410_528_528_0, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0, 202410_532_532_0] 2024.10.22 01:41:48.580476 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 5.83 MiB, will set to 1.68 GiB (RSS), difference: 38.03 MiB 2024.10.22 01:41:48.658955 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:41:48.772001 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:41:48.782849 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:41:48.813645 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:41:48.832075 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:41:48.865511 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:41:48.889247 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:41:48.927738 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:41:48.965050 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RefererHash 2024.10.22 01:41:49.291409 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_927_470, 202410_928_933_1] 2024.10.22 01:41:49.308025 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_927_470, 202410_928_933_1] 2024.10.22 01:41:49.327389 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_927_470, 202410_928_933_1] 2024.10.22 01:41:49.536642 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 9340 2024.10.22 01:41:49.772897 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:49.776811 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:49.822679 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_608_608_0 (state Temporary) 2024.10.22 01:41:49.867844 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_608_608_0 to 202410_621_621_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:49.889324 [ 1236 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_616_123 to 202410_621_621_0 2024.10.22 01:41:49.904928 [ 1236 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:49.909502 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9340 2024.10.22 01:41:49.927481 [ 1236 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:49.968969 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_616_123 to 202410_621_621_0 into Compact with storage Full 2024.10.22 01:41:50.104748 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:50.115918 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_616_123, total 9367 rows starting from the beginning of the part 2024.10.22 01:41:50.164689 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_617_617_0, total 11 rows starting from the beginning of the part 2024.10.22 01:41:50.280328 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 4929 2024.10.22 01:41:50.302563 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_618_618_0, total 30 rows starting from the beginning of the part 2024.10.22 01:41:50.302444 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_124_23/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:41:50.356810 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_619_619_0, total 6 rows starting from the beginning of the part 2024.10.22 01:41:50.403623 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_620_620_0, total 16 rows starting from the beginning of the part 2024.10.22 01:41:50.450876 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_621_621_0, total 16 rows starting from the beginning of the part 2024.10.22 01:41:50.453391 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 6.15 MiB, will set to 1.68 GiB (RSS), difference: -16.92 MiB 2024.10.22 01:41:50.534619 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 67370 2024.10.22 01:41:50.664479 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:41:50.665727 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:41:50.887934 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:50.894779 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:41:51.155515 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 510 2024.10.22 01:41:51.195523 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1027_1027_0 (state Temporary) 2024.10.22 01:41:51.365211 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1027_1027_0 to 202410_1040_1040_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:51.387720 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:51.392649 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:41:51.440536 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67370 2024.10.22 01:41:51.666816 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_492_492_0 (state Temporary) 2024.10.22 01:41:51.683382 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} MergeTask::MergeProjectionsStage: Merge sorted 9446 rows, containing 12 columns (12 merged, 0 gathered) in 1.720159274 sec., 5491.351959539533 rows/sec., 873.54 KiB/sec. 2024.10.22 01:41:51.721726 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} MergedBlockOutputStream: filled checksums 202410_1_621_124 (state Temporary) 2024.10.22 01:41:51.821142 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 144 entries to flush up to offset 69009 2024.10.22 01:41:51.918652 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_492_492_0 to 202410_502_502_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:52.032057 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_621_124 to 202410_1_621_124 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:52.037645 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_621_124} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_616_123, 202410_621_621_0] -> 202410_1_621_124 2024.10.22 01:41:52.116757 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 510 2024.10.22 01:41:52.171504 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.59 MiB. 2024.10.22 01:41:52.180267 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:52.182543 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:41:52.240341 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_597_597_0 (state Temporary) 2024.10.22 01:41:52.483684 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_597_597_0 to 202410_603_603_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:52.643853 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 69009 2024.10.22 01:41:52.723734 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 4.77 MiB, will set to 1.69 GiB (RSS), difference: 5.21 MiB 2024.10.22 01:41:53.375896 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(1c101d4921c05ef4cdaae5d48dbdc76e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1c101d4921c05ef4cdaae5d48dbdc76e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:41:53.389166 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 1c101d4921c05ef4cdaae5d48dbdc76e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:41:53.395588 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:41:53.677602 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1659 entries to flush up to offset 908793 2024.10.22 01:41:54.422516 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_933_471, 202410_934_934_0] 2024.10.22 01:41:54.428321 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_933_471, 202410_934_934_0] 2024.10.22 01:41:54.439721 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_933_471, 202410_934_934_0] 2024.10.22 01:41:54.545936 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:41:54.638645 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:54.644554 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:54.856573 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_583_583_0 (state Temporary) 2024.10.22 01:41:54.913052 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_583_583_0 to 202410_590_590_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:54.943972 [ 1253 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_580_114 to 202410_581_586_1 2024.10.22 01:41:54.954641 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 908793 2024.10.22 01:41:54.956229 [ 1253 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:54.960292 [ 1253 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:54.993248 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_586_115} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_580_114 to 202410_581_586_1 into Wide with storage Full 2024.10.22 01:41:55.039612 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_586_115} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:41:55.051603 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_586_115} MergeTreeSequentialSource: Reading 112 marks from part 202410_1_580_114, total 906469 rows starting from the beginning of the part 2024.10.22 01:41:55.107116 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_586_115} MergeTreeSequentialSource: Reading 3 marks from part 202410_581_586_1, total 9625 rows starting from the beginning of the part 2024.10.22 01:41:55.464228 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 67416 2024.10.22 01:41:55.471893 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 7.14 MiB, will set to 1.69 GiB (RSS), difference: -9.33 MiB 2024.10.22 01:41:55.729381 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:55.735708 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:55.785425 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1028_1028_0 (state Temporary) 2024.10.22 01:41:55.842073 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1028_1028_0 to 202410_1041_1041_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:55.925648 [ 1181 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:41:55.965683 [ 1181 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:41:55.981816 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67416 2024.10.22 01:41:56.312746 [ 1171 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:41:57.356767 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 11.17 MiB, will set to 1.70 GiB (RSS), difference: 53.73 MiB 2024.10.22 01:41:57.412039 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 9353 2024.10.22 01:41:57.556165 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:41:57.564616 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:41:57.658910 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vlb/afnzdpmooofkvcpmgwqaruqjxtzpy. 2024.10.22 01:41:57.692094 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_609_609_0 (state Temporary) 2024.10.22 01:41:57.860371 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_609_609_0 to 202410_622_622_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:41:57.911299 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9353 2024.10.22 01:41:58.017532 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_520_104, 202410_521_526_1] 2024.10.22 01:41:58.026270 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_520_104, 202410_521_526_1] 2024.10.22 01:41:58.054248 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_520_104, 202410_521_526_1] 2024.10.22 01:41:58.090826 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_935_935_0, 202410_936_936_0, 202410_937_937_0, 202410_938_938_0, 202410_939_939_0, 202410_940_940_0] 2024.10.22 01:41:58.096129 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_935_935_0, 202410_936_936_0, 202410_937_937_0, 202410_938_938_0, 202410_939_939_0, 202410_940_940_0] 2024.10.22 01:41:58.236292 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_935_935_0, 202410_936_936_0, 202410_937_937_0, 202410_938_938_0, 202410_939_939_0, 202410_940_940_0] 2024.10.22 01:41:58.617488 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 13.29 MiB, will set to 1.70 GiB (RSS), difference: 46.76 MiB 2024.10.22 01:41:59.988334 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 25 entries to flush up to offset 67441 2024.10.22 01:42:00.165654 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_586_115/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:42:00.269375 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 81 entries to flush up to offset 69090 2024.10.22 01:42:00.467318 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.97 GiB, free memory in arenas 14.58 MiB, will set to 1.69 GiB (RSS), difference: 90.40 MiB 2024.10.22 01:42:00.540555 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:00.551157 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:42:00.551412 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:00.556285 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:42:00.631227 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1029_1029_0 (state Temporary) 2024.10.22 01:42:00.637436 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_598_598_0 (state Temporary) 2024.10.22 01:42:00.665573 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:42:00.668297 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:42:00.702304 [ 1211 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 248 ms. 2024.10.22 01:42:00.703004 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_598_598_0 to 202410_604_604_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:00.852523 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1029_1029_0 to 202410_1042_1042_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:00.863767 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 69090 2024.10.22 01:42:00.915766 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67441 2024.10.22 01:42:01.961085 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1796 entries to flush up to offset 910589 2024.10.22 01:42:02.591150 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 14.71 MiB, will set to 1.69 GiB (RSS), difference: 34.00 MiB 2024.10.22 01:42:02.640229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2ff94ae3c91dfbdf85c62a2a51ef42c4) : [0, 198]: Setting download as failed: File segment: [0, 198], key: 2ff94ae3c91dfbdf85c62a2a51ef42c4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:02.643940 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2ff94ae3c91dfbdf85c62a2a51ef42c4): No space left in cache to reserve 199 bytes, will continue without cache download 2024.10.22 01:42:02.696695 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:02.708128 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:42:02.862055 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_584_584_0 (state Temporary) 2024.10.22 01:42:03.068808 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_584_584_0 to 202410_591_591_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:03.170719 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 910589 2024.10.22 01:42:03.817730 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(289764df6017be7cdf99541eef76d19b) : [0, 2106194]: Setting download as failed: File segment: [0, 2106194], key: 289764df6017be7cdf99541eef76d19b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:03.820462 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(289764df6017be7cdf99541eef76d19b): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:42:03.990776 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 289764df6017be7cdf99541eef76d19b, offset: 0, size: 0 2024.10.22 01:42:03.993192 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 289764df6017be7cdf99541eef76d19b for removal 2024.10.22 01:42:03.996250 [ 1476 ] {} CacheMetadata: Key 289764df6017be7cdf99541eef76d19b is removed from metadata 2024.10.22 01:42:04.003343 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2ff94ae3c91dfbdf85c62a2a51ef42c4, offset: 0, size: 0 2024.10.22 01:42:04.005134 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2ff94ae3c91dfbdf85c62a2a51ef42c4 for removal 2024.10.22 01:42:04.009280 [ 1476 ] {} CacheMetadata: Key 2ff94ae3c91dfbdf85c62a2a51ef42c4 is removed from metadata 2024.10.22 01:42:04.367010 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 13.93 MiB, will set to 1.69 GiB (RSS), difference: 3.59 MiB 2024.10.22 01:42:04.665179 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(6f074b0cc5235babe2bd3caa2361be6f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6f074b0cc5235babe2bd3caa2361be6f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:42:04.772319 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 146, file segment info: File segment: [0, 33554431], key: 6f074b0cc5235babe2bd3caa2361be6f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:42:04.780931 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:04.984775 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 67479 2024.10.22 01:42:05.120904 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:05.124371 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:42:05.175105 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1030_1030_0 (state Temporary) 2024.10.22 01:42:05.248027 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1030_1030_0 to 202410_1043_1043_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:05.290510 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67479 2024.10.22 01:42:05.415432 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 9358 2024.10.22 01:42:05.737626 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:05.744421 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:42:05.877050 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_610_610_0 (state Temporary) 2024.10.22 01:42:05.946739 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_610_610_0 to 202410_623_623_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:06.001375 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9358 2024.10.22 01:42:06.014760 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b71e5029fdfa8d087760fe05407efa1c) : [0, 139]: Setting download as failed: File segment: [0, 139], key: b71e5029fdfa8d087760fe05407efa1c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:06.015820 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b71e5029fdfa8d087760fe05407efa1c): No space left in cache to reserve 140 bytes, will continue without cache download 2024.10.22 01:42:06.345307 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_124_23/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:42:06.992073 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(134b43eea404d205a36546bfb2d6b310) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 134b43eea404d205a36546bfb2d6b310, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:42:06.994358 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 520, file segment info: File segment: [0, 33554431], key: 134b43eea404d205a36546bfb2d6b310, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:42:06.999906 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:07.120803 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 511 2024.10.22 01:42:07.541505 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:07.548809 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:42:07.604139 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_493_493_0 (state Temporary) 2024.10.22 01:42:07.846972 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0d6e9210f13958dc0d6b583ce4f0800e) : [0, 14273]: Setting download as failed: File segment: [0, 14273], key: 0d6e9210f13958dc0d6b583ce4f0800e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:07.866598 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_493_493_0 to 202410_503_503_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:07.953082 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0d6e9210f13958dc0d6b583ce4f0800e): No space left in cache to reserve 14274 bytes, will continue without cache download 2024.10.22 01:42:07.968949 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0d6e9210f13958dc0d6b583ce4f0800e, offset: 0, size: 0 2024.10.22 01:42:07.970065 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0d6e9210f13958dc0d6b583ce4f0800e for removal 2024.10.22 01:42:07.971814 [ 1476 ] {} CacheMetadata: Key 0d6e9210f13958dc0d6b583ce4f0800e is removed from metadata 2024.10.22 01:42:07.972783 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b71e5029fdfa8d087760fe05407efa1c, offset: 0, size: 0 2024.10.22 01:42:07.974671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b71e5029fdfa8d087760fe05407efa1c for removal 2024.10.22 01:42:07.977089 [ 1476 ] {} CacheMetadata: Key b71e5029fdfa8d087760fe05407efa1c is removed from metadata 2024.10.22 01:42:07.910842 [ 1229 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_498_102 to 202410_503_503_0 2024.10.22 01:42:08.036353 [ 1229 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:08.042459 [ 1229 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:42:08.151767 [ 1229 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 269 ms. 2024.10.22 01:42:08.171275 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:08.173313 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.95 GiB. 2024.10.22 01:42:08.173642 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 511 2024.10.22 01:42:08.280979 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_498_102 to 202410_503_503_0 into Compact with storage Full 2024.10.22 01:42:08.376185 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 57 entries to flush up to offset 69147 2024.10.22 01:42:08.425147 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 10.68 MiB, will set to 1.69 GiB (RSS), difference: -4.76 MiB 2024.10.22 01:42:08.542595 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:08.595058 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_498_102, total 542 rows starting from the beginning of the part 2024.10.22 01:42:08.641205 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(7431a10858fdcb4624492718a46b8ed8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7431a10858fdcb4624492718a46b8ed8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:42:08.646874 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 26944, file segment info: File segment: [0, 33554431], key: 7431a10858fdcb4624492718a46b8ed8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:42:08.652620 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:08.695324 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:08.699516 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:42:08.749150 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_599_599_0 (state Temporary) 2024.10.22 01:42:08.788515 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_599_599_0 to 202410_605_605_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:08.940935 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 69147 2024.10.22 01:42:09.005196 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_499_499_0, total 1 rows starting from the beginning of the part 2024.10.22 01:42:09.231241 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_500_500_0, total 1 rows starting from the beginning of the part 2024.10.22 01:42:09.359431 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 67532 2024.10.22 01:42:09.484361 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:09.495936 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:42:09.555978 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_501_501_0, total 1 rows starting from the beginning of the part 2024.10.22 01:42:09.582658 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3838adc51d1dc7f3bbfc5d4966c81989) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 3838adc51d1dc7f3bbfc5d4966c81989, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:09.584678 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3838adc51d1dc7f3bbfc5d4966c81989): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:42:09.665682 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1031_1031_0 (state Temporary) 2024.10.22 01:42:09.813396 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1031_1031_0 to 202410_1044_1044_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:09.867712 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1033_498 to 202410_1034_1039_1 2024.10.22 01:42:09.872411 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.52 MiB using storage policy from min volume index 0 2024.10.22 01:42:09.877154 [ 1267 ] {} DiskLocal: Reserved 2.52 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:42:09.925508 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67532 2024.10.22 01:42:09.933233 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1039_499} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1033_498 to 202410_1034_1039_1 into Wide with storage Full 2024.10.22 01:42:09.995500 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_502_502_0, total 1 rows starting from the beginning of the part 2024.10.22 01:42:10.068043 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1039_499} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:10.092074 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1039_499} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1033_498, total 67795 rows starting from the beginning of the part 2024.10.22 01:42:10.174975 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 911359 2024.10.22 01:42:10.366637 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_503_503_0, total 1 rows starting from the beginning of the part 2024.10.22 01:42:10.478534 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_586_115/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:42:10.666280 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:42:10.668930 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:42:10.696505 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1039_499} MergeTreeSequentialSource: Reading 2 marks from part 202410_1034_1039_1, total 313 rows starting from the beginning of the part 2024.10.22 01:42:10.778074 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a634f33db60fd5ffadc0bb98e78bccc3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a634f33db60fd5ffadc0bb98e78bccc3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:42:10.780933 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 598, file segment info: File segment: [0, 33554431], key: a634f33db60fd5ffadc0bb98e78bccc3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:42:10.783400 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:10.826954 [ 1710 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xcp/lbvulsefsskcrndudsokexwnmbrnb, size 26944 2024.10.22 01:42:10.973019 [ 1256 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:42:11.025115 [ 1256 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:42:11.499141 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:42:11.514137 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:42:11.623155 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f0ae771780c988291e46988ea093d0cc) : [0, 2343]: Setting download as failed: File segment: [0, 2343], key: f0ae771780c988291e46988ea093d0cc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:11.627418 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f0ae771780c988291e46988ea093d0cc): No space left in cache to reserve 2344 bytes, will continue without cache download 2024.10.22 01:42:11.652070 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:42:11.666620 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f0ae771780c988291e46988ea093d0cc, offset: 0, size: 0 2024.10.22 01:42:11.668024 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f0ae771780c988291e46988ea093d0cc for removal 2024.10.22 01:42:11.671780 [ 1476 ] {} CacheMetadata: Key f0ae771780c988291e46988ea093d0cc is removed from metadata 2024.10.22 01:42:11.674652 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3838adc51d1dc7f3bbfc5d4966c81989, offset: 0, size: 0 2024.10.22 01:42:11.678079 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3838adc51d1dc7f3bbfc5d4966c81989 for removal 2024.10.22 01:42:11.682365 [ 1476 ] {} CacheMetadata: Key 3838adc51d1dc7f3bbfc5d4966c81989 is removed from metadata 2024.10.22 01:42:11.704506 [ 1182 ] {} DatabaseCatalog: Execution took 230 ms. 2024.10.22 01:42:11.724490 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:11.731699 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:42:11.767067 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:42:11.782308 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:42:11.810799 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:42:11.823327 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:42:11.846543 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 01:42:11.971484 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_123_123_0 (state Temporary) 2024.10.22 01:42:12.016875 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ttg/oubsrsvzkrzponvtpfzqlpyercjbm (LogSeriesLimiter: on interval from 2024-10-22 01:41:46 to 2024-10-22 01:42:12 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:42:12.058182 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_585_585_0 (state Temporary) 2024.10.22 01:42:12.111033 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_585_585_0 to 202410_592_592_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:12.157098 [ 1242 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_587_587_0 to 202410_592_592_0 2024.10.22 01:42:12.170515 [ 1242 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:12.173279 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:42:12.238315 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_587_587_0 to 202410_592_592_0 into Compact with storage Full 2024.10.22 01:42:12.266171 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 911359 2024.10.22 01:42:12.289021 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:12.296619 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_587_587_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:42:12.379983 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(5ad71f4b1be76b8c8e01f5a063d91f21) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5ad71f4b1be76b8c8e01f5a063d91f21, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:42:12.383126 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1165, file segment info: File segment: [0, 33554431], key: 5ad71f4b1be76b8c8e01f5a063d91f21, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 01:42:12.385801 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:12.451498 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_588_588_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:42:12.517584 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 10.35 MiB, will set to 1.70 GiB (RSS), difference: -56.10 MiB 2024.10.22 01:42:12.656138 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_589_589_0, total 1550 rows starting from the beginning of the part 2024.10.22 01:42:12.717644 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_590_590_0, total 1659 rows starting from the beginning of the part 2024.10.22 01:42:12.854018 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_591_591_0, total 1796 rows starting from the beginning of the part 2024.10.22 01:42:12.897054 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_592_592_0, total 770 rows starting from the beginning of the part 2024.10.22 01:42:13.021570 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column URLHash 2024.10.22 01:42:13.031354 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column URLHash 2024.10.22 01:42:13.061018 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column URLHash 2024.10.22 01:42:13.070978 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column URLHash 2024.10.22 01:42:13.087129 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(50a21a6cf704458c8cd70ea640ffaa60) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 50a21a6cf704458c8cd70ea640ffaa60, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:13.101131 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(50a21a6cf704458c8cd70ea640ffaa60): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:42:13.253556 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column URLHash 2024.10.22 01:42:13.272395 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column URLHash 2024.10.22 01:42:13.290121 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column URLHash 2024.10.22 01:42:13.317675 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column URLHash 2024.10.22 01:42:13.186997 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} MergeTask::MergeProjectionsStage: Merge sorted 547 rows, containing 32 columns (32 merged, 0 gathered) in 4.913468374 sec., 111.32665530005099 rows/sec., 10.23 KiB/sec. 2024.10.22 01:42:13.373545 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} MergedBlockOutputStream: filled checksums 202410_1_503_103 (state Temporary) 2024.10.22 01:42:13.474655 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column URLHash 2024.10.22 01:42:13.504971 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 9363 2024.10.22 01:42:13.893608 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:13.902304 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:13.941348 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 67615 2024.10.22 01:42:13.944039 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_611_611_0 (state Temporary) 2024.10.22 01:42:14.013747 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_611_611_0 to 202410_624_624_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:14.082195 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9363 2024.10.22 01:42:14.082334 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} MergeTask::MergeProjectionsStage: Merge sorted 8855 rows, containing 4 columns (4 merged, 0 gathered) in 1.853982075 sec., 4776.205832518634 rows/sec., 74.63 KiB/sec. 2024.10.22 01:42:14.237484 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} MergedBlockOutputStream: filled checksums 202410_587_592_1 (state Temporary) 2024.10.22 01:42:14.366062 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7c82d4080346ebdc20c05607f69b43c5) : [0, 948077]: Setting download as failed: File segment: [0, 948077], key: 7c82d4080346ebdc20c05607f69b43c5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:14.367507 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7c82d4080346ebdc20c05607f69b43c5): No space left in cache to reserve 948078 bytes, will continue without cache download 2024.10.22 01:42:14.377036 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_1039_499/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:42:14.385084 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7c82d4080346ebdc20c05607f69b43c5, offset: 0, size: 0 2024.10.22 01:42:14.385803 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7c82d4080346ebdc20c05607f69b43c5 for removal 2024.10.22 01:42:14.387450 [ 1476 ] {} CacheMetadata: Key 7c82d4080346ebdc20c05607f69b43c5 is removed from metadata 2024.10.22 01:42:14.389259 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 50a21a6cf704458c8cd70ea640ffaa60, offset: 0, size: 0 2024.10.22 01:42:14.390398 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 50a21a6cf704458c8cd70ea640ffaa60 for removal 2024.10.22 01:42:14.393370 [ 1476 ] {} CacheMetadata: Key 50a21a6cf704458c8cd70ea640ffaa60 is removed from metadata 2024.10.22 01:42:14.404983 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_587_592_1 to 202410_587_592_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:14.411423 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_587_592_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_587_587_0, 202410_592_592_0] -> 202410_587_592_1 2024.10.22 01:42:14.452211 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_503_103 to 202410_1_503_103 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:14.456691 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_503_103} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_498_102, 202410_503_503_0] -> 202410_1_503_103 2024.10.22 01:42:14.556647 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:42:14.582793 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:14.588735 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:14.663353 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:42:14.667934 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1032_1032_0 (state Temporary) 2024.10.22 01:42:14.725824 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1032_1032_0 to 202410_1045_1045_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:14.757351 [ 1219 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1040_1040_0 to 202410_1045_1045_0 2024.10.22 01:42:14.768455 [ 1219 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:14.770881 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:14.775745 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67615 2024.10.22 01:42:14.809484 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1040_1040_0 to 202410_1045_1045_0 into Compact with storage Full 2024.10.22 01:42:14.863400 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:14.882368 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1040_1040_0, total 93 rows starting from the beginning of the part 2024.10.22 01:42:14.952234 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1041_1041_0, total 46 rows starting from the beginning of the part 2024.10.22 01:42:15.005242 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1042_1042_0, total 25 rows starting from the beginning of the part 2024.10.22 01:42:15.286457 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1043_1043_0, total 38 rows starting from the beginning of the part 2024.10.22 01:42:15.345640 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1044_1044_0, total 53 rows starting from the beginning of the part 2024.10.22 01:42:15.404560 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1045_1045_0, total 83 rows starting from the beginning of the part 2024.10.22 01:42:15.468703 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 12.40 MiB, will set to 1.70 GiB (RSS), difference: 8.54 MiB 2024.10.22 01:42:15.629154 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1039_499} MergeTask::MergeProjectionsStage: Merge sorted 68108 rows, containing 13 columns (13 merged, 0 gathered) in 5.706883107 sec., 11934.36044212286 rows/sec., 1.97 MiB/sec. 2024.10.22 01:42:15.675871 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 512 2024.10.22 01:42:15.788425 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1039_499} MergedBlockOutputStream: filled checksums 202410_1_1039_499 (state Temporary) 2024.10.22 01:42:15.802319 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(57fcec67ba1fe3dfd23032dcaab5ea37) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 57fcec67ba1fe3dfd23032dcaab5ea37, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:15.806769 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(57fcec67ba1fe3dfd23032dcaab5ea37): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:42:16.229349 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1039_499} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1039_499 to 202410_1_1039_499 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:16.235811 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1039_499} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1033_498, 202410_1034_1039_1] -> 202410_1_1039_499 2024.10.22 01:42:16.267314 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.50 MiB. 2024.10.22 01:42:16.376587 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:16.379040 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:16.452466 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 203 entries to flush up to offset 69350 2024.10.22 01:42:16.655887 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:16.659215 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:16.689572 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_494_494_0 (state Temporary) 2024.10.22 01:42:16.723447 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_600_600_0 (state Temporary) 2024.10.22 01:42:16.758784 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} MergeTask::MergeProjectionsStage: Merge sorted 338 rows, containing 13 columns (13 merged, 0 gathered) in 1.957375989 sec., 172.68016053097708 rows/sec., 30.95 KiB/sec. 2024.10.22 01:42:16.774344 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e893d57b412a2a43573f764c140033dd) : [0, 2666]: Setting download as failed: File segment: [0, 2666], key: e893d57b412a2a43573f764c140033dd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:16.778204 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e893d57b412a2a43573f764c140033dd): No space left in cache to reserve 2667 bytes, will continue without cache download 2024.10.22 01:42:16.796751 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e893d57b412a2a43573f764c140033dd, offset: 0, size: 0 2024.10.22 01:42:16.799477 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e893d57b412a2a43573f764c140033dd for removal 2024.10.22 01:42:16.805656 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 57fcec67ba1fe3dfd23032dcaab5ea37, offset: 0, size: 0 2024.10.22 01:42:16.808295 [ 1476 ] {} CacheMetadata: Key e893d57b412a2a43573f764c140033dd is removed from metadata 2024.10.22 01:42:16.809467 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 57fcec67ba1fe3dfd23032dcaab5ea37 for removal 2024.10.22 01:42:16.817771 [ 1476 ] {} CacheMetadata: Key 57fcec67ba1fe3dfd23032dcaab5ea37 is removed from metadata 2024.10.22 01:42:16.819506 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} MergedBlockOutputStream: filled checksums 202410_1040_1045_1 (state Temporary) 2024.10.22 01:42:16.872963 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_494_494_0 to 202410_504_504_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:16.917371 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_600_600_0 to 202410_606_606_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:16.963989 [ 1262 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_601_19 to 202410_606_606_0 2024.10.22 01:42:16.977446 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 69350 2024.10.22 01:42:17.072368 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1040_1045_1 to 202410_1040_1045_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:17.072439 [ 1262 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:17.076284 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1040_1045_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1040_1040_0, 202410_1045_1045_0] -> 202410_1040_1045_1 2024.10.22 01:42:17.078195 [ 1262 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:17.095455 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_601_19 to 202410_606_606_0 into Compact with storage Full 2024.10.22 01:42:17.116247 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 512 2024.10.22 01:42:17.117968 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:42:17.167946 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_542_7, 202410_543_543_0, 202410_544_544_0, 202410_545_545_0, 202410_546_546_0, 202410_547_547_0] 2024.10.22 01:42:17.171216 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_542_7, 202410_543_543_0, 202410_544_544_0, 202410_545_545_0, 202410_546_546_0, 202410_547_547_0] 2024.10.22 01:42:17.188766 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:17.198694 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_542_7, 202410_543_543_0, 202410_544_544_0, 202410_545_545_0, 202410_546_546_0, 202410_547_547_0] 2024.10.22 01:42:17.201798 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_601_19, total 9569 rows starting from the beginning of the part 2024.10.22 01:42:17.244688 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_602_602_0, total 171 rows starting from the beginning of the part 2024.10.22 01:42:17.408719 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_603_603_0, total 144 rows starting from the beginning of the part 2024.10.22 01:42:17.455418 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_604_604_0, total 81 rows starting from the beginning of the part 2024.10.22 01:42:17.469289 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 10.97 MiB, will set to 1.70 GiB (RSS), difference: 36.95 MiB 2024.10.22 01:42:17.503701 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_605_605_0, total 57 rows starting from the beginning of the part 2024.10.22 01:42:17.569270 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_606_606_0, total 203 rows starting from the beginning of the part 2024.10.22 01:42:17.782534 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7a1e89ec2ea3981ac2e37c84655abe49) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7a1e89ec2ea3981ac2e37c84655abe49, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:42:17.799885 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 7a1e89ec2ea3981ac2e37c84655abe49, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:42:17.804083 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:18.807823 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 67703 2024.10.22 01:42:18.813948 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7dc46f126b909721f3a7005c4e751401) : [0, 136]: Setting download as failed: File segment: [0, 136], key: 7dc46f126b909721f3a7005c4e751401, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:18.815576 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7dc46f126b909721f3a7005c4e751401): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:42:19.007726 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} MergeTask::MergeProjectionsStage: Merge sorted 10225 rows, containing 13 columns (13 merged, 0 gathered) in 1.922730315 sec., 5317.958488629749 rows/sec., 2.31 MiB/sec. 2024.10.22 01:42:19.054996 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} MergedBlockOutputStream: filled checksums 202410_507_606_20 (state Temporary) 2024.10.22 01:42:19.268992 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1434 entries to flush up to offset 912793 2024.10.22 01:42:19.288935 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:19.291571 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:19.473271 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_606_20 to 202410_507_606_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:19.489746 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_606_20} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_601_19, 202410_606_606_0] -> 202410_507_606_20 2024.10.22 01:42:19.570913 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1033_1033_0 (state Temporary) 2024.10.22 01:42:19.659457 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.92 MiB. 2024.10.22 01:42:19.755047 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1033_1033_0 to 202410_1046_1046_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:19.988648 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f2b7a96270a33dfddc8ca38cbd08b257) : [0, 4936]: Setting download as failed: File segment: [0, 4936], key: f2b7a96270a33dfddc8ca38cbd08b257, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:19.989684 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f2b7a96270a33dfddc8ca38cbd08b257): No space left in cache to reserve 4937 bytes, will continue without cache download 2024.10.22 01:42:20.019111 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f2b7a96270a33dfddc8ca38cbd08b257, offset: 0, size: 0 2024.10.22 01:42:20.022669 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f2b7a96270a33dfddc8ca38cbd08b257 for removal 2024.10.22 01:42:20.027131 [ 1476 ] {} CacheMetadata: Key f2b7a96270a33dfddc8ca38cbd08b257 is removed from metadata 2024.10.22 01:42:20.028187 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7dc46f126b909721f3a7005c4e751401, offset: 0, size: 0 2024.10.22 01:42:20.032705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7dc46f126b909721f3a7005c4e751401 for removal 2024.10.22 01:42:20.045788 [ 1476 ] {} CacheMetadata: Key 7dc46f126b909721f3a7005c4e751401 is removed from metadata 2024.10.22 01:42:20.109422 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67703 2024.10.22 01:42:20.244426 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:20.349571 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:20.502994 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_586_586_0 (state Temporary) 2024.10.22 01:42:20.597250 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_586_586_0 to 202410_593_593_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:20.666570 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:42:20.667509 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:42:20.719019 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 14.03 MiB, will set to 1.70 GiB (RSS), difference: -2.73 MiB 2024.10.22 01:42:20.757611 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 912793 2024.10.22 01:42:21.184167 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f9c9a643347527e79852c5afd3d9df88) : [0, 138]: Setting download as failed: File segment: [0, 138], key: f9c9a643347527e79852c5afd3d9df88, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:21.186396 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f9c9a643347527e79852c5afd3d9df88): No space left in cache to reserve 139 bytes, will continue without cache download 2024.10.22 01:42:21.187530 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_123_123_0 to 202410_126_126_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:21.471228 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_124_23/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:42:21.587924 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 38 entries to flush up to offset 9401 2024.10.22 01:42:21.654990 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_586_115/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:42:21.701212 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:21.711747 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:21.768958 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_612_612_0 (state Temporary) 2024.10.22 01:42:21.818363 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_612_612_0 to 202410_625_625_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:21.983217 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9401 2024.10.22 01:42:22.086150 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8392da079cc0a11221dec68def2f1391) : [0, 72248]: Setting download as failed: File segment: [0, 72248], key: 8392da079cc0a11221dec68def2f1391, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:22.088044 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8392da079cc0a11221dec68def2f1391): No space left in cache to reserve 72249 bytes, will continue without cache download 2024.10.22 01:42:22.123362 [ 1155 ] {} DatabaseCatalog: Execution took 412 ms. 2024.10.22 01:42:22.127014 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8392da079cc0a11221dec68def2f1391, offset: 0, size: 0 2024.10.22 01:42:22.130645 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8392da079cc0a11221dec68def2f1391 for removal 2024.10.22 01:42:22.136015 [ 1476 ] {} CacheMetadata: Key 8392da079cc0a11221dec68def2f1391 is removed from metadata 2024.10.22 01:42:22.138558 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f9c9a643347527e79852c5afd3d9df88, offset: 0, size: 0 2024.10.22 01:42:22.141512 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f9c9a643347527e79852c5afd3d9df88 for removal 2024.10.22 01:42:22.160725 [ 1476 ] {} CacheMetadata: Key f9c9a643347527e79852c5afd3d9df88 is removed from metadata 2024.10.22 01:42:22.733284 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_586_115} MergeTask::MergeProjectionsStage: Merge sorted 916094 rows, containing 4 columns (4 merged, 0 gathered) in 27.745297919 sec., 33017.99110878021 rows/sec., 495.10 KiB/sec. 2024.10.22 01:42:22.769366 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 11.23 MiB, will set to 1.70 GiB (RSS), difference: 8.19 MiB 2024.10.22 01:42:22.888002 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(abfc2d57196bf6073dc58bc49fe74fa6) : [0, 136]: Setting download as failed: File segment: [0, 136], key: abfc2d57196bf6073dc58bc49fe74fa6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:22.889270 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(abfc2d57196bf6073dc58bc49fe74fa6): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:42:22.961614 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_586_115} MergedBlockOutputStream: filled checksums 202410_1_586_115 (state Temporary) 2024.10.22 01:42:23.163679 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_934_472, 202410_935_940_1] 2024.10.22 01:42:23.169440 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_934_472, 202410_935_940_1] 2024.10.22 01:42:23.179197 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_934_472, 202410_935_940_1] 2024.10.22 01:42:23.972080 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(df566cc10a1276eb2e945b156d78ad28) : [0, 48179]: Setting download as failed: File segment: [0, 48179], key: df566cc10a1276eb2e945b156d78ad28, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:23.974156 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(df566cc10a1276eb2e945b156d78ad28): No space left in cache to reserve 48180 bytes, will continue without cache download 2024.10.22 01:42:23.982583 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: df566cc10a1276eb2e945b156d78ad28, offset: 0, size: 0 2024.10.22 01:42:23.984526 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key df566cc10a1276eb2e945b156d78ad28 for removal 2024.10.22 01:42:23.987148 [ 1476 ] {} CacheMetadata: Key df566cc10a1276eb2e945b156d78ad28 is removed from metadata 2024.10.22 01:42:23.990612 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: abfc2d57196bf6073dc58bc49fe74fa6, offset: 0, size: 0 2024.10.22 01:42:23.995808 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key abfc2d57196bf6073dc58bc49fe74fa6 for removal 2024.10.22 01:42:24.004683 [ 1476 ] {} CacheMetadata: Key abfc2d57196bf6073dc58bc49fe74fa6 is removed from metadata 2024.10.22 01:42:24.130865 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 67769 2024.10.22 01:42:24.388754 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:24.396947 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:24.446870 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 13.18 MiB, will set to 1.70 GiB (RSS), difference: 12.84 MiB 2024.10.22 01:42:24.502896 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1034_1034_0 (state Temporary) 2024.10.22 01:42:24.504795 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 85 entries to flush up to offset 69435 2024.10.22 01:42:24.561192 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4929 2024.10.22 01:42:24.618506 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 513 2024.10.22 01:42:24.640348 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1034_1034_0 to 202410_1047_1047_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:24.730648 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:24.735703 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:24.780129 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67769 2024.10.22 01:42:24.782649 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:24.785802 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:24.798730 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_601_601_0 (state Temporary) 2024.10.22 01:42:24.864485 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_495_495_0 (state Temporary) 2024.10.22 01:42:24.945118 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_601_601_0 to 202410_607_607_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:25.088457 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 69435 2024.10.22 01:42:25.151451 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_495_495_0 to 202410_505_505_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:25.329835 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 513 2024.10.22 01:42:25.340074 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b61729c0b9f8b7ac25a34680d8e627d8) : [0, 167]: Setting download as failed: File segment: [0, 167], key: b61729c0b9f8b7ac25a34680d8e627d8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:25.344927 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b61729c0b9f8b7ac25a34680d8e627d8): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:42:25.415384 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_586_115} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_586_115 to 202410_1_586_115 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:25.423774 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_586_115} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_580_114, 202410_581_586_1] -> 202410_1_586_115 2024.10.22 01:42:25.458130 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:42:26.032476 [ 1235 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:42:26.067546 [ 1235 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:42:26.383851 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_941_941_0, 202410_942_942_0, 202410_943_943_0, 202410_944_944_0, 202410_945_945_0, 202410_946_946_0] 2024.10.22 01:42:26.386474 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_941_941_0, 202410_942_942_0, 202410_943_943_0, 202410_944_944_0, 202410_945_945_0, 202410_946_946_0] 2024.10.22 01:42:26.419233 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_941_941_0, 202410_942_942_0, 202410_943_943_0, 202410_944_944_0, 202410_945_945_0, 202410_946_946_0] 2024.10.22 01:42:26.663059 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6ef50cc8407b77260298ba9672b0c61f) : [0, 109599]: Setting download as failed: File segment: [0, 109599], key: 6ef50cc8407b77260298ba9672b0c61f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:26.664733 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6ef50cc8407b77260298ba9672b0c61f): No space left in cache to reserve 109600 bytes, will continue without cache download 2024.10.22 01:42:26.677353 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6ef50cc8407b77260298ba9672b0c61f, offset: 0, size: 0 2024.10.22 01:42:26.679326 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6ef50cc8407b77260298ba9672b0c61f for removal 2024.10.22 01:42:26.681296 [ 1476 ] {} CacheMetadata: Key 6ef50cc8407b77260298ba9672b0c61f is removed from metadata 2024.10.22 01:42:26.682443 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b61729c0b9f8b7ac25a34680d8e627d8, offset: 0, size: 0 2024.10.22 01:42:26.686246 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b61729c0b9f8b7ac25a34680d8e627d8 for removal 2024.10.22 01:42:26.689812 [ 1476 ] {} CacheMetadata: Key b61729c0b9f8b7ac25a34680d8e627d8 is removed from metadata 2024.10.22 01:42:27.862796 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1646 entries to flush up to offset 914439 2024.10.22 01:42:28.163525 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(61ff50e65ba0599021f2922135c0eb46) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 61ff50e65ba0599021f2922135c0eb46, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:28.165097 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(61ff50e65ba0599021f2922135c0eb46): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:42:28.616725 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:28.620405 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:42:28.637968 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 9.23 MiB, will set to 1.71 GiB (RSS), difference: -3.97 MiB 2024.10.22 01:42:28.769178 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_561_112, 202410_562_562_0, 202410_563_563_0, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0] 2024.10.22 01:42:28.772511 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_561_112, 202410_562_562_0, 202410_563_563_0, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0] 2024.10.22 01:42:28.791889 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 67815 2024.10.22 01:42:28.900490 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:28.906574 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:28.910911 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_561_112, 202410_562_562_0, 202410_563_563_0, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0] 2024.10.22 01:42:28.960825 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_587_587_0 (state Temporary) 2024.10.22 01:42:28.968799 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a45b46d3310a8ce7f855eff301d72a50) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a45b46d3310a8ce7f855eff301d72a50, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:42:28.970661 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 869764, file segment info: File segment: [0, 33554431], key: a45b46d3310a8ce7f855eff301d72a50, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:42:28.972270 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:29.055857 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1035_1035_0 (state Temporary) 2024.10.22 01:42:29.068814 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_587_587_0 to 202410_594_594_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:29.103395 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 914439 2024.10.22 01:42:29.352523 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1035_1035_0 to 202410_1048_1048_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:29.393771 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67815 2024.10.22 01:42:29.488256 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 9410 2024.10.22 01:42:29.716131 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:29.719622 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:42:29.761503 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_613_613_0 (state Temporary) 2024.10.22 01:42:29.910435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(59ce7a13f681bb6e1a7fc20b292ca057) : [0, 21014]: Setting download as failed: File segment: [0, 21014], key: 59ce7a13f681bb6e1a7fc20b292ca057, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:29.917417 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(59ce7a13f681bb6e1a7fc20b292ca057): No space left in cache to reserve 21015 bytes, will continue without cache download 2024.10.22 01:42:29.932303 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_613_613_0 to 202410_626_626_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:29.954666 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 59ce7a13f681bb6e1a7fc20b292ca057, offset: 0, size: 0 2024.10.22 01:42:29.957200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 59ce7a13f681bb6e1a7fc20b292ca057 for removal 2024.10.22 01:42:29.961352 [ 1476 ] {} CacheMetadata: Key 59ce7a13f681bb6e1a7fc20b292ca057 is removed from metadata 2024.10.22 01:42:29.965118 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 61ff50e65ba0599021f2922135c0eb46, offset: 0, size: 0 2024.10.22 01:42:29.966654 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 61ff50e65ba0599021f2922135c0eb46 for removal 2024.10.22 01:42:29.968965 [ 1476 ] {} CacheMetadata: Key 61ff50e65ba0599021f2922135c0eb46 is removed from metadata 2024.10.22 01:42:29.969418 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_621_124 to 202410_626_626_0 2024.10.22 01:42:29.977165 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:29.980545 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:42:30.048871 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_621_124 to 202410_626_626_0 into Compact with storage Full 2024.10.22 01:42:30.082657 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9410 2024.10.22 01:42:30.097311 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:30.127761 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_621_124, total 9446 rows starting from the beginning of the part 2024.10.22 01:42:30.286538 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_622_622_0, total 13 rows starting from the beginning of the part 2024.10.22 01:42:30.363795 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_623_623_0, total 5 rows starting from the beginning of the part 2024.10.22 01:42:30.546630 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_624_624_0, total 5 rows starting from the beginning of the part 2024.10.22 01:42:30.587966 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_625_625_0, total 38 rows starting from the beginning of the part 2024.10.22 01:42:30.638105 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_626_626_0, total 9 rows starting from the beginning of the part 2024.10.22 01:42:30.667566 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:42:30.668661 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:42:30.692028 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(96aa6068289f2cd27e4a0e67dab366c2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 96aa6068289f2cd27e4a0e67dab366c2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:42:30.708902 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 505, file segment info: File segment: [0, 33554431], key: 96aa6068289f2cd27e4a0e67dab366c2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:42:30.712236 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:30.971919 [ 1688 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ttg/oubsrsvzkrzponvtpfzqlpyercjbm, size 869764 2024.10.22 01:42:31.309699 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:42:31.341136 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:42:31.353069 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:42:31.465578 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:42:31.472625 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:42:31.586374 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:42:31.635610 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:42:31.674537 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column FUniqID 2024.10.22 01:42:31.701788 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(674690abb4ed77fdfdb5920085a1a136) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 674690abb4ed77fdfdb5920085a1a136, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:31.704069 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(674690abb4ed77fdfdb5920085a1a136): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:42:31.844537 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jej/qutcrsdvoyrowwnfehlvfrdzocrkz (LogSeriesLimiter: on interval from 2024-10-22 01:42:12 to 2024-10-22 01:42:31 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:42:32.103360 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 4971 2024.10.22 01:42:32.327652 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} MergeTask::MergeProjectionsStage: Merge sorted 9516 rows, containing 12 columns (12 merged, 0 gathered) in 2.298273436 sec., 4140.499494508364 rows/sec., 658.67 KiB/sec. 2024.10.22 01:42:32.346953 [ 1248 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 01:42:32.386340 [ 1250 ] {} DatabaseCatalog: Execution took 255 ms. 2024.10.22 01:42:32.504438 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} MergedBlockOutputStream: filled checksums 202410_1_626_125 (state Temporary) 2024.10.22 01:42:32.601752 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 69526 2024.10.22 01:42:32.761459 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_626_125 to 202410_1_626_125 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:32.765744 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_626_125} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_621_124, 202410_626_626_0] -> 202410_1_626_125 2024.10.22 01:42:32.770657 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8c74a7007ab84396f29e8c857e9f9cf3) : [0, 27572]: Setting download as failed: File segment: [0, 27572], key: 8c74a7007ab84396f29e8c857e9f9cf3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:32.771738 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8c74a7007ab84396f29e8c857e9f9cf3): No space left in cache to reserve 27573 bytes, will continue without cache download 2024.10.22 01:42:32.786596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8c74a7007ab84396f29e8c857e9f9cf3, offset: 0, size: 0 2024.10.22 01:42:32.787858 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8c74a7007ab84396f29e8c857e9f9cf3 for removal 2024.10.22 01:42:32.793088 [ 1476 ] {} CacheMetadata: Key 8c74a7007ab84396f29e8c857e9f9cf3 is removed from metadata 2024.10.22 01:42:32.797597 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 674690abb4ed77fdfdb5920085a1a136, offset: 0, size: 0 2024.10.22 01:42:32.802295 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 674690abb4ed77fdfdb5920085a1a136 for removal 2024.10.22 01:42:32.809225 [ 1476 ] {} CacheMetadata: Key 674690abb4ed77fdfdb5920085a1a136 is removed from metadata 2024.10.22 01:42:32.814341 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.59 MiB. 2024.10.22 01:42:32.834711 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 514 2024.10.22 01:42:32.975602 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.038889362 sec. 2024.10.22 01:42:33.011959 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:33.014223 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:42:33.237077 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_602_602_0 (state Temporary) 2024.10.22 01:42:33.276247 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_602_602_0 to 202410_608_608_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:33.333260 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 69526 2024.10.22 01:42:33.408231 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 67894 2024.10.22 01:42:33.579119 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:33.581215 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.94 GiB. 2024.10.22 01:42:33.811919 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f886fa8b269707775d111717badaee6a) : [0, 151]: Setting download as failed: File segment: [0, 151], key: f886fa8b269707775d111717badaee6a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:33.814093 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f886fa8b269707775d111717badaee6a): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:42:33.845217 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_496_496_0 (state Temporary) 2024.10.22 01:42:34.025302 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:34.032029 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.93 GiB. 2024.10.22 01:42:34.220880 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_496_496_0 to 202410_506_506_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:34.299036 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1036_1036_0 (state Temporary) 2024.10.22 01:42:34.318255 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 514 2024.10.22 01:42:34.455661 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1036_1036_0 to 202410_1049_1049_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:34.492666 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67894 2024.10.22 01:42:34.668314 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 12.18 MiB, will set to 1.70 GiB (RSS), difference: 3.86 MiB 2024.10.22 01:42:34.742500 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(501fe3760931b9fe8060290f32485295) : [0, 35344]: Setting download as failed: File segment: [0, 35344], key: 501fe3760931b9fe8060290f32485295, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:34.744400 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(501fe3760931b9fe8060290f32485295): No space left in cache to reserve 35345 bytes, will continue without cache download 2024.10.22 01:42:34.761381 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 501fe3760931b9fe8060290f32485295, offset: 0, size: 0 2024.10.22 01:42:34.763976 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 501fe3760931b9fe8060290f32485295 for removal 2024.10.22 01:42:34.766391 [ 1476 ] {} CacheMetadata: Key 501fe3760931b9fe8060290f32485295 is removed from metadata 2024.10.22 01:42:34.766923 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f886fa8b269707775d111717badaee6a, offset: 0, size: 0 2024.10.22 01:42:34.767851 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f886fa8b269707775d111717badaee6a for removal 2024.10.22 01:42:34.771520 [ 1476 ] {} CacheMetadata: Key f886fa8b269707775d111717badaee6a is removed from metadata 2024.10.22 01:42:35.954381 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(9b48810186d7830c1bba0e67810aac50) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9b48810186d7830c1bba0e67810aac50, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:42:35.960676 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1208, file segment info: File segment: [0, 33554431], key: 9b48810186d7830c1bba0e67810aac50, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:42:35.970538 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:36.380093 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 915209 2024.10.22 01:42:36.568877 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column CLID 2024.10.22 01:42:36.601525 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column CLID 2024.10.22 01:42:36.644436 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column CLID 2024.10.22 01:42:36.684823 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column CLID 2024.10.22 01:42:36.722548 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column CLID 2024.10.22 01:42:36.771663 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column CLID 2024.10.22 01:42:36.796009 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(976f4c64b1c70a48708e50699d9cce5b) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 976f4c64b1c70a48708e50699d9cce5b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:36.805381 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(976f4c64b1c70a48708e50699d9cce5b): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:42:36.820986 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column CLID 2024.10.22 01:42:36.876758 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column CLID 2024.10.22 01:42:36.989787 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column CLID 2024.10.22 01:42:37.103124 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_124_23/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:42:37.245477 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:37.253796 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:37.271293 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mkg/iluwosumfsvhoiksypotnedgkhvos (LogSeriesLimiter: on interval from 2024-10-22 01:42:31 to 2024-10-22 01:42:37 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:42:37.511383 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_588_588_0 (state Temporary) 2024.10.22 01:42:37.576974 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_588_588_0 to 202410_595_595_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:37.585543 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 9422 2024.10.22 01:42:37.644031 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 915209 2024.10.22 01:42:37.792446 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:37.815944 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:37.837171 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a3960979d8465fde72e3b42b1193fb0a) : [0, 34008]: Setting download as failed: File segment: [0, 34008], key: a3960979d8465fde72e3b42b1193fb0a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:37.842904 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a3960979d8465fde72e3b42b1193fb0a): No space left in cache to reserve 34009 bytes, will continue without cache download 2024.10.22 01:42:37.855071 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_614_614_0 (state Temporary) 2024.10.22 01:42:37.962361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a3960979d8465fde72e3b42b1193fb0a, offset: 0, size: 0 2024.10.22 01:42:37.963946 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a3960979d8465fde72e3b42b1193fb0a for removal 2024.10.22 01:42:37.966325 [ 1476 ] {} CacheMetadata: Key a3960979d8465fde72e3b42b1193fb0a is removed from metadata 2024.10.22 01:42:38.052499 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_614_614_0 to 202410_627_627_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:38.066783 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 976f4c64b1c70a48708e50699d9cce5b, offset: 0, size: 0 2024.10.22 01:42:38.067845 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 976f4c64b1c70a48708e50699d9cce5b for removal 2024.10.22 01:42:38.069567 [ 1476 ] {} CacheMetadata: Key 976f4c64b1c70a48708e50699d9cce5b is removed from metadata 2024.10.22 01:42:38.089287 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9422 2024.10.22 01:42:38.367589 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 11.89 MiB, will set to 1.67 GiB (RSS), difference: 4.36 MiB 2024.10.22 01:42:38.527460 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 67953 2024.10.22 01:42:38.958629 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:38.960869 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:39.057957 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(76bd4b3bdce1a79ae6a5694b5f9353ce) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 76bd4b3bdce1a79ae6a5694b5f9353ce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:39.059467 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(76bd4b3bdce1a79ae6a5694b5f9353ce): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:42:39.114350 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1037_1037_0 (state Temporary) 2024.10.22 01:42:39.161913 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1037_1037_0 to 202410_1050_1050_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:39.163309 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:42:39.229826 [ 1162 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1039_499 to 202410_1040_1045_1 2024.10.22 01:42:39.238694 [ 1162 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.54 MiB using storage policy from min volume index 0 2024.10.22 01:42:39.240973 [ 1162 ] {} DiskLocal: Reserved 2.54 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:39.240979 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 67953 2024.10.22 01:42:39.266323 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1045_500} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1039_499 to 202410_1040_1045_1 into Wide with storage Full 2024.10.22 01:42:39.305611 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1045_500} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:39.422192 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1045_500} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1039_499, total 68108 rows starting from the beginning of the part 2024.10.22 01:42:39.564988 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1045_500} MergeTreeSequentialSource: Reading 2 marks from part 202410_1040_1045_1, total 338 rows starting from the beginning of the part 2024.10.22 01:42:40.166437 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(9b115df86358e9199e28f4e246808406) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9b115df86358e9199e28f4e246808406, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 01:42:40.168580 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 9b115df86358e9199e28f4e246808406, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 01:42:40.172540 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:40.669120 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:42:40.667588 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:42:40.854928 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 154 entries to flush up to offset 69680 2024.10.22 01:42:41.017749 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_453_93, 202410_454_454_0, 202410_455_455_0, 202410_456_456_0, 202410_457_457_0, 202410_458_458_0] 2024.10.22 01:42:41.023877 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_453_93, 202410_454_454_0, 202410_455_455_0, 202410_456_456_0, 202410_457_457_0, 202410_458_458_0] 2024.10.22 01:42:41.059931 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_453_93, 202410_454_454_0, 202410_455_455_0, 202410_456_456_0, 202410_457_457_0, 202410_458_458_0] 2024.10.22 01:42:41.072093 [ 1251 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:42:41.087147 [ 1251 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:42:41.104470 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_533_533_0, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0, 202410_538_538_0] 2024.10.22 01:42:41.108887 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_533_533_0, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0, 202410_538_538_0] 2024.10.22 01:42:41.132554 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:41.140221 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:41.147707 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_533_533_0, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0, 202410_538_538_0] 2024.10.22 01:42:41.183523 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(04d24cca4ce7300f974d55f050561bef) : [0, 19773]: Setting download as failed: File segment: [0, 19773], key: 04d24cca4ce7300f974d55f050561bef, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:41.184840 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(04d24cca4ce7300f974d55f050561bef): No space left in cache to reserve 19774 bytes, will continue without cache download 2024.10.22 01:42:41.191200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 04d24cca4ce7300f974d55f050561bef, offset: 0, size: 0 2024.10.22 01:42:41.192737 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 04d24cca4ce7300f974d55f050561bef for removal 2024.10.22 01:42:41.197042 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 76bd4b3bdce1a79ae6a5694b5f9353ce, offset: 0, size: 0 2024.10.22 01:42:41.197486 [ 1476 ] {} CacheMetadata: Key 04d24cca4ce7300f974d55f050561bef is removed from metadata 2024.10.22 01:42:41.201664 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 76bd4b3bdce1a79ae6a5694b5f9353ce for removal 2024.10.22 01:42:41.205601 [ 1476 ] {} CacheMetadata: Key 76bd4b3bdce1a79ae6a5694b5f9353ce is removed from metadata 2024.10.22 01:42:41.275766 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_603_603_0 (state Temporary) 2024.10.22 01:42:41.373401 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_603_603_0 to 202410_609_609_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:41.409620 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 69680 2024.10.22 01:42:41.489696 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 12.16 MiB, will set to 1.68 GiB (RSS), difference: -41.21 MiB 2024.10.22 01:42:41.723718 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_1045_500/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:42:41.820516 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 515 2024.10.22 01:42:42.110611 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fe60de0bc1b8deffb53ba129a33acb16) : [0, 140]: Setting download as failed: File segment: [0, 140], key: fe60de0bc1b8deffb53ba129a33acb16, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:42.112091 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fe60de0bc1b8deffb53ba129a33acb16): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:42:42.126531 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:42.131236 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:42.188234 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_497_497_0 (state Temporary) 2024.10.22 01:42:42.595012 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_497_497_0 to 202410_507_507_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:42.732200 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1045_500} MergeTask::MergeProjectionsStage: Merge sorted 68446 rows, containing 13 columns (13 merged, 0 gathered) in 3.458644141 sec., 19789.835903791525 rows/sec., 3.27 MiB/sec. 2024.10.22 01:42:42.974507 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1045_500} MergedBlockOutputStream: filled checksums 202410_1_1045_500 (state Temporary) 2024.10.22 01:42:43.028968 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 515 2024.10.22 01:42:43.261535 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 68008 2024.10.22 01:42:43.400745 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(07c3986e40b0a808028a41f9fa7c65bf) : [0, 16670]: Setting download as failed: File segment: [0, 16670], key: 07c3986e40b0a808028a41f9fa7c65bf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:43.402581 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(07c3986e40b0a808028a41f9fa7c65bf): No space left in cache to reserve 16671 bytes, will continue without cache download 2024.10.22 01:42:43.413779 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 07c3986e40b0a808028a41f9fa7c65bf, offset: 0, size: 0 2024.10.22 01:42:43.416759 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 07c3986e40b0a808028a41f9fa7c65bf for removal 2024.10.22 01:42:43.419485 [ 1476 ] {} CacheMetadata: Key 07c3986e40b0a808028a41f9fa7c65bf is removed from metadata 2024.10.22 01:42:43.420783 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fe60de0bc1b8deffb53ba129a33acb16, offset: 0, size: 0 2024.10.22 01:42:43.424770 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fe60de0bc1b8deffb53ba129a33acb16 for removal 2024.10.22 01:42:43.431209 [ 1476 ] {} CacheMetadata: Key fe60de0bc1b8deffb53ba129a33acb16 is removed from metadata 2024.10.22 01:42:43.477102 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 9.74 MiB, will set to 1.68 GiB (RSS), difference: 33.01 MiB 2024.10.22 01:42:43.533312 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:43.638765 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:43.780620 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1038_1038_0 (state Temporary) 2024.10.22 01:42:43.913748 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1045_500} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1045_500 to 202410_1_1045_500 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:43.924287 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1045_500} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1039_499, 202410_1040_1045_1] -> 202410_1_1045_500 2024.10.22 01:42:43.941507 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1038_1038_0 to 202410_1051_1051_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:43.971519 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1045_500 to 202410_1046_1046_0 2024.10.22 01:42:43.975665 [ 1157 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.54 MiB using storage policy from min volume index 0 2024.10.22 01:42:43.977332 [ 1157 ] {} DiskLocal: Reserved 2.54 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:44.040279 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.54 MiB. 2024.10.22 01:42:44.111321 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68008 2024.10.22 01:42:44.163259 [ 1157 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 201 ms. 2024.10.22 01:42:44.273534 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1046_501} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1045_500 to 202410_1046_1046_0 into Wide with storage Full 2024.10.22 01:42:44.296638 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1f8a9d9405590dbf6586d93196a84556) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 1f8a9d9405590dbf6586d93196a84556, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:44.300111 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1f8a9d9405590dbf6586d93196a84556): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:42:44.434785 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1046_501} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:44.453556 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1046_501} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1045_500, total 68446 rows starting from the beginning of the part 2024.10.22 01:42:44.628455 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1046_501} MergeTreeSequentialSource: Reading 2 marks from part 202410_1046_1046_0, total 88 rows starting from the beginning of the part 2024.10.22 01:42:44.647092 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 916749 2024.10.22 01:42:45.371052 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7319f64375530596f8dafc8790333040) : [0, 11931]: Setting download as failed: File segment: [0, 11931], key: 7319f64375530596f8dafc8790333040, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:45.372114 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7319f64375530596f8dafc8790333040): No space left in cache to reserve 11932 bytes, will continue without cache download 2024.10.22 01:42:45.380842 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7319f64375530596f8dafc8790333040, offset: 0, size: 0 2024.10.22 01:42:45.383352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7319f64375530596f8dafc8790333040 for removal 2024.10.22 01:42:45.386563 [ 1476 ] {} CacheMetadata: Key 7319f64375530596f8dafc8790333040 is removed from metadata 2024.10.22 01:42:45.388278 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1f8a9d9405590dbf6586d93196a84556, offset: 0, size: 0 2024.10.22 01:42:45.391140 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1f8a9d9405590dbf6586d93196a84556 for removal 2024.10.22 01:42:45.413630 [ 1476 ] {} CacheMetadata: Key 1f8a9d9405590dbf6586d93196a84556 is removed from metadata 2024.10.22 01:42:45.506832 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 6.88 MiB, will set to 1.67 GiB (RSS), difference: -40.09 MiB 2024.10.22 01:42:45.592108 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 9430 2024.10.22 01:42:45.888095 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:45.891308 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:46.041816 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_615_615_0 (state Temporary) 2024.10.22 01:42:46.195593 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_615_615_0 to 202410_628_628_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:46.325686 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:46.331879 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:46.365539 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9430 2024.10.22 01:42:46.481790 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_589_589_0 (state Temporary) 2024.10.22 01:42:46.681112 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_589_589_0 to 202410_596_596_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:46.728034 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(76a7d0ef726d2106acfe1e84abd4a51e) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 76a7d0ef726d2106acfe1e84abd4a51e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:46.732720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(76a7d0ef726d2106acfe1e84abd4a51e): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:42:46.715762 [ 1185 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_586_115 to 202410_587_592_1 2024.10.22 01:42:46.754881 [ 1185 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:46.757445 [ 1185 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:46.830463 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 916749 2024.10.22 01:42:46.976216 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_592_116} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_586_115 to 202410_587_592_1 into Wide with storage Full 2024.10.22 01:42:47.040154 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_592_116} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:47.055343 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_592_116} MergeTreeSequentialSource: Reading 113 marks from part 202410_1_586_115, total 916094 rows starting from the beginning of the part 2024.10.22 01:42:47.084528 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_592_116} MergeTreeSequentialSource: Reading 3 marks from part 202410_587_592_1, total 8855 rows starting from the beginning of the part 2024.10.22 01:42:47.729122 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 5.41 MiB, will set to 1.67 GiB (RSS), difference: -8.46 MiB 2024.10.22 01:42:48.085334 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_526_105, 202410_527_532_1] 2024.10.22 01:42:48.087931 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_526_105, 202410_527_532_1] 2024.10.22 01:42:48.099807 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_526_105, 202410_527_532_1] 2024.10.22 01:42:48.121014 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 68070 2024.10.22 01:42:48.182707 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1046_501} MergeTask::MergeProjectionsStage: Merge sorted 68534 rows, containing 13 columns (13 merged, 0 gathered) in 3.912228407 sec., 17517.893351363316 rows/sec., 2.89 MiB/sec. 2024.10.22 01:42:48.340688 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b734094611a498be01d735492d70e864) : [0, 36269]: Setting download as failed: File segment: [0, 36269], key: b734094611a498be01d735492d70e864, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:48.343827 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b734094611a498be01d735492d70e864): No space left in cache to reserve 36270 bytes, will continue without cache download 2024.10.22 01:42:48.363345 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b734094611a498be01d735492d70e864, offset: 0, size: 0 2024.10.22 01:42:48.378175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b734094611a498be01d735492d70e864 for removal 2024.10.22 01:42:48.380354 [ 1476 ] {} CacheMetadata: Key b734094611a498be01d735492d70e864 is removed from metadata 2024.10.22 01:42:48.381261 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 76a7d0ef726d2106acfe1e84abd4a51e, offset: 0, size: 0 2024.10.22 01:42:48.382360 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 76a7d0ef726d2106acfe1e84abd4a51e for removal 2024.10.22 01:42:48.386747 [ 1476 ] {} CacheMetadata: Key 76a7d0ef726d2106acfe1e84abd4a51e is removed from metadata 2024.10.22 01:42:48.513331 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:48.529612 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:48.533286 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1046_501} MergedBlockOutputStream: filled checksums 202410_1_1046_501 (state Temporary) 2024.10.22 01:42:48.590529 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1039_1039_0 (state Temporary) 2024.10.22 01:42:48.664011 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1039_1039_0 to 202410_1052_1052_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:48.699954 [ 1207 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1047_1047_0 to 202410_1052_1052_0 2024.10.22 01:42:48.708055 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_592_116/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:42:48.750341 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68070 2024.10.22 01:42:48.757203 [ 1207 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:48.761169 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:48.888623 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1047_1047_0 to 202410_1052_1052_0 into Compact with storage Full 2024.10.22 01:42:48.949004 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:48.964213 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1047_1047_0, total 66 rows starting from the beginning of the part 2024.10.22 01:42:48.976675 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:48.979579 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:48.984805 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 179 entries to flush up to offset 69859 2024.10.22 01:42:49.243511 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1048_1048_0, total 46 rows starting from the beginning of the part 2024.10.22 01:42:49.278780 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:49.391063 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:49.422463 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1049_1049_0, total 79 rows starting from the beginning of the part 2024.10.22 01:42:49.480378 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_604_604_0 (state Temporary) 2024.10.22 01:42:49.496751 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1050_1050_0, total 59 rows starting from the beginning of the part 2024.10.22 01:42:49.560311 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_604_604_0 to 202410_610_610_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:49.612056 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1051_1051_0, total 55 rows starting from the beginning of the part 2024.10.22 01:42:49.625405 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 69859 2024.10.22 01:42:49.664595 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1046_501} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1046_501 to 202410_1_1046_501 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:49.675169 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1046_501} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1045_500, 202410_1046_1046_0] -> 202410_1_1046_501 2024.10.22 01:42:49.702513 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.47 MiB. 2024.10.22 01:42:49.728329 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1052_1052_0, total 62 rows starting from the beginning of the part 2024.10.22 01:42:49.731399 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 10.95 MiB, will set to 1.68 GiB (RSS), difference: 49.02 MiB 2024.10.22 01:42:49.767612 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(508fda6067580c846422f21dac2571f6) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 508fda6067580c846422f21dac2571f6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:49.768994 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(508fda6067580c846422f21dac2571f6): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:42:50.668048 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:42:50.669336 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:42:50.720500 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(1d7a79f63a949cc66c29613a5807dc99) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1d7a79f63a949cc66c29613a5807dc99, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:42:50.725783 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 821, file segment info: File segment: [0, 33554431], key: 1d7a79f63a949cc66c29613a5807dc99, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:42:50.729392 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:50.897384 [ 1713 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jej/qutcrsdvoyrowwnfehlvfrdzocrkz, size 2621248 2024.10.22 01:42:51.170984 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column HID 2024.10.22 01:42:51.174551 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} MergeTask::MergeProjectionsStage: Merge sorted 367 rows, containing 13 columns (13 merged, 0 gathered) in 2.289388416 sec., 160.30482090112926 rows/sec., 27.76 KiB/sec. 2024.10.22 01:42:51.215550 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} MergedBlockOutputStream: filled checksums 202410_1047_1052_1 (state Temporary) 2024.10.22 01:42:51.278690 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column HID 2024.10.22 01:42:51.313039 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column HID 2024.10.22 01:42:51.328298 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column HID 2024.10.22 01:42:51.347604 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column HID 2024.10.22 01:42:51.362443 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_940_473, 202410_941_946_1] 2024.10.22 01:42:51.365284 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_940_473, 202410_941_946_1] 2024.10.22 01:42:51.366425 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(93a8cc401f7311bf488a0112137a2106) : [0, 1274]: Setting download as failed: File segment: [0, 1274], key: 93a8cc401f7311bf488a0112137a2106, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:51.367487 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(93a8cc401f7311bf488a0112137a2106): No space left in cache to reserve 1275 bytes, will continue without cache download 2024.10.22 01:42:51.370414 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column HID 2024.10.22 01:42:51.370679 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1047_1052_1 to 202410_1047_1052_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:51.374514 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1047_1052_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1047_1047_0, 202410_1052_1052_0] -> 202410_1047_1052_1 2024.10.22 01:42:51.382302 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column HID 2024.10.22 01:42:51.376200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 93a8cc401f7311bf488a0112137a2106, offset: 0, size: 0 2024.10.22 01:42:51.387897 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 93a8cc401f7311bf488a0112137a2106 for removal 2024.10.22 01:42:51.393572 [ 1476 ] {} CacheMetadata: Key 93a8cc401f7311bf488a0112137a2106 is removed from metadata 2024.10.22 01:42:51.394311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 508fda6067580c846422f21dac2571f6, offset: 0, size: 0 2024.10.22 01:42:51.398237 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 508fda6067580c846422f21dac2571f6 for removal 2024.10.22 01:42:51.403280 [ 1476 ] {} CacheMetadata: Key 508fda6067580c846422f21dac2571f6 is removed from metadata 2024.10.22 01:42:51.403453 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_940_473, 202410_941_946_1] 2024.10.22 01:42:51.504796 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:42:51.505430 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column HID 2024.10.22 01:42:51.662129 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rmt/kjkhzmqgcmiivnrbwsvmdspnvamxm (LogSeriesLimiter: on interval from 2024-10-22 01:42:37 to 2024-10-22 01:42:51 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:42:51.683879 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_124_124_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:42:52.554939 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_124_124_0 (state Temporary) 2024.10.22 01:42:52.673723 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c744696a962c5727d397f419b2a8572e) : [0, 140]: Setting download as failed: File segment: [0, 140], key: c744696a962c5727d397f419b2a8572e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:52.674694 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c744696a962c5727d397f419b2a8572e): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:42:52.701722 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 12.49 MiB, will set to 1.68 GiB (RSS), difference: -23.69 MiB 2024.10.22 01:42:52.758689 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 68148 2024.10.22 01:42:52.955823 [ 1270 ] {} DatabaseCatalog: Execution took 454 ms. 2024.10.22 01:42:53.069832 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:53.078347 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:53.145448 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1040_1040_0 (state Temporary) 2024.10.22 01:42:53.302397 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1040_1040_0 to 202410_1053_1053_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:53.572847 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5adfe44a157507a7745d91710e8967f6) : [0, 1272]: Setting download as failed: File segment: [0, 1272], key: 5adfe44a157507a7745d91710e8967f6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:53.573990 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5adfe44a157507a7745d91710e8967f6): No space left in cache to reserve 1273 bytes, will continue without cache download 2024.10.22 01:42:53.581231 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5adfe44a157507a7745d91710e8967f6, offset: 0, size: 0 2024.10.22 01:42:53.593453 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5adfe44a157507a7745d91710e8967f6 for removal 2024.10.22 01:42:53.598676 [ 1476 ] {} CacheMetadata: Key 5adfe44a157507a7745d91710e8967f6 is removed from metadata 2024.10.22 01:42:53.600687 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c744696a962c5727d397f419b2a8572e, offset: 0, size: 0 2024.10.22 01:42:53.603371 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c744696a962c5727d397f419b2a8572e for removal 2024.10.22 01:42:53.610132 [ 1476 ] {} CacheMetadata: Key c744696a962c5727d397f419b2a8572e is removed from metadata 2024.10.22 01:42:53.653420 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68148 2024.10.22 01:42:53.843886 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1352 entries to flush up to offset 918101 2024.10.22 01:42:53.867958 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 9445 2024.10.22 01:42:53.974719 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:53.977607 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:54.018659 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_616_616_0 (state Temporary) 2024.10.22 01:42:54.180474 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_616_616_0 to 202410_629_629_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:54.269278 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9445 2024.10.22 01:42:54.723367 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9f8851726d92909e3198615ef371a854) : [0, 136]: Setting download as failed: File segment: [0, 136], key: 9f8851726d92909e3198615ef371a854, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:54.728395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9f8851726d92909e3198615ef371a854): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:42:54.859312 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:54.863789 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:54.994520 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_590_590_0 (state Temporary) 2024.10.22 01:42:55.081800 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_590_590_0 to 202410_597_597_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:55.219598 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 918101 2024.10.22 01:42:55.317322 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(1259eab84f22c693f60df9e067fc4d44) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1259eab84f22c693f60df9e067fc4d44, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 01:42:55.320571 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 1259eab84f22c693f60df9e067fc4d44, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 01:42:55.326292 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:55.503210 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_946_474, 202410_947_947_0] 2024.10.22 01:42:55.507525 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_946_474, 202410_947_947_0] 2024.10.22 01:42:55.520774 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_946_474, 202410_947_947_0] 2024.10.22 01:42:56.004963 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d24f708dc8a368ee4806c07df639d905) : [0, 57241]: Setting download as failed: File segment: [0, 57241], key: d24f708dc8a368ee4806c07df639d905, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:56.007068 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d24f708dc8a368ee4806c07df639d905): No space left in cache to reserve 57242 bytes, will continue without cache download 2024.10.22 01:42:56.092972 [ 1232 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:42:56.166499 [ 1232 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:42:56.242868 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d24f708dc8a368ee4806c07df639d905, offset: 0, size: 0 2024.10.22 01:42:56.245764 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d24f708dc8a368ee4806c07df639d905 for removal 2024.10.22 01:42:56.248109 [ 1476 ] {} CacheMetadata: Key d24f708dc8a368ee4806c07df639d905 is removed from metadata 2024.10.22 01:42:56.252058 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9f8851726d92909e3198615ef371a854, offset: 0, size: 0 2024.10.22 01:42:56.256415 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9f8851726d92909e3198615ef371a854 for removal 2024.10.22 01:42:56.259603 [ 1476 ] {} CacheMetadata: Key 9f8851726d92909e3198615ef371a854 is removed from metadata 2024.10.22 01:42:57.268439 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 98 entries to flush up to offset 69957 2024.10.22 01:42:57.408358 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:57.413321 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:57.606596 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_605_605_0 (state Temporary) 2024.10.22 01:42:57.753634 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 11.42 MiB, will set to 1.68 GiB (RSS), difference: -3.49 MiB 2024.10.22 01:42:57.750767 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_547_8, 202410_548_548_0, 202410_549_549_0, 202410_550_550_0, 202410_551_551_0, 202410_552_552_0] 2024.10.22 01:42:57.768692 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 68197 2024.10.22 01:42:57.786971 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7b340b4758af36dab988da7009f6f23d) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 7b340b4758af36dab988da7009f6f23d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:42:57.788491 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7b340b4758af36dab988da7009f6f23d): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:42:57.772075 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_547_8, 202410_548_548_0, 202410_549_549_0, 202410_550_550_0, 202410_551_551_0, 202410_552_552_0] 2024.10.22 01:42:57.851396 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_605_605_0 to 202410_611_611_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:57.907457 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_606_20 to 202410_611_611_0 2024.10.22 01:42:57.942123 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_547_8, 202410_548_548_0, 202410_549_549_0, 202410_550_550_0, 202410_551_551_0, 202410_552_552_0] 2024.10.22 01:42:58.025225 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 69957 2024.10.22 01:42:58.031661 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 516 2024.10.22 01:42:58.038954 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:58.066933 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:58.067872 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:58.069147 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:58.093823 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_606_20 to 202410_611_611_0 into Compact with storage Full 2024.10.22 01:42:58.368506 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:58.424444 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:58.439187 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_606_20, total 10225 rows starting from the beginning of the part 2024.10.22 01:42:58.478783 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_607_607_0, total 85 rows starting from the beginning of the part 2024.10.22 01:42:58.480800 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1041_1041_0 (state Temporary) 2024.10.22 01:42:58.482628 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:58.528322 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_608_608_0, total 91 rows starting from the beginning of the part 2024.10.22 01:42:58.557163 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_498_498_0 (state Temporary) 2024.10.22 01:42:58.580626 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1041_1041_0 to 202410_1054_1054_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:58.626725 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_609_609_0, total 154 rows starting from the beginning of the part 2024.10.22 01:42:58.674447 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_610_610_0, total 179 rows starting from the beginning of the part 2024.10.22 01:42:58.709183 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_611_611_0, total 98 rows starting from the beginning of the part 2024.10.22 01:42:58.757298 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_498_498_0 to 202410_508_508_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:42:58.763341 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68197 2024.10.22 01:42:58.791286 [ 1219 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_503_103 to 202410_508_508_0 2024.10.22 01:42:58.807587 [ 1219 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:42:58.812876 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:42:58.936308 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 516 2024.10.22 01:42:59.062199 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_503_103 to 202410_508_508_0 into Compact with storage Full 2024.10.22 01:42:59.208875 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(cc4db1d2a1591be8f3e7dc8034a39d80) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cc4db1d2a1591be8f3e7dc8034a39d80, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:42:59.214800 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 31050, file segment info: File segment: [0, 33554431], key: cc4db1d2a1591be8f3e7dc8034a39d80, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:42:59.219083 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:42:59.290244 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_947_475, 202410_948_948_0] 2024.10.22 01:42:59.294821 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_947_475, 202410_948_948_0] 2024.10.22 01:42:59.315673 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_947_475, 202410_948_948_0] 2024.10.22 01:42:59.732736 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:42:59.781817 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_503_103, total 547 rows starting from the beginning of the part 2024.10.22 01:42:59.890363 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_592_116/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:43:00.083956 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_504_504_0, total 1 rows starting from the beginning of the part 2024.10.22 01:43:00.283433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(52e45685cd7a990d9490fb2d2d304016) : [0, 20731]: Setting download as failed: File segment: [0, 20731], key: 52e45685cd7a990d9490fb2d2d304016, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:00.290767 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(52e45685cd7a990d9490fb2d2d304016): No space left in cache to reserve 20732 bytes, will continue without cache download 2024.10.22 01:43:00.314652 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 52e45685cd7a990d9490fb2d2d304016, offset: 0, size: 0 2024.10.22 01:43:00.316870 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 52e45685cd7a990d9490fb2d2d304016 for removal 2024.10.22 01:43:00.319295 [ 1476 ] {} CacheMetadata: Key 52e45685cd7a990d9490fb2d2d304016 is removed from metadata 2024.10.22 01:43:00.324220 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7b340b4758af36dab988da7009f6f23d, offset: 0, size: 0 2024.10.22 01:43:00.331073 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7b340b4758af36dab988da7009f6f23d for removal 2024.10.22 01:43:00.336924 [ 1476 ] {} CacheMetadata: Key 7b340b4758af36dab988da7009f6f23d is removed from metadata 2024.10.22 01:43:00.412884 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_505_505_0, total 1 rows starting from the beginning of the part 2024.10.22 01:43:00.668202 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:43:00.671364 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:43:00.866673 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_506_506_0, total 1 rows starting from the beginning of the part 2024.10.22 01:43:00.932083 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_124_124_0 to 202410_127_127_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:01.026230 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_124_23/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:43:01.073175 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(11ac2ceff08607d4f3796da186944bd1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 11ac2ceff08607d4f3796da186944bd1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:43:01.084134 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} MergeTask::MergeProjectionsStage: Merge sorted 10832 rows, containing 13 columns (13 merged, 0 gathered) in 2.995663424 sec., 3615.893532370344 rows/sec., 1.57 MiB/sec. 2024.10.22 01:43:01.085440 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 229, file segment info: File segment: [0, 33554431], key: 11ac2ceff08607d4f3796da186944bd1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:43:01.089367 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:01.089146 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_507_507_0, total 1 rows starting from the beginning of the part 2024.10.22 01:43:01.375233 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} MergedBlockOutputStream: filled checksums 202410_507_611_21 (state Temporary) 2024.10.22 01:43:01.415967 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_508_508_0, total 1 rows starting from the beginning of the part 2024.10.22 01:43:01.505218 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 10.49 MiB, will set to 1.68 GiB (RSS), difference: 4.70 MiB 2024.10.22 01:43:01.626455 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_611_21 to 202410_507_611_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:01.631325 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_611_21} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_606_20, 202410_611_611_0] -> 202410_507_611_21 2024.10.22 01:43:01.776134 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 9458 2024.10.22 01:43:01.787650 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.36 MiB. 2024.10.22 01:43:02.226497 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 958 entries to flush up to offset 919059 2024.10.22 01:43:02.268109 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:02.272718 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:02.388074 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fd6d25efd02c589c5ec01c5279dcdbc1) : [0, 150]: Setting download as failed: File segment: [0, 150], key: fd6d25efd02c589c5ec01c5279dcdbc1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:02.389691 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fd6d25efd02c589c5ec01c5279dcdbc1): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 01:43:02.630526 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_617_617_0 (state Temporary) 2024.10.22 01:43:02.681447 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_617_617_0 to 202410_630_630_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:02.766768 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 68275 2024.10.22 01:43:02.891406 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9458 2024.10.22 01:43:02.916646 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:02.921452 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:02.906222 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 4971 2024.10.22 01:43:03.280234 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:03.284170 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:03.289842 [ 1171 ] {} DatabaseCatalog: Execution took 333 ms. 2024.10.22 01:43:03.296104 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_591_591_0 (state Temporary) 2024.10.22 01:43:03.389435 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_591_591_0 to 202410_598_598_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:03.426406 [ 1194 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_593_593_0 to 202410_598_598_0 2024.10.22 01:43:03.433200 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 919059 2024.10.22 01:43:03.439984 [ 1194 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:03.443453 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:03.559748 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_593_593_0 to 202410_598_598_0 into Compact with storage Full 2024.10.22 01:43:03.599035 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:43:03.604903 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(cd6ed14af0d662ee5b49c477bacd60e3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cd6ed14af0d662ee5b49c477bacd60e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:43:03.612206 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_593_593_0, total 1434 rows starting from the beginning of the part 2024.10.22 01:43:03.646849 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1042_1042_0 (state Temporary) 2024.10.22 01:43:03.652195 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_594_594_0, total 1646 rows starting from the beginning of the part 2024.10.22 01:43:03.675842 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_595_595_0, total 770 rows starting from the beginning of the part 2024.10.22 01:43:03.692886 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_596_596_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:43:03.711516 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 13496, file segment info: File segment: [0, 33554431], key: cd6ed14af0d662ee5b49c477bacd60e3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:43:03.715126 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:03.748505 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} MergeTask::MergeProjectionsStage: Merge sorted 552 rows, containing 32 columns (32 merged, 0 gathered) in 4.701829619 sec., 117.40110653295028 rows/sec., 10.78 KiB/sec. 2024.10.22 01:43:03.815145 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1042_1042_0 to 202410_1055_1055_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:03.818223 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} MergedBlockOutputStream: filled checksums 202410_1_508_104 (state Temporary) 2024.10.22 01:43:03.842960 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_597_597_0, total 1352 rows starting from the beginning of the part 2024.10.22 01:43:03.938973 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_598_598_0, total 958 rows starting from the beginning of the part 2024.10.22 01:43:03.983683 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68275 2024.10.22 01:43:03.990758 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_948_476, 202410_949_949_0] 2024.10.22 01:43:03.997143 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_948_476, 202410_949_949_0] 2024.10.22 01:43:04.112850 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_948_476, 202410_949_949_0] 2024.10.22 01:43:04.290439 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5e73ec02162fd546c740c3f0ee8ad208) : [0, 16567]: Setting download as failed: File segment: [0, 16567], key: 5e73ec02162fd546c740c3f0ee8ad208, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:04.295838 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5e73ec02162fd546c740c3f0ee8ad208): No space left in cache to reserve 16568 bytes, will continue without cache download 2024.10.22 01:43:04.321512 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5e73ec02162fd546c740c3f0ee8ad208, offset: 0, size: 0 2024.10.22 01:43:04.323985 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5e73ec02162fd546c740c3f0ee8ad208 for removal 2024.10.22 01:43:04.329239 [ 1476 ] {} CacheMetadata: Key 5e73ec02162fd546c740c3f0ee8ad208 is removed from metadata 2024.10.22 01:43:04.331350 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fd6d25efd02c589c5ec01c5279dcdbc1, offset: 0, size: 0 2024.10.22 01:43:04.343431 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fd6d25efd02c589c5ec01c5279dcdbc1 for removal 2024.10.22 01:43:04.347486 [ 1476 ] {} CacheMetadata: Key fd6d25efd02c589c5ec01c5279dcdbc1 is removed from metadata 2024.10.22 01:43:04.869548 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} MergeTask::MergeProjectionsStage: Merge sorted 7700 rows, containing 4 columns (4 merged, 0 gathered) in 1.315223748 sec., 5854.517158551185 rows/sec., 91.48 KiB/sec. 2024.10.22 01:43:04.912572 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} MergedBlockOutputStream: filled checksums 202410_593_598_1 (state Temporary) 2024.10.22 01:43:05.069558 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_508_104 to 202410_1_508_104 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:05.072459 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_508_104} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_503_103, 202410_508_508_0] -> 202410_1_508_104 2024.10.22 01:43:05.076690 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_593_598_1 to 202410_593_598_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:05.079917 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_593_598_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_593_593_0, 202410_598_598_0] -> 202410_593_598_1 2024.10.22 01:43:05.122943 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:43:05.313728 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:43:05.320936 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(909b754bf66ea0121f6f27f20f2892dd) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 909b754bf66ea0121f6f27f20f2892dd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:43:05.327215 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 573, file segment info: File segment: [0, 33554431], key: 909b754bf66ea0121f6f27f20f2892dd, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 01:43:05.330465 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:05.373316 [ 1727 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mkg/iluwosumfsvhoiksypotnedgkhvos, size 13496 2024.10.22 01:43:05.753599 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 117 entries to flush up to offset 70074 2024.10.22 01:43:05.817094 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column YCLID 2024.10.22 01:43:05.835533 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column YCLID 2024.10.22 01:43:05.972824 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column YCLID 2024.10.22 01:43:05.973044 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:05.974858 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:06.018417 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column YCLID 2024.10.22 01:43:06.019968 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_606_606_0 (state Temporary) 2024.10.22 01:43:06.071790 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column YCLID 2024.10.22 01:43:06.084816 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_606_606_0 to 202410_612_612_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:06.187788 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 70074 2024.10.22 01:43:06.271178 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(866e300d391cde58647e50ba9343bb84) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 866e300d391cde58647e50ba9343bb84, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:06.272344 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(866e300d391cde58647e50ba9343bb84): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:43:06.310791 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column YCLID 2024.10.22 01:43:06.345036 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column YCLID 2024.10.22 01:43:06.384059 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column YCLID 2024.10.22 01:43:06.439170 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 517 2024.10.22 01:43:06.556665 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 15.21 MiB, will set to 1.68 GiB (RSS), difference: 16.01 MiB 2024.10.22 01:43:06.601206 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column YCLID 2024.10.22 01:43:06.714288 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tma/wlupgdgdndcldnnuyhjxozucjismi (LogSeriesLimiter: on interval from 2024-10-22 01:42:51 to 2024-10-22 01:43:06 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:43:06.807889 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:06.812498 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:06.884033 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_499_499_0 (state Temporary) 2024.10.22 01:43:07.136914 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fc5e497e4339055918462ad104da8ee1) : [0, 45758]: Setting download as failed: File segment: [0, 45758], key: fc5e497e4339055918462ad104da8ee1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:07.139629 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fc5e497e4339055918462ad104da8ee1): No space left in cache to reserve 45759 bytes, will continue without cache download 2024.10.22 01:43:07.159344 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_499_499_0 to 202410_509_509_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:07.169426 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fc5e497e4339055918462ad104da8ee1, offset: 0, size: 0 2024.10.22 01:43:07.173067 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fc5e497e4339055918462ad104da8ee1 for removal 2024.10.22 01:43:07.175245 [ 1476 ] {} CacheMetadata: Key fc5e497e4339055918462ad104da8ee1 is removed from metadata 2024.10.22 01:43:07.176278 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 866e300d391cde58647e50ba9343bb84, offset: 0, size: 0 2024.10.22 01:43:07.180137 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 866e300d391cde58647e50ba9343bb84 for removal 2024.10.22 01:43:07.182413 [ 1476 ] {} CacheMetadata: Key 866e300d391cde58647e50ba9343bb84 is removed from metadata 2024.10.22 01:43:07.523651 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 517 2024.10.22 01:43:07.992957 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 68361 2024.10.22 01:43:08.159511 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a53667928351443bca0d7177bcf6d374) : [0, 151]: Setting download as failed: File segment: [0, 151], key: a53667928351443bca0d7177bcf6d374, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:08.162364 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a53667928351443bca0d7177bcf6d374): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:43:08.585047 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:08.589068 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:08.668481 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 16.52 MiB, will set to 1.68 GiB (RSS), difference: -13.62 MiB 2024.10.22 01:43:08.830297 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1043_1043_0 (state Temporary) 2024.10.22 01:43:08.883105 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1043_1043_0 to 202410_1056_1056_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:08.955469 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68361 2024.10.22 01:43:09.269412 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(54928591fc6040fdca700be35f6f7828) : [0, 43580]: Setting download as failed: File segment: [0, 43580], key: 54928591fc6040fdca700be35f6f7828, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:09.271109 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(54928591fc6040fdca700be35f6f7828): No space left in cache to reserve 43581 bytes, will continue without cache download 2024.10.22 01:43:09.309680 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 54928591fc6040fdca700be35f6f7828, offset: 0, size: 0 2024.10.22 01:43:09.311297 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 54928591fc6040fdca700be35f6f7828 for removal 2024.10.22 01:43:09.315838 [ 1476 ] {} CacheMetadata: Key 54928591fc6040fdca700be35f6f7828 is removed from metadata 2024.10.22 01:43:09.417717 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a53667928351443bca0d7177bcf6d374, offset: 0, size: 0 2024.10.22 01:43:09.419882 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a53667928351443bca0d7177bcf6d374 for removal 2024.10.22 01:43:09.423687 [ 1476 ] {} CacheMetadata: Key a53667928351443bca0d7177bcf6d374 is removed from metadata 2024.10.22 01:43:10.268871 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(3c0a59f3f81267b981d7a4484fc453e5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3c0a59f3f81267b981d7a4484fc453e5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:43:10.279129 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 975, file segment info: File segment: [0, 33554431], key: 3c0a59f3f81267b981d7a4484fc453e5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:43:10.281489 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_566_113, 202410_567_567_0, 202410_568_568_0, 202410_569_569_0, 202410_570_570_0, 202410_571_571_0] 2024.10.22 01:43:10.282571 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:10.283180 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_566_113, 202410_567_567_0, 202410_568_568_0, 202410_569_569_0, 202410_570_570_0, 202410_571_571_0] 2024.10.22 01:43:10.334531 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_566_113, 202410_567_567_0, 202410_568_568_0, 202410_569_569_0, 202410_570_570_0, 202410_571_571_0] 2024.10.22 01:43:10.395503 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 9478 2024.10.22 01:43:10.438220 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 867 entries to flush up to offset 919926 2024.10.22 01:43:10.465149 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:43:10.482280 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:43:10.514970 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:43:10.565605 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:43:10.580565 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 5009 2024.10.22 01:43:10.604551 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:43:10.668455 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:43:10.668807 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:10.667748 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:43:10.670228 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:43:10.672661 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:10.735094 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:43:10.768658 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_592_116/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:43:10.779159 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 01:43:10.785406 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_618_618_0 (state Temporary) 2024.10.22 01:43:10.838832 [ 1698 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rmt/kjkhzmqgcmiivnrbwsvmdspnvamxm, size 9572285 2024.10.22 01:43:10.879821 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_618_618_0 to 202410_631_631_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:10.914137 [ 1199 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_626_125 to 202410_631_631_0 2024.10.22 01:43:10.954796 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9478 2024.10.22 01:43:10.959368 [ 1199 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:10.963307 [ 1199 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:10.976916 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_626_125 to 202410_631_631_0 into Compact with storage Full 2024.10.22 01:43:11.130991 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:43:11.148526 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_626_125, total 9516 rows starting from the beginning of the part 2024.10.22 01:43:11.169546 [ 1197 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:43:11.179496 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_627_627_0, total 12 rows starting from the beginning of the part 2024.10.22 01:43:11.199870 [ 1197 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:43:11.265503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cda10a7f3e0e68c0f5a5fb6e715489fb) : [0, 151]: Setting download as failed: File segment: [0, 151], key: cda10a7f3e0e68c0f5a5fb6e715489fb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:11.266909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cda10a7f3e0e68c0f5a5fb6e715489fb): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:43:11.323477 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_628_628_0, total 8 rows starting from the beginning of the part 2024.10.22 01:43:11.372522 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 7 old parts to remove. Parts: [202410_1_949_477, 202410_950_950_0, 202410_951_951_0, 202410_952_952_0, 202410_953_953_0, 202410_954_954_0, 202410_955_955_0] 2024.10.22 01:43:11.374371 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 7 parts from filesystem (serially): Parts: [202410_1_949_477, 202410_950_950_0, 202410_951_951_0, 202410_952_952_0, 202410_953_953_0, 202410_954_954_0, 202410_955_955_0] 2024.10.22 01:43:11.439378 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:11.443956 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:11.478133 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_629_629_0, total 15 rows starting from the beginning of the part 2024.10.22 01:43:11.520062 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 7 parts from memory: Parts: [202410_1_949_477, 202410_950_950_0, 202410_951_951_0, 202410_952_952_0, 202410_953_953_0, 202410_954_954_0, 202410_955_955_0] 2024.10.22 01:43:11.527271 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_630_630_0, total 13 rows starting from the beginning of the part 2024.10.22 01:43:11.559547 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_631_631_0, total 20 rows starting from the beginning of the part 2024.10.22 01:43:11.687381 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_592_592_0 (state Temporary) 2024.10.22 01:43:11.857452 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_592_592_0 to 202410_599_599_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:12.052192 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_631_126/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:43:12.058518 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 919926 2024.10.22 01:43:12.163354 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_592_116} MergeTask::MergeProjectionsStage: Merge sorted 924949 rows, containing 4 columns (4 merged, 0 gathered) in 25.294528587 sec., 36567.15707583391 rows/sec., 548.25 KiB/sec. 2024.10.22 01:43:12.227697 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_592_116} MergedBlockOutputStream: filled checksums 202410_1_592_116 (state Temporary) 2024.10.22 01:43:12.563036 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 15.64 MiB, will set to 1.66 GiB (RSS), difference: 3.11 MiB 2024.10.22 01:43:12.701115 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(87403f100236f79f4968c1973eba8cce) : [0, 47571]: Setting download as failed: File segment: [0, 47571], key: 87403f100236f79f4968c1973eba8cce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:12.707081 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(87403f100236f79f4968c1973eba8cce): No space left in cache to reserve 47572 bytes, will continue without cache download 2024.10.22 01:43:12.747452 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 87403f100236f79f4968c1973eba8cce, offset: 0, size: 0 2024.10.22 01:43:12.750929 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 87403f100236f79f4968c1973eba8cce for removal 2024.10.22 01:43:12.754075 [ 1476 ] {} CacheMetadata: Key 87403f100236f79f4968c1973eba8cce is removed from metadata 2024.10.22 01:43:12.756917 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cda10a7f3e0e68c0f5a5fb6e715489fb, offset: 0, size: 0 2024.10.22 01:43:12.759301 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cda10a7f3e0e68c0f5a5fb6e715489fb for removal 2024.10.22 01:43:12.761404 [ 1476 ] {} CacheMetadata: Key cda10a7f3e0e68c0f5a5fb6e715489fb is removed from metadata 2024.10.22 01:43:12.969043 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 68439 2024.10.22 01:43:13.338878 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:13.348517 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:13.421696 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1044_1044_0 (state Temporary) 2024.10.22 01:43:13.491602 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} MergeTask::MergeProjectionsStage: Merge sorted 9584 rows, containing 12 columns (12 merged, 0 gathered) in 2.517288363 sec., 3807.2714039714488 rows/sec., 605.72 KiB/sec. 2024.10.22 01:43:13.571712 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} MergedBlockOutputStream: filled checksums 202410_1_631_126 (state Temporary) 2024.10.22 01:43:13.792290 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1044_1044_0 to 202410_1057_1057_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:13.810278 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 85 entries to flush up to offset 70159 2024.10.22 01:43:13.854775 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(93f73699bd7f0aaff47d4820371cfe8d) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 93f73699bd7f0aaff47d4820371cfe8d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:13.856463 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(93f73699bd7f0aaff47d4820371cfe8d): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:43:13.892886 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68439 2024.10.22 01:43:13.928160 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1046_501 to 202410_1047_1052_1 2024.10.22 01:43:13.954806 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.55 MiB using storage policy from min volume index 0 2024.10.22 01:43:13.956094 [ 1239 ] {} DiskLocal: Reserved 2.55 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:14.068117 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1052_502} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1046_501 to 202410_1047_1052_1 into Wide with storage Full 2024.10.22 01:43:14.150830 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1052_502} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:43:14.167239 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_631_126 to 202410_1_631_126 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:14.172690 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_631_126} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_626_125, 202410_631_631_0] -> 202410_1_631_126 2024.10.22 01:43:14.179671 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1052_502} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1046_501, total 68534 rows starting from the beginning of the part 2024.10.22 01:43:14.206705 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.59 MiB. 2024.10.22 01:43:14.372989 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:14.374670 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:14.421567 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_592_116} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_592_116 to 202410_1_592_116 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:14.424906 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_592_116} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_586_115, 202410_587_592_1] -> 202410_1_592_116 2024.10.22 01:43:14.478958 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_607_607_0 (state Temporary) 2024.10.22 01:43:14.481528 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cea43ea8ae9337d066447684dbb4a465) : [0, 29345]: Setting download as failed: File segment: [0, 29345], key: cea43ea8ae9337d066447684dbb4a465, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:14.482931 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cea43ea8ae9337d066447684dbb4a465): No space left in cache to reserve 29346 bytes, will continue without cache download 2024.10.22 01:43:14.508579 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cea43ea8ae9337d066447684dbb4a465, offset: 0, size: 0 2024.10.22 01:43:14.512306 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cea43ea8ae9337d066447684dbb4a465 for removal 2024.10.22 01:43:14.515225 [ 1476 ] {} CacheMetadata: Key cea43ea8ae9337d066447684dbb4a465 is removed from metadata 2024.10.22 01:43:14.516579 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 93f73699bd7f0aaff47d4820371cfe8d, offset: 0, size: 0 2024.10.22 01:43:14.518003 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 93f73699bd7f0aaff47d4820371cfe8d for removal 2024.10.22 01:43:14.521827 [ 1476 ] {} CacheMetadata: Key 93f73699bd7f0aaff47d4820371cfe8d is removed from metadata 2024.10.22 01:43:14.514682 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 14.02 MiB, will set to 1.67 GiB (RSS), difference: 8.41 MiB 2024.10.22 01:43:14.527623 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_607_607_0 to 202410_613_613_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:14.561567 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:43:14.690265 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 70159 2024.10.22 01:43:14.697235 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1052_502} MergeTreeSequentialSource: Reading 2 marks from part 202410_1047_1052_1, total 367 rows starting from the beginning of the part 2024.10.22 01:43:15.029038 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 518 2024.10.22 01:43:15.179819 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:15.185840 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:15.257507 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_500_500_0 (state Temporary) 2024.10.22 01:43:15.356390 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_500_500_0 to 202410_510_510_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:15.427180 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e5fcf40dd993c6fb58ddbfd54f7e16a4) : [0, 153]: Setting download as failed: File segment: [0, 153], key: e5fcf40dd993c6fb58ddbfd54f7e16a4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:15.430723 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e5fcf40dd993c6fb58ddbfd54f7e16a4): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:43:15.462826 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 518 2024.10.22 01:43:16.538504 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4e5da299809026c7e878540ba17fbb4f) : [0, 963961]: Setting download as failed: File segment: [0, 963961], key: 4e5da299809026c7e878540ba17fbb4f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:16.542616 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4e5da299809026c7e878540ba17fbb4f): No space left in cache to reserve 963962 bytes, will continue without cache download 2024.10.22 01:43:16.573128 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 12.00 MiB, will set to 1.67 GiB (RSS), difference: -44.81 MiB 2024.10.22 01:43:16.676922 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4e5da299809026c7e878540ba17fbb4f, offset: 0, size: 0 2024.10.22 01:43:16.780060 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4e5da299809026c7e878540ba17fbb4f for removal 2024.10.22 01:43:16.782444 [ 1476 ] {} CacheMetadata: Key 4e5da299809026c7e878540ba17fbb4f is removed from metadata 2024.10.22 01:43:16.885938 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e5fcf40dd993c6fb58ddbfd54f7e16a4, offset: 0, size: 0 2024.10.22 01:43:16.887816 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e5fcf40dd993c6fb58ddbfd54f7e16a4 for removal 2024.10.22 01:43:16.891178 [ 1476 ] {} CacheMetadata: Key e5fcf40dd993c6fb58ddbfd54f7e16a4 is removed from metadata 2024.10.22 01:43:17.144497 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_124_23/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:43:17.749058 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(09875f530f923fe742f2e1c63c2eab04) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 09875f530f923fe742f2e1c63c2eab04, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:17.855434 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(09875f530f923fe742f2e1c63c2eab04): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:43:17.917558 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 68497 2024.10.22 01:43:18.236699 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:18.248248 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:18.260992 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} MergeTask::MergeProjectionsStage: Merge sorted 4912 rows, containing 679 columns (679 merged, 0 gathered) in 139.081586397 sec., 35.31739986038836 rows/sec., 47.45 KiB/sec. 2024.10.22 01:43:18.347304 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1045_1045_0 (state Temporary) 2024.10.22 01:43:18.401515 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1045_1045_0 to 202410_1058_1058_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:18.430562 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1053_1053_0 to 202410_1058_1058_0 2024.10.22 01:43:18.452109 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:18.455350 [ 1241 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:18.459263 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 9494 2024.10.22 01:43:18.482615 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 8.87 MiB, will set to 1.67 GiB (RSS), difference: 11.50 MiB 2024.10.22 01:43:18.482921 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1053_1053_0 to 202410_1058_1058_0 into Compact with storage Full 2024.10.22 01:43:18.558315 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68497 2024.10.22 01:43:18.564239 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:18.565763 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:18.590277 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_619_619_0 (state Temporary) 2024.10.22 01:43:18.641719 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_619_619_0 to 202410_632_632_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:18.658500 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:43:18.665404 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1053_1053_0, total 78 rows starting from the beginning of the part 2024.10.22 01:43:18.672988 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9494 2024.10.22 01:43:18.693208 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1054_1054_0, total 49 rows starting from the beginning of the part 2024.10.22 01:43:18.775535 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1052_502} MergeTask::MergeProjectionsStage: Merge sorted 68901 rows, containing 13 columns (13 merged, 0 gathered) in 4.713184381 sec., 14618.778819211233 rows/sec., 2.41 MiB/sec. 2024.10.22 01:43:18.797770 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c72ccd46dbcadcee99c63fbf62afd797) : [0, 19108]: Setting download as failed: File segment: [0, 19108], key: c72ccd46dbcadcee99c63fbf62afd797, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:18.801324 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c72ccd46dbcadcee99c63fbf62afd797): No space left in cache to reserve 19109 bytes, will continue without cache download 2024.10.22 01:43:18.930709 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c72ccd46dbcadcee99c63fbf62afd797, offset: 0, size: 0 2024.10.22 01:43:18.943392 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c72ccd46dbcadcee99c63fbf62afd797 for removal 2024.10.22 01:43:18.950037 [ 1476 ] {} CacheMetadata: Key c72ccd46dbcadcee99c63fbf62afd797 is removed from metadata 2024.10.22 01:43:18.951117 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 09875f530f923fe742f2e1c63c2eab04, offset: 0, size: 0 2024.10.22 01:43:18.955303 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 09875f530f923fe742f2e1c63c2eab04 for removal 2024.10.22 01:43:18.961342 [ 1476 ] {} CacheMetadata: Key 09875f530f923fe742f2e1c63c2eab04 is removed from metadata 2024.10.22 01:43:18.987733 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1055_1055_0, total 78 rows starting from the beginning of the part 2024.10.22 01:43:19.059655 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1056_1056_0, total 86 rows starting from the beginning of the part 2024.10.22 01:43:19.061164 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 921468 2024.10.22 01:43:19.071610 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1052_502} MergedBlockOutputStream: filled checksums 202410_1_1052_502 (state Temporary) 2024.10.22 01:43:19.137420 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1057_1057_0, total 78 rows starting from the beginning of the part 2024.10.22 01:43:19.181120 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1058_1058_0, total 58 rows starting from the beginning of the part 2024.10.22 01:43:20.054800 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:20.059899 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:20.111674 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1052_502} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1052_502 to 202410_1_1052_502 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:20.117034 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1052_502} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1046_501, 202410_1047_1052_1] -> 202410_1_1052_502 2024.10.22 01:43:20.148217 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.55 MiB. 2024.10.22 01:43:20.344185 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0cafed0d73c7de2a2fa26e8ce7b941f9) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 0cafed0d73c7de2a2fa26e8ce7b941f9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:20.346949 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0cafed0d73c7de2a2fa26e8ce7b941f9): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:43:20.400093 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_593_593_0 (state Temporary) 2024.10.22 01:43:20.471828 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_593_593_0 to 202410_600_600_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:20.507914 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 921468 2024.10.22 01:43:20.589551 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 7.06 MiB, will set to 1.66 GiB (RSS), difference: 26.47 MiB 2024.10.22 01:43:20.668999 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:43:20.670881 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:43:20.784010 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} MergeTask::MergeProjectionsStage: Merge sorted 427 rows, containing 13 columns (13 merged, 0 gathered) in 2.315138721 sec., 184.438192030049 rows/sec., 33.18 KiB/sec. 2024.10.22 01:43:20.824851 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} MergedBlockOutputStream: filled checksums 202410_1053_1058_1 (state Temporary) 2024.10.22 01:43:21.155626 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(937fe0d2c1e5d49616338c0487936603) : [0, 15836]: Setting download as failed: File segment: [0, 15836], key: 937fe0d2c1e5d49616338c0487936603, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:21.158079 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(937fe0d2c1e5d49616338c0487936603): No space left in cache to reserve 15837 bytes, will continue without cache download 2024.10.22 01:43:21.166206 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 937fe0d2c1e5d49616338c0487936603, offset: 0, size: 0 2024.10.22 01:43:21.167315 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 937fe0d2c1e5d49616338c0487936603 for removal 2024.10.22 01:43:21.170354 [ 1476 ] {} CacheMetadata: Key 937fe0d2c1e5d49616338c0487936603 is removed from metadata 2024.10.22 01:43:21.171486 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0cafed0d73c7de2a2fa26e8ce7b941f9, offset: 0, size: 0 2024.10.22 01:43:21.173604 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0cafed0d73c7de2a2fa26e8ce7b941f9 for removal 2024.10.22 01:43:21.177638 [ 1476 ] {} CacheMetadata: Key 0cafed0d73c7de2a2fa26e8ce7b941f9 is removed from metadata 2024.10.22 01:43:21.308415 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} MergedBlockOutputStream: filled checksums 202410_1_124_23 (state Temporary) 2024.10.22 01:43:21.465248 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1053_1058_1 to 202410_1053_1058_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:21.468176 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1053_1058_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1053_1053_0, 202410_1058_1058_0] -> 202410_1053_1058_1 2024.10.22 01:43:21.500461 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:43:22.215046 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 166 entries to flush up to offset 70325 2024.10.22 01:43:22.305000 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d9ca3635a89c1036595283282883280e) : [0, 136]: Setting download as failed: File segment: [0, 136], key: d9ca3635a89c1036595283282883280e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:22.308439 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d9ca3635a89c1036595283282883280e): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:43:22.384683 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:22.387606 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:22.557053 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_608_608_0 (state Temporary) 2024.10.22 01:43:22.569145 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 68566 2024.10.22 01:43:22.640463 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_608_608_0 to 202410_614_614_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:22.652466 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 4.81 MiB, will set to 1.65 GiB (RSS), difference: 7.32 MiB 2024.10.22 01:43:22.674841 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 70325 2024.10.22 01:43:22.853647 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:22.857165 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:22.912054 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1046_1046_0 (state Temporary) 2024.10.22 01:43:22.961299 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1046_1046_0 to 202410_1059_1059_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:22.964858 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 519 2024.10.22 01:43:23.112044 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68566 2024.10.22 01:43:23.221055 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:23.225468 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:23.292901 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_501_501_0 (state Temporary) 2024.10.22 01:43:23.385080 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(60662530b94a6e48a4c58cc10ad51515) : [0, 27238]: Setting download as failed: File segment: [0, 27238], key: 60662530b94a6e48a4c58cc10ad51515, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:23.387550 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(60662530b94a6e48a4c58cc10ad51515): No space left in cache to reserve 27239 bytes, will continue without cache download 2024.10.22 01:43:23.416542 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 60662530b94a6e48a4c58cc10ad51515, offset: 0, size: 0 2024.10.22 01:43:23.419609 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 60662530b94a6e48a4c58cc10ad51515 for removal 2024.10.22 01:43:23.425062 [ 1476 ] {} CacheMetadata: Key 60662530b94a6e48a4c58cc10ad51515 is removed from metadata 2024.10.22 01:43:23.486711 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_501_501_0 to 202410_511_511_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:23.531630 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d9ca3635a89c1036595283282883280e, offset: 0, size: 0 2024.10.22 01:43:23.535508 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d9ca3635a89c1036595283282883280e for removal 2024.10.22 01:43:23.540512 [ 1476 ] {} CacheMetadata: Key d9ca3635a89c1036595283282883280e is removed from metadata 2024.10.22 01:43:23.618948 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 519 2024.10.22 01:43:23.690531 [ 1150 ] {} DatabaseCatalog: Execution took 234 ms. 2024.10.22 01:43:24.621366 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 3.38 MiB, will set to 1.65 GiB (RSS), difference: -3.51 MiB 2024.10.22 01:43:24.690452 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(58d0e8ce10b7cd8c0d199087e7ce1286) : [0, 136]: Setting download as failed: File segment: [0, 136], key: 58d0e8ce10b7cd8c0d199087e7ce1286, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:24.695503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(58d0e8ce10b7cd8c0d199087e7ce1286): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:43:26.056683 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(3281b005ddd72389454e819899d12795) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3281b005ddd72389454e819899d12795, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:43:26.061017 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 57, file segment info: File segment: [0, 33554431], key: 3281b005ddd72389454e819899d12795, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:43:26.063528 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:26.174463 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 9509 2024.10.22 01:43:26.206251 [ 1190 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:43:26.268451 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:26.270803 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:26.292820 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_620_620_0 (state Temporary) 2024.10.22 01:43:26.348708 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_620_620_0 to 202410_633_633_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:26.355527 [ 1190 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:43:26.490159 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9509 2024.10.22 01:43:26.702786 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 3.55 MiB, will set to 1.65 GiB (RSS), difference: 1.08 MiB 2024.10.22 01:43:26.801714 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f4e64901222e51ac180dc7ef33cf2883) : [0, 32878]: Setting download as failed: File segment: [0, 32878], key: f4e64901222e51ac180dc7ef33cf2883, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:26.802691 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:26.804358 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f4e64901222e51ac180dc7ef33cf2883): No space left in cache to reserve 32879 bytes, will continue without cache download 2024.10.22 01:43:26.807378 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:26.821482 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f4e64901222e51ac180dc7ef33cf2883, offset: 0, size: 0 2024.10.22 01:43:26.824596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f4e64901222e51ac180dc7ef33cf2883 for removal 2024.10.22 01:43:26.836797 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 58d0e8ce10b7cd8c0d199087e7ce1286, offset: 0, size: 0 2024.10.22 01:43:26.839913 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 58d0e8ce10b7cd8c0d199087e7ce1286 for removal 2024.10.22 01:43:26.930759 [ 1476 ] {} CacheMetadata: Key f4e64901222e51ac180dc7ef33cf2883 is removed from metadata 2024.10.22 01:43:26.934907 [ 1476 ] {} CacheMetadata: Key 58d0e8ce10b7cd8c0d199087e7ce1286 is removed from metadata 2024.10.22 01:43:27.118753 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 68615 2024.10.22 01:43:27.526243 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1443 entries to flush up to offset 922911 2024.10.22 01:43:27.662443 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:27.664506 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:27.727424 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(7dea6b897bb42021fe9f41adb3b6513c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7dea6b897bb42021fe9f41adb3b6513c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:43:27.730301 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 57, file segment info: File segment: [0, 33554431], key: 7dea6b897bb42021fe9f41adb3b6513c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:43:27.732895 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:27.806733 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1047_1047_0 (state Temporary) 2024.10.22 01:43:27.866170 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1047_1047_0 to 202410_1060_1060_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:27.916126 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68615 2024.10.22 01:43:27.984491 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:27.988183 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:28.149717 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_594_594_0 (state Temporary) 2024.10.22 01:43:28.235663 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_594_594_0 to 202410_601_601_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:28.373541 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 922911 2024.10.22 01:43:28.657142 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2c623e7141181fd388f15760afdddfbe) : [0, 167]: Setting download as failed: File segment: [0, 167], key: 2c623e7141181fd388f15760afdddfbe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:28.669925 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2c623e7141181fd388f15760afdddfbe): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 01:43:29.084856 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:43:29.303063 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_458_94, 202410_459_459_0, 202410_460_460_0, 202410_461_461_0, 202410_462_462_0, 202410_463_463_0] 2024.10.22 01:43:29.305172 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_458_94, 202410_459_459_0, 202410_460_460_0, 202410_461_461_0, 202410_462_462_0, 202410_463_463_0] 2024.10.22 01:43:29.326697 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_125_125_0 (state Temporary) 2024.10.22 01:43:29.328637 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(c294c377c0047efd7e8ae07b13cbab37) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c294c377c0047efd7e8ae07b13cbab37, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:43:29.335138 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 452, file segment info: File segment: [0, 33554431], key: c294c377c0047efd7e8ae07b13cbab37, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:43:29.338688 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:29.443012 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_458_94, 202410_459_459_0, 202410_460_460_0, 202410_461_461_0, 202410_462_462_0, 202410_463_463_0] 2024.10.22 01:43:29.785267 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:43:29.818507 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:43:29.860658 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:43:29.864082 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_539_539_0, 202410_540_540_0, 202410_541_541_0, 202410_542_542_0, 202410_543_543_0, 202410_544_544_0] 2024.10.22 01:43:29.867289 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_539_539_0, 202410_540_540_0, 202410_541_541_0, 202410_542_542_0, 202410_543_543_0, 202410_544_544_0] 2024.10.22 01:43:29.873564 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:43:29.906218 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:43:29.935782 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:43:29.963773 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:43:30.006425 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_539_539_0, 202410_540_540_0, 202410_541_541_0, 202410_542_542_0, 202410_543_543_0, 202410_544_544_0] 2024.10.22 01:43:30.085452 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column IsEvent 2024.10.22 01:43:30.206525 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 51 entries to flush up to offset 70376 2024.10.22 01:43:30.234516 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rzr/pihqdwqnjqonclmqqnrwnyrksridb (LogSeriesLimiter: on interval from 2024-10-22 01:43:06 to 2024-10-22 01:43:30 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:43:30.279990 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(3ade1a7f0513bcc06d476aa3a6ebb91e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3ade1a7f0513bcc06d476aa3a6ebb91e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:43:30.283188 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 17202, file segment info: File segment: [0, 33554431], key: 3ade1a7f0513bcc06d476aa3a6ebb91e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:43:30.290133 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:30.432708 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:30.437206 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:30.476149 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_609_609_0 (state Temporary) 2024.10.22 01:43:30.514310 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_609_609_0 to 202410_615_615_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:30.573567 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 70376 2024.10.22 01:43:30.601540 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 4.82 MiB, will set to 1.65 GiB (RSS), difference: -15.19 MiB 2024.10.22 01:43:30.669597 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:43:30.772491 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:43:30.863854 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(44fac6d6587364387c025298f81f13f5) : [0, 107868]: Setting download as failed: File segment: [0, 107868], key: 44fac6d6587364387c025298f81f13f5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:30.866714 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(44fac6d6587364387c025298f81f13f5): No space left in cache to reserve 107869 bytes, will continue without cache download 2024.10.22 01:43:30.881769 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 44fac6d6587364387c025298f81f13f5, offset: 0, size: 0 2024.10.22 01:43:30.883284 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 44fac6d6587364387c025298f81f13f5 for removal 2024.10.22 01:43:30.886013 [ 1476 ] {} CacheMetadata: Key 44fac6d6587364387c025298f81f13f5 is removed from metadata 2024.10.22 01:43:30.887943 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2c623e7141181fd388f15760afdddfbe, offset: 0, size: 0 2024.10.22 01:43:30.889190 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2c623e7141181fd388f15760afdddfbe for removal 2024.10.22 01:43:30.904849 [ 1476 ] {} CacheMetadata: Key 2c623e7141181fd388f15760afdddfbe is removed from metadata 2024.10.22 01:43:30.922217 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_125_125_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:43:31.121821 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 520 2024.10.22 01:43:31.378377 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:31.380064 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:31.852285 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_502_502_0 (state Temporary) 2024.10.22 01:43:31.858212 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(54e237e7a32da1b5ad6730a5d387e597) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 54e237e7a32da1b5ad6730a5d387e597, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:43:31.865442 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 542, file segment info: File segment: [0, 33554431], key: 54e237e7a32da1b5ad6730a5d387e597, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:43:31.870979 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:31.919346 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 68680 2024.10.22 01:43:32.145836 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_502_502_0 to 202410_512_512_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:32.204498 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:32.208518 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:32.253077 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 520 2024.10.22 01:43:32.268938 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1048_1048_0 (state Temporary) 2024.10.22 01:43:32.327442 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1048_1048_0 to 202410_1061_1061_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:32.480137 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68680 2024.10.22 01:43:32.958380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2952caf362f6349dc1a66bac6dfce5a1) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 2952caf362f6349dc1a66bac6dfce5a1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:32.960312 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2952caf362f6349dc1a66bac6dfce5a1): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:43:33.786115 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(359a9fa0bae72c3968e01eb229b1cafe) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 359a9fa0bae72c3968e01eb229b1cafe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:43:33.790785 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 5569, file segment info: File segment: [0, 33554431], key: 359a9fa0bae72c3968e01eb229b1cafe, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:43:33.796706 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:33.994444 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 9515 2024.10.22 01:43:34.289621 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:34.292670 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:34.339205 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_621_621_0 (state Temporary) 2024.10.22 01:43:34.486173 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_621_621_0 to 202410_634_634_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:34.536949 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(64e90a93cf5f37158ccd2412ac79b5e8) : [0, 139801]: Setting download as failed: File segment: [0, 139801], key: 64e90a93cf5f37158ccd2412ac79b5e8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:34.540284 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(64e90a93cf5f37158ccd2412ac79b5e8): No space left in cache to reserve 139802 bytes, will continue without cache download 2024.10.22 01:43:34.550884 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9515 2024.10.22 01:43:34.569271 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 64e90a93cf5f37158ccd2412ac79b5e8, offset: 0, size: 0 2024.10.22 01:43:34.571480 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 64e90a93cf5f37158ccd2412ac79b5e8 for removal 2024.10.22 01:43:34.578430 [ 1476 ] {} CacheMetadata: Key 64e90a93cf5f37158ccd2412ac79b5e8 is removed from metadata 2024.10.22 01:43:34.581019 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2952caf362f6349dc1a66bac6dfce5a1, offset: 0, size: 0 2024.10.22 01:43:34.585229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2952caf362f6349dc1a66bac6dfce5a1 for removal 2024.10.22 01:43:34.589671 [ 1476 ] {} CacheMetadata: Key 2952caf362f6349dc1a66bac6dfce5a1 is removed from metadata 2024.10.22 01:43:34.629532 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 4.66 MiB, will set to 1.65 GiB (RSS), difference: 2.66 MiB 2024.10.22 01:43:35.378504 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 923681 2024.10.22 01:43:35.818269 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(8a69e637afbbb47ca5633b9920b70228) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8a69e637afbbb47ca5633b9920b70228, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:43:35.825733 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 568, file segment info: File segment: [0, 33554431], key: 8a69e637afbbb47ca5633b9920b70228, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:43:35.828915 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:35.868713 [ 1692 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tma/wlupgdgdndcldnnuyhjxozucjismi, size 5569 2024.10.22 01:43:35.961087 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 247 ms. 2024.10.22 01:43:36.081213 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ShareService 2024.10.22 01:43:36.108654 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ShareService 2024.10.22 01:43:36.129235 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ShareService 2024.10.22 01:43:36.168654 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ShareService 2024.10.22 01:43:36.308934 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ShareService 2024.10.22 01:43:36.353172 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ShareService 2024.10.22 01:43:36.377907 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:36.379067 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ShareService 2024.10.22 01:43:36.384222 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:36.441185 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ShareService 2024.10.22 01:43:36.566872 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_595_595_0 (state Temporary) 2024.10.22 01:43:36.574105 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ShareService 2024.10.22 01:43:36.747355 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4fe15f9a8ca3197d8d1a23096bcb0dd0) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 4fe15f9a8ca3197d8d1a23096bcb0dd0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:36.855421 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 68724 2024.10.22 01:43:36.858792 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4fe15f9a8ca3197d8d1a23096bcb0dd0): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:43:37.028191 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/okp/nwjglxnyuxqauatdjnozezdyeedyf (LogSeriesLimiter: on interval from 2024-10-22 01:43:30 to 2024-10-22 01:43:37 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:43:37.074048 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_595_595_0 to 202410_602_602_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:37.187633 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:37.207954 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:37.278397 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1049_1049_0 (state Temporary) 2024.10.22 01:43:37.318362 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1049_1049_0 to 202410_1062_1062_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:37.314708 [ 1233 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_592_116 to 202410_593_598_1 2024.10.22 01:43:37.383570 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68724 2024.10.22 01:43:37.418943 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 923681 2024.10.22 01:43:37.454726 [ 1233 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:37.457447 [ 1233 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:37.468828 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_598_117} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_592_116 to 202410_593_598_1 into Wide with storage Full 2024.10.22 01:43:37.500976 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_598_117} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:43:37.507681 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_598_117} MergeTreeSequentialSource: Reading 114 marks from part 202410_1_592_116, total 924949 rows starting from the beginning of the part 2024.10.22 01:43:37.539878 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_598_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_593_598_1, total 7700 rows starting from the beginning of the part 2024.10.22 01:43:37.563301 [ 1233 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 378 ms. 2024.10.22 01:43:37.669970 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bf35715cb79536a6ee2a7da832ec39bf) : [0, 949411]: Setting download as failed: File segment: [0, 949411], key: bf35715cb79536a6ee2a7da832ec39bf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:37.671348 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bf35715cb79536a6ee2a7da832ec39bf): No space left in cache to reserve 949412 bytes, will continue without cache download 2024.10.22 01:43:37.698830 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bf35715cb79536a6ee2a7da832ec39bf, offset: 0, size: 0 2024.10.22 01:43:37.702914 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bf35715cb79536a6ee2a7da832ec39bf for removal 2024.10.22 01:43:37.706410 [ 1476 ] {} CacheMetadata: Key bf35715cb79536a6ee2a7da832ec39bf is removed from metadata 2024.10.22 01:43:37.708748 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4fe15f9a8ca3197d8d1a23096bcb0dd0, offset: 0, size: 0 2024.10.22 01:43:37.709004 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_125_125_0 to 202410_128_128_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:37.711200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4fe15f9a8ca3197d8d1a23096bcb0dd0 for removal 2024.10.22 01:43:37.714689 [ 1476 ] {} CacheMetadata: Key 4fe15f9a8ca3197d8d1a23096bcb0dd0 is removed from metadata 2024.10.22 01:43:38.179538 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 74 entries to flush up to offset 70450 2024.10.22 01:43:38.308716 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:38.367625 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fb73c53edb8a3d31de33ff0cdbcff6c6) : [0, 140]: Setting download as failed: File segment: [0, 140], key: fb73c53edb8a3d31de33ff0cdbcff6c6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:38.412086 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fb73c53edb8a3d31de33ff0cdbcff6c6): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:43:38.413504 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:38.458007 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_610_610_0 (state Temporary) 2024.10.22 01:43:38.521737 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_610_610_0 to 202410_616_616_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:38.568437 [ 1245 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_611_21 to 202410_616_616_0 2024.10.22 01:43:38.576400 [ 1245 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:38.579101 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_598_117/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:43:38.581174 [ 1245 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:38.631132 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_611_21 to 202410_616_616_0 into Compact with storage Full 2024.10.22 01:43:38.672104 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 70450 2024.10.22 01:43:39.041772 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:43:39.081445 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 6.37 MiB, will set to 1.65 GiB (RSS), difference: -15.12 MiB 2024.10.22 01:43:39.112122 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_955_478, 202410_956_956_0, 202410_957_957_0, 202410_958_958_0, 202410_959_959_0, 202410_960_960_0] 2024.10.22 01:43:39.115515 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_955_478, 202410_956_956_0, 202410_957_957_0, 202410_958_958_0, 202410_959_959_0, 202410_960_960_0] 2024.10.22 01:43:39.150406 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_955_478, 202410_956_956_0, 202410_957_957_0, 202410_958_958_0, 202410_959_959_0, 202410_960_960_0] 2024.10.22 01:43:39.162556 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_611_21, total 10832 rows starting from the beginning of the part 2024.10.22 01:43:39.257639 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_612_612_0, total 117 rows starting from the beginning of the part 2024.10.22 01:43:39.267715 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_532_106, 202410_533_538_1] 2024.10.22 01:43:39.319589 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_613_613_0, total 85 rows starting from the beginning of the part 2024.10.22 01:43:39.270865 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_532_106, 202410_533_538_1] 2024.10.22 01:43:39.357522 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_532_106, 202410_533_538_1] 2024.10.22 01:43:39.427512 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_614_614_0, total 166 rows starting from the beginning of the part 2024.10.22 01:43:39.710408 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_615_615_0, total 51 rows starting from the beginning of the part 2024.10.22 01:43:39.714690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f511edef8326c37e2e430fdee6f6ebc6) : [0, 34377]: Setting download as failed: File segment: [0, 34377], key: f511edef8326c37e2e430fdee6f6ebc6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:39.715964 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f511edef8326c37e2e430fdee6f6ebc6): No space left in cache to reserve 34378 bytes, will continue without cache download 2024.10.22 01:43:39.791409 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_616_616_0, total 74 rows starting from the beginning of the part 2024.10.22 01:43:39.846571 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f511edef8326c37e2e430fdee6f6ebc6, offset: 0, size: 0 2024.10.22 01:43:39.848653 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f511edef8326c37e2e430fdee6f6ebc6 for removal 2024.10.22 01:43:39.858395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fb73c53edb8a3d31de33ff0cdbcff6c6, offset: 0, size: 0 2024.10.22 01:43:39.860392 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fb73c53edb8a3d31de33ff0cdbcff6c6 for removal 2024.10.22 01:43:39.886765 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_124_23 to 202410_1_124_23 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:39.891797 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_124_23} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 7 parts: [202410_1_114_22, 202410_124_124_0] -> 202410_1_124_23 2024.10.22 01:43:39.954513 [ 1476 ] {} CacheMetadata: Key f511edef8326c37e2e430fdee6f6ebc6 is removed from metadata 2024.10.22 01:43:39.958289 [ 1476 ] {} CacheMetadata: Key fb73c53edb8a3d31de33ff0cdbcff6c6 is removed from metadata 2024.10.22 01:43:40.417526 [ 1242 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 01:43:40.669835 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:43:40.673221 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:43:40.693722 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5009 2024.10.22 01:43:40.805472 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 49.54 MiB. 2024.10.22 01:43:41.000991 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(68a4025f1a80d1f81ae3120cd4b0ce78) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 68a4025f1a80d1f81ae3120cd4b0ce78, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:41.005705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(68a4025f1a80d1f81ae3120cd4b0ce78): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:43:41.358690 [ 1218 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:43:41.375416 [ 1218 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:43:41.464020 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 13.10 MiB, will set to 1.66 GiB (RSS), difference: 9.24 MiB 2024.10.22 01:43:41.490622 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 68800 2024.10.22 01:43:41.881076 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_552_9, 202410_553_553_0, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0, 202410_557_557_0] 2024.10.22 01:43:41.883720 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_552_9, 202410_553_553_0, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0, 202410_557_557_0] 2024.10.22 01:43:41.892703 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:41.897429 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:43:41.911482 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_552_9, 202410_553_553_0, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0, 202410_557_557_0] 2024.10.22 01:43:42.024469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a7d10d34dc9cf956976bfa84d8a49410) : [0, 26744]: Setting download as failed: File segment: [0, 26744], key: a7d10d34dc9cf956976bfa84d8a49410, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:42.026568 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a7d10d34dc9cf956976bfa84d8a49410): No space left in cache to reserve 26745 bytes, will continue without cache download 2024.10.22 01:43:42.053059 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 9528 2024.10.22 01:43:42.150571 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a7d10d34dc9cf956976bfa84d8a49410, offset: 0, size: 0 2024.10.22 01:43:42.149820 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} MergeTask::MergeProjectionsStage: Merge sorted 11325 rows, containing 13 columns (13 merged, 0 gathered) in 3.531087169 sec., 3207.2275358773504 rows/sec., 1.39 MiB/sec. 2024.10.22 01:43:42.152557 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a7d10d34dc9cf956976bfa84d8a49410 for removal 2024.10.22 01:43:42.154951 [ 1476 ] {} CacheMetadata: Key a7d10d34dc9cf956976bfa84d8a49410 is removed from metadata 2024.10.22 01:43:42.156885 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 68a4025f1a80d1f81ae3120cd4b0ce78, offset: 0, size: 0 2024.10.22 01:43:42.260768 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 68a4025f1a80d1f81ae3120cd4b0ce78 for removal 2024.10.22 01:43:42.242258 [ 1203 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 01:43:42.264056 [ 1476 ] {} CacheMetadata: Key 68a4025f1a80d1f81ae3120cd4b0ce78 is removed from metadata 2024.10.22 01:43:42.268186 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:42.270997 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:42.301093 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1050_1050_0 (state Temporary) 2024.10.22 01:43:42.319603 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_622_622_0 (state Temporary) 2024.10.22 01:43:42.438544 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} MergedBlockOutputStream: filled checksums 202410_507_616_22 (state Temporary) 2024.10.22 01:43:42.461627 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1050_1050_0 to 202410_1063_1063_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:42.462661 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_622_622_0 to 202410_635_635_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:42.480858 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1052_502 to 202410_1053_1058_1 2024.10.22 01:43:42.486163 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.57 MiB using storage policy from min volume index 0 2024.10.22 01:43:42.488922 [ 1152 ] {} DiskLocal: Reserved 2.57 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:42.497281 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9528 2024.10.22 01:43:42.502775 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68800 2024.10.22 01:43:42.507823 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1058_503} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1052_502 to 202410_1053_1058_1 into Wide with storage Full 2024.10.22 01:43:42.577674 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1058_503} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:43:42.586221 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1058_503} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1052_502, total 68901 rows starting from the beginning of the part 2024.10.22 01:43:42.716737 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_616_22 to 202410_507_616_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:42.720450 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_616_22} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_611_21, 202410_616_616_0] -> 202410_507_616_22 2024.10.22 01:43:42.780981 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1058_503} MergeTreeSequentialSource: Reading 2 marks from part 202410_1053_1058_1, total 427 rows starting from the beginning of the part 2024.10.22 01:43:42.965829 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.58 MiB. 2024.10.22 01:43:43.142715 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(50a89de6b6a12e3737c0f992cb9b4f12) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 50a89de6b6a12e3737c0f992cb9b4f12, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:43.144435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(50a89de6b6a12e3737c0f992cb9b4f12): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:43:43.530618 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 16.53 MiB, will set to 1.66 GiB (RSS), difference: -27.78 MiB 2024.10.22 01:43:44.251366 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(08bff31fbbcc5dc815e8a28bd23ef8f2) : [0, 23284]: Setting download as failed: File segment: [0, 23284], key: 08bff31fbbcc5dc815e8a28bd23ef8f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:44.254514 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(08bff31fbbcc5dc815e8a28bd23ef8f2): No space left in cache to reserve 23285 bytes, will continue without cache download 2024.10.22 01:43:44.275100 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 08bff31fbbcc5dc815e8a28bd23ef8f2, offset: 0, size: 0 2024.10.22 01:43:44.277640 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 08bff31fbbcc5dc815e8a28bd23ef8f2 for removal 2024.10.22 01:43:44.280270 [ 1476 ] {} CacheMetadata: Key 08bff31fbbcc5dc815e8a28bd23ef8f2 is removed from metadata 2024.10.22 01:43:44.280457 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 50a89de6b6a12e3737c0f992cb9b4f12, offset: 0, size: 0 2024.10.22 01:43:44.282304 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 50a89de6b6a12e3737c0f992cb9b4f12 for removal 2024.10.22 01:43:44.285584 [ 1476 ] {} CacheMetadata: Key 50a89de6b6a12e3737c0f992cb9b4f12 is removed from metadata 2024.10.22 01:43:44.422624 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 924836 2024.10.22 01:43:44.680845 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:43:44.908131 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(67a8bca42a9744b9edd64dd4f2ae4dc1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 67a8bca42a9744b9edd64dd4f2ae4dc1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:43:44.957171 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 57, file segment info: File segment: [0, 33554431], key: 67a8bca42a9744b9edd64dd4f2ae4dc1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:43:44.961386 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:45.796785 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d95ce1dcd43f50226f94a9ce3c97a7c7) : [0, 151]: Setting download as failed: File segment: [0, 151], key: d95ce1dcd43f50226f94a9ce3c97a7c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:45.799304 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d95ce1dcd43f50226f94a9ce3c97a7c7): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:43:45.857032 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:45.861085 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:46.009694 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_596_596_0 (state Temporary) 2024.10.22 01:43:46.107588 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_596_596_0 to 202410_603_603_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:46.140314 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 924836 2024.10.22 01:43:46.191306 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 186 entries to flush up to offset 70636 2024.10.22 01:43:46.532954 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:46.545280 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(8ba95b09686c2258d0b345807fb1b61b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8ba95b09686c2258d0b345807fb1b61b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:43:46.606318 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 13.35 MiB, will set to 1.67 GiB (RSS), difference: 4.77 MiB 2024.10.22 01:43:46.619540 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 68861 2024.10.22 01:43:46.622595 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 57, file segment info: File segment: [0, 33554431], key: 8ba95b09686c2258d0b345807fb1b61b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:43:46.623802 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:46.627579 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:46.697702 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_611_611_0 (state Temporary) 2024.10.22 01:43:46.788010 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_611_611_0 to 202410_617_617_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:46.944202 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 70636 2024.10.22 01:43:46.960906 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1058_503} MergeTask::MergeProjectionsStage: Merge sorted 69328 rows, containing 13 columns (13 merged, 0 gathered) in 4.464426895 sec., 15528.98090405398 rows/sec., 2.57 MiB/sec. 2024.10.22 01:43:46.996061 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:47.001602 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:47.086297 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_1058_503/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:43:47.180480 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1051_1051_0 (state Temporary) 2024.10.22 01:43:47.256583 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 521 2024.10.22 01:43:47.269313 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1058_503} MergedBlockOutputStream: filled checksums 202410_1_1058_503 (state Temporary) 2024.10.22 01:43:47.341021 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1051_1051_0 to 202410_1064_1064_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:47.364014 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d594e2ff190a96ba134a1ff10b296495) : [0, 35838]: Setting download as failed: File segment: [0, 35838], key: d594e2ff190a96ba134a1ff10b296495, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:47.365634 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d594e2ff190a96ba134a1ff10b296495): No space left in cache to reserve 35839 bytes, will continue without cache download 2024.10.22 01:43:47.371305 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1059_1059_0 to 202410_1064_1064_0 2024.10.22 01:43:47.400710 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:47.407584 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:47.484628 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d594e2ff190a96ba134a1ff10b296495, offset: 0, size: 0 2024.10.22 01:43:47.490366 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d594e2ff190a96ba134a1ff10b296495 for removal 2024.10.22 01:43:47.497015 [ 1476 ] {} CacheMetadata: Key d594e2ff190a96ba134a1ff10b296495 is removed from metadata 2024.10.22 01:43:47.531389 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68861 2024.10.22 01:43:47.558477 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1059_1059_0 to 202410_1064_1064_0 into Compact with storage Full 2024.10.22 01:43:47.600238 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:47.605165 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:47.606644 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d95ce1dcd43f50226f94a9ce3c97a7c7, offset: 0, size: 0 2024.10.22 01:43:47.608047 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d95ce1dcd43f50226f94a9ce3c97a7c7 for removal 2024.10.22 01:43:47.678219 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_503_503_0 (state Temporary) 2024.10.22 01:43:47.713004 [ 1476 ] {} CacheMetadata: Key d95ce1dcd43f50226f94a9ce3c97a7c7 is removed from metadata 2024.10.22 01:43:47.742737 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:43:47.757517 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1059_1059_0, total 69 rows starting from the beginning of the part 2024.10.22 01:43:47.794942 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1058_503} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1058_503 to 202410_1_1058_503 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:47.805828 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1058_503} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1052_502, 202410_1053_1058_1] -> 202410_1_1058_503 2024.10.22 01:43:47.817163 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1060_1060_0, total 49 rows starting from the beginning of the part 2024.10.22 01:43:47.847349 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.60 MiB. 2024.10.22 01:43:47.863616 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_503_503_0 to 202410_513_513_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:47.892218 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1061_1061_0, total 65 rows starting from the beginning of the part 2024.10.22 01:43:47.897199 [ 1240 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_508_104 to 202410_513_513_0 2024.10.22 01:43:47.934295 [ 1240 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:47.943571 [ 1240 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:48.019297 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 521 2024.10.22 01:43:48.066874 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1062_1062_0, total 44 rows starting from the beginning of the part 2024.10.22 01:43:48.074536 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_508_104 to 202410_513_513_0 into Compact with storage Full 2024.10.22 01:43:48.166583 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1063_1063_0, total 76 rows starting from the beginning of the part 2024.10.22 01:43:48.285465 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1064_1064_0, total 61 rows starting from the beginning of the part 2024.10.22 01:43:48.389086 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:43:48.410612 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_508_104, total 552 rows starting from the beginning of the part 2024.10.22 01:43:48.433592 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(6bf87f0fb161cdda328b78497b875657) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6bf87f0fb161cdda328b78497b875657, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:43:48.440545 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 452, file segment info: File segment: [0, 33554431], key: 6bf87f0fb161cdda328b78497b875657, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:43:48.445791 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:43:48.327857 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 5047 2024.10.22 01:43:48.467944 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_509_509_0, total 1 rows starting from the beginning of the part 2024.10.22 01:43:48.610813 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_510_510_0, total 1 rows starting from the beginning of the part 2024.10.22 01:43:48.642771 [ 1689 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rzr/pihqdwqnjqonclmqqnrwnyrksridb, size 57 2024.10.22 01:43:48.778260 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:43:48.790311 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:43:48.814387 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:43:48.854470 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:43:48.872959 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_511_511_0, total 1 rows starting from the beginning of the part 2024.10.22 01:43:48.884258 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:43:48.964565 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_512_512_0, total 1 rows starting from the beginning of the part 2024.10.22 01:43:48.966872 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:43:49.015659 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:43:49.030472 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_513_513_0, total 1 rows starting from the beginning of the part 2024.10.22 01:43:49.066256 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column IsParameter 2024.10.22 01:43:49.162499 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uri/jdkorukihivsixzvzskikmkfyoeol (LogSeriesLimiter: on interval from 2024-10-22 01:43:37 to 2024-10-22 01:43:49 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:43:49.272370 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a2dda8b0971ed88997c76906eddceb48) : [0, 140]: Setting download as failed: File segment: [0, 140], key: a2dda8b0971ed88997c76906eddceb48, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:49.273413 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a2dda8b0971ed88997c76906eddceb48): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:43:49.568011 [ 1211 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 01:43:49.711207 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_571_114, 202410_572_572_0, 202410_573_573_0, 202410_574_574_0, 202410_575_575_0, 202410_576_576_0] 2024.10.22 01:43:49.724166 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_571_114, 202410_572_572_0, 202410_573_573_0, 202410_574_574_0, 202410_575_575_0, 202410_576_576_0] 2024.10.22 01:43:49.757035 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_571_114, 202410_572_572_0, 202410_573_573_0, 202410_574_574_0, 202410_575_575_0, 202410_576_576_0] 2024.10.22 01:43:49.773803 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} MergeTask::MergeProjectionsStage: Merge sorted 364 rows, containing 13 columns (13 merged, 0 gathered) in 2.231741823 sec., 163.10130331773595 rows/sec., 29.28 KiB/sec. 2024.10.22 01:43:49.911351 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} MergedBlockOutputStream: filled checksums 202410_1059_1064_1 (state Temporary) 2024.10.22 01:43:50.104239 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 9544 2024.10.22 01:43:50.168359 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1059_1064_1 to 202410_1059_1064_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:50.172710 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1059_1064_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1059_1059_0, 202410_1064_1064_0] -> 202410_1059_1064_1 2024.10.22 01:43:50.201024 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:43:50.227158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(28fbb56b39888e9f46ed8a06b19daae6) : [0, 24588]: Setting download as failed: File segment: [0, 24588], key: 28fbb56b39888e9f46ed8a06b19daae6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:50.229299 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(28fbb56b39888e9f46ed8a06b19daae6): No space left in cache to reserve 24589 bytes, will continue without cache download 2024.10.22 01:43:50.345526 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 28fbb56b39888e9f46ed8a06b19daae6, offset: 0, size: 0 2024.10.22 01:43:50.420618 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:50.466402 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 11.46 MiB, will set to 1.65 GiB (RSS), difference: 21.79 MiB 2024.10.22 01:43:50.496436 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_598_117/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:43:50.555800 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 28fbb56b39888e9f46ed8a06b19daae6 for removal 2024.10.22 01:43:50.557552 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:50.559150 [ 1476 ] {} CacheMetadata: Key 28fbb56b39888e9f46ed8a06b19daae6 is removed from metadata 2024.10.22 01:43:50.559878 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a2dda8b0971ed88997c76906eddceb48, offset: 0, size: 0 2024.10.22 01:43:50.562134 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a2dda8b0971ed88997c76906eddceb48 for removal 2024.10.22 01:43:50.563722 [ 1476 ] {} CacheMetadata: Key a2dda8b0971ed88997c76906eddceb48 is removed from metadata 2024.10.22 01:43:50.673796 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:43:50.675085 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:43:50.782751 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} MergeTask::MergeProjectionsStage: Merge sorted 557 rows, containing 32 columns (32 merged, 0 gathered) in 2.716163523 sec., 205.06865484475472 rows/sec., 18.83 KiB/sec. 2024.10.22 01:43:50.911998 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_623_623_0 (state Temporary) 2024.10.22 01:43:50.974573 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_623_623_0 to 202410_636_636_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:51.029319 [ 1186 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_631_126 to 202410_636_636_0 2024.10.22 01:43:51.048205 [ 1186 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:51.054403 [ 1186 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:51.079589 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} MergedBlockOutputStream: filled checksums 202410_1_513_105 (state Temporary) 2024.10.22 01:43:51.098599 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_631_126 to 202410_636_636_0 into Compact with storage Full 2024.10.22 01:43:51.138499 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9544 2024.10.22 01:43:51.185155 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:43:51.219614 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_631_126, total 9584 rows starting from the beginning of the part 2024.10.22 01:43:51.286327 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_632_632_0, total 16 rows starting from the beginning of the part 2024.10.22 01:43:51.384677 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_633_633_0, total 15 rows starting from the beginning of the part 2024.10.22 01:43:51.554464 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 110 entries to flush up to offset 68971 2024.10.22 01:43:51.562651 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_634_634_0, total 6 rows starting from the beginning of the part 2024.10.22 01:43:51.605143 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_635_635_0, total 13 rows starting from the beginning of the part 2024.10.22 01:43:51.773947 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_636_636_0, total 16 rows starting from the beginning of the part 2024.10.22 01:43:51.846886 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_513_105 to 202410_1_513_105 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:51.854168 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_513_105} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_508_104, 202410_513_513_0] -> 202410_1_513_105 2024.10.22 01:43:51.884786 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:43:51.918894 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8ef2a33bea90992d09ff0b49e8c85ccf) : [0, 138]: Setting download as failed: File segment: [0, 138], key: 8ef2a33bea90992d09ff0b49e8c85ccf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:51.921525 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8ef2a33bea90992d09ff0b49e8c85ccf): No space left in cache to reserve 139 bytes, will continue without cache download 2024.10.22 01:43:52.161071 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:52.165168 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:52.338284 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1052_1052_0 (state Temporary) 2024.10.22 01:43:52.380370 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1052_1052_0 to 202410_1065_1065_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:52.460855 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 68971 2024.10.22 01:43:52.670765 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 8.82 MiB, will set to 1.65 GiB (RSS), difference: -5.60 MiB 2024.10.22 01:43:52.883054 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(49efbb40073186f20d0c4003417d2e5e) : [0, 72481]: Setting download as failed: File segment: [0, 72481], key: 49efbb40073186f20d0c4003417d2e5e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:52.885377 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(49efbb40073186f20d0c4003417d2e5e): No space left in cache to reserve 72482 bytes, will continue without cache download 2024.10.22 01:43:52.923625 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 49efbb40073186f20d0c4003417d2e5e, offset: 0, size: 0 2024.10.22 01:43:52.927070 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 49efbb40073186f20d0c4003417d2e5e for removal 2024.10.22 01:43:52.931527 [ 1476 ] {} CacheMetadata: Key 49efbb40073186f20d0c4003417d2e5e is removed from metadata 2024.10.22 01:43:52.936286 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8ef2a33bea90992d09ff0b49e8c85ccf, offset: 0, size: 0 2024.10.22 01:43:52.938962 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8ef2a33bea90992d09ff0b49e8c85ccf for removal 2024.10.22 01:43:52.943004 [ 1476 ] {} CacheMetadata: Key 8ef2a33bea90992d09ff0b49e8c85ccf is removed from metadata 2024.10.22 01:43:53.068643 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} MergeTask::MergeProjectionsStage: Merge sorted 9650 rows, containing 12 columns (12 merged, 0 gathered) in 1.994156587 sec., 4839.138542534122 rows/sec., 769.84 KiB/sec. 2024.10.22 01:43:53.091301 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} MergedBlockOutputStream: filled checksums 202410_1_636_127 (state Temporary) 2024.10.22 01:43:53.148833 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1226 entries to flush up to offset 926062 2024.10.22 01:43:53.688325 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_636_127 to 202410_1_636_127 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:53.716241 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_636_127} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_631_126, 202410_636_636_0] -> 202410_1_636_127 2024.10.22 01:43:53.769176 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.59 MiB. 2024.10.22 01:43:53.856730 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:53.860456 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:54.088357 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_597_597_0 (state Temporary) 2024.10.22 01:43:54.102566 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(18cb933004d95098fa2c464b6c65f5b6) : [0, 136]: Setting download as failed: File segment: [0, 136], key: 18cb933004d95098fa2c464b6c65f5b6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:54.104704 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(18cb933004d95098fa2c464b6c65f5b6): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:43:54.272898 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_597_597_0 to 202410_604_604_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:54.307638 [ 1199 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_599_599_0 to 202410_604_604_0 2024.10.22 01:43:54.348519 [ 1199 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:54.355235 [ 1199 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:54.368665 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_599_599_0 to 202410_604_604_0 into Compact with storage Full 2024.10.22 01:43:54.387633 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:43:54.392199 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_599_599_0, total 867 rows starting from the beginning of the part 2024.10.22 01:43:54.412998 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_600_600_0, total 1542 rows starting from the beginning of the part 2024.10.22 01:43:54.422506 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 926062 2024.10.22 01:43:54.423776 [ 1168 ] {} DatabaseCatalog: Execution took 430 ms. 2024.10.22 01:43:54.442886 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_601_601_0, total 1443 rows starting from the beginning of the part 2024.10.22 01:43:54.468984 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_602_602_0, total 770 rows starting from the beginning of the part 2024.10.22 01:43:54.484743 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 70720 2024.10.22 01:43:54.487252 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_603_603_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:43:54.513676 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_604_604_0, total 1226 rows starting from the beginning of the part 2024.10.22 01:43:54.561313 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 10.48 MiB, will set to 1.66 GiB (RSS), difference: 9.53 MiB 2024.10.22 01:43:54.740687 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:54.745281 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:54.787651 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_612_612_0 (state Temporary) 2024.10.22 01:43:54.939976 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_612_612_0 to 202410_618_618_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:54.975697 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d4022f3359b2fda3de370108bf984552) : [0, 15908]: Setting download as failed: File segment: [0, 15908], key: d4022f3359b2fda3de370108bf984552, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:54.977812 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d4022f3359b2fda3de370108bf984552): No space left in cache to reserve 15909 bytes, will continue without cache download 2024.10.22 01:43:55.003089 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 70720 2024.10.22 01:43:55.014738 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d4022f3359b2fda3de370108bf984552, offset: 0, size: 0 2024.10.22 01:43:55.017270 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d4022f3359b2fda3de370108bf984552 for removal 2024.10.22 01:43:55.025566 [ 1476 ] {} CacheMetadata: Key d4022f3359b2fda3de370108bf984552 is removed from metadata 2024.10.22 01:43:55.030433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 18cb933004d95098fa2c464b6c65f5b6, offset: 0, size: 0 2024.10.22 01:43:55.031666 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 18cb933004d95098fa2c464b6c65f5b6 for removal 2024.10.22 01:43:55.048934 [ 1476 ] {} CacheMetadata: Key 18cb933004d95098fa2c464b6c65f5b6 is removed from metadata 2024.10.22 01:43:55.524777 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 522 2024.10.22 01:43:55.709681 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:55.713695 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:55.826620 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_504_504_0 (state Temporary) 2024.10.22 01:43:55.891114 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_504_504_0 to 202410_514_514_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:56.204037 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} MergeTask::MergeProjectionsStage: Merge sorted 7003 rows, containing 4 columns (4 merged, 0 gathered) in 1.839168939 sec., 3807.698059433136 rows/sec., 59.50 KiB/sec. 2024.10.22 01:43:56.279469 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 522 2024.10.22 01:43:56.377971 [ 1229 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:43:56.425762 [ 1229 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:43:56.472075 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 69043 2024.10.22 01:43:56.584820 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 11.20 MiB, will set to 1.66 GiB (RSS), difference: -4.77 MiB 2024.10.22 01:43:56.611082 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} MergedBlockOutputStream: filled checksums 202410_599_604_1 (state Temporary) 2024.10.22 01:43:56.636684 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:56.640655 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:56.688465 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1053_1053_0 (state Temporary) 2024.10.22 01:43:56.716101 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(489d304b1a6094e49b88031a9245a68a) : [0, 136]: Setting download as failed: File segment: [0, 136], key: 489d304b1a6094e49b88031a9245a68a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:56.717269 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(489d304b1a6094e49b88031a9245a68a): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:43:56.738277 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1053_1053_0 to 202410_1066_1066_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:56.789616 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_599_604_1 to 202410_599_604_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:56.792794 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_599_604_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_599_599_0, 202410_604_604_0] -> 202410_599_604_1 2024.10.22 01:43:56.814561 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69043 2024.10.22 01:43:56.859115 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:43:57.641004 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(07004b4d0a17e0003403137d3e6a66bc) : [0, 4671]: Setting download as failed: File segment: [0, 4671], key: 07004b4d0a17e0003403137d3e6a66bc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:57.644736 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(07004b4d0a17e0003403137d3e6a66bc): No space left in cache to reserve 4672 bytes, will continue without cache download 2024.10.22 01:43:57.659033 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 07004b4d0a17e0003403137d3e6a66bc, offset: 0, size: 0 2024.10.22 01:43:57.661073 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 07004b4d0a17e0003403137d3e6a66bc for removal 2024.10.22 01:43:57.665218 [ 1476 ] {} CacheMetadata: Key 07004b4d0a17e0003403137d3e6a66bc is removed from metadata 2024.10.22 01:43:57.665398 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 489d304b1a6094e49b88031a9245a68a, offset: 0, size: 0 2024.10.22 01:43:57.671666 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 489d304b1a6094e49b88031a9245a68a for removal 2024.10.22 01:43:57.678794 [ 1476 ] {} CacheMetadata: Key 489d304b1a6094e49b88031a9245a68a is removed from metadata 2024.10.22 01:43:58.580465 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 15.36 MiB, will set to 1.66 GiB (RSS), difference: 11.38 MiB 2024.10.22 01:43:58.639947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1348fd98520de6d85af013e4121ffce6) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 1348fd98520de6d85af013e4121ffce6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:58.642274 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1348fd98520de6d85af013e4121ffce6): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:43:58.647625 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 34 entries to flush up to offset 9578 2024.10.22 01:43:58.882635 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:43:58.887241 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:43:59.035026 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_624_624_0 (state Temporary) 2024.10.22 01:43:59.080414 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_624_624_0 to 202410_637_637_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:43:59.240866 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9578 2024.10.22 01:43:59.741299 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dc1644497ff295e6a9265af7d436428f) : [0, 11151]: Setting download as failed: File segment: [0, 11151], key: dc1644497ff295e6a9265af7d436428f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:43:59.744690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dc1644497ff295e6a9265af7d436428f): No space left in cache to reserve 11152 bytes, will continue without cache download 2024.10.22 01:43:59.759261 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dc1644497ff295e6a9265af7d436428f, offset: 0, size: 0 2024.10.22 01:43:59.761265 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dc1644497ff295e6a9265af7d436428f for removal 2024.10.22 01:43:59.766606 [ 1476 ] {} CacheMetadata: Key dc1644497ff295e6a9265af7d436428f is removed from metadata 2024.10.22 01:43:59.767073 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1348fd98520de6d85af013e4121ffce6, offset: 0, size: 0 2024.10.22 01:43:59.768756 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1348fd98520de6d85af013e4121ffce6 for removal 2024.10.22 01:43:59.770788 [ 1476 ] {} CacheMetadata: Key 1348fd98520de6d85af013e4121ffce6 is removed from metadata 2024.10.22 01:44:00.604432 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 12.52 MiB, will set to 1.66 GiB (RSS), difference: -1.81 MiB 2024.10.22 01:44:00.774660 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:44:00.789338 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:44:00.819280 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 69084 2024.10.22 01:44:00.885532 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(366ada0ac4bc9f2cb812130ea9eb3457) : [0, 136]: Setting download as failed: File segment: [0, 136], key: 366ada0ac4bc9f2cb812130ea9eb3457, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:00.887734 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(366ada0ac4bc9f2cb812130ea9eb3457): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:44:01.132325 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:01.137045 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:01.320967 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_598_117/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:44:01.375996 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1054_1054_0 (state Temporary) 2024.10.22 01:44:01.416883 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1054_1054_0 to 202410_1067_1067_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:01.426774 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1504 entries to flush up to offset 927566 2024.10.22 01:44:01.567410 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69084 2024.10.22 01:44:01.917790 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:01.920840 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:01.927933 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e05ccc58c365caa0d3cef9d1f32d078b) : [0, 12347]: Setting download as failed: File segment: [0, 12347], key: e05ccc58c365caa0d3cef9d1f32d078b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:01.932236 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e05ccc58c365caa0d3cef9d1f32d078b): No space left in cache to reserve 12348 bytes, will continue without cache download 2024.10.22 01:44:02.066716 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e05ccc58c365caa0d3cef9d1f32d078b, offset: 0, size: 0 2024.10.22 01:44:02.068282 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e05ccc58c365caa0d3cef9d1f32d078b for removal 2024.10.22 01:44:02.070017 [ 1476 ] {} CacheMetadata: Key e05ccc58c365caa0d3cef9d1f32d078b is removed from metadata 2024.10.22 01:44:02.071808 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 366ada0ac4bc9f2cb812130ea9eb3457, offset: 0, size: 0 2024.10.22 01:44:02.073128 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 366ada0ac4bc9f2cb812130ea9eb3457 for removal 2024.10.22 01:44:02.074566 [ 1476 ] {} CacheMetadata: Key 366ada0ac4bc9f2cb812130ea9eb3457 is removed from metadata 2024.10.22 01:44:02.146474 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:02.150427 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:02.378084 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_598_598_0 (state Temporary) 2024.10.22 01:44:02.383858 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 7 old parts to remove. Parts: [202410_1_960_479, 202410_961_961_0, 202410_962_962_0, 202410_963_963_0, 202410_964_964_0, 202410_965_965_0, 202410_966_966_0] 2024.10.22 01:44:02.385710 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 7 parts from filesystem (serially): Parts: [202410_1_960_479, 202410_961_961_0, 202410_962_962_0, 202410_963_963_0, 202410_964_964_0, 202410_965_965_0, 202410_966_966_0] 2024.10.22 01:44:02.417799 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 7 parts from memory: Parts: [202410_1_960_479, 202410_961_961_0, 202410_962_962_0, 202410_963_963_0, 202410_964_964_0, 202410_965_965_0, 202410_966_966_0] 2024.10.22 01:44:02.516879 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 65 entries to flush up to offset 70785 2024.10.22 01:44:02.568553 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_598_598_0 to 202410_605_605_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:02.586119 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 927566 2024.10.22 01:44:02.735577 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:02.740464 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:02.780740 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_613_613_0 (state Temporary) 2024.10.22 01:44:02.814282 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_613_613_0 to 202410_619_619_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:02.883845 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 70785 2024.10.22 01:44:03.377769 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(e13e685bb12daa7b83395ac1df147309) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e13e685bb12daa7b83395ac1df147309, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:44:03.380740 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 274, file segment info: File segment: [0, 33554431], key: e13e685bb12daa7b83395ac1df147309, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:44:03.382236 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:03.452525 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 16.49 MiB, will set to 1.66 GiB (RSS), difference: -3.00 MiB 2024.10.22 01:44:03.781166 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 523 2024.10.22 01:44:04.096665 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_598_117} MergeTask::MergeProjectionsStage: Merge sorted 932649 rows, containing 4 columns (4 merged, 0 gathered) in 26.631420655 sec., 35020.62515109936 rows/sec., 525.27 KiB/sec. 2024.10.22 01:44:04.259543 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6d346bb9908b3ed7346ba1f5fffcea98) : [0, 138]: Setting download as failed: File segment: [0, 138], key: 6d346bb9908b3ed7346ba1f5fffcea98, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:04.264883 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6d346bb9908b3ed7346ba1f5fffcea98): No space left in cache to reserve 139 bytes, will continue without cache download 2024.10.22 01:44:04.265249 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:04.268658 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:04.351887 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_598_117} MergedBlockOutputStream: filled checksums 202410_1_598_117 (state Temporary) 2024.10.22 01:44:04.403422 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_126_126_0 (state Temporary) 2024.10.22 01:44:04.447014 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_505_505_0 (state Temporary) 2024.10.22 01:44:04.677533 [ 1158 ] {} DatabaseCatalog: Execution took 251 ms. 2024.10.22 01:44:04.729197 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_505_505_0 to 202410_515_515_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:04.896407 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 523 2024.10.22 01:44:05.170692 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(1291db0e520636a4dd7e30c4477f3c30) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1291db0e520636a4dd7e30c4477f3c30, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:44:05.172696 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 54, file segment info: File segment: [0, 33554431], key: 1291db0e520636a4dd7e30c4477f3c30, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:44:05.174623 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:05.569515 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 69137 2024.10.22 01:44:05.987874 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:05.991616 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:06.099837 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c0b28b5daa1960e8c812fa910849368b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c0b28b5daa1960e8c812fa910849368b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:44:06.119739 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 212, file segment info: File segment: [0, 33554431], key: c0b28b5daa1960e8c812fa910849368b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:44:06.122453 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:06.160746 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1055_1055_0 (state Temporary) 2024.10.22 01:44:06.199812 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1055_1055_0 to 202410_1068_1068_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:06.452424 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69137 2024.10.22 01:44:06.752134 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 9584 2024.10.22 01:44:07.053576 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:07.058435 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:07.085751 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_625_625_0 (state Temporary) 2024.10.22 01:44:07.169361 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_625_625_0 to 202410_638_638_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:07.175208 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(65b8f5b13218cc36fdcee876d6a3d838) : [0, 72052]: Setting download as failed: File segment: [0, 72052], key: 65b8f5b13218cc36fdcee876d6a3d838, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:07.176872 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(65b8f5b13218cc36fdcee876d6a3d838): No space left in cache to reserve 72053 bytes, will continue without cache download 2024.10.22 01:44:07.209763 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9584 2024.10.22 01:44:07.401467 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 65b8f5b13218cc36fdcee876d6a3d838, offset: 0, size: 0 2024.10.22 01:44:07.406337 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 65b8f5b13218cc36fdcee876d6a3d838 for removal 2024.10.22 01:44:07.412724 [ 1476 ] {} CacheMetadata: Key 65b8f5b13218cc36fdcee876d6a3d838 is removed from metadata 2024.10.22 01:44:07.414046 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6d346bb9908b3ed7346ba1f5fffcea98, offset: 0, size: 0 2024.10.22 01:44:07.419668 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6d346bb9908b3ed7346ba1f5fffcea98 for removal 2024.10.22 01:44:07.426723 [ 1476 ] {} CacheMetadata: Key 6d346bb9908b3ed7346ba1f5fffcea98 is removed from metadata 2024.10.22 01:44:07.553161 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_598_117} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_598_117 to 202410_1_598_117 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:07.560841 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_598_117} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_592_116, 202410_593_598_1] -> 202410_1_598_117 2024.10.22 01:44:07.563697 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 13.11 MiB, will set to 1.66 GiB (RSS), difference: 5.92 MiB 2024.10.22 01:44:07.575819 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:44:07.987084 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a2d7b9ea8238d25bf9d85e2dcacae618) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a2d7b9ea8238d25bf9d85e2dcacae618, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:44:08.014667 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 773, file segment info: File segment: [0, 33554431], key: a2d7b9ea8238d25bf9d85e2dcacae618, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:44:08.016890 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:08.207260 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_126_126_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:44:08.457762 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 01:44:08.582962 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(02a19bbbd0b46b12840642e63587c41a) : [0, 138]: Setting download as failed: File segment: [0, 138], key: 02a19bbbd0b46b12840642e63587c41a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:08.583705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(02a19bbbd0b46b12840642e63587c41a): No space left in cache to reserve 139 bytes, will continue without cache download 2024.10.22 01:44:09.589514 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 735 entries to flush up to offset 928301 2024.10.22 01:44:09.648144 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(cce680d033832906870c1cfb4963c3f6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cce680d033832906870c1cfb4963c3f6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:44:09.652865 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 264, file segment info: File segment: [0, 33554431], key: cce680d033832906870c1cfb4963c3f6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:44:09.656594 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:10.069199 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:10.072119 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:10.377308 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_599_599_0 (state Temporary) 2024.10.22 01:44:10.464247 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 69178 2024.10.22 01:44:10.493719 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 44 entries to flush up to offset 70829 2024.10.22 01:44:10.567628 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_599_599_0 to 202410_606_606_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:10.608484 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 928301 2024.10.22 01:44:10.619245 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:10.622368 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:10.635998 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8699ae57d3f5d95a0beaf4f0814467f5) : [0, 75760]: Setting download as failed: File segment: [0, 75760], key: 8699ae57d3f5d95a0beaf4f0814467f5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:10.639034 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8699ae57d3f5d95a0beaf4f0814467f5): No space left in cache to reserve 75761 bytes, will continue without cache download 2024.10.22 01:44:10.672865 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:44:10.719252 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_614_614_0 (state Temporary) 2024.10.22 01:44:10.770924 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8699ae57d3f5d95a0beaf4f0814467f5, offset: 0, size: 0 2024.10.22 01:44:10.773442 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8699ae57d3f5d95a0beaf4f0814467f5 for removal 2024.10.22 01:44:10.777257 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:44:10.793301 [ 1476 ] {} CacheMetadata: Key 8699ae57d3f5d95a0beaf4f0814467f5 is removed from metadata 2024.10.22 01:44:10.801292 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 02a19bbbd0b46b12840642e63587c41a, offset: 0, size: 0 2024.10.22 01:44:10.803675 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 02a19bbbd0b46b12840642e63587c41a for removal 2024.10.22 01:44:10.811706 [ 1476 ] {} CacheMetadata: Key 02a19bbbd0b46b12840642e63587c41a is removed from metadata 2024.10.22 01:44:10.877725 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_614_614_0 to 202410_620_620_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:10.941781 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 70829 2024.10.22 01:44:11.000907 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:11.008091 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:11.279713 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1056_1056_0 (state Temporary) 2024.10.22 01:44:11.431519 [ 1170 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:44:11.466053 [ 1170 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:44:11.656025 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1056_1056_0 to 202410_1069_1069_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:11.677768 [ 1164 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1058_503 to 202410_1059_1064_1 2024.10.22 01:44:11.682242 [ 1164 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.58 MiB using storage policy from min volume index 0 2024.10.22 01:44:11.684099 [ 1164 ] {} DiskLocal: Reserved 2.58 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:11.704127 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69178 2024.10.22 01:44:11.756920 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1064_504} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1058_503 to 202410_1059_1064_1 into Wide with storage Full 2024.10.22 01:44:11.818832 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1064_504} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:44:11.835405 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1064_504} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1058_503, total 69328 rows starting from the beginning of the part 2024.10.22 01:44:12.016703 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1064_504} MergeTreeSequentialSource: Reading 2 marks from part 202410_1059_1064_1, total 364 rows starting from the beginning of the part 2024.10.22 01:44:12.128326 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(f643ac0bf821bc801ca84fb319f25863) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f643ac0bf821bc801ca84fb319f25863, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:44:12.130811 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4507, file segment info: File segment: [0, 33554431], key: f643ac0bf821bc801ca84fb319f25863, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:44:12.135394 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:12.401079 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 524 2024.10.22 01:44:12.647091 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:12.659450 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:12.818813 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_506_506_0 (state Temporary) 2024.10.22 01:44:13.131003 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(aff8b112bb4fabe4a3c192d208fc1579) : [0, 138]: Setting download as failed: File segment: [0, 138], key: aff8b112bb4fabe4a3c192d208fc1579, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:13.132433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(aff8b112bb4fabe4a3c192d208fc1579): No space left in cache to reserve 139 bytes, will continue without cache download 2024.10.22 01:44:13.313644 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_506_506_0 to 202410_516_516_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:13.386064 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 524 2024.10.22 01:44:13.725415 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_126_126_0 to 202410_129_129_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:14.079590 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d0c2327c69787f7dd816fbd6eb762d86) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d0c2327c69787f7dd816fbd6eb762d86, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:44:14.082497 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 532, file segment info: File segment: [0, 33554431], key: d0c2327c69787f7dd816fbd6eb762d86, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:44:14.084689 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:14.127965 [ 1683 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/okp/nwjglxnyuxqauatdjnozezdyeedyf, size 773 2024.10.22 01:44:14.158083 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1_1064_504/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:44:14.430835 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:44:14.461080 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:44:14.474132 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:44:14.500551 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:44:14.548389 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:44:14.576993 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:44:14.615901 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:44:14.678217 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:44:14.704349 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ShareURL 2024.10.22 01:44:14.712984 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 9593 2024.10.22 01:44:14.844106 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/khd/liuioroaepxffhfugtxcyumqtmgqr (LogSeriesLimiter: on interval from 2024-10-22 01:43:49 to 2024-10-22 01:44:14 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:44:14.860704 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:14.865130 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:14.913493 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_626_626_0 (state Temporary) 2024.10.22 01:44:14.933527 [ 1176 ] {} DatabaseCatalog: Execution took 255 ms. 2024.10.22 01:44:14.971326 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_626_626_0 to 202410_639_639_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:15.043893 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9593 2024.10.22 01:44:15.118110 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(19fda24a0a44adbc31f626bf3e1119f5) : [0, 82609]: Setting download as failed: File segment: [0, 82609], key: 19fda24a0a44adbc31f626bf3e1119f5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:15.118866 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(19fda24a0a44adbc31f626bf3e1119f5): No space left in cache to reserve 82610 bytes, will continue without cache download 2024.10.22 01:44:15.147783 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 19fda24a0a44adbc31f626bf3e1119f5, offset: 0, size: 0 2024.10.22 01:44:15.154685 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 19fda24a0a44adbc31f626bf3e1119f5 for removal 2024.10.22 01:44:15.159700 [ 1476 ] {} CacheMetadata: Key 19fda24a0a44adbc31f626bf3e1119f5 is removed from metadata 2024.10.22 01:44:15.161406 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: aff8b112bb4fabe4a3c192d208fc1579, offset: 0, size: 0 2024.10.22 01:44:15.163316 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key aff8b112bb4fabe4a3c192d208fc1579 for removal 2024.10.22 01:44:15.165558 [ 1476 ] {} CacheMetadata: Key aff8b112bb4fabe4a3c192d208fc1579 is removed from metadata 2024.10.22 01:44:15.241667 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1064_504} MergeTask::MergeProjectionsStage: Merge sorted 69692 rows, containing 13 columns (13 merged, 0 gathered) in 3.501119806 sec., 19905.631301324283 rows/sec., 3.29 MiB/sec. 2024.10.22 01:44:15.432017 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1064_504} MergedBlockOutputStream: filled checksums 202410_1_1064_504 (state Temporary) 2024.10.22 01:44:15.605456 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 5.96 MiB, will set to 1.65 GiB (RSS), difference: -36.71 MiB 2024.10.22 01:44:15.711560 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 69256 2024.10.22 01:44:15.860253 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:15.866378 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:16.017072 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1057_1057_0 (state Temporary) 2024.10.22 01:44:16.169656 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1057_1057_0 to 202410_1070_1070_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:16.215287 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(33b45b69f267a596aac113b34128947a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 33b45b69f267a596aac113b34128947a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:44:16.217116 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 461, file segment info: File segment: [0, 33554431], key: 33b45b69f267a596aac113b34128947a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:44:16.218801 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:16.207804 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1064_504} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1064_504 to 202410_1_1064_504 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:16.238419 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1064_504} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1058_503, 202410_1059_1064_1] -> 202410_1_1064_504 2024.10.22 01:44:16.257087 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1065_1065_0 to 202410_1070_1070_0 2024.10.22 01:44:16.263069 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.59 MiB. 2024.10.22 01:44:16.265678 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:16.267812 [ 1264 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:16.355100 [ 1744 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uri/jdkorukihivsixzvzskikmkfyoeol, size 4507 2024.10.22 01:44:16.424825 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1065_1065_0 to 202410_1070_1070_0 into Compact with storage Full 2024.10.22 01:44:16.459372 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69256 2024.10.22 01:44:16.480832 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:44:16.489763 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1065_1065_0, total 110 rows starting from the beginning of the part 2024.10.22 01:44:16.869549 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e7d81a95654081293379cb2db8c71066) : [0, 138]: Setting download as failed: File segment: [0, 138], key: e7d81a95654081293379cb2db8c71066, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:16.869536 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5047 2024.10.22 01:44:16.872322 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e7d81a95654081293379cb2db8c71066): No space left in cache to reserve 139 bytes, will continue without cache download 2024.10.22 01:44:16.908689 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1066_1066_0, total 72 rows starting from the beginning of the part 2024.10.22 01:44:16.925174 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:44:16.935283 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:44:16.942873 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1067_1067_0, total 41 rows starting from the beginning of the part 2024.10.22 01:44:16.948789 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:44:17.065821 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:44:17.073757 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1068_1068_0, total 53 rows starting from the beginning of the part 2024.10.22 01:44:17.187066 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:44:17.311883 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1069_1069_0, total 41 rows starting from the beginning of the part 2024.10.22 01:44:17.312724 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:44:17.347512 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:44:17.350355 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1070_1070_0, total 78 rows starting from the beginning of the part 2024.10.22 01:44:17.371830 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column DontCountHits 2024.10.22 01:44:17.611922 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 929456 2024.10.22 01:44:17.857789 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(daaefc8549cfc43868510f5a22c8de36) : [0, 83179]: Setting download as failed: File segment: [0, 83179], key: daaefc8549cfc43868510f5a22c8de36, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:17.860185 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(daaefc8549cfc43868510f5a22c8de36): No space left in cache to reserve 83180 bytes, will continue without cache download 2024.10.22 01:44:17.871987 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: daaefc8549cfc43868510f5a22c8de36, offset: 0, size: 0 2024.10.22 01:44:17.873806 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key daaefc8549cfc43868510f5a22c8de36 for removal 2024.10.22 01:44:17.881073 [ 1476 ] {} CacheMetadata: Key daaefc8549cfc43868510f5a22c8de36 is removed from metadata 2024.10.22 01:44:17.882989 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e7d81a95654081293379cb2db8c71066, offset: 0, size: 0 2024.10.22 01:44:17.884391 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e7d81a95654081293379cb2db8c71066 for removal 2024.10.22 01:44:17.893798 [ 1476 ] {} CacheMetadata: Key e7d81a95654081293379cb2db8c71066 is removed from metadata 2024.10.22 01:44:18.280862 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.97 GiB, free memory in arenas 7.81 MiB, will set to 1.65 GiB (RSS), difference: 27.59 MiB 2024.10.22 01:44:18.399760 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:18.407384 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:18.464034 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} MergeTask::MergeProjectionsStage: Merge sorted 395 rows, containing 13 columns (13 merged, 0 gathered) in 2.047460407 sec., 192.92192349583246 rows/sec., 34.72 KiB/sec. 2024.10.22 01:44:18.474396 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 152 entries to flush up to offset 70981 2024.10.22 01:44:18.536819 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_600_600_0 (state Temporary) 2024.10.22 01:44:18.582929 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} MergedBlockOutputStream: filled checksums 202410_1065_1070_1 (state Temporary) 2024.10.22 01:44:18.697497 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:18.705424 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:18.713644 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1065_1070_1 to 202410_1065_1070_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:18.719799 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1070_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1065_1065_0, 202410_1070_1070_0] -> 202410_1065_1070_1 2024.10.22 01:44:18.762435 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_600_600_0 to 202410_607_607_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:18.816880 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 929456 2024.10.22 01:44:18.862617 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_615_615_0 (state Temporary) 2024.10.22 01:44:18.950465 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_615_615_0 to 202410_621_621_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:18.960597 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:44:18.973334 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_616_22 to 202410_621_621_0 2024.10.22 01:44:18.982169 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:18.982650 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 70981 2024.10.22 01:44:18.984016 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:19.033300 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_616_22 to 202410_621_621_0 into Compact with storage Full 2024.10.22 01:44:19.231972 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:44:19.354735 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_616_22, total 11325 rows starting from the beginning of the part 2024.10.22 01:44:19.381171 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_617_617_0, total 186 rows starting from the beginning of the part 2024.10.22 01:44:19.422420 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_618_618_0, total 84 rows starting from the beginning of the part 2024.10.22 01:44:19.554955 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_619_619_0, total 65 rows starting from the beginning of the part 2024.10.22 01:44:19.579049 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_620_620_0, total 44 rows starting from the beginning of the part 2024.10.22 01:44:19.720958 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_621_621_0, total 152 rows starting from the beginning of the part 2024.10.22 01:44:19.771592 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ce309473b0609ccf6467d3ef4ee6061d) : [0, 138]: Setting download as failed: File segment: [0, 138], key: ce309473b0609ccf6467d3ef4ee6061d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:19.772606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ce309473b0609ccf6467d3ef4ee6061d): No space left in cache to reserve 139 bytes, will continue without cache download 2024.10.22 01:44:20.468020 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 69330 2024.10.22 01:44:20.562892 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_463_95, 202410_464_464_0, 202410_465_465_0, 202410_466_466_0, 202410_467_467_0, 202410_468_468_0] 2024.10.22 01:44:20.564484 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_463_95, 202410_464_464_0, 202410_465_465_0, 202410_466_466_0, 202410_467_467_0, 202410_468_468_0] 2024.10.22 01:44:20.567817 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_557_10, 202410_558_558_0, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0, 202410_562_562_0] 2024.10.22 01:44:20.569347 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_557_10, 202410_558_558_0, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0, 202410_562_562_0] 2024.10.22 01:44:20.674271 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:44:20.675967 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:44:20.697053 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_463_95, 202410_464_464_0, 202410_465_465_0, 202410_466_466_0, 202410_467_467_0, 202410_468_468_0] 2024.10.22 01:44:20.714398 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_557_10, 202410_558_558_0, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0, 202410_562_562_0] 2024.10.22 01:44:20.783490 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:20.785535 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:20.890280 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 525 2024.10.22 01:44:20.930923 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d0fe6ccd3a95f0184e9e8bdc365ffc88) : [0, 73821]: Setting download as failed: File segment: [0, 73821], key: d0fe6ccd3a95f0184e9e8bdc365ffc88, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:20.932164 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d0fe6ccd3a95f0184e9e8bdc365ffc88): No space left in cache to reserve 73822 bytes, will continue without cache download 2024.10.22 01:44:20.948110 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d0fe6ccd3a95f0184e9e8bdc365ffc88, offset: 0, size: 0 2024.10.22 01:44:20.950918 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d0fe6ccd3a95f0184e9e8bdc365ffc88 for removal 2024.10.22 01:44:20.955245 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ce309473b0609ccf6467d3ef4ee6061d, offset: 0, size: 0 2024.10.22 01:44:20.956671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ce309473b0609ccf6467d3ef4ee6061d for removal 2024.10.22 01:44:20.992982 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1058_1058_0 (state Temporary) 2024.10.22 01:44:21.026691 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_621_23/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:44:21.044243 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_545_545_0, 202410_546_546_0, 202410_547_547_0, 202410_548_548_0, 202410_549_549_0, 202410_550_550_0] 2024.10.22 01:44:21.052746 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_545_545_0, 202410_546_546_0, 202410_547_547_0, 202410_548_548_0, 202410_549_549_0, 202410_550_550_0] 2024.10.22 01:44:20.953821 [ 1476 ] {} CacheMetadata: Key d0fe6ccd3a95f0184e9e8bdc365ffc88 is removed from metadata 2024.10.22 01:44:21.059831 [ 1476 ] {} CacheMetadata: Key ce309473b0609ccf6467d3ef4ee6061d is removed from metadata 2024.10.22 01:44:21.084486 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1058_1058_0 to 202410_1071_1071_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:21.185371 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_545_545_0, 202410_546_546_0, 202410_547_547_0, 202410_548_548_0, 202410_549_549_0, 202410_550_550_0] 2024.10.22 01:44:21.197299 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:21.254888 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69330 2024.10.22 01:44:21.263762 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:21.321389 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 4.03 MiB, will set to 1.65 GiB (RSS), difference: 1.77 MiB 2024.10.22 01:44:21.415015 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} MergeTask::MergeProjectionsStage: Merge sorted 11856 rows, containing 13 columns (13 merged, 0 gathered) in 2.387797162 sec., 4965.245871248757 rows/sec., 2.15 MiB/sec. 2024.10.22 01:44:21.500708 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_507_507_0 (state Temporary) 2024.10.22 01:44:21.580997 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} MergedBlockOutputStream: filled checksums 202410_507_621_23 (state Temporary) 2024.10.22 01:44:21.782201 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_507_507_0 to 202410_517_517_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:21.852028 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_621_23 to 202410_507_621_23 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:21.858913 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_621_23} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_616_22, 202410_621_621_0] -> 202410_507_621_23 2024.10.22 01:44:21.887280 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.57 MiB. 2024.10.22 01:44:21.956583 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 525 2024.10.22 01:44:22.341004 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(91218547d9c3fb5c7b26acdd195fef02) : [0, 138]: Setting download as failed: File segment: [0, 138], key: 91218547d9c3fb5c7b26acdd195fef02, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:22.342588 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(91218547d9c3fb5c7b26acdd195fef02): No space left in cache to reserve 139 bytes, will continue without cache download 2024.10.22 01:44:22.547118 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 9616 2024.10.22 01:44:22.763478 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:22.766322 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:22.806796 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_627_627_0 (state Temporary) 2024.10.22 01:44:23.061771 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_627_627_0 to 202410_640_640_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:23.111530 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9616 2024.10.22 01:44:23.242608 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3b90ff835b8981501fb73c58c76bc7ac) : [0, 70320]: Setting download as failed: File segment: [0, 70320], key: 3b90ff835b8981501fb73c58c76bc7ac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:23.245127 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3b90ff835b8981501fb73c58c76bc7ac): No space left in cache to reserve 70321 bytes, will continue without cache download 2024.10.22 01:44:23.275862 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3b90ff835b8981501fb73c58c76bc7ac, offset: 0, size: 0 2024.10.22 01:44:23.287635 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3b90ff835b8981501fb73c58c76bc7ac for removal 2024.10.22 01:44:23.289821 [ 1476 ] {} CacheMetadata: Key 3b90ff835b8981501fb73c58c76bc7ac is removed from metadata 2024.10.22 01:44:23.292664 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 91218547d9c3fb5c7b26acdd195fef02, offset: 0, size: 0 2024.10.22 01:44:23.305491 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 91218547d9c3fb5c7b26acdd195fef02 for removal 2024.10.22 01:44:23.312306 [ 1476 ] {} CacheMetadata: Key 91218547d9c3fb5c7b26acdd195fef02 is removed from metadata 2024.10.22 01:44:24.348007 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fca9f6c7367abb46e0ed8f416ae77cba) : [0, 138]: Setting download as failed: File segment: [0, 138], key: fca9f6c7367abb46e0ed8f416ae77cba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:24.350136 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fca9f6c7367abb46e0ed8f416ae77cba): No space left in cache to reserve 139 bytes, will continue without cache download 2024.10.22 01:44:24.386522 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 36 entries to flush up to offset 5083 2024.10.22 01:44:24.469156 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 4.02 MiB, will set to 1.65 GiB (RSS), difference: 4.73 MiB 2024.10.22 01:44:24.542738 [ 1246 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 5 parts from 202410_125_125_0 to 202410_129_129_0 2024.10.22 01:44:24.558564 [ 1246 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:24.561558 [ 1246 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:24.640360 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_125_129_1} MergeTask::PrepareStage: Merging 5 parts: from 202410_125_125_0 to 202410_129_129_0 into Compact with storage Full 2024.10.22 01:44:25.161254 [ 1224 ] {} DatabaseCatalog: Execution took 229 ms. 2024.10.22 01:44:25.236370 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(80abfdbb0ddfd6a38d43eef281751206) : [0, 84025]: Setting download as failed: File segment: [0, 84025], key: 80abfdbb0ddfd6a38d43eef281751206, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:25.238591 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(80abfdbb0ddfd6a38d43eef281751206): No space left in cache to reserve 84026 bytes, will continue without cache download 2024.10.22 01:44:25.271560 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 69395 2024.10.22 01:44:25.272432 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 80abfdbb0ddfd6a38d43eef281751206, offset: 0, size: 0 2024.10.22 01:44:25.273820 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 80abfdbb0ddfd6a38d43eef281751206 for removal 2024.10.22 01:44:25.277303 [ 1476 ] {} CacheMetadata: Key 80abfdbb0ddfd6a38d43eef281751206 is removed from metadata 2024.10.22 01:44:25.279670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fca9f6c7367abb46e0ed8f416ae77cba, offset: 0, size: 0 2024.10.22 01:44:25.280955 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fca9f6c7367abb46e0ed8f416ae77cba for removal 2024.10.22 01:44:25.282786 [ 1476 ] {} CacheMetadata: Key fca9f6c7367abb46e0ed8f416ae77cba is removed from metadata 2024.10.22 01:44:25.507232 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:25.511525 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:25.773062 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1059_1059_0 (state Temporary) 2024.10.22 01:44:25.822747 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 930611 2024.10.22 01:44:26.026558 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1059_1059_0 to 202410_1072_1072_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:26.084530 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69395 2024.10.22 01:44:26.159659 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(da80e7bb6b01709aea271bb506ff2bd1) : [0, 138]: Setting download as failed: File segment: [0, 138], key: da80e7bb6b01709aea271bb506ff2bd1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:26.163410 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(da80e7bb6b01709aea271bb506ff2bd1): No space left in cache to reserve 139 bytes, will continue without cache download 2024.10.22 01:44:26.461481 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:26.464177 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:26.468816 [ 1153 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:44:26.488863 [ 1153 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:44:26.591399 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 3.29 MiB, will set to 1.64 GiB (RSS), difference: -1.97 MiB 2024.10.22 01:44:26.632061 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 56 entries to flush up to offset 71037 2024.10.22 01:44:26.751085 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:26.755403 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:26.803792 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_616_616_0 (state Temporary) 2024.10.22 01:44:26.805738 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_601_601_0 (state Temporary) 2024.10.22 01:44:26.965616 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_616_616_0 to 202410_622_622_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:26.987977 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_601_601_0 to 202410_608_608_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:27.022833 [ 1167 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_598_117 to 202410_599_604_1 2024.10.22 01:44:27.057548 [ 1167 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:27.061593 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:27.093368 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_604_118} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_598_117 to 202410_599_604_1 into Wide with storage Full 2024.10.22 01:44:27.122912 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_604_118} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:44:27.128744 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 930611 2024.10.22 01:44:27.132121 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_604_118} MergeTreeSequentialSource: Reading 115 marks from part 202410_1_598_117, total 932649 rows starting from the beginning of the part 2024.10.22 01:44:27.213708 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 71037 2024.10.22 01:44:27.259359 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a64b6c44b9c895ee76ceb2d807b4da7c) : [0, 81045]: Setting download as failed: File segment: [0, 81045], key: a64b6c44b9c895ee76ceb2d807b4da7c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:27.260476 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a64b6c44b9c895ee76ceb2d807b4da7c): No space left in cache to reserve 81046 bytes, will continue without cache download 2024.10.22 01:44:27.288106 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a64b6c44b9c895ee76ceb2d807b4da7c, offset: 0, size: 0 2024.10.22 01:44:27.289673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a64b6c44b9c895ee76ceb2d807b4da7c for removal 2024.10.22 01:44:27.292814 [ 1476 ] {} CacheMetadata: Key a64b6c44b9c895ee76ceb2d807b4da7c is removed from metadata 2024.10.22 01:44:27.303602 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: da80e7bb6b01709aea271bb506ff2bd1, offset: 0, size: 0 2024.10.22 01:44:27.308192 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key da80e7bb6b01709aea271bb506ff2bd1 for removal 2024.10.22 01:44:27.308633 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_604_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_599_604_1, total 7003 rows starting from the beginning of the part 2024.10.22 01:44:27.412333 [ 1476 ] {} CacheMetadata: Key da80e7bb6b01709aea271bb506ff2bd1 is removed from metadata 2024.10.22 01:44:28.237091 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(92212fac1cb205b2b663992d06bfb1fb) : [0, 136]: Setting download as failed: File segment: [0, 136], key: 92212fac1cb205b2b663992d06bfb1fb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:28.241730 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(92212fac1cb205b2b663992d06bfb1fb): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:44:28.675344 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 4.13 MiB, will set to 1.65 GiB (RSS), difference: -9.48 MiB 2024.10.22 01:44:28.784858 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_604_118/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:44:29.165223 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_576_115, 202410_577_577_0, 202410_578_578_0, 202410_579_579_0, 202410_580_580_0, 202410_581_581_0] 2024.10.22 01:44:29.268192 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_576_115, 202410_577_577_0, 202410_578_578_0, 202410_579_579_0, 202410_580_580_0, 202410_581_581_0] 2024.10.22 01:44:29.417267 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_576_115, 202410_577_577_0, 202410_578_578_0, 202410_579_579_0, 202410_580_580_0, 202410_581_581_0] 2024.10.22 01:44:29.631600 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(92232216f0dff934d6dddf95da6eb6da) : [0, 62293]: Setting download as failed: File segment: [0, 62293], key: 92232216f0dff934d6dddf95da6eb6da, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:29.634606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(92232216f0dff934d6dddf95da6eb6da): No space left in cache to reserve 62294 bytes, will continue without cache download 2024.10.22 01:44:29.664882 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 92232216f0dff934d6dddf95da6eb6da, offset: 0, size: 0 2024.10.22 01:44:29.667645 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 92232216f0dff934d6dddf95da6eb6da for removal 2024.10.22 01:44:29.670400 [ 1476 ] {} CacheMetadata: Key 92232216f0dff934d6dddf95da6eb6da is removed from metadata 2024.10.22 01:44:29.670959 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 92212fac1cb205b2b663992d06bfb1fb, offset: 0, size: 0 2024.10.22 01:44:29.673374 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 92212fac1cb205b2b663992d06bfb1fb for removal 2024.10.22 01:44:29.685624 [ 1476 ] {} CacheMetadata: Key 92212fac1cb205b2b663992d06bfb1fb is removed from metadata 2024.10.22 01:44:30.090299 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 69454 2024.10.22 01:44:30.268536 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:30.280852 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:30.428276 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1060_1060_0 (state Temporary) 2024.10.22 01:44:30.489611 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1060_1060_0 to 202410_1073_1073_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:30.615253 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 9621 2024.10.22 01:44:30.664250 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69454 2024.10.22 01:44:30.665382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(82feb4fb8b15d00d0a7ab237af3dbc21) : [0, 136]: Setting download as failed: File segment: [0, 136], key: 82feb4fb8b15d00d0a7ab237af3dbc21, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:30.667318 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(82feb4fb8b15d00d0a7ab237af3dbc21): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:44:30.672068 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:44:30.683043 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:44:30.784089 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1_966_480, 202410_967_967_0, 202410_968_968_0, 202410_969_969_0, 202410_970_970_0, 202410_971_971_0] 2024.10.22 01:44:30.787543 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1_966_480, 202410_967_967_0, 202410_968_968_0, 202410_969_969_0, 202410_970_970_0, 202410_971_971_0] 2024.10.22 01:44:30.832146 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1_966_480, 202410_967_967_0, 202410_968_968_0, 202410_969_969_0, 202410_970_970_0, 202410_971_971_0] 2024.10.22 01:44:30.871192 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:30.874828 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:31.159810 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_628_628_0 (state Temporary) 2024.10.22 01:44:31.305451 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_628_628_0 to 202410_641_641_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:31.372980 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9621 2024.10.22 01:44:31.435615 [ 1179 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 5 parts from 202410_1_636_127 to 202410_640_640_0 2024.10.22 01:44:31.450978 [ 1179 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:31.455516 [ 1179 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:31.470331 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_640_128} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_636_127 to 202410_640_640_0 into Compact with storage Full 2024.10.22 01:44:31.502683 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_640_128} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:44:31.511571 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 7.73 MiB, will set to 1.65 GiB (RSS), difference: 6.43 MiB 2024.10.22 01:44:31.511849 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(28169a4974202aad13dbb45afa6bf879) : [0, 62657]: Setting download as failed: File segment: [0, 62657], key: 28169a4974202aad13dbb45afa6bf879, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:31.514700 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(28169a4974202aad13dbb45afa6bf879): No space left in cache to reserve 62658 bytes, will continue without cache download 2024.10.22 01:44:31.536184 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 28169a4974202aad13dbb45afa6bf879, offset: 0, size: 0 2024.10.22 01:44:31.543658 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 28169a4974202aad13dbb45afa6bf879 for removal 2024.10.22 01:44:31.549013 [ 1476 ] {} CacheMetadata: Key 28169a4974202aad13dbb45afa6bf879 is removed from metadata 2024.10.22 01:44:31.551808 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 82feb4fb8b15d00d0a7ab237af3dbc21, offset: 0, size: 0 2024.10.22 01:44:31.553477 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 82feb4fb8b15d00d0a7ab237af3dbc21 for removal 2024.10.22 01:44:31.556906 [ 1476 ] {} CacheMetadata: Key 82feb4fb8b15d00d0a7ab237af3dbc21 is removed from metadata 2024.10.22 01:44:31.621483 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_640_128} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_636_127, total 9650 rows starting from the beginning of the part 2024.10.22 01:44:31.745728 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_640_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_637_637_0, total 34 rows starting from the beginning of the part 2024.10.22 01:44:31.866088 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_640_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_638_638_0, total 6 rows starting from the beginning of the part 2024.10.22 01:44:31.882676 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_640_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_639_639_0, total 9 rows starting from the beginning of the part 2024.10.22 01:44:31.913478 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_640_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_640_640_0, total 23 rows starting from the beginning of the part 2024.10.22 01:44:32.303660 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(d8b50fd83031e036d2e6aa8fdff0742a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d8b50fd83031e036d2e6aa8fdff0742a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:44:32.316021 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 967013, file segment info: File segment: [0, 33554431], key: d8b50fd83031e036d2e6aa8fdff0742a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:44:32.321698 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:32.915923 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_538_107, 202410_539_544_1] 2024.10.22 01:44:32.920172 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_538_107, 202410_539_544_1] 2024.10.22 01:44:33.053796 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_538_107, 202410_539_544_1] 2024.10.22 01:44:33.161595 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_640_128} MergeTask::MergeProjectionsStage: Merge sorted 9722 rows, containing 12 columns (12 merged, 0 gathered) in 1.694349619 sec., 5737.894877763123 rows/sec., 913.01 KiB/sec. 2024.10.22 01:44:33.294892 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_640_128} MergedBlockOutputStream: filled checksums 202410_1_640_128 (state Temporary) 2024.10.22 01:44:33.368265 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(04765c78fd5748d2a516d758c3c0ac0d) : [0, 136]: Setting download as failed: File segment: [0, 136], key: 04765c78fd5748d2a516d758c3c0ac0d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:33.371871 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(04765c78fd5748d2a516d758c3c0ac0d): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:44:33.663640 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_640_128} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_640_128 to 202410_1_640_128 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:33.666323 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_640_128} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 5 parts: [202410_1_636_127, 202410_640_640_0] -> 202410_1_640_128 2024.10.22 01:44:33.806942 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.57 MiB. 2024.10.22 01:44:34.133259 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 931766 2024.10.22 01:44:34.371875 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(8697d1795fd323bda640c1cc81978fc7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8697d1795fd323bda640c1cc81978fc7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:44:34.389775 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 506, file segment info: File segment: [0, 33554431], key: 8697d1795fd323bda640c1cc81978fc7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:44:34.496265 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:34.668672 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 69509 2024.10.22 01:44:34.719190 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 104 entries to flush up to offset 71141 2024.10.22 01:44:34.879133 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:34.881709 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:34.937768 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_617_617_0 (state Temporary) 2024.10.22 01:44:34.986793 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_617_617_0 to 202410_623_623_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:34.909528 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:35.010446 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:35.085194 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1061_1061_0 (state Temporary) 2024.10.22 01:44:35.142176 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 71141 2024.10.22 01:44:35.176277 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:35.183680 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:35.184136 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column WithHash 2024.10.22 01:44:35.200101 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column WithHash 2024.10.22 01:44:35.220336 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column WithHash 2024.10.22 01:44:35.247722 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1061_1061_0 to 202410_1074_1074_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:35.260476 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fd8ff209edb729e7ebdf513fd102e790) : [0, 52560]: Setting download as failed: File segment: [0, 52560], key: fd8ff209edb729e7ebdf513fd102e790, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:35.262463 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fd8ff209edb729e7ebdf513fd102e790): No space left in cache to reserve 52561 bytes, will continue without cache download 2024.10.22 01:44:35.263796 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column WithHash 2024.10.22 01:44:35.274046 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column WithHash 2024.10.22 01:44:35.316523 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column WithHash 2024.10.22 01:44:35.336584 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69509 2024.10.22 01:44:35.339951 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fd8ff209edb729e7ebdf513fd102e790, offset: 0, size: 0 2024.10.22 01:44:35.348021 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fd8ff209edb729e7ebdf513fd102e790 for removal 2024.10.22 01:44:35.355762 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column WithHash 2024.10.22 01:44:35.356081 [ 1476 ] {} CacheMetadata: Key fd8ff209edb729e7ebdf513fd102e790 is removed from metadata 2024.10.22 01:44:35.357060 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 04765c78fd5748d2a516d758c3c0ac0d, offset: 0, size: 0 2024.10.22 01:44:35.369186 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column WithHash 2024.10.22 01:44:35.438838 [ 1196 ] {} DatabaseCatalog: Execution took 273 ms. 2024.10.22 01:44:35.460977 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 04765c78fd5748d2a516d758c3c0ac0d for removal 2024.10.22 01:44:35.463184 [ 1476 ] {} CacheMetadata: Key 04765c78fd5748d2a516d758c3c0ac0d is removed from metadata 2024.10.22 01:44:35.474813 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 9.05 MiB, will set to 1.65 GiB (RSS), difference: -2.17 MiB 2024.10.22 01:44:35.475848 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_602_602_0 (state Temporary) 2024.10.22 01:44:35.588907 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/cra/ngvjlhmcgragjawzxhqxoxbysrcit (LogSeriesLimiter: on interval from 2024-10-22 01:44:14 to 2024-10-22 01:44:35 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:44:35.756498 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 384 ms. 2024.10.22 01:44:35.772637 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_602_602_0 to 202410_609_609_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:35.844922 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 931766 2024.10.22 01:44:35.970868 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.184689522 sec. 2024.10.22 01:44:36.416481 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9e6da2e9ed792103185eb4baa3c43a4b) : [0, 138]: Setting download as failed: File segment: [0, 138], key: 9e6da2e9ed792103185eb4baa3c43a4b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:36.418377 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9e6da2e9ed792103185eb4baa3c43a4b): No space left in cache to reserve 139 bytes, will continue without cache download 2024.10.22 01:44:36.497359 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_125_129_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:44:36.961631 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 526 2024.10.22 01:44:37.163245 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7b5a795e9655c5a0bb5683dcff11e7c3) : [0, 85200]: Setting download as failed: File segment: [0, 85200], key: 7b5a795e9655c5a0bb5683dcff11e7c3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:37.164448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7b5a795e9655c5a0bb5683dcff11e7c3): No space left in cache to reserve 85201 bytes, will continue without cache download 2024.10.22 01:44:37.181224 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7b5a795e9655c5a0bb5683dcff11e7c3, offset: 0, size: 0 2024.10.22 01:44:37.183205 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7b5a795e9655c5a0bb5683dcff11e7c3 for removal 2024.10.22 01:44:37.187486 [ 1476 ] {} CacheMetadata: Key 7b5a795e9655c5a0bb5683dcff11e7c3 is removed from metadata 2024.10.22 01:44:37.190864 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9e6da2e9ed792103185eb4baa3c43a4b, offset: 0, size: 0 2024.10.22 01:44:37.193688 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9e6da2e9ed792103185eb4baa3c43a4b for removal 2024.10.22 01:44:37.293557 [ 1476 ] {} CacheMetadata: Key 9e6da2e9ed792103185eb4baa3c43a4b is removed from metadata 2024.10.22 01:44:37.345542 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:37.348700 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:37.397310 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_508_508_0 (state Temporary) 2024.10.22 01:44:37.399549 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_125_129_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_125_125_0, total 38 rows starting from the beginning of the part 2024.10.22 01:44:37.442455 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 9.13 MiB, will set to 1.66 GiB (RSS), difference: -1.87 MiB 2024.10.22 01:44:37.580426 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_508_508_0 to 202410_518_518_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:37.608772 [ 1207 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_513_105 to 202410_518_518_0 2024.10.22 01:44:37.757733 [ 1207 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:37.760244 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:37.777048 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_513_105 to 202410_518_518_0 into Compact with storage Full 2024.10.22 01:44:37.867375 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 526 2024.10.22 01:44:37.961784 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b2e2e1fbe747535270ac1605b11a2494) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b2e2e1fbe747535270ac1605b11a2494, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:44:37.964792 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1900, file segment info: File segment: [0, 33554431], key: b2e2e1fbe747535270ac1605b11a2494, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:44:37.971298 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:38.316743 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:44:38.466565 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_513_105, total 557 rows starting from the beginning of the part 2024.10.22 01:44:38.582368 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_514_514_0, total 1 rows starting from the beginning of the part 2024.10.22 01:44:38.690654 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a4444fb32ee9e0f399224e6168c97e5f) : [0, 136]: Setting download as failed: File segment: [0, 136], key: a4444fb32ee9e0f399224e6168c97e5f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:38.692461 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a4444fb32ee9e0f399224e6168c97e5f): No space left in cache to reserve 137 bytes, will continue without cache download 2024.10.22 01:44:38.875396 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 9632 2024.10.22 01:44:39.083396 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_515_515_0, total 1 rows starting from the beginning of the part 2024.10.22 01:44:39.105396 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:39.122267 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:39.168974 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:39.172447 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:39.272611 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_629_629_0 (state Temporary) 2024.10.22 01:44:39.331552 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_629_629_0 to 202410_642_642_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:39.346340 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 69586 2024.10.22 01:44:39.412313 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_516_516_0, total 1 rows starting from the beginning of the part 2024.10.22 01:44:39.476516 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9632 2024.10.22 01:44:39.536087 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d9d9babbbdb07bb8fd64102dba1b7e0b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d9d9babbbdb07bb8fd64102dba1b7e0b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:44:39.552547 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_604_118/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:44:39.557585 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 226, file segment info: File segment: [0, 33554431], key: d9d9babbbdb07bb8fd64102dba1b7e0b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:44:39.560045 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:39.819783 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_517_517_0, total 1 rows starting from the beginning of the part 2024.10.22 01:44:39.938402 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:39.946719 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:40.121155 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_518_518_0, total 1 rows starting from the beginning of the part 2024.10.22 01:44:40.216037 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1062_1062_0 (state Temporary) 2024.10.22 01:44:40.273826 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1062_1062_0 to 202410_1075_1075_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:40.290450 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1065_1070_1 to 202410_1075_1075_0 2024.10.22 01:44:40.308780 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_125_129_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_126_126_0, total 41 rows starting from the beginning of the part 2024.10.22 01:44:40.310521 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:40.314630 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:40.334816 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69586 2024.10.22 01:44:40.353416 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} MergeTask::PrepareStage: Merging 6 parts: from 202410_1065_1070_1 to 202410_1075_1075_0 into Compact with storage Full 2024.10.22 01:44:40.386173 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:44:40.456805 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 9.64 MiB, will set to 1.66 GiB (RSS), difference: 5.32 MiB 2024.10.22 01:44:40.532084 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1065_1070_1, total 395 rows starting from the beginning of the part 2024.10.22 01:44:40.540782 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(22a85385735fe45ba4dc7bdee0aa40ee) : [0, 24608]: Setting download as failed: File segment: [0, 24608], key: 22a85385735fe45ba4dc7bdee0aa40ee, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:40.543163 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(22a85385735fe45ba4dc7bdee0aa40ee): No space left in cache to reserve 24609 bytes, will continue without cache download 2024.10.22 01:44:40.664051 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 22a85385735fe45ba4dc7bdee0aa40ee, offset: 0, size: 0 2024.10.22 01:44:40.666780 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 22a85385735fe45ba4dc7bdee0aa40ee for removal 2024.10.22 01:44:40.668668 [ 1476 ] {} CacheMetadata: Key 22a85385735fe45ba4dc7bdee0aa40ee is removed from metadata 2024.10.22 01:44:40.668915 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a4444fb32ee9e0f399224e6168c97e5f, offset: 0, size: 0 2024.10.22 01:44:40.675428 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:44:40.677510 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:44:40.718231 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1071_1071_0, total 74 rows starting from the beginning of the part 2024.10.22 01:44:40.771357 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a4444fb32ee9e0f399224e6168c97e5f for removal 2024.10.22 01:44:40.776433 [ 1476 ] {} CacheMetadata: Key a4444fb32ee9e0f399224e6168c97e5f is removed from metadata 2024.10.22 01:44:40.898772 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1072_1072_0, total 65 rows starting from the beginning of the part 2024.10.22 01:44:41.035244 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1073_1073_0, total 59 rows starting from the beginning of the part 2024.10.22 01:44:41.233672 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1074_1074_0, total 55 rows starting from the beginning of the part 2024.10.22 01:44:41.268471 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(aabec3f9c8de384080c9e7cdd6c3167c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: aabec3f9c8de384080c9e7cdd6c3167c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:44:41.281997 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 773, file segment info: File segment: [0, 33554431], key: aabec3f9c8de384080c9e7cdd6c3167c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:44:41.284965 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:41.305827 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1075_1075_0, total 77 rows starting from the beginning of the part 2024.10.22 01:44:41.495977 [ 1254 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:44:41.540228 [ 1254 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:44:42.074224 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e446c84bc6fb8feb6abf296591770b4d) : [0, 140]: Setting download as failed: File segment: [0, 140], key: e446c84bc6fb8feb6abf296591770b4d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:42.075006 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_125_129_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_127_127_0, total 42 rows starting from the beginning of the part 2024.10.22 01:44:42.075429 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e446c84bc6fb8feb6abf296591770b4d): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:44:42.116413 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} MergeTask::MergeProjectionsStage: Merge sorted 562 rows, containing 32 columns (32 merged, 0 gathered) in 4.34506516 sec., 129.3421339623823 rows/sec., 11.87 KiB/sec. 2024.10.22 01:44:42.142957 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} MergedBlockOutputStream: filled checksums 202410_1_518_106 (state Temporary) 2024.10.22 01:44:42.266624 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_127_127_0 (state Temporary) 2024.10.22 01:44:42.519122 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} MergeTask::MergeProjectionsStage: Merge sorted 725 rows, containing 13 columns (13 merged, 0 gathered) in 2.175603909 sec., 333.24080592098255 rows/sec., 59.35 KiB/sec. 2024.10.22 01:44:42.572030 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} MergedBlockOutputStream: filled checksums 202410_1065_1075_2 (state Temporary) 2024.10.22 01:44:42.656951 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 93 entries to flush up to offset 71234 2024.10.22 01:44:42.761956 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a25792b99fe457685aed186a4128d5bc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a25792b99fe457685aed186a4128d5bc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:44:42.767263 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 532, file segment info: File segment: [0, 33554431], key: a25792b99fe457685aed186a4128d5bc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1396, kind: Regular, unbound: 0 2024.10.22 01:44:42.769594 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:42.812961 [ 1684 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/khd/liuioroaepxffhfugtxcyumqtmgqr, size 773 2024.10.22 01:44:42.831065 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1065_1075_2 to 202410_1065_1075_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:42.839310 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1075_2} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1065_1070_1, 202410_1075_1075_0] -> 202410_1065_1075_2 2024.10.22 01:44:42.852240 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 932921 2024.10.22 01:44:42.884329 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:44:42.973208 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:42.975299 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:43.035243 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_618_618_0 (state Temporary) 2024.10.22 01:44:43.056092 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_518_106 to 202410_1_518_106 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:43.058139 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_518_106} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_513_105, 202410_518_518_0] -> 202410_1_518_106 2024.10.22 01:44:43.071406 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_618_618_0 to 202410_624_624_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:43.083897 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:44:43.249331 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 71234 2024.10.22 01:44:43.324590 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:44:43.343183 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:44:43.459221 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:44:43.474975 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:44:43.500458 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:44:43.660447 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:44:43.732798 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:44:43.737509 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 9.73 MiB, will set to 1.66 GiB (RSS), difference: -7.53 MiB 2024.10.22 01:44:43.757535 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:44:43.765602 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_125_129_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_128_128_0, total 38 rows starting from the beginning of the part 2024.10.22 01:44:43.778875 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ShareTitle 2024.10.22 01:44:43.855710 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hag/kdrgaugrmlrikyuuedjlbwuwlcxfv (LogSeriesLimiter: on interval from 2024-10-22 01:44:35 to 2024-10-22 01:44:43 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:44:43.880336 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:43.882583 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:43.981114 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e27dc1988219d7f18ba6291e489984d5) : [0, 1167]: Setting download as failed: File segment: [0, 1167], key: e27dc1988219d7f18ba6291e489984d5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:43.982423 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e27dc1988219d7f18ba6291e489984d5): No space left in cache to reserve 1168 bytes, will continue without cache download 2024.10.22 01:44:44.039669 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_603_603_0 (state Temporary) 2024.10.22 01:44:44.127576 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e27dc1988219d7f18ba6291e489984d5, offset: 0, size: 0 2024.10.22 01:44:44.131609 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e27dc1988219d7f18ba6291e489984d5 for removal 2024.10.22 01:44:44.133251 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_603_603_0 to 202410_610_610_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:44.139613 [ 1476 ] {} CacheMetadata: Key e27dc1988219d7f18ba6291e489984d5 is removed from metadata 2024.10.22 01:44:44.141667 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e446c84bc6fb8feb6abf296591770b4d, offset: 0, size: 0 2024.10.22 01:44:44.147015 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e446c84bc6fb8feb6abf296591770b4d for removal 2024.10.22 01:44:44.153352 [ 1476 ] {} CacheMetadata: Key e446c84bc6fb8feb6abf296591770b4d is removed from metadata 2024.10.22 01:44:44.162650 [ 1182 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_605_605_0 to 202410_610_610_0 2024.10.22 01:44:44.172366 [ 1182 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:44.174451 [ 1182 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:44.175935 [ 1261 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 212 ms. 2024.10.22 01:44:44.273039 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 932921 2024.10.22 01:44:44.308088 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_605_605_0 to 202410_610_610_0 into Compact with storage Full 2024.10.22 01:44:44.349428 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 97 entries to flush up to offset 69683 2024.10.22 01:44:44.471420 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:44:44.479176 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_605_605_0, total 1504 rows starting from the beginning of the part 2024.10.22 01:44:44.505557 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_606_606_0, total 735 rows starting from the beginning of the part 2024.10.22 01:44:44.533605 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_607_607_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:44:44.562532 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_608_608_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:44:44.688458 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_609_609_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:44:44.715355 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_610_610_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:44:44.823281 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:44.827146 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:44.912266 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1063_1063_0 (state Temporary) 2024.10.22 01:44:45.055865 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1063_1063_0 to 202410_1076_1076_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:45.067740 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(33b582208884519c0e32876a31291544) : [0, 169]: Setting download as failed: File segment: [0, 169], key: 33b582208884519c0e32876a31291544, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:45.068877 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(33b582208884519c0e32876a31291544): No space left in cache to reserve 170 bytes, will continue without cache download 2024.10.22 01:44:45.208397 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69683 2024.10.22 01:44:45.369419 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 527 2024.10.22 01:44:45.523045 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_125_129_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_129_129_0, total 38 rows starting from the beginning of the part 2024.10.22 01:44:45.575460 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 8.61 MiB, will set to 1.66 GiB (RSS), difference: -13.49 MiB 2024.10.22 01:44:45.576304 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:45.582332 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:45.684691 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_509_509_0 (state Temporary) 2024.10.22 01:44:45.748581 [ 1205 ] {} DatabaseCatalog: Execution took 295 ms. 2024.10.22 01:44:45.792569 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} MergeTask::MergeProjectionsStage: Merge sorted 6859 rows, containing 4 columns (4 merged, 0 gathered) in 1.596379625 sec., 4296.59705785834 rows/sec., 67.13 KiB/sec. 2024.10.22 01:44:45.959214 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} MergedBlockOutputStream: filled checksums 202410_605_610_1 (state Temporary) 2024.10.22 01:44:45.963583 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_509_509_0 to 202410_519_519_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:46.031259 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_605_610_1 to 202410_605_610_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:46.041329 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_605_610_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_605_605_0, 202410_610_610_0] -> 202410_605_610_1 2024.10.22 01:44:46.078140 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:44:46.165072 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(29bb35941070e25ad66fbbdd0e920210) : [0, 236605]: Setting download as failed: File segment: [0, 236605], key: 29bb35941070e25ad66fbbdd0e920210, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:46.165925 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(29bb35941070e25ad66fbbdd0e920210): No space left in cache to reserve 236606 bytes, will continue without cache download 2024.10.22 01:44:46.248186 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 527 2024.10.22 01:44:46.281391 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 29bb35941070e25ad66fbbdd0e920210, offset: 0, size: 0 2024.10.22 01:44:46.284112 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 29bb35941070e25ad66fbbdd0e920210 for removal 2024.10.22 01:44:46.286685 [ 1476 ] {} CacheMetadata: Key 29bb35941070e25ad66fbbdd0e920210 is removed from metadata 2024.10.22 01:44:46.289788 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 33b582208884519c0e32876a31291544, offset: 0, size: 0 2024.10.22 01:44:46.295360 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 33b582208884519c0e32876a31291544 for removal 2024.10.22 01:44:46.304051 [ 1476 ] {} CacheMetadata: Key 33b582208884519c0e32876a31291544 is removed from metadata 2024.10.22 01:44:46.978305 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 9659 2024.10.22 01:44:47.184690 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:47.188595 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:47.241592 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_630_630_0 (state Temporary) 2024.10.22 01:44:47.400746 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_630_630_0 to 202410_643_643_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:47.495268 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(50f976760ac2fc3ad1ef653c3ffce49c) : [0, 193]: Setting download as failed: File segment: [0, 193], key: 50f976760ac2fc3ad1ef653c3ffce49c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:47.498320 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(50f976760ac2fc3ad1ef653c3ffce49c): No space left in cache to reserve 194 bytes, will continue without cache download 2024.10.22 01:44:47.561584 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9659 2024.10.22 01:44:47.665125 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 9.62 MiB, will set to 1.66 GiB (RSS), difference: 4.89 MiB 2024.10.22 01:44:48.277032 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c0c7b1ed0c29b9380e8d8db9a0345214) : [0, 1794231]: Setting download as failed: File segment: [0, 1794231], key: c0c7b1ed0c29b9380e8d8db9a0345214, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:48.278772 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c0c7b1ed0c29b9380e8d8db9a0345214): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:44:48.547130 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c0c7b1ed0c29b9380e8d8db9a0345214, offset: 0, size: 0 2024.10.22 01:44:48.552833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c0c7b1ed0c29b9380e8d8db9a0345214 for removal 2024.10.22 01:44:48.555517 [ 1476 ] {} CacheMetadata: Key c0c7b1ed0c29b9380e8d8db9a0345214 is removed from metadata 2024.10.22 01:44:48.558825 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 50f976760ac2fc3ad1ef653c3ffce49c, offset: 0, size: 0 2024.10.22 01:44:48.559975 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 50f976760ac2fc3ad1ef653c3ffce49c for removal 2024.10.22 01:44:48.561431 [ 1476 ] {} CacheMetadata: Key 50f976760ac2fc3ad1ef653c3ffce49c is removed from metadata 2024.10.22 01:44:49.219623 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 69737 2024.10.22 01:44:49.467490 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(431b6dca2ac72b4eacd8390f364163ef) : [0, 143]: Setting download as failed: File segment: [0, 143], key: 431b6dca2ac72b4eacd8390f364163ef, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:49.468564 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(431b6dca2ac72b4eacd8390f364163ef): No space left in cache to reserve 144 bytes, will continue without cache download 2024.10.22 01:44:49.527803 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 9.99 MiB, will set to 1.66 GiB (RSS), difference: 3.54 MiB 2024.10.22 01:44:49.592679 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:49.607150 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:49.779797 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1064_1064_0 (state Temporary) 2024.10.22 01:44:49.820555 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1064_1064_0 to 202410_1077_1077_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:49.896541 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69737 2024.10.22 01:44:50.233154 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a375614c8f3e9d3c1e2d9c213c2952cb) : [0, 322123]: Setting download as failed: File segment: [0, 322123], key: a375614c8f3e9d3c1e2d9c213c2952cb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:50.236072 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a375614c8f3e9d3c1e2d9c213c2952cb): No space left in cache to reserve 322124 bytes, will continue without cache download 2024.10.22 01:44:50.270465 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a375614c8f3e9d3c1e2d9c213c2952cb, offset: 0, size: 0 2024.10.22 01:44:50.272258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a375614c8f3e9d3c1e2d9c213c2952cb for removal 2024.10.22 01:44:50.274141 [ 1476 ] {} CacheMetadata: Key a375614c8f3e9d3c1e2d9c213c2952cb is removed from metadata 2024.10.22 01:44:50.275588 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 431b6dca2ac72b4eacd8390f364163ef, offset: 0, size: 0 2024.10.22 01:44:50.277109 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 431b6dca2ac72b4eacd8390f364163ef for removal 2024.10.22 01:44:50.281335 [ 1476 ] {} CacheMetadata: Key 431b6dca2ac72b4eacd8390f364163ef is removed from metadata 2024.10.22 01:44:50.332268 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_127_127_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:44:50.569158 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_604_118/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:44:50.673525 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:44:50.676359 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:44:50.765755 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 92 entries to flush up to offset 71326 2024.10.22 01:44:50.925219 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:51.030508 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:51.083102 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_619_619_0 (state Temporary) 2024.10.22 01:44:51.192521 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(8471f8fe03e84a1ecaca27b0837643de) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8471f8fe03e84a1ecaca27b0837643de, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:44:51.199083 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 31607, file segment info: File segment: [0, 33554431], key: 8471f8fe03e84a1ecaca27b0837643de, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:44:51.213030 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:51.275951 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 934076 2024.10.22 01:44:51.280870 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_619_619_0 to 202410_625_625_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:51.437427 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 71326 2024.10.22 01:44:52.012223 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:52.017038 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:52.152521 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_604_604_0 (state Temporary) 2024.10.22 01:44:52.233022 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7ce1d59794e030a50823734f00aa0a95) : [0, 140]: Setting download as failed: File segment: [0, 140], key: 7ce1d59794e030a50823734f00aa0a95, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:52.239334 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7ce1d59794e030a50823734f00aa0a95): No space left in cache to reserve 141 bytes, will continue without cache download 2024.10.22 01:44:52.466056 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_604_604_0 to 202410_611_611_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:52.499450 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 934076 2024.10.22 01:44:52.979739 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(ba5e7351c77d231338f1f07b88190d28) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ba5e7351c77d231338f1f07b88190d28, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:44:52.992440 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 483, file segment info: File segment: [0, 33554431], key: ba5e7351c77d231338f1f07b88190d28, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 01:44:52.994281 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:44:53.030101 [ 1720 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/cra/ngvjlhmcgragjawzxhqxoxbysrcit, size 31607 2024.10.22 01:44:53.221538 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_127_127_0 to 202410_130_130_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:53.495369 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column HitColor 2024.10.22 01:44:53.523559 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column HitColor 2024.10.22 01:44:53.524713 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_604_118} MergeTask::MergeProjectionsStage: Merge sorted 939652 rows, containing 4 columns (4 merged, 0 gathered) in 26.44238789 sec., 35535.82240412403 rows/sec., 532.86 KiB/sec. 2024.10.22 01:44:53.536813 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column HitColor 2024.10.22 01:44:53.671876 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column HitColor 2024.10.22 01:44:53.678662 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column HitColor 2024.10.22 01:44:53.683704 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column HitColor 2024.10.22 01:44:53.720985 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column HitColor 2024.10.22 01:44:53.753204 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 528 2024.10.22 01:44:53.766491 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(44dd741e019f8ea0b94a19dee8cbbe63) : [0, 45025]: Setting download as failed: File segment: [0, 45025], key: 44dd741e019f8ea0b94a19dee8cbbe63, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:44:53.828014 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(44dd741e019f8ea0b94a19dee8cbbe63): No space left in cache to reserve 45026 bytes, will continue without cache download 2024.10.22 01:44:53.842913 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column HitColor 2024.10.22 01:44:53.862796 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 44dd741e019f8ea0b94a19dee8cbbe63, offset: 0, size: 0 2024.10.22 01:44:53.872040 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 44dd741e019f8ea0b94a19dee8cbbe63 for removal 2024.10.22 01:44:53.874299 [ 1476 ] {} CacheMetadata: Key 44dd741e019f8ea0b94a19dee8cbbe63 is removed from metadata 2024.10.22 01:44:53.876464 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7ce1d59794e030a50823734f00aa0a95, offset: 0, size: 0 2024.10.22 01:44:53.878075 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7ce1d59794e030a50823734f00aa0a95 for removal 2024.10.22 01:44:53.879872 [ 1476 ] {} CacheMetadata: Key 7ce1d59794e030a50823734f00aa0a95 is removed from metadata 2024.10.22 01:44:53.925172 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 69798 2024.10.22 01:44:53.980473 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/sdt/eqiqlfebhzghwrtwhbrtofcpjhqpv (LogSeriesLimiter: on interval from 2024-10-22 01:44:43 to 2024-10-22 01:44:53 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:44:54.025278 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_604_118} MergedBlockOutputStream: filled checksums 202410_1_604_118 (state Temporary) 2024.10.22 01:44:54.086180 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:54.128994 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:54.204556 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_510_510_0 (state Temporary) 2024.10.22 01:44:54.221757 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:54.231104 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:54.276064 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_510_510_0 to 202410_520_520_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:54.290702 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1065_1065_0 (state Temporary) 2024.10.22 01:44:54.374861 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 528 2024.10.22 01:44:54.453618 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 6.62 MiB, will set to 1.66 GiB (RSS), difference: 3.57 MiB 2024.10.22 01:44:54.567437 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1065_1065_0 to 202410_1078_1078_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:54.752416 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69798 2024.10.22 01:44:54.801021 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_6_6_0 to 201403_6_6_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:54.874453 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:44:55.064812 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 9667 2024.10.22 01:44:55.284157 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:55.287582 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:55.351258 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_971_481, 202410_972_972_0] 2024.10.22 01:44:55.356411 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_971_481, 202410_972_972_0] 2024.10.22 01:44:55.367123 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_971_481, 202410_972_972_0] 2024.10.22 01:44:55.432088 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_631_631_0 (state Temporary) 2024.10.22 01:44:55.478649 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_631_631_0 to 202410_644_644_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:55.648159 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9667 2024.10.22 01:44:55.962820 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_604_118} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_604_118 to 202410_1_604_118 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:55.972985 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_604_118} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_598_117, 202410_599_604_1] -> 202410_1_604_118 2024.10.22 01:44:56.042372 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:44:56.060974 [ 1163 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 261 ms. 2024.10.22 01:44:56.504830 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.03 GiB, peak 1.97 GiB, free memory in arenas 6.95 MiB, will set to 1.39 GiB (RSS), difference: 360.44 MiB 2024.10.22 01:44:56.544968 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_973_973_0, 202410_974_974_0, 202410_975_975_0, 202410_976_976_0, 202410_977_977_0, 202410_978_978_0] 2024.10.22 01:44:56.546662 [ 1186 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:44:56.548841 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_973_973_0, 202410_974_974_0, 202410_975_975_0, 202410_976_976_0, 202410_977_977_0, 202410_978_978_0] 2024.10.22 01:44:56.676096 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_973_973_0, 202410_974_974_0, 202410_975_975_0, 202410_976_976_0, 202410_977_977_0, 202410_978_978_0] 2024.10.22 01:44:56.682917 [ 1186 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:44:56.956522 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5083 2024.10.22 01:44:57.413763 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.97 GiB, free memory in arenas 9.58 MiB, will set to 1.20 GiB (RSS), difference: -254.32 MiB 2024.10.22 01:44:58.759531 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 37 entries to flush up to offset 69835 2024.10.22 01:44:58.954563 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 55 entries to flush up to offset 71381 2024.10.22 01:44:58.957792 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:59.061985 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:59.137489 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1066_1066_0 (state Temporary) 2024.10.22 01:44:59.170683 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1066_1066_0 to 202410_1079_1079_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:59.264102 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:59.266342 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:59.322862 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_620_620_0 (state Temporary) 2024.10.22 01:44:59.328525 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69835 2024.10.22 01:44:59.360325 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 1.97 GiB, free memory in arenas 9.96 MiB, will set to 1.26 GiB (RSS), difference: -139.98 MiB 2024.10.22 01:44:59.387064 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_620_620_0 to 202410_626_626_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:44:59.419333 [ 1264 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 5 parts from 202410_507_621_23 to 202410_625_625_0 2024.10.22 01:44:59.443772 [ 1264 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:44:59.450104 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 71381 2024.10.22 01:44:59.470613 [ 1264 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:44:59.484482 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_625_24} MergeTask::PrepareStage: Merging 5 parts: from 202410_507_621_23 to 202410_625_625_0 into Compact with storage Full 2024.10.22 01:44:59.504925 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1610 entries to flush up to offset 935686 2024.10.22 01:44:59.543144 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_625_24} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:44:59.559908 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_625_24} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_621_23, total 11856 rows starting from the beginning of the part 2024.10.22 01:44:59.576083 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_625_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_622_622_0, total 56 rows starting from the beginning of the part 2024.10.22 01:44:59.587709 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_625_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_623_623_0, total 104 rows starting from the beginning of the part 2024.10.22 01:44:59.614557 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_625_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_624_624_0, total 93 rows starting from the beginning of the part 2024.10.22 01:44:59.736729 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_625_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_625_625_0, total 92 rows starting from the beginning of the part 2024.10.22 01:45:00.387754 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:00.391422 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:00.473191 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 1.97 GiB, free memory in arenas 7.91 MiB, will set to 1.30 GiB (RSS), difference: -77.37 MiB 2024.10.22 01:45:00.571067 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_605_605_0 (state Temporary) 2024.10.22 01:45:00.674871 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:45:00.677255 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:45:00.763511 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_562_11, 202410_563_563_0, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0, 202410_567_567_0] 2024.10.22 01:45:00.766263 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_562_11, 202410_563_563_0, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0, 202410_567_567_0] 2024.10.22 01:45:00.772437 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_605_605_0 to 202410_612_612_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:00.784148 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 38 2024.10.22 01:45:00.800217 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_562_11, 202410_563_563_0, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0, 202410_567_567_0] 2024.10.22 01:45:00.816456 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 935686 2024.10.22 01:45:00.943501 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:00.953288 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:01.104153 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_32_32_0 (state Temporary) 2024.10.22 01:45:01.288644 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_32_32_0 to 202410_32_32_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:01.355626 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 38 2024.10.22 01:45:01.357267 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Found 6 old parts to remove. Parts: [202410_1_25_6, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0] 2024.10.22 01:45:01.359651 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 6 parts from filesystem (serially): Parts: [202410_1_25_6, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0] 2024.10.22 01:45:01.378322 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 6 parts from memory: Parts: [202410_1_25_6, 202410_26_26_0, 202410_27_27_0, 202410_28_28_0, 202410_29_29_0, 202410_30_30_0] 2024.10.22 01:45:01.511873 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_625_24} MergeTask::MergeProjectionsStage: Merge sorted 12201 rows, containing 13 columns (13 merged, 0 gathered) in 2.031131438 sec., 6006.996776148565 rows/sec., 2.61 MiB/sec. 2024.10.22 01:45:01.559903 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_625_24} MergedBlockOutputStream: filled checksums 202410_507_625_24 (state Temporary) 2024.10.22 01:45:01.879579 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 529 2024.10.22 01:45:02.167005 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:02.171682 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:02.329100 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_625_24} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_625_24 to 202410_507_625_24 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:02.338200 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_625_24} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 5 parts: [202410_507_621_23, 202410_625_625_0] -> 202410_507_625_24 2024.10.22 01:45:02.360004 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_511_511_0 (state Temporary) 2024.10.22 01:45:02.381563 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.46 MiB. 2024.10.22 01:45:02.605540 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 1.97 GiB, free memory in arenas 5.20 MiB, will set to 1.39 GiB (RSS), difference: -67.09 MiB 2024.10.22 01:45:02.759668 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_511_511_0 to 202410_521_521_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:02.960136 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 529 2024.10.22 01:45:03.150357 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 9682 2024.10.22 01:45:03.277791 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:03.283011 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:03.328323 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_632_632_0 (state Temporary) 2024.10.22 01:45:03.344637 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 69895 2024.10.22 01:45:03.480245 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_632_632_0 to 202410_645_645_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:03.611215 [ 1249 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_640_128 to 202410_645_645_0 2024.10.22 01:45:03.626493 [ 1249 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:03.629613 [ 1249 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:03.639776 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9682 2024.10.22 01:45:03.733427 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:03.740515 [ 1249 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 250 ms. 2024.10.22 01:45:03.743370 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:03.764249 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_640_128 to 202410_645_645_0 into Compact with storage Full 2024.10.22 01:45:03.827644 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1067_1067_0 (state Temporary) 2024.10.22 01:45:03.985001 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:45:04.007407 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1067_1067_0 to 202410_1080_1080_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:04.048920 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1065_1075_2 to 202410_1080_1080_0 2024.10.22 01:45:04.066810 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:04.069191 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:04.102563 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_1065_1075_2 to 202410_1080_1080_0 into Compact with storage Full 2024.10.22 01:45:04.114164 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_640_128, total 9722 rows starting from the beginning of the part 2024.10.22 01:45:04.173801 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69895 2024.10.22 01:45:04.227214 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:45:04.228888 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_641_641_0, total 5 rows starting from the beginning of the part 2024.10.22 01:45:04.247626 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1065_1075_2, total 725 rows starting from the beginning of the part 2024.10.22 01:45:04.303638 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1076_1076_0, total 97 rows starting from the beginning of the part 2024.10.22 01:45:04.369274 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1077_1077_0, total 54 rows starting from the beginning of the part 2024.10.22 01:45:04.480874 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 5123 2024.10.22 01:45:04.506758 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_642_642_0, total 11 rows starting from the beginning of the part 2024.10.22 01:45:04.507445 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1078_1078_0, total 61 rows starting from the beginning of the part 2024.10.22 01:45:04.605525 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_643_643_0, total 27 rows starting from the beginning of the part 2024.10.22 01:45:04.652163 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.97 GiB, free memory in arenas 6.25 MiB, will set to 1.43 GiB (RSS), difference: -25.26 MiB 2024.10.22 01:45:04.767437 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1079_1079_0, total 37 rows starting from the beginning of the part 2024.10.22 01:45:04.821810 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1080_1080_0, total 60 rows starting from the beginning of the part 2024.10.22 01:45:04.890966 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_644_644_0, total 8 rows starting from the beginning of the part 2024.10.22 01:45:05.057436 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_125_129_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:45:05.212523 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_645_645_0, total 15 rows starting from the beginning of the part 2024.10.22 01:45:05.778758 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:45:06.081731 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_468_96, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0, 202410_472_472_0, 202410_473_473_0] 2024.10.22 01:45:06.083773 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_468_96, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0, 202410_472_472_0, 202410_473_473_0] 2024.10.22 01:45:06.113545 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_468_96, 202410_469_469_0, 202410_470_470_0, 202410_471_471_0, 202410_472_472_0, 202410_473_473_0] 2024.10.22 01:45:06.745677 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 1.97 GiB, free memory in arenas 5.41 MiB, will set to 1.44 GiB (RSS), difference: -23.80 MiB 2024.10.22 01:45:06.850469 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} MergeTask::MergeProjectionsStage: Merge sorted 1034 rows, containing 13 columns (13 merged, 0 gathered) in 2.761335195 sec., 374.45653170693754 rows/sec., 66.22 KiB/sec. 2024.10.22 01:45:06.920332 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} MergedBlockOutputStream: filled checksums 202410_1065_1080_3 (state Temporary) 2024.10.22 01:45:06.969708 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 71481 2024.10.22 01:45:07.131617 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} MergeTask::MergeProjectionsStage: Merge sorted 9788 rows, containing 12 columns (12 merged, 0 gathered) in 3.353864216 sec., 2918.4246497831386 rows/sec., 464.38 KiB/sec. 2024.10.22 01:45:07.187388 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1065_1080_3 to 202410_1065_1080_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:07.206665 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1080_3} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1065_1075_2, 202410_1080_1080_0] -> 202410_1065_1080_3 2024.10.22 01:45:07.239819 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:45:07.275646 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} MergedBlockOutputStream: filled checksums 202410_1_645_129 (state Temporary) 2024.10.22 01:45:07.324796 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:07.329205 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:07.503990 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_621_621_0 (state Temporary) 2024.10.22 01:45:07.572432 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_645_129 to 202410_1_645_129 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:07.575011 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_645_129} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_640_128, 202410_645_645_0] -> 202410_1_645_129 2024.10.22 01:45:07.626350 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.59 MiB. 2024.10.22 01:45:07.816004 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_621_621_0 to 202410_627_627_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:07.882581 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 71481 2024.10.22 01:45:07.944064 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1855 entries to flush up to offset 937541 2024.10.22 01:45:08.182606 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 69951 2024.10.22 01:45:08.445562 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:08.457103 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:08.489524 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.97 GiB, free memory in arenas 4.25 MiB, will set to 1.47 GiB (RSS), difference: -53.27 MiB 2024.10.22 01:45:08.613772 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1068_1068_0 (state Temporary) 2024.10.22 01:45:08.660690 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1068_1068_0 to 202410_1081_1081_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:08.692268 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 69951 2024.10.22 01:45:08.750826 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:08.754859 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:08.862066 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_606_606_0 (state Temporary) 2024.10.22 01:45:09.045650 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_606_606_0 to 202410_613_613_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:09.071349 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 937541 2024.10.22 01:45:09.366504 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(7dc8a76a66d0195a4bce1b8947e874fc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7dc8a76a66d0195a4bce1b8947e874fc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:09.386896 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 742913, file segment info: File segment: [0, 33554431], key: 7dc8a76a66d0195a4bce1b8947e874fc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:09.389413 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:10.015423 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7236a0d09a2079eb47b879ae385fca51) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7236a0d09a2079eb47b879ae385fca51, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:45:10.017585 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1831, file segment info: File segment: [0, 33554431], key: 7236a0d09a2079eb47b879ae385fca51, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:45:10.025075 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:10.674946 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:45:10.677381 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:45:10.733155 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_581_116, 202410_582_582_0, 202410_583_583_0, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0] 2024.10.22 01:45:10.750912 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_581_116, 202410_582_582_0, 202410_583_583_0, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0] 2024.10.22 01:45:10.775665 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_581_116, 202410_582_582_0, 202410_583_583_0, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0] 2024.10.22 01:45:10.838273 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(e2889a3b4c31f232bef8260255a69a2e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e2889a3b4c31f232bef8260255a69a2e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:10.843173 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 511, file segment info: File segment: [0, 33554431], key: e2889a3b4c31f232bef8260255a69a2e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:10.847392 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:11.098767 [ 1729 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/sdt/eqiqlfebhzghwrtwhbrtofcpjhqpv, size 742913 2024.10.22 01:45:11.142485 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 9701 2024.10.22 01:45:11.265110 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_551_551_0, 202410_552_552_0, 202410_553_553_0, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0] 2024.10.22 01:45:11.267188 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_551_551_0, 202410_552_552_0, 202410_553_553_0, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0] 2024.10.22 01:45:11.306619 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_551_551_0, 202410_552_552_0, 202410_553_553_0, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0] 2024.10.22 01:45:11.378825 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:11.381742 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:11.424041 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_633_633_0 (state Temporary) 2024.10.22 01:45:11.472642 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_633_633_0 to 202410_646_646_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:11.514162 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9701 2024.10.22 01:45:11.522184 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.97 GiB, free memory in arenas 4.40 MiB, will set to 1.51 GiB (RSS), difference: -13.13 MiB 2024.10.22 01:45:11.602595 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(fde4cb8e9165de7e384ca85d7107e283) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fde4cb8e9165de7e384ca85d7107e283, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:45:11.608117 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 238, file segment info: File segment: [0, 33554431], key: fde4cb8e9165de7e384ca85d7107e283, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:45:11.611381 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:11.682184 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:45:11.685305 [ 1165 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:45:11.688927 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:45:11.717356 [ 1165 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:45:11.830486 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:45:11.854279 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:45:11.879129 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:45:11.921265 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:45:11.940209 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:45:12.006038 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 01:45:12.076160 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xjm/pzwphhdzeutvbzrqcjlhupdtvebnv (LogSeriesLimiter: on interval from 2024-10-22 01:44:53 to 2024-10-22 01:45:12 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:45:12.710747 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 70002 2024.10.22 01:45:12.932585 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:12.939128 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:12.943896 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(478045f753180b6c9272e4b5eccc2cf0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 478045f753180b6c9272e4b5eccc2cf0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:45:12.962643 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 773, file segment info: File segment: [0, 33554431], key: 478045f753180b6c9272e4b5eccc2cf0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:45:12.966453 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:12.978723 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1069_1069_0 (state Temporary) 2024.10.22 01:45:13.116783 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1069_1069_0 to 202410_1082_1082_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:13.180929 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70002 2024.10.22 01:45:13.477578 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 3.21 MiB, will set to 1.56 GiB (RSS), difference: -86.50 MiB 2024.10.22 01:45:13.739417 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(00a847497a02870c8b66f7da01000a21) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 00a847497a02870c8b66f7da01000a21, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:45:13.755067 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 532, file segment info: File segment: [0, 33554431], key: 00a847497a02870c8b66f7da01000a21, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:45:13.758831 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:13.885864 [ 1687 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hag/kdrgaugrmlrikyuuedjlbwuwlcxfv, size 773 2024.10.22 01:45:14.080663 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:45:14.214680 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:45:14.246326 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:45:14.267533 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:45:14.310418 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:45:14.368192 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:45:14.409831 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:45:14.449719 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:45:14.466094 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 7 old parts to remove. Parts: [202410_1_104_21, 202410_105_109_1, 202410_110_110_0, 202410_111_111_0, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0] 2024.10.22 01:45:14.468017 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from filesystem (serially): Parts: [202410_1_104_21, 202410_105_109_1, 202410_110_110_0, 202410_111_111_0, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0] 2024.10.22 01:45:14.570971 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 01:45:14.604174 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from memory: Parts: [202410_1_104_21, 202410_105_109_1, 202410_110_110_0, 202410_111_111_0, 202410_112_112_0, 202410_113_113_0, 202410_114_114_0] 2024.10.22 01:45:15.410904 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 80 entries to flush up to offset 71561 2024.10.22 01:45:15.576156 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:15.579287 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:15.581199 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 5.17 MiB, will set to 1.62 GiB (RSS), difference: -30.96 MiB 2024.10.22 01:45:15.582181 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(392c35869a2db4990578a9e68bea478e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 392c35869a2db4990578a9e68bea478e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, kind: Regular, unbound: 0 2024.10.22 01:45:15.616152 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 392c35869a2db4990578a9e68bea478e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, kind: Regular, unbound: 0 2024.10.22 01:45:15.622646 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:15.842460 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_622_622_0 (state Temporary) 2024.10.22 01:45:15.985109 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_622_622_0 to 202410_628_628_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:16.076222 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1476 entries to flush up to offset 939017 2024.10.22 01:45:16.250258 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 71561 2024.10.22 01:45:16.444532 [ 1169 ] {} DatabaseCatalog: Execution took 367 ms. 2024.10.22 01:45:16.788107 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:16.796752 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:17.141640 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_607_607_0 (state Temporary) 2024.10.22 01:45:17.187709 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 70043 2024.10.22 01:45:17.274123 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_607_607_0 to 202410_614_614_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:17.317062 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 939017 2024.10.22 01:45:17.310976 [ 1196 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_604_118 to 202410_605_610_1 2024.10.22 01:45:17.343720 [ 1196 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:17.349069 [ 1196 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:17.477177 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_610_119} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_604_118 to 202410_605_610_1 into Wide with storage Full 2024.10.22 01:45:17.509807 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_610_119} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:45:17.518843 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:17.521767 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:17.522874 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_610_119} MergeTreeSequentialSource: Reading 116 marks from part 202410_1_604_118, total 939652 rows starting from the beginning of the part 2024.10.22 01:45:17.575314 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_610_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_605_610_1, total 6859 rows starting from the beginning of the part 2024.10.22 01:45:17.606259 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1070_1070_0 (state Temporary) 2024.10.22 01:45:17.637000 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 5.07 MiB, will set to 1.68 GiB (RSS), difference: -32.75 MiB 2024.10.22 01:45:17.673362 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1070_1070_0 to 202410_1083_1083_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:17.718512 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70043 2024.10.22 01:45:17.962105 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 530 2024.10.22 01:45:18.248807 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_610_119/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:45:18.344892 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:18.352010 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:18.523614 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_512_512_0 (state Temporary) 2024.10.22 01:45:18.749920 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_512_512_0 to 202410_522_522_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:18.932954 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 530 2024.10.22 01:45:19.015801 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 9707 2024.10.22 01:45:19.163205 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:19.165242 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:19.218272 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_634_634_0 (state Temporary) 2024.10.22 01:45:19.390194 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_634_634_0 to 202410_647_647_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:19.437063 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 5.85 MiB, will set to 1.71 GiB (RSS), difference: -8.57 MiB 2024.10.22 01:45:19.460338 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9707 2024.10.22 01:45:19.466758 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:45:19.777703 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 245.84 MiB using storage policy from min volume index 0 2024.10.22 01:45:20.071129 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_544_108, 202410_545_550_1] 2024.10.22 01:45:20.072758 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_544_108, 202410_545_550_1] 2024.10.22 01:45:20.089157 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_544_108, 202410_545_550_1] 2024.10.22 01:45:20.139534 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_125_129_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:45:20.546340 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nvl/bfggxsvoyhnbenrqkbathapkzjvlb (LogSeriesLimiter: on interval from 2024-10-22 01:45:12 to 2024-10-22 01:45:20 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:45:20.676185 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:45:20.680618 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:45:20.725070 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 8.13 MiB, will set to 1.72 GiB (RSS), difference: -9.79 MiB 2024.10.22 01:45:21.423205 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:21.425078 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:21.737492 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 70085 2024.10.22 01:45:21.972079 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:21.976232 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:22.032197 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1071_1071_0 (state Temporary) 2024.10.22 01:45:22.081381 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_972_482, 202410_973_978_1] 2024.10.22 01:45:22.083933 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1071_1071_0 to 202410_1084_1084_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:22.084254 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_972_482, 202410_973_978_1] 2024.10.22 01:45:22.101353 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_972_482, 202410_973_978_1] 2024.10.22 01:45:22.132777 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70085 2024.10.22 01:45:22.358139 [ 1239 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 251 ms. 2024.10.22 01:45:22.465404 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.86 GiB, peak 1.97 GiB, free memory in arenas 10.95 MiB, will set to 1.72 GiB (RSS), difference: -147.25 MiB 2024.10.22 01:45:23.763939 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 85 entries to flush up to offset 71646 2024.10.22 01:45:23.882779 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:23.885009 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:45:23.955059 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_623_623_0 (state Temporary) 2024.10.22 01:45:23.990937 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_623_623_0 to 202410_629_629_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:24.076433 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:45:24.192102 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 71646 2024.10.22 01:45:24.296880 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.86 GiB, peak 1.97 GiB, free memory in arenas 12.29 MiB, will set to 1.72 GiB (RSS), difference: -141.41 MiB 2024.10.22 01:45:24.322975 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1604 entries to flush up to offset 940621 2024.10.22 01:45:24.604881 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_128_128_0 (state Temporary) 2024.10.22 01:45:25.012901 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ehr/kjgsncfkstvdxtztourucumoekghr (LogSeriesLimiter: on interval from 2024-10-22 01:45:20 to 2024-10-22 01:45:25 accepted series 1 / 240 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:45:25.573323 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:25.576231 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:45:25.586780 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.83 GiB, peak 1.97 GiB, free memory in arenas 11.39 MiB, will set to 1.72 GiB (RSS), difference: -121.21 MiB 2024.10.22 01:45:26.084622 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_608_608_0 (state Temporary) 2024.10.22 01:45:26.147471 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 70111 2024.10.22 01:45:26.360658 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_608_608_0 to 202410_615_615_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:26.365985 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:26.369176 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:45:26.409265 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 940621 2024.10.22 01:45:26.409386 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1072_1072_0 (state Temporary) 2024.10.22 01:45:26.451450 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 531 2024.10.22 01:45:26.479442 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1072_1072_0 to 202410_1085_1085_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:26.514535 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1065_1080_3 to 202410_1085_1085_0 2024.10.22 01:45:26.546297 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70111 2024.10.22 01:45:26.555743 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:26.560901 [ 1241 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:45:26.690725 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_1065_1080_3 to 202410_1085_1085_0 into Compact with storage Full 2024.10.22 01:45:26.722286 [ 1205 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:45:26.754208 [ 1205 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:45:26.763056 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:45:26.776265 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1065_1080_3, total 1034 rows starting from the beginning of the part 2024.10.22 01:45:26.796395 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1081_1081_0, total 56 rows starting from the beginning of the part 2024.10.22 01:45:26.825443 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:26.832437 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:26.851388 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1082_1082_0, total 51 rows starting from the beginning of the part 2024.10.22 01:45:26.881730 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1083_1083_0, total 41 rows starting from the beginning of the part 2024.10.22 01:45:26.920448 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1084_1084_0, total 42 rows starting from the beginning of the part 2024.10.22 01:45:26.962716 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 9712 2024.10.22 01:45:26.973304 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1085_1085_0, total 26 rows starting from the beginning of the part 2024.10.22 01:45:27.144754 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_513_513_0 (state Temporary) 2024.10.22 01:45:27.233032 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_978_483, 202410_979_979_0] 2024.10.22 01:45:27.239541 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_978_483, 202410_979_979_0] 2024.10.22 01:45:27.255217 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_978_483, 202410_979_979_0] 2024.10.22 01:45:27.263379 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_513_513_0 to 202410_523_523_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:27.296580 [ 1224 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_518_106 to 202410_523_523_0 2024.10.22 01:45:27.328943 [ 1224 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:27.333754 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:45:27.382658 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_518_106 to 202410_523_523_0 into Compact with storage Full 2024.10.22 01:45:27.387045 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 531 2024.10.22 01:45:27.488522 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:27.495460 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:45:27.573002 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_635_635_0 (state Temporary) 2024.10.22 01:45:27.580148 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:45:27.607496 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_518_106, total 562 rows starting from the beginning of the part 2024.10.22 01:45:27.643385 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_635_635_0 to 202410_648_648_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:27.765771 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.85 GiB, peak 1.97 GiB, free memory in arenas 9.16 MiB, will set to 1.72 GiB (RSS), difference: -136.52 MiB 2024.10.22 01:45:27.766388 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_519_519_0, total 1 rows starting from the beginning of the part 2024.10.22 01:45:27.863365 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9712 2024.10.22 01:45:27.927176 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_520_520_0, total 1 rows starting from the beginning of the part 2024.10.22 01:45:28.000135 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_521_521_0, total 1 rows starting from the beginning of the part 2024.10.22 01:45:28.058983 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_522_522_0, total 1 rows starting from the beginning of the part 2024.10.22 01:45:28.131331 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_523_523_0, total 1 rows starting from the beginning of the part 2024.10.22 01:45:28.469429 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} MergeTask::MergeProjectionsStage: Merge sorted 1250 rows, containing 13 columns (13 merged, 0 gathered) in 1.796749519 sec., 695.700756717442 rows/sec., 123.24 KiB/sec. 2024.10.22 01:45:28.653668 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(27b99a928414d3044c04a84bffd6e2f9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 27b99a928414d3044c04a84bffd6e2f9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:45:28.657464 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 981, file segment info: File segment: [0, 33554431], key: 27b99a928414d3044c04a84bffd6e2f9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:45:28.660321 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} MergedBlockOutputStream: filled checksums 202410_1065_1085_4 (state Temporary) 2024.10.22 01:45:28.760017 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:28.908217 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_610_119/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:45:28.921722 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column Age 2024.10.22 01:45:28.933497 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column Age 2024.10.22 01:45:28.947842 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1065_1085_4 to 202410_1065_1085_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:29.040793 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1085_4} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1065_1080_3, 202410_1085_1085_0] -> 202410_1065_1085_4 2024.10.22 01:45:29.130569 [ 1694 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xjm/pzwphhdzeutvbzrqcjlhupdtvebnv, size 10246423 2024.10.22 01:45:29.149526 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column Age 2024.10.22 01:45:29.172060 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column Age 2024.10.22 01:45:29.177127 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:45:29.191750 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column Age 2024.10.22 01:45:29.247713 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column Age 2024.10.22 01:45:29.265177 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column Age 2024.10.22 01:45:29.387645 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column Age 2024.10.22 01:45:29.617519 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} MergeTask::MergeProjectionsStage: Merge sorted 567 rows, containing 32 columns (32 merged, 0 gathered) in 2.240366836 sec., 253.0835535007 rows/sec., 23.21 KiB/sec. 2024.10.22 01:45:29.773377 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} MergedBlockOutputStream: filled checksums 202410_1_523_107 (state Temporary) 2024.10.22 01:45:30.285801 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_523_107 to 202410_1_523_107 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:30.295791 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_523_107} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_518_106, 202410_523_523_0] -> 202410_1_523_107 2024.10.22 01:45:30.347948 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:45:30.468766 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 11.48 MiB, will set to 1.72 GiB (RSS), difference: 17.94 MiB 2024.10.22 01:45:30.551774 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 70183 2024.10.22 01:45:30.676952 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:45:30.783423 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:45:30.961815 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:30.966275 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:31.121995 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1073_1073_0 (state Temporary) 2024.10.22 01:45:31.284994 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1073_1073_0 to 202410_1086_1086_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:31.447633 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70183 2024.10.22 01:45:31.710565 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 482 entries to flush up to offset 72128 2024.10.22 01:45:32.560632 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:32.570067 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_979_484, 202410_980_980_0] 2024.10.22 01:45:32.568750 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:32.582237 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_979_484, 202410_980_980_0] 2024.10.22 01:45:32.627310 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_979_484, 202410_980_980_0] 2024.10.22 01:45:32.710955 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 13.31 MiB, will set to 1.73 GiB (RSS), difference: 7.64 MiB 2024.10.22 01:45:32.888855 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_624_624_0 (state Temporary) 2024.10.22 01:45:32.978656 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_624_624_0 to 202410_630_630_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:33.007304 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_625_24 to 202410_630_630_0 2024.10.22 01:45:33.024671 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:33.039239 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 72128 2024.10.22 01:45:33.037805 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:33.172227 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_625_24 to 202410_630_630_0 into Compact with storage Full 2024.10.22 01:45:33.298785 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:45:33.311265 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_625_24, total 12201 rows starting from the beginning of the part 2024.10.22 01:45:33.413405 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 942546 2024.10.22 01:45:33.468480 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_626_626_0, total 55 rows starting from the beginning of the part 2024.10.22 01:45:33.616243 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_627_627_0, total 100 rows starting from the beginning of the part 2024.10.22 01:45:33.660351 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_628_628_0, total 80 rows starting from the beginning of the part 2024.10.22 01:45:33.565482 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_125_129_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:45:33.688232 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_128_128_0 to 202410_131_131_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:33.785701 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_629_629_0, total 85 rows starting from the beginning of the part 2024.10.22 01:45:33.831213 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_630_630_0, total 482 rows starting from the beginning of the part 2024.10.22 01:45:34.387275 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:34.493614 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:34.615961 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 14.34 MiB, will set to 1.73 GiB (RSS), difference: -8.99 MiB 2024.10.22 01:45:34.885237 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_609_609_0 (state Temporary) 2024.10.22 01:45:34.888459 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 532 2024.10.22 01:45:34.974279 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_609_609_0 to 202410_616_616_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:35.019194 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 942546 2024.10.22 01:45:35.032392 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_611_611_0 to 202410_616_616_0 2024.10.22 01:45:35.055689 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:35.062254 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:35.074679 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_611_611_0 to 202410_616_616_0 into Compact with storage Full 2024.10.22 01:45:35.125413 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:45:35.143247 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_611_611_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:45:35.176637 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_612_612_0, total 1610 rows starting from the beginning of the part 2024.10.22 01:45:35.215591 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_613_613_0, total 1855 rows starting from the beginning of the part 2024.10.22 01:45:35.339136 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_614_614_0, total 1476 rows starting from the beginning of the part 2024.10.22 01:45:35.365663 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 9732 2024.10.22 01:45:35.376986 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_615_615_0, total 1604 rows starting from the beginning of the part 2024.10.22 01:45:35.424248 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_616_616_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:45:35.462801 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 70234 2024.10.22 01:45:35.488036 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:35.498533 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:35.565139 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:35.572524 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:35.573348 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_636_636_0 (state Temporary) 2024.10.22 01:45:35.671335 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_514_514_0 (state Temporary) 2024.10.22 01:45:35.744744 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} MergeTask::MergeProjectionsStage: Merge sorted 13003 rows, containing 13 columns (13 merged, 0 gathered) in 2.576248128 sec., 5047.262279854435 rows/sec., 2.19 MiB/sec. 2024.10.22 01:45:35.843175 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_636_636_0 to 202410_649_649_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:35.866455 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:35.871218 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:35.910782 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_514_514_0 to 202410_524_524_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:36.015773 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9732 2024.10.22 01:45:36.056212 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1074_1074_0 (state Temporary) 2024.10.22 01:45:36.118273 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1074_1074_0 to 202410_1087_1087_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:36.143304 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 532 2024.10.22 01:45:36.179987 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70234 2024.10.22 01:45:36.219673 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} MergedBlockOutputStream: filled checksums 202410_507_630_25 (state Temporary) 2024.10.22 01:45:36.363370 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_630_25 to 202410_507_630_25 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:36.367083 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_630_25} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_625_24, 202410_630_630_0] -> 202410_507_630_25 2024.10.22 01:45:36.476699 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_980_485, 202410_981_981_0] 2024.10.22 01:45:36.478382 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_980_485, 202410_981_981_0] 2024.10.22 01:45:36.489576 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_980_485, 202410_981_981_0] 2024.10.22 01:45:36.509768 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.80 MiB. 2024.10.22 01:45:36.694750 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 12.54 MiB, will set to 1.73 GiB (RSS), difference: 16.57 MiB 2024.10.22 01:45:37.013529 [ 1185 ] {} DatabaseCatalog: Execution took 373 ms. 2024.10.22 01:45:37.087284 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5123 2024.10.22 01:45:37.308920 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} MergeTask::MergeProjectionsStage: Merge sorted 9625 rows, containing 4 columns (4 merged, 0 gathered) in 2.241292793 sec., 4294.396533135151 rows/sec., 67.10 KiB/sec. 2024.10.22 01:45:37.460474 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} MergedBlockOutputStream: filled checksums 202410_611_616_1 (state Temporary) 2024.10.22 01:45:37.561788 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_611_616_1 to 202410_611_616_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:37.565242 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_611_616_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_611_611_0, 202410_616_616_0] -> 202410_611_616_1 2024.10.22 01:45:37.618977 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:45:38.717508 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 16.51 MiB, will set to 1.74 GiB (RSS), difference: 34.19 MiB 2024.10.22 01:45:38.921982 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:45:39.935018 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_125_129_1} MergeTask::MergeProjectionsStage: Merge sorted 197 rows, containing 679 columns (679 merged, 0 gathered) in 75.305384654 sec., 2.616014789714456 rows/sec., 3.84 KiB/sec. 2024.10.22 01:45:40.187450 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 70269 2024.10.22 01:45:40.352065 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:40.355208 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:45:40.411440 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1075_1075_0 (state Temporary) 2024.10.22 01:45:40.483069 [ 1723 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ehr/kjgsncfkstvdxtztourucumoekghr, size 142 2024.10.22 01:45:40.485152 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1075_1075_0 to 202410_1088_1088_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:40.630602 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70269 2024.10.22 01:45:40.569742 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_610_119/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:45:40.676634 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:45:40.754509 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:45:41.365657 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 114 entries to flush up to offset 72242 2024.10.22 01:45:41.431794 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_567_12, 202410_568_568_0, 202410_569_569_0, 202410_570_570_0, 202410_571_571_0, 202410_572_572_0] 2024.10.22 01:45:41.438652 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_567_12, 202410_568_568_0, 202410_569_569_0, 202410_570_570_0, 202410_571_571_0, 202410_572_572_0] 2024.10.22 01:45:41.575096 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_567_12, 202410_568_568_0, 202410_569_569_0, 202410_570_570_0, 202410_571_571_0, 202410_572_572_0] 2024.10.22 01:45:41.631028 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 1.97 GiB, free memory in arenas 15.14 MiB, will set to 1.73 GiB (RSS), difference: 202.74 MiB 2024.10.22 01:45:41.760191 [ 1209 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:45:41.831146 [ 1209 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:45:41.895834 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:41.902301 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:45:41.972868 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_625_625_0 (state Temporary) 2024.10.22 01:45:42.077653 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1344 entries to flush up to offset 943890 2024.10.22 01:45:42.084322 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_625_625_0 to 202410_631_631_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:42.263947 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_981_486, 202410_982_982_0] 2024.10.22 01:45:42.266656 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_981_486, 202410_982_982_0] 2024.10.22 01:45:42.275684 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 72242 2024.10.22 01:45:42.280595 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_981_486, 202410_982_982_0] 2024.10.22 01:45:42.977477 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_125_129_1} MergedBlockOutputStream: filled checksums 202410_125_129_1 (state Temporary) 2024.10.22 01:45:42.979914 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_8_8_0 (state Temporary) 2024.10.22 01:45:42.998977 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zxr/evqzewizfwcalaqghahqqbsylfmqk (LogSeriesLimiter: on interval from 2024-10-22 01:45:25 to 2024-10-22 01:45:42 accepted series 1 / 121 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:45:43.011535 [ 1694 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nvl/bfggxsvoyhnbenrqkbathapkzjvlb, size 245 2024.10.22 01:45:43.159510 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:43.162276 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:45:43.251284 [ 1680 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zxr/evqzewizfwcalaqghahqqbsylfmqk, size 4 2024.10.22 01:45:43.315334 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.97 GiB, free memory in arenas 12.15 MiB, will set to 1.72 GiB (RSS), difference: 96.67 MiB 2024.10.22 01:45:43.520296 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 9751 2024.10.22 01:45:43.614103 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_610_610_0 (state Temporary) 2024.10.22 01:45:43.645758 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 533 2024.10.22 01:45:43.675005 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_610_610_0 to 202410_617_617_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:43.726621 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:43.728767 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:45:43.763439 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_637_637_0 (state Temporary) 2024.10.22 01:45:43.811425 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_637_637_0 to 202410_650_650_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:43.818713 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 943890 2024.10.22 01:45:43.870942 [ 1256 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_645_129 to 202410_650_650_0 2024.10.22 01:45:43.873509 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9751 2024.10.22 01:45:43.880806 [ 1256 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:43.883760 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:45:44.011338 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_645_129 to 202410_650_650_0 into Compact with storage Full 2024.10.22 01:45:44.176667 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:45:44.191930 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_645_129, total 9788 rows starting from the beginning of the part 2024.10.22 01:45:44.251120 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_646_646_0, total 19 rows starting from the beginning of the part 2024.10.22 01:45:44.260821 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:44.264392 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:45:44.277335 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_647_647_0, total 6 rows starting from the beginning of the part 2024.10.22 01:45:44.329030 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_648_648_0, total 5 rows starting from the beginning of the part 2024.10.22 01:45:44.347417 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 15.68 MiB, will set to 1.72 GiB (RSS), difference: 2.80 MiB 2024.10.22 01:45:44.364401 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_649_649_0, total 20 rows starting from the beginning of the part 2024.10.22 01:45:44.399796 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_650_650_0, total 19 rows starting from the beginning of the part 2024.10.22 01:45:44.537700 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_515_515_0 (state Temporary) 2024.10.22 01:45:44.654948 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 70328 2024.10.22 01:45:44.740950 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 5163 2024.10.22 01:45:44.916621 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:44.921359 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:45:44.979681 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1076_1076_0 (state Temporary) 2024.10.22 01:45:45.044352 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1076_1076_0 to 202410_1089_1089_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:45.159905 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_515_515_0 to 202410_525_525_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:45.211777 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70328 2024.10.22 01:45:45.378590 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 533 2024.10.22 01:45:45.575141 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} MergeTask::MergeProjectionsStage: Merge sorted 9857 rows, containing 12 columns (12 merged, 0 gathered) in 1.57949287 sec., 6240.610633462372 rows/sec., 993.05 KiB/sec. 2024.10.22 01:45:45.663362 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(133253c8c4f70bd33d6ede93b87b42dc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 133253c8c4f70bd33d6ede93b87b42dc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:45:45.689253 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 741641, file segment info: File segment: [0, 33554431], key: 133253c8c4f70bd33d6ede93b87b42dc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:45:45.697706 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:45.720520 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} MergedBlockOutputStream: filled checksums 202410_1_650_130 (state Temporary) 2024.10.22 01:45:46.162745 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_610_119} MergeTask::MergeProjectionsStage: Merge sorted 946511 rows, containing 4 columns (4 merged, 0 gathered) in 28.691305059 sec., 32989.47182965784 rows/sec., 494.55 KiB/sec. 2024.10.22 01:45:46.396380 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_610_119} MergedBlockOutputStream: filled checksums 202410_1_610_119 (state Temporary) 2024.10.22 01:45:46.468767 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_650_130 to 202410_1_650_130 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:46.472226 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_650_130} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_645_129, 202410_650_650_0] -> 202410_1_650_130 2024.10.22 01:45:46.565192 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(e6714b10814d7a014fa5752216aa3e7c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e6714b10814d7a014fa5752216aa3e7c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:45:46.570596 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 505, file segment info: File segment: [0, 33554431], key: e6714b10814d7a014fa5752216aa3e7c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:45:46.578467 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:46.608607 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.59 MiB. 2024.10.22 01:45:47.028669 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column Sex 2024.10.22 01:45:47.060822 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column Sex 2024.10.22 01:45:47.070355 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column Sex 2024.10.22 01:45:47.086062 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column Sex 2024.10.22 01:45:47.109509 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column Sex 2024.10.22 01:45:47.159618 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column Sex 2024.10.22 01:45:47.176225 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column Sex 2024.10.22 01:45:47.201116 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column Sex 2024.10.22 01:45:47.292498 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/skr/ckusoqxpeuinsxyklmenxxqoeamdi (LogSeriesLimiter: on interval from 2024-10-22 01:45:42 to 2024-10-22 01:45:47 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:45:47.514393 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_982_487, 202410_983_983_0] 2024.10.22 01:45:47.517355 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_982_487, 202410_983_983_0] 2024.10.22 01:45:47.580125 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_982_487, 202410_983_983_0] 2024.10.22 01:45:47.641115 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_125_129_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:45:47.696715 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 12.93 MiB, will set to 1.71 GiB (RSS), difference: 2.86 MiB 2024.10.22 01:45:49.222900 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 70364 2024.10.22 01:45:49.372122 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:49.378603 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:45:49.416099 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_610_119} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_610_119 to 202410_1_610_119 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:49.424433 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_610_119} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_604_118, 202410_605_610_1] -> 202410_1_610_119 2024.10.22 01:45:49.438971 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1077_1077_0 (state Temporary) 2024.10.22 01:45:49.463721 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:45:49.476633 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1077_1077_0 to 202410_1090_1090_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:49.516866 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1065_1085_4 to 202410_1090_1090_0 2024.10.22 01:45:49.532756 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:49.539951 [ 1271 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:45:49.542646 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70364 2024.10.22 01:45:49.741095 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_1065_1085_4 to 202410_1090_1090_0 into Compact with storage Full 2024.10.22 01:45:49.819845 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:45:49.856498 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1065_1085_4, total 1250 rows starting from the beginning of the part 2024.10.22 01:45:49.888506 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 72342 2024.10.22 01:45:50.018987 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:50.024030 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:45:50.133384 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1086_1086_0, total 72 rows starting from the beginning of the part 2024.10.22 01:45:50.174489 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_626_626_0 (state Temporary) 2024.10.22 01:45:50.201111 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1087_1087_0, total 51 rows starting from the beginning of the part 2024.10.22 01:45:50.294411 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1088_1088_0, total 35 rows starting from the beginning of the part 2024.10.22 01:45:50.327311 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_586_117, 202410_587_587_0, 202410_588_588_0, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0] 2024.10.22 01:45:50.329906 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_586_117, 202410_587_587_0, 202410_588_588_0, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0] 2024.10.22 01:45:50.360851 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_586_117, 202410_587_587_0, 202410_588_588_0, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0] 2024.10.22 01:45:50.371925 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_626_626_0 to 202410_632_632_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:50.423107 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1089_1089_0, total 59 rows starting from the beginning of the part 2024.10.22 01:45:50.472785 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 72342 2024.10.22 01:45:50.647827 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1090_1090_0, total 36 rows starting from the beginning of the part 2024.10.22 01:45:50.677246 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:45:50.679253 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:45:50.822686 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2121 entries to flush up to offset 946011 2024.10.22 01:45:50.822688 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(daaa2c16eba68fae3ee1d9a10691af38) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: daaa2c16eba68fae3ee1d9a10691af38, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:50.828506 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 695534, file segment info: File segment: [0, 33554431], key: daaa2c16eba68fae3ee1d9a10691af38, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:50.832801 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:50.885635 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xxv/mlplsnnfcqoezetnxfwdbcjadwubd. 2024.10.22 01:45:51.375485 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 9767 2024.10.22 01:45:51.449288 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 5.63 MiB, will set to 1.71 GiB (RSS), difference: -7.22 MiB 2024.10.22 01:45:51.618878 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c2561e208be47209f949d1d8d2cdadb9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c2561e208be47209f949d1d8d2cdadb9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:51.627486 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 781, file segment info: File segment: [0, 33554431], key: c2561e208be47209f949d1d8d2cdadb9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:51.635745 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:51.641210 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:51.642236 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:45:51.792185 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:51.800141 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:45:51.901739 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_611_611_0 (state Temporary) 2024.10.22 01:45:51.972912 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_611_611_0 to 202410_618_618_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:52.026964 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 946011 2024.10.22 01:45:52.041612 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_638_638_0 (state Temporary) 2024.10.22 01:45:52.083638 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_638_638_0 to 202410_651_651_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:52.340417 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9767 2024.10.22 01:45:52.591640 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} MergeTask::MergeProjectionsStage: Merge sorted 1503 rows, containing 13 columns (13 merged, 0 gathered) in 2.848808125 sec., 527.5890597230026 rows/sec., 92.52 KiB/sec. 2024.10.22 01:45:52.745477 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} MergedBlockOutputStream: filled checksums 202410_1065_1090_5 (state Temporary) 2024.10.22 01:45:52.882793 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 534 2024.10.22 01:45:52.886919 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7f975de0ec48b82d05b6ec618bc05161) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7f975de0ec48b82d05b6ec618bc05161, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:52.896970 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 807764, file segment info: File segment: [0, 33554431], key: 7f975de0ec48b82d05b6ec618bc05161, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:52.904804 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:52.960990 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1065_1090_5 to 202410_1065_1090_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:52.963223 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1090_5} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1065_1085_4, 202410_1090_1090_0] -> 202410_1065_1090_5 2024.10.22 01:45:52.997633 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:45:53.263602 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:53.266808 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:45:53.555615 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 70426 2024.10.22 01:45:53.559705 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 3.42 MiB, will set to 1.70 GiB (RSS), difference: 3.55 MiB 2024.10.22 01:45:53.638979 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_516_516_0 (state Temporary) 2024.10.22 01:45:53.752859 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_516_516_0 to 202410_526_526_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:53.829303 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(24bbf8d55b6a8be1a7012948ca8ce490) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 24bbf8d55b6a8be1a7012948ca8ce490, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:53.852253 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 934, file segment info: File segment: [0, 33554431], key: 24bbf8d55b6a8be1a7012948ca8ce490, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:45:53.856177 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:45:53.956210 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 534 2024.10.22 01:45:54.144680 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_983_488, 202410_984_984_0] 2024.10.22 01:45:54.148034 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_983_488, 202410_984_984_0] 2024.10.22 01:45:54.165152 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_983_488, 202410_984_984_0] 2024.10.22 01:45:54.185183 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:54.187168 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:45:54.469228 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1078_1078_0 (state Temporary) 2024.10.22 01:45:54.626709 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1078_1078_0 to 202410_1091_1091_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:54.756454 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:45:54.793251 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:45:54.803482 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70426 2024.10.22 01:45:54.960990 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:45:55.005144 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:45:55.130475 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:45:55.207356 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:45:55.364630 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:45:55.409695 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:45:55.620829 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 01:45:55.622359 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 2.64 MiB, will set to 1.70 GiB (RSS), difference: -1.94 MiB 2024.10.22 01:45:55.698426 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vou/xyqjheuaeinxicuyzcqlxvlxywpre (LogSeriesLimiter: on interval from 2024-10-22 01:45:47 to 2024-10-22 01:45:55 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:45:56.786058 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_985_985_0, 202410_986_986_0, 202410_987_987_0, 202410_988_988_0, 202410_989_989_0, 202410_990_990_0] 2024.10.22 01:45:56.787480 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_985_985_0, 202410_986_986_0, 202410_987_987_0, 202410_988_988_0, 202410_989_989_0, 202410_990_990_0] 2024.10.22 01:45:56.813002 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_985_985_0, 202410_986_986_0, 202410_987_987_0, 202410_988_988_0, 202410_989_989_0, 202410_990_990_0] 2024.10.22 01:45:56.952949 [ 1186 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:45:56.976453 [ 1186 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:45:57.497531 [ 1169 ] {} DatabaseCatalog: Execution took 333 ms. 2024.10.22 01:45:57.504857 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 3.08 MiB, will set to 1.70 GiB (RSS), difference: -6.85 MiB 2024.10.22 01:45:57.707527 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_473_97, 202410_474_474_0, 202410_475_475_0, 202410_476_476_0, 202410_477_477_0, 202410_478_478_0] 2024.10.22 01:45:57.710068 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_473_97, 202410_474_474_0, 202410_475_475_0, 202410_476_476_0, 202410_477_477_0, 202410_478_478_0] 2024.10.22 01:45:57.850128 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_473_97, 202410_474_474_0, 202410_475_475_0, 202410_476_476_0, 202410_477_477_0, 202410_478_478_0] 2024.10.22 01:45:57.984371 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 70 entries to flush up to offset 72412 2024.10.22 01:45:58.166523 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_125_129_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_125_129_1 to 202410_125_129_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:58.170854 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_125_129_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 5 parts: [202410_125_125_0, 202410_129_129_0] -> 202410_125_129_1 2024.10.22 01:45:58.344212 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 01:45:58.451636 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:58.455309 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:45:58.458355 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 01:45:58.598749 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_627_627_0 (state Temporary) 2024.10.22 01:45:58.651268 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_627_627_0 to 202410_633_633_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:58.704203 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 72412 2024.10.22 01:45:58.864480 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:58.931547 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 70473 2024.10.22 01:45:58.935962 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.75 MiB. 2024.10.22 01:45:58.936448 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:45:59.034260 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 947551 2024.10.22 01:45:59.071567 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:59.074620 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:45:59.154847 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1079_1079_0 (state Temporary) 2024.10.22 01:45:59.190154 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1079_1079_0 to 202410_1092_1092_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:45:59.339994 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70473 2024.10.22 01:45:59.367724 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 2.63 MiB, will set to 1.70 GiB (RSS), difference: 4.80 MiB 2024.10.22 01:45:59.843784 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 9786 2024.10.22 01:45:59.857576 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:59.925734 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:45:59.978824 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:45:59.980408 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:00.124056 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_639_639_0 (state Temporary) 2024.10.22 01:46:00.287614 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_639_639_0 to 202410_652_652_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:00.378308 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_612_612_0 (state Temporary) 2024.10.22 01:46:00.466985 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9786 2024.10.22 01:46:00.474343 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_8_8_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:46:00.476705 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_612_612_0 to 202410_619_619_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:00.612298 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 947551 2024.10.22 01:46:00.680010 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:46:00.683546 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:46:01.240247 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a0d24a49019615b2230383543e1b0ea2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a0d24a49019615b2230383543e1b0ea2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, kind: Regular, unbound: 0 2024.10.22 01:46:01.242756 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 643102, file segment info: File segment: [0, 33554431], key: a0d24a49019615b2230383543e1b0ea2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, kind: Regular, unbound: 0 2024.10.22 01:46:01.246128 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:01.422661 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 3.20 MiB, will set to 1.70 GiB (RSS), difference: -2.99 MiB 2024.10.22 01:46:01.458726 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 535 2024.10.22 01:46:02.021531 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:02.034688 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:02.175511 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(4d9e3853cff0ec8694bfc942315491d8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4d9e3853cff0ec8694bfc942315491d8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, kind: Regular, unbound: 0 2024.10.22 01:46:02.178557 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 498, file segment info: File segment: [0, 33554431], key: 4d9e3853cff0ec8694bfc942315491d8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1386, kind: Regular, unbound: 0 2024.10.22 01:46:02.181664 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:02.232694 [ 1729 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/skr/ckusoqxpeuinsxyklmenxxqoeamdi, size 643102 2024.10.22 01:46:02.274873 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_517_517_0 (state Temporary) 2024.10.22 01:46:02.386796 [ 1200 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 01:46:02.432668 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_129_129_0 (state Temporary) 2024.10.22 01:46:02.664044 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_517_517_0 to 202410_527_527_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:02.834453 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 535 2024.10.22 01:46:02.927087 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column Income 2024.10.22 01:46:03.027760 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_557_557_0, 202410_558_558_0, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0, 202410_562_562_0] 2024.10.22 01:46:03.032324 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_557_557_0, 202410_558_558_0, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0, 202410_562_562_0] 2024.10.22 01:46:03.037165 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column Income 2024.10.22 01:46:03.056634 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column Income 2024.10.22 01:46:03.071159 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column Income 2024.10.22 01:46:03.079645 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column Income 2024.10.22 01:46:03.109469 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column Income 2024.10.22 01:46:03.171453 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_557_557_0, 202410_558_558_0, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0, 202410_562_562_0] 2024.10.22 01:46:03.247558 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column Income 2024.10.22 01:46:03.375908 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column Income 2024.10.22 01:46:03.451523 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 70524 2024.10.22 01:46:03.499986 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lqk/irqfsnnxhmyjkmlvspoioxdnhharh (LogSeriesLimiter: on interval from 2024-10-22 01:45:55 to 2024-10-22 01:46:03 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:46:03.553244 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 3.85 MiB, will set to 1.70 GiB (RSS), difference: -4.88 MiB 2024.10.22 01:46:03.569269 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:03.575614 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:03.828006 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1080_1080_0 (state Temporary) 2024.10.22 01:46:04.077815 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1080_1080_0 to 202410_1093_1093_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:04.153753 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70524 2024.10.22 01:46:05.011793 [ 1245 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:46:06.316575 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 39 entries to flush up to offset 72451 2024.10.22 01:46:06.455344 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:06.459863 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:06.489010 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_628_628_0 (state Temporary) 2024.10.22 01:46:06.769267 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_628_628_0 to 202410_634_634_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:06.917325 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 72451 2024.10.22 01:46:07.040997 [ 1263 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 01:46:07.618665 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 949091 2024.10.22 01:46:07.893141 [ 1260 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 01:46:07.953784 [ 1182 ] {} DatabaseCatalog: Execution took 450 ms. 2024.10.22 01:46:07.970996 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 9791 2024.10.22 01:46:08.159431 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 21 entries to flush up to offset 70545 2024.10.22 01:46:08.077698 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:08.181022 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:08.215374 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_640_640_0 (state Temporary) 2024.10.22 01:46:08.364903 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_640_640_0 to 202410_653_653_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:08.376914 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:08.382781 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:08.429052 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1081_1081_0 (state Temporary) 2024.10.22 01:46:08.437376 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:08.445665 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:08.470562 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1081_1081_0 to 202410_1094_1094_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:08.520745 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9791 2024.10.22 01:46:08.686691 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70545 2024.10.22 01:46:08.814235 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_613_613_0 (state Temporary) 2024.10.22 01:46:09.366546 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 4.04 MiB, will set to 1.71 GiB (RSS), difference: -7.68 MiB 2024.10.22 01:46:09.536304 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_613_613_0 to 202410_620_620_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:09.569320 [ 1222 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_610_119 to 202410_611_616_1 2024.10.22 01:46:09.571718 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 949091 2024.10.22 01:46:09.574440 [ 1222 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:09.576461 [ 1222 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:09.591701 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_616_120} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_610_119 to 202410_611_616_1 into Wide with storage Full 2024.10.22 01:46:09.743517 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_616_120} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:46:09.753807 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_616_120} MergeTreeSequentialSource: Reading 117 marks from part 202410_1_610_119, total 946511 rows starting from the beginning of the part 2024.10.22 01:46:09.782640 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_616_120} MergeTreeSequentialSource: Reading 3 marks from part 202410_611_616_1, total 9625 rows starting from the beginning of the part 2024.10.22 01:46:10.681738 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:46:10.808141 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:46:10.904472 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_129_129_0 to 202410_132_132_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:10.913950 [ 1154 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 263 ms. 2024.10.22 01:46:11.471585 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 5.57 MiB, will set to 1.71 GiB (RSS), difference: 1.52 MiB 2024.10.22 01:46:11.980196 [ 1274 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:46:12.013481 [ 1274 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:46:12.689747 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 70576 2024.10.22 01:46:13.129201 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:13.131658 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:13.187848 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1082_1082_0 (state Temporary) 2024.10.22 01:46:13.344135 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1082_1082_0 to 202410_1095_1095_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:13.370371 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1065_1090_5 to 202410_1095_1095_0 2024.10.22 01:46:13.378379 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:13.383085 [ 1253 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:13.397417 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70576 2024.10.22 01:46:13.423781 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_1065_1090_5 to 202410_1095_1095_0 into Compact with storage Full 2024.10.22 01:46:13.438644 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 8.84 MiB, will set to 1.71 GiB (RSS), difference: 5.91 MiB 2024.10.22 01:46:13.508743 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:46:13.521667 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1065_1090_5, total 1503 rows starting from the beginning of the part 2024.10.22 01:46:13.627136 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5163 2024.10.22 01:46:13.658717 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1091_1091_0, total 62 rows starting from the beginning of the part 2024.10.22 01:46:13.783973 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1092_1092_0, total 47 rows starting from the beginning of the part 2024.10.22 01:46:13.828985 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1093_1093_0, total 51 rows starting from the beginning of the part 2024.10.22 01:46:13.864682 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1094_1094_0, total 21 rows starting from the beginning of the part 2024.10.22 01:46:14.251856 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_616_120/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:46:14.277810 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_550_109, 202410_551_556_1] 2024.10.22 01:46:14.280843 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_550_109, 202410_551_556_1] 2024.10.22 01:46:14.293202 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_550_109, 202410_551_556_1] 2024.10.22 01:46:14.427298 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 68 entries to flush up to offset 72519 2024.10.22 01:46:14.555317 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:46:14.573240 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1095_1095_0, total 31 rows starting from the beginning of the part 2024.10.22 01:46:14.775940 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:14.779104 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:14.827465 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_629_629_0 (state Temporary) 2024.10.22 01:46:14.961362 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_629_629_0 to 202410_635_635_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:14.979845 [ 1171 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 4 parts from 202410_507_630_25 to 202410_633_633_0 2024.10.22 01:46:14.991808 [ 1171 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:15.018279 [ 1171 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:15.037053 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_633_26} MergeTask::PrepareStage: Merging 4 parts: from 202410_507_630_25 to 202410_633_633_0 into Compact with storage Full 2024.10.22 01:46:15.072446 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_633_26} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:46:15.078510 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_633_26} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_630_25, total 13003 rows starting from the beginning of the part 2024.10.22 01:46:15.103108 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_633_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_631_631_0, total 114 rows starting from the beginning of the part 2024.10.22 01:46:15.165207 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 72519 2024.10.22 01:46:15.200681 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_633_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_632_632_0, total 100 rows starting from the beginning of the part 2024.10.22 01:46:15.244066 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_633_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_633_633_0, total 70 rows starting from the beginning of the part 2024.10.22 01:46:15.272083 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1065_1095_6/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:46:15.518692 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 13.68 MiB, will set to 1.71 GiB (RSS), difference: -2.58 MiB 2024.10.22 01:46:16.126702 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 9797 2024.10.22 01:46:16.263849 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} MergeTask::MergeProjectionsStage: Merge sorted 1715 rows, containing 13 columns (13 merged, 0 gathered) in 2.851008953 sec., 601.5414291124465 rows/sec., 104.90 KiB/sec. 2024.10.22 01:46:16.265646 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:16.268235 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:16.339102 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_633_26/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:46:16.388335 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_641_641_0 (state Temporary) 2024.10.22 01:46:16.451531 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} MergedBlockOutputStream: filled checksums 202410_1065_1095_6 (state Temporary) 2024.10.22 01:46:16.474515 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_641_641_0 to 202410_654_654_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:16.512186 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1065_1095_6 to 202410_1065_1095_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:16.516054 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1065_1095_6} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1065_1090_5, 202410_1095_1095_0] -> 202410_1065_1095_6 2024.10.22 01:46:16.570802 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:46:16.575960 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 951016 2024.10.22 01:46:16.728971 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9797 2024.10.22 01:46:17.409215 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 70630 2024.10.22 01:46:17.424268 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 10.97 MiB, will set to 1.72 GiB (RSS), difference: 8.54 MiB 2024.10.22 01:46:17.775164 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:17.777565 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:17.816697 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_633_26} MergeTask::MergeProjectionsStage: Merge sorted 13287 rows, containing 13 columns (13 merged, 0 gathered) in 2.785516857 sec., 4770.030368550736 rows/sec., 2.07 MiB/sec. 2024.10.22 01:46:17.839042 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 536 2024.10.22 01:46:17.940437 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1083_1083_0 (state Temporary) 2024.10.22 01:46:17.951091 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:17.955599 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:18.012643 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1083_1083_0 to 202410_1096_1096_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:18.117138 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_633_26} MergedBlockOutputStream: filled checksums 202410_507_633_26 (state Temporary) 2024.10.22 01:46:18.139339 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:18.149102 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:18.165465 [ 1183 ] {} DatabaseCatalog: Execution took 209 ms. 2024.10.22 01:46:18.171461 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70630 2024.10.22 01:46:18.272564 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_614_614_0 (state Temporary) 2024.10.22 01:46:18.307288 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_518_518_0 (state Temporary) 2024.10.22 01:46:18.366518 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_614_614_0 to 202410_621_621_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:18.372121 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_633_26} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_633_26 to 202410_507_633_26 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:18.373519 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_633_26} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 4 parts: [202410_507_630_25, 202410_633_633_0] -> 202410_507_633_26 2024.10.22 01:46:18.397319 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 951016 2024.10.22 01:46:18.576064 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_518_518_0 to 202410_528_528_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:18.696250 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.73 MiB. 2024.10.22 01:46:18.729086 [ 1276 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_523_107 to 202410_528_528_0 2024.10.22 01:46:18.740120 [ 1276 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:18.748360 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:18.764331 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 536 2024.10.22 01:46:18.769271 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xxv/mlplsnnfcqoezetnxfwdbcjadwubd. 2024.10.22 01:46:18.753144 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a8e890157c02a05debd7322801c7cb98) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a8e890157c02a05debd7322801c7cb98, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:46:18.788095 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 630290, file segment info: File segment: [0, 33554431], key: a8e890157c02a05debd7322801c7cb98, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:46:18.791778 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:18.867262 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_523_107 to 202410_528_528_0 into Compact with storage Full 2024.10.22 01:46:18.974664 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:46:19.022350 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_523_107, total 567 rows starting from the beginning of the part 2024.10.22 01:46:19.206214 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_524_524_0, total 1 rows starting from the beginning of the part 2024.10.22 01:46:19.563311 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_525_525_0, total 1 rows starting from the beginning of the part 2024.10.22 01:46:19.566827 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.97 GiB, free memory in arenas 12.36 MiB, will set to 1.71 GiB (RSS), difference: 94.63 MiB 2024.10.22 01:46:19.702445 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_526_526_0, total 1 rows starting from the beginning of the part 2024.10.22 01:46:19.791760 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(298a42f594a5185987db0b6d4b168db2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 298a42f594a5185987db0b6d4b168db2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:46:19.799233 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 501, file segment info: File segment: [0, 33554431], key: 298a42f594a5185987db0b6d4b168db2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:46:19.808166 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:19.955517 [ 1723 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lqk/irqfsnnxhmyjkmlvspoioxdnhharh, size 630290 2024.10.22 01:46:19.964279 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_527_527_0, total 1 rows starting from the beginning of the part 2024.10.22 01:46:20.163581 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column Interests 2024.10.22 01:46:20.174064 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column Interests 2024.10.22 01:46:20.181216 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column Interests 2024.10.22 01:46:20.183891 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_528_528_0, total 1 rows starting from the beginning of the part 2024.10.22 01:46:20.203075 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column Interests 2024.10.22 01:46:20.219157 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column Interests 2024.10.22 01:46:20.243517 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column Interests 2024.10.22 01:46:20.373437 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column Interests 2024.10.22 01:46:20.390496 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column Interests 2024.10.22 01:46:20.474212 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hex/hxnyouztnodpkeguqwzarkafmjqnm (LogSeriesLimiter: on interval from 2024-10-22 01:46:03 to 2024-10-22 01:46:20 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:46:20.682754 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:46:20.685248 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:46:21.158891 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 5200 2024.10.22 01:46:21.269922 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.97 GiB, free memory in arenas 13.07 MiB, will set to 1.70 GiB (RSS), difference: 85.95 MiB 2024.10.22 01:46:21.270295 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_572_13, 202410_573_573_0, 202410_574_574_0, 202410_575_575_0, 202410_576_576_0, 202410_577_577_0] 2024.10.22 01:46:21.272983 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_572_13, 202410_573_573_0, 202410_574_574_0, 202410_575_575_0, 202410_576_576_0, 202410_577_577_0] 2024.10.22 01:46:21.294421 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_572_13, 202410_573_573_0, 202410_574_574_0, 202410_575_575_0, 202410_576_576_0, 202410_577_577_0] 2024.10.22 01:46:21.655780 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_984_489, 202410_985_990_1] 2024.10.22 01:46:21.658813 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_984_489, 202410_985_990_1] 2024.10.22 01:46:21.773490 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_984_489, 202410_985_990_1] 2024.10.22 01:46:22.186498 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 70693 2024.10.22 01:46:22.402556 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9974d90e6f091fce66a13a167ee51747) : [0, 264]: Setting download as failed: File segment: [0, 264], key: 9974d90e6f091fce66a13a167ee51747, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:22.406599 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9974d90e6f091fce66a13a167ee51747): No space left in cache to reserve 265 bytes, will continue without cache download 2024.10.22 01:46:22.488945 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:22.497777 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:22.662351 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1084_1084_0 (state Temporary) 2024.10.22 01:46:22.678623 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 92 entries to flush up to offset 72611 2024.10.22 01:46:22.785666 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:22.788898 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:22.802544 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1084_1084_0 to 202410_1097_1097_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:22.847570 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_630_630_0 (state Temporary) 2024.10.22 01:46:22.975485 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} MergeTask::MergeProjectionsStage: Merge sorted 572 rows, containing 32 columns (32 merged, 0 gathered) in 4.112280264 sec., 139.09557794672722 rows/sec., 12.75 KiB/sec. 2024.10.22 01:46:22.980101 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_630_630_0 to 202410_636_636_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:23.000762 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} MergedBlockOutputStream: filled checksums 202410_1_528_108 (state Temporary) 2024.10.22 01:46:23.037409 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 72611 2024.10.22 01:46:23.110529 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70693 2024.10.22 01:46:23.576848 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 14.79 MiB, will set to 1.70 GiB (RSS), difference: -1.85 MiB 2024.10.22 01:46:23.652931 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0b8153f31b282f5438cc5d167ccf39e5) : [0, 4080777]: Setting download as failed: File segment: [0, 4080777], key: 0b8153f31b282f5438cc5d167ccf39e5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:23.655056 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0b8153f31b282f5438cc5d167ccf39e5): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:46:23.846934 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0b8153f31b282f5438cc5d167ccf39e5, offset: 0, size: 0 2024.10.22 01:46:23.849396 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0b8153f31b282f5438cc5d167ccf39e5 for removal 2024.10.22 01:46:23.852750 [ 1476 ] {} CacheMetadata: Key 0b8153f31b282f5438cc5d167ccf39e5 is removed from metadata 2024.10.22 01:46:23.856556 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9974d90e6f091fce66a13a167ee51747, offset: 0, size: 0 2024.10.22 01:46:23.858256 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9974d90e6f091fce66a13a167ee51747 for removal 2024.10.22 01:46:23.861086 [ 1476 ] {} CacheMetadata: Key 9974d90e6f091fce66a13a167ee51747 is removed from metadata 2024.10.22 01:46:23.876553 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_528_108 to 202410_1_528_108 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:23.879849 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_528_108} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_523_107, 202410_528_528_0] -> 202410_1_528_108 2024.10.22 01:46:24.143708 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:46:24.237420 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 9822 2024.10.22 01:46:24.460234 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:24.462362 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:24.629372 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_642_642_0 (state Temporary) 2024.10.22 01:46:24.676507 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_642_642_0 to 202410_655_655_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:24.764089 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2d0081208d9591caeb2d2061c3ce7944) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 2d0081208d9591caeb2d2061c3ce7944, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:24.765257 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2d0081208d9591caeb2d2061c3ce7944): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:46:24.825476 [ 1151 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_650_130 to 202410_655_655_0 2024.10.22 01:46:24.847622 [ 1151 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:24.853121 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:24.875062 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9822 2024.10.22 01:46:24.902327 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_650_130 to 202410_655_655_0 into Compact with storage Full 2024.10.22 01:46:25.127868 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:46:25.145340 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_650_130, total 9857 rows starting from the beginning of the part 2024.10.22 01:46:25.207135 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_651_651_0, total 16 rows starting from the beginning of the part 2024.10.22 01:46:25.244123 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_616_120/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:46:25.259105 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_652_652_0, total 19 rows starting from the beginning of the part 2024.10.22 01:46:25.404295 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 952556 2024.10.22 01:46:25.495843 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_653_653_0, total 5 rows starting from the beginning of the part 2024.10.22 01:46:25.576124 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 13.28 MiB, will set to 1.70 GiB (RSS), difference: 3.98 MiB 2024.10.22 01:46:25.660585 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_654_654_0, total 6 rows starting from the beginning of the part 2024.10.22 01:46:25.771286 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_991_991_0, 202410_992_992_0, 202410_993_993_0, 202410_994_994_0, 202410_995_995_0, 202410_996_996_0] 2024.10.22 01:46:25.773629 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_991_991_0, 202410_992_992_0, 202410_993_993_0, 202410_994_994_0, 202410_995_995_0, 202410_996_996_0] 2024.10.22 01:46:25.816558 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_991_991_0, 202410_992_992_0, 202410_993_993_0, 202410_994_994_0, 202410_995_995_0, 202410_996_996_0] 2024.10.22 01:46:25.918852 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_655_655_0, total 25 rows starting from the beginning of the part 2024.10.22 01:46:26.081776 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(22be2f6202125752fb79609e575c359a) : [0, 72041]: Setting download as failed: File segment: [0, 72041], key: 22be2f6202125752fb79609e575c359a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:26.085236 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(22be2f6202125752fb79609e575c359a): No space left in cache to reserve 72042 bytes, will continue without cache download 2024.10.22 01:46:26.093452 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:26.100784 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:26.135320 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 22be2f6202125752fb79609e575c359a, offset: 0, size: 0 2024.10.22 01:46:26.139159 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 22be2f6202125752fb79609e575c359a for removal 2024.10.22 01:46:26.145040 [ 1476 ] {} CacheMetadata: Key 22be2f6202125752fb79609e575c359a is removed from metadata 2024.10.22 01:46:26.148558 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2d0081208d9591caeb2d2061c3ce7944, offset: 0, size: 0 2024.10.22 01:46:26.154936 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2d0081208d9591caeb2d2061c3ce7944 for removal 2024.10.22 01:46:26.172744 [ 1476 ] {} CacheMetadata: Key 2d0081208d9591caeb2d2061c3ce7944 is removed from metadata 2024.10.22 01:46:26.267856 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 537 2024.10.22 01:46:26.287785 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_615_615_0 (state Temporary) 2024.10.22 01:46:26.642506 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_615_615_0 to 202410_622_622_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:26.672598 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:26.675225 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:26.673315 [ 1249 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_617_617_0 to 202410_622_622_0 2024.10.22 01:46:26.693637 [ 1249 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:26.697197 [ 1249 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:26.778661 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 952556 2024.10.22 01:46:26.836932 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_519_519_0 (state Temporary) 2024.10.22 01:46:26.976361 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_617_617_0 to 202410_622_622_0 into Compact with storage Full 2024.10.22 01:46:27.018944 [ 1215 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:46:27.024336 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:46:27.034175 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_519_519_0 to 202410_529_529_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:27.034607 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_617_617_0, total 1344 rows starting from the beginning of the part 2024.10.22 01:46:27.053758 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_618_618_0, total 2121 rows starting from the beginning of the part 2024.10.22 01:46:27.069800 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_619_619_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:46:27.094921 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_620_620_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:46:27.102977 [ 1215 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:46:27.113592 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_621_621_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:46:27.148493 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 537 2024.10.22 01:46:27.234631 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 70775 2024.10.22 01:46:27.277423 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_622_622_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:46:27.277529 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(f9ff7001240f777c41707629ff5bdba8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f9ff7001240f777c41707629ff5bdba8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:46:27.304541 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 629438, file segment info: File segment: [0, 33554431], key: f9ff7001240f777c41707629ff5bdba8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:46:27.308240 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:27.377559 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} MergeTask::MergeProjectionsStage: Merge sorted 9928 rows, containing 12 columns (12 merged, 0 gathered) in 2.488912345 sec., 3988.8909788022283 rows/sec., 634.74 KiB/sec. 2024.10.22 01:46:27.425484 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} MergedBlockOutputStream: filled checksums 202410_1_655_131 (state Temporary) 2024.10.22 01:46:27.751251 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:27.753805 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:27.854863 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_655_131 to 202410_1_655_131 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:27.857568 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_655_131} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_650_130, 202410_655_655_0] -> 202410_1_655_131 2024.10.22 01:46:27.876710 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.59 MiB. 2024.10.22 01:46:27.953549 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1085_1085_0 (state Temporary) 2024.10.22 01:46:27.990703 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1085_1085_0 to 202410_1098_1098_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:28.259093 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70775 2024.10.22 01:46:28.260999 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} MergeTask::MergeProjectionsStage: Merge sorted 10010 rows, containing 4 columns (4 merged, 0 gathered) in 1.492897835 sec., 6705.080391519223 rows/sec., 104.77 KiB/sec. 2024.10.22 01:46:28.279610 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bedcc6314ac58f48072afe857bc2e309) : [0, 176]: Setting download as failed: File segment: [0, 176], key: bedcc6314ac58f48072afe857bc2e309, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:28.280478 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bedcc6314ac58f48072afe857bc2e309): No space left in cache to reserve 177 bytes, will continue without cache download 2024.10.22 01:46:28.397767 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} MergedBlockOutputStream: filled checksums 202410_617_622_1 (state Temporary) 2024.10.22 01:46:28.562448 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_617_622_1 to 202410_617_622_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:28.568159 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_617_622_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_617_617_0, 202410_622_622_0] -> 202410_617_622_1 2024.10.22 01:46:28.593601 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:46:29.180607 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(5a145ecd3a0da188a755f56a8fbecd5e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5a145ecd3a0da188a755f56a8fbecd5e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:46:29.190173 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 742, file segment info: File segment: [0, 33554431], key: 5a145ecd3a0da188a755f56a8fbecd5e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:46:29.196783 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:29.326211 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 13.36 MiB, will set to 1.70 GiB (RSS), difference: 4.42 MiB 2024.10.22 01:46:30.052476 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cb8646ee54cc020b1696aa6651e4bd6f) : [0, 4559]: Setting download as failed: File segment: [0, 4559], key: cb8646ee54cc020b1696aa6651e4bd6f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:30.055061 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cb8646ee54cc020b1696aa6651e4bd6f): No space left in cache to reserve 4560 bytes, will continue without cache download 2024.10.22 01:46:30.068243 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cb8646ee54cc020b1696aa6651e4bd6f, offset: 0, size: 0 2024.10.22 01:46:30.070493 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cb8646ee54cc020b1696aa6651e4bd6f for removal 2024.10.22 01:46:30.072434 [ 1476 ] {} CacheMetadata: Key cb8646ee54cc020b1696aa6651e4bd6f is removed from metadata 2024.10.22 01:46:30.073584 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bedcc6314ac58f48072afe857bc2e309, offset: 0, size: 0 2024.10.22 01:46:30.076241 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bedcc6314ac58f48072afe857bc2e309 for removal 2024.10.22 01:46:30.179029 [ 1476 ] {} CacheMetadata: Key bedcc6314ac58f48072afe857bc2e309 is removed from metadata 2024.10.22 01:46:30.562323 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 87 entries to flush up to offset 72698 2024.10.22 01:46:30.683077 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:46:30.703925 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:46:30.780044 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:30.784927 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:30.839956 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_631_631_0 (state Temporary) 2024.10.22 01:46:31.015809 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_631_631_0 to 202410_637_637_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:31.163996 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 72698 2024.10.22 01:46:31.217805 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_591_118, 202410_592_592_0, 202410_593_593_0, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0] 2024.10.22 01:46:31.223720 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_591_118, 202410_592_592_0, 202410_593_593_0, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0] 2024.10.22 01:46:31.247416 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(f4e452570b794a53ee6b8be333f49350) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f4e452570b794a53ee6b8be333f49350, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:46:31.253350 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_591_118, 202410_592_592_0, 202410_593_593_0, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0] 2024.10.22 01:46:31.275675 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 807764, file segment info: File segment: [0, 33554431], key: f4e452570b794a53ee6b8be333f49350, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:46:31.277161 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:31.863863 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:46:32.251126 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1b95c17dff301598ce4ec8d0c4e547fa) : [0, 240]: Setting download as failed: File segment: [0, 240], key: 1b95c17dff301598ce4ec8d0c4e547fa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:32.252425 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1b95c17dff301598ce4ec8d0c4e547fa): No space left in cache to reserve 241 bytes, will continue without cache download 2024.10.22 01:46:32.262942 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 70828 2024.10.22 01:46:32.377588 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 9841 2024.10.22 01:46:32.476997 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:32.479708 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:32.481716 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:32.485384 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:32.631764 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1086_1086_0 (state Temporary) 2024.10.22 01:46:32.680430 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_643_643_0 (state Temporary) 2024.10.22 01:46:32.776630 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1086_1086_0 to 202410_1099_1099_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:32.839721 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_643_643_0 to 202410_656_656_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:32.881414 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9841 2024.10.22 01:46:32.920764 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70828 2024.10.22 01:46:33.021368 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d203db6d0b2f905981bc40a24462a4ea) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d203db6d0b2f905981bc40a24462a4ea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:46:33.027089 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 934, file segment info: File segment: [0, 33554431], key: d203db6d0b2f905981bc40a24462a4ea, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:46:33.033560 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:33.086745 [ 1695 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vou/xyqjheuaeinxicuyzcqlxvlxywpre, size 807764 2024.10.22 01:46:33.241294 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:46:33.254165 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:46:33.281376 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 11.36 MiB, will set to 1.70 GiB (RSS), difference: 1.58 MiB 2024.10.22 01:46:33.283674 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:46:33.423761 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:46:33.464410 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:46:33.489828 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:46:33.595998 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:46:33.661636 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:46:33.681290 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 01:46:33.768340 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/iic/fiicfoalcbvrevszrvdywqrkbvbsp (LogSeriesLimiter: on interval from 2024-10-22 01:46:20 to 2024-10-22 01:46:33 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:46:33.782098 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 953326 2024.10.22 01:46:34.084479 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e3d5baa958c6ca4473fc6a8a0ee642f5) : [0, 1643377]: Setting download as failed: File segment: [0, 1643377], key: e3d5baa958c6ca4473fc6a8a0ee642f5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:34.086899 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e3d5baa958c6ca4473fc6a8a0ee642f5): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:46:34.335639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e3d5baa958c6ca4473fc6a8a0ee642f5, offset: 0, size: 0 2024.10.22 01:46:34.339863 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e3d5baa958c6ca4473fc6a8a0ee642f5 for removal 2024.10.22 01:46:34.344313 [ 1476 ] {} CacheMetadata: Key e3d5baa958c6ca4473fc6a8a0ee642f5 is removed from metadata 2024.10.22 01:46:34.348095 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1b95c17dff301598ce4ec8d0c4e547fa, offset: 0, size: 0 2024.10.22 01:46:34.349542 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1b95c17dff301598ce4ec8d0c4e547fa for removal 2024.10.22 01:46:34.362248 [ 1476 ] {} CacheMetadata: Key 1b95c17dff301598ce4ec8d0c4e547fa is removed from metadata 2024.10.22 01:46:34.566178 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:34.569781 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:34.670256 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_616_616_0 (state Temporary) 2024.10.22 01:46:34.671533 [ 1162 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 215 ms. 2024.10.22 01:46:34.736677 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 538 2024.10.22 01:46:34.791617 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_616_616_0 to 202410_623_623_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:34.944710 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 953326 2024.10.22 01:46:35.117163 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(6fa586a05d27218446720af217114c80) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6fa586a05d27218446720af217114c80, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:46:35.120781 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 874833, file segment info: File segment: [0, 33554431], key: 6fa586a05d27218446720af217114c80, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:46:35.124183 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:35.222865 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:35.225804 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:35.385098 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_616_120/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:46:35.495517 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_520_520_0 (state Temporary) 2024.10.22 01:46:35.585111 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_520_520_0 to 202410_530_530_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:35.663552 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 538 2024.10.22 01:46:35.998824 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5bc285cb6ac2bfcd600fb94c7ca3558a) : [0, 176]: Setting download as failed: File segment: [0, 176], key: 5bc285cb6ac2bfcd600fb94c7ca3558a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:36.002338 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5bc285cb6ac2bfcd600fb94c7ca3558a): No space left in cache to reserve 177 bytes, will continue without cache download 2024.10.22 01:46:36.257394 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:36.261641 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:46:36.521262 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 7.80 MiB, will set to 1.70 GiB (RSS), difference: -12.74 MiB 2024.10.22 01:46:36.948747 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 70885 2024.10.22 01:46:36.865406 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(09ec90228ce860fd44b144993c35ee34) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 09ec90228ce860fd44b144993c35ee34, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:46:36.977944 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 635, file segment info: File segment: [0, 33554431], key: 09ec90228ce860fd44b144993c35ee34, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:46:36.979989 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:37.065502 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:37.068312 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:37.102412 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1087_1087_0 (state Temporary) 2024.10.22 01:46:37.236273 [ 1690 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hex/hxnyouztnodpkeguqwzarkafmjqnm, size 874833 2024.10.22 01:46:37.263222 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1087_1087_0 to 202410_1100_1100_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:37.274672 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1064_504 to 202410_1065_1095_6 2024.10.22 01:46:37.277429 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.64 MiB using storage policy from min volume index 0 2024.10.22 01:46:37.279196 [ 1277 ] {} DiskLocal: Reserved 2.64 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:37.305472 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70885 2024.10.22 01:46:37.318564 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1095_505} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1064_504 to 202410_1065_1095_6 into Wide with storage Full 2024.10.22 01:46:37.474898 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1095_505} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:46:37.513791 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1095_505} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1064_504, total 69692 rows starting from the beginning of the part 2024.10.22 01:46:37.621090 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column Robotness 2024.10.22 01:46:37.770259 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(979cad8b6c33493e6474bc3b8e5e81c7) : [0, 42962]: Setting download as failed: File segment: [0, 42962], key: 979cad8b6c33493e6474bc3b8e5e81c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:37.771133 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(979cad8b6c33493e6474bc3b8e5e81c7): No space left in cache to reserve 42963 bytes, will continue without cache download 2024.10.22 01:46:37.808931 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 979cad8b6c33493e6474bc3b8e5e81c7, offset: 0, size: 0 2024.10.22 01:46:37.811534 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 979cad8b6c33493e6474bc3b8e5e81c7 for removal 2024.10.22 01:46:37.820560 [ 1476 ] {} CacheMetadata: Key 979cad8b6c33493e6474bc3b8e5e81c7 is removed from metadata 2024.10.22 01:46:37.821588 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5bc285cb6ac2bfcd600fb94c7ca3558a, offset: 0, size: 0 2024.10.22 01:46:37.827494 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5bc285cb6ac2bfcd600fb94c7ca3558a for removal 2024.10.22 01:46:37.832771 [ 1476 ] {} CacheMetadata: Key 5bc285cb6ac2bfcd600fb94c7ca3558a is removed from metadata 2024.10.22 01:46:37.854278 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column Robotness 2024.10.22 01:46:37.855125 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1095_505} MergeTreeSequentialSource: Reading 2 marks from part 202410_1065_1095_6, total 1715 rows starting from the beginning of the part 2024.10.22 01:46:37.861457 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column Robotness 2024.10.22 01:46:37.979632 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column Robotness 2024.10.22 01:46:38.000462 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column Robotness 2024.10.22 01:46:38.027748 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column Robotness 2024.10.22 01:46:38.070850 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column Robotness 2024.10.22 01:46:38.094823 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column Robotness 2024.10.22 01:46:38.175476 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/phz/ovcoadcsgpvxuwfltgycydoodzbrl (LogSeriesLimiter: on interval from 2024-10-22 01:46:33 to 2024-10-22 01:46:38 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:46:38.327600 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.162457755 sec. 2024.10.22 01:46:38.669663 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 125 entries to flush up to offset 72823 2024.10.22 01:46:38.863170 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dd8b8b3bb55c64b72426fd276944265d) : [0, 209]: Setting download as failed: File segment: [0, 209], key: dd8b8b3bb55c64b72426fd276944265d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:38.864262 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dd8b8b3bb55c64b72426fd276944265d): No space left in cache to reserve 210 bytes, will continue without cache download 2024.10.22 01:46:38.912364 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:38.918312 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:38.951292 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_616_120} MergeTask::MergeProjectionsStage: Merge sorted 956136 rows, containing 4 columns (4 merged, 0 gathered) in 29.36891142 sec., 32556.058558877292 rows/sec., 488.21 KiB/sec. 2024.10.22 01:46:38.989583 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_130_130_0 (state Temporary) 2024.10.22 01:46:39.088871 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_632_632_0 (state Temporary) 2024.10.22 01:46:39.095053 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_616_120} MergedBlockOutputStream: filled checksums 202410_1_616_120 (state Temporary) 2024.10.22 01:46:39.168601 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_632_632_0 to 202410_638_638_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:39.220271 [ 1173 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_633_26 to 202410_638_638_0 2024.10.22 01:46:39.242279 [ 1173 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:39.248032 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 72823 2024.10.22 01:46:39.258792 [ 1173 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:39.274804 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_633_26 to 202410_638_638_0 into Compact with storage Full 2024.10.22 01:46:39.353957 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:46:39.372864 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_633_26, total 13287 rows starting from the beginning of the part 2024.10.22 01:46:39.470499 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_634_634_0, total 39 rows starting from the beginning of the part 2024.10.22 01:46:39.474553 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 11.18 MiB, will set to 1.70 GiB (RSS), difference: -41.88 MiB 2024.10.22 01:46:39.573546 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_635_635_0, total 68 rows starting from the beginning of the part 2024.10.22 01:46:39.743429 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5abaf258756e759eab9168544a0eaf56) : [0, 31190]: Setting download as failed: File segment: [0, 31190], key: 5abaf258756e759eab9168544a0eaf56, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:39.745162 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5abaf258756e759eab9168544a0eaf56): No space left in cache to reserve 31191 bytes, will continue without cache download 2024.10.22 01:46:39.759855 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_636_636_0, total 92 rows starting from the beginning of the part 2024.10.22 01:46:39.828475 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_637_637_0, total 87 rows starting from the beginning of the part 2024.10.22 01:46:39.869930 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5abaf258756e759eab9168544a0eaf56, offset: 0, size: 0 2024.10.22 01:46:39.872455 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5abaf258756e759eab9168544a0eaf56 for removal 2024.10.22 01:46:39.874373 [ 1476 ] {} CacheMetadata: Key 5abaf258756e759eab9168544a0eaf56 is removed from metadata 2024.10.22 01:46:39.876092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dd8b8b3bb55c64b72426fd276944265d, offset: 0, size: 0 2024.10.22 01:46:39.877533 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dd8b8b3bb55c64b72426fd276944265d for removal 2024.10.22 01:46:39.878709 [ 1476 ] {} CacheMetadata: Key dd8b8b3bb55c64b72426fd276944265d is removed from metadata 2024.10.22 01:46:40.003326 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_638_638_0, total 125 rows starting from the beginning of the part 2024.10.22 01:46:40.386744 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 9847 2024.10.22 01:46:40.639042 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:40.668003 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:40.684746 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:46:40.706302 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_644_644_0 (state Temporary) 2024.10.22 01:46:40.794266 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:46:40.860657 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_644_644_0 to 202410_657_657_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:40.899894 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(61a272c90b876136c42b4909ba340629) : [0, 220]: Setting download as failed: File segment: [0, 220], key: 61a272c90b876136c42b4909ba340629, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:40.903519 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(61a272c90b876136c42b4909ba340629): No space left in cache to reserve 221 bytes, will continue without cache download 2024.10.22 01:46:40.917540 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9847 2024.10.22 01:46:41.310799 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 70961 2024.10.22 01:46:41.426165 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 8.18 MiB, will set to 1.72 GiB (RSS), difference: 1.66 MiB 2024.10.22 01:46:41.472347 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:41.477477 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:41.545537 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1088_1088_0 (state Temporary) 2024.10.22 01:46:41.764196 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1088_1088_0 to 202410_1101_1101_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:41.889074 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1096_0 to 202410_1101_1101_0 2024.10.22 01:46:41.914536 [ 1187 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:41.933716 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 70961 2024.10.22 01:46:41.951772 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 954481 2024.10.22 01:46:42.008932 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(52cf918e227158755883d490e7e554bc) : [0, 393591]: Setting download as failed: File segment: [0, 393591], key: 52cf918e227158755883d490e7e554bc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:42.010753 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(52cf918e227158755883d490e7e554bc): No space left in cache to reserve 393592 bytes, will continue without cache download 2024.10.22 01:46:42.046681 [ 1187 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:42.056412 [ 1187 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 283 ms. 2024.10.22 01:46:42.109043 [ 1196 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:46:42.132531 [ 1196 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:46:42.155439 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 52cf918e227158755883d490e7e554bc, offset: 0, size: 0 2024.10.22 01:46:42.157372 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 52cf918e227158755883d490e7e554bc for removal 2024.10.22 01:46:42.162455 [ 1476 ] {} CacheMetadata: Key 52cf918e227158755883d490e7e554bc is removed from metadata 2024.10.22 01:46:42.167755 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 61a272c90b876136c42b4909ba340629, offset: 0, size: 0 2024.10.22 01:46:42.170352 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1095_505} MergeTask::MergeProjectionsStage: Merge sorted 71407 rows, containing 13 columns (13 merged, 0 gathered) in 4.858554414 sec., 14697.169963608849 rows/sec., 2.43 MiB/sec. 2024.10.22 01:46:42.172564 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 61a272c90b876136c42b4909ba340629 for removal 2024.10.22 01:46:42.173693 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1096_0 to 202410_1101_1101_0 into Compact with storage Full 2024.10.22 01:46:42.177310 [ 1476 ] {} CacheMetadata: Key 61a272c90b876136c42b4909ba340629 is removed from metadata 2024.10.22 01:46:42.271672 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:46:42.360794 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_616_120} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_616_120 to 202410_1_616_120 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:42.400306 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1096_0, total 54 rows starting from the beginning of the part 2024.10.22 01:46:42.445239 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_616_120} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_610_119, 202410_611_616_1] -> 202410_1_616_120 2024.10.22 01:46:42.462387 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} MergeTask::MergeProjectionsStage: Merge sorted 13698 rows, containing 13 columns (13 merged, 0 gathered) in 3.198111064 sec., 4283.153313277177 rows/sec., 1.86 MiB/sec. 2024.10.22 01:46:42.465130 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1097_1097_0, total 63 rows starting from the beginning of the part 2024.10.22 01:46:42.476020 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1095_505} MergedBlockOutputStream: filled checksums 202410_1_1095_505 (state Temporary) 2024.10.22 01:46:42.489401 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:46:42.543022 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1098_1098_0, total 82 rows starting from the beginning of the part 2024.10.22 01:46:42.628227 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:42.634638 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:42.648404 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} MergedBlockOutputStream: filled checksums 202410_507_638_27 (state Temporary) 2024.10.22 01:46:42.695473 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1099_1099_0, total 53 rows starting from the beginning of the part 2024.10.22 01:46:42.863969 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_617_617_0 (state Temporary) 2024.10.22 01:46:42.880889 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1100_1100_0, total 57 rows starting from the beginning of the part 2024.10.22 01:46:42.948739 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1101_1101_0, total 76 rows starting from the beginning of the part 2024.10.22 01:46:43.033998 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1095_505} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1095_505 to 202410_1_1095_505 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:43.037098 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_638_27 to 202410_507_638_27 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:43.041300 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1095_505} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1064_504, 202410_1065_1095_6] -> 202410_1_1095_505 2024.10.22 01:46:43.047275 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_638_27} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_633_26, 202410_638_638_0] -> 202410_507_638_27 2024.10.22 01:46:43.062168 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 51.94 MiB. 2024.10.22 01:46:43.083610 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.81 MiB. 2024.10.22 01:46:43.143518 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_617_617_0 to 202410_624_624_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:43.166603 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 539 2024.10.22 01:46:43.177463 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 954481 2024.10.22 01:46:43.469372 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(eeb286e18a00658f81def6e4d4420429) : [0, 215]: Setting download as failed: File segment: [0, 215], key: eeb286e18a00658f81def6e4d4420429, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:43.470381 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(eeb286e18a00658f81def6e4d4420429): No space left in cache to reserve 216 bytes, will continue without cache download 2024.10.22 01:46:43.619465 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 1.97 GiB, free memory in arenas 5.45 MiB, will set to 1.70 GiB (RSS), difference: 39.97 MiB 2024.10.22 01:46:43.756295 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:43.758629 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:43.913565 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_521_521_0 (state Temporary) 2024.10.22 01:46:43.983704 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_521_521_0 to 202410_531_531_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:44.085576 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 539 2024.10.22 01:46:44.269307 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8f4b9f94198271e5110ed3d091322f92) : [0, 256980]: Setting download as failed: File segment: [0, 256980], key: 8f4b9f94198271e5110ed3d091322f92, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:44.270658 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8f4b9f94198271e5110ed3d091322f92): No space left in cache to reserve 256981 bytes, will continue without cache download 2024.10.22 01:46:44.300349 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8f4b9f94198271e5110ed3d091322f92, offset: 0, size: 0 2024.10.22 01:46:44.304262 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8f4b9f94198271e5110ed3d091322f92 for removal 2024.10.22 01:46:44.308711 [ 1476 ] {} CacheMetadata: Key 8f4b9f94198271e5110ed3d091322f92 is removed from metadata 2024.10.22 01:46:44.309762 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: eeb286e18a00658f81def6e4d4420429, offset: 0, size: 0 2024.10.22 01:46:44.311419 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key eeb286e18a00658f81def6e4d4420429 for removal 2024.10.22 01:46:44.313772 [ 1476 ] {} CacheMetadata: Key eeb286e18a00658f81def6e4d4420429 is removed from metadata 2024.10.22 01:46:44.479229 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} MergeTask::MergeProjectionsStage: Merge sorted 385 rows, containing 13 columns (13 merged, 0 gathered) in 2.41233318 sec., 159.5965280384694 rows/sec., 28.66 KiB/sec. 2024.10.22 01:46:44.514643 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} MergedBlockOutputStream: filled checksums 202410_1096_1101_1 (state Temporary) 2024.10.22 01:46:44.812374 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1101_1 to 202410_1096_1101_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:44.821350 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1101_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1096_0, 202410_1101_1101_0] -> 202410_1096_1101_1 2024.10.22 01:46:44.958910 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:46:45.069165 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5dedaf42de4aec1cdca2fe8bb21ecd6b) : [0, 215]: Setting download as failed: File segment: [0, 215], key: 5dedaf42de4aec1cdca2fe8bb21ecd6b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:45.071993 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5dedaf42de4aec1cdca2fe8bb21ecd6b): No space left in cache to reserve 216 bytes, will continue without cache download 2024.10.22 01:46:45.630390 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 3.20 MiB, will set to 1.69 GiB (RSS), difference: -1.74 MiB 2024.10.22 01:46:45.954942 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 71034 2024.10.22 01:46:46.239956 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ba325ff9168cfe28ce6024f4d517198e) : [0, 564529]: Setting download as failed: File segment: [0, 564529], key: ba325ff9168cfe28ce6024f4d517198e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:46.243664 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ba325ff9168cfe28ce6024f4d517198e): No space left in cache to reserve 564530 bytes, will continue without cache download 2024.10.22 01:46:46.283871 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ba325ff9168cfe28ce6024f4d517198e, offset: 0, size: 0 2024.10.22 01:46:46.286388 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ba325ff9168cfe28ce6024f4d517198e for removal 2024.10.22 01:46:46.303407 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5dedaf42de4aec1cdca2fe8bb21ecd6b, offset: 0, size: 0 2024.10.22 01:46:46.306313 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5dedaf42de4aec1cdca2fe8bb21ecd6b for removal 2024.10.22 01:46:46.311041 [ 1476 ] {} CacheMetadata: Key ba325ff9168cfe28ce6024f4d517198e is removed from metadata 2024.10.22 01:46:46.317011 [ 1476 ] {} CacheMetadata: Key 5dedaf42de4aec1cdca2fe8bb21ecd6b is removed from metadata 2024.10.22 01:46:46.442113 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:46.445805 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:46.497762 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1089_1089_0 (state Temporary) 2024.10.22 01:46:46.770996 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 106 entries to flush up to offset 72929 2024.10.22 01:46:46.874912 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1089_1089_0 to 202410_1102_1102_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:46.926573 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71034 2024.10.22 01:46:47.221290 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:47.229720 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:47.291744 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_633_633_0 (state Temporary) 2024.10.22 01:46:47.570249 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3338dfba82fd0e2390cca4aa4ebea903) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 3338dfba82fd0e2390cca4aa4ebea903, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:47.572352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3338dfba82fd0e2390cca4aa4ebea903): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:46:47.648065 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_633_633_0 to 202410_639_639_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:47.697686 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_130_130_0 to 202410_133_133_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:47.783442 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 72929 2024.10.22 01:46:48.371009 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7bd9fdea1b830fb3c891bb36a2ad2349) : [0, 6443]: Setting download as failed: File segment: [0, 6443], key: 7bd9fdea1b830fb3c891bb36a2ad2349, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:48.374616 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7bd9fdea1b830fb3c891bb36a2ad2349): No space left in cache to reserve 6444 bytes, will continue without cache download 2024.10.22 01:46:48.385761 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7bd9fdea1b830fb3c891bb36a2ad2349, offset: 0, size: 0 2024.10.22 01:46:48.388036 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7bd9fdea1b830fb3c891bb36a2ad2349 for removal 2024.10.22 01:46:48.392366 [ 1476 ] {} CacheMetadata: Key 7bd9fdea1b830fb3c891bb36a2ad2349 is removed from metadata 2024.10.22 01:46:48.394876 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3338dfba82fd0e2390cca4aa4ebea903, offset: 0, size: 0 2024.10.22 01:46:48.400078 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3338dfba82fd0e2390cca4aa4ebea903 for removal 2024.10.22 01:46:48.418345 [ 1476 ] {} CacheMetadata: Key 3338dfba82fd0e2390cca4aa4ebea903 is removed from metadata 2024.10.22 01:46:48.420972 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 9874 2024.10.22 01:46:48.461722 [ 1233 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 01:46:48.613825 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:48.619896 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:48.678392 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_645_645_0 (state Temporary) 2024.10.22 01:46:48.834586 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_645_645_0 to 202410_658_658_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:48.977019 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_478_98, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0, 202410_482_482_0, 202410_483_483_0] 2024.10.22 01:46:48.983312 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9874 2024.10.22 01:46:48.978660 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_478_98, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0, 202410_482_482_0, 202410_483_483_0] 2024.10.22 01:46:49.123082 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_478_98, 202410_479_479_0, 202410_480_480_0, 202410_481_481_0, 202410_482_482_0, 202410_483_483_0] 2024.10.22 01:46:49.382537 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dfe1ad34a9fb81103399dc88bfc542d9) : [0, 154]: Setting download as failed: File segment: [0, 154], key: dfe1ad34a9fb81103399dc88bfc542d9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:49.384828 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dfe1ad34a9fb81103399dc88bfc542d9): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:46:49.669759 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 1.81 MiB, will set to 1.69 GiB (RSS), difference: 2.48 MiB 2024.10.22 01:46:50.180749 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 956021 2024.10.22 01:46:50.285367 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5200 2024.10.22 01:46:50.367563 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f1005439fc350f1c3208b9882675d0c4) : [0, 103846]: Setting download as failed: File segment: [0, 103846], key: f1005439fc350f1c3208b9882675d0c4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:50.368821 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f1005439fc350f1c3208b9882675d0c4): No space left in cache to reserve 103847 bytes, will continue without cache download 2024.10.22 01:46:50.602863 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f1005439fc350f1c3208b9882675d0c4, offset: 0, size: 0 2024.10.22 01:46:50.605827 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f1005439fc350f1c3208b9882675d0c4 for removal 2024.10.22 01:46:50.610869 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dfe1ad34a9fb81103399dc88bfc542d9, offset: 0, size: 0 2024.10.22 01:46:50.613579 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dfe1ad34a9fb81103399dc88bfc542d9 for removal 2024.10.22 01:46:50.633490 [ 1262 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 01:46:50.683077 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:46:50.685646 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:46:50.710856 [ 1476 ] {} CacheMetadata: Key f1005439fc350f1c3208b9882675d0c4 is removed from metadata 2024.10.22 01:46:50.717673 [ 1476 ] {} CacheMetadata: Key dfe1ad34a9fb81103399dc88bfc542d9 is removed from metadata 2024.10.22 01:46:50.947419 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 71091 2024.10.22 01:46:51.057697 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:51.061234 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:51.322228 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1090_1090_0 (state Temporary) 2024.10.22 01:46:51.444100 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:51.448990 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:51.499543 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 5.42 MiB, will set to 1.69 GiB (RSS), difference: -4.19 MiB 2024.10.22 01:46:51.560902 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ccb61f741d24aeb8330d3ec2c851f2f2) : [0, 155]: Setting download as failed: File segment: [0, 155], key: ccb61f741d24aeb8330d3ec2c851f2f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:51.562983 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ccb61f741d24aeb8330d3ec2c851f2f2): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:46:51.568619 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1090_1090_0 to 202410_1103_1103_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:51.589202 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 540 2024.10.22 01:46:51.618784 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71091 2024.10.22 01:46:51.651795 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_618_618_0 (state Temporary) 2024.10.22 01:46:51.707996 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_563_563_0, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0, 202410_567_567_0, 202410_568_568_0] 2024.10.22 01:46:51.709361 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_618_618_0 to 202410_625_625_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:51.713687 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_563_563_0, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0, 202410_567_567_0, 202410_568_568_0] 2024.10.22 01:46:51.860067 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 956021 2024.10.22 01:46:51.860067 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_563_563_0, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0, 202410_567_567_0, 202410_568_568_0] 2024.10.22 01:46:51.885569 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:51.891776 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:52.162377 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_522_522_0 (state Temporary) 2024.10.22 01:46:52.214081 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 8 old parts to remove. Parts: [202410_1_990_490, 202410_991_996_1, 202410_997_997_0, 202410_998_998_0, 202410_999_999_0, 202410_1000_1000_0, 202410_1001_1001_0, 202410_1002_1002_0] 2024.10.22 01:46:52.217584 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 8 parts from filesystem (serially): Parts: [202410_1_990_490, 202410_991_996_1, 202410_997_997_0, 202410_998_998_0, 202410_999_999_0, 202410_1000_1000_0, 202410_1001_1001_0, 202410_1002_1002_0] 2024.10.22 01:46:52.263954 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 8 parts from memory: Parts: [202410_1_990_490, 202410_991_996_1, 202410_997_997_0, 202410_998_998_0, 202410_999_999_0, 202410_1000_1000_0, 202410_1001_1001_0, 202410_1002_1002_0] 2024.10.22 01:46:52.539041 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_522_522_0 to 202410_532_532_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:52.577000 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(ad493f53f03973cfca0a94a55292cf4b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ad493f53f03973cfca0a94a55292cf4b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:46:52.579638 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 372376, file segment info: File segment: [0, 33554431], key: ad493f53f03973cfca0a94a55292cf4b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:46:52.580746 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 540 2024.10.22 01:46:52.582913 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:53.268763 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1b4ceb167eda3155790267eab8fcc5ed) : [0, 121708]: Setting download as failed: File segment: [0, 121708], key: 1b4ceb167eda3155790267eab8fcc5ed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:53.269766 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1b4ceb167eda3155790267eab8fcc5ed): No space left in cache to reserve 121709 bytes, will continue without cache download 2024.10.22 01:46:53.287707 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1b4ceb167eda3155790267eab8fcc5ed, offset: 0, size: 0 2024.10.22 01:46:53.325593 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1b4ceb167eda3155790267eab8fcc5ed for removal 2024.10.22 01:46:53.329051 [ 1476 ] {} CacheMetadata: Key 1b4ceb167eda3155790267eab8fcc5ed is removed from metadata 2024.10.22 01:46:53.330893 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ccb61f741d24aeb8330d3ec2c851f2f2, offset: 0, size: 0 2024.10.22 01:46:53.332505 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ccb61f741d24aeb8330d3ec2c851f2f2 for removal 2024.10.22 01:46:53.336514 [ 1476 ] {} CacheMetadata: Key ccb61f741d24aeb8330d3ec2c851f2f2 is removed from metadata 2024.10.22 01:46:54.262961 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:46:54.265616 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(91d93e9ff3afcd8b1edf2fce819ad138) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 91d93e9ff3afcd8b1edf2fce819ad138, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:46:54.276629 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 494, file segment info: File segment: [0, 33554431], key: 91d93e9ff3afcd8b1edf2fce819ad138, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:46:54.278157 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:46:54.335621 [ 1711 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/phz/ovcoadcsgpvxuwfltgycydoodzbrl, size 372376 2024.10.22 01:46:54.350771 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 6.59 MiB, will set to 1.69 GiB (RSS), difference: 10.69 MiB 2024.10.22 01:46:54.767618 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(eecda2c4a71ef53cc84627b3ceb93a29) : [0, 180]: Setting download as failed: File segment: [0, 180], key: eecda2c4a71ef53cc84627b3ceb93a29, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:54.769082 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(eecda2c4a71ef53cc84627b3ceb93a29): No space left in cache to reserve 181 bytes, will continue without cache download 2024.10.22 01:46:54.892170 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:46:54.908714 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:46:54.917456 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:46:55.060802 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:46:55.077933 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:46:55.209202 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:46:55.306214 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 41 entries to flush up to offset 72970 2024.10.22 01:46:55.307088 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:46:55.345162 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 01:46:55.392623 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/cxe/qduxgyizhzxczagjupisarmugvtsb (LogSeriesLimiter: on interval from 2024-10-22 01:46:38 to 2024-10-22 01:46:55 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:46:55.626832 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 71146 2024.10.22 01:46:55.826024 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:55.828814 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:55.853281 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1d13c412f1c01fb5366f835d4a4dd828) : [0, 181217]: Setting download as failed: File segment: [0, 181217], key: 1d13c412f1c01fb5366f835d4a4dd828, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:55.855887 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1d13c412f1c01fb5366f835d4a4dd828): No space left in cache to reserve 181218 bytes, will continue without cache download 2024.10.22 01:46:55.880873 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_634_634_0 (state Temporary) 2024.10.22 01:46:55.979236 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:55.981609 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:56.014852 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_634_634_0 to 202410_640_640_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:56.027550 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1091_1091_0 (state Temporary) 2024.10.22 01:46:56.072027 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1091_1091_0 to 202410_1104_1104_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:55.975176 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1d13c412f1c01fb5366f835d4a4dd828, offset: 0, size: 0 2024.10.22 01:46:56.089158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1d13c412f1c01fb5366f835d4a4dd828 for removal 2024.10.22 01:46:56.166111 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 72970 2024.10.22 01:46:56.194321 [ 1476 ] {} CacheMetadata: Key 1d13c412f1c01fb5366f835d4a4dd828 is removed from metadata 2024.10.22 01:46:56.195268 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: eecda2c4a71ef53cc84627b3ceb93a29, offset: 0, size: 0 2024.10.22 01:46:56.197340 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key eecda2c4a71ef53cc84627b3ceb93a29 for removal 2024.10.22 01:46:56.200784 [ 1476 ] {} CacheMetadata: Key eecda2c4a71ef53cc84627b3ceb93a29 is removed from metadata 2024.10.22 01:46:56.220958 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71146 2024.10.22 01:46:56.418723 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 6.82 MiB, will set to 1.69 GiB (RSS), difference: -5.78 MiB 2024.10.22 01:46:56.696416 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 9880 2024.10.22 01:46:56.800195 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:56.803574 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:46:56.835532 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_646_646_0 (state Temporary) 2024.10.22 01:46:56.883249 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_646_646_0 to 202410_659_659_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:46:56.938093 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9880 2024.10.22 01:46:57.060542 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6c60ac8460519cff2d1e8a78eb2d0a25) : [0, 184]: Setting download as failed: File segment: [0, 184], key: 6c60ac8460519cff2d1e8a78eb2d0a25, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:57.062754 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6c60ac8460519cff2d1e8a78eb2d0a25): No space left in cache to reserve 185 bytes, will continue without cache download 2024.10.22 01:46:57.140039 [ 1166 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:46:57.264243 [ 1166 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:46:57.758810 [ 1181 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 01:46:57.817297 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 36 entries to flush up to offset 5236 2024.10.22 01:46:58.251979 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0c39075de6bb7036c01dea2f88f67a7c) : [0, 176963]: Setting download as failed: File segment: [0, 176963], key: 0c39075de6bb7036c01dea2f88f67a7c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:58.253398 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0c39075de6bb7036c01dea2f88f67a7c): No space left in cache to reserve 176964 bytes, will continue without cache download 2024.10.22 01:46:58.277744 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0c39075de6bb7036c01dea2f88f67a7c, offset: 0, size: 0 2024.10.22 01:46:58.279959 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0c39075de6bb7036c01dea2f88f67a7c for removal 2024.10.22 01:46:58.283106 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6c60ac8460519cff2d1e8a78eb2d0a25, offset: 0, size: 0 2024.10.22 01:46:58.284121 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6c60ac8460519cff2d1e8a78eb2d0a25 for removal 2024.10.22 01:46:58.383490 [ 1476 ] {} CacheMetadata: Key 6c60ac8460519cff2d1e8a78eb2d0a25 is removed from metadata 2024.10.22 01:46:58.395708 [ 1476 ] {} CacheMetadata: Key 0c39075de6bb7036c01dea2f88f67a7c is removed from metadata 2024.10.22 01:46:58.790646 [ 1268 ] {} DatabaseCatalog: Execution took 314 ms. 2024.10.22 01:46:58.898945 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 7.23 MiB, will set to 1.70 GiB (RSS), difference: -1.18 MiB 2024.10.22 01:46:58.964140 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 957561 2024.10.22 01:46:59.269357 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f2ffcf19026bd2f73ea3ac163ccd7034) : [0, 156]: Setting download as failed: File segment: [0, 156], key: f2ffcf19026bd2f73ea3ac163ccd7034, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:46:59.271635 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f2ffcf19026bd2f73ea3ac163ccd7034): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:46:59.981041 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:46:59.986243 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:00.243020 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 71193 2024.10.22 01:47:00.263382 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_619_619_0 (state Temporary) 2024.10.22 01:47:00.468010 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_619_619_0 to 202410_626_626_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:00.516354 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 957561 2024.10.22 01:47:00.501345 [ 1172 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_616_120 to 202410_617_622_1 2024.10.22 01:47:00.536619 [ 1172 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:00.547819 [ 1172 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:00.585222 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_622_121} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_616_120 to 202410_617_622_1 into Wide with storage Full 2024.10.22 01:47:00.592832 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(795487f84e7b259f8e1c8aa51b14b485) : [0, 89476]: Setting download as failed: File segment: [0, 89476], key: 795487f84e7b259f8e1c8aa51b14b485, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:00.597526 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(795487f84e7b259f8e1c8aa51b14b485): No space left in cache to reserve 89477 bytes, will continue without cache download 2024.10.22 01:47:00.687663 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:47:00.691542 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:47:00.757770 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_622_121} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:00.764249 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_622_121} MergeTreeSequentialSource: Reading 118 marks from part 202410_1_616_120, total 956136 rows starting from the beginning of the part 2024.10.22 01:47:00.817686 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:00.840239 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:00.845764 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 795487f84e7b259f8e1c8aa51b14b485, offset: 0, size: 0 2024.10.22 01:47:00.850225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 795487f84e7b259f8e1c8aa51b14b485 for removal 2024.10.22 01:47:00.852457 [ 1476 ] {} CacheMetadata: Key 795487f84e7b259f8e1c8aa51b14b485 is removed from metadata 2024.10.22 01:47:00.851493 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_622_121} MergeTreeSequentialSource: Reading 3 marks from part 202410_617_622_1, total 10010 rows starting from the beginning of the part 2024.10.22 01:47:00.853284 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f2ffcf19026bd2f73ea3ac163ccd7034, offset: 0, size: 0 2024.10.22 01:47:00.855582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f2ffcf19026bd2f73ea3ac163ccd7034 for removal 2024.10.22 01:47:00.860760 [ 1476 ] {} CacheMetadata: Key f2ffcf19026bd2f73ea3ac163ccd7034 is removed from metadata 2024.10.22 01:47:00.971601 [ 1164 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 01:47:01.112535 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1092_1092_0 (state Temporary) 2024.10.22 01:47:01.254317 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1092_1092_0 to 202410_1105_1105_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:01.628834 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 6.80 MiB, will set to 1.70 GiB (RSS), difference: -11.02 MiB 2024.10.22 01:47:01.794502 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71193 2024.10.22 01:47:02.022812 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(123590a0a658e18e9fc0965cfb73bedd) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 123590a0a658e18e9fc0965cfb73bedd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:02.027158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(123590a0a658e18e9fc0965cfb73bedd): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:47:02.856158 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_577_14, 202410_578_578_0, 202410_579_579_0, 202410_580_580_0, 202410_581_581_0, 202410_582_582_0] 2024.10.22 01:47:02.860512 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_577_14, 202410_578_578_0, 202410_579_579_0, 202410_580_580_0, 202410_581_581_0, 202410_582_582_0] 2024.10.22 01:47:02.897085 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_577_14, 202410_578_578_0, 202410_579_579_0, 202410_580_580_0, 202410_581_581_0, 202410_582_582_0] 2024.10.22 01:47:03.082994 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(98a7c3e1189727dd346e9e0d9fbcdb42) : [0, 84069]: Setting download as failed: File segment: [0, 84069], key: 98a7c3e1189727dd346e9e0d9fbcdb42, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:03.087922 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(98a7c3e1189727dd346e9e0d9fbcdb42): No space left in cache to reserve 84070 bytes, will continue without cache download 2024.10.22 01:47:03.109763 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_556_110, 202410_557_562_1] 2024.10.22 01:47:03.114368 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_556_110, 202410_557_562_1] 2024.10.22 01:47:03.147519 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_556_110, 202410_557_562_1] 2024.10.22 01:47:03.152550 [ 1191 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 254 ms. 2024.10.22 01:47:03.161755 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 98a7c3e1189727dd346e9e0d9fbcdb42, offset: 0, size: 0 2024.10.22 01:47:03.163382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 98a7c3e1189727dd346e9e0d9fbcdb42 for removal 2024.10.22 01:47:03.166053 [ 1476 ] {} CacheMetadata: Key 98a7c3e1189727dd346e9e0d9fbcdb42 is removed from metadata 2024.10.22 01:47:03.167585 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 123590a0a658e18e9fc0965cfb73bedd, offset: 0, size: 0 2024.10.22 01:47:03.169940 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_622_121/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:47:03.169654 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 123590a0a658e18e9fc0965cfb73bedd for removal 2024.10.22 01:47:03.174715 [ 1476 ] {} CacheMetadata: Key 123590a0a658e18e9fc0965cfb73bedd is removed from metadata 2024.10.22 01:47:03.459912 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 8.79 MiB, will set to 1.70 GiB (RSS), difference: 1.92 MiB 2024.10.22 01:47:03.670440 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 81 entries to flush up to offset 73051 2024.10.22 01:47:03.910484 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:03.913795 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:03.961015 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_635_635_0 (state Temporary) 2024.10.22 01:47:04.009049 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_635_635_0 to 202410_641_641_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:04.061310 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73051 2024.10.22 01:47:04.063010 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(cd26f1f1074677337b66efa14609acc5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cd26f1f1074677337b66efa14609acc5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:04.074956 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: cd26f1f1074677337b66efa14609acc5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:04.077387 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:04.441079 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 4 entries to flush up to offset 9884 2024.10.22 01:47:04.777552 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:04.782927 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:04.923788 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_647_647_0 (state Temporary) 2024.10.22 01:47:04.990284 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_647_647_0 to 202410_660_660_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:05.014705 [ 1193 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 4 parts from 202410_1_655_131 to 202410_658_658_0 2024.10.22 01:47:05.042402 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9884 2024.10.22 01:47:05.143369 [ 1193 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:05.152918 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:05.174627 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_658_132} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_655_131 to 202410_658_658_0 into Compact with storage Full 2024.10.22 01:47:05.250913 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6e0c04daa375fa85f9d25b3d49063468) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 6e0c04daa375fa85f9d25b3d49063468, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:05.253148 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6e0c04daa375fa85f9d25b3d49063468): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 01:47:05.259752 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_658_132} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:05.275535 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_658_132} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_655_131, total 9928 rows starting from the beginning of the part 2024.10.22 01:47:05.314165 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_658_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_656_656_0, total 19 rows starting from the beginning of the part 2024.10.22 01:47:05.459959 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_658_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_657_657_0, total 6 rows starting from the beginning of the part 2024.10.22 01:47:05.484295 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_658_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_658_658_0, total 27 rows starting from the beginning of the part 2024.10.22 01:47:05.805002 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 71266 2024.10.22 01:47:06.077499 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:06.082706 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:06.259094 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1093_1093_0 (state Temporary) 2024.10.22 01:47:06.310579 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1093_1093_0 to 202410_1106_1106_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:06.344076 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ed546c868e9dce38ebd34237d8a5ceca) : [0, 51868]: Setting download as failed: File segment: [0, 51868], key: ed546c868e9dce38ebd34237d8a5ceca, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:06.346407 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ed546c868e9dce38ebd34237d8a5ceca): No space left in cache to reserve 51869 bytes, will continue without cache download 2024.10.22 01:47:06.362681 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1101_1 to 202410_1106_1106_0 2024.10.22 01:47:06.370480 [ 1231 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:06.389464 [ 1231 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:06.468997 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71266 2024.10.22 01:47:06.464462 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ed546c868e9dce38ebd34237d8a5ceca, offset: 0, size: 0 2024.10.22 01:47:06.476667 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ed546c868e9dce38ebd34237d8a5ceca for removal 2024.10.22 01:47:06.479668 [ 1476 ] {} CacheMetadata: Key ed546c868e9dce38ebd34237d8a5ceca is removed from metadata 2024.10.22 01:47:06.483389 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6e0c04daa375fa85f9d25b3d49063468, offset: 0, size: 0 2024.10.22 01:47:06.485761 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6e0c04daa375fa85f9d25b3d49063468 for removal 2024.10.22 01:47:06.489808 [ 1476 ] {} CacheMetadata: Key 6e0c04daa375fa85f9d25b3d49063468 is removed from metadata 2024.10.22 01:47:06.543909 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1101_1 to 202410_1106_1106_0 into Compact with storage Full 2024.10.22 01:47:06.551717 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 14.82 MiB, will set to 1.71 GiB (RSS), difference: 7.58 MiB 2024.10.22 01:47:06.594368 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:06.621044 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_658_132} MergeTask::MergeProjectionsStage: Merge sorted 9980 rows, containing 12 columns (12 merged, 0 gathered) in 1.451280398 sec., 6876.686279063214 rows/sec., 1.07 MiB/sec. 2024.10.22 01:47:06.725248 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1101_1, total 385 rows starting from the beginning of the part 2024.10.22 01:47:06.769343 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_658_132} MergedBlockOutputStream: filled checksums 202410_1_658_132 (state Temporary) 2024.10.22 01:47:06.778839 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1102_1102_0, total 73 rows starting from the beginning of the part 2024.10.22 01:47:06.858402 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1103_1103_0, total 57 rows starting from the beginning of the part 2024.10.22 01:47:06.936837 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1104_1104_0, total 55 rows starting from the beginning of the part 2024.10.22 01:47:07.031806 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1105_1105_0, total 47 rows starting from the beginning of the part 2024.10.22 01:47:07.104651 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1106_1106_0, total 73 rows starting from the beginning of the part 2024.10.22 01:47:07.123647 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_658_132} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_658_132 to 202410_1_658_132 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:07.127638 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_658_132} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 4 parts: [202410_1_655_131, 202410_658_658_0] -> 202410_1_658_132 2024.10.22 01:47:07.274689 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.55 MiB. 2024.10.22 01:47:07.363236 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7b4446735b388ca4372d503d11361e6e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7b4446735b388ca4372d503d11361e6e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:07.374184 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 330477, file segment info: File segment: [0, 33554431], key: 7b4446735b388ca4372d503d11361e6e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:07.376252 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:07.520580 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 958716 2024.10.22 01:47:07.582748 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 541 2024.10.22 01:47:07.860741 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:07.864480 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:07.952412 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_523_523_0 (state Temporary) 2024.10.22 01:47:08.145417 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:08.151711 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:08.173617 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} MergeTask::MergeProjectionsStage: Merge sorted 690 rows, containing 13 columns (13 merged, 0 gathered) in 1.747442192 sec., 394.86284762889596 rows/sec., 69.59 KiB/sec. 2024.10.22 01:47:08.217658 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} MergedBlockOutputStream: filled checksums 202410_1096_1106_2 (state Temporary) 2024.10.22 01:47:08.238720 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_523_523_0 to 202410_533_533_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:08.273100 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_620_620_0 (state Temporary) 2024.10.22 01:47:08.275182 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_528_108 to 202410_533_533_0 2024.10.22 01:47:08.283359 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:08.286487 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:08.321827 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(28cd61d77285a05b0b67fc69926885e3) : [0, 186]: Setting download as failed: File segment: [0, 186], key: 28cd61d77285a05b0b67fc69926885e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:08.325274 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(28cd61d77285a05b0b67fc69926885e3): No space left in cache to reserve 187 bytes, will continue without cache download 2024.10.22 01:47:08.327249 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_528_108 to 202410_533_533_0 into Compact with storage Full 2024.10.22 01:47:08.417474 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 541 2024.10.22 01:47:08.446554 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_620_620_0 to 202410_627_627_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:08.463439 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 958716 2024.10.22 01:47:08.557513 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1106_2 to 202410_1096_1106_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:08.560598 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1106_2} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1101_1, 202410_1106_1106_0] -> 202410_1096_1106_2 2024.10.22 01:47:08.616444 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:47:08.677195 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:08.720836 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_528_108, total 572 rows starting from the beginning of the part 2024.10.22 01:47:09.061789 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(2a87de3ddf831cbdf1e9a4ecc9f113f2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2a87de3ddf831cbdf1e9a4ecc9f113f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:09.076536 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 747, file segment info: File segment: [0, 33554431], key: 2a87de3ddf831cbdf1e9a4ecc9f113f2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:09.079962 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:09.082439 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_529_529_0, total 1 rows starting from the beginning of the part 2024.10.22 01:47:09.135840 [ 1225 ] {} DatabaseCatalog: Execution took 338 ms. 2024.10.22 01:47:09.290354 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_530_530_0, total 1 rows starting from the beginning of the part 2024.10.22 01:47:09.693440 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_531_531_0, total 1 rows starting from the beginning of the part 2024.10.22 01:47:09.899048 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6304a253c6ed5ec0087340d11548a307) : [0, 179984]: Setting download as failed: File segment: [0, 179984], key: 6304a253c6ed5ec0087340d11548a307, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:09.903321 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6304a253c6ed5ec0087340d11548a307): No space left in cache to reserve 179985 bytes, will continue without cache download 2024.10.22 01:47:09.965761 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_532_532_0, total 1 rows starting from the beginning of the part 2024.10.22 01:47:10.146670 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_533_533_0, total 1 rows starting from the beginning of the part 2024.10.22 01:47:10.358718 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6304a253c6ed5ec0087340d11548a307, offset: 0, size: 0 2024.10.22 01:47:10.361140 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6304a253c6ed5ec0087340d11548a307 for removal 2024.10.22 01:47:10.364720 [ 1476 ] {} CacheMetadata: Key 6304a253c6ed5ec0087340d11548a307 is removed from metadata 2024.10.22 01:47:10.365037 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 28cd61d77285a05b0b67fc69926885e3, offset: 0, size: 0 2024.10.22 01:47:10.367478 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 28cd61d77285a05b0b67fc69926885e3 for removal 2024.10.22 01:47:10.386166 [ 1476 ] {} CacheMetadata: Key 28cd61d77285a05b0b67fc69926885e3 is removed from metadata 2024.10.22 01:47:10.478884 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 71342 2024.10.22 01:47:10.488810 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 17.19 MiB, will set to 1.71 GiB (RSS), difference: 7.60 MiB 2024.10.22 01:47:10.690132 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:47:10.695975 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:47:10.733684 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:10.740117 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:10.800968 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1094_1094_0 (state Temporary) 2024.10.22 01:47:11.059438 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1094_1094_0 to 202410_1107_1107_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:11.116309 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71342 2024.10.22 01:47:11.151008 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(1bbc813bfa14aef875ea361a39c79c07) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1bbc813bfa14aef875ea361a39c79c07, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:11.173977 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 807764, file segment info: File segment: [0, 33554431], key: 1bbc813bfa14aef875ea361a39c79c07, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:11.179496 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:11.335512 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_596_119, 202410_597_597_0, 202410_598_598_0, 202410_599_599_0, 202410_600_600_0, 202410_601_601_0] 2024.10.22 01:47:11.345205 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_596_119, 202410_597_597_0, 202410_598_598_0, 202410_599_599_0, 202410_600_600_0, 202410_601_601_0] 2024.10.22 01:47:11.471444 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_596_119, 202410_597_597_0, 202410_598_598_0, 202410_599_599_0, 202410_600_600_0, 202410_601_601_0] 2024.10.22 01:47:11.574697 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 77 entries to flush up to offset 73128 2024.10.22 01:47:11.800352 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:11.804028 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:11.861362 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_636_636_0 (state Temporary) 2024.10.22 01:47:12.002357 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_636_636_0 to 202410_642_642_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:12.108777 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73128 2024.10.22 01:47:12.200218 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ae8f905bc5a92df8246a4d0761f9ab8b) : [0, 153]: Setting download as failed: File segment: [0, 153], key: ae8f905bc5a92df8246a4d0761f9ab8b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:12.204944 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ae8f905bc5a92df8246a4d0761f9ab8b): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:47:12.268564 [ 1257 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:47:12.292601 [ 1257 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:47:12.544398 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 9902 2024.10.22 01:47:12.803422 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} MergeTask::MergeProjectionsStage: Merge sorted 577 rows, containing 32 columns (32 merged, 0 gathered) in 4.484078393 sec., 128.6775005764267 rows/sec., 11.79 KiB/sec. 2024.10.22 01:47:12.853562 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} MergedBlockOutputStream: filled checksums 202410_1_533_109 (state Temporary) 2024.10.22 01:47:12.877011 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:12.882874 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:12.937301 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_648_648_0 (state Temporary) 2024.10.22 01:47:12.995104 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_648_648_0 to 202410_661_661_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:13.158551 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9902 2024.10.22 01:47:13.147397 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b58ee33423676924c64024b62c2784ec) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b58ee33423676924c64024b62c2784ec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:13.172696 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 934, file segment info: File segment: [0, 33554431], key: b58ee33423676924c64024b62c2784ec, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:13.177405 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:13.242879 [ 1676 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/iic/fiicfoalcbvrevszrvdywqrkbvbsp, size 807764 2024.10.22 01:47:13.475053 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_533_109 to 202410_1_533_109 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:13.476920 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_533_109} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_528_108, 202410_533_533_0] -> 202410_1_533_109 2024.10.22 01:47:13.648157 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:47:13.864530 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:47:13.901289 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:47:13.929475 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:47:13.968807 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_622_121/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:47:14.042674 [ 1241 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 438 ms. 2024.10.22 01:47:14.106727 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:47:14.149784 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:47:14.177315 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:47:14.235846 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:47:14.260973 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:47:14.394071 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 01:47:14.419887 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9e40853a9e1dd88ca8a4ea9b511d1f04) : [0, 3439]: Setting download as failed: File segment: [0, 3439], key: 9e40853a9e1dd88ca8a4ea9b511d1f04, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:14.495043 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 14.05 MiB, will set to 1.71 GiB (RSS), difference: 5.63 MiB 2024.10.22 01:47:14.523370 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9e40853a9e1dd88ca8a4ea9b511d1f04): No space left in cache to reserve 3440 bytes, will continue without cache download 2024.10.22 01:47:14.654338 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ynt/vbleqtivtilovurhqcteasusnskos (LogSeriesLimiter: on interval from 2024-10-22 01:46:55 to 2024-10-22 01:47:14 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:47:14.646944 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9e40853a9e1dd88ca8a4ea9b511d1f04, offset: 0, size: 0 2024.10.22 01:47:14.658874 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9e40853a9e1dd88ca8a4ea9b511d1f04 for removal 2024.10.22 01:47:14.659853 [ 1476 ] {} CacheMetadata: Key 9e40853a9e1dd88ca8a4ea9b511d1f04 is removed from metadata 2024.10.22 01:47:14.662309 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ae8f905bc5a92df8246a4d0761f9ab8b, offset: 0, size: 0 2024.10.22 01:47:14.663387 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ae8f905bc5a92df8246a4d0761f9ab8b for removal 2024.10.22 01:47:14.666094 [ 1476 ] {} CacheMetadata: Key ae8f905bc5a92df8246a4d0761f9ab8b is removed from metadata 2024.10.22 01:47:15.131847 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 71403 2024.10.22 01:47:15.344217 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(f6703dbac530b46ede917a3d4e1710e3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f6703dbac530b46ede917a3d4e1710e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:47:15.347809 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: f6703dbac530b46ede917a3d4e1710e3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:47:15.351751 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:15.466879 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 959486 2024.10.22 01:47:15.480423 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:15.484958 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:15.545188 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:15.552702 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:15.576837 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1095_1095_0 (state Temporary) 2024.10.22 01:47:15.622848 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1095_1095_0 to 202410_1108_1108_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:15.817206 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71403 2024.10.22 01:47:15.920440 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 542 2024.10.22 01:47:16.093725 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:16.101031 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:16.134150 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:16.142302 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:16.152621 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bedd701bb9ef1dff4212d1ab4920220e) : [0, 153]: Setting download as failed: File segment: [0, 153], key: bedd701bb9ef1dff4212d1ab4920220e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:16.155294 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bedd701bb9ef1dff4212d1ab4920220e): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:47:16.284682 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_621_621_0 (state Temporary) 2024.10.22 01:47:16.406369 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_621_621_0 to 202410_628_628_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:16.439675 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:47:16.463496 [ 1237 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_623_623_0 to 202410_628_628_0 2024.10.22 01:47:16.473612 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_524_524_0 (state Temporary) 2024.10.22 01:47:16.491754 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 959486 2024.10.22 01:47:16.584140 [ 1237 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:16.590570 [ 1237 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:16.631722 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_623_623_0 to 202410_628_628_0 into Compact with storage Full 2024.10.22 01:47:16.661695 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:16.669159 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_623_623_0, total 770 rows starting from the beginning of the part 2024.10.22 01:47:16.677829 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_624_624_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:47:16.690914 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_625_625_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:47:16.704630 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_626_626_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:47:16.713439 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_627_627_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:47:16.737174 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_628_628_0, total 770 rows starting from the beginning of the part 2024.10.22 01:47:16.782689 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_524_524_0 to 202410_534_534_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:16.908044 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 542 2024.10.22 01:47:17.069190 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dc2ff36a438d47e93110e215ae3d6422) : [0, 4409]: Setting download as failed: File segment: [0, 4409], key: dc2ff36a438d47e93110e215ae3d6422, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:17.071699 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dc2ff36a438d47e93110e215ae3d6422): No space left in cache to reserve 4410 bytes, will continue without cache download 2024.10.22 01:47:17.083203 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dc2ff36a438d47e93110e215ae3d6422, offset: 0, size: 0 2024.10.22 01:47:17.117738 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dc2ff36a438d47e93110e215ae3d6422 for removal 2024.10.22 01:47:17.127083 [ 1476 ] {} CacheMetadata: Key dc2ff36a438d47e93110e215ae3d6422 is removed from metadata 2024.10.22 01:47:17.128725 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bedd701bb9ef1dff4212d1ab4920220e, offset: 0, size: 0 2024.10.22 01:47:17.135744 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bedd701bb9ef1dff4212d1ab4920220e for removal 2024.10.22 01:47:17.143097 [ 1476 ] {} CacheMetadata: Key bedd701bb9ef1dff4212d1ab4920220e is removed from metadata 2024.10.22 01:47:17.567192 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 12.89 MiB, will set to 1.71 GiB (RSS), difference: -23.77 MiB 2024.10.22 01:47:17.583463 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} MergeTask::MergeProjectionsStage: Merge sorted 6930 rows, containing 4 columns (4 merged, 0 gathered) in 0.972432099 sec., 7126.4615875251975 rows/sec., 111.35 KiB/sec. 2024.10.22 01:47:17.875128 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} MergedBlockOutputStream: filled checksums 202410_623_628_1 (state Temporary) 2024.10.22 01:47:18.046348 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_623_628_1 to 202410_623_628_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:18.051140 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_623_628_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_623_623_0, 202410_628_628_0] -> 202410_623_628_1 2024.10.22 01:47:18.173717 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:47:18.358742 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(17bb75f5172572fef0132f1ca50a5c36) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 17bb75f5172572fef0132f1ca50a5c36, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:18.363514 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(17bb75f5172572fef0132f1ca50a5c36): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:47:19.410452 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_131_131_0 (state Temporary) 2024.10.22 01:47:19.479246 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(16d38671302dab334df5f7e1e38feee1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 16d38671302dab334df5f7e1e38feee1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:19.482447 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1028, file segment info: File segment: [0, 33554431], key: 16d38671302dab334df5f7e1e38feee1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:19.484790 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:19.615691 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 73 entries to flush up to offset 73201 2024.10.22 01:47:19.752227 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:19.757568 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:19.819151 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_637_637_0 (state Temporary) 2024.10.22 01:47:19.842692 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 71464 2024.10.22 01:47:19.964224 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_637_637_0 to 202410_643_643_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:20.011120 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73201 2024.10.22 01:47:20.011806 [ 1166 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_638_27 to 202410_643_643_0 2024.10.22 01:47:20.060356 [ 1166 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:20.062796 [ 1166 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:20.074866 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_638_27 to 202410_643_643_0 into Compact with storage Full 2024.10.22 01:47:20.103459 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:20.108031 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:20.124929 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(428479004e0c0b97d90fd2c1b68f9ce2) : [0, 43603]: Setting download as failed: File segment: [0, 43603], key: 428479004e0c0b97d90fd2c1b68f9ce2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:20.126291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(428479004e0c0b97d90fd2c1b68f9ce2): No space left in cache to reserve 43604 bytes, will continue without cache download 2024.10.22 01:47:20.158059 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1096_1096_0 (state Temporary) 2024.10.22 01:47:20.180937 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_996_491, 202410_997_1002_1] 2024.10.22 01:47:20.188219 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_996_491, 202410_997_1002_1] 2024.10.22 01:47:20.202273 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1096_1096_0 to 202410_1109_1109_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:20.268819 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71464 2024.10.22 01:47:20.269448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 428479004e0c0b97d90fd2c1b68f9ce2, offset: 0, size: 0 2024.10.22 01:47:20.270937 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 428479004e0c0b97d90fd2c1b68f9ce2 for removal 2024.10.22 01:47:20.272623 [ 1476 ] {} CacheMetadata: Key 428479004e0c0b97d90fd2c1b68f9ce2 is removed from metadata 2024.10.22 01:47:20.273454 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 17bb75f5172572fef0132f1ca50a5c36, offset: 0, size: 0 2024.10.22 01:47:20.275790 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 17bb75f5172572fef0132f1ca50a5c36 for removal 2024.10.22 01:47:20.279436 [ 1476 ] {} CacheMetadata: Key 17bb75f5172572fef0132f1ca50a5c36 is removed from metadata 2024.10.22 01:47:20.283653 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_996_491, 202410_997_1002_1] 2024.10.22 01:47:20.359346 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:20.369629 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_638_27, total 13698 rows starting from the beginning of the part 2024.10.22 01:47:20.408504 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_639_639_0, total 106 rows starting from the beginning of the part 2024.10.22 01:47:20.457650 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_640_640_0, total 41 rows starting from the beginning of the part 2024.10.22 01:47:20.478356 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_641_641_0, total 81 rows starting from the beginning of the part 2024.10.22 01:47:20.532550 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_642_642_0, total 77 rows starting from the beginning of the part 2024.10.22 01:47:20.661478 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 9922 2024.10.22 01:47:20.675632 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_643_643_0, total 73 rows starting from the beginning of the part 2024.10.22 01:47:20.691161 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:47:20.693927 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:47:20.771888 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:20.777188 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:20.807210 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(21a1c76d17142ad64d69431790da91c0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 21a1c76d17142ad64d69431790da91c0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:20.811810 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 794, file segment info: File segment: [0, 33554431], key: 21a1c76d17142ad64d69431790da91c0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:47:20.817778 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:20.845798 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_649_649_0 (state Temporary) 2024.10.22 01:47:20.896165 [ 1725 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/cxe/qduxgyizhzxczagjupisarmugvtsb, size 1198987 2024.10.22 01:47:20.901598 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_649_649_0 to 202410_662_662_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:20.958018 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9922 2024.10.22 01:47:21.062658 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:47:21.191095 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:47:21.209320 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:47:21.332899 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:47:21.351463 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:47:21.384244 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:47:21.432868 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:47:21.676856 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column RemoteIP 2024.10.22 01:47:21.849036 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 13.98 MiB, will set to 1.71 GiB (RSS), difference: -5.62 MiB 2024.10.22 01:47:21.911606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(59fba5eb55fcc42cd887e871ca1ddef1) : [0, 151]: Setting download as failed: File segment: [0, 151], key: 59fba5eb55fcc42cd887e871ca1ddef1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:21.924245 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/enu/nbpiffqmmvqdtejjopxkgmeuncnrt (LogSeriesLimiter: on interval from 2024-10-22 01:47:14 to 2024-10-22 01:47:21 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:47:21.955629 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(59fba5eb55fcc42cd887e871ca1ddef1): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:47:22.271555 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_643_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:47:22.778291 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} MergeTask::MergeProjectionsStage: Merge sorted 14076 rows, containing 13 columns (13 merged, 0 gathered) in 2.706853586 sec., 5200.133495510016 rows/sec., 2.25 MiB/sec. 2024.10.22 01:47:22.949268 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} MergedBlockOutputStream: filled checksums 202410_507_643_28 (state Temporary) 2024.10.22 01:47:23.112763 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_643_28 to 202410_507_643_28 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:23.118831 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_643_28} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_638_27, 202410_643_643_0] -> 202410_507_643_28 2024.10.22 01:47:23.163696 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.05 MiB. 2024.10.22 01:47:23.268032 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1a98f991e06eee05b161a1efc5fdaacb) : [0, 40430]: Setting download as failed: File segment: [0, 40430], key: 1a98f991e06eee05b161a1efc5fdaacb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:23.268835 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1a98f991e06eee05b161a1efc5fdaacb): No space left in cache to reserve 40431 bytes, will continue without cache download 2024.10.22 01:47:23.389234 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1a98f991e06eee05b161a1efc5fdaacb, offset: 0, size: 0 2024.10.22 01:47:23.410919 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1a98f991e06eee05b161a1efc5fdaacb for removal 2024.10.22 01:47:23.415329 [ 1476 ] {} CacheMetadata: Key 1a98f991e06eee05b161a1efc5fdaacb is removed from metadata 2024.10.22 01:47:23.416799 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 59fba5eb55fcc42cd887e871ca1ddef1, offset: 0, size: 0 2024.10.22 01:47:23.420742 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 59fba5eb55fcc42cd887e871ca1ddef1 for removal 2024.10.22 01:47:23.424929 [ 1476 ] {} CacheMetadata: Key 59fba5eb55fcc42cd887e871ca1ddef1 is removed from metadata 2024.10.22 01:47:23.500549 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 960641 2024.10.22 01:47:23.684514 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 11.42 MiB, will set to 1.71 GiB (RSS), difference: -1.70 MiB 2024.10.22 01:47:24.278637 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 71534 2024.10.22 01:47:24.413663 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 543 2024.10.22 01:47:24.522861 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:24.529003 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:24.557378 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:24.559445 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:24.561352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(be6c854c20e45447295181432aa5eb73) : [0, 216]: Setting download as failed: File segment: [0, 216], key: be6c854c20e45447295181432aa5eb73, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:24.562954 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(be6c854c20e45447295181432aa5eb73): No space left in cache to reserve 217 bytes, will continue without cache download 2024.10.22 01:47:24.644191 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1097_1097_0 (state Temporary) 2024.10.22 01:47:24.760108 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_622_622_0 (state Temporary) 2024.10.22 01:47:24.760868 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1003_1003_0, 202410_1004_1004_0, 202410_1005_1005_0, 202410_1006_1006_0, 202410_1007_1007_0, 202410_1008_1008_0] 2024.10.22 01:47:24.762939 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1003_1003_0, 202410_1004_1004_0, 202410_1005_1005_0, 202410_1006_1006_0, 202410_1007_1007_0, 202410_1008_1008_0] 2024.10.22 01:47:24.779127 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_131_131_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:47:24.783571 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1097_1097_0 to 202410_1110_1110_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:24.841788 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71534 2024.10.22 01:47:24.864846 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:24.866967 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:24.920354 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1003_1003_0, 202410_1004_1004_0, 202410_1005_1005_0, 202410_1006_1006_0, 202410_1007_1007_0, 202410_1008_1008_0] 2024.10.22 01:47:24.937788 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_622_622_0 to 202410_629_629_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:24.983622 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_622_121/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:47:25.017425 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 960641 2024.10.22 01:47:25.159800 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_525_525_0 (state Temporary) 2024.10.22 01:47:25.345788 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_525_525_0 to 202410_535_535_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:25.538628 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 543 2024.10.22 01:47:25.655668 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c0feaff49392ad481673e3067fc5a344) : [0, 300483]: Setting download as failed: File segment: [0, 300483], key: c0feaff49392ad481673e3067fc5a344, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:25.657691 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c0feaff49392ad481673e3067fc5a344): No space left in cache to reserve 300484 bytes, will continue without cache download 2024.10.22 01:47:25.780821 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c0feaff49392ad481673e3067fc5a344, offset: 0, size: 0 2024.10.22 01:47:25.783575 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c0feaff49392ad481673e3067fc5a344 for removal 2024.10.22 01:47:25.790444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: be6c854c20e45447295181432aa5eb73, offset: 0, size: 0 2024.10.22 01:47:25.790505 [ 1476 ] {} CacheMetadata: Key c0feaff49392ad481673e3067fc5a344 is removed from metadata 2024.10.22 01:47:25.792423 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key be6c854c20e45447295181432aa5eb73 for removal 2024.10.22 01:47:25.797733 [ 1476 ] {} CacheMetadata: Key be6c854c20e45447295181432aa5eb73 is removed from metadata 2024.10.22 01:47:26.772509 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(157d1c62800b301e03666c2dfdeaeaa1) : [0, 160]: Setting download as failed: File segment: [0, 160], key: 157d1c62800b301e03666c2dfdeaeaa1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:26.775150 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(157d1c62800b301e03666c2dfdeaeaa1): No space left in cache to reserve 161 bytes, will continue without cache download 2024.10.22 01:47:27.303196 [ 1201 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:47:27.344552 [ 1201 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:47:27.538913 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 73 entries to flush up to offset 73274 2024.10.22 01:47:27.705634 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e3dcd16a156b452b9996d7ce67f7c970) : [0, 254064]: Setting download as failed: File segment: [0, 254064], key: e3dcd16a156b452b9996d7ce67f7c970, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:27.706948 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e3dcd16a156b452b9996d7ce67f7c970): No space left in cache to reserve 254065 bytes, will continue without cache download 2024.10.22 01:47:27.754510 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e3dcd16a156b452b9996d7ce67f7c970, offset: 0, size: 0 2024.10.22 01:47:27.759550 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:27.760311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e3dcd16a156b452b9996d7ce67f7c970 for removal 2024.10.22 01:47:27.761445 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:27.763368 [ 1476 ] {} CacheMetadata: Key e3dcd16a156b452b9996d7ce67f7c970 is removed from metadata 2024.10.22 01:47:27.765149 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 157d1c62800b301e03666c2dfdeaeaa1, offset: 0, size: 0 2024.10.22 01:47:27.766659 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 157d1c62800b301e03666c2dfdeaeaa1 for removal 2024.10.22 01:47:27.767964 [ 1476 ] {} CacheMetadata: Key 157d1c62800b301e03666c2dfdeaeaa1 is removed from metadata 2024.10.22 01:47:27.940727 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_638_638_0 (state Temporary) 2024.10.22 01:47:28.079599 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_638_638_0 to 202410_644_644_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:28.136269 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73274 2024.10.22 01:47:28.459826 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 9934 2024.10.22 01:47:28.576821 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:28.681004 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:28.739640 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_650_650_0 (state Temporary) 2024.10.22 01:47:28.779535 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ea6aa36e29127ecdc92d2e238c90e69d) : [0, 178]: Setting download as failed: File segment: [0, 178], key: ea6aa36e29127ecdc92d2e238c90e69d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:28.782288 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ea6aa36e29127ecdc92d2e238c90e69d): No space left in cache to reserve 179 bytes, will continue without cache download 2024.10.22 01:47:28.853670 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 71590 2024.10.22 01:47:28.968913 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:28.973700 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:28.978411 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_650_650_0 to 202410_663_663_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:29.051746 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9934 2024.10.22 01:47:29.060451 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_658_132 to 202410_663_663_0 2024.10.22 01:47:29.070455 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:29.082375 [ 1220 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:29.121435 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_658_132 to 202410_663_663_0 into Compact with storage Full 2024.10.22 01:47:29.149366 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1098_1098_0 (state Temporary) 2024.10.22 01:47:29.225084 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1098_1098_0 to 202410_1111_1111_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:29.263669 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1106_2 to 202410_1111_1111_0 2024.10.22 01:47:29.265982 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:29.270382 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_658_132, total 9980 rows starting from the beginning of the part 2024.10.22 01:47:29.270252 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:29.273403 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71590 2024.10.22 01:47:29.274778 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:29.282413 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_659_659_0, total 6 rows starting from the beginning of the part 2024.10.22 01:47:29.296872 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1106_2 to 202410_1111_1111_0 into Compact with storage Full 2024.10.22 01:47:29.375401 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:29.383956 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1106_2, total 690 rows starting from the beginning of the part 2024.10.22 01:47:29.400526 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_660_660_0, total 4 rows starting from the beginning of the part 2024.10.22 01:47:29.536936 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_661_661_0, total 18 rows starting from the beginning of the part 2024.10.22 01:47:29.558951 [ 1213 ] {} DatabaseCatalog: Execution took 273 ms. 2024.10.22 01:47:29.566312 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_662_662_0, total 20 rows starting from the beginning of the part 2024.10.22 01:47:29.572715 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1107_1107_0, total 76 rows starting from the beginning of the part 2024.10.22 01:47:29.618198 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_663_663_0, total 12 rows starting from the beginning of the part 2024.10.22 01:47:29.644815 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1108_1108_0, total 61 rows starting from the beginning of the part 2024.10.22 01:47:29.717374 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(223815c382958e5b1d949e11249b7c7e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 223815c382958e5b1d949e11249b7c7e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:47:29.745027 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 223815c382958e5b1d949e11249b7c7e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1392, kind: Regular, unbound: 0 2024.10.22 01:47:29.749087 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:29.935959 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1109_1109_0, total 61 rows starting from the beginning of the part 2024.10.22 01:47:29.949110 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_131_131_0 to 202410_134_134_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:29.990380 [ 1214 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 7 parts from 202410_1_124_23 to 202410_134_134_0 2024.10.22 01:47:29.989715 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1110_1110_0, total 70 rows starting from the beginning of the part 2024.10.22 01:47:30.041608 [ 1214 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 2.60 MiB using storage policy from min volume index 0 2024.10.22 01:47:30.049558 [ 1214 ] {} DiskLocal: Reserved 2.60 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:30.064047 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} MergeTask::PrepareStage: Merging 7 parts: from 202410_1_124_23 to 202410_134_134_0 into Compact with storage Full 2024.10.22 01:47:30.073057 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1111_1111_0, total 56 rows starting from the beginning of the part 2024.10.22 01:47:30.075217 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_663_133/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:47:30.194213 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 5 old parts to remove. Parts: [202410_115_115_0, 202410_116_116_0, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0] 2024.10.22 01:47:30.203712 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from filesystem (serially): Parts: [202410_115_115_0, 202410_116_116_0, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0] 2024.10.22 01:47:30.336802 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from memory: Parts: [202410_115_115_0, 202410_116_116_0, 202410_117_117_0, 202410_118_118_0, 202410_119_119_0] 2024.10.22 01:47:30.400822 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_622_121} MergeTask::MergeProjectionsStage: Merge sorted 966146 rows, containing 4 columns (4 merged, 0 gathered) in 29.823159801 sec., 32395.82949783893 rows/sec., 485.74 KiB/sec. 2024.10.22 01:47:30.499973 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2c70d13c9d03af9599a66e23ad05e982) : [0, 107506]: Setting download as failed: File segment: [0, 107506], key: 2c70d13c9d03af9599a66e23ad05e982, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:30.502747 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2c70d13c9d03af9599a66e23ad05e982): No space left in cache to reserve 107507 bytes, will continue without cache download 2024.10.22 01:47:30.542837 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 9.95 MiB, will set to 1.70 GiB (RSS), difference: -2.37 MiB 2024.10.22 01:47:30.653807 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2c70d13c9d03af9599a66e23ad05e982, offset: 0, size: 0 2024.10.22 01:47:30.657189 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2c70d13c9d03af9599a66e23ad05e982 for removal 2024.10.22 01:47:30.660213 [ 1476 ] {} CacheMetadata: Key 2c70d13c9d03af9599a66e23ad05e982 is removed from metadata 2024.10.22 01:47:30.660801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ea6aa36e29127ecdc92d2e238c90e69d, offset: 0, size: 0 2024.10.22 01:47:30.663236 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ea6aa36e29127ecdc92d2e238c90e69d for removal 2024.10.22 01:47:30.665488 [ 1476 ] {} CacheMetadata: Key ea6aa36e29127ecdc92d2e238c90e69d is removed from metadata 2024.10.22 01:47:30.687748 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:47:30.695555 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:47:30.760046 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_622_121} MergedBlockOutputStream: filled checksums 202410_1_622_121 (state Temporary) 2024.10.22 01:47:30.919147 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} MergeTask::MergeProjectionsStage: Merge sorted 10040 rows, containing 12 columns (12 merged, 0 gathered) in 1.802210508 sec., 5570.936333703809 rows/sec., 886.61 KiB/sec. 2024.10.22 01:47:31.053329 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} MergedBlockOutputStream: filled checksums 202410_1_663_133 (state Temporary) 2024.10.22 01:47:31.554909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0c3c9bd3d1c846f06ad20d118d29aa22) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 0c3c9bd3d1c846f06ad20d118d29aa22, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:31.556428 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0c3c9bd3d1c846f06ad20d118d29aa22): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:47:31.622640 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} MergeTask::MergeProjectionsStage: Merge sorted 1014 rows, containing 13 columns (13 merged, 0 gathered) in 2.339282082 sec., 433.4663219123482 rows/sec., 77.15 KiB/sec. 2024.10.22 01:47:31.655536 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_663_133 to 202410_1_663_133 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:31.658743 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_663_133} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_658_132, 202410_663_663_0] -> 202410_1_663_133 2024.10.22 01:47:31.677556 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.59 MiB. 2024.10.22 01:47:31.951080 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} MergedBlockOutputStream: filled checksums 202410_1096_1111_3 (state Temporary) 2024.10.22 01:47:32.024891 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 962181 2024.10.22 01:47:32.269541 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1111_3 to 202410_1096_1111_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:32.275096 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1111_3} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1106_2, 202410_1111_1111_0] -> 202410_1096_1111_3 2024.10.22 01:47:32.319326 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:47:32.342332 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2e62cdf477c2c30b3d21cc93dbafb004) : [0, 192653]: Setting download as failed: File segment: [0, 192653], key: 2e62cdf477c2c30b3d21cc93dbafb004, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:32.347709 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2e62cdf477c2c30b3d21cc93dbafb004): No space left in cache to reserve 192654 bytes, will continue without cache download 2024.10.22 01:47:32.468926 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2e62cdf477c2c30b3d21cc93dbafb004, offset: 0, size: 0 2024.10.22 01:47:32.471697 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2e62cdf477c2c30b3d21cc93dbafb004 for removal 2024.10.22 01:47:32.474893 [ 1476 ] {} CacheMetadata: Key 2e62cdf477c2c30b3d21cc93dbafb004 is removed from metadata 2024.10.22 01:47:32.476774 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0c3c9bd3d1c846f06ad20d118d29aa22, offset: 0, size: 0 2024.10.22 01:47:32.478030 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0c3c9bd3d1c846f06ad20d118d29aa22 for removal 2024.10.22 01:47:32.480677 [ 1476 ] {} CacheMetadata: Key 0c3c9bd3d1c846f06ad20d118d29aa22 is removed from metadata 2024.10.22 01:47:32.505827 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 8.64 MiB, will set to 1.69 GiB (RSS), difference: 15.81 MiB 2024.10.22 01:47:32.962688 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:32.965784 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:33.043092 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 544 2024.10.22 01:47:33.027637 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5236 2024.10.22 01:47:33.204124 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(84bee8c965210a8eb12e5b82e2160d94) : [0, 188]: Setting download as failed: File segment: [0, 188], key: 84bee8c965210a8eb12e5b82e2160d94, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:33.207127 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(84bee8c965210a8eb12e5b82e2160d94): No space left in cache to reserve 189 bytes, will continue without cache download 2024.10.22 01:47:33.240623 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:33.244503 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:33.408780 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 71676 2024.10.22 01:47:33.411249 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_623_623_0 (state Temporary) 2024.10.22 01:47:33.477675 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_623_623_0 to 202410_630_630_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:33.505154 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_526_526_0 (state Temporary) 2024.10.22 01:47:33.607656 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:33.637316 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 962181 2024.10.22 01:47:33.674823 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_622_121} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_622_121 to 202410_1_622_121 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:33.677979 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_622_121} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_616_120, 202410_617_622_1] -> 202410_1_622_121 2024.10.22 01:47:33.705536 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:47:33.749578 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:33.776067 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_526_526_0 to 202410_536_536_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:33.824073 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1099_1099_0 (state Temporary) 2024.10.22 01:47:33.945200 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 544 2024.10.22 01:47:34.066112 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1099_1099_0 to 202410_1112_1112_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:34.136726 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71676 2024.10.22 01:47:34.510532 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 8.07 MiB, will set to 1.69 GiB (RSS), difference: 2.88 MiB 2024.10.22 01:47:34.684494 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d3621dc504c256d6f159fbe1dd66e28e) : [0, 330041]: Setting download as failed: File segment: [0, 330041], key: d3621dc504c256d6f159fbe1dd66e28e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:34.686990 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d3621dc504c256d6f159fbe1dd66e28e): No space left in cache to reserve 330042 bytes, will continue without cache download 2024.10.22 01:47:34.841103 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d3621dc504c256d6f159fbe1dd66e28e, offset: 0, size: 0 2024.10.22 01:47:34.845573 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d3621dc504c256d6f159fbe1dd66e28e for removal 2024.10.22 01:47:34.849095 [ 1476 ] {} CacheMetadata: Key d3621dc504c256d6f159fbe1dd66e28e is removed from metadata 2024.10.22 01:47:34.851456 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 84bee8c965210a8eb12e5b82e2160d94, offset: 0, size: 0 2024.10.22 01:47:34.853327 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 84bee8c965210a8eb12e5b82e2160d94 for removal 2024.10.22 01:47:34.856146 [ 1476 ] {} CacheMetadata: Key 84bee8c965210a8eb12e5b82e2160d94 is removed from metadata 2024.10.22 01:47:35.657357 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 78 entries to flush up to offset 73352 2024.10.22 01:47:35.752529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4ce3085fe974a1244fee96279f529093) : [0, 189]: Setting download as failed: File segment: [0, 189], key: 4ce3085fe974a1244fee96279f529093, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:35.754051 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4ce3085fe974a1244fee96279f529093): No space left in cache to reserve 190 bytes, will continue without cache download 2024.10.22 01:47:35.795392 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:35.798659 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:35.941549 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_639_639_0 (state Temporary) 2024.10.22 01:47:35.980199 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_639_639_0 to 202410_645_645_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:36.030937 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73352 2024.10.22 01:47:36.412845 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 8.00 MiB, will set to 1.69 GiB (RSS), difference: -2.18 MiB 2024.10.22 01:47:36.554488 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 24 entries to flush up to offset 9958 2024.10.22 01:47:36.755048 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:36.757416 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:36.786622 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_651_651_0 (state Temporary) 2024.10.22 01:47:36.925938 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a01f894e58fe4b45daf096941023a1d0) : [0, 345723]: Setting download as failed: File segment: [0, 345723], key: a01f894e58fe4b45daf096941023a1d0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:36.928284 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a01f894e58fe4b45daf096941023a1d0): No space left in cache to reserve 345724 bytes, will continue without cache download 2024.10.22 01:47:37.066106 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a01f894e58fe4b45daf096941023a1d0, offset: 0, size: 0 2024.10.22 01:47:37.067898 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a01f894e58fe4b45daf096941023a1d0 for removal 2024.10.22 01:47:37.071255 [ 1476 ] {} CacheMetadata: Key a01f894e58fe4b45daf096941023a1d0 is removed from metadata 2024.10.22 01:47:37.073652 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4ce3085fe974a1244fee96279f529093, offset: 0, size: 0 2024.10.22 01:47:37.075400 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4ce3085fe974a1244fee96279f529093 for removal 2024.10.22 01:47:37.077380 [ 1476 ] {} CacheMetadata: Key 4ce3085fe974a1244fee96279f529093 is removed from metadata 2024.10.22 01:47:37.155524 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_651_651_0 to 202410_664_664_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:37.257136 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9958 2024.10.22 01:47:37.957726 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9608de464cc586b7c5734cb4d245befc) : [0, 186]: Setting download as failed: File segment: [0, 186], key: 9608de464cc586b7c5734cb4d245befc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:37.960352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9608de464cc586b7c5734cb4d245befc): No space left in cache to reserve 187 bytes, will continue without cache download 2024.10.22 01:47:38.072753 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_483_99, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0, 202410_487_487_0, 202410_488_488_0] 2024.10.22 01:47:38.075645 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_483_99, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0, 202410_487_487_0, 202410_488_488_0] 2024.10.22 01:47:38.114725 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_483_99, 202410_484_484_0, 202410_485_485_0, 202410_486_486_0, 202410_487_487_0, 202410_488_488_0] 2024.10.22 01:47:38.153818 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 71728 2024.10.22 01:47:38.387175 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:38.394402 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:38.548697 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1100_1100_0 (state Temporary) 2024.10.22 01:47:38.566431 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 7.22 MiB, will set to 1.69 GiB (RSS), difference: -3.40 MiB 2024.10.22 01:47:38.692268 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1100_1100_0 to 202410_1113_1113_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:38.802913 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(70c6c1361cad57deb91c51b1f940d9bf) : [0, 141842]: Setting download as failed: File segment: [0, 141842], key: 70c6c1361cad57deb91c51b1f940d9bf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:38.805393 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(70c6c1361cad57deb91c51b1f940d9bf): No space left in cache to reserve 141843 bytes, will continue without cache download 2024.10.22 01:47:38.833703 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 70c6c1361cad57deb91c51b1f940d9bf, offset: 0, size: 0 2024.10.22 01:47:38.839442 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 70c6c1361cad57deb91c51b1f940d9bf for removal 2024.10.22 01:47:38.843315 [ 1476 ] {} CacheMetadata: Key 70c6c1361cad57deb91c51b1f940d9bf is removed from metadata 2024.10.22 01:47:38.846221 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71728 2024.10.22 01:47:38.848321 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9608de464cc586b7c5734cb4d245befc, offset: 0, size: 0 2024.10.22 01:47:38.851004 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9608de464cc586b7c5734cb4d245befc for removal 2024.10.22 01:47:38.854471 [ 1476 ] {} CacheMetadata: Key 9608de464cc586b7c5734cb4d245befc is removed from metadata 2024.10.22 01:47:39.621559 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(c3d1d86c9d7b4f29e0804fe4964abda1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c3d1d86c9d7b4f29e0804fe4964abda1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:47:39.626926 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 879, file segment info: File segment: [0, 33554431], key: c3d1d86c9d7b4f29e0804fe4964abda1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:47:39.631410 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:39.708958 [ 1742 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/enu/nbpiffqmmvqdtejjopxkgmeuncnrt, size 2552763 2024.10.22 01:47:39.848303 [ 1150 ] {} DatabaseCatalog: Execution took 287 ms. 2024.10.22 01:47:40.171806 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:47:40.176509 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:47:40.184916 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:47:40.206515 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:47:40.223078 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:47:40.236968 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:47:40.259486 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:47:40.272936 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 01:47:40.317691 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/upj/zqcpcauejapnsgxsumxjtnvwffmdf (LogSeriesLimiter: on interval from 2024-10-22 01:47:21 to 2024-10-22 01:47:40 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:47:40.620732 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d6c74036e85b73fe4560415d250c2b09) : [0, 240]: Setting download as failed: File segment: [0, 240], key: d6c74036e85b73fe4560415d250c2b09, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:40.622042 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d6c74036e85b73fe4560415d250c2b09): No space left in cache to reserve 241 bytes, will continue without cache download 2024.10.22 01:47:40.663206 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 5279 2024.10.22 01:47:40.689008 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:47:40.693776 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:47:40.725413 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 963721 2024.10.22 01:47:41.352767 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(935a4a58b038c7ad049fcc755c05e8de) : [0, 1662390]: Setting download as failed: File segment: [0, 1662390], key: 935a4a58b038c7ad049fcc755c05e8de, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:41.354588 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(935a4a58b038c7ad049fcc755c05e8de): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:47:41.448877 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 545 2024.10.22 01:47:41.506682 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 935a4a58b038c7ad049fcc755c05e8de, offset: 0, size: 0 2024.10.22 01:47:41.508485 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 935a4a58b038c7ad049fcc755c05e8de for removal 2024.10.22 01:47:41.512767 [ 1476 ] {} CacheMetadata: Key 935a4a58b038c7ad049fcc755c05e8de is removed from metadata 2024.10.22 01:47:41.517063 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d6c74036e85b73fe4560415d250c2b09, offset: 0, size: 0 2024.10.22 01:47:41.521398 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d6c74036e85b73fe4560415d250c2b09 for removal 2024.10.22 01:47:41.525572 [ 1476 ] {} CacheMetadata: Key d6c74036e85b73fe4560415d250c2b09 is removed from metadata 2024.10.22 01:47:41.539108 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_569_569_0, 202410_570_570_0, 202410_571_571_0, 202410_572_572_0, 202410_573_573_0, 202410_574_574_0] 2024.10.22 01:47:41.546104 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_569_569_0, 202410_570_570_0, 202410_571_571_0, 202410_572_572_0, 202410_573_573_0, 202410_574_574_0] 2024.10.22 01:47:41.552323 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:41.556813 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:47:41.680777 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_624_624_0 (state Temporary) 2024.10.22 01:47:41.741197 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:41.747154 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:47:41.772763 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 6.82 MiB, will set to 1.69 GiB (RSS), difference: -13.94 MiB 2024.10.22 01:47:41.775359 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_569_569_0, 202410_570_570_0, 202410_571_571_0, 202410_572_572_0, 202410_573_573_0, 202410_574_574_0] 2024.10.22 01:47:41.792368 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_624_624_0 to 202410_631_631_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:41.805237 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_527_527_0 (state Temporary) 2024.10.22 01:47:41.957354 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 963721 2024.10.22 01:47:41.972047 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_527_527_0 to 202410_537_537_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:42.044997 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 545 2024.10.22 01:47:42.353950 [ 1228 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:47:42.370634 [ 1228 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:47:42.616350 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(330eb6aa84769ca7b6a7ef4ff2868995) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 330eb6aa84769ca7b6a7ef4ff2868995, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:42.622615 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(330eb6aa84769ca7b6a7ef4ff2868995): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:47:42.866830 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 71790 2024.10.22 01:47:43.090731 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:43.105657 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:47:43.167819 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1101_1101_0 (state Temporary) 2024.10.22 01:47:43.209606 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1101_1101_0 to 202410_1114_1114_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:43.360897 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71790 2024.10.22 01:47:43.514479 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cac86f81b63e8936557b07bfd6d0bec7) : [0, 80894]: Setting download as failed: File segment: [0, 80894], key: cac86f81b63e8936557b07bfd6d0bec7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:43.516717 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cac86f81b63e8936557b07bfd6d0bec7): No space left in cache to reserve 80895 bytes, will continue without cache download 2024.10.22 01:47:43.556098 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cac86f81b63e8936557b07bfd6d0bec7, offset: 0, size: 0 2024.10.22 01:47:43.557337 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 48 entries to flush up to offset 73400 2024.10.22 01:47:43.557733 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cac86f81b63e8936557b07bfd6d0bec7 for removal 2024.10.22 01:47:43.559738 [ 1476 ] {} CacheMetadata: Key cac86f81b63e8936557b07bfd6d0bec7 is removed from metadata 2024.10.22 01:47:43.560754 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 330eb6aa84769ca7b6a7ef4ff2868995, offset: 0, size: 0 2024.10.22 01:47:43.561967 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 330eb6aa84769ca7b6a7ef4ff2868995 for removal 2024.10.22 01:47:43.564260 [ 1476 ] {} CacheMetadata: Key 330eb6aa84769ca7b6a7ef4ff2868995 is removed from metadata 2024.10.22 01:47:43.767131 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:43.770403 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:47:43.807597 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_640_640_0 (state Temporary) 2024.10.22 01:47:43.969346 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_582_15, 202410_583_583_0, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0, 202410_587_587_0] 2024.10.22 01:47:43.970148 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_640_640_0 to 202410_646_646_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:43.971189 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_582_15, 202410_583_583_0, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0, 202410_587_587_0] 2024.10.22 01:47:43.996573 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_582_15, 202410_583_583_0, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0, 202410_587_587_0] 2024.10.22 01:47:44.017452 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73400 2024.10.22 01:47:44.612942 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 6.58 MiB, will set to 1.69 GiB (RSS), difference: 11.39 MiB 2024.10.22 01:47:44.759522 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 9964 2024.10.22 01:47:44.827957 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(151a71a670a65f1c45f07b18f532a1b8) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 151a71a670a65f1c45f07b18f532a1b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:44.833200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(151a71a670a65f1c45f07b18f532a1b8): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:47:45.060787 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:45.063318 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:47:45.095766 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_652_652_0 (state Temporary) 2024.10.22 01:47:45.169252 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_652_652_0 to 202410_665_665_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:45.222094 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9964 2024.10.22 01:47:45.671421 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fec03a50a2b860c601d555e4278aba05) : [0, 69218]: Setting download as failed: File segment: [0, 69218], key: fec03a50a2b860c601d555e4278aba05, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:45.673238 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fec03a50a2b860c601d555e4278aba05): No space left in cache to reserve 69219 bytes, will continue without cache download 2024.10.22 01:47:45.694986 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fec03a50a2b860c601d555e4278aba05, offset: 0, size: 0 2024.10.22 01:47:45.698930 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fec03a50a2b860c601d555e4278aba05 for removal 2024.10.22 01:47:45.707083 [ 1476 ] {} CacheMetadata: Key fec03a50a2b860c601d555e4278aba05 is removed from metadata 2024.10.22 01:47:45.708378 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 151a71a670a65f1c45f07b18f532a1b8, offset: 0, size: 0 2024.10.22 01:47:45.712258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 151a71a670a65f1c45f07b18f532a1b8 for removal 2024.10.22 01:47:45.819412 [ 1476 ] {} CacheMetadata: Key 151a71a670a65f1c45f07b18f532a1b8 is removed from metadata 2024.10.22 01:47:46.207719 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 203 ms. 2024.10.22 01:47:46.302804 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 3.85 MiB, will set to 1.69 GiB (RSS), difference: 1.03 MiB 2024.10.22 01:47:46.820826 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(391144318e48b6a1834fa30bd7452cf6) : [0, 211]: Setting download as failed: File segment: [0, 211], key: 391144318e48b6a1834fa30bd7452cf6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:46.831398 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(391144318e48b6a1834fa30bd7452cf6): No space left in cache to reserve 212 bytes, will continue without cache download 2024.10.22 01:47:47.072194 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:47.379836 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 71835 2024.10.22 01:47:47.825536 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:47.829314 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:47:47.855959 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d76382f47141f8009ebb2b56b3d1cb97) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d76382f47141f8009ebb2b56b3d1cb97, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:47:47.861000 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 104742, file segment info: File segment: [0, 33554431], key: d76382f47141f8009ebb2b56b3d1cb97, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:47:47.862476 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:47.876641 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1102_1102_0 (state Temporary) 2024.10.22 01:47:47.929715 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_124_23, total 4912 rows starting from the beginning of the part 2024.10.22 01:47:48.030666 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1102_1102_0 to 202410_1115_1115_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:48.374792 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71835 2024.10.22 01:47:48.651229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e40206075201e8f58c85779e5faed64c) : [0, 128747]: Setting download as failed: File segment: [0, 128747], key: e40206075201e8f58c85779e5faed64c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:48.653368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e40206075201e8f58c85779e5faed64c): No space left in cache to reserve 128748 bytes, will continue without cache download 2024.10.22 01:47:48.669141 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e40206075201e8f58c85779e5faed64c, offset: 0, size: 0 2024.10.22 01:47:48.671074 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e40206075201e8f58c85779e5faed64c for removal 2024.10.22 01:47:48.673954 [ 1476 ] {} CacheMetadata: Key e40206075201e8f58c85779e5faed64c is removed from metadata 2024.10.22 01:47:48.675030 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 391144318e48b6a1834fa30bd7452cf6, offset: 0, size: 0 2024.10.22 01:47:48.676793 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 391144318e48b6a1834fa30bd7452cf6 for removal 2024.10.22 01:47:48.680179 [ 1476 ] {} CacheMetadata: Key 391144318e48b6a1834fa30bd7452cf6 is removed from metadata 2024.10.22 01:47:48.997966 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 844 entries to flush up to offset 964565 2024.10.22 01:47:49.560361 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:49.563220 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:47:49.595630 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1002_492, 202410_1003_1008_1] 2024.10.22 01:47:49.597065 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1002_492, 202410_1003_1008_1] 2024.10.22 01:47:49.630404 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1002_492, 202410_1003_1008_1] 2024.10.22 01:47:49.802710 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_625_625_0 (state Temporary) 2024.10.22 01:47:50.003569 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(0a4ec5e55f068d0e7234e68078b70bcf) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0a4ec5e55f068d0e7234e68078b70bcf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:47:50.006306 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 692, file segment info: File segment: [0, 33554431], key: 0a4ec5e55f068d0e7234e68078b70bcf, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:47:50.009343 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:50.070971 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_625_625_0 to 202410_632_632_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:50.116466 [ 1151 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_622_121 to 202410_623_628_1 2024.10.22 01:47:50.125719 [ 1151 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:50.135016 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:50.149072 [ 1200 ] {} DatabaseCatalog: Execution took 299 ms. 2024.10.22 01:47:50.159739 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_628_122} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_622_121 to 202410_623_628_1 into Wide with storage Full 2024.10.22 01:47:50.193477 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 964565 2024.10.22 01:47:50.264346 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_628_122} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:50.268463 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_628_122} MergeTreeSequentialSource: Reading 119 marks from part 202410_1_622_121, total 966146 rows starting from the beginning of the part 2024.10.22 01:47:50.405628 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_628_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_623_628_1, total 6930 rows starting from the beginning of the part 2024.10.22 01:47:50.505668 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 3.11 MiB, will set to 1.69 GiB (RSS), difference: 1.53 MiB 2024.10.22 01:47:50.689706 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:47:50.700488 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:47:50.925634 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_125_129_1, total 197 rows starting from the beginning of the part 2024.10.22 01:47:51.126833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a2d01cbd0e4827018a4b7ef69d422012) : [0, 213]: Setting download as failed: File segment: [0, 213], key: a2d01cbd0e4827018a4b7ef69d422012, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:51.130006 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a2d01cbd0e4827018a4b7ef69d422012): No space left in cache to reserve 214 bytes, will continue without cache download 2024.10.22 01:47:51.531197 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 78 entries to flush up to offset 73478 2024.10.22 01:47:51.767847 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:51.770519 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:51.783450 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_628_122/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:47:51.846969 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_641_641_0 (state Temporary) 2024.10.22 01:47:51.892508 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_641_641_0 to 202410_647_647_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:51.948502 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73478 2024.10.22 01:47:52.062933 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(fd10ec934336ab619e2b4ed765b54ccf) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fd10ec934336ab619e2b4ed765b54ccf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:47:52.065164 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 807764, file segment info: File segment: [0, 33554431], key: fd10ec934336ab619e2b4ed765b54ccf, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:47:52.067145 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:52.321132 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_562_111, 202410_563_568_1] 2024.10.22 01:47:52.323484 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_562_111, 202410_563_568_1] 2024.10.22 01:47:52.344207 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_562_111, 202410_563_568_1] 2024.10.22 01:47:52.380236 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 71892 2024.10.22 01:47:52.535583 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:52.539433 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:52.577920 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1103_1103_0 (state Temporary) 2024.10.22 01:47:52.615777 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1103_1103_0 to 202410_1116_1116_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:52.652926 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1111_3 to 202410_1116_1116_0 2024.10.22 01:47:52.660231 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:52.667488 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:52.700798 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1111_3 to 202410_1116_1116_0 into Compact with storage Full 2024.10.22 01:47:52.707158 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_130_130_0, total 36 rows starting from the beginning of the part 2024.10.22 01:47:52.725142 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 9969 2024.10.22 01:47:52.742160 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:52.759497 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1111_3, total 1014 rows starting from the beginning of the part 2024.10.22 01:47:52.766627 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71892 2024.10.22 01:47:52.782631 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(90a2f068124aca327b64f2714d8254c4) : [0, 192706]: Setting download as failed: File segment: [0, 192706], key: 90a2f068124aca327b64f2714d8254c4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:52.785068 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(90a2f068124aca327b64f2714d8254c4): No space left in cache to reserve 192707 bytes, will continue without cache download 2024.10.22 01:47:52.797552 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1112_1112_0, total 86 rows starting from the beginning of the part 2024.10.22 01:47:52.848267 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1113_1113_0, total 52 rows starting from the beginning of the part 2024.10.22 01:47:52.936279 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:52.941180 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:52.950769 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 90a2f068124aca327b64f2714d8254c4, offset: 0, size: 0 2024.10.22 01:47:52.952646 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 90a2f068124aca327b64f2714d8254c4 for removal 2024.10.22 01:47:52.958528 [ 1476 ] {} CacheMetadata: Key 90a2f068124aca327b64f2714d8254c4 is removed from metadata 2024.10.22 01:47:52.959557 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a2d01cbd0e4827018a4b7ef69d422012, offset: 0, size: 0 2024.10.22 01:47:52.961795 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a2d01cbd0e4827018a4b7ef69d422012 for removal 2024.10.22 01:47:52.970789 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1114_1114_0, total 62 rows starting from the beginning of the part 2024.10.22 01:47:52.975791 [ 1476 ] {} CacheMetadata: Key a2d01cbd0e4827018a4b7ef69d422012 is removed from metadata 2024.10.22 01:47:52.988562 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1115_1115_0, total 45 rows starting from the beginning of the part 2024.10.22 01:47:53.018847 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1116_1116_0, total 57 rows starting from the beginning of the part 2024.10.22 01:47:53.161123 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_653_653_0 (state Temporary) 2024.10.22 01:47:53.306897 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_653_653_0 to 202410_666_666_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:53.368322 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9969 2024.10.22 01:47:53.585293 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_601_120, 202410_602_602_0, 202410_603_603_0, 202410_604_604_0, 202410_605_605_0, 202410_606_606_0] 2024.10.22 01:47:53.587735 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_601_120, 202410_602_602_0, 202410_603_603_0, 202410_604_604_0, 202410_605_605_0, 202410_606_606_0] 2024.10.22 01:47:53.631204 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_601_120, 202410_602_602_0, 202410_603_603_0, 202410_604_604_0, 202410_605_605_0, 202410_606_606_0] 2024.10.22 01:47:53.875557 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d29abadcfadce91f6d5e55863794ff67) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d29abadcfadce91f6d5e55863794ff67, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:47:53.877621 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 934, file segment info: File segment: [0, 33554431], key: d29abadcfadce91f6d5e55863794ff67, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 01:47:53.881071 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:53.938900 [ 1684 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ynt/vbleqtivtilovurhqcteasusnskos, size 807764 2024.10.22 01:47:54.326703 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:47:54.352150 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_131_131_0, total 40 rows starting from the beginning of the part 2024.10.22 01:47:54.454592 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:47:54.455259 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} MergeTask::MergeProjectionsStage: Merge sorted 1316 rows, containing 13 columns (13 merged, 0 gathered) in 1.765313532 sec., 745.4766397836688 rows/sec., 132.24 KiB/sec. 2024.10.22 01:47:54.483964 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:47:54.498336 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} MergedBlockOutputStream: filled checksums 202410_1096_1116_4 (state Temporary) 2024.10.22 01:47:54.553929 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:47:54.573430 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:47:54.581520 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 9.43 MiB, will set to 1.70 GiB (RSS), difference: -9.39 MiB 2024.10.22 01:47:54.628489 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:47:54.664347 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1116_4 to 202410_1096_1116_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:54.680102 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1116_4} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1111_3, 202410_1116_1116_0] -> 202410_1096_1116_4 2024.10.22 01:47:54.739953 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:47:54.742476 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:47:54.771491 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:47:54.903932 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 01:47:54.973356 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3e7c2c127620fcd27171df5e7b0890fb) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 3e7c2c127620fcd27171df5e7b0890fb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:54.974614 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3e7c2c127620fcd27171df5e7b0890fb): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:47:55.093793 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zjo/rukdlgjhenxlgngklgjmaoxzcgdwj (LogSeriesLimiter: on interval from 2024-10-22 01:47:40 to 2024-10-22 01:47:55 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:47:55.453384 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1008_493, 202410_1009_1009_0] 2024.10.22 01:47:55.457639 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1008_493, 202410_1009_1009_0] 2024.10.22 01:47:55.474420 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1008_493, 202410_1009_1009_0] 2024.10.22 01:47:55.869357 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(01739eca089b35f941ba773d6a3813ba) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 01739eca089b35f941ba773d6a3813ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:47:55.871464 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 01739eca089b35f941ba773d6a3813ba, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:47:55.873122 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:47:56.322285 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_132_132_0, total 40 rows starting from the beginning of the part 2024.10.22 01:47:56.615011 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(32d73bffabc54fe83c572e7762daf236) : [0, 81636]: Setting download as failed: File segment: [0, 81636], key: 32d73bffabc54fe83c572e7762daf236, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:56.629836 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(32d73bffabc54fe83c572e7762daf236): No space left in cache to reserve 81637 bytes, will continue without cache download 2024.10.22 01:47:56.675751 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 32d73bffabc54fe83c572e7762daf236, offset: 0, size: 0 2024.10.22 01:47:56.687763 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 32d73bffabc54fe83c572e7762daf236 for removal 2024.10.22 01:47:56.700236 [ 1476 ] {} CacheMetadata: Key 32d73bffabc54fe83c572e7762daf236 is removed from metadata 2024.10.22 01:47:56.702724 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3e7c2c127620fcd27171df5e7b0890fb, offset: 0, size: 0 2024.10.22 01:47:56.706976 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3e7c2c127620fcd27171df5e7b0890fb for removal 2024.10.22 01:47:56.713167 [ 1476 ] {} CacheMetadata: Key 3e7c2c127620fcd27171df5e7b0890fb is removed from metadata 2024.10.22 01:47:56.777010 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 71965 2024.10.22 01:47:56.937210 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:56.941048 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:57.042553 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:57.048561 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 546 2024.10.22 01:47:57.048539 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:57.199370 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 852 entries to flush up to offset 965417 2024.10.22 01:47:57.221799 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1104_1104_0 (state Temporary) 2024.10.22 01:47:57.357729 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:57.361549 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:57.374920 [ 1204 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:47:57.498585 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1104_1104_0 to 202410_1117_1117_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:57.508481 [ 1204 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:47:57.611465 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_133_133_0, total 37 rows starting from the beginning of the part 2024.10.22 01:47:57.638319 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_528_528_0 (state Temporary) 2024.10.22 01:47:57.701573 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 71965 2024.10.22 01:47:57.755121 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_528_528_0 to 202410_538_538_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:57.792201 [ 1154 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_533_109 to 202410_538_538_0 2024.10.22 01:47:57.816619 [ 1154 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:57.823956 [ 1154 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:57.856096 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 546 2024.10.22 01:47:57.883931 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_533_109 to 202410_538_538_0 into Compact with storage Full 2024.10.22 01:47:57.952133 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:57.966723 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:57.999085 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1010_1010_0, 202410_1011_1011_0, 202410_1012_1012_0, 202410_1013_1013_0, 202410_1014_1014_0, 202410_1015_1015_0] 2024.10.22 01:47:58.011405 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fc7d05532d18c0eb6524b5e029c72829) : [0, 155]: Setting download as failed: File segment: [0, 155], key: fc7d05532d18c0eb6524b5e029c72829, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:58.013013 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1010_1010_0, 202410_1011_1011_0, 202410_1012_1012_0, 202410_1013_1013_0, 202410_1014_1014_0, 202410_1015_1015_0] 2024.10.22 01:47:58.014116 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fc7d05532d18c0eb6524b5e029c72829): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:47:58.049577 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1010_1010_0, 202410_1011_1011_0, 202410_1012_1012_0, 202410_1013_1013_0, 202410_1014_1014_0, 202410_1015_1015_0] 2024.10.22 01:47:58.123782 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:47:58.209629 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_626_626_0 (state Temporary) 2024.10.22 01:47:58.247802 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_533_109, total 577 rows starting from the beginning of the part 2024.10.22 01:47:58.272976 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_534_534_0, total 1 rows starting from the beginning of the part 2024.10.22 01:47:58.329336 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_626_626_0 to 202410_633_633_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:47:58.330943 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_535_535_0, total 1 rows starting from the beginning of the part 2024.10.22 01:47:58.362403 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_536_536_0, total 1 rows starting from the beginning of the part 2024.10.22 01:47:58.369569 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 965417 2024.10.22 01:47:58.447007 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_537_537_0, total 1 rows starting from the beginning of the part 2024.10.22 01:47:58.493778 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_538_538_0, total 1 rows starting from the beginning of the part 2024.10.22 01:47:58.656260 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 11.83 MiB, will set to 1.70 GiB (RSS), difference: -6.63 MiB 2024.10.22 01:47:59.215478 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(41abac4cd8f4edfeefc2175cf5003679) : [0, 106455]: Setting download as failed: File segment: [0, 106455], key: 41abac4cd8f4edfeefc2175cf5003679, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:47:59.220648 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(41abac4cd8f4edfeefc2175cf5003679): No space left in cache to reserve 106456 bytes, will continue without cache download 2024.10.22 01:47:59.361444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 41abac4cd8f4edfeefc2175cf5003679, offset: 0, size: 0 2024.10.22 01:47:59.365739 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 41abac4cd8f4edfeefc2175cf5003679 for removal 2024.10.22 01:47:59.368337 [ 1476 ] {} CacheMetadata: Key 41abac4cd8f4edfeefc2175cf5003679 is removed from metadata 2024.10.22 01:47:59.369179 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fc7d05532d18c0eb6524b5e029c72829, offset: 0, size: 0 2024.10.22 01:47:59.373639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fc7d05532d18c0eb6524b5e029c72829 for removal 2024.10.22 01:47:59.378780 [ 1476 ] {} CacheMetadata: Key fc7d05532d18c0eb6524b5e029c72829 is removed from metadata 2024.10.22 01:47:59.461038 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 73569 2024.10.22 01:47:59.514974 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_134_134_0, total 36 rows starting from the beginning of the part 2024.10.22 01:47:59.803929 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:47:59.806749 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:47:59.916752 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_538_110/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:47:59.959889 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_642_642_0 (state Temporary) 2024.10.22 01:48:00.006566 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_642_642_0 to 202410_648_648_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:00.163470 [ 1272 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_643_28 to 202410_648_648_0 2024.10.22 01:48:00.166810 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73569 2024.10.22 01:48:00.183177 [ 1272 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:00.197710 [ 1272 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:00.271643 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_132_132_0 (state Temporary) 2024.10.22 01:48:00.400956 [ 1195 ] {} DatabaseCatalog: Execution took 251 ms. 2024.10.22 01:48:00.432654 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_643_28 to 202410_648_648_0 into Compact with storage Full 2024.10.22 01:48:00.467199 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5706cf60b7255533d88791fb8f7cc667) : [0, 213]: Setting download as failed: File segment: [0, 213], key: 5706cf60b7255533d88791fb8f7cc667, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:00.469278 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5706cf60b7255533d88791fb8f7cc667): No space left in cache to reserve 214 bytes, will continue without cache download 2024.10.22 01:48:00.470951 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:48:00.501797 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_643_28, total 14076 rows starting from the beginning of the part 2024.10.22 01:48:00.693468 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:48:00.752134 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_644_644_0, total 73 rows starting from the beginning of the part 2024.10.22 01:48:00.753187 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:48:00.832635 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_645_645_0, total 78 rows starting from the beginning of the part 2024.10.22 01:48:00.870070 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 9981 2024.10.22 01:48:00.872091 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} MergeTask::MergeProjectionsStage: Merge sorted 582 rows, containing 32 columns (32 merged, 0 gathered) in 2.995240581 sec., 194.30826481580712 rows/sec., 17.79 KiB/sec. 2024.10.22 01:48:00.875840 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_646_646_0, total 48 rows starting from the beginning of the part 2024.10.22 01:48:00.923999 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} MergedBlockOutputStream: filled checksums 202410_1_538_110 (state Temporary) 2024.10.22 01:48:00.948151 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_647_647_0, total 78 rows starting from the beginning of the part 2024.10.22 01:48:01.016018 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_648_648_0, total 91 rows starting from the beginning of the part 2024.10.22 01:48:01.180023 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:01.184195 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:01.242902 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_654_654_0 (state Temporary) 2024.10.22 01:48:01.339744 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_538_110 to 202410_1_538_110 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:01.344523 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_538_110} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_533_109, 202410_538_538_0] -> 202410_1_538_110 2024.10.22 01:48:01.366564 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(59edbb4cbf3706e83d3eb79e77ef830d) : [0, 501682]: Setting download as failed: File segment: [0, 501682], key: 59edbb4cbf3706e83d3eb79e77ef830d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:01.369227 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(59edbb4cbf3706e83d3eb79e77ef830d): No space left in cache to reserve 501683 bytes, will continue without cache download 2024.10.22 01:48:01.382205 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:48:01.386673 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_654_654_0 to 202410_667_667_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:01.452461 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 9981 2024.10.22 01:48:01.475131 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 9.45 MiB, will set to 1.71 GiB (RSS), difference: -7.50 MiB 2024.10.22 01:48:01.556215 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 59edbb4cbf3706e83d3eb79e77ef830d, offset: 0, size: 0 2024.10.22 01:48:01.558671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 59edbb4cbf3706e83d3eb79e77ef830d for removal 2024.10.22 01:48:01.562224 [ 1476 ] {} CacheMetadata: Key 59edbb4cbf3706e83d3eb79e77ef830d is removed from metadata 2024.10.22 01:48:01.565712 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5706cf60b7255533d88791fb8f7cc667, offset: 0, size: 0 2024.10.22 01:48:01.567014 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5706cf60b7255533d88791fb8f7cc667 for removal 2024.10.22 01:48:01.586548 [ 1476 ] {} CacheMetadata: Key 5706cf60b7255533d88791fb8f7cc667 is removed from metadata 2024.10.22 01:48:01.736073 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 72058 2024.10.22 01:48:02.094706 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_132_132_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:48:02.325388 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:02.335397 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:02.545822 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1105_1105_0 (state Temporary) 2024.10.22 01:48:02.665618 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1105_1105_0 to 202410_1118_1118_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:02.703376 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5c73fcf981e31a0d0c632d611fc432ea) : [0, 233]: Setting download as failed: File segment: [0, 233], key: 5c73fcf981e31a0d0c632d611fc432ea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:02.705755 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5c73fcf981e31a0d0c632d611fc432ea): No space left in cache to reserve 234 bytes, will continue without cache download 2024.10.22 01:48:02.732396 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72058 2024.10.22 01:48:03.042314 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} MergeTask::MergeProjectionsStage: Merge sorted 14444 rows, containing 13 columns (13 merged, 0 gathered) in 2.830973143 sec., 5102.1324719080885 rows/sec., 2.21 MiB/sec. 2024.10.22 01:48:03.097320 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_628_122/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:48:03.351200 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} MergedBlockOutputStream: filled checksums 202410_507_648_29 (state Temporary) 2024.10.22 01:48:03.579410 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_648_29 to 202410_507_648_29 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:03.584193 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_648_29} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_643_28, 202410_648_648_0] -> 202410_507_648_29 2024.10.22 01:48:03.659528 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.01 MiB. 2024.10.22 01:48:03.697083 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 7.30 MiB, will set to 1.71 GiB (RSS), difference: 4.11 MiB 2024.10.22 01:48:04.203499 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cf20f2a2a60cec94ea2842f497ca3faf) : [0, 1159555]: Setting download as failed: File segment: [0, 1159555], key: cf20f2a2a60cec94ea2842f497ca3faf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:04.205088 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cf20f2a2a60cec94ea2842f497ca3faf): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:48:04.273192 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cf20f2a2a60cec94ea2842f497ca3faf, offset: 0, size: 0 2024.10.22 01:48:04.275695 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cf20f2a2a60cec94ea2842f497ca3faf for removal 2024.10.22 01:48:04.281640 [ 1476 ] {} CacheMetadata: Key cf20f2a2a60cec94ea2842f497ca3faf is removed from metadata 2024.10.22 01:48:04.286199 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5c73fcf981e31a0d0c632d611fc432ea, offset: 0, size: 0 2024.10.22 01:48:04.292167 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5c73fcf981e31a0d0c632d611fc432ea for removal 2024.10.22 01:48:04.306981 [ 1476 ] {} CacheMetadata: Key 5c73fcf981e31a0d0c632d611fc432ea is removed from metadata 2024.10.22 01:48:05.358434 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 547 2024.10.22 01:48:05.374007 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1384 entries to flush up to offset 966801 2024.10.22 01:48:05.527333 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 7.30 MiB, will set to 1.70 GiB (RSS), difference: -7.79 MiB 2024.10.22 01:48:05.664012 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:05.667069 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:05.846547 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_529_529_0 (state Temporary) 2024.10.22 01:48:06.014680 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_529_529_0 to 202410_539_539_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:06.018930 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:06.024060 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:06.129426 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 547 2024.10.22 01:48:06.237509 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(eeae0c75a535ecca7cb85129565f83c5) : [0, 151]: Setting download as failed: File segment: [0, 151], key: eeae0c75a535ecca7cb85129565f83c5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:06.239596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(eeae0c75a535ecca7cb85129565f83c5): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 01:48:06.374740 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_627_627_0 (state Temporary) 2024.10.22 01:48:06.568769 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_627_627_0 to 202410_634_634_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:06.604791 [ 1169 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_629_629_0 to 202410_634_634_0 2024.10.22 01:48:06.658366 [ 1169 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:06.662853 [ 1169 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:06.724377 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_629_629_0 to 202410_634_634_0 into Compact with storage Full 2024.10.22 01:48:06.750805 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 72100 2024.10.22 01:48:06.775265 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:48:06.786929 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_629_629_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:48:06.819786 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 966801 2024.10.22 01:48:06.925640 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_630_630_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:48:06.953028 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_631_631_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:48:07.110788 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_632_632_0, total 844 rows starting from the beginning of the part 2024.10.22 01:48:07.140320 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_633_633_0, total 852 rows starting from the beginning of the part 2024.10.22 01:48:07.167302 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_634_634_0, total 1384 rows starting from the beginning of the part 2024.10.22 01:48:07.193683 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:07.200031 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:07.410856 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1106_1106_0 (state Temporary) 2024.10.22 01:48:07.456368 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1106_1106_0 to 202410_1119_1119_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:07.503522 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72100 2024.10.22 01:48:07.677642 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 110 entries to flush up to offset 73679 2024.10.22 01:48:07.689424 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(083310cefd996040da99425a103e6fee) : [0, 76721]: Setting download as failed: File segment: [0, 76721], key: 083310cefd996040da99425a103e6fee, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:07.692671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(083310cefd996040da99425a103e6fee): No space left in cache to reserve 76722 bytes, will continue without cache download 2024.10.22 01:48:07.739969 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 083310cefd996040da99425a103e6fee, offset: 0, size: 0 2024.10.22 01:48:07.771932 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 083310cefd996040da99425a103e6fee for removal 2024.10.22 01:48:07.774493 [ 1476 ] {} CacheMetadata: Key 083310cefd996040da99425a103e6fee is removed from metadata 2024.10.22 01:48:07.776291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: eeae0c75a535ecca7cb85129565f83c5, offset: 0, size: 0 2024.10.22 01:48:07.778014 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key eeae0c75a535ecca7cb85129565f83c5 for removal 2024.10.22 01:48:07.782644 [ 1476 ] {} CacheMetadata: Key eeae0c75a535ecca7cb85129565f83c5 is removed from metadata 2024.10.22 01:48:08.032806 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:08.038609 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:08.209646 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_643_643_0 (state Temporary) 2024.10.22 01:48:08.265460 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_643_643_0 to 202410_649_649_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:08.309365 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73679 2024.10.22 01:48:08.466104 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} MergeTask::MergeProjectionsStage: Merge sorted 7315 rows, containing 4 columns (4 merged, 0 gathered) in 1.74588934 sec., 4189.841722729117 rows/sec., 65.47 KiB/sec. 2024.10.22 01:48:08.535095 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} MergedBlockOutputStream: filled checksums 202410_629_634_1 (state Temporary) 2024.10.22 01:48:08.787042 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_629_634_1 to 202410_629_634_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:08.793188 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_629_634_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_629_629_0, 202410_634_634_0] -> 202410_629_634_1 2024.10.22 01:48:08.866138 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:48:08.955488 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 10008 2024.10.22 01:48:09.056509 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(432d69fe66b2400213f40995bbf1f216) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 432d69fe66b2400213f40995bbf1f216, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:48:09.070958 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 825, file segment info: File segment: [0, 33554431], key: 432d69fe66b2400213f40995bbf1f216, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 01:48:09.073743 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:48:09.252277 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:09.257451 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:09.258523 [ 1705 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/upj/zqcpcauejapnsgxsumxjtnvwffmdf, size 2695475 2024.10.22 01:48:09.287881 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_655_655_0 (state Temporary) 2024.10.22 01:48:09.453715 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_655_655_0 to 202410_668_668_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:09.632316 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10008 2024.10.22 01:48:09.714916 [ 1208 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_663_133 to 202410_668_668_0 2024.10.22 01:48:09.731529 [ 1208 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:09.736124 [ 1208 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:09.742218 [ 1208 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 273 ms. 2024.10.22 01:48:09.760559 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_663_133 to 202410_668_668_0 into Compact with storage Full 2024.10.22 01:48:09.784069 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:48:09.800953 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_663_133, total 10040 rows starting from the beginning of the part 2024.10.22 01:48:09.917171 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_664_664_0, total 24 rows starting from the beginning of the part 2024.10.22 01:48:09.952361 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_665_665_0, total 6 rows starting from the beginning of the part 2024.10.22 01:48:09.963689 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column WindowName 2024.10.22 01:48:09.972753 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_666_666_0, total 5 rows starting from the beginning of the part 2024.10.22 01:48:09.984803 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column WindowName 2024.10.22 01:48:09.988701 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_667_667_0, total 12 rows starting from the beginning of the part 2024.10.22 01:48:10.007643 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column WindowName 2024.10.22 01:48:10.035831 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_668_668_0, total 27 rows starting from the beginning of the part 2024.10.22 01:48:10.036523 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column WindowName 2024.10.22 01:48:10.074365 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column WindowName 2024.10.22 01:48:10.088348 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column WindowName 2024.10.22 01:48:10.096752 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d2af2f60628de62daa3d733fed450e36) : [0, 157]: Setting download as failed: File segment: [0, 157], key: d2af2f60628de62daa3d733fed450e36, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:10.099749 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d2af2f60628de62daa3d733fed450e36): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:48:10.124520 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column WindowName 2024.10.22 01:48:10.152345 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column WindowName 2024.10.22 01:48:10.352218 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pyx/kwazpxhzldfyzhasatgjhiavmuoym (LogSeriesLimiter: on interval from 2024-10-22 01:47:55 to 2024-10-22 01:48:10 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:48:10.619897 [ 1213 ] {} DatabaseCatalog: Execution took 217 ms. 2024.10.22 01:48:10.699287 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:48:10.702704 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:48:11.258904 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_132_132_0 to 202410_135_135_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:11.284967 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2c4c62cda2845dd2275cc9a689f05deb) : [0, 196148]: Setting download as failed: File segment: [0, 196148], key: 2c4c62cda2845dd2275cc9a689f05deb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:11.287592 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2c4c62cda2845dd2275cc9a689f05deb): No space left in cache to reserve 196149 bytes, will continue without cache download 2024.10.22 01:48:11.331263 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2c4c62cda2845dd2275cc9a689f05deb, offset: 0, size: 0 2024.10.22 01:48:11.336844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2c4c62cda2845dd2275cc9a689f05deb for removal 2024.10.22 01:48:11.346403 [ 1476 ] {} CacheMetadata: Key 2c4c62cda2845dd2275cc9a689f05deb is removed from metadata 2024.10.22 01:48:11.349154 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d2af2f60628de62daa3d733fed450e36, offset: 0, size: 0 2024.10.22 01:48:11.351300 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d2af2f60628de62daa3d733fed450e36 for removal 2024.10.22 01:48:11.356592 [ 1476 ] {} CacheMetadata: Key d2af2f60628de62daa3d733fed450e36 is removed from metadata 2024.10.22 01:48:11.481136 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 9.91 MiB, will set to 1.71 GiB (RSS), difference: 6.12 MiB 2024.10.22 01:48:11.510560 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 72179 2024.10.22 01:48:11.844665 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:11.849326 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:12.045634 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1107_1107_0 (state Temporary) 2024.10.22 01:48:12.139966 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1107_1107_0 to 202410_1120_1120_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:12.203393 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} MergeTask::MergeProjectionsStage: Merge sorted 10114 rows, containing 12 columns (12 merged, 0 gathered) in 2.452662762 sec., 4123.681476597556 rows/sec., 656.34 KiB/sec. 2024.10.22 01:48:12.305478 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72179 2024.10.22 01:48:12.378828 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} MergedBlockOutputStream: filled checksums 202410_1_668_134 (state Temporary) 2024.10.22 01:48:12.513401 [ 1198 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:48:12.577735 [ 1198 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:48:12.710991 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f4c141c088b6897db9fc10c41795e912) : [0, 153]: Setting download as failed: File segment: [0, 153], key: f4c141c088b6897db9fc10c41795e912, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:12.717304 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f4c141c088b6897db9fc10c41795e912): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:48:12.874411 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_668_134 to 202410_1_668_134 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:12.876814 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_668_134} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_663_133, 202410_668_668_0] -> 202410_1_668_134 2024.10.22 01:48:12.919429 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.59 MiB. 2024.10.22 01:48:13.540503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cf11e70a6b231347bd3f37cd326d78d5) : [0, 5804]: Setting download as failed: File segment: [0, 5804], key: cf11e70a6b231347bd3f37cd326d78d5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:13.543910 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cf11e70a6b231347bd3f37cd326d78d5): No space left in cache to reserve 5805 bytes, will continue without cache download 2024.10.22 01:48:13.562294 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cf11e70a6b231347bd3f37cd326d78d5, offset: 0, size: 0 2024.10.22 01:48:13.565116 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cf11e70a6b231347bd3f37cd326d78d5 for removal 2024.10.22 01:48:13.569488 [ 1476 ] {} CacheMetadata: Key cf11e70a6b231347bd3f37cd326d78d5 is removed from metadata 2024.10.22 01:48:13.574999 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f4c141c088b6897db9fc10c41795e912, offset: 0, size: 0 2024.10.22 01:48:13.577081 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f4c141c088b6897db9fc10c41795e912 for removal 2024.10.22 01:48:13.580815 [ 1476 ] {} CacheMetadata: Key f4c141c088b6897db9fc10c41795e912 is removed from metadata 2024.10.22 01:48:13.633328 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 548 2024.10.22 01:48:13.817411 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:13.823048 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:13.829002 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 967956 2024.10.22 01:48:13.890594 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_530_530_0 (state Temporary) 2024.10.22 01:48:14.113636 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_530_530_0 to 202410_540_540_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:14.232387 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 548 2024.10.22 01:48:14.541093 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_134_24/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:48:14.607376 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 12.01 MiB, will set to 1.71 GiB (RSS), difference: 5.39 MiB 2024.10.22 01:48:14.752904 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3a5cdaab6eb5eeed32f77eb837329a73) : [0, 240]: Setting download as failed: File segment: [0, 240], key: 3a5cdaab6eb5eeed32f77eb837329a73, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:14.753775 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3a5cdaab6eb5eeed32f77eb837329a73): No space left in cache to reserve 241 bytes, will continue without cache download 2024.10.22 01:48:14.772041 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5279 2024.10.22 01:48:14.940052 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:15.035243 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:15.048956 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_628_122/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:48:15.288158 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_628_628_0 (state Temporary) 2024.10.22 01:48:15.608197 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_628_628_0 to 202410_635_635_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:15.740277 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 967956 2024.10.22 01:48:15.831095 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 113 entries to flush up to offset 73792 2024.10.22 01:48:15.967300 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:15.971158 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:15.997565 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9c3da46b6ecf7a361ac6ccd8aa525322) : [0, 1648286]: Setting download as failed: File segment: [0, 1648286], key: 9c3da46b6ecf7a361ac6ccd8aa525322, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:16.000355 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9c3da46b6ecf7a361ac6ccd8aa525322): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:48:16.134459 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_644_644_0 (state Temporary) 2024.10.22 01:48:16.281772 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9c3da46b6ecf7a361ac6ccd8aa525322, offset: 0, size: 0 2024.10.22 01:48:16.284252 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9c3da46b6ecf7a361ac6ccd8aa525322 for removal 2024.10.22 01:48:16.288992 [ 1476 ] {} CacheMetadata: Key 9c3da46b6ecf7a361ac6ccd8aa525322 is removed from metadata 2024.10.22 01:48:16.299143 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3a5cdaab6eb5eeed32f77eb837329a73, offset: 0, size: 0 2024.10.22 01:48:16.318762 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_644_644_0 to 202410_650_650_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:16.397254 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3a5cdaab6eb5eeed32f77eb837329a73 for removal 2024.10.22 01:48:16.409655 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 72230 2024.10.22 01:48:16.411158 [ 1476 ] {} CacheMetadata: Key 3a5cdaab6eb5eeed32f77eb837329a73 is removed from metadata 2024.10.22 01:48:16.458382 [ 1182 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 01:48:16.773054 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:16.776451 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:16.802679 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73792 2024.10.22 01:48:16.939693 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1108_1108_0 (state Temporary) 2024.10.22 01:48:17.140096 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1108_1108_0 to 202410_1121_1121_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:17.141531 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10021 2024.10.22 01:48:17.175386 [ 1232 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1116_4 to 202410_1121_1121_0 2024.10.22 01:48:17.223840 [ 1232 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:17.229091 [ 1232 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:17.265783 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1116_4 to 202410_1121_1121_0 into Compact with storage Full 2024.10.22 01:48:17.308895 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72230 2024.10.22 01:48:17.320645 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:48:17.352657 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1116_4, total 1316 rows starting from the beginning of the part 2024.10.22 01:48:17.552433 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1117_1117_0, total 73 rows starting from the beginning of the part 2024.10.22 01:48:17.560456 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:17.563850 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:17.605454 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_656_656_0 (state Temporary) 2024.10.22 01:48:17.709476 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1118_1118_0, total 93 rows starting from the beginning of the part 2024.10.22 01:48:17.764382 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_656_656_0 to 202410_669_669_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:17.765273 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1119_1119_0, total 42 rows starting from the beginning of the part 2024.10.22 01:48:17.814409 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10021 2024.10.22 01:48:17.935790 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1120_1120_0, total 79 rows starting from the beginning of the part 2024.10.22 01:48:17.971366 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2b7e0630bbb12164032c70a5539c8f95) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 2b7e0630bbb12164032c70a5539c8f95, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:17.973582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2b7e0630bbb12164032c70a5539c8f95): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 01:48:17.989086 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1121_1121_0, total 51 rows starting from the beginning of the part 2024.10.22 01:48:19.258760 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(61ae3c9ea4c1a681f0708fe490beb351) : [0, 130602]: Setting download as failed: File segment: [0, 130602], key: 61ae3c9ea4c1a681f0708fe490beb351, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:19.260911 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(61ae3c9ea4c1a681f0708fe490beb351): No space left in cache to reserve 130603 bytes, will continue without cache download 2024.10.22 01:48:19.330558 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 20.01 MiB, will set to 1.72 GiB (RSS), difference: 3.14 MiB 2024.10.22 01:48:19.407612 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} MergeTask::MergeProjectionsStage: Merge sorted 1654 rows, containing 13 columns (13 merged, 0 gathered) in 2.148108292 sec., 769.9798032342403 rows/sec., 136.11 KiB/sec. 2024.10.22 01:48:19.419020 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 61ae3c9ea4c1a681f0708fe490beb351, offset: 0, size: 0 2024.10.22 01:48:19.422616 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 61ae3c9ea4c1a681f0708fe490beb351 for removal 2024.10.22 01:48:19.429057 [ 1476 ] {} CacheMetadata: Key 61ae3c9ea4c1a681f0708fe490beb351 is removed from metadata 2024.10.22 01:48:19.430189 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2b7e0630bbb12164032c70a5539c8f95, offset: 0, size: 0 2024.10.22 01:48:19.433001 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2b7e0630bbb12164032c70a5539c8f95 for removal 2024.10.22 01:48:19.436800 [ 1476 ] {} CacheMetadata: Key 2b7e0630bbb12164032c70a5539c8f95 is removed from metadata 2024.10.22 01:48:19.466435 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} MergedBlockOutputStream: filled checksums 202410_1096_1121_5 (state Temporary) 2024.10.22 01:48:19.645212 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1121_5 to 202410_1096_1121_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:19.655861 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1121_5} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1116_4, 202410_1121_1121_0] -> 202410_1096_1121_5 2024.10.22 01:48:19.687376 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:48:20.271911 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(48e2b9565d473cc0cc4b01baaf124c6c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 48e2b9565d473cc0cc4b01baaf124c6c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:48:20.276067 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 48e2b9565d473cc0cc4b01baaf124c6c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 01:48:20.279474 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:48:20.702174 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:48:20.711247 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:48:20.953126 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_628_122} MergeTask::MergeProjectionsStage: Merge sorted 973076 rows, containing 4 columns (4 merged, 0 gathered) in 30.806768678 sec., 31586.435116608045 rows/sec., 473.54 KiB/sec. 2024.10.22 01:48:21.058604 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_628_122} MergedBlockOutputStream: filled checksums 202410_1_628_122 (state Temporary) 2024.10.22 01:48:21.084346 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(036e378010f00c3d633b8aab6448d837) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 036e378010f00c3d633b8aab6448d837, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:21.086171 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(036e378010f00c3d633b8aab6448d837): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 01:48:21.313744 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 72283 2024.10.22 01:48:21.544007 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:21.548810 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:21.686494 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1109_1109_0 (state Temporary) 2024.10.22 01:48:21.736356 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 549 2024.10.22 01:48:21.847391 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1109_1109_0 to 202410_1122_1122_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:21.938751 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72283 2024.10.22 01:48:22.018540 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_7_7_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:48:22.163972 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:22.168190 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:22.196025 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(591546f06565f4b2fc88852571843f9e) : [0, 106566]: Setting download as failed: File segment: [0, 106566], key: 591546f06565f4b2fc88852571843f9e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:22.201264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(591546f06565f4b2fc88852571843f9e): No space left in cache to reserve 106567 bytes, will continue without cache download 2024.10.22 01:48:22.262342 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_531_531_0 (state Temporary) 2024.10.22 01:48:22.343866 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 5321 2024.10.22 01:48:22.365410 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 591546f06565f4b2fc88852571843f9e, offset: 0, size: 0 2024.10.22 01:48:22.367863 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 591546f06565f4b2fc88852571843f9e for removal 2024.10.22 01:48:22.372550 [ 1476 ] {} CacheMetadata: Key 591546f06565f4b2fc88852571843f9e is removed from metadata 2024.10.22 01:48:22.374300 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 036e378010f00c3d633b8aab6448d837, offset: 0, size: 0 2024.10.22 01:48:22.375706 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 036e378010f00c3d633b8aab6448d837 for removal 2024.10.22 01:48:22.378618 [ 1476 ] {} CacheMetadata: Key 036e378010f00c3d633b8aab6448d837 is removed from metadata 2024.10.22 01:48:22.455194 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_531_531_0 to 202410_541_541_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:22.458060 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 16.18 MiB, will set to 1.72 GiB (RSS), difference: 3.65 MiB 2024.10.22 01:48:22.555486 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 549 2024.10.22 01:48:22.748080 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1157 entries to flush up to offset 969113 2024.10.22 01:48:23.164154 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_587_16, 202410_588_588_0, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0, 202410_592_592_0] 2024.10.22 01:48:23.166345 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_587_16, 202410_588_588_0, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0, 202410_592_592_0] 2024.10.22 01:48:23.203175 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_587_16, 202410_588_588_0, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0, 202410_592_592_0] 2024.10.22 01:48:23.250804 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(05a1879d4e940ff90f4c4a50ff3c28fc) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 05a1879d4e940ff90f4c4a50ff3c28fc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:23.253459 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(05a1879d4e940ff90f4c4a50ff3c28fc): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:48:23.684308 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1009_494, 202410_1010_1015_1] 2024.10.22 01:48:23.692104 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1009_494, 202410_1010_1015_1] 2024.10.22 01:48:23.763860 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1009_494, 202410_1010_1015_1] 2024.10.22 01:48:23.918123 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:23.923515 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:24.149203 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_629_629_0 (state Temporary) 2024.10.22 01:48:24.172146 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_628_122} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_628_122 to 202410_1_628_122 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:24.175898 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_628_122} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_622_121, 202410_623_628_1] -> 202410_1_628_122 2024.10.22 01:48:24.201298 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:48:24.207433 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_629_629_0 to 202410_636_636_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:24.285439 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(14669bee3d056a414951e7e00c648c31) : [0, 101933]: Setting download as failed: File segment: [0, 101933], key: 14669bee3d056a414951e7e00c648c31, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:24.287784 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(14669bee3d056a414951e7e00c648c31): No space left in cache to reserve 101934 bytes, will continue without cache download 2024.10.22 01:48:24.324998 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 73871 2024.10.22 01:48:24.348199 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 969113 2024.10.22 01:48:24.584397 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:24.590365 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:24.651100 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 14669bee3d056a414951e7e00c648c31, offset: 0, size: 0 2024.10.22 01:48:24.651305 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_645_645_0 (state Temporary) 2024.10.22 01:48:24.652577 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 14669bee3d056a414951e7e00c648c31 for removal 2024.10.22 01:48:24.655974 [ 1476 ] {} CacheMetadata: Key 14669bee3d056a414951e7e00c648c31 is removed from metadata 2024.10.22 01:48:24.656594 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 05a1879d4e940ff90f4c4a50ff3c28fc, offset: 0, size: 0 2024.10.22 01:48:24.658145 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 05a1879d4e940ff90f4c4a50ff3c28fc for removal 2024.10.22 01:48:24.660885 [ 1476 ] {} CacheMetadata: Key 05a1879d4e940ff90f4c4a50ff3c28fc is removed from metadata 2024.10.22 01:48:24.683722 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_645_645_0 to 202410_651_651_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:24.949134 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73871 2024.10.22 01:48:24.980938 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1016_1016_0, 202410_1017_1017_0, 202410_1018_1018_0, 202410_1019_1019_0, 202410_1020_1020_0, 202410_1021_1021_0] 2024.10.22 01:48:24.984847 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1016_1016_0, 202410_1017_1017_0, 202410_1018_1018_0, 202410_1019_1019_0, 202410_1020_1020_0, 202410_1021_1021_0] 2024.10.22 01:48:25.235806 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1016_1016_0, 202410_1017_1017_0, 202410_1018_1018_0, 202410_1019_1019_0, 202410_1020_1020_0, 202410_1021_1021_0] 2024.10.22 01:48:25.317182 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 10037 2024.10.22 01:48:25.445601 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:25.447299 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:25.479020 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_657_657_0 (state Temporary) 2024.10.22 01:48:25.533548 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_657_657_0 to 202410_670_670_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:25.540649 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 9.41 MiB, will set to 1.72 GiB (RSS), difference: -1.51 MiB 2024.10.22 01:48:25.588104 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10037 2024.10.22 01:48:25.950554 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 72347 2024.10.22 01:48:26.097005 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(90a24abc9c8bcc1f46f3a4e9771ecf4e) : [0, 184]: Setting download as failed: File segment: [0, 184], key: 90a24abc9c8bcc1f46f3a4e9771ecf4e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:26.101395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(90a24abc9c8bcc1f46f3a4e9771ecf4e): No space left in cache to reserve 185 bytes, will continue without cache download 2024.10.22 01:48:26.293464 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:26.299459 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:26.478272 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1110_1110_0 (state Temporary) 2024.10.22 01:48:26.647375 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1110_1110_0 to 202410_1123_1123_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:27.047796 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8b7c38e7f5431c457ec1718e6a520eea) : [0, 175814]: Setting download as failed: File segment: [0, 175814], key: 8b7c38e7f5431c457ec1718e6a520eea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:27.050777 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8b7c38e7f5431c457ec1718e6a520eea): No space left in cache to reserve 175815 bytes, will continue without cache download 2024.10.22 01:48:27.082360 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72347 2024.10.22 01:48:27.084181 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8b7c38e7f5431c457ec1718e6a520eea, offset: 0, size: 0 2024.10.22 01:48:27.088494 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8b7c38e7f5431c457ec1718e6a520eea for removal 2024.10.22 01:48:27.092919 [ 1476 ] {} CacheMetadata: Key 8b7c38e7f5431c457ec1718e6a520eea is removed from metadata 2024.10.22 01:48:27.094730 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 90a24abc9c8bcc1f46f3a4e9771ecf4e, offset: 0, size: 0 2024.10.22 01:48:27.097524 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 90a24abc9c8bcc1f46f3a4e9771ecf4e for removal 2024.10.22 01:48:27.103699 [ 1476 ] {} CacheMetadata: Key 90a24abc9c8bcc1f46f3a4e9771ecf4e is removed from metadata 2024.10.22 01:48:27.291005 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_488_100, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0, 202410_492_492_0, 202410_493_493_0] 2024.10.22 01:48:27.295803 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_488_100, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0, 202410_492_492_0, 202410_493_493_0] 2024.10.22 01:48:27.460753 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 8.91 MiB, will set to 1.72 GiB (RSS), difference: 4.46 MiB 2024.10.22 01:48:27.526411 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_488_100, 202410_489_489_0, 202410_490_490_0, 202410_491_491_0, 202410_492_492_0, 202410_493_493_0] 2024.10.22 01:48:27.583847 [ 1202 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:48:27.633099 [ 1202 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:48:27.878053 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(adc2cb1769bf9ca044fb1b16f3fbb26b) : [0, 155]: Setting download as failed: File segment: [0, 155], key: adc2cb1769bf9ca044fb1b16f3fbb26b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:27.879334 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(adc2cb1769bf9ca044fb1b16f3fbb26b): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 01:48:28.909370 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_134_24/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:48:29.055991 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(abe684d6add0544bf6e7a37491548466) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: abe684d6add0544bf6e7a37491548466, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:48:29.058601 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 16275, file segment info: File segment: [0, 33554431], key: abe684d6add0544bf6e7a37491548466, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:48:29.061372 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:48:29.923323 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(282fa14611d67db42b207ebef93ff1cd) : [0, 84909]: Setting download as failed: File segment: [0, 84909], key: 282fa14611d67db42b207ebef93ff1cd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:29.926353 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(282fa14611d67db42b207ebef93ff1cd): No space left in cache to reserve 84910 bytes, will continue without cache download 2024.10.22 01:48:30.045540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 282fa14611d67db42b207ebef93ff1cd, offset: 0, size: 0 2024.10.22 01:48:30.048815 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 282fa14611d67db42b207ebef93ff1cd for removal 2024.10.22 01:48:30.051175 [ 1476 ] {} CacheMetadata: Key 282fa14611d67db42b207ebef93ff1cd is removed from metadata 2024.10.22 01:48:30.052091 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: adc2cb1769bf9ca044fb1b16f3fbb26b, offset: 0, size: 0 2024.10.22 01:48:30.053271 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key adc2cb1769bf9ca044fb1b16f3fbb26b for removal 2024.10.22 01:48:30.055627 [ 1476 ] {} CacheMetadata: Key adc2cb1769bf9ca044fb1b16f3fbb26b is removed from metadata 2024.10.22 01:48:30.574905 [ 1168 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 01:48:30.673343 [ 1212 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef) (MergerMutator): Selected 4 parts from 202410_1_6_1 to 202410_9_9_0 2024.10.22 01:48:30.700426 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:48:30.700553 [ 1212 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:30.704436 [ 1212 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:30.704588 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:48:30.748155 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_9_2} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_6_1 to 202410_9_9_0 into Compact with storage Full 2024.10.22 01:48:31.096837 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 72390 2024.10.22 01:48:31.262785 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c8ed86266229d54cbd631ae550614774) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c8ed86266229d54cbd631ae550614774, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:48:31.274526 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 684, file segment info: File segment: [0, 33554431], key: c8ed86266229d54cbd631ae550614774, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:48:31.276078 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:48:31.353192 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 970268 2024.10.22 01:48:31.454244 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:31.461088 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:31.588820 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 8.83 MiB, will set to 1.71 GiB (RSS), difference: 2.42 MiB 2024.10.22 01:48:31.633058 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1111_1111_0 (state Temporary) 2024.10.22 01:48:31.673033 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1111_1111_0 to 202410_1124_1124_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:31.828354 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_9_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:48:31.836173 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72390 2024.10.22 01:48:32.128768 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:32.199525 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_6_1, total 20 rows starting from the beginning of the part 2024.10.22 01:48:32.235828 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:32.560611 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_630_630_0 (state Temporary) 2024.10.22 01:48:32.563776 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 48 entries to flush up to offset 73919 2024.10.22 01:48:32.596221 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a0ffb58271b2eba1c12b2e193d3c857f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a0ffb58271b2eba1c12b2e193d3c857f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:48:32.623562 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 883, file segment info: File segment: [0, 33554431], key: a0ffb58271b2eba1c12b2e193d3c857f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 01:48:32.635451 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:48:32.665544 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_630_630_0 to 202410_637_637_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:32.805270 [ 1704 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pyx/kwazpxhzldfyzhasatgjhiavmuoym, size 2576718 2024.10.22 01:48:32.864580 [ 1229 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 01:48:32.864762 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 970268 2024.10.22 01:48:32.912325 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:32.916572 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:32.977622 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_646_646_0 (state Temporary) 2024.10.22 01:48:33.010910 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_575_575_0, 202410_576_576_0, 202410_577_577_0, 202410_578_578_0, 202410_579_579_0, 202410_580_580_0] 2024.10.22 01:48:33.082607 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_575_575_0, 202410_576_576_0, 202410_577_577_0, 202410_578_578_0, 202410_579_579_0, 202410_580_580_0] 2024.10.22 01:48:33.091549 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 4 entries to flush up to offset 10041 2024.10.22 01:48:33.121058 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_646_646_0 to 202410_652_652_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:33.140941 [ 1200 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 01:48:33.176560 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 73919 2024.10.22 01:48:33.223014 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_575_575_0, 202410_576_576_0, 202410_577_577_0, 202410_578_578_0, 202410_579_579_0, 202410_580_580_0] 2024.10.22 01:48:33.330657 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:48:33.364740 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:48:33.462004 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(68b6a445162566c776daf03844055d02) : [0, 217]: Setting download as failed: File segment: [0, 217], key: 68b6a445162566c776daf03844055d02, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:33.467186 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(68b6a445162566c776daf03844055d02): No space left in cache to reserve 218 bytes, will continue without cache download 2024.10.22 01:48:33.468750 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_7_7_0, total 3 rows starting from the beginning of the part 2024.10.22 01:48:33.471750 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:48:33.481007 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:48:33.509240 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:48:33.519365 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_606_121, 202410_607_607_0, 202410_608_608_0, 202410_609_609_0, 202410_610_610_0, 202410_611_611_0] 2024.10.22 01:48:33.594484 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_606_121, 202410_607_607_0, 202410_608_608_0, 202410_609_609_0, 202410_610_610_0, 202410_611_611_0] 2024.10.22 01:48:33.647376 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:48:33.662591 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:48:33.670192 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column OpenerName 2024.10.22 01:48:33.696406 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:33.712167 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:33.746329 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_606_121, 202410_607_607_0, 202410_608_608_0, 202410_609_609_0, 202410_610_610_0, 202410_611_611_0] 2024.10.22 01:48:33.764399 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_658_658_0 (state Temporary) 2024.10.22 01:48:33.822816 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_658_658_0 to 202410_671_671_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:33.841370 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fhd/dbmszrdzvzrvcxqdqujnfucoeyvnl (LogSeriesLimiter: on interval from 2024-10-22 01:48:10 to 2024-10-22 01:48:33 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:48:34.061157 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10041 2024.10.22 01:48:34.359979 [ 1249 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 342 ms. 2024.10.22 01:48:34.442268 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_8_8_0, total 1 rows starting from the beginning of the part 2024.10.22 01:48:34.522339 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(bd17c4101ac8d377adbd1d23ae20c571) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: bd17c4101ac8d377adbd1d23ae20c571, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:48:34.528080 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 807764, file segment info: File segment: [0, 33554431], key: bd17c4101ac8d377adbd1d23ae20c571, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:48:34.532541 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:48:35.013753 [ 1386 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_9_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_9_9_0, total 1 rows starting from the beginning of the part 2024.10.22 01:48:35.252314 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(66d083637139a603ca44b6a07c381718) : [0, 379327]: Setting download as failed: File segment: [0, 379327], key: 66d083637139a603ca44b6a07c381718, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:35.254662 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(66d083637139a603ca44b6a07c381718): No space left in cache to reserve 379328 bytes, will continue without cache download 2024.10.22 01:48:35.306516 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 66d083637139a603ca44b6a07c381718, offset: 0, size: 0 2024.10.22 01:48:35.308916 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 66d083637139a603ca44b6a07c381718 for removal 2024.10.22 01:48:35.313054 [ 1476 ] {} CacheMetadata: Key 66d083637139a603ca44b6a07c381718 is removed from metadata 2024.10.22 01:48:35.317380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 68b6a445162566c776daf03844055d02, offset: 0, size: 0 2024.10.22 01:48:35.322252 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 68b6a445162566c776daf03844055d02 for removal 2024.10.22 01:48:35.326839 [ 1476 ] {} CacheMetadata: Key 68b6a445162566c776daf03844055d02 is removed from metadata 2024.10.22 01:48:35.656189 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 7.96 MiB, will set to 1.71 GiB (RSS), difference: -3.86 MiB 2024.10.22 01:48:35.846345 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 72459 2024.10.22 01:48:36.288468 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:36.299129 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:36.360063 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(0baeeed16f892dcdbeb02bc91654ed30) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0baeeed16f892dcdbeb02bc91654ed30, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:48:36.364780 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 934, file segment info: File segment: [0, 33554431], key: 0baeeed16f892dcdbeb02bc91654ed30, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 01:48:36.366408 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1112_1112_0 (state Temporary) 2024.10.22 01:48:36.367213 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:48:36.420970 [ 1681 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zjo/rukdlgjhenxlgngklgjmaoxzcgdwj, size 807764 2024.10.22 01:48:36.518393 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1112_1112_0 to 202410_1125_1125_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:36.695105 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72459 2024.10.22 01:48:36.812158 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:48:36.842593 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:48:37.004723 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:48:37.139628 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:48:37.176569 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:48:37.208670 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:48:37.252365 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:48:37.277506 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:48:37.312093 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 01:48:37.559458 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 550 2024.10.22 01:48:37.658803 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0f0d123b63bcbababd97199a85607645) : [0, 216]: Setting download as failed: File segment: [0, 216], key: 0f0d123b63bcbababd97199a85607645, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:37.660940 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0f0d123b63bcbababd97199a85607645): No space left in cache to reserve 217 bytes, will continue without cache download 2024.10.22 01:48:38.117255 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:38.121579 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:38.222637 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_532_532_0 (state Temporary) 2024.10.22 01:48:38.623612 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_532_532_0 to 202410_542_542_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:38.800694 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 550 2024.10.22 01:48:39.159745 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(16c157642f5142a2878fde580c7a6112) : [0, 319045]: Setting download as failed: File segment: [0, 319045], key: 16c157642f5142a2878fde580c7a6112, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:39.160965 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(16c157642f5142a2878fde580c7a6112): No space left in cache to reserve 319046 bytes, will continue without cache download 2024.10.22 01:48:39.308529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 16c157642f5142a2878fde580c7a6112, offset: 0, size: 0 2024.10.22 01:48:39.310527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 16c157642f5142a2878fde580c7a6112 for removal 2024.10.22 01:48:39.314158 [ 1476 ] {} CacheMetadata: Key 16c157642f5142a2878fde580c7a6112 is removed from metadata 2024.10.22 01:48:39.318691 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0f0d123b63bcbababd97199a85607645, offset: 0, size: 0 2024.10.22 01:48:39.320301 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0f0d123b63bcbababd97199a85607645 for removal 2024.10.22 01:48:39.328502 [ 1476 ] {} CacheMetadata: Key 0f0d123b63bcbababd97199a85607645 is removed from metadata 2024.10.22 01:48:39.678576 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 3.30 MiB, will set to 1.72 GiB (RSS), difference: -16.33 MiB 2024.10.22 01:48:39.975350 [ 1421 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): /var/lib/clickhouse/store/3c8/3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef/tmp_merge_202410_1_9_2/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:48:40.251104 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 971038 2024.10.22 01:48:40.495137 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(257117831dd49cff8a497f956c506bf0) : [0, 209]: Setting download as failed: File segment: [0, 209], key: 257117831dd49cff8a497f956c506bf0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:40.499561 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(257117831dd49cff8a497f956c506bf0): No space left in cache to reserve 210 bytes, will continue without cache download 2024.10.22 01:48:40.666229 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:40.668924 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:40.694131 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 82 entries to flush up to offset 74001 2024.10.22 01:48:40.703775 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:48:40.708445 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:48:40.709330 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 72502 2024.10.22 01:48:40.722987 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:40.796114 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_631_631_0 (state Temporary) 2024.10.22 01:48:40.829142 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:40.873423 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_631_631_0 to 202410_638_638_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:40.930738 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 971038 2024.10.22 01:48:40.921950 [ 1224 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_628_122 to 202410_629_634_1 2024.10.22 01:48:40.947693 [ 1224 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:40.952993 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:40.980195 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.286260467 sec. 2024.10.22 01:48:40.998916 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_634_123} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_628_122 to 202410_629_634_1 into Wide with storage Full 2024.10.22 01:48:41.162576 [ 1152 ] {} DatabaseCatalog: Execution took 254 ms. 2024.10.22 01:48:41.172572 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:41.179021 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:41.245009 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_647_647_0 (state Temporary) 2024.10.22 01:48:41.245439 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_634_123} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:48:41.254211 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_634_123} MergeTreeSequentialSource: Reading 120 marks from part 202410_1_628_122, total 973076 rows starting from the beginning of the part 2024.10.22 01:48:41.350301 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_634_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_629_634_1, total 7315 rows starting from the beginning of the part 2024.10.22 01:48:41.404706 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_647_647_0 to 202410_653_653_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:41.452982 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_648_29 to 202410_653_653_0 2024.10.22 01:48:41.459770 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74001 2024.10.22 01:48:41.477127 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:41.481474 [ 1154 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:41.555301 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:41.556939 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:41.563173 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 10047 2024.10.22 01:48:41.656208 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_648_29 to 202410_653_653_0 into Compact with storage Full 2024.10.22 01:48:41.767848 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:41.771228 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:41.783368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(05c9c5cb5b544a3928dbf6e11d293bb7) : [0, 9051]: Setting download as failed: File segment: [0, 9051], key: 05c9c5cb5b544a3928dbf6e11d293bb7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:41.786572 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(05c9c5cb5b544a3928dbf6e11d293bb7): No space left in cache to reserve 9052 bytes, will continue without cache download 2024.10.22 01:48:41.808580 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_659_659_0 (state Temporary) 2024.10.22 01:48:41.817009 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:48:41.863311 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_648_29, total 14444 rows starting from the beginning of the part 2024.10.22 01:48:41.875800 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1113_1113_0 (state Temporary) 2024.10.22 01:48:42.062886 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_659_659_0 to 202410_672_672_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:42.131204 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 05c9c5cb5b544a3928dbf6e11d293bb7, offset: 0, size: 0 2024.10.22 01:48:42.134622 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10047 2024.10.22 01:48:42.135832 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 05c9c5cb5b544a3928dbf6e11d293bb7 for removal 2024.10.22 01:48:42.139567 [ 1476 ] {} CacheMetadata: Key 05c9c5cb5b544a3928dbf6e11d293bb7 is removed from metadata 2024.10.22 01:48:42.146086 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 257117831dd49cff8a497f956c506bf0, offset: 0, size: 0 2024.10.22 01:48:42.152415 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 257117831dd49cff8a497f956c506bf0 for removal 2024.10.22 01:48:42.169029 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_649_649_0, total 110 rows starting from the beginning of the part 2024.10.22 01:48:42.171419 [ 1476 ] {} CacheMetadata: Key 257117831dd49cff8a497f956c506bf0 is removed from metadata 2024.10.22 01:48:42.243860 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_568_112, 202410_569_574_1] 2024.10.22 01:48:42.267848 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_650_650_0, total 113 rows starting from the beginning of the part 2024.10.22 01:48:42.248577 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_568_112, 202410_569_574_1] 2024.10.22 01:48:42.280273 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_568_112, 202410_569_574_1] 2024.10.22 01:48:42.359979 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_651_651_0, total 79 rows starting from the beginning of the part 2024.10.22 01:48:42.444642 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1113_1113_0 to 202410_1126_1126_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:42.460231 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_652_652_0, total 48 rows starting from the beginning of the part 2024.10.22 01:48:42.525675 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 4.72 MiB, will set to 1.72 GiB (RSS), difference: -19.73 MiB 2024.10.22 01:48:42.575160 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1121_5 to 202410_1126_1126_0 2024.10.22 01:48:42.609175 [ 1271 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:42.617565 [ 1271 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:42.624437 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72502 2024.10.22 01:48:42.647189 [ 1162 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:48:42.660488 [ 1397 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_9_2} MergeTask::MergeProjectionsStage: Merge sorted 25 rows, containing 72 columns (72 merged, 0 gathered) in 11.921496109 sec., 2.0970522299735963 rows/sec., 3.35 KiB/sec. 2024.10.22 01:48:42.669564 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1121_5 to 202410_1126_1126_0 into Compact with storage Full 2024.10.22 01:48:42.681263 [ 1162 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:48:42.682747 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_653_653_0, total 82 rows starting from the beginning of the part 2024.10.22 01:48:42.738567 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:48:42.754533 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1121_5, total 1654 rows starting from the beginning of the part 2024.10.22 01:48:42.808592 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1122_1122_0, total 53 rows starting from the beginning of the part 2024.10.22 01:48:42.966654 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1123_1123_0, total 64 rows starting from the beginning of the part 2024.10.22 01:48:42.970774 [ 1399 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_9_2} MergedBlockOutputStream: filled checksums 202410_1_9_2 (state Temporary) 2024.10.22 01:48:43.016473 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1124_1124_0, total 43 rows starting from the beginning of the part 2024.10.22 01:48:43.064759 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1125_1125_0, total 69 rows starting from the beginning of the part 2024.10.22 01:48:43.119307 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1126_1126_0, total 43 rows starting from the beginning of the part 2024.10.22 01:48:43.217080 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(e328962a7afb51f68d37327dd3977064) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e328962a7afb51f68d37327dd3977064, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:48:43.224583 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: e328962a7afb51f68d37327dd3977064, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 01:48:43.232349 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:48:43.475390 [ 1388 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_9_2} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Renaming temporary part tmp_merge_202410_1_9_2 to 202410_1_9_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:43.486792 [ 1388 ] {3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef::202410_1_9_2} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef) (MergerMutator): Merged 4 parts: [202410_1_6_1, 202410_9_9_0] -> 202410_1_9_2 2024.10.22 01:48:43.553433 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.35 MiB. 2024.10.22 01:48:44.016767 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_634_123/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:48:44.193591 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_133_133_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:48:44.354906 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} MergeTask::MergeProjectionsStage: Merge sorted 1926 rows, containing 13 columns (13 merged, 0 gathered) in 1.695004355 sec., 1136.280266371351 rows/sec., 201.55 KiB/sec. 2024.10.22 01:48:44.444644 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bffe096fcbc588451f3d804ede100850) : [0, 185]: Setting download as failed: File segment: [0, 185], key: bffe096fcbc588451f3d804ede100850, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:44.448686 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bffe096fcbc588451f3d804ede100850): No space left in cache to reserve 186 bytes, will continue without cache download 2024.10.22 01:48:44.511493 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} MergedBlockOutputStream: filled checksums 202410_1096_1126_6 (state Temporary) 2024.10.22 01:48:44.638548 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_133_133_0 (state Temporary) 2024.10.22 01:48:44.835531 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1126_6 to 202410_1096_1126_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:44.844337 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1126_6} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1121_5, 202410_1126_1126_0] -> 202410_1096_1126_6 2024.10.22 01:48:44.910487 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:48:45.247564 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} MergeTask::MergeProjectionsStage: Merge sorted 14876 rows, containing 13 columns (13 merged, 0 gathered) in 3.723320735 sec., 3995.3581919930944 rows/sec., 1.73 MiB/sec. 2024.10.22 01:48:45.520699 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} MergedBlockOutputStream: filled checksums 202410_507_653_30 (state Temporary) 2024.10.22 01:48:45.647161 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 11.05 MiB, will set to 1.73 GiB (RSS), difference: 12.14 MiB 2024.10.22 01:48:45.805839 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(843e20db923ff902e7efe2738a9fe690) : [0, 213426]: Setting download as failed: File segment: [0, 213426], key: 843e20db923ff902e7efe2738a9fe690, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:45.909105 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(843e20db923ff902e7efe2738a9fe690): No space left in cache to reserve 213427 bytes, will continue without cache download 2024.10.22 01:48:45.973373 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 843e20db923ff902e7efe2738a9fe690, offset: 0, size: 0 2024.10.22 01:48:45.975990 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 843e20db923ff902e7efe2738a9fe690 for removal 2024.10.22 01:48:45.979611 [ 1476 ] {} CacheMetadata: Key 843e20db923ff902e7efe2738a9fe690 is removed from metadata 2024.10.22 01:48:45.980024 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bffe096fcbc588451f3d804ede100850, offset: 0, size: 0 2024.10.22 01:48:45.983489 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bffe096fcbc588451f3d804ede100850 for removal 2024.10.22 01:48:45.995654 [ 1476 ] {} CacheMetadata: Key bffe096fcbc588451f3d804ede100850 is removed from metadata 2024.10.22 01:48:46.008916 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_653_30 to 202410_507_653_30 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:46.013351 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_653_30} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_648_29, 202410_653_653_0] -> 202410_507_653_30 2024.10.22 01:48:46.174971 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.09 MiB. 2024.10.22 01:48:46.304770 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 551 2024.10.22 01:48:46.650729 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 102 entries to flush up to offset 72604 2024.10.22 01:48:46.669460 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:46.672303 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:47.038747 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_533_533_0 (state Temporary) 2024.10.22 01:48:47.081993 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:47.087722 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:47.128272 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ec6a29fc0bdeb9523e030bec4e9c9cf9) : [0, 215]: Setting download as failed: File segment: [0, 215], key: ec6a29fc0bdeb9523e030bec4e9c9cf9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:47.130617 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ec6a29fc0bdeb9523e030bec4e9c9cf9): No space left in cache to reserve 216 bytes, will continue without cache download 2024.10.22 01:48:47.177112 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1114_1114_0 (state Temporary) 2024.10.22 01:48:47.257132 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1114_1114_0 to 202410_1127_1127_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:47.261350 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_533_533_0 to 202410_543_543_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:47.291918 [ 1170 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_538_110 to 202410_543_543_0 2024.10.22 01:48:47.312827 [ 1170 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:47.417649 [ 1170 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:47.492711 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_538_110 to 202410_543_543_0 into Compact with storage Full 2024.10.22 01:48:47.522784 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72604 2024.10.22 01:48:47.531449 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 14.87 MiB, will set to 1.73 GiB (RSS), difference: 5.19 MiB 2024.10.22 01:48:47.771071 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 551 2024.10.22 01:48:47.810779 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:48:47.933931 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 972193 2024.10.22 01:48:47.994107 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_538_110, total 582 rows starting from the beginning of the part 2024.10.22 01:48:48.157270 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_539_539_0, total 1 rows starting from the beginning of the part 2024.10.22 01:48:48.426601 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_540_540_0, total 1 rows starting from the beginning of the part 2024.10.22 01:48:48.632573 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7f28813fe90b19e7a7be18b8e0d63c79) : [0, 236909]: Setting download as failed: File segment: [0, 236909], key: 7f28813fe90b19e7a7be18b8e0d63c79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:48.637491 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7f28813fe90b19e7a7be18b8e0d63c79): No space left in cache to reserve 236910 bytes, will continue without cache download 2024.10.22 01:48:48.677314 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7f28813fe90b19e7a7be18b8e0d63c79, offset: 0, size: 0 2024.10.22 01:48:48.681064 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7f28813fe90b19e7a7be18b8e0d63c79 for removal 2024.10.22 01:48:48.688501 [ 1476 ] {} CacheMetadata: Key 7f28813fe90b19e7a7be18b8e0d63c79 is removed from metadata 2024.10.22 01:48:48.696356 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ec6a29fc0bdeb9523e030bec4e9c9cf9, offset: 0, size: 0 2024.10.22 01:48:48.700530 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ec6a29fc0bdeb9523e030bec4e9c9cf9 for removal 2024.10.22 01:48:48.709460 [ 1476 ] {} CacheMetadata: Key ec6a29fc0bdeb9523e030bec4e9c9cf9 is removed from metadata 2024.10.22 01:48:48.742417 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:48.746244 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:48.855866 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_541_541_0, total 1 rows starting from the beginning of the part 2024.10.22 01:48:48.963531 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_632_632_0 (state Temporary) 2024.10.22 01:48:48.970553 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 119 entries to flush up to offset 74120 2024.10.22 01:48:49.098458 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_632_632_0 to 202410_639_639_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:49.148793 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_542_542_0, total 1 rows starting from the beginning of the part 2024.10.22 01:48:49.150672 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 972193 2024.10.22 01:48:49.272033 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:49.274737 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:49.328941 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_648_648_0 (state Temporary) 2024.10.22 01:48:49.409400 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_648_648_0 to 202410_654_654_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:49.425499 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_543_543_0, total 1 rows starting from the beginning of the part 2024.10.22 01:48:49.461040 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74120 2024.10.22 01:48:49.637499 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 10072 2024.10.22 01:48:49.779149 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:49.781752 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:49.831163 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_660_660_0 (state Temporary) 2024.10.22 01:48:49.870741 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e23f9aae548577455f5dda8a702e1ff5) : [0, 218]: Setting download as failed: File segment: [0, 218], key: e23f9aae548577455f5dda8a702e1ff5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:49.872625 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e23f9aae548577455f5dda8a702e1ff5): No space left in cache to reserve 219 bytes, will continue without cache download 2024.10.22 01:48:50.004779 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_660_660_0 to 202410_673_673_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:50.060773 [ 1169 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_668_134 to 202410_673_673_0 2024.10.22 01:48:50.070223 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10072 2024.10.22 01:48:50.220204 [ 1169 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:50.225289 [ 1169 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:50.335491 [ 1169 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 302 ms. 2024.10.22 01:48:50.372077 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_668_134 to 202410_673_673_0 into Compact with storage Full 2024.10.22 01:48:50.484756 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:48:50.508976 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_668_134, total 10114 rows starting from the beginning of the part 2024.10.22 01:48:50.530634 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1015_495, 202410_1016_1021_1] 2024.10.22 01:48:50.533963 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1015_495, 202410_1016_1021_1] 2024.10.22 01:48:50.552239 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1015_495, 202410_1016_1021_1] 2024.10.22 01:48:50.570504 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_669_669_0, total 13 rows starting from the beginning of the part 2024.10.22 01:48:50.620103 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_670_670_0, total 16 rows starting from the beginning of the part 2024.10.22 01:48:50.672092 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_671_671_0, total 4 rows starting from the beginning of the part 2024.10.22 01:48:50.705359 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:48:50.709669 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:48:50.715309 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 12.50 MiB, will set to 1.73 GiB (RSS), difference: -2.06 MiB 2024.10.22 01:48:50.732520 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_672_672_0, total 6 rows starting from the beginning of the part 2024.10.22 01:48:50.997750 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_673_673_0, total 25 rows starting from the beginning of the part 2024.10.22 01:48:51.405780 [ 1184 ] {} DatabaseCatalog: Execution took 245 ms. 2024.10.22 01:48:51.543211 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 72676 2024.10.22 01:48:51.657503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e6d0c3028fe72d23a9e09c8e74a266d4) : [0, 357455]: Setting download as failed: File segment: [0, 357455], key: e6d0c3028fe72d23a9e09c8e74a266d4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:51.661525 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e6d0c3028fe72d23a9e09c8e74a266d4): No space left in cache to reserve 357456 bytes, will continue without cache download 2024.10.22 01:48:51.962069 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e6d0c3028fe72d23a9e09c8e74a266d4, offset: 0, size: 0 2024.10.22 01:48:51.964239 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e6d0c3028fe72d23a9e09c8e74a266d4 for removal 2024.10.22 01:48:51.966589 [ 1476 ] {} CacheMetadata: Key e6d0c3028fe72d23a9e09c8e74a266d4 is removed from metadata 2024.10.22 01:48:51.968512 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e23f9aae548577455f5dda8a702e1ff5, offset: 0, size: 0 2024.10.22 01:48:51.970539 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e23f9aae548577455f5dda8a702e1ff5 for removal 2024.10.22 01:48:51.972727 [ 1476 ] {} CacheMetadata: Key e23f9aae548577455f5dda8a702e1ff5 is removed from metadata 2024.10.22 01:48:52.142757 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:52.147780 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:52.149048 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} MergeTask::MergeProjectionsStage: Merge sorted 587 rows, containing 32 columns (32 merged, 0 gathered) in 4.673333355 sec., 125.6062761651635 rows/sec., 11.49 KiB/sec. 2024.10.22 01:48:52.226377 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1115_1115_0 (state Temporary) 2024.10.22 01:48:52.263218 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} MergedBlockOutputStream: filled checksums 202410_1_543_111 (state Temporary) 2024.10.22 01:48:52.274443 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1022_1022_0, 202410_1023_1023_0, 202410_1024_1024_0, 202410_1025_1025_0, 202410_1026_1026_0, 202410_1027_1027_0] 2024.10.22 01:48:52.275505 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1022_1022_0, 202410_1023_1023_0, 202410_1024_1024_0, 202410_1025_1025_0, 202410_1026_1026_0, 202410_1027_1027_0] 2024.10.22 01:48:52.280649 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1115_1115_0 to 202410_1128_1128_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:52.302417 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1022_1022_0, 202410_1023_1023_0, 202410_1024_1024_0, 202410_1025_1025_0, 202410_1026_1026_0, 202410_1027_1027_0] 2024.10.22 01:48:52.438998 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a2f761e2c0d311319af91e5da96c6d76) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a2f761e2c0d311319af91e5da96c6d76, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 01:48:52.531305 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_543_111 to 202410_1_543_111 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:52.537381 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_543_111} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_538_110, 202410_543_543_0] -> 202410_1_543_111 2024.10.22 01:48:52.550670 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 47662, file segment info: File segment: [0, 33554431], key: a2f761e2c0d311319af91e5da96c6d76, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 01:48:52.554716 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:48:52.555495 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72676 2024.10.22 01:48:52.579711 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:48:52.614418 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} MergeTask::MergeProjectionsStage: Merge sorted 10178 rows, containing 12 columns (12 merged, 0 gathered) in 2.247642689 sec., 4528.299827108329 rows/sec., 720.68 KiB/sec. 2024.10.22 01:48:52.963872 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} MergedBlockOutputStream: filled checksums 202410_1_673_135 (state Temporary) 2024.10.22 01:48:53.146640 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_133_133_0 to 202410_136_136_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:53.510684 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_673_135 to 202410_1_673_135 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:53.517137 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_673_135} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_668_134, 202410_673_673_0] -> 202410_1_673_135 2024.10.22 01:48:53.575581 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.62 MiB. 2024.10.22 01:48:53.883148 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(41a1f21c630db47c90b9a0c15bdfaa0f) : [0, 219]: Setting download as failed: File segment: [0, 219], key: 41a1f21c630db47c90b9a0c15bdfaa0f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:53.885840 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(41a1f21c630db47c90b9a0c15bdfaa0f): No space left in cache to reserve 220 bytes, will continue without cache download 2024.10.22 01:48:54.475508 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 11.73 MiB, will set to 1.73 GiB (RSS), difference: 7.32 MiB 2024.10.22 01:48:55.073348 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(630ad6d6ac2d9135ce5be44a6bdb36c2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 630ad6d6ac2d9135ce5be44a6bdb36c2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 01:48:55.076968 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 797, file segment info: File segment: [0, 33554431], key: 630ad6d6ac2d9135ce5be44a6bdb36c2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 01:48:55.080303 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:48:55.127310 [ 1728 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fhd/dbmszrdzvzrvcxqdqujnfucoeyvnl, size 47662 2024.10.22 01:48:55.273771 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 552 2024.10.22 01:48:55.395511 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:48:55.444905 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:48:55.458336 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:48:55.467371 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:48:55.481629 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:48:55.507279 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:48:55.550998 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:48:55.569625 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column HistoryLength 2024.10.22 01:48:55.625746 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rom/rneezooasulspmlngmionskosiaze (LogSeriesLimiter: on interval from 2024-10-22 01:48:33 to 2024-10-22 01:48:55 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:48:55.701004 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:55.705446 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:55.863406 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_634_123/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:48:55.886893 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_534_534_0 (state Temporary) 2024.10.22 01:48:56.096120 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_534_534_0 to 202410_544_544_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:56.156393 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 972963 2024.10.22 01:48:56.166471 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 552 2024.10.22 01:48:56.183098 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(53f3f0a52132a29e49a58f8edaca70bc) : [0, 431910]: Setting download as failed: File segment: [0, 431910], key: 53f3f0a52132a29e49a58f8edaca70bc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:56.185072 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(53f3f0a52132a29e49a58f8edaca70bc): No space left in cache to reserve 431911 bytes, will continue without cache download 2024.10.22 01:48:56.464519 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 53f3f0a52132a29e49a58f8edaca70bc, offset: 0, size: 0 2024.10.22 01:48:56.466125 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 53f3f0a52132a29e49a58f8edaca70bc for removal 2024.10.22 01:48:56.468047 [ 1476 ] {} CacheMetadata: Key 53f3f0a52132a29e49a58f8edaca70bc is removed from metadata 2024.10.22 01:48:56.469961 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 41a1f21c630db47c90b9a0c15bdfaa0f, offset: 0, size: 0 2024.10.22 01:48:56.471296 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 41a1f21c630db47c90b9a0c15bdfaa0f for removal 2024.10.22 01:48:56.475217 [ 1476 ] {} CacheMetadata: Key 41a1f21c630db47c90b9a0c15bdfaa0f is removed from metadata 2024.10.22 01:48:56.574117 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 72739 2024.10.22 01:48:56.913006 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:56.928976 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:56.934940 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:48:56.974463 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 88 entries to flush up to offset 74208 2024.10.22 01:48:57.019788 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:57.076979 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1116_1116_0 (state Temporary) 2024.10.22 01:48:57.142786 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1116_1116_0 to 202410_1129_1129_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:57.155954 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5321 2024.10.22 01:48:57.186373 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_633_633_0 (state Temporary) 2024.10.22 01:48:57.203758 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:57.207965 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:57.271589 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_649_649_0 (state Temporary) 2024.10.22 01:48:57.311743 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72739 2024.10.22 01:48:57.337279 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_649_649_0 to 202410_655_655_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:57.382812 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74208 2024.10.22 01:48:57.454734 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_633_633_0 to 202410_640_640_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:57.479084 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 972963 2024.10.22 01:48:57.571981 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 10093 2024.10.22 01:48:57.579480 [ 1256 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_635_635_0 to 202410_640_640_0 2024.10.22 01:48:57.599644 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6bd91d33d9d3acb09062f39e77c00939) : [0, 224]: Setting download as failed: File segment: [0, 224], key: 6bd91d33d9d3acb09062f39e77c00939, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:57.601131 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6bd91d33d9d3acb09062f39e77c00939): No space left in cache to reserve 225 bytes, will continue without cache download 2024.10.22 01:48:57.614027 [ 1256 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:57.620994 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:57.694594 [ 1226 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:48:57.757053 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_635_635_0 to 202410_640_640_0 into Compact with storage Full 2024.10.22 01:48:57.760499 [ 1226 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:48:57.775191 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:48:57.779795 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:48:57.789465 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:48:57.810435 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_635_635_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:48:57.838527 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_636_636_0, total 1157 rows starting from the beginning of the part 2024.10.22 01:48:57.847061 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_661_661_0 (state Temporary) 2024.10.22 01:48:57.861006 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_637_637_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:48:57.876745 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_638_638_0, total 770 rows starting from the beginning of the part 2024.10.22 01:48:57.904675 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_639_639_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:48:57.917037 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_661_661_0 to 202410_674_674_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:57.930704 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_640_640_0, total 770 rows starting from the beginning of the part 2024.10.22 01:48:57.967795 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10093 2024.10.22 01:48:58.380189 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 12.24 MiB, will set to 1.73 GiB (RSS), difference: -5.50 MiB 2024.10.22 01:48:58.661591 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0ae298979f1f9b8907905b4cc5ed301f) : [0, 623196]: Setting download as failed: File segment: [0, 623196], key: 0ae298979f1f9b8907905b4cc5ed301f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:48:58.663616 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0ae298979f1f9b8907905b4cc5ed301f): No space left in cache to reserve 623197 bytes, will continue without cache download 2024.10.22 01:48:58.812183 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0ae298979f1f9b8907905b4cc5ed301f, offset: 0, size: 0 2024.10.22 01:48:58.815993 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0ae298979f1f9b8907905b4cc5ed301f for removal 2024.10.22 01:48:58.820755 [ 1476 ] {} CacheMetadata: Key 0ae298979f1f9b8907905b4cc5ed301f is removed from metadata 2024.10.22 01:48:58.826933 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6bd91d33d9d3acb09062f39e77c00939, offset: 0, size: 0 2024.10.22 01:48:58.829791 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6bd91d33d9d3acb09062f39e77c00939 for removal 2024.10.22 01:48:58.835744 [ 1476 ] {} CacheMetadata: Key 6bd91d33d9d3acb09062f39e77c00939 is removed from metadata 2024.10.22 01:48:59.374022 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} MergeTask::MergeProjectionsStage: Merge sorted 6162 rows, containing 4 columns (4 merged, 0 gathered) in 1.627417012 sec., 3786.368186250716 rows/sec., 59.16 KiB/sec. 2024.10.22 01:48:59.449247 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} MergedBlockOutputStream: filled checksums 202410_635_640_1 (state Temporary) 2024.10.22 01:48:59.852289 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_635_640_1 to 202410_635_640_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:48:59.859591 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_635_640_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_635_635_0, 202410_640_640_0] -> 202410_635_640_1 2024.10.22 01:49:00.006526 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:49:00.247298 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ce45b2773d83103dcd89787e385d67e0) : [0, 223]: Setting download as failed: File segment: [0, 223], key: ce45b2773d83103dcd89787e385d67e0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:00.249642 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ce45b2773d83103dcd89787e385d67e0): No space left in cache to reserve 224 bytes, will continue without cache download 2024.10.22 01:49:00.510758 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 14.09 MiB, will set to 1.73 GiB (RSS), difference: 7.19 MiB 2024.10.22 01:49:00.707762 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:49:00.709759 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:49:00.824841 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_134_24/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:49:01.147168 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2236f39697b0779a39c8bb255906eaa9) : [0, 616205]: Setting download as failed: File segment: [0, 616205], key: 2236f39697b0779a39c8bb255906eaa9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:01.149161 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2236f39697b0779a39c8bb255906eaa9): No space left in cache to reserve 616206 bytes, will continue without cache download 2024.10.22 01:49:01.356833 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 72800 2024.10.22 01:49:01.499254 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2236f39697b0779a39c8bb255906eaa9, offset: 0, size: 0 2024.10.22 01:49:01.501507 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2236f39697b0779a39c8bb255906eaa9 for removal 2024.10.22 01:49:01.504938 [ 1476 ] {} CacheMetadata: Key 2236f39697b0779a39c8bb255906eaa9 is removed from metadata 2024.10.22 01:49:01.507164 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ce45b2773d83103dcd89787e385d67e0, offset: 0, size: 0 2024.10.22 01:49:01.509928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ce45b2773d83103dcd89787e385d67e0 for removal 2024.10.22 01:49:01.513556 [ 1476 ] {} CacheMetadata: Key ce45b2773d83103dcd89787e385d67e0 is removed from metadata 2024.10.22 01:49:01.770230 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:01.774848 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:01.946305 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1117_1117_0 (state Temporary) 2024.10.22 01:49:02.222301 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1117_1117_0 to 202410_1130_1130_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:02.412807 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72800 2024.10.22 01:49:02.473286 [ 1224 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:49:02.797563 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4d77c461be90f9c24602db19af7a8d40) : [0, 219]: Setting download as failed: File segment: [0, 219], key: 4d77c461be90f9c24602db19af7a8d40, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:02.804179 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4d77c461be90f9c24602db19af7a8d40): No space left in cache to reserve 220 bytes, will continue without cache download 2024.10.22 01:49:03.668994 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 553 2024.10.22 01:49:03.974177 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f6cc984758dd10e70c0672ead6babe37) : [0, 426740]: Setting download as failed: File segment: [0, 426740], key: f6cc984758dd10e70c0672ead6babe37, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:03.975673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f6cc984758dd10e70c0672ead6babe37): No space left in cache to reserve 426741 bytes, will continue without cache download 2024.10.22 01:49:04.133271 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:04.137994 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:04.212917 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_535_535_0 (state Temporary) 2024.10.22 01:49:04.225451 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f6cc984758dd10e70c0672ead6babe37, offset: 0, size: 0 2024.10.22 01:49:04.235337 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f6cc984758dd10e70c0672ead6babe37 for removal 2024.10.22 01:49:04.240168 [ 1476 ] {} CacheMetadata: Key f6cc984758dd10e70c0672ead6babe37 is removed from metadata 2024.10.22 01:49:04.243288 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4d77c461be90f9c24602db19af7a8d40, offset: 0, size: 0 2024.10.22 01:49:04.246653 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4d77c461be90f9c24602db19af7a8d40 for removal 2024.10.22 01:49:04.251199 [ 1476 ] {} CacheMetadata: Key 4d77c461be90f9c24602db19af7a8d40 is removed from metadata 2024.10.22 01:49:04.482129 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1536 entries to flush up to offset 974499 2024.10.22 01:49:04.502960 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_535_535_0 to 202410_545_545_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:04.773623 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 5363 2024.10.22 01:49:04.810070 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 553 2024.10.22 01:49:05.173084 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 76 entries to flush up to offset 74284 2024.10.22 01:49:05.359600 [ 1271 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:49:05.377486 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:05.383518 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:05.470395 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 10103 2024.10.22 01:49:05.494987 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_634_634_0 (state Temporary) 2024.10.22 01:49:05.509255 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7794c5653eab66378ea5ea26cd724ca9) : [0, 216]: Setting download as failed: File segment: [0, 216], key: 7794c5653eab66378ea5ea26cd724ca9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:05.511965 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7794c5653eab66378ea5ea26cd724ca9): No space left in cache to reserve 217 bytes, will continue without cache download 2024.10.22 01:49:05.580115 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:05.584524 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:05.614567 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:05.624859 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:05.662590 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_650_650_0 (state Temporary) 2024.10.22 01:49:05.684359 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_662_662_0 (state Temporary) 2024.10.22 01:49:05.743563 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_662_662_0 to 202410_675_675_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:05.757720 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_650_650_0 to 202410_656_656_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:05.760421 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 12.43 MiB, will set to 1.73 GiB (RSS), difference: -13.38 MiB 2024.10.22 01:49:05.764505 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_634_634_0 to 202410_641_641_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:05.808054 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10103 2024.10.22 01:49:05.819756 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74284 2024.10.22 01:49:05.906107 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 974499 2024.10.22 01:49:06.087098 [ 1253 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 293 ms. 2024.10.22 01:49:06.630242 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(71981efbe9e715df70f90f5755eb6146) : [0, 322651]: Setting download as failed: File segment: [0, 322651], key: 71981efbe9e715df70f90f5755eb6146, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:06.715374 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 72853 2024.10.22 01:49:06.715950 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(71981efbe9e715df70f90f5755eb6146): No space left in cache to reserve 322652 bytes, will continue without cache download 2024.10.22 01:49:06.903686 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 71981efbe9e715df70f90f5755eb6146, offset: 0, size: 0 2024.10.22 01:49:06.907082 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 71981efbe9e715df70f90f5755eb6146 for removal 2024.10.22 01:49:06.909736 [ 1476 ] {} CacheMetadata: Key 71981efbe9e715df70f90f5755eb6146 is removed from metadata 2024.10.22 01:49:06.911795 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7794c5653eab66378ea5ea26cd724ca9, offset: 0, size: 0 2024.10.22 01:49:06.916236 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7794c5653eab66378ea5ea26cd724ca9 for removal 2024.10.22 01:49:06.921249 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_634_123/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:49:06.921617 [ 1476 ] {} CacheMetadata: Key 7794c5653eab66378ea5ea26cd724ca9 is removed from metadata 2024.10.22 01:49:07.080353 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:07.089505 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:07.173072 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 5 old parts to remove. Parts: [202410_507_592_17, 202410_593_593_0, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0] 2024.10.22 01:49:07.175938 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from filesystem (serially): Parts: [202410_507_592_17, 202410_593_593_0, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0] 2024.10.22 01:49:07.259239 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1118_1118_0 (state Temporary) 2024.10.22 01:49:07.300154 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1118_1118_0 to 202410_1131_1131_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:07.361172 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72853 2024.10.22 01:49:07.534428 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from memory: Parts: [202410_507_592_17, 202410_593_593_0, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0] 2024.10.22 01:49:07.652089 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1126_6 to 202410_1131_1131_0 2024.10.22 01:49:07.660315 [ 1269 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:07.762602 [ 1269 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:49:07.766813 [ 1269 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 452 ms. 2024.10.22 01:49:07.786975 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1126_6 to 202410_1131_1131_0 into Compact with storage Full 2024.10.22 01:49:07.862903 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:49:07.874400 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1126_6, total 1926 rows starting from the beginning of the part 2024.10.22 01:49:07.877413 [ 1277 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 208 ms. 2024.10.22 01:49:07.940720 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1127_1127_0, total 102 rows starting from the beginning of the part 2024.10.22 01:49:07.963495 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(315803728eea99e137314e16464d1845) : [0, 221]: Setting download as failed: File segment: [0, 221], key: 315803728eea99e137314e16464d1845, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:07.965647 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(315803728eea99e137314e16464d1845): No space left in cache to reserve 222 bytes, will continue without cache download 2024.10.22 01:49:08.020938 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1128_1128_0, total 72 rows starting from the beginning of the part 2024.10.22 01:49:08.072667 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1129_1129_0, total 63 rows starting from the beginning of the part 2024.10.22 01:49:08.131471 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1130_1130_0, total 61 rows starting from the beginning of the part 2024.10.22 01:49:08.183665 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1131_1131_0, total 53 rows starting from the beginning of the part 2024.10.22 01:49:08.469824 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 12.27 MiB, will set to 1.74 GiB (RSS), difference: 12.64 MiB 2024.10.22 01:49:09.458845 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a8ed246560536e482bb6890056695d3c) : [0, 472898]: Setting download as failed: File segment: [0, 472898], key: a8ed246560536e482bb6890056695d3c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:09.460513 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a8ed246560536e482bb6890056695d3c): No space left in cache to reserve 472899 bytes, will continue without cache download 2024.10.22 01:49:09.626308 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a8ed246560536e482bb6890056695d3c, offset: 0, size: 0 2024.10.22 01:49:09.628827 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a8ed246560536e482bb6890056695d3c for removal 2024.10.22 01:49:09.646382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 315803728eea99e137314e16464d1845, offset: 0, size: 0 2024.10.22 01:49:09.648801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 315803728eea99e137314e16464d1845 for removal 2024.10.22 01:49:09.662342 [ 1476 ] {} CacheMetadata: Key a8ed246560536e482bb6890056695d3c is removed from metadata 2024.10.22 01:49:09.664793 [ 1476 ] {} CacheMetadata: Key 315803728eea99e137314e16464d1845 is removed from metadata 2024.10.22 01:49:10.069160 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} MergeTask::MergeProjectionsStage: Merge sorted 2277 rows, containing 13 columns (13 merged, 0 gathered) in 2.289793105 sec., 994.4129864955637 rows/sec., 175.64 KiB/sec. 2024.10.22 01:49:10.123753 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} MergedBlockOutputStream: filled checksums 202410_1096_1131_7 (state Temporary) 2024.10.22 01:49:10.534420 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1131_7 to 202410_1096_1131_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:10.540953 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1131_7} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1126_6, 202410_1131_1131_0] -> 202410_1096_1131_7 2024.10.22 01:49:10.560027 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(08cf9264ad334000ebeafa9306abe0b2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 08cf9264ad334000ebeafa9306abe0b2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:49:10.563064 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 08cf9264ad334000ebeafa9306abe0b2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:49:10.565780 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:10.704276 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:49:10.763223 [ 1164 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 213 ms. 2024.10.22 01:49:10.813044 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:49:10.818478 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:49:11.173474 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_634_123} MergeTask::MergeProjectionsStage: Merge sorted 980391 rows, containing 4 columns (4 merged, 0 gathered) in 30.201415249 sec., 32461.756905000064 rows/sec., 486.82 KiB/sec. 2024.10.22 01:49:11.273972 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_634_123} MergedBlockOutputStream: filled checksums 202410_1_634_123 (state Temporary) 2024.10.22 01:49:11.367842 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 72908 2024.10.22 01:49:11.770638 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e16c57a08d6d14bcf739032d4a1ebf2e) : [0, 218]: Setting download as failed: File segment: [0, 218], key: e16c57a08d6d14bcf739032d4a1ebf2e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:11.771909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e16c57a08d6d14bcf739032d4a1ebf2e): No space left in cache to reserve 219 bytes, will continue without cache download 2024.10.22 01:49:11.824316 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:11.836995 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:11.883607 [ 1163 ] {} DatabaseCatalog: Execution took 233 ms. 2024.10.22 01:49:12.111477 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1119_1119_0 (state Temporary) 2024.10.22 01:49:12.268463 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1119_1119_0 to 202410_1132_1132_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:12.320094 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 554 2024.10.22 01:49:12.439441 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_611_122, 202410_612_612_0, 202410_613_613_0, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0] 2024.10.22 01:49:12.442152 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_611_122, 202410_612_612_0, 202410_613_613_0, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0] 2024.10.22 01:49:12.465467 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 12.99 MiB, will set to 1.75 GiB (RSS), difference: 11.40 MiB 2024.10.22 01:49:12.468814 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_611_122, 202410_612_612_0, 202410_613_613_0, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0] 2024.10.22 01:49:12.576226 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72908 2024.10.22 01:49:12.607721 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:12.613462 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:49:12.780144 [ 1187 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:49:12.693722 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_536_536_0 (state Temporary) 2024.10.22 01:49:12.821216 [ 1187 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:49:12.865404 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ad1f98f9a9dca2aaca426d7de6b10c1b) : [0, 383216]: Setting download as failed: File segment: [0, 383216], key: ad1f98f9a9dca2aaca426d7de6b10c1b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:12.866681 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ad1f98f9a9dca2aaca426d7de6b10c1b): No space left in cache to reserve 383217 bytes, will continue without cache download 2024.10.22 01:49:12.913641 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 778 entries to flush up to offset 975277 2024.10.22 01:49:12.921418 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_536_536_0 to 202410_546_546_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:12.923191 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ad1f98f9a9dca2aaca426d7de6b10c1b, offset: 0, size: 0 2024.10.22 01:49:12.925183 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ad1f98f9a9dca2aaca426d7de6b10c1b for removal 2024.10.22 01:49:12.931997 [ 1476 ] {} CacheMetadata: Key ad1f98f9a9dca2aaca426d7de6b10c1b is removed from metadata 2024.10.22 01:49:12.935334 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e16c57a08d6d14bcf739032d4a1ebf2e, offset: 0, size: 0 2024.10.22 01:49:12.937135 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e16c57a08d6d14bcf739032d4a1ebf2e for removal 2024.10.22 01:49:12.954733 [ 1476 ] {} CacheMetadata: Key e16c57a08d6d14bcf739032d4a1ebf2e is removed from metadata 2024.10.22 01:49:13.028625 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 554 2024.10.22 01:49:13.311122 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10116 2024.10.22 01:49:13.439464 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_134_24/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:49:13.443129 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 105 entries to flush up to offset 74389 2024.10.22 01:49:13.564432 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:13.567606 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:49:13.583859 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:13.593507 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:13.721042 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_663_663_0 (state Temporary) 2024.10.22 01:49:13.801551 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:13.825212 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_663_663_0 to 202410_676_676_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:13.908359 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:13.948539 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_635_635_0 (state Temporary) 2024.10.22 01:49:13.955944 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_651_651_0 (state Temporary) 2024.10.22 01:49:14.072657 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_635_635_0 to 202410_642_642_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:14.140412 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 975277 2024.10.22 01:49:14.210253 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_651_651_0 to 202410_657_657_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:14.261135 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d2290ab348281db0f530727bf18ef445) : [0, 214]: Setting download as failed: File segment: [0, 214], key: d2290ab348281db0f530727bf18ef445, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:14.262833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d2290ab348281db0f530727bf18ef445): No space left in cache to reserve 215 bytes, will continue without cache download 2024.10.22 01:49:14.265639 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74389 2024.10.22 01:49:14.400427 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10116 2024.10.22 01:49:15.239420 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_634_123} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_634_123 to 202410_1_634_123 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:15.256949 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_634_123} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_628_122, 202410_629_634_1] -> 202410_1_634_123 2024.10.22 01:49:15.277117 [ 1395 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_634_123}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 01:49:15.290158 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:49:15.360646 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1bad2bbdaa7b758271d44b3b50f855dc) : [0, 258720]: Setting download as failed: File segment: [0, 258720], key: 1bad2bbdaa7b758271d44b3b50f855dc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:15.361783 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1bad2bbdaa7b758271d44b3b50f855dc): No space left in cache to reserve 258721 bytes, will continue without cache download 2024.10.22 01:49:15.603302 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1bad2bbdaa7b758271d44b3b50f855dc, offset: 0, size: 0 2024.10.22 01:49:15.607885 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1bad2bbdaa7b758271d44b3b50f855dc for removal 2024.10.22 01:49:15.617809 [ 1476 ] {} CacheMetadata: Key 1bad2bbdaa7b758271d44b3b50f855dc is removed from metadata 2024.10.22 01:49:15.621234 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d2290ab348281db0f530727bf18ef445, offset: 0, size: 0 2024.10.22 01:49:15.633288 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d2290ab348281db0f530727bf18ef445 for removal 2024.10.22 01:49:15.641006 [ 1476 ] {} CacheMetadata: Key d2290ab348281db0f530727bf18ef445 is removed from metadata 2024.10.22 01:49:16.692067 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 72970 2024.10.22 01:49:16.696438 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(9984c4471e336290b002d97b63e1089a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9984c4471e336290b002d97b63e1089a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:49:16.712103 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 651, file segment info: File segment: [0, 33554431], key: 9984c4471e336290b002d97b63e1089a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:49:16.718522 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:16.804708 [ 1688 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rom/rneezooasulspmlngmionskosiaze, size 1429468 2024.10.22 01:49:16.858155 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:16.860166 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:16.906818 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1120_1120_0 (state Temporary) 2024.10.22 01:49:16.959087 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1120_1120_0 to 202410_1133_1133_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:17.132068 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 72970 2024.10.22 01:49:17.534817 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:49:17.547092 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:49:17.579135 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:49:17.592946 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:49:17.669124 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(efc079438944a0a100bc73d63e5e67b1) : [0, 214]: Setting download as failed: File segment: [0, 214], key: efc079438944a0a100bc73d63e5e67b1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:17.670168 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(efc079438944a0a100bc73d63e5e67b1): No space left in cache to reserve 215 bytes, will continue without cache download 2024.10.22 01:49:17.720398 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:49:17.739247 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:49:17.757428 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:49:17.781962 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 01:49:17.824456 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lqy/zkksdmazvktdybmijbgjyzekxzdsx (LogSeriesLimiter: on interval from 2024-10-22 01:48:55 to 2024-10-22 01:49:17 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:49:18.574239 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c621b105e1bfe5d0d53a105f45d9218c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c621b105e1bfe5d0d53a105f45d9218c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:49:18.590635 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 874, file segment info: File segment: [0, 33554431], key: c621b105e1bfe5d0d53a105f45d9218c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:49:18.593819 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:19.371096 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 18.82 MiB, will set to 1.73 GiB (RSS), difference: 6.96 MiB 2024.10.22 01:49:19.663618 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9a2554771069ef7adfd5d61c56dc4158) : [0, 257550]: Setting download as failed: File segment: [0, 257550], key: 9a2554771069ef7adfd5d61c56dc4158, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:19.664334 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9a2554771069ef7adfd5d61c56dc4158): No space left in cache to reserve 257551 bytes, will continue without cache download 2024.10.22 01:49:19.783774 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9a2554771069ef7adfd5d61c56dc4158, offset: 0, size: 0 2024.10.22 01:49:19.786881 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9a2554771069ef7adfd5d61c56dc4158 for removal 2024.10.22 01:49:19.798985 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: efc079438944a0a100bc73d63e5e67b1, offset: 0, size: 0 2024.10.22 01:49:19.801452 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key efc079438944a0a100bc73d63e5e67b1 for removal 2024.10.22 01:49:19.898215 [ 1476 ] {} CacheMetadata: Key 9a2554771069ef7adfd5d61c56dc4158 is removed from metadata 2024.10.22 01:49:19.905383 [ 1476 ] {} CacheMetadata: Key efc079438944a0a100bc73d63e5e67b1 is removed from metadata 2024.10.22 01:49:20.357578 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(6c316594caa4b39be26eec4430300c88) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6c316594caa4b39be26eec4430300c88, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:49:20.364560 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 807764, file segment info: File segment: [0, 33554431], key: 6c316594caa4b39be26eec4430300c88, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:49:20.368376 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:20.817766 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:49:20.823430 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:49:21.145720 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 976432 2024.10.22 01:49:21.160241 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1021_496, 202410_1022_1027_1] 2024.10.22 01:49:21.455903 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 73008 2024.10.22 01:49:21.456916 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1021_496, 202410_1022_1027_1] 2024.10.22 01:49:21.532811 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1021_496, 202410_1022_1027_1] 2024.10.22 01:49:21.554332 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cd2063be6fe6e72f18500a565550d5b5) : [0, 214]: Setting download as failed: File segment: [0, 214], key: cd2063be6fe6e72f18500a565550d5b5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:21.556009 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cd2063be6fe6e72f18500a565550d5b5): No space left in cache to reserve 215 bytes, will continue without cache download 2024.10.22 01:49:21.582022 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:21.587666 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:21.815934 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 52 entries to flush up to offset 74441 2024.10.22 01:49:21.854611 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1121_1121_0 (state Temporary) 2024.10.22 01:49:21.903525 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 10123 2024.10.22 01:49:22.003898 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1121_1121_0 to 202410_1134_1134_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:22.071393 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73008 2024.10.22 01:49:22.120644 [ 1266 ] {} DatabaseCatalog: Execution took 236 ms. 2024.10.22 01:49:22.172873 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:22.175343 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:22.199857 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1028_1028_0, 202410_1029_1029_0, 202410_1030_1030_0, 202410_1031_1031_0, 202410_1032_1032_0, 202410_1033_1033_0] 2024.10.22 01:49:22.216925 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1028_1028_0, 202410_1029_1029_0, 202410_1030_1030_0, 202410_1031_1031_0, 202410_1032_1032_0, 202410_1033_1033_0] 2024.10.22 01:49:22.225732 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_652_652_0 (state Temporary) 2024.10.22 01:49:22.247300 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:22.251605 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:22.255128 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:22.259051 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:22.266441 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_652_652_0 to 202410_658_658_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:22.303826 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 5 parts from 202410_507_653_30 to 202410_657_657_0 2024.10.22 01:49:22.308793 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_664_664_0 (state Temporary) 2024.10.22 01:49:22.323492 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:22.329968 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74441 2024.10.22 01:49:22.351615 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1028_1028_0, 202410_1029_1029_0, 202410_1030_1030_0, 202410_1031_1031_0, 202410_1032_1032_0, 202410_1033_1033_0] 2024.10.22 01:49:22.358781 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:22.365984 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_664_664_0 to 202410_677_677_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:22.396268 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_581_581_0, 202410_582_582_0, 202410_583_583_0, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0] 2024.10.22 01:49:22.405385 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_581_581_0, 202410_582_582_0, 202410_583_583_0, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0] 2024.10.22 01:49:22.446897 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_581_581_0, 202410_582_582_0, 202410_583_583_0, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0] 2024.10.22 01:49:22.505206 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_657_31} MergeTask::PrepareStage: Merging 5 parts: from 202410_507_653_30 to 202410_657_657_0 into Compact with storage Full 2024.10.22 01:49:22.536321 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10123 2024.10.22 01:49:22.567553 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_657_31} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:49:22.570839 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_636_636_0 (state Temporary) 2024.10.22 01:49:22.660723 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:22.663829 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:22.665788 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_636_636_0 to 202410_643_643_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:22.698717 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_657_31} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_653_30, total 14876 rows starting from the beginning of the part 2024.10.22 01:49:22.795262 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_657_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_654_654_0, total 119 rows starting from the beginning of the part 2024.10.22 01:49:22.825173 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 976432 2024.10.22 01:49:22.862865 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7b0953db17f56bba0ab72eec1d8bafd1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7b0953db17f56bba0ab72eec1d8bafd1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:49:22.865160 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 934, file segment info: File segment: [0, 33554431], key: 7b0953db17f56bba0ab72eec1d8bafd1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:49:22.868564 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:22.984596 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_657_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_655_655_0, total 88 rows starting from the beginning of the part 2024.10.22 01:49:23.066731 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_657_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_656_656_0, total 76 rows starting from the beginning of the part 2024.10.22 01:49:23.261182 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_657_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_657_657_0, total 105 rows starting from the beginning of the part 2024.10.22 01:49:23.263954 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IslandID 2024.10.22 01:49:23.273817 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IslandID 2024.10.22 01:49:23.293346 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IslandID 2024.10.22 01:49:23.315038 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IslandID 2024.10.22 01:49:23.337084 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IslandID 2024.10.22 01:49:23.367594 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IslandID 2024.10.22 01:49:23.384129 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IslandID 2024.10.22 01:49:23.410550 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IslandID 2024.10.22 01:49:23.541412 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IslandID 2024.10.22 01:49:23.809793 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kcb/zvsyzrepgjbsoznffixqbvjenrujt (LogSeriesLimiter: on interval from 2024-10-22 01:49:17 to 2024-10-22 01:49:23 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:49:24.226327 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8edf043a1da09be9960c3c3eac11a981) : [0, 217033]: Setting download as failed: File segment: [0, 217033], key: 8edf043a1da09be9960c3c3eac11a981, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:24.228269 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8edf043a1da09be9960c3c3eac11a981): No space left in cache to reserve 217034 bytes, will continue without cache download 2024.10.22 01:49:24.368751 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8edf043a1da09be9960c3c3eac11a981, offset: 0, size: 0 2024.10.22 01:49:24.375619 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8edf043a1da09be9960c3c3eac11a981 for removal 2024.10.22 01:49:24.379238 [ 1476 ] {} CacheMetadata: Key 8edf043a1da09be9960c3c3eac11a981 is removed from metadata 2024.10.22 01:49:24.380010 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cd2063be6fe6e72f18500a565550d5b5, offset: 0, size: 0 2024.10.22 01:49:24.395996 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cd2063be6fe6e72f18500a565550d5b5 for removal 2024.10.22 01:49:24.401554 [ 1476 ] {} CacheMetadata: Key cd2063be6fe6e72f18500a565550d5b5 is removed from metadata 2024.10.22 01:49:25.027100 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_493_101, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0, 202410_497_497_0, 202410_498_498_0] 2024.10.22 01:49:25.030332 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_493_101, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0, 202410_497_497_0, 202410_498_498_0] 2024.10.22 01:49:25.104002 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_657_31} MergeTask::MergeProjectionsStage: Merge sorted 15264 rows, containing 13 columns (13 merged, 0 gathered) in 2.610084485 sec., 5848.086561075436 rows/sec., 2.53 MiB/sec. 2024.10.22 01:49:25.160768 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_493_101, 202410_494_494_0, 202410_495_495_0, 202410_496_496_0, 202410_497_497_0, 202410_498_498_0] 2024.10.22 01:49:25.202583 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_657_31} MergedBlockOutputStream: filled checksums 202410_507_657_31 (state Temporary) 2024.10.22 01:49:25.481485 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b5793b0f95ac9fb406acf070fcc06998) : [0, 220]: Setting download as failed: File segment: [0, 220], key: b5793b0f95ac9fb406acf070fcc06998, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:25.483467 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b5793b0f95ac9fb406acf070fcc06998): No space left in cache to reserve 221 bytes, will continue without cache download 2024.10.22 01:49:25.583360 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_657_31} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_657_31 to 202410_507_657_31 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:25.592893 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_657_31} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 5 parts: [202410_507_653_30, 202410_657_657_0] -> 202410_507_657_31 2024.10.22 01:49:25.746706 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.05 MiB. 2024.10.22 01:49:26.075125 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 73087 2024.10.22 01:49:26.089010 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_134_134_0 (state Temporary) 2024.10.22 01:49:26.452069 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dfddc876610d766a43f67330da747cd0) : [0, 419213]: Setting download as failed: File segment: [0, 419213], key: dfddc876610d766a43f67330da747cd0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:26.454875 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dfddc876610d766a43f67330da747cd0): No space left in cache to reserve 419214 bytes, will continue without cache download 2024.10.22 01:49:26.510941 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:26.515280 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:26.613920 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dfddc876610d766a43f67330da747cd0, offset: 0, size: 0 2024.10.22 01:49:26.617503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dfddc876610d766a43f67330da747cd0 for removal 2024.10.22 01:49:26.619380 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1122_1122_0 (state Temporary) 2024.10.22 01:49:26.622462 [ 1476 ] {} CacheMetadata: Key dfddc876610d766a43f67330da747cd0 is removed from metadata 2024.10.22 01:49:26.626645 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b5793b0f95ac9fb406acf070fcc06998, offset: 0, size: 0 2024.10.22 01:49:26.630998 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b5793b0f95ac9fb406acf070fcc06998 for removal 2024.10.22 01:49:26.639998 [ 1476 ] {} CacheMetadata: Key b5793b0f95ac9fb406acf070fcc06998 is removed from metadata 2024.10.22 01:49:26.644410 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 21.50 MiB, will set to 1.73 GiB (RSS), difference: -12.60 MiB 2024.10.22 01:49:26.719052 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1122_1122_0 to 202410_1135_1135_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:26.803911 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73087 2024.10.22 01:49:27.841127 [ 1271 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:49:27.872191 [ 1271 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:49:27.948909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fe4416807b049de92844c29b4ac4f8b0) : [0, 216]: Setting download as failed: File segment: [0, 216], key: fe4416807b049de92844c29b4ac4f8b0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:27.951639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fe4416807b049de92844c29b4ac4f8b0): No space left in cache to reserve 217 bytes, will continue without cache download 2024.10.22 01:49:28.032519 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 555 2024.10.22 01:49:28.304068 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:28.306761 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:28.310375 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 16.09 MiB, will set to 1.73 GiB (RSS), difference: 1.91 MiB 2024.10.22 01:49:28.383837 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_537_537_0 (state Temporary) 2024.10.22 01:49:28.581613 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_537_537_0 to 202410_547_547_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:28.866967 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 555 2024.10.22 01:49:29.115986 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bb0c44f2a980fcc8c6c3eb9ed15928ce) : [0, 302754]: Setting download as failed: File segment: [0, 302754], key: bb0c44f2a980fcc8c6c3eb9ed15928ce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:29.118345 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bb0c44f2a980fcc8c6c3eb9ed15928ce): No space left in cache to reserve 302755 bytes, will continue without cache download 2024.10.22 01:49:29.180066 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bb0c44f2a980fcc8c6c3eb9ed15928ce, offset: 0, size: 0 2024.10.22 01:49:29.182684 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bb0c44f2a980fcc8c6c3eb9ed15928ce for removal 2024.10.22 01:49:29.190525 [ 1476 ] {} CacheMetadata: Key bb0c44f2a980fcc8c6c3eb9ed15928ce is removed from metadata 2024.10.22 01:49:29.200455 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fe4416807b049de92844c29b4ac4f8b0, offset: 0, size: 0 2024.10.22 01:49:29.204750 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fe4416807b049de92844c29b4ac4f8b0 for removal 2024.10.22 01:49:29.208394 [ 1476 ] {} CacheMetadata: Key fe4416807b049de92844c29b4ac4f8b0 is removed from metadata 2024.10.22 01:49:29.844626 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 977587 2024.10.22 01:49:29.846706 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 65 entries to flush up to offset 74506 2024.10.22 01:49:29.973606 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:29.979455 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:30.029751 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_653_653_0 (state Temporary) 2024.10.22 01:49:30.038686 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 10135 2024.10.22 01:49:30.067550 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_653_653_0 to 202410_659_659_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:30.180189 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74506 2024.10.22 01:49:30.255493 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:30.257442 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:30.299598 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e55112a4e013f8ffffc873244cbd0485) : [0, 154]: Setting download as failed: File segment: [0, 154], key: e55112a4e013f8ffffc873244cbd0485, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:30.301229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e55112a4e013f8ffffc873244cbd0485): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 01:49:30.394778 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_665_665_0 (state Temporary) 2024.10.22 01:49:30.405456 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:30.412873 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:30.554332 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_665_665_0 to 202410_678_678_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:30.569731 [ 1194 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_673_135 to 202410_678_678_0 2024.10.22 01:49:30.575821 [ 1194 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:30.584364 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10135 2024.10.22 01:49:30.577826 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:30.596399 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 12.01 MiB, will set to 1.72 GiB (RSS), difference: -2.60 MiB 2024.10.22 01:49:30.752627 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_637_637_0 (state Temporary) 2024.10.22 01:49:30.752675 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_673_135 to 202410_678_678_0 into Compact with storage Full 2024.10.22 01:49:30.813983 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:49:30.822380 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:49:30.859138 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 73145 2024.10.22 01:49:30.892232 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:49:30.909558 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_673_135, total 10178 rows starting from the beginning of the part 2024.10.22 01:49:30.914863 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_637_637_0 to 202410_644_644_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:31.052813 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_674_674_0, total 21 rows starting from the beginning of the part 2024.10.22 01:49:31.060306 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 977587 2024.10.22 01:49:31.051521 [ 1238 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_634_123 to 202410_635_640_1 2024.10.22 01:49:31.066339 [ 1238 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:31.068152 [ 1238 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:31.078837 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_675_675_0, total 10 rows starting from the beginning of the part 2024.10.22 01:49:31.168362 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_640_124} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_634_123 to 202410_635_640_1 into Wide with storage Full 2024.10.22 01:49:31.202392 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(28f8d6975bfb5a93a3f63ee6a7586ab8) : [0, 2271]: Setting download as failed: File segment: [0, 2271], key: 28f8d6975bfb5a93a3f63ee6a7586ab8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:31.204350 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(28f8d6975bfb5a93a3f63ee6a7586ab8): No space left in cache to reserve 2272 bytes, will continue without cache download 2024.10.22 01:49:31.238917 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:31.243739 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:31.265985 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_640_124} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:49:31.250402 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 28f8d6975bfb5a93a3f63ee6a7586ab8, offset: 0, size: 0 2024.10.22 01:49:31.269138 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 28f8d6975bfb5a93a3f63ee6a7586ab8 for removal 2024.10.22 01:49:31.271357 [ 1476 ] {} CacheMetadata: Key 28f8d6975bfb5a93a3f63ee6a7586ab8 is removed from metadata 2024.10.22 01:49:31.272098 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_640_124} MergeTreeSequentialSource: Reading 121 marks from part 202410_1_634_123, total 980391 rows starting from the beginning of the part 2024.10.22 01:49:31.272954 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e55112a4e013f8ffffc873244cbd0485, offset: 0, size: 0 2024.10.22 01:49:31.274559 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e55112a4e013f8ffffc873244cbd0485 for removal 2024.10.22 01:49:31.276030 [ 1476 ] {} CacheMetadata: Key e55112a4e013f8ffffc873244cbd0485 is removed from metadata 2024.10.22 01:49:31.306073 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_640_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_635_640_1, total 6162 rows starting from the beginning of the part 2024.10.22 01:49:31.310072 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_676_676_0, total 13 rows starting from the beginning of the part 2024.10.22 01:49:31.352358 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_677_677_0, total 7 rows starting from the beginning of the part 2024.10.22 01:49:31.416719 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1123_1123_0 (state Temporary) 2024.10.22 01:49:31.427029 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_678_678_0, total 12 rows starting from the beginning of the part 2024.10.22 01:49:31.484400 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1123_1123_0 to 202410_1136_1136_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:31.632071 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1131_7 to 202410_1136_1136_0 2024.10.22 01:49:31.658328 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:31.661188 [ 1179 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:31.815771 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73145 2024.10.22 01:49:31.811263 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1131_7 to 202410_1136_1136_0 into Compact with storage Full 2024.10.22 01:49:32.101730 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:49:32.111274 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1131_7, total 2277 rows starting from the beginning of the part 2024.10.22 01:49:32.260264 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1132_1132_0, total 55 rows starting from the beginning of the part 2024.10.22 01:49:32.435081 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9147f2939cd1a2ab20a7a1a7badea2a5) : [0, 238]: Setting download as failed: File segment: [0, 238], key: 9147f2939cd1a2ab20a7a1a7badea2a5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:32.438784 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1133_1133_0, total 62 rows starting from the beginning of the part 2024.10.22 01:49:32.439446 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9147f2939cd1a2ab20a7a1a7badea2a5): No space left in cache to reserve 239 bytes, will continue without cache download 2024.10.22 01:49:32.470653 [ 1164 ] {} DatabaseCatalog: Execution took 349 ms. 2024.10.22 01:49:32.585681 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1134_1134_0, total 38 rows starting from the beginning of the part 2024.10.22 01:49:32.535140 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 10.26 MiB, will set to 1.72 GiB (RSS), difference: -21.03 MiB 2024.10.22 01:49:32.777824 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_134_24/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:49:32.798636 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1135_1135_0, total 79 rows starting from the beginning of the part 2024.10.22 01:49:32.968627 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1136_1136_0, total 58 rows starting from the beginning of the part 2024.10.22 01:49:33.409199 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1638bf9205956f3c253d6665ee590e43) : [0, 1625226]: Setting download as failed: File segment: [0, 1625226], key: 1638bf9205956f3c253d6665ee590e43, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:33.411084 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1638bf9205956f3c253d6665ee590e43): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:49:33.507726 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1638bf9205956f3c253d6665ee590e43, offset: 0, size: 0 2024.10.22 01:49:33.509771 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1638bf9205956f3c253d6665ee590e43 for removal 2024.10.22 01:49:33.514549 [ 1476 ] {} CacheMetadata: Key 1638bf9205956f3c253d6665ee590e43 is removed from metadata 2024.10.22 01:49:33.521210 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9147f2939cd1a2ab20a7a1a7badea2a5, offset: 0, size: 0 2024.10.22 01:49:33.527514 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9147f2939cd1a2ab20a7a1a7badea2a5 for removal 2024.10.22 01:49:33.633037 [ 1476 ] {} CacheMetadata: Key 9147f2939cd1a2ab20a7a1a7badea2a5 is removed from metadata 2024.10.22 01:49:33.853077 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_134_134_0 to 202410_137_137_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:33.899170 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} MergeTask::MergeProjectionsStage: Merge sorted 10241 rows, containing 12 columns (12 merged, 0 gathered) in 3.154937268 sec., 3246.0233374123623 rows/sec., 516.65 KiB/sec. 2024.10.22 01:49:33.963947 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} MergedBlockOutputStream: filled checksums 202410_1_678_136 (state Temporary) 2024.10.22 01:49:34.044271 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a549e2adff3d88c9ce5d12be3916aa62) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a549e2adff3d88c9ce5d12be3916aa62, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:49:34.135352 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 385488, file segment info: File segment: [0, 33554431], key: a549e2adff3d88c9ce5d12be3916aa62, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:49:34.146877 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:34.159587 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_574_113, 202410_575_580_1] 2024.10.22 01:49:34.163086 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_574_113, 202410_575_580_1] 2024.10.22 01:49:34.189447 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_574_113, 202410_575_580_1] 2024.10.22 01:49:34.469403 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_678_136 to 202410_1_678_136 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:34.473684 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_678_136} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_673_135, 202410_678_678_0] -> 202410_1_678_136 2024.10.22 01:49:34.542619 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.57 MiB. 2024.10.22 01:49:34.571243 [ 1175 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 01:49:34.760235 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c1e98e6e1ea9afa9f939e99b7f35967b) : [0, 244]: Setting download as failed: File segment: [0, 244], key: c1e98e6e1ea9afa9f939e99b7f35967b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:34.761566 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c1e98e6e1ea9afa9f939e99b7f35967b): No space left in cache to reserve 245 bytes, will continue without cache download 2024.10.22 01:49:35.018911 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} MergeTask::MergeProjectionsStage: Merge sorted 2569 rows, containing 13 columns (13 merged, 0 gathered) in 3.319500832 sec., 773.911539721524 rows/sec., 139.84 KiB/sec. 2024.10.22 01:49:35.086682 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} MergedBlockOutputStream: filled checksums 202410_1096_1136_8 (state Temporary) 2024.10.22 01:49:35.198696 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1136_8 to 202410_1096_1136_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:35.205589 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1136_8} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1131_7, 202410_1136_1136_0] -> 202410_1096_1136_8 2024.10.22 01:49:35.253136 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:49:35.467556 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_640_124/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:49:35.826950 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 73222 2024.10.22 01:49:35.863945 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(757dc188f642d49a77f52d3b8de8ede4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 757dc188f642d49a77f52d3b8de8ede4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:49:35.865124 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 627, file segment info: File segment: [0, 33554431], key: 757dc188f642d49a77f52d3b8de8ede4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:49:35.878111 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:35.913715 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 20.18 MiB, will set to 1.73 GiB (RSS), difference: 22.47 MiB 2024.10.22 01:49:35.960978 [ 1677 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lqy/zkksdmazvktdybmijbgjyzekxzdsx, size 385488 2024.10.22 01:49:36.165772 [ 1234 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 01:49:36.180725 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:49:36.211891 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:49:36.242035 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:49:36.254654 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:49:36.265221 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:49:36.305462 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:49:36.435902 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:49:36.448430 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 01:49:36.451285 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:36.455277 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:36.470681 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 556 2024.10.22 01:49:36.587794 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/oqp/wiexgnifypybamshjrrdpfbefjzpt (LogSeriesLimiter: on interval from 2024-10-22 01:49:23 to 2024-10-22 01:49:36 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:49:36.620696 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1124_1124_0 (state Temporary) 2024.10.22 01:49:36.779154 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1124_1124_0 to 202410_1137_1137_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:37.037373 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73222 2024.10.22 01:49:37.109529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f062804d22653d0c5888fa58f9a0995b) : [0, 2015783]: Setting download as failed: File segment: [0, 2015783], key: f062804d22653d0c5888fa58f9a0995b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:37.111558 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f062804d22653d0c5888fa58f9a0995b): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:49:37.257091 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:37.265393 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:37.456275 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_538_538_0 (state Temporary) 2024.10.22 01:49:37.545075 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f062804d22653d0c5888fa58f9a0995b, offset: 0, size: 0 2024.10.22 01:49:37.549640 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f062804d22653d0c5888fa58f9a0995b for removal 2024.10.22 01:49:37.554391 [ 1476 ] {} CacheMetadata: Key f062804d22653d0c5888fa58f9a0995b is removed from metadata 2024.10.22 01:49:37.559603 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c1e98e6e1ea9afa9f939e99b7f35967b, offset: 0, size: 0 2024.10.22 01:49:37.561319 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c1e98e6e1ea9afa9f939e99b7f35967b for removal 2024.10.22 01:49:37.563697 [ 1476 ] {} CacheMetadata: Key c1e98e6e1ea9afa9f939e99b7f35967b is removed from metadata 2024.10.22 01:49:37.639934 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_538_538_0 to 202410_548_548_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:37.665147 [ 1208 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_543_111 to 202410_548_548_0 2024.10.22 01:49:37.684960 [ 1208 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:37.692774 [ 1208 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:37.693392 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 74621 2024.10.22 01:49:37.703449 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 556 2024.10.22 01:49:37.769228 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_543_111 to 202410_548_548_0 into Compact with storage Full 2024.10.22 01:49:37.842802 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:37.845676 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:37.873433 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:49:37.895409 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_543_111, total 587 rows starting from the beginning of the part 2024.10.22 01:49:37.899383 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_654_654_0 (state Temporary) 2024.10.22 01:49:37.976672 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_654_654_0 to 202410_660_660_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:38.065692 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1157 entries to flush up to offset 978744 2024.10.22 01:49:38.068128 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5363 2024.10.22 01:49:38.086886 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 10157 2024.10.22 01:49:38.153667 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74621 2024.10.22 01:49:38.161137 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_544_544_0, total 1 rows starting from the beginning of the part 2024.10.22 01:49:38.268501 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:38.272651 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:38.304836 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_545_545_0, total 1 rows starting from the beginning of the part 2024.10.22 01:49:38.333791 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_666_666_0 (state Temporary) 2024.10.22 01:49:38.375050 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_546_546_0, total 1 rows starting from the beginning of the part 2024.10.22 01:49:38.386757 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_666_666_0 to 202410_679_679_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:38.439291 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_547_547_0, total 1 rows starting from the beginning of the part 2024.10.22 01:49:38.474896 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 12.93 MiB, will set to 1.72 GiB (RSS), difference: -7.02 MiB 2024.10.22 01:49:38.494058 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_548_548_0, total 1 rows starting from the beginning of the part 2024.10.22 01:49:38.552890 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10157 2024.10.22 01:49:38.639983 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3ce78f9740ad24aae52146cda546cd63) : [0, 229]: Setting download as failed: File segment: [0, 229], key: 3ce78f9740ad24aae52146cda546cd63, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:38.643385 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3ce78f9740ad24aae52146cda546cd63): No space left in cache to reserve 230 bytes, will continue without cache download 2024.10.22 01:49:38.833255 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:38.839454 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:38.941365 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_638_638_0 (state Temporary) 2024.10.22 01:49:39.104732 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_638_638_0 to 202410_645_645_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:39.200490 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 978744 2024.10.22 01:49:39.684898 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_596_18, 202410_597_597_0, 202410_598_598_0, 202410_599_599_0, 202410_600_600_0, 202410_601_601_0] 2024.10.22 01:49:39.694802 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_596_18, 202410_597_597_0, 202410_598_598_0, 202410_599_599_0, 202410_600_600_0, 202410_601_601_0] 2024.10.22 01:49:39.749230 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_596_18, 202410_597_597_0, 202410_598_598_0, 202410_599_599_0, 202410_600_600_0, 202410_601_601_0] 2024.10.22 01:49:39.779799 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(34efa7964d2aef7c7af778020c7569d2) : [0, 981587]: Setting download as failed: File segment: [0, 981587], key: 34efa7964d2aef7c7af778020c7569d2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:39.781720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(34efa7964d2aef7c7af778020c7569d2): No space left in cache to reserve 981588 bytes, will continue without cache download 2024.10.22 01:49:39.948966 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 34efa7964d2aef7c7af778020c7569d2, offset: 0, size: 0 2024.10.22 01:49:39.952832 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 34efa7964d2aef7c7af778020c7569d2 for removal 2024.10.22 01:49:39.957194 [ 1476 ] {} CacheMetadata: Key 34efa7964d2aef7c7af778020c7569d2 is removed from metadata 2024.10.22 01:49:39.959564 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3ce78f9740ad24aae52146cda546cd63, offset: 0, size: 0 2024.10.22 01:49:39.960432 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3ce78f9740ad24aae52146cda546cd63 for removal 2024.10.22 01:49:39.964285 [ 1476 ] {} CacheMetadata: Key 3ce78f9740ad24aae52146cda546cd63 is removed from metadata 2024.10.22 01:49:40.312575 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} MergeTask::MergeProjectionsStage: Merge sorted 592 rows, containing 32 columns (32 merged, 0 gathered) in 2.548393167 sec., 232.30324412496748 rows/sec., 21.25 KiB/sec. 2024.10.22 01:49:40.372070 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} MergedBlockOutputStream: filled checksums 202410_1_548_112 (state Temporary) 2024.10.22 01:49:40.816636 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:49:40.822610 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:49:40.839739 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 15.32 MiB, will set to 1.72 GiB (RSS), difference: 1.81 MiB 2024.10.22 01:49:40.943801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4bf61f14f770167cbb68486238896264) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 4bf61f14f770167cbb68486238896264, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:40.947439 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4bf61f14f770167cbb68486238896264): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 01:49:41.048092 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 73308 2024.10.22 01:49:41.051408 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_548_112 to 202410_1_548_112 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:41.055341 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_548_112} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_543_111, 202410_548_548_0] -> 202410_1_548_112 2024.10.22 01:49:41.131207 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:49:41.311551 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:41.317077 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:41.391994 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1125_1125_0 (state Temporary) 2024.10.22 01:49:41.665182 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1125_1125_0 to 202410_1138_1138_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:41.839699 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73308 2024.10.22 01:49:42.215782 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(28ed84fdc128096e88022aa787271f6b) : [0, 160950]: Setting download as failed: File segment: [0, 160950], key: 28ed84fdc128096e88022aa787271f6b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:49:42.217140 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(28ed84fdc128096e88022aa787271f6b): No space left in cache to reserve 160951 bytes, will continue without cache download 2024.10.22 01:49:42.267431 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 28ed84fdc128096e88022aa787271f6b, offset: 0, size: 0 2024.10.22 01:49:42.272203 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 28ed84fdc128096e88022aa787271f6b for removal 2024.10.22 01:49:42.376957 [ 1476 ] {} CacheMetadata: Key 28ed84fdc128096e88022aa787271f6b is removed from metadata 2024.10.22 01:49:42.377627 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4bf61f14f770167cbb68486238896264, offset: 0, size: 0 2024.10.22 01:49:42.380221 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4bf61f14f770167cbb68486238896264 for removal 2024.10.22 01:49:42.382456 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nit/cxccobazsvtnymxnhrzeojpgqpwxb. 2024.10.22 01:49:42.383909 [ 1476 ] {} CacheMetadata: Key 4bf61f14f770167cbb68486238896264 is removed from metadata 2024.10.22 01:49:42.472442 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 15.52 MiB, will set to 1.72 GiB (RSS), difference: 3.66 MiB 2024.10.22 01:49:42.536494 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/cql/reswptjtwlshzznxfwxflwioyinhl. 2024.10.22 01:49:42.876771 [ 1262 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:49:43.023712 [ 1262 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:49:43.092011 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/cql/reswptjtwlshzznxfwxflwioyinhl. 2024.10.22 01:49:43.098955 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_7_7_0 to 201403_7_7_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:43.133503 [ 1262 ] {} DNSCacheUpdater: Execution took 255 ms. 2024.10.22 01:49:43.217170 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nit/cxccobazsvtnymxnhrzeojpgqpwxb. 2024.10.22 01:49:43.331509 [ 1260 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4) (MergerMutator): Selected 7 parts from 201403_1_1_0 to 201403_7_7_0 2024.10.22 01:49:43.344414 [ 1260 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 437.23 MiB using storage policy from min volume index 0 2024.10.22 01:49:43.383440 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTask::PrepareStage: Merging 7 parts: from 201403_1_1_0 to 201403_7_7_0 into Wide with storage Full 2024.10.22 01:49:43.420897 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_8_8_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:49:44.007766 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Vertical 2024.10.22 01:49:44.043260 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part 2024.10.22 01:49:44.074511 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part 2024.10.22 01:49:44.127890 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part 2024.10.22 01:49:44.200597 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part 2024.10.22 01:49:44.252991 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part 2024.10.22 01:49:44.365436 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part 2024.10.22 01:49:44.455382 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part 2024.10.22 01:49:44.498424 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} SortDescription: Compile sort description UInt32 1 1Date 1 1UInt32 1 1 2024.10.22 01:49:44.771417 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_134_24/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:49:45.070655 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(578fc983dd02ab04e81ded808383c7b1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 578fc983dd02ab04e81ded808383c7b1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:49:45.081136 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 230670, file segment info: File segment: [0, 33554431], key: 578fc983dd02ab04e81ded808383c7b1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:49:45.085077 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:45.184330 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 1.97 GiB, free memory in arenas 15.75 MiB, will set to 1.50 GiB (RSS), difference: 341.29 MiB 2024.10.22 01:49:45.219200 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 557 2024.10.22 01:49:45.244254 [ 1218 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 01:49:45.489317 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:45.497005 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:45.637355 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 5404 2024.10.22 01:49:45.659773 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_539_539_0 (state Temporary) 2024.10.22 01:49:45.665783 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 74 entries to flush up to offset 74695 2024.10.22 01:49:45.832742 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_539_539_0 to 202410_549_549_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:45.874242 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 73360 2024.10.22 01:49:45.879074 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_640_124/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:49:45.933805 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(44d79e42a2d008a9048f8ddf8d64c292) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 44d79e42a2d008a9048f8ddf8d64c292, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:49:45.942800 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 919, file segment info: File segment: [0, 33554431], key: 44d79e42a2d008a9048f8ddf8d64c292, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 01:49:45.947732 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:45.974831 [ 1698 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kcb/zvsyzrepgjbsoznffixqbvjenrujt, size 230670 2024.10.22 01:49:45.993132 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:46.000061 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:46.065297 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 10169 2024.10.22 01:49:46.147036 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_655_655_0 (state Temporary) 2024.10.22 01:49:46.180478 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_655_655_0 to 202410_661_661_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:46.205260 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 979899 2024.10.22 01:49:46.248645 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74695 2024.10.22 01:49:46.257940 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:49:46.272939 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:49:46.295765 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:49:46.297918 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 557 2024.10.22 01:49:46.316835 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:46.318922 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:46.321339 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:49:46.348920 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:46.353997 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:46.353907 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:49:46.359837 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_667_667_0 (state Temporary) 2024.10.22 01:49:46.380708 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:49:46.464348 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1126_1126_0 (state Temporary) 2024.10.22 01:49:46.502140 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 39 2024.10.22 01:49:46.503987 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:49:46.550848 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:49:46.564650 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_667_667_0 to 202410_680_680_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:46.569565 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RequestNum 2024.10.22 01:49:46.613539 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10169 2024.10.22 01:49:46.619887 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1126_1126_0 to 202410_1139_1139_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:46.649279 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xin/wsdldbbdqwilmbchhuerajnxhwfvf (LogSeriesLimiter: on interval from 2024-10-22 01:49:36 to 2024-10-22 01:49:46 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:49:46.692916 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73360 2024.10.22 01:49:46.769703 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:46.771573 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:46.928597 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_33_33_0 (state Temporary) 2024.10.22 01:49:47.159981 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:47.164405 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:47.227677 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_33_33_0 to 202410_33_33_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:47.348830 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_639_639_0 (state Temporary) 2024.10.22 01:49:47.453464 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 39 2024.10.22 01:49:47.541684 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_639_639_0 to 202410_646_646_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:47.575367 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 979899 2024.10.22 01:49:47.669392 [ 1271 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_641_641_0 to 202410_646_646_0 2024.10.22 01:49:47.686351 [ 1271 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:47.693300 [ 1271 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:47.834724 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_641_641_0 to 202410_646_646_0 into Compact with storage Full 2024.10.22 01:49:47.840284 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.97 GiB, free memory in arenas 12.40 MiB, will set to 1.52 GiB (RSS), difference: -105.07 MiB 2024.10.22 01:49:47.856964 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:49:47.865102 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_641_641_0, total 1536 rows starting from the beginning of the part 2024.10.22 01:49:47.874835 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_642_642_0, total 778 rows starting from the beginning of the part 2024.10.22 01:49:47.908742 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_643_643_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:49:47.980207 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1027_497, 202410_1028_1033_1] 2024.10.22 01:49:47.985167 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1027_497, 202410_1028_1033_1] 2024.10.22 01:49:48.047763 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_644_644_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:49:48.167454 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_645_645_0, total 1157 rows starting from the beginning of the part 2024.10.22 01:49:48.306992 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_646_646_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:49:48.390363 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1027_497, 202410_1028_1033_1] 2024.10.22 01:49:49.054653 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1034_1034_0, 202410_1035_1035_0, 202410_1036_1036_0, 202410_1037_1037_0, 202410_1038_1038_0, 202410_1039_1039_0] 2024.10.22 01:49:49.059555 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1034_1034_0, 202410_1035_1035_0, 202410_1036_1036_0, 202410_1037_1037_0, 202410_1038_1038_0, 202410_1039_1039_0] 2024.10.22 01:49:49.121148 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1034_1034_0, 202410_1035_1035_0, 202410_1036_1036_0, 202410_1037_1037_0, 202410_1038_1038_0, 202410_1039_1039_0] 2024.10.22 01:49:49.574296 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 1.97 GiB, free memory in arenas 12.86 MiB, will set to 1.56 GiB (RSS), difference: -7.15 MiB 2024.10.22 01:49:49.832659 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} MergeTask::MergeProjectionsStage: Merge sorted 6936 rows, containing 4 columns (4 merged, 0 gathered) in 2.109414367 sec., 3288.1164120752387 rows/sec., 51.38 KiB/sec. 2024.10.22 01:49:49.974912 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} MergedBlockOutputStream: filled checksums 202410_641_646_1 (state Temporary) 2024.10.22 01:49:50.105562 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_641_646_1 to 202410_641_646_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:50.108389 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_641_646_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_641_641_0, 202410_646_646_0] -> 202410_641_646_1 2024.10.22 01:49:50.211011 [ 1277 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 01:49:50.267775 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:49:50.784337 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 73430 2024.10.22 01:49:50.819756 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:49:50.822344 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:49:51.054971 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:51.059526 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:51.236504 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(20934e241bcc0d7681213b7d9d36c603) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 20934e241bcc0d7681213b7d9d36c603, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 01:49:51.240798 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 20934e241bcc0d7681213b7d9d36c603, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 01:49:51.246662 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:51.326569 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1127_1127_0 (state Temporary) 2024.10.22 01:49:51.466372 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1127_1127_0 to 202410_1140_1140_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:51.541194 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73430 2024.10.22 01:49:51.580313 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 12.73 MiB, will set to 1.62 GiB (RSS), difference: -74.00 MiB 2024.10.22 01:49:52.322348 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(90ba9199657691b535f1a0b52e1f0cb4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 90ba9199657691b535f1a0b52e1f0cb4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:49:52.331772 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 705717, file segment info: File segment: [0, 33554431], key: 90ba9199657691b535f1a0b52e1f0cb4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:49:52.335815 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:52.474326 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_616_123, 202410_617_617_0, 202410_618_618_0, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0] 2024.10.22 01:49:52.476189 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_616_123, 202410_617_617_0, 202410_618_618_0, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0] 2024.10.22 01:49:52.514757 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_616_123, 202410_617_617_0, 202410_618_618_0, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0] 2024.10.22 01:49:53.071001 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(9dd9df2b7a3efe738acf48bdfc3209a7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9dd9df2b7a3efe738acf48bdfc3209a7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:49:53.072891 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 632, file segment info: File segment: [0, 33554431], key: 9dd9df2b7a3efe738acf48bdfc3209a7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:49:53.077382 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:49:53.309279 [ 1710 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/oqp/wiexgnifypybamshjrrdpfbefjzpt, size 705717 2024.10.22 01:49:53.336807 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} MergeTask::MergeProjectionsStage: Merge sorted 5298 rows, containing 679 columns (679 merged, 0 gathered) in 143.277597816 sec., 36.9771693604453 rows/sec., 49.69 KiB/sec. 2024.10.22 01:49:53.649545 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:49:53.771320 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 87 entries to flush up to offset 74782 2024.10.22 01:49:53.801704 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 558 2024.10.22 01:49:53.904979 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:49:53.927161 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:49:53.968844 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:49:54.016750 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:54.020077 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:54.040258 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:49:54.060304 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:54.064140 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:54.117378 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10182 2024.10.22 01:49:54.193607 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vae/iqrzvwmbuzgmfinolhtgmswoexrgz (LogSeriesLimiter: on interval from 2024-10-22 01:49:46 to 2024-10-22 01:49:54 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:49:54.227165 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_540_540_0 (state Temporary) 2024.10.22 01:49:54.260405 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:54.264176 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:54.328618 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_656_656_0 (state Temporary) 2024.10.22 01:49:54.351358 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:49:54.395540 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:49:54.419861 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_668_668_0 (state Temporary) 2024.10.22 01:49:54.441672 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 01:49:54.481411 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_668_668_0 to 202410_681_681_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:54.523849 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_540_540_0 to 202410_550_550_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:54.557441 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_656_656_0 to 202410_662_662_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:54.579421 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 981439 2024.10.22 01:49:54.665038 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10182 2024.10.22 01:49:54.695427 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74782 2024.10.22 01:49:54.646715 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_657_31 to 202410_662_662_0 2024.10.22 01:49:54.772928 [ 1176 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:54.776913 [ 1176 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:54.794671 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 558 2024.10.22 01:49:54.800757 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 208 ms. 2024.10.22 01:49:54.892341 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_657_31 to 202410_662_662_0 into Compact with storage Full 2024.10.22 01:49:55.003170 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.80 GiB, peak 1.97 GiB, free memory in arenas 13.05 MiB, will set to 1.70 GiB (RSS), difference: -93.46 MiB 2024.10.22 01:49:55.104378 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:49:55.118150 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_657_31, total 15264 rows starting from the beginning of the part 2024.10.22 01:49:55.173218 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_658_658_0, total 52 rows starting from the beginning of the part 2024.10.22 01:49:55.232974 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_659_659_0, total 65 rows starting from the beginning of the part 2024.10.22 01:49:55.359072 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_660_660_0, total 115 rows starting from the beginning of the part 2024.10.22 01:49:55.375757 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_134_24/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:49:55.564587 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 73493 2024.10.22 01:49:55.645274 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_661_661_0, total 74 rows starting from the beginning of the part 2024.10.22 01:49:55.712779 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_662_662_0, total 87 rows starting from the beginning of the part 2024.10.22 01:49:55.985753 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:55.991841 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:56.207294 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1128_1128_0 (state Temporary) 2024.10.22 01:49:56.258712 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:56.261387 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:56.310053 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1128_1128_0 to 202410_1141_1141_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:56.567439 [ 1207 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1136_8 to 202410_1141_1141_0 2024.10.22 01:49:56.567622 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73493 2024.10.22 01:49:56.575591 [ 1207 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:49:56.579394 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:49:56.590476 [ 1207 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 310 ms. 2024.10.22 01:49:56.649488 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1136_8 to 202410_1141_1141_0 into Compact with storage Full 2024.10.22 01:49:56.800367 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:49:56.836850 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1136_8, total 2569 rows starting from the beginning of the part 2024.10.22 01:49:56.857006 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_640_640_0 (state Temporary) 2024.10.22 01:49:56.989989 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1137_1137_0, total 77 rows starting from the beginning of the part 2024.10.22 01:49:57.057757 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1138_1138_0, total 86 rows starting from the beginning of the part 2024.10.22 01:49:57.117239 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1139_1139_0, total 52 rows starting from the beginning of the part 2024.10.22 01:49:57.161449 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 261.76 MiB using storage policy from min volume index 0 2024.10.22 01:49:57.168571 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_640_640_0 to 202410_647_647_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:57.267240 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1140_1140_0, total 70 rows starting from the beginning of the part 2024.10.22 01:49:57.275197 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} MergedBlockOutputStream: filled checksums 202410_1_134_24 (state Temporary) 2024.10.22 01:49:57.325731 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 981439 2024.10.22 01:49:57.344732 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1141_1141_0, total 63 rows starting from the beginning of the part 2024.10.22 01:49:57.470779 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_640_124/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:49:57.663529 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.97 GiB, free memory in arenas 13.13 MiB, will set to 1.75 GiB (RSS), difference: -39.79 MiB 2024.10.22 01:49:57.985084 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_8_8_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:49:58.003131 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} MergeTask::MergeProjectionsStage: Merge sorted 15657 rows, containing 13 columns (13 merged, 0 gathered) in 3.198969945 sec., 4894.3879652486075 rows/sec., 2.12 MiB/sec. 2024.10.22 01:49:58.037299 [ 1211 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:49:58.091256 [ 1211 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:49:58.361574 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} MergedBlockOutputStream: filled checksums 202410_507_662_32 (state Temporary) 2024.10.22 01:49:58.605255 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_662_32 to 202410_507_662_32 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:58.607944 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_662_32} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_657_31, 202410_662_662_0] -> 202410_507_662_32 2024.10.22 01:49:58.662661 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.08 MiB. 2024.10.22 01:49:58.970550 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} MergeTask::MergeProjectionsStage: Merge sorted 2917 rows, containing 13 columns (13 merged, 0 gathered) in 2.364925916 sec., 1233.442443276942 rows/sec., 222.71 KiB/sec. 2024.10.22 01:49:59.009731 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rde/xhxafbpndakfuqxycfjroqcrnydje (LogSeriesLimiter: on interval from 2024-10-22 01:49:54 to 2024-10-22 01:49:59 accepted series 1 / 48 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:49:59.052912 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} MergedBlockOutputStream: filled checksums 202410_1096_1141_9 (state Temporary) 2024.10.22 01:49:59.252846 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1141_9 to 202410_1096_1141_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:49:59.257346 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1141_9} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1136_8, 202410_1141_1141_0] -> 202410_1096_1141_9 2024.10.22 01:49:59.314253 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:49:59.434507 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.83 GiB, peak 1.97 GiB, free memory in arenas 12.86 MiB, will set to 1.74 GiB (RSS), difference: -84.76 MiB 2024.10.22 01:50:00.682689 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 73537 2024.10.22 01:50:00.817677 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:50:00.821662 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:50:01.056580 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:01.060385 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:01.254854 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1129_1129_0 (state Temporary) 2024.10.22 01:50:01.305439 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1129_1129_0 to 202410_1142_1142_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:01.410100 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73537 2024.10.22 01:50:01.616326 [ 1254 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 01:50:01.633034 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.88 GiB, peak 1.97 GiB, free memory in arenas 11.14 MiB, will set to 1.74 GiB (RSS), difference: -140.13 MiB 2024.10.22 01:50:02.169652 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 10202 2024.10.22 01:50:02.223452 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 151 entries to flush up to offset 74933 2024.10.22 01:50:02.282962 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:02.297219 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:02.299598 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 559 2024.10.22 01:50:02.446695 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_669_669_0 (state Temporary) 2024.10.22 01:50:02.631843 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:02.632371 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:02.639366 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:02.655545 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:02.701382 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_669_669_0 to 202410_682_682_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:02.707432 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_541_541_0 (state Temporary) 2024.10.22 01:50:02.714606 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_657_657_0 (state Temporary) 2024.10.22 01:50:02.779676 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_657_657_0 to 202410_663_663_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:02.875602 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74933 2024.10.22 01:50:02.939833 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_541_541_0 to 202410_551_551_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:03.002451 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10202 2024.10.22 01:50:03.042356 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 559 2024.10.22 01:50:03.108720 [ 1266 ] {} DatabaseCatalog: Execution took 313 ms. 2024.10.22 01:50:03.545448 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.87 GiB, peak 1.97 GiB, free memory in arenas 8.75 MiB, will set to 1.74 GiB (RSS), difference: -135.15 MiB 2024.10.22 01:50:04.031431 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/adi/mcubbdjldgrcrxxpurttqvneqffqf (LogSeriesLimiter: on interval from 2024-10-22 01:49:59 to 2024-10-22 01:50:04 accepted series 1 / 230 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:50:04.161656 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_640_124} MergeTask::MergeProjectionsStage: Merge sorted 986553 rows, containing 4 columns (4 merged, 0 gathered) in 33.00935382 sec., 29887.073990592886 rows/sec., 448.08 KiB/sec. 2024.10.22 01:50:04.234918 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:04.238739 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:04.336461 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_640_124} MergedBlockOutputStream: filled checksums 202410_1_640_124 (state Temporary) 2024.10.22 01:50:04.344174 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 983364 2024.10.22 01:50:04.751181 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(3b2562287c02de184463f6d3d14da81c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3b2562287c02de184463f6d3d14da81c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:50:04.754512 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 3b2562287c02de184463f6d3d14da81c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 01:50:04.771321 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:50:05.293144 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:05.300497 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:05.366522 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.89 GiB, peak 1.97 GiB, free memory in arenas 9.88 MiB, will set to 1.74 GiB (RSS), difference: -153.73 MiB 2024.10.22 01:50:05.524763 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 73579 2024.10.22 01:50:05.557375 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_641_641_0 (state Temporary) 2024.10.22 01:50:05.647096 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:05.651712 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:05.663486 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_641_641_0 to 202410_648_648_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:05.724199 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1130_1130_0 (state Temporary) 2024.10.22 01:50:05.921772 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 983364 2024.10.22 01:50:06.062293 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1130_1130_0 to 202410_1143_1143_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:06.108735 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73579 2024.10.22 01:50:07.105674 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_640_124} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_640_124 to 202410_1_640_124 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:07.128357 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_640_124} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_634_123, 202410_635_640_1] -> 202410_1_640_124 2024.10.22 01:50:07.264140 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:50:07.663426 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:50:07.719433 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 8.56 MiB, will set to 1.75 GiB (RSS), difference: -21.59 MiB 2024.10.22 01:50:08.009388 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_135_135_0 (state Temporary) 2024.10.22 01:50:09.475469 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.97 GiB, free memory in arenas 9.35 MiB, will set to 1.76 GiB (RSS), difference: -8.66 MiB 2024.10.22 01:50:10.116044 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 16 entries to flush up to offset 73595 2024.10.22 01:50:10.127342 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(be06943bf52eb7ed9c6b90e7f3f08a68) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: be06943bf52eb7ed9c6b90e7f3f08a68, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:50:10.232470 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1098, file segment info: File segment: [0, 33554431], key: be06943bf52eb7ed9c6b90e7f3f08a68, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:50:10.236723 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:50:10.363865 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:10.416192 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 63 entries to flush up to offset 74996 2024.10.22 01:50:10.365805 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:10.517067 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 10210 2024.10.22 01:50:10.538127 [ 1743 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xin/wsdldbbdqwilmbchhuerajnxhwfvf, size 5862448 2024.10.22 01:50:10.641724 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1131_1131_0 (state Temporary) 2024.10.22 01:50:10.658039 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:10.662789 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.87 GiB. 2024.10.22 01:50:10.716604 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_670_670_0 (state Temporary) 2024.10.22 01:50:10.734631 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1131_1131_0 to 202410_1144_1144_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:10.774566 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:10.781458 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.87 GiB. 2024.10.22 01:50:10.807402 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73595 2024.10.22 01:50:10.819244 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:50:10.826268 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:50:10.897541 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:50:10.913018 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:50:10.951523 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_658_658_0 (state Temporary) 2024.10.22 01:50:10.953256 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:50:10.978591 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:50:11.022802 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:50:11.023345 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_658_658_0 to 202410_664_664_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:11.054162 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:50:11.070751 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_670_670_0 to 202410_683_683_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:11.076932 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:50:11.098414 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 74996 2024.10.22 01:50:11.105804 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:50:11.107188 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 5 parts from 202410_1_678_136 to 202410_682_682_0 2024.10.22 01:50:11.126943 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10210 2024.10.22 01:50:11.153715 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:11.160777 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:11.246783 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RequestTry 2024.10.22 01:50:11.205837 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_682_137} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_678_136 to 202410_682_682_0 into Compact with storage Full 2024.10.22 01:50:11.400493 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yqv/imgmzhuzcgeazrlvznnriarefncka (LogSeriesLimiter: on interval from 2024-10-22 01:50:04 to 2024-10-22 01:50:11 accepted series 1 / 86 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:50:11.479965 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_682_137} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:50:11.641310 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_682_137} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_678_136, total 10241 rows starting from the beginning of the part 2024.10.22 01:50:11.680838 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_682_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_679_679_0, total 22 rows starting from the beginning of the part 2024.10.22 01:50:11.764838 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_682_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_680_680_0, total 12 rows starting from the beginning of the part 2024.10.22 01:50:11.945348 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_682_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_681_681_0, total 13 rows starting from the beginning of the part 2024.10.22 01:50:12.045507 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_682_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_682_682_0, total 20 rows starting from the beginning of the part 2024.10.22 01:50:12.659817 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 7.48 MiB, will set to 1.76 GiB (RSS), difference: -4.67 MiB 2024.10.22 01:50:12.931083 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 984519 2024.10.22 01:50:13.112440 [ 1228 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:50:13.248191 [ 1228 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:50:13.516885 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:13.519853 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:13.650942 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_642_642_0 (state Temporary) 2024.10.22 01:50:13.708223 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_642_642_0 to 202410_649_649_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:13.719283 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_682_137} MergeTask::MergeProjectionsStage: Merge sorted 10308 rows, containing 12 columns (12 merged, 0 gathered) in 2.520693147 sec., 4089.351380301111 rows/sec., 650.94 KiB/sec. 2024.10.22 01:50:13.771085 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 984519 2024.10.22 01:50:13.774167 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_682_137} MergedBlockOutputStream: filled checksums 202410_1_682_137 (state Temporary) 2024.10.22 01:50:13.871365 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_682_137} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_682_137 to 202410_1_682_137 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:13.873275 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_682_137} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 5 parts: [202410_1_678_136, 202410_682_682_0] -> 202410_1_682_137 2024.10.22 01:50:13.901811 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.50 MiB. 2024.10.22 01:50:14.411395 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 6.69 MiB, will set to 1.76 GiB (RSS), difference: 10.11 MiB 2024.10.22 01:50:14.523866 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(baf66a267427a0a4ee2a863be5cd823b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: baf66a267427a0a4ee2a863be5cd823b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:14.525922 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 57, file segment info: File segment: [0, 33554431], key: baf66a267427a0a4ee2a863be5cd823b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:14.528581 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:50:14.819700 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 73657 2024.10.22 01:50:14.883056 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_498_102, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0, 202410_502_502_0, 202410_503_503_0] 2024.10.22 01:50:14.885612 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_498_102, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0, 202410_502_502_0, 202410_503_503_0] 2024.10.22 01:50:15.063359 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_498_102, 202410_499_499_0, 202410_500_500_0, 202410_501_501_0, 202410_502_502_0, 202410_503_503_0] 2024.10.22 01:50:15.059918 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_134_24/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:50:15.158781 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_587_587_0, 202410_588_588_0, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0, 202410_592_592_0] 2024.10.22 01:50:15.160197 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_587_587_0, 202410_588_588_0, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0, 202410_592_592_0] 2024.10.22 01:50:15.186707 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_587_587_0, 202410_588_588_0, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0, 202410_592_592_0] 2024.10.22 01:50:15.233392 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:15.238619 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:15.406020 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1132_1132_0 (state Temporary) 2024.10.22 01:50:15.456282 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1132_1132_0 to 202410_1145_1145_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:15.464346 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_135_135_0 to 202410_138_138_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:15.496215 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73657 2024.10.22 01:50:15.572826 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(9ffa5015e6f2419b052861845faaea8f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9ffa5015e6f2419b052861845faaea8f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:15.579194 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 57, file segment info: File segment: [0, 33554431], key: 9ffa5015e6f2419b052861845faaea8f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:15.583430 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:50:16.012968 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_134_24 to 202410_1_134_24 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:16.023156 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_134_24} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 7 parts: [202410_1_124_23, 202410_134_134_0] -> 202410_1_134_24 2024.10.22 01:50:16.300557 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(030891c153701bafa5fb6897d90947ee) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 030891c153701bafa5fb6897d90947ee, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:16.305256 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 452, file segment info: File segment: [0, 33554431], key: 030891c153701bafa5fb6897d90947ee, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:16.308035 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:50:16.428099 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1033_498, 202410_1034_1039_1] 2024.10.22 01:50:16.434725 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1033_498, 202410_1034_1039_1] 2024.10.22 01:50:16.457731 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 4.04 MiB, will set to 1.76 GiB (RSS), difference: 14.99 MiB 2024.10.22 01:50:16.481643 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1033_498, 202410_1034_1039_1] 2024.10.22 01:50:16.718216 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:50:16.838765 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:50:16.860426 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:50:16.982959 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 54.55 MiB. 2024.10.22 01:50:16.983846 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:50:17.142676 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:50:17.169580 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:50:17.200804 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:50:17.250333 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column SocialAction 2024.10.22 01:50:17.312893 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vbx/uvdyckarmsgaowmqtdihzsufsjwex (LogSeriesLimiter: on interval from 2024-10-22 01:50:11 to 2024-10-22 01:50:17 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:50:18.019711 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1040_1040_0, 202410_1041_1041_0, 202410_1042_1042_0, 202410_1043_1043_0, 202410_1044_1044_0, 202410_1045_1045_0] 2024.10.22 01:50:18.022800 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1040_1040_0, 202410_1041_1041_0, 202410_1042_1042_0, 202410_1043_1043_0, 202410_1044_1044_0, 202410_1045_1045_0] 2024.10.22 01:50:18.047842 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 560 2024.10.22 01:50:18.061554 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1040_1040_0, 202410_1041_1041_0, 202410_1042_1042_0, 202410_1043_1043_0, 202410_1044_1044_0, 202410_1045_1045_0] 2024.10.22 01:50:18.402985 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:18.405108 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:18.459867 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.97 GiB, free memory in arenas 5.11 MiB, will set to 1.76 GiB (RSS), difference: 2.57 MiB 2024.10.22 01:50:18.468239 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_542_542_0 (state Temporary) 2024.10.22 01:50:18.618445 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 75075 2024.10.22 01:50:18.638067 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 10230 2024.10.22 01:50:18.674259 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_542_542_0 to 202410_552_552_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:18.758205 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 560 2024.10.22 01:50:18.799287 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5404 2024.10.22 01:50:18.845199 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:18.848991 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:18.893139 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_671_671_0 (state Temporary) 2024.10.22 01:50:18.950645 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_671_671_0 to 202410_684_684_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:19.002626 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10230 2024.10.22 01:50:19.007655 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:19.016670 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:19.057700 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_659_659_0 (state Temporary) 2024.10.22 01:50:19.221254 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_659_659_0 to 202410_665_665_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:19.364788 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 75075 2024.10.22 01:50:19.381273 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_601_19, 202410_602_602_0, 202410_603_603_0, 202410_604_604_0, 202410_605_605_0, 202410_606_606_0] 2024.10.22 01:50:19.385364 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_601_19, 202410_602_602_0, 202410_603_603_0, 202410_604_604_0, 202410_605_605_0, 202410_606_606_0] 2024.10.22 01:50:19.421774 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_601_19, 202410_602_602_0, 202410_603_603_0, 202410_604_604_0, 202410_605_605_0, 202410_606_606_0] 2024.10.22 01:50:19.507878 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 73719 2024.10.22 01:50:19.863540 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:19.876163 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:19.933283 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1133_1133_0 (state Temporary) 2024.10.22 01:50:20.003775 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1133_1133_0 to 202410_1146_1146_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:20.040076 [ 1211 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1141_9 to 202410_1146_1146_0 2024.10.22 01:50:20.056510 [ 1211 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:20.059612 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:20.168620 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73719 2024.10.22 01:50:20.185219 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1141_9 to 202410_1146_1146_0 into Compact with storage Full 2024.10.22 01:50:20.287642 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:50:20.342354 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1141_9, total 2917 rows starting from the beginning of the part 2024.10.22 01:50:20.501279 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1142_1142_0, total 44 rows starting from the beginning of the part 2024.10.22 01:50:20.526919 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:50:20.720306 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1143_1143_0, total 42 rows starting from the beginning of the part 2024.10.22 01:50:20.777061 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1593 entries to flush up to offset 986112 2024.10.22 01:50:20.789334 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1144_1144_0, total 16 rows starting from the beginning of the part 2024.10.22 01:50:20.818991 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:50:20.822589 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:50:20.971097 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1145_1145_0, total 62 rows starting from the beginning of the part 2024.10.22 01:50:21.031904 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1146_1146_0, total 62 rows starting from the beginning of the part 2024.10.22 01:50:21.345210 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:21.350508 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:21.511632 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_643_643_0 (state Temporary) 2024.10.22 01:50:21.577138 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_643_643_0 to 202410_650_650_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:21.612524 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 986112 2024.10.22 01:50:21.715668 [ 1225 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_640_124 to 202410_641_646_1 2024.10.22 01:50:21.850694 [ 1225 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:21.856009 [ 1225 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:21.861385 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 272 ms. 2024.10.22 01:50:21.881470 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_646_125} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_640_124 to 202410_641_646_1 into Wide with storage Full 2024.10.22 01:50:21.950440 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_646_125} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:50:21.958856 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_646_125} MergeTreeSequentialSource: Reading 122 marks from part 202410_1_640_124, total 986553 rows starting from the beginning of the part 2024.10.22 01:50:22.032364 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_646_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_641_646_1, total 6936 rows starting from the beginning of the part 2024.10.22 01:50:22.853538 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} MergeTask::MergeProjectionsStage: Merge sorted 3143 rows, containing 13 columns (13 merged, 0 gathered) in 2.78690949 sec., 1127.7725420497957 rows/sec., 203.11 KiB/sec. 2024.10.22 01:50:23.080551 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} MergedBlockOutputStream: filled checksums 202410_1096_1146_10 (state Temporary) 2024.10.22 01:50:23.145823 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 5.28 MiB, will set to 1.77 GiB (RSS), difference: 38.00 MiB 2024.10.22 01:50:23.593488 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 01:50:24.176929 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 73757 2024.10.22 01:50:24.364571 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:24.375665 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.87 GiB. 2024.10.22 01:50:24.410269 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_646_125/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:50:24.551047 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1134_1134_0 (state Temporary) 2024.10.22 01:50:24.637375 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1134_1134_0 to 202410_1147_1147_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:24.672421 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1146_10 to 202410_1096_1146_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:24.676906 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1146_10} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1141_9, 202410_1146_1146_0] -> 202410_1096_1146_10 2024.10.22 01:50:24.699992 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73757 2024.10.22 01:50:24.749585 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column WatchID 2024.10.22 01:50:24.786003 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column WatchID 2024.10.22 01:50:24.893466 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column WatchID 2024.10.22 01:50:24.896183 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:50:24.999711 [ 1702 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/adi/mcubbdjldgrcrxxpurttqvneqffqf, size 152 2024.10.22 01:50:25.013474 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column WatchID 2024.10.22 01:50:25.026108 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column WatchID 2024.10.22 01:50:25.245698 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column WatchID 2024.10.22 01:50:25.367365 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column WatchID 2024.10.22 01:50:25.610965 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fyk/wzfxdcqocziailspwfubikihrsywk (LogSeriesLimiter: on interval from 2024-10-22 01:50:17 to 2024-10-22 01:50:25 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:50:25.745380 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 1.97 GiB, free memory in arenas 9.52 MiB, will set to 1.77 GiB (RSS), difference: 155.80 MiB 2024.10.22 01:50:26.064757 [ 1688 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rde/xhxafbpndakfuqxycfjroqcrnydje, size 7523 2024.10.22 01:50:26.259815 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 561 2024.10.22 01:50:26.459749 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 5445 2024.10.22 01:50:26.507282 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10243 2024.10.22 01:50:26.672317 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:26.676765 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.86 GiB. 2024.10.22 01:50:26.756505 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:26.764575 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_543_543_0 (state Temporary) 2024.10.22 01:50:26.764915 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 01:50:26.874363 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 140 entries to flush up to offset 75215 2024.10.22 01:50:26.974224 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_543_543_0 to 202410_553_553_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:27.054514 [ 1217 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_548_112 to 202410_553_553_0 2024.10.22 01:50:27.067815 [ 1217 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:27.070663 [ 1217 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 01:50:27.150756 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_672_672_0 (state Temporary) 2024.10.22 01:50:27.224004 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_672_672_0 to 202410_685_685_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:27.224615 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 561 2024.10.22 01:50:27.262800 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_580_114, 202410_581_586_1] 2024.10.22 01:50:27.267048 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_580_114, 202410_581_586_1] 2024.10.22 01:50:27.276216 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10243 2024.10.22 01:50:27.324653 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_548_112 to 202410_553_553_0 into Compact with storage Full 2024.10.22 01:50:27.377800 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_580_114, 202410_581_586_1] 2024.10.22 01:50:27.512574 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:27.516589 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 01:50:27.703514 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_660_660_0 (state Temporary) 2024.10.22 01:50:27.778999 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:50:27.782813 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_660_660_0 to 202410_666_666_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:27.858324 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 75215 2024.10.22 01:50:27.864520 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_548_112, total 592 rows starting from the beginning of the part 2024.10.22 01:50:27.867594 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_9_9_0 (state Temporary) 2024.10.22 01:50:28.254300 [ 1237 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:50:28.275504 [ 1237 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:50:28.405785 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_549_549_0, total 1 rows starting from the beginning of the part 2024.10.22 01:50:28.410814 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 12.69 MiB, will set to 1.76 GiB (RSS), difference: 130.62 MiB 2024.10.22 01:50:28.602285 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_550_550_0, total 1 rows starting from the beginning of the part 2024.10.22 01:50:28.618863 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1106 entries to flush up to offset 987218 2024.10.22 01:50:28.705758 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 73813 2024.10.22 01:50:28.903812 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_551_551_0, total 1 rows starting from the beginning of the part 2024.10.22 01:50:29.011960 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:29.018718 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 01:50:29.069043 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(6113a3ec149e5357aafd30437f50b725) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6113a3ec149e5357aafd30437f50b725, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:50:29.071144 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 6113a3ec149e5357aafd30437f50b725, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 01:50:29.073151 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:50:29.099607 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_552_552_0, total 1 rows starting from the beginning of the part 2024.10.22 01:50:29.171941 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1135_1135_0 (state Temporary) 2024.10.22 01:50:29.320831 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1135_1135_0 to 202410_1148_1148_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:29.363979 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_553_553_0, total 1 rows starting from the beginning of the part 2024.10.22 01:50:29.367003 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73813 2024.10.22 01:50:29.666986 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:29.670097 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 01:50:29.819715 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_644_644_0 (state Temporary) 2024.10.22 01:50:29.952786 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(e954ecf97cb2f046b32958161971ccc8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e954ecf97cb2f046b32958161971ccc8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:50:29.957437 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: e954ecf97cb2f046b32958161971ccc8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1388, kind: Regular, unbound: 0 2024.10.22 01:50:29.959719 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:50:30.030612 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_644_644_0 to 202410_651_651_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:30.167753 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 987218 2024.10.22 01:50:30.553055 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.97 GiB, free memory in arenas 13.34 MiB, will set to 1.76 GiB (RSS), difference: -1.20 MiB 2024.10.22 01:50:30.823100 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:50:30.840806 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:50:31.228467 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d4206866d86c947920808089216c3224) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d4206866d86c947920808089216c3224, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:50:31.239841 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 609, file segment info: File segment: [0, 33554431], key: d4206866d86c947920808089216c3224, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 01:50:31.244583 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:50:31.385820 [ 1732 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yqv/imgmzhuzcgeazrlvznnriarefncka, size 1096558 2024.10.22 01:50:31.881731 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_553_113/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:50:31.925398 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c43ea813d2bb6c5426cfa8c637ef1673, offset: 0, size: 0 2024.10.22 01:50:31.932856 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c43ea813d2bb6c5426cfa8c637ef1673 for removal 2024.10.22 01:50:31.939084 [ 1476 ] {} CacheMetadata: Key c43ea813d2bb6c5426cfa8c637ef1673 is removed from metadata 2024.10.22 01:50:32.154934 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} MergeTask::MergeProjectionsStage: Merge sorted 597 rows, containing 32 columns (32 merged, 0 gathered) in 4.968614358 sec., 120.1542234886405 rows/sec., 10.98 KiB/sec. 2024.10.22 01:50:32.240790 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_621_124, 202410_622_622_0, 202410_623_623_0, 202410_624_624_0, 202410_625_625_0, 202410_626_626_0] 2024.10.22 01:50:32.246348 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_621_124, 202410_622_622_0, 202410_623_623_0, 202410_624_624_0, 202410_625_625_0, 202410_626_626_0] 2024.10.22 01:50:32.296477 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} MergedBlockOutputStream: filled checksums 202410_1_553_113 (state Temporary) 2024.10.22 01:50:32.369761 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_621_124, 202410_622_622_0, 202410_623_623_0, 202410_624_624_0, 202410_625_625_0, 202410_626_626_0] 2024.10.22 01:50:33.396587 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 73849 2024.10.22 01:50:33.535927 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:33.547229 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 01:50:33.614827 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1136_1136_0 (state Temporary) 2024.10.22 01:50:33.762470 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1136_1136_0 to 202410_1149_1149_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:33.876290 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c43ea813d2bb6c5426cfa8c637ef1673) : [0, 26853891]: Setting download as failed: File segment: [0, 26853891], key: c43ea813d2bb6c5426cfa8c637ef1673, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:33.893271 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(c43ea813d2bb6c5426cfa8c637ef1673): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:50:33.938465 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73849 2024.10.22 01:50:34.041480 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_553_113 to 202410_1_553_113 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:34.047949 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_553_113} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_548_112, 202410_553_553_0] -> 202410_1_553_113 2024.10.22 01:50:34.085253 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:50:34.731287 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 562 2024.10.22 01:50:34.779220 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10256 2024.10.22 01:50:35.152828 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:35.160624 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 01:50:35.231968 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_544_544_0 (state Temporary) 2024.10.22 01:50:35.259146 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:35.262327 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 01:50:35.279093 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c43ea813d2bb6c5426cfa8c637ef1673, offset: 0, size: 0 2024.10.22 01:50:35.281087 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c43ea813d2bb6c5426cfa8c637ef1673 for removal 2024.10.22 01:50:35.284231 [ 1476 ] {} CacheMetadata: Key c43ea813d2bb6c5426cfa8c637ef1673 is removed from metadata 2024.10.22 01:50:35.293412 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pyo/iucnzirkndatqgnkyapobhofidlaw. 2024.10.22 01:50:35.303211 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_673_673_0 (state Temporary) 2024.10.22 01:50:35.354388 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_544_544_0 to 202410_554_554_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:35.383992 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 96 entries to flush up to offset 75311 2024.10.22 01:50:35.440713 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 562 2024.10.22 01:50:35.458400 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 77ec80561addcf51b5fd7ccc2d66457c, offset: 0, size: 0 2024.10.22 01:50:35.459348 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 77ec80561addcf51b5fd7ccc2d66457c for removal 2024.10.22 01:50:35.463497 [ 1476 ] {} CacheMetadata: Key 77ec80561addcf51b5fd7ccc2d66457c is removed from metadata 2024.10.22 01:50:35.543150 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pyo/iucnzirkndatqgnkyapobhofidlaw. 2024.10.22 01:50:35.594687 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.97 GiB, free memory in arenas 12.63 MiB, will set to 1.76 GiB (RSS), difference: 6.43 MiB 2024.10.22 01:50:35.595912 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:35.601019 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 01:50:35.650719 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_673_673_0 to 202410_686_686_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:35.772284 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_661_661_0 (state Temporary) 2024.10.22 01:50:35.804164 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10256 2024.10.22 01:50:35.829049 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_661_661_0 to 202410_667_667_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:35.851975 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_646_125/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:50:35.866973 [ 1156 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_662_32 to 202410_667_667_0 2024.10.22 01:50:35.891270 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 75311 2024.10.22 01:50:35.900751 [ 1156 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:35.906254 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 01:50:35.947420 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_662_32 to 202410_667_667_0 into Compact with storage Full 2024.10.22 01:50:36.022873 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:50:36.055177 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_662_32, total 15657 rows starting from the beginning of the part 2024.10.22 01:50:36.159410 [ 1192 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 01:50:36.379977 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_663_663_0, total 151 rows starting from the beginning of the part 2024.10.22 01:50:36.550742 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_664_664_0, total 63 rows starting from the beginning of the part 2024.10.22 01:50:36.623452 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_665_665_0, total 79 rows starting from the beginning of the part 2024.10.22 01:50:36.764888 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(77ec80561addcf51b5fd7ccc2d66457c) : [0, 501907]: Setting download as failed: File segment: [0, 501907], key: 77ec80561addcf51b5fd7ccc2d66457c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:36.768787 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(77ec80561addcf51b5fd7ccc2d66457c): No space left in cache to reserve 501908 bytes, will continue without cache download 2024.10.22 01:50:36.791828 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_666_666_0, total 140 rows starting from the beginning of the part 2024.10.22 01:50:36.956355 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_667_667_0, total 96 rows starting from the beginning of the part 2024.10.22 01:50:37.112917 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:50:37.198196 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 988758 2024.10.22 01:50:37.356617 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 77ec80561addcf51b5fd7ccc2d66457c, offset: 0, size: 0 2024.10.22 01:50:37.360338 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 77ec80561addcf51b5fd7ccc2d66457c for removal 2024.10.22 01:50:37.363638 [ 1476 ] {} CacheMetadata: Key 77ec80561addcf51b5fd7ccc2d66457c is removed from metadata 2024.10.22 01:50:37.368255 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/joa/uvwsudilnfximdksetazbxhkjoxaa, upload id 4b41fd86-bb5b-46f6-a981-76839d59ef91. 2024.10.22 01:50:37.447872 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} AWSClient: Got dangerous response with successful code 200, checking its body: ' http://localhost:11111/test/common/joa/uvwsudilnfximdksetazbxhkjoxaatestcommon/joa/uvwsudilnfximdksetazbxhkjoxaa"353d632fe8' 2024.10.22 01:50:37.454759 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Multipart upload has completed. Details: bucket test, key common/joa/uvwsudilnfximdksetazbxhkjoxaa, upload id 4b41fd86-bb5b-46f6-a981-76839d59ef91, Parts: 3 2024.10.22 01:50:37.597679 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ad616a7930721a2d9dac02121ddaba3a, offset: 0, size: 0 2024.10.22 01:50:37.604148 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ad616a7930721a2d9dac02121ddaba3a for removal 2024.10.22 01:50:37.612034 [ 1476 ] {} CacheMetadata: Key ad616a7930721a2d9dac02121ddaba3a is removed from metadata 2024.10.22 01:50:37.668935 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/joa/uvwsudilnfximdksetazbxhkjoxaa, upload id 4b41fd86-bb5b-46f6-a981-76839d59ef91. 2024.10.22 01:50:37.672466 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gmh/hkapthxcptwdjnqujadtruvnjqsck. 2024.10.22 01:50:37.858464 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 12.52 MiB, will set to 1.77 GiB (RSS), difference: -2.32 MiB 2024.10.22 01:50:37.878970 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gmh/hkapthxcptwdjnqujadtruvnjqsck. 2024.10.22 01:50:37.948947 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 73917 2024.10.22 01:50:37.955407 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:37.957798 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:50:38.080294 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/otn/whrdzloqnvzorfrmtknrdviajrqrn. 2024.10.22 01:50:38.086873 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:38.093343 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:50:38.121347 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f1f5f41618307d2fcb4ec8c635c7db0e, offset: 0, size: 0 2024.10.22 01:50:38.136292 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f1f5f41618307d2fcb4ec8c635c7db0e for removal 2024.10.22 01:50:38.140606 [ 1476 ] {} CacheMetadata: Key f1f5f41618307d2fcb4ec8c635c7db0e is removed from metadata 2024.10.22 01:50:38.196636 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/otn/whrdzloqnvzorfrmtknrdviajrqrn. 2024.10.22 01:50:38.196957 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_645_645_0 (state Temporary) 2024.10.22 01:50:38.276353 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_645_645_0 to 202410_652_652_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:38.418571 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 988758 2024.10.22 01:50:38.415273 [ 1247 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_647_647_0 to 202410_652_652_0 2024.10.22 01:50:38.454387 [ 1247 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:38.456339 [ 1247 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:50:38.478696 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_647_647_0 to 202410_652_652_0 into Compact with storage Full 2024.10.22 01:50:38.510739 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} MergeTask::MergeProjectionsStage: Merge sorted 16186 rows, containing 13 columns (13 merged, 0 gathered) in 2.570098236 sec., 6297.813746291369 rows/sec., 2.73 MiB/sec. 2024.10.22 01:50:38.632431 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:50:38.657029 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_647_647_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:50:38.682335 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_648_648_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:50:38.817427 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_649_649_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:50:38.852186 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1137_1137_0 (state Temporary) 2024.10.22 01:50:38.908319 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} MergedBlockOutputStream: filled checksums 202410_507_667_33 (state Temporary) 2024.10.22 01:50:38.959220 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1137_1137_0 to 202410_1150_1150_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:38.959217 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_650_650_0, total 1593 rows starting from the beginning of the part 2024.10.22 01:50:38.991124 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_651_651_0, total 1106 rows starting from the beginning of the part 2024.10.22 01:50:39.033331 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_652_652_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:50:39.044264 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73917 2024.10.22 01:50:39.200771 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_667_33 to 202410_507_667_33 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:39.208515 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_667_33} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_662_32, 202410_667_667_0] -> 202410_507_667_33 2024.10.22 01:50:39.326691 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(f1f5f41618307d2fcb4ec8c635c7db0e) : [0, 12044091]: Setting download as failed: File segment: [0, 12044091], key: f1f5f41618307d2fcb4ec8c635c7db0e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:39.333297 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(f1f5f41618307d2fcb4ec8c635c7db0e): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:50:39.337367 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 57 bytes for fcd3c9c2e9d86650fa03805a82ac0d9a:0) 2024.10.22 01:50:39.357754 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.15 MiB. 2024.10.22 01:50:39.448629 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: da4831e16754e9f9499d0bafe7e1ec8f, offset: 0, size: 1393 2024.10.22 01:50:39.451608 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key da4831e16754e9f9499d0bafe7e1ec8f for removal 2024.10.22 01:50:39.455309 [ 1476 ] {} CacheMetadata: Key da4831e16754e9f9499d0bafe7e1ec8f is removed from metadata 2024.10.22 01:50:39.577565 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 12.61 MiB, will set to 1.77 GiB (RSS), difference: 12.44 MiB 2024.10.22 01:50:39.715061 [ 1680 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vbx/uvdyckarmsgaowmqtdihzsufsjwex, size 57 2024.10.22 01:50:39.823796 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.112754626 sec. 2024.10.22 01:50:40.259169 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:50:40.274128 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:50:40.362899 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f1f5f41618307d2fcb4ec8c635c7db0e, offset: 0, size: 0 2024.10.22 01:50:40.366676 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f1f5f41618307d2fcb4ec8c635c7db0e for removal 2024.10.22 01:50:40.369841 [ 1476 ] {} CacheMetadata: Key f1f5f41618307d2fcb4ec8c635c7db0e is removed from metadata 2024.10.22 01:50:40.372239 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/art/gvnqnyjohyulrxmyhwswpbctgglet. 2024.10.22 01:50:40.501483 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:50:40.511625 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} MergeTask::MergeProjectionsStage: Merge sorted 8859 rows, containing 4 columns (4 merged, 0 gathered) in 2.044495497 sec., 4333.098318386758 rows/sec., 67.70 KiB/sec. 2024.10.22 01:50:40.522159 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5a9443235694259bda78a336928c8852, offset: 0, size: 0 2024.10.22 01:50:40.533403 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5a9443235694259bda78a336928c8852 for removal 2024.10.22 01:50:40.541816 [ 1476 ] {} CacheMetadata: Key 5a9443235694259bda78a336928c8852 is removed from metadata 2024.10.22 01:50:40.552039 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:50:40.606075 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} MergedBlockOutputStream: filled checksums 202410_647_652_1 (state Temporary) 2024.10.22 01:50:40.607376 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:50:40.643360 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:50:40.754912 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:50:40.799180 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_647_652_1 to 202410_647_652_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:40.805178 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_647_652_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_647_647_0, 202410_652_652_0] -> 202410_647_652_1 2024.10.22 01:50:40.831429 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:50:40.838534 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:50:40.892568 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/art/gvnqnyjohyulrxmyhwswpbctgglet. 2024.10.22 01:50:40.909291 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:50:40.964520 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column HTTPError 2024.10.22 01:50:41.171613 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ugz/fkaqfdcfgexwgsbvtpkfyzimnulbq (LogSeriesLimiter: on interval from 2024-10-22 01:50:25 to 2024-10-22 01:50:41 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:50:41.723950 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 13.00 MiB, will set to 1.77 GiB (RSS), difference: -9.02 MiB 2024.10.22 01:50:42.948045 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 563 2024.10.22 01:50:43.046896 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ehr/kjgsncfkstvdxtztourucumoekghr. 2024.10.22 01:50:43.048383 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:43.056747 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 73984 2024.10.22 01:50:43.058647 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:50:43.101515 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(5a9443235694259bda78a336928c8852) : [0, 2928448]: Setting download as failed: File segment: [0, 2928448], key: 5a9443235694259bda78a336928c8852, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:43.104118 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(5a9443235694259bda78a336928c8852): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:50:43.289095 [ 1225 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:50:43.307765 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 10275 2024.10.22 01:50:43.449677 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:43.457106 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:50:43.462272 [ 1225 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:50:43.465758 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:43.470812 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:50:43.512569 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 108 entries to flush up to offset 75419 2024.10.22 01:50:43.630418 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_545_545_0 (state Temporary) 2024.10.22 01:50:43.697488 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:43.704635 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:50:43.756290 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_674_674_0 (state Temporary) 2024.10.22 01:50:43.756974 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1138_1138_0 (state Temporary) 2024.10.22 01:50:43.795568 [ 1230 ] {} DatabaseCatalog: Execution took 268 ms. 2024.10.22 01:50:43.940221 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_545_545_0 to 202410_555_555_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:43.949188 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_674_674_0 to 202410_687_687_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:43.979373 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_682_137 to 202410_687_687_0 2024.10.22 01:50:43.988056 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:43.994618 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:50:44.010357 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:44.016415 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:50:44.016797 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1138_1138_0 to 202410_1151_1151_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:44.063507 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_682_137 to 202410_687_687_0 into Compact with storage Full 2024.10.22 01:50:44.095786 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1146_10 to 202410_1151_1151_0 2024.10.22 01:50:44.107617 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10275 2024.10.22 01:50:44.148631 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:44.150775 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:50:44.156935 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_662_662_0 (state Temporary) 2024.10.22 01:50:44.162663 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 563 2024.10.22 01:50:44.166635 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 73984 2024.10.22 01:50:44.217008 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1146_10 to 202410_1151_1151_0 into Compact with storage Full 2024.10.22 01:50:44.306737 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:50:44.324234 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_682_137, total 10308 rows starting from the beginning of the part 2024.10.22 01:50:44.346460 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_662_662_0 to 202410_668_668_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:44.355829 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_683_683_0, total 8 rows starting from the beginning of the part 2024.10.22 01:50:44.376279 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_684_684_0, total 20 rows starting from the beginning of the part 2024.10.22 01:50:44.408144 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 75419 2024.10.22 01:50:44.430347 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_685_685_0, total 13 rows starting from the beginning of the part 2024.10.22 01:50:44.449187 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:50:44.464613 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1039_499, 202410_1040_1045_1] 2024.10.22 01:50:44.468204 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1146_10, total 3143 rows starting from the beginning of the part 2024.10.22 01:50:44.475844 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5a9443235694259bda78a336928c8852, offset: 0, size: 0 2024.10.22 01:50:44.479276 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5a9443235694259bda78a336928c8852 for removal 2024.10.22 01:50:44.468676 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1039_499, 202410_1040_1045_1] 2024.10.22 01:50:44.492608 [ 1476 ] {} CacheMetadata: Key 5a9443235694259bda78a336928c8852 is removed from metadata 2024.10.22 01:50:44.563593 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1147_1147_0, total 38 rows starting from the beginning of the part 2024.10.22 01:50:44.574767 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_686_686_0, total 13 rows starting from the beginning of the part 2024.10.22 01:50:44.630831 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5da4d73e1aeb7c1b02da9df1d42d7149, offset: 0, size: 0 2024.10.22 01:50:44.635514 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5da4d73e1aeb7c1b02da9df1d42d7149 for removal 2024.10.22 01:50:44.643246 [ 1476 ] {} CacheMetadata: Key 5da4d73e1aeb7c1b02da9df1d42d7149 is removed from metadata 2024.10.22 01:50:44.675940 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1039_499, 202410_1040_1045_1] 2024.10.22 01:50:44.685474 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_687_687_0, total 19 rows starting from the beginning of the part 2024.10.22 01:50:44.712856 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/btf/amvgwphczqiajftivzttflrplrkod. 2024.10.22 01:50:44.749203 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1148_1148_0, total 56 rows starting from the beginning of the part 2024.10.22 01:50:45.013698 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1149_1149_0, total 36 rows starting from the beginning of the part 2024.10.22 01:50:45.069780 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d4518825df9b0bc95fa7d8bf9b6edb46, offset: 0, size: 0 2024.10.22 01:50:45.072419 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d4518825df9b0bc95fa7d8bf9b6edb46 for removal 2024.10.22 01:50:45.080571 [ 1476 ] {} CacheMetadata: Key d4518825df9b0bc95fa7d8bf9b6edb46 is removed from metadata 2024.10.22 01:50:45.086219 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1150_1150_0, total 68 rows starting from the beginning of the part 2024.10.22 01:50:45.166749 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/btf/amvgwphczqiajftivzttflrplrkod. 2024.10.22 01:50:45.220179 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1151_1151_0, total 67 rows starting from the beginning of the part 2024.10.22 01:50:45.428832 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 989913 2024.10.22 01:50:45.432029 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 10.20 MiB, will set to 1.76 GiB (RSS), difference: -10.77 MiB 2024.10.22 01:50:46.009403 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_646_125/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:50:46.099741 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} MergeTask::MergeProjectionsStage: Merge sorted 10381 rows, containing 12 columns (12 merged, 0 gathered) in 2.045358436 sec., 5075.394032305446 rows/sec., 807.99 KiB/sec. 2024.10.22 01:50:46.262655 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} MergedBlockOutputStream: filled checksums 202410_1_687_138 (state Temporary) 2024.10.22 01:50:46.653456 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_687_138 to 202410_1_687_138 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:46.748937 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:46.752241 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:50:46.759329 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_687_138} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_682_137, 202410_687_687_0] -> 202410_1_687_138 2024.10.22 01:50:46.809700 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} MergeTask::MergeProjectionsStage: Merge sorted 3408 rows, containing 13 columns (13 merged, 0 gathered) in 2.601482945 sec., 1310.0220420626283 rows/sec., 235.18 KiB/sec. 2024.10.22 01:50:46.816082 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.56 MiB. 2024.10.22 01:50:46.968765 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} MergedBlockOutputStream: filled checksums 202410_1096_1151_11 (state Temporary) 2024.10.22 01:50:47.151545 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d4518825df9b0bc95fa7d8bf9b6edb46) : [0, 1890900]: Setting download as failed: File segment: [0, 1890900], key: d4518825df9b0bc95fa7d8bf9b6edb46, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:47.153446 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(d4518825df9b0bc95fa7d8bf9b6edb46): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:50:47.156748 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 1140 bytes for b91e6740ee3930b79365bcd4a92491f1:0) 2024.10.22 01:50:47.158617 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_646_646_0 (state Temporary) 2024.10.22 01:50:47.160802 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5d15e314ae5ff3d939475b9ba4964407, offset: 0, size: 992 2024.10.22 01:50:47.165378 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 5d15e314ae5ff3d939475b9ba4964407 for removal 2024.10.22 01:50:47.170228 [ 1476 ] {} CacheMetadata: Key 5d15e314ae5ff3d939475b9ba4964407 is removed from metadata 2024.10.22 01:50:47.262165 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_646_646_0 to 202410_653_653_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:47.312216 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 989913 2024.10.22 01:50:47.356461 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1151_11 to 202410_1096_1151_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:47.359620 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1151_11} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1146_10, 202410_1151_1151_0] -> 202410_1096_1151_11 2024.10.22 01:50:47.421460 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:50:47.852986 [ 1728 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fyk/wzfxdcqocziailspwfubikihrsywk, size 22970616 2024.10.22 01:50:47.865206 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_136_136_0 (state Temporary) 2024.10.22 01:50:48.059550 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d4518825df9b0bc95fa7d8bf9b6edb46, offset: 0, size: 0 2024.10.22 01:50:48.064401 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d4518825df9b0bc95fa7d8bf9b6edb46 for removal 2024.10.22 01:50:48.068187 [ 1476 ] {} CacheMetadata: Key d4518825df9b0bc95fa7d8bf9b6edb46 is removed from metadata 2024.10.22 01:50:48.070515 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/bfq/zdrneonesomalqwcqxfogkrsbduny. 2024.10.22 01:50:48.155911 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:50:48.169561 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:50:48.184410 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 98 entries to flush up to offset 74082 2024.10.22 01:50:48.205312 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:50:48.241490 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0e7ffea1ab346168c7db85476117ca41, offset: 0, size: 0 2024.10.22 01:50:48.247836 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0e7ffea1ab346168c7db85476117ca41 for removal 2024.10.22 01:50:48.252732 [ 1476 ] {} CacheMetadata: Key 0e7ffea1ab346168c7db85476117ca41 is removed from metadata 2024.10.22 01:50:48.262325 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:50:48.274924 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:50:48.307453 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:50:48.354612 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column JavaEnable 2024.10.22 01:50:48.357242 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/bfq/zdrneonesomalqwcqxfogkrsbduny. 2024.10.22 01:50:48.434581 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/znm/tushejfdwcrxtxmtesrwqxplirgyi (LogSeriesLimiter: on interval from 2024-10-22 01:50:41 to 2024-10-22 01:50:48 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:50:48.779925 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 14.89 MiB, will set to 1.75 GiB (RSS), difference: 26.16 MiB 2024.10.22 01:50:49.033827 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:49.038226 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.87 GiB. 2024.10.22 01:50:49.131022 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1139_1139_0 (state Temporary) 2024.10.22 01:50:49.206026 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1139_1139_0 to 202410_1152_1152_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:49.271342 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 74082 2024.10.22 01:50:50.032322 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(0e7ffea1ab346168c7db85476117ca41) : [0, 56294]: Setting download as failed: File segment: [0, 56294], key: 0e7ffea1ab346168c7db85476117ca41, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:50.035496 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(0e7ffea1ab346168c7db85476117ca41): No space left in cache to reserve 56295 bytes, will continue without cache download 2024.10.22 01:50:50.158539 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0e7ffea1ab346168c7db85476117ca41, offset: 0, size: 0 2024.10.22 01:50:50.159800 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0e7ffea1ab346168c7db85476117ca41 for removal 2024.10.22 01:50:50.161518 [ 1476 ] {} CacheMetadata: Key 0e7ffea1ab346168c7db85476117ca41 is removed from metadata 2024.10.22 01:50:50.163602 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ahj/eswlkgbbeibcozlddwibhqnlcvkdr. 2024.10.22 01:50:50.296540 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 84269666b4fea5967eabf859348b9299, offset: 0, size: 0 2024.10.22 01:50:50.299964 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 84269666b4fea5967eabf859348b9299 for removal 2024.10.22 01:50:50.303619 [ 1476 ] {} CacheMetadata: Key 84269666b4fea5967eabf859348b9299 is removed from metadata 2024.10.22 01:50:50.359412 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ahj/eswlkgbbeibcozlddwibhqnlcvkdr. 2024.10.22 01:50:50.626416 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1045_500, 202410_1046_1046_0] 2024.10.22 01:50:50.630956 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1045_500, 202410_1046_1046_0] 2024.10.22 01:50:50.712622 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1045_500, 202410_1046_1046_0] 2024.10.22 01:50:50.836847 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:50:50.932168 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:50:51.611357 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 10295 2024.10.22 01:50:51.665101 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 564 2024.10.22 01:50:51.724874 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 10.26 MiB, will set to 1.75 GiB (RSS), difference: -4.97 MiB 2024.10.22 01:50:51.753612 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(84269666b4fea5967eabf859348b9299) : [0, 760400]: Setting download as failed: File segment: [0, 760400], key: 84269666b4fea5967eabf859348b9299, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:51.755296 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(84269666b4fea5967eabf859348b9299): No space left in cache to reserve 760401 bytes, will continue without cache download 2024.10.22 01:50:51.858518 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:51.861398 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:51.913052 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_675_675_0 (state Temporary) 2024.10.22 01:50:52.046269 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:52.051977 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 92 entries to flush up to offset 75511 2024.10.22 01:50:52.052851 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:52.176774 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 84269666b4fea5967eabf859348b9299, offset: 0, size: 0 2024.10.22 01:50:52.178623 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 84269666b4fea5967eabf859348b9299 for removal 2024.10.22 01:50:52.186560 [ 1476 ] {} CacheMetadata: Key 84269666b4fea5967eabf859348b9299 is removed from metadata 2024.10.22 01:50:52.226696 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_546_546_0 (state Temporary) 2024.10.22 01:50:52.279644 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:52.284175 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_675_675_0 to 202410_688_688_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:52.286813 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:52.289207 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/flz/xqekkwsexcjdtqyuxvishwaxsvegm. 2024.10.22 01:50:52.359958 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_663_663_0 (state Temporary) 2024.10.22 01:50:52.375360 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10295 2024.10.22 01:50:52.407864 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_663_663_0 to 202410_669_669_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:52.445239 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_546_546_0 to 202410_556_556_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:52.483149 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 75511 2024.10.22 01:50:52.562564 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 18798fb2b518b2b7e9c467b3d4080c5c, offset: 0, size: 0 2024.10.22 01:50:52.563875 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 18798fb2b518b2b7e9c467b3d4080c5c for removal 2024.10.22 01:50:52.567454 [ 1476 ] {} CacheMetadata: Key 18798fb2b518b2b7e9c467b3d4080c5c is removed from metadata 2024.10.22 01:50:52.630884 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8d4af8f09f4f8c4bcb906efb387bc932, offset: 0, size: 0 2024.10.22 01:50:52.632994 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8d4af8f09f4f8c4bcb906efb387bc932 for removal 2024.10.22 01:50:52.637215 [ 1476 ] {} CacheMetadata: Key 8d4af8f09f4f8c4bcb906efb387bc932 is removed from metadata 2024.10.22 01:50:52.649601 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/flz/xqekkwsexcjdtqyuxvishwaxsvegm. 2024.10.22 01:50:52.672956 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 564 2024.10.22 01:50:52.808051 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1047_1047_0, 202410_1048_1048_0, 202410_1049_1049_0, 202410_1050_1050_0, 202410_1051_1051_0, 202410_1052_1052_0] 2024.10.22 01:50:52.817530 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1047_1047_0, 202410_1048_1048_0, 202410_1049_1049_0, 202410_1050_1050_0, 202410_1051_1051_0, 202410_1052_1052_0] 2024.10.22 01:50:52.842615 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1047_1047_0, 202410_1048_1048_0, 202410_1049_1049_0, 202410_1050_1050_0, 202410_1051_1051_0, 202410_1052_1052_0] 2024.10.22 01:50:53.393220 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 74150 2024.10.22 01:50:53.761326 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_136_136_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:50:53.958425 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:53.963993 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:50:53.987194 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(18798fb2b518b2b7e9c467b3d4080c5c) : [0, 863614]: Setting download as failed: File segment: [0, 863614], key: 18798fb2b518b2b7e9c467b3d4080c5c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:50:53.991960 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(18798fb2b518b2b7e9c467b3d4080c5c): No space left in cache to reserve 863615 bytes, will continue without cache download 2024.10.22 01:50:54.017641 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1140_1140_0 (state Temporary) 2024.10.22 01:50:54.114950 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_646_125} MergeTask::MergeProjectionsStage: Merge sorted 993489 rows, containing 4 columns (4 merged, 0 gathered) in 32.240199086 sec., 30815.225344914612 rows/sec., 462.13 KiB/sec. 2024.10.22 01:50:54.163548 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1140_1140_0 to 202410_1153_1153_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:54.168400 [ 1190 ] {} DatabaseCatalog: Execution took 378 ms. 2024.10.22 01:50:54.268980 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 74150 2024.10.22 01:50:54.318939 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1334 entries to flush up to offset 991247 2024.10.22 01:50:54.358533 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_646_125} MergedBlockOutputStream: filled checksums 202410_1_646_125 (state Temporary) 2024.10.22 01:50:54.580212 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 18798fb2b518b2b7e9c467b3d4080c5c, offset: 0, size: 0 2024.10.22 01:50:54.583620 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 18798fb2b518b2b7e9c467b3d4080c5c for removal 2024.10.22 01:50:54.587895 [ 1476 ] {} CacheMetadata: Key 18798fb2b518b2b7e9c467b3d4080c5c is removed from metadata 2024.10.22 01:50:54.599867 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/cku/suotwqdzaqkgweepwawbfznftrvtq, upload id 6755a331-9324-40b3-ab88-8585eef3da56. 2024.10.22 01:50:54.871270 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} AWSClient: Got dangerous response with successful code 200, checking its body: ' http://localhost:11111/test/common/cku/suotwqdzaqkgweepwawbfznftrvtqtestcommon/cku/suotwqdzaqkgweepwawbfznftrvtq"2eec07f15e' 2024.10.22 01:50:54.878773 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Multipart upload has completed. Details: bucket test, key common/cku/suotwqdzaqkgweepwawbfznftrvtq, upload id 6755a331-9324-40b3-ab88-8585eef3da56, Parts: 3 2024.10.22 01:50:54.929374 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2d023c5b0df17a8362683d072e6260ab, offset: 0, size: 0 2024.10.22 01:50:54.935699 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 2d023c5b0df17a8362683d072e6260ab for removal 2024.10.22 01:50:54.949183 [ 1476 ] {} CacheMetadata: Key 2d023c5b0df17a8362683d072e6260ab is removed from metadata 2024.10.22 01:50:55.107341 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d9c5fe7653874480d13eb250620f5bdd, offset: 0, size: 0 2024.10.22 01:50:55.109272 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d9c5fe7653874480d13eb250620f5bdd for removal 2024.10.22 01:50:55.113652 [ 1476 ] {} CacheMetadata: Key d9c5fe7653874480d13eb250620f5bdd is removed from metadata 2024.10.22 01:50:55.143271 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/cku/suotwqdzaqkgweepwawbfznftrvtq, upload id 6755a331-9324-40b3-ab88-8585eef3da56. 2024.10.22 01:50:55.239679 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} AWSClient: Got dangerous response with successful code 200, checking its body: ' http://localhost:11111/test/common/khf/zlbzidyddpzgjxcnskafirglnaristestcommon/khf/zlbzidyddpzgjxcnskafirglnaris"8ea315d8da' 2024.10.22 01:50:55.280863 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d6cdff6104ed493a25c0fbaf0a76f64d, offset: 0, size: 0 2024.10.22 01:50:55.283747 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d6cdff6104ed493a25c0fbaf0a76f64d for removal 2024.10.22 01:50:55.295294 [ 1476 ] {} CacheMetadata: Key d6cdff6104ed493a25c0fbaf0a76f64d is removed from metadata 2024.10.22 01:50:55.399194 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8d27870e158bd4f1963b84844c38f6bd, offset: 0, size: 0 2024.10.22 01:50:55.403437 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8d27870e158bd4f1963b84844c38f6bd for removal 2024.10.22 01:50:55.409637 [ 1476 ] {} CacheMetadata: Key 8d27870e158bd4f1963b84844c38f6bd is removed from metadata 2024.10.22 01:50:55.511305 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ocu/rrilnxmbhtaihtbopyrzcjljoedqm. 2024.10.22 01:50:55.560953 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:55.563986 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:50:55.566718 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 71b2fe73bb175b10c89d55dd990feb3d, offset: 0, size: 0 2024.10.22 01:50:55.568318 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 71b2fe73bb175b10c89d55dd990feb3d for removal 2024.10.22 01:50:55.571512 [ 1476 ] {} CacheMetadata: Key 71b2fe73bb175b10c89d55dd990feb3d is removed from metadata 2024.10.22 01:50:55.638901 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f95524f22e27b3bd3946f87396cbf07f, offset: 0, size: 0 2024.10.22 01:50:55.646045 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f95524f22e27b3bd3946f87396cbf07f for removal 2024.10.22 01:50:55.654060 [ 1476 ] {} CacheMetadata: Key f95524f22e27b3bd3946f87396cbf07f is removed from metadata 2024.10.22 01:50:55.752888 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_647_647_0 (state Temporary) 2024.10.22 01:50:55.754184 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ocu/rrilnxmbhtaihtbopyrzcjljoedqm. 2024.10.22 01:50:55.757030 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mwb/wcsthspxdcwhoiavzscyfkpndonfu. 2024.10.22 01:50:55.836234 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 17c54cd28559ba947e530ef27b487d2d, offset: 0, size: 0 2024.10.22 01:50:55.840533 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 17c54cd28559ba947e530ef27b487d2d for removal 2024.10.22 01:50:55.844695 [ 1476 ] {} CacheMetadata: Key 17c54cd28559ba947e530ef27b487d2d is removed from metadata 2024.10.22 01:50:55.849457 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_647_647_0 to 202410_654_654_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:55.859672 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 10.23 MiB, will set to 1.74 GiB (RSS), difference: 6.80 MiB 2024.10.22 01:50:55.924518 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 991247 2024.10.22 01:50:55.962372 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: adc28596f83bc528d9d4d301a6b2c896, offset: 0, size: 0 2024.10.22 01:50:55.966356 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key adc28596f83bc528d9d4d301a6b2c896 for removal 2024.10.22 01:50:55.969290 [ 1476 ] {} CacheMetadata: Key adc28596f83bc528d9d4d301a6b2c896 is removed from metadata 2024.10.22 01:50:55.974604 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mwb/wcsthspxdcwhoiavzscyfkpndonfu. 2024.10.22 01:50:56.144260 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d25d8f83156d8e6492d6d906d130570c, offset: 0, size: 0 2024.10.22 01:50:56.147918 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d25d8f83156d8e6492d6d906d130570c for removal 2024.10.22 01:50:56.150885 [ 1476 ] {} CacheMetadata: Key d25d8f83156d8e6492d6d906d130570c is removed from metadata 2024.10.22 01:50:56.452405 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:50:56.543422 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ccced49eb1b2e1798b8643c4ec57aa54, offset: 0, size: 0 2024.10.22 01:50:56.550528 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ccced49eb1b2e1798b8643c4ec57aa54 for removal 2024.10.22 01:50:56.559191 [ 1476 ] {} CacheMetadata: Key ccced49eb1b2e1798b8643c4ec57aa54 is removed from metadata 2024.10.22 01:50:56.561214 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pqf/vyfqqiicweffbarajxrsvwavaodxc. 2024.10.22 01:50:56.625760 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b530611f0b23313f64ede9cc2e532194, offset: 0, size: 0 2024.10.22 01:50:56.629558 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b530611f0b23313f64ede9cc2e532194 for removal 2024.10.22 01:50:56.635719 [ 1476 ] {} CacheMetadata: Key b530611f0b23313f64ede9cc2e532194 is removed from metadata 2024.10.22 01:50:56.677758 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_646_125/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:50:56.952181 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5594f1a3deaaf7d955a8a63015de2dae, offset: 0, size: 0 2024.10.22 01:50:56.954665 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5594f1a3deaaf7d955a8a63015de2dae for removal 2024.10.22 01:50:56.958352 [ 1476 ] {} CacheMetadata: Key 5594f1a3deaaf7d955a8a63015de2dae is removed from metadata 2024.10.22 01:50:56.977742 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pqf/vyfqqiicweffbarajxrsvwavaodxc. 2024.10.22 01:50:57.079465 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8ce6bb802f2a9a49bca2f164ca348145, offset: 0, size: 0 2024.10.22 01:50:57.084173 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8ce6bb802f2a9a49bca2f164ca348145 for removal 2024.10.22 01:50:57.092360 [ 1476 ] {} CacheMetadata: Key 8ce6bb802f2a9a49bca2f164ca348145 is removed from metadata 2024.10.22 01:50:57.166833 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ed71bb3d1639813d7b8d6c4390183154, offset: 0, size: 0 2024.10.22 01:50:57.170651 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ed71bb3d1639813d7b8d6c4390183154 for removal 2024.10.22 01:50:57.174577 [ 1476 ] {} CacheMetadata: Key ed71bb3d1639813d7b8d6c4390183154 is removed from metadata 2024.10.22 01:50:57.175179 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qvk/axjxmtphfcibbqolldznjigcmcxdg. 2024.10.22 01:50:57.263473 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8c63c007df184e84b6da0dc0ace4a078, offset: 0, size: 0 2024.10.22 01:50:57.266830 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8c63c007df184e84b6da0dc0ace4a078 for removal 2024.10.22 01:50:57.270878 [ 1476 ] {} CacheMetadata: Key 8c63c007df184e84b6da0dc0ace4a078 is removed from metadata 2024.10.22 01:50:57.331297 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 21839aabc968578f6ecd46ea20129a16, offset: 0, size: 0 2024.10.22 01:50:57.336066 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 21839aabc968578f6ecd46ea20129a16 for removal 2024.10.22 01:50:57.341712 [ 1476 ] {} CacheMetadata: Key 21839aabc968578f6ecd46ea20129a16 is removed from metadata 2024.10.22 01:50:57.358264 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qvk/axjxmtphfcibbqolldznjigcmcxdg. 2024.10.22 01:50:57.430462 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1a41da72912ad243a87a79ddf1d299a8, offset: 0, size: 0 2024.10.22 01:50:57.438889 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1a41da72912ad243a87a79ddf1d299a8 for removal 2024.10.22 01:50:57.445484 [ 1476 ] {} CacheMetadata: Key 1a41da72912ad243a87a79ddf1d299a8 is removed from metadata 2024.10.22 01:50:57.448925 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_136_136_0 to 202410_139_139_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:57.634606 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 02df69b053a8cf9880d45def16c3a517, offset: 0, size: 0 2024.10.22 01:50:57.638687 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 02df69b053a8cf9880d45def16c3a517 for removal 2024.10.22 01:50:57.643488 [ 1476 ] {} CacheMetadata: Key 02df69b053a8cf9880d45def16c3a517 is removed from metadata 2024.10.22 01:50:57.644668 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xxb/hdhihkzgphhpvkqfplouaswrlwmfo. 2024.10.22 01:50:57.637835 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_646_125} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_646_125 to 202410_1_646_125 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:57.677209 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_646_125} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_640_124, 202410_641_646_1] -> 202410_1_646_125 2024.10.22 01:50:57.705581 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:50:57.760955 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 10.69 MiB, will set to 1.74 GiB (RSS), difference: 14.17 MiB 2024.10.22 01:50:57.797374 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5c2ad3b96e90900db108d9065bd616d0, offset: 0, size: 0 2024.10.22 01:50:57.799399 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5c2ad3b96e90900db108d9065bd616d0 for removal 2024.10.22 01:50:57.801728 [ 1476 ] {} CacheMetadata: Key 5c2ad3b96e90900db108d9065bd616d0 is removed from metadata 2024.10.22 01:50:58.058407 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a0a4f5ed4a694b52cedd35e4ad77bbfb, offset: 0, size: 0 2024.10.22 01:50:58.060060 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a0a4f5ed4a694b52cedd35e4ad77bbfb for removal 2024.10.22 01:50:58.062878 [ 1476 ] {} CacheMetadata: Key a0a4f5ed4a694b52cedd35e4ad77bbfb is removed from metadata 2024.10.22 01:50:58.076266 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xxb/hdhihkzgphhpvkqfplouaswrlwmfo. 2024.10.22 01:50:58.141097 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4f352059417ed946cab52c91559f0627, offset: 0, size: 0 2024.10.22 01:50:58.145370 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 4f352059417ed946cab52c91559f0627 for removal 2024.10.22 01:50:58.153360 [ 1476 ] {} CacheMetadata: Key 4f352059417ed946cab52c91559f0627 is removed from metadata 2024.10.22 01:50:58.208526 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2a655a65a8c48b1b186363561eb204ea, offset: 0, size: 0 2024.10.22 01:50:58.213609 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 2a655a65a8c48b1b186363561eb204ea for removal 2024.10.22 01:50:58.219680 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nsh/nsufyjdwjbzxsnxzfdbfeetejumbo. 2024.10.22 01:50:58.220070 [ 1476 ] {} CacheMetadata: Key 2a655a65a8c48b1b186363561eb204ea is removed from metadata 2024.10.22 01:50:58.294881 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 111 entries to flush up to offset 74261 2024.10.22 01:50:58.407863 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9728596eb88f5a798d1bc0698f21cbbe, offset: 0, size: 0 2024.10.22 01:50:58.412520 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9728596eb88f5a798d1bc0698f21cbbe for removal 2024.10.22 01:50:58.416777 [ 1476 ] {} CacheMetadata: Key 9728596eb88f5a798d1bc0698f21cbbe is removed from metadata 2024.10.22 01:50:58.471395 [ 1169 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:50:58.478885 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: dd24ee55601a9242a4339a2931da681c, offset: 0, size: 0 2024.10.22 01:50:58.482713 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key dd24ee55601a9242a4339a2931da681c for removal 2024.10.22 01:50:58.498833 [ 1169 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:50:58.517635 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nsh/nsufyjdwjbzxsnxzfdbfeetejumbo. 2024.10.22 01:50:58.580578 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 68b4974d08899fa686583f5ed8468608, offset: 0, size: 0 2024.10.22 01:50:58.584612 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 68b4974d08899fa686583f5ed8468608 for removal 2024.10.22 01:50:58.654212 [ 1476 ] {} CacheMetadata: Key 68b4974d08899fa686583f5ed8468608 is removed from metadata 2024.10.22 01:50:58.659619 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7c482176003bd02037dba9eac8337b49, offset: 0, size: 0 2024.10.22 01:50:58.660542 [ 1476 ] {} CacheMetadata: Key dd24ee55601a9242a4339a2931da681c is removed from metadata 2024.10.22 01:50:58.662413 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7c482176003bd02037dba9eac8337b49 for removal 2024.10.22 01:50:58.665379 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nxa/juvbfuwgugjeefcfufvwxlojxzfbn. 2024.10.22 01:50:58.666964 [ 1476 ] {} CacheMetadata: Key 7c482176003bd02037dba9eac8337b49 is removed from metadata 2024.10.22 01:50:58.706573 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d1d912b816a61728559f458ddbf1688f, offset: 0, size: 0 2024.10.22 01:50:58.709939 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d1d912b816a61728559f458ddbf1688f for removal 2024.10.22 01:50:58.717175 [ 1476 ] {} CacheMetadata: Key d1d912b816a61728559f458ddbf1688f is removed from metadata 2024.10.22 01:50:58.867432 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c169f75f7b37ad446191050ba4de8bbe, offset: 0, size: 0 2024.10.22 01:50:58.869075 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c169f75f7b37ad446191050ba4de8bbe for removal 2024.10.22 01:50:58.873459 [ 1476 ] {} CacheMetadata: Key c169f75f7b37ad446191050ba4de8bbe is removed from metadata 2024.10.22 01:50:58.904664 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nxa/juvbfuwgugjeefcfufvwxlojxzfbn. 2024.10.22 01:50:58.954303 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f90074af79542f3d8c17805a08f984b7, offset: 0, size: 0 2024.10.22 01:50:58.956744 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f90074af79542f3d8c17805a08f984b7 for removal 2024.10.22 01:50:58.960990 [ 1476 ] {} CacheMetadata: Key f90074af79542f3d8c17805a08f984b7 is removed from metadata 2024.10.22 01:50:58.965835 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:50:58.969409 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:50:59.128424 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cf21a592f69761aebb973eca63aca0fe, offset: 0, size: 0 2024.10.22 01:50:59.131790 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cf21a592f69761aebb973eca63aca0fe for removal 2024.10.22 01:50:59.138141 [ 1476 ] {} CacheMetadata: Key cf21a592f69761aebb973eca63aca0fe is removed from metadata 2024.10.22 01:50:59.140967 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hhf/laqpseenvemgijhkptekgpxvmkmal. 2024.10.22 01:50:59.183070 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1141_1141_0 (state Temporary) 2024.10.22 01:50:59.194305 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cc26314d5edf6d6b65e81bff64f64518, offset: 0, size: 0 2024.10.22 01:50:59.203660 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cc26314d5edf6d6b65e81bff64f64518 for removal 2024.10.22 01:50:59.207575 [ 1476 ] {} CacheMetadata: Key cc26314d5edf6d6b65e81bff64f64518 is removed from metadata 2024.10.22 01:50:59.280335 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1141_1141_0 to 202410_1154_1154_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:50:59.373497 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 74261 2024.10.22 01:50:59.486608 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3d09ef420d6bfa021bfeae762e1b5088, offset: 0, size: 0 2024.10.22 01:50:59.499876 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3d09ef420d6bfa021bfeae762e1b5088 for removal 2024.10.22 01:50:59.533246 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hhf/laqpseenvemgijhkptekgpxvmkmal. 2024.10.22 01:50:59.608468 [ 1476 ] {} CacheMetadata: Key 3d09ef420d6bfa021bfeae762e1b5088 is removed from metadata 2024.10.22 01:50:59.640604 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/wgg/ncjwzhbuhdghjcqtvqdaeccrxfncb. 2024.10.22 01:50:59.843388 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7c8cb6b0ebccd928fff4f7f84f431530, offset: 0, size: 0 2024.10.22 01:50:59.845155 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7c8cb6b0ebccd928fff4f7f84f431530 for removal 2024.10.22 01:50:59.849379 [ 1476 ] {} CacheMetadata: Key 7c8cb6b0ebccd928fff4f7f84f431530 is removed from metadata 2024.10.22 01:50:59.877580 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 10305 2024.10.22 01:50:59.912569 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 7.12 MiB, will set to 1.73 GiB (RSS), difference: 10.47 MiB 2024.10.22 01:51:00.002457 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5e72ee16f6455ae9550450893e41673a, offset: 0, size: 0 2024.10.22 01:51:00.008940 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5e72ee16f6455ae9550450893e41673a for removal 2024.10.22 01:51:00.012287 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 45 entries to flush up to offset 75556 2024.10.22 01:51:00.013411 [ 1476 ] {} CacheMetadata: Key 5e72ee16f6455ae9550450893e41673a is removed from metadata 2024.10.22 01:51:00.031717 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/wgg/ncjwzhbuhdghjcqtvqdaeccrxfncb. 2024.10.22 01:51:00.304076 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:00.307943 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:00.364814 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_676_676_0 (state Temporary) 2024.10.22 01:51:00.373639 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:00.379212 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:00.446752 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_664_664_0 (state Temporary) 2024.10.22 01:51:00.697058 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_676_676_0 to 202410_689_689_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:00.705217 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_664_664_0 to 202410_670_670_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:00.780204 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 75556 2024.10.22 01:51:00.795105 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10305 2024.10.22 01:51:00.838798 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:51:00.935684 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:51:01.257292 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5445 2024.10.22 01:51:01.357009 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 764 bytes for 5e72ee16f6455ae9550450893e41673a:0) 2024.10.22 01:51:01.361027 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bb47cbdc3b909290cce39e177c9901be, offset: 0, size: 1318 2024.10.22 01:51:01.365191 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bb47cbdc3b909290cce39e177c9901be for removal 2024.10.22 01:51:01.471012 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f4742b33c06627f72d41aabe53687fa2, offset: 0, size: 4 2024.10.22 01:51:01.474815 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f4742b33c06627f72d41aabe53687fa2 for removal 2024.10.22 01:51:01.479568 [ 1476 ] {} CacheMetadata: Key bb47cbdc3b909290cce39e177c9901be is removed from metadata 2024.10.22 01:51:01.479664 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9a3a0bd3f118cd7526e869ebd64e09ca, offset: 0, size: 4 2024.10.22 01:51:01.485585 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9a3a0bd3f118cd7526e869ebd64e09ca for removal 2024.10.22 01:51:01.491517 [ 1476 ] {} CacheMetadata: Key 9a3a0bd3f118cd7526e869ebd64e09ca is removed from metadata 2024.10.22 01:51:01.508894 [ 1476 ] {} CacheMetadata: Key f4742b33c06627f72d41aabe53687fa2 is removed from metadata 2024.10.22 01:51:01.718645 [ 1730 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ugz/fkaqfdcfgexwgsbvtpkfyzimnulbq, size 57 2024.10.22 01:51:01.923483 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_606_20, 202410_607_607_0, 202410_608_608_0, 202410_609_609_0, 202410_610_610_0, 202410_611_611_0] 2024.10.22 01:51:02.031781 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_606_20, 202410_607_607_0, 202410_608_608_0, 202410_609_609_0, 202410_610_610_0, 202410_611_611_0] 2024.10.22 01:51:02.094297 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_606_20, 202410_607_607_0, 202410_608_608_0, 202410_609_609_0, 202410_610_610_0, 202410_611_611_0] 2024.10.22 01:51:02.351351 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:51:02.377362 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:51:02.399896 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:51:02.431740 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:51:02.448073 [ 1212 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 203 ms. 2024.10.22 01:51:02.456350 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:51:02.476439 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7c8cb6b0ebccd928fff4f7f84f431530) : [0, 1337555]: Setting download as failed: File segment: [0, 1337555], key: 7c8cb6b0ebccd928fff4f7f84f431530, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:02.477733 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(7c8cb6b0ebccd928fff4f7f84f431530): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:51:02.568694 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:51:02.705674 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:51:02.728294 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column SendTiming 2024.10.22 01:51:02.815892 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/dnd/mdvxhfatfeleyprlfonvshhvfhzaj (LogSeriesLimiter: on interval from 2024-10-22 01:50:48 to 2024-10-22 01:51:02 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:51:02.819712 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7c8cb6b0ebccd928fff4f7f84f431530, offset: 0, size: 0 2024.10.22 01:51:02.875317 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7c8cb6b0ebccd928fff4f7f84f431530 for removal 2024.10.22 01:51:02.878520 [ 1476 ] {} CacheMetadata: Key 7c8cb6b0ebccd928fff4f7f84f431530 is removed from metadata 2024.10.22 01:51:02.959428 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 63176e9bcb40909575b0f70e468936ba, offset: 0, size: 0 2024.10.22 01:51:02.962506 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 63176e9bcb40909575b0f70e468936ba for removal 2024.10.22 01:51:02.975050 [ 1476 ] {} CacheMetadata: Key 63176e9bcb40909575b0f70e468936ba is removed from metadata 2024.10.22 01:51:02.985401 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1549 entries to flush up to offset 992796 2024.10.22 01:51:03.134891 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9fa83dbc4668d5b7e8e1ed099cafde02, offset: 0, size: 0 2024.10.22 01:51:03.137175 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9fa83dbc4668d5b7e8e1ed099cafde02 for removal 2024.10.22 01:51:03.143151 [ 1476 ] {} CacheMetadata: Key 9fa83dbc4668d5b7e8e1ed099cafde02 is removed from metadata 2024.10.22 01:51:03.388518 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 103 entries to flush up to offset 74364 2024.10.22 01:51:03.866906 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:03.869221 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:03.972556 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:03.974963 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:04.071899 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 767 bytes for 9fa83dbc4668d5b7e8e1ed099cafde02:0) 2024.10.22 01:51:04.074677 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a5c8cba1bd1f83e9ab902c7ad3a47a0b, offset: 0, size: 728 2024.10.22 01:51:04.084849 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a5c8cba1bd1f83e9ab902c7ad3a47a0b for removal 2024.10.22 01:51:04.093818 [ 1476 ] {} CacheMetadata: Key a5c8cba1bd1f83e9ab902c7ad3a47a0b is removed from metadata 2024.10.22 01:51:04.095992 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bf44230068eb957704f341a50be7bc70, offset: 0, size: 578 2024.10.22 01:51:04.103447 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bf44230068eb957704f341a50be7bc70 for removal 2024.10.22 01:51:04.146038 [ 1476 ] {} CacheMetadata: Key bf44230068eb957704f341a50be7bc70 is removed from metadata 2024.10.22 01:51:04.249128 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1142_1142_0 (state Temporary) 2024.10.22 01:51:04.467916 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_648_648_0 (state Temporary) 2024.10.22 01:51:04.506867 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1142_1142_0 to 202410_1155_1155_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:04.571665 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 74364 2024.10.22 01:51:04.704712 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 7.38 MiB, will set to 1.73 GiB (RSS), difference: -14.28 MiB 2024.10.22 01:51:04.769564 [ 1209 ] {} DatabaseCatalog: Execution took 588 ms. 2024.10.22 01:51:04.977596 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_648_648_0 to 202410_655_655_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:05.022664 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 992796 2024.10.22 01:51:05.155390 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_593_593_0, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0, 202410_597_597_0, 202410_598_598_0] 2024.10.22 01:51:05.158985 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_593_593_0, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0, 202410_597_597_0, 202410_598_598_0] 2024.10.22 01:51:05.187951 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_593_593_0, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0, 202410_597_597_0, 202410_598_598_0] 2024.10.22 01:51:05.332596 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(63176e9bcb40909575b0f70e468936ba) : [0, 1300758]: Setting download as failed: File segment: [0, 1300758], key: 63176e9bcb40909575b0f70e468936ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:05.334133 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(63176e9bcb40909575b0f70e468936ba): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:51:05.550208 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 63176e9bcb40909575b0f70e468936ba, offset: 0, size: 0 2024.10.22 01:51:05.552554 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 63176e9bcb40909575b0f70e468936ba for removal 2024.10.22 01:51:05.555813 [ 1476 ] {} CacheMetadata: Key 63176e9bcb40909575b0f70e468936ba is removed from metadata 2024.10.22 01:51:05.558145 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kqy/uuqadegslbwgkjkxxswtczyehjilh. 2024.10.22 01:51:05.831540 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6d2e7d37c977b7e001ff81a38968f7c7, offset: 0, size: 0 2024.10.22 01:51:05.833227 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6d2e7d37c977b7e001ff81a38968f7c7 for removal 2024.10.22 01:51:05.837479 [ 1476 ] {} CacheMetadata: Key 6d2e7d37c977b7e001ff81a38968f7c7 is removed from metadata 2024.10.22 01:51:05.863232 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_503_103, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0, 202410_507_507_0, 202410_508_508_0] 2024.10.22 01:51:05.866352 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_503_103, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0, 202410_507_507_0, 202410_508_508_0] 2024.10.22 01:51:06.000855 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cee83f6db1d1f0b14199b4490f242333, offset: 0, size: 0 2024.10.22 01:51:06.004265 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cee83f6db1d1f0b14199b4490f242333 for removal 2024.10.22 01:51:06.005801 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_503_103, 202410_504_504_0, 202410_505_505_0, 202410_506_506_0, 202410_507_507_0, 202410_508_508_0] 2024.10.22 01:51:06.007820 [ 1476 ] {} CacheMetadata: Key cee83f6db1d1f0b14199b4490f242333 is removed from metadata 2024.10.22 01:51:06.017693 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kqy/uuqadegslbwgkjkxxswtczyehjilh. 2024.10.22 01:51:06.659914 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(527de1273fa8dee69d244a8d5ddf44f4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 527de1273fa8dee69d244a8d5ddf44f4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:51:06.668650 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 466500, file segment info: File segment: [0, 33554431], key: 527de1273fa8dee69d244a8d5ddf44f4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 01:51:06.671989 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:51:07.314525 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 7.59 MiB, will set to 1.73 GiB (RSS), difference: 10.93 MiB 2024.10.22 01:51:07.675869 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 565 2024.10.22 01:51:07.780827 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 522 bytes for 772497ef3004e3e7f6ed9a775069dd47:0) 2024.10.22 01:51:07.784110 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0ea9232faefcdd705877811b32e2dcd9, offset: 0, size: 29886 2024.10.22 01:51:07.793614 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 0ea9232faefcdd705877811b32e2dcd9 for removal 2024.10.22 01:51:07.801009 [ 1476 ] {} CacheMetadata: Key 0ea9232faefcdd705877811b32e2dcd9 is removed from metadata 2024.10.22 01:51:07.856564 [ 1673 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/znm/tushejfdwcrxtxmtesrwqxplirgyi, size 466500 2024.10.22 01:51:08.028436 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:08.033050 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:08.295291 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 66 entries to flush up to offset 75622 2024.10.22 01:51:08.301658 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 4 entries to flush up to offset 10309 2024.10.22 01:51:08.306941 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_547_547_0 (state Temporary) 2024.10.22 01:51:08.412113 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:08.421160 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:08.497787 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_547_547_0 to 202410_557_557_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:08.567055 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_677_677_0 (state Temporary) 2024.10.22 01:51:08.562383 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Title 2024.10.22 01:51:08.596205 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Title 2024.10.22 01:51:08.629467 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_677_677_0 to 202410_690_690_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:08.671416 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 565 2024.10.22 01:51:08.721312 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:08.814981 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 5487 2024.10.22 01:51:08.831393 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(6d2e7d37c977b7e001ff81a38968f7c7) : [0, 641309]: Setting download as failed: File segment: [0, 641309], key: 6d2e7d37c977b7e001ff81a38968f7c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:08.865242 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 74423 2024.10.22 01:51:08.868654 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:08.869632 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(6d2e7d37c977b7e001ff81a38968f7c7): No space left in cache to reserve 641310 bytes, will continue without cache download 2024.10.22 01:51:08.927707 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_665_665_0 (state Temporary) 2024.10.22 01:51:08.931740 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ehr/kjgsncfkstvdxtztourucumoekghr. 2024.10.22 01:51:08.942999 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10309 2024.10.22 01:51:08.972161 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_665_665_0 to 202410_671_671_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:09.098480 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Title 2024.10.22 01:51:09.120295 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Title 2024.10.22 01:51:09.137453 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 75622 2024.10.22 01:51:09.155636 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Title 2024.10.22 01:51:09.193480 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Title 2024.10.22 01:51:09.234601 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6d2e7d37c977b7e001ff81a38968f7c7, offset: 0, size: 0 2024.10.22 01:51:09.234416 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Title 2024.10.22 01:51:09.240240 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6d2e7d37c977b7e001ff81a38968f7c7 for removal 2024.10.22 01:51:09.246812 [ 1476 ] {} CacheMetadata: Key 6d2e7d37c977b7e001ff81a38968f7c7 is removed from metadata 2024.10.22 01:51:09.251486 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/bds/qezqtbfndhmavcemljodlcgdisyxe. 2024.10.22 01:51:09.315788 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:09.321027 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:09.357579 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qjy/ynbgwzltucfosnaxkvyfequqbjehn (LogSeriesLimiter: on interval from 2024-10-22 01:51:02 to 2024-10-22 01:51:09 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:51:09.405560 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d14f1f70c4c8ce2814cc91992ea41234, offset: 0, size: 0 2024.10.22 01:51:09.408574 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d14f1f70c4c8ce2814cc91992ea41234 for removal 2024.10.22 01:51:09.416306 [ 1476 ] {} CacheMetadata: Key d14f1f70c4c8ce2814cc91992ea41234 is removed from metadata 2024.10.22 01:51:09.501287 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1143_1143_0 (state Temporary) 2024.10.22 01:51:09.663495 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ab05bf47e3493b1c04171bd4c6e09037, offset: 0, size: 0 2024.10.22 01:51:09.664434 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ab05bf47e3493b1c04171bd4c6e09037 for removal 2024.10.22 01:51:09.668236 [ 1476 ] {} CacheMetadata: Key ab05bf47e3493b1c04171bd4c6e09037 is removed from metadata 2024.10.22 01:51:09.672429 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/bds/qezqtbfndhmavcemljodlcgdisyxe. 2024.10.22 01:51:09.747675 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1143_1143_0 to 202410_1156_1156_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:09.838479 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 74423 2024.10.22 01:51:09.915238 [ 1261 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1151_11 to 202410_1156_1156_0 2024.10.22 01:51:09.925484 [ 1261 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:09.930475 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:09.970186 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1151_11 to 202410_1156_1156_0 into Compact with storage Full 2024.10.22 01:51:10.015794 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:51:10.042886 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1151_11, total 3408 rows starting from the beginning of the part 2024.10.22 01:51:10.186691 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1152_1152_0, total 98 rows starting from the beginning of the part 2024.10.22 01:51:10.340971 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1153_1153_0, total 68 rows starting from the beginning of the part 2024.10.22 01:51:10.457203 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1154_1154_0, total 111 rows starting from the beginning of the part 2024.10.22 01:51:10.484632 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1155_1155_0, total 103 rows starting from the beginning of the part 2024.10.22 01:51:10.550136 [ 1244 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 5 parts from 202410_135_135_0 to 202410_139_139_0 2024.10.22 01:51:10.564214 [ 1244 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:10.575662 [ 1244 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:10.618616 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_135_139_1} MergeTask::PrepareStage: Merging 5 parts: from 202410_135_135_0 to 202410_139_139_0 into Compact with storage Full 2024.10.22 01:51:10.619254 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.97 GiB, free memory in arenas 6.86 MiB, will set to 1.73 GiB (RSS), difference: 139.47 MiB 2024.10.22 01:51:10.836737 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:51:10.838439 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1156_1156_0, total 59 rows starting from the beginning of the part 2024.10.22 01:51:10.943315 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:51:11.499596 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(d14f1f70c4c8ce2814cc91992ea41234) : [0, 601453]: Setting download as failed: File segment: [0, 601453], key: d14f1f70c4c8ce2814cc91992ea41234, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:11.501426 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(d14f1f70c4c8ce2814cc91992ea41234): No space left in cache to reserve 601454 bytes, will continue without cache download 2024.10.22 01:51:11.825454 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d14f1f70c4c8ce2814cc91992ea41234, offset: 0, size: 0 2024.10.22 01:51:11.827692 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d14f1f70c4c8ce2814cc91992ea41234 for removal 2024.10.22 01:51:11.830863 [ 1476 ] {} CacheMetadata: Key d14f1f70c4c8ce2814cc91992ea41234 is removed from metadata 2024.10.22 01:51:11.835382 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lur/hzrighiudyvdrbomqxybujugrgrun. 2024.10.22 01:51:11.877469 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0091f0081388ddb6e4a4f8ed442bcd1b, offset: 0, size: 0 2024.10.22 01:51:11.880118 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0091f0081388ddb6e4a4f8ed442bcd1b for removal 2024.10.22 01:51:11.885762 [ 1476 ] {} CacheMetadata: Key 0091f0081388ddb6e4a4f8ed442bcd1b is removed from metadata 2024.10.22 01:51:11.972790 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ded603209da8b8d3b2b89a2313cbbf01, offset: 0, size: 0 2024.10.22 01:51:11.975513 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ded603209da8b8d3b2b89a2313cbbf01 for removal 2024.10.22 01:51:11.986996 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lur/hzrighiudyvdrbomqxybujugrgrun. 2024.10.22 01:51:12.034216 [ 1476 ] {} CacheMetadata: Key ded603209da8b8d3b2b89a2313cbbf01 is removed from metadata 2024.10.22 01:51:12.059674 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 967 entries to flush up to offset 993763 2024.10.22 01:51:12.541999 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(df0f7b02353c159b75ae5ab4232f6eed) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: df0f7b02353c159b75ae5ab4232f6eed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:51:12.545772 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: df0f7b02353c159b75ae5ab4232f6eed, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:51:12.548769 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:51:12.946443 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} MergeTask::MergeProjectionsStage: Merge sorted 3847 rows, containing 13 columns (13 merged, 0 gathered) in 2.980295958 sec., 1290.811400684388 rows/sec., 229.65 KiB/sec. 2024.10.22 01:51:13.025062 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} MergedBlockOutputStream: filled checksums 202410_1096_1156_12 (state Temporary) 2024.10.22 01:51:13.168707 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:13.175108 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:13.243185 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1156_12 to 202410_1096_1156_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:13.259242 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1156_12} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1151_11, 202410_1156_1156_0] -> 202410_1096_1156_12 2024.10.22 01:51:13.400016 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:51:13.406468 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_649_649_0 (state Temporary) 2024.10.22 01:51:13.507607 [ 1158 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:51:13.531651 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_649_649_0 to 202410_656_656_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:13.535912 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(61d270474d68163096fd6a7a00b65f74) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 61d270474d68163096fd6a7a00b65f74, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:51:13.537677 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 61d270474d68163096fd6a7a00b65f74, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:51:13.547890 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:51:13.559846 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 993763 2024.10.22 01:51:13.569560 [ 1153 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_646_125 to 202410_647_652_1 2024.10.22 01:51:13.619864 [ 1153 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:13.629077 [ 1153 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:13.643062 [ 1158 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:51:13.655567 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_652_126} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_646_125 to 202410_647_652_1 into Wide with storage Full 2024.10.22 01:51:13.703542 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_652_126} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:51:13.710996 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_652_126} MergeTreeSequentialSource: Reading 123 marks from part 202410_1_646_125, total 993489 rows starting from the beginning of the part 2024.10.22 01:51:13.771279 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_652_126} MergeTreeSequentialSource: Reading 3 marks from part 202410_647_652_1, total 8859 rows starting from the beginning of the part 2024.10.22 01:51:13.847179 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 74516 2024.10.22 01:51:14.123188 [ 1159 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 01:51:14.373436 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:14.376288 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:14.437323 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_626_125, 202410_627_627_0, 202410_628_628_0, 202410_629_629_0, 202410_630_630_0, 202410_631_631_0] 2024.10.22 01:51:14.440954 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_626_125, 202410_627_627_0, 202410_628_628_0, 202410_629_629_0, 202410_630_630_0, 202410_631_631_0] 2024.10.22 01:51:14.468579 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_626_125, 202410_627_627_0, 202410_628_628_0, 202410_629_629_0, 202410_630_630_0, 202410_631_631_0] 2024.10.22 01:51:14.763822 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1144_1144_0 (state Temporary) 2024.10.22 01:51:14.839870 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1144_1144_0 to 202410_1157_1157_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:14.906461 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 74516 2024.10.22 01:51:15.073798 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(0091f0081388ddb6e4a4f8ed442bcd1b) : [0, 374537]: Setting download as failed: File segment: [0, 374537], key: 0091f0081388ddb6e4a4f8ed442bcd1b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:15.076154 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(0091f0081388ddb6e4a4f8ed442bcd1b): No space left in cache to reserve 374538 bytes, will continue without cache download 2024.10.22 01:51:15.866546 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0091f0081388ddb6e4a4f8ed442bcd1b, offset: 0, size: 0 2024.10.22 01:51:15.867726 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0091f0081388ddb6e4a4f8ed442bcd1b for removal 2024.10.22 01:51:15.871129 [ 1476 ] {} CacheMetadata: Key 0091f0081388ddb6e4a4f8ed442bcd1b is removed from metadata 2024.10.22 01:51:15.872075 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rvv/fylhznkzzfkehevenqiqkljlraewa. 2024.10.22 01:51:15.930394 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 394ce66a62df8c3de1e5a6f3aca7285d, offset: 0, size: 0 2024.10.22 01:51:15.936644 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 394ce66a62df8c3de1e5a6f3aca7285d for removal 2024.10.22 01:51:15.942436 [ 1476 ] {} CacheMetadata: Key 394ce66a62df8c3de1e5a6f3aca7285d is removed from metadata 2024.10.22 01:51:15.974955 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 4.94 MiB, will set to 1.75 GiB (RSS), difference: 27.73 MiB 2024.10.22 01:51:15.976158 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7f81e4ca157e9fdf27a44597f0b86045, offset: 0, size: 0 2024.10.22 01:51:15.980251 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7f81e4ca157e9fdf27a44597f0b86045 for removal 2024.10.22 01:51:15.985455 [ 1476 ] {} CacheMetadata: Key 7f81e4ca157e9fdf27a44597f0b86045 is removed from metadata 2024.10.22 01:51:16.035051 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_586_115, 202410_587_592_1] 2024.10.22 01:51:16.042810 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_586_115, 202410_587_592_1] 2024.10.22 01:51:16.074291 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_586_115, 202410_587_592_1] 2024.10.22 01:51:16.204894 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rvv/fylhznkzzfkehevenqiqkljlraewa. 2024.10.22 01:51:16.359285 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cfe62333ff881be926efc365bc7e15a4, offset: 0, size: 0 2024.10.22 01:51:16.361102 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cfe62333ff881be926efc365bc7e15a4 for removal 2024.10.22 01:51:16.365274 [ 1476 ] {} CacheMetadata: Key cfe62333ff881be926efc365bc7e15a4 is removed from metadata 2024.10.22 01:51:16.368026 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 566 2024.10.22 01:51:16.408307 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ac079411ace282d173204d4d5fd67062, offset: 0, size: 0 2024.10.22 01:51:16.412487 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ac079411ace282d173204d4d5fd67062 for removal 2024.10.22 01:51:16.419688 [ 1476 ] {} CacheMetadata: Key ac079411ace282d173204d4d5fd67062 is removed from metadata 2024.10.22 01:51:16.419918 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/juu/xasdrhhybkhansjfomnqsjyfsxhdq. 2024.10.22 01:51:16.445328 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10322 2024.10.22 01:51:16.484480 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a247986bb9524d2f7ad5b04abe81692f, offset: 0, size: 0 2024.10.22 01:51:16.488841 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a247986bb9524d2f7ad5b04abe81692f for removal 2024.10.22 01:51:16.495705 [ 1476 ] {} CacheMetadata: Key a247986bb9524d2f7ad5b04abe81692f is removed from metadata 2024.10.22 01:51:16.599619 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8de966fa81db70114fa077f83a3cab75, offset: 0, size: 0 2024.10.22 01:51:16.602002 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8de966fa81db70114fa077f83a3cab75 for removal 2024.10.22 01:51:16.609107 [ 1476 ] {} CacheMetadata: Key 8de966fa81db70114fa077f83a3cab75 is removed from metadata 2024.10.22 01:51:16.640311 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/juu/xasdrhhybkhansjfomnqsjyfsxhdq. 2024.10.22 01:51:16.654254 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 122 entries to flush up to offset 75744 2024.10.22 01:51:16.684043 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(196326e4baa77fa73a4a9deeb91a0a80) : [0, 2237845]: Setting download as failed: File segment: [0, 2237845], key: 196326e4baa77fa73a4a9deeb91a0a80, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:51:16.685475 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(196326e4baa77fa73a4a9deeb91a0a80): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:51:16.693073 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:16.699670 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:16.723278 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1c324bc4f89757f09559647065814638, offset: 0, size: 0 2024.10.22 01:51:16.741506 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1c324bc4f89757f09559647065814638 for removal 2024.10.22 01:51:16.749711 [ 1476 ] {} CacheMetadata: Key 1c324bc4f89757f09559647065814638 is removed from metadata 2024.10.22 01:51:16.757229 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_678_678_0 (state Temporary) 2024.10.22 01:51:16.819733 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_678_678_0 to 202410_691_691_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:16.859202 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:16.861503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 196326e4baa77fa73a4a9deeb91a0a80, offset: 0, size: 0 2024.10.22 01:51:16.865712 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:16.901478 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0bd680259737ef847b2f5b7212065aba, offset: 0, size: 0 2024.10.22 01:51:16.906895 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0bd680259737ef847b2f5b7212065aba for removal 2024.10.22 01:51:16.917632 [ 1476 ] {} CacheMetadata: Key 0bd680259737ef847b2f5b7212065aba is removed from metadata 2024.10.22 01:51:16.962676 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:16.967216 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 196326e4baa77fa73a4a9deeb91a0a80 for removal 2024.10.22 01:51:16.969169 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:16.969782 [ 1476 ] {} CacheMetadata: Key 196326e4baa77fa73a4a9deeb91a0a80 is removed from metadata 2024.10.22 01:51:17.005066 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10322 2024.10.22 01:51:17.029193 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/jkl/lzodacwtthrgoriormwqyhrpbattv. 2024.10.22 01:51:17.055796 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_548_548_0 (state Temporary) 2024.10.22 01:51:17.058428 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_666_666_0 (state Temporary) 2024.10.22 01:51:17.105243 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b35e437890688b4fac4a4d6d8137486d, offset: 0, size: 0 2024.10.22 01:51:17.108057 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b35e437890688b4fac4a4d6d8137486d for removal 2024.10.22 01:51:17.115092 [ 1476 ] {} CacheMetadata: Key b35e437890688b4fac4a4d6d8137486d is removed from metadata 2024.10.22 01:51:17.171753 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d30705bab8facd285ffaad566038dba3, offset: 0, size: 0 2024.10.22 01:51:17.172966 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_666_666_0 to 202410_672_672_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:17.175609 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d30705bab8facd285ffaad566038dba3 for removal 2024.10.22 01:51:17.184812 [ 1476 ] {} CacheMetadata: Key d30705bab8facd285ffaad566038dba3 is removed from metadata 2024.10.22 01:51:17.220563 [ 1190 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_667_33 to 202410_672_672_0 2024.10.22 01:51:17.249624 [ 1190 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:17.253453 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 75744 2024.10.22 01:51:17.254974 [ 1190 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:17.284807 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_667_33 to 202410_672_672_0 into Compact with storage Full 2024.10.22 01:51:17.349830 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_548_548_0 to 202410_558_558_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:17.373273 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:51:17.379018 [ 1228 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_553_113 to 202410_558_558_0 2024.10.22 01:51:17.389752 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} MergeTreeSequentialSource: Reading 3 marks from part 202410_507_667_33, total 16186 rows starting from the beginning of the part 2024.10.22 01:51:17.407830 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/jkl/lzodacwtthrgoriormwqyhrpbattv. 2024.10.22 01:51:17.439572 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_668_668_0, total 108 rows starting from the beginning of the part 2024.10.22 01:51:17.466949 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a23949621e6307f9093ef31c58d7053e, offset: 0, size: 0 2024.10.22 01:51:17.471115 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a23949621e6307f9093ef31c58d7053e for removal 2024.10.22 01:51:17.475165 [ 1476 ] {} CacheMetadata: Key a23949621e6307f9093ef31c58d7053e is removed from metadata 2024.10.22 01:51:17.505496 [ 1228 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:17.509342 [ 1228 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:17.516126 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_669_669_0, total 92 rows starting from the beginning of the part 2024.10.22 01:51:17.568640 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_553_113 to 202410_558_558_0 into Compact with storage Full 2024.10.22 01:51:17.764595 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 566 2024.10.22 01:51:17.793307 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_670_670_0, total 45 rows starting from the beginning of the part 2024.10.22 01:51:17.819598 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 09fc569b5f4319b60040cde830b470a9, offset: 0, size: 0 2024.10.22 01:51:17.821803 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 09fc569b5f4319b60040cde830b470a9 for removal 2024.10.22 01:51:17.830203 [ 1476 ] {} CacheMetadata: Key 09fc569b5f4319b60040cde830b470a9 is removed from metadata 2024.10.22 01:51:17.836259 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_671_671_0, total 66 rows starting from the beginning of the part 2024.10.22 01:51:17.870897 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_672_672_0, total 122 rows starting from the beginning of the part 2024.10.22 01:51:17.962298 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:51:17.999034 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_553_113, total 597 rows starting from the beginning of the part 2024.10.22 01:51:18.136784 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_554_554_0, total 1 rows starting from the beginning of the part 2024.10.22 01:51:18.228470 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_555_555_0, total 1 rows starting from the beginning of the part 2024.10.22 01:51:18.543895 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_556_556_0, total 1 rows starting from the beginning of the part 2024.10.22 01:51:18.650809 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_557_557_0, total 1 rows starting from the beginning of the part 2024.10.22 01:51:18.698882 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.97 GiB, free memory in arenas 9.33 MiB, will set to 1.76 GiB (RSS), difference: -1.66 MiB 2024.10.22 01:51:18.749414 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_558_558_0, total 1 rows starting from the beginning of the part 2024.10.22 01:51:18.952493 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 107 entries to flush up to offset 74623 2024.10.22 01:51:19.644113 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a50cf3b05b7fda5aebbc1b3fb630e5a0) : [0, 489542]: Setting download as failed: File segment: [0, 489542], key: a50cf3b05b7fda5aebbc1b3fb630e5a0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:51:19.648435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a50cf3b05b7fda5aebbc1b3fb630e5a0): No space left in cache to reserve 489543 bytes, will continue without cache download 2024.10.22 01:51:19.749648 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:19.754696 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:19.888471 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a50cf3b05b7fda5aebbc1b3fb630e5a0, offset: 0, size: 0 2024.10.22 01:51:19.892315 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1145_1145_0 (state Temporary) 2024.10.22 01:51:20.000432 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a50cf3b05b7fda5aebbc1b3fb630e5a0 for removal 2024.10.22 01:51:20.005680 [ 1476 ] {} CacheMetadata: Key a50cf3b05b7fda5aebbc1b3fb630e5a0 is removed from metadata 2024.10.22 01:51:20.079205 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_652_126/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:51:20.093530 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1145_1145_0 to 202410_1158_1158_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:20.299510 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 74623 2024.10.22 01:51:20.563617 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 772 entries to flush up to offset 994535 2024.10.22 01:51:20.633825 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1046_501, 202410_1047_1052_1] 2024.10.22 01:51:20.639167 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1046_501, 202410_1047_1052_1] 2024.10.22 01:51:20.762980 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1046_501, 202410_1047_1052_1] 2024.10.22 01:51:20.843497 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:51:20.845919 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:51:21.067769 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:21.075102 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:21.083024 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} MergeTask::MergeProjectionsStage: Merge sorted 16619 rows, containing 13 columns (13 merged, 0 gathered) in 3.809523475 sec., 4362.487883080967 rows/sec., 1.89 MiB/sec. 2024.10.22 01:51:21.127158 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} MergedBlockOutputStream: filled checksums 202410_507_672_34 (state Temporary) 2024.10.22 01:51:21.235074 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a23949621e6307f9093ef31c58d7053e) : [0, 1291111]: Setting download as failed: File segment: [0, 1291111], key: a23949621e6307f9093ef31c58d7053e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:21.238924 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(a23949621e6307f9093ef31c58d7053e): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:51:21.276854 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} MergeTask::MergeProjectionsStage: Merge sorted 602 rows, containing 32 columns (32 merged, 0 gathered) in 3.701608092 sec., 162.6320196622263 rows/sec., 14.86 KiB/sec. 2024.10.22 01:51:21.308251 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_650_650_0 (state Temporary) 2024.10.22 01:51:21.426856 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_650_650_0 to 202410_657_657_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:21.441155 [ 1748 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/dnd/mdvxhfatfeleyprlfonvshhvfhzaj, size 2297058 2024.10.22 01:51:21.474606 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 994535 2024.10.22 01:51:21.555618 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} MergedBlockOutputStream: filled checksums 202410_1_558_114 (state Temporary) 2024.10.22 01:51:21.566365 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_672_34 to 202410_507_672_34 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:21.570736 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_672_34} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_667_33, 202410_672_672_0] -> 202410_507_672_34 2024.10.22 01:51:21.624872 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.09 MiB. 2024.10.22 01:51:21.743159 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:51:21.763787 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:51:21.818337 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:51:21.853237 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:51:21.859157 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:51:21.870995 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:51:21.896691 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:51:21.927613 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column DNSTiming 2024.10.22 01:51:21.979663 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tya/setmktoqqnaurzllfdsodugnsvrec (LogSeriesLimiter: on interval from 2024-10-22 01:51:09 to 2024-10-22 01:51:21 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:51:22.261255 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a23949621e6307f9093ef31c58d7053e, offset: 0, size: 0 2024.10.22 01:51:22.266786 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a23949621e6307f9093ef31c58d7053e for removal 2024.10.22 01:51:22.271264 [ 1476 ] {} CacheMetadata: Key a23949621e6307f9093ef31c58d7053e is removed from metadata 2024.10.22 01:51:22.273383 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hkj/ddduvuiyzcbktmsizgpppftnlunzx. 2024.10.22 01:51:22.434925 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 03d074cf9aa4ed018d8b4bd5bd7b4f27, offset: 0, size: 0 2024.10.22 01:51:22.437326 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 03d074cf9aa4ed018d8b4bd5bd7b4f27 for removal 2024.10.22 01:51:22.441779 [ 1476 ] {} CacheMetadata: Key 03d074cf9aa4ed018d8b4bd5bd7b4f27 is removed from metadata 2024.10.22 01:51:22.451053 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1053_1053_0, 202410_1054_1054_0, 202410_1055_1055_0, 202410_1056_1056_0, 202410_1057_1057_0, 202410_1058_1058_0] 2024.10.22 01:51:22.452944 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1053_1053_0, 202410_1054_1054_0, 202410_1055_1055_0, 202410_1056_1056_0, 202410_1057_1057_0, 202410_1058_1058_0] 2024.10.22 01:51:22.483552 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4fb972563368ac623396d6b2be25b487, offset: 0, size: 0 2024.10.22 01:51:22.488058 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 4fb972563368ac623396d6b2be25b487 for removal 2024.10.22 01:51:22.493192 [ 1476 ] {} CacheMetadata: Key 4fb972563368ac623396d6b2be25b487 is removed from metadata 2024.10.22 01:51:22.503079 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hkj/ddduvuiyzcbktmsizgpppftnlunzx. 2024.10.22 01:51:22.507759 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gdl/jpcblunzqlcrgesiixeeylwbllaaw. 2024.10.22 01:51:22.549376 [ 1231 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 262 ms. 2024.10.22 01:51:22.672669 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1053_1053_0, 202410_1054_1054_0, 202410_1055_1055_0, 202410_1056_1056_0, 202410_1057_1057_0, 202410_1058_1058_0] 2024.10.22 01:51:22.758460 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b85ea7799c8c1988c1b95344fd3310d1, offset: 0, size: 0 2024.10.22 01:51:22.762795 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b85ea7799c8c1988c1b95344fd3310d1 for removal 2024.10.22 01:51:22.765280 [ 1476 ] {} CacheMetadata: Key b85ea7799c8c1988c1b95344fd3310d1 is removed from metadata 2024.10.22 01:51:22.820135 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 511d621a95ad2cdda277ba396f6c1f73, offset: 0, size: 0 2024.10.22 01:51:22.923723 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 511d621a95ad2cdda277ba396f6c1f73 for removal 2024.10.22 01:51:22.934907 [ 1476 ] {} CacheMetadata: Key 511d621a95ad2cdda277ba396f6c1f73 is removed from metadata 2024.10.22 01:51:22.943284 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gdl/jpcblunzqlcrgesiixeeylwbllaaw. 2024.10.22 01:51:23.096134 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_558_114 to 202410_1_558_114 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:23.106071 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_558_114} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_553_113, 202410_558_558_0] -> 202410_1_558_114 2024.10.22 01:51:23.203949 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:51:23.549791 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 9.32 MiB, will set to 1.76 GiB (RSS), difference: 9.97 MiB 2024.10.22 01:51:23.597620 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_135_139_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:51:23.686908 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cdf1786f1f267ae3eb6d41b6863dc21b) : [0, 37464]: Setting download as failed: File segment: [0, 37464], key: cdf1786f1f267ae3eb6d41b6863dc21b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:51:23.693550 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cdf1786f1f267ae3eb6d41b6863dc21b): No space left in cache to reserve 37465 bytes, will continue without cache download 2024.10.22 01:51:23.756138 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cdf1786f1f267ae3eb6d41b6863dc21b, offset: 0, size: 0 2024.10.22 01:51:23.757726 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cdf1786f1f267ae3eb6d41b6863dc21b for removal 2024.10.22 01:51:23.760558 [ 1476 ] {} CacheMetadata: Key cdf1786f1f267ae3eb6d41b6863dc21b is removed from metadata 2024.10.22 01:51:24.319391 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 74698 2024.10.22 01:51:24.507520 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 10341 2024.10.22 01:51:24.569637 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:51:24.773171 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 120 entries to flush up to offset 75864 2024.10.22 01:51:24.783381 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:24.788065 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:24.860889 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:24.864254 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:24.872159 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1146_1146_0 (state Temporary) 2024.10.22 01:51:24.904796 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_679_679_0 (state Temporary) 2024.10.22 01:51:25.070141 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_679_679_0 to 202410_692_692_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:25.072082 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 16 file segments (while reserving 25115 bytes for b85ea7799c8c1988c1b95344fd3310d1:0) 2024.10.22 01:51:25.077754 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1dd3826dfb9fac8451532a27a1130cad, offset: 0, size: 164 2024.10.22 01:51:25.088511 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1dd3826dfb9fac8451532a27a1130cad for removal 2024.10.22 01:51:25.098184 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0a12c07debcfa516ee8ab3181007aa27, offset: 0, size: 154 2024.10.22 01:51:25.098274 [ 1476 ] {} CacheMetadata: Key 1dd3826dfb9fac8451532a27a1130cad is removed from metadata 2024.10.22 01:51:25.104786 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0a12c07debcfa516ee8ab3181007aa27 for removal 2024.10.22 01:51:25.112331 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9fe7977d57cc99bc35fd5d753f801ffb, offset: 0, size: 208 2024.10.22 01:51:25.117013 [ 1221 ] {} DatabaseCatalog: Execution took 210 ms. 2024.10.22 01:51:25.119433 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9fe7977d57cc99bc35fd5d753f801ffb for removal 2024.10.22 01:51:25.132529 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ded603209da8b8d3b2b89a2313cbbf01, offset: 0, size: 577 2024.10.22 01:51:25.135036 [ 1200 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_687_138 to 202410_692_692_0 2024.10.22 01:51:25.136241 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ded603209da8b8d3b2b89a2313cbbf01 for removal 2024.10.22 01:51:25.137655 [ 1476 ] {} CacheMetadata: Key 9fe7977d57cc99bc35fd5d753f801ffb is removed from metadata 2024.10.22 01:51:25.146969 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 825f4973398a6752cbf9867e7bcf4486, offset: 0, size: 585 2024.10.22 01:51:25.153104 [ 1476 ] {} CacheMetadata: Key ded603209da8b8d3b2b89a2313cbbf01 is removed from metadata 2024.10.22 01:51:25.154203 [ 1200 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:25.155326 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 825f4973398a6752cbf9867e7bcf4486 for removal 2024.10.22 01:51:25.153781 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10341 2024.10.22 01:51:25.159357 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:25.161479 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ab05bf47e3493b1c04171bd4c6e09037, offset: 0, size: 583 2024.10.22 01:51:25.164226 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ab05bf47e3493b1c04171bd4c6e09037 for removal 2024.10.22 01:51:25.165384 [ 1476 ] {} CacheMetadata: Key 825f4973398a6752cbf9867e7bcf4486 is removed from metadata 2024.10.22 01:51:25.168530 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5e72ee16f6455ae9550450893e41673a, offset: 0, size: 764 2024.10.22 01:51:25.171344 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5e72ee16f6455ae9550450893e41673a for removal 2024.10.22 01:51:25.175634 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8d4af8f09f4f8c4bcb906efb387bc932, offset: 0, size: 600 2024.10.22 01:51:25.179952 [ 1476 ] {} CacheMetadata: Key 5e72ee16f6455ae9550450893e41673a is removed from metadata 2024.10.22 01:51:25.180273 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8d4af8f09f4f8c4bcb906efb387bc932 for removal 2024.10.22 01:51:25.186538 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 22e743d125216f35bde0b20534394377, offset: 0, size: 1170 2024.10.22 01:51:25.192545 [ 1476 ] {} CacheMetadata: Key 8d4af8f09f4f8c4bcb906efb387bc932 is removed from metadata 2024.10.22 01:51:25.202976 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1146_1146_0 to 202410_1159_1159_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:25.204335 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 22e743d125216f35bde0b20534394377 for removal 2024.10.22 01:51:25.213723 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cee83f6db1d1f0b14199b4490f242333, offset: 0, size: 582 2024.10.22 01:51:25.214998 [ 1476 ] {} CacheMetadata: Key ab05bf47e3493b1c04171bd4c6e09037 is removed from metadata 2024.10.22 01:51:25.219099 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cee83f6db1d1f0b14199b4490f242333 for removal 2024.10.22 01:51:25.228888 [ 1476 ] {} CacheMetadata: Key 22e743d125216f35bde0b20534394377 is removed from metadata 2024.10.22 01:51:25.229452 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9fe31d1095e5972eec844c52929b942f, offset: 0, size: 562 2024.10.22 01:51:25.232129 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_687_138 to 202410_692_692_0 into Compact with storage Full 2024.10.22 01:51:25.234778 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9fe31d1095e5972eec844c52929b942f for removal 2024.10.22 01:51:25.242305 [ 1476 ] {} CacheMetadata: Key 9fe31d1095e5972eec844c52929b942f is removed from metadata 2024.10.22 01:51:25.241678 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 480392562f4723b951800261affff950, offset: 0, size: 2509 2024.10.22 01:51:25.253116 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 480392562f4723b951800261affff950 for removal 2024.10.22 01:51:25.263118 [ 1476 ] {} CacheMetadata: Key 480392562f4723b951800261affff950 is removed from metadata 2024.10.22 01:51:25.268812 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 567 2024.10.22 01:51:25.272220 [ 1476 ] {} CacheMetadata: Key cee83f6db1d1f0b14199b4490f242333 is removed from metadata 2024.10.22 01:51:25.280378 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 74698 2024.10.22 01:51:25.261795 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9fa83dbc4668d5b7e8e1ed099cafde02, offset: 0, size: 767 2024.10.22 01:51:25.367284 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9fa83dbc4668d5b7e8e1ed099cafde02 for removal 2024.10.22 01:51:25.370723 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 37f61175a46eddb10770e06c8be7fa17, offset: 0, size: 6985 2024.10.22 01:51:25.374768 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 37f61175a46eddb10770e06c8be7fa17 for removal 2024.10.22 01:51:25.380265 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0a38fc00cb2c2261d2192578b0673ad4, offset: 0, size: 1014 2024.10.22 01:51:25.384005 [ 1476 ] {} CacheMetadata: Key 0a12c07debcfa516ee8ab3181007aa27 is removed from metadata 2024.10.22 01:51:25.386493 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0a38fc00cb2c2261d2192578b0673ad4 for removal 2024.10.22 01:51:25.393791 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ccbc9be1064a70a167fac274ee3f8748, offset: 0, size: 1037 2024.10.22 01:51:25.398266 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ccbc9be1064a70a167fac274ee3f8748 for removal 2024.10.22 01:51:25.452008 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:51:25.473255 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:25.474993 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_687_138, total 10381 rows starting from the beginning of the part 2024.10.22 01:51:25.484703 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:25.512616 [ 1476 ] {} CacheMetadata: Key ccbc9be1064a70a167fac274ee3f8748 is removed from metadata 2024.10.22 01:51:25.530267 [ 1476 ] {} CacheMetadata: Key 0a38fc00cb2c2261d2192578b0673ad4 is removed from metadata 2024.10.22 01:51:25.562470 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_667_667_0 (state Temporary) 2024.10.22 01:51:25.579879 [ 1476 ] {} CacheMetadata: Key 37f61175a46eddb10770e06c8be7fa17 is removed from metadata 2024.10.22 01:51:25.589319 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:25.596590 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:25.598658 [ 1476 ] {} CacheMetadata: Key 9fa83dbc4668d5b7e8e1ed099cafde02 is removed from metadata 2024.10.22 01:51:25.517650 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_135_139_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_135_135_0, total 43 rows starting from the beginning of the part 2024.10.22 01:51:25.633175 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_688_688_0, total 20 rows starting from the beginning of the part 2024.10.22 01:51:25.634758 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.97 GiB, free memory in arenas 9.10 MiB, will set to 1.77 GiB (RSS), difference: -10.61 MiB 2024.10.22 01:51:25.672780 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:25.681432 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:25.689695 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_689_689_0, total 10 rows starting from the beginning of the part 2024.10.22 01:51:25.738434 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_667_667_0 to 202410_673_673_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:25.760378 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_690_690_0, total 4 rows starting from the beginning of the part 2024.10.22 01:51:25.851205 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 19613c091bb646fc15ce08801a6c0d98, offset: 0, size: 0 2024.10.22 01:51:25.853552 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 19613c091bb646fc15ce08801a6c0d98 for removal 2024.10.22 01:51:25.855983 [ 1476 ] {} CacheMetadata: Key 19613c091bb646fc15ce08801a6c0d98 is removed from metadata 2024.10.22 01:51:25.911574 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 75864 2024.10.22 01:51:25.913382 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_549_549_0 (state Temporary) 2024.10.22 01:51:25.942615 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_691_691_0, total 13 rows starting from the beginning of the part 2024.10.22 01:51:25.996959 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_692_692_0, total 19 rows starting from the beginning of the part 2024.10.22 01:51:26.002613 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7b5b22cb80bd986bfbb9bfefb0f33ee7, offset: 0, size: 0 2024.10.22 01:51:26.007807 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_549_549_0 to 202410_559_559_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:26.007219 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7b5b22cb80bd986bfbb9bfefb0f33ee7 for removal 2024.10.22 01:51:26.016160 [ 1476 ] {} CacheMetadata: Key 7b5b22cb80bd986bfbb9bfefb0f33ee7 is removed from metadata 2024.10.22 01:51:26.203599 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 567 2024.10.22 01:51:26.862431 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 9 file segments (while reserving 22571 bytes for 19e1fa9ebb7c83e2068a7015ff1a95b4:0) 2024.10.22 01:51:26.866467 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b85ea7799c8c1988c1b95344fd3310d1, offset: 0, size: 25115 2024.10.22 01:51:26.869713 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b85ea7799c8c1988c1b95344fd3310d1 for removal 2024.10.22 01:51:26.873008 [ 1476 ] {} CacheMetadata: Key b85ea7799c8c1988c1b95344fd3310d1 is removed from metadata 2024.10.22 01:51:26.874202 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 511d621a95ad2cdda277ba396f6c1f73, offset: 0, size: 567 2024.10.22 01:51:26.878523 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 511d621a95ad2cdda277ba396f6c1f73 for removal 2024.10.22 01:51:26.895324 [ 1476 ] {} CacheMetadata: Key 511d621a95ad2cdda277ba396f6c1f73 is removed from metadata 2024.10.22 01:51:26.895782 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9b1f4545ff8a59683d36ff12c9142c1d, offset: 0, size: 142 2024.10.22 01:51:26.901384 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9b1f4545ff8a59683d36ff12c9142c1d for removal 2024.10.22 01:51:26.905489 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e6ca6373162a116375d8c7f7c2c4b35a, offset: 0, size: 153 2024.10.22 01:51:26.906293 [ 1476 ] {} CacheMetadata: Key 9b1f4545ff8a59683d36ff12c9142c1d is removed from metadata 2024.10.22 01:51:27.012989 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e6ca6373162a116375d8c7f7c2c4b35a for removal 2024.10.22 01:51:27.120865 [ 1476 ] {} CacheMetadata: Key e6ca6373162a116375d8c7f7c2c4b35a is removed from metadata 2024.10.22 01:51:27.120947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 09fc569b5f4319b60040cde830b470a9, offset: 0, size: 763 2024.10.22 01:51:27.127574 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 09fc569b5f4319b60040cde830b470a9 for removal 2024.10.22 01:51:27.134423 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 35d2b4b22b5be5ade4c2c6a212f86864, offset: 0, size: 142 2024.10.22 01:51:27.139110 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 35d2b4b22b5be5ade4c2c6a212f86864 for removal 2024.10.22 01:51:27.146808 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0e91f5859a49a61a06f47924e32daef5, offset: 0, size: 164 2024.10.22 01:51:27.151705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0e91f5859a49a61a06f47924e32daef5 for removal 2024.10.22 01:51:27.160927 [ 1476 ] {} CacheMetadata: Key 0e91f5859a49a61a06f47924e32daef5 is removed from metadata 2024.10.22 01:51:27.158755 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 941cb26595fc7b2772be1ba2e6d1cfa1, offset: 0, size: 5015 2024.10.22 01:51:27.170388 [ 1476 ] {} CacheMetadata: Key 35d2b4b22b5be5ade4c2c6a212f86864 is removed from metadata 2024.10.22 01:51:27.179044 [ 1476 ] {} CacheMetadata: Key 09fc569b5f4319b60040cde830b470a9 is removed from metadata 2024.10.22 01:51:27.264333 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 941cb26595fc7b2772be1ba2e6d1cfa1 for removal 2024.10.22 01:51:27.266359 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} MergeTask::MergeProjectionsStage: Merge sorted 10447 rows, containing 12 columns (12 merged, 0 gathered) in 2.060259987 sec., 5070.7192616074435 rows/sec., 807.24 KiB/sec. 2024.10.22 01:51:27.267195 [ 1476 ] {} CacheMetadata: Key 941cb26595fc7b2772be1ba2e6d1cfa1 is removed from metadata 2024.10.22 01:51:27.268708 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 62899557aacb6f32b60b283ba23207c4, offset: 0, size: 2521 2024.10.22 01:51:27.274763 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 62899557aacb6f32b60b283ba23207c4 for removal 2024.10.22 01:51:27.279490 [ 1476 ] {} CacheMetadata: Key 62899557aacb6f32b60b283ba23207c4 is removed from metadata 2024.10.22 01:51:27.328565 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} MergedBlockOutputStream: filled checksums 202410_1_692_139 (state Temporary) 2024.10.22 01:51:28.076940 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_692_139 to 202410_1_692_139 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:28.083715 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_692_139} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_687_138, 202410_692_692_0] -> 202410_1_692_139 2024.10.22 01:51:28.161933 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.55 MiB. 2024.10.22 01:51:28.417501 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 28948 bytes for 19613c091bb646fc15ce08801a6c0d98:0) 2024.10.22 01:51:28.421341 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 19e1fa9ebb7c83e2068a7015ff1a95b4, offset: 0, size: 22571 2024.10.22 01:51:28.427174 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 19e1fa9ebb7c83e2068a7015ff1a95b4 for removal 2024.10.22 01:51:28.433194 [ 1476 ] {} CacheMetadata: Key 19e1fa9ebb7c83e2068a7015ff1a95b4 is removed from metadata 2024.10.22 01:51:28.433462 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cc5f219e23a6a8712ead8123c2777215, offset: 0, size: 142 2024.10.22 01:51:28.445629 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cc5f219e23a6a8712ead8123c2777215 for removal 2024.10.22 01:51:28.450203 [ 1476 ] {} CacheMetadata: Key cc5f219e23a6a8712ead8123c2777215 is removed from metadata 2024.10.22 01:51:28.478382 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 995690 2024.10.22 01:51:28.579858 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uec/qskmyvjyodgubladlziwfvvnsroci. 2024.10.22 01:51:28.652717 [ 1263 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:51:28.753939 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9bd2df032fdbcae4a61262aed359478d, offset: 0, size: 0 2024.10.22 01:51:28.755746 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9bd2df032fdbcae4a61262aed359478d for removal 2024.10.22 01:51:28.760579 [ 1476 ] {} CacheMetadata: Key 9bd2df032fdbcae4a61262aed359478d is removed from metadata 2024.10.22 01:51:28.810350 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6f2eb6646b33798e0767d439f3274699, offset: 0, size: 0 2024.10.22 01:51:28.812970 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6f2eb6646b33798e0767d439f3274699 for removal 2024.10.22 01:51:28.816301 [ 1476 ] {} CacheMetadata: Key 6f2eb6646b33798e0767d439f3274699 is removed from metadata 2024.10.22 01:51:28.822073 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uec/qskmyvjyodgubladlziwfvvnsroci. 2024.10.22 01:51:28.821815 [ 1263 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:51:29.276823 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_135_139_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_136_136_0, total 42 rows starting from the beginning of the part 2024.10.22 01:51:29.421051 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 148 entries to flush up to offset 74846 2024.10.22 01:51:29.472452 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:29.476239 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:29.663654 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c004863decc87ea36f8699c9262d6607) : [0, 48827]: Setting download as failed: File segment: [0, 48827], key: c004863decc87ea36f8699c9262d6607, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:51:29.667180 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c004863decc87ea36f8699c9262d6607): No space left in cache to reserve 48828 bytes, will continue without cache download 2024.10.22 01:51:29.708529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c004863decc87ea36f8699c9262d6607, offset: 0, size: 0 2024.10.22 01:51:29.711026 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c004863decc87ea36f8699c9262d6607 for removal 2024.10.22 01:51:29.719371 [ 1476 ] {} CacheMetadata: Key c004863decc87ea36f8699c9262d6607 is removed from metadata 2024.10.22 01:51:29.842671 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_651_651_0 (state Temporary) 2024.10.22 01:51:30.057177 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_137_137_0 (state Temporary) 2024.10.22 01:51:30.057704 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_651_651_0 to 202410_658_658_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:30.110410 [ 1242 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_653_653_0 to 202410_658_658_0 2024.10.22 01:51:30.128736 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:30.136038 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:30.148614 [ 1242 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:30.153478 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:30.208144 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 995690 2024.10.22 01:51:30.274620 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1147_1147_0 (state Temporary) 2024.10.22 01:51:30.353389 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1147_1147_0 to 202410_1160_1160_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:30.374542 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_653_653_0 to 202410_658_658_0 into Compact with storage Full 2024.10.22 01:51:30.396620 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_652_126/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:51:30.503464 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:51:30.509715 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_653_653_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:51:30.511466 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 74846 2024.10.22 01:51:30.520990 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_654_654_0, total 1334 rows starting from the beginning of the part 2024.10.22 01:51:30.669582 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_655_655_0, total 1549 rows starting from the beginning of the part 2024.10.22 01:51:30.784816 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_656_656_0, total 967 rows starting from the beginning of the part 2024.10.22 01:51:30.815943 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_657_657_0, total 772 rows starting from the beginning of the part 2024.10.22 01:51:30.834935 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_658_658_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:51:30.843784 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:51:30.846952 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:51:30.950658 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(9bd2df032fdbcae4a61262aed359478d) : [0, 313302]: Setting download as failed: File segment: [0, 313302], key: 9bd2df032fdbcae4a61262aed359478d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:30.953201 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(9bd2df032fdbcae4a61262aed359478d): No space left in cache to reserve 313303 bytes, will continue without cache download 2024.10.22 01:51:31.433718 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_135_139_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_137_137_0, total 42 rows starting from the beginning of the part 2024.10.22 01:51:31.453925 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 9.95 MiB, will set to 1.77 GiB (RSS), difference: -1.67 MiB 2024.10.22 01:51:31.465574 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9bd2df032fdbcae4a61262aed359478d, offset: 0, size: 0 2024.10.22 01:51:31.568214 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9bd2df032fdbcae4a61262aed359478d for removal 2024.10.22 01:51:31.571982 [ 1476 ] {} CacheMetadata: Key 9bd2df032fdbcae4a61262aed359478d is removed from metadata 2024.10.22 01:51:31.572811 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tsi/vxjjsvcdgwlhjubqofltkumyxaroe. 2024.10.22 01:51:31.636152 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 76ed00f48ff5d936685fbb4a7546980d, offset: 0, size: 0 2024.10.22 01:51:31.644707 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 76ed00f48ff5d936685fbb4a7546980d for removal 2024.10.22 01:51:31.652082 [ 1476 ] {} CacheMetadata: Key 76ed00f48ff5d936685fbb4a7546980d is removed from metadata 2024.10.22 01:51:31.702820 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 77bb2587621b9d45be807ad7114f0b0c, offset: 0, size: 0 2024.10.22 01:51:31.704902 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 77bb2587621b9d45be807ad7114f0b0c for removal 2024.10.22 01:51:31.707356 [ 1476 ] {} CacheMetadata: Key 77bb2587621b9d45be807ad7114f0b0c is removed from metadata 2024.10.22 01:51:31.713508 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tsi/vxjjsvcdgwlhjubqofltkumyxaroe. 2024.10.22 01:51:31.770092 [ 1205 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 01:51:31.912101 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 11281 bytes for b9bbdb80adc1e4754546efd65f894726:0) 2024.10.22 01:51:31.918552 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 19613c091bb646fc15ce08801a6c0d98, offset: 0, size: 28948 2024.10.22 01:51:31.922504 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 19613c091bb646fc15ce08801a6c0d98 for removal 2024.10.22 01:51:31.933465 [ 1476 ] {} CacheMetadata: Key 19613c091bb646fc15ce08801a6c0d98 is removed from metadata 2024.10.22 01:51:31.936327 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3b1e0872db659c85b605787951944a01, offset: 0, size: 174 2024.10.22 01:51:31.942704 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3b1e0872db659c85b605787951944a01 for removal 2024.10.22 01:51:31.947311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7b5b22cb80bd986bfbb9bfefb0f33ee7, offset: 0, size: 564 2024.10.22 01:51:31.951957 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7b5b22cb80bd986bfbb9bfefb0f33ee7 for removal 2024.10.22 01:51:31.983054 [ 1476 ] {} CacheMetadata: Key 7b5b22cb80bd986bfbb9bfefb0f33ee7 is removed from metadata 2024.10.22 01:51:31.998215 [ 1476 ] {} CacheMetadata: Key 3b1e0872db659c85b605787951944a01 is removed from metadata 2024.10.22 01:51:32.139957 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} MergeTask::MergeProjectionsStage: Merge sorted 6932 rows, containing 4 columns (4 merged, 0 gathered) in 1.775315268 sec., 3904.658583716974 rows/sec., 61.01 KiB/sec. 2024.10.22 01:51:32.219801 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} MergedBlockOutputStream: filled checksums 202410_653_658_1 (state Temporary) 2024.10.22 01:51:32.580473 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_653_658_1 to 202410_653_658_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:32.585441 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_653_658_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_653_653_0, 202410_658_658_0] -> 202410_653_658_1 2024.10.22 01:51:32.646821 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:51:32.660175 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 10361 2024.10.22 01:51:32.908364 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:32.920480 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:32.951474 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(b05b8be1fed58e2286a5400cf2247b40) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b05b8be1fed58e2286a5400cf2247b40, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:51:32.965287 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_680_680_0 (state Temporary) 2024.10.22 01:51:32.980198 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: b05b8be1fed58e2286a5400cf2247b40, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:51:32.991114 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:51:33.061459 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_680_680_0 to 202410_693_693_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:33.256546 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10361 2024.10.22 01:51:33.440544 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 106 entries to flush up to offset 75970 2024.10.22 01:51:33.602694 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_135_139_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_138_138_0, total 41 rows starting from the beginning of the part 2024.10.22 01:51:33.617810 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:33.622097 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:33.706761 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 568 2024.10.22 01:51:33.802255 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_668_668_0 (state Temporary) 2024.10.22 01:51:33.868212 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_668_668_0 to 202410_674_674_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:33.947051 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 75970 2024.10.22 01:51:34.079892 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:34.086460 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:34.228331 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(76ed00f48ff5d936685fbb4a7546980d) : [0, 302506]: Setting download as failed: File segment: [0, 302506], key: 76ed00f48ff5d936685fbb4a7546980d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:34.232766 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(76ed00f48ff5d936685fbb4a7546980d): No space left in cache to reserve 302507 bytes, will continue without cache download 2024.10.22 01:51:34.318188 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_137_137_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:51:34.360742 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_550_550_0 (state Temporary) 2024.10.22 01:51:34.536250 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 74931 2024.10.22 01:51:34.580493 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 76ed00f48ff5d936685fbb4a7546980d, offset: 0, size: 0 2024.10.22 01:51:34.583265 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 76ed00f48ff5d936685fbb4a7546980d for removal 2024.10.22 01:51:34.591617 [ 1476 ] {} CacheMetadata: Key 76ed00f48ff5d936685fbb4a7546980d is removed from metadata 2024.10.22 01:51:34.592832 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tda/jazfrqidqdwwseitnbkmehevzngoj. 2024.10.22 01:51:34.655304 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_550_550_0 to 202410_560_560_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:34.766363 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 12.76 MiB, will set to 1.78 GiB (RSS), difference: 6.12 MiB 2024.10.22 01:51:34.794870 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6535868ab3c0774ad33d8dca37d7bec0, offset: 0, size: 0 2024.10.22 01:51:34.797498 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6535868ab3c0774ad33d8dca37d7bec0 for removal 2024.10.22 01:51:34.808353 [ 1476 ] {} CacheMetadata: Key 6535868ab3c0774ad33d8dca37d7bec0 is removed from metadata 2024.10.22 01:51:34.877267 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ab159d9b6e5db5bb131b6c0b75147c19, offset: 0, size: 0 2024.10.22 01:51:34.878976 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ab159d9b6e5db5bb131b6c0b75147c19 for removal 2024.10.22 01:51:34.885408 [ 1476 ] {} CacheMetadata: Key ab159d9b6e5db5bb131b6c0b75147c19 is removed from metadata 2024.10.22 01:51:34.912097 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 568 2024.10.22 01:51:34.997373 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tda/jazfrqidqdwwseitnbkmehevzngoj. 2024.10.22 01:51:35.002876 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mcw/pwgyuqsqkcchhzehrwlkajtzvgxvh. 2024.10.22 01:51:35.128682 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:35.136855 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:35.167779 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e572de6d198553ef2c83bfee1c2b68a1, offset: 0, size: 0 2024.10.22 01:51:35.170062 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e572de6d198553ef2c83bfee1c2b68a1 for removal 2024.10.22 01:51:35.175219 [ 1476 ] {} CacheMetadata: Key e572de6d198553ef2c83bfee1c2b68a1 is removed from metadata 2024.10.22 01:51:35.339853 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e08d763794e5ebb98d2c34e9b401b042, offset: 0, size: 0 2024.10.22 01:51:35.347968 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e08d763794e5ebb98d2c34e9b401b042 for removal 2024.10.22 01:51:35.348219 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1148_1148_0 (state Temporary) 2024.10.22 01:51:35.352338 [ 1476 ] {} CacheMetadata: Key e08d763794e5ebb98d2c34e9b401b042 is removed from metadata 2024.10.22 01:51:35.367695 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mcw/pwgyuqsqkcchhzehrwlkajtzvgxvh. 2024.10.22 01:51:35.372314 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qvb/beqxshdapezxrjcaomaazucfcldmm. 2024.10.22 01:51:35.522608 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1148_1148_0 to 202410_1161_1161_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:35.537326 [ 1188 ] {} DatabaseCatalog: Execution took 418 ms. 2024.10.22 01:51:35.544424 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a4286881e5b66bc4fd280e9f7ea9d505, offset: 0, size: 0 2024.10.22 01:51:35.544940 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_135_139_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_139_139_0, total 41 rows starting from the beginning of the part 2024.10.22 01:51:35.587584 [ 1273 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1156_12 to 202410_1161_1161_0 2024.10.22 01:51:35.620656 [ 1273 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:35.649189 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a4286881e5b66bc4fd280e9f7ea9d505 for removal 2024.10.22 01:51:35.653417 [ 1476 ] {} CacheMetadata: Key a4286881e5b66bc4fd280e9f7ea9d505 is removed from metadata 2024.10.22 01:51:35.707438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 17727 bytes for 96915ecf73c8addd3b575d794c4dd8d3:0) 2024.10.22 01:51:35.713448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b9bbdb80adc1e4754546efd65f894726, offset: 0, size: 11281 2024.10.22 01:51:35.719164 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 74931 2024.10.22 01:51:35.722844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b9bbdb80adc1e4754546efd65f894726 for removal 2024.10.22 01:51:35.725230 [ 1273 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:35.725698 [ 1476 ] {} CacheMetadata: Key b9bbdb80adc1e4754546efd65f894726 is removed from metadata 2024.10.22 01:51:35.727234 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4c4329cf9d0074f295b23c4515aa0dc0, offset: 0, size: 154 2024.10.22 01:51:35.739258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4c4329cf9d0074f295b23c4515aa0dc0 for removal 2024.10.22 01:51:35.749834 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6f2eb6646b33798e0767d439f3274699, offset: 0, size: 577 2024.10.22 01:51:35.755982 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6f2eb6646b33798e0767d439f3274699 for removal 2024.10.22 01:51:35.772151 [ 1476 ] {} CacheMetadata: Key 6f2eb6646b33798e0767d439f3274699 is removed from metadata 2024.10.22 01:51:35.815190 [ 1476 ] {} CacheMetadata: Key 4c4329cf9d0074f295b23c4515aa0dc0 is removed from metadata 2024.10.22 01:51:35.891635 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1156_12 to 202410_1161_1161_0 into Compact with storage Full 2024.10.22 01:51:35.921069 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 132ebf21a23d4cfc5e404154f1bda3c9, offset: 0, size: 0 2024.10.22 01:51:35.926986 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 132ebf21a23d4cfc5e404154f1bda3c9 for removal 2024.10.22 01:51:35.930414 [ 1476 ] {} CacheMetadata: Key 132ebf21a23d4cfc5e404154f1bda3c9 is removed from metadata 2024.10.22 01:51:35.937511 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qvb/beqxshdapezxrjcaomaazucfcldmm. 2024.10.22 01:51:36.007168 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:51:36.024113 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1156_12, total 3847 rows starting from the beginning of the part 2024.10.22 01:51:36.073525 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1157_1157_0, total 93 rows starting from the beginning of the part 2024.10.22 01:51:36.236182 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1158_1158_0, total 107 rows starting from the beginning of the part 2024.10.22 01:51:36.411694 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1159_1159_0, total 75 rows starting from the beginning of the part 2024.10.22 01:51:36.462508 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1160_1160_0, total 148 rows starting from the beginning of the part 2024.10.22 01:51:36.497241 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1161_1161_0, total 85 rows starting from the beginning of the part 2024.10.22 01:51:37.180861 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a4286881e5b66bc4fd280e9f7ea9d505) : [0, 2203513]: Setting download as failed: File segment: [0, 2203513], key: a4286881e5b66bc4fd280e9f7ea9d505, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:37.192798 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(a4286881e5b66bc4fd280e9f7ea9d505): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:51:37.261024 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 996845 2024.10.22 01:51:37.523978 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.97 GiB, free memory in arenas 10.87 MiB, will set to 1.78 GiB (RSS), difference: 2.52 MiB 2024.10.22 01:51:38.015771 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a4286881e5b66bc4fd280e9f7ea9d505, offset: 0, size: 0 2024.10.22 01:51:38.020291 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a4286881e5b66bc4fd280e9f7ea9d505 for removal 2024.10.22 01:51:38.028197 [ 1476 ] {} CacheMetadata: Key a4286881e5b66bc4fd280e9f7ea9d505 is removed from metadata 2024.10.22 01:51:38.037345 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} MergeTask::MergeProjectionsStage: Merge sorted 4355 rows, containing 13 columns (13 merged, 0 gathered) in 2.266350328 sec., 1921.5917090113703 rows/sec., 339.02 KiB/sec. 2024.10.22 01:51:38.044880 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gym/tandbbjzpxfjhbnklivneukgyrbzk. 2024.10.22 01:51:38.099153 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7dab4cd80a02bb858d9d71c71cc8ec31, offset: 0, size: 0 2024.10.22 01:51:38.103655 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7dab4cd80a02bb858d9d71c71cc8ec31 for removal 2024.10.22 01:51:38.107337 [ 1476 ] {} CacheMetadata: Key 7dab4cd80a02bb858d9d71c71cc8ec31 is removed from metadata 2024.10.22 01:51:38.230721 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 5 file segments (while reserving 17211 bytes for da1e64bbe89e5838dc772886232b22f6:0) 2024.10.22 01:51:38.235980 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 96915ecf73c8addd3b575d794c4dd8d3, offset: 0, size: 17727 2024.10.22 01:51:38.241609 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 96915ecf73c8addd3b575d794c4dd8d3 for removal 2024.10.22 01:51:38.246057 [ 1476 ] {} CacheMetadata: Key 96915ecf73c8addd3b575d794c4dd8d3 is removed from metadata 2024.10.22 01:51:38.246851 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 60d303eb63d233b64e9bdeb22424f1d5, offset: 0, size: 164 2024.10.22 01:51:38.253241 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 60d303eb63d233b64e9bdeb22424f1d5 for removal 2024.10.22 01:51:38.258201 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 617d5e2fd9625f564cbec77b17f16889, offset: 0, size: 14973 2024.10.22 01:51:38.267366 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 617d5e2fd9625f564cbec77b17f16889 for removal 2024.10.22 01:51:38.272775 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 77328e266c7070db2b5687a49a2dc112, offset: 0, size: 154 2024.10.22 01:51:38.278596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 77328e266c7070db2b5687a49a2dc112 for removal 2024.10.22 01:51:38.284585 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 77bb2587621b9d45be807ad7114f0b0c, offset: 0, size: 577 2024.10.22 01:51:38.300413 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 77bb2587621b9d45be807ad7114f0b0c for removal 2024.10.22 01:51:38.317738 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} MergedBlockOutputStream: filled checksums 202410_1096_1161_13 (state Temporary) 2024.10.22 01:51:38.511958 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ed0b98044ad70a4b9fd3cb28a97fe529, offset: 0, size: 0 2024.10.22 01:51:38.513466 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ed0b98044ad70a4b9fd3cb28a97fe529 for removal 2024.10.22 01:51:38.521056 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gym/tandbbjzpxfjhbnklivneukgyrbzk. 2024.10.22 01:51:38.531616 [ 1476 ] {} CacheMetadata: Key ed0b98044ad70a4b9fd3cb28a97fe529 is removed from metadata 2024.10.22 01:51:38.537120 [ 1476 ] {} CacheMetadata: Key 77bb2587621b9d45be807ad7114f0b0c is removed from metadata 2024.10.22 01:51:38.549460 [ 1476 ] {} CacheMetadata: Key 77328e266c7070db2b5687a49a2dc112 is removed from metadata 2024.10.22 01:51:38.556814 [ 1476 ] {} CacheMetadata: Key 617d5e2fd9625f564cbec77b17f16889 is removed from metadata 2024.10.22 01:51:38.570768 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:38.571892 [ 1476 ] {} CacheMetadata: Key 60d303eb63d233b64e9bdeb22424f1d5 is removed from metadata 2024.10.22 01:51:38.574565 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:39.078769 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1161_13 to 202410_1096_1161_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:39.111204 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_652_652_0 (state Temporary) 2024.10.22 01:51:39.188600 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1161_13} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1156_12, 202410_1161_1161_0] -> 202410_1096_1161_13 2024.10.22 01:51:39.306731 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_652_652_0 to 202410_659_659_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:39.314049 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:51:39.358419 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 996845 2024.10.22 01:51:39.746910 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 100 entries to flush up to offset 75031 2024.10.22 01:51:39.999835 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_137_137_0 to 202410_140_140_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:40.198869 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 7 old parts to remove. Parts: [202410_1_114_22, 202410_115_119_1, 202410_120_120_0, 202410_121_121_0, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0] 2024.10.22 01:51:40.203739 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from filesystem (serially): Parts: [202410_1_114_22, 202410_115_119_1, 202410_120_120_0, 202410_121_121_0, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0] 2024.10.22 01:51:40.203936 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:40.210658 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:40.346131 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from memory: Parts: [202410_1_114_22, 202410_115_119_1, 202410_120_120_0, 202410_121_121_0, 202410_122_122_0, 202410_123_123_0, 202410_124_124_0] 2024.10.22 01:51:40.358807 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(7dab4cd80a02bb858d9d71c71cc8ec31) : [0, 1580284]: Setting download as failed: File segment: [0, 1580284], key: 7dab4cd80a02bb858d9d71c71cc8ec31, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:40.360687 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(7dab4cd80a02bb858d9d71c71cc8ec31): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:51:40.516042 [ 1276 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 01:51:40.522711 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1149_1149_0 (state Temporary) 2024.10.22 01:51:40.555887 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.97 GiB, free memory in arenas 9.47 MiB, will set to 1.78 GiB (RSS), difference: -12.79 MiB 2024.10.22 01:51:40.594582 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1149_1149_0 to 202410_1162_1162_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:40.666808 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 75031 2024.10.22 01:51:40.763352 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7dab4cd80a02bb858d9d71c71cc8ec31, offset: 0, size: 0 2024.10.22 01:51:40.771623 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7dab4cd80a02bb858d9d71c71cc8ec31 for removal 2024.10.22 01:51:40.773748 [ 1476 ] {} CacheMetadata: Key 7dab4cd80a02bb858d9d71c71cc8ec31 is removed from metadata 2024.10.22 01:51:40.847296 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:51:40.849363 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:51:40.860718 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 10375 2024.10.22 01:51:41.045304 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ab00c2358cf2493acb979a968dc5bf4d, offset: 0, size: 0 2024.10.22 01:51:41.057801 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ab00c2358cf2493acb979a968dc5bf4d for removal 2024.10.22 01:51:41.061590 [ 1476 ] {} CacheMetadata: Key ab00c2358cf2493acb979a968dc5bf4d is removed from metadata 2024.10.22 01:51:41.092032 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:41.094611 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:41.109835 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 08349b656a1ad96fd38f7c372d6d0d21, offset: 0, size: 0 2024.10.22 01:51:41.114091 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 08349b656a1ad96fd38f7c372d6d0d21 for removal 2024.10.22 01:51:41.118969 [ 1476 ] {} CacheMetadata: Key 08349b656a1ad96fd38f7c372d6d0d21 is removed from metadata 2024.10.22 01:51:41.147315 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_681_681_0 (state Temporary) 2024.10.22 01:51:41.198402 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_652_126/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:51:41.336189 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_681_681_0 to 202410_694_694_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:41.418829 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10375 2024.10.22 01:51:41.461823 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 76061 2024.10.22 01:51:41.698236 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:41.704140 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:41.783362 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_669_669_0 (state Temporary) 2024.10.22 01:51:41.771722 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 7742 bytes for a14fe093bbb039f7a834379f474440d3:0) 2024.10.22 01:51:41.795144 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8f2b138bc62fdcf6d53c78d941f4d2bd, offset: 0, size: 164 2024.10.22 01:51:41.904801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8f2b138bc62fdcf6d53c78d941f4d2bd for removal 2024.10.22 01:51:41.909677 [ 1476 ] {} CacheMetadata: Key 8f2b138bc62fdcf6d53c78d941f4d2bd is removed from metadata 2024.10.22 01:51:41.914965 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 132ebf21a23d4cfc5e404154f1bda3c9, offset: 0, size: 1020 2024.10.22 01:51:41.923308 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 132ebf21a23d4cfc5e404154f1bda3c9 for removal 2024.10.22 01:51:41.962223 [ 1476 ] {} CacheMetadata: Key 132ebf21a23d4cfc5e404154f1bda3c9 is removed from metadata 2024.10.22 01:51:42.058512 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_669_669_0 to 202410_675_675_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:42.211937 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 76061 2024.10.22 01:51:42.414980 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 569 2024.10.22 01:51:42.574339 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 1.97 GiB, free memory in arenas 12.02 MiB, will set to 1.78 GiB (RSS), difference: 19.46 MiB 2024.10.22 01:51:42.765424 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:42.768616 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:42.942479 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_551_551_0 (state Temporary) 2024.10.22 01:51:43.107489 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_551_551_0 to 202410_561_561_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:43.107996 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 754 bytes for 08349b656a1ad96fd38f7c372d6d0d21:0) 2024.10.22 01:51:43.112524 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: da1e64bbe89e5838dc772886232b22f6, offset: 0, size: 17211 2024.10.22 01:51:43.117615 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key da1e64bbe89e5838dc772886232b22f6 for removal 2024.10.22 01:51:43.123319 [ 1476 ] {} CacheMetadata: Key da1e64bbe89e5838dc772886232b22f6 is removed from metadata 2024.10.22 01:51:43.206993 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 569 2024.10.22 01:51:43.316894 [ 1717 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tya/setmktoqqnaurzllfdsodugnsvrec, size 2641034 2024.10.22 01:51:43.358353 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_611_21, 202410_612_612_0, 202410_613_613_0, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0] 2024.10.22 01:51:43.361488 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_611_21, 202410_612_612_0, 202410_613_613_0, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0] 2024.10.22 01:51:43.407135 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_611_21, 202410_612_612_0, 202410_613_613_0, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0] 2024.10.22 01:51:43.831594 [ 1199 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:51:43.858570 [ 1199 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:51:43.967017 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5487 2024.10.22 01:51:44.310741 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(ab00c2358cf2493acb979a968dc5bf4d) : [0, 805764]: Setting download as failed: File segment: [0, 805764], key: ab00c2358cf2493acb979a968dc5bf4d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:44.317531 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(ab00c2358cf2493acb979a968dc5bf4d): No space left in cache to reserve 805765 bytes, will continue without cache download 2024.10.22 01:51:44.344827 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:51:44.354190 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:51:44.397814 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:51:44.429129 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:51:44.455089 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:51:44.476998 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:51:44.524715 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:51:44.662138 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 01:51:44.696015 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/sxi/alumqirmcsartnhsdyvfqfyiwhlub (LogSeriesLimiter: on interval from 2024-10-22 01:51:21 to 2024-10-22 01:51:44 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:51:44.779597 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 75104 2024.10.22 01:51:44.794212 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 1.97 GiB, free memory in arenas 10.30 MiB, will set to 1.78 GiB (RSS), difference: 8.57 MiB 2024.10.22 01:51:44.805313 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ab00c2358cf2493acb979a968dc5bf4d, offset: 0, size: 0 2024.10.22 01:51:44.812698 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ab00c2358cf2493acb979a968dc5bf4d for removal 2024.10.22 01:51:44.816214 [ 1476 ] {} CacheMetadata: Key ab00c2358cf2493acb979a968dc5bf4d is removed from metadata 2024.10.22 01:51:45.062335 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: fa4256509c2fb5ea6d422f2d6bee8254, offset: 0, size: 0 2024.10.22 01:51:45.063597 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key fa4256509c2fb5ea6d422f2d6bee8254 for removal 2024.10.22 01:51:45.067059 [ 1476 ] {} CacheMetadata: Key fa4256509c2fb5ea6d422f2d6bee8254 is removed from metadata 2024.10.22 01:51:45.217523 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1a418f828a5ad2535ff48c0b8c3c76ba, offset: 0, size: 0 2024.10.22 01:51:45.223336 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1a418f828a5ad2535ff48c0b8c3c76ba for removal 2024.10.22 01:51:45.230910 [ 1476 ] {} CacheMetadata: Key 1a418f828a5ad2535ff48c0b8c3c76ba is removed from metadata 2024.10.22 01:51:45.232496 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rhq/xfhyoxvsgpdlfbyahngeydsgumcwx. 2024.10.22 01:51:45.314168 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0d67991a970ae80aee8a378f99c4de29, offset: 0, size: 0 2024.10.22 01:51:45.315959 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0d67991a970ae80aee8a378f99c4de29 for removal 2024.10.22 01:51:45.322943 [ 1476 ] {} CacheMetadata: Key 0d67991a970ae80aee8a378f99c4de29 is removed from metadata 2024.10.22 01:51:45.400034 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:45.404048 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:45.463981 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1150_1150_0 (state Temporary) 2024.10.22 01:51:45.467250 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b56d2036adb80b3e1189486b023949b6, offset: 0, size: 0 2024.10.22 01:51:45.470435 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b56d2036adb80b3e1189486b023949b6 for removal 2024.10.22 01:51:45.473792 [ 1476 ] {} CacheMetadata: Key b56d2036adb80b3e1189486b023949b6 is removed from metadata 2024.10.22 01:51:45.485728 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rhq/xfhyoxvsgpdlfbyahngeydsgumcwx. 2024.10.22 01:51:45.546691 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: dd55f2af6afe1571d0e9d486249faa07, offset: 0, size: 0 2024.10.22 01:51:45.549670 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key dd55f2af6afe1571d0e9d486249faa07 for removal 2024.10.22 01:51:45.554445 [ 1476 ] {} CacheMetadata: Key dd55f2af6afe1571d0e9d486249faa07 is removed from metadata 2024.10.22 01:51:45.635342 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1150_1150_0 to 202410_1163_1163_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:45.802716 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b40d32af54b02850343aafff2cbb445d, offset: 0, size: 0 2024.10.22 01:51:45.805218 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 75104 2024.10.22 01:51:45.806627 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b40d32af54b02850343aafff2cbb445d for removal 2024.10.22 01:51:45.811866 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/oqw/jjityjqworspfzbpfmtzddrezymnw. 2024.10.22 01:51:45.812267 [ 1476 ] {} CacheMetadata: Key b40d32af54b02850343aafff2cbb445d is removed from metadata 2024.10.22 01:51:45.853417 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a1b36dc14df2ca3bd48782e749fc2895, offset: 0, size: 0 2024.10.22 01:51:45.855166 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a1b36dc14df2ca3bd48782e749fc2895 for removal 2024.10.22 01:51:45.856924 [ 1476 ] {} CacheMetadata: Key a1b36dc14df2ca3bd48782e749fc2895 is removed from metadata 2024.10.22 01:51:45.964389 [ 1227 ] {} DatabaseCatalog: Execution took 352 ms. 2024.10.22 01:51:46.002725 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b446d4fadaea5a88488e4f6595cbc45b, offset: 0, size: 0 2024.10.22 01:51:46.004284 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b446d4fadaea5a88488e4f6595cbc45b for removal 2024.10.22 01:51:46.008247 [ 1476 ] {} CacheMetadata: Key b446d4fadaea5a88488e4f6595cbc45b is removed from metadata 2024.10.22 01:51:46.032800 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/oqw/jjityjqworspfzbpfmtzddrezymnw. 2024.10.22 01:51:46.037534 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/sat/ywqcnpbjsiaxzmvdmdurkzxohxsmf. 2024.10.22 01:51:46.188105 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: dc7f8c5dcd1fb817cb0fae9f68f00d4a, offset: 0, size: 0 2024.10.22 01:51:46.194838 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key dc7f8c5dcd1fb817cb0fae9f68f00d4a for removal 2024.10.22 01:51:46.199106 [ 1476 ] {} CacheMetadata: Key dc7f8c5dcd1fb817cb0fae9f68f00d4a is removed from metadata 2024.10.22 01:51:46.348852 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1e86f91c9ae4ad6e181cac537505f5eb, offset: 0, size: 0 2024.10.22 01:51:46.350491 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1e86f91c9ae4ad6e181cac537505f5eb for removal 2024.10.22 01:51:46.352788 [ 1476 ] {} CacheMetadata: Key 1e86f91c9ae4ad6e181cac537505f5eb is removed from metadata 2024.10.22 01:51:46.365677 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 998385 2024.10.22 01:51:46.460066 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/sat/ywqcnpbjsiaxzmvdmdurkzxohxsmf. 2024.10.22 01:51:46.504767 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 5 file segments (while reserving 18053 bytes for 14a52a921ce2cfcaf3f77761844c11ab:0) 2024.10.22 01:51:46.531316 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a14fe093bbb039f7a834379f474440d3, offset: 0, size: 7742 2024.10.22 01:51:46.537713 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a14fe093bbb039f7a834379f474440d3 for removal 2024.10.22 01:51:46.545268 [ 1476 ] {} CacheMetadata: Key a14fe093bbb039f7a834379f474440d3 is removed from metadata 2024.10.22 01:51:46.546330 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 145132b8528db514393e0dc45974410b, offset: 0, size: 9618 2024.10.22 01:51:46.553978 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 145132b8528db514393e0dc45974410b for removal 2024.10.22 01:51:46.557770 [ 1476 ] {} CacheMetadata: Key 145132b8528db514393e0dc45974410b is removed from metadata 2024.10.22 01:51:46.558137 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e2fcf721046f2b98ff6565f76a86d228, offset: 0, size: 154 2024.10.22 01:51:46.566935 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e2fcf721046f2b98ff6565f76a86d228 for removal 2024.10.22 01:51:46.570788 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 31254e1e1201c583c09a38dc0c8f6449, offset: 0, size: 154 2024.10.22 01:51:46.571477 [ 1476 ] {} CacheMetadata: Key e2fcf721046f2b98ff6565f76a86d228 is removed from metadata 2024.10.22 01:51:46.575040 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 31254e1e1201c583c09a38dc0c8f6449 for removal 2024.10.22 01:51:46.592528 [ 1476 ] {} CacheMetadata: Key 31254e1e1201c583c09a38dc0c8f6449 is removed from metadata 2024.10.22 01:51:46.594466 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ed0b98044ad70a4b9fd3cb28a97fe529, offset: 0, size: 616 2024.10.22 01:51:46.615096 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ed0b98044ad70a4b9fd3cb28a97fe529 for removal 2024.10.22 01:51:46.628739 [ 1476 ] {} CacheMetadata: Key ed0b98044ad70a4b9fd3cb28a97fe529 is removed from metadata 2024.10.22 01:51:46.630445 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.97 GiB, free memory in arenas 12.45 MiB, will set to 1.79 GiB (RSS), difference: 4.50 MiB 2024.10.22 01:51:46.890591 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_135_139_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:51:47.149182 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:47.152391 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.01 GiB. 2024.10.22 01:51:47.645148 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_653_653_0 (state Temporary) 2024.10.22 01:51:47.742470 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_653_653_0 to 202410_660_660_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:47.780982 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 998385 2024.10.22 01:51:48.047559 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(dc7f8c5dcd1fb817cb0fae9f68f00d4a) : [0, 1076177]: Setting download as failed: File segment: [0, 1076177], key: dc7f8c5dcd1fb817cb0fae9f68f00d4a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:48.049560 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(dc7f8c5dcd1fb817cb0fae9f68f00d4a): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:51:48.200716 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1052_502, 202410_1053_1058_1] 2024.10.22 01:51:48.204900 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1052_502, 202410_1053_1058_1] 2024.10.22 01:51:48.281499 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1052_502, 202410_1053_1058_1] 2024.10.22 01:51:48.406667 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: dc7f8c5dcd1fb817cb0fae9f68f00d4a, offset: 0, size: 0 2024.10.22 01:51:48.503520 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.79 GiB, peak 1.97 GiB, free memory in arenas 14.92 MiB, will set to 1.79 GiB (RSS), difference: 6.04 MiB 2024.10.22 01:51:48.511004 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key dc7f8c5dcd1fb817cb0fae9f68f00d4a for removal 2024.10.22 01:51:48.514236 [ 1476 ] {} CacheMetadata: Key dc7f8c5dcd1fb817cb0fae9f68f00d4a is removed from metadata 2024.10.22 01:51:48.519505 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/veu/tadkuihrcrctknzypdbunepmsvrpi. 2024.10.22 01:51:48.620316 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b983d63f46e3dbc1b8a1d11abe1e0902, offset: 0, size: 0 2024.10.22 01:51:48.631069 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b983d63f46e3dbc1b8a1d11abe1e0902 for removal 2024.10.22 01:51:48.633754 [ 1476 ] {} CacheMetadata: Key b983d63f46e3dbc1b8a1d11abe1e0902 is removed from metadata 2024.10.22 01:51:48.811024 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f8e3548932750f4cdc5bc7fcf3152172, offset: 0, size: 0 2024.10.22 01:51:48.815720 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f8e3548932750f4cdc5bc7fcf3152172 for removal 2024.10.22 01:51:48.825209 [ 1476 ] {} CacheMetadata: Key f8e3548932750f4cdc5bc7fcf3152172 is removed from metadata 2024.10.22 01:51:48.933282 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 10380 2024.10.22 01:51:48.944048 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/veu/tadkuihrcrctknzypdbunepmsvrpi. 2024.10.22 01:51:49.068602 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:49.072842 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:49.241406 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_682_682_0 (state Temporary) 2024.10.22 01:51:49.294407 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_682_682_0 to 202410_695_695_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:49.381573 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10380 2024.10.22 01:51:49.742397 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 76145 2024.10.22 01:51:49.929833 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 75200 2024.10.22 01:51:50.167326 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 9 file segments (while reserving 27959 bytes for 587e89524478cfef0d462af9ade4e33c:0) 2024.10.22 01:51:50.171140 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 14a52a921ce2cfcaf3f77761844c11ab, offset: 0, size: 18053 2024.10.22 01:51:50.175913 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 14a52a921ce2cfcaf3f77761844c11ab for removal 2024.10.22 01:51:50.181216 [ 1476 ] {} CacheMetadata: Key 14a52a921ce2cfcaf3f77761844c11ab is removed from metadata 2024.10.22 01:51:50.181064 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 210485172f6585a09e4cb4718ee6b4b9, offset: 0, size: 2455 2024.10.22 01:51:50.191116 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 210485172f6585a09e4cb4718ee6b4b9 for removal 2024.10.22 01:51:50.195058 [ 1476 ] {} CacheMetadata: Key 210485172f6585a09e4cb4718ee6b4b9 is removed from metadata 2024.10.22 01:51:50.199594 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 11ea0461bca7fb2781dfd8effdad9fe8, offset: 0, size: 154 2024.10.22 01:51:50.203276 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 11ea0461bca7fb2781dfd8effdad9fe8 for removal 2024.10.22 01:51:50.207545 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c5786389a21d8010cc225eff5aef27ad, offset: 0, size: 154 2024.10.22 01:51:50.210643 [ 1476 ] {} CacheMetadata: Key 11ea0461bca7fb2781dfd8effdad9fe8 is removed from metadata 2024.10.22 01:51:50.216562 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c5786389a21d8010cc225eff5aef27ad for removal 2024.10.22 01:51:50.219056 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:50.230919 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2a49dd5587c168e92baf8e806c2ee196, offset: 0, size: 5890 2024.10.22 01:51:50.233138 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:50.232543 [ 1476 ] {} CacheMetadata: Key c5786389a21d8010cc225eff5aef27ad is removed from metadata 2024.10.22 01:51:50.239293 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2a49dd5587c168e92baf8e806c2ee196 for removal 2024.10.22 01:51:50.248054 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0e23ab79f2f091730d7df405e0c2aa67, offset: 0, size: 2569 2024.10.22 01:51:50.252381 [ 1476 ] {} CacheMetadata: Key 2a49dd5587c168e92baf8e806c2ee196 is removed from metadata 2024.10.22 01:51:50.253205 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0e23ab79f2f091730d7df405e0c2aa67 for removal 2024.10.22 01:51:50.257985 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d2218111d260c87ca09e7d88489c3b1a, offset: 0, size: 154 2024.10.22 01:51:50.262836 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d2218111d260c87ca09e7d88489c3b1a for removal 2024.10.22 01:51:50.268373 [ 1476 ] {} CacheMetadata: Key d2218111d260c87ca09e7d88489c3b1a is removed from metadata 2024.10.22 01:51:50.268807 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3be6c94e497f55c64aa659a76db7bdb2, offset: 0, size: 164 2024.10.22 01:51:50.275809 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3be6c94e497f55c64aa659a76db7bdb2 for removal 2024.10.22 01:51:50.283894 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 08349b656a1ad96fd38f7c372d6d0d21, offset: 0, size: 754 2024.10.22 01:51:50.292286 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 08349b656a1ad96fd38f7c372d6d0d21 for removal 2024.10.22 01:51:50.305309 [ 1476 ] {} CacheMetadata: Key 08349b656a1ad96fd38f7c372d6d0d21 is removed from metadata 2024.10.22 01:51:50.310247 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_670_670_0 (state Temporary) 2024.10.22 01:51:50.327149 [ 1476 ] {} CacheMetadata: Key 3be6c94e497f55c64aa659a76db7bdb2 is removed from metadata 2024.10.22 01:51:50.455377 [ 1476 ] {} CacheMetadata: Key 0e23ab79f2f091730d7df405e0c2aa67 is removed from metadata 2024.10.22 01:51:50.607507 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_652_126} MergeTask::MergeProjectionsStage: Merge sorted 1002348 rows, containing 4 columns (4 merged, 0 gathered) in 36.955585365 sec., 27123.044868592628 rows/sec., 406.68 KiB/sec. 2024.10.22 01:51:50.660234 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_670_670_0 to 202410_676_676_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:50.715128 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.97 GiB, free memory in arenas 14.29 MiB, will set to 1.79 GiB (RSS), difference: 4.25 MiB 2024.10.22 01:51:50.743433 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 76145 2024.10.22 01:51:50.753373 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:50.757713 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 297.02 GiB. 2024.10.22 01:51:50.848704 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:51:50.854362 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:51:50.858222 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_652_126} MergedBlockOutputStream: filled checksums 202410_1_652_126 (state Temporary) 2024.10.22 01:51:50.869711 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1151_1151_0 (state Temporary) 2024.10.22 01:51:50.948347 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1151_1151_0 to 202410_1164_1164_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:51.419182 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 75200 2024.10.22 01:51:51.421320 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1059_1059_0, 202410_1060_1060_0, 202410_1061_1061_0, 202410_1062_1062_0, 202410_1063_1063_0, 202410_1064_1064_0] 2024.10.22 01:51:51.435968 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1059_1059_0, 202410_1060_1060_0, 202410_1061_1061_0, 202410_1062_1062_0, 202410_1063_1063_0, 202410_1064_1064_0] 2024.10.22 01:51:51.447558 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/qjy/ynbgwzltucfosnaxkvyfequqbjehn, upload id d779e18e-d6b3-409a-89f5-1c3ba6b1d8ce 2024.10.22 01:51:51.459835 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1059_1059_0, 202410_1060_1060_0, 202410_1061_1061_0, 202410_1062_1062_0, 202410_1063_1063_0, 202410_1064_1064_0] 2024.10.22 01:51:51.512031 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 5530 2024.10.22 01:51:51.676419 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(b983d63f46e3dbc1b8a1d11abe1e0902) : [0, 2050585]: Setting download as failed: File segment: [0, 2050585], key: b983d63f46e3dbc1b8a1d11abe1e0902, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:51.677750 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(b983d63f46e3dbc1b8a1d11abe1e0902): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:51:52.363171 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_652_126/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:51:52.583811 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b983d63f46e3dbc1b8a1d11abe1e0902, offset: 0, size: 0 2024.10.22 01:51:52.585302 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b983d63f46e3dbc1b8a1d11abe1e0902 for removal 2024.10.22 01:51:52.595229 [ 1476 ] {} CacheMetadata: Key b983d63f46e3dbc1b8a1d11abe1e0902 is removed from metadata 2024.10.22 01:51:52.596182 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/sfr/gnwlsiznjxglqzhuudamjfcgosrpy. 2024.10.22 01:51:52.679708 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_508_104, 202410_509_509_0, 202410_510_510_0, 202410_511_511_0, 202410_512_512_0, 202410_513_513_0] 2024.10.22 01:51:52.681573 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_508_104, 202410_509_509_0, 202410_510_510_0, 202410_511_511_0, 202410_512_512_0, 202410_513_513_0] 2024.10.22 01:51:52.736449 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 43046fa1481d6d8d922ae4adefe38b2f, offset: 0, size: 0 2024.10.22 01:51:52.738816 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 43046fa1481d6d8d922ae4adefe38b2f for removal 2024.10.22 01:51:52.744326 [ 1476 ] {} CacheMetadata: Key 43046fa1481d6d8d922ae4adefe38b2f is removed from metadata 2024.10.22 01:51:52.794464 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b305f7b9b5b4bb9878c4ac7266103455, offset: 0, size: 0 2024.10.22 01:51:52.797050 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b305f7b9b5b4bb9878c4ac7266103455 for removal 2024.10.22 01:51:52.799339 [ 1476 ] {} CacheMetadata: Key b305f7b9b5b4bb9878c4ac7266103455 is removed from metadata 2024.10.22 01:51:52.814996 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/sfr/gnwlsiznjxglqzhuudamjfcgosrpy. 2024.10.22 01:51:52.836235 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_508_104, 202410_509_509_0, 202410_510_510_0, 202410_511_511_0, 202410_512_512_0, 202410_513_513_0] 2024.10.22 01:51:53.147961 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_631_126, 202410_632_632_0, 202410_633_633_0, 202410_634_634_0, 202410_635_635_0, 202410_636_636_0] 2024.10.22 01:51:53.151051 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_631_126, 202410_632_632_0, 202410_633_633_0, 202410_634_634_0, 202410_635_635_0, 202410_636_636_0] 2024.10.22 01:51:53.176238 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_631_126, 202410_632_632_0, 202410_633_633_0, 202410_634_634_0, 202410_635_635_0, 202410_636_636_0] 2024.10.22 01:51:53.175877 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(37205d77145d65d8ed5b6247cee4ff5c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 37205d77145d65d8ed5b6247cee4ff5c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:51:53.212601 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 37205d77145d65d8ed5b6247cee4ff5c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:51:53.226874 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:51:54.044740 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a7aeaf9db6b59ca3296e877a122abecf) : [0, 127399]: Setting download as failed: File segment: [0, 127399], key: a7aeaf9db6b59ca3296e877a122abecf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:51:54.046882 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a7aeaf9db6b59ca3296e877a122abecf): No space left in cache to reserve 127400 bytes, will continue without cache download 2024.10.22 01:51:54.076358 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a7aeaf9db6b59ca3296e877a122abecf, offset: 0, size: 0 2024.10.22 01:51:54.078475 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a7aeaf9db6b59ca3296e877a122abecf for removal 2024.10.22 01:51:54.086155 [ 1476 ] {} CacheMetadata: Key a7aeaf9db6b59ca3296e877a122abecf is removed from metadata 2024.10.22 01:51:54.720131 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:51:54.749657 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_652_126} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_652_126 to 202410_1_652_126 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:54.755542 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_652_126} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_646_125, 202410_647_652_1] -> 202410_1_652_126 2024.10.22 01:51:54.779010 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:51:54.793111 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 999540 2024.10.22 01:51:55.431070 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 75282 2024.10.22 01:51:55.513594 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 8.41 MiB, will set to 1.74 GiB (RSS), difference: -4.31 MiB 2024.10.22 01:51:55.555965 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(43046fa1481d6d8d922ae4adefe38b2f) : [0, 2199652]: Setting download as failed: File segment: [0, 2199652], key: 43046fa1481d6d8d922ae4adefe38b2f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:55.557202 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(43046fa1481d6d8d922ae4adefe38b2f): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:51:55.673732 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:55.679522 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:51:55.816249 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:55.827196 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:51:55.829473 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_654_654_0 (state Temporary) 2024.10.22 01:51:55.911111 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1152_1152_0 (state Temporary) 2024.10.22 01:51:56.098200 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 43046fa1481d6d8d922ae4adefe38b2f, offset: 0, size: 0 2024.10.22 01:51:56.101091 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 43046fa1481d6d8d922ae4adefe38b2f for removal 2024.10.22 01:51:56.103746 [ 1476 ] {} CacheMetadata: Key 43046fa1481d6d8d922ae4adefe38b2f is removed from metadata 2024.10.22 01:51:56.168150 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/wjn/gygddtdkoowzcbsovvmwctbgrdfyu. 2024.10.22 01:51:56.200938 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1152_1152_0 to 202410_1165_1165_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:56.242958 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_654_654_0 to 202410_661_661_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:56.276735 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 999540 2024.10.22 01:51:56.321337 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 75282 2024.10.22 01:51:56.333277 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9eaf90f81bdb172b415bf3d2743f70b8, offset: 0, size: 0 2024.10.22 01:51:56.336437 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9eaf90f81bdb172b415bf3d2743f70b8 for removal 2024.10.22 01:51:56.341573 [ 1476 ] {} CacheMetadata: Key 9eaf90f81bdb172b415bf3d2743f70b8 is removed from metadata 2024.10.22 01:51:56.462878 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_599_599_0, 202410_600_600_0, 202410_601_601_0, 202410_602_602_0, 202410_603_603_0, 202410_604_604_0] 2024.10.22 01:51:56.464900 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_599_599_0, 202410_600_600_0, 202410_601_601_0, 202410_602_602_0, 202410_603_603_0, 202410_604_604_0] 2024.10.22 01:51:56.595789 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_599_599_0, 202410_600_600_0, 202410_601_601_0, 202410_602_602_0, 202410_603_603_0, 202410_604_604_0] 2024.10.22 01:51:56.597678 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 75adc223457f6adb949ceabee9db99c9, offset: 0, size: 0 2024.10.22 01:51:56.601736 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 75adc223457f6adb949ceabee9db99c9 for removal 2024.10.22 01:51:56.607014 [ 1476 ] {} CacheMetadata: Key 75adc223457f6adb949ceabee9db99c9 is removed from metadata 2024.10.22 01:51:56.613090 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/wjn/gygddtdkoowzcbsovvmwctbgrdfyu. 2024.10.22 01:51:56.620816 [ 1237 ] {} DatabaseCatalog: Execution took 733 ms. 2024.10.22 01:51:56.884713 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 10388 2024.10.22 01:51:57.062670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(16304da4cd389d21e9dbe273c7eeb4d9) : [0, 124273]: Setting download as failed: File segment: [0, 124273], key: 16304da4cd389d21e9dbe273c7eeb4d9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:51:57.065071 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(16304da4cd389d21e9dbe273c7eeb4d9): No space left in cache to reserve 124274 bytes, will continue without cache download 2024.10.22 01:51:57.217671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 16304da4cd389d21e9dbe273c7eeb4d9, offset: 0, size: 0 2024.10.22 01:51:57.250652 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:57.315440 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:51:57.323596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 16304da4cd389d21e9dbe273c7eeb4d9 for removal 2024.10.22 01:51:57.432247 [ 1476 ] {} CacheMetadata: Key 16304da4cd389d21e9dbe273c7eeb4d9 is removed from metadata 2024.10.22 01:51:57.502681 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 8.23 MiB, will set to 1.74 GiB (RSS), difference: 8.47 MiB 2024.10.22 01:51:57.567932 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_683_683_0 (state Temporary) 2024.10.22 01:51:57.608939 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_683_683_0 to 202410_696_696_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:57.673091 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10388 2024.10.22 01:51:58.211267 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 570 2024.10.22 01:51:58.250323 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(9eaf90f81bdb172b415bf3d2743f70b8) : [0, 976457]: Setting download as failed: File segment: [0, 976457], key: 9eaf90f81bdb172b415bf3d2743f70b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:51:58.251932 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(9eaf90f81bdb172b415bf3d2743f70b8): No space left in cache to reserve 976458 bytes, will continue without cache download 2024.10.22 01:51:58.255240 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 69 entries to flush up to offset 76214 2024.10.22 01:51:58.298744 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_135_139_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:51:58.592455 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9eaf90f81bdb172b415bf3d2743f70b8, offset: 0, size: 0 2024.10.22 01:51:58.595409 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9eaf90f81bdb172b415bf3d2743f70b8 for removal 2024.10.22 01:51:58.600256 [ 1476 ] {} CacheMetadata: Key 9eaf90f81bdb172b415bf3d2743f70b8 is removed from metadata 2024.10.22 01:51:58.608478 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fki/cmcycbnwcockxlgwfmwfckzjsyttm. 2024.10.22 01:51:58.657173 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: fd1e5409b8d370a7a458bcd756b57dbf, offset: 0, size: 0 2024.10.22 01:51:58.659156 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key fd1e5409b8d370a7a458bcd756b57dbf for removal 2024.10.22 01:51:58.661519 [ 1476 ] {} CacheMetadata: Key fd1e5409b8d370a7a458bcd756b57dbf is removed from metadata 2024.10.22 01:51:58.696124 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 69a64914ed0da6057de6fc86644063f2, offset: 0, size: 0 2024.10.22 01:51:58.702446 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 69a64914ed0da6057de6fc86644063f2 for removal 2024.10.22 01:51:58.706935 [ 1476 ] {} CacheMetadata: Key 69a64914ed0da6057de6fc86644063f2 is removed from metadata 2024.10.22 01:51:58.754873 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:58.756625 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:51:58.813108 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fki/cmcycbnwcockxlgwfmwfckzjsyttm. 2024.10.22 01:51:58.863349 [ 1194 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:51:58.887564 [ 1194 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:51:58.891627 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:58.899278 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:51:58.920173 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_552_552_0 (state Temporary) 2024.10.22 01:51:59.049990 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_671_671_0 (state Temporary) 2024.10.22 01:51:59.088844 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_671_671_0 to 202410_677_677_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:59.104075 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_552_552_0 to 202410_562_562_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:51:59.119443 [ 1240 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_672_34 to 202410_673_673_0 2024.10.22 01:51:59.153567 [ 1240 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:51:59.156115 [ 1240 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.92 GiB. 2024.10.22 01:51:59.201341 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 570 2024.10.22 01:51:59.217519 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_673_35} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_672_34 to 202410_673_673_0 into Compact with storage Full 2024.10.22 01:51:59.234453 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 76214 2024.10.22 01:51:59.581129 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_673_35} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:51:59.589356 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_673_35} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_672_34, total 16619 rows starting from the beginning of the part 2024.10.22 01:51:59.769499 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_673_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_673_673_0, total 120 rows starting from the beginning of the part 2024.10.22 01:51:59.904925 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1273 bytes for 18a13d4358a62386c174a39c8e43df0b:0) 2024.10.22 01:51:59.915248 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: dd6c591a251649cb5d45196acf3cb4b2, offset: 0, size: 154 2024.10.22 01:51:59.921182 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key dd6c591a251649cb5d45196acf3cb4b2 for removal 2024.10.22 01:51:59.931634 [ 1476 ] {} CacheMetadata: Key dd6c591a251649cb5d45196acf3cb4b2 is removed from metadata 2024.10.22 01:51:59.932311 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1e86f91c9ae4ad6e181cac537505f5eb, offset: 0, size: 604 2024.10.22 01:51:59.935424 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 1e86f91c9ae4ad6e181cac537505f5eb for removal 2024.10.22 01:51:59.953057 [ 1476 ] {} CacheMetadata: Key 1e86f91c9ae4ad6e181cac537505f5eb is removed from metadata 2024.10.22 01:52:00.346348 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 75365 2024.10.22 01:52:00.628280 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:52:00.640355 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:52:00.655258 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:52:00.671574 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:52:00.684836 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:52:00.794891 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 1168 bytes for 69a64914ed0da6057de6fc86644063f2:0) 2024.10.22 01:52:00.798847 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 959ff03cc4598327cb62802dfccc4552, offset: 0, size: 1268 2024.10.22 01:52:00.803704 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 959ff03cc4598327cb62802dfccc4552 for removal 2024.10.22 01:52:00.807853 [ 1476 ] {} CacheMetadata: Key 959ff03cc4598327cb62802dfccc4552 is removed from metadata 2024.10.22 01:52:00.819418 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:52:00.849007 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:52:00.851563 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:52:00.852051 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column GoodEvent 2024.10.22 01:52:00.955131 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:00.959568 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:00.962137 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uvu/tgqgvxtisovrrtajegyzmhaeewols (LogSeriesLimiter: on interval from 2024-10-22 01:51:44 to 2024-10-22 01:52:00 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:52:01.160300 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1153_1153_0 (state Temporary) 2024.10.22 01:52:01.399794 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1153_1153_0 to 202410_1166_1166_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:01.412490 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_673_35} MergeTask::MergeProjectionsStage: Merge sorted 16739 rows, containing 13 columns (13 merged, 0 gathered) in 2.215563016 sec., 7555.190206334443 rows/sec., 3.27 MiB/sec. 2024.10.22 01:52:01.451150 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_673_35} MergedBlockOutputStream: filled checksums 202410_507_673_35 (state Temporary) 2024.10.22 01:52:01.540824 [ 1164 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1161_13 to 202410_1166_1166_0 2024.10.22 01:52:01.561251 [ 1164 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:01.656223 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 75365 2024.10.22 01:52:01.565765 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:01.671436 [ 1164 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 01:52:01.691008 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1161_13 to 202410_1166_1166_0 into Compact with storage Full 2024.10.22 01:52:01.714374 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 166 bytes for 241bd7c1498facdb5093ebd91f7b4e40:0) 2024.10.22 01:52:01.718465 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3c8fd4de014a4b6561011eb5a53ad992, offset: 0, size: 154 2024.10.22 01:52:01.731250 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3c8fd4de014a4b6561011eb5a53ad992 for removal 2024.10.22 01:52:01.736506 [ 1476 ] {} CacheMetadata: Key 3c8fd4de014a4b6561011eb5a53ad992 is removed from metadata 2024.10.22 01:52:01.798651 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 5.69 MiB, will set to 1.74 GiB (RSS), difference: 23.15 MiB 2024.10.22 01:52:01.926268 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:01.968638 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1161_13, total 4355 rows starting from the beginning of the part 2024.10.22 01:52:01.993179 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_673_35} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_673_35 to 202410_507_673_35 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:02.002040 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_673_35} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_672_34, 202410_673_673_0] -> 202410_507_673_35 2024.10.22 01:52:02.068048 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.51 MiB. 2024.10.22 01:52:02.172145 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1162_1162_0, total 100 rows starting from the beginning of the part 2024.10.22 01:52:02.265460 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1163_1163_0, total 73 rows starting from the beginning of the part 2024.10.22 01:52:02.378884 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1164_1164_0, total 96 rows starting from the beginning of the part 2024.10.22 01:52:02.567447 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1165_1165_0, total 82 rows starting from the beginning of the part 2024.10.22 01:52:02.572911 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(fd1e5409b8d370a7a458bcd756b57dbf) : [0, 12247216]: Setting download as failed: File segment: [0, 12247216], key: fd1e5409b8d370a7a458bcd756b57dbf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:52:02.585678 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(fd1e5409b8d370a7a458bcd756b57dbf): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:52:02.761768 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1166_1166_0, total 83 rows starting from the beginning of the part 2024.10.22 01:52:03.281803 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1000310 2024.10.22 01:52:03.589233 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bd8e23d74c29b2640031778e29b28638) : [0, 189674]: Setting download as failed: File segment: [0, 189674], key: bd8e23d74c29b2640031778e29b28638, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:03.595002 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bd8e23d74c29b2640031778e29b28638): No space left in cache to reserve 189675 bytes, will continue without cache download 2024.10.22 01:52:03.859282 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bd8e23d74c29b2640031778e29b28638, offset: 0, size: 0 2024.10.22 01:52:03.872056 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bd8e23d74c29b2640031778e29b28638 for removal 2024.10.22 01:52:03.875603 [ 1476 ] {} CacheMetadata: Key bd8e23d74c29b2640031778e29b28638 is removed from metadata 2024.10.22 01:52:04.042245 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: fd1e5409b8d370a7a458bcd756b57dbf, offset: 0, size: 0 2024.10.22 01:52:04.044138 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key fd1e5409b8d370a7a458bcd756b57dbf for removal 2024.10.22 01:52:04.045372 [ 1476 ] {} CacheMetadata: Key fd1e5409b8d370a7a458bcd756b57dbf is removed from metadata 2024.10.22 01:52:04.046920 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qif/sftyspystusxrvdclsgrxcwsamhzt. 2024.10.22 01:52:04.079775 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 989d035a5c7d674edd359d493cddead3, offset: 0, size: 0 2024.10.22 01:52:04.188149 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 989d035a5c7d674edd359d493cddead3 for removal 2024.10.22 01:52:04.194284 [ 1476 ] {} CacheMetadata: Key 989d035a5c7d674edd359d493cddead3 is removed from metadata 2024.10.22 01:52:04.219044 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:04.223370 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:04.244340 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a59b789b9b488cc11bd5fa8d6a6533a7, offset: 0, size: 0 2024.10.22 01:52:04.245986 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a59b789b9b488cc11bd5fa8d6a6533a7 for removal 2024.10.22 01:52:04.247729 [ 1476 ] {} CacheMetadata: Key a59b789b9b488cc11bd5fa8d6a6533a7 is removed from metadata 2024.10.22 01:52:04.251448 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qif/sftyspystusxrvdclsgrxcwsamhzt. 2024.10.22 01:52:04.359937 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_655_655_0 (state Temporary) 2024.10.22 01:52:04.454932 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_655_655_0 to 202410_662_662_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:04.484357 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1000310 2024.10.22 01:52:04.477274 [ 1184 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_652_126 to 202410_653_658_1 2024.10.22 01:52:04.603313 [ 1184 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:04.606798 [ 1184 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:04.746993 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_658_127} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_652_126 to 202410_653_658_1 into Wide with storage Full 2024.10.22 01:52:04.899334 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_658_127} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:04.907551 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_658_127} MergeTreeSequentialSource: Reading 124 marks from part 202410_1_652_126, total 1002348 rows starting from the beginning of the part 2024.10.22 01:52:04.969082 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} MergeTask::MergeProjectionsStage: Merge sorted 4789 rows, containing 13 columns (13 merged, 0 gathered) in 3.296995553 sec., 1452.5345645802877 rows/sec., 254.43 KiB/sec. 2024.10.22 01:52:04.982941 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_658_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_653_658_1, total 6932 rows starting from the beginning of the part 2024.10.22 01:52:05.059458 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 895 bytes for d8abf6a6b7076e2ceeb6b28383ed35a9:0) 2024.10.22 01:52:05.064017 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a94414b69b19190be11f18f84b57a3d9, offset: 0, size: 1268 2024.10.22 01:52:05.068764 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a94414b69b19190be11f18f84b57a3d9 for removal 2024.10.22 01:52:05.070695 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} MergedBlockOutputStream: filled checksums 202410_1096_1166_14 (state Temporary) 2024.10.22 01:52:05.070847 [ 1476 ] {} CacheMetadata: Key a94414b69b19190be11f18f84b57a3d9 is removed from metadata 2024.10.22 01:52:05.175284 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 10396 2024.10.22 01:52:05.205204 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:52:05.300573 [ 1682 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/sxi/alumqirmcsartnhsdyvfqfyiwhlub, size 3221811 2024.10.22 01:52:05.306868 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1166_14 to 202410_1096_1166_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:05.312986 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1166_14} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1161_13, 202410_1166_1166_0] -> 202410_1096_1166_14 2024.10.22 01:52:05.362161 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:52:05.448178 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:05.452776 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:05.496952 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_684_684_0 (state Temporary) 2024.10.22 01:52:05.676668 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_684_684_0 to 202410_697_697_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:05.777901 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 75453 2024.10.22 01:52:05.826462 [ 1168 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 3 parts from 202410_1_692_139 to 202410_694_694_0 2024.10.22 01:52:05.844697 [ 1168 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:05.851277 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:52:05.859241 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10396 2024.10.22 01:52:05.867673 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:52:05.849176 [ 1168 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:06.000300 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:52:06.107183 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:06.111365 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 580 bytes for a59b789b9b488cc11bd5fa8d6a6533a7:0) 2024.10.22 01:52:06.119794 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 587e89524478cfef0d462af9ade4e33c, offset: 0, size: 27959 2024.10.22 01:52:06.119790 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:06.124412 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 587e89524478cfef0d462af9ade4e33c for removal 2024.10.22 01:52:06.128077 [ 1476 ] {} CacheMetadata: Key 587e89524478cfef0d462af9ade4e33c is removed from metadata 2024.10.22 01:52:06.130157 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e028c5d145ff739e38f8fc3a912c9240, offset: 0, size: 142 2024.10.22 01:52:06.131943 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_694_140} MergeTask::PrepareStage: Merging 3 parts: from 202410_1_692_139 to 202410_694_694_0 into Compact with storage Full 2024.10.22 01:52:06.142554 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:52:06.162555 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:52:06.175679 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 6.27 MiB, will set to 1.73 GiB (RSS), difference: -11.45 MiB 2024.10.22 01:52:06.180716 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:52:06.237051 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e028c5d145ff739e38f8fc3a912c9240 for removal 2024.10.22 01:52:06.240503 [ 1476 ] {} CacheMetadata: Key e028c5d145ff739e38f8fc3a912c9240 is removed from metadata 2024.10.22 01:52:06.250033 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1154_1154_0 (state Temporary) 2024.10.22 01:52:06.327600 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:52:06.343150 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1154_1154_0 to 202410_1167_1167_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:06.368896 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_694_140} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:06.383464 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_694_140} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_692_139, total 10447 rows starting from the beginning of the part 2024.10.22 01:52:06.427463 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 75453 2024.10.22 01:52:06.452161 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 01:52:06.489532 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mqg/zvdwetmgyrotxtoralhrtwuidtjsh (LogSeriesLimiter: on interval from 2024-10-22 01:52:00 to 2024-10-22 01:52:06 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:52:06.552921 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_694_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_693_693_0, total 20 rows starting from the beginning of the part 2024.10.22 01:52:06.598001 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_694_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_694_694_0, total 14 rows starting from the beginning of the part 2024.10.22 01:52:06.703071 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 571 2024.10.22 01:52:06.750744 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 129 entries to flush up to offset 76343 2024.10.22 01:52:06.901662 [ 1269 ] {} DatabaseCatalog: Execution took 277 ms. 2024.10.22 01:52:07.036682 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:07.047606 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:07.100576 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_672_672_0 (state Temporary) 2024.10.22 01:52:07.140973 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:07.145377 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:07.232538 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_672_672_0 to 202410_678_678_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:07.302275 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 76343 2024.10.22 01:52:07.304632 [ 1209 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_673_35 to 202410_678_678_0 2024.10.22 01:52:07.221838 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_553_553_0 (state Temporary) 2024.10.22 01:52:07.325511 [ 1209 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.02 MiB using storage policy from min volume index 0 2024.10.22 01:52:07.330909 [ 1209 ] {} DiskLocal: Reserved 1.02 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:07.373368 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_673_35 to 202410_678_678_0 into Compact with storage Full 2024.10.22 01:52:07.434627 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:07.454670 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_673_35, total 16739 rows starting from the beginning of the part 2024.10.22 01:52:07.501209 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_553_553_0 to 202410_563_563_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:07.534230 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3191d8536c5f99a48fc5e59999704f52) : [0, 192129]: Setting download as failed: File segment: [0, 192129], key: 3191d8536c5f99a48fc5e59999704f52, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:07.535728 [ 1221 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_558_114 to 202410_563_563_0 2024.10.22 01:52:07.537265 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3191d8536c5f99a48fc5e59999704f52): No space left in cache to reserve 192130 bytes, will continue without cache download 2024.10.22 01:52:07.547136 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_674_674_0, total 106 rows starting from the beginning of the part 2024.10.22 01:52:07.548909 [ 1221 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:07.569403 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3191d8536c5f99a48fc5e59999704f52, offset: 0, size: 0 2024.10.22 01:52:07.571368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3191d8536c5f99a48fc5e59999704f52 for removal 2024.10.22 01:52:07.574496 [ 1476 ] {} CacheMetadata: Key 3191d8536c5f99a48fc5e59999704f52 is removed from metadata 2024.10.22 01:52:07.577458 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_694_140} MergeTask::MergeProjectionsStage: Merge sorted 10481 rows, containing 12 columns (12 merged, 0 gathered) in 1.672143851 sec., 6268.001400556536 rows/sec., 997.87 KiB/sec. 2024.10.22 01:52:07.578529 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:07.610487 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 571 2024.10.22 01:52:07.659276 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_694_140} MergedBlockOutputStream: filled checksums 202410_1_694_140 (state Temporary) 2024.10.22 01:52:07.661040 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_675_675_0, total 91 rows starting from the beginning of the part 2024.10.22 01:52:07.711262 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_558_114 to 202410_563_563_0 into Compact with storage Full 2024.10.22 01:52:07.750642 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_676_676_0, total 84 rows starting from the beginning of the part 2024.10.22 01:52:07.850414 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_677_677_0, total 69 rows starting from the beginning of the part 2024.10.22 01:52:07.963631 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:07.974760 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_558_114, total 602 rows starting from the beginning of the part 2024.10.22 01:52:07.999953 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_694_140} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_694_140 to 202410_1_694_140 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:08.002893 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_694_140} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 3 parts: [202410_1_692_139, 202410_694_694_0] -> 202410_1_694_140 2024.10.22 01:52:08.049065 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_658_127/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:52:08.053703 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_678_678_0, total 129 rows starting from the beginning of the part 2024.10.22 01:52:08.078960 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_592_116, 202410_593_598_1] 2024.10.22 01:52:08.080623 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_592_116, 202410_593_598_1] 2024.10.22 01:52:08.095724 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_592_116, 202410_593_598_1] 2024.10.22 01:52:08.247501 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.46 MiB. 2024.10.22 01:52:08.252461 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_559_559_0, total 1 rows starting from the beginning of the part 2024.10.22 01:52:08.583336 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_560_560_0, total 1 rows starting from the beginning of the part 2024.10.22 01:52:08.641834 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_561_561_0, total 1 rows starting from the beginning of the part 2024.10.22 01:52:08.657397 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(989d035a5c7d674edd359d493cddead3) : [0, 576372]: Setting download as failed: File segment: [0, 576372], key: 989d035a5c7d674edd359d493cddead3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:52:08.658861 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(989d035a5c7d674edd359d493cddead3): No space left in cache to reserve 576373 bytes, will continue without cache download 2024.10.22 01:52:08.796197 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_562_562_0, total 1 rows starting from the beginning of the part 2024.10.22 01:52:09.059414 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_563_563_0, total 1 rows starting from the beginning of the part 2024.10.22 01:52:09.188187 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 989d035a5c7d674edd359d493cddead3, offset: 0, size: 0 2024.10.22 01:52:09.192214 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 989d035a5c7d674edd359d493cddead3 for removal 2024.10.22 01:52:09.197441 [ 1476 ] {} CacheMetadata: Key 989d035a5c7d674edd359d493cddead3 is removed from metadata 2024.10.22 01:52:09.202934 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rqe/uhktjlphpxtywbffcraunnjfgetmj. 2024.10.22 01:52:09.242972 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 206908e73cbfb79534f402d50e9a4ae1, offset: 0, size: 0 2024.10.22 01:52:09.244135 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 206908e73cbfb79534f402d50e9a4ae1 for removal 2024.10.22 01:52:09.245606 [ 1476 ] {} CacheMetadata: Key 206908e73cbfb79534f402d50e9a4ae1 is removed from metadata 2024.10.22 01:52:09.292112 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8c14f3c4d613ee31fe5e2e7940b77d21, offset: 0, size: 0 2024.10.22 01:52:09.299299 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8c14f3c4d613ee31fe5e2e7940b77d21 for removal 2024.10.22 01:52:09.301792 [ 1476 ] {} CacheMetadata: Key 8c14f3c4d613ee31fe5e2e7940b77d21 is removed from metadata 2024.10.22 01:52:09.312036 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rqe/uhktjlphpxtywbffcraunnjfgetmj. 2024.10.22 01:52:09.827050 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(be1bc192a4d7752f1e46587d620e0cf3) : [0, 559183]: Setting download as failed: File segment: [0, 559183], key: be1bc192a4d7752f1e46587d620e0cf3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:09.828862 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(be1bc192a4d7752f1e46587d620e0cf3): No space left in cache to reserve 559184 bytes, will continue without cache download 2024.10.22 01:52:09.871980 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: be1bc192a4d7752f1e46587d620e0cf3, offset: 0, size: 0 2024.10.22 01:52:09.879015 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key be1bc192a4d7752f1e46587d620e0cf3 for removal 2024.10.22 01:52:09.882578 [ 1476 ] {} CacheMetadata: Key be1bc192a4d7752f1e46587d620e0cf3 is removed from metadata 2024.10.22 01:52:10.413566 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} MergeTask::MergeProjectionsStage: Merge sorted 17218 rows, containing 13 columns (13 merged, 0 gathered) in 3.045221591 sec., 5654.1041383940465 rows/sec., 2.45 MiB/sec. 2024.10.22 01:52:10.511351 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 9.06 MiB, will set to 1.74 GiB (RSS), difference: -11.01 MiB 2024.10.22 01:52:10.558273 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 101 entries to flush up to offset 75554 2024.10.22 01:52:10.624124 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:10.630445 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:10.643331 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} MergedBlockOutputStream: filled checksums 202410_507_678_36 (state Temporary) 2024.10.22 01:52:10.689463 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_678_36 to 202410_507_678_36 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:10.714899 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_678_36} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_673_35, 202410_678_678_0] -> 202410_507_678_36 2024.10.22 01:52:10.753450 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:10.769072 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:10.770774 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(206908e73cbfb79534f402d50e9a4ae1) : [0, 500666]: Setting download as failed: File segment: [0, 500666], key: 206908e73cbfb79534f402d50e9a4ae1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:52:10.772297 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(206908e73cbfb79534f402d50e9a4ae1): No space left in cache to reserve 500667 bytes, will continue without cache download 2024.10.22 01:52:10.782414 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.89 MiB. 2024.10.22 01:52:10.851573 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:52:10.854034 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:52:11.198338 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1155_1155_0 (state Temporary) 2024.10.22 01:52:11.201159 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 206908e73cbfb79534f402d50e9a4ae1, offset: 0, size: 0 2024.10.22 01:52:11.207361 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 206908e73cbfb79534f402d50e9a4ae1 for removal 2024.10.22 01:52:11.212228 [ 1476 ] {} CacheMetadata: Key 206908e73cbfb79534f402d50e9a4ae1 is removed from metadata 2024.10.22 01:52:11.215068 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uix/jhrhxjytpeethcbqdkrgpdtswkjns. 2024.10.22 01:52:11.268255 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e2fd614c651e8c05501f9521f127e3eb, offset: 0, size: 0 2024.10.22 01:52:11.271036 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e2fd614c651e8c05501f9521f127e3eb for removal 2024.10.22 01:52:11.274850 [ 1476 ] {} CacheMetadata: Key e2fd614c651e8c05501f9521f127e3eb is removed from metadata 2024.10.22 01:52:11.296594 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1155_1155_0 to 202410_1168_1168_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:11.317827 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 08d814b841b118884f2e4bf659eb7e51, offset: 0, size: 0 2024.10.22 01:52:11.325431 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 08d814b841b118884f2e4bf659eb7e51 for removal 2024.10.22 01:52:11.332305 [ 1476 ] {} CacheMetadata: Key 08d814b841b118884f2e4bf659eb7e51 is removed from metadata 2024.10.22 01:52:11.339820 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uix/jhrhxjytpeethcbqdkrgpdtswkjns. 2024.10.22 01:52:11.361804 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 75554 2024.10.22 01:52:11.301820 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} MergeTask::MergeProjectionsStage: Merge sorted 607 rows, containing 32 columns (32 merged, 0 gathered) in 3.595700765 sec., 168.8127126451803 rows/sec., 15.42 KiB/sec. 2024.10.22 01:52:11.446867 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} MergedBlockOutputStream: filled checksums 202410_1_563_115 (state Temporary) 2024.10.22 01:52:11.493309 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1001465 2024.10.22 01:52:11.551382 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_135_139_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:52:11.753596 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_563_115 to 202410_1_563_115 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:11.756448 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_563_115} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_558_114, 202410_563_563_0] -> 202410_1_563_115 2024.10.22 01:52:11.777594 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:52:11.916074 [ 1262 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 01:52:12.155771 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 11 file segments (while reserving 9036 bytes for c39079719b61087600123674c7ae48fc:0) 2024.10.22 01:52:12.158990 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 546bfa4e881fb7247f3141388356378b, offset: 0, size: 164 2024.10.22 01:52:12.160869 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:12.162403 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 546bfa4e881fb7247f3141388356378b for removal 2024.10.22 01:52:12.163521 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:12.164960 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a59b789b9b488cc11bd5fa8d6a6533a7, offset: 0, size: 580 2024.10.22 01:52:12.171603 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a59b789b9b488cc11bd5fa8d6a6533a7 for removal 2024.10.22 01:52:12.176082 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ebfd29616532ab97bf4b2bf9643e1016, offset: 0, size: 14524 2024.10.22 01:52:12.176245 [ 1476 ] {} CacheMetadata: Key 546bfa4e881fb7247f3141388356378b is removed from metadata 2024.10.22 01:52:12.181273 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ebfd29616532ab97bf4b2bf9643e1016 for removal 2024.10.22 01:52:12.193140 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 69a64914ed0da6057de6fc86644063f2, offset: 0, size: 1168 2024.10.22 01:52:12.196973 [ 1476 ] {} CacheMetadata: Key ebfd29616532ab97bf4b2bf9643e1016 is removed from metadata 2024.10.22 01:52:12.197393 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 69a64914ed0da6057de6fc86644063f2 for removal 2024.10.22 01:52:12.210127 [ 1476 ] {} CacheMetadata: Key 69a64914ed0da6057de6fc86644063f2 is removed from metadata 2024.10.22 01:52:12.235011 [ 1476 ] {} CacheMetadata: Key a59b789b9b488cc11bd5fa8d6a6533a7 is removed from metadata 2024.10.22 01:52:12.203000 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 390d7d15d9efd22521f778290c945d1d, offset: 0, size: 166 2024.10.22 01:52:12.241632 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 390d7d15d9efd22521f778290c945d1d for removal 2024.10.22 01:52:12.245046 [ 1476 ] {} CacheMetadata: Key 390d7d15d9efd22521f778290c945d1d is removed from metadata 2024.10.22 01:52:12.245385 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 241bd7c1498facdb5093ebd91f7b4e40, offset: 0, size: 166 2024.10.22 01:52:12.249606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 241bd7c1498facdb5093ebd91f7b4e40 for removal 2024.10.22 01:52:12.251824 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0f2beb50f9e0cac75818b622332aff2f, offset: 0, size: 155 2024.10.22 01:52:12.252718 [ 1476 ] {} CacheMetadata: Key 241bd7c1498facdb5093ebd91f7b4e40 is removed from metadata 2024.10.22 01:52:12.255773 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0f2beb50f9e0cac75818b622332aff2f for removal 2024.10.22 01:52:12.257687 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 75adc223457f6adb949ceabee9db99c9, offset: 0, size: 753 2024.10.22 01:52:12.260456 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 75adc223457f6adb949ceabee9db99c9 for removal 2024.10.22 01:52:12.263243 [ 1476 ] {} CacheMetadata: Key 0f2beb50f9e0cac75818b622332aff2f is removed from metadata 2024.10.22 01:52:12.265303 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b305f7b9b5b4bb9878c4ac7266103455, offset: 0, size: 795 2024.10.22 01:52:12.268975 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b305f7b9b5b4bb9878c4ac7266103455 for removal 2024.10.22 01:52:12.269520 [ 1476 ] {} CacheMetadata: Key 75adc223457f6adb949ceabee9db99c9 is removed from metadata 2024.10.22 01:52:12.271576 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1f87721327040a19ee5304e78c406816, offset: 0, size: 155 2024.10.22 01:52:12.273622 [ 1476 ] {} CacheMetadata: Key b305f7b9b5b4bb9878c4ac7266103455 is removed from metadata 2024.10.22 01:52:12.338839 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1f87721327040a19ee5304e78c406816 for removal 2024.10.22 01:52:12.381333 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f8e3548932750f4cdc5bc7fcf3152172, offset: 0, size: 793 2024.10.22 01:52:12.392731 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f8e3548932750f4cdc5bc7fcf3152172 for removal 2024.10.22 01:52:12.395041 [ 1476 ] {} CacheMetadata: Key 1f87721327040a19ee5304e78c406816 is removed from metadata 2024.10.22 01:52:12.427280 [ 1476 ] {} CacheMetadata: Key f8e3548932750f4cdc5bc7fcf3152172 is removed from metadata 2024.10.22 01:52:12.445755 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_656_656_0 (state Temporary) 2024.10.22 01:52:12.625415 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_656_656_0 to 202410_663_663_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:12.666207 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1001465 2024.10.22 01:52:13.266406 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(914f14352ee5c8db5bbe9351f51dacb0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 914f14352ee5c8db5bbe9351f51dacb0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:52:13.268948 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 914f14352ee5c8db5bbe9351f51dacb0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1400, kind: Regular, unbound: 0 2024.10.22 01:52:13.271749 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:52:13.364480 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 31 entries to flush up to offset 10427 2024.10.22 01:52:13.480483 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:13.485472 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:13.539396 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_685_685_0 (state Temporary) 2024.10.22 01:52:13.578698 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_685_685_0 to 202410_698_698_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:13.733629 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10427 2024.10.22 01:52:13.895699 [ 1218 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:52:13.930486 [ 1218 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:52:14.546946 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(e2fd614c651e8c05501f9521f127e3eb) : [0, 935637]: Setting download as failed: File segment: [0, 935637], key: e2fd614c651e8c05501f9521f127e3eb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:52:14.651319 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(e2fd614c651e8c05501f9521f127e3eb): No space left in cache to reserve 935638 bytes, will continue without cache download 2024.10.22 01:52:14.825250 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 126 entries to flush up to offset 76469 2024.10.22 01:52:15.114817 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 572 2024.10.22 01:52:15.323815 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:15.336671 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:15.383498 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 75646 2024.10.22 01:52:15.431415 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:15.435235 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:15.440694 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_138_138_0 (state Temporary) 2024.10.22 01:52:15.613066 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e2fd614c651e8c05501f9521f127e3eb, offset: 0, size: 0 2024.10.22 01:52:15.617532 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e2fd614c651e8c05501f9521f127e3eb for removal 2024.10.22 01:52:15.620607 [ 1476 ] {} CacheMetadata: Key e2fd614c651e8c05501f9521f127e3eb is removed from metadata 2024.10.22 01:52:15.684798 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_673_673_0 (state Temporary) 2024.10.22 01:52:15.712197 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_554_554_0 (state Temporary) 2024.10.22 01:52:15.713275 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 14222 bytes for d53d9c454c03a14d9d4bdf116477ccb2:0) 2024.10.22 01:52:15.752859 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_673_673_0 to 202410_679_679_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:15.769436 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:15.771201 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:15.838107 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_554_554_0 to 202410_564_564_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:15.838763 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b68471eaf488b72b03f9ad045a8da931, offset: 0, size: 0 2024.10.22 01:52:15.844829 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b68471eaf488b72b03f9ad045a8da931 for removal 2024.10.22 01:52:15.849202 [ 1476 ] {} CacheMetadata: Key b68471eaf488b72b03f9ad045a8da931 is removed from metadata 2024.10.22 01:52:15.851115 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qwb/dpnbhphpgwfmbszskwcozyedxsvql. 2024.10.22 01:52:15.857629 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1156_1156_0 (state Temporary) 2024.10.22 01:52:15.894381 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d4b685b24b3b7ce2faa71b44ffc336d4, offset: 0, size: 0 2024.10.22 01:52:15.908424 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d4b685b24b3b7ce2faa71b44ffc336d4 for removal 2024.10.22 01:52:15.917004 [ 1476 ] {} CacheMetadata: Key d4b685b24b3b7ce2faa71b44ffc336d4 is removed from metadata 2024.10.22 01:52:15.923793 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 76469 2024.10.22 01:52:15.928878 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: abe655a31518caa58910a0298cf796b4, offset: 0, size: 10377 2024.10.22 01:52:15.935312 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key abe655a31518caa58910a0298cf796b4 for removal 2024.10.22 01:52:15.938210 [ 1476 ] {} CacheMetadata: Key abe655a31518caa58910a0298cf796b4 is removed from metadata 2024.10.22 01:52:15.938963 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 58c4af0c7b781fa6597c31ddcd940aff, offset: 0, size: 154 2024.10.22 01:52:15.958230 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 58c4af0c7b781fa6597c31ddcd940aff for removal 2024.10.22 01:52:15.958477 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c2c0e5985303f508088d8cde091b0e2b, offset: 0, size: 0 2024.10.22 01:52:15.961088 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c2c0e5985303f508088d8cde091b0e2b for removal 2024.10.22 01:52:15.961456 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8c14f3c4d613ee31fe5e2e7940b77d21, offset: 0, size: 581 2024.10.22 01:52:15.962065 [ 1476 ] {} CacheMetadata: Key 58c4af0c7b781fa6597c31ddcd940aff is removed from metadata 2024.10.22 01:52:15.966431 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8c14f3c4d613ee31fe5e2e7940b77d21 for removal 2024.10.22 01:52:15.967767 [ 1476 ] {} CacheMetadata: Key c2c0e5985303f508088d8cde091b0e2b is removed from metadata 2024.10.22 01:52:15.968949 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1de9679d995af29a0ed973806052d7fd, offset: 0, size: 154 2024.10.22 01:52:15.971268 [ 1476 ] {} CacheMetadata: Key 8c14f3c4d613ee31fe5e2e7940b77d21 is removed from metadata 2024.10.22 01:52:15.973466 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qwb/dpnbhphpgwfmbszskwcozyedxsvql. 2024.10.22 01:52:15.973567 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1de9679d995af29a0ed973806052d7fd for removal 2024.10.22 01:52:15.977672 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qjh/nqirnmvrobagvysohkhbkcjzkiewc. 2024.10.22 01:52:15.979026 [ 1476 ] {} CacheMetadata: Key 1de9679d995af29a0ed973806052d7fd is removed from metadata 2024.10.22 01:52:16.049391 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1156_1156_0 to 202410_1169_1169_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:16.050798 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 255fcf15322de323d6f97814cd234232, offset: 0, size: 0 2024.10.22 01:52:16.053728 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 255fcf15322de323d6f97814cd234232 for removal 2024.10.22 01:52:16.056758 [ 1476 ] {} CacheMetadata: Key 255fcf15322de323d6f97814cd234232 is removed from metadata 2024.10.22 01:52:16.093417 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b0badf436c93df2620a4720cf8785284, offset: 0, size: 0 2024.10.22 01:52:16.097941 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b0badf436c93df2620a4720cf8785284 for removal 2024.10.22 01:52:16.107532 [ 1476 ] {} CacheMetadata: Key b0badf436c93df2620a4720cf8785284 is removed from metadata 2024.10.22 01:52:16.119931 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qjh/nqirnmvrobagvysohkhbkcjzkiewc. 2024.10.22 01:52:16.127119 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/scp/cfdcvlkarmybtghpvascanojugkxl. 2024.10.22 01:52:16.137433 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 572 2024.10.22 01:52:16.162276 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 780c5a48f5900df0127c219bb9942e95, offset: 0, size: 0 2024.10.22 01:52:16.243761 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 780c5a48f5900df0127c219bb9942e95 for removal 2024.10.22 01:52:16.248007 [ 1476 ] {} CacheMetadata: Key 780c5a48f5900df0127c219bb9942e95 is removed from metadata 2024.10.22 01:52:16.307058 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 75646 2024.10.22 01:52:16.380152 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cc036cf12ee7050eed9deacbab9ccef8, offset: 0, size: 0 2024.10.22 01:52:16.383332 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cc036cf12ee7050eed9deacbab9ccef8 for removal 2024.10.22 01:52:16.390097 [ 1476 ] {} CacheMetadata: Key cc036cf12ee7050eed9deacbab9ccef8 is removed from metadata 2024.10.22 01:52:16.417387 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 13.32 MiB, will set to 1.74 GiB (RSS), difference: 10.94 MiB 2024.10.22 01:52:16.501778 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/scp/cfdcvlkarmybtghpvascanojugkxl. 2024.10.22 01:52:17.375031 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1058_503, 202410_1059_1064_1] 2024.10.22 01:52:17.376895 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1058_503, 202410_1059_1064_1] 2024.10.22 01:52:17.439945 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1058_503, 202410_1059_1064_1] 2024.10.22 01:52:17.500476 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d5cb7b24107b176ce1a0b79f56ea5f92) : [0, 32623]: Setting download as failed: File segment: [0, 32623], key: d5cb7b24107b176ce1a0b79f56ea5f92, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:17.504117 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d5cb7b24107b176ce1a0b79f56ea5f92): No space left in cache to reserve 32624 bytes, will continue without cache download 2024.10.22 01:52:17.553340 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d5cb7b24107b176ce1a0b79f56ea5f92, offset: 0, size: 0 2024.10.22 01:52:17.554939 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d5cb7b24107b176ce1a0b79f56ea5f92 for removal 2024.10.22 01:52:17.557132 [ 1476 ] {} CacheMetadata: Key d5cb7b24107b176ce1a0b79f56ea5f92 is removed from metadata 2024.10.22 01:52:18.577181 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 25592 bytes for 99926f4ce1fb7030568b50b688a53465:0) 2024.10.22 01:52:18.605329 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_658_127/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:52:18.622695 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1065_1065_0, 202410_1066_1066_0, 202410_1067_1067_0, 202410_1068_1068_0, 202410_1069_1069_0, 202410_1070_1070_0] 2024.10.22 01:52:18.646623 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1065_1065_0, 202410_1066_1066_0, 202410_1067_1067_0, 202410_1068_1068_0, 202410_1069_1069_0, 202410_1070_1070_0] 2024.10.22 01:52:18.673508 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1065_1065_0, 202410_1066_1066_0, 202410_1067_1067_0, 202410_1068_1068_0, 202410_1069_1069_0, 202410_1070_1070_0] 2024.10.22 01:52:18.793798 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 08d814b841b118884f2e4bf659eb7e51, offset: 0, size: 990 2024.10.22 01:52:18.922606 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 08d814b841b118884f2e4bf659eb7e51 for removal 2024.10.22 01:52:18.928192 [ 1476 ] {} CacheMetadata: Key 08d814b841b118884f2e4bf659eb7e51 is removed from metadata 2024.10.22 01:52:18.929400 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c39079719b61087600123674c7ae48fc, offset: 0, size: 9036 2024.10.22 01:52:18.935673 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c39079719b61087600123674c7ae48fc for removal 2024.10.22 01:52:18.940972 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d53d9c454c03a14d9d4bdf116477ccb2, offset: 0, size: 14222 2024.10.22 01:52:18.947307 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d53d9c454c03a14d9d4bdf116477ccb2 for removal 2024.10.22 01:52:18.950418 [ 1476 ] {} CacheMetadata: Key d53d9c454c03a14d9d4bdf116477ccb2 is removed from metadata 2024.10.22 01:52:18.951432 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 55c092fa1f113f2caa20e5826dc515dc, offset: 0, size: 154 2024.10.22 01:52:18.956262 [ 1476 ] {} CacheMetadata: Key c39079719b61087600123674c7ae48fc is removed from metadata 2024.10.22 01:52:19.055032 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 55c092fa1f113f2caa20e5826dc515dc for removal 2024.10.22 01:52:19.057532 [ 1476 ] {} CacheMetadata: Key 55c092fa1f113f2caa20e5826dc515dc is removed from metadata 2024.10.22 01:52:19.058536 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 493d0a3dbcc7b191aa5feb3fd461d93a, offset: 0, size: 142 2024.10.22 01:52:19.064245 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 493d0a3dbcc7b191aa5feb3fd461d93a for removal 2024.10.22 01:52:19.068216 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: f540f3ed2c70f074ee2336fff9dbeb84, offset: 0, size: 37 2024.10.22 01:52:19.070942 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key f540f3ed2c70f074ee2336fff9dbeb84 for removal 2024.10.22 01:52:19.336044 [ 1476 ] {} CacheMetadata: Key f540f3ed2c70f074ee2336fff9dbeb84 is removed from metadata 2024.10.22 01:52:19.352136 [ 1476 ] {} CacheMetadata: Key 493d0a3dbcc7b191aa5feb3fd461d93a is removed from metadata 2024.10.22 01:52:19.675086 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 614 entries to flush up to offset 1002079 2024.10.22 01:52:20.001834 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(780c5a48f5900df0127c219bb9942e95) : [0, 1483774]: Setting download as failed: File segment: [0, 1483774], key: 780c5a48f5900df0127c219bb9942e95, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:52:20.006600 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(780c5a48f5900df0127c219bb9942e95): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:52:20.059030 [ 1686 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uvu/tgqgvxtisovrrtajegyzmhaeewols, size 25592 2024.10.22 01:52:20.112518 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:20.118665 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:20.327442 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 75742 2024.10.22 01:52:20.570295 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_657_657_0 (state Temporary) 2024.10.22 01:52:20.604290 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 13.21 MiB, will set to 1.74 GiB (RSS), difference: -5.60 MiB 2024.10.22 01:52:20.631392 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_657_657_0 to 202410_664_664_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:20.671014 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1002079 2024.10.22 01:52:20.671524 [ 1157 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_659_659_0 to 202410_664_664_0 2024.10.22 01:52:20.692922 [ 1157 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:20.741630 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:20.744804 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:20.798734 [ 1157 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:20.841063 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_659_659_0 to 202410_664_664_0 into Compact with storage Full 2024.10.22 01:52:20.851309 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:52:20.856465 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:52:20.898222 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column EventTime 2024.10.22 01:52:20.915068 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column EventTime 2024.10.22 01:52:20.939034 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column EventTime 2024.10.22 01:52:20.985162 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:21.018873 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1157_1157_0 (state Temporary) 2024.10.22 01:52:21.051091 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column EventTime 2024.10.22 01:52:21.055948 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_659_659_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:52:21.096134 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1157_1157_0 to 202410_1170_1170_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:21.095214 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3e77ac35b084fd2061a8191d20017fbb) : [0, 47614]: Setting download as failed: File segment: [0, 47614], key: 3e77ac35b084fd2061a8191d20017fbb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:21.100381 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3e77ac35b084fd2061a8191d20017fbb): No space left in cache to reserve 47615 bytes, will continue without cache download 2024.10.22 01:52:21.136342 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3e77ac35b084fd2061a8191d20017fbb, offset: 0, size: 0 2024.10.22 01:52:21.138434 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3e77ac35b084fd2061a8191d20017fbb for removal 2024.10.22 01:52:21.143632 [ 1476 ] {} CacheMetadata: Key 3e77ac35b084fd2061a8191d20017fbb is removed from metadata 2024.10.22 01:52:21.168204 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_660_660_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:52:21.168450 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column EventTime 2024.10.22 01:52:21.192417 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column EventTime 2024.10.22 01:52:21.192370 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_661_661_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:52:21.218733 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_662_662_0, total 770 rows starting from the beginning of the part 2024.10.22 01:52:21.219304 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column EventTime 2024.10.22 01:52:21.221379 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 780c5a48f5900df0127c219bb9942e95, offset: 0, size: 0 2024.10.22 01:52:21.229782 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 780c5a48f5900df0127c219bb9942e95 for removal 2024.10.22 01:52:21.238620 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 10433 2024.10.22 01:52:21.245607 [ 1476 ] {} CacheMetadata: Key 780c5a48f5900df0127c219bb9942e95 is removed from metadata 2024.10.22 01:52:21.247035 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_663_663_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:52:21.278595 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 75742 2024.10.22 01:52:21.278784 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_664_664_0, total 614 rows starting from the beginning of the part 2024.10.22 01:52:21.300553 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gsb/lecjacxmehmhwqzwavcbwiputdzme (LogSeriesLimiter: on interval from 2024-10-22 01:52:06 to 2024-10-22 01:52:21 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:52:21.477085 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 43424f77e7d7a752af051891d1688115, offset: 0, size: 0 2024.10.22 01:52:21.478670 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 43424f77e7d7a752af051891d1688115 for removal 2024.10.22 01:52:21.503668 [ 1476 ] {} CacheMetadata: Key 43424f77e7d7a752af051891d1688115 is removed from metadata 2024.10.22 01:52:21.527796 [ 1186 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 308 ms. 2024.10.22 01:52:21.537139 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d62c1799e2aef9238717dd69d14f1f65, offset: 0, size: 0 2024.10.22 01:52:21.539150 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d62c1799e2aef9238717dd69d14f1f65 for removal 2024.10.22 01:52:21.543872 [ 1476 ] {} CacheMetadata: Key d62c1799e2aef9238717dd69d14f1f65 is removed from metadata 2024.10.22 01:52:21.646732 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tej/btsmkwpzcgfqzvdwcgqcegblabsbb. 2024.10.22 01:52:21.662541 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:21.664024 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:21.675748 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c8fc994da8d134980a087b5d6a37b0ef, offset: 0, size: 0 2024.10.22 01:52:21.679101 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c8fc994da8d134980a087b5d6a37b0ef for removal 2024.10.22 01:52:21.682758 [ 1476 ] {} CacheMetadata: Key c8fc994da8d134980a087b5d6a37b0ef is removed from metadata 2024.10.22 01:52:21.722341 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_686_686_0 (state Temporary) 2024.10.22 01:52:21.744478 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 69cc27d6bbce59fc0548a6ca753473ba, offset: 0, size: 0 2024.10.22 01:52:21.747479 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 69cc27d6bbce59fc0548a6ca753473ba for removal 2024.10.22 01:52:21.750261 [ 1476 ] {} CacheMetadata: Key 69cc27d6bbce59fc0548a6ca753473ba is removed from metadata 2024.10.22 01:52:21.765511 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tej/btsmkwpzcgfqzvdwcgqcegblabsbb. 2024.10.22 01:52:21.918315 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 590a8acd55ca9ed4df37153b0cd27093, offset: 0, size: 0 2024.10.22 01:52:21.921671 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 590a8acd55ca9ed4df37153b0cd27093 for removal 2024.10.22 01:52:21.928906 [ 1476 ] {} CacheMetadata: Key 590a8acd55ca9ed4df37153b0cd27093 is removed from metadata 2024.10.22 01:52:21.971307 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_686_686_0 to 202410_699_699_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:21.971881 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ed3f7402827d7c5de2677605e00b99cd, offset: 0, size: 0 2024.10.22 01:52:21.974221 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ed3f7402827d7c5de2677605e00b99cd for removal 2024.10.22 01:52:21.977387 [ 1476 ] {} CacheMetadata: Key ed3f7402827d7c5de2677605e00b99cd is removed from metadata 2024.10.22 01:52:21.978219 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kph/yrnywuhxpjgyursowlqkzieuxefzf. 2024.10.22 01:52:22.010715 [ 1183 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 5 parts from 202410_1_694_140 to 202410_698_698_0 2024.10.22 01:52:22.025142 [ 1183 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:22.131318 [ 1183 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:22.240469 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7562a0cb9668413c597572a08bad6683, offset: 0, size: 0 2024.10.22 01:52:22.242834 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7562a0cb9668413c597572a08bad6683 for removal 2024.10.22 01:52:22.247350 [ 1476 ] {} CacheMetadata: Key 7562a0cb9668413c597572a08bad6683 is removed from metadata 2024.10.22 01:52:22.259652 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_698_141} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_694_140 to 202410_698_698_0 into Compact with storage Full 2024.10.22 01:52:22.267571 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} MergeTask::MergeProjectionsStage: Merge sorted 6389 rows, containing 4 columns (4 merged, 0 gathered) in 1.438413442 sec., 4441.699314987352 rows/sec., 69.40 KiB/sec. 2024.10.22 01:52:22.356854 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10433 2024.10.22 01:52:22.372634 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f94b59c404f81ee908204e484648f460) : [0, 111356]: Setting download as failed: File segment: [0, 111356], key: f94b59c404f81ee908204e484648f460, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:22.374065 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f94b59c404f81ee908204e484648f460): No space left in cache to reserve 111357 bytes, will continue without cache download 2024.10.22 01:52:22.411091 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_698_141} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:22.433148 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_698_141} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_694_140, total 10481 rows starting from the beginning of the part 2024.10.22 01:52:22.449183 [ 1163 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 270 ms. 2024.10.22 01:52:22.459355 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_698_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_695_695_0, total 5 rows starting from the beginning of the part 2024.10.22 01:52:22.483905 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_698_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_696_696_0, total 8 rows starting from the beginning of the part 2024.10.22 01:52:22.531103 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_698_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_697_697_0, total 8 rows starting from the beginning of the part 2024.10.22 01:52:22.549112 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f94b59c404f81ee908204e484648f460, offset: 0, size: 0 2024.10.22 01:52:22.549521 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} MergedBlockOutputStream: filled checksums 202410_659_664_1 (state Temporary) 2024.10.22 01:52:22.550548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f94b59c404f81ee908204e484648f460 for removal 2024.10.22 01:52:22.552491 [ 1476 ] {} CacheMetadata: Key f94b59c404f81ee908204e484648f460 is removed from metadata 2024.10.22 01:52:22.565665 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c0df5e84f8e88f79ff3e63d84a8e8d08, offset: 0, size: 0 2024.10.22 01:52:22.569572 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c0df5e84f8e88f79ff3e63d84a8e8d08 for removal 2024.10.22 01:52:22.573081 [ 1476 ] {} CacheMetadata: Key c0df5e84f8e88f79ff3e63d84a8e8d08 is removed from metadata 2024.10.22 01:52:22.577507 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_698_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_698_698_0, total 31 rows starting from the beginning of the part 2024.10.22 01:52:22.603568 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kph/yrnywuhxpjgyursowlqkzieuxefzf. 2024.10.22 01:52:22.614960 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zyt/xsrqgzxbpwxdivelzovyjiiyghmod. 2024.10.22 01:52:22.733179 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_659_664_1 to 202410_659_664_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:22.740153 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_659_664_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_659_659_0, 202410_664_664_0] -> 202410_659_664_1 2024.10.22 01:52:22.784098 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c2a2428a2f79041dff54aae80c712112, offset: 0, size: 0 2024.10.22 01:52:22.791772 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c2a2428a2f79041dff54aae80c712112 for removal 2024.10.22 01:52:22.795713 [ 1476 ] {} CacheMetadata: Key c2a2428a2f79041dff54aae80c712112 is removed from metadata 2024.10.22 01:52:22.819490 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_616_22, 202410_617_617_0, 202410_618_618_0, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0] 2024.10.22 01:52:22.826322 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:52:22.823233 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_616_22, 202410_617_617_0, 202410_618_618_0, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0] 2024.10.22 01:52:22.869300 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 636478c81f03e131df4a42f58ee60777, offset: 0, size: 0 2024.10.22 01:52:22.873148 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_616_22, 202410_617_617_0, 202410_618_618_0, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0] 2024.10.22 01:52:22.873086 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 636478c81f03e131df4a42f58ee60777 for removal 2024.10.22 01:52:22.876302 [ 1476 ] {} CacheMetadata: Key 636478c81f03e131df4a42f58ee60777 is removed from metadata 2024.10.22 01:52:22.883491 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zyt/xsrqgzxbpwxdivelzovyjiiyghmod. 2024.10.22 01:52:23.453613 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 76569 2024.10.22 01:52:23.666732 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 9.69 MiB, will set to 1.75 GiB (RSS), difference: -1.10 MiB 2024.10.22 01:52:23.743617 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 573 2024.10.22 01:52:23.814396 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(96304bdaef039666a5bbf455c2106516) : [0, 127135]: Setting download as failed: File segment: [0, 127135], key: 96304bdaef039666a5bbf455c2106516, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:23.819225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(96304bdaef039666a5bbf455c2106516): No space left in cache to reserve 127136 bytes, will continue without cache download 2024.10.22 01:52:23.851145 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 96304bdaef039666a5bbf455c2106516, offset: 0, size: 0 2024.10.22 01:52:23.853915 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 96304bdaef039666a5bbf455c2106516 for removal 2024.10.22 01:52:23.857415 [ 1476 ] {} CacheMetadata: Key 96304bdaef039666a5bbf455c2106516 is removed from metadata 2024.10.22 01:52:23.915418 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_698_141} MergeTask::MergeProjectionsStage: Merge sorted 10533 rows, containing 12 columns (12 merged, 0 gathered) in 1.706603735 sec., 6171.907270553348 rows/sec., 982.62 KiB/sec. 2024.10.22 01:52:24.056477 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:24.062866 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:24.071363 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_698_141} MergedBlockOutputStream: filled checksums 202410_1_698_141 (state Temporary) 2024.10.22 01:52:24.153387 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:24.217205 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_674_674_0 (state Temporary) 2024.10.22 01:52:24.257783 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:24.417000 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_555_555_0 (state Temporary) 2024.10.22 01:52:24.564146 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_674_674_0 to 202410_680_680_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:24.608468 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_555_555_0 to 202410_565_565_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:24.614851 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 76569 2024.10.22 01:52:24.675092 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_698_141} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_698_141 to 202410_1_698_141 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:24.679728 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_698_141} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 5 parts: [202410_1_694_140, 202410_698_698_0] -> 202410_1_698_141 2024.10.22 01:52:24.731918 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.50 MiB. 2024.10.22 01:52:24.736854 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 573 2024.10.22 01:52:24.886548 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(c2a2428a2f79041dff54aae80c712112) : [0, 829653]: Setting download as failed: File segment: [0, 829653], key: c2a2428a2f79041dff54aae80c712112, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:52:24.889655 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(c2a2428a2f79041dff54aae80c712112): No space left in cache to reserve 829654 bytes, will continue without cache download 2024.10.22 01:52:24.983184 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c2a2428a2f79041dff54aae80c712112, offset: 0, size: 0 2024.10.22 01:52:24.989509 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c2a2428a2f79041dff54aae80c712112 for removal 2024.10.22 01:52:24.996640 [ 1476 ] {} CacheMetadata: Key c2a2428a2f79041dff54aae80c712112 is removed from metadata 2024.10.22 01:52:25.003336 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/iyr/ujfqvgdmvcrfxzxfvlztfemmvpsje. 2024.10.22 01:52:25.170282 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bee28f16fcdbf64249b9fab13d00f177, offset: 0, size: 0 2024.10.22 01:52:25.171931 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bee28f16fcdbf64249b9fab13d00f177 for removal 2024.10.22 01:52:25.176279 [ 1476 ] {} CacheMetadata: Key bee28f16fcdbf64249b9fab13d00f177 is removed from metadata 2024.10.22 01:52:25.220468 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e3ddb9a73e55005e081fac5a8d8e1b6f, offset: 0, size: 0 2024.10.22 01:52:25.225768 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e3ddb9a73e55005e081fac5a8d8e1b6f for removal 2024.10.22 01:52:25.233173 [ 1476 ] {} CacheMetadata: Key e3ddb9a73e55005e081fac5a8d8e1b6f is removed from metadata 2024.10.22 01:52:25.240092 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/iyr/ujfqvgdmvcrfxzxfvlztfemmvpsje. 2024.10.22 01:52:25.297268 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 142 entries to flush up to offset 75884 2024.10.22 01:52:25.477603 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 8.36 MiB, will set to 1.74 GiB (RSS), difference: -2.89 MiB 2024.10.22 01:52:25.611091 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_138_138_0 to 202410_141_141_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:25.686262 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_135_139_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:52:26.140645 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:26.147335 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:26.350563 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1158_1158_0 (state Temporary) 2024.10.22 01:52:26.446307 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1158_1158_0 to 202410_1171_1171_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:26.462753 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1166_14 to 202410_1171_1171_0 2024.10.22 01:52:26.472487 [ 1264 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:26.474390 [ 1264 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.91 GiB. 2024.10.22 01:52:26.484138 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 75884 2024.10.22 01:52:26.516147 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1166_14 to 202410_1171_1171_0 into Compact with storage Full 2024.10.22 01:52:26.572049 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(945d7759c6d9937b9d89b9b3d86f621a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 945d7759c6d9937b9d89b9b3d86f621a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:52:26.577023 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 945d7759c6d9937b9d89b9b3d86f621a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:52:26.581086 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:52:26.849332 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:27.011380 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1166_14, total 4789 rows starting from the beginning of the part 2024.10.22 01:52:27.261632 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1167_1167_0, total 88 rows starting from the beginning of the part 2024.10.22 01:52:27.386095 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1168_1168_0, total 101 rows starting from the beginning of the part 2024.10.22 01:52:27.673299 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 926 entries to flush up to offset 1003005 2024.10.22 01:52:27.723708 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1169_1169_0, total 92 rows starting from the beginning of the part 2024.10.22 01:52:27.967893 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b660f799517322cefee06a724c1dece8) : [0, 116788]: Setting download as failed: File segment: [0, 116788], key: b660f799517322cefee06a724c1dece8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:27.969143 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b660f799517322cefee06a724c1dece8): No space left in cache to reserve 116789 bytes, will continue without cache download 2024.10.22 01:52:27.998344 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b660f799517322cefee06a724c1dece8, offset: 0, size: 0 2024.10.22 01:52:28.000873 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b660f799517322cefee06a724c1dece8 for removal 2024.10.22 01:52:28.006800 [ 1476 ] {} CacheMetadata: Key b660f799517322cefee06a724c1dece8 is removed from metadata 2024.10.22 01:52:28.087941 [ 1723 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mqg/zvdwetmgyrotxtoralhrtwuidtjsh, size 5110016 2024.10.22 01:52:28.110491 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1170_1170_0, total 96 rows starting from the beginning of the part 2024.10.22 01:52:28.371316 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1171_1171_0, total 142 rows starting from the beginning of the part 2024.10.22 01:52:28.371754 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:28.387521 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:28.470846 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:52:28.599775 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:52:28.610807 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:52:28.637726 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:52:28.651002 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:52:28.768734 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:52:28.791953 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:52:28.840859 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 01:52:28.910821 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xpa/hebccyzvbkhjltamwqbgltswqaohs (LogSeriesLimiter: on interval from 2024-10-22 01:52:21 to 2024-10-22 01:52:28 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:52:28.952008 [ 1198 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:52:28.880455 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5530 2024.10.22 01:52:29.071659 [ 1198 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:52:29.131377 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_658_658_0 (state Temporary) 2024.10.22 01:52:29.297757 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(bee28f16fcdbf64249b9fab13d00f177) : [0, 3358152]: Setting download as failed: File segment: [0, 3358152], key: bee28f16fcdbf64249b9fab13d00f177, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:52:29.301379 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(bee28f16fcdbf64249b9fab13d00f177): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:52:29.416747 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_658_658_0 to 202410_665_665_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:29.553601 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1003005 2024.10.22 01:52:29.565337 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_658_127/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:52:29.771518 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 13.26 MiB, will set to 1.74 GiB (RSS), difference: -3.53 MiB 2024.10.22 01:52:29.858792 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 10452 2024.10.22 01:52:30.188659 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:30.197653 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:30.274211 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_687_687_0 (state Temporary) 2024.10.22 01:52:30.448409 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_687_687_0 to 202410_700_700_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:30.508107 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} MergeTask::MergeProjectionsStage: Merge sorted 5308 rows, containing 13 columns (13 merged, 0 gathered) in 4.007667805 sec., 1324.4610726911285 rows/sec., 230.90 KiB/sec. 2024.10.22 01:52:30.518069 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10452 2024.10.22 01:52:30.616065 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 75942 2024.10.22 01:52:30.692540 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bee28f16fcdbf64249b9fab13d00f177, offset: 0, size: 0 2024.10.22 01:52:30.695662 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bee28f16fcdbf64249b9fab13d00f177 for removal 2024.10.22 01:52:30.704747 [ 1476 ] {} CacheMetadata: Key bee28f16fcdbf64249b9fab13d00f177 is removed from metadata 2024.10.22 01:52:30.736657 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} MergedBlockOutputStream: filled checksums 202410_1096_1171_15 (state Temporary) 2024.10.22 01:52:30.812152 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fyd/wgxgemeqbrumiqcxbrmycouzujlvx. 2024.10.22 01:52:30.852533 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:52:30.857399 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:52:30.869039 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:30.874740 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:30.910798 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1171_15 to 202410_1096_1171_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:30.917189 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1171_15} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1166_14, 202410_1171_1171_0] -> 202410_1096_1171_15 2024.10.22 01:52:30.983178 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a201f0cb708f13bd91bd5481715086cc, offset: 0, size: 0 2024.10.22 01:52:30.988704 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a201f0cb708f13bd91bd5481715086cc for removal 2024.10.22 01:52:31.065479 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(db181c1b0a73e2b556bc76469438e4d9) : [0, 489593]: Setting download as failed: File segment: [0, 489593], key: db181c1b0a73e2b556bc76469438e4d9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:31.068950 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:52:31.099803 [ 1476 ] {} CacheMetadata: Key a201f0cb708f13bd91bd5481715086cc is removed from metadata 2024.10.22 01:52:31.165703 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1159_1159_0 (state Temporary) 2024.10.22 01:52:31.168131 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(db181c1b0a73e2b556bc76469438e4d9): No space left in cache to reserve 489594 bytes, will continue without cache download 2024.10.22 01:52:31.175890 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a8192f922b61cd8c55b1e1aa89f34c4a, offset: 0, size: 0 2024.10.22 01:52:31.182697 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a8192f922b61cd8c55b1e1aa89f34c4a for removal 2024.10.22 01:52:31.218614 [ 1476 ] {} CacheMetadata: Key a8192f922b61cd8c55b1e1aa89f34c4a is removed from metadata 2024.10.22 01:52:31.316796 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fyd/wgxgemeqbrumiqcxbrmycouzujlvx. 2024.10.22 01:52:31.317323 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: db181c1b0a73e2b556bc76469438e4d9, offset: 0, size: 0 2024.10.22 01:52:31.321133 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key db181c1b0a73e2b556bc76469438e4d9 for removal 2024.10.22 01:52:31.327928 [ 1476 ] {} CacheMetadata: Key db181c1b0a73e2b556bc76469438e4d9 is removed from metadata 2024.10.22 01:52:31.360270 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1159_1159_0 to 202410_1172_1172_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:31.627347 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 75942 2024.10.22 01:52:31.650365 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 13.89 MiB, will set to 1.75 GiB (RSS), difference: 13.21 MiB 2024.10.22 01:52:32.134548 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 89 entries to flush up to offset 76658 2024.10.22 01:52:32.239747 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 574 2024.10.22 01:52:32.578936 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:32.585205 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:32.636569 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:32.642482 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:32.756523 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1164 bytes for a8192f922b61cd8c55b1e1aa89f34c4a:0) 2024.10.22 01:52:32.762518 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cc036cf12ee7050eed9deacbab9ccef8, offset: 0, size: 1002 2024.10.22 01:52:32.782528 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cc036cf12ee7050eed9deacbab9ccef8 for removal 2024.10.22 01:52:32.787016 [ 1476 ] {} CacheMetadata: Key cc036cf12ee7050eed9deacbab9ccef8 is removed from metadata 2024.10.22 01:52:32.791098 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f9284209590bb9fb42e3de9986473822, offset: 0, size: 142 2024.10.22 01:52:32.805893 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_556_556_0 (state Temporary) 2024.10.22 01:52:32.867267 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_675_675_0 (state Temporary) 2024.10.22 01:52:32.876168 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_556_556_0 to 202410_566_566_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:32.897375 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f9284209590bb9fb42e3de9986473822 for removal 2024.10.22 01:52:32.903483 [ 1476 ] {} CacheMetadata: Key f9284209590bb9fb42e3de9986473822 is removed from metadata 2024.10.22 01:52:32.919430 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_675_675_0 to 202410_681_681_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:32.985047 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 76658 2024.10.22 01:52:33.080467 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 574 2024.10.22 01:52:33.513791 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 5 old parts to remove. Parts: [202410_1_636_127, 202410_637_637_0, 202410_638_638_0, 202410_639_639_0, 202410_640_640_0] 2024.10.22 01:52:33.517051 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 5 parts from filesystem (serially): Parts: [202410_1_636_127, 202410_637_637_0, 202410_638_638_0, 202410_639_639_0, 202410_640_640_0] 2024.10.22 01:52:33.658585 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 5 parts from memory: Parts: [202410_1_636_127, 202410_637_637_0, 202410_638_638_0, 202410_639_639_0, 202410_640_640_0] 2024.10.22 01:52:33.672574 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(07d21f4354cd951a076d1fe76dea51ac) : [0, 27357]: Setting download as failed: File segment: [0, 27357], key: 07d21f4354cd951a076d1fe76dea51ac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:33.674975 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(07d21f4354cd951a076d1fe76dea51ac): No space left in cache to reserve 27358 bytes, will continue without cache download 2024.10.22 01:52:33.797627 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 07d21f4354cd951a076d1fe76dea51ac, offset: 0, size: 0 2024.10.22 01:52:33.800393 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 07d21f4354cd951a076d1fe76dea51ac for removal 2024.10.22 01:52:33.803035 [ 1476 ] {} CacheMetadata: Key 07d21f4354cd951a076d1fe76dea51ac is removed from metadata 2024.10.22 01:52:34.774777 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(197b8031d4cc8fd87597911fa521a5bb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 197b8031d4cc8fd87597911fa521a5bb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 01:52:34.780371 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 197b8031d4cc8fd87597911fa521a5bb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 01:52:34.792297 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:52:35.476305 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 13.52 MiB, will set to 1.75 GiB (RSS), difference: 3.13 MiB 2024.10.22 01:52:35.636367 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 76002 2024.10.22 01:52:35.724317 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(a201f0cb708f13bd91bd5481715086cc) : [0, 10643044]: Setting download as failed: File segment: [0, 10643044], key: a201f0cb708f13bd91bd5481715086cc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:52:35.727643 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(a201f0cb708f13bd91bd5481715086cc): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:52:35.861238 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:35.864971 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:36.032202 [ 1218 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 01:52:36.146749 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1160_1160_0 (state Temporary) 2024.10.22 01:52:36.189422 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_135_139_1} MergeTask::MergeProjectionsStage: Merge sorted 209 rows, containing 679 columns (679 merged, 0 gathered) in 85.589754896 sec., 2.4418810435192344 rows/sec., 3.52 KiB/sec. 2024.10.22 01:52:36.198544 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1160_1160_0 to 202410_1173_1173_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:36.474757 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 76002 2024.10.22 01:52:36.555603 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1004160 2024.10.22 01:52:36.653297 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 45 entries to flush up to offset 5575 2024.10.22 01:52:37.034975 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(92157a5f115cade62db79fec7ce31b49) : [0, 18659]: Setting download as failed: File segment: [0, 18659], key: 92157a5f115cade62db79fec7ce31b49, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:37.037195 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(92157a5f115cade62db79fec7ce31b49): No space left in cache to reserve 18660 bytes, will continue without cache download 2024.10.22 01:52:37.057673 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a201f0cb708f13bd91bd5481715086cc, offset: 0, size: 0 2024.10.22 01:52:37.058979 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a201f0cb708f13bd91bd5481715086cc for removal 2024.10.22 01:52:37.067208 [ 1476 ] {} CacheMetadata: Key a201f0cb708f13bd91bd5481715086cc is removed from metadata 2024.10.22 01:52:37.114335 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3278c808920ea1492a0b63299dbd5429, offset: 0, size: 0 2024.10.22 01:52:37.118932 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3278c808920ea1492a0b63299dbd5429 for removal 2024.10.22 01:52:37.119866 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:37.127123 [ 1476 ] {} CacheMetadata: Key 3278c808920ea1492a0b63299dbd5429 is removed from metadata 2024.10.22 01:52:37.127428 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:37.158835 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 92157a5f115cade62db79fec7ce31b49, offset: 0, size: 0 2024.10.22 01:52:37.161519 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 92157a5f115cade62db79fec7ce31b49 for removal 2024.10.22 01:52:37.165577 [ 1476 ] {} CacheMetadata: Key 92157a5f115cade62db79fec7ce31b49 is removed from metadata 2024.10.22 01:52:37.176386 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b50106425d8a6d97a2648c14e592d227, offset: 0, size: 0 2024.10.22 01:52:37.178990 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b50106425d8a6d97a2648c14e592d227 for removal 2024.10.22 01:52:37.181670 [ 1476 ] {} CacheMetadata: Key b50106425d8a6d97a2648c14e592d227 is removed from metadata 2024.10.22 01:52:37.183620 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ioe/zwnvicuysczqxsntvhsdqelgnoloi. 2024.10.22 01:52:37.231463 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_658_127} MergeTask::MergeProjectionsStage: Merge sorted 1009280 rows, containing 4 columns (4 merged, 0 gathered) in 32.593590011 sec., 30965.597826424713 rows/sec., 464.20 KiB/sec. 2024.10.22 01:52:37.378966 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_659_659_0 (state Temporary) 2024.10.22 01:52:37.405368 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_658_127} MergedBlockOutputStream: filled checksums 202410_1_658_127 (state Temporary) 2024.10.22 01:52:37.424929 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0de67dc98442ed28a259cfef237868e9, offset: 0, size: 0 2024.10.22 01:52:37.428794 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0de67dc98442ed28a259cfef237868e9 for removal 2024.10.22 01:52:37.433651 [ 1476 ] {} CacheMetadata: Key 0de67dc98442ed28a259cfef237868e9 is removed from metadata 2024.10.22 01:52:37.466588 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d5388608342c7727dd2fcdfd61d1d64b, offset: 0, size: 0 2024.10.22 01:52:37.469016 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d5388608342c7727dd2fcdfd61d1d64b for removal 2024.10.22 01:52:37.473427 [ 1154 ] {} DatabaseCatalog: Execution took 261 ms. 2024.10.22 01:52:37.478073 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ioe/zwnvicuysczqxsntvhsdqelgnoloi. 2024.10.22 01:52:37.543051 [ 1476 ] {} CacheMetadata: Key d5388608342c7727dd2fcdfd61d1d64b is removed from metadata 2024.10.22 01:52:37.565310 [ 1203 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 01:52:37.584860 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e2bee378f08b3d84fcfc4ff8553f4cd7, offset: 0, size: 0 2024.10.22 01:52:37.591368 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e2bee378f08b3d84fcfc4ff8553f4cd7 for removal 2024.10.22 01:52:37.599203 [ 1476 ] {} CacheMetadata: Key e2bee378f08b3d84fcfc4ff8553f4cd7 is removed from metadata 2024.10.22 01:52:37.604175 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/bsg/efmeumeqireazpzpywycfzlcqhoit. 2024.10.22 01:52:37.663062 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_135_139_1} MergedBlockOutputStream: filled checksums 202410_135_139_1 (state Temporary) 2024.10.22 01:52:37.692341 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_659_659_0 to 202410_666_666_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:37.750486 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1004160 2024.10.22 01:52:37.861036 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 54ee2164aafaa71b23e28eb15017738c, offset: 0, size: 0 2024.10.22 01:52:37.863618 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 54ee2164aafaa71b23e28eb15017738c for removal 2024.10.22 01:52:37.866444 [ 1476 ] {} CacheMetadata: Key 54ee2164aafaa71b23e28eb15017738c is removed from metadata 2024.10.22 01:52:37.914650 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 42b27cc33d4887e6f16ad5016136d19b, offset: 0, size: 0 2024.10.22 01:52:37.916612 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 42b27cc33d4887e6f16ad5016136d19b for removal 2024.10.22 01:52:37.919010 [ 1476 ] {} CacheMetadata: Key 42b27cc33d4887e6f16ad5016136d19b is removed from metadata 2024.10.22 01:52:38.020623 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10465 2024.10.22 01:52:38.031434 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/bsg/efmeumeqireazpzpywycfzlcqhoit. 2024.10.22 01:52:38.137369 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/bgm/zigmiypixbeeomiocjcwyzlamfcns. 2024.10.22 01:52:38.275440 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 92e4200dd6b8f06508a432f2bf4a73b0, offset: 0, size: 0 2024.10.22 01:52:38.280131 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 92e4200dd6b8f06508a432f2bf4a73b0 for removal 2024.10.22 01:52:38.293375 [ 1476 ] {} CacheMetadata: Key 92e4200dd6b8f06508a432f2bf4a73b0 is removed from metadata 2024.10.22 01:52:38.353807 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b60c9d604ba2e6cad8532d4ed7bb2972, offset: 0, size: 0 2024.10.22 01:52:38.355415 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b60c9d604ba2e6cad8532d4ed7bb2972 for removal 2024.10.22 01:52:38.358435 [ 1476 ] {} CacheMetadata: Key b60c9d604ba2e6cad8532d4ed7bb2972 is removed from metadata 2024.10.22 01:52:38.364197 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/bgm/zigmiypixbeeomiocjcwyzlamfcns. 2024.10.22 01:52:38.367684 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:38.368636 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 12.05 MiB, will set to 1.74 GiB (RSS), difference: 10.48 MiB 2024.10.22 01:52:38.371243 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:38.507834 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(058c662622be13fc1d4f7b2988aa3ca8) : [0, 18320]: Setting download as failed: File segment: [0, 18320], key: 058c662622be13fc1d4f7b2988aa3ca8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:38.511420 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(058c662622be13fc1d4f7b2988aa3ca8): No space left in cache to reserve 18321 bytes, will continue without cache download 2024.10.22 01:52:38.548349 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 058c662622be13fc1d4f7b2988aa3ca8, offset: 0, size: 0 2024.10.22 01:52:38.550689 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 058c662622be13fc1d4f7b2988aa3ca8 for removal 2024.10.22 01:52:38.553404 [ 1476 ] {} CacheMetadata: Key 058c662622be13fc1d4f7b2988aa3ca8 is removed from metadata 2024.10.22 01:52:38.640905 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_688_688_0 (state Temporary) 2024.10.22 01:52:38.897316 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_688_688_0 to 202410_701_701_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:38.963374 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10465 2024.10.22 01:52:39.876473 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 576 bytes for b60c9d604ba2e6cad8532d4ed7bb2972:0) 2024.10.22 01:52:39.879427 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 20bc5710a5de9047a75403bacef81856, offset: 0, size: 155 2024.10.22 01:52:39.887090 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 20bc5710a5de9047a75403bacef81856 for removal 2024.10.22 01:52:39.893426 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9b0e7424f98ba6bfb98db1d6390ef26c, offset: 0, size: 144 2024.10.22 01:52:39.895037 [ 1476 ] {} CacheMetadata: Key 20bc5710a5de9047a75403bacef81856 is removed from metadata 2024.10.22 01:52:39.900979 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9b0e7424f98ba6bfb98db1d6390ef26c for removal 2024.10.22 01:52:39.905758 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 636478c81f03e131df4a42f58ee60777, offset: 0, size: 587 2024.10.22 01:52:39.911779 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 636478c81f03e131df4a42f58ee60777 for removal 2024.10.22 01:52:39.919148 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7f8712db1adb95141bfac72bcc799e6e, offset: 0, size: 174 2024.10.22 01:52:39.935393 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7f8712db1adb95141bfac72bcc799e6e for removal 2024.10.22 01:52:40.042402 [ 1476 ] {} CacheMetadata: Key 7f8712db1adb95141bfac72bcc799e6e is removed from metadata 2024.10.22 01:52:40.052514 [ 1476 ] {} CacheMetadata: Key 636478c81f03e131df4a42f58ee60777 is removed from metadata 2024.10.22 01:52:40.062308 [ 1476 ] {} CacheMetadata: Key 9b0e7424f98ba6bfb98db1d6390ef26c is removed from metadata 2024.10.22 01:52:40.542978 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 70 entries to flush up to offset 76728 2024.10.22 01:52:40.595019 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 76090 2024.10.22 01:52:40.853101 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:52:40.855115 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:40.859519 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:40.923238 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_658_127/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:52:40.937813 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1161_1161_0 (state Temporary) 2024.10.22 01:52:40.956262 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:52:40.956739 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:40.960510 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:41.019675 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_676_676_0 (state Temporary) 2024.10.22 01:52:41.091177 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_676_676_0 to 202410_682_682_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:41.106562 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1161_1161_0 to 202410_1174_1174_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:41.150266 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 76728 2024.10.22 01:52:41.190922 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 76090 2024.10.22 01:52:41.322284 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_658_127} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_658_127 to 202410_1_658_127 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:41.338220 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_658_127} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_652_126, 202410_653_658_1] -> 202410_1_658_127 2024.10.22 01:52:41.365578 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:52:41.432051 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileSegment(92e4200dd6b8f06508a432f2bf4a73b0) : [0, 225736]: Setting download as failed: File segment: [0, 225736], key: 92e4200dd6b8f06508a432f2bf4a73b0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 01:52:41.434208 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CachedOnDiskReadBufferFromFile(92e4200dd6b8f06508a432f2bf4a73b0): No space left in cache to reserve 225737 bytes, will continue without cache download 2024.10.22 01:52:41.962648 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 92e4200dd6b8f06508a432f2bf4a73b0, offset: 0, size: 0 2024.10.22 01:52:41.965376 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 92e4200dd6b8f06508a432f2bf4a73b0 for removal 2024.10.22 01:52:41.969678 [ 1476 ] {} CacheMetadata: Key 92e4200dd6b8f06508a432f2bf4a73b0 is removed from metadata 2024.10.22 01:52:41.970377 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ymh/xhtqlwnawfmikgivdyayaryghzbof. 2024.10.22 01:52:42.088839 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_135_139_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:52:42.220011 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ymh/xhtqlwnawfmikgivdyayaryghzbof. 2024.10.22 01:52:42.383110 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6f39743885fba03f9dad2e6bbf9600e4) : [0, 21303]: Setting download as failed: File segment: [0, 21303], key: 6f39743885fba03f9dad2e6bbf9600e4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:52:42.385375 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6f39743885fba03f9dad2e6bbf9600e4): No space left in cache to reserve 21304 bytes, will continue without cache download 2024.10.22 01:52:42.415921 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6f39743885fba03f9dad2e6bbf9600e4, offset: 0, size: 0 2024.10.22 01:52:42.417981 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6f39743885fba03f9dad2e6bbf9600e4 for removal 2024.10.22 01:52:42.525449 [ 1476 ] {} CacheMetadata: Key 6f39743885fba03f9dad2e6bbf9600e4 is removed from metadata 2024.10.22 01:52:42.567304 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.063657434 sec. 2024.10.22 01:52:43.070472 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1065_1070_1, 202410_1071_1071_0, 202410_1072_1072_0, 202410_1073_1073_0, 202410_1074_1074_0, 202410_1075_1075_0] 2024.10.22 01:52:43.073013 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1065_1070_1, 202410_1071_1071_0, 202410_1072_1072_0, 202410_1073_1073_0, 202410_1074_1074_0, 202410_1075_1075_0] 2024.10.22 01:52:43.111911 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1065_1070_1, 202410_1071_1071_0, 202410_1072_1072_0, 202410_1073_1073_0, 202410_1074_1074_0, 202410_1075_1075_0] 2024.10.22 01:52:43.262700 [ 1179 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 01:52:43.855288 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1026 bytes for 619d0cc99443727a826097a145c78339:0) 2024.10.22 01:52:43.857015 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: e3ddb9a73e55005e081fac5a8d8e1b6f, offset: 0, size: 1011 2024.10.22 01:52:43.860591 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key e3ddb9a73e55005e081fac5a8d8e1b6f for removal 2024.10.22 01:52:43.863580 [ 1476 ] {} CacheMetadata: Key e3ddb9a73e55005e081fac5a8d8e1b6f is removed from metadata 2024.10.22 01:52:43.864335 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2ba1cd3582cd1a413b67cec9ab0996e8, offset: 0, size: 155 2024.10.22 01:52:43.872165 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 2ba1cd3582cd1a413b67cec9ab0996e8 for removal 2024.10.22 01:52:43.892322 [ 1476 ] {} CacheMetadata: Key 2ba1cd3582cd1a413b67cec9ab0996e8 is removed from metadata 2024.10.22 01:52:44.075292 [ 1178 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:52:44.118735 [ 1178 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:52:44.298641 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gym/mcrlshuecevpqhtkjjwtnvixvctpw. 2024.10.22 01:52:44.457819 [ 1676 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gsb/lecjacxmehmhwqzwavcbwiputdzme, size 10648784 2024.10.22 01:52:44.504644 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 9.21 MiB, will set to 1.74 GiB (RSS), difference: 24.71 MiB 2024.10.22 01:52:44.540649 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gym/mcrlshuecevpqhtkjjwtnvixvctpw. 2024.10.22 01:52:44.573266 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:52:44.583193 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_513_105, 202410_514_514_0, 202410_515_515_0, 202410_516_516_0, 202410_517_517_0, 202410_518_518_0] 2024.10.22 01:52:44.584200 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:52:44.589270 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_513_105, 202410_514_514_0, 202410_515_515_0, 202410_516_516_0, 202410_517_517_0, 202410_518_518_0] 2024.10.22 01:52:44.709102 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:52:44.730343 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_513_105, 202410_514_514_0, 202410_515_515_0, 202410_516_516_0, 202410_517_517_0, 202410_518_518_0] 2024.10.22 01:52:44.738713 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:52:44.755370 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1005315 2024.10.22 01:52:44.759991 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:52:44.800541 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:52:44.942119 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ClientIP 2024.10.22 01:52:45.032393 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ljt/bxbrcyjmsjmzhkayntdhhhhvjlycp (LogSeriesLimiter: on interval from 2024-10-22 01:52:28 to 2024-10-22 01:52:45 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:52:45.308483 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 76151 2024.10.22 01:52:45.325497 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 11 file segments (while reserving 27464 bytes for 32bc3a0df004580a4b14004734b6122f:0) 2024.10.22 01:52:45.332258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b64a559ec845c123b4426a5e08a5bc55, offset: 0, size: 1136283 2024.10.22 01:52:45.343876 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b64a559ec845c123b4426a5e08a5bc55 for removal 2024.10.22 01:52:45.347536 [ 1476 ] {} CacheMetadata: Key b64a559ec845c123b4426a5e08a5bc55 is removed from metadata 2024.10.22 01:52:45.347401 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 382e0de37bc9f337f0370a5a81ef2917, offset: 0, size: 610 2024.10.22 01:52:45.353366 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 382e0de37bc9f337f0370a5a81ef2917 for removal 2024.10.22 01:52:45.356301 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1e3ca9b21239c906dfa3b534f0fa38bc, offset: 0, size: 164 2024.10.22 01:52:45.362846 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1e3ca9b21239c906dfa3b534f0fa38bc for removal 2024.10.22 01:52:45.368409 [ 1476 ] {} CacheMetadata: Key 1e3ca9b21239c906dfa3b534f0fa38bc is removed from metadata 2024.10.22 01:52:45.369269 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d712685e8be2d131a8d545978d980133, offset: 0, size: 154 2024.10.22 01:52:45.372725 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d712685e8be2d131a8d545978d980133 for removal 2024.10.22 01:52:45.377891 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1cac2e7e3e1c1630358e85e1d7c46835, offset: 0, size: 154 2024.10.22 01:52:45.381726 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1cac2e7e3e1c1630358e85e1d7c46835 for removal 2024.10.22 01:52:45.386839 [ 1476 ] {} CacheMetadata: Key d712685e8be2d131a8d545978d980133 is removed from metadata 2024.10.22 01:52:45.390726 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b60c9d604ba2e6cad8532d4ed7bb2972, offset: 0, size: 576 2024.10.22 01:52:45.403143 [ 1476 ] {} CacheMetadata: Key 1cac2e7e3e1c1630358e85e1d7c46835 is removed from metadata 2024.10.22 01:52:45.420859 [ 1476 ] {} CacheMetadata: Key 382e0de37bc9f337f0370a5a81ef2917 is removed from metadata 2024.10.22 01:52:45.497718 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b60c9d604ba2e6cad8532d4ed7bb2972 for removal 2024.10.22 01:52:45.503367 [ 1476 ] {} CacheMetadata: Key b60c9d604ba2e6cad8532d4ed7bb2972 is removed from metadata 2024.10.22 01:52:45.503720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a8192f922b61cd8c55b1e1aa89f34c4a, offset: 0, size: 1164 2024.10.22 01:52:45.508358 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a8192f922b61cd8c55b1e1aa89f34c4a for removal 2024.10.22 01:52:45.513615 [ 1476 ] {} CacheMetadata: Key a8192f922b61cd8c55b1e1aa89f34c4a is removed from metadata 2024.10.22 01:52:45.514327 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c9ba5cd484762847e470e7337e9f8fe6, offset: 0, size: 1268 2024.10.22 01:52:45.518541 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c9ba5cd484762847e470e7337e9f8fe6 for removal 2024.10.22 01:52:45.529311 [ 1476 ] {} CacheMetadata: Key c9ba5cd484762847e470e7337e9f8fe6 is removed from metadata 2024.10.22 01:52:45.530876 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e96ecab1d6c0d24cfc64b7b6f1b50894, offset: 0, size: 154 2024.10.22 01:52:45.534826 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e96ecab1d6c0d24cfc64b7b6f1b50894 for removal 2024.10.22 01:52:45.544484 [ 1476 ] {} CacheMetadata: Key e96ecab1d6c0d24cfc64b7b6f1b50894 is removed from metadata 2024.10.22 01:52:45.543901 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bf920ae5f1e629f971678bb567e357ea, offset: 0, size: 153 2024.10.22 01:52:45.552501 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bf920ae5f1e629f971678bb567e357ea for removal 2024.10.22 01:52:45.558409 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:45.560768 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:45.584543 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fa13c3d710569fdec293ad99b0b5b628, offset: 0, size: 154 2024.10.22 01:52:45.593030 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fa13c3d710569fdec293ad99b0b5b628 for removal 2024.10.22 01:52:45.631237 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1162_1162_0 (state Temporary) 2024.10.22 01:52:45.661741 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:45.668098 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:45.684164 [ 1476 ] {} CacheMetadata: Key bf920ae5f1e629f971678bb567e357ea is removed from metadata 2024.10.22 01:52:45.696845 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1162_1162_0 to 202410_1175_1175_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:45.700520 [ 1476 ] {} CacheMetadata: Key fa13c3d710569fdec293ad99b0b5b628 is removed from metadata 2024.10.22 01:52:45.759732 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 76151 2024.10.22 01:52:45.958534 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qxp/zcuyismhxdwwmnvaehxtampbiubzn. 2024.10.22 01:52:46.123119 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_660_660_0 (state Temporary) 2024.10.22 01:52:46.184627 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qxp/zcuyismhxdwwmnvaehxtampbiubzn. 2024.10.22 01:52:46.188391 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qaa/wrpsnnrqwkmayizzhhvgstkkaxnkj. 2024.10.22 01:52:46.307025 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_660_660_0 to 202410_667_667_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:46.340906 [ 1187 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 01:52:46.386790 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qaa/wrpsnnrqwkmayizzhhvgstkkaxnkj. 2024.10.22 01:52:46.444874 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_605_605_0, 202410_606_606_0, 202410_607_607_0, 202410_608_608_0, 202410_609_609_0, 202410_610_610_0] 2024.10.22 01:52:46.447785 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_605_605_0, 202410_606_606_0, 202410_607_607_0, 202410_608_608_0, 202410_609_609_0, 202410_610_610_0] 2024.10.22 01:52:46.452278 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1005315 2024.10.22 01:52:46.465465 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 10473 2024.10.22 01:52:46.469592 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_605_605_0, 202410_606_606_0, 202410_607_607_0, 202410_608_608_0, 202410_609_609_0, 202410_610_610_0] 2024.10.22 01:52:46.666506 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:46.668191 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:46.619678 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_135_139_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_135_139_1 to 202410_135_139_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:46.751404 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_135_139_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 5 parts: [202410_135_135_0, 202410_139_139_0] -> 202410_135_139_1 2024.10.22 01:52:46.907216 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_689_689_0 (state Temporary) 2024.10.22 01:52:47.062267 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_689_689_0 to 202410_702_702_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:47.110178 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10473 2024.10.22 01:52:47.346145 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.75 MiB. 2024.10.22 01:52:47.449319 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 6.10 MiB, will set to 1.72 GiB (RSS), difference: -10.07 MiB 2024.10.22 01:52:48.005785 [ 1185 ] {} DatabaseCatalog: Execution took 528 ms. 2024.10.22 01:52:48.084189 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 575 2024.10.22 01:52:48.365725 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:48.370432 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:48.498801 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:52:48.512361 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:52:48.541631 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_557_557_0 (state Temporary) 2024.10.22 01:52:48.544901 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:52:48.556446 [ 1695 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xpa/hebccyzvbkhjltamwqbgltswqaohs, size 5035280 2024.10.22 01:52:48.582655 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:52:48.625548 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/djf/yjhtsvwkjelmwclbxhsbjhbwrzglj. 2024.10.22 01:52:48.645503 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:52:48.648654 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_557_557_0 to 202410_567_567_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:48.659059 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 65 entries to flush up to offset 76793 2024.10.22 01:52:48.680312 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:52:48.744806 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 575 2024.10.22 01:52:48.845339 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:52:48.895093 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:48.901676 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:48.903083 [ 1234 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 01:52:48.959878 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/djf/yjhtsvwkjelmwclbxhsbjhbwrzglj. 2024.10.22 01:52:48.964930 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_677_677_0 (state Temporary) 2024.10.22 01:52:48.976908 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column FetchTiming 2024.10.22 01:52:49.009515 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_677_677_0 to 202410_683_683_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:49.075998 [ 1207 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_678_36 to 202410_679_679_0 2024.10.22 01:52:49.098303 [ 1207 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.02 MiB using storage policy from min volume index 0 2024.10.22 01:52:49.103295 [ 1207 ] {} DiskLocal: Reserved 1.02 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:49.158511 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 76793 2024.10.22 01:52:49.262684 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_679_37} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_678_36 to 202410_679_679_0 into Compact with storage Full 2024.10.22 01:52:49.364843 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_679_37} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:49.372461 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_679_37} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_678_36, total 17218 rows starting from the beginning of the part 2024.10.22 01:52:49.405656 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_679_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_679_679_0, total 126 rows starting from the beginning of the part 2024.10.22 01:52:49.449064 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/loo/mqhulmrkodkpoqdwopopewriwlxpd. 2024.10.22 01:52:49.763598 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 76247 2024.10.22 01:52:49.767650 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 4.74 MiB, will set to 1.72 GiB (RSS), difference: 2.94 MiB 2024.10.22 01:52:49.839618 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/loo/mqhulmrkodkpoqdwopopewriwlxpd. 2024.10.22 01:52:50.040583 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:50.046621 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:50.146218 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 01:52:50.231301 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1163_1163_0 (state Temporary) 2024.10.22 01:52:50.372011 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1163_1163_0 to 202410_1176_1176_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:50.447755 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 76247 2024.10.22 01:52:50.531249 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1171_15 to 202410_1176_1176_0 2024.10.22 01:52:50.547859 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:50.551411 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:50.584185 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1171_15 to 202410_1176_1176_0 into Compact with storage Full 2024.10.22 01:52:50.686076 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:50.706356 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1171_15, total 5308 rows starting from the beginning of the part 2024.10.22 01:52:50.768402 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1172_1172_0, total 58 rows starting from the beginning of the part 2024.10.22 01:52:50.804947 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1173_1173_0, total 60 rows starting from the beginning of the part 2024.10.22 01:52:50.855757 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:52:50.856991 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1174_1174_0, total 88 rows starting from the beginning of the part 2024.10.22 01:52:50.857815 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:52:50.986263 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1175_1175_0, total 61 rows starting from the beginning of the part 2024.10.22 01:52:51.079122 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1176_1176_0, total 96 rows starting from the beginning of the part 2024.10.22 01:52:51.376347 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mvy/rwideehywmuyubpkarbzcxnofbsie. 2024.10.22 01:52:51.530766 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 1048576 bytes for 6b6f7c3a5cef32f95b4be00b91f2b9ce:0) 2024.10.22 01:52:51.554938 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 72fa03b33383026f7ac69d496a6b1161, offset: 0, size: 692289 2024.10.22 01:52:51.561220 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 72fa03b33383026f7ac69d496a6b1161 for removal 2024.10.22 01:52:51.567217 [ 1476 ] {} CacheMetadata: Key 72fa03b33383026f7ac69d496a6b1161 is removed from metadata 2024.10.22 01:52:51.630751 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mvy/rwideehywmuyubpkarbzcxnofbsie. 2024.10.22 01:52:51.647393 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 3.82 MiB, will set to 1.72 GiB (RSS), difference: 3.47 MiB 2024.10.22 01:52:52.122456 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_679_37} MergeTask::MergeProjectionsStage: Merge sorted 17344 rows, containing 13 columns (13 merged, 0 gathered) in 2.966318589 sec., 5846.9781581509005 rows/sec., 2.53 MiB/sec. 2024.10.22 01:52:52.171112 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_679_37} MergedBlockOutputStream: filled checksums 202410_507_679_37 (state Temporary) 2024.10.22 01:52:52.372143 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_679_37} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_679_37 to 202410_507_679_37 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:52.387010 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_679_37} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_678_36, 202410_679_679_0] -> 202410_507_679_37 2024.10.22 01:52:52.515776 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.80 MiB. 2024.10.22 01:52:52.934150 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 13 file segments (while reserving 432915 bytes for 2cd8ef4f95687a07afb97a135913f4d2:0) 2024.10.22 01:52:52.937640 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e034d8189fdaaf64ec6ac6172158daef, offset: 0, size: 12051897 2024.10.22 01:52:52.957542 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e034d8189fdaaf64ec6ac6172158daef for removal 2024.10.22 01:52:52.961732 [ 1476 ] {} CacheMetadata: Key e034d8189fdaaf64ec6ac6172158daef is removed from metadata 2024.10.22 01:52:52.972456 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d74ffd24b15e5523cffc5f451235fa19, offset: 0, size: 1168 2024.10.22 01:52:52.975898 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d74ffd24b15e5523cffc5f451235fa19 for removal 2024.10.22 01:52:52.978254 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0fb0cf25a35bb80ac71dffaac502292a, offset: 0, size: 142 2024.10.22 01:52:52.979742 [ 1476 ] {} CacheMetadata: Key d74ffd24b15e5523cffc5f451235fa19 is removed from metadata 2024.10.22 01:52:53.086801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0fb0cf25a35bb80ac71dffaac502292a for removal 2024.10.22 01:52:53.093501 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8d7b6044b7b70573d9ba5eb39204d426, offset: 0, size: 142 2024.10.22 01:52:53.093410 [ 1476 ] {} CacheMetadata: Key 0fb0cf25a35bb80ac71dffaac502292a is removed from metadata 2024.10.22 01:52:53.106704 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8d7b6044b7b70573d9ba5eb39204d426 for removal 2024.10.22 01:52:53.116928 [ 1476 ] {} CacheMetadata: Key 8d7b6044b7b70573d9ba5eb39204d426 is removed from metadata 2024.10.22 01:52:53.199288 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} MergeTask::MergeProjectionsStage: Merge sorted 5671 rows, containing 13 columns (13 merged, 0 gathered) in 2.629634051 sec., 2156.573838798378 rows/sec., 375.05 KiB/sec. 2024.10.22 01:52:53.212386 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e00d7689f6df74777989e07f0e4eae77, offset: 0, size: 45420 2024.10.22 01:52:53.218125 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e00d7689f6df74777989e07f0e4eae77 for removal 2024.10.22 01:52:53.220726 [ 1476 ] {} CacheMetadata: Key e00d7689f6df74777989e07f0e4eae77 is removed from metadata 2024.10.22 01:52:53.222250 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 35702f2a7a0639ead8b7e203872882eb, offset: 0, size: 4964 2024.10.22 01:52:53.226985 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 35702f2a7a0639ead8b7e203872882eb for removal 2024.10.22 01:52:53.231744 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 84e130dad461e463ca777adec3e93089, offset: 0, size: 4964 2024.10.22 01:52:53.235008 [ 1476 ] {} CacheMetadata: Key 35702f2a7a0639ead8b7e203872882eb is removed from metadata 2024.10.22 01:52:53.236874 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 84e130dad461e463ca777adec3e93089 for removal 2024.10.22 01:52:53.244150 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2ec1e1e3d857039c3dd66e87205169ea, offset: 0, size: 5015 2024.10.22 01:52:53.248604 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2ec1e1e3d857039c3dd66e87205169ea for removal 2024.10.22 01:52:53.251573 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 21ab79686dcf3f1a25d30e1e1ec57af5, offset: 0, size: 154 2024.10.22 01:52:53.251692 [ 1476 ] {} CacheMetadata: Key 84e130dad461e463ca777adec3e93089 is removed from metadata 2024.10.22 01:52:53.254556 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 21ab79686dcf3f1a25d30e1e1ec57af5 for removal 2024.10.22 01:52:53.259308 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4e4cce8d83f78f839c2d8c848ac78b43, offset: 0, size: 565 2024.10.22 01:52:53.262398 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4e4cce8d83f78f839c2d8c848ac78b43 for removal 2024.10.22 01:52:53.277361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ce4753f3094710b7a97d67a3df67f8f9, offset: 0, size: 142 2024.10.22 01:52:53.277712 [ 1476 ] {} CacheMetadata: Key 4e4cce8d83f78f839c2d8c848ac78b43 is removed from metadata 2024.10.22 01:52:53.283721 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ce4753f3094710b7a97d67a3df67f8f9 for removal 2024.10.22 01:52:53.290493 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c4c023360911dbb1fab7b8363fc77dc5, offset: 0, size: 582 2024.10.22 01:52:53.295734 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c4c023360911dbb1fab7b8363fc77dc5 for removal 2024.10.22 01:52:53.299301 [ 1476 ] {} CacheMetadata: Key ce4753f3094710b7a97d67a3df67f8f9 is removed from metadata 2024.10.22 01:52:53.300408 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 32bc3a0df004580a4b14004734b6122f, offset: 0, size: 27464 2024.10.22 01:52:53.304904 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 32bc3a0df004580a4b14004734b6122f for removal 2024.10.22 01:52:53.371011 [ 1476 ] {} CacheMetadata: Key 32bc3a0df004580a4b14004734b6122f is removed from metadata 2024.10.22 01:52:53.394122 [ 1476 ] {} CacheMetadata: Key c4c023360911dbb1fab7b8363fc77dc5 is removed from metadata 2024.10.22 01:52:53.416185 [ 1476 ] {} CacheMetadata: Key 21ab79686dcf3f1a25d30e1e1ec57af5 is removed from metadata 2024.10.22 01:52:53.437375 [ 1476 ] {} CacheMetadata: Key 2ec1e1e3d857039c3dd66e87205169ea is removed from metadata 2024.10.22 01:52:53.478166 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 1006857 2024.10.22 01:52:53.507206 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} MergedBlockOutputStream: filled checksums 202410_1096_1176_16 (state Temporary) 2024.10.22 01:52:53.689629 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1176_16 to 202410_1096_1176_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:53.696752 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1176_16} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1171_15, 202410_1176_1176_0] -> 202410_1096_1176_16 2024.10.22 01:52:53.746449 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.67 MiB. 2024.10.22 01:52:54.151118 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:54.155459 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:54.456455 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 76329 2024.10.22 01:52:54.480250 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:54.484976 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:54.516040 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nvl/bfggxsvoyhnbenrqkbathapkzjvlb. 2024.10.22 01:52:54.527023 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_661_661_0 (state Temporary) 2024.10.22 01:52:54.611894 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 10493 2024.10.22 01:52:54.776438 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_661_661_0 to 202410_668_668_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:54.819430 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1006857 2024.10.22 01:52:54.820975 [ 1250 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_658_127 to 202410_659_664_1 2024.10.22 01:52:54.847088 [ 1250 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:54.854446 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.90 GiB. 2024.10.22 01:52:54.856608 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zxr/evqzewizfwcalaqghahqqbsylfmqk. 2024.10.22 01:52:54.859634 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:54.869159 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:54.880754 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:54.880930 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_664_128} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_658_127 to 202410_659_664_1 into Wide with storage Full 2024.10.22 01:52:54.892645 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:54.928742 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_664_128} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:54.946952 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_664_128} MergeTreeSequentialSource: Reading 125 marks from part 202410_1_658_127, total 1009280 rows starting from the beginning of the part 2024.10.22 01:52:54.954923 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yci/zliaujazuovlbfwlrksqbqsfbgssw. 2024.10.22 01:52:54.972384 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_664_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_659_664_1, total 6389 rows starting from the beginning of the part 2024.10.22 01:52:55.049712 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_690_690_0 (state Temporary) 2024.10.22 01:52:55.069703 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1164_1164_0 (state Temporary) 2024.10.22 01:52:55.112131 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_690_690_0 to 202410_703_703_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:55.163874 [ 1268 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_698_141 to 202410_703_703_0 2024.10.22 01:52:55.178383 [ 1268 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:55.197660 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:55.233457 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_698_141 to 202410_703_703_0 into Compact with storage Full 2024.10.22 01:52:55.250400 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1164_1164_0 to 202410_1177_1177_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:55.275450 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yci/zliaujazuovlbfwlrksqbqsfbgssw. 2024.10.22 01:52:55.394331 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:55.446967 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10493 2024.10.22 01:52:55.450863 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 76329 2024.10.22 01:52:55.506124 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_698_141, total 10533 rows starting from the beginning of the part 2024.10.22 01:52:55.558193 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_699_699_0, total 6 rows starting from the beginning of the part 2024.10.22 01:52:55.601969 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_700_700_0, total 19 rows starting from the beginning of the part 2024.10.22 01:52:55.655008 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_701_701_0, total 13 rows starting from the beginning of the part 2024.10.22 01:52:55.684337 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_702_702_0, total 8 rows starting from the beginning of the part 2024.10.22 01:52:55.738337 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_703_703_0, total 20 rows starting from the beginning of the part 2024.10.22 01:52:55.787625 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 3.71 MiB, will set to 1.71 GiB (RSS), difference: 3.24 MiB 2024.10.22 01:52:55.812120 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zxr/evqzewizfwcalaqghahqqbsylfmqk. 2024.10.22 01:52:55.821576 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_8_8_0 to 201403_8_8_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:55.856978 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nvl/bfggxsvoyhnbenrqkbathapkzjvlb. 2024.10.22 01:52:55.878104 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:52:56.099235 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_598_117, 202410_599_604_1] 2024.10.22 01:52:56.102434 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_598_117, 202410_599_604_1] 2024.10.22 01:52:56.128852 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_598_117, 202410_599_604_1] 2024.10.22 01:52:56.186244 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_703_142/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:52:56.246663 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 576 2024.10.22 01:52:56.418738 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_139_139_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:52:56.568157 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:56.591188 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:56.678721 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 128 entries to flush up to offset 76921 2024.10.22 01:52:56.853165 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:56.857817 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:56.896139 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_558_558_0 (state Temporary) 2024.10.22 01:52:56.958794 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_678_678_0 (state Temporary) 2024.10.22 01:52:56.979214 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_664_128/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:52:56.989820 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_558_558_0 to 202410_568_568_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:57.031810 [ 1261 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_563_115 to 202410_568_568_0 2024.10.22 01:52:57.045676 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_678_678_0 to 202410_684_684_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:57.051283 [ 1261 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:52:57.075663 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:57.182274 [ 1262 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_679_37 to 202410_684_684_0 2024.10.22 01:52:57.183746 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 576 2024.10.22 01:52:57.201580 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 76921 2024.10.22 01:52:57.208802 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_563_115 to 202410_568_568_0 into Compact with storage Full 2024.10.22 01:52:57.212875 [ 1262 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 01:52:57.221833 [ 1262 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:52:57.133909 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} MergeTask::MergeProjectionsStage: Merge sorted 10599 rows, containing 12 columns (12 merged, 0 gathered) in 1.910308322 sec., 5548.319021561589 rows/sec., 883.40 KiB/sec. 2024.10.22 01:52:57.250454 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tzw/kqsxcgcwjjrwjxnerchephocnllgv. 2024.10.22 01:52:57.387077 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_679_37 to 202410_684_684_0 into Compact with storage Full 2024.10.22 01:52:57.385600 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tzw/kqsxcgcwjjrwjxnerchephocnllgv. 2024.10.22 01:52:57.409205 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:57.428267 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_563_115, total 607 rows starting from the beginning of the part 2024.10.22 01:52:57.471718 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:52:57.472610 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_564_564_0, total 1 rows starting from the beginning of the part 2024.10.22 01:52:57.480877 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_679_37, total 17344 rows starting from the beginning of the part 2024.10.22 01:52:57.506364 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_565_565_0, total 1 rows starting from the beginning of the part 2024.10.22 01:52:57.529468 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_680_680_0, total 100 rows starting from the beginning of the part 2024.10.22 01:52:57.537442 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} MergedBlockOutputStream: filled checksums 202410_1_703_142 (state Temporary) 2024.10.22 01:52:57.575932 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_681_681_0, total 89 rows starting from the beginning of the part 2024.10.22 01:52:57.591037 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_566_566_0, total 1 rows starting from the beginning of the part 2024.10.22 01:52:57.639059 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_567_567_0, total 1 rows starting from the beginning of the part 2024.10.22 01:52:57.672146 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_568_568_0, total 1 rows starting from the beginning of the part 2024.10.22 01:52:57.704881 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.24 GiB, peak 1.97 GiB, free memory in arenas 8.44 MiB, will set to 1.46 GiB (RSS), difference: 230.93 MiB 2024.10.22 01:52:57.741618 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_682_682_0, total 70 rows starting from the beginning of the part 2024.10.22 01:52:57.853674 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_703_142 to 202410_1_703_142 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:52:57.857039 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_703_142} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_698_141, 202410_703_703_0] -> 202410_1_703_142 2024.10.22 01:52:57.888024 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.52 MiB. 2024.10.22 01:52:57.918297 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_683_683_0, total 65 rows starting from the beginning of the part 2024.10.22 01:52:57.976170 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_684_684_0, total 128 rows starting from the beginning of the part 2024.10.22 01:52:58.215303 [ 1252 ] {} DatabaseCatalog: Execution took 206 ms. 2024.10.22 01:52:58.596254 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_139_139_0 (state Temporary) 2024.10.22 01:52:58.765617 [ 1218 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 01:52:59.128714 [ 1219 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:52:59.265402 [ 1219 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:52:59.458652 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 97 entries to flush up to offset 76426 2024.10.22 01:52:59.612434 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 1.97 GiB, free memory in arenas 11.49 MiB, will set to 1.50 GiB (RSS), difference: -65.35 MiB 2024.10.22 01:52:59.895892 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/bxa/otkxhtneavuakbhuyjkhelceulttf. 2024.10.22 01:53:00.063727 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:00.068651 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:53:00.160579 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/bxa/otkxhtneavuakbhuyjkhelceulttf. 2024.10.22 01:53:00.262567 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1165_1165_0 (state Temporary) 2024.10.22 01:53:00.302732 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/prb/kcrmxsvzxnmnsnnjvmeycmxzlswmz. 2024.10.22 01:53:00.351309 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1165_1165_0 to 202410_1178_1178_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:00.430534 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 76426 2024.10.22 01:53:00.566241 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} MergeTask::MergeProjectionsStage: Merge sorted 612 rows, containing 32 columns (32 merged, 0 gathered) in 3.366870786 sec., 181.77115752252692 rows/sec., 16.59 KiB/sec. 2024.10.22 01:53:00.641652 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} MergedBlockOutputStream: filled checksums 202410_1_568_116 (state Temporary) 2024.10.22 01:53:00.658633 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/prb/kcrmxsvzxnmnsnnjvmeycmxzlswmz. 2024.10.22 01:53:00.855262 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:53:00.858627 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:53:01.031079 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_568_116/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:53:01.323425 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} MergeTask::MergeProjectionsStage: Merge sorted 17796 rows, containing 13 columns (13 merged, 0 gathered) in 4.04897442 sec., 4395.187065666865 rows/sec., 1.90 MiB/sec. 2024.10.22 01:53:01.460002 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} MergedBlockOutputStream: filled checksums 202410_507_684_38 (state Temporary) 2024.10.22 01:53:01.588763 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/psr/ndmijurdyagcxvfkanqwxykgepnvn. 2024.10.22 01:53:01.593773 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_568_116 to 202410_1_568_116 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:01.601795 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_568_116} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_563_115, 202410_568_568_0] -> 202410_1_568_116 2024.10.22 01:53:01.703044 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.97 GiB, free memory in arenas 12.17 MiB, will set to 1.52 GiB (RSS), difference: 26.59 MiB 2024.10.22 01:53:01.722397 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/psr/ndmijurdyagcxvfkanqwxykgepnvn. 2024.10.22 01:53:01.770571 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:53:01.816422 [ 1701 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ljt/bxbrcyjmsjmzhkayntdhhhhvjlycp, size 2746789 2024.10.22 01:53:01.827942 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1008397 2024.10.22 01:53:01.942682 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_684_38 to 202410_507_684_38 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:01.947888 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_684_38} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_679_37, 202410_684_684_0] -> 202410_507_684_38 2024.10.22 01:53:01.996766 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.34 MiB. 2024.10.22 01:53:02.297073 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:53:02.315747 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:53:02.336806 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:53:02.351707 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:53:02.462173 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:53:02.476138 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 40 2024.10.22 01:53:02.486676 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:53:02.519907 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ClientIP6 2024.10.22 01:53:02.669555 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nnw/fvqzhecuclwhjjsifwrbhyjcqorgd (LogSeriesLimiter: on interval from 2024-10-22 01:52:45 to 2024-10-22 01:53:02 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:53:02.716565 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:02.728256 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:53:02.796896 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_34_34_0 (state Temporary) 2024.10.22 01:53:02.917398 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 5 old parts to remove. Parts: [202410_507_621_23, 202410_622_622_0, 202410_623_623_0, 202410_624_624_0, 202410_625_625_0] 2024.10.22 01:53:02.920624 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from filesystem (serially): Parts: [202410_507_621_23, 202410_622_622_0, 202410_623_623_0, 202410_624_624_0, 202410_625_625_0] 2024.10.22 01:53:02.949679 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 10521 2024.10.22 01:53:02.951877 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from memory: Parts: [202410_507_621_23, 202410_622_622_0, 202410_623_623_0, 202410_624_624_0, 202410_625_625_0] 2024.10.22 01:53:03.038270 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_34_34_0 to 202410_34_34_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:03.068498 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:03.077295 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:53:03.154514 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ikt/kqdmqhwiixhppikmzqlbkeaiqeuvl. 2024.10.22 01:53:03.233123 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_662_662_0 (state Temporary) 2024.10.22 01:53:03.236269 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 40 2024.10.22 01:53:03.351193 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:03.354721 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:53:03.356527 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_662_662_0 to 202410_669_669_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:03.369172 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ikt/kqdmqhwiixhppikmzqlbkeaiqeuvl. 2024.10.22 01:53:03.505822 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_691_691_0 (state Temporary) 2024.10.22 01:53:03.523743 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1008397 2024.10.22 01:53:03.611787 [ 1240 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 207 ms. 2024.10.22 01:53:03.624188 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_691_691_0 to 202410_704_704_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:03.719550 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.97 GiB, free memory in arenas 11.25 MiB, will set to 1.53 GiB (RSS), difference: 6.41 MiB 2024.10.22 01:53:03.897040 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10521 2024.10.22 01:53:04.457237 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 76487 2024.10.22 01:53:04.480053 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pkc/cylnveqnofeswefoxymhfnixbrkmc. 2024.10.22 01:53:04.691270 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 577 2024.10.22 01:53:04.739059 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 99 entries to flush up to offset 77020 2024.10.22 01:53:04.926627 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pkc/cylnveqnofeswefoxymhfnixbrkmc. 2024.10.22 01:53:05.072630 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:05.072924 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:05.078305 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:53:05.084770 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:53:05.166274 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:05.169097 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.89 GiB. 2024.10.22 01:53:05.169296 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_559_559_0 (state Temporary) 2024.10.22 01:53:05.202312 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1166_1166_0 (state Temporary) 2024.10.22 01:53:05.272336 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1166_1166_0 to 202410_1179_1179_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:05.366689 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_679_679_0 (state Temporary) 2024.10.22 01:53:05.460617 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 76487 2024.10.22 01:53:05.480969 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_679_679_0 to 202410_685_685_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:05.538624 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_559_559_0 to 202410_569_569_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:05.671330 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77020 2024.10.22 01:53:05.672583 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 577 2024.10.22 01:53:06.101796 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kqn/xtkixutlmufgqmnmxqqcphcaxkrjw. 2024.10.22 01:53:06.260535 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 76d9b3500f21e8cdb0626ffa5cbaa840, offset: 0, size: 0 2024.10.22 01:53:06.263277 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 76d9b3500f21e8cdb0626ffa5cbaa840 for removal 2024.10.22 01:53:06.268955 [ 1476 ] {} CacheMetadata: Key 76d9b3500f21e8cdb0626ffa5cbaa840 is removed from metadata 2024.10.22 01:53:06.438509 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kqn/xtkixutlmufgqmnmxqqcphcaxkrjw. 2024.10.22 01:53:06.658449 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 13.70 MiB, will set to 1.58 GiB (RSS), difference: -53.53 MiB 2024.10.22 01:53:06.835595 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:53:06.947667 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:53:06.956032 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:53:06.968743 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:53:06.978638 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:53:07.006333 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:53:07.033284 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:53:07.060544 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 01:53:07.108479 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/brk/egwwtoukviprcmhaccjrdzfdiwmwe (LogSeriesLimiter: on interval from 2024-10-22 01:53:02 to 2024-10-22 01:53:07 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:53:07.492164 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_640_128, 202410_641_641_0, 202410_642_642_0, 202410_643_643_0, 202410_644_644_0, 202410_645_645_0] 2024.10.22 01:53:07.506440 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_640_128, 202410_641_641_0, 202410_642_642_0, 202410_643_643_0, 202410_644_644_0, 202410_645_645_0] 2024.10.22 01:53:07.557454 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_640_128, 202410_641_641_0, 202410_642_642_0, 202410_643_643_0, 202410_644_644_0, 202410_645_645_0] 2024.10.22 01:53:07.840571 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 1048576 bytes for 76d9b3500f21e8cdb0626ffa5cbaa840:0) 2024.10.22 01:53:07.843443 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0cfd7a53e55b3d0e1c8aa1209ba77ae7, offset: 0, size: 4611304 2024.10.22 01:53:07.850847 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0cfd7a53e55b3d0e1c8aa1209ba77ae7 for removal 2024.10.22 01:53:07.896215 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1065_1075_2, 202410_1076_1076_0, 202410_1077_1077_0, 202410_1078_1078_0, 202410_1079_1079_0, 202410_1080_1080_0] 2024.10.22 01:53:07.902758 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1065_1075_2, 202410_1076_1076_0, 202410_1077_1077_0, 202410_1078_1078_0, 202410_1079_1079_0, 202410_1080_1080_0] 2024.10.22 01:53:07.941089 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1065_1075_2, 202410_1076_1076_0, 202410_1077_1077_0, 202410_1078_1078_0, 202410_1079_1079_0, 202410_1080_1080_0] 2024.10.22 01:53:07.853720 [ 1476 ] {} CacheMetadata: Key 0cfd7a53e55b3d0e1c8aa1209ba77ae7 is removed from metadata 2024.10.22 01:53:08.259378 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_664_128/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:53:08.468244 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/oic/hsmqmahlqedwywttleiuvajaqwous. 2024.10.22 01:53:08.534661 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d3f9f35ada37427e56d6b835a1211363, offset: 0, size: 0 2024.10.22 01:53:08.539451 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d3f9f35ada37427e56d6b835a1211363 for removal 2024.10.22 01:53:08.544992 [ 1476 ] {} CacheMetadata: Key d3f9f35ada37427e56d6b835a1211363 is removed from metadata 2024.10.22 01:53:08.594889 [ 1176 ] {} DatabaseCatalog: Execution took 375 ms. 2024.10.22 01:53:08.658899 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 10.94 MiB, will set to 1.60 GiB (RSS), difference: -28.25 MiB 2024.10.22 01:53:08.819936 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/oic/hsmqmahlqedwywttleiuvajaqwous. 2024.10.22 01:53:08.886620 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_139_139_0 to 202410_142_142_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:09.320464 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 1048576 bytes for 8430e472d0ef29ae6e5e0d47d80698e8:0) 2024.10.22 01:53:09.329027 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 820753cede6aa3e7956e92e665fe3023, offset: 0, size: 1590061 2024.10.22 01:53:09.334828 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 820753cede6aa3e7956e92e665fe3023 for removal 2024.10.22 01:53:09.340181 [ 1476 ] {} CacheMetadata: Key 820753cede6aa3e7956e92e665fe3023 is removed from metadata 2024.10.22 01:53:09.468890 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 76545 2024.10.22 01:53:09.641638 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 1048576 bytes for d3f9f35ada37427e56d6b835a1211363:0) 2024.10.22 01:53:09.644579 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a200e7576e2e88de8f2ae31d69302bed, offset: 0, size: 2637590 2024.10.22 01:53:09.658442 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a200e7576e2e88de8f2ae31d69302bed for removal 2024.10.22 01:53:09.660928 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a786b49181eadb0e04d04d56844bc5b7, offset: 0, size: 962 2024.10.22 01:53:09.660881 [ 1476 ] {} CacheMetadata: Key a200e7576e2e88de8f2ae31d69302bed is removed from metadata 2024.10.22 01:53:09.665441 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a786b49181eadb0e04d04d56844bc5b7 for removal 2024.10.22 01:53:09.667857 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 009a48c05d96f7e8b877435b59505fc0, offset: 0, size: 1238 2024.10.22 01:53:09.670979 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 009a48c05d96f7e8b877435b59505fc0 for removal 2024.10.22 01:53:09.785164 [ 1476 ] {} CacheMetadata: Key 009a48c05d96f7e8b877435b59505fc0 is removed from metadata 2024.10.22 01:53:09.800192 [ 1476 ] {} CacheMetadata: Key a786b49181eadb0e04d04d56844bc5b7 is removed from metadata 2024.10.22 01:53:09.940058 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:09.947029 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:10.101403 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1167_1167_0 (state Temporary) 2024.10.22 01:53:10.265312 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1167_1167_0 to 202410_1180_1180_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:10.276967 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for d3f9f35ada37427e56d6b835a1211363:0) 2024.10.22 01:53:10.278867 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 99d355c68d546f1104404e14576821c5, offset: 0, size: 338623 2024.10.22 01:53:10.285521 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 99d355c68d546f1104404e14576821c5 for removal 2024.10.22 01:53:10.292690 [ 1476 ] {} CacheMetadata: Key 99d355c68d546f1104404e14576821c5 is removed from metadata 2024.10.22 01:53:10.293534 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1ad8b972adf66a004c8b1d8f2a6e4ae2, offset: 0, size: 990 2024.10.22 01:53:10.302187 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1ad8b972adf66a004c8b1d8f2a6e4ae2 for removal 2024.10.22 01:53:10.306938 [ 1476 ] {} CacheMetadata: Key 1ad8b972adf66a004c8b1d8f2a6e4ae2 is removed from metadata 2024.10.22 01:53:10.441058 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 76545 2024.10.22 01:53:10.506035 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:53:10.533039 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1009937 2024.10.22 01:53:10.548044 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 8.63 MiB, will set to 1.64 GiB (RSS), difference: -15.11 MiB 2024.10.22 01:53:10.739645 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 470698 bytes for d3f9f35ada37427e56d6b835a1211363:0) 2024.10.22 01:53:10.746390 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3684660e3eb08e82a657cf4a2ea2bda1, offset: 0, size: 772347 2024.10.22 01:53:10.749599 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3684660e3eb08e82a657cf4a2ea2bda1 for removal 2024.10.22 01:53:10.753579 [ 1476 ] {} CacheMetadata: Key 3684660e3eb08e82a657cf4a2ea2bda1 is removed from metadata 2024.10.22 01:53:10.755150 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f6aa7c4c094bf04d1f2c67dc57d66fd1, offset: 0, size: 739 2024.10.22 01:53:10.758564 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f6aa7c4c094bf04d1f2c67dc57d66fd1 for removal 2024.10.22 01:53:10.760995 [ 1476 ] {} CacheMetadata: Key f6aa7c4c094bf04d1f2c67dc57d66fd1 is removed from metadata 2024.10.22 01:53:10.855391 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:53:10.856993 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:53:10.952777 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 23409d7e68f7cf704ed3c02293b23e58, offset: 0, size: 0 2024.10.22 01:53:10.956044 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 23409d7e68f7cf704ed3c02293b23e58 for removal 2024.10.22 01:53:10.958565 [ 1476 ] {} CacheMetadata: Key 23409d7e68f7cf704ed3c02293b23e58 is removed from metadata 2024.10.22 01:53:11.281159 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:11.289401 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:11.383933 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_663_663_0 (state Temporary) 2024.10.22 01:53:11.401238 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 10529 2024.10.22 01:53:11.562886 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_663_663_0 to 202410_670_670_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:11.636100 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:11.644604 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:11.672827 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_692_692_0 (state Temporary) 2024.10.22 01:53:11.704020 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1009937 2024.10.22 01:53:11.705915 [ 1213 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_665_665_0 to 202410_670_670_0 2024.10.22 01:53:11.735876 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_692_692_0 to 202410_705_705_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:11.755513 [ 1213 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:11.757665 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:11.754118 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5575 2024.10.22 01:53:11.784661 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10529 2024.10.22 01:53:11.794183 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_665_665_0 to 202410_670_670_0 into Compact with storage Full 2024.10.22 01:53:11.856562 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:53:11.862391 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_665_665_0, total 926 rows starting from the beginning of the part 2024.10.22 01:53:11.884560 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 31 file segments (while reserving 1048576 bytes for 23409d7e68f7cf704ed3c02293b23e58:0) 2024.10.22 01:53:11.895753 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5d6bce733ea8c725a036f0c6f649a679, offset: 0, size: 142 2024.10.22 01:53:11.899451 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_666_666_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:53:11.900158 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5d6bce733ea8c725a036f0c6f649a679 for removal 2024.10.22 01:53:11.903661 [ 1476 ] {} CacheMetadata: Key 5d6bce733ea8c725a036f0c6f649a679 is removed from metadata 2024.10.22 01:53:11.904316 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7f7c0f49bd4691a6fd7527b380a96737, offset: 0, size: 49261 2024.10.22 01:53:12.008624 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7f7c0f49bd4691a6fd7527b380a96737 for removal 2024.10.22 01:53:12.012078 [ 1476 ] {} CacheMetadata: Key 7f7c0f49bd4691a6fd7527b380a96737 is removed from metadata 2024.10.22 01:53:12.012421 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f26d1a845d4f7c994ef17d232210a306, offset: 0, size: 142 2024.10.22 01:53:12.017482 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f26d1a845d4f7c994ef17d232210a306 for removal 2024.10.22 01:53:12.022898 [ 1476 ] {} CacheMetadata: Key f26d1a845d4f7c994ef17d232210a306 is removed from metadata 2024.10.22 01:53:12.027349 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 65d899d6ec5658f4385321febc1dbb15, offset: 0, size: 142 2024.10.22 01:53:12.031589 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_667_667_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:53:12.032930 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 65d899d6ec5658f4385321febc1dbb15 for removal 2024.10.22 01:53:12.037674 [ 1476 ] {} CacheMetadata: Key 65d899d6ec5658f4385321febc1dbb15 is removed from metadata 2024.10.22 01:53:12.054765 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_668_668_0, total 1542 rows starting from the beginning of the part 2024.10.22 01:53:12.072950 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_669_669_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:53:12.101352 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_670_670_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:53:12.141667 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 38f45f09f513f3b6c19374c241d6cfe8, offset: 0, size: 50030 2024.10.22 01:53:12.148633 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 38f45f09f513f3b6c19374c241d6cfe8 for removal 2024.10.22 01:53:12.151695 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2203467956cbd8e92608a7bd93a390f1, offset: 0, size: 142 2024.10.22 01:53:12.152156 [ 1476 ] {} CacheMetadata: Key 38f45f09f513f3b6c19374c241d6cfe8 is removed from metadata 2024.10.22 01:53:12.156500 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 2203467956cbd8e92608a7bd93a390f1 for removal 2024.10.22 01:53:12.163102 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: dc2f3d7595e1afaa97bee9825df98cf0, offset: 0, size: 46049 2024.10.22 01:53:12.162967 [ 1476 ] {} CacheMetadata: Key 2203467956cbd8e92608a7bd93a390f1 is removed from metadata 2024.10.22 01:53:12.169210 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key dc2f3d7595e1afaa97bee9825df98cf0 for removal 2024.10.22 01:53:12.172070 [ 1476 ] {} CacheMetadata: Key dc2f3d7595e1afaa97bee9825df98cf0 is removed from metadata 2024.10.22 01:53:12.172800 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 83900c727691ea8db7db6f4e7076eb77, offset: 0, size: 142 2024.10.22 01:53:12.175451 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 83900c727691ea8db7db6f4e7076eb77 for removal 2024.10.22 01:53:12.182344 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7c8679b3600e0534b10803ccb535f320, offset: 0, size: 4964 2024.10.22 01:53:12.190630 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7c8679b3600e0534b10803ccb535f320 for removal 2024.10.22 01:53:12.202550 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 901e87c1ff9f2e9cbf54831d9646b984, offset: 0, size: 56 2024.10.22 01:53:12.204468 [ 1476 ] {} CacheMetadata: Key 7c8679b3600e0534b10803ccb535f320 is removed from metadata 2024.10.22 01:53:12.208255 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 901e87c1ff9f2e9cbf54831d9646b984 for removal 2024.10.22 01:53:12.221583 [ 1476 ] {} CacheMetadata: Key 901e87c1ff9f2e9cbf54831d9646b984 is removed from metadata 2024.10.22 01:53:12.227248 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4f36811a1fd02fb31f0c971cd850dfac, offset: 0, size: 142 2024.10.22 01:53:12.238192 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 4f36811a1fd02fb31f0c971cd850dfac for removal 2024.10.22 01:53:12.244614 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 09fb2f4930267c194905cc69db952dea, offset: 0, size: 164 2024.10.22 01:53:12.246882 [ 1476 ] {} CacheMetadata: Key 4f36811a1fd02fb31f0c971cd850dfac is removed from metadata 2024.10.22 01:53:12.247582 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 09fb2f4930267c194905cc69db952dea for removal 2024.10.22 01:53:12.258341 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 32810d41081302178cbca1384ce392dc, offset: 0, size: 142 2024.10.22 01:53:12.262568 [ 1476 ] {} CacheMetadata: Key 09fb2f4930267c194905cc69db952dea is removed from metadata 2024.10.22 01:53:12.262768 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 32810d41081302178cbca1384ce392dc for removal 2024.10.22 01:53:12.268715 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b92a5ce2c69fc08aafeac78613949307, offset: 0, size: 58 2024.10.22 01:53:12.271314 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b92a5ce2c69fc08aafeac78613949307 for removal 2024.10.22 01:53:12.277952 [ 1476 ] {} CacheMetadata: Key b92a5ce2c69fc08aafeac78613949307 is removed from metadata 2024.10.22 01:53:12.278853 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8dc3169b451bb67261b6cdd586fe0859, offset: 0, size: 4 2024.10.22 01:53:12.287915 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8dc3169b451bb67261b6cdd586fe0859 for removal 2024.10.22 01:53:12.301018 [ 1476 ] {} CacheMetadata: Key 8dc3169b451bb67261b6cdd586fe0859 is removed from metadata 2024.10.22 01:53:12.304026 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4eb9c0f0bb953e5b36c9bb7156053e5b, offset: 0, size: 52584 2024.10.22 01:53:12.307108 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 4eb9c0f0bb953e5b36c9bb7156053e5b for removal 2024.10.22 01:53:12.312302 [ 1476 ] {} CacheMetadata: Key 32810d41081302178cbca1384ce392dc is removed from metadata 2024.10.22 01:53:12.325411 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e7be6351e18933e60dfa42594aa0a4ce, offset: 0, size: 56 2024.10.22 01:53:12.330858 [ 1476 ] {} CacheMetadata: Key 4eb9c0f0bb953e5b36c9bb7156053e5b is removed from metadata 2024.10.22 01:53:12.334653 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e7be6351e18933e60dfa42594aa0a4ce for removal 2024.10.22 01:53:12.343984 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 255d9952c594567f4e18d2ee723ec076, offset: 0, size: 2500 2024.10.22 01:53:12.346630 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 255d9952c594567f4e18d2ee723ec076 for removal 2024.10.22 01:53:12.345217 [ 1476 ] {} CacheMetadata: Key e7be6351e18933e60dfa42594aa0a4ce is removed from metadata 2024.10.22 01:53:12.351413 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1440ec4480a54cd5e6ce5d54c04169b9, offset: 0, size: 56 2024.10.22 01:53:12.355050 [ 1476 ] {} CacheMetadata: Key 255d9952c594567f4e18d2ee723ec076 is removed from metadata 2024.10.22 01:53:12.355704 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1440ec4480a54cd5e6ce5d54c04169b9 for removal 2024.10.22 01:53:12.359148 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: da391b35d27ae4afbf4f78c38cfcdc4e, offset: 0, size: 535 2024.10.22 01:53:12.360803 [ 1476 ] {} CacheMetadata: Key 1440ec4480a54cd5e6ce5d54c04169b9 is removed from metadata 2024.10.22 01:53:12.362641 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key da391b35d27ae4afbf4f78c38cfcdc4e for removal 2024.10.22 01:53:12.367095 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 97a0d16a0aa2d9db7d7960f2ed986096, offset: 0, size: 58 2024.10.22 01:53:12.367516 [ 1476 ] {} CacheMetadata: Key da391b35d27ae4afbf4f78c38cfcdc4e is removed from metadata 2024.10.22 01:53:12.371018 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 97a0d16a0aa2d9db7d7960f2ed986096 for removal 2024.10.22 01:53:12.373522 [ 1476 ] {} CacheMetadata: Key 97a0d16a0aa2d9db7d7960f2ed986096 is removed from metadata 2024.10.22 01:53:12.373833 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: fb919e83f08c464ca033c3e2e63825c0, offset: 0, size: 58 2024.10.22 01:53:12.377411 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key fb919e83f08c464ca033c3e2e63825c0 for removal 2024.10.22 01:53:12.378740 [ 1476 ] {} CacheMetadata: Key 83900c727691ea8db7db6f4e7076eb77 is removed from metadata 2024.10.22 01:53:12.383068 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: aeac369e39a6f8084daff4407c1b90fc, offset: 0, size: 45363 2024.10.22 01:53:12.385034 [ 1476 ] {} CacheMetadata: Key fb919e83f08c464ca033c3e2e63825c0 is removed from metadata 2024.10.22 01:53:12.390387 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key aeac369e39a6f8084daff4407c1b90fc for removal 2024.10.22 01:53:12.397684 [ 1476 ] {} CacheMetadata: Key aeac369e39a6f8084daff4407c1b90fc is removed from metadata 2024.10.22 01:53:12.400005 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9cb36d082034ed9c6e5ef16d1954bd05, offset: 0, size: 51317 2024.10.22 01:53:12.406881 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9cb36d082034ed9c6e5ef16d1954bd05 for removal 2024.10.22 01:53:12.412405 [ 1476 ] {} CacheMetadata: Key 9cb36d082034ed9c6e5ef16d1954bd05 is removed from metadata 2024.10.22 01:53:12.413213 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: aca2bfe7bbadbd0f9b2a47d1b6c4947c, offset: 0, size: 142 2024.10.22 01:53:12.418956 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key aca2bfe7bbadbd0f9b2a47d1b6c4947c for removal 2024.10.22 01:53:12.428547 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bed7008320449c04e54ed24cbfc0eeb7, offset: 0, size: 599 2024.10.22 01:53:12.434406 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bed7008320449c04e54ed24cbfc0eeb7 for removal 2024.10.22 01:53:12.442916 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d8029618340ed2d3105f0b4c8d136189, offset: 0, size: 535 2024.10.22 01:53:12.446834 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d8029618340ed2d3105f0b4c8d136189 for removal 2024.10.22 01:53:12.450895 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c83446186e5bb3b6cf674777b35bf252, offset: 0, size: 535 2024.10.22 01:53:12.455225 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c83446186e5bb3b6cf674777b35bf252 for removal 2024.10.22 01:53:12.457557 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 164d3e407138c1a648892c2e6fbae707, offset: 0, size: 906181 2024.10.22 01:53:12.460525 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 164d3e407138c1a648892c2e6fbae707 for removal 2024.10.22 01:53:12.464301 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: df97416ee53929f5f14d9c7d4fdf2043, offset: 0, size: 53621 2024.10.22 01:53:12.468029 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key df97416ee53929f5f14d9c7d4fdf2043 for removal 2024.10.22 01:53:12.473165 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7e765cbe3b54542b5946151963ba6985, offset: 0, size: 753 2024.10.22 01:53:12.478236 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7e765cbe3b54542b5946151963ba6985 for removal 2024.10.22 01:53:12.529252 [ 1476 ] {} CacheMetadata: Key 7e765cbe3b54542b5946151963ba6985 is removed from metadata 2024.10.22 01:53:12.540405 [ 1476 ] {} CacheMetadata: Key df97416ee53929f5f14d9c7d4fdf2043 is removed from metadata 2024.10.22 01:53:12.554082 [ 1476 ] {} CacheMetadata: Key 164d3e407138c1a648892c2e6fbae707 is removed from metadata 2024.10.22 01:53:12.570833 [ 1476 ] {} CacheMetadata: Key c83446186e5bb3b6cf674777b35bf252 is removed from metadata 2024.10.22 01:53:12.594457 [ 1476 ] {} CacheMetadata: Key d8029618340ed2d3105f0b4c8d136189 is removed from metadata 2024.10.22 01:53:12.604742 [ 1476 ] {} CacheMetadata: Key bed7008320449c04e54ed24cbfc0eeb7 is removed from metadata 2024.10.22 01:53:12.618736 [ 1476 ] {} CacheMetadata: Key aca2bfe7bbadbd0f9b2a47d1b6c4947c is removed from metadata 2024.10.22 01:53:12.681374 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 1.97 GiB, free memory in arenas 15.17 MiB, will set to 1.69 GiB (RSS), difference: -91.09 MiB 2024.10.22 01:53:13.174245 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 578 2024.10.22 01:53:13.207012 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 77091 2024.10.22 01:53:13.456562 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:13.459126 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:13.516871 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_680_680_0 (state Temporary) 2024.10.22 01:53:13.527998 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:13.511045 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} MergeTask::MergeProjectionsStage: Merge sorted 7858 rows, containing 4 columns (4 merged, 0 gathered) in 1.739076796 sec., 4518.489360604407 rows/sec., 70.60 KiB/sec. 2024.10.22 01:53:13.644461 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:13.672491 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_680_680_0 to 202410_686_686_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:13.727238 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_560_560_0 (state Temporary) 2024.10.22 01:53:13.750837 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77091 2024.10.22 01:53:13.727078 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 1048576 bytes for 23409d7e68f7cf704ed3c02293b23e58:0) 2024.10.22 01:53:13.756567 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b0cafada75e4e52ee202182c84a45b4c, offset: 0, size: 761257 2024.10.22 01:53:13.760531 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b0cafada75e4e52ee202182c84a45b4c for removal 2024.10.22 01:53:13.767996 [ 1476 ] {} CacheMetadata: Key b0cafada75e4e52ee202182c84a45b4c is removed from metadata 2024.10.22 01:53:13.768417 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f77ca80383de7fa96701fc8e28dabd15, offset: 0, size: 51766 2024.10.22 01:53:13.774338 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f77ca80383de7fa96701fc8e28dabd15 for removal 2024.10.22 01:53:13.780686 [ 1476 ] {} CacheMetadata: Key f77ca80383de7fa96701fc8e28dabd15 is removed from metadata 2024.10.22 01:53:13.780879 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 042061a10196bb8871211ff541089231, offset: 0, size: 141 2024.10.22 01:53:13.892208 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 042061a10196bb8871211ff541089231 for removal 2024.10.22 01:53:13.925428 [ 1476 ] {} CacheMetadata: Key 042061a10196bb8871211ff541089231 is removed from metadata 2024.10.22 01:53:13.925226 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 93433c155d91d5e4867b6cc36f57d775, offset: 0, size: 51814 2024.10.22 01:53:13.926083 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} MergedBlockOutputStream: filled checksums 202410_665_670_1 (state Temporary) 2024.10.22 01:53:13.927993 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_560_560_0 to 202410_570_570_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:13.931310 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 93433c155d91d5e4867b6cc36f57d775 for removal 2024.10.22 01:53:13.935890 [ 1476 ] {} CacheMetadata: Key 93433c155d91d5e4867b6cc36f57d775 is removed from metadata 2024.10.22 01:53:13.936360 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8dec6eab618921401715c61071fc928b, offset: 0, size: 142 2024.10.22 01:53:13.941696 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8dec6eab618921401715c61071fc928b for removal 2024.10.22 01:53:13.947633 [ 1476 ] {} CacheMetadata: Key 8dec6eab618921401715c61071fc928b is removed from metadata 2024.10.22 01:53:13.947912 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 35280db9ffc644e4b11013d96b4f7a08, offset: 0, size: 589 2024.10.22 01:53:13.951781 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 35280db9ffc644e4b11013d96b4f7a08 for removal 2024.10.22 01:53:13.955054 [ 1476 ] {} CacheMetadata: Key 35280db9ffc644e4b11013d96b4f7a08 is removed from metadata 2024.10.22 01:53:13.955068 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: df43b7ae87af54ab23cbdcdee62298eb, offset: 0, size: 47567 2024.10.22 01:53:13.968797 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key df43b7ae87af54ab23cbdcdee62298eb for removal 2024.10.22 01:53:13.972125 [ 1476 ] {} CacheMetadata: Key df43b7ae87af54ab23cbdcdee62298eb is removed from metadata 2024.10.22 01:53:13.972577 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9bd23ab05a22fe448f3fc25eff6973a0, offset: 0, size: 142 2024.10.22 01:53:13.978427 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9bd23ab05a22fe448f3fc25eff6973a0 for removal 2024.10.22 01:53:13.989310 [ 1476 ] {} CacheMetadata: Key 9bd23ab05a22fe448f3fc25eff6973a0 is removed from metadata 2024.10.22 01:53:14.005590 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_665_670_1 to 202410_665_670_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:14.130787 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_665_670_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_665_665_0, 202410_670_670_0] -> 202410_665_670_1 2024.10.22 01:53:14.197545 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:53:14.270330 [ 1248 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:53:14.305493 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 578 2024.10.22 01:53:14.309733 [ 1248 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:53:14.448940 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 272.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:14.452178 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 200 entries to flush up to offset 76745 2024.10.22 01:53:15.252558 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 23409d7e68f7cf704ed3c02293b23e58:0) 2024.10.22 01:53:15.255650 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7dcb5817ff7187b9691da1cfa2b23a05, offset: 0, size: 750094 2024.10.22 01:53:15.261468 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7dcb5817ff7187b9691da1cfa2b23a05 for removal 2024.10.22 01:53:15.267374 [ 1476 ] {} CacheMetadata: Key 7dcb5817ff7187b9691da1cfa2b23a05 is removed from metadata 2024.10.22 01:53:15.267615 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ce7ad6ebc9c480ad2b382d797c9e58c3, offset: 0, size: 53620 2024.10.22 01:53:15.271713 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ce7ad6ebc9c480ad2b382d797c9e58c3 for removal 2024.10.22 01:53:15.277477 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c0b8a339431dcafa491fce21ae90c1af, offset: 0, size: 587 2024.10.22 01:53:15.283168 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c0b8a339431dcafa491fce21ae90c1af for removal 2024.10.22 01:53:15.292560 [ 1476 ] {} CacheMetadata: Key ce7ad6ebc9c480ad2b382d797c9e58c3 is removed from metadata 2024.10.22 01:53:15.296695 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7a8a45f81a665b1a6252d6c7137d3db2, offset: 0, size: 142 2024.10.22 01:53:15.304853 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7a8a45f81a665b1a6252d6c7137d3db2 for removal 2024.10.22 01:53:15.309319 [ 1476 ] {} CacheMetadata: Key c0b8a339431dcafa491fce21ae90c1af is removed from metadata 2024.10.22 01:53:15.315514 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d6b7318f9a48259697354ed579fb446e, offset: 0, size: 44212 2024.10.22 01:53:15.323856 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d6b7318f9a48259697354ed579fb446e for removal 2024.10.22 01:53:15.334659 [ 1476 ] {} CacheMetadata: Key d6b7318f9a48259697354ed579fb446e is removed from metadata 2024.10.22 01:53:15.336704 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ee4ec134a488a76b61820d0e62dab275, offset: 0, size: 142 2024.10.22 01:53:15.347867 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ee4ec134a488a76b61820d0e62dab275 for removal 2024.10.22 01:53:15.356730 [ 1476 ] {} CacheMetadata: Key ee4ec134a488a76b61820d0e62dab275 is removed from metadata 2024.10.22 01:53:15.470072 [ 1476 ] {} CacheMetadata: Key 7a8a45f81a665b1a6252d6c7137d3db2 is removed from metadata 2024.10.22 01:53:15.573595 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lsh/tcvagrkxswftkyqbvyegnovbgfixe (LogSeriesLimiter: on interval from 2024-10-22 01:53:07 to 2024-10-22 01:53:15 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:53:15.774765 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:15.780817 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:15.831397 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 13.60 MiB, will set to 1.70 GiB (RSS), difference: 4.13 MiB 2024.10.22 01:53:15.900544 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1168_1168_0 (state Temporary) 2024.10.22 01:53:16.299374 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 7 file segments (while reserving 1048576 bytes for 810f22ef34688d3ae2626571a4e17381:0) 2024.10.22 01:53:16.304752 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 89e39cd17adb9efcfe98e51a910a72d6, offset: 0, size: 758 2024.10.22 01:53:16.307851 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 89e39cd17adb9efcfe98e51a910a72d6 for removal 2024.10.22 01:53:16.312851 [ 1476 ] {} CacheMetadata: Key 89e39cd17adb9efcfe98e51a910a72d6 is removed from metadata 2024.10.22 01:53:16.312879 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d550d52017fde09e8bc17abc445da762, offset: 0, size: 1166208 2024.10.22 01:53:16.319191 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d550d52017fde09e8bc17abc445da762 for removal 2024.10.22 01:53:16.322389 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b3faaa0c064a7b3427a600cc9d23404e, offset: 0, size: 180 2024.10.22 01:53:16.327875 [ 1476 ] {} CacheMetadata: Key d550d52017fde09e8bc17abc445da762 is removed from metadata 2024.10.22 01:53:16.329672 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b3faaa0c064a7b3427a600cc9d23404e for removal 2024.10.22 01:53:16.333003 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4ea765d7140aa1f635981ca613c1686a, offset: 0, size: 1268 2024.10.22 01:53:16.336159 [ 1476 ] {} CacheMetadata: Key b3faaa0c064a7b3427a600cc9d23404e is removed from metadata 2024.10.22 01:53:16.335976 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4ea765d7140aa1f635981ca613c1686a for removal 2024.10.22 01:53:16.346377 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 82cbbfdcbbc245d1630ba972ffa41efe, offset: 0, size: 154 2024.10.22 01:53:16.352886 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 82cbbfdcbbc245d1630ba972ffa41efe for removal 2024.10.22 01:53:16.359056 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6cd621e374b8a0fb283bcde7ac1c0830, offset: 0, size: 37020 2024.10.22 01:53:16.361104 [ 1476 ] {} CacheMetadata: Key 82cbbfdcbbc245d1630ba972ffa41efe is removed from metadata 2024.10.22 01:53:16.362219 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6cd621e374b8a0fb283bcde7ac1c0830 for removal 2024.10.22 01:53:16.365308 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a972af4378748d77aea040446de52e01, offset: 0, size: 142 2024.10.22 01:53:16.368369 [ 1476 ] {} CacheMetadata: Key 6cd621e374b8a0fb283bcde7ac1c0830 is removed from metadata 2024.10.22 01:53:16.368605 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a972af4378748d77aea040446de52e01 for removal 2024.10.22 01:53:16.370370 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1168_1168_0 to 202410_1181_1181_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:16.389109 [ 1476 ] {} CacheMetadata: Key a972af4378748d77aea040446de52e01 is removed from metadata 2024.10.22 01:53:16.404712 [ 1476 ] {} CacheMetadata: Key 4ea765d7140aa1f635981ca613c1686a is removed from metadata 2024.10.22 01:53:16.446548 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1176_16 to 202410_1181_1181_0 2024.10.22 01:53:16.448152 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 76745 2024.10.22 01:53:16.457643 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:16.461217 [ 1266 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:16.528929 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1176_16 to 202410_1181_1181_0 into Compact with storage Full 2024.10.22 01:53:16.537690 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/efn/yjdyrfqvgkbcxetnnmievckqyuwwg. 2024.10.22 01:53:16.580358 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:53:16.593834 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1176_16, total 5671 rows starting from the beginning of the part 2024.10.22 01:53:16.700887 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 98252f8c71e856785e9602b1eb18feee, offset: 0, size: 0 2024.10.22 01:53:16.792583 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1177_1177_0, total 82 rows starting from the beginning of the part 2024.10.22 01:53:16.805100 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 98252f8c71e856785e9602b1eb18feee for removal 2024.10.22 01:53:16.810170 [ 1476 ] {} CacheMetadata: Key 98252f8c71e856785e9602b1eb18feee is removed from metadata 2024.10.22 01:53:16.831190 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1178_1178_0, total 97 rows starting from the beginning of the part 2024.10.22 01:53:16.858689 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1179_1179_0, total 61 rows starting from the beginning of the part 2024.10.22 01:53:16.884143 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/efn/yjdyrfqvgkbcxetnnmievckqyuwwg. 2024.10.22 01:53:16.904985 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1180_1180_0, total 58 rows starting from the beginning of the part 2024.10.22 01:53:16.934307 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1181_1181_0, total 200 rows starting from the beginning of the part 2024.10.22 01:53:17.439227 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 1048576 bytes for 8430e472d0ef29ae6e5e0d47d80698e8:0) 2024.10.22 01:53:17.441829 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a3bf8db61fe9ad20a39381a00b92533b, offset: 0, size: 531486 2024.10.22 01:53:17.549532 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a3bf8db61fe9ad20a39381a00b92533b for removal 2024.10.22 01:53:17.555463 [ 1476 ] {} CacheMetadata: Key a3bf8db61fe9ad20a39381a00b92533b is removed from metadata 2024.10.22 01:53:17.559110 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2a771c5ffb8bb7bbf6f4d6b6493f18c6, offset: 0, size: 582 2024.10.22 01:53:17.562423 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 2a771c5ffb8bb7bbf6f4d6b6493f18c6 for removal 2024.10.22 01:53:17.565642 [ 1476 ] {} CacheMetadata: Key 2a771c5ffb8bb7bbf6f4d6b6493f18c6 is removed from metadata 2024.10.22 01:53:17.566255 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2cd8ef4f95687a07afb97a135913f4d2, offset: 0, size: 432915 2024.10.22 01:53:17.573527 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 2cd8ef4f95687a07afb97a135913f4d2 for removal 2024.10.22 01:53:17.580413 [ 1476 ] {} CacheMetadata: Key 2cd8ef4f95687a07afb97a135913f4d2 is removed from metadata 2024.10.22 01:53:18.651587 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 98252f8c71e856785e9602b1eb18feee:0) 2024.10.22 01:53:18.653749 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 08f212c87a27b9dcac14002b8642e7d2, offset: 0, size: 864677 2024.10.22 01:53:18.656983 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 08f212c87a27b9dcac14002b8642e7d2 for removal 2024.10.22 01:53:18.661109 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0867f4e8cd13a0d0c1c9ae07cd3211bf, offset: 0, size: 158 2024.10.22 01:53:18.661706 [ 1476 ] {} CacheMetadata: Key 08f212c87a27b9dcac14002b8642e7d2 is removed from metadata 2024.10.22 01:53:18.666843 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0867f4e8cd13a0d0c1c9ae07cd3211bf for removal 2024.10.22 01:53:18.670783 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7d0d8d7726806a281132660c5cb14408, offset: 0, size: 534861 2024.10.22 01:53:18.677468 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7d0d8d7726806a281132660c5cb14408 for removal 2024.10.22 01:53:18.682020 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5e40a8cc72e51bbc5d8dda7e37a5be7b, offset: 0, size: 181 2024.10.22 01:53:18.689091 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5e40a8cc72e51bbc5d8dda7e37a5be7b for removal 2024.10.22 01:53:18.693486 [ 1476 ] {} CacheMetadata: Key 7d0d8d7726806a281132660c5cb14408 is removed from metadata 2024.10.22 01:53:18.723573 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1151 entries to flush up to offset 1011088 2024.10.22 01:53:18.730668 [ 1476 ] {} CacheMetadata: Key 5e40a8cc72e51bbc5d8dda7e37a5be7b is removed from metadata 2024.10.22 01:53:18.750540 [ 1476 ] {} CacheMetadata: Key 0867f4e8cd13a0d0c1c9ae07cd3211bf is removed from metadata 2024.10.22 01:53:18.855083 [ 1217 ] {} DatabaseCatalog: Execution took 257 ms. 2024.10.22 01:53:18.897408 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} MergeTask::MergeProjectionsStage: Merge sorted 6169 rows, containing 13 columns (13 merged, 0 gathered) in 2.368818687 sec., 2604.251660903923 rows/sec., 449.57 KiB/sec. 2024.10.22 01:53:19.017069 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} MergedBlockOutputStream: filled checksums 202410_1096_1181_17 (state Temporary) 2024.10.22 01:53:19.289577 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 10541 2024.10.22 01:53:19.309832 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 5618 2024.10.22 01:53:19.497722 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1181_17 to 202410_1096_1181_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:19.532717 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1181_17} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1176_16, 202410_1181_1181_0] -> 202410_1096_1181_17 2024.10.22 01:53:19.541494 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:19.545705 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:19.583242 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.80 MiB. 2024.10.22 01:53:19.603343 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_693_693_0 (state Temporary) 2024.10.22 01:53:19.606118 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:19.612690 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:19.758443 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_693_693_0 to 202410_706_706_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:19.763810 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.96 GiB, peak 1.97 GiB, free memory in arenas 13.28 MiB, will set to 1.71 GiB (RSS), difference: -261.61 MiB 2024.10.22 01:53:19.808911 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10541 2024.10.22 01:53:19.841382 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_664_128/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:53:19.858369 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_664_664_0 (state Temporary) 2024.10.22 01:53:19.997199 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_664_664_0 to 202410_671_671_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:20.048874 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1011088 2024.10.22 01:53:20.099265 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jve/uotfevpplelxxcpzyydignkpujqql (LogSeriesLimiter: on interval from 2024-10-22 01:53:15 to 2024-10-22 01:53:20 accepted series 1 / 197 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:53:20.371248 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 98252f8c71e856785e9602b1eb18feee:0) 2024.10.22 01:53:20.373477 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6bcf41483bfa0ab63a436581c4a1c4f0, offset: 0, size: 2799559 2024.10.22 01:53:20.397081 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6bcf41483bfa0ab63a436581c4a1c4f0 for removal 2024.10.22 01:53:20.401820 [ 1476 ] {} CacheMetadata: Key 6bcf41483bfa0ab63a436581c4a1c4f0 is removed from metadata 2024.10.22 01:53:20.402815 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9221ed3e0276d4c7589f2453ee7a8678, offset: 0, size: 1031 2024.10.22 01:53:20.407827 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9221ed3e0276d4c7589f2453ee7a8678 for removal 2024.10.22 01:53:20.417387 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 75e9e273407b25997a9a6ef8a50f1ec3, offset: 0, size: 54180 2024.10.22 01:53:20.427798 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 75e9e273407b25997a9a6ef8a50f1ec3 for removal 2024.10.22 01:53:20.436222 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 19b0d736fc460b0f6683b3fcfddc72ad, offset: 0, size: 154 2024.10.22 01:53:20.445773 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 19b0d736fc460b0f6683b3fcfddc72ad for removal 2024.10.22 01:53:20.459839 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 121 entries to flush up to offset 76866 2024.10.22 01:53:20.528624 [ 1476 ] {} CacheMetadata: Key 9221ed3e0276d4c7589f2453ee7a8678 is removed from metadata 2024.10.22 01:53:20.542585 [ 1476 ] {} CacheMetadata: Key 19b0d736fc460b0f6683b3fcfddc72ad is removed from metadata 2024.10.22 01:53:20.652428 [ 1476 ] {} CacheMetadata: Key 75e9e273407b25997a9a6ef8a50f1ec3 is removed from metadata 2024.10.22 01:53:20.855483 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:53:20.859051 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:53:20.912953 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vgr/uydwxzksooxgoefebqhbpkrehfcjh. 2024.10.22 01:53:20.953491 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 096af66b26a92fdf7f85204b2c2959fd, offset: 0, size: 0 2024.10.22 01:53:20.955645 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 096af66b26a92fdf7f85204b2c2959fd for removal 2024.10.22 01:53:20.958698 [ 1476 ] {} CacheMetadata: Key 096af66b26a92fdf7f85204b2c2959fd is removed from metadata 2024.10.22 01:53:21.001247 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:21.011812 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vgr/uydwxzksooxgoefebqhbpkrehfcjh. 2024.10.22 01:53:21.017354 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:21.210356 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1169_1169_0 (state Temporary) 2024.10.22 01:53:21.262171 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 77162 2024.10.22 01:53:21.282320 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1169_1169_0 to 202410_1182_1182_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:21.366344 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:21.370924 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:21.421500 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_681_681_0 (state Temporary) 2024.10.22 01:53:21.478945 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_681_681_0 to 202410_687_687_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:21.537316 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 76866 2024.10.22 01:53:21.558893 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77162 2024.10.22 01:53:21.810586 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 579 2024.10.22 01:53:22.218206 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 096af66b26a92fdf7f85204b2c2959fd:0) 2024.10.22 01:53:22.225650 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b02482912600d053054de9321d6cdf8f, offset: 0, size: 2176921 2024.10.22 01:53:22.235448 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b02482912600d053054de9321d6cdf8f for removal 2024.10.22 01:53:22.243624 [ 1476 ] {} CacheMetadata: Key b02482912600d053054de9321d6cdf8f is removed from metadata 2024.10.22 01:53:22.244529 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 68135861c18967d1c38b68430e8e22fc, offset: 0, size: 1022 2024.10.22 01:53:22.248671 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 68135861c18967d1c38b68430e8e22fc for removal 2024.10.22 01:53:22.253092 [ 1476 ] {} CacheMetadata: Key 68135861c18967d1c38b68430e8e22fc is removed from metadata 2024.10.22 01:53:22.284210 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:22.288416 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:22.569566 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_561_561_0 (state Temporary) 2024.10.22 01:53:22.576156 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.86 GiB, peak 1.97 GiB, free memory in arenas 16.00 MiB, will set to 1.70 GiB (RSS), difference: -157.00 MiB 2024.10.22 01:53:22.771418 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_561_561_0 to 202410_571_571_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:23.035032 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 579 2024.10.22 01:53:23.070732 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 096af66b26a92fdf7f85204b2c2959fd:0) 2024.10.22 01:53:23.072689 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ff8a12368c7e050fad5a0bd1a638bf90, offset: 0, size: 767 2024.10.22 01:53:23.086676 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ff8a12368c7e050fad5a0bd1a638bf90 for removal 2024.10.22 01:53:23.092493 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d29cf84e32a807b80eb8b48691857d3a, offset: 0, size: 59579 2024.10.22 01:53:23.092779 [ 1476 ] {} CacheMetadata: Key ff8a12368c7e050fad5a0bd1a638bf90 is removed from metadata 2024.10.22 01:53:23.104281 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d29cf84e32a807b80eb8b48691857d3a for removal 2024.10.22 01:53:23.107701 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4d65c76ffb9deeacc0eff90ae45d6b51, offset: 0, size: 1353456 2024.10.22 01:53:23.113149 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 4d65c76ffb9deeacc0eff90ae45d6b51 for removal 2024.10.22 01:53:23.119282 [ 1476 ] {} CacheMetadata: Key 4d65c76ffb9deeacc0eff90ae45d6b51 is removed from metadata 2024.10.22 01:53:23.126791 [ 1476 ] {} CacheMetadata: Key d29cf84e32a807b80eb8b48691857d3a is removed from metadata 2024.10.22 01:53:23.117833 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 570ed97eaa85cbe97490fdaba817d424, offset: 0, size: 961 2024.10.22 01:53:23.254596 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 570ed97eaa85cbe97490fdaba817d424 for removal 2024.10.22 01:53:23.255867 [ 1476 ] {} CacheMetadata: Key 570ed97eaa85cbe97490fdaba817d424 is removed from metadata 2024.10.22 01:53:24.319805 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 656275 bytes for 8430e472d0ef29ae6e5e0d47d80698e8:0) 2024.10.22 01:53:24.327678 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 91302091f50805f2638898f91aee8388, offset: 0, size: 2018645 2024.10.22 01:53:24.336690 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 91302091f50805f2638898f91aee8388 for removal 2024.10.22 01:53:24.342924 [ 1476 ] {} CacheMetadata: Key 91302091f50805f2638898f91aee8388 is removed from metadata 2024.10.22 01:53:24.343953 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7d77b2092f0d446121a6ff699c5b317e, offset: 0, size: 1017 2024.10.22 01:53:24.347829 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 7d77b2092f0d446121a6ff699c5b317e for removal 2024.10.22 01:53:24.350647 [ 1476 ] {} CacheMetadata: Key 7d77b2092f0d446121a6ff699c5b317e is removed from metadata 2024.10.22 01:53:24.423904 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/plz/lhxfwmybrfgokkeydmbywohnyimgz. 2024.10.22 01:53:24.476486 [ 1719 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nnw/fvqzhecuclwhjjsifwrbhyjcqorgd, size 2753427 2024.10.22 01:53:24.531021 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cd8664dde187b099f8377096eb650839, offset: 0, size: 0 2024.10.22 01:53:24.533169 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cd8664dde187b099f8377096eb650839 for removal 2024.10.22 01:53:24.536295 [ 1476 ] {} CacheMetadata: Key cd8664dde187b099f8377096eb650839 is removed from metadata 2024.10.22 01:53:24.803705 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/plz/lhxfwmybrfgokkeydmbywohnyimgz. 2024.10.22 01:53:25.120943 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_664_128} MergeTask::MergeProjectionsStage: Merge sorted 1015669 rows, containing 4 columns (4 merged, 0 gathered) in 30.248373049 sec., 33577.64063391759 rows/sec., 503.49 KiB/sec. 2024.10.22 01:53:25.173340 [ 1696 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/brk/egwwtoukviprcmhaccjrdzfdiwmwe, size 2320384 2024.10.22 01:53:25.266989 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RegionID 2024.10.22 01:53:25.273057 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RegionID 2024.10.22 01:53:25.275162 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_664_128} MergedBlockOutputStream: filled checksums 202410_1_664_128 (state Temporary) 2024.10.22 01:53:25.381255 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RegionID 2024.10.22 01:53:25.496114 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RegionID 2024.10.22 01:53:25.519905 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RegionID 2024.10.22 01:53:25.552502 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RegionID 2024.10.22 01:53:25.565125 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 76936 2024.10.22 01:53:25.584163 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RegionID 2024.10.22 01:53:25.693590 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:53:25.710849 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:53:25.759812 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gxl/sxqmdlbxffsnvmotishhsdxffzdzx (LogSeriesLimiter: on interval from 2024-10-22 01:53:20 to 2024-10-22 01:53:25 accepted series 1 / 166 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:53:25.852419 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:53:25.886547 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:53:25.942667 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:53:25.965288 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.81 GiB, peak 1.97 GiB, free memory in arenas 15.16 MiB, will set to 1.71 GiB (RSS), difference: -108.23 MiB 2024.10.22 01:53:25.969742 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:53:26.013754 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:53:26.027799 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for cd8664dde187b099f8377096eb650839:0) 2024.10.22 01:53:26.035235 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b901de3434d8b6261ce3a19b340302f1, offset: 0, size: 2826516 2024.10.22 01:53:26.051949 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b901de3434d8b6261ce3a19b340302f1 for removal 2024.10.22 01:53:26.054577 [ 1476 ] {} CacheMetadata: Key b901de3434d8b6261ce3a19b340302f1 is removed from metadata 2024.10.22 01:53:26.055884 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d967dcf8a62f65c17917722c7c42e682, offset: 0, size: 1035 2024.10.22 01:53:26.063040 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d967dcf8a62f65c17917722c7c42e682 for removal 2024.10.22 01:53:26.071401 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 01:53:26.184369 [ 1476 ] {} CacheMetadata: Key d967dcf8a62f65c17917722c7c42e682 is removed from metadata 2024.10.22 01:53:26.269178 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:26.275192 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:26.469995 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1170_1170_0 (state Temporary) 2024.10.22 01:53:26.544270 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1170_1170_0 to 202410_1183_1183_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:26.661014 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 76936 2024.10.22 01:53:27.053620 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1159 entries to flush up to offset 1012247 2024.10.22 01:53:27.106681 [ 1276 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Selected 5 parts from 202410_1_30_7 to 202410_34_34_0 2024.10.22 01:53:27.124843 [ 1276 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:27.130729 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:27.156698 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_34_8} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_30_7 to 202410_34_34_0 into Compact with storage Full 2024.10.22 01:53:27.314936 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10554 2024.10.22 01:53:27.472288 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:27.475758 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.87 GiB. 2024.10.22 01:53:27.557348 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 11.89 MiB, will set to 1.71 GiB (RSS), difference: -15.01 MiB 2024.10.22 01:53:27.632812 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_694_694_0 (state Temporary) 2024.10.22 01:53:27.661758 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 751357 bytes for cd8664dde187b099f8377096eb650839:0) 2024.10.22 01:53:27.663125 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_34_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:53:27.666184 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f8ec25a0412ed1dbc7f4b238871dba96, offset: 0, size: 1048576 2024.10.22 01:53:27.671338 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d4556f01e35f2bcaa99692cd75ce02bd, offset: 0, size: 1053 2024.10.22 01:53:27.673152 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_694_694_0 to 202410_707_707_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:27.674135 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d4556f01e35f2bcaa99692cd75ce02bd for removal 2024.10.22 01:53:27.675272 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_34_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_30_7, total 36 rows starting from the beginning of the part 2024.10.22 01:53:27.677658 [ 1476 ] {} CacheMetadata: Key d4556f01e35f2bcaa99692cd75ce02bd is removed from metadata 2024.10.22 01:53:27.751165 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10554 2024.10.22 01:53:27.875921 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_34_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_31_31_0, total 1 rows starting from the beginning of the part 2024.10.22 01:53:27.903100 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qrd/rweqosuxphxypxxbanjkdnvhmvgwh. 2024.10.22 01:53:27.951923 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_34_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_32_32_0, total 1 rows starting from the beginning of the part 2024.10.22 01:53:27.992333 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5bec27c39a4f955b239f9aacfff68355, offset: 0, size: 0 2024.10.22 01:53:27.995438 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5bec27c39a4f955b239f9aacfff68355 for removal 2024.10.22 01:53:27.997841 [ 1476 ] {} CacheMetadata: Key 5bec27c39a4f955b239f9aacfff68355 is removed from metadata 2024.10.22 01:53:28.020870 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_34_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_33_33_0, total 1 rows starting from the beginning of the part 2024.10.22 01:53:28.055854 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:28.070438 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_34_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_34_34_0, total 1 rows starting from the beginning of the part 2024.10.22 01:53:28.158787 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.87 GiB. 2024.10.22 01:53:28.174394 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qrd/rweqosuxphxypxxbanjkdnvhmvgwh. 2024.10.22 01:53:28.295461 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_665_665_0 (state Temporary) 2024.10.22 01:53:28.467557 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_665_665_0 to 202410_672_672_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:28.520518 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1012247 2024.10.22 01:53:29.042342 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 1048576 bytes for 5bec27c39a4f955b239f9aacfff68355:0) 2024.10.22 01:53:29.046537 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f8ec25a0412ed1dbc7f4b238871dba96, offset: 1048576, size: 2471030 2024.10.22 01:53:29.074422 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 89 entries to flush up to offset 77251 2024.10.22 01:53:29.083818 [ 1202 ] {} DatabaseCatalog: Execution took 227 ms. 2024.10.22 01:53:29.154642 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f8ec25a0412ed1dbc7f4b238871dba96 for removal 2024.10.22 01:53:29.158309 [ 1476 ] {} CacheMetadata: Key f8ec25a0412ed1dbc7f4b238871dba96 is removed from metadata 2024.10.22 01:53:29.170861 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_664_128} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_664_128 to 202410_1_664_128 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:29.207735 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_664_128} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_658_127, 202410_659_664_1] -> 202410_1_664_128 2024.10.22 01:53:29.254570 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:53:29.316060 [ 1193 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:53:29.365668 [ 1193 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:53:29.461636 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:29.468516 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:29.637555 [ 1390 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_34_8} MergeTask::MergeProjectionsStage: Merge sorted 40 rows, containing 27 columns (27 merged, 0 gathered) in 2.485153004 sec., 16.095588454963398 rows/sec., 43.70 KiB/sec. 2024.10.22 01:53:29.674194 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_682_682_0 (state Temporary) 2024.10.22 01:53:29.733624 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_682_682_0 to 202410_688_688_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:29.743719 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1065_1080_3, 202410_1081_1081_0, 202410_1082_1082_0, 202410_1083_1083_0, 202410_1084_1084_0, 202410_1085_1085_0] 2024.10.22 01:53:29.750785 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1065_1080_3, 202410_1081_1081_0, 202410_1082_1082_0, 202410_1083_1083_0, 202410_1084_1084_0, 202410_1085_1085_0] 2024.10.22 01:53:29.791756 [ 1388 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_34_8} MergedBlockOutputStream: filled checksums 202410_1_34_8 (state Temporary) 2024.10.22 01:53:29.809688 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77251 2024.10.22 01:53:29.843138 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/dad/ebvcohxfzeigtqjaearyccukivdhj. 2024.10.22 01:53:29.873587 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9872bbfcdab5b61b9be1b446b980ecf5, offset: 0, size: 0 2024.10.22 01:53:29.875843 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9872bbfcdab5b61b9be1b446b980ecf5 for removal 2024.10.22 01:53:29.878431 [ 1476 ] {} CacheMetadata: Key 9872bbfcdab5b61b9be1b446b980ecf5 is removed from metadata 2024.10.22 01:53:29.897496 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1065_1080_3, 202410_1081_1081_0, 202410_1082_1082_0, 202410_1083_1083_0, 202410_1084_1084_0, 202410_1085_1085_0] 2024.10.22 01:53:29.948109 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/dad/ebvcohxfzeigtqjaearyccukivdhj. 2024.10.22 01:53:30.222941 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_518_106, 202410_519_519_0, 202410_520_520_0, 202410_521_521_0, 202410_522_522_0, 202410_523_523_0] 2024.10.22 01:53:30.229651 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_518_106, 202410_519_519_0, 202410_520_520_0, 202410_521_521_0, 202410_522_522_0, 202410_523_523_0] 2024.10.22 01:53:30.260283 [ 1390 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_34_8} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_merge_202410_1_34_8 to 202410_1_34_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:30.265249 [ 1390 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_34_8} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Merged 5 parts: [202410_1_30_7, 202410_34_34_0] -> 202410_1_34_8 2024.10.22 01:53:30.268753 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_518_106, 202410_519_519_0, 202410_520_520_0, 202410_521_521_0, 202410_522_522_0, 202410_523_523_0] 2024.10.22 01:53:30.301828 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.24 MiB. 2024.10.22 01:53:30.669423 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 90 entries to flush up to offset 77026 2024.10.22 01:53:30.859488 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:53:30.864680 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:53:30.875641 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:30.882364 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:31.013655 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 9872bbfcdab5b61b9be1b446b980ecf5:0) 2024.10.22 01:53:31.017648 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 76d9b3500f21e8cdb0626ffa5cbaa840, offset: 0, size: 5743662 2024.10.22 01:53:31.027707 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 76d9b3500f21e8cdb0626ffa5cbaa840 for removal 2024.10.22 01:53:31.041527 [ 1476 ] {} CacheMetadata: Key 76d9b3500f21e8cdb0626ffa5cbaa840 is removed from metadata 2024.10.22 01:53:31.048275 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0ddfe740366066a96c7392346853ea3d, offset: 0, size: 1098 2024.10.22 01:53:31.053795 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0ddfe740366066a96c7392346853ea3d for removal 2024.10.22 01:53:31.165292 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 01:53:31.167132 [ 1476 ] {} CacheMetadata: Key 0ddfe740366066a96c7392346853ea3d is removed from metadata 2024.10.22 01:53:31.184533 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1171_1171_0 (state Temporary) 2024.10.22 01:53:31.409553 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1171_1171_0 to 202410_1184_1184_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:31.494161 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 77026 2024.10.22 01:53:31.533148 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 7.08 MiB, will set to 1.73 GiB (RSS), difference: 2.35 MiB 2024.10.22 01:53:31.655855 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tvu/huefavmxcoccggbzkqqjsechxlgfz. 2024.10.22 01:53:31.850412 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bb1126d74011b3caa96a5fcb91af0ac9, offset: 0, size: 0 2024.10.22 01:53:31.853542 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bb1126d74011b3caa96a5fcb91af0ac9 for removal 2024.10.22 01:53:31.857343 [ 1476 ] {} CacheMetadata: Key bb1126d74011b3caa96a5fcb91af0ac9 is removed from metadata 2024.10.22 01:53:32.313698 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tvu/huefavmxcoccggbzkqqjsechxlgfz. 2024.10.22 01:53:33.464407 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pvp/hayqaxtysacyiceddyilajnkijuql. 2024.10.22 01:53:33.520472 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f661ce116c3ef0f653e2199ac45c8023, offset: 0, size: 0 2024.10.22 01:53:33.530927 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f661ce116c3ef0f653e2199ac45c8023 for removal 2024.10.22 01:53:33.543053 [ 1476 ] {} CacheMetadata: Key f661ce116c3ef0f653e2199ac45c8023 is removed from metadata 2024.10.22 01:53:33.701237 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pvp/hayqaxtysacyiceddyilajnkijuql. 2024.10.22 01:53:33.707852 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 6.20 MiB, will set to 1.73 GiB (RSS), difference: 15.42 MiB 2024.10.22 01:53:34.780485 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for f661ce116c3ef0f653e2199ac45c8023:0) 2024.10.22 01:53:34.790312 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d3f9f35ada37427e56d6b835a1211363, offset: 0, size: 5713578 2024.10.22 01:53:34.803847 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d3f9f35ada37427e56d6b835a1211363 for removal 2024.10.22 01:53:34.814169 [ 1476 ] {} CacheMetadata: Key d3f9f35ada37427e56d6b835a1211363 is removed from metadata 2024.10.22 01:53:34.817770 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 29383d832410694452c1711cba4fa422, offset: 0, size: 1098 2024.10.22 01:53:34.830497 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 29383d832410694452c1711cba4fa422 for removal 2024.10.22 01:53:34.834262 [ 1476 ] {} CacheMetadata: Key 29383d832410694452c1711cba4fa422 is removed from metadata 2024.10.22 01:53:35.253465 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10567 2024.10.22 01:53:35.376391 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/cmu/avugrslubmlwktkrgwstsfbyddaxh. 2024.10.22 01:53:35.507922 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 37 entries to flush up to offset 77063 2024.10.22 01:53:35.534462 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 4.63 MiB, will set to 1.74 GiB (RSS), difference: 3.68 MiB 2024.10.22 01:53:35.537808 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 911feed3b8162db7c90be37a15c082ef, offset: 0, size: 0 2024.10.22 01:53:35.543381 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1013787 2024.10.22 01:53:35.543491 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 911feed3b8162db7c90be37a15c082ef for removal 2024.10.22 01:53:35.548159 [ 1476 ] {} CacheMetadata: Key 911feed3b8162db7c90be37a15c082ef is removed from metadata 2024.10.22 01:53:35.606522 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/cmu/avugrslubmlwktkrgwstsfbyddaxh. 2024.10.22 01:53:35.643524 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:35.648577 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:35.783411 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:35.791542 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_695_695_0 (state Temporary) 2024.10.22 01:53:35.798757 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:35.868714 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1172_1172_0 (state Temporary) 2024.10.22 01:53:35.954526 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_695_695_0 to 202410_708_708_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:35.956431 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1172_1172_0 to 202410_1185_1185_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:36.000724 [ 1230 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_703_142 to 202410_704_704_0 2024.10.22 01:53:36.012587 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10567 2024.10.22 01:53:36.026720 [ 1230 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:36.034501 [ 1230 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:36.148569 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 77063 2024.10.22 01:53:36.183051 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_704_143} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_703_142 to 202410_704_704_0 into Compact with storage Full 2024.10.22 01:53:36.269790 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_704_143} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:53:36.276114 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_704_143} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_703_142, total 10599 rows starting from the beginning of the part 2024.10.22 01:53:36.333392 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_704_143} MergeTreeSequentialSource: Reading 2 marks from part 202410_704_704_0, total 28 rows starting from the beginning of the part 2024.10.22 01:53:36.678499 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:36.686869 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:36.669778 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for aeaf59ec551ddaed69d45f3bc5a4c53a:0) 2024.10.22 01:53:36.779399 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 23409d7e68f7cf704ed3c02293b23e58, offset: 0, size: 3564870 2024.10.22 01:53:36.791542 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 23409d7e68f7cf704ed3c02293b23e58 for removal 2024.10.22 01:53:36.798619 [ 1476 ] {} CacheMetadata: Key 23409d7e68f7cf704ed3c02293b23e58 is removed from metadata 2024.10.22 01:53:36.798521 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 97fb0aa5140433f3a873e34452149a87, offset: 0, size: 1053 2024.10.22 01:53:36.804657 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 97fb0aa5140433f3a873e34452149a87 for removal 2024.10.22 01:53:36.936787 [ 1476 ] {} CacheMetadata: Key 97fb0aa5140433f3a873e34452149a87 is removed from metadata 2024.10.22 01:53:36.937233 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_666_666_0 (state Temporary) 2024.10.22 01:53:37.005077 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_625_24, 202410_626_626_0, 202410_627_627_0, 202410_628_628_0, 202410_629_629_0, 202410_630_630_0] 2024.10.22 01:53:37.010619 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_625_24, 202410_626_626_0, 202410_627_627_0, 202410_628_628_0, 202410_629_629_0, 202410_630_630_0] 2024.10.22 01:53:37.059258 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_666_666_0 to 202410_673_673_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:37.071004 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_625_24, 202410_626_626_0, 202410_627_627_0, 202410_628_628_0, 202410_629_629_0, 202410_630_630_0] 2024.10.22 01:53:37.162833 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1013787 2024.10.22 01:53:37.347973 [ 1274 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 01:53:37.384645 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_704_143} MergeTask::MergeProjectionsStage: Merge sorted 10627 rows, containing 12 columns (12 merged, 0 gathered) in 1.229323648 sec., 8644.590883197587 rows/sec., 1.34 MiB/sec. 2024.10.22 01:53:37.435759 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 64 entries to flush up to offset 77315 2024.10.22 01:53:37.443977 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_704_143} MergedBlockOutputStream: filled checksums 202410_1_704_143 (state Temporary) 2024.10.22 01:53:37.629328 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_704_143} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_704_143 to 202410_1_704_143 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:37.640174 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_704_143} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_703_142, 202410_704_704_0] -> 202410_1_704_143 2024.10.22 01:53:37.663852 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 01:53:37.668257 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:37.670132 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:37.673939 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 4.96 MiB, will set to 1.75 GiB (RSS), difference: 11.06 MiB 2024.10.22 01:53:37.815095 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_683_683_0 (state Temporary) 2024.10.22 01:53:37.969948 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_683_683_0 to 202410_689_689_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:38.032239 [ 1267 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_684_38 to 202410_685_685_0 2024.10.22 01:53:38.042101 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 580 2024.10.22 01:53:38.047370 [ 1267 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.05 MiB using storage policy from min volume index 0 2024.10.22 01:53:38.050226 [ 1267 ] {} DiskLocal: Reserved 1.05 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:38.053971 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77315 2024.10.22 01:53:38.175636 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_685_39} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_684_38 to 202410_685_685_0 into Compact with storage Full 2024.10.22 01:53:38.225200 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_685_39} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:53:38.250162 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_685_39} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_684_38, total 17796 rows starting from the beginning of the part 2024.10.22 01:53:38.295334 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_685_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_685_685_0, total 99 rows starting from the beginning of the part 2024.10.22 01:53:38.648287 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:38.650159 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nrj/nktopxjmlnmbrtmweacjtyfmfdvnc. 2024.10.22 01:53:38.651417 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:38.777362 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_611_611_0, 202410_612_612_0, 202410_613_613_0, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0] 2024.10.22 01:53:38.784161 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_611_611_0, 202410_612_612_0, 202410_613_613_0, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0] 2024.10.22 01:53:38.832757 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_562_562_0 (state Temporary) 2024.10.22 01:53:39.033643 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_611_611_0, 202410_612_612_0, 202410_613_613_0, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0] 2024.10.22 01:53:39.036991 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_562_562_0 to 202410_572_572_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:39.072589 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nrj/nktopxjmlnmbrtmweacjtyfmfdvnc. 2024.10.22 01:53:39.147072 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 580 2024.10.22 01:53:39.308095 [ 1262 ] {} DatabaseCatalog: Execution took 218 ms. 2024.10.22 01:53:39.561987 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 2.78 MiB, will set to 1.74 GiB (RSS), difference: -12.00 MiB 2024.10.22 01:53:39.603139 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/msf/mwkvghgxfzrtyeajlfqafhcjqhilv. 2024.10.22 01:53:39.859832 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/msf/mwkvghgxfzrtyeajlfqafhcjqhilv. 2024.10.22 01:53:40.151782 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 77137 2024.10.22 01:53:40.640664 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_685_39} MergeTask::MergeProjectionsStage: Merge sorted 17895 rows, containing 13 columns (13 merged, 0 gathered) in 2.543085203 sec., 7036.728450501703 rows/sec., 3.05 MiB/sec. 2024.10.22 01:53:40.634968 [ 1232 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 01:53:40.859221 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:53:40.871568 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:53:40.899230 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_685_39} MergedBlockOutputStream: filled checksums 202410_507_685_39 (state Temporary) 2024.10.22 01:53:40.907436 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:40.913477 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:40.983010 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:41.065402 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_685_39} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_685_39 to 202410_507_685_39 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:41.068512 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_685_39} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_684_38, 202410_685_685_0] -> 202410_507_685_39 2024.10.22 01:53:41.077543 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:41.092137 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1173_1173_0 (state Temporary) 2024.10.22 01:53:41.118897 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ayl/bifobpyjvkgszdqoqehehmajxtxtd. 2024.10.22 01:53:41.157112 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1173_1173_0 to 202410_1186_1186_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:41.177458 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1181_17 to 202410_1186_1186_0 2024.10.22 01:53:41.197539 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:41.199904 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.18 MiB. 2024.10.22 01:53:41.207581 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 77137 2024.10.22 01:53:41.227133 [ 1218 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.88 GiB. 2024.10.22 01:53:41.270916 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1181_17 to 202410_1186_1186_0 into Compact with storage Full 2024.10.22 01:53:41.279694 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ayl/bifobpyjvkgszdqoqehehmajxtxtd. 2024.10.22 01:53:41.373249 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 3.23 MiB, will set to 1.74 GiB (RSS), difference: 33.30 MiB 2024.10.22 01:53:41.553717 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 352459d1c4cfdc6748c41e77b6bef95d, offset: 0, size: 0 2024.10.22 01:53:41.555153 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 352459d1c4cfdc6748c41e77b6bef95d for removal 2024.10.22 01:53:41.559805 [ 1476 ] {} CacheMetadata: Key 352459d1c4cfdc6748c41e77b6bef95d is removed from metadata 2024.10.22 01:53:41.658716 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:53:41.667151 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1181_17, total 6169 rows starting from the beginning of the part 2024.10.22 01:53:41.764771 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1182_1182_0, total 121 rows starting from the beginning of the part 2024.10.22 01:53:41.950810 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1183_1183_0, total 70 rows starting from the beginning of the part 2024.10.22 01:53:42.015819 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1184_1184_0, total 90 rows starting from the beginning of the part 2024.10.22 01:53:42.272194 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1185_1185_0, total 37 rows starting from the beginning of the part 2024.10.22 01:53:42.376619 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1186_1186_0, total 74 rows starting from the beginning of the part 2024.10.22 01:53:42.601630 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a44d6eef407ee51c19666ece288c4ded, offset: 0, size: 0 2024.10.22 01:53:42.709145 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a44d6eef407ee51c19666ece288c4ded for removal 2024.10.22 01:53:42.718779 [ 1476 ] {} CacheMetadata: Key a44d6eef407ee51c19666ece288c4ded is removed from metadata 2024.10.22 01:53:43.102974 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ojq/hfybufjjavwwigwkpnvhjhphetysk. 2024.10.22 01:53:43.295416 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 698c991925e310ea37acbce74e6d9224, offset: 0, size: 0 2024.10.22 01:53:43.312167 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 698c991925e310ea37acbce74e6d9224 for removal 2024.10.22 01:53:43.315830 [ 1476 ] {} CacheMetadata: Key 698c991925e310ea37acbce74e6d9224 is removed from metadata 2024.10.22 01:53:43.526569 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 10579 2024.10.22 01:53:43.602614 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ojq/hfybufjjavwwigwkpnvhjhphetysk. 2024.10.22 01:53:43.693495 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:43.707289 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.87 GiB. 2024.10.22 01:53:43.773482 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_696_696_0 (state Temporary) 2024.10.22 01:53:43.795108 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1096_1186_18/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:53:43.874469 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 1.97 GiB, free memory in arenas 3.16 MiB, will set to 1.74 GiB (RSS), difference: 126.72 MiB 2024.10.22 01:53:43.905109 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fwl/xavtbxrnjphzigvcvahyzelmsnrms. 2024.10.22 01:53:43.975899 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b162f2da5992839e249c59b049b0d4e7, offset: 0, size: 0 2024.10.22 01:53:43.979037 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b162f2da5992839e249c59b049b0d4e7 for removal 2024.10.22 01:53:43.982994 [ 1476 ] {} CacheMetadata: Key b162f2da5992839e249c59b049b0d4e7 is removed from metadata 2024.10.22 01:53:44.002803 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_696_696_0 to 202410_709_709_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:44.061160 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10579 2024.10.22 01:53:44.051417 [ 1195 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_704_143 to 202410_709_709_0 2024.10.22 01:53:44.075210 [ 1195 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:44.076806 [ 1195 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.87 GiB. 2024.10.22 01:53:44.160161 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_704_143 to 202410_709_709_0 into Compact with storage Full 2024.10.22 01:53:44.168026 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 1015329 2024.10.22 01:53:44.242039 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:53:44.268870 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_704_143, total 10627 rows starting from the beginning of the part 2024.10.22 01:53:44.306410 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fwl/xavtbxrnjphzigvcvahyzelmsnrms. 2024.10.22 01:53:44.354166 [ 1188 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 262 ms. 2024.10.22 01:53:44.370953 [ 1253 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:53:44.405013 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} MergeTask::MergeProjectionsStage: Merge sorted 6561 rows, containing 13 columns (13 merged, 0 gathered) in 3.107331233 sec., 2111.458196127236 rows/sec., 362.92 KiB/sec. 2024.10.22 01:53:44.427702 [ 1253 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:53:44.548258 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 26335d5b4dcead96aa0d86894252461f, offset: 0, size: 0 2024.10.22 01:53:44.550865 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 26335d5b4dcead96aa0d86894252461f for removal 2024.10.22 01:53:44.558508 [ 1476 ] {} CacheMetadata: Key 26335d5b4dcead96aa0d86894252461f is removed from metadata 2024.10.22 01:53:44.558760 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_705_705_0, total 8 rows starting from the beginning of the part 2024.10.22 01:53:44.624146 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 50d122ca54e36f39a94b0127142bc915, offset: 0, size: 0 2024.10.22 01:53:44.629496 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 50d122ca54e36f39a94b0127142bc915 for removal 2024.10.22 01:53:44.636866 [ 1476 ] {} CacheMetadata: Key 50d122ca54e36f39a94b0127142bc915 is removed from metadata 2024.10.22 01:53:44.637548 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rpx/joqzudphqrxprngyxsttlkyajtyul. 2024.10.22 01:53:44.717050 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} MergedBlockOutputStream: filled checksums 202410_1096_1186_18 (state Temporary) 2024.10.22 01:53:44.725507 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ef41ada3f15ab6a65c05ae70181a09a7, offset: 0, size: 0 2024.10.22 01:53:44.735750 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ef41ada3f15ab6a65c05ae70181a09a7 for removal 2024.10.22 01:53:44.744953 [ 1476 ] {} CacheMetadata: Key ef41ada3f15ab6a65c05ae70181a09a7 is removed from metadata 2024.10.22 01:53:44.813182 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_706_706_0, total 12 rows starting from the beginning of the part 2024.10.22 01:53:44.976986 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 28 file segments (while reserving 2097152 bytes for 163535c645584a69033b089d7370229d:0) 2024.10.22 01:53:44.983499 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2add8f85280bad6a88585bb1d20e4187, offset: 0, size: 1029 2024.10.22 01:53:44.989805 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2add8f85280bad6a88585bb1d20e4187 for removal 2024.10.22 01:53:44.997347 [ 1476 ] {} CacheMetadata: Key 2add8f85280bad6a88585bb1d20e4187 is removed from metadata 2024.10.22 01:53:45.002838 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 02c0bf267302d1ce9f1af75aa3ce181b, offset: 0, size: 950 2024.10.22 01:53:45.005325 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: db95e82f6610c50c8010750e16083aa3, offset: 0, size: 0 2024.10.22 01:53:45.006564 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 02c0bf267302d1ce9f1af75aa3ce181b for removal 2024.10.22 01:53:45.008691 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key db95e82f6610c50c8010750e16083aa3 for removal 2024.10.22 01:53:45.013962 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 88193f83256faf63b5cbc93d82512275, offset: 0, size: 537 2024.10.22 01:53:45.016989 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 88193f83256faf63b5cbc93d82512275 for removal 2024.10.22 01:53:45.026973 [ 1476 ] {} CacheMetadata: Key db95e82f6610c50c8010750e16083aa3 is removed from metadata 2024.10.22 01:53:45.030981 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0d518ff4d75744ff81a47989b7ba41e5, offset: 0, size: 819 2024.10.22 01:53:45.033169 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rpx/joqzudphqrxprngyxsttlkyajtyul. 2024.10.22 01:53:45.034653 [ 1476 ] {} CacheMetadata: Key 88193f83256faf63b5cbc93d82512275 is removed from metadata 2024.10.22 01:53:45.042575 [ 1476 ] {} CacheMetadata: Key 02c0bf267302d1ce9f1af75aa3ce181b is removed from metadata 2024.10.22 01:53:45.136698 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0d518ff4d75744ff81a47989b7ba41e5 for removal 2024.10.22 01:53:45.139627 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_707_707_0, total 13 rows starting from the beginning of the part 2024.10.22 01:53:45.144548 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d059763f830ef5e637cc473be6465e0e, offset: 0, size: 154 2024.10.22 01:53:45.152431 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d059763f830ef5e637cc473be6465e0e for removal 2024.10.22 01:53:45.157081 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 43d024c81c23252f8694752ee9fa14c5, offset: 0, size: 483 2024.10.22 01:53:45.166532 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 43d024c81c23252f8694752ee9fa14c5 for removal 2024.10.22 01:53:45.172811 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 82fe4ec6ec2c99457dcb1746dba3f90d, offset: 0, size: 532 2024.10.22 01:53:45.178998 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 82fe4ec6ec2c99457dcb1746dba3f90d for removal 2024.10.22 01:53:45.190067 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5ce313874270e2c5b0b055086fb30f50, offset: 0, size: 508 2024.10.22 01:53:45.213017 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5ce313874270e2c5b0b055086fb30f50 for removal 2024.10.22 01:53:45.227624 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: dc2232227d1c9a176ea0151b5a695ad0, offset: 0, size: 0 2024.10.22 01:53:45.238270 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 102 entries to flush up to offset 77239 2024.10.22 01:53:45.242266 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key dc2232227d1c9a176ea0151b5a695ad0 for removal 2024.10.22 01:53:45.242593 [ 1476 ] {} CacheMetadata: Key 0d518ff4d75744ff81a47989b7ba41e5 is removed from metadata 2024.10.22 01:53:45.243608 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4d8a40d8fc5481727a8fc2239bd63262, offset: 0, size: 593 2024.10.22 01:53:45.251806 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4d8a40d8fc5481727a8fc2239bd63262 for removal 2024.10.22 01:53:45.255233 [ 1476 ] {} CacheMetadata: Key dc2232227d1c9a176ea0151b5a695ad0 is removed from metadata 2024.10.22 01:53:45.264755 [ 1476 ] {} CacheMetadata: Key 4d8a40d8fc5481727a8fc2239bd63262 is removed from metadata 2024.10.22 01:53:45.263081 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 957d5a54e3867b9d53cdacf68b37bcd9, offset: 0, size: 931 2024.10.22 01:53:45.270518 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1186_18 to 202410_1096_1186_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:45.273258 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1186_18} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1181_17, 202410_1186_1186_0] -> 202410_1096_1186_18 2024.10.22 01:53:45.274583 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 957d5a54e3867b9d53cdacf68b37bcd9 for removal 2024.10.22 01:53:45.299130 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9c9f2a34468110994812c0e70c6ab006, offset: 0, size: 62 2024.10.22 01:53:45.316073 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9c9f2a34468110994812c0e70c6ab006 for removal 2024.10.22 01:53:45.329671 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 60705ae02c8ab04c5516a7357e755577, offset: 0, size: 0 2024.10.22 01:53:45.332963 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 60705ae02c8ab04c5516a7357e755577 for removal 2024.10.22 01:53:45.377128 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.84 MiB. 2024.10.22 01:53:45.378052 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_708_708_0, total 13 rows starting from the beginning of the part 2024.10.22 01:53:45.388158 [ 1476 ] {} CacheMetadata: Key 5ce313874270e2c5b0b055086fb30f50 is removed from metadata 2024.10.22 01:53:45.419995 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:45.439098 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 01:53:45.439061 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1924da7d9695307b645e67392836a757, offset: 0, size: 532 2024.10.22 01:53:45.450165 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1924da7d9695307b645e67392836a757 for removal 2024.10.22 01:53:45.455072 [ 1476 ] {} CacheMetadata: Key 60705ae02c8ab04c5516a7357e755577 is removed from metadata 2024.10.22 01:53:45.460379 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 44053028bfa74e6b186721ca4d98eca9, offset: 0, size: 153 2024.10.22 01:53:45.462350 [ 1476 ] {} CacheMetadata: Key 1924da7d9695307b645e67392836a757 is removed from metadata 2024.10.22 01:53:45.467952 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 44053028bfa74e6b186721ca4d98eca9 for removal 2024.10.22 01:53:45.470503 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9a471337fc256661140bcd9d58ee46f4, offset: 0, size: 746 2024.10.22 01:53:45.475982 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9a471337fc256661140bcd9d58ee46f4 for removal 2024.10.22 01:53:45.478870 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b68012a0c4dfd82d8772743aa0809b28, offset: 0, size: 0 2024.10.22 01:53:45.483017 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4fde0178fcd94e40ab4a1b80cf6c746e, offset: 0, size: 620 2024.10.22 01:53:45.486181 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b68012a0c4dfd82d8772743aa0809b28 for removal 2024.10.22 01:53:45.492413 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4fde0178fcd94e40ab4a1b80cf6c746e for removal 2024.10.22 01:53:45.504088 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4997ac6aac5125f34d488a822e401427, offset: 0, size: 1192 2024.10.22 01:53:45.511881 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4997ac6aac5125f34d488a822e401427 for removal 2024.10.22 01:53:45.526652 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9ff6bb0752c66e19127bde610b01b27e, offset: 0, size: 596 2024.10.22 01:53:45.533355 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9ff6bb0752c66e19127bde610b01b27e for removal 2024.10.22 01:53:45.544896 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c072f64e01b77c5f096fc266bea7a7f5, offset: 0, size: 13242 2024.10.22 01:53:45.549816 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 56e2db7f3f6594c06f12b6fafca1c01e, offset: 0, size: 0 2024.10.22 01:53:45.551287 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c072f64e01b77c5f096fc266bea7a7f5 for removal 2024.10.22 01:53:45.554083 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 56e2db7f3f6594c06f12b6fafca1c01e for removal 2024.10.22 01:53:45.565307 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 139 entries to flush up to offset 77454 2024.10.22 01:53:45.568164 [ 1476 ] {} CacheMetadata: Key 56e2db7f3f6594c06f12b6fafca1c01e is removed from metadata 2024.10.22 01:53:45.567632 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 98252f8c71e856785e9602b1eb18feee, offset: 0, size: 2544326 2024.10.22 01:53:45.575913 [ 1476 ] {} CacheMetadata: Key c072f64e01b77c5f096fc266bea7a7f5 is removed from metadata 2024.10.22 01:53:45.577091 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 98252f8c71e856785e9602b1eb18feee for removal 2024.10.22 01:53:45.649820 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_709_709_0, total 12 rows starting from the beginning of the part 2024.10.22 01:53:45.676619 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f5fb348d856804aa3b60d23d624ca4a6, offset: 0, size: 0 2024.10.22 01:53:45.686679 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f5fb348d856804aa3b60d23d624ca4a6 for removal 2024.10.22 01:53:45.700034 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_140_140_0 (state Temporary) 2024.10.22 01:53:45.752278 [ 1476 ] {} CacheMetadata: Key f5fb348d856804aa3b60d23d624ca4a6 is removed from metadata 2024.10.22 01:53:45.733469 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 23f96213b040ac0468e024eba01523fc, offset: 0, size: 523 2024.10.22 01:53:45.762774 [ 1476 ] {} CacheMetadata: Key 98252f8c71e856785e9602b1eb18feee is removed from metadata 2024.10.22 01:53:45.763370 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 23f96213b040ac0468e024eba01523fc for removal 2024.10.22 01:53:45.775985 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e4813e0bfd68222ecff8ba51e60af7cd, offset: 0, size: 603 2024.10.22 01:53:45.788165 [ 1476 ] {} CacheMetadata: Key 23f96213b040ac0468e024eba01523fc is removed from metadata 2024.10.22 01:53:45.829049 [ 1476 ] {} CacheMetadata: Key 9ff6bb0752c66e19127bde610b01b27e is removed from metadata 2024.10.22 01:53:45.829373 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_667_667_0 (state Temporary) 2024.10.22 01:53:45.859356 [ 1476 ] {} CacheMetadata: Key 4997ac6aac5125f34d488a822e401427 is removed from metadata 2024.10.22 01:53:45.860651 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: baad079fcc9e94d1ab9ecad23685f5a3, offset: 0, size: 0 2024.10.22 01:53:45.862307 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key baad079fcc9e94d1ab9ecad23685f5a3 for removal 2024.10.22 01:53:45.868743 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gln/lebwdiracosjqaleofgyhgzptmhxt. 2024.10.22 01:53:45.872071 [ 1476 ] {} CacheMetadata: Key baad079fcc9e94d1ab9ecad23685f5a3 is removed from metadata 2024.10.22 01:53:45.885087 [ 1476 ] {} CacheMetadata: Key 4fde0178fcd94e40ab4a1b80cf6c746e is removed from metadata 2024.10.22 01:53:45.893275 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e4813e0bfd68222ecff8ba51e60af7cd for removal 2024.10.22 01:53:45.901192 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 955669c4fd657afdc4488833158064f0, offset: 0, size: 716 2024.10.22 01:53:45.908092 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 955669c4fd657afdc4488833158064f0 for removal 2024.10.22 01:53:45.923270 [ 1476 ] {} CacheMetadata: Key e4813e0bfd68222ecff8ba51e60af7cd is removed from metadata 2024.10.22 01:53:45.939560 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 47dd784321d777431785b288055435ce, offset: 0, size: 0 2024.10.22 01:53:45.985695 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:46.029452 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 01:53:46.031647 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b16410573e98950a9f29d0bffe7f6c71, offset: 0, size: 572 2024.10.22 01:53:46.039858 [ 1476 ] {} CacheMetadata: Key 955669c4fd657afdc4488833158064f0 is removed from metadata 2024.10.22 01:53:46.050615 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b16410573e98950a9f29d0bffe7f6c71 for removal 2024.10.22 01:53:46.051464 [ 1476 ] {} CacheMetadata: Key b68012a0c4dfd82d8772743aa0809b28 is removed from metadata 2024.10.22 01:53:46.054864 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3d1a1b41ac58088ff3925c39a4a9d216, offset: 0, size: 676 2024.10.22 01:53:46.056831 [ 1476 ] {} CacheMetadata: Key b16410573e98950a9f29d0bffe7f6c71 is removed from metadata 2024.10.22 01:53:46.061826 [ 1476 ] {} CacheMetadata: Key 9a471337fc256661140bcd9d58ee46f4 is removed from metadata 2024.10.22 01:53:46.062564 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3d1a1b41ac58088ff3925c39a4a9d216 for removal 2024.10.22 01:53:46.069372 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e09735ac397494c446e75d6502bcb913, offset: 0, size: 162 2024.10.22 01:53:46.070317 [ 1476 ] {} CacheMetadata: Key 3d1a1b41ac58088ff3925c39a4a9d216 is removed from metadata 2024.10.22 01:53:46.075254 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e09735ac397494c446e75d6502bcb913 for removal 2024.10.22 01:53:46.081540 [ 1476 ] {} CacheMetadata: Key e09735ac397494c446e75d6502bcb913 is removed from metadata 2024.10.22 01:53:46.082554 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e77938b233158aa3c6a38a5f85e17331, offset: 0, size: 8963 2024.10.22 01:53:46.090864 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e77938b233158aa3c6a38a5f85e17331 for removal 2024.10.22 01:53:46.091846 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:46.092534 [ 1476 ] {} CacheMetadata: Key 44053028bfa74e6b186721ca4d98eca9 is removed from metadata 2024.10.22 01:53:46.100079 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 01:53:46.103757 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 771ef480ef63565eeec7d97ce09e763e, offset: 0, size: 27651 2024.10.22 01:53:46.107041 [ 1476 ] {} CacheMetadata: Key e77938b233158aa3c6a38a5f85e17331 is removed from metadata 2024.10.22 01:53:46.113677 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 771ef480ef63565eeec7d97ce09e763e for removal 2024.10.22 01:53:46.118415 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 47dd784321d777431785b288055435ce for removal 2024.10.22 01:53:46.121434 [ 1476 ] {} CacheMetadata: Key 771ef480ef63565eeec7d97ce09e763e is removed from metadata 2024.10.22 01:53:46.131736 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 60483008caf9fe47d12a268676efd817, offset: 0, size: 51513 2024.10.22 01:53:46.155001 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 60483008caf9fe47d12a268676efd817 for removal 2024.10.22 01:53:46.146859 [ 1476 ] {} CacheMetadata: Key 47dd784321d777431785b288055435ce is removed from metadata 2024.10.22 01:53:46.175489 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_667_667_0 to 202410_674_674_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:46.180702 [ 1476 ] {} CacheMetadata: Key 60483008caf9fe47d12a268676efd817 is removed from metadata 2024.10.22 01:53:46.203990 [ 1476 ] {} CacheMetadata: Key 9c9f2a34468110994812c0e70c6ab006 is removed from metadata 2024.10.22 01:53:46.219157 [ 1476 ] {} CacheMetadata: Key 957d5a54e3867b9d53cdacf68b37bcd9 is removed from metadata 2024.10.22 01:53:46.234644 [ 1476 ] {} CacheMetadata: Key 82fe4ec6ec2c99457dcb1746dba3f90d is removed from metadata 2024.10.22 01:53:46.239269 [ 1206 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_664_128 to 202410_665_670_1 2024.10.22 01:53:46.248545 [ 1476 ] {} CacheMetadata: Key 43d024c81c23252f8694752ee9fa14c5 is removed from metadata 2024.10.22 01:53:46.255901 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_645_129, 202410_646_646_0, 202410_647_647_0, 202410_648_648_0, 202410_649_649_0, 202410_650_650_0] 2024.10.22 01:53:46.259069 [ 1476 ] {} CacheMetadata: Key d059763f830ef5e637cc473be6465e0e is removed from metadata 2024.10.22 01:53:46.259470 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_645_129, 202410_646_646_0, 202410_647_647_0, 202410_648_648_0, 202410_649_649_0, 202410_650_650_0] 2024.10.22 01:53:46.269111 [ 1206 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:46.273077 [ 1206 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 01:53:46.277039 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_684_684_0 (state Temporary) 2024.10.22 01:53:46.280205 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1174_1174_0 (state Temporary) 2024.10.22 01:53:46.357316 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_670_129} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_664_128 to 202410_665_670_1 into Wide with storage Full 2024.10.22 01:53:46.357550 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bd2839eab57038b9cf6ac68d870fc7d1, offset: 0, size: 0 2024.10.22 01:53:46.372429 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bd2839eab57038b9cf6ac68d870fc7d1 for removal 2024.10.22 01:53:46.383225 [ 1476 ] {} CacheMetadata: Key bd2839eab57038b9cf6ac68d870fc7d1 is removed from metadata 2024.10.22 01:53:46.416154 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1174_1174_0 to 202410_1187_1187_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:46.428660 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_645_129, 202410_646_646_0, 202410_647_647_0, 202410_648_648_0, 202410_649_649_0, 202410_650_650_0] 2024.10.22 01:53:46.445075 [ 1740 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gxl/sxqmdlbxffsnvmotishhsdxffzdzx, size 1717842 2024.10.22 01:53:46.453554 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1015329 2024.10.22 01:53:46.466897 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_670_129} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:53:46.486508 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_670_129} MergeTreeSequentialSource: Reading 125 marks from part 202410_1_664_128, total 1015669 rows starting from the beginning of the part 2024.10.22 01:53:46.506854 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gln/lebwdiracosjqaleofgyhgzptmhxt. 2024.10.22 01:53:46.562433 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_684_684_0 to 202410_690_690_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:46.572335 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 77239 2024.10.22 01:53:46.604157 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_670_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_665_670_1, total 7858 rows starting from the beginning of the part 2024.10.22 01:53:46.649358 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 581 2024.10.22 01:53:46.652936 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 45f406d7b82d80d8ed5acc1ac2907f2d, offset: 0, size: 0 2024.10.22 01:53:46.656667 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 45f406d7b82d80d8ed5acc1ac2907f2d for removal 2024.10.22 01:53:46.665595 [ 1476 ] {} CacheMetadata: Key 45f406d7b82d80d8ed5acc1ac2907f2d is removed from metadata 2024.10.22 01:53:46.657750 [ 1160 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_685_39 to 202410_690_690_0 2024.10.22 01:53:46.705702 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77454 2024.10.22 01:53:46.742030 [ 1160 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.09 MiB using storage policy from min volume index 0 2024.10.22 01:53:46.749674 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6e5213e822c6083487f5663f01400153, offset: 0, size: 0 2024.10.22 01:53:46.750226 [ 1160 ] {} DiskLocal: Reserved 1.09 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:53:46.751319 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6e5213e822c6083487f5663f01400153 for removal 2024.10.22 01:53:46.756273 [ 1476 ] {} CacheMetadata: Key 6e5213e822c6083487f5663f01400153 is removed from metadata 2024.10.22 01:53:46.756961 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zij/rujpvsfkukkcvcvtdhvdeccvhotnz. 2024.10.22 01:53:46.784540 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_685_39 to 202410_690_690_0 into Compact with storage Full 2024.10.22 01:53:46.798517 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:53:46.823191 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:53:46.861722 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:53:46.925884 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6488f33ca90ffaa5af6c1bcd86c5cfd0, offset: 0, size: 0 2024.10.22 01:53:46.944458 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6488f33ca90ffaa5af6c1bcd86c5cfd0 for removal 2024.10.22 01:53:46.948327 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:53:46.952151 [ 1476 ] {} CacheMetadata: Key 6488f33ca90ffaa5af6c1bcd86c5cfd0 is removed from metadata 2024.10.22 01:53:47.020882 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:53:47.091252 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 93e458e593c8d13af148507e000ea30c, offset: 0, size: 0 2024.10.22 01:53:47.101438 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 93e458e593c8d13af148507e000ea30c for removal 2024.10.22 01:53:47.101354 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:53:47.105130 [ 1476 ] {} CacheMetadata: Key 93e458e593c8d13af148507e000ea30c is removed from metadata 2024.10.22 01:53:47.155089 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:53:47.166845 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_685_39, total 17895 rows starting from the beginning of the part 2024.10.22 01:53:47.200709 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_140_140_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:53:47.216510 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:47.230671 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column CounterClass 2024.10.22 01:53:47.235109 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:53:47.258584 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zij/rujpvsfkukkcvcvtdhvdeccvhotnz. 2024.10.22 01:53:47.283146 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_686_686_0, total 71 rows starting from the beginning of the part 2024.10.22 01:53:47.336131 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5c82085508709e9daa49f735d7a862c1, offset: 0, size: 0 2024.10.22 01:53:47.344308 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5c82085508709e9daa49f735d7a862c1 for removal 2024.10.22 01:53:47.350261 [ 1476 ] {} CacheMetadata: Key 5c82085508709e9daa49f735d7a862c1 is removed from metadata 2024.10.22 01:53:47.396547 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_687_687_0, total 71 rows starting from the beginning of the part 2024.10.22 01:53:47.443836 [ 1707 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jve/uotfevpplelxxcpzyydignkpujqql, size 66509 2024.10.22 01:53:47.468393 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/keh/vcgpoerlhsbmzampgyokafnnoesrk (LogSeriesLimiter: on interval from 2024-10-22 01:53:25 to 2024-10-22 01:53:47 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:53:47.468723 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_563_563_0 (state Temporary) 2024.10.22 01:53:47.497993 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_709_144/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:53:47.508468 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d402c8ef6477953e0e19f80a18d39f94, offset: 0, size: 0 2024.10.22 01:53:47.510203 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d402c8ef6477953e0e19f80a18d39f94 for removal 2024.10.22 01:53:47.513906 [ 1476 ] {} CacheMetadata: Key d402c8ef6477953e0e19f80a18d39f94 is removed from metadata 2024.10.22 01:53:47.517128 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ysn/zwtdxtrhoxjzxdbwynspkvuymhlqz. 2024.10.22 01:53:47.522170 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_670_129/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:53:47.590551 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 947856338a3480e5faeebe424f8a1708, offset: 0, size: 0 2024.10.22 01:53:47.602433 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 947856338a3480e5faeebe424f8a1708 for removal 2024.10.22 01:53:47.605490 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_563_563_0 to 202410_573_573_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:47.616892 [ 1719 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lsh/tcvagrkxswftkyqbvyegnovbgfixe, size 312 2024.10.22 01:53:47.618280 [ 1476 ] {} CacheMetadata: Key 947856338a3480e5faeebe424f8a1708 is removed from metadata 2024.10.22 01:53:47.635625 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_688_688_0, total 89 rows starting from the beginning of the part 2024.10.22 01:53:47.575713 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_10_10_0 (state Temporary) 2024.10.22 01:53:47.754391 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_689_689_0, total 64 rows starting from the beginning of the part 2024.10.22 01:53:47.772099 [ 1163 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_568_116 to 202410_573_573_0 2024.10.22 01:53:47.781362 [ 1163 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:47.785807 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:53:47.870981 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e2630261314480e51cc5023763a1a645, offset: 0, size: 0 2024.10.22 01:53:47.872653 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e2630261314480e51cc5023763a1a645 for removal 2024.10.22 01:53:47.875132 [ 1476 ] {} CacheMetadata: Key e2630261314480e51cc5023763a1a645 is removed from metadata 2024.10.22 01:53:47.899862 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ysn/zwtdxtrhoxjzxdbwynspkvuymhlqz. 2024.10.22 01:53:47.944069 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_690_690_0, total 139 rows starting from the beginning of the part 2024.10.22 01:53:47.959552 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0051ca12b77879d9bac8677f4f8c496c, offset: 0, size: 0 2024.10.22 01:53:47.967276 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_568_116 to 202410_573_573_0 into Compact with storage Full 2024.10.22 01:53:47.967830 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0051ca12b77879d9bac8677f4f8c496c for removal 2024.10.22 01:53:47.970445 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:53:47.970659 [ 1476 ] {} CacheMetadata: Key 0051ca12b77879d9bac8677f4f8c496c is removed from metadata 2024.10.22 01:53:47.990276 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:53:48.012444 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:53:48.023964 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:53:48.031374 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 581 2024.10.22 01:53:48.041745 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_568_116, total 612 rows starting from the beginning of the part 2024.10.22 01:53:48.056952 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:53:48.070719 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:53:48.089652 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_569_569_0, total 1 rows starting from the beginning of the part 2024.10.22 01:53:48.101665 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:53:48.117075 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2606b332f81b04157634f9c923884b91, offset: 0, size: 0 2024.10.22 01:53:48.137149 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} MergeTask::MergeProjectionsStage: Merge sorted 10685 rows, containing 12 columns (12 merged, 0 gathered) in 4.014288029 sec., 2661.742237430268 rows/sec., 423.80 KiB/sec. 2024.10.22 01:53:48.153372 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:53:48.181907 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_570_570_0, total 1 rows starting from the beginning of the part 2024.10.22 01:53:48.226192 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 2606b332f81b04157634f9c923884b91 for removal 2024.10.22 01:53:48.239644 [ 1476 ] {} CacheMetadata: Key 2606b332f81b04157634f9c923884b91 is removed from metadata 2024.10.22 01:53:48.245229 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hhe/wlxuzypemqdxqqkatpvvjauyggfxk. 2024.10.22 01:53:48.264445 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 01:53:48.299167 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_571_571_0, total 1 rows starting from the beginning of the part 2024.10.22 01:53:48.302744 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d7b242d66415fc9debc74ffc3547a528, offset: 0, size: 0 2024.10.22 01:53:48.305374 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d7b242d66415fc9debc74ffc3547a528 for removal 2024.10.22 01:53:48.310825 [ 1476 ] {} CacheMetadata: Key d7b242d66415fc9debc74ffc3547a528 is removed from metadata 2024.10.22 01:53:48.357100 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_572_572_0, total 1 rows starting from the beginning of the part 2024.10.22 01:53:48.380187 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3035d6814fd3fbd30ba35847217f1205, offset: 0, size: 0 2024.10.22 01:53:48.386492 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3035d6814fd3fbd30ba35847217f1205 for removal 2024.10.22 01:53:48.391544 [ 1476 ] {} CacheMetadata: Key 3035d6814fd3fbd30ba35847217f1205 is removed from metadata 2024.10.22 01:53:48.405463 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hhe/wlxuzypemqdxqqkatpvvjauyggfxk. 2024.10.22 01:53:48.414080 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_573_573_0, total 1 rows starting from the beginning of the part 2024.10.22 01:53:48.446788 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4c064fbeef40627e0209c72dfe7cb47a, offset: 0, size: 0 2024.10.22 01:53:48.448300 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 4c064fbeef40627e0209c72dfe7cb47a for removal 2024.10.22 01:53:48.450761 [ 1476 ] {} CacheMetadata: Key 4c064fbeef40627e0209c72dfe7cb47a is removed from metadata 2024.10.22 01:53:48.478133 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} MergedBlockOutputStream: filled checksums 202410_1_709_144 (state Temporary) 2024.10.22 01:53:48.608196 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.97 GiB, free memory in arenas 4.58 MiB, will set to 1.70 GiB (RSS), difference: 218.96 MiB 2024.10.22 01:53:48.669234 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 445d7591deea600b4eb19e1bc67ec1d8, offset: 0, size: 0 2024.10.22 01:53:48.671705 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_709_144 to 202410_1_709_144 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:48.672688 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 445d7591deea600b4eb19e1bc67ec1d8 for removal 2024.10.22 01:53:48.692517 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tmx/epqubnnvscnkcummwiirokrhersxw. 2024.10.22 01:53:48.693650 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_709_144} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_704_143, 202410_709_709_0] -> 202410_1_709_144 2024.10.22 01:53:48.747503 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 251f9b604f97d7027d387853e0788706, offset: 0, size: 0 2024.10.22 01:53:48.748819 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 251f9b604f97d7027d387853e0788706 for removal 2024.10.22 01:53:48.755017 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.53 MiB. 2024.10.22 01:53:48.799534 [ 1476 ] {} CacheMetadata: Key 445d7591deea600b4eb19e1bc67ec1d8 is removed from metadata 2024.10.22 01:53:48.805343 [ 1476 ] {} CacheMetadata: Key 251f9b604f97d7027d387853e0788706 is removed from metadata 2024.10.22 01:53:49.006168 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 690ac04fd9f9b4c56fe023b8a967b1f9, offset: 0, size: 0 2024.10.22 01:53:49.008996 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 690ac04fd9f9b4c56fe023b8a967b1f9 for removal 2024.10.22 01:53:49.011968 [ 1476 ] {} CacheMetadata: Key 690ac04fd9f9b4c56fe023b8a967b1f9 is removed from metadata 2024.10.22 01:53:49.014308 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_10_10_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:53:49.032009 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tmx/epqubnnvscnkcummwiirokrhersxw. 2024.10.22 01:53:49.075422 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c8d65f8435a267d1ee23a161727ab924, offset: 0, size: 0 2024.10.22 01:53:49.077423 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c8d65f8435a267d1ee23a161727ab924 for removal 2024.10.22 01:53:49.089322 [ 1476 ] {} CacheMetadata: Key c8d65f8435a267d1ee23a161727ab924 is removed from metadata 2024.10.22 01:53:49.261452 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ab63d1745910e27630448f8b5fa5c85b, offset: 0, size: 0 2024.10.22 01:53:49.262847 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ab63d1745910e27630448f8b5fa5c85b for removal 2024.10.22 01:53:49.267339 [ 1476 ] {} CacheMetadata: Key ab63d1745910e27630448f8b5fa5c85b is removed from metadata 2024.10.22 01:53:49.329431 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 154613fad93885bd3aadb530b8f0e9f2, offset: 0, size: 0 2024.10.22 01:53:49.336524 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 154613fad93885bd3aadb530b8f0e9f2 for removal 2024.10.22 01:53:49.344494 [ 1476 ] {} CacheMetadata: Key 154613fad93885bd3aadb530b8f0e9f2 is removed from metadata 2024.10.22 01:53:49.455665 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6c81ce8932f8daea67dc6eabbeb1f6ae, offset: 0, size: 0 2024.10.22 01:53:49.457063 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6c81ce8932f8daea67dc6eabbeb1f6ae for removal 2024.10.22 01:53:49.461482 [ 1476 ] {} CacheMetadata: Key 6c81ce8932f8daea67dc6eabbeb1f6ae is removed from metadata 2024.10.22 01:53:49.543836 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3202df41fd9cdf29e925397d2ef19251, offset: 0, size: 0 2024.10.22 01:53:49.549532 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3202df41fd9cdf29e925397d2ef19251 for removal 2024.10.22 01:53:49.554719 [ 1476 ] {} CacheMetadata: Key 3202df41fd9cdf29e925397d2ef19251 is removed from metadata 2024.10.22 01:53:49.601787 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5c53b297ae4cb53f23d4af85e493d145, offset: 0, size: 0 2024.10.22 01:53:49.607116 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5c53b297ae4cb53f23d4af85e493d145 for removal 2024.10.22 01:53:49.612148 [ 1476 ] {} CacheMetadata: Key 5c53b297ae4cb53f23d4af85e493d145 is removed from metadata 2024.10.22 01:53:49.612781 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/biu/gqqykazxexqodovzxjgdggxztsehi. 2024.10.22 01:53:49.671992 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2162dbf687a74f0b1568f3cd43a171ff, offset: 0, size: 0 2024.10.22 01:53:49.674045 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 2162dbf687a74f0b1568f3cd43a171ff for removal 2024.10.22 01:53:49.679129 [ 1476 ] {} CacheMetadata: Key 2162dbf687a74f0b1568f3cd43a171ff is removed from metadata 2024.10.22 01:53:49.733580 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f402c0cb8bc34619dcdd603577452d54, offset: 0, size: 0 2024.10.22 01:53:49.737836 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f402c0cb8bc34619dcdd603577452d54 for removal 2024.10.22 01:53:49.743099 [ 1476 ] {} CacheMetadata: Key f402c0cb8bc34619dcdd603577452d54 is removed from metadata 2024.10.22 01:53:49.754946 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/biu/gqqykazxexqodovzxjgdggxztsehi. 2024.10.22 01:53:50.059463 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a346e1ff991ba59852720a0f2e03d763, offset: 0, size: 0 2024.10.22 01:53:50.063510 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a346e1ff991ba59852720a0f2e03d763 for removal 2024.10.22 01:53:50.069470 [ 1476 ] {} CacheMetadata: Key a346e1ff991ba59852720a0f2e03d763 is removed from metadata 2024.10.22 01:53:50.198711 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8cece9b16c216b34f65fcca240bb5ff7, offset: 0, size: 0 2024.10.22 01:53:50.202643 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8cece9b16c216b34f65fcca240bb5ff7 for removal 2024.10.22 01:53:50.206570 [ 1476 ] {} CacheMetadata: Key 8cece9b16c216b34f65fcca240bb5ff7 is removed from metadata 2024.10.22 01:53:50.470430 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 64ac586e033c77eaae46c70872936e0c, offset: 0, size: 0 2024.10.22 01:53:50.472067 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 64ac586e033c77eaae46c70872936e0c for removal 2024.10.22 01:53:50.477037 [ 1476 ] {} CacheMetadata: Key 64ac586e033c77eaae46c70872936e0c is removed from metadata 2024.10.22 01:53:50.565676 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1c101d4921c05ef4cdaae5d48dbdc76e, offset: 0, size: 0 2024.10.22 01:53:50.569785 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1c101d4921c05ef4cdaae5d48dbdc76e for removal 2024.10.22 01:53:50.582950 [ 1476 ] {} CacheMetadata: Key 1c101d4921c05ef4cdaae5d48dbdc76e is removed from metadata 2024.10.22 01:53:50.572202 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} MergeTask::MergeProjectionsStage: Merge sorted 18329 rows, containing 13 columns (13 merged, 0 gathered) in 3.795244089 sec., 4829.465396738017 rows/sec., 2.09 MiB/sec. 2024.10.22 01:53:50.596600 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 8.68 MiB, will set to 1.71 GiB (RSS), difference: 17.79 MiB 2024.10.22 01:53:50.665724 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_604_118, 202410_605_610_1] 2024.10.22 01:53:50.711382 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 276 entries to flush up to offset 77515 2024.10.22 01:53:50.720192 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_604_118, 202410_605_610_1] 2024.10.22 01:53:50.767720 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} MergeTask::MergeProjectionsStage: Merge sorted 617 rows, containing 32 columns (32 merged, 0 gathered) in 2.822704699 sec., 218.58467880773526 rows/sec., 19.94 KiB/sec. 2024.10.22 01:53:50.835602 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5ad71f4b1be76b8c8e01f5a063d91f21, offset: 0, size: 0 2024.10.22 01:53:50.734598 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_573_117/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:53:50.840772 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5ad71f4b1be76b8c8e01f5a063d91f21 for removal 2024.10.22 01:53:50.846357 [ 1476 ] {} CacheMetadata: Key 5ad71f4b1be76b8c8e01f5a063d91f21 is removed from metadata 2024.10.22 01:53:50.848803 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} MergedBlockOutputStream: filled checksums 202410_1_573_117 (state Temporary) 2024.10.22 01:53:50.899793 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} MergedBlockOutputStream: filled checksums 202410_507_690_40 (state Temporary) 2024.10.22 01:53:50.961466 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:53:50.964802 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:53:51.034737 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_604_118, 202410_605_610_1] 2024.10.22 01:53:51.186356 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_690_40 to 202410_507_690_40 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:51.194304 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_690_40} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_685_39, 202410_690_690_0] -> 202410_507_690_40 2024.10.22 01:53:51.351511 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.44 MiB. 2024.10.22 01:53:51.364386 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_573_117 to 202410_1_573_117 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:51.369470 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_573_117} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_568_116, 202410_573_573_0] -> 202410_1_573_117 2024.10.22 01:53:51.563484 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 33 entries to flush up to offset 10612 2024.10.22 01:53:51.656529 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:53:51.800443 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:51.804938 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:53:51.861436 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_697_697_0 (state Temporary) 2024.10.22 01:53:51.936361 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_697_697_0 to 202410_710_710_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:51.953222 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:51.956187 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:53:52.012377 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10612 2024.10.22 01:53:52.090047 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1175_1175_0 (state Temporary) 2024.10.22 01:53:52.174650 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 1048576 bytes for 1c101d4921c05ef4cdaae5d48dbdc76e:0) 2024.10.22 01:53:52.177056 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 096af66b26a92fdf7f85204b2c2959fd, offset: 0, size: 5042029 2024.10.22 01:53:52.189820 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 096af66b26a92fdf7f85204b2c2959fd for removal 2024.10.22 01:53:52.194554 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9e0246c5c10017e702b550c3be98dbd4, offset: 0, size: 1078 2024.10.22 01:53:52.194993 [ 1476 ] {} CacheMetadata: Key 096af66b26a92fdf7f85204b2c2959fd is removed from metadata 2024.10.22 01:53:52.200552 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9e0246c5c10017e702b550c3be98dbd4 for removal 2024.10.22 01:53:52.208093 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 05e129dfe5bee454fe37897dc6dd3fc2, offset: 0, size: 62 2024.10.22 01:53:52.213295 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 05e129dfe5bee454fe37897dc6dd3fc2 for removal 2024.10.22 01:53:52.224149 [ 1476 ] {} CacheMetadata: Key 9e0246c5c10017e702b550c3be98dbd4 is removed from metadata 2024.10.22 01:53:52.317108 [ 1476 ] {} CacheMetadata: Key 05e129dfe5bee454fe37897dc6dd3fc2 is removed from metadata 2024.10.22 01:53:52.405439 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1175_1175_0 to 202410_1188_1188_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:52.461618 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 77515 2024.10.22 01:53:52.525268 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 10.21 MiB, will set to 1.71 GiB (RSS), difference: 8.87 MiB 2024.10.22 01:53:53.364896 [ 1239 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 222 ms. 2024.10.22 01:53:53.456768 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 1016871 2024.10.22 01:53:53.478691 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1065_1085_4, 202410_1086_1086_0, 202410_1087_1087_0, 202410_1088_1088_0, 202410_1089_1089_0, 202410_1090_1090_0] 2024.10.22 01:53:53.483861 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1065_1085_4, 202410_1086_1086_0, 202410_1087_1087_0, 202410_1088_1088_0, 202410_1089_1089_0, 202410_1090_1090_0] 2024.10.22 01:53:53.754465 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 1c101d4921c05ef4cdaae5d48dbdc76e:0) 2024.10.22 01:53:53.756080 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1065_1085_4, 202410_1086_1086_0, 202410_1087_1087_0, 202410_1088_1088_0, 202410_1089_1089_0, 202410_1090_1090_0] 2024.10.22 01:53:53.756630 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cd8664dde187b099f8377096eb650839, offset: 0, size: 3897085 2024.10.22 01:53:53.761707 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cd8664dde187b099f8377096eb650839 for removal 2024.10.22 01:53:53.764734 [ 1476 ] {} CacheMetadata: Key cd8664dde187b099f8377096eb650839 is removed from metadata 2024.10.22 01:53:53.765092 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1aaf52b081ae86ea82433931dae87d45, offset: 0, size: 1057 2024.10.22 01:53:53.768469 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1aaf52b081ae86ea82433931dae87d45 for removal 2024.10.22 01:53:53.794774 [ 1476 ] {} CacheMetadata: Key 1aaf52b081ae86ea82433931dae87d45 is removed from metadata 2024.10.22 01:53:54.267644 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 167 entries to flush up to offset 77621 2024.10.22 01:53:54.506338 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:54.510617 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:53:54.567584 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:54.571004 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:53:54.585976 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 1c101d4921c05ef4cdaae5d48dbdc76e:0) 2024.10.22 01:53:54.592659 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5bec27c39a4f955b239f9aacfff68355, offset: 0, size: 2113279 2024.10.22 01:53:54.604458 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5bec27c39a4f955b239f9aacfff68355 for removal 2024.10.22 01:53:54.608320 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f6de1428c62b3c0aba427dc0a9321960, offset: 0, size: 1019 2024.10.22 01:53:54.623632 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f6de1428c62b3c0aba427dc0a9321960 for removal 2024.10.22 01:53:54.708569 [ 1476 ] {} CacheMetadata: Key 5bec27c39a4f955b239f9aacfff68355 is removed from metadata 2024.10.22 01:53:54.733029 [ 1476 ] {} CacheMetadata: Key f6de1428c62b3c0aba427dc0a9321960 is removed from metadata 2024.10.22 01:53:54.858061 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_685_685_0 (state Temporary) 2024.10.22 01:53:54.862825 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_668_668_0 (state Temporary) 2024.10.22 01:53:54.970978 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_685_685_0 to 202410_691_691_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:55.059345 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77621 2024.10.22 01:53:55.068233 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_668_668_0 to 202410_675_675_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:55.115638 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1016871 2024.10.22 01:53:55.352458 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_140_140_0 to 202410_143_143_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:55.494601 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7a1e89ec2ea3981ac2e37c84655abe49, offset: 0, size: 0 2024.10.22 01:53:55.504492 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7a1e89ec2ea3981ac2e37c84655abe49 for removal 2024.10.22 01:53:55.512071 [ 1476 ] {} CacheMetadata: Key 7a1e89ec2ea3981ac2e37c84655abe49 is removed from metadata 2024.10.22 01:53:55.534914 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 582 2024.10.22 01:53:55.568258 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9b48810186d7830c1bba0e67810aac50, offset: 0, size: 0 2024.10.22 01:53:55.570549 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9b48810186d7830c1bba0e67810aac50 for removal 2024.10.22 01:53:55.576252 [ 1476 ] {} CacheMetadata: Key 9b48810186d7830c1bba0e67810aac50 is removed from metadata 2024.10.22 01:53:56.034611 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:56.039809 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:53:56.208412 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_564_564_0 (state Temporary) 2024.10.22 01:53:56.433966 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_564_564_0 to 202410_574_574_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:56.536057 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 77601 2024.10.22 01:53:56.549342 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 582 2024.10.22 01:53:56.569146 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 8.26 MiB, will set to 1.71 GiB (RSS), difference: 5.04 MiB 2024.10.22 01:53:56.777142 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 7a1e89ec2ea3981ac2e37c84655abe49:0) 2024.10.22 01:53:56.783716 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9872bbfcdab5b61b9be1b446b980ecf5, offset: 0, size: 2104455 2024.10.22 01:53:56.794123 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9872bbfcdab5b61b9be1b446b980ecf5 for removal 2024.10.22 01:53:56.802131 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 992c1319f4f6dc1c1304ab7e3c6719ad, offset: 0, size: 1019 2024.10.22 01:53:56.801785 [ 1476 ] {} CacheMetadata: Key 9872bbfcdab5b61b9be1b446b980ecf5 is removed from metadata 2024.10.22 01:53:56.809551 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 992c1319f4f6dc1c1304ab7e3c6719ad for removal 2024.10.22 01:53:56.836739 [ 1476 ] {} CacheMetadata: Key 992c1319f4f6dc1c1304ab7e3c6719ad is removed from metadata 2024.10.22 01:53:56.857125 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:56.859154 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:53:56.951388 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1176_1176_0 (state Temporary) 2024.10.22 01:53:57.116475 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1176_1176_0 to 202410_1189_1189_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:53:57.197531 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 77601 2024.10.22 01:53:58.138385 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_670_129/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:53:58.150689 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 7a1e89ec2ea3981ac2e37c84655abe49:0) 2024.10.22 01:53:58.154080 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bb1126d74011b3caa96a5fcb91af0ac9, offset: 0, size: 1763794 2024.10.22 01:53:58.163068 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bb1126d74011b3caa96a5fcb91af0ac9 for removal 2024.10.22 01:53:58.166595 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 40052c0c4c6326d33973a0127f0fea5c, offset: 0, size: 1009 2024.10.22 01:53:58.166922 [ 1476 ] {} CacheMetadata: Key bb1126d74011b3caa96a5fcb91af0ac9 is removed from metadata 2024.10.22 01:53:58.169671 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 40052c0c4c6326d33973a0127f0fea5c for removal 2024.10.22 01:53:58.198321 [ 1476 ] {} CacheMetadata: Key 40052c0c4c6326d33973a0127f0fea5c is removed from metadata 2024.10.22 01:53:58.496441 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5618 2024.10.22 01:53:58.596847 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 16.07 MiB, will set to 1.71 GiB (RSS), difference: 7.09 MiB 2024.10.22 01:53:59.119162 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 7a1e89ec2ea3981ac2e37c84655abe49:0) 2024.10.22 01:53:59.249575 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f661ce116c3ef0f653e2199ac45c8023, offset: 0, size: 4286889 2024.10.22 01:53:59.255781 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f661ce116c3ef0f653e2199ac45c8023 for removal 2024.10.22 01:53:59.265586 [ 1476 ] {} CacheMetadata: Key f661ce116c3ef0f653e2199ac45c8023 is removed from metadata 2024.10.22 01:53:59.267369 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: ec2c387540ea42a8b9a168305fe0118f, offset: 0, size: 1064 2024.10.22 01:53:59.277421 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key ec2c387540ea42a8b9a168305fe0118f for removal 2024.10.22 01:53:59.414931 [ 1476 ] {} CacheMetadata: Key ec2c387540ea42a8b9a168305fe0118f is removed from metadata 2024.10.22 01:53:59.434567 [ 1262 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:53:59.477552 [ 1262 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:53:59.516477 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 10619 2024.10.22 01:53:59.661349 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:53:59.672115 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:53:59.732098 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_698_698_0 (state Temporary) 2024.10.22 01:53:59.969228 [ 1246 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 269 ms. 2024.10.22 01:54:00.107804 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_698_698_0 to 202410_711_711_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:00.139828 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 5 old parts to remove. Parts: [202410_125_125_0, 202410_126_126_0, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0] 2024.10.22 01:54:00.145164 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from filesystem (serially): Parts: [202410_125_125_0, 202410_126_126_0, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0] 2024.10.22 01:54:00.178179 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10619 2024.10.22 01:54:00.272307 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from memory: Parts: [202410_125_125_0, 202410_126_126_0, 202410_127_127_0, 202410_128_128_0, 202410_129_129_0] 2024.10.22 01:54:00.962400 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:54:00.964584 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:54:01.081617 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 7a1e89ec2ea3981ac2e37c84655abe49:0) 2024.10.22 01:54:01.093131 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 911feed3b8162db7c90be37a15c082ef, offset: 0, size: 2823753 2024.10.22 01:54:01.099915 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 911feed3b8162db7c90be37a15c082ef for removal 2024.10.22 01:54:01.102156 [ 1476 ] {} CacheMetadata: Key 911feed3b8162db7c90be37a15c082ef is removed from metadata 2024.10.22 01:54:01.103718 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f4616837cd539e26231db9bcb41b5d1f, offset: 0, size: 1035 2024.10.22 01:54:01.119900 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f4616837cd539e26231db9bcb41b5d1f for removal 2024.10.22 01:54:01.133155 [ 1476 ] {} CacheMetadata: Key f4616837cd539e26231db9bcb41b5d1f is removed from metadata 2024.10.22 01:54:01.217450 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 77654 2024.10.22 01:54:01.471737 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:01.475208 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:01.663182 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1177_1177_0 (state Temporary) 2024.10.22 01:54:01.735661 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 17.14 MiB, will set to 1.72 GiB (RSS), difference: 2.28 MiB 2024.10.22 01:54:01.751166 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1177_1177_0 to 202410_1190_1190_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:01.829468 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 77654 2024.10.22 01:54:02.122628 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1018026 2024.10.22 01:54:02.534621 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 7a1e89ec2ea3981ac2e37c84655abe49:0) 2024.10.22 01:54:02.540367 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1c101d4921c05ef4cdaae5d48dbdc76e, offset: 0, size: 10783113 2024.10.22 01:54:02.555189 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1c101d4921c05ef4cdaae5d48dbdc76e for removal 2024.10.22 01:54:02.562431 [ 1476 ] {} CacheMetadata: Key 1c101d4921c05ef4cdaae5d48dbdc76e is removed from metadata 2024.10.22 01:54:02.562301 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5ad71f4b1be76b8c8e01f5a063d91f21, offset: 0, size: 1165 2024.10.22 01:54:02.567024 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5ad71f4b1be76b8c8e01f5a063d91f21 for removal 2024.10.22 01:54:02.570212 [ 1476 ] {} CacheMetadata: Key 5ad71f4b1be76b8c8e01f5a063d91f21 is removed from metadata 2024.10.22 01:54:02.571429 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cdd9b8c40abad66bfb176dd564485d1e, offset: 0, size: 14892 2024.10.22 01:54:02.582906 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cdd9b8c40abad66bfb176dd564485d1e for removal 2024.10.22 01:54:02.589165 [ 1476 ] {} CacheMetadata: Key cdd9b8c40abad66bfb176dd564485d1e is removed from metadata 2024.10.22 01:54:02.589377 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5f60a37f716d174e21031e8f04581537, offset: 0, size: 564 2024.10.22 01:54:02.597292 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5f60a37f716d174e21031e8f04581537 for removal 2024.10.22 01:54:02.602560 [ 1476 ] {} CacheMetadata: Key 5f60a37f716d174e21031e8f04581537 is removed from metadata 2024.10.22 01:54:02.669389 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 50 entries to flush up to offset 77671 2024.10.22 01:54:02.930825 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:02.936706 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:02.938999 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:02.950308 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:03.002635 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_686_686_0 (state Temporary) 2024.10.22 01:54:03.197219 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_669_669_0 (state Temporary) 2024.10.22 01:54:03.201160 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_686_686_0 to 202410_692_692_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:03.281794 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_669_669_0 to 202410_676_676_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:03.375731 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77671 2024.10.22 01:54:03.331840 [ 1161 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_671_671_0 to 202410_676_676_0 2024.10.22 01:54:03.398998 [ 1161 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:03.403358 [ 1161 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:03.448458 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1018026 2024.10.22 01:54:03.542282 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_671_671_0 to 202410_676_676_0 into Compact with storage Full 2024.10.22 01:54:03.558904 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 10.73 MiB, will set to 1.71 GiB (RSS), difference: 2.75 MiB 2024.10.22 01:54:03.566538 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:03.604963 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_671_671_0, total 1151 rows starting from the beginning of the part 2024.10.22 01:54:03.648995 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cc4db1d2a1591be8f3e7dc8034a39d80, offset: 0, size: 0 2024.10.22 01:54:03.651833 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cc4db1d2a1591be8f3e7dc8034a39d80 for removal 2024.10.22 01:54:03.655101 [ 1476 ] {} CacheMetadata: Key cc4db1d2a1591be8f3e7dc8034a39d80 is removed from metadata 2024.10.22 01:54:03.753063 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_672_672_0, total 1159 rows starting from the beginning of the part 2024.10.22 01:54:03.787627 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_673_673_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:54:03.835742 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_674_674_0, total 1542 rows starting from the beginning of the part 2024.10.22 01:54:03.904425 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 11ac2ceff08607d4f3796da186944bd1, offset: 0, size: 0 2024.10.22 01:54:03.907537 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 11ac2ceff08607d4f3796da186944bd1 for removal 2024.10.22 01:54:03.906103 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_675_675_0, total 1542 rows starting from the beginning of the part 2024.10.22 01:54:03.913336 [ 1476 ] {} CacheMetadata: Key 11ac2ceff08607d4f3796da186944bd1 is removed from metadata 2024.10.22 01:54:03.913640 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mkg/iluwosumfsvhoiksypotnedgkhvos. 2024.10.22 01:54:03.965794 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: cd6ed14af0d662ee5b49c477bacd60e3, offset: 0, size: 0 2024.10.22 01:54:03.967309 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key cd6ed14af0d662ee5b49c477bacd60e3 for removal 2024.10.22 01:54:03.970667 [ 1476 ] {} CacheMetadata: Key cd6ed14af0d662ee5b49c477bacd60e3 is removed from metadata 2024.10.22 01:54:04.052694 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 583 2024.10.22 01:54:04.083477 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_676_676_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:54:04.165299 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 909b754bf66ea0121f6f27f20f2892dd, offset: 0, size: 0 2024.10.22 01:54:04.168173 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 909b754bf66ea0121f6f27f20f2892dd for removal 2024.10.22 01:54:04.170878 [ 1476 ] {} CacheMetadata: Key 909b754bf66ea0121f6f27f20f2892dd is removed from metadata 2024.10.22 01:54:04.291829 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mkg/iluwosumfsvhoiksypotnedgkhvos. 2024.10.22 01:54:04.357743 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3ade1a7f0513bcc06d476aa3a6ebb91e, offset: 0, size: 0 2024.10.22 01:54:04.364063 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 3ade1a7f0513bcc06d476aa3a6ebb91e for removal 2024.10.22 01:54:04.369266 [ 1476 ] {} CacheMetadata: Key 3ade1a7f0513bcc06d476aa3a6ebb91e is removed from metadata 2024.10.22 01:54:04.630627 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 54e237e7a32da1b5ad6730a5d387e597, offset: 0, size: 0 2024.10.22 01:54:04.632468 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 54e237e7a32da1b5ad6730a5d387e597 for removal 2024.10.22 01:54:04.638952 [ 1476 ] {} CacheMetadata: Key 54e237e7a32da1b5ad6730a5d387e597 is removed from metadata 2024.10.22 01:54:04.741478 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tma/wlupgdgdndcldnnuyhjxozucjismi. 2024.10.22 01:54:04.761121 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:04.770482 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:04.801491 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 359a9fa0bae72c3968e01eb229b1cafe, offset: 0, size: 0 2024.10.22 01:54:04.803155 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 359a9fa0bae72c3968e01eb229b1cafe for removal 2024.10.22 01:54:04.812143 [ 1476 ] {} CacheMetadata: Key 359a9fa0bae72c3968e01eb229b1cafe is removed from metadata 2024.10.22 01:54:04.893976 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/adi/mcubbdjldgrcrxxpurttqvneqffqf. 2024.10.22 01:54:04.957746 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_565_565_0 (state Temporary) 2024.10.22 01:54:05.061055 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8a69e637afbbb47ca5633b9920b70228, offset: 0, size: 0 2024.10.22 01:54:05.062809 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 8a69e637afbbb47ca5633b9920b70228 for removal 2024.10.22 01:54:05.075563 [ 1476 ] {} CacheMetadata: Key 8a69e637afbbb47ca5633b9920b70228 is removed from metadata 2024.10.22 01:54:05.085689 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tma/wlupgdgdndcldnnuyhjxozucjismi. 2024.10.22 01:54:05.144002 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_565_565_0 to 202410_575_575_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:05.335651 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 583 2024.10.22 01:54:05.341033 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e13e685bb12daa7b83395ac1df147309, offset: 0, size: 0 2024.10.22 01:54:05.342571 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e13e685bb12daa7b83395ac1df147309 for removal 2024.10.22 01:54:05.346834 [ 1476 ] {} CacheMetadata: Key e13e685bb12daa7b83395ac1df147309 is removed from metadata 2024.10.22 01:54:05.390683 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c0b28b5daa1960e8c812fa910849368b, offset: 0, size: 0 2024.10.22 01:54:05.398416 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c0b28b5daa1960e8c812fa910849368b for removal 2024.10.22 01:54:05.402988 [ 1476 ] {} CacheMetadata: Key c0b28b5daa1960e8c812fa910849368b is removed from metadata 2024.10.22 01:54:05.404719 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/okp/nwjglxnyuxqauatdjnozezdyeedyf. 2024.10.22 01:54:05.451306 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a2d7b9ea8238d25bf9d85e2dcacae618, offset: 0, size: 0 2024.10.22 01:54:05.452752 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a2d7b9ea8238d25bf9d85e2dcacae618 for removal 2024.10.22 01:54:05.455007 [ 1476 ] {} CacheMetadata: Key a2d7b9ea8238d25bf9d85e2dcacae618 is removed from metadata 2024.10.22 01:54:05.467345 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 12.43 MiB, will set to 1.71 GiB (RSS), difference: -2.41 MiB 2024.10.22 01:54:05.577715 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} MergeTask::MergeProjectionsStage: Merge sorted 8089 rows, containing 4 columns (4 merged, 0 gathered) in 2.046466092 sec., 3952.6674942826266 rows/sec., 61.76 KiB/sec. 2024.10.22 01:54:05.608447 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d0c2327c69787f7dd816fbd6eb762d86, offset: 0, size: 0 2024.10.22 01:54:05.612605 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d0c2327c69787f7dd816fbd6eb762d86 for removal 2024.10.22 01:54:05.620527 [ 1476 ] {} CacheMetadata: Key d0c2327c69787f7dd816fbd6eb762d86 is removed from metadata 2024.10.22 01:54:05.639334 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/okp/nwjglxnyuxqauatdjnozezdyeedyf. 2024.10.22 01:54:05.678955 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b2e2e1fbe747535270ac1605b11a2494, offset: 0, size: 0 2024.10.22 01:54:05.681992 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b2e2e1fbe747535270ac1605b11a2494 for removal 2024.10.22 01:54:05.689504 [ 1476 ] {} CacheMetadata: Key b2e2e1fbe747535270ac1605b11a2494 is removed from metadata 2024.10.22 01:54:05.829759 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d9d9babbbdb07bb8fd64102dba1b7e0b, offset: 0, size: 0 2024.10.22 01:54:05.831740 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d9d9babbbdb07bb8fd64102dba1b7e0b for removal 2024.10.22 01:54:05.840898 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/khd/liuioroaepxffhfugtxcyumqtmgqr. 2024.10.22 01:54:05.845695 [ 1476 ] {} CacheMetadata: Key d9d9babbbdb07bb8fd64102dba1b7e0b is removed from metadata 2024.10.22 01:54:05.860811 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} MergedBlockOutputStream: filled checksums 202410_671_676_1 (state Temporary) 2024.10.22 01:54:05.942656 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 103 entries to flush up to offset 77757 2024.10.22 01:54:05.955151 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_671_676_1 to 202410_671_676_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:05.958850 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_671_676_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_671_671_0, 202410_676_676_0] -> 202410_671_676_1 2024.10.22 01:54:05.968939 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: aabec3f9c8de384080c9e7cdd6c3167c, offset: 0, size: 0 2024.10.22 01:54:05.970728 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key aabec3f9c8de384080c9e7cdd6c3167c for removal 2024.10.22 01:54:05.975377 [ 1476 ] {} CacheMetadata: Key aabec3f9c8de384080c9e7cdd6c3167c is removed from metadata 2024.10.22 01:54:06.049113 [ 1341 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/keh/vcgpoerlhsbmzampgyokafnnoesrk, size 30642 2024.10.22 01:54:06.049236 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 47 entries to flush up to offset 5665 2024.10.22 01:54:06.049593 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: a25792b99fe457685aed186a4128d5bc, offset: 0, size: 0 2024.10.22 01:54:06.051909 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key a25792b99fe457685aed186a4128d5bc for removal 2024.10.22 01:54:06.055307 [ 1476 ] {} CacheMetadata: Key a25792b99fe457685aed186a4128d5bc is removed from metadata 2024.10.22 01:54:06.089056 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:54:06.096261 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:54:06.103983 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:54:06.150693 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/khd/liuioroaepxffhfugtxcyumqtmgqr. 2024.10.22 01:54:06.247677 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:54:06.271224 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:54:06.292265 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OS 2024.10.22 01:54:06.301718 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OS 2024.10.22 01:54:06.405368 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OS 2024.10.22 01:54:06.406224 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:54:06.441745 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7236a0d09a2079eb47b879ae385fca51, offset: 0, size: 0 2024.10.22 01:54:06.443421 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7236a0d09a2079eb47b879ae385fca51 for removal 2024.10.22 01:54:06.448130 [ 1476 ] {} CacheMetadata: Key 7236a0d09a2079eb47b879ae385fca51 is removed from metadata 2024.10.22 01:54:06.544030 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:54:06.588118 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:54:06.630490 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: fde4cb8e9165de7e384ca85d7107e283, offset: 0, size: 0 2024.10.22 01:54:06.632158 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key fde4cb8e9165de7e384ca85d7107e283 for removal 2024.10.22 01:54:06.636574 [ 1476 ] {} CacheMetadata: Key fde4cb8e9165de7e384ca85d7107e283 is removed from metadata 2024.10.22 01:54:06.637169 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hag/kdrgaugrmlrikyuuedjlbwuwlcxfv. 2024.10.22 01:54:06.643488 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OS 2024.10.22 01:54:06.664521 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OS 2024.10.22 01:54:06.666837 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 01:54:06.671006 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OS 2024.10.22 01:54:06.684615 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 478045f753180b6c9272e4b5eccc2cf0, offset: 0, size: 0 2024.10.22 01:54:06.692398 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 478045f753180b6c9272e4b5eccc2cf0 for removal 2024.10.22 01:54:06.696297 [ 1476 ] {} CacheMetadata: Key 478045f753180b6c9272e4b5eccc2cf0 is removed from metadata 2024.10.22 01:54:06.699602 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OS 2024.10.22 01:54:06.709535 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:06.716096 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:06.815948 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1178_1178_0 (state Temporary) 2024.10.22 01:54:06.862622 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lys/iuyviuklpewsbkgghpwfhrocllrmh (LogSeriesLimiter: on interval from 2024-10-22 01:53:47 to 2024-10-22 01:54:06 accepted series 1 / 12 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:54:06.963322 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 00a847497a02870c8b66f7da01000a21, offset: 0, size: 0 2024.10.22 01:54:06.964733 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 00a847497a02870c8b66f7da01000a21 for removal 2024.10.22 01:54:06.966817 [ 1476 ] {} CacheMetadata: Key 00a847497a02870c8b66f7da01000a21 is removed from metadata 2024.10.22 01:54:06.984731 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hag/kdrgaugrmlrikyuuedjlbwuwlcxfv. 2024.10.22 01:54:07.016458 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1178_1178_0 to 202410_1191_1191_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:07.048471 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: daaa2c16eba68fae3ee1d9a10691af38, offset: 0, size: 0 2024.10.22 01:54:07.059633 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1186_18 to 202410_1191_1191_0 2024.10.22 01:54:07.062088 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key daaa2c16eba68fae3ee1d9a10691af38 for removal 2024.10.22 01:54:07.064590 [ 1476 ] {} CacheMetadata: Key daaa2c16eba68fae3ee1d9a10691af38 is removed from metadata 2024.10.22 01:54:07.067466 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:07.069389 [ 1179 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:07.104239 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c2561e208be47209f949d1d8d2cdadb9, offset: 0, size: 0 2024.10.22 01:54:07.110771 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c2561e208be47209f949d1d8d2cdadb9 for removal 2024.10.22 01:54:07.116815 [ 1476 ] {} CacheMetadata: Key c2561e208be47209f949d1d8d2cdadb9 is removed from metadata 2024.10.22 01:54:07.130958 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 77757 2024.10.22 01:54:07.148075 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1186_18 to 202410_1191_1191_0 into Compact with storage Full 2024.10.22 01:54:07.172502 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7f975de0ec48b82d05b6ec618bc05161, offset: 0, size: 0 2024.10.22 01:54:07.175685 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7f975de0ec48b82d05b6ec618bc05161 for removal 2024.10.22 01:54:07.184889 [ 1476 ] {} CacheMetadata: Key 7f975de0ec48b82d05b6ec618bc05161 is removed from metadata 2024.10.22 01:54:07.196759 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:07.246409 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1186_18, total 6561 rows starting from the beginning of the part 2024.10.22 01:54:07.249824 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 24bbf8d55b6a8be1a7012948ca8ce490, offset: 0, size: 0 2024.10.22 01:54:07.252886 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 24bbf8d55b6a8be1a7012948ca8ce490 for removal 2024.10.22 01:54:07.254991 [ 1476 ] {} CacheMetadata: Key 24bbf8d55b6a8be1a7012948ca8ce490 is removed from metadata 2024.10.22 01:54:07.266231 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1187_1187_0, total 102 rows starting from the beginning of the part 2024.10.22 01:54:07.304457 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1188_1188_0, total 276 rows starting from the beginning of the part 2024.10.22 01:54:07.344603 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1189_1189_0, total 86 rows starting from the beginning of the part 2024.10.22 01:54:07.351369 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f9ff7001240f777c41707629ff5bdba8, offset: 0, size: 0 2024.10.22 01:54:07.353011 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f9ff7001240f777c41707629ff5bdba8 for removal 2024.10.22 01:54:07.357266 [ 1476 ] {} CacheMetadata: Key f9ff7001240f777c41707629ff5bdba8 is removed from metadata 2024.10.22 01:54:07.368300 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1190_1190_0, total 53 rows starting from the beginning of the part 2024.10.22 01:54:07.399446 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5a145ecd3a0da188a755f56a8fbecd5e, offset: 0, size: 0 2024.10.22 01:54:07.402249 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 5a145ecd3a0da188a755f56a8fbecd5e for removal 2024.10.22 01:54:07.404200 [ 1476 ] {} CacheMetadata: Key 5a145ecd3a0da188a755f56a8fbecd5e is removed from metadata 2024.10.22 01:54:07.405428 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vou/xyqjheuaeinxicuyzcqlxvlxywpre. 2024.10.22 01:54:07.499421 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1191_1191_0, total 103 rows starting from the beginning of the part 2024.10.22 01:54:07.564998 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: f4e452570b794a53ee6b8be333f49350, offset: 0, size: 0 2024.10.22 01:54:07.566304 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key f4e452570b794a53ee6b8be333f49350 for removal 2024.10.22 01:54:07.571551 [ 1476 ] {} CacheMetadata: Key f4e452570b794a53ee6b8be333f49350 is removed from metadata 2024.10.22 01:54:07.683304 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10632 2024.10.22 01:54:07.702326 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 14.82 MiB, will set to 1.71 GiB (RSS), difference: 2.74 MiB 2024.10.22 01:54:07.829449 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} S3ObjectStorage: Objects with paths [common/hre/yjjusmtzaqlwotxgpsutmuybooqcy] were removed from S3 2024.10.22 01:54:07.998553 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d203db6d0b2f905981bc40a24462a4ea, offset: 0, size: 0 2024.10.22 01:54:08.000266 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d203db6d0b2f905981bc40a24462a4ea for removal 2024.10.22 01:54:08.039114 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:08.042815 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:08.089358 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_699_699_0 (state Temporary) 2024.10.22 01:54:08.135858 [ 1476 ] {} CacheMetadata: Key d203db6d0b2f905981bc40a24462a4ea is removed from metadata 2024.10.22 01:54:08.243271 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} S3ObjectStorage: Objects with paths [common/emp/hkgedubjecoijnzpkyddvvuvdhxfp] were removed from S3 2024.10.22 01:54:08.258429 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vou/xyqjheuaeinxicuyzcqlxvlxywpre. 2024.10.22 01:54:08.378728 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_699_699_0 to 202410_712_712_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:08.419227 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7b4446735b388ca4372d503d11361e6e, offset: 0, size: 0 2024.10.22 01:54:08.421031 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7b4446735b388ca4372d503d11361e6e for removal 2024.10.22 01:54:08.433140 [ 1476 ] {} CacheMetadata: Key 7b4446735b388ca4372d503d11361e6e is removed from metadata 2024.10.22 01:54:08.467277 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2a87de3ddf831cbdf1e9a4ecc9f113f2, offset: 0, size: 0 2024.10.22 01:54:08.471674 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 2a87de3ddf831cbdf1e9a4ecc9f113f2 for removal 2024.10.22 01:54:08.472481 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10632 2024.10.22 01:54:08.476185 [ 1476 ] {} CacheMetadata: Key 2a87de3ddf831cbdf1e9a4ecc9f113f2 is removed from metadata 2024.10.22 01:54:08.477589 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/iic/fiicfoalcbvrevszrvdywqrkbvbsp. 2024.10.22 01:54:08.514854 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1bbc813bfa14aef875ea361a39c79c07, offset: 0, size: 0 2024.10.22 01:54:08.520741 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 1bbc813bfa14aef875ea361a39c79c07 for removal 2024.10.22 01:54:08.535726 [ 1476 ] {} CacheMetadata: Key 1bbc813bfa14aef875ea361a39c79c07 is removed from metadata 2024.10.22 01:54:08.595851 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} S3ObjectStorage: Objects with paths [common/vou/xyqjheuaeinxicuyzcqlxvlxywpre] were removed from S3 2024.10.22 01:54:08.658267 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: b58ee33423676924c64024b62c2784ec, offset: 0, size: 0 2024.10.22 01:54:08.659644 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key b58ee33423676924c64024b62c2784ec for removal 2024.10.22 01:54:08.725173 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} MergeTask::MergeProjectionsStage: Merge sorted 7181 rows, containing 13 columns (13 merged, 0 gathered) in 1.588252505 sec., 4521.3213751550165 rows/sec., 771.75 KiB/sec. 2024.10.22 01:54:08.661828 [ 1476 ] {} CacheMetadata: Key b58ee33423676924c64024b62c2784ec is removed from metadata 2024.10.22 01:54:08.808749 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} S3ObjectStorage: Objects with paths [common/iix/juqakyoksqfxioazockezppyoiipz] were removed from S3 2024.10.22 01:54:08.854869 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} MergedBlockOutputStream: filled checksums 202410_1096_1191_19 (state Temporary) 2024.10.22 01:54:08.861205 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/iic/fiicfoalcbvrevszrvdywqrkbvbsp. 2024.10.22 01:54:08.911955 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d76382f47141f8009ebb2b56b3d1cb97, offset: 0, size: 0 2024.10.22 01:54:08.914569 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d76382f47141f8009ebb2b56b3d1cb97 for removal 2024.10.22 01:54:08.918500 [ 1476 ] {} CacheMetadata: Key d76382f47141f8009ebb2b56b3d1cb97 is removed from metadata 2024.10.22 01:54:09.163907 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0a4ec5e55f068d0e7234e68078b70bcf, offset: 0, size: 0 2024.10.22 01:54:09.166808 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0a4ec5e55f068d0e7234e68078b70bcf for removal 2024.10.22 01:54:09.170133 [ 1476 ] {} CacheMetadata: Key 0a4ec5e55f068d0e7234e68078b70bcf is removed from metadata 2024.10.22 01:54:09.171960 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ynt/vbleqtivtilovurhqcteasusnskos. 2024.10.22 01:54:09.174680 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1191_19 to 202410_1096_1191_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:09.182260 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1191_19} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1186_18, 202410_1191_1191_0] -> 202410_1096_1191_19 2024.10.22 01:54:09.256811 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.88 MiB. 2024.10.22 01:54:09.348033 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: fd10ec934336ab619e2b4ed765b54ccf, offset: 0, size: 0 2024.10.22 01:54:09.350427 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key fd10ec934336ab619e2b4ed765b54ccf for removal 2024.10.22 01:54:09.352951 [ 1476 ] {} CacheMetadata: Key fd10ec934336ab619e2b4ed765b54ccf is removed from metadata 2024.10.22 01:54:09.483270 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} S3ObjectStorage: Objects with paths [common/iic/fiicfoalcbvrevszrvdywqrkbvbsp] were removed from S3 2024.10.22 01:54:09.655269 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d29abadcfadce91f6d5e55863794ff67, offset: 0, size: 0 2024.10.22 01:54:09.660845 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d29abadcfadce91f6d5e55863794ff67 for removal 2024.10.22 01:54:09.664123 [ 1476 ] {} CacheMetadata: Key d29abadcfadce91f6d5e55863794ff67 is removed from metadata 2024.10.22 01:54:09.697316 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} S3ObjectStorage: Objects with paths [common/yxb/nwgochxmfigzsepjbjqjcyqmorcla] were removed from S3 2024.10.22 01:54:09.723848 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ynt/vbleqtivtilovurhqcteasusnskos. 2024.10.22 01:54:09.727500 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 12.99 MiB, will set to 1.71 GiB (RSS), difference: 11.42 MiB 2024.10.22 01:54:09.797193 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: abe684d6add0544bf6e7a37491548466, offset: 0, size: 0 2024.10.22 01:54:09.801213 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key abe684d6add0544bf6e7a37491548466 for removal 2024.10.22 01:54:09.805316 [ 1476 ] {} CacheMetadata: Key abe684d6add0544bf6e7a37491548466 is removed from metadata 2024.10.22 01:54:09.861344 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c8ed86266229d54cbd631ae550614774, offset: 0, size: 0 2024.10.22 01:54:09.863990 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c8ed86266229d54cbd631ae550614774 for removal 2024.10.22 01:54:09.868048 [ 1476 ] {} CacheMetadata: Key c8ed86266229d54cbd631ae550614774 is removed from metadata 2024.10.22 01:54:09.868759 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zjo/rukdlgjhenxlgngklgjmaoxzcgdwj. 2024.10.22 01:54:09.869776 [ 1192 ] {} DatabaseCatalog: Execution took 286 ms. 2024.10.22 01:54:09.944711 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: bd17c4101ac8d377adbd1d23ae20c571, offset: 0, size: 0 2024.10.22 01:54:09.947262 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key bd17c4101ac8d377adbd1d23ae20c571 for removal 2024.10.22 01:54:09.948795 [ 1476 ] {} CacheMetadata: Key bd17c4101ac8d377adbd1d23ae20c571 is removed from metadata 2024.10.22 01:54:09.954746 [ 1169 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 349 ms. 2024.10.22 01:54:09.955809 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_670_129/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:54:10.010278 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} S3ObjectStorage: Objects with paths [common/ynt/vbleqtivtilovurhqcteasusnskos] were removed from S3 2024.10.22 01:54:10.274509 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0baeeed16f892dcdbeb02bc91654ed30, offset: 0, size: 0 2024.10.22 01:54:10.279616 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 0baeeed16f892dcdbeb02bc91654ed30 for removal 2024.10.22 01:54:10.388165 [ 1476 ] {} CacheMetadata: Key 0baeeed16f892dcdbeb02bc91654ed30 is removed from metadata 2024.10.22 01:54:10.434674 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} S3ObjectStorage: Objects with paths [common/pvl/stffsggefgnkgvhanawdigfjftvdq] were removed from S3 2024.10.22 01:54:10.452468 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1019951 2024.10.22 01:54:10.454428 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zjo/rukdlgjhenxlgngklgjmaoxzcgdwj. 2024.10.22 01:54:10.623438 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e328962a7afb51f68d37327dd3977064, offset: 0, size: 0 2024.10.22 01:54:10.628165 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e328962a7afb51f68d37327dd3977064 for removal 2024.10.22 01:54:10.633123 [ 1476 ] {} CacheMetadata: Key e328962a7afb51f68d37327dd3977064 is removed from metadata 2024.10.22 01:54:10.697211 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: c621b105e1bfe5d0d53a105f45d9218c, offset: 0, size: 0 2024.10.22 01:54:10.699686 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key c621b105e1bfe5d0d53a105f45d9218c for removal 2024.10.22 01:54:10.704966 [ 1476 ] {} CacheMetadata: Key c621b105e1bfe5d0d53a105f45d9218c is removed from metadata 2024.10.22 01:54:10.865215 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6c316594caa4b39be26eec4430300c88, offset: 0, size: 0 2024.10.22 01:54:10.870879 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 6c316594caa4b39be26eec4430300c88 for removal 2024.10.22 01:54:10.879931 [ 1476 ] {} CacheMetadata: Key 6c316594caa4b39be26eec4430300c88 is removed from metadata 2024.10.22 01:54:10.888392 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 77771 2024.10.22 01:54:10.964698 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:54:10.966759 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:54:11.039779 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} S3ObjectStorage: Objects with paths [common/zjo/rukdlgjhenxlgngklgjmaoxzcgdwj] were removed from S3 2024.10.22 01:54:11.102602 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7b0953db17f56bba0ab72eec1d8bafd1, offset: 0, size: 0 2024.10.22 01:54:11.105160 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7b0953db17f56bba0ab72eec1d8bafd1 for removal 2024.10.22 01:54:11.110389 [ 1476 ] {} CacheMetadata: Key 7b0953db17f56bba0ab72eec1d8bafd1 is removed from metadata 2024.10.22 01:54:11.139415 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:11.154659 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} S3ObjectStorage: Objects with paths [common/lag/exnjjzptoxcblkgvpalauakmocslm] were removed from S3 2024.10.22 01:54:11.333882 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 168 entries to flush up to offset 77925 2024.10.22 01:54:11.337595 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:11.359282 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rde/xhxafbpndakfuqxycfjroqcrnydje. 2024.10.22 01:54:11.367214 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kcb/zvsyzrepgjbsoznffixqbvjenrujt. 2024.10.22 01:54:11.422424 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 578fc983dd02ab04e81ded808383c7b1, offset: 0, size: 0 2024.10.22 01:54:11.424704 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 578fc983dd02ab04e81ded808383c7b1 for removal 2024.10.22 01:54:11.435118 [ 1476 ] {} CacheMetadata: Key 578fc983dd02ab04e81ded808383c7b1 is removed from metadata 2024.10.22 01:54:11.450764 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_687_687_0 (state Temporary) 2024.10.22 01:54:11.475626 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:11.480299 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 44d79e42a2d008a9048f8ddf8d64c292, offset: 0, size: 0 2024.10.22 01:54:11.487234 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 44d79e42a2d008a9048f8ddf8d64c292 for removal 2024.10.22 01:54:11.493409 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:11.497099 [ 1476 ] {} CacheMetadata: Key 44d79e42a2d008a9048f8ddf8d64c292 is removed from metadata 2024.10.22 01:54:11.504712 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kcb/zvsyzrepgjbsoznffixqbvjenrujt. 2024.10.22 01:54:11.512669 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xin/wsdldbbdqwilmbchhuerajnxhwfvf. 2024.10.22 01:54:11.570373 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 20934e241bcc0d7681213b7d9d36c603, offset: 0, size: 0 2024.10.22 01:54:11.572184 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 20934e241bcc0d7681213b7d9d36c603 for removal 2024.10.22 01:54:11.576989 [ 1476 ] {} CacheMetadata: Key 20934e241bcc0d7681213b7d9d36c603 is removed from metadata 2024.10.22 01:54:11.636833 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_687_687_0 to 202410_693_693_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:11.701836 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77771 2024.10.22 01:54:11.742180 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: be06943bf52eb7ed9c6b90e7f3f08a68, offset: 0, size: 0 2024.10.22 01:54:11.746981 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key be06943bf52eb7ed9c6b90e7f3f08a68 for removal 2024.10.22 01:54:11.751848 [ 1476 ] {} CacheMetadata: Key be06943bf52eb7ed9c6b90e7f3f08a68 is removed from metadata 2024.10.22 01:54:11.755897 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xin/wsdldbbdqwilmbchhuerajnxhwfvf. 2024.10.22 01:54:11.772009 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_670_670_0 (state Temporary) 2024.10.22 01:54:11.963503 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_670_670_0 to 202410_677_677_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:12.115977 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1019951 2024.10.22 01:54:12.222435 [ 1247 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 208 ms. 2024.10.22 01:54:12.473310 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:12.477924 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:12.840352 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 584 2024.10.22 01:54:12.884795 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1179_1179_0 (state Temporary) 2024.10.22 01:54:13.062232 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1179_1179_0 to 202410_1192_1192_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:13.165669 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:13.173748 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:13.245544 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_566_566_0 (state Temporary) 2024.10.22 01:54:13.337064 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_566_566_0 to 202410_576_576_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:13.407493 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yqv/imgmzhuzcgeazrlvznnriarefncka. 2024.10.22 01:54:13.430505 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 77925 2024.10.22 01:54:13.473179 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: e954ecf97cb2f046b32958161971ccc8, offset: 0, size: 0 2024.10.22 01:54:13.477463 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key e954ecf97cb2f046b32958161971ccc8 for removal 2024.10.22 01:54:13.482421 [ 1476 ] {} CacheMetadata: Key e954ecf97cb2f046b32958161971ccc8 is removed from metadata 2024.10.22 01:54:13.550844 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 584 2024.10.22 01:54:13.646855 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d4206866d86c947920808089216c3224, offset: 0, size: 0 2024.10.22 01:54:13.649424 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d4206866d86c947920808089216c3224 for removal 2024.10.22 01:54:13.652621 [ 1476 ] {} CacheMetadata: Key d4206866d86c947920808089216c3224 is removed from metadata 2024.10.22 01:54:13.658916 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yqv/imgmzhuzcgeazrlvznnriarefncka. 2024.10.22 01:54:13.957477 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 14.17 MiB, will set to 1.71 GiB (RSS), difference: 11.09 MiB 2024.10.22 01:54:14.438357 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_10_10_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:54:14.487722 [ 1209 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:54:14.531804 [ 1209 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:54:14.582121 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for e954ecf97cb2f046b32958161971ccc8:0) 2024.10.22 01:54:14.588780 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7a1e89ec2ea3981ac2e37c84655abe49, offset: 0, size: 13393684 2024.10.22 01:54:14.618719 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 7a1e89ec2ea3981ac2e37c84655abe49 for removal 2024.10.22 01:54:14.625508 [ 1476 ] {} CacheMetadata: Key 7a1e89ec2ea3981ac2e37c84655abe49 is removed from metadata 2024.10.22 01:54:14.627179 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9b48810186d7830c1bba0e67810aac50, offset: 0, size: 1208 2024.10.22 01:54:14.635241 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key 9b48810186d7830c1bba0e67810aac50 for removal 2024.10.22 01:54:14.810158 [ 1476 ] {} CacheMetadata: Key 9b48810186d7830c1bba0e67810aac50 is removed from metadata 2024.10.22 01:54:15.459488 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergeTask::MergeProjectionsStage: Merge sorted 3342336 rows, containing 133 columns (3 merged, 130 gathered) in 3518.059372896 sec., 950.0510496639664 rows/sec., 693.63 KiB/sec. 2024.10.22 01:54:15.977058 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 10644 2024.10.22 01:54:16.079454 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} MergedBlockOutputStream: filled checksums 201403_8_16_1 (state Temporary) 2024.10.22 01:54:16.248147 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uye/urbnbpyfcjykuwakeeubhfzslychg (LogSeriesLimiter: on interval from 2024-10-22 01:54:06 to 2024-10-22 01:54:16 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:54:16.437538 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:16.449420 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1065_1090_5, 202410_1091_1091_0, 202410_1092_1092_0, 202410_1093_1093_0, 202410_1094_1094_0, 202410_1095_1095_0] 2024.10.22 01:54:16.451454 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1065_1090_5, 202410_1091_1091_0, 202410_1092_1092_0, 202410_1093_1093_0, 202410_1094_1094_0, 202410_1095_1095_0] 2024.10.22 01:54:16.476649 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1065_1090_5, 202410_1091_1091_0, 202410_1092_1092_0, 202410_1093_1093_0, 202410_1094_1094_0, 202410_1095_1095_0] 2024.10.22 01:54:16.501575 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:16.511282 [ 1741 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uye/urbnbpyfcjykuwakeeubhfzslychg, size 4 2024.10.22 01:54:16.826259 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_700_700_0 (state Temporary) 2024.10.22 01:54:16.874599 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_700_700_0 to 202410_713_713_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:16.925364 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 14.12 MiB, will set to 1.71 GiB (RSS), difference: 7.48 MiB 2024.10.22 01:54:16.951941 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10644 2024.10.22 01:54:17.465288 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 77999 2024.10.22 01:54:17.872259 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:17.876109 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:18.070236 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1180_1180_0 (state Temporary) 2024.10.22 01:54:18.125818 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1180_1180_0 to 202410_1193_1193_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:18.204318 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 77999 2024.10.22 01:54:19.131428 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1021106 2024.10.22 01:54:19.237678 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 55 entries to flush up to offset 77826 2024.10.22 01:54:19.423375 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Remove from cache. Key: d9a81ae8138ca5ce745534dda577ab1c, offset: 0, size: 0 2024.10.22 01:54:19.426859 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} CacheMetadata: Submitting key d9a81ae8138ca5ce745534dda577ab1c for removal 2024.10.22 01:54:19.438767 [ 1476 ] {} CacheMetadata: Key d9a81ae8138ca5ce745534dda577ab1c is removed from metadata 2024.10.22 01:54:19.451942 [ 1275 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 256 ms. 2024.10.22 01:54:19.595366 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:19.601108 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:19.655512 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_688_688_0 (state Temporary) 2024.10.22 01:54:19.716668 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_688_688_0 to 202410_694_694_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:19.821287 [ 1276 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 324 ms. 2024.10.22 01:54:19.850383 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 4 old parts to remove. Parts: [202410_507_630_25, 202410_631_631_0, 202410_632_632_0, 202410_633_633_0] 2024.10.22 01:54:19.854688 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from filesystem (serially): Parts: [202410_507_630_25, 202410_631_631_0, 202410_632_632_0, 202410_633_633_0] 2024.10.22 01:54:19.886352 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from memory: Parts: [202410_507_630_25, 202410_631_631_0, 202410_632_632_0, 202410_633_633_0] 2024.10.22 01:54:20.015526 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:20.019987 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77826 2024.10.22 01:54:20.024134 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:20.061159 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_670_129} MergeTask::MergeProjectionsStage: Merge sorted 1023527 rows, containing 4 columns (4 merged, 0 gathered) in 33.722500212 sec., 30351.456551723368 rows/sec., 455.03 KiB/sec. 2024.10.22 01:54:20.149587 [ 1226 ] {} DatabaseCatalog: Execution took 275 ms. 2024.10.22 01:54:20.171623 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_670_129} MergedBlockOutputStream: filled checksums 202410_1_670_129 (state Temporary) 2024.10.22 01:54:20.298980 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_671_671_0 (state Temporary) 2024.10.22 01:54:20.572071 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_671_671_0 to 202410_678_678_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:20.659883 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1021106 2024.10.22 01:54:20.965746 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:54:21.037210 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:54:21.300592 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_670_129/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:54:21.466913 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 13.49 MiB, will set to 1.70 GiB (RSS), difference: 6.54 MiB 2024.10.22 01:54:22.216837 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 78032 2024.10.22 01:54:22.474608 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:22.482336 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 01:54:22.631152 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1181_1181_0 (state Temporary) 2024.10.22 01:54:22.672090 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1181_1181_0 to 202410_1194_1194_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:22.850659 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 78032 2024.10.22 01:54:22.981126 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uye/urbnbpyfcjykuwakeeubhfzslychg. 2024.10.22 01:54:23.145147 [ 1679 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lys/iuyviuklpewsbkgghpwfhrocllrmh, size 679840 2024.10.22 01:54:23.548349 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:54:23.562840 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:54:23.587249 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:54:23.618268 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:54:23.656715 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:54:23.702144 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:54:23.839892 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UserAgent 2024.10.22 01:54:23.881683 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/svi/vxbuenrohahrfmtaorypktyjitkma (LogSeriesLimiter: on interval from 2024-10-22 01:54:16 to 2024-10-22 01:54:23 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:54:23.970291 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uye/urbnbpyfcjykuwakeeubhfzslychg. 2024.10.22 01:54:23.982516 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_merge_201403_8_16_1 to 201403_8_16_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:23.993003 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_8_16_1} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937) (MergerMutator): Merged 9 parts: [201403_8_8_0, 201403_16_16_0] -> 201403_8_16_1 2024.10.22 01:54:24.258550 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 349.99 MiB. 2024.10.22 01:54:24.455544 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 10659 2024.10.22 01:54:24.504453 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_670_129} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_670_129 to 202410_1_670_129 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:24.542636 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_670_129} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_664_128, 202410_665_670_1] -> 202410_1_670_129 2024.10.22 01:54:24.571469 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:54:24.683717 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:24.694535 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:25.075413 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_701_701_0 (state Temporary) 2024.10.22 01:54:25.250363 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_701_701_0 to 202410_714_714_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:25.282701 [ 1203 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_709_144 to 202410_710_710_0 2024.10.22 01:54:25.289512 [ 1203 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:25.293145 [ 1203 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:25.318803 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_710_145} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_709_144 to 202410_710_710_0 into Compact with storage Full 2024.10.22 01:54:25.333539 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:54:25.354584 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:54:25.370230 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_710_145} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:25.375410 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:54:25.364917 [ 1154 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 292 ms. 2024.10.22 01:54:25.382482 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_710_145} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_709_144, total 10685 rows starting from the beginning of the part 2024.10.22 01:54:25.448842 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:54:25.467954 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:54:25.429837 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10659 2024.10.22 01:54:25.547975 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_710_145} MergeTreeSequentialSource: Reading 2 marks from part 202410_710_710_0, total 33 rows starting from the beginning of the part 2024.10.22 01:54:25.606672 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_523_107, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0, 202410_527_527_0, 202410_528_528_0] 2024.10.22 01:54:25.630966 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:54:25.612286 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_523_107, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0, 202410_527_527_0, 202410_528_528_0] 2024.10.22 01:54:25.752734 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 41 2024.10.22 01:54:25.860061 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:54:25.891565 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 01:54:25.907571 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 8.39 MiB, will set to 1.70 GiB (RSS), difference: 59.32 MiB 2024.10.22 01:54:25.967534 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_523_107, 202410_524_524_0, 202410_525_525_0, 202410_526_526_0, 202410_527_527_0, 202410_528_528_0] 2024.10.22 01:54:26.035806 [ 1161 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 01:54:26.042299 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_710_145/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:54:26.159921 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:26.162880 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:26.234932 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_35_35_0 (state Temporary) 2024.10.22 01:54:26.265897 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:26.271152 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:26.332251 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_35_35_0 to 202410_35_35_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:26.409070 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 41 2024.10.22 01:54:26.536312 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/adi/mcubbdjldgrcrxxpurttqvneqffqf. 2024.10.22 01:54:26.858756 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 78092 2024.10.22 01:54:27.083086 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:27.091643 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:27.251859 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_710_145} MergeTask::MergeProjectionsStage: Merge sorted 10718 rows, containing 12 columns (12 merged, 0 gathered) in 1.946146804 sec., 5507.292655400317 rows/sec., 877.05 KiB/sec. 2024.10.22 01:54:27.283665 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1182_1182_0 (state Temporary) 2024.10.22 01:54:27.419893 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_710_145} MergedBlockOutputStream: filled checksums 202410_1_710_145 (state Temporary) 2024.10.22 01:54:27.446936 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1182_1182_0 to 202410_1195_1195_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:27.534973 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 75 entries to flush up to offset 77901 2024.10.22 01:54:27.550533 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rde/xhxafbpndakfuqxycfjroqcrnydje. 2024.10.22 01:54:27.608103 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 78092 2024.10.22 01:54:27.666136 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1022646 2024.10.22 01:54:27.686920 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 6.83 MiB, will set to 1.69 GiB (RSS), difference: 63.95 MiB 2024.10.22 01:54:27.792079 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:27.805285 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:27.889260 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_689_689_0 (state Temporary) 2024.10.22 01:54:27.997338 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_689_689_0 to 202410_695_695_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:28.143903 [ 1251 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_690_40 to 202410_691_691_0 2024.10.22 01:54:28.151271 [ 1251 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.09 MiB using storage policy from min volume index 0 2024.10.22 01:54:28.155218 [ 1251 ] {} DiskLocal: Reserved 1.09 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:28.174134 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_691_41} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_690_40 to 202410_691_691_0 into Compact with storage Full 2024.10.22 01:54:28.204260 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77901 2024.10.22 01:54:28.252325 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_710_145} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_710_145 to 202410_1_710_145 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:28.254018 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_691_41} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:28.255301 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_710_145} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_709_144, 202410_710_710_0] -> 202410_1_710_145 2024.10.22 01:54:28.276854 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_691_41} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_690_40, total 18329 rows starting from the beginning of the part 2024.10.22 01:54:28.301514 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_141_141_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:54:28.367558 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_691_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_691_691_0, total 167 rows starting from the beginning of the part 2024.10.22 01:54:28.387198 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:28.394916 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:28.397270 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 01:54:28.503658 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_672_672_0 (state Temporary) 2024.10.22 01:54:28.553739 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 585 2024.10.22 01:54:28.583763 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_650_130, 202410_651_651_0, 202410_652_652_0, 202410_653_653_0, 202410_654_654_0, 202410_655_655_0] 2024.10.22 01:54:28.588478 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_650_130, 202410_651_651_0, 202410_652_652_0, 202410_653_653_0, 202410_654_654_0, 202410_655_655_0] 2024.10.22 01:54:28.638127 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_650_130, 202410_651_651_0, 202410_652_652_0, 202410_653_653_0, 202410_654_654_0, 202410_655_655_0] 2024.10.22 01:54:28.651652 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_617_617_0, 202410_618_618_0, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0, 202410_622_622_0] 2024.10.22 01:54:28.655194 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_617_617_0, 202410_618_618_0, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0, 202410_622_622_0] 2024.10.22 01:54:28.728190 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_672_672_0 to 202410_679_679_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:28.847296 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_617_617_0, 202410_618_618_0, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0, 202410_622_622_0] 2024.10.22 01:54:28.876621 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1022646 2024.10.22 01:54:28.935188 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:28.943699 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:29.049590 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_567_567_0 (state Temporary) 2024.10.22 01:54:29.246400 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_567_567_0 to 202410_577_577_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:29.318779 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 585 2024.10.22 01:54:29.559695 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_141_141_0 (state Temporary) 2024.10.22 01:54:29.641339 [ 1157 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:54:29.664194 [ 1157 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:54:29.918293 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 1.97 GiB, free memory in arenas 7.82 MiB, will set to 1.69 GiB (RSS), difference: 43.80 MiB 2024.10.22 01:54:30.475191 [ 1180 ] {} DatabaseCatalog: Execution took 327 ms. 2024.10.22 01:54:30.525270 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_691_41} MergeTask::MergeProjectionsStage: Merge sorted 18496 rows, containing 13 columns (13 merged, 0 gathered) in 2.348167884 sec., 7876.779222656296 rows/sec., 3.41 MiB/sec. 2024.10.22 01:54:30.577394 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_691_41} MergedBlockOutputStream: filled checksums 202410_507_691_41 (state Temporary) 2024.10.22 01:54:30.965327 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:54:31.070749 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:54:31.242008 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_691_41} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_691_41 to 202410_507_691_41 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:31.258780 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_691_41} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_690_40, 202410_691_691_0] -> 202410_507_691_41 2024.10.22 01:54:31.314561 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.70 MiB. 2024.10.22 01:54:31.859735 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 78149 2024.10.22 01:54:32.206425 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:32.208820 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:32.256120 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1183_1183_0 (state Temporary) 2024.10.22 01:54:32.313829 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1183_1183_0 to 202410_1196_1196_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:32.351172 [ 1258 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1191_19 to 202410_1196_1196_0 2024.10.22 01:54:32.369815 [ 1258 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:32.374729 [ 1258 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:32.411463 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1191_19 to 202410_1196_1196_0 into Compact with storage Full 2024.10.22 01:54:32.589152 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:32.669604 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 78149 2024.10.22 01:54:32.731700 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1191_19, total 7181 rows starting from the beginning of the part 2024.10.22 01:54:32.712459 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 1.97 GiB, free memory in arenas 5.67 MiB, will set to 1.68 GiB (RSS), difference: 5.96 MiB 2024.10.22 01:54:32.917104 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1192_1192_0, total 168 rows starting from the beginning of the part 2024.10.22 01:54:33.010831 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1193_1193_0, total 74 rows starting from the beginning of the part 2024.10.22 01:54:33.033599 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 10675 2024.10.22 01:54:33.310728 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1194_1194_0, total 33 rows starting from the beginning of the part 2024.10.22 01:54:33.463260 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1195_1195_0, total 60 rows starting from the beginning of the part 2024.10.22 01:54:33.560885 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1196_1196_0, total 57 rows starting from the beginning of the part 2024.10.22 01:54:33.563432 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:33.566288 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:33.606166 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_702_702_0 (state Temporary) 2024.10.22 01:54:33.654771 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_702_702_0 to 202410_715_715_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:33.679049 [ 1247 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_710_145 to 202410_715_715_0 2024.10.22 01:54:33.696078 [ 1247 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:33.700118 [ 1247 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:33.750604 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_710_145 to 202410_715_715_0 into Compact with storage Full 2024.10.22 01:54:33.814250 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:33.817720 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10675 2024.10.22 01:54:33.949676 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_710_145, total 10718 rows starting from the beginning of the part 2024.10.22 01:54:33.974458 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_711_711_0, total 7 rows starting from the beginning of the part 2024.10.22 01:54:34.119790 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_712_712_0, total 13 rows starting from the beginning of the part 2024.10.22 01:54:34.148701 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_713_713_0, total 12 rows starting from the beginning of the part 2024.10.22 01:54:34.176408 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_714_714_0, total 15 rows starting from the beginning of the part 2024.10.22 01:54:34.310947 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_715_715_0, total 16 rows starting from the beginning of the part 2024.10.22 01:54:34.464964 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 5.27 MiB, will set to 1.68 GiB (RSS), difference: -9.77 MiB 2024.10.22 01:54:35.468459 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} MergeTask::MergeProjectionsStage: Merge sorted 7573 rows, containing 13 columns (13 merged, 0 gathered) in 3.056443243 sec., 2477.716547606116 rows/sec., 421.66 KiB/sec. 2024.10.22 01:54:35.721720 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 92 entries to flush up to offset 77993 2024.10.22 01:54:35.751213 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} MergedBlockOutputStream: filled checksums 202410_1096_1196_20 (state Temporary) 2024.10.22 01:54:35.889730 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1024186 2024.10.22 01:54:35.899268 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} MergeTask::MergeProjectionsStage: Merge sorted 10781 rows, containing 12 columns (12 merged, 0 gathered) in 2.154417482 sec., 5004.136890864684 rows/sec., 796.84 KiB/sec. 2024.10.22 01:54:35.955166 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} MergedBlockOutputStream: filled checksums 202410_1_715_146 (state Temporary) 2024.10.22 01:54:36.006877 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1196_20 to 202410_1096_1196_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:36.013310 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1196_20} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1191_19, 202410_1196_1196_0] -> 202410_1096_1196_20 2024.10.22 01:54:36.164334 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.94 MiB. 2024.10.22 01:54:36.183362 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:36.186532 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:36.242078 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_690_690_0 (state Temporary) 2024.10.22 01:54:36.290956 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_715_146 to 202410_1_715_146 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:36.296115 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_715_146} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_710_145, 202410_715_715_0] -> 202410_1_715_146 2024.10.22 01:54:36.331896 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.53 MiB. 2024.10.22 01:54:36.507428 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 4.73 MiB, will set to 1.68 GiB (RSS), difference: 11.05 MiB 2024.10.22 01:54:36.517153 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_690_690_0 to 202410_696_696_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:36.632522 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for a6882731fa7ce17c413a4aec19d1585e:0) 2024.10.22 01:54:36.635858 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 20934e241bcc0d7681213b7d9d36c603, offset: 0, size: 5862448 2024.10.22 01:54:36.645677 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 20934e241bcc0d7681213b7d9d36c603 for removal 2024.10.22 01:54:36.649766 [ 1476 ] {} CacheMetadata: Key 20934e241bcc0d7681213b7d9d36c603 is removed from metadata 2024.10.22 01:54:36.649752 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: be06943bf52eb7ed9c6b90e7f3f08a68, offset: 0, size: 1098 2024.10.22 01:54:36.653337 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key be06943bf52eb7ed9c6b90e7f3f08a68 for removal 2024.10.22 01:54:36.670339 [ 1476 ] {} CacheMetadata: Key be06943bf52eb7ed9c6b90e7f3f08a68 is removed from metadata 2024.10.22 01:54:36.670627 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 77993 2024.10.22 01:54:36.748826 [ 1239 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_691_41 to 202410_692_692_0 2024.10.22 01:54:36.799032 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 78209 2024.10.22 01:54:36.808870 [ 1239 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.09 MiB using storage policy from min volume index 0 2024.10.22 01:54:36.815403 [ 1239 ] {} DiskLocal: Reserved 1.09 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:36.824560 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 586 2024.10.22 01:54:36.824637 [ 1239 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 200 ms. 2024.10.22 01:54:36.870111 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:36.877114 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:36.963818 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_692_42} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_691_41 to 202410_692_692_0 into Compact with storage Full 2024.10.22 01:54:37.188335 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_673_673_0 (state Temporary) 2024.10.22 01:54:37.206329 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_692_42} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:37.261161 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_692_42} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_691_41, total 18496 rows starting from the beginning of the part 2024.10.22 01:54:37.262109 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:37.268318 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:37.275328 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_673_673_0 to 202410_680_680_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:37.297589 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:37.301455 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:37.328812 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1024186 2024.10.22 01:54:37.422590 [ 1233 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_670_129 to 202410_671_676_1 2024.10.22 01:54:37.429451 [ 1233 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:37.436265 [ 1233 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:37.548247 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1184_1184_0 (state Temporary) 2024.10.22 01:54:37.556359 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_692_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_692_692_0, total 50 rows starting from the beginning of the part 2024.10.22 01:54:37.563917 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_676_130} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_670_129 to 202410_671_676_1 into Wide with storage Full 2024.10.22 01:54:37.600047 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1184_1184_0 to 202410_1197_1197_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:37.708874 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_568_568_0 (state Temporary) 2024.10.22 01:54:37.731895 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_676_130} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:37.745525 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_676_130} MergeTreeSequentialSource: Reading 126 marks from part 202410_1_670_129, total 1023527 rows starting from the beginning of the part 2024.10.22 01:54:37.867386 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 78209 2024.10.22 01:54:37.891565 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_676_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_671_676_1, total 8089 rows starting from the beginning of the part 2024.10.22 01:54:38.019018 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_568_568_0 to 202410_578_578_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:38.061816 [ 1256 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_573_117 to 202410_578_578_0 2024.10.22 01:54:38.071486 [ 1256 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:38.075529 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:38.114440 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 586 2024.10.22 01:54:38.164904 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_573_117 to 202410_578_578_0 into Compact with storage Full 2024.10.22 01:54:38.253253 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:38.283770 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_573_117, total 617 rows starting from the beginning of the part 2024.10.22 01:54:38.365516 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_692_42/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:54:38.468015 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_574_574_0, total 1 rows starting from the beginning of the part 2024.10.22 01:54:38.514809 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 4.98 MiB, will set to 1.69 GiB (RSS), difference: -24.81 MiB 2024.10.22 01:54:38.574483 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_575_575_0, total 1 rows starting from the beginning of the part 2024.10.22 01:54:38.871688 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_576_576_0, total 1 rows starting from the beginning of the part 2024.10.22 01:54:38.924939 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_692_42} MergeTask::MergeProjectionsStage: Merge sorted 18546 rows, containing 13 columns (13 merged, 0 gathered) in 1.991192799 sec., 9314.015202000537 rows/sec., 4.03 MiB/sec. 2024.10.22 01:54:38.970278 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_577_577_0, total 1 rows starting from the beginning of the part 2024.10.22 01:54:39.112949 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_692_42} MergedBlockOutputStream: filled checksums 202410_507_692_42 (state Temporary) 2024.10.22 01:54:39.173102 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_578_578_0, total 1 rows starting from the beginning of the part 2024.10.22 01:54:39.534000 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_141_141_0 to 202410_144_144_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:39.659808 [ 1277 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 7 parts from 202410_1_134_24 to 202410_144_144_0 2024.10.22 01:54:39.785614 [ 1277 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 2.77 MiB using storage policy from min volume index 0 2024.10.22 01:54:39.790539 [ 1277 ] {} DiskLocal: Reserved 2.77 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:39.830886 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} MergeTask::PrepareStage: Merging 7 parts: from 202410_1_134_24 to 202410_144_144_0 into Compact with storage Full 2024.10.22 01:54:40.062665 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_692_42} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_692_42 to 202410_507_692_42 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:40.066712 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_692_42} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_691_41, 202410_692_692_0] -> 202410_507_692_42 2024.10.22 01:54:40.113632 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.64 MiB. 2024.10.22 01:54:40.337650 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 1.97 GiB, free memory in arenas 8.61 MiB, will set to 1.69 GiB (RSS), difference: 18.69 MiB 2024.10.22 01:54:40.520208 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.057791695 sec. 2024.10.22 01:54:40.965743 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:54:40.967021 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:54:41.128347 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} MergeTask::MergeProjectionsStage: Merge sorted 622 rows, containing 32 columns (32 merged, 0 gathered) in 3.042217497 sec., 204.45612472263022 rows/sec., 18.65 KiB/sec. 2024.10.22 01:54:41.160312 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} MergedBlockOutputStream: filled checksums 202410_1_578_118 (state Temporary) 2024.10.22 01:54:41.319874 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 10698 2024.10.22 01:54:41.500334 [ 1709 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/svi/vxbuenrohahrfmtaorypktyjitkma, size 809752 2024.10.22 01:54:41.524030 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:41.535124 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:41.581159 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_703_703_0 (state Temporary) 2024.10.22 01:54:41.640866 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column URL 2024.10.22 01:54:41.644291 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_703_703_0 to 202410_716_716_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:41.646269 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_578_118 to 202410_1_578_118 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:41.650462 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column URL 2024.10.22 01:54:41.654064 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_578_118} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_573_117, 202410_578_578_0] -> 202410_1_578_118 2024.10.22 01:54:41.672048 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column URL 2024.10.22 01:54:41.701285 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column URL 2024.10.22 01:54:41.702058 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:54:41.710661 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10698 2024.10.22 01:54:41.736056 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column URL 2024.10.22 01:54:41.757619 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column URL 2024.10.22 01:54:41.759205 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5665 2024.10.22 01:54:41.796408 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column URL 2024.10.22 01:54:41.875087 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 78289 2024.10.22 01:54:41.972853 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yzg/lybmtyaukyezpjwhkeryghzsrauqc (LogSeriesLimiter: on interval from 2024-10-22 01:54:23 to 2024-10-22 01:54:41 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:54:42.062983 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:42.066573 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:42.241810 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1185_1185_0 (state Temporary) 2024.10.22 01:54:42.302415 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1185_1185_0 to 202410_1198_1198_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:42.361580 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 78289 2024.10.22 01:54:42.537409 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 1.97 GiB, free memory in arenas 13.86 MiB, will set to 1.68 GiB (RSS), difference: -8.80 MiB 2024.10.22 01:54:42.809001 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_610_119, 202410_611_616_1] 2024.10.22 01:54:42.812226 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_610_119, 202410_611_616_1] 2024.10.22 01:54:42.843751 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_676_130/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:54:42.940165 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_610_119, 202410_611_616_1] 2024.10.22 01:54:43.084904 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_633_26, 202410_634_634_0, 202410_635_635_0, 202410_636_636_0, 202410_637_637_0, 202410_638_638_0] 2024.10.22 01:54:43.090528 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_633_26, 202410_634_634_0, 202410_635_635_0, 202410_636_636_0, 202410_637_637_0, 202410_638_638_0] 2024.10.22 01:54:43.128916 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_633_26, 202410_634_634_0, 202410_635_635_0, 202410_636_636_0, 202410_637_637_0, 202410_638_638_0] 2024.10.22 01:54:43.631205 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1_1064_504, 202410_1065_1095_6] 2024.10.22 01:54:43.644545 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1_1064_504, 202410_1065_1095_6] 2024.10.22 01:54:43.748557 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 923031 bytes for a89b593e7d300c0ac159376ad10f5dd0:0) 2024.10.22 01:54:43.751038 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e954ecf97cb2f046b32958161971ccc8, offset: 0, size: 1096558 2024.10.22 01:54:43.756413 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e954ecf97cb2f046b32958161971ccc8 for removal 2024.10.22 01:54:43.759594 [ 1476 ] {} CacheMetadata: Key e954ecf97cb2f046b32958161971ccc8 is removed from metadata 2024.10.22 01:54:43.760777 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d4206866d86c947920808089216c3224, offset: 0, size: 609 2024.10.22 01:54:43.770988 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d4206866d86c947920808089216c3224 for removal 2024.10.22 01:54:43.774046 [ 1476 ] {} CacheMetadata: Key d4206866d86c947920808089216c3224 is removed from metadata 2024.10.22 01:54:43.776240 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1_1064_504, 202410_1065_1095_6] 2024.10.22 01:54:43.990882 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:54:44.018920 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:54:44.031340 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:54:44.140686 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:54:44.158947 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:54:44.270420 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:54:44.277227 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:54:44.302198 [ 1400 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 01:54:44.306002 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 120 entries to flush up to offset 78113 2024.10.22 01:54:44.344492 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1025726 2024.10.22 01:54:44.541068 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:44.543385 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:44.544150 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 16.78 MiB, will set to 1.71 GiB (RSS), difference: -4.86 MiB 2024.10.22 01:54:44.572486 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_691_691_0 (state Temporary) 2024.10.22 01:54:44.612106 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_691_691_0 to 202410_697_697_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:44.654659 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 78113 2024.10.22 01:54:44.657235 [ 1278 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_692_42 to 202410_693_693_0 2024.10.22 01:54:44.667695 [ 1159 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:54:44.682495 [ 1159 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:54:44.764945 [ 1278 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.10 MiB using storage policy from min volume index 0 2024.10.22 01:54:44.800939 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 14 file segments (while reserving 1048576 bytes for e8e6c8f81a5292584a98e2ac3638fab6:0) 2024.10.22 01:54:44.804879 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d9a81ae8138ca5ce745534dda577ab1c, offset: 0, size: 4125906 2024.10.22 01:54:44.812166 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d9a81ae8138ca5ce745534dda577ab1c for removal 2024.10.22 01:54:44.818622 [ 1476 ] {} CacheMetadata: Key d9a81ae8138ca5ce745534dda577ab1c is removed from metadata 2024.10.22 01:54:44.821451 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d18283b25bc6a7cd8b5035c70960ff33, offset: 0, size: 294022 2024.10.22 01:54:44.831175 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d18283b25bc6a7cd8b5035c70960ff33 for removal 2024.10.22 01:54:44.835972 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c891629664cf8442487a11062083d798, offset: 0, size: 7 2024.10.22 01:54:44.838948 [ 1476 ] {} CacheMetadata: Key d18283b25bc6a7cd8b5035c70960ff33 is removed from metadata 2024.10.22 01:54:44.841573 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c891629664cf8442487a11062083d798 for removal 2024.10.22 01:54:44.846118 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: aa07fa6844270120d3edc6a9b876cc6f, offset: 0, size: 16411 2024.10.22 01:54:44.846452 [ 1476 ] {} CacheMetadata: Key c891629664cf8442487a11062083d798 is removed from metadata 2024.10.22 01:54:44.848569 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key aa07fa6844270120d3edc6a9b876cc6f for removal 2024.10.22 01:54:44.852792 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 31728113065c5f352e4ff7f0f090454e, offset: 0, size: 969 2024.10.22 01:54:44.855389 [ 1476 ] {} CacheMetadata: Key aa07fa6844270120d3edc6a9b876cc6f is removed from metadata 2024.10.22 01:54:44.857407 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 31728113065c5f352e4ff7f0f090454e for removal 2024.10.22 01:54:44.861478 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 578ec765761972542aba02373ecc75d2, offset: 0, size: 395831 2024.10.22 01:54:44.863779 [ 1476 ] {} CacheMetadata: Key 31728113065c5f352e4ff7f0f090454e is removed from metadata 2024.10.22 01:54:44.865631 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 578ec765761972542aba02373ecc75d2 for removal 2024.10.22 01:54:44.867396 [ 1278 ] {} DiskLocal: Reserved 1.10 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 01:54:44.869665 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2df8b659fe5d8a06762a97224219d45f, offset: 0, size: 10 2024.10.22 01:54:44.872597 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 2df8b659fe5d8a06762a97224219d45f for removal 2024.10.22 01:54:44.876509 [ 1278 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 01:54:44.877071 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4c524b5b1e8ac6273ca50e0cd59b8601, offset: 0, size: 4 2024.10.22 01:54:44.879460 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 4c524b5b1e8ac6273ca50e0cd59b8601 for removal 2024.10.22 01:54:44.883904 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: b322436de0a250fe332359a5a7b2a9a5, offset: 0, size: 739 2024.10.22 01:54:44.892273 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key b322436de0a250fe332359a5a7b2a9a5 for removal 2024.10.22 01:54:44.898694 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 82cf18e6b34c4a46b81f8edbc3673498, offset: 0, size: 1 2024.10.22 01:54:44.900668 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_693_43} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_692_42 to 202410_693_693_0 into Compact with storage Full 2024.10.22 01:54:44.901569 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 82cf18e6b34c4a46b81f8edbc3673498 for removal 2024.10.22 01:54:44.904236 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a3b58fdb0e69b90c01bb85f4414c433a, offset: 0, size: 9726 2024.10.22 01:54:44.907904 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a3b58fdb0e69b90c01bb85f4414c433a for removal 2024.10.22 01:54:44.909301 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 646192fbe0f569d16eb451dc9055a875, offset: 0, size: 1029 2024.10.22 01:54:44.953713 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_693_43} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:44.967009 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_693_43} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_692_42, total 18546 rows starting from the beginning of the part 2024.10.22 01:54:44.973231 [ 1476 ] {} CacheMetadata: Key a3b58fdb0e69b90c01bb85f4414c433a is removed from metadata 2024.10.22 01:54:44.980734 [ 1476 ] {} CacheMetadata: Key 82cf18e6b34c4a46b81f8edbc3673498 is removed from metadata 2024.10.22 01:54:44.999108 [ 1476 ] {} CacheMetadata: Key b322436de0a250fe332359a5a7b2a9a5 is removed from metadata 2024.10.22 01:54:45.004681 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_693_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_693_693_0, total 100 rows starting from the beginning of the part 2024.10.22 01:54:45.012347 [ 1476 ] {} CacheMetadata: Key 4c524b5b1e8ac6273ca50e0cd59b8601 is removed from metadata 2024.10.22 01:54:45.013058 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 646192fbe0f569d16eb451dc9055a875 for removal 2024.10.22 01:54:45.017145 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1be62c9029e8cc6cf6f705eca75a0373, offset: 0, size: 3034 2024.10.22 01:54:45.023163 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 1be62c9029e8cc6cf6f705eca75a0373 for removal 2024.10.22 01:54:45.029232 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7575e67911ca0d0779bceb0c9649f706, offset: 0, size: 3320 2024.10.22 01:54:45.029486 [ 1476 ] {} CacheMetadata: Key 1be62c9029e8cc6cf6f705eca75a0373 is removed from metadata 2024.10.22 01:54:45.036857 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 7575e67911ca0d0779bceb0c9649f706 for removal 2024.10.22 01:54:45.148476 [ 1476 ] {} CacheMetadata: Key 7575e67911ca0d0779bceb0c9649f706 is removed from metadata 2024.10.22 01:54:45.174186 [ 1476 ] {} CacheMetadata: Key 646192fbe0f569d16eb451dc9055a875 is removed from metadata 2024.10.22 01:54:45.192322 [ 1476 ] {} CacheMetadata: Key 2df8b659fe5d8a06762a97224219d45f is removed from metadata 2024.10.22 01:54:45.216560 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:45.222820 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:45.298328 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1096_0, 202410_1097_1097_0, 202410_1098_1098_0, 202410_1099_1099_0, 202410_1100_1100_0, 202410_1101_1101_0] 2024.10.22 01:54:45.301548 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1096_0, 202410_1097_1097_0, 202410_1098_1098_0, 202410_1099_1099_0, 202410_1100_1100_0, 202410_1101_1101_0] 2024.10.22 01:54:45.311730 [ 1476 ] {} CacheMetadata: Key 578ec765761972542aba02373ecc75d2 is removed from metadata 2024.10.22 01:54:45.343187 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1096_0, 202410_1097_1097_0, 202410_1098_1098_0, 202410_1099_1099_0, 202410_1100_1100_0, 202410_1101_1101_0] 2024.10.22 01:54:45.620707 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 587 2024.10.22 01:54:45.655040 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_674_674_0 (state Temporary) 2024.10.22 01:54:45.850739 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_674_674_0 to 202410_681_681_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:45.990812 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1025726 2024.10.22 01:54:46.144172 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:46.146620 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:46.200724 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_693_43} MergeTask::MergeProjectionsStage: Merge sorted 18646 rows, containing 13 columns (13 merged, 0 gathered) in 1.307571029 sec., 14260.028393455634 rows/sec., 6.17 MiB/sec. 2024.10.22 01:54:46.271522 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_693_43} MergedBlockOutputStream: filled checksums 202410_507_693_43 (state Temporary) 2024.10.22 01:54:46.400501 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_569_569_0 (state Temporary) 2024.10.22 01:54:46.566373 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_693_43} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_693_43 to 202410_507_693_43 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:46.633837 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 17.47 MiB, will set to 1.72 GiB (RSS), difference: -7.70 MiB 2024.10.22 01:54:46.723096 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 107 entries to flush up to offset 78396 2024.10.22 01:54:46.730185 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_693_43} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_692_42, 202410_693_693_0] -> 202410_507_693_43 2024.10.22 01:54:46.783565 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.83 MiB. 2024.10.22 01:54:46.816991 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_569_569_0 to 202410_579_579_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:47.009347 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 587 2024.10.22 01:54:47.273752 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:47.277531 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:47.391549 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1186_1186_0 (state Temporary) 2024.10.22 01:54:47.470156 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1186_1186_0 to 202410_1199_1199_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:47.531466 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 78396 2024.10.22 01:54:48.159860 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 99887 bytes for b5e13cebef581fffe9cf6c38bd327f51:0) 2024.10.22 01:54:48.163128 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b394d006019ed453c0092ff4eecabecd, offset: 0, size: 2303438 2024.10.22 01:54:48.171225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b394d006019ed453c0092ff4eecabecd for removal 2024.10.22 01:54:48.178790 [ 1476 ] {} CacheMetadata: Key b394d006019ed453c0092ff4eecabecd is removed from metadata 2024.10.22 01:54:48.180360 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bcd892856dbf10b323fa93a35193c359, offset: 0, size: 212 2024.10.22 01:54:48.210540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bcd892856dbf10b323fa93a35193c359 for removal 2024.10.22 01:54:48.216775 [ 1476 ] {} CacheMetadata: Key bcd892856dbf10b323fa93a35193c359 is removed from metadata 2024.10.22 01:54:48.468613 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 1.97 GiB, free memory in arenas 13.89 MiB, will set to 1.72 GiB (RSS), difference: 13.10 MiB 2024.10.22 01:54:49.220962 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 10714 2024.10.22 01:54:49.309451 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:49.415990 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:49.419520 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 30 file segments (while reserving 4194304 bytes for e8e6c8f81a5292584a98e2ac3638fab6:0) 2024.10.22 01:54:49.431045 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 803a8ff5bbfe01e680ce03d817f7e4b3, offset: 0, size: 156 2024.10.22 01:54:49.439422 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 803a8ff5bbfe01e680ce03d817f7e4b3 for removal 2024.10.22 01:54:49.449169 [ 1476 ] {} CacheMetadata: Key 803a8ff5bbfe01e680ce03d817f7e4b3 is removed from metadata 2024.10.22 01:54:49.450708 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 671000761efef997ee21fb90635e693e, offset: 0, size: 98802 2024.10.22 01:54:49.453809 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 671000761efef997ee21fb90635e693e for removal 2024.10.22 01:54:49.457627 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 735480b659ccc571c1a4edf836db5718, offset: 0, size: 44592 2024.10.22 01:54:49.464063 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 735480b659ccc571c1a4edf836db5718 for removal 2024.10.22 01:54:49.466662 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d2037681f3419e8cf2377da2ddf93c26, offset: 0, size: 168 2024.10.22 01:54:49.468925 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d2037681f3419e8cf2377da2ddf93c26 for removal 2024.10.22 01:54:49.471860 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5bb8e73cf6078709355e34db4d0db5a9, offset: 0, size: 156 2024.10.22 01:54:49.474650 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 5bb8e73cf6078709355e34db4d0db5a9 for removal 2024.10.22 01:54:49.477787 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3dadcc3d080c56375e1fc085094f8180, offset: 0, size: 168 2024.10.22 01:54:49.488196 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 3dadcc3d080c56375e1fc085094f8180 for removal 2024.10.22 01:54:49.495476 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: eacf42ac632a0b2385b1a68bf47ecee0, offset: 0, size: 181 2024.10.22 01:54:49.499647 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key eacf42ac632a0b2385b1a68bf47ecee0 for removal 2024.10.22 01:54:49.507642 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6e5fd2d269500bd96500856e2b3c81b0, offset: 0, size: 97770 2024.10.22 01:54:49.520676 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 6e5fd2d269500bd96500856e2b3c81b0 for removal 2024.10.22 01:54:49.528057 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: aae3bf388e759aa30b5cd5ae5c126545, offset: 0, size: 137397 2024.10.22 01:54:49.533535 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key aae3bf388e759aa30b5cd5ae5c126545 for removal 2024.10.22 01:54:49.539051 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: b946e45055d6277842d0e2929423f07e, offset: 0, size: 37404 2024.10.22 01:54:49.543921 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key b946e45055d6277842d0e2929423f07e for removal 2024.10.22 01:54:49.546747 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: b7dd884e55b796a84ff80b497025bc96, offset: 0, size: 156 2024.10.22 01:54:49.551468 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key b7dd884e55b796a84ff80b497025bc96 for removal 2024.10.22 01:54:49.385722 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 5708 2024.10.22 01:54:49.562960 [ 1476 ] {} CacheMetadata: Key b7dd884e55b796a84ff80b497025bc96 is removed from metadata 2024.10.22 01:54:49.570276 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_704_704_0 (state Temporary) 2024.10.22 01:54:49.583051 [ 1476 ] {} CacheMetadata: Key b946e45055d6277842d0e2929423f07e is removed from metadata 2024.10.22 01:54:49.601792 [ 1476 ] {} CacheMetadata: Key aae3bf388e759aa30b5cd5ae5c126545 is removed from metadata 2024.10.22 01:54:49.618162 [ 1476 ] {} CacheMetadata: Key 6e5fd2d269500bd96500856e2b3c81b0 is removed from metadata 2024.10.22 01:54:49.640557 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_704_704_0 to 202410_717_717_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:49.643135 [ 1476 ] {} CacheMetadata: Key eacf42ac632a0b2385b1a68bf47ecee0 is removed from metadata 2024.10.22 01:54:49.652990 [ 1476 ] {} CacheMetadata: Key 3dadcc3d080c56375e1fc085094f8180 is removed from metadata 2024.10.22 01:54:49.663223 [ 1476 ] {} CacheMetadata: Key 5bb8e73cf6078709355e34db4d0db5a9 is removed from metadata 2024.10.22 01:54:49.673717 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10714 2024.10.22 01:54:49.555540 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 41041dacd7d88d4cf590693f5762d7fb, offset: 0, size: 65031 2024.10.22 01:54:49.676735 [ 1476 ] {} CacheMetadata: Key d2037681f3419e8cf2377da2ddf93c26 is removed from metadata 2024.10.22 01:54:49.680536 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 41041dacd7d88d4cf590693f5762d7fb for removal 2024.10.22 01:54:49.688717 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3d6f0b46802c2bb0187982a7c1e91375, offset: 0, size: 17129 2024.10.22 01:54:49.695686 [ 1476 ] {} CacheMetadata: Key 41041dacd7d88d4cf590693f5762d7fb is removed from metadata 2024.10.22 01:54:49.696467 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 3d6f0b46802c2bb0187982a7c1e91375 for removal 2024.10.22 01:54:49.722739 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c7ecc7cdce609e7314846b0c3955b3d7, offset: 0, size: 3262 2024.10.22 01:54:49.727107 [ 1476 ] {} CacheMetadata: Key 3d6f0b46802c2bb0187982a7c1e91375 is removed from metadata 2024.10.22 01:54:49.727092 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c7ecc7cdce609e7314846b0c3955b3d7 for removal 2024.10.22 01:54:49.734176 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 66818eee3092f565e0adf816070a124c, offset: 0, size: 167 2024.10.22 01:54:49.741087 [ 1476 ] {} CacheMetadata: Key c7ecc7cdce609e7314846b0c3955b3d7 is removed from metadata 2024.10.22 01:54:49.741397 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 66818eee3092f565e0adf816070a124c for removal 2024.10.22 01:54:49.745635 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 317c51ad29e56ca61a507e07c73475c2, offset: 0, size: 156 2024.10.22 01:54:49.751878 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 317c51ad29e56ca61a507e07c73475c2 for removal 2024.10.22 01:54:49.753192 [ 1476 ] {} CacheMetadata: Key 66818eee3092f565e0adf816070a124c is removed from metadata 2024.10.22 01:54:49.754492 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: f2ea4f9e6678dcceea98222e55e80557, offset: 0, size: 922184 2024.10.22 01:54:49.767001 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key f2ea4f9e6678dcceea98222e55e80557 for removal 2024.10.22 01:54:49.771198 [ 1476 ] {} CacheMetadata: Key 317c51ad29e56ca61a507e07c73475c2 is removed from metadata 2024.10.22 01:54:49.775960 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8c922033a1bd3d26f79c164e48e22655, offset: 0, size: 156 2024.10.22 01:54:49.794642 [ 1476 ] {} CacheMetadata: Key f2ea4f9e6678dcceea98222e55e80557 is removed from metadata 2024.10.22 01:54:49.999807 [ 1476 ] {} CacheMetadata: Key 735480b659ccc571c1a4edf836db5718 is removed from metadata 2024.10.22 01:54:50.010146 [ 1476 ] {} CacheMetadata: Key 671000761efef997ee21fb90635e693e is removed from metadata 2024.10.22 01:54:50.099901 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 8c922033a1bd3d26f79c164e48e22655 for removal 2024.10.22 01:54:50.102957 [ 1476 ] {} CacheMetadata: Key 8c922033a1bd3d26f79c164e48e22655 is removed from metadata 2024.10.22 01:54:50.103900 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d6ad3a39965c4f875f9821b804ff3004, offset: 0, size: 156 2024.10.22 01:54:50.108534 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d6ad3a39965c4f875f9821b804ff3004 for removal 2024.10.22 01:54:50.118613 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: b070b7226562547b0c505b844a56b513, offset: 0, size: 155 2024.10.22 01:54:50.129982 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key b070b7226562547b0c505b844a56b513 for removal 2024.10.22 01:54:50.133813 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0c6a0e2f1558bf1305193a5c30de68ac, offset: 0, size: 167 2024.10.22 01:54:50.139269 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 0c6a0e2f1558bf1305193a5c30de68ac for removal 2024.10.22 01:54:50.148815 [ 1476 ] {} CacheMetadata: Key b070b7226562547b0c505b844a56b513 is removed from metadata 2024.10.22 01:54:50.153102 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0c19759be5ce14ad1814b1ccaaa178e4, offset: 0, size: 2573 2024.10.22 01:54:50.155807 [ 1476 ] {} CacheMetadata: Key 0c6a0e2f1558bf1305193a5c30de68ac is removed from metadata 2024.10.22 01:54:50.162207 [ 1476 ] {} CacheMetadata: Key d6ad3a39965c4f875f9821b804ff3004 is removed from metadata 2024.10.22 01:54:50.255820 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 0c19759be5ce14ad1814b1ccaaa178e4 for removal 2024.10.22 01:54:50.259201 [ 1476 ] {} CacheMetadata: Key 0c19759be5ce14ad1814b1ccaaa178e4 is removed from metadata 2024.10.22 01:54:50.259356 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 087fc1e9dffcc54c5f7ef393d9c6c955, offset: 0, size: 23187 2024.10.22 01:54:50.263854 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 087fc1e9dffcc54c5f7ef393d9c6c955 for removal 2024.10.22 01:54:50.267658 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a2bd114ded4539fb8ff896f1e86c25b6, offset: 0, size: 294167 2024.10.22 01:54:50.268608 [ 1476 ] {} CacheMetadata: Key 087fc1e9dffcc54c5f7ef393d9c6c955 is removed from metadata 2024.10.22 01:54:50.270058 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a2bd114ded4539fb8ff896f1e86c25b6 for removal 2024.10.22 01:54:50.275586 [ 1476 ] {} CacheMetadata: Key a2bd114ded4539fb8ff896f1e86c25b6 is removed from metadata 2024.10.22 01:54:50.274920 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d219b1fe350af21f33a0d3c0f49ea2c9, offset: 0, size: 167 2024.10.22 01:54:50.286906 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d219b1fe350af21f33a0d3c0f49ea2c9 for removal 2024.10.22 01:54:50.296586 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 75c3fff1979f05761dcc8d5c5ac4ff10, offset: 0, size: 48871 2024.10.22 01:54:50.296875 [ 1476 ] {} CacheMetadata: Key d219b1fe350af21f33a0d3c0f49ea2c9 is removed from metadata 2024.10.22 01:54:50.304746 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 75c3fff1979f05761dcc8d5c5ac4ff10 for removal 2024.10.22 01:54:50.313679 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8fb0956dc448268e81deeccf3465336a, offset: 0, size: 151 2024.10.22 01:54:50.321529 [ 1476 ] {} CacheMetadata: Key 75c3fff1979f05761dcc8d5c5ac4ff10 is removed from metadata 2024.10.22 01:54:50.324193 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 8fb0956dc448268e81deeccf3465336a for removal 2024.10.22 01:54:50.331810 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2257586cfb0d97c42b9cbcec76259eac, offset: 0, size: 57912 2024.10.22 01:54:50.335179 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 2257586cfb0d97c42b9cbcec76259eac for removal 2024.10.22 01:54:50.337820 [ 1476 ] {} CacheMetadata: Key 8fb0956dc448268e81deeccf3465336a is removed from metadata 2024.10.22 01:54:50.340216 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: dd749e1385b677372a239dc9b1c9f60d, offset: 0, size: 220387 2024.10.22 01:54:50.346358 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key dd749e1385b677372a239dc9b1c9f60d for removal 2024.10.22 01:54:50.348490 [ 1476 ] {} CacheMetadata: Key 2257586cfb0d97c42b9cbcec76259eac is removed from metadata 2024.10.22 01:54:50.351286 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 19b0808d0ad5eba72dcbc8dcac3ec089, offset: 0, size: 156 2024.10.22 01:54:50.354191 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 19b0808d0ad5eba72dcbc8dcac3ec089 for removal 2024.10.22 01:54:50.355204 [ 1476 ] {} CacheMetadata: Key dd749e1385b677372a239dc9b1c9f60d is removed from metadata 2024.10.22 01:54:50.365634 [ 1476 ] {} CacheMetadata: Key 19b0808d0ad5eba72dcbc8dcac3ec089 is removed from metadata 2024.10.22 01:54:50.966699 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:54:50.968586 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:54:51.068725 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 14 file segments (while reserving 189872 bytes for 75259b77190ec9a0153cd9bf6a607fea:0) 2024.10.22 01:54:51.071382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 28283ffb81052415efc20e23b8f7c03c, offset: 0, size: 162731 2024.10.22 01:54:51.074472 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 28283ffb81052415efc20e23b8f7c03c for removal 2024.10.22 01:54:51.076971 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4e7bdfcb932a4ab579320fb0ace42807, offset: 0, size: 25282 2024.10.22 01:54:51.077494 [ 1476 ] {} CacheMetadata: Key 28283ffb81052415efc20e23b8f7c03c is removed from metadata 2024.10.22 01:54:51.084982 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4e7bdfcb932a4ab579320fb0ace42807 for removal 2024.10.22 01:54:51.093654 [ 1476 ] {} CacheMetadata: Key 4e7bdfcb932a4ab579320fb0ace42807 is removed from metadata 2024.10.22 01:54:51.092072 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5318499f4c55defe8fbd7bcc355d8c5b, offset: 0, size: 2846 2024.10.22 01:54:51.104676 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5318499f4c55defe8fbd7bcc355d8c5b for removal 2024.10.22 01:54:51.122467 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 54e3b65b27e0296a1dcd2b376739e5a8, offset: 0, size: 156 2024.10.22 01:54:51.125058 [ 1476 ] {} CacheMetadata: Key 5318499f4c55defe8fbd7bcc355d8c5b is removed from metadata 2024.10.22 01:54:51.126644 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 54e3b65b27e0296a1dcd2b376739e5a8 for removal 2024.10.22 01:54:51.135947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 67c55aad49697602b953f10730605995, offset: 0, size: 156 2024.10.22 01:54:51.142209 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 67c55aad49697602b953f10730605995 for removal 2024.10.22 01:54:51.147209 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0d8d86e11053fdcfc37f2573322985d7, offset: 0, size: 24126 2024.10.22 01:54:51.147155 [ 1476 ] {} CacheMetadata: Key 54e3b65b27e0296a1dcd2b376739e5a8 is removed from metadata 2024.10.22 01:54:51.153091 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0d8d86e11053fdcfc37f2573322985d7 for removal 2024.10.22 01:54:51.157449 [ 1476 ] {} CacheMetadata: Key 67c55aad49697602b953f10730605995 is removed from metadata 2024.10.22 01:54:51.158656 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cfbce0e57354112bb95b4906c83f9665, offset: 0, size: 156 2024.10.22 01:54:51.263919 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cfbce0e57354112bb95b4906c83f9665 for removal 2024.10.22 01:54:51.266744 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 97b292b2fc23ff42a98ead7a3e851938, offset: 0, size: 27402 2024.10.22 01:54:51.269752 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 97b292b2fc23ff42a98ead7a3e851938 for removal 2024.10.22 01:54:51.272298 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: de2521ad585f19119b6cfb3ad36d0976, offset: 0, size: 94866 2024.10.22 01:54:51.276677 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key de2521ad585f19119b6cfb3ad36d0976 for removal 2024.10.22 01:54:51.281331 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c6fd0c43e8027bf0cbeb1eefc2c13d1f, offset: 0, size: 156 2024.10.22 01:54:51.288453 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c6fd0c43e8027bf0cbeb1eefc2c13d1f for removal 2024.10.22 01:54:51.293027 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b4816cc70fe6448425cc6e1f7b301fdf, offset: 0, size: 155 2024.10.22 01:54:51.298939 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b4816cc70fe6448425cc6e1f7b301fdf for removal 2024.10.22 01:54:51.302469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 03d33d6f2e37a27586d20f8972751636, offset: 0, size: 168 2024.10.22 01:54:51.305186 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 03d33d6f2e37a27586d20f8972751636 for removal 2024.10.22 01:54:51.308815 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4f5d67c640beb05ea1fdc9c95a43c88f, offset: 0, size: 2134 2024.10.22 01:54:51.312673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4f5d67c640beb05ea1fdc9c95a43c88f for removal 2024.10.22 01:54:51.318260 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 55fb983e94113782b73114d29d31480e, offset: 0, size: 156 2024.10.22 01:54:51.323679 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 55fb983e94113782b73114d29d31480e for removal 2024.10.22 01:54:51.469951 [ 1476 ] {} CacheMetadata: Key 55fb983e94113782b73114d29d31480e is removed from metadata 2024.10.22 01:54:51.484709 [ 1476 ] {} CacheMetadata: Key 4f5d67c640beb05ea1fdc9c95a43c88f is removed from metadata 2024.10.22 01:54:51.506938 [ 1476 ] {} CacheMetadata: Key 03d33d6f2e37a27586d20f8972751636 is removed from metadata 2024.10.22 01:54:51.530691 [ 1476 ] {} CacheMetadata: Key b4816cc70fe6448425cc6e1f7b301fdf is removed from metadata 2024.10.22 01:54:51.546685 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 159 entries to flush up to offset 78555 2024.10.22 01:54:51.551983 [ 1476 ] {} CacheMetadata: Key c6fd0c43e8027bf0cbeb1eefc2c13d1f is removed from metadata 2024.10.22 01:54:51.554978 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_9_9_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:54:51.562426 [ 1476 ] {} CacheMetadata: Key de2521ad585f19119b6cfb3ad36d0976 is removed from metadata 2024.10.22 01:54:51.570342 [ 1476 ] {} CacheMetadata: Key 97b292b2fc23ff42a98ead7a3e851938 is removed from metadata 2024.10.22 01:54:51.582542 [ 1476 ] {} CacheMetadata: Key cfbce0e57354112bb95b4906c83f9665 is removed from metadata 2024.10.22 01:54:51.599188 [ 1476 ] {} CacheMetadata: Key 0d8d86e11053fdcfc37f2573322985d7 is removed from metadata 2024.10.22 01:54:51.852290 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:51.855233 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:51.987052 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1187_1187_0 (state Temporary) 2024.10.22 01:54:52.160207 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1187_1187_0 to 202410_1200_1200_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:52.163101 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 236143 bytes for 4840100cc3626da522f517ea391342ba:0) 2024.10.22 01:54:52.163509 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 78197 2024.10.22 01:54:52.164812 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fdeb9eb51128ae3f91a8400928638c23, offset: 0, size: 139777 2024.10.22 01:54:52.171357 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fdeb9eb51128ae3f91a8400928638c23 for removal 2024.10.22 01:54:52.174623 [ 1476 ] {} CacheMetadata: Key fdeb9eb51128ae3f91a8400928638c23 is removed from metadata 2024.10.22 01:54:52.176747 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7b29b769dd0e80372afc4c0311db3568, offset: 0, size: 157 2024.10.22 01:54:52.189714 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7b29b769dd0e80372afc4c0311db3568 for removal 2024.10.22 01:54:52.224813 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 78555 2024.10.22 01:54:52.303259 [ 1476 ] {} CacheMetadata: Key 7b29b769dd0e80372afc4c0311db3568 is removed from metadata 2024.10.22 01:54:52.615046 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:52.618496 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:52.668976 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_692_692_0 (state Temporary) 2024.10.22 01:54:52.732170 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 12.46 MiB, will set to 1.72 GiB (RSS), difference: -15.12 MiB 2024.10.22 01:54:52.755098 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_692_692_0 to 202410_698_698_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:52.775876 [ 1208 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_693_43 to 202410_694_694_0 2024.10.22 01:54:52.815494 [ 1208 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.10 MiB using storage policy from min volume index 0 2024.10.22 01:54:52.820795 [ 1208 ] {} DiskLocal: Reserved 1.10 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:52.830883 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 78197 2024.10.22 01:54:52.958152 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_694_44} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_693_43 to 202410_694_694_0 into Compact with storage Full 2024.10.22 01:54:52.997317 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1026881 2024.10.22 01:54:53.007394 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_694_44} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:53.017955 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_694_44} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_693_43, total 18646 rows starting from the beginning of the part 2024.10.22 01:54:53.047743 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_694_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_694_694_0, total 55 rows starting from the beginning of the part 2024.10.22 01:54:53.234181 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 6 file segments (while reserving 383691 bytes for cb645196d58ef917b4115275a48c15d4:0) 2024.10.22 01:54:53.241322 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8c7e707003eca5a67bbea1473dab2793, offset: 0, size: 167 2024.10.22 01:54:53.257603 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8c7e707003eca5a67bbea1473dab2793 for removal 2024.10.22 01:54:53.260695 [ 1476 ] {} CacheMetadata: Key 8c7e707003eca5a67bbea1473dab2793 is removed from metadata 2024.10.22 01:54:53.263422 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2dc3f738d56e21e70f7d6f6abcdd7cde, offset: 0, size: 169 2024.10.22 01:54:53.369318 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2dc3f738d56e21e70f7d6f6abcdd7cde for removal 2024.10.22 01:54:53.372219 [ 1476 ] {} CacheMetadata: Key 2dc3f738d56e21e70f7d6f6abcdd7cde is removed from metadata 2024.10.22 01:54:53.372973 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e101d76cfb6afc5a4b5efbde6e8f0e0a, offset: 0, size: 81804 2024.10.22 01:54:53.377516 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e101d76cfb6afc5a4b5efbde6e8f0e0a for removal 2024.10.22 01:54:53.382794 [ 1476 ] {} CacheMetadata: Key e101d76cfb6afc5a4b5efbde6e8f0e0a is removed from metadata 2024.10.22 01:54:53.383577 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 45caab0915f1fdf834d7b34c057e9b5d, offset: 0, size: 191333 2024.10.22 01:54:53.402330 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 45caab0915f1fdf834d7b34c057e9b5d for removal 2024.10.22 01:54:53.405635 [ 1476 ] {} CacheMetadata: Key 45caab0915f1fdf834d7b34c057e9b5d is removed from metadata 2024.10.22 01:54:53.406876 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8c4f4935457bf181859416957636406a, offset: 0, size: 157 2024.10.22 01:54:53.413045 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8c4f4935457bf181859416957636406a for removal 2024.10.22 01:54:53.421262 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cc572365e38f8c78270c08821a048a84, offset: 0, size: 59379 2024.10.22 01:54:53.426891 [ 1476 ] {} CacheMetadata: Key 8c4f4935457bf181859416957636406a is removed from metadata 2024.10.22 01:54:53.427492 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cc572365e38f8c78270c08821a048a84 for removal 2024.10.22 01:54:53.441359 [ 1476 ] {} CacheMetadata: Key cc572365e38f8c78270c08821a048a84 is removed from metadata 2024.10.22 01:54:53.597105 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_676_130/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:54:54.072207 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:54.075333 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:54.375315 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_675_675_0 (state Temporary) 2024.10.22 01:54:54.501216 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for fb4763fa40288d4098bfca6554b133c5:0) 2024.10.22 01:54:54.505088 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9c5b11d4219a08ec4bdfeed24024a941, offset: 0, size: 925497 2024.10.22 01:54:54.511865 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9c5b11d4219a08ec4bdfeed24024a941 for removal 2024.10.22 01:54:54.515560 [ 1476 ] {} CacheMetadata: Key 9c5b11d4219a08ec4bdfeed24024a941 is removed from metadata 2024.10.22 01:54:54.515887 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 588 2024.10.22 01:54:54.516251 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 197530751ab35901904e8b1e59e4cd30, offset: 0, size: 195309 2024.10.22 01:54:54.519151 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 197530751ab35901904e8b1e59e4cd30 for removal 2024.10.22 01:54:54.523163 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4e97fea87dbffbb2a41e804696dd5fb9, offset: 0, size: 167 2024.10.22 01:54:54.528239 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4e97fea87dbffbb2a41e804696dd5fb9 for removal 2024.10.22 01:54:54.534462 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 892b8bfec606a12a4ee945baaf6ecb3f, offset: 0, size: 168 2024.10.22 01:54:54.544847 [ 1476 ] {} CacheMetadata: Key 4e97fea87dbffbb2a41e804696dd5fb9 is removed from metadata 2024.10.22 01:54:54.545034 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 892b8bfec606a12a4ee945baaf6ecb3f for removal 2024.10.22 01:54:54.548902 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 50c00292868f8614ff6ccc4052614dc2, offset: 0, size: 77121 2024.10.22 01:54:54.552516 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 50c00292868f8614ff6ccc4052614dc2 for removal 2024.10.22 01:54:54.553169 [ 1476 ] {} CacheMetadata: Key 892b8bfec606a12a4ee945baaf6ecb3f is removed from metadata 2024.10.22 01:54:54.555373 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 87a7e85562faa2af490945393eb20800, offset: 0, size: 169 2024.10.22 01:54:54.558689 [ 1476 ] {} CacheMetadata: Key 50c00292868f8614ff6ccc4052614dc2 is removed from metadata 2024.10.22 01:54:54.559230 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 87a7e85562faa2af490945393eb20800 for removal 2024.10.22 01:54:54.563723 [ 1476 ] {} CacheMetadata: Key 87a7e85562faa2af490945393eb20800 is removed from metadata 2024.10.22 01:54:54.569142 [ 1476 ] {} CacheMetadata: Key 197530751ab35901904e8b1e59e4cd30 is removed from metadata 2024.10.22 01:54:54.615941 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_694_44} MergeTask::MergeProjectionsStage: Merge sorted 18701 rows, containing 13 columns (13 merged, 0 gathered) in 1.751386508 sec., 10677.825776650325 rows/sec., 4.62 MiB/sec. 2024.10.22 01:54:54.655947 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_694_44} MergedBlockOutputStream: filled checksums 202410_507_694_44 (state Temporary) 2024.10.22 01:54:54.678212 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_675_675_0 to 202410_682_682_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:54.722610 [ 1221 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_677_677_0 to 202410_682_682_0 2024.10.22 01:54:54.758476 [ 1221 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:54.761108 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:54.775680 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_677_677_0 to 202410_682_682_0 into Compact with storage Full 2024.10.22 01:54:54.834743 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:54.840656 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1026881 2024.10.22 01:54:54.847422 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_677_677_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:54:54.860637 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_678_678_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:54:54.892538 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:54.894504 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_679_679_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:54:54.896956 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:55.039922 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_680_680_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:54:55.060003 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_681_681_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:54:55.103116 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_682_682_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:54:55.130168 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_694_44} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_694_44 to 202410_507_694_44 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:55.134327 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_694_44} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_693_43, 202410_694_694_0] -> 202410_507_694_44 2024.10.22 01:54:55.178283 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_570_570_0 (state Temporary) 2024.10.22 01:54:55.161767 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.79 MiB. 2024.10.22 01:54:55.380687 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_570_570_0 to 202410_580_580_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:55.506117 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 588 2024.10.22 01:54:55.925816 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(e8e6c8f81a5292584a98e2ac3638fab6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e8e6c8f81a5292584a98e2ac3638fab6, state: DOWNLOADING, downloaded size: 8388608, reserved size: 8388608, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 8388608, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:54:55.930775 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 8388608, file segment info: File segment: [0, 33554431], key: e8e6c8f81a5292584a98e2ac3638fab6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 8388608, reserved size: 8388608, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 8388608, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:54:55.934567 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:54:56.356967 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 103 entries to flush up to offset 78658 2024.10.22 01:54:56.361804 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 10.06 MiB, will set to 1.73 GiB (RSS), difference: -15.58 MiB 2024.10.22 01:54:56.507285 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} MergeTask::MergeProjectionsStage: Merge sorted 8855 rows, containing 4 columns (4 merged, 0 gathered) in 1.736784738 sec., 5098.501734991639 rows/sec., 79.66 KiB/sec. 2024.10.22 01:54:56.546593 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} MergedBlockOutputStream: filled checksums 202410_677_682_1 (state Temporary) 2024.10.22 01:54:56.609140 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 8 file segments (while reserving 375980 bytes for 27e4540ea05c1e471ac94c868255b5c0:0) 2024.10.22 01:54:56.614652 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fef09fe8854c544f6df3f627c9cdf8a7, offset: 0, size: 65823 2024.10.22 01:54:56.619407 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fef09fe8854c544f6df3f627c9cdf8a7 for removal 2024.10.22 01:54:56.621683 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_677_682_1 to 202410_677_682_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:56.628699 [ 1476 ] {} CacheMetadata: Key fef09fe8854c544f6df3f627c9cdf8a7 is removed from metadata 2024.10.22 01:54:56.633400 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_677_682_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_677_677_0, 202410_682_682_0] -> 202410_677_682_1 2024.10.22 01:54:56.633701 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7a97ba1fc080968c1b3178f09bfbaec1, offset: 0, size: 151 2024.10.22 01:54:56.650650 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7a97ba1fc080968c1b3178f09bfbaec1 for removal 2024.10.22 01:54:56.655537 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9e2aa0218c07e193d0c12b2ea6578487, offset: 0, size: 108267 2024.10.22 01:54:56.658011 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:56.661543 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:56.663277 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:54:56.734641 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1188_1188_0 (state Temporary) 2024.10.22 01:54:56.758262 [ 1476 ] {} CacheMetadata: Key 7a97ba1fc080968c1b3178f09bfbaec1 is removed from metadata 2024.10.22 01:54:56.759394 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9e2aa0218c07e193d0c12b2ea6578487 for removal 2024.10.22 01:54:56.760624 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cbdda555540d1422c6ac4a61ff8e4f7b, offset: 0, size: 36495 2024.10.22 01:54:56.764632 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cbdda555540d1422c6ac4a61ff8e4f7b for removal 2024.10.22 01:54:56.771807 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 33488f0fafb0e1cd3ab4dc6529422c98, offset: 0, size: 156 2024.10.22 01:54:56.777404 [ 1476 ] {} CacheMetadata: Key cbdda555540d1422c6ac4a61ff8e4f7b is removed from metadata 2024.10.22 01:54:56.777636 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 33488f0fafb0e1cd3ab4dc6529422c98 for removal 2024.10.22 01:54:56.784299 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a296082f6f4a5442a30c77ba69c7d7b6, offset: 0, size: 153 2024.10.22 01:54:56.812360 [ 1476 ] {} CacheMetadata: Key 33488f0fafb0e1cd3ab4dc6529422c98 is removed from metadata 2024.10.22 01:54:56.830794 [ 1476 ] {} CacheMetadata: Key 9e2aa0218c07e193d0c12b2ea6578487 is removed from metadata 2024.10.22 01:54:56.793812 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a296082f6f4a5442a30c77ba69c7d7b6 for removal 2024.10.22 01:54:56.921192 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1188_1188_0 to 202410_1201_1201_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:57.000083 [ 1476 ] {} CacheMetadata: Key a296082f6f4a5442a30c77ba69c7d7b6 is removed from metadata 2024.10.22 01:54:57.001813 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 44167336c2dbc079dc7e7124e41f661c, offset: 0, size: 41515 2024.10.22 01:54:57.005728 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 44167336c2dbc079dc7e7124e41f661c for removal 2024.10.22 01:54:57.009499 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f3b14987f12a2a132505a8da474ac4b4, offset: 0, size: 156 2024.10.22 01:54:57.009975 [ 1476 ] {} CacheMetadata: Key 44167336c2dbc079dc7e7124e41f661c is removed from metadata 2024.10.22 01:54:57.013098 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f3b14987f12a2a132505a8da474ac4b4 for removal 2024.10.22 01:54:57.057389 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1196_20 to 202410_1201_1201_0 2024.10.22 01:54:57.069287 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:57.076228 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:57.084720 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 78658 2024.10.22 01:54:57.123291 [ 1476 ] {} CacheMetadata: Key f3b14987f12a2a132505a8da474ac4b4 is removed from metadata 2024.10.22 01:54:57.176811 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 10730 2024.10.22 01:54:57.184405 [ 1250 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 01:54:57.307061 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1196_20 to 202410_1201_1201_0 into Compact with storage Full 2024.10.22 01:54:57.403310 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:54:57.407252 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:54:57.468761 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_705_705_0 (state Temporary) 2024.10.22 01:54:57.469623 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:57.474865 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1196_20, total 7573 rows starting from the beginning of the part 2024.10.22 01:54:57.505068 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1197_1197_0, total 60 rows starting from the beginning of the part 2024.10.22 01:54:57.547027 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_705_705_0 to 202410_718_718_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:57.583561 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10730 2024.10.22 01:54:57.690085 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1198_1198_0, total 80 rows starting from the beginning of the part 2024.10.22 01:54:57.719337 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1199_1199_0, total 107 rows starting from the beginning of the part 2024.10.22 01:54:57.747077 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1200_1200_0, total 159 rows starting from the beginning of the part 2024.10.22 01:54:57.782315 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1201_1201_0, total 103 rows starting from the beginning of the part 2024.10.22 01:54:58.093522 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 244085 bytes for 3fb19e1e9e8c322839d9675cdc55e0c7:0) 2024.10.22 01:54:58.096675 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5b5a7a896b8e6c6e6576ad2e86d02060, offset: 0, size: 261296 2024.10.22 01:54:58.100634 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5b5a7a896b8e6c6e6576ad2e86d02060 for removal 2024.10.22 01:54:58.103260 [ 1476 ] {} CacheMetadata: Key 5b5a7a896b8e6c6e6576ad2e86d02060 is removed from metadata 2024.10.22 01:54:58.104990 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f835beb27218581e6d3fe5787781e5eb, offset: 0, size: 180 2024.10.22 01:54:58.131563 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f835beb27218581e6d3fe5787781e5eb for removal 2024.10.22 01:54:58.139813 [ 1476 ] {} CacheMetadata: Key f835beb27218581e6d3fe5787781e5eb is removed from metadata 2024.10.22 01:54:58.303394 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 11.38 MiB, will set to 1.73 GiB (RSS), difference: 10.06 MiB 2024.10.22 01:54:58.895330 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:54:59.146448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 160601 bytes for 71c5001b136e57e26a419ba39b6904a5:0) 2024.10.22 01:54:59.150921 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9fde7459b3f77b09b510a9b88e281ac7, offset: 0, size: 732670 2024.10.22 01:54:59.153235 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9fde7459b3f77b09b510a9b88e281ac7 for removal 2024.10.22 01:54:59.157072 [ 1476 ] {} CacheMetadata: Key 9fde7459b3f77b09b510a9b88e281ac7 is removed from metadata 2024.10.22 01:54:59.158280 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5c0a66f599f87be353878be0e8d07b59, offset: 0, size: 165 2024.10.22 01:54:59.160913 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5c0a66f599f87be353878be0e8d07b59 for removal 2024.10.22 01:54:59.194469 [ 1476 ] {} CacheMetadata: Key 5c0a66f599f87be353878be0e8d07b59 is removed from metadata 2024.10.22 01:54:59.273186 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} MergeTask::MergeProjectionsStage: Merge sorted 8082 rows, containing 13 columns (13 merged, 0 gathered) in 2.076253317 sec., 3892.5886036284655 rows/sec., 659.00 KiB/sec. 2024.10.22 01:54:59.299504 [ 1267 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 01:54:59.474900 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} MergedBlockOutputStream: filled checksums 202410_1096_1201_21 (state Temporary) 2024.10.22 01:54:59.693075 [ 1257 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:54:59.745012 [ 1257 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:54:59.753411 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1201_21 to 202410_1096_1201_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:54:59.756136 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1201_21} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1196_20, 202410_1201_1201_0] -> 202410_1096_1201_21 2024.10.22 01:54:59.791820 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.10 MiB. 2024.10.22 01:54:59.911183 [ 1261 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:55:00.347851 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 78312 2024.10.22 01:55:00.449577 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_134_24, total 5298 rows starting from the beginning of the part 2024.10.22 01:55:00.450826 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 1.97 GiB, free memory in arenas 10.60 MiB, will set to 1.73 GiB (RSS), difference: 6.59 MiB 2024.10.22 01:55:00.537525 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:00.545030 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:00.610565 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_693_693_0 (state Temporary) 2024.10.22 01:55:00.850837 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_693_693_0 to 202410_699_699_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:00.871571 [ 1246 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_694_44 to 202410_695_695_0 2024.10.22 01:55:00.876787 [ 1246 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 01:55:00.898604 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 78312 2024.10.22 01:55:00.882521 [ 1246 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:00.968070 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:55:00.969551 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:55:01.005441 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_695_45} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_694_44 to 202410_695_695_0 into Compact with storage Full 2024.10.22 01:55:01.064599 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_695_45} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:01.070377 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_695_45} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_694_44, total 18701 rows starting from the beginning of the part 2024.10.22 01:55:01.094829 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_695_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_695_695_0, total 75 rows starting from the beginning of the part 2024.10.22 01:55:01.107170 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 78754 2024.10.22 01:55:01.153093 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 112548 bytes for 48b57aef463227eb185c7c598870bb73:0) 2024.10.22 01:55:01.166663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6afe9d4e44dc89a965b8af242c6bea6b, offset: 0, size: 96373 2024.10.22 01:55:01.170136 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6afe9d4e44dc89a965b8af242c6bea6b for removal 2024.10.22 01:55:01.173250 [ 1476 ] {} CacheMetadata: Key 6afe9d4e44dc89a965b8af242c6bea6b is removed from metadata 2024.10.22 01:55:01.173840 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 838d3ae63bdb0483d9b72d816c9e736b, offset: 0, size: 157 2024.10.22 01:55:01.300533 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 838d3ae63bdb0483d9b72d816c9e736b for removal 2024.10.22 01:55:01.310835 [ 1476 ] {} CacheMetadata: Key 838d3ae63bdb0483d9b72d816c9e736b is removed from metadata 2024.10.22 01:55:01.312033 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 03292208fde7e32b7895a747d4a2455b, offset: 0, size: 47550 2024.10.22 01:55:01.317232 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 03292208fde7e32b7895a747d4a2455b for removal 2024.10.22 01:55:01.324270 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bfa41bce1f5f9e32cbcf09e75bf97924, offset: 0, size: 156 2024.10.22 01:55:01.327123 [ 1476 ] {} CacheMetadata: Key 03292208fde7e32b7895a747d4a2455b is removed from metadata 2024.10.22 01:55:01.332518 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bfa41bce1f5f9e32cbcf09e75bf97924 for removal 2024.10.22 01:55:01.337017 [ 1476 ] {} CacheMetadata: Key bfa41bce1f5f9e32cbcf09e75bf97924 is removed from metadata 2024.10.22 01:55:01.802787 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 813505 bytes for fb4763fa40288d4098bfca6554b133c5:0) 2024.10.22 01:55:01.909025 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a89b593e7d300c0ac159376ad10f5dd0, offset: 0, size: 923031 2024.10.22 01:55:01.913280 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a89b593e7d300c0ac159376ad10f5dd0 for removal 2024.10.22 01:55:01.917405 [ 1476 ] {} CacheMetadata: Key a89b593e7d300c0ac159376ad10f5dd0 is removed from metadata 2024.10.22 01:55:01.918513 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ba0320d1cc524febb1d03848900583fa, offset: 0, size: 167 2024.10.22 01:55:01.919705 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:01.948095 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1028036 2024.10.22 01:55:02.040556 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ba0320d1cc524febb1d03848900583fa for removal 2024.10.22 01:55:02.048560 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: aea9898106cb50087dd455ae2f98759b, offset: 0, size: 6109 2024.10.22 01:55:02.052202 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key aea9898106cb50087dd455ae2f98759b for removal 2024.10.22 01:55:02.054031 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8240697e921afb9f265bcde36b9b0980, offset: 0, size: 156 2024.10.22 01:55:02.056913 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8240697e921afb9f265bcde36b9b0980 for removal 2024.10.22 01:55:02.152290 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:02.349259 [ 1476 ] {} CacheMetadata: Key 8240697e921afb9f265bcde36b9b0980 is removed from metadata 2024.10.22 01:55:02.361333 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1189_1189_0 (state Temporary) 2024.10.22 01:55:02.402879 [ 1476 ] {} CacheMetadata: Key aea9898106cb50087dd455ae2f98759b is removed from metadata 2024.10.22 01:55:02.430901 [ 1476 ] {} CacheMetadata: Key ba0320d1cc524febb1d03848900583fa is removed from metadata 2024.10.22 01:55:02.553704 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1189_1189_0 to 202410_1202_1202_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:02.662103 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 78754 2024.10.22 01:55:02.741543 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_695_45} MergeTask::MergeProjectionsStage: Merge sorted 18776 rows, containing 13 columns (13 merged, 0 gathered) in 1.748126025 sec., 10740.644399479152 rows/sec., 4.65 MiB/sec. 2024.10.22 01:55:02.773118 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_695_45} MergedBlockOutputStream: filled checksums 202410_507_695_45 (state Temporary) 2024.10.22 01:55:02.831067 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:02.836562 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:02.888330 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:55:02.908191 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:55:02.948923 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:55:02.990518 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:55:03.012377 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 589 2024.10.22 01:55:03.035722 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:55:03.073813 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:55:03.114063 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:55:03.144290 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_695_45} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_695_45 to 202410_507_695_45 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:03.149433 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_695_45} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_694_44, 202410_695_695_0] -> 202410_507_695_45 2024.10.22 01:55:03.166146 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 01:55:03.194328 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.79 MiB. 2024.10.22 01:55:03.255467 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_676_676_0 (state Temporary) 2024.10.22 01:55:03.353676 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_676_676_0 to 202410_683_683_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:03.357504 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 220545 bytes for 578bf39e88810c94973b420e51254111:0) 2024.10.22 01:55:03.360716 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b05878ee6795d35d4c52a4ed4ce8edc4, offset: 0, size: 59365 2024.10.22 01:55:03.366352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b05878ee6795d35d4c52a4ed4ce8edc4 for removal 2024.10.22 01:55:03.378177 [ 1476 ] {} CacheMetadata: Key b05878ee6795d35d4c52a4ed4ce8edc4 is removed from metadata 2024.10.22 01:55:03.380320 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8764684d428cdb3f2bd8b49e651fb575, offset: 0, size: 73279 2024.10.22 01:55:03.391651 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8764684d428cdb3f2bd8b49e651fb575 for removal 2024.10.22 01:55:03.399795 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5d0e1cf6ee8667923f437008fd7d478b, offset: 0, size: 156 2024.10.22 01:55:03.404281 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5d0e1cf6ee8667923f437008fd7d478b for removal 2024.10.22 01:55:03.406411 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1028036 2024.10.22 01:55:03.410369 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 253baf54f99b1904bd12bc80a57a4b87, offset: 0, size: 157 2024.10.22 01:55:03.418931 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 253baf54f99b1904bd12bc80a57a4b87 for removal 2024.10.22 01:55:03.428888 [ 1476 ] {} CacheMetadata: Key 253baf54f99b1904bd12bc80a57a4b87 is removed from metadata 2024.10.22 01:55:03.482367 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fgz/jskzccfiorchjqwhpkvotonuaguwp (LogSeriesLimiter: on interval from 2024-10-22 01:54:41 to 2024-10-22 01:55:03 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:55:03.541507 [ 1476 ] {} CacheMetadata: Key 5d0e1cf6ee8667923f437008fd7d478b is removed from metadata 2024.10.22 01:55:03.551036 [ 1476 ] {} CacheMetadata: Key 8764684d428cdb3f2bd8b49e651fb575 is removed from metadata 2024.10.22 01:55:03.553035 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:03.555788 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:03.565453 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 11.39 MiB, will set to 1.73 GiB (RSS), difference: 1.84 MiB 2024.10.22 01:55:03.718623 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_135_139_1, total 209 rows starting from the beginning of the part 2024.10.22 01:55:03.845018 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_571_571_0 (state Temporary) 2024.10.22 01:55:04.017193 [ 1154 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:55:04.125134 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_571_571_0 to 202410_581_581_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:04.229504 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 6 file segments (while reserving 192987 bytes for c600b434a339d3c787519eb6373c96d8:0) 2024.10.22 01:55:04.239910 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: adc21c7e00a5641279d7f3c339b92c1e, offset: 0, size: 48318 2024.10.22 01:55:04.346406 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 589 2024.10.22 01:55:04.347688 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key adc21c7e00a5641279d7f3c339b92c1e for removal 2024.10.22 01:55:04.350768 [ 1476 ] {} CacheMetadata: Key adc21c7e00a5641279d7f3c339b92c1e is removed from metadata 2024.10.22 01:55:04.351480 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4b186e867e31e85988806f08144215b7, offset: 0, size: 59552 2024.10.22 01:55:04.365173 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4b186e867e31e85988806f08144215b7 for removal 2024.10.22 01:55:04.367478 [ 1476 ] {} CacheMetadata: Key 4b186e867e31e85988806f08144215b7 is removed from metadata 2024.10.22 01:55:04.368160 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1cf7bea6984f0749af26590596df53b2, offset: 0, size: 156 2024.10.22 01:55:04.369734 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1cf7bea6984f0749af26590596df53b2 for removal 2024.10.22 01:55:04.371628 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cfb56e8559d7e962ed9a0c0dd858abd0, offset: 0, size: 168 2024.10.22 01:55:04.374907 [ 1476 ] {} CacheMetadata: Key 1cf7bea6984f0749af26590596df53b2 is removed from metadata 2024.10.22 01:55:04.475849 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cfb56e8559d7e962ed9a0c0dd858abd0 for removal 2024.10.22 01:55:04.484682 [ 1476 ] {} CacheMetadata: Key cfb56e8559d7e962ed9a0c0dd858abd0 is removed from metadata 2024.10.22 01:55:04.484937 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dc86c261a57fbabd17b420c94ad582fc, offset: 0, size: 98795 2024.10.22 01:55:04.490936 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dc86c261a57fbabd17b420c94ad582fc for removal 2024.10.22 01:55:04.500581 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 46be9160aee728007c0978aa7b525e7e, offset: 0, size: 156 2024.10.22 01:55:04.505784 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 46be9160aee728007c0978aa7b525e7e for removal 2024.10.22 01:55:04.598995 [ 1476 ] {} CacheMetadata: Key dc86c261a57fbabd17b420c94ad582fc is removed from metadata 2024.10.22 01:55:04.619992 [ 1476 ] {} CacheMetadata: Key 46be9160aee728007c0978aa7b525e7e is removed from metadata 2024.10.22 01:55:04.825187 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_676_130/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:55:05.092771 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 10745 2024.10.22 01:55:05.206875 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:05.209159 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:05.246912 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_706_706_0 (state Temporary) 2024.10.22 01:55:05.357817 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_706_706_0 to 202410_719_719_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:05.412309 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10745 2024.10.22 01:55:05.816746 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 14 file segments (while reserving 823748 bytes for cb97e6f87568c753f36a1b578923e0a7:0) 2024.10.22 01:55:05.822724 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4840100cc3626da522f517ea391342ba, offset: 0, size: 236143 2024.10.22 01:55:05.827158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4840100cc3626da522f517ea391342ba for removal 2024.10.22 01:55:05.833382 [ 1476 ] {} CacheMetadata: Key 4840100cc3626da522f517ea391342ba is removed from metadata 2024.10.22 01:55:05.835039 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d456c4c831edb17b538b834cb0bf0b66, offset: 0, size: 156 2024.10.22 01:55:05.839221 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d456c4c831edb17b538b834cb0bf0b66 for removal 2024.10.22 01:55:05.844612 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 75259b77190ec9a0153cd9bf6a607fea, offset: 0, size: 189872 2024.10.22 01:55:05.849349 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 75259b77190ec9a0153cd9bf6a607fea for removal 2024.10.22 01:55:05.854247 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 343d94b54b8e899fda7e0f91e3eddbd0, offset: 0, size: 168 2024.10.22 01:55:05.855366 [ 1476 ] {} CacheMetadata: Key 75259b77190ec9a0153cd9bf6a607fea is removed from metadata 2024.10.22 01:55:05.856542 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 343d94b54b8e899fda7e0f91e3eddbd0 for removal 2024.10.22 01:55:05.860997 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b5e13cebef581fffe9cf6c38bd327f51, offset: 0, size: 99887 2024.10.22 01:55:05.866578 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b5e13cebef581fffe9cf6c38bd327f51 for removal 2024.10.22 01:55:05.873680 [ 1476 ] {} CacheMetadata: Key b5e13cebef581fffe9cf6c38bd327f51 is removed from metadata 2024.10.22 01:55:05.869362 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bba219e69fcf3b517503ce7798f73aff, offset: 0, size: 5128 2024.10.22 01:55:05.882285 [ 1476 ] {} CacheMetadata: Key 343d94b54b8e899fda7e0f91e3eddbd0 is removed from metadata 2024.10.22 01:55:05.884109 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bba219e69fcf3b517503ce7798f73aff for removal 2024.10.22 01:55:05.889329 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fae2e44c4a9b76a5764b47dafbc31d78, offset: 0, size: 155 2024.10.22 01:55:05.893569 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fae2e44c4a9b76a5764b47dafbc31d78 for removal 2024.10.22 01:55:05.895372 [ 1476 ] {} CacheMetadata: Key bba219e69fcf3b517503ce7798f73aff is removed from metadata 2024.10.22 01:55:05.897193 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bc41f8f8a6eac82364e19e5d2e5d17fa, offset: 0, size: 151 2024.10.22 01:55:05.903550 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bc41f8f8a6eac82364e19e5d2e5d17fa for removal 2024.10.22 01:55:05.903972 [ 1476 ] {} CacheMetadata: Key fae2e44c4a9b76a5764b47dafbc31d78 is removed from metadata 2024.10.22 01:55:05.907882 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 309c07800fb5333bd67b5e0159f87e67, offset: 0, size: 155 2024.10.22 01:55:05.911220 [ 1476 ] {} CacheMetadata: Key bc41f8f8a6eac82364e19e5d2e5d17fa is removed from metadata 2024.10.22 01:55:05.914451 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 309c07800fb5333bd67b5e0159f87e67 for removal 2024.10.22 01:55:05.919627 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5b74e531ec46b3b1ddbf655acab03587, offset: 0, size: 167718 2024.10.22 01:55:05.927767 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5b74e531ec46b3b1ddbf655acab03587 for removal 2024.10.22 01:55:06.037475 [ 1476 ] {} CacheMetadata: Key 5b74e531ec46b3b1ddbf655acab03587 is removed from metadata 2024.10.22 01:55:06.048589 [ 1476 ] {} CacheMetadata: Key 309c07800fb5333bd67b5e0159f87e67 is removed from metadata 2024.10.22 01:55:06.056594 [ 1476 ] {} CacheMetadata: Key d456c4c831edb17b538b834cb0bf0b66 is removed from metadata 2024.10.22 01:55:05.936028 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9a2b340f074c0385a8e15710caf19a68, offset: 0, size: 152 2024.10.22 01:55:06.156231 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9a2b340f074c0385a8e15710caf19a68 for removal 2024.10.22 01:55:06.159606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a384f042dd5ca7b8da150104b9e6c087, offset: 0, size: 207786 2024.10.22 01:55:06.159962 [ 1476 ] {} CacheMetadata: Key 9a2b340f074c0385a8e15710caf19a68 is removed from metadata 2024.10.22 01:55:06.164704 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a384f042dd5ca7b8da150104b9e6c087 for removal 2024.10.22 01:55:06.264706 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ec94dafa83f63ecc7c4cabe107807e33, offset: 0, size: 80934 2024.10.22 01:55:06.267870 [ 1476 ] {} CacheMetadata: Key a384f042dd5ca7b8da150104b9e6c087 is removed from metadata 2024.10.22 01:55:06.267914 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ec94dafa83f63ecc7c4cabe107807e33 for removal 2024.10.22 01:55:06.269582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 79e3c28aa32a1d488688795ee4fef74e, offset: 0, size: 156 2024.10.22 01:55:06.271373 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 79e3c28aa32a1d488688795ee4fef74e for removal 2024.10.22 01:55:06.278300 [ 1476 ] {} CacheMetadata: Key 79e3c28aa32a1d488688795ee4fef74e is removed from metadata 2024.10.22 01:55:06.298038 [ 1476 ] {} CacheMetadata: Key ec94dafa83f63ecc7c4cabe107807e33 is removed from metadata 2024.10.22 01:55:06.375883 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_140_140_0, total 42 rows starting from the beginning of the part 2024.10.22 01:55:06.387846 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 1.97 GiB, free memory in arenas 11.27 MiB, will set to 1.74 GiB (RSS), difference: -4.84 MiB 2024.10.22 01:55:06.668764 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 146 entries to flush up to offset 78900 2024.10.22 01:55:07.045422 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:07.049238 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:07.229328 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 4 old parts to remove. Parts: [202410_1_655_131, 202410_656_656_0, 202410_657_657_0, 202410_658_658_0] 2024.10.22 01:55:07.248782 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from filesystem (serially): Parts: [202410_1_655_131, 202410_656_656_0, 202410_657_657_0, 202410_658_658_0] 2024.10.22 01:55:07.360277 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 6 file segments (while reserving 1042948 bytes for c4be8ad2809c561768e439069f244240:0) 2024.10.22 01:55:07.362763 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3fb19e1e9e8c322839d9675cdc55e0c7, offset: 0, size: 244085 2024.10.22 01:55:07.366575 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3fb19e1e9e8c322839d9675cdc55e0c7 for removal 2024.10.22 01:55:07.371943 [ 1476 ] {} CacheMetadata: Key 3fb19e1e9e8c322839d9675cdc55e0c7 is removed from metadata 2024.10.22 01:55:07.372556 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2f150af0b45fdf00e7453fb6edeac3b3, offset: 0, size: 156 2024.10.22 01:55:07.374823 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from memory: Parts: [202410_1_655_131, 202410_656_656_0, 202410_657_657_0, 202410_658_658_0] 2024.10.22 01:55:07.379235 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2f150af0b45fdf00e7453fb6edeac3b3 for removal 2024.10.22 01:55:07.387439 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 59d3fbede869c8eb369540af77597f14, offset: 0, size: 159 2024.10.22 01:55:07.387906 [ 1476 ] {} CacheMetadata: Key 2f150af0b45fdf00e7453fb6edeac3b3 is removed from metadata 2024.10.22 01:55:07.395226 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 59d3fbede869c8eb369540af77597f14 for removal 2024.10.22 01:55:07.400790 [ 1476 ] {} CacheMetadata: Key 59d3fbede869c8eb369540af77597f14 is removed from metadata 2024.10.22 01:55:07.412139 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 27e4540ea05c1e471ac94c868255b5c0, offset: 0, size: 375980 2024.10.22 01:55:07.416044 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 27e4540ea05c1e471ac94c868255b5c0 for removal 2024.10.22 01:55:07.425179 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ea4418313e686252a1f491ca48d9289d, offset: 0, size: 159 2024.10.22 01:55:07.425300 [ 1476 ] {} CacheMetadata: Key 27e4540ea05c1e471ac94c868255b5c0 is removed from metadata 2024.10.22 01:55:07.432927 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ea4418313e686252a1f491ca48d9289d for removal 2024.10.22 01:55:07.437104 [ 1476 ] {} CacheMetadata: Key ea4418313e686252a1f491ca48d9289d is removed from metadata 2024.10.22 01:55:07.438662 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cb645196d58ef917b4115275a48c15d4, offset: 0, size: 383691 2024.10.22 01:55:07.446888 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cb645196d58ef917b4115275a48c15d4 for removal 2024.10.22 01:55:07.453226 [ 1476 ] {} CacheMetadata: Key cb645196d58ef917b4115275a48c15d4 is removed from metadata 2024.10.22 01:55:07.486168 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:07.495054 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:07.604204 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1190_1190_0 (state Temporary) 2024.10.22 01:55:07.673647 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1190_1190_0 to 202410_1203_1203_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:07.921780 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 78900 2024.10.22 01:55:08.321373 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_141_141_0, total 43 rows starting from the beginning of the part 2024.10.22 01:55:08.326918 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 6 file segments (while reserving 576010 bytes for dec16703181e81e9d740fcc221456846:0) 2024.10.22 01:55:08.333188 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f468a7592ec8b6fcdd56c5bb7226ab2f, offset: 0, size: 200916 2024.10.22 01:55:08.438248 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f468a7592ec8b6fcdd56c5bb7226ab2f for removal 2024.10.22 01:55:08.442712 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cc1f09e8f7910684b64b60f54d945318, offset: 0, size: 155 2024.10.22 01:55:08.448527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cc1f09e8f7910684b64b60f54d945318 for removal 2024.10.22 01:55:08.452504 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 053539c87f76a5b92c2577ebe8b00041, offset: 0, size: 156 2024.10.22 01:55:08.442434 [ 1476 ] {} CacheMetadata: Key f468a7592ec8b6fcdd56c5bb7226ab2f is removed from metadata 2024.10.22 01:55:08.456363 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 053539c87f76a5b92c2577ebe8b00041 for removal 2024.10.22 01:55:08.458881 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d569e2662c71843d56cd3bea27686085, offset: 0, size: 268627 2024.10.22 01:55:08.462510 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d569e2662c71843d56cd3bea27686085 for removal 2024.10.22 01:55:08.463122 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 1.97 GiB, free memory in arenas 9.93 MiB, will set to 1.75 GiB (RSS), difference: -8.88 MiB 2024.10.22 01:55:08.465824 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 71c5001b136e57e26a419ba39b6904a5, offset: 0, size: 160601 2024.10.22 01:55:08.469639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 71c5001b136e57e26a419ba39b6904a5 for removal 2024.10.22 01:55:08.469758 [ 1476 ] {} CacheMetadata: Key d569e2662c71843d56cd3bea27686085 is removed from metadata 2024.10.22 01:55:08.472287 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0df195ac6f24d9cb7331a82989af18bc, offset: 0, size: 155 2024.10.22 01:55:08.476274 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0df195ac6f24d9cb7331a82989af18bc for removal 2024.10.22 01:55:08.476875 [ 1476 ] {} CacheMetadata: Key 71c5001b136e57e26a419ba39b6904a5 is removed from metadata 2024.10.22 01:55:08.489008 [ 1476 ] {} CacheMetadata: Key 0df195ac6f24d9cb7331a82989af18bc is removed from metadata 2024.10.22 01:55:08.496909 [ 1476 ] {} CacheMetadata: Key 053539c87f76a5b92c2577ebe8b00041 is removed from metadata 2024.10.22 01:55:08.508084 [ 1476 ] {} CacheMetadata: Key cc1f09e8f7910684b64b60f54d945318 is removed from metadata 2024.10.22 01:55:08.518393 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 113 entries to flush up to offset 78425 2024.10.22 01:55:08.665014 [ 1277 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 252 ms. 2024.10.22 01:55:08.860682 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:08.863906 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:08.984056 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_694_694_0 (state Temporary) 2024.10.22 01:55:09.140051 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1101_1, 202410_1102_1102_0, 202410_1103_1103_0, 202410_1104_1104_0, 202410_1105_1105_0, 202410_1106_1106_0] 2024.10.22 01:55:09.145325 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1101_1, 202410_1102_1102_0, 202410_1103_1103_0, 202410_1104_1104_0, 202410_1105_1105_0, 202410_1106_1106_0] 2024.10.22 01:55:09.147125 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_694_694_0 to 202410_700_700_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:09.191569 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1101_1, 202410_1102_1102_0, 202410_1103_1103_0, 202410_1104_1104_0, 202410_1105_1105_0, 202410_1106_1106_0] 2024.10.22 01:55:09.200631 [ 1157 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_695_45 to 202410_696_696_0 2024.10.22 01:55:09.210669 [ 1157 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 01:55:09.215087 [ 1157 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:09.232043 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_696_46} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_695_45 to 202410_696_696_0 into Compact with storage Full 2024.10.22 01:55:09.262680 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_696_46} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:09.269589 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_696_46} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_695_45, total 18776 rows starting from the beginning of the part 2024.10.22 01:55:09.294931 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_696_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_696_696_0, total 92 rows starting from the beginning of the part 2024.10.22 01:55:09.338768 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 78425 2024.10.22 01:55:09.769223 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_142_142_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:55:09.816596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_9_9_0 to 201403_9_9_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:09.858512 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_10_10_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:55:09.864348 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_142_142_0, total 45 rows starting from the beginning of the part 2024.10.22 01:55:10.155813 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_676_130} MergeTask::MergeProjectionsStage: Merge sorted 1031616 rows, containing 4 columns (4 merged, 0 gathered) in 32.597643642 sec., 31646.9500473595 rows/sec., 474.60 KiB/sec. 2024.10.22 01:55:10.236801 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_676_130} MergedBlockOutputStream: filled checksums 202410_1_676_130 (state Temporary) 2024.10.22 01:55:10.259508 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 15 file segments (while reserving 1048576 bytes for 63ee9bf5d16cc4b7293c2d3c556ac111:0) 2024.10.22 01:55:10.261598 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 32be5fd6785173462ccc2e0ae646d8b4, offset: 0, size: 189 2024.10.22 01:55:10.267282 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 32be5fd6785173462ccc2e0ae646d8b4 for removal 2024.10.22 01:55:10.269698 [ 1476 ] {} CacheMetadata: Key 32be5fd6785173462ccc2e0ae646d8b4 is removed from metadata 2024.10.22 01:55:10.370877 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c600b434a339d3c787519eb6373c96d8, offset: 0, size: 192987 2024.10.22 01:55:10.374355 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c600b434a339d3c787519eb6373c96d8 for removal 2024.10.22 01:55:10.377773 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cb97e6f87568c753f36a1b578923e0a7, offset: 0, size: 823748 2024.10.22 01:55:10.385687 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cb97e6f87568c753f36a1b578923e0a7 for removal 2024.10.22 01:55:10.392893 [ 1476 ] {} CacheMetadata: Key c600b434a339d3c787519eb6373c96d8 is removed from metadata 2024.10.22 01:55:10.400119 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 578bf39e88810c94973b420e51254111, offset: 0, size: 220545 2024.10.22 01:55:10.409142 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 578bf39e88810c94973b420e51254111 for removal 2024.10.22 01:55:10.412969 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 30b853e98636e1009354fb2cbc11fafd, offset: 0, size: 153 2024.10.22 01:55:10.412844 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1029576 2024.10.22 01:55:10.414752 [ 1476 ] {} CacheMetadata: Key 578bf39e88810c94973b420e51254111 is removed from metadata 2024.10.22 01:55:10.420555 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 30b853e98636e1009354fb2cbc11fafd for removal 2024.10.22 01:55:10.434477 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2c2e5973d1e626a3513037e3411d35ea, offset: 0, size: 156 2024.10.22 01:55:10.440174 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2c2e5973d1e626a3513037e3411d35ea for removal 2024.10.22 01:55:10.443480 [ 1476 ] {} CacheMetadata: Key 30b853e98636e1009354fb2cbc11fafd is removed from metadata 2024.10.22 01:55:10.444961 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d666e79928e63f79c93aaf2dab26e67a, offset: 0, size: 1300 2024.10.22 01:55:10.448185 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d666e79928e63f79c93aaf2dab26e67a for removal 2024.10.22 01:55:10.452868 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 35f9996b6a19eecc726a58b736d2da1e, offset: 0, size: 156 2024.10.22 01:55:10.453525 [ 1476 ] {} CacheMetadata: Key d666e79928e63f79c93aaf2dab26e67a is removed from metadata 2024.10.22 01:55:10.455389 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 35f9996b6a19eecc726a58b736d2da1e for removal 2024.10.22 01:55:10.459034 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 102f4ca8ad34b70ef1d314be5a7e0c45, offset: 0, size: 96140 2024.10.22 01:55:10.460756 [ 1476 ] {} CacheMetadata: Key 35f9996b6a19eecc726a58b736d2da1e is removed from metadata 2024.10.22 01:55:10.463956 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 102f4ca8ad34b70ef1d314be5a7e0c45 for removal 2024.10.22 01:55:10.467894 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ce65623487ea851c6803d2841623d8a5, offset: 0, size: 155 2024.10.22 01:55:10.473455 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ce65623487ea851c6803d2841623d8a5 for removal 2024.10.22 01:55:10.475668 [ 1476 ] {} CacheMetadata: Key 102f4ca8ad34b70ef1d314be5a7e0c45 is removed from metadata 2024.10.22 01:55:10.477237 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ef8c1e31091a4f54a71c204140547630, offset: 0, size: 153 2024.10.22 01:55:10.504330 [ 1476 ] {} CacheMetadata: Key ce65623487ea851c6803d2841623d8a5 is removed from metadata 2024.10.22 01:55:10.527785 [ 1476 ] {} CacheMetadata: Key 2c2e5973d1e626a3513037e3411d35ea is removed from metadata 2024.10.22 01:55:10.546176 [ 1476 ] {} CacheMetadata: Key cb97e6f87568c753f36a1b578923e0a7 is removed from metadata 2024.10.22 01:55:10.583025 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ef8c1e31091a4f54a71c204140547630 for removal 2024.10.22 01:55:10.590088 [ 1476 ] {} CacheMetadata: Key ef8c1e31091a4f54a71c204140547630 is removed from metadata 2024.10.22 01:55:10.594996 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e3a90cc8cb2b8c72ea6309c60aa42ccd, offset: 0, size: 113057 2024.10.22 01:55:10.605350 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 1.97 GiB, free memory in arenas 13.60 MiB, will set to 1.76 GiB (RSS), difference: 277.97 MiB 2024.10.22 01:55:10.701184 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e3a90cc8cb2b8c72ea6309c60aa42ccd for removal 2024.10.22 01:55:10.704440 [ 1476 ] {} CacheMetadata: Key e3a90cc8cb2b8c72ea6309c60aa42ccd is removed from metadata 2024.10.22 01:55:10.704858 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 48b57aef463227eb185c7c598870bb73, offset: 0, size: 112548 2024.10.22 01:55:10.709756 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 48b57aef463227eb185c7c598870bb73 for removal 2024.10.22 01:55:10.713576 [ 1476 ] {} CacheMetadata: Key 48b57aef463227eb185c7c598870bb73 is removed from metadata 2024.10.22 01:55:10.714162 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dec86d0a84ee0d3314c3e4d6452a76b5, offset: 0, size: 152 2024.10.22 01:55:10.717298 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dec86d0a84ee0d3314c3e4d6452a76b5 for removal 2024.10.22 01:55:10.728863 [ 1476 ] {} CacheMetadata: Key dec86d0a84ee0d3314c3e4d6452a76b5 is removed from metadata 2024.10.22 01:55:10.728921 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d947a60e38f481d307bff52a796a21de, offset: 0, size: 4 2024.10.22 01:55:10.734602 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d947a60e38f481d307bff52a796a21de for removal 2024.10.22 01:55:10.738839 [ 1476 ] {} CacheMetadata: Key d947a60e38f481d307bff52a796a21de is removed from metadata 2024.10.22 01:55:10.960987 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_696_46} MergeTask::MergeProjectionsStage: Merge sorted 18868 rows, containing 13 columns (13 merged, 0 gathered) in 1.736840743 sec., 10863.402459922603 rows/sec., 4.70 MiB/sec. 2024.10.22 01:55:10.969772 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:55:10.972894 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:55:11.055271 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_142_142_0 (state Temporary) 2024.10.22 01:55:11.159193 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_696_46} MergedBlockOutputStream: filled checksums 202410_507_696_46 (state Temporary) 2024.10.22 01:55:11.346561 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:11.350184 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:11.360482 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_696_46} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_696_46 to 202410_507_696_46 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:11.362983 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_696_46} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_695_45, 202410_696_696_0] -> 202410_507_696_46 2024.10.22 01:55:11.419896 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 42 2024.10.22 01:55:11.484775 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.80 MiB. 2024.10.22 01:55:11.487821 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_677_677_0 (state Temporary) 2024.10.22 01:55:11.570638 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_677_677_0 to 202410_684_684_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:11.633435 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1029576 2024.10.22 01:55:11.694194 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:11.699253 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:11.849103 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 590 2024.10.22 01:55:11.854948 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_36_36_0 (state Temporary) 2024.10.22 01:55:11.942002 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 140 entries to flush up to offset 79040 2024.10.22 01:55:11.945119 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_36_36_0 to 202410_36_36_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:12.041735 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 42 2024.10.22 01:55:12.083649 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_143_143_0, total 43 rows starting from the beginning of the part 2024.10.22 01:55:12.380386 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:12.394994 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:12.474247 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:12.478757 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.80 GiB. 2024.10.22 01:55:12.552482 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 1.97 GiB, free memory in arenas 9.75 MiB, will set to 1.49 GiB (RSS), difference: -9.90 MiB 2024.10.22 01:55:12.561311 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_572_572_0 (state Temporary) 2024.10.22 01:55:12.801205 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1191_1191_0 (state Temporary) 2024.10.22 01:55:12.920400 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 10754 2024.10.22 01:55:12.946235 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_572_572_0 to 202410_582_582_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:12.951146 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_676_130} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_676_130 to 202410_1_676_130 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:12.954220 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_676_130} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_670_129, 202410_671_676_1] -> 202410_1_676_130 2024.10.22 01:55:12.963272 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1191_1191_0 to 202410_1204_1204_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:12.973384 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:55:13.029670 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79040 2024.10.22 01:55:13.051622 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 590 2024.10.22 01:55:13.083463 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:13.098446 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:13.153206 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_707_707_0 (state Temporary) 2024.10.22 01:55:13.288741 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_707_707_0 to 202410_720_720_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:13.423815 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 4 parts from 202410_1_715_146 to 202410_718_718_0 2024.10.22 01:55:13.442369 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:13.445781 [ 1220 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.81 GiB. 2024.10.22 01:55:13.470918 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_718_147} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_715_146 to 202410_718_718_0 into Compact with storage Full 2024.10.22 01:55:13.543945 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10754 2024.10.22 01:55:13.556341 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_718_147} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:13.662808 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_718_147} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_715_146, total 10781 rows starting from the beginning of the part 2024.10.22 01:55:13.675459 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_718_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_716_716_0, total 23 rows starting from the beginning of the part 2024.10.22 01:55:13.738274 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_718_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_717_717_0, total 16 rows starting from the beginning of the part 2024.10.22 01:55:13.757315 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_718_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_718_718_0, total 16 rows starting from the beginning of the part 2024.10.22 01:55:14.147311 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_528_108, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0, 202410_532_532_0, 202410_533_533_0] 2024.10.22 01:55:14.149575 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_528_108, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0, 202410_532_532_0, 202410_533_533_0] 2024.10.22 01:55:14.183849 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_528_108, 202410_529_529_0, 202410_530_530_0, 202410_531_531_0, 202410_532_532_0, 202410_533_533_0] 2024.10.22 01:55:14.289426 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_144_144_0, total 47 rows starting from the beginning of the part 2024.10.22 01:55:14.442168 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 63ee9bf5d16cc4b7293c2d3c556ac111:0) 2024.10.22 01:55:14.448808 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c4be8ad2809c561768e439069f244240, offset: 0, size: 1042948 2024.10.22 01:55:14.454778 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c4be8ad2809c561768e439069f244240 for removal 2024.10.22 01:55:14.457430 [ 1476 ] {} CacheMetadata: Key c4be8ad2809c561768e439069f244240 is removed from metadata 2024.10.22 01:55:14.462272 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 634282541c46a755764bf3143b0b68b5, offset: 0, size: 192 2024.10.22 01:55:14.467789 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 634282541c46a755764bf3143b0b68b5 for removal 2024.10.22 01:55:14.578350 [ 1476 ] {} CacheMetadata: Key 634282541c46a755764bf3143b0b68b5 is removed from metadata 2024.10.22 01:55:14.748528 [ 1211 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:55:14.778832 [ 1211 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:55:14.811163 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 1.97 GiB, free memory in arenas 7.23 MiB, will set to 1.52 GiB (RSS), difference: -63.13 MiB 2024.10.22 01:55:15.138839 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/yzg/lybmtyaukyezpjwhkeryghzsrauqc, upload id 7c633de3-ee5e-4b70-8aa7-5cba34ff7a5f 2024.10.22 01:55:15.250827 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_718_147} MergeTask::MergeProjectionsStage: Merge sorted 10836 rows, containing 12 columns (12 merged, 0 gathered) in 1.785809425 sec., 6067.836717795349 rows/sec., 966.37 KiB/sec. 2024.10.22 01:55:15.402960 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_718_147} MergedBlockOutputStream: filled checksums 202410_1_718_147 (state Temporary) 2024.10.22 01:55:15.951819 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_718_147} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_718_147 to 202410_1_718_147 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:15.958269 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_718_147} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 4 parts: [202410_1_715_146, 202410_718_718_0] -> 202410_1_718_147 2024.10.22 01:55:16.017324 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.49 MiB. 2024.10.22 01:55:16.542833 [ 1233 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 271 ms. 2024.10.22 01:55:16.652056 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 1.97 GiB, free memory in arenas 1.99 MiB, will set to 1.54 GiB (RSS), difference: 16.91 MiB 2024.10.22 01:55:16.853695 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 113 entries to flush up to offset 78538 2024.10.22 01:55:17.063517 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 79096 2024.10.22 01:55:17.155913 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:17.166853 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.71 GiB. 2024.10.22 01:55:17.238132 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_695_695_0 (state Temporary) 2024.10.22 01:55:17.299814 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_695_695_0 to 202410_701_701_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:17.347648 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:17.351277 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.71 GiB. 2024.10.22 01:55:17.360422 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_696_46 to 202410_697_697_0 2024.10.22 01:55:17.367889 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.12 MiB using storage policy from min volume index 0 2024.10.22 01:55:17.399656 [ 1254 ] {} DiskLocal: Reserved 1.12 MiB on local disk `default`, having unreserved 296.71 GiB. 2024.10.22 01:55:17.464375 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_697_47} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_696_46 to 202410_697_697_0 into Compact with storage Full 2024.10.22 01:55:17.555644 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1192_1192_0 (state Temporary) 2024.10.22 01:55:17.556846 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 78538 2024.10.22 01:55:17.652944 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_697_47} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:17.670307 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_697_47} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_696_46, total 18868 rows starting from the beginning of the part 2024.10.22 01:55:17.771360 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_697_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_697_697_0, total 120 rows starting from the beginning of the part 2024.10.22 01:55:17.849533 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1192_1192_0 to 202410_1205_1205_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:17.927565 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79096 2024.10.22 01:55:18.041108 [ 1215 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 206 ms. 2024.10.22 01:55:18.169763 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_623_623_0, 202410_624_624_0, 202410_625_625_0, 202410_626_626_0, 202410_627_627_0, 202410_628_628_0] 2024.10.22 01:55:18.172192 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_623_623_0, 202410_624_624_0, 202410_625_625_0, 202410_626_626_0, 202410_627_627_0, 202410_628_628_0] 2024.10.22 01:55:18.239127 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_623_623_0, 202410_624_624_0, 202410_625_625_0, 202410_626_626_0, 202410_627_627_0, 202410_628_628_0] 2024.10.22 01:55:18.545314 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 1.97 GiB, free memory in arenas 1.53 MiB, will set to 1.54 GiB (RSS), difference: -89.75 MiB 2024.10.22 01:55:18.647783 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1031116 2024.10.22 01:55:19.613665 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:19.666711 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.71 GiB. 2024.10.22 01:55:19.751262 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_697_47} MergeTask::MergeProjectionsStage: Merge sorted 18988 rows, containing 13 columns (13 merged, 0 gathered) in 2.294140416 sec., 8276.738366828895 rows/sec., 3.58 MiB/sec. 2024.10.22 01:55:19.905647 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_697_47} MergedBlockOutputStream: filled checksums 202410_507_697_47 (state Temporary) 2024.10.22 01:55:19.948768 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_678_678_0 (state Temporary) 2024.10.22 01:55:20.156900 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_678_678_0 to 202410_685_685_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:20.299884 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1031116 2024.10.22 01:55:20.352008 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_697_47} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_697_47 to 202410_507_697_47 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:20.455716 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_697_47} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_696_46, 202410_697_697_0] -> 202410_507_697_47 2024.10.22 01:55:20.569721 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 1.97 GiB, free memory in arenas 2.38 MiB, will set to 1.58 GiB (RSS), difference: -4.50 MiB 2024.10.22 01:55:20.581230 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 01:55:20.595342 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.84 MiB. 2024.10.22 01:55:20.969769 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:55:20.977116 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:55:21.048247 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 10771 2024.10.22 01:55:21.265022 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:21.273630 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.71 GiB. 2024.10.22 01:55:21.328358 [ 1261 ] {} DatabaseCatalog: Execution took 234 ms. 2024.10.22 01:55:21.434210 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_708_708_0 (state Temporary) 2024.10.22 01:55:21.600131 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_708_708_0 to 202410_721_721_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:21.670454 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10771 2024.10.22 01:55:21.934797 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 79141 2024.10.22 01:55:22.195718 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:22.206612 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.71 GiB. 2024.10.22 01:55:22.399267 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 1.97 GiB, free memory in arenas 2.55 MiB, will set to 1.61 GiB (RSS), difference: -24.73 MiB 2024.10.22 01:55:22.453226 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1193_1193_0 (state Temporary) 2024.10.22 01:55:22.513655 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1193_1193_0 to 202410_1206_1206_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:22.550269 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1096_1201_21 to 202410_1206_1206_0 2024.10.22 01:55:22.557361 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:22.559506 [ 1244 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.71 GiB. 2024.10.22 01:55:22.573587 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} MergeTask::PrepareStage: Merging 6 parts: from 202410_1096_1201_21 to 202410_1206_1206_0 into Compact with storage Full 2024.10.22 01:55:22.661633 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:22.678083 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1096_1201_21, total 8082 rows starting from the beginning of the part 2024.10.22 01:55:22.691708 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79141 2024.10.22 01:55:22.835841 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(63ee9bf5d16cc4b7293c2d3c556ac111) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 63ee9bf5d16cc4b7293c2d3c556ac111, state: DOWNLOADING, downloaded size: 2097152, reserved size: 2097152, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 2097152, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:55:22.842768 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1202_1202_0, total 96 rows starting from the beginning of the part 2024.10.22 01:55:22.858520 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 2097152, file segment info: File segment: [0, 33554431], key: 63ee9bf5d16cc4b7293c2d3c556ac111, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 2097152, reserved size: 2097152, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, current write offset: 2097152, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1393, kind: Regular, unbound: 0 2024.10.22 01:55:22.871850 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:55:22.878459 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1203_1203_0, total 146 rows starting from the beginning of the part 2024.10.22 01:55:22.934535 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1204_1204_0, total 140 rows starting from the beginning of the part 2024.10.22 01:55:22.973184 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1205_1205_0, total 56 rows starting from the beginning of the part 2024.10.22 01:55:23.032590 [ 1419 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_10_10_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:55:23.052906 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1206_1206_0, total 45 rows starting from the beginning of the part 2024.10.22 01:55:23.197031 [ 1686 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fgz/jskzccfiorchjqwhpkvotonuaguwp, size 4260536 2024.10.22 01:55:23.229969 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_142_142_0 to 202410_145_145_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:23.294142 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_144_25/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:55:23.301757 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_638_27, 202410_639_639_0, 202410_640_640_0, 202410_641_641_0, 202410_642_642_0, 202410_643_643_0] 2024.10.22 01:55:23.306883 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_638_27, 202410_639_639_0, 202410_640_640_0, 202410_641_641_0, 202410_642_642_0, 202410_643_643_0] 2024.10.22 01:55:23.458432 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_638_27, 202410_639_639_0, 202410_640_640_0, 202410_641_641_0, 202410_642_642_0, 202410_643_643_0] 2024.10.22 01:55:23.502499 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:55:23.631757 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:55:23.645594 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:55:23.658219 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:55:23.673912 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:55:23.704227 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:55:23.822437 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:55:23.849199 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 01:55:24.016841 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vao/dixvkvueutjsuygphjsxsuqexdgrl (LogSeriesLimiter: on interval from 2024-10-22 01:55:03 to 2024-10-22 01:55:23 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:55:24.456682 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 1.97 GiB, free memory in arenas 2.46 MiB, will set to 1.63 GiB (RSS), difference: -73.65 MiB 2024.10.22 01:55:24.532622 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} MergeTask::MergeProjectionsStage: Merge sorted 8565 rows, containing 13 columns (13 merged, 0 gathered) in 1.967090435 sec., 4354.146534193279 rows/sec., 732.82 KiB/sec. 2024.10.22 01:55:24.695453 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} MergedBlockOutputStream: filled checksums 202410_1096_1206_22 (state Temporary) 2024.10.22 01:55:24.961333 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1096_1206_22 to 202410_1096_1206_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:24.964951 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1096_1206_22} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1096_1201_21, 202410_1206_1206_0] -> 202410_1096_1206_22 2024.10.22 01:55:24.995632 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.65 MiB. 2024.10.22 01:55:25.084776 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 107 entries to flush up to offset 78645 2024.10.22 01:55:25.388869 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:25.395243 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:55:25.454410 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_696_696_0 (state Temporary) 2024.10.22 01:55:25.507402 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_696_696_0 to 202410_702_702_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:25.553139 [ 1195 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_697_47 to 202410_698_698_0 2024.10.22 01:55:25.570865 [ 1195 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.12 MiB using storage policy from min volume index 0 2024.10.22 01:55:25.574947 [ 1195 ] {} DiskLocal: Reserved 1.12 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:55:25.641031 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_698_48} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_697_47 to 202410_698_698_0 into Compact with storage Full 2024.10.22 01:55:25.657004 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 78645 2024.10.22 01:55:25.702918 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_698_48} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:25.707741 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_698_48} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_697_47, total 18988 rows starting from the beginning of the part 2024.10.22 01:55:25.736582 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_698_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_698_698_0, total 84 rows starting from the beginning of the part 2024.10.22 01:55:26.248954 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5708 2024.10.22 01:55:26.439459 [ 1251 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 01:55:26.590514 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 1.97 GiB, free memory in arenas 4.76 MiB, will set to 1.69 GiB (RSS), difference: -50.25 MiB 2024.10.22 01:55:26.704771 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 79200 2024.10.22 01:55:26.877611 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:26.879925 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_698_48} MergeTask::MergeProjectionsStage: Merge sorted 19072 rows, containing 13 columns (13 merged, 0 gathered) in 1.262561802 sec., 15105.79519338254 rows/sec., 6.54 MiB/sec. 2024.10.22 01:55:26.881413 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:55:26.956069 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1194_1194_0 (state Temporary) 2024.10.22 01:55:27.098860 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1194_1194_0 to 202410_1207_1207_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:27.172255 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_698_48} MergedBlockOutputStream: filled checksums 202410_507_698_48 (state Temporary) 2024.10.22 01:55:27.268674 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79200 2024.10.22 01:55:27.316074 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1033041 2024.10.22 01:55:27.491451 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_698_48} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_698_48 to 202410_507_698_48 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:27.498781 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_698_48} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_697_47, 202410_698_698_0] -> 202410_507_698_48 2024.10.22 01:55:27.592711 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.80 MiB. 2024.10.22 01:55:27.768578 [ 1190 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 01:55:28.063088 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 591 2024.10.22 01:55:28.402483 [ 1170 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 288 ms. 2024.10.22 01:55:28.548358 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:28.551328 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:55:28.731212 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_573_573_0 (state Temporary) 2024.10.22 01:55:29.016120 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_573_573_0 to 202410_583_583_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:29.073498 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:29.088050 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 01:55:29.101304 [ 1178 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_578_118 to 202410_583_583_0 2024.10.22 01:55:29.172047 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 10787 2024.10.22 01:55:29.179289 [ 1178 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:29.187845 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:55:29.239914 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 2.00 GiB, peak 2.00 GiB, free memory in arenas 4.39 MiB, will set to 1.83 GiB (RSS), difference: -173.33 MiB 2024.10.22 01:55:29.307705 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_578_118 to 202410_583_583_0 into Compact with storage Full 2024.10.22 01:55:29.434480 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 591 2024.10.22 01:55:29.467653 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:29.470271 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:55:29.519795 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_679_679_0 (state Temporary) 2024.10.22 01:55:29.642286 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_709_709_0 (state Temporary) 2024.10.22 01:55:29.759427 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:29.788341 [ 1191 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:55:29.816144 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_709_709_0 to 202410_722_722_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:29.921438 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_578_118, total 622 rows starting from the beginning of the part 2024.10.22 01:55:29.952235 [ 1191 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:55:29.985017 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10787 2024.10.22 01:55:30.033727 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_679_679_0 to 202410_686_686_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:30.138305 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1033041 2024.10.22 01:55:30.137239 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_676_130 to 202410_677_682_1 2024.10.22 01:55:30.155975 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:30.160713 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:55:30.161332 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_579_579_0, total 1 rows starting from the beginning of the part 2024.10.22 01:55:30.459374 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_580_580_0, total 1 rows starting from the beginning of the part 2024.10.22 01:55:30.465376 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_682_131} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_676_130 to 202410_677_682_1 into Wide with storage Full 2024.10.22 01:55:30.472611 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_682_131} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:30.477140 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_682_131} MergeTreeSequentialSource: Reading 127 marks from part 202410_1_676_130, total 1031616 rows starting from the beginning of the part 2024.10.22 01:55:30.530517 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_682_131} MergeTreeSequentialSource: Reading 3 marks from part 202410_677_682_1, total 8855 rows starting from the beginning of the part 2024.10.22 01:55:30.555672 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_581_581_0, total 1 rows starting from the beginning of the part 2024.10.22 01:55:30.755452 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_582_582_0, total 1 rows starting from the beginning of the part 2024.10.22 01:55:30.955027 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_583_583_0, total 1 rows starting from the beginning of the part 2024.10.22 01:55:30.994763 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:55:31.016536 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:55:31.286294 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 79254 2024.10.22 01:55:31.402687 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_658_132, 202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0, 202410_663_663_0] 2024.10.22 01:55:31.405733 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_658_132, 202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0, 202410_663_663_0] 2024.10.22 01:55:31.459139 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 2.00 GiB, free memory in arenas 7.34 MiB, will set to 1.57 GiB (RSS), difference: 284.94 MiB 2024.10.22 01:55:31.534509 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_658_132, 202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0, 202410_663_663_0] 2024.10.22 01:55:31.553402 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:31.555874 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 01:55:31.708178 [ 1193 ] {} DatabaseCatalog: Execution took 385 ms. 2024.10.22 01:55:31.845130 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1195_1195_0 (state Temporary) 2024.10.22 01:55:31.882520 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1195_1195_0 to 202410_1208_1208_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:31.924394 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Referer 2024.10.22 01:55:31.965955 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79254 2024.10.22 01:55:32.136117 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Referer 2024.10.22 01:55:32.147590 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Referer 2024.10.22 01:55:32.166326 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Referer 2024.10.22 01:55:32.257129 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1106_2, 202410_1107_1107_0, 202410_1108_1108_0, 202410_1109_1109_0, 202410_1110_1110_0, 202410_1111_1111_0] 2024.10.22 01:55:32.260126 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1106_2, 202410_1107_1107_0, 202410_1108_1108_0, 202410_1109_1109_0, 202410_1110_1110_0, 202410_1111_1111_0] 2024.10.22 01:55:32.316656 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Referer 2024.10.22 01:55:32.330787 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Referer 2024.10.22 01:55:32.398071 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1106_2, 202410_1107_1107_0, 202410_1108_1108_0, 202410_1109_1109_0, 202410_1110_1110_0, 202410_1111_1111_0] 2024.10.22 01:55:32.448912 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Referer 2024.10.22 01:55:32.599458 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vhz/rzjqegybqeudsumpyaenjsgosdqmc (LogSeriesLimiter: on interval from 2024-10-22 01:55:23 to 2024-10-22 01:55:32 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:55:32.809705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 489.89 MiB using storage policy from min volume index 0 2024.10.22 01:55:33.166669 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 139 entries to flush up to offset 78784 2024.10.22 01:55:33.357504 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} MergeTask::MergeProjectionsStage: Merge sorted 627 rows, containing 32 columns (32 merged, 0 gathered) in 4.056013008 sec., 154.58530304595118 rows/sec., 14.09 KiB/sec. 2024.10.22 01:55:33.371804 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 497c362e0eabd9dd8b5457ecb4ead15c:0) 2024.10.22 01:55:33.382396 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} MergedBlockOutputStream: filled checksums 202410_1_583_119 (state Temporary) 2024.10.22 01:55:33.489012 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 10.19 MiB, will set to 1.56 GiB (RSS), difference: -1.57 MiB 2024.10.22 01:55:33.491349 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dec16703181e81e9d740fcc221456846, offset: 0, size: 576010 2024.10.22 01:55:33.495410 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dec16703181e81e9d740fcc221456846 for removal 2024.10.22 01:55:33.497590 [ 1476 ] {} CacheMetadata: Key dec16703181e81e9d740fcc221456846 is removed from metadata 2024.10.22 01:55:33.593079 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_682_131/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:55:33.599164 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f0527c5f57d361ac9af116a121c6747f, offset: 0, size: 163 2024.10.22 01:55:33.601784 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f0527c5f57d361ac9af116a121c6747f for removal 2024.10.22 01:55:33.631618 [ 1476 ] {} CacheMetadata: Key f0527c5f57d361ac9af116a121c6747f is removed from metadata 2024.10.22 01:55:33.643753 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_583_119 to 202410_1_583_119 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:33.642331 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_616_120, 202410_617_622_1] 2024.10.22 01:55:33.647339 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_583_119} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_578_118, 202410_583_583_0] -> 202410_1_583_119 2024.10.22 01:55:33.648228 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_616_120, 202410_617_622_1] 2024.10.22 01:55:33.667786 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_616_120, 202410_617_622_1] 2024.10.22 01:55:33.695092 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:55:33.755765 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:33.771167 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:33.793681 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 44 entries to flush up to offset 5752 2024.10.22 01:55:33.936403 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_697_697_0 (state Temporary) 2024.10.22 01:55:34.287623 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_697_697_0 to 202410_703_703_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:34.336743 [ 1214 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_698_48 to 202410_699_699_0 2024.10.22 01:55:34.352115 [ 1214 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.13 MiB using storage policy from min volume index 0 2024.10.22 01:55:34.355774 [ 1214 ] {} DiskLocal: Reserved 1.13 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:34.394872 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_699_49} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_698_48 to 202410_699_699_0 into Compact with storage Full 2024.10.22 01:55:34.512230 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 78784 2024.10.22 01:55:34.663574 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_699_49} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:34.671566 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_699_49} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_698_48, total 19072 rows starting from the beginning of the part 2024.10.22 01:55:34.817506 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_699_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_699_699_0, total 115 rows starting from the beginning of the part 2024.10.22 01:55:35.543016 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 2.00 GiB, free memory in arenas 15.87 MiB, will set to 1.59 GiB (RSS), difference: -121.42 MiB 2024.10.22 01:55:35.832072 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(c259d95559a99dfe19648d2ad76fa9c4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c259d95559a99dfe19648d2ad76fa9c4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:55:35.887578 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: c259d95559a99dfe19648d2ad76fa9c4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:55:35.895616 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:55:36.036863 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 79312 2024.10.22 01:55:36.406942 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:36.413025 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:36.466138 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_699_49} MergeTask::MergeProjectionsStage: Merge sorted 19187 rows, containing 13 columns (13 merged, 0 gathered) in 2.087996256 sec., 9189.192722383887 rows/sec., 3.98 MiB/sec. 2024.10.22 01:55:36.474673 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1196_1196_0 (state Temporary) 2024.10.22 01:55:36.539443 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1196_1196_0 to 202410_1209_1209_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:36.545407 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_699_49} MergedBlockOutputStream: filled checksums 202410_507_699_49 (state Temporary) 2024.10.22 01:55:36.604408 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_699_49} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_699_49 to 202410_507_699_49 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:36.612423 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_699_49} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_698_48, 202410_699_699_0] -> 202410_507_699_49 2024.10.22 01:55:36.659505 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.84 MiB. 2024.10.22 01:55:36.848632 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79312 2024.10.22 01:55:36.942740 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 592 2024.10.22 01:55:37.012059 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/efp/etamwtqeurlurppiwqagwnfrlscle (LogSeriesLimiter: on interval from 2024-10-22 01:55:32 to 2024-10-22 01:55:37 accepted series 1 / 150 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:55:37.150649 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 1034583 2024.10.22 01:55:37.489036 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 10802 2024.10.22 01:55:37.572689 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 2.00 GiB, free memory in arenas 13.25 MiB, will set to 1.59 GiB (RSS), difference: -142.91 MiB 2024.10.22 01:55:37.631799 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:37.635571 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:37.701675 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:37.703985 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:37.757589 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_710_710_0 (state Temporary) 2024.10.22 01:55:37.809188 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_574_574_0 (state Temporary) 2024.10.22 01:55:38.003499 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:38.006664 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:38.031518 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_710_710_0 to 202410_723_723_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:38.063618 [ 1250 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 4 parts from 202410_1_718_147 to 202410_721_721_0 2024.10.22 01:55:38.087828 [ 1250 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:38.096190 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:38.147526 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_721_148} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_718_147 to 202410_721_721_0 into Compact with storage Full 2024.10.22 01:55:38.148935 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_680_680_0 (state Temporary) 2024.10.22 01:55:38.156471 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_574_574_0 to 202410_584_584_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:38.168905 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_721_148} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:38.172247 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10802 2024.10.22 01:55:38.260978 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 592 2024.10.22 01:55:38.274935 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_721_148} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_718_147, total 10836 rows starting from the beginning of the part 2024.10.22 01:55:38.302659 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_721_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_719_719_0, total 15 rows starting from the beginning of the part 2024.10.22 01:55:38.347250 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_721_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_720_720_0, total 9 rows starting from the beginning of the part 2024.10.22 01:55:38.369408 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_721_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_721_721_0, total 17 rows starting from the beginning of the part 2024.10.22 01:55:38.477820 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_680_680_0 to 202410_687_687_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:38.632185 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1034583 2024.10.22 01:55:38.928553 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_144_25/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:55:39.024072 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_721_148} MergeTask::MergeProjectionsStage: Merge sorted 10877 rows, containing 12 columns (12 merged, 0 gathered) in 0.88510358 sec., 12288.957186231242 rows/sec., 1.91 MiB/sec. 2024.10.22 01:55:39.065780 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_721_148} MergedBlockOutputStream: filled checksums 202410_1_721_148 (state Temporary) 2024.10.22 01:55:39.545706 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_721_148} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_721_148 to 202410_1_721_148 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:39.549317 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_721_148} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 4 parts: [202410_1_718_147, 202410_721_721_0] -> 202410_1_721_148 2024.10.22 01:55:39.558631 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.78 GiB, peak 2.00 GiB, free memory in arenas 10.65 MiB, will set to 1.60 GiB (RSS), difference: -190.80 MiB 2024.10.22 01:55:39.567409 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.49 MiB. 2024.10.22 01:55:40.401774 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(497c362e0eabd9dd8b5457ecb4ead15c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 497c362e0eabd9dd8b5457ecb4ead15c, state: DOWNLOADING, downloaded size: 1048576, reserved size: 1048576, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 1048576, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:55:40.435197 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 497c362e0eabd9dd8b5457ecb4ead15c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 1048576, reserved size: 1048576, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 1048576, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:55:40.443116 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:55:40.853752 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 79362 2024.10.22 01:55:40.980329 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:55:40.993449 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:55:41.114514 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:41.123076 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:41.178571 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1197_1197_0 (state Temporary) 2024.10.22 01:55:41.236790 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1197_1197_0 to 202410_1210_1210_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:41.256643 [ 1673 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vao/dixvkvueutjsuygphjsxsuqexdgrl, size 2237741 2024.10.22 01:55:41.313449 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79362 2024.10.22 01:55:41.869705 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:55:41.874316 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:55:41.898892 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:55:41.919860 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:55:41.951226 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:55:41.981058 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:55:42.028785 [ 1262 ] {} DatabaseCatalog: Execution took 320 ms. 2024.10.22 01:55:42.027037 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:55:42.064818 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 78899 2024.10.22 01:55:42.066575 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:55:42.105380 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column RedirectCount 2024.10.22 01:55:42.170469 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mbv/gkkabnyekbznvxfjwxfxfetjdyllt (LogSeriesLimiter: on interval from 2024-10-22 01:55:37 to 2024-10-22 01:55:42 accepted series 1 / 205 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:55:42.666699 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 2.00 GiB, free memory in arenas 11.33 MiB, will set to 1.61 GiB (RSS), difference: -89.73 MiB 2024.10.22 01:55:42.951051 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:42.957278 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:43.007587 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_698_698_0 (state Temporary) 2024.10.22 01:55:43.079180 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_698_698_0 to 202410_704_704_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:43.115565 [ 1217 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_699_49 to 202410_700_700_0 2024.10.22 01:55:43.145741 [ 1217 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.14 MiB using storage policy from min volume index 0 2024.10.22 01:55:43.157421 [ 1217 ] {} DiskLocal: Reserved 1.14 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:43.167859 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_700_50} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_699_49 to 202410_700_700_0 into Compact with storage Full 2024.10.22 01:55:43.184506 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_700_50} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:43.191141 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_700_50} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_699_49, total 19187 rows starting from the beginning of the part 2024.10.22 01:55:43.200128 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_700_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_700_700_0, total 113 rows starting from the beginning of the part 2024.10.22 01:55:43.350493 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 78899 2024.10.22 01:55:44.483470 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_700_50} MergeTask::MergeProjectionsStage: Merge sorted 19300 rows, containing 13 columns (13 merged, 0 gathered) in 1.320628633 sec., 14614.25227178154 rows/sec., 6.33 MiB/sec. 2024.10.22 01:55:44.565328 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_700_50} MergedBlockOutputStream: filled checksums 202410_507_700_50 (state Temporary) 2024.10.22 01:55:44.586986 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 13.38 MiB, will set to 1.64 GiB (RSS), difference: 1.93 MiB 2024.10.22 01:55:44.854421 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_682_131/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:55:44.957407 [ 1167 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:55:44.991623 [ 1167 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:55:45.068056 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_700_50} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_700_50 to 202410_507_700_50 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:45.099796 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_700_50} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_699_49, 202410_700_700_0] -> 202410_507_700_50 2024.10.22 01:55:45.153208 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.96 MiB. 2024.10.22 01:55:45.325282 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 79405 2024.10.22 01:55:45.438410 [ 1237 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 01:55:45.643218 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1036123 2024.10.22 01:55:45.675068 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10815 2024.10.22 01:55:45.765133 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 593 2024.10.22 01:55:45.844355 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:45.847087 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:46.096997 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:46.100463 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:46.102939 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1198_1198_0 (state Temporary) 2024.10.22 01:55:46.115272 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:46.135039 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:46.176485 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_711_711_0 (state Temporary) 2024.10.22 01:55:46.199427 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1198_1198_0 to 202410_1211_1211_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:46.200317 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_575_575_0 (state Temporary) 2024.10.22 01:55:46.250758 [ 1188 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1095_505 to 202410_1096_1206_22 2024.10.22 01:55:46.259790 [ 1188 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 2.96 MiB using storage policy from min volume index 0 2024.10.22 01:55:46.264186 [ 1188 ] {} DiskLocal: Reserved 2.96 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:46.273166 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_575_575_0 to 202410_585_585_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:46.282606 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79405 2024.10.22 01:55:46.294547 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1206_506} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1095_505 to 202410_1096_1206_22 into Wide with storage Full 2024.10.22 01:55:46.355807 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_711_711_0 to 202410_724_724_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:46.364484 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 593 2024.10.22 01:55:46.411869 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10815 2024.10.22 01:55:46.454184 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1206_506} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:46.469080 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1206_506} MergeTreeSequentialSource: Reading 10 marks from part 202410_1_1095_505, total 71407 rows starting from the beginning of the part 2024.10.22 01:55:46.525135 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 10.67 MiB, will set to 1.64 GiB (RSS), difference: -4.42 MiB 2024.10.22 01:55:46.698830 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1206_506} MergeTreeSequentialSource: Reading 3 marks from part 202410_1096_1206_22, total 8565 rows starting from the beginning of the part 2024.10.22 01:55:46.790616 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:46.795816 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:46.928854 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_681_681_0 (state Temporary) 2024.10.22 01:55:47.049325 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_681_681_0 to 202410_688_688_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:47.105740 [ 1181 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_683_683_0 to 202410_688_688_0 2024.10.22 01:55:47.144175 [ 1181 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:47.152629 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:47.192068 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_683_683_0 to 202410_688_688_0 into Compact with storage Full 2024.10.22 01:55:47.219203 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:47.237441 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_683_683_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:55:47.264335 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1036123 2024.10.22 01:55:47.353035 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_684_684_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:55:47.369691 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_685_685_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:55:47.494979 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_686_686_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:55:47.651807 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_687_687_0, total 1542 rows starting from the beginning of the part 2024.10.22 01:55:47.676864 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_688_688_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:55:48.555263 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 2.00 GiB, free memory in arenas 12.71 MiB, will set to 1.66 GiB (RSS), difference: -43.38 MiB 2024.10.22 01:55:48.817584 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} MergeTask::MergeProjectionsStage: Merge sorted 9242 rows, containing 4 columns (4 merged, 0 gathered) in 1.63713809 sec., 5645.217136203824 rows/sec., 88.21 KiB/sec. 2024.10.22 01:55:48.873560 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} MergedBlockOutputStream: filled checksums 202410_683_688_1 (state Temporary) 2024.10.22 01:55:49.139497 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_683_688_1 to 202410_683_688_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:49.143757 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_683_688_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_683_683_0, 202410_688_688_0] -> 202410_683_688_1 2024.10.22 01:55:49.247364 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:55:50.401487 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 79455 2024.10.22 01:55:50.671249 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:50.687865 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:50.765626 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1199_1199_0 (state Temporary) 2024.10.22 01:55:50.795599 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 2.00 GiB, free memory in arenas 13.00 MiB, will set to 1.68 GiB (RSS), difference: -7.50 MiB 2024.10.22 01:55:50.835145 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1199_1199_0 to 202410_1212_1212_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:50.858467 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 209 entries to flush up to offset 79108 2024.10.22 01:55:50.876478 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1207_0 to 202410_1212_1212_0 2024.10.22 01:55:50.889803 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79455 2024.10.22 01:55:50.917084 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:50.981553 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:55:51.024371 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:55:51.028926 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 01:55:51.144059 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:51.148658 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:51.164349 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1207_0 to 202410_1212_1212_0 into Compact with storage Full 2024.10.22 01:55:51.215972 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_699_699_0 (state Temporary) 2024.10.22 01:55:51.282370 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:51.301206 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_699_699_0 to 202410_705_705_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:51.303881 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1207_0, total 59 rows starting from the beginning of the part 2024.10.22 01:55:51.382987 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1208_1208_0, total 54 rows starting from the beginning of the part 2024.10.22 01:55:51.445815 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1209_1209_0, total 58 rows starting from the beginning of the part 2024.10.22 01:55:51.448149 [ 1196 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_507_700_50 to 202410_705_705_0 2024.10.22 01:55:51.486721 [ 1196 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.19 MiB using storage policy from min volume index 0 2024.10.22 01:55:51.498884 [ 1196 ] {} DiskLocal: Reserved 1.19 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:51.558102 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 79108 2024.10.22 01:55:51.583455 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1210_1210_0, total 50 rows starting from the beginning of the part 2024.10.22 01:55:51.628875 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1206_506} MergeTask::MergeProjectionsStage: Merge sorted 79972 rows, containing 13 columns (13 merged, 0 gathered) in 5.340359638 sec., 14975.021425701218 rows/sec., 2.48 MiB/sec. 2024.10.22 01:55:51.655539 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} MergeTask::PrepareStage: Merging 6 parts: from 202410_507_700_50 to 202410_705_705_0 into Compact with storage Full 2024.10.22 01:55:51.749395 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1206_506} MergedBlockOutputStream: filled checksums 202410_1_1206_506 (state Temporary) 2024.10.22 01:55:51.755829 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1211_1211_0, total 43 rows starting from the beginning of the part 2024.10.22 01:55:51.806273 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1212_1212_0, total 50 rows starting from the beginning of the part 2024.10.22 01:55:51.827605 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:55:51.939283 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_700_50, total 19300 rows starting from the beginning of the part 2024.10.22 01:55:51.966510 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_701_701_0, total 113 rows starting from the beginning of the part 2024.10.22 01:55:51.996814 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_702_702_0, total 107 rows starting from the beginning of the part 2024.10.22 01:55:52.032080 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_703_703_0, total 139 rows starting from the beginning of the part 2024.10.22 01:55:52.068724 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_704_704_0, total 115 rows starting from the beginning of the part 2024.10.22 01:55:52.097214 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_705_705_0, total 209 rows starting from the beginning of the part 2024.10.22 01:55:52.175178 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_144_25/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:55:52.227312 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:52.231500 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:52.329816 [ 1258 ] {} DatabaseCatalog: Execution took 301 ms. 2024.10.22 01:55:52.560035 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 13.07 MiB, will set to 1.68 GiB (RSS), difference: 26.62 MiB 2024.10.22 01:55:52.954775 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} MergeTask::MergeProjectionsStage: Merge sorted 314 rows, containing 13 columns (13 merged, 0 gathered) in 1.797866132 sec., 174.65149068173224 rows/sec., 29.16 KiB/sec. 2024.10.22 01:55:52.968811 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1212_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:55:53.072452 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1206_506} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1206_506 to 202410_1_1206_506 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:53.076923 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1206_506} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1095_505, 202410_1096_1206_22] -> 202410_1_1206_506 2024.10.22 01:55:53.171563 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 54.91 MiB. 2024.10.22 01:55:53.171266 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} MergedBlockOutputStream: filled checksums 202410_1207_1212_1 (state Temporary) 2024.10.22 01:55:53.656763 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1212_1 to 202410_1207_1212_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:53.660029 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1212_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1207_0, 202410_1212_1212_0] -> 202410_1207_1212_1 2024.10.22 01:55:53.784187 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:55:53.866359 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 594 2024.10.22 01:55:53.917212 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 10838 2024.10.22 01:55:54.175817 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:54.179280 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:54.239236 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_712_712_0 (state Temporary) 2024.10.22 01:55:54.304650 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_712_712_0 to 202410_725_725_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:54.320602 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} MergeTask::MergeProjectionsStage: Merge sorted 19983 rows, containing 13 columns (13 merged, 0 gathered) in 2.781296424 sec., 7184.778949688823 rows/sec., 3.11 MiB/sec. 2024.10.22 01:55:54.345199 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1037663 2024.10.22 01:55:54.362231 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10838 2024.10.22 01:55:54.471418 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} MergedBlockOutputStream: filled checksums 202410_507_705_51 (state Temporary) 2024.10.22 01:55:54.585628 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:54.599614 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:54.664935 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_576_576_0 (state Temporary) 2024.10.22 01:55:54.699620 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 2.00 GiB, free memory in arenas 12.05 MiB, will set to 1.69 GiB (RSS), difference: 15.19 MiB 2024.10.22 01:55:54.781744 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_576_576_0 to 202410_586_586_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:54.843266 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_705_51 to 202410_507_705_51 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:54.849020 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_705_51} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_507_700_50, 202410_705_705_0] -> 202410_507_705_51 2024.10.22 01:55:54.903253 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.45 MiB. 2024.10.22 01:55:54.903993 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 79525 2024.10.22 01:55:55.011152 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 594 2024.10.22 01:55:54.997567 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1111_3, 202410_1112_1112_0, 202410_1113_1113_0, 202410_1114_1114_0, 202410_1115_1115_0, 202410_1116_1116_0] 2024.10.22 01:55:55.024684 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1111_3, 202410_1112_1112_0, 202410_1113_1113_0, 202410_1114_1114_0, 202410_1115_1115_0, 202410_1116_1116_0] 2024.10.22 01:55:55.093419 [ 1694 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mbv/gkkabnyekbznvxfjwxfxfetjdyllt, size 11916 2024.10.22 01:55:55.112199 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:55.126601 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:55.167794 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1111_3, 202410_1112_1112_0, 202410_1113_1113_0, 202410_1114_1114_0, 202410_1115_1115_0, 202410_1116_1116_0] 2024.10.22 01:55:55.229398 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1200_1200_0 (state Temporary) 2024.10.22 01:55:55.310959 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1200_1200_0 to 202410_1213_1213_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:55.369263 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79525 2024.10.22 01:55:55.416329 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:55:55.456627 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:55:55.473089 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:55:55.498392 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:55.507112 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:55.519183 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:55:55.556375 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:55:55.573220 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:55:55.654548 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:55:55.710621 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_682_682_0 (state Temporary) 2024.10.22 01:55:55.735158 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 01:55:55.792700 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pto/czkxbkwzclmpsyijafzaxxbfoafqg (LogSeriesLimiter: on interval from 2024-10-22 01:55:42 to 2024-10-22 01:55:55 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:55:55.995487 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_682_682_0 to 202410_689_689_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:56.087847 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_143_143_0 (state Temporary) 2024.10.22 01:55:56.166418 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_682_131/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:55:56.244667 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1037663 2024.10.22 01:55:57.798465 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:55:58.533005 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 2.00 GiB, free memory in arenas 15.97 MiB, will set to 1.70 GiB (RSS), difference: 14.87 MiB 2024.10.22 01:55:58.660804 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_682_131} MergeTask::MergeProjectionsStage: Merge sorted 1040471 rows, containing 4 columns (4 merged, 0 gathered) in 28.200426151 sec., 36895.57719549229 rows/sec., 553.21 KiB/sec. 2024.10.22 01:55:58.784787 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_682_131} MergedBlockOutputStream: filled checksums 202410_1_682_131 (state Temporary) 2024.10.22 01:55:59.209185 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 102 entries to flush up to offset 79210 2024.10.22 01:55:59.390304 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 79558 2024.10.22 01:55:59.438370 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:59.443518 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:59.500628 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_700_700_0 (state Temporary) 2024.10.22 01:55:59.533115 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:55:59.540888 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:55:59.583956 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1201_1201_0 (state Temporary) 2024.10.22 01:55:59.606622 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_700_700_0 to 202410_706_706_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:59.642461 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1201_1201_0 to 202410_1214_1214_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:55:59.711715 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 79210 2024.10.22 01:55:59.812571 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79558 2024.10.22 01:55:59.998280 [ 1272 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:56:00.078603 [ 1272 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:56:00.590440 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 2.00 GiB, free memory in arenas 14.45 MiB, will set to 1.70 GiB (RSS), difference: 11.17 MiB 2024.10.22 01:56:00.986928 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:56:00.993637 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:56:01.444446 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_533_109, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0, 202410_538_538_0] 2024.10.22 01:56:01.447594 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_533_109, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0, 202410_538_538_0] 2024.10.22 01:56:01.457905 [ 1276 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 01:56:01.676643 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_533_109, 202410_534_534_0, 202410_535_535_0, 202410_536_536_0, 202410_537_537_0, 202410_538_538_0] 2024.10.22 01:56:01.737241 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_682_131} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_682_131 to 202410_1_682_131 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:01.744483 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_682_131} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_676_130, 202410_677_682_1] -> 202410_1_682_131 2024.10.22 01:56:01.761278 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.51 MiB. 2024.10.22 01:56:01.865134 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 10854 2024.10.22 01:56:01.979908 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:02.087011 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:56:02.238189 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_713_713_0 (state Temporary) 2024.10.22 01:56:02.286540 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_713_713_0 to 202410_726_726_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:02.340278 [ 1259 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_721_148 to 202410_726_726_0 2024.10.22 01:56:02.349388 [ 1259 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:02.352598 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:56:02.383194 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10854 2024.10.22 01:56:02.444281 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_721_148 to 202410_726_726_0 into Compact with storage Full 2024.10.22 01:56:02.465209 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:56:02.472371 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_721_148, total 10877 rows starting from the beginning of the part 2024.10.22 01:56:02.518349 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 595 2024.10.22 01:56:02.567281 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 2.00 GiB, free memory in arenas 10.98 MiB, will set to 1.71 GiB (RSS), difference: 13.22 MiB 2024.10.22 01:56:02.602546 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_722_722_0, total 16 rows starting from the beginning of the part 2024.10.22 01:56:02.638714 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_723_723_0, total 15 rows starting from the beginning of the part 2024.10.22 01:56:02.659105 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_724_724_0, total 13 rows starting from the beginning of the part 2024.10.22 01:56:02.661711 [ 1267 ] {} DatabaseCatalog: Execution took 332 ms. 2024.10.22 01:56:02.669834 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_725_725_0, total 23 rows starting from the beginning of the part 2024.10.22 01:56:02.688253 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_726_726_0, total 16 rows starting from the beginning of the part 2024.10.22 01:56:02.806552 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:02.814326 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:56:02.937620 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_577_577_0 (state Temporary) 2024.10.22 01:56:03.072755 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_144_25/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:56:03.118866 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_577_577_0 to 202410_587_587_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:03.250716 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1039588 2024.10.22 01:56:03.327521 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 595 2024.10.22 01:56:03.766299 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_643_28, 202410_644_644_0, 202410_645_645_0, 202410_646_646_0, 202410_647_647_0, 202410_648_648_0] 2024.10.22 01:56:03.767896 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_643_28, 202410_644_644_0, 202410_645_645_0, 202410_646_646_0, 202410_647_647_0, 202410_648_648_0] 2024.10.22 01:56:03.816083 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_643_28, 202410_644_644_0, 202410_645_645_0, 202410_646_646_0, 202410_647_647_0, 202410_648_648_0] 2024.10.22 01:56:03.923719 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 79611 2024.10.22 01:56:04.138845 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} MergeTask::MergeProjectionsStage: Merge sorted 10960 rows, containing 12 columns (12 merged, 0 gathered) in 1.734537485 sec., 6318.687312773757 rows/sec., 1006.58 KiB/sec. 2024.10.22 01:56:04.173186 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} MergedBlockOutputStream: filled checksums 202410_1_726_149 (state Temporary) 2024.10.22 01:56:04.275816 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_726_149 to 202410_1_726_149 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:04.286508 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_726_149} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_721_148, 202410_726_726_0] -> 202410_1_726_149 2024.10.22 01:56:04.455308 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:04.460295 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:56:04.464034 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.54 MiB. 2024.10.22 01:56:04.469045 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:04.475224 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.64 GiB. 2024.10.22 01:56:04.484223 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_143_143_0 to 202410_146_146_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:04.562388 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1202_1202_0 (state Temporary) 2024.10.22 01:56:04.615499 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1202_1202_0 to 202410_1215_1215_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:04.656619 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/vhz/rzjqegybqeudsumpyaenjsgosdqmc, upload id 4b24fb1b-3d41-4c17-9d1a-7004c1f4c55a 2024.10.22 01:56:04.660803 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 2.00 GiB, free memory in arenas 8.59 MiB, will set to 1.70 GiB (RSS), difference: -3.95 MiB 2024.10.22 01:56:04.670546 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_683_683_0 (state Temporary) 2024.10.22 01:56:04.805679 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79611 2024.10.22 01:56:04.858955 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_683_683_0 to 202410_690_690_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:05.046910 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1039588 2024.10.22 01:56:06.585834 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 2.00 GiB, free memory in arenas 5.40 MiB, will set to 1.66 GiB (RSS), difference: -11.46 MiB 2024.10.22 01:56:07.243043 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 68 entries to flush up to offset 79278 2024.10.22 01:56:07.617345 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:07.628536 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.54 GiB. 2024.10.22 01:56:07.668562 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_701_701_0 (state Temporary) 2024.10.22 01:56:07.755796 [ 1675 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/efp/etamwtqeurlurppiwqagwnfrlscle, size 258 2024.10.22 01:56:07.767854 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_701_701_0 to 202410_707_707_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:07.901962 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 79278 2024.10.22 01:56:08.751869 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 8.40 MiB, will set to 1.65 GiB (RSS), difference: 128.60 MiB 2024.10.22 01:56:08.851664 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5752 2024.10.22 01:56:08.931035 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 28 entries to flush up to offset 79639 2024.10.22 01:56:09.092715 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:09.104302 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 01:56:09.212028 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_629_629_0, 202410_630_630_0, 202410_631_631_0, 202410_632_632_0, 202410_633_633_0, 202410_634_634_0] 2024.10.22 01:56:09.230527 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_629_629_0, 202410_630_630_0, 202410_631_631_0, 202410_632_632_0, 202410_633_633_0, 202410_634_634_0] 2024.10.22 01:56:09.264981 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_629_629_0, 202410_630_630_0, 202410_631_631_0, 202410_632_632_0, 202410_633_633_0, 202410_634_634_0] 2024.10.22 01:56:09.265147 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1203_1203_0 (state Temporary) 2024.10.22 01:56:09.532416 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1203_1203_0 to 202410_1216_1216_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:09.650498 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79639 2024.10.22 01:56:09.893640 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 10868 2024.10.22 01:56:10.134839 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:10.141731 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.50 GiB. 2024.10.22 01:56:10.304360 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_714_714_0 (state Temporary) 2024.10.22 01:56:10.577381 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_714_714_0 to 202410_727_727_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:10.667930 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10868 2024.10.22 01:56:10.986313 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:56:11.000317 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:56:11.213092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_11_11_0 (state Temporary) 2024.10.22 01:56:11.237435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wmd/qaqvhjvwcoaaifjteibmrybavwtlb (LogSeriesLimiter: on interval from 2024-10-22 01:55:55 to 2024-10-22 01:56:11 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:56:11.278740 [ 1702 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wmd/qaqvhjvwcoaaifjteibmrybavwtlb, size 4 2024.10.22 01:56:11.514956 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 9.08 MiB, will set to 1.60 GiB (RSS), difference: 239.28 MiB 2024.10.22 01:56:12.055587 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1292 entries to flush up to offset 1040880 2024.10.22 01:56:12.198284 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:56:12.209755 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:56:12.233783 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:56:12.343891 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:56:12.353220 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:56:12.372228 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:56:12.383468 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column URLDomain 2024.10.22 01:56:12.866707 [ 1180 ] {} DatabaseCatalog: Execution took 203 ms. 2024.10.22 01:56:13.035331 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_663_133, 202410_664_664_0, 202410_665_665_0, 202410_666_666_0, 202410_667_667_0, 202410_668_668_0] 2024.10.22 01:56:13.047911 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_663_133, 202410_664_664_0, 202410_665_665_0, 202410_666_666_0, 202410_667_667_0, 202410_668_668_0] 2024.10.22 01:56:13.076538 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_663_133, 202410_664_664_0, 202410_665_665_0, 202410_666_666_0, 202410_667_667_0, 202410_668_668_0] 2024.10.22 01:56:13.152412 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:13.254772 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:13.660804 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 79675 2024.10.22 01:56:13.660949 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_684_684_0 (state Temporary) 2024.10.22 01:56:13.790998 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:13.796821 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:13.839041 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_684_684_0 to 202410_691_691_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:13.876906 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1040880 2024.10.22 01:56:13.947303 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1204_1204_0 (state Temporary) 2024.10.22 01:56:13.979161 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1204_1204_0 to 202410_1217_1217_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:14.029168 [ 1175 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1212_1 to 202410_1217_1217_0 2024.10.22 01:56:14.049138 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79675 2024.10.22 01:56:14.050557 [ 1175 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:14.056419 [ 1175 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:14.086673 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1212_1 to 202410_1217_1217_0 into Compact with storage Full 2024.10.22 01:56:14.236681 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:56:14.247593 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1212_1, total 314 rows starting from the beginning of the part 2024.10.22 01:56:14.263398 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1213_1213_0, total 70 rows starting from the beginning of the part 2024.10.22 01:56:14.302391 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1214_1214_0, total 33 rows starting from the beginning of the part 2024.10.22 01:56:14.336765 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1215_1215_0, total 53 rows starting from the beginning of the part 2024.10.22 01:56:14.472334 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1216_1216_0, total 28 rows starting from the beginning of the part 2024.10.22 01:56:14.511054 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1217_1217_0, total 36 rows starting from the beginning of the part 2024.10.22 01:56:14.777517 [ 1690 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pto/czkxbkwzclmpsyijafzaxxbfoafqg, size 48592 2024.10.22 01:56:15.061237 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:56:15.079790 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:56:15.096579 [ 1231 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:56:15.111304 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:56:15.138708 [ 1231 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:56:15.140216 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:56:15.175511 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:56:15.196204 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:56:15.248452 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:56:15.276260 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 01:56:15.416389 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 96 entries to flush up to offset 79374 2024.10.22 01:56:15.436820 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1217_2/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:56:15.567675 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:15.570477 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:15.590822 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 8.25 MiB, will set to 1.58 GiB (RSS), difference: -10.75 MiB 2024.10.22 01:56:15.612975 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_702_702_0 (state Temporary) 2024.10.22 01:56:15.654842 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} MergeTask::MergeProjectionsStage: Merge sorted 534 rows, containing 13 columns (13 merged, 0 gathered) in 1.57466675 sec., 339.11937240054124 rows/sec., 55.49 KiB/sec. 2024.10.22 01:56:15.696031 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} MergedBlockOutputStream: filled checksums 202410_1207_1217_2 (state Temporary) 2024.10.22 01:56:15.748535 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_702_702_0 to 202410_708_708_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:15.983737 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1217_2 to 202410_1207_1217_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:15.989713 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1217_2} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1212_1, 202410_1217_1217_0] -> 202410_1207_1217_2 2024.10.22 01:56:15.999139 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 79374 2024.10.22 01:56:16.056040 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:56:16.464155 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 5795 2024.10.22 01:56:16.540669 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 7.46 MiB, will set to 1.58 GiB (RSS), difference: 1.90 MiB 2024.10.22 01:56:17.031967 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_144_25/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:56:18.063398 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 79722 2024.10.22 01:56:18.169254 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 10879 2024.10.22 01:56:18.220718 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:18.228006 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:18.272779 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1205_1205_0 (state Temporary) 2024.10.22 01:56:18.320103 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1205_1205_0 to 202410_1218_1218_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:18.332280 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 596 2024.10.22 01:56:18.387852 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79722 2024.10.22 01:56:18.588021 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:18.595392 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:18.760953 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_715_715_0 (state Temporary) 2024.10.22 01:56:18.838243 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_715_715_0 to 202410_728_728_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:18.847864 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:18.850216 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:18.966942 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10879 2024.10.22 01:56:19.005939 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_578_578_0 (state Temporary) 2024.10.22 01:56:19.203627 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_578_578_0 to 202410_588_588_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:19.237771 [ 1198 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_583_119 to 202410_588_588_0 2024.10.22 01:56:19.355811 [ 1198 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:19.359829 [ 1198 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:19.370384 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 596 2024.10.22 01:56:19.600130 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_583_119 to 202410_588_588_0 into Compact with storage Full 2024.10.22 01:56:19.741026 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:56:19.758166 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_583_119, total 627 rows starting from the beginning of the part 2024.10.22 01:56:19.812468 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_584_584_0, total 1 rows starting from the beginning of the part 2024.10.22 01:56:19.857510 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_585_585_0, total 1 rows starting from the beginning of the part 2024.10.22 01:56:19.967436 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1116_4, 202410_1117_1117_0, 202410_1118_1118_0, 202410_1119_1119_0, 202410_1120_1120_0, 202410_1121_1121_0] 2024.10.22 01:56:19.968743 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1116_4, 202410_1117_1117_0, 202410_1118_1118_0, 202410_1119_1119_0, 202410_1120_1120_0, 202410_1121_1121_0] 2024.10.22 01:56:20.025097 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_586_586_0, total 1 rows starting from the beginning of the part 2024.10.22 01:56:20.109177 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1116_4, 202410_1117_1117_0, 202410_1118_1118_0, 202410_1119_1119_0, 202410_1120_1120_0, 202410_1121_1121_0] 2024.10.22 01:56:20.125057 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_587_587_0, total 1 rows starting from the beginning of the part 2024.10.22 01:56:20.201273 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_588_588_0, total 1 rows starting from the beginning of the part 2024.10.22 01:56:20.364322 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 7.34 MiB, will set to 1.58 GiB (RSS), difference: 1.79 MiB 2024.10.22 01:56:20.881600 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2001 entries to flush up to offset 1042881 2024.10.22 01:56:20.986565 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:56:20.990434 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:56:21.528476 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:56:21.892489 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:21.897682 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:22.058713 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_685_685_0 (state Temporary) 2024.10.22 01:56:22.261630 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_685_685_0 to 202410_692_692_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:22.279434 [ 1274 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_682_131 to 202410_683_688_1 2024.10.22 01:56:22.294360 [ 1274 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:22.299134 [ 1274 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:22.317031 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_688_132} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_682_131 to 202410_683_688_1 into Wide with storage Full 2024.10.22 01:56:22.343859 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_688_132} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:56:22.356239 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_688_132} MergeTreeSequentialSource: Reading 129 marks from part 202410_1_682_131, total 1040471 rows starting from the beginning of the part 2024.10.22 01:56:22.394065 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_688_132} MergeTreeSequentialSource: Reading 3 marks from part 202410_683_688_1, total 9242 rows starting from the beginning of the part 2024.10.22 01:56:22.394299 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1042881 2024.10.22 01:56:22.407226 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 79769 2024.10.22 01:56:22.556816 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 6.55 MiB, will set to 1.59 GiB (RSS), difference: -4.64 MiB 2024.10.22 01:56:22.646950 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} MergeTask::MergeProjectionsStage: Merge sorted 632 rows, containing 32 columns (32 merged, 0 gathered) in 3.164531728 sec., 199.7135925066004 rows/sec., 18.20 KiB/sec. 2024.10.22 01:56:22.683547 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} MergedBlockOutputStream: filled checksums 202410_1_588_120 (state Temporary) 2024.10.22 01:56:22.748888 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:22.752255 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:22.811906 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1206_1206_0 (state Temporary) 2024.10.22 01:56:22.868620 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1206_1206_0 to 202410_1219_1219_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:22.932476 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79769 2024.10.22 01:56:23.504263 [ 1259 ] {} DatabaseCatalog: Execution took 534 ms. 2024.10.22 01:56:23.509421 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_688_132/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:56:23.515425 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 121 entries to flush up to offset 79495 2024.10.22 01:56:23.724995 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_588_120 to 202410_1_588_120 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:23.732229 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_588_120} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_583_119, 202410_588_588_0] -> 202410_1_588_120 2024.10.22 01:56:23.801747 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:56:23.896898 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:24.003695 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:24.253072 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_703_703_0 (state Temporary) 2024.10.22 01:56:24.316294 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_703_703_0 to 202410_709_709_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:24.414726 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 8.73 MiB, will set to 1.59 GiB (RSS), difference: -3.94 MiB 2024.10.22 01:56:24.453573 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 79495 2024.10.22 01:56:24.603644 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_622_121, 202410_623_628_1] 2024.10.22 01:56:24.608616 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_622_121, 202410_623_628_1] 2024.10.22 01:56:24.627874 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_622_121, 202410_623_628_1] 2024.10.22 01:56:24.846867 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(910a53cba624c5db84ec4df33a43214b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 910a53cba624c5db84ec4df33a43214b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 01:56:24.862757 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 910a53cba624c5db84ec4df33a43214b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 01:56:24.865214 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:56:26.468980 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 10892 2024.10.22 01:56:26.599008 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:26.602261 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:26.651207 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_716_716_0 (state Temporary) 2024.10.22 01:56:26.730993 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_716_716_0 to 202410_729_729_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:26.781044 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10892 2024.10.22 01:56:26.873553 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 597 2024.10.22 01:56:26.949931 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 36 entries to flush up to offset 79805 2024.10.22 01:56:27.280764 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:27.287025 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:27.301045 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:27.403656 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:27.547406 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_144_25/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:56:27.558303 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_579_579_0 (state Temporary) 2024.10.22 01:56:27.569366 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1207_1207_0 (state Temporary) 2024.10.22 01:56:27.639105 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1207_1207_0 to 202410_1220_1220_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:27.679803 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79805 2024.10.22 01:56:27.853521 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_579_579_0 to 202410_589_589_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:27.950942 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 597 2024.10.22 01:56:28.560765 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 16.53 MiB, will set to 1.60 GiB (RSS), difference: 3.23 MiB 2024.10.22 01:56:29.492915 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1570 entries to flush up to offset 1044451 2024.10.22 01:56:30.143146 [ 1253 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:56:30.162008 [ 1253 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:56:30.511492 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 13.17 MiB, will set to 1.61 GiB (RSS), difference: 4.57 MiB 2024.10.22 01:56:30.671099 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:30.675600 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:30.991127 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:56:30.996813 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:56:31.025592 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_686_686_0 (state Temporary) 2024.10.22 01:56:31.104668 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_686_686_0 to 202410_693_693_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:31.159981 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1044451 2024.10.22 01:56:31.957281 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 24 entries to flush up to offset 79829 2024.10.22 01:56:31.965401 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 83 entries to flush up to offset 79578 2024.10.22 01:56:32.397709 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:32.403481 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:32.466284 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1208_1208_0 (state Temporary) 2024.10.22 01:56:32.488882 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:32.492415 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:32.519183 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1208_1208_0 to 202410_1221_1221_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:32.578684 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 25.91 MiB, will set to 1.61 GiB (RSS), difference: 20.46 MiB 2024.10.22 01:56:32.667500 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_704_704_0 (state Temporary) 2024.10.22 01:56:32.690825 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79829 2024.10.22 01:56:32.726975 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_704_704_0 to 202410_710_710_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:32.757702 [ 1179 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_705_51 to 202410_706_706_0 2024.10.22 01:56:32.764788 [ 1179 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.18 MiB using storage policy from min volume index 0 2024.10.22 01:56:32.768419 [ 1179 ] {} DiskLocal: Reserved 1.18 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:32.869837 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 79578 2024.10.22 01:56:32.911752 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_706_52} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_705_51 to 202410_706_706_0 into Compact with storage Full 2024.10.22 01:56:33.120018 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_706_52} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:56:33.144500 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_706_52} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_705_51, total 19983 rows starting from the beginning of the part 2024.10.22 01:56:33.302870 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_706_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_706_706_0, total 102 rows starting from the beginning of the part 2024.10.22 01:56:33.735642 [ 1186 ] {} DatabaseCatalog: Execution took 230 ms. 2024.10.22 01:56:34.390498 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 10898 2024.10.22 01:56:34.491125 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_706_52} MergeTask::MergeProjectionsStage: Merge sorted 20085 rows, containing 13 columns (13 merged, 0 gathered) in 1.585877903 sec., 12664.909424619178 rows/sec., 5.49 MiB/sec. 2024.10.22 01:56:34.550912 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_706_52} MergedBlockOutputStream: filled checksums 202410_507_706_52 (state Temporary) 2024.10.22 01:56:34.581153 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:34.608683 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:56:34.621683 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 16.74 MiB, will set to 1.61 GiB (RSS), difference: -6.19 MiB 2024.10.22 01:56:34.697574 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:34.732835 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:56:34.753688 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_717_717_0 (state Temporary) 2024.10.22 01:56:34.796929 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_717_717_0 to 202410_730_730_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:34.812098 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_688_132/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:56:34.850786 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:56:34.875052 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:56:34.911266 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:56:34.953479 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:56:34.964786 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10898 2024.10.22 01:56:34.979467 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RefererDomain 2024.10.22 01:56:34.988013 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_706_52} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_706_52 to 202410_507_706_52 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:35.000297 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_706_52} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_705_51, 202410_706_706_0] -> 202410_507_706_52 2024.10.22 01:56:35.074809 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jrk/jxcwuqujrrtxtzbnhjuaeswgxzmcy (LogSeriesLimiter: on interval from 2024-10-22 01:56:11 to 2024-10-22 01:56:35 accepted series 1 / 14 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:56:35.149477 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.96 MiB. 2024.10.22 01:56:35.453336 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 598 2024.10.22 01:56:36.021108 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:36.025591 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:36.188432 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_580_580_0 (state Temporary) 2024.10.22 01:56:36.363367 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_580_580_0 to 202410_590_590_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:36.472181 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 598 2024.10.22 01:56:36.599603 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 14.20 MiB, will set to 1.60 GiB (RSS), difference: -16.21 MiB 2024.10.22 01:56:36.652028 [ 1175 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 264 ms. 2024.10.22 01:56:36.701136 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 79879 2024.10.22 01:56:36.964818 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:36.971733 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:37.136227 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1209_1209_0 (state Temporary) 2024.10.22 01:56:37.638499 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 78562 bytes for f55b3c44e54efd5a8f12e8d79a305e54:0) 2024.10.22 01:56:37.643019 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 258c5941de49141cba5c878e9a98c62e, offset: 0, size: 73357 2024.10.22 01:56:37.645366 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1209_1209_0 to 202410_1222_1222_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:37.649273 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 258c5941de49141cba5c878e9a98c62e for removal 2024.10.22 01:56:37.651049 [ 1476 ] {} CacheMetadata: Key 258c5941de49141cba5c878e9a98c62e is removed from metadata 2024.10.22 01:56:37.651462 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2691e9f082a33cc842e533b08c196c1c, offset: 0, size: 157 2024.10.22 01:56:37.654660 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2691e9f082a33cc842e533b08c196c1c for removal 2024.10.22 01:56:37.668904 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1217_2 to 202410_1222_1222_0 2024.10.22 01:56:37.676935 [ 1476 ] {} CacheMetadata: Key 2691e9f082a33cc842e533b08c196c1c is removed from metadata 2024.10.22 01:56:37.682995 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:37.696572 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79879 2024.10.22 01:56:37.707280 [ 1184 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:37.950540 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1217_2 to 202410_1222_1222_0 into Compact with storage Full 2024.10.22 01:56:38.166091 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1329 entries to flush up to offset 1045780 2024.10.22 01:56:38.167138 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:56:38.186567 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1217_2, total 534 rows starting from the beginning of the part 2024.10.22 01:56:38.345491 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1218_1218_0, total 47 rows starting from the beginning of the part 2024.10.22 01:56:38.364066 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:56:38.415869 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:56:38.451165 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1219_1219_0, total 47 rows starting from the beginning of the part 2024.10.22 01:56:38.452879 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:56:38.469958 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:56:38.513065 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:56:38.528925 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1220_1220_0, total 36 rows starting from the beginning of the part 2024.10.22 01:56:38.650076 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:56:38.666562 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:38.671127 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:38.678832 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:56:38.742255 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1221_1221_0, total 24 rows starting from the beginning of the part 2024.10.22 01:56:38.827313 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ParamPrice 2024.10.22 01:56:38.900240 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:38.909193 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:39.175561 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_687_687_0 (state Temporary) 2024.10.22 01:56:39.299357 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1222_1222_0, total 50 rows starting from the beginning of the part 2024.10.22 01:56:39.323875 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_687_687_0 to 202410_694_694_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:39.462448 [ 1214 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_689_689_0 to 202410_694_694_0 2024.10.22 01:56:39.515382 [ 1214 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:39.524844 [ 1214 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:39.535801 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 23.09 MiB, will set to 1.60 GiB (RSS), difference: 7.48 MiB 2024.10.22 01:56:39.559498 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1045780 2024.10.22 01:56:39.566246 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_689_689_0 to 202410_694_694_0 into Compact with storage Full 2024.10.22 01:56:39.596861 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:56:39.608942 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_689_689_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:56:39.849239 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_690_690_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:56:39.862002 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_691_691_0, total 1292 rows starting from the beginning of the part 2024.10.22 01:56:39.985445 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_692_692_0, total 2001 rows starting from the beginning of the part 2024.10.22 01:56:40.011668 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_693_693_0, total 1570 rows starting from the beginning of the part 2024.10.22 01:56:40.038716 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_694_694_0, total 1329 rows starting from the beginning of the part 2024.10.22 01:56:40.390638 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 128 entries to flush up to offset 79706 2024.10.22 01:56:40.648833 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:40.651198 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:40.707314 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_705_705_0 (state Temporary) 2024.10.22 01:56:40.762641 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_705_705_0 to 202410_711_711_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:40.818590 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_706_52 to 202410_707_707_0 2024.10.22 01:56:40.839063 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.18 MiB using storage policy from min volume index 0 2024.10.22 01:56:40.842019 [ 1221 ] {} DiskLocal: Reserved 1.18 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:40.848435 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 79706 2024.10.22 01:56:40.855193 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_707_53} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_706_52 to 202410_707_707_0 into Compact with storage Full 2024.10.22 01:56:40.968208 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_707_53} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:56:40.980153 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_707_53} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_706_52, total 20085 rows starting from the beginning of the part 2024.10.22 01:56:40.992784 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:56:40.996447 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:56:41.145248 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_707_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_707_707_0, total 68 rows starting from the beginning of the part 2024.10.22 01:56:41.251648 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} MergeTask::MergeProjectionsStage: Merge sorted 738 rows, containing 13 columns (13 merged, 0 gathered) in 3.407210828 sec., 216.59945253026765 rows/sec., 34.89 KiB/sec. 2024.10.22 01:56:41.371643 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} MergedBlockOutputStream: filled checksums 202410_1207_1222_3 (state Temporary) 2024.10.22 01:56:41.646620 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 21.12 MiB, will set to 1.59 GiB (RSS), difference: -32.19 MiB 2024.10.22 01:56:41.662478 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} MergeTask::MergeProjectionsStage: Merge sorted 9657 rows, containing 4 columns (4 merged, 0 gathered) in 2.103995056 sec., 4589.839682589064 rows/sec., 71.72 KiB/sec. 2024.10.22 01:56:41.701687 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1222_3 to 202410_1207_1222_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:41.702630 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} MergedBlockOutputStream: filled checksums 202410_689_694_1 (state Temporary) 2024.10.22 01:56:41.708954 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 79950 2024.10.22 01:56:41.714475 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1222_3} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1217_2, 202410_1222_1222_0] -> 202410_1207_1222_3 2024.10.22 01:56:41.764225 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:56:41.777679 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_689_694_1 to 202410_689_694_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:41.780914 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_689_694_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_689_689_0, 202410_694_694_0] -> 202410_689_694_1 2024.10.22 01:56:41.828938 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:56:41.917100 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.147418955 sec. 2024.10.22 01:56:41.951282 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:41.955547 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:42.111308 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1210_1210_0 (state Temporary) 2024.10.22 01:56:42.264632 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1210_1210_0 to 202410_1223_1223_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:42.311973 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 79950 2024.10.22 01:56:42.439895 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(6d45eb13dd510ea93cceeb50df43fc0a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6d45eb13dd510ea93cceeb50df43fc0a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:56:42.447331 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 6d45eb13dd510ea93cceeb50df43fc0a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 01:56:42.452590 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:56:42.466768 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 10921 2024.10.22 01:56:42.581477 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_144_144_0 (state Temporary) 2024.10.22 01:56:42.588615 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:42.595563 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:42.653965 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_718_718_0 (state Temporary) 2024.10.22 01:56:42.910605 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_718_718_0 to 202410_731_731_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:42.955859 [ 1256 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_726_149 to 202410_731_731_0 2024.10.22 01:56:42.968943 [ 1256 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:42.976243 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:43.042984 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_726_149 to 202410_731_731_0 into Compact with storage Full 2024.10.22 01:56:43.074165 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10921 2024.10.22 01:56:43.102191 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_707_53} MergeTask::MergeProjectionsStage: Merge sorted 20153 rows, containing 13 columns (13 merged, 0 gathered) in 2.24101997 sec., 8992.780193743654 rows/sec., 3.89 MiB/sec. 2024.10.22 01:56:43.117026 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:56:43.144665 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_726_149, total 10960 rows starting from the beginning of the part 2024.10.22 01:56:43.260977 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_727_727_0, total 14 rows starting from the beginning of the part 2024.10.22 01:56:43.280593 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_728_728_0, total 11 rows starting from the beginning of the part 2024.10.22 01:56:43.332282 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_729_729_0, total 13 rows starting from the beginning of the part 2024.10.22 01:56:43.354917 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_730_730_0, total 6 rows starting from the beginning of the part 2024.10.22 01:56:43.359846 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_707_53} MergedBlockOutputStream: filled checksums 202410_507_707_53 (state Temporary) 2024.10.22 01:56:43.410005 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_731_731_0, total 23 rows starting from the beginning of the part 2024.10.22 01:56:43.476973 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 22.52 MiB, will set to 1.60 GiB (RSS), difference: 22.73 MiB 2024.10.22 01:56:43.695750 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_707_53} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_707_53 to 202410_507_707_53 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:43.701703 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_707_53} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_706_52, 202410_707_707_0] -> 202410_507_707_53 2024.10.22 01:56:43.958042 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.92 MiB. 2024.10.22 01:56:43.973755 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 599 2024.10.22 01:56:44.051515 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 221 ms. 2024.10.22 01:56:44.257283 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:44.269724 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:44.414277 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} MergeTask::MergeProjectionsStage: Merge sorted 11027 rows, containing 12 columns (12 merged, 0 gathered) in 1.398789963 sec., 7883.242153346792 rows/sec., 1.23 MiB/sec. 2024.10.22 01:56:44.460753 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_581_581_0 (state Temporary) 2024.10.22 01:56:44.462496 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} MergedBlockOutputStream: filled checksums 202410_1_731_150 (state Temporary) 2024.10.22 01:56:44.596894 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1121_5, 202410_1122_1122_0, 202410_1123_1123_0, 202410_1124_1124_0, 202410_1125_1125_0, 202410_1126_1126_0] 2024.10.22 01:56:44.604392 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1121_5, 202410_1122_1122_0, 202410_1123_1123_0, 202410_1124_1124_0, 202410_1125_1125_0, 202410_1126_1126_0] 2024.10.22 01:56:44.641578 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1121_5, 202410_1122_1122_0, 202410_1123_1123_0, 202410_1124_1124_0, 202410_1125_1125_0, 202410_1126_1126_0] 2024.10.22 01:56:44.657502 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_581_581_0 to 202410_591_591_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:44.799805 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 599 2024.10.22 01:56:44.870830 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_731_150 to 202410_1_731_150 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:44.874408 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_731_150} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_726_149, 202410_731_731_0] -> 202410_1_731_150 2024.10.22 01:56:44.926485 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.54 MiB. 2024.10.22 01:56:45.024724 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_688_132/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:56:45.165081 [ 1226 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:56:45.175810 [ 1226 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:56:45.577152 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 21.90 MiB, will set to 1.59 GiB (RSS), difference: 6.44 MiB 2024.10.22 01:56:46.031431 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/jve/uotfevpplelxxcpzyydignkpujqql. 2024.10.22 01:56:46.267586 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_144_25/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:56:46.329332 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 80012 2024.10.22 01:56:46.462797 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:46.465566 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:46.516234 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1211_1211_0 (state Temporary) 2024.10.22 01:56:46.562964 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1893 entries to flush up to offset 1047673 2024.10.22 01:56:46.664369 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1211_1211_0 to 202410_1224_1224_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:46.818970 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80012 2024.10.22 01:56:47.442613 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_648_29, 202410_649_649_0, 202410_650_650_0, 202410_651_651_0, 202410_652_652_0, 202410_653_653_0] 2024.10.22 01:56:47.444520 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_648_29, 202410_649_649_0, 202410_650_650_0, 202410_651_651_0, 202410_652_652_0, 202410_653_653_0] 2024.10.22 01:56:47.466091 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_648_29, 202410_649_649_0, 202410_650_650_0, 202410_651_651_0, 202410_652_652_0, 202410_653_653_0] 2024.10.22 01:56:47.522652 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 20.02 MiB, will set to 1.59 GiB (RSS), difference: 1.87 MiB 2024.10.22 01:56:47.784371 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:47.791233 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:47.928201 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_688_688_0 (state Temporary) 2024.10.22 01:56:47.997578 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_688_688_0 to 202410_695_695_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:48.144925 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1047673 2024.10.22 01:56:48.364086 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 118 entries to flush up to offset 79824 2024.10.22 01:56:48.513785 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:48.629187 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:48.682498 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_706_706_0 (state Temporary) 2024.10.22 01:56:48.863562 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_706_706_0 to 202410_712_712_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:48.903895 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_707_53 to 202410_708_708_0 2024.10.22 01:56:48.923316 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.19 MiB using storage policy from min volume index 0 2024.10.22 01:56:48.938858 [ 1224 ] {} DiskLocal: Reserved 1.19 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:48.943281 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 79824 2024.10.22 01:56:49.033767 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_708_54} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_707_53 to 202410_708_708_0 into Compact with storage Full 2024.10.22 01:56:49.118360 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_144_144_0 to 202410_147_147_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:49.235533 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_708_54} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:56:49.268730 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_708_54} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_707_53, total 20153 rows starting from the beginning of the part 2024.10.22 01:56:49.388753 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_708_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_708_708_0, total 96 rows starting from the beginning of the part 2024.10.22 01:56:49.455046 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 2.00 GiB, free memory in arenas 26.33 MiB, will set to 1.60 GiB (RSS), difference: 113.68 MiB 2024.10.22 01:56:50.577228 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 10937 2024.10.22 01:56:50.833217 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 80044 2024.10.22 01:56:50.901390 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:50.902985 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:50.950663 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_719_719_0 (state Temporary) 2024.10.22 01:56:50.994706 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:56:50.997370 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/jve/uotfevpplelxxcpzyydignkpujqql. 2024.10.22 01:56:51.015479 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:56:51.019539 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_719_719_0 to 202410_732_732_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:51.067137 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:51.073721 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:51.172109 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10937 2024.10.22 01:56:51.259823 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1212_1212_0 (state Temporary) 2024.10.22 01:56:51.299061 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1212_1212_0 to 202410_1225_1225_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:51.467059 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80044 2024.10.22 01:56:51.645398 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 20.66 MiB, will set to 1.59 GiB (RSS), difference: 85.40 MiB 2024.10.22 01:56:51.681391 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_708_54/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:56:51.705246 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_708_54} MergeTask::MergeProjectionsStage: Merge sorted 20249 rows, containing 13 columns (13 merged, 0 gathered) in 2.744638076 sec., 7377.657614336761 rows/sec., 3.19 MiB/sec. 2024.10.22 01:56:51.772520 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_708_54} MergedBlockOutputStream: filled checksums 202410_507_708_54 (state Temporary) 2024.10.22 01:56:51.975293 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(51020561b966c23f563b06d39946a1b6) : [0, 2237841]: Setting download as failed: File segment: [0, 2237841], key: 51020561b966c23f563b06d39946a1b6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:56:51.988584 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(51020561b966c23f563b06d39946a1b6): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:56:52.047746 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 51020561b966c23f563b06d39946a1b6, offset: 0, size: 0 2024.10.22 01:56:52.051210 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 51020561b966c23f563b06d39946a1b6 for removal 2024.10.22 01:56:52.053670 [ 1476 ] {} CacheMetadata: Key 51020561b966c23f563b06d39946a1b6 is removed from metadata 2024.10.22 01:56:52.245766 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_708_54} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_708_54 to 202410_507_708_54 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:52.249610 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_708_54} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_707_53, 202410_708_708_0] -> 202410_507_708_54 2024.10.22 01:56:52.267153 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.96 MiB. 2024.10.22 01:56:52.302810 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 600 2024.10.22 01:56:52.316700 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5795 2024.10.22 01:56:52.592794 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:52.596782 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:52.673666 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_582_582_0 (state Temporary) 2024.10.22 01:56:52.779885 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_538_110, 202410_539_539_0, 202410_540_540_0, 202410_541_541_0, 202410_542_542_0, 202410_543_543_0] 2024.10.22 01:56:52.788476 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_582_582_0 to 202410_592_592_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:52.789747 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_538_110, 202410_539_539_0, 202410_540_540_0, 202410_541_541_0, 202410_542_542_0, 202410_543_543_0] 2024.10.22 01:56:52.882767 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 600 2024.10.22 01:56:52.947815 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_538_110, 202410_539_539_0, 202410_540_540_0, 202410_541_541_0, 202410_542_542_0, 202410_543_543_0] 2024.10.22 01:56:53.509505 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_688_132} MergeTask::MergeProjectionsStage: Merge sorted 1049713 rows, containing 4 columns (4 merged, 0 gathered) in 31.201817104 sec., 33642.68806849167 rows/sec., 504.35 KiB/sec. 2024.10.22 01:56:53.717071 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(226344a5f03367f71ed7fb3ce84b056f) : [0, 1102136]: Setting download as failed: File segment: [0, 1102136], key: 226344a5f03367f71ed7fb3ce84b056f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:56:53.720591 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(226344a5f03367f71ed7fb3ce84b056f): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:56:53.725185 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_688_132} MergedBlockOutputStream: filled checksums 202410_1_688_132 (state Temporary) 2024.10.22 01:56:53.860171 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 226344a5f03367f71ed7fb3ce84b056f, offset: 0, size: 0 2024.10.22 01:56:53.863179 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 226344a5f03367f71ed7fb3ce84b056f for removal 2024.10.22 01:56:53.866370 [ 1476 ] {} CacheMetadata: Key 226344a5f03367f71ed7fb3ce84b056f is removed from metadata 2024.10.22 01:56:54.444174 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_668_134, 202410_669_669_0, 202410_670_670_0, 202410_671_671_0, 202410_672_672_0, 202410_673_673_0] 2024.10.22 01:56:54.604503 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_668_134, 202410_669_669_0, 202410_670_670_0, 202410_671_671_0, 202410_672_672_0, 202410_673_673_0] 2024.10.22 01:56:54.663376 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_668_134, 202410_669_669_0, 202410_670_670_0, 202410_671_671_0, 202410_672_672_0, 202410_673_673_0] 2024.10.22 01:56:54.900555 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 17234 bytes for 68149b7779f549488f7e8e9b59f4d349:0) 2024.10.22 01:56:54.904843 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 452fc8d761c74478d0efece457be6880, offset: 0, size: 28036 2024.10.22 01:56:54.910988 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 452fc8d761c74478d0efece457be6880 for removal 2024.10.22 01:56:54.919175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 53b14ad33d56c189d38b9e0b06948955, offset: 0, size: 154 2024.10.22 01:56:54.927725 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 53b14ad33d56c189d38b9e0b06948955 for removal 2024.10.22 01:56:54.931429 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 595ee0dab931014823f4f086e30a3bda, offset: 0, size: 208 2024.10.22 01:56:54.941086 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 595ee0dab931014823f4f086e30a3bda for removal 2024.10.22 01:56:54.917653 [ 1476 ] {} CacheMetadata: Key 452fc8d761c74478d0efece457be6880 is removed from metadata 2024.10.22 01:56:55.045616 [ 1476 ] {} CacheMetadata: Key 595ee0dab931014823f4f086e30a3bda is removed from metadata 2024.10.22 01:56:55.057096 [ 1476 ] {} CacheMetadata: Key 53b14ad33d56c189d38b9e0b06948955 is removed from metadata 2024.10.22 01:56:55.249272 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1049213 2024.10.22 01:56:55.498305 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:56:55.592540 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 80101 2024.10.22 01:56:55.626193 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 20.63 MiB, will set to 1.58 GiB (RSS), difference: 10.22 MiB 2024.10.22 01:56:55.998688 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_688_132/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:56:56.085794 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:56.094308 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:56.346616 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b7f924643e98c6536cd13fe4fc90d580) : [0, 142864]: Setting download as failed: File segment: [0, 142864], key: b7f924643e98c6536cd13fe4fc90d580, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:56:56.347037 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1213_1213_0 (state Temporary) 2024.10.22 01:56:56.348747 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b7f924643e98c6536cd13fe4fc90d580): No space left in cache to reserve 142865 bytes, will continue without cache download 2024.10.22 01:56:56.368244 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:56.383792 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:56.393609 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1213_1213_0 to 202410_1226_1226_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:56.436164 [ 1341 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jrk/jxcwuqujrrtxtzbnhjuaeswgxzmcy, size 2833854 2024.10.22 01:56:56.471952 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80101 2024.10.22 01:56:56.497840 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b7f924643e98c6536cd13fe4fc90d580, offset: 0, size: 0 2024.10.22 01:56:56.501279 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b7f924643e98c6536cd13fe4fc90d580 for removal 2024.10.22 01:56:56.504564 [ 1476 ] {} CacheMetadata: Key b7f924643e98c6536cd13fe4fc90d580 is removed from metadata 2024.10.22 01:56:56.541596 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 79914 2024.10.22 01:56:56.751391 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_689_689_0 (state Temporary) 2024.10.22 01:56:56.911481 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_689_689_0 to 202410_696_696_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:56.925480 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:56.932185 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:56.955569 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1049213 2024.10.22 01:56:56.963781 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:56:56.970550 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:56:57.000880 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:56:57.092858 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_707_707_0 (state Temporary) 2024.10.22 01:56:57.139596 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:56:57.160496 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:56:57.228369 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_688_132} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_688_132 to 202410_1_688_132 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:57.258066 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_707_707_0 to 202410_713_713_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:57.263712 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:56:57.292551 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:56:57.318124 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 01:56:57.327741 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(453ef9f47d213835318c871fd0f5b800) : [0, 92333]: Setting download as failed: File segment: [0, 92333], key: 453ef9f47d213835318c871fd0f5b800, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:56:57.332112 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(453ef9f47d213835318c871fd0f5b800): No space left in cache to reserve 92334 bytes, will continue without cache download 2024.10.22 01:56:57.333620 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_688_132} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_682_131, 202410_683_688_1] -> 202410_1_688_132 2024.10.22 01:56:57.384203 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_708_54 to 202410_709_709_0 2024.10.22 01:56:57.394451 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.20 MiB using storage policy from min volume index 0 2024.10.22 01:56:57.408044 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 79914 2024.10.22 01:56:57.423255 [ 1193 ] {} DiskLocal: Reserved 1.20 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:57.445228 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 01:56:57.451737 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 453ef9f47d213835318c871fd0f5b800, offset: 0, size: 0 2024.10.22 01:56:57.452849 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Refresh 2024.10.22 01:56:57.455629 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 453ef9f47d213835318c871fd0f5b800 for removal 2024.10.22 01:56:57.460341 [ 1476 ] {} CacheMetadata: Key 453ef9f47d213835318c871fd0f5b800 is removed from metadata 2024.10.22 01:56:57.473768 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 15.20 MiB, will set to 1.57 GiB (RSS), difference: 1.01 MiB 2024.10.22 01:56:57.476251 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Refresh 2024.10.22 01:56:57.480913 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_709_55} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_708_54 to 202410_709_709_0 into Compact with storage Full 2024.10.22 01:56:57.481632 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fqq/trfifnmaugyjxqbexltabhjsedkwj (LogSeriesLimiter: on interval from 2024-10-22 01:56:35 to 2024-10-22 01:56:57 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:56:57.629170 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Refresh 2024.10.22 01:56:57.659952 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Refresh 2024.10.22 01:56:57.695157 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Refresh 2024.10.22 01:56:57.705010 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_709_55} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:56:57.711416 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Refresh 2024.10.22 01:56:57.723100 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_709_55} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_708_54, total 20249 rows starting from the beginning of the part 2024.10.22 01:56:57.739129 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Refresh 2024.10.22 01:56:57.781367 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_709_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_709_709_0, total 121 rows starting from the beginning of the part 2024.10.22 01:56:58.481154 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(233b64842135035fe7b1cd03a10b2afe) : [0, 202897]: Setting download as failed: File segment: [0, 202897], key: 233b64842135035fe7b1cd03a10b2afe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:56:58.485449 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(233b64842135035fe7b1cd03a10b2afe): No space left in cache to reserve 202898 bytes, will continue without cache download 2024.10.22 01:56:58.617228 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 233b64842135035fe7b1cd03a10b2afe, offset: 0, size: 0 2024.10.22 01:56:58.625016 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 233b64842135035fe7b1cd03a10b2afe for removal 2024.10.22 01:56:58.633366 [ 1476 ] {} CacheMetadata: Key 233b64842135035fe7b1cd03a10b2afe is removed from metadata 2024.10.22 01:56:58.675073 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 10949 2024.10.22 01:56:58.776854 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_709_55} MergeTask::MergeProjectionsStage: Merge sorted 20370 rows, containing 13 columns (13 merged, 0 gathered) in 1.305522307 sec., 15602.95055149908 rows/sec., 6.76 MiB/sec. 2024.10.22 01:56:58.790695 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:56:58.794913 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:56:58.930923 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_720_720_0 (state Temporary) 2024.10.22 01:56:58.950922 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_709_55} MergedBlockOutputStream: filled checksums 202410_507_709_55 (state Temporary) 2024.10.22 01:56:58.969001 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_720_720_0 to 202410_733_733_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:59.012525 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10949 2024.10.22 01:56:59.153670 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_144_25/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:56:59.353118 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_709_55} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_709_55 to 202410_507_709_55 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:56:59.362819 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_709_55} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_708_54, 202410_709_709_0] -> 202410_507_709_55 2024.10.22 01:56:59.496839 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.96 MiB. 2024.10.22 01:56:59.510447 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 12.61 MiB, will set to 1.56 GiB (RSS), difference: -19.48 MiB 2024.10.22 01:56:59.761125 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 7 file segments (while reserving 39547 bytes for b9e94700f8fa58d0ad36c4e944e748ba:0) 2024.10.22 01:56:59.766556 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e8e216162d032f787013338eec9dbf1c, offset: 0, size: 14190 2024.10.22 01:56:59.769096 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e8e216162d032f787013338eec9dbf1c for removal 2024.10.22 01:56:59.770947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 68149b7779f549488f7e8e9b59f4d349, offset: 0, size: 17234 2024.10.22 01:56:59.771085 [ 1476 ] {} CacheMetadata: Key e8e216162d032f787013338eec9dbf1c is removed from metadata 2024.10.22 01:56:59.773241 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_635_635_0, 202410_636_636_0, 202410_637_637_0, 202410_638_638_0, 202410_639_639_0, 202410_640_640_0] 2024.10.22 01:56:59.774762 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 68149b7779f549488f7e8e9b59f4d349 for removal 2024.10.22 01:56:59.774850 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_635_635_0, 202410_636_636_0, 202410_637_637_0, 202410_638_638_0, 202410_639_639_0, 202410_640_640_0] 2024.10.22 01:56:59.776764 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 261f074ad3dceafc1f120f886b209602, offset: 0, size: 142 2024.10.22 01:56:59.782341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 261f074ad3dceafc1f120f886b209602 for removal 2024.10.22 01:56:59.790614 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f725b21eeb4cd168690d13d194aaa3ce, offset: 0, size: 160 2024.10.22 01:56:59.796824 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f725b21eeb4cd168690d13d194aaa3ce for removal 2024.10.22 01:56:59.801551 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b94ae28a3d54fbdf5fd20bc7b50d43b9, offset: 0, size: 2509 2024.10.22 01:56:59.805474 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b94ae28a3d54fbdf5fd20bc7b50d43b9 for removal 2024.10.22 01:56:59.812842 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_635_635_0, 202410_636_636_0, 202410_637_637_0, 202410_638_638_0, 202410_639_639_0, 202410_640_640_0] 2024.10.22 01:56:59.914954 [ 1476 ] {} CacheMetadata: Key f725b21eeb4cd168690d13d194aaa3ce is removed from metadata 2024.10.22 01:56:59.921677 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d297d93ec6477ea6d67da1b0be559630, offset: 0, size: 164 2024.10.22 01:56:59.934815 [ 1476 ] {} CacheMetadata: Key b94ae28a3d54fbdf5fd20bc7b50d43b9 is removed from metadata 2024.10.22 01:56:59.935482 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 5838 2024.10.22 01:56:59.941445 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d297d93ec6477ea6d67da1b0be559630 for removal 2024.10.22 01:56:59.946622 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e24a569897fb0ac2a5dcd55094f134a0, offset: 0, size: 164 2024.10.22 01:56:59.951425 [ 1476 ] {} CacheMetadata: Key d297d93ec6477ea6d67da1b0be559630 is removed from metadata 2024.10.22 01:56:59.952746 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e24a569897fb0ac2a5dcd55094f134a0 for removal 2024.10.22 01:56:59.956985 [ 1476 ] {} CacheMetadata: Key e24a569897fb0ac2a5dcd55094f134a0 is removed from metadata 2024.10.22 01:56:59.970647 [ 1476 ] {} CacheMetadata: Key 261f074ad3dceafc1f120f886b209602 is removed from metadata 2024.10.22 01:56:59.980615 [ 1476 ] {} CacheMetadata: Key 68149b7779f549488f7e8e9b59f4d349 is removed from metadata 2024.10.22 01:57:00.198334 [ 1211 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:57:00.245581 [ 1211 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:57:00.342417 [ 1151 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 251 ms. 2024.10.22 01:57:00.497000 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 105 entries to flush up to offset 80206 2024.10.22 01:57:00.999428 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:57:01.006463 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:57:01.112749 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:01.118872 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:57:01.178174 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1214_1214_0 (state Temporary) 2024.10.22 01:57:01.247457 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1214_1214_0 to 202410_1227_1227_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:01.267920 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1222_3 to 202410_1227_1227_0 2024.10.22 01:57:01.287637 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80206 2024.10.22 01:57:01.390866 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:01.415055 [ 1239 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:57:01.441236 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1222_3 to 202410_1227_1227_0 into Compact with storage Full 2024.10.22 01:57:01.453772 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 8.70 MiB, will set to 1.56 GiB (RSS), difference: -6.06 MiB 2024.10.22 01:57:01.456762 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9fd744aaa8df24bf03fa785b2511e83d) : [0, 47077]: Setting download as failed: File segment: [0, 47077], key: 9fd744aaa8df24bf03fa785b2511e83d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:01.458910 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9fd744aaa8df24bf03fa785b2511e83d): No space left in cache to reserve 47078 bytes, will continue without cache download 2024.10.22 01:57:01.473363 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:01.474690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9fd744aaa8df24bf03fa785b2511e83d, offset: 0, size: 0 2024.10.22 01:57:01.478570 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9fd744aaa8df24bf03fa785b2511e83d for removal 2024.10.22 01:57:01.485803 [ 1476 ] {} CacheMetadata: Key 9fd744aaa8df24bf03fa785b2511e83d is removed from metadata 2024.10.22 01:57:01.495673 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1222_3, total 738 rows starting from the beginning of the part 2024.10.22 01:57:01.529053 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1223_1223_0, total 71 rows starting from the beginning of the part 2024.10.22 01:57:01.654032 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1224_1224_0, total 62 rows starting from the beginning of the part 2024.10.22 01:57:01.680422 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1225_1225_0, total 32 rows starting from the beginning of the part 2024.10.22 01:57:01.705737 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1226_1226_0, total 57 rows starting from the beginning of the part 2024.10.22 01:57:01.738171 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1227_1227_0, total 105 rows starting from the beginning of the part 2024.10.22 01:57:02.467215 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8e57e7916badb2b25ca97363b1835172) : [0, 64847]: Setting download as failed: File segment: [0, 64847], key: 8e57e7916badb2b25ca97363b1835172, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:02.468336 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8e57e7916badb2b25ca97363b1835172): No space left in cache to reserve 64848 bytes, will continue without cache download 2024.10.22 01:57:02.475753 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8e57e7916badb2b25ca97363b1835172, offset: 0, size: 0 2024.10.22 01:57:02.476861 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8e57e7916badb2b25ca97363b1835172 for removal 2024.10.22 01:57:02.492630 [ 1476 ] {} CacheMetadata: Key 8e57e7916badb2b25ca97363b1835172 is removed from metadata 2024.10.22 01:57:02.668020 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 7.36 MiB, will set to 1.56 GiB (RSS), difference: -3.17 MiB 2024.10.22 01:57:02.779147 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} MergeTask::MergeProjectionsStage: Merge sorted 1065 rows, containing 13 columns (13 merged, 0 gathered) in 1.347939083 sec., 790.095052092202 rows/sec., 128.27 KiB/sec. 2024.10.22 01:57:02.814848 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} MergedBlockOutputStream: filled checksums 202410_1207_1227_4 (state Temporary) 2024.10.22 01:57:02.874191 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1227_4 to 202410_1207_1227_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:02.886030 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1222_3, 202410_1227_1227_0] -> 202410_1207_1227_4 2024.10.22 01:57:02.911816 [ 1395 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1227_4}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 01:57:02.928321 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:57:03.227058 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9a951cd47d87c265ca03afcef5810190) : [0, 62908]: Setting download as failed: File segment: [0, 62908], key: 9a951cd47d87c265ca03afcef5810190, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:03.228829 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9a951cd47d87c265ca03afcef5810190): No space left in cache to reserve 62909 bytes, will continue without cache download 2024.10.22 01:57:03.248607 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9a951cd47d87c265ca03afcef5810190, offset: 0, size: 0 2024.10.22 01:57:03.251022 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9a951cd47d87c265ca03afcef5810190 for removal 2024.10.22 01:57:03.255484 [ 1476 ] {} CacheMetadata: Key 9a951cd47d87c265ca03afcef5810190 is removed from metadata 2024.10.22 01:57:03.958631 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1051138 2024.10.22 01:57:04.163377 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 5 file segments (while reserving 34892 bytes for cac168bd6b9334713b5567b07f5693ee:0) 2024.10.22 01:57:04.165768 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 42684df72015de7406e36f99e4892930, offset: 0, size: 154 2024.10.22 01:57:04.170083 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 42684df72015de7406e36f99e4892930 for removal 2024.10.22 01:57:04.172528 [ 1476 ] {} CacheMetadata: Key 42684df72015de7406e36f99e4892930 is removed from metadata 2024.10.22 01:57:04.174089 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b9e94700f8fa58d0ad36c4e944e748ba, offset: 0, size: 39547 2024.10.22 01:57:04.179047 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b9e94700f8fa58d0ad36c4e944e748ba for removal 2024.10.22 01:57:04.185094 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7464010c453b29c9e135fa05c5be9766, offset: 0, size: 176 2024.10.22 01:57:04.194628 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7464010c453b29c9e135fa05c5be9766 for removal 2024.10.22 01:57:04.199425 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d4d9cde7fff1acfaac5f2bbb7407802b, offset: 0, size: 144 2024.10.22 01:57:04.201781 [ 1476 ] {} CacheMetadata: Key 7464010c453b29c9e135fa05c5be9766 is removed from metadata 2024.10.22 01:57:04.203135 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d4d9cde7fff1acfaac5f2bbb7407802b for removal 2024.10.22 01:57:04.208696 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c3de53ba1ec3edb6719c0cfb6c148f51, offset: 0, size: 146 2024.10.22 01:57:04.210673 [ 1476 ] {} CacheMetadata: Key d4d9cde7fff1acfaac5f2bbb7407802b is removed from metadata 2024.10.22 01:57:04.217510 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c3de53ba1ec3edb6719c0cfb6c148f51 for removal 2024.10.22 01:57:04.220789 [ 1476 ] {} CacheMetadata: Key b9e94700f8fa58d0ad36c4e944e748ba is removed from metadata 2024.10.22 01:57:04.234216 [ 1476 ] {} CacheMetadata: Key c3de53ba1ec3edb6719c0cfb6c148f51 is removed from metadata 2024.10.22 01:57:04.349323 [ 1268 ] {} DatabaseCatalog: Execution took 340 ms. 2024.10.22 01:57:04.451050 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 5.20 MiB, will set to 1.55 GiB (RSS), difference: 1.51 MiB 2024.10.22 01:57:05.033046 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 102 entries to flush up to offset 80016 2024.10.22 01:57:05.070459 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:05.074247 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:57:05.302349 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 80270 2024.10.22 01:57:05.416438 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:05.423537 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:05.433210 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_690_690_0 (state Temporary) 2024.10.22 01:57:05.475915 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_708_708_0 (state Temporary) 2024.10.22 01:57:05.534395 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_708_708_0 to 202410_714_714_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:05.572170 [ 1270 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_709_55 to 202410_710_710_0 2024.10.22 01:57:05.577420 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_690_690_0 to 202410_697_697_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:05.582235 [ 1270 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.20 MiB using storage policy from min volume index 0 2024.10.22 01:57:05.591531 [ 1270 ] {} DiskLocal: Reserved 1.20 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 01:57:05.599420 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 80016 2024.10.22 01:57:05.640271 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_710_56} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_709_55 to 202410_710_710_0 into Compact with storage Full 2024.10.22 01:57:05.668790 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_710_56} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:05.717661 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1051138 2024.10.22 01:57:05.753341 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:05.756699 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:05.804135 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_710_56} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_709_55, total 20370 rows starting from the beginning of the part 2024.10.22 01:57:05.943760 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1215_1215_0 (state Temporary) 2024.10.22 01:57:05.971307 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_710_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_710_710_0, total 83 rows starting from the beginning of the part 2024.10.22 01:57:06.079391 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1215_1215_0 to 202410_1228_1228_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:06.167464 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 5 file segments (while reserving 32797 bytes for ddb581caae36f8bf644aa97cc696e84d:0) 2024.10.22 01:57:06.169435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cac168bd6b9334713b5567b07f5693ee, offset: 0, size: 34892 2024.10.22 01:57:06.178153 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cac168bd6b9334713b5567b07f5693ee for removal 2024.10.22 01:57:06.180566 [ 1476 ] {} CacheMetadata: Key cac168bd6b9334713b5567b07f5693ee is removed from metadata 2024.10.22 01:57:06.181311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9be64f443591f1745848449d61a98d7d, offset: 0, size: 154 2024.10.22 01:57:06.192611 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9be64f443591f1745848449d61a98d7d for removal 2024.10.22 01:57:06.197519 [ 1476 ] {} CacheMetadata: Key 9be64f443591f1745848449d61a98d7d is removed from metadata 2024.10.22 01:57:06.198577 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 29b06b4a6f32fa8cee4b52d041e696d2, offset: 0, size: 164 2024.10.22 01:57:06.202137 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 29b06b4a6f32fa8cee4b52d041e696d2 for removal 2024.10.22 01:57:06.210160 [ 1476 ] {} CacheMetadata: Key 29b06b4a6f32fa8cee4b52d041e696d2 is removed from metadata 2024.10.22 01:57:06.209750 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cd9c7504a9cc3a239cbf5b0aafe8c52e, offset: 0, size: 163 2024.10.22 01:57:06.214596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cd9c7504a9cc3a239cbf5b0aafe8c52e for removal 2024.10.22 01:57:06.220588 [ 1476 ] {} CacheMetadata: Key cd9c7504a9cc3a239cbf5b0aafe8c52e is removed from metadata 2024.10.22 01:57:06.320832 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1ecd2b49f5ab7131fad4f781a3ff895f, offset: 0, size: 154 2024.10.22 01:57:06.330581 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1ecd2b49f5ab7131fad4f781a3ff895f for removal 2024.10.22 01:57:06.337573 [ 1476 ] {} CacheMetadata: Key 1ecd2b49f5ab7131fad4f781a3ff895f is removed from metadata 2024.10.22 01:57:06.413609 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80270 2024.10.22 01:57:06.494707 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 5.79 MiB, will set to 1.55 GiB (RSS), difference: -1.32 MiB 2024.10.22 01:57:06.514500 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 10964 2024.10.22 01:57:06.620015 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:06.629662 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:06.662059 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_721_721_0 (state Temporary) 2024.10.22 01:57:06.897158 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_721_721_0 to 202410_734_734_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:06.954251 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10964 2024.10.22 01:57:07.354316 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 20228 bytes for 293d8647756547f1ee3192cf53c9d717:0) 2024.10.22 01:57:07.357028 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ddb581caae36f8bf644aa97cc696e84d, offset: 0, size: 32797 2024.10.22 01:57:07.360375 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ddb581caae36f8bf644aa97cc696e84d for removal 2024.10.22 01:57:07.363236 [ 1476 ] {} CacheMetadata: Key ddb581caae36f8bf644aa97cc696e84d is removed from metadata 2024.10.22 01:57:07.364104 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3195990e39fb777136eb8c33083fd198, offset: 0, size: 154 2024.10.22 01:57:07.368056 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3195990e39fb777136eb8c33083fd198 for removal 2024.10.22 01:57:07.382194 [ 1476 ] {} CacheMetadata: Key 3195990e39fb777136eb8c33083fd198 is removed from metadata 2024.10.22 01:57:07.663870 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_710_56/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:57:07.757570 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_710_56} MergeTask::MergeProjectionsStage: Merge sorted 20453 rows, containing 13 columns (13 merged, 0 gathered) in 2.14216085 sec., 9547.83577526403 rows/sec., 4.13 MiB/sec. 2024.10.22 01:57:07.894591 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 601 2024.10.22 01:57:08.026514 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_710_56} MergedBlockOutputStream: filled checksums 202410_507_710_56 (state Temporary) 2024.10.22 01:57:08.188024 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:08.191913 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:08.316697 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_710_56} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_710_56 to 202410_507_710_56 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:08.356721 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_583_583_0 (state Temporary) 2024.10.22 01:57:08.427782 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_710_56} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_709_55, 202410_710_710_0] -> 202410_507_710_56 2024.10.22 01:57:08.456965 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.92 MiB. 2024.10.22 01:57:08.546547 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_583_583_0 to 202410_593_593_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:08.576809 [ 1223 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_588_120 to 202410_593_593_0 2024.10.22 01:57:08.593624 [ 1223 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:08.597727 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:08.651179 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_588_120 to 202410_593_593_0 into Compact with storage Full 2024.10.22 01:57:08.717557 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 601 2024.10.22 01:57:08.829461 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e2f130d649921df09b22a8babeb31b81) : [0, 64750]: Setting download as failed: File segment: [0, 64750], key: e2f130d649921df09b22a8babeb31b81, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:08.830681 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e2f130d649921df09b22a8babeb31b81): No space left in cache to reserve 64751 bytes, will continue without cache download 2024.10.22 01:57:08.844261 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e2f130d649921df09b22a8babeb31b81, offset: 0, size: 0 2024.10.22 01:57:08.855395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e2f130d649921df09b22a8babeb31b81 for removal 2024.10.22 01:57:08.857759 [ 1476 ] {} CacheMetadata: Key e2f130d649921df09b22a8babeb31b81 is removed from metadata 2024.10.22 01:57:08.907369 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:08.937583 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_588_120, total 632 rows starting from the beginning of the part 2024.10.22 01:57:09.248340 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_589_589_0, total 1 rows starting from the beginning of the part 2024.10.22 01:57:09.473480 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_590_590_0, total 1 rows starting from the beginning of the part 2024.10.22 01:57:09.473739 [ 1205 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 01:57:09.595856 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_591_591_0, total 1 rows starting from the beginning of the part 2024.10.22 01:57:09.664322 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_592_592_0, total 1 rows starting from the beginning of the part 2024.10.22 01:57:09.740899 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_593_593_0, total 1 rows starting from the beginning of the part 2024.10.22 01:57:09.975135 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_144_25/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:57:10.439814 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 80351 2024.10.22 01:57:10.783133 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:10.794833 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:10.880407 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1216_1216_0 (state Temporary) 2024.10.22 01:57:10.898992 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_593_121/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:57:10.939893 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 15685 bytes for 62b90b4031ab662356ba99aad996bcaf:0) 2024.10.22 01:57:10.945179 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 293d8647756547f1ee3192cf53c9d717, offset: 0, size: 20228 2024.10.22 01:57:10.948915 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 293d8647756547f1ee3192cf53c9d717 for removal 2024.10.22 01:57:10.951964 [ 1476 ] {} CacheMetadata: Key 293d8647756547f1ee3192cf53c9d717 is removed from metadata 2024.10.22 01:57:10.951765 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 907d2cf7550828cd3512d9d3b3c18844, offset: 0, size: 154 2024.10.22 01:57:10.955462 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 907d2cf7550828cd3512d9d3b3c18844 for removal 2024.10.22 01:57:10.958294 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1216_1216_0 to 202410_1229_1229_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:10.966583 [ 1476 ] {} CacheMetadata: Key 907d2cf7550828cd3512d9d3b3c18844 is removed from metadata 2024.10.22 01:57:11.001305 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:57:11.005330 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:57:11.028321 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80351 2024.10.22 01:57:11.089092 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1126_6, 202410_1127_1127_0, 202410_1128_1128_0, 202410_1129_1129_0, 202410_1130_1130_0, 202410_1131_1131_0] 2024.10.22 01:57:11.097063 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1126_6, 202410_1127_1127_0, 202410_1128_1128_0, 202410_1129_1129_0, 202410_1130_1130_0, 202410_1131_1131_0] 2024.10.22 01:57:11.117041 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} MergeTask::MergeProjectionsStage: Merge sorted 637 rows, containing 32 columns (32 merged, 0 gathered) in 2.475309116 sec., 257.34159660405015 rows/sec., 23.44 KiB/sec. 2024.10.22 01:57:11.139285 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1126_6, 202410_1127_1127_0, 202410_1128_1128_0, 202410_1129_1129_0, 202410_1130_1130_0, 202410_1131_1131_0] 2024.10.22 01:57:11.167821 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} MergedBlockOutputStream: filled checksums 202410_1_593_121 (state Temporary) 2024.10.22 01:57:11.199336 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} MergeTask::MergeProjectionsStage: Merge sorted 5727 rows, containing 679 columns (679 merged, 0 gathered) in 151.37859555 sec., 37.83229709056447 rows/sec., 50.64 KiB/sec. 2024.10.22 01:57:11.667043 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(19faf7e65588a1b6864c69678e1287a0) : [0, 109748]: Setting download as failed: File segment: [0, 109748], key: 19faf7e65588a1b6864c69678e1287a0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:11.670646 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(19faf7e65588a1b6864c69678e1287a0): No space left in cache to reserve 109749 bytes, will continue without cache download 2024.10.22 01:57:11.684395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 19faf7e65588a1b6864c69678e1287a0, offset: 0, size: 0 2024.10.22 01:57:11.687225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 19faf7e65588a1b6864c69678e1287a0 for removal 2024.10.22 01:57:11.693967 [ 1476 ] {} CacheMetadata: Key 19faf7e65588a1b6864c69678e1287a0 is removed from metadata 2024.10.22 01:57:11.919873 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_593_121 to 202410_1_593_121 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:11.927462 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_593_121} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_588_120, 202410_593_593_0] -> 202410_1_593_121 2024.10.22 01:57:12.059392 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:57:12.411899 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 4.81 MiB, will set to 1.55 GiB (RSS), difference: 3.92 MiB 2024.10.22 01:57:12.468032 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7f56b5f2437cd5a96b9da20643891a1c) : [0, 86548]: Setting download as failed: File segment: [0, 86548], key: 7f56b5f2437cd5a96b9da20643891a1c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:12.469162 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7f56b5f2437cd5a96b9da20643891a1c): No space left in cache to reserve 86549 bytes, will continue without cache download 2024.10.22 01:57:12.481623 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7f56b5f2437cd5a96b9da20643891a1c, offset: 0, size: 0 2024.10.22 01:57:12.487378 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7f56b5f2437cd5a96b9da20643891a1c for removal 2024.10.22 01:57:12.490777 [ 1476 ] {} CacheMetadata: Key 7f56b5f2437cd5a96b9da20643891a1c is removed from metadata 2024.10.22 01:57:12.731193 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1635 entries to flush up to offset 1052773 2024.10.22 01:57:13.120354 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 80 entries to flush up to offset 80096 2024.10.22 01:57:13.223375 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} MergedBlockOutputStream: filled checksums 202410_1_144_25 (state Temporary) 2024.10.22 01:57:13.576023 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 9 file segments (while reserving 32976 bytes for 0a4633740af5a094f23fbb7225e625cb:0) 2024.10.22 01:57:13.585817 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 62b90b4031ab662356ba99aad996bcaf, offset: 0, size: 15685 2024.10.22 01:57:13.595098 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 62b90b4031ab662356ba99aad996bcaf for removal 2024.10.22 01:57:13.601352 [ 1476 ] {} CacheMetadata: Key 62b90b4031ab662356ba99aad996bcaf is removed from metadata 2024.10.22 01:57:13.612747 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:13.619804 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:13.600983 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 58efd1c4006d304b5bf09daffc834d3b, offset: 0, size: 154 2024.10.22 01:57:13.657739 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:13.659569 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:13.697052 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_709_709_0 (state Temporary) 2024.10.22 01:57:13.713819 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_691_691_0 (state Temporary) 2024.10.22 01:57:13.729182 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 58efd1c4006d304b5bf09daffc834d3b for removal 2024.10.22 01:57:13.736953 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f8e515f8c2f6b4dfab7169c37c193987, offset: 0, size: 17786 2024.10.22 01:57:13.737127 [ 1476 ] {} CacheMetadata: Key 58efd1c4006d304b5bf09daffc834d3b is removed from metadata 2024.10.22 01:57:13.743617 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f8e515f8c2f6b4dfab7169c37c193987 for removal 2024.10.22 01:57:13.747128 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7c7da3e0be53f87100e4e88054b94bf9, offset: 0, size: 154 2024.10.22 01:57:13.750480 [ 1476 ] {} CacheMetadata: Key f8e515f8c2f6b4dfab7169c37c193987 is removed from metadata 2024.10.22 01:57:13.750787 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7c7da3e0be53f87100e4e88054b94bf9 for removal 2024.10.22 01:57:13.756563 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 04c601cba87f524a1663dca4b698ea23, offset: 0, size: 1742 2024.10.22 01:57:13.760439 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 04c601cba87f524a1663dca4b698ea23 for removal 2024.10.22 01:57:13.764689 [ 1476 ] {} CacheMetadata: Key 04c601cba87f524a1663dca4b698ea23 is removed from metadata 2024.10.22 01:57:13.764396 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 78337b9e00ce1377f982a0eaafbccfad, offset: 0, size: 1831 2024.10.22 01:57:13.768409 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 78337b9e00ce1377f982a0eaafbccfad for removal 2024.10.22 01:57:13.770361 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_709_709_0 to 202410_715_715_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:13.770481 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 234c0ec5bbacac77463f121348a898ad, offset: 0, size: 154 2024.10.22 01:57:13.773704 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 234c0ec5bbacac77463f121348a898ad for removal 2024.10.22 01:57:13.775607 [ 1476 ] {} CacheMetadata: Key 78337b9e00ce1377f982a0eaafbccfad is removed from metadata 2024.10.22 01:57:13.778176 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e65ab7fb07514d508abb3a76b4a0281f, offset: 0, size: 154 2024.10.22 01:57:13.786235 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e65ab7fb07514d508abb3a76b4a0281f for removal 2024.10.22 01:57:13.791379 [ 1476 ] {} CacheMetadata: Key 234c0ec5bbacac77463f121348a898ad is removed from metadata 2024.10.22 01:57:13.792575 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3d9b8c427070933fe13bf067f1247d3f, offset: 0, size: 164 2024.10.22 01:57:13.797185 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_691_691_0 to 202410_698_698_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:13.800407 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3d9b8c427070933fe13bf067f1247d3f for removal 2024.10.22 01:57:13.805437 [ 1476 ] {} CacheMetadata: Key 3d9b8c427070933fe13bf067f1247d3f is removed from metadata 2024.10.22 01:57:13.814703 [ 1238 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_710_56 to 202410_711_711_0 2024.10.22 01:57:13.825027 [ 1238 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.21 MiB using storage policy from min volume index 0 2024.10.22 01:57:13.828468 [ 1476 ] {} CacheMetadata: Key e65ab7fb07514d508abb3a76b4a0281f is removed from metadata 2024.10.22 01:57:13.833617 [ 1238 ] {} DiskLocal: Reserved 1.21 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:13.835334 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 80096 2024.10.22 01:57:13.836873 [ 1476 ] {} CacheMetadata: Key 7c7da3e0be53f87100e4e88054b94bf9 is removed from metadata 2024.10.22 01:57:13.879148 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_711_57} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_710_56 to 202410_711_711_0 into Compact with storage Full 2024.10.22 01:57:13.930189 [ 1195 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_688_132 to 202410_689_694_1 2024.10.22 01:57:13.950488 [ 1195 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:13.953513 [ 1195 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:13.989557 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_694_133} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_688_132 to 202410_689_694_1 into Wide with storage Full 2024.10.22 01:57:14.024894 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_711_57} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:14.035085 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1052773 2024.10.22 01:57:14.042496 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_694_133} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:14.043621 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_711_57} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_710_56, total 20453 rows starting from the beginning of the part 2024.10.22 01:57:14.053332 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_694_133} MergeTreeSequentialSource: Reading 130 marks from part 202410_1_688_132, total 1049713 rows starting from the beginning of the part 2024.10.22 01:57:14.066359 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_711_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_711_711_0, total 128 rows starting from the beginning of the part 2024.10.22 01:57:14.080241 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_694_133} MergeTreeSequentialSource: Reading 3 marks from part 202410_689_694_1, total 9657 rows starting from the beginning of the part 2024.10.22 01:57:14.456379 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 10979 2024.10.22 01:57:14.575824 [ 1206 ] {} DatabaseCatalog: Execution took 224 ms. 2024.10.22 01:57:14.660171 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:14.666984 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:14.703806 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_722_722_0 (state Temporary) 2024.10.22 01:57:14.754500 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_722_722_0 to 202410_735_735_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:14.756288 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4e38fcd611ee8b850312a9a084ff98e7) : [0, 90483]: Setting download as failed: File segment: [0, 90483], key: 4e38fcd611ee8b850312a9a084ff98e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:14.757280 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4e38fcd611ee8b850312a9a084ff98e7): No space left in cache to reserve 90484 bytes, will continue without cache download 2024.10.22 01:57:14.833605 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10979 2024.10.22 01:57:14.873963 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4e38fcd611ee8b850312a9a084ff98e7, offset: 0, size: 0 2024.10.22 01:57:14.877002 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4e38fcd611ee8b850312a9a084ff98e7 for removal 2024.10.22 01:57:14.889719 [ 1476 ] {} CacheMetadata: Key 4e38fcd611ee8b850312a9a084ff98e7 is removed from metadata 2024.10.22 01:57:15.045589 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 103 entries to flush up to offset 80454 2024.10.22 01:57:15.251677 [ 1172 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:57:15.267969 [ 1172 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:57:15.346154 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 4.97 MiB, will set to 1.55 GiB (RSS), difference: -8.10 MiB 2024.10.22 01:57:15.480692 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:15.484266 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:15.548879 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_711_57} MergeTask::MergeProjectionsStage: Merge sorted 20581 rows, containing 13 columns (13 merged, 0 gathered) in 1.689042286 sec., 12185.01169010993 rows/sec., 5.28 MiB/sec. 2024.10.22 01:57:15.632820 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_711_57} MergedBlockOutputStream: filled checksums 202410_507_711_57 (state Temporary) 2024.10.22 01:57:15.658423 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1217_1217_0 (state Temporary) 2024.10.22 01:57:15.715165 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1217_1217_0 to 202410_1230_1230_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:15.750454 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a117ce3937c80d491329ce7e976eaf47) : [0, 91244]: Setting download as failed: File segment: [0, 91244], key: a117ce3937c80d491329ce7e976eaf47, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:15.752076 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a117ce3937c80d491329ce7e976eaf47): No space left in cache to reserve 91245 bytes, will continue without cache download 2024.10.22 01:57:15.767254 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_711_57} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_711_57 to 202410_507_711_57 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:15.769728 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_711_57} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_710_56, 202410_711_711_0] -> 202410_507_711_57 2024.10.22 01:57:15.770024 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80454 2024.10.22 01:57:15.770095 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a117ce3937c80d491329ce7e976eaf47, offset: 0, size: 0 2024.10.22 01:57:15.771664 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a117ce3937c80d491329ce7e976eaf47 for removal 2024.10.22 01:57:15.775807 [ 1476 ] {} CacheMetadata: Key a117ce3937c80d491329ce7e976eaf47 is removed from metadata 2024.10.22 01:57:15.808106 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.96 MiB. 2024.10.22 01:57:16.073002 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_628_122, 202410_629_634_1] 2024.10.22 01:57:16.075760 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_628_122, 202410_629_634_1] 2024.10.22 01:57:16.214504 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_628_122, 202410_629_634_1] 2024.10.22 01:57:16.221478 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 602 2024.10.22 01:57:16.379580 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:16.387766 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:16.444004 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_584_584_0 (state Temporary) 2024.10.22 01:57:16.502615 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_584_584_0 to 202410_594_594_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:16.700790 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 602 2024.10.22 01:57:16.873020 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:16.875561 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:17.038636 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(00311fd1cb94c5d4a764647975cf17ea) : [0, 48499]: Setting download as failed: File segment: [0, 48499], key: 00311fd1cb94c5d4a764647975cf17ea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:17.044779 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(00311fd1cb94c5d4a764647975cf17ea): No space left in cache to reserve 48500 bytes, will continue without cache download 2024.10.22 01:57:17.071042 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 00311fd1cb94c5d4a764647975cf17ea, offset: 0, size: 0 2024.10.22 01:57:17.073382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 00311fd1cb94c5d4a764647975cf17ea for removal 2024.10.22 01:57:17.076804 [ 1476 ] {} CacheMetadata: Key 00311fd1cb94c5d4a764647975cf17ea is removed from metadata 2024.10.22 01:57:17.266651 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_694_133/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:57:17.507510 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 8.80 MiB, will set to 1.54 GiB (RSS), difference: -1.77 MiB 2024.10.22 01:57:18.144556 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(fef7a65aa3b8b1b57109793f473de3ad) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fef7a65aa3b8b1b57109793f473de3ad, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:57:18.148220 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 76659, file segment info: File segment: [0, 33554431], key: fef7a65aa3b8b1b57109793f473de3ad, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:57:18.150892 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:57:18.866773 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0fa09e92940d35e44c167aeba971330b) : [0, 1103749]: Setting download as failed: File segment: [0, 1103749], key: 0fa09e92940d35e44c167aeba971330b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:18.868737 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0fa09e92940d35e44c167aeba971330b): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:57:18.923508 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0fa09e92940d35e44c167aeba971330b, offset: 0, size: 0 2024.10.22 01:57:18.932049 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0fa09e92940d35e44c167aeba971330b for removal 2024.10.22 01:57:18.939431 [ 1476 ] {} CacheMetadata: Key 0fa09e92940d35e44c167aeba971330b is removed from metadata 2024.10.22 01:57:19.003963 [ 1685 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fqq/trfifnmaugyjxqbexltabhjsedkwj, size 323 2024.10.22 01:57:19.165372 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:57:19.316141 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:57:19.349744 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:57:19.431983 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:57:19.566094 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:57:19.612159 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:57:19.648788 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:57:19.662273 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsRobot 2024.10.22 01:57:19.756887 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:57:19.775709 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 80509 2024.10.22 01:57:19.860118 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:57:19.870871 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:57:19.899248 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:57:19.899646 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kng/fhsunvtnkuyrccdtvmgrgmhkmstlm (LogSeriesLimiter: on interval from 2024-10-22 01:56:57 to 2024-10-22 01:57:19 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:57:20.003240 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:20.007629 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:20.020109 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:57:20.054477 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:57:20.076641 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 01:57:20.264375 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1218_1218_0 (state Temporary) 2024.10.22 01:57:20.320867 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 30847 bytes for 4b9bfb285815e34a89e36597c48c53ca:0) 2024.10.22 01:57:20.322979 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0a4633740af5a094f23fbb7225e625cb, offset: 0, size: 32976 2024.10.22 01:57:20.330406 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0a4633740af5a094f23fbb7225e625cb for removal 2024.10.22 01:57:20.343255 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 02c4ab267bc99f98d735081059d58c06, offset: 0, size: 155 2024.10.22 01:57:20.348680 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 02c4ab267bc99f98d735081059d58c06 for removal 2024.10.22 01:57:20.352382 [ 1476 ] {} CacheMetadata: Key 0a4633740af5a094f23fbb7225e625cb is removed from metadata 2024.10.22 01:57:20.376436 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_145_145_0 (state Temporary) 2024.10.22 01:57:20.351805 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: af9bffb2e41bdde19012d339f1a364e5, offset: 0, size: 154 2024.10.22 01:57:20.391285 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key af9bffb2e41bdde19012d339f1a364e5 for removal 2024.10.22 01:57:20.401197 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 733766aac540f8b941400c78c8d8f0b0, offset: 0, size: 144 2024.10.22 01:57:20.412511 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 733766aac540f8b941400c78c8d8f0b0 for removal 2024.10.22 01:57:20.436213 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1218_1218_0 to 202410_1231_1231_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:20.460041 [ 1476 ] {} CacheMetadata: Key 733766aac540f8b941400c78c8d8f0b0 is removed from metadata 2024.10.22 01:57:20.482977 [ 1476 ] {} CacheMetadata: Key af9bffb2e41bdde19012d339f1a364e5 is removed from metadata 2024.10.22 01:57:20.495737 [ 1476 ] {} CacheMetadata: Key 02c4ab267bc99f98d735081059d58c06 is removed from metadata 2024.10.22 01:57:20.508474 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80509 2024.10.22 01:57:20.806498 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 14.00 MiB, will set to 1.55 GiB (RSS), difference: -10.52 MiB 2024.10.22 01:57:21.015050 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:57:21.032452 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:57:21.041002 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1445 entries to flush up to offset 1054218 2024.10.22 01:57:21.351694 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 118 entries to flush up to offset 80214 2024.10.22 01:57:21.570524 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:21.573243 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:21.734672 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_710_710_0 (state Temporary) 2024.10.22 01:57:21.803740 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 6 file segments (while reserving 26198 bytes for 364e6397b5ef72eb9889f3799e7a0caf:0) 2024.10.22 01:57:21.807994 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4b9bfb285815e34a89e36597c48c53ca, offset: 0, size: 30847 2024.10.22 01:57:21.812185 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4b9bfb285815e34a89e36597c48c53ca for removal 2024.10.22 01:57:21.814670 [ 1476 ] {} CacheMetadata: Key 4b9bfb285815e34a89e36597c48c53ca is removed from metadata 2024.10.22 01:57:21.816677 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 928593d478c2cd879f8723b0d991c16c, offset: 0, size: 160 2024.10.22 01:57:21.822173 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 928593d478c2cd879f8723b0d991c16c for removal 2024.10.22 01:57:21.837160 [ 1476 ] {} CacheMetadata: Key 928593d478c2cd879f8723b0d991c16c is removed from metadata 2024.10.22 01:57:21.828917 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8718ed5d4a9c61ceb7df11324833996a, offset: 0, size: 154 2024.10.22 01:57:21.851343 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8718ed5d4a9c61ceb7df11324833996a for removal 2024.10.22 01:57:21.857012 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e7e43bcd87238d5e24dcfa1e6af4da73, offset: 0, size: 164 2024.10.22 01:57:21.857295 [ 1476 ] {} CacheMetadata: Key 8718ed5d4a9c61ceb7df11324833996a is removed from metadata 2024.10.22 01:57:21.859879 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e7e43bcd87238d5e24dcfa1e6af4da73 for removal 2024.10.22 01:57:21.874441 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2760c45dfc134eace2936e2bcd14def4, offset: 0, size: 165 2024.10.22 01:57:21.875857 [ 1476 ] {} CacheMetadata: Key e7e43bcd87238d5e24dcfa1e6af4da73 is removed from metadata 2024.10.22 01:57:21.882856 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2760c45dfc134eace2936e2bcd14def4 for removal 2024.10.22 01:57:21.888348 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3d003fb587b4acc2332e72f987c669bc, offset: 0, size: 165 2024.10.22 01:57:21.894148 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_710_710_0 to 202410_716_716_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:21.894529 [ 1476 ] {} CacheMetadata: Key 2760c45dfc134eace2936e2bcd14def4 is removed from metadata 2024.10.22 01:57:21.943771 [ 1208 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_711_57 to 202410_712_712_0 2024.10.22 01:57:21.951140 [ 1208 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.22 MiB using storage policy from min volume index 0 2024.10.22 01:57:21.955617 [ 1208 ] {} DiskLocal: Reserved 1.22 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:21.957204 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 80214 2024.10.22 01:57:21.993637 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3d003fb587b4acc2332e72f987c669bc for removal 2024.10.22 01:57:21.997803 [ 1476 ] {} CacheMetadata: Key 3d003fb587b4acc2332e72f987c669bc is removed from metadata 2024.10.22 01:57:22.143935 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_712_58} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_711_57 to 202410_712_712_0 into Compact with storage Full 2024.10.22 01:57:22.267707 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:22.271399 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:22.340981 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 10987 2024.10.22 01:57:22.440879 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_712_58} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:22.468045 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_712_58} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_711_57, total 20581 rows starting from the beginning of the part 2024.10.22 01:57:22.546408 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_712_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_712_712_0, total 118 rows starting from the beginning of the part 2024.10.22 01:57:22.552809 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:22.554894 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:22.598974 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_692_692_0 (state Temporary) 2024.10.22 01:57:22.704318 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_692_692_0 to 202410_699_699_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:22.708627 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_723_723_0 (state Temporary) 2024.10.22 01:57:22.852922 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1054218 2024.10.22 01:57:22.966234 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_723_723_0 to 202410_736_736_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:22.968562 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ee9e49ce72713a72cec73a8802b9cc09) : [0, 46449]: Setting download as failed: File segment: [0, 46449], key: ee9e49ce72713a72cec73a8802b9cc09, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:22.969925 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ee9e49ce72713a72cec73a8802b9cc09): No space left in cache to reserve 46450 bytes, will continue without cache download 2024.10.22 01:57:23.091516 [ 1151 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_731_150 to 202410_732_732_0 2024.10.22 01:57:23.097609 [ 1151 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:23.103940 [ 1151 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:23.107647 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ee9e49ce72713a72cec73a8802b9cc09, offset: 0, size: 0 2024.10.22 01:57:23.113252 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ee9e49ce72713a72cec73a8802b9cc09 for removal 2024.10.22 01:57:23.123382 [ 1476 ] {} CacheMetadata: Key ee9e49ce72713a72cec73a8802b9cc09 is removed from metadata 2024.10.22 01:57:23.140516 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 10987 2024.10.22 01:57:23.252337 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_732_151} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_731_150 to 202410_732_732_0 into Compact with storage Full 2024.10.22 01:57:23.315914 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_732_151} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:23.331596 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_732_151} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_731_150, total 11027 rows starting from the beginning of the part 2024.10.22 01:57:23.456514 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.71 MiB, will set to 1.55 GiB (RSS), difference: -12.73 MiB 2024.10.22 01:57:23.459029 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_732_151} MergeTreeSequentialSource: Reading 2 marks from part 202410_732_732_0, total 16 rows starting from the beginning of the part 2024.10.22 01:57:24.196522 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_712_58/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:57:24.202698 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 603 2024.10.22 01:57:24.304869 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_732_151} MergeTask::MergeProjectionsStage: Merge sorted 11043 rows, containing 12 columns (12 merged, 0 gathered) in 1.073704966 sec., 10284.948239682446 rows/sec., 1.60 MiB/sec. 2024.10.22 01:57:24.367238 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_732_151} MergedBlockOutputStream: filled checksums 202410_1_732_151 (state Temporary) 2024.10.22 01:57:24.419963 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8ce655acd1290832febf97600bb99021) : [0, 71758]: Setting download as failed: File segment: [0, 71758], key: 8ce655acd1290832febf97600bb99021, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:24.424184 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8ce655acd1290832febf97600bb99021): No space left in cache to reserve 71759 bytes, will continue without cache download 2024.10.22 01:57:24.483101 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:24.491378 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:24.552473 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8ce655acd1290832febf97600bb99021, offset: 0, size: 0 2024.10.22 01:57:24.615860 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_585_585_0 (state Temporary) 2024.10.22 01:57:24.710147 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 95 entries to flush up to offset 80604 2024.10.22 01:57:24.711950 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8ce655acd1290832febf97600bb99021 for removal 2024.10.22 01:57:24.720046 [ 1476 ] {} CacheMetadata: Key 8ce655acd1290832febf97600bb99021 is removed from metadata 2024.10.22 01:57:24.819030 [ 1233 ] {} DatabaseCatalog: Execution took 241 ms. 2024.10.22 01:57:24.864342 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_732_151} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_732_151 to 202410_1_732_151 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:24.877990 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_732_151} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_731_150, 202410_732_732_0] -> 202410_1_732_151 2024.10.22 01:57:24.899017 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_585_585_0 to 202410_595_595_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:24.913609 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 01:57:25.202837 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 603 2024.10.22 01:57:25.308653 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:25.317381 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:25.351004 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_712_58} MergeTask::MergeProjectionsStage: Merge sorted 20699 rows, containing 13 columns (13 merged, 0 gathered) in 3.220099214 sec., 6428.062809371562 rows/sec., 2.78 MiB/sec. 2024.10.22 01:57:25.374438 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 5 old parts to remove. Parts: [202410_507_653_30, 202410_654_654_0, 202410_655_655_0, 202410_656_656_0, 202410_657_657_0] 2024.10.22 01:57:25.378514 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from filesystem (serially): Parts: [202410_507_653_30, 202410_654_654_0, 202410_655_655_0, 202410_656_656_0, 202410_657_657_0] 2024.10.22 01:57:25.434653 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from memory: Parts: [202410_507_653_30, 202410_654_654_0, 202410_655_655_0, 202410_656_656_0, 202410_657_657_0] 2024.10.22 01:57:25.466843 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(66ad0a3ba15991220f1c87ddfd0d4235) : [0, 179305]: Setting download as failed: File segment: [0, 179305], key: 66ad0a3ba15991220f1c87ddfd0d4235, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:25.469044 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(66ad0a3ba15991220f1c87ddfd0d4235): No space left in cache to reserve 179306 bytes, will continue without cache download 2024.10.22 01:57:25.522981 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_712_58} MergedBlockOutputStream: filled checksums 202410_507_712_58 (state Temporary) 2024.10.22 01:57:25.619273 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1219_1219_0 (state Temporary) 2024.10.22 01:57:25.650397 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_144_25 to 202410_1_144_25 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:25.655828 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_144_25} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 7 parts: [202410_1_134_24, 202410_144_144_0] -> 202410_1_144_25 2024.10.22 01:57:25.678435 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1219_1219_0 to 202410_1232_1232_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:25.716817 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1227_4 to 202410_1232_1232_0 2024.10.22 01:57:25.734206 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:25.740439 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:25.741223 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 66ad0a3ba15991220f1c87ddfd0d4235, offset: 0, size: 0 2024.10.22 01:57:25.744477 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 66ad0a3ba15991220f1c87ddfd0d4235 for removal 2024.10.22 01:57:25.747585 [ 1476 ] {} CacheMetadata: Key 66ad0a3ba15991220f1c87ddfd0d4235 is removed from metadata 2024.10.22 01:57:25.755072 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80604 2024.10.22 01:57:25.760173 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_712_58} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_712_58 to 202410_507_712_58 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:25.770041 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_712_58} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_711_57, 202410_712_712_0] -> 202410_507_712_58 2024.10.22 01:57:25.813665 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.26 MiB. 2024.10.22 01:57:25.860778 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1227_4 to 202410_1232_1232_0 into Compact with storage Full 2024.10.22 01:57:26.128349 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:26.158452 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1227_4, total 1065 rows starting from the beginning of the part 2024.10.22 01:57:26.348211 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1228_1228_0, total 64 rows starting from the beginning of the part 2024.10.22 01:57:26.368881 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 54.58 MiB. 2024.10.22 01:57:26.383937 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 9.17 MiB, will set to 1.55 GiB (RSS), difference: 11.14 MiB 2024.10.22 01:57:26.445298 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1229_1229_0, total 81 rows starting from the beginning of the part 2024.10.22 01:57:26.464283 [ 1272 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 01:57:26.601737 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1230_1230_0, total 103 rows starting from the beginning of the part 2024.10.22 01:57:26.674208 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1231_1231_0, total 55 rows starting from the beginning of the part 2024.10.22 01:57:26.854548 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1232_1232_0, total 95 rows starting from the beginning of the part 2024.10.22 01:57:26.861558 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d579e7d419d20cda33dda57e49990d70) : [0, 1070126]: Setting download as failed: File segment: [0, 1070126], key: d579e7d419d20cda33dda57e49990d70, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:26.864235 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d579e7d419d20cda33dda57e49990d70): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:57:27.103143 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d579e7d419d20cda33dda57e49990d70, offset: 0, size: 0 2024.10.22 01:57:27.231674 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d579e7d419d20cda33dda57e49990d70 for removal 2024.10.22 01:57:27.236460 [ 1476 ] {} CacheMetadata: Key d579e7d419d20cda33dda57e49990d70 is removed from metadata 2024.10.22 01:57:28.093402 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_694_133/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:57:28.362488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c3b0d16232a2929f5aff82513f8801cd) : [0, 39787]: Setting download as failed: File segment: [0, 39787], key: c3b0d16232a2929f5aff82513f8801cd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:28.364472 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c3b0d16232a2929f5aff82513f8801cd): No space left in cache to reserve 39788 bytes, will continue without cache download 2024.10.22 01:57:28.372288 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c3b0d16232a2929f5aff82513f8801cd, offset: 0, size: 0 2024.10.22 01:57:28.374605 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c3b0d16232a2929f5aff82513f8801cd for removal 2024.10.22 01:57:28.380877 [ 1476 ] {} CacheMetadata: Key c3b0d16232a2929f5aff82513f8801cd is removed from metadata 2024.10.22 01:57:28.714487 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_145_145_0 to 202410_148_148_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:28.857571 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 13.59 MiB, will set to 1.55 GiB (RSS), difference: -1.11 MiB 2024.10.22 01:57:29.070678 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} MergeTask::MergeProjectionsStage: Merge sorted 1463 rows, containing 13 columns (13 merged, 0 gathered) in 3.206206534 sec., 456.3024822280522 rows/sec., 76.92 KiB/sec. 2024.10.22 01:57:29.116161 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} MergedBlockOutputStream: filled checksums 202410_1207_1232_5 (state Temporary) 2024.10.22 01:57:29.470277 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 110 entries to flush up to offset 80324 2024.10.22 01:57:29.503899 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1232_5 to 202410_1207_1232_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:29.513311 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1232_5} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1227_4, 202410_1232_1232_0] -> 202410_1207_1232_5 2024.10.22 01:57:29.552350 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:57:29.609195 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:29.616338 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:29.760794 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 80670 2024.10.22 01:57:29.775002 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(186e7ceaa4c16f7c952289d59203fa76) : [0, 1100420]: Setting download as failed: File segment: [0, 1100420], key: 186e7ceaa4c16f7c952289d59203fa76, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:29.783641 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(186e7ceaa4c16f7c952289d59203fa76): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:57:29.856830 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1164 entries to flush up to offset 1055382 2024.10.22 01:57:29.859350 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_711_711_0 (state Temporary) 2024.10.22 01:57:29.929144 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 186e7ceaa4c16f7c952289d59203fa76, offset: 0, size: 0 2024.10.22 01:57:29.934101 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 186e7ceaa4c16f7c952289d59203fa76 for removal 2024.10.22 01:57:29.943049 [ 1476 ] {} CacheMetadata: Key 186e7ceaa4c16f7c952289d59203fa76 is removed from metadata 2024.10.22 01:57:29.976681 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:29.983985 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:30.028991 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_711_711_0 to 202410_717_717_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:30.106221 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 80324 2024.10.22 01:57:30.076501 [ 1236 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_712_58 to 202410_713_713_0 2024.10.22 01:57:30.192774 [ 1236 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.22 MiB using storage policy from min volume index 0 2024.10.22 01:57:30.198829 [ 1236 ] {} DiskLocal: Reserved 1.22 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:30.225031 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_713_59} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_712_58 to 202410_713_713_0 into Compact with storage Full 2024.10.22 01:57:30.254740 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1220_1220_0 (state Temporary) 2024.10.22 01:57:30.270744 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:30.274665 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:30.274791 [ 1182 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:57:30.300712 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1220_1220_0 to 202410_1233_1233_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:30.353557 [ 1182 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:57:30.357943 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_713_59} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:30.360622 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80670 2024.10.22 01:57:30.368729 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_713_59} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_712_58, total 20699 rows starting from the beginning of the part 2024.10.22 01:57:30.388702 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_693_693_0 (state Temporary) 2024.10.22 01:57:30.415330 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_713_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_713_713_0, total 90 rows starting from the beginning of the part 2024.10.22 01:57:30.492018 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_693_693_0 to 202410_700_700_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:30.556283 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1055382 2024.10.22 01:57:30.547658 [ 1217 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_695_695_0 to 202410_700_700_0 2024.10.22 01:57:30.654263 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 29 entries to flush up to offset 11016 2024.10.22 01:57:30.679598 [ 1217 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:30.686676 [ 1217 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:30.800046 [ 1217 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 277 ms. 2024.10.22 01:57:30.944185 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_695_695_0 to 202410_700_700_0 into Compact with storage Full 2024.10.22 01:57:30.957119 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:30.962076 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_695_695_0, total 1893 rows starting from the beginning of the part 2024.10.22 01:57:30.970713 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_696_696_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:57:30.986078 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1717477751227b6d1b61f99352cf236b) : [0, 58409]: Setting download as failed: File segment: [0, 58409], key: 1717477751227b6d1b61f99352cf236b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:30.885809 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:30.989014 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1717477751227b6d1b61f99352cf236b): No space left in cache to reserve 58410 bytes, will continue without cache download 2024.10.22 01:57:30.993478 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:30.994061 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_697_697_0, total 1925 rows starting from the beginning of the part 2024.10.22 01:57:31.003480 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:57:31.009825 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:57:31.012143 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_698_698_0, total 1635 rows starting from the beginning of the part 2024.10.22 01:57:31.047232 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_724_724_0 (state Temporary) 2024.10.22 01:57:31.049580 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_699_699_0, total 1445 rows starting from the beginning of the part 2024.10.22 01:57:31.055857 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1717477751227b6d1b61f99352cf236b, offset: 0, size: 0 2024.10.22 01:57:31.060118 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1717477751227b6d1b61f99352cf236b for removal 2024.10.22 01:57:31.063853 [ 1476 ] {} CacheMetadata: Key 1717477751227b6d1b61f99352cf236b is removed from metadata 2024.10.22 01:57:31.070405 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_700_700_0, total 1164 rows starting from the beginning of the part 2024.10.22 01:57:31.106359 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_724_724_0 to 202410_737_737_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:31.257320 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_732_151 to 202410_737_737_0 2024.10.22 01:57:31.266777 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:31.269687 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:31.324318 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_732_151 to 202410_737_737_0 into Compact with storage Full 2024.10.22 01:57:31.464176 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 15.33 MiB, will set to 1.56 GiB (RSS), difference: -3.23 MiB 2024.10.22 01:57:31.480362 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11016 2024.10.22 01:57:31.504832 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:31.534278 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_732_151, total 11043 rows starting from the beginning of the part 2024.10.22 01:57:31.688041 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_733_733_0, total 12 rows starting from the beginning of the part 2024.10.22 01:57:31.745606 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_734_734_0, total 15 rows starting from the beginning of the part 2024.10.22 01:57:31.777081 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_735_735_0, total 15 rows starting from the beginning of the part 2024.10.22 01:57:32.023834 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_736_736_0, total 8 rows starting from the beginning of the part 2024.10.22 01:57:32.063952 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_737_737_0, total 29 rows starting from the beginning of the part 2024.10.22 01:57:32.121661 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_713_59} MergeTask::MergeProjectionsStage: Merge sorted 20789 rows, containing 13 columns (13 merged, 0 gathered) in 1.90341683 sec., 10921.937681931708 rows/sec., 4.73 MiB/sec. 2024.10.22 01:57:32.209564 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5838 2024.10.22 01:57:32.354249 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} MergeTask::MergeProjectionsStage: Merge sorted 9602 rows, containing 4 columns (4 merged, 0 gathered) in 1.421872695 sec., 6753.0658924426425 rows/sec., 105.52 KiB/sec. 2024.10.22 01:57:32.426309 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} MergedBlockOutputStream: filled checksums 202410_695_700_1 (state Temporary) 2024.10.22 01:57:32.444066 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(51bec9b316709edbb2782330540d04d8) : [0, 47352]: Setting download as failed: File segment: [0, 47352], key: 51bec9b316709edbb2782330540d04d8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:32.446116 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(51bec9b316709edbb2782330540d04d8): No space left in cache to reserve 47353 bytes, will continue without cache download 2024.10.22 01:57:32.566828 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 51bec9b316709edbb2782330540d04d8, offset: 0, size: 0 2024.10.22 01:57:32.569136 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 51bec9b316709edbb2782330540d04d8 for removal 2024.10.22 01:57:32.571167 [ 1476 ] {} CacheMetadata: Key 51bec9b316709edbb2782330540d04d8 is removed from metadata 2024.10.22 01:57:32.639259 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_713_59} MergedBlockOutputStream: filled checksums 202410_507_713_59 (state Temporary) 2024.10.22 01:57:32.664166 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_695_700_1 to 202410_695_700_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:32.668066 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_695_700_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_695_695_0, 202410_700_700_0] -> 202410_695_700_1 2024.10.22 01:57:32.705720 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 604 2024.10.22 01:57:32.718243 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:57:32.811016 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_713_59} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_713_59 to 202410_507_713_59 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:32.818457 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_713_59} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_712_58, 202410_713_713_0] -> 202410_507_713_59 2024.10.22 01:57:32.973335 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.22 MiB. 2024.10.22 01:57:32.996977 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:33.003696 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:33.171900 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_586_586_0 (state Temporary) 2024.10.22 01:57:33.386825 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_586_586_0 to 202410_596_596_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:33.401034 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} MergeTask::MergeProjectionsStage: Merge sorted 11122 rows, containing 12 columns (12 merged, 0 gathered) in 2.101348984 sec., 5292.790528695923 rows/sec., 843.22 KiB/sec. 2024.10.22 01:57:33.443268 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(84fd609fd40560552c09ced79fb2370c) : [0, 48549]: Setting download as failed: File segment: [0, 48549], key: 84fd609fd40560552c09ced79fb2370c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:33.444706 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(84fd609fd40560552c09ced79fb2370c): No space left in cache to reserve 48550 bytes, will continue without cache download 2024.10.22 01:57:33.455002 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 84fd609fd40560552c09ced79fb2370c, offset: 0, size: 0 2024.10.22 01:57:33.458734 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 84fd609fd40560552c09ced79fb2370c for removal 2024.10.22 01:57:33.460722 [ 1476 ] {} CacheMetadata: Key 84fd609fd40560552c09ced79fb2370c is removed from metadata 2024.10.22 01:57:33.466462 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 604 2024.10.22 01:57:33.550782 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} MergedBlockOutputStream: filled checksums 202410_1_737_152 (state Temporary) 2024.10.22 01:57:33.657478 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_737_152 to 202410_1_737_152 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:33.660874 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_737_152} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_732_151, 202410_737_737_0] -> 202410_1_737_152 2024.10.22 01:57:33.694752 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.54 MiB. 2024.10.22 01:57:34.319071 [ 1166 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 01:57:34.372692 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 99 entries to flush up to offset 80769 2024.10.22 01:57:34.394903 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_673_135, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 01:57:34.398157 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_673_135, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 01:57:34.483645 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_673_135, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 01:57:34.592340 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 14.30 MiB, will set to 1.55 GiB (RSS), difference: 20.89 MiB 2024.10.22 01:57:34.960866 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:34.963515 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:35.047802 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1221_1221_0 (state Temporary) 2024.10.22 01:57:35.055958 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8e72da7b8589320941a26a32f5060500) : [0, 70402]: Setting download as failed: File segment: [0, 70402], key: 8e72da7b8589320941a26a32f5060500, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:35.058298 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8e72da7b8589320941a26a32f5060500): No space left in cache to reserve 70403 bytes, will continue without cache download 2024.10.22 01:57:35.100012 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1221_1221_0 to 202410_1234_1234_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:35.103158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8e72da7b8589320941a26a32f5060500, offset: 0, size: 0 2024.10.22 01:57:35.105029 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8e72da7b8589320941a26a32f5060500 for removal 2024.10.22 01:57:35.108314 [ 1476 ] {} CacheMetadata: Key 8e72da7b8589320941a26a32f5060500 is removed from metadata 2024.10.22 01:57:35.162720 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80769 2024.10.22 01:57:35.639697 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 36028 bytes for c16d0e8b1ee79e47b78319163114ffd5:0) 2024.10.22 01:57:35.644760 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 364e6397b5ef72eb9889f3799e7a0caf, offset: 0, size: 26198 2024.10.22 01:57:35.659860 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 364e6397b5ef72eb9889f3799e7a0caf for removal 2024.10.22 01:57:35.663727 [ 1476 ] {} CacheMetadata: Key 364e6397b5ef72eb9889f3799e7a0caf is removed from metadata 2024.10.22 01:57:35.664058 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 523ce009af8805b84e3f7a2a22f30cf0, offset: 0, size: 154 2024.10.22 01:57:35.668927 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 523ce009af8805b84e3f7a2a22f30cf0 for removal 2024.10.22 01:57:35.700126 [ 1476 ] {} CacheMetadata: Key 523ce009af8805b84e3f7a2a22f30cf0 is removed from metadata 2024.10.22 01:57:36.235194 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1131_7, 202410_1132_1132_0, 202410_1133_1133_0, 202410_1134_1134_0, 202410_1135_1135_0, 202410_1136_1136_0] 2024.10.22 01:57:36.238360 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1131_7, 202410_1132_1132_0, 202410_1133_1133_0, 202410_1134_1134_0, 202410_1135_1135_0, 202410_1136_1136_0] 2024.10.22 01:57:36.258825 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:57:36.266366 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:57:36.277833 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1131_7, 202410_1132_1132_0, 202410_1133_1133_0, 202410_1134_1134_0, 202410_1135_1135_0, 202410_1136_1136_0] 2024.10.22 01:57:36.286833 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:57:36.325488 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:57:36.348867 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:57:36.370801 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:57:36.399164 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:57:36.435539 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 01:57:36.546895 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qqj/grperotgusvxrhcanoqcvnlqypnoo (LogSeriesLimiter: on interval from 2024-10-22 01:57:19 to 2024-10-22 01:57:36 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:57:36.650231 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c4c625d5c198509d41d38492dbe2b4c6) : [0, 46207]: Setting download as failed: File segment: [0, 46207], key: c4c625d5c198509d41d38492dbe2b4c6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:36.654867 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c4c625d5c198509d41d38492dbe2b4c6): No space left in cache to reserve 46208 bytes, will continue without cache download 2024.10.22 01:57:36.668291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c4c625d5c198509d41d38492dbe2b4c6, offset: 0, size: 0 2024.10.22 01:57:36.679379 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c4c625d5c198509d41d38492dbe2b4c6 for removal 2024.10.22 01:57:36.682325 [ 1476 ] {} CacheMetadata: Key c4c625d5c198509d41d38492dbe2b4c6 is removed from metadata 2024.10.22 01:57:37.415476 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 14.53 MiB, will set to 1.55 GiB (RSS), difference: -2.38 MiB 2024.10.22 01:57:37.448048 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(05f823cb4847efff8960650dbbdffcd6) : [0, 160780]: Setting download as failed: File segment: [0, 160780], key: 05f823cb4847efff8960650dbbdffcd6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:37.449397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(05f823cb4847efff8960650dbbdffcd6): No space left in cache to reserve 160781 bytes, will continue without cache download 2024.10.22 01:57:37.560837 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1148 entries to flush up to offset 1056530 2024.10.22 01:57:37.568860 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 05f823cb4847efff8960650dbbdffcd6, offset: 0, size: 0 2024.10.22 01:57:37.570904 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 05f823cb4847efff8960650dbbdffcd6 for removal 2024.10.22 01:57:37.577278 [ 1476 ] {} CacheMetadata: Key 05f823cb4847efff8960650dbbdffcd6 is removed from metadata 2024.10.22 01:57:37.654659 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 111 entries to flush up to offset 80435 2024.10.22 01:57:37.834407 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:37.838471 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:37.884580 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_712_712_0 (state Temporary) 2024.10.22 01:57:38.020780 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:38.024459 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:38.045350 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_712_712_0 to 202410_718_718_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:38.090342 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 80435 2024.10.22 01:57:38.103462 [ 1163 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_713_59 to 202410_714_714_0 2024.10.22 01:57:38.111592 [ 1163 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.23 MiB using storage policy from min volume index 0 2024.10.22 01:57:38.115550 [ 1163 ] {} DiskLocal: Reserved 1.23 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:38.135732 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_714_60} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_713_59 to 202410_714_714_0 into Compact with storage Full 2024.10.22 01:57:38.271347 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_714_60} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:38.276844 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(b2f5e5840afe5e90bd54aeb3da132c30) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b2f5e5840afe5e90bd54aeb3da132c30, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:57:38.279870 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 15581, file segment info: File segment: [0, 33554431], key: b2f5e5840afe5e90bd54aeb3da132c30, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:57:38.283505 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:57:38.405253 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_714_60} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_713_59, total 20789 rows starting from the beginning of the part 2024.10.22 01:57:38.433671 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_694_694_0 (state Temporary) 2024.10.22 01:57:38.436371 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_714_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_714_714_0, total 102 rows starting from the beginning of the part 2024.10.22 01:57:38.508194 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_694_694_0 to 202410_701_701_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:38.732500 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1056530 2024.10.22 01:57:38.755637 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_694_133/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:57:38.983513 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 11038 2024.10.22 01:57:39.063143 [ 1185 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 01:57:39.119355 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:39.167504 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:39.169009 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 80840 2024.10.22 01:57:39.221790 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_725_725_0 (state Temporary) 2024.10.22 01:57:39.381595 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_725_725_0 to 202410_738_738_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:39.416097 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 577 bytes for b16895e12ff677b2eb3959172b8aaa64:0) 2024.10.22 01:57:39.420761 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 109ff692c2f1957f2ff4640a12b3ac2b, offset: 0, size: 176 2024.10.22 01:57:39.431709 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 109ff692c2f1957f2ff4640a12b3ac2b for removal 2024.10.22 01:57:39.437062 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: ef03239cb3e7a9bb38522952432ae155, offset: 0, size: 144 2024.10.22 01:57:39.436395 [ 1476 ] {} CacheMetadata: Key 109ff692c2f1957f2ff4640a12b3ac2b is removed from metadata 2024.10.22 01:57:39.444027 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key ef03239cb3e7a9bb38522952432ae155 for removal 2024.10.22 01:57:39.447074 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: b959879f6f74ac423672f76806c53dad, offset: 0, size: 142 2024.10.22 01:57:39.447961 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11038 2024.10.22 01:57:39.450303 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key b959879f6f74ac423672f76806c53dad for removal 2024.10.22 01:57:39.453643 [ 1476 ] {} CacheMetadata: Key b959879f6f74ac423672f76806c53dad is removed from metadata 2024.10.22 01:57:39.463812 [ 1476 ] {} CacheMetadata: Key ef03239cb3e7a9bb38522952432ae155 is removed from metadata 2024.10.22 01:57:39.464779 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 15.53 MiB, will set to 1.56 GiB (RSS), difference: -1.88 MiB 2024.10.22 01:57:39.500070 [ 1746 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kng/fhsunvtnkuyrccdtvmgrgmhkmstlm, size 15581 2024.10.22 01:57:39.828669 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:39.833036 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:39.856006 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 5878 2024.10.22 01:57:39.878912 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1222_1222_0 (state Temporary) 2024.10.22 01:57:40.043029 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1222_1222_0 to 202410_1235_1235_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:40.051362 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:57:40.057754 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_714_60} MergeTask::MergeProjectionsStage: Merge sorted 20891 rows, containing 13 columns (13 merged, 0 gathered) in 1.93031091 sec., 10822.608882213695 rows/sec., 4.69 MiB/sec. 2024.10.22 01:57:40.096903 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:57:40.097363 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80840 2024.10.22 01:57:40.114661 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:57:40.166580 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:57:40.212233 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:57:40.404452 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_714_60} MergedBlockOutputStream: filled checksums 202410_507_714_60 (state Temporary) 2024.10.22 01:57:40.405212 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:57:40.446773 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(abc21623bf21f6f1920e6d1df6e3f1d2) : [0, 66508]: Setting download as failed: File segment: [0, 66508], key: abc21623bf21f6f1920e6d1df6e3f1d2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:40.449418 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(abc21623bf21f6f1920e6d1df6e3f1d2): No space left in cache to reserve 66509 bytes, will continue without cache download 2024.10.22 01:57:40.453334 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RefererCategories 2024.10.22 01:57:40.465519 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: abc21623bf21f6f1920e6d1df6e3f1d2, offset: 0, size: 0 2024.10.22 01:57:40.467352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key abc21623bf21f6f1920e6d1df6e3f1d2 for removal 2024.10.22 01:57:40.470067 [ 1476 ] {} CacheMetadata: Key abc21623bf21f6f1920e6d1df6e3f1d2 is removed from metadata 2024.10.22 01:57:40.742348 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_714_60} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_714_60 to 202410_507_714_60 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:40.745515 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_714_60} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_713_59, 202410_714_714_0] -> 202410_507_714_60 2024.10.22 01:57:40.755880 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.25 MiB. 2024.10.22 01:57:41.008677 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:57:41.013280 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:57:41.498710 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f5443ace549d31ceabaa1709db0f4359) : [0, 4963]: Setting download as failed: File segment: [0, 4963], key: f5443ace549d31ceabaa1709db0f4359, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:41.500147 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f5443ace549d31ceabaa1709db0f4359): No space left in cache to reserve 4964 bytes, will continue without cache download 2024.10.22 01:57:41.519884 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f5443ace549d31ceabaa1709db0f4359, offset: 0, size: 0 2024.10.22 01:57:41.523394 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f5443ace549d31ceabaa1709db0f4359 for removal 2024.10.22 01:57:41.528633 [ 1476 ] {} CacheMetadata: Key f5443ace549d31ceabaa1709db0f4359 is removed from metadata 2024.10.22 01:57:42.391005 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 164 bytes for 326f963336ef553d430d34d9c3ecc49b:0) 2024.10.22 01:57:42.396542 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 93a21d41e2bfa958d043d08725eae212, offset: 0, size: 160 2024.10.22 01:57:42.399831 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 93a21d41e2bfa958d043d08725eae212 for removal 2024.10.22 01:57:42.405132 [ 1476 ] {} CacheMetadata: Key 93a21d41e2bfa958d043d08725eae212 is removed from metadata 2024.10.22 01:57:42.561135 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 15.05 MiB, will set to 1.55 GiB (RSS), difference: -10.40 MiB 2024.10.22 01:57:42.691403 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_543_111, 202410_544_544_0, 202410_545_545_0, 202410_546_546_0, 202410_547_547_0, 202410_548_548_0] 2024.10.22 01:57:42.695118 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_543_111, 202410_544_544_0, 202410_545_545_0, 202410_546_546_0, 202410_547_547_0, 202410_548_548_0] 2024.10.22 01:57:42.824424 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_543_111, 202410_544_544_0, 202410_545_545_0, 202410_546_546_0, 202410_547_547_0, 202410_548_548_0] 2024.10.22 01:57:43.353149 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6c8615516d66df296b2cb6462ad05b9a) : [0, 40590]: Setting download as failed: File segment: [0, 40590], key: 6c8615516d66df296b2cb6462ad05b9a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:43.354221 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6c8615516d66df296b2cb6462ad05b9a): No space left in cache to reserve 40591 bytes, will continue without cache download 2024.10.22 01:57:43.375582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6c8615516d66df296b2cb6462ad05b9a, offset: 0, size: 0 2024.10.22 01:57:43.378534 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6c8615516d66df296b2cb6462ad05b9a for removal 2024.10.22 01:57:43.382851 [ 1476 ] {} CacheMetadata: Key 6c8615516d66df296b2cb6462ad05b9a is removed from metadata 2024.10.22 01:57:44.114156 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 80900 2024.10.22 01:57:44.432736 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:44.440667 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:44.505405 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1223_1223_0 (state Temporary) 2024.10.22 01:57:44.552723 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1223_1223_0 to 202410_1236_1236_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:44.637071 [ 1229 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 259 ms. 2024.10.22 01:57:44.654991 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 142 bytes for 69f98a845ff1af5d15191f4c8e1867a9:0) 2024.10.22 01:57:44.656737 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: afc2d344d94c983ed290b5fa71a72584, offset: 0, size: 154 2024.10.22 01:57:44.657059 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80900 2024.10.22 01:57:44.659328 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key afc2d344d94c983ed290b5fa71a72584 for removal 2024.10.22 01:57:44.660987 [ 1476 ] {} CacheMetadata: Key afc2d344d94c983ed290b5fa71a72584 is removed from metadata 2024.10.22 01:57:45.202061 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:57:45.245753 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_694_133} MergeTask::MergeProjectionsStage: Merge sorted 1059370 rows, containing 4 columns (4 merged, 0 gathered) in 31.269042198 sec., 33879.19570071636 rows/sec., 508.08 KiB/sec. 2024.10.22 01:57:45.356551 [ 1233 ] {} DatabaseCatalog: Execution took 286 ms. 2024.10.22 01:57:45.359927 [ 1233 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:57:45.387414 [ 1233 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:57:45.470599 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_694_133} MergedBlockOutputStream: filled checksums 202410_1_694_133 (state Temporary) 2024.10.22 01:57:45.490640 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 12.93 MiB, will set to 1.55 GiB (RSS), difference: 2.50 MiB 2024.10.22 01:57:45.614152 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 86 entries to flush up to offset 80521 2024.10.22 01:57:45.741526 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1057685 2024.10.22 01:57:45.794368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(07682a2b69a0c94fa3ca3bee90c88822) : [0, 35062]: Setting download as failed: File segment: [0, 35062], key: 07682a2b69a0c94fa3ca3bee90c88822, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:45.795976 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(07682a2b69a0c94fa3ca3bee90c88822): No space left in cache to reserve 35063 bytes, will continue without cache download 2024.10.22 01:57:45.829265 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 07682a2b69a0c94fa3ca3bee90c88822, offset: 0, size: 0 2024.10.22 01:57:45.831977 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 07682a2b69a0c94fa3ca3bee90c88822 for removal 2024.10.22 01:57:45.835018 [ 1476 ] {} CacheMetadata: Key 07682a2b69a0c94fa3ca3bee90c88822 is removed from metadata 2024.10.22 01:57:45.848658 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:45.850750 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:45.902141 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_713_713_0 (state Temporary) 2024.10.22 01:57:46.069610 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_713_713_0 to 202410_719_719_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:46.110996 [ 1215 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_714_60 to 202410_715_715_0 2024.10.22 01:57:46.123953 [ 1215 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.23 MiB using storage policy from min volume index 0 2024.10.22 01:57:46.129784 [ 1215 ] {} DiskLocal: Reserved 1.23 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:46.138537 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 80521 2024.10.22 01:57:46.143414 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:46.146443 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:46.162744 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_715_61} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_714_60 to 202410_715_715_0 into Compact with storage Full 2024.10.22 01:57:46.275007 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_695_695_0 (state Temporary) 2024.10.22 01:57:46.354778 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_715_61} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:46.398195 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_715_61} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_714_60, total 20891 rows starting from the beginning of the part 2024.10.22 01:57:46.474105 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_715_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_715_715_0, total 80 rows starting from the beginning of the part 2024.10.22 01:57:46.476784 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_695_695_0 to 202410_702_702_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:46.520977 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1057685 2024.10.22 01:57:46.951539 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 11046 2024.10.22 01:57:47.001427 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 146 bytes for 95d5c813618453fb43a34fb76ff0194e:0) 2024.10.22 01:57:47.005019 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 073b69397b5ef6b3fce839c7700cad22, offset: 0, size: 160 2024.10.22 01:57:47.014899 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 073b69397b5ef6b3fce839c7700cad22 for removal 2024.10.22 01:57:47.024252 [ 1476 ] {} CacheMetadata: Key 073b69397b5ef6b3fce839c7700cad22 is removed from metadata 2024.10.22 01:57:47.171609 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:47.173593 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:47.540954 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_726_726_0 (state Temporary) 2024.10.22 01:57:47.584484 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 10.23 MiB, will set to 1.55 GiB (RSS), difference: -6.53 MiB 2024.10.22 01:57:47.710506 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_726_726_0 to 202410_739_739_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:47.768407 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11046 2024.10.22 01:57:47.965688 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_715_61} MergeTask::MergeProjectionsStage: Merge sorted 20971 rows, containing 13 columns (13 merged, 0 gathered) in 1.808055208 sec., 11598.650255374281 rows/sec., 5.02 MiB/sec. 2024.10.22 01:57:48.141743 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_715_61} MergedBlockOutputStream: filled checksums 202410_507_715_61 (state Temporary) 2024.10.22 01:57:48.247331 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(15acaf7d5a03964dac790bc270a8461c) : [0, 146961]: Setting download as failed: File segment: [0, 146961], key: 15acaf7d5a03964dac790bc270a8461c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:48.248239 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(15acaf7d5a03964dac790bc270a8461c): No space left in cache to reserve 146962 bytes, will continue without cache download 2024.10.22 01:57:48.263822 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 15acaf7d5a03964dac790bc270a8461c, offset: 0, size: 0 2024.10.22 01:57:48.265951 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 15acaf7d5a03964dac790bc270a8461c for removal 2024.10.22 01:57:48.270596 [ 1476 ] {} CacheMetadata: Key 15acaf7d5a03964dac790bc270a8461c is removed from metadata 2024.10.22 01:57:48.319061 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_715_61} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_715_61 to 202410_507_715_61 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:48.326178 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_715_61} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_714_60, 202410_715_715_0] -> 202410_507_715_61 2024.10.22 01:57:48.406402 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_694_133} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_694_133 to 202410_1_694_133 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:48.425064 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_694_133} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_688_132, 202410_689_694_1] -> 202410_1_694_133 2024.10.22 01:57:48.468038 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 605 2024.10.22 01:57:48.544858 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 01:57:48.551859 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.22 MiB. 2024.10.22 01:57:48.664304 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 80967 2024.10.22 01:57:48.765017 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:48.843598 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:48.849497 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:48.867805 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:49.017340 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_587_587_0 (state Temporary) 2024.10.22 01:57:49.038236 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1224_1224_0 (state Temporary) 2024.10.22 01:57:49.069766 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1224_1224_0 to 202410_1237_1237_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:49.196537 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1232_5 to 202410_1237_1237_0 2024.10.22 01:57:49.220286 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 147 bytes for 6df2b70c8d2dd788de4be36613e1ed11:0) 2024.10.22 01:57:49.226694 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6179867e7abfb4a25bcff19e67ee70b7, offset: 0, size: 154 2024.10.22 01:57:49.232159 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6179867e7abfb4a25bcff19e67ee70b7 for removal 2024.10.22 01:57:49.234928 [ 1476 ] {} CacheMetadata: Key 6179867e7abfb4a25bcff19e67ee70b7 is removed from metadata 2024.10.22 01:57:49.236848 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 80967 2024.10.22 01:57:49.315025 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_587_587_0 to 202410_597_597_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:49.414267 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:49.458845 [ 1267 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:49.462906 [ 1267 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 295 ms. 2024.10.22 01:57:49.473313 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1232_5 to 202410_1237_1237_0 into Compact with storage Full 2024.10.22 01:57:49.474755 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 605 2024.10.22 01:57:49.553747 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:49.573351 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1232_5, total 1463 rows starting from the beginning of the part 2024.10.22 01:57:49.584303 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 7.88 MiB, will set to 1.55 GiB (RSS), difference: 14.50 MiB 2024.10.22 01:57:49.848151 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1233_1233_0, total 66 rows starting from the beginning of the part 2024.10.22 01:57:49.916295 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1234_1234_0, total 99 rows starting from the beginning of the part 2024.10.22 01:57:49.970010 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1235_1235_0, total 71 rows starting from the beginning of the part 2024.10.22 01:57:50.151139 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1236_1236_0, total 60 rows starting from the beginning of the part 2024.10.22 01:57:50.162590 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(82d1793a06037b262a0661085b985ab0) : [0, 249187]: Setting download as failed: File segment: [0, 249187], key: 82d1793a06037b262a0661085b985ab0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:50.164166 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(82d1793a06037b262a0661085b985ab0): No space left in cache to reserve 249188 bytes, will continue without cache download 2024.10.22 01:57:50.227536 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1237_1237_0, total 67 rows starting from the beginning of the part 2024.10.22 01:57:50.308044 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 82d1793a06037b262a0661085b985ab0, offset: 0, size: 0 2024.10.22 01:57:50.315007 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 82d1793a06037b262a0661085b985ab0 for removal 2024.10.22 01:57:50.317314 [ 1476 ] {} CacheMetadata: Key 82d1793a06037b262a0661085b985ab0 is removed from metadata 2024.10.22 01:57:50.660184 [ 1246 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 330 ms. 2024.10.22 01:57:50.712021 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_641_641_0, 202410_642_642_0, 202410_643_643_0, 202410_644_644_0, 202410_645_645_0, 202410_646_646_0] 2024.10.22 01:57:50.817513 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_641_641_0, 202410_642_642_0, 202410_643_643_0, 202410_644_644_0, 202410_645_645_0, 202410_646_646_0] 2024.10.22 01:57:50.856270 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(b6c9c04a8682ef60aa46ed74cd0404e6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b6c9c04a8682ef60aa46ed74cd0404e6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:57:50.859032 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: b6c9c04a8682ef60aa46ed74cd0404e6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 01:57:50.860572 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:57:51.008925 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:57:51.015540 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:57:51.060696 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_641_641_0, 202410_642_642_0, 202410_643_643_0, 202410_644_644_0, 202410_645_645_0, 202410_646_646_0] 2024.10.22 01:57:51.487792 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for f7220facd3e6a15d8352d8c38453b41f:0) 2024.10.22 01:57:51.493646 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4fd365bda4354da37f4f54d6212fd595, offset: 0, size: 154 2024.10.22 01:57:51.499320 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4fd365bda4354da37f4f54d6212fd595 for removal 2024.10.22 01:57:51.502038 [ 1476 ] {} CacheMetadata: Key 4fd365bda4354da37f4f54d6212fd595 is removed from metadata 2024.10.22 01:57:51.652455 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} MergeTask::MergeProjectionsStage: Merge sorted 1826 rows, containing 13 columns (13 merged, 0 gathered) in 2.18206301 sec., 836.8227643435467 rows/sec., 140.93 KiB/sec. 2024.10.22 01:57:51.676447 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} MergedBlockOutputStream: filled checksums 202410_1207_1237_6 (state Temporary) 2024.10.22 01:57:51.942041 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1237_6 to 202410_1207_1237_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:51.945076 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1237_6} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1232_5, 202410_1237_1237_0] -> 202410_1207_1237_6 2024.10.22 01:57:51.965228 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:57:52.453223 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 5.28 MiB, will set to 1.54 GiB (RSS), difference: -2.49 MiB 2024.10.22 01:57:52.575871 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2d66e51880166c60cb9332b0bebdfad4) : [0, 599638]: Setting download as failed: File segment: [0, 599638], key: 2d66e51880166c60cb9332b0bebdfad4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:52.578487 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2d66e51880166c60cb9332b0bebdfad4): No space left in cache to reserve 599639 bytes, will continue without cache download 2024.10.22 01:57:52.717078 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2d66e51880166c60cb9332b0bebdfad4, offset: 0, size: 0 2024.10.22 01:57:52.722656 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2d66e51880166c60cb9332b0bebdfad4 for removal 2024.10.22 01:57:52.727074 [ 1476 ] {} CacheMetadata: Key 2d66e51880166c60cb9332b0bebdfad4 is removed from metadata 2024.10.22 01:57:53.247656 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 81024 2024.10.22 01:57:53.313944 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 71 bytes for 23ea390ea3992a193356dfe9fcdb7a37:0) 2024.10.22 01:57:53.317455 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 470ccd05893cf188741db28f5a067caa, offset: 0, size: 154 2024.10.22 01:57:53.339162 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 470ccd05893cf188741db28f5a067caa for removal 2024.10.22 01:57:53.345293 [ 1476 ] {} CacheMetadata: Key 470ccd05893cf188741db28f5a067caa is removed from metadata 2024.10.22 01:57:53.464818 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:53.527778 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1160 entries to flush up to offset 1058845 2024.10.22 01:57:53.568623 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:53.644297 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1225_1225_0 (state Temporary) 2024.10.22 01:57:53.649088 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 77 entries to flush up to offset 80598 2024.10.22 01:57:53.673626 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1225_1225_0 to 202410_1238_1238_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:53.753087 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81024 2024.10.22 01:57:53.911954 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:53.916441 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:53.971546 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_714_714_0 (state Temporary) 2024.10.22 01:57:54.011764 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_714_714_0 to 202410_720_720_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:54.044700 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_715_61 to 202410_716_716_0 2024.10.22 01:57:54.061830 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.24 MiB using storage policy from min volume index 0 2024.10.22 01:57:54.064900 [ 1150 ] {} DiskLocal: Reserved 1.24 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:54.067423 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 80598 2024.10.22 01:57:54.194784 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_716_62} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_715_61 to 202410_716_716_0 into Compact with storage Full 2024.10.22 01:57:54.346419 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_716_62} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:54.349625 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for c0441dafe021a52ffb87e505e0ec7d1d:0) 2024.10.22 01:57:54.453311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7a4b190c88f82f6aa31606e11a31c1bf, offset: 0, size: 164 2024.10.22 01:57:54.453441 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_716_62} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_715_61, total 20971 rows starting from the beginning of the part 2024.10.22 01:57:54.456844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7a4b190c88f82f6aa31606e11a31c1bf for removal 2024.10.22 01:57:54.458517 [ 1476 ] {} CacheMetadata: Key 7a4b190c88f82f6aa31606e11a31c1bf is removed from metadata 2024.10.22 01:57:54.465178 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:54.466935 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:54.600078 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_696_696_0 (state Temporary) 2024.10.22 01:57:54.653482 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_696_696_0 to 202410_703_703_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:54.681421 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1058845 2024.10.22 01:57:54.713190 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_716_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_716_716_0, total 118 rows starting from the beginning of the part 2024.10.22 01:57:55.270343 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 11065 2024.10.22 01:57:55.299221 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 142 bytes for 86ea4b0dae121d534c151e3b47a92a7f:0) 2024.10.22 01:57:55.304387 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 535c038c9901b79ff939d1db29c917c3, offset: 0, size: 154 2024.10.22 01:57:55.312904 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 535c038c9901b79ff939d1db29c917c3 for removal 2024.10.22 01:57:55.317401 [ 1476 ] {} CacheMetadata: Key 535c038c9901b79ff939d1db29c917c3 is removed from metadata 2024.10.22 01:57:55.534779 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 5.00 MiB, will set to 1.54 GiB (RSS), difference: -15.64 MiB 2024.10.22 01:57:55.619395 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:55.622435 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:55.857389 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_727_727_0 (state Temporary) 2024.10.22 01:57:56.108725 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_727_727_0 to 202410_740_740_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:56.153431 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11065 2024.10.22 01:57:56.164237 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b9870854113f9b81c47427e57ac77598) : [0, 592023]: Setting download as failed: File segment: [0, 592023], key: b9870854113f9b81c47427e57ac77598, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:56.165164 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b9870854113f9b81c47427e57ac77598): No space left in cache to reserve 592024 bytes, will continue without cache download 2024.10.22 01:57:56.204806 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b9870854113f9b81c47427e57ac77598, offset: 0, size: 0 2024.10.22 01:57:56.211302 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b9870854113f9b81c47427e57ac77598 for removal 2024.10.22 01:57:56.219307 [ 1476 ] {} CacheMetadata: Key b9870854113f9b81c47427e57ac77598 is removed from metadata 2024.10.22 01:57:56.372628 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:56.379109 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:57:56.808089 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 355 bytes for d9d81a548164ac01c185913aebfb9f61:0) 2024.10.22 01:57:56.811418 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8afb3db4e859cb81d67a3fafa7e85a1f, offset: 0, size: 144 2024.10.22 01:57:56.815905 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8afb3db4e859cb81d67a3fafa7e85a1f for removal 2024.10.22 01:57:56.824578 [ 1476 ] {} CacheMetadata: Key 8afb3db4e859cb81d67a3fafa7e85a1f is removed from metadata 2024.10.22 01:57:56.822924 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c6aef097f79227d45f083b9bcd45e376, offset: 0, size: 146 2024.10.22 01:57:56.830584 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c6aef097f79227d45f083b9bcd45e376 for removal 2024.10.22 01:57:56.855182 [ 1476 ] {} CacheMetadata: Key c6aef097f79227d45f083b9bcd45e376 is removed from metadata 2024.10.22 01:57:56.976137 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 606 2024.10.22 01:57:56.987507 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_716_62} MergeTask::MergeProjectionsStage: Merge sorted 21089 rows, containing 13 columns (13 merged, 0 gathered) in 2.903440216 sec., 7263.452467106008 rows/sec., 3.15 MiB/sec. 2024.10.22 01:57:57.176594 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_716_62} MergedBlockOutputStream: filled checksums 202410_507_716_62 (state Temporary) 2024.10.22 01:57:57.353672 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:57.359268 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:57:57.564126 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 149 bytes for dd456863022b4c8acc6b242e22703bf8:0) 2024.10.22 01:57:57.566333 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: eb86dbfc6c4d3ab7a3160a244fc78cb5, offset: 0, size: 142 2024.10.22 01:57:57.568791 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key eb86dbfc6c4d3ab7a3160a244fc78cb5 for removal 2024.10.22 01:57:57.570623 [ 1476 ] {} CacheMetadata: Key eb86dbfc6c4d3ab7a3160a244fc78cb5 is removed from metadata 2024.10.22 01:57:57.591223 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_716_62} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_716_62 to 202410_507_716_62 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:57.598627 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_716_62} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_715_61, 202410_716_716_0] -> 202410_507_716_62 2024.10.22 01:57:57.632067 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_588_588_0 (state Temporary) 2024.10.22 01:57:57.742248 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.26 MiB. 2024.10.22 01:57:57.760774 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 81095 2024.10.22 01:57:57.845635 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_588_588_0 to 202410_598_598_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:57.881675 [ 1157 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_593_121 to 202410_598_598_0 2024.10.22 01:57:57.933009 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 606 2024.10.22 01:57:57.999774 [ 1157 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:58.002498 [ 1157 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:57:58.025206 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_593_121 to 202410_598_598_0 into Compact with storage Full 2024.10.22 01:57:58.135805 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:57:58.139979 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:57:58.235089 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:57:58.291524 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1226_1226_0 (state Temporary) 2024.10.22 01:57:58.347745 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_593_121, total 637 rows starting from the beginning of the part 2024.10.22 01:57:58.423305 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_594_594_0, total 1 rows starting from the beginning of the part 2024.10.22 01:57:58.439443 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1226_1226_0 to 202410_1239_1239_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:57:58.464088 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_595_595_0, total 1 rows starting from the beginning of the part 2024.10.22 01:57:58.497825 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 438 bytes for 425f60facf0ec3084f136a3ab661234a:0) 2024.10.22 01:57:58.503487 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 69f98a845ff1af5d15191f4c8e1867a9, offset: 0, size: 142 2024.10.22 01:57:58.512403 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 69f98a845ff1af5d15191f4c8e1867a9 for removal 2024.10.22 01:57:58.525840 [ 1476 ] {} CacheMetadata: Key 69f98a845ff1af5d15191f4c8e1867a9 is removed from metadata 2024.10.22 01:57:58.527148 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 95d5c813618453fb43a34fb76ff0194e, offset: 0, size: 146 2024.10.22 01:57:58.530322 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_596_596_0, total 1 rows starting from the beginning of the part 2024.10.22 01:57:58.532408 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_657_31, 202410_658_658_0, 202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0] 2024.10.22 01:57:58.535434 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_657_31, 202410_658_658_0, 202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0] 2024.10.22 01:57:58.564415 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_597_597_0, total 1 rows starting from the beginning of the part 2024.10.22 01:57:58.564758 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_657_31, 202410_658_658_0, 202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0] 2024.10.22 01:57:58.584040 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81095 2024.10.22 01:57:58.635753 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 95d5c813618453fb43a34fb76ff0194e for removal 2024.10.22 01:57:58.640874 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 326f963336ef553d430d34d9c3ecc49b, offset: 0, size: 164 2024.10.22 01:57:58.644010 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 326f963336ef553d430d34d9c3ecc49b for removal 2024.10.22 01:57:58.648909 [ 1476 ] {} CacheMetadata: Key 326f963336ef553d430d34d9c3ecc49b is removed from metadata 2024.10.22 01:57:58.658904 [ 1476 ] {} CacheMetadata: Key 95d5c813618453fb43a34fb76ff0194e is removed from metadata 2024.10.22 01:57:58.715614 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_598_598_0, total 1 rows starting from the beginning of the part 2024.10.22 01:57:58.765974 [ 1712 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qqj/grperotgusvxrhcanoqcvnlqypnoo, size 355 2024.10.22 01:57:59.195234 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:57:59.239477 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:57:59.266597 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:57:59.335962 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:57:59.358314 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:57:59.383124 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_146_146_0 (state Temporary) 2024.10.22 01:57:59.399679 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:57:59.449641 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:57:59.460095 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1136_8, 202410_1137_1137_0, 202410_1138_1138_0, 202410_1139_1139_0, 202410_1140_1140_0, 202410_1141_1141_0] 2024.10.22 01:57:59.464885 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 3.91 MiB, will set to 1.54 GiB (RSS), difference: 2.62 MiB 2024.10.22 01:57:59.481649 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column GoalsReached 2024.10.22 01:57:59.461764 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1136_8, 202410_1137_1137_0, 202410_1138_1138_0, 202410_1139_1139_0, 202410_1140_1140_0, 202410_1141_1141_0] 2024.10.22 01:57:59.568801 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wym/xfuksglqhcbeivciwqnnmbxlqlkdf (LogSeriesLimiter: on interval from 2024-10-22 01:57:36 to 2024-10-22 01:57:59 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:57:59.596318 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1136_8, 202410_1137_1137_0, 202410_1138_1138_0, 202410_1139_1139_0, 202410_1140_1140_0, 202410_1141_1141_0] 2024.10.22 01:57:59.600262 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fe5e3afda81ad0a52cad5e67ec3f0494) : [0, 290867]: Setting download as failed: File segment: [0, 290867], key: fe5e3afda81ad0a52cad5e67ec3f0494, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:57:59.601185 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fe5e3afda81ad0a52cad5e67ec3f0494): No space left in cache to reserve 290868 bytes, will continue without cache download 2024.10.22 01:57:59.749476 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fe5e3afda81ad0a52cad5e67ec3f0494, offset: 0, size: 0 2024.10.22 01:57:59.750690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fe5e3afda81ad0a52cad5e67ec3f0494 for removal 2024.10.22 01:57:59.752206 [ 1476 ] {} CacheMetadata: Key fe5e3afda81ad0a52cad5e67ec3f0494 is removed from metadata 2024.10.22 01:58:00.121265 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(e93dc74f5b73428e62201780fb5415ae) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e93dc74f5b73428e62201780fb5415ae, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:58:00.124190 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: e93dc74f5b73428e62201780fb5415ae, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 01:58:00.130148 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:00.177177 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} MergeTask::MergeProjectionsStage: Merge sorted 642 rows, containing 32 columns (32 merged, 0 gathered) in 2.165008842 sec., 296.5345857003202 rows/sec., 26.99 KiB/sec. 2024.10.22 01:58:00.245581 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} MergedBlockOutputStream: filled checksums 202410_1_598_122 (state Temporary) 2024.10.22 01:58:00.506438 [ 1193 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:58:00.570266 [ 1193 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:58:00.654980 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 146 bytes for 386c35af6431508924dee11593a7977a:0) 2024.10.22 01:58:00.659134 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6df2b70c8d2dd788de4be36613e1ed11, offset: 0, size: 147 2024.10.22 01:58:00.664999 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6df2b70c8d2dd788de4be36613e1ed11 for removal 2024.10.22 01:58:00.666808 [ 1476 ] {} CacheMetadata: Key 6df2b70c8d2dd788de4be36613e1ed11 is removed from metadata 2024.10.22 01:58:01.011256 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:58:01.016649 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:58:01.228259 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_598_122 to 202410_1_598_122 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:01.235531 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_598_122} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_593_121, 202410_598_598_0] -> 202410_1_598_122 2024.10.22 01:58:01.379967 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:58:01.445116 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(db9c3729b3b42b8a4ba8082635334ffe) : [0, 175038]: Setting download as failed: File segment: [0, 175038], key: db9c3729b3b42b8a4ba8082635334ffe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:01.549229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(db9c3729b3b42b8a4ba8082635334ffe): No space left in cache to reserve 175039 bytes, will continue without cache download 2024.10.22 01:58:01.563250 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: db9c3729b3b42b8a4ba8082635334ffe, offset: 0, size: 0 2024.10.22 01:58:01.564786 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key db9c3729b3b42b8a4ba8082635334ffe for removal 2024.10.22 01:58:01.567308 [ 1476 ] {} CacheMetadata: Key db9c3729b3b42b8a4ba8082635334ffe is removed from metadata 2024.10.22 01:58:01.575517 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 94 entries to flush up to offset 80692 2024.10.22 01:58:01.687141 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1150 entries to flush up to offset 1059995 2024.10.22 01:58:01.945459 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:01.948465 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:58:02.014657 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_715_715_0 (state Temporary) 2024.10.22 01:58:02.172360 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_715_715_0 to 202410_721_721_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:02.218687 [ 1235 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_716_62 to 202410_717_717_0 2024.10.22 01:58:02.248994 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 80692 2024.10.22 01:58:02.251163 [ 1235 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.25 MiB using storage policy from min volume index 0 2024.10.22 01:58:02.258695 [ 1235 ] {} DiskLocal: Reserved 1.25 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 01:58:02.299836 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_717_63} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_716_62 to 202410_717_717_0 into Compact with storage Full 2024.10.22 01:58:02.356184 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:02.358770 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_717_63} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:02.360319 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:02.368608 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_717_63} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_716_62, total 21089 rows starting from the beginning of the part 2024.10.22 01:58:02.574346 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 153 bytes for 7f5d9c74b46f4b2122d194999b9d422c:0) 2024.10.22 01:58:02.576966 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f7220facd3e6a15d8352d8c38453b41f, offset: 0, size: 155 2024.10.22 01:58:02.581397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f7220facd3e6a15d8352d8c38453b41f for removal 2024.10.22 01:58:02.587371 [ 1476 ] {} CacheMetadata: Key f7220facd3e6a15d8352d8c38453b41f is removed from metadata 2024.10.22 01:58:02.593432 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 91 entries to flush up to offset 81186 2024.10.22 01:58:02.604389 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_717_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_717_717_0, total 110 rows starting from the beginning of the part 2024.10.22 01:58:02.623316 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_697_697_0 (state Temporary) 2024.10.22 01:58:02.770681 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 3.08 MiB, will set to 1.55 GiB (RSS), difference: -14.23 MiB 2024.10.22 01:58:02.830886 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_697_697_0 to 202410_704_704_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:02.866462 [ 1207 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_694_133 to 202410_695_700_1 2024.10.22 01:58:02.869177 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1059995 2024.10.22 01:58:02.884962 [ 1207 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:02.892357 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:03.043607 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_700_134} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_694_133 to 202410_695_700_1 into Wide with storage Full 2024.10.22 01:58:03.071816 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:03.074683 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:03.081146 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_700_134} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:03.096540 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_700_134} MergeTreeSequentialSource: Reading 131 marks from part 202410_1_694_133, total 1059370 rows starting from the beginning of the part 2024.10.22 01:58:03.099675 [ 1207 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 252 ms. 2024.10.22 01:58:03.133838 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_700_134} MergeTreeSequentialSource: Reading 3 marks from part 202410_695_700_1, total 9602 rows starting from the beginning of the part 2024.10.22 01:58:03.292014 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1227_1227_0 (state Temporary) 2024.10.22 01:58:03.345238 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1227_1227_0 to 202410_1240_1240_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:03.423886 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81186 2024.10.22 01:58:03.582239 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(743f3d7eb54a735d8f0a742949324fde) : [0, 472963]: Setting download as failed: File segment: [0, 472963], key: 743f3d7eb54a735d8f0a742949324fde, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:03.587993 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(743f3d7eb54a735d8f0a742949324fde): No space left in cache to reserve 472964 bytes, will continue without cache download 2024.10.22 01:58:03.654909 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 11081 2024.10.22 01:58:03.851830 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 743f3d7eb54a735d8f0a742949324fde, offset: 0, size: 0 2024.10.22 01:58:03.855379 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 743f3d7eb54a735d8f0a742949324fde for removal 2024.10.22 01:58:03.859077 [ 1476 ] {} CacheMetadata: Key 743f3d7eb54a735d8f0a742949324fde is removed from metadata 2024.10.22 01:58:03.898792 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:03.903871 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:03.957303 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_728_728_0 (state Temporary) 2024.10.22 01:58:04.119533 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_728_728_0 to 202410_741_741_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:04.196926 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11081 2024.10.22 01:58:04.804050 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_717_63} MergeTask::MergeProjectionsStage: Merge sorted 21199 rows, containing 13 columns (13 merged, 0 gathered) in 2.517117522 sec., 8421.934937370796 rows/sec., 3.65 MiB/sec. 2024.10.22 01:58:04.874759 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 149 bytes for 2b566416d6bc41c1d8e9f8df5d88bc7d:0) 2024.10.22 01:58:04.880767 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c0441dafe021a52ffb87e505e0ec7d1d, offset: 0, size: 155 2024.10.22 01:58:04.897826 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c0441dafe021a52ffb87e505e0ec7d1d for removal 2024.10.22 01:58:04.903541 [ 1476 ] {} CacheMetadata: Key c0441dafe021a52ffb87e505e0ec7d1d is removed from metadata 2024.10.22 01:58:05.074460 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_717_63} MergedBlockOutputStream: filled checksums 202410_507_717_63 (state Temporary) 2024.10.22 01:58:05.435551 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 607 2024.10.22 01:58:05.466813 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_717_63} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_717_63 to 202410_507_717_63 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:05.469040 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_717_63} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_716_62, 202410_717_717_0] -> 202410_507_717_63 2024.10.22 01:58:05.498782 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.25 MiB. 2024.10.22 01:58:05.647045 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 9.25 MiB, will set to 1.55 GiB (RSS), difference: 3.08 MiB 2024.10.22 01:58:05.766274 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9857324dc654b7c3ea8545ebeae99cc6) : [0, 381309]: Setting download as failed: File segment: [0, 381309], key: 9857324dc654b7c3ea8545ebeae99cc6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:05.770959 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9857324dc654b7c3ea8545ebeae99cc6): No space left in cache to reserve 381310 bytes, will continue without cache download 2024.10.22 01:58:05.772558 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:05.776444 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:05.807109 [ 1198 ] {} DatabaseCatalog: Execution took 317 ms. 2024.10.22 01:58:05.908050 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9857324dc654b7c3ea8545ebeae99cc6, offset: 0, size: 0 2024.10.22 01:58:05.914095 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9857324dc654b7c3ea8545ebeae99cc6 for removal 2024.10.22 01:58:05.920579 [ 1476 ] {} CacheMetadata: Key 9857324dc654b7c3ea8545ebeae99cc6 is removed from metadata 2024.10.22 01:58:06.052208 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_589_589_0 (state Temporary) 2024.10.22 01:58:06.218725 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_589_589_0 to 202410_599_599_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:06.307755 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 607 2024.10.22 01:58:06.471284 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_700_134/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:58:07.039469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 142 bytes for 97db17d5d418d1a94c64c1a246689577:0) 2024.10.22 01:58:07.045822 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dd456863022b4c8acc6b242e22703bf8, offset: 0, size: 149 2024.10.22 01:58:07.051733 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dd456863022b4c8acc6b242e22703bf8 for removal 2024.10.22 01:58:07.054028 [ 1476 ] {} CacheMetadata: Key dd456863022b4c8acc6b242e22703bf8 is removed from metadata 2024.10.22 01:58:07.439322 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 81244 2024.10.22 01:58:07.453188 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_146_146_0 to 202410_149_149_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:07.645369 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:07.649762 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:07.798523 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1228_1228_0 (state Temporary) 2024.10.22 01:58:07.846796 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1228_1228_0 to 202410_1241_1241_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:07.908301 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81244 2024.10.22 01:58:08.071402 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(af1853f288e1ea8e714981d9490305a3) : [0, 63537]: Setting download as failed: File segment: [0, 63537], key: af1853f288e1ea8e714981d9490305a3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:08.073244 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(af1853f288e1ea8e714981d9490305a3): No space left in cache to reserve 63538 bytes, will continue without cache download 2024.10.22 01:58:08.221835 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: af1853f288e1ea8e714981d9490305a3, offset: 0, size: 0 2024.10.22 01:58:08.227199 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key af1853f288e1ea8e714981d9490305a3 for removal 2024.10.22 01:58:08.231171 [ 1476 ] {} CacheMetadata: Key af1853f288e1ea8e714981d9490305a3 is removed from metadata 2024.10.22 01:58:08.358598 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_634_123, 202410_635_640_1] 2024.10.22 01:58:08.361262 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_634_123, 202410_635_640_1] 2024.10.22 01:58:08.494462 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_634_123, 202410_635_640_1] 2024.10.22 01:58:09.133211 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(267c8691c11fdd20accf5ed66a0458d5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 267c8691c11fdd20accf5ed66a0458d5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:58:09.136574 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1012, file segment info: File segment: [0, 33554431], key: 267c8691c11fdd20accf5ed66a0458d5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:58:09.142121 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:09.428355 [ 1270 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 01:58:09.762621 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 101 entries to flush up to offset 80793 2024.10.22 01:58:09.768985 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 142 bytes for 50765eef39719d9168fa8c04b4eaf0e3:0) 2024.10.22 01:58:09.775611 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 386c35af6431508924dee11593a7977a, offset: 0, size: 146 2024.10.22 01:58:09.780533 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 386c35af6431508924dee11593a7977a for removal 2024.10.22 01:58:09.786699 [ 1476 ] {} CacheMetadata: Key 386c35af6431508924dee11593a7977a is removed from metadata 2024.10.22 01:58:10.102428 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1061150 2024.10.22 01:58:10.107169 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:10.217025 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:10.599168 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_716_716_0 (state Temporary) 2024.10.22 01:58:10.654404 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5878 2024.10.22 01:58:10.679935 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 11.03 MiB, will set to 1.55 GiB (RSS), difference: 7.09 MiB 2024.10.22 01:58:10.752567 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_716_716_0 to 202410_722_722_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:10.775101 [ 1260 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_717_63 to 202410_718_718_0 2024.10.22 01:58:10.787330 [ 1260 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.25 MiB using storage policy from min volume index 0 2024.10.22 01:58:10.791523 [ 1260 ] {} DiskLocal: Reserved 1.25 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:10.802482 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 80793 2024.10.22 01:58:10.817104 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_718_64} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_717_63 to 202410_718_718_0 into Compact with storage Full 2024.10.22 01:58:10.819400 [ 1259 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 261 ms. 2024.10.22 01:58:10.821833 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(2dd65d2e402b6d3c239c7a0f17dac5f7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2dd65d2e402b6d3c239c7a0f17dac5f7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:58:10.827686 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 872, file segment info: File segment: [0, 33554431], key: 2dd65d2e402b6d3c239c7a0f17dac5f7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:58:10.831851 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:10.928467 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_718_64} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:10.943467 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_718_64} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_717_63, total 21199 rows starting from the beginning of the part 2024.10.22 01:58:10.979218 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_718_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_718_718_0, total 111 rows starting from the beginning of the part 2024.10.22 01:58:11.012103 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:58:11.014499 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:58:11.245127 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:58:11.254710 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:58:11.319602 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:58:11.323173 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:11.334478 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:11.352416 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:58:11.407908 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:58:11.458879 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:58:11.494598 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column URLCategories 2024.10.22 01:58:11.566835 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_698_698_0 (state Temporary) 2024.10.22 01:58:11.660522 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_698_698_0 to 202410_705_705_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:11.699290 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 11090 2024.10.22 01:58:11.746379 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uxi/axmmvnesbxmkevqjfiewmxykpmtso (LogSeriesLimiter: on interval from 2024-10-22 01:57:59 to 2024-10-22 01:58:11 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:58:11.681651 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1061150 2024.10.22 01:58:11.828588 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a7c608a0b11f9917a5bdd89abea9d1c3) : [0, 63146]: Setting download as failed: File segment: [0, 63146], key: a7c608a0b11f9917a5bdd89abea9d1c3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:11.831840 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a7c608a0b11f9917a5bdd89abea9d1c3): No space left in cache to reserve 63147 bytes, will continue without cache download 2024.10.22 01:58:11.856570 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a7c608a0b11f9917a5bdd89abea9d1c3, offset: 0, size: 0 2024.10.22 01:58:11.857495 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a7c608a0b11f9917a5bdd89abea9d1c3 for removal 2024.10.22 01:58:11.859543 [ 1476 ] {} CacheMetadata: Key a7c608a0b11f9917a5bdd89abea9d1c3 is removed from metadata 2024.10.22 01:58:11.947429 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:12.033091 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 81308 2024.10.22 01:58:12.037233 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:12.150348 [ 1246 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 01:58:12.177481 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_729_729_0 (state Temporary) 2024.10.22 01:58:12.299339 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_718_64} MergeTask::MergeProjectionsStage: Merge sorted 21310 rows, containing 13 columns (13 merged, 0 gathered) in 1.491976648 sec., 14283.065374090223 rows/sec., 6.19 MiB/sec. 2024.10.22 01:58:12.318681 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:12.325774 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:12.331786 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_729_729_0 to 202410_742_742_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:12.377335 [ 1205 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 4 parts from 202410_1_737_152 to 202410_740_740_0 2024.10.22 01:58:12.395371 [ 1205 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:12.399073 [ 1205 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:12.444622 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_740_153} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_737_152 to 202410_740_740_0 into Compact with storage Full 2024.10.22 01:58:12.453657 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11090 2024.10.22 01:58:12.468419 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1229_1229_0 (state Temporary) 2024.10.22 01:58:12.489729 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_740_153} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:12.504059 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_740_153} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_737_152, total 11122 rows starting from the beginning of the part 2024.10.22 01:58:12.556507 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_740_153} MergeTreeSequentialSource: Reading 2 marks from part 202410_738_738_0, total 22 rows starting from the beginning of the part 2024.10.22 01:58:12.573021 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_740_153} MergeTreeSequentialSource: Reading 2 marks from part 202410_739_739_0, total 8 rows starting from the beginning of the part 2024.10.22 01:58:12.607896 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_740_153} MergeTreeSequentialSource: Reading 2 marks from part 202410_740_740_0, total 19 rows starting from the beginning of the part 2024.10.22 01:58:12.731398 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1229_1229_0 to 202410_1242_1242_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:12.760797 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1237_6 to 202410_1242_1242_0 2024.10.22 01:58:12.780721 [ 1152 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:12.783662 [ 1152 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:12.820724 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_718_64} MergedBlockOutputStream: filled checksums 202410_507_718_64 (state Temporary) 2024.10.22 01:58:12.853418 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1237_6 to 202410_1242_1242_0 into Compact with storage Full 2024.10.22 01:58:12.890133 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81308 2024.10.22 01:58:12.903483 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_718_64} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_718_64 to 202410_507_718_64 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:12.907488 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_718_64} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_717_63, 202410_718_718_0] -> 202410_507_718_64 2024.10.22 01:58:12.918444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 142 bytes for b649162de0721d0a983c64ee67cbfc10:0) 2024.10.22 01:58:12.924065 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7f5d9c74b46f4b2122d194999b9d422c, offset: 0, size: 153 2024.10.22 01:58:12.932361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7f5d9c74b46f4b2122d194999b9d422c for removal 2024.10.22 01:58:12.939525 [ 1476 ] {} CacheMetadata: Key 7f5d9c74b46f4b2122d194999b9d422c is removed from metadata 2024.10.22 01:58:12.947489 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.25 MiB. 2024.10.22 01:58:13.042845 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:13.196547 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1237_6, total 1826 rows starting from the beginning of the part 2024.10.22 01:58:13.249444 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_740_153} MergeTask::MergeProjectionsStage: Merge sorted 11171 rows, containing 12 columns (12 merged, 0 gathered) in 0.838054507 sec., 13329.68190814825 rows/sec., 2.07 MiB/sec. 2024.10.22 01:58:13.280160 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_740_153} MergedBlockOutputStream: filled checksums 202410_1_740_153 (state Temporary) 2024.10.22 01:58:13.491905 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1238_1238_0, total 57 rows starting from the beginning of the part 2024.10.22 01:58:13.552267 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 15.29 MiB, will set to 1.55 GiB (RSS), difference: -7.13 MiB 2024.10.22 01:58:13.672040 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1239_1239_0, total 71 rows starting from the beginning of the part 2024.10.22 01:58:13.759255 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1240_1240_0, total 91 rows starting from the beginning of the part 2024.10.22 01:58:13.810605 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 608 2024.10.22 01:58:13.852361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6b29c14673e6aa91077ee2370bb6b72e) : [0, 51369]: Setting download as failed: File segment: [0, 51369], key: 6b29c14673e6aa91077ee2370bb6b72e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:13.854492 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6b29c14673e6aa91077ee2370bb6b72e): No space left in cache to reserve 51370 bytes, will continue without cache download 2024.10.22 01:58:13.899426 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6b29c14673e6aa91077ee2370bb6b72e, offset: 0, size: 0 2024.10.22 01:58:13.901573 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6b29c14673e6aa91077ee2370bb6b72e for removal 2024.10.22 01:58:13.911020 [ 1476 ] {} CacheMetadata: Key 6b29c14673e6aa91077ee2370bb6b72e is removed from metadata 2024.10.22 01:58:13.991995 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1241_1241_0, total 58 rows starting from the beginning of the part 2024.10.22 01:58:14.030774 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 5 old parts to remove. Parts: [202410_1_678_136, 202410_679_679_0, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0] 2024.10.22 01:58:14.032050 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_740_153} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_740_153 to 202410_1_740_153 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:14.033425 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 5 parts from filesystem (serially): Parts: [202410_1_678_136, 202410_679_679_0, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0] 2024.10.22 01:58:14.036440 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_740_153} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 4 parts: [202410_1_737_152, 202410_740_740_0] -> 202410_1_740_153 2024.10.22 01:58:14.075441 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.50 MiB. 2024.10.22 01:58:14.068994 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1242_1242_0, total 64 rows starting from the beginning of the part 2024.10.22 01:58:14.180414 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:14.284945 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:14.360217 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 5 parts from memory: Parts: [202410_1_678_136, 202410_679_679_0, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0] 2024.10.22 01:58:14.461803 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_590_590_0 (state Temporary) 2024.10.22 01:58:14.651597 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_590_590_0 to 202410_600_600_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:14.658015 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 151 bytes for 97033d1c06da59226b5e9ca37cf3fa84:0) 2024.10.22 01:58:14.659142 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2b566416d6bc41c1d8e9f8df5d88bc7d, offset: 0, size: 149 2024.10.22 01:58:14.660825 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2b566416d6bc41c1d8e9f8df5d88bc7d for removal 2024.10.22 01:58:14.663287 [ 1476 ] {} CacheMetadata: Key 2b566416d6bc41c1d8e9f8df5d88bc7d is removed from metadata 2024.10.22 01:58:14.849717 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 608 2024.10.22 01:58:15.502625 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} MergeTask::MergeProjectionsStage: Merge sorted 2167 rows, containing 13 columns (13 merged, 0 gathered) in 2.654459031 sec., 816.3621945913544 rows/sec., 137.28 KiB/sec. 2024.10.22 01:58:15.575606 [ 1238 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:58:15.628440 [ 1238 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:58:15.659544 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} MergedBlockOutputStream: filled checksums 202410_1207_1242_7 (state Temporary) 2024.10.22 01:58:15.823265 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5bb0d69969211ffd6c9807e11647f7a6) : [0, 403822]: Setting download as failed: File segment: [0, 403822], key: 5bb0d69969211ffd6c9807e11647f7a6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:15.828135 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5bb0d69969211ffd6c9807e11647f7a6): No space left in cache to reserve 403823 bytes, will continue without cache download 2024.10.22 01:58:15.855474 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5bb0d69969211ffd6c9807e11647f7a6, offset: 0, size: 0 2024.10.22 01:58:15.857280 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5bb0d69969211ffd6c9807e11647f7a6 for removal 2024.10.22 01:58:15.859407 [ 1476 ] {} CacheMetadata: Key 5bb0d69969211ffd6c9807e11647f7a6 is removed from metadata 2024.10.22 01:58:16.260145 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1242_7 to 202410_1207_1242_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:16.264459 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1242_7} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1237_6, 202410_1242_1242_0] -> 202410_1207_1242_7 2024.10.22 01:58:16.403807 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:58:16.453529 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 15.31 MiB, will set to 1.55 GiB (RSS), difference: 4.95 MiB 2024.10.22 01:58:16.900021 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 81386 2024.10.22 01:58:16.967696 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for d6c5361e3ca24149d427205c5a433e10:0) 2024.10.22 01:58:16.969314 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 97db17d5d418d1a94c64c1a246689577, offset: 0, size: 142 2024.10.22 01:58:16.973014 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 97db17d5d418d1a94c64c1a246689577 for removal 2024.10.22 01:58:16.986865 [ 1476 ] {} CacheMetadata: Key 97db17d5d418d1a94c64c1a246689577 is removed from metadata 2024.10.22 01:58:16.996847 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_700_134/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:58:17.390732 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:17.397122 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:17.470181 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1230_1230_0 (state Temporary) 2024.10.22 01:58:17.629017 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1230_1230_0 to 202410_1243_1243_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:17.695921 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 7 old parts to remove. Parts: [202410_1_124_23, 202410_125_129_1, 202410_130_130_0, 202410_131_131_0, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0] 2024.10.22 01:58:17.698515 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from filesystem (serially): Parts: [202410_1_124_23, 202410_125_129_1, 202410_130_130_0, 202410_131_131_0, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0] 2024.10.22 01:58:17.753270 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from memory: Parts: [202410_1_124_23, 202410_125_129_1, 202410_130_130_0, 202410_131_131_0, 202410_132_132_0, 202410_133_133_0, 202410_134_134_0] 2024.10.22 01:58:17.814125 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81386 2024.10.22 01:58:17.976166 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6d24b089682a0de9ee628dc810e5b846) : [0, 397839]: Setting download as failed: File segment: [0, 397839], key: 6d24b089682a0de9ee628dc810e5b846, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:17.979560 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6d24b089682a0de9ee628dc810e5b846): No space left in cache to reserve 397840 bytes, will continue without cache download 2024.10.22 01:58:18.127428 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6d24b089682a0de9ee628dc810e5b846, offset: 0, size: 0 2024.10.22 01:58:18.131862 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6d24b089682a0de9ee628dc810e5b846 for removal 2024.10.22 01:58:18.140880 [ 1476 ] {} CacheMetadata: Key 6d24b089682a0de9ee628dc810e5b846 is removed from metadata 2024.10.22 01:58:18.297645 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 5917 2024.10.22 01:58:18.325716 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 103 entries to flush up to offset 80896 2024.10.22 01:58:18.580485 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 12.93 MiB, will set to 1.55 GiB (RSS), difference: -1.76 MiB 2024.10.22 01:58:18.675231 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:18.679400 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:18.737308 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_717_717_0 (state Temporary) 2024.10.22 01:58:18.788675 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1062305 2024.10.22 01:58:18.863927 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_717_717_0 to 202410_723_723_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:18.959045 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_718_64 to 202410_719_719_0 2024.10.22 01:58:18.961591 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.26 MiB using storage policy from min volume index 0 2024.10.22 01:58:18.964297 [ 1191 ] {} DiskLocal: Reserved 1.26 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:19.003417 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 80896 2024.10.22 01:58:19.017652 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_719_65} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_718_64 to 202410_719_719_0 into Compact with storage Full 2024.10.22 01:58:19.065687 [ 1155 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 212 ms. 2024.10.22 01:58:19.100240 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 154 bytes for e7ce091aad014637df76d3630dc79c8a:0) 2024.10.22 01:58:19.103448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 50765eef39719d9168fa8c04b4eaf0e3, offset: 0, size: 142 2024.10.22 01:58:19.110719 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 50765eef39719d9168fa8c04b4eaf0e3 for removal 2024.10.22 01:58:19.113931 [ 1476 ] {} CacheMetadata: Key 50765eef39719d9168fa8c04b4eaf0e3 is removed from metadata 2024.10.22 01:58:19.301586 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_719_65} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:19.440507 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_719_65} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_718_64, total 21310 rows starting from the beginning of the part 2024.10.22 01:58:19.538720 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_719_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_719_719_0, total 86 rows starting from the beginning of the part 2024.10.22 01:58:19.743644 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:19.747688 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:19.955410 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 11110 2024.10.22 01:58:19.987151 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_699_699_0 (state Temporary) 2024.10.22 01:58:20.008965 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ab7079a16d105f91742f652ee2825ff3) : [0, 1267]: Setting download as failed: File segment: [0, 1267], key: ab7079a16d105f91742f652ee2825ff3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:20.013005 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ab7079a16d105f91742f652ee2825ff3): No space left in cache to reserve 1268 bytes, will continue without cache download 2024.10.22 01:58:20.049729 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ab7079a16d105f91742f652ee2825ff3, offset: 0, size: 0 2024.10.22 01:58:20.052297 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ab7079a16d105f91742f652ee2825ff3 for removal 2024.10.22 01:58:20.056622 [ 1476 ] {} CacheMetadata: Key ab7079a16d105f91742f652ee2825ff3 is removed from metadata 2024.10.22 01:58:20.244063 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_719_65/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:58:20.273255 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_699_699_0 to 202410_706_706_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:20.280630 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:20.292865 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:20.342996 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_730_730_0 (state Temporary) 2024.10.22 01:58:20.398563 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_730_730_0 to 202410_743_743_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:20.432510 [ 1196 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_701_701_0 to 202410_706_706_0 2024.10.22 01:58:20.447232 [ 1196 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:20.462618 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11110 2024.10.22 01:58:20.467729 [ 1196 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:20.534704 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 11.10 MiB, will set to 1.55 GiB (RSS), difference: -3.79 MiB 2024.10.22 01:58:20.603333 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_701_701_0 to 202410_706_706_0 into Compact with storage Full 2024.10.22 01:58:20.648053 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:20.657319 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_701_701_0, total 1148 rows starting from the beginning of the part 2024.10.22 01:58:20.751454 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1062305 2024.10.22 01:58:20.766218 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_702_702_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:58:20.773583 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_703_703_0, total 1160 rows starting from the beginning of the part 2024.10.22 01:58:20.787236 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_704_704_0, total 1150 rows starting from the beginning of the part 2024.10.22 01:58:20.768814 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(974b61f4d096d6fe80b2d655a95b6c06) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 974b61f4d096d6fe80b2d655a95b6c06, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:58:20.793491 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 154470, file segment info: File segment: [0, 33554431], key: 974b61f4d096d6fe80b2d655a95b6c06, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:58:20.796266 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:20.826579 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_705_705_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:58:20.858843 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_706_706_0, total 1155 rows starting from the beginning of the part 2024.10.22 01:58:21.012669 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:58:21.119367 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:58:21.202469 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_719_65} MergeTask::MergeProjectionsStage: Merge sorted 21396 rows, containing 13 columns (13 merged, 0 gathered) in 2.209752751 sec., 9682.53121998263 rows/sec., 4.19 MiB/sec. 2024.10.22 01:58:21.355023 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_719_65} MergedBlockOutputStream: filled checksums 202410_507_719_65 (state Temporary) 2024.10.22 01:58:21.417419 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_719_65} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_719_65 to 202410_507_719_65 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:21.422425 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_719_65} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_718_64, 202410_719_719_0] -> 202410_507_719_65 2024.10.22 01:58:21.463952 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.22 MiB. 2024.10.22 01:58:21.474409 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 188 bytes for 4358eacfa1bdf0622c3d7f2a09d17c47:0) 2024.10.22 01:58:21.479176 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b649162de0721d0a983c64ee67cbfc10, offset: 0, size: 142 2024.10.22 01:58:21.481656 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b649162de0721d0a983c64ee67cbfc10 for removal 2024.10.22 01:58:21.484985 [ 1476 ] {} CacheMetadata: Key b649162de0721d0a983c64ee67cbfc10 is removed from metadata 2024.10.22 01:58:21.599761 [ 1231 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 5 parts from 202410_145_145_0 to 202410_149_149_0 2024.10.22 01:58:21.715808 [ 1231 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:21.719033 [ 1231 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:21.842718 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 81472 2024.10.22 01:58:21.844369 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_145_149_1} MergeTask::PrepareStage: Merging 5 parts: from 202410_145_145_0 to 202410_149_149_0 into Compact with storage Full 2024.10.22 01:58:21.966881 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:21.970119 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:22.092779 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(172fb7139d6e3e143efb1afe07549d26) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 172fb7139d6e3e143efb1afe07549d26, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:58:22.096132 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 685, file segment info: File segment: [0, 33554431], key: 172fb7139d6e3e143efb1afe07549d26, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:58:22.099416 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:22.151913 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1231_1231_0 (state Temporary) 2024.10.22 01:58:22.156618 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} MergeTask::MergeProjectionsStage: Merge sorted 6923 rows, containing 4 columns (4 merged, 0 gathered) in 1.66687474 sec., 4153.281487725946 rows/sec., 64.90 KiB/sec. 2024.10.22 01:58:22.187050 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1231_1231_0 to 202410_1244_1244_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:22.296220 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} MergedBlockOutputStream: filled checksums 202410_701_706_1 (state Temporary) 2024.10.22 01:58:22.323737 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81472 2024.10.22 01:58:22.352397 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 609 2024.10.22 01:58:22.550534 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_701_706_1 to 202410_701_706_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:22.554176 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_701_706_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_701_701_0, 202410_706_706_0] -> 202410_701_706_1 2024.10.22 01:58:22.601745 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:58:22.839901 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:22.842960 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:22.904356 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_591_591_0 (state Temporary) 2024.10.22 01:58:22.967441 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_591_591_0 to 202410_601_601_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:23.028164 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 609 2024.10.22 01:58:23.044069 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cb458b9ad08bdadc0414b7f47c84319b) : [0, 974316]: Setting download as failed: File segment: [0, 974316], key: cb458b9ad08bdadc0414b7f47c84319b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:23.046583 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cb458b9ad08bdadc0414b7f47c84319b): No space left in cache to reserve 974317 bytes, will continue without cache download 2024.10.22 01:58:23.079447 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cb458b9ad08bdadc0414b7f47c84319b, offset: 0, size: 0 2024.10.22 01:58:23.080783 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cb458b9ad08bdadc0414b7f47c84319b for removal 2024.10.22 01:58:23.095703 [ 1476 ] {} CacheMetadata: Key cb458b9ad08bdadc0414b7f47c84319b is removed from metadata 2024.10.22 01:58:23.467440 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 11.19 MiB, will set to 1.55 GiB (RSS), difference: 14.88 MiB 2024.10.22 01:58:23.858410 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(e3788eea9ccef41e41514bd0c9c29169) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e3788eea9ccef41e41514bd0c9c29169, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:58:23.861090 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 487644, file segment info: File segment: [0, 33554431], key: e3788eea9ccef41e41514bd0c9c29169, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:58:23.864240 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:24.343176 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 196 bytes for bfc49cae30ae8d384d3dad4477ad0b87:0) 2024.10.22 01:58:24.345004 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 97033d1c06da59226b5e9ca37cf3fa84, offset: 0, size: 151 2024.10.22 01:58:24.347673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 97033d1c06da59226b5e9ca37cf3fa84 for removal 2024.10.22 01:58:24.350109 [ 1476 ] {} CacheMetadata: Key 97033d1c06da59226b5e9ca37cf3fa84 is removed from metadata 2024.10.22 01:58:24.353577 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1141_9, 202410_1142_1142_0, 202410_1143_1143_0, 202410_1144_1144_0, 202410_1145_1145_0, 202410_1146_1146_0] 2024.10.22 01:58:24.356818 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1141_9, 202410_1142_1142_0, 202410_1143_1143_0, 202410_1144_1144_0, 202410_1145_1145_0, 202410_1146_1146_0] 2024.10.22 01:58:24.606013 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1141_9, 202410_1142_1142_0, 202410_1143_1143_0, 202410_1144_1144_0, 202410_1145_1145_0, 202410_1146_1146_0] 2024.10.22 01:58:24.864012 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(6b826047edc63f9d5549aa90972b4c6e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6b826047edc63f9d5549aa90972b4c6e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:58:24.865934 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 757, file segment info: File segment: [0, 33554431], key: 6b826047edc63f9d5549aa90972b4c6e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 01:58:24.867446 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:25.003909 [ 1692 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wym/xfuksglqhcbeivciwqnnmbxlqlkdf, size 487644 2024.10.22 01:58:25.194650 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:58:25.314886 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:58:25.435516 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:58:25.460386 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fa275c126836d3d30729e492ba7b921e) : [0, 1449188]: Setting download as failed: File segment: [0, 1449188], key: fa275c126836d3d30729e492ba7b921e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:25.462409 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fa275c126836d3d30729e492ba7b921e): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:58:25.566741 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:58:25.620658 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:58:25.634435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fa275c126836d3d30729e492ba7b921e, offset: 0, size: 0 2024.10.22 01:58:25.637421 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fa275c126836d3d30729e492ba7b921e for removal 2024.10.22 01:58:25.671046 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:58:25.703472 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:58:25.745042 [ 1476 ] {} CacheMetadata: Key fa275c126836d3d30729e492ba7b921e is removed from metadata 2024.10.22 01:58:25.752412 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 01:58:25.942172 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/agd/zvonotuuivvdipouvoqunrxokewxw (LogSeriesLimiter: on interval from 2024-10-22 01:58:11 to 2024-10-22 01:58:25 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:58:26.434870 [ 1254 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 01:58:26.440028 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 81527 2024.10.22 01:58:26.524573 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 80996 2024.10.22 01:58:26.659634 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 439a625e53519d47fda4321d7c36a8c8:0) 2024.10.22 01:58:26.661109 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d6c5361e3ca24149d427205c5a433e10, offset: 0, size: 152 2024.10.22 01:58:26.663227 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d6c5361e3ca24149d427205c5a433e10 for removal 2024.10.22 01:58:26.664475 [ 1476 ] {} CacheMetadata: Key d6c5361e3ca24149d427205c5a433e10 is removed from metadata 2024.10.22 01:58:26.755895 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:26.759887 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:26.784803 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:26.795099 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:26.883543 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1232_1232_0 (state Temporary) 2024.10.22 01:58:26.937412 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_718_718_0 (state Temporary) 2024.10.22 01:58:26.950636 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1232_1232_0 to 202410_1245_1245_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:26.999165 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_718_718_0 to 202410_724_724_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:27.044799 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_719_65 to 202410_720_720_0 2024.10.22 01:58:27.051501 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81527 2024.10.22 01:58:27.063618 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 80996 2024.10.22 01:58:27.053776 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.26 MiB using storage policy from min volume index 0 2024.10.22 01:58:27.157822 [ 1250 ] {} DiskLocal: Reserved 1.26 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:27.299008 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_700_134/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:58:27.325359 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_720_66} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_719_65 to 202410_720_720_0 into Compact with storage Full 2024.10.22 01:58:27.363327 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_720_66} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:27.368641 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_720_66} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_719_65, total 21396 rows starting from the beginning of the part 2024.10.22 01:58:27.382438 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_720_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_720_720_0, total 77 rows starting from the beginning of the part 2024.10.22 01:58:27.682438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8ac08fde46cebf4985562199952a39d9) : [0, 443052]: Setting download as failed: File segment: [0, 443052], key: 8ac08fde46cebf4985562199952a39d9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:27.688194 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8ac08fde46cebf4985562199952a39d9): No space left in cache to reserve 443053 bytes, will continue without cache download 2024.10.22 01:58:27.704556 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8ac08fde46cebf4985562199952a39d9, offset: 0, size: 0 2024.10.22 01:58:27.754977 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1063845 2024.10.22 01:58:27.807485 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8ac08fde46cebf4985562199952a39d9 for removal 2024.10.22 01:58:27.813585 [ 1476 ] {} CacheMetadata: Key 8ac08fde46cebf4985562199952a39d9 is removed from metadata 2024.10.22 01:58:27.978014 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 11126 2024.10.22 01:58:28.086688 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:28.192971 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:28.332263 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_731_731_0 (state Temporary) 2024.10.22 01:58:28.412202 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_731_731_0 to 202410_744_744_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:28.579100 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11126 2024.10.22 01:58:28.661139 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 11.74 MiB, will set to 1.55 GiB (RSS), difference: -27.43 MiB 2024.10.22 01:58:28.668392 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 154 bytes for a5f5fefb81d6c8dbb05cb0ef1b0dc92b:0) 2024.10.22 01:58:28.671392 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e7ce091aad014637df76d3630dc79c8a, offset: 0, size: 154 2024.10.22 01:58:28.675115 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e7ce091aad014637df76d3630dc79c8a for removal 2024.10.22 01:58:28.695614 [ 1476 ] {} CacheMetadata: Key e7ce091aad014637df76d3630dc79c8a is removed from metadata 2024.10.22 01:58:28.822317 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:28.827108 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:28.941158 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_700_700_0 (state Temporary) 2024.10.22 01:58:29.026282 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_700_700_0 to 202410_707_707_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:29.080345 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1063845 2024.10.22 01:58:29.158308 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_720_66} MergeTask::MergeProjectionsStage: Merge sorted 21473 rows, containing 13 columns (13 merged, 0 gathered) in 1.953625016 sec., 10991.362121255721 rows/sec., 4.76 MiB/sec. 2024.10.22 01:58:29.234333 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_720_66} MergedBlockOutputStream: filled checksums 202410_507_720_66 (state Temporary) 2024.10.22 01:58:29.341379 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_720_66} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_720_66 to 202410_507_720_66 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:29.345638 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_720_66} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_719_65, 202410_720_720_0] -> 202410_507_720_66 2024.10.22 01:58:29.463554 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.22 MiB. 2024.10.22 01:58:29.640281 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0c58b1541584fa864cc069536dd4be75) : [0, 40279]: Setting download as failed: File segment: [0, 40279], key: 0c58b1541584fa864cc069536dd4be75, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:58:29.643237 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0c58b1541584fa864cc069536dd4be75): No space left in cache to reserve 40280 bytes, will continue without cache download 2024.10.22 01:58:29.659917 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0c58b1541584fa864cc069536dd4be75, offset: 0, size: 0 2024.10.22 01:58:29.661190 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0c58b1541584fa864cc069536dd4be75 for removal 2024.10.22 01:58:29.663539 [ 1476 ] {} CacheMetadata: Key 0c58b1541584fa864cc069536dd4be75 is removed from metadata 2024.10.22 01:58:29.663763 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lsh/tcvagrkxswftkyqbvyegnovbgfixe. 2024.10.22 01:58:30.094942 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_10_10_0 to 201403_10_10_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:30.117184 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lsh/tcvagrkxswftkyqbvyegnovbgfixe. 2024.10.22 01:58:30.177793 [ 1419 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:58:30.314799 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(59388841bc6ec7e7dcb99211f7640280) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 59388841bc6ec7e7dcb99211f7640280, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:58:30.316534 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 59388841bc6ec7e7dcb99211f7640280, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:58:30.323567 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:30.454501 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 9.44 MiB, will set to 1.55 GiB (RSS), difference: 193.81 MiB 2024.10.22 01:58:30.636740 [ 1252 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:58:30.678551 [ 1252 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:58:31.015028 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:58:31.024000 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:58:31.056953 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 81596 2024.10.22 01:58:31.234435 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:31.239415 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:31.389676 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1233_1233_0 (state Temporary) 2024.10.22 01:58:31.549477 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1233_1233_0 to 202410_1246_1246_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:31.707742 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81596 2024.10.22 01:58:32.084256 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(4c98f9dd3a1f386a96c1c79df6c12933) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4c98f9dd3a1f386a96c1c79df6c12933, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:58:32.088345 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 4c98f9dd3a1f386a96c1c79df6c12933, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 01:58:32.092210 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:32.302944 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 885.84 MiB, peak 2.00 GiB, free memory in arenas 12.06 MiB, will set to 1.07 GiB (RSS), difference: 211.97 MiB 2024.10.22 01:58:33.520929 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:33.525484 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:34.504774 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.08 GiB, peak 2.00 GiB, free memory in arenas 10.73 MiB, will set to 1.09 GiB (RSS), difference: 15.88 MiB 2024.10.22 01:58:34.567168 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 43 2024.10.22 01:58:34.676257 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 89 entries to flush up to offset 81085 2024.10.22 01:58:34.770562 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(08078fa2aa4616a3b72783d7b5503da3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 08078fa2aa4616a3b72783d7b5503da3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:58:34.796888 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 980, file segment info: File segment: [0, 33554431], key: 08078fa2aa4616a3b72783d7b5503da3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:58:34.800142 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:34.955958 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:34.957830 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:35.001227 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_719_719_0 (state Temporary) 2024.10.22 01:58:35.066511 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:35.067161 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_700_134} MergeTask::MergeProjectionsStage: Merge sorted 1068972 rows, containing 4 columns (4 merged, 0 gathered) in 32.043899374 sec., 33359.61043702908 rows/sec., 500.48 KiB/sec. 2024.10.22 01:58:35.068767 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:35.157954 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_719_719_0 to 202410_725_725_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:35.165152 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_37_37_0 (state Temporary) 2024.10.22 01:58:35.211161 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_720_66 to 202410_721_721_0 2024.10.22 01:58:35.236050 [ 1150 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.27 MiB using storage policy from min volume index 0 2024.10.22 01:58:35.237020 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 81085 2024.10.22 01:58:35.240661 [ 1150 ] {} DiskLocal: Reserved 1.27 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:35.255062 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_37_37_0 to 202410_37_37_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:35.333982 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_700_134} MergedBlockOutputStream: filled checksums 202410_1_700_134 (state Temporary) 2024.10.22 01:58:35.358564 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_721_67} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_720_66 to 202410_721_721_0 into Compact with storage Full 2024.10.22 01:58:35.520242 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 43 2024.10.22 01:58:35.535511 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_721_67} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:35.725067 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 81631 2024.10.22 01:58:35.734663 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(cc7f39cae3274865d6989ec69962526c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cc7f39cae3274865d6989ec69962526c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:58:35.742636 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 824, file segment info: File segment: [0, 33554431], key: cc7f39cae3274865d6989ec69962526c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 01:58:35.746857 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:35.754313 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_721_67} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_720_66, total 21473 rows starting from the beginning of the part 2024.10.22 01:58:35.802769 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_548_112, 202410_549_549_0, 202410_550_550_0, 202410_551_551_0, 202410_552_552_0, 202410_553_553_0] 2024.10.22 01:58:35.804082 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_548_112, 202410_549_549_0, 202410_550_550_0, 202410_551_551_0, 202410_552_552_0, 202410_553_553_0] 2024.10.22 01:58:35.812674 [ 1747 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uxi/axmmvnesbxmkevqjfiewmxykpmtso, size 1148739 2024.10.22 01:58:35.852005 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_721_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_721_721_0, total 94 rows starting from the beginning of the part 2024.10.22 01:58:35.937733 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_548_112, 202410_549_549_0, 202410_550_550_0, 202410_551_551_0, 202410_552_552_0, 202410_553_553_0] 2024.10.22 01:58:36.035944 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:36.044115 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:36.081720 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 11135 2024.10.22 01:58:36.087871 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1065385 2024.10.22 01:58:36.090278 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1234_1234_0 (state Temporary) 2024.10.22 01:58:36.111954 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:58:36.133138 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:58:36.155644 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1234_1234_0 to 202410_1247_1247_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:36.208969 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1242_7 to 202410_1247_1247_0 2024.10.22 01:58:36.246611 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81631 2024.10.22 01:58:36.252979 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:36.254852 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:36.256432 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:58:36.258430 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:36.271147 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:58:36.276836 [ 1195 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:36.341245 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:58:36.342426 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_732_732_0 (state Temporary) 2024.10.22 01:58:36.401265 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_732_732_0 to 202410_745_745_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:36.348192 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1242_7 to 202410_1247_1247_0 into Compact with storage Full 2024.10.22 01:58:36.478756 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:58:36.494039 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:36.516239 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1242_7, total 2167 rows starting from the beginning of the part 2024.10.22 01:58:36.519887 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column URLRegions 2024.10.22 01:58:36.548344 [ 1206 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 4 parts from 202410_1_740_153 to 202410_743_743_0 2024.10.22 01:58:36.554614 [ 1181 ] {} DatabaseCatalog: Execution took 358 ms. 2024.10.22 01:58:36.564133 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1243_1243_0, total 78 rows starting from the beginning of the part 2024.10.22 01:58:36.590253 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11135 2024.10.22 01:58:36.598807 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_147_147_0 (state Temporary) 2024.10.22 01:58:36.601079 [ 1206 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:36.607464 [ 1206 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:36.603128 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1244_1244_0, total 86 rows starting from the beginning of the part 2024.10.22 01:58:36.631490 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_743_154} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_740_153 to 202410_743_743_0 into Compact with storage Full 2024.10.22 01:58:36.649147 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1245_1245_0, total 55 rows starting from the beginning of the part 2024.10.22 01:58:36.679705 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1246_1246_0, total 69 rows starting from the beginning of the part 2024.10.22 01:58:36.717220 [ 1206 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 202 ms. 2024.10.22 01:58:36.740274 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1247_1247_0, total 35 rows starting from the beginning of the part 2024.10.22 01:58:36.742829 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_743_154} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:36.772230 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_743_154} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_740_153, total 11171 rows starting from the beginning of the part 2024.10.22 01:58:36.780436 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.22 GiB, peak 2.00 GiB, free memory in arenas 11.70 MiB, will set to 1.11 GiB (RSS), difference: -113.06 MiB 2024.10.22 01:58:36.852293 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/srm/bplgmyopewojsefljypgckywjamxm (LogSeriesLimiter: on interval from 2024-10-22 01:58:25 to 2024-10-22 01:58:36 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:58:36.867802 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_743_154} MergeTreeSequentialSource: Reading 2 marks from part 202410_741_741_0, total 16 rows starting from the beginning of the part 2024.10.22 01:58:36.918388 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:36.923923 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:36.996087 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_743_154} MergeTreeSequentialSource: Reading 2 marks from part 202410_742_742_0, total 9 rows starting from the beginning of the part 2024.10.22 01:58:37.094176 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_743_154} MergeTreeSequentialSource: Reading 2 marks from part 202410_743_743_0, total 20 rows starting from the beginning of the part 2024.10.22 01:58:37.098417 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_701_701_0 (state Temporary) 2024.10.22 01:58:37.160897 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_701_701_0 to 202410_708_708_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:37.208061 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1065385 2024.10.22 01:58:37.577680 [ 1179 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 351 ms. 2024.10.22 01:58:37.707772 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_721_67} MergeTask::MergeProjectionsStage: Merge sorted 21567 rows, containing 13 columns (13 merged, 0 gathered) in 2.345017316 sec., 9196.947013076982 rows/sec., 3.98 MiB/sec. 2024.10.22 01:58:37.796789 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_721_67/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:58:37.882257 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_721_67} MergedBlockOutputStream: filled checksums 202410_507_721_67 (state Temporary) 2024.10.22 01:58:37.984722 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_721_67} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_721_67 to 202410_507_721_67 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:38.010989 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_721_67} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_720_66, 202410_721_721_0] -> 202410_507_721_67 2024.10.22 01:58:38.033216 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 610 2024.10.22 01:58:38.049665 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1247_8/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:58:38.056122 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_147_147_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:58:38.102956 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.22 MiB. 2024.10.22 01:58:38.118641 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_743_154} MergeTask::MergeProjectionsStage: Merge sorted 11216 rows, containing 12 columns (12 merged, 0 gathered) in 1.501214349 sec., 7471.284835154476 rows/sec., 1.16 MiB/sec. 2024.10.22 01:58:38.198366 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_743_154} MergedBlockOutputStream: filled checksums 202410_1_743_154 (state Temporary) 2024.10.22 01:58:38.280687 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_743_154} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_743_154 to 202410_1_743_154 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:38.284353 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_743_154} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 4 parts: [202410_1_740_153, 202410_743_743_0] -> 202410_1_743_154 2024.10.22 01:58:38.337123 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.10 GiB, peak 2.00 GiB, free memory in arenas 8.46 MiB, will set to 1.15 GiB (RSS), difference: 47.21 MiB 2024.10.22 01:58:38.359443 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} MergeTask::MergeProjectionsStage: Merge sorted 2490 rows, containing 13 columns (13 merged, 0 gathered) in 2.043669579 sec., 1218.3965674228007 rows/sec., 205.89 KiB/sec. 2024.10.22 01:58:38.420105 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.49 MiB. 2024.10.22 01:58:38.488727 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:38.492475 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:38.559565 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} MergedBlockOutputStream: filled checksums 202410_1207_1247_8 (state Temporary) 2024.10.22 01:58:38.676719 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_592_592_0 (state Temporary) 2024.10.22 01:58:38.771599 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_700_134/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:58:38.831670 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1247_8 to 202410_1207_1247_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:38.837627 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1247_8} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1242_7, 202410_1247_1247_0] -> 202410_1207_1247_8 2024.10.22 01:58:38.866701 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:58:39.057168 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_700_134} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_700_134 to 202410_1_700_134 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:39.061047 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_700_134} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_694_133, 202410_695_700_1] -> 202410_1_700_134 2024.10.22 01:58:39.075716 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 01:58:39.094471 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_592_592_0 to 202410_602_602_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:39.187542 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 610 2024.10.22 01:58:39.340573 [ 1163 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 01:58:39.475676 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_145_149_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:39.478974 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_662_32, 202410_663_663_0, 202410_664_664_0, 202410_665_665_0, 202410_666_666_0, 202410_667_667_0] 2024.10.22 01:58:39.504238 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_662_32, 202410_663_663_0, 202410_664_664_0, 202410_665_665_0, 202410_666_666_0, 202410_667_667_0] 2024.10.22 01:58:39.656832 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_662_32, 202410_663_663_0, 202410_664_664_0, 202410_665_665_0, 202410_666_666_0, 202410_667_667_0] 2024.10.22 01:58:40.254936 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 94 entries to flush up to offset 81725 2024.10.22 01:58:40.399234 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.21 GiB, peak 2.00 GiB, free memory in arenas 4.08 MiB, will set to 1.18 GiB (RSS), difference: -28.17 MiB 2024.10.22 01:58:40.838855 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:40.846439 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:40.946428 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1235_1235_0 (state Temporary) 2024.10.22 01:58:40.984934 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1235_1235_0 to 202410_1248_1248_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:41.015842 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:58:41.024716 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:58:41.045181 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_145_149_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_145_145_0, total 43 rows starting from the beginning of the part 2024.10.22 01:58:41.147365 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81725 2024.10.22 01:58:41.639272 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_647_647_0, 202410_648_648_0, 202410_649_649_0, 202410_650_650_0, 202410_651_651_0, 202410_652_652_0] 2024.10.22 01:58:41.641171 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_647_647_0, 202410_648_648_0, 202410_649_649_0, 202410_650_650_0, 202410_651_651_0, 202410_652_652_0] 2024.10.22 01:58:41.674996 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_647_647_0, 202410_648_648_0, 202410_649_649_0, 202410_650_650_0, 202410_651_651_0, 202410_652_652_0] 2024.10.22 01:58:41.932754 [ 1234 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 251 ms. 2024.10.22 01:58:42.543955 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.28 GiB, peak 2.00 GiB, free memory in arenas 3.12 MiB, will set to 1.22 GiB (RSS), difference: -58.88 MiB 2024.10.22 01:58:42.754323 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 119 entries to flush up to offset 81204 2024.10.22 01:58:42.896132 [ 1208 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 222 ms. 2024.10.22 01:58:42.931372 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.16236481 sec. 2024.10.22 01:58:43.081114 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:43.087576 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:43.156593 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_720_720_0 (state Temporary) 2024.10.22 01:58:43.250533 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_720_720_0 to 202410_726_726_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:43.311887 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 81204 2024.10.22 01:58:43.316796 [ 1269 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_721_67 to 202410_722_722_0 2024.10.22 01:58:43.335200 [ 1269 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.27 MiB using storage policy from min volume index 0 2024.10.22 01:58:43.341823 [ 1269 ] {} DiskLocal: Reserved 1.27 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:43.370791 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_722_68} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_721_67 to 202410_722_722_0 into Compact with storage Full 2024.10.22 01:58:43.417062 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_722_68} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:43.433673 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_722_68} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_721_67, total 21567 rows starting from the beginning of the part 2024.10.22 01:58:43.460713 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_722_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_722_722_0, total 101 rows starting from the beginning of the part 2024.10.22 01:58:43.723309 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_147_147_0 to 202410_150_150_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:44.088163 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_145_149_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_146_146_0, total 44 rows starting from the beginning of the part 2024.10.22 01:58:44.101041 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 11160 2024.10.22 01:58:44.224297 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1066925 2024.10.22 01:58:44.363956 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:44.366665 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:44.417703 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_733_733_0 (state Temporary) 2024.10.22 01:58:44.500857 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 3.00 MiB, will set to 1.28 GiB (RSS), difference: -70.83 MiB 2024.10.22 01:58:44.667574 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_733_733_0 to 202410_746_746_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:44.924588 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11160 2024.10.22 01:58:44.995827 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_722_68} MergeTask::MergeProjectionsStage: Merge sorted 21668 rows, containing 13 columns (13 merged, 0 gathered) in 1.629862478 sec., 13294.37317103511 rows/sec., 5.76 MiB/sec. 2024.10.22 01:58:45.144115 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:45.147966 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:45.167034 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_722_68} MergedBlockOutputStream: filled checksums 202410_507_722_68 (state Temporary) 2024.10.22 01:58:45.280826 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_702_702_0 (state Temporary) 2024.10.22 01:58:45.315618 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 81769 2024.10.22 01:58:45.320867 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 01:58:45.392607 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_702_702_0 to 202410_709_709_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:45.538010 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1066925 2024.10.22 01:58:45.553557 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_722_68} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_722_68 to 202410_507_722_68 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:45.556522 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_722_68} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_721_67, 202410_722_722_0] -> 202410_507_722_68 2024.10.22 01:58:45.567357 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:45.570692 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:45.582172 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.25 MiB. 2024.10.22 01:58:45.636624 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1236_1236_0 (state Temporary) 2024.10.22 01:58:45.674880 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1236_1236_0 to 202410_1249_1249_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:45.685390 [ 1215 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:58:45.745265 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81769 2024.10.22 01:58:45.765168 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_145_149_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_147_147_0, total 43 rows starting from the beginning of the part 2024.10.22 01:58:45.839931 [ 1215 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:58:46.359920 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5917 2024.10.22 01:58:46.554590 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 3.92 MiB, will set to 1.33 GiB (RSS), difference: -12.01 MiB 2024.10.22 01:58:46.691077 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 611 2024.10.22 01:58:46.780882 [ 1266 ] {} DatabaseCatalog: Execution took 228 ms. 2024.10.22 01:58:46.904654 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_682_137, 202410_683_683_0, 202410_684_684_0, 202410_685_685_0, 202410_686_686_0, 202410_687_687_0] 2024.10.22 01:58:46.917002 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:46.925611 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:47.007903 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_682_137, 202410_683_683_0, 202410_684_684_0, 202410_685_685_0, 202410_686_686_0, 202410_687_687_0] 2024.10.22 01:58:47.034321 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_682_137, 202410_683_683_0, 202410_684_684_0, 202410_685_685_0, 202410_686_686_0, 202410_687_687_0] 2024.10.22 01:58:47.219077 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 262.61 MiB using storage policy from min volume index 0 2024.10.22 01:58:47.276677 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_593_593_0 (state Temporary) 2024.10.22 01:58:47.475526 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_593_593_0 to 202410_603_603_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:47.709598 [ 1257 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_598_122 to 202410_603_603_0 2024.10.22 01:58:47.760230 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_145_149_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_148_148_0, total 43 rows starting from the beginning of the part 2024.10.22 01:58:47.831534 [ 1257 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:47.836997 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:47.842196 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 611 2024.10.22 01:58:47.860740 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_598_122 to 202410_603_603_0 into Compact with storage Full 2024.10.22 01:58:48.084645 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:48.102942 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_598_122, total 642 rows starting from the beginning of the part 2024.10.22 01:58:48.273100 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_599_599_0, total 1 rows starting from the beginning of the part 2024.10.22 01:58:48.301362 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 4.97 MiB, will set to 1.32 GiB (RSS), difference: -11.63 MiB 2024.10.22 01:58:48.351092 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_600_600_0, total 1 rows starting from the beginning of the part 2024.10.22 01:58:48.387435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mmo/vkxvzsmlryvinqeeorwqvyfftfgkw (LogSeriesLimiter: on interval from 2024-10-22 01:58:36 to 2024-10-22 01:58:48 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:58:48.553481 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1146_10, 202410_1147_1147_0, 202410_1148_1148_0, 202410_1149_1149_0, 202410_1150_1150_0, 202410_1151_1151_0] 2024.10.22 01:58:48.555260 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1146_10, 202410_1147_1147_0, 202410_1148_1148_0, 202410_1149_1149_0, 202410_1150_1150_0, 202410_1151_1151_0] 2024.10.22 01:58:48.583212 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1146_10, 202410_1147_1147_0, 202410_1148_1148_0, 202410_1149_1149_0, 202410_1150_1150_0, 202410_1151_1151_0] 2024.10.22 01:58:48.841446 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_601_601_0, total 1 rows starting from the beginning of the part 2024.10.22 01:58:48.900560 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_602_602_0, total 1 rows starting from the beginning of the part 2024.10.22 01:58:48.915358 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(911c3483032fd75f5c7c25209c5323b8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 911c3483032fd75f5c7c25209c5323b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:58:48.920904 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 764, file segment info: File segment: [0, 33554431], key: 911c3483032fd75f5c7c25209c5323b8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:58:48.927114 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:48.972187 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_603_603_0, total 1 rows starting from the beginning of the part 2024.10.22 01:58:49.511180 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_145_149_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_149_149_0, total 40 rows starting from the beginning of the part 2024.10.22 01:58:49.702768 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 544 bytes for b968d623b345ba3595fd035d340c01ec:0) 2024.10.22 01:58:49.709080 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 439a625e53519d47fda4321d7c36a8c8, offset: 0, size: 152 2024.10.22 01:58:49.717129 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 439a625e53519d47fda4321d7c36a8c8 for removal 2024.10.22 01:58:49.721389 [ 1476 ] {} CacheMetadata: Key 439a625e53519d47fda4321d7c36a8c8 is removed from metadata 2024.10.22 01:58:49.721730 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: bfc49cae30ae8d384d3dad4477ad0b87, offset: 0, size: 196 2024.10.22 01:58:49.727806 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key bfc49cae30ae8d384d3dad4477ad0b87 for removal 2024.10.22 01:58:49.739564 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4358eacfa1bdf0622c3d7f2a09d17c47, offset: 0, size: 188 2024.10.22 01:58:49.746333 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4358eacfa1bdf0622c3d7f2a09d17c47 for removal 2024.10.22 01:58:49.749217 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 49cda6f7d508d661c090cd0eccb0d2bd, offset: 0, size: 4 2024.10.22 01:58:49.749485 [ 1476 ] {} CacheMetadata: Key 4358eacfa1bdf0622c3d7f2a09d17c47 is removed from metadata 2024.10.22 01:58:49.750520 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 81827 2024.10.22 01:58:49.752958 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 49cda6f7d508d661c090cd0eccb0d2bd for removal 2024.10.22 01:58:49.756748 [ 1476 ] {} CacheMetadata: Key 49cda6f7d508d661c090cd0eccb0d2bd is removed from metadata 2024.10.22 01:58:49.765609 [ 1476 ] {} CacheMetadata: Key bfc49cae30ae8d384d3dad4477ad0b87 is removed from metadata 2024.10.22 01:58:49.960716 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:49.964200 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:50.292051 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_603_123/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:58:50.345778 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1237_1237_0 (state Temporary) 2024.10.22 01:58:50.355313 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(fd3266c1aeb09f2411a16e3692a3c00e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fd3266c1aeb09f2411a16e3692a3c00e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:58:50.359734 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 5959, file segment info: File segment: [0, 33554431], key: fd3266c1aeb09f2411a16e3692a3c00e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:58:50.362282 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:50.492302 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1237_1237_0 to 202410_1250_1250_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:50.544852 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81827 2024.10.22 01:58:50.570245 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 4.55 MiB, will set to 1.32 GiB (RSS), difference: -182.39 MiB 2024.10.22 01:58:50.831887 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 95 entries to flush up to offset 81299 2024.10.22 01:58:50.960172 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:50.962207 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:51.024272 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:58:51.027154 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:58:51.047705 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(9e33abb83d2454027c46cc358f792683) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9e33abb83d2454027c46cc358f792683, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:58:51.049720 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 477, file segment info: File segment: [0, 33554431], key: 9e33abb83d2454027c46cc358f792683, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 01:58:51.052191 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:58:51.105132 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_721_721_0 (state Temporary) 2024.10.22 01:58:51.146588 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_721_721_0 to 202410_727_727_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:51.171666 [ 1152 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_722_68 to 202410_723_723_0 2024.10.22 01:58:51.188181 [ 1721 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/agd/zvonotuuivvdipouvoqunrxokewxw, size 5959 2024.10.22 01:58:51.199284 [ 1152 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.28 MiB using storage policy from min volume index 0 2024.10.22 01:58:51.202398 [ 1152 ] {} DiskLocal: Reserved 1.28 MiB on local disk `default`, having unreserved 296.42 GiB. 2024.10.22 01:58:51.236857 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_723_69} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_722_68 to 202410_723_723_0 into Compact with storage Full 2024.10.22 01:58:51.266223 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_723_69} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:51.274884 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_723_69} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_722_68, total 21668 rows starting from the beginning of the part 2024.10.22 01:58:51.293573 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 81299 2024.10.22 01:58:51.315926 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_723_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_723_723_0, total 103 rows starting from the beginning of the part 2024.10.22 01:58:51.433260 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} MergeTask::MergeProjectionsStage: Merge sorted 647 rows, containing 32 columns (32 merged, 0 gathered) in 3.565030895 sec., 181.4851032307814 rows/sec., 16.51 KiB/sec. 2024.10.22 01:58:51.560853 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:58:51.571117 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} MergedBlockOutputStream: filled checksums 202410_1_603_123 (state Temporary) 2024.10.22 01:58:51.585426 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:58:51.643549 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:58:51.687397 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:58:51.840199 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:58:51.976080 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:58:52.051674 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_603_123 to 202410_1_603_123 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:52.058650 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_603_123} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_598_122, 202410_603_603_0] -> 202410_1_603_123 2024.10.22 01:58:52.091801 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:58:52.109291 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:58:52.143474 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 01:58:52.427825 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 11176 2024.10.22 01:58:52.545184 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1068465 2024.10.22 01:58:52.557518 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_723_69} MergeTask::MergeProjectionsStage: Merge sorted 21771 rows, containing 13 columns (13 merged, 0 gathered) in 1.337762992 sec., 16274.183192533703 rows/sec., 7.05 MiB/sec. 2024.10.22 01:58:52.615070 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:52.622974 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:52.637550 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 4.71 MiB, will set to 1.33 GiB (RSS), difference: -170.45 MiB 2024.10.22 01:58:52.666162 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_734_734_0 (state Temporary) 2024.10.22 01:58:52.716353 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_734_734_0 to 202410_747_747_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:52.738938 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_723_69} MergedBlockOutputStream: filled checksums 202410_507_723_69 (state Temporary) 2024.10.22 01:58:52.871130 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11176 2024.10.22 01:58:53.052043 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qpb/hnllljucvowdnddplkbujffkxkgvi (LogSeriesLimiter: on interval from 2024-10-22 01:58:48 to 2024-10-22 01:58:53 accepted series 1 / 273 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:58:53.439372 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:53.443140 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:53.423088 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_723_69} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_723_69 to 202410_507_723_69 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:53.450959 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_723_69} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_722_68, 202410_723_723_0] -> 202410_507_723_69 2024.10.22 01:58:53.573204 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.25 MiB. 2024.10.22 01:58:53.581626 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_703_703_0 (state Temporary) 2024.10.22 01:58:53.830449 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_703_703_0 to 202410_710_710_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:53.862196 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_700_134 to 202410_701_706_1 2024.10.22 01:58:53.868826 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1068465 2024.10.22 01:58:53.870889 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:53.873732 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:53.874247 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 35 entries to flush up to offset 5952 2024.10.22 01:58:53.953787 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_706_135} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_700_134 to 202410_701_706_1 into Wide with storage Full 2024.10.22 01:58:53.965307 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_706_135} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:53.968435 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_706_135} MergeTreeSequentialSource: Reading 132 marks from part 202410_1_700_134, total 1068972 rows starting from the beginning of the part 2024.10.22 01:58:54.010749 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_706_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_701_706_1, total 6923 rows starting from the beginning of the part 2024.10.22 01:58:54.439273 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 5.82 MiB, will set to 1.32 GiB (RSS), difference: -194.29 MiB 2024.10.22 01:58:54.552313 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 81903 2024.10.22 01:58:54.818572 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:54.828160 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:54.990790 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1238_1238_0 (state Temporary) 2024.10.22 01:58:55.144556 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1238_1238_0 to 202410_1251_1251_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:55.345607 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 612 2024.10.22 01:58:55.535090 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81903 2024.10.22 01:58:55.581393 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_145_149_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:58:55.816243 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:55.921502 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:56.112181 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_594_594_0 (state Temporary) 2024.10.22 01:58:56.392008 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_594_594_0 to 202410_604_604_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:56.463202 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 612 2024.10.22 01:58:56.875862 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 8.91 MiB, will set to 1.33 GiB (RSS), difference: -14.41 MiB 2024.10.22 01:58:57.008681 [ 1164 ] {} DatabaseCatalog: Execution took 225 ms. 2024.10.22 01:58:57.316775 [ 1425 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Found 4 old parts to remove. Parts: [202410_1_6_1, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0] 2024.10.22 01:58:57.318737 [ 1425 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Removing 4 parts from filesystem (serially): Parts: [202410_1_6_1, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0] 2024.10.22 01:58:57.444727 [ 1425 ] {} system.query_log (3c89ec5e-dff4-4fa3-aa5c-2c2b424d63ef): Removing 4 parts from memory: Parts: [202410_1_6_1, 202410_7_7_0, 202410_8_8_0, 202410_9_9_0] 2024.10.22 01:58:57.986948 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_640_124, 202410_641_646_1] 2024.10.22 01:58:57.991917 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_640_124, 202410_641_646_1] 2024.10.22 01:58:58.025780 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_640_124, 202410_641_646_1] 2024.10.22 01:58:58.535381 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 10.25 MiB, will set to 1.35 GiB (RSS), difference: 4.23 MiB 2024.10.22 01:58:59.034347 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 127 entries to flush up to offset 81426 2024.10.22 01:58:59.316386 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:59.320751 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:59.367555 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_722_722_0 (state Temporary) 2024.10.22 01:58:59.450580 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_722_722_0 to 202410_728_728_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:58:59.482614 [ 1184 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_723_69 to 202410_724_724_0 2024.10.22 01:58:59.496367 [ 1184 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.28 MiB using storage policy from min volume index 0 2024.10.22 01:58:59.500285 [ 1184 ] {} DiskLocal: Reserved 1.28 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:58:59.523554 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 81426 2024.10.22 01:58:59.542789 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_724_70} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_723_69 to 202410_724_724_0 into Compact with storage Full 2024.10.22 01:58:59.549421 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 81934 2024.10.22 01:58:59.612575 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_724_70} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:58:59.625259 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_724_70} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_723_69, total 21771 rows starting from the beginning of the part 2024.10.22 01:58:59.661912 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_724_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_724_724_0, total 100 rows starting from the beginning of the part 2024.10.22 01:58:59.852126 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:58:59.866243 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:00.044851 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1239_1239_0 (state Temporary) 2024.10.22 01:59:00.147431 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1239_1239_0 to 202410_1252_1252_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:00.208678 [ 1212 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1247_8 to 202410_1252_1252_0 2024.10.22 01:59:00.230680 [ 1212 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:00.257382 [ 1212 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:00.332489 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1247_8 to 202410_1252_1252_0 into Compact with storage Full 2024.10.22 01:59:00.363839 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:00.372689 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 11185 2024.10.22 01:59:00.378499 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 81934 2024.10.22 01:59:00.517713 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1247_8, total 2490 rows starting from the beginning of the part 2024.10.22 01:59:00.544213 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1248_1248_0, total 94 rows starting from the beginning of the part 2024.10.22 01:59:00.550016 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:00.555229 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:00.563611 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_706_135/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:59:00.576585 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1249_1249_0, total 44 rows starting from the beginning of the part 2024.10.22 01:59:00.613705 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1250_1250_0, total 58 rows starting from the beginning of the part 2024.10.22 01:59:00.643845 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1251_1251_0, total 76 rows starting from the beginning of the part 2024.10.22 01:59:00.676465 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1252_1252_0, total 31 rows starting from the beginning of the part 2024.10.22 01:59:00.688745 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 2.00 GiB, free memory in arenas 12.70 MiB, will set to 1.36 GiB (RSS), difference: -6.71 MiB 2024.10.22 01:59:00.799393 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(6bbadeb43959403c6d1895d53c0cb1ff) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6bbadeb43959403c6d1895d53c0cb1ff, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:59:00.805468 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_735_735_0 (state Temporary) 2024.10.22 01:59:00.804685 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 764813, file segment info: File segment: [0, 33554431], key: 6bbadeb43959403c6d1895d53c0cb1ff, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:59:00.813623 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:00.851050 [ 1197 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:59:00.873116 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1070005 2024.10.22 01:59:00.878945 [ 1197 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:59:01.017472 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:59:01.020966 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:59:01.055022 [ 1168 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 01:59:01.222412 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_735_735_0 to 202410_748_748_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:01.262181 [ 1214 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 4 parts from 202410_1_743_154 to 202410_746_746_0 2024.10.22 01:59:01.267302 [ 1214 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:01.270039 [ 1214 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:01.278976 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11185 2024.10.22 01:59:01.285424 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_746_155} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_743_154 to 202410_746_746_0 into Compact with storage Full 2024.10.22 01:59:01.353610 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_746_155} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:01.363200 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_746_155} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_743_154, total 11216 rows starting from the beginning of the part 2024.10.22 01:59:01.498974 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_746_155} MergeTreeSequentialSource: Reading 2 marks from part 202410_744_744_0, total 16 rows starting from the beginning of the part 2024.10.22 01:59:01.662507 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_746_155} MergeTreeSequentialSource: Reading 2 marks from part 202410_745_745_0, total 9 rows starting from the beginning of the part 2024.10.22 01:59:01.693296 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:01.699287 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:01.715209 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_746_155} MergeTreeSequentialSource: Reading 2 marks from part 202410_746_746_0, total 25 rows starting from the beginning of the part 2024.10.22 01:59:01.796582 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_724_70} MergeTask::MergeProjectionsStage: Merge sorted 21871 rows, containing 13 columns (13 merged, 0 gathered) in 2.260998902 sec., 9673.15816945054 rows/sec., 4.19 MiB/sec. 2024.10.22 01:59:01.847030 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_704_704_0 (state Temporary) 2024.10.22 01:59:01.975353 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_704_704_0 to 202410_711_711_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:02.011449 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(51ac5f2ffde20036c410027069415b07) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 51ac5f2ffde20036c410027069415b07, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:59:02.039348 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1070005 2024.10.22 01:59:02.073282 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_724_70} MergedBlockOutputStream: filled checksums 202410_507_724_70 (state Temporary) 2024.10.22 01:59:02.081663 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} MergeTask::MergeProjectionsStage: Merge sorted 2793 rows, containing 13 columns (13 merged, 0 gathered) in 1.784311245 sec., 1565.3098683464275 rows/sec., 264.12 KiB/sec. 2024.10.22 01:59:02.116134 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1086, file segment info: File segment: [0, 33554431], key: 51ac5f2ffde20036c410027069415b07, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:59:02.125568 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:02.147236 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} MergedBlockOutputStream: filled checksums 202410_1207_1252_9 (state Temporary) 2024.10.22 01:59:02.310661 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1252_9 to 202410_1207_1252_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:02.321534 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1252_9} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1247_8, 202410_1252_1252_0] -> 202410_1207_1252_9 2024.10.22 01:59:02.363210 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:59:02.575106 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_724_70} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_724_70 to 202410_507_724_70 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:02.580092 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_724_70} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_723_69, 202410_724_724_0] -> 202410_507_724_70 2024.10.22 01:59:02.616702 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.50 MiB. 2024.10.22 01:59:03.066210 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(0c71c4276a7e701d0efc9df6a75a3430) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0c71c4276a7e701d0efc9df6a75a3430, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:59:03.083043 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 907024, file segment info: File segment: [0, 33554431], key: 0c71c4276a7e701d0efc9df6a75a3430, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:59:03.086073 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:03.267203 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_746_155} MergeTask::MergeProjectionsStage: Merge sorted 11266 rows, containing 12 columns (12 merged, 0 gathered) in 1.991155089 sec., 5658.022352070035 rows/sec., 901.63 KiB/sec. 2024.10.22 01:59:03.303072 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_746_155} MergedBlockOutputStream: filled checksums 202410_1_746_155 (state Temporary) 2024.10.22 01:59:03.456266 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_746_155} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_746_155 to 202410_1_746_155 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:03.458851 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_746_155} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 4 parts: [202410_1_743_154, 202410_746_746_0] -> 202410_1_746_155 2024.10.22 01:59:03.676352 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 12.55 MiB, will set to 1.37 GiB (RSS), difference: 23.68 MiB 2024.10.22 01:59:03.788330 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.50 MiB. 2024.10.22 01:59:03.965803 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 613 2024.10.22 01:59:04.246983 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:04.249732 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:04.253160 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(c26e8ed5240c564088d6fc5ca047b707) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c26e8ed5240c564088d6fc5ca047b707, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:59:04.256883 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 815, file segment info: File segment: [0, 33554431], key: c26e8ed5240c564088d6fc5ca047b707, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 01:59:04.260410 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:04.317151 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_595_595_0 (state Temporary) 2024.10.22 01:59:04.389026 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 82015 2024.10.22 01:59:04.422681 [ 1686 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/srm/bplgmyopewojsefljypgckywjamxm, size 907024 2024.10.22 01:59:04.515855 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_595_595_0 to 202410_605_605_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:04.618368 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 613 2024.10.22 01:59:04.736127 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:04.739046 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:04.824848 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:59:04.858629 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:59:04.874705 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:59:04.909040 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:59:04.935101 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:59:04.936501 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1240_1240_0 (state Temporary) 2024.10.22 01:59:04.962335 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:59:04.997777 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1240_1240_0 to 202410_1253_1253_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:05.030392 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RefererRegions 2024.10.22 01:59:05.196286 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82015 2024.10.22 01:59:05.252101 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_145_149_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 01:59:05.289531 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vtb/ctligvdpoicwsncxrxywltuexikpv (LogSeriesLimiter: on interval from 2024-10-22 01:58:53 to 2024-10-22 01:59:05 accepted series 1 / 86 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:59:05.463685 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 2.00 GiB, free memory in arenas 11.09 MiB, will set to 1.38 GiB (RSS), difference: 5.68 MiB 2024.10.22 01:59:07.151058 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 119 entries to flush up to offset 81545 2024.10.22 01:59:07.345445 [ 1260 ] {} DatabaseCatalog: Execution took 333 ms. 2024.10.22 01:59:07.516884 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:07.521421 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 01:59:07.525307 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:07.552917 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 2.00 GiB, free memory in arenas 10.04 MiB, will set to 1.38 GiB (RSS), difference: -3.27 MiB 2024.10.22 01:59:07.675792 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_723_723_0 (state Temporary) 2024.10.22 01:59:07.733774 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_723_723_0 to 202410_729_729_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:07.755758 [ 1242 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_724_70 to 202410_725_725_0 2024.10.22 01:59:07.763638 [ 1242 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.29 MiB using storage policy from min volume index 0 2024.10.22 01:59:07.772787 [ 1242 ] {} DiskLocal: Reserved 1.29 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:07.774088 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 81545 2024.10.22 01:59:07.803110 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_725_71} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_724_70 to 202410_725_725_0 into Compact with storage Full 2024.10.22 01:59:08.155521 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_725_71} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:08.261788 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_725_71} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_724_70, total 21871 rows starting from the beginning of the part 2024.10.22 01:59:08.270424 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_725_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_725_725_0, total 89 rows starting from the beginning of the part 2024.10.22 01:59:08.782700 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 11205 2024.10.22 01:59:08.904017 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:08.908597 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:09.044654 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1071545 2024.10.22 01:59:09.056607 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_736_736_0 (state Temporary) 2024.10.22 01:59:09.138461 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_736_736_0 to 202410_749_749_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:09.207288 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 82056 2024.10.22 01:59:09.336571 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11205 2024.10.22 01:59:09.455062 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_725_71} MergeTask::MergeProjectionsStage: Merge sorted 21960 rows, containing 13 columns (13 merged, 0 gathered) in 1.671270382 sec., 13139.705122830328 rows/sec., 5.69 MiB/sec. 2024.10.22 01:59:09.544574 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 2.00 GiB, free memory in arenas 13.27 MiB, will set to 1.40 GiB (RSS), difference: 8.50 MiB 2024.10.22 01:59:09.747388 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_725_71} MergedBlockOutputStream: filled checksums 202410_507_725_71 (state Temporary) 2024.10.22 01:59:09.789167 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:09.793580 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:09.827033 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:09.832796 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:09.842465 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1241_1241_0 (state Temporary) 2024.10.22 01:59:09.987385 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1241_1241_0 to 202410_1254_1254_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:10.055641 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82056 2024.10.22 01:59:10.173696 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_705_705_0 (state Temporary) 2024.10.22 01:59:10.334538 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_725_71} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_725_71 to 202410_507_725_71 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:10.342278 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_725_71} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_724_70, 202410_725_725_0] -> 202410_507_725_71 2024.10.22 01:59:10.374888 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.47 MiB. 2024.10.22 01:59:10.601062 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_705_705_0 to 202410_712_712_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:10.666009 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1071545 2024.10.22 01:59:10.650823 [ 1177 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_707_707_0 to 202410_712_712_0 2024.10.22 01:59:10.716370 [ 1177 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:10.727088 [ 1177 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 01:59:10.760413 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_707_707_0 to 202410_712_712_0 into Compact with storage Full 2024.10.22 01:59:10.794549 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_706_135/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:59:10.810349 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:10.825975 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_707_707_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:59:10.899864 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_708_708_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:59:10.941189 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_709_709_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:59:11.023009 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:59:11.029705 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:59:10.957828 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_710_710_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:59:11.191284 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_711_711_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:59:11.226569 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_712_712_0, total 1540 rows starting from the beginning of the part 2024.10.22 01:59:11.590866 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 13.54 MiB, will set to 1.39 GiB (RSS), difference: 75.00 MiB 2024.10.22 01:59:11.984644 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(f217dd9667706836223d543a5ab5e1e3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f217dd9667706836223d543a5ab5e1e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:11.991775 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 9422, file segment info: File segment: [0, 33554431], key: f217dd9667706836223d543a5ab5e1e3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:12.000221 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:12.123519 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 614 2024.10.22 01:59:12.570387 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:12.577685 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 01:59:12.655697 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_596_596_0 (state Temporary) 2024.10.22 01:59:12.860449 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_596_596_0 to 202410_606_606_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:12.900422 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} MergeTask::MergeProjectionsStage: Merge sorted 9240 rows, containing 4 columns (4 merged, 0 gathered) in 2.156551733 sec., 4284.617826972389 rows/sec., 66.95 KiB/sec. 2024.10.22 01:59:12.970145 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} MergedBlockOutputStream: filled checksums 202410_707_712_1 (state Temporary) 2024.10.22 01:59:12.997595 [ 1738 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qpb/hnllljucvowdnddplkbujffkxkgvi, size 186 2024.10.22 01:59:13.003581 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_707_712_1 to 202410_707_712_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:13.009420 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_707_712_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_707_707_0, 202410_712_712_0] -> 202410_707_712_1 2024.10.22 01:59:13.058383 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 01:59:13.261618 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 614 2024.10.22 01:59:13.341143 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(ca393420dcd0c5217cd0e6529781412a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ca393420dcd0c5217cd0e6529781412a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:13.344801 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 488, file segment info: File segment: [0, 33554431], key: ca393420dcd0c5217cd0e6529781412a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:13.357175 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:13.388386 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1151_11, 202410_1152_1152_0, 202410_1153_1153_0, 202410_1154_1154_0, 202410_1155_1155_0, 202410_1156_1156_0] 2024.10.22 01:59:13.394644 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1151_11, 202410_1152_1152_0, 202410_1153_1153_0, 202410_1154_1154_0, 202410_1155_1155_0, 202410_1156_1156_0] 2024.10.22 01:59:13.413817 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:13.422405 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 01:59:13.438944 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1151_11, 202410_1152_1152_0, 202410_1153_1153_0, 202410_1154_1154_0, 202410_1155_1155_0, 202410_1156_1156_0] 2024.10.22 01:59:13.713567 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.27 GiB, peak 2.00 GiB, free memory in arenas 12.58 MiB, will set to 1.39 GiB (RSS), difference: 118.66 MiB 2024.10.22 01:59:14.065542 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 82113 2024.10.22 01:59:14.247215 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(f2a6e42ac3976d69bb059fe8977bd3fe) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f2a6e42ac3976d69bb059fe8977bd3fe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:14.288655 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 5881, file segment info: File segment: [0, 33554431], key: f2a6e42ac3976d69bb059fe8977bd3fe, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:14.296921 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:14.373549 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:14.482425 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.38 GiB. 2024.10.22 01:59:14.482230 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:59:14.715936 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1242_1242_0 (state Temporary) 2024.10.22 01:59:15.135368 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1242_1242_0 to 202410_1255_1255_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:15.315222 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 141 entries to flush up to offset 81686 2024.10.22 01:59:15.363520 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82113 2024.10.22 01:59:15.555662 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_12_12_0 (state Temporary) 2024.10.22 01:59:15.571627 [ 1746 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mmo/vkxvzsmlryvinqeeorwqvyfftfgkw, size 359 2024.10.22 01:59:15.575221 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uaz/ndrrbelygdxgcuzkhxftfmpxgefda (LogSeriesLimiter: on interval from 2024-10-22 01:59:05 to 2024-10-22 01:59:15 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:59:15.728001 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:15.731905 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:15.879426 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_724_724_0 (state Temporary) 2024.10.22 01:59:15.885038 [ 1157 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:59:16.116617 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(b1645db16dbb6ef6bea73b11d3130c25) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b1645db16dbb6ef6bea73b11d3130c25, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:16.128688 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 475, file segment info: File segment: [0, 33554431], key: b1645db16dbb6ef6bea73b11d3130c25, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:16.139808 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:16.158629 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_724_724_0 to 202410_730_730_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:16.198370 [ 1186 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_725_71 to 202410_726_726_0 2024.10.22 01:59:16.202125 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 81686 2024.10.22 01:59:16.203013 [ 1186 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.29 MiB using storage policy from min volume index 0 2024.10.22 01:59:16.205437 [ 1186 ] {} DiskLocal: Reserved 1.29 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:16.232827 [ 1157 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:59:16.239078 [ 1157 ] {} DNSCacheUpdater: Execution took 353 ms. 2024.10.22 01:59:16.254623 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_726_72} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_725_71 to 202410_726_726_0 into Compact with storage Full 2024.10.22 01:59:16.303850 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_726_72} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:16.314363 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_726_72} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_725_71, total 21960 rows starting from the beginning of the part 2024.10.22 01:59:16.339170 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_726_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_726_726_0, total 119 rows starting from the beginning of the part 2024.10.22 01:59:16.458978 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:59:16.551826 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.23 GiB, peak 2.00 GiB, free memory in arenas 13.28 MiB, will set to 1.35 GiB (RSS), difference: 129.82 MiB 2024.10.22 01:59:16.571770 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:59:16.581777 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:59:16.607652 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:59:16.638513 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:59:16.673567 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:59:16.721776 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:59:16.769502 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 01:59:16.839512 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 11221 2024.10.22 01:59:17.082957 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:17.088102 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:17.142273 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_737_737_0 (state Temporary) 2024.10.22 01:59:17.218488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 4 bytes for 97e1363b086da6573d4723551055f525:0) 2024.10.22 01:59:17.238125 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a5f5fefb81d6c8dbb05cb0ef1b0dc92b, offset: 0, size: 154 2024.10.22 01:59:17.243467 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a5f5fefb81d6c8dbb05cb0ef1b0dc92b for removal 2024.10.22 01:59:17.246453 [ 1476 ] {} CacheMetadata: Key a5f5fefb81d6c8dbb05cb0ef1b0dc92b is removed from metadata 2024.10.22 01:59:17.261214 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_148_148_0 (state Temporary) 2024.10.22 01:59:17.293168 [ 1740 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uaz/ndrrbelygdxgcuzkhxftfmpxgefda, size 4 2024.10.22 01:59:17.418312 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_737_737_0 to 202410_750_750_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:17.492840 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11221 2024.10.22 01:59:17.672641 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1072700 2024.10.22 01:59:17.897655 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_726_72} MergeTask::MergeProjectionsStage: Merge sorted 22079 rows, containing 13 columns (13 merged, 0 gathered) in 1.652874088 sec., 13357.944298537519 rows/sec., 5.79 MiB/sec. 2024.10.22 01:59:18.207700 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_726_72} MergedBlockOutputStream: filled checksums 202410_507_726_72 (state Temporary) 2024.10.22 01:59:18.270304 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(a015f89918ba9ab10b336f8d66414b21) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a015f89918ba9ab10b336f8d66414b21, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:59:18.287595 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: a015f89918ba9ab10b336f8d66414b21, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 01:59:18.292493 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:18.366395 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_726_72} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_726_72 to 202410_507_726_72 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:18.368804 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_726_72} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_725_71, 202410_726_726_0] -> 202410_507_726_72 2024.10.22 01:59:18.414117 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.51 MiB. 2024.10.22 01:59:18.944081 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:18.948051 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:19.128205 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_706_706_0 (state Temporary) 2024.10.22 01:59:19.225231 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_706_706_0 to 202410_713_713_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:19.263920 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1072700 2024.10.22 01:59:19.374359 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 82180 2024.10.22 01:59:19.574136 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:19.584026 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:19.657834 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1243_1243_0 (state Temporary) 2024.10.22 01:59:19.800533 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1243_1243_0 to 202410_1256_1256_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:19.882667 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82180 2024.10.22 01:59:20.872585 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_145_149_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:59:21.024491 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:59:21.028913 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:59:21.460035 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 12.63 MiB, will set to 1.36 GiB (RSS), difference: 2.84 MiB 2024.10.22 01:59:22.144185 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_706_135/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:59:22.185305 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_667_33, 202410_668_668_0, 202410_669_669_0, 202410_670_670_0, 202410_671_671_0, 202410_672_672_0] 2024.10.22 01:59:22.187756 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_667_33, 202410_668_668_0, 202410_669_669_0, 202410_670_670_0, 202410_671_671_0, 202410_672_672_0] 2024.10.22 01:59:22.231076 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_667_33, 202410_668_668_0, 202410_669_669_0, 202410_670_670_0, 202410_671_671_0, 202410_672_672_0] 2024.10.22 01:59:23.396056 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/efp/etamwtqeurlurppiwqagwnfrlscle. 2024.10.22 01:59:23.475823 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_553_113, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0, 202410_557_557_0, 202410_558_558_0] 2024.10.22 01:59:23.478989 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_553_113, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0, 202410_557_557_0, 202410_558_558_0] 2024.10.22 01:59:23.615154 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_553_113, 202410_554_554_0, 202410_555_555_0, 202410_556_556_0, 202410_557_557_0, 202410_558_558_0] 2024.10.22 01:59:23.676272 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 13.27 MiB, will set to 1.35 GiB (RSS), difference: -3.13 MiB 2024.10.22 01:59:23.767436 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 98 entries to flush up to offset 81784 2024.10.22 01:59:23.958701 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 20 entries to flush up to offset 82200 2024.10.22 01:59:23.963209 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:23.970716 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:24.145375 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_725_725_0 (state Temporary) 2024.10.22 01:59:24.188240 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:24.191879 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.37 GiB. 2024.10.22 01:59:24.223127 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_148_148_0 to 202410_151_151_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:24.293250 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_725_725_0 to 202410_731_731_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:24.327274 [ 1245 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_726_72 to 202410_727_727_0 2024.10.22 01:59:24.337708 [ 1245 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.30 MiB using storage policy from min volume index 0 2024.10.22 01:59:24.342115 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 81784 2024.10.22 01:59:24.343257 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1244_1244_0 (state Temporary) 2024.10.22 01:59:24.343882 [ 1245 ] {} DiskLocal: Reserved 1.30 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:24.371958 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_727_73} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_726_72 to 202410_727_727_0 into Compact with storage Full 2024.10.22 01:59:24.475215 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1244_1244_0 to 202410_1257_1257_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:24.477106 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_727_73} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:24.519343 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1252_9 to 202410_1257_1257_0 2024.10.22 01:59:24.529396 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_727_73} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_726_72, total 22079 rows starting from the beginning of the part 2024.10.22 01:59:24.554245 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:24.562978 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:24.568226 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82200 2024.10.22 01:59:24.603404 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1252_9 to 202410_1257_1257_0 into Compact with storage Full 2024.10.22 01:59:24.764757 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:24.798199 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1252_9, total 2793 rows starting from the beginning of the part 2024.10.22 01:59:24.852676 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_727_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_727_727_0, total 95 rows starting from the beginning of the part 2024.10.22 01:59:24.864624 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1253_1253_0, total 81 rows starting from the beginning of the part 2024.10.22 01:59:24.967885 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1254_1254_0, total 41 rows starting from the beginning of the part 2024.10.22 01:59:24.995650 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 11230 2024.10.22 01:59:25.062175 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1255_1255_0, total 57 rows starting from the beginning of the part 2024.10.22 01:59:25.196587 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1256_1256_0, total 67 rows starting from the beginning of the part 2024.10.22 01:59:25.203070 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:25.206195 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:25.250646 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_738_738_0 (state Temporary) 2024.10.22 01:59:25.268406 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_706_135} MergeTask::MergeProjectionsStage: Merge sorted 1075895 rows, containing 4 columns (4 merged, 0 gathered) in 31.374615669 sec., 34291.89416535383 rows/sec., 514.39 KiB/sec. 2024.10.22 01:59:25.319360 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_706_135} MergedBlockOutputStream: filled checksums 202410_1_706_135 (state Temporary) 2024.10.22 01:59:25.348249 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_738_738_0 to 202410_751_751_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:25.378330 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1257_1257_0, total 20 rows starting from the beginning of the part 2024.10.22 01:59:25.384013 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11230 2024.10.22 01:59:25.400968 [ 1248 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 4 parts from 202410_1_746_155 to 202410_749_749_0 2024.10.22 01:59:25.412717 [ 1248 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:25.415932 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:25.439546 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_749_156} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_746_155 to 202410_749_749_0 into Compact with storage Full 2024.10.22 01:59:25.451395 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 12.36 MiB, will set to 1.35 GiB (RSS), difference: -8.12 MiB 2024.10.22 01:59:25.482391 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_749_156} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:25.492577 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_749_156} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_746_155, total 11266 rows starting from the beginning of the part 2024.10.22 01:59:25.653830 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_749_156} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_747_0, total 16 rows starting from the beginning of the part 2024.10.22 01:59:25.669133 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_749_156} MergeTreeSequentialSource: Reading 2 marks from part 202410_748_748_0, total 9 rows starting from the beginning of the part 2024.10.22 01:59:25.692212 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_749_156} MergeTreeSequentialSource: Reading 2 marks from part 202410_749_749_0, total 20 rows starting from the beginning of the part 2024.10.22 01:59:26.268854 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1074625 2024.10.22 01:59:26.566590 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_12_12_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:59:26.571296 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_749_156} MergeTask::MergeProjectionsStage: Merge sorted 11311 rows, containing 12 columns (12 merged, 0 gathered) in 1.141667793 sec., 9907.435481102337 rows/sec., 1.54 MiB/sec. 2024.10.22 01:59:26.746832 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_749_156} MergedBlockOutputStream: filled checksums 202410_1_749_156 (state Temporary) 2024.10.22 01:59:26.810850 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_727_73} MergeTask::MergeProjectionsStage: Merge sorted 22174 rows, containing 13 columns (13 merged, 0 gathered) in 2.4415154 sec., 9082.0643605197 rows/sec., 3.93 MiB/sec. 2024.10.22 01:59:27.055077 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_727_73} MergedBlockOutputStream: filled checksums 202410_507_727_73 (state Temporary) 2024.10.22 01:59:27.124984 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:27.129111 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:27.271920 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_707_707_0 (state Temporary) 2024.10.22 01:59:27.292456 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} MergeTask::MergeProjectionsStage: Merge sorted 3059 rows, containing 13 columns (13 merged, 0 gathered) in 2.695168666 sec., 1134.9939017137565 rows/sec., 192.81 KiB/sec. 2024.10.22 01:59:27.368494 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5952 2024.10.22 01:59:27.416764 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_727_73} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_727_73 to 202410_507_727_73 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:27.427724 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_727_73} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_726_72, 202410_727_727_0] -> 202410_507_727_73 2024.10.22 01:59:27.460940 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.50 MiB. 2024.10.22 01:59:27.475755 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} MergedBlockOutputStream: filled checksums 202410_1207_1257_10 (state Temporary) 2024.10.22 01:59:27.477746 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_707_707_0 to 202410_714_714_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:27.516316 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 12.16 MiB, will set to 1.35 GiB (RSS), difference: 19.83 MiB 2024.10.22 01:59:27.530401 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_749_156} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_749_156 to 202410_1_749_156 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:27.535130 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_749_156} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 4 parts: [202410_1_746_155, 202410_749_749_0] -> 202410_1_749_156 2024.10.22 01:59:27.542256 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1074625 2024.10.22 01:59:27.574400 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.49 MiB. 2024.10.22 01:59:27.764758 [ 1265 ] {} DatabaseCatalog: Execution took 266 ms. 2024.10.22 01:59:27.870101 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1257_10 to 202410_1207_1257_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:27.888820 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1257_10} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1252_9, 202410_1257_1257_0] -> 202410_1207_1257_10 2024.10.22 01:59:27.947353 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:59:28.264334 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 615 2024.10.22 01:59:28.463692 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:28.465402 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:28.571103 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 82276 2024.10.22 01:59:28.759220 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_706_135} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_706_135 to 202410_1_706_135 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:28.764859 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_706_135} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_700_134, 202410_701_706_1] -> 202410_1_706_135 2024.10.22 01:59:28.799794 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 01:59:28.843923 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_597_597_0 (state Temporary) 2024.10.22 01:59:29.060257 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:29.064975 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:29.121512 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1245_1245_0 (state Temporary) 2024.10.22 01:59:29.222875 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_597_597_0 to 202410_607_607_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:29.262508 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1245_1245_0 to 202410_1258_1258_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:29.408689 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 615 2024.10.22 01:59:29.424871 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82276 2024.10.22 01:59:29.517256 [ 1182 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 01:59:29.634365 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 12.30 MiB, will set to 1.35 GiB (RSS), difference: 3.95 MiB 2024.10.22 01:59:29.807216 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_687_138, 202410_688_688_0, 202410_689_689_0, 202410_690_690_0, 202410_691_691_0, 202410_692_692_0] 2024.10.22 01:59:29.814891 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_687_138, 202410_688_688_0, 202410_689_689_0, 202410_690_690_0, 202410_691_691_0, 202410_692_692_0] 2024.10.22 01:59:29.842495 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_687_138, 202410_688_688_0, 202410_689_689_0, 202410_690_690_0, 202410_691_691_0, 202410_692_692_0] 2024.10.22 01:59:31.022365 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:59:31.024948 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:59:31.239959 [ 1224 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:59:31.267701 [ 1224 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:59:31.957769 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 106 entries to flush up to offset 81890 2024.10.22 01:59:32.076108 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:32.091532 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:32.152845 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_726_726_0 (state Temporary) 2024.10.22 01:59:32.166809 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_145_149_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 01:59:32.324674 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_726_726_0 to 202410_732_732_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:32.355191 [ 1158 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_727_73 to 202410_728_728_0 2024.10.22 01:59:32.360221 [ 1158 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 01:59:32.363248 [ 1158 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:32.365138 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 81890 2024.10.22 01:59:32.371799 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_728_74} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_727_73 to 202410_728_728_0 into Compact with storage Full 2024.10.22 01:59:32.438836 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_728_74} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:32.449365 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_728_74} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_727_73, total 22174 rows starting from the beginning of the part 2024.10.22 01:59:32.473832 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_728_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_728_728_0, total 127 rows starting from the beginning of the part 2024.10.22 01:59:32.657093 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d9ab26553571fd25b62700193c6b8d0b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d9ab26553571fd25b62700193c6b8d0b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:32.659142 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 872160, file segment info: File segment: [0, 33554431], key: d9ab26553571fd25b62700193c6b8d0b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:32.663722 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:32.676966 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_653_653_0, 202410_654_654_0, 202410_655_655_0, 202410_656_656_0, 202410_657_657_0, 202410_658_658_0] 2024.10.22 01:59:32.679575 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_653_653_0, 202410_654_654_0, 202410_655_655_0, 202410_656_656_0, 202410_657_657_0, 202410_658_658_0] 2024.10.22 01:59:32.710534 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_653_653_0, 202410_654_654_0, 202410_655_655_0, 202410_656_656_0, 202410_657_657_0, 202410_658_658_0] 2024.10.22 01:59:32.890159 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 11253 2024.10.22 01:59:33.216382 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:33.222426 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:33.259414 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_739_739_0 (state Temporary) 2024.10.22 01:59:33.304199 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_739_739_0 to 202410_752_752_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:33.432437 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 82322 2024.10.22 01:59:33.453062 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(3a0a6d1461c93755d120578225d093d2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3a0a6d1461c93755d120578225d093d2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:33.454878 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1086, file segment info: File segment: [0, 33554431], key: 3a0a6d1461c93755d120578225d093d2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:33.459467 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:33.466133 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11253 2024.10.22 01:59:33.536855 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 01:59:33.567248 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:33.641112 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:33.679709 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1246_1246_0 (state Temporary) 2024.10.22 01:59:33.751195 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1246_1246_0 to 202410_1259_1259_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:34.071786 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82322 2024.10.22 01:59:34.273745 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_728_74} MergeTask::MergeProjectionsStage: Merge sorted 22301 rows, containing 13 columns (13 merged, 0 gathered) in 1.894559181 sec., 11771.075944024575 rows/sec., 5.10 MiB/sec. 2024.10.22 01:59:34.346150 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_728_74} MergedBlockOutputStream: filled checksums 202410_507_728_74 (state Temporary) 2024.10.22 01:59:34.400129 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_728_74} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_728_74 to 202410_507_728_74 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:34.403646 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_728_74} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_727_73, 202410_728_728_0] -> 202410_507_728_74 2024.10.22 01:59:34.464659 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(21d2d46086fc0543ff81d6ed28e01821) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 21d2d46086fc0543ff81d6ed28e01821, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:34.468018 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 866, file segment info: File segment: [0, 33554431], key: 21d2d46086fc0543ff81d6ed28e01821, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 01:59:34.471097 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:34.545032 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.51 MiB. 2024.10.22 01:59:34.553089 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1076165 2024.10.22 01:59:34.761469 [ 1713 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vtb/ctligvdpoicwsncxrxywltuexikpv, size 2514155 2024.10.22 01:59:34.926685 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 5993 2024.10.22 01:59:35.243072 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:59:35.253768 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:59:35.263221 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:59:35.269588 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:59:35.297817 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:59:35.311058 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:59:35.448865 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 01:59:35.492162 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bke/vxmdqnjsmbjtuycggtuoafdrugehu (LogSeriesLimiter: on interval from 2024-10-22 01:59:15 to 2024-10-22 01:59:35 accepted series 1 / 12 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:59:35.680816 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(3ee93c3d7f2a68e09c8b550402ccbd20) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3ee93c3d7f2a68e09c8b550402ccbd20, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:59:35.685529 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 14475, file segment info: File segment: [0, 33554431], key: 3ee93c3d7f2a68e09c8b550402ccbd20, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:59:35.689134 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:35.900316 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:35.906460 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:36.206024 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_708_708_0 (state Temporary) 2024.10.22 01:59:36.377430 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_708_708_0 to 202410_715_715_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:36.405622 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(4f90a366a2b3732ae42e3f9188ba69c5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4f90a366a2b3732ae42e3f9188ba69c5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:59:36.407738 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 503, file segment info: File segment: [0, 33554431], key: 4f90a366a2b3732ae42e3f9188ba69c5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:59:36.411457 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:36.526078 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1076165 2024.10.22 01:59:36.542678 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 4.72 MiB, will set to 1.34 GiB (RSS), difference: -1.50 MiB 2024.10.22 01:59:36.915242 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 616 2024.10.22 01:59:37.050824 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(2b3474e0d743bdbf768e353d11a1e29f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2b3474e0d743bdbf768e353d11a1e29f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:59:37.055929 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 5875, file segment info: File segment: [0, 33554431], key: 2b3474e0d743bdbf768e353d11a1e29f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:59:37.059288 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:37.259652 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:37.261138 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:37.415028 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_598_598_0 (state Temporary) 2024.10.22 01:59:37.665455 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_598_598_0 to 202410_608_608_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:37.698537 [ 1180 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_603_123 to 202410_608_608_0 2024.10.22 01:59:37.759680 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(524e4e2d7af5d3bf04da0f8a80bdae97) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 524e4e2d7af5d3bf04da0f8a80bdae97, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:59:37.766871 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 475, file segment info: File segment: [0, 33554431], key: 524e4e2d7af5d3bf04da0f8a80bdae97, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 01:59:37.768158 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:37.835558 [ 1180 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:37.844463 [ 1180 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:37.850195 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 616 2024.10.22 01:59:37.867244 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_603_123 to 202410_608_608_0 into Compact with storage Full 2024.10.22 01:59:38.045727 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:59:38.077031 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 82381 2024.10.22 01:59:38.128261 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:38.171094 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:59:38.180599 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_603_123, total 647 rows starting from the beginning of the part 2024.10.22 01:59:38.207182 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:59:38.256935 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:59:38.286275 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:59:38.305556 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:59:38.460618 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:59:38.467400 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_604_604_0, total 1 rows starting from the beginning of the part 2024.10.22 01:59:38.481433 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:38.487512 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:38.557691 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1247_1247_0 (state Temporary) 2024.10.22 01:59:38.600151 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 01:59:38.608060 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1247_1247_0 to 202410_1260_1260_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:38.667354 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82381 2024.10.22 01:59:38.870679 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_605_605_0, total 1 rows starting from the beginning of the part 2024.10.22 01:59:39.154919 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_606_606_0, total 1 rows starting from the beginning of the part 2024.10.22 01:59:39.450420 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_607_607_0, total 1 rows starting from the beginning of the part 2024.10.22 01:59:39.625365 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_608_608_0, total 1 rows starting from the beginning of the part 2024.10.22 01:59:39.872146 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 76 entries to flush up to offset 81966 2024.10.22 01:59:40.100263 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:40.107913 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:40.230227 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_727_727_0 (state Temporary) 2024.10.22 01:59:40.371886 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_727_727_0 to 202410_733_733_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:40.400011 [ 1220 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_728_74 to 202410_729_729_0 2024.10.22 01:59:40.414956 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 3.92 MiB, will set to 1.34 GiB (RSS), difference: -6.94 MiB 2024.10.22 01:59:40.420720 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 81966 2024.10.22 01:59:40.508303 [ 1220 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 01:59:40.511636 [ 1220 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:40.550840 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_729_75} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_728_74 to 202410_729_729_0 into Compact with storage Full 2024.10.22 01:59:40.585075 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_729_75} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:40.589684 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1156_12, 202410_1157_1157_0, 202410_1158_1158_0, 202410_1159_1159_0, 202410_1160_1160_0, 202410_1161_1161_0] 2024.10.22 01:59:40.594363 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1156_12, 202410_1157_1157_0, 202410_1158_1158_0, 202410_1159_1159_0, 202410_1160_1160_0, 202410_1161_1161_0] 2024.10.22 01:59:40.602407 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_729_75} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_728_74, total 22301 rows starting from the beginning of the part 2024.10.22 01:59:40.649092 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_729_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_729_729_0, total 119 rows starting from the beginning of the part 2024.10.22 01:59:40.718291 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1156_12, 202410_1157_1157_0, 202410_1158_1158_0, 202410_1159_1159_0, 202410_1160_1160_0, 202410_1161_1161_0] 2024.10.22 01:59:40.969124 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 11262 2024.10.22 01:59:41.022731 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:59:41.029343 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:59:41.092192 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:41.096399 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:41.152963 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_740_740_0 (state Temporary) 2024.10.22 01:59:41.197208 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_740_740_0 to 202410_753_753_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:41.265505 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11262 2024.10.22 01:59:41.591354 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_729_75} MergeTask::MergeProjectionsStage: Merge sorted 22420 rows, containing 13 columns (13 merged, 0 gathered) in 1.046291152 sec., 21428.069956573618 rows/sec., 9.28 MiB/sec. 2024.10.22 01:59:41.968453 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_729_75} MergedBlockOutputStream: filled checksums 202410_507_729_75 (state Temporary) 2024.10.22 01:59:42.073130 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} MergeTask::MergeProjectionsStage: Merge sorted 652 rows, containing 32 columns (32 merged, 0 gathered) in 4.212703839 sec., 154.7699588952757 rows/sec., 14.08 KiB/sec. 2024.10.22 01:59:42.140626 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} MergedBlockOutputStream: filled checksums 202410_1_608_124 (state Temporary) 2024.10.22 01:59:42.278499 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_729_75} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_729_75 to 202410_507_729_75 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:42.282348 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_729_75} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_728_74, 202410_729_729_0] -> 202410_507_729_75 2024.10.22 01:59:42.315054 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.51 MiB. 2024.10.22 01:59:42.473525 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_608_124 to 202410_1_608_124 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:42.481012 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_608_124} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_603_123, 202410_608_608_0] -> 202410_1_608_124 2024.10.22 01:59:42.545619 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 01:59:42.684235 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 82436 2024.10.22 01:59:42.809355 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 4.29 MiB, will set to 1.34 GiB (RSS), difference: 4.43 MiB 2024.10.22 01:59:43.168290 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:43.170732 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:43.211635 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1248_1248_0 (state Temporary) 2024.10.22 01:59:43.352214 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1248_1248_0 to 202410_1261_1261_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:43.401443 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82436 2024.10.22 01:59:43.627091 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1698 entries to flush up to offset 1077863 2024.10.22 01:59:44.847085 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:44.849782 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:45.042449 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_709_709_0 (state Temporary) 2024.10.22 01:59:45.111283 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_709_709_0 to 202410_716_716_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:45.153712 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_706_135 to 202410_707_712_1 2024.10.22 01:59:45.159969 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:45.161715 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:45.164396 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1077863 2024.10.22 01:59:45.194612 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_712_136} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_706_135 to 202410_707_712_1 into Wide with storage Full 2024.10.22 01:59:45.268601 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_712_136} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:45.275538 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_712_136} MergeTreeSequentialSource: Reading 133 marks from part 202410_1_706_135, total 1075895 rows starting from the beginning of the part 2024.10.22 01:59:45.346638 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_712_136} MergeTreeSequentialSource: Reading 3 marks from part 202410_707_712_1, total 9240 rows starting from the beginning of the part 2024.10.22 01:59:45.351885 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 617 2024.10.22 01:59:45.404361 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 4.38 MiB, will set to 1.34 GiB (RSS), difference: -7.79 MiB 2024.10.22 01:59:45.743813 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:45.747973 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:45.906225 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_599_599_0 (state Temporary) 2024.10.22 01:59:46.232818 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_599_599_0 to 202410_609_609_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:46.274425 [ 1255 ] {} DNSResolver: Updating DNS cache 2024.10.22 01:59:46.297061 [ 1255 ] {} DNSResolver: Updated DNS cache 2024.10.22 01:59:46.424754 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 617 2024.10.22 01:59:46.459994 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 216 ms. 2024.10.22 01:59:46.663425 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 3.97 MiB, will set to 1.34 GiB (RSS), difference: -6.66 MiB 2024.10.22 01:59:47.428905 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 82467 2024.10.22 01:59:47.487079 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_145_149_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:59:47.794976 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:47.798615 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:47.954049 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1249_1249_0 (state Temporary) 2024.10.22 01:59:48.001187 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1249_1249_0 to 202410_1262_1262_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:48.036108 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 112 entries to flush up to offset 82078 2024.10.22 01:59:48.060946 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82467 2024.10.22 01:59:48.138339 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1257_10 to 202410_1262_1262_0 2024.10.22 01:59:48.158929 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:48.161417 [ 1263 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:48.169708 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1257_10 to 202410_1262_1262_0 into Compact with storage Full 2024.10.22 01:59:48.209469 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:48.217732 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1257_10, total 3059 rows starting from the beginning of the part 2024.10.22 01:59:48.232711 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:48.238566 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:48.254227 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1258_1258_0, total 76 rows starting from the beginning of the part 2024.10.22 01:59:48.273791 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1259_1259_0, total 46 rows starting from the beginning of the part 2024.10.22 01:59:48.274894 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_728_728_0 (state Temporary) 2024.10.22 01:59:48.302298 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 5.53 MiB, will set to 1.34 GiB (RSS), difference: 2.52 MiB 2024.10.22 01:59:48.303109 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1260_1260_0, total 59 rows starting from the beginning of the part 2024.10.22 01:59:48.310424 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(c6fbfb2b3f29b3c920a4e4e69b1c3f39) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c6fbfb2b3f29b3c920a4e4e69b1c3f39, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:59:48.324018 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1261_1261_0, total 55 rows starting from the beginning of the part 2024.10.22 01:59:48.343413 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: c6fbfb2b3f29b3c920a4e4e69b1c3f39, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 01:59:48.347269 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:48.353542 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_728_728_0 to 202410_734_734_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:48.358027 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1262_1262_0, total 31 rows starting from the beginning of the part 2024.10.22 01:59:48.375758 [ 1242 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_729_75 to 202410_730_730_0 2024.10.22 01:59:48.398510 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 82078 2024.10.22 01:59:48.405462 [ 1242 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.32 MiB using storage policy from min volume index 0 2024.10.22 01:59:48.410432 [ 1242 ] {} DiskLocal: Reserved 1.32 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 01:59:48.533480 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_730_76} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_729_75 to 202410_730_730_0 into Compact with storage Full 2024.10.22 01:59:48.634184 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_730_76} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:48.698138 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_730_76} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_729_75, total 22420 rows starting from the beginning of the part 2024.10.22 01:59:48.771199 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_730_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_730_730_0, total 141 rows starting from the beginning of the part 2024.10.22 01:59:48.771273 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 11278 2024.10.22 01:59:48.867475 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:48.869426 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 01:59:48.914457 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_741_741_0 (state Temporary) 2024.10.22 01:59:48.957743 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_741_741_0 to 202410_754_754_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:49.020651 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11278 2024.10.22 01:59:49.116400 [ 1182 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 4 parts from 202410_1_749_156 to 202410_752_752_0 2024.10.22 01:59:49.133714 [ 1182 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:49.240676 [ 1182 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 01:59:49.252564 [ 1182 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 268 ms. 2024.10.22 01:59:49.262354 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_752_157} MergeTask::PrepareStage: Merging 4 parts: from 202410_1_749_156 to 202410_752_752_0 into Compact with storage Full 2024.10.22 01:59:49.458759 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_752_157} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:49.468567 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_752_157} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_749_156, total 11311 rows starting from the beginning of the part 2024.10.22 01:59:49.499052 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_752_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_750_750_0, total 16 rows starting from the beginning of the part 2024.10.22 01:59:49.531968 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_752_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_751_751_0, total 9 rows starting from the beginning of the part 2024.10.22 01:59:49.558009 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_752_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_752_752_0, total 23 rows starting from the beginning of the part 2024.10.22 01:59:49.669259 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:49.675212 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 01:59:49.751242 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/efp/etamwtqeurlurppiwqagwnfrlscle. 2024.10.22 01:59:49.760582 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} MergeTask::MergeProjectionsStage: Merge sorted 3326 rows, containing 13 columns (13 merged, 0 gathered) in 1.595236591 sec., 2084.957189901871 rows/sec., 354.74 KiB/sec. 2024.10.22 01:59:49.863951 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_712_136/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:59:49.988514 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} MergedBlockOutputStream: filled checksums 202410_1207_1262_11 (state Temporary) 2024.10.22 01:59:50.139080 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1262_11 to 202410_1207_1262_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:50.145345 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1262_11} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1257_10, 202410_1262_1262_0] -> 202410_1207_1262_11 2024.10.22 01:59:50.168231 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 01:59:50.653146 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_145_149_1} MergeTask::MergeProjectionsStage: Merge sorted 213 rows, containing 679 columns (679 merged, 0 gathered) in 88.814380527 sec., 2.3982602674940345 rows/sec., 3.65 KiB/sec. 2024.10.22 01:59:50.702652 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_730_76} MergeTask::MergeProjectionsStage: Merge sorted 22561 rows, containing 13 columns (13 merged, 0 gathered) in 2.282113986 sec., 9886.009260888864 rows/sec., 4.28 MiB/sec. 2024.10.22 01:59:50.709424 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.27 GiB, peak 2.00 GiB, free memory in arenas 10.21 MiB, will set to 1.35 GiB (RSS), difference: 86.50 MiB 2024.10.22 01:59:50.891129 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_730_76} MergedBlockOutputStream: filled checksums 202410_507_730_76 (state Temporary) 2024.10.22 01:59:51.049253 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_752_157} MergeTask::MergeProjectionsStage: Merge sorted 11359 rows, containing 12 columns (12 merged, 0 gathered) in 1.794537771 sec., 6329.763677066679 rows/sec., 1008.82 KiB/sec. 2024.10.22 01:59:51.099282 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_752_157} MergedBlockOutputStream: filled checksums 202410_1_752_157 (state Temporary) 2024.10.22 01:59:51.127582 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 01:59:51.137194 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 01:59:51.176843 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_730_76} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_730_76 to 202410_507_730_76 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:51.181258 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_730_76} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_729_75, 202410_730_730_0] -> 202410_507_730_76 2024.10.22 01:59:51.331307 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.51 MiB. 2024.10.22 01:59:51.528179 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_752_157} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_752_157 to 202410_1_752_157 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:51.535591 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_752_157} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 4 parts: [202410_1_749_156, 202410_752_752_0] -> 202410_1_752_157 2024.10.22 01:59:51.565267 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.49 MiB. 2024.10.22 01:59:51.608665 [ 1386 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_145_149_1} MergedBlockOutputStream: filled checksums 202410_145_149_1 (state Temporary) 2024.10.22 01:59:51.887640 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d57571a61537325e31082d0b4b97116a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d57571a61537325e31082d0b4b97116a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:59:51.892358 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 673, file segment info: File segment: [0, 33554431], key: d57571a61537325e31082d0b4b97116a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 01:59:51.895658 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 01:59:52.053307 [ 1674 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bke/vxmdqnjsmbjtuycggtuoafdrugehu, size 1198636 2024.10.22 01:59:52.070482 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 82546 2024.10.22 01:59:52.181065 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1430 entries to flush up to offset 1079293 2024.10.22 01:59:52.227674 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:59:52.260384 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:59:52.279696 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:59:52.296023 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:59:52.315644 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:59:52.338565 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:59:52.358136 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 01:59:52.414979 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:52.419036 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 01:59:52.481344 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1250_1250_0 (state Temporary) 2024.10.22 01:59:52.555570 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/dvd/haiselxjgshsmavwjvqpuopbultop (LogSeriesLimiter: on interval from 2024-10-22 01:59:35 to 2024-10-22 01:59:52 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 01:59:52.739088 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1250_1250_0 to 202410_1263_1263_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:52.790418 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:52.807353 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 01:59:52.848170 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82546 2024.10.22 01:59:52.892365 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_710_710_0 (state Temporary) 2024.10.22 01:59:53.164471 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_710_710_0 to 202410_717_717_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:53.200990 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1079293 2024.10.22 01:59:53.453051 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 2.00 GiB, free memory in arenas 13.44 MiB, will set to 1.33 GiB (RSS), difference: 83.23 MiB 2024.10.22 01:59:53.560349 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_149_149_0 (state Temporary) 2024.10.22 01:59:53.928670 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 618 2024.10.22 01:59:54.233258 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:54.237031 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 01:59:54.306101 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_600_600_0 (state Temporary) 2024.10.22 01:59:54.415852 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_646_125, 202410_647_652_1] 2024.10.22 01:59:54.423656 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_646_125, 202410_647_652_1] 2024.10.22 01:59:54.449582 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_646_125, 202410_647_652_1] 2024.10.22 01:59:54.565072 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_600_600_0 to 202410_610_610_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:54.639097 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d7352e49a41ea41658d63569ea800eb2) : [0, 273]: Setting download as failed: File segment: [0, 273], key: d7352e49a41ea41658d63569ea800eb2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:59:54.641166 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d7352e49a41ea41658d63569ea800eb2): No space left in cache to reserve 274 bytes, will continue without cache download 2024.10.22 01:59:54.643683 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 618 2024.10.22 01:59:55.514566 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 01:59:55.726187 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 15.34 MiB, will set to 1.33 GiB (RSS), difference: 13.31 MiB 2024.10.22 01:59:55.757304 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4884e472455813f963999689b1f6db53) : [0, 4278234]: Setting download as failed: File segment: [0, 4278234], key: 4884e472455813f963999689b1f6db53, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:59:55.859517 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4884e472455813f963999689b1f6db53): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 01:59:55.924471 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 111 entries to flush up to offset 82189 2024.10.22 01:59:56.065533 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4884e472455813f963999689b1f6db53, offset: 0, size: 0 2024.10.22 01:59:56.067049 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4884e472455813f963999689b1f6db53 for removal 2024.10.22 01:59:56.068960 [ 1476 ] {} CacheMetadata: Key 4884e472455813f963999689b1f6db53 is removed from metadata 2024.10.22 01:59:56.072070 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d7352e49a41ea41658d63569ea800eb2, offset: 0, size: 0 2024.10.22 01:59:56.073351 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d7352e49a41ea41658d63569ea800eb2 for removal 2024.10.22 01:59:56.075595 [ 1476 ] {} CacheMetadata: Key d7352e49a41ea41658d63569ea800eb2 is removed from metadata 2024.10.22 01:59:56.161337 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:56.223734 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 01:59:56.267178 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_729_729_0 (state Temporary) 2024.10.22 01:59:56.327719 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_729_729_0 to 202410_735_735_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:56.363354 [ 1179 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_730_76 to 202410_731_731_0 2024.10.22 01:59:56.378242 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 82189 2024.10.22 01:59:56.378868 [ 1179 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 01:59:56.382874 [ 1179 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 01:59:56.423940 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_731_77} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_730_76 to 202410_731_731_0 into Compact with storage Full 2024.10.22 01:59:56.453244 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_731_77} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 01:59:56.462940 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_731_77} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_730_76, total 22561 rows starting from the beginning of the part 2024.10.22 01:59:56.476002 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_731_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_731_731_0, total 98 rows starting from the beginning of the part 2024.10.22 01:59:56.624765 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 11298 2024.10.22 01:59:56.860849 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:56.904299 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 82603 2024.10.22 01:59:56.905599 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 01:59:56.945793 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_742_742_0 (state Temporary) 2024.10.22 01:59:57.114641 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(be611863210e7bc1c2f4a595ad1a0088) : [0, 161]: Setting download as failed: File segment: [0, 161], key: be611863210e7bc1c2f4a595ad1a0088, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:59:57.115463 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_742_742_0 to 202410_755_755_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:57.119669 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(be611863210e7bc1c2f4a595ad1a0088): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 01:59:57.150862 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 01:59:57.157368 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 01:59:57.272632 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11298 2024.10.22 01:59:57.312811 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1251_1251_0 (state Temporary) 2024.10.22 01:59:57.363457 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 11.94 MiB, will set to 1.34 GiB (RSS), difference: -17.04 MiB 2024.10.22 01:59:57.367381 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1251_1251_0 to 202410_1264_1264_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:57.447209 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82603 2024.10.22 01:59:57.946821 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_731_77} MergeTask::MergeProjectionsStage: Merge sorted 22659 rows, containing 13 columns (13 merged, 0 gathered) in 1.538384387 sec., 14729.088640965258 rows/sec., 6.38 MiB/sec. 2024.10.22 01:59:58.041007 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_731_77} MergedBlockOutputStream: filled checksums 202410_507_731_77 (state Temporary) 2024.10.22 01:59:58.098995 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4a3a6fac14b343281bd38a5b9db8a334) : [0, 104130]: Setting download as failed: File segment: [0, 104130], key: 4a3a6fac14b343281bd38a5b9db8a334, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:59:58.100713 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4a3a6fac14b343281bd38a5b9db8a334): No space left in cache to reserve 104131 bytes, will continue without cache download 2024.10.22 01:59:58.211562 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_731_77} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_731_77 to 202410_507_731_77 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:58.222963 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_731_77} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_730_76, 202410_731_731_0] -> 202410_507_731_77 2024.10.22 01:59:58.265035 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.50 MiB. 2024.10.22 01:59:58.274935 [ 1202 ] {} DatabaseCatalog: Execution took 244 ms. 2024.10.22 01:59:58.448699 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4a3a6fac14b343281bd38a5b9db8a334, offset: 0, size: 0 2024.10.22 01:59:58.450310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4a3a6fac14b343281bd38a5b9db8a334 for removal 2024.10.22 01:59:58.452244 [ 1476 ] {} CacheMetadata: Key 4a3a6fac14b343281bd38a5b9db8a334 is removed from metadata 2024.10.22 01:59:58.453175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: be611863210e7bc1c2f4a595ad1a0088, offset: 0, size: 0 2024.10.22 01:59:58.555088 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key be611863210e7bc1c2f4a595ad1a0088 for removal 2024.10.22 01:59:58.556787 [ 1476 ] {} CacheMetadata: Key be611863210e7bc1c2f4a595ad1a0088 is removed from metadata 2024.10.22 01:59:58.949442 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_145_149_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_145_149_1 to 202410_145_149_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 01:59:58.952775 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_145_149_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 5 parts: [202410_145_145_0, 202410_149_149_0] -> 202410_145_149_1 2024.10.22 01:59:59.205900 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(733aafae87d3f0dfa26f6ad0a92a52c9) : [0, 185]: Setting download as failed: File segment: [0, 185], key: 733aafae87d3f0dfa26f6ad0a92a52c9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 01:59:59.208762 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(733aafae87d3f0dfa26f6ad0a92a52c9): No space left in cache to reserve 186 bytes, will continue without cache download 2024.10.22 01:59:59.372807 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 9.90 MiB, will set to 1.33 GiB (RSS), difference: 21.25 MiB 2024.10.22 01:59:59.766251 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.75 MiB. 2024.10.22 01:59:59.942634 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(babc8a118b8021aa985cd34c605af1da) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: babc8a118b8021aa985cd34c605af1da, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:59:59.950994 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 9153, file segment info: File segment: [0, 33554431], key: babc8a118b8021aa985cd34c605af1da, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 01:59:59.953081 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:00.206889 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1492 entries to flush up to offset 1080785 2024.10.22 02:00:00.615712 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_712_136/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:00:00.928417 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:00.934182 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:00.947593 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dfdd019ed691132e8adf0506588c23c1) : [0, 4788]: Setting download as failed: File segment: [0, 4788], key: dfdd019ed691132e8adf0506588c23c1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:00.951347 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dfdd019ed691132e8adf0506588c23c1): No space left in cache to reserve 4789 bytes, will continue without cache download 2024.10.22 02:00:00.969830 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dfdd019ed691132e8adf0506588c23c1, offset: 0, size: 0 2024.10.22 02:00:00.971269 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dfdd019ed691132e8adf0506588c23c1 for removal 2024.10.22 02:00:00.975159 [ 1476 ] {} CacheMetadata: Key dfdd019ed691132e8adf0506588c23c1 is removed from metadata 2024.10.22 02:00:00.979192 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 733aafae87d3f0dfa26f6ad0a92a52c9, offset: 0, size: 0 2024.10.22 02:00:00.981640 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 733aafae87d3f0dfa26f6ad0a92a52c9 for removal 2024.10.22 02:00:00.984221 [ 1476 ] {} CacheMetadata: Key 733aafae87d3f0dfa26f6ad0a92a52c9 is removed from metadata 2024.10.22 02:00:00.991520 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_149_149_0 to 202410_152_152_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:01.071868 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_711_711_0 (state Temporary) 2024.10.22 02:00:01.135842 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:00:01.140156 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:00:01.260396 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_711_711_0 to 202410_718_718_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:01.295559 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1080785 2024.10.22 02:00:01.305026 [ 1247 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:00:01.306064 [ 1220 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_713_713_0 to 202410_718_718_0 2024.10.22 02:00:01.325749 [ 1220 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:01.330304 [ 1220 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:01.356980 [ 1247 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:00:01.366558 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_713_713_0 to 202410_718_718_0 into Compact with storage Full 2024.10.22 02:00:01.402789 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:01.412655 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_713_713_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:00:01.426745 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_714_714_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:00:01.439971 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_715_715_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:00:01.450622 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_716_716_0, total 1698 rows starting from the beginning of the part 2024.10.22 02:00:01.464586 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 82668 2024.10.22 02:00:01.483378 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_717_717_0, total 1430 rows starting from the beginning of the part 2024.10.22 02:00:01.528464 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_718_718_0, total 1492 rows starting from the beginning of the part 2024.10.22 02:00:01.997274 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:02.001456 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:02.148894 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 619 2024.10.22 02:00:02.160197 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1252_1252_0 (state Temporary) 2024.10.22 02:00:02.226712 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(ae2ad8a9784c137101a6e339c732f01b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ae2ad8a9784c137101a6e339c732f01b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:02.352258 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:02.354555 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 543, file segment info: File segment: [0, 33554431], key: ae2ad8a9784c137101a6e339c732f01b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:02.355671 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:02.358605 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:02.413243 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1252_1252_0 to 202410_1265_1265_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:02.460052 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_601_601_0 (state Temporary) 2024.10.22 02:00:02.597660 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82668 2024.10.22 02:00:02.603360 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_601_601_0 to 202410_611_611_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:02.740880 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 619 2024.10.22 02:00:03.099337 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(23d7ae0aadb34ad4eef683aac3c1db2c) : [0, 257]: Setting download as failed: File segment: [0, 257], key: 23d7ae0aadb34ad4eef683aac3c1db2c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:03.102247 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(23d7ae0aadb34ad4eef683aac3c1db2c): No space left in cache to reserve 258 bytes, will continue without cache download 2024.10.22 02:00:03.176757 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} MergeTask::MergeProjectionsStage: Merge sorted 9240 rows, containing 4 columns (4 merged, 0 gathered) in 1.798474453 sec., 5137.6876577740295 rows/sec., 80.28 KiB/sec. 2024.10.22 02:00:03.261134 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} MergedBlockOutputStream: filled checksums 202410_713_718_1 (state Temporary) 2024.10.22 02:00:03.343745 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_713_718_1 to 202410_713_718_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:03.346400 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_713_718_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_713_713_0, 202410_718_718_0] -> 202410_713_718_1 2024.10.22 02:00:03.410803 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:00:03.659475 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 9.75 MiB, will set to 1.33 GiB (RSS), difference: 4.86 MiB 2024.10.22 02:00:03.902118 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 88 entries to flush up to offset 82277 2024.10.22 02:00:04.060085 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_672_34, 202410_673_673_0] 2024.10.22 02:00:04.063623 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_672_34, 202410_673_673_0] 2024.10.22 02:00:04.079443 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_672_34, 202410_673_673_0] 2024.10.22 02:00:04.433503 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(eabc1d53c0f6f1eead2c3140c83cbfdc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: eabc1d53c0f6f1eead2c3140c83cbfdc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:04.440708 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:04.443304 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 5507, file segment info: File segment: [0, 33554431], key: eabc1d53c0f6f1eead2c3140c83cbfdc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:04.444153 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:04.446174 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:04.599835 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_730_730_0 (state Temporary) 2024.10.22 02:00:04.653570 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_730_730_0 to 202410_736_736_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:04.722675 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_731_77 to 202410_732_732_0 2024.10.22 02:00:04.747728 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.34 MiB using storage policy from min volume index 0 2024.10.22 02:00:04.751529 [ 1254 ] {} DiskLocal: Reserved 1.34 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:04.776367 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 11321 2024.10.22 02:00:04.812280 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 5993 2024.10.22 02:00:04.872846 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_732_78} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_731_77 to 202410_732_732_0 into Compact with storage Full 2024.10.22 02:00:04.950876 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 82277 2024.10.22 02:00:05.063067 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_732_78} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:05.105250 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:05.109594 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:05.171212 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_743_743_0 (state Temporary) 2024.10.22 02:00:05.195410 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_732_78} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_731_77, total 22659 rows starting from the beginning of the part 2024.10.22 02:00:05.252319 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1211f86f9d3ac713c92ce084b00da796) : [0, 2017386]: Setting download as failed: File segment: [0, 2017386], key: 1211f86f9d3ac713c92ce084b00da796, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:05.253682 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1211f86f9d3ac713c92ce084b00da796): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:00:05.353418 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_743_743_0 to 202410_756_756_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:05.440084 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11321 2024.10.22 02:00:05.534082 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_732_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_732_732_0, total 106 rows starting from the beginning of the part 2024.10.22 02:00:05.628363 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1211f86f9d3ac713c92ce084b00da796, offset: 0, size: 0 2024.10.22 02:00:05.631147 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1211f86f9d3ac713c92ce084b00da796 for removal 2024.10.22 02:00:05.634453 [ 1476 ] {} CacheMetadata: Key 1211f86f9d3ac713c92ce084b00da796 is removed from metadata 2024.10.22 02:00:05.639203 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 23d7ae0aadb34ad4eef683aac3c1db2c, offset: 0, size: 0 2024.10.22 02:00:05.641477 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 23d7ae0aadb34ad4eef683aac3c1db2c for removal 2024.10.22 02:00:05.644798 [ 1476 ] {} CacheMetadata: Key 23d7ae0aadb34ad4eef683aac3c1db2c is removed from metadata 2024.10.22 02:00:06.136337 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1161_13, 202410_1162_1162_0, 202410_1163_1163_0, 202410_1164_1164_0, 202410_1165_1165_0, 202410_1166_1166_0] 2024.10.22 02:00:06.139165 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1161_13, 202410_1162_1162_0, 202410_1163_1163_0, 202410_1164_1164_0, 202410_1165_1165_0, 202410_1166_1166_0] 2024.10.22 02:00:06.175176 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_732_78/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:00:06.265756 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1161_13, 202410_1162_1162_0, 202410_1163_1163_0, 202410_1164_1164_0, 202410_1165_1165_0, 202410_1166_1166_0] 2024.10.22 02:00:06.303141 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(75e86ed9d18a10bee91339be9f52b975) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 75e86ed9d18a10bee91339be9f52b975, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:06.309660 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 471, file segment info: File segment: [0, 33554431], key: 75e86ed9d18a10bee91339be9f52b975, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:06.314237 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:06.481399 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:00:06.533442 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:00:06.552072 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:00:06.583534 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:00:06.620191 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 82739 2024.10.22 02:00:06.638247 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:00:06.673836 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:00:06.710457 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:00:06.742845 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:00:06.884354 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:06.907002 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:06.906821 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/baq/gvgkqkhmtgmbtyncgcudvwaegawmt (LogSeriesLimiter: on interval from 2024-10-22 01:59:52 to 2024-10-22 02:00:06 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:00:06.994699 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1253_1253_0 (state Temporary) 2024.10.22 02:00:07.241118 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1253_1253_0 to 202410_1266_1266_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:07.360782 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(7c793707a530c779f2c74db669997d31) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7c793707a530c779f2c74db669997d31, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:00:07.375764 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82739 2024.10.22 02:00:07.388155 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 7c793707a530c779f2c74db669997d31, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:00:07.396117 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:07.605343 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 14.93 MiB, will set to 1.34 GiB (RSS), difference: -4.44 MiB 2024.10.22 02:00:08.159786 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_732_78} MergeTask::MergeProjectionsStage: Merge sorted 22765 rows, containing 13 columns (13 merged, 0 gathered) in 3.296344332 sec., 6906.135314506943 rows/sec., 2.99 MiB/sec. 2024.10.22 02:00:08.194911 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a3d3f67dfa90f6edf2349748996ed58d) : [0, 187]: Setting download as failed: File segment: [0, 187], key: a3d3f67dfa90f6edf2349748996ed58d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:08.197609 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a3d3f67dfa90f6edf2349748996ed58d): No space left in cache to reserve 188 bytes, will continue without cache download 2024.10.22 02:00:08.300334 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1081555 2024.10.22 02:00:08.320990 [ 1257 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 02:00:08.549998 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_732_78} MergedBlockOutputStream: filled checksums 202410_507_732_78 (state Temporary) 2024.10.22 02:00:08.597263 [ 1265 ] {} DatabaseCatalog: Execution took 323 ms. 2024.10.22 02:00:08.656431 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_732_78} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_732_78 to 202410_507_732_78 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:08.663734 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_732_78} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_731_77, 202410_732_732_0] -> 202410_507_732_78 2024.10.22 02:00:08.782984 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.50 MiB. 2024.10.22 02:00:08.807265 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:08.811426 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:09.008831 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_712_712_0 (state Temporary) 2024.10.22 02:00:09.202471 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 3 old parts to remove. Parts: [202410_1_692_139, 202410_693_693_0, 202410_694_694_0] 2024.10.22 02:00:09.223620 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 3 parts from filesystem (serially): Parts: [202410_1_692_139, 202410_693_693_0, 202410_694_694_0] 2024.10.22 02:00:09.241449 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 3 parts from memory: Parts: [202410_1_692_139, 202410_693_693_0, 202410_694_694_0] 2024.10.22 02:00:09.264200 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_712_712_0 to 202410_719_719_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:09.300792 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1081555 2024.10.22 02:00:09.322282 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f09b6d0cafae442bb50c3e85f3949ba7) : [0, 79784]: Setting download as failed: File segment: [0, 79784], key: f09b6d0cafae442bb50c3e85f3949ba7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:09.324319 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f09b6d0cafae442bb50c3e85f3949ba7): No space left in cache to reserve 79785 bytes, will continue without cache download 2024.10.22 02:00:09.358892 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f09b6d0cafae442bb50c3e85f3949ba7, offset: 0, size: 0 2024.10.22 02:00:09.361673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f09b6d0cafae442bb50c3e85f3949ba7 for removal 2024.10.22 02:00:09.365181 [ 1476 ] {} CacheMetadata: Key f09b6d0cafae442bb50c3e85f3949ba7 is removed from metadata 2024.10.22 02:00:09.368469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a3d3f67dfa90f6edf2349748996ed58d, offset: 0, size: 0 2024.10.22 02:00:09.370745 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a3d3f67dfa90f6edf2349748996ed58d for removal 2024.10.22 02:00:09.379379 [ 1476 ] {} CacheMetadata: Key a3d3f67dfa90f6edf2349748996ed58d is removed from metadata 2024.10.22 02:00:09.454230 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 16.66 MiB, will set to 1.33 GiB (RSS), difference: 7.16 MiB 2024.10.22 02:00:10.250996 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(500440f4cd52c0d45288b47a85025d59) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 500440f4cd52c0d45288b47a85025d59, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 02:00:10.252839 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 679, file segment info: File segment: [0, 33554431], key: 500440f4cd52c0d45288b47a85025d59, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 02:00:10.355028 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:10.408047 [ 1730 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/dvd/haiselxjgshsmavwjvqpuopbultop, size 1173061 2024.10.22 02:00:10.855779 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:00:10.917713 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_673_35, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 02:00:10.920164 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_673_35, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 02:00:10.951406 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_673_35, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 02:00:11.088842 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:00:11.116872 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:00:11.129044 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:00:11.132164 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:00:11.189276 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:00:11.221681 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a0f343cac6852bf0aa8007ec298c57ae) : [0, 223]: Setting download as failed: File segment: [0, 223], key: a0f343cac6852bf0aa8007ec298c57ae, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:11.224082 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:00:11.224749 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a0f343cac6852bf0aa8007ec298c57ae): No space left in cache to reserve 224 bytes, will continue without cache download 2024.10.22 02:00:11.252644 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:00:11.273566 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_558_114, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0, 202410_562_562_0, 202410_563_563_0] 2024.10.22 02:00:11.276345 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_558_114, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0, 202410_562_562_0, 202410_563_563_0] 2024.10.22 02:00:11.283397 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:00:11.389173 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 82801 2024.10.22 02:00:11.420775 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wev/bxellflrfinshsgghddyrjlzyfjax (LogSeriesLimiter: on interval from 2024-10-22 02:00:06 to 2024-10-22 02:00:11 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:00:11.524807 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_558_114, 202410_559_559_0, 202410_560_560_0, 202410_561_561_0, 202410_562_562_0, 202410_563_563_0] 2024.10.22 02:00:11.750851 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:11.753327 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:11.912559 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1254_1254_0 (state Temporary) 2024.10.22 02:00:11.965709 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1254_1254_0 to 202410_1267_1267_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:11.991138 [ 1204 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1262_11 to 202410_1267_1267_0 2024.10.22 02:00:12.003476 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82801 2024.10.22 02:00:12.007432 [ 1204 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:12.034830 [ 1204 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:12.042980 [ 1219 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 02:00:12.076688 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1262_11 to 202410_1267_1267_0 into Compact with storage Full 2024.10.22 02:00:12.124215 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:12.131507 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1262_11, total 3326 rows starting from the beginning of the part 2024.10.22 02:00:12.148520 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1263_1263_0, total 79 rows starting from the beginning of the part 2024.10.22 02:00:12.195181 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e108a5ef83ca17e75ba22dc0cd8a0c00) : [0, 60812]: Setting download as failed: File segment: [0, 60812], key: e108a5ef83ca17e75ba22dc0cd8a0c00, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:12.199224 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e108a5ef83ca17e75ba22dc0cd8a0c00): No space left in cache to reserve 60813 bytes, will continue without cache download 2024.10.22 02:00:12.227965 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e108a5ef83ca17e75ba22dc0cd8a0c00, offset: 0, size: 0 2024.10.22 02:00:12.232030 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e108a5ef83ca17e75ba22dc0cd8a0c00 for removal 2024.10.22 02:00:12.234865 [ 1476 ] {} CacheMetadata: Key e108a5ef83ca17e75ba22dc0cd8a0c00 is removed from metadata 2024.10.22 02:00:12.237425 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a0f343cac6852bf0aa8007ec298c57ae, offset: 0, size: 0 2024.10.22 02:00:12.239888 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a0f343cac6852bf0aa8007ec298c57ae for removal 2024.10.22 02:00:12.342352 [ 1476 ] {} CacheMetadata: Key a0f343cac6852bf0aa8007ec298c57ae is removed from metadata 2024.10.22 02:00:12.369167 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 6031 2024.10.22 02:00:12.413479 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1264_1264_0, total 57 rows starting from the beginning of the part 2024.10.22 02:00:12.460085 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 86 entries to flush up to offset 82363 2024.10.22 02:00:12.471365 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1265_1265_0, total 65 rows starting from the beginning of the part 2024.10.22 02:00:12.547878 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1266_1266_0, total 71 rows starting from the beginning of the part 2024.10.22 02:00:12.794355 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:12.877624 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_712_136/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:00:12.893742 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1267_1267_0, total 62 rows starting from the beginning of the part 2024.10.22 02:00:12.909572 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:12.943239 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 11328 2024.10.22 02:00:12.953269 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_731_731_0 (state Temporary) 2024.10.22 02:00:12.991401 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_731_731_0 to 202410_737_737_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:13.047551 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_732_78 to 202410_733_733_0 2024.10.22 02:00:13.052558 [ 1237 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.34 MiB using storage policy from min volume index 0 2024.10.22 02:00:13.067483 [ 1237 ] {} DiskLocal: Reserved 1.34 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:13.079914 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_733_79} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_732_78 to 202410_733_733_0 into Compact with storage Full 2024.10.22 02:00:13.084824 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3d60c4cdfdf91a81a170dcfd3f301ec9) : [0, 237]: Setting download as failed: File segment: [0, 237], key: 3d60c4cdfdf91a81a170dcfd3f301ec9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:13.088573 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3d60c4cdfdf91a81a170dcfd3f301ec9): No space left in cache to reserve 238 bytes, will continue without cache download 2024.10.22 02:00:13.132616 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_733_79} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:13.156755 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 82363 2024.10.22 02:00:13.212026 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_733_79} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_732_78, total 22765 rows starting from the beginning of the part 2024.10.22 02:00:13.369579 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_733_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_733_733_0, total 76 rows starting from the beginning of the part 2024.10.22 02:00:13.470452 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 17.03 MiB, will set to 1.33 GiB (RSS), difference: -8.40 MiB 2024.10.22 02:00:13.471130 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:13.473825 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:13.646935 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_744_744_0 (state Temporary) 2024.10.22 02:00:13.716557 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_744_744_0 to 202410_757_757_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:13.749226 [ 1188 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 5 parts from 202410_1_752_157 to 202410_756_756_0 2024.10.22 02:00:13.764048 [ 1188 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:13.766540 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11328 2024.10.22 02:00:13.771151 [ 1188 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:13.822132 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_756_158} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_752_157 to 202410_756_756_0 into Compact with storage Full 2024.10.22 02:00:14.031197 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_756_158} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:14.099252 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_756_158} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_752_157, total 11359 rows starting from the beginning of the part 2024.10.22 02:00:14.152139 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(180df0a45bb9416da7244d3cf72237af) : [0, 573235]: Setting download as failed: File segment: [0, 573235], key: 180df0a45bb9416da7244d3cf72237af, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:14.153130 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(180df0a45bb9416da7244d3cf72237af): No space left in cache to reserve 573236 bytes, will continue without cache download 2024.10.22 02:00:14.158511 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_756_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_753_753_0, total 9 rows starting from the beginning of the part 2024.10.22 02:00:14.253443 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_756_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_754_754_0, total 16 rows starting from the beginning of the part 2024.10.22 02:00:14.307227 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 180df0a45bb9416da7244d3cf72237af, offset: 0, size: 0 2024.10.22 02:00:14.310636 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 180df0a45bb9416da7244d3cf72237af for removal 2024.10.22 02:00:14.316947 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} MergeTask::MergeProjectionsStage: Merge sorted 3660 rows, containing 13 columns (13 merged, 0 gathered) in 2.256468457 sec., 1622.0036174873062 rows/sec., 277.64 KiB/sec. 2024.10.22 02:00:14.327971 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3d60c4cdfdf91a81a170dcfd3f301ec9, offset: 0, size: 0 2024.10.22 02:00:14.342371 [ 1476 ] {} CacheMetadata: Key 180df0a45bb9416da7244d3cf72237af is removed from metadata 2024.10.22 02:00:14.390912 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} MergedBlockOutputStream: filled checksums 202410_1207_1267_12 (state Temporary) 2024.10.22 02:00:14.431351 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3d60c4cdfdf91a81a170dcfd3f301ec9 for removal 2024.10.22 02:00:14.440201 [ 1476 ] {} CacheMetadata: Key 3d60c4cdfdf91a81a170dcfd3f301ec9 is removed from metadata 2024.10.22 02:00:14.631106 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_756_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_755_755_0, total 20 rows starting from the beginning of the part 2024.10.22 02:00:14.640633 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1267_12 to 202410_1207_1267_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:14.644367 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1267_12} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1262_11, 202410_1267_1267_0] -> 202410_1207_1267_12 2024.10.22 02:00:14.755922 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_756_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_756_756_0, total 23 rows starting from the beginning of the part 2024.10.22 02:00:14.767123 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:00:15.052409 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_733_79} MergeTask::MergeProjectionsStage: Merge sorted 22841 rows, containing 13 columns (13 merged, 0 gathered) in 1.980046659 sec., 11535.586748009053 rows/sec., 5.00 MiB/sec. 2024.10.22 02:00:15.417801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e04e7e759aeb661bcbf940fbca9acfb4) : [0, 232]: Setting download as failed: File segment: [0, 232], key: e04e7e759aeb661bcbf940fbca9acfb4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:15.423096 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e04e7e759aeb661bcbf940fbca9acfb4): No space left in cache to reserve 233 bytes, will continue without cache download 2024.10.22 02:00:15.444040 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_733_79} MergedBlockOutputStream: filled checksums 202410_507_733_79 (state Temporary) 2024.10.22 02:00:15.659305 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_733_79} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_733_79 to 202410_507_733_79 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:15.663177 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_733_79} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_732_78, 202410_733_733_0] -> 202410_507_733_79 2024.10.22 02:00:15.710277 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.47 MiB. 2024.10.22 02:00:16.029708 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 82883 2024.10.22 02:00:16.303860 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1082710 2024.10.22 02:00:16.360929 [ 1178 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:00:16.385049 [ 1178 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:00:16.449421 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:16.452441 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:16.612527 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 16.00 MiB, will set to 1.33 GiB (RSS), difference: 3.95 MiB 2024.10.22 02:00:16.618442 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_756_158} MergeTask::MergeProjectionsStage: Merge sorted 11427 rows, containing 12 columns (12 merged, 0 gathered) in 2.807395111 sec., 4070.3212580325676 rows/sec., 648.81 KiB/sec. 2024.10.22 02:00:16.644653 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1255_1255_0 (state Temporary) 2024.10.22 02:00:16.846893 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_756_158} MergedBlockOutputStream: filled checksums 202410_1_756_158 (state Temporary) 2024.10.22 02:00:16.853800 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(79bf835e21a7dc358c3e120e365801c7) : [0, 370682]: Setting download as failed: File segment: [0, 370682], key: 79bf835e21a7dc358c3e120e365801c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:16.856035 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(79bf835e21a7dc358c3e120e365801c7): No space left in cache to reserve 370683 bytes, will continue without cache download 2024.10.22 02:00:16.923241 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1255_1255_0 to 202410_1268_1268_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:16.993171 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82883 2024.10.22 02:00:17.018563 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 79bf835e21a7dc358c3e120e365801c7, offset: 0, size: 0 2024.10.22 02:00:17.022764 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 79bf835e21a7dc358c3e120e365801c7 for removal 2024.10.22 02:00:17.028393 [ 1476 ] {} CacheMetadata: Key 79bf835e21a7dc358c3e120e365801c7 is removed from metadata 2024.10.22 02:00:17.031741 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e04e7e759aeb661bcbf940fbca9acfb4, offset: 0, size: 0 2024.10.22 02:00:17.036801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e04e7e759aeb661bcbf940fbca9acfb4 for removal 2024.10.22 02:00:17.043609 [ 1476 ] {} CacheMetadata: Key e04e7e759aeb661bcbf940fbca9acfb4 is removed from metadata 2024.10.22 02:00:17.376089 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_756_158/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:00:17.451440 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:17.453622 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:17.476594 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_756_158} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_756_158 to 202410_1_756_158 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:17.485091 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_756_158} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 5 parts: [202410_1_752_157, 202410_756_756_0] -> 202410_1_756_158 2024.10.22 02:00:17.532338 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.52 MiB. 2024.10.22 02:00:17.745685 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 620 2024.10.22 02:00:17.928509 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_713_713_0 (state Temporary) 2024.10.22 02:00:18.114808 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_713_713_0 to 202410_720_720_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:18.119329 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:18.124974 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:18.222593 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3846c5cf04659f5f55a97f637624e5a0) : [0, 227]: Setting download as failed: File segment: [0, 227], key: 3846c5cf04659f5f55a97f637624e5a0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:18.225369 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3846c5cf04659f5f55a97f637624e5a0): No space left in cache to reserve 228 bytes, will continue without cache download 2024.10.22 02:00:18.251630 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1082710 2024.10.22 02:00:18.283438 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_602_602_0 (state Temporary) 2024.10.22 02:00:18.365265 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 13.93 MiB, will set to 1.33 GiB (RSS), difference: -8.14 MiB 2024.10.22 02:00:18.399554 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_602_602_0 to 202410_612_612_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:18.571696 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 620 2024.10.22 02:00:18.841637 [ 1164 ] {} DatabaseCatalog: Execution took 217 ms. 2024.10.22 02:00:19.170741 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(63493a20558fcad00292e7b86a84d35f) : [0, 810685]: Setting download as failed: File segment: [0, 810685], key: 63493a20558fcad00292e7b86a84d35f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:19.174131 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(63493a20558fcad00292e7b86a84d35f): No space left in cache to reserve 810686 bytes, will continue without cache download 2024.10.22 02:00:19.334477 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 63493a20558fcad00292e7b86a84d35f, offset: 0, size: 0 2024.10.22 02:00:19.337170 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 63493a20558fcad00292e7b86a84d35f for removal 2024.10.22 02:00:19.344894 [ 1476 ] {} CacheMetadata: Key 63493a20558fcad00292e7b86a84d35f is removed from metadata 2024.10.22 02:00:19.347107 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3846c5cf04659f5f55a97f637624e5a0, offset: 0, size: 0 2024.10.22 02:00:19.348878 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3846c5cf04659f5f55a97f637624e5a0 for removal 2024.10.22 02:00:19.355302 [ 1476 ] {} CacheMetadata: Key 3846c5cf04659f5f55a97f637624e5a0 is removed from metadata 2024.10.22 02:00:19.555791 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 13.86 MiB, will set to 1.33 GiB (RSS), difference: 7.31 MiB 2024.10.22 02:00:20.244858 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(02394372c8b0540d1fe532ea0a3033db) : [0, 158]: Setting download as failed: File segment: [0, 158], key: 02394372c8b0540d1fe532ea0a3033db, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:20.249670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(02394372c8b0540d1fe532ea0a3033db): No space left in cache to reserve 159 bytes, will continue without cache download 2024.10.22 02:00:20.372282 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_712_136} MergeTask::MergeProjectionsStage: Merge sorted 1085135 rows, containing 4 columns (4 merged, 0 gathered) in 35.192870986 sec., 30833.943625448323 rows/sec., 462.43 KiB/sec. 2024.10.22 02:00:20.671470 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 113 entries to flush up to offset 82476 2024.10.22 02:00:20.682491 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_712_136} MergedBlockOutputStream: filled checksums 202410_1_712_136 (state Temporary) 2024.10.22 02:00:20.987441 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:20.992985 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:21.005427 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 82938 2024.10.22 02:00:21.135356 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:00:21.139566 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:00:21.158096 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_732_732_0 (state Temporary) 2024.10.22 02:00:21.252503 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_732_732_0 to 202410_738_738_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:21.271442 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 11350 2024.10.22 02:00:21.272535 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:21.291403 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e457eec26201bcdb39aaa3458c911b0a) : [0, 10222]: Setting download as failed: File segment: [0, 10222], key: e457eec26201bcdb39aaa3458c911b0a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:21.294619 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e457eec26201bcdb39aaa3458c911b0a): No space left in cache to reserve 10223 bytes, will continue without cache download 2024.10.22 02:00:21.311846 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e457eec26201bcdb39aaa3458c911b0a, offset: 0, size: 0 2024.10.22 02:00:21.314511 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e457eec26201bcdb39aaa3458c911b0a for removal 2024.10.22 02:00:21.315827 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 82476 2024.10.22 02:00:21.322713 [ 1476 ] {} CacheMetadata: Key e457eec26201bcdb39aaa3458c911b0a is removed from metadata 2024.10.22 02:00:21.323963 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 02394372c8b0540d1fe532ea0a3033db, offset: 0, size: 0 2024.10.22 02:00:21.327540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 02394372c8b0540d1fe532ea0a3033db for removal 2024.10.22 02:00:21.335000 [ 1476 ] {} CacheMetadata: Key 02394372c8b0540d1fe532ea0a3033db is removed from metadata 2024.10.22 02:00:21.370771 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 11.43 MiB, will set to 1.33 GiB (RSS), difference: 5.13 MiB 2024.10.22 02:00:21.371285 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:21.375991 [ 1220 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_733_79 to 202410_734_734_0 2024.10.22 02:00:21.381698 [ 1220 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.35 MiB using storage policy from min volume index 0 2024.10.22 02:00:21.392041 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 02:00:21.476239 [ 1220 ] {} DiskLocal: Reserved 1.35 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:21.489041 [ 1220 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 02:00:21.525832 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:21.536057 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:21.541335 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_734_80} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_733_79 to 202410_734_734_0 into Compact with storage Full 2024.10.22 02:00:21.567856 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_745_745_0 (state Temporary) 2024.10.22 02:00:21.597400 [ 1247 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 210 ms. 2024.10.22 02:00:21.650386 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1256_1256_0 (state Temporary) 2024.10.22 02:00:21.688399 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_734_80} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:21.694764 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1256_1256_0 to 202410_1269_1269_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:21.721977 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_734_80} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_733_79, total 22841 rows starting from the beginning of the part 2024.10.22 02:00:21.776069 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 82938 2024.10.22 02:00:21.930363 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_745_745_0 to 202410_758_758_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:22.017590 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_734_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_734_734_0, total 112 rows starting from the beginning of the part 2024.10.22 02:00:22.079411 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11350 2024.10.22 02:00:22.465584 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b67d1791e384cd0f53aefe8c57c9e5c2) : [0, 163]: Setting download as failed: File segment: [0, 163], key: b67d1791e384cd0f53aefe8c57c9e5c2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:22.466928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b67d1791e384cd0f53aefe8c57c9e5c2): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 02:00:22.862339 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0, 202410_663_663_0, 202410_664_664_0] 2024.10.22 02:00:23.076913 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0, 202410_663_663_0, 202410_664_664_0] 2024.10.22 02:00:23.116322 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0, 202410_663_663_0, 202410_664_664_0] 2024.10.22 02:00:23.342501 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_712_136/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:00:23.353613 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9c131fb4f5f41019ab753ee4785e804e) : [0, 150405]: Setting download as failed: File segment: [0, 150405], key: 9c131fb4f5f41019ab753ee4785e804e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:23.355691 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9c131fb4f5f41019ab753ee4785e804e): No space left in cache to reserve 150406 bytes, will continue without cache download 2024.10.22 02:00:23.372630 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9c131fb4f5f41019ab753ee4785e804e, offset: 0, size: 0 2024.10.22 02:00:23.374973 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9c131fb4f5f41019ab753ee4785e804e for removal 2024.10.22 02:00:23.379246 [ 1476 ] {} CacheMetadata: Key 9c131fb4f5f41019ab753ee4785e804e is removed from metadata 2024.10.22 02:00:23.380975 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b67d1791e384cd0f53aefe8c57c9e5c2, offset: 0, size: 0 2024.10.22 02:00:23.383480 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b67d1791e384cd0f53aefe8c57c9e5c2 for removal 2024.10.22 02:00:23.401015 [ 1476 ] {} CacheMetadata: Key b67d1791e384cd0f53aefe8c57c9e5c2 is removed from metadata 2024.10.22 02:00:23.766882 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_734_80} MergeTask::MergeProjectionsStage: Merge sorted 22953 rows, containing 13 columns (13 merged, 0 gathered) in 2.244499856 sec., 10226.331687499114 rows/sec., 4.43 MiB/sec. 2024.10.22 02:00:24.371248 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_734_80} MergedBlockOutputStream: filled checksums 202410_507_734_80 (state Temporary) 2024.10.22 02:00:24.660799 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a11c7d8f8dd32dc4aeb20538f22a516c) : [0, 162]: Setting download as failed: File segment: [0, 162], key: a11c7d8f8dd32dc4aeb20538f22a516c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:24.662646 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a11c7d8f8dd32dc4aeb20538f22a516c): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 02:00:24.569549 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_712_136} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_712_136 to 202410_1_712_136 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:24.665405 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_734_80} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_734_80 to 202410_507_734_80 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:24.681072 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_734_80} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_733_79, 202410_734_734_0] -> 202410_507_734_80 2024.10.22 02:00:24.697397 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_712_136} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_706_135, 202410_707_712_1] -> 202410_1_712_136 2024.10.22 02:00:24.705053 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 6.70 MiB, will set to 1.33 GiB (RSS), difference: 2.76 MiB 2024.10.22 02:00:24.723154 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:00:24.822501 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.50 MiB. 2024.10.22 02:00:25.265144 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1084250 2024.10.22 02:00:25.530136 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2e31c4d82d6773d7f7070b4fbd34c71e) : [0, 170003]: Setting download as failed: File segment: [0, 170003], key: 2e31c4d82d6773d7f7070b4fbd34c71e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:25.533385 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2e31c4d82d6773d7f7070b4fbd34c71e): No space left in cache to reserve 170004 bytes, will continue without cache download 2024.10.22 02:00:25.561623 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2e31c4d82d6773d7f7070b4fbd34c71e, offset: 0, size: 0 2024.10.22 02:00:25.563748 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2e31c4d82d6773d7f7070b4fbd34c71e for removal 2024.10.22 02:00:25.565431 [ 1476 ] {} CacheMetadata: Key 2e31c4d82d6773d7f7070b4fbd34c71e is removed from metadata 2024.10.22 02:00:25.566162 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a11c7d8f8dd32dc4aeb20538f22a516c, offset: 0, size: 0 2024.10.22 02:00:25.567068 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a11c7d8f8dd32dc4aeb20538f22a516c for removal 2024.10.22 02:00:25.568547 [ 1476 ] {} CacheMetadata: Key a11c7d8f8dd32dc4aeb20538f22a516c is removed from metadata 2024.10.22 02:00:25.784452 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 83008 2024.10.22 02:00:26.024602 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:26.030348 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:26.053587 [ 1198 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 284 ms. 2024.10.22 02:00:26.073520 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 621 2024.10.22 02:00:26.149546 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 5 old parts to remove. Parts: [202410_1_694_140, 202410_695_695_0, 202410_696_696_0, 202410_697_697_0, 202410_698_698_0] 2024.10.22 02:00:26.153291 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 5 parts from filesystem (serially): Parts: [202410_1_694_140, 202410_695_695_0, 202410_696_696_0, 202410_697_697_0, 202410_698_698_0] 2024.10.22 02:00:26.175843 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 5 parts from memory: Parts: [202410_1_694_140, 202410_695_695_0, 202410_696_696_0, 202410_697_697_0, 202410_698_698_0] 2024.10.22 02:00:26.370957 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1257_1257_0 (state Temporary) 2024.10.22 02:00:26.543793 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 6.36 MiB, will set to 1.32 GiB (RSS), difference: -7.88 MiB 2024.10.22 02:00:26.637288 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6afc17808e2fe709c283bf7586d48fd3) : [0, 191]: Setting download as failed: File segment: [0, 191], key: 6afc17808e2fe709c283bf7586d48fd3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:26.642551 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6afc17808e2fe709c283bf7586d48fd3): No space left in cache to reserve 192 bytes, will continue without cache download 2024.10.22 02:00:26.685633 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:26.695197 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:26.768905 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1257_1257_0 to 202410_1270_1270_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:26.788657 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:26.794940 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:26.848020 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83008 2024.10.22 02:00:26.856018 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_603_603_0 (state Temporary) 2024.10.22 02:00:26.921120 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_714_714_0 (state Temporary) 2024.10.22 02:00:27.014702 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_714_714_0 to 202410_721_721_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:27.137699 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_603_603_0 to 202410_613_613_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:27.158453 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1084250 2024.10.22 02:00:27.164877 [ 1250 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_608_124 to 202410_613_613_0 2024.10.22 02:00:27.173267 [ 1250 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:27.180625 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:27.202950 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_608_124 to 202410_613_613_0 into Compact with storage Full 2024.10.22 02:00:27.212897 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 621 2024.10.22 02:00:27.366648 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:27.475825 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a29b8353fa636250426855ee20fadf4a) : [0, 260109]: Setting download as failed: File segment: [0, 260109], key: a29b8353fa636250426855ee20fadf4a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:27.479521 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a29b8353fa636250426855ee20fadf4a): No space left in cache to reserve 260110 bytes, will continue without cache download 2024.10.22 02:00:27.514630 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_608_124, total 652 rows starting from the beginning of the part 2024.10.22 02:00:27.504877 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a29b8353fa636250426855ee20fadf4a, offset: 0, size: 0 2024.10.22 02:00:27.528578 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a29b8353fa636250426855ee20fadf4a for removal 2024.10.22 02:00:27.534940 [ 1476 ] {} CacheMetadata: Key a29b8353fa636250426855ee20fadf4a is removed from metadata 2024.10.22 02:00:27.539240 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6afc17808e2fe709c283bf7586d48fd3, offset: 0, size: 0 2024.10.22 02:00:27.542112 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6afc17808e2fe709c283bf7586d48fd3 for removal 2024.10.22 02:00:27.544554 [ 1476 ] {} CacheMetadata: Key 6afc17808e2fe709c283bf7586d48fd3 is removed from metadata 2024.10.22 02:00:27.680302 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_609_609_0, total 1 rows starting from the beginning of the part 2024.10.22 02:00:27.997672 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_610_610_0, total 1 rows starting from the beginning of the part 2024.10.22 02:00:28.261683 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_611_611_0, total 1 rows starting from the beginning of the part 2024.10.22 02:00:28.471293 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 5.48 MiB, will set to 1.32 GiB (RSS), difference: 1.89 MiB 2024.10.22 02:00:28.473779 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(9a3d10feb355bd85fc48ce003ef2a8ea) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9a3d10feb355bd85fc48ce003ef2a8ea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:00:28.476448 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 598170, file segment info: File segment: [0, 33554431], key: 9a3d10feb355bd85fc48ce003ef2a8ea, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:00:28.481711 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:28.668127 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_612_612_0, total 1 rows starting from the beginning of the part 2024.10.22 02:00:28.840263 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 82547 2024.10.22 02:00:28.865441 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_613_613_0, total 1 rows starting from the beginning of the part 2024.10.22 02:00:28.977224 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:28.984648 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:29.041606 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_733_733_0 (state Temporary) 2024.10.22 02:00:29.096740 [ 1226 ] {} DatabaseCatalog: Execution took 253 ms. 2024.10.22 02:00:29.191112 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_733_733_0 to 202410_739_739_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:29.228138 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_734_80 to 202410_735_735_0 2024.10.22 02:00:29.231906 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.35 MiB using storage policy from min volume index 0 2024.10.22 02:00:29.233685 [ 1257 ] {} DiskLocal: Reserved 1.35 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:29.252953 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_735_81} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_734_80 to 202410_735_735_0 into Compact with storage Full 2024.10.22 02:00:29.315484 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_735_81} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:29.332686 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_735_81} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_734_80, total 22953 rows starting from the beginning of the part 2024.10.22 02:00:29.397388 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_735_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_735_735_0, total 111 rows starting from the beginning of the part 2024.10.22 02:00:29.447196 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 82547 2024.10.22 02:00:29.583419 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 11362 2024.10.22 02:00:29.780327 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:29.788120 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:29.725835 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d11da4198d6d8f0e3f84168fc420edac) : [0, 190]: Setting download as failed: File segment: [0, 190], key: d11da4198d6d8f0e3f84168fc420edac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:29.934197 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d11da4198d6d8f0e3f84168fc420edac): No space left in cache to reserve 191 bytes, will continue without cache download 2024.10.22 02:00:29.948370 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_746_746_0 (state Temporary) 2024.10.22 02:00:29.998659 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_746_746_0 to 202410_759_759_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:30.365607 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11362 2024.10.22 02:00:30.491485 [ 1241 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 02:00:30.648648 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1166_14, 202410_1167_1167_0, 202410_1168_1168_0, 202410_1169_1169_0, 202410_1170_1170_0, 202410_1171_1171_0] 2024.10.22 02:00:30.651644 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1166_14, 202410_1167_1167_0, 202410_1168_1168_0, 202410_1169_1169_0, 202410_1170_1170_0, 202410_1171_1171_0] 2024.10.22 02:00:30.680214 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1166_14, 202410_1167_1167_0, 202410_1168_1168_0, 202410_1169_1169_0, 202410_1170_1170_0, 202410_1171_1171_0] 2024.10.22 02:00:30.717464 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(ea6542d0a3840e2a14494812f8125029) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ea6542d0a3840e2a14494812f8125029, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:00:30.725026 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 521, file segment info: File segment: [0, 33554431], key: ea6542d0a3840e2a14494812f8125029, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:00:30.728603 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:30.771858 [ 1690 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wev/bxellflrfinshsgghddyrjlzyfjax, size 598170 2024.10.22 02:00:30.862700 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 83085 2024.10.22 02:00:30.946969 [ 1262 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 02:00:30.969607 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_735_81} MergeTask::MergeProjectionsStage: Merge sorted 23064 rows, containing 13 columns (13 merged, 0 gathered) in 1.719819687 sec., 13410.70821222667 rows/sec., 5.81 MiB/sec. 2024.10.22 02:00:31.040053 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:31.045023 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:31.061333 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_735_81} MergedBlockOutputStream: filled checksums 202410_507_735_81 (state Temporary) 2024.10.22 02:00:31.113814 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1258_1258_0 (state Temporary) 2024.10.22 02:00:31.134531 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:00:31.138158 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:00:31.162435 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_735_81} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_735_81 to 202410_507_735_81 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:31.164794 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_735_81} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_734_80, 202410_735_735_0] -> 202410_507_735_81 2024.10.22 02:00:31.183311 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.50 MiB. 2024.10.22 02:00:31.253645 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1258_1258_0 to 202410_1271_1271_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:31.304185 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83085 2024.10.22 02:00:31.316816 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:00:31.356121 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:00:31.373553 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:31.377718 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:31.387165 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:00:31.393424 [ 1223 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:00:31.400380 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} MergeTask::MergeProjectionsStage: Merge sorted 657 rows, containing 32 columns (32 merged, 0 gathered) in 4.206634668 sec., 156.18185363179248 rows/sec., 14.20 KiB/sec. 2024.10.22 02:00:31.402459 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:00:31.390772 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(287e3ac0203014b87d4a231a4390035f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 287e3ac0203014b87d4a231a4390035f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:31.411853 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 67224, file segment info: File segment: [0, 33554431], key: 287e3ac0203014b87d4a231a4390035f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:31.418159 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:31.421054 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:00:31.439553 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} MergedBlockOutputStream: filled checksums 202410_1_613_125 (state Temporary) 2024.10.22 02:00:31.540912 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:00:31.547982 [ 1223 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:00:31.759621 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:00:31.790273 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_613_125 to 202410_1_613_125 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:31.799487 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_613_125} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_608_124, 202410_613_613_0] -> 202410_1_613_125 2024.10.22 02:00:31.854973 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:00:31.924487 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/otb/czmwimxnjevpgfmakxklrpgiolxou (LogSeriesLimiter: on interval from 2024-10-22 02:00:11 to 2024-10-22 02:00:31 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:00:32.135417 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(63731221b3f36b0bf29e6477cc0f6e5a) : [0, 258326]: Setting download as failed: File segment: [0, 258326], key: 63731221b3f36b0bf29e6477cc0f6e5a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:32.137355 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(63731221b3f36b0bf29e6477cc0f6e5a): No space left in cache to reserve 258327 bytes, will continue without cache download 2024.10.22 02:00:32.264937 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 63731221b3f36b0bf29e6477cc0f6e5a, offset: 0, size: 0 2024.10.22 02:00:32.276375 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 63731221b3f36b0bf29e6477cc0f6e5a for removal 2024.10.22 02:00:32.284334 [ 1476 ] {} CacheMetadata: Key 63731221b3f36b0bf29e6477cc0f6e5a is removed from metadata 2024.10.22 02:00:32.285673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d11da4198d6d8f0e3f84168fc420edac, offset: 0, size: 0 2024.10.22 02:00:32.291830 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d11da4198d6d8f0e3f84168fc420edac for removal 2024.10.22 02:00:32.295581 [ 1476 ] {} CacheMetadata: Key d11da4198d6d8f0e3f84168fc420edac is removed from metadata 2024.10.22 02:00:32.715324 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 4.30 MiB, will set to 1.32 GiB (RSS), difference: -4.18 MiB 2024.10.22 02:00:32.883064 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(3376c4421f3130dee889637d34e76ed9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3376c4421f3130dee889637d34e76ed9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:32.899670 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 687, file segment info: File segment: [0, 33554431], key: 3376c4421f3130dee889637d34e76ed9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:32.905212 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:33.916414 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(10350b10607a9bd88d332c1455df4e22) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 10350b10607a9bd88d332c1455df4e22, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:33.921678 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(10350b10607a9bd88d332c1455df4e22): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 02:00:34.164668 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1200 entries to flush up to offset 1085450 2024.10.22 02:00:34.315711 [ 1237 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 02:00:34.716787 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 622 2024.10.22 02:00:34.872533 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:34.880691 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:35.000059 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:35.000332 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_715_715_0 (state Temporary) 2024.10.22 02:00:35.002921 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:35.048680 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(ed97b2210e88c9ca922d67830321ede7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ed97b2210e88c9ca922d67830321ede7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:35.051940 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 31880, file segment info: File segment: [0, 33554431], key: ed97b2210e88c9ca922d67830321ede7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:35.057446 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:35.240991 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_715_715_0 to 202410_722_722_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:35.271627 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1085450 2024.10.22 02:00:35.283579 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_604_604_0 (state Temporary) 2024.10.22 02:00:35.270017 [ 1188 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_712_136 to 202410_713_718_1 2024.10.22 02:00:35.302352 [ 1188 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:35.308717 [ 1188 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:35.312002 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 83149 2024.10.22 02:00:35.360351 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_150_150_0 (state Temporary) 2024.10.22 02:00:35.474918 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_718_137} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_712_136 to 202410_713_718_1 into Wide with storage Full 2024.10.22 02:00:35.623782 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_604_604_0 to 202410_614_614_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:35.631910 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_718_137} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:35.639752 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_718_137} MergeTreeSequentialSource: Reading 134 marks from part 202410_1_712_136, total 1085135 rows starting from the beginning of the part 2024.10.22 02:00:35.670218 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_718_137} MergeTreeSequentialSource: Reading 3 marks from part 202410_713_718_1, total 9240 rows starting from the beginning of the part 2024.10.22 02:00:35.701000 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 622 2024.10.22 02:00:35.729763 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:35.733816 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:35.765004 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 3.91 MiB, will set to 1.32 GiB (RSS), difference: -19.61 MiB 2024.10.22 02:00:35.886465 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1259_1259_0 (state Temporary) 2024.10.22 02:00:35.964476 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(14d498a4519adf033e5f2db25f2a0171) : [0, 123821]: Setting download as failed: File segment: [0, 123821], key: 14d498a4519adf033e5f2db25f2a0171, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:35.966598 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(14d498a4519adf033e5f2db25f2a0171): No space left in cache to reserve 123822 bytes, will continue without cache download 2024.10.22 02:00:36.009171 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 14d498a4519adf033e5f2db25f2a0171, offset: 0, size: 0 2024.10.22 02:00:36.012669 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 14d498a4519adf033e5f2db25f2a0171 for removal 2024.10.22 02:00:36.017110 [ 1476 ] {} CacheMetadata: Key 14d498a4519adf033e5f2db25f2a0171 is removed from metadata 2024.10.22 02:00:36.022755 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 10350b10607a9bd88d332c1455df4e22, offset: 0, size: 0 2024.10.22 02:00:36.027006 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 10350b10607a9bd88d332c1455df4e22 for removal 2024.10.22 02:00:36.030925 [ 1476 ] {} CacheMetadata: Key 10350b10607a9bd88d332c1455df4e22 is removed from metadata 2024.10.22 02:00:36.030856 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1259_1259_0 to 202410_1272_1272_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:36.064092 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1267_12 to 202410_1272_1272_0 2024.10.22 02:00:36.081664 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:36.085695 [ 1171 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:36.106223 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83149 2024.10.22 02:00:36.249262 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1267_12 to 202410_1272_1272_0 into Compact with storage Full 2024.10.22 02:00:36.345409 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:36.354747 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1267_12, total 3660 rows starting from the beginning of the part 2024.10.22 02:00:36.376192 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1268_1268_0, total 82 rows starting from the beginning of the part 2024.10.22 02:00:36.468116 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1269_1269_0, total 55 rows starting from the beginning of the part 2024.10.22 02:00:36.671416 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1270_1270_0, total 70 rows starting from the beginning of the part 2024.10.22 02:00:36.696229 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(c339e32872a077820cb26a6c0e9e544e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c339e32872a077820cb26a6c0e9e544e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:36.701435 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 620, file segment info: File segment: [0, 33554431], key: c339e32872a077820cb26a6c0e9e544e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:00:36.706324 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:36.714497 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:00:36.759250 [ 1708 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/baq/gvgkqkhmtgmbtyncgcudvwaegawmt, size 31880 2024.10.22 02:00:36.864292 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1271_1271_0, total 77 rows starting from the beginning of the part 2024.10.22 02:00:36.939652 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:00:36.954360 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1272_1272_0, total 64 rows starting from the beginning of the part 2024.10.22 02:00:36.966704 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 82662 2024.10.22 02:00:36.974420 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:00:37.004004 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:00:37.044231 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:00:37.198870 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:00:37.245537 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:00:37.273941 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:00:37.315909 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:00:37.567629 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:37.570316 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:00:37.634295 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qpy/oirdylbiomlsoqlytlqqlosufvtzx (LogSeriesLimiter: on interval from 2024-10-22 02:00:31 to 2024-10-22 02:00:37 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:00:37.650540 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_734_734_0 (state Temporary) 2024.10.22 02:00:37.702796 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(61931e1cb7f631904d011a8d5494f4cf) : [0, 159]: Setting download as failed: File segment: [0, 159], key: 61931e1cb7f631904d011a8d5494f4cf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:37.705558 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(61931e1cb7f631904d011a8d5494f4cf): No space left in cache to reserve 160 bytes, will continue without cache download 2024.10.22 02:00:37.867678 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 11377 2024.10.22 02:00:37.945626 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_734_734_0 to 202410_740_740_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:37.975037 [ 1277 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_735_81 to 202410_736_736_0 2024.10.22 02:00:37.980412 [ 1277 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.36 MiB using storage policy from min volume index 0 2024.10.22 02:00:37.983627 [ 1277 ] {} DiskLocal: Reserved 1.36 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:00:38.007507 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_736_82} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_735_81 to 202410_736_736_0 into Compact with storage Full 2024.10.22 02:00:38.009501 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 82662 2024.10.22 02:00:38.060561 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_736_82} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:38.070678 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_736_82} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_735_81, total 23064 rows starting from the beginning of the part 2024.10.22 02:00:38.168982 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:38.181702 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:00:38.238607 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_747_747_0 (state Temporary) 2024.10.22 02:00:38.252971 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_736_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_736_736_0, total 88 rows starting from the beginning of the part 2024.10.22 02:00:38.275400 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_747_747_0 to 202410_760_760_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:38.458817 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11377 2024.10.22 02:00:38.503080 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 7.26 MiB, will set to 1.32 GiB (RSS), difference: -3.23 MiB 2024.10.22 02:00:38.759374 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9697793edcea10e3df690e89e9b00c73) : [0, 116776]: Setting download as failed: File segment: [0, 116776], key: 9697793edcea10e3df690e89e9b00c73, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:38.760853 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9697793edcea10e3df690e89e9b00c73): No space left in cache to reserve 116777 bytes, will continue without cache download 2024.10.22 02:00:38.773793 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9697793edcea10e3df690e89e9b00c73, offset: 0, size: 0 2024.10.22 02:00:38.795898 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9697793edcea10e3df690e89e9b00c73 for removal 2024.10.22 02:00:38.798135 [ 1476 ] {} CacheMetadata: Key 9697793edcea10e3df690e89e9b00c73 is removed from metadata 2024.10.22 02:00:38.800389 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 61931e1cb7f631904d011a8d5494f4cf, offset: 0, size: 0 2024.10.22 02:00:38.803077 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 61931e1cb7f631904d011a8d5494f4cf for removal 2024.10.22 02:00:38.810509 [ 1476 ] {} CacheMetadata: Key 61931e1cb7f631904d011a8d5494f4cf is removed from metadata 2024.10.22 02:00:38.817316 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} MergeTask::MergeProjectionsStage: Merge sorted 4008 rows, containing 13 columns (13 merged, 0 gathered) in 2.574844606 sec., 1556.5987907232955 rows/sec., 266.78 KiB/sec. 2024.10.22 02:00:38.981413 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} MergedBlockOutputStream: filled checksums 202410_1207_1272_13 (state Temporary) 2024.10.22 02:00:39.175098 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1272_13 to 202410_1207_1272_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:39.181274 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1272_13} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1267_12, 202410_1272_1272_0] -> 202410_1207_1272_13 2024.10.22 02:00:39.243602 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:00:39.351370 [ 1202 ] {} DatabaseCatalog: Execution took 242 ms. 2024.10.22 02:00:39.791045 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_736_82} MergeTask::MergeProjectionsStage: Merge sorted 23152 rows, containing 13 columns (13 merged, 0 gathered) in 1.78893705 sec., 12941.763378426313 rows/sec., 5.61 MiB/sec. 2024.10.22 02:00:39.905804 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e8e11d3b16f581c71c9b3f5133b0aaec) : [0, 160]: Setting download as failed: File segment: [0, 160], key: e8e11d3b16f581c71c9b3f5133b0aaec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:39.908566 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e8e11d3b16f581c71c9b3f5133b0aaec): No space left in cache to reserve 161 bytes, will continue without cache download 2024.10.22 02:00:40.071022 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_736_82} MergedBlockOutputStream: filled checksums 202410_507_736_82 (state Temporary) 2024.10.22 02:00:40.124208 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 83237 2024.10.22 02:00:40.217069 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_736_82} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_736_82 to 202410_507_736_82 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:40.228442 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_736_82} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_735_81, 202410_736_736_0] -> 202410_507_736_82 2024.10.22 02:00:40.257765 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.47 MiB. 2024.10.22 02:00:40.466851 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 11.47 MiB, will set to 1.33 GiB (RSS), difference: 11.32 MiB 2024.10.22 02:00:40.637807 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:40.643354 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:00:40.728435 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1260_1260_0 (state Temporary) 2024.10.22 02:00:40.744054 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(09ab767797e960504c717518dc966d07) : [0, 71179]: Setting download as failed: File segment: [0, 71179], key: 09ab767797e960504c717518dc966d07, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:40.745348 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(09ab767797e960504c717518dc966d07): No space left in cache to reserve 71180 bytes, will continue without cache download 2024.10.22 02:00:40.763827 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 09ab767797e960504c717518dc966d07, offset: 0, size: 0 2024.10.22 02:00:40.766270 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 09ab767797e960504c717518dc966d07 for removal 2024.10.22 02:00:40.767830 [ 1476 ] {} CacheMetadata: Key 09ab767797e960504c717518dc966d07 is removed from metadata 2024.10.22 02:00:40.768564 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e8e11d3b16f581c71c9b3f5133b0aaec, offset: 0, size: 0 2024.10.22 02:00:40.771119 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e8e11d3b16f581c71c9b3f5133b0aaec for removal 2024.10.22 02:00:40.775060 [ 1476 ] {} CacheMetadata: Key e8e11d3b16f581c71c9b3f5133b0aaec is removed from metadata 2024.10.22 02:00:40.893606 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1260_1260_0 to 202410_1273_1273_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:40.962650 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83237 2024.10.22 02:00:41.142090 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:00:41.145430 [ 1185 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 02:00:41.157226 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:00:41.272719 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_718_137/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:00:41.402541 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_652_126, 202410_653_658_1] 2024.10.22 02:00:41.407815 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_652_126, 202410_653_658_1] 2024.10.22 02:00:41.441409 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_652_126, 202410_653_658_1] 2024.10.22 02:00:41.980250 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3fd4f097655ea8887dcf804c579aa5d7) : [0, 191]: Setting download as failed: File segment: [0, 191], key: 3fd4f097655ea8887dcf804c579aa5d7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:41.985675 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3fd4f097655ea8887dcf804c579aa5d7): No space left in cache to reserve 192 bytes, will continue without cache download 2024.10.22 02:00:42.276222 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1495 entries to flush up to offset 1086945 2024.10.22 02:00:42.521596 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 10.67 MiB, will set to 1.33 GiB (RSS), difference: -4.67 MiB 2024.10.22 02:00:42.762652 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(984d725322fceb3b253c4436586f9744) : [0, 253672]: Setting download as failed: File segment: [0, 253672], key: 984d725322fceb3b253c4436586f9744, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:42.763719 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(984d725322fceb3b253c4436586f9744): No space left in cache to reserve 253673 bytes, will continue without cache download 2024.10.22 02:00:42.792650 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 984d725322fceb3b253c4436586f9744, offset: 0, size: 0 2024.10.22 02:00:42.809246 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 984d725322fceb3b253c4436586f9744 for removal 2024.10.22 02:00:42.816043 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3fd4f097655ea8887dcf804c579aa5d7, offset: 0, size: 0 2024.10.22 02:00:42.813827 [ 1476 ] {} CacheMetadata: Key 984d725322fceb3b253c4436586f9744 is removed from metadata 2024.10.22 02:00:42.828589 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3fd4f097655ea8887dcf804c579aa5d7 for removal 2024.10.22 02:00:42.833722 [ 1476 ] {} CacheMetadata: Key 3fd4f097655ea8887dcf804c579aa5d7 is removed from metadata 2024.10.22 02:00:43.008830 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.088473427 sec. 2024.10.22 02:00:43.049684 [ 1247 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 02:00:43.207208 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 623 2024.10.22 02:00:43.260463 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:43.264669 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:43.462120 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_150_150_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:00:43.548611 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_716_716_0 (state Temporary) 2024.10.22 02:00:43.567975 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(81b4a28d625f348783a5f348d762949b) : [0, 158]: Setting download as failed: File segment: [0, 158], key: 81b4a28d625f348783a5f348d762949b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:43.569281 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(81b4a28d625f348783a5f348d762949b): No space left in cache to reserve 159 bytes, will continue without cache download 2024.10.22 02:00:43.605007 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:43.611409 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:43.613801 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_716_716_0 to 202410_723_723_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:43.653385 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1086945 2024.10.22 02:00:43.672671 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_605_605_0 (state Temporary) 2024.10.22 02:00:43.980417 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_605_605_0 to 202410_615_615_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:44.308999 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(433200db65e60a82be87f890e56ffea3) : [0, 4722]: Setting download as failed: File segment: [0, 4722], key: 433200db65e60a82be87f890e56ffea3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:44.313032 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(433200db65e60a82be87f890e56ffea3): No space left in cache to reserve 4723 bytes, will continue without cache download 2024.10.22 02:00:44.347651 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 433200db65e60a82be87f890e56ffea3, offset: 0, size: 0 2024.10.22 02:00:44.349606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 433200db65e60a82be87f890e56ffea3 for removal 2024.10.22 02:00:44.353076 [ 1476 ] {} CacheMetadata: Key 433200db65e60a82be87f890e56ffea3 is removed from metadata 2024.10.22 02:00:44.353425 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 81b4a28d625f348783a5f348d762949b, offset: 0, size: 0 2024.10.22 02:00:44.354637 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 81b4a28d625f348783a5f348d762949b for removal 2024.10.22 02:00:44.356864 [ 1476 ] {} CacheMetadata: Key 81b4a28d625f348783a5f348d762949b is removed from metadata 2024.10.22 02:00:44.386428 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 623 2024.10.22 02:00:44.817231 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 11.00 MiB, will set to 1.33 GiB (RSS), difference: 1.90 MiB 2024.10.22 02:00:44.973359 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 83299 2024.10.22 02:00:45.123254 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:45.129839 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:45.191249 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1261_1261_0 (state Temporary) 2024.10.22 02:00:45.355685 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e14af6b2e9d261cbc5354dc45ee16244) : [0, 159]: Setting download as failed: File segment: [0, 159], key: e14af6b2e9d261cbc5354dc45ee16244, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:45.367806 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e14af6b2e9d261cbc5354dc45ee16244): No space left in cache to reserve 160 bytes, will continue without cache download 2024.10.22 02:00:45.455060 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1261_1261_0 to 202410_1274_1274_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:45.576549 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 89 entries to flush up to offset 82751 2024.10.22 02:00:45.630406 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83299 2024.10.22 02:00:45.960575 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 11393 2024.10.22 02:00:46.043849 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:46.142490 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:46.146736 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:46.148398 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:00:46.273121 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_735_735_0 (state Temporary) 2024.10.22 02:00:46.317147 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_735_735_0 to 202410_741_741_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:46.334544 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0120be485969b64ed9b30d429b12cfc1) : [0, 5153]: Setting download as failed: File segment: [0, 5153], key: 0120be485969b64ed9b30d429b12cfc1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:46.339406 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0120be485969b64ed9b30d429b12cfc1): No space left in cache to reserve 5154 bytes, will continue without cache download 2024.10.22 02:00:46.364550 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_748_748_0 (state Temporary) 2024.10.22 02:00:46.366966 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_736_82 to 202410_737_737_0 2024.10.22 02:00:46.358407 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0120be485969b64ed9b30d429b12cfc1, offset: 0, size: 0 2024.10.22 02:00:46.384455 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0120be485969b64ed9b30d429b12cfc1 for removal 2024.10.22 02:00:46.392107 [ 1476 ] {} CacheMetadata: Key 0120be485969b64ed9b30d429b12cfc1 is removed from metadata 2024.10.22 02:00:46.397286 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e14af6b2e9d261cbc5354dc45ee16244, offset: 0, size: 0 2024.10.22 02:00:46.400689 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 82751 2024.10.22 02:00:46.400773 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e14af6b2e9d261cbc5354dc45ee16244 for removal 2024.10.22 02:00:46.408279 [ 1476 ] {} CacheMetadata: Key e14af6b2e9d261cbc5354dc45ee16244 is removed from metadata 2024.10.22 02:00:46.451613 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_748_748_0 to 202410_761_761_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:46.486702 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11393 2024.10.22 02:00:46.516646 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.36 MiB using storage policy from min volume index 0 2024.10.22 02:00:46.521002 [ 1254 ] {} DiskLocal: Reserved 1.36 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:46.544880 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 3 parts from 202410_1_756_158 to 202410_758_758_0 2024.10.22 02:00:46.554579 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:46.555278 [ 1230 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:00:46.556910 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:00:46.564645 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_737_83} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_736_82 to 202410_737_737_0 into Compact with storage Full 2024.10.22 02:00:46.621378 [ 1230 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:00:46.623164 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_737_83} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:46.631893 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_737_83} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_736_82, total 23152 rows starting from the beginning of the part 2024.10.22 02:00:46.636277 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_758_159} MergeTask::PrepareStage: Merging 3 parts: from 202410_1_756_158 to 202410_758_758_0 into Compact with storage Full 2024.10.22 02:00:46.665387 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_737_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_737_737_0, total 86 rows starting from the beginning of the part 2024.10.22 02:00:46.749587 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_758_159} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:46.763461 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_758_159} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_756_158, total 11427 rows starting from the beginning of the part 2024.10.22 02:00:46.820953 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_758_159} MergeTreeSequentialSource: Reading 2 marks from part 202410_757_757_0, total 7 rows starting from the beginning of the part 2024.10.22 02:00:46.878116 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_758_159} MergeTreeSequentialSource: Reading 2 marks from part 202410_758_758_0, total 22 rows starting from the beginning of the part 2024.10.22 02:00:47.463522 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 10.54 MiB, will set to 1.33 GiB (RSS), difference: -21.91 MiB 2024.10.22 02:00:47.591236 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(32a5d1d4efc1ce01fa42d867d664c32b) : [0, 160]: Setting download as failed: File segment: [0, 160], key: 32a5d1d4efc1ce01fa42d867d664c32b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:47.593788 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(32a5d1d4efc1ce01fa42d867d664c32b): No space left in cache to reserve 161 bytes, will continue without cache download 2024.10.22 02:00:47.948405 [ 1268 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 257 ms. 2024.10.22 02:00:48.411443 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_758_159} MergeTask::MergeProjectionsStage: Merge sorted 11456 rows, containing 12 columns (12 merged, 0 gathered) in 1.803614972 sec., 6351.688236041101 rows/sec., 1012.48 KiB/sec. 2024.10.22 02:00:48.465510 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_758_159} MergedBlockOutputStream: filled checksums 202410_1_758_159 (state Temporary) 2024.10.22 02:00:48.508083 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(bce44e061f30882a96101a05bba89b5f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: bce44e061f30882a96101a05bba89b5f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:00:48.520565 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 551240, file segment info: File segment: [0, 33554431], key: bce44e061f30882a96101a05bba89b5f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:00:48.535885 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:48.570605 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_150_150_0 to 202410_153_153_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:48.668146 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_737_83} MergeTask::MergeProjectionsStage: Merge sorted 23238 rows, containing 13 columns (13 merged, 0 gathered) in 2.109860644 sec., 11013.997567130316 rows/sec., 4.77 MiB/sec. 2024.10.22 02:00:48.701711 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 5 old parts to remove. Parts: [202410_135_135_0, 202410_136_136_0, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0] 2024.10.22 02:00:48.707107 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from filesystem (serially): Parts: [202410_135_135_0, 202410_136_136_0, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0] 2024.10.22 02:00:48.742378 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from memory: Parts: [202410_135_135_0, 202410_136_136_0, 202410_137_137_0, 202410_138_138_0, 202410_139_139_0] 2024.10.22 02:00:48.749311 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_758_159} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_758_159 to 202410_1_758_159 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:48.765953 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_758_159} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 3 parts: [202410_1_756_158, 202410_758_758_0] -> 202410_1_758_159 2024.10.22 02:00:48.804319 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.47 MiB. 2024.10.22 02:00:48.961577 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_737_83} MergedBlockOutputStream: filled checksums 202410_507_737_83 (state Temporary) 2024.10.22 02:00:49.367358 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(48a63490085a419f7039729aed19915b) : [0, 63890]: Setting download as failed: File segment: [0, 63890], key: 48a63490085a419f7039729aed19915b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:49.368500 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(48a63490085a419f7039729aed19915b): No space left in cache to reserve 63891 bytes, will continue without cache download 2024.10.22 02:00:49.404562 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_737_83} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_737_83 to 202410_507_737_83 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:49.411793 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_737_83} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_736_82, 202410_737_737_0] -> 202410_507_737_83 2024.10.22 02:00:49.522500 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 48a63490085a419f7039729aed19915b, offset: 0, size: 0 2024.10.22 02:00:49.530942 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 48a63490085a419f7039729aed19915b for removal 2024.10.22 02:00:49.540015 [ 1476 ] {} CacheMetadata: Key 48a63490085a419f7039729aed19915b is removed from metadata 2024.10.22 02:00:49.547670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 32a5d1d4efc1ce01fa42d867d664c32b, offset: 0, size: 0 2024.10.22 02:00:49.549395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 32a5d1d4efc1ce01fa42d867d664c32b for removal 2024.10.22 02:00:49.552295 [ 1476 ] {} CacheMetadata: Key 32a5d1d4efc1ce01fa42d867d664c32b is removed from metadata 2024.10.22 02:00:49.560172 [ 1260 ] {} DatabaseCatalog: Execution took 209 ms. 2024.10.22 02:00:49.636365 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 83373 2024.10.22 02:00:49.652435 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.47 MiB. 2024.10.22 02:00:49.960138 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:49.962312 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:50.043301 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1262_1262_0 (state Temporary) 2024.10.22 02:00:50.085647 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1262_1262_0 to 202410_1275_1275_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:50.344421 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83373 2024.10.22 02:00:50.426541 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(448070d89db741b0f53cb2392c534727) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 448070d89db741b0f53cb2392c534727, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:00:50.431460 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 526, file segment info: File segment: [0, 33554431], key: 448070d89db741b0f53cb2392c534727, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:00:50.534522 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:50.586507 [ 1724 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/otb/czmwimxnjevpgfmakxklrpgiolxou, size 551240 2024.10.22 02:00:50.658743 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1088100 2024.10.22 02:00:50.848812 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:00:50.854329 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:00:50.862381 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:00:50.870715 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:00:50.888023 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:00:50.928529 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:00:50.953134 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:00:50.989174 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qal/rrhrnitmvveqoismwiinprijubdlj (LogSeriesLimiter: on interval from 2024-10-22 02:00:37 to 2024-10-22 02:00:50 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:00:51.138601 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:00:51.243973 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:00:51.580085 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:51.589163 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:00:51.626778 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 2.00 GiB, free memory in arenas 10.64 MiB, will set to 1.32 GiB (RSS), difference: 27.93 MiB 2024.10.22 02:00:51.643764 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5afe85b220a328ab27cdec1e45aabe77) : [0, 163]: Setting download as failed: File segment: [0, 163], key: 5afe85b220a328ab27cdec1e45aabe77, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:51.645690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5afe85b220a328ab27cdec1e45aabe77): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 02:00:51.881023 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_717_717_0 (state Temporary) 2024.10.22 02:00:51.888779 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 624 2024.10.22 02:00:52.000711 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_717_717_0 to 202410_724_724_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:51.910009 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6031 2024.10.22 02:00:52.145575 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1088100 2024.10.22 02:00:52.247431 [ 1217 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_719_719_0 to 202410_724_724_0 2024.10.22 02:00:52.258785 [ 1217 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:52.262541 [ 1217 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:00:52.267175 [ 1217 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 257 ms. 2024.10.22 02:00:52.307822 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_718_137/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:00:52.406868 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_719_719_0 to 202410_724_724_0 into Compact with storage Full 2024.10.22 02:00:52.456055 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:52.464815 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_719_719_0, total 770 rows starting from the beginning of the part 2024.10.22 02:00:52.467974 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:52.470678 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:00:52.630228 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_720_720_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:00:52.630948 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_606_606_0 (state Temporary) 2024.10.22 02:00:52.756044 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_721_721_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:00:52.850313 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_606_606_0 to 202410_616_616_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:52.882246 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ca763727613a69c748bd56244147744f) : [0, 68484]: Setting download as failed: File segment: [0, 68484], key: ca763727613a69c748bd56244147744f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:52.885071 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ca763727613a69c748bd56244147744f): No space left in cache to reserve 68485 bytes, will continue without cache download 2024.10.22 02:00:52.893488 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_722_722_0, total 1200 rows starting from the beginning of the part 2024.10.22 02:00:52.898693 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ca763727613a69c748bd56244147744f, offset: 0, size: 0 2024.10.22 02:00:52.901219 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ca763727613a69c748bd56244147744f for removal 2024.10.22 02:00:52.907034 [ 1476 ] {} CacheMetadata: Key ca763727613a69c748bd56244147744f is removed from metadata 2024.10.22 02:00:52.908271 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5afe85b220a328ab27cdec1e45aabe77, offset: 0, size: 0 2024.10.22 02:00:52.911175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5afe85b220a328ab27cdec1e45aabe77 for removal 2024.10.22 02:00:52.915736 [ 1476 ] {} CacheMetadata: Key 5afe85b220a328ab27cdec1e45aabe77 is removed from metadata 2024.10.22 02:00:53.033228 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_723_723_0, total 1495 rows starting from the beginning of the part 2024.10.22 02:00:53.086440 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 624 2024.10.22 02:00:53.137350 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1171_15, 202410_1172_1172_0, 202410_1173_1173_0, 202410_1174_1174_0, 202410_1175_1175_0, 202410_1176_1176_0] 2024.10.22 02:00:53.140427 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1171_15, 202410_1172_1172_0, 202410_1173_1173_0, 202410_1174_1174_0, 202410_1175_1175_0, 202410_1176_1176_0] 2024.10.22 02:00:53.164394 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_724_724_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:00:53.271514 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1171_15, 202410_1172_1172_0, 202410_1173_1173_0, 202410_1174_1174_0, 202410_1175_1175_0, 202410_1176_1176_0] 2024.10.22 02:00:53.407019 [ 1219 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 217 ms. 2024.10.22 02:00:53.689299 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_678_36, 202410_679_679_0] 2024.10.22 02:00:53.694715 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_678_36, 202410_679_679_0] 2024.10.22 02:00:53.704140 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_678_36, 202410_679_679_0] 2024.10.22 02:00:53.753442 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 15.22 MiB, will set to 1.33 GiB (RSS), difference: -2.13 MiB 2024.10.22 02:00:53.789389 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6ba59ec1d372e5c4f370f342d9dd6b91) : [0, 233]: Setting download as failed: File segment: [0, 233], key: 6ba59ec1d372e5c4f370f342d9dd6b91, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:53.794575 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6ba59ec1d372e5c4f370f342d9dd6b91): No space left in cache to reserve 234 bytes, will continue without cache download 2024.10.22 02:00:53.914348 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 104 entries to flush up to offset 82855 2024.10.22 02:00:53.989120 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 11406 2024.10.22 02:00:54.109163 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:54.115272 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:54.120247 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} MergeTask::MergeProjectionsStage: Merge sorted 7315 rows, containing 4 columns (4 merged, 0 gathered) in 1.829151114 sec., 3999.1228411979087 rows/sec., 62.49 KiB/sec. 2024.10.22 02:00:54.145215 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:54.148079 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:54.178721 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} MergedBlockOutputStream: filled checksums 202410_719_724_1 (state Temporary) 2024.10.22 02:00:54.198536 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_736_736_0 (state Temporary) 2024.10.22 02:00:54.258487 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_719_724_1 to 202410_719_724_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:54.260614 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_736_736_0 to 202410_742_742_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:54.261601 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_719_724_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_719_719_0, 202410_724_724_0] -> 202410_719_724_1 2024.10.22 02:00:54.270928 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_749_749_0 (state Temporary) 2024.10.22 02:00:54.367368 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 83454 2024.10.22 02:00:54.436960 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:00:54.501340 [ 1167 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_737_83 to 202410_738_738_0 2024.10.22 02:00:54.521308 [ 1167 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 02:00:54.530489 [ 1167 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:54.531663 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 82855 2024.10.22 02:00:54.536004 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_749_749_0 to 202410_762_762_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:54.559229 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:54.562782 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_738_84} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_737_83 to 202410_738_738_0 into Compact with storage Full 2024.10.22 02:00:54.563411 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:00:54.654031 [ 1167 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 268 ms. 2024.10.22 02:00:54.666986 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_738_84} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:54.683768 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1263_1263_0 (state Temporary) 2024.10.22 02:00:54.692575 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11406 2024.10.22 02:00:54.697661 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_738_84} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_737_83, total 23238 rows starting from the beginning of the part 2024.10.22 02:00:54.765974 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(95b6fc8fe62d732019032002355e83a2) : [0, 436222]: Setting download as failed: File segment: [0, 436222], key: 95b6fc8fe62d732019032002355e83a2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:54.767573 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(95b6fc8fe62d732019032002355e83a2): No space left in cache to reserve 436223 bytes, will continue without cache download 2024.10.22 02:00:54.780741 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_738_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_738_738_0, total 113 rows starting from the beginning of the part 2024.10.22 02:00:54.805416 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 95b6fc8fe62d732019032002355e83a2, offset: 0, size: 0 2024.10.22 02:00:54.808593 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 95b6fc8fe62d732019032002355e83a2 for removal 2024.10.22 02:00:54.816567 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6ba59ec1d372e5c4f370f342d9dd6b91, offset: 0, size: 0 2024.10.22 02:00:54.825287 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6ba59ec1d372e5c4f370f342d9dd6b91 for removal 2024.10.22 02:00:54.843883 [ 1476 ] {} CacheMetadata: Key 95b6fc8fe62d732019032002355e83a2 is removed from metadata 2024.10.22 02:00:54.848247 [ 1476 ] {} CacheMetadata: Key 6ba59ec1d372e5c4f370f342d9dd6b91 is removed from metadata 2024.10.22 02:00:54.848565 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1263_1263_0 to 202410_1276_1276_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:54.911740 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83454 2024.10.22 02:00:55.575050 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 14.77 MiB, will set to 1.33 GiB (RSS), difference: -17.52 MiB 2024.10.22 02:00:55.936137 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(98fe1284c06727d53fc4f76a3befe336) : [0, 168]: Setting download as failed: File segment: [0, 168], key: 98fe1284c06727d53fc4f76a3befe336, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:55.937559 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(98fe1284c06727d53fc4f76a3befe336): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 02:00:56.255295 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_738_84/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:00:56.433741 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_738_84} MergeTask::MergeProjectionsStage: Merge sorted 23351 rows, containing 13 columns (13 merged, 0 gathered) in 1.877435894 sec., 12437.70829918947 rows/sec., 5.39 MiB/sec. 2024.10.22 02:00:56.795624 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e93d9cabe8c8de9b0564cec008692771) : [0, 272946]: Setting download as failed: File segment: [0, 272946], key: e93d9cabe8c8de9b0564cec008692771, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:56.797572 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e93d9cabe8c8de9b0564cec008692771): No space left in cache to reserve 272947 bytes, will continue without cache download 2024.10.22 02:00:56.831201 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_738_84} MergedBlockOutputStream: filled checksums 202410_507_738_84 (state Temporary) 2024.10.22 02:00:57.141621 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e93d9cabe8c8de9b0564cec008692771, offset: 0, size: 0 2024.10.22 02:00:57.144168 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e93d9cabe8c8de9b0564cec008692771 for removal 2024.10.22 02:00:57.149287 [ 1476 ] {} CacheMetadata: Key e93d9cabe8c8de9b0564cec008692771 is removed from metadata 2024.10.22 02:00:57.151919 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 98fe1284c06727d53fc4f76a3befe336, offset: 0, size: 0 2024.10.22 02:00:57.153196 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 98fe1284c06727d53fc4f76a3befe336 for removal 2024.10.22 02:00:57.156941 [ 1476 ] {} CacheMetadata: Key 98fe1284c06727d53fc4f76a3befe336 is removed from metadata 2024.10.22 02:00:57.251214 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_738_84} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_738_84 to 202410_507_738_84 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:57.253888 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_738_84} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_737_83, 202410_738_738_0] -> 202410_507_738_84 2024.10.22 02:00:57.265513 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.51 MiB. 2024.10.22 02:00:57.455721 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_698_141, 202410_699_699_0, 202410_700_700_0, 202410_701_701_0, 202410_702_702_0, 202410_703_703_0] 2024.10.22 02:00:57.458626 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_698_141, 202410_699_699_0, 202410_700_700_0, 202410_701_701_0, 202410_702_702_0, 202410_703_703_0] 2024.10.22 02:00:57.492767 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_698_141, 202410_699_699_0, 202410_700_700_0, 202410_701_701_0, 202410_702_702_0, 202410_703_703_0] 2024.10.22 02:00:57.656625 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 2.00 GiB, free memory in arenas 16.16 MiB, will set to 1.33 GiB (RSS), difference: 27.40 MiB 2024.10.22 02:00:57.935698 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3b401cdc4e700a68abece132074e4bd6) : [0, 191]: Setting download as failed: File segment: [0, 191], key: 3b401cdc4e700a68abece132074e4bd6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:57.939284 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3b401cdc4e700a68abece132074e4bd6): No space left in cache to reserve 192 bytes, will continue without cache download 2024.10.22 02:00:58.670765 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ae9af716496e6b156c21ab7df3689c91) : [0, 165093]: Setting download as failed: File segment: [0, 165093], key: ae9af716496e6b156c21ab7df3689c91, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:00:58.673255 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ae9af716496e6b156c21ab7df3689c91): No space left in cache to reserve 165094 bytes, will continue without cache download 2024.10.22 02:00:58.816215 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ae9af716496e6b156c21ab7df3689c91, offset: 0, size: 0 2024.10.22 02:00:58.820391 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ae9af716496e6b156c21ab7df3689c91 for removal 2024.10.22 02:00:58.825028 [ 1476 ] {} CacheMetadata: Key ae9af716496e6b156c21ab7df3689c91 is removed from metadata 2024.10.22 02:00:58.828213 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3b401cdc4e700a68abece132074e4bd6, offset: 0, size: 0 2024.10.22 02:00:58.833607 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3b401cdc4e700a68abece132074e4bd6 for removal 2024.10.22 02:00:58.837220 [ 1476 ] {} CacheMetadata: Key 3b401cdc4e700a68abece132074e4bd6 is removed from metadata 2024.10.22 02:00:58.931242 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 83511 2024.10.22 02:00:59.151119 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1089640 2024.10.22 02:00:59.168563 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:59.174443 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:59.220065 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1264_1264_0 (state Temporary) 2024.10.22 02:00:59.256041 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1264_1264_0 to 202410_1277_1277_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:00:59.333025 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1272_13 to 202410_1277_1277_0 2024.10.22 02:00:59.343233 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:00:59.348151 [ 1202 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:00:59.371372 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1272_13 to 202410_1277_1277_0 into Compact with storage Full 2024.10.22 02:00:59.441540 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(08ab16db79c939be63d26b97e598835a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 08ab16db79c939be63d26b97e598835a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:00:59.447414 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 60398, file segment info: File segment: [0, 33554431], key: 08ab16db79c939be63d26b97e598835a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:00:59.450410 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:00:59.455046 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:00:59.464362 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1272_13, total 4008 rows starting from the beginning of the part 2024.10.22 02:00:59.550225 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83511 2024.10.22 02:00:59.637787 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1273_1273_0, total 88 rows starting from the beginning of the part 2024.10.22 02:00:59.642276 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 47 entries to flush up to offset 6078 2024.10.22 02:00:59.662854 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1274_1274_0, total 62 rows starting from the beginning of the part 2024.10.22 02:00:59.696536 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1275_1275_0, total 74 rows starting from the beginning of the part 2024.10.22 02:00:59.769470 [ 1163 ] {} DatabaseCatalog: Execution took 208 ms. 2024.10.22 02:00:59.796360 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1276_1276_0, total 81 rows starting from the beginning of the part 2024.10.22 02:00:59.947961 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1277_1277_0, total 57 rows starting from the beginning of the part 2024.10.22 02:01:00.110658 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(22a664fbd2698e58892eb01bc85864e0) : [0, 165]: Setting download as failed: File segment: [0, 165], key: 22a664fbd2698e58892eb01bc85864e0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:00.114892 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(22a664fbd2698e58892eb01bc85864e0): No space left in cache to reserve 166 bytes, will continue without cache download 2024.10.22 02:01:00.536243 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:00.541498 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:00.804222 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(abca8b03a9234238a95cd16040dbd779) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: abca8b03a9234238a95cd16040dbd779, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:01:00.837731 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 677, file segment info: File segment: [0, 33554431], key: abca8b03a9234238a95cd16040dbd779, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:01:00.944791 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:00.992025 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_718_718_0 (state Temporary) 2024.10.22 02:01:01.044100 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_718_718_0 to 202410_725_725_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:01.142647 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:01:01.144405 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:01:01.165619 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1089640 2024.10.22 02:01:01.174247 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_563_115, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0, 202410_567_567_0, 202410_568_568_0] 2024.10.22 02:01:01.175588 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_563_115, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0, 202410_567_567_0, 202410_568_568_0] 2024.10.22 02:01:01.223285 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_563_115, 202410_564_564_0, 202410_565_565_0, 202410_566_566_0, 202410_567_567_0, 202410_568_568_0] 2024.10.22 02:01:01.343353 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} MergeTask::MergeProjectionsStage: Merge sorted 4370 rows, containing 13 columns (13 merged, 0 gathered) in 1.977163774 sec., 2210.236732771536 rows/sec., 379.34 KiB/sec. 2024.10.22 02:01:01.731376 [ 1184 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:01:01.756964 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8f2d2ce4cea79b08f34dc3d457c009f9) : [0, 164384]: Setting download as failed: File segment: [0, 164384], key: 8f2d2ce4cea79b08f34dc3d457c009f9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:01.758462 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8f2d2ce4cea79b08f34dc3d457c009f9): No space left in cache to reserve 164385 bytes, will continue without cache download 2024.10.22 02:01:01.767697 [ 1184 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:01:01.792551 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} MergedBlockOutputStream: filled checksums 202410_1207_1277_14 (state Temporary) 2024.10.22 02:01:01.877635 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8f2d2ce4cea79b08f34dc3d457c009f9, offset: 0, size: 0 2024.10.22 02:01:01.880263 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8f2d2ce4cea79b08f34dc3d457c009f9 for removal 2024.10.22 02:01:01.885737 [ 1476 ] {} CacheMetadata: Key 8f2d2ce4cea79b08f34dc3d457c009f9 is removed from metadata 2024.10.22 02:01:01.886235 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 22a664fbd2698e58892eb01bc85864e0, offset: 0, size: 0 2024.10.22 02:01:01.889562 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 22a664fbd2698e58892eb01bc85864e0 for removal 2024.10.22 02:01:01.893713 [ 1476 ] {} CacheMetadata: Key 22a664fbd2698e58892eb01bc85864e0 is removed from metadata 2024.10.22 02:01:02.050862 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 82939 2024.10.22 02:01:02.197152 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 11421 2024.10.22 02:01:02.238365 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1277_14 to 202410_1207_1277_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:02.348738 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:02.354413 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:01:02.363885 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1277_14} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1272_13, 202410_1277_1277_0] -> 202410_1207_1277_14 2024.10.22 02:01:02.411833 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:01:02.432287 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_750_750_0 (state Temporary) 2024.10.22 02:01:02.473100 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_750_750_0 to 202410_763_763_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:02.558640 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_679_37, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0, 202410_683_683_0, 202410_684_684_0] 2024.10.22 02:01:02.560211 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_679_37, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0, 202410_683_683_0, 202410_684_684_0] 2024.10.22 02:01:02.571285 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 14.39 MiB, will set to 1.32 GiB (RSS), difference: -5.06 MiB 2024.10.22 02:01:02.616071 [ 1197 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_758_159 to 202410_759_759_0 2024.10.22 02:01:02.629615 [ 1197 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:02.644941 [ 1197 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:01:02.660139 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_759_160} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_758_159 to 202410_759_759_0 into Compact with storage Full 2024.10.22 02:01:02.668718 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11421 2024.10.22 02:01:02.695392 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_679_37, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0, 202410_683_683_0, 202410_684_684_0] 2024.10.22 02:01:02.759773 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:02.769466 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:01:02.768374 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(9cad1cfa2b652edaadd345b7ae793e5c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9cad1cfa2b652edaadd345b7ae793e5c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:01:02.777552 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 30231, file segment info: File segment: [0, 33554431], key: 9cad1cfa2b652edaadd345b7ae793e5c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:01:02.781743 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:02.929170 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_737_737_0 (state Temporary) 2024.10.22 02:01:02.964598 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_737_737_0 to 202410_743_743_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:03.003363 [ 1208 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_738_84 to 202410_739_739_0 2024.10.22 02:01:03.024508 [ 1208 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 02:01:03.036778 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 82939 2024.10.22 02:01:03.049055 [ 1208 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:01:03.073051 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_739_85} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_738_84 to 202410_739_739_0 into Compact with storage Full 2024.10.22 02:01:03.122590 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_759_160} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:03.134848 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_759_160} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_758_159, total 11456 rows starting from the beginning of the part 2024.10.22 02:01:03.163090 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_718_137/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:01:03.171748 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_759_160} MergeTreeSequentialSource: Reading 2 marks from part 202410_759_759_0, total 12 rows starting from the beginning of the part 2024.10.22 02:01:03.252084 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_739_85} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:03.368282 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_739_85} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_738_84, total 23351 rows starting from the beginning of the part 2024.10.22 02:01:03.444619 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_739_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_739_739_0, total 71 rows starting from the beginning of the part 2024.10.22 02:01:03.552703 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 83598 2024.10.22 02:01:03.654742 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4eeb713597c163cc6d9f2380c3a177e4) : [0, 192]: Setting download as failed: File segment: [0, 192], key: 4eeb713597c163cc6d9f2380c3a177e4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:03.656311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4eeb713597c163cc6d9f2380c3a177e4): No space left in cache to reserve 193 bytes, will continue without cache download 2024.10.22 02:01:03.925767 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_759_160} MergeTask::MergeProjectionsStage: Merge sorted 11468 rows, containing 12 columns (12 merged, 0 gathered) in 1.273292243 sec., 9006.573363692438 rows/sec., 1.40 MiB/sec. 2024.10.22 02:01:04.069717 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_759_160} MergedBlockOutputStream: filled checksums 202410_1_759_160 (state Temporary) 2024.10.22 02:01:04.084911 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:04.089527 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.35 GiB. 2024.10.22 02:01:04.174344 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1265_1265_0 (state Temporary) 2024.10.22 02:01:04.246180 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1265_1265_0 to 202410_1278_1278_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:04.342780 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83598 2024.10.22 02:01:04.499132 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(2d79faf8d553d46888f05be38d4616f3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2d79faf8d553d46888f05be38d4616f3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:01:04.516779 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 612, file segment info: File segment: [0, 33554431], key: 2d79faf8d553d46888f05be38d4616f3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:01:04.520148 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:04.553186 [ 1735 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qpy/oirdylbiomlsoqlytlqqlosufvtzx, size 30231 2024.10.22 02:01:04.792934 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_759_160} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_759_160 to 202410_1_759_160 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:04.799974 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_759_160} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_758_159, 202410_759_759_0] -> 202410_1_759_160 2024.10.22 02:01:04.840509 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:01:05.068631 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:01:05.085187 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:01:05.117760 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:01:05.155267 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:01:05.211263 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:01:05.270348 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:01:05.310274 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:01:05.349441 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:01:05.398501 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c551a87e7861e273ac27d910e28f7ab9) : [0, 345772]: Setting download as failed: File segment: [0, 345772], key: c551a87e7861e273ac27d910e28f7ab9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:05.405022 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c551a87e7861e273ac27d910e28f7ab9): No space left in cache to reserve 345773 bytes, will continue without cache download 2024.10.22 02:01:05.476165 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kfv/mmdzlqjgnpuevmpmrxuimybbzvrbv (LogSeriesLimiter: on interval from 2024-10-22 02:00:50 to 2024-10-22 02:01:05 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:01:05.473723 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c551a87e7861e273ac27d910e28f7ab9, offset: 0, size: 0 2024.10.22 02:01:05.513348 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c551a87e7861e273ac27d910e28f7ab9 for removal 2024.10.22 02:01:05.518731 [ 1476 ] {} CacheMetadata: Key c551a87e7861e273ac27d910e28f7ab9 is removed from metadata 2024.10.22 02:01:05.528978 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4eeb713597c163cc6d9f2380c3a177e4, offset: 0, size: 0 2024.10.22 02:01:05.532911 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4eeb713597c163cc6d9f2380c3a177e4 for removal 2024.10.22 02:01:05.537108 [ 1476 ] {} CacheMetadata: Key 4eeb713597c163cc6d9f2380c3a177e4 is removed from metadata 2024.10.22 02:01:05.651488 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_739_85} MergeTask::MergeProjectionsStage: Merge sorted 23422 rows, containing 13 columns (13 merged, 0 gathered) in 2.583889806 sec., 9064.62804474565 rows/sec., 3.93 MiB/sec. 2024.10.22 02:01:05.868578 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_739_85} MergedBlockOutputStream: filled checksums 202410_507_739_85 (state Temporary) 2024.10.22 02:01:06.257323 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_739_85} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_739_85 to 202410_507_739_85 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:06.275281 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_739_85} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_738_84, 202410_739_739_0] -> 202410_507_739_85 2024.10.22 02:01:06.312732 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.47 MiB. 2024.10.22 02:01:06.864434 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(c871040e84a5dfcb025052ecb66cacb7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c871040e84a5dfcb025052ecb66cacb7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:01:06.974962 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 356072, file segment info: File segment: [0, 33554431], key: c871040e84a5dfcb025052ecb66cacb7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:01:06.978650 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:07.780183 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_718_137} MergeTask::MergeProjectionsStage: Merge sorted 1094375 rows, containing 4 columns (4 merged, 0 gathered) in 32.313085957 sec., 33867.85779161786 rows/sec., 508.11 KiB/sec. 2024.10.22 02:01:07.899509 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ee3ee86ab6ed7d40949484352120c69c) : [0, 195]: Setting download as failed: File segment: [0, 195], key: ee3ee86ab6ed7d40949484352120c69c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:07.901444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ee3ee86ab6ed7d40949484352120c69c): No space left in cache to reserve 196 bytes, will continue without cache download 2024.10.22 02:01:07.966467 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_718_137} MergedBlockOutputStream: filled checksums 202410_1_718_137 (state Temporary) 2024.10.22 02:01:08.089039 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 625 2024.10.22 02:01:08.189698 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1090795 2024.10.22 02:01:08.329436 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:08.333647 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:08.350399 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 83648 2024.10.22 02:01:08.588815 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_607_607_0 (state Temporary) 2024.10.22 02:01:08.591087 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(74cffffd5874ac5111189d50da436ee0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 74cffffd5874ac5111189d50da436ee0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:01:08.624544 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 519, file segment info: File segment: [0, 33554431], key: 74cffffd5874ac5111189d50da436ee0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:01:08.631022 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:08.680893 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:08.684504 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:08.698505 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_607_607_0 to 202410_617_617_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:08.744664 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1266_1266_0 (state Temporary) 2024.10.22 02:01:08.766483 [ 1696 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qal/rrhrnitmvveqoismwiinprijubdlj, size 356072 2024.10.22 02:01:08.786733 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 625 2024.10.22 02:01:08.971340 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1266_1266_0 to 202410_1279_1279_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:09.046768 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83648 2024.10.22 02:01:09.049660 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:01:09.054761 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:01:09.067997 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:01:09.143066 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:09.144392 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:09.171613 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:01:09.191712 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:01:09.220971 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:01:09.345991 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6095b71905819f520eaf979c5cdc1e08) : [0, 386319]: Setting download as failed: File segment: [0, 386319], key: 6095b71905819f520eaf979c5cdc1e08, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:09.348586 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6095b71905819f520eaf979c5cdc1e08): No space left in cache to reserve 386320 bytes, will continue without cache download 2024.10.22 02:01:09.349607 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:01:09.370939 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_719_719_0 (state Temporary) 2024.10.22 02:01:09.490966 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6095b71905819f520eaf979c5cdc1e08, offset: 0, size: 0 2024.10.22 02:01:09.494670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6095b71905819f520eaf979c5cdc1e08 for removal 2024.10.22 02:01:09.498828 [ 1476 ] {} CacheMetadata: Key 6095b71905819f520eaf979c5cdc1e08 is removed from metadata 2024.10.22 02:01:09.500118 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ee3ee86ab6ed7d40949484352120c69c, offset: 0, size: 0 2024.10.22 02:01:09.502871 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ee3ee86ab6ed7d40949484352120c69c for removal 2024.10.22 02:01:09.508132 [ 1476 ] {} CacheMetadata: Key ee3ee86ab6ed7d40949484352120c69c is removed from metadata 2024.10.22 02:01:09.671097 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 9.80 MiB, will set to 1.32 GiB (RSS), difference: -4.70 MiB 2024.10.22 02:01:09.715839 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_719_719_0 to 202410_726_726_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:09.957167 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1090795 2024.10.22 02:01:09.995347 [ 1257 ] {} DatabaseCatalog: Execution took 224 ms. 2024.10.22 02:01:10.180777 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 11440 2024.10.22 02:01:10.289214 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:10.314723 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:10.366337 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_751_751_0 (state Temporary) 2024.10.22 02:01:10.447340 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(47f3241b37a12058f530a79c0c4e6297) : [0, 190]: Setting download as failed: File segment: [0, 190], key: 47f3241b37a12058f530a79c0c4e6297, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:10.552073 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(47f3241b37a12058f530a79c0c4e6297): No space left in cache to reserve 191 bytes, will continue without cache download 2024.10.22 02:01:10.628732 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_751_751_0 to 202410_764_764_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:10.651049 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 102 entries to flush up to offset 83041 2024.10.22 02:01:10.749300 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_718_137} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_718_137 to 202410_1_718_137 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:10.753287 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_718_137} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_712_136, 202410_713_718_1] -> 202410_1_718_137 2024.10.22 02:01:10.779399 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11440 2024.10.22 02:01:10.804155 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:01:10.837557 [ 1242 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_759_160 to 202410_760_760_0 2024.10.22 02:01:10.842688 [ 1242 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:10.845687 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:10.847495 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:10.851759 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:10.864307 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_760_161} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_759_160 to 202410_760_760_0 into Compact with storage Full 2024.10.22 02:01:10.952408 [ 1261 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 301 ms. 2024.10.22 02:01:11.055588 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_760_161} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:11.087925 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_760_161} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_759_160, total 11468 rows starting from the beginning of the part 2024.10.22 02:01:11.143034 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:01:11.144846 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:01:11.227556 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_738_738_0 (state Temporary) 2024.10.22 02:01:11.259396 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_760_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_760_760_0, total 15 rows starting from the beginning of the part 2024.10.22 02:01:11.271863 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_738_738_0 to 202410_744_744_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:11.322281 [ 1244 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_739_85 to 202410_740_740_0 2024.10.22 02:01:11.332537 [ 1244 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 02:01:11.334935 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 83041 2024.10.22 02:01:11.335959 [ 1244 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:11.450851 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_740_86} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_739_85 to 202410_740_740_0 into Wide with storage Full 2024.10.22 02:01:11.468988 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_740_86} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:11.475403 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_740_86} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_739_85, total 23422 rows starting from the beginning of the part 2024.10.22 02:01:11.632084 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_740_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_740_740_0, total 115 rows starting from the beginning of the part 2024.10.22 02:01:11.695644 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(66d32d9f9041465d219d4d50b0d8e5d3) : [0, 197351]: Setting download as failed: File segment: [0, 197351], key: 66d32d9f9041465d219d4d50b0d8e5d3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:11.697553 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(66d32d9f9041465d219d4d50b0d8e5d3): No space left in cache to reserve 197352 bytes, will continue without cache download 2024.10.22 02:01:11.941310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 66d32d9f9041465d219d4d50b0d8e5d3, offset: 0, size: 0 2024.10.22 02:01:11.944565 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 66d32d9f9041465d219d4d50b0d8e5d3 for removal 2024.10.22 02:01:11.947176 [ 1476 ] {} CacheMetadata: Key 66d32d9f9041465d219d4d50b0d8e5d3 is removed from metadata 2024.10.22 02:01:11.948259 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 47f3241b37a12058f530a79c0c4e6297, offset: 0, size: 0 2024.10.22 02:01:11.950197 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 47f3241b37a12058f530a79c0c4e6297 for removal 2024.10.22 02:01:11.955006 [ 1476 ] {} CacheMetadata: Key 47f3241b37a12058f530a79c0c4e6297 is removed from metadata 2024.10.22 02:01:12.121458 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_760_161} MergeTask::MergeProjectionsStage: Merge sorted 11483 rows, containing 12 columns (12 merged, 0 gathered) in 1.263716207 sec., 9086.692040818307 rows/sec., 1.41 MiB/sec. 2024.10.22 02:01:12.162858 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_760_161} MergedBlockOutputStream: filled checksums 202410_1_760_161 (state Temporary) 2024.10.22 02:01:12.470183 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 7.52 MiB, will set to 1.32 GiB (RSS), difference: -37.37 MiB 2024.10.22 02:01:12.542350 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_760_161} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_760_161 to 202410_1_760_161 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:12.554661 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_760_161} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_759_160, 202410_760_760_0] -> 202410_1_760_161 2024.10.22 02:01:12.591027 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:01:13.008234 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a9e144700f9d4b17efe74ff647949d09) : [0, 257]: Setting download as failed: File segment: [0, 257], key: a9e144700f9d4b17efe74ff647949d09, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:13.011194 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a9e144700f9d4b17efe74ff647949d09): No space left in cache to reserve 258 bytes, will continue without cache download 2024.10.22 02:01:13.071214 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 83733 2024.10.22 02:01:13.377395 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_740_86} MergeTask::MergeProjectionsStage: Merge sorted 23537 rows, containing 13 columns (13 merged, 0 gathered) in 2.032808393 sec., 11578.562977725762 rows/sec., 5.02 MiB/sec. 2024.10.22 02:01:13.437407 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:13.441106 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:13.628228 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1267_1267_0 (state Temporary) 2024.10.22 02:01:13.666197 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1267_1267_0 to 202410_1280_1280_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:13.732607 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83733 2024.10.22 02:01:13.755909 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_740_86} MergedBlockOutputStream: filled checksums 202410_507_740_86 (state Temporary) 2024.10.22 02:01:14.138559 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_740_86} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_740_86 to 202410_507_740_86 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:14.145683 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_740_86} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_739_85, 202410_740_740_0] -> 202410_507_740_86 2024.10.22 02:01:14.165592 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.07 MiB. 2024.10.22 02:01:14.294415 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(43e0d316670a488472ae48fbc937e570) : [0, 2049509]: Setting download as failed: File segment: [0, 2049509], key: 43e0d316670a488472ae48fbc937e570, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:14.295996 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(43e0d316670a488472ae48fbc937e570): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:01:14.400037 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_665_665_0, 202410_666_666_0, 202410_667_667_0, 202410_668_668_0, 202410_669_669_0, 202410_670_670_0] 2024.10.22 02:01:14.401783 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_665_665_0, 202410_666_666_0, 202410_667_667_0, 202410_668_668_0, 202410_669_669_0, 202410_670_670_0] 2024.10.22 02:01:14.472310 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.28 GiB, peak 2.00 GiB, free memory in arenas 7.22 MiB, will set to 1.32 GiB (RSS), difference: 41.79 MiB 2024.10.22 02:01:14.546821 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 43e0d316670a488472ae48fbc937e570, offset: 0, size: 0 2024.10.22 02:01:14.548474 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_665_665_0, 202410_666_666_0, 202410_667_667_0, 202410_668_668_0, 202410_669_669_0, 202410_670_670_0] 2024.10.22 02:01:14.548929 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 43e0d316670a488472ae48fbc937e570 for removal 2024.10.22 02:01:14.550987 [ 1476 ] {} CacheMetadata: Key 43e0d316670a488472ae48fbc937e570 is removed from metadata 2024.10.22 02:01:14.553607 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a9e144700f9d4b17efe74ff647949d09, offset: 0, size: 0 2024.10.22 02:01:14.555970 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a9e144700f9d4b17efe74ff647949d09 for removal 2024.10.22 02:01:14.558891 [ 1476 ] {} CacheMetadata: Key a9e144700f9d4b17efe74ff647949d09 is removed from metadata 2024.10.22 02:01:15.369401 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3ceb4db05e2f50effb8d7e12e19d4b3b) : [0, 159]: Setting download as failed: File segment: [0, 159], key: 3ceb4db05e2f50effb8d7e12e19d4b3b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:15.370498 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3ceb4db05e2f50effb8d7e12e19d4b3b): No space left in cache to reserve 160 bytes, will continue without cache download 2024.10.22 02:01:15.402650 [ 1253 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 221 ms. 2024.10.22 02:01:16.248041 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(68e3f6941e8301c99d30ba675baa54f8) : [0, 106043]: Setting download as failed: File segment: [0, 106043], key: 68e3f6941e8301c99d30ba675baa54f8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:16.249361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(68e3f6941e8301c99d30ba675baa54f8): No space left in cache to reserve 106044 bytes, will continue without cache download 2024.10.22 02:01:16.265643 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 68e3f6941e8301c99d30ba675baa54f8, offset: 0, size: 0 2024.10.22 02:01:16.267143 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 68e3f6941e8301c99d30ba675baa54f8 for removal 2024.10.22 02:01:16.280258 [ 1476 ] {} CacheMetadata: Key 68e3f6941e8301c99d30ba675baa54f8 is removed from metadata 2024.10.22 02:01:16.281237 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3ceb4db05e2f50effb8d7e12e19d4b3b, offset: 0, size: 0 2024.10.22 02:01:16.283613 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3ceb4db05e2f50effb8d7e12e19d4b3b for removal 2024.10.22 02:01:16.290831 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 626 2024.10.22 02:01:16.387212 [ 1476 ] {} CacheMetadata: Key 3ceb4db05e2f50effb8d7e12e19d4b3b is removed from metadata 2024.10.22 02:01:16.395460 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 7.27 MiB, will set to 1.32 GiB (RSS), difference: 1.21 MiB 2024.10.22 02:01:16.527666 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:16.538428 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:16.692609 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_608_608_0 (state Temporary) 2024.10.22 02:01:16.772384 [ 1240 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:01:16.791024 [ 1240 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:01:16.850063 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_608_608_0 to 202410_618_618_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:16.863750 [ 1181 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_613_125 to 202410_618_618_0 2024.10.22 02:01:16.875081 [ 1181 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:16.884490 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:16.888124 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 626 2024.10.22 02:01:16.960270 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1092335 2024.10.22 02:01:16.978736 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:16.983639 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:17.027760 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_613_125 to 202410_618_618_0 into Compact with storage Full 2024.10.22 02:01:17.245640 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:17.275005 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_613_125, total 657 rows starting from the beginning of the part 2024.10.22 02:01:17.411285 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(667e044e36ecf39e6c4acfd81c0cd9ad) : [0, 162]: Setting download as failed: File segment: [0, 162], key: 667e044e36ecf39e6c4acfd81c0cd9ad, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:17.415082 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(667e044e36ecf39e6c4acfd81c0cd9ad): No space left in cache to reserve 163 bytes, will continue without cache download 2024.10.22 02:01:17.505439 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_614_614_0, total 1 rows starting from the beginning of the part 2024.10.22 02:01:17.676247 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:17.680652 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:17.743357 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_615_615_0, total 1 rows starting from the beginning of the part 2024.10.22 02:01:17.745408 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 83790 2024.10.22 02:01:17.891355 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_720_720_0 (state Temporary) 2024.10.22 02:01:17.952223 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:17.954433 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:18.068847 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_616_616_0, total 1 rows starting from the beginning of the part 2024.10.22 02:01:18.085448 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_720_720_0 to 202410_727_727_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:18.122369 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1268_1268_0 (state Temporary) 2024.10.22 02:01:18.147691 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1092335 2024.10.22 02:01:18.243123 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_617_617_0, total 1 rows starting from the beginning of the part 2024.10.22 02:01:18.286938 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 11454 2024.10.22 02:01:18.298792 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1268_1268_0 to 202410_1281_1281_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:18.362735 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83790 2024.10.22 02:01:18.396964 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_618_618_0, total 1 rows starting from the beginning of the part 2024.10.22 02:01:18.411606 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:18.415784 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:18.453076 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_752_752_0 (state Temporary) 2024.10.22 02:01:18.505547 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_752_752_0 to 202410_765_765_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:18.550736 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dff1309ca435504a8ccf201f5ff2497a) : [0, 92818]: Setting download as failed: File segment: [0, 92818], key: dff1309ca435504a8ccf201f5ff2497a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:18.551716 [ 1157 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_760_161 to 202410_761_761_0 2024.10.22 02:01:18.552093 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dff1309ca435504a8ccf201f5ff2497a): No space left in cache to reserve 92819 bytes, will continue without cache download 2024.10.22 02:01:18.556369 [ 1157 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:18.560592 [ 1157 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:18.561282 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dff1309ca435504a8ccf201f5ff2497a, offset: 0, size: 0 2024.10.22 02:01:18.563542 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dff1309ca435504a8ccf201f5ff2497a for removal 2024.10.22 02:01:18.565696 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 667e044e36ecf39e6c4acfd81c0cd9ad, offset: 0, size: 0 2024.10.22 02:01:18.567320 [ 1476 ] {} CacheMetadata: Key dff1309ca435504a8ccf201f5ff2497a is removed from metadata 2024.10.22 02:01:18.567500 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 667e044e36ecf39e6c4acfd81c0cd9ad for removal 2024.10.22 02:01:18.583284 [ 1476 ] {} CacheMetadata: Key 667e044e36ecf39e6c4acfd81c0cd9ad is removed from metadata 2024.10.22 02:01:18.664025 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11454 2024.10.22 02:01:18.573770 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_761_162} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_760_161 to 202410_761_761_0 into Compact with storage Full 2024.10.22 02:01:18.850613 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 149 entries to flush up to offset 83190 2024.10.22 02:01:18.879660 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_761_162} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:18.953313 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:18.958028 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:18.992209 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_761_162} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_760_161, total 11483 rows starting from the beginning of the part 2024.10.22 02:01:18.994422 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_739_739_0 (state Temporary) 2024.10.22 02:01:19.048864 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_761_162} MergeTreeSequentialSource: Reading 2 marks from part 202410_761_761_0, total 16 rows starting from the beginning of the part 2024.10.22 02:01:19.164048 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_739_739_0 to 202410_745_745_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:19.203174 [ 1155 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_740_86 to 202410_741_741_0 2024.10.22 02:01:19.215409 [ 1155 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 02:01:19.220120 [ 1155 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:19.311198 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_741_87} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_740_86 to 202410_741_741_0 into Wide with storage Full 2024.10.22 02:01:19.404298 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(87ae3da2d13cef5a322b92f861de6988) : [0, 228]: Setting download as failed: File segment: [0, 228], key: 87ae3da2d13cef5a322b92f861de6988, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:19.410279 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(87ae3da2d13cef5a322b92f861de6988): No space left in cache to reserve 229 bytes, will continue without cache download 2024.10.22 02:01:19.436274 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 83190 2024.10.22 02:01:19.446732 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_741_87} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:19.473763 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_741_87} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_740_86, total 23537 rows starting from the beginning of the part 2024.10.22 02:01:19.606125 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_741_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_741_741_0, total 89 rows starting from the beginning of the part 2024.10.22 02:01:19.643017 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1176_16, 202410_1177_1177_0, 202410_1178_1178_0, 202410_1179_1179_0, 202410_1180_1180_0, 202410_1181_1181_0] 2024.10.22 02:01:19.670676 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1176_16, 202410_1177_1177_0, 202410_1178_1178_0, 202410_1179_1179_0, 202410_1180_1180_0, 202410_1181_1181_0] 2024.10.22 02:01:19.696293 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1176_16, 202410_1177_1177_0, 202410_1178_1178_0, 202410_1179_1179_0, 202410_1180_1180_0, 202410_1181_1181_0] 2024.10.22 02:01:19.958748 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 6.84 MiB, will set to 1.32 GiB (RSS), difference: -11.83 MiB 2024.10.22 02:01:20.252173 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_761_162} MergeTask::MergeProjectionsStage: Merge sorted 11499 rows, containing 12 columns (12 merged, 0 gathered) in 1.683610291 sec., 6829.965379440651 rows/sec., 1.06 MiB/sec. 2024.10.22 02:01:20.313646 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_761_162} MergedBlockOutputStream: filled checksums 202410_1_761_162 (state Temporary) 2024.10.22 02:01:20.374998 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ecb76e7d5cec4afc9e0709d10ba701bf) : [0, 172735]: Setting download as failed: File segment: [0, 172735], key: ecb76e7d5cec4afc9e0709d10ba701bf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:20.377224 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ecb76e7d5cec4afc9e0709d10ba701bf): No space left in cache to reserve 172736 bytes, will continue without cache download 2024.10.22 02:01:20.475451 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} MergeTask::MergeProjectionsStage: Merge sorted 662 rows, containing 32 columns (32 merged, 0 gathered) in 3.560451188 sec., 185.93149155679424 rows/sec., 16.90 KiB/sec. 2024.10.22 02:01:20.506966 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_151_151_0 (state Temporary) 2024.10.22 02:01:20.634743 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} MergedBlockOutputStream: filled checksums 202410_1_618_126 (state Temporary) 2024.10.22 02:01:20.657954 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ecb76e7d5cec4afc9e0709d10ba701bf, offset: 0, size: 0 2024.10.22 02:01:20.659856 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ecb76e7d5cec4afc9e0709d10ba701bf for removal 2024.10.22 02:01:20.662836 [ 1476 ] {} CacheMetadata: Key ecb76e7d5cec4afc9e0709d10ba701bf is removed from metadata 2024.10.22 02:01:20.664074 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 87ae3da2d13cef5a322b92f861de6988, offset: 0, size: 0 2024.10.22 02:01:20.668555 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 87ae3da2d13cef5a322b92f861de6988 for removal 2024.10.22 02:01:20.675519 [ 1476 ] {} CacheMetadata: Key 87ae3da2d13cef5a322b92f861de6988 is removed from metadata 2024.10.22 02:01:20.733524 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_761_162} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_761_162 to 202410_1_761_162 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:20.740157 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_761_162} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_760_161, 202410_761_761_0] -> 202410_1_761_162 2024.10.22 02:01:20.766396 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:01:21.014754 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_618_126 to 202410_1_618_126 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:21.030237 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_618_126} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_613_125, 202410_618_618_0] -> 202410_1_618_126 2024.10.22 02:01:21.146017 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:01:21.148111 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:01:21.191697 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:01:21.543796 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(471d34429378c4d6de4c6618bae51188) : [0, 231]: Setting download as failed: File segment: [0, 231], key: 471d34429378c4d6de4c6618bae51188, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:21.546023 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(471d34429378c4d6de4c6618bae51188): No space left in cache to reserve 232 bytes, will continue without cache download 2024.10.22 02:01:21.663294 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 6.20 MiB, will set to 1.33 GiB (RSS), difference: -25.84 MiB 2024.10.22 02:01:21.733706 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_151_151_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:01:21.848987 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_741_87} MergeTask::MergeProjectionsStage: Merge sorted 23626 rows, containing 13 columns (13 merged, 0 gathered) in 2.551917045 sec., 9258.137934495437 rows/sec., 4.01 MiB/sec. 2024.10.22 02:01:21.942680 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_741_87} MergedBlockOutputStream: filled checksums 202410_507_741_87 (state Temporary) 2024.10.22 02:01:22.343204 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6606cc53db3f87f88be280fb624dc551) : [0, 296993]: Setting download as failed: File segment: [0, 296993], key: 6606cc53db3f87f88be280fb624dc551, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:22.346567 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6606cc53db3f87f88be280fb624dc551): No space left in cache to reserve 296994 bytes, will continue without cache download 2024.10.22 02:01:22.368319 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 83872 2024.10.22 02:01:22.574244 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_741_87} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_741_87 to 202410_507_741_87 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:22.580619 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_741_87} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_740_86, 202410_741_741_0] -> 202410_507_741_87 2024.10.22 02:01:22.631101 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.51 MiB. 2024.10.22 02:01:22.682426 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6606cc53db3f87f88be280fb624dc551, offset: 0, size: 0 2024.10.22 02:01:22.684940 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6606cc53db3f87f88be280fb624dc551 for removal 2024.10.22 02:01:22.691467 [ 1476 ] {} CacheMetadata: Key 6606cc53db3f87f88be280fb624dc551 is removed from metadata 2024.10.22 02:01:22.691844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 471d34429378c4d6de4c6618bae51188, offset: 0, size: 0 2024.10.22 02:01:22.695329 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 471d34429378c4d6de4c6618bae51188 for removal 2024.10.22 02:01:22.698627 [ 1476 ] {} CacheMetadata: Key 471d34429378c4d6de4c6618bae51188 is removed from metadata 2024.10.22 02:01:22.840090 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:22.844655 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:22.907319 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1269_1269_0 (state Temporary) 2024.10.22 02:01:23.074156 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1269_1269_0 to 202410_1282_1282_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:23.214503 [ 1208 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1277_14 to 202410_1282_1282_0 2024.10.22 02:01:23.243363 [ 1208 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:23.245815 [ 1208 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:23.273195 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1277_14 to 202410_1282_1282_0 into Compact with storage Full 2024.10.22 02:01:23.299287 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(eaeee72668b18709b857b580a06fd1a4) : [0, 166]: Setting download as failed: File segment: [0, 166], key: eaeee72668b18709b857b580a06fd1a4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:23.301381 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(eaeee72668b18709b857b580a06fd1a4): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 02:01:23.439928 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83872 2024.10.22 02:01:23.453591 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:23.471187 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1277_14, total 4370 rows starting from the beginning of the part 2024.10.22 02:01:23.568547 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1278_1278_0, total 87 rows starting from the beginning of the part 2024.10.22 02:01:23.871972 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1279_1279_0, total 50 rows starting from the beginning of the part 2024.10.22 02:01:24.061617 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1280_1280_0, total 85 rows starting from the beginning of the part 2024.10.22 02:01:24.136565 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1281_1281_0, total 57 rows starting from the beginning of the part 2024.10.22 02:01:24.194850 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1282_1282_0, total 82 rows starting from the beginning of the part 2024.10.22 02:01:24.324923 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 2.00 GiB, free memory in arenas 4.35 MiB, will set to 1.32 GiB (RSS), difference: 28.72 MiB 2024.10.22 02:01:24.385624 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dce48bcbfe498364160819f5efe2eccd) : [0, 147762]: Setting download as failed: File segment: [0, 147762], key: dce48bcbfe498364160819f5efe2eccd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:24.389604 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dce48bcbfe498364160819f5efe2eccd): No space left in cache to reserve 147763 bytes, will continue without cache download 2024.10.22 02:01:24.391302 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 627 2024.10.22 02:01:24.433067 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dce48bcbfe498364160819f5efe2eccd, offset: 0, size: 0 2024.10.22 02:01:24.435746 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dce48bcbfe498364160819f5efe2eccd for removal 2024.10.22 02:01:24.441363 [ 1476 ] {} CacheMetadata: Key dce48bcbfe498364160819f5efe2eccd is removed from metadata 2024.10.22 02:01:24.442912 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: eaeee72668b18709b857b580a06fd1a4, offset: 0, size: 0 2024.10.22 02:01:24.447255 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key eaeee72668b18709b857b580a06fd1a4 for removal 2024.10.22 02:01:24.451219 [ 1476 ] {} CacheMetadata: Key eaeee72668b18709b857b580a06fd1a4 is removed from metadata 2024.10.22 02:01:24.976196 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:24.992486 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:25.152775 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1093875 2024.10.22 02:01:25.161595 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(284c87fd2262dd3b1d6ac33faff16ed2) : [0, 225]: Setting download as failed: File segment: [0, 225], key: 284c87fd2262dd3b1d6ac33faff16ed2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:25.164241 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(284c87fd2262dd3b1d6ac33faff16ed2): No space left in cache to reserve 226 bytes, will continue without cache download 2024.10.22 02:01:25.257320 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_609_609_0 (state Temporary) 2024.10.22 02:01:25.360461 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_609_609_0 to 202410_619_619_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:25.485757 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 627 2024.10.22 02:01:25.945636 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:25.948025 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:26.165700 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 11472 2024.10.22 02:01:26.266732 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} MergeTask::MergeProjectionsStage: Merge sorted 4731 rows, containing 13 columns (13 merged, 0 gathered) in 3.006745234 sec., 1573.46220973506 rows/sec., 272.24 KiB/sec. 2024.10.22 02:01:26.269104 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_721_721_0 (state Temporary) 2024.10.22 02:01:26.283410 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:26.288262 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:26.334924 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2ff30dedfeea859bb4679bb7a5b8690c) : [0, 681615]: Setting download as failed: File segment: [0, 681615], key: 2ff30dedfeea859bb4679bb7a5b8690c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:26.338267 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2ff30dedfeea859bb4679bb7a5b8690c): No space left in cache to reserve 681616 bytes, will continue without cache download 2024.10.22 02:01:26.430234 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_753_753_0 (state Temporary) 2024.10.22 02:01:26.448756 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_721_721_0 to 202410_728_728_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:26.459981 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} MergedBlockOutputStream: filled checksums 202410_1207_1282_15 (state Temporary) 2024.10.22 02:01:26.488054 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2ff30dedfeea859bb4679bb7a5b8690c, offset: 0, size: 0 2024.10.22 02:01:26.492347 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2ff30dedfeea859bb4679bb7a5b8690c for removal 2024.10.22 02:01:26.502021 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1093875 2024.10.22 02:01:26.503801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 284c87fd2262dd3b1d6ac33faff16ed2, offset: 0, size: 0 2024.10.22 02:01:26.508316 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 284c87fd2262dd3b1d6ac33faff16ed2 for removal 2024.10.22 02:01:26.486671 [ 1254 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_718_137 to 202410_719_724_1 2024.10.22 02:01:26.559730 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 4.13 MiB, will set to 1.32 GiB (RSS), difference: -9.58 MiB 2024.10.22 02:01:26.599617 [ 1476 ] {} CacheMetadata: Key 2ff30dedfeea859bb4679bb7a5b8690c is removed from metadata 2024.10.22 02:01:26.621687 [ 1476 ] {} CacheMetadata: Key 284c87fd2262dd3b1d6ac33faff16ed2 is removed from metadata 2024.10.22 02:01:26.628516 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_753_753_0 to 202410_766_766_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:26.687344 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11472 2024.10.22 02:01:26.707262 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1282_15 to 202410_1207_1282_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:26.707937 [ 1198 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_761_162 to 202410_762_762_0 2024.10.22 02:01:26.716867 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1282_15} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1277_14, 202410_1282_1282_0] -> 202410_1207_1282_15 2024.10.22 02:01:26.720543 [ 1198 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:26.722694 [ 1254 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:26.725510 [ 1198 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:26.747685 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:26.757415 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_762_163} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_761_162 to 202410_762_762_0 into Compact with storage Full 2024.10.22 02:01:26.758081 [ 1254 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 296 ms. 2024.10.22 02:01:26.783407 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:01:26.793691 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_762_163} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:26.801555 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_724_138} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_718_137 to 202410_719_724_1 into Wide with storage Full 2024.10.22 02:01:26.908496 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_762_163} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_761_162, total 11499 rows starting from the beginning of the part 2024.10.22 02:01:26.956520 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_762_163} MergeTreeSequentialSource: Reading 2 marks from part 202410_762_762_0, total 13 rows starting from the beginning of the part 2024.10.22 02:01:26.962164 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 165 entries to flush up to offset 83355 2024.10.22 02:01:27.023448 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_724_138} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:27.031869 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_724_138} MergeTreeSequentialSource: Reading 135 marks from part 202410_1_718_137, total 1094375 rows starting from the beginning of the part 2024.10.22 02:01:27.059100 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_724_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_719_724_1, total 7315 rows starting from the beginning of the part 2024.10.22 02:01:27.104454 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:27.110833 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:27.293243 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_740_740_0 (state Temporary) 2024.10.22 02:01:27.450736 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 90 entries to flush up to offset 83962 2024.10.22 02:01:27.578311 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_151_151_0 to 202410_154_154_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:27.634700 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a6586e019a4491cf570b26cd11a97853) : [0, 257]: Setting download as failed: File segment: [0, 257], key: a6586e019a4491cf570b26cd11a97853, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:27.637353 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a6586e019a4491cf570b26cd11a97853): No space left in cache to reserve 258 bytes, will continue without cache download 2024.10.22 02:01:27.653061 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_740_740_0 to 202410_746_746_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:27.673163 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_741_87 to 202410_742_742_0 2024.10.22 02:01:27.699350 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 02:01:27.702177 [ 1268 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:27.700484 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_762_163} MergeTask::MergeProjectionsStage: Merge sorted 11512 rows, containing 12 columns (12 merged, 0 gathered) in 0.941457199 sec., 12227.852750213024 rows/sec., 1.90 MiB/sec. 2024.10.22 02:01:27.735208 [ 1255 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 7 parts from 202410_1_144_25 to 202410_154_154_0 2024.10.22 02:01:27.750468 [ 1255 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 2.93 MiB using storage policy from min volume index 0 2024.10.22 02:01:27.752290 [ 1255 ] {} DiskLocal: Reserved 2.93 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:27.758179 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_742_88} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_741_87 to 202410_742_742_0 into Wide with storage Full 2024.10.22 02:01:27.788988 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} MergeTask::PrepareStage: Merging 7 parts: from 202410_1_144_25 to 202410_154_154_0 into Compact with storage Full 2024.10.22 02:01:27.800496 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 83355 2024.10.22 02:01:27.817256 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:27.824510 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:27.845394 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_762_163} MergedBlockOutputStream: filled checksums 202410_1_762_163 (state Temporary) 2024.10.22 02:01:27.887200 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_742_88} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:27.935791 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_762_163} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_762_163 to 202410_1_762_163 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:27.949523 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_762_163} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_761_162, 202410_762_762_0] -> 202410_1_762_163 2024.10.22 02:01:27.978126 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:01:28.001812 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_742_88} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_741_87, total 23626 rows starting from the beginning of the part 2024.10.22 02:01:28.099549 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1270_1270_0 (state Temporary) 2024.10.22 02:01:28.170281 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_742_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_742_742_0, total 104 rows starting from the beginning of the part 2024.10.22 02:01:28.195605 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1270_1270_0 to 202410_1283_1283_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:28.259085 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 83962 2024.10.22 02:01:28.418346 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(b1a98b3fafe8351382417808b0afb7f5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b1a98b3fafe8351382417808b0afb7f5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:01:28.425588 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 867630, file segment info: File segment: [0, 33554431], key: b1a98b3fafe8351382417808b0afb7f5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:01:28.430261 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:28.498809 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 5.16 MiB, will set to 1.32 GiB (RSS), difference: 4.44 MiB 2024.10.22 02:01:29.116227 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a88fb34be907706abb4c0d4d9920cc5b) : [0, 1942465]: Setting download as failed: File segment: [0, 1942465], key: a88fb34be907706abb4c0d4d9920cc5b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:29.120033 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a88fb34be907706abb4c0d4d9920cc5b): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:01:29.207302 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a88fb34be907706abb4c0d4d9920cc5b, offset: 0, size: 0 2024.10.22 02:01:29.210688 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a88fb34be907706abb4c0d4d9920cc5b for removal 2024.10.22 02:01:29.215471 [ 1476 ] {} CacheMetadata: Key a88fb34be907706abb4c0d4d9920cc5b is removed from metadata 2024.10.22 02:01:29.221681 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a6586e019a4491cf570b26cd11a97853, offset: 0, size: 0 2024.10.22 02:01:29.228287 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a6586e019a4491cf570b26cd11a97853 for removal 2024.10.22 02:01:29.235810 [ 1476 ] {} CacheMetadata: Key a6586e019a4491cf570b26cd11a97853 is removed from metadata 2024.10.22 02:01:29.865823 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(9dd08dac8d0e2401e3a5dd51adc0ed95) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9dd08dac8d0e2401e3a5dd51adc0ed95, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:01:29.894830 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 101568, file segment info: File segment: [0, 33554431], key: 9dd08dac8d0e2401e3a5dd51adc0ed95, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:01:29.906287 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:29.915479 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_658_127, 202410_659_664_1] 2024.10.22 02:01:29.920701 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_658_127, 202410_659_664_1] 2024.10.22 02:01:30.041562 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_658_127, 202410_659_664_1] 2024.10.22 02:01:30.526694 [ 1261 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 244 ms. 2024.10.22 02:01:30.654452 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(657b9926dda7bc432671e0bc5c3e5dc2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 657b9926dda7bc432671e0bc5c3e5dc2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:01:30.664418 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1046, file segment info: File segment: [0, 33554431], key: 657b9926dda7bc432671e0bc5c3e5dc2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:01:30.665732 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:30.715579 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6078 2024.10.22 02:01:31.082743 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:01:31.144616 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:01:31.045509 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_742_88} MergeTask::MergeProjectionsStage: Merge sorted 23730 rows, containing 13 columns (13 merged, 0 gathered) in 3.294531743 sec., 7202.844547004263 rows/sec., 3.12 MiB/sec. 2024.10.22 02:01:31.203694 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:01:31.203984 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:01:31.224631 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:01:31.262661 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:01:31.276608 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:01:31.313126 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:01:31.352218 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:01:31.478818 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_724_138/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:01:31.488488 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/idd/chzqcgqmhecpjmwabcopsmktsacjg (LogSeriesLimiter: on interval from 2024-10-22 02:01:05 to 2024-10-22 02:01:31 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:01:31.571587 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_742_88} MergedBlockOutputStream: filled checksums 202410_507_742_88 (state Temporary) 2024.10.22 02:01:31.571890 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(08e70dd39ea98614a8cef82ed36fbb40) : [0, 164]: Setting download as failed: File segment: [0, 164], key: 08e70dd39ea98614a8cef82ed36fbb40, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:31.575318 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(08e70dd39ea98614a8cef82ed36fbb40): No space left in cache to reserve 165 bytes, will continue without cache download 2024.10.22 02:01:31.743907 [ 1223 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 02:01:31.795000 [ 1217 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:01:31.818108 [ 1217 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:01:32.102693 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_742_88} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_742_88 to 202410_507_742_88 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:32.118986 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_742_88} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_741_87, 202410_742_742_0] -> 202410_507_742_88 2024.10.22 02:01:32.151693 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.54 MiB. 2024.10.22 02:01:32.234138 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a5efe52cc12a0e595b951b34badad8d5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a5efe52cc12a0e595b951b34badad8d5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:01:32.251068 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 682, file segment info: File segment: [0, 33554431], key: a5efe52cc12a0e595b951b34badad8d5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:01:32.252474 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:32.264580 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 84036 2024.10.22 02:01:32.391314 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 13.51 MiB, will set to 1.32 GiB (RSS), difference: 12.24 MiB 2024.10.22 02:01:32.459518 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:32.461360 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:32.504703 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1271_1271_0 (state Temporary) 2024.10.22 02:01:32.681010 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1271_1271_0 to 202410_1284_1284_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:32.742491 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84036 2024.10.22 02:01:32.866173 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(35322e5a8a706d74ccb1595033794bb1) : [0, 178929]: Setting download as failed: File segment: [0, 178929], key: 35322e5a8a706d74ccb1595033794bb1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:32.867415 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(35322e5a8a706d74ccb1595033794bb1): No space left in cache to reserve 178930 bytes, will continue without cache download 2024.10.22 02:01:32.989776 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 628 2024.10.22 02:01:32.996720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 35322e5a8a706d74ccb1595033794bb1, offset: 0, size: 0 2024.10.22 02:01:32.999446 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 35322e5a8a706d74ccb1595033794bb1 for removal 2024.10.22 02:01:33.007838 [ 1476 ] {} CacheMetadata: Key 35322e5a8a706d74ccb1595033794bb1 is removed from metadata 2024.10.22 02:01:33.009065 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 08e70dd39ea98614a8cef82ed36fbb40, offset: 0, size: 0 2024.10.22 02:01:33.012343 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 08e70dd39ea98614a8cef82ed36fbb40 for removal 2024.10.22 02:01:33.015612 [ 1476 ] {} CacheMetadata: Key 08e70dd39ea98614a8cef82ed36fbb40 is removed from metadata 2024.10.22 02:01:33.483631 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:33.489011 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:33.511223 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1094645 2024.10.22 02:01:33.730255 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(5eb64c19f39046b07406b4d9b888b91d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5eb64c19f39046b07406b4d9b888b91d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:01:33.747779 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 29724, file segment info: File segment: [0, 33554431], key: 5eb64c19f39046b07406b4d9b888b91d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:01:33.752934 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:33.788555 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_610_610_0 (state Temporary) 2024.10.22 02:01:34.098601 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:34.101411 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:34.131476 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_610_610_0 to 202410_620_620_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:34.191194 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 628 2024.10.22 02:01:34.193389 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 11492 2024.10.22 02:01:34.503013 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:34.512207 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:34.525647 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_722_722_0 (state Temporary) 2024.10.22 02:01:34.546228 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b5b7fc102223734a1e8426c910debc5c) : [0, 159]: Setting download as failed: File segment: [0, 159], key: b5b7fc102223734a1e8426c910debc5c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:34.549356 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b5b7fc102223734a1e8426c910debc5c): No space left in cache to reserve 160 bytes, will continue without cache download 2024.10.22 02:01:34.568010 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_754_754_0 (state Temporary) 2024.10.22 02:01:34.748867 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_754_754_0 to 202410_767_767_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:34.749385 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_722_722_0 to 202410_729_729_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:34.770223 [ 1167 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_762_163 to 202410_767_767_0 2024.10.22 02:01:34.814225 [ 1167 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:34.819412 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:34.872761 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_762_163 to 202410_767_767_0 into Compact with storage Full 2024.10.22 02:01:34.894718 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1094645 2024.10.22 02:01:34.904151 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11492 2024.10.22 02:01:35.004289 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:35.029775 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:35.037075 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_762_163, total 11512 rows starting from the beginning of the part 2024.10.22 02:01:35.056343 [ 1253 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 02:01:35.058473 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_763_763_0, total 15 rows starting from the beginning of the part 2024.10.22 02:01:35.195423 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_764_764_0, total 19 rows starting from the beginning of the part 2024.10.22 02:01:35.239016 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_765_765_0, total 14 rows starting from the beginning of the part 2024.10.22 02:01:35.270151 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(de4f1b99ee9a92393f79a307bbf89006) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: de4f1b99ee9a92393f79a307bbf89006, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:01:35.273811 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 605, file segment info: File segment: [0, 33554431], key: de4f1b99ee9a92393f79a307bbf89006, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:01:35.278783 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:35.341698 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 195 entries to flush up to offset 83550 2024.10.22 02:01:35.380596 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_766_766_0, total 18 rows starting from the beginning of the part 2024.10.22 02:01:35.420627 [ 1678 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kfv/mmdzlqjgnpuevmpmrxuimybbzvrbv, size 29724 2024.10.22 02:01:35.543337 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_767_767_0, total 20 rows starting from the beginning of the part 2024.10.22 02:01:35.669724 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:01:35.679316 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:01:35.717143 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} MergeTreeSequentialSource: Reading 4 marks from part 202410_1_144_25, total 5727 rows starting from the beginning of the part 2024.10.22 02:01:35.821677 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:01:35.847637 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:01:35.874640 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:01:35.897047 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2f4e0363a3fe23bb247ec9112a97a67b) : [0, 8524]: Setting download as failed: File segment: [0, 8524], key: 2f4e0363a3fe23bb247ec9112a97a67b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:35.901775 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2f4e0363a3fe23bb247ec9112a97a67b): No space left in cache to reserve 8525 bytes, will continue without cache download 2024.10.22 02:01:35.903589 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:35.912812 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:01:35.914301 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:35.950672 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:01:35.984499 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:01:35.955800 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2f4e0363a3fe23bb247ec9112a97a67b, offset: 0, size: 0 2024.10.22 02:01:35.993042 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2f4e0363a3fe23bb247ec9112a97a67b for removal 2024.10.22 02:01:36.002278 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b5b7fc102223734a1e8426c910debc5c, offset: 0, size: 0 2024.10.22 02:01:36.009092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b5b7fc102223734a1e8426c910debc5c for removal 2024.10.22 02:01:36.016182 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_741_741_0 (state Temporary) 2024.10.22 02:01:36.092064 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_11_11_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:01:36.107835 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fcv/xsqdanfzpjtlujwaihmeejyifktgy (LogSeriesLimiter: on interval from 2024-10-22 02:01:31 to 2024-10-22 02:01:36 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:01:36.122957 [ 1476 ] {} CacheMetadata: Key 2f4e0363a3fe23bb247ec9112a97a67b is removed from metadata 2024.10.22 02:01:36.128194 [ 1476 ] {} CacheMetadata: Key b5b7fc102223734a1e8426c910debc5c is removed from metadata 2024.10.22 02:01:36.176386 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_741_741_0 to 202410_747_747_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:36.212227 [ 1228 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_742_88 to 202410_743_743_0 2024.10.22 02:01:36.225662 [ 1228 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 02:01:36.229646 [ 1228 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:36.231829 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 83550 2024.10.22 02:01:36.264470 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_743_89} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_742_88 to 202410_743_743_0 into Wide with storage Full 2024.10.22 02:01:36.467387 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_743_89} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:36.498433 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_743_89} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_742_88, total 23730 rows starting from the beginning of the part 2024.10.22 02:01:36.668130 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 14.34 MiB, will set to 1.33 GiB (RSS), difference: -8.16 MiB 2024.10.22 02:01:36.756431 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_743_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_743_743_0, total 84 rows starting from the beginning of the part 2024.10.22 02:01:36.757407 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 84123 2024.10.22 02:01:37.006792 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:37.014568 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:37.165778 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f36f42be6b136de9762801e5b65af24d) : [0, 257]: Setting download as failed: File segment: [0, 257], key: f36f42be6b136de9762801e5b65af24d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:37.168921 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f36f42be6b136de9762801e5b65af24d): No space left in cache to reserve 258 bytes, will continue without cache download 2024.10.22 02:01:37.293339 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} MergeTask::MergeProjectionsStage: Merge sorted 11598 rows, containing 12 columns (12 merged, 0 gathered) in 2.425824318 sec., 4781.055212424497 rows/sec., 762.35 KiB/sec. 2024.10.22 02:01:37.392025 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1272_1272_0 (state Temporary) 2024.10.22 02:01:37.459981 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1272_1272_0 to 202410_1285_1285_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:37.460752 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} MergedBlockOutputStream: filled checksums 202410_1_767_164 (state Temporary) 2024.10.22 02:01:37.614649 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84123 2024.10.22 02:01:37.635335 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_767_164 to 202410_1_767_164 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:37.744597 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_767_164} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_762_163, 202410_767_767_0] -> 202410_1_767_164 2024.10.22 02:01:37.884329 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.54 MiB. 2024.10.22 02:01:38.087093 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_145_149_1, total 213 rows starting from the beginning of the part 2024.10.22 02:01:38.298542 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 6117 2024.10.22 02:01:38.299813 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_703_142, 202410_704_704_0] 2024.10.22 02:01:38.303958 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_703_142, 202410_704_704_0] 2024.10.22 02:01:38.327913 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d6818ceaa745ec3f8b6608209fd2271d) : [0, 2027648]: Setting download as failed: File segment: [0, 2027648], key: d6818ceaa745ec3f8b6608209fd2271d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:38.332115 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_703_142, 202410_704_704_0] 2024.10.22 02:01:38.332376 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d6818ceaa745ec3f8b6608209fd2271d): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:01:38.469158 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 15.85 MiB, will set to 1.33 GiB (RSS), difference: -51.52 MiB 2024.10.22 02:01:38.680657 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d6818ceaa745ec3f8b6608209fd2271d, offset: 0, size: 0 2024.10.22 02:01:38.683928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d6818ceaa745ec3f8b6608209fd2271d for removal 2024.10.22 02:01:38.687884 [ 1476 ] {} CacheMetadata: Key d6818ceaa745ec3f8b6608209fd2271d is removed from metadata 2024.10.22 02:01:38.695458 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f36f42be6b136de9762801e5b65af24d, offset: 0, size: 0 2024.10.22 02:01:38.697650 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f36f42be6b136de9762801e5b65af24d for removal 2024.10.22 02:01:38.700208 [ 1476 ] {} CacheMetadata: Key f36f42be6b136de9762801e5b65af24d is removed from metadata 2024.10.22 02:01:39.561816 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(83f2cac393005d73f8ae681d3a59b3df) : [0, 166]: Setting download as failed: File segment: [0, 166], key: 83f2cac393005d73f8ae681d3a59b3df, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:39.563521 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(83f2cac393005d73f8ae681d3a59b3df): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 02:01:39.831925 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_743_89} MergeTask::MergeProjectionsStage: Merge sorted 23814 rows, containing 13 columns (13 merged, 0 gathered) in 3.572511603 sec., 6665.89857398988 rows/sec., 2.89 MiB/sec. 2024.10.22 02:01:39.997767 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_743_89} MergedBlockOutputStream: filled checksums 202410_507_743_89 (state Temporary) 2024.10.22 02:01:40.372012 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 15.46 MiB, will set to 1.33 GiB (RSS), difference: 16.85 MiB 2024.10.22 02:01:40.445225 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_150_150_0, total 39 rows starting from the beginning of the part 2024.10.22 02:01:40.541437 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(495065e98362861e36a741fa3b37865a) : [0, 173755]: Setting download as failed: File segment: [0, 173755], key: 495065e98362861e36a741fa3b37865a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:40.545703 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(495065e98362861e36a741fa3b37865a): No space left in cache to reserve 173756 bytes, will continue without cache download 2024.10.22 02:01:40.566296 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 495065e98362861e36a741fa3b37865a, offset: 0, size: 0 2024.10.22 02:01:40.567876 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 495065e98362861e36a741fa3b37865a for removal 2024.10.22 02:01:40.571055 [ 1476 ] {} CacheMetadata: Key 495065e98362861e36a741fa3b37865a is removed from metadata 2024.10.22 02:01:40.573283 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 83f2cac393005d73f8ae681d3a59b3df, offset: 0, size: 0 2024.10.22 02:01:40.577078 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 83f2cac393005d73f8ae681d3a59b3df for removal 2024.10.22 02:01:40.583762 [ 1476 ] {} CacheMetadata: Key 83f2cac393005d73f8ae681d3a59b3df is removed from metadata 2024.10.22 02:01:40.608816 [ 1187 ] {} DatabaseCatalog: Execution took 278 ms. 2024.10.22 02:01:41.147866 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:01:41.150386 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:01:41.150761 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_743_89} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_743_89 to 202410_507_743_89 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:41.154911 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_743_89} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_742_88, 202410_743_743_0] -> 202410_507_743_89 2024.10.22 02:01:41.174195 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.51 MiB. 2024.10.22 02:01:41.440841 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(25003c8bdf5694658df5b63604a73ffa) : [0, 165]: Setting download as failed: File segment: [0, 165], key: 25003c8bdf5694658df5b63604a73ffa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:41.443088 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(25003c8bdf5694658df5b63604a73ffa): No space left in cache to reserve 166 bytes, will continue without cache download 2024.10.22 02:01:41.601512 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_684_38, 202410_685_685_0] 2024.10.22 02:01:41.606265 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_684_38, 202410_685_685_0] 2024.10.22 02:01:41.646364 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 84175 2024.10.22 02:01:41.695338 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 629 2024.10.22 02:01:41.723187 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_684_38, 202410_685_685_0] 2024.10.22 02:01:41.767295 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_151_151_0, total 35 rows starting from the beginning of the part 2024.10.22 02:01:41.899737 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1096185 2024.10.22 02:01:41.917410 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:41.923793 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:41.963385 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:41.966090 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:42.058990 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_724_138/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:01:42.083561 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_611_611_0 (state Temporary) 2024.10.22 02:01:42.170917 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1273_1273_0 (state Temporary) 2024.10.22 02:01:42.262763 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_611_611_0 to 202410_621_621_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:42.315956 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(788b2c955750af3b115f153065c65697) : [0, 144792]: Setting download as failed: File segment: [0, 144792], key: 788b2c955750af3b115f153065c65697, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:42.318703 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(788b2c955750af3b115f153065c65697): No space left in cache to reserve 144793 bytes, will continue without cache download 2024.10.22 02:01:42.345393 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1273_1273_0 to 202410_1286_1286_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:42.362701 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 788b2c955750af3b115f153065c65697, offset: 0, size: 0 2024.10.22 02:01:42.364482 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 788b2c955750af3b115f153065c65697 for removal 2024.10.22 02:01:42.368229 [ 1476 ] {} CacheMetadata: Key 788b2c955750af3b115f153065c65697 is removed from metadata 2024.10.22 02:01:42.369104 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 25003c8bdf5694658df5b63604a73ffa, offset: 0, size: 0 2024.10.22 02:01:42.370669 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 25003c8bdf5694658df5b63604a73ffa for removal 2024.10.22 02:01:42.374563 [ 1476 ] {} CacheMetadata: Key 25003c8bdf5694658df5b63604a73ffa is removed from metadata 2024.10.22 02:01:42.414331 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 11508 2024.10.22 02:01:42.449355 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 629 2024.10.22 02:01:42.502904 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 2.00 GiB, free memory in arenas 10.41 MiB, will set to 1.33 GiB (RSS), difference: 37.00 MiB 2024.10.22 02:01:42.546839 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84175 2024.10.22 02:01:42.551786 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:42.555377 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:42.618105 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Found 5 old parts to remove. Parts: [202410_1_30_7, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0] 2024.10.22 02:01:42.624887 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from filesystem (serially): Parts: [202410_1_30_7, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0] 2024.10.22 02:01:42.661509 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from memory: Parts: [202410_1_30_7, 202410_31_31_0, 202410_32_32_0, 202410_33_33_0, 202410_34_34_0] 2024.10.22 02:01:42.681475 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_755_755_0 (state Temporary) 2024.10.22 02:01:42.697948 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:42.701743 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:42.844963 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_755_755_0 to 202410_768_768_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:42.895391 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11508 2024.10.22 02:01:42.948833 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_723_723_0 (state Temporary) 2024.10.22 02:01:43.128928 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_723_723_0 to 202410_730_730_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:43.169637 [ 1207 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_725_725_0 to 202410_730_730_0 2024.10.22 02:01:43.185348 [ 1207 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:43.194560 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1096185 2024.10.22 02:01:43.226251 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_152_152_0, total 41 rows starting from the beginning of the part 2024.10.22 02:01:43.296106 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:43.503291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ca3b4e0a8d05764e493122148922debb) : [0, 166]: Setting download as failed: File segment: [0, 166], key: ca3b4e0a8d05764e493122148922debb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:43.505122 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ca3b4e0a8d05764e493122148922debb): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 02:01:43.548994 [ 1211 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 244 ms. 2024.10.22 02:01:43.550068 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_725_725_0 to 202410_730_730_0 into Compact with storage Full 2024.10.22 02:01:43.565278 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:43.570935 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_725_725_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:01:43.580798 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_726_726_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:01:43.598027 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_727_727_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:01:43.615543 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_728_728_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:01:43.640882 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_729_729_0, total 770 rows starting from the beginning of the part 2024.10.22 02:01:43.654418 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_730_730_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:01:43.752055 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 194 entries to flush up to offset 83744 2024.10.22 02:01:43.925804 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:43.932487 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:44.074800 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_742_742_0 (state Temporary) 2024.10.22 02:01:44.128659 [ 1269 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 249 ms. 2024.10.22 02:01:44.151875 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_742_742_0 to 202410_748_748_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:44.190552 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_743_89 to 202410_744_744_0 2024.10.22 02:01:44.191074 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 83744 2024.10.22 02:01:44.305781 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 02:01:44.311320 [ 1221 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:44.350823 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_744_90} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_743_89 to 202410_744_744_0 into Wide with storage Full 2024.10.22 02:01:44.430540 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_744_90} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:44.454297 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_744_90} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_743_89, total 23814 rows starting from the beginning of the part 2024.10.22 02:01:44.554194 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} MergeTask::MergeProjectionsStage: Merge sorted 8085 rows, containing 4 columns (4 merged, 0 gathered) in 1.011910044 sec., 7989.8406463489955 rows/sec., 124.84 KiB/sec. 2024.10.22 02:01:44.566952 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(171873a47c94af29ab872225c6528162) : [0, 142724]: Setting download as failed: File segment: [0, 142724], key: 171873a47c94af29ab872225c6528162, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:44.569091 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(171873a47c94af29ab872225c6528162): No space left in cache to reserve 142725 bytes, will continue without cache download 2024.10.22 02:01:44.652733 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} MergedBlockOutputStream: filled checksums 202410_725_730_1 (state Temporary) 2024.10.22 02:01:44.800639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 171873a47c94af29ab872225c6528162, offset: 0, size: 0 2024.10.22 02:01:44.803278 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 171873a47c94af29ab872225c6528162 for removal 2024.10.22 02:01:44.807458 [ 1476 ] {} CacheMetadata: Key 171873a47c94af29ab872225c6528162 is removed from metadata 2024.10.22 02:01:44.809233 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ca3b4e0a8d05764e493122148922debb, offset: 0, size: 0 2024.10.22 02:01:44.811836 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ca3b4e0a8d05764e493122148922debb for removal 2024.10.22 02:01:44.815124 [ 1476 ] {} CacheMetadata: Key ca3b4e0a8d05764e493122148922debb is removed from metadata 2024.10.22 02:01:44.787385 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_725_730_1 to 202410_725_730_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:44.908587 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_725_730_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_725_725_0, 202410_730_730_0] -> 202410_725_730_1 2024.10.22 02:01:44.946808 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_744_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_744_744_0, total 102 rows starting from the beginning of the part 2024.10.22 02:01:44.967172 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_153_153_0, total 38 rows starting from the beginning of the part 2024.10.22 02:01:44.971445 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:01:45.904085 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 2.00 GiB, free memory in arenas 11.80 MiB, will set to 1.33 GiB (RSS), difference: -45.11 MiB 2024.10.22 02:01:46.051283 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3c2a814a5d05bdcca1326be63100f2fd) : [0, 191]: Setting download as failed: File segment: [0, 191], key: 3c2a814a5d05bdcca1326be63100f2fd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:46.053605 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3c2a814a5d05bdcca1326be63100f2fd): No space left in cache to reserve 192 bytes, will continue without cache download 2024.10.22 02:01:46.560755 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 84260 2024.10.22 02:01:46.666050 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1181_17, 202410_1182_1182_0, 202410_1183_1183_0, 202410_1184_1184_0, 202410_1185_1185_0, 202410_1186_1186_0] 2024.10.22 02:01:46.668038 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1181_17, 202410_1182_1182_0, 202410_1183_1183_0, 202410_1184_1184_0, 202410_1185_1185_0, 202410_1186_1186_0] 2024.10.22 02:01:46.706130 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_507_744_90/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:01:46.833581 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_154_154_0, total 47 rows starting from the beginning of the part 2024.10.22 02:01:46.841472 [ 1215 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:01:46.860213 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1181_17, 202410_1182_1182_0, 202410_1183_1183_0, 202410_1184_1184_0, 202410_1185_1185_0, 202410_1186_1186_0] 2024.10.22 02:01:46.874952 [ 1215 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:01:46.969776 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:46.976670 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:47.045567 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(be553b6d6d6e57c68d8ee36d3d80529a) : [0, 241297]: Setting download as failed: File segment: [0, 241297], key: be553b6d6d6e57c68d8ee36d3d80529a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:47.048238 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(be553b6d6d6e57c68d8ee36d3d80529a): No space left in cache to reserve 241298 bytes, will continue without cache download 2024.10.22 02:01:47.075522 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1274_1274_0 (state Temporary) 2024.10.22 02:01:47.080414 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: be553b6d6d6e57c68d8ee36d3d80529a, offset: 0, size: 0 2024.10.22 02:01:47.083433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key be553b6d6d6e57c68d8ee36d3d80529a for removal 2024.10.22 02:01:47.093673 [ 1476 ] {} CacheMetadata: Key be553b6d6d6e57c68d8ee36d3d80529a is removed from metadata 2024.10.22 02:01:47.097623 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3c2a814a5d05bdcca1326be63100f2fd, offset: 0, size: 0 2024.10.22 02:01:47.109544 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3c2a814a5d05bdcca1326be63100f2fd for removal 2024.10.22 02:01:47.115820 [ 1476 ] {} CacheMetadata: Key 3c2a814a5d05bdcca1326be63100f2fd is removed from metadata 2024.10.22 02:01:47.208715 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_744_90} MergeTask::MergeProjectionsStage: Merge sorted 23916 rows, containing 13 columns (13 merged, 0 gathered) in 2.863862994 sec., 8350.958146428704 rows/sec., 3.62 MiB/sec. 2024.10.22 02:01:47.244704 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1274_1274_0 to 202410_1287_1287_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:47.258170 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1282_15 to 202410_1287_1287_0 2024.10.22 02:01:47.266330 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:47.267993 [ 1278 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:47.304639 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84260 2024.10.22 02:01:47.314945 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_744_90} MergedBlockOutputStream: filled checksums 202410_507_744_90 (state Temporary) 2024.10.22 02:01:47.314973 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1282_15 to 202410_1287_1287_0 into Compact with storage Full 2024.10.22 02:01:47.348184 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:47.456615 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1282_15, total 4731 rows starting from the beginning of the part 2024.10.22 02:01:47.494231 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1283_1283_0, total 90 rows starting from the beginning of the part 2024.10.22 02:01:47.500048 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 9.49 MiB, will set to 1.34 GiB (RSS), difference: 11.55 MiB 2024.10.22 02:01:47.738352 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1284_1284_0, total 74 rows starting from the beginning of the part 2024.10.22 02:01:47.763002 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1285_1285_0, total 87 rows starting from the beginning of the part 2024.10.22 02:01:47.779733 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1286_1286_0, total 52 rows starting from the beginning of the part 2024.10.22 02:01:47.816237 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_1287_1287_0, total 85 rows starting from the beginning of the part 2024.10.22 02:01:47.893490 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b0226e35a8781b8e8a65efe55944f41b) : [0, 160]: Setting download as failed: File segment: [0, 160], key: b0226e35a8781b8e8a65efe55944f41b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:47.896527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b0226e35a8781b8e8a65efe55944f41b): No space left in cache to reserve 161 bytes, will continue without cache download 2024.10.22 02:01:48.335756 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_744_90} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_744_90 to 202410_507_744_90 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:48.340781 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_744_90} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_743_89, 202410_744_744_0] -> 202410_507_744_90 2024.10.22 02:01:48.358226 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.55 MiB. 2024.10.22 02:01:48.830522 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ac887279b227efef72ef0b720976055e) : [0, 104865]: Setting download as failed: File segment: [0, 104865], key: ac887279b227efef72ef0b720976055e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:48.834832 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ac887279b227efef72ef0b720976055e): No space left in cache to reserve 104866 bytes, will continue without cache download 2024.10.22 02:01:48.879738 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} MergeTask::MergeProjectionsStage: Merge sorted 5119 rows, containing 13 columns (13 merged, 0 gathered) in 1.584231746 sec., 3231.219177954789 rows/sec., 559.90 KiB/sec. 2024.10.22 02:01:49.063134 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ac887279b227efef72ef0b720976055e, offset: 0, size: 0 2024.10.22 02:01:49.065389 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ac887279b227efef72ef0b720976055e for removal 2024.10.22 02:01:49.068788 [ 1476 ] {} CacheMetadata: Key ac887279b227efef72ef0b720976055e is removed from metadata 2024.10.22 02:01:49.070536 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b0226e35a8781b8e8a65efe55944f41b, offset: 0, size: 0 2024.10.22 02:01:49.072741 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b0226e35a8781b8e8a65efe55944f41b for removal 2024.10.22 02:01:49.073476 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} MergedBlockOutputStream: filled checksums 202410_1207_1287_16 (state Temporary) 2024.10.22 02:01:49.079977 [ 1476 ] {} CacheMetadata: Key b0226e35a8781b8e8a65efe55944f41b is removed from metadata 2024.10.22 02:01:49.371560 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(fe3875eff19c5855d198a68c4a8cb72d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fe3875eff19c5855d198a68c4a8cb72d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:01:49.375038 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 8600, file segment info: File segment: [0, 33554431], key: fe3875eff19c5855d198a68c4a8cb72d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:01:49.380703 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:49.410879 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1287_16 to 202410_1207_1287_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:49.418775 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1287_16} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1282_15, 202410_1287_1287_0] -> 202410_1207_1287_16 2024.10.22 02:01:49.552377 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:01:49.624645 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 2.00 GiB, free memory in arenas 8.00 MiB, will set to 1.33 GiB (RSS), difference: 31.48 MiB 2024.10.22 02:01:49.800802 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_704_143, 202410_705_705_0, 202410_706_706_0, 202410_707_707_0, 202410_708_708_0, 202410_709_709_0] 2024.10.22 02:01:49.837003 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_704_143, 202410_705_705_0, 202410_706_706_0, 202410_707_707_0, 202410_708_708_0, 202410_709_709_0] 2024.10.22 02:01:49.965092 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_704_143, 202410_705_705_0, 202410_706_706_0, 202410_707_707_0, 202410_708_708_0, 202410_709_709_0] 2024.10.22 02:01:49.993562 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4e38d6c23a4bbac38aaecd497145ecc9) : [0, 237]: Setting download as failed: File segment: [0, 237], key: 4e38d6c23a4bbac38aaecd497145ecc9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:49.996044 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4e38d6c23a4bbac38aaecd497145ecc9): No space left in cache to reserve 238 bytes, will continue without cache download 2024.10.22 02:01:50.201595 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1243 entries to flush up to offset 1097428 2024.10.22 02:01:50.399819 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 11529 2024.10.22 02:01:50.604181 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:50.614888 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:50.757705 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_756_756_0 (state Temporary) 2024.10.22 02:01:50.797700 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(78ce3da7fdf15b41fdc5eaf7953a742a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 78ce3da7fdf15b41fdc5eaf7953a742a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:01:50.800330 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 771, file segment info: File segment: [0, 33554431], key: 78ce3da7fdf15b41fdc5eaf7953a742a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:01:50.825742 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:50.851095 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_756_756_0 to 202410_769_769_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:50.960916 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11529 2024.10.22 02:01:51.029703 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:51.035393 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:51.087624 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_568_116, 202410_569_569_0, 202410_570_570_0, 202410_571_571_0, 202410_572_572_0, 202410_573_573_0] 2024.10.22 02:01:51.093600 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_568_116, 202410_569_569_0, 202410_570_570_0, 202410_571_571_0, 202410_572_572_0, 202410_573_573_0] 2024.10.22 02:01:51.159220 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:01:51.162789 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:01:51.255863 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_724_724_0 (state Temporary) 2024.10.22 02:01:51.319585 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_568_116, 202410_569_569_0, 202410_570_570_0, 202410_571_571_0, 202410_572_572_0, 202410_573_573_0] 2024.10.22 02:01:51.404176 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_724_724_0 to 202410_731_731_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:51.420262 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 84339 2024.10.22 02:01:51.453075 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1097428 2024.10.22 02:01:51.455864 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 8.89 MiB, will set to 1.33 GiB (RSS), difference: 2.23 MiB 2024.10.22 02:01:51.519928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fde3634b4e867b2ab54b50490665816c) : [0, 547381]: Setting download as failed: File segment: [0, 547381], key: fde3634b4e867b2ab54b50490665816c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:51.523842 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fde3634b4e867b2ab54b50490665816c): No space left in cache to reserve 547382 bytes, will continue without cache download 2024.10.22 02:01:51.658949 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fde3634b4e867b2ab54b50490665816c, offset: 0, size: 0 2024.10.22 02:01:51.662454 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fde3634b4e867b2ab54b50490665816c for removal 2024.10.22 02:01:51.667808 [ 1476 ] {} CacheMetadata: Key fde3634b4e867b2ab54b50490665816c is removed from metadata 2024.10.22 02:01:51.669092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4e38d6c23a4bbac38aaecd497145ecc9, offset: 0, size: 0 2024.10.22 02:01:51.671155 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4e38d6c23a4bbac38aaecd497145ecc9 for removal 2024.10.22 02:01:51.672763 [ 1476 ] {} CacheMetadata: Key 4e38d6c23a4bbac38aaecd497145ecc9 is removed from metadata 2024.10.22 02:01:51.735518 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 167 entries to flush up to offset 83911 2024.10.22 02:01:51.895324 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:51.902336 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:51.971167 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1275_1275_0 (state Temporary) 2024.10.22 02:01:52.020571 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1275_1275_0 to 202410_1288_1288_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:52.052178 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_724_138/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:01:52.266911 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_685_39, 202410_686_686_0, 202410_687_687_0, 202410_688_688_0, 202410_689_689_0, 202410_690_690_0] 2024.10.22 02:01:52.283067 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84339 2024.10.22 02:01:52.288532 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_685_39, 202410_686_686_0, 202410_687_687_0, 202410_688_688_0, 202410_689_689_0, 202410_690_690_0] 2024.10.22 02:01:52.301320 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(f20a7c895b602b397e6d277aa691a361) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f20a7c895b602b397e6d277aa691a361, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:01:52.304416 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:52.307925 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 40929, file segment info: File segment: [0, 33554431], key: f20a7c895b602b397e6d277aa691a361, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:01:52.308986 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:52.311502 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:52.421687 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_685_39, 202410_686_686_0, 202410_687_687_0, 202410_688_688_0, 202410_689_689_0, 202410_690_690_0] 2024.10.22 02:01:52.465032 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_743_743_0 (state Temporary) 2024.10.22 02:01:52.731253 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_743_743_0 to 202410_749_749_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:52.767478 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 83911 2024.10.22 02:01:52.768448 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_744_90 to 202410_745_745_0 2024.10.22 02:01:52.874600 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.39 MiB using storage policy from min volume index 0 2024.10.22 02:01:52.877096 [ 1219 ] {} DiskLocal: Reserved 1.39 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:01:52.915849 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_745_91} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_744_90 to 202410_745_745_0 into Wide with storage Full 2024.10.22 02:01:52.953215 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_745_91} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:01:52.957806 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_745_91} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_744_90, total 23916 rows starting from the beginning of the part 2024.10.22 02:01:53.001817 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(74e950f21ae668f51790b8d707736168) : [0, 233]: Setting download as failed: File segment: [0, 233], key: 74e950f21ae668f51790b8d707736168, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:53.005663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(74e950f21ae668f51790b8d707736168): No space left in cache to reserve 234 bytes, will continue without cache download 2024.10.22 02:01:53.307931 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_745_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_745_745_0, total 149 rows starting from the beginning of the part 2024.10.22 02:01:53.751529 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(5600ab567793bef02ac93a32dc11390b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5600ab567793bef02ac93a32dc11390b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:01:53.755541 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 783, file segment info: File segment: [0, 33554431], key: 5600ab567793bef02ac93a32dc11390b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:01:53.758780 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:53.909961 [ 1727 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/idd/chzqcgqmhecpjmwabcopsmktsacjg, size 40929 2024.10.22 02:01:54.133236 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:01:54.146524 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:01:54.161357 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:01:54.201234 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:01:54.227094 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:01:54.246344 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:01:54.287264 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:01:54.388262 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 6.84 MiB, will set to 1.34 GiB (RSS), difference: -48.06 MiB 2024.10.22 02:01:54.445018 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f101d64fd6dbe2decd224f97099d96fe) : [0, 382500]: Setting download as failed: File segment: [0, 382500], key: f101d64fd6dbe2decd224f97099d96fe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:54.446585 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f101d64fd6dbe2decd224f97099d96fe): No space left in cache to reserve 382501 bytes, will continue without cache download 2024.10.22 02:01:54.518856 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f101d64fd6dbe2decd224f97099d96fe, offset: 0, size: 0 2024.10.22 02:01:54.524815 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f101d64fd6dbe2decd224f97099d96fe for removal 2024.10.22 02:01:54.531817 [ 1476 ] {} CacheMetadata: Key f101d64fd6dbe2decd224f97099d96fe is removed from metadata 2024.10.22 02:01:54.533000 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 74e950f21ae668f51790b8d707736168, offset: 0, size: 0 2024.10.22 02:01:54.537087 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 74e950f21ae668f51790b8d707736168 for removal 2024.10.22 02:01:54.641579 [ 1476 ] {} CacheMetadata: Key 74e950f21ae668f51790b8d707736168 is removed from metadata 2024.10.22 02:01:54.672843 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bjf/kmbsjruxtucijdncvfbtrrstaswze (LogSeriesLimiter: on interval from 2024-10-22 02:01:36 to 2024-10-22 02:01:54 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:01:54.905832 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_745_91} MergeTask::MergeProjectionsStage: Merge sorted 24065 rows, containing 13 columns (13 merged, 0 gathered) in 2.005300082 sec., 12000.697659174582 rows/sec., 5.20 MiB/sec. 2024.10.22 02:01:55.018055 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_745_91} MergedBlockOutputStream: filled checksums 202410_507_745_91 (state Temporary) 2024.10.22 02:01:55.124238 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:55.131714 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:55.417187 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(37ce6fa815099f067a67c8fb2b5141f0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 37ce6fa815099f067a67c8fb2b5141f0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:01:55.422620 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 36011, file segment info: File segment: [0, 33554431], key: 37ce6fa815099f067a67c8fb2b5141f0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:01:55.425509 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:56.107703 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_154_26/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:01:56.228918 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_745_91} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_745_91 to 202410_507_745_91 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:56.235011 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_745_91} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_744_90, 202410_745_745_0] -> 202410_507_745_91 2024.10.22 02:01:56.287101 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f2f5bf0d5019e1c7c6b2307a5851bfac) : [0, 223]: Setting download as failed: File segment: [0, 223], key: f2f5bf0d5019e1c7c6b2307a5851bfac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:56.295229 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 84410 2024.10.22 02:01:56.296017 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f2f5bf0d5019e1c7c6b2307a5851bfac): No space left in cache to reserve 224 bytes, will continue without cache download 2024.10.22 02:01:56.355678 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.56 MiB. 2024.10.22 02:01:56.519598 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 2.00 GiB, free memory in arenas 8.97 MiB, will set to 1.33 GiB (RSS), difference: 33.94 MiB 2024.10.22 02:01:56.569577 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:56.574396 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:56.649395 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1276_1276_0 (state Temporary) 2024.10.22 02:01:56.700223 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1276_1276_0 to 202410_1289_1289_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:56.948315 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(64e6e2cd967a82f2168ea13282cbd3ab) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 64e6e2cd967a82f2168ea13282cbd3ab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:01:56.952964 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 593, file segment info: File segment: [0, 33554431], key: 64e6e2cd967a82f2168ea13282cbd3ab, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:01:56.972105 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84410 2024.10.22 02:01:57.057238 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:57.452500 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 630 2024.10.22 02:01:57.563815 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_152_152_0 (state Temporary) 2024.10.22 02:01:57.611818 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(09f04be0a0a29cacc7a14f41516aa587) : [0, 9507]: Setting download as failed: File segment: [0, 9507], key: 09f04be0a0a29cacc7a14f41516aa587, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:57.614405 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(09f04be0a0a29cacc7a14f41516aa587): No space left in cache to reserve 9508 bytes, will continue without cache download 2024.10.22 02:01:57.659043 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 09f04be0a0a29cacc7a14f41516aa587, offset: 0, size: 0 2024.10.22 02:01:57.672442 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 09f04be0a0a29cacc7a14f41516aa587 for removal 2024.10.22 02:01:57.679316 [ 1476 ] {} CacheMetadata: Key 09f04be0a0a29cacc7a14f41516aa587 is removed from metadata 2024.10.22 02:01:57.680252 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f2f5bf0d5019e1c7c6b2307a5851bfac, offset: 0, size: 0 2024.10.22 02:01:57.681376 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f2f5bf0d5019e1c7c6b2307a5851bfac for removal 2024.10.22 02:01:57.687705 [ 1476 ] {} CacheMetadata: Key f2f5bf0d5019e1c7c6b2307a5851bfac is removed from metadata 2024.10.22 02:01:58.140109 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:58.143438 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:58.199849 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_612_612_0 (state Temporary) 2024.10.22 02:01:58.455584 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1067 entries to flush up to offset 1098495 2024.10.22 02:01:58.463404 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 11537 2024.10.22 02:01:58.473263 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(ca9e2f6cf20fbf99eba3b97517f9f7d8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ca9e2f6cf20fbf99eba3b97517f9f7d8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:01:58.479842 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 13220, file segment info: File segment: [0, 33554431], key: ca9e2f6cf20fbf99eba3b97517f9f7d8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:01:58.483457 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:01:58.491419 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_612_612_0 to 202410_622_622_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:58.558170 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 630 2024.10.22 02:01:58.775271 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:58.783381 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:58.841475 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_757_757_0 (state Temporary) 2024.10.22 02:01:58.988215 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_757_757_0 to 202410_770_770_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:01:59.196322 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4e629691d9e2848691bbbb5f85aa1a37) : [0, 191]: Setting download as failed: File segment: [0, 191], key: 4e629691d9e2848691bbbb5f85aa1a37, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:01:59.198784 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4e629691d9e2848691bbbb5f85aa1a37): No space left in cache to reserve 192 bytes, will continue without cache download 2024.10.22 02:01:59.246624 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11537 2024.10.22 02:01:59.414446 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 16.77 MiB, will set to 1.35 GiB (RSS), difference: 6.00 MiB 2024.10.22 02:01:59.489082 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:01:59.493718 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:01:59.797643 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_725_725_0 (state Temporary) 2024.10.22 02:01:59.872341 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(43bd04e34ac04fd94bb09592809e5171) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 43bd04e34ac04fd94bb09592809e5171, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:01:59.898926 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_725_725_0 to 202410_732_732_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:00.029086 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1098495 2024.10.22 02:02:00.084285 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 529, file segment info: File segment: [0, 33554431], key: 43bd04e34ac04fd94bb09592809e5171, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:02:00.088041 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:00.151783 [ 1709 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fcv/xsqdanfzpjtlujwaihmeejyifktgy, size 13220 2024.10.22 02:02:00.399333 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 181 entries to flush up to offset 84092 2024.10.22 02:02:00.557221 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:02:00.709067 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:02:00.831442 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:02:00.861402 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:02:00.876787 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:02:00.903014 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:02:00.912155 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3eb324eeb2623ce2c4f51858820a4030) : [0, 230659]: Setting download as failed: File segment: [0, 230659], key: 3eb324eeb2623ce2c4f51858820a4030, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:00.915655 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3eb324eeb2623ce2c4f51858820a4030): No space left in cache to reserve 230660 bytes, will continue without cache download 2024.10.22 02:02:00.921093 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:00.924458 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:00.969706 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:02:00.975861 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3eb324eeb2623ce2c4f51858820a4030, offset: 0, size: 0 2024.10.22 02:02:00.998248 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:02:00.998417 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 84472 2024.10.22 02:02:01.000756 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3eb324eeb2623ce2c4f51858820a4030 for removal 2024.10.22 02:02:01.011455 [ 1476 ] {} CacheMetadata: Key 3eb324eeb2623ce2c4f51858820a4030 is removed from metadata 2024.10.22 02:02:01.014131 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4e629691d9e2848691bbbb5f85aa1a37, offset: 0, size: 0 2024.10.22 02:02:01.021024 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4e629691d9e2848691bbbb5f85aa1a37 for removal 2024.10.22 02:02:01.028234 [ 1476 ] {} CacheMetadata: Key 4e629691d9e2848691bbbb5f85aa1a37 is removed from metadata 2024.10.22 02:02:01.120386 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/svu/afqixrkjudeplvuqaqwpnhssbwtft (LogSeriesLimiter: on interval from 2024-10-22 02:01:54 to 2024-10-22 02:02:01 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:02:01.149524 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:02:01.156503 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_744_744_0 (state Temporary) 2024.10.22 02:02:01.157061 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:02:01.224702 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:01.229689 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:01.269536 [ 1175 ] {} DatabaseCatalog: Execution took 555 ms. 2024.10.22 02:02:01.410061 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1277_1277_0 (state Temporary) 2024.10.22 02:02:01.473065 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1277_1277_0 to 202410_1290_1290_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:01.527693 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_744_744_0 to 202410_750_750_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:01.563112 [ 1212 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_507_745_91 to 202410_746_746_0 2024.10.22 02:02:01.565933 [ 1212 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.40 MiB using storage policy from min volume index 0 2024.10.22 02:02:01.567633 [ 1212 ] {} DiskLocal: Reserved 1.40 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:01.653069 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84472 2024.10.22 02:02:01.711841 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_746_92} MergeTask::PrepareStage: Merging 2 parts: from 202410_507_745_91 to 202410_746_746_0 into Wide with storage Full 2024.10.22 02:02:01.764446 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 84092 2024.10.22 02:02:01.815823 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(47c5752087dc89bf717c15e4577be9aa) : [0, 232]: Setting download as failed: File segment: [0, 232], key: 47c5752087dc89bf717c15e4577be9aa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:01.817788 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(47c5752087dc89bf717c15e4577be9aa): No space left in cache to reserve 233 bytes, will continue without cache download 2024.10.22 02:02:01.882346 [ 1234 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:02:01.893516 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_746_92} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:01.932796 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_746_92} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_745_91, total 24065 rows starting from the beginning of the part 2024.10.22 02:02:01.943596 [ 1234 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:02:02.120647 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_724_138} MergeTask::MergeProjectionsStage: Merge sorted 1101690 rows, containing 4 columns (4 merged, 0 gathered) in 35.36277815 sec., 31153.943712422948 rows/sec., 467.32 KiB/sec. 2024.10.22 02:02:02.225816 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_746_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_746_746_0, total 165 rows starting from the beginning of the part 2024.10.22 02:02:02.498580 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 15.21 MiB, will set to 1.35 GiB (RSS), difference: -4.79 MiB 2024.10.22 02:02:02.622190 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_724_138} MergedBlockOutputStream: filled checksums 202410_1_724_138 (state Temporary) 2024.10.22 02:02:02.833094 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8b6e50a3f004a53724d4ab08f35069a7) : [0, 351866]: Setting download as failed: File segment: [0, 351866], key: 8b6e50a3f004a53724d4ab08f35069a7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:02.836734 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8b6e50a3f004a53724d4ab08f35069a7): No space left in cache to reserve 351867 bytes, will continue without cache download 2024.10.22 02:02:02.971065 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8b6e50a3f004a53724d4ab08f35069a7, offset: 0, size: 0 2024.10.22 02:02:02.972454 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8b6e50a3f004a53724d4ab08f35069a7 for removal 2024.10.22 02:02:02.975851 [ 1476 ] {} CacheMetadata: Key 8b6e50a3f004a53724d4ab08f35069a7 is removed from metadata 2024.10.22 02:02:02.978232 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 47c5752087dc89bf717c15e4577be9aa, offset: 0, size: 0 2024.10.22 02:02:02.981340 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 47c5752087dc89bf717c15e4577be9aa for removal 2024.10.22 02:02:02.984396 [ 1476 ] {} CacheMetadata: Key 47c5752087dc89bf717c15e4577be9aa is removed from metadata 2024.10.22 02:02:03.238745 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_724_138/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:02:03.957750 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8dcb0c7fba20f82806fb0dbc5e7875b8) : [0, 235]: Setting download as failed: File segment: [0, 235], key: 8dcb0c7fba20f82806fb0dbc5e7875b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:03.958711 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8dcb0c7fba20f82806fb0dbc5e7875b8): No space left in cache to reserve 236 bytes, will continue without cache download 2024.10.22 02:02:04.449374 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 2.00 GiB, free memory in arenas 10.93 MiB, will set to 1.36 GiB (RSS), difference: -35.68 MiB 2024.10.22 02:02:04.633123 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a66962ceffe5ce73cc882b161f9f281d) : [0, 523494]: Setting download as failed: File segment: [0, 523494], key: a66962ceffe5ce73cc882b161f9f281d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:04.636283 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a66962ceffe5ce73cc882b161f9f281d): No space left in cache to reserve 523495 bytes, will continue without cache download 2024.10.22 02:02:04.677468 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a66962ceffe5ce73cc882b161f9f281d, offset: 0, size: 0 2024.10.22 02:02:04.679855 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a66962ceffe5ce73cc882b161f9f281d for removal 2024.10.22 02:02:04.683335 [ 1476 ] {} CacheMetadata: Key a66962ceffe5ce73cc882b161f9f281d is removed from metadata 2024.10.22 02:02:04.694355 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8dcb0c7fba20f82806fb0dbc5e7875b8, offset: 0, size: 0 2024.10.22 02:02:04.696681 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8dcb0c7fba20f82806fb0dbc5e7875b8 for removal 2024.10.22 02:02:04.702843 [ 1476 ] {} CacheMetadata: Key 8dcb0c7fba20f82806fb0dbc5e7875b8 is removed from metadata 2024.10.22 02:02:05.101384 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_746_92} MergeTask::MergeProjectionsStage: Merge sorted 24230 rows, containing 13 columns (13 merged, 0 gathered) in 3.413912603 sec., 7097.428322771859 rows/sec., 3.07 MiB/sec. 2024.10.22 02:02:05.187873 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_152_152_0 to 202410_155_155_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:05.204511 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_746_92} MergedBlockOutputStream: filled checksums 202410_507_746_92 (state Temporary) 2024.10.22 02:02:05.478986 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9a0ca1e566333f6ace9dca4964e43fe0) : [0, 238]: Setting download as failed: File segment: [0, 238], key: 9a0ca1e566333f6ace9dca4964e43fe0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:05.480217 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9a0ca1e566333f6ace9dca4964e43fe0): No space left in cache to reserve 239 bytes, will continue without cache download 2024.10.22 02:02:05.667035 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 84531 2024.10.22 02:02:05.923668 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_724_138} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_724_138 to 202410_1_724_138 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:05.935474 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_724_138} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_718_137, 202410_719_724_1] -> 202410_1_724_138 2024.10.22 02:02:05.965662 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:02:06.003717 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:06.014927 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:06.059485 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 631 2024.10.22 02:02:06.088038 [ 1198 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 02:02:06.129201 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_671_671_0, 202410_672_672_0, 202410_673_673_0, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0] 2024.10.22 02:02:06.156272 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_671_671_0, 202410_672_672_0, 202410_673_673_0, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0] 2024.10.22 02:02:06.179736 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1278_1278_0 (state Temporary) 2024.10.22 02:02:06.234751 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1278_1278_0 to 202410_1291_1291_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:06.264047 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_746_92} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_507_746_92 to 202410_507_746_92 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:06.270622 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_507_746_92} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_507_745_91, 202410_746_746_0] -> 202410_507_746_92 2024.10.22 02:02:06.293051 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_671_671_0, 202410_672_672_0, 202410_673_673_0, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0] 2024.10.22 02:02:06.339469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(61fe51191e227a1c5e7cf34068279c77) : [0, 642950]: Setting download as failed: File segment: [0, 642950], key: 61fe51191e227a1c5e7cf34068279c77, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:06.342122 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(61fe51191e227a1c5e7cf34068279c77): No space left in cache to reserve 642951 bytes, will continue without cache download 2024.10.22 02:02:06.379750 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84531 2024.10.22 02:02:06.393803 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.57 MiB. 2024.10.22 02:02:06.429303 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:06.433569 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:06.593695 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_613_613_0 (state Temporary) 2024.10.22 02:02:06.601678 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 61fe51191e227a1c5e7cf34068279c77, offset: 0, size: 0 2024.10.22 02:02:06.607450 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 61fe51191e227a1c5e7cf34068279c77 for removal 2024.10.22 02:02:06.618859 [ 1476 ] {} CacheMetadata: Key 61fe51191e227a1c5e7cf34068279c77 is removed from metadata 2024.10.22 02:02:06.627329 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9a0ca1e566333f6ace9dca4964e43fe0, offset: 0, size: 0 2024.10.22 02:02:06.631432 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9a0ca1e566333f6ace9dca4964e43fe0 for removal 2024.10.22 02:02:06.636627 [ 1476 ] {} CacheMetadata: Key 9a0ca1e566333f6ace9dca4964e43fe0 is removed from metadata 2024.10.22 02:02:06.644675 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 6.23 MiB, will set to 1.35 GiB (RSS), difference: 35.59 MiB 2024.10.22 02:02:06.692533 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_613_613_0 to 202410_623_623_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:06.730196 [ 1215 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_618_126 to 202410_623_623_0 2024.10.22 02:02:06.744028 [ 1215 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:06.745628 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:06.748011 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 11551 2024.10.22 02:02:06.824640 [ 1248 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 265 ms. 2024.10.22 02:02:06.863101 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 631 2024.10.22 02:02:06.981509 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_618_126 to 202410_623_623_0 into Compact with storage Full 2024.10.22 02:02:07.032228 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1100035 2024.10.22 02:02:07.288377 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:07.291080 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:07.329097 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_758_758_0 (state Temporary) 2024.10.22 02:02:07.374859 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:07.412312 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_618_126, total 662 rows starting from the beginning of the part 2024.10.22 02:02:07.415956 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(de231c71c131013e414aed9a8a8970de) : [0, 245]: Setting download as failed: File segment: [0, 245], key: de231c71c131013e414aed9a8a8970de, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:07.420003 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(de231c71c131013e414aed9a8a8970de): No space left in cache to reserve 246 bytes, will continue without cache download 2024.10.22 02:02:07.464534 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_758_758_0 to 202410_771_771_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:07.507701 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11551 2024.10.22 02:02:07.580014 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:07.587767 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:07.663749 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_619_619_0, total 1 rows starting from the beginning of the part 2024.10.22 02:02:07.801555 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_726_726_0 (state Temporary) 2024.10.22 02:02:07.849312 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_620_620_0, total 1 rows starting from the beginning of the part 2024.10.22 02:02:07.871649 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_726_726_0 to 202410_733_733_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:07.907897 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1100035 2024.10.22 02:02:08.127722 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_621_621_0, total 1 rows starting from the beginning of the part 2024.10.22 02:02:08.158886 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3b6406d00a8285555487371ab8231dfc) : [0, 1011268]: Setting download as failed: File segment: [0, 1011268], key: 3b6406d00a8285555487371ab8231dfc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:08.160183 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3b6406d00a8285555487371ab8231dfc): No space left in cache to reserve 1011269 bytes, will continue without cache download 2024.10.22 02:02:08.197711 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6117 2024.10.22 02:02:08.330476 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3b6406d00a8285555487371ab8231dfc, offset: 0, size: 0 2024.10.22 02:02:08.334721 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3b6406d00a8285555487371ab8231dfc for removal 2024.10.22 02:02:08.339623 [ 1476 ] {} CacheMetadata: Key 3b6406d00a8285555487371ab8231dfc is removed from metadata 2024.10.22 02:02:08.369203 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_622_622_0, total 1 rows starting from the beginning of the part 2024.10.22 02:02:08.433365 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 9.22 MiB, will set to 1.34 GiB (RSS), difference: 4.15 MiB 2024.10.22 02:02:08.446201 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: de231c71c131013e414aed9a8a8970de, offset: 0, size: 0 2024.10.22 02:02:08.448733 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key de231c71c131013e414aed9a8a8970de for removal 2024.10.22 02:02:08.450465 [ 1476 ] {} CacheMetadata: Key de231c71c131013e414aed9a8a8970de is removed from metadata 2024.10.22 02:02:08.536146 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_623_623_0, total 1 rows starting from the beginning of the part 2024.10.22 02:02:09.302227 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 184 entries to flush up to offset 84276 2024.10.22 02:02:09.302554 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(497e90798314b541c69eb09a05639b38) : [0, 245]: Setting download as failed: File segment: [0, 245], key: 497e90798314b541c69eb09a05639b38, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:09.308629 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(497e90798314b541c69eb09a05639b38): No space left in cache to reserve 246 bytes, will continue without cache download 2024.10.22 02:02:09.639936 [ 1259 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 02:02:09.799909 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 9.10 MiB, will set to 1.34 GiB (RSS), difference: -4.83 MiB 2024.10.22 02:02:09.909130 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:09.917039 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:09.959555 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_745_745_0 (state Temporary) 2024.10.22 02:02:10.134232 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_745_745_0 to 202410_751_751_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:10.174370 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 84276 2024.10.22 02:02:10.251743 [ 1241 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_1_506_234 to 202410_507_746_92 2024.10.22 02:02:10.262190 [ 1241 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 4.39 MiB using storage policy from min volume index 0 2024.10.22 02:02:10.267568 [ 1241 ] {} DiskLocal: Reserved 4.39 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:10.307548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bb9ea980fbd868b1aabaa4dfb459c796) : [0, 1009483]: Setting download as failed: File segment: [0, 1009483], key: bb9ea980fbd868b1aabaa4dfb459c796, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:10.315113 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bb9ea980fbd868b1aabaa4dfb459c796): No space left in cache to reserve 1009484 bytes, will continue without cache download 2024.10.22 02:02:10.450692 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_746_235} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_506_234 to 202410_507_746_92 into Wide with storage Full 2024.10.22 02:02:10.492753 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 84613 2024.10.22 02:02:10.535797 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_746_235} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:10.545249 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_746_235} MergeTreeSequentialSource: Reading 9 marks from part 202410_1_506_234, total 59229 rows starting from the beginning of the part 2024.10.22 02:02:10.807028 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} MergeTask::MergeProjectionsStage: Merge sorted 667 rows, containing 32 columns (32 merged, 0 gathered) in 3.835011095 sec., 173.9238775266177 rows/sec., 15.80 KiB/sec. 2024.10.22 02:02:10.830207 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_746_235} MergeTreeSequentialSource: Reading 4 marks from part 202410_507_746_92, total 24230 rows starting from the beginning of the part 2024.10.22 02:02:10.934228 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bb9ea980fbd868b1aabaa4dfb459c796, offset: 0, size: 0 2024.10.22 02:02:10.938792 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bb9ea980fbd868b1aabaa4dfb459c796 for removal 2024.10.22 02:02:10.941577 [ 1476 ] {} CacheMetadata: Key bb9ea980fbd868b1aabaa4dfb459c796 is removed from metadata 2024.10.22 02:02:10.943873 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:10.946587 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 497e90798314b541c69eb09a05639b38, offset: 0, size: 0 2024.10.22 02:02:10.950385 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 497e90798314b541c69eb09a05639b38 for removal 2024.10.22 02:02:10.953370 [ 1476 ] {} CacheMetadata: Key 497e90798314b541c69eb09a05639b38 is removed from metadata 2024.10.22 02:02:11.004170 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:11.027428 [ 1261 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 350 ms. 2024.10.22 02:02:11.045152 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_154_26/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:02:11.076015 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1279_1279_0 (state Temporary) 2024.10.22 02:02:11.122968 [ 1182 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 02:02:11.138196 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} MergedBlockOutputStream: filled checksums 202410_1_623_127 (state Temporary) 2024.10.22 02:02:11.151609 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:02:11.153824 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:02:11.257417 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1279_1279_0 to 202410_1292_1292_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:11.370590 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1287_16 to 202410_1292_1292_0 2024.10.22 02:02:11.383277 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:11.385543 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:11.407603 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84613 2024.10.22 02:02:11.433035 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1287_16 to 202410_1292_1292_0 into Compact with storage Full 2024.10.22 02:02:11.461834 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:11.572078 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1287_16, total 5119 rows starting from the beginning of the part 2024.10.22 02:02:11.697343 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1288_1288_0, total 79 rows starting from the beginning of the part 2024.10.22 02:02:11.724147 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1289_1289_0, total 71 rows starting from the beginning of the part 2024.10.22 02:02:11.738047 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1186_18, 202410_1187_1187_0, 202410_1188_1188_0, 202410_1189_1189_0, 202410_1190_1190_0, 202410_1191_1191_0] 2024.10.22 02:02:11.741103 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1186_18, 202410_1187_1187_0, 202410_1188_1188_0, 202410_1189_1189_0, 202410_1190_1190_0, 202410_1191_1191_0] 2024.10.22 02:02:11.762905 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1186_18, 202410_1187_1187_0, 202410_1188_1188_0, 202410_1189_1189_0, 202410_1190_1190_0, 202410_1191_1191_0] 2024.10.22 02:02:11.784797 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(76677448946d9013168f9846ba1cbda5) : [0, 237]: Setting download as failed: File segment: [0, 237], key: 76677448946d9013168f9846ba1cbda5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:11.790562 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(76677448946d9013168f9846ba1cbda5): No space left in cache to reserve 238 bytes, will continue without cache download 2024.10.22 02:02:11.940626 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_623_127 to 202410_1_623_127 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:11.947700 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_623_127} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_618_126, 202410_623_623_0] -> 202410_1_623_127 2024.10.22 02:02:11.958769 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1290_1290_0, total 62 rows starting from the beginning of the part 2024.10.22 02:02:11.971555 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1291_1291_0, total 59 rows starting from the beginning of the part 2024.10.22 02:02:11.986722 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:02:12.197656 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_1292_1292_0, total 82 rows starting from the beginning of the part 2024.10.22 02:02:12.430718 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 2.00 GiB, free memory in arenas 8.36 MiB, will set to 1.36 GiB (RSS), difference: -56.62 MiB 2024.10.22 02:02:12.555127 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1b86340a5995d70a2b7c48372b5e245b) : [0, 657385]: Setting download as failed: File segment: [0, 657385], key: 1b86340a5995d70a2b7c48372b5e245b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:12.556989 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1b86340a5995d70a2b7c48372b5e245b): No space left in cache to reserve 657386 bytes, will continue without cache download 2024.10.22 02:02:12.715411 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1b86340a5995d70a2b7c48372b5e245b, offset: 0, size: 0 2024.10.22 02:02:12.719201 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1b86340a5995d70a2b7c48372b5e245b for removal 2024.10.22 02:02:12.733585 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 76677448946d9013168f9846ba1cbda5, offset: 0, size: 0 2024.10.22 02:02:12.737085 [ 1476 ] {} CacheMetadata: Key 1b86340a5995d70a2b7c48372b5e245b is removed from metadata 2024.10.22 02:02:12.737428 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 76677448946d9013168f9846ba1cbda5 for removal 2024.10.22 02:02:12.742104 [ 1476 ] {} CacheMetadata: Key 76677448946d9013168f9846ba1cbda5 is removed from metadata 2024.10.22 02:02:13.169584 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_1_746_235/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:02:13.269391 [ 1204 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 221 ms. 2024.10.22 02:02:13.448320 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(e4a1c9a281c56708d92f74fcfd17c796) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e4a1c9a281c56708d92f74fcfd17c796, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:02:13.451396 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1437, file segment info: File segment: [0, 33554431], key: e4a1c9a281c56708d92f74fcfd17c796, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:02:13.455618 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:13.592125 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} MergeTask::MergeProjectionsStage: Merge sorted 5472 rows, containing 13 columns (13 merged, 0 gathered) in 2.177381382 sec., 2513.1104937499645 rows/sec., 437.38 KiB/sec. 2024.10.22 02:02:13.880321 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} MergedBlockOutputStream: filled checksums 202410_1207_1292_17 (state Temporary) 2024.10.22 02:02:14.190377 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9d949208c99e3afa9bdc813d37fa1c38) : [0, 234]: Setting download as failed: File segment: [0, 234], key: 9d949208c99e3afa9bdc813d37fa1c38, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:14.193254 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9d949208c99e3afa9bdc813d37fa1c38): No space left in cache to reserve 235 bytes, will continue without cache download 2024.10.22 02:02:14.266308 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1292_17 to 202410_1207_1292_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:14.270096 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1292_17} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1287_16, 202410_1292_1292_0] -> 202410_1207_1292_17 2024.10.22 02:02:14.350656 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:02:14.366671 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 632 2024.10.22 02:02:14.668037 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:14.670349 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:14.751308 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_614_614_0 (state Temporary) 2024.10.22 02:02:14.912660 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1101575 2024.10.22 02:02:14.927515 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_614_614_0 to 202410_624_624_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:15.011126 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 11570 2024.10.22 02:02:15.133337 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 632 2024.10.22 02:02:15.136344 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:15.140366 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:15.171466 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_759_759_0 (state Temporary) 2024.10.22 02:02:15.321113 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 6.91 MiB, will set to 1.36 GiB (RSS), difference: 20.77 MiB 2024.10.22 02:02:15.329309 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_759_759_0 to 202410_772_772_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:15.395360 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(fc3a9cf2f632a76e8984371ce124f34c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fc3a9cf2f632a76e8984371ce124f34c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:02:15.403833 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 766, file segment info: File segment: [0, 33554431], key: fc3a9cf2f632a76e8984371ce124f34c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:02:15.409237 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:15.560201 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_767_164 to 202410_768_768_0 2024.10.22 02:02:15.671533 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11570 2024.10.22 02:02:15.774981 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 84692 2024.10.22 02:02:15.811222 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:15.813592 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:15.822855 [ 1251 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 273 ms. 2024.10.22 02:02:15.731616 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 6154 2024.10.22 02:02:15.862197 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_768_165} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_767_164 to 202410_768_768_0 into Compact with storage Full 2024.10.22 02:02:16.004368 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_746_235} MergeTask::MergeProjectionsStage: Merge sorted 83459 rows, containing 13 columns (13 merged, 0 gathered) in 5.560949487 sec., 15008.048570681072 rows/sec., 6.52 MiB/sec. 2024.10.22 02:02:16.007039 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(82d4f8fe59c19de238674a72883cc40d) : [0, 465505]: Setting download as failed: File segment: [0, 465505], key: 82d4f8fe59c19de238674a72883cc40d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:16.009608 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(82d4f8fe59c19de238674a72883cc40d): No space left in cache to reserve 465506 bytes, will continue without cache download 2024.10.22 02:02:16.027869 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_768_165} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:16.049419 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_768_165} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_767_164, total 11598 rows starting from the beginning of the part 2024.10.22 02:02:16.076377 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_768_165} MergeTreeSequentialSource: Reading 2 marks from part 202410_768_768_0, total 16 rows starting from the beginning of the part 2024.10.22 02:02:16.185299 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 82d4f8fe59c19de238674a72883cc40d, offset: 0, size: 0 2024.10.22 02:02:16.190295 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 82d4f8fe59c19de238674a72883cc40d for removal 2024.10.22 02:02:16.196145 [ 1476 ] {} CacheMetadata: Key 82d4f8fe59c19de238674a72883cc40d is removed from metadata 2024.10.22 02:02:16.204551 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9d949208c99e3afa9bdc813d37fa1c38, offset: 0, size: 0 2024.10.22 02:02:16.210165 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9d949208c99e3afa9bdc813d37fa1c38 for removal 2024.10.22 02:02:16.217217 [ 1476 ] {} CacheMetadata: Key 9d949208c99e3afa9bdc813d37fa1c38 is removed from metadata 2024.10.22 02:02:16.298427 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:16.303687 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:16.307906 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_746_235} MergedBlockOutputStream: filled checksums 202410_1_746_235 (state Temporary) 2024.10.22 02:02:16.370042 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:16.374407 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:16.383140 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1280_1280_0 (state Temporary) 2024.10.22 02:02:16.438409 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1280_1280_0 to 202410_1293_1293_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:16.471826 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84692 2024.10.22 02:02:16.777228 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(f5e7fbcc64671009459e9f09154d6e01) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f5e7fbcc64671009459e9f09154d6e01, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:02:16.883652 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 37171, file segment info: File segment: [0, 33554431], key: f5e7fbcc64671009459e9f09154d6e01, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:02:16.887937 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:16.950892 [ 1172 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:02:17.003416 [ 1172 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:02:17.085144 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_727_727_0 (state Temporary) 2024.10.22 02:02:17.180404 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_727_727_0 to 202410_734_734_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:17.216827 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1101575 2024.10.22 02:02:17.227330 [ 1160 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_724_138 to 202410_725_730_1 2024.10.22 02:02:17.238736 [ 1160 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:17.242239 [ 1160 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:17.348142 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_768_165} MergeTask::MergeProjectionsStage: Merge sorted 11614 rows, containing 12 columns (12 merged, 0 gathered) in 1.491131794 sec., 7788.7146171333 rows/sec., 1.21 MiB/sec. 2024.10.22 02:02:17.400902 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_768_165} MergedBlockOutputStream: filled checksums 202410_1_768_165 (state Temporary) 2024.10.22 02:02:17.455919 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_730_139} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_724_138 to 202410_725_730_1 into Wide with storage Full 2024.10.22 02:02:17.463058 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3a7746745d392653615fc9124da5fc56) : [0, 243]: Setting download as failed: File segment: [0, 243], key: 3a7746745d392653615fc9124da5fc56, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:17.464184 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3a7746745d392653615fc9124da5fc56): No space left in cache to reserve 244 bytes, will continue without cache download 2024.10.22 02:02:17.468444 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_730_139} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:17.490285 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_730_139} MergeTreeSequentialSource: Reading 136 marks from part 202410_1_724_138, total 1101690 rows starting from the beginning of the part 2024.10.22 02:02:17.500158 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_768_165} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_768_165 to 202410_1_768_165 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:17.505086 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_768_165} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_767_164, 202410_768_768_0] -> 202410_1_768_165 2024.10.22 02:02:17.558391 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:02:17.708943 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 207 entries to flush up to offset 84483 2024.10.22 02:02:17.730515 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_730_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_725_730_1, total 8085 rows starting from the beginning of the part 2024.10.22 02:02:17.944380 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_746_235} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_1_746_235 to 202410_1_746_235 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:17.951543 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_1_746_235} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_1_506_234, 202410_507_746_92] -> 202410_1_746_235 2024.10.22 02:02:18.174868 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 77.64 MiB. 2024.10.22 02:02:18.345071 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d46606af851ae8cf3f6a9beb523166ca) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d46606af851ae8cf3f6a9beb523166ca, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:02:18.351455 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 770, file segment info: File segment: [0, 33554431], key: d46606af851ae8cf3f6a9beb523166ca, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:02:18.357335 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:18.359969 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:18.448077 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 4.84 MiB, will set to 1.34 GiB (RSS), difference: 29.73 MiB 2024.10.22 02:02:18.353819 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:18.495716 [ 1732 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bjf/kmbsjruxtucijdncvfbtrrstaswze, size 37171 2024.10.22 02:02:18.731743 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_746_746_0 (state Temporary) 2024.10.22 02:02:18.875069 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_746_746_0 to 202410_752_752_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:18.914066 [ 1183 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_747_0 to 202410_752_752_0 2024.10.22 02:02:18.973053 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:02:18.979263 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:02:18.991691 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:02:19.021148 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 84483 2024.10.22 02:02:19.030229 [ 1183 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:19.035103 [ 1183 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:19.052441 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_747_0 to 202410_752_752_0 into Compact with storage Full 2024.10.22 02:02:19.170710 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:02:19.185139 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:02:19.200836 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:02:19.220592 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:02:19.225785 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:19.239519 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_747_0, total 195 rows starting from the beginning of the part 2024.10.22 02:02:19.256752 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vrd/lkouhxgnkrkkjsznktmaexpvcjqix (LogSeriesLimiter: on interval from 2024-10-22 02:02:01 to 2024-10-22 02:02:19 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:02:19.258783 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_748_748_0, total 194 rows starting from the beginning of the part 2024.10.22 02:02:19.294911 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_749_749_0, total 167 rows starting from the beginning of the part 2024.10.22 02:02:19.356764 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_750_750_0, total 181 rows starting from the beginning of the part 2024.10.22 02:02:19.422603 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_751_751_0, total 184 rows starting from the beginning of the part 2024.10.22 02:02:19.428860 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(30bbf25de92647b23931223f18afc6bb) : [0, 902342]: Setting download as failed: File segment: [0, 902342], key: 30bbf25de92647b23931223f18afc6bb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:19.434274 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(30bbf25de92647b23931223f18afc6bb): No space left in cache to reserve 902343 bytes, will continue without cache download 2024.10.22 02:02:19.544786 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_752_752_0, total 207 rows starting from the beginning of the part 2024.10.22 02:02:19.688488 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_730_139/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:02:19.796313 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 30bbf25de92647b23931223f18afc6bb, offset: 0, size: 0 2024.10.22 02:02:19.798926 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 30bbf25de92647b23931223f18afc6bb for removal 2024.10.22 02:02:19.802244 [ 1476 ] {} CacheMetadata: Key 30bbf25de92647b23931223f18afc6bb is removed from metadata 2024.10.22 02:02:19.805641 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3a7746745d392653615fc9124da5fc56, offset: 0, size: 0 2024.10.22 02:02:19.809223 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3a7746745d392653615fc9124da5fc56 for removal 2024.10.22 02:02:19.815737 [ 1476 ] {} CacheMetadata: Key 3a7746745d392653615fc9124da5fc56 is removed from metadata 2024.10.22 02:02:20.478953 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 84784 2024.10.22 02:02:20.819591 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:20.827775 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:20.869231 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(89aeef6d377d2c9bf7f21c4b27246e5b) : [0, 237]: Setting download as failed: File segment: [0, 237], key: 89aeef6d377d2c9bf7f21c4b27246e5b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:20.877568 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} MergeTask::MergeProjectionsStage: Merge sorted 1128 rows, containing 13 columns (13 merged, 0 gathered) in 1.830084165 sec., 616.3650948807592 rows/sec., 279.85 KiB/sec. 2024.10.22 02:02:20.941293 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1281_1281_0 (state Temporary) 2024.10.22 02:02:20.953709 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} MergedBlockOutputStream: filled checksums 202410_747_752_1 (state Temporary) 2024.10.22 02:02:20.971244 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(89aeef6d377d2c9bf7f21c4b27246e5b): No space left in cache to reserve 238 bytes, will continue without cache download 2024.10.22 02:02:20.975729 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1281_1281_0 to 202410_1294_1294_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:21.056062 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84784 2024.10.22 02:02:21.150350 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:02:21.151387 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:02:21.240227 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_752_1 to 202410_747_752_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:21.255243 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_752_1} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_747_0, 202410_752_752_0] -> 202410_747_752_1 2024.10.22 02:02:21.338122 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 9.02 MiB, will set to 1.35 GiB (RSS), difference: 4.51 MiB 2024.10.22 02:02:21.392405 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:02:21.649437 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(e15f7466b57b672d527ba58148035faf) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e15f7466b57b672d527ba58148035faf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:02:21.651914 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 32699, file segment info: File segment: [0, 33554431], key: e15f7466b57b672d527ba58148035faf, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:02:21.654569 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:22.031837 [ 1235 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 222 ms. 2024.10.22 02:02:22.426980 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0089ebe857a55520e07e7a0d50839acb) : [0, 579268]: Setting download as failed: File segment: [0, 579268], key: 0089ebe857a55520e07e7a0d50839acb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:22.429359 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0089ebe857a55520e07e7a0d50839acb): No space left in cache to reserve 579269 bytes, will continue without cache download 2024.10.22 02:02:22.570225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0089ebe857a55520e07e7a0d50839acb, offset: 0, size: 0 2024.10.22 02:02:22.595661 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0089ebe857a55520e07e7a0d50839acb for removal 2024.10.22 02:02:22.599605 [ 1476 ] {} CacheMetadata: Key 0089ebe857a55520e07e7a0d50839acb is removed from metadata 2024.10.22 02:02:22.601364 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 89aeef6d377d2c9bf7f21c4b27246e5b, offset: 0, size: 0 2024.10.22 02:02:22.603737 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 89aeef6d377d2c9bf7f21c4b27246e5b for removal 2024.10.22 02:02:22.605712 [ 1476 ] {} CacheMetadata: Key 89aeef6d377d2c9bf7f21c4b27246e5b is removed from metadata 2024.10.22 02:02:22.636934 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 633 2024.10.22 02:02:23.036501 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:23.040266 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:23.082056 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_615_615_0 (state Temporary) 2024.10.22 02:02:23.260043 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_615_615_0 to 202410_625_625_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:23.278392 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 11589 2024.10.22 02:02:23.351783 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 633 2024.10.22 02:02:23.409621 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:23.447945 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(1225c9d632daf4efc6341dfd75d09871) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1225c9d632daf4efc6341dfd75d09871, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:02:23.515761 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:23.559519 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_760_760_0 (state Temporary) 2024.10.22 02:02:23.569502 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 575, file segment info: File segment: [0, 33554431], key: 1225c9d632daf4efc6341dfd75d09871, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:02:23.571672 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:23.628941 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_760_760_0 to 202410_773_773_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:23.666595 [ 1208 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_768_165 to 202410_769_769_0 2024.10.22 02:02:23.669093 [ 1208 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:23.670804 [ 1208 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:23.687513 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11589 2024.10.22 02:02:23.827945 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_769_166} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_768_165 to 202410_769_769_0 into Compact with storage Full 2024.10.22 02:02:23.861106 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_769_166} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:23.869103 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_769_166} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_768_165, total 11614 rows starting from the beginning of the part 2024.10.22 02:02:24.011745 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_769_166} MergeTreeSequentialSource: Reading 2 marks from part 202410_769_769_0, total 21 rows starting from the beginning of the part 2024.10.22 02:02:24.229192 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1102730 2024.10.22 02:02:24.288529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1a28f3f8d16fe0d6025130b816aac5b8) : [0, 231]: Setting download as failed: File segment: [0, 231], key: 1a28f3f8d16fe0d6025130b816aac5b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:24.290539 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1a28f3f8d16fe0d6025130b816aac5b8): No space left in cache to reserve 232 bytes, will continue without cache download 2024.10.22 02:02:24.884513 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(5341f09889b634dcf7e1496a9baa706b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5341f09889b634dcf7e1496a9baa706b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:02:24.887951 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 8384, file segment info: File segment: [0, 33554431], key: 5341f09889b634dcf7e1496a9baa706b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:02:24.891020 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:24.959215 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_769_166} MergeTask::MergeProjectionsStage: Merge sorted 11635 rows, containing 12 columns (12 merged, 0 gathered) in 1.143181372 sec., 10177.737570762307 rows/sec., 1.58 MiB/sec. 2024.10.22 02:02:24.991207 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_769_166} MergedBlockOutputStream: filled checksums 202410_1_769_166 (state Temporary) 2024.10.22 02:02:25.066919 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 84842 2024.10.22 02:02:25.091394 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_769_166} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_769_166 to 202410_1_769_166 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:25.099390 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_769_166} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_768_165, 202410_769_769_0] -> 202410_1_769_166 2024.10.22 02:02:25.143321 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:25.146018 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:25.231746 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:02:25.273989 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_728_728_0 (state Temporary) 2024.10.22 02:02:25.322937 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:25.344502 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:02:25.489642 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_728_728_0 to 202410_735_735_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:25.503968 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1282_1282_0 (state Temporary) 2024.10.22 02:02:25.559441 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1282_1282_0 to 202410_1295_1295_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:25.560120 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 11.30 MiB, will set to 1.35 GiB (RSS), difference: 3.24 MiB 2024.10.22 02:02:25.622567 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84842 2024.10.22 02:02:25.629405 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_664_128, 202410_665_670_1] 2024.10.22 02:02:25.632065 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_664_128, 202410_665_670_1] 2024.10.22 02:02:25.644137 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_664_128, 202410_665_670_1] 2024.10.22 02:02:25.652967 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1102730 2024.10.22 02:02:25.854458 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6f4ed34a04a6e67d02859e0657464312) : [0, 329085]: Setting download as failed: File segment: [0, 329085], key: 6f4ed34a04a6e67d02859e0657464312, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:25.855840 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6f4ed34a04a6e67d02859e0657464312): No space left in cache to reserve 329086 bytes, will continue without cache download 2024.10.22 02:02:25.892475 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6f4ed34a04a6e67d02859e0657464312, offset: 0, size: 0 2024.10.22 02:02:25.996201 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6f4ed34a04a6e67d02859e0657464312 for removal 2024.10.22 02:02:26.004101 [ 1476 ] {} CacheMetadata: Key 6f4ed34a04a6e67d02859e0657464312 is removed from metadata 2024.10.22 02:02:26.007556 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1a28f3f8d16fe0d6025130b816aac5b8, offset: 0, size: 0 2024.10.22 02:02:26.011239 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1a28f3f8d16fe0d6025130b816aac5b8 for removal 2024.10.22 02:02:26.016862 [ 1476 ] {} CacheMetadata: Key 1a28f3f8d16fe0d6025130b816aac5b8 is removed from metadata 2024.10.22 02:02:26.675560 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 117 entries to flush up to offset 84600 2024.10.22 02:02:27.040834 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(3a01a6e9862ee4d4ca66c2ec6d67d1ce) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3a01a6e9862ee4d4ca66c2ec6d67d1ce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:02:27.101257 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:27.107600 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:27.161684 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 492, file segment info: File segment: [0, 33554431], key: 3a01a6e9862ee4d4ca66c2ec6d67d1ce, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:02:27.164541 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:27.203642 [ 1676 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/svu/afqixrkjudeplvuqaqwpnhssbwtft, size 8384 2024.10.22 02:02:27.258632 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_747_747_0 (state Temporary) 2024.10.22 02:02:27.352131 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_747_747_0 to 202410_753_753_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:27.382822 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 84600 2024.10.22 02:02:27.549821 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_154_26/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:02:27.632750 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b026242e4fbbdf4bd4c4d7de64b7da1a) : [0, 231]: Setting download as failed: File segment: [0, 231], key: b026242e4fbbdf4bd4c4d7de64b7da1a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:27.633971 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b026242e4fbbdf4bd4c4d7de64b7da1a): No space left in cache to reserve 232 bytes, will continue without cache download 2024.10.22 02:02:27.845619 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column FromTag 2024.10.22 02:02:27.865161 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column FromTag 2024.10.22 02:02:27.884329 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column FromTag 2024.10.22 02:02:27.903208 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column FromTag 2024.10.22 02:02:28.030521 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column FromTag 2024.10.22 02:02:28.057159 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column FromTag 2024.10.22 02:02:28.103139 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column FromTag 2024.10.22 02:02:28.129318 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column FromTag 2024.10.22 02:02:28.262794 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_709_144, 202410_710_710_0] 2024.10.22 02:02:28.264471 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_709_144, 202410_710_710_0] 2024.10.22 02:02:28.278425 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_709_144, 202410_710_710_0] 2024.10.22 02:02:28.298091 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xox/ldsyqaqljqmhaavuufjmogombczae (LogSeriesLimiter: on interval from 2024-10-22 02:02:19 to 2024-10-22 02:02:28 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:02:28.678673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4653ae8532291878b843266e9b4905c7) : [0, 326822]: Setting download as failed: File segment: [0, 326822], key: 4653ae8532291878b843266e9b4905c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:28.680813 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4653ae8532291878b843266e9b4905c7): No space left in cache to reserve 326823 bytes, will continue without cache download 2024.10.22 02:02:28.716672 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4653ae8532291878b843266e9b4905c7, offset: 0, size: 0 2024.10.22 02:02:28.720144 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4653ae8532291878b843266e9b4905c7 for removal 2024.10.22 02:02:28.728636 [ 1476 ] {} CacheMetadata: Key 4653ae8532291878b843266e9b4905c7 is removed from metadata 2024.10.22 02:02:28.733345 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b026242e4fbbdf4bd4c4d7de64b7da1a, offset: 0, size: 0 2024.10.22 02:02:28.737231 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b026242e4fbbdf4bd4c4d7de64b7da1a for removal 2024.10.22 02:02:28.740874 [ 1476 ] {} CacheMetadata: Key b026242e4fbbdf4bd4c4d7de64b7da1a is removed from metadata 2024.10.22 02:02:29.635858 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 84901 2024.10.22 02:02:29.652892 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(37ee1ff32187e163ae11eda6e5a02f67) : [0, 230]: Setting download as failed: File segment: [0, 230], key: 37ee1ff32187e163ae11eda6e5a02f67, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:29.654161 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(37ee1ff32187e163ae11eda6e5a02f67): No space left in cache to reserve 231 bytes, will continue without cache download 2024.10.22 02:02:29.910267 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:29.915853 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:29.985373 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1283_1283_0 (state Temporary) 2024.10.22 02:02:30.145070 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1283_1283_0 to 202410_1296_1296_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:30.344372 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84901 2024.10.22 02:02:30.531565 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3d4bf10777a0f0d4f1004fc2c9732bd8) : [0, 276154]: Setting download as failed: File segment: [0, 276154], key: 3d4bf10777a0f0d4f1004fc2c9732bd8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:30.535576 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3d4bf10777a0f0d4f1004fc2c9732bd8): No space left in cache to reserve 276155 bytes, will continue without cache download 2024.10.22 02:02:30.562361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3d4bf10777a0f0d4f1004fc2c9732bd8, offset: 0, size: 0 2024.10.22 02:02:30.564382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3d4bf10777a0f0d4f1004fc2c9732bd8 for removal 2024.10.22 02:02:30.567697 [ 1476 ] {} CacheMetadata: Key 3d4bf10777a0f0d4f1004fc2c9732bd8 is removed from metadata 2024.10.22 02:02:30.568757 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 37ee1ff32187e163ae11eda6e5a02f67, offset: 0, size: 0 2024.10.22 02:02:30.571980 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 37ee1ff32187e163ae11eda6e5a02f67 for removal 2024.10.22 02:02:30.575383 [ 1476 ] {} CacheMetadata: Key 37ee1ff32187e163ae11eda6e5a02f67 is removed from metadata 2024.10.22 02:02:31.152145 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:02:31.154418 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:02:31.199767 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 11597 2024.10.22 02:02:31.213139 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_690_40, 202410_691_691_0] 2024.10.22 02:02:31.215342 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_690_40, 202410_691_691_0] 2024.10.22 02:02:31.240035 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_690_40, 202410_691_691_0] 2024.10.22 02:02:31.566831 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_730_139/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:02:31.653681 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:31.674557 [ 1242 ] {} DatabaseCatalog: Execution took 238 ms. 2024.10.22 02:02:31.768275 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:31.811778 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_761_761_0 (state Temporary) 2024.10.22 02:02:31.838469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c59398a693ef54029ca3eac5d27d5615) : [0, 238]: Setting download as failed: File segment: [0, 238], key: c59398a693ef54029ca3eac5d27d5615, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:31.842577 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c59398a693ef54029ca3eac5d27d5615): No space left in cache to reserve 239 bytes, will continue without cache download 2024.10.22 02:02:31.870289 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_761_761_0 to 202410_774_774_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:31.902286 [ 1262 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_769_166 to 202410_770_770_0 2024.10.22 02:02:31.912147 [ 1262 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:31.919746 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11597 2024.10.22 02:02:31.919775 [ 1262 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:31.956062 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_770_167} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_769_166 to 202410_770_770_0 into Compact with storage Full 2024.10.22 02:02:32.007946 [ 1252 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:02:32.040002 [ 1252 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:02:32.063939 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_770_167} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:32.112298 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_770_167} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_769_166, total 11635 rows starting from the beginning of the part 2024.10.22 02:02:32.233621 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_770_167} MergeTreeSequentialSource: Reading 2 marks from part 202410_770_770_0, total 8 rows starting from the beginning of the part 2024.10.22 02:02:32.261749 [ 1261 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 331 ms. 2024.10.22 02:02:32.491965 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:32.496037 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:32.566039 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 10.93 MiB, will set to 1.35 GiB (RSS), difference: -7.14 MiB 2024.10.22 02:02:32.655648 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1103500 2024.10.22 02:02:32.888116 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ae56cbb52f8c45fa0ed9e2d5c9501edf) : [0, 651501]: Setting download as failed: File segment: [0, 651501], key: ae56cbb52f8c45fa0ed9e2d5c9501edf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:32.890059 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ae56cbb52f8c45fa0ed9e2d5c9501edf): No space left in cache to reserve 651502 bytes, will continue without cache download 2024.10.22 02:02:32.952686 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ae56cbb52f8c45fa0ed9e2d5c9501edf, offset: 0, size: 0 2024.10.22 02:02:32.954431 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ae56cbb52f8c45fa0ed9e2d5c9501edf for removal 2024.10.22 02:02:32.958050 [ 1476 ] {} CacheMetadata: Key ae56cbb52f8c45fa0ed9e2d5c9501edf is removed from metadata 2024.10.22 02:02:32.963680 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c59398a693ef54029ca3eac5d27d5615, offset: 0, size: 0 2024.10.22 02:02:32.965582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c59398a693ef54029ca3eac5d27d5615 for removal 2024.10.22 02:02:32.968727 [ 1476 ] {} CacheMetadata: Key c59398a693ef54029ca3eac5d27d5615 is removed from metadata 2024.10.22 02:02:33.167817 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_770_167} MergeTask::MergeProjectionsStage: Merge sorted 11643 rows, containing 12 columns (12 merged, 0 gathered) in 1.216289937 sec., 9572.553094303863 rows/sec., 1.49 MiB/sec. 2024.10.22 02:02:33.247306 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:33.249480 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:33.452707 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_770_167} MergedBlockOutputStream: filled checksums 202410_1_770_167 (state Temporary) 2024.10.22 02:02:33.610952 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_729_729_0 (state Temporary) 2024.10.22 02:02:33.676409 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_729_729_0 to 202410_736_736_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:33.824809 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1103500 2024.10.22 02:02:33.910237 [ 1224 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_731_731_0 to 202410_736_736_0 2024.10.22 02:02:33.933479 [ 1224 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:33.940292 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:33.956174 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_731_731_0 to 202410_736_736_0 into Compact with storage Full 2024.10.22 02:02:33.959758 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(a104e597c3fc4798f8304c89baa6ac84) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a104e597c3fc4798f8304c89baa6ac84, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:02:33.968553 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: a104e597c3fc4798f8304c89baa6ac84, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:02:33.972460 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:34.007333 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:34.020784 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_731_731_0, total 1243 rows starting from the beginning of the part 2024.10.22 02:02:34.056609 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_732_732_0, total 1067 rows starting from the beginning of the part 2024.10.22 02:02:34.106783 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_733_733_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:02:34.153638 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_734_734_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:02:34.186643 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_770_167} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_770_167 to 202410_1_770_167 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:34.190212 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_770_167} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_769_166, 202410_770_770_0] -> 202410_1_770_167 2024.10.22 02:02:34.196891 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_735_735_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:02:34.241075 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:02:34.282578 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_736_736_0, total 770 rows starting from the beginning of the part 2024.10.22 02:02:34.356234 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 84978 2024.10.22 02:02:34.463045 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 12.48 MiB, will set to 1.36 GiB (RSS), difference: 2.97 MiB 2024.10.22 02:02:34.653731 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:34.657746 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:34.750867 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1284_1284_0 (state Temporary) 2024.10.22 02:02:34.821686 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1284_1284_0 to 202410_1297_1297_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:34.871257 [ 1162 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1292_17 to 202410_1297_1297_0 2024.10.22 02:02:34.899687 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 84978 2024.10.22 02:02:34.927305 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 84690 2024.10.22 02:02:34.993021 [ 1162 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:34.999622 [ 1162 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:35.006882 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(82e8cde028cbea6c5e96db810ba7f458) : [0, 235]: Setting download as failed: File segment: [0, 235], key: 82e8cde028cbea6c5e96db810ba7f458, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:35.011426 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(82e8cde028cbea6c5e96db810ba7f458): No space left in cache to reserve 236 bytes, will continue without cache download 2024.10.22 02:02:35.052361 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1292_17 to 202410_1297_1297_0 into Compact with storage Full 2024.10.22 02:02:35.081105 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:35.094729 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1292_17, total 5472 rows starting from the beginning of the part 2024.10.22 02:02:35.127587 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_153_153_0 (state Temporary) 2024.10.22 02:02:35.138949 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:35.144303 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:02:35.211007 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1293_1293_0, total 79 rows starting from the beginning of the part 2024.10.22 02:02:35.233374 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_748_748_0 (state Temporary) 2024.10.22 02:02:35.235222 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1294_1294_0, total 92 rows starting from the beginning of the part 2024.10.22 02:02:35.256505 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1295_1295_0, total 58 rows starting from the beginning of the part 2024.10.22 02:02:35.293597 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1296_1296_0, total 59 rows starting from the beginning of the part 2024.10.22 02:02:35.316256 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_1297_1297_0, total 77 rows starting from the beginning of the part 2024.10.22 02:02:35.349223 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_748_748_0 to 202410_754_754_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:35.458243 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} MergeTask::MergeProjectionsStage: Merge sorted 7315 rows, containing 4 columns (4 merged, 0 gathered) in 1.510804222 sec., 4841.792135261851 rows/sec., 75.65 KiB/sec. 2024.10.22 02:02:35.485819 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} MergedBlockOutputStream: filled checksums 202410_731_736_1 (state Temporary) 2024.10.22 02:02:35.571610 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 84690 2024.10.22 02:02:35.676058 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_731_736_1 to 202410_731_736_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:35.681691 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_731_736_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_731_731_0, 202410_736_736_0] -> 202410_731_736_1 2024.10.22 02:02:35.733397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(70498a7692a1f7794651f65df95913ed) : [0, 465533]: Setting download as failed: File segment: [0, 465533], key: 70498a7692a1f7794651f65df95913ed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:35.735208 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(70498a7692a1f7794651f65df95913ed): No space left in cache to reserve 465534 bytes, will continue without cache download 2024.10.22 02:02:35.772041 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 70498a7692a1f7794651f65df95913ed, offset: 0, size: 0 2024.10.22 02:02:35.775457 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 70498a7692a1f7794651f65df95913ed for removal 2024.10.22 02:02:35.780682 [ 1476 ] {} CacheMetadata: Key 70498a7692a1f7794651f65df95913ed is removed from metadata 2024.10.22 02:02:35.781144 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 82e8cde028cbea6c5e96db810ba7f458, offset: 0, size: 0 2024.10.22 02:02:35.786260 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 82e8cde028cbea6c5e96db810ba7f458 for removal 2024.10.22 02:02:35.790761 [ 1476 ] {} CacheMetadata: Key 82e8cde028cbea6c5e96db810ba7f458 is removed from metadata 2024.10.22 02:02:35.843843 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:02:36.273020 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1191_19, 202410_1192_1192_0, 202410_1193_1193_0, 202410_1194_1194_0, 202410_1195_1195_0, 202410_1196_1196_0] 2024.10.22 02:02:36.276373 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1191_19, 202410_1192_1192_0, 202410_1193_1193_0, 202410_1194_1194_0, 202410_1195_1195_0, 202410_1196_1196_0] 2024.10.22 02:02:36.370574 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 13.59 MiB, will set to 1.36 GiB (RSS), difference: -3.00 MiB 2024.10.22 02:02:36.484095 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(32013f481d559a216dcd26a89945c0a2) : [0, 160]: Setting download as failed: File segment: [0, 160], key: 32013f481d559a216dcd26a89945c0a2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:36.485727 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(32013f481d559a216dcd26a89945c0a2): No space left in cache to reserve 161 bytes, will continue without cache download 2024.10.22 02:02:36.564737 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} MergeTask::MergeProjectionsStage: Merge sorted 5837 rows, containing 13 columns (13 merged, 0 gathered) in 1.530966228 sec., 3812.624924865423 rows/sec., 665.49 KiB/sec. 2024.10.22 02:02:36.633564 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} MergedBlockOutputStream: filled checksums 202410_1207_1297_18 (state Temporary) 2024.10.22 02:02:36.708993 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_710_145, 202410_711_711_0, 202410_712_712_0, 202410_713_713_0, 202410_714_714_0, 202410_715_715_0] 2024.10.22 02:02:36.723393 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_710_145, 202410_711_711_0, 202410_712_712_0, 202410_713_713_0, 202410_714_714_0, 202410_715_715_0] 2024.10.22 02:02:36.753726 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_710_145, 202410_711_711_0, 202410_712_712_0, 202410_713_713_0, 202410_714_714_0, 202410_715_715_0] 2024.10.22 02:02:36.954846 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1297_18 to 202410_1207_1297_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:36.959384 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1297_18} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1292_17, 202410_1297_1297_0] -> 202410_1207_1297_18 2024.10.22 02:02:37.006794 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.69 MiB. 2024.10.22 02:02:37.040645 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1191_19, 202410_1192_1192_0, 202410_1193_1193_0, 202410_1194_1194_0, 202410_1195_1195_0, 202410_1196_1196_0] 2024.10.22 02:02:37.259110 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(0e754cafac643132455892686e86d081) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0e754cafac643132455892686e86d081, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:02:37.261552 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 681, file segment info: File segment: [0, 33554431], key: 0e754cafac643132455892686e86d081, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:02:37.263964 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:37.519192 [ 1741 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vrd/lkouhxgnkrkkjsznktmaexpvcjqix, size 1202203 2024.10.22 02:02:37.624154 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:02:37.647459 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:02:37.655204 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:02:37.661358 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:02:37.680981 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:02:37.733506 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:02:37.758277 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:02:37.810384 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yos/tzjoedndrazvhqtbvsejzqoovpbko (LogSeriesLimiter: on interval from 2024-10-22 02:02:28 to 2024-10-22 02:02:37 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:02:37.967785 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b5f5be48bcc7146d32dff5aaac7e842b) : [0, 2403]: Setting download as failed: File segment: [0, 2403], key: b5f5be48bcc7146d32dff5aaac7e842b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:37.969235 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b5f5be48bcc7146d32dff5aaac7e842b): No space left in cache to reserve 2404 bytes, will continue without cache download 2024.10.22 02:02:37.986591 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b5f5be48bcc7146d32dff5aaac7e842b, offset: 0, size: 0 2024.10.22 02:02:37.990111 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b5f5be48bcc7146d32dff5aaac7e842b for removal 2024.10.22 02:02:37.996324 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 32013f481d559a216dcd26a89945c0a2, offset: 0, size: 0 2024.10.22 02:02:38.000185 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 32013f481d559a216dcd26a89945c0a2 for removal 2024.10.22 02:02:38.094493 [ 1476 ] {} CacheMetadata: Key 32013f481d559a216dcd26a89945c0a2 is removed from metadata 2024.10.22 02:02:38.099762 [ 1476 ] {} CacheMetadata: Key b5f5be48bcc7146d32dff5aaac7e842b is removed from metadata 2024.10.22 02:02:38.353772 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 634 2024.10.22 02:02:38.577201 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 10.31 MiB, will set to 1.35 GiB (RSS), difference: -2.54 MiB 2024.10.22 02:02:38.732705 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:38.736958 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:38.887821 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_616_616_0 (state Temporary) 2024.10.22 02:02:38.915307 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 85057 2024.10.22 02:02:38.983725 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_616_616_0 to 202410_626_626_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:39.077636 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 634 2024.10.22 02:02:39.180333 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:39.186415 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:39.193619 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a7aa88895868f394163ed9de5feadd79) : [0, 249]: Setting download as failed: File segment: [0, 249], key: a7aa88895868f394163ed9de5feadd79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:39.196083 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a7aa88895868f394163ed9de5feadd79): No space left in cache to reserve 250 bytes, will continue without cache download 2024.10.22 02:02:39.429168 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 11619 2024.10.22 02:02:39.456158 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1285_1285_0 (state Temporary) 2024.10.22 02:02:39.499311 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1285_1285_0 to 202410_1298_1298_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:39.655366 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:39.659039 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:39.803969 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_762_762_0 (state Temporary) 2024.10.22 02:02:39.870318 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85057 2024.10.22 02:02:39.954165 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_762_762_0 to 202410_775_775_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:39.964294 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5d9383201dbdb6680406c4554afaaf1f) : [0, 1870298]: Setting download as failed: File segment: [0, 1870298], key: 5d9383201dbdb6680406c4554afaaf1f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:39.965724 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5d9383201dbdb6680406c4554afaaf1f): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:02:39.973158 [ 1155 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_770_167 to 202410_771_771_0 2024.10.22 02:02:40.008063 [ 1155 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:40.012822 [ 1155 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:40.031605 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11619 2024.10.22 02:02:40.061340 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_771_168} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_770_167 to 202410_771_771_0 into Compact with storage Full 2024.10.22 02:02:40.064078 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5d9383201dbdb6680406c4554afaaf1f, offset: 0, size: 0 2024.10.22 02:02:40.066488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5d9383201dbdb6680406c4554afaaf1f for removal 2024.10.22 02:02:40.072743 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a7aa88895868f394163ed9de5feadd79, offset: 0, size: 0 2024.10.22 02:02:40.074682 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a7aa88895868f394163ed9de5feadd79 for removal 2024.10.22 02:02:40.170762 [ 1476 ] {} CacheMetadata: Key 5d9383201dbdb6680406c4554afaaf1f is removed from metadata 2024.10.22 02:02:40.180436 [ 1476 ] {} CacheMetadata: Key a7aa88895868f394163ed9de5feadd79 is removed from metadata 2024.10.22 02:02:40.220259 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_771_168} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:40.242115 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_771_168} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_770_167, total 11643 rows starting from the beginning of the part 2024.10.22 02:02:40.458615 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_771_168} MergeTreeSequentialSource: Reading 2 marks from part 202410_771_771_0, total 14 rows starting from the beginning of the part 2024.10.22 02:02:40.464642 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 12.67 MiB, will set to 1.36 GiB (RSS), difference: 3.13 MiB 2024.10.22 02:02:40.828493 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1105040 2024.10.22 02:02:41.163180 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:02:41.164629 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:02:41.263444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d6edda175f4491a4a6a1aea311023d49) : [0, 256]: Setting download as failed: File segment: [0, 256], key: d6edda175f4491a4a6a1aea311023d49, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:41.264697 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d6edda175f4491a4a6a1aea311023d49): No space left in cache to reserve 257 bytes, will continue without cache download 2024.10.22 02:02:41.283067 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:41.290578 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:02:41.512880 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_691_41, 202410_692_692_0] 2024.10.22 02:02:41.516367 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_691_41, 202410_692_692_0] 2024.10.22 02:02:41.541488 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_691_41, 202410_692_692_0] 2024.10.22 02:02:41.659301 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_730_730_0 (state Temporary) 2024.10.22 02:02:41.669895 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_771_168} MergeTask::MergeProjectionsStage: Merge sorted 11657 rows, containing 12 columns (12 merged, 0 gathered) in 1.613748312 sec., 7223.555193407385 rows/sec., 1.12 MiB/sec. 2024.10.22 02:02:41.769044 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_730_730_0 to 202410_737_737_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:41.928654 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1105040 2024.10.22 02:02:41.958493 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_730_139/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:02:42.148609 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_771_168} MergedBlockOutputStream: filled checksums 202410_1_771_168 (state Temporary) 2024.10.22 02:02:42.186370 [ 1185 ] {} DatabaseCatalog: Execution took 401 ms. 2024.10.22 02:02:42.300923 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(de37f86686f75a14339a6b717c67502c) : [0, 2324837]: Setting download as failed: File segment: [0, 2324837], key: de37f86686f75a14339a6b717c67502c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:42.303280 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(de37f86686f75a14339a6b717c67502c): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:02:42.458998 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_153_153_0 to 202410_156_156_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:42.466820 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_771_168} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_771_168 to 202410_1_771_168 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:42.472001 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_771_168} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_770_167, 202410_771_771_0] -> 202410_1_771_168 2024.10.22 02:02:42.526557 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_573_117, 202410_574_574_0, 202410_575_575_0, 202410_576_576_0, 202410_577_577_0, 202410_578_578_0] 2024.10.22 02:02:42.531160 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_154_26/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:02:42.532037 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_573_117, 202410_574_574_0, 202410_575_575_0, 202410_576_576_0, 202410_577_577_0, 202410_578_578_0] 2024.10.22 02:02:42.576306 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_573_117, 202410_574_574_0, 202410_575_575_0, 202410_576_576_0, 202410_577_577_0, 202410_578_578_0] 2024.10.22 02:02:42.632391 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:02:42.663322 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: de37f86686f75a14339a6b717c67502c, offset: 0, size: 0 2024.10.22 02:02:42.665536 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key de37f86686f75a14339a6b717c67502c for removal 2024.10.22 02:02:42.678349 [ 1476 ] {} CacheMetadata: Key de37f86686f75a14339a6b717c67502c is removed from metadata 2024.10.22 02:02:42.684275 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d6edda175f4491a4a6a1aea311023d49, offset: 0, size: 0 2024.10.22 02:02:42.690115 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d6edda175f4491a4a6a1aea311023d49 for removal 2024.10.22 02:02:42.695986 [ 1476 ] {} CacheMetadata: Key d6edda175f4491a4a6a1aea311023d49 is removed from metadata 2024.10.22 02:02:43.111482 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 97 entries to flush up to offset 84787 2024.10.22 02:02:43.418814 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:43.423960 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:43.448858 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5b4b31da91e7b86d988fda83b664c824) : [0, 242]: Setting download as failed: File segment: [0, 242], key: 5b4b31da91e7b86d988fda83b664c824, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:43.555381 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 12.13 MiB, will set to 1.36 GiB (RSS), difference: 4.09 MiB 2024.10.22 02:02:43.528912 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5b4b31da91e7b86d988fda83b664c824): No space left in cache to reserve 243 bytes, will continue without cache download 2024.10.22 02:02:43.669511 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_749_749_0 (state Temporary) 2024.10.22 02:02:43.715109 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.069209364 sec. 2024.10.22 02:02:43.893697 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 85130 2024.10.22 02:02:43.931036 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_749_749_0 to 202410_755_755_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:43.968091 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 84787 2024.10.22 02:02:44.102817 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:44.106538 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:44.321768 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(746c63328d5ca2c062111c510e428daa) : [0, 856760]: Setting download as failed: File segment: [0, 856760], key: 746c63328d5ca2c062111c510e428daa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:44.324605 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(746c63328d5ca2c062111c510e428daa): No space left in cache to reserve 856761 bytes, will continue without cache download 2024.10.22 02:02:44.327277 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1286_1286_0 (state Temporary) 2024.10.22 02:02:44.399212 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1286_1286_0 to 202410_1299_1299_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:44.460031 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 746c63328d5ca2c062111c510e428daa, offset: 0, size: 0 2024.10.22 02:02:44.461458 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 746c63328d5ca2c062111c510e428daa for removal 2024.10.22 02:02:44.466213 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5b4b31da91e7b86d988fda83b664c824, offset: 0, size: 0 2024.10.22 02:02:44.467684 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5b4b31da91e7b86d988fda83b664c824 for removal 2024.10.22 02:02:44.523811 [ 1476 ] {} CacheMetadata: Key 746c63328d5ca2c062111c510e428daa is removed from metadata 2024.10.22 02:02:44.535637 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85130 2024.10.22 02:02:44.541294 [ 1207 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 249 ms. 2024.10.22 02:02:44.632160 [ 1476 ] {} CacheMetadata: Key 5b4b31da91e7b86d988fda83b664c824 is removed from metadata 2024.10.22 02:02:45.408738 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(981f1ccd2e336e3dbaf46b4c9c46c7b6) : [0, 167]: Setting download as failed: File segment: [0, 167], key: 981f1ccd2e336e3dbaf46b4c9c46c7b6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:45.439580 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6154 2024.10.22 02:02:45.366192 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 16.83 MiB, will set to 1.37 GiB (RSS), difference: 3.10 MiB 2024.10.22 02:02:45.512757 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(981f1ccd2e336e3dbaf46b4c9c46c7b6): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 02:02:46.242946 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a9f01ca9e217c4750371cc6df2769ce3) : [0, 191426]: Setting download as failed: File segment: [0, 191426], key: a9f01ca9e217c4750371cc6df2769ce3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:02:46.244633 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a9f01ca9e217c4750371cc6df2769ce3): No space left in cache to reserve 191427 bytes, will continue without cache download 2024.10.22 02:02:46.274133 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a9f01ca9e217c4750371cc6df2769ce3, offset: 0, size: 0 2024.10.22 02:02:46.276542 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a9f01ca9e217c4750371cc6df2769ce3 for removal 2024.10.22 02:02:46.282378 [ 1476 ] {} CacheMetadata: Key a9f01ca9e217c4750371cc6df2769ce3 is removed from metadata 2024.10.22 02:02:46.284538 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 981f1ccd2e336e3dbaf46b4c9c46c7b6, offset: 0, size: 0 2024.10.22 02:02:46.286499 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 981f1ccd2e336e3dbaf46b4c9c46c7b6 for removal 2024.10.22 02:02:46.288964 [ 1476 ] {} CacheMetadata: Key 981f1ccd2e336e3dbaf46b4c9c46c7b6 is removed from metadata 2024.10.22 02:02:46.432808 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/wmd/qaqvhjvwcoaaifjteibmrybavwtlb. 2024.10.22 02:02:46.581481 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 635 2024.10.22 02:02:47.000978 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:47.005381 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:47.052967 [ 1198 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:02:47.066754 [ 1198 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:02:47.071025 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_617_617_0 (state Temporary) 2024.10.22 02:02:47.248333 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_692_42, 202410_693_693_0] 2024.10.22 02:02:47.251441 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_692_42, 202410_693_693_0] 2024.10.22 02:02:47.262035 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_692_42, 202410_693_693_0] 2024.10.22 02:02:47.267650 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_617_617_0 to 202410_627_627_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:47.354639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/wmd/qaqvhjvwcoaaifjteibmrybavwtlb. 2024.10.22 02:02:47.358838 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_11_11_0 to 201403_11_11_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:47.469494 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 19.45 MiB, will set to 1.37 GiB (RSS), difference: 61.15 MiB 2024.10.22 02:02:47.537049 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 11630 2024.10.22 02:02:47.576539 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 635 2024.10.22 02:02:47.737829 [ 1250 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 365 ms. 2024.10.22 02:02:47.743072 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_12_12_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:02:47.763187 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:47.765383 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:48.012931 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_763_763_0 (state Temporary) 2024.10.22 02:02:48.056725 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_763_763_0 to 202410_776_776_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:48.085304 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_771_168 to 202410_772_772_0 2024.10.22 02:02:48.196620 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11630 2024.10.22 02:02:48.199966 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:48.203578 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:48.243423 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_772_169} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_771_168 to 202410_772_772_0 into Compact with storage Full 2024.10.22 02:02:48.296783 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_772_169} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:48.316111 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_772_169} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_771_168, total 11657 rows starting from the beginning of the part 2024.10.22 02:02:48.347005 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_772_169} MergeTreeSequentialSource: Reading 2 marks from part 202410_772_772_0, total 19 rows starting from the beginning of the part 2024.10.22 02:02:48.545595 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 85189 2024.10.22 02:02:48.664603 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(f845286c1e24cc032e6be7fbc2a5c6b1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f845286c1e24cc032e6be7fbc2a5c6b1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:02:48.671593 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 13008, file segment info: File segment: [0, 33554431], key: f845286c1e24cc032e6be7fbc2a5c6b1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:02:48.674435 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:48.677045 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:48.683831 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:48.764569 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1287_1287_0 (state Temporary) 2024.10.22 02:02:49.020615 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1287_1287_0 to 202410_1300_1300_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:49.081943 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1106580 2024.10.22 02:02:49.341231 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_772_169} MergeTask::MergeProjectionsStage: Merge sorted 11676 rows, containing 12 columns (12 merged, 0 gathered) in 1.103554698 sec., 10580.354577041546 rows/sec., 1.65 MiB/sec. 2024.10.22 02:02:49.506969 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_772_169} MergedBlockOutputStream: filled checksums 202410_1_772_169 (state Temporary) 2024.10.22 02:02:49.587422 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 982.24 MiB, peak 2.00 GiB, free memory in arenas 21.59 MiB, will set to 1.13 GiB (RSS), difference: 174.93 MiB 2024.10.22 02:02:49.952316 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(d869f12035091d3314ac847bab514e22) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d869f12035091d3314ac847bab514e22, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:02:49.954932 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 510, file segment info: File segment: [0, 33554431], key: d869f12035091d3314ac847bab514e22, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:02:49.956947 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:50.016789 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_772_169} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_772_169 to 202410_1_772_169 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:50.029192 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_772_169} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_771_168, 202410_772_772_0] -> 202410_1_772_169 2024.10.22 02:02:50.044410 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85189 2024.10.22 02:02:50.070737 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:50.074409 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:50.162268 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:02:50.528633 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_731_731_0 (state Temporary) 2024.10.22 02:02:50.560794 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 44 2024.10.22 02:02:50.756211 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_731_731_0 to 202410_738_738_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:50.831083 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:50.801270 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(537179d766582fb3aaea09113b12ab20) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 537179d766582fb3aaea09113b12ab20, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:02:50.840868 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:50.906317 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1106580 2024.10.22 02:02:50.940832 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 14250, file segment info: File segment: [0, 33554431], key: 537179d766582fb3aaea09113b12ab20, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:02:50.944846 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:51.094724 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_38_38_0 (state Temporary) 2024.10.22 02:02:51.164649 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:02:51.167046 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:02:51.368496 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_38_38_0 to 202410_38_38_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:51.437305 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 44 2024.10.22 02:02:51.484283 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 105 entries to flush up to offset 84892 2024.10.22 02:02:51.604818 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:51.615548 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:51.769070 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_750_750_0 (state Temporary) 2024.10.22 02:02:51.874287 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(42b265dfee6b192487ab79adc2cb16a9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 42b265dfee6b192487ab79adc2cb16a9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:02:51.884955 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 509, file segment info: File segment: [0, 33554431], key: 42b265dfee6b192487ab79adc2cb16a9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:02:51.890739 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:51.932324 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_750_750_0 to 202410_756_756_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:51.959460 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_730_139} MergeTask::MergeProjectionsStage: Merge sorted 1109775 rows, containing 4 columns (4 merged, 0 gathered) in 34.50764095 sec., 32160.26855060922 rows/sec., 482.33 KiB/sec. 2024.10.22 02:02:52.091904 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 84892 2024.10.22 02:02:52.137411 [ 1746 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xox/ldsyqaqljqmhaavuufjmogombczae, size 14250 2024.10.22 02:02:52.143493 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_730_139} MergedBlockOutputStream: filled checksums 202410_1_730_139 (state Temporary) 2024.10.22 02:02:52.411174 [ 1222 ] {} DatabaseCatalog: Execution took 227 ms. 2024.10.22 02:02:52.551370 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:02:52.559109 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:02:52.575394 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:02:52.632512 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:02:52.648187 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(1b77af38f1a623d44b39661dfdc51eeb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1b77af38f1a623d44b39661dfdc51eeb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:02:52.670654 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 633185, file segment info: File segment: [0, 33554431], key: 1b77af38f1a623d44b39661dfdc51eeb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:02:52.681647 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:52.683843 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:02:52.763036 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.23 GiB, peak 2.00 GiB, free memory in arenas 27.15 MiB, will set to 1.18 GiB (RSS), difference: -45.02 MiB 2024.10.22 02:02:52.829275 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:02:52.954009 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:02:52.955704 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 6191 2024.10.22 02:02:53.135184 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_730_139/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:02:53.016881 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:02:53.330311 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jcc/pqraihiyvimmabzvykrjunforelqk (LogSeriesLimiter: on interval from 2024-10-22 02:02:37 to 2024-10-22 02:02:53 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:02:53.597267 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(bc327432b8a0ec20862822a8592469b7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: bc327432b8a0ec20862822a8592469b7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:02:53.631030 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 662, file segment info: File segment: [0, 33554431], key: bc327432b8a0ec20862822a8592469b7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:02:53.633738 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:02:53.780286 [ 1684 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yos/tzjoedndrazvhqtbvsejzqoovpbko, size 633185 2024.10.22 02:02:54.050041 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 85256 2024.10.22 02:02:54.041819 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:02:54.072971 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:02:54.108820 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:02:54.142941 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:02:54.262834 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:02:54.306183 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:02:54.359473 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:54.361754 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:54.372624 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:02:54.459973 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1288_1288_0 (state Temporary) 2024.10.22 02:02:54.738791 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1288_1288_0 to 202410_1301_1301_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:54.769549 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85256 2024.10.22 02:02:55.083742 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 636 2024.10.22 02:02:55.435493 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_730_139} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_730_139 to 202410_1_730_139 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:55.442887 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_730_139} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_724_138, 202410_725_730_1] -> 202410_1_730_139 2024.10.22 02:02:55.552411 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.24 GiB, peak 2.00 GiB, free memory in arenas 18.04 MiB, will set to 1.21 GiB (RSS), difference: -37.23 MiB 2024.10.22 02:02:55.566070 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:02:55.587942 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:55.599335 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:55.661809 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_618_618_0 (state Temporary) 2024.10.22 02:02:55.699554 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 11642 2024.10.22 02:02:55.765109 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_693_43, 202410_694_694_0] 2024.10.22 02:02:55.766782 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_693_43, 202410_694_694_0] 2024.10.22 02:02:55.863299 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_154_26/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:02:55.867372 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_618_618_0 to 202410_628_628_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:55.884611 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_693_43, 202410_694_694_0] 2024.10.22 02:02:55.898198 [ 1165 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_623_127 to 202410_628_628_0 2024.10.22 02:02:55.913362 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:55.918984 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:55.921367 [ 1165 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:55.934406 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:55.981402 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 636 2024.10.22 02:02:56.027079 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_623_127 to 202410_628_628_0 into Compact with storage Full 2024.10.22 02:02:56.063419 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_764_764_0 (state Temporary) 2024.10.22 02:02:56.121012 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_764_764_0 to 202410_777_777_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:56.155867 [ 1195 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_772_169 to 202410_773_773_0 2024.10.22 02:02:56.160630 [ 1195 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:56.164340 [ 1195 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:56.167125 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11642 2024.10.22 02:02:56.347959 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:56.370092 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_623_127, total 667 rows starting from the beginning of the part 2024.10.22 02:02:56.387840 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_773_170} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_772_169 to 202410_773_773_0 into Compact with storage Full 2024.10.22 02:02:56.459275 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_624_624_0, total 1 rows starting from the beginning of the part 2024.10.22 02:02:56.558089 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_625_625_0, total 1 rows starting from the beginning of the part 2024.10.22 02:02:56.591270 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_773_170} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:56.635048 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_773_170} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_772_169, total 11676 rows starting from the beginning of the part 2024.10.22 02:02:56.681590 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_626_626_0, total 1 rows starting from the beginning of the part 2024.10.22 02:02:56.700468 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_773_170} MergeTreeSequentialSource: Reading 2 marks from part 202410_773_773_0, total 19 rows starting from the beginning of the part 2024.10.22 02:02:56.812324 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_627_627_0, total 1 rows starting from the beginning of the part 2024.10.22 02:02:57.063364 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_628_628_0, total 1 rows starting from the beginning of the part 2024.10.22 02:02:57.145201 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_677_677_0, 202410_678_678_0, 202410_679_679_0, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0] 2024.10.22 02:02:57.149467 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_677_677_0, 202410_678_678_0, 202410_679_679_0, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0] 2024.10.22 02:02:57.174753 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_677_677_0, 202410_678_678_0, 202410_679_679_0, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0] 2024.10.22 02:02:57.711354 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_773_170} MergeTask::MergeProjectionsStage: Merge sorted 11695 rows, containing 12 columns (12 merged, 0 gathered) in 1.438826176 sec., 8128.153487249317 rows/sec., 1.27 MiB/sec. 2024.10.22 02:02:57.776430 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_773_170} MergedBlockOutputStream: filled checksums 202410_1_773_170 (state Temporary) 2024.10.22 02:02:57.803340 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 2.00 GiB, free memory in arenas 14.50 MiB, will set to 1.23 GiB (RSS), difference: -61.72 MiB 2024.10.22 02:02:57.944945 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1107735 2024.10.22 02:02:57.967737 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_12_12_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:02:58.083008 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_773_170} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_773_170 to 202410_1_773_170 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:58.096508 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_773_170} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_772_169, 202410_773_773_0] -> 202410_1_773_170 2024.10.22 02:02:58.127931 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:02:58.776928 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 85318 2024.10.22 02:02:58.933972 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:58.940937 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:58.945055 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:58.948228 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:59.004297 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1289_1289_0 (state Temporary) 2024.10.22 02:02:59.057694 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1289_1289_0 to 202410_1302_1302_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:59.072676 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1297_18 to 202410_1302_1302_0 2024.10.22 02:02:59.103152 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:59.108269 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:59.136892 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85318 2024.10.22 02:02:59.146949 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1297_18 to 202410_1302_1302_0 into Compact with storage Full 2024.10.22 02:02:59.160224 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_732_732_0 (state Temporary) 2024.10.22 02:02:59.255317 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1196_20, 202410_1197_1197_0, 202410_1198_1198_0, 202410_1199_1199_0, 202410_1200_1200_0, 202410_1201_1201_0] 2024.10.22 02:02:59.259559 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1196_20, 202410_1197_1197_0, 202410_1198_1198_0, 202410_1199_1199_0, 202410_1200_1200_0, 202410_1201_1201_0] 2024.10.22 02:02:59.301560 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1196_20, 202410_1197_1197_0, 202410_1198_1198_0, 202410_1199_1199_0, 202410_1200_1200_0, 202410_1201_1201_0] 2024.10.22 02:02:59.315017 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:02:59.342815 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_732_732_0 to 202410_739_739_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:02:59.351229 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1297_18, total 5837 rows starting from the beginning of the part 2024.10.22 02:02:59.352135 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 9.10 MiB, will set to 1.28 GiB (RSS), difference: -34.03 MiB 2024.10.22 02:02:59.366862 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1107735 2024.10.22 02:02:59.470537 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1298_1298_0, total 79 rows starting from the beginning of the part 2024.10.22 02:02:59.604780 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 87 entries to flush up to offset 84979 2024.10.22 02:02:59.633147 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_628_128/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:02:59.658764 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} MergeTask::MergeProjectionsStage: Merge sorted 672 rows, containing 32 columns (32 merged, 0 gathered) in 3.690399966 sec., 182.09408361998666 rows/sec., 16.53 KiB/sec. 2024.10.22 02:02:59.693463 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1299_1299_0, total 73 rows starting from the beginning of the part 2024.10.22 02:02:59.754783 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:02:59.781353 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1300_1300_0, total 59 rows starting from the beginning of the part 2024.10.22 02:02:59.849298 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:02:59.896288 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_751_751_0 (state Temporary) 2024.10.22 02:02:59.921686 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} MergedBlockOutputStream: filled checksums 202410_1_628_128 (state Temporary) 2024.10.22 02:03:00.153719 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_751_751_0 to 202410_757_757_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:00.205354 [ 1187 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_752_1 to 202410_757_757_0 2024.10.22 02:03:00.244435 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 84979 2024.10.22 02:03:00.261778 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_628_128 to 202410_1_628_128 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:00.276423 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_628_128} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_623_127, 202410_628_628_0] -> 202410_1_628_128 2024.10.22 02:03:00.313411 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1301_1301_0, total 67 rows starting from the beginning of the part 2024.10.22 02:03:00.339994 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:03:00.441647 [ 1187 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:00.442993 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_1302_1302_0, total 62 rows starting from the beginning of the part 2024.10.22 02:03:00.444369 [ 1187 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:00.448623 [ 1187 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 267 ms. 2024.10.22 02:03:00.469662 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_752_1 to 202410_757_757_0 into Compact with storage Full 2024.10.22 02:03:00.602406 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 7.36 MiB, will set to 1.31 GiB (RSS), difference: -31.39 MiB 2024.10.22 02:03:00.659077 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:00.687556 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_752_1, total 1128 rows starting from the beginning of the part 2024.10.22 02:03:00.714068 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_753_753_0, total 117 rows starting from the beginning of the part 2024.10.22 02:03:00.754724 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_754_754_0, total 90 rows starting from the beginning of the part 2024.10.22 02:03:00.771899 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_755_755_0, total 97 rows starting from the beginning of the part 2024.10.22 02:03:00.808460 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_756_756_0, total 105 rows starting from the beginning of the part 2024.10.22 02:03:00.839059 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_757_757_0, total 87 rows starting from the beginning of the part 2024.10.22 02:03:00.990987 [ 1185 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 340 ms. 2024.10.22 02:03:01.164182 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:03:01.165559 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:03:01.911151 [ 1235 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 02:03:02.056325 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1302_19/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:03:02.071367 [ 1220 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:03:02.226442 [ 1220 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:03:02.355528 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} MergeTask::MergeProjectionsStage: Merge sorted 1624 rows, containing 13 columns (13 merged, 0 gathered) in 1.893100736 sec., 857.8518665791697 rows/sec., 386.66 KiB/sec. 2024.10.22 02:03:02.490649 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} MergeTask::MergeProjectionsStage: Merge sorted 6177 rows, containing 13 columns (13 merged, 0 gathered) in 3.362911581 sec., 1836.8011918300863 rows/sec., 321.23 KiB/sec. 2024.10.22 02:03:02.503297 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} MergedBlockOutputStream: filled checksums 202410_747_757_2 (state Temporary) 2024.10.22 02:03:02.577612 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 2.00 GiB, free memory in arenas 17.25 MiB, will set to 1.36 GiB (RSS), difference: -14.97 MiB 2024.10.22 02:03:02.744660 [ 1234 ] {} DatabaseCatalog: Execution took 311 ms. 2024.10.22 02:03:02.745595 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} MergedBlockOutputStream: filled checksums 202410_1207_1302_19 (state Temporary) 2024.10.22 02:03:02.747436 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_757_2 to 202410_747_757_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:02.752294 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_757_2} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_752_1, 202410_757_757_0] -> 202410_747_757_2 2024.10.22 02:03:02.881069 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:03:02.938682 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1302_19 to 202410_1207_1302_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:02.943356 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1302_19} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1297_18, 202410_1302_1302_0] -> 202410_1207_1302_19 2024.10.22 02:03:03.148580 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 85387 2024.10.22 02:03:03.156403 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.73 MiB. 2024.10.22 02:03:03.269211 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:03.271906 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:03.321102 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1290_1290_0 (state Temporary) 2024.10.22 02:03:03.466331 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1290_1290_0 to 202410_1303_1303_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:03.486038 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 637 2024.10.22 02:03:03.492313 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 2.00 GiB, free memory in arenas 17.11 MiB, will set to 1.34 GiB (RSS), difference: -79.17 MiB 2024.10.22 02:03:03.669690 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 30 entries to flush up to offset 11672 2024.10.22 02:03:03.727208 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85387 2024.10.22 02:03:03.824104 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:03.827878 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:03.866501 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_765_765_0 (state Temporary) 2024.10.22 02:03:03.917364 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:03.925358 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:04.108902 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_765_765_0 to 202410_778_778_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:04.150256 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_619_619_0 (state Temporary) 2024.10.22 02:03:04.171653 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11672 2024.10.22 02:03:04.140332 [ 1269 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_773_170 to 202410_778_778_0 2024.10.22 02:03:04.262195 [ 1269 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:04.265448 [ 1269 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:04.387026 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_773_170 to 202410_778_778_0 into Compact with storage Full 2024.10.22 02:03:04.531357 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_619_619_0 to 202410_629_629_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:04.550839 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:04.558878 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_773_170, total 11695 rows starting from the beginning of the part 2024.10.22 02:03:04.638719 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_694_44, 202410_695_695_0] 2024.10.22 02:03:04.640959 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_694_44, 202410_695_695_0] 2024.10.22 02:03:04.653721 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_694_44, 202410_695_695_0] 2024.10.22 02:03:04.685563 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} MergeTreeSequentialSource: Reading 2 marks from part 202410_774_774_0, total 8 rows starting from the beginning of the part 2024.10.22 02:03:04.718438 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} MergeTreeSequentialSource: Reading 2 marks from part 202410_775_775_0, total 22 rows starting from the beginning of the part 2024.10.22 02:03:04.815337 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 637 2024.10.22 02:03:04.868364 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} MergeTreeSequentialSource: Reading 2 marks from part 202410_776_776_0, total 11 rows starting from the beginning of the part 2024.10.22 02:03:04.917588 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} MergeTreeSequentialSource: Reading 2 marks from part 202410_777_777_0, total 12 rows starting from the beginning of the part 2024.10.22 02:03:04.964993 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} MergeTreeSequentialSource: Reading 2 marks from part 202410_778_778_0, total 30 rows starting from the beginning of the part 2024.10.22 02:03:05.447999 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 2.00 GiB, free memory in arenas 16.25 MiB, will set to 1.39 GiB (RSS), difference: -87.30 MiB 2024.10.22 02:03:06.306982 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} MergeTask::MergeProjectionsStage: Merge sorted 11778 rows, containing 12 columns (12 merged, 0 gathered) in 2.03185244 sec., 5796.680786524044 rows/sec., 924.63 KiB/sec. 2024.10.22 02:03:06.360606 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} MergedBlockOutputStream: filled checksums 202410_1_778_171 (state Temporary) 2024.10.22 02:03:06.371291 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2310 entries to flush up to offset 1110045 2024.10.22 02:03:06.645113 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_778_171 to 202410_1_778_171 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:06.649415 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_778_171} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_773_170, 202410_778_778_0] -> 202410_1_778_171 2024.10.22 02:03:06.697330 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.54 MiB. 2024.10.22 02:03:07.442319 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:07.447408 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:07.574658 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_733_733_0 (state Temporary) 2024.10.22 02:03:07.663758 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_733_733_0 to 202410_740_740_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:07.589768 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 2.00 GiB, free memory in arenas 15.13 MiB, will set to 1.41 GiB (RSS), difference: 3.19 MiB 2024.10.22 02:03:07.700729 [ 1273 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_730_139 to 202410_731_736_1 2024.10.22 02:03:07.727974 [ 1273 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:07.730748 [ 1273 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:07.736970 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 85435 2024.10.22 02:03:07.757427 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 94 entries to flush up to offset 85073 2024.10.22 02:03:07.765358 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_736_140} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_730_139 to 202410_731_736_1 into Wide with storage Full 2024.10.22 02:03:07.801721 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_736_140} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:07.808120 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1110045 2024.10.22 02:03:07.812349 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_736_140} MergeTreeSequentialSource: Reading 137 marks from part 202410_1_730_139, total 1109775 rows starting from the beginning of the part 2024.10.22 02:03:07.862849 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_736_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_731_736_1, total 7315 rows starting from the beginning of the part 2024.10.22 02:03:07.870109 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:07.873577 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:07.904158 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:07.911927 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:07.955293 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1291_1291_0 (state Temporary) 2024.10.22 02:03:08.009083 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1291_1291_0 to 202410_1304_1304_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:08.077944 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85435 2024.10.22 02:03:08.133823 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_752_752_0 (state Temporary) 2024.10.22 02:03:08.166239 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_154_26/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:03:08.297764 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_752_752_0 to 202410_758_758_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:08.354206 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85073 2024.10.22 02:03:08.696846 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:08.706871 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:08.960830 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(043f04c882f417234b035ddae93603af) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 043f04c882f417234b035ddae93603af, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 02:03:08.967213 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 18211, file segment info: File segment: [0, 33554431], key: 043f04c882f417234b035ddae93603af, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 02:03:08.968586 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:09.373394 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 15.07 MiB, will set to 1.47 GiB (RSS), difference: -34.09 MiB 2024.10.22 02:03:09.656899 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_736_140/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:03:09.659964 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(dc8b8f7f720c98946eae05d2c896deeb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: dc8b8f7f720c98946eae05d2c896deeb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:03:09.664220 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 431, file segment info: File segment: [0, 33554431], key: dc8b8f7f720c98946eae05d2c896deeb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:03:09.666741 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:10.695606 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(aef446d6bc73b1a668d119a9da55b5e7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: aef446d6bc73b1a668d119a9da55b5e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 02:03:10.710963 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 511, file segment info: File segment: [0, 33554431], key: aef446d6bc73b1a668d119a9da55b5e7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 02:03:10.713749 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:10.770310 [ 1261 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 02:03:11.052021 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:03:11.057646 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:03:11.065458 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:03:11.096227 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:03:11.118566 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:03:11.134416 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_154_154_0 (state Temporary) 2024.10.22 02:03:11.164636 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:03:11.168216 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:03:11.255124 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:03:11.269184 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:03:11.333366 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xxt/efdtfzkvaeuacxjzdrkqvwdikdqwu (LogSeriesLimiter: on interval from 2024-10-22 02:02:53 to 2024-10-22 02:03:11 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:03:11.515208 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(202414651a10896f92770f368be4c06e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 202414651a10896f92770f368be4c06e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:03:11.519749 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 9642, file segment info: File segment: [0, 33554431], key: 202414651a10896f92770f368be4c06e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:03:11.526635 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:11.675073 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 11684 2024.10.22 02:03:11.792689 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:11.798883 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:11.963395 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_766_766_0 (state Temporary) 2024.10.22 02:03:12.100372 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 85486 2024.10.22 02:03:12.162149 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_766_766_0 to 202410_779_779_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:12.256937 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11684 2024.10.22 02:03:12.303715 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_695_45, 202410_696_696_0] 2024.10.22 02:03:12.307985 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_695_45, 202410_696_696_0] 2024.10.22 02:03:12.320768 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 638 2024.10.22 02:03:12.325653 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_695_45, 202410_696_696_0] 2024.10.22 02:03:12.407678 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(26ad8d56cec4cd01cc7bff00d848021f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 26ad8d56cec4cd01cc7bff00d848021f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:03:12.428318 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 499, file segment info: File segment: [0, 33554431], key: 26ad8d56cec4cd01cc7bff00d848021f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:03:12.442218 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:12.485503 [ 1745 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jcc/pqraihiyvimmabzvykrjunforelqk, size 9642 2024.10.22 02:03:12.539622 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_670_129, 202410_671_676_1] 2024.10.22 02:03:12.562368 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_670_129, 202410_671_676_1] 2024.10.22 02:03:12.694623 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 26.14 MiB, will set to 1.53 GiB (RSS), difference: -90.10 MiB 2024.10.22 02:03:12.713218 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:12.716713 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:12.723772 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:12.728955 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_670_129, 202410_671_676_1] 2024.10.22 02:03:12.729355 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:12.786561 [ 1154 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Selected 5 parts from 202410_1_34_8 to 202410_38_38_0 2024.10.22 02:03:12.819888 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1292_1292_0 (state Temporary) 2024.10.22 02:03:12.822238 [ 1154 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:12.831777 [ 1154 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:12.859854 [ 1392 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_38_9} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_34_8 to 202410_38_38_0 into Compact with storage Full 2024.10.22 02:03:12.929608 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_620_620_0 (state Temporary) 2024.10.22 02:03:12.955692 [ 1219 ] {} DatabaseCatalog: Execution took 230 ms. 2024.10.22 02:03:13.064514 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1292_1292_0 to 202410_1305_1305_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:13.130707 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85486 2024.10.22 02:03:13.231928 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_620_620_0 to 202410_630_630_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:13.246405 [ 1392 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_38_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:13.351538 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:03:13.382210 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:03:13.405086 [ 1392 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_38_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_34_8, total 40 rows starting from the beginning of the part 2024.10.22 02:03:13.412114 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 638 2024.10.22 02:03:13.431290 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:03:13.443866 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:03:13.557454 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:03:13.578787 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:03:13.615313 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:03:13.633310 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:03:13.646026 [ 1392 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_38_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_35_35_0, total 1 rows starting from the beginning of the part 2024.10.22 02:03:13.952207 [ 1392 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_38_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_36_36_0, total 1 rows starting from the beginning of the part 2024.10.22 02:03:14.053718 [ 1392 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_38_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_37_37_0, total 1 rows starting from the beginning of the part 2024.10.22 02:03:14.334182 [ 1392 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_38_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_38_38_0, total 1 rows starting from the beginning of the part 2024.10.22 02:03:14.430606 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 30.52 MiB, will set to 1.58 GiB (RSS), difference: -11.99 MiB 2024.10.22 02:03:14.822571 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1190 entries to flush up to offset 1111235 2024.10.22 02:03:15.556108 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_12_12_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:03:15.862462 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 128 entries to flush up to offset 85201 2024.10.22 02:03:16.097331 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:16.099930 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:16.146020 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:16.150560 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:16.273726 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_734_734_0 (state Temporary) 2024.10.22 02:03:16.351128 [ 1391 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_38_9} MergeTask::MergeProjectionsStage: Merge sorted 44 rows, containing 27 columns (27 merged, 0 gathered) in 3.494455189 sec., 12.59137622897702 rows/sec., 35.02 KiB/sec. 2024.10.22 02:03:16.369172 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_734_734_0 to 202410_741_741_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:16.383658 [ 1385 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_38_9} MergedBlockOutputStream: filled checksums 202410_1_38_9 (state Temporary) 2024.10.22 02:03:16.407702 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_753_753_0 (state Temporary) 2024.10.22 02:03:16.537440 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1111235 2024.10.22 02:03:16.521028 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 2.00 GiB, free memory in arenas 31.69 MiB, will set to 1.63 GiB (RSS), difference: -102.23 MiB 2024.10.22 02:03:16.552179 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 4 old parts to remove. Parts: [202410_1_715_146, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0] 2024.10.22 02:03:16.559201 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from filesystem (serially): Parts: [202410_1_715_146, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0] 2024.10.22 02:03:16.585830 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_753_753_0 to 202410_759_759_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:16.588548 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from memory: Parts: [202410_1_715_146, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0] 2024.10.22 02:03:16.944391 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85201 2024.10.22 02:03:17.147112 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 85550 2024.10.22 02:03:17.234546 [ 1158 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:03:17.264283 [ 1158 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:03:17.309979 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:17.318326 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:17.402098 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1293_1293_0 (state Temporary) 2024.10.22 02:03:17.458466 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_38_9} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_merge_202410_1_38_9 to 202410_1_38_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:17.461171 [ 1395 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_38_9} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Merged 5 parts: [202410_1_34_8, 202410_38_38_0] -> 202410_1_38_9 2024.10.22 02:03:17.496186 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.24 MiB. 2024.10.22 02:03:17.556240 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1293_1293_0 to 202410_1306_1306_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:17.752607 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85550 2024.10.22 02:03:18.015978 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(ed3c5fc983fb77b3521847057998c497) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ed3c5fc983fb77b3521847057998c497, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:03:18.023728 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: ed3c5fc983fb77b3521847057998c497, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:03:18.027634 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:18.675080 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.81 GiB, peak 2.00 GiB, free memory in arenas 22.09 MiB, will set to 1.77 GiB (RSS), difference: -40.63 MiB 2024.10.22 02:03:18.740548 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_154_26/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:03:19.763595 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 11696 2024.10.22 02:03:19.983654 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:20.000904 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:20.146214 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_767_767_0 (state Temporary) 2024.10.22 02:03:20.189834 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_767_767_0 to 202410_780_780_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:20.478876 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11696 2024.10.22 02:03:20.649340 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_154_154_0 to 202410_157_157_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:20.757761 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 2.00 GiB, free memory in arenas 18.90 MiB, will set to 1.59 GiB (RSS), difference: 294.00 MiB 2024.10.22 02:03:21.167807 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:03:21.169613 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:03:21.226792 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_736_140/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:03:21.327615 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 495.61 MiB using storage policy from min volume index 0 2024.10.22 02:03:21.341243 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_696_46, 202410_697_697_0] 2024.10.22 02:03:21.346553 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_696_46, 202410_697_697_0] 2024.10.22 02:03:21.356725 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_696_46, 202410_697_697_0] 2024.10.22 02:03:21.752555 [ 1264 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 02:03:21.760798 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 85582 2024.10.22 02:03:22.200130 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:22.203301 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:22.261617 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1294_1294_0 (state Temporary) 2024.10.22 02:03:22.309123 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1294_1294_0 to 202410_1307_1307_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:22.350963 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1302_19 to 202410_1307_1307_0 2024.10.22 02:03:22.360133 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:22.362643 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:22.392916 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85582 2024.10.22 02:03:22.435430 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 15.20 MiB, will set to 1.58 GiB (RSS), difference: -5.22 MiB 2024.10.22 02:03:22.497319 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1302_19 to 202410_1307_1307_0 into Compact with storage Full 2024.10.22 02:03:22.578813 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:22.722808 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1302_19, total 6177 rows starting from the beginning of the part 2024.10.22 02:03:22.901729 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1303_1303_0, total 69 rows starting from the beginning of the part 2024.10.22 02:03:22.942602 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ynk/rbaiwwzdiwimaharlfjgwivtlylhm (LogSeriesLimiter: on interval from 2024-10-22 02:03:11 to 2024-10-22 02:03:22 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:03:23.067131 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1304_1304_0, total 48 rows starting from the beginning of the part 2024.10.22 02:03:23.171451 [ 1269 ] {} DatabaseCatalog: Execution took 215 ms. 2024.10.22 02:03:23.251960 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1305_1305_0, total 51 rows starting from the beginning of the part 2024.10.22 02:03:23.547385 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1890 entries to flush up to offset 1113125 2024.10.22 02:03:23.551926 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1306_1306_0, total 64 rows starting from the beginning of the part 2024.10.22 02:03:23.618143 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 18.66 MiB, will set to 1.58 GiB (RSS), difference: -32.45 MiB 2024.10.22 02:03:23.725356 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_1307_1307_0, total 32 rows starting from the beginning of the part 2024.10.22 02:03:23.713165 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6191 2024.10.22 02:03:24.396775 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1096_1201_21, 202410_1202_1202_0, 202410_1203_1203_0, 202410_1204_1204_0, 202410_1205_1205_0, 202410_1206_1206_0] 2024.10.22 02:03:24.399292 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1096_1201_21, 202410_1202_1202_0, 202410_1203_1203_0, 202410_1204_1204_0, 202410_1205_1205_0, 202410_1206_1206_0] 2024.10.22 02:03:24.444282 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1096_1201_21, 202410_1202_1202_0, 202410_1203_1203_0, 202410_1204_1204_0, 202410_1205_1205_0, 202410_1206_1206_0] 2024.10.22 02:03:24.447305 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:24.450496 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:24.460465 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 85272 2024.10.22 02:03:24.617740 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:24.628422 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:24.665654 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_754_754_0 (state Temporary) 2024.10.22 02:03:24.741803 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_735_735_0 (state Temporary) 2024.10.22 02:03:24.817084 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_754_754_0 to 202410_760_760_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:24.869296 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85272 2024.10.22 02:03:24.876622 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_735_735_0 to 202410_742_742_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:24.920439 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1113125 2024.10.22 02:03:24.909676 [ 1216 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_737_737_0 to 202410_742_742_0 2024.10.22 02:03:24.952056 [ 1216 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:25.024182 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:25.134868 [ 1216 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 02:03:25.198023 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_737_737_0 to 202410_742_742_0 into Compact with storage Full 2024.10.22 02:03:25.206804 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} MergeTask::MergeProjectionsStage: Merge sorted 6441 rows, containing 13 columns (13 merged, 0 gathered) in 2.828560509 sec., 2277.1300028780825 rows/sec., 397.78 KiB/sec. 2024.10.22 02:03:25.234412 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:25.249602 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_737_737_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:03:25.262636 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_738_738_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:03:25.275092 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_739_739_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:03:25.291846 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_740_740_0, total 2310 rows starting from the beginning of the part 2024.10.22 02:03:25.295236 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 2.00 GiB, free memory in arenas 20.02 MiB, will set to 1.58 GiB (RSS), difference: -184.70 MiB 2024.10.22 02:03:25.343381 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_741_741_0, total 1190 rows starting from the beginning of the part 2024.10.22 02:03:25.357087 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_742_742_0, total 1890 rows starting from the beginning of the part 2024.10.22 02:03:25.387317 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} MergedBlockOutputStream: filled checksums 202410_1207_1307_20 (state Temporary) 2024.10.22 02:03:25.727358 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1307_20 to 202410_1207_1307_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:25.848995 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1307_20} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1302_19, 202410_1307_1307_0] -> 202410_1207_1307_20 2024.10.22 02:03:25.876297 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.87 MiB. 2024.10.22 02:03:26.106515 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} MergeTask::MergeProjectionsStage: Merge sorted 9625 rows, containing 4 columns (4 merged, 0 gathered) in 0.94732961 sec., 10160.13845487211 rows/sec., 158.75 KiB/sec. 2024.10.22 02:03:26.154212 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(97f57942050b008931772a7a4156afff) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 97f57942050b008931772a7a4156afff, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:03:26.156417 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 24042, file segment info: File segment: [0, 33554431], key: 97f57942050b008931772a7a4156afff, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:03:26.158582 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:26.273510 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} MergedBlockOutputStream: filled checksums 202410_737_742_1 (state Temporary) 2024.10.22 02:03:26.340011 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_737_742_1 to 202410_737_742_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:26.344478 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_737_742_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_737_737_0, 202410_742_742_0] -> 202410_737_742_1 2024.10.22 02:03:26.403345 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 85644 2024.10.22 02:03:26.407678 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:03:26.771149 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:26.826869 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(13c8fb4abad869946379df6ce3dc7eb0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 13c8fb4abad869946379df6ce3dc7eb0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:03:26.851297 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 509, file segment info: File segment: [0, 33554431], key: 13c8fb4abad869946379df6ce3dc7eb0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:03:26.854388 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:26.874881 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:26.979923 [ 1698 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xxt/efdtfzkvaeuacxjzdrkqvwdikdqwu, size 24042 2024.10.22 02:03:27.024999 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gch/zemoffkbxrfxtdfiibwdtpvjzbdne (LogSeriesLimiter: on interval from 2024-10-22 02:03:22 to 2024-10-22 02:03:27 accepted series 1 / 234 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:03:27.115925 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1295_1295_0 (state Temporary) 2024.10.22 02:03:27.266075 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1295_1295_0 to 202410_1308_1308_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:27.325450 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 2.00 GiB, free memory in arenas 19.87 MiB, will set to 1.58 GiB (RSS), difference: -178.97 MiB 2024.10.22 02:03:27.327375 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85644 2024.10.22 02:03:27.465336 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:03:27.472666 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:03:27.480541 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:03:27.496424 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:03:27.519096 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:03:27.554788 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:03:27.583294 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:03:27.680977 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_697_47, 202410_698_698_0] 2024.10.22 02:03:27.684284 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_697_47, 202410_698_698_0] 2024.10.22 02:03:27.701326 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_697_47, 202410_698_698_0] 2024.10.22 02:03:27.983078 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 11716 2024.10.22 02:03:28.196655 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:28.207642 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:28.272697 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_768_768_0 (state Temporary) 2024.10.22 02:03:28.416609 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 639 2024.10.22 02:03:28.445394 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_768_768_0 to 202410_781_781_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:28.483612 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11716 2024.10.22 02:03:28.674439 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:28.688967 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:28.853721 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_621_621_0 (state Temporary) 2024.10.22 02:03:28.957289 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_621_621_0 to 202410_631_631_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:29.200290 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 639 2024.10.22 02:03:29.639088 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 2.00 GiB, free memory in arenas 20.76 MiB, will set to 1.58 GiB (RSS), difference: -152.86 MiB 2024.10.22 02:03:30.455266 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:03:31.177509 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:03:31.179632 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:03:31.258045 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 6230 2024.10.22 02:03:31.416095 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 18.91 MiB, will set to 1.59 GiB (RSS), difference: -7.00 MiB 2024.10.22 02:03:31.572330 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 40 entries to flush up to offset 85684 2024.10.22 02:03:31.833020 [ 1222 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 250 ms. 2024.10.22 02:03:31.924116 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_154_26/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:03:31.933162 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1114665 2024.10.22 02:03:31.995378 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:32.001348 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:32.059558 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1296_1296_0 (state Temporary) 2024.10.22 02:03:32.144533 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1296_1296_0 to 202410_1309_1309_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:32.270771 [ 1159 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:03:32.309697 [ 1159 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:03:32.332860 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85684 2024.10.22 02:03:32.383980 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 74 entries to flush up to offset 85346 2024.10.22 02:03:32.422884 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(7d6f4b6cd6134cf4b68221d511ac00d9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7d6f4b6cd6134cf4b68221d511ac00d9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:03:32.425272 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 967, file segment info: File segment: [0, 33554431], key: 7d6f4b6cd6134cf4b68221d511ac00d9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:03:32.428606 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:32.716937 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:32.725444 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:03:32.756129 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:32.832484 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column URLHash 2024.10.22 02:03:32.831536 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:32.850396 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column URLHash 2024.10.22 02:03:32.859490 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column URLHash 2024.10.22 02:03:32.974901 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column URLHash 2024.10.22 02:03:32.999698 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_755_755_0 (state Temporary) 2024.10.22 02:03:33.063238 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_755_755_0 to 202410_761_761_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:33.097830 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column URLHash 2024.10.22 02:03:33.133952 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column URLHash 2024.10.22 02:03:33.150986 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column URLHash 2024.10.22 02:03:33.155283 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_736_736_0 (state Temporary) 2024.10.22 02:03:33.164357 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column URLHash 2024.10.22 02:03:33.246324 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85346 2024.10.22 02:03:33.262750 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ksg/brnctonrigceoeohhiiyzrrhlbmea (LogSeriesLimiter: on interval from 2024-10-22 02:03:27 to 2024-10-22 02:03:33 accepted series 1 / 123 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:03:33.302553 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_736_736_0 to 202410_743_743_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:33.457214 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1114665 2024.10.22 02:03:33.560284 [ 1237 ] {} DatabaseCatalog: Execution took 390 ms. 2024.10.22 02:03:33.578694 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_736_140/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:03:33.602695 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_578_118, 202410_579_579_0, 202410_580_580_0, 202410_581_581_0, 202410_582_582_0, 202410_583_583_0] 2024.10.22 02:03:33.608092 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_578_118, 202410_579_579_0, 202410_580_580_0, 202410_581_581_0, 202410_582_582_0, 202410_583_583_0] 2024.10.22 02:03:33.734211 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_578_118, 202410_579_579_0, 202410_580_580_0, 202410_581_581_0, 202410_582_582_0, 202410_583_583_0] 2024.10.22 02:03:34.478856 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 7.52 MiB, will set to 1.59 GiB (RSS), difference: -13.16 MiB 2024.10.22 02:03:35.988821 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 11721 2024.10.22 02:03:36.106645 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:36.109720 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:36.164243 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_769_769_0 (state Temporary) 2024.10.22 02:03:36.210591 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_769_769_0 to 202410_782_782_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:36.260696 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11721 2024.10.22 02:03:36.340188 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 85731 2024.10.22 02:03:36.556610 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 9.32 MiB, will set to 1.60 GiB (RSS), difference: 3.02 MiB 2024.10.22 02:03:36.595379 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:36.594391 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} MergeTask::MergeProjectionsStage: Merge sorted 6140 rows, containing 679 columns (679 merged, 0 gathered) in 128.815328648 sec., 47.66513476651624 rows/sec., 66.13 KiB/sec. 2024.10.22 02:03:36.610377 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:36.659589 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1297_1297_0 (state Temporary) 2024.10.22 02:03:36.702500 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 640 2024.10.22 02:03:36.846417 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1297_1297_0 to 202410_1310_1310_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:36.900611 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85731 2024.10.22 02:03:37.154914 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:37.159176 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:37.242506 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_622_622_0 (state Temporary) 2024.10.22 02:03:37.401713 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(59d7f6e13394049dc663985c4824fa28) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 59d7f6e13394049dc663985c4824fa28, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:03:37.407688 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 59d7f6e13394049dc663985c4824fa28, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:03:37.410727 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:37.422180 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_622_622_0 to 202410_632_632_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:37.441036 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_698_48, 202410_699_699_0] 2024.10.22 02:03:37.446894 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_698_48, 202410_699_699_0] 2024.10.22 02:03:37.462800 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_698_48, 202410_699_699_0] 2024.10.22 02:03:37.671299 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 640 2024.10.22 02:03:38.466263 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} MergedBlockOutputStream: filled checksums 202410_1_154_26 (state Temporary) 2024.10.22 02:03:39.370372 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 15.34 MiB, will set to 1.62 GiB (RSS), difference: 8.27 MiB 2024.10.22 02:03:39.739803 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 4 old parts to remove. Parts: [202410_1_718_147, 202410_719_719_0, 202410_720_720_0, 202410_721_721_0] 2024.10.22 02:03:39.742905 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from filesystem (serially): Parts: [202410_1_718_147, 202410_719_719_0, 202410_720_720_0, 202410_721_721_0] 2024.10.22 02:03:39.857262 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from memory: Parts: [202410_1_718_147, 202410_719_719_0, 202410_720_720_0, 202410_721_721_0] 2024.10.22 02:03:40.347576 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_736_140} MergeTask::MergeProjectionsStage: Merge sorted 1117090 rows, containing 4 columns (4 merged, 0 gathered) in 32.588626483 sec., 34278.52353896335 rows/sec., 514.24 KiB/sec. 2024.10.22 02:03:40.484998 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_736_140} MergedBlockOutputStream: filled checksums 202410_1_736_140 (state Temporary) 2024.10.22 02:03:40.496541 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1116205 2024.10.22 02:03:40.767102 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 63 entries to flush up to offset 85409 2024.10.22 02:03:40.927388 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 29 entries to flush up to offset 85760 2024.10.22 02:03:41.012075 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:41.016704 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:41.074666 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_756_756_0 (state Temporary) 2024.10.22 02:03:41.119722 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_756_756_0 to 202410_762_762_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:41.159673 [ 1175 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_757_2 to 202410_762_762_0 2024.10.22 02:03:41.167809 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:03:41.200521 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85409 2024.10.22 02:03:41.275188 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:03:41.314196 [ 1175 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:41.327824 [ 1175 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:41.351099 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:41.353102 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:41.401066 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:41.409002 [ 1175 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 263 ms. 2024.10.22 02:03:41.412973 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:41.426818 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 02:03:41.469085 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 12.62 MiB, will set to 1.62 GiB (RSS), difference: 6.94 MiB 2024.10.22 02:03:41.502946 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_737_737_0 (state Temporary) 2024.10.22 02:03:41.538895 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_757_2 to 202410_762_762_0 into Compact with storage Full 2024.10.22 02:03:41.629229 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:41.630861 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_737_737_0 to 202410_744_744_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:41.654282 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_757_2, total 1624 rows starting from the beginning of the part 2024.10.22 02:03:41.660581 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1298_1298_0 (state Temporary) 2024.10.22 02:03:41.701074 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1116205 2024.10.22 02:03:41.706418 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_758_758_0, total 94 rows starting from the beginning of the part 2024.10.22 02:03:41.711770 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1298_1298_0 to 202410_1311_1311_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:41.965487 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_759_759_0, total 128 rows starting from the beginning of the part 2024.10.22 02:03:42.150786 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_760_760_0, total 71 rows starting from the beginning of the part 2024.10.22 02:03:42.201256 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_761_761_0, total 74 rows starting from the beginning of the part 2024.10.22 02:03:42.245541 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85760 2024.10.22 02:03:42.260071 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_762_762_0, total 63 rows starting from the beginning of the part 2024.10.22 02:03:42.674864 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(385fe2d7e244c09d371682480de91447) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 385fe2d7e244c09d371682480de91447, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:03:42.692712 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 246447, file segment info: File segment: [0, 33554431], key: 385fe2d7e244c09d371682480de91447, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:03:42.696465 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:42.945671 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_154_26/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:03:43.440564 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} MergeTask::MergeProjectionsStage: Merge sorted 2054 rows, containing 13 columns (13 merged, 0 gathered) in 1.910898669 sec., 1074.8869279787018 rows/sec., 482.13 KiB/sec. 2024.10.22 02:03:43.611298 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} MergedBlockOutputStream: filled checksums 202410_747_762_3 (state Temporary) 2024.10.22 02:03:43.760279 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_736_140} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_736_140 to 202410_1_736_140 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:43.762654 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 11727 2024.10.22 02:03:43.765687 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_736_140} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_730_139, 202410_731_736_1] -> 202410_1_736_140 2024.10.22 02:03:43.800041 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:03:43.829499 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(f057fce657467b7b67b8267126044a20) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f057fce657467b7b67b8267126044a20, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:03:43.844344 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 509, file segment info: File segment: [0, 33554431], key: f057fce657467b7b67b8267126044a20, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:03:43.848194 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:43.858260 [ 1210 ] {} DatabaseCatalog: Execution took 297 ms. 2024.10.22 02:03:43.935209 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:43.939750 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:43.971932 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_770_770_0 (state Temporary) 2024.10.22 02:03:44.027997 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_762_3 to 202410_747_762_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:44.034762 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_762_3} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_757_2, 202410_762_762_0] -> 202410_747_762_3 2024.10.22 02:03:44.067680 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.29 MiB. 2024.10.22 02:03:44.260529 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_770_770_0 to 202410_783_783_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:44.294530 [ 1229 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_778_171 to 202410_779_779_0 2024.10.22 02:03:44.319204 [ 1229 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:44.323994 [ 1229 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:44.343476 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11727 2024.10.22 02:03:44.467370 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:03:44.470175 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_779_172} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_778_171 to 202410_779_779_0 into Compact with storage Full 2024.10.22 02:03:44.471245 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:03:44.484346 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:03:44.518258 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:03:44.553653 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:03:44.592659 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:03:44.645345 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:03:44.751462 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_779_172} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:44.776937 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_779_172} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_778_171, total 11778 rows starting from the beginning of the part 2024.10.22 02:03:44.812236 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uuv/lmsawyjijnbwqurzmjmrxupdhhxok (LogSeriesLimiter: on interval from 2024-10-22 02:03:33 to 2024-10-22 02:03:44 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:03:44.844027 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_779_172} MergeTreeSequentialSource: Reading 2 marks from part 202410_779_779_0, total 12 rows starting from the beginning of the part 2024.10.22 02:03:45.173160 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 641 2024.10.22 02:03:45.355071 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_699_49, 202410_700_700_0] 2024.10.22 02:03:45.359793 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_699_49, 202410_700_700_0] 2024.10.22 02:03:45.371887 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_699_49, 202410_700_700_0] 2024.10.22 02:03:45.475672 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:45.581355 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:45.747397 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 7.76 MiB, will set to 1.62 GiB (RSS), difference: 2.60 MiB 2024.10.22 02:03:45.787397 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_623_623_0 (state Temporary) 2024.10.22 02:03:45.793792 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_779_172} MergeTask::MergeProjectionsStage: Merge sorted 11790 rows, containing 12 columns (12 merged, 0 gathered) in 1.325915807 sec., 8891.967301208893 rows/sec., 1.39 MiB/sec. 2024.10.22 02:03:45.876571 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_623_623_0 to 202410_633_633_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:45.953539 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_779_172} MergedBlockOutputStream: filled checksums 202410_1_779_172 (state Temporary) 2024.10.22 02:03:46.067320 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 641 2024.10.22 02:03:46.013781 [ 1256 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_628_128 to 202410_633_633_0 2024.10.22 02:03:46.155291 [ 1256 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:46.167150 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:46.192674 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_628_128 to 202410_633_633_0 into Compact with storage Full 2024.10.22 02:03:46.249653 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 85845 2024.10.22 02:03:46.431516 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:46.440100 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:46.472175 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_779_172} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_779_172 to 202410_1_779_172 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:46.474609 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_779_172} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_778_171, 202410_779_779_0] -> 202410_1_779_172 2024.10.22 02:03:46.487268 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:46.504019 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:03:46.509704 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_628_128, total 672 rows starting from the beginning of the part 2024.10.22 02:03:46.607713 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1299_1299_0 (state Temporary) 2024.10.22 02:03:46.781786 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1299_1299_0 to 202410_1312_1312_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:46.822676 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_629_629_0, total 1 rows starting from the beginning of the part 2024.10.22 02:03:46.843541 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85845 2024.10.22 02:03:46.909459 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1307_20 to 202410_1312_1312_0 2024.10.22 02:03:46.912825 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_630_630_0, total 1 rows starting from the beginning of the part 2024.10.22 02:03:46.925644 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:46.929621 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:46.988300 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_631_631_0, total 1 rows starting from the beginning of the part 2024.10.22 02:03:47.016884 [ 1248 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 02:03:47.060391 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1307_20 to 202410_1312_1312_0 into Compact with storage Full 2024.10.22 02:03:47.108866 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:47.126615 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_632_632_0, total 1 rows starting from the beginning of the part 2024.10.22 02:03:47.204589 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_633_633_0, total 1 rows starting from the beginning of the part 2024.10.22 02:03:47.230833 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1307_20, total 6441 rows starting from the beginning of the part 2024.10.22 02:03:47.327943 [ 1276 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:03:47.354277 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1308_1308_0, total 62 rows starting from the beginning of the part 2024.10.22 02:03:47.393268 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1309_1309_0, total 40 rows starting from the beginning of the part 2024.10.22 02:03:47.403227 [ 1276 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:03:47.436254 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1310_1310_0, total 47 rows starting from the beginning of the part 2024.10.22 02:03:47.472243 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1311_1311_0, total 29 rows starting from the beginning of the part 2024.10.22 02:03:47.521780 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_1312_1312_0, total 85 rows starting from the beginning of the part 2024.10.22 02:03:47.562716 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 6.14 MiB, will set to 1.63 GiB (RSS), difference: 2.33 MiB 2024.10.22 02:03:47.746691 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:47.751082 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:48.720526 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 97 entries to flush up to offset 85506 2024.10.22 02:03:48.809293 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1117745 2024.10.22 02:03:48.846846 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} MergeTask::MergeProjectionsStage: Merge sorted 6704 rows, containing 13 columns (13 merged, 0 gathered) in 1.794006455 sec., 3736.8873346668083 rows/sec., 652.48 KiB/sec. 2024.10.22 02:03:48.872948 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:48.888161 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:48.948480 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_757_757_0 (state Temporary) 2024.10.22 02:03:48.997391 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_757_757_0 to 202410_763_763_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:49.082399 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85506 2024.10.22 02:03:49.156246 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} MergedBlockOutputStream: filled checksums 202410_1207_1312_21 (state Temporary) 2024.10.22 02:03:49.263953 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} MergeTask::MergeProjectionsStage: Merge sorted 677 rows, containing 32 columns (32 merged, 0 gathered) in 3.08343279 sec., 219.56048537707872 rows/sec., 19.93 KiB/sec. 2024.10.22 02:03:49.293705 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} MergedBlockOutputStream: filled checksums 202410_1_633_129 (state Temporary) 2024.10.22 02:03:49.574989 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1312_21 to 202410_1207_1312_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:49.581742 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1312_21} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1307_20, 202410_1312_1312_0] -> 202410_1207_1312_21 2024.10.22 02:03:49.629335 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.90 MiB. 2024.10.22 02:03:49.637358 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:49.646155 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:49.722538 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 5.55 MiB, will set to 1.63 GiB (RSS), difference: -9.64 MiB 2024.10.22 02:03:50.104491 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_738_738_0 (state Temporary) 2024.10.22 02:03:50.190816 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_738_738_0 to 202410_745_745_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:50.292854 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1117745 2024.10.22 02:03:50.459010 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_633_129 to 202410_1_633_129 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:50.462672 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_633_129} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_628_128, 202410_633_633_0] -> 202410_1_633_129 2024.10.22 02:03:50.566617 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_683_683_0, 202410_684_684_0, 202410_685_685_0, 202410_686_686_0, 202410_687_687_0, 202410_688_688_0] 2024.10.22 02:03:50.568460 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_683_683_0, 202410_684_684_0, 202410_685_685_0, 202410_686_686_0, 202410_687_687_0, 202410_688_688_0] 2024.10.22 02:03:50.668468 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:03:50.688610 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_155_155_0 (state Temporary) 2024.10.22 02:03:50.713234 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_683_683_0, 202410_684_684_0, 202410_685_685_0, 202410_686_686_0, 202410_687_687_0, 202410_688_688_0] 2024.10.22 02:03:50.858740 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 85905 2024.10.22 02:03:51.151992 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:51.155752 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:51.167931 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:03:51.170098 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:03:51.299433 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1300_1300_0 (state Temporary) 2024.10.22 02:03:51.461233 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:03:51.649522 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1300_1300_0 to 202410_1313_1313_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:51.711227 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85905 2024.10.22 02:03:51.846055 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 33 entries to flush up to offset 11760 2024.10.22 02:03:52.055751 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:52.058969 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:52.214335 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_771_771_0 (state Temporary) 2024.10.22 02:03:52.312984 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_771_771_0 to 202410_784_784_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:52.352370 [ 1177 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_779_172 to 202410_784_784_0 2024.10.22 02:03:52.360311 [ 1177 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:52.363880 [ 1177 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:52.381047 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_779_172 to 202410_784_784_0 into Compact with storage Full 2024.10.22 02:03:52.425047 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11760 2024.10.22 02:03:52.557670 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:52.666833 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_779_172, total 11790 rows starting from the beginning of the part 2024.10.22 02:03:52.687764 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} MergeTreeSequentialSource: Reading 2 marks from part 202410_780_780_0, total 12 rows starting from the beginning of the part 2024.10.22 02:03:52.718295 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} MergeTreeSequentialSource: Reading 2 marks from part 202410_781_781_0, total 20 rows starting from the beginning of the part 2024.10.22 02:03:52.754539 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} MergeTreeSequentialSource: Reading 2 marks from part 202410_782_782_0, total 5 rows starting from the beginning of the part 2024.10.22 02:03:52.778687 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(b26836c3e2d35a4b51ad218eb13d6fe2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b26836c3e2d35a4b51ad218eb13d6fe2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 02:03:52.780144 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} MergeTreeSequentialSource: Reading 2 marks from part 202410_783_783_0, total 6 rows starting from the beginning of the part 2024.10.22 02:03:52.781920 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 990, file segment info: File segment: [0, 33554431], key: b26836c3e2d35a4b51ad218eb13d6fe2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 02:03:52.784686 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:03:52.835873 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} MergeTreeSequentialSource: Reading 2 marks from part 202410_784_784_0, total 33 rows starting from the beginning of the part 2024.10.22 02:03:52.977350 [ 1702 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ksg/brnctonrigceoeohhiiyzrrhlbmea, size 11261387 2024.10.22 02:03:53.278910 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column CLID 2024.10.22 02:03:53.321829 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column CLID 2024.10.22 02:03:53.351271 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column CLID 2024.10.22 02:03:53.361493 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column CLID 2024.10.22 02:03:53.372521 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column CLID 2024.10.22 02:03:53.402639 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column CLID 2024.10.22 02:03:53.446259 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column CLID 2024.10.22 02:03:53.471626 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column CLID 2024.10.22 02:03:53.550668 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 4.82 MiB, will set to 1.63 GiB (RSS), difference: 10.72 MiB 2024.10.22 02:03:53.567803 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ouv/zqzpxrmixbybpbkwyavgwlhisnogf (LogSeriesLimiter: on interval from 2024-10-22 02:03:44 to 2024-10-22 02:03:53 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:03:53.570827 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 642 2024.10.22 02:03:53.671754 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 8 old parts to remove. Parts: [202410_1_1095_505, 202410_1096_1206_22, 202410_1207_1207_0, 202410_1208_1208_0, 202410_1209_1209_0, 202410_1210_1210_0, 202410_1211_1211_0, 202410_1212_1212_0] 2024.10.22 02:03:53.830467 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 8 parts from filesystem (serially): Parts: [202410_1_1095_505, 202410_1096_1206_22, 202410_1207_1207_0, 202410_1208_1208_0, 202410_1209_1209_0, 202410_1210_1210_0, 202410_1211_1211_0, 202410_1212_1212_0] 2024.10.22 02:03:54.054815 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 8 parts from memory: Parts: [202410_1_1095_505, 202410_1096_1206_22, 202410_1207_1207_0, 202410_1208_1208_0, 202410_1209_1209_0, 202410_1210_1210_0, 202410_1211_1211_0, 202410_1212_1212_0] 2024.10.22 02:03:54.100560 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:54.116881 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:54.155536 [ 1161 ] {} DatabaseCatalog: Execution took 293 ms. 2024.10.22 02:03:54.463737 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_624_624_0 (state Temporary) 2024.10.22 02:03:54.577071 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} MergeTask::MergeProjectionsStage: Merge sorted 11866 rows, containing 12 columns (12 merged, 0 gathered) in 2.205992902 sec., 5378.983762478126 rows/sec., 858.03 KiB/sec. 2024.10.22 02:03:54.643212 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} MergedBlockOutputStream: filled checksums 202410_1_784_173 (state Temporary) 2024.10.22 02:03:54.756738 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_624_624_0 to 202410_634_634_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:54.999379 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_784_173 to 202410_1_784_173 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:55.003939 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_779_172, 202410_784_784_0] -> 202410_1_784_173 2024.10.22 02:03:55.012251 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 642 2024.10.22 02:03:55.059736 [ 1397 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {5276570e-a364-404e-a403-3e186cba01a8::202410_1_784_173}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 02:03:55.066484 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_154_26 to 202410_1_154_26 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:55.080302 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.54 MiB. 2024.10.22 02:03:55.171439 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_154_26} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 7 parts: [202410_1_144_25, 202410_154_154_0] -> 202410_1_154_26 2024.10.22 02:03:55.414149 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_507_700_50, 202410_701_701_0, 202410_702_702_0, 202410_703_703_0, 202410_704_704_0, 202410_705_705_0] 2024.10.22 02:03:55.417535 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_507_700_50, 202410_701_701_0, 202410_702_702_0, 202410_703_703_0, 202410_704_704_0, 202410_705_705_0] 2024.10.22 02:03:55.458535 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_507_700_50, 202410_701_701_0, 202410_702_702_0, 202410_703_703_0, 202410_704_704_0, 202410_705_705_0] 2024.10.22 02:03:55.530526 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 4.81 MiB, will set to 1.63 GiB (RSS), difference: 3.71 MiB 2024.10.22 02:03:55.724799 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 85967 2024.10.22 02:03:56.055164 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:56.057723 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:56.230442 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 54.55 MiB. 2024.10.22 02:03:56.245755 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1301_1301_0 (state Temporary) 2024.10.22 02:03:56.288436 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1301_1301_0 to 202410_1314_1314_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:56.452717 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85967 2024.10.22 02:03:56.602456 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 72 entries to flush up to offset 85578 2024.10.22 02:03:56.869199 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:56.871349 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:56.931643 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_758_758_0 (state Temporary) 2024.10.22 02:03:57.087170 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_758_758_0 to 202410_764_764_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:57.244181 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85578 2024.10.22 02:03:57.300544 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1119285 2024.10.22 02:03:57.505322 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 4.49 MiB, will set to 1.63 GiB (RSS), difference: 2.86 MiB 2024.10.22 02:03:58.402828 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:58.411626 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:58.749234 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_739_739_0 (state Temporary) 2024.10.22 02:03:58.809125 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_739_739_0 to 202410_746_746_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:58.863063 [ 1189 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_736_140 to 202410_737_742_1 2024.10.22 02:03:58.871576 [ 1189 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:03:58.876233 [ 1189 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:03:58.963293 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1119285 2024.10.22 02:03:59.010892 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_742_141} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_736_140 to 202410_737_742_1 into Wide with storage Full 2024.10.22 02:03:59.057007 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_742_141} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:03:59.064090 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_742_141} MergeTreeSequentialSource: Reading 138 marks from part 202410_1_736_140, total 1117090 rows starting from the beginning of the part 2024.10.22 02:03:59.322290 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_742_141} MergeTreeSequentialSource: Reading 3 marks from part 202410_737_742_1, total 9625 rows starting from the beginning of the part 2024.10.22 02:03:59.416673 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_155_155_0 to 202410_158_158_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:03:59.626467 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 5.42 MiB, will set to 1.64 GiB (RSS), difference: -6.07 MiB 2024.10.22 02:03:59.930631 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 11781 2024.10.22 02:04:00.159087 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:00.164725 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:04:00.472728 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 85999 2024.10.22 02:04:00.528072 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_772_772_0 (state Temporary) 2024.10.22 02:04:00.789479 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_772_772_0 to 202410_785_785_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:00.939996 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:00.945080 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:04:00.945285 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11781 2024.10.22 02:04:01.022343 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1302_1302_0 (state Temporary) 2024.10.22 02:04:01.093411 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1302_1302_0 to 202410_1315_1315_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:01.171003 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:04:01.176246 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:04:01.287765 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 85999 2024.10.22 02:04:01.423981 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_676_130, 202410_677_682_1] 2024.10.22 02:04:01.427548 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_676_130, 202410_677_682_1] 2024.10.22 02:04:01.478600 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_676_130, 202410_677_682_1] 2024.10.22 02:04:01.505663 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 6.21 MiB, will set to 1.64 GiB (RSS), difference: 26.82 MiB 2024.10.22 02:04:01.883181 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d025fc5dd260ff5f18a67dc8da1d12f2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d025fc5dd260ff5f18a67dc8da1d12f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:04:01.889736 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 230914, file segment info: File segment: [0, 33554431], key: d025fc5dd260ff5f18a67dc8da1d12f2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:04:01.907770 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:02.420430 [ 1208 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:04:02.478538 [ 1208 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:04:02.514710 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 643 2024.10.22 02:04:02.934298 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(ef848a7656700ccda6787aa021e78fdf) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ef848a7656700ccda6787aa021e78fdf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:04:02.941153 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 513, file segment info: File segment: [0, 33554431], key: ef848a7656700ccda6787aa021e78fdf, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:04:02.943769 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:03.009449 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_742_141/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:04:03.064807 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:03.065562 [ 1692 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uuv/lmsawyjijnbwqurzmjmrxupdhhxok, size 230914 2024.10.22 02:04:03.068778 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:04:03.099141 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6230 2024.10.22 02:04:03.223272 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_625_625_0 (state Temporary) 2024.10.22 02:04:03.310306 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_625_625_0 to 202410_635_635_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:03.347371 [ 1736 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gch/zemoffkbxrfxtdfiibwdtpvjzbdne, size 238 2024.10.22 02:04:03.487896 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 11.37 MiB, will set to 1.63 GiB (RSS), difference: 110.32 MiB 2024.10.22 02:04:03.552568 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 643 2024.10.22 02:04:03.762676 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:04:03.771404 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:04:03.803523 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:04:04.042185 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:04:04.162326 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:04:04.232074 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:04:04.360613 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:04:04.519760 [ 1249 ] {} DatabaseCatalog: Execution took 355 ms. 2024.10.22 02:04:04.560835 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_721_148, 202410_722_722_0, 202410_723_723_0, 202410_724_724_0, 202410_725_725_0, 202410_726_726_0] 2024.10.22 02:04:04.580959 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_721_148, 202410_722_722_0, 202410_723_723_0, 202410_724_724_0, 202410_725_725_0, 202410_726_726_0] 2024.10.22 02:04:04.671291 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lcb/duwywxfzlwhcgroxtxlqqdvwmappl (LogSeriesLimiter: on interval from 2024-10-22 02:03:53 to 2024-10-22 02:04:04 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:04:04.728888 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_721_148, 202410_722_722_0, 202410_723_723_0, 202410_724_724_0, 202410_725_725_0, 202410_726_726_0] 2024.10.22 02:04:04.758794 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 85678 2024.10.22 02:04:05.129016 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:05.151716 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:04:05.308139 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 86048 2024.10.22 02:04:05.456016 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_759_759_0 (state Temporary) 2024.10.22 02:04:05.553167 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_759_759_0 to 202410_765_765_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:05.755083 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:05.757233 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:04:05.823684 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1303_1303_0 (state Temporary) 2024.10.22 02:04:05.852033 [ 1216 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 281 ms. 2024.10.22 02:04:05.857170 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 2.00 GiB, free memory in arenas 13.74 MiB, will set to 1.62 GiB (RSS), difference: 150.67 MiB 2024.10.22 02:04:05.912016 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1303_1303_0 to 202410_1316_1316_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:05.977706 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1120825 2024.10.22 02:04:06.002544 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86048 2024.10.22 02:04:06.435162 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85678 2024.10.22 02:04:07.283008 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_13_13_0 (state Temporary) 2024.10.22 02:04:07.404364 [ 1215 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 02:04:07.420008 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:07.423857 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:04:07.501640 [ 1746 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ynk/rbaiwwzdiwimaharlfjgwivtlylhm, size 683 2024.10.22 02:04:07.680120 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_740_740_0 (state Temporary) 2024.10.22 02:04:08.066892 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_740_740_0 to 202410_747_747_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:08.241372 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1120825 2024.10.22 02:04:08.448803 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 11787 2024.10.22 02:04:08.589112 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 14.38 MiB, will set to 1.57 GiB (RSS), difference: 62.26 MiB 2024.10.22 02:04:08.631764 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:08.638775 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:04:08.670798 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_773_773_0 (state Temporary) 2024.10.22 02:04:08.747133 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_773_773_0 to 202410_786_786_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:08.822331 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11787 2024.10.22 02:04:10.017834 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 86075 2024.10.22 02:04:10.162775 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:10.165264 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:04:10.227137 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1304_1304_0 (state Temporary) 2024.10.22 02:04:10.495003 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1304_1304_0 to 202410_1317_1317_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:10.518640 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 16.73 MiB, will set to 1.58 GiB (RSS), difference: 2.02 MiB 2024.10.22 02:04:10.570879 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86075 2024.10.22 02:04:10.647297 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1312_21 to 202410_1317_1317_0 2024.10.22 02:04:10.667139 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:10.669417 [ 1252 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:04:10.811809 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1312_21 to 202410_1317_1317_0 into Compact with storage Full 2024.10.22 02:04:10.850246 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:04:10.862616 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1312_21, total 6704 rows starting from the beginning of the part 2024.10.22 02:04:10.898938 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1313_1313_0, total 60 rows starting from the beginning of the part 2024.10.22 02:04:10.959961 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1314_1314_0, total 62 rows starting from the beginning of the part 2024.10.22 02:04:10.983240 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1315_1315_0, total 32 rows starting from the beginning of the part 2024.10.22 02:04:11.046297 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 6269 2024.10.22 02:04:11.057519 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1316_1316_0, total 49 rows starting from the beginning of the part 2024.10.22 02:04:11.170470 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:04:11.174234 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:04:11.210195 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_1317_1317_0, total 27 rows starting from the beginning of the part 2024.10.22 02:04:12.669469 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 15.11 MiB, will set to 1.58 GiB (RSS), difference: -5.86 MiB 2024.10.22 02:04:12.979510 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1317_22/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:04:12.970275 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} MergeTask::MergeProjectionsStage: Merge sorted 6934 rows, containing 13 columns (13 merged, 0 gathered) in 2.272259738 sec., 3051.587758230129 rows/sec., 536.26 KiB/sec. 2024.10.22 02:04:13.371494 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} MergedBlockOutputStream: filled checksums 202410_1207_1317_22 (state Temporary) 2024.10.22 02:04:13.951252 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 97 entries to flush up to offset 85775 2024.10.22 02:04:14.047711 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1317_22 to 202410_1207_1317_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:14.051793 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1317_22} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1312_21, 202410_1317_1317_0] -> 202410_1207_1317_22 2024.10.22 02:04:14.079956 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.96 MiB. 2024.10.22 02:04:14.302791 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:14.303190 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_742_141/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:04:14.306853 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:04:14.552981 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 15.23 MiB, will set to 1.58 GiB (RSS), difference: 6.28 MiB 2024.10.22 02:04:14.580473 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 86108 2024.10.22 02:04:14.601358 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_760_760_0 (state Temporary) 2024.10.22 02:04:14.668290 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_760_760_0 to 202410_766_766_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:14.853299 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:14.858598 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:04:14.870830 [ 1167 ] {} DatabaseCatalog: Execution took 352 ms. 2024.10.22 02:04:14.870710 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(982f51aa958ece5823fdc2196e0e7d52) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 982f51aa958ece5823fdc2196e0e7d52, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:04:14.912476 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 7186, file segment info: File segment: [0, 33554431], key: 982f51aa958ece5823fdc2196e0e7d52, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:04:14.923897 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1305_1305_0 (state Temporary) 2024.10.22 02:04:14.932619 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:14.943190 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85775 2024.10.22 02:04:14.999595 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1305_1305_0 to 202410_1318_1318_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:15.185529 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86108 2024.10.22 02:04:15.301496 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1680 entries to flush up to offset 1122505 2024.10.22 02:04:15.911769 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(f55d542d803c311f486819585b1e9b30) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f55d542d803c311f486819585b1e9b30, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:04:15.915466 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 165, file segment info: File segment: [0, 33554431], key: f55d542d803c311f486819585b1e9b30, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:04:16.026602 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:16.324779 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 11798 2024.10.22 02:04:16.372853 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 14.61 MiB, will set to 1.58 GiB (RSS), difference: 1.95 MiB 2024.10.22 02:04:16.433296 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:16.441509 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:16.450830 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:16.454360 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:16.456167 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1212_1, 202410_1213_1213_0, 202410_1214_1214_0, 202410_1215_1215_0, 202410_1216_1216_0, 202410_1217_1217_0] 2024.10.22 02:04:16.457363 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1212_1, 202410_1213_1213_0, 202410_1214_1214_0, 202410_1215_1215_0, 202410_1216_1216_0, 202410_1217_1217_0] 2024.10.22 02:04:16.476691 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1212_1, 202410_1213_1213_0, 202410_1214_1214_0, 202410_1215_1215_0, 202410_1216_1216_0, 202410_1217_1217_0] 2024.10.22 02:04:16.615640 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_741_741_0 (state Temporary) 2024.10.22 02:04:16.665576 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_741_741_0 to 202410_748_748_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:16.677646 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_774_774_0 (state Temporary) 2024.10.22 02:04:16.711263 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_743_743_0 to 202410_748_748_0 2024.10.22 02:04:16.725799 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:16.739516 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_774_774_0 to 202410_787_787_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:16.835645 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1122505 2024.10.22 02:04:16.854157 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:04:16.866701 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11798 2024.10.22 02:04:16.882548 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_743_743_0 to 202410_748_748_0 into Compact with storage Full 2024.10.22 02:04:16.958204 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:04:16.969231 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_743_743_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:04:17.079161 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(b1109b990b61c12ee188fe12776e61d5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b1109b990b61c12ee188fe12776e61d5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:04:17.085250 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 2804, file segment info: File segment: [0, 33554431], key: b1109b990b61c12ee188fe12776e61d5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:04:17.089625 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:17.159811 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_744_744_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:04:17.207889 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_745_745_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:04:17.248198 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_746_746_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:04:17.374760 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_747_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:04:17.428892 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_748_748_0, total 1680 rows starting from the beginning of the part 2024.10.22 02:04:17.496933 [ 1228 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:04:17.549821 [ 1228 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:04:17.831954 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(69487ef70545e960a39cc2d75d020c09) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 69487ef70545e960a39cc2d75d020c09, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:04:17.852571 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 451, file segment info: File segment: [0, 33554431], key: 69487ef70545e960a39cc2d75d020c09, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:04:17.854067 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:17.880572 [ 1739 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ouv/zqzpxrmixbybpbkwyavgwlhisnogf, size 2804 2024.10.22 02:04:18.237351 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column YCLID 2024.10.22 02:04:18.265650 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column YCLID 2024.10.22 02:04:18.284104 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column YCLID 2024.10.22 02:04:18.323646 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column YCLID 2024.10.22 02:04:18.450588 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column YCLID 2024.10.22 02:04:18.494258 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column YCLID 2024.10.22 02:04:18.528179 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column YCLID 2024.10.22 02:04:18.555997 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 644 2024.10.22 02:04:18.560957 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column YCLID 2024.10.22 02:04:18.752564 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 13.71 MiB, will set to 1.58 GiB (RSS), difference: -2.65 MiB 2024.10.22 02:04:18.750697 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} MergeTask::MergeProjectionsStage: Merge sorted 9380 rows, containing 4 columns (4 merged, 0 gathered) in 1.887590695 sec., 4969.297647443637 rows/sec., 77.65 KiB/sec. 2024.10.22 02:04:18.954367 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fko/zxdguxigfjxwjtrzyekycphwkybcr (LogSeriesLimiter: on interval from 2024-10-22 02:04:04 to 2024-10-22 02:04:18 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:04:19.082426 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} MergedBlockOutputStream: filled checksums 202410_743_748_1 (state Temporary) 2024.10.22 02:04:19.147462 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:19.152463 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:04:19.191577 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 86175 2024.10.22 02:04:19.217833 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_626_626_0 (state Temporary) 2024.10.22 02:04:19.366746 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_626_626_0 to 202410_636_636_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:19.367689 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_743_748_1 to 202410_743_748_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:19.371646 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_743_748_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_743_743_0, 202410_748_748_0] -> 202410_743_748_1 2024.10.22 02:04:19.464538 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:19.466359 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:19.545280 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:04:19.557754 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1306_1306_0 (state Temporary) 2024.10.22 02:04:19.640993 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1306_1306_0 to 202410_1319_1319_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:19.697522 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 644 2024.10.22 02:04:19.728237 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86175 2024.10.22 02:04:20.466622 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 13.18 MiB, will set to 1.58 GiB (RSS), difference: -4.62 MiB 2024.10.22 02:04:21.172031 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:04:21.174681 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:04:22.074842 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:04:22.569012 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 70 entries to flush up to offset 85845 2024.10.22 02:04:22.825490 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:22.830344 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:22.874498 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_761_761_0 (state Temporary) 2024.10.22 02:04:23.044180 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_761_761_0 to 202410_767_767_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:23.073102 [ 1235 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_762_3 to 202410_767_767_0 2024.10.22 02:04:23.094856 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85845 2024.10.22 02:04:23.205045 [ 1235 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:23.211092 [ 1235 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:23.254529 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_762_3 to 202410_767_767_0 into Compact with storage Full 2024.10.22 02:04:23.294385 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:04:23.311069 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_762_3, total 2054 rows starting from the beginning of the part 2024.10.22 02:04:23.342603 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_763_763_0, total 97 rows starting from the beginning of the part 2024.10.22 02:04:23.366629 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_764_764_0, total 72 rows starting from the beginning of the part 2024.10.22 02:04:23.391674 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_765_765_0, total 100 rows starting from the beginning of the part 2024.10.22 02:04:23.443868 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_766_766_0, total 97 rows starting from the beginning of the part 2024.10.22 02:04:23.461819 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_767_767_0, total 70 rows starting from the beginning of the part 2024.10.22 02:04:23.596963 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qpb/hnllljucvowdnddplkbujffkxkgvi. 2024.10.22 02:04:23.744613 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 86209 2024.10.22 02:04:23.871318 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1785 entries to flush up to offset 1124290 2024.10.22 02:04:24.205307 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:24.214386 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:24.273462 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1307_1307_0 (state Temporary) 2024.10.22 02:04:24.342348 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1307_1307_0 to 202410_1320_1320_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:24.369329 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 11811 2024.10.22 02:04:24.391598 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86209 2024.10.22 02:04:24.588791 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:24.595154 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:24.638900 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_775_775_0 (state Temporary) 2024.10.22 02:04:24.672038 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_775_775_0 to 202410_788_788_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:24.743343 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:24.747944 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11811 2024.10.22 02:04:24.748071 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:24.867946 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 11.98 MiB, will set to 1.58 GiB (RSS), difference: -9.75 MiB 2024.10.22 02:04:25.095844 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} MergeTask::MergeProjectionsStage: Merge sorted 2490 rows, containing 13 columns (13 merged, 0 gathered) in 1.867084824 sec., 1333.629821201953 rows/sec., 595.83 KiB/sec. 2024.10.22 02:04:25.108325 [ 1227 ] {} DatabaseCatalog: Execution took 237 ms. 2024.10.22 02:04:25.161839 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} MergedBlockOutputStream: filled checksums 202410_747_767_4 (state Temporary) 2024.10.22 02:04:25.195040 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_742_742_0 (state Temporary) 2024.10.22 02:04:25.247983 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_583_119, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0, 202410_587_587_0, 202410_588_588_0] 2024.10.22 02:04:25.249670 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_583_119, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0, 202410_587_587_0, 202410_588_588_0] 2024.10.22 02:04:25.280259 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_583_119, 202410_584_584_0, 202410_585_585_0, 202410_586_586_0, 202410_587_587_0, 202410_588_588_0] 2024.10.22 02:04:25.315463 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_742_742_0 to 202410_749_749_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:25.367012 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1124290 2024.10.22 02:04:25.381661 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_742_141/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:04:25.560100 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(46ee0179e971a9575a3b838770d419dc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 46ee0179e971a9575a3b838770d419dc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:04:25.564424 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 355500, file segment info: File segment: [0, 33554431], key: 46ee0179e971a9575a3b838770d419dc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:04:25.567501 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:25.666448 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_767_4 to 202410_747_767_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:25.888655 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_767_4} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_762_3, 202410_767_767_0] -> 202410_747_767_4 2024.10.22 02:04:26.041354 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.58 MiB. 2024.10.22 02:04:26.415314 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 13.68 MiB, will set to 1.58 GiB (RSS), difference: 10.15 MiB 2024.10.22 02:04:26.730682 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(fc1ff35d324129669f387fcb9aab2607) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fc1ff35d324129669f387fcb9aab2607, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:04:26.747696 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 988, file segment info: File segment: [0, 33554431], key: fc1ff35d324129669f387fcb9aab2607, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:04:26.750672 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:26.779105 [ 1731 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lcb/duwywxfzlwhcgroxtxlqqdvwmappl, size 355500 2024.10.22 02:04:27.250546 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Params 2024.10.22 02:04:27.303177 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 645 2024.10.22 02:04:27.367048 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Params 2024.10.22 02:04:27.483866 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Params 2024.10.22 02:04:27.539773 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Params 2024.10.22 02:04:27.575083 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Params 2024.10.22 02:04:27.593777 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Params 2024.10.22 02:04:27.677839 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Params 2024.10.22 02:04:27.721523 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:27.730454 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:27.767319 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hdy/oxdloczekdnzbdkgganqvtlwchdwu (LogSeriesLimiter: on interval from 2024-10-22 02:04:18 to 2024-10-22 02:04:27 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:04:27.815911 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_627_627_0 (state Temporary) 2024.10.22 02:04:28.004951 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_627_627_0 to 202410_637_637_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:28.311947 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 645 2024.10.22 02:04:28.370818 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 16.13 MiB, will set to 1.58 GiB (RSS), difference: 1.93 MiB 2024.10.22 02:04:28.408271 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 86261 2024.10.22 02:04:28.472029 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:28.475344 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:28.667846 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:28.671157 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:28.964315 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1308_1308_0 (state Temporary) 2024.10.22 02:04:29.103756 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1308_1308_0 to 202410_1321_1321_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:29.253241 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86261 2024.10.22 02:04:30.623443 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 51 entries to flush up to offset 85896 2024.10.22 02:04:30.729091 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 14.98 MiB, will set to 1.58 GiB (RSS), difference: -9.89 MiB 2024.10.22 02:04:30.859806 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:30.864728 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:31.050671 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_762_762_0 (state Temporary) 2024.10.22 02:04:31.109264 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_762_762_0 to 202410_768_768_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:31.172390 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:04:31.174571 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:04:31.192187 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85896 2024.10.22 02:04:31.423735 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_156_156_0 (state Temporary) 2024.10.22 02:04:32.250981 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 11823 2024.10.22 02:04:32.371461 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1125830 2024.10.22 02:04:32.479048 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:32.481453 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:32.520690 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_776_776_0 (state Temporary) 2024.10.22 02:04:32.553750 [ 1263 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:04:32.565415 [ 1263 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:04:32.604920 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_776_776_0 to 202410_789_789_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:32.762176 [ 1242 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_784_173 to 202410_785_785_0 2024.10.22 02:04:32.768404 [ 1242 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:32.772067 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:32.837212 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_785_174} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_784_173 to 202410_785_785_0 into Compact with storage Full 2024.10.22 02:04:32.897399 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11823 2024.10.22 02:04:33.124342 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_785_174} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:04:33.166666 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_785_174} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_784_173, total 11866 rows starting from the beginning of the part 2024.10.22 02:04:33.208290 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:33.213744 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:33.264460 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 86296 2024.10.22 02:04:33.392312 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_785_174} MergeTreeSequentialSource: Reading 2 marks from part 202410_785_785_0, total 21 rows starting from the beginning of the part 2024.10.22 02:04:33.397667 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:33.405802 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:33.434544 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_743_743_0 (state Temporary) 2024.10.22 02:04:33.596628 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1309_1309_0 (state Temporary) 2024.10.22 02:04:33.624979 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_743_743_0 to 202410_750_750_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:33.724566 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1125830 2024.10.22 02:04:33.744205 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1309_1309_0 to 202410_1322_1322_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:33.856622 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1317_22 to 202410_1322_1322_0 2024.10.22 02:04:33.864594 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:33.866811 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:33.869771 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86296 2024.10.22 02:04:33.916613 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1317_22 to 202410_1322_1322_0 into Compact with storage Full 2024.10.22 02:04:34.111612 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_742_141} MergeTask::MergeProjectionsStage: Merge sorted 1126715 rows, containing 4 columns (4 merged, 0 gathered) in 35.110605661 sec., 32090.446142645935 rows/sec., 481.32 KiB/sec. 2024.10.22 02:04:34.167397 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:04:34.179767 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1317_22, total 6934 rows starting from the beginning of the part 2024.10.22 02:04:34.329152 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_742_141} MergedBlockOutputStream: filled checksums 202410_1_742_141 (state Temporary) 2024.10.22 02:04:34.332441 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_1318_1318_0, total 33 rows starting from the beginning of the part 2024.10.22 02:04:34.532773 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 12.68 MiB, will set to 1.58 GiB (RSS), difference: -11.02 MiB 2024.10.22 02:04:34.597698 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_1319_1319_0, total 67 rows starting from the beginning of the part 2024.10.22 02:04:34.662315 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_1320_1320_0, total 34 rows starting from the beginning of the part 2024.10.22 02:04:34.828312 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_1321_1321_0, total 52 rows starting from the beginning of the part 2024.10.22 02:04:34.863183 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_785_174} MergeTask::MergeProjectionsStage: Merge sorted 11887 rows, containing 12 columns (12 merged, 0 gathered) in 2.058980779 sec., 5773.244763252838 rows/sec., 920.94 KiB/sec. 2024.10.22 02:04:34.885147 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_1322_1322_0, total 35 rows starting from the beginning of the part 2024.10.22 02:04:34.904766 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_785_174} MergedBlockOutputStream: filled checksums 202410_1_785_174 (state Temporary) 2024.10.22 02:04:35.439791 [ 1152 ] {} DatabaseCatalog: Execution took 326 ms. 2024.10.22 02:04:35.687505 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_785_174} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_785_174 to 202410_1_785_174 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:35.696935 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_785_174} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_784_173, 202410_785_785_0] -> 202410_1_785_174 2024.10.22 02:04:35.691291 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_705_51, 202410_706_706_0] 2024.10.22 02:04:35.721190 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_705_51, 202410_706_706_0] 2024.10.22 02:04:35.815709 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 646 2024.10.22 02:04:35.839060 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_705_51, 202410_706_706_0] 2024.10.22 02:04:35.853563 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:04:35.983626 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:35.991693 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:36.170675 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_628_628_0 (state Temporary) 2024.10.22 02:04:36.367810 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_628_628_0 to 202410_638_638_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:36.401141 [ 1203 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_633_129 to 202410_638_638_0 2024.10.22 02:04:36.439485 [ 1203 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:36.444915 [ 1203 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:36.445656 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 646 2024.10.22 02:04:36.452785 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.23 MiB, will set to 1.58 GiB (RSS), difference: 9.67 MiB 2024.10.22 02:04:36.477830 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_633_129 to 202410_638_638_0 into Compact with storage Full 2024.10.22 02:04:36.714681 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} MergeTask::MergeProjectionsStage: Merge sorted 7155 rows, containing 13 columns (13 merged, 0 gathered) in 2.80787421 sec., 2548.191074414263 rows/sec., 447.60 KiB/sec. 2024.10.22 02:04:36.823790 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} MergedBlockOutputStream: filled checksums 202410_1207_1322_23 (state Temporary) 2024.10.22 02:04:37.010300 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:04:37.074823 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_633_129, total 677 rows starting from the beginning of the part 2024.10.22 02:04:37.172854 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1322_23 to 202410_1207_1322_23 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:37.243688 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_742_141/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:04:37.281554 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1322_23} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1317_22, 202410_1322_1322_0] -> 202410_1207_1322_23 2024.10.22 02:04:37.330336 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.52 MiB. 2024.10.22 02:04:37.602367 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_634_634_0, total 1 rows starting from the beginning of the part 2024.10.22 02:04:37.615444 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_156_156_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:04:37.872597 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 86350 2024.10.22 02:04:37.914223 [ 1194 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 02:04:37.965633 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_635_635_0, total 1 rows starting from the beginning of the part 2024.10.22 02:04:38.145178 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:38.148340 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:38.199719 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1310_1310_0 (state Temporary) 2024.10.22 02:04:38.254114 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(d224758c8c0afca919b47ca3d8d0fb3e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d224758c8c0afca919b47ca3d8d0fb3e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 02:04:38.259096 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 15402, file segment info: File segment: [0, 33554431], key: d224758c8c0afca919b47ca3d8d0fb3e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 02:04:38.263522 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:38.265171 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1310_1310_0 to 202410_1323_1323_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:38.435838 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86350 2024.10.22 02:04:38.523922 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_636_636_0, total 1 rows starting from the beginning of the part 2024.10.22 02:04:38.545069 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_742_141} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_742_141 to 202410_1_742_141 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:38.551725 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_742_141} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_736_140, 202410_737_742_1] -> 202410_1_742_141 2024.10.22 02:04:38.591640 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:04:38.700095 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 83 entries to flush up to offset 85979 2024.10.22 02:04:38.825779 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:38.825598 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_637_637_0, total 1 rows starting from the beginning of the part 2024.10.22 02:04:38.831204 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:38.869403 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_763_763_0 (state Temporary) 2024.10.22 02:04:39.030556 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_763_763_0 to 202410_769_769_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:39.049513 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_638_638_0, total 1 rows starting from the beginning of the part 2024.10.22 02:04:39.167689 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 85979 2024.10.22 02:04:39.310554 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a21f419d1b14dddca517dda61f7bbd5a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a21f419d1b14dddca517dda61f7bbd5a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 02:04:39.312973 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 495, file segment info: File segment: [0, 33554431], key: a21f419d1b14dddca517dda61f7bbd5a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 02:04:39.317726 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:40.311342 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(6b7a886b8f02cc9fa206f703e8c6192c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6b7a886b8f02cc9fa206f703e8c6192c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 02:04:40.315033 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 5128, file segment info: File segment: [0, 33554431], key: 6b7a886b8f02cc9fa206f703e8c6192c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 02:04:40.318243 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:40.404752 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 11842 2024.10.22 02:04:40.553691 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:40.556257 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:40.734967 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1127370 2024.10.22 02:04:40.740864 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_777_777_0 (state Temporary) 2024.10.22 02:04:40.884327 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_777_777_0 to 202410_790_790_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:40.916031 [ 1178 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_785_174 to 202410_786_786_0 2024.10.22 02:04:40.944466 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11842 2024.10.22 02:04:41.030789 [ 1178 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:41.036428 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:41.059226 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(071320489c6a35bd7699238e23399b37) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 071320489c6a35bd7699238e23399b37, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 02:04:41.059970 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_786_175} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_785_174 to 202410_786_786_0 into Compact with storage Full 2024.10.22 02:04:41.061013 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 473, file segment info: File segment: [0, 33554431], key: 071320489c6a35bd7699238e23399b37, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1398, kind: Regular, unbound: 0 2024.10.22 02:04:41.063555 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:41.099952 [ 1733 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fko/zxdguxigfjxwjtrzyekycphwkybcr, size 5128 2024.10.22 02:04:41.130651 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_786_175} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:04:41.129338 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_156_156_0 to 202410_159_159_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:41.146095 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_786_175} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_785_174, total 11887 rows starting from the beginning of the part 2024.10.22 02:04:41.171895 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:04:41.173188 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:04:41.183601 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_786_175} MergeTreeSequentialSource: Reading 2 marks from part 202410_786_786_0, total 6 rows starting from the beginning of the part 2024.10.22 02:04:41.181544 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1217_2, 202410_1218_1218_0, 202410_1219_1219_0, 202410_1220_1220_0, 202410_1221_1221_0, 202410_1222_1222_0] 2024.10.22 02:04:41.193452 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1217_2, 202410_1218_1218_0, 202410_1219_1219_0, 202410_1220_1220_0, 202410_1221_1221_0, 202410_1222_1222_0] 2024.10.22 02:04:41.261170 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ShareService 2024.10.22 02:04:41.364615 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1217_2, 202410_1218_1218_0, 202410_1219_1219_0, 202410_1220_1220_0, 202410_1221_1221_0, 202410_1222_1222_0] 2024.10.22 02:04:41.375459 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ShareService 2024.10.22 02:04:41.458274 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ShareService 2024.10.22 02:04:41.523187 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ShareService 2024.10.22 02:04:41.553376 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} MergeTask::MergeProjectionsStage: Merge sorted 682 rows, containing 32 columns (32 merged, 0 gathered) in 5.083776448 sec., 134.15224036224183 rows/sec., 12.17 KiB/sec. 2024.10.22 02:04:41.579897 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 3.65 MiB, will set to 1.57 GiB (RSS), difference: -1.30 MiB 2024.10.22 02:04:41.612182 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_689_689_0, 202410_690_690_0, 202410_691_691_0, 202410_692_692_0, 202410_693_693_0, 202410_694_694_0] 2024.10.22 02:04:41.617515 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_689_689_0, 202410_690_690_0, 202410_691_691_0, 202410_692_692_0, 202410_693_693_0, 202410_694_694_0] 2024.10.22 02:04:41.646290 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} MergedBlockOutputStream: filled checksums 202410_1_638_130 (state Temporary) 2024.10.22 02:04:41.654105 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_689_689_0, 202410_690_690_0, 202410_691_691_0, 202410_692_692_0, 202410_693_693_0, 202410_694_694_0] 2024.10.22 02:04:41.667528 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ShareService 2024.10.22 02:04:41.683162 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ShareService 2024.10.22 02:04:41.729957 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ShareService 2024.10.22 02:04:41.755817 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:41.758639 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:41.764399 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ShareService 2024.10.22 02:04:41.841752 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/aif/ffscdlgarrairpcpmuobjkpywqmpb (LogSeriesLimiter: on interval from 2024-10-22 02:04:27 to 2024-10-22 02:04:41 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:04:42.014561 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_744_744_0 (state Temporary) 2024.10.22 02:04:42.089293 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_744_744_0 to 202410_751_751_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:42.145561 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1127370 2024.10.22 02:04:42.243764 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_638_130 to 202410_1_638_130 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:42.247496 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_638_130} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_633_129, 202410_638_638_0] -> 202410_1_638_130 2024.10.22 02:04:42.294932 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:04:42.323589 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_786_175} MergeTask::MergeProjectionsStage: Merge sorted 11893 rows, containing 12 columns (12 merged, 0 gathered) in 1.269177227 sec., 9370.637722607042 rows/sec., 1.46 MiB/sec. 2024.10.22 02:04:42.360620 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_786_175} MergedBlockOutputStream: filled checksums 202410_1_786_175 (state Temporary) 2024.10.22 02:04:42.467573 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 86428 2024.10.22 02:04:42.667635 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_786_175} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_786_175 to 202410_1_786_175 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:42.670074 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_786_175} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_785_174, 202410_786_786_0] -> 202410_1_786_175 2024.10.22 02:04:42.705196 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.44 MiB. 2024.10.22 02:04:42.936177 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:42.942629 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:42.995888 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1311_1311_0 (state Temporary) 2024.10.22 02:04:43.167582 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1311_1311_0 to 202410_1324_1324_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:43.238732 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86428 2024.10.22 02:04:43.401634 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(1f12913eec21c79cdd56c0e59200e5a5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1f12913eec21c79cdd56c0e59200e5a5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:04:43.429302 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 1f12913eec21c79cdd56c0e59200e5a5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:04:43.432197 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 5.07 MiB, will set to 1.56 GiB (RSS), difference: 66.37 MiB 2024.10.22 02:04:43.433410 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:43.751340 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.194032471 sec. 2024.10.22 02:04:43.783952 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_706_52, 202410_707_707_0] 2024.10.22 02:04:43.789429 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_706_52, 202410_707_707_0] 2024.10.22 02:04:43.807482 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_706_52, 202410_707_707_0] 2024.10.22 02:04:43.947906 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 647 2024.10.22 02:04:44.041780 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qpb/hnllljucvowdnddplkbujffkxkgvi. 2024.10.22 02:04:44.320288 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6269 2024.10.22 02:04:44.341118 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:44.345101 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:44.414844 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_629_629_0 (state Temporary) 2024.10.22 02:04:44.501078 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_629_629_0 to 202410_639_639_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:44.603580 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 647 2024.10.22 02:04:45.410861 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_726_149, 202410_727_727_0, 202410_728_728_0, 202410_729_729_0, 202410_730_730_0, 202410_731_731_0] 2024.10.22 02:04:45.419349 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_726_149, 202410_727_727_0, 202410_728_728_0, 202410_729_729_0, 202410_730_730_0, 202410_731_731_0] 2024.10.22 02:04:45.559440 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_726_149, 202410_727_727_0, 202410_728_728_0, 202410_729_729_0, 202410_730_730_0, 202410_731_731_0] 2024.10.22 02:04:45.643731 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 2.00 GiB, free memory in arenas 7.99 MiB, will set to 1.56 GiB (RSS), difference: 69.79 MiB 2024.10.22 02:04:45.665265 [ 1208 ] {} DatabaseCatalog: Execution took 221 ms. 2024.10.22 02:04:46.688042 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 69 entries to flush up to offset 86048 2024.10.22 02:04:46.934192 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:46.940864 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:47.180838 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_764_764_0 (state Temporary) 2024.10.22 02:04:47.224929 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_764_764_0 to 202410_770_770_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:47.255225 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 86463 2024.10.22 02:04:47.289214 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 86048 2024.10.22 02:04:47.427393 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 8.52 MiB, will set to 1.55 GiB (RSS), difference: 56.83 MiB 2024.10.22 02:04:47.567917 [ 1209 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:04:47.608981 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:47.612537 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:47.664239 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1312_1312_0 (state Temporary) 2024.10.22 02:04:47.697755 [ 1209 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:04:47.819717 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1312_1312_0 to 202410_1325_1325_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:47.975241 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86463 2024.10.22 02:04:48.056303 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4f876b3bea13c1c015526f0ebd1c67dc) : [0, 207]: Setting download as failed: File segment: [0, 207], key: 4f876b3bea13c1c015526f0ebd1c67dc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:04:48.059184 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4f876b3bea13c1c015526f0ebd1c67dc): No space left in cache to reserve 208 bytes, will continue without cache download 2024.10.22 02:04:48.448557 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 11859 2024.10.22 02:04:48.666368 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:48.668939 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:48.710798 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_778_778_0 (state Temporary) 2024.10.22 02:04:49.149838 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1128910 2024.10.22 02:04:49.161573 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_778_778_0 to 202410_791_791_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:49.199562 [ 1256 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_786_175 to 202410_787_787_0 2024.10.22 02:04:49.206300 [ 1256 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:49.211168 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:49.256163 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_787_176} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_786_175 to 202410_787_787_0 into Compact with storage Full 2024.10.22 02:04:49.304933 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_787_176} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:04:49.314178 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_787_176} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_786_175, total 11893 rows starting from the beginning of the part 2024.10.22 02:04:49.315186 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11859 2024.10.22 02:04:49.339520 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_787_176} MergeTreeSequentialSource: Reading 2 marks from part 202410_787_787_0, total 11 rows starting from the beginning of the part 2024.10.22 02:04:49.508880 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 8.93 MiB, will set to 1.55 GiB (RSS), difference: 1.96 MiB 2024.10.22 02:04:49.581376 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(25ba0debfb56277c481c5003d11dec00) : [0, 2237845]: Setting download as failed: File segment: [0, 2237845], key: 25ba0debfb56277c481c5003d11dec00, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:04:49.584979 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(25ba0debfb56277c481c5003d11dec00): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:04:49.653414 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 25ba0debfb56277c481c5003d11dec00, offset: 0, size: 0 2024.10.22 02:04:49.656537 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 25ba0debfb56277c481c5003d11dec00 for removal 2024.10.22 02:04:49.658977 [ 1476 ] {} CacheMetadata: Key 25ba0debfb56277c481c5003d11dec00 is removed from metadata 2024.10.22 02:04:49.661318 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4f876b3bea13c1c015526f0ebd1c67dc, offset: 0, size: 0 2024.10.22 02:04:49.662945 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4f876b3bea13c1c015526f0ebd1c67dc for removal 2024.10.22 02:04:49.763218 [ 1476 ] {} CacheMetadata: Key 4f876b3bea13c1c015526f0ebd1c67dc is removed from metadata 2024.10.22 02:04:50.227279 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:50.232523 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:50.456715 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(daeab82278b9bbdd0bcaf559cd0ab45d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: daeab82278b9bbdd0bcaf559cd0ab45d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:04:50.459139 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 922, file segment info: File segment: [0, 33554431], key: daeab82278b9bbdd0bcaf559cd0ab45d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:04:50.464286 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:50.566874 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_745_745_0 (state Temporary) 2024.10.22 02:04:50.567048 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_787_176} MergeTask::MergeProjectionsStage: Merge sorted 11904 rows, containing 12 columns (12 merged, 0 gathered) in 1.215825396 sec., 9790.879545009931 rows/sec., 1.53 MiB/sec. 2024.10.22 02:04:50.736036 [ 1690 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hdy/oxdloczekdnzbdkgganqvtlwchdwu, size 2147943 2024.10.22 02:04:50.822476 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_787_176} MergedBlockOutputStream: filled checksums 202410_1_787_176 (state Temporary) 2024.10.22 02:04:50.902971 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_787_176} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_787_176 to 202410_1_787_176 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:50.910566 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_787_176} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_786_175, 202410_787_787_0] -> 202410_1_787_176 2024.10.22 02:04:50.953409 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_745_745_0 to 202410_752_752_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:50.955708 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:04:50.974186 [ 1273 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_742_141 to 202410_743_748_1 2024.10.22 02:04:50.982875 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1128910 2024.10.22 02:04:50.997249 [ 1273 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:51.002663 [ 1273 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:51.031906 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_748_142} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_742_141 to 202410_743_748_1 into Wide with storage Full 2024.10.22 02:04:51.035834 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:04:51.044694 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:04:51.165217 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_748_142} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:04:51.170563 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_748_142} MergeTreeSequentialSource: Reading 139 marks from part 202410_1_742_141, total 1126715 rows starting from the beginning of the part 2024.10.22 02:04:51.172750 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:04:51.183642 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:04:51.287125 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:04:51.296671 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:04:51.311612 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:04:51.347324 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:04:51.367126 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:04:51.368981 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_748_142} MergeTreeSequentialSource: Reading 3 marks from part 202410_743_748_1, total 9380 rows starting from the beginning of the part 2024.10.22 02:04:51.535520 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lcs/owcvemwbbnvceyptctxdobanrofzx (LogSeriesLimiter: on interval from 2024-10-22 02:04:41 to 2024-10-22 02:04:51 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:04:51.587701 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(85ee2372ad80667d83f1fb8da68bc494) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 85ee2372ad80667d83f1fb8da68bc494, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:04:51.589359 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(85ee2372ad80667d83f1fb8da68bc494): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:04:51.954841 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 6310 2024.10.22 02:04:51.985452 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 86532 2024.10.22 02:04:52.107354 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 648 2024.10.22 02:04:52.363077 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:52.365536 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:52.368665 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.89 MiB, will set to 1.55 GiB (RSS), difference: -11.93 MiB 2024.10.22 02:04:52.552494 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1313_1313_0 (state Temporary) 2024.10.22 02:04:52.616210 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1313_1313_0 to 202410_1326_1326_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:52.639168 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:52.645002 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:52.699717 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_630_630_0 (state Temporary) 2024.10.22 02:04:52.745063 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86532 2024.10.22 02:04:52.988128 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_630_630_0 to 202410_640_640_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:53.054960 [ 1244 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 5 parts from 202410_155_155_0 to 202410_159_159_0 2024.10.22 02:04:53.064350 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d84ec41fd8b8f7a6a6611376b14eb1a4) : [0, 41210]: Setting download as failed: File segment: [0, 41210], key: d84ec41fd8b8f7a6a6611376b14eb1a4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:04:53.067243 [ 1244 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:53.069612 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d84ec41fd8b8f7a6a6611376b14eb1a4): No space left in cache to reserve 41211 bytes, will continue without cache download 2024.10.22 02:04:53.069734 [ 1244 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:53.123121 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_155_159_1} MergeTask::PrepareStage: Merging 5 parts: from 202410_155_155_0 to 202410_159_159_0 into Compact with storage Full 2024.10.22 02:04:53.135175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d84ec41fd8b8f7a6a6611376b14eb1a4, offset: 0, size: 0 2024.10.22 02:04:53.142580 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d84ec41fd8b8f7a6a6611376b14eb1a4 for removal 2024.10.22 02:04:53.148117 [ 1476 ] {} CacheMetadata: Key d84ec41fd8b8f7a6a6611376b14eb1a4 is removed from metadata 2024.10.22 02:04:53.151793 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 85ee2372ad80667d83f1fb8da68bc494, offset: 0, size: 0 2024.10.22 02:04:53.155594 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 85ee2372ad80667d83f1fb8da68bc494 for removal 2024.10.22 02:04:53.169352 [ 1476 ] {} CacheMetadata: Key 85ee2372ad80667d83f1fb8da68bc494 is removed from metadata 2024.10.22 02:04:53.274931 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 648 2024.10.22 02:04:53.372814 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_707_53, 202410_708_708_0] 2024.10.22 02:04:53.413825 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_707_53, 202410_708_708_0] 2024.10.22 02:04:53.438769 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_707_53, 202410_708_708_0] 2024.10.22 02:04:54.292697 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bd090ddf125a7621dbc460d8fbcbc247) : [0, 163]: Setting download as failed: File segment: [0, 163], key: bd090ddf125a7621dbc460d8fbcbc247, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:04:54.295580 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bd090ddf125a7621dbc460d8fbcbc247): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 02:04:54.327642 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 11.41 MiB, will set to 1.56 GiB (RSS), difference: 3.82 MiB 2024.10.22 02:04:54.608192 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_748_142/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:04:54.801155 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 86127 2024.10.22 02:04:55.047521 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:55.051040 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:55.094413 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_765_765_0 (state Temporary) 2024.10.22 02:04:55.095579 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6665cc0b1226e95df3b8dcfc04656484) : [0, 2508]: Setting download as failed: File segment: [0, 2508], key: 6665cc0b1226e95df3b8dcfc04656484, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:04:55.099070 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6665cc0b1226e95df3b8dcfc04656484): No space left in cache to reserve 2509 bytes, will continue without cache download 2024.10.22 02:04:55.126278 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6665cc0b1226e95df3b8dcfc04656484, offset: 0, size: 0 2024.10.22 02:04:55.128236 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6665cc0b1226e95df3b8dcfc04656484 for removal 2024.10.22 02:04:55.132213 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bd090ddf125a7621dbc460d8fbcbc247, offset: 0, size: 0 2024.10.22 02:04:55.135085 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bd090ddf125a7621dbc460d8fbcbc247 for removal 2024.10.22 02:04:55.250603 [ 1476 ] {} CacheMetadata: Key 6665cc0b1226e95df3b8dcfc04656484 is removed from metadata 2024.10.22 02:04:55.254622 [ 1476 ] {} CacheMetadata: Key bd090ddf125a7621dbc460d8fbcbc247 is removed from metadata 2024.10.22 02:04:55.258862 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_765_765_0 to 202410_771_771_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:55.311637 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 86127 2024.10.22 02:04:55.970435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(87c17e454920fc6b96bfd702a1ad5dc7) : [0, 159]: Setting download as failed: File segment: [0, 159], key: 87c17e454920fc6b96bfd702a1ad5dc7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:04:55.971934 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(87c17e454920fc6b96bfd702a1ad5dc7): No space left in cache to reserve 160 bytes, will continue without cache download 2024.10.22 02:04:56.003918 [ 1203 ] {} DatabaseCatalog: Execution took 336 ms. 2024.10.22 02:04:56.818856 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 11867 2024.10.22 02:04:56.861480 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 86581 2024.10.22 02:04:56.911667 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(82d60e061a99d908b6edd22f7725656a) : [0, 1077090]: Setting download as failed: File segment: [0, 1077090], key: 82d60e061a99d908b6edd22f7725656a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:04:56.914240 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 14.94 MiB, will set to 1.56 GiB (RSS), difference: 3.03 MiB 2024.10.22 02:04:56.915440 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(82d60e061a99d908b6edd22f7725656a): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:04:57.087172 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 82d60e061a99d908b6edd22f7725656a, offset: 0, size: 0 2024.10.22 02:04:57.091666 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 82d60e061a99d908b6edd22f7725656a for removal 2024.10.22 02:04:57.100255 [ 1476 ] {} CacheMetadata: Key 82d60e061a99d908b6edd22f7725656a is removed from metadata 2024.10.22 02:04:57.106986 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 87c17e454920fc6b96bfd702a1ad5dc7, offset: 0, size: 0 2024.10.22 02:04:57.113246 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 87c17e454920fc6b96bfd702a1ad5dc7 for removal 2024.10.22 02:04:57.125517 [ 1476 ] {} CacheMetadata: Key 87c17e454920fc6b96bfd702a1ad5dc7 is removed from metadata 2024.10.22 02:04:57.155624 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:57.188515 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:57.232869 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:57.251215 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:57.297317 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_779_779_0 (state Temporary) 2024.10.22 02:04:57.362868 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_682_131, 202410_683_688_1] 2024.10.22 02:04:57.364746 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_682_131, 202410_683_688_1] 2024.10.22 02:04:57.387298 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_682_131, 202410_683_688_1] 2024.10.22 02:04:57.409259 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1314_1314_0 (state Temporary) 2024.10.22 02:04:57.467757 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_779_779_0 to 202410_792_792_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:57.504405 [ 1195 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_787_176 to 202410_788_788_0 2024.10.22 02:04:57.538979 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11867 2024.10.22 02:04:57.554827 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1314_1314_0 to 202410_1327_1327_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:57.591043 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1322_23 to 202410_1327_1327_0 2024.10.22 02:04:57.605642 [ 1166 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:57.629990 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86581 2024.10.22 02:04:57.639854 [ 1195 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:57.643556 [ 1195 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:57.651412 [ 1166 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:57.704184 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_788_177} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_787_176 to 202410_788_788_0 into Compact with storage Full 2024.10.22 02:04:57.757592 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_788_177} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:04:57.768155 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_788_177} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_787_176, total 11904 rows starting from the beginning of the part 2024.10.22 02:04:57.768604 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1322_23 to 202410_1327_1327_0 into Compact with storage Full 2024.10.22 02:04:57.801160 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_788_177} MergeTreeSequentialSource: Reading 2 marks from part 202410_788_788_0, total 13 rows starting from the beginning of the part 2024.10.22 02:04:57.801722 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:04:57.836729 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1322_23, total 7155 rows starting from the beginning of the part 2024.10.22 02:04:57.884394 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_1323_1323_0, total 54 rows starting from the beginning of the part 2024.10.22 02:04:57.989244 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1130450 2024.10.22 02:04:58.078294 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_1324_1324_0, total 78 rows starting from the beginning of the part 2024.10.22 02:04:58.155203 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_1325_1325_0, total 35 rows starting from the beginning of the part 2024.10.22 02:04:58.188536 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_1326_1326_0, total 69 rows starting from the beginning of the part 2024.10.22 02:04:58.356234 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_1327_1327_0, total 49 rows starting from the beginning of the part 2024.10.22 02:04:58.696021 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_788_177} MergeTask::MergeProjectionsStage: Merge sorted 11917 rows, containing 12 columns (12 merged, 0 gathered) in 1.005113921 sec., 11856.367473394093 rows/sec., 1.85 MiB/sec. 2024.10.22 02:04:58.755351 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_788_177} MergedBlockOutputStream: filled checksums 202410_1_788_177 (state Temporary) 2024.10.22 02:04:58.848687 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:04:58.850847 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:04:58.862498 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b518674b07e687a532f6ed1281869659) : [0, 163]: Setting download as failed: File segment: [0, 163], key: b518674b07e687a532f6ed1281869659, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:04:58.863462 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b518674b07e687a532f6ed1281869659): No space left in cache to reserve 164 bytes, will continue without cache download 2024.10.22 02:04:58.975660 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_788_177} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_788_177 to 202410_1_788_177 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:58.979260 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_788_177} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_787_176, 202410_788_788_0] -> 202410_1_788_177 2024.10.22 02:04:59.093195 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_746_746_0 (state Temporary) 2024.10.22 02:04:59.213580 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:04:59.517316 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_746_746_0 to 202410_753_753_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:04:59.564344 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1130450 2024.10.22 02:04:59.586569 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(828196a318e5a9005387010bc356c863) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 828196a318e5a9005387010bc356c863, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:04:59.595443 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 828196a318e5a9005387010bc356c863, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:04:59.610653 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.89 MiB, will set to 1.56 GiB (RSS), difference: -1.12 MiB 2024.10.22 02:04:59.610576 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:04:59.709073 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} MergeTask::MergeProjectionsStage: Merge sorted 7440 rows, containing 13 columns (13 merged, 0 gathered) in 2.04884875 sec., 3631.3075818798243 rows/sec., 637.92 KiB/sec. 2024.10.22 02:04:59.914970 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} MergedBlockOutputStream: filled checksums 202410_1207_1327_24 (state Temporary) 2024.10.22 02:05:00.151500 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1327_24 to 202410_1207_1327_24 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:00.158865 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1327_24} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1322_23, 202410_1327_1327_0] -> 202410_1207_1327_24 2024.10.22 02:05:00.213587 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.52 MiB. 2024.10.22 02:05:00.242222 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_708_54, 202410_709_709_0] 2024.10.22 02:05:00.320958 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_708_54, 202410_709_709_0] 2024.10.22 02:05:00.436929 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_708_54, 202410_709_709_0] 2024.10.22 02:05:00.764973 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5a1c2a8357742b916336f4db20135672) : [0, 35000]: Setting download as failed: File segment: [0, 35000], key: 5a1c2a8357742b916336f4db20135672, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:00.766504 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5a1c2a8357742b916336f4db20135672): No space left in cache to reserve 35001 bytes, will continue without cache download 2024.10.22 02:05:00.792867 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5a1c2a8357742b916336f4db20135672, offset: 0, size: 0 2024.10.22 02:05:00.794744 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5a1c2a8357742b916336f4db20135672 for removal 2024.10.22 02:05:00.801255 [ 1476 ] {} CacheMetadata: Key 5a1c2a8357742b916336f4db20135672 is removed from metadata 2024.10.22 02:05:00.802970 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b518674b07e687a532f6ed1281869659, offset: 0, size: 0 2024.10.22 02:05:00.806566 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b518674b07e687a532f6ed1281869659 for removal 2024.10.22 02:05:00.813635 [ 1476 ] {} CacheMetadata: Key b518674b07e687a532f6ed1281869659 is removed from metadata 2024.10.22 02:05:01.175625 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:05:01.180055 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:05:01.642402 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 86658 2024.10.22 02:05:01.956302 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:01.969278 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:02.147671 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1315_1315_0 (state Temporary) 2024.10.22 02:05:02.356469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fb24db7cd91e6952baa7ae374357479e) : [0, 141]: Setting download as failed: File segment: [0, 141], key: fb24db7cd91e6952baa7ae374357479e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:02.357537 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fb24db7cd91e6952baa7ae374357479e): No space left in cache to reserve 142 bytes, will continue without cache download 2024.10.22 02:05:02.549036 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1315_1315_0 to 202410_1328_1328_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:02.549091 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1222_3, 202410_1223_1223_0, 202410_1224_1224_0, 202410_1225_1225_0, 202410_1226_1226_0, 202410_1227_1227_0] 2024.10.22 02:05:02.557057 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1222_3, 202410_1223_1223_0, 202410_1224_1224_0, 202410_1225_1225_0, 202410_1226_1226_0, 202410_1227_1227_0] 2024.10.22 02:05:02.581084 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1222_3, 202410_1223_1223_0, 202410_1224_1224_0, 202410_1225_1225_0, 202410_1226_1226_0, 202410_1227_1227_0] 2024.10.22 02:05:02.635109 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.62 MiB, will set to 1.57 GiB (RSS), difference: 5.53 MiB 2024.10.22 02:05:02.711807 [ 1199 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:05:02.737350 [ 1199 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:05:02.808161 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86658 2024.10.22 02:05:02.843899 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 87 entries to flush up to offset 86214 2024.10.22 02:05:03.327381 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(445904fdd952fd1b3f0d0112c9b321d0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 445904fdd952fd1b3f0d0112c9b321d0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 02:05:03.339118 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 227, file segment info: File segment: [0, 33554431], key: 445904fdd952fd1b3f0d0112c9b321d0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 02:05:03.342419 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:03.400951 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:03.410650 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:03.766102 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_155_159_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:03.874473 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_766_766_0 (state Temporary) 2024.10.22 02:05:04.229214 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_766_766_0 to 202410_772_772_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:04.258930 [ 1264 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_767_4 to 202410_772_772_0 2024.10.22 02:05:04.278688 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 86214 2024.10.22 02:05:04.285427 [ 1264 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:04.291740 [ 1264 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:04.323255 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_767_4 to 202410_772_772_0 into Compact with storage Full 2024.10.22 02:05:04.424273 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d1a0fe88fed331e5dc99f69cfbf095e8) : [0, 25398]: Setting download as failed: File segment: [0, 25398], key: d1a0fe88fed331e5dc99f69cfbf095e8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:04.427513 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d1a0fe88fed331e5dc99f69cfbf095e8): No space left in cache to reserve 25399 bytes, will continue without cache download 2024.10.22 02:05:04.445619 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d1a0fe88fed331e5dc99f69cfbf095e8, offset: 0, size: 0 2024.10.22 02:05:04.447826 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d1a0fe88fed331e5dc99f69cfbf095e8 for removal 2024.10.22 02:05:04.451546 [ 1476 ] {} CacheMetadata: Key d1a0fe88fed331e5dc99f69cfbf095e8 is removed from metadata 2024.10.22 02:05:04.452204 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fb24db7cd91e6952baa7ae374357479e, offset: 0, size: 0 2024.10.22 02:05:04.454412 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fb24db7cd91e6952baa7ae374357479e for removal 2024.10.22 02:05:04.460082 [ 1476 ] {} CacheMetadata: Key fb24db7cd91e6952baa7ae374357479e is removed from metadata 2024.10.22 02:05:04.497536 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:04.526079 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_767_4, total 2490 rows starting from the beginning of the part 2024.10.22 02:05:04.557067 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_768_768_0, total 51 rows starting from the beginning of the part 2024.10.22 02:05:04.577189 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_769_769_0, total 83 rows starting from the beginning of the part 2024.10.22 02:05:04.611474 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_770_770_0, total 69 rows starting from the beginning of the part 2024.10.22 02:05:04.651350 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_771_771_0, total 79 rows starting from the beginning of the part 2024.10.22 02:05:04.678249 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_772_772_0, total 87 rows starting from the beginning of the part 2024.10.22 02:05:05.043956 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 11882 2024.10.22 02:05:05.161438 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:05.164820 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:05.196720 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_780_780_0 (state Temporary) 2024.10.22 02:05:05.240299 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_155_159_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_155_155_0, total 39 rows starting from the beginning of the part 2024.10.22 02:05:05.346854 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_780_780_0 to 202410_793_793_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:05.386873 [ 1211 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_788_177 to 202410_789_789_0 2024.10.22 02:05:05.398234 [ 1211 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:05.403721 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:05.436554 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_789_178} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_788_177 to 202410_789_789_0 into Compact with storage Full 2024.10.22 02:05:05.499702 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11882 2024.10.22 02:05:05.480124 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(2c6327ed4299cd2ad358240b960503e7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2c6327ed4299cd2ad358240b960503e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 02:05:05.513441 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 180, file segment info: File segment: [0, 33554431], key: 2c6327ed4299cd2ad358240b960503e7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 02:05:05.516754 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:05.655059 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_789_178} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:05.668787 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_789_178} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_788_177, total 11917 rows starting from the beginning of the part 2024.10.22 02:05:05.672146 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} MergeTask::MergeProjectionsStage: Merge sorted 2859 rows, containing 13 columns (13 merged, 0 gathered) in 1.366498344 sec., 2092.208902084114 rows/sec., 932.12 KiB/sec. 2024.10.22 02:05:05.711015 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_789_178} MergeTreeSequentialSource: Reading 2 marks from part 202410_789_789_0, total 12 rows starting from the beginning of the part 2024.10.22 02:05:05.725448 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} MergedBlockOutputStream: filled checksums 202410_747_772_5 (state Temporary) 2024.10.22 02:05:06.016846 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_748_142/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:05:06.263775 [ 1269 ] {} DatabaseCatalog: Execution took 248 ms. 2024.10.22 02:05:06.343969 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_772_5 to 202410_747_772_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:06.349332 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_772_5} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_767_4, 202410_772_772_0] -> 202410_747_772_5 2024.10.22 02:05:06.477518 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1a4cad76ea18bd76cb399c3269b4b782) : [0, 145]: Setting download as failed: File segment: [0, 145], key: 1a4cad76ea18bd76cb399c3269b4b782, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:06.482955 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1a4cad76ea18bd76cb399c3269b4b782): No space left in cache to reserve 146 bytes, will continue without cache download 2024.10.22 02:05:06.568731 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1131220 2024.10.22 02:05:06.589424 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.20 MiB. 2024.10.22 02:05:06.580204 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.55 MiB, will set to 1.57 GiB (RSS), difference: 2.43 MiB 2024.10.22 02:05:06.816211 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:05:06.816560 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 86729 2024.10.22 02:05:06.890826 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_789_178} MergeTask::MergeProjectionsStage: Merge sorted 11929 rows, containing 12 columns (12 merged, 0 gathered) in 1.468400742 sec., 8123.804121586312 rows/sec., 1.27 MiB/sec. 2024.10.22 02:05:06.968419 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_789_178} MergedBlockOutputStream: filled checksums 202410_1_789_178 (state Temporary) 2024.10.22 02:05:07.127061 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:07.136185 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:07.259484 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_789_178} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_789_178 to 202410_1_789_178 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:07.263123 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_789_178} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_788_177, 202410_789_789_0] -> 202410_1_789_178 2024.10.22 02:05:07.301394 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1316_1316_0 (state Temporary) 2024.10.22 02:05:07.310151 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:05:07.343526 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:07.347386 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:07.360551 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1316_1316_0 to 202410_1329_1329_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:07.523060 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86729 2024.10.22 02:05:07.660855 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_747_747_0 (state Temporary) 2024.10.22 02:05:07.827341 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_747_747_0 to 202410_754_754_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:07.871144 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1131220 2024.10.22 02:05:07.861329 [ 1198 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_749_749_0 to 202410_754_754_0 2024.10.22 02:05:07.874726 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(7b64bb48e8731ad1b6f3f1afb99b1cc7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7b64bb48e8731ad1b6f3f1afb99b1cc7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 02:05:07.915091 [ 1198 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:07.920656 [ 1198 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:07.959353 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_749_749_0 to 202410_754_754_0 into Compact with storage Full 2024.10.22 02:05:07.996541 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:08.003288 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_749_749_0, total 1785 rows starting from the beginning of the part 2024.10.22 02:05:08.013956 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 535, file segment info: File segment: [0, 33554431], key: 7b64bb48e8731ad1b6f3f1afb99b1cc7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 02:05:08.016802 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:08.029068 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_750_750_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:05:08.047400 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_751_751_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:05:08.060189 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_752_752_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:05:08.070287 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_753_753_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:05:08.075607 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_155_159_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_156_156_0, total 37 rows starting from the beginning of the part 2024.10.22 02:05:08.087166 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_754_754_0, total 770 rows starting from the beginning of the part 2024.10.22 02:05:08.166145 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_709_55, 202410_710_710_0] 2024.10.22 02:05:08.166609 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:08.180120 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:08.179276 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_709_55, 202410_710_710_0] 2024.10.22 02:05:08.225620 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_709_55, 202410_710_710_0] 2024.10.22 02:05:08.381136 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 649 2024.10.22 02:05:08.695413 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:08.701542 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:08.770890 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_631_631_0 (state Temporary) 2024.10.22 02:05:08.952799 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_631_631_0 to 202410_641_641_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:09.050539 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 649 2024.10.22 02:05:09.257464 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(09dedb76b60d2deb861892df7119dc8d) : [0, 241199]: Setting download as failed: File segment: [0, 241199], key: 09dedb76b60d2deb861892df7119dc8d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:09.259428 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(09dedb76b60d2deb861892df7119dc8d): No space left in cache to reserve 241200 bytes, will continue without cache download 2024.10.22 02:05:09.291726 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 09dedb76b60d2deb861892df7119dc8d, offset: 0, size: 0 2024.10.22 02:05:09.296658 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 09dedb76b60d2deb861892df7119dc8d for removal 2024.10.22 02:05:09.306302 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1a4cad76ea18bd76cb399c3269b4b782, offset: 0, size: 0 2024.10.22 02:05:09.308254 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1a4cad76ea18bd76cb399c3269b4b782 for removal 2024.10.22 02:05:09.402891 [ 1476 ] {} CacheMetadata: Key 09dedb76b60d2deb861892df7119dc8d is removed from metadata 2024.10.22 02:05:09.408452 [ 1476 ] {} CacheMetadata: Key 1a4cad76ea18bd76cb399c3269b4b782 is removed from metadata 2024.10.22 02:05:09.679735 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} MergeTask::MergeProjectionsStage: Merge sorted 8715 rows, containing 4 columns (4 merged, 0 gathered) in 1.730081619 sec., 5037.334599877048 rows/sec., 78.71 KiB/sec. 2024.10.22 02:05:09.748722 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} MergedBlockOutputStream: filled checksums 202410_749_754_1 (state Temporary) 2024.10.22 02:05:09.805636 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_155_159_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_157_157_0, total 37 rows starting from the beginning of the part 2024.10.22 02:05:09.909792 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_749_754_1 to 202410_749_754_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:10.017220 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_749_754_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_749_749_0, 202410_754_754_0] -> 202410_749_754_1 2024.10.22 02:05:10.067903 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:05:10.549000 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d507421bdbb84158a3b5c35c91407548) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d507421bdbb84158a3b5c35c91407548, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:05:10.551008 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 902, file segment info: File segment: [0, 33554431], key: d507421bdbb84158a3b5c35c91407548, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:05:10.552839 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:10.600798 [ 1732 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lcs/owcvemwbbnvceyptctxdobanrofzx, size 2027602 2024.10.22 02:05:10.930650 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:05:10.944502 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:05:10.962332 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:05:10.970182 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:05:11.095747 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:05:11.128132 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:05:11.142729 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:05:11.163298 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_588_120, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0, 202410_592_592_0, 202410_593_593_0] 2024.10.22 02:05:11.167196 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_588_120, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0, 202410_592_592_0, 202410_593_593_0] 2024.10.22 02:05:11.174181 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tnd/eeivugcdrnnwdgqdkidofnkbejvck (LogSeriesLimiter: on interval from 2024-10-22 02:04:51 to 2024-10-22 02:05:11 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:05:11.275207 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:05:11.288718 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:05:11.339030 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(4ae8c71ca649ce0a8d198968cf4a0bf6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4ae8c71ca649ce0a8d198968cf4a0bf6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 02:05:11.352541 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 441, file segment info: File segment: [0, 33554431], key: 4ae8c71ca649ce0a8d198968cf4a0bf6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1394, kind: Regular, unbound: 0 2024.10.22 02:05:11.354825 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:11.389784 [ 1706 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/aif/ffscdlgarrairpcpmuobjkpywqmpb, size 535 2024.10.22 02:05:11.499119 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.10 MiB, will set to 1.56 GiB (RSS), difference: -6.35 MiB 2024.10.22 02:05:11.501500 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_588_120, 202410_589_589_0, 202410_590_590_0, 202410_591_591_0, 202410_592_592_0, 202410_593_593_0] 2024.10.22 02:05:11.533575 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 86808 2024.10.22 02:05:11.489656 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:05:11.648012 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:05:11.666301 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:05:11.699405 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_155_159_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_158_158_0, total 39 rows starting from the beginning of the part 2024.10.22 02:05:11.703787 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:05:11.739000 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:05:11.747614 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_157_157_0 (state Temporary) 2024.10.22 02:05:11.775315 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:05:11.808526 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 112 entries to flush up to offset 86326 2024.10.22 02:05:11.816272 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:05:11.835261 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:11.854465 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:12.158998 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1317_1317_0 (state Temporary) 2024.10.22 02:05:12.166578 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:05:12.196059 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:12.205429 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:12.249645 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1317_1317_0 to 202410_1330_1330_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:12.335582 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86808 2024.10.22 02:05:12.378387 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_767_767_0 (state Temporary) 2024.10.22 02:05:12.661486 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_767_767_0 to 202410_773_773_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:12.732347 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 86326 2024.10.22 02:05:12.773675 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b83aba64019bbf8d3ec0031681704f75) : [0, 145]: Setting download as failed: File segment: [0, 145], key: b83aba64019bbf8d3ec0031681704f75, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:12.877046 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b83aba64019bbf8d3ec0031681704f75): No space left in cache to reserve 146 bytes, will continue without cache download 2024.10.22 02:05:13.001554 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 11905 2024.10.22 02:05:13.342835 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:13.351300 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:13.421351 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_781_781_0 (state Temporary) 2024.10.22 02:05:13.505541 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_781_781_0 to 202410_794_794_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:13.569462 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_155_159_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_159_159_0, total 39 rows starting from the beginning of the part 2024.10.22 02:05:13.653611 [ 1223 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_789_178 to 202410_790_790_0 2024.10.22 02:05:13.668029 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11905 2024.10.22 02:05:13.671327 [ 1223 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:13.675067 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:13.725278 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_790_179} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_789_178 to 202410_790_790_0 into Compact with storage Full 2024.10.22 02:05:13.835012 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(940b4e97092c4a6b85d3dafacef9934d) : [0, 155359]: Setting download as failed: File segment: [0, 155359], key: 940b4e97092c4a6b85d3dafacef9934d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:13.837411 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(940b4e97092c4a6b85d3dafacef9934d): No space left in cache to reserve 155360 bytes, will continue without cache download 2024.10.22 02:05:13.863486 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 940b4e97092c4a6b85d3dafacef9934d, offset: 0, size: 0 2024.10.22 02:05:13.865773 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 940b4e97092c4a6b85d3dafacef9934d for removal 2024.10.22 02:05:13.868231 [ 1476 ] {} CacheMetadata: Key 940b4e97092c4a6b85d3dafacef9934d is removed from metadata 2024.10.22 02:05:13.869033 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b83aba64019bbf8d3ec0031681704f75, offset: 0, size: 0 2024.10.22 02:05:13.870406 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b83aba64019bbf8d3ec0031681704f75 for removal 2024.10.22 02:05:13.873583 [ 1476 ] {} CacheMetadata: Key b83aba64019bbf8d3ec0031681704f75 is removed from metadata 2024.10.22 02:05:13.881286 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_790_179} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:13.997464 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_790_179} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_789_178, total 11929 rows starting from the beginning of the part 2024.10.22 02:05:14.023621 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_790_179} MergeTreeSequentialSource: Reading 2 marks from part 202410_790_790_0, total 19 rows starting from the beginning of the part 2024.10.22 02:05:14.582720 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 10.40 MiB, will set to 1.56 GiB (RSS), difference: -18.78 MiB 2024.10.22 02:05:14.613773 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_790_179} MergeTask::MergeProjectionsStage: Merge sorted 11948 rows, containing 12 columns (12 merged, 0 gathered) in 0.903077954 sec., 13230.308576439902 rows/sec., 2.06 MiB/sec. 2024.10.22 02:05:14.854340 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_790_179} MergedBlockOutputStream: filled checksums 202410_1_790_179 (state Temporary) 2024.10.22 02:05:14.875104 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1131990 2024.10.22 02:05:14.904322 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7ca77d6cb24650842e2056209141d053) : [0, 177]: Setting download as failed: File segment: [0, 177], key: 7ca77d6cb24650842e2056209141d053, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:14.905335 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7ca77d6cb24650842e2056209141d053): No space left in cache to reserve 178 bytes, will continue without cache download 2024.10.22 02:05:15.120953 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_790_179} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_790_179 to 202410_1_790_179 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:15.124771 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_790_179} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_789_178, 202410_790_790_0] -> 202410_1_790_179 2024.10.22 02:05:15.152883 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:05:15.211219 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_157_157_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:05:15.282204 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:15.291983 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:15.407601 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_710_56, 202410_711_711_0] 2024.10.22 02:05:15.410437 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_710_56, 202410_711_711_0] 2024.10.22 02:05:15.429436 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_710_56, 202410_711_711_0] 2024.10.22 02:05:15.449178 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_748_748_0 (state Temporary) 2024.10.22 02:05:15.514769 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_748_748_0 to 202410_755_755_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:15.551980 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1131990 2024.10.22 02:05:15.771022 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8acbd10d33f544a8940aa331e9537643) : [0, 335694]: Setting download as failed: File segment: [0, 335694], key: 8acbd10d33f544a8940aa331e9537643, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:15.774551 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8acbd10d33f544a8940aa331e9537643): No space left in cache to reserve 335695 bytes, will continue without cache download 2024.10.22 02:05:15.917442 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8acbd10d33f544a8940aa331e9537643, offset: 0, size: 0 2024.10.22 02:05:15.920092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8acbd10d33f544a8940aa331e9537643 for removal 2024.10.22 02:05:15.924366 [ 1476 ] {} CacheMetadata: Key 8acbd10d33f544a8940aa331e9537643 is removed from metadata 2024.10.22 02:05:15.930482 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7ca77d6cb24650842e2056209141d053, offset: 0, size: 0 2024.10.22 02:05:15.935975 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7ca77d6cb24650842e2056209141d053 for removal 2024.10.22 02:05:16.140697 [ 1476 ] {} CacheMetadata: Key 7ca77d6cb24650842e2056209141d053 is removed from metadata 2024.10.22 02:05:16.449674 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 86880 2024.10.22 02:05:16.514702 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 8.72 MiB, will set to 1.56 GiB (RSS), difference: 1.93 MiB 2024.10.22 02:05:16.553107 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 650 2024.10.22 02:05:16.564325 [ 1150 ] {} DatabaseCatalog: Execution took 299 ms. 2024.10.22 02:05:16.669104 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_748_142/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:05:16.932071 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:16.936119 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:16.989055 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:16.993045 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:17.108854 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(23dcc92707f3782943ce3ae5d78a6de0) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 23dcc92707f3782943ce3ae5d78a6de0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:17.112994 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(23dcc92707f3782943ce3ae5d78a6de0): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 02:05:17.130311 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1318_1318_0 (state Temporary) 2024.10.22 02:05:17.302336 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_632_632_0 (state Temporary) 2024.10.22 02:05:17.328392 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1318_1318_0 to 202410_1331_1331_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:17.481028 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86880 2024.10.22 02:05:17.492500 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_632_632_0 to 202410_642_642_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:17.690426 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 650 2024.10.22 02:05:17.747047 [ 1195 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:05:17.779917 [ 1195 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:05:18.254866 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f43d602b6e2d9695e2d1df70dba1c448) : [0, 5459]: Setting download as failed: File segment: [0, 5459], key: f43d602b6e2d9695e2d1df70dba1c448, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:18.257086 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f43d602b6e2d9695e2d1df70dba1c448): No space left in cache to reserve 5460 bytes, will continue without cache download 2024.10.22 02:05:18.269957 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f43d602b6e2d9695e2d1df70dba1c448, offset: 0, size: 0 2024.10.22 02:05:18.272673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f43d602b6e2d9695e2d1df70dba1c448 for removal 2024.10.22 02:05:18.276697 [ 1476 ] {} CacheMetadata: Key f43d602b6e2d9695e2d1df70dba1c448 is removed from metadata 2024.10.22 02:05:18.277573 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 23dcc92707f3782943ce3ae5d78a6de0, offset: 0, size: 0 2024.10.22 02:05:18.283406 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 23dcc92707f3782943ce3ae5d78a6de0 for removal 2024.10.22 02:05:18.292887 [ 1476 ] {} CacheMetadata: Key 23dcc92707f3782943ce3ae5d78a6de0 is removed from metadata 2024.10.22 02:05:19.354803 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7f340d539e91c973d3ef7360d6e526d2) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 7f340d539e91c973d3ef7360d6e526d2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:19.359930 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7f340d539e91c973d3ef7360d6e526d2): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:05:19.546994 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 13.47 MiB, will set to 1.57 GiB (RSS), difference: 2.66 MiB 2024.10.22 02:05:20.254862 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3edb58a7ecd8b7987d6bba11fa87fda4) : [0, 62838]: Setting download as failed: File segment: [0, 62838], key: 3edb58a7ecd8b7987d6bba11fa87fda4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:20.254987 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 86405 2024.10.22 02:05:20.263523 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3edb58a7ecd8b7987d6bba11fa87fda4): No space left in cache to reserve 62839 bytes, will continue without cache download 2024.10.22 02:05:20.405592 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3edb58a7ecd8b7987d6bba11fa87fda4, offset: 0, size: 0 2024.10.22 02:05:20.510285 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3edb58a7ecd8b7987d6bba11fa87fda4 for removal 2024.10.22 02:05:20.512310 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:20.517025 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7f340d539e91c973d3ef7360d6e526d2, offset: 0, size: 0 2024.10.22 02:05:20.518761 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7f340d539e91c973d3ef7360d6e526d2 for removal 2024.10.22 02:05:20.518775 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:20.522967 [ 1476 ] {} CacheMetadata: Key 3edb58a7ecd8b7987d6bba11fa87fda4 is removed from metadata 2024.10.22 02:05:20.529473 [ 1476 ] {} CacheMetadata: Key 7f340d539e91c973d3ef7360d6e526d2 is removed from metadata 2024.10.22 02:05:20.668159 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_768_768_0 (state Temporary) 2024.10.22 02:05:20.707207 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_768_768_0 to 202410_774_774_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:20.750844 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 86405 2024.10.22 02:05:20.975437 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_157_157_0 to 202410_160_160_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:21.172347 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 11914 2024.10.22 02:05:21.174922 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:05:21.181315 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:05:21.459175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4e858e849e7bdb05e4bcf114b0220d5b) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 4e858e849e7bdb05e4bcf114b0220d5b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:21.462007 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4e858e849e7bdb05e4bcf114b0220d5b): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 02:05:21.476384 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.39 MiB, will set to 1.57 GiB (RSS), difference: 3.41 MiB 2024.10.22 02:05:21.489583 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:21.511157 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 02:05:21.538338 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 86931 2024.10.22 02:05:21.543247 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:21.623837 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_782_782_0 (state Temporary) 2024.10.22 02:05:21.664155 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_782_782_0 to 202410_795_795_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:21.760063 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11914 2024.10.22 02:05:21.815334 [ 1169 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_790_179 to 202410_791_791_0 2024.10.22 02:05:21.829587 [ 1169 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:21.835511 [ 1169 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:21.856037 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:21.858629 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:21.936500 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_791_180} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_790_179 to 202410_791_791_0 into Compact with storage Full 2024.10.22 02:05:21.982155 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_791_180} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:22.058163 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_791_180} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_790_179, total 11948 rows starting from the beginning of the part 2024.10.22 02:05:22.083774 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_791_180} MergeTreeSequentialSource: Reading 2 marks from part 202410_791_791_0, total 17 rows starting from the beginning of the part 2024.10.22 02:05:22.092008 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1319_1319_0 (state Temporary) 2024.10.22 02:05:22.229613 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(34165d4f58e920af445b532e063e805f) : [0, 71504]: Setting download as failed: File segment: [0, 71504], key: 34165d4f58e920af445b532e063e805f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:22.236542 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(34165d4f58e920af445b532e063e805f): No space left in cache to reserve 71505 bytes, will continue without cache download 2024.10.22 02:05:22.263489 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1319_1319_0 to 202410_1332_1332_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:22.292625 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1327_24 to 202410_1332_1332_0 2024.10.22 02:05:22.303382 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:22.310566 [ 1196 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:22.361743 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 34165d4f58e920af445b532e063e805f, offset: 0, size: 0 2024.10.22 02:05:22.363806 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 34165d4f58e920af445b532e063e805f for removal 2024.10.22 02:05:22.367086 [ 1476 ] {} CacheMetadata: Key 34165d4f58e920af445b532e063e805f is removed from metadata 2024.10.22 02:05:22.368498 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4e858e849e7bdb05e4bcf114b0220d5b, offset: 0, size: 0 2024.10.22 02:05:22.371019 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4e858e849e7bdb05e4bcf114b0220d5b for removal 2024.10.22 02:05:22.370685 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1327_24 to 202410_1332_1332_0 into Compact with storage Full 2024.10.22 02:05:22.373383 [ 1476 ] {} CacheMetadata: Key 4e858e849e7bdb05e4bcf114b0220d5b is removed from metadata 2024.10.22 02:05:22.467795 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 86931 2024.10.22 02:05:22.588294 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:22.642301 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1327_24, total 7440 rows starting from the beginning of the part 2024.10.22 02:05:22.655187 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1228 entries to flush up to offset 1133218 2024.10.22 02:05:22.845733 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_1328_1328_0, total 77 rows starting from the beginning of the part 2024.10.22 02:05:23.064773 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_791_180} MergeTask::MergeProjectionsStage: Merge sorted 11965 rows, containing 12 columns (12 merged, 0 gathered) in 1.202313461 sec., 9951.647709282364 rows/sec., 1.55 MiB/sec. 2024.10.22 02:05:23.101028 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_791_180} MergedBlockOutputStream: filled checksums 202410_1_791_180 (state Temporary) 2024.10.22 02:05:23.174371 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_1329_1329_0, total 71 rows starting from the beginning of the part 2024.10.22 02:05:23.252775 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_1330_1330_0, total 79 rows starting from the beginning of the part 2024.10.22 02:05:23.364205 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_1331_1331_0, total 72 rows starting from the beginning of the part 2024.10.22 02:05:23.436226 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:23.443781 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:23.576222 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(775eb0aca0a026314ad1d0d01aed3a07) : [0, 164]: Setting download as failed: File segment: [0, 164], key: 775eb0aca0a026314ad1d0d01aed3a07, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:23.579226 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(775eb0aca0a026314ad1d0d01aed3a07): No space left in cache to reserve 165 bytes, will continue without cache download 2024.10.22 02:05:23.588845 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_1332_1332_0, total 51 rows starting from the beginning of the part 2024.10.22 02:05:23.606881 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6310 2024.10.22 02:05:23.611879 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_791_180} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_791_180 to 202410_1_791_180 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:23.653464 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_791_180} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_790_179, 202410_791_791_0] -> 202410_1_791_180 2024.10.22 02:05:23.681564 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.45 MiB. 2024.10.22 02:05:23.778911 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_749_749_0 (state Temporary) 2024.10.22 02:05:23.851460 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_749_749_0 to 202410_756_756_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:23.868956 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1133218 2024.10.22 02:05:24.374312 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.73 MiB, will set to 1.57 GiB (RSS), difference: -2.61 MiB 2024.10.22 02:05:24.553129 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0f8e22e5f64670998bc84056f4ad01e2) : [0, 110965]: Setting download as failed: File segment: [0, 110965], key: 0f8e22e5f64670998bc84056f4ad01e2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:24.555229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0f8e22e5f64670998bc84056f4ad01e2): No space left in cache to reserve 110966 bytes, will continue without cache download 2024.10.22 02:05:24.580834 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0f8e22e5f64670998bc84056f4ad01e2, offset: 0, size: 0 2024.10.22 02:05:24.587290 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0f8e22e5f64670998bc84056f4ad01e2 for removal 2024.10.22 02:05:24.591619 [ 1476 ] {} CacheMetadata: Key 0f8e22e5f64670998bc84056f4ad01e2 is removed from metadata 2024.10.22 02:05:24.592725 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 775eb0aca0a026314ad1d0d01aed3a07, offset: 0, size: 0 2024.10.22 02:05:24.595059 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 775eb0aca0a026314ad1d0d01aed3a07 for removal 2024.10.22 02:05:24.603044 [ 1476 ] {} CacheMetadata: Key 775eb0aca0a026314ad1d0d01aed3a07 is removed from metadata 2024.10.22 02:05:25.193236 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 651 2024.10.22 02:05:25.237625 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} MergeTask::MergeProjectionsStage: Merge sorted 7790 rows, containing 13 columns (13 merged, 0 gathered) in 2.898409052 sec., 2687.6813659633854 rows/sec., 472.38 KiB/sec. 2024.10.22 02:05:25.320396 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} MergedBlockOutputStream: filled checksums 202410_1207_1332_25 (state Temporary) 2024.10.22 02:05:25.401079 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:25.489671 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1332_25 to 202410_1207_1332_25 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:25.494570 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1332_25} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1327_24, 202410_1332_1332_0] -> 202410_1207_1332_25 2024.10.22 02:05:25.508712 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:25.564228 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_633_633_0 (state Temporary) 2024.10.22 02:05:25.628694 [ 1234 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 02:05:25.646958 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.60 MiB. 2024.10.22 02:05:25.658847 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_633_633_0 to 202410_643_643_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:25.716464 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_711_57, 202410_712_712_0] 2024.10.22 02:05:25.720241 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_711_57, 202410_712_712_0] 2024.10.22 02:05:25.786611 [ 1208 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_638_130 to 202410_643_643_0 2024.10.22 02:05:25.809285 [ 1208 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:25.816120 [ 1208 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:25.833486 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_711_57, 202410_712_712_0] 2024.10.22 02:05:25.869734 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 651 2024.10.22 02:05:25.939647 [ 1208 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 270 ms. 2024.10.22 02:05:25.943702 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7d3954c8c3241db1df339a9ec1afd5d0) : [0, 164]: Setting download as failed: File segment: [0, 164], key: 7d3954c8c3241db1df339a9ec1afd5d0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:25.946368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7d3954c8c3241db1df339a9ec1afd5d0): No space left in cache to reserve 165 bytes, will continue without cache download 2024.10.22 02:05:25.950726 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_638_130 to 202410_643_643_0 into Compact with storage Full 2024.10.22 02:05:26.043922 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:26.023544 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_731_150, 202410_732_732_0] 2024.10.22 02:05:26.050904 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_731_150, 202410_732_732_0] 2024.10.22 02:05:26.069272 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_638_130, total 682 rows starting from the beginning of the part 2024.10.22 02:05:26.162165 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_731_150, 202410_732_732_0] 2024.10.22 02:05:26.330380 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_639_639_0, total 1 rows starting from the beginning of the part 2024.10.22 02:05:26.476867 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 87020 2024.10.22 02:05:26.497188 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 15.31 MiB, will set to 1.57 GiB (RSS), difference: 8.20 MiB 2024.10.22 02:05:26.556790 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_640_640_0, total 1 rows starting from the beginning of the part 2024.10.22 02:05:26.766788 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(c6371d397bcb5243fe5a0027de2e7900) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c6371d397bcb5243fe5a0027de2e7900, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 02:05:26.769493 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_641_641_0, total 1 rows starting from the beginning of the part 2024.10.22 02:05:26.785818 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: c6371d397bcb5243fe5a0027de2e7900, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 02:05:26.789363 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_155_159_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:05:26.793129 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:26.815468 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 7 old parts to remove. Parts: [202410_1_134_24, 202410_135_139_1, 202410_140_140_0, 202410_141_141_0, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0] 2024.10.22 02:05:26.823623 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from filesystem (serially): Parts: [202410_1_134_24, 202410_135_139_1, 202410_140_140_0, 202410_141_141_0, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0] 2024.10.22 02:05:26.867467 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from memory: Parts: [202410_1_134_24, 202410_135_139_1, 202410_140_140_0, 202410_141_141_0, 202410_142_142_0, 202410_143_143_0, 202410_144_144_0] 2024.10.22 02:05:26.888367 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_642_642_0, total 1 rows starting from the beginning of the part 2024.10.22 02:05:26.914179 [ 1245 ] {} DatabaseCatalog: Execution took 348 ms. 2024.10.22 02:05:26.991742 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:26.997812 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:27.075118 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_643_643_0, total 1 rows starting from the beginning of the part 2024.10.22 02:05:27.227041 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1320_1320_0 (state Temporary) 2024.10.22 02:05:27.277282 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1320_1320_0 to 202410_1333_1333_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:27.369536 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87020 2024.10.22 02:05:27.962779 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8f36d9a7f365aa7ff4bd3b9580e1890c) : [0, 107612]: Setting download as failed: File segment: [0, 107612], key: 8f36d9a7f365aa7ff4bd3b9580e1890c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:27.964310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8f36d9a7f365aa7ff4bd3b9580e1890c): No space left in cache to reserve 107613 bytes, will continue without cache download 2024.10.22 02:05:28.001833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8f36d9a7f365aa7ff4bd3b9580e1890c, offset: 0, size: 0 2024.10.22 02:05:28.005797 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8f36d9a7f365aa7ff4bd3b9580e1890c for removal 2024.10.22 02:05:28.012124 [ 1476 ] {} CacheMetadata: Key 8f36d9a7f365aa7ff4bd3b9580e1890c is removed from metadata 2024.10.22 02:05:28.013739 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7d3954c8c3241db1df339a9ec1afd5d0, offset: 0, size: 0 2024.10.22 02:05:28.016876 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7d3954c8c3241db1df339a9ec1afd5d0 for removal 2024.10.22 02:05:28.038105 [ 1476 ] {} CacheMetadata: Key 7d3954c8c3241db1df339a9ec1afd5d0 is removed from metadata 2024.10.22 02:05:28.179124 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_643_131/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:05:28.262255 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 106 entries to flush up to offset 86511 2024.10.22 02:05:28.375280 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_748_142/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:05:28.488998 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:28.493340 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:28.632196 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_769_769_0 (state Temporary) 2024.10.22 02:05:28.673560 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_769_769_0 to 202410_775_775_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:28.753370 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 86511 2024.10.22 02:05:28.792338 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(97a14235a735539692678f9cb7593efe) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 97a14235a735539692678f9cb7593efe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:28.797242 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(97a14235a735539692678f9cb7593efe): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:05:29.263883 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 11930 2024.10.22 02:05:29.275541 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_748_142} MergeTask::MergeProjectionsStage: Merge sorted 1136095 rows, containing 4 columns (4 merged, 0 gathered) in 38.260773346 sec., 29693.466719192016 rows/sec., 445.32 KiB/sec. 2024.10.22 02:05:29.527235 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 15.12 MiB, will set to 1.57 GiB (RSS), difference: 1.17 MiB 2024.10.22 02:05:29.565821 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:29.570571 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:29.611158 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_783_783_0 (state Temporary) 2024.10.22 02:05:29.743625 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_748_142} MergedBlockOutputStream: filled checksums 202410_1_748_142 (state Temporary) 2024.10.22 02:05:29.842652 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} MergeTask::MergeProjectionsStage: Merge sorted 687 rows, containing 32 columns (32 merged, 0 gathered) in 3.900088894 sec., 176.149831112029 rows/sec., 15.98 KiB/sec. 2024.10.22 02:05:29.896836 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_783_783_0 to 202410_796_796_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:29.901056 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} MergedBlockOutputStream: filled checksums 202410_1_643_131 (state Temporary) 2024.10.22 02:05:29.929063 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d327fc854ddd7c3e71892edd0cfb728f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d327fc854ddd7c3e71892edd0cfb728f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:05:29.948051 [ 1275 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_791_180 to 202410_792_792_0 2024.10.22 02:05:29.936813 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 537, file segment info: File segment: [0, 33554431], key: d327fc854ddd7c3e71892edd0cfb728f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:05:29.953930 [ 1275 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:29.958541 [ 1275 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:29.965723 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:29.967574 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11930 2024.10.22 02:05:29.982953 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_792_181} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_791_180 to 202410_792_792_0 into Compact with storage Full 2024.10.22 02:05:30.080445 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1227_4, 202410_1228_1228_0, 202410_1229_1229_0, 202410_1230_1230_0, 202410_1231_1231_0, 202410_1232_1232_0] 2024.10.22 02:05:30.079727 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_792_181} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:30.093206 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1227_4, 202410_1228_1228_0, 202410_1229_1229_0, 202410_1230_1230_0, 202410_1231_1231_0, 202410_1232_1232_0] 2024.10.22 02:05:30.117774 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_792_181} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_791_180, total 11965 rows starting from the beginning of the part 2024.10.22 02:05:30.130662 [ 1708 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tnd/eeivugcdrnnwdgqdkidofnkbejvck, size 1217502 2024.10.22 02:05:30.165267 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_792_181} MergeTreeSequentialSource: Reading 2 marks from part 202410_792_792_0, total 8 rows starting from the beginning of the part 2024.10.22 02:05:30.269738 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 02:05:30.283894 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 02:05:30.311267 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 02:05:30.332789 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1227_4, 202410_1228_1228_0, 202410_1229_1229_0, 202410_1230_1230_0, 202410_1231_1231_0, 202410_1232_1232_0] 2024.10.22 02:05:30.348177 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 02:05:30.374433 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 02:05:30.440969 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 02:05:30.461392 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SearchEngineID 2024.10.22 02:05:30.536263 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gbz/aqlozhxptfyzxgzlqadwdwtkhrecw (LogSeriesLimiter: on interval from 2024-10-22 02:05:11 to 2024-10-22 02:05:30 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:05:30.752475 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_643_131 to 202410_1_643_131 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:30.755659 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_643_131} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_638_130, 202410_643_643_0] -> 202410_1_643_131 2024.10.22 02:05:30.867482 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:05:30.872053 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1082 entries to flush up to offset 1134300 2024.10.22 02:05:30.977281 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1f61057a37dcfcdba11f79f91b5d0dea) : [0, 52813]: Setting download as failed: File segment: [0, 52813], key: 1f61057a37dcfcdba11f79f91b5d0dea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:30.983312 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1f61057a37dcfcdba11f79f91b5d0dea): No space left in cache to reserve 52814 bytes, will continue without cache download 2024.10.22 02:05:31.125609 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1f61057a37dcfcdba11f79f91b5d0dea, offset: 0, size: 0 2024.10.22 02:05:31.144540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1f61057a37dcfcdba11f79f91b5d0dea for removal 2024.10.22 02:05:31.148358 [ 1476 ] {} CacheMetadata: Key 1f61057a37dcfcdba11f79f91b5d0dea is removed from metadata 2024.10.22 02:05:31.149083 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 97a14235a735539692678f9cb7593efe, offset: 0, size: 0 2024.10.22 02:05:31.151140 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 97a14235a735539692678f9cb7593efe for removal 2024.10.22 02:05:31.154901 [ 1476 ] {} CacheMetadata: Key 97a14235a735539692678f9cb7593efe is removed from metadata 2024.10.22 02:05:31.166221 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 6350 2024.10.22 02:05:31.180260 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:05:31.183857 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:05:31.388090 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 87105 2024.10.22 02:05:31.539879 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_792_181} MergeTask::MergeProjectionsStage: Merge sorted 11973 rows, containing 12 columns (12 merged, 0 gathered) in 1.574873398 sec., 7602.515869024794 rows/sec., 1.18 MiB/sec. 2024.10.22 02:05:31.739466 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:31.743587 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:31.832007 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_792_181} MergedBlockOutputStream: filled checksums 202410_1_792_181 (state Temporary) 2024.10.22 02:05:31.865518 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:31.868263 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:32.059037 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7171410ac62cbc336e98b23379bb481a) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 7171410ac62cbc336e98b23379bb481a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:32.059293 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1321_1321_0 (state Temporary) 2024.10.22 02:05:32.061437 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7171410ac62cbc336e98b23379bb481a): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:05:32.120809 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1321_1321_0 to 202410_1334_1334_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:32.165403 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_750_750_0 (state Temporary) 2024.10.22 02:05:32.171357 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_792_181} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_792_181 to 202410_1_792_181 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:32.179072 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_792_181} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_791_180, 202410_792_792_0] -> 202410_1_792_181 2024.10.22 02:05:32.253069 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_750_750_0 to 202410_757_757_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:32.307996 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87105 2024.10.22 02:05:32.213928 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_695_695_0, 202410_696_696_0, 202410_697_697_0, 202410_698_698_0, 202410_699_699_0, 202410_700_700_0] 2024.10.22 02:05:32.343711 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_695_695_0, 202410_696_696_0, 202410_697_697_0, 202410_698_698_0, 202410_699_699_0, 202410_700_700_0] 2024.10.22 02:05:32.352028 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.51 MiB. 2024.10.22 02:05:32.365129 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_695_695_0, 202410_696_696_0, 202410_697_697_0, 202410_698_698_0, 202410_699_699_0, 202410_700_700_0] 2024.10.22 02:05:32.495614 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1134300 2024.10.22 02:05:32.574781 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 7.53 MiB, will set to 1.57 GiB (RSS), difference: 9.54 MiB 2024.10.22 02:05:32.618210 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_712_58, 202410_713_713_0] 2024.10.22 02:05:32.628187 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_712_58, 202410_713_713_0] 2024.10.22 02:05:32.637724 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_712_58, 202410_713_713_0] 2024.10.22 02:05:32.788049 [ 1182 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:05:32.806581 [ 1182 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:05:32.983918 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(37517bb36bf387e9cf2b603a565905e3) : [0, 49881]: Setting download as failed: File segment: [0, 49881], key: 37517bb36bf387e9cf2b603a565905e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:32.985338 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(37517bb36bf387e9cf2b603a565905e3): No space left in cache to reserve 49882 bytes, will continue without cache download 2024.10.22 02:05:33.116727 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 37517bb36bf387e9cf2b603a565905e3, offset: 0, size: 0 2024.10.22 02:05:33.119850 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 37517bb36bf387e9cf2b603a565905e3 for removal 2024.10.22 02:05:33.126325 [ 1476 ] {} CacheMetadata: Key 37517bb36bf387e9cf2b603a565905e3 is removed from metadata 2024.10.22 02:05:33.235945 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7171410ac62cbc336e98b23379bb481a, offset: 0, size: 0 2024.10.22 02:05:33.243243 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7171410ac62cbc336e98b23379bb481a for removal 2024.10.22 02:05:33.248101 [ 1476 ] {} CacheMetadata: Key 7171410ac62cbc336e98b23379bb481a is removed from metadata 2024.10.22 02:05:33.374784 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 652 2024.10.22 02:05:33.434911 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_732_151, 202410_733_733_0, 202410_734_734_0, 202410_735_735_0, 202410_736_736_0, 202410_737_737_0] 2024.10.22 02:05:33.437702 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_732_151, 202410_733_733_0, 202410_734_734_0, 202410_735_735_0, 202410_736_736_0, 202410_737_737_0] 2024.10.22 02:05:33.456836 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_732_151, 202410_733_733_0, 202410_734_734_0, 202410_735_735_0, 202410_736_736_0, 202410_737_737_0] 2024.10.22 02:05:33.636290 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:33.639745 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:33.673118 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(22c8d0b56e1492a61d1de4ebd63ca86f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 22c8d0b56e1492a61d1de4ebd63ca86f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:05:33.690545 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1169, file segment info: File segment: [0, 33554431], key: 22c8d0b56e1492a61d1de4ebd63ca86f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:05:33.697319 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:33.891253 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_634_634_0 (state Temporary) 2024.10.22 02:05:33.966937 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_634_634_0 to 202410_644_644_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:34.071558 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_748_142} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_748_142 to 202410_1_748_142 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:34.079519 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_748_142} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_742_141, 202410_743_748_1] -> 202410_1_748_142 2024.10.22 02:05:34.113285 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:05:34.404703 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 652 2024.10.22 02:05:34.665241 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b671373f367216e7ecc38f500768e36f) : [0, 153]: Setting download as failed: File segment: [0, 153], key: b671373f367216e7ecc38f500768e36f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:34.667378 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b671373f367216e7ecc38f500768e36f): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:05:35.880770 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(7dadebb03892dc885f458696db38bfa0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7dadebb03892dc885f458696db38bfa0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:05:35.912125 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 192, file segment info: File segment: [0, 33554431], key: 7dadebb03892dc885f458696db38bfa0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:05:35.916218 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:36.272732 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 86582 2024.10.22 02:05:36.324137 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 87161 2024.10.22 02:05:36.696381 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:36.701578 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:36.717710 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:36.723562 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:36.762955 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1322_1322_0 (state Temporary) 2024.10.22 02:05:36.868990 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_770_770_0 (state Temporary) 2024.10.22 02:05:36.870202 [ 1194 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 216 ms. 2024.10.22 02:05:37.011267 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1322_1322_0 to 202410_1335_1335_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:37.072341 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87161 2024.10.22 02:05:37.164861 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f2510b256f9e4ee32e0b3ffaadad3627) : [0, 30274]: Setting download as failed: File segment: [0, 30274], key: f2510b256f9e4ee32e0b3ffaadad3627, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:37.166124 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f2510b256f9e4ee32e0b3ffaadad3627): No space left in cache to reserve 30275 bytes, will continue without cache download 2024.10.22 02:05:37.196295 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f2510b256f9e4ee32e0b3ffaadad3627, offset: 0, size: 0 2024.10.22 02:05:37.200630 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f2510b256f9e4ee32e0b3ffaadad3627 for removal 2024.10.22 02:05:37.203543 [ 1476 ] {} CacheMetadata: Key f2510b256f9e4ee32e0b3ffaadad3627 is removed from metadata 2024.10.22 02:05:37.204549 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b671373f367216e7ecc38f500768e36f, offset: 0, size: 0 2024.10.22 02:05:37.273133 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_770_770_0 to 202410_776_776_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:37.308110 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b671373f367216e7ecc38f500768e36f for removal 2024.10.22 02:05:37.312607 [ 1476 ] {} CacheMetadata: Key b671373f367216e7ecc38f500768e36f is removed from metadata 2024.10.22 02:05:37.339349 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 86582 2024.10.22 02:05:37.468961 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 11949 2024.10.22 02:05:37.823773 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_155_159_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:05:37.855285 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(99693e6b820f7023bc1470827234a386) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 99693e6b820f7023bc1470827234a386, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:05:37.857269 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 535, file segment info: File segment: [0, 33554431], key: 99693e6b820f7023bc1470827234a386, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:05:37.859536 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:37.894382 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:37.909527 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:37.957440 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_784_784_0 (state Temporary) 2024.10.22 02:05:38.107244 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_784_784_0 to 202410_797_797_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:38.153664 [ 1190 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_792_181 to 202410_793_793_0 2024.10.22 02:05:38.181570 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11949 2024.10.22 02:05:38.258203 [ 1190 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:38.260441 [ 1190 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:38.290924 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_793_182} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_792_181 to 202410_793_793_0 into Compact with storage Full 2024.10.22 02:05:38.323306 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_793_182} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:38.344155 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_793_182} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_792_181, total 11973 rows starting from the beginning of the part 2024.10.22 02:05:38.363709 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_793_182} MergeTreeSequentialSource: Reading 2 marks from part 202410_793_793_0, total 15 rows starting from the beginning of the part 2024.10.22 02:05:38.594841 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6a42324561d458dd077b66ca25c65250) : [0, 164]: Setting download as failed: File segment: [0, 164], key: 6a42324561d458dd077b66ca25c65250, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:38.596616 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6a42324561d458dd077b66ca25c65250): No space left in cache to reserve 165 bytes, will continue without cache download 2024.10.22 02:05:39.143079 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_793_182} MergeTask::MergeProjectionsStage: Merge sorted 11988 rows, containing 12 columns (12 merged, 0 gathered) in 0.863980421 sec., 13875.31442683005 rows/sec., 2.16 MiB/sec. 2024.10.22 02:05:39.253166 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(c8cf284c1c7a182aaae198c49e765034) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c8cf284c1c7a182aaae198c49e765034, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:05:39.275781 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 441, file segment info: File segment: [0, 33554431], key: c8cf284c1c7a182aaae198c49e765034, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:05:39.279456 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:39.327288 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_793_182} MergedBlockOutputStream: filled checksums 202410_1_793_182 (state Temporary) 2024.10.22 02:05:39.499005 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1135455 2024.10.22 02:05:39.647790 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:05:39.659419 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_793_182} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_793_182 to 202410_1_793_182 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:39.659666 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:05:39.661089 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_793_182} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_792_181, 202410_793_793_0] -> 202410_1_793_182 2024.10.22 02:05:39.696908 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.51 MiB. 2024.10.22 02:05:39.767872 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:05:39.782559 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:05:39.813242 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:05:39.861832 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:05:39.907861 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:05:40.045745 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:05:40.085174 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/idh/wtjdhlvzkwbhrdycquoedqhanujtp (LogSeriesLimiter: on interval from 2024-10-22 02:05:30 to 2024-10-22 02:05:40 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:05:40.231530 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:40.235850 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:40.354411 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e2cbc561728dcbccc0f7495abf0713dd) : [0, 105681]: Setting download as failed: File segment: [0, 105681], key: e2cbc561728dcbccc0f7495abf0713dd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:40.356067 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e2cbc561728dcbccc0f7495abf0713dd): No space left in cache to reserve 105682 bytes, will continue without cache download 2024.10.22 02:05:40.460273 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_751_751_0 (state Temporary) 2024.10.22 02:05:40.368103 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e2cbc561728dcbccc0f7495abf0713dd, offset: 0, size: 0 2024.10.22 02:05:40.481251 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e2cbc561728dcbccc0f7495abf0713dd for removal 2024.10.22 02:05:40.490793 [ 1476 ] {} CacheMetadata: Key e2cbc561728dcbccc0f7495abf0713dd is removed from metadata 2024.10.22 02:05:40.492476 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6a42324561d458dd077b66ca25c65250, offset: 0, size: 0 2024.10.22 02:05:40.494496 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6a42324561d458dd077b66ca25c65250 for removal 2024.10.22 02:05:40.500460 [ 1476 ] {} CacheMetadata: Key 6a42324561d458dd077b66ca25c65250 is removed from metadata 2024.10.22 02:05:40.527679 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_713_59, 202410_714_714_0] 2024.10.22 02:05:40.533081 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_713_59, 202410_714_714_0] 2024.10.22 02:05:40.546891 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_751_751_0 to 202410_758_758_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:40.549302 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_713_59, 202410_714_714_0] 2024.10.22 02:05:40.585376 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1135455 2024.10.22 02:05:40.592114 [ 1189 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_748_142 to 202410_749_754_1 2024.10.22 02:05:40.600978 [ 1189 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:40.603350 [ 1189 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:40.629280 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_754_143} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_748_142 to 202410_749_754_1 into Wide with storage Full 2024.10.22 02:05:40.674622 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_754_143} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:40.694768 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_754_143} MergeTreeSequentialSource: Reading 140 marks from part 202410_1_748_142, total 1136095 rows starting from the beginning of the part 2024.10.22 02:05:40.772720 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_754_143} MergeTreeSequentialSource: Reading 3 marks from part 202410_749_754_1, total 8715 rows starting from the beginning of the part 2024.10.22 02:05:41.111243 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 87241 2024.10.22 02:05:41.179725 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:05:41.184868 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:05:41.318856 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1eace9889b3ce7188f47967101bde75a) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 1eace9889b3ce7188f47967101bde75a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:41.320957 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1eace9889b3ce7188f47967101bde75a): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:05:41.592690 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:41.596859 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:41.672104 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1323_1323_0 (state Temporary) 2024.10.22 02:05:41.758122 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1323_1323_0 to 202410_1336_1336_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:41.819384 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87241 2024.10.22 02:05:41.907966 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 653 2024.10.22 02:05:42.131153 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2b376616cf05f01ed72cf1bfb1e532c5) : [0, 2106]: Setting download as failed: File segment: [0, 2106], key: 2b376616cf05f01ed72cf1bfb1e532c5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:42.133248 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2b376616cf05f01ed72cf1bfb1e532c5): No space left in cache to reserve 2107 bytes, will continue without cache download 2024.10.22 02:05:42.154920 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2b376616cf05f01ed72cf1bfb1e532c5, offset: 0, size: 0 2024.10.22 02:05:42.167026 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2b376616cf05f01ed72cf1bfb1e532c5 for removal 2024.10.22 02:05:42.172673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1eace9889b3ce7188f47967101bde75a, offset: 0, size: 0 2024.10.22 02:05:42.174664 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1eace9889b3ce7188f47967101bde75a for removal 2024.10.22 02:05:42.295626 [ 1476 ] {} CacheMetadata: Key 2b376616cf05f01ed72cf1bfb1e532c5 is removed from metadata 2024.10.22 02:05:42.300653 [ 1476 ] {} CacheMetadata: Key 1eace9889b3ce7188f47967101bde75a is removed from metadata 2024.10.22 02:05:42.599753 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:42.607336 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:42.666808 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_635_635_0 (state Temporary) 2024.10.22 02:05:42.839473 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 5.90 MiB, will set to 1.56 GiB (RSS), difference: -17.29 MiB 2024.10.22 02:05:42.843269 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_635_635_0 to 202410_645_645_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:43.007414 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 653 2024.10.22 02:05:43.229441 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(462e841fe4fe7fea82f80aa5a1c432cf) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 462e841fe4fe7fea82f80aa5a1c432cf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:43.236768 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(462e841fe4fe7fea82f80aa5a1c432cf): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:05:43.253663 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 02:05:44.056099 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bf49375871b90e4e2de7cc325a6d5a15) : [0, 2372]: Setting download as failed: File segment: [0, 2372], key: bf49375871b90e4e2de7cc325a6d5a15, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:44.068173 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bf49375871b90e4e2de7cc325a6d5a15): No space left in cache to reserve 2373 bytes, will continue without cache download 2024.10.22 02:05:44.084995 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bf49375871b90e4e2de7cc325a6d5a15, offset: 0, size: 0 2024.10.22 02:05:44.090158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bf49375871b90e4e2de7cc325a6d5a15 for removal 2024.10.22 02:05:44.095759 [ 1476 ] {} CacheMetadata: Key bf49375871b90e4e2de7cc325a6d5a15 is removed from metadata 2024.10.22 02:05:44.096949 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 462e841fe4fe7fea82f80aa5a1c432cf, offset: 0, size: 0 2024.10.22 02:05:44.099425 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 462e841fe4fe7fea82f80aa5a1c432cf for removal 2024.10.22 02:05:44.107793 [ 1476 ] {} CacheMetadata: Key 462e841fe4fe7fea82f80aa5a1c432cf is removed from metadata 2024.10.22 02:05:44.752315 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_754_143/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:05:44.848429 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 93 entries to flush up to offset 86675 2024.10.22 02:05:45.127443 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:45.135132 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:45.193677 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_771_771_0 (state Temporary) 2024.10.22 02:05:45.370247 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 10.71 MiB, will set to 1.57 GiB (RSS), difference: 5.99 MiB 2024.10.22 02:05:45.465827 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d0488ee78074b787905da69ed5e0ce7d) : [0, 153]: Setting download as failed: File segment: [0, 153], key: d0488ee78074b787905da69ed5e0ce7d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:45.468024 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d0488ee78074b787905da69ed5e0ce7d): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:05:45.515971 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_771_771_0 to 202410_777_777_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:45.543367 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_772_5 to 202410_777_777_0 2024.10.22 02:05:45.551519 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:45.553377 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:45.578481 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_772_5 to 202410_777_777_0 into Compact with storage Full 2024.10.22 02:05:45.592741 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 86675 2024.10.22 02:05:45.656876 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:45.668140 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_772_5, total 2859 rows starting from the beginning of the part 2024.10.22 02:05:45.756457 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 11957 2024.10.22 02:05:45.809189 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_773_773_0, total 112 rows starting from the beginning of the part 2024.10.22 02:05:45.846348 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 87295 2024.10.22 02:05:45.852417 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_774_774_0, total 79 rows starting from the beginning of the part 2024.10.22 02:05:45.885179 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_775_775_0, total 106 rows starting from the beginning of the part 2024.10.22 02:05:46.026305 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_776_776_0, total 71 rows starting from the beginning of the part 2024.10.22 02:05:46.059837 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_777_777_0, total 93 rows starting from the beginning of the part 2024.10.22 02:05:46.133263 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:46.139860 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:46.274103 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_785_785_0 (state Temporary) 2024.10.22 02:05:46.296151 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:46.300093 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:46.368250 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1324_1324_0 (state Temporary) 2024.10.22 02:05:46.399437 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_785_785_0 to 202410_798_798_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:46.408584 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.58 MiB, will set to 1.57 GiB (RSS), difference: -6.37 MiB 2024.10.22 02:05:46.445024 [ 1239 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_793_182 to 202410_794_794_0 2024.10.22 02:05:46.449286 [ 1239 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:46.453621 [ 1239 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:46.468090 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11957 2024.10.22 02:05:46.471896 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7a061a8ff37a9cc1aa65dad78e09b8f1) : [0, 27022]: Setting download as failed: File segment: [0, 27022], key: 7a061a8ff37a9cc1aa65dad78e09b8f1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:46.473083 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7a061a8ff37a9cc1aa65dad78e09b8f1): No space left in cache to reserve 27023 bytes, will continue without cache download 2024.10.22 02:05:46.498570 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7a061a8ff37a9cc1aa65dad78e09b8f1, offset: 0, size: 0 2024.10.22 02:05:46.515352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7a061a8ff37a9cc1aa65dad78e09b8f1 for removal 2024.10.22 02:05:46.520872 [ 1476 ] {} CacheMetadata: Key 7a061a8ff37a9cc1aa65dad78e09b8f1 is removed from metadata 2024.10.22 02:05:46.533671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d0488ee78074b787905da69ed5e0ce7d, offset: 0, size: 0 2024.10.22 02:05:46.535567 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1324_1324_0 to 202410_1337_1337_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:46.539090 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d0488ee78074b787905da69ed5e0ce7d for removal 2024.10.22 02:05:46.543051 [ 1476 ] {} CacheMetadata: Key d0488ee78074b787905da69ed5e0ce7d is removed from metadata 2024.10.22 02:05:46.575366 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_794_183} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_793_182 to 202410_794_794_0 into Compact with storage Full 2024.10.22 02:05:46.621116 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87295 2024.10.22 02:05:46.659567 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_794_183} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:46.668916 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_794_183} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_793_182, total 11988 rows starting from the beginning of the part 2024.10.22 02:05:46.700121 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1332_25 to 202410_1337_1337_0 2024.10.22 02:05:46.717620 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_794_183} MergeTreeSequentialSource: Reading 2 marks from part 202410_794_794_0, total 23 rows starting from the beginning of the part 2024.10.22 02:05:46.722741 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:46.736238 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:46.816073 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1332_25 to 202410_1337_1337_0 into Compact with storage Full 2024.10.22 02:05:47.014439 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:47.039346 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1332_25, total 7790 rows starting from the beginning of the part 2024.10.22 02:05:47.067678 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_1333_1333_0, total 89 rows starting from the beginning of the part 2024.10.22 02:05:47.144090 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_1334_1334_0, total 85 rows starting from the beginning of the part 2024.10.22 02:05:47.186912 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_1335_1335_0, total 56 rows starting from the beginning of the part 2024.10.22 02:05:47.329727 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5e897ffa283d283f579ff450130a42c9) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 5e897ffa283d283f579ff450130a42c9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:47.440865 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5e897ffa283d283f579ff450130a42c9): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:05:47.441676 [ 1256 ] {} DatabaseCatalog: Execution took 292 ms. 2024.10.22 02:05:47.442452 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_1336_1336_0, total 80 rows starting from the beginning of the part 2024.10.22 02:05:47.452389 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} MergeTask::MergeProjectionsStage: Merge sorted 3320 rows, containing 13 columns (13 merged, 0 gathered) in 1.881196475 sec., 1764.8342659157915 rows/sec., 785.61 KiB/sec. 2024.10.22 02:05:47.516838 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} MergedBlockOutputStream: filled checksums 202410_747_777_6 (state Temporary) 2024.10.22 02:05:47.594579 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1136995 2024.10.22 02:05:47.604053 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_1337_1337_0, total 54 rows starting from the beginning of the part 2024.10.22 02:05:47.803035 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:47.806639 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:47.813905 [ 1268 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:05:47.917737 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_777_6 to 202410_747_777_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:47.928652 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_777_6} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_772_5, 202410_777_777_0] -> 202410_747_777_6 2024.10.22 02:05:47.936374 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_794_183} MergeTask::MergeProjectionsStage: Merge sorted 12011 rows, containing 12 columns (12 merged, 0 gathered) in 1.367447924 sec., 8783.51547374904 rows/sec., 1.37 MiB/sec. 2024.10.22 02:05:47.961567 [ 1268 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:05:48.047633 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.74 MiB. 2024.10.22 02:05:48.392386 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_794_183} MergedBlockOutputStream: filled checksums 202410_1_794_183 (state Temporary) 2024.10.22 02:05:48.446975 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:48.451058 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:05:48.466563 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(2c1082dd879cb370601b0886a4fed734) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2c1082dd879cb370601b0886a4fed734, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:05:48.481068 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 459400, file segment info: File segment: [0, 33554431], key: 2c1082dd879cb370601b0886a4fed734, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:05:48.488582 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:48.545029 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.73 MiB, will set to 1.57 GiB (RSS), difference: -2.49 MiB 2024.10.22 02:05:48.566394 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_752_752_0 (state Temporary) 2024.10.22 02:05:48.805815 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_752_752_0 to 202410_759_759_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:48.841448 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_794_183} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_794_183 to 202410_1_794_183 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:48.844856 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1136995 2024.10.22 02:05:48.846332 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_794_183} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_793_182, 202410_794_794_0] -> 202410_1_794_183 2024.10.22 02:05:48.872264 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.51 MiB. 2024.10.22 02:05:48.983811 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_688_132, 202410_689_694_1] 2024.10.22 02:05:48.989412 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_688_132, 202410_689_694_1] 2024.10.22 02:05:49.047800 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_688_132, 202410_689_694_1] 2024.10.22 02:05:49.231645 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_714_60, 202410_715_715_0] 2024.10.22 02:05:49.235105 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_714_60, 202410_715_715_0] 2024.10.22 02:05:49.250429 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_714_60, 202410_715_715_0] 2024.10.22 02:05:49.307096 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} MergeTask::MergeProjectionsStage: Merge sorted 8154 rows, containing 13 columns (13 merged, 0 gathered) in 2.527906592 sec., 3225.5938671961817 rows/sec., 567.22 KiB/sec. 2024.10.22 02:05:49.379330 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} MergedBlockOutputStream: filled checksums 202410_1207_1337_26 (state Temporary) 2024.10.22 02:05:49.459772 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c4a4f5b3811695b0f061bd9c499b0776) : [0, 26575]: Setting download as failed: File segment: [0, 26575], key: c4a4f5b3811695b0f061bd9c499b0776, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:49.461557 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c4a4f5b3811695b0f061bd9c499b0776): No space left in cache to reserve 26576 bytes, will continue without cache download 2024.10.22 02:05:49.478313 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c4a4f5b3811695b0f061bd9c499b0776, offset: 0, size: 0 2024.10.22 02:05:49.502515 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c4a4f5b3811695b0f061bd9c499b0776 for removal 2024.10.22 02:05:49.509424 [ 1476 ] {} CacheMetadata: Key c4a4f5b3811695b0f061bd9c499b0776 is removed from metadata 2024.10.22 02:05:49.539812 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1337_26 to 202410_1207_1337_26 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:49.543120 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1337_26} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1332_25, 202410_1337_1337_0] -> 202410_1207_1337_26 2024.10.22 02:05:49.613001 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5e897ffa283d283f579ff450130a42c9, offset: 0, size: 0 2024.10.22 02:05:49.616702 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5e897ffa283d283f579ff450130a42c9 for removal 2024.10.22 02:05:49.621109 [ 1476 ] {} CacheMetadata: Key 5e897ffa283d283f579ff450130a42c9 is removed from metadata 2024.10.22 02:05:49.801202 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.65 MiB. 2024.10.22 02:05:50.102281 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(5940ca0b91069767a5f6032a640f88e4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5940ca0b91069767a5f6032a640f88e4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:05:50.116730 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 661, file segment info: File segment: [0, 33554431], key: 5940ca0b91069767a5f6032a640f88e4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:05:50.123626 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:50.165467 [ 1743 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gbz/aqlozhxptfyzxgzlqadwdwtkhrecw, size 459400 2024.10.22 02:05:50.647579 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 87387 2024.10.22 02:05:50.702170 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_158_158_0 (state Temporary) 2024.10.22 02:05:50.725633 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 02:05:50.748459 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 02:05:50.760687 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 02:05:50.790918 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 02:05:50.833439 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 02:05:50.864027 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 02:05:50.865120 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:50.869199 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:50.883391 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SearchPhrase 2024.10.22 02:05:50.966075 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kxm/xfgjriapapymbopnkmkpmjyknyzwa (LogSeriesLimiter: on interval from 2024-10-22 02:05:40 to 2024-10-22 02:05:50 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:05:51.049510 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(581cfcfacf3275fef600dda8e72786f8) : [0, 145]: Setting download as failed: File segment: [0, 145], key: 581cfcfacf3275fef600dda8e72786f8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:51.054432 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(581cfcfacf3275fef600dda8e72786f8): No space left in cache to reserve 146 bytes, will continue without cache download 2024.10.22 02:05:51.059593 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1325_1325_0 (state Temporary) 2024.10.22 02:05:51.144843 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1325_1325_0 to 202410_1338_1338_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:51.180965 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:05:51.191876 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:05:51.216543 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87387 2024.10.22 02:05:51.296452 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1232_5, 202410_1233_1233_0, 202410_1234_1234_0, 202410_1235_1235_0, 202410_1236_1236_0, 202410_1237_1237_0] 2024.10.22 02:05:51.299569 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1232_5, 202410_1233_1233_0, 202410_1234_1234_0, 202410_1235_1235_0, 202410_1236_1236_0, 202410_1237_1237_0] 2024.10.22 02:05:51.396915 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.25 MiB, will set to 1.57 GiB (RSS), difference: 7.87 MiB 2024.10.22 02:05:51.530963 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1232_5, 202410_1233_1233_0, 202410_1234_1234_0, 202410_1235_1235_0, 202410_1236_1236_0, 202410_1237_1237_0] 2024.10.22 02:05:52.161381 [ 1223 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 319 ms. 2024.10.22 02:05:52.172372 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b6214438f5ffe1c8138639de59eda9ea) : [0, 183491]: Setting download as failed: File segment: [0, 183491], key: b6214438f5ffe1c8138639de59eda9ea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:52.174834 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b6214438f5ffe1c8138639de59eda9ea): No space left in cache to reserve 183492 bytes, will continue without cache download 2024.10.22 02:05:52.218478 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b6214438f5ffe1c8138639de59eda9ea, offset: 0, size: 0 2024.10.22 02:05:52.237268 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b6214438f5ffe1c8138639de59eda9ea for removal 2024.10.22 02:05:52.241684 [ 1476 ] {} CacheMetadata: Key b6214438f5ffe1c8138639de59eda9ea is removed from metadata 2024.10.22 02:05:52.243519 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 581cfcfacf3275fef600dda8e72786f8, offset: 0, size: 0 2024.10.22 02:05:52.245341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 581cfcfacf3275fef600dda8e72786f8 for removal 2024.10.22 02:05:52.248199 [ 1476 ] {} CacheMetadata: Key 581cfcfacf3275fef600dda8e72786f8 is removed from metadata 2024.10.22 02:05:53.104454 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 86790 2024.10.22 02:05:53.467657 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:53.470853 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5264dae2df3277390b89e042a2054310) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 5264dae2df3277390b89e042a2054310, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:53.472958 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5264dae2df3277390b89e042a2054310): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 02:05:53.469789 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:53.702332 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_155_159_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:05:53.724091 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_772_772_0 (state Temporary) 2024.10.22 02:05:53.764281 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_772_772_0 to 202410_778_778_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:53.909634 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 86790 2024.10.22 02:05:53.975768 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 11979 2024.10.22 02:05:54.071012 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:54.074967 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:54.143309 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_786_786_0 (state Temporary) 2024.10.22 02:05:54.221303 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_786_786_0 to 202410_799_799_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:54.260648 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11979 2024.10.22 02:05:54.351746 [ 1159 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_794_183 to 202410_795_795_0 2024.10.22 02:05:54.419654 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 11.10 MiB, will set to 1.57 GiB (RSS), difference: -2.74 MiB 2024.10.22 02:05:54.452135 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(582962dcf06d216a289bef01c51b8280) : [0, 128696]: Setting download as failed: File segment: [0, 128696], key: 582962dcf06d216a289bef01c51b8280, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:54.453258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(582962dcf06d216a289bef01c51b8280): No space left in cache to reserve 128697 bytes, will continue without cache download 2024.10.22 02:05:54.456873 [ 1159 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:54.465156 [ 1159 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:54.467678 [ 1159 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 02:05:54.485537 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 582962dcf06d216a289bef01c51b8280, offset: 0, size: 0 2024.10.22 02:05:54.490761 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 582962dcf06d216a289bef01c51b8280 for removal 2024.10.22 02:05:54.498463 [ 1476 ] {} CacheMetadata: Key 582962dcf06d216a289bef01c51b8280 is removed from metadata 2024.10.22 02:05:54.500534 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_795_184} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_794_183 to 202410_795_795_0 into Compact with storage Full 2024.10.22 02:05:54.502853 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5264dae2df3277390b89e042a2054310, offset: 0, size: 0 2024.10.22 02:05:54.506782 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5264dae2df3277390b89e042a2054310 for removal 2024.10.22 02:05:54.510296 [ 1476 ] {} CacheMetadata: Key 5264dae2df3277390b89e042a2054310 is removed from metadata 2024.10.22 02:05:54.649687 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_795_184} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:54.655276 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_795_184} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_794_183, total 12011 rows starting from the beginning of the part 2024.10.22 02:05:54.669213 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_795_184} MergeTreeSequentialSource: Reading 2 marks from part 202410_795_795_0, total 9 rows starting from the beginning of the part 2024.10.22 02:05:55.245056 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 87451 2024.10.22 02:05:55.569749 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:55.600104 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:55.689545 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1326_1326_0 (state Temporary) 2024.10.22 02:05:55.700174 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e062c8d7d21165587e4b28be6ffbf001) : [0, 164]: Setting download as failed: File segment: [0, 164], key: e062c8d7d21165587e4b28be6ffbf001, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:55.731556 [ 1237 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 02:05:55.747197 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1326_1326_0 to 202410_1339_1339_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:55.701840 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e062c8d7d21165587e4b28be6ffbf001): No space left in cache to reserve 165 bytes, will continue without cache download 2024.10.22 02:05:55.846106 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_795_184} MergeTask::MergeProjectionsStage: Merge sorted 12020 rows, containing 12 columns (12 merged, 0 gathered) in 1.358321231 sec., 8849.15859788987 rows/sec., 1.38 MiB/sec. 2024.10.22 02:05:55.851911 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1138150 2024.10.22 02:05:55.906667 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87451 2024.10.22 02:05:56.008681 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_754_143/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:05:56.033510 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_795_184} MergedBlockOutputStream: filled checksums 202410_1_795_184 (state Temporary) 2024.10.22 02:05:56.321527 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_795_184} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_795_184 to 202410_1_795_184 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:56.325583 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_795_184} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_794_183, 202410_795_795_0] -> 202410_1_795_184 2024.10.22 02:05:56.353369 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:05:56.489319 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.75 MiB, will set to 1.57 GiB (RSS), difference: 1.12 MiB 2024.10.22 02:05:56.768903 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:56.770978 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:56.917240 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f38ec4826117413e7862c70bce790d24) : [0, 67235]: Setting download as failed: File segment: [0, 67235], key: f38ec4826117413e7862c70bce790d24, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:56.919715 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f38ec4826117413e7862c70bce790d24): No space left in cache to reserve 67236 bytes, will continue without cache download 2024.10.22 02:05:56.943135 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f38ec4826117413e7862c70bce790d24, offset: 0, size: 0 2024.10.22 02:05:56.946754 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f38ec4826117413e7862c70bce790d24 for removal 2024.10.22 02:05:56.952868 [ 1476 ] {} CacheMetadata: Key f38ec4826117413e7862c70bce790d24 is removed from metadata 2024.10.22 02:05:56.954470 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e062c8d7d21165587e4b28be6ffbf001, offset: 0, size: 0 2024.10.22 02:05:56.957289 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e062c8d7d21165587e4b28be6ffbf001 for removal 2024.10.22 02:05:56.960061 [ 1476 ] {} CacheMetadata: Key e062c8d7d21165587e4b28be6ffbf001 is removed from metadata 2024.10.22 02:05:57.138814 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_753_753_0 (state Temporary) 2024.10.22 02:05:57.463107 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_753_753_0 to 202410_760_760_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:57.487242 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1138150 2024.10.22 02:05:57.480674 [ 1206 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_755_755_0 to 202410_760_760_0 2024.10.22 02:05:57.506663 [ 1206 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:57.515029 [ 1206 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:57.646284 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_755_755_0 to 202410_760_760_0 into Compact with storage Full 2024.10.22 02:05:57.659264 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:05:57.663276 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_755_755_0, total 770 rows starting from the beginning of the part 2024.10.22 02:05:57.681050 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_756_756_0, total 1228 rows starting from the beginning of the part 2024.10.22 02:05:57.715679 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_757_757_0, total 1082 rows starting from the beginning of the part 2024.10.22 02:05:57.753749 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_758_758_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:05:57.764764 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ecac160df849e752c0e0a5cf4d6b0821) : [0, 154]: Setting download as failed: File segment: [0, 154], key: ecac160df849e752c0e0a5cf4d6b0821, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:57.765924 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ecac160df849e752c0e0a5cf4d6b0821): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 02:05:57.773197 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_759_759_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:05:57.811652 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_760_760_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:05:58.011167 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 654 2024.10.22 02:05:58.435563 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(0d0cb551a02686297d9f192618b17f54) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0d0cb551a02686297d9f192618b17f54, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:05:58.436897 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 14.47 MiB, will set to 1.57 GiB (RSS), difference: -5.20 MiB 2024.10.22 02:05:58.447934 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 0d0cb551a02686297d9f192618b17f54, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:05:58.450239 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:05:58.573192 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_715_61, 202410_716_716_0] 2024.10.22 02:05:58.574839 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_715_61, 202410_716_716_0] 2024.10.22 02:05:58.597209 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:05:58.602452 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:05:58.662542 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_636_636_0 (state Temporary) 2024.10.22 02:05:58.681160 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_158_158_0 to 202410_161_161_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:58.691435 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_715_61, 202410_716_716_0] 2024.10.22 02:05:58.865405 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_636_636_0 to 202410_646_646_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:58.904280 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} MergeTask::MergeProjectionsStage: Merge sorted 6930 rows, containing 4 columns (4 merged, 0 gathered) in 1.242382393 sec., 5577.992765388458 rows/sec., 87.16 KiB/sec. 2024.10.22 02:05:58.937496 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 654 2024.10.22 02:05:59.076880 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} MergedBlockOutputStream: filled checksums 202410_755_760_1 (state Temporary) 2024.10.22 02:05:59.362682 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(87481fc2040f9a0772472d1a3a5a7764) : [0, 92884]: Setting download as failed: File segment: [0, 92884], key: 87481fc2040f9a0772472d1a3a5a7764, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:05:59.363717 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(87481fc2040f9a0772472d1a3a5a7764): No space left in cache to reserve 92885 bytes, will continue without cache download 2024.10.22 02:05:59.401713 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 87481fc2040f9a0772472d1a3a5a7764, offset: 0, size: 0 2024.10.22 02:05:59.404502 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 87481fc2040f9a0772472d1a3a5a7764 for removal 2024.10.22 02:05:59.409173 [ 1476 ] {} CacheMetadata: Key 87481fc2040f9a0772472d1a3a5a7764 is removed from metadata 2024.10.22 02:05:59.475700 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_755_760_1 to 202410_755_760_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:05:59.480795 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_755_760_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_755_755_0, 202410_760_760_0] -> 202410_755_760_1 2024.10.22 02:05:59.513443 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ecac160df849e752c0e0a5cf4d6b0821, offset: 0, size: 0 2024.10.22 02:05:59.515769 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ecac160df849e752c0e0a5cf4d6b0821 for removal 2024.10.22 02:05:59.519248 [ 1476 ] {} CacheMetadata: Key ecac160df849e752c0e0a5cf4d6b0821 is removed from metadata 2024.10.22 02:05:59.850083 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:05:59.939408 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 87521 2024.10.22 02:05:59.968961 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(158da33bc5348cfb5d4dea48b95a29ed) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 158da33bc5348cfb5d4dea48b95a29ed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:05:59.960295 [ 1191 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 02:05:59.999283 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1567, file segment info: File segment: [0, 33554431], key: 158da33bc5348cfb5d4dea48b95a29ed, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:06:00.005828 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:00.115200 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:00.124469 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:00.209329 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1327_1327_0 (state Temporary) 2024.10.22 02:06:00.274115 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1327_1327_0 to 202410_1340_1340_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:00.317683 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87521 2024.10.22 02:06:00.467722 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 16.79 MiB, will set to 1.57 GiB (RSS), difference: 17.01 MiB 2024.10.22 02:06:01.002981 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(96d11c5c2a352b05737453b5b9a634ed) : [0, 165]: Setting download as failed: File segment: [0, 165], key: 96d11c5c2a352b05737453b5b9a634ed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:01.005490 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(96d11c5c2a352b05737453b5b9a634ed): No space left in cache to reserve 166 bytes, will continue without cache download 2024.10.22 02:06:01.182042 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:06:01.164518 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6350 2024.10.22 02:06:01.288455 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:06:01.427273 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 89 entries to flush up to offset 86879 2024.10.22 02:06:01.551614 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(8bfc3182818aaf3294b4f8e6353548a5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8bfc3182818aaf3294b4f8e6353548a5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:06:01.561201 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 195, file segment info: File segment: [0, 33554431], key: 8bfc3182818aaf3294b4f8e6353548a5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:06:01.563281 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:01.650968 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:01.654595 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:01.694185 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_773_773_0 (state Temporary) 2024.10.22 02:06:01.762652 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 11995 2024.10.22 02:06:01.855203 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_773_773_0 to 202410_779_779_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:02.050885 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_593_121, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0, 202410_597_597_0, 202410_598_598_0] 2024.10.22 02:06:02.053601 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_593_121, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0, 202410_597_597_0, 202410_598_598_0] 2024.10.22 02:06:02.095958 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_593_121, 202410_594_594_0, 202410_595_595_0, 202410_596_596_0, 202410_597_597_0, 202410_598_598_0] 2024.10.22 02:06:02.144161 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 86879 2024.10.22 02:06:02.165189 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:02.167628 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:02.201366 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_787_787_0 (state Temporary) 2024.10.22 02:06:02.462490 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_787_787_0 to 202410_800_800_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:02.488145 [ 1175 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_795_184 to 202410_796_796_0 2024.10.22 02:06:02.495231 [ 1175 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:02.507166 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 11995 2024.10.22 02:06:02.522979 [ 1175 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:02.551396 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_796_185} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_795_184 to 202410_796_796_0 into Compact with storage Full 2024.10.22 02:06:02.593643 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_796_185} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:06:02.627634 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_796_185} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_795_184, total 12020 rows starting from the beginning of the part 2024.10.22 02:06:02.752160 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_796_185} MergeTreeSequentialSource: Reading 2 marks from part 202410_796_796_0, total 16 rows starting from the beginning of the part 2024.10.22 02:06:02.860176 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6caa202b7e8bcff34dab94a4351fb37c) : [0, 151384]: Setting download as failed: File segment: [0, 151384], key: 6caa202b7e8bcff34dab94a4351fb37c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:02.861762 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6caa202b7e8bcff34dab94a4351fb37c): No space left in cache to reserve 151385 bytes, will continue without cache download 2024.10.22 02:06:02.881373 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6caa202b7e8bcff34dab94a4351fb37c, offset: 0, size: 0 2024.10.22 02:06:02.883921 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6caa202b7e8bcff34dab94a4351fb37c for removal 2024.10.22 02:06:02.887046 [ 1476 ] {} CacheMetadata: Key 6caa202b7e8bcff34dab94a4351fb37c is removed from metadata 2024.10.22 02:06:02.889770 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 96d11c5c2a352b05737453b5b9a634ed, offset: 0, size: 0 2024.10.22 02:06:02.891619 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 96d11c5c2a352b05737453b5b9a634ed for removal 2024.10.22 02:06:02.898905 [ 1476 ] {} CacheMetadata: Key 96d11c5c2a352b05737453b5b9a634ed is removed from metadata 2024.10.22 02:06:02.968871 [ 1247 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:06:02.997583 [ 1247 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:06:03.508889 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 16.86 MiB, will set to 1.57 GiB (RSS), difference: -6.15 MiB 2024.10.22 02:06:03.681539 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(9cde6c63b0891173004b3f6ff09ce3e3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9cde6c63b0891173004b3f6ff09ce3e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:06:03.700448 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 535, file segment info: File segment: [0, 33554431], key: 9cde6c63b0891173004b3f6ff09ce3e3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:06:03.703549 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:03.865219 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_796_185} MergeTask::MergeProjectionsStage: Merge sorted 12036 rows, containing 12 columns (12 merged, 0 gathered) in 1.321770574 sec., 9105.967583751035 rows/sec., 1.42 MiB/sec. 2024.10.22 02:06:03.999087 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_796_185} MergedBlockOutputStream: filled checksums 202410_1_796_185 (state Temporary) 2024.10.22 02:06:04.339091 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 87578 2024.10.22 02:06:04.493422 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1139305 2024.10.22 02:06:04.639224 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e8ff7bba8a73fbd294c5a5faab42f07d) : [0, 165]: Setting download as failed: File segment: [0, 165], key: e8ff7bba8a73fbd294c5a5faab42f07d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:04.643088 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e8ff7bba8a73fbd294c5a5faab42f07d): No space left in cache to reserve 166 bytes, will continue without cache download 2024.10.22 02:06:04.668952 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:04.670291 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:04.737432 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_796_185} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_796_185 to 202410_1_796_185 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:04.747108 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_796_185} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_795_184, 202410_796_796_0] -> 202410_1_796_185 2024.10.22 02:06:04.753240 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1328_1328_0 (state Temporary) 2024.10.22 02:06:04.772669 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:06:04.891336 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_155_159_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:06:04.950961 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1328_1328_0 to 202410_1341_1341_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:04.984385 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87578 2024.10.22 02:06:05.351774 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:05.356188 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:05.477279 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_754_754_0 (state Temporary) 2024.10.22 02:06:05.578268 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_754_754_0 to 202410_761_761_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:05.616527 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1139305 2024.10.22 02:06:05.796333 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(c912855c504c101be42e57e4e54baa09) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c912855c504c101be42e57e4e54baa09, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:06:05.800317 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 441, file segment info: File segment: [0, 33554431], key: c912855c504c101be42e57e4e54baa09, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:06:05.805440 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:05.849642 [ 1703 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/idh/wtjdhlvzkwbhrdycquoedqhanujtp, size 535 2024.10.22 02:06:06.172540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5b801f7000a925bfa8acae16e89934df) : [0, 160922]: Setting download as failed: File segment: [0, 160922], key: 5b801f7000a925bfa8acae16e89934df, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:06.174734 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5b801f7000a925bfa8acae16e89934df): No space left in cache to reserve 160923 bytes, will continue without cache download 2024.10.22 02:06:06.206363 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5b801f7000a925bfa8acae16e89934df, offset: 0, size: 0 2024.10.22 02:06:06.231803 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5b801f7000a925bfa8acae16e89934df for removal 2024.10.22 02:06:06.239300 [ 1476 ] {} CacheMetadata: Key 5b801f7000a925bfa8acae16e89934df is removed from metadata 2024.10.22 02:06:06.240808 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e8ff7bba8a73fbd294c5a5faab42f07d, offset: 0, size: 0 2024.10.22 02:06:06.242565 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e8ff7bba8a73fbd294c5a5faab42f07d for removal 2024.10.22 02:06:06.244951 [ 1476 ] {} CacheMetadata: Key e8ff7bba8a73fbd294c5a5faab42f07d is removed from metadata 2024.10.22 02:06:06.355864 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:06:06.395722 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:06:06.441638 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 655 2024.10.22 02:06:06.566885 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:06:06.600912 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:06.608118 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:06.695613 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:06:06.740903 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:06:06.771406 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_637_637_0 (state Temporary) 2024.10.22 02:06:06.816324 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:06:06.868491 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:06:06.898248 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_716_62, 202410_717_717_0] 2024.10.22 02:06:06.903598 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_716_62, 202410_717_717_0] 2024.10.22 02:06:07.017723 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:06:07.127848 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_716_62, 202410_717_717_0] 2024.10.22 02:06:07.123869 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rco/aucjajhfmgeqczfldlxfybwzilnyt (LogSeriesLimiter: on interval from 2024-10-22 02:05:50 to 2024-10-22 02:06:07 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:06:07.296744 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_637_637_0 to 202410_647_647_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:07.384054 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 655 2024.10.22 02:06:07.514613 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 13.63 MiB, will set to 1.57 GiB (RSS), difference: -3.46 MiB 2024.10.22 02:06:07.575842 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b5e2b41032e1a6ce1ae3fa32c1b30920) : [0, 164]: Setting download as failed: File segment: [0, 164], key: b5e2b41032e1a6ce1ae3fa32c1b30920, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:07.592498 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b5e2b41032e1a6ce1ae3fa32c1b30920): No space left in cache to reserve 165 bytes, will continue without cache download 2024.10.22 02:06:07.749664 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_754_143/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:06:08.659236 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(55d97f8bca3575d67cdaa3c7b26f0053) : [0, 78325]: Setting download as failed: File segment: [0, 78325], key: 55d97f8bca3575d67cdaa3c7b26f0053, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:08.661338 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(55d97f8bca3575d67cdaa3c7b26f0053): No space left in cache to reserve 78326 bytes, will continue without cache download 2024.10.22 02:06:08.688990 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 55d97f8bca3575d67cdaa3c7b26f0053, offset: 0, size: 0 2024.10.22 02:06:08.695697 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 55d97f8bca3575d67cdaa3c7b26f0053 for removal 2024.10.22 02:06:08.700136 [ 1476 ] {} CacheMetadata: Key 55d97f8bca3575d67cdaa3c7b26f0053 is removed from metadata 2024.10.22 02:06:08.700922 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b5e2b41032e1a6ce1ae3fa32c1b30920, offset: 0, size: 0 2024.10.22 02:06:08.703122 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b5e2b41032e1a6ce1ae3fa32c1b30920 for removal 2024.10.22 02:06:08.707666 [ 1476 ] {} CacheMetadata: Key b5e2b41032e1a6ce1ae3fa32c1b30920 is removed from metadata 2024.10.22 02:06:08.750864 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 37 entries to flush up to offset 6387 2024.10.22 02:06:08.997680 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 87639 2024.10.22 02:06:09.435390 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:09.442928 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:09.464285 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 16.32 MiB, will set to 1.57 GiB (RSS), difference: 2.55 MiB 2024.10.22 02:06:09.657245 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 81 entries to flush up to offset 86960 2024.10.22 02:06:09.746562 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1329_1329_0 (state Temporary) 2024.10.22 02:06:09.780358 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:09.787669 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:09.798973 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1329_1329_0 to 202410_1342_1342_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:09.827948 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1337_26 to 202410_1342_1342_0 2024.10.22 02:06:09.855427 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:09.857126 [ 1266 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:09.926464 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bfb72d3ec3aeeba1689c83b650c95b99) : [0, 159]: Setting download as failed: File segment: [0, 159], key: bfb72d3ec3aeeba1689c83b650c95b99, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:09.928760 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bfb72d3ec3aeeba1689c83b650c95b99): No space left in cache to reserve 160 bytes, will continue without cache download 2024.10.22 02:06:09.937730 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_155_159_1} MergeTask::MergeProjectionsStage: Merge sorted 191 rows, containing 679 columns (679 merged, 0 gathered) in 76.828482718 sec., 2.486057165817892 rows/sec., 3.66 KiB/sec. 2024.10.22 02:06:09.950221 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87639 2024.10.22 02:06:10.005801 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1337_26 to 202410_1342_1342_0 into Compact with storage Full 2024.10.22 02:06:10.010773 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 12004 2024.10.22 02:06:10.046750 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_774_774_0 (state Temporary) 2024.10.22 02:06:10.085023 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_774_774_0 to 202410_780_780_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:10.095379 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:06:10.115506 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:10.121242 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:10.147704 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 86960 2024.10.22 02:06:10.164261 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_788_788_0 (state Temporary) 2024.10.22 02:06:10.165245 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_1207_1337_26, total 8154 rows starting from the beginning of the part 2024.10.22 02:06:10.227754 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_788_788_0 to 202410_801_801_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:10.267469 [ 1274 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_796_185 to 202410_797_797_0 2024.10.22 02:06:10.275681 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_1338_1338_0, total 92 rows starting from the beginning of the part 2024.10.22 02:06:10.288489 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12004 2024.10.22 02:06:10.372000 [ 1274 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:10.375326 [ 1274 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:10.397930 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_797_186} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_796_185 to 202410_797_797_0 into Compact with storage Full 2024.10.22 02:06:10.467474 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_1339_1339_0, total 64 rows starting from the beginning of the part 2024.10.22 02:06:10.690569 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0ce5b37feba23005de8f9236781eafc9) : [0, 1106974]: Setting download as failed: File segment: [0, 1106974], key: 0ce5b37feba23005de8f9236781eafc9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:10.692366 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0ce5b37feba23005de8f9236781eafc9): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:06:10.737665 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_797_186} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:06:10.843569 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_1340_1340_0, total 70 rows starting from the beginning of the part 2024.10.22 02:06:10.859259 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_797_186} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_796_185, total 12036 rows starting from the beginning of the part 2024.10.22 02:06:10.882412 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_797_186} MergeTreeSequentialSource: Reading 2 marks from part 202410_797_797_0, total 19 rows starting from the beginning of the part 2024.10.22 02:06:10.942995 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0ce5b37feba23005de8f9236781eafc9, offset: 0, size: 0 2024.10.22 02:06:10.945792 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0ce5b37feba23005de8f9236781eafc9 for removal 2024.10.22 02:06:10.949010 [ 1476 ] {} CacheMetadata: Key 0ce5b37feba23005de8f9236781eafc9 is removed from metadata 2024.10.22 02:06:10.951338 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bfb72d3ec3aeeba1689c83b650c95b99, offset: 0, size: 0 2024.10.22 02:06:10.953993 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bfb72d3ec3aeeba1689c83b650c95b99 for removal 2024.10.22 02:06:10.956846 [ 1476 ] {} CacheMetadata: Key bfb72d3ec3aeeba1689c83b650c95b99 is removed from metadata 2024.10.22 02:06:11.050250 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_1341_1341_0, total 57 rows starting from the beginning of the part 2024.10.22 02:06:11.194551 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:06:11.203354 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:06:11.413491 [ 1263 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 425 ms. 2024.10.22 02:06:11.534393 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_1342_1342_0, total 61 rows starting from the beginning of the part 2024.10.22 02:06:11.547407 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_155_159_1} MergedBlockOutputStream: filled checksums 202410_155_159_1 (state Temporary) 2024.10.22 02:06:11.626692 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 15.72 MiB, will set to 1.57 GiB (RSS), difference: -5.18 MiB 2024.10.22 02:06:12.102595 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0f806e93e6f4a0f3fb694a7282d2a050) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 0f806e93e6f4a0f3fb694a7282d2a050, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:12.105383 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0f806e93e6f4a0f3fb694a7282d2a050): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:06:12.128574 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_797_186} MergeTask::MergeProjectionsStage: Merge sorted 12055 rows, containing 12 columns (12 merged, 0 gathered) in 1.741583396 sec., 6921.862040995251 rows/sec., 1.08 MiB/sec. 2024.10.22 02:06:12.276947 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_797_186} MergedBlockOutputStream: filled checksums 202410_1_797_186 (state Temporary) 2024.10.22 02:06:12.622201 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1328 entries to flush up to offset 1140633 2024.10.22 02:06:12.623899 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_797_186} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_797_186 to 202410_1_797_186 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:12.630766 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_797_186} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_796_185, 202410_797_797_0] -> 202410_1_797_186 2024.10.22 02:06:12.684577 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:06:12.785668 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1342_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:06:12.866886 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_717_63, 202410_718_718_0] 2024.10.22 02:06:12.869147 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_717_63, 202410_718_718_0] 2024.10.22 02:06:12.877786 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_717_63, 202410_718_718_0] 2024.10.22 02:06:13.224323 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:06:13.308894 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(73c05fa8075b022f5520ba76a92071ba) : [0, 46529]: Setting download as failed: File segment: [0, 46529], key: 73c05fa8075b022f5520ba76a92071ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:13.312488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(73c05fa8075b022f5520ba76a92071ba): No space left in cache to reserve 46530 bytes, will continue without cache download 2024.10.22 02:06:13.394698 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} MergeTask::MergeProjectionsStage: Merge sorted 8498 rows, containing 13 columns (13 merged, 0 gathered) in 3.417191963 sec., 2486.837172746798 rows/sec., 437.81 KiB/sec. 2024.10.22 02:06:13.444347 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 73c05fa8075b022f5520ba76a92071ba, offset: 0, size: 0 2024.10.22 02:06:13.448144 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 73c05fa8075b022f5520ba76a92071ba for removal 2024.10.22 02:06:13.451207 [ 1476 ] {} CacheMetadata: Key 73c05fa8075b022f5520ba76a92071ba is removed from metadata 2024.10.22 02:06:13.452129 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0f806e93e6f4a0f3fb694a7282d2a050, offset: 0, size: 0 2024.10.22 02:06:13.455625 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0f806e93e6f4a0f3fb694a7282d2a050 for removal 2024.10.22 02:06:13.459984 [ 1476 ] {} CacheMetadata: Key 0f806e93e6f4a0f3fb694a7282d2a050 is removed from metadata 2024.10.22 02:06:13.466374 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} MergedBlockOutputStream: filled checksums 202410_1207_1342_27 (state Temporary) 2024.10.22 02:06:13.559723 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 14.01 MiB, will set to 1.57 GiB (RSS), difference: 4.35 MiB 2024.10.22 02:06:13.667632 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1342_27 to 202410_1207_1342_27 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:13.671610 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1342_27} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1337_26, 202410_1342_1342_0] -> 202410_1207_1342_27 2024.10.22 02:06:13.729390 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.76 MiB. 2024.10.22 02:06:13.883440 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:13.889283 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:13.908281 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_754_143} MergeTask::MergeProjectionsStage: Merge sorted 1144810 rows, containing 4 columns (4 merged, 0 gathered) in 33.294736741 sec., 34384.11328809972 rows/sec., 515.85 KiB/sec. 2024.10.22 02:06:13.968476 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 87722 2024.10.22 02:06:14.084237 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_754_143} MergedBlockOutputStream: filled checksums 202410_1_754_143 (state Temporary) 2024.10.22 02:06:14.163594 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(78b988b45670c07be35ef9a25b2ff9fa) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 78b988b45670c07be35ef9a25b2ff9fa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 02:06:14.166643 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1073, file segment info: File segment: [0, 33554431], key: 78b988b45670c07be35ef9a25b2ff9fa, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 02:06:14.168822 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:14.183230 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_755_755_0 (state Temporary) 2024.10.22 02:06:14.324034 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_755_755_0 to 202410_762_762_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:14.368213 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1140633 2024.10.22 02:06:14.448559 [ 1685 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kxm/xfgjriapapymbopnkmkpmjyknyzwa, size 3120927 2024.10.22 02:06:14.565260 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:14.568269 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:14.674712 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1330_1330_0 (state Temporary) 2024.10.22 02:06:14.762859 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 02:06:14.787727 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 4 old parts to remove. Parts: [202410_1_737_152, 202410_738_738_0, 202410_739_739_0, 202410_740_740_0] 2024.10.22 02:06:14.792396 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from filesystem (serially): Parts: [202410_1_737_152, 202410_738_738_0, 202410_739_739_0, 202410_740_740_0] 2024.10.22 02:06:14.815314 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 02:06:14.831440 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from memory: Parts: [202410_1_737_152, 202410_738_738_0, 202410_739_739_0, 202410_740_740_0] 2024.10.22 02:06:14.856850 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 02:06:14.882768 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 02:06:14.888591 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 656 2024.10.22 02:06:14.934573 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 02:06:15.041019 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 02:06:15.101685 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1330_1330_0 to 202410_1343_1343_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:15.114892 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column AdvEngineID 2024.10.22 02:06:15.251962 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87722 2024.10.22 02:06:15.251682 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/oom/yxkgwnytplpdmzdbugdkbglfxejow (LogSeriesLimiter: on interval from 2024-10-22 02:06:07 to 2024-10-22 02:06:15 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:06:15.408109 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:15.414141 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:15.489346 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_638_638_0 (state Temporary) 2024.10.22 02:06:15.692730 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_638_638_0 to 202410_648_648_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:15.727347 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(117fc50bef6f8077a661264b3028a304) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 117fc50bef6f8077a661264b3028a304, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:15.734848 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(117fc50bef6f8077a661264b3028a304): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:06:15.753792 [ 1188 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_643_131 to 202410_648_648_0 2024.10.22 02:06:15.868242 [ 1188 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:15.869704 [ 1188 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:15.892575 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_643_131 to 202410_648_648_0 into Compact with storage Full 2024.10.22 02:06:16.011518 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 656 2024.10.22 02:06:16.344188 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:06:16.364300 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_643_131, total 687 rows starting from the beginning of the part 2024.10.22 02:06:16.735025 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_644_644_0, total 1 rows starting from the beginning of the part 2024.10.22 02:06:16.767567 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1237_6, 202410_1238_1238_0, 202410_1239_1239_0, 202410_1240_1240_0, 202410_1241_1241_0, 202410_1242_1242_0] 2024.10.22 02:06:16.769752 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1237_6, 202410_1238_1238_0, 202410_1239_1239_0, 202410_1240_1240_0, 202410_1241_1241_0, 202410_1242_1242_0] 2024.10.22 02:06:16.802634 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1237_6, 202410_1238_1238_0, 202410_1239_1239_0, 202410_1240_1240_0, 202410_1241_1241_0, 202410_1242_1242_0] 2024.10.22 02:06:16.879372 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_645_645_0, total 1 rows starting from the beginning of the part 2024.10.22 02:06:17.055934 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b3982fd26c1c78d5a57b8cf412582c9d) : [0, 40495]: Setting download as failed: File segment: [0, 40495], key: b3982fd26c1c78d5a57b8cf412582c9d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:17.057490 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b3982fd26c1c78d5a57b8cf412582c9d): No space left in cache to reserve 40496 bytes, will continue without cache download 2024.10.22 02:06:17.147110 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_646_646_0, total 1 rows starting from the beginning of the part 2024.10.22 02:06:17.179466 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b3982fd26c1c78d5a57b8cf412582c9d, offset: 0, size: 0 2024.10.22 02:06:17.181572 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b3982fd26c1c78d5a57b8cf412582c9d for removal 2024.10.22 02:06:17.188659 [ 1476 ] {} CacheMetadata: Key b3982fd26c1c78d5a57b8cf412582c9d is removed from metadata 2024.10.22 02:06:17.204170 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_647_647_0, total 1 rows starting from the beginning of the part 2024.10.22 02:06:17.271887 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_648_648_0, total 1 rows starting from the beginning of the part 2024.10.22 02:06:17.290368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 117fc50bef6f8077a661264b3028a304, offset: 0, size: 0 2024.10.22 02:06:17.292937 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 117fc50bef6f8077a661264b3028a304 for removal 2024.10.22 02:06:17.300244 [ 1476 ] {} CacheMetadata: Key 117fc50bef6f8077a661264b3028a304 is removed from metadata 2024.10.22 02:06:17.623678 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 9.76 MiB, will set to 1.56 GiB (RSS), difference: 2.34 MiB 2024.10.22 02:06:17.657167 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 95 entries to flush up to offset 87055 2024.10.22 02:06:17.795711 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 12019 2024.10.22 02:06:17.884348 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:17.895307 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:17.958980 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:17.959712 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_754_143} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_754_143 to 202410_1_754_143 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:17.961328 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_775_775_0 (state Temporary) 2024.10.22 02:06:17.964166 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_754_143} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_748_142, 202410_749_754_1] -> 202410_1_754_143 2024.10.22 02:06:17.965091 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_754_143/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:06:17.962018 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:06:17.991746 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:06:18.013520 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_789_789_0 (state Temporary) 2024.10.22 02:06:18.024669 [ 1171 ] {} DatabaseCatalog: Execution took 334 ms. 2024.10.22 02:06:18.112349 [ 1172 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:06:18.143527 [ 1172 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:06:18.160008 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_789_789_0 to 202410_802_802_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:18.165606 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_775_775_0 to 202410_781_781_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:18.236494 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87055 2024.10.22 02:06:18.292983 [ 1190 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_797_186 to 202410_798_798_0 2024.10.22 02:06:18.305376 [ 1190 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:18.324077 [ 1190 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:18.325569 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12019 2024.10.22 02:06:18.354149 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_798_187} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_797_186 to 202410_798_798_0 into Compact with storage Full 2024.10.22 02:06:18.364809 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0d35aa3731c943daf3fa99c813f3b89f) : [0, 143]: Setting download as failed: File segment: [0, 143], key: 0d35aa3731c943daf3fa99c813f3b89f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:18.367164 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0d35aa3731c943daf3fa99c813f3b89f): No space left in cache to reserve 144 bytes, will continue without cache download 2024.10.22 02:06:18.444510 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_798_187} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:06:18.456227 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_798_187} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_797_186, total 12055 rows starting from the beginning of the part 2024.10.22 02:06:18.700466 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_798_187} MergeTreeSequentialSource: Reading 2 marks from part 202410_798_798_0, total 8 rows starting from the beginning of the part 2024.10.22 02:06:19.325551 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 87806 2024.10.22 02:06:19.479983 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9c592d52ab611b3eb51b07cdd0151e4b) : [0, 75059]: Setting download as failed: File segment: [0, 75059], key: 9c592d52ab611b3eb51b07cdd0151e4b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:19.483608 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9c592d52ab611b3eb51b07cdd0151e4b): No space left in cache to reserve 75060 bytes, will continue without cache download 2024.10.22 02:06:19.553361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9c592d52ab611b3eb51b07cdd0151e4b, offset: 0, size: 0 2024.10.22 02:06:19.555987 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9c592d52ab611b3eb51b07cdd0151e4b for removal 2024.10.22 02:06:19.563318 [ 1476 ] {} CacheMetadata: Key 9c592d52ab611b3eb51b07cdd0151e4b is removed from metadata 2024.10.22 02:06:19.563524 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0d35aa3731c943daf3fa99c813f3b89f, offset: 0, size: 0 2024.10.22 02:06:19.567100 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0d35aa3731c943daf3fa99c813f3b89f for removal 2024.10.22 02:06:19.573050 [ 1476 ] {} CacheMetadata: Key 0d35aa3731c943daf3fa99c813f3b89f is removed from metadata 2024.10.22 02:06:19.601221 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:19.609641 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:19.667843 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1331_1331_0 (state Temporary) 2024.10.22 02:06:19.854209 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1331_1331_0 to 202410_1344_1344_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:19.860488 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} MergeTask::MergeProjectionsStage: Merge sorted 692 rows, containing 32 columns (32 merged, 0 gathered) in 3.984728803 sec., 173.66301051128272 rows/sec., 15.74 KiB/sec. 2024.10.22 02:06:19.864155 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_798_187} MergeTask::MergeProjectionsStage: Merge sorted 12063 rows, containing 12 columns (12 merged, 0 gathered) in 1.525681017 sec., 7906.633080956792 rows/sec., 1.23 MiB/sec. 2024.10.22 02:06:20.027711 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} MergedBlockOutputStream: filled checksums 202410_1_648_132 (state Temporary) 2024.10.22 02:06:20.043359 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_798_187} MergedBlockOutputStream: filled checksums 202410_1_798_187 (state Temporary) 2024.10.22 02:06:20.128910 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87806 2024.10.22 02:06:20.155578 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_155_159_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:06:20.162521 [ 1183 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 260 ms. 2024.10.22 02:06:20.398267 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_798_187} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_798_187 to 202410_1_798_187 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:20.404905 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_798_187} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_797_186, 202410_798_798_0] -> 202410_1_798_187 2024.10.22 02:06:20.467974 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:06:20.600941 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_648_132 to 202410_1_648_132 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:20.709275 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_643_131, 202410_648_648_0] -> 202410_1_648_132 2024.10.22 02:06:20.725160 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 5.18 MiB, will set to 1.56 GiB (RSS), difference: -4.20 MiB 2024.10.22 02:06:20.734141 [ 1397 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_648_132}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 02:06:20.751934 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:06:20.868951 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bf1dece2c50e5f504de6d9102b3098a7) : [0, 143]: Setting download as failed: File segment: [0, 143], key: bf1dece2c50e5f504de6d9102b3098a7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:20.870334 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bf1dece2c50e5f504de6d9102b3098a7): No space left in cache to reserve 144 bytes, will continue without cache download 2024.10.22 02:06:21.196536 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:06:21.202737 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:06:21.473005 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1300 entries to flush up to offset 1141933 2024.10.22 02:06:21.962560 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_718_64, 202410_719_719_0] 2024.10.22 02:06:21.965412 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_718_64, 202410_719_719_0] 2024.10.22 02:06:21.976366 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_718_64, 202410_719_719_0] 2024.10.22 02:06:22.014312 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bb2234e3f5aaf693e9df5a2af60c3a26) : [0, 125144]: Setting download as failed: File segment: [0, 125144], key: bb2234e3f5aaf693e9df5a2af60c3a26, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:22.015829 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bb2234e3f5aaf693e9df5a2af60c3a26): No space left in cache to reserve 125145 bytes, will continue without cache download 2024.10.22 02:06:22.081278 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bb2234e3f5aaf693e9df5a2af60c3a26, offset: 0, size: 0 2024.10.22 02:06:22.093141 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bb2234e3f5aaf693e9df5a2af60c3a26 for removal 2024.10.22 02:06:22.101379 [ 1476 ] {} CacheMetadata: Key bb2234e3f5aaf693e9df5a2af60c3a26 is removed from metadata 2024.10.22 02:06:22.104656 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bf1dece2c50e5f504de6d9102b3098a7, offset: 0, size: 0 2024.10.22 02:06:22.110356 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bf1dece2c50e5f504de6d9102b3098a7 for removal 2024.10.22 02:06:22.119936 [ 1476 ] {} CacheMetadata: Key bf1dece2c50e5f504de6d9102b3098a7 is removed from metadata 2024.10.22 02:06:22.492339 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 3.14 MiB, will set to 1.55 GiB (RSS), difference: -7.15 MiB 2024.10.22 02:06:22.677694 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:22.685306 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:22.902854 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a35ab3b658062f063c599f456353bd96) : [0, 154]: Setting download as failed: File segment: [0, 154], key: a35ab3b658062f063c599f456353bd96, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:22.908862 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a35ab3b658062f063c599f456353bd96): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 02:06:22.942025 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_756_756_0 (state Temporary) 2024.10.22 02:06:23.145803 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_756_756_0 to 202410_763_763_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:23.176058 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1141933 2024.10.22 02:06:23.220931 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_155_159_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_155_159_1 to 202410_155_159_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:23.330811 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_155_159_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 5 parts: [202410_155_155_0, 202410_159_159_0] -> 202410_155_159_1 2024.10.22 02:06:23.353538 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_701_701_0, 202410_702_702_0, 202410_703_703_0, 202410_704_704_0, 202410_705_705_0, 202410_706_706_0] 2024.10.22 02:06:23.357595 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_701_701_0, 202410_702_702_0, 202410_703_703_0, 202410_704_704_0, 202410_705_705_0, 202410_706_706_0] 2024.10.22 02:06:23.515960 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 657 2024.10.22 02:06:23.596610 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_701_701_0, 202410_702_702_0, 202410_703_703_0, 202410_704_704_0, 202410_705_705_0, 202410_706_706_0] 2024.10.22 02:06:23.712677 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:23.716140 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:23.844981 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_639_639_0 (state Temporary) 2024.10.22 02:06:23.927822 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_639_639_0 to 202410_649_649_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:23.948316 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(aabccbb0d860e09b7a120173f1f85d1c) : [0, 91311]: Setting download as failed: File segment: [0, 91311], key: aabccbb0d860e09b7a120173f1f85d1c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:23.951058 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(aabccbb0d860e09b7a120173f1f85d1c): No space left in cache to reserve 91312 bytes, will continue without cache download 2024.10.22 02:06:23.966441 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: aabccbb0d860e09b7a120173f1f85d1c, offset: 0, size: 0 2024.10.22 02:06:23.968031 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key aabccbb0d860e09b7a120173f1f85d1c for removal 2024.10.22 02:06:23.970133 [ 1476 ] {} CacheMetadata: Key aabccbb0d860e09b7a120173f1f85d1c is removed from metadata 2024.10.22 02:06:23.972422 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a35ab3b658062f063c599f456353bd96, offset: 0, size: 0 2024.10.22 02:06:23.974892 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a35ab3b658062f063c599f456353bd96 for removal 2024.10.22 02:06:23.979854 [ 1476 ] {} CacheMetadata: Key a35ab3b658062f063c599f456353bd96 is removed from metadata 2024.10.22 02:06:24.040905 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 657 2024.10.22 02:06:24.077784 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.75 MiB. 2024.10.22 02:06:24.252185 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 87878 2024.10.22 02:06:24.517726 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:24.523563 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:24.524594 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 4.15 MiB, will set to 1.55 GiB (RSS), difference: 7.55 MiB 2024.10.22 02:06:24.676657 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1332_1332_0 (state Temporary) 2024.10.22 02:06:24.829166 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1332_1332_0 to 202410_1345_1345_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:24.883888 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87878 2024.10.22 02:06:25.049264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fba6959cbee88c0676cffc643b82a6fc) : [0, 175]: Setting download as failed: File segment: [0, 175], key: fba6959cbee88c0676cffc643b82a6fc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:25.051276 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fba6959cbee88c0676cffc643b82a6fc): No space left in cache to reserve 176 bytes, will continue without cache download 2024.10.22 02:06:25.175672 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:25.179528 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:25.669972 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(07118185b07904d07ac498ed16d46281) : [0, 272819]: Setting download as failed: File segment: [0, 272819], key: 07118185b07904d07ac498ed16d46281, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:25.671855 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(07118185b07904d07ac498ed16d46281): No space left in cache to reserve 272820 bytes, will continue without cache download 2024.10.22 02:06:25.692680 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 07118185b07904d07ac498ed16d46281, offset: 0, size: 0 2024.10.22 02:06:25.695229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 07118185b07904d07ac498ed16d46281 for removal 2024.10.22 02:06:25.700049 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fba6959cbee88c0676cffc643b82a6fc, offset: 0, size: 0 2024.10.22 02:06:25.703311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fba6959cbee88c0676cffc643b82a6fc for removal 2024.10.22 02:06:25.750413 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 87134 2024.10.22 02:06:25.799422 [ 1476 ] {} CacheMetadata: Key 07118185b07904d07ac498ed16d46281 is removed from metadata 2024.10.22 02:06:25.813414 [ 1476 ] {} CacheMetadata: Key fba6959cbee88c0676cffc643b82a6fc is removed from metadata 2024.10.22 02:06:25.833101 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 12044 2024.10.22 02:06:25.871818 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:25.873620 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:26.032687 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_776_776_0 (state Temporary) 2024.10.22 02:06:26.176279 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_776_776_0 to 202410_782_782_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:26.209423 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:26.225348 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:26.251007 [ 1239 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_777_6 to 202410_782_782_0 2024.10.22 02:06:26.256582 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87134 2024.10.22 02:06:26.261052 [ 1239 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:26.264395 [ 1239 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:26.310196 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_777_6 to 202410_782_782_0 into Compact with storage Full 2024.10.22 02:06:26.369663 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:06:26.379747 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_777_6, total 3320 rows starting from the beginning of the part 2024.10.22 02:06:26.420370 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_778_778_0, total 115 rows starting from the beginning of the part 2024.10.22 02:06:26.468324 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_790_790_0 (state Temporary) 2024.10.22 02:06:26.468312 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_779_779_0, total 89 rows starting from the beginning of the part 2024.10.22 02:06:26.519547 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_790_790_0 to 202410_803_803_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:26.558738 [ 1254 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_798_187 to 202410_803_803_0 2024.10.22 02:06:26.559107 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_780_780_0, total 81 rows starting from the beginning of the part 2024.10.22 02:06:26.566977 [ 1254 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:26.570891 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:26.580294 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12044 2024.10.22 02:06:26.605205 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_781_781_0, total 95 rows starting from the beginning of the part 2024.10.22 02:06:26.632381 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_798_187 to 202410_803_803_0 into Compact with storage Full 2024.10.22 02:06:26.654531 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_782_782_0, total 79 rows starting from the beginning of the part 2024.10.22 02:06:26.691480 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 2.93 MiB, will set to 1.55 GiB (RSS), difference: -6.44 MiB 2024.10.22 02:06:26.692999 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:06:26.703917 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_798_187, total 12063 rows starting from the beginning of the part 2024.10.22 02:06:26.785639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4faa713a018e8925da76cd4363f27c5b) : [0, 158]: Setting download as failed: File segment: [0, 158], key: 4faa713a018e8925da76cd4363f27c5b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:26.790646 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4faa713a018e8925da76cd4363f27c5b): No space left in cache to reserve 159 bytes, will continue without cache download 2024.10.22 02:06:26.838126 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} MergeTreeSequentialSource: Reading 2 marks from part 202410_799_799_0, total 22 rows starting from the beginning of the part 2024.10.22 02:06:26.860317 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} MergeTreeSequentialSource: Reading 2 marks from part 202410_800_800_0, total 16 rows starting from the beginning of the part 2024.10.22 02:06:26.884263 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} MergeTreeSequentialSource: Reading 2 marks from part 202410_801_801_0, total 9 rows starting from the beginning of the part 2024.10.22 02:06:26.917620 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} MergeTreeSequentialSource: Reading 2 marks from part 202410_802_802_0, total 15 rows starting from the beginning of the part 2024.10.22 02:06:26.958724 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} MergeTreeSequentialSource: Reading 2 marks from part 202410_803_803_0, total 25 rows starting from the beginning of the part 2024.10.22 02:06:27.435399 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_782_7/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:06:27.826506 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b37f224397d45735894762d8e77f9afe) : [0, 967546]: Setting download as failed: File segment: [0, 967546], key: b37f224397d45735894762d8e77f9afe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:27.829151 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b37f224397d45735894762d8e77f9afe): No space left in cache to reserve 967547 bytes, will continue without cache download 2024.10.22 02:06:27.840978 [ 1250 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 247 ms. 2024.10.22 02:06:27.735656 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} MergeTask::MergeProjectionsStage: Merge sorted 3779 rows, containing 13 columns (13 merged, 0 gathered) in 1.437568767 sec., 2628.7438115995187 rows/sec., 1.14 MiB/sec. 2024.10.22 02:06:27.864711 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b37f224397d45735894762d8e77f9afe, offset: 0, size: 0 2024.10.22 02:06:27.870534 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b37f224397d45735894762d8e77f9afe for removal 2024.10.22 02:06:27.877547 [ 1476 ] {} CacheMetadata: Key b37f224397d45735894762d8e77f9afe is removed from metadata 2024.10.22 02:06:27.889995 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4faa713a018e8925da76cd4363f27c5b, offset: 0, size: 0 2024.10.22 02:06:27.893571 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4faa713a018e8925da76cd4363f27c5b for removal 2024.10.22 02:06:27.900216 [ 1476 ] {} CacheMetadata: Key 4faa713a018e8925da76cd4363f27c5b is removed from metadata 2024.10.22 02:06:28.127577 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} MergedBlockOutputStream: filled checksums 202410_747_782_7 (state Temporary) 2024.10.22 02:06:28.242219 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_782_7 to 202410_747_782_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:28.256054 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_159_159_0 (state Temporary) 2024.10.22 02:06:28.360953 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_782_7} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_777_6, 202410_782_782_0] -> 202410_747_782_7 2024.10.22 02:06:28.392757 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_803_188/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:06:28.450905 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 3.90 MiB, will set to 1.55 GiB (RSS), difference: -8.34 MiB 2024.10.22 02:06:28.485650 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} MergeTask::MergeProjectionsStage: Merge sorted 12150 rows, containing 12 columns (12 merged, 0 gathered) in 1.872747022 sec., 6487.795659140555 rows/sec., 1.01 MiB/sec. 2024.10.22 02:06:28.599719 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.74 MiB. 2024.10.22 02:06:28.755765 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} MergedBlockOutputStream: filled checksums 202410_1_803_188 (state Temporary) 2024.10.22 02:06:28.892663 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 87954 2024.10.22 02:06:29.035642 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_803_188 to 202410_1_803_188 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:29.043399 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_803_188} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_798_187, 202410_803_803_0] -> 202410_1_803_188 2024.10.22 02:06:29.075292 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:29.083497 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:29.090450 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.59 MiB. 2024.10.22 02:06:29.256070 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1333_1333_0 (state Temporary) 2024.10.22 02:06:29.335114 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_719_65, 202410_720_720_0] 2024.10.22 02:06:29.339111 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_719_65, 202410_720_720_0] 2024.10.22 02:06:29.341019 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(68c7fafe5493efc9d5298d9a7ca7a878) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 68c7fafe5493efc9d5298d9a7ca7a878, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:29.344515 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(68c7fafe5493efc9d5298d9a7ca7a878): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 02:06:29.349723 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_719_65, 202410_720_720_0] 2024.10.22 02:06:29.424618 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1333_1333_0 to 202410_1346_1346_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:29.591547 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 87954 2024.10.22 02:06:30.186338 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1607 entries to flush up to offset 1143540 2024.10.22 02:06:30.327969 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(edf71c7088b81787bedafcaf26f5a8b2) : [0, 95820]: Setting download as failed: File segment: [0, 95820], key: edf71c7088b81787bedafcaf26f5a8b2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:30.330430 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(edf71c7088b81787bedafcaf26f5a8b2): No space left in cache to reserve 95821 bytes, will continue without cache download 2024.10.22 02:06:30.358795 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: edf71c7088b81787bedafcaf26f5a8b2, offset: 0, size: 0 2024.10.22 02:06:30.360499 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key edf71c7088b81787bedafcaf26f5a8b2 for removal 2024.10.22 02:06:30.364679 [ 1476 ] {} CacheMetadata: Key edf71c7088b81787bedafcaf26f5a8b2 is removed from metadata 2024.10.22 02:06:30.368412 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 68c7fafe5493efc9d5298d9a7ca7a878, offset: 0, size: 0 2024.10.22 02:06:30.370815 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 68c7fafe5493efc9d5298d9a7ca7a878 for removal 2024.10.22 02:06:30.373823 [ 1476 ] {} CacheMetadata: Key 68c7fafe5493efc9d5298d9a7ca7a878 is removed from metadata 2024.10.22 02:06:31.197540 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:06:31.204051 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:06:31.256281 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b998dfb7c140ca32e8182a2e978cfef9) : [0, 153]: Setting download as failed: File segment: [0, 153], key: b998dfb7c140ca32e8182a2e978cfef9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:31.260093 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b998dfb7c140ca32e8182a2e978cfef9): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:06:31.401223 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:31.401693 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 3.95 MiB, will set to 1.55 GiB (RSS), difference: 5.58 MiB 2024.10.22 02:06:31.407466 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:31.543384 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 658 2024.10.22 02:06:31.680922 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_757_757_0 (state Temporary) 2024.10.22 02:06:31.716247 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:31.723893 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:31.799433 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_640_640_0 (state Temporary) 2024.10.22 02:06:31.901765 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_757_757_0 to 202410_764_764_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:32.033456 [ 1241 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_754_143 to 202410_755_760_1 2024.10.22 02:06:32.043720 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1143540 2024.10.22 02:06:32.058825 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_640_640_0 to 202410_650_650_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:32.060063 [ 1241 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:32.062380 [ 1241 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:32.097764 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_760_144} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_754_143 to 202410_755_760_1 into Wide with storage Full 2024.10.22 02:06:32.134838 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2982e580efc5086214dc4eb32e11d7b6) : [0, 3753]: Setting download as failed: File segment: [0, 3753], key: 2982e580efc5086214dc4eb32e11d7b6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:32.138229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2982e580efc5086214dc4eb32e11d7b6): No space left in cache to reserve 3754 bytes, will continue without cache download 2024.10.22 02:06:32.150391 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_760_144} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:06:32.157510 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_760_144} MergeTreeSequentialSource: Reading 141 marks from part 202410_1_754_143, total 1144810 rows starting from the beginning of the part 2024.10.22 02:06:32.158377 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2982e580efc5086214dc4eb32e11d7b6, offset: 0, size: 0 2024.10.22 02:06:32.160547 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2982e580efc5086214dc4eb32e11d7b6 for removal 2024.10.22 02:06:32.164469 [ 1476 ] {} CacheMetadata: Key 2982e580efc5086214dc4eb32e11d7b6 is removed from metadata 2024.10.22 02:06:32.165590 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b998dfb7c140ca32e8182a2e978cfef9, offset: 0, size: 0 2024.10.22 02:06:32.167100 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b998dfb7c140ca32e8182a2e978cfef9 for removal 2024.10.22 02:06:32.170441 [ 1476 ] {} CacheMetadata: Key b998dfb7c140ca32e8182a2e978cfef9 is removed from metadata 2024.10.22 02:06:32.187050 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_760_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_755_760_1, total 6930 rows starting from the beginning of the part 2024.10.22 02:06:32.188463 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 658 2024.10.22 02:06:32.993800 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(328bbe6c7c24bfde732ea941b52a972d) : [0, 159]: Setting download as failed: File segment: [0, 159], key: 328bbe6c7c24bfde732ea941b52a972d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:33.000499 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(328bbe6c7c24bfde732ea941b52a972d): No space left in cache to reserve 160 bytes, will continue without cache download 2024.10.22 02:06:33.147315 [ 1232 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:06:33.179337 [ 1232 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:06:33.531683 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 6.51 MiB, will set to 1.55 GiB (RSS), difference: -12.69 MiB 2024.10.22 02:06:33.616799 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 88013 2024.10.22 02:06:33.735493 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:33.740926 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:33.775089 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 92 entries to flush up to offset 87226 2024.10.22 02:06:33.902509 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1334_1334_0 (state Temporary) 2024.10.22 02:06:33.995725 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a31d6605b5f14bb1ebecbf3054f52c99) : [0, 1078313]: Setting download as failed: File segment: [0, 1078313], key: a31d6605b5f14bb1ebecbf3054f52c99, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:33.999210 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a31d6605b5f14bb1ebecbf3054f52c99): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:06:34.002661 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:34.009072 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:34.012989 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1334_1334_0 to 202410_1347_1347_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:34.045648 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a31d6605b5f14bb1ebecbf3054f52c99, offset: 0, size: 0 2024.10.22 02:06:34.046840 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a31d6605b5f14bb1ebecbf3054f52c99 for removal 2024.10.22 02:06:34.048595 [ 1476 ] {} CacheMetadata: Key a31d6605b5f14bb1ebecbf3054f52c99 is removed from metadata 2024.10.22 02:06:34.050582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 328bbe6c7c24bfde732ea941b52a972d, offset: 0, size: 0 2024.10.22 02:06:34.051741 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 328bbe6c7c24bfde732ea941b52a972d for removal 2024.10.22 02:06:34.064069 [ 1476 ] {} CacheMetadata: Key 328bbe6c7c24bfde732ea941b52a972d is removed from metadata 2024.10.22 02:06:34.083551 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 12064 2024.10.22 02:06:34.156778 [ 1177 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1342_27 to 202410_1347_1347_0 2024.10.22 02:06:34.170923 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_777_777_0 (state Temporary) 2024.10.22 02:06:34.204223 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88013 2024.10.22 02:06:34.302506 [ 1177 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:34.308748 [ 1177 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:34.351452 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1342_27 to 202410_1347_1347_0 into Compact with storage Full 2024.10.22 02:06:34.363142 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:34.432455 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_777_777_0 to 202410_783_783_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:34.439975 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:34.489703 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:06:34.492656 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87226 2024.10.22 02:06:34.498817 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1342_27, total 8498 rows starting from the beginning of the part 2024.10.22 02:06:34.503545 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_791_791_0 (state Temporary) 2024.10.22 02:06:34.634461 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_1343_1343_0, total 83 rows starting from the beginning of the part 2024.10.22 02:06:34.670433 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_1344_1344_0, total 84 rows starting from the beginning of the part 2024.10.22 02:06:34.692774 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_1345_1345_0, total 72 rows starting from the beginning of the part 2024.10.22 02:06:34.695321 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_760_144/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:06:34.832938 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_1346_1346_0, total 76 rows starting from the beginning of the part 2024.10.22 02:06:34.836947 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(dbca09781f09cd861728556dad97971a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: dbca09781f09cd861728556dad97971a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:06:34.840240 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_791_791_0 to 202410_804_804_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:34.844086 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 808, file segment info: File segment: [0, 33554431], key: dbca09781f09cd861728556dad97971a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:06:34.847365 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:34.961022 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_1347_1347_0, total 59 rows starting from the beginning of the part 2024.10.22 02:06:35.000379 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12064 2024.10.22 02:06:35.515812 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 11.95 MiB, will set to 1.56 GiB (RSS), difference: 4.03 MiB 2024.10.22 02:06:35.836083 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a219373a78c1d99f4767dfa86833461e) : [0, 154]: Setting download as failed: File segment: [0, 154], key: a219373a78c1d99f4767dfa86833461e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:35.839593 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a219373a78c1d99f4767dfa86833461e): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 02:06:36.196787 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_159_159_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:06:36.263975 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_159_159_0 to 202410_162_162_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:36.832828 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(1cd979eb7e79dfb6cbf6c95ff2a04377) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1cd979eb7e79dfb6cbf6c95ff2a04377, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:06:36.860309 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 313, file segment info: File segment: [0, 33554431], key: 1cd979eb7e79dfb6cbf6c95ff2a04377, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:06:36.864513 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:37.044272 [ 1173 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 02:06:37.261627 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} MergeTask::MergeProjectionsStage: Merge sorted 8872 rows, containing 13 columns (13 merged, 0 gathered) in 2.916681054 sec., 3041.813566770609 rows/sec., 538.03 KiB/sec. 2024.10.22 02:06:37.280003 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} MergedBlockOutputStream: filled checksums 202410_1207_1347_28 (state Temporary) 2024.10.22 02:06:37.687926 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1347_28 to 202410_1207_1347_28 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:37.692594 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1347_28} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1342_27, 202410_1347_1347_0] -> 202410_1207_1347_28 2024.10.22 02:06:37.703611 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d64fd5c0af9b3c3fee16fb831cade799) : [0, 72239]: Setting download as failed: File segment: [0, 72239], key: d64fd5c0af9b3c3fee16fb831cade799, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:37.705594 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d64fd5c0af9b3c3fee16fb831cade799): No space left in cache to reserve 72240 bytes, will continue without cache download 2024.10.22 02:06:37.740747 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d64fd5c0af9b3c3fee16fb831cade799, offset: 0, size: 0 2024.10.22 02:06:37.743364 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d64fd5c0af9b3c3fee16fb831cade799 for removal 2024.10.22 02:06:37.746394 [ 1476 ] {} CacheMetadata: Key d64fd5c0af9b3c3fee16fb831cade799 is removed from metadata 2024.10.22 02:06:37.747159 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a219373a78c1d99f4767dfa86833461e, offset: 0, size: 0 2024.10.22 02:06:37.748978 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a219373a78c1d99f4767dfa86833461e for removal 2024.10.22 02:06:37.751436 [ 1476 ] {} CacheMetadata: Key a219373a78c1d99f4767dfa86833461e is removed from metadata 2024.10.22 02:06:37.834972 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 6.68 MiB. 2024.10.22 02:06:38.212372 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 88076 2024.10.22 02:06:38.297726 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1242_7, 202410_1243_1243_0, 202410_1244_1244_0, 202410_1245_1245_0, 202410_1246_1246_0, 202410_1247_1247_0] 2024.10.22 02:06:38.303776 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1242_7, 202410_1243_1243_0, 202410_1244_1244_0, 202410_1245_1245_0, 202410_1246_1246_0, 202410_1247_1247_0] 2024.10.22 02:06:38.317363 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_720_66, 202410_721_721_0] 2024.10.22 02:06:38.322422 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_720_66, 202410_721_721_0] 2024.10.22 02:06:38.335870 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_720_66, 202410_721_721_0] 2024.10.22 02:06:38.373448 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:38.375923 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:38.443587 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1242_7, 202410_1243_1243_0, 202410_1244_1244_0, 202410_1245_1245_0, 202410_1246_1246_0, 202410_1247_1247_0] 2024.10.22 02:06:38.496104 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(cd0578ead26793da5c55b8002993ce0f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cd0578ead26793da5c55b8002993ce0f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:06:38.500431 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 7427, file segment info: File segment: [0, 33554431], key: cd0578ead26793da5c55b8002993ce0f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:06:38.503335 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:38.641703 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1335_1335_0 (state Temporary) 2024.10.22 02:06:38.672109 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1335_1335_0 to 202410_1348_1348_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:38.879887 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88076 2024.10.22 02:06:39.048010 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1144695 2024.10.22 02:06:39.296946 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_694_133, 202410_695_700_1] 2024.10.22 02:06:39.302691 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_694_133, 202410_695_700_1] 2024.10.22 02:06:39.320136 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_694_133, 202410_695_700_1] 2024.10.22 02:06:39.368937 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 4 old parts to remove. Parts: [202410_1_740_153, 202410_741_741_0, 202410_742_742_0, 202410_743_743_0] 2024.10.22 02:06:39.372318 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from filesystem (serially): Parts: [202410_1_740_153, 202410_741_741_0, 202410_742_742_0, 202410_743_743_0] 2024.10.22 02:06:39.425817 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from memory: Parts: [202410_1_740_153, 202410_741_741_0, 202410_742_742_0, 202410_743_743_0] 2024.10.22 02:06:39.517004 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(7192a021262a722a5e21e3bb39641993) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7192a021262a722a5e21e3bb39641993, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:06:39.540786 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 64795, file segment info: File segment: [0, 33554431], key: 7192a021262a722a5e21e3bb39641993, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:06:39.546472 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 14.30 MiB, will set to 1.56 GiB (RSS), difference: 14.59 MiB 2024.10.22 02:06:39.546641 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:39.774207 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6387 2024.10.22 02:06:40.217404 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:40.226921 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:40.456330 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_758_758_0 (state Temporary) 2024.10.22 02:06:40.527806 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(a51c1199cf11662de0c452e037dd579c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a51c1199cf11662de0c452e037dd579c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:06:40.530531 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 513, file segment info: File segment: [0, 33554431], key: a51c1199cf11662de0c452e037dd579c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:06:40.533047 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:40.568876 [ 1341 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/oom/yxkgwnytplpdmzdbugdkbglfxejow, size 7427 2024.10.22 02:06:40.634652 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_758_758_0 to 202410_765_765_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:40.665942 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1144695 2024.10.22 02:06:41.132167 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsArtifical 2024.10.22 02:06:41.144542 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsArtifical 2024.10.22 02:06:41.198684 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:06:41.204035 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:06:41.264252 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsArtifical 2024.10.22 02:06:41.285634 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsArtifical 2024.10.22 02:06:41.316119 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsArtifical 2024.10.22 02:06:41.344276 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsArtifical 2024.10.22 02:06:41.332481 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(4f6e13f943fbe08dba9f2ae5ac75cee6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4f6e13f943fbe08dba9f2ae5ac75cee6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:06:41.356945 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 510, file segment info: File segment: [0, 33554431], key: 4f6e13f943fbe08dba9f2ae5ac75cee6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:06:41.360429 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:41.361654 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsArtifical 2024.10.22 02:06:41.556906 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pdn/ziljsenxmoapgnbkhoukgsinbwnlp (LogSeriesLimiter: on interval from 2024-10-22 02:06:15 to 2024-10-22 02:06:41 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:06:42.028528 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 67 entries to flush up to offset 87293 2024.10.22 02:06:42.209362 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(065c42b57049f824884dc10c360f1d3a) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 065c42b57049f824884dc10c360f1d3a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:42.214669 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(065c42b57049f824884dc10c360f1d3a): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:06:42.261489 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:42.264945 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:42.321364 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_778_778_0 (state Temporary) 2024.10.22 02:06:42.468907 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_778_778_0 to 202410_784_784_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:42.506553 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 12077 2024.10.22 02:06:42.539720 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87293 2024.10.22 02:06:42.842653 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(be4ad479f453141f3b6a20a4adb59d0c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: be4ad479f453141f3b6a20a4adb59d0c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:06:42.845164 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:42.846433 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 268037, file segment info: File segment: [0, 33554431], key: be4ad479f453141f3b6a20a4adb59d0c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:06:42.848097 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:42.848541 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:42.872151 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_792_792_0 (state Temporary) 2024.10.22 02:06:43.029378 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 88140 2024.10.22 02:06:43.032240 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_792_792_0 to 202410_805_805_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:43.082947 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12077 2024.10.22 02:06:43.167267 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:43.170085 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:43.343611 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1336_1336_0 (state Temporary) 2024.10.22 02:06:43.370413 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cff03f52e2812303c961d87ecfedf3b1) : [0, 61454]: Setting download as failed: File segment: [0, 61454], key: cff03f52e2812303c961d87ecfedf3b1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:43.375393 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cff03f52e2812303c961d87ecfedf3b1): No space left in cache to reserve 61455 bytes, will continue without cache download 2024.10.22 02:06:43.392083 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cff03f52e2812303c961d87ecfedf3b1, offset: 0, size: 0 2024.10.22 02:06:43.404607 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cff03f52e2812303c961d87ecfedf3b1 for removal 2024.10.22 02:06:43.407836 [ 1476 ] {} CacheMetadata: Key cff03f52e2812303c961d87ecfedf3b1 is removed from metadata 2024.10.22 02:06:43.411751 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 065c42b57049f824884dc10c360f1d3a, offset: 0, size: 0 2024.10.22 02:06:43.514671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 065c42b57049f824884dc10c360f1d3a for removal 2024.10.22 02:06:43.520225 [ 1476 ] {} CacheMetadata: Key 065c42b57049f824884dc10c360f1d3a is removed from metadata 2024.10.22 02:06:43.653198 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1336_1336_0 to 202410_1349_1349_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:44.080699 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88140 2024.10.22 02:06:44.593191 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(26d4cd4913470ab4340d601fbbab4c7e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 26d4cd4913470ab4340d601fbbab4c7e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:06:44.596729 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 750, file segment info: File segment: [0, 33554431], key: 26d4cd4913470ab4340d601fbbab4c7e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1385, kind: Regular, unbound: 0 2024.10.22 02:06:44.600137 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:44.746885 [ 1682 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rco/aucjajhfmgeqczfldlxfybwzilnyt, size 268037 2024.10.22 02:06:45.032051 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:06:45.056552 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:06:45.089426 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:06:45.222843 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:06:45.253078 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:06:45.275305 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:06:45.483895 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4171d5ca6e6eae97d374d76e2265703d) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 4171d5ca6e6eae97d374d76e2265703d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:45.488812 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4171d5ca6e6eae97d374d76e2265703d): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:06:45.528553 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:06:45.567563 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_721_67, 202410_722_722_0] 2024.10.22 02:06:45.568906 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_721_67, 202410_722_722_0] 2024.10.22 02:06:45.593185 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_721_67, 202410_722_722_0] 2024.10.22 02:06:45.656537 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:06:45.856396 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.064711602 sec. 2024.10.22 02:06:46.065191 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/njs/mfcewlufsigbfixowmpmqfkwppjnl (LogSeriesLimiter: on interval from 2024-10-22 02:06:41 to 2024-10-22 02:06:46 accepted series 1 / 5 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:06:46.670926 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(59e1d3fe4fa62a5e89e86f3df5def84b) : [0, 62513]: Setting download as failed: File segment: [0, 62513], key: 59e1d3fe4fa62a5e89e86f3df5def84b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:46.672409 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(59e1d3fe4fa62a5e89e86f3df5def84b): No space left in cache to reserve 62514 bytes, will continue without cache download 2024.10.22 02:06:46.694265 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 59e1d3fe4fa62a5e89e86f3df5def84b, offset: 0, size: 0 2024.10.22 02:06:46.699326 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 59e1d3fe4fa62a5e89e86f3df5def84b for removal 2024.10.22 02:06:46.702785 [ 1476 ] {} CacheMetadata: Key 59e1d3fe4fa62a5e89e86f3df5def84b is removed from metadata 2024.10.22 02:06:46.706275 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4171d5ca6e6eae97d374d76e2265703d, offset: 0, size: 0 2024.10.22 02:06:46.708206 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4171d5ca6e6eae97d374d76e2265703d for removal 2024.10.22 02:06:46.715147 [ 1476 ] {} CacheMetadata: Key 4171d5ca6e6eae97d374d76e2265703d is removed from metadata 2024.10.22 02:06:47.180690 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_760_144/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:06:47.203387 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 659 2024.10.22 02:06:47.341529 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 6426 2024.10.22 02:06:47.584527 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:47.587428 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:47.670264 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1145465 2024.10.22 02:06:47.657946 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 14.95 MiB, will set to 1.56 GiB (RSS), difference: -4.24 MiB 2024.10.22 02:06:47.766762 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0620b15f877a139440b029e2ad5e3eb3) : [0, 164]: Setting download as failed: File segment: [0, 164], key: 0620b15f877a139440b029e2ad5e3eb3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:47.768952 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0620b15f877a139440b029e2ad5e3eb3): No space left in cache to reserve 165 bytes, will continue without cache download 2024.10.22 02:06:47.960306 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_641_641_0 (state Temporary) 2024.10.22 02:06:48.081349 [ 1164 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 214 ms. 2024.10.22 02:06:48.092028 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 88194 2024.10.22 02:06:48.160034 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_641_641_0 to 202410_651_651_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:48.219480 [ 1272 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:06:48.256388 [ 1272 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:06:48.281366 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 659 2024.10.22 02:06:48.571968 [ 1278 ] {} DatabaseCatalog: Execution took 249 ms. 2024.10.22 02:06:48.639610 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:48.646489 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:48.684932 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1337_1337_0 (state Temporary) 2024.10.22 02:06:48.751694 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1337_1337_0 to 202410_1350_1350_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:48.817529 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88194 2024.10.22 02:06:48.892170 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bb44f483bba71b6795276d4a8631b347) : [0, 94123]: Setting download as failed: File segment: [0, 94123], key: bb44f483bba71b6795276d4a8631b347, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:48.896511 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bb44f483bba71b6795276d4a8631b347): No space left in cache to reserve 94124 bytes, will continue without cache download 2024.10.22 02:06:48.914001 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bb44f483bba71b6795276d4a8631b347, offset: 0, size: 0 2024.10.22 02:06:48.915363 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bb44f483bba71b6795276d4a8631b347 for removal 2024.10.22 02:06:48.920158 [ 1476 ] {} CacheMetadata: Key bb44f483bba71b6795276d4a8631b347 is removed from metadata 2024.10.22 02:06:48.927154 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0620b15f877a139440b029e2ad5e3eb3, offset: 0, size: 0 2024.10.22 02:06:48.928794 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0620b15f877a139440b029e2ad5e3eb3 for removal 2024.10.22 02:06:48.935393 [ 1476 ] {} CacheMetadata: Key 0620b15f877a139440b029e2ad5e3eb3 is removed from metadata 2024.10.22 02:06:49.183926 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:49.188888 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:49.360455 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_759_759_0 (state Temporary) 2024.10.22 02:06:49.634938 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_759_759_0 to 202410_766_766_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:49.677505 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1145465 2024.10.22 02:06:49.711200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ecd202015b4d5c7beb8eb09df2870556) : [0, 153]: Setting download as failed: File segment: [0, 153], key: ecd202015b4d5c7beb8eb09df2870556, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:49.714785 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ecd202015b4d5c7beb8eb09df2870556): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:06:49.770814 [ 1273 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_761_761_0 to 202410_766_766_0 2024.10.22 02:06:49.825466 [ 1273 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:49.833063 [ 1273 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:49.867151 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_761_761_0 to 202410_766_766_0 into Compact with storage Full 2024.10.22 02:06:50.026837 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:06:50.050824 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 51 entries to flush up to offset 87344 2024.10.22 02:06:50.141039 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_761_761_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:06:50.157115 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_762_762_0, total 1328 rows starting from the beginning of the part 2024.10.22 02:06:50.170442 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_763_763_0, total 1300 rows starting from the beginning of the part 2024.10.22 02:06:50.214804 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_764_764_0, total 1607 rows starting from the beginning of the part 2024.10.22 02:06:50.264506 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_765_765_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:06:50.312499 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_766_766_0, total 770 rows starting from the beginning of the part 2024.10.22 02:06:50.315250 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:50.321756 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:50.366196 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_779_779_0 (state Temporary) 2024.10.22 02:06:50.536701 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_779_779_0 to 202410_785_785_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:50.588355 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 12083 2024.10.22 02:06:50.778253 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(93f6fa116284f6f1b37851c10af0e74f) : [0, 43323]: Setting download as failed: File segment: [0, 43323], key: 93f6fa116284f6f1b37851c10af0e74f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:50.779922 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87344 2024.10.22 02:06:50.782940 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(93f6fa116284f6f1b37851c10af0e74f): No space left in cache to reserve 43324 bytes, will continue without cache download 2024.10.22 02:06:50.801823 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 93f6fa116284f6f1b37851c10af0e74f, offset: 0, size: 0 2024.10.22 02:06:50.807708 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 93f6fa116284f6f1b37851c10af0e74f for removal 2024.10.22 02:06:50.820992 [ 1476 ] {} CacheMetadata: Key 93f6fa116284f6f1b37851c10af0e74f is removed from metadata 2024.10.22 02:06:50.826544 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ecd202015b4d5c7beb8eb09df2870556, offset: 0, size: 0 2024.10.22 02:06:50.828877 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ecd202015b4d5c7beb8eb09df2870556 for removal 2024.10.22 02:06:50.940218 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:50.942525 [ 1476 ] {} CacheMetadata: Key ecd202015b4d5c7beb8eb09df2870556 is removed from metadata 2024.10.22 02:06:50.946367 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:51.099248 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_793_793_0 (state Temporary) 2024.10.22 02:06:51.115871 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} MergeTask::MergeProjectionsStage: Merge sorted 7315 rows, containing 4 columns (4 merged, 0 gathered) in 1.269239704 sec., 5763.292762546608 rows/sec., 90.05 KiB/sec. 2024.10.22 02:06:51.201244 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:06:51.207643 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:06:51.258668 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_793_793_0 to 202410_806_806_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:51.330159 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} MergedBlockOutputStream: filled checksums 202410_761_766_1 (state Temporary) 2024.10.22 02:06:51.358940 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12083 2024.10.22 02:06:51.426635 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_761_766_1 to 202410_761_766_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:51.435239 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_761_766_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_761_761_0, 202410_766_766_0] -> 202410_761_766_1 2024.10.22 02:06:51.574212 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:06:52.209755 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8ba63df2b791bae6ca5df8475bf223be) : [0, 147]: Setting download as failed: File segment: [0, 147], key: 8ba63df2b791bae6ca5df8475bf223be, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:52.211958 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8ba63df2b791bae6ca5df8475bf223be): No space left in cache to reserve 148 bytes, will continue without cache download 2024.10.22 02:06:52.449747 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 14.26 MiB, will set to 1.56 GiB (RSS), difference: 1.15 MiB 2024.10.22 02:06:52.609131 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_598_122, 202410_599_599_0, 202410_600_600_0, 202410_601_601_0, 202410_602_602_0, 202410_603_603_0] 2024.10.22 02:06:52.612745 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_598_122, 202410_599_599_0, 202410_600_600_0, 202410_601_601_0, 202410_602_602_0, 202410_603_603_0] 2024.10.22 02:06:52.744477 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_598_122, 202410_599_599_0, 202410_600_600_0, 202410_601_601_0, 202410_602_602_0, 202410_603_603_0] 2024.10.22 02:06:52.935769 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 88264 2024.10.22 02:06:53.117620 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f3c6b548d1c1acca198f495c4766b7e4) : [0, 240523]: Setting download as failed: File segment: [0, 240523], key: f3c6b548d1c1acca198f495c4766b7e4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:53.121285 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f3c6b548d1c1acca198f495c4766b7e4): No space left in cache to reserve 240524 bytes, will continue without cache download 2024.10.22 02:06:53.147145 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f3c6b548d1c1acca198f495c4766b7e4, offset: 0, size: 0 2024.10.22 02:06:53.149433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f3c6b548d1c1acca198f495c4766b7e4 for removal 2024.10.22 02:06:53.152107 [ 1476 ] {} CacheMetadata: Key f3c6b548d1c1acca198f495c4766b7e4 is removed from metadata 2024.10.22 02:06:53.153151 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8ba63df2b791bae6ca5df8475bf223be, offset: 0, size: 0 2024.10.22 02:06:53.154453 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8ba63df2b791bae6ca5df8475bf223be for removal 2024.10.22 02:06:53.156066 [ 1476 ] {} CacheMetadata: Key 8ba63df2b791bae6ca5df8475bf223be is removed from metadata 2024.10.22 02:06:53.398575 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:53.400654 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:53.575046 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1338_1338_0 (state Temporary) 2024.10.22 02:06:53.754660 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1338_1338_0 to 202410_1351_1351_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:53.816141 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88264 2024.10.22 02:06:53.951464 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b590dcf424d2b53062f045d4b72838a4) : [0, 146]: Setting download as failed: File segment: [0, 146], key: b590dcf424d2b53062f045d4b72838a4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:53.952816 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b590dcf424d2b53062f045d4b72838a4): No space left in cache to reserve 147 bytes, will continue without cache download 2024.10.22 02:06:54.225644 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_722_68, 202410_723_723_0] 2024.10.22 02:06:54.230676 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_722_68, 202410_723_723_0] 2024.10.22 02:06:54.343140 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_722_68, 202410_723_723_0] 2024.10.22 02:06:54.650933 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 15.79 MiB, will set to 1.56 GiB (RSS), difference: 1.76 MiB 2024.10.22 02:06:55.259458 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2f9faef6c1241ef141ef8e5679ba0b2e) : [0, 220576]: Setting download as failed: File segment: [0, 220576], key: 2f9faef6c1241ef141ef8e5679ba0b2e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:55.264208 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2f9faef6c1241ef141ef8e5679ba0b2e): No space left in cache to reserve 220577 bytes, will continue without cache download 2024.10.22 02:06:55.306167 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2f9faef6c1241ef141ef8e5679ba0b2e, offset: 0, size: 0 2024.10.22 02:06:55.309209 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2f9faef6c1241ef141ef8e5679ba0b2e for removal 2024.10.22 02:06:55.314573 [ 1476 ] {} CacheMetadata: Key 2f9faef6c1241ef141ef8e5679ba0b2e is removed from metadata 2024.10.22 02:06:55.323041 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b590dcf424d2b53062f045d4b72838a4, offset: 0, size: 0 2024.10.22 02:06:55.325742 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b590dcf424d2b53062f045d4b72838a4 for removal 2024.10.22 02:06:55.331097 [ 1476 ] {} CacheMetadata: Key b590dcf424d2b53062f045d4b72838a4 is removed from metadata 2024.10.22 02:06:55.786513 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 660 2024.10.22 02:06:56.432859 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:56.458403 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(29455a8f0b8d3ebf669fdbdb05337062) : [0, 141]: Setting download as failed: File segment: [0, 141], key: 29455a8f0b8d3ebf669fdbdb05337062, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:56.460032 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(29455a8f0b8d3ebf669fdbdb05337062): No space left in cache to reserve 142 bytes, will continue without cache download 2024.10.22 02:06:56.476875 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:56.661014 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_642_642_0 (state Temporary) 2024.10.22 02:06:56.682119 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1147005 2024.10.22 02:06:57.060303 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_642_642_0 to 202410_652_652_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:57.159158 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 660 2024.10.22 02:06:57.615170 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(edcecb0666f80bdaf403fad84f0014ed) : [0, 4963]: Setting download as failed: File segment: [0, 4963], key: edcecb0666f80bdaf403fad84f0014ed, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:06:57.627660 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(edcecb0666f80bdaf403fad84f0014ed): No space left in cache to reserve 4964 bytes, will continue without cache download 2024.10.22 02:06:57.765123 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: edcecb0666f80bdaf403fad84f0014ed, offset: 0, size: 0 2024.10.22 02:06:57.766488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key edcecb0666f80bdaf403fad84f0014ed for removal 2024.10.22 02:06:57.783254 [ 1476 ] {} CacheMetadata: Key edcecb0666f80bdaf403fad84f0014ed is removed from metadata 2024.10.22 02:06:57.786794 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 29455a8f0b8d3ebf669fdbdb05337062, offset: 0, size: 0 2024.10.22 02:06:57.794706 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 29455a8f0b8d3ebf669fdbdb05337062 for removal 2024.10.22 02:06:57.798725 [ 1476 ] {} CacheMetadata: Key 29455a8f0b8d3ebf669fdbdb05337062 is removed from metadata 2024.10.22 02:06:57.827427 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 88309 2024.10.22 02:06:57.973941 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:57.980149 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:58.100844 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_760_760_0 (state Temporary) 2024.10.22 02:06:58.167193 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:58.169208 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:58.247061 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1339_1339_0 (state Temporary) 2024.10.22 02:06:58.279161 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_760_760_0 to 202410_767_767_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:58.328536 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_760_144/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:06:58.333525 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1147005 2024.10.22 02:06:58.391008 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 87415 2024.10.22 02:06:58.521197 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1339_1339_0 to 202410_1352_1352_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:58.627400 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1347_28 to 202410_1352_1352_0 2024.10.22 02:06:58.640727 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:58.647007 [ 1240 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:58.669118 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:58.673093 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:58.673356 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1347_28 to 202410_1352_1352_0 into Compact with storage Full 2024.10.22 02:06:58.707088 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88309 2024.10.22 02:06:58.802484 [ 1206 ] {} DatabaseCatalog: Execution took 227 ms. 2024.10.22 02:06:58.830681 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:06:58.840402 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_780_780_0 (state Temporary) 2024.10.22 02:06:58.862030 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 12095 2024.10.22 02:06:58.880837 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1347_28, total 8872 rows starting from the beginning of the part 2024.10.22 02:06:59.011324 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_780_780_0 to 202410_786_786_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:59.073075 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87415 2024.10.22 02:06:59.091725 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:06:59.099942 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:06:59.107428 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_1348_1348_0, total 63 rows starting from the beginning of the part 2024.10.22 02:06:59.167432 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_794_794_0 (state Temporary) 2024.10.22 02:06:59.314965 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_1349_1349_0, total 64 rows starting from the beginning of the part 2024.10.22 02:06:59.317028 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_794_794_0 to 202410_807_807_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:06:59.329329 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(cb2a913bf7bad7106f02f5458237a66d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cb2a913bf7bad7106f02f5458237a66d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:06:59.333657 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 915037, file segment info: File segment: [0, 33554431], key: cb2a913bf7bad7106f02f5458237a66d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:06:59.360476 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:06:59.402795 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12095 2024.10.22 02:06:59.474520 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_1350_1350_0, total 54 rows starting from the beginning of the part 2024.10.22 02:06:59.575418 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 14.63 MiB, will set to 1.57 GiB (RSS), difference: 1.17 MiB 2024.10.22 02:06:59.628961 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_1351_1351_0, total 70 rows starting from the beginning of the part 2024.10.22 02:06:59.716342 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_1352_1352_0, total 45 rows starting from the beginning of the part 2024.10.22 02:07:00.147710 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(288f5bbf44dcb9104a5b832754b4a70c) : [0, 165]: Setting download as failed: File segment: [0, 165], key: 288f5bbf44dcb9104a5b832754b4a70c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:00.149299 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(288f5bbf44dcb9104a5b832754b4a70c): No space left in cache to reserve 166 bytes, will continue without cache download 2024.10.22 02:07:01.204529 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:07:01.211118 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:07:01.491510 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(f67ad569706b7a722b03956b71cc1aaf) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f67ad569706b7a722b03956b71cc1aaf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:01.500684 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 530, file segment info: File segment: [0, 33554431], key: f67ad569706b7a722b03956b71cc1aaf, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:01.507863 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:01.772841 [ 1735 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pdn/ziljsenxmoapgnbkhoukgsinbwnlp, size 915037 2024.10.22 02:07:01.809668 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 02:07:01.948942 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 02:07:01.967909 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 02:07:01.995854 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 02:07:02.028528 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 02:07:02.060506 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 02:07:02.075107 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column WindowClientWidth 2024.10.22 02:07:02.175024 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qdi/njpumepiglfziqfnzrbgmiqityqhi (LogSeriesLimiter: on interval from 2024-10-22 02:06:46 to 2024-10-22 02:07:02 accepted series 1 / 1 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:07:02.549229 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1352_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:07:02.563909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c47937b831694c0c13012ad9c1117b79) : [0, 163350]: Setting download as failed: File segment: [0, 163350], key: c47937b831694c0c13012ad9c1117b79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:02.565315 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c47937b831694c0c13012ad9c1117b79): No space left in cache to reserve 163351 bytes, will continue without cache download 2024.10.22 02:07:02.591964 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c47937b831694c0c13012ad9c1117b79, offset: 0, size: 0 2024.10.22 02:07:02.595303 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c47937b831694c0c13012ad9c1117b79 for removal 2024.10.22 02:07:02.612577 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 288f5bbf44dcb9104a5b832754b4a70c, offset: 0, size: 0 2024.10.22 02:07:02.615190 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 288f5bbf44dcb9104a5b832754b4a70c for removal 2024.10.22 02:07:02.618341 [ 1476 ] {} CacheMetadata: Key c47937b831694c0c13012ad9c1117b79 is removed from metadata 2024.10.22 02:07:02.628337 [ 1476 ] {} CacheMetadata: Key 288f5bbf44dcb9104a5b832754b4a70c is removed from metadata 2024.10.22 02:07:02.696484 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1247_8, 202410_1248_1248_0, 202410_1249_1249_0, 202410_1250_1250_0, 202410_1251_1251_0, 202410_1252_1252_0] 2024.10.22 02:07:02.707550 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1247_8, 202410_1248_1248_0, 202410_1249_1249_0, 202410_1250_1250_0, 202410_1251_1251_0, 202410_1252_1252_0] 2024.10.22 02:07:02.712722 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 88375 2024.10.22 02:07:02.750423 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1247_8, 202410_1248_1248_0, 202410_1249_1249_0, 202410_1250_1250_0, 202410_1251_1251_0, 202410_1252_1252_0] 2024.10.22 02:07:02.933709 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:02.936695 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:03.063782 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} MergeTask::MergeProjectionsStage: Merge sorted 9168 rows, containing 13 columns (13 merged, 0 gathered) in 4.396013523 sec., 2085.525886586314 rows/sec., 369.33 KiB/sec. 2024.10.22 02:07:03.100995 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1340_1340_0 (state Temporary) 2024.10.22 02:07:03.110435 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} MergedBlockOutputStream: filled checksums 202410_1207_1352_29 (state Temporary) 2024.10.22 02:07:03.189280 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1340_1340_0 to 202410_1353_1353_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:03.260372 [ 1199 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:07:03.314875 [ 1199 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:07:03.339926 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_723_69, 202410_724_724_0] 2024.10.22 02:07:03.341446 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88375 2024.10.22 02:07:03.343997 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_723_69, 202410_724_724_0] 2024.10.22 02:07:03.355060 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_723_69, 202410_724_724_0] 2024.10.22 02:07:03.471345 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1352_29 to 202410_1207_1352_29 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:03.475911 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1352_29} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1347_28, 202410_1352_1352_0] -> 202410_1207_1352_29 2024.10.22 02:07:03.532052 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.68 MiB. 2024.10.22 02:07:03.548058 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.71 MiB, will set to 1.57 GiB (RSS), difference: -5.19 MiB 2024.10.22 02:07:03.655433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5d651d6fab3d03afa93f9d67b232a58e) : [0, 146]: Setting download as failed: File segment: [0, 146], key: 5d651d6fab3d03afa93f9d67b232a58e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:03.656850 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5d651d6fab3d03afa93f9d67b232a58e): No space left in cache to reserve 147 bytes, will continue without cache download 2024.10.22 02:07:04.538618 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(94151b5e16b15fb76c80b73935e211e1) : [0, 212474]: Setting download as failed: File segment: [0, 212474], key: 94151b5e16b15fb76c80b73935e211e1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:04.544871 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(94151b5e16b15fb76c80b73935e211e1): No space left in cache to reserve 212475 bytes, will continue without cache download 2024.10.22 02:07:04.565264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 94151b5e16b15fb76c80b73935e211e1, offset: 0, size: 0 2024.10.22 02:07:04.567728 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 94151b5e16b15fb76c80b73935e211e1 for removal 2024.10.22 02:07:04.570384 [ 1476 ] {} CacheMetadata: Key 94151b5e16b15fb76c80b73935e211e1 is removed from metadata 2024.10.22 02:07:04.572181 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5d651d6fab3d03afa93f9d67b232a58e, offset: 0, size: 0 2024.10.22 02:07:04.573777 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5d651d6fab3d03afa93f9d67b232a58e for removal 2024.10.22 02:07:04.576940 [ 1476 ] {} CacheMetadata: Key 5d651d6fab3d03afa93f9d67b232a58e is removed from metadata 2024.10.22 02:07:04.673278 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 661 2024.10.22 02:07:04.707157 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 4 old parts to remove. Parts: [202410_1_743_154, 202410_744_744_0, 202410_745_745_0, 202410_746_746_0] 2024.10.22 02:07:04.710315 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from filesystem (serially): Parts: [202410_1_743_154, 202410_744_744_0, 202410_745_745_0, 202410_746_746_0] 2024.10.22 02:07:04.732831 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from memory: Parts: [202410_1_743_154, 202410_744_744_0, 202410_745_745_0, 202410_746_746_0] 2024.10.22 02:07:05.060791 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:05.064530 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:05.335478 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_643_643_0 (state Temporary) 2024.10.22 02:07:05.371186 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1148160 2024.10.22 02:07:05.491227 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 10.43 MiB, will set to 1.56 GiB (RSS), difference: -6.40 MiB 2024.10.22 02:07:05.501115 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(208356d051ebfa55bba31a826ba0fdc9) : [0, 146]: Setting download as failed: File segment: [0, 146], key: 208356d051ebfa55bba31a826ba0fdc9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:05.502458 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_643_643_0 to 202410_653_653_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:05.504274 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(208356d051ebfa55bba31a826ba0fdc9): No space left in cache to reserve 147 bytes, will continue without cache download 2024.10.22 02:07:05.546373 [ 1234 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_648_132 to 202410_653_653_0 2024.10.22 02:07:05.565183 [ 1234 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:05.568776 [ 1234 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:05.608017 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 661 2024.10.22 02:07:05.612290 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_648_132 to 202410_653_653_0 into Compact with storage Full 2024.10.22 02:07:05.851294 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:07:05.865440 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_648_132, total 692 rows starting from the beginning of the part 2024.10.22 02:07:05.913067 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:05.918254 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:06.006373 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:06.025375 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:06.043445 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_649_649_0, total 1 rows starting from the beginning of the part 2024.10.22 02:07:06.074524 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_761_761_0 (state Temporary) 2024.10.22 02:07:06.249211 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_650_650_0, total 1 rows starting from the beginning of the part 2024.10.22 02:07:06.369290 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_761_761_0 to 202410_768_768_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:06.459494 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_651_651_0, total 1 rows starting from the beginning of the part 2024.10.22 02:07:06.592753 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 72 entries to flush up to offset 87487 2024.10.22 02:07:06.625638 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1148160 2024.10.22 02:07:06.676017 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7c5a29cf3ab8f4d168443da71f08ac05) : [0, 257879]: Setting download as failed: File segment: [0, 257879], key: 7c5a29cf3ab8f4d168443da71f08ac05, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:06.677623 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7c5a29cf3ab8f4d168443da71f08ac05): No space left in cache to reserve 257880 bytes, will continue without cache download 2024.10.22 02:07:06.774666 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_652_652_0, total 1 rows starting from the beginning of the part 2024.10.22 02:07:06.840166 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7c5a29cf3ab8f4d168443da71f08ac05, offset: 0, size: 0 2024.10.22 02:07:06.843802 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7c5a29cf3ab8f4d168443da71f08ac05 for removal 2024.10.22 02:07:06.848812 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 208356d051ebfa55bba31a826ba0fdc9, offset: 0, size: 0 2024.10.22 02:07:06.850356 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:06.851255 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 208356d051ebfa55bba31a826ba0fdc9 for removal 2024.10.22 02:07:06.853066 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:06.855706 [ 1476 ] {} CacheMetadata: Key 7c5a29cf3ab8f4d168443da71f08ac05 is removed from metadata 2024.10.22 02:07:06.860211 [ 1476 ] {} CacheMetadata: Key 208356d051ebfa55bba31a826ba0fdc9 is removed from metadata 2024.10.22 02:07:06.878667 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_653_653_0, total 1 rows starting from the beginning of the part 2024.10.22 02:07:06.895329 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_781_781_0 (state Temporary) 2024.10.22 02:07:07.013149 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 12107 2024.10.22 02:07:07.186282 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_781_781_0 to 202410_787_787_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:07.256749 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87487 2024.10.22 02:07:07.267403 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:07.355553 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 88451 2024.10.22 02:07:07.365074 [ 1269 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_782_7 to 202410_787_787_0 2024.10.22 02:07:07.370858 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:07.386827 [ 1269 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:07.401688 [ 1269 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:07.451180 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_782_7 to 202410_787_787_0 into Compact with storage Full 2024.10.22 02:07:07.521333 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_795_795_0 (state Temporary) 2024.10.22 02:07:07.643438 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:07:07.655512 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_782_7, total 3779 rows starting from the beginning of the part 2024.10.22 02:07:07.754638 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ae3476dca7dddfa875a160957dd47572) : [0, 149]: Setting download as failed: File segment: [0, 149], key: ae3476dca7dddfa875a160957dd47572, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:07.756187 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ae3476dca7dddfa875a160957dd47572): No space left in cache to reserve 150 bytes, will continue without cache download 2024.10.22 02:07:07.799206 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_795_795_0 to 202410_808_808_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:07.800199 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_783_783_0, total 92 rows starting from the beginning of the part 2024.10.22 02:07:07.810141 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:07.817278 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:07.864087 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_784_784_0, total 67 rows starting from the beginning of the part 2024.10.22 02:07:07.902876 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1341_1341_0 (state Temporary) 2024.10.22 02:07:07.945400 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_785_785_0, total 51 rows starting from the beginning of the part 2024.10.22 02:07:07.948473 [ 1239 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_803_188 to 202410_804_804_0 2024.10.22 02:07:07.955452 [ 1239 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:07.959071 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12107 2024.10.22 02:07:07.959174 [ 1239 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:07.966406 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1341_1341_0 to 202410_1354_1354_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:07.995598 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_786_786_0, total 71 rows starting from the beginning of the part 2024.10.22 02:07:08.005809 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_804_189} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_803_188 to 202410_804_804_0 into Compact with storage Full 2024.10.22 02:07:08.054373 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88451 2024.10.22 02:07:08.060551 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_787_787_0, total 72 rows starting from the beginning of the part 2024.10.22 02:07:08.172829 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_804_189} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:07:08.274452 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_804_189} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_803_188, total 12150 rows starting from the beginning of the part 2024.10.22 02:07:08.311024 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_804_189} MergeTreeSequentialSource: Reading 2 marks from part 202410_804_804_0, total 20 rows starting from the beginning of the part 2024.10.22 02:07:08.343212 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_160_160_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:07:08.496365 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_160_160_0 (state Temporary) 2024.10.22 02:07:08.548490 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 13.16 MiB, will set to 1.57 GiB (RSS), difference: -18.31 MiB 2024.10.22 02:07:08.585717 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(28b8d3194ff7b8a493840c15d9acf8a4) : [0, 333886]: Setting download as failed: File segment: [0, 333886], key: 28b8d3194ff7b8a493840c15d9acf8a4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:08.595165 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(28b8d3194ff7b8a493840c15d9acf8a4): No space left in cache to reserve 333887 bytes, will continue without cache download 2024.10.22 02:07:08.808600 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_760_144/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:07:08.905559 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_804_189} MergeTask::MergeProjectionsStage: Merge sorted 12170 rows, containing 12 columns (12 merged, 0 gathered) in 0.909441856 sec., 13381.83405537033 rows/sec., 2.09 MiB/sec. 2024.10.22 02:07:08.948980 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 28b8d3194ff7b8a493840c15d9acf8a4, offset: 0, size: 0 2024.10.22 02:07:08.962641 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 28b8d3194ff7b8a493840c15d9acf8a4 for removal 2024.10.22 02:07:08.964618 [ 1476 ] {} CacheMetadata: Key 28b8d3194ff7b8a493840c15d9acf8a4 is removed from metadata 2024.10.22 02:07:08.965399 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ae3476dca7dddfa875a160957dd47572, offset: 0, size: 0 2024.10.22 02:07:08.966174 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_804_189} MergedBlockOutputStream: filled checksums 202410_1_804_189 (state Temporary) 2024.10.22 02:07:08.966227 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ae3476dca7dddfa875a160957dd47572 for removal 2024.10.22 02:07:08.967699 [ 1476 ] {} CacheMetadata: Key ae3476dca7dddfa875a160957dd47572 is removed from metadata 2024.10.22 02:07:09.011270 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_760_144} MergeTask::MergeProjectionsStage: Merge sorted 1151740 rows, containing 4 columns (4 merged, 0 gathered) in 36.925509471 sec., 31190.903429634094 rows/sec., 467.86 KiB/sec. 2024.10.22 02:07:09.062829 [ 1256 ] {} DatabaseCatalog: Execution took 261 ms. 2024.10.22 02:07:09.174493 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_760_144} MergedBlockOutputStream: filled checksums 202410_1_760_144 (state Temporary) 2024.10.22 02:07:09.293414 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_804_189} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_804_189 to 202410_1_804_189 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:09.302244 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_804_189} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_803_188, 202410_804_804_0] -> 202410_1_804_189 2024.10.22 02:07:09.516512 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:07:09.657006 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} MergeTask::MergeProjectionsStage: Merge sorted 697 rows, containing 32 columns (32 merged, 0 gathered) in 4.061145081 sec., 171.62647137648514 rows/sec., 15.55 KiB/sec. 2024.10.22 02:07:09.724941 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} MergedBlockOutputStream: filled checksums 202410_1_653_133 (state Temporary) 2024.10.22 02:07:09.736104 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} MergeTask::MergeProjectionsStage: Merge sorted 4132 rows, containing 13 columns (13 merged, 0 gathered) in 2.299441207 sec., 1796.9583164036949 rows/sec., 798.09 KiB/sec. 2024.10.22 02:07:09.797438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4a3d69eff3d14bbe5889d58216d31864) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 4a3d69eff3d14bbe5889d58216d31864, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:09.800288 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4a3d69eff3d14bbe5889d58216d31864): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 02:07:09.879391 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} MergedBlockOutputStream: filled checksums 202410_747_787_8 (state Temporary) 2024.10.22 02:07:10.007637 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_787_8 to 202410_747_787_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:10.013240 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_787_8} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_782_7, 202410_787_787_0] -> 202410_747_787_8 2024.10.22 02:07:10.049070 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.04 MiB. 2024.10.22 02:07:10.321520 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 9.47 MiB, will set to 1.56 GiB (RSS), difference: 17.16 MiB 2024.10.22 02:07:10.333498 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_653_133 to 202410_1_653_133 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:10.444536 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_653_133} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_648_132, 202410_653_653_0] -> 202410_1_653_133 2024.10.22 02:07:10.470388 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:07:10.932347 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_724_70, 202410_725_725_0] 2024.10.22 02:07:10.934474 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_724_70, 202410_725_725_0] 2024.10.22 02:07:10.952574 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_724_70, 202410_725_725_0] 2024.10.22 02:07:10.959929 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(eedb40d81e69c8167ffe3981ea1e7524) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: eedb40d81e69c8167ffe3981ea1e7524, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:10.961301 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: eedb40d81e69c8167ffe3981ea1e7524, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:10.963566 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:11.204365 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:07:11.287856 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:07:11.607544 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b1885cdadd06a178e5fe6d1bb9508c0b) : [0, 523420]: Setting download as failed: File segment: [0, 523420], key: b1885cdadd06a178e5fe6d1bb9508c0b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:11.612150 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b1885cdadd06a178e5fe6d1bb9508c0b): No space left in cache to reserve 523421 bytes, will continue without cache download 2024.10.22 02:07:11.671663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b1885cdadd06a178e5fe6d1bb9508c0b, offset: 0, size: 0 2024.10.22 02:07:11.674833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b1885cdadd06a178e5fe6d1bb9508c0b for removal 2024.10.22 02:07:11.681017 [ 1476 ] {} CacheMetadata: Key b1885cdadd06a178e5fe6d1bb9508c0b is removed from metadata 2024.10.22 02:07:11.686182 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4a3d69eff3d14bbe5889d58216d31864, offset: 0, size: 0 2024.10.22 02:07:11.691655 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4a3d69eff3d14bbe5889d58216d31864 for removal 2024.10.22 02:07:11.713140 [ 1476 ] {} CacheMetadata: Key 4a3d69eff3d14bbe5889d58216d31864 is removed from metadata 2024.10.22 02:07:12.067094 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 88530 2024.10.22 02:07:12.551200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(07fd0d90539f4ce46defd7a5e4b0fb54) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 07fd0d90539f4ce46defd7a5e4b0fb54, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:12.554962 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(07fd0d90539f4ce46defd7a5e4b0fb54): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 02:07:12.561267 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:12.564746 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:12.630884 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1342_1342_0 (state Temporary) 2024.10.22 02:07:12.668377 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1342_1342_0 to 202410_1355_1355_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:12.740905 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88530 2024.10.22 02:07:12.775289 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_760_144} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_760_144 to 202410_1_760_144 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:12.780111 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_760_144} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_754_143, 202410_755_760_1] -> 202410_1_760_144 2024.10.22 02:07:12.801458 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:07:13.112917 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 662 2024.10.22 02:07:13.167998 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_707_707_0, 202410_708_708_0, 202410_709_709_0, 202410_710_710_0, 202410_711_711_0, 202410_712_712_0] 2024.10.22 02:07:13.169363 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_707_707_0, 202410_708_708_0, 202410_709_709_0, 202410_710_710_0, 202410_711_711_0, 202410_712_712_0] 2024.10.22 02:07:13.301100 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_707_707_0, 202410_708_708_0, 202410_709_709_0, 202410_710_710_0, 202410_711_711_0, 202410_712_712_0] 2024.10.22 02:07:13.363468 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:13.366407 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:13.387412 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 8.50 MiB, will set to 1.56 GiB (RSS), difference: 3.50 MiB 2024.10.22 02:07:13.540951 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_644_644_0 (state Temporary) 2024.10.22 02:07:13.622599 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_644_644_0 to 202410_654_654_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:13.634132 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1149315 2024.10.22 02:07:13.693816 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 662 2024.10.22 02:07:13.788915 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(01112aaadf70a5d3ac24b37068ef638e) : [0, 513984]: Setting download as failed: File segment: [0, 513984], key: 01112aaadf70a5d3ac24b37068ef638e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:13.790648 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(01112aaadf70a5d3ac24b37068ef638e): No space left in cache to reserve 513985 bytes, will continue without cache download 2024.10.22 02:07:13.933494 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 01112aaadf70a5d3ac24b37068ef638e, offset: 0, size: 0 2024.10.22 02:07:13.937426 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 01112aaadf70a5d3ac24b37068ef638e for removal 2024.10.22 02:07:13.940611 [ 1476 ] {} CacheMetadata: Key 01112aaadf70a5d3ac24b37068ef638e is removed from metadata 2024.10.22 02:07:13.943639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 07fd0d90539f4ce46defd7a5e4b0fb54, offset: 0, size: 0 2024.10.22 02:07:13.945173 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 07fd0d90539f4ce46defd7a5e4b0fb54 for removal 2024.10.22 02:07:13.947531 [ 1476 ] {} CacheMetadata: Key 07fd0d90539f4ce46defd7a5e4b0fb54 is removed from metadata 2024.10.22 02:07:14.128585 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:14.132717 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:14.457682 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_762_762_0 (state Temporary) 2024.10.22 02:07:14.487632 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(effa0a8575e1817ff0fb1a331f1c80ee) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: effa0a8575e1817ff0fb1a331f1c80ee, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:14.489698 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 120664, file segment info: File segment: [0, 33554431], key: effa0a8575e1817ff0fb1a331f1c80ee, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:14.491929 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:14.643963 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_762_762_0 to 202410_769_769_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:14.675865 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1149315 2024.10.22 02:07:14.771329 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 87578 2024.10.22 02:07:15.036256 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:15.041158 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:15.200325 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_782_782_0 (state Temporary) 2024.10.22 02:07:15.256219 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_782_782_0 to 202410_788_788_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:15.263078 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(91d0e28475e255cc300d28fd2273efdd) : [0, 148]: Setting download as failed: File segment: [0, 148], key: 91d0e28475e255cc300d28fd2273efdd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:15.264456 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(91d0e28475e255cc300d28fd2273efdd): No space left in cache to reserve 149 bytes, will continue without cache download 2024.10.22 02:07:15.430195 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87578 2024.10.22 02:07:15.460265 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 12134 2024.10.22 02:07:15.656153 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:15.659080 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:15.693259 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_796_796_0 (state Temporary) 2024.10.22 02:07:15.859134 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_796_796_0 to 202410_809_809_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:15.879946 [ 1177 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_804_189 to 202410_809_809_0 2024.10.22 02:07:16.002094 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12134 2024.10.22 02:07:16.003555 [ 1177 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:16.011597 [ 1177 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:16.073382 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(7d30a2650fb4ed2506c5c4f382de8067) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7d30a2650fb4ed2506c5c4f382de8067, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:16.122985 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 484, file segment info: File segment: [0, 33554431], key: 7d30a2650fb4ed2506c5c4f382de8067, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:16.125497 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:16.149962 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_804_189 to 202410_809_809_0 into Compact with storage Full 2024.10.22 02:07:16.168101 [ 1707 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/njs/mfcewlufsigbfixowmpmqfkwppjnl, size 484 2024.10.22 02:07:16.279879 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:07:16.370478 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 7.53 MiB, will set to 1.56 GiB (RSS), difference: 4.62 MiB 2024.10.22 02:07:16.393059 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_804_189, total 12170 rows starting from the beginning of the part 2024.10.22 02:07:16.514663 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} MergeTreeSequentialSource: Reading 2 marks from part 202410_805_805_0, total 13 rows starting from the beginning of the part 2024.10.22 02:07:16.559805 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} MergeTreeSequentialSource: Reading 2 marks from part 202410_806_806_0, total 6 rows starting from the beginning of the part 2024.10.22 02:07:16.590705 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} MergeTreeSequentialSource: Reading 2 marks from part 202410_807_807_0, total 12 rows starting from the beginning of the part 2024.10.22 02:07:16.642633 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} MergeTreeSequentialSource: Reading 2 marks from part 202410_808_808_0, total 12 rows starting from the beginning of the part 2024.10.22 02:07:16.669252 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} MergeTreeSequentialSource: Reading 2 marks from part 202410_809_809_0, total 27 rows starting from the beginning of the part 2024.10.22 02:07:16.751997 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 88598 2024.10.22 02:07:16.772161 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_160_160_0 to 202410_163_163_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:16.868109 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3e7fbd0caf3de6af2e3fc95eda4fae9b) : [0, 300927]: Setting download as failed: File segment: [0, 300927], key: 3e7fbd0caf3de6af2e3fc95eda4fae9b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:16.870426 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3e7fbd0caf3de6af2e3fc95eda4fae9b): No space left in cache to reserve 300928 bytes, will continue without cache download 2024.10.22 02:07:17.109113 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3e7fbd0caf3de6af2e3fc95eda4fae9b, offset: 0, size: 0 2024.10.22 02:07:17.115793 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3e7fbd0caf3de6af2e3fc95eda4fae9b for removal 2024.10.22 02:07:17.119639 [ 1476 ] {} CacheMetadata: Key 3e7fbd0caf3de6af2e3fc95eda4fae9b is removed from metadata 2024.10.22 02:07:17.122366 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 91d0e28475e255cc300d28fd2273efdd, offset: 0, size: 0 2024.10.22 02:07:17.183707 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:17.187184 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:17.227075 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 91d0e28475e255cc300d28fd2273efdd for removal 2024.10.22 02:07:17.232143 [ 1476 ] {} CacheMetadata: Key 91d0e28475e255cc300d28fd2273efdd is removed from metadata 2024.10.22 02:07:17.361412 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1343_1343_0 (state Temporary) 2024.10.22 02:07:17.658812 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1343_1343_0 to 202410_1356_1356_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:17.735607 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88598 2024.10.22 02:07:18.094359 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(295570838d3db2f529a6e889b6f695d2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 295570838d3db2f529a6e889b6f695d2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:18.127512 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 268037, file segment info: File segment: [0, 33554431], key: 295570838d3db2f529a6e889b6f695d2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:18.131699 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:18.321423 [ 1194 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:07:18.558908 [ 1194 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:07:18.561929 [ 1194 ] {} DNSCacheUpdater: Execution took 241 ms. 2024.10.22 02:07:18.624379 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} MergeTask::MergeProjectionsStage: Merge sorted 12240 rows, containing 12 columns (12 merged, 0 gathered) in 2.567806989 sec., 4766.713406589299 rows/sec., 760.73 KiB/sec. 2024.10.22 02:07:18.959364 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_725_71, 202410_726_726_0] 2024.10.22 02:07:18.959801 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(52b9aa15e4933b773ce46a5b7962fe53) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 52b9aa15e4933b773ce46a5b7962fe53, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:07:18.960710 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 693, file segment info: File segment: [0, 33554431], key: 52b9aa15e4933b773ce46a5b7962fe53, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:07:18.960883 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_725_71, 202410_726_726_0] 2024.10.22 02:07:18.961924 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:18.965507 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} MergedBlockOutputStream: filled checksums 202410_1_809_190 (state Temporary) 2024.10.22 02:07:18.971836 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_725_71, 202410_726_726_0] 2024.10.22 02:07:19.019802 [ 1691 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qdi/njpumepiglfziqfnzrbgmiqityqhi, size 1740305 2024.10.22 02:07:19.096075 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_809_190 to 202410_1_809_190 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:19.101349 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_809_190} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_804_189, 202410_809_809_0] -> 202410_1_809_190 2024.10.22 02:07:19.145117 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.59 MiB. 2024.10.22 02:07:19.288498 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 02:07:19.298725 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 02:07:19.327984 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 02:07:19.447124 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 02:07:19.454522 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 02:07:19.466903 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 02:07:19.486502 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column WindowClientHeight 2024.10.22 02:07:19.628482 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nih/zfrawmfywdhkqiciiqlclmcbjxhzn (LogSeriesLimiter: on interval from 2024-10-22 02:07:02 to 2024-10-22 02:07:19 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:07:19.665574 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(1a83d9930e011d9489f15d66c9b83aba) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1a83d9930e011d9489f15d66c9b83aba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:19.673697 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 750, file segment info: File segment: [0, 33554431], key: 1a83d9930e011d9489f15d66c9b83aba, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:07:19.676362 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:20.076984 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6426 2024.10.22 02:07:20.197426 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:07:20.337446 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:07:20.358859 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:07:20.433652 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:07:20.443969 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 9.45 MiB, will set to 1.55 GiB (RSS), difference: 2.68 MiB 2024.10.22 02:07:20.465140 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:07:20.618886 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:07:20.692038 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:07:20.747264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(69dfd2ca4abc1b5696c371957afb73ac) : [0, 146]: Setting download as failed: File segment: [0, 146], key: 69dfd2ca4abc1b5696c371957afb73ac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:20.750163 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(69dfd2ca4abc1b5696c371957afb73ac): No space left in cache to reserve 147 bytes, will continue without cache download 2024.10.22 02:07:20.835493 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:07:21.204806 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:07:21.208909 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:07:21.682816 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1150470 2024.10.22 02:07:21.746721 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 88657 2024.10.22 02:07:21.900122 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3fc70238a45ae59d0ae155ff1d57d917) : [0, 214429]: Setting download as failed: File segment: [0, 214429], key: 3fc70238a45ae59d0ae155ff1d57d917, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:21.903186 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3fc70238a45ae59d0ae155ff1d57d917): No space left in cache to reserve 214430 bytes, will continue without cache download 2024.10.22 02:07:21.949455 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3fc70238a45ae59d0ae155ff1d57d917, offset: 0, size: 0 2024.10.22 02:07:21.951957 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3fc70238a45ae59d0ae155ff1d57d917 for removal 2024.10.22 02:07:21.956841 [ 1476 ] {} CacheMetadata: Key 3fc70238a45ae59d0ae155ff1d57d917 is removed from metadata 2024.10.22 02:07:21.957687 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 69dfd2ca4abc1b5696c371957afb73ac, offset: 0, size: 0 2024.10.22 02:07:21.960209 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 69dfd2ca4abc1b5696c371957afb73ac for removal 2024.10.22 02:07:21.963377 [ 1476 ] {} CacheMetadata: Key 69dfd2ca4abc1b5696c371957afb73ac is removed from metadata 2024.10.22 02:07:22.212774 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:22.214724 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:22.267378 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1344_1344_0 (state Temporary) 2024.10.22 02:07:22.329599 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1344_1344_0 to 202410_1357_1357_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:22.360781 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1352_29 to 202410_1357_1357_0 2024.10.22 02:07:22.368931 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88657 2024.10.22 02:07:22.371470 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:22.373819 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:22.411424 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1352_29 to 202410_1357_1357_0 into Compact with storage Full 2024.10.22 02:07:22.443694 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 8.77 MiB, will set to 1.56 GiB (RSS), difference: -6.11 MiB 2024.10.22 02:07:22.500970 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:07:22.525167 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1352_29, total 9168 rows starting from the beginning of the part 2024.10.22 02:07:22.657111 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_1353_1353_0, total 66 rows starting from the beginning of the part 2024.10.22 02:07:22.762566 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:22.766493 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:22.828326 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_1354_1354_0, total 76 rows starting from the beginning of the part 2024.10.22 02:07:22.908262 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_1355_1355_0, total 79 rows starting from the beginning of the part 2024.10.22 02:07:22.947623 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 62 entries to flush up to offset 87640 2024.10.22 02:07:22.982235 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_763_763_0 (state Temporary) 2024.10.22 02:07:22.995741 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_1356_1356_0, total 68 rows starting from the beginning of the part 2024.10.22 02:07:23.068837 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_763_763_0 to 202410_770_770_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:23.080493 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_1357_1357_0, total 59 rows starting from the beginning of the part 2024.10.22 02:07:23.104852 [ 1196 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_760_144 to 202410_761_766_1 2024.10.22 02:07:23.199952 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b0d9dc6da30293940dd8a2f589a5239e) : [0, 152]: Setting download as failed: File segment: [0, 152], key: b0d9dc6da30293940dd8a2f589a5239e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:23.202640 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b0d9dc6da30293940dd8a2f589a5239e): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 02:07:23.216975 [ 1196 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:23.225016 [ 1196 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:23.269166 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:23.269667 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_766_145} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_760_144 to 202410_761_766_1 into Wide with storage Full 2024.10.22 02:07:23.271324 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:23.312665 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1150470 2024.10.22 02:07:23.329072 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_783_783_0 (state Temporary) 2024.10.22 02:07:23.370746 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_783_783_0 to 202410_789_789_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:23.427257 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87640 2024.10.22 02:07:23.429597 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_766_145} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:07:23.445140 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_766_145} MergeTreeSequentialSource: Reading 142 marks from part 202410_1_760_144, total 1151740 rows starting from the beginning of the part 2024.10.22 02:07:23.467083 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_766_145} MergeTreeSequentialSource: Reading 2 marks from part 202410_761_766_1, total 7315 rows starting from the beginning of the part 2024.10.22 02:07:23.520787 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 12147 2024.10.22 02:07:23.933809 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:23.940533 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:23.968111 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_797_797_0 (state Temporary) 2024.10.22 02:07:24.032215 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_797_797_0 to 202410_810_810_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:24.070344 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12147 2024.10.22 02:07:24.290772 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 9.00 MiB, will set to 1.56 GiB (RSS), difference: -17.42 MiB 2024.10.22 02:07:24.388276 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(99733f79b3c7a95553746d650996af97) : [0, 484663]: Setting download as failed: File segment: [0, 484663], key: 99733f79b3c7a95553746d650996af97, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:24.392500 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(99733f79b3c7a95553746d650996af97): No space left in cache to reserve 484664 bytes, will continue without cache download 2024.10.22 02:07:24.732509 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 99733f79b3c7a95553746d650996af97, offset: 0, size: 0 2024.10.22 02:07:24.735425 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 99733f79b3c7a95553746d650996af97 for removal 2024.10.22 02:07:24.740485 [ 1476 ] {} CacheMetadata: Key 99733f79b3c7a95553746d650996af97 is removed from metadata 2024.10.22 02:07:24.743961 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b0d9dc6da30293940dd8a2f589a5239e, offset: 0, size: 0 2024.10.22 02:07:24.746882 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b0d9dc6da30293940dd8a2f589a5239e for removal 2024.10.22 02:07:24.755486 [ 1476 ] {} CacheMetadata: Key b0d9dc6da30293940dd8a2f589a5239e is removed from metadata 2024.10.22 02:07:25.083208 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} MergeTask::MergeProjectionsStage: Merge sorted 9516 rows, containing 13 columns (13 merged, 0 gathered) in 2.678827484 sec., 3552.300421298798 rows/sec., 629.75 KiB/sec. 2024.10.22 02:07:25.142994 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} MergedBlockOutputStream: filled checksums 202410_1207_1357_30 (state Temporary) 2024.10.22 02:07:25.196082 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1357_30 to 202410_1207_1357_30 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:25.302745 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1357_30} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1352_29, 202410_1357_1357_0] -> 202410_1207_1357_30 2024.10.22 02:07:25.352599 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.69 MiB. 2024.10.22 02:07:25.483915 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ec4eb777ed1ee4dce3ce63f8ce0c68e4) : [0, 151]: Setting download as failed: File segment: [0, 151], key: ec4eb777ed1ee4dce3ce63f8ce0c68e4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:25.485830 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ec4eb777ed1ee4dce3ce63f8ce0c68e4): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 02:07:26.381832 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_766_145/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:07:26.357769 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 13.61 MiB, will set to 1.56 GiB (RSS), difference: 10.69 MiB 2024.10.22 02:07:26.461342 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 335 ms. 2024.10.22 02:07:26.500326 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 88725 2024.10.22 02:07:26.734691 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(16d41fe80fe10eef5d76675f9d0803f5) : [0, 403640]: Setting download as failed: File segment: [0, 403640], key: 16d41fe80fe10eef5d76675f9d0803f5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:26.736233 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(16d41fe80fe10eef5d76675f9d0803f5): No space left in cache to reserve 403641 bytes, will continue without cache download 2024.10.22 02:07:26.787267 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 16d41fe80fe10eef5d76675f9d0803f5, offset: 0, size: 0 2024.10.22 02:07:26.790570 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 16d41fe80fe10eef5d76675f9d0803f5 for removal 2024.10.22 02:07:26.795482 [ 1476 ] {} CacheMetadata: Key 16d41fe80fe10eef5d76675f9d0803f5 is removed from metadata 2024.10.22 02:07:26.798240 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ec4eb777ed1ee4dce3ce63f8ce0c68e4, offset: 0, size: 0 2024.10.22 02:07:26.801394 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ec4eb777ed1ee4dce3ce63f8ce0c68e4 for removal 2024.10.22 02:07:26.808287 [ 1476 ] {} CacheMetadata: Key ec4eb777ed1ee4dce3ce63f8ce0c68e4 is removed from metadata 2024.10.22 02:07:26.877489 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:26.882241 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:26.947917 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1345_1345_0 (state Temporary) 2024.10.22 02:07:27.197563 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1345_1345_0 to 202410_1358_1358_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:27.446500 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88725 2024.10.22 02:07:27.555168 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(ce28ca56131bdf94a9319d65f0d76c18) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ce28ca56131bdf94a9319d65f0d76c18, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:07:27.578216 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: ce28ca56131bdf94a9319d65f0d76c18, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:07:27.583261 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:27.654697 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_726_72, 202410_727_727_0] 2024.10.22 02:07:27.741631 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 6466 2024.10.22 02:07:27.657719 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_726_72, 202410_727_727_0] 2024.10.22 02:07:27.769291 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_726_72, 202410_727_727_0] 2024.10.22 02:07:28.057008 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 4 old parts to remove. Parts: [202410_1_746_155, 202410_747_747_0, 202410_748_748_0, 202410_749_749_0] 2024.10.22 02:07:28.059710 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from filesystem (serially): Parts: [202410_1_746_155, 202410_747_747_0, 202410_748_748_0, 202410_749_749_0] 2024.10.22 02:07:28.089947 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from memory: Parts: [202410_1_746_155, 202410_747_747_0, 202410_748_748_0, 202410_749_749_0] 2024.10.22 02:07:28.243146 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1252_9, 202410_1253_1253_0, 202410_1254_1254_0, 202410_1255_1255_0, 202410_1256_1256_0, 202410_1257_1257_0] 2024.10.22 02:07:28.245622 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1252_9, 202410_1253_1253_0, 202410_1254_1254_0, 202410_1255_1255_0, 202410_1256_1256_0, 202410_1257_1257_0] 2024.10.22 02:07:28.379380 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1252_9, 202410_1253_1253_0, 202410_1254_1254_0, 202410_1255_1255_0, 202410_1256_1256_0, 202410_1257_1257_0] 2024.10.22 02:07:28.483312 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4276c7782256e23e80d50ee587cacce6) : [0, 145]: Setting download as failed: File segment: [0, 145], key: 4276c7782256e23e80d50ee587cacce6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:28.488445 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4276c7782256e23e80d50ee587cacce6): No space left in cache to reserve 146 bytes, will continue without cache download 2024.10.22 02:07:28.621658 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_700_134, 202410_701_706_1] 2024.10.22 02:07:28.629540 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_700_134, 202410_701_706_1] 2024.10.22 02:07:28.643780 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_700_134, 202410_701_706_1] 2024.10.22 02:07:28.699679 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 663 2024.10.22 02:07:28.700234 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 16.00 MiB, will set to 1.56 GiB (RSS), difference: 2.03 MiB 2024.10.22 02:07:29.115267 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:29.121748 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:29.211496 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_645_645_0 (state Temporary) 2024.10.22 02:07:29.456610 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(29b76d7b654a22b76b8fb7d7e3ef11ea) : [0, 183099]: Setting download as failed: File segment: [0, 183099], key: 29b76d7b654a22b76b8fb7d7e3ef11ea, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:29.550065 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_645_645_0 to 202410_655_655_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:29.552994 [ 1209 ] {} DatabaseCatalog: Execution took 275 ms. 2024.10.22 02:07:29.571560 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(29b76d7b654a22b76b8fb7d7e3ef11ea): No space left in cache to reserve 183100 bytes, will continue without cache download 2024.10.22 02:07:29.709613 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 29b76d7b654a22b76b8fb7d7e3ef11ea, offset: 0, size: 0 2024.10.22 02:07:29.713510 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 29b76d7b654a22b76b8fb7d7e3ef11ea for removal 2024.10.22 02:07:29.721148 [ 1476 ] {} CacheMetadata: Key 29b76d7b654a22b76b8fb7d7e3ef11ea is removed from metadata 2024.10.22 02:07:29.725151 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4276c7782256e23e80d50ee587cacce6, offset: 0, size: 0 2024.10.22 02:07:29.728275 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4276c7782256e23e80d50ee587cacce6 for removal 2024.10.22 02:07:29.732017 [ 1476 ] {} CacheMetadata: Key 4276c7782256e23e80d50ee587cacce6 is removed from metadata 2024.10.22 02:07:29.752277 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 663 2024.10.22 02:07:30.316902 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1152010 2024.10.22 02:07:31.066641 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e3172e9ff47c60b88a8d52e2f36aceb6) : [0, 145]: Setting download as failed: File segment: [0, 145], key: e3172e9ff47c60b88a8d52e2f36aceb6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:31.068109 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e3172e9ff47c60b88a8d52e2f36aceb6): No space left in cache to reserve 146 bytes, will continue without cache download 2024.10.22 02:07:31.100897 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:31.110530 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:31.154442 [ 1256 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 02:07:31.209015 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:07:31.224663 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:07:31.342551 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 92 entries to flush up to offset 87732 2024.10.22 02:07:31.373618 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_764_764_0 (state Temporary) 2024.10.22 02:07:31.470646 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 88786 2024.10.22 02:07:31.473816 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_764_764_0 to 202410_771_771_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:31.528747 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1152010 2024.10.22 02:07:31.573337 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 12158 2024.10.22 02:07:31.583284 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 14.01 MiB, will set to 1.57 GiB (RSS), difference: 2.63 MiB 2024.10.22 02:07:31.643996 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:31.650504 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:31.746498 [ 1268 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 02:07:31.807418 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1346_1346_0 (state Temporary) 2024.10.22 02:07:31.881211 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:31.881658 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1346_1346_0 to 202410_1359_1359_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:31.889318 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:31.946091 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_798_798_0 (state Temporary) 2024.10.22 02:07:31.966842 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88786 2024.10.22 02:07:31.972407 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2593f1529fbeb6296c0fb7e92b9bdb29) : [0, 182341]: Setting download as failed: File segment: [0, 182341], key: 2593f1529fbeb6296c0fb7e92b9bdb29, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:31.974690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2593f1529fbeb6296c0fb7e92b9bdb29): No space left in cache to reserve 182342 bytes, will continue without cache download 2024.10.22 02:07:32.011816 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_798_798_0 to 202410_811_811_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:32.014700 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2593f1529fbeb6296c0fb7e92b9bdb29, offset: 0, size: 0 2024.10.22 02:07:32.016738 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2593f1529fbeb6296c0fb7e92b9bdb29 for removal 2024.10.22 02:07:32.020109 [ 1476 ] {} CacheMetadata: Key 2593f1529fbeb6296c0fb7e92b9bdb29 is removed from metadata 2024.10.22 02:07:32.022545 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e3172e9ff47c60b88a8d52e2f36aceb6, offset: 0, size: 0 2024.10.22 02:07:32.030389 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e3172e9ff47c60b88a8d52e2f36aceb6 for removal 2024.10.22 02:07:32.085120 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12158 2024.10.22 02:07:32.041780 [ 1476 ] {} CacheMetadata: Key e3172e9ff47c60b88a8d52e2f36aceb6 is removed from metadata 2024.10.22 02:07:32.216812 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:32.224643 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:32.297115 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_784_784_0 (state Temporary) 2024.10.22 02:07:32.364484 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_784_784_0 to 202410_790_790_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:32.515508 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87732 2024.10.22 02:07:32.955710 [ 1260 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 02:07:33.285970 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f6b13d2b15e6248f42b9ec1ea07ab1a8) : [0, 145]: Setting download as failed: File segment: [0, 145], key: f6b13d2b15e6248f42b9ec1ea07ab1a8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:33.288404 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f6b13d2b15e6248f42b9ec1ea07ab1a8): No space left in cache to reserve 146 bytes, will continue without cache download 2024.10.22 02:07:33.562787 [ 1226 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:07:33.562705 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.54 MiB, will set to 1.56 GiB (RSS), difference: -1.92 MiB 2024.10.22 02:07:33.592801 [ 1226 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:07:34.423817 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(91b8ebaac581a0d422e0213025cb44b1) : [0, 153581]: Setting download as failed: File segment: [0, 153581], key: 91b8ebaac581a0d422e0213025cb44b1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:34.428215 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(91b8ebaac581a0d422e0213025cb44b1): No space left in cache to reserve 153582 bytes, will continue without cache download 2024.10.22 02:07:34.662751 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 91b8ebaac581a0d422e0213025cb44b1, offset: 0, size: 0 2024.10.22 02:07:34.766895 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 91b8ebaac581a0d422e0213025cb44b1 for removal 2024.10.22 02:07:34.769147 [ 1476 ] {} CacheMetadata: Key 91b8ebaac581a0d422e0213025cb44b1 is removed from metadata 2024.10.22 02:07:34.770757 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f6b13d2b15e6248f42b9ec1ea07ab1a8, offset: 0, size: 0 2024.10.22 02:07:34.772089 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f6b13d2b15e6248f42b9ec1ea07ab1a8 for removal 2024.10.22 02:07:34.775061 [ 1476 ] {} CacheMetadata: Key f6b13d2b15e6248f42b9ec1ea07ab1a8 is removed from metadata 2024.10.22 02:07:35.399475 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_727_73, 202410_728_728_0] 2024.10.22 02:07:35.402889 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_727_73, 202410_728_728_0] 2024.10.22 02:07:35.424315 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_727_73, 202410_728_728_0] 2024.10.22 02:07:35.855926 [ 1261 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 02:07:35.921909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fc821ad3ae86aba338e0aec842fdf081) : [0, 151]: Setting download as failed: File segment: [0, 151], key: fc821ad3ae86aba338e0aec842fdf081, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:35.924097 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fc821ad3ae86aba338e0aec842fdf081): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 02:07:35.975247 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 88835 2024.10.22 02:07:36.347551 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:36.351363 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:36.395445 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1347_1347_0 (state Temporary) 2024.10.22 02:07:36.558592 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1347_1347_0 to 202410_1360_1360_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:36.617043 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88835 2024.10.22 02:07:36.734535 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 17.08 MiB, will set to 1.57 GiB (RSS), difference: 5.05 MiB 2024.10.22 02:07:36.881524 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d8449b882651269c4ae6c818c604bad9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d8449b882651269c4ae6c818c604bad9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:07:36.899741 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_766_145/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:07:36.884505 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 697, file segment info: File segment: [0, 33554431], key: d8449b882651269c4ae6c818c604bad9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:07:36.911597 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:37.093491 [ 1698 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nih/zfrawmfywdhkqiciiqlclmcbjxhzn, size 1863307 2024.10.22 02:07:37.146368 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 02:07:37.150294 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 02:07:37.169323 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 02:07:37.189324 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 02:07:37.197454 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 02:07:37.224951 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 02:07:37.235685 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ClientTimeZone 2024.10.22 02:07:37.254034 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 664 2024.10.22 02:07:37.282557 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wsf/yueyqftsjtcrvrfpkyyjhhgrcgogg (LogSeriesLimiter: on interval from 2024-10-22 02:07:19 to 2024-10-22 02:07:37 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:07:37.562491 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:37.567170 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:37.850388 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_646_646_0 (state Temporary) 2024.10.22 02:07:37.955858 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c70e08dfd9412ee8e3ec20211b525bd8) : [0, 429252]: Setting download as failed: File segment: [0, 429252], key: c70e08dfd9412ee8e3ec20211b525bd8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:37.957209 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c70e08dfd9412ee8e3ec20211b525bd8): No space left in cache to reserve 429253 bytes, will continue without cache download 2024.10.22 02:07:38.081917 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c70e08dfd9412ee8e3ec20211b525bd8, offset: 0, size: 0 2024.10.22 02:07:38.111815 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c70e08dfd9412ee8e3ec20211b525bd8 for removal 2024.10.22 02:07:38.119014 [ 1476 ] {} CacheMetadata: Key c70e08dfd9412ee8e3ec20211b525bd8 is removed from metadata 2024.10.22 02:07:38.129575 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_646_646_0 to 202410_656_656_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:38.130463 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fc821ad3ae86aba338e0aec842fdf081, offset: 0, size: 0 2024.10.22 02:07:38.133207 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fc821ad3ae86aba338e0aec842fdf081 for removal 2024.10.22 02:07:38.141480 [ 1476 ] {} CacheMetadata: Key fc821ad3ae86aba338e0aec842fdf081 is removed from metadata 2024.10.22 02:07:38.307996 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 664 2024.10.22 02:07:38.541042 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1078 entries to flush up to offset 1153088 2024.10.22 02:07:39.261050 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b736a4270e1fe3697e94acf597e64758) : [0, 146]: Setting download as failed: File segment: [0, 146], key: b736a4270e1fe3697e94acf597e64758, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:39.263422 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b736a4270e1fe3697e94acf597e64758): No space left in cache to reserve 147 bytes, will continue without cache download 2024.10.22 02:07:39.591453 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 12163 2024.10.22 02:07:39.672348 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 13.84 MiB, will set to 1.57 GiB (RSS), difference: -4.28 MiB 2024.10.22 02:07:39.756584 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:39.762287 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:39.787063 [ 1237 ] {} DatabaseCatalog: Execution took 245 ms. 2024.10.22 02:07:39.822383 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_799_799_0 (state Temporary) 2024.10.22 02:07:39.875172 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_799_799_0 to 202410_812_812_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:39.947396 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12163 2024.10.22 02:07:40.070897 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 54 entries to flush up to offset 87786 2024.10.22 02:07:40.199982 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:40.203717 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:40.309254 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:40.314459 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:40.409448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8ef90887bf3d4b0e00911cb92a35116a) : [0, 258577]: Setting download as failed: File segment: [0, 258577], key: 8ef90887bf3d4b0e00911cb92a35116a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:40.411587 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8ef90887bf3d4b0e00911cb92a35116a): No space left in cache to reserve 258578 bytes, will continue without cache download 2024.10.22 02:07:40.423455 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_765_765_0 (state Temporary) 2024.10.22 02:07:40.491216 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_785_785_0 (state Temporary) 2024.10.22 02:07:40.555137 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_785_785_0 to 202410_791_791_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:40.595295 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8ef90887bf3d4b0e00911cb92a35116a, offset: 0, size: 0 2024.10.22 02:07:40.599028 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8ef90887bf3d4b0e00911cb92a35116a for removal 2024.10.22 02:07:40.603048 [ 1476 ] {} CacheMetadata: Key 8ef90887bf3d4b0e00911cb92a35116a is removed from metadata 2024.10.22 02:07:40.606469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b736a4270e1fe3697e94acf597e64758, offset: 0, size: 0 2024.10.22 02:07:40.607642 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b736a4270e1fe3697e94acf597e64758 for removal 2024.10.22 02:07:40.610418 [ 1476 ] {} CacheMetadata: Key b736a4270e1fe3697e94acf597e64758 is removed from metadata 2024.10.22 02:07:40.628151 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 88896 2024.10.22 02:07:40.673495 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_765_765_0 to 202410_772_772_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:40.716539 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_767_767_0 to 202410_772_772_0 2024.10.22 02:07:40.828923 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87786 2024.10.22 02:07:40.833482 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:40.836893 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:40.866107 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_767_767_0 to 202410_772_772_0 into Compact with storage Full 2024.10.22 02:07:40.934894 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1153088 2024.10.22 02:07:41.055273 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:07:41.132551 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_767_767_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:07:41.162516 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_768_768_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:07:41.175927 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:41.183257 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:41.214785 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:07:41.214384 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_769_769_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:07:41.225631 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:07:41.266061 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_770_770_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:07:41.320806 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_771_771_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:07:41.364021 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_772_772_0, total 1078 rows starting from the beginning of the part 2024.10.22 02:07:41.423270 [ 1173 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 02:07:41.471583 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1348_1348_0 (state Temporary) 2024.10.22 02:07:41.546466 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1348_1348_0 to 202410_1361_1361_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:41.716013 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88896 2024.10.22 02:07:42.225435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(79d6e0eefc75ee757d922d163ec24a32) : [0, 153]: Setting download as failed: File segment: [0, 153], key: 79d6e0eefc75ee757d922d163ec24a32, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:42.230776 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(79d6e0eefc75ee757d922d163ec24a32): No space left in cache to reserve 154 bytes, will continue without cache download 2024.10.22 02:07:42.507811 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_603_123, 202410_604_604_0, 202410_605_605_0, 202410_606_606_0, 202410_607_607_0, 202410_608_608_0] 2024.10.22 02:07:42.533173 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_728_74, 202410_729_729_0] 2024.10.22 02:07:42.536059 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_728_74, 202410_729_729_0] 2024.10.22 02:07:42.517719 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_603_123, 202410_604_604_0, 202410_605_605_0, 202410_606_606_0, 202410_607_607_0, 202410_608_608_0] 2024.10.22 02:07:42.550718 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_728_74, 202410_729_729_0] 2024.10.22 02:07:42.567091 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_603_123, 202410_604_604_0, 202410_605_605_0, 202410_606_606_0, 202410_607_607_0, 202410_608_608_0] 2024.10.22 02:07:42.645807 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 13.48 MiB, will set to 1.56 GiB (RSS), difference: -10.27 MiB 2024.10.22 02:07:42.700858 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} MergeTask::MergeProjectionsStage: Merge sorted 7623 rows, containing 4 columns (4 merged, 0 gathered) in 1.849084312 sec., 4122.58108001297 rows/sec., 64.42 KiB/sec. 2024.10.22 02:07:42.886246 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} MergedBlockOutputStream: filled checksums 202410_767_772_1 (state Temporary) 2024.10.22 02:07:42.975606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ff6dffaf9f9fdde793595903cb7c8099) : [0, 1267]: Setting download as failed: File segment: [0, 1267], key: ff6dffaf9f9fdde793595903cb7c8099, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:42.982527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ff6dffaf9f9fdde793595903cb7c8099): No space left in cache to reserve 1268 bytes, will continue without cache download 2024.10.22 02:07:42.975606 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_767_772_1 to 202410_767_772_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:43.008206 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_767_772_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_767_767_0, 202410_772_772_0] -> 202410_767_772_1 2024.10.22 02:07:43.075560 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:07:43.101713 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ff6dffaf9f9fdde793595903cb7c8099, offset: 0, size: 0 2024.10.22 02:07:43.103849 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ff6dffaf9f9fdde793595903cb7c8099 for removal 2024.10.22 02:07:43.107223 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 79d6e0eefc75ee757d922d163ec24a32, offset: 0, size: 0 2024.10.22 02:07:43.108529 [ 1476 ] {} CacheMetadata: Key ff6dffaf9f9fdde793595903cb7c8099 is removed from metadata 2024.10.22 02:07:43.109567 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 79d6e0eefc75ee757d922d163ec24a32 for removal 2024.10.22 02:07:43.134752 [ 1476 ] {} CacheMetadata: Key 79d6e0eefc75ee757d922d163ec24a32 is removed from metadata 2024.10.22 02:07:44.368996 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1c4763fa1de949ec3ce5d63e2cf3c5da) : [0, 187]: Setting download as failed: File segment: [0, 187], key: 1c4763fa1de949ec3ce5d63e2cf3c5da, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:44.371144 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1c4763fa1de949ec3ce5d63e2cf3c5da): No space left in cache to reserve 188 bytes, will continue without cache download 2024.10.22 02:07:44.556057 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 12.65 MiB, will set to 1.56 GiB (RSS), difference: 4.47 MiB 2024.10.22 02:07:45.470927 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:45.479515 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:45.596697 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f608d174e666e7ecf7ccf355d09323a8) : [0, 966413]: Setting download as failed: File segment: [0, 966413], key: f608d174e666e7ecf7ccf355d09323a8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:45.600121 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f608d174e666e7ecf7ccf355d09323a8): No space left in cache to reserve 966414 bytes, will continue without cache download 2024.10.22 02:07:45.660840 [ 1193 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 02:07:45.732234 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 88949 2024.10.22 02:07:45.810134 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 665 2024.10.22 02:07:45.855221 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f608d174e666e7ecf7ccf355d09323a8, offset: 0, size: 0 2024.10.22 02:07:45.857472 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f608d174e666e7ecf7ccf355d09323a8 for removal 2024.10.22 02:07:45.860384 [ 1476 ] {} CacheMetadata: Key f608d174e666e7ecf7ccf355d09323a8 is removed from metadata 2024.10.22 02:07:45.863223 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1c4763fa1de949ec3ce5d63e2cf3c5da, offset: 0, size: 0 2024.10.22 02:07:45.868526 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1c4763fa1de949ec3ce5d63e2cf3c5da for removal 2024.10.22 02:07:45.874676 [ 1476 ] {} CacheMetadata: Key 1c4763fa1de949ec3ce5d63e2cf3c5da is removed from metadata 2024.10.22 02:07:45.974821 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:45.978234 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:46.157600 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1349_1349_0 (state Temporary) 2024.10.22 02:07:46.350913 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:46.354117 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:46.378414 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1349_1349_0 to 202410_1362_1362_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:46.429027 [ 1199 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1357_30 to 202410_1362_1362_0 2024.10.22 02:07:46.457703 [ 1199 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:46.460605 [ 1199 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:46.480007 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1357_30 to 202410_1362_1362_0 into Compact with storage Full 2024.10.22 02:07:46.509786 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_647_647_0 (state Temporary) 2024.10.22 02:07:46.567167 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 88949 2024.10.22 02:07:46.601580 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:07:46.634262 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1357_30, total 9516 rows starting from the beginning of the part 2024.10.22 02:07:46.723368 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_647_647_0 to 202410_657_657_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:46.814925 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_1358_1358_0, total 68 rows starting from the beginning of the part 2024.10.22 02:07:46.909069 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_1359_1359_0, total 61 rows starting from the beginning of the part 2024.10.22 02:07:46.951793 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 665 2024.10.22 02:07:46.975084 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(55b52bd72c0085067be2cad237c74993) : [0, 191]: Setting download as failed: File segment: [0, 191], key: 55b52bd72c0085067be2cad237c74993, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:46.976465 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(55b52bd72c0085067be2cad237c74993): No space left in cache to reserve 192 bytes, will continue without cache download 2024.10.22 02:07:47.087927 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_1360_1360_0, total 49 rows starting from the beginning of the part 2024.10.22 02:07:47.164083 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_1361_1361_0, total 61 rows starting from the beginning of the part 2024.10.22 02:07:47.297822 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_1362_1362_0, total 53 rows starting from the beginning of the part 2024.10.22 02:07:47.450495 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 12176 2024.10.22 02:07:47.612053 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.41 MiB, will set to 1.56 GiB (RSS), difference: -2.20 MiB 2024.10.22 02:07:47.870146 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_766_145/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:07:47.944881 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1232 entries to flush up to offset 1154320 2024.10.22 02:07:47.951647 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(1a150f66c4a6576129f351f646efeea4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1a150f66c4a6576129f351f646efeea4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:07:47.958453 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 43132, file segment info: File segment: [0, 33554431], key: 1a150f66c4a6576129f351f646efeea4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:07:47.962101 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:48.029740 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:48.035149 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:48.294910 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_800_800_0 (state Temporary) 2024.10.22 02:07:48.356187 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_800_800_0 to 202410_813_813_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:48.425993 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12176 2024.10.22 02:07:48.546025 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 80 entries to flush up to offset 87866 2024.10.22 02:07:48.615917 [ 1214 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:07:48.665355 [ 1214 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:07:48.691358 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:48.700700 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:48.830483 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_766_766_0 (state Temporary) 2024.10.22 02:07:48.918154 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:48.922693 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:48.960241 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(71ca0ee48967e5c945b9c76d1ae1ab43) : [0, 1213670]: Setting download as failed: File segment: [0, 1213670], key: 71ca0ee48967e5c945b9c76d1ae1ab43, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:48.961102 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_161_161_0 (state Temporary) 2024.10.22 02:07:48.962258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(71ca0ee48967e5c945b9c76d1ae1ab43): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:07:48.965323 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_766_766_0 to 202410_773_773_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:48.967865 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_786_786_0 (state Temporary) 2024.10.22 02:07:49.007057 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1154320 2024.10.22 02:07:49.130184 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_786_786_0 to 202410_792_792_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:49.168206 [ 1207 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_787_8 to 202410_792_792_0 2024.10.22 02:07:49.183502 [ 1207 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:49.190656 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:49.216377 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_787_8 to 202410_792_792_0 into Compact with storage Full 2024.10.22 02:07:49.249622 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 71ca0ee48967e5c945b9c76d1ae1ab43, offset: 0, size: 0 2024.10.22 02:07:49.252147 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 71ca0ee48967e5c945b9c76d1ae1ab43 for removal 2024.10.22 02:07:49.255815 [ 1476 ] {} CacheMetadata: Key 71ca0ee48967e5c945b9c76d1ae1ab43 is removed from metadata 2024.10.22 02:07:49.295035 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87866 2024.10.22 02:07:49.360063 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 55b52bd72c0085067be2cad237c74993, offset: 0, size: 0 2024.10.22 02:07:49.363137 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 55b52bd72c0085067be2cad237c74993 for removal 2024.10.22 02:07:49.368355 [ 1476 ] {} CacheMetadata: Key 55b52bd72c0085067be2cad237c74993 is removed from metadata 2024.10.22 02:07:49.399601 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:07:49.437777 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_787_8, total 4132 rows starting from the beginning of the part 2024.10.22 02:07:49.649234 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_788_788_0, total 91 rows starting from the beginning of the part 2024.10.22 02:07:49.656182 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:07:49.693038 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} MergeTask::MergeProjectionsStage: Merge sorted 9808 rows, containing 13 columns (13 merged, 0 gathered) in 3.220559307 sec., 3045.433747697788 rows/sec., 538.96 KiB/sec. 2024.10.22 02:07:49.858281 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_789_789_0, total 62 rows starting from the beginning of the part 2024.10.22 02:07:49.861602 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} MergedBlockOutputStream: filled checksums 202410_1207_1362_31 (state Temporary) 2024.10.22 02:07:49.974590 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1362_31 to 202410_1207_1362_31 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:49.999174 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1362_31} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1357_30, 202410_1362_1362_0] -> 202410_1207_1362_31 2024.10.22 02:07:50.001689 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(cce833800d0365dff389c2ace19cabf4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cce833800d0365dff389c2ace19cabf4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:07:50.011741 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 449, file segment info: File segment: [0, 33554431], key: cce833800d0365dff389c2ace19cabf4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:07:50.016437 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:50.056138 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.68 MiB. 2024.10.22 02:07:50.060391 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_790_790_0, total 92 rows starting from the beginning of the part 2024.10.22 02:07:50.113699 [ 1213 ] {} DatabaseCatalog: Execution took 325 ms. 2024.10.22 02:07:50.145225 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_791_791_0, total 54 rows starting from the beginning of the part 2024.10.22 02:07:50.212174 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_792_792_0, total 80 rows starting from the beginning of the part 2024.10.22 02:07:50.579407 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 89035 2024.10.22 02:07:50.938042 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1257_10, 202410_1258_1258_0, 202410_1259_1259_0, 202410_1260_1260_0, 202410_1261_1261_0, 202410_1262_1262_0] 2024.10.22 02:07:50.941094 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1257_10, 202410_1258_1258_0, 202410_1259_1259_0, 202410_1260_1260_0, 202410_1261_1261_0, 202410_1262_1262_0] 2024.10.22 02:07:50.963157 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1257_10, 202410_1258_1258_0, 202410_1259_1259_0, 202410_1260_1260_0, 202410_1261_1261_0, 202410_1262_1262_0] 2024.10.22 02:07:50.967316 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:50.973127 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:51.051206 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bb283ee8dcb9c877f276d381dd2e9a97) : [0, 151]: Setting download as failed: File segment: [0, 151], key: bb283ee8dcb9c877f276d381dd2e9a97, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:51.053334 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bb283ee8dcb9c877f276d381dd2e9a97): No space left in cache to reserve 152 bytes, will continue without cache download 2024.10.22 02:07:51.104413 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1350_1350_0 (state Temporary) 2024.10.22 02:07:51.163353 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1350_1350_0 to 202410_1363_1363_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:51.221128 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:07:51.231945 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:07:51.286800 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 4 old parts to remove. Parts: [202410_1_749_156, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0] 2024.10.22 02:07:51.290154 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from filesystem (serially): Parts: [202410_1_749_156, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0] 2024.10.22 02:07:51.332946 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 4 parts from memory: Parts: [202410_1_749_156, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0] 2024.10.22 02:07:51.359574 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89035 2024.10.22 02:07:51.667379 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_792_9/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:07:51.725200 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_729_75, 202410_730_730_0] 2024.10.22 02:07:51.827715 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_729_75, 202410_730_730_0] 2024.10.22 02:07:51.852961 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_729_75, 202410_730_730_0] 2024.10.22 02:07:52.171261 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} MergeTask::MergeProjectionsStage: Merge sorted 4511 rows, containing 13 columns (13 merged, 0 gathered) in 2.973321791 sec., 1517.1583559016133 rows/sec., 672.90 KiB/sec. 2024.10.22 02:07:52.286630 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(ecdc2f899cbb9c16b05a74b96beb1a72) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ecdc2f899cbb9c16b05a74b96beb1a72, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:07:52.292986 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 268037, file segment info: File segment: [0, 33554431], key: ecdc2f899cbb9c16b05a74b96beb1a72, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:07:52.298506 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:52.557070 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} MergedBlockOutputStream: filled checksums 202410_747_792_9 (state Temporary) 2024.10.22 02:07:53.052650 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_792_9 to 202410_747_792_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:53.056224 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_792_9} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_787_8, 202410_792_792_0] -> 202410_747_792_9 2024.10.22 02:07:53.233771 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6e1fe79d4d9630fd3e1aa399c2de4305) : [0, 434263]: Setting download as failed: File segment: [0, 434263], key: 6e1fe79d4d9630fd3e1aa399c2de4305, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:53.237684 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6e1fe79d4d9630fd3e1aa399c2de4305): No space left in cache to reserve 434264 bytes, will continue without cache download 2024.10.22 02:07:53.307212 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.16 MiB. 2024.10.22 02:07:53.376864 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6e1fe79d4d9630fd3e1aa399c2de4305, offset: 0, size: 0 2024.10.22 02:07:53.379859 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6e1fe79d4d9630fd3e1aa399c2de4305 for removal 2024.10.22 02:07:53.388025 [ 1476 ] {} CacheMetadata: Key 6e1fe79d4d9630fd3e1aa399c2de4305 is removed from metadata 2024.10.22 02:07:53.392440 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bb283ee8dcb9c877f276d381dd2e9a97, offset: 0, size: 0 2024.10.22 02:07:53.396494 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bb283ee8dcb9c877f276d381dd2e9a97 for removal 2024.10.22 02:07:53.402165 [ 1476 ] {} CacheMetadata: Key bb283ee8dcb9c877f276d381dd2e9a97 is removed from metadata 2024.10.22 02:07:54.453579 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 666 2024.10.22 02:07:54.560240 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.45 MiB, will set to 1.57 GiB (RSS), difference: 6.55 MiB 2024.10.22 02:07:54.662019 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(4a4660b6315734677be8d356bf27f73b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4a4660b6315734677be8d356bf27f73b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:07:54.664611 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 958171, file segment info: File segment: [0, 33554431], key: 4a4660b6315734677be8d356bf27f73b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:07:54.668052 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:54.851802 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:54.855205 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:55.196392 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_648_648_0 (state Temporary) 2024.10.22 02:07:55.299106 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_648_648_0 to 202410_658_658_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:55.341502 [ 1260 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_653_133 to 202410_658_658_0 2024.10.22 02:07:55.353055 [ 1260 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:55.355814 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:55.368624 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 89083 2024.10.22 02:07:55.395342 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_653_133 to 202410_658_658_0 into Compact with storage Full 2024.10.22 02:07:55.479512 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 666 2024.10.22 02:07:55.644745 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:07:55.657622 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_653_133, total 697 rows starting from the beginning of the part 2024.10.22 02:07:55.662603 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(53ee5cc328cfa24ac8e77f783593b91b) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 53ee5cc328cfa24ac8e77f783593b91b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:55.664225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(53ee5cc328cfa24ac8e77f783593b91b): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 02:07:55.789243 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:55.793923 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:55.820644 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_654_654_0, total 1 rows starting from the beginning of the part 2024.10.22 02:07:55.842329 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1351_1351_0 (state Temporary) 2024.10.22 02:07:55.930732 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 12195 2024.10.22 02:07:55.994417 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1351_1351_0 to 202410_1364_1364_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:56.008624 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_655_655_0, total 1 rows starting from the beginning of the part 2024.10.22 02:07:56.013422 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1155090 2024.10.22 02:07:56.086933 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_656_656_0, total 1 rows starting from the beginning of the part 2024.10.22 02:07:56.200781 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:56.205157 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:56.257524 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_801_801_0 (state Temporary) 2024.10.22 02:07:56.295565 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89083 2024.10.22 02:07:56.334827 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_801_801_0 to 202410_814_814_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:56.353732 [ 1257 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_809_190 to 202410_810_810_0 2024.10.22 02:07:56.355819 [ 1257 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:56.362932 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12195 2024.10.22 02:07:56.357556 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:56.376401 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_657_657_0, total 1 rows starting from the beginning of the part 2024.10.22 02:07:56.385162 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_810_191} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_809_190 to 202410_810_810_0 into Compact with storage Full 2024.10.22 02:07:56.448007 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(640365821c5cb491f027aa3d217cc50b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 640365821c5cb491f027aa3d217cc50b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:07:56.451524 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 677, file segment info: File segment: [0, 33554431], key: 640365821c5cb491f027aa3d217cc50b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:07:56.454612 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:56.495044 [ 1681 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wsf/yueyqftsjtcrvrfpkyyjhhgrcgogg, size 958171 2024.10.22 02:07:56.575971 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_810_191} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:07:56.598797 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:56.604095 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:56.628488 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_810_191} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_809_190, total 12240 rows starting from the beginning of the part 2024.10.22 02:07:56.643763 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_658_658_0, total 1 rows starting from the beginning of the part 2024.10.22 02:07:56.674474 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_810_191} MergeTreeSequentialSource: Reading 2 marks from part 202410_810_810_0, total 13 rows starting from the beginning of the part 2024.10.22 02:07:56.812065 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 74 entries to flush up to offset 87940 2024.10.22 02:07:56.846049 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_767_767_0 (state Temporary) 2024.10.22 02:07:56.875254 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 02:07:56.897373 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 02:07:56.921221 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 02:07:56.954344 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 02:07:56.957307 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:07:56.960165 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:07:56.974992 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 02:07:57.010390 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_767_767_0 to 202410_774_774_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:57.090778 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 02:07:57.120250 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ClientEventTime 2024.10.22 02:07:57.158518 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_787_787_0 (state Temporary) 2024.10.22 02:07:57.159803 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1155090 2024.10.22 02:07:57.231596 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fxu/cspaoypeyqlnxrrguhxiejjhqvmws (LogSeriesLimiter: on interval from 2024-10-22 02:07:37 to 2024-10-22 02:07:57 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:07:57.232208 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_787_787_0 to 202410_793_793_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:57.271857 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(79035f84b076ec44f727c9361fd25d80) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 79035f84b076ec44f727c9361fd25d80, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:07:57.278881 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 750, file segment info: File segment: [0, 33554431], key: 79035f84b076ec44f727c9361fd25d80, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1397, kind: Regular, unbound: 0 2024.10.22 02:07:57.284578 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:07:57.298971 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 87940 2024.10.22 02:07:57.338465 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_161_161_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:07:57.681664 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_810_191} MergeTask::MergeProjectionsStage: Merge sorted 12253 rows, containing 12 columns (12 merged, 0 gathered) in 1.293518429 sec., 9472.6133971455 rows/sec., 1.48 MiB/sec. 2024.10.22 02:07:57.857455 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_810_191} MergedBlockOutputStream: filled checksums 202410_1_810_191 (state Temporary) 2024.10.22 02:07:57.932280 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 02:07:57.947014 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 02:07:58.002430 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 02:07:58.053711 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_810_191} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_810_191 to 202410_1_810_191 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:58.059542 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_810_191} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_809_190, 202410_810_810_0] -> 202410_1_810_191 2024.10.22 02:07:58.073696 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:07:58.154966 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 02:07:58.202589 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 02:07:58.214857 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a4295ff707c90cc7fd98f6a5222be828) : [0, 105218]: Setting download as failed: File segment: [0, 105218], key: a4295ff707c90cc7fd98f6a5222be828, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:07:58.218483 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a4295ff707c90cc7fd98f6a5222be828): No space left in cache to reserve 105219 bytes, will continue without cache download 2024.10.22 02:07:58.258671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a4295ff707c90cc7fd98f6a5222be828, offset: 0, size: 0 2024.10.22 02:07:58.260607 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a4295ff707c90cc7fd98f6a5222be828 for removal 2024.10.22 02:07:58.264287 [ 1476 ] {} CacheMetadata: Key a4295ff707c90cc7fd98f6a5222be828 is removed from metadata 2024.10.22 02:07:58.364684 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 53ee5cc328cfa24ac8e77f783593b91b, offset: 0, size: 0 2024.10.22 02:07:58.364402 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 02:07:58.366704 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 53ee5cc328cfa24ac8e77f783593b91b for removal 2024.10.22 02:07:58.367422 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_730_76, 202410_731_731_0] 2024.10.22 02:07:58.368041 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mmo/vkxvzsmlryvinqeeorwqvyfftfgkw. 2024.10.22 02:07:58.368415 [ 1476 ] {} CacheMetadata: Key 53ee5cc328cfa24ac8e77f783593b91b is removed from metadata 2024.10.22 02:07:58.369304 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_730_76, 202410_731_731_0] 2024.10.22 02:07:58.443353 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 02:07:58.491792 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ParsedParams.Key4 2024.10.22 02:07:58.516783 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uaz/ndrrbelygdxgcuzkhxftfmpxgefda. 2024.10.22 02:07:58.527952 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_730_76, 202410_731_731_0] 2024.10.22 02:07:58.651889 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} MergeTask::MergeProjectionsStage: Merge sorted 702 rows, containing 32 columns (32 merged, 0 gathered) in 3.26997176 sec., 214.68075308393492 rows/sec., 19.45 KiB/sec. 2024.10.22 02:07:58.745673 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} MergedBlockOutputStream: filled checksums 202410_1_658_134 (state Temporary) 2024.10.22 02:07:59.086976 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_766_145/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:07:59.332878 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uaz/ndrrbelygdxgcuzkhxftfmpxgefda. 2024.10.22 02:07:59.342657 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_12_12_0 to 201403_12_12_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:59.362534 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mmo/vkxvzsmlryvinqeeorwqvyfftfgkw. 2024.10.22 02:07:59.433005 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:07:59.744613 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_766_145} MergeTask::MergeProjectionsStage: Merge sorted 1159055 rows, containing 4 columns (4 merged, 0 gathered) in 36.477669064 sec., 31774.371272639168 rows/sec., 476.52 KiB/sec. 2024.10.22 02:07:59.769633 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_161_161_0 to 202410_164_164_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:59.781604 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.23 GiB, peak 2.00 GiB, free memory in arenas 10.39 MiB, will set to 1.57 GiB (RSS), difference: 351.12 MiB 2024.10.22 02:07:59.848670 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_658_134 to 202410_1_658_134 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:07:59.853755 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_658_134} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_653_133, 202410_658_658_0] -> 202410_1_658_134 2024.10.22 02:07:59.879903 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:07:59.921226 [ 1159 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 7 parts from 202410_1_154_26 to 202410_164_164_0 2024.10.22 02:07:59.941230 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_766_145} MergedBlockOutputStream: filled checksums 202410_1_766_145 (state Temporary) 2024.10.22 02:07:59.949680 [ 1159 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 3.10 MiB using storage policy from min volume index 0 2024.10.22 02:07:59.954234 [ 1159 ] {} DiskLocal: Reserved 3.10 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:00.006510 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} MergeTask::PrepareStage: Merging 7 parts: from 202410_1_154_26 to 202410_164_164_0 into Compact with storage Full 2024.10.22 02:08:00.010798 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 5 old parts to remove. Parts: [202410_145_145_0, 202410_146_146_0, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0] 2024.10.22 02:08:00.015497 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from filesystem (serially): Parts: [202410_145_145_0, 202410_146_146_0, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0] 2024.10.22 02:08:00.040685 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from memory: Parts: [202410_145_145_0, 202410_146_146_0, 202410_147_147_0, 202410_148_148_0, 202410_149_149_0] 2024.10.22 02:08:00.310255 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 105 entries to flush up to offset 89188 2024.10.22 02:08:00.897737 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:00.904171 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:01.225530 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:08:01.232570 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:08:01.287681 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1352_1352_0 (state Temporary) 2024.10.22 02:08:01.362800 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1352_1352_0 to 202410_1365_1365_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:01.411487 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89188 2024.10.22 02:08:01.499297 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(59b26b63dfe87ba47c91a635469327bc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 59b26b63dfe87ba47c91a635469327bc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:08:01.503485 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 59b26b63dfe87ba47c91a635469327bc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:08:01.507736 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:08:01.621600 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.07 GiB, peak 2.00 GiB, free memory in arenas 2.71 MiB, will set to 1.08 GiB (RSS), difference: 7.32 MiB 2024.10.22 02:08:02.511534 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_766_145} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_766_145 to 202410_1_766_145 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:02.520886 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_766_145} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_760_144, 202410_761_766_1] -> 202410_1_766_145 2024.10.22 02:08:02.545546 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:08:02.638613 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6466 2024.10.22 02:08:02.981724 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 667 2024.10.22 02:08:03.336502 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:03.342926 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:03.452767 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 2.00 GiB, free memory in arenas 6.25 MiB, will set to 1.11 GiB (RSS), difference: -49.26 MiB 2024.10.22 02:08:03.525912 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_649_649_0 (state Temporary) 2024.10.22 02:08:03.604323 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_649_649_0 to 202410_659_659_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:03.670736 [ 1156 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:08:03.714382 [ 1156 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:08:03.731703 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_713_713_0, 202410_714_714_0, 202410_715_715_0, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0] 2024.10.22 02:08:03.735595 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_713_713_0, 202410_714_714_0, 202410_715_715_0, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0] 2024.10.22 02:08:03.762641 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_713_713_0, 202410_714_714_0, 202410_715_715_0, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0] 2024.10.22 02:08:03.864460 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 12216 2024.10.22 02:08:04.115506 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 667 2024.10.22 02:08:04.162130 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1156630 2024.10.22 02:08:04.263175 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:04.266857 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:04.310735 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_802_802_0 (state Temporary) 2024.10.22 02:08:04.469175 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_802_802_0 to 202410_815_815_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:04.521595 [ 1191 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_810_191 to 202410_815_815_0 2024.10.22 02:08:04.538303 [ 1191 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:04.544892 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:04.577660 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_810_191 to 202410_815_815_0 into Compact with storage Full 2024.10.22 02:08:04.618854 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:08:04.621013 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12216 2024.10.22 02:08:04.631648 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_810_191, total 12253 rows starting from the beginning of the part 2024.10.22 02:08:04.647772 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} MergeTreeSequentialSource: Reading 2 marks from part 202410_811_811_0, total 11 rows starting from the beginning of the part 2024.10.22 02:08:04.681243 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} MergeTreeSequentialSource: Reading 2 marks from part 202410_812_812_0, total 5 rows starting from the beginning of the part 2024.10.22 02:08:04.716698 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} MergeTreeSequentialSource: Reading 2 marks from part 202410_813_813_0, total 13 rows starting from the beginning of the part 2024.10.22 02:08:04.813832 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 80 entries to flush up to offset 88020 2024.10.22 02:08:04.856127 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} MergeTreeSequentialSource: Reading 2 marks from part 202410_814_814_0, total 19 rows starting from the beginning of the part 2024.10.22 02:08:04.907250 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} MergeTreeSequentialSource: Reading 2 marks from part 202410_815_815_0, total 21 rows starting from the beginning of the part 2024.10.22 02:08:04.978739 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:05.087143 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:05.165666 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:05.170058 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:05.213029 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_788_788_0 (state Temporary) 2024.10.22 02:08:05.266286 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_788_788_0 to 202410_794_794_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:05.399845 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_768_768_0 (state Temporary) 2024.10.22 02:08:05.455861 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 89242 2024.10.22 02:08:05.494541 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 88020 2024.10.22 02:08:05.512661 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 2.00 GiB, free memory in arenas 5.84 MiB, will set to 1.14 GiB (RSS), difference: -27.01 MiB 2024.10.22 02:08:05.531639 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_768_768_0 to 202410_775_775_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:05.850683 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1156630 2024.10.22 02:08:05.947483 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} MergeTask::MergeProjectionsStage: Merge sorted 12322 rows, containing 12 columns (12 merged, 0 gathered) in 1.37970087 sec., 8930.921381531056 rows/sec., 1.39 MiB/sec. 2024.10.22 02:08:06.081024 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:06.084854 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:06.124045 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} MergedBlockOutputStream: filled checksums 202410_1_815_192 (state Temporary) 2024.10.22 02:08:06.224407 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_815_192 to 202410_1_815_192 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:06.230596 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_815_192} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_810_191, 202410_815_815_0] -> 202410_1_815_192 2024.10.22 02:08:06.456090 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1353_1353_0 (state Temporary) 2024.10.22 02:08:06.470992 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.60 MiB. 2024.10.22 02:08:06.479246 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 45 2024.10.22 02:08:06.615467 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1353_1353_0 to 202410_1366_1366_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:06.687872 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89242 2024.10.22 02:08:07.161364 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:07.164910 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:07.213364 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_39_39_0 (state Temporary) 2024.10.22 02:08:07.474213 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_39_39_0 to 202410_39_39_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:07.559384 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 45 2024.10.22 02:08:07.700315 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 2.00 GiB, free memory in arenas 5.93 MiB, will set to 1.16 GiB (RSS), difference: -1.92 MiB 2024.10.22 02:08:09.240259 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_731_77, 202410_732_732_0] 2024.10.22 02:08:09.245108 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_731_77, 202410_732_732_0] 2024.10.22 02:08:09.256755 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_731_77, 202410_732_732_0] 2024.10.22 02:08:09.545399 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.22 GiB, peak 2.00 GiB, free memory in arenas 5.81 MiB, will set to 1.20 GiB (RSS), difference: -18.43 MiB 2024.10.22 02:08:10.266710 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 6509 2024.10.22 02:08:10.708370 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 27 entries to flush up to offset 89269 2024.10.22 02:08:10.969264 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:10.973681 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:11.231994 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:08:11.232476 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1354_1354_0 (state Temporary) 2024.10.22 02:08:11.325683 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:08:11.368364 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1354_1354_0 to 202410_1367_1367_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:11.418376 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1362_31 to 202410_1367_1367_0 2024.10.22 02:08:11.437049 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:11.499801 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.25 GiB, peak 2.00 GiB, free memory in arenas 6.05 MiB, will set to 1.23 GiB (RSS), difference: -25.46 MiB 2024.10.22 02:08:11.547029 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:11.577374 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1362_31 to 202410_1367_1367_0 into Compact with storage Full 2024.10.22 02:08:11.619526 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 668 2024.10.22 02:08:11.642634 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:08:11.650548 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1362_31, total 9808 rows starting from the beginning of the part 2024.10.22 02:08:11.643895 [ 1198 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 02:08:11.666532 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89269 2024.10.22 02:08:11.696762 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_1363_1363_0, total 86 rows starting from the beginning of the part 2024.10.22 02:08:11.976509 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_1364_1364_0, total 48 rows starting from the beginning of the part 2024.10.22 02:08:12.060020 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_1365_1365_0, total 105 rows starting from the beginning of the part 2024.10.22 02:08:12.126181 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 12229 2024.10.22 02:08:12.232477 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:08:12.289361 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:12.293824 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:12.352132 [ 1419 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:08:12.353212 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_1366_1366_0, total 54 rows starting from the beginning of the part 2024.10.22 02:08:12.412226 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_1367_1367_0, total 27 rows starting from the beginning of the part 2024.10.22 02:08:12.452266 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_650_650_0 (state Temporary) 2024.10.22 02:08:12.454507 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:12.461915 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:12.495581 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_803_803_0 (state Temporary) 2024.10.22 02:08:12.517713 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_650_650_0 to 202410_660_660_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:12.549486 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_803_803_0 to 202410_816_816_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:12.589346 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 668 2024.10.22 02:08:12.739504 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12229 2024.10.22 02:08:12.854958 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1158170 2024.10.22 02:08:13.032139 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 74 entries to flush up to offset 88094 2024.10.22 02:08:13.248230 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_154_26, total 6140 rows starting from the beginning of the part 2024.10.22 02:08:13.276793 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:13.282339 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:13.360228 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_789_789_0 (state Temporary) 2024.10.22 02:08:13.430997 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_789_789_0 to 202410_795_795_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:13.403546 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 2.00 GiB, free memory in arenas 5.70 MiB, will set to 1.26 GiB (RSS), difference: -40.11 MiB 2024.10.22 02:08:13.483278 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 88094 2024.10.22 02:08:13.849474 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:13.852355 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:14.381790 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 2.00 GiB, free memory in arenas 5.39 MiB, will set to 1.29 GiB (RSS), difference: -12.24 MiB 2024.10.22 02:08:14.411926 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_769_769_0 (state Temporary) 2024.10.22 02:08:14.535468 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1262_11, 202410_1263_1263_0, 202410_1264_1264_0, 202410_1265_1265_0, 202410_1266_1266_0, 202410_1267_1267_0] 2024.10.22 02:08:14.540523 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1262_11, 202410_1263_1263_0, 202410_1264_1264_0, 202410_1265_1265_0, 202410_1266_1266_0, 202410_1267_1267_0] 2024.10.22 02:08:14.564750 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1262_11, 202410_1263_1263_0, 202410_1264_1264_0, 202410_1265_1265_0, 202410_1266_1266_0, 202410_1267_1267_0] 2024.10.22 02:08:14.713033 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_769_769_0 to 202410_776_776_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:14.761722 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1158170 2024.10.22 02:08:14.753138 [ 1236 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_766_145 to 202410_767_772_1 2024.10.22 02:08:14.767523 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} MergeTask::MergeProjectionsStage: Merge sorted 10128 rows, containing 13 columns (13 merged, 0 gathered) in 3.197949952 sec., 3167.028925410775 rows/sec., 560.24 KiB/sec. 2024.10.22 02:08:14.768859 [ 1236 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:14.772670 [ 1236 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:14.799579 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_772_146} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_766_145 to 202410_767_772_1 into Wide with storage Full 2024.10.22 02:08:14.849399 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_772_146} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:08:14.854279 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_772_146} MergeTreeSequentialSource: Reading 143 marks from part 202410_1_766_145, total 1159055 rows starting from the beginning of the part 2024.10.22 02:08:14.886241 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_772_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_767_772_1, total 7623 rows starting from the beginning of the part 2024.10.22 02:08:15.169518 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} MergedBlockOutputStream: filled checksums 202410_1207_1367_32 (state Temporary) 2024.10.22 02:08:15.173148 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_155_159_1, total 191 rows starting from the beginning of the part 2024.10.22 02:08:15.251221 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_732_78, 202410_733_733_0] 2024.10.22 02:08:15.255447 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_732_78, 202410_733_733_0] 2024.10.22 02:08:15.266499 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_732_78, 202410_733_733_0] 2024.10.22 02:08:15.623968 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1367_32 to 202410_1207_1367_32 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:15.634065 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1367_32} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1362_31, 202410_1367_1367_0] -> 202410_1207_1367_32 2024.10.22 02:08:15.667050 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.72 MiB. 2024.10.22 02:08:15.671198 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 89338 2024.10.22 02:08:16.251399 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:16.254628 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:16.257649 [ 1246 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 217 ms. 2024.10.22 02:08:16.327907 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1355_1355_0 (state Temporary) 2024.10.22 02:08:16.484065 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1355_1355_0 to 202410_1368_1368_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:16.543613 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89338 2024.10.22 02:08:16.626857 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 8.04 MiB, will set to 1.33 GiB (RSS), difference: -44.64 MiB 2024.10.22 02:08:16.977618 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_160_160_0, total 41 rows starting from the beginning of the part 2024.10.22 02:08:17.031470 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(f3b35eb150c478bd1419f3d5c32c1918) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f3b35eb150c478bd1419f3d5c32c1918, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:08:17.037315 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1028, file segment info: File segment: [0, 33554431], key: f3b35eb150c478bd1419f3d5c32c1918, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:08:17.047052 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:08:17.333120 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 02:08:17.355843 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 02:08:17.373462 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 02:08:17.515464 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 02:08:17.523646 [ 1712 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fxu/cspaoypeyqlnxrrguhxiejjhqvmws, size 10745530 2024.10.22 02:08:17.551354 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 02:08:17.561974 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 02:08:17.576803 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SilverlightVersion1 2024.10.22 02:08:17.993116 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/txu/xrfmlizhlsoztfiedgxpiwumvidtb (LogSeriesLimiter: on interval from 2024-10-22 02:07:57 to 2024-10-22 02:08:17 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:08:18.252995 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 5 old parts to remove. Parts: [202410_1_752_157, 202410_753_753_0, 202410_754_754_0, 202410_755_755_0, 202410_756_756_0] 2024.10.22 02:08:18.255435 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 5 parts from filesystem (serially): Parts: [202410_1_752_157, 202410_753_753_0, 202410_754_754_0, 202410_755_755_0, 202410_756_756_0] 2024.10.22 02:08:18.276931 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 5 parts from memory: Parts: [202410_1_752_157, 202410_753_753_0, 202410_754_754_0, 202410_755_755_0, 202410_756_756_0] 2024.10.22 02:08:18.450670 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 2.00 GiB, free memory in arenas 9.25 MiB, will set to 1.35 GiB (RSS), difference: -55.73 MiB 2024.10.22 02:08:18.723936 [ 1188 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:08:18.778994 [ 1188 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:08:18.808897 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_161_161_0, total 40 rows starting from the beginning of the part 2024.10.22 02:08:20.242826 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 12241 2024.10.22 02:08:20.309737 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 2.00 GiB, free memory in arenas 9.71 MiB, will set to 1.39 GiB (RSS), difference: -74.09 MiB 2024.10.22 02:08:20.509123 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:20.512895 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:20.551769 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 89370 2024.10.22 02:08:20.558292 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_162_162_0, total 37 rows starting from the beginning of the part 2024.10.22 02:08:20.562155 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_804_804_0 (state Temporary) 2024.10.22 02:08:20.763874 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:20.776656 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:20.816133 [ 1173 ] {} DatabaseCatalog: Execution took 446 ms. 2024.10.22 02:08:20.856065 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_804_804_0 to 202410_817_817_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:20.914786 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12241 2024.10.22 02:08:21.009505 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 98 entries to flush up to offset 88192 2024.10.22 02:08:21.151348 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1356_1356_0 (state Temporary) 2024.10.22 02:08:21.184285 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1356_1356_0 to 202410_1369_1369_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:21.224327 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:08:21.229364 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:08:21.268781 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89370 2024.10.22 02:08:21.326859 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:21.334982 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:21.394603 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_790_790_0 (state Temporary) 2024.10.22 02:08:21.548984 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_790_790_0 to 202410_796_796_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:21.617152 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 88192 2024.10.22 02:08:21.764800 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1160095 2024.10.22 02:08:21.854580 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_163_163_0, total 39 rows starting from the beginning of the part 2024.10.22 02:08:22.062176 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_772_146/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:08:22.121063 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_13_13_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:08:22.424042 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.41 GiB, peak 2.00 GiB, free memory in arenas 10.39 MiB, will set to 1.41 GiB (RSS), difference: 2.72 MiB 2024.10.22 02:08:23.026709 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:23.041032 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:23.206727 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_770_770_0 (state Temporary) 2024.10.22 02:08:23.343204 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_770_770_0 to 202410_777_777_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:23.400826 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1160095 2024.10.22 02:08:23.649639 [ 1388 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_164_164_0, total 40 rows starting from the beginning of the part 2024.10.22 02:08:24.526634 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_706_135, 202410_707_712_1] 2024.10.22 02:08:24.532031 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_706_135, 202410_707_712_1] 2024.10.22 02:08:24.553379 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_706_135, 202410_707_712_1] 2024.10.22 02:08:24.592749 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_733_79, 202410_734_734_0] 2024.10.22 02:08:24.600382 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 11.63 MiB, will set to 1.46 GiB (RSS), difference: -38.62 MiB 2024.10.22 02:08:24.656924 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_733_79, 202410_734_734_0] 2024.10.22 02:08:24.721558 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_733_79, 202410_734_734_0] 2024.10.22 02:08:25.279003 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 37 entries to flush up to offset 89407 2024.10.22 02:08:25.447931 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:25.450900 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:25.536618 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1357_1357_0 (state Temporary) 2024.10.22 02:08:25.696663 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1357_1357_0 to 202410_1370_1370_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:25.763296 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89407 2024.10.22 02:08:26.427565 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 10.61 MiB, will set to 1.52 GiB (RSS), difference: -28.16 MiB 2024.10.22 02:08:26.903451 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(43f2e27ee77abb066acc73ce74177e40) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 43f2e27ee77abb066acc73ce74177e40, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:26.914654 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4469, file segment info: File segment: [0, 33554431], key: 43f2e27ee77abb066acc73ce74177e40, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:26.923987 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:08:27.593352 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 669 2024.10.22 02:08:28.060077 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:28.067242 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:28.055190 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(c10982cf8dde3622665bf46facf3bcf0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c10982cf8dde3622665bf46facf3bcf0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:28.090788 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 433, file segment info: File segment: [0, 33554431], key: c10982cf8dde3622665bf46facf3bcf0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:28.095899 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:08:28.144328 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_651_651_0 (state Temporary) 2024.10.22 02:08:28.427603 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 5 entries to flush up to offset 12246 2024.10.22 02:08:28.434363 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_651_651_0 to 202410_661_661_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:28.540393 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 669 2024.10.22 02:08:28.647142 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 10.46 MiB, will set to 1.56 GiB (RSS), difference: -54.95 MiB 2024.10.22 02:08:28.676025 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:28.679673 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:28.723854 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_805_805_0 (state Temporary) 2024.10.22 02:08:28.966569 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_805_805_0 to 202410_818_818_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:29.041799 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12246 2024.10.22 02:08:29.117408 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(a6fb733ea04a128cf230a5dce80a8785) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a6fb733ea04a128cf230a5dce80a8785, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:29.224172 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 268037, file segment info: File segment: [0, 33554431], key: a6fb733ea04a128cf230a5dce80a8785, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:29.228993 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:08:29.241139 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 66 entries to flush up to offset 88258 2024.10.22 02:08:29.309569 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:29.316510 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:29.469353 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:29.472187 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:29.524519 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_791_791_0 (state Temporary) 2024.10.22 02:08:29.589717 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_791_791_0 to 202410_797_797_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:29.640602 [ 1213 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_792_9 to 202410_797_797_0 2024.10.22 02:08:29.655657 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 88258 2024.10.22 02:08:29.674652 [ 1213 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:29.683347 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:29.732872 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_792_9 to 202410_797_797_0 into Compact with storage Full 2024.10.22 02:08:29.773299 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 89448 2024.10.22 02:08:29.959333 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:29.962570 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:30.037737 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_164_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:08:30.140791 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1358_1358_0 (state Temporary) 2024.10.22 02:08:30.156548 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:08:30.169207 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_792_9, total 4511 rows starting from the beginning of the part 2024.10.22 02:08:30.197560 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_793_793_0, total 74 rows starting from the beginning of the part 2024.10.22 02:08:30.251145 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_794_794_0, total 80 rows starting from the beginning of the part 2024.10.22 02:08:30.338067 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1358_1358_0 to 202410_1371_1371_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:30.387915 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_795_795_0, total 74 rows starting from the beginning of the part 2024.10.22 02:08:30.397812 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 10.53 MiB, will set to 1.59 GiB (RSS), difference: -38.88 MiB 2024.10.22 02:08:30.408785 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1161635 2024.10.22 02:08:30.435631 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_796_796_0, total 98 rows starting from the beginning of the part 2024.10.22 02:08:30.405379 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(eeea61b3ab48ccd8c4319dd3ba8607e8) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: eeea61b3ab48ccd8c4319dd3ba8607e8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:30.451253 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 750, file segment info: File segment: [0, 33554431], key: eeea61b3ab48ccd8c4319dd3ba8607e8, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:30.457735 [ 1395 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:08:30.576738 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_797_797_0, total 66 rows starting from the beginning of the part 2024.10.22 02:08:30.674334 [ 1233 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 293 ms. 2024.10.22 02:08:30.809144 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89448 2024.10.22 02:08:30.889097 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 02:08:30.959508 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 02:08:31.034400 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 02:08:31.119399 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 02:08:31.168468 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 02:08:31.229746 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:08:31.235640 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:08:31.311024 [ 1254 ] {} DatabaseCatalog: Execution took 484 ms. 2024.10.22 02:08:31.372620 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 02:08:31.427451 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_734_80, 202410_735_735_0] 2024.10.22 02:08:31.431912 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_734_80, 202410_735_735_0] 2024.10.22 02:08:31.442250 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_608_124, 202410_609_609_0, 202410_610_610_0, 202410_611_611_0, 202410_612_612_0, 202410_613_613_0] 2024.10.22 02:08:31.446791 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_608_124, 202410_609_609_0, 202410_610_610_0, 202410_611_611_0, 202410_612_612_0, 202410_613_613_0] 2024.10.22 02:08:31.447143 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_734_80, 202410_735_735_0] 2024.10.22 02:08:31.474775 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_608_124, 202410_609_609_0, 202410_610_610_0, 202410_611_611_0, 202410_612_612_0, 202410_613_613_0] 2024.10.22 02:08:31.549025 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 02:08:31.632325 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ParsedParams.Key5 2024.10.22 02:08:31.659910 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 254.08 MiB using storage policy from min volume index 0 2024.10.22 02:08:31.663845 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:31.681709 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:31.896338 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/phm/vfykellvelazdywlknhkpefcfuybn (LogSeriesLimiter: on interval from 2024-10-22 02:08:17 to 2024-10-22 02:08:31 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:08:32.009702 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_771_771_0 (state Temporary) 2024.10.22 02:08:32.311281 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_771_771_0 to 202410_778_778_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:32.381426 [ 1255 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_773_773_0 to 202410_778_778_0 2024.10.22 02:08:32.453293 [ 1255 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:32.456340 [ 1255 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:32.536185 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1161635 2024.10.22 02:08:32.551395 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 10.79 MiB, will set to 1.59 GiB (RSS), difference: -9.56 MiB 2024.10.22 02:08:32.639728 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_773_773_0 to 202410_778_778_0 into Compact with storage Full 2024.10.22 02:08:32.672098 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:08:32.679046 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_773_773_0, total 1232 rows starting from the beginning of the part 2024.10.22 02:08:32.711313 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_774_774_0, total 770 rows starting from the beginning of the part 2024.10.22 02:08:32.809394 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_162_162_0 (state Temporary) 2024.10.22 02:08:32.947951 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_775_775_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:08:32.989026 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_776_776_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:08:33.020193 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_777_777_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:08:33.058554 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_778_778_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:08:33.116836 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} MergeTask::MergeProjectionsStage: Merge sorted 4903 rows, containing 13 columns (13 merged, 0 gathered) in 3.395982523 sec., 1443.764791718865 rows/sec., 639.87 KiB/sec. 2024.10.22 02:08:33.167525 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_772_146/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:08:33.187766 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} MergedBlockOutputStream: filled checksums 202410_747_797_10 (state Temporary) 2024.10.22 02:08:33.776889 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_797_10 to 202410_747_797_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:33.785579 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_797_10} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_792_9, 202410_797_797_0] -> 202410_747_797_10 2024.10.22 02:08:33.796209 [ 1265 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:08:33.949157 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(6c06b71366caa2f9039e83a998f3ba44) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6c06b71366caa2f9039e83a998f3ba44, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:08:33.949254 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.34 MiB. 2024.10.22 02:08:33.952429 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 540553, file segment info: File segment: [0, 33554431], key: 6c06b71366caa2f9039e83a998f3ba44, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:08:33.954889 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:08:34.059190 [ 1265 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:08:34.144314 [ 1265 ] {} DNSCacheUpdater: Execution took 273 ms. 2024.10.22 02:08:34.507023 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.70 GiB, peak 2.00 GiB, free memory in arenas 12.14 MiB, will set to 1.60 GiB (RSS), difference: -102.17 MiB 2024.10.22 02:08:34.540406 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} MergeTask::MergeProjectionsStage: Merge sorted 8547 rows, containing 4 columns (4 merged, 0 gathered) in 2.053212061 sec., 4162.745856771002 rows/sec., 65.04 KiB/sec. 2024.10.22 02:08:34.571534 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} MergedBlockOutputStream: filled checksums 202410_773_778_1 (state Temporary) 2024.10.22 02:08:34.629633 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_773_778_1 to 202410_773_778_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:34.637585 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_773_778_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_773_773_0, 202410_778_778_0] -> 202410_773_778_1 2024.10.22 02:08:34.689400 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:08:34.820983 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 89523 2024.10.22 02:08:35.045411 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(faa2c9419f88c50e5e2974237f3e710a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: faa2c9419f88c50e5e2974237f3e710a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:08:35.049012 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 522, file segment info: File segment: [0, 33554431], key: faa2c9419f88c50e5e2974237f3e710a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:08:35.051915 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:08:35.061041 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:35.073689 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:35.114211 [ 1709 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/txu/xrfmlizhlsoztfiedgxpiwumvidtb, size 540553 2024.10.22 02:08:35.195570 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1359_1359_0 (state Temporary) 2024.10.22 02:08:35.216947 [ 1263 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 332 ms. 2024.10.22 02:08:35.592166 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1359_1359_0 to 202410_1372_1372_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:35.634857 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1367_32 to 202410_1372_1372_0 2024.10.22 02:08:35.653438 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:35.660405 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89523 2024.10.22 02:08:35.770153 [ 1262 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:35.928710 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1367_32 to 202410_1372_1372_0 into Compact with storage Full 2024.10.22 02:08:35.931631 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 02:08:35.944141 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 02:08:35.955396 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 02:08:35.967944 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 02:08:35.998822 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 02:08:36.009275 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fya/cczmzaqgfnidjzjfpqrpvncgnfpor (LogSeriesLimiter: on interval from 2024-10-22 02:08:31 to 2024-10-22 02:08:36 accepted series 1 / 127 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:08:36.043525 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 670 2024.10.22 02:08:36.155981 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:08:36.171629 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1367_32, total 10128 rows starting from the beginning of the part 2024.10.22 02:08:36.215456 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_1368_1368_0, total 69 rows starting from the beginning of the part 2024.10.22 02:08:36.269508 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_1369_1369_0, total 32 rows starting from the beginning of the part 2024.10.22 02:08:36.348776 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 02:08:36.364699 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SilverlightVersion2 2024.10.22 02:08:36.458116 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:36.462401 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:36.518610 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_652_652_0 (state Temporary) 2024.10.22 02:08:36.547101 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 12266 2024.10.22 02:08:36.555016 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_1370_1370_0, total 37 rows starting from the beginning of the part 2024.10.22 02:08:36.601223 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_1371_1371_0, total 41 rows starting from the beginning of the part 2024.10.22 02:08:36.679116 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_1372_1372_0, total 75 rows starting from the beginning of the part 2024.10.22 02:08:36.693478 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_652_652_0 to 202410_662_662_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:36.720054 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:36.725166 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:36.768815 [ 1223 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 213 ms. 2024.10.22 02:08:36.772000 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_806_806_0 (state Temporary) 2024.10.22 02:08:36.783090 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:08:36.816439 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 670 2024.10.22 02:08:36.860467 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_806_806_0 to 202410_819_819_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:36.923944 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 2.00 GiB, free memory in arenas 10.07 MiB, will set to 1.59 GiB (RSS), difference: -157.65 MiB 2024.10.22 02:08:37.120285 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12266 2024.10.22 02:08:37.164916 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 117 entries to flush up to offset 88375 2024.10.22 02:08:37.523515 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:37.537429 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:37.690312 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_792_792_0 (state Temporary) 2024.10.22 02:08:37.752681 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_792_792_0 to 202410_798_798_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:38.040148 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 88375 2024.10.22 02:08:38.308126 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} MergeTask::MergeProjectionsStage: Merge sorted 10382 rows, containing 13 columns (13 merged, 0 gathered) in 2.497595916 sec., 4156.797315967425 rows/sec., 734.67 KiB/sec. 2024.10.22 02:08:38.475490 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} MergedBlockOutputStream: filled checksums 202410_1207_1372_33 (state Temporary) 2024.10.22 02:08:38.514800 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 2.00 GiB, free memory in arenas 9.58 MiB, will set to 1.59 GiB (RSS), difference: -160.31 MiB 2024.10.22 02:08:38.660291 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1372_33 to 202410_1207_1372_33 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:38.664166 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1372_33} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1367_32, 202410_1372_1372_0] -> 202410_1207_1372_33 2024.10.22 02:08:38.712368 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.70 MiB. 2024.10.22 02:08:39.531120 [ 1161 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 02:08:39.545280 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1759 entries to flush up to offset 1163394 2024.10.22 02:08:39.764977 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 89582 2024.10.22 02:08:40.201131 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:40.208726 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:40.174191 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1267_12, 202410_1268_1268_0, 202410_1269_1269_0, 202410_1270_1270_0, 202410_1271_1271_0, 202410_1272_1272_0] 2024.10.22 02:08:40.229298 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1267_12, 202410_1268_1268_0, 202410_1269_1269_0, 202410_1270_1270_0, 202410_1271_1271_0, 202410_1272_1272_0] 2024.10.22 02:08:40.267857 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1267_12, 202410_1268_1268_0, 202410_1269_1269_0, 202410_1270_1270_0, 202410_1271_1271_0, 202410_1272_1272_0] 2024.10.22 02:08:40.292460 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:40.367566 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_772_772_0 (state Temporary) 2024.10.22 02:08:40.397196 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:40.461471 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1360_1360_0 (state Temporary) 2024.10.22 02:08:40.462061 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_772_772_0 to 202410_779_779_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:40.563234 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1163394 2024.10.22 02:08:40.607105 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1360_1360_0 to 202410_1373_1373_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:40.666823 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89582 2024.10.22 02:08:40.741941 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 2.00 GiB, free memory in arenas 9.04 MiB, will set to 1.60 GiB (RSS), difference: -129.84 MiB 2024.10.22 02:08:41.231173 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:08:41.234025 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:08:41.672987 [ 1173 ] {} DatabaseCatalog: Execution took 329 ms. 2024.10.22 02:08:41.800407 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_735_81, 202410_736_736_0] 2024.10.22 02:08:41.802898 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_735_81, 202410_736_736_0] 2024.10.22 02:08:41.824303 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_735_81, 202410_736_736_0] 2024.10.22 02:08:42.408663 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_162_162_0 to 202410_165_165_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:42.497504 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_164_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:08:42.550707 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 10.68 MiB, will set to 1.61 GiB (RSS), difference: -1.18 MiB 2024.10.22 02:08:44.198277 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_772_146/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:08:44.319282 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 671 2024.10.22 02:08:44.474528 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 8.62 MiB, will set to 1.62 GiB (RSS), difference: 4.37 MiB 2024.10.22 02:08:44.623126 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 12279 2024.10.22 02:08:44.691121 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 28 entries to flush up to offset 89610 2024.10.22 02:08:44.696547 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:44.702466 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:44.707523 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.096667 sec. 2024.10.22 02:08:44.850549 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:44.853973 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:44.881395 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_653_653_0 (state Temporary) 2024.10.22 02:08:44.917733 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1361_1361_0 (state Temporary) 2024.10.22 02:08:44.967222 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:44.977331 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_653_653_0 to 202410_663_663_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:44.978978 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:45.039533 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_807_807_0 (state Temporary) 2024.10.22 02:08:45.043284 [ 1163 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_658_134 to 202410_663_663_0 2024.10.22 02:08:45.060818 [ 1163 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:45.063395 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1361_1361_0 to 202410_1374_1374_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:45.065119 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:45.103506 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_807_807_0 to 202410_820_820_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:45.174861 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_658_134 to 202410_663_663_0 into Compact with storage Full 2024.10.22 02:08:45.193811 [ 1231 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_815_192 to 202410_816_816_0 2024.10.22 02:08:45.204847 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12279 2024.10.22 02:08:45.216769 [ 1231 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:45.218942 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 671 2024.10.22 02:08:45.224438 [ 1231 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:45.264686 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_816_193} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_815_192 to 202410_816_816_0 into Compact with storage Full 2024.10.22 02:08:45.268045 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89610 2024.10.22 02:08:45.289525 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6509 2024.10.22 02:08:45.373415 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_816_193} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:08:45.424857 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_816_193} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_815_192, total 12322 rows starting from the beginning of the part 2024.10.22 02:08:45.568075 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_816_193} MergeTreeSequentialSource: Reading 2 marks from part 202410_816_816_0, total 13 rows starting from the beginning of the part 2024.10.22 02:08:45.670488 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 85 entries to flush up to offset 88460 2024.10.22 02:08:45.696185 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:08:45.839286 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_658_134, total 702 rows starting from the beginning of the part 2024.10.22 02:08:45.970116 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_659_659_0, total 1 rows starting from the beginning of the part 2024.10.22 02:08:46.065564 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:46.076304 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:46.112476 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_660_660_0, total 1 rows starting from the beginning of the part 2024.10.22 02:08:46.252487 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_661_661_0, total 1 rows starting from the beginning of the part 2024.10.22 02:08:46.267731 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_793_793_0 (state Temporary) 2024.10.22 02:08:46.345348 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_793_793_0 to 202410_799_799_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:46.392358 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_662_662_0, total 1 rows starting from the beginning of the part 2024.10.22 02:08:46.486048 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 88460 2024.10.22 02:08:46.527416 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 10.52 MiB, will set to 1.63 GiB (RSS), difference: -10.48 MiB 2024.10.22 02:08:46.657692 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_663_663_0, total 1 rows starting from the beginning of the part 2024.10.22 02:08:46.970423 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_816_193} MergeTask::MergeProjectionsStage: Merge sorted 12335 rows, containing 12 columns (12 merged, 0 gathered) in 1.719585768 sec., 7173.2391774482285 rows/sec., 1.12 MiB/sec. 2024.10.22 02:08:47.171097 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_816_193} MergedBlockOutputStream: filled checksums 202410_1_816_193 (state Temporary) 2024.10.22 02:08:47.569412 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1706 entries to flush up to offset 1165100 2024.10.22 02:08:48.241737 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_816_193} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_816_193 to 202410_1_816_193 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:48.249465 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_816_193} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_815_192, 202410_816_816_0] -> 202410_1_816_193 2024.10.22 02:08:48.277586 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:08:48.466722 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_663_135/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:08:48.511314 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 12.10 MiB, will set to 1.64 GiB (RSS), difference: 8.80 MiB 2024.10.22 02:08:48.660758 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:48.664635 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:48.814321 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_773_773_0 (state Temporary) 2024.10.22 02:08:48.920410 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_773_773_0 to 202410_780_780_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:48.975671 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1165100 2024.10.22 02:08:49.062776 [ 1269 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:08:49.133408 [ 1269 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:08:49.272641 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 89668 2024.10.22 02:08:49.318351 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 3 old parts to remove. Parts: [202410_1_756_158, 202410_757_757_0, 202410_758_758_0] 2024.10.22 02:08:49.328484 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 3 parts from filesystem (serially): Parts: [202410_1_756_158, 202410_757_757_0, 202410_758_758_0] 2024.10.22 02:08:49.351616 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 3 parts from memory: Parts: [202410_1_756_158, 202410_757_757_0, 202410_758_758_0] 2024.10.22 02:08:49.460134 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:49.462876 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:49.376355 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} MergeTask::MergeProjectionsStage: Merge sorted 707 rows, containing 32 columns (32 merged, 0 gathered) in 4.22380803 sec., 167.3845011369989 rows/sec., 15.16 KiB/sec. 2024.10.22 02:08:49.529352 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_736_82, 202410_737_737_0] 2024.10.22 02:08:49.540957 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_736_82, 202410_737_737_0] 2024.10.22 02:08:49.558115 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_736_82, 202410_737_737_0] 2024.10.22 02:08:49.563032 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} MergedBlockOutputStream: filled checksums 202410_1_663_135 (state Temporary) 2024.10.22 02:08:49.595029 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1362_1362_0 (state Temporary) 2024.10.22 02:08:49.657650 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1362_1362_0 to 202410_1375_1375_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:49.715104 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89668 2024.10.22 02:08:50.655730 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 13.25 MiB, will set to 1.64 GiB (RSS), difference: 4.41 MiB 2024.10.22 02:08:50.666698 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_663_135 to 202410_1_663_135 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:50.670395 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_663_135} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_658_134, 202410_663_663_0] -> 202410_1_663_135 2024.10.22 02:08:50.952831 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:08:51.233371 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:08:51.240919 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:08:51.703533 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_772_146} MergeTask::MergeProjectionsStage: Merge sorted 1166678 rows, containing 4 columns (4 merged, 0 gathered) in 36.918270076 sec., 31601.643240549332 rows/sec., 474.06 KiB/sec. 2024.10.22 02:08:51.904061 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_772_146} MergedBlockOutputStream: filled checksums 202410_1_772_146 (state Temporary) 2024.10.22 02:08:52.154817 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(1b76a3c4b0a5018f47081259b8c77247) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1b76a3c4b0a5018f47081259b8c77247, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:52.184221 [ 1250 ] {} DatabaseCatalog: Execution took 508 ms. 2024.10.22 02:08:52.279316 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 463523, file segment info: File segment: [0, 33554431], key: 1b76a3c4b0a5018f47081259b8c77247, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:52.287040 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:08:52.565607 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 14.07 MiB, will set to 1.65 GiB (RSS), difference: 16.65 MiB 2024.10.22 02:08:52.709146 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 12295 2024.10.22 02:08:52.724831 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 672 2024.10.22 02:08:52.870472 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 6551 2024.10.22 02:08:52.940748 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:52.953220 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:52.974050 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_808_808_0 (state Temporary) 2024.10.22 02:08:53.003899 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:53.006724 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:53.041314 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_808_808_0 to 202410_821_821_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:53.074704 [ 1276 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_816_193 to 202410_817_817_0 2024.10.22 02:08:53.081243 [ 1276 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:53.088887 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:53.107183 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_817_194} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_816_193 to 202410_817_817_0 into Compact with storage Full 2024.10.22 02:08:53.117204 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12295 2024.10.22 02:08:53.177004 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_817_194} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:08:53.201590 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_817_194} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_816_193, total 12335 rows starting from the beginning of the part 2024.10.22 02:08:53.208656 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_654_654_0 (state Temporary) 2024.10.22 02:08:53.372383 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_817_194} MergeTreeSequentialSource: Reading 2 marks from part 202410_817_817_0, total 12 rows starting from the beginning of the part 2024.10.22 02:08:53.515464 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_654_654_0 to 202410_664_664_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:53.565111 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(88fa6329319372d42a61c2192a30f5b1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 88fa6329319372d42a61c2192a30f5b1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:53.591354 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 521, file segment info: File segment: [0, 33554431], key: 88fa6329319372d42a61c2192a30f5b1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:08:53.595140 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:08:53.730302 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 89717 2024.10.22 02:08:53.758133 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 672 2024.10.22 02:08:53.948750 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:53.951493 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:54.005637 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 02:08:54.015266 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 122 entries to flush up to offset 88582 2024.10.22 02:08:54.016961 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1363_1363_0 (state Temporary) 2024.10.22 02:08:54.051382 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 02:08:54.067196 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 02:08:54.106862 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 02:08:54.145537 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 02:08:54.177172 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 02:08:54.209069 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SilverlightVersion3 2024.10.22 02:08:54.210362 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:54.217395 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:54.249256 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1363_1363_0 to 202410_1376_1376_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:54.359987 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89717 2024.10.22 02:08:54.372451 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_794_794_0 (state Temporary) 2024.10.22 02:08:54.395042 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pqv/zveunaqtophfdfzdokwkvuzjxefxv (LogSeriesLimiter: on interval from 2024-10-22 02:08:36 to 2024-10-22 02:08:54 accepted series 1 / 232 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:08:54.437416 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_794_794_0 to 202410_800_800_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:54.477804 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 88582 2024.10.22 02:08:54.711388 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_817_194} MergeTask::MergeProjectionsStage: Merge sorted 12347 rows, containing 12 columns (12 merged, 0 gathered) in 1.61052018 sec., 7666.467116233216 rows/sec., 1.19 MiB/sec. 2024.10.22 02:08:54.773378 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_817_194} MergedBlockOutputStream: filled checksums 202410_1_817_194 (state Temporary) 2024.10.22 02:08:54.803076 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_719_719_0, 202410_720_720_0, 202410_721_721_0, 202410_722_722_0, 202410_723_723_0, 202410_724_724_0] 2024.10.22 02:08:54.810895 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_719_719_0, 202410_720_720_0, 202410_721_721_0, 202410_722_722_0, 202410_723_723_0, 202410_724_724_0] 2024.10.22 02:08:54.944929 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_719_719_0, 202410_720_720_0, 202410_721_721_0, 202410_722_722_0, 202410_723_723_0, 202410_724_724_0] 2024.10.22 02:08:55.108486 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_772_146/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:08:55.457027 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_817_194} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_817_194 to 202410_1_817_194 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:55.460809 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_817_194} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_816_193, 202410_817_817_0] -> 202410_1_817_194 2024.10.22 02:08:55.484358 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:08:55.671521 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_772_146} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_772_146 to 202410_1_772_146 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:55.675384 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_772_146} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_766_145, 202410_767_772_1] -> 202410_1_772_146 2024.10.22 02:08:55.696676 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:08:55.739374 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 11.64 MiB, will set to 1.65 GiB (RSS), difference: 3.63 MiB 2024.10.22 02:08:55.999442 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1169 entries to flush up to offset 1166269 2024.10.22 02:08:56.715184 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:56.727495 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.24 GiB. 2024.10.22 02:08:56.830229 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_164_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:08:57.003983 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_774_774_0 (state Temporary) 2024.10.22 02:08:57.199524 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_774_774_0 to 202410_781_781_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:57.364097 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1166269 2024.10.22 02:08:57.948790 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 11.07 MiB, will set to 1.66 GiB (RSS), difference: 31.88 MiB 2024.10.22 02:08:58.335815 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:08:58.383493 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 89760 2024.10.22 02:08:58.675123 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_737_83, 202410_738_738_0] 2024.10.22 02:08:58.688966 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_737_83, 202410_738_738_0] 2024.10.22 02:08:58.761277 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:58.801737 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_737_83, 202410_738_738_0] 2024.10.22 02:08:58.950444 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:59.008780 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1364_1364_0 (state Temporary) 2024.10.22 02:08:59.073122 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1364_1364_0 to 202410_1377_1377_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:08:59.157063 [ 1203 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1372_33 to 202410_1377_1377_0 2024.10.22 02:08:59.185180 [ 1219 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 307 ms. 2024.10.22 02:08:59.189661 [ 1203 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:08:59.196538 [ 1203 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.23 GiB. 2024.10.22 02:08:59.294967 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1372_33 to 202410_1377_1377_0 into Compact with storage Full 2024.10.22 02:08:59.318821 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89760 2024.10.22 02:08:59.370591 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:08:59.407316 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1372_33, total 10382 rows starting from the beginning of the part 2024.10.22 02:08:59.460864 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_1373_1373_0, total 59 rows starting from the beginning of the part 2024.10.22 02:08:59.746022 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_1374_1374_0, total 28 rows starting from the beginning of the part 2024.10.22 02:08:59.793717 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_1375_1375_0, total 58 rows starting from the beginning of the part 2024.10.22 02:08:59.858097 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_1376_1376_0, total 49 rows starting from the beginning of the part 2024.10.22 02:08:59.930282 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_1377_1377_0, total 43 rows starting from the beginning of the part 2024.10.22 02:09:00.462797 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(ec85081fc1371c498e1d5a8e3777c639) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ec85081fc1371c498e1d5a8e3777c639, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:09:00.477494 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1884, file segment info: File segment: [0, 33554431], key: ec85081fc1371c498e1d5a8e3777c639, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:09:00.480491 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:00.620269 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 12307 2024.10.22 02:09:00.667964 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 2.00 GiB, free memory in arenas 9.09 MiB, will set to 1.65 GiB (RSS), difference: 205.59 MiB 2024.10.22 02:09:01.043466 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:01.054403 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.21 GiB. 2024.10.22 02:09:01.203670 [ 1682 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fya/cczmzaqgfnidjzjfpqrpvncgnfpor, size 53888 2024.10.22 02:09:01.237639 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:09:01.242671 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:09:01.322804 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_809_809_0 (state Temporary) 2024.10.22 02:09:01.499356 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_809_809_0 to 202410_822_822_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:01.565603 [ 1166 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_817_194 to 202410_818_818_0 2024.10.22 02:09:01.635990 [ 1166 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:01.644449 [ 1166 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.21 GiB. 2024.10.22 02:09:01.674654 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_818_195} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_817_194 to 202410_818_818_0 into Compact with storage Full 2024.10.22 02:09:01.698412 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12307 2024.10.22 02:09:02.017341 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_818_195} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:02.111892 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 124 entries to flush up to offset 88706 2024.10.22 02:09:02.152750 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_818_195} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_817_194, total 12347 rows starting from the beginning of the part 2024.10.22 02:09:02.268048 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(b07fe82cc34ac381db91cd5ad4b46127) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b07fe82cc34ac381db91cd5ad4b46127, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:09:02.269787 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 428, file segment info: File segment: [0, 33554431], key: b07fe82cc34ac381db91cd5ad4b46127, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:09:02.276369 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:02.464048 [ 1173 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 252 ms. 2024.10.22 02:09:02.487455 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_14_14_0 (state Temporary) 2024.10.22 02:09:02.569562 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1377_34/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:09:02.597626 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:02.602105 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1272_13, 202410_1273_1273_0, 202410_1274_1274_0, 202410_1275_1275_0, 202410_1276_1276_0, 202410_1277_1277_0] 2024.10.22 02:09:02.606911 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:02.607419 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xka/bnqyqnnzmurgnbpvgebwtjklcgipl (LogSeriesLimiter: on interval from 2024-10-22 02:08:54 to 2024-10-22 02:09:02 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:09:02.607979 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1272_13, 202410_1273_1273_0, 202410_1274_1274_0, 202410_1275_1275_0, 202410_1276_1276_0, 202410_1277_1277_0] 2024.10.22 02:09:02.611486 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_818_195} MergeTreeSequentialSource: Reading 2 marks from part 202410_818_818_0, total 5 rows starting from the beginning of the part 2024.10.22 02:09:02.709460 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_795_795_0 (state Temporary) 2024.10.22 02:09:02.710533 [ 1206 ] {} DatabaseCatalog: Execution took 515 ms. 2024.10.22 02:09:02.739318 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1272_13, 202410_1273_1273_0, 202410_1274_1274_0, 202410_1275_1275_0, 202410_1276_1276_0, 202410_1277_1277_0] 2024.10.22 02:09:02.760119 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} MergeTask::MergeProjectionsStage: Merge sorted 10619 rows, containing 13 columns (13 merged, 0 gathered) in 3.479582131 sec., 3051.803233898145 rows/sec., 539.00 KiB/sec. 2024.10.22 02:09:02.876028 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_795_795_0 to 202410_801_801_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:03.033396 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 88706 2024.10.22 02:09:03.109593 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(1cef721a4bb0af14ab229b485e0758c3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1cef721a4bb0af14ab229b485e0758c3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:09:03.145216 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} MergedBlockOutputStream: filled checksums 202410_1207_1377_34 (state Temporary) 2024.10.22 02:09:03.152895 [ 1676 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xka/bnqyqnnzmurgnbpvgebwtjklcgipl, size 4 2024.10.22 02:09:03.222003 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 268037, file segment info: File segment: [0, 33554431], key: 1cef721a4bb0af14ab229b485e0758c3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:09:03.226938 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:03.257551 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1377_34 to 202410_1207_1377_34 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:03.260773 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1377_34} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1372_33, 202410_1377_1377_0] -> 202410_1207_1377_34 2024.10.22 02:09:03.391563 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.70 MiB. 2024.10.22 02:09:03.439216 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 89826 2024.10.22 02:09:03.689513 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:03.692642 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:03.989137 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_818_195} MergeTask::MergeProjectionsStage: Merge sorted 12352 rows, containing 12 columns (12 merged, 0 gathered) in 2.301442007 sec., 5367.069846831036 rows/sec., 856.44 KiB/sec. 2024.10.22 02:09:04.139456 [ 1277 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:09:04.169101 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1365_1365_0 (state Temporary) 2024.10.22 02:09:04.295748 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(d8dbbb2d37047102f59dbedc439cf01e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d8dbbb2d37047102f59dbedc439cf01e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:09:04.299323 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 750, file segment info: File segment: [0, 33554431], key: d8dbbb2d37047102f59dbedc439cf01e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1399, kind: Regular, unbound: 0 2024.10.22 02:09:04.303946 [ 1399 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:04.325536 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1365_1365_0 to 202410_1378_1378_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:04.353510 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_818_195} MergedBlockOutputStream: filled checksums 202410_1_818_195 (state Temporary) 2024.10.22 02:09:04.360372 [ 1277 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:09:04.363688 [ 1277 ] {} DNSCacheUpdater: Execution took 223 ms. 2024.10.22 02:09:04.368813 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1141 entries to flush up to offset 1167410 2024.10.22 02:09:04.386728 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89826 2024.10.22 02:09:04.453633 [ 1713 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/phm/vfykellvelazdywlknhkpefcfuybn, size 268037 2024.10.22 02:09:04.652236 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_818_195} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_818_195 to 202410_1_818_195 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:04.656420 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_818_195} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_817_194, 202410_818_818_0] -> 202410_1_818_195 2024.10.22 02:09:04.725759 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:09:04.742515 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 02:09:04.754521 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 02:09:04.807434 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 02:09:04.864728 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 02:09:04.867425 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 6.18 MiB, will set to 1.62 GiB (RSS), difference: 112.88 MiB 2024.10.22 02:09:04.925312 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 02:09:05.051188 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 02:09:05.102292 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 02:09:05.270014 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column ParsedParams.ValueDouble 2024.10.22 02:09:05.465267 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:05.470639 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:05.566392 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_758_159, 202410_759_759_0] 2024.10.22 02:09:05.567976 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_758_159, 202410_759_759_0] 2024.10.22 02:09:05.786603 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_758_159, 202410_759_759_0] 2024.10.22 02:09:05.937123 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_775_775_0 (state Temporary) 2024.10.22 02:09:05.938949 [ 1163 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 02:09:06.033626 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_775_775_0 to 202410_782_782_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:06.071215 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1167410 2024.10.22 02:09:06.074406 [ 1196 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_772_146 to 202410_773_778_1 2024.10.22 02:09:06.088211 [ 1196 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:06.097563 [ 1196 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:06.160098 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_778_147} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_772_146 to 202410_773_778_1 into Wide with storage Full 2024.10.22 02:09:06.184383 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_778_147} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:06.205019 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_778_147} MergeTreeSequentialSource: Reading 144 marks from part 202410_1_772_146, total 1166678 rows starting from the beginning of the part 2024.10.22 02:09:06.256997 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_778_147} MergeTreeSequentialSource: Reading 3 marks from part 202410_773_778_1, total 8547 rows starting from the beginning of the part 2024.10.22 02:09:06.468488 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 3.46 MiB, will set to 1.62 GiB (RSS), difference: -18.94 MiB 2024.10.22 02:09:06.580197 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_738_84, 202410_739_739_0] 2024.10.22 02:09:06.586461 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_738_84, 202410_739_739_0] 2024.10.22 02:09:06.604537 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_738_84, 202410_739_739_0] 2024.10.22 02:09:08.415631 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 89875 2024.10.22 02:09:08.675252 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:08.679900 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:08.760208 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 673 2024.10.22 02:09:08.849119 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_164_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:09:08.864762 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1366_1366_0 (state Temporary) 2024.10.22 02:09:08.942484 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1366_1366_0 to 202410_1379_1379_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:09.115111 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89875 2024.10.22 02:09:09.175208 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:09.184381 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:09.202045 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 12322 2024.10.22 02:09:09.274002 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_655_655_0 (state Temporary) 2024.10.22 02:09:09.459824 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:09.464218 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:09.520551 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_810_810_0 (state Temporary) 2024.10.22 02:09:09.574980 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_810_810_0 to 202410_823_823_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:09.586495 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_655_655_0 to 202410_665_665_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:09.647075 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12322 2024.10.22 02:09:09.649654 [ 1207 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_818_195 to 202410_819_819_0 2024.10.22 02:09:09.658998 [ 1207 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:09.660340 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:09.699867 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_819_196} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_818_195 to 202410_819_819_0 into Compact with storage Full 2024.10.22 02:09:09.771099 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 673 2024.10.22 02:09:09.923303 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_819_196} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:10.047473 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_819_196} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_818_195, total 12352 rows starting from the beginning of the part 2024.10.22 02:09:10.216398 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_819_196} MergeTreeSequentialSource: Reading 2 marks from part 202410_819_819_0, total 20 rows starting from the beginning of the part 2024.10.22 02:09:10.647020 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_712_136, 202410_713_718_1] 2024.10.22 02:09:10.650985 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_712_136, 202410_713_718_1] 2024.10.22 02:09:10.656395 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 114 entries to flush up to offset 88820 2024.10.22 02:09:10.661402 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_712_136, 202410_713_718_1] 2024.10.22 02:09:10.861625 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:10.870573 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:10.897037 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 11.18 MiB, will set to 1.63 GiB (RSS), difference: -4.36 MiB 2024.10.22 02:09:10.931271 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_796_796_0 (state Temporary) 2024.10.22 02:09:10.996760 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_796_796_0 to 202410_802_802_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:11.039884 [ 1239 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_797_10 to 202410_802_802_0 2024.10.22 02:09:11.165455 [ 1239 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:11.169284 [ 1239 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:11.178098 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 88820 2024.10.22 02:09:11.240952 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:09:11.271739 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(f346c62ee76ff95af1ec0d47ad105226) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f346c62ee76ff95af1ec0d47ad105226, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:09:11.275866 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 852584, file segment info: File segment: [0, 33554431], key: f346c62ee76ff95af1ec0d47ad105226, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:09:11.280270 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:11.333357 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:09:11.344626 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_797_10 to 202410_802_802_0 into Compact with storage Full 2024.10.22 02:09:11.403554 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:11.415673 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_797_10, total 4903 rows starting from the beginning of the part 2024.10.22 02:09:11.457886 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_798_798_0, total 117 rows starting from the beginning of the part 2024.10.22 02:09:11.475858 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_819_196} MergeTask::MergeProjectionsStage: Merge sorted 12372 rows, containing 12 columns (12 merged, 0 gathered) in 1.80677667 sec., 6847.553549603892 rows/sec., 1.07 MiB/sec. 2024.10.22 02:09:11.521657 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_819_196} MergedBlockOutputStream: filled checksums 202410_1_819_196 (state Temporary) 2024.10.22 02:09:11.631967 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_799_799_0, total 85 rows starting from the beginning of the part 2024.10.22 02:09:11.826395 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_800_800_0, total 122 rows starting from the beginning of the part 2024.10.22 02:09:11.833824 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_819_196} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_819_196 to 202410_1_819_196 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:11.847902 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_819_196} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_818_195, 202410_819_819_0] -> 202410_1_819_196 2024.10.22 02:09:11.893678 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:09:11.950628 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_801_801_0, total 124 rows starting from the beginning of the part 2024.10.22 02:09:12.003250 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_778_147/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:09:12.104156 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_802_802_0, total 114 rows starting from the beginning of the part 2024.10.22 02:09:12.366518 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(4e288cd16d24a3d8740174e03ec0775f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4e288cd16d24a3d8740174e03ec0775f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:09:12.371351 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 875, file segment info: File segment: [0, 33554431], key: 4e288cd16d24a3d8740174e03ec0775f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:09:12.386834 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:12.655593 [ 1341 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pqv/zveunaqtophfdfzdokwkvuzjxefxv, size 852584 2024.10.22 02:09:12.977048 [ 1198 ] {} DatabaseCatalog: Execution took 267 ms. 2024.10.22 02:09:13.080179 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_759_160, 202410_760_760_0] 2024.10.22 02:09:13.084384 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_759_160, 202410_760_760_0] 2024.10.22 02:09:13.092153 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1168950 2024.10.22 02:09:13.106486 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_759_160, 202410_760_760_0] 2024.10.22 02:09:13.152349 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 89942 2024.10.22 02:09:13.263753 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 02:09:13.288540 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 02:09:13.311444 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 02:09:13.446690 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 02:09:13.573117 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 02:09:13.582177 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 9.71 MiB, will set to 1.63 GiB (RSS), difference: 6.06 MiB 2024.10.22 02:09:13.642152 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 02:09:13.672447 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:13.678365 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:13.762848 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} MergeTask::MergeProjectionsStage: Merge sorted 5465 rows, containing 13 columns (13 merged, 0 gathered) in 2.581592078 sec., 2116.9107414653295 rows/sec., 938.48 KiB/sec. 2024.10.22 02:09:13.779414 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SilverlightVersion4 2024.10.22 02:09:13.833097 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1367_1367_0 (state Temporary) 2024.10.22 02:09:13.882077 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ife/disjuffxourbxrvexuroilyzzcxou (LogSeriesLimiter: on interval from 2024-10-22 02:09:02 to 2024-10-22 02:09:13 accepted series 1 / 12 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:09:13.899398 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1367_1367_0 to 202410_1380_1380_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:13.951578 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} MergedBlockOutputStream: filled checksums 202410_747_802_11 (state Temporary) 2024.10.22 02:09:14.020411 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89942 2024.10.22 02:09:14.111026 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:14.128605 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:14.208792 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_802_11 to 202410_747_802_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:14.212252 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_802_11} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_797_10, 202410_802_802_0] -> 202410_747_802_11 2024.10.22 02:09:14.366131 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.65 MiB. 2024.10.22 02:09:14.419865 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_776_776_0 (state Temporary) 2024.10.22 02:09:14.494396 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_776_776_0 to 202410_783_783_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:14.541333 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1168950 2024.10.22 02:09:14.817310 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:14.826443 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:14.842617 [ 1421 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:09:14.848477 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Found 9 old parts to remove. Parts: [201403_8_8_0, 201403_9_9_0, 201403_10_10_0, 201403_11_11_0, 201403_12_12_0, 201403_13_13_0, 201403_14_14_0, 201403_15_15_0, 201403_16_16_0] 2024.10.22 02:09:14.849981 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Removing 9 parts from filesystem (serially): Parts: [201403_8_8_0, 201403_9_9_0, 201403_10_10_0, 201403_11_11_0, 201403_12_12_0, 201403_13_13_0, 201403_14_14_0, 201403_15_15_0, 201403_16_16_0] 2024.10.22 02:09:15.252861 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_739_85, 202410_740_740_0] 2024.10.22 02:09:15.254325 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_739_85, 202410_740_740_0] 2024.10.22 02:09:15.267099 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_739_85, 202410_740_740_0] 2024.10.22 02:09:17.148832 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 12337 2024.10.22 02:09:17.269367 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gch/zemoffkbxrfxtdfiibwdtpvjzbdne. 2024.10.22 02:09:17.274618 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 674 2024.10.22 02:09:17.310575 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:17.331040 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:17.470925 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_811_811_0 (state Temporary) 2024.10.22 02:09:17.547191 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_811_811_0 to 202410_824_824_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:17.570343 [ 1261 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_819_196 to 202410_820_820_0 2024.10.22 02:09:17.575061 [ 1261 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:17.577532 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:17.581466 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:17.586891 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:17.645751 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_820_197} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_819_196 to 202410_820_820_0 into Compact with storage Full 2024.10.22 02:09:17.652700 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12337 2024.10.22 02:09:17.763347 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_656_656_0 (state Temporary) 2024.10.22 02:09:17.916531 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 11.79 MiB, will set to 1.63 GiB (RSS), difference: -7.92 MiB 2024.10.22 02:09:17.952693 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_820_197} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:18.028228 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 89992 2024.10.22 02:09:18.055989 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_820_197} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_819_196, total 12372 rows starting from the beginning of the part 2024.10.22 02:09:18.128401 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_820_197} MergeTreeSequentialSource: Reading 2 marks from part 202410_820_820_0, total 13 rows starting from the beginning of the part 2024.10.22 02:09:18.392066 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:18.404905 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:18.496629 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_163_163_0 (state Temporary) 2024.10.22 02:09:18.557476 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_656_656_0 to 202410_666_666_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:18.664412 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1368_1368_0 (state Temporary) 2024.10.22 02:09:18.716101 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 106 entries to flush up to offset 88926 2024.10.22 02:09:18.758671 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 674 2024.10.22 02:09:18.867928 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:18.870375 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:18.945668 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1368_1368_0 to 202410_1381_1381_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:19.092232 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_797_797_0 (state Temporary) 2024.10.22 02:09:19.150947 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 89992 2024.10.22 02:09:19.152515 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_797_797_0 to 202410_803_803_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:19.231040 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 88926 2024.10.22 02:09:19.136728 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_820_197} MergeTask::MergeProjectionsStage: Merge sorted 12385 rows, containing 12 columns (12 merged, 0 gathered) in 1.502518024 sec., 8242.829571540635 rows/sec., 1.28 MiB/sec. 2024.10.22 02:09:19.306823 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_820_197} MergedBlockOutputStream: filled checksums 202410_1_820_197 (state Temporary) 2024.10.22 02:09:19.366889 [ 1219 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:09:19.420697 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_163_163_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:09:19.420496 [ 1204 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 02:09:19.499856 [ 1219 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:09:19.700756 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_820_197} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_820_197 to 202410_1_820_197 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:19.710158 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_820_197} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_819_196, 202410_820_820_0] -> 202410_1_820_197 2024.10.22 02:09:20.072115 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:09:21.243404 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:09:21.246138 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:09:21.361781 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_760_161, 202410_761_761_0] 2024.10.22 02:09:21.466359 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_760_161, 202410_761_761_0] 2024.10.22 02:09:21.474775 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_760_161, 202410_761_761_0] 2024.10.22 02:09:21.561400 [ 1238 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 360 ms. 2024.10.22 02:09:21.665797 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1529 entries to flush up to offset 1170479 2024.10.22 02:09:21.736260 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_613_125, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0, 202410_617_617_0, 202410_618_618_0] 2024.10.22 02:09:21.739223 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_613_125, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0, 202410_617_617_0, 202410_618_618_0] 2024.10.22 02:09:21.765606 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_613_125, 202410_614_614_0, 202410_615_615_0, 202410_616_616_0, 202410_617_617_0, 202410_618_618_0] 2024.10.22 02:09:22.971592 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:23.008818 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_740_86, 202410_741_741_0] 2024.10.22 02:09:23.013794 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_740_86, 202410_741_741_0] 2024.10.22 02:09:23.016691 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:23.039471 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_740_86, 202410_741_741_0] 2024.10.22 02:09:23.161077 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 41 entries to flush up to offset 90033 2024.10.22 02:09:23.213495 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_777_777_0 (state Temporary) 2024.10.22 02:09:23.313494 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_777_777_0 to 202410_784_784_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:23.302304 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(211593ca6bcf205729a7c16a879e921e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 211593ca6bcf205729a7c16a879e921e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 02:09:23.333109 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 211593ca6bcf205729a7c16a879e921e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1396, kind: Regular, unbound: 0 2024.10.22 02:09:23.340267 [ 1396 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:23.350250 [ 1160 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_779_779_0 to 202410_784_784_0 2024.10.22 02:09:23.355440 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1170479 2024.10.22 02:09:23.360778 [ 1160 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:23.363869 [ 1160 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:23.385761 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_779_779_0 to 202410_784_784_0 into Compact with storage Full 2024.10.22 02:09:23.420701 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:23.427792 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:23.447428 [ 1172 ] {} DatabaseCatalog: Execution took 469 ms. 2024.10.22 02:09:23.546804 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:23.552170 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_778_147/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:09:23.553177 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_779_779_0, total 1759 rows starting from the beginning of the part 2024.10.22 02:09:23.573107 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_780_780_0, total 1706 rows starting from the beginning of the part 2024.10.22 02:09:23.585443 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1369_1369_0 (state Temporary) 2024.10.22 02:09:23.601279 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_781_781_0, total 1169 rows starting from the beginning of the part 2024.10.22 02:09:23.630743 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_782_782_0, total 1141 rows starting from the beginning of the part 2024.10.22 02:09:23.653116 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_783_783_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:09:23.665077 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_784_784_0, total 1529 rows starting from the beginning of the part 2024.10.22 02:09:24.067896 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1369_1369_0 to 202410_1382_1382_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:24.115371 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1377_34 to 202410_1382_1382_0 2024.10.22 02:09:24.154897 [ 1277 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:24.157076 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:24.221244 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1377_34 to 202410_1382_1382_0 into Compact with storage Full 2024.10.22 02:09:24.244894 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90033 2024.10.22 02:09:24.519262 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:24.559712 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1377_34, total 10619 rows starting from the beginning of the part 2024.10.22 02:09:24.764058 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_1378_1378_0, total 66 rows starting from the beginning of the part 2024.10.22 02:09:24.880531 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_1379_1379_0, total 49 rows starting from the beginning of the part 2024.10.22 02:09:25.113314 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} MergeTask::MergeProjectionsStage: Merge sorted 8844 rows, containing 4 columns (4 merged, 0 gathered) in 1.73640147 sec., 5093.292163591638 rows/sec., 79.58 KiB/sec. 2024.10.22 02:09:25.133134 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_1380_1380_0, total 67 rows starting from the beginning of the part 2024.10.22 02:09:25.155554 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 12346 2024.10.22 02:09:25.162853 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} MergedBlockOutputStream: filled checksums 202410_779_784_1 (state Temporary) 2024.10.22 02:09:25.224977 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_779_784_1 to 202410_779_784_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:25.239378 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_779_784_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_779_779_0, 202410_784_784_0] -> 202410_779_784_1 2024.10.22 02:09:25.287482 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:09:25.305449 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:25.316533 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_1381_1381_0, total 50 rows starting from the beginning of the part 2024.10.22 02:09:25.318817 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:25.410699 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 12.52 MiB, will set to 1.63 GiB (RSS), difference: 2.19 MiB 2024.10.22 02:09:25.426573 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_1382_1382_0, total 41 rows starting from the beginning of the part 2024.10.22 02:09:25.468343 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_812_812_0 (state Temporary) 2024.10.22 02:09:25.864064 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_812_812_0 to 202410_825_825_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:26.001107 [ 1192 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_820_197 to 202410_821_821_0 2024.10.22 02:09:26.008062 [ 1192 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:26.037023 [ 1192 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:26.051663 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12346 2024.10.22 02:09:26.071065 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_821_198} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_820_197 to 202410_821_821_0 into Compact with storage Full 2024.10.22 02:09:26.219524 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_821_198} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:26.262814 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 675 2024.10.22 02:09:26.349163 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_821_198} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_820_197, total 12385 rows starting from the beginning of the part 2024.10.22 02:09:26.408820 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_821_198} MergeTreeSequentialSource: Reading 2 marks from part 202410_821_821_0, total 16 rows starting from the beginning of the part 2024.10.22 02:09:26.497431 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:26.515001 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:26.759827 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 92 entries to flush up to offset 89018 2024.10.22 02:09:26.994468 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:27.002902 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1277_14, 202410_1278_1278_0, 202410_1279_1279_0, 202410_1280_1280_0, 202410_1281_1281_0, 202410_1282_1282_0] 2024.10.22 02:09:27.005258 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1277_14, 202410_1278_1278_0, 202410_1279_1279_0, 202410_1280_1280_0, 202410_1281_1281_0, 202410_1282_1282_0] 2024.10.22 02:09:27.006935 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:27.025177 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_657_657_0 (state Temporary) 2024.10.22 02:09:27.199062 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_798_798_0 (state Temporary) 2024.10.22 02:09:27.250208 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1277_14, 202410_1278_1278_0, 202410_1279_1279_0, 202410_1280_1280_0, 202410_1281_1281_0, 202410_1282_1282_0] 2024.10.22 02:09:27.361384 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_798_798_0 to 202410_804_804_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:27.380346 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_657_657_0 to 202410_667_667_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:27.410928 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_761_162, 202410_762_762_0] 2024.10.22 02:09:27.414065 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_761_162, 202410_762_762_0] 2024.10.22 02:09:27.426499 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_761_162, 202410_762_762_0] 2024.10.22 02:09:27.457707 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 89018 2024.10.22 02:09:27.531026 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_821_198} MergeTask::MergeProjectionsStage: Merge sorted 12401 rows, containing 12 columns (12 merged, 0 gathered) in 1.468103693 sec., 8446.951028819461 rows/sec., 1.32 MiB/sec. 2024.10.22 02:09:27.621232 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} MergeTask::MergeProjectionsStage: Merge sorted 10892 rows, containing 13 columns (13 merged, 0 gathered) in 3.440402214 sec., 3165.90890323151 rows/sec., 558.58 KiB/sec. 2024.10.22 02:09:27.708515 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_821_198} MergedBlockOutputStream: filled checksums 202410_1_821_198 (state Temporary) 2024.10.22 02:09:27.738649 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 11.06 MiB, will set to 1.63 GiB (RSS), difference: -6.30 MiB 2024.10.22 02:09:27.840558 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} MergedBlockOutputStream: filled checksums 202410_1207_1382_35 (state Temporary) 2024.10.22 02:09:27.952490 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 675 2024.10.22 02:09:28.141835 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1382_35 to 202410_1207_1382_35 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:28.247899 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1382_35} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1377_34, 202410_1382_1382_0] -> 202410_1207_1382_35 2024.10.22 02:09:28.260662 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 90112 2024.10.22 02:09:28.329441 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.71 MiB. 2024.10.22 02:09:28.355034 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_163_163_0 to 202410_166_166_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:28.387415 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_821_198} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_821_198 to 202410_1_821_198 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:28.414306 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_821_198} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_820_197, 202410_821_821_0] -> 202410_1_821_198 2024.10.22 02:09:28.453540 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:09:28.694146 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:28.696577 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:28.954601 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1370_1370_0 (state Temporary) 2024.10.22 02:09:29.084938 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:09:29.122419 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1370_1370_0 to 202410_1383_1383_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:29.195070 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90112 2024.10.22 02:09:29.557664 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 6.70 MiB, will set to 1.63 GiB (RSS), difference: 5.68 MiB 2024.10.22 02:09:30.359625 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1509 entries to flush up to offset 1171988 2024.10.22 02:09:30.839341 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:30.849250 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:09:30.964375 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6551 2024.10.22 02:09:31.171621 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_778_778_0 (state Temporary) 2024.10.22 02:09:31.244734 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:09:31.246550 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:09:31.274643 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_778_778_0 to 202410_785_785_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:31.406389 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1171988 2024.10.22 02:09:31.584428 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 14.78 MiB, will set to 1.63 GiB (RSS), difference: 8.78 MiB 2024.10.22 02:09:31.645684 [ 1164 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 252 ms. 2024.10.22 02:09:31.868587 [ 1419 ] {} S3ObjectStorage: Objects with paths [common/onn/nvjtminrmtlwdonfmwrqewfhvlcom, common/hrs/ygtxtbimaujtvffgnaqjovnafpbpq, common/abk/ulnoewafzvoohdwzonwqoqekbkmcb, common/muo/qifudbmywgtowhxxgflqixufiusjo, common/pou/tbceesqlbqmvgjpqqkzyaehrheagj, common/dvw/jwwubrezrglubjuqxjkcrzjaxfpha, common/kef/eriwtbthacjuiindrpvycfmdtvtez, common/yip/xvdbzmwixkyywmbiyrimroqjjxgig, common/ubj/ptoyethkuykbnqpnbgtijxlkbtbmk, common/rwf/chajmnowoxokbgzjcrohwgmxaogbg, common/boy/amgpunvcoiuypwtvywpnssbnayopw, common/nei/ykpthwlbqmbofldtbkxvlquhiefik, common/vfb/pvgrxuxdaqbhrywjsnitslxhvadql, common/hbx/jfetsozqeuyvbfzhmtmvqhsgmtfvj, common/pdl/ozoekydifpnzfymfpfjsuxcvgggwd, common/hxu/wousmqiyizclgjmgvctckgfhjnamr, common/xgi/vybdhrfaunjybuaufcsxulgbbdiez, common/rxi/ktkyvoyjletrxndlxlceasygfkrgy, common/qft/jlyowcssvbcwrkxtetdjnqlvxrcii, common/ihv/yahbehqxhrgifhyaajprtfytcbycy, common/ykd/nuzmwnprhogkaonxrmkzoyevnemul, common/iev/lcctzroanlrsukgznwaqkmyeqsqdy, common/pjd/fhfuarcopooynntgeorvxcdxjvnsc, common/ork/drgznpxowmkqhurcnmgrjhziqxvsf, common/wqn/acekqllxygetsifhhfiyhxvhjzqvx, common/zwf/hrjbgqwyydzedubibzyelvjxrnaml, common/oin/vrjdynzennfqckbeneedybnnanmzc, common/bty/mvahxifnrqqwfuuzlpbqciwtffbuk, common/erw/czxrmftifbvgsxryltciitvgxcsdx, common/vqq/bzuyujiarrbphuhzluoachvvmdfmm, common/cwu/zuztbvsehjfwsjpbxnorswmtzjkgp, common/img/kwrfixqhylnskqdplugsqznojkizi, common/vdq/vaskzxjgoikpkalunqmyldtwnlpzt, common/hio/downgqeevdoxspledwnpzlytaglak, common/rou/gqlfoscyaqdaubuhdmqtbrlctkggx, common/bep/yayqodpmbxkfmwswzashnizoduiio, common/wet/gvxplwaopzmmubtntdfhbnbukbaqa, common/ajv/ifvvlymaxbpefkhuwznfkwouohfxh, common/gkb/atstpvcvneqqveuoyryrjylxrquud, common/mob/pbjflcyvneddjyycknwaaibjguuxs, common/nfe/psqdvvatjoakyuvpyqeojnbceooxh, common/bja/fhioqbkxcygjfvomqftqhrscjffcg, common/txk/mvamnlemyexadrmzuubmgdajdgwba, common/qyk/qqvvvvjhmktmpppfbmnezpeuthdzj, common/ofh/kgsgeqyvhegcaueqzwsrxfttqzcam, common/gvt/yzsegwcbfklvgkgsfnwizvdklbnhg, common/dzk/izxtgqyiqmgamynnbrjzwbyugxsdn, common/rng/fozdprgybcznxnyglsldqogjlmocp, common/oyb/vlbjnmhomyzjnszfhllwufomgypne, common/qtd/frqtmieikybnjgeepsxxihdqozgda, common/qan/gvqkobkrsjpwqpfgcmwpvsuakjugw, common/zar/afzgsuianyxmgxphagqediapipblm, common/fom/foolgpcbkragfzpnmdbyyznxpckax, common/zjl/zwvftiyribvwdviwxcfdpspodgwlp, common/sgr/wzodeonoyzsdppomqmwtrhedcytws, common/gbf/tpirdliuzvvtzfpqlnstefotqlbjz, common/qhb/xamsnoqtzkmlycavkwmzkqjjwgiqw, common/ane/mzsnkqpokzipockmtceqyhznqyoox, common/rlb/mugozdtwstegeeyuyumjttggpxtkg, common/vzw/yxjsgwakggedcrlecovmagsycdsux, common/dce/pjeqykwikoouazlgthkuqkwrjmkcd, common/cdj/iaoichimayqxfoscgrljlhbhetpeu, common/gfu/jnkgxupjdaycfjmrvlfpxlduuqjze, common/nod/vkxxlwsmirtjjtjcrwmzgsssdbpfa, common/prd/wofusmhghaovwurlqfvlhkumixomn, common/qvx/vtowmcirtdrunklcrkcfhxqrnsrcq, common/fgw/dnyduiirgqmfmgerrdhqhmgtnropw, common/rrd/eevoqmxbrpolcshamxbxymndujxop, common/uyb/ltiqftblajmkhcadzmjacaorltwqs, common/iwh/vlsnfxdngwyzszaectveqlfulbvyk, common/pwe/kbzyisyilfspwoveterxhqlyydstg, common/vkn/pneomotxdehkfcgxvkhdnnxpdsvrr, common/yam/fyiuocucoydnarhtycpnamnafjzwp, common/ort/ekzgomgepabhdxyejjrnhdsabytty, common/hbs/llkxspiqdkgbxwctopedblvmwkgnx, common/ige/xxukikopyidvgvslntryurphdvudm, common/fpk/fvhqqyjknrlsulifejnwoanoggaei, common/spk/fltpnrdhhacbmoprvvjayphzofzdp, common/mqh/eaifpwxtgngtsmwxkwucsyvbalgnf, common/amv/rngudatealpacxkyfmzbiccnslyem, common/wrm/rdbmsagxldfihkrouqyozfkfufcdy, common/zxb/rkutsyusneensioakysncltwzhuhh, common/bjh/rokctvhuwvexuwxshidtjtncpvxfe, common/ezw/pxcqdbjinzusuthhpmyvmkakavool, common/hrb/mlobmlnibavgxaqlhpcqfjwbyuwlb, common/teb/bcaqfmjxgoifpcfzmypyqjduodref, common/llh/lzguqxssblxhhmxyzwbhlrntxxvrs, common/eum/ivshuqwlvionndxldoegyyzybuttd, common/yqb/edbockqzdgzoshzbwjjbeuppditwo, common/raf/ysspsbzjwnbqilkhfurrsapcyjraq, common/xmi/ulvesqjdlkawaezrirkpkqbeuexdf, common/ecp/mxpxabcycffnrwsgbucxwbgqhqwoh, common/qfn/jlcgcwebvgokqdgtuikoehchkmybj, common/kdm/wcsaqmgxpdqorrnfxeupobhkestjf, common/uih/poobtlfiftnfobllrxnfkwsmoryga, common/huo/ddmfalejbbwxofkvvsnvthzmsugjy, common/kzi/jrwmostwsvjnyoejulptgcumttkdr, common/yro/bcfndcrinkbruddibbmughtmvepxv, common/ojr/hcddaprofhcunlaorrtabjgcwzanr, common/cnb/rffwseujltkinqkxciclzdkrldnxq, common/keq/rebhizcznkaagotjxnodchmjclces, common/eum/szrwadgdrovtpyqbpdrrhrvxbicms, common/wnr/isjliemtsgnhwblhnytpzrzhpsbci, common/sto/ikiqiiqpgkwkpqjulgjvshfpbyege, common/dqx/lqiomtxulopkvvkeqlcyrpudphvvd, common/dmy/nphbftdgwgwtbaqtzidsazqnxuzri, common/lzp/wsuwyxcqyuhquaikibftzrbixiayh, common/pio/wmcnyxcdyqeripanwqlgedxvkxcpj, common/eih/ilvklykkjqlowrmzgpkthrhzkaqhp, common/xaf/zqnehtpwbyqfzlcrudjjvnekujxmi, common/tcc/qdvbdmrvgazmfuiptijnnixyinhco, common/ztm/ufkuasnaqjbqhcjiimfujovzeqxuz, common/gbu/igtdqjvhcoqyqycisdcggdtjztqbw, common/pma/jvpnttrjjtzfbmhlxfnfbaxfgnauc, common/ipw/yeomylkrtiejmtlzvaictdvcujpli, common/vao/sghednwnbiubduvgbpjlqvyaewfge, common/hdq/tsfejtcqiphtubxnwpnovhxusrtqt, common/uzj/aseogqargoydernmxrqhfmopslhdm, common/mpp/dvilbfgpiayfvyxaxxtqlupeuxpoa, common/pwo/wqrhsznpdrearbzurrhlrtcicyhgg, common/afz/mwyqcbuzrvndurmoxcwebavcleuhp, common/irs/wuuhgqyxhozpazlgktdjszdrdazim, common/jku/wpwckusruytvntgvnjhaajbkfqduz, common/vca/tlifyzkjpwwsorbfrivopgbgyvjct, common/qxd/hwmygcnhrthtsgxyyvqvylfxhbaon, common/rcf/hbcdoenalqknbsfjwmfjwiygqxgab, common/iqi/uuzajvtmcvaxywwykpkumgkmmroso, common/rms/ucesrjiwijocipjrbesynapkpvnyh, common/ion/ycfygontvyknmkcxdqnlljlqrkoxs, common/grv/kfecaqwaanlxvynjspmuefrsklgoa, common/mku/ddnrzqynkjvfeycujvmjdnehnuicj, common/vtv/olkwjkxgxtlqvnfkvvensjteeqnbu, common/aqh/dpjycsjsonczmlxywzimaeckmhoay, common/jvw/pnzfxvdvuqeipqcwxffewmcmvlzta, common/uye/svqjdllhbpclddghrjxvvqoewqcny, common/gcc/dizochuznftktirlqwgidzqpbapmf, common/jdp/ycbvfhuhhifdafruihbggwarlteta, common/soh/nhkjnusjebodeypunvrdvolesgxog, common/ijg/efpjnebusxlwfarzxoqnydrqqbepl, common/hmv/wraddmeqbxjzexfqodchkliydbbau, common/vet/ntczhmohwbudizlrmpowixrteblol, common/fpy/bdthajormtoshzvanxqsdeurccafn, common/ypr/oghgrfrbludguyxjhvrtjlkbokfot, common/lhy/cmireqhvexzllblrclitrefnldeye, common/yus/dknssbfioypszfdicgvwpxdlksykn, common/kls/eetfxbskqyyyjbeajtrapetffnlkz, common/hwv/gsjlygwjvhgawbmenlqjrrfirflub, common/pll/mvefbwxyifquflefsteuzjatllyun, common/wke/apptcdxuzgvmanshntvjbvjampmxg, common/cyn/wtjruztcfeycspeiwspwsvfzkjflz, common/izs/jkjysivefxhbvlexldbkistxdeakx, common/eio/paxluuejhvaxjxxcmfmciwduffdww, common/slb/wuzigqcqgdswadfliznfvctoyyxlx, common/tqs/euetpcjpamnwlhghghxppexsllqql, common/inw/jqrpeeteeihfmrrdimdhvqctnrvuo, common/cri/tlcjrpogskykpzvwciamxbqkenhvy, common/tev/iddormknlfgognkgysgjtlkimdfad, common/axi/wkxcmhdekqhieeowjltnbrbnfylbl, common/crm/zhbpnoatvyyibffsnelrpfjcfovjr, common/abs/qffbsmrbcybbwnjnuowboizkqwjpr, common/rpu/tfsfljedzekxpfuevrzuejacwxfyh, common/mij/xcuiicgakomdwzpvrotfepxanayyy, common/wwv/ppqpihljfdlfpsnaflevlgvkgvmov, common/dhh/afetoisqedglvzjnbvnyamylgfgog, common/ags/xnerfnhvvofzneoefbsuwmxohtguu, common/wof/wziluvquazazomwygdsgrlrbtxkfz, common/aja/hgabjcyneppfjjpxjpgyihrotmhul, common/ipq/ocwxzwpiwzsjucmurdnwbevulvukk, common/lsc/wntybqqtonshfjxrsjtxqrqcsygwg, common/sqq/dslrfaocmfbpmyqfgijcsxqxiqqws, common/rhd/pifyscmbluiubpgztildpnlrqsgcg, common/zdm/logxgohhiiivnffpkpbhfydbkjrjm, common/nug/hdgqrbcpovzhwezxxszchgkbdsupi, common/nng/rezkxaqbawvywcxyhyqdnfeenzkwt, common/dgs/igkeqsespfmoiflaexbbdqmrcvrzh, common/sol/inffyitovcwdniozpfxlsyelshydn, common/vyz/dtwhrpiunxsjghcdmaqtippexxrqt, common/zof/jerexxjimvvudgigwqxprbynovhno, common/unu/lxkkenzfjgalivdvfaovgljqslrwr, common/hjp/qabrhmgqavxwbfxnvfbpssnhtwczc, common/gmg/aamebeyefmksuamaolafmryuwmkyj, common/icx/jgdadtzgcjpowvxrsjywhddcejclc, common/xxb/hdshrhdaykgjozgojmbciuetxyaah, common/isc/mkbltjmvmxcmmkvskyssaymcpniqy, common/enh/vqnvzcztdnfcocptjrfwkfmxmcgiv, common/ckg/ibijooifothqaqntgxpecixntyqqg, common/zvp/ysfduqneddgbdokozbfjdafduccan, common/mmg/btxuhlobszsfljsedmlwqmmktuank, common/tuy/eeaipbnfztngkvlzpasadwlvbgqdv, common/qbi/xyggrhfxlezapnsgkzuhhfiqngnxd, common/hap/ycmemkjfdoqerxtpbqpdmkaueevvo, common/ugb/hzktepthifdnmnynojgirkqtsoqnj, common/ddx/syryllrbcllebsetemcfaccwuzege, common/rjt/wklhugzrgbtnnqamoiymyoogytait, common/vxf/yavlnrjoeatdrptaqjzrhkikanhzp, common/ofo/khwoswaggkovpcxqtrapxwzgbpfyr, common/mpd/vwmprxklqlykrvfiptagterjqdurb, common/seg/ulewajxljmewcmupxezeglgyduxds, common/msv/jwfdtvgwbcweykolrokqvtbusgfaf, common/tbo/zleshvvjjwtqjmkgjtsgtesphpqbx, common/imk/khoivfhoyvxxfylkbmqlwvxtbyqtc, common/fqp/addhyiaqlqavtjabgetcxolutxqap, common/rdq/dpgohmjjcbppfqyjokmthzeuwizic, common/etv/niorxvlttjsqfbcosaxptlonpjdjr, common/ysq/nwrxarcusdvjpqjwmqvfjzcbhueju, common/pcm/wcvfxbfvnlpsyalvggpruimygxsoe, common/qdj/wzypgfoimwruiiyxcimvfjgcpwcna, common/auh/pqznkjurcabufrjfbvlifvnutudzi, common/tnb/hudicoovrrxqropqpcsdtuvjsrhiu, common/xqv/zhpelvgwiaterpwmsdokwiqfqbthm, common/qfz/zxjrjgwdknohuxvfclvsbvrmdpppc, common/jml/qpruieisvppntmisibicxnderrvam, common/ebx/rlzwimcctziwmrqhyqphkwabkipqq, common/aqo/ucwecoxxjcxecidjvrnnhokevqlcv, common/rwb/yjyctdduxltwwbaubydwsiugzftdp, common/mtm/fmnwlcrygqcrqeokddizznubbujfc, common/zsb/bmxumqrxrkftqbjtmychpcfcaafeq, common/uii/hbltbvrwqpceiyizukdxsfcrhvxsj, common/ygc/tfhvpxnmybcqsmzfpvuabeyaftrtv, common/gal/oteiikhnimxztketvfobcfkkfbzoq, common/bmy/spuydrmaqjqzunqwpczlzmmypsbay, common/uzu/walyhpqlqvlkbgzgxmccwnqcoxhae, common/nvf/nlgdzgluelbvnrsjpvvdcqywlrbij, common/iyr/tjhiueaxnlzzbpfthxnkbrnqktrne, common/uxh/ppqvvhukgxilvtjpmxwdrxyhmufzs, common/bxs/pneafyyyiqihnysblpselpdjdztph, common/fju/wuckysibrbxipzylxtznhxlysfcxq, common/gfz/cgxgshmqkeetpuxrirawjysdnsjpj, common/myi/ujwebwvbfyzkgwmdukrtforfagusx, common/bfn/znktojzstxpzaeahstowmywlqydat, common/roo/vuxcybwoidwfynswbhtcqzswpoaxj, common/vqe/jmddthcdgnoqyghhuwnvxztucwtct, common/jkp/mpjvgewcfwljbgelpkxgicbrhqdex, common/gau/gxgvzntwvfmsztxmvccapytnndebh, common/dft/hawohmrnyyilvtiawfoinqdkwszfd, common/lsf/zmepnkdhazaqsyglbmiybtmapzdqy, common/cks/xupjcqipdkcdulzhthcuovswskwsw, common/hdb/aqprcpiisqhrvgdyfoiessxugbwit, common/rpd/uqyinvlhpoztlltaqawtovcfhjsig, common/tvu/dtvwvlgjnhsbgksfgjkrfkfqmuieo, common/ljn/cizvhqknagjeecadisiysctazosfr, common/lvb/nrswguimiitkocdodhaeoqivtnszg, common/fbt/ehncayzzeeoyaouvwbbayigrwnnoo, common/ill/tmbvadghkxzhbfmsjdrqnkulfidgp, common/syv/xgssjjrgyufhuhzgzwxnitsbhrxjd, common/mnj/uvzplknnqlyajrflktkqtcvfpjhhj, common/tcd/xgvpdvpxsooufdmbfcpawkblnnezh, common/vvd/urnwcpaokbiyththjgsjmgrpzdmqi, common/log/xsaxbxbrmfjkxwqmyeeijabldjvon, common/zrq/hrmcruotaqhclovlxuizqavnhjcil, common/cwk/iqzwlibghsgvmpbrwvopqssstttlm, common/zig/tqymnkvnzvetvzuivadpugyxxdrqf, common/njf/bakfgvlabhgjljymikdigcnknsyyz, common/luy/zrgzewleabcdmljsotbgwojubukaj, common/zmh/ccqrynutyfopbocxloxacaynerguv, common/lhu/xdaqvxfraokcaiyimjuacchdckqmw, common/ayn/fkzcjcuvpiuilyrhloxmodliujydb, common/ktb/tztmkgxlzkvcptmbpmpktaqnbmbxa, common/yfo/cebtilbmukgbpayypkswlyjmmxrln, common/xow/ftplhqerttayvwkywhzldccrfnnqf, common/dwk/gjtgistgeqkugzfbskkzrpqoplfbx, common/gxq/mohkgbkaihrbnfmiktnazzoxmbaff, common/tev/rxgkhmzflwbtaxtqduzlwinngurfz, common/gbw/whzkftfyjhqsjbdruegnphflrucwf, common/dme/cxtkjovrghnepydfmaihdbnmzehya, common/rwq/rjnnlunighmpybtaecodbbfooavhe, common/lky/ahinveiwdaaqqyymloodhogslbudp, common/heh/hrveskaddleakxzximdzwszoxolwu, common/pnw/vnfkitqywkppcdyxqfslhtbdhkimo, common/wmj/fiidnslqekwwcbaldaabapawiqkjr, common/qwg/czdfkdiepwpfakyibjtqbiyhizoki, common/qui/fzfmpdeqtkwwbnlwbufuheuwmufxe, common/jmv/vixuyvfrneueeqbvuuskmwrugomdg, common/agc/dnhscfphorecuyzlfytuykbxbtcmt, common/vgk/fpouqcgvqfosvcqbracwnfndrzqpr, common/zls/fnvuhywohrovjsqhkkwbreuqresom, common/ebu/dgacxrofwqwwqbzmfmwvkkcwistsa, common/arf/vutdpjmkuqqnxzidfmamyajddobpx, common/mjv/lzyqqmehbqogtgwgkmkyizuinhddk, common/xyj/nduromgpwqddjdjzafjundhxmdqmk, common/pjp/vdwwkvnvkquvsqavikeieeoeqgrdp, common/hjm/eshevhfpqdvdrvireeaybqhwgqira, common/sjv/cxrczqhlokjfiepfvcivtpsigcxes, common/fwv/lqmastlyxfpewjzklkbrtdkjmwehr, common/jtv/usjssyyorknpycwzxlalqsqqnhsov, common/hka/syhpyubxjlycvfzrwgvmaflbwpwyb, common/rix/lekquzeazbhqvtrtzmeolugdglrlm, common/gle/mpqrusndcjvlxffdgeiemdqqpvkmj, common/wbf/jtbyfwasukpfsuqydffmomhzbinxs, common/hbl/jdliqjpdxiuuvlgdyzpxylgovyktl, common/snx/idnfjikonlvmklmfxffrwhlzezlfq, common/eke/tivfpdqcgtqsvpxpvqrlbjrmlftho, common/nyj/yoqlcdkznyowtfqofbnlogvwafqbn, common/too/srizatbbrcuuqrdkamjuhkrcblool, common/oki/lvcozgxbuzzqjtssyoncygacrfhym, common/tfn/yrypzhaeudticfrdlocnnzzrhudmo, common/fkd/kiztbuyoijnfgsebzkirptdrdjxgk, common/cox/gtziygnortguzuevqeeorujauhvxv, common/evt/lfmvgjlwalxqwiymamynkilzuojpk, common/pxb/hvfouypvctrjrrmsqjhcoocfrjzqn, common/byc/vrznieigmnvebsaehufbfdiezcxvd, common/ugg/lwsqukzpcfexupywqlktkhhoqjulb, common/iqs/mknzppjlfexztgutlqchvhjfjzkgb, common/tfd/affrfgjblbgzrqilmfrimptelplmm, common/grx/toggmehvdqnwemgvihdfiykhlzsxt, common/gok/cfexnbxnutjiyctozapgtkjrvrfey, common/uxf/chacilsvxbihlxggonszyacsrzxna, common/mby/knuwmxxqhkmxooigbxprotchzpote, common/feb/zygbmgchkwhztptgwyufgzzeaecvo, common/bek/xrmyztttkfdpbdsvmxqvxugqlftog, common/ppa/lcjucuxlhenvfiymvisssumrjqhqa, common/ddz/ltnmafsshezbujtkyekczwxnsblqh, common/zff/pksozksrroupjbmhjnwijjzkqnvwp, common/fns/tgvoajknypejwyyhldyxnjlpxpgyv, common/qgn/otslvjggwecehcarztdqhelplzllt, common/wuw/qovvcmeehxnhngvjpjfkpjxnvegrx, common/dms/vhdjtucwhdvpsnjcqytuybabglnex, common/iuw/miqtdhkcegvubhgvjcnijwolchclu, common/ujo/mhodnijqsnkqvjowkxzxscbvlbkaq, common/nwz/plfihhkpploypejnixvpliklzizru, common/esq/oicsuknxjwwmwjezahadhcypiaypj, common/efb/gsgxocbvovivzinokjftorplvrcqx, common/isj/dwrbqnenzxdontehscrqoaktfqjrl, common/jvp/nxfrzmpmrojuxhwwkxstuzkvntzcu, common/fhf/mxhhfgmnxeggyrlketfhsylmttigu, common/fen/yjycbndshdxmkyuzicxwafhwvwaol, common/iqb/rvuphybqatgeheqiknijuknxmjnen, common/exk/laggkeqwvayrnurztnolfsuhashvn, common/qzt/oatdikihxfcwkrylzfiqukwzopama, common/abp/oqqiknknfakltimyxqxivmnymrqzw, common/uoo/jarljwlyskkezsblauellxygbmizi, common/ivp/nxrlcecqqznwjhnplsjbxagasudml, common/pmc/driswryozfatkocagzjomveaczqlt, common/biy/mpfphggdwqyjztqbbqhcaaphhiipl, common/ptr/xjcsduuviwuejvzelajccabskbdqn, common/bht/vfxedipjiuwwwyygmnszleuhgogkn, common/mai/raclwtgstcdsyekgddnixffrmzrhx, common/bhr/attsryhvnlsfmlikgvmvqtmtimzit, common/ecc/lhdbmdidyslgkffcqfpdvhmkbonvw, common/fpn/ebjktjqgttdditfsswshvpgcwpths, common/mtj/hqrujmhfqrsvtskqfeanmbejckovm, common/tmk/wrjukzkvqpletryshegwenzqqhzkx, common/cip/onhsjuyffnmwczntpmnowwfwztmfe, common/fje/puqjkgviubnggrpfqukgttmeeswnc, common/kkj/dnbusmmiaqftnkaaqvkzrsjdoaojs, common/jws/iyqcolrztltrpdjkogoyvcfgdbzhk, common/gsg/rbsdwdvzyjdwztssvaiwegetkwexa, common/eem/dagfggvzvbbirbmwvnjabexssmend, common/ewo/zfcdocyacetwssdtejegujvmluley, common/zuv/fmjmegjhadvpvvscqipqnacmtkqxv, common/mqt/pvwdkkkeecgxrustqvxifmhcbbxoe, common/qts/tfghpcjaqlqybreljymayeaqejwrs, common/dlz/xrnqpmikpznzsestkgiekxpppyfsu, common/llf/nqtohfirzlevvvleokorlfklrveox, common/loj/gawdhiezdzchtjsutdxyiordjzzyr, common/bjc/crsqcdjhcvbhstiwiojeeulmwlaor, common/pfh/cxbsapvnkqlheijsxjmbnctkajotx, common/jys/tonvxoqabecrnhqucoijvvhmptsnx, common/ujv/gayueywyzsuwsnricwxvmpgqlxomb, common/cpu/qbkuglbkytswqfgsemnuzzohogajj, common/fgc/brexoykjvbtcrowdkmpfwznqbpgqb, common/xcc/zsssdelnefegvcxaxidynpbcttaoh, common/lte/elwuvvwgoylbyivcneaewwdcyalpc, common/lpe/qgsqzrpjqpbymlprhjbatsdunljtr, common/peh/ucrhknbbiaxryqwvxuqqlfatxuxtc, common/brm/zxhnwhqemszweuyvgydxrkagtyjjt, common/grg/aqultrtzvjgenvsdrrkpcapwmmqwm] were removed from S3 2024.10.22 02:09:32.143191 [ 1419 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CounterClass.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsNotBounce.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/MobilePhoneModel.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SearchEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CounterClass.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/MobilePhoneModel.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsMobile.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsNotBounce.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/MobilePhone.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/MobilePhone.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/IsMobile.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SearchEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_8_8_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:09:32.756313 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(1bdcb403fd6598a943fed5c47e4e28b1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1bdcb403fd6598a943fed5c47e4e28b1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:32.759050 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 86, file segment info: File segment: [0, 33554431], key: 1bdcb403fd6598a943fed5c47e4e28b1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:32.763015 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:33.216600 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 90142 2024.10.22 02:09:33.241233 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_741_87, 202410_742_742_0] 2024.10.22 02:09:33.244053 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_741_87, 202410_742_742_0] 2024.10.22 02:09:33.366277 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_741_87, 202410_742_742_0] 2024.10.22 02:09:33.476602 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:33.480811 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:33.563788 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 12369 2024.10.22 02:09:33.632839 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 15.47 MiB, will set to 1.63 GiB (RSS), difference: 1.19 MiB 2024.10.22 02:09:33.756218 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1371_1371_0 (state Temporary) 2024.10.22 02:09:33.768190 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:33.775543 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:33.808178 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(0dc300830a3ecb4d7f6b73b7fb64393b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0dc300830a3ecb4d7f6b73b7fb64393b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:33.813032 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1371_1371_0 to 202410_1384_1384_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:33.814842 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 136, file segment info: File segment: [0, 33554431], key: 0dc300830a3ecb4d7f6b73b7fb64393b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:33.819996 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:33.871148 [ 1276 ] {} DatabaseCatalog: Execution took 421 ms. 2024.10.22 02:09:33.940483 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_813_813_0 (state Temporary) 2024.10.22 02:09:33.975917 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90142 2024.10.22 02:09:34.093476 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_813_813_0 to 202410_826_826_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:34.155494 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12369 2024.10.22 02:09:34.241029 [ 1260 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_821_198 to 202410_822_822_0 2024.10.22 02:09:34.253383 [ 1260 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:34.275680 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:34.445453 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_822_199} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_821_198 to 202410_822_822_0 into Compact with storage Full 2024.10.22 02:09:34.508098 [ 1267 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:09:34.550500 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(37ddd321c3079d4e97eafe6a414a1bb6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 37ddd321c3079d4e97eafe6a414a1bb6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:34.552900 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 465, file segment info: File segment: [0, 33554431], key: 37ddd321c3079d4e97eafe6a414a1bb6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:34.557056 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:34.596256 [ 1729 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ife/disjuffxourbxrvexuroilyzzcxou, size 136 2024.10.22 02:09:34.606351 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_778_147/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:09:34.611716 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_822_199} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:34.625468 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_822_199} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_821_198, total 12401 rows starting from the beginning of the part 2024.10.22 02:09:34.696388 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_822_199} MergeTreeSequentialSource: Reading 2 marks from part 202410_822_822_0, total 12 rows starting from the beginning of the part 2024.10.22 02:09:34.837649 [ 1267 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:09:34.843505 [ 1267 ] {} DNSCacheUpdater: Execution took 335 ms. 2024.10.22 02:09:34.972713 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 113 entries to flush up to offset 89131 2024.10.22 02:09:35.169573 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:35.277710 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:35.331741 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_164_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:09:35.456582 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 676 2024.10.22 02:09:35.460050 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_799_799_0 (state Temporary) 2024.10.22 02:09:35.501528 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column PageCharset 2024.10.22 02:09:35.521763 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column PageCharset 2024.10.22 02:09:35.643636 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column PageCharset 2024.10.22 02:09:35.645304 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_799_799_0 to 202410_805_805_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:35.664383 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column PageCharset 2024.10.22 02:09:35.686262 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column PageCharset 2024.10.22 02:09:35.704702 [ 1236 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 255 ms. 2024.10.22 02:09:35.705370 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_822_199/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:09:35.708428 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column PageCharset 2024.10.22 02:09:35.714660 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 89131 2024.10.22 02:09:35.714749 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 16.20 MiB, will set to 1.64 GiB (RSS), difference: -1.94 MiB 2024.10.22 02:09:35.724753 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column PageCharset 2024.10.22 02:09:35.781577 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xgp/bxmyqebhhxeihzonuxunrgcptzzov (LogSeriesLimiter: on interval from 2024-10-22 02:09:13 to 2024-10-22 02:09:35 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:09:35.783974 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:35.792248 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:36.041129 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_658_658_0 (state Temporary) 2024.10.22 02:09:36.133094 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_658_658_0 to 202410_668_668_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:36.277441 [ 1261 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_663_135 to 202410_668_668_0 2024.10.22 02:09:36.304512 [ 1261 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:36.311548 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:36.353796 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_663_135 to 202410_668_668_0 into Compact with storage Full 2024.10.22 02:09:36.505311 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:36.510626 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 676 2024.10.22 02:09:36.525697 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_663_135, total 707 rows starting from the beginning of the part 2024.10.22 02:09:36.546717 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_822_199} MergeTask::MergeProjectionsStage: Merge sorted 12413 rows, containing 12 columns (12 merged, 0 gathered) in 2.252650509 sec., 5510.397618452761 rows/sec., 879.38 KiB/sec. 2024.10.22 02:09:36.615162 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_664_664_0, total 1 rows starting from the beginning of the part 2024.10.22 02:09:36.698912 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_822_199} MergedBlockOutputStream: filled checksums 202410_1_822_199 (state Temporary) 2024.10.22 02:09:36.871331 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_665_665_0, total 1 rows starting from the beginning of the part 2024.10.22 02:09:37.034889 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_666_666_0, total 1 rows starting from the beginning of the part 2024.10.22 02:09:37.200526 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_667_667_0, total 1 rows starting from the beginning of the part 2024.10.22 02:09:37.265536 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_668_668_0, total 1 rows starting from the beginning of the part 2024.10.22 02:09:37.471829 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_762_163, 202410_763_763_0, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0, 202410_767_767_0] 2024.10.22 02:09:37.471977 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_822_199} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_822_199 to 202410_1_822_199 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:37.475837 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_762_163, 202410_763_763_0, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0, 202410_767_767_0] 2024.10.22 02:09:37.480683 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_822_199} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_821_198, 202410_822_822_0] -> 202410_1_822_199 2024.10.22 02:09:37.532201 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:09:37.557360 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_762_163, 202410_763_763_0, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0, 202410_767_767_0] 2024.10.22 02:09:37.574188 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(38cabb814b99b87a2fd8a70947666a3b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 38cabb814b99b87a2fd8a70947666a3b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:37.600600 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 531, file segment info: File segment: [0, 33554431], key: 38cabb814b99b87a2fd8a70947666a3b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:37.612153 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:37.664455 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 19.52 MiB, will set to 1.64 GiB (RSS), difference: 2.35 MiB 2024.10.22 02:09:37.988807 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 90224 2024.10.22 02:09:38.311459 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:38.316787 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.21 GiB. 2024.10.22 02:09:38.392122 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1372_1372_0 (state Temporary) 2024.10.22 02:09:38.415879 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1201 entries to flush up to offset 1173189 2024.10.22 02:09:38.459243 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1372_1372_0 to 202410_1385_1385_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:38.520538 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90224 2024.10.22 02:09:38.595652 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 46 entries to flush up to offset 6597 2024.10.22 02:09:38.632900 [ 1152 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 02:09:38.833425 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(7642fe22b6cd460164f2cfbd6a9102c6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7642fe22b6cd460164f2cfbd6a9102c6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:38.844555 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 268037, file segment info: File segment: [0, 33554431], key: 7642fe22b6cd460164f2cfbd6a9102c6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:38.848637 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:39.392480 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:39.399340 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:39.508911 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_779_779_0 (state Temporary) 2024.10.22 02:09:39.576684 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_779_779_0 to 202410_786_786_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:39.923925 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1173189 2024.10.22 02:09:39.950761 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(0cc095798131343869a0ca0a8fe068f3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0cc095798131343869a0ca0a8fe068f3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:39.975530 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 750, file segment info: File segment: [0, 33554431], key: 0cc095798131343869a0ca0a8fe068f3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1390, kind: Regular, unbound: 0 2024.10.22 02:09:39.979249 [ 1390 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:09:40.152290 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} MergeTask::MergeProjectionsStage: Merge sorted 712 rows, containing 32 columns (32 merged, 0 gathered) in 3.807287431 sec., 187.00978397446346 rows/sec., 16.93 KiB/sec. 2024.10.22 02:09:40.175467 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} MergedBlockOutputStream: filled checksums 202410_1_668_136 (state Temporary) 2024.10.22 02:09:40.231589 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column IslandID 2024.10.22 02:09:40.245657 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column IslandID 2024.10.22 02:09:40.272590 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column IslandID 2024.10.22 02:09:40.292887 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column IslandID 2024.10.22 02:09:40.305206 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column IslandID 2024.10.22 02:09:40.333936 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column IslandID 2024.10.22 02:09:40.350352 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column IslandID 2024.10.22 02:09:40.363407 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column IslandID 2024.10.22 02:09:40.410454 [ 1397 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/syd/jgsfdgcknijtquhlfqwifrzmhqfwu (LogSeriesLimiter: on interval from 2024-10-22 02:09:35 to 2024-10-22 02:09:40 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:09:40.796271 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 2.00 GiB, free memory in arenas 19.98 MiB, will set to 1.65 GiB (RSS), difference: -6.71 MiB 2024.10.22 02:09:41.066432 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_668_136 to 202410_1_668_136 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:41.076619 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_668_136} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_663_135, 202410_668_668_0] -> 202410_1_668_136 2024.10.22 02:09:41.138308 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:09:41.244129 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:09:41.247353 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:09:41.660659 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 12385 2024.10.22 02:09:41.765081 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:41.767815 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:41.804692 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_814_814_0 (state Temporary) 2024.10.22 02:09:41.958329 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_814_814_0 to 202410_827_827_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:41.984749 [ 1181 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_822_199 to 202410_823_823_0 2024.10.22 02:09:42.016640 [ 1181 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:42.019825 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:42.054710 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_823_200} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_822_199 to 202410_823_823_0 into Compact with storage Full 2024.10.22 02:09:42.191054 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_742_88, 202410_743_743_0] 2024.10.22 02:09:42.195371 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_742_88, 202410_743_743_0] 2024.10.22 02:09:42.322402 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_742_88, 202410_743_743_0] 2024.10.22 02:09:42.343499 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_823_200} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:42.355161 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_823_200} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_822_199, total 12413 rows starting from the beginning of the part 2024.10.22 02:09:42.389299 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_823_200} MergeTreeSequentialSource: Reading 2 marks from part 202410_823_823_0, total 15 rows starting from the beginning of the part 2024.10.22 02:09:42.428794 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12385 2024.10.22 02:09:42.648056 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 90277 2024.10.22 02:09:43.064905 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_778_147} MergeTask::MergeProjectionsStage: Merge sorted 1175225 rows, containing 4 columns (4 merged, 0 gathered) in 36.913473281 sec., 31837.29125280954 rows/sec., 477.51 KiB/sec. 2024.10.22 02:09:43.270902 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:43.274996 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:43.339627 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1373_1373_0 (state Temporary) 2024.10.22 02:09:43.342254 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_778_147} MergedBlockOutputStream: filled checksums 202410_1_778_147 (state Temporary) 2024.10.22 02:09:43.347702 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 153 entries to flush up to offset 89284 2024.10.22 02:09:43.388754 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1373_1373_0 to 202410_1386_1386_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:43.453622 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90277 2024.10.22 02:09:43.512034 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:43.538823 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:43.580519 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_800_800_0 (state Temporary) 2024.10.22 02:09:43.763359 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_823_200} MergeTask::MergeProjectionsStage: Merge sorted 12428 rows, containing 12 columns (12 merged, 0 gathered) in 1.713538209 sec., 7252.829224772776 rows/sec., 1.13 MiB/sec. 2024.10.22 02:09:43.850907 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_800_800_0 to 202410_806_806_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:43.859668 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_823_200} MergedBlockOutputStream: filled checksums 202410_1_823_200 (state Temporary) 2024.10.22 02:09:43.901244 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 89284 2024.10.22 02:09:44.012910 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 677 2024.10.22 02:09:44.106590 [ 1202 ] {} DatabaseCatalog: Execution took 235 ms. 2024.10.22 02:09:44.246275 [ 1189 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 02:09:44.325753 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_725_725_0, 202410_726_726_0, 202410_727_727_0, 202410_728_728_0, 202410_729_729_0, 202410_730_730_0] 2024.10.22 02:09:44.335620 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_725_725_0, 202410_726_726_0, 202410_727_727_0, 202410_728_728_0, 202410_729_729_0, 202410_730_730_0] 2024.10.22 02:09:44.364092 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_725_725_0, 202410_726_726_0, 202410_727_727_0, 202410_728_728_0, 202410_729_729_0, 202410_730_730_0] 2024.10.22 02:09:44.453242 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:44.455572 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.22 GiB. 2024.10.22 02:09:44.470462 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_823_200} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_823_200 to 202410_1_823_200 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:44.475323 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_823_200} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_822_199, 202410_823_823_0] -> 202410_1_823_200 2024.10.22 02:09:44.522518 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:09:44.630851 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_659_659_0 (state Temporary) 2024.10.22 02:09:44.828338 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_659_659_0 to 202410_669_669_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:45.032020 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 677 2024.10.22 02:09:45.359827 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_778_147/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:09:45.652715 [ 1231 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 02:09:46.463931 [ 1419 ] {} S3ObjectStorage: Objects with paths [common/riq/fgltoduxyuzftijarsfreerrsbnay, common/zel/bgyzroncfzszmesnrpcqjczpgytri, common/mty/bxcwirmxsucngaguofslwquepgmnb, common/uhn/sxyirgeekjzhdzmlqbdkzladobogx, common/ujk/ffzeptmlratzoxdchlwfltbiadtct, common/bzq/cvzkhrrepihmblatvfkhrttzlxyrm, common/mqs/yczxcxqoidwtdkymdaqjakzobtyry, common/fcc/dddklxpujqrjlyccyfqgcamtolfkf, common/epc/hhglludkexnigcyehuktxihzpgrrt, common/oev/ctsmdmstmndbcgznhbpavcdwjggtk, common/oym/ywovlemzjayfeleubpsvyscmmyxtq, common/yjz/zblitvgdvkpjcewgjorvvfivfvpsh, common/wdb/zmodnoyeekihybrwrzjvicwgwsspt, common/thv/pzybfcrilijsdlazecswmbbevozij, common/xsp/hmzynrzxeubcfgwwrshtiqducgcbr, common/gte/crdsstpisesffsgqtgdgnthdsdxzm, common/zom/pnvhxeliighzskpmiwljpnlqxyzty, common/uhh/lgoyhkpxnnvuffxkyryutphghkgwd, common/fbi/nykcowdtudrvcyvdfapijdfoysjyu, common/lrf/ifbnuauaxmhsmepmyweryeycdyhwa, common/aom/gctytxfiriitetctictluevjogwhj, common/ntp/uqkdfpguliezjvriysemmghcemmie, common/spy/hzrjysihjlvjgdoodstdscsqexeva, common/lxv/kiitvpadmognbhuudaifsdvlwibbv, common/ecn/ppcrnoqxemtsqohmhowjqvbnqwzoe, common/cju/lctezpardqgipcejnvdlbkodyxreh, common/lah/cgriwyoohokbcbugkljdbvstizxav, common/qev/xozbfstwgbdoftzwmdpchrqdidejx, common/ssr/mhroseupivignndsenounlpvautef, common/hvg/uelaglofkmamgqtvurjxeccyionxb, common/kic/lcwqcamtefiiflzbfwdrsjjpalywy, common/qhd/szecmkhnpflxuneyhxydqtctgfmaj, common/tgk/lehvdbewknafjdulbalfzhaxingcd, common/bug/qmpalijxgxpioorccwszjuziwmhoa, common/ntw/ibdsbxdwyweszouvqpczixrtddlaw, common/bbf/yjfllmwkfoeadrnpwinjflgjqgmjr, common/sad/ajgakghuukzmiminqxubgwzydlhgh, common/xtf/ldoqchljzdughgeavlbbdtxhbbllu, common/ouq/djkagmpqbjdcyzimfyklhblxpusfx, common/nto/jhoxtwlgfmbdbbxvalumlnmcphdcy, common/dqd/kvfhwbxdqhllikavlqjnnkrkfychw, common/woy/xxmdhpcdyeucvyscuixgstoeqlsrw, common/yad/oghwncqwvdxcjasrabiouxewfinjv, common/cax/acxfocorikyxpqekhqmzlmrygppma, common/efm/kyybehgqnssprgsndhtumtzzpbdzq, common/kbf/kuwcovshvwaanzctnwjgehjjkfgst, common/yqi/lpuotrvgueziijjztfchtwmewgvwk, common/dks/crzrmgucczcetzvetukagvytfkejj, common/bbi/mkyxgidwvpdsyayyfmfukzqsejaez, common/cbu/zludbzriqnculessiuqmwamvodtso, common/mmq/eayizjvwkomvojxhjpvocnqngblyr, common/zuf/xfbfkelsijbvtvffhljsvopbovfzw, common/uum/buvvnrtypiocjupyegkhpxhmgrjmz, common/zeo/jyzhbwwhqtzmuxghuaylaepudzxsz, common/xlr/prwgfwbguxhovqzdrkivvbcvkwrom, common/pia/dkchkosnapajxvmjpzszvtnxzkaxg, common/hol/rtybzbqnnumvzldosxbevmrgomnzp, common/ibp/uuxaxqknvvcqmmhuscrryfexefrkt, common/bxg/whtjvgmazzhrnhvtybxuxckcgubjf, common/iuc/gngrbnvvxqytwwzzhkrixmwwllstj, common/toi/asjqiyykssimyeerjrhiluohxdchd, common/xes/cxpesssrebijgipsltxzgtoqvjuat, common/iti/gbpfqcntzrunajcqzzfaecsaeswzr, common/gvo/fkqjvmfpmobieezsaxvefezxvxwvh, common/uuj/chlnkdmkiievtsajaikkwdyjdkplt, common/kge/eahjktjpvvgdoockuycwqeusnmimg, common/pcb/jquydclxnxsbesbnafgilnkvshegj, common/afs/nfvkxzybjibchwhtlxsqhgcfddxrt, common/tov/btodzryegnvsaalebohpphvpgbubr, common/pje/vceumghggjbnpbedpwqdttxntsunu, common/fwp/ktnuhexojuwettudhtjcvmrfknuar, common/qfe/stqbrkzqipamduhmtgxjonganhfdo, common/hvu/qghfhhahyzhbvmzdvghurtufiplaf, common/kir/krikgahkbhnannbwhljplqoljcghz, common/cfm/hrbyujmidnorchhdlzkqnmhhfgxlm, common/cox/brkxhaqzmslhtuwexahsaoexeiewe, common/eok/qlmxuylkxhriklhlohytqcfnawxfb, common/ngt/clvmizretkkcxzrsfdpzlljecuwpw, common/yua/oflpjenvqyzllyefxlrorutyvppni, common/qan/dvmdhohdjbmrjxtpayvjfwavmddzt, common/dfr/vccrppfmkhozvvtfhmjchhnwajmga, common/jmu/gifjxvhrahjfromeulukqdezfapli, common/xbw/nkxkttxqhnratwcussuefytlnnvtv, common/gco/nzjxozakombytgysvlfzhqrgbmlcp, common/pvc/odmswrfjprwxwhviinozcpbvrrxpc, common/brx/rwaqvsxnbnrhhjlgvgxhekcelwixd, common/pwj/iwhdtqwvoiutwojjbwqnbwrdwacky, common/zdw/coqjvcdrssdypckzgxlvdbmhowkml, common/tpk/nubdbhssuhlnwzxzdgvsdyzrnzjnb, common/mub/hrdqkygdzuzsdpelozgbtwmvjhszc, common/wdi/xioltphkkfapljobdbbjufoowcxxr, common/jgs/rrogqjkkwbvqorgtxqoqxwwkzutli, common/vvf/noocopgbgiiwqbanhgfbakqirralu, common/fig/qubnepskyxzirvrdcuuxxbidsgxkp, common/bxj/xkjmmjjhosbbcvivvgbfasirvylgx, common/wqx/noossdmhxpkwwmxbpqjignlnbusib, common/dtv/cbtzemiylsteocwqpugssrxeucjyg, common/kxw/gmxmslgdxxcmtfivyaiutmtvlwaxk, common/fxn/xdfkkzvgpdwkqdpkctbgowashlczn, common/rtb/sxbbbozwnmqjkrwpkchnwwxyvkrnn, common/qrt/nvcfcegcegwrigjwpyqnujnomwtfm, common/gib/dponywpletjutzbjuwtcsmmmfzyes, common/iyh/imxpxzjcfrmxrxvbakfifqwmdmfby, common/mdy/dolqufholbgbbmjliqxdwfngtbvrv, common/mds/jqeufxureowusosayjfuqcupzoinj, common/kek/rtltzwnbwzfkevjvhzxwltseidsry, common/dfk/gwgcpwdyejbgiyndjtlnuznhzjslz, common/nrn/mlcbskvrdpynjxmistqpbuljuujkn, common/xal/imlmiqonmsifizulviknmfrbrhfxe, common/hwy/klteuizsmdywhsjbkkvebikdcfleg, common/lmp/dmptolqepxzyqfgxrjgbdpbatteco, common/xjy/gaxfyrjquofkliukjvsnforcmyrqa, common/taq/edveaygmtykrbtqqujmzbrkjfzlpw, common/cbg/jnzlqhcardmwssyswokzieugywcfl, common/olt/rwwkmrlqmwnpqebldtzkukufmyvyq, common/snk/kprspnzjojwzwuepdfjxfiinvfsiq, common/qsz/rmocpxopvfrgxgcqandlxrsgqgyjq, common/mao/ziktrcujnbdsxzyfughixbohovmhr, common/neo/tfisjszbikkckrphewevpzcgbqsxk, common/pog/cuegtevkcwbglfyladvnkcowpovmq, common/cqa/cycwgpakniokwxdstyhgxqykiaigd, common/qwr/twkoocpnwtaxufcibmuaqhqwmulxz, common/jci/wsxfsbtjwyozqbvsnqkckaidatffc, common/oqw/pycvwkvpueaokwkdfvdihmbwdwycp, common/ipx/kqwmvddgfgxkqqzgkeqkpwuwnvjfg, common/fol/aogbwnbwrffabhnuzfizumvlaxjwe, common/hcz/odvxyidizkilzlnbvymtqereuhmdx, common/pem/sjgusdgmgxrniextwtbyhglbueoad, common/nqh/mqkpprcznxtpvuiccbxxgejrpqbra, common/ahp/qiqkjowdljzwvzoxpcjtrdqzrdqfz, common/nii/hpknwxezwviqzpvgjxcvtdvnwsvnt, common/zsk/ypisaipcyrpllfyzdutllzkeizeaz, common/iny/asqyjxgqfmwamahsfltctesbkkbef, common/xkq/ialwoywcspqfucdvbgkwtgvvctczo, common/phv/oogmzlttrcmvnuzlsydlhcwjrufze, common/uzj/zivcgcquuzdcrntdmcictostnvrem, common/lhc/vpybhoeislcbtysvrxslbmqfofigt, common/wbj/rforammwzgvllbbftfixabifqhfwl, common/hsq/pnggthqwiekgxemlowbpktydodtep, common/gkw/yzvsiezzlbilgssmtyhrkuxocqsum, common/pfx/ojakpqzapikrkyowjjtinnylheuue, common/rqd/sitiygwjloixlhuuuxgvclswfguof, common/vab/odebguyafmudcyzdopljrvnzruyhy, common/agq/pfwmnimdffqmmynqgadygrqterdec, common/ujg/gwxxavbryudmeywdwqezgehrvekph, common/upb/tjhuetwmfbeqrhsdzrrscdzywoakf, common/yoy/pbqjzeeaontfqawoictyiicbivmpc, common/tht/sbbxlsffavvbwjyusmjtchdupkxpk, common/tcb/cknbufmthwxgwfgyunvweoelpeteq, common/qzx/emhtelarkzclmvvwqiseociifqvrx, common/xsj/krwhdixjfieklwugdgfpgwlhqsheg, common/wrb/fhsiobgdqqlyjrbwijgxucqntxems, common/sbq/erkpjcovashmtraxjzcqmgyqqqwzu, common/hfy/qpufsxwluaioehtkrdebkfatqtyzt, common/nch/yoctfhdvtmckmtbyjrzpolgfvrftm, common/zge/owsnqvsagegsiwytkshxbcgcmwfiq, common/afg/pmwynhfjcjiymttgszqpiwzrjasmk, common/bzq/gyeogwruwkpykevvhjtfcczlyzsau, common/scp/mopmlufxlsustkryxtadfnoeocfss, common/dds/zuagdezfjldqcijhlqbjhifdwgzfr, common/wsx/vojmirwqaopvdehpbsununflktsnk, common/pbm/ubrafhoaxihazkubsfiiqcxnnssey, common/tgw/drodmrgmuccduextshiiguyfdbshf, common/ixf/xevgqpdarsloigxxzloebozevbpqh, common/sfw/ccqnxlfdwggrrsqtaukkrcyropjnd, common/nrf/iilcbzgeigkzdhmwgvywouncfqbil, common/mmk/xxxxdikugligltekpxmaeedgxfdqe, common/ado/einqnztjvqymbrollarvnlcmwbxgt, common/rqh/fqbefyvqtbaxrebkmvfjmcrnxwmmw, common/ons/bnybmwscabfhhhhshpwhgmeahfobc, common/qgg/yjgdufnqksagvggrcieudorgalmed, common/snl/tuzknulpzmtggxkzwcpopaejwfihm, common/jac/tuguycqznizcrnweqiesxxkzxrsor, common/ncj/yamexstvwpokhcaugkrlpoqmbtvom, common/eyr/grhaisfgfuteosvdvhhauuilvorzx, common/ndg/qhshxodlxejygokdpudbknrlsiguw, common/xgj/ewrvvtvbblknuuwfnqlhxkfwfkpof, common/nee/xfkbnscpfjfiwyojjfdbljkpmscma, common/qns/miptwjdopqtynrkpiuyxezxnfzltt, common/vyy/rqrjltkqonfspzmvjoewhrazmpdyb, common/jzg/xvzubvvgjncuezzczhhwttxmewkhq, common/tds/pksjfqmdqmnzohxzjrizcuagyovqh, common/oer/cdvaobrnybiqwkqmcdpvcgyckgbqp, common/bxh/munyvqgjnhjdfyhbuyxrrllvawwhc, common/idy/oogvzxhyixoobekzaqpnixasoikvx, common/vzu/xzlevkvlchfztuogedbzpxibxrpui, common/pqm/hmrqeqcsmjhutvjynsvbwzqqgbhfn, common/mph/bnvqfnzlmpzaakoqzlmulnwergvrt, common/awf/mbmwmpevwreltchcgvfsmvlwpqiyc, common/pcg/gtqewprljuitwizwtfpuvieoqhpci, common/zqv/wjecfqmeocmaitxjvpabjjscnsoll, common/wzo/naskptrdbullfirykqsjhgdlnwxek, common/xji/wioktetwmpplvpbthxwbrjgzznrro, common/nka/pbabsxxjplcdfycmqgninspmzalts, common/gxi/sfolemounhwlbhkkkehxthimwvheh, common/chy/wkvdbjgcxyuohjmqrdrbxilpynbvh, common/osg/vqwqewnondmpxtptkrazkjzyxzzmu, common/hgm/rtufhjjudsootlnxjdyurobqurkbh, common/xwo/kvxjdifoolpwckyqovkbdrvnhroml, common/ino/wfhyztfalulkjqbcifgcmbhwhejzs, common/frw/wterqnyhlpzaocrxdspaxjippxrmb, common/bex/iukunxlpregjigehmdriugmgkpnei, common/lgp/srposrrjcgvcuovhnbnfkahclbcdp, common/mkq/ubhhrymqaurgglvoqsfdmimxibkdf, common/yry/guytlgucpezqlrikbhjxtxhkxpuol, common/rlx/msclasastnacaytydospknjmefarb, common/ajh/hbhkbnbzgifnpclqeuwjjonhlewkd, common/gno/dbigujsoiqhabuelqqatyijvncqux, common/fbp/lcslgoyprvtiwgscovgohfgxtvweg, common/xjc/wfaknrlhrfquwgpsgkqmhjtadrhtp, common/gml/kewnfmncrmnuubqddkqyglogjnvlw, common/sxh/yvhctllyjaausicrxgigazbpfgwgz, common/wsl/shgdyyxrhddrejsieflrnkznppeyd, common/seq/pejfbmrtoypilecgspmruvjysiijd, common/bsm/kqsukfjoxyxzxdvbruvxeecqmmamz, common/pkc/jmjhyvrunkozhechyjoeiaauoncma, common/gvp/resbuzsuhlbjtpjqcqqibynnbnwaf, common/zes/gcyxhpcabosirvturlpzkntehszuj, common/nzo/drtuntjrlbiefbdttzvlyrowmxgge, common/eqw/uxdslikyxagsohmliuzvdhbhvndip, common/xae/rjehxcjysydrbaxjlzinttxideelg, common/vcx/qkcybqxvgfqrxzcrgsiaqadlyizlz, common/jnx/ghtstreuxzprhqdvihkegxrrksdam, common/nrc/znqzihkyfzzwqapyrgdyfqpygomxj, common/bry/vkgawzoovsjrghynuucyzjkbnjarp, common/epd/idffxdogwvmsjllwbrvaabbggagus, common/znz/aksgwblwvibwxfoidldknrobkfbpr, common/vys/lyzrfzszcnbrrbzifwjaoblrhwnqu, common/rvy/clzhqkcuayutqveffrnrlshiysmeo, common/fmx/kypgbpoqcuieyqyakwjzkxxttbmyr, common/tdc/luqtvgfxamniehwaagggilnyywcex, common/kdp/mtnrckezowlfbdnsuivpwcazyvrsu, common/oxl/mwqevahckkmvlcgdrvywvvkurmcfd, common/ikm/ooyqrqpvafkflyhduahwmbnjjqjgk, common/pfa/vpbnxaqsfgvosfrzmqoggxqgvbmfj, common/fjx/dejllixkhoejxgnqjrkmnoxgurzsh, common/omd/yoihuwpuoimeohnxhseuxzylvrygq, common/cnp/ipjdufdkbapfwoxoscyjirhuzaaqc, common/hze/jpjlehtzmwoekvozyesinscoyeqgy, common/nid/qyupvbobyxnncvhxqpwdhbxmrnsqo, common/oxs/hchxplpnmcgnaajjwaktcpxnrnada, common/dae/cxzsogzkkxbbvatqsmhzmmrzczqfo, common/fyc/tplqbmkhpimdwpolqpbgsewzitcgn, common/fmm/nqjplmpwqsmgunmpphaetcsrakznl, common/spa/sipcuuboefyldpqzjcofjassqqnyi, common/nrs/mhkhpxuxfjhtjsctuchcfbdtgtrkz, common/ocd/pillrfayqtqryeqacuxciqakycymz, common/nuy/znieqjbzrabwyrrhsobyubwxpveeu, common/lwg/fuoiauhwzccvigkrgpoegrvcjcmfk, common/gti/qbfwqugroreuuqfovtxonzhdrfwuj, common/ehv/rvbdwwwwiljoklbwuoafrbpsgpgii, common/oba/gmwwfgcrgngjpaejzkuytykimbmcz, common/dwo/lkzbjidpnjccjxphhfighyzdsnvje, common/zjc/bmmcqumgbdqboewisdrdtnlanmmnr, common/gdw/kufbvfatdxmjudfctdcdeauduahpw, common/voa/yvvjgyecduswtswituwodsvbpuuik, common/nrs/urnhxdlylhoghuctyhgrczuuoynfh, common/bpc/agznscxzdapayzeomispfursilaug, common/sts/irzzjwminjzoyjounlgzvvccnaugy, common/tvo/wglcprodoqeavuqlapeemtefzzkfz, common/dxv/zbfxwqvfcmdnqkgqfrmvfunwswljg, common/zof/clxzlmevlmvsqevicaftzreohhlcb, common/zme/lhiatyogzvoxebxlweowkurskpqku, common/odx/mjhswucxqamklxjosymrfvwgwupok, common/zzi/pfufesbfsdhpwuuivvotibnedmyam, common/zcy/cviajvzzeuobdhcerhbwoapdfdgez, common/tkm/ozilltvnadtiovhosmtcxwkdzmaxx, common/fpa/obhdhyuhdqtxjsfjpsntbmbkkznhl, common/vwz/nymwtzjboinzddpwmktsclktxntir, common/gth/xwdskkqhjjmxszbxwrthursiritfw, common/xkl/kcasftrqisyznmeixcpktordgqmfe, common/cwv/dddbwrdouzrbzaiyubuumikasrtit, common/sxr/mezedgzcbwnkamojfbuxtakiycuxh, common/nex/cuffnloyngjqorknigoumdfgpmqyp, common/mkt/hbtkywbyokfbftdatlduraxvoetsb, common/vog/gfselrmtesxjppkqlfhsclnjlgszl, common/dmx/uduusfhovxczgwmsxmjwbpvcttnzo, common/lvw/fzshkgtovyqisagiqddcnpeyyfkjt, common/mrf/ptacnuwyiexfexyodrbmyzgfigxju, common/uwd/hkofoattzhbzhhoyyjaiijhzkahml, common/dsz/anslkakstbdzgdjdmnfutgahldxml, common/kms/assgiczlfxbnthwbgmxxyetyggpzz, common/goj/fciiejrjskpkoelfwjvnulzjkwmlv, common/msz/kogegmeymrzsogukzwsqzmxzgxtmo, common/hyo/fyppxdwiqeyythcnlctpstivraxom, common/zys/rcybhmcslishmptthoxkmfsmmchqe, common/ylp/ptsuludzmlprfnkuliptkksvsnszl, common/sfr/oxckphuqziblsygutrrskgtotowyi, common/ojo/ctkngmgthwhqnqtfzbaqhfpzvthal, common/tfa/carhfmjfeuxiaaahphofhaikijokb, common/hcq/mpvdjeoijrmbmsfyqfgyhiivjewte, common/evp/ceecjykxcrdixsvrmniksccalqdri, common/bbi/agrhyvjbnnutijxonurghjvwsvzbi, common/iqa/qkubkzzlkoajcwkixugqambvfweau, common/pwa/jqrtrrhobhpyvsbdwjxvsbjbbsvsx, common/ufg/hfgyqmxbaheblqwndbmgtmfzaoswi, common/huu/nminoywyqoazbrfqlffzryuomtngz, common/bef/xnijcogmdcgelufrfttvxhbmsgeub, common/zbo/uqxupmqzerzcgmluzcoyhhhwoxzyk, common/qkv/ywmrhozhowztjhxfhuknquepufivv, common/cev/lksxpztlekvyeverixcalsbtogvgg, common/wbp/mxhldrmdfyrrbwazudyolbfccdouv, common/wlf/qvkbtlrmgjzgtmuisewgvwqrysras, common/czl/lkqnahccmahznuphwjwmlwicnbrhg, common/nvb/zolndvhpdtjauzwngipwetqvqhrzz, common/tkh/gdaunhetmkbnxfdjexussxqwkrilq, common/riy/nahbwnbpkgsabzqdlqncxixtvmgat, common/rpp/zosinezvfaiaourovwnmbxunboioi, common/aey/vuunnfuzkujccpvtwqogtdzqhmdtc, common/igt/fbsbrayonbgeewxeeyugzymjmryvc, common/jzo/gpqxqtjiezddpaolbhjtyyenffosr, common/suf/tlezuiuokdfeqyzlfaaiddmjodfic, common/kkl/pnwkqrkpkbzvgeljlaewyofbhctlm, common/hff/ymmezdguiztpuqwelqrktntzxvvvv, common/aho/icmmnzjeporudcpnhzkmmzfgxmizq, common/tlt/ijpsthyedzunwhrldeougsddxolbz, common/xsn/uakhykvuxvpmzzpctutuxemajfigt, common/iza/fthnsuwllrrextuihualsqlnjlbuo, common/hll/jtrddqoxlcnmsghehovwnjghwzuko, common/rjg/vwancjwguudecatjgumhscdjwwyfq, common/mvm/kqnraszqovwkufnwmzutbmxdbinzr, common/exp/ydykxwwuzjcegsnhkllvykfaekabi, common/cgf/tbqbqtuwlqbfdbgeacsgahcjtrjcs, common/ysh/clluydbrcwcsyhfugcelftnxboxrf, common/qif/aullqjcwntzzvkiohbmouzqpepwwo, common/cvy/dgtimmmljrrqlqnrnziurisydyrzw, common/rbm/ruwdgelfsxobbbdszjptwvgjrrmal, common/urn/fcevqnknlbhzxfontwabovuxaxtbb, common/isu/atowxpptjphwylxaptdmebrtezzdz, common/sbc/ybjfablawunojxvdpnslszniosduj, common/nnh/yebqpycnzgbsdsvpeqducremmwiul, common/sls/ldwjdphqwquqnodhztglurpbacifv, common/bkh/edozbfnpwfzcfphmhnjaeicwfwcqb, common/lvr/msznbmbkkryboexsmvxkeqpsragsn, common/kfi/gqazagqmqeyujuhsrntkjuipphxtn, common/qsm/olmbkttqyrzhyqqazbsaogthqbhxc, common/hnn/zaqfelyaysdgkdnsmxcesfalvluxk, common/zlk/ddqxtnwxztcotawqqtuizqojdcfri, common/nst/kywvqwmflatvqxsrdiywdiznamoan, common/ojf/tgcnkhwpxhbenrnfpyggoymbnuavu, common/ysa/bspnkcfvjrbgxtjcdugewtvzimmvo, common/cpy/tmyqittmrechjejqnjgjxlnpuiggc, common/ife/uqvnhbiisxqwhpftdkzeyffjxopvv, common/htl/vqkfdjrckythlptakabdesndkbvdk, common/dvl/voldvdlzeizxmsugfajzmhaqszorh, common/ipn/zismvcjsbrspmhnllxlvtawmrmrwo, common/ufy/mdjtjsehjgjafronubcgqppvjqhfm, common/ljq/qvsgjtvxdkpiqtejmfjpgyoexgbqj, common/iku/hxrpwjfztvqabafypklorkkqackvu, common/lmo/vrgowkyhmychxuxpqikfdtnoetjck, common/dvx/iyhkleadozwovbvkitlzqoslhjqft, common/rog/qcaubjxoidnzvdmyckkgzzblbskzd, common/rzj/mbixvvpdeevytnvoicyadgadtlqol, common/wyp/tmgqxkbxdfavzifnzazmuykkgbrqn, common/pxi/qzgjpphyhywpdtviqrajxklimnjlc, common/rll/sddnvrbojsrsmpitqcagnuiqkguex, common/pfn/jzvsikphinfdwdbvzembwuwnsqddv, common/crk/ijyqbozfrkpompfysqlxifkabynvc, common/jjt/onojpxhmshzvawcximokyhvgmyssi, common/qby/opxniktacnjyujgclkhkyelnsdjfk, common/kax/bemkewnbxiujmvtfhxzkxvflfouxi] were removed from S3 2024.10.22 02:09:46.673764 [ 1419 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsNotBounce.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/IsNotBounce.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_9_9_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:09:46.810239 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_778_147} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_778_147 to 202410_1_778_147 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:46.814437 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_778_147} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_772_146, 202410_773_778_1] -> 202410_1_778_147 2024.10.22 02:09:46.814660 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 2.00 GiB, free memory in arenas 9.73 MiB, will set to 1.65 GiB (RSS), difference: -2.41 MiB 2024.10.22 02:09:46.859370 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:09:46.931349 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1174729 2024.10.22 02:09:47.463877 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 90316 2024.10.22 02:09:47.711276 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:47.716750 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:47.881280 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_780_780_0 (state Temporary) 2024.10.22 02:09:47.968090 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_780_780_0 to 202410_787_787_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:47.982793 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:47.989059 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:48.059279 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1374_1374_0 (state Temporary) 2024.10.22 02:09:48.114213 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1374_1374_0 to 202410_1387_1387_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:48.179914 [ 1208 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1382_35 to 202410_1387_1387_0 2024.10.22 02:09:48.205446 [ 1208 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:48.232869 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90316 2024.10.22 02:09:48.250404 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1174729 2024.10.22 02:09:48.257742 [ 1208 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:48.270754 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1382_35 to 202410_1387_1387_0 into Compact with storage Full 2024.10.22 02:09:48.405106 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:48.445347 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1382_35, total 10892 rows starting from the beginning of the part 2024.10.22 02:09:48.535958 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_1383_1383_0, total 79 rows starting from the beginning of the part 2024.10.22 02:09:48.731811 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_1384_1384_0, total 30 rows starting from the beginning of the part 2024.10.22 02:09:48.837833 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_1385_1385_0, total 82 rows starting from the beginning of the part 2024.10.22 02:09:48.954194 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_1386_1386_0, total 53 rows starting from the beginning of the part 2024.10.22 02:09:49.163822 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_1387_1387_0, total 39 rows starting from the beginning of the part 2024.10.22 02:09:49.244275 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_164_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:09:49.273281 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_743_89, 202410_744_744_0] 2024.10.22 02:09:49.276766 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_743_89, 202410_744_744_0] 2024.10.22 02:09:49.449054 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_743_89, 202410_744_744_0] 2024.10.22 02:09:49.766523 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1282_15, 202410_1283_1283_0, 202410_1284_1284_0, 202410_1285_1285_0, 202410_1286_1286_0, 202410_1287_1287_0] 2024.10.22 02:09:49.768139 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1282_15, 202410_1283_1283_0, 202410_1284_1284_0, 202410_1285_1285_0, 202410_1286_1286_0, 202410_1287_1287_0] 2024.10.22 02:09:49.847157 [ 1221 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:09:49.880076 [ 1221 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:09:49.906146 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1282_15, 202410_1283_1283_0, 202410_1284_1284_0, 202410_1285_1285_0, 202410_1286_1286_0, 202410_1287_1287_0] 2024.10.22 02:09:49.932095 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 12397 2024.10.22 02:09:50.066996 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:50.071738 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:50.173113 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_815_815_0 (state Temporary) 2024.10.22 02:09:50.233447 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_815_815_0 to 202410_828_828_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:50.299716 [ 1229 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_823_200 to 202410_824_824_0 2024.10.22 02:09:50.316646 [ 1229 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:50.320812 [ 1229 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:50.350714 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12397 2024.10.22 02:09:50.383228 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_824_201} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_823_200 to 202410_824_824_0 into Compact with storage Full 2024.10.22 02:09:50.509628 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_824_201} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:50.539362 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_824_201} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_823_200, total 12428 rows starting from the beginning of the part 2024.10.22 02:09:50.576528 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_824_201} MergeTreeSequentialSource: Reading 2 marks from part 202410_824_824_0, total 15 rows starting from the beginning of the part 2024.10.22 02:09:50.578717 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 13.04 MiB, will set to 1.66 GiB (RSS), difference: 4.99 MiB 2024.10.22 02:09:51.247240 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:09:51.254523 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:09:51.405548 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1387_36/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:09:51.444153 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 118 entries to flush up to offset 89402 2024.10.22 02:09:51.674752 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:51.676001 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:51.684067 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} MergeTask::MergeProjectionsStage: Merge sorted 11175 rows, containing 13 columns (13 merged, 0 gathered) in 3.418381293 sec., 3269.091140559316 rows/sec., 604.39 KiB/sec. 2024.10.22 02:09:51.724956 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_801_801_0 (state Temporary) 2024.10.22 02:09:51.762130 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} MergedBlockOutputStream: filled checksums 202410_1207_1387_36 (state Temporary) 2024.10.22 02:09:51.876275 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1387_36 to 202410_1207_1387_36 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:51.881401 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1387_36} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1382_35, 202410_1387_1387_0] -> 202410_1207_1387_36 2024.10.22 02:09:51.912091 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_801_801_0 to 202410_807_807_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:52.041045 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.68 MiB. 2024.10.22 02:09:52.045353 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_802_11 to 202410_807_807_0 2024.10.22 02:09:52.054289 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 89402 2024.10.22 02:09:52.054889 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:52.060464 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:52.099028 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_824_201} MergeTask::MergeProjectionsStage: Merge sorted 12443 rows, containing 12 columns (12 merged, 0 gathered) in 1.622629826 sec., 7668.41567967086 rows/sec., 1.20 MiB/sec. 2024.10.22 02:09:52.104772 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_802_11 to 202410_807_807_0 into Compact with storage Full 2024.10.22 02:09:52.151311 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:52.156323 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_824_201} MergedBlockOutputStream: filled checksums 202410_1_824_201 (state Temporary) 2024.10.22 02:09:52.158657 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_802_11, total 5465 rows starting from the beginning of the part 2024.10.22 02:09:52.165574 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_803_803_0, total 106 rows starting from the beginning of the part 2024.10.22 02:09:52.182873 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_804_804_0, total 92 rows starting from the beginning of the part 2024.10.22 02:09:52.243888 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 90385 2024.10.22 02:09:52.292431 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_824_201} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_824_201 to 202410_1_824_201 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:52.298496 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_824_201} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_823_200, 202410_824_824_0] -> 202410_1_824_201 2024.10.22 02:09:52.308624 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_805_805_0, total 113 rows starting from the beginning of the part 2024.10.22 02:09:52.331386 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_806_806_0, total 153 rows starting from the beginning of the part 2024.10.22 02:09:52.355368 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:09:52.360511 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_807_807_0, total 118 rows starting from the beginning of the part 2024.10.22 02:09:52.463533 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 15.98 MiB, will set to 1.64 GiB (RSS), difference: 2.98 MiB 2024.10.22 02:09:52.496498 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:52.498699 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:52.536120 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 678 2024.10.22 02:09:52.658039 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1375_1375_0 (state Temporary) 2024.10.22 02:09:52.700736 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1375_1375_0 to 202410_1388_1388_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:52.792366 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90385 2024.10.22 02:09:52.829151 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:52.832229 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:52.886324 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_660_660_0 (state Temporary) 2024.10.22 02:09:52.972690 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_660_660_0 to 202410_670_670_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:53.158363 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 678 2024.10.22 02:09:53.589527 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} MergeTask::MergeProjectionsStage: Merge sorted 6047 rows, containing 13 columns (13 merged, 0 gathered) in 1.490477347 sec., 4057.089503689049 rows/sec., 1.76 MiB/sec. 2024.10.22 02:09:53.668050 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} MergedBlockOutputStream: filled checksums 202410_747_807_12 (state Temporary) 2024.10.22 02:09:53.832654 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_807_12 to 202410_747_807_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:53.839644 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_807_12} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_802_11, 202410_807_807_0] -> 202410_747_807_12 2024.10.22 02:09:53.872049 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.13 MiB. 2024.10.22 02:09:54.470408 [ 1273 ] {} DatabaseCatalog: Execution took 360 ms. 2024.10.22 02:09:54.580079 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 12.18 MiB, will set to 1.64 GiB (RSS), difference: -3.50 MiB 2024.10.22 02:09:55.327836 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1929 entries to flush up to offset 1176658 2024.10.22 02:09:56.325682 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:56.331863 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:56.454471 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_744_90, 202410_745_745_0] 2024.10.22 02:09:56.465011 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_744_90, 202410_745_745_0] 2024.10.22 02:09:56.481745 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_744_90, 202410_745_745_0] 2024.10.22 02:09:56.560271 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 6.73 MiB, will set to 1.65 GiB (RSS), difference: -3.56 MiB 2024.10.22 02:09:56.572953 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_781_781_0 (state Temporary) 2024.10.22 02:09:56.768274 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_781_781_0 to 202410_788_788_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:56.803130 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 90420 2024.10.22 02:09:56.814698 [ 1242 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_778_147 to 202410_779_784_1 2024.10.22 02:09:56.830797 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1176658 2024.10.22 02:09:56.843130 [ 1242 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:56.850930 [ 1242 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:56.968684 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_784_148} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_778_147 to 202410_779_784_1 into Wide with storage Full 2024.10.22 02:09:57.167108 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_784_148} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:57.176570 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_784_148} MergeTreeSequentialSource: Reading 145 marks from part 202410_1_778_147, total 1175225 rows starting from the beginning of the part 2024.10.22 02:09:57.187884 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:57.195352 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:57.209303 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:57.214487 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:57.227880 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_784_148} MergeTreeSequentialSource: Reading 3 marks from part 202410_779_784_1, total 8844 rows starting from the beginning of the part 2024.10.22 02:09:57.448184 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1376_1376_0 (state Temporary) 2024.10.22 02:09:57.526685 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1376_1376_0 to 202410_1389_1389_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:57.712326 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90420 2024.10.22 02:09:57.853112 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 12420 2024.10.22 02:09:58.184847 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:58.193257 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:58.243463 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_816_816_0 (state Temporary) 2024.10.22 02:09:58.469704 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_816_816_0 to 202410_829_829_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:09:58.585778 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 15.65 MiB, will set to 1.65 GiB (RSS), difference: -4.20 MiB 2024.10.22 02:09:58.743697 [ 1152 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_824_201 to 202410_825_825_0 2024.10.22 02:09:58.751846 [ 1152 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:58.756122 [ 1152 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:09:58.868377 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_825_202} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_824_201 to 202410_825_825_0 into Compact with storage Full 2024.10.22 02:09:58.925388 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_825_202} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:09:58.932114 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12420 2024.10.22 02:09:58.940156 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_825_202} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_824_201, total 12443 rows starting from the beginning of the part 2024.10.22 02:09:58.967468 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_825_202} MergeTreeSequentialSource: Reading 2 marks from part 202410_825_825_0, total 9 rows starting from the beginning of the part 2024.10.22 02:09:59.628364 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 96 entries to flush up to offset 89498 2024.10.22 02:09:59.854495 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:09:59.859821 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:10:00.034269 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_802_802_0 (state Temporary) 2024.10.22 02:10:00.235286 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_802_802_0 to 202410_808_808_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:00.439352 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 89498 2024.10.22 02:10:00.752948 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_825_202} MergeTask::MergeProjectionsStage: Merge sorted 12452 rows, containing 12 columns (12 merged, 0 gathered) in 1.912134857 sec., 6512.09299093908 rows/sec., 1.01 MiB/sec. 2024.10.22 02:10:00.801837 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 20.07 MiB, will set to 1.64 GiB (RSS), difference: 1.83 MiB 2024.10.22 02:10:01.150937 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_825_202} MergedBlockOutputStream: filled checksums 202410_1_825_202 (state Temporary) 2024.10.22 02:10:01.247179 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:10:01.249292 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:10:01.573341 [ 1419 ] {} S3ObjectStorage: Objects with paths [common/nja/vdpjpashofokjiwlfjbfdvdzwajms, common/ayg/slcweprkixfbpdnmamcnyoayjebhy, common/ikc/knagijckjdfizejyezunxngyjpspz, common/ivs/izfiftxciabqwiusbeyifohscfphq, common/lzp/wyymalxgjngvljtulrvlynaobktef, common/mai/deczeawhusnyfnbsxrakphlazciqs, common/bzy/zgkkjinfxogypmscdctattejvzxoz, common/rny/gmhjifjmeglhhuvgxlmvrbtqkbhqi, common/zbz/qgbpttkxvzudehyswbkmywaggburz, common/yfr/vojilyijspitotajqcecvhqdwvpxz, common/vmq/deibtrxkooxvolqdbgcxxtrarfgeo, common/rqf/xkezlemcanxtfjinqvjmcxgjifane, common/iim/fkkuqwcfdhtpybdsisqmwavmhjamt, common/low/gtbmoxystnbcjuabluteineyptxva, common/ymq/gsxqwfzqzdgnepglywplpblqwtpwv, common/dwx/eifscfugcsuceaqqygyssvkotyqgl, common/kpo/slhipqjvlltswlrhlwnrgzfqcwtuv, common/yck/oavakgnrspyehwwtsmeczjaqdgyss, common/phs/egaejczcbeawkmhkxejxjkzujjxiz, common/vzx/jrqrgizuhpqvjukzbnesrcforxuta, common/gfe/mcnbekrvsxncufynndjukgbsuffyd, common/fuw/kkvptcwukjneuefnnvaudnggdiizs, common/qyn/gjahclrifvwgjthhsezhdyozyocjy, common/kxg/xyvhorgmehapbppwmmcgiwnwitkgh, common/joj/fkatpoidrpeyuzifmuopjrpjgzyxa, common/jbr/jvlafuwwgfusldusjgxcwjhhkdhcf, common/wyy/aquzicgxwiyhiwtuctbhcpozhwens, common/pui/xndakcwdtgorochrrepppwostdvtv, common/dox/hwzjnlzrtjvljyzjzchvikqzqdktq, common/lal/reoszrizkwluhiafwtgkpqqdigotz, common/tpl/uanadmzaevumspgezqdlzcwmuaduo, common/hnt/wfnjtvznqvvpydfyadrhvzeqwadlg, common/kki/dhujuzsgaxxhhvmovhyjofcqdziyx, common/mqp/akbzuzhoomhnszxgbvrtnwgznxsff, common/dwy/oerdigwxyhotulwljzaspkatmuaej, common/sle/fthckupnwhvtqnedwaoibicornnjv, common/ofo/izobnapomksawgzlqsdskaahgunno, common/onz/qunmdgbxcdhlivquhxxxcgjmshawf, common/uxz/khxorrigvhenpcfolegiflxkmjeca, common/pas/zljhfpjzkwhfgflmvpauuampemlcb, common/jzw/fgxkejqphfoseetdftlcuuejolvfe, common/fap/axwrsydpexygwttlinjtqywkisqdr, common/eao/zgrlmidngupyeswiuedfdcfvvvvxb, common/cmr/fvqlnxkqezcufzeofdplbiizdxjij, common/nrz/aegbwyargyjryxllvfbrkeuzghuxt, common/nut/cjzrxawwkzfipgpfihbslymxcdwsa, common/yxr/lpqhhwriuvihlgkojjnbhyktjrzkj, common/adp/rbyptjhnxwuhbepwluiywtelluqaj, common/fzo/ddnemtjgsjodsjzxzwbbrshguwjey, common/jsa/dnsdlrpewzqttuwvnqegwbkwxckdy, common/hql/kcpwljkgzmiszsbtygnzxgqswurau, common/ddr/xksdwxvurmjqnlzrcfgvwpgiazmcl, common/vdd/uueubkqpkzcnbgfguqwxytnulmsay, common/kkv/reypidvcbarchmxvxrtouduuftbui, common/yot/uafpyivddmsxlmxslviqqmvzyljpc, common/ffy/betyjpoxbovcwlkppckpwfhapruir, common/tvd/qshzhwpwcwdpttjosjdxzkysztmhn, common/mhv/xdzbrozmcouhywckxakwtahlvxlay, common/ufu/gbwpnrblzkwzcfrbupkdbbxomcrrd, common/csq/xjiuqewoqdjkwxqwkwdysekfrgaei, common/sdo/icmpjgistiwouydmijaxlrflahddd, common/rmu/upxsitiwgzbcvxyjgannrwhmpazxr, common/yin/ffhqehglahkletlpfderzqdqzukxi, common/obf/oskiejhdbjmolmiffaqerrguhqnya, common/qzg/lccvwnrfaioxlhhkcxutgursfcxkp, common/jzx/lueiletcsxlcexfqzhtelqvjhgwtj, common/lqh/gpnrbkwosjnxwkivoyfelavbtdguw, common/hdy/zrjowiatylxokqxlkldviesbqcjzu, common/lhm/oerygtolpgmvwgewdiqxedxehcbdo, common/nyw/tmadhgqgapfotroplsqzantxludfq, common/bgr/onekilfvpngmqlhsnyflsnagkybfy, common/hon/fusxmweakrtivjiicvipyurhrqcmd, common/uww/eccqkjlxaqvmtmlakmbnbvyxkxkri, common/lwn/cvpofpstcqkudygcqcmafvtkptrln, common/mik/mlnyxelakbqtvstydxkxdgpuzjeeg, common/aez/mfbzrqyuszqxuxzjxjajupvjruyin, common/sdh/ttximxhuskddupszkdcgxbvzfoifr, common/jrz/yxaozjcixljaxzjvnbfiexydsxjiv, common/clg/rgpsiughwmparoqdjacsjioxjrkwa, common/opg/zdoqjqrqnbvzufgftkxivkzzwvfju, common/tbs/pmdhwadqrhkodxkwajykjoffazfrr, common/aqt/tmnarcwgxylrfprdrzcydqfaqikpq, common/onc/pocukjfyhwrkihtrehrurtpfjkhmv, common/kys/whnnbaknwawpldzkpicalgqifhmzo, common/yhc/ajhdubknfxnmwqjrpdnomqockbkws, common/qcm/oknqitutjgeorakxaypvnplhknrgm, common/ycl/ywhhoomkqqxdpjkllxkqyipaafzku, common/bhw/nfpzpilswglyjbsmvkbbdsdnlqykj, common/ohm/dfzzjqlqpxghfbnrwlquxctcrvaeq, common/vey/bmxmckchdcgklmmvxulzqqwcnyzuk, common/kcx/xtasyvnuajqxqbqqmonyemyhmznnr, common/hkp/wpntubbbievayeexfyxrfgahzmswv, common/vlq/lzrvnduopdfqatimklvqkqmyaendo, common/mnr/jyqvqpscvkwcysrbysocgactquzjk, common/gtw/ooxuwxntioufmeqlljxvhrkjzqewi, common/rqb/gprwkffzxswkxcmcjpqfkfbeqycoz, common/ntn/afvhdwwqztphydwqgtzfkdjbfoscj, common/vvi/svfefdbyuvhxkeauafkztsndfggnb, common/vnv/fwumzedtwbuzgnagpxtqjdbkownbt, common/hth/kxdlmrlocorpykdtqoqnapxqhgfnn, common/mpt/vsuzjhoyofmnkikqorwwxfacpkdok, common/nis/ikhvpdtwbillcwihcqxzokqvmulpr, common/cgu/zskymhkndzltpxdyjjxdffkfpuxxj, common/xya/wzufffepefmylbuzckbnzxzsfqpha, common/pck/dqbbmerhjboschomhlghvocxpzlav, common/wkx/cjmlypltgmyrudizdqzaognieodrp, common/drc/yxvsmxmorzblvmoexpnikqjcminpi, common/htu/svsnxmhmwelshrtauixjboaiwcmql, common/eee/dsuvhlcgrcdrcoftilluwpbbgjmnu, common/vij/amhfspspyjwxtrjvbeetnyigiopix, common/trb/jqfcuugxllrbsliykbtdfkawxriup, common/vly/vwulqsjmbnbityvjdejmrxqspebpw, common/hky/lugxtdbsbpwsfhmgvtgfoxokkgbvo, common/arm/azpneqvkdakuqrahypjkzlslhoyyi, common/jyq/uudyciebhchmyjduklvqfikrzkluw, common/skk/nkggmhbpjaawajkidtjbydfluzzfc, common/gsf/nywcypgbdiyqxypmcttqjkccoovab, common/oei/zfuoitzanjuuowqzmjjcqagzoljyc, common/zwl/vrxiscdbrmbpxpwxblgdrrlpaxjuq, common/pfu/xevcgnfimqqzxcbbtqtklpnouqzvm, common/upb/ofizgydertipfkxsleiklywgnilqf, common/lan/gyhjfeyjwhpzzrvfnxzlywbbhknjm, common/xuz/pkdwtulpmqmkhwcbibdjeynyjsrsl, common/zzu/qqxpkpttshhwymeimdjwchnxmxlyj, common/zbl/hkrpkcirxnagbtpsgyqgurhddbaie, common/tqd/jessxmoizhukkkuadvebqirphtviy, common/zgr/msaldsdtadqlsuhzjqkzwyymlinta, common/wsu/oaayzqubcyggpaykbndgvfdgcibkj, common/red/etboydepbwzcagbcczqkpuruozaew, common/sci/eatfmnbrvlhuprdowrimaqmpqofig, common/mja/fhiyyfkefgfqlrlwvrosyrhatgckm, common/vgw/etqwfnokoljlepwrdvadagthdbirf, common/ope/cmswhzqgeksqcqsfbdhpwockbxlzc, common/sif/fxpxjorvzpxuenfkzkxmqadjgmrqx, common/fvn/sfxkgvyofwjjeycmocyiznnagunsp, common/coc/nhkpvuaqqokykvnbwsjgcpypsaper, common/tkk/xgzlryrifzuyuuczzijniwlowtixm, common/ybv/cmrjgtcengbldooprevpvqmzywcxd, common/oeu/isiwoyrfsnkctrejwvqggbnavwhbv, common/vss/ktkkdezlwwhtrespqhycltdfzpyun, common/csd/tlqgmdjfxumzqskocsdhdktvpdkac, common/utv/ehivlklxdduotzuhhvgpwjqghdmml, common/lkh/ujkgwuhcqzgehwcqxmcymqsnijwlt, common/qeh/aqjpswmsuictqmqaiohmwghjbvhhy, common/ahf/ubrwbwpotgpmyxtrurrebnimzixtt, common/ghp/siaukkgiityrocllovqqlyhcptviq, common/rnx/jxiytofsxbftwagyxvvjvmscqdrni, common/mvm/vtqrwoftkdxxeqnacpzbbalfyjfcr, common/bsb/eggspsvtbxdylrjjasogddrtmiljd, common/dap/axvebsmcaqblbadnqcuitkflolrut, common/pzg/jixqrciuqikzwchnrlsokcgmjjalz, common/vme/ioaeaxhlkcimplfvavnhidllounav, common/kyp/gzrlgllzwwwetdmqcgfaxfqqkyqnc, common/gwv/jkzaldgfnbabfllfgapcaaajwtmmq, common/lkk/xyhektdfmwwqfodqhjmssztleuccj, common/fik/lzncjggpfmfahkcfrqrlqugbvtvcb, common/nth/vdzndfjpvebmoerwtrodiuowswxsq, common/hls/dsfagjqjfeagapdepmzoidthcayvs, common/fuz/dkarsptfjuonwufmdsnoqhdjbknki, common/gaa/mowvgbltywzhxdsfjzbtozbjclrjc, common/ftv/vswzoucacljoqwvmrfrtdenfxzdes, common/ryk/baehpvgeajaxcherntfxixcbcidyc, common/viv/uexwurvpteyvrcoqgjhaxpwukmmmu, common/big/uwzbfuyqzdctspnoavjirlioygrva, common/snv/wqchxknrmvntqwccdlgrckbqenfkd, common/xuw/fbncdufcnvzvgutjuphugmizyajfw, common/vmm/ycjkgsqsmbomvsheakjqpcfbulvov, common/tzs/oysqtbvtzmjilvvxffotwhbxixchg, common/egx/vatgibnhpljzykfvfxwdbijoyxeqe, common/fbd/dxyphbrojlwhuxqszcrjaqthyouow, common/xyl/xntqkqnwzlgoicvccirnvbagsqjed, common/pqn/ryejtszcvqbrpogcalauedwmoghci, common/ufm/kuexanzbbjxabbjqgaxggzxhwfuef, common/tmu/rukhplwbwezvnjgfbyzrdbkkxtqpd, common/rpl/axnxuxszaapzqlauqwrdukelwfdcp, common/eiw/zczoitmbbewskewbdqlkisdpufzxz, common/cgb/qswtfyfkubzqljjsmvidpxusradty, common/vil/zemxgprczgzlbbjizhjuuxogkofvx, common/bnc/odlgdliquvdcndofcuawrgsoyalss, common/gom/ookkrsbzwccdcxrytytbqqedqrjfd, common/tbq/yjphbvcktaedymjiciscwbxbaxbvm, common/guo/zbwbjvwhvufandjocvgznugubtzks, common/ghv/qxcdmnsvevwbijecgetqjtyfoqtmy, common/wpf/uhvbweftivenfczspdfxbgoehxniw, common/uoa/tegygjvwcbzrkbuwqluqswnahafot, common/ead/adjnneskshstvtdlcyhpshomivqrm, common/rnc/pmzhkwvsonynqvcckmqucwwcumyjs, common/kpp/unttxsiurhkffmgfeohbyrywjnemx, common/xlx/idloojlhbvehocrovyrfjirgmpcgd, common/ydm/fhgvysckqogvymhzuzznfkvymnavr, common/uyh/nyhxtfpameuzdnxtzsbvatipfrunp, common/xfz/ffeplntxoaykqolyejqqljpjyvshx, common/aqz/wnwzprndjwlrqfahwwefpcuprziwp, common/nlg/haniohrpcumprgpxgnffowuqcaiwk, common/bbo/ivbyakooinuiwjljvrdkkwtepstnj, common/ybw/oymrgyoaiksxonfytbviyssiyclmo, common/htn/fnzqovsrnetqvawxhvgnawbzrhrti, common/olt/phxizkxasqosykkirclbopikkvkyj, common/dki/sficnahbrewmdljzsycflizttmwfh, common/ryz/pjxsvosmqpmojbqoiivsltuqgffix, common/cug/fbwuchxbgsfutewdxremlokdttqkg, common/xfy/rvcknywnoxvlrxuarmuiigmlozaon, common/rnh/rftxkkliwpbnzmmojdjskcfpjbnwk, common/wbj/jjichmxxcakaiirogcgxesmgzlono, common/bjc/uwvnitolwfbdyzlrpewzzuzhygvfv, common/btl/epsxeeoulmreesyfcmmvuxhfasghg, common/kxq/nnxpqrxtfvsireeymrzlygdkrmeag, common/jfb/myblujppyckbzkqybmttqslnjcvhd, common/vbx/igtnvofjfyszbgvjnejsssfjzgtau, common/nlf/ydlknltrcmngaufvltjhxtiiiiaie, common/xxg/bcxiazzkqofqbkiovrqdycervtqfc, common/zbl/rvyipgiouqwxwfuwvzupownjnnozm, common/cvr/muqdpvnmfxluoluyimhpfwsamzvjr, common/qem/owauixlembjirixykctmhqvionibr, common/ymk/porvudayjtrokgnwidydbnslspurv, common/zsq/whxcfyzrhugxiswenxutyaodgkuln, common/jho/reqpqoufyxbxbdlztxfmlhkoktbxb, common/xic/paoqdwgmvpknychkihmiqjjazqyjr, common/taq/rduehvlprqgawbrktcptqamcgentv, common/nje/wqzgfmywryzreyojubrghbebbiuor, common/bpp/mbpikkfpnttamvjrcjxhazfjdazod, common/jfs/vypdxtyxdxjnkhkstqqfxlpryjygz, common/llk/iuycvruhcejkktwlvxhpobiypkodh, common/yrd/pvhieqyyygvnqbhsdlveeybfmpzhp, common/vwg/pedhwdfzthzcqtlzjyzwxfvmyzfro, common/zgk/oyixnwstsopompjnniptxpqsaizyr, common/exc/deoqvptptlldbgvgsuhabszigqsxp, common/gif/uqbltjfnlhbulpglskufoikpimpxy, common/xmf/vfkzdbqjyaphjvsjqixapnbifygus, common/uda/eqegiucjpxbodqtwutwaoakqpizil, common/dze/rygdsowisbauogzaenpembknlrjxk, common/qdt/szwufxdvvsgbwktsocrnlvduetlhw, common/qdt/atnekfcjydzobhyydjiqxuttucwrv, common/ekf/vpxmytqsoqvejvrdiegpphczrbacw, common/yjr/idjdqwvqjcfnnkmnlzkdxiahrhrmc, common/ztk/qcbqpediewidlsduatakvixkijwwf, common/qml/yypyycwccexfgiguvhwcphwtsivwa, common/fsl/xbcqwlovabqxvedtsujeclmirupfn, common/ecf/clqxsdubbnradepwrvcptrzvkmypm, common/qzb/aaldtxustozevrlvxpawrabfqdeft, common/udx/fqoyibsatroyqhfbqqitapycjxhjb, common/idv/ndogzzbpqsdskltkeohfmffalqujc, common/uiw/rvgzwmbsnvgsmkfnrrcqqmrrjocnf, common/chy/ldhixmpklitbzfafwkaoufnraktks, common/vqf/jzjcwkarnytvuyzqtbnlkirryieen, common/zja/zfmysvzzsgsrqjtnvlwqkbpnmvsht, common/zlw/zqqvhyldzpnwmilraunhjgvcqkqux, common/jsa/znejchqkledqxvwbelmuqfhnhrqnb, common/twp/spakvqoyiodnsvtipfecpneyqlchp, common/emi/hovtxhnywzjtizlcuxshyevcfndap, common/dii/uwrxtsaehgpregliqbqydeivyjsbe, common/pak/qqqovoundmwngysecspfevlrhqjqf, common/xrq/wgnnvjpnejalhxcxnnrajyzicgntr, common/svi/wuapuybmqdxdhtxlnncbethmrzmoe, common/jou/lcruvbuvnbcxvjnbvqcohtrugnojg, common/yvf/oybjkhkxgveuddfgqwxxchupvhzsm, common/wad/zsvmxnoitytjmzhrerfmehnggxipt, common/dsi/kesgzmjlrogddcxcfcyejbsjjnazl, common/rfo/vfwjesfeivptymobyvvldxcoolxms, common/yuz/rttifgiqezspyboggzscnzyjmivrh, common/lxf/mlaqbhnwnoeiixciolaaeyrqdnfcj, common/mvq/wpjzfpspysjkdlfueswamevdjpwto, common/vgq/fktfixhbfkjbzeneixrnjkuhskods, common/voz/jobcaxbndeznytlxzmerwnoaukthj, common/bmq/vkiurlrhtdxechztrxgjumqwdeque, common/thb/argvepurguajimyfpocagwykmaqzg, common/zbd/ebkkbvvkfvzmayfunilytfkuitcaj, common/fmw/upwxraumckjuuduqmeuznlmxiinpt, common/xip/whmfalmndvjjtxrhxedoliecmqudp, common/jmc/cxeclkkuqjxhhudpkmxupsafjdqxz, common/auk/ckixsvkwpjmckariidmdorafeplxh, common/irz/xgcwegbrocvgzxetfdviuopjrfzvv, common/xxl/muzkngsefkakqydtjierbhtxawtiv, common/ihq/cevhjyzwypziqsgmjwnwvpsxfqajl, common/edx/rxaqugvfrmosovdeenkvwjzhwaykt, common/ysz/lnqttlthpwcbcuzzogakkvsvcmzpx, common/bbq/jykyyhfagtnzkxycptmswaqpcnmad, common/mjg/svcfgcjpddqobiqcshmwryzypqlhj, common/uue/lzdwvcrorlyvxuwoerhanvtaupxts, common/yzo/xfvfnebriivqsszkjxekvasmftfzg, common/dbv/pgarcjehrwsjjpolwfrptterbmsac, common/tak/nbontifgtbyiljqyssavvudctwhdq, common/nbj/caeirhjqjgszsyhnhaxctkdzgybsm, common/ycw/srfbxolfbbjmriyymixiaxgobyikb, common/oid/crtbmfsfkfjdwnygrfgqzgsmdwdvs, common/ens/hgxjdxxbbapnnsmlpvioggbrbblnd, common/lqh/qbrczvfionmrgigdyzvkdfylhfobo, common/rio/ajyteqcvmjpzljzuwakbvvnavjloq, common/pis/pmimdtopwzylmjkesfizafnkjfvxq, common/hrd/vcqrrtlljmvxerqdbvmadrrhkxrhd, common/qny/iqwqwtnwifyikkjvafoyeddpmriny, common/vzw/lgnauxrdxghouqkvtfaoedlfdtpkh, common/xea/baviczibucffvnkftensuwfibmcpa, common/phe/zvunkpwazqynhgwhwepdmhlsbldwn, common/clb/fxhizqwovtpyogzlfzwlebjebgmel, common/rmu/troiiupwzmywqdjxgrczmetxgioto, common/utf/bwnskdbaryobbtzowzrjlhnoduidk, common/izs/ztkxypmmoriecohoqymhoxhvcmjln, common/ggn/bdgaeteeqrgoubavuwqdntleimzvv, common/zfs/okhbqhhvpzagphbjbrufxxkjvqucj, common/kbb/yxbwimpacudrpmpgvsczvoclpzoxs, common/qxx/oaoqhgewfkhunimvxpmjcjazegack, common/xjg/iupzououpaoecafyjwopsbcfjqexg, common/jxq/whdqffeumbeunswisrjskqflhmakk, common/ipy/kvumoqvefkcacqrucpoheydcaoqsw, common/cok/jppjpvrgcifjhttgrmzigaitsbdad, common/djf/dxnfbtflwljwhysdopphpgyflzmjl, common/hjc/yhobqnqjyatzdvkqxngvlssxpypeb, common/pyg/vopnbrbcdtdzkoyafimbocadjhbbd, common/byu/yqbhiwtldhnpyfpsnlugewtdmsvet, common/scb/muhxruoqxyljdipgchubraxyygirb, common/eeh/bmdwozpswfwsdrihlghsywkmblewo, common/wdy/npvuzljlfkghlemmjhebrtytbhcdn, common/bdk/xutwnislrmobkaycqzrhkdunqiaez, common/ctw/qsfxctmjagawixnbylfythulohdvf, common/oip/eoebbcxjxiisdqvromtofdsgxujgo, common/exn/fqdvnwoyuagfjqvnsyoljuqeqetrx, common/lzf/akpkqoptetetyzhhoabxakxsagbow, common/vmt/bcmyfafpdbsqntdrkhdoyonumptdq, common/nuj/vxhlmkmqerjgevmgmsnwywtjwybxy, common/jcc/mbxrqoydamjzsalrqfyxabekfcyxq, common/pgo/bjccfubgooaxzgkzlybaqaqykepeh, common/xgt/mukcrkysnsydbpifdijbqbjhdtvcu, common/mlx/ydxwsabghemccdhiaidmznvaybwwi, common/adx/adpthqbprmxdjqaalvzmtrenezhub, common/ljj/nbgxbpdatahtcndhtrpxnpotwiggm, common/jel/hzolvfduvsxcmotsccmbxcsxmoqtx, common/jzt/vfcewkuqnsvexxcpqmmkqzcgtpygz, common/kdy/suwqwnyqsoddklpswpliloribakfa, common/qgm/joklryzoovryxkjesuzwaaaczqcou, common/ijk/hqjclkyzbrkvoymktgzbwwlgknloe, common/ogm/dulvxlloclkkchsccigqaczjnuqmr, common/pta/rkjvxdhhstfbmcvrhrvgedvkzrspy, common/rwj/rmcjmnavxvwsexojjkxwqpoookvsy, common/shi/jplfsfbbgtdyxuemqqzoksptedxlz, common/yvu/uocgfmevpcuxtevhyfvnxyfcuqwwo, common/kfv/yiipcizrtiwfpmnfyexghsjglkhfv, common/khv/agmvevuuwthbekahwmuffzddtujtl, common/qzx/rszvoqczeunwvnvpphlcptwpdsugp, common/awf/ceeuxxmwmskatuingxbpnljjnnnnu, common/lye/txteyuiwlqakhqcajtyclelmvusjn, common/qrz/ifxeqjilwgrcmcdykgvxehwxfvfjd, common/nts/vzxmdgqlrhoqlumetodubslsmoems, common/jci/dapacqtdgjxlojcuvadyvltdwwpol, common/sqj/rtsknoucggcjjohemjwrxlcxfkxzz, common/ofz/avrvdmfybxzrmiyglqcqdmdhlazzj, common/oho/coxatfxpzmjxebpckqkwivfbnervz, common/gec/amdxxmwxckbcsmqrdkhloijwlohem, common/lrd/tbantgnqeixkjorpseoqebsnayrfa, common/kvv/uwrkdwhbtafwbrqefndaxsbaududa, common/wba/kreneddodvkhijthqycvlkvkkynwu, common/hnp/mrcyhsgdehsqqtiiqartvmjgjmrzo, common/sod/ycljmwnuddfuuewepraijhsqxjoqp, common/vgz/ndifxpuquikrecevnzgudepxczdsf, common/lfp/dmprgrvgmuiswewrhsfhbnkanymhi, common/lin/bearebdlptespumefjowbqmdhesof, common/xwd/wwfngezqzocrhgjarpzqnayhqkhjs, common/aue/zczetepungwjyiocllqrrscgfqzvn, common/ttl/olfdhfzlkfaorwmneomclniptxuil, common/mlt/bubcspgogsipxsiolubpdpvxscomv, common/sea/qoxficdfsonlfqlyvrqslbilivrft, common/bfw/gkawzqwaoibtalpjhiwhbtrezrjan, common/dfp/yswumxiwfjtqaqddcieevvktlhcym, common/bvr/bjfyrgsoyxlioortapcppydewheca, common/hql/jwtowgikkvbnbxkxbsvzvkxwarnmw, common/tuu/uazomncyhvuzequiqvbasdoslpfzc, common/hra/eoljuqppaaszcdmqfpvvotogboxup, common/qqa/uvffilwlwxrllldpsvznikrurrauv, common/lcg/owoinbsaflcnvwtupdfkhyteccqrj, common/oke/akwhgmoyujbjylwhcwodzufqxyhje, common/xei/vrtzhigpkxnxmcctbkkcxrzofguxu] were removed from S3 2024.10.22 02:10:01.647481 [ 1419 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CounterClass.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsNotBounce.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SearchEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CounterClass.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/IsNotBounce.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SearchEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WithHash.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/WithHash.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_10_10_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:10:01.711818 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_825_202} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_825_202 to 202410_1_825_202 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:01.724950 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_825_202} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_824_201, 202410_825_825_0] -> 202410_1_825_202 2024.10.22 02:10:01.726972 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 90464 2024.10.22 02:10:01.757455 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:10:01.866064 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_164_164_0 (state Temporary) 2024.10.22 02:10:02.152434 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:02.155661 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:02.232980 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1377_1377_0 (state Temporary) 2024.10.22 02:10:02.351610 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_784_148/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:10:02.468549 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1377_1377_0 to 202410_1390_1390_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:02.471076 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 20.02 MiB, will set to 1.64 GiB (RSS), difference: -6.26 MiB 2024.10.22 02:10:02.545069 [ 1181 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 02:10:02.549300 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90464 2024.10.22 02:10:03.279424 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(b3c63db6f609fa7d1dfd8b5b371312c2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b3c63db6f609fa7d1dfd8b5b371312c2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 02:10:03.301653 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 186096, file segment info: File segment: [0, 33554431], key: b3c63db6f609fa7d1dfd8b5b371312c2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 02:10:03.305299 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:10:03.840345 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1178198 2024.10.22 02:10:04.238162 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(fc2a9f15dffa804d881554dca1bee8e2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fc2a9f15dffa804d881554dca1bee8e2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 02:10:04.244188 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 745, file segment info: File segment: [0, 33554431], key: fc2a9f15dffa804d881554dca1bee8e2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1391, kind: Regular, unbound: 0 2024.10.22 02:10:04.246796 [ 1391 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:10:04.293662 [ 1708 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/syd/jgsfdgcknijtquhlfqwifrzmhqfwu, size 186096 2024.10.22 02:10:04.500636 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column RequestNum 2024.10.22 02:10:04.537572 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column RequestNum 2024.10.22 02:10:04.552838 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column RequestNum 2024.10.22 02:10:04.575909 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column RequestNum 2024.10.22 02:10:04.614246 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column RequestNum 2024.10.22 02:10:04.641791 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column RequestNum 2024.10.22 02:10:04.653480 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 24.45 MiB, will set to 1.64 GiB (RSS), difference: 16.34 MiB 2024.10.22 02:10:04.664305 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column RequestNum 2024.10.22 02:10:04.833751 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column RequestNum 2024.10.22 02:10:04.907324 [ 1385 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nvs/gcovvqfodomvdpbaqmlvudxijguwy (LogSeriesLimiter: on interval from 2024-10-22 02:09:40 to 2024-10-22 02:10:04 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:10:04.981564 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:04.988982 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:05.003193 [ 1156 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:10:05.039358 [ 1156 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:10:05.158752 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_782_782_0 (state Temporary) 2024.10.22 02:10:05.353506 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_718_137, 202410_719_724_1] 2024.10.22 02:10:05.354454 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_782_782_0 to 202410_789_789_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:05.355122 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d0af58282d740950254444259a1826b0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d0af58282d740950254444259a1826b0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 02:10:05.356881 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 97679, file segment info: File segment: [0, 33554431], key: d0af58282d740950254444259a1826b0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 02:10:05.360192 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:10:05.355128 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_718_137, 202410_719_724_1] 2024.10.22 02:10:05.384903 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_718_137, 202410_719_724_1] 2024.10.22 02:10:05.390447 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1178198 2024.10.22 02:10:05.816059 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gch/zemoffkbxrfxtdfiibwdtpvjzbdne. 2024.10.22 02:10:06.275769 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(e7ca05d04e0f2c7365afc0f9e6db6b86) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e7ca05d04e0f2c7365afc0f9e6db6b86, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 02:10:06.283496 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1093, file segment info: File segment: [0, 33554431], key: e7ca05d04e0f2c7365afc0f9e6db6b86, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 02:10:06.291483 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:10:06.434851 [ 1702 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xgp/bxmyqebhhxeihzonuxunrgcptzzov, size 97679 2024.10.22 02:10:06.435545 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 12427 2024.10.22 02:10:06.556345 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 90513 2024.10.22 02:10:06.565139 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:06.567554 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:06.721554 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_817_817_0 (state Temporary) 2024.10.22 02:10:06.795823 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:06.799926 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:06.867383 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1378_1378_0 (state Temporary) 2024.10.22 02:10:06.869210 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_817_817_0 to 202410_830_830_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:06.913735 [ 1176 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_825_202 to 202410_826_826_0 2024.10.22 02:10:06.936129 [ 1176 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:06.941459 [ 1176 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:06.944236 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12427 2024.10.22 02:10:06.967091 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_826_203} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_825_202 to 202410_826_826_0 into Compact with storage Full 2024.10.22 02:10:07.073148 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column CodeVersion 2024.10.22 02:10:07.112372 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_826_203} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:07.118120 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1378_1378_0 to 202410_1391_1391_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:07.185681 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_826_203} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_825_202, total 12452 rows starting from the beginning of the part 2024.10.22 02:10:07.187318 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column CodeVersion 2024.10.22 02:10:07.213222 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column CodeVersion 2024.10.22 02:10:07.216900 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_826_203} MergeTreeSequentialSource: Reading 2 marks from part 202410_826_826_0, total 23 rows starting from the beginning of the part 2024.10.22 02:10:07.274250 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_507_745_91, 202410_746_746_0] 2024.10.22 02:10:07.317826 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_507_745_91, 202410_746_746_0] 2024.10.22 02:10:07.336392 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column CodeVersion 2024.10.22 02:10:07.364349 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_507_745_91, 202410_746_746_0] 2024.10.22 02:10:07.463494 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90513 2024.10.22 02:10:07.484089 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column CodeVersion 2024.10.22 02:10:07.533049 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column CodeVersion 2024.10.22 02:10:07.548496 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column CodeVersion 2024.10.22 02:10:07.660359 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 26.23 MiB, will set to 1.64 GiB (RSS), difference: 145.90 MiB 2024.10.22 02:10:07.959975 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 92 entries to flush up to offset 89590 2024.10.22 02:10:08.136987 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:08.146802 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:08.160980 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 679 2024.10.22 02:10:08.290936 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_803_803_0 (state Temporary) 2024.10.22 02:10:08.360258 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_803_803_0 to 202410_809_809_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:08.376822 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_826_203} MergeTask::MergeProjectionsStage: Merge sorted 12475 rows, containing 12 columns (12 merged, 0 gathered) in 1.415431438 sec., 8813.567132313477 rows/sec., 1.37 MiB/sec. 2024.10.22 02:10:08.434205 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_826_203} MergedBlockOutputStream: filled checksums 202410_1_826_203 (state Temporary) 2024.10.22 02:10:08.523111 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:08.529001 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:08.542535 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 89590 2024.10.22 02:10:08.603335 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_661_661_0 (state Temporary) 2024.10.22 02:10:08.766301 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_826_203} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_826_203 to 202410_1_826_203 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:08.780042 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_826_203} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_825_202, 202410_826_826_0] -> 202410_1_826_203 2024.10.22 02:10:08.817096 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.51 MiB. 2024.10.22 02:10:09.102388 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_661_661_0 to 202410_671_671_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:09.208839 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 679 2024.10.22 02:10:09.550513 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 13.46 MiB, will set to 1.63 GiB (RSS), difference: 39.33 MiB 2024.10.22 02:10:10.153201 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_164_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:10:11.249552 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:10:11.253082 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:10:11.455624 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_164_164_0 to 202410_167_167_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:11.468056 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 90564 2024.10.22 02:10:11.523575 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 15.10 MiB, will set to 1.61 GiB (RSS), difference: -3.03 MiB 2024.10.22 02:10:11.973364 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_618_126, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0, 202410_622_622_0, 202410_623_623_0] 2024.10.22 02:10:11.975315 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_618_126, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0, 202410_622_622_0, 202410_623_623_0] 2024.10.22 02:10:12.028258 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:12.032687 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:12.036920 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_618_126, 202410_619_619_0, 202410_620_620_0, 202410_621_621_0, 202410_622_622_0, 202410_623_623_0] 2024.10.22 02:10:12.188670 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1379_1379_0 (state Temporary) 2024.10.22 02:10:12.243872 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1379_1379_0 to 202410_1392_1392_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:12.275540 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1387_36 to 202410_1392_1392_0 2024.10.22 02:10:12.291995 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90564 2024.10.22 02:10:12.295368 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:12.300142 [ 1209 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:12.343372 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1387_36 to 202410_1392_1392_0 into Compact with storage Full 2024.10.22 02:10:12.390357 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:12.399516 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1179738 2024.10.22 02:10:12.411512 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1387_36, total 11175 rows starting from the beginning of the part 2024.10.22 02:10:12.453160 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_1388_1388_0, total 69 rows starting from the beginning of the part 2024.10.22 02:10:12.478256 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_1389_1389_0, total 35 rows starting from the beginning of the part 2024.10.22 02:10:12.625813 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_1390_1390_0, total 44 rows starting from the beginning of the part 2024.10.22 02:10:12.667499 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_1391_1391_0, total 49 rows starting from the beginning of the part 2024.10.22 02:10:12.715790 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_1392_1392_0, total 51 rows starting from the beginning of the part 2024.10.22 02:10:13.325279 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_784_148/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:10:13.559185 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:13.561743 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:13.621210 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 13.61 MiB, will set to 1.60 GiB (RSS), difference: -21.11 MiB 2024.10.22 02:10:13.846482 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_783_783_0 (state Temporary) 2024.10.22 02:10:13.957828 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_783_783_0 to 202410_790_790_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:14.014709 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1179738 2024.10.22 02:10:14.012539 [ 1268 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_785_785_0 to 202410_790_790_0 2024.10.22 02:10:14.058983 [ 1268 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:14.066342 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:14.089746 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ce71a03246c0fa2106694fc16148b0f1) : [0, 279]: Setting download as failed: File segment: [0, 279], key: ce71a03246c0fa2106694fc16148b0f1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:14.098102 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ce71a03246c0fa2106694fc16148b0f1): No space left in cache to reserve 280 bytes, will continue without cache download 2024.10.22 02:10:14.143739 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_785_785_0 to 202410_790_790_0 into Compact with storage Full 2024.10.22 02:10:14.170290 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:14.188708 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_785_785_0, total 1509 rows starting from the beginning of the part 2024.10.22 02:10:14.224961 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_786_786_0, total 1201 rows starting from the beginning of the part 2024.10.22 02:10:14.248172 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_787_787_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:10:14.266370 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_788_788_0, total 1929 rows starting from the beginning of the part 2024.10.22 02:10:14.307376 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_789_789_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:10:14.441169 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_790_790_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:10:14.446408 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 12437 2024.10.22 02:10:14.578296 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:10:14.644083 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:14.643980 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} MergeTask::MergeProjectionsStage: Merge sorted 11423 rows, containing 13 columns (13 merged, 0 gathered) in 2.309258214 sec., 4946.610097886611 rows/sec., 933.71 KiB/sec. 2024.10.22 02:10:14.648943 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:14.723351 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_818_818_0 (state Temporary) 2024.10.22 02:10:14.884918 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_818_818_0 to 202410_831_831_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:14.888938 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} MergedBlockOutputStream: filled checksums 202410_1207_1392_37 (state Temporary) 2024.10.22 02:10:14.938890 [ 1273 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_826_203 to 202410_827_827_0 2024.10.22 02:10:14.954277 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12437 2024.10.22 02:10:14.959421 [ 1273 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:14.963415 [ 1273 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:10:14.996842 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_827_204} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_826_203 to 202410_827_827_0 into Compact with storage Full 2024.10.22 02:10:15.055400 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_827_204} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:15.048481 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1392_37 to 202410_1207_1392_37 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:15.064118 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1392_37} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1387_36, 202410_1392_1392_0] -> 202410_1207_1392_37 2024.10.22 02:10:15.076835 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_827_204} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_826_203, total 12475 rows starting from the beginning of the part 2024.10.22 02:10:15.136751 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.01 MiB. 2024.10.22 02:10:15.151930 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6597 2024.10.22 02:10:15.214639 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_827_204} MergeTreeSequentialSource: Reading 2 marks from part 202410_827_827_0, total 16 rows starting from the beginning of the part 2024.10.22 02:10:15.262489 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(41b5efd63bb4e2f1f8e04b7c142b77c4) : [0, 4409847]: Setting download as failed: File segment: [0, 4409847], key: 41b5efd63bb4e2f1f8e04b7c142b77c4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:15.266120 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(41b5efd63bb4e2f1f8e04b7c142b77c4): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:10:15.474107 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1287_16, 202410_1288_1288_0, 202410_1289_1289_0, 202410_1290_1290_0, 202410_1291_1291_0, 202410_1292_1292_0] 2024.10.22 02:10:15.481649 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1287_16, 202410_1288_1288_0, 202410_1289_1289_0, 202410_1290_1290_0, 202410_1291_1291_0, 202410_1292_1292_0] 2024.10.22 02:10:15.577838 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 41b5efd63bb4e2f1f8e04b7c142b77c4, offset: 0, size: 0 2024.10.22 02:10:15.580621 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 41b5efd63bb4e2f1f8e04b7c142b77c4 for removal 2024.10.22 02:10:15.587709 [ 1476 ] {} CacheMetadata: Key 41b5efd63bb4e2f1f8e04b7c142b77c4 is removed from metadata 2024.10.22 02:10:15.619492 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1287_16, 202410_1288_1288_0, 202410_1289_1289_0, 202410_1290_1290_0, 202410_1291_1291_0, 202410_1292_1292_0] 2024.10.22 02:10:15.694185 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ce71a03246c0fa2106694fc16148b0f1, offset: 0, size: 0 2024.10.22 02:10:15.699274 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ce71a03246c0fa2106694fc16148b0f1 for removal 2024.10.22 02:10:15.708115 [ 1476 ] {} CacheMetadata: Key ce71a03246c0fa2106694fc16148b0f1 is removed from metadata 2024.10.22 02:10:15.864072 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} MergeTask::MergeProjectionsStage: Merge sorted 9259 rows, containing 4 columns (4 merged, 0 gathered) in 1.7776207 sec., 5208.647716579808 rows/sec., 81.39 KiB/sec. 2024.10.22 02:10:15.913461 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} MergedBlockOutputStream: filled checksums 202410_785_790_1 (state Temporary) 2024.10.22 02:10:15.957837 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_785_790_1 to 202410_785_790_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:15.962327 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_785_790_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_785_785_0, 202410_790_790_0] -> 202410_785_790_1 2024.10.22 02:10:16.043829 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_827_204} MergeTask::MergeProjectionsStage: Merge sorted 12491 rows, containing 12 columns (12 merged, 0 gathered) in 1.06439019 sec., 11735.35806450828 rows/sec., 1.83 MiB/sec. 2024.10.22 02:10:16.061018 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 150 entries to flush up to offset 89740 2024.10.22 02:10:16.117453 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:10:16.214483 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_827_204} MergedBlockOutputStream: filled checksums 202410_1_827_204 (state Temporary) 2024.10.22 02:10:16.297634 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:16.304837 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 90645 2024.10.22 02:10:16.309821 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.34 GiB. 2024.10.22 02:10:16.396861 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_804_804_0 (state Temporary) 2024.10.22 02:10:16.491048 [ 1419 ] {} S3ObjectStorage: Objects with paths [common/cvp/quvoptqnqkrctxciinupwkcqdwtxe, common/wxx/mwigakjecfiyqrateomoozyxzqtif, common/ttk/uvofwcvazeaadfzrfhkxccmagprzy, common/bhr/ryohjeeafzpsldiksurkifesvcgkn, common/huc/cmkxnfjehdptdwavbbumsrzhfwfil, common/nlj/cpqczdzlmndnsooshbahfjlufkevw, common/yay/iizcrrvfhscbgyrmjbtbjzjwlonqb, common/syc/ovrriuprqsyvhdvmujkzgifjjpprb, common/lkv/tnetsxsbbhasqzscduhqwadlhfxaf, common/edi/ogvnheavfloaladxbwloxnrnpcmmc, common/lmf/nvokdcyadtetgbffkrhiximhjviqf, common/nxd/hcufchjemtuiufcwyeiucrrrznxmg, common/qgq/roziohjjlifowikfllidgxjvqoyil, common/bqg/hjzikxabizocpswkhztlkviudzscw, common/oiv/uasqgdjprxokdjkuupppcafthriyk, common/let/bfexddtgkroagsarjprhiycjiqkpc, common/ewz/fwjnllchhjretauoveosprgdugljb, common/hir/baacrvfnnauxrzjsiissrciaejrsn, common/udk/rxuixitgzhpkdfsxwfwxuhfjugqfi, common/ouk/izsnyompqvzxtaptnaakjfnfjltlw, common/cpj/itmcztixtoifyposxulhnvxzgmhtc, common/uno/zxppfkyvhdmrexyqkupcjhitkygdi, common/xfr/ktqflzhzwuswrpospfhjhegxnaakg, common/wti/oskafmtuueuvbfaitdsbuvuuwnpib, common/soy/vmvjjktdomyvzfmhgfqqhjvrkwcrp, common/mew/goomhwkwiewfnpkabudajphgwvecs, common/nxi/lvatxrxmcgmyruknfedvmtkvcjtwu, common/rfh/dvircbamenjepakyfmktxkozymliz, common/bqi/qqpveubyecwcnbcurlemcvymeekjf, common/afk/rpuiiifallgttomwxehzgisjeaewy, common/xnm/amvqyivqhoggvbghsbyeaavhrnkiz, common/srs/crzpmaelgpcenainjnpmnraujsirl, common/xbb/esczjfnezpmxaumrqgqfglewmomol, common/slm/gnvbhaitucpyulabklkrwhiontoov, common/geb/yrenzpfumfcifbquwoypspspvrzdq, common/zcl/ofvbusmlealsnxebshfkdnihhrlmk, common/qxi/vccnajgghepqekutrgoghimffjnfc, common/hmj/fldhdpipowpyucgrktnqlcahzusyy, common/pay/cyktvgodqrazzjtibaqqhkqueuhza, common/aai/uqekiwihqwxffbmyatghnrrqaifof, common/ctz/guxkpjcysshgeqknbeysdamdydhul, common/orm/qehyskejzbqnrvrbdstgxhzitfvlf, common/ekh/zpblhvyjpmzfevupxbmevvmodxvam, common/uxl/vfkgsoumxmwqadyuhvnstitmomxar, common/ywf/ucqxdgkuhrfgmgnzaiifqzrwlrdvc, common/med/ndibemulcdvrmnhxnfpalqzsjvciz, common/yfb/hzwuiuzaxuanckhefsqqxxllclrkx, common/qzx/qimlpcdqnczhcbbpnwxcoehqjpsrv, common/dub/pzhflzpppdpuexqofnmtfvfnfsfzj, common/gem/locybswjqnibdgnnqqjxfzxmkbihp, common/gjd/bddedolrepuaeczszbmtkkhqmjwjw, common/xhr/wtxvgtebhbludurnisadbhuhvcqkn, common/ylc/qjmtmtzjnvvjpkactzyylincxtneo, common/wcj/rarjuyjydfhothicwypnrlfabdedf, common/fwe/ijvhbywklnwxfmzhyuxvzazrrbqzn, common/erf/stbupcxylwygbyjaskvdknyudcmwd, common/jlc/oepnrtwiketjaklirpgxnoyatjtun, common/bht/esjspeiigkrltstelezrdtkjscjhw, common/izw/pmqkvwfvsmmftyjoaddbkuemqyiwg, common/jdl/jimsevbvycdnzrghlefbkbqvsecfh, common/aec/gkbuqnsbiqrkntwdgpgzysglrwxbm, common/wyh/mdtmzbmxounoywldqqlvlymmshheu, common/vkk/ujnwjoahaewcefoffoagwmjzphtcg, common/ydn/ejtpuflcgjerzhjhngzrftbfjigwf, common/xth/kyqvgedlovpryhigdsnljasxeapim, common/bby/aroqkvsbvngmszmfnmyljnkckaqdn, common/zxz/ljktpicnogcorvvojfdssxzghvfqb, common/oqe/zajqxivxjgmyftiumhegmvjnrjgpu, common/fum/buipzmnsycmgutzmnhulojkhiyqyv, common/bie/etrzwqsrevucylhuymvazbgnkrkyc, common/dxy/vaddvehbgtfbbjhdqypbruxybffbo, common/cse/vrqkhmfprveirmmiyidmmtvefuxzr, common/luu/pxdgwxkuvbuntsqrlzqarfrcrrodu, common/mkl/cljkvkvlaydgxwtqksjoqhyphovuj, common/xmc/urfdigdiwzqjxpbzbzhqjjdmporsr, common/kvs/qiovfygvvwhusnqhsqvnnkovkqcqr, common/phy/atzlbrsbylfkphzvlbkzjvkmawmuv, common/cry/xpejqekxnwmzfehzmcqkxkrfaehhz, common/fei/rkzcmnjbyzotgwgjafrlryszacokz, common/jul/jhcebfqvkubthrmfqznrifqdhkgvg, common/wpl/gydgpehxtoupqxvdcppmpzppsforr, common/bij/siggcqstwzalydptgcoojrnavqvgh, common/nex/cewoseywncxykcikttgqvvqydmlte, common/ufk/zagckivkfrtmpkjwofwmcsjsrqdnz, common/ogi/nwrqfbtccaknrzfuqtdkqojzsufit, common/jri/jswksnqzjsmasmvufmdmapkbrmamn, common/cxp/kwixcjjpqgizctjznpngpfflgmtkd, common/fpq/orzsuvfdcdponperjktqfhqvfkjdw, common/gjr/rlolkbcgrgvtrxaawjpqpajysjeof, common/gak/uiukzozaxcrkuunrzbmwjkmpjmtju, common/ghi/qjegnzuyaiyonbflnopindbjcnowu, common/wbn/ncthnvgpnnylqenbjyqeccmcnyqba, common/khl/ofclhtmqgfjeavoyiphlrswtomwzx, common/zby/uqfnkogoupqomkhdwdmjdveizyzlb, common/yls/lwsaoouyngyfxtakqnilykvsyxvuj, common/qfx/rhkpmcudioqrfngbluchjncgdgkzs, common/wyn/kprhbmmzbbefwaxjhcttvswmkbmyr, common/zii/ttxzoqxchjeyqxquyskbstadugqng, common/oty/rkgachtxrtorywbxdidipkhswkioz, common/gih/kvybkwwspuovklogjmenpvvvkxdzh, common/rwj/kkmcdacrueuqzqzksxgbzanklefzj, common/kyt/jccxzzzpbnxdgbjvwersmmdkjetgm, common/lcp/uciphtmpqzsxbkmvmdcuipmqcbhky, common/ipt/aewiwtvbosvxwrcvwkgltboqnttsj, common/erc/dfugexzvtvtusifdivkhiyagweugk, common/gcd/ujyoaobjjubiuqjbisfouisytsoqo, common/vck/najberpjqiyvasoowgrunylyridaa, common/pjx/pwjrxnoudnhrzreiiogpnyjtqolhk, common/ljb/notecoikcimfpbuvmlndaoedfleea, common/bza/opbznjhkuhwwdbojsinunihscjrrd, common/azm/yefxubmxihmpvyifwcfdggjvjsizm, common/oag/aurkuemovmhfgdzstanuzblnhkcyi, common/yvq/kjhelaywmukwenlchvrqpsxbrnilj, common/yga/rrfmsbmiiopsyijjpcqgxtvisgzxq, common/wap/lijwvsxjqvmzhhddzwfcvfeunpata, common/chp/dpynbecgibnzgneietnktubnobala, common/yof/zidahkfrhvzxufucltqynhjwyvmxl, common/api/hjtiijjdjxjvspnopmclkgkjljinh, common/glu/xhdxzzacnyrnksfaxqlqbwcxnyhoi, common/fld/sfulmpvixqzdpkcftafzfhcuozoug, common/igz/xtqmrwftpnoufzulgyshqwuyhhzxp, common/cjo/medtgqyateeqbkqfxxghdcvrhhccm, common/snh/vbmenugkosirmdhohwjpfuomchula, common/mdg/urdpngpoyybgpmyaedxkegjqfodws, common/sjs/pauesfqlxfkbqunackbulfxmqejdt, common/eqt/fowpelyxgxubnkoqikmvwovkiojok, common/qvg/pfezznwkibjmsouijpfsqhvaabmmr, common/sla/qxqvcdtolmxyfylajrsrmwuqkhubl, common/vur/fjrufhsbntqpiblqdjatskxnesezh, common/cbg/tcgephsdompyxyillywvbtmethrwy, common/rum/seguyvobwaumjyfokdmmsdsuxjqoh, common/uqr/igaeokxrmgjukdpjrrvbalotloqvk, common/kew/ytdqmykggymkzuhtbdpviicyfyklu, common/ekb/godhpogzsmlupmzklaebgklcjiolc, common/amk/jcclfxlupzqyrrmjlegemcfdqihtd, common/nhr/jaldecsuauzcazdwxdshwgvrvghih, common/itf/fprxyvnimivhewlduljlvrwhxmsuy, common/xqd/ckkgngokadbnbqerykwbjyoqdjdqj, common/smj/tcvncqkbnyqzmqtdifigmdeocsgkv, common/coy/bnoxkfelxrvijjgqdkahdavvaniva, common/waa/orctppszshxxkyiawdyhtciphgcxm, common/qtf/oayctiqbyelfqyirykztnxbrmhrqp, common/fuq/drraggwouhqenszhfrttofpowuhds, common/ivh/rglrxogxhlubshvkuuysmmzbuohjs, common/bit/pjraltuakbcobrgjeulgsqldutruz, common/eam/iijrciznfjzybxarlhmbeyijrsgzl, common/xyj/hpttaspndaecmcxbugfijhnvulbmm, common/pko/ygnnpeisjwvxnzxcosyhuitxoftnc, common/toc/sfbczjlfbylhxrsztkpdqgyueutrb, common/kcb/voophsxbvhxxfnkygawnhyouhlncy, common/ura/xwsktyrjbcxtkhdfgosuufpzriykb, common/wbj/fhcsinqauoehnrffhhnhnyempckde, common/yie/lpskuuspbelzweelmftqudbpujfkn, common/adf/xteyzydhkoarjrvmopoyyklkpsswt, common/qgh/gbyjmpplzgfedhgjxaoyiywnstsui, common/jnz/lvmsqidfsylqysanaygssctaiheee, common/rxf/qveaobmsccmluynjakrkmirbuoxbx, common/avj/qictnkzwxikmlotxzozvtucyrflzq, common/qct/xqeukgksefibkefgemdqkpquxhzyp, common/hos/jqulkevbuqxxlwgpkxfgnammlvrjq, common/ntm/qhkdsrdrotucmeqlcpznzxqonmoxx, common/nxm/bouoioxkowwgyezljgwaunvwydoxg, common/psm/tlvguijpyvnyjgarpuwnoqslpovld, common/pjv/wgccpnuqjnlfyjuxveyebvmgcqkel, common/xfq/wngjvluhcobnobowcamsnkhqkxbzv, common/fla/kkfnynpsxpmfzvilucgemyrdqbcay, common/trx/wcqedxmmjquxdeoprosrmpatrbsbu, common/irn/idzystjstuqphueszvwwejevjgjvk, common/kgh/knbokrfunnsgmapmbonfnihtemfvb, common/qas/crcfijpglcsgqwjyzhkrmrnykzsyh, common/eye/foaftfshqmfehslpmiwgiolvjsepn, common/wao/ftndipvquixjjjhrriqttibmyqhqe, common/vnc/wmpiimyevowyvhvlmqpqbmxeazxqa, common/mhe/jeehfjpiokbliuduxrbisotfzftjq, common/sab/zfeetcnrtmpbusitvalbrhattyiyx, common/oqu/llyzzlasdylsxzrrjohefvcetlrjb, common/hox/axcrhjdifwctobawpnnerrqewoxki, common/rar/nzwbwifozcvoysxjgbvgyjbuomjek, common/vzf/zwoildypmswxstdwmuggtmtphbosj, common/ehr/rfzmbmxacpaacxypmurdnjtkmezsi, common/jcy/ijzezufpityowncyupnyfxrnborkw, common/kgn/dzrnwfkowtcmghumolpqdfzhlvpeq, common/wux/entxphhaowtsmrepcdhlswwlwbgpo, common/vjm/buqxnknfphsaexhnjregjcrjvydxq, common/vqw/hqgwanvnuppgwendknaqdrmaeikbd, common/xqc/qcfxmqrqbuspfcxifqwzckikoayxo, common/bfo/jkwozbwijjntjgxzhcschvfwfaskh, common/ris/hnfymkcrrsqyrclbsrtwpkimkxlxt, common/btc/jxtlpieoshobjsqberfvnvaxmqoul, common/naq/aiwgsgfqyxowheivlskfvxkxofieu, common/hwz/naidawlgwdsjsxfdapuxruocngulx, common/prs/ccktfegploemdpambwymddpsrpkdp, common/dkw/tgpcupmiuzoyexkaannemgbezkawm, common/dgv/zcxxbfazbspyfgegdqgezivaygkyx, common/bso/bsmlnogpeitcsrwdoojtumayysgrx, common/hel/tgamymzpmsjdwxqczfigodfjslnqa, common/rwt/abvgfoelmfnpojnroaphkvrfnildb, common/upb/fxpsfpkhyvtaxnrjjvdykulwhgnvb, common/wxa/qapgcpaltzgdynfezvuuqeflwaipn, common/gyg/fejyxsxxsbtezznzrjufoxzmezcqi, common/tum/avljqisvnwhenbjhrilyqrjypkrvj, common/mbu/lejggoeidxxbxorzhukbffymynont, common/atu/dpbnrzuyxprzrxkefyhvrerbsmoiy, common/fez/tucgcqbvzbtexqaqigzidwitqsiox, common/rvm/yexasrbxnthyzkumhftvucdffzwzq, common/tgj/wblolmewqzirfneydsxgqdfwzqdlv, common/sdb/pwcbnheszkipsjboigztyowkpowpu, common/bta/hjjgfrzvvwlccfzzjvolydmeketfa, common/vvd/otwiwmcynrmorfrhtdufvsbvngwtd, common/maj/paafjzgigdujmtkpqqkvkoicnxhcb, common/oqi/bgparvuiumpukfzodlfygmtcmyutc, common/iuh/ueehbuylzypmamaagkpzznbgtcang, common/bvj/xakmcfkyyykokzdcbqctwpkhmmxfj, common/ugn/hlffgpmnuntnuxzehbrdskmzvdhzx, common/ups/tsesscbucdtlyjqyjldxlzbefducs, common/cgq/zgtltcuiyclhcsrsdszfjguavmffo, common/sbl/wccniyqvegqansfdmbfazzdregafe, common/cmj/jlpbysqrfeikwwbetowajyejoawtd, common/qtg/gyojfdvsusojborikoqzjfxkdkysq, common/uiu/xrxslvgdpardvodovhhylklbencqn, common/rfc/ydohzxdsiwllatkaobnyhiajsnuff, common/cul/hxedhoniroxfglykbcjudwqlhfxxd, common/pky/jbtuygkpatqtuymlvfduskrenkach, common/hpe/uwjdigilebkufrdexystfokxaxmsn, common/azl/ckbebpbqctrgznmztuhafflxcpyvd, common/mmz/poscxakzmzxlsuoqssxelpjscbkpc, common/oph/ggnbkfpmotlojwsluqonznpqsdfrr, common/wuj/kbsfqifpkjbzxayjdnzbsvugnnolp, common/ufg/lncykazywrnluiypeouncdvlesuox, common/snb/vpgqngbryfwyqfgfaxmdzlybaieem, common/urz/dkevdfgxjckxvxpntdypxjnyzdgow, common/sat/iscbazgruvlxsiqotuqqysldpmsal, common/eeu/svnopeaybjflgtpvgmnneismaypub, common/cmf/wwwbliyylaelfyzyumuevvfhkpcge, common/lce/aglysmnzhtlsoeyusvqavsmhmtwfp, common/zjs/khmdiazeozrbbzgtizwxenpkloozg, common/dtd/ijdiltzlfwxqfguahxkwdfzyjgllw, common/xtn/ctxcvdexktjftamftdlzkcqifyklp, common/gbg/lnhqjzhmhkqsdmfjlivhjnxgwocxy, common/cco/swfztcqosloyccpfeoeniuaofcbad, common/utf/vnsiwvjxnciqmuntgvqpahsaetbmn, common/gna/oouhrgdpmignriqyzwqspoocqxqsu, common/seg/hfjrzrrtpsjqmlmbvylzungawcmof, common/vnc/ffoifnubiwxexoaqygnkeqwjziubi, common/vdt/hkwhohrkngllczuarnwnxnbzqhqgw, common/xsl/pbyblcfupxzwvoomgcpcoabbxnfpm, common/uir/rzbvhybkbgbantxiwqwkxanlgldyq, common/xun/qtfysiklugthzhdsduyfohfbetcvw, common/wsm/jyingpqemexunhkunsrkfunzybggt, common/yuv/hcmeecfamaqnqxgwhciutrzsqzfcw, common/okb/lwhyvjlhkkajdhmvclbemvxlucgwp, common/web/lsxyyadlpwiimqmqsjtxkaqdbihbp, common/lri/bdsffolumizfsjhhpehooeeqeuubt, common/xnz/kjvnvdwokmviyfvojqclgalvclnhg, common/kmi/ummznnnjppexadiwgmvwpkphugwed, common/jdp/wdkdafxkkeciqjuoxpoqdrhqieywp, common/nls/gqjctxuruarzefrqyjrxkhbmqzpit, common/fvv/wfkzlxfzcrtqafysyepwjggxqqkef, common/gza/lkocnklsekzkcamepwqmanwfrkgwp, common/keg/xoouvlfwvathuinaoqbseykbrqmdm, common/fax/hwgzmxpmbdnzhdgktiptkqatgoxkw, common/xxr/amrgftmgaliiaytetqxyvheqmdteu, common/vxc/rnuehecjhuswyuouwmgipovstjvlw, common/bdn/dlexqhckrmaeudtouaanwpzrjrkaq, common/fer/lkkqgxqrlpxtcverdjhgcnkmojnzw, common/pdw/zlrchnyqumzjwvvlstnuovmwpyhnt, common/sor/ounueewbyjaisyelindozgbgexigc, common/psh/xrzeodtbgxczspfptacwkpcpwbjmv, common/cbj/wuwpdavvviqlgoiycgwlkushtrogr, common/lfn/vmzxxwmzrogojcckjitngczyzpkro, common/kvo/iexwulwmmtkoxeeogpiirvswzxamu, common/dzg/jblhejfgtqsyobzwtzmljagqhiexb, common/wvq/kxnwxuufealeckdlomveobdjgracb, common/vxf/dvqhaijrrzfvnvhlxkakovmoonhhx, common/ion/vaxlrffxtuptxtnywruolwfophdbo, common/usd/oqbosfghbeqyqmxvtcgiyvstelirv, common/djq/aumgpuilyzznjwxhpieoefhllhnjk, common/mha/hdspvmgzzjjxvlabwobftdtpsrwam, common/kao/mbzfgbmobgoljhijxaoyiohqzscpf, common/yif/qadipsflrulugfdjmynwydfdcmxsw, common/zqd/lxuuqtalygxrwvallexrvzhbhvpdu, common/xeg/ogzgtovxveczyhnjynquaxcuxovyl, common/can/bkcrxhbsusqmepfltorbjgmexnldc, common/qrq/knvfwyhyjvvdzmdvprqxeviyoiufb, common/tdf/brthulqlgeefnbryqkyedndofctpi, common/sgj/upegxxecyldsiljlnhvcijbgposrw, common/rll/kzxxnnoegpogobrhhfejnbtoognkb, common/cwi/zrovmswppewjooifwfirpxytrijyy, common/ros/vxnddowkkhbzpwrrfjsaomeqejtne, common/qww/geqmlufwelbdypurvkjkhzszaxbwf, common/czc/sujpaxaxmnzbcniuyeiebynhtdqft, common/ibo/zajmgsoimajoczropspabilmukoax, common/isf/gvvuvwvgjvaxpomabgrbhduiufnqo, common/mfv/zajdbwvwxfhgsednnysvltyjuisoo, common/qqf/xwdupdtueszkbhazigyhdckmabykt, common/tre/ciniwzxjecpjwbawnyzukhogkhvtm, common/huj/lwzhvkyrpmgpeyoewbresfsqpkqgw, common/aqb/vnrtibihqwuvuoccgrssjucnzwwsp, common/iza/fzxjkxleefafbrgwamfsfbilllhpz, common/mni/kqbfdetgyhxcaopfxtmbidumethej, common/nnn/nnirzlwvzlmuymbzicoiemddlroze, common/vyu/pvbrosgydmbnlyihjeypbjgccfuue, common/rxi/niguzhppebxiuxqudbdatsdqfzwql, common/yvx/uovcmhnlguaphzyxhkwetaxtpxard, common/hxk/iylwugxegfgqtzpaclvrfaykcdupu, common/bxi/jhctcomrlwmoplzjdypkliormmwkv, common/cgi/jbtljvseltfgfgfsykddknupsyoti, common/mww/gcoqoidxwjyocqennvnfzncecwzqg, common/zss/wgbgbuhmyoykikfsjkbklytmsbawk, common/wxn/qchpbrhgscgdpxxmsktmltkuwgwdg, common/jfm/ntstrrbdvnzpulnelgadvkhzrgsin, common/scw/opyzilylkcntkchzddjrodnbvzbox, common/tih/eaolceatdjfgsbluhylvqrtafpzoz, common/xec/ofjnqgvhswbwzplshhcvvuviwmato, common/oly/bxkkjwsoodcagdraawgxwutlvivog, common/hec/bpkuzbijvpbafclzbojazatbzzocq, common/iia/tasqnigtazgyalmgujeqjgtsbevay, common/vtf/bbzouzuttujqyqrxjkernzbvwuihc, common/dun/uesxwbkbeagwgzywbrfsldxvkcnrw, common/pmr/feajoowxbcpmxyqqltdotqwqvobfb, common/jwl/towwqhgymedvecyfqvryvxlkuutrh, common/str/qzscxnfbidqlphqfkplecbvqpryqm, common/fcl/rwuggpskhlgoimqueukilyxycuwdf, common/zrl/owlzkybptrtfeitopsdgsuhluxokd, common/ocy/vveqoahnajqidwvocnaggnifzsyec, common/uch/lbwnjlgnfpsmcswhaiatdjpekegsi, common/zqx/kzokjxhilprqbbpccwujmgketcczf, common/cxg/vahmskpkgzmzgnkydpjbksyjeaqjx, common/vow/fryzqwqwwjxnrnjtsxsetxbtybwnf, common/hsk/wylltvvbyzgkauxxtcevxeruyktnk, common/tly/fhyjivleocaiovdaqqzusrdbmbqpv, common/qpv/npwobjwdwrvhczcvxncgsvkiieceg, common/how/bnkqjenogbefupnbpyznjdvnafnen, common/ibj/bddursvfrbkxparkgzcppjtxxjchq, common/dif/quxdknwkjicqodbzrwuqrbohiufkt, common/xil/apwgqxmhvfooumzkxqckruhonzenz, common/rlh/zvjtpaputxutptwyuwnegpvrdnhvk, common/bhz/ijbblhcheiwznlrybulpgsvoictwd, common/xjs/ztejunhfejipsngadsywgazerwdnn, common/hjg/wtryblqlaudtkwbqonntavvgppicz, common/rss/rfqtbmheaxzjvvpnhprmwthsosygi, common/cgm/pubcwtaxdhmtwcpkmuubhxjxctmsq, common/dgh/msorgdnqyuvbarrptqayxsjhwmoyx, common/ert/sqrvzzcezvnbwioedvwihrgyfdevp, common/sfs/uisgjgxzeyxaprbmgtinrtrnwhdad, common/hbq/pfsnfogtpqqeogrwnlupbapqkiafy, common/dgy/pdlpgmnbpoaddoydxgbjuclzttnfr, common/qqh/xjbmlvqfcaknosollxtikfadjgjzv, common/diz/cbtyuoilhbnuekqxzyfxdaonfagwp, common/dya/bxgrpfydxcnlfrhjyrbciaenryamo, common/gja/nfbjgcyjutjvqsdomzhpfwnbathnj, common/vds/rvrzewjhytmdpossnaunglvonnjrs, common/rez/oarfmyljwbicpjdlapgrirpdgrxmi, common/fqv/rltpwfznmeoecrqsdkxpfwzgkywsa, common/zec/fhygbzdtvguuadfzclnnbqhxalemb, common/rsm/kmoyoinbinseyqvcvqcpljmchadpr, common/zyg/zvqxhrtwlvdimqqkfmhcijbuvojrq, common/bru/moqszfuxwqtuzhkiptwqeqgzivomo, common/ays/udsfbwtqaagxjfbrksfiufdpcwqjt, common/btf/dwovdtxfkrafiidmgwdrjturbsmda, common/kwc/nniznlkymxvadxjjbdmtuieqkgrro] were removed from S3 2024.10.22 02:10:16.559754 [ 1419 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_11_11_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:10:16.599214 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:16.602308 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.39 GiB. 2024.10.22 02:10:16.608274 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_804_804_0 to 202410_810_810_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:16.688541 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 89740 2024.10.22 02:10:16.713286 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1380_1380_0 (state Temporary) 2024.10.22 02:10:16.713557 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 680 2024.10.22 02:10:16.836399 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 13.79 MiB, will set to 1.60 GiB (RSS), difference: 5.09 MiB 2024.10.22 02:10:16.858430 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1380_1380_0 to 202410_1393_1393_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:16.950934 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90645 2024.10.22 02:10:17.096159 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c062629c616db63325966111e39ab917) : [0, 165]: Setting download as failed: File segment: [0, 165], key: c062629c616db63325966111e39ab917, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:17.140700 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:17.200267 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c062629c616db63325966111e39ab917): No space left in cache to reserve 166 bytes, will continue without cache download 2024.10.22 02:10:17.247320 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.39 GiB. 2024.10.22 02:10:17.427913 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_662_662_0 (state Temporary) 2024.10.22 02:10:17.803388 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_662_662_0 to 202410_672_672_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:17.818344 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_827_204} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_827_204 to 202410_1_827_204 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:17.828971 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_827_204} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_826_203, 202410_827_827_0] -> 202410_1_827_204 2024.10.22 02:10:17.865504 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:10:17.882464 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 680 2024.10.22 02:10:17.936567 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_1_506_234, 202410_507_746_92] 2024.10.22 02:10:17.941025 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_1_506_234, 202410_507_746_92] 2024.10.22 02:10:18.137103 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_1_506_234, 202410_507_746_92] 2024.10.22 02:10:18.169478 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_767_164, 202410_768_768_0] 2024.10.22 02:10:18.172172 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_767_164, 202410_768_768_0] 2024.10.22 02:10:18.198081 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_767_164, 202410_768_768_0] 2024.10.22 02:10:18.364403 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f4464e401a5e8318fec4148638b365fb) : [0, 92526]: Setting download as failed: File segment: [0, 92526], key: f4464e401a5e8318fec4148638b365fb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:18.366563 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f4464e401a5e8318fec4148638b365fb): No space left in cache to reserve 92527 bytes, will continue without cache download 2024.10.22 02:10:18.504610 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 14.57 MiB, will set to 1.60 GiB (RSS), difference: 8.54 MiB 2024.10.22 02:10:18.528499 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f4464e401a5e8318fec4148638b365fb, offset: 0, size: 0 2024.10.22 02:10:18.533395 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f4464e401a5e8318fec4148638b365fb for removal 2024.10.22 02:10:18.540419 [ 1476 ] {} CacheMetadata: Key f4464e401a5e8318fec4148638b365fb is removed from metadata 2024.10.22 02:10:18.541620 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c062629c616db63325966111e39ab917, offset: 0, size: 0 2024.10.22 02:10:18.543933 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c062629c616db63325966111e39ab917 for removal 2024.10.22 02:10:18.548018 [ 1476 ] {} CacheMetadata: Key c062629c616db63325966111e39ab917 is removed from metadata 2024.10.22 02:10:19.042854 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(be026635971499dff795106d1ff92433) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: be026635971499dff795106d1ff92433, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:10:19.058027 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: be026635971499dff795106d1ff92433, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1388, kind: Regular, unbound: 0 2024.10.22 02:10:19.062727 [ 1388 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:10:19.781505 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ab8e818af62df34381ee22f60ab1678f) : [0, 182]: Setting download as failed: File segment: [0, 182], key: ab8e818af62df34381ee22f60ab1678f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:19.788337 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ab8e818af62df34381ee22f60ab1678f): No space left in cache to reserve 183 bytes, will continue without cache download 2024.10.22 02:10:20.052862 [ 1237 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:10:20.199989 [ 1237 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:10:20.976689 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 90692 2024.10.22 02:10:21.228928 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1180893 2024.10.22 02:10:21.247889 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_747_0, 202410_748_748_0, 202410_749_749_0, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0] 2024.10.22 02:10:21.248906 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:10:21.252435 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:21.258256 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.39 GiB. 2024.10.22 02:10:21.261507 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:10:21.251185 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_747_0, 202410_748_748_0, 202410_749_749_0, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0] 2024.10.22 02:10:21.406637 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_747_0, 202410_748_748_0, 202410_749_749_0, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0] 2024.10.22 02:10:21.457297 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1381_1381_0 (state Temporary) 2024.10.22 02:10:21.535896 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(929d9b544d073508370a0f4bc361f256) : [0, 4916]: Setting download as failed: File segment: [0, 4916], key: 929d9b544d073508370a0f4bc361f256, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:21.539033 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(929d9b544d073508370a0f4bc361f256): No space left in cache to reserve 4917 bytes, will continue without cache download 2024.10.22 02:10:21.563976 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 929d9b544d073508370a0f4bc361f256, offset: 0, size: 0 2024.10.22 02:10:21.566167 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 929d9b544d073508370a0f4bc361f256 for removal 2024.10.22 02:10:21.570601 [ 1476 ] {} CacheMetadata: Key 929d9b544d073508370a0f4bc361f256 is removed from metadata 2024.10.22 02:10:21.671554 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ab8e818af62df34381ee22f60ab1678f, offset: 0, size: 0 2024.10.22 02:10:21.675663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ab8e818af62df34381ee22f60ab1678f for removal 2024.10.22 02:10:21.682791 [ 1476 ] {} CacheMetadata: Key ab8e818af62df34381ee22f60ab1678f is removed from metadata 2024.10.22 02:10:21.713283 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1381_1381_0 to 202410_1394_1394_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:21.741506 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 02:10:21.769381 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90692 2024.10.22 02:10:22.267792 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_164_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:10:22.358725 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:22.362347 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.39 GiB. 2024.10.22 02:10:22.456306 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 12459 2024.10.22 02:10:22.471787 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_784_784_0 (state Temporary) 2024.10.22 02:10:22.545746 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_784_784_0 to 202410_791_791_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:22.572833 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:22.578326 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.39 GiB. 2024.10.22 02:10:22.581782 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1180893 2024.10.22 02:10:22.627737 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_819_819_0 (state Temporary) 2024.10.22 02:10:22.691892 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_819_819_0 to 202410_832_832_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:22.728913 [ 1178 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_827_204 to 202410_828_828_0 2024.10.22 02:10:22.741471 [ 1178 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:22.748136 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12459 2024.10.22 02:10:22.863486 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.39 GiB. 2024.10.22 02:10:22.781102 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 44 entries to flush up to offset 6641 2024.10.22 02:10:22.998384 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_828_205} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_827_204 to 202410_828_828_0 into Compact with storage Full 2024.10.22 02:10:23.049231 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_828_205} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:23.069085 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_828_205} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_827_204, total 12491 rows starting from the beginning of the part 2024.10.22 02:10:23.267882 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_828_205} MergeTreeSequentialSource: Reading 2 marks from part 202410_828_828_0, total 12 rows starting from the beginning of the part 2024.10.22 02:10:23.763541 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_784_148/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:10:24.075720 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 18.71 MiB, will set to 1.61 GiB (RSS), difference: 1.21 MiB 2024.10.22 02:10:24.324450 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 81 entries to flush up to offset 89821 2024.10.22 02:10:24.464465 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:24.466783 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.39 GiB. 2024.10.22 02:10:24.609575 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_805_805_0 (state Temporary) 2024.10.22 02:10:24.664216 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_805_805_0 to 202410_811_811_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:24.728567 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_828_205} MergeTask::MergeProjectionsStage: Merge sorted 12503 rows, containing 12 columns (12 merged, 0 gathered) in 1.738702735 sec., 7190.993462145788 rows/sec., 1.12 MiB/sec. 2024.10.22 02:10:24.766183 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 89821 2024.10.22 02:10:24.766892 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_828_205} MergedBlockOutputStream: filled checksums 202410_1_828_205 (state Temporary) 2024.10.22 02:10:25.193010 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_828_205} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_828_205 to 202410_1_828_205 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:25.299286 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_828_205} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_827_204, 202410_828_828_0] -> 202410_1_828_205 2024.10.22 02:10:25.327142 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:10:25.386854 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 681 2024.10.22 02:10:25.582879 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:25.588899 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.39 GiB. 2024.10.22 02:10:25.659863 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_663_663_0 (state Temporary) 2024.10.22 02:10:25.745131 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_663_663_0 to 202410_673_673_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:25.771811 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 90751 2024.10.22 02:10:25.767536 [ 1192 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_668_136 to 202410_673_673_0 2024.10.22 02:10:25.801216 [ 1192 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:25.808042 [ 1192 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.39 GiB. 2024.10.22 02:10:25.814733 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 681 2024.10.22 02:10:25.975507 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_668_136 to 202410_673_673_0 into Compact with storage Full 2024.10.22 02:10:26.016274 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d430d4bef486602f916e4154b133e409) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d430d4bef486602f916e4154b133e409, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:10:26.028507 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: d430d4bef486602f916e4154b133e409, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:10:26.040505 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:10:26.070656 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:26.073610 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.39 GiB. 2024.10.22 02:10:26.240657 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1382_1382_0 (state Temporary) 2024.10.22 02:10:26.264202 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:26.321285 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_668_136, total 712 rows starting from the beginning of the part 2024.10.22 02:10:26.376758 [ 1201 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 02:10:26.468581 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_768_165, 202410_769_769_0] 2024.10.22 02:10:26.471124 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_768_165, 202410_769_769_0] 2024.10.22 02:10:26.504551 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1382_1382_0 to 202410_1395_1395_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:26.566497 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90751 2024.10.22 02:10:26.592983 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_768_165, 202410_769_769_0] 2024.10.22 02:10:26.604628 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_669_669_0, total 1 rows starting from the beginning of the part 2024.10.22 02:10:26.670348 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 16.72 MiB, will set to 1.60 GiB (RSS), difference: 6.42 MiB 2024.10.22 02:10:26.828108 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_670_670_0, total 1 rows starting from the beginning of the part 2024.10.22 02:10:26.931367 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_671_671_0, total 1 rows starting from the beginning of the part 2024.10.22 02:10:27.046822 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_672_672_0, total 1 rows starting from the beginning of the part 2024.10.22 02:10:27.154472 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_673_673_0, total 1 rows starting from the beginning of the part 2024.10.22 02:10:28.752484 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(55418e8492db2c3333e6eed3e221f440) : [0, 263]: Setting download as failed: File segment: [0, 263], key: 55418e8492db2c3333e6eed3e221f440, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:28.754807 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(55418e8492db2c3333e6eed3e221f440): No space left in cache to reserve 264 bytes, will continue without cache download 2024.10.22 02:10:29.404444 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} MergeTask::MergeProjectionsStage: Merge sorted 717 rows, containing 32 columns (32 merged, 0 gathered) in 3.438908763 sec., 208.49637178932042 rows/sec., 18.87 KiB/sec. 2024.10.22 02:10:29.460545 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} MergedBlockOutputStream: filled checksums 202410_1_673_137 (state Temporary) 2024.10.22 02:10:29.595685 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1182433 2024.10.22 02:10:29.800202 [ 1419 ] {} S3ObjectStorage: Objects with paths [common/hoo/uxpnajejzjeprvltbzjpvqiekkjdv, common/rkd/jqraittuxfhygqqiamkvqbhgyrjec, common/mvf/yawfxllxhfahzduzvxcpfpzyetvim, common/ews/ejrtixaulzrfqufikquspgcvjfdgs, common/mhx/gpquzwcudphacvdeqdvvnwilrlygn, common/vpc/dzhatsxsagueeajotkuhxsfvnjbro, common/pmo/gczyljkzdgdnxwfisxsqdqoolekcp, common/eqe/mqumiphbfwbapmticcjhhwpcrtgge, common/ime/yiujopjkjgsovxkobftevxvvefjpr, common/qko/tzkgrqiyyhvjaekcqhexubusawhnh, common/aeb/gqjtfmnwobygycuzdoavklhlbvslf, common/shv/whnvjfzmohxxjrymkiithyavvkgch, common/buy/umxxxojivcjlfbtczhdelmcmjxgaw, common/smd/bcqwunxxllqfwrlxifcxxprccgmwe, common/pfw/snhexyfkujrwzvruccpkkvzuecxhi, common/jvk/yrrzyfqjbewvtzflqookrjvnqqrks, common/vgv/vfxihgmcaedlbogqrwsjtjjlakkab, common/akl/bbuzfyxaylpknsjlyidamrndxbify, common/rfs/andxmhyhzhnbwvwketsuvhujikpzd, common/gqu/yaufbobhyiorkuhxackvzfvpxmbfd, common/idc/sdvycusbscwxvbofzlcethldrtjve, common/mlj/mduewenpuxeokqltuctmhruhktknb, common/uip/nhowvqvukrzakhoriodacbqbdmjbh, common/bsg/nvdotggzokmwgfaipygmpjdufupse, common/pyn/bnbtayglmomwxhbisyiwojnehhvkv, common/dfl/hgsydzlihxokkbrmyslryeznojdcc, common/bif/dltjazrtfgsdmiihydbczusbflixf, common/ond/czcybvphutisgcobirhudgnxrfsgo, common/lvi/critplzjnbswvdgjpolfrkwjtmucd, common/zkk/bzbndqqebkvpgbmmpxxpkygxwcqvn, common/cfy/glupterxjroyvjlcdrkoxtlrxiest, common/shj/ekqiwsuuprobikrfnykipssjclzpr, common/htp/wutrgvivatnbeqaxyskmwscslgocb, common/yib/jkgciyoibfxzmutqrxkmwiotuxrlc, common/hiz/aewnwtihoavfvpvljpmvyuxvzvkbn, common/jic/ifyotpocbuksyylnnbmxijragacdp, common/afa/tlxighsxnvdvlmfibkwziydumjwbl, common/ufj/wqzpzfzbsztryufqgfuawtwtnvsmn, common/xte/yeseujczjjyeqmbuvcuzibfbtbirc, common/mpc/xzajjsyjlqxuiryxkliejriixprfx, common/oqh/qmwjecjqhuyzrswmxkgilokpyqdja, common/ukr/ulrfjdqrhuzmkigbkvuqflavfurhi, common/def/feadodnyqomfebdrlzioytphqbmzr, common/czh/wtalfslprklnkzqbphttibwlzewnt, common/jkx/omgqsurmdxalynkbfpnzcbuddqzlr, common/cvl/qleopcqxzpziujrxgnseejsjltiju, common/dsr/ejkgoxibqsogjbumrfgugyjqpshpb, common/rrg/sbnijbsqrndcamixovyyvvmlcbqha, common/pvk/dfsrnuucflizeqjisffdshxrlcuxu, common/wec/zbsiwyczxhjzijdtfsjlvsfezpuyo, common/vee/ewybjkbnskknatggumolgepzhkffm, common/ukm/hormnpkourcvshfpgziajokcwrvuu, common/vwp/groytsvefogsapqufkkecbhshpcmg, common/dts/yadsnqwqjhhnwcokilumxvrukezot, common/uhi/yffpvezddzxaixakmciubkfynsgjn, common/qmr/pgaqrhikbkeemhqfvnvzqqgbpwnvi, common/ucv/atscvhyxfxjnaqpgzzuksylrcqjbs, common/szq/zhchrocdjfhfsykycoojaynjrbwyd, common/exc/knfrdsckolhhvexhlrwyazthghygm, common/kvi/ziglgjjgizqsqybxdwyfmylqygphj, common/pmp/dybmektpjxprbmdkvyssllujsykjd, common/vjj/mwgiswknhidbpvvgyjiviswdtwcqw, common/gel/syolmghzggwgwyrcuruwsbrlovapg, common/mub/wryjvetwhrozhijwgubhoyzaoinjf, common/fbm/djbhtcknitxudnnsqgedxoiugkctf, common/urf/nadjhnqpkvgbktscnqdejbaxstruk, common/hna/fhgziqncaffvmmkrfplvjoijuabzp, common/tmb/cloxgcsiccuosstoqhohcshkmvahp, common/ynl/tzatzoplqdanupgrnacvwnmvorukx, common/hol/kpzrldswaopdvrwkyabyyyxsvrahk, common/ach/lbrtfdkqpudmscqffpdfjsnotskzm, common/eee/szlbdxlbucuxwxiqfrfvnsfuwurxz, common/ohz/rleotlrjeffrxzajmsdcvovhscfkf, common/jik/pmwwszizeldjetqgddjhkigmdpyyu, common/vex/btyjvvcfnyircjbapctodshshgpgr, common/lua/rjfapxggosextbzjcqsupibacjgfu, common/nba/rreicqjpjabsinuqzxnwnqhieulyq, common/jry/xzxrryyompmmomkqfieoamcbmvzlc, common/itj/ssexhgdelgyxnsefpjvbugwdqktaj, common/tup/kkkawpkahpcvvwaqbexyvjyohlbnm, common/hqd/eqhmmwnfkhtuholqotxotrwuyhzza, common/jhn/qzlqqmcyfbqadohcahwesqgwgbunl, common/ukv/aenmpgrcfwzmmjjnpvrpgezmxsebu, common/bbr/momvvwvghotfdffdmepczhzavmgts, common/pwg/unoxulqpedhcvoaoxnmdnxoewpbml, common/ncf/lpzdyhhegsfjitnrgifitrjvtvqxq, common/rdd/phikxnydjjrcckvrzgdycggqmblwy, common/uet/gbcomzukhtojcjwzvxbrbcpeigrae, common/iar/kjkrlgsiicnhjaltevefyyuppajll, common/cap/penyjmulhyhxclinhtzagxabaykdw, common/lpd/bragwjagzsmdpcfkuzqzgawsemugj, common/utc/uhpbbbydnfmiqcgfsqodmlvxwiexc, common/xil/mrgsepgisnxjcpcoxghvpijdktzde, common/rgu/tzuohrfnbdgbdttzqzdovnfmklrbu, common/qlk/lmqyuemwvlibvqydgonimlzsshjgr, common/yij/mfraidlhyqgcrqewjehjoeqldusep, common/mgs/sgpekxujyajazpwequxrlfrcbtjur, common/nqx/stnizplluunuvexhogafvcvlcczkm, common/oor/xhmxdxnoriuskpvsszhnmaeoupjbe, common/kyv/noiijozavatqmysbakrpgczgkgjul, common/tjv/fkavfeyzwkmvigrcsoezjnnatgvch, common/dke/zkrlnwmuqvlluzwhtcxwimykklclf, common/wym/lsrqplwqvmcqfcrhqsseppynvzdpc, common/jab/qaqpbizwtprklbxhwglbvreiawdhk, common/ixe/wtygqcbbhmyvlnmzvjnjmiljbxzvl, common/fsm/zknjvwrpnpxykaogbjsltvmyavvbi, common/sff/sewrkhrxsmpkhnruwyalaolljyzpi, common/mxo/gkoorabrfveoasxvtmjnwpqbohdlp, common/ljh/peaoaflnfdhcsevodhnhflkipunlw, common/hra/uwnbzlqmwpbclsrzxrykghlbujzcv, common/kki/vciscipfctkpyyjbyebyyjhnaaojx, common/dws/hzujkqsajrwdhrkennebxvmitnmdk, common/qkq/zacvhsnekvmyjgnfujdjstcabkqhx, common/zkj/pvoaztizinqfwkalmdqotsjvaqqsl, common/rts/muttiuaededjmijsfefkxyjlosycf, common/vbu/hpbomudprxlchlrcavgdrqyljbdoz, common/xpk/fxssufjjybepsomrzmxbasbryunuh, common/cuo/gvxaiwsghzvynjcnkszbjtabizywu, common/xcj/blcqgsjbqzgzprhmxszsswgxkosiz, common/peq/mtoptlscojuexepezeqmtbkwvcnhr, common/pqs/cvrkxbbkdkqqhskxjookhbrogylhk, common/veg/eezfenrpoockwhrmfpssbfjdnroqp, common/zcs/amesenfrnpxgvxwvuddswosldthym, common/css/kjiuqdjuiusxyhdkwalkpixbbtskj, common/arw/vmlplfqcmrehszxhqsedqakicwiom, common/ojt/oqrzzglerwzsfsuowjsmfwbmabvpo, common/ihl/fxknvwyuehdjicbnsrvlfrxlrjidv, common/sez/cexvxkrnrktibrsikwaiememfzqnu, common/rwf/prvjhjxyxwixhnalfogxhbfgjfhpi, common/umw/glpioyhqpswutuuwjimrdateobazp, common/yor/plblwbgxhcgvikzyfwytneyidngid, common/lnb/umzvhutbtiwthkuogrmnhrfutgvwp, common/ysi/kexcttrgepuvevttaimsvqwzkkwwg, common/maq/dfzfzygwibybblntfxgybqgmugzmu, common/dmt/lhljgfuwecbtvovywipxqaorvcwqi, common/stt/qcdnfkaxkbxzjtqtndwkrdvhqzstl, common/ypn/idemfokhrtvetlbnhxwemkkvxddqa, common/wqq/dkcuylgtkswsvusnbmfnyqlozzqcs, common/ocd/fdbvpjowjvcqcjyftaatzkbpdlkqv, common/sqr/xuwxjzgmtofsaqjuhpfzlhjqqtlce, common/nsy/bcwvxyfjnzktjeoqoruiaoyyqqhgz, common/nvn/tvmtblmbemvjslwppzkcwpioyvchh, common/uhz/dkyhumfdtexvxqpmfazxjllxscmtd, common/mqj/jvgknquwacmqpsyqgwgnjgonmajpp, common/jav/tvuwqzwkrpoubstnltfsembmmochl, common/yia/vzsodelcguofnsipagrfwzrzonnrj, common/umx/nvxkcccxwqeoxremmxiyujegsvtml, common/qkd/xoiqcxpjknkyywrnxnyplhpfuqjiq, common/xwa/lvnuzteqtvztmgkialmhseemcyuvl, common/aee/ltcgrafzgwuhhyoboesizkupxyyad, common/dtx/lolvgalaarreubpwgjfhpaofttrcq, common/zuo/cuvzpplbucoqtjzlpdbxmsvgwxviq, common/rjq/rmthtbecwzacydvgavrunkkrvcaqo, common/jtc/oqtcyipneaoirfixgoxsnxltcbenr, common/sma/etsxlwsbhozjxusyqbpmnvfabnnpr, common/yff/uaguvqvdsutplmztbpxqxyuqsodxz, common/yip/hsyqlwfoamkyvcepvfvuvunbfvpcs, common/rzx/ahqkylfpxbwaeawsdtpnpidlvpjkl, common/dcw/nrynrtbsxxubraiawlhoejqvpegjy, common/kdf/bnrhvqnbmxjhitcsbzggersossaqq, common/uwf/avnhkfywoninoqeffmmmfijkndfma, common/tkh/dxkpmkosazjylulolyhkqgqtozzjl, common/lua/hxdmgghlczerhpslokguzzpwrtxau, common/xwy/eszkpxigngfydqbgyprcbbgbgiieo, common/ray/cyuwlqowuobstcocwqkxcwwlqdzdg, common/ngk/zieyfjcsbshweooohgiztsaomnkdd, common/ozp/irhyrsfzuhwbjpbzhwpurlndxhzjf, common/hal/mxrpqzrvqglbjbgmpfckmbeaavyoj, common/jim/yttnmsqllfsotuxtwiyweptkgfhgg, common/jaz/zggiawhsuyvjcpkquoppvdrnibsoy, common/cvm/ztjmoaxbvfvafrolabzcewiwbsnal, common/nzu/khexxpzdndkcwvmxfnupkwwqwwbmw, common/jvd/bomwmwirxrpddwfkrgjeahrolpxqc, common/awt/rtxdlosfxgdnqvdswjtpykdhfoxps, common/vuh/zqwgrjfhzakcqkyrajnkxbiyxysae, common/cdd/qjxrvzrwgxaktoybisqudxvleiduu, common/obr/uwmnunlyqjpsulvkxkkuyjufagfbk, common/xuc/ptuofyooiuhgiixtgjwuvwpmdkysl, common/arp/qvddqbcypkyvpnihzvcaavnngwdtg, common/emz/griqhutltochvnkjsxwicdgtllgxr, common/okh/usjnsjnkdaxelssfzvcgwfnjopwgu, common/fzd/zfjytorhfduwiqcefgdgqfpqwauap, common/ylo/kwuvcomjxushzxpwxjjotdyefwsij, common/ufq/xkiveaxekwbzwuothpmhxoyebzvia, common/pwj/nzrduvnzjcghdpokjynxrtejgoxnl, common/wld/qoyoaisjntxwhavbhjauhwnlkefnk, common/uyo/mikyjxwdhmiairmmllmdnybvxpsle, common/gek/iksiuksoffcmdvofjfmuatsrxajip, common/exo/jaondhytxexnlkwjbvpyzxwrdjypt, common/duw/hekgrwzarmtippesgqilcwzsggtyf, common/zrv/bqzxhceksnzaxwctxwcgczzlxosqp, common/rkx/gihpnlcuhploonwgmmfckcclginvk, common/zzs/kcvbnbhwfoltfjbwcrjjnmdqcffzf, common/gbk/bcxxjgqaobpsosihjelcvjxcyeuwm, common/igq/fqjvfgeeaibutznuummoecdwurmmt, common/afx/keofmxdwujmmgerjzeorksnqkfzry, common/zmm/txqzkzzlbimyrukmqobnnyhponiib, common/rho/ymlegggpdmwwdnodcrmdyipmtdkdo, common/eqi/fbbiulrlrnqkucjnditodtpgxzzsa, common/zld/cioogdtjbidlzyqqicjdsetwodblr, common/gfv/vcgxpfbnmmevordjwikiqzsdhkgws, common/vyx/byzhcuzjtgwgbnmcijgablgpnefou, common/duw/bpyosgvzrajzbolktdlogngfppgrv, common/zut/lnwwbsgrlagifqswyiayvrgwkrtse, common/ujf/yhuvfhgdzpapetbvugqnfracgfgmx, common/lpu/fsatsjyqdepftgempykrrkooramrb, common/ztw/ocdbwdxrzmjdlannwgvpdtrbdshld, common/vvd/fuiaqzqaskldogskgubawfkqoyilh, common/ixo/iyflugjwrmdwflzojejcqoykukwsj, common/nov/wkvmxfhlfepcxiyujsxlghwbkwqje, common/xvv/izqnibvwezzxadbppdkdxjpbsgvcd, common/fct/gkhdjksagcltcyldsdisnsfrwwbtv, common/din/ecwilkcqsuihhjyzyydyafgdwacym, common/jqj/ltpixswsiakwyflqvqsxnnnfpioqt, common/ynb/asfssmqsbayofofvcyfqvdkekumpz, common/qxb/eyeastagefeffusynnmgpjzxbbvsf, common/upq/jmynijjnxnxteixashgoxqjoafzcm, common/yto/bljetrhtiqynlwldlkqvueycllafh, common/dbu/ahjqpclpyvklosjgfuhkxbuzrxjrs, common/oqo/ynkdzsqlyduvbovoognmurlppdpxq, common/cnp/huibcxbkyxzqgubkgsqaifyjsyege, common/ssx/xqfhyjtujyhsrddfslvwypvlzoigr, common/bhq/iaaxrjapdyiyaghnidkixhbnfafew, common/qah/zkpgcdirtauvjfuxnhaqsovqummsc, common/zqa/vnvunpzycusorrikdelyuqesonicl, common/uoh/oxkslqyprixewxjyyhqjzeaknotmb, common/jod/qpyjecrgtogfkdxfeoqbmrxxbobmk, common/qbq/xvstkjysohifdpjduvnnfodkhmjzu, common/tlq/qkgincaqohjkdslgtodfpozvcptpz, common/bxy/yfvzibtaahtywgxuojhmeivkcbkas, common/bcw/dpmbokgfvdwvxbbgliowhozubfclu, common/aup/wkswofkxwqbsfqgtxqidavjctetjj, common/vkh/xsmfftslomgbuipsuagjkazkbudtj, common/uof/nchryeuwdtelrghvrjrilqrpyjliu, common/afl/zziykacfchbawlrxhyxmckpdmrzae, common/ena/vgmwyfehncplanvfxptxpqhlfqsif, common/dvo/wshtzdmletjytdiglmaooidvtoorx, common/mej/rwpdkvqlrtprpsvigrqfzrtekxzme, common/oaa/cwxkdszyixknmdstuqyefdbbyducc, common/srq/odqgxndmlssdicrapavlamiliamge, common/qkw/dqjrykxbivxqoqiwslqfhcpyxtqyq, common/agh/jmisvdrvvgndhejbvwehpmombzhud, common/klc/jjywxcdlygxxvskvyilvwovynoecs, common/xti/yygfvrrvmxnmbbpnnolworqnccdfe, common/drx/ydunoamqzebyajzpaijzjvbqttlrt, common/yiv/cbuhfymmbqovrieymsouqytjgatpc, common/mun/kclowyqopdleshaoiatoycktbdigm, common/fzt/nyzlhfndbahircpdmxtaihhndodtj, common/thp/errxlgfwqnjfgqmflkescgmzrgpst, common/bfe/kbbnlzytgtghdynelnpdvlyvplckw, common/pnv/hyigpdyecuwmfoeuqkqyhrbsppuhd, common/pxa/qvjwgwalqlznrlancujtvodvhrzee, common/pmu/aaelrvvyvwsbqcsqwasokiflxsnoy, common/eyt/rhtfkucmufuxvunzjrliagjjpxwny, common/dpb/ejqytnfzbymynuailuycjuuhjqkzx, common/wkm/qcheeszithgvqgvzcsyrsyndfuxaz, common/dsq/seothgncndnuvwkejugpbximyxmbk, common/xzv/fjwleerbwkpfmccogtjvnvoiyygzs, common/pli/xrkhfbyjrmjqfrdvxqfrvvtzqtheg, common/jun/sriapwnqvnmrtgmuleklmuowtpcrw, common/hgf/hxyibjayieuetngxaadbwywvvitzd, common/zch/yphuoyzpklqwcmtxvfzzjcookxepg, common/iwe/orhoxogoekxcxmrgfcyrmtmxsxguf, common/srm/lkjmnenipgbayadrdvrqyxfwynpoj, common/toz/ovjuyqievkqxnpkproipqpfuudiot, common/djk/nlfuixjnmcdfejgenkjveerqxuowg, common/rmi/lnrzkrpfruulhtrwwqoloifdshhve, common/nte/sbjkysqrxtajsfoiadtaolkatzbex, common/pss/eilblhdkjwcpqjcnhtjppcsuhlhql, common/ggm/jkffftwvkxipjhxwbpcnesemghlvx, common/ewr/onpuosewadkmctvslohxevyteaxzi, common/jkh/qmpxzarwhnjtlamsspqwftvroqnah, common/wuf/nxvzwjiyomcrfycxkktmlmigokijg, common/wkw/mmxedvkxuzfuqghjukuvqlkcisdij, common/lpc/onwfbusriidfqzmmwopyunpqlxmpz, common/pox/rpyngapswrpxndamwsrjjutcamfqb, common/tfg/qkztuzngqxyiczluebzxfgbzrjgcg, common/uta/lryamauwwsdfbiyrkwwumflpryrev, common/gqp/zikgffqydieqoytlrjtdgkprlzuds, common/duc/kbynbfwsonbbjpcvjblzvlounljwm, common/uqi/lquqfyuxmkdqnkjlbmxxbaelvqesp, common/vol/fxvgxdzwvflslzaudkbufojqjwqvj, common/upr/qyxsyifxxgeonedurqrhqbxjlesrh, common/ksd/nkqyncmpxevbbvcfsgkahhjagzyrq, common/bdv/twrkhrntpkxxojgytkcjsishmyqhw, common/zrp/ezjdtvrzbidztfpittflutsbijmnr, common/dtr/xvxbbeuvsgllmisycunqvjyblahij, common/bpd/mqdgyfefdpsodoahsbqpkvcvstkaw, common/bpb/iyhczzynuvsahsqbajjtybrvvpuon, common/ipq/bzfuaoubqpjhifqychtronwqsccxx, common/faj/fzgjddjlejowjchazyqejlhzuoeyz, common/uvc/zrxzhsmavfpbjxwtzewpyiacircza, common/mvp/oeqrljhqjuaqitqmbvjpdpvxegalu, common/yss/gvufqvgcyofrmiurwappaqokeipuz, common/ipf/cnwydetywrptwfwxhkezosauprvct, common/ocm/uvfrnwgxyzhfmbffmucwlivnhjflu, common/ihu/hpokhoxvlecwmoteywowqbdrvzmwa, common/hdi/rpgkdbvmuwhubtpfsnbcqvwcdlweg, common/bxp/dwajeyrbsgfwupomdzklolkhtnvom, common/aiz/cyvzghglxnbnyrmzfjyyhqbcmncnz, common/mnr/qwluimkgdyoxcfaivfqywtfstobtl, common/ubf/osasxbspjlcuoksdiwinketnqxdhv, common/qyh/dgutbhycriazdlumhiryttifululd, common/qdh/eisrfdwlqofxmzxxrqxdglaueszoe, common/tkk/qdscwzouzjmvdlyyttptcelsoscvn, common/zjv/cdkqxmnphoiwdyebmgdzdfyetftlm, common/mjh/vqntbvwoabuegerkueomqzurkvotd, common/auj/vgvmoxmcmwzpcuvsbdsxtycmzahqa, common/ihq/hmsfudoeygnphnqdoaaozlvjdoryd, common/iam/ctdgjuvzkuupnivpnadcqaraozdji, common/qqw/cxosrdaskcrwypdtaulrxxedxcspo, common/bdv/uxeefeibtslwqpgcqqghbqskjxsch, common/xni/uumtfrqxecmiqgopsrgqdbvrpjvgw, common/fes/hzgwybltwownhzjjznsdxyxfwhcbb, common/zfw/wbdkgcmtqayzcjjrbaeoccsofdcpn, common/wrg/grxswfmbnbxadqolbpzixdkjiorcn, common/aaj/xpjwttpemekmpguewyjnmjaupzxkt, common/hnp/ypdxeuvokyixzycdfdloizbsjpzjv, common/wlb/pmwucprvoprfldolovcewqsedoqmq, common/eol/omviamjmjyalgqngbicbmdhxxfiiy, common/sna/kalihbakwtosefoixjlfxyxvgpkrf, common/mhl/zuayodzqyauqioeutqersoczngbsh, common/jse/aevvviayyzadfmbivumyalshajdjw, common/lyi/znhprgonfakckjozqxpirhscvovzz, common/lvh/rxaqbrxsezhdwtavxgsxkuicpyqvb, common/vih/loqeqjdlooesjazalyljdfsuqyfyw, common/vvi/ormubcajshrxpaozuubgjnmtyzrpr, common/opi/fiifqxwxgzcxtdivnkjglrcyvkymn, common/pjp/qqnovlavpgyopxewcuyvwkpqhapco, common/iou/jotirzdktxxbwujjdrpejhrredlmy, common/bau/ozxonedqufsuzrscsbmrimtavgmhv, common/zvl/wwzwnnrlehsboaqkoqyehefernwhy, common/igg/upumrxxjantdwgkiywjrumahnhfhg, common/vfc/wzirftadnuqvcwtmsfqylhvwxdimm, common/jup/rysqtkfajryiblgaptoixkrrbigat, common/fcp/lmlovbemyzoymjayydekijslovzsq, common/ygn/qygwecqbdyfjvapypmypgslxnbeew, common/gbm/bghsxduymdwbstvaiczwymiyhskaw, common/iyo/byfvrrlndmmgjsbxonztfofayphkq, common/yys/regrxzputmjdbuhjvdxuocpjbrqlc, common/loc/qkvqkyatiltbspbesnvtafbeqheor, common/ezt/srdpxauulbmyqftffianceybkuxul, common/ssw/vxwkuvjuqqnbqtifbfduvedbewrsd, common/ybo/pmwnykamdcgvhqovnqaqouxwbnwli, common/hbd/tekmzwmfasyqikjtktdarnhesrdeu, common/kra/ylklytnptjnlziyhugwukiveyahnr, common/qsl/orywrmhtjodolmkgcqgvgbcfpwarw, common/vsj/cztqoiaxrjtffnelzuprgfguzsnwg, common/hyk/vwjeedcxugxipidflifvjxvylonpc, common/ohz/ezktpfcgyvolbocrjmdxfwdqpbbdd, common/ejr/vgzufqvlxmytgtepuroocvoyictrb, common/dft/uqsuqgnqyjiwhcvdyomujxegovlyu, common/rpx/ihdhdhkcsqlgoedsnjcmathiwjuek, common/ehb/zwodhevvochlplbocihlefrroimrq, common/ipb/unzqiochjfwiojoehhyeuegfuqaif, common/hih/uqmudvspxulydpguqjurmbaootcju, common/dee/pztghrpeobzbptzfnhdbyynmlgnth, common/pov/uflvbfabefsdpicvfxxnhggdivpow, common/mun/smjlvpcenbhrwljvcuqqjuocmmudu, common/yea/ckaywwgvtguglvzonslximtbmmrlk, common/xtx/ytbghrwqooftqaqljolejhipualbk, common/mwv/bzsvitgszlwmlmuqompqktpnwfopc, common/dav/uixsjawlkcqqgvhbkxobgqkolojfp] were removed from S3 2024.10.22 02:10:29.952504 [ 1419 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_12_12_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:10:30.070506 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(54bafe0cff62d02fb9d30d1de4f59175) : [0, 2146313]: Setting download as failed: File segment: [0, 2146313], key: 54bafe0cff62d02fb9d30d1de4f59175, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:30.072055 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(54bafe0cff62d02fb9d30d1de4f59175): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:10:30.227856 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_673_137/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:10:30.253560 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 12467 2024.10.22 02:10:30.446316 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 54bafe0cff62d02fb9d30d1de4f59175, offset: 0, size: 0 2024.10.22 02:10:30.451039 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 54bafe0cff62d02fb9d30d1de4f59175 for removal 2024.10.22 02:10:30.456063 [ 1476 ] {} CacheMetadata: Key 54bafe0cff62d02fb9d30d1de4f59175 is removed from metadata 2024.10.22 02:10:30.470437 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 55418e8492db2c3333e6eed3e221f440, offset: 0, size: 0 2024.10.22 02:10:30.471463 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 55418e8492db2c3333e6eed3e221f440 for removal 2024.10.22 02:10:30.474463 [ 1476 ] {} CacheMetadata: Key 55418e8492db2c3333e6eed3e221f440 is removed from metadata 2024.10.22 02:10:30.588629 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 90794 2024.10.22 02:10:30.980969 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:30.987461 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 02:10:31.045050 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:31.071769 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:31.095430 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 02:10:31.066700 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_673_137 to 202410_1_673_137 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:31.119585 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_673_137} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_668_136, 202410_673_673_0] -> 202410_1_673_137 2024.10.22 02:10:31.167180 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_820_820_0 (state Temporary) 2024.10.22 02:10:31.175344 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 02:10:31.177421 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:10:31.218133 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_820_820_0 to 202410_833_833_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:31.251591 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:10:31.253567 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:10:31.255427 [ 1233 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_828_205 to 202410_829_829_0 2024.10.22 02:10:31.281825 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12467 2024.10.22 02:10:31.293513 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1383_1383_0 (state Temporary) 2024.10.22 02:10:31.368565 [ 1233 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:31.371641 [ 1233 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:31.411468 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_829_206} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_828_205 to 202410_829_829_0 into Compact with storage Full 2024.10.22 02:10:31.430204 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_785_785_0 (state Temporary) 2024.10.22 02:10:31.463480 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(41b0d3a72d8a8cb8c56779ed2329869e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 41b0d3a72d8a8cb8c56779ed2329869e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:10:31.467891 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 903, file segment info: File segment: [0, 33554431], key: 41b0d3a72d8a8cb8c56779ed2329869e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1387, kind: Regular, unbound: 0 2024.10.22 02:10:31.470708 [ 1387 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:10:31.548782 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_829_206} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:31.557328 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_829_206} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_828_205, total 12503 rows starting from the beginning of the part 2024.10.22 02:10:31.563573 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1383_1383_0 to 202410_1396_1396_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:31.636850 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_785_785_0 to 202410_792_792_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:31.684303 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_829_206} MergeTreeSequentialSource: Reading 2 marks from part 202410_829_829_0, total 23 rows starting from the beginning of the part 2024.10.22 02:10:31.684904 [ 1733 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nvs/gcovvqfodomvdpbaqmlvudxijguwy, size 4220766 2024.10.22 02:10:31.712217 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1182433 2024.10.22 02:10:31.951626 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90794 2024.10.22 02:10:32.254094 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(755b8671224672191a0bea6769790f0b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 755b8671224672191a0bea6769790f0b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 02:10:32.256091 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 883, file segment info: File segment: [0, 33554431], key: 755b8671224672191a0bea6769790f0b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1392, kind: Regular, unbound: 0 2024.10.22 02:10:32.257743 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:10:32.281696 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 72 entries to flush up to offset 89893 2024.10.22 02:10:32.355834 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_17_17_0, total 294912 rows starting from the beginning of the part, column RequestTry 2024.10.22 02:10:32.365834 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_18_18_0, total 270336 rows starting from the beginning of the part, column RequestTry 2024.10.22 02:10:32.386914 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_19_19_0, total 286720 rows starting from the beginning of the part, column RequestTry 2024.10.22 02:10:32.531969 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 34 marks from part 201403_20_20_0, total 270336 rows starting from the beginning of the part, column RequestTry 2024.10.22 02:10:32.556798 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_21_21_0, total 286720 rows starting from the beginning of the part, column RequestTry 2024.10.22 02:10:32.589805 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 71 marks from part 201403_22_22_0, total 573440 rows starting from the beginning of the part, column RequestTry 2024.10.22 02:10:32.628078 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 37 marks from part 201403_23_23_0, total 294912 rows starting from the beginning of the part, column RequestTry 2024.10.22 02:10:32.761662 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTreeSequentialSource: Reading 50 marks from part 201403_24_24_0, total 395178 rows starting from the beginning of the part, column RequestTry 2024.10.22 02:10:32.761808 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:32.766744 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:32.840305 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_806_806_0 (state Temporary) 2024.10.22 02:10:32.855871 [ 1398 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vch/uzrpekbtmdwesjzhggatimtimlkxo (LogSeriesLimiter: on interval from 2024-10-22 02:10:04 to 2024-10-22 02:10:32 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:10:32.933375 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_806_806_0 to 202410_812_812_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:32.957821 [ 1255 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_807_12 to 202410_812_812_0 2024.10.22 02:10:32.967952 [ 1255 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:32.972544 [ 1255 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:33.019864 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 89893 2024.10.22 02:10:33.031134 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsLink 2024.10.22 02:10:33.050006 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_807_12 to 202410_812_812_0 into Compact with storage Full 2024.10.22 02:10:33.086829 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:33.099558 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_807_12, total 6047 rows starting from the beginning of the part 2024.10.22 02:10:33.155885 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsLink 2024.10.22 02:10:33.170155 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsLink 2024.10.22 02:10:33.201120 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_808_808_0, total 96 rows starting from the beginning of the part 2024.10.22 02:10:33.212930 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsLink 2024.10.22 02:10:33.242944 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsLink 2024.10.22 02:10:33.250003 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_829_206} MergeTask::MergeProjectionsStage: Merge sorted 12526 rows, containing 12 columns (12 merged, 0 gathered) in 1.850783979 sec., 6767.94274325194 rows/sec., 1.05 MiB/sec. 2024.10.22 02:10:33.266108 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsLink 2024.10.22 02:10:33.297518 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsLink 2024.10.22 02:10:33.332275 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 682 2024.10.22 02:10:33.332339 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_829_206} MergedBlockOutputStream: filled checksums 202410_1_829_206 (state Temporary) 2024.10.22 02:10:33.362989 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_809_809_0, total 92 rows starting from the beginning of the part 2024.10.22 02:10:33.365989 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_164_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:10:33.402640 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_810_810_0, total 150 rows starting from the beginning of the part 2024.10.22 02:10:33.432591 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(56030acd323a092060c8974369a449df) : [0, 184]: Setting download as failed: File segment: [0, 184], key: 56030acd323a092060c8974369a449df, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:33.439524 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(56030acd323a092060c8974369a449df): No space left in cache to reserve 185 bytes, will continue without cache download 2024.10.22 02:10:33.556797 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_811_811_0, total 81 rows starting from the beginning of the part 2024.10.22 02:10:33.603044 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_812_812_0, total 72 rows starting from the beginning of the part 2024.10.22 02:10:33.884891 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:33.918168 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:34.003999 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_664_664_0 (state Temporary) 2024.10.22 02:10:34.116506 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_664_664_0 to 202410_674_674_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:34.225585 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 682 2024.10.22 02:10:34.199320 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_829_206} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_829_206 to 202410_1_829_206 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:34.241492 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_829_206} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_828_205, 202410_829_829_0] -> 202410_1_829_206 2024.10.22 02:10:34.303211 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:10:34.557819 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c8515fa42d350024b9756235900cfbbf) : [0, 72851]: Setting download as failed: File segment: [0, 72851], key: c8515fa42d350024b9756235900cfbbf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:34.561261 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c8515fa42d350024b9756235900cfbbf): No space left in cache to reserve 72852 bytes, will continue without cache download 2024.10.22 02:10:34.818085 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c8515fa42d350024b9756235900cfbbf, offset: 0, size: 0 2024.10.22 02:10:34.819572 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c8515fa42d350024b9756235900cfbbf for removal 2024.10.22 02:10:34.827971 [ 1476 ] {} CacheMetadata: Key c8515fa42d350024b9756235900cfbbf is removed from metadata 2024.10.22 02:10:34.829382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 56030acd323a092060c8974369a449df, offset: 0, size: 0 2024.10.22 02:10:34.831766 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 56030acd323a092060c8974369a449df for removal 2024.10.22 02:10:34.848706 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 14.15 MiB, will set to 1.60 GiB (RSS), difference: -5.59 MiB 2024.10.22 02:10:34.936213 [ 1476 ] {} CacheMetadata: Key 56030acd323a092060c8974369a449df is removed from metadata 2024.10.22 02:10:35.021168 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} MergeTask::MergeProjectionsStage: Merge sorted 6538 rows, containing 13 columns (13 merged, 0 gathered) in 1.976178146 sec., 3308.40618455053 rows/sec., 1.43 MiB/sec. 2024.10.22 02:10:35.205717 [ 1226 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:10:35.248873 [ 1226 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:10:35.249089 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_784_148/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:10:35.223357 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_731_731_0, 202410_732_732_0, 202410_733_733_0, 202410_734_734_0, 202410_735_735_0, 202410_736_736_0] 2024.10.22 02:10:35.261616 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_731_731_0, 202410_732_732_0, 202410_733_733_0, 202410_734_734_0, 202410_735_735_0, 202410_736_736_0] 2024.10.22 02:10:35.317477 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_731_731_0, 202410_732_732_0, 202410_733_733_0, 202410_734_734_0, 202410_735_735_0, 202410_736_736_0] 2024.10.22 02:10:35.317386 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} MergedBlockOutputStream: filled checksums 202410_747_812_13 (state Temporary) 2024.10.22 02:10:35.371210 [ 1259 ] {} DatabaseCatalog: Execution took 401 ms. 2024.10.22 02:10:35.645751 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_769_166, 202410_770_770_0] 2024.10.22 02:10:35.649493 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_769_166, 202410_770_770_0] 2024.10.22 02:10:35.659646 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_769_166, 202410_770_770_0] 2024.10.22 02:10:35.887761 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_812_13 to 202410_747_812_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:35.895238 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_812_13} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_807_12, 202410_812_812_0] -> 202410_747_812_13 2024.10.22 02:10:36.015204 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 104 entries to flush up to offset 90898 2024.10.22 02:10:36.069184 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.29 MiB. 2024.10.22 02:10:36.240875 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:36.247443 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:36.593533 [ 1244 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 02:10:36.597291 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_784_148} MergeTask::MergeProjectionsStage: Merge sorted 1184069 rows, containing 4 columns (4 merged, 0 gathered) in 39.635450706 sec., 29873.98853574172 rows/sec., 447.99 KiB/sec. 2024.10.22 02:10:36.639912 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1384_1384_0 (state Temporary) 2024.10.22 02:10:36.741430 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1292_17, 202410_1293_1293_0, 202410_1294_1294_0, 202410_1295_1295_0, 202410_1296_1296_0, 202410_1297_1297_0] 2024.10.22 02:10:36.743490 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1292_17, 202410_1293_1293_0, 202410_1294_1294_0, 202410_1295_1295_0, 202410_1296_1296_0, 202410_1297_1297_0] 2024.10.22 02:10:36.765905 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1292_17, 202410_1293_1293_0, 202410_1294_1294_0, 202410_1295_1295_0, 202410_1296_1296_0, 202410_1297_1297_0] 2024.10.22 02:10:36.768723 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 13.95 MiB, will set to 1.60 GiB (RSS), difference: 4.32 MiB 2024.10.22 02:10:36.813626 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1384_1384_0 to 202410_1397_1397_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:36.852324 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_784_148} MergedBlockOutputStream: filled checksums 202410_1_784_148 (state Temporary) 2024.10.22 02:10:36.963736 [ 1205 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1392_37 to 202410_1397_1397_0 2024.10.22 02:10:36.975410 [ 1205 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:36.984658 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90898 2024.10.22 02:10:36.987815 [ 1205 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:37.144325 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1392_37 to 202410_1397_1397_0 into Compact with storage Full 2024.10.22 02:10:37.193438 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:37.203208 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1392_37, total 11423 rows starting from the beginning of the part 2024.10.22 02:10:37.231526 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_1393_1393_0, total 81 rows starting from the beginning of the part 2024.10.22 02:10:37.285512 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_1394_1394_0, total 47 rows starting from the beginning of the part 2024.10.22 02:10:37.431185 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_1395_1395_0, total 59 rows starting from the beginning of the part 2024.10.22 02:10:37.475068 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_1396_1396_0, total 43 rows starting from the beginning of the part 2024.10.22 02:10:37.516632 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_1397_1397_0, total 104 rows starting from the beginning of the part 2024.10.22 02:10:38.581785 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 9.84 MiB, will set to 1.60 GiB (RSS), difference: 2.99 MiB 2024.10.22 02:10:38.727871 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1183588 2024.10.22 02:10:38.792083 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 12490 2024.10.22 02:10:39.007907 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:39.013614 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:39.053232 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_821_821_0 (state Temporary) 2024.10.22 02:10:39.132070 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_821_821_0 to 202410_834_834_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:39.214617 [ 1219 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_829_206 to 202410_830_830_0 2024.10.22 02:10:39.228639 [ 1219 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:39.236543 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:39.257075 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12490 2024.10.22 02:10:39.274846 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_830_207} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_829_206 to 202410_830_830_0 into Compact with storage Full 2024.10.22 02:10:39.368364 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_830_207} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:39.377344 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_830_207} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_829_206, total 12526 rows starting from the beginning of the part 2024.10.22 02:10:39.425041 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_830_207} MergeTreeSequentialSource: Reading 2 marks from part 202410_830_830_0, total 7 rows starting from the beginning of the part 2024.10.22 02:10:39.597661 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:39.603655 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:39.714942 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_786_786_0 (state Temporary) 2024.10.22 02:10:39.808924 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_786_786_0 to 202410_793_793_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:39.892452 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} MergeTask::MergeProjectionsStage: Merge sorted 11757 rows, containing 13 columns (13 merged, 0 gathered) in 2.755515673 sec., 4266.714980140126 rows/sec., 838.21 KiB/sec. 2024.10.22 02:10:39.954618 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1183588 2024.10.22 02:10:40.097394 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} MergedBlockOutputStream: filled checksums 202410_1207_1397_38 (state Temporary) 2024.10.22 02:10:40.388492 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 6.05 MiB, will set to 1.60 GiB (RSS), difference: -2.48 MiB 2024.10.22 02:10:40.427381 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1397_38 to 202410_1207_1397_38 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:40.433214 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1397_38} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1392_37, 202410_1397_1397_0] -> 202410_1207_1397_38 2024.10.22 02:10:40.434578 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_830_207} MergeTask::MergeProjectionsStage: Merge sorted 12533 rows, containing 12 columns (12 merged, 0 gathered) in 1.185448237 sec., 10572.372212317863 rows/sec., 1.65 MiB/sec. 2024.10.22 02:10:40.492915 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_830_207} MergedBlockOutputStream: filled checksums 202410_1_830_207 (state Temporary) 2024.10.22 02:10:40.532610 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 128 entries to flush up to offset 90021 2024.10.22 02:10:40.568577 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 7.69 MiB. 2024.10.22 02:10:40.638885 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_784_148} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_784_148 to 202410_1_784_148 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:40.645508 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_784_148} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_778_147, 202410_779_784_1] -> 202410_1_784_148 2024.10.22 02:10:40.697729 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_830_207} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_830_207 to 202410_1_830_207 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:40.704596 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_830_207} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_829_206, 202410_830_830_0] -> 202410_1_830_207 2024.10.22 02:10:40.712665 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:40.717594 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:40.763636 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:10:40.848662 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:10:41.009030 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 90961 2024.10.22 02:10:41.068357 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_807_807_0 (state Temporary) 2024.10.22 02:10:41.150345 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_807_807_0 to 202410_813_813_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:41.250183 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:10:41.252167 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:10:41.288694 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90021 2024.10.22 02:10:41.375943 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:41.381633 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:41.653164 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1385_1385_0 (state Temporary) 2024.10.22 02:10:41.679750 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1385_1385_0 to 202410_1398_1398_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:41.731085 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 683 2024.10.22 02:10:41.777632 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 90961 2024.10.22 02:10:41.875934 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bfcc2642645a8f046b11b7640c42356f) : [0, 228]: Setting download as failed: File segment: [0, 228], key: bfcc2642645a8f046b11b7640c42356f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:41.877953 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bfcc2642645a8f046b11b7640c42356f): No space left in cache to reserve 229 bytes, will continue without cache download 2024.10.22 02:10:42.103967 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:42.108788 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:42.172884 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_665_665_0 (state Temporary) 2024.10.22 02:10:42.313394 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:42.317308 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:10:42.667916 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_665_665_0 to 202410_675_675_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:42.761418 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 4.58 MiB, will set to 1.59 GiB (RSS), difference: 6.72 MiB 2024.10.22 02:10:42.788605 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 683 2024.10.22 02:10:42.796114 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(220456326750451f39b41e8d6dbb5070) : [0, 52219]: Setting download as failed: File segment: [0, 52219], key: 220456326750451f39b41e8d6dbb5070, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:42.799964 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(220456326750451f39b41e8d6dbb5070): No space left in cache to reserve 52220 bytes, will continue without cache download 2024.10.22 02:10:42.872169 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 220456326750451f39b41e8d6dbb5070, offset: 0, size: 0 2024.10.22 02:10:42.875994 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 220456326750451f39b41e8d6dbb5070 for removal 2024.10.22 02:10:42.882759 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bfcc2642645a8f046b11b7640c42356f, offset: 0, size: 0 2024.10.22 02:10:42.883914 [ 1476 ] {} CacheMetadata: Key 220456326750451f39b41e8d6dbb5070 is removed from metadata 2024.10.22 02:10:42.888782 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bfcc2642645a8f046b11b7640c42356f for removal 2024.10.22 02:10:42.893719 [ 1476 ] {} CacheMetadata: Key bfcc2642645a8f046b11b7640c42356f is removed from metadata 2024.10.22 02:10:43.382383 [ 1419 ] {} S3ObjectStorage: Objects with paths [common/tve/xrsguitofxtqbgmlnjtzsukmujqjc, common/krw/fbkfohdkrpjlrnggmxejgrxnjbxgv, common/nsh/dzkjpjmqwaeekicltiuprswlzndmk, common/sos/vabdpkpmerpikjzbdvtkvjfkcnzkr, common/dcp/mhlmogfydgplndabtdogiwplinydk, common/gev/oeobjsqdpdroribrkxmviraoajzmb, common/sju/wdfpnznhqfrctkvoncohtacrwdylg, common/xny/kpvqgqrcppqprwzovuuccfqyohnin, common/rkb/pxqxfwukfxvuhslcrrkujeihhspxl, common/fku/cuouldillzkrbapspjcztjugutzdd, common/tyy/czgxzkqyqxjkwsrraiiadozgmqnne, common/hld/bhzqtwwixkvlasienuxqpxbdfpyxv, common/bsj/nlhufgjqzxtdhxcgsbdwjduyxpjgd, common/ifn/ppwrutmetxpighnrnjneobqnnhuei, common/zxr/iooiytezcunualeqevrxdqzmsfbvn, common/juv/uhvejybrlpqyqpmzxlcxdkribbcwl, common/ykh/qhdkkqaacpfloeahojrjzfybmhgsx, common/rxq/qrennroozhupaarijqpkjczwwodzw, common/fmz/ezjpzmjcdeeyymilorczzrfmufvoa, common/ybs/dlsvvkovsihwcpsvwcjpimyhndvkx, common/kls/sbyyytqdcmsebjshvatstwgjvfonz, common/dkm/lyskxmbxwpjhrigemshyadbladoiw, common/fnp/uqtzhdqeasuyzxrdhlusyrhfseyul, common/tva/pvdmktvpsgzpjrhoetujgdkvwvgdd, common/ckm/wcgheqauxdeitvsupiswokuersikz, common/vff/usflvqrcsjxjetllailtikxuwjgeg, common/yjo/rvbtnlkagurxdhyrtvtorweijgkaw, common/iws/krdfslxjdtlwabbqotfpntdgdukyo, common/vvl/bmtwerysodmrwuupqexhhuzjebzdm, common/ocq/fkcjrujphvdywtpdmmtmrvksgtqxy, common/bbu/davbosyhrryalpvjnhrsdahkgkavj, common/afi/zngztebxgzlbvlshsfxwnmrdoafkv, common/kei/ufdrgjsrgoxvnabtqkfhoilfsuqie, common/qvr/pwlwapwwgjtspfzkezqlghqvcuqno, common/iqx/hkgxladmenqunawybvlopgasseqqa, common/kgl/qkkpzgjxhzrktlygysfvtdqxxcevb, common/uzh/xnrcucpfcvqtroizpwqcxabijrduo, common/jpu/vhvledjdgcdyfattiwufnrkhurooy, common/pjk/hwogarxngwifjmclfmgkqlxxrzsct, common/zuz/pkfkhtzlfdzwhqbfjrdyhmobfsrvo, common/fjl/zqzlseqkvtrqlipmilsghnwdtgrod, common/fgy/krpmtfjchobfnmhhyqiezhwahsyer, common/yod/gzlatixldxfgayfaxdzowdloayggu, common/acd/feqsmxilyxyhpvrwmpmynbfndoztm, common/olo/iozlsygnecupocqwkeoacqtdsewcf, common/zql/urbjylpcnkfbwmjsrsuwflqhkqznd, common/tqc/odhkxrbkisvdjxlmhtacqidtdazki, common/mlg/yzvssmlxjlzyntorzpsqghdebfzpx, common/soh/kzjthhcqopxqxbebsdqbfxdmroosc, common/iqf/vmbccpuinrvdgpevlcbgyhrkhsgjw, common/nqm/wlwhibesgsqvkkuvesdcaxaabwtpy, common/rwr/dfuzwtdqnahonoqtqngfqgfthdsid, common/vkh/njrqmpkdcsorcraaxlsadmpwdhvae, common/ixe/hxugtvmhewriimerhoqgqzgmoinwc, common/fje/ghcjdnwwtqvvrfyryizyuimepzrrb, common/jgt/kcygmqolljyqamdhbjccxafhiqyxw, common/xqp/fqrngutzukheutcjkjqjubzuhhkmb, common/qyu/qaklznwfqtuzedmclityysniwytha, common/bzn/zhqgxqtbexkgwunazgntnqpajmapl, common/hns/pajktzgbzrfrdbsnaddikpmkzswlu, common/xxj/fpwrvlsunvdqkbemrehiclgulbsjq, common/ohi/fnncoxatqxljweqflschnvsmjcsdy, common/wbt/sjeobzdjsbhnnhiovzrhlqwxdhzih, common/yks/xnnmifcrhdvkwthimjmfrgjlqvnkn, common/koi/xybzqwydszeipajdoykbzzvkboyrr, common/kwa/npjjomvnrshjxyjjzbtnxnuyiutgx, common/qav/hjvnxgmhkuanhsjtnajviazwfkpzc, common/elh/jxbadktrfxzcbgoslwnipymssbwad, common/kln/plgkwndpleybyzmqxqeqbibgmtgvv, common/oiw/rilrachwlcxjwiypxvcmmsmikvvdy, common/wns/mnoyrwudzkokopyrufgkbhdxscqys, common/axt/ycdiydbyhxkivfbomrewndhepqdww, common/djt/zxeghbphbbkbhyrupujswqwgzfwkp, common/grj/dkeufowsnlzlgwgrfarjzxwuftgob, common/won/mwqgyebylzhqucqkwqtkcyyyrmqnw, common/oxm/silxnfryokjgivlrrfzcgmbjmjcxj, common/flj/oajpxktggkiuaigokqtxmetayrusa, common/bfs/bzjkwoigdelonrvvykxvmudvxsngh, common/xph/hmlqttnmjiijihjbzvzqyuotzckpk, common/gtv/uszuervvmrgtmpmlesjisknnrjqsl, common/poo/jmddcugbeidjzourpcylxlqltlfvl, common/vuc/dfetwpnczdiuvfhuluqrnhfkycwma, common/fpz/pepeqfcbaistgfrlxmzhvmenqzmnr, common/gaf/biorybmapeoecanvvokgqldzdtgqy, common/kwf/rcifmooyvoebddlmcjecdssmunhhc, common/zng/bgfxiavttjthojtuwttjrxbwxeuxu, common/hko/woekcxwcpaoloboszwbryepitrynd, common/nri/dvnuoxvtbnbpccndviykvcglktvxh, common/zqv/lhwrksufafxlufiulaodhpstrrnsm, common/hei/tgzbbqwimoacnsuxwddxvpfimbsby, common/dry/gsnvltdslrscwovkzfbtcubrgtdly, common/fen/ewjwfcdfginhazqxtpoahmtkjkjid, common/ipf/fsxbqftyffjoylswclkuozyoymrxb, common/inw/pwoipvupcsjqukknuamxzletcoual, common/dau/jasqenwvuxeufbfdlyiyfaccwlegl, common/psh/ynwaslxqrvzzlltmbjzwaxiolvzqi, common/qvc/ewqlokuiledgakcocinrdamxougfo, common/ssf/msrlhqjznmwleufmbyefqfvzjrtpp, common/qzx/hzbwomoivozlardqpfzrhonoxapnd, common/nyg/tdlfbnsqytwvticowyzvlybfjflvf, common/dxp/wwtvibbqkxjskaxijdztrsauzabbm, common/whu/wmjtwcqrvbmaktmtspraatpfupqvn, common/zck/jxhgadtkukzvloorinxvvvoobekhb, common/ogb/ydjtsbmqfzsgttvzmfcasqgmgwwig, common/xea/udiyqrmhvmvexoabbakfztwadpfjv, common/zwt/matlzuklbwlpvbqjyfnxfdxqtmhfe, common/aov/onpsoizkvvljxboawyemnvibllpoi, common/kqg/fowvpjxtkiqomyuixvnysmlwditua, common/yai/vhiturvutdofgxzwsuepryeijtkea, common/dql/owewoqjqynjcnhtqslvamckurlatw, common/qwx/maqkqsatnrpbanozqtfytcdyjwqit, common/cvy/ieaibnbxbizjfnwuorcyutrpaqjoh, common/aon/fyjrreiogfrycumlyfsqyqotaplrj, common/fnj/kbjxrowsibyfempabnuxfmuuhwycg, common/ixi/bfjaftjuumqpukzfxogolkkrhfdwo, common/nsy/cbwnaaolyihnrhdxxiqtxsalxscvh, common/sat/rbrxfrthxxysrzfqjqtcjnxbgvumf, common/hbz/pfdkswoyhsdpdcnqojkatuzbxmqra, common/lqa/jjntqophvlqugjyimskhqytnngjna, common/epm/rtmvnzhguankcngnvwcjprshudvxj, common/fdn/vcopparyvkpuslboalecubayuytvp, common/hvq/kbwqxflubuqovqeqjlnhkngdfhddb, common/zkg/cbtiotiqwhhcvojluntoazrbkweid, common/avt/asjqufbyluvxuzrvqsimhnyfhgkiz, common/ewe/bxsbvgldcdjfiqaeoyukzhfjiuaod, common/inc/chyztveflioyqhecrmsnwjkhogkhv, common/tjj/fdqbqoggkxqqbkdkiyfauvlfurswa, common/dgv/iwfyiherhzywdneyhjcanwlqyacqh, common/ece/terelxihmkjiwtmyzblirtdejcunv, common/kgi/fcrwofkpergncwbsvcmcpwxkjuder, common/gnl/cblkqpjlgstpnsgtwgsiikonzeyiv, common/osq/iyanfoxofkjwwugjwobcjjtcznzwe, common/tnd/yclofplphabxxtdzxrsgyitqkyssp, common/xes/hcgzkdeszyihuqxnjaaipodaejtux, common/kvx/vgwnnmbqbedpftbvwzolhhfihbbas, common/bwf/flihtmfwrxxtejzivfwixtoloplzx, common/ycq/lcfbaggeygirfxoomgupoewituppa, common/qfl/ifklmvntgjtrjtayfgwkpwicqrcqu, common/xey/xzonehkkqmfiuellurfckoihquubs, common/nnd/hbiantxyejojakufberlgcazxvjaf, common/lga/sfnbdvzrslikkvhlwtmoazfoufdwf, common/zfh/jsxrdpzrnfhczzvzzsvqqtecthyif, common/dek/alweqhogzihkbnmpnajfgrnqxwbwf, common/syp/vognrutgmzuxrzziwfpjhftkkbrqk, common/yxf/rcgazeayblvtbnpjqofoucxwarkji, common/hnd/mgrmxktiybyzhdimeictavfhfbznf, common/jtm/fvlkhbntkamuaegqcuxcpseqmvtnq, common/agu/qfocejdqvrtshymrfhfroyjylqgqm, common/ffl/ftirufghqsuimdscpnbtuydciopnq, common/hzw/bnccidwreruqaeablspnuifpbgcsq, common/iwe/yajhxhfvdnbcjwlkpbzxjzijfcdzs, common/wde/hghpezvblsnraxjrvwoseiqofaxfr, common/oql/rgiyzhaxwexgwftopjbqcakaaexfd, common/egz/ijdvymemfypjtpxrlyaqnlddxroyx, common/lpx/fkpjjvnnzrtnvtvieyxaadmfmqufv, common/ifi/qiomkgflvjlbapazpivmymuprodpm, common/ckn/dmugvwkmgzbrodfyfodzwylmifyes, common/aoe/kwrrfnpdziqtjbgpevwndwjyauekp, common/qtx/yftgjoaqorbouwniquixdhaohzruj, common/xjj/cemfvvvcswvotihrrxwzymeedpqni, common/kyy/tlgjpgkkguuedsebjzeeptqotjswp, common/fgx/ogrizfxpfbpevqoctdmtjjgsnuitj, common/bdm/vmhaycuamvhxfircniqpxsmkdmchn, common/hcu/bcwxpsjqtccacjjajqwwmphrzmceb, common/yam/icvbgvuzdhwmigsqbptpgukyjiqfd, common/ukv/zsnqpgfmzfzxdsoopbowsdjfotrli, common/umc/lnsvpccfzyspokhvbcnwmszeuyqii, common/zam/xjvydjmdzrizkkntoavjqgjpicmqw, common/dtl/oschxngokhtrwiigvpvoavcyalymm, common/ebp/ortkravuokzuaoyuxcqzlikyjhaqi, common/mfv/xlyhyfiuukhsoqjfkdvzhcdnflsma, common/pyq/gfelhzrhyzdxdirhvswwbdcnnkwih, common/lqh/gjgjegeujxbyesbgxrihdluvcbpbk, common/nab/jzulgrtopsxlxrmxwojggwuyxrrao, common/kgs/ntooggtokfwxqdzigbyxxywtunurf, common/nzo/tjunrangyfakhxnlzopxjzazlkvsn, common/gjm/zlfmimhcxojsrypkysbvxkcaantyh, common/hts/ikqtrxdnypzujouitlaoygwjmwdwf, common/fzf/sotblhtqotiumbpicdkdmkjynuzbr, common/nrc/pqvqpwqwihooteaxitbdrazwpshml, common/kit/yvdjrqhdmswpgdzcnkpzukoqukzdr, common/jmh/gtahjaxjyqyrwkngpggkpfdqabnvy, common/mpr/xxahlgpsedibesgjcickbbejmfzqq, common/qdx/kduuluihinzghuifyxfkeybyvdtry, common/wrw/mmiltipkdyqyzzdslzgsguqdaxhow, common/zmf/yxfpefzdblpohwydeqrpstuelwgas, common/fdd/awzgpliyraswiiymmwrpseorrwvaw, common/ygp/ujzknmujcdaoyabekvphvnlpfjygw, common/hrq/twtbjtgpniwqsnsiknnhzxqdwhhbm, common/jce/yofqbpbrhntmyctkeeqinbczvfhgo, common/sbc/ntfymgszyyqnadkyfxejtrtwyrhci, common/mzb/jcwegfwduregkooyekegzowzorqnp, common/sbr/cclwkdcqsfgrpkwkbflowagifozxg, common/ven/keaadgeejuzodozfulibdkyoqzslo, common/gxi/xbhlmiwwbjaturyvukmxrzopjbdpv, common/ast/riurddflidunctbkapbllenucaxeq, common/uzb/mvfhvvrwfgnetrpotqaoxnqmoqbyj, common/brn/cskblijirstiedultibjtkifhsfvt, common/utv/ogxfukripzlkgluahunbknmxxwfsu, common/kxj/obyyqaaacdkyaneipijqnaeckdlzz, common/ipk/qlwoedtfvymvplqywefabavurmrho, common/ccv/dmgtxhdsmoholefbtseluyeshxorb, common/vvs/bfvonpskbxxcjxaorleieleihlbrs, common/ets/gynfuxxqlrvioetmmslfcpzwnostk, common/ybc/rarvlhmvncopvnaowkmwmjlxgfemi, common/tzj/dmomwnmrnguusvgpoknsmprlmnsks, common/wky/hmgufiladwydyxhnuejbcprdafcfj, common/tcx/ezpwfxvvtncbtqawoozdifijdaqkd, common/kqj/zmbsdsoyiuiugqgwohytaapeaajxn, common/kqr/nfqaaufekfixphnplxkxqicpnxmkk, common/jyu/zmoueahrrevlkjtgtywrwdfefjmbm, common/gqf/mkyfpwenntkbmqgqzinfftfstggdc, common/vnt/prrixcizxejuauhiwabaywbiudlne, common/yja/zbwgxaeszzgbduaouqdctiyikgwoj, common/qgm/ucywkukwtlhwcxedktvsghijvoepe, common/jao/xibqsippofxizsebspjdvhlgahbjt, common/bcz/fponuhhkjrggzypxkltqputjwyyjm, common/vtd/lcufatopnuyhkajoiwjsszabpkmdd, common/ivr/dmlasrswhxfrcnqphkhapyftjaelm, common/eeg/kguvjjobejnamymnqgutihyknoyat, common/zok/csibbuamkjuozimhbmqdgfdettzok, common/pqq/wciloqdstsrjkqfovtitfakmqevxj, common/lga/spyjkhftyoemucydifpoewcfepjie, common/ert/nsoibxmjhydoxvkmrzoxvrnkxmdsf, common/nfi/rdqprgmiuristnsahfcxjpzlkmndl, common/mih/xywaafjzmrhdujabshpiysoptxqox, common/zvp/ujolxreaboubaulxrygzyonmxxyei, common/xos/yvajtgakzjonyabcltdbtumpyjddh, common/yet/bnjoiufmazyngxsdovenjggbgouso, common/rcs/khfadkdpwxgjrscibhdmcydxqxfvp, common/uso/tjdbpjdibmltjnjwelflexnsbdlvh, common/dmd/gqhlqoncxzwhocdpgpwujmwbegxve, common/yyk/bzrseihpcfvyjxronahzrdvqhlhba, common/oit/htqltizvflhapebhkqmtekuirquhf, common/ufa/poiyhtlquovcgklqpykexiewwxfnx, common/lpj/vnmgyfstdfceolwyzmbrawqliakvl, common/kdi/rjcyorftjxnpstfepanjbanvirsat, common/wtb/fmwmbpjcicwqlyjdjnwpetzvlprmr, common/bqi/wirdxnjlklaeeefvshiuvcpiswcar, common/iiz/iwgtxexoyzylgscklgtmxnzpgddtk, common/mbn/qigavgaqtbhzxeqeulfvihzntqouz, common/efi/pvvmhnetymhhlczxjddptsbhxcjpp, common/cnc/gaaxrmtlzsbsmfsozrzcipnrwhozo, common/usf/wwjztpwnlreoxgxihdeabnmlkuhly, common/jip/ktmnetabsisrxupbgzxolqaivjifh, common/nqt/tioojskdtnmbcrmhdcjdmrabfpqkw, common/mum/iandgtidvtexjnlwquadblfjbtrhk, common/qju/axxxbcimprazzrnuzgexegpykjsit, common/noy/tmszxzymsseypiesrgpzxtpxmzqrv, common/olb/xbajxhqfvxyhkyjvvhqqfhrqjgznx, common/umn/mrsykcybmevpnylonpkwkzlciakyc, common/odb/ycsajwbzwhjadlehxwcpfcwbgryxa, common/roe/zbebqxyxjsdodsnsxdclaihziyikq, common/zkw/nlgupklqciyugircspqcbtesgpgjh, common/ydf/tcklqcctbglgpomeewrtqrdnesrec, common/bxq/xeryyrchsqgpuvaxkbzadlkgdxxlt, common/gsd/vsglwxvobviptvejshpkzvxpfovgz, common/jem/dyfvjasqoaaiezzutucdeextlroih, common/dbw/xvldioforeeomhduldkthbgmlhgmx, common/ppa/ptvchiyuqouepasbfmydtlrltrzqf, common/lqz/sbgcohxpacqlbangrolyjqdsoxioz, common/jjx/cryitjzwdtzhwjjyxjpzxubibeuoc, common/fvg/dgtcaheezsabcqbkaljllxwxbgdlw, common/aua/xgpfeqspecsocwtcdfhpiiixdxbvq, common/frh/anrwzxpipdxhyzhebrmganlwedsww, common/bxi/uauyuibtmvwkdwdhjhcnzvrzuieqp, common/krh/thwxnulhfzqxptqqdxbyabndtfvqx, common/nnv/kwqfxxyornaoakbukokvraigfvjhn, common/hop/owxfttwcbhtugzzwbdwvjdslozarx, common/pvw/szsyiaqtofztrhbqkgxfkqtdkcqsw, common/oty/ulriubkwotweyonwtrmmgnkhnmckw, common/tto/wwichyhkvhjdklzgwqqimyzwbgknu, common/vhw/fnxmnnadwfihhjlnwzlqwdyxvluov, common/ubt/hmzitfjfssudhvlvkzwdxviomzgsl, common/bor/lhlgabsbybudytyzlczfyuibyktqe, common/ebk/nwcbotasnzmjdbamjlclaeqyhywlr, common/kim/iyzoglktzlvcynmvdymnaewlroxpw, common/azl/aoytryzzrhqfenvciyhfzukiuxrsa, common/etx/kyemhngngjyfmfwgpigbyptdmouex, common/sap/tkekwxjfiuhynvhzymshhnpqkyfhj, common/ftw/uwmsyqqvgyfpnlgabgumucluoddge, common/hoa/jeeorehavicwungmevqpdpfhjhctm, common/yjc/shsmahxeylyevdjduzugazryhwnxb, common/mcg/ytajkngbycqronwzrovbdjdptvvls, common/yri/ilkulsnmozdmhmuumkppqvywlftrn, common/fvy/jskfibgpocmjeofqptowxbtdtlokx, common/ycg/gofbrkupovjplxyuyvytlnpfmyosl, common/ajl/lbntavelqokouqfmksvybxngigawu, common/ifz/ocknalptxackkycijfqaltugsujkh, common/ibi/kidjclrvooemkkglkmuidzpujhojs, common/vee/irnrelirfhjcjtgskfgrayeuqzryl, common/vmq/vsyfzakyqsxfspyjbcjxbrtacmvpm, common/fso/krbtnjodlwbcbxmjyeoihdqzxtbrg, common/uho/sqicdswkyuzmwkhcoxgrrsyefunrd, common/bac/tttbjzxytmnzojcbsmczmqmpumwad, common/xpg/brjpysqadurzaczdiyukwqdnhyphj, common/beq/tvuhlzdrjjwnqfjmcqgoppripgwue, common/thp/zfmlguppqzyxdvrlwwmwwmlganvnc, common/bqj/fannnndhqvtnxzzhosfwsvgzpvlhu, common/xqc/spkxkcsyxiypjkwpmpzwbvgfelcnp, common/aks/rutqnsjhhzxmtdsjqfmtyfesvgjaq, common/ckt/qrjstaccuprvwiivakcsncpyytvyx, common/nnv/syuhesilqxettekzxbhiqwxjboliu, common/nqq/nbnskdnnfdjadizgxqlomtguxnigp, common/oob/tefmhejtwsdymbhlsehvlpbgvjieq, common/fyw/mklajisclgtifvdbzxijvlmnnoakg, common/tke/szvemunjgfsbcoghkraofcuoxejwb, common/bhh/hclskhhgkgfjkpjvevkxnrtthhjzq, common/lgq/ajtjboiqekkyvxzweniabiltdwnut, common/zji/gljncammlmfrzfswqjmsehdprlqeg, common/dot/zkhqxscbbtewhyhbvkxaksxiqmpgp, common/zrm/znozeudxbimqdjmgqfarxoixinylq, common/jai/aplgazjjhqcmnslfxkuoknkqlfboi, common/euj/ymomfgqpskvruryahpbshdvbeaqax, common/ors/nnhugfmqrimzejhqalkczqwswrfdy, common/aic/rneeacgyeakappyhcvaplgeautyks, common/dvu/ncbzycnpscksffzzsonmlewwnvfht, common/pgz/xwufomusixqivxhtbzhygeiqddsuz, common/xls/bnvjleiooegnjsxggobhefsjbtchj, common/sev/krnzzdmgfzyzpavjyliwfrmmgobmv, common/spr/ujifuzdzaqfzxqkmezpjefmbgupco, common/wau/eashkttytbaudacabekbyarutqbym, common/czl/haisdzmgmtmnjjdwrumjeytkyrdwl, common/hta/uxqxkjvaoadmchzkauoitvnoacssc, common/xol/xtmamoxegnggzmjkmmhbfhrotrqga, common/ycf/sfgixvgiccqzwospetdeamncgcrpq, common/oyr/doaoiocojoywggtbugptasbkfnpaw, common/okn/qsssvolgpcnmzhpexhexkzhlxdage, common/tpo/ckcwjwihgtbsqfoactwrqecoywwgp, common/lhf/uvwgamgzxisumbkoiwixafldmmghn, common/xgb/vnfuxjlmpvepwvxpgfpevpngidscx, common/lwl/vmmsiwqoxjjcerxjryaohqhbponbn, common/hfc/nknwwvvhdjneiyzayjlankjllrawg, common/tul/uetqjxdzhiejuznvidnjtnoapfdrt, common/edl/xnjcuwmwiswzvdjjriwvzrywjyyst, common/qih/yxmcdyovyjorlajxelwedugmkwhcg, common/lzp/qncilmmhezwwykcnvellcbtzleffw, common/tuq/gjdnhgmjnhdmxxqhjvkrnkvkopdhn, common/lmp/vkzemagptaabpgcvggzqrzrcfkzbu, common/lkl/gzypqxgflzwyqpnauberqjbbpexli, common/yzp/rahqklpsmfkndcwcdlwrbwhqirrpv, common/eqt/vljtjwpwjvwqldwuccbhnoimxxnwq, common/vbp/prvjorsnqiqpjtgskjclbenpdelms, common/tvq/szbsggqayogiswhagajlvihwlphra, common/zut/gklpcjdsstxxdtojpxaoyzochiuht, common/wfr/hzidcnipymnqhiafsrzntzcynynmo, common/ivz/jptuhbpfpiwpbmujtziwtbydbsxvq, common/uww/bmelfpxfywkasdngrqbjrfuajszrm, common/jxi/bgkjmokbddlmroabpmyzemjrcmuqy, common/hjn/sjmvoaorsuffwvdbpmipiqtsaoabj, common/zqb/uxnwysbnaddyzmmphnqdypeehagpe, common/gpd/abmxxjtnspjfeqvhiqwmlkrrmilst, common/sjk/pouggosretgqvkfvlqpjknmrbgtnr, common/cws/apkeoiicvnskimetgjhdekblbcvsz, common/jlk/jvibgypcbcbibfxzehqxjdazxstzn, common/vxu/nxwnhdelcphpcfjseajlhqpbcejbi, common/pbv/weikfaqadxstrtgcaluvxxjezlwql, common/mbp/ktjwjojtddlevdbkqfujlpfyodklz, common/dui/vpdwelikpivpyoczvduwdqykmbnsb, common/udb/azjubljgagcplffbzvjzmhlxacsxm, common/uah/xopsyqarvuvcbrydagwpgmvakscmw, common/ipt/mtoflqwxltywenhwinzptoeimlmvj, common/jqu/tbweuordmjairmirxtwvpjmrgfswu] were removed from S3 2024.10.22 02:10:43.390872 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_770_167, 202410_771_771_0] 2024.10.22 02:10:43.394843 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_770_167, 202410_771_771_0] 2024.10.22 02:10:43.405451 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_770_167, 202410_771_771_0] 2024.10.22 02:10:43.436170 [ 1419 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_13_13_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:10:44.052165 [ 1186 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 02:10:44.207241 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8c4c7be67cc880445c89fe784aa75752) : [0, 242]: Setting download as failed: File segment: [0, 242], key: 8c4c7be67cc880445c89fe784aa75752, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:44.208694 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8c4c7be67cc880445c89fe784aa75752): No space left in cache to reserve 243 bytes, will continue without cache download 2024.10.22 02:10:44.901438 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 4.25 MiB, will set to 1.59 GiB (RSS), difference: -3.96 MiB 2024.10.22 02:10:45.249132 [ 1183 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 311 ms. 2024.10.22 02:10:45.641816 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d6aea94440c9c2bf1417d5f9ce0bcd33) : [0, 565533]: Setting download as failed: File segment: [0, 565533], key: d6aea94440c9c2bf1417d5f9ce0bcd33, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:45.643448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d6aea94440c9c2bf1417d5f9ce0bcd33): No space left in cache to reserve 565534 bytes, will continue without cache download 2024.10.22 02:10:45.809211 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 91003 2024.10.22 02:10:45.810673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d6aea94440c9c2bf1417d5f9ce0bcd33, offset: 0, size: 0 2024.10.22 02:10:45.813844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d6aea94440c9c2bf1417d5f9ce0bcd33 for removal 2024.10.22 02:10:45.818820 [ 1476 ] {} CacheMetadata: Key d6aea94440c9c2bf1417d5f9ce0bcd33 is removed from metadata 2024.10.22 02:10:45.841284 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8c4c7be67cc880445c89fe784aa75752, offset: 0, size: 0 2024.10.22 02:10:45.843786 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8c4c7be67cc880445c89fe784aa75752 for removal 2024.10.22 02:10:45.845713 [ 1476 ] {} CacheMetadata: Key 8c4c7be67cc880445c89fe784aa75752 is removed from metadata 2024.10.22 02:10:46.031089 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.198099838 sec. 2024.10.22 02:10:46.473258 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:46.475534 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:46.545799 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1386_1386_0 (state Temporary) 2024.10.22 02:10:46.572728 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1386_1386_0 to 202410_1399_1399_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:46.740666 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91003 2024.10.22 02:10:46.759569 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 12509 2024.10.22 02:10:46.885660 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3be7c84a423740b7050bcaab4edcfe1a) : [0, 238]: Setting download as failed: File segment: [0, 238], key: 3be7c84a423740b7050bcaab4edcfe1a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:46.893038 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3be7c84a423740b7050bcaab4edcfe1a): No space left in cache to reserve 239 bytes, will continue without cache download 2024.10.22 02:10:46.963449 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 1185130 2024.10.22 02:10:46.979097 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:46.983962 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:47.037206 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_822_822_0 (state Temporary) 2024.10.22 02:10:47.074934 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_822_822_0 to 202410_835_835_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:47.134638 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12509 2024.10.22 02:10:47.214567 [ 1212 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_830_207 to 202410_831_831_0 2024.10.22 02:10:47.221474 [ 1212 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:47.227669 [ 1212 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:47.273688 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_831_208} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_830_207 to 202410_831_831_0 into Compact with storage Full 2024.10.22 02:10:47.514362 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:47.518077 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:47.568391 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_831_208} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:47.572984 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_831_208} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_830_207, total 12533 rows starting from the beginning of the part 2024.10.22 02:10:47.610855 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_787_787_0 (state Temporary) 2024.10.22 02:10:47.800303 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d612bfc61991f12539e13588f69d52f2) : [0, 366937]: Setting download as failed: File segment: [0, 366937], key: d612bfc61991f12539e13588f69d52f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:47.814147 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d612bfc61991f12539e13588f69d52f2): No space left in cache to reserve 366938 bytes, will continue without cache download 2024.10.22 02:10:47.814328 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_165_165_0 (state Temporary) 2024.10.22 02:10:47.835760 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_831_208} MergeTreeSequentialSource: Reading 2 marks from part 202410_831_831_0, total 10 rows starting from the beginning of the part 2024.10.22 02:10:47.870615 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_787_787_0 to 202410_794_794_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:48.011686 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1185130 2024.10.22 02:10:48.015307 [ 1264 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_784_148 to 202410_785_790_1 2024.10.22 02:10:48.040296 [ 1264 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:48.052831 [ 1264 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:48.074756 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_790_149} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_784_148 to 202410_785_790_1 into Wide with storage Full 2024.10.22 02:10:48.116998 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_790_149} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:48.127231 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_790_149} MergeTreeSequentialSource: Reading 146 marks from part 202410_1_784_148, total 1184069 rows starting from the beginning of the part 2024.10.22 02:10:48.161134 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d612bfc61991f12539e13588f69d52f2, offset: 0, size: 0 2024.10.22 02:10:48.162611 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d612bfc61991f12539e13588f69d52f2 for removal 2024.10.22 02:10:48.167443 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_790_149} MergeTreeSequentialSource: Reading 3 marks from part 202410_785_790_1, total 9259 rows starting from the beginning of the part 2024.10.22 02:10:48.168815 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3be7c84a423740b7050bcaab4edcfe1a, offset: 0, size: 0 2024.10.22 02:10:48.166978 [ 1476 ] {} CacheMetadata: Key d612bfc61991f12539e13588f69d52f2 is removed from metadata 2024.10.22 02:10:48.171271 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3be7c84a423740b7050bcaab4edcfe1a for removal 2024.10.22 02:10:48.173583 [ 1476 ] {} CacheMetadata: Key 3be7c84a423740b7050bcaab4edcfe1a is removed from metadata 2024.10.22 02:10:48.687866 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(b0caf4f74b56c51bd6a35143a2025322) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b0caf4f74b56c51bd6a35143a2025322, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:10:48.690784 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 969018, file segment info: File segment: [0, 33554431], key: b0caf4f74b56c51bd6a35143a2025322, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:10:48.692806 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:10:48.825608 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 90112 2024.10.22 02:10:49.055722 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:49.058082 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:49.125388 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_808_808_0 (state Temporary) 2024.10.22 02:10:49.212498 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_831_208} MergeTask::MergeProjectionsStage: Merge sorted 12543 rows, containing 12 columns (12 merged, 0 gathered) in 1.840500667 sec., 6814.993455256379 rows/sec., 1.06 MiB/sec. 2024.10.22 02:10:49.271334 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_831_208} MergedBlockOutputStream: filled checksums 202410_1_831_208 (state Temporary) 2024.10.22 02:10:49.281238 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_808_808_0 to 202410_814_814_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:49.454671 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90112 2024.10.22 02:10:49.573447 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegment(5fd29699e006415d61b563e3f2ba1676) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5fd29699e006415d61b563e3f2ba1676, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:10:49.575007 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 506, file segment info: File segment: [0, 33554431], key: 5fd29699e006415d61b563e3f2ba1676, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, current write offset: 0, caller id: 18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1:1389, kind: Regular, unbound: 0 2024.10.22 02:10:49.579002 [ 1389 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:10:49.746741 [ 1718 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vch/uzrpekbtmdwesjzhggatimtimlkxo, size 969018 2024.10.22 02:10:49.749220 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_831_208} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_831_208 to 202410_1_831_208 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:49.752520 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_831_208} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_830_207, 202410_831_831_0] -> 202410_1_831_208 2024.10.22 02:10:49.775878 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:10:49.973527 [ 1394 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} MergeTask::MergeProjectionsStage: Merge sorted 6528 rows, containing 679 columns (679 merged, 0 gathered) in 169.97404942 sec., 38.40586267301038 rows/sec., 51.59 KiB/sec. 2024.10.22 02:10:50.115550 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_771_168, 202410_772_772_0] 2024.10.22 02:10:50.118277 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_771_168, 202410_772_772_0] 2024.10.22 02:10:50.136347 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_771_168, 202410_772_772_0] 2024.10.22 02:10:50.258404 [ 1213 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:10:50.282917 [ 1213 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:10:50.460175 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 7.87 MiB, will set to 1.59 GiB (RSS), difference: -10.21 MiB 2024.10.22 02:10:50.745379 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 91078 2024.10.22 02:10:51.252493 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:10:51.254610 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:10:51.366765 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:51.369219 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:51.441541 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1387_1387_0 (state Temporary) 2024.10.22 02:10:51.492802 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1387_1387_0 to 202410_1400_1400_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:51.541830 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91078 2024.10.22 02:10:51.857528 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} MergedBlockOutputStream: filled checksums 202410_1_164_27 (state Temporary) 2024.10.22 02:10:52.022219 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_790_149/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:10:52.120532 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_165_165_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:10:52.717233 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 19.41 MiB, will set to 1.60 GiB (RSS), difference: 9.92 MiB 2024.10.22 02:10:54.640825 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 12516 2024.10.22 02:10:54.775245 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:54.780517 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:54.934728 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_823_823_0 (state Temporary) 2024.10.22 02:10:55.020610 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1186285 2024.10.22 02:10:55.049430 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fcdfb2b47c488f6c00f8de7a14ca6cf1) : [0, 232]: Setting download as failed: File segment: [0, 232], key: fcdfb2b47c488f6c00f8de7a14ca6cf1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:55.051617 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fcdfb2b47c488f6c00f8de7a14ca6cf1): No space left in cache to reserve 233 bytes, will continue without cache download 2024.10.22 02:10:55.363902 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_823_823_0 to 202410_836_836_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:55.415592 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_831_208 to 202410_832_832_0 2024.10.22 02:10:55.423777 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:55.428762 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:55.549243 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12516 2024.10.22 02:10:55.583427 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_832_209} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_831_208 to 202410_832_832_0 into Compact with storage Full 2024.10.22 02:10:55.648013 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 25 entries to flush up to offset 91103 2024.10.22 02:10:55.683185 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_165_165_0 to 202410_168_168_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:55.891395 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:55.896140 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:55.943880 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_832_209} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:10:55.950835 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1388_1388_0 (state Temporary) 2024.10.22 02:10:55.964174 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_832_209} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_831_208, total 12543 rows starting from the beginning of the part 2024.10.22 02:10:56.029634 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1388_1388_0 to 202410_1401_1401_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:56.039555 [ 1276 ] {} DatabaseCatalog: Execution took 560 ms. 2024.10.22 02:10:56.066653 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_832_209} MergeTreeSequentialSource: Reading 2 marks from part 202410_832_832_0, total 22 rows starting from the beginning of the part 2024.10.22 02:10:56.239749 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_724_138, 202410_725_730_1] 2024.10.22 02:10:56.242645 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_724_138, 202410_725_730_1] 2024.10.22 02:10:56.252948 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_724_138, 202410_725_730_1] 2024.10.22 02:10:56.445149 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91103 2024.10.22 02:10:56.484253 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9a46a7804085d1b03ef0a867f2982595) : [0, 804403]: Setting download as failed: File segment: [0, 804403], key: 9a46a7804085d1b03ef0a867f2982595, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:10:56.487229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9a46a7804085d1b03ef0a867f2982595): No space left in cache to reserve 804404 bytes, will continue without cache download 2024.10.22 02:10:56.538801 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:56.545498 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:56.660969 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9a46a7804085d1b03ef0a867f2982595, offset: 0, size: 0 2024.10.22 02:10:56.663754 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9a46a7804085d1b03ef0a867f2982595 for removal 2024.10.22 02:10:56.668716 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fcdfb2b47c488f6c00f8de7a14ca6cf1, offset: 0, size: 0 2024.10.22 02:10:56.670116 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fcdfb2b47c488f6c00f8de7a14ca6cf1 for removal 2024.10.22 02:10:56.764249 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 15.87 MiB, will set to 1.60 GiB (RSS), difference: -3.79 MiB 2024.10.22 02:10:56.766722 [ 1476 ] {} CacheMetadata: Key 9a46a7804085d1b03ef0a867f2982595 is removed from metadata 2024.10.22 02:10:56.769470 [ 1476 ] {} CacheMetadata: Key fcdfb2b47c488f6c00f8de7a14ca6cf1 is removed from metadata 2024.10.22 02:10:56.867538 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_788_788_0 (state Temporary) 2024.10.22 02:10:56.967015 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 71 entries to flush up to offset 90183 2024.10.22 02:10:56.982566 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_788_788_0 to 202410_795_795_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:57.031076 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1186285 2024.10.22 02:10:57.282195 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(0f0a74c323f8eb52353e88cfb6271459) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0f0a74c323f8eb52353e88cfb6271459, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 02:10:57.283675 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:57.285170 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 374482, file segment info: File segment: [0, 33554431], key: 0f0a74c323f8eb52353e88cfb6271459, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 02:10:57.293162 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.51 GiB. 2024.10.22 02:10:57.296169 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:10:57.360829 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_809_809_0 (state Temporary) 2024.10.22 02:10:57.408233 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_809_809_0 to 202410_815_815_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:57.565206 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_832_209} MergeTask::MergeProjectionsStage: Merge sorted 12565 rows, containing 12 columns (12 merged, 0 gathered) in 2.100942631 sec., 5980.648788120097 rows/sec., 954.68 KiB/sec. 2024.10.22 02:10:57.580022 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90183 2024.10.22 02:10:57.795674 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 684 2024.10.22 02:10:57.838295 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_832_209} MergedBlockOutputStream: filled checksums 202410_1_832_209 (state Temporary) 2024.10.22 02:10:57.911726 [ 1419 ] {} S3ObjectStorage: Objects with paths [common/xmu/hkmjutvqotundaygjaxkuweckebag, common/fhg/ubfumfkgtgrvvurnumglmbdixsget, common/xgd/sbhcmvstcsnyyslbbaghpexukcfqp, common/exs/ylkjxrlvozryljhazlfpfjyflsnxw, common/cey/nuwiewhbankqvzggmiuvunianbslf, common/vmt/ogtjjdektqwjlruocklaqpjagfarv, common/ayz/jeghokundgmmmmkndgyldrjcofdxv, common/cen/dgtzxepfdgagyjlkoxnmruivjuzms, common/gae/jpqqmfvltjdovbiurlzcqsgtutdeo, common/ify/iclcctoqzebpdoidzqkvfftpbijef, common/kgj/gfkcswigroxxtyngsyvsooofjqmlm, common/dcp/ogwcjinxrvqfcbhyesxyvliagolnn, common/lem/tbccbzvlrgjekhwlpbjotlmcceidf, common/tme/kbcvxsezexsbyiebbfmysjntadhen, common/qdn/skbuivxhpyumkwiflyvgcqjqsuvyd, common/aie/nwlycrtfcbuxgmrkijfuoqxijiynr, common/xbg/jfgffrctdcallkkbeygcakdhyxllj, common/yua/skupslyshwhxtewlwsizaalqjmgmt, common/bge/xrbgjhkgdgscijzkdeuucfleaqnva, common/fkl/cjzgkpvholnfglljhsaksyzqcfadf, common/kqt/lhjyhiqftgqvduvzzzwboaejblknl, common/qcb/vjzjladsfawiiajcrrnkoetekddli, common/rxc/uegqdxrvosheuqkhukgndnifqcfev, common/gby/bvjhjuymbhodphtksdstqpnourdmx, common/ulf/bugfkdmzihbdvvgrgvlavuhpygavc, common/jxf/rvtugwrdxwjobfrmhtrhedqlnryxv, common/gof/lmsptjdrnirnmpfwrafuktxgspprm, common/ozm/canruxndzfeazvaduqbxsbvtcmonk, common/rzk/lnkznrlcnnbrptklunnfsprxzbgfj, common/shb/edkohtzfearjfnhabnzhjzjnjlwzk, common/lww/tbixnahkgjroonzgmyqpeykuveyec, common/yrl/svpziktmrpntwiqgqotduwagbqsty, common/sjj/bxjvukpeyivmxtdayaibuwmmxxmhf, common/mxi/jrnbybakglcgqzdipqpdaqkkdekpr, common/ttr/lvdqaexoowdnwblydmbqeyxkmlijc, common/trt/dbgdvhqggixvdhndhwqxrmvtmasjq, common/abn/visfpkbgiusqqvateltgmqufgglxg, common/hkv/janylrobsbvsbzlicroddxdecgiog, common/ytl/mrvthxqbeyheizmllswikqpmfezuz, common/irp/edwbdcypuimbjmdfgjstzhoxyquma, common/czu/glhhgjardtutknqjfpsxwbiuqkurt, common/onp/nvkabazfmotwovijycymbookvaumt, common/jtf/ggnhbreppkvrgegbxbupofktzjpjr, common/fqq/ozgaxvlmowyuxniuynyzoggigdthj, common/lfy/eysjjqlbujyaomhotpqagyojpxzdd, common/bhs/iifqcgwbcwkghkofpcjcnjftkjbzy, common/eup/xvzfdgbavtyjcopwaljdxdofrgint, common/itd/nccxhlyymtqwxrknpudvcuzptdjjc, common/amo/olbplnsooeahxzkrfzcvxubznvohv, common/sog/tariabvezsrenppkdbhagqoixmzxo, common/bfo/xfljgtfdulynxkqqccelwydnvshmg, common/oxd/pxbaahyrzmolzjnmoleeayemkrbea, common/fkx/vjaekrbzrijgdhuqawssewxvngcew, common/fbl/mxqanvetvxfiwicdxkxepnhpcuecp, common/nlj/eqtsjxjbglabxurjqhnoyydosnonm, common/jkf/vosdrgnmgwlamekwekbymiephozfa, common/rtj/aaqurnvugdprcvricchhutqdjclax, common/hqi/sxedqrdtdftyxfhftkathsuscvzzl, common/mht/tvxuhtnznhnazuujgsmpfhslrnrtl, common/trt/uhnmjedxzzvjcgxwtdlijntmaoujm, common/lku/tegemohtayqenulikeiclqicxcdow, common/ner/bneoignxkxjejtdsspkzqfzxwujgz, common/zsj/fbsjjjmpupqykdyttpglzlnrtboet, common/peg/hpwdxoyfifzjxdxmwvuamyrlyutiv, common/kem/kwxfcipdbysqsotootjrpmfviyvzr, common/ooq/vjpyvofjwtfmsvotcmznuewgxknrj, common/luf/bpngjhxpbocrzvylcdtrvcnjoweiw, common/pgi/hckarqcsrrnzypwtvvsrlqhtuhmvi, common/vjr/tmituuhiglptwyzrjkwvmuuchpqcs, common/nxe/diyyqskgojovineqkmwiliszpdtya, common/iwf/kwtstpjbeaestmmufyigbzphekbea, common/ivv/dknjmygiozndsmawufovhbfsvpynt, common/chz/ualkroagimbopifocwrrmboleubwq, common/vdz/dqwcmaeqxmvyusarcjjiaevvaiipx, common/xjb/esdjttnuapolyqbxfzxcldassuftm, common/feq/wjvqxovfauslocdzpzphpiahjoedm, common/sjr/tzemnputuphjosqfgnjhsnwieibwb, common/xhb/porqfyfdlwevmutpuyimcehuyehti, common/iia/efzydsjulsmkxqlimkdsjvjjrjynk, common/evq/jbbubiifeisjiwaqaliiblxuyqjaf, common/usc/phjblottoxxfucpudrhpxsnmngfxh, common/tcu/pcfpjouxxwmppdsyvovlokapizvim, common/gmp/ntsaotarqmleiwyxkqakmiwkxgpib, common/spu/bjhyxuoiozeiqlsrhnkvolvetcbsj, common/ngf/epvhhkwukcsllwevwwjfdkvtauntx, common/gfa/jqhrviktsqlhbiudfqljehxyofixc, common/rty/kzyejmtvukaaeitewaabiphmugqaz, common/xbx/eygoxxrtzswoapabypaiatcjrgpyv, common/oqj/nuvwmnuckmkmacqtaxematnbrrnpy, common/gjy/brwaaxshtflmfajlznihyentcmuyv, common/nll/fkqfjedmpumhowlpzfvyhgnuplfry, common/tun/cwalbaycdvaenrmclkwpjyulurdab, common/sqh/fnwmtynpsclbffrpichlhpkyytnxk, common/wmm/xpbndmqkfiqksxadftmnhesiacahr, common/sfv/yyfadlybeiexzllofcjnqsiizfrtt, common/qll/xpzuyodnzabzhyobjvbecusotxtey, common/dcu/yxxstlueedtkgcvazmfewafjzsrlo, common/uxy/itrmzqjezpkummkgethroydzdltyy, common/our/mldmsuhmkmuaareqfpklazikspqhq, common/rrh/slhrtfatnwyznhuhoqjnhydiboeuc, common/mtz/geigfnppqtigqxkofcdnrelevgfbs, common/bnl/wniqvocycdjpcizgcrdqpscxxtqvh, common/ntj/llqmfvdzfusuzjgyrifbjkxtqndaa, common/xbe/fwexgdzbydymollycvphmbfuzhidb, common/psv/pjaxpjmyjanxnetyqdarckmvvfwwu, common/zdl/xwzianbnoooxgsxuuligqanqpvapp, common/nbq/ynlmzccacixssndrmmknfjeuqowjj, common/ndx/bostlnxpsfuvliizywznrwxrrfgjc, common/psq/mqopbquczufxnkrxxxzorwpgujswp, common/avt/auhmhokewddbxzzljyuhbhjtugkjf, common/uwz/pdswrzquljxejhdfutttfzbbjkfqo, common/drt/lecmzduyrmoiqqmujuoauihvdbvyx, common/afh/xuflmsobovyfjecogemsfhxxmshqq, common/qzt/lfodexhdljlptlnjhmmcxgzrbjqsf, common/uid/uitwividiqewnbceboyfrtwqtsweq, common/lfo/lzumpeshswrtphkmsrirkjagnslyo, common/ldf/ltcwegiaftalrfhmsqlnoeabdhxhk, common/dvu/zlhunwjnmdeehhqoxorlyahxucvdu, common/ztg/ulzhlwxsirlbkldnwujpfchndhkvv, common/kfe/gcdoramsjavipdnasyvwqysdhrete, common/daa/tscnmbuydspxhbionlgngswxgbvcm, common/cro/hosapjiumouvaudqsmdvmlvgewhqo, common/hxg/gemdcjxgckzpcqqmzbdazhhyrhutf, common/uqa/qtvbpzcjvrhmfpbkygyewyguxbfjq, common/kjm/siknimfshclzuwzhlvnkecscnpkzi, common/eto/objfvnqfjvetkurmuyuarrlzztnog, common/bin/ngqsehtfslchzgwptylelmtakynwq, common/qdd/wdfzsvvphgacnscasggkdnyvoeblj, common/kzl/ubyxcjvjibdwfpkenzvwjzrbyekbe, common/pfd/bwpiuoimjedjdoqynjqjpqzxxkbuv, common/ikk/ialhlbizcmereiyiqundeeabagwyj, common/xcf/mlhfpzgvofttllrmfqfiiyddmemrh, common/jur/seeepwiuwwazzegvfuedrgbtjtgle, common/mjk/jrnknurvkipibertemgjspskdgfaz, common/eja/abdoaylqxgxudledazgaxhocgbyjm, common/syf/sxemohpfsdoeovzsdskbqbwayvezg, common/zet/svzdudpgixppppnbfgjrovztwwnsw, common/vnf/mvvtemzkvqebyhubyntwdxdemyrxi, common/dmi/olncjadbcuqpzjweetkbeihedkniv, common/mmv/ecjfvbowhagxhmabtlqqbjhndkxxk, common/ysj/odiwwlmxuhaiqkqgjjccjxvegnihd, common/pgi/wmzqiydyxbzlxsswtpikfhxsnagsv, common/hjg/itygqtfvxqigathppmxqtbnqriull, common/oef/vdjzyyxzcsrcddjrsiasjaayhgzwu, common/pwo/ebpyqpftibxkdppihfydcpnwedraf, common/dwc/wnbnflgnzquuxegjilrsrwquoddxt, common/amo/iaaonalokqzqgjfqdktcsgorgxitv, common/jtq/jrixgrqiytezflwtcmkbmuhbxyzay, common/dpa/nstdqzkfrxgendhdnupqhdyctycek, common/jhs/smpybmbcxmoblqwcgznchuifbbjhn, common/oww/yowzaupfzrnxsofpnkvqbosqdkwqm, common/wkd/xpfgutsfxpgompvfcxndxvgzwtiaa, common/syd/hzksuahjdcpqhlntavcedzaqckflo, common/oma/nawadfcsrevnzpyfplmjwllqzsyew, common/egj/fzpxeblolbakywckcdqretdupkqmd, common/hpu/sknrguobwisfpgvvgkzjinyetghxb, common/ezg/uhhfsubwkgzmohicdnpynndwobljw, common/rit/ylllvvravyetsdvyamdrbrjlzvvvb, common/mzz/svcitvfcinjgljdffhwpwinvarldw, common/dsn/mpaysbhwsfcrcuwoupeuggfzlsyaf, common/idd/cakpfqobtlkmzmqqbrjtnpyhejoui, common/niz/bxjdhgvtmxgkwuurxwikhksabslmu, common/vea/pwpndaqhvveqtcbngxdcmvmeqrgvw, common/mum/xwekasxboovdqrpcuvyetakkunkdo, common/xxw/auefsofyktgrqdoxoajmyltmqpgno, common/nhl/kakiiwwgpwezxtgejkpgiieywddzo, common/ngv/cuobankbnkkmgzjzoqrrrudftppdm, common/pzo/pcgipwggpotswmxbnzbkdzloajhpt, common/zvv/yfuwtxlavnjgnleoggqdgytvbxpgv, common/off/jrzcbrficfcmbcymwnyfvgfpdfmfr, common/rmf/yslnhhowyadzqzbrnocvlmrtyizoc, common/swu/upfxoxaykiedqhlbxqvptdxpzgpnv, common/rio/rqvgajzxibfjkayrtojybdghqnvpb, common/qnt/ndrihaxpphntmamxsdwymnnytybid, common/bfz/aszciagbckwcsaadiqxtmkkyedqug, common/brd/aedbspnsnynsdbeyrvcvyeweezusk, common/bki/jqqdnjfxoljkbmvxeyvkghbyvbcff, common/eys/meearrtbvpiglcikhcjsptdvonnre, common/uah/buxegeofimjvdvqofpnlvbzfvkgrq, common/liw/isakzrxwpmlvrfsbqctqveucvsdok, common/zdv/hhxteuwhiucorlsvnighrjhogkvgn, common/kud/teodkzpswmtcdkymudmlrrrpqvpvc, common/ldu/jebvrchwybafyvpaniqiodfwvopvb, common/lgb/ndowoavuurabhvrwwkmaqotmsrswg, common/wun/fwzlckwjsfayngjlgrrahpirqfqtn, common/ofv/jfwlgxrttmfqwwqmitmbuydmwuaxz, common/edu/dhzhmpkbmhfcjsoynrxmzpipafdqu, common/yei/qucltiofvfsnekxoemfkxdleiejmx, common/pil/zerhhfxxpnjgnpygyvljvqmwqnysr, common/rab/alppzdotnoncbuunbkiydgsnpjlca, common/exa/ivlvbluxnnnkxevhqijfybpjzbgge, common/mom/afpvruaxgoiralhlnyiydzyrorzqn, common/kvn/ruynshylejpjdoozrcflcmdbakctd, common/bgb/rwwbrzhiqjombjeavmpgqveibdsyn, common/ykb/rjrepjrbpyjznnbfygsvqoshzhnfh, common/kxb/pcuwsytkwkhtdlcyurumzkfdzxwdt, common/enh/fznbnobxcylvglmnxqpkwisfvdmzd, common/paj/fgktvicmqdvaulwmtuxrgnvqbcgqm, common/gay/dcpwqtbieksvdtdxvagjhlsncwemc, common/pbo/gwfmujrsoblurwxwpdxkwttgmllyz, common/tou/nqephxljblbvohhtswfijyftlbyoy, common/tbx/iigcnhbnttzbjfmaciqmuysviaodx, common/xrb/onuaexbqydwgipxchdkcgkaxeiapb, common/xdr/zruburuqucgxvzbrsboktxneqdxhc, common/hnp/baaxjxocayligoewedwhodrjjkood, common/hpu/vhaqtokzpxgqwfwfmrihvafnscytf, common/swp/xabjzyoglsazanrlgjhxsxublpqcp, common/utv/mnpktfyqwyqijcifzaamndveqckqg, common/nyj/xhjadnsosiqdcwldgthfinjeqjmlv, common/rkh/lrzuwhdvrdgefvpthmapcsihughfg, common/tui/stfhiyscdifwfmdnvvcxmwnoxnmfp, common/lsr/qefgefliouimstohelohxkedywxiu, common/tbj/jlotwkvdzeedkknycueemqnnwwkxl, common/ezs/xposcsetnkjvudkccvnwqqaddobbn, common/exq/xnecdaltwvlfzxyhhzslghfygqvhv, common/ipq/hnwkegusgzorvuogyfnlbnvejbugt, common/hfc/ygannktlkfdoradufshndeketwjhz, common/nwn/cvctzmrxxqpsnqbfvkhylotyvjakd, common/jaz/lbqzlcqvaxwefdmlybdyiqhkxfbam, common/nzf/usawkcjzjnqiiedhzvwygbvdtrmaj, common/npj/xrowkotjgwwbssotljoygowyiinvr, common/axe/rikwwilzfcudzrtljyqkukemprfgs, common/csn/rgkzphyymueybjnqwkgdflxvjqzuz, common/eyg/gnnykjqxumkaolhtrreiwhdxhczin, common/fyz/galywyychtujmxguvxzmqimocnmwd, common/snz/rzbnvhovvwvhsneixewbbtywnduvg, common/gvp/mcybnczxhveblucffliaqrzhpimie, common/gjk/ibnytbwrxmudvdqtwmiqtbusmvgvp, common/cvi/fxgyxjeacnuagizmlvupfhafgggtb, common/jml/crjmwzneukhdzblimlshdgnqwkncb, common/ubm/eilxbvxjbxksdilozfpscptcwokff, common/wau/inxqvvmldbzzccabdufblkgqgiwxa, common/ulw/mybfaqziosztivyflafetgggpqxab, common/skq/mocwijdnhyiesxbooqshnquniajbj, common/trn/upusghvuvasacasygpfykbegsnguy, common/zbk/unyeajqvfpqvrgrhlqtmyahcgevzn, common/zgf/klachyldmnhpkrtftkrqlutuadetq, common/evu/ffpekcjqfnevhmkftkzrghoykrjrx, common/ttn/bumuvspssiuvnfumzhiqwidqlcwra, common/wsp/jsuaaptaztulopgxhugcuzulciggw, common/inc/mtnemwgocmvzvszxdsebahecnqpyn, common/ffr/cxzhggcnanwqvbbsazteguihjqkfl, common/bxw/cjradmlmrakyrvqqgycgqnpycfkhu, common/gba/qyjwpupkrppepkytcnqddyxkhyano, common/ojr/uyhxkjxpgkltllvtvbqmxwxubwsgp, common/mkj/kknftwunporbevkzcgfqwxgvrwrjj, common/scj/fciogdgzwhkhminoyghrsqdyivmaa, common/iwi/ouoqbzrrewewplotftiuujgwlywhd, common/ozd/lxabvwykzzbfqbugwtxusgbwrfpud, common/jgb/xcvvpgirxhtnmyzxirfxrasqywijw, common/uun/phqnqodjnjdbgcyawsightrukpfaa, common/cwm/egcgvlfdytnchyhmupflkgxrrveoq, common/rui/jgxnwtffknmkgctjldvxsxhihrhwh, common/lbz/gqyferkwfratsmonecccxrpglqiqd, common/uuy/herzypargqmqbapimlmhdrsquvzsx, common/hgf/wdwocwawmpjnuiepwieylyrqznjjn, common/koj/xilfforrvzpzvmayhqdptsbsmedip, common/pab/friutduzonnivqkdspcjydzlhtdnx, common/iju/ctrdhnxdkymzcexcyygbsbielpigu, common/vtw/ivlbvuvxgpyqacrcsqeawfnozusys, common/cyi/xsldmdyqsmzcolvpimjhfvjcabned, common/ymt/rtxsfvfzhxgtufplpzoiwshoxbtrf, common/wgq/uuyxagvykveyjntsmjpawtbepdyjj, common/oxp/mtkrddbnzlodddoldbjfaeumwiqlg, common/ujs/gatnfbjifboscshfhprqzqsoiqcuw, common/pjs/mafznoegywnxgpyrotdxijxwxkgka, common/qco/oatkulgztuvimoytapczyqzedxeyp, common/mln/jdxahpxgppifjsgrusutloyegkmoz, common/ywc/hsnyqljqdflmiyozjsvnrntnozyhm, common/udi/ibdzdoamtizsfeydpwebuhaufbsnu, common/kax/egwenlkanwlkgqdxabmfqoulbblmy, common/jvh/oexvwtbbohhjgxtmkanyuxgrrqukf, common/hqe/zximvjgulgpbccyyndtrvuzgkljvj, common/ded/qvrgspghyviabkdyisdamjtnllkrx, common/svp/bhktohkynbddxarfdrirsuixondlr, common/rve/hdnpvvpqsboityvsgfnturlacadfr, common/hpe/jfyhxdfzlgazbycqpnfxpfdjvsdhi, common/adt/fbsdicazqgpjpdyiersbyugqhibxc, common/xbj/omuznzokoqkdkwddshwfdnmvllejd, common/qtk/dfoioaakwmmwvfhiahciuhgoxskwn, common/ilz/zavhigzifcreztfqvanvyrcmlqzia, common/kdq/emqbwphvwozfnszrvvrgycfizrxwj, common/yzb/xljioystijscxramyuwzchkxsodel, common/cgt/ucrfiqxntiinsrnquqitowwjsireg, common/rwc/vscrgxgpmxnofovgclalvauhpmlhq, common/xzd/bofxqihnenzofapgirzhbxtsbyvvh, common/ygr/xhaxgckllgszvktrrietrczhcfzuv, common/krs/ucdmygijagqdsvyvldcvzejauiwqp, common/sbr/glqcwkxpskcybgtpccjnsfbuaodxl, common/ivv/dgbpjfhkuoyriyvnuonajkknmbgtg, common/wbq/nuoueiwybplilisbxcrbjuroppbhd, common/ehf/kpbhzbxoknbejrlknphztuwmjimyc, common/tjx/ycmnxnyqnfgfzsddmnwgiipmyswov, common/ewy/eudjgozwcytmlllegktafiygngzgg, common/cek/hwzvryfpudhxvfcyxnjqyknefmddv, common/rey/ulkguuhphxtxxctiybephuitvurzs, common/trf/mugclvuuxpwnrxhkonlwejesgubmn, common/vrk/zobrfvhctqnujhimatnyowbpgrllg, common/nvh/zqbxjeqrpdjjwnzaemmexlpdkkbbv, common/gzn/pkooilnzzfzektlrilborzqailurn, common/urm/yquinwggzqhthziymqjnmvpnhbqzc, common/zob/bhbnnmcsiugaxrejyuucffsaznemi, common/bbr/bitosxqwpsymxfxaxjytoqgcnnigx, common/pxo/tbaniyycphmafwsaajwbhnampbsgo, common/ujq/thdzmwekvcrvehiyeouxmqvkqehvi, common/xuz/fdjzydlfxvcocrwpeiqtluaaqunjq, common/khz/lpmgfvqhqztssdakzmyyigsnqowdh, common/vjy/vfkyjjhamkctmtoahpxsxgsfkdvau, common/tbk/lrmcuxzaivzofmvbxcrldonpbkrcy, common/jjg/awbzqvjvmhvhswninyfiafaxxvdcg, common/qub/tqmpfuscpsrauqeaudbkxgphflcdm, common/fhg/xqbmztimrezawmpmtyjtrrmfajajy, common/bsn/jjzrpdkdkepyzkhimtxjaqzmejmhg, common/rrr/eewhggiquqrpmpvuhajmaaqnhpjtg, common/hxf/woxkrjjtuklkzhsxfxxifencxbtjh, common/ssd/mukjijundbsuhgwqtwxhevorwtxqb, common/pvh/rfklnyweaggexayjgoqqrkyhrevfn, common/cvp/lprppsnvcfwogbtvjsxcsvfoxlpmh, common/qmk/zzuuuhgwqwwqhjuiaorpslqvszfrc, common/viz/aeurtktuppeotxjkwswezfyfokasu, common/oeg/ykgpgucoiwsgkzulpoimvkcpudejx, common/abp/czyhtzilltqvzavmacftpenozojdh, common/euy/boiemwbmltuyucdwjyvlfwtpviqpf, common/hoq/jojnmupkvetetnaqdieilseocecrh, common/ryw/tiafxfmitfpkysyinuhcvmrjumjgs, common/waz/twytnbkeiaractohewlzzwxrdcqtm, common/urk/omndmommzulligjrkteorzjchclqr, common/moh/wexpbkswfwvdsinwaxamnehiycsvj, common/wzz/guevqpuiolmzlytbrkpsdufluowoh, common/egq/pjmzjbmncamdbebhsiqhynwbtkfwv, common/wsb/jnmxbagaxmxkxgeykjptbxaetuehd, common/xmb/rlvyisoywygpbntlsrxnmgcwdbfen, common/qkg/afqralblxtzkvrqbwdmxmuyhnjxyn, common/gml/vjltcykugngwfplwxmshjnyzhpjci, common/atg/gvksskzgcujixzqcwjeqnhfkldsrg, common/akz/ipmyloozonghsvcwqbrchfeiwjpat, common/cla/tncorlzanwcwlzbjbzrlkzxooprfg, common/koi/yvsvrxslmwneljoterrpjepdgdspb, common/wju/vvudtkrqiwdzgayavgpthimchriig, common/nwv/dygybsiidmlsyryryykmtbgfxtxah, common/ztj/tkegkodwnbrqklmifzcmjrehepcim, common/nsu/zqyxqrlhpanymdqilexgocjnyqour, common/xay/pewophpmboufpcgnpuxjgixvospyh, common/tub/lzojvalfrevzxlxhddufdjcxnskmv, common/bop/yntfzcltpwkxispwqwriffoxceqbm, common/zir/glczsjqdzgkvgnivhmrqizphalshk, common/xuh/zxzvqrsiycxlpskupeyttnsiowkwt, common/veu/mjxwwntjlxdtxvqtayionaidbplpc, common/jby/ywxgviqbabamfsnrhgodzfrzwzfgx, common/hex/hytxestsqnqbmatdbxruuryxpyaid, common/hts/iyzwnpetudvaqajaqswfvwxswbnym, common/pis/pamcxmfgdgawvgdrnkwuxdhmsluyo, common/tlr/ipchejcryyhgngsyxkkocfugdwyrt, common/kqv/jdgoiraxgmivzuzcuxfbzuszmtviv, common/xgn/zgixeutzcfmubtlcrejpiqgzvjlez, common/ohc/eozwbxdkiaezccekdkzwzinyohywy, common/ala/vofulckbypngymwpngqiouokzivqt, common/zuw/okdwmuxafpkoihsvgzsqvcflntrja, common/opu/bfbhivdvqwhybexztzkhxblpubpbb, common/rbp/uhhnuzepphbmcwzibmyrqooqdiglu, common/gtp/dcowlcnsekyicpdalonzalpktciml] were removed from S3 2024.10.22 02:10:58.013654 [ 1419 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_14_14_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:10:58.119466 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6641 2024.10.22 02:10:58.128998 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_832_209} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_832_209 to 202410_1_832_209 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:58.146192 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_832_209} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_831_208, 202410_832_832_0] -> 202410_1_832_209 2024.10.22 02:10:58.161642 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:10:58.164442 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.56 GiB. 2024.10.22 02:10:58.263466 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:10:58.301582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 163 bytes for 724d78779ce49f0663227a6ae189c81b:0) 2024.10.22 02:10:58.306456 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8263d06ef15c31f8bcd6d8d149c894d9, offset: 0, size: 1074 2024.10.22 02:10:58.309436 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8263d06ef15c31f8bcd6d8d149c894d9 for removal 2024.10.22 02:10:58.312467 [ 1476 ] {} CacheMetadata: Key 8263d06ef15c31f8bcd6d8d149c894d9 is removed from metadata 2024.10.22 02:10:58.313532 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 97e1363b086da6573d4723551055f525, offset: 0, size: 4 2024.10.22 02:10:58.319652 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 97e1363b086da6573d4723551055f525 for removal 2024.10.22 02:10:58.326803 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1d68dd705f3cc80bd2e95b7dc9ca9233, offset: 0, size: 4 2024.10.22 02:10:58.331512 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1d68dd705f3cc80bd2e95b7dc9ca9233 for removal 2024.10.22 02:10:58.344079 [ 1476 ] {} CacheMetadata: Key 1d68dd705f3cc80bd2e95b7dc9ca9233 is removed from metadata 2024.10.22 02:10:58.358686 [ 1476 ] {} CacheMetadata: Key 97e1363b086da6573d4723551055f525 is removed from metadata 2024.10.22 02:10:58.663527 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_666_666_0 (state Temporary) 2024.10.22 02:10:58.749570 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_666_666_0 to 202410_676_676_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:10:58.764976 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsDownload 2024.10.22 02:10:58.901207 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_772_169, 202410_773_773_0] 2024.10.22 02:10:58.903754 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_772_169, 202410_773_773_0] 2024.10.22 02:10:58.921750 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_772_169, 202410_773_773_0] 2024.10.22 02:10:58.974149 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsDownload 2024.10.22 02:10:59.115800 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsDownload 2024.10.22 02:10:59.154259 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsDownload 2024.10.22 02:10:59.155767 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 684 2024.10.22 02:10:59.172612 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsDownload 2024.10.22 02:10:59.219683 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsDownload 2024.10.22 02:10:59.243866 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsDownload 2024.10.22 02:10:59.489023 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hup/axqcwhbupyyjwdlvbgxqduuiyakdz (LogSeriesLimiter: on interval from 2024-10-22 02:10:32 to 2024-10-22 02:10:59 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:10:59.546690 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 17.40 MiB, will set to 1.59 GiB (RSS), difference: 13.94 MiB 2024.10.22 02:10:59.547168 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 11065 bytes for ee8717e2478a948149a96769e52085e1:0) 2024.10.22 02:10:59.548694 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0da1bb5a82e7fa189fa7dc9a2bff1627, offset: 0, size: 21472705 2024.10.22 02:10:59.563293 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0da1bb5a82e7fa189fa7dc9a2bff1627 for removal 2024.10.22 02:10:59.666643 [ 1476 ] {} CacheMetadata: Key 0da1bb5a82e7fa189fa7dc9a2bff1627 is removed from metadata 2024.10.22 02:10:59.874232 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/izl/afuseuleisjxiesaxxmamsfdzqfoj, upload id b7174a93-d837-4ec5-919a-707e569e6e1a. 2024.10.22 02:10:59.988510 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} AWSClient: Got dangerous response with successful code 200, checking its body: ' http://localhost:11111/test/common/izl/afuseuleisjxiesaxxmamsfdzqfojtestcommon/izl/afuseuleisjxiesaxxmamsfdzqfoj"e60f691a03' 2024.10.22 02:10:59.997562 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Multipart upload has completed. Details: bucket test, key common/izl/afuseuleisjxiesaxxmamsfdzqfoj, upload id b7174a93-d837-4ec5-919a-707e569e6e1a, Parts: 2 2024.10.22 02:11:00.127681 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/izl/afuseuleisjxiesaxxmamsfdzqfoj, upload id b7174a93-d837-4ec5-919a-707e569e6e1a. 2024.10.22 02:11:00.144505 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/dou/paybvzcggfmainboorlqeamwmbcfw. 2024.10.22 02:11:00.350346 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/dou/paybvzcggfmainboorlqeamwmbcfw. 2024.10.22 02:11:00.448327 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 91186 2024.10.22 02:11:00.988592 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:00.993698 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:11:01.252921 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:11:01.266270 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:11:01.299826 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1389_1389_0 (state Temporary) 2024.10.22 02:11:01.490317 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1389_1389_0 to 202410_1402_1402_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:01.559311 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91186 2024.10.22 02:11:01.628500 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 18.24 MiB, will set to 1.59 GiB (RSS), difference: 2.58 MiB 2024.10.22 02:11:01.629088 [ 1162 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1397_38 to 202410_1402_1402_0 2024.10.22 02:11:01.668457 [ 1162 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:01.670773 [ 1162 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:11:01.712138 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1397_38 to 202410_1402_1402_0 into Compact with storage Full 2024.10.22 02:11:01.764348 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_623_127, 202410_624_624_0, 202410_625_625_0, 202410_626_626_0, 202410_627_627_0, 202410_628_628_0] 2024.10.22 02:11:01.767863 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_623_127, 202410_624_624_0, 202410_625_625_0, 202410_626_626_0, 202410_627_627_0, 202410_628_628_0] 2024.10.22 02:11:01.811359 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_623_127, 202410_624_624_0, 202410_625_625_0, 202410_626_626_0, 202410_627_627_0, 202410_628_628_0] 2024.10.22 02:11:01.836078 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:01.901699 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1397_38, total 11757 rows starting from the beginning of the part 2024.10.22 02:11:02.101456 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_1398_1398_0, total 63 rows starting from the beginning of the part 2024.10.22 02:11:02.298124 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_790_149/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:11:02.320837 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_1399_1399_0, total 42 rows starting from the beginning of the part 2024.10.22 02:11:02.896273 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_752_1, 202410_753_753_0, 202410_754_754_0, 202410_755_755_0, 202410_756_756_0, 202410_757_757_0] 2024.10.22 02:11:02.908441 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_1400_1400_0, total 75 rows starting from the beginning of the part 2024.10.22 02:11:02.898071 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_752_1, 202410_753_753_0, 202410_754_754_0, 202410_755_755_0, 202410_756_756_0, 202410_757_757_0] 2024.10.22 02:11:02.951382 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_752_1, 202410_753_753_0, 202410_754_754_0, 202410_755_755_0, 202410_756_756_0, 202410_757_757_0] 2024.10.22 02:11:02.972446 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_1401_1401_0, total 25 rows starting from the beginning of the part 2024.10.22 02:11:03.051245 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 12526 2024.10.22 02:11:03.059534 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_1402_1402_0, total 83 rows starting from the beginning of the part 2024.10.22 02:11:03.255688 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:03.258298 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:11:03.421314 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_824_824_0 (state Temporary) 2024.10.22 02:11:03.505833 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 16.09 MiB, will set to 1.59 GiB (RSS), difference: -13.64 MiB 2024.10.22 02:11:03.567646 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_824_824_0 to 202410_837_837_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:03.618229 [ 1171 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_832_209 to 202410_833_833_0 2024.10.22 02:11:03.631438 [ 1171 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:03.635334 [ 1171 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:11:03.670574 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_833_210} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_832_209 to 202410_833_833_0 into Compact with storage Full 2024.10.22 02:11:03.691942 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12526 2024.10.22 02:11:03.724920 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1297_18, 202410_1298_1298_0, 202410_1299_1299_0, 202410_1300_1300_0, 202410_1301_1301_0, 202410_1302_1302_0] 2024.10.22 02:11:03.748707 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1297_18, 202410_1298_1298_0, 202410_1299_1299_0, 202410_1300_1300_0, 202410_1301_1301_0, 202410_1302_1302_0] 2024.10.22 02:11:03.768714 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1297_18, 202410_1298_1298_0, 202410_1299_1299_0, 202410_1300_1300_0, 202410_1301_1301_0, 202410_1302_1302_0] 2024.10.22 02:11:03.788797 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_833_210} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:03.798798 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_833_210} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_832_209, total 12565 rows starting from the beginning of the part 2024.10.22 02:11:03.822361 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_833_210} MergeTreeSequentialSource: Reading 2 marks from part 202410_833_833_0, total 8 rows starting from the beginning of the part 2024.10.22 02:11:04.040182 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1708 entries to flush up to offset 1187993 2024.10.22 02:11:04.363592 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_164_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:11:04.734332 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_833_210} MergeTask::MergeProjectionsStage: Merge sorted 12573 rows, containing 12 columns (12 merged, 0 gathered) in 1.068000429 sec., 11772.46718128425 rows/sec., 1.84 MiB/sec. 2024.10.22 02:11:04.807674 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_833_210} MergedBlockOutputStream: filled checksums 202410_1_833_210 (state Temporary) 2024.10.22 02:11:04.812847 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} MergeTask::MergeProjectionsStage: Merge sorted 12045 rows, containing 13 columns (13 merged, 0 gathered) in 3.117149082 sec., 3864.1077738482063 rows/sec., 787.15 KiB/sec. 2024.10.22 02:11:04.879057 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} MergedBlockOutputStream: filled checksums 202410_1207_1402_39 (state Temporary) 2024.10.22 02:11:05.103069 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 103 entries to flush up to offset 90286 2024.10.22 02:11:05.180768 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:05.188945 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:11:05.203422 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1402_39 to 202410_1207_1402_39 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:05.207809 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1402_39} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1397_38, 202410_1402_1402_0] -> 202410_1207_1402_39 2024.10.22 02:11:05.281521 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_833_210} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_833_210 to 202410_1_833_210 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:05.291614 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_833_210} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_832_209, 202410_833_833_0] -> 202410_1_833_210 2024.10.22 02:11:05.294469 [ 1277 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:11:05.346592 [ 1277 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:11:05.362792 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.50 MiB. 2024.10.22 02:11:05.370270 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.75 MiB. 2024.10.22 02:11:05.465562 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:05.468278 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:11:05.532347 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 15.77 MiB, will set to 1.59 GiB (RSS), difference: 8.17 MiB 2024.10.22 02:11:05.532538 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_789_789_0 (state Temporary) 2024.10.22 02:11:05.543842 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_810_810_0 (state Temporary) 2024.10.22 02:11:05.566957 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 91252 2024.10.22 02:11:05.655531 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 6684 2024.10.22 02:11:05.700710 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_789_789_0 to 202410_796_796_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:05.744511 [ 1156 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_791_791_0 to 202410_796_796_0 2024.10.22 02:11:05.761026 [ 1156 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:05.764043 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:11:05.803477 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_810_810_0 to 202410_816_816_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:05.861110 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1187993 2024.10.22 02:11:05.888822 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_791_791_0 to 202410_796_796_0 into Compact with storage Full 2024.10.22 02:11:05.929615 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:05.967566 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:05.969125 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:11:06.013710 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1390_1390_0 (state Temporary) 2024.10.22 02:11:06.041428 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90286 2024.10.22 02:11:06.043360 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_791_791_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:11:06.057575 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_792_792_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:11:06.070477 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_793_793_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:11:06.103484 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_794_794_0, total 1542 rows starting from the beginning of the part 2024.10.22 02:11:06.105815 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1390_1390_0 to 202410_1403_1403_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:06.141636 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_795_795_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:11:06.176363 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_796_796_0, total 1708 rows starting from the beginning of the part 2024.10.22 02:11:06.303721 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91252 2024.10.22 02:11:06.668822 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 685 2024.10.22 02:11:06.706988 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_773_170, 202410_774_774_0, 202410_775_775_0, 202410_776_776_0, 202410_777_777_0, 202410_778_778_0] 2024.10.22 02:11:06.709322 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_773_170, 202410_774_774_0, 202410_775_775_0, 202410_776_776_0, 202410_777_777_0, 202410_778_778_0] 2024.10.22 02:11:06.745039 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_773_170, 202410_774_774_0, 202410_775_775_0, 202410_776_776_0, 202410_777_777_0, 202410_778_778_0] 2024.10.22 02:11:07.179821 [ 1163 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 214 ms. 2024.10.22 02:11:07.241253 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:07.251616 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:11:07.322577 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_667_667_0 (state Temporary) 2024.10.22 02:11:07.499838 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_667_667_0 to 202410_677_677_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:07.680232 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 13.76 MiB, will set to 1.59 GiB (RSS), difference: -7.83 MiB 2024.10.22 02:11:07.720452 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} MergeTask::MergeProjectionsStage: Merge sorted 8255 rows, containing 4 columns (4 merged, 0 gathered) in 1.843129445 sec., 4478.795573687989 rows/sec., 69.98 KiB/sec. 2024.10.22 02:11:07.758440 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} MergedBlockOutputStream: filled checksums 202410_791_796_1 (state Temporary) 2024.10.22 02:11:07.797038 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 685 2024.10.22 02:11:07.801016 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_791_796_1 to 202410_791_796_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:07.822473 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_791_796_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_791_791_0, 202410_796_796_0] -> 202410_791_796_1 2024.10.22 02:11:07.863042 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:11:08.233029 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_164_27 to 202410_1_164_27 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:08.239906 [ 1385 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_164_27} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 7 parts: [202410_1_154_26, 202410_164_164_0] -> 202410_1_164_27 2024.10.22 02:11:09.031234 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 68.87 MiB. 2024.10.22 02:11:09.686825 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 10.13 MiB, will set to 1.58 GiB (RSS), difference: 13.03 MiB 2024.10.22 02:11:10.315982 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 91294 2024.10.22 02:11:10.376212 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vfy/qaxpxrrkahsptutwfluembgciztss. 2024.10.22 02:11:10.567979 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:10.572017 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:11:10.605436 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vfy/qaxpxrrkahsptutwfluembgciztss. 2024.10.22 02:11:10.747879 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1391_1391_0 (state Temporary) 2024.10.22 02:11:10.793575 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1391_1391_0 to 202410_1404_1404_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:10.958890 [ 1151 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 02:11:10.970339 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91294 2024.10.22 02:11:11.196364 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 31 entries to flush up to offset 12557 2024.10.22 02:11:11.256399 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:11:11.259479 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:11:11.499651 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:11.502844 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:11:11.624431 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/thv/eqzkchuvdxtsakojyomcelmwdvite. 2024.10.22 02:11:11.764024 [ 1419 ] {} S3ObjectStorage: Objects with paths [common/bno/cwmgiehmlrynpcpmdvmsipuegjxnt, common/pjj/hjsvehhyjhjoeenqhcrzzbodocthg, common/uyt/cfovhhjxiywaxsbgijwqlqypstqpi, common/unb/vgxsddvguwdickdnpaqwxiaebxgsp, common/wps/vevppuwknurwqhnvfeqeitmljhnxx, common/hyz/ovvboppxndnhdyyawxugibvldkize, common/pmw/ezdqixunwoiciweoqocnybgqlvwip, common/cin/omkvbnchctyuifyvzeenhxqpwyymn, common/aqe/ryjtkqyipnkuocpuugbebpmelrzxq, common/pgi/mpseqkchgtdentdwweltgqodwvmjs, common/lfl/xftrnoleitswtbktlrbfpzhhxuxfy, common/smk/ytzfhsceysxtsjffddcrvaxtzmppd, common/rux/tazvrpvlegwjljfsmzaaygbywlvok, common/szp/dahhramrxjwtqdcyrrpnkjysbjhrp, common/low/flitzntiqxwdgnhanojswyxvkhuuh, common/ytk/ttlbkkykfwnlassxierztoswkybqd, common/bpu/phjmtjxegzhcsylnjgknwptpbqgtg, common/abw/wchmdyughbhllbwuxdswfvbuyqjsk, common/scc/bdkgdvcmynbjbwegysjhwukxwjtsd, common/ewz/vvzkkectmxfolxassbxcynozqgnfc, common/krf/kfvqmbxouinfsrfgkadnmkcgvpbmz, common/fcs/xceuzjjtpepfrzdhojugvoonvtxwn, common/nkh/hjtwsaukknjhvarxfaqngfrhpznop, common/rtq/jqolfxkphlvxuumtpxsqwalbqvyso, common/wpb/rygzposrwlinwwamajbannnlfhmrn, common/mzs/hrkgjbvowscwkddhnexvxfeioczfl, common/vbc/lhvhtgcrpuinfhbjnxanxparvpjxk, common/lgw/uqkdmlisjsmznsxafksgjxzxqfdeh, common/hgu/iqutyetclypovgliptdjqyyzhfyev, common/rjd/rncnghndgaljoalkxppktpnlyqkzo, common/nty/xinbpngliaxxghpfjelgjleknwusq, common/quj/hdufpknvwrembtwasacoyvxtqfmxl, common/gvk/qumamncwnhskyxezeynfuutgffukz, common/faw/axervyulfrmwxfttcyugufcpkysqt, common/uio/ofcgkhuxqsxhceyacfnfqgmqiqkom, common/vzq/whpqniwxrwxtsqfelhkdanbkemtmh, common/rrx/wtkwbzcngknotuiicjethafvjmejb, common/pox/ezbsqobjifkfdlfwkojknosvwdwfp, common/wrp/aqediqsgymodxlxvukqmrtagxqayh, common/nte/cicwqffgkiqameotdynusoqrdsplk, common/goo/psvwpawzsmspeuukszhawwomelqwn, common/nzn/ywhhpllnlwdsazjsrkqfutfonzqnx, common/kmj/shlgvpzajakgjtsokxpusuihaieco, common/ldm/mdjatllcplnddvisakvdkgxqhqwtn, common/lgx/tpraceudoxnpyeseullreubsksyfj, common/iec/toujlsdjizyjzvdwqporxpezitklw, common/nnk/kztlnhpitbzwkequbjymjqipfgcrz, common/djr/dfvoodftvujnnylolitllsseypijc, common/ngu/nillkagrleqieetbggxtfpztjtvvu, common/kju/fcfqjksyiadihwuywbuxmjsvogiou, common/gcc/zqugthxmjnzhrpzgnvcvsytaqqnmy, common/czs/kdpnrbpccuhrexcknawzrlvccberp, common/wyx/aemjgtoskyiwqdeksfueatuizcszn, common/itc/ueslvwyjenxuxrzczhjrffwxxqpif, common/ivg/nqrjkroagquberaxhlvgrxisqukuu, common/lms/fhpbiqjvbafofzmwmwfyyiqlugkjy, common/ttv/rwidqbyorabonscydeufzlzztbdwh, common/fve/idkwtszvtvljuqajjqfwuuymaxvwg, common/cbd/ajrurigzinbtzhfptbwpyrwdgdfwz, common/plb/xsipftlvdmzcpyagwhwrcnhaybgmk, common/gsu/trbvuznfrvekoedmexkilmyypbsth, common/ioq/sjwvolgzlvjtspvwuzmlwfodjernv, common/gaf/plzolybbhtmylyacvvwgpmnqgiwqh, common/lgf/lbqtegqmqkdekkexhlfoyphdovvhw, common/uce/njxcjngzwmfgnaohumcfzsenxcsbl, common/cju/jbeyvabzcejptnsgvbykhvmdfdhxy, common/xxg/gfpmhbqqjliaejwuqiidoykvidime, common/dlt/vororwaokotwcjmnyvkmdlehnusqv, common/znx/wlrfqqkvylsanbxdmekdnltwddocy, common/vmv/qwsbiqsrsymscrinqsyncxemijmxz, common/aej/gmzbzmkixkbjfpmbpditsiofyaycc, common/ssq/zylydnihgcbrcvrbgzcihemhuhlwx, common/tyt/ufsfoogryuekhyrpgnqewviymyigv, common/orj/xywrrobblvktxrbpnlutnarybkaxx, common/tgz/ycqxcwiupmlykrgzrkzdaathvsijq, common/waw/ntduspqdlartgqtsmyfcurbjqsgry, common/wma/yzzufqxjjdiexqbtlfwxzipgkfabo, common/sak/vyfrtosrfvlcduqlcaderqrcybhcn, common/bbp/fzxjrbhukayjjxfottxoadvdugzzy, common/qbi/lfokvtsozlvbilgwwaigdlclcfhtk, common/ysy/pomehfazjnnpnlldvwtljxnhtnlwl, common/hst/qbjtqhmofrpscplqjyjyjromasuwg, common/jxt/aiqckpcxwkaiqdxixyzhidcktorwm, common/jlf/zlswiacehrafdztegbiykdymtlgqf, common/ian/vamdsjwpsrrcgrzzjbghikbtmkvae, common/mze/jkrsxegwuycdxpyitwelevmcpdgqq, common/gas/dzpbqslfapawywwwkzkmiebjppnwk, common/usg/wipbrzvuzpdvpvilnhlgprsxdhlpo, common/xlf/mfjztjiiccjhyuvbcggywtsjjevjf, common/ecv/tlpjqtmcjyywsntssyzotujzyqcwa, common/zbt/fdokdjjqdemqiypwjcfbtplgjqnye, common/aev/sieibhwfuipmnxxvonvbemezixqzx, common/ips/rroddgkqbnvwwrsangkzibdfvdovn, common/bxw/hyfhegsjkicxjmkpxdmmsjlsbnrzw, common/xfu/clmpsdjlzobxajlaoijervqidttil, common/poa/byyibfsvatllmlyuvymvqiwdgzodk, common/lro/idewmkhvhscvknvlhmzidxcovdfdq, common/oxy/yanbjdkrdomuszlljoeqdvadmbroq, common/iul/ykmpbhcdizhvitvmvdumhevarlgke, common/gfq/pnsqywevkcvpmrsgiybcghhgcdmeo, common/stu/yidnbanfbsleexeukpmvifyizrrfx, common/onk/kugfnpfqlhvmoxltcmmlnsazdnpna, common/myd/mxhykujrcqlcijywlnszpqwziaivp, common/yqy/iwexyajfwblzinrysummxlwlgqrgg, common/yos/wmflrmdwpznbtyemxrmrsxajrapsi, common/oao/sahfiznuaehdehatiznoomaobbesr, common/rin/npxuwktuipkvjffeucvtnuandrroy, common/qfm/opcatxosruwmbvikywonaxrwodguv, common/oyp/luomrhdzixhucohejahxdwojkmrji, common/daj/jjlvrazwxcklylceunkgenqcnhcaq, common/fhn/gdktxxujfujmkbipxrhrpdskujlyw, common/onw/suzayzdaauxeklhwesxnxiaopatyd, common/pkk/kuuwhyaqnfwyphpvgrjuudvjdndwz, common/vmi/aeuakvyppsggphavegtdehgkqldmc, common/xme/dgbcdlrptdosdmzypfempbzrkcwad, common/olb/ffqrrxttqmodctfkyiufudivvotij, common/azl/axcbgkzjgmqisxsrjhugatynunarf, common/imx/kqnaathrqhdwuvtkrfyskswnnrmge, common/ssb/nbvdxqisoivvonrpwnjjvdmovubrz, common/fnj/krfodkonvbiujcktoebcixnnpvgus, common/nbc/hoxnkwnpyfeicsaoolskbuzapnoac, common/gsu/lzgijiuhmewqjovuijxxsyxkjftzz, common/hpa/lvljynelkbwktchtxyqbeovrvlbei, common/fno/srfjazehejkqdpvzoblyhethcmxuj, common/cji/yuczilkaygohxrohjxocrqirfuryq, common/nkb/hfpjwkmhqrsnkejviuovhgsslpyms, common/pcd/dhriantmkbapxqztnxdhhppphvmsz, common/uea/mqvttnjpzjypiovfxkwackdmacahn, common/zzp/qydecsaczivyjymuhmkjqqvrtymzs, common/git/sqnihcndpphsicbckuxubbkspmmvv, common/vsu/hllansnyozbrcxxjmznantfrvcdwe, common/qpk/ghmsqzmmqakrpkyokzrzoevgxljnn, common/unh/jnojwhibggkvoezesoglszroupcco, common/fhn/lqfsieiycyiuiivcibpegxejuawjd, common/xbw/wunkijfciszsfqqwbhxkzqepuxbwb, common/cdw/sjywocjgnnqbqjiqkdprdobhgdjmt, common/gxz/lekwzqkxnoskuahcqyzvkhzkjvndh, common/ess/xzejnnydzqaidrtyvurdhhxnwcmfg, common/zdr/wpwiqyqfxffqcdpocjlsoxdnenytu, common/lwo/qwzmwreshlbnuhcqtgdtmfydwdxmc, common/pyp/yoyedixbibzpkylmpptshdhjioeap, common/ngm/ednxyopwdewcfhwrhkrwyrgxttuxu, common/doa/ttkfuqgztchggycaekqyvkpskeqzt, common/nin/zipwgfakbkumysmvzhzujxnojrfmo, common/gxx/ybczqxfwasskquwveqvevvzpsubyi, common/fnd/dwacvognkpcgfgjmxjzmrkpzziayt, common/tqi/lzxckawujixxtjhwhjrjplqcjrzbq, common/gfn/joxhclxfvpzimgcgttsqzqtdrjdmf, common/rwp/qodkpqyxkhrbkksewjwlonuplbshf, common/tds/xyjsfkunyzjkjuvjdxflkvppyvdwp, common/jzm/zchbqcuckqefxaefgrtxpulrxfurj, common/yww/suvabxvxottvdkydselkmknxpuxva, common/ixo/patlrdcwkapvzmqohocniethmosco, common/sxm/jdipbkbkvtrofeizdwmkutustatxm, common/alm/ukldypsofhcvjmdwycfkwbfvqbgrp, common/uzm/hivdmccejxazlnnileslhwsaxpemg, common/qcn/zlkbbnyjihmczgzjsyzoownoyktox, common/nju/oytzadmgtaexkdupcsccrtlmtzfdn, common/psa/uqbcswocrwiizrhqsbzlnmapvabfg, common/xsw/ysxyvbtqsxsnkykjdspzvuiwobdcj, common/pup/edavwifqrfljcliambfadonxckdky, common/tcz/zqbqqndeaavcfcqxcoplmflzhhtse, common/kgv/houqviwhzsljrwbxctnlwwoidfkum, common/mxi/fouciyphxepckoywwjamqljvungjn, common/fbo/dkcdaqhxysjxibwjiljgdgwkozjxt, common/uga/chcpcigjhrpuucstwcmhlpdwmzpzb, common/akg/pqecbcsewadllgpytbrjqundshyhm, common/jyr/syykgjzauckfcctogttdscyegegxz, common/gms/cgewzebzoelynryeezbfogoszczug, common/vhy/aconhehkevksgbpktgbgpssqmmkjy, common/hho/daprgcursgrevkfajcnpepkzzebqk, common/vac/ipkjxrahzezhfagbxemzjhhtchrmi, common/amt/ockhhvtqgvyqgvjehkeuozzxpvanb, common/fbt/vtxxvspnzfuzdtkgnzahijrmzdafr, common/cqr/gqubrtpyjvwiyipxgzrjatznkznvw, common/ivh/bwqqhzcqvchvmlkyileigetmkkuvi, common/ntx/hvybvaeomroojzwyaafmpvcohvocl, common/pky/uwvpmnbdkowyqkmjcuhhajaakacgf, common/mph/bnmxyuirvyzwgvjwqppallfmpbvdf, common/npt/gyfsycsbjhebdgdbxewojjkxxqdst, common/faa/flddxghasiymlrarsjtqvndrcjvez, common/dhb/vlfixhsnknsbzghuuawqynemgqfqj, common/dda/degrhrjuuwvcwqxwmtpycxxzgsrot, common/vwv/yhovggvxoildhzskpxmdwgmzhabhe, common/phd/idqxojwyoecwiyiexucsgetsqxtep, common/dwx/fmphizawlewsnljfoowlycrnlbqvl, common/gyz/xtjsnbrnciptbzfrhjisgvltnetvg, common/vne/plyufalgpsoonoefcbdzdqqyrhzkp, common/jcy/bmxaredmratqjgimalfjpfurpswgj, common/jhb/fmdkkziutqruleeeqpdxpdjllzuwu, common/abl/ikztikehatwyxsahearbhqmamimza, common/bzc/hgktwfmcvwlmmoqckduqumrksqyks, common/orj/mauqvhgzrlioaxzlrjqlyrguaawll, common/sxj/fuusvivdrhrgiejgwhvhxgmpxuhga, common/vtk/eksbwoonasounnotxlshblpubytni, common/aab/hwbvcgiqrbgarmulywietkypnlqry, common/kzy/bqtzkcmvainontfohtfisvnhavrvf, common/mpt/fxwhuhztssekyliuswskjtsmmybjr, common/fxk/qzeagnheckwesuwfrfvkjgcgvojrf, common/xbc/engmpyrvrmjychlyefilrsmnxvoyu, common/djg/bonagcupzdafreokcdgsauhcrcaut, common/bwd/dgrissfptfcuewypmjvdwtrvllfkc, common/myd/vsomijlnnrzeprvzuhntpivlbwotj, common/vob/yhstcnpmhnomxanorrzwwtyvimcih, common/axm/bsdlpamvugjusjdtornxmljejvooi, common/mnw/atcdkiivgpldvermqmofdvvoczwha, common/yqk/ppmiftwrarpdrszsldttsdlmzuaqd, common/uib/qrxouddmytecofrunfjcehngqhlwl, common/ynd/gumsxuhkltaxogeqdajnywzdghwij, common/lou/fcbnztotxkgyjcgwnywparmoaxbmx, common/tvm/eugulonvpsopmzugxljzaioqhxuro, common/bqs/cxkfsivctgerrysqncskoomsrdigi, common/ybu/jeijojikormkykcjaoqkghzgappav, common/odt/qivznuvrstbbcmbmsixsdkyanozoi, common/hsg/sxdhdoujbkhcfitdoqeibuuxddlwq, common/ozk/uwyabvbqxgubmczqwomothzyihcgx, common/ete/hvvvclhqntelqxgwnrymjsahaygng, common/cee/icxlqwmksviegkrgdbuzrepptoxfu, common/amq/fstohmnjxqtcklvuirhsnjlzuaqer, common/yjr/zjswcxayahxlkcrfuneiyvqnjzdju, common/phq/mrdupnzdvryjiqmowvvclfvaapdbc, common/qwb/hfasafzbnkqewvxypkofogsyqcfzu, common/ppi/mmylqaetnmpphxgcqcvhlkbqgngun, common/usr/lvewtlyuomfeauibwbotwwohjotjd, common/nro/rkjfcadbhmffnqhknpxcnemqqtfeq, common/wlr/qjvkmowzljgjasfoqkovxfoxepcnr, common/dyg/efhdhxtxgnhfetugayqbglkichrdn, common/vpl/rcmxkdmnbmtjnazewjympavprntnr, common/iek/ddofbyhxomezhzcsuczzsyueszewm, common/rsj/wbhqbspxijcsfjajldjnrotlqwpzx, common/cwt/xifdpzyjbvxtlphqesxbsgjuturhf, common/fxo/qxkmhhlnyvqwteskkyrtktrsouznu, common/meg/vbjdhfciqivkzhnxnzmaxbmfjwryq, common/osu/kuewepbtxhutzjymjnsibhcoyrjkf, common/ahq/umxazvqflfrsiuiemnlvpfysqywgk, common/vet/fnrlxkeowehvhegqyyurnvjugywnv, common/lup/pknhryerniaqowqtxinmpzedbfriq, common/ock/ayxdgfqiuhfjldrzixleiubfceokr, common/qaj/nfqmqgavniffjmbslivxmcflkqwbx, common/bch/hnnzqsvmtebezdcvcvhshbdbkjlcu, common/gdq/mnqhplshfjevmtxpgdcqzhldwisox, common/uxs/jcdazcqtngfdalazozdmcuqxrnbtn, common/pou/pfrpaosfexmwbnzpwmnhmtfvgqjha, common/ncd/yzkvynarcuvxcvilbxfgdezdirjok, common/htn/qjofeqjthwpyjsjncjaxzljarfoaf, common/ouv/ibpistfgypzdfzvyvpfbuylpoboki, common/gjd/lahwgwztqfqfqqgfxsdxpapqgqhwn, common/hqs/rpecghwvkmglkmaukpbkzogqytgib, common/vle/tcrbtchwtwbwczsbiyllnqhuybduq, common/viy/uakedcubyjeyrrgairksavhewtwdg, common/fxx/prmyurdlgduvbyrimizaxjwlklumd, common/kng/wdpfianmvukgtoolkshoeetfnfnku, common/dvk/svhxxuxzykydfmlsoashkqfhltikz, common/qjm/fojknpxkkjdbgtoiydtskwsakxqln, common/frc/wviydifiptnhpnixhianiwfsvfojr, common/gly/vtequlgtyvwlbqyxaxndoujsdepsy, common/oql/kunabmmqevkohfcyzeuwfriebehyn, common/aij/duexvclwtjslgfydertebiitytjkz, common/bvy/nyqkvnejwvjsmslluimrjbfkudnua, common/pta/punmkubpawebikhdcbvnqksrhhyyx, common/ten/grjuuigtvsirxhydtavvfzgjzgvmg, common/bcm/sscjbwymxoebfnjmcouqqwymsnrph, common/voa/tyjyywxchaeqvzkfhxbeudmwjhjux, common/jph/ztenveusnqvxmxnqcmayhtypcjiwb, common/mmj/fjnjmzglglegtenpspoalwytqdjce, common/wkw/yfztfkhvizerlwcouajiuxxpkphtp, common/flt/lildpdnijckvzznpmyerqjheklxwx, common/sby/tmbozegbnixwmrdqkexolygnamprf, common/kln/npoqagokymhzehjlxkwcknyqfsbva, common/kym/kalueasfqzmiiggboygiqpmjhmpqz, common/jiw/vpogklaihycqhstlymfhsjwjzjgzw, common/skv/ugjnjzzxaoxrubikenqscqpqlsafo, common/jdc/itjygbbnrwwgukuuztkgqwvjlfdbg, common/ilv/uqbmcmbftojzevtjgjfbtkythhucm, common/our/vsfajspkrzyrgplygtdugyzuwqmgu, common/vpj/yqawciqntnmydvmzvvkdxwxbdnwcz, common/nwq/slqxfmgnidbphwnphnrklekrkgraj, common/ozb/hxgpalojlikpidpuudpdvmiczclhm, common/ugp/qinphqgyfyjlubwalzigyfxxzaqas, common/hts/deqtlkkekbbxddgpkjysbvrwzfvza, common/dle/emndnbqkbvyorkmkmipuzcfisdsnd, common/spl/zklhtznifrcfgmgnoxbjjghuieqbh, common/fgk/dlwfwimpkmaweiccuohckrihzzzzn, common/cbo/tqzelqmcroebywfccxlovrmpjdqeu, common/ctl/kzhrwijtenqtuxzzokqpckznjxkug, common/vry/cemommrmdbmlgepabtqxkavrojjzv, common/bld/ysfhubohvkdvwqlvqkdesdinxpjct, common/ymb/jyyxqzyledyhidiwqenoclaqlfbgm, common/jkk/lpdlvkwnrlsfkcoigvaievaangjsf, common/ldw/mjngjkjqljtnmqsmfnmszvegucmkm, common/tlh/ztjykrmnlaotowzusdncauepuuvha, common/duv/xfbgztwevgwqpdwdereavvqsctwli, common/zsq/cestgdklhycazmsbfebaeroqdppdw, common/evc/fswhrvoajlsgdqudmlhpblkwkwpkq, common/hgv/ogtpziwwhannqmlpnjrdttbiqerzv, common/exv/wzpxjxxkaeddmqzyhpfjkpmdqhosj, common/zvw/kaozkolkhbvysyiaeuvbgmcaslavi, common/ofp/cejdutlfaaqcfmntnmadaldfzmger, common/jpm/fuxbggoelxvwcngenaeiuouhbknkb, common/lix/vhaommhjwpnqafbhogmivmuljasjh, common/ehf/nwmfvlsicjbxwimaqskrbclasejuj, common/wiv/imbwjzswjmeecoizfuevqdtngkzjp, common/kge/wjshaztonlmngmuwszcdijviczybi, common/ooa/soyqvocnycbozbjtmghmhejabtqad, common/cdw/khhmninmqfomoucdrbcngtwniycxq, common/cnt/uyvnzbrqrepbtyhkosstgssjcdoye, common/nws/jwzvntkmanqnurupckdzjqmxuhgzz, common/rlh/wnpxsnsmaamlkftdlvjphhassuyyi, common/eis/vkybutzntspvwaiwgdellnyoyvmia, common/kmw/xjrafnhtvgweokkgijudkwwvhtkse, common/mop/cjfebwbtrlyhrwefprvizwkjhsnif, common/yfk/zetmjgwvlltpkiqyourmiufypngfl, common/yby/witwxhjaknihpmhpekdjkjsnqkyku, common/fxa/ykkksnyusahvbjkbmlvuxmuczscjx, common/tvm/tigxbvlmjqpdmbkvhruxzcdieyfnc, common/nnl/jbvffvxkmmabixbcqjlodwkfuihaq, common/rco/kffjdirpwscpxpgdwfeptzivnbvva, common/sly/gqevzedzmjjovpgpjieachfpetipx, common/lsj/mnhzninsqmjsccnjxwabhrnpmpbjh, common/lur/gmeibkzotfjdcixyckhbxwtiifrze, common/mho/kiuuzmhyroqhlepitvfltwbbrfbvn, common/rml/cputefhadxjhtzbfdsxqccqjbmxaz, common/wcl/ephmolxenknxaeazoeacedhzxbzof, common/ppj/sypnqauvxevzxqzxkuawystbuxxmr, common/cxz/fixfcgumgqpakguccoxpiosjymlnw, common/rjz/vysaexgqaxkuyxrzhrhckchywppsd, common/ran/gwdzuacmrgxfwmddxtmruuihtdvbp, common/fad/nfvomusnwnsheotbdqgwyhdrrfpxi, common/iet/tlienqiacbprixiyvxfixhbwjxjfp, common/hrl/objpcucypkumtnxzsymygidvnqdhr, common/wpf/fdryuocfoooaeobpuooopgfayzznc, common/kiy/qzzgxtafkpyatszrgmicchzaugcnp, common/cor/xkmzatcqvmqxqmbhogqempmisyfvd, common/kem/eymziiqarghlamendcneerpvwznnw, common/aai/fyhmrltqzoqhfroqbbofctygccuex, common/bet/dncavgkbqsheuqxcuvsgipfxgpnjs, common/cea/wmhovttnardhuzdnlvarixbjteftw, common/wvp/wscdlgsgyhmizjfbtqjfxuastzolz, common/vyl/jsenjxxebptplpvzpwuanybapqcyd, common/ahd/oqvhzpukundlpxschbknpctxdqnhg, common/omu/bfdflwhirchxccjcwiqephxfctooe, common/cft/nnekcwzlkijldhhfhftbwtzbvdxvr, common/bir/npmqluicagtimtooudloaeotsppew, common/ipk/mkqogvdqkjondaeysdlkyzxkcfqvu, common/muc/ccwazpxwawhalmuikdvnmpncbnakf, common/bwo/ywozmidsfvitotuejvdepfugocelf, common/ysg/rqgifwzkxlklcfzfrtrlvxfbejjon, common/jcr/njeohokhyuydupenedlzqbwisqyxo, common/ueh/aorunjdndohtzhydkmprkydxsdcrg, common/lmn/dttjcfrhaausaztslidveimtxomvm, common/hrj/wcglipakfbaziexiqrrsvmmlekejs, common/psj/iuzjnehznnhpwfojohuamjqvqtinw, common/nnl/gpdppspwffdheespzqcvfccrhkzsb, common/zhm/wuwchvyxomjysidzuttsetiapareh, common/brb/wakvdiaiczkscktncsynhfcqmdpab, common/ilk/dutpqfzbnsvlbtmdbmbbqeucxgoxy, common/xgr/edazgbefwhlkctvlnjhrszrzkcrxx, common/gsk/emfiqjgqrzbhajzimijqqxuxubzbe, common/jrs/cbreowjxgjavdhvkcxbjbaotwbogq, common/gcn/przyvmisebljwkckpxlzindemeety, common/yam/mhbesezqdaznihpchdxxssrgedwur] were removed from S3 2024.10.22 02:11:11.817770 [ 1419 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_15_15_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:11:11.854589 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_825_825_0 (state Temporary) 2024.10.22 02:11:11.895705 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_825_825_0 to 202410_838_838_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:11.959526 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12557 2024.10.22 02:11:12.036650 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/thv/eqzkchuvdxtsakojyomcelmwdvite. 2024.10.22 02:11:12.056435 [ 1225 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_833_210 to 202410_838_838_0 2024.10.22 02:11:12.064467 [ 1225 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:12.067833 [ 1225 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:11:12.171542 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_833_210 to 202410_838_838_0 into Compact with storage Full 2024.10.22 02:11:12.253694 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/iwu/tdlhrfljubpzccqxdmrlclztopijj. 2024.10.22 02:11:12.255753 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:12.268953 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_833_210, total 12573 rows starting from the beginning of the part 2024.10.22 02:11:12.307112 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} MergeTreeSequentialSource: Reading 2 marks from part 202410_834_834_0, total 23 rows starting from the beginning of the part 2024.10.22 02:11:12.461471 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} MergeTreeSequentialSource: Reading 2 marks from part 202410_835_835_0, total 19 rows starting from the beginning of the part 2024.10.22 02:11:12.502336 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} MergeTreeSequentialSource: Reading 2 marks from part 202410_836_836_0, total 7 rows starting from the beginning of the part 2024.10.22 02:11:12.555153 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} MergeTreeSequentialSource: Reading 2 marks from part 202410_837_837_0, total 10 rows starting from the beginning of the part 2024.10.22 02:11:12.578776 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} MergeTreeSequentialSource: Reading 2 marks from part 202410_838_838_0, total 31 rows starting from the beginning of the part 2024.10.22 02:11:12.697791 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/iwu/tdlhrfljubpzccqxdmrlclztopijj. 2024.10.22 02:11:12.799033 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.38 MiB, will set to 1.58 GiB (RSS), difference: 9.09 MiB 2024.10.22 02:11:12.865218 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1372 entries to flush up to offset 1189365 2024.10.22 02:11:13.547761 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/cmm/wwzwchshbpxxslxqvxvyafkxetnlm. 2024.10.22 02:11:13.558039 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 90370 2024.10.22 02:11:13.669667 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_790_149/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:11:13.806716 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:13.817506 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:11:13.855777 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/cmm/wwzwchshbpxxslxqvxvyafkxetnlm. 2024.10.22 02:11:13.886933 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_811_811_0 (state Temporary) 2024.10.22 02:11:13.937493 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_811_811_0 to 202410_817_817_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:13.968495 [ 1272 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_812_13 to 202410_817_817_0 2024.10.22 02:11:14.007120 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90370 2024.10.22 02:11:14.008184 [ 1272 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:14.020001 [ 1272 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:11:14.254545 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_812_13 to 202410_817_817_0 into Compact with storage Full 2024.10.22 02:11:14.420013 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} MergeTask::MergeProjectionsStage: Merge sorted 12663 rows, containing 12 columns (12 merged, 0 gathered) in 2.252774441 sec., 5621.068745071047 rows/sec., 897.54 KiB/sec. 2024.10.22 02:11:14.507489 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:14.512011 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:11:14.520786 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:14.532882 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} MergedBlockOutputStream: filled checksums 202410_1_838_211 (state Temporary) 2024.10.22 02:11:14.571761 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_812_13, total 6538 rows starting from the beginning of the part 2024.10.22 02:11:14.665202 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_813_813_0, total 128 rows starting from the beginning of the part 2024.10.22 02:11:14.758636 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_790_790_0 (state Temporary) 2024.10.22 02:11:14.831158 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} AWSClient: Got dangerous response with successful code 200, checking its body: ' http://localhost:11111/test/common/gnc/qomjpfjvbedldjhwdarjbkajzppwxtestcommon/gnc/qomjpfjvbedldjhwdarjbkajzppwx"037d2d977c' 2024.10.22 02:11:14.840554 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_814_814_0, total 91 rows starting from the beginning of the part 2024.10.22 02:11:14.891116 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_790_790_0 to 202410_797_797_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:14.952169 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1189365 2024.10.22 02:11:14.952887 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_815_815_0, total 71 rows starting from the beginning of the part 2024.10.22 02:11:14.979020 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 91356 2024.10.22 02:11:14.996502 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 13.68 MiB, will set to 1.58 GiB (RSS), difference: 5.73 MiB 2024.10.22 02:11:15.011273 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_838_211 to 202410_1_838_211 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:15.024489 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_838_211} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_833_210, 202410_838_838_0] -> 202410_1_838_211 2024.10.22 02:11:15.100700 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.60 MiB. 2024.10.22 02:11:15.143206 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kqt/cieukdiolxqgnecddjyxkttjkbzsh, upload id 3eb7eec4-f9e7-425a-b396-a3c05fc93ab5. 2024.10.22 02:11:15.207505 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} AWSClient: Got dangerous response with successful code 200, checking its body: ' http://localhost:11111/test/common/kqt/cieukdiolxqgnecddjyxkttjkbzshtestcommon/kqt/cieukdiolxqgnecddjyxkttjkbzsh"552326e294' 2024.10.22 02:11:15.213071 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_816_816_0, total 103 rows starting from the beginning of the part 2024.10.22 02:11:15.214547 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Multipart upload has completed. Details: bucket test, key common/kqt/cieukdiolxqgnecddjyxkttjkbzsh, upload id 3eb7eec4-f9e7-425a-b396-a3c05fc93ab5, Parts: 4 2024.10.22 02:11:15.299869 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 686 2024.10.22 02:11:15.310022 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_817_817_0, total 84 rows starting from the beginning of the part 2024.10.22 02:11:15.329724 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:15.335888 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:11:15.459825 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kqt/cieukdiolxqgnecddjyxkttjkbzsh, upload id 3eb7eec4-f9e7-425a-b396-a3c05fc93ab5. 2024.10.22 02:11:15.610627 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1392_1392_0 (state Temporary) 2024.10.22 02:11:15.668729 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1392_1392_0 to 202410_1405_1405_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:15.765239 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:15.768152 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:11:15.793203 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5fe338a91331f5aef30ae3d4a87f7527, offset: 0, size: 0 2024.10.22 02:11:15.800830 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5fe338a91331f5aef30ae3d4a87f7527 for removal 2024.10.22 02:11:15.810986 [ 1476 ] {} CacheMetadata: Key 5fe338a91331f5aef30ae3d4a87f7527 is removed from metadata 2024.10.22 02:11:15.853012 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_668_668_0 (state Temporary) 2024.10.22 02:11:16.055640 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 264 ms. 2024.10.22 02:11:16.067077 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91356 2024.10.22 02:11:16.165063 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_668_668_0 to 202410_678_678_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:16.255334 [ 1265 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_673_137 to 202410_678_678_0 2024.10.22 02:11:16.267688 [ 1265 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:16.277276 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:11:16.339368 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_673_137 to 202410_678_678_0 into Compact with storage Full 2024.10.22 02:11:16.340902 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/jud/qisashfzuxqifyrvhfepwvrdkiyuc. 2024.10.22 02:11:16.572562 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 686 2024.10.22 02:11:16.850828 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 15.41 MiB, will set to 1.58 GiB (RSS), difference: 11.40 MiB 2024.10.22 02:11:16.915353 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:17.054890 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_673_137, total 717 rows starting from the beginning of the part 2024.10.22 02:11:17.389650 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} MergeTask::MergeProjectionsStage: Merge sorted 7015 rows, containing 13 columns (13 merged, 0 gathered) in 3.252544823 sec., 2156.772736964062 rows/sec., 957.00 KiB/sec. 2024.10.22 02:11:17.608695 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} MergedBlockOutputStream: filled checksums 202410_747_817_14 (state Temporary) 2024.10.22 02:11:17.674638 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_817_14 to 202410_747_817_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:17.679750 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_817_14} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_812_13, 202410_817_817_0] -> 202410_747_817_14 2024.10.22 02:11:17.681328 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_674_674_0, total 1 rows starting from the beginning of the part 2024.10.22 02:11:17.854856 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.50 MiB. 2024.10.22 02:11:17.956477 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_675_675_0, total 1 rows starting from the beginning of the part 2024.10.22 02:11:18.193237 [ 1704 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hup/axqcwhbupyyjwdlvbgxqduuiyakdz, size 11122 2024.10.22 02:11:18.559467 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_676_676_0, total 1 rows starting from the beginning of the part 2024.10.22 02:11:18.569483 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 13.92 MiB, will set to 1.57 GiB (RSS), difference: 3.22 MiB 2024.10.22 02:11:18.741058 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_677_677_0, total 1 rows starting from the beginning of the part 2024.10.22 02:11:19.047926 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 02:11:19.048566 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_678_678_0, total 1 rows starting from the beginning of the part 2024.10.22 02:11:19.263911 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 02:11:19.273410 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 02:11:19.406681 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 02:11:19.436376 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 02:11:19.458197 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 02:11:19.462654 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 12576 2024.10.22 02:11:19.483677 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsNotBounce 2024.10.22 02:11:19.566348 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yfq/kqgtvkflytuoalbhsoohlfhixmruu (LogSeriesLimiter: on interval from 2024-10-22 02:10:59 to 2024-10-22 02:11:19 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:11:19.764220 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:19.767185 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:11:19.919581 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_826_826_0 (state Temporary) 2024.10.22 02:11:19.989445 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_826_826_0 to 202410_839_839_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:20.077821 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 91415 2024.10.22 02:11:20.148008 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12576 2024.10.22 02:11:20.355977 [ 1222 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:11:20.426687 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:11:20.521050 [ 1222 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:11:20.594721 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 11.13 MiB, will set to 1.58 GiB (RSS), difference: -6.09 MiB 2024.10.22 02:11:20.617721 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:20.624649 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:11:20.708026 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1393_1393_0 (state Temporary) 2024.10.22 02:11:20.858560 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1393_1393_0 to 202410_1406_1406_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:21.025768 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91415 2024.10.22 02:11:21.254657 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:11:21.258321 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:11:21.531254 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 98 entries to flush up to offset 90468 2024.10.22 02:11:21.923276 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} MergeTask::MergeProjectionsStage: Merge sorted 722 rows, containing 32 columns (32 merged, 0 gathered) in 5.596733314 sec., 129.0038241046695 rows/sec., 11.67 KiB/sec. 2024.10.22 02:11:21.956335 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1191290 2024.10.22 02:11:21.999193 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:22.007602 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:11:22.060655 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_812_812_0 (state Temporary) 2024.10.22 02:11:22.158988 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} MergedBlockOutputStream: filled checksums 202410_1_678_138 (state Temporary) 2024.10.22 02:11:22.241634 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_812_812_0 to 202410_818_818_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:22.303251 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90468 2024.10.22 02:11:22.357160 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 15.52 MiB, will set to 1.57 GiB (RSS), difference: -2.92 MiB 2024.10.22 02:11:22.739112 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:22.743100 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:11:22.948693 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_791_791_0 (state Temporary) 2024.10.22 02:11:23.048889 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_791_791_0 to 202410_798_798_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:23.121220 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1191290 2024.10.22 02:11:23.135359 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/jud/qisashfzuxqifyrvhfepwvrdkiyuc. 2024.10.22 02:11:23.156615 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_678_138 to 202410_1_678_138 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:23.160676 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_678_138} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_673_137, 202410_678_678_0] -> 202410_1_678_138 2024.10.22 02:11:23.228660 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:11:23.424751 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:23.434927 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:11:23.872033 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7224c683c7e4d907ede53e5c5bc41069, offset: 0, size: 0 2024.10.22 02:11:23.875813 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7224c683c7e4d907ede53e5c5bc41069 for removal 2024.10.22 02:11:23.988261 [ 1476 ] {} CacheMetadata: Key 7224c683c7e4d907ede53e5c5bc41069 is removed from metadata 2024.10.22 02:11:24.049758 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/twt/kggbxpjqkqdvlmyppvfpceyosfzny. 2024.10.22 02:11:24.077813 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 687 2024.10.22 02:11:24.098343 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 26563b59a80c026e95df5d49bf4cf6aa, offset: 0, size: 0 2024.10.22 02:11:24.101090 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 26563b59a80c026e95df5d49bf4cf6aa for removal 2024.10.22 02:11:24.108843 [ 1476 ] {} CacheMetadata: Key 26563b59a80c026e95df5d49bf4cf6aa is removed from metadata 2024.10.22 02:11:24.265103 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/twt/kggbxpjqkqdvlmyppvfpceyosfzny. 2024.10.22 02:11:24.324536 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:24.332524 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:11:24.334878 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 78e8fe78031bac7e24081e0017dec75a, offset: 0, size: 0 2024.10.22 02:11:24.338396 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 78e8fe78031bac7e24081e0017dec75a for removal 2024.10.22 02:11:24.343804 [ 1476 ] {} CacheMetadata: Key 78e8fe78031bac7e24081e0017dec75a is removed from metadata 2024.10.22 02:11:24.453745 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 15.60 MiB, will set to 1.58 GiB (RSS), difference: 19.66 MiB 2024.10.22 02:11:24.544458 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f8936e9a207c2cb6582aa6f6296cb315, offset: 0, size: 0 2024.10.22 02:11:24.547394 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f8936e9a207c2cb6582aa6f6296cb315 for removal 2024.10.22 02:11:24.560635 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_790_149/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:11:24.560890 [ 1476 ] {} CacheMetadata: Key f8936e9a207c2cb6582aa6f6296cb315 is removed from metadata 2024.10.22 02:11:24.568992 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_669_669_0 (state Temporary) 2024.10.22 02:11:24.625610 [ 1171 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 252 ms. 2024.10.22 02:11:24.834596 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 72cd5490af0710e6e47b150babe9aefe, offset: 0, size: 0 2024.10.22 02:11:24.839530 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 72cd5490af0710e6e47b150babe9aefe for removal 2024.10.22 02:11:24.844467 [ 1476 ] {} CacheMetadata: Key 72cd5490af0710e6e47b150babe9aefe is removed from metadata 2024.10.22 02:11:24.867329 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_669_669_0 to 202410_679_679_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:24.983036 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5293158adbe19cdfd587aadf03f0f407, offset: 0, size: 0 2024.10.22 02:11:24.986769 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5293158adbe19cdfd587aadf03f0f407 for removal 2024.10.22 02:11:25.003993 [ 1476 ] {} CacheMetadata: Key 5293158adbe19cdfd587aadf03f0f407 is removed from metadata 2024.10.22 02:11:25.044172 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 91477 2024.10.22 02:11:25.248423 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2d429385fa389a92f1f9be112574fa0a, offset: 0, size: 0 2024.10.22 02:11:25.251209 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2d429385fa389a92f1f9be112574fa0a for removal 2024.10.22 02:11:25.256984 [ 1476 ] {} CacheMetadata: Key 2d429385fa389a92f1f9be112574fa0a is removed from metadata 2024.10.22 02:11:25.291073 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:25.299258 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:11:25.327445 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lji/qeahogkxpotwmzbrlfsauzscurdyx. 2024.10.22 02:11:25.354995 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 687 2024.10.22 02:11:25.455633 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1394_1394_0 (state Temporary) 2024.10.22 02:11:25.478262 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3dc363cdc529598442857f5e3b75381e, offset: 0, size: 0 2024.10.22 02:11:25.487105 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3dc363cdc529598442857f5e3b75381e for removal 2024.10.22 02:11:25.497227 [ 1476 ] {} CacheMetadata: Key 3dc363cdc529598442857f5e3b75381e is removed from metadata 2024.10.22 02:11:25.526465 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1394_1394_0 to 202410_1407_1407_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:25.563229 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1402_39 to 202410_1407_1407_0 2024.10.22 02:11:25.571042 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91477 2024.10.22 02:11:25.572817 [ 1172 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:25.668634 [ 1172 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:11:25.681128 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1402_39 to 202410_1407_1407_0 into Compact with storage Full 2024.10.22 02:11:25.696642 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lji/qeahogkxpotwmzbrlfsauzscurdyx. 2024.10.22 02:11:25.699825 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qvy/hlkywmoyhijywhrfirejnycgvzqjv. 2024.10.22 02:11:25.764164 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 44715bb5e051bfb6b9605d1c58ee8fab, offset: 0, size: 0 2024.10.22 02:11:25.779357 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 44715bb5e051bfb6b9605d1c58ee8fab for removal 2024.10.22 02:11:25.791879 [ 1476 ] {} CacheMetadata: Key 44715bb5e051bfb6b9605d1c58ee8fab is removed from metadata 2024.10.22 02:11:25.891093 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:25.922490 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1402_39, total 12045 rows starting from the beginning of the part 2024.10.22 02:11:25.964938 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qvy/hlkywmoyhijywhrfirejnycgvzqjv. 2024.10.22 02:11:25.983054 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_1403_1403_0, total 66 rows starting from the beginning of the part 2024.10.22 02:11:26.105628 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_1404_1404_0, total 42 rows starting from the beginning of the part 2024.10.22 02:11:26.295924 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_1405_1405_0, total 62 rows starting from the beginning of the part 2024.10.22 02:11:26.297060 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1302_19, 202410_1303_1303_0, 202410_1304_1304_0, 202410_1305_1305_0, 202410_1306_1306_0, 202410_1307_1307_0] 2024.10.22 02:11:26.332737 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1302_19, 202410_1303_1303_0, 202410_1304_1304_0, 202410_1305_1305_0, 202410_1306_1306_0, 202410_1307_1307_0] 2024.10.22 02:11:26.356302 [ 1419 ] {} S3ObjectStorage: Objects with paths [common/qtx/lxnanocoyfbzamzauizjqwmdmrksd, common/rpo/ooyayurflwzomoehhjrlymxqywpyl, common/mqk/yyhxtffjawbrbnwvjymcvdjzpffsh, common/zsb/eqoulqvgtczinswwoxuvruuuqiuxi, common/oun/xwfzabqcisytscjfapyrmuuzhmchz, common/lrg/rybjqoyvjeqjgwskrvmgnfrajsdlj, common/qry/gfqhacsvzxuufqueqqzkpouvcotfu, common/wim/vqdzzpybuoaevcgmxnyujdbgvbrpy, common/dss/xtpsrdfvohbetqkemzwvrllsdgiuo, common/dsi/rrlehpufhlqcdewrjlrabnjltpgnc, common/mbn/kdioxgvbdktmlsdiiiamfswyvoape, common/tlz/shfssdnvbaqdognhmkpbugekcbizt, common/pfd/hihqtukikixyvpczkpiytmaqpctlh, common/mvz/bsmnmulwcjqclmuicwpjayzlizfcc, common/eqc/ajxlyhnpsirqeabhtemcmnoxammrc, common/iyy/ecfpsinxwvjajflbiufjyoqpebwqm, common/gio/mytauruedcysjdsffctfgoponhbnz, common/png/ydalcqgjezmdivxpfbcpvjsxeuplq, common/mnm/kbkwoxxsnwgidclsinnawabjatknb, common/vav/sodvjtejmkhnhdwzuxlxfodyghxka, common/udn/ydcfnhpgkidzlfmhagvbpyddvgari, common/spe/kzebyvvwfjnaslqngpxggahiqctdb, common/ezx/wgzgpceomnwfdztzzkmlvhxqgzlvm, common/wrx/tlluyoltzofsssevzycoypoaiuyks, common/kjk/smopfcvhiinfhirshopkfcxqrlugk, common/kul/tawqooxtqhoyzksdsvxszhkzeflkb, common/xhc/fzvozotogpczpktctcexuehjgptqx, common/igs/zprihfaqdzruwvvcwmcswxeubbgvg, common/xgv/abxgrzdfekikgkhkygfrftzzjvktd, common/oom/expozddkdiwysqeembqcykkqswiwq, common/ped/viznnwbvmjrcndtmtnjskgxghzjwb, common/dfd/ggtyateoaplrgjyxkwpxjmkyegyel, common/eoh/whztpjdlkkkdyajmyrogopjuabxvg, common/bpk/lhrlsspvdjtivwvobwiwnpzistktz, common/nju/hxvrpneperqgdcbctnncszgsxcrxk, common/vwm/eveonnmilnodvgenxymsuqwowsjcp, common/bkp/wvkxfitcsaobfbvzefsuspwjrajfu, common/ftn/yvhrqezirvxvlzgnfhcdiljjhxhsf, common/xlh/ttjrhmsjxpjqcyegpnthnyzkxoxph, common/ctd/otduvydfbyzewcfpsmekaeostlvwl, common/wok/twxrnqjwsdxcodhchqjzebkhphpfc, common/jmj/vexfvyaqabjeqtsltcoynpsdhocfs, common/exh/fiatyihorndrylyftnokrwtzepnyd, common/hgb/pgbglybyncvgpesvasgjlydsrojhd, common/xqp/nkmewhpcwemhrvyapmhnkqrzbjlnt, common/gvy/sjqgshimlfptovhrjjjkzyokccguh, common/tia/uymumuzsuwkvpahywotdzlwggejtb, common/cxr/uwxvngicpkfjvftvvlbqxlyygteze, common/mng/omtimeigdstvsrcodakctoznfedra, common/opc/wgvmpiecycmyewcqxmqftshjqcmkq, common/xae/tethwymxkulttsbyshrjmsrmpyhnu, common/pnx/eooflzhvcxezkhttkegbnvrnyvwpf, common/itu/qjayrmbelceedfvibcmiydjjeybpr, common/kml/cwqyghyzapzapatejswiqjifoufhv, common/zyg/jmteqqsgvulresizvehswncuxsqnf, common/jpy/ghgcqgqictbbgalaldamgdjyeblfo, common/wuh/ocpwolrurognytsorqxccylmgjfla, common/hkx/udlmoufjqbfyzjhiufpbfohlpthke, common/lhg/detmmusrttjisjywqplvcddtvefuv, common/jmj/zgryjxpyogstzircxrvsnvkwgnvop, common/yfl/aaxecmmkwrpzzsjrvbhukmacwopbh, common/onn/noyhmntjcvsabdlydttyrfscikqjb, common/qhx/prifiiqnuuxrgqjjoirzgokzxjffp, common/gyv/iimyaefirelsdulpghkbqedtsumoi, common/hxe/yvkggfeiohxxflwmxurmpboxndnqd, common/wcs/uzaeffsljisoqdckzxggusnmcowex, common/wxl/qmlmgbavpfradyjxthybcsehhbbpc, common/dmr/cptihokkbwwpvenylpqmtypyvxpdk, common/mqk/dzsatoqipytejfmjadfumyggrogvs, common/dnj/czowejblvdxnxyaousadeepgtlpzj, common/lui/ncdnrnftejwyyiqwaeqtvvenvxixs, common/xci/wdbpbtgymzdjpeebjciywzmmclvej, common/rgo/aqqxkovgkohooydcbvblrcimqtfmf, common/gsw/wcoenwplefvylvxpfnedeaisertqf, common/ptw/kcwstjyxtyqdzyiuagmezcnvrhtis, common/ccw/kstbxaujlsganediwifygfrxtnuik, common/nxr/mdsqnutbmchtugtclzognmvupqupk, common/bqy/dghknpskqqiwsqetgyczwrqmdvyql, common/tkp/aqzlifxbahrcnqqwclvtrdripjsfm, common/yqd/jgzocltffqnqefdfzjcuxgxhxxpsn, common/fed/ilndxojzfexctfzlhzhqnxqhivgwc, common/nhf/vtjvscorocmiokumnfuczcpexvtru, common/toc/shqvngqdyztclguxhkmqvfbmopzto, common/myq/aanzunyqdaidjzlostrwvlpmhparf, common/sik/jojvsmmlldywibybzbjmygvyveqgy, common/pvo/hgoxwzhmkwmvtybkquttvndmxmmbs, common/iqn/ehzkeyusvuoxhkhjzjbdgrdjvpltj, common/ead/tphxmlcipelfycfuxfwsosjdtjqay, common/qvg/mntjvzqxjontbdwttjeqiscekoxnk, common/cyd/sgqivsaanlsoydaewerohnfvwvqeh, common/ltr/omxvrvjyjjriyzbaedpteoyiksoow, common/vfd/yggeuomrugxpemljdhuizgxqfggzz, common/efq/hgimexuvqmtxmivwphgzibaeoiziv, common/orf/eyzdjqfrdfkpawaymeegukehtvkcd, common/anx/dzqreworxxlegaourcbizdsvqhzvm, common/iji/aklpshwcvyybunlmplnprcpkhyemx, common/fil/achzmltebmnwwefbbsdttrapsmlhr, common/gwq/qvlznkbsrhzbgipbdyfahrmiotkpn, common/fkv/fliffzhqkbuzduqqpvvmahhbbcnuz, common/biv/lbkdujgkmglrclgddhwpjcxwwqjtr, common/hoc/aiiqhjjbclcqotokfkwlufcuwqqep, common/gki/irawoelaojyjhsgaflgjmitbphfvv, common/xwf/ugueohyuxkyppfjhtwrdegnjgfjkv, common/xhl/xmaxsujxtxhwvjnufpxpeugxiqobe, common/ota/hnsbveuoyoavjchbtbwdqnasmhgap, common/kcb/afllxcfpstsxlmumubcuzvldbstjz, common/nvu/fyilvdgigtfwghdnzlfqijungczfl, common/anv/izbiemcvzuaoowtqdaagjnjxaaqho, common/jbj/jcfqfiritzltvcuxbrjodjeppmtzh, common/oyd/herbwyvqjafuybdsyiiutoejqvnvn, common/hwv/zcfupnvsnhcpifwddhrohresvidyt, common/rma/arbndpumebndbizajtgxpkkgdnknw, common/lrv/ayvcjdjpqelxpilmncyjrxgciqmxt, common/adb/swmisyumjabvhyavfwozjniwbcagy, common/wdd/xofswepyobbkkulxzpxphfdjizcdz, common/znl/whndieroseufismpwogdvajvdkqlg, common/bdy/svlcfcdcijzwydbfuvzuyjboopkrq, common/rkj/jwbgwkdeznxptfgsbsyeybxklamom, common/oya/cctasbqopotrbaqodtbhkuxkecevg, common/hzd/ieqtridyczjubezmxvqjtdlqvnlvp, common/jdl/uwbgcgedsipuzihgoosxfnntccejp, common/erz/paxlhksjcilwooioluthnyavkfvkc, common/gkx/wengqvzeeqjlvirfkzqmqvewvatfb, common/ckt/rxmsxczhosrxptxbogksnwasahdke, common/fvy/zxcsleshixoizeklnmgoreuyrbbou, common/qig/gchccvndoarxjabqietfvxoheyvbt, common/ymg/nispnxmghkhttrmvmhymdzcbzbacm, common/xhy/ijwirwoafnoworxfnplxmxvifgyfc, common/hds/iylxecvetmnqjfbynkidfzfmwtbev, common/oos/vsegrhgxjxqxpygfjmmuinayaahwh, common/szu/efkcmlkslzhmnhuimciihzrxnzzuc, common/ezy/mlfvreedvovtxqqqncgkpydiihbjv, common/ljl/ztlgfaagshboqegazehoqnravdyfu, common/hrc/vrcuhsmqedinczxxwafeooxxsswww, common/juy/cofrdwpvhmgwjlftgdvqttzcwjxxj, common/zud/vowwluddpsysqxywxdsgkavekkwcl, common/xnr/xosbrcrxyhawplbdbexjaogumehxn, common/sxc/ixxxbqhvclwcvjatlngcjmzrepcnn, common/ckp/imrowjhkcntpztmfpeectrajdzfyo, common/ofb/lchlriiapeiicxofbtujljzazrrsy, common/otm/tmxwrerofcioysclzppdqbxxxcjtn, common/hhw/swbyxyyqoeqozewgbhkjpjbmyjruo, common/yzp/jzwduyklesxirbauufuimurutfdnl, common/zap/kjhdwgpnlstzeasuiuzicvxjnzsjt, common/ith/zfwzqrwsdhgckmnfzukgxukzvkmuz, common/yxq/vbkdvldkazdvzyptljdabbgodzbeg, common/hlr/aqjjxmtrwwfaverhymjgctjjjlvkd, common/fhv/unoiuomtlqqkhqxjblapdxckrsbyu, common/wxd/daaepipmthhwwvtetbucxpkciehic, common/rcd/akpzecixhfxfizqumqvvbwdfipbni, common/dbk/mvkjlehiuovfxvjjekrdmdlncdjxw, common/shd/swvftupprpqienyiyryvpofasqghl, common/iuc/eatvvwvniqzuxcjsopqfwazperuqr, common/qio/fscnasaxloploetlibnxalwkglvee, common/jse/ukwnrvbhqqdwcltpeufoxegiaexwy, common/ifh/wyjzrkzriydovtfewthzemtufrmub, common/fbr/xyknvwirwjbfgpiapkelejtqslmvu, common/urf/xhqnzmxmfwlbdgjnnciztgmppucbu, common/ffj/gjhvdrsnyikycpxcegwppnpstrzhp, common/eyl/qskfmufcwjopepigudfvgunnzccxf, common/hol/qbsbeljqlkxbkokfjrebihwntyrld, common/til/xppkwgqgtygpjtntqgbphxgrluvey, common/cor/vnqwovyfpncearobqxyygondezpcr, common/kab/fyfrqepwkqytbssafngzyltyluvdf, common/cgn/ushlteitxayrabzqnexjhmyninjba, common/llf/mwkbdspnvoplanqcfyccysarxjzwr, common/lrk/wkhnkzepqzcembnvghnrvqyglylkj, common/iwx/bzcuqyvdwdxfekltkyqqugzbehqrm, common/pht/mwcvstihdtiilazfhgbdkkskpwmke, common/fns/clnbdluzioulxpqgfzoqcubdkwian, common/owl/zqlvknlihsdvtlacxisxwyxbqcxqi, common/zsn/gtpjqmyczsmwpcaijriycvwbmhfat, common/uuo/amouttyswlerjbwessookczcotlco, common/zvh/kgaiwcxvfazyhkeiogiieqzlmyctb, common/vcw/ptrmqnfgbijwtdlxensawgykrlzsd, common/nnp/doptlaarfupmjtijdtaddgifthlgz, common/act/igubgcfkbizgjebfcodevyqlpbvpn, common/iiw/hyfulvlbykuhyysfzwckoxqjwcjej, common/bkw/ibjnsrhfslsbqmoqrttrusqwdqmox, common/wki/ceazkvdskoyltrekmwzmhrjqmwyuf, common/eaw/lrpmqceooywujlnohxhlhfjcxmkze, common/taj/rmmsrqfiinoxcacszbqdxnqashrdt, common/xni/dcykgpzeviofimqfcqvhaefpyjave, common/vom/lgrpddbkrxatwudsgpmraqvulptjp, common/cqf/mfjfjsqpcdcszegpncyptcmlyrlmm, common/tlg/lalimfjrfcrtevkjimkodkohzefqx, common/dta/plgrkdjvhrimljzbwqrlwwoflnnjg, common/uhi/icsztibwpznekpntewvgxaexunygk, common/jmp/jnzqjxwlfgblvfvdaiwalynwdtage, common/tst/aoqbgvymuybcbhakwvidfjiedonbg, common/qrs/lpgsykoyljexckzymzdfcsyslhepy, common/klu/vaosenouggiuwqvfbcesrlxqyyfnf, common/fia/wkuhlqhjdspalffxpsazddmotwoyp, common/bwm/jeuzaqvtncoyqyotyjtjdwgjnrafd, common/gje/fnavridubbdjnejwwrwdidkutzgmk, common/qyd/lbgcpdpjwhgerwlhutzdcoerkvrwe, common/hrw/zuimxxasooszgtkryzhseyteavuem, common/ota/opugqzvispvfsglxdhgfwxhqnfgwy, common/ndr/kbjmfiwnivmxqdymmpxyztyujleoi, common/yuv/jbyxyjgcfrxokonfvldsbacvvtbpd, common/ffi/mdxkldqkvnplolwclkuvqybruspsj, common/kqf/lttbrshyjwxrflgauynucsgcrwkbt, common/ysh/qbbgapehzxrsiflxpiggnfwjieglb, common/lcp/suzjcydbpltdipoiuxlcembbfetny, common/hdi/dapjcilzthybebjgkrzhzgkajqzdq, common/eoq/ubgjcswhhxdlhchvetqpawirrfczg, common/ksr/dxojrneccnnevxgnsvjztyqtbgbow, common/rmz/adabbvtgmtrqugdxmbdrdjdjktttc, common/wwj/elqgqhludfwtidvwxmhluywuusucs, common/tcl/izpzcbpbhpcseoqasknjestbflkpb, common/nzg/hplqiwvdxduvcthswjqczadqvpoba, common/umv/jbnvrznmnpxlnasopwbsnvryirdne, common/fuj/crqjozjkmrzdjrsrixwsifszamscc, common/rgq/jankxfoieuoyzrrauffuqtzxpaard, common/xvj/giiivsastuvbkdoejzqlwabkqlmra, common/yki/jmpglnctislrgnhztyutgglhzvmwo, common/nvd/inabvzgfdgoyzcfzsgiekyboyrxky, common/wrp/ttlmqlbkfzxohlxtxcnvtbqtteqdo, common/lyj/sffsupwuctnmyxvijpnjoyswmcoyu, common/qeb/gnkcqkejdapyuerkzvtpzqggypwgm, common/pgk/ugubcyawafbzebnhydqgwigjlffli, common/gfe/mewibuzkpzlxzwxfpqbbjkbnndpkw, common/jzw/aahvmuwodzxebquvbulsbwkovqydd, common/tdx/rhxwumsavdgytxnfvofstccoehobu, common/qmv/yavieygzipblkyoxqapnldyvoudrp, common/nco/isqnqpxgobgqskeriqryftdokgwfw, common/pcs/jkppsgxrrskksymvbigoswpuqowub, common/osq/icpklgruboswghgmpztislbhymqbt, common/ggz/wxtfebbzvtimuuebfudfzaqzckcxf, common/wia/filshcmaidtyyfxkzdrfbqpofklof, common/zgq/aenlldfhborgslhodbgjvolxeazyf, common/sdz/jnfrqlbydlaqfyeshivzosnkrdxgm, common/wnd/wdnsxmzlcypmgqqcpsuosuxqguyct, common/ist/cbaaummkaueroiowfycdggzpqkrqm, common/hxl/kbfvtibpecxhjlubzdtcirkvnbrjs, common/qcs/lppikltsmzgojptwhdqzcdkkjjaqg, common/ijm/dvjrzqpiosvsjksjykemswedwyghl, common/ywb/dawpdjdammfyuxdyxklcwxtpqxirc, common/wmw/wlzgpbnxtcvkrrkafiwqvxbswgtvw, common/nmr/usyykzftqhbhyojkygkoiznbviqav, common/ddf/ezrxkaxptiqzdzakvhbgalmggbhld, common/bed/swuwychzsvbfkurcpncutqasnhqvk, common/fsn/oizsyjxqkiumjghwrtuczswrmvlgx, common/oje/njrpawkdcmokdkhxerizdfxxdzpdv, common/yvz/woiqszwfeakkhlduminiylyphdnbr, common/rnx/rnoydtueyshlqkseudwxxpjsdkllq, common/lan/dqdtgyvkpcwyfhsevjgcokoplpzjh, common/zli/ksvufsphrpvnsnujppkajtdhmoors, common/xwy/zpqtydgeqtqkoxgdwnhndkxsiiomv, common/usm/icvtirjowdyfabeepedeqyikdpakt, common/kdh/rammzujjjtpcskxviywvffnfqsfdo, common/rrl/zmmvslkcbvqsihtppkpgygbtbaims, common/poy/dejogflmnmfjjvavixarrjdosocvq, common/txh/zizxjzfmvsnteaqpvrnfklvggxeni, common/swg/jadwqldgrxzauaadmgdcdrsmuovqe, common/psw/nncgixbhyqvzoefskqexszhdcftcf, common/wdv/zotsngbumjhaibzaviyrcttagiett, common/qln/ksqohgllmsjampdwwazmxicxlhupw, common/dky/llwpakyqarpusojntefmpubypfptm, common/xmw/wkwqzoqpaxaakntcsirfagiytcdez, common/orp/giolmyyodubdfjpjklxukdmsoyrib, common/voz/rlpiawsepimfmcncawliwnhisdlou, common/wyq/xapuypdwasgsvvytkxrfuusltjqca, common/kvt/pucwolsxlyhfafcuxnazwauwtkkky, common/hht/bqylccltzdkfmtlswehvewhgguamc, common/gms/ooptpiihecdejxnekytwpmdyumxxk, common/oky/nljzcnvmwbffvugtdossvwjudlasz, common/vfs/ibhqabncmibbblxmcsywnwxfwtucd, common/ibh/ipsjlxiqblzgjujjslfphhihmzzzx, common/tdx/nommoxzqnsscpmcqgkugrectkqans, common/fff/iqykofaoojimuzfieoefcdrwpjtfm, common/wrs/admooabrowhtkgmdztuplfeisibdt, common/uko/mpinbqrkphggvbbjhllhilkufgtzh, common/lea/bxhwzohxkfiwsakunzfxaibzmfksj, common/tda/grvejkjeferbfmefhfkdluewebbqg, common/hbg/bljpxzcepylahkpvxhmwzoxyzraad, common/qmr/taewimxddumkvsqtwzoduyapwstpq, common/qih/tcqowopkupntljrevuqzbmbgxwyzp, common/isn/nentydziqyffioaupvvqtocqfsypz, common/jdh/cshpelicazvrfcggsncxuvtsulqje, common/exi/slrnislibrqjsysvqignajzxmlruf, common/rkc/fiarhpmzsqtbyrunglgpyphmwpgjy, common/ohe/mfvyquvcqcrsxkozuvlzpdbwnlaab, common/wgf/ixokfwsxnuqfuygxnjjrrlyoxwisr, common/wox/aqgdxjprminbciyyddlwlrcqseawj, common/iwc/merspnwdzkpxhjhiamswwowllreym, common/dlz/ceyttgwnbljzrxgewcyctbgduiepz, common/ksn/dqqcpjvertjnsfrvsrnlmzwbgegnd, common/bhb/dovngrggzhsiwbeibxetfohpdgzeg, common/ilv/rccoodzhvwffmbdtirregrgxbfpxe, common/ggt/iokoabqltobbxgxtiupqozfjhepsy, common/rse/befhcfrhqfslixtodgzfhysozrukz, common/ehj/nszdxhmnexgnivtpgfgpakvkebbdu, common/nct/ligcddjahxpdluthyikoonkrwahnn, common/nxz/yszllrqterpvqaqjmkjjobwffdzwo, common/xdr/zvsvprnpkifrhruggqsoiavjmmjci, common/fvn/bnerpgljugankmbtcsunywgujdkkf, common/izs/zwajtcwedtmgtobxpjmtojphvjzzu, common/qsn/qothyurtctjwckrnadbwxbrbfwqxo, common/ctg/rhdnxilvyzhsumwwqdfxiiwudlbmy, common/omv/okoleqvlczmxswwvofmbxctgcldkz, common/oop/yqgjchrdllbpbtwhycyrkxpxjkogv, common/mmq/ytwaodslqgxwcimmthqplkiryxgon, common/pkm/msdnmekgelcdhbibeyibpjzmwftxu, common/lor/ivrswipvusebzzhrordhascojjhlu, common/nen/rjrwrxnrqxmgyzzdeonwamavjuspq, common/faa/vcpjjvpunjgqrxzoloodyqcykhoxe, common/xho/foipifrinicsvoudtpoxqmywcgnrm, common/idh/tjsulifzxwbslqkmbxlpuveqpubvu, common/btw/ksdpqphbeccezaingvzsuxyzvnwgb, common/fxs/vswlvgzskcwaatuqhmynxnbzvoqov, common/qgl/fahgpwlbztkjaifanzidlbnmzkqth, common/lcj/artdqasspexvivmoxffuxbyfdstpx, common/uys/pmrnriffmrkzkxayzyuezlcnyzkta, common/akl/rceeytucpqnhazizjazndgpapuqap, common/wdq/htpdzgcvansrismdtlldfvxubwgbw, common/llc/mshxyqtxkomqcjkgkoqcrabmeowcn, common/zae/teierrhaocfwljjssnzxivnohizvk, common/dnf/tnurelyskyvbicabvfqfywdtldpga, common/qhd/sppwdxmmanbyukuveqxmvsktwuxat, common/yeq/tnndhakcfehlwpadtszjjttufjslj, common/pkq/eljydlvsgfztospdnomwsplhzdjjo, common/mep/qwknpqduhddenczpohytthfuydsth, common/cqx/ahzgsauacniwbyjoyvfjpeiozerkg, common/qvm/hzhkmuswjhseixskcyrwlyfomuszy, common/xtu/wjpaorubgollifznxwfggokoelrvy, common/aec/gbldbaiyagoxbohrvslobjhahmdpd, common/ydk/ghfjojivrlcsnjzkrmjpkhjfxeiha, common/uib/tgsabqattyspqfutnetxldsqhvdjm, common/zle/rwzdcadrsvzltlzszncxeanrkcjpw, common/pjc/januphtdhhypdggfroxtaazapupjw, common/grd/phpskbybxdfxsjwouwzowokxoncxh, common/nij/iaxglgsfphbocmkytpjpoynsxpbcs, common/ubv/sijmmzqjfmxijbtywqrvjhzkegpmu, common/feq/kctitrwaysnzwjiifdyqgouikonjq, common/lbn/johaxulurpupaequxohtmyptltzsd, common/vml/vaamrjoyjvuioitkoijngsjzlyakj, common/gle/vttndtwtpfztkyofknhzqurbwtuuz, common/cuk/hvdevwkiuqslngotveiqxfyndhgit, common/hck/almptfarwzvpvzmybqjunofoftejs, common/tmi/alqisagdmaaqexqdilihmvlshqnnr, common/mvg/niiynvsppfukcsdhuoxgnkhrzdcdz, common/wdr/zrjiuzongupoqccdccmsvesajzzlh, common/tag/pbtihkoqxcfnzopjoaziimgndgpyn, common/lhh/iqxvbepntlmptkaldvssrwbxgfkma, common/wfi/dzwsuaksavtidjndytiglbqsefmrr, common/xgk/ogtdnonwoufiwbjwdpiwrpircvjqm, common/fje/lgmfwpzyopjxxthrbcvmatqozczhu, common/usn/gujdtpyixqfgksoakeiwthaefhasi, common/wql/kvnohnkcyciyqbbiazpqcjdwviaac, common/azw/mlqohuvzaijznbwxkyeyzozycmzjd, common/eyr/qajsbjnikhgxwxsffpklbkmsnbhnl, common/unl/xhuynakqohahrwyyvlsrfvvtludrj, common/uyj/dzmnafcedyttjqdofuzoqhlzgnwiq, common/ulb/asvpckmxoxpqasoxxjihgfjimpfqw, common/kzc/fxcfcmxaigggpzdwfwdpiuudzdleq, common/qvs/opityahpqxxcklcccpwqtcwqtmetk, common/ech/gnqlxuxuamziemjkojorbcvelgycp, common/yya/chxwdniwmednbxrnskkqqhhmjqwow, common/ijv/zzpenbjwqyagpaajsafgnikyswarc, common/arx/mwisopcrzzpcvactpcidnbhzmwzta] were removed from S3 2024.10.22 02:11:26.412921 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_1406_1406_0, total 59 rows starting from the beginning of the part 2024.10.22 02:11:26.474096 [ 1419 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_16_16_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:11:26.507697 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1302_19, 202410_1303_1303_0, 202410_1304_1304_0, 202410_1305_1305_0, 202410_1306_1306_0, 202410_1307_1307_0] 2024.10.22 02:11:26.519714 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_1407_1407_0, total 62 rows starting from the beginning of the part 2024.10.22 02:11:26.753022 [ 1158 ] {} DatabaseCatalog: Execution took 385 ms. 2024.10.22 02:11:26.865827 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/iad/iqrgjmbhylitrlnczxayrktdmgnrx. 2024.10.22 02:11:26.957939 [ 1419 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Removing 9 parts from memory: Parts: [201403_8_8_0, 201403_9_9_0, 201403_10_10_0, 201403_11_11_0, 201403_12_12_0, 201403_13_13_0, 201403_14_14_0, 201403_15_15_0, 201403_16_16_0] 2024.10.22 02:11:26.973305 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.70 MiB, will set to 1.57 GiB (RSS), difference: -2.26 MiB 2024.10.22 02:11:26.975865 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 094c5b448040b68d5ed10e2821a5a8f5, offset: 0, size: 0 2024.10.22 02:11:26.978266 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 094c5b448040b68d5ed10e2821a5a8f5 for removal 2024.10.22 02:11:26.988524 [ 1476 ] {} CacheMetadata: Key 094c5b448040b68d5ed10e2821a5a8f5 is removed from metadata 2024.10.22 02:11:27.048700 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/iad/iqrgjmbhylitrlnczxayrktdmgnrx. 2024.10.22 02:11:27.188464 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_737_737_0, 202410_738_738_0, 202410_739_739_0, 202410_740_740_0, 202410_741_741_0, 202410_742_742_0] 2024.10.22 02:11:27.192790 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_737_737_0, 202410_738_738_0, 202410_739_739_0, 202410_740_740_0, 202410_741_741_0, 202410_742_742_0] 2024.10.22 02:11:27.230605 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_737_737_0, 202410_738_738_0, 202410_739_739_0, 202410_740_740_0, 202410_741_741_0, 202410_742_742_0] 2024.10.22 02:11:27.650653 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 12589 2024.10.22 02:11:27.885958 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:27.891628 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:27.940299 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_827_827_0 (state Temporary) 2024.10.22 02:11:28.004840 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rxm/jqlyjlpikrdbdjozqyeujmritaaop. 2024.10.22 02:11:28.089329 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_827_827_0 to 202410_840_840_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:28.161777 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cc8c2033ab645069f7b62ee049cfeab2, offset: 0, size: 0 2024.10.22 02:11:28.165336 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cc8c2033ab645069f7b62ee049cfeab2 for removal 2024.10.22 02:11:28.173367 [ 1476 ] {} CacheMetadata: Key cc8c2033ab645069f7b62ee049cfeab2 is removed from metadata 2024.10.22 02:11:28.175571 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12589 2024.10.22 02:11:28.252461 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rxm/jqlyjlpikrdbdjozqyeujmritaaop. 2024.10.22 02:11:28.349006 [ 1193 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 201 ms. 2024.10.22 02:11:28.351436 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_790_149} MergeTask::MergeProjectionsStage: Merge sorted 1193328 rows, containing 4 columns (4 merged, 0 gathered) in 40.289704418 sec., 29618.68341399059 rows/sec., 444.30 KiB/sec. 2024.10.22 02:11:28.501321 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_166_166_0 (state Temporary) 2024.10.22 02:11:28.654417 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_790_149} MergedBlockOutputStream: filled checksums 202410_1_790_149 (state Temporary) 2024.10.22 02:11:28.709681 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} MergeTask::MergeProjectionsStage: Merge sorted 12336 rows, containing 13 columns (13 merged, 0 gathered) in 3.035743689 sec., 4063.584170396014 rows/sec., 839.52 KiB/sec. 2024.10.22 02:11:28.886694 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} MergedBlockOutputStream: filled checksums 202410_1207_1407_40 (state Temporary) 2024.10.22 02:11:28.898462 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6799854bd0156bf3de78b2ed3279b084, offset: 0, size: 0 2024.10.22 02:11:28.911259 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6799854bd0156bf3de78b2ed3279b084 for removal 2024.10.22 02:11:28.915571 [ 1476 ] {} CacheMetadata: Key 6799854bd0156bf3de78b2ed3279b084 is removed from metadata 2024.10.22 02:11:29.390946 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 9.29 MiB, will set to 1.57 GiB (RSS), difference: 13.01 MiB 2024.10.22 02:11:29.582366 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 91545 2024.10.22 02:11:29.640633 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1407_40 to 202410_1207_1407_40 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:29.667188 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1407_40} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1402_39, 202410_1407_1407_0] -> 202410_1207_1407_40 2024.10.22 02:11:29.703393 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.70 MiB. 2024.10.22 02:11:29.829325 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 90552 2024.10.22 02:11:29.849323 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:29.853057 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:29.853807 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/asy/oqeiodzqtntfufunzpqutsdtvjusn. 2024.10.22 02:11:29.918270 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1ca58f6febc896266271404bca92c4dc, offset: 0, size: 0 2024.10.22 02:11:29.921839 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1ca58f6febc896266271404bca92c4dc for removal 2024.10.22 02:11:29.932348 [ 1476 ] {} CacheMetadata: Key 1ca58f6febc896266271404bca92c4dc is removed from metadata 2024.10.22 02:11:30.090458 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/asy/oqeiodzqtntfufunzpqutsdtvjusn. 2024.10.22 02:11:30.091788 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 9 entries to flush up to offset 54 2024.10.22 02:11:30.152908 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:30.154967 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:30.129363 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1363 entries to flush up to offset 1192653 2024.10.22 02:11:30.262360 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1395_1395_0 (state Temporary) 2024.10.22 02:11:30.316696 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_813_813_0 (state Temporary) 2024.10.22 02:11:30.341263 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1395_1395_0 to 202410_1408_1408_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:30.389403 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91545 2024.10.22 02:11:30.442967 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tgp/wzkdrovmvejmxepgesbrcbyeryhbk. 2024.10.22 02:11:30.510799 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_813_813_0 to 202410_819_819_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:30.592870 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a0b3152a1f704865954fabbbe48809de, offset: 0, size: 0 2024.10.22 02:11:30.595322 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a0b3152a1f704865954fabbbe48809de for removal 2024.10.22 02:11:30.623446 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90552 2024.10.22 02:11:30.653339 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tgp/wzkdrovmvejmxepgesbrcbyeryhbk. 2024.10.22 02:11:30.689601 [ 1476 ] {} CacheMetadata: Key a0b3152a1f704865954fabbbe48809de is removed from metadata 2024.10.22 02:11:30.860278 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:30.861787 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:30.908878 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_40_40_0 (state Temporary) 2024.10.22 02:11:30.976128 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/dou/huaohdngpzsqnoriwwknuohapeaer. 2024.10.22 02:11:31.022422 [ 1197 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 233 ms. 2024.10.22 02:11:31.023989 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_40_40_0 to 202410_40_40_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:31.171735 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Found 5 old parts to remove. Parts: [202410_1_34_8, 202410_35_35_0, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0] 2024.10.22 02:11:31.174712 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from filesystem (serially): Parts: [202410_1_34_8, 202410_35_35_0, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0] 2024.10.22 02:11:31.195058 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 54 2024.10.22 02:11:31.255835 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:11:31.260968 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:11:31.307293 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from memory: Parts: [202410_1_34_8, 202410_35_35_0, 202410_36_36_0, 202410_37_37_0, 202410_38_38_0] 2024.10.22 02:11:31.375606 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:31.383284 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:31.410255 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/dou/huaohdngpzsqnoriwwknuohapeaer. 2024.10.22 02:11:31.565615 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 5.44 MiB, will set to 1.56 GiB (RSS), difference: 13.33 MiB 2024.10.22 02:11:31.782500 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_792_792_0 (state Temporary) 2024.10.22 02:11:32.032502 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_166_166_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:11:32.155909 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_792_792_0 to 202410_799_799_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:32.209153 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1192653 2024.10.22 02:11:32.439583 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/jbs/lblfzbrtclwkmjygreywcbbozksli. 2024.10.22 02:11:32.515962 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 723dc7a7b2d3e0ad7b20e55b9094b531, offset: 0, size: 0 2024.10.22 02:11:32.519396 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 723dc7a7b2d3e0ad7b20e55b9094b531 for removal 2024.10.22 02:11:32.526788 [ 1476 ] {} CacheMetadata: Key 723dc7a7b2d3e0ad7b20e55b9094b531 is removed from metadata 2024.10.22 02:11:32.754394 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/jbs/lblfzbrtclwkmjygreywcbbozksli. 2024.10.22 02:11:32.771069 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_790_149} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_790_149 to 202410_1_790_149 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:32.774066 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_790_149} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_784_148, 202410_785_790_1] -> 202410_1_790_149 2024.10.22 02:11:32.826647 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:11:32.858046 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 688 2024.10.22 02:11:33.309122 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:33.317842 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:11:33.495195 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_670_670_0 (state Temporary) 2024.10.22 02:11:33.580603 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 5.36 MiB, will set to 1.57 GiB (RSS), difference: 5.37 MiB 2024.10.22 02:11:33.723880 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 1048576 bytes for 58242d715d331feada47591edfd5c171:0) 2024.10.22 02:11:33.729456 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 205ad1f9404ca48beb078c3585fdafb0, offset: 33554432, size: 12827132 2024.10.22 02:11:33.744523 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 205ad1f9404ca48beb078c3585fdafb0 for removal 2024.10.22 02:11:33.747388 [ 1476 ] {} CacheMetadata: Key 205ad1f9404ca48beb078c3585fdafb0 is removed from metadata 2024.10.22 02:11:33.754956 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ttu/kjjxikrxnklanuegcihzhyuecpthz. 2024.10.22 02:11:33.844704 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_670_670_0 to 202410_680_680_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:33.913783 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2de86453f2a5277468887d680076d836, offset: 0, size: 0 2024.10.22 02:11:34.024637 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2de86453f2a5277468887d680076d836 for removal 2024.10.22 02:11:34.031033 [ 1476 ] {} CacheMetadata: Key 2de86453f2a5277468887d680076d836 is removed from metadata 2024.10.22 02:11:34.079232 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 688 2024.10.22 02:11:34.100462 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ttu/kjjxikrxnklanuegcihzhyuecpthz. 2024.10.22 02:11:34.103378 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/htc/gejaewyruozfpktlzpwmcjsifkijw. 2024.10.22 02:11:34.407374 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 91617 2024.10.22 02:11:34.520524 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/htc/gejaewyruozfpktlzpwmcjsifkijw. 2024.10.22 02:11:34.656853 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:34.660379 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:34.747754 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1396_1396_0 (state Temporary) 2024.10.22 02:11:34.817711 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1396_1396_0 to 202410_1409_1409_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:34.872107 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91617 2024.10.22 02:11:35.071513 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tuw/cxgwqanxofsohocnwoatwvvqcqfpg. 2024.10.22 02:11:35.146967 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 905e3e1d29f3083d31db4f49c1a1f53c, offset: 0, size: 0 2024.10.22 02:11:35.149459 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 905e3e1d29f3083d31db4f49c1a1f53c for removal 2024.10.22 02:11:35.153240 [ 1476 ] {} CacheMetadata: Key 905e3e1d29f3083d31db4f49c1a1f53c is removed from metadata 2024.10.22 02:11:35.336425 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tuw/cxgwqanxofsohocnwoatwvvqcqfpg. 2024.10.22 02:11:35.530723 [ 1272 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:11:35.563390 [ 1272 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:11:35.613373 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/yfu/uevrmbejvrcwpcqlzqikesgucqtje. 2024.10.22 02:11:35.617656 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 4.55 MiB, will set to 1.56 GiB (RSS), difference: 10.71 MiB 2024.10.22 02:11:35.655472 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ed62a9761553b77e99d8e597c4bd2b1d, offset: 0, size: 0 2024.10.22 02:11:35.657784 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ed62a9761553b77e99d8e597c4bd2b1d for removal 2024.10.22 02:11:35.661574 [ 1476 ] {} CacheMetadata: Key ed62a9761553b77e99d8e597c4bd2b1d is removed from metadata 2024.10.22 02:11:35.684189 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 12606 2024.10.22 02:11:35.732049 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/yfu/uevrmbejvrcwpcqlzqikesgucqtje. 2024.10.22 02:11:35.913191 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:35.922028 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:36.060531 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_828_828_0 (state Temporary) 2024.10.22 02:11:36.135791 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_828_828_0 to 202410_841_841_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:36.203711 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12606 2024.10.22 02:11:36.277370 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zec/imipawnyssrcebykreacgzkeipuwg. 2024.10.22 02:11:36.325228 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 40178c05a655880f8eec986d5941facc, offset: 0, size: 0 2024.10.22 02:11:36.328517 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 40178c05a655880f8eec986d5941facc for removal 2024.10.22 02:11:36.333187 [ 1476 ] {} CacheMetadata: Key 40178c05a655880f8eec986d5941facc is removed from metadata 2024.10.22 02:11:36.596204 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zec/imipawnyssrcebykreacgzkeipuwg. 2024.10.22 02:11:36.855787 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_166_166_0 to 202410_169_169_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:36.871438 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/luq/tsfkenobwaznbjfmnjjoggvifgdwx. 2024.10.22 02:11:37.175497 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2025a2164ba53a82c7324e2a93262791, offset: 0, size: 0 2024.10.22 02:11:37.180930 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2025a2164ba53a82c7324e2a93262791 for removal 2024.10.22 02:11:37.188949 [ 1476 ] {} CacheMetadata: Key 2025a2164ba53a82c7324e2a93262791 is removed from metadata 2024.10.22 02:11:37.561642 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/luq/tsfkenobwaznbjfmnjjoggvifgdwx. 2024.10.22 02:11:37.764171 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 3.75 MiB, will set to 1.56 GiB (RSS), difference: 8.31 MiB 2024.10.22 02:11:37.820504 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4fea9ec1a2efc15b1457520400adb037, offset: 0, size: 0 2024.10.22 02:11:37.825579 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4fea9ec1a2efc15b1457520400adb037 for removal 2024.10.22 02:11:37.830481 [ 1476 ] {} CacheMetadata: Key 4fea9ec1a2efc15b1457520400adb037 is removed from metadata 2024.10.22 02:11:37.873222 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lsq/bzmkkcycmhsmnsnnzehapptpogapa. 2024.10.22 02:11:37.925265 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 76a95a32353445c6f9853ea0755a2189, offset: 0, size: 0 2024.10.22 02:11:37.933609 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 76a95a32353445c6f9853ea0755a2189 for removal 2024.10.22 02:11:37.939727 [ 1476 ] {} CacheMetadata: Key 76a95a32353445c6f9853ea0755a2189 is removed from metadata 2024.10.22 02:11:38.004785 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lsq/bzmkkcycmhsmnsnnzehapptpogapa. 2024.10.22 02:11:38.140665 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 41 entries to flush up to offset 90593 2024.10.22 02:11:38.153746 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 228c9e1375ccb068f726f0c52a9ce0fa, offset: 0, size: 0 2024.10.22 02:11:38.156067 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 228c9e1375ccb068f726f0c52a9ce0fa for removal 2024.10.22 02:11:38.161544 [ 1476 ] {} CacheMetadata: Key 228c9e1375ccb068f726f0c52a9ce0fa is removed from metadata 2024.10.22 02:11:38.572261 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:38.581057 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:38.658105 [ 1226 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 02:11:38.698404 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_814_814_0 (state Temporary) 2024.10.22 02:11:38.854175 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_814_814_0 to 202410_820_820_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:38.887592 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 91672 2024.10.22 02:11:38.996609 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90593 2024.10.22 02:11:39.156060 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6684 2024.10.22 02:11:39.222452 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1717 entries to flush up to offset 1194370 2024.10.22 02:11:39.325592 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:39.329739 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:39.399633 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1397_1397_0 (state Temporary) 2024.10.22 02:11:39.443088 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 7.34 MiB, will set to 1.56 GiB (RSS), difference: 3.51 MiB 2024.10.22 02:11:39.659442 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1397_1397_0 to 202410_1410_1410_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:39.673540 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nad/utlnvzdcymrpoyttiuxcgmeubgvwv. 2024.10.22 02:11:39.738667 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a31f15e82f7c611c1971940927d7f5c3, offset: 0, size: 0 2024.10.22 02:11:39.741805 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a31f15e82f7c611c1971940927d7f5c3 for removal 2024.10.22 02:11:39.745697 [ 1476 ] {} CacheMetadata: Key a31f15e82f7c611c1971940927d7f5c3 is removed from metadata 2024.10.22 02:11:39.836629 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91672 2024.10.22 02:11:39.897989 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nad/utlnvzdcymrpoyttiuxcgmeubgvwv. 2024.10.22 02:11:40.153207 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:40.158436 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:11:40.183563 [ 1677 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yfq/kqgtvkflytuoalbhsoohlfhixmruu, size 717780 2024.10.22 02:11:40.282250 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_793_793_0 (state Temporary) 2024.10.22 02:11:40.443160 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 40d91b3cf883fae717c36e245f21c2eb:0) 2024.10.22 02:11:40.445659 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: da9e066c23a8d06c5418589e537446e7, offset: 0, size: 2753496 2024.10.22 02:11:40.450213 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key da9e066c23a8d06c5418589e537446e7 for removal 2024.10.22 02:11:40.455208 [ 1476 ] {} CacheMetadata: Key da9e066c23a8d06c5418589e537446e7 is removed from metadata 2024.10.22 02:11:40.456833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e513832a7763288efe4b31de443b9dd2, offset: 0, size: 1122 2024.10.22 02:11:40.461481 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e513832a7763288efe4b31de443b9dd2 for removal 2024.10.22 02:11:40.483655 [ 1476 ] {} CacheMetadata: Key e513832a7763288efe4b31de443b9dd2 is removed from metadata 2024.10.22 02:11:40.551581 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ict/mvnvryfdkufeiwdxfzvojtjncocwe. 2024.10.22 02:11:40.551559 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_793_793_0 to 202410_800_800_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:40.580020 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e83736d078e3073fa5d4371d90162ff4, offset: 0, size: 0 2024.10.22 02:11:40.583219 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e83736d078e3073fa5d4371d90162ff4 for removal 2024.10.22 02:11:40.586234 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1194370 2024.10.22 02:11:40.587062 [ 1476 ] {} CacheMetadata: Key e83736d078e3073fa5d4371d90162ff4 is removed from metadata 2024.10.22 02:11:40.584580 [ 1223 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_790_149 to 202410_791_796_1 2024.10.22 02:11:40.623778 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 257321d5e7af4e1f6c3b8aa0195f5918, offset: 0, size: 0 2024.10.22 02:11:40.679495 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FUniqID 2024.10.22 02:11:40.691246 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FUniqID 2024.10.22 02:11:40.723532 [ 1223 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:40.727721 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 257321d5e7af4e1f6c3b8aa0195f5918 for removal 2024.10.22 02:11:40.733001 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:11:40.733312 [ 1476 ] {} CacheMetadata: Key 257321d5e7af4e1f6c3b8aa0195f5918 is removed from metadata 2024.10.22 02:11:40.748649 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ict/mvnvryfdkufeiwdxfzvojtjncocwe. 2024.10.22 02:11:40.771449 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_796_150} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_790_149 to 202410_791_796_1 into Wide with storage Full 2024.10.22 02:11:40.795565 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_796_150} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:40.808285 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_796_150} MergeTreeSequentialSource: Reading 147 marks from part 202410_1_790_149, total 1193328 rows starting from the beginning of the part 2024.10.22 02:11:40.815666 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FUniqID 2024.10.22 02:11:40.851412 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FUniqID 2024.10.22 02:11:40.854828 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_796_150} MergeTreeSequentialSource: Reading 3 marks from part 202410_791_796_1, total 8255 rows starting from the beginning of the part 2024.10.22 02:11:40.927753 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FUniqID 2024.10.22 02:11:41.048350 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FUniqID 2024.10.22 02:11:41.056280 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 670068 bytes for e83736d078e3073fa5d4371d90162ff4:0) 2024.10.22 02:11:41.059757 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1df1b5cc5d849f9e7fc7257b12796a66, offset: 33554432, size: 19098690 2024.10.22 02:11:41.076188 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1df1b5cc5d849f9e7fc7257b12796a66 for removal 2024.10.22 02:11:41.090555 [ 1476 ] {} CacheMetadata: Key 1df1b5cc5d849f9e7fc7257b12796a66 is removed from metadata 2024.10.22 02:11:41.093591 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e07fd60feba440f7c6d07b235a000240, offset: 0, size: 825 2024.10.22 02:11:41.110824 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e07fd60feba440f7c6d07b235a000240 for removal 2024.10.22 02:11:41.115812 [ 1476 ] {} CacheMetadata: Key e07fd60feba440f7c6d07b235a000240 is removed from metadata 2024.10.22 02:11:41.244344 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FUniqID 2024.10.22 02:11:41.256257 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:11:41.259002 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:11:41.426422 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tfn/hlahcdwboinsaontofrsooggajvsj (LogSeriesLimiter: on interval from 2024-10-22 02:11:19 to 2024-10-22 02:11:41 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:11:41.531756 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 387c01fe4621a9131a769eea429b3cb2, offset: 0, size: 0 2024.10.22 02:11:41.533325 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 387c01fe4621a9131a769eea429b3cb2 for removal 2024.10.22 02:11:41.548697 [ 1476 ] {} CacheMetadata: Key 387c01fe4621a9131a769eea429b3cb2 is removed from metadata 2024.10.22 02:11:41.576728 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 7.64 MiB, will set to 1.56 GiB (RSS), difference: -5.06 MiB 2024.10.22 02:11:41.582479 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 689 2024.10.22 02:11:41.715398 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8589f389096ec58cc8d3c15efcf44db3, offset: 0, size: 0 2024.10.22 02:11:41.734613 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8589f389096ec58cc8d3c15efcf44db3 for removal 2024.10.22 02:11:41.746463 [ 1476 ] {} CacheMetadata: Key 8589f389096ec58cc8d3c15efcf44db3 is removed from metadata 2024.10.22 02:11:41.747597 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/upa/hcgstehqtpbqdvioznvkinnrruoat. 2024.10.22 02:11:41.773354 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_796_150/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:11:41.838725 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ynk/rbaiwwzdiwimaharlfjgwivtlylhm. 2024.10.22 02:11:41.902689 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4c5d1d1ff48f86cee636e42c003926d1, offset: 0, size: 0 2024.10.22 02:11:41.903106 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:41.906027 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4c5d1d1ff48f86cee636e42c003926d1 for removal 2024.10.22 02:11:41.912810 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.86 GiB. 2024.10.22 02:11:41.913016 [ 1476 ] {} CacheMetadata: Key 4c5d1d1ff48f86cee636e42c003926d1 is removed from metadata 2024.10.22 02:11:41.960364 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9f91f974cc5502b4802ffc8954836e8f, offset: 0, size: 0 2024.10.22 02:11:41.964657 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9f91f974cc5502b4802ffc8954836e8f for removal 2024.10.22 02:11:41.967983 [ 1476 ] {} CacheMetadata: Key 9f91f974cc5502b4802ffc8954836e8f is removed from metadata 2024.10.22 02:11:41.980200 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/upa/hcgstehqtpbqdvioznvkinnrruoat. 2024.10.22 02:11:41.991868 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_671_671_0 (state Temporary) 2024.10.22 02:11:42.141465 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8a088ac1aebe4bbe77961f3dc7ff002e, offset: 0, size: 0 2024.10.22 02:11:42.145334 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8a088ac1aebe4bbe77961f3dc7ff002e for removal 2024.10.22 02:11:42.149329 [ 1476 ] {} CacheMetadata: Key 8a088ac1aebe4bbe77961f3dc7ff002e is removed from metadata 2024.10.22 02:11:42.286785 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b603ee0238a597110dda86687f3e1468, offset: 0, size: 0 2024.10.22 02:11:42.291134 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b603ee0238a597110dda86687f3e1468 for removal 2024.10.22 02:11:42.294340 [ 1476 ] {} CacheMetadata: Key b603ee0238a597110dda86687f3e1468 is removed from metadata 2024.10.22 02:11:42.296337 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gtg/ypvlyktxmahrequroabolwcyoiwxo. 2024.10.22 02:11:42.348280 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f320dfd6c342db9bf50a3d166d51c3b8, offset: 0, size: 0 2024.10.22 02:11:42.352262 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_671_671_0 to 202410_681_681_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:42.352632 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f320dfd6c342db9bf50a3d166d51c3b8 for removal 2024.10.22 02:11:42.354942 [ 1476 ] {} CacheMetadata: Key f320dfd6c342db9bf50a3d166d51c3b8 is removed from metadata 2024.10.22 02:11:42.499913 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8eb0295d74d355f1bb4b016b95b3e31f, offset: 0, size: 0 2024.10.22 02:11:42.502083 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8eb0295d74d355f1bb4b016b95b3e31f for removal 2024.10.22 02:11:42.507666 [ 1476 ] {} CacheMetadata: Key 8eb0295d74d355f1bb4b016b95b3e31f is removed from metadata 2024.10.22 02:11:42.628009 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gtg/ypvlyktxmahrequroabolwcyoiwxo. 2024.10.22 02:11:42.630240 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/otn/tjojijlebekgqazhenqkofvpnrbsl. 2024.10.22 02:11:42.649840 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 689 2024.10.22 02:11:42.666514 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e92a51f7cbd183b911faf451db733549, offset: 0, size: 0 2024.10.22 02:11:42.668289 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e92a51f7cbd183b911faf451db733549 for removal 2024.10.22 02:11:42.671421 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_13_13_0 to 201403_13_13_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:42.688303 [ 1476 ] {} CacheMetadata: Key e92a51f7cbd183b911faf451db733549 is removed from metadata 2024.10.22 02:11:42.694735 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ynk/rbaiwwzdiwimaharlfjgwivtlylhm. 2024.10.22 02:11:42.741184 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: df157de9f7babe97767710714308be65, offset: 0, size: 0 2024.10.22 02:11:42.745272 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key df157de9f7babe97767710714308be65 for removal 2024.10.22 02:11:42.749665 [ 1476 ] {} CacheMetadata: Key df157de9f7babe97767710714308be65 is removed from metadata 2024.10.22 02:11:42.754212 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/otn/tjojijlebekgqazhenqkofvpnrbsl. 2024.10.22 02:11:42.771834 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:11:43.446845 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.26 GiB, peak 2.00 GiB, free memory in arenas 9.89 MiB, will set to 1.47 GiB (RSS), difference: 212.51 MiB 2024.10.22 02:11:43.471939 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_730_139, 202410_731_736_1] 2024.10.22 02:11:43.474597 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_730_139, 202410_731_736_1] 2024.10.22 02:11:43.502148 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_730_139, 202410_731_736_1] 2024.10.22 02:11:43.708107 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 7 entries to flush up to offset 12613 2024.10.22 02:11:43.845525 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 113 entries to flush up to offset 91785 2024.10.22 02:11:44.103326 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3a009de3d5821f9639ed3193caf89aa4, offset: 0, size: 0 2024.10.22 02:11:44.107565 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3a009de3d5821f9639ed3193caf89aa4 for removal 2024.10.22 02:11:44.112925 [ 1476 ] {} CacheMetadata: Key 3a009de3d5821f9639ed3193caf89aa4 is removed from metadata 2024.10.22 02:11:44.168905 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:44.173595 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 46f5ab324c71650c46104464f092d870, offset: 0, size: 0 2024.10.22 02:11:44.174697 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.86 GiB. 2024.10.22 02:11:44.177646 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 46f5ab324c71650c46104464f092d870 for removal 2024.10.22 02:11:44.183350 [ 1476 ] {} CacheMetadata: Key 46f5ab324c71650c46104464f092d870 is removed from metadata 2024.10.22 02:11:44.264820 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_829_829_0 (state Temporary) 2024.10.22 02:11:44.345479 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_829_829_0 to 202410_842_842_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:44.554085 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12613 2024.10.22 02:11:44.689476 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:44.700916 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.86 GiB. 2024.10.22 02:11:44.791940 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1398_1398_0 (state Temporary) 2024.10.22 02:11:44.857119 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tdg/vfoipsuwzibnysqtxhgzeqmgfwziz. 2024.10.22 02:11:44.905246 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 94025f4a68c64eca351c2ef17bb93212, offset: 0, size: 0 2024.10.22 02:11:44.911521 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 94025f4a68c64eca351c2ef17bb93212 for removal 2024.10.22 02:11:44.919673 [ 1476 ] {} CacheMetadata: Key 94025f4a68c64eca351c2ef17bb93212 is removed from metadata 2024.10.22 02:11:44.955261 [ 1253 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 481 ms. 2024.10.22 02:11:44.975181 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1398_1398_0 to 202410_1411_1411_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:45.015495 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_757_2, 202410_758_758_0, 202410_759_759_0, 202410_760_760_0, 202410_761_761_0, 202410_762_762_0] 2024.10.22 02:11:45.017692 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_757_2, 202410_758_758_0, 202410_759_759_0, 202410_760_760_0, 202410_761_761_0, 202410_762_762_0] 2024.10.22 02:11:45.033830 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91785 2024.10.22 02:11:45.269746 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_757_2, 202410_758_758_0, 202410_759_759_0, 202410_760_760_0, 202410_761_761_0, 202410_762_762_0] 2024.10.22 02:11:45.275760 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8c4ea26bbfaed64da73da3792b48a04b, offset: 0, size: 0 2024.10.22 02:11:45.278261 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8c4ea26bbfaed64da73da3792b48a04b for removal 2024.10.22 02:11:45.295455 [ 1476 ] {} CacheMetadata: Key 8c4ea26bbfaed64da73da3792b48a04b is removed from metadata 2024.10.22 02:11:45.415845 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tdg/vfoipsuwzibnysqtxhgzeqmgfwziz. 2024.10.22 02:11:45.681682 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.30 GiB, peak 2.00 GiB, free memory in arenas 13.99 MiB, will set to 1.33 GiB (RSS), difference: 35.66 MiB 2024.10.22 02:11:46.201778 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 55 2024.10.22 02:11:46.355227 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:46.358315 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:46.517332 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 80 entries to flush up to offset 90673 2024.10.22 02:11:46.520834 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 41baa0e522ccd28c615f56e41060325e, offset: 0, size: 0 2024.10.22 02:11:46.527379 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 41baa0e522ccd28c615f56e41060325e for removal 2024.10.22 02:11:46.534681 [ 1476 ] {} CacheMetadata: Key 41baa0e522ccd28c615f56e41060325e is removed from metadata 2024.10.22 02:11:46.586697 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2cf5b4cceb65005feb8446a0b08b7e4b, offset: 0, size: 0 2024.10.22 02:11:46.591930 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2cf5b4cceb65005feb8446a0b08b7e4b for removal 2024.10.22 02:11:46.595213 [ 1476 ] {} CacheMetadata: Key 2cf5b4cceb65005feb8446a0b08b7e4b is removed from metadata 2024.10.22 02:11:46.618230 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_41_41_0 (state Temporary) 2024.10.22 02:11:46.695944 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_41_41_0 to 202410_41_41_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:46.725180 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 6725 2024.10.22 02:11:47.125023 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:47.128997 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:47.168055 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 55 2024.10.22 02:11:47.226435 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_815_815_0 (state Temporary) 2024.10.22 02:11:47.269823 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/zrs/lbarnitbdfbmatfyjardwhmcydfkm. 2024.10.22 02:11:47.275659 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_815_815_0 to 202410_821_821_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:47.426942 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: eb57a587309c9d5d6b6f68ee27aa181a, offset: 0, size: 0 2024.10.22 02:11:47.429186 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key eb57a587309c9d5d6b6f68ee27aa181a for removal 2024.10.22 02:11:47.432615 [ 1476 ] {} CacheMetadata: Key eb57a587309c9d5d6b6f68ee27aa181a is removed from metadata 2024.10.22 02:11:47.452590 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90673 2024.10.22 02:11:47.572913 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 50cce7bddc13e44fa300d0ec9181d60e, offset: 0, size: 0 2024.10.22 02:11:47.575842 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 50cce7bddc13e44fa300d0ec9181d60e for removal 2024.10.22 02:11:47.590454 [ 1476 ] {} CacheMetadata: Key 50cce7bddc13e44fa300d0ec9181d60e is removed from metadata 2024.10.22 02:11:47.597070 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/zrs/lbarnitbdfbmatfyjardwhmcydfkm. 2024.10.22 02:11:47.761206 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 2.00 GiB, free memory in arenas 14.63 MiB, will set to 1.37 GiB (RSS), difference: -52.09 MiB 2024.10.22 02:11:47.858472 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1195910 2024.10.22 02:11:47.992184 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_778_171, 202410_779_779_0] 2024.10.22 02:11:48.000412 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_778_171, 202410_779_779_0] 2024.10.22 02:11:48.016727 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_778_171, 202410_779_779_0] 2024.10.22 02:11:49.060075 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 91847 2024.10.22 02:11:49.084535 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:49.108707 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:11:49.383610 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6d1f6e259eeb129818313861baf4cd20, offset: 0, size: 0 2024.10.22 02:11:49.388309 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6d1f6e259eeb129818313861baf4cd20 for removal 2024.10.22 02:11:49.396779 [ 1476 ] {} CacheMetadata: Key 6d1f6e259eeb129818313861baf4cd20 is removed from metadata 2024.10.22 02:11:49.460273 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 765e7a6dc95b33e280db7f02a2cc45e7, offset: 0, size: 0 2024.10.22 02:11:49.467353 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 765e7a6dc95b33e280db7f02a2cc45e7 for removal 2024.10.22 02:11:49.468823 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:49.469676 [ 1476 ] {} CacheMetadata: Key 765e7a6dc95b33e280db7f02a2cc45e7 is removed from metadata 2024.10.22 02:11:49.473238 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:11:49.488359 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_794_794_0 (state Temporary) 2024.10.22 02:11:49.493590 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 2.00 GiB, free memory in arenas 16.06 MiB, will set to 1.39 GiB (RSS), difference: -10.41 MiB 2024.10.22 02:11:49.673592 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1399_1399_0 (state Temporary) 2024.10.22 02:11:49.754654 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_794_794_0 to 202410_801_801_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:49.816539 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1195910 2024.10.22 02:11:49.844193 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1399_1399_0 to 202410_1412_1412_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:49.871520 [ 1212 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 5 parts from 202410_1207_1407_40 to 202410_1411_1411_0 2024.10.22 02:11:49.886709 [ 1212 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:49.899420 [ 1212 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:11:49.915710 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91847 2024.10.22 02:11:49.979037 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hxg/zujqsmnqxswfgrsrekztxvrkixjtc. 2024.10.22 02:11:50.114237 [ 1212 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 258 ms. 2024.10.22 02:11:50.131145 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1411_41} MergeTask::PrepareStage: Merging 5 parts: from 202410_1207_1407_40 to 202410_1411_1411_0 into Compact with storage Full 2024.10.22 02:11:50.140578 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 43e36fe9063f4a160b7186b6548005ba, offset: 0, size: 0 2024.10.22 02:11:50.142583 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 43e36fe9063f4a160b7186b6548005ba for removal 2024.10.22 02:11:50.148381 [ 1476 ] {} CacheMetadata: Key 43e36fe9063f4a160b7186b6548005ba is removed from metadata 2024.10.22 02:11:50.186395 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 841c6ba4504b8760a79901a4639eada8, offset: 0, size: 0 2024.10.22 02:11:50.188991 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 841c6ba4504b8760a79901a4639eada8 for removal 2024.10.22 02:11:50.191303 [ 1476 ] {} CacheMetadata: Key 841c6ba4504b8760a79901a4639eada8 is removed from metadata 2024.10.22 02:11:50.201627 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1411_41} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:50.203432 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hxg/zujqsmnqxswfgrsrekztxvrkixjtc. 2024.10.22 02:11:50.221789 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1411_41} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1407_40, total 12336 rows starting from the beginning of the part 2024.10.22 02:11:50.265183 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1411_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_1408_1408_0, total 68 rows starting from the beginning of the part 2024.10.22 02:11:50.307245 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1411_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_1409_1409_0, total 72 rows starting from the beginning of the part 2024.10.22 02:11:50.351156 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1411_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_1410_1410_0, total 55 rows starting from the beginning of the part 2024.10.22 02:11:50.269742 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1307_20, 202410_1308_1308_0, 202410_1309_1309_0, 202410_1310_1310_0, 202410_1311_1311_0, 202410_1312_1312_0] 2024.10.22 02:11:50.372626 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1307_20, 202410_1308_1308_0, 202410_1309_1309_0, 202410_1310_1310_0, 202410_1311_1311_0, 202410_1312_1312_0] 2024.10.22 02:11:50.422085 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1411_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_1411_1411_0, total 113 rows starting from the beginning of the part 2024.10.22 02:11:50.520708 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1307_20, 202410_1308_1308_0, 202410_1309_1309_0, 202410_1310_1310_0, 202410_1311_1311_0, 202410_1312_1312_0] 2024.10.22 02:11:50.569673 [ 1247 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:11:50.615885 [ 1247 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:11:51.166742 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qre/zblkpezrhijnvpdxzjndypkvdvboh. 2024.10.22 02:11:51.256446 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:11:51.259288 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:11:51.317510 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0a587d3c9177ec3e605c58c6e72f7ae1, offset: 0, size: 0 2024.10.22 02:11:51.326488 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0a587d3c9177ec3e605c58c6e72f7ae1 for removal 2024.10.22 02:11:51.331163 [ 1476 ] {} CacheMetadata: Key 0a587d3c9177ec3e605c58c6e72f7ae1 is removed from metadata 2024.10.22 02:11:51.379937 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c3738ae5e1f45a9dacd27a5d3c8d85f7, offset: 0, size: 0 2024.10.22 02:11:51.387096 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c3738ae5e1f45a9dacd27a5d3c8d85f7 for removal 2024.10.22 02:11:51.391887 [ 1476 ] {} CacheMetadata: Key c3738ae5e1f45a9dacd27a5d3c8d85f7 is removed from metadata 2024.10.22 02:11:51.402272 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qre/zblkpezrhijnvpdxzjndypkvdvboh. 2024.10.22 02:11:51.643581 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 2.00 GiB, free memory in arenas 18.25 MiB, will set to 1.42 GiB (RSS), difference: -35.84 MiB 2024.10.22 02:11:51.801080 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 811946 bytes for 0a587d3c9177ec3e605c58c6e72f7ae1:0) 2024.10.22 02:11:51.808520 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: af6cd34767f5df4976ea7ce243d474ec, offset: 0, size: 16777216 2024.10.22 02:11:51.822415 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key af6cd34767f5df4976ea7ce243d474ec for removal 2024.10.22 02:11:51.828442 [ 1476 ] {} CacheMetadata: Key af6cd34767f5df4976ea7ce243d474ec is removed from metadata 2024.10.22 02:11:51.830328 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b2b3c69a7605507ef900c9401cb5bc50, offset: 0, size: 1084 2024.10.22 02:11:51.940276 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b2b3c69a7605507ef900c9401cb5bc50 for removal 2024.10.22 02:11:52.054951 [ 1476 ] {} CacheMetadata: Key b2b3c69a7605507ef900c9401cb5bc50 is removed from metadata 2024.10.22 02:11:52.058373 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 12619 2024.10.22 02:11:52.169522 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:52.175425 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.86 GiB. 2024.10.22 02:11:52.249230 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_830_830_0 (state Temporary) 2024.10.22 02:11:52.353645 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_830_830_0 to 202410_843_843_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:52.374870 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_838_211 to 202410_839_839_0 2024.10.22 02:11:52.381469 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:52.394672 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12619 2024.10.22 02:11:52.411622 [ 1228 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.86 GiB. 2024.10.22 02:11:52.461380 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_839_212} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_838_211 to 202410_839_839_0 into Compact with storage Full 2024.10.22 02:11:52.557655 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_839_212} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:52.562750 [ 1227 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 5 parts from 202410_165_165_0 to 202410_169_169_0 2024.10.22 02:11:52.580878 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_839_212} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_838_211, total 12663 rows starting from the beginning of the part 2024.10.22 02:11:52.615381 [ 1227 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:52.619944 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.86 GiB. 2024.10.22 02:11:52.658912 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_165_169_1} MergeTask::PrepareStage: Merging 5 parts: from 202410_165_165_0 to 202410_169_169_0 into Compact with storage Full 2024.10.22 02:11:52.661018 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/avv/serunvytomfqarnpaekhdupvbgaux. 2024.10.22 02:11:52.704311 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 331786e2d1d0325ec29a316d2fba5717, offset: 0, size: 0 2024.10.22 02:11:52.706722 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_839_212} MergeTreeSequentialSource: Reading 2 marks from part 202410_839_839_0, total 19 rows starting from the beginning of the part 2024.10.22 02:11:52.709811 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 331786e2d1d0325ec29a316d2fba5717 for removal 2024.10.22 02:11:52.691545 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1411_41} MergeTask::MergeProjectionsStage: Merge sorted 12644 rows, containing 13 columns (13 merged, 0 gathered) in 2.571947734 sec., 4916.118563706395 rows/sec., 1.00 MiB/sec. 2024.10.22 02:11:52.714553 [ 1476 ] {} CacheMetadata: Key 331786e2d1d0325ec29a316d2fba5717 is removed from metadata 2024.10.22 02:11:52.764583 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cb514d3392bea46c9cb138f56e180337, offset: 0, size: 0 2024.10.22 02:11:52.767327 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cb514d3392bea46c9cb138f56e180337 for removal 2024.10.22 02:11:52.770491 [ 1476 ] {} CacheMetadata: Key cb514d3392bea46c9cb138f56e180337 is removed from metadata 2024.10.22 02:11:52.791243 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_796_150/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:11:52.804550 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/avv/serunvytomfqarnpaekhdupvbgaux. 2024.10.22 02:11:52.913414 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_628_128, 202410_629_629_0, 202410_630_630_0, 202410_631_631_0, 202410_632_632_0, 202410_633_633_0] 2024.10.22 02:11:52.917033 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1411_41} MergedBlockOutputStream: filled checksums 202410_1207_1411_41 (state Temporary) 2024.10.22 02:11:52.916090 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_628_128, 202410_629_629_0, 202410_630_630_0, 202410_631_631_0, 202410_632_632_0, 202410_633_633_0] 2024.10.22 02:11:52.957376 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_628_128, 202410_629_629_0, 202410_630_630_0, 202410_631_631_0, 202410_632_632_0, 202410_633_633_0] 2024.10.22 02:11:52.967931 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f8426d6bddcfec811503fc87d85776bc, offset: 0, size: 0 2024.10.22 02:11:52.969523 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f8426d6bddcfec811503fc87d85776bc for removal 2024.10.22 02:11:52.971432 [ 1476 ] {} CacheMetadata: Key f8426d6bddcfec811503fc87d85776bc is removed from metadata 2024.10.22 02:11:53.091589 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:11:53.121049 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 66c1740e566adcc1b4c289ffb9675021, offset: 0, size: 0 2024.10.22 02:11:53.123423 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 66c1740e566adcc1b4c289ffb9675021 for removal 2024.10.22 02:11:53.128874 [ 1476 ] {} CacheMetadata: Key 66c1740e566adcc1b4c289ffb9675021 is removed from metadata 2024.10.22 02:11:53.140750 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/urb/llltzypemipddhnlafsaynqwetptw. 2024.10.22 02:11:53.184925 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cf89f1f402501bfbc1f3539e1e95d5d2, offset: 0, size: 0 2024.10.22 02:11:53.189740 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cf89f1f402501bfbc1f3539e1e95d5d2 for removal 2024.10.22 02:11:53.196693 [ 1476 ] {} CacheMetadata: Key cf89f1f402501bfbc1f3539e1e95d5d2 is removed from metadata 2024.10.22 02:11:53.231126 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1411_41} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1411_41 to 202410_1207_1411_41 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:53.236489 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1411_41} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 5 parts: [202410_1207_1407_40, 202410_1411_1411_0] -> 202410_1207_1411_41 2024.10.22 02:11:53.258383 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e2a2970d43f3a71749170bb380a16fd9, offset: 0, size: 0 2024.10.22 02:11:53.261494 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e2a2970d43f3a71749170bb380a16fd9 for removal 2024.10.22 02:11:53.264866 [ 1476 ] {} CacheMetadata: Key e2a2970d43f3a71749170bb380a16fd9 is removed from metadata 2024.10.22 02:11:53.275855 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/urb/llltzypemipddhnlafsaynqwetptw. 2024.10.22 02:11:53.280523 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.88 MiB. 2024.10.22 02:11:53.466074 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 2.00 GiB, free memory in arenas 13.32 MiB, will set to 1.46 GiB (RSS), difference: -13.28 MiB 2024.10.22 02:11:53.869615 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_839_212} MergeTask::MergeProjectionsStage: Merge sorted 12682 rows, containing 12 columns (12 merged, 0 gathered) in 1.414087298 sec., 8968.328912887244 rows/sec., 1.40 MiB/sec. 2024.10.22 02:11:53.923474 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 110 entries to flush up to offset 91957 2024.10.22 02:11:54.153056 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_839_212} MergedBlockOutputStream: filled checksums 202410_1_839_212 (state Temporary) 2024.10.22 02:11:54.415714 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_839_212} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_839_212 to 202410_1_839_212 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:54.420739 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_839_212} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_838_211, 202410_839_839_0] -> 202410_1_839_212 2024.10.22 02:11:54.455882 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.45 MiB. 2024.10.22 02:11:54.472645 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: edf018e753e77ac5144e81cbc9a16bec, offset: 0, size: 0 2024.10.22 02:11:54.474346 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key edf018e753e77ac5144e81cbc9a16bec for removal 2024.10.22 02:11:54.478310 [ 1476 ] {} CacheMetadata: Key edf018e753e77ac5144e81cbc9a16bec is removed from metadata 2024.10.22 02:11:54.602943 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:54.606983 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:54.645136 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2c2563c21f8410f0c9c64c1fa4bf14d3, offset: 0, size: 0 2024.10.22 02:11:54.651324 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2c2563c21f8410f0c9c64c1fa4bf14d3 for removal 2024.10.22 02:11:54.655057 [ 1476 ] {} CacheMetadata: Key 2c2563c21f8410f0c9c64c1fa4bf14d3 is removed from metadata 2024.10.22 02:11:54.691313 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: df38d8eb26dace05c1476e15b1103dc3, offset: 0, size: 0 2024.10.22 02:11:54.696064 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key df38d8eb26dace05c1476e15b1103dc3 for removal 2024.10.22 02:11:54.700579 [ 1476 ] {} CacheMetadata: Key df38d8eb26dace05c1476e15b1103dc3 is removed from metadata 2024.10.22 02:11:54.725803 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1400_1400_0 (state Temporary) 2024.10.22 02:11:54.762828 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b2e04141c1cb063520148eb5c9cf4f0f, offset: 0, size: 0 2024.10.22 02:11:54.764935 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b2e04141c1cb063520148eb5c9cf4f0f for removal 2024.10.22 02:11:54.768203 [ 1476 ] {} CacheMetadata: Key b2e04141c1cb063520148eb5c9cf4f0f is removed from metadata 2024.10.22 02:11:54.816287 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6f074b0cc5235babe2bd3caa2361be6f, offset: 0, size: 0 2024.10.22 02:11:54.818931 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6f074b0cc5235babe2bd3caa2361be6f for removal 2024.10.22 02:11:54.824492 [ 1476 ] {} CacheMetadata: Key 6f074b0cc5235babe2bd3caa2361be6f is removed from metadata 2024.10.22 02:11:54.861598 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 134b43eea404d205a36546bfb2d6b310, offset: 0, size: 0 2024.10.22 02:11:54.863954 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 134b43eea404d205a36546bfb2d6b310 for removal 2024.10.22 02:11:54.867301 [ 1476 ] {} CacheMetadata: Key 134b43eea404d205a36546bfb2d6b310 is removed from metadata 2024.10.22 02:11:54.868054 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xcp/lbvulsefsskcrndudsokexwnmbrnb. 2024.10.22 02:11:54.962349 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 90764 2024.10.22 02:11:55.023426 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7431a10858fdcb4624492718a46b8ed8, offset: 0, size: 0 2024.10.22 02:11:55.026057 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7431a10858fdcb4624492718a46b8ed8 for removal 2024.10.22 02:11:55.029290 [ 1476 ] {} CacheMetadata: Key 7431a10858fdcb4624492718a46b8ed8 is removed from metadata 2024.10.22 02:11:55.110440 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1400_1400_0 to 202410_1413_1413_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:55.162613 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 91957 2024.10.22 02:11:55.211925 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a634f33db60fd5ffadc0bb98e78bccc3, offset: 0, size: 0 2024.10.22 02:11:55.214935 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a634f33db60fd5ffadc0bb98e78bccc3 for removal 2024.10.22 02:11:55.219232 [ 1476 ] {} CacheMetadata: Key a634f33db60fd5ffadc0bb98e78bccc3 is removed from metadata 2024.10.22 02:11:55.235297 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xcp/lbvulsefsskcrndudsokexwnmbrnb. 2024.10.22 02:11:55.239246 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ttg/oubsrsvzkrzponvtpfzqlpyercjbm. 2024.10.22 02:11:55.281421 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a45b46d3310a8ce7f855eff301d72a50, offset: 0, size: 0 2024.10.22 02:11:55.286853 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a45b46d3310a8ce7f855eff301d72a50 for removal 2024.10.22 02:11:55.293324 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 11.59 MiB, will set to 1.49 GiB (RSS), difference: -19.73 MiB 2024.10.22 02:11:55.295923 [ 1476 ] {} CacheMetadata: Key a45b46d3310a8ce7f855eff301d72a50 is removed from metadata 2024.10.22 02:11:55.347905 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 96aa6068289f2cd27e4a0e67dab366c2, offset: 0, size: 0 2024.10.22 02:11:55.349572 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 96aa6068289f2cd27e4a0e67dab366c2 for removal 2024.10.22 02:11:55.351338 [ 1476 ] {} CacheMetadata: Key 96aa6068289f2cd27e4a0e67dab366c2 is removed from metadata 2024.10.22 02:11:55.355765 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ttg/oubsrsvzkrzponvtpfzqlpyercjbm. 2024.10.22 02:11:55.367173 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 7 old parts to remove. Parts: [202410_1_144_25, 202410_145_149_1, 202410_150_150_0, 202410_151_151_0, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0] 2024.10.22 02:11:55.368580 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from filesystem (serially): Parts: [202410_1_144_25, 202410_145_149_1, 202410_150_150_0, 202410_151_151_0, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0] 2024.10.22 02:11:55.467802 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:55.474410 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.86 GiB. 2024.10.22 02:11:55.621483 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from memory: Parts: [202410_1_144_25, 202410_145_149_1, 202410_150_150_0, 202410_151_151_0, 202410_152_152_0, 202410_153_153_0, 202410_154_154_0] 2024.10.22 02:11:55.752344 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_779_172, 202410_780_780_0, 202410_781_781_0, 202410_782_782_0, 202410_783_783_0, 202410_784_784_0] 2024.10.22 02:11:55.755701 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_779_172, 202410_780_780_0, 202410_781_781_0, 202410_782_782_0, 202410_783_783_0, 202410_784_784_0] 2024.10.22 02:11:55.809678 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_816_816_0 (state Temporary) 2024.10.22 02:11:55.870091 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_816_816_0 to 202410_822_822_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:55.891856 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_779_172, 202410_780_780_0, 202410_781_781_0, 202410_782_782_0, 202410_783_783_0, 202410_784_784_0] 2024.10.22 02:11:55.904898 [ 1215 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_817_14 to 202410_822_822_0 2024.10.22 02:11:55.942215 [ 1215 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:55.950941 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.86 GiB. 2024.10.22 02:11:55.954945 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90764 2024.10.22 02:11:55.961503 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_817_14 to 202410_822_822_0 into Compact with storage Full 2024.10.22 02:11:56.091943 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:56.103045 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_817_14, total 7015 rows starting from the beginning of the part 2024.10.22 02:11:56.172173 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_818_818_0, total 98 rows starting from the beginning of the part 2024.10.22 02:11:56.271531 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/jej/qutcrsdvoyrowwnfehlvfrdzocrkz. 2024.10.22 02:11:56.408892 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_819_819_0, total 84 rows starting from the beginning of the part 2024.10.22 02:11:56.426360 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9b115df86358e9199e28f4e246808406, offset: 0, size: 0 2024.10.22 02:11:56.439302 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9b115df86358e9199e28f4e246808406 for removal 2024.10.22 02:11:56.446822 [ 1476 ] {} CacheMetadata: Key 9b115df86358e9199e28f4e246808406 is removed from metadata 2024.10.22 02:11:56.533407 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_820_820_0, total 41 rows starting from the beginning of the part 2024.10.22 02:11:56.556788 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_821_821_0, total 80 rows starting from the beginning of the part 2024.10.22 02:11:56.580309 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1d7a79f63a949cc66c29613a5807dc99, offset: 0, size: 0 2024.10.22 02:11:56.596019 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1d7a79f63a949cc66c29613a5807dc99 for removal 2024.10.22 02:11:56.596962 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_822_822_0, total 91 rows starting from the beginning of the part 2024.10.22 02:11:56.598875 [ 1476 ] {} CacheMetadata: Key 1d7a79f63a949cc66c29613a5807dc99 is removed from metadata 2024.10.22 02:11:56.614838 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/jej/qutcrsdvoyrowwnfehlvfrdzocrkz. 2024.10.22 02:11:56.828468 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1197835 2024.10.22 02:11:57.311510 [ 1276 ] {} DatabaseCatalog: Execution took 341 ms. 2024.10.22 02:11:57.549444 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 2.00 GiB, free memory in arenas 11.76 MiB, will set to 1.56 GiB (RSS), difference: -126.45 MiB 2024.10.22 02:11:57.654284 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 690 2024.10.22 02:11:57.766116 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_822_15/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:11:57.856844 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} MergeTask::MergeProjectionsStage: Merge sorted 7409 rows, containing 13 columns (13 merged, 0 gathered) in 1.898543202 sec., 3902.4658444406577 rows/sec., 1.69 MiB/sec. 2024.10.22 02:11:57.933088 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} MergedBlockOutputStream: filled checksums 202410_747_822_15 (state Temporary) 2024.10.22 02:11:57.960571 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:57.968190 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:57.970620 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:57.972335 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:58.052473 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_672_672_0 (state Temporary) 2024.10.22 02:11:58.189579 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rmt/kjkhzmqgcmiivnrbwsvmdspnvamxm. 2024.10.22 02:11:58.235756 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_795_795_0 (state Temporary) 2024.10.22 02:11:58.263741 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1259eab84f22c693f60df9e067fc4d44, offset: 0, size: 0 2024.10.22 02:11:58.270166 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_822_15 to 202410_747_822_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:58.275667 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_822_15} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_817_14, 202410_822_822_0] -> 202410_747_822_15 2024.10.22 02:11:58.297967 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_795_795_0 to 202410_802_802_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:58.353258 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.64 MiB. 2024.10.22 02:11:58.358262 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_672_672_0 to 202410_682_682_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:11:58.362276 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1197835 2024.10.22 02:11:58.349077 [ 1252 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_797_797_0 to 202410_802_802_0 2024.10.22 02:11:58.367068 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1259eab84f22c693f60df9e067fc4d44 for removal 2024.10.22 02:11:58.370776 [ 1476 ] {} CacheMetadata: Key 1259eab84f22c693f60df9e067fc4d44 is removed from metadata 2024.10.22 02:11:58.375290 [ 1252 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:58.383335 [ 1252 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:11:58.424175 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3c0a59f3f81267b981d7a4484fc453e5, offset: 0, size: 0 2024.10.22 02:11:58.428130 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3c0a59f3f81267b981d7a4484fc453e5 for removal 2024.10.22 02:11:58.433815 [ 1476 ] {} CacheMetadata: Key 3c0a59f3f81267b981d7a4484fc453e5 is removed from metadata 2024.10.22 02:11:58.442581 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_797_797_0 to 202410_802_802_0 into Compact with storage Full 2024.10.22 02:11:58.444003 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rmt/kjkhzmqgcmiivnrbwsvmdspnvamxm. 2024.10.22 02:11:58.479549 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:11:58.497146 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_797_797_0, total 1372 rows starting from the beginning of the part 2024.10.22 02:11:58.548641 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_798_798_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:11:58.552678 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 690 2024.10.22 02:11:58.606474 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_799_799_0, total 1363 rows starting from the beginning of the part 2024.10.22 02:11:58.706561 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 264.11 MiB using storage policy from min volume index 0 2024.10.22 02:11:58.753469 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_800_800_0, total 1717 rows starting from the beginning of the part 2024.10.22 02:11:58.940579 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_801_801_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:11:59.132331 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_802_802_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:11:59.275433 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 117 entries to flush up to offset 92074 2024.10.22 02:11:59.292343 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:11:59.442934 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/sdh/fsjhgcedsqptjjbciwzdszcvmuubv (LogSeriesLimiter: on interval from 2024-10-22 02:11:41 to 2024-10-22 02:11:59 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:11:59.499419 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 10.52 MiB, will set to 1.56 GiB (RSS), difference: 36.76 MiB 2024.10.22 02:11:59.816003 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:11:59.900727 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 12640 2024.10.22 02:11:59.903517 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:11:59.971774 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1403827 bytes for f4b88f4935a83b534d483ae40a10490a:0) 2024.10.22 02:11:59.974984 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: fcc0799ccbebe353ead076f68434238b, offset: 0, size: 1976947 2024.10.22 02:11:59.988710 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key fcc0799ccbebe353ead076f68434238b for removal 2024.10.22 02:12:00.002478 [ 1476 ] {} CacheMetadata: Key fcc0799ccbebe353ead076f68434238b is removed from metadata 2024.10.22 02:12:00.006648 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: fcd63e5916b77d8e7950f0f856ddbda4, offset: 0, size: 1103 2024.10.22 02:12:00.012190 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3281b005ddd72389454e819899d12795, offset: 0, size: 0 2024.10.22 02:12:00.016717 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3281b005ddd72389454e819899d12795 for removal 2024.10.22 02:12:00.017068 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key fcd63e5916b77d8e7950f0f856ddbda4 for removal 2024.10.22 02:12:00.039788 [ 1476 ] {} CacheMetadata: Key fcd63e5916b77d8e7950f0f856ddbda4 is removed from metadata 2024.10.22 02:12:00.054256 [ 1476 ] {} CacheMetadata: Key 3281b005ddd72389454e819899d12795 is removed from metadata 2024.10.22 02:12:00.204328 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7dea6b897bb42021fe9f41adb3b6513c, offset: 0, size: 0 2024.10.22 02:12:00.212347 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7dea6b897bb42021fe9f41adb3b6513c for removal 2024.10.22 02:12:00.222270 [ 1476 ] {} CacheMetadata: Key 7dea6b897bb42021fe9f41adb3b6513c is removed from metadata 2024.10.22 02:12:00.255338 [ 1739 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tfn/hlahcdwboinsaontofrsooggajvsj, size 3500979 2024.10.22 02:12:00.270720 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1401_1401_0 (state Temporary) 2024.10.22 02:12:00.275037 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:00.279681 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:00.357700 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1401_1401_0 to 202410_1414_1414_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:00.383742 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c294c377c0047efd7e8ae07b13cbab37, offset: 0, size: 0 2024.10.22 02:12:00.389430 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c294c377c0047efd7e8ae07b13cbab37 for removal 2024.10.22 02:12:00.396102 [ 1476 ] {} CacheMetadata: Key c294c377c0047efd7e8ae07b13cbab37 is removed from metadata 2024.10.22 02:12:00.433087 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_831_831_0 (state Temporary) 2024.10.22 02:12:00.546963 [ 1154 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 323 ms. 2024.10.22 02:12:00.618472 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_831_831_0 to 202410_844_844_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:00.649460 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 92074 2024.10.22 02:12:00.665827 [ 1204 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_839_212 to 202410_840_840_0 2024.10.22 02:12:00.670947 [ 1204 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:00.677074 [ 1204 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:00.808665 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 67a8bca42a9744b9edd64dd4f2ae4dc1, offset: 0, size: 0 2024.10.22 02:12:00.814884 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 67a8bca42a9744b9edd64dd4f2ae4dc1 for removal 2024.10.22 02:12:00.815990 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12640 2024.10.22 02:12:00.825696 [ 1476 ] {} CacheMetadata: Key 67a8bca42a9744b9edd64dd4f2ae4dc1 is removed from metadata 2024.10.22 02:12:00.825035 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rzr/pihqdwqnjqonclmqqnrwnyrksridb. 2024.10.22 02:12:00.856194 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_840_213} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_839_212 to 202410_840_840_0 into Compact with storage Full 2024.10.22 02:12:00.957717 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column HID 2024.10.22 02:12:00.977132 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column HID 2024.10.22 02:12:00.984758 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8ba95b09686c2258d0b345807fb1b61b, offset: 0, size: 0 2024.10.22 02:12:00.988177 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8ba95b09686c2258d0b345807fb1b61b for removal 2024.10.22 02:12:00.992768 [ 1476 ] {} CacheMetadata: Key 8ba95b09686c2258d0b345807fb1b61b is removed from metadata 2024.10.22 02:12:00.995107 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column HID 2024.10.22 02:12:00.999449 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_840_213} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:01.021735 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_840_213} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_839_212, total 12682 rows starting from the beginning of the part 2024.10.22 02:12:01.035060 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column HID 2024.10.22 02:12:01.073543 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_840_213} MergeTreeSequentialSource: Reading 2 marks from part 202410_840_840_0, total 13 rows starting from the beginning of the part 2024.10.22 02:12:01.078032 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column HID 2024.10.22 02:12:01.132034 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column HID 2024.10.22 02:12:01.160545 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column HID 2024.10.22 02:12:01.250621 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6bf87f0fb161cdda328b78497b875657, offset: 0, size: 0 2024.10.22 02:12:01.255276 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6bf87f0fb161cdda328b78497b875657 for removal 2024.10.22 02:12:01.259815 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:12:01.259499 [ 1476 ] {} CacheMetadata: Key 6bf87f0fb161cdda328b78497b875657 is removed from metadata 2024.10.22 02:12:01.266551 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:12:01.334569 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rzr/pihqdwqnjqonclmqqnrwnyrksridb. 2024.10.22 02:12:01.383346 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1291db0e520636a4dd7e30c4477f3c30, offset: 0, size: 0 2024.10.22 02:12:01.387505 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1291db0e520636a4dd7e30c4477f3c30 for removal 2024.10.22 02:12:01.396580 [ 1476 ] {} CacheMetadata: Key 1291db0e520636a4dd7e30c4477f3c30 is removed from metadata 2024.10.22 02:12:01.469771 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} MergeTask::MergeProjectionsStage: Merge sorted 9842 rows, containing 4 columns (4 merged, 0 gathered) in 3.03996253 sec., 3237.5399048092872 rows/sec., 50.59 KiB/sec. 2024.10.22 02:12:01.746580 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} MergedBlockOutputStream: filled checksums 202410_797_802_1 (state Temporary) 2024.10.22 02:12:01.769538 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cce680d033832906870c1cfb4963c3f6, offset: 0, size: 0 2024.10.22 02:12:01.771543 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cce680d033832906870c1cfb4963c3f6 for removal 2024.10.22 02:12:01.772597 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 2.00 GiB, free memory in arenas 11.03 MiB, will set to 1.56 GiB (RSS), difference: -108.13 MiB 2024.10.22 02:12:01.784356 [ 1476 ] {} CacheMetadata: Key cce680d033832906870c1cfb4963c3f6 is removed from metadata 2024.10.22 02:12:01.785304 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uri/jdkorukihivsixzvzskikmkfyoeol. 2024.10.22 02:12:01.801288 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_797_802_1 to 202410_797_802_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:01.914152 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_797_802_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_797_797_0, 202410_802_802_0] -> 202410_797_802_1 2024.10.22 02:12:01.954936 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f643ac0bf821bc801ca84fb319f25863, offset: 0, size: 0 2024.10.22 02:12:01.958761 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f643ac0bf821bc801ca84fb319f25863 for removal 2024.10.22 02:12:01.960903 [ 1476 ] {} CacheMetadata: Key f643ac0bf821bc801ca84fb319f25863 is removed from metadata 2024.10.22 02:12:01.985690 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:12:02.012015 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 33b45b69f267a596aac113b34128947a, offset: 0, size: 0 2024.10.22 02:12:02.018310 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 33b45b69f267a596aac113b34128947a for removal 2024.10.22 02:12:02.027312 [ 1476 ] {} CacheMetadata: Key 33b45b69f267a596aac113b34128947a is removed from metadata 2024.10.22 02:12:02.048007 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uri/jdkorukihivsixzvzskikmkfyoeol. 2024.10.22 02:12:02.103378 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d8b50fd83031e036d2e6aa8fdff0742a, offset: 0, size: 0 2024.10.22 02:12:02.116173 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d8b50fd83031e036d2e6aa8fdff0742a for removal 2024.10.22 02:12:02.124266 [ 1476 ] {} CacheMetadata: Key d8b50fd83031e036d2e6aa8fdff0742a is removed from metadata 2024.10.22 02:12:02.271306 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8697d1795fd323bda640c1cc81978fc7, offset: 0, size: 0 2024.10.22 02:12:02.274160 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8697d1795fd323bda640c1cc81978fc7 for removal 2024.10.22 02:12:02.280383 [ 1476 ] {} CacheMetadata: Key 8697d1795fd323bda640c1cc81978fc7 is removed from metadata 2024.10.22 02:12:02.205800 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_840_213} MergeTask::MergeProjectionsStage: Merge sorted 12695 rows, containing 12 columns (12 merged, 0 gathered) in 1.513693669 sec., 8386.769569028302 rows/sec., 1.31 MiB/sec. 2024.10.22 02:12:02.425499 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_840_213} MergedBlockOutputStream: filled checksums 202410_1_840_213 (state Temporary) 2024.10.22 02:12:02.492598 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_840_213} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_840_213 to 202410_1_840_213 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:02.500790 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_840_213} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_839_212, 202410_840_840_0] -> 202410_1_840_213 2024.10.22 02:12:02.652222 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.45 MiB. 2024.10.22 02:12:02.839290 [ 1253 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 02:12:03.244589 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/cra/ngvjlhmcgragjawzxhqxoxbysrcit. 2024.10.22 02:12:03.387863 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8471f8fe03e84a1ecaca27b0837643de, offset: 0, size: 0 2024.10.22 02:12:03.390772 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8471f8fe03e84a1ecaca27b0837643de for removal 2024.10.22 02:12:03.399328 [ 1476 ] {} CacheMetadata: Key 8471f8fe03e84a1ecaca27b0837643de is removed from metadata 2024.10.22 02:12:03.468623 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 127 entries to flush up to offset 90891 2024.10.22 02:12:03.547153 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ba5e7351c77d231338f1f07b88190d28, offset: 0, size: 0 2024.10.22 02:12:03.550045 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ba5e7351c77d231338f1f07b88190d28 for removal 2024.10.22 02:12:03.552778 [ 1476 ] {} CacheMetadata: Key ba5e7351c77d231338f1f07b88190d28 is removed from metadata 2024.10.22 02:12:03.559155 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/cra/ngvjlhmcgragjawzxhqxoxbysrcit. 2024.10.22 02:12:03.605072 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 2.00 GiB, free memory in arenas 10.77 MiB, will set to 1.57 GiB (RSS), difference: -104.59 MiB 2024.10.22 02:12:03.658712 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_796_150/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:12:03.829965 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:03.835810 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:03.980657 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_817_817_0 (state Temporary) 2024.10.22 02:12:04.046924 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vgi/djselzodcomnpacpzggvbbyhygsgv (LogSeriesLimiter: on interval from 2024-10-22 02:11:59 to 2024-10-22 02:12:04 accepted series 1 / 167 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:12:04.066728 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_817_817_0 to 202410_823_823_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:04.195680 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90891 2024.10.22 02:12:04.458636 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/sdt/eqiqlfebhzghwrtwhbrtofcpjhqpv. 2024.10.22 02:12:04.563397 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 9 file segments (while reserving 1048576 bytes for 24c2dab7d4d02b4f182eed2dcc18bdb1:0) 2024.10.22 02:12:04.566681 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8fb335c31e375251c15a56172221a090, offset: 0, size: 723 2024.10.22 02:12:04.569203 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 8fb335c31e375251c15a56172221a090 for removal 2024.10.22 02:12:04.571221 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: e3297eac6a3aa9f15740bad46be3a31d, offset: 0, size: 730 2024.10.22 02:12:04.571620 [ 1476 ] {} CacheMetadata: Key 8fb335c31e375251c15a56172221a090 is removed from metadata 2024.10.22 02:12:04.574747 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key e3297eac6a3aa9f15740bad46be3a31d for removal 2024.10.22 02:12:04.578688 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: cec8b008e2c4297f30f35b8a4832c84d, offset: 0, size: 84341 2024.10.22 02:12:04.592922 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key cec8b008e2c4297f30f35b8a4832c84d for removal 2024.10.22 02:12:04.597826 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: ca25f686318e95ad689cb8c204bccd8c, offset: 0, size: 713 2024.10.22 02:12:04.601908 [ 1476 ] {} CacheMetadata: Key cec8b008e2c4297f30f35b8a4832c84d is removed from metadata 2024.10.22 02:12:04.603205 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key ca25f686318e95ad689cb8c204bccd8c for removal 2024.10.22 02:12:04.607763 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: da971a9da574cd8a5dbe04f54d82ecca, offset: 0, size: 414 2024.10.22 02:12:04.610828 [ 1476 ] {} CacheMetadata: Key ca25f686318e95ad689cb8c204bccd8c is removed from metadata 2024.10.22 02:12:04.616260 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key da971a9da574cd8a5dbe04f54d82ecca for removal 2024.10.22 02:12:04.626881 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c73fddfcc92d372f2d6fcd7602383dd6, offset: 0, size: 930 2024.10.22 02:12:04.630794 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c73fddfcc92d372f2d6fcd7602383dd6 for removal 2024.10.22 02:12:04.636011 [ 1476 ] {} CacheMetadata: Key da971a9da574cd8a5dbe04f54d82ecca is removed from metadata 2024.10.22 02:12:04.644931 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0d876fd2273c4253da611c75d7d0745c, offset: 0, size: 72477 2024.10.22 02:12:04.650231 [ 1476 ] {} CacheMetadata: Key c73fddfcc92d372f2d6fcd7602383dd6 is removed from metadata 2024.10.22 02:12:04.651633 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 0d876fd2273c4253da611c75d7d0745c for removal 2024.10.22 02:12:04.654057 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1335f89f2e90f8420580e44cac511bf4, offset: 0, size: 1359 2024.10.22 02:12:04.656137 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 1335f89f2e90f8420580e44cac511bf4 for removal 2024.10.22 02:12:04.657139 [ 1476 ] {} CacheMetadata: Key 0d876fd2273c4253da611c75d7d0745c is removed from metadata 2024.10.22 02:12:04.771093 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7dc8a76a66d0195a4bce1b8947e874fc, offset: 0, size: 0 2024.10.22 02:12:04.861268 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 131 entries to flush up to offset 92205 2024.10.22 02:12:04.862780 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7dc8a76a66d0195a4bce1b8947e874fc for removal 2024.10.22 02:12:04.864214 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7ba4c546dcf2d99aab8f9261f68a9ec7, offset: 0, size: 807 2024.10.22 02:12:04.869499 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 7ba4c546dcf2d99aab8f9261f68a9ec7 for removal 2024.10.22 02:12:04.870118 [ 1476 ] {} CacheMetadata: Key 7dc8a76a66d0195a4bce1b8947e874fc is removed from metadata 2024.10.22 02:12:04.876861 [ 1476 ] {} CacheMetadata: Key 7ba4c546dcf2d99aab8f9261f68a9ec7 is removed from metadata 2024.10.22 02:12:04.888434 [ 1476 ] {} CacheMetadata: Key 1335f89f2e90f8420580e44cac511bf4 is removed from metadata 2024.10.22 02:12:04.904461 [ 1476 ] {} CacheMetadata: Key e3297eac6a3aa9f15740bad46be3a31d is removed from metadata 2024.10.22 02:12:04.908175 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e2889a3b4c31f232bef8260255a69a2e, offset: 0, size: 0 2024.10.22 02:12:04.913602 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e2889a3b4c31f232bef8260255a69a2e for removal 2024.10.22 02:12:04.929509 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/sdt/eqiqlfebhzghwrtwhbrtofcpjhqpv. 2024.10.22 02:12:04.939465 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xjm/pzwphhdzeutvbzrqcjlhupdtvebnv. 2024.10.22 02:12:04.976925 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 392c35869a2db4990578a9e68bea478e, offset: 0, size: 0 2024.10.22 02:12:04.980841 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 392c35869a2db4990578a9e68bea478e for removal 2024.10.22 02:12:05.042774 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 27b99a928414d3044c04a84bffd6e2f9, offset: 0, size: 0 2024.10.22 02:12:05.045794 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 27b99a928414d3044c04a84bffd6e2f9 for removal 2024.10.22 02:12:05.053355 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xjm/pzwphhdzeutvbzrqcjlhupdtvebnv. 2024.10.22 02:12:05.121764 [ 1476 ] {} CacheMetadata: Key 27b99a928414d3044c04a84bffd6e2f9 is removed from metadata 2024.10.22 02:12:05.232718 [ 1476 ] {} CacheMetadata: Key 392c35869a2db4990578a9e68bea478e is removed from metadata 2024.10.22 02:12:05.241096 [ 1476 ] {} CacheMetadata: Key e2889a3b4c31f232bef8260255a69a2e is removed from metadata 2024.10.22 02:12:05.368328 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1199375 2024.10.22 02:12:05.626620 [ 1158 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:12:05.640787 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:05.647312 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:05.654863 [ 1158 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:12:05.752914 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1402_1402_0 (state Temporary) 2024.10.22 02:12:05.805824 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1402_1402_0 to 202410_1415_1415_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:05.870300 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 92205 2024.10.22 02:12:06.054303 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 2.00 GiB, free memory in arenas 10.56 MiB, will set to 1.56 GiB (RSS), difference: -121.29 MiB 2024.10.22 02:12:06.056433 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 691 2024.10.22 02:12:06.133390 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:06.140042 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:06.254867 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 37 file segments (while reserving 1048576 bytes for 392c35869a2db4990578a9e68bea478e:0) 2024.10.22 02:12:06.259020 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1d1443b387b6849ac461a1da19fc0dcb, offset: 0, size: 260305 2024.10.22 02:12:06.263420 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1d1443b387b6849ac461a1da19fc0dcb for removal 2024.10.22 02:12:06.267592 [ 1476 ] {} CacheMetadata: Key 1d1443b387b6849ac461a1da19fc0dcb is removed from metadata 2024.10.22 02:12:06.267662 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ad14964be69e67a9e465d22392c03837, offset: 0, size: 614 2024.10.22 02:12:06.272860 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ad14964be69e67a9e465d22392c03837 for removal 2024.10.22 02:12:06.277160 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dc1ee3af2889bdb6cda4ff305e796d31, offset: 0, size: 167 2024.10.22 02:12:06.279884 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_796_796_0 (state Temporary) 2024.10.22 02:12:06.282667 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dc1ee3af2889bdb6cda4ff305e796d31 for removal 2024.10.22 02:12:06.290946 [ 1476 ] {} CacheMetadata: Key dc1ee3af2889bdb6cda4ff305e796d31 is removed from metadata 2024.10.22 02:12:06.291263 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d841444c7484557e9507473c20a121dd, offset: 0, size: 438761 2024.10.22 02:12:06.299735 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d841444c7484557e9507473c20a121dd for removal 2024.10.22 02:12:06.306626 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ecc4e2f7c3b531b92d820551c9b69e28, offset: 0, size: 23913 2024.10.22 02:12:06.308957 [ 1476 ] {} CacheMetadata: Key d841444c7484557e9507473c20a121dd is removed from metadata 2024.10.22 02:12:06.309225 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ecc4e2f7c3b531b92d820551c9b69e28 for removal 2024.10.22 02:12:06.317225 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ee8717e2478a948149a96769e52085e1, offset: 0, size: 11065 2024.10.22 02:12:06.320656 [ 1476 ] {} CacheMetadata: Key ecc4e2f7c3b531b92d820551c9b69e28 is removed from metadata 2024.10.22 02:12:06.327665 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ee8717e2478a948149a96769e52085e1 for removal 2024.10.22 02:12:06.338561 [ 1476 ] {} CacheMetadata: Key ee8717e2478a948149a96769e52085e1 is removed from metadata 2024.10.22 02:12:06.346857 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: be4e0db3b004cf8633de4bf2f36309a3, offset: 0, size: 495 2024.10.22 02:12:06.349603 [ 1476 ] {} CacheMetadata: Key ad14964be69e67a9e465d22392c03837 is removed from metadata 2024.10.22 02:12:06.349703 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key be4e0db3b004cf8633de4bf2f36309a3 for removal 2024.10.22 02:12:06.352892 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 724d78779ce49f0663227a6ae189c81b, offset: 0, size: 163 2024.10.22 02:12:06.355335 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 724d78779ce49f0663227a6ae189c81b for removal 2024.10.22 02:12:06.356144 [ 1476 ] {} CacheMetadata: Key be4e0db3b004cf8633de4bf2f36309a3 is removed from metadata 2024.10.22 02:12:06.357257 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d0f79f148309301393f8c3a7533be7b5, offset: 0, size: 4 2024.10.22 02:12:06.360658 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d0f79f148309301393f8c3a7533be7b5 for removal 2024.10.22 02:12:06.372372 [ 1476 ] {} CacheMetadata: Key 724d78779ce49f0663227a6ae189c81b is removed from metadata 2024.10.22 02:12:06.373505 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 01a14955519cf3b2bba7903e2c592808, offset: 0, size: 69 2024.10.22 02:12:06.376518 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 01a14955519cf3b2bba7903e2c592808 for removal 2024.10.22 02:12:06.381029 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 31d1c92754696cd477203b9379ad37db, offset: 0, size: 37 2024.10.22 02:12:06.386931 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 31d1c92754696cd477203b9379ad37db for removal 2024.10.22 02:12:06.392878 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9d286bcb27bfb4a2d5d5401c5b38208b, offset: 0, size: 537 2024.10.22 02:12:06.394843 [ 1476 ] {} CacheMetadata: Key 31d1c92754696cd477203b9379ad37db is removed from metadata 2024.10.22 02:12:06.401812 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9d286bcb27bfb4a2d5d5401c5b38208b for removal 2024.10.22 02:12:06.405959 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: bdf64d1224b3b0d231648cbbace1224b, offset: 0, size: 629 2024.10.22 02:12:06.407349 [ 1476 ] {} CacheMetadata: Key 9d286bcb27bfb4a2d5d5401c5b38208b is removed from metadata 2024.10.22 02:12:06.429041 [ 1476 ] {} CacheMetadata: Key 01a14955519cf3b2bba7903e2c592808 is removed from metadata 2024.10.22 02:12:06.451958 [ 1476 ] {} CacheMetadata: Key d0f79f148309301393f8c3a7533be7b5 is removed from metadata 2024.10.22 02:12:06.458763 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:06.460257 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:06.409835 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key bdf64d1224b3b0d231648cbbace1224b for removal 2024.10.22 02:12:06.515911 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 67bdb1c9cd995996cad4bb8d70806bda, offset: 0, size: 761 2024.10.22 02:12:06.520715 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 67bdb1c9cd995996cad4bb8d70806bda for removal 2024.10.22 02:12:06.524740 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9a6cd5aa43b6c7de2fd46605664ba325, offset: 0, size: 698 2024.10.22 02:12:06.531274 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9a6cd5aa43b6c7de2fd46605664ba325 for removal 2024.10.22 02:12:06.537300 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 71b3792a583c0180ed32fddfcda1a67a, offset: 0, size: 33336 2024.10.22 02:12:06.542656 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 71b3792a583c0180ed32fddfcda1a67a for removal 2024.10.22 02:12:06.548938 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5771fd256ae06936f0c21c5de6a91af9, offset: 0, size: 35590 2024.10.22 02:12:06.562531 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5771fd256ae06936f0c21c5de6a91af9 for removal 2024.10.22 02:12:06.565203 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0329cf53a1fa7c606b3832266bc7b513, offset: 0, size: 941 2024.10.22 02:12:06.565557 [ 1476 ] {} CacheMetadata: Key 5771fd256ae06936f0c21c5de6a91af9 is removed from metadata 2024.10.22 02:12:06.568568 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0329cf53a1fa7c606b3832266bc7b513 for removal 2024.10.22 02:12:06.571054 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 258ba14c06af87a09cf6451ed38782f1, offset: 0, size: 195 2024.10.22 02:12:06.573828 [ 1476 ] {} CacheMetadata: Key 0329cf53a1fa7c606b3832266bc7b513 is removed from metadata 2024.10.22 02:12:06.576662 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 258ba14c06af87a09cf6451ed38782f1 for removal 2024.10.22 02:12:06.581526 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d2684604fb977e712050e54e4cee4f86, offset: 0, size: 1013 2024.10.22 02:12:06.599572 [ 1476 ] {} CacheMetadata: Key 258ba14c06af87a09cf6451ed38782f1 is removed from metadata 2024.10.22 02:12:06.600640 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_796_796_0 to 202410_803_803_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:06.599295 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d2684604fb977e712050e54e4cee4f86 for removal 2024.10.22 02:12:06.607693 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: af494e564cb714c49057c3683e59dd37, offset: 0, size: 6788 2024.10.22 02:12:06.608979 [ 1476 ] {} CacheMetadata: Key d2684604fb977e712050e54e4cee4f86 is removed from metadata 2024.10.22 02:12:06.614975 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key af494e564cb714c49057c3683e59dd37 for removal 2024.10.22 02:12:06.622883 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5633902a05ede7b13e202192d4dde352, offset: 0, size: 163275 2024.10.22 02:12:06.631508 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5633902a05ede7b13e202192d4dde352 for removal 2024.10.22 02:12:06.635424 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 51a09b6d2d92a0927fa1ebb269413a37, offset: 0, size: 144926 2024.10.22 02:12:06.637706 [ 1476 ] {} CacheMetadata: Key 5633902a05ede7b13e202192d4dde352 is removed from metadata 2024.10.22 02:12:06.640479 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 51a09b6d2d92a0927fa1ebb269413a37 for removal 2024.10.22 02:12:06.644660 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d224e665116bfd3521f5186e311b8c53, offset: 0, size: 720 2024.10.22 02:12:06.645254 [ 1476 ] {} CacheMetadata: Key 51a09b6d2d92a0927fa1ebb269413a37 is removed from metadata 2024.10.22 02:12:06.648340 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d224e665116bfd3521f5186e311b8c53 for removal 2024.10.22 02:12:06.650937 [ 1476 ] {} CacheMetadata: Key d224e665116bfd3521f5186e311b8c53 is removed from metadata 2024.10.22 02:12:06.651626 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: caee001a89d03b7cce4a5968828cac21, offset: 0, size: 571 2024.10.22 02:12:06.657140 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key caee001a89d03b7cce4a5968828cac21 for removal 2024.10.22 02:12:06.658142 [ 1476 ] {} CacheMetadata: Key af494e564cb714c49057c3683e59dd37 is removed from metadata 2024.10.22 02:12:06.659531 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dc6289cb7b6670de21c509fff217dd53, offset: 0, size: 702 2024.10.22 02:12:06.660882 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dc6289cb7b6670de21c509fff217dd53 for removal 2024.10.22 02:12:06.661715 [ 1476 ] {} CacheMetadata: Key caee001a89d03b7cce4a5968828cac21 is removed from metadata 2024.10.22 02:12:06.664450 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c85234315424aafeecbcf8ecb07048ce, offset: 0, size: 812 2024.10.22 02:12:06.666130 [ 1476 ] {} CacheMetadata: Key dc6289cb7b6670de21c509fff217dd53 is removed from metadata 2024.10.22 02:12:06.666688 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c85234315424aafeecbcf8ecb07048ce for removal 2024.10.22 02:12:06.667796 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8393068da9622d7a941f8fd5c8307cf7, offset: 0, size: 993 2024.10.22 02:12:06.669957 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8393068da9622d7a941f8fd5c8307cf7 for removal 2024.10.22 02:12:06.670573 [ 1476 ] {} CacheMetadata: Key c85234315424aafeecbcf8ecb07048ce is removed from metadata 2024.10.22 02:12:06.671745 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f377502b9558cba069661b33ca0dd67a, offset: 0, size: 725 2024.10.22 02:12:06.674497 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f377502b9558cba069661b33ca0dd67a for removal 2024.10.22 02:12:06.675152 [ 1476 ] {} CacheMetadata: Key 8393068da9622d7a941f8fd5c8307cf7 is removed from metadata 2024.10.22 02:12:06.684708 [ 1476 ] {} CacheMetadata: Key f377502b9558cba069661b33ca0dd67a is removed from metadata 2024.10.22 02:12:06.697702 [ 1476 ] {} CacheMetadata: Key 71b3792a583c0180ed32fddfcda1a67a is removed from metadata 2024.10.22 02:12:06.680984 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 77fafe87234133ba6a522d2868e12e83, offset: 0, size: 974 2024.10.22 02:12:06.707815 [ 1476 ] {} CacheMetadata: Key 9a6cd5aa43b6c7de2fd46605664ba325 is removed from metadata 2024.10.22 02:12:06.708533 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_673_673_0 (state Temporary) 2024.10.22 02:12:06.712276 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 77fafe87234133ba6a522d2868e12e83 for removal 2024.10.22 02:12:06.721607 [ 1476 ] {} CacheMetadata: Key 77fafe87234133ba6a522d2868e12e83 is removed from metadata 2024.10.22 02:12:06.720080 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d66af0e0055b41fc07239585cb978b1a, offset: 0, size: 168 2024.10.22 02:12:06.735288 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d66af0e0055b41fc07239585cb978b1a for removal 2024.10.22 02:12:06.743435 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d6900ec23ff13ce40a5d1f0bac342af8, offset: 0, size: 947 2024.10.22 02:12:06.748208 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d6900ec23ff13ce40a5d1f0bac342af8 for removal 2024.10.22 02:12:06.748699 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1199375 2024.10.22 02:12:06.749137 [ 1476 ] {} CacheMetadata: Key d66af0e0055b41fc07239585cb978b1a is removed from metadata 2024.10.22 02:12:06.751685 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a47fb910b86535cf9695ddcb24d3e4ac, offset: 0, size: 817 2024.10.22 02:12:06.754011 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a47fb910b86535cf9695ddcb24d3e4ac for removal 2024.10.22 02:12:06.755529 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a32af5c9bb509cdab755132c9d69ff0c, offset: 0, size: 1159 2024.10.22 02:12:06.759005 [ 1476 ] {} CacheMetadata: Key a47fb910b86535cf9695ddcb24d3e4ac is removed from metadata 2024.10.22 02:12:06.760109 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a32af5c9bb509cdab755132c9d69ff0c for removal 2024.10.22 02:12:06.765347 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 658b6ca5c061b0b8425147469cfd809d, offset: 0, size: 765 2024.10.22 02:12:06.769270 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 658b6ca5c061b0b8425147469cfd809d for removal 2024.10.22 02:12:06.769732 [ 1476 ] {} CacheMetadata: Key a32af5c9bb509cdab755132c9d69ff0c is removed from metadata 2024.10.22 02:12:06.773928 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c71f1da4a3efc171e967fe6f1bf48605, offset: 0, size: 728 2024.10.22 02:12:06.776594 [ 1476 ] {} CacheMetadata: Key 658b6ca5c061b0b8425147469cfd809d is removed from metadata 2024.10.22 02:12:06.778800 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c71f1da4a3efc171e967fe6f1bf48605 for removal 2024.10.22 02:12:06.784553 [ 1476 ] {} CacheMetadata: Key c71f1da4a3efc171e967fe6f1bf48605 is removed from metadata 2024.10.22 02:12:06.786190 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8ba6e3f2429e411398b2ef98d1051545, offset: 0, size: 1018 2024.10.22 02:12:06.789007 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8ba6e3f2429e411398b2ef98d1051545 for removal 2024.10.22 02:12:06.808073 [ 1476 ] {} CacheMetadata: Key 8ba6e3f2429e411398b2ef98d1051545 is removed from metadata 2024.10.22 02:12:06.828650 [ 1476 ] {} CacheMetadata: Key d6900ec23ff13ce40a5d1f0bac342af8 is removed from metadata 2024.10.22 02:12:06.847324 [ 1476 ] {} CacheMetadata: Key 67bdb1c9cd995996cad4bb8d70806bda is removed from metadata 2024.10.22 02:12:06.854712 [ 1476 ] {} CacheMetadata: Key bdf64d1224b3b0d231648cbbace1224b is removed from metadata 2024.10.22 02:12:06.874463 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_673_673_0 to 202410_683_683_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:06.992834 [ 1180 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_678_138 to 202410_683_683_0 2024.10.22 02:12:07.022923 [ 1180 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:07.030077 [ 1180 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:07.055694 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_678_138 to 202410_683_683_0 into Compact with storage Full 2024.10.22 02:12:07.065461 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 691 2024.10.22 02:12:07.260679 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:07.269129 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_678_138, total 722 rows starting from the beginning of the part 2024.10.22 02:12:07.447665 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_679_679_0, total 1 rows starting from the beginning of the part 2024.10.22 02:12:07.457091 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 24c2dab7d4d02b4f182eed2dcc18bdb1:0) 2024.10.22 02:12:07.461056 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: fe2a71089fb2b02b63c123fb480a7765, offset: 0, size: 10235190 2024.10.22 02:12:07.471950 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key fe2a71089fb2b02b63c123fb480a7765 for removal 2024.10.22 02:12:07.474632 [ 1476 ] {} CacheMetadata: Key fe2a71089fb2b02b63c123fb480a7765 is removed from metadata 2024.10.22 02:12:07.478612 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 96c3dd217be3e106ce2f8b5337e65a3e, offset: 0, size: 980 2024.10.22 02:12:07.488177 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 96c3dd217be3e106ce2f8b5337e65a3e for removal 2024.10.22 02:12:07.507974 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_680_680_0, total 1 rows starting from the beginning of the part 2024.10.22 02:12:07.509272 [ 1476 ] {} CacheMetadata: Key 96c3dd217be3e106ce2f8b5337e65a3e is removed from metadata 2024.10.22 02:12:07.532646 [ 1163 ] {} DatabaseCatalog: Execution took 215 ms. 2024.10.22 02:12:07.668601 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_681_681_0, total 1 rows starting from the beginning of the part 2024.10.22 02:12:07.679815 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:07.682902 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.85 GiB. 2024.10.22 02:12:07.776023 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_165_169_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:07.906105 [ 1199 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 02:12:07.970448 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_682_682_0, total 1 rows starting from the beginning of the part 2024.10.22 02:12:08.054526 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_683_683_0, total 1 rows starting from the beginning of the part 2024.10.22 02:12:08.324255 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 12656 2024.10.22 02:12:08.487474 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:08.495937 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:08.651556 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_832_832_0 (state Temporary) 2024.10.22 02:12:08.695098 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_832_832_0 to 202410_845_845_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:08.754612 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12656 2024.10.22 02:12:08.801081 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 133253c8c4f70bd33d6ede93b87b42dc, offset: 0, size: 0 2024.10.22 02:12:08.804941 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 133253c8c4f70bd33d6ede93b87b42dc for removal 2024.10.22 02:12:08.809602 [ 1476 ] {} CacheMetadata: Key 133253c8c4f70bd33d6ede93b87b42dc is removed from metadata 2024.10.22 02:12:08.845354 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_840_213 to 202410_841_841_0 2024.10.22 02:12:08.852338 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:08.852412 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 2.00 GiB, free memory in arenas 8.21 MiB, will set to 1.57 GiB (RSS), difference: -180.32 MiB 2024.10.22 02:12:08.854524 [ 1220 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:08.955750 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e6714b10814d7a014fa5752216aa3e7c, offset: 0, size: 0 2024.10.22 02:12:08.958465 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e6714b10814d7a014fa5752216aa3e7c for removal 2024.10.22 02:12:08.960760 [ 1476 ] {} CacheMetadata: Key e6714b10814d7a014fa5752216aa3e7c is removed from metadata 2024.10.22 02:12:08.995568 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_841_214} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_840_213 to 202410_841_841_0 into Compact with storage Full 2024.10.22 02:12:09.026114 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xid/zntugubdeidjhueqzmntmopyvromd (LogSeriesLimiter: on interval from 2024-10-22 02:12:04 to 2024-10-22 02:12:09 accepted series 1 / 185 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:12:09.036471 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_841_214} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:09.049634 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_841_214} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_840_213, total 12695 rows starting from the beginning of the part 2024.10.22 02:12:09.066423 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_841_214} MergeTreeSequentialSource: Reading 2 marks from part 202410_841_841_0, total 17 rows starting from the beginning of the part 2024.10.22 02:12:09.358445 [ 1168 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 202 ms. 2024.10.22 02:12:09.668703 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_165_169_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_165_165_0, total 43 rows starting from the beginning of the part 2024.10.22 02:12:09.729080 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 741641 bytes for 133253c8c4f70bd33d6ede93b87b42dc:0) 2024.10.22 02:12:09.733675 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 291a25324d90f375b31aef04636b4024, offset: 0, size: 883 2024.10.22 02:12:09.740260 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 291a25324d90f375b31aef04636b4024 for removal 2024.10.22 02:12:09.744362 [ 1476 ] {} CacheMetadata: Key 291a25324d90f375b31aef04636b4024 is removed from metadata 2024.10.22 02:12:09.745191 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9a1e86da7b9033c82765e42fe439d035, offset: 0, size: 163 2024.10.22 02:12:09.750880 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9a1e86da7b9033c82765e42fe439d035 for removal 2024.10.22 02:12:09.755561 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4956f9262aa73d7970728c0a63fe3bbd, offset: 0, size: 113708 2024.10.22 02:12:09.759745 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4956f9262aa73d7970728c0a63fe3bbd for removal 2024.10.22 02:12:09.764055 [ 1476 ] {} CacheMetadata: Key 4956f9262aa73d7970728c0a63fe3bbd is removed from metadata 2024.10.22 02:12:09.771496 [ 1476 ] {} CacheMetadata: Key 9a1e86da7b9033c82765e42fe439d035 is removed from metadata 2024.10.22 02:12:09.765352 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3e0562db4044ab25f02c02279a778d7e, offset: 0, size: 251187 2024.10.22 02:12:09.802635 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3e0562db4044ab25f02c02279a778d7e for removal 2024.10.22 02:12:09.817026 [ 1476 ] {} CacheMetadata: Key 3e0562db4044ab25f02c02279a778d7e is removed from metadata 2024.10.22 02:12:09.818646 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a27c1ccd84115eb57904d0a80378d88e, offset: 0, size: 2608541 2024.10.22 02:12:09.828642 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a27c1ccd84115eb57904d0a80378d88e for removal 2024.10.22 02:12:09.833087 [ 1476 ] {} CacheMetadata: Key a27c1ccd84115eb57904d0a80378d88e is removed from metadata 2024.10.22 02:12:09.832877 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7afbbf2316a88407deea8ef4b6811aa7, offset: 0, size: 121685 2024.10.22 02:12:09.858885 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7afbbf2316a88407deea8ef4b6811aa7 for removal 2024.10.22 02:12:09.863200 [ 1476 ] {} CacheMetadata: Key 7afbbf2316a88407deea8ef4b6811aa7 is removed from metadata 2024.10.22 02:12:09.866856 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9e0385106aa8e866527b2db63a6896b6, offset: 0, size: 162 2024.10.22 02:12:09.872530 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9e0385106aa8e866527b2db63a6896b6 for removal 2024.10.22 02:12:10.022972 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 221 entries to flush up to offset 92426 2024.10.22 02:12:10.026432 [ 1476 ] {} CacheMetadata: Key 9e0385106aa8e866527b2db63a6896b6 is removed from metadata 2024.10.22 02:12:10.024525 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: da1beedf7b6b1c41bd48bb77ee7ce4a7, offset: 0, size: 195 2024.10.22 02:12:10.046961 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key da1beedf7b6b1c41bd48bb77ee7ce4a7 for removal 2024.10.22 02:12:10.049656 [ 1476 ] {} CacheMetadata: Key da1beedf7b6b1c41bd48bb77ee7ce4a7 is removed from metadata 2024.10.22 02:12:10.084113 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_841_214} MergeTask::MergeProjectionsStage: Merge sorted 12712 rows, containing 12 columns (12 merged, 0 gathered) in 1.11428111 sec., 11408.252267688536 rows/sec., 1.78 MiB/sec. 2024.10.22 02:12:10.268403 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/skr/ckusoqxpeuinsxyklmenxxqoeamdi. 2024.10.22 02:12:10.302738 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_841_214} MergedBlockOutputStream: filled checksums 202410_1_841_214 (state Temporary) 2024.10.22 02:12:10.402977 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_841_214} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_841_214 to 202410_1_841_214 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:10.413406 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_841_214} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_840_213, 202410_841_841_0] -> 202410_1_841_214 2024.10.22 02:12:10.423264 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a0d24a49019615b2230383543e1b0ea2, offset: 0, size: 0 2024.10.22 02:12:10.435177 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a0d24a49019615b2230383543e1b0ea2 for removal 2024.10.22 02:12:10.445175 [ 1476 ] {} CacheMetadata: Key a0d24a49019615b2230383543e1b0ea2 is removed from metadata 2024.10.22 02:12:10.465759 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.45 MiB. 2024.10.22 02:12:10.531181 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} MergeTask::MergeProjectionsStage: Merge sorted 727 rows, containing 32 columns (32 merged, 0 gathered) in 3.479822131 sec., 208.91872418521612 rows/sec., 18.89 KiB/sec. 2024.10.22 02:12:10.553703 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} MergedBlockOutputStream: filled checksums 202410_1_683_139 (state Temporary) 2024.10.22 02:12:10.727717 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4d9e3853cff0ec8694bfc942315491d8, offset: 0, size: 0 2024.10.22 02:12:10.733000 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4d9e3853cff0ec8694bfc942315491d8 for removal 2024.10.22 02:12:10.739845 [ 1476 ] {} CacheMetadata: Key 4d9e3853cff0ec8694bfc942315491d8 is removed from metadata 2024.10.22 02:12:10.743938 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/skr/ckusoqxpeuinsxyklmenxxqoeamdi. 2024.10.22 02:12:10.831858 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 02:12:10.847743 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_683_139 to 202410_1_683_139 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:10.934819 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 8.47 MiB, will set to 1.58 GiB (RSS), difference: -25.84 MiB 2024.10.22 02:12:10.951764 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_683_139} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_678_138, 202410_683_683_0] -> 202410_1_683_139 2024.10.22 02:12:11.097775 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:12:11.260756 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:12:11.264074 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:12:11.284147 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:11.296832 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:11.365412 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 643102 bytes for a0d24a49019615b2230383543e1b0ea2:0) 2024.10.22 02:12:11.377456 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a27314c515e1964e9d82d0dbdb7e807f, offset: 0, size: 248811 2024.10.22 02:12:11.386991 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a27314c515e1964e9d82d0dbdb7e807f for removal 2024.10.22 02:12:11.391039 [ 1476 ] {} CacheMetadata: Key a27314c515e1964e9d82d0dbdb7e807f is removed from metadata 2024.10.22 02:12:11.392638 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e120da532b7118f51b438e302878a506, offset: 0, size: 195 2024.10.22 02:12:11.398597 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e120da532b7118f51b438e302878a506 for removal 2024.10.22 02:12:11.402650 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d351d1e1011ceab5ee1f039496b7637e, offset: 0, size: 70007 2024.10.22 02:12:11.411526 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d351d1e1011ceab5ee1f039496b7637e for removal 2024.10.22 02:12:11.417392 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8566df602c8830fe1c46bd8a76f0ced9, offset: 0, size: 162 2024.10.22 02:12:11.418795 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1403_1403_0 (state Temporary) 2024.10.22 02:12:11.424318 [ 1476 ] {} CacheMetadata: Key d351d1e1011ceab5ee1f039496b7637e is removed from metadata 2024.10.22 02:12:11.428478 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8566df602c8830fe1c46bd8a76f0ced9 for removal 2024.10.22 02:12:11.483531 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_167_167_0 (state Temporary) 2024.10.22 02:12:11.538621 [ 1476 ] {} CacheMetadata: Key 8566df602c8830fe1c46bd8a76f0ced9 is removed from metadata 2024.10.22 02:12:11.561841 [ 1476 ] {} CacheMetadata: Key e120da532b7118f51b438e302878a506 is removed from metadata 2024.10.22 02:12:11.631708 [ 1249 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 02:12:11.745097 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 96 entries to flush up to offset 90987 2024.10.22 02:12:11.850083 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1403_1403_0 to 202410_1416_1416_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:11.871300 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1411_41 to 202410_1416_1416_0 2024.10.22 02:12:11.889064 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 92426 2024.10.22 02:12:11.897567 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:11.902323 [ 1200 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:11.928968 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1411_41 to 202410_1416_1416_0 into Compact with storage Full 2024.10.22 02:12:11.991434 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:11.996243 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:12.059242 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:12.065277 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_818_818_0 (state Temporary) 2024.10.22 02:12:12.069312 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1411_41, total 12644 rows starting from the beginning of the part 2024.10.22 02:12:12.080237 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lqk/irqfsnnxhmyjkmlvspoioxdnhharh. 2024.10.22 02:12:12.089401 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_1412_1412_0, total 62 rows starting from the beginning of the part 2024.10.22 02:12:12.145553 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a8e890157c02a05debd7322801c7cb98, offset: 0, size: 0 2024.10.22 02:12:12.150874 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a8e890157c02a05debd7322801c7cb98 for removal 2024.10.22 02:12:12.155080 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_1413_1413_0, total 110 rows starting from the beginning of the part 2024.10.22 02:12:12.155134 [ 1476 ] {} CacheMetadata: Key a8e890157c02a05debd7322801c7cb98 is removed from metadata 2024.10.22 02:12:12.197053 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 298a42f594a5185987db0b6d4b168db2, offset: 0, size: 0 2024.10.22 02:12:12.200135 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 298a42f594a5185987db0b6d4b168db2 for removal 2024.10.22 02:12:12.203030 [ 1476 ] {} CacheMetadata: Key 298a42f594a5185987db0b6d4b168db2 is removed from metadata 2024.10.22 02:12:12.201804 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_1414_1414_0, total 117 rows starting from the beginning of the part 2024.10.22 02:12:12.213349 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lqk/irqfsnnxhmyjkmlvspoioxdnhharh. 2024.10.22 02:12:12.218853 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_818_818_0 to 202410_824_824_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:12.245660 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_1415_1415_0, total 131 rows starting from the beginning of the part 2024.10.22 02:12:12.269187 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 90987 2024.10.22 02:12:12.270808 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_165_169_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_166_166_0, total 42 rows starting from the beginning of the part 2024.10.22 02:12:12.279925 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_1416_1416_0, total 221 rows starting from the beginning of the part 2024.10.22 02:12:12.854568 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 630290 bytes for a8e890157c02a05debd7322801c7cb98:0) 2024.10.22 02:12:12.856654 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 68d0aeb65cd309474589284dc046b8a6, offset: 0, size: 859 2024.10.22 02:12:12.862327 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 68d0aeb65cd309474589284dc046b8a6 for removal 2024.10.22 02:12:12.877201 [ 1476 ] {} CacheMetadata: Key 68d0aeb65cd309474589284dc046b8a6 is removed from metadata 2024.10.22 02:12:12.879406 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 20d9ebcd19cf85380021461fdb43be3c, offset: 0, size: 1684746 2024.10.22 02:12:12.884193 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 20d9ebcd19cf85380021461fdb43be3c for removal 2024.10.22 02:12:12.886910 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c8f755e25ac663b908783b0b5eba0db7, offset: 0, size: 4680 2024.10.22 02:12:12.892111 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c8f755e25ac663b908783b0b5eba0db7 for removal 2024.10.22 02:12:12.895887 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e9c9f656f6ae94572439fb5c46a3da7c, offset: 0, size: 163 2024.10.22 02:12:12.899107 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e9c9f656f6ae94572439fb5c46a3da7c for removal 2024.10.22 02:12:12.922442 [ 1476 ] {} CacheMetadata: Key e9c9f656f6ae94572439fb5c46a3da7c is removed from metadata 2024.10.22 02:12:12.941447 [ 1476 ] {} CacheMetadata: Key c8f755e25ac663b908783b0b5eba0db7 is removed from metadata 2024.10.22 02:12:12.958633 [ 1476 ] {} CacheMetadata: Key 20d9ebcd19cf85380021461fdb43be3c is removed from metadata 2024.10.22 02:12:13.211054 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hex/hxnyouztnodpkeguqwzarkafmjqnm. 2024.10.22 02:12:13.266934 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6fa586a05d27218446720af217114c80, offset: 0, size: 0 2024.10.22 02:12:13.268938 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6fa586a05d27218446720af217114c80 for removal 2024.10.22 02:12:13.272073 [ 1476 ] {} CacheMetadata: Key 6fa586a05d27218446720af217114c80 is removed from metadata 2024.10.22 02:12:13.325533 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 09ec90228ce860fd44b144993c35ee34, offset: 0, size: 0 2024.10.22 02:12:13.327747 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 09ec90228ce860fd44b144993c35ee34 for removal 2024.10.22 02:12:13.331080 [ 1476 ] {} CacheMetadata: Key 09ec90228ce860fd44b144993c35ee34 is removed from metadata 2024.10.22 02:12:13.335156 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hex/hxnyouztnodpkeguqwzarkafmjqnm. 2024.10.22 02:12:13.575534 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 11.68 MiB, will set to 1.59 GiB (RSS), difference: -2.56 MiB 2024.10.22 02:12:13.635849 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/phz/ovcoadcsgpvxuwfltgycydoodzbrl. 2024.10.22 02:12:13.753424 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1200530 2024.10.22 02:12:13.782604 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ad493f53f03973cfca0a94a55292cf4b, offset: 0, size: 0 2024.10.22 02:12:13.785117 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ad493f53f03973cfca0a94a55292cf4b for removal 2024.10.22 02:12:13.789711 [ 1476 ] {} CacheMetadata: Key ad493f53f03973cfca0a94a55292cf4b is removed from metadata 2024.10.22 02:12:13.942651 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 91d93e9ff3afcd8b1edf2fce819ad138, offset: 0, size: 0 2024.10.22 02:12:13.946596 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 91d93e9ff3afcd8b1edf2fce819ad138 for removal 2024.10.22 02:12:13.950895 [ 1476 ] {} CacheMetadata: Key 91d93e9ff3afcd8b1edf2fce819ad138 is removed from metadata 2024.10.22 02:12:14.058144 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/phz/ovcoadcsgpvxuwfltgycydoodzbrl. 2024.10.22 02:12:14.266446 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} MergeTask::MergeProjectionsStage: Merge sorted 13285 rows, containing 13 columns (13 merged, 0 gathered) in 2.347587656 sec., 5659.00061965567 rows/sec., 1.14 MiB/sec. 2024.10.22 02:12:14.451254 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} MergedBlockOutputStream: filled checksums 202410_1207_1416_42 (state Temporary) 2024.10.22 02:12:14.568197 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 692 2024.10.22 02:12:14.571755 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:14.575078 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:14.716242 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1416_42 to 202410_1207_1416_42 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:14.728145 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 372376 bytes for ad493f53f03973cfca0a94a55292cf4b:0) 2024.10.22 02:12:14.732253 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1416_42} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1411_41, 202410_1416_1416_0] -> 202410_1207_1416_42 2024.10.22 02:12:14.742196 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b186bcfb09112ce7b7c90df3052a5b9a, offset: 0, size: 41938 2024.10.22 02:12:14.745167 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b186bcfb09112ce7b7c90df3052a5b9a for removal 2024.10.22 02:12:14.747289 [ 1476 ] {} CacheMetadata: Key b186bcfb09112ce7b7c90df3052a5b9a is removed from metadata 2024.10.22 02:12:14.748986 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ad02d6e84669abf5d034f5eeadb01563, offset: 0, size: 55127 2024.10.22 02:12:14.752948 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ad02d6e84669abf5d034f5eeadb01563 for removal 2024.10.22 02:12:14.755051 [ 1476 ] {} CacheMetadata: Key ad02d6e84669abf5d034f5eeadb01563 is removed from metadata 2024.10.22 02:12:14.755763 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b31534f28e5fe760c29c44dc16f061b3, offset: 0, size: 161 2024.10.22 02:12:14.758173 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b31534f28e5fe760c29c44dc16f061b3 for removal 2024.10.22 02:12:14.760117 [ 1476 ] {} CacheMetadata: Key b31534f28e5fe760c29c44dc16f061b3 is removed from metadata 2024.10.22 02:12:14.760664 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f1bb91832da637d32965ad32324e8ebc, offset: 0, size: 5062 2024.10.22 02:12:14.773723 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f1bb91832da637d32965ad32324e8ebc for removal 2024.10.22 02:12:14.775792 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.94 MiB. 2024.10.22 02:12:14.782388 [ 1476 ] {} CacheMetadata: Key f1bb91832da637d32965ad32324e8ebc is removed from metadata 2024.10.22 02:12:14.783048 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b11e7e384c53c79b147f78e8ebf9f67d, offset: 0, size: 483 2024.10.22 02:12:14.792164 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b11e7e384c53c79b147f78e8ebf9f67d for removal 2024.10.22 02:12:14.798520 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 54999f4089ddbc2dd3fcef393a10740f, offset: 0, size: 162 2024.10.22 02:12:14.807351 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 54999f4089ddbc2dd3fcef393a10740f for removal 2024.10.22 02:12:14.807417 [ 1476 ] {} CacheMetadata: Key b11e7e384c53c79b147f78e8ebf9f67d is removed from metadata 2024.10.22 02:12:14.817045 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 74efcf9ed0b07bd481ee8ac19f7dc7d1, offset: 0, size: 163 2024.10.22 02:12:14.858579 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 74efcf9ed0b07bd481ee8ac19f7dc7d1 for removal 2024.10.22 02:12:14.861279 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_796_150/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:12:14.863919 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ab69463017db48cdff90770166e5f4fd, offset: 0, size: 59294 2024.10.22 02:12:14.870532 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ab69463017db48cdff90770166e5f4fd for removal 2024.10.22 02:12:14.909017 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:14.914517 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:14.938402 [ 1476 ] {} CacheMetadata: Key ab69463017db48cdff90770166e5f4fd is removed from metadata 2024.10.22 02:12:14.943490 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_165_169_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_167_167_0, total 46 rows starting from the beginning of the part 2024.10.22 02:12:14.950744 [ 1476 ] {} CacheMetadata: Key 74efcf9ed0b07bd481ee8ac19f7dc7d1 is removed from metadata 2024.10.22 02:12:14.954975 [ 1476 ] {} CacheMetadata: Key 54999f4089ddbc2dd3fcef393a10740f is removed from metadata 2024.10.22 02:12:14.986360 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1312_21, 202410_1313_1313_0, 202410_1314_1314_0, 202410_1315_1315_0, 202410_1316_1316_0, 202410_1317_1317_0] 2024.10.22 02:12:14.993192 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1312_21, 202410_1313_1313_0, 202410_1314_1314_0, 202410_1315_1315_0, 202410_1316_1316_0, 202410_1317_1317_0] 2024.10.22 02:12:15.025340 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1312_21, 202410_1313_1313_0, 202410_1314_1314_0, 202410_1315_1315_0, 202410_1316_1316_0, 202410_1317_1317_0] 2024.10.22 02:12:15.027828 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_797_797_0 (state Temporary) 2024.10.22 02:12:15.132096 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_797_797_0 to 202410_804_804_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:15.163496 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cd26f1f1074677337b66efa14609acc5, offset: 0, size: 0 2024.10.22 02:12:15.170986 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cd26f1f1074677337b66efa14609acc5 for removal 2024.10.22 02:12:15.173004 [ 1476 ] {} CacheMetadata: Key cd26f1f1074677337b66efa14609acc5 is removed from metadata 2024.10.22 02:12:15.184943 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_674_674_0 (state Temporary) 2024.10.22 02:12:15.205600 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1200530 2024.10.22 02:12:15.213631 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 16d38671302dab334df5f7e1e38feee1, offset: 0, size: 0 2024.10.22 02:12:15.216035 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 16d38671302dab334df5f7e1e38feee1 for removal 2024.10.22 02:12:15.223711 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/cxe/qduxgyizhzxczagjupisarmugvtsb. 2024.10.22 02:12:15.232668 [ 1476 ] {} CacheMetadata: Key 16d38671302dab334df5f7e1e38feee1 is removed from metadata 2024.10.22 02:12:15.275725 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_674_674_0 to 202410_684_684_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:15.341338 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 692 2024.10.22 02:12:15.371373 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f6703dbac530b46ede917a3d4e1710e3, offset: 0, size: 0 2024.10.22 02:12:15.373072 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f6703dbac530b46ede917a3d4e1710e3 for removal 2024.10.22 02:12:15.375840 [ 1476 ] {} CacheMetadata: Key f6703dbac530b46ede917a3d4e1710e3 is removed from metadata 2024.10.22 02:12:15.436799 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 21a1c76d17142ad64d69431790da91c0, offset: 0, size: 0 2024.10.22 02:12:15.439780 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 21a1c76d17142ad64d69431790da91c0 for removal 2024.10.22 02:12:15.444048 [ 1476 ] {} CacheMetadata: Key 21a1c76d17142ad64d69431790da91c0 is removed from metadata 2024.10.22 02:12:15.451536 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/cxe/qduxgyizhzxczagjupisarmugvtsb. 2024.10.22 02:12:15.454312 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/enu/nbpiffqmmvqdtejjopxkgmeuncnrt. 2024.10.22 02:12:15.504905 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 223815c382958e5b1d949e11249b7c7e, offset: 0, size: 0 2024.10.22 02:12:15.510808 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 223815c382958e5b1d949e11249b7c7e for removal 2024.10.22 02:12:15.518457 [ 1476 ] {} CacheMetadata: Key 223815c382958e5b1d949e11249b7c7e is removed from metadata 2024.10.22 02:12:15.577049 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c3d1d86c9d7b4f29e0804fe4964abda1, offset: 0, size: 0 2024.10.22 02:12:15.589748 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c3d1d86c9d7b4f29e0804fe4964abda1 for removal 2024.10.22 02:12:15.596848 [ 1476 ] {} CacheMetadata: Key c3d1d86c9d7b4f29e0804fe4964abda1 is removed from metadata 2024.10.22 02:12:15.619327 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/enu/nbpiffqmmvqdtejjopxkgmeuncnrt. 2024.10.22 02:12:15.764122 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 10.55 MiB, will set to 1.60 GiB (RSS), difference: 17.86 MiB 2024.10.22 02:12:15.928695 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 174 entries to flush up to offset 92600 2024.10.22 02:12:16.255712 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 12678 2024.10.22 02:12:16.329790 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 7 file segments (while reserving 1048576 bytes for 223815c382958e5b1d949e11249b7c7e:0) 2024.10.22 02:12:16.333146 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5034bc0c18f16ff332a291234f5a662a, offset: 0, size: 673748 2024.10.22 02:12:16.345448 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5034bc0c18f16ff332a291234f5a662a for removal 2024.10.22 02:12:16.348577 [ 1476 ] {} CacheMetadata: Key 5034bc0c18f16ff332a291234f5a662a is removed from metadata 2024.10.22 02:12:16.348837 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 395f7274852379b69d2dc73bcee915e2, offset: 0, size: 503 2024.10.22 02:12:16.353377 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 395f7274852379b69d2dc73bcee915e2 for removal 2024.10.22 02:12:16.358332 [ 1476 ] {} CacheMetadata: Key 395f7274852379b69d2dc73bcee915e2 is removed from metadata 2024.10.22 02:12:16.359585 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7b3fd0ef47028e6ca0309994f1a295d2, offset: 0, size: 268328 2024.10.22 02:12:16.362800 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7b3fd0ef47028e6ca0309994f1a295d2 for removal 2024.10.22 02:12:16.366995 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: bcab6a94c03115afa37d83e196359d9c, offset: 0, size: 171 2024.10.22 02:12:16.370392 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key bcab6a94c03115afa37d83e196359d9c for removal 2024.10.22 02:12:16.380214 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 148ddfaa60b3bbaae4f982333bc84c6b, offset: 0, size: 192 2024.10.22 02:12:16.389738 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 148ddfaa60b3bbaae4f982333bc84c6b for removal 2024.10.22 02:12:16.402564 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 75f5af682c1daa54ac4d8f49eadfb883, offset: 0, size: 428704 2024.10.22 02:12:16.411403 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 75f5af682c1daa54ac4d8f49eadfb883 for removal 2024.10.22 02:12:16.519361 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a2f2ffc1f8511e77fca8a2d58be51c12, offset: 0, size: 237 2024.10.22 02:12:16.520401 [ 1476 ] {} CacheMetadata: Key 75f5af682c1daa54ac4d8f49eadfb883 is removed from metadata 2024.10.22 02:12:16.531083 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a2f2ffc1f8511e77fca8a2d58be51c12 for removal 2024.10.22 02:12:16.536408 [ 1476 ] {} CacheMetadata: Key 148ddfaa60b3bbaae4f982333bc84c6b is removed from metadata 2024.10.22 02:12:16.651765 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:16.653975 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:16.655026 [ 1476 ] {} CacheMetadata: Key bcab6a94c03115afa37d83e196359d9c is removed from metadata 2024.10.22 02:12:16.667732 [ 1476 ] {} CacheMetadata: Key a2f2ffc1f8511e77fca8a2d58be51c12 is removed from metadata 2024.10.22 02:12:16.677010 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:16.683223 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:16.694678 [ 1476 ] {} CacheMetadata: Key 7b3fd0ef47028e6ca0309994f1a295d2 is removed from metadata 2024.10.22 02:12:16.775932 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_165_169_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_168_168_0, total 44 rows starting from the beginning of the part 2024.10.22 02:12:16.922587 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_833_833_0 (state Temporary) 2024.10.22 02:12:17.005387 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_833_833_0 to 202410_846_846_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:17.043424 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1404_1404_0 (state Temporary) 2024.10.22 02:12:17.077100 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1404_1404_0 to 202410_1417_1417_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:17.100160 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12678 2024.10.22 02:12:17.141701 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 92600 2024.10.22 02:12:17.173002 [ 1276 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_841_214 to 202410_842_842_0 2024.10.22 02:12:17.181263 [ 1276 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:17.192615 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:17.329502 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_842_215} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_841_214 to 202410_842_842_0 into Compact with storage Full 2024.10.22 02:12:17.360626 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_842_215} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:17.373191 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_842_215} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_841_214, total 12712 rows starting from the beginning of the part 2024.10.22 02:12:17.404463 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_842_215} MergeTreeSequentialSource: Reading 2 marks from part 202410_842_842_0, total 7 rows starting from the beginning of the part 2024.10.22 02:12:17.447133 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 1048576 bytes for 223815c382958e5b1d949e11249b7c7e:0) 2024.10.22 02:12:17.450374 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b458f0b5b53a68c94d5593e0767166f3, offset: 0, size: 198 2024.10.22 02:12:17.452433 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b458f0b5b53a68c94d5593e0767166f3 for removal 2024.10.22 02:12:17.454150 [ 1476 ] {} CacheMetadata: Key b458f0b5b53a68c94d5593e0767166f3 is removed from metadata 2024.10.22 02:12:17.454379 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6447f9f9373b49e7e3e2cc603eddd396, offset: 0, size: 190450 2024.10.22 02:12:17.457313 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6447f9f9373b49e7e3e2cc603eddd396 for removal 2024.10.22 02:12:17.461362 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d5cbb8dc3512d690af71098b2f53e9c1, offset: 0, size: 169 2024.10.22 02:12:17.463501 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d5cbb8dc3512d690af71098b2f53e9c1 for removal 2024.10.22 02:12:17.467376 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e42b3bb9cac8d0bb3f3603a5c54f8274, offset: 0, size: 375313 2024.10.22 02:12:17.470917 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e42b3bb9cac8d0bb3f3603a5c54f8274 for removal 2024.10.22 02:12:17.471115 [ 1476 ] {} CacheMetadata: Key d5cbb8dc3512d690af71098b2f53e9c1 is removed from metadata 2024.10.22 02:12:17.473369 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 704b36288f7c196870b83d61e0069952, offset: 0, size: 151173 2024.10.22 02:12:17.477046 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 704b36288f7c196870b83d61e0069952 for removal 2024.10.22 02:12:17.479806 [ 1476 ] {} CacheMetadata: Key e42b3bb9cac8d0bb3f3603a5c54f8274 is removed from metadata 2024.10.22 02:12:17.495815 [ 1476 ] {} CacheMetadata: Key 704b36288f7c196870b83d61e0069952 is removed from metadata 2024.10.22 02:12:17.511123 [ 1476 ] {} CacheMetadata: Key 6447f9f9373b49e7e3e2cc603eddd396 is removed from metadata 2024.10.22 02:12:17.524812 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_167_167_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:12:17.754496 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 6.68 MiB, will set to 1.61 GiB (RSS), difference: 2.39 MiB 2024.10.22 02:12:17.871285 [ 1242 ] {} DatabaseCatalog: Execution took 340 ms. 2024.10.22 02:12:18.202662 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 455611 bytes for 223815c382958e5b1d949e11249b7c7e:0) 2024.10.22 02:12:18.206775 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 396c90e5847a24278471c10649506ae4, offset: 0, size: 738189 2024.10.22 02:12:18.213593 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 396c90e5847a24278471c10649506ae4 for removal 2024.10.22 02:12:18.219049 [ 1476 ] {} CacheMetadata: Key 396c90e5847a24278471c10649506ae4 is removed from metadata 2024.10.22 02:12:18.220466 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 22277ec29f877abf9babd17f8e8978a8, offset: 0, size: 377121 2024.10.22 02:12:18.225294 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 22277ec29f877abf9babd17f8e8978a8 for removal 2024.10.22 02:12:18.230892 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7ece380b5ec6478b2dbd5ca902d2f521, offset: 0, size: 498 2024.10.22 02:12:18.236228 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7ece380b5ec6478b2dbd5ca902d2f521 for removal 2024.10.22 02:12:18.248430 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 84e5577bf8e9a06a0751e9c854fa14fe, offset: 0, size: 196 2024.10.22 02:12:18.249091 [ 1476 ] {} CacheMetadata: Key 7ece380b5ec6478b2dbd5ca902d2f521 is removed from metadata 2024.10.22 02:12:18.253090 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 84e5577bf8e9a06a0751e9c854fa14fe for removal 2024.10.22 02:12:18.258147 [ 1476 ] {} CacheMetadata: Key 84e5577bf8e9a06a0751e9c854fa14fe is removed from metadata 2024.10.22 02:12:18.279238 [ 1476 ] {} CacheMetadata: Key 22277ec29f877abf9babd17f8e8978a8 is removed from metadata 2024.10.22 02:12:18.306934 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_842_215} MergeTask::MergeProjectionsStage: Merge sorted 12719 rows, containing 12 columns (12 merged, 0 gathered) in 0.983248462 sec., 12935.692748635085 rows/sec., 2.02 MiB/sec. 2024.10.22 02:12:18.308063 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/upj/zqcpcauejapnsgxsumxjtnvwffmdf. 2024.10.22 02:12:18.367803 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 01739eca089b35f941ba773d6a3813ba, offset: 0, size: 0 2024.10.22 02:12:18.370531 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 01739eca089b35f941ba773d6a3813ba for removal 2024.10.22 02:12:18.409421 [ 1476 ] {} CacheMetadata: Key 01739eca089b35f941ba773d6a3813ba is removed from metadata 2024.10.22 02:12:18.419404 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 432d69fe66b2400213f40995bbf1f216, offset: 0, size: 0 2024.10.22 02:12:18.421817 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 432d69fe66b2400213f40995bbf1f216 for removal 2024.10.22 02:12:18.429108 [ 1476 ] {} CacheMetadata: Key 432d69fe66b2400213f40995bbf1f216 is removed from metadata 2024.10.22 02:12:18.470786 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_842_215} MergedBlockOutputStream: filled checksums 202410_1_842_215 (state Temporary) 2024.10.22 02:12:18.511002 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_167_167_0 to 202410_170_170_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:18.645132 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/upj/zqcpcauejapnsgxsumxjtnvwffmdf. 2024.10.22 02:12:18.649602 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pyx/kwazpxhzldfyzhasatgjhiavmuoym. 2024.10.22 02:12:18.684871 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 48e2b9565d473cc0cc4b01baaf124c6c, offset: 0, size: 0 2024.10.22 02:12:18.699899 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 48e2b9565d473cc0cc4b01baaf124c6c for removal 2024.10.22 02:12:18.702271 [ 1476 ] {} CacheMetadata: Key 48e2b9565d473cc0cc4b01baaf124c6c is removed from metadata 2024.10.22 02:12:18.885102 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_842_215} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_842_215 to 202410_1_842_215 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:18.896835 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_842_215} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_841_214, 202410_842_842_0] -> 202410_1_842_215 2024.10.22 02:12:18.929019 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a0ffb58271b2eba1c12b2e193d3c857f, offset: 0, size: 0 2024.10.22 02:12:18.931155 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a0ffb58271b2eba1c12b2e193d3c857f for removal 2024.10.22 02:12:18.933362 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.45 MiB. 2024.10.22 02:12:18.935618 [ 1476 ] {} CacheMetadata: Key a0ffb58271b2eba1c12b2e193d3c857f is removed from metadata 2024.10.22 02:12:18.942791 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pyx/kwazpxhzldfyzhasatgjhiavmuoym. 2024.10.22 02:12:19.000058 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_165_169_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_169_169_0, total 43 rows starting from the beginning of the part 2024.10.22 02:12:19.638362 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_743_743_0, 202410_744_744_0, 202410_745_745_0, 202410_746_746_0, 202410_747_747_0, 202410_748_748_0] 2024.10.22 02:12:19.640859 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_743_743_0, 202410_744_744_0, 202410_745_745_0, 202410_746_746_0, 202410_747_747_0, 202410_748_748_0] 2024.10.22 02:12:19.764488 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_743_743_0, 202410_744_744_0, 202410_745_745_0, 202410_746_746_0, 202410_747_747_0, 202410_748_748_0] 2024.10.22 02:12:19.792325 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 99 entries to flush up to offset 91086 2024.10.22 02:12:19.943722 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 48e2b9565d473cc0cc4b01baaf124c6c:0) 2024.10.22 02:12:19.950629 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4f0338e3059f4a1adc8f552f946888e4, offset: 0, size: 2177825 2024.10.22 02:12:19.954269 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4f0338e3059f4a1adc8f552f946888e4 for removal 2024.10.22 02:12:19.960355 [ 1476 ] {} CacheMetadata: Key 4f0338e3059f4a1adc8f552f946888e4 is removed from metadata 2024.10.22 02:12:19.962623 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ddaecfd342c0fe6c19accc5b3d4c3748, offset: 0, size: 264 2024.10.22 02:12:19.966490 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ddaecfd342c0fe6c19accc5b3d4c3748 for removal 2024.10.22 02:12:19.969411 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 08e8375e8b396cc0c736a2b11ff08ff7, offset: 0, size: 176821 2024.10.22 02:12:19.973364 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 08e8375e8b396cc0c736a2b11ff08ff7 for removal 2024.10.22 02:12:19.973811 [ 1476 ] {} CacheMetadata: Key ddaecfd342c0fe6c19accc5b3d4c3748 is removed from metadata 2024.10.22 02:12:19.975748 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a33514fc98041be5757a851c850f48f3, offset: 0, size: 194 2024.10.22 02:12:19.978112 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a33514fc98041be5757a851c850f48f3 for removal 2024.10.22 02:12:19.980013 [ 1476 ] {} CacheMetadata: Key 08e8375e8b396cc0c736a2b11ff08ff7 is removed from metadata 2024.10.22 02:12:20.007226 [ 1476 ] {} CacheMetadata: Key a33514fc98041be5757a851c850f48f3 is removed from metadata 2024.10.22 02:12:20.064173 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:20.081989 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.84 GiB. 2024.10.22 02:12:20.355587 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_819_819_0 (state Temporary) 2024.10.22 02:12:20.410939 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_819_819_0 to 202410_825_825_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:20.493323 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 91086 2024.10.22 02:12:20.615208 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 4.78 MiB, will set to 1.61 GiB (RSS), difference: 17.31 MiB 2024.10.22 02:12:20.629284 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 14 file segments (while reserving 4099308 bytes for 24c2dab7d4d02b4f182eed2dcc18bdb1:0) 2024.10.22 02:12:20.634790 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: e095c5f94e37781635ccdccf2d5b5f73, offset: 0, size: 1984714 2024.10.22 02:12:20.643468 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key e095c5f94e37781635ccdccf2d5b5f73 for removal 2024.10.22 02:12:20.647593 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 622f04b76ae30acd58ca6e79be1f5ba1, offset: 0, size: 629560 2024.10.22 02:12:20.647593 [ 1476 ] {} CacheMetadata: Key e095c5f94e37781635ccdccf2d5b5f73 is removed from metadata 2024.10.22 02:12:20.654798 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 622f04b76ae30acd58ca6e79be1f5ba1 for removal 2024.10.22 02:12:20.658581 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 584b8ebf0834be2434cc86326c92ebcd, offset: 0, size: 163 2024.10.22 02:12:20.658431 [ 1476 ] {} CacheMetadata: Key 622f04b76ae30acd58ca6e79be1f5ba1 is removed from metadata 2024.10.22 02:12:20.662157 [ 1204 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:12:20.664117 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 584b8ebf0834be2434cc86326c92ebcd for removal 2024.10.22 02:12:20.667023 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9b4fb64c5a01666c85c028b8f724de00, offset: 0, size: 185137 2024.10.22 02:12:20.670369 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 9b4fb64c5a01666c85c028b8f724de00 for removal 2024.10.22 02:12:20.670692 [ 1476 ] {} CacheMetadata: Key 584b8ebf0834be2434cc86326c92ebcd is removed from metadata 2024.10.22 02:12:20.675871 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 63b73f52a5a759d072137697c4b3a8f0, offset: 0, size: 278500 2024.10.22 02:12:20.681139 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 63b73f52a5a759d072137697c4b3a8f0 for removal 2024.10.22 02:12:20.688401 [ 1476 ] {} CacheMetadata: Key 9b4fb64c5a01666c85c028b8f724de00 is removed from metadata 2024.10.22 02:12:20.693757 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d89663e39ed984095909e4313b96f904, offset: 0, size: 237 2024.10.22 02:12:20.699508 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d89663e39ed984095909e4313b96f904 for removal 2024.10.22 02:12:20.707707 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9d71d7184b77417f941716a9c24fd94d, offset: 0, size: 187006 2024.10.22 02:12:20.716200 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 9d71d7184b77417f941716a9c24fd94d for removal 2024.10.22 02:12:20.719286 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c0d788f3137f82e88ed9a19ac834b17c, offset: 0, size: 230 2024.10.22 02:12:20.724597 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c0d788f3137f82e88ed9a19ac834b17c for removal 2024.10.22 02:12:20.832208 [ 1476 ] {} CacheMetadata: Key c0d788f3137f82e88ed9a19ac834b17c is removed from metadata 2024.10.22 02:12:20.833712 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 65ce0e539938ea7047e0607c604e3f9e, offset: 0, size: 164 2024.10.22 02:12:20.837831 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 65ce0e539938ea7047e0607c604e3f9e for removal 2024.10.22 02:12:20.841294 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3f543e329c154b0d83cd2101e320d89f, offset: 0, size: 228 2024.10.22 02:12:20.842864 [ 1476 ] {} CacheMetadata: Key 65ce0e539938ea7047e0607c604e3f9e is removed from metadata 2024.10.22 02:12:20.845166 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 3f543e329c154b0d83cd2101e320d89f for removal 2024.10.22 02:12:20.850159 [ 1476 ] {} CacheMetadata: Key 3f543e329c154b0d83cd2101e320d89f is removed from metadata 2024.10.22 02:12:20.850358 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 02f913d7e9d96cc955101725643e2d02, offset: 0, size: 111928 2024.10.22 02:12:20.852978 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 02f913d7e9d96cc955101725643e2d02 for removal 2024.10.22 02:12:20.862306 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9d60c1cdaadc370a1cd777f295f8e8b4, offset: 0, size: 263 2024.10.22 02:12:20.867845 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 9d60c1cdaadc370a1cd777f295f8e8b4 for removal 2024.10.22 02:12:20.868513 [ 1476 ] {} CacheMetadata: Key 02f913d7e9d96cc955101725643e2d02 is removed from metadata 2024.10.22 02:12:20.870939 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: e402db6b0365abaaeab9ac0d32606086, offset: 0, size: 166 2024.10.22 02:12:20.874185 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key e402db6b0365abaaeab9ac0d32606086 for removal 2024.10.22 02:12:20.883923 [ 1476 ] {} CacheMetadata: Key e402db6b0365abaaeab9ac0d32606086 is removed from metadata 2024.10.22 02:12:20.883519 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 42e315c53c12a72628cafcebb3ca9d12, offset: 0, size: 98105 2024.10.22 02:12:20.895056 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 42e315c53c12a72628cafcebb3ca9d12 for removal 2024.10.22 02:12:20.895444 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_796_150} MergeTask::MergeProjectionsStage: Merge sorted 1201583 rows, containing 4 columns (4 merged, 0 gathered) in 40.124982057 sec., 29946.007160653124 rows/sec., 449.16 KiB/sec. 2024.10.22 02:12:20.926599 [ 1476 ] {} CacheMetadata: Key 42e315c53c12a72628cafcebb3ca9d12 is removed from metadata 2024.10.22 02:12:20.943181 [ 1476 ] {} CacheMetadata: Key 9d60c1cdaadc370a1cd777f295f8e8b4 is removed from metadata 2024.10.22 02:12:20.954219 [ 1476 ] {} CacheMetadata: Key 9d71d7184b77417f941716a9c24fd94d is removed from metadata 2024.10.22 02:12:20.964396 [ 1476 ] {} CacheMetadata: Key d89663e39ed984095909e4313b96f904 is removed from metadata 2024.10.22 02:12:20.970465 [ 1476 ] {} CacheMetadata: Key 63b73f52a5a759d072137697c4b3a8f0 is removed from metadata 2024.10.22 02:12:20.925834 [ 1204 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:12:21.032202 [ 1204 ] {} DNSCacheUpdater: Execution took 370 ms. 2024.10.22 02:12:21.051537 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_796_150} MergedBlockOutputStream: filled checksums 202410_1_796_150 (state Temporary) 2024.10.22 02:12:21.155322 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 168 entries to flush up to offset 92768 2024.10.22 02:12:21.260753 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:12:21.265719 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:12:21.715099 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 02:12:21.731132 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 02:12:21.868043 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 02:12:21.886379 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 479566 bytes for 48e2b9565d473cc0cc4b01baaf124c6c:0) 2024.10.22 02:12:21.926609 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 02:12:21.926694 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 52d7c186b0b95bbbe64cd3c6321c4baa, offset: 0, size: 202180 2024.10.22 02:12:21.934449 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 52d7c186b0b95bbbe64cd3c6321c4baa for removal 2024.10.22 02:12:21.945737 [ 1476 ] {} CacheMetadata: Key 52d7c186b0b95bbbe64cd3c6321c4baa is removed from metadata 2024.10.22 02:12:21.949799 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 46cce3efa2fbe9f5bcbab80eadab38a1, offset: 0, size: 167 2024.10.22 02:12:21.958696 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 46cce3efa2fbe9f5bcbab80eadab38a1 for removal 2024.10.22 02:12:21.962925 [ 1476 ] {} CacheMetadata: Key 46cce3efa2fbe9f5bcbab80eadab38a1 is removed from metadata 2024.10.22 02:12:21.980817 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:21.985713 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 02:12:21.996947 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 02:12:22.020469 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 02:12:22.067430 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsOldCounter 2024.10.22 02:12:22.099558 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1405_1405_0 (state Temporary) 2024.10.22 02:12:22.171341 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fhd/dbmszrdzvzrvcxqdqujnfucoeyvnl. 2024.10.22 02:12:22.228592 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1202070 2024.10.22 02:12:22.297693 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1405_1405_0 to 202410_1418_1418_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:22.326283 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a2f761e2c0d311319af91e5da96c6d76, offset: 0, size: 0 2024.10.22 02:12:22.330923 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a2f761e2c0d311319af91e5da96c6d76 for removal 2024.10.22 02:12:22.336065 [ 1476 ] {} CacheMetadata: Key a2f761e2c0d311319af91e5da96c6d76 is removed from metadata 2024.10.22 02:12:22.458012 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/eph/hdngfahzcnxwhxutuvvhvjtjnkqaa (LogSeriesLimiter: on interval from 2024-10-22 02:12:09 to 2024-10-22 02:12:22 accepted series 1 / 3 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:12:22.465721 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 92768 2024.10.22 02:12:22.506517 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 630ad6d6ac2d9135ce5be44a6bdb36c2, offset: 0, size: 0 2024.10.22 02:12:22.514237 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 630ad6d6ac2d9135ce5be44a6bdb36c2 for removal 2024.10.22 02:12:22.523981 [ 1476 ] {} CacheMetadata: Key 630ad6d6ac2d9135ce5be44a6bdb36c2 is removed from metadata 2024.10.22 02:12:22.534917 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fhd/dbmszrdzvzrvcxqdqujnfucoeyvnl. 2024.10.22 02:12:22.847335 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 693 2024.10.22 02:12:23.063334 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6725 2024.10.22 02:12:23.111936 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:23.135852 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 02:12:23.139413 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 3.68 MiB, will set to 1.60 GiB (RSS), difference: 5.42 MiB 2024.10.22 02:12:23.322240 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_798_798_0 (state Temporary) 2024.10.22 02:12:23.325179 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:23.328551 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.83 GiB. 2024.10.22 02:12:23.453347 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_798_798_0 to 202410_805_805_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:23.481824 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1202070 2024.10.22 02:12:23.562668 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 47662 bytes for a2f761e2c0d311319af91e5da96c6d76:0) 2024.10.22 02:12:23.567692 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 44715bb5e051bfb6b9605d1c58ee8fab, offset: 0, size: 1186335 2024.10.22 02:12:23.571617 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 44715bb5e051bfb6b9605d1c58ee8fab for removal 2024.10.22 02:12:23.574950 [ 1476 ] {} CacheMetadata: Key 44715bb5e051bfb6b9605d1c58ee8fab is removed from metadata 2024.10.22 02:12:23.580709 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f89006b3bcb9bb36e0b46ed658f6641e, offset: 0, size: 640 2024.10.22 02:12:23.589024 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f89006b3bcb9bb36e0b46ed658f6641e for removal 2024.10.22 02:12:23.589419 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_675_675_0 (state Temporary) 2024.10.22 02:12:23.646718 [ 1476 ] {} CacheMetadata: Key f89006b3bcb9bb36e0b46ed658f6641e is removed from metadata 2024.10.22 02:12:23.776010 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_675_675_0 to 202410_685_685_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:23.851774 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rom/rneezooasulspmlngmionskosiaze. 2024.10.22 02:12:23.863656 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 693 2024.10.22 02:12:23.997110 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 08cf9264ad334000ebeafa9306abe0b2, offset: 0, size: 0 2024.10.22 02:12:24.000071 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 08cf9264ad334000ebeafa9306abe0b2 for removal 2024.10.22 02:12:24.003572 [ 1476 ] {} CacheMetadata: Key 08cf9264ad334000ebeafa9306abe0b2 is removed from metadata 2024.10.22 02:12:24.047546 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9984c4471e336290b002d97b63e1089a, offset: 0, size: 0 2024.10.22 02:12:24.049194 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9984c4471e336290b002d97b63e1089a for removal 2024.10.22 02:12:24.061360 [ 1476 ] {} CacheMetadata: Key 9984c4471e336290b002d97b63e1089a is removed from metadata 2024.10.22 02:12:24.065758 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rom/rneezooasulspmlngmionskosiaze. 2024.10.22 02:12:24.602481 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 12688 2024.10.22 02:12:24.855542 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_796_150} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_796_150 to 202410_1_796_150 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:24.952971 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:24.955950 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 02:12:24.965693 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_796_150} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_790_149, 202410_791_796_1] -> 202410_1_796_150 2024.10.22 02:12:24.999266 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:12:25.100917 [ 1682 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vgi/djselzodcomnpacpzggvbbyhygsgv, size 64777 2024.10.22 02:12:25.112791 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_834_834_0 (state Temporary) 2024.10.22 02:12:25.279362 [ 1424 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:12:25.391292 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_834_834_0 to 202410_847_847_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:25.512894 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12688 2024.10.22 02:12:25.548214 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:12:25.554971 [ 1169 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_842_215 to 202410_843_843_0 2024.10.22 02:12:25.563325 [ 1169 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:25.565813 [ 1169 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 02:12:25.574008 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 380892 bytes for 08cf9264ad334000ebeafa9306abe0b2:0) 2024.10.22 02:12:25.578204 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 237cb1427bdfecaec845955457540d86, offset: 0, size: 264 2024.10.22 02:12:25.587735 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 237cb1427bdfecaec845955457540d86 for removal 2024.10.22 02:12:25.595535 [ 1476 ] {} CacheMetadata: Key 237cb1427bdfecaec845955457540d86 is removed from metadata 2024.10.22 02:12:25.597806 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6a61d37cf668c47d8d9485fb2cd09b1d, offset: 0, size: 2153403 2024.10.22 02:12:25.604090 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6a61d37cf668c47d8d9485fb2cd09b1d for removal 2024.10.22 02:12:25.610591 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: be028f8cbc7d304896d0fe5f0da36d84, offset: 0, size: 7701 2024.10.22 02:12:25.616865 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key be028f8cbc7d304896d0fe5f0da36d84 for removal 2024.10.22 02:12:25.624139 [ 1476 ] {} CacheMetadata: Key 6a61d37cf668c47d8d9485fb2cd09b1d is removed from metadata 2024.10.22 02:12:25.648070 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9df9bed1cc82d527fb4f0cb4e23aa7c7, offset: 0, size: 161 2024.10.22 02:12:25.653321 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9df9bed1cc82d527fb4f0cb4e23aa7c7 for removal 2024.10.22 02:12:25.654480 [ 1476 ] {} CacheMetadata: Key be028f8cbc7d304896d0fe5f0da36d84 is removed from metadata 2024.10.22 02:12:25.721632 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_843_216} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_842_215 to 202410_843_843_0 into Compact with storage Full 2024.10.22 02:12:25.729640 [ 1476 ] {} CacheMetadata: Key 9df9bed1cc82d527fb4f0cb4e23aa7c7 is removed from metadata 2024.10.22 02:12:25.788294 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/lqy/zkksdmazvktdybmijbgjyzekxzdsx. 2024.10.22 02:12:25.808095 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_843_216} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:25.962963 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 2.00 GiB, free memory in arenas 5.52 MiB, will set to 1.60 GiB (RSS), difference: 126.49 MiB 2024.10.22 02:12:25.990240 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_843_216} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_842_215, total 12719 rows starting from the beginning of the part 2024.10.22 02:12:25.988237 [ 1202 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 02:12:26.025422 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a549e2adff3d88c9ce5d12be3916aa62, offset: 0, size: 0 2024.10.22 02:12:26.032499 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a549e2adff3d88c9ce5d12be3916aa62 for removal 2024.10.22 02:12:26.035348 [ 1476 ] {} CacheMetadata: Key a549e2adff3d88c9ce5d12be3916aa62 is removed from metadata 2024.10.22 02:12:26.051237 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_843_216} MergeTreeSequentialSource: Reading 2 marks from part 202410_843_843_0, total 6 rows starting from the beginning of the part 2024.10.22 02:12:26.067318 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_762_3, 202410_763_763_0, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0, 202410_767_767_0] 2024.10.22 02:12:26.144809 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_762_3, 202410_763_763_0, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0, 202410_767_767_0] 2024.10.22 02:12:26.217584 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 757dc188f642d49a77f52d3b8de8ede4, offset: 0, size: 0 2024.10.22 02:12:26.227202 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 757dc188f642d49a77f52d3b8de8ede4 for removal 2024.10.22 02:12:26.236019 [ 1476 ] {} CacheMetadata: Key 757dc188f642d49a77f52d3b8de8ede4 is removed from metadata 2024.10.22 02:12:26.251428 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/lqy/zkksdmazvktdybmijbgjyzekxzdsx. 2024.10.22 02:12:26.254279 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/oqp/wiexgnifypybamshjrrdpfbefjzpt. 2024.10.22 02:12:26.493493 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 105 entries to flush up to offset 92873 2024.10.22 02:12:26.582750 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_762_3, 202410_763_763_0, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0, 202410_767_767_0] 2024.10.22 02:12:26.707923 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 90ba9199657691b535f1a0b52e1f0cb4, offset: 0, size: 0 2024.10.22 02:12:26.713020 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 90ba9199657691b535f1a0b52e1f0cb4 for removal 2024.10.22 02:12:26.718416 [ 1476 ] {} CacheMetadata: Key 90ba9199657691b535f1a0b52e1f0cb4 is removed from metadata 2024.10.22 02:12:26.935239 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9dd9df2b7a3efe738acf48bdfc3209a7, offset: 0, size: 0 2024.10.22 02:12:26.943315 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9dd9df2b7a3efe738acf48bdfc3209a7 for removal 2024.10.22 02:12:26.952340 [ 1476 ] {} CacheMetadata: Key 9dd9df2b7a3efe738acf48bdfc3209a7 is removed from metadata 2024.10.22 02:12:26.962476 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/oqp/wiexgnifypybamshjrrdpfbefjzpt. 2024.10.22 02:12:27.045640 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:27.051904 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.82 GiB. 2024.10.22 02:12:27.258591 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: baf66a267427a0a4ee2a863be5cd823b, offset: 0, size: 0 2024.10.22 02:12:27.259997 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key baf66a267427a0a4ee2a863be5cd823b for removal 2024.10.22 02:12:27.264394 [ 1476 ] {} CacheMetadata: Key baf66a267427a0a4ee2a863be5cd823b is removed from metadata 2024.10.22 02:12:27.360003 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9ffa5015e6f2419b052861845faaea8f, offset: 0, size: 0 2024.10.22 02:12:27.364068 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9ffa5015e6f2419b052861845faaea8f for removal 2024.10.22 02:12:27.376650 [ 1476 ] {} CacheMetadata: Key 9ffa5015e6f2419b052861845faaea8f is removed from metadata 2024.10.22 02:12:27.535861 [ 1708 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xid/zntugubdeidjhueqzmntmopyvromd, size 162 2024.10.22 02:12:27.540919 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1406_1406_0 (state Temporary) 2024.10.22 02:12:27.560512 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 030891c153701bafa5fb6897d90947ee, offset: 0, size: 0 2024.10.22 02:12:27.565406 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 030891c153701bafa5fb6897d90947ee for removal 2024.10.22 02:12:27.570230 [ 1476 ] {} CacheMetadata: Key 030891c153701bafa5fb6897d90947ee is removed from metadata 2024.10.22 02:12:27.655614 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_843_216} MergeTask::MergeProjectionsStage: Merge sorted 12725 rows, containing 12 columns (12 merged, 0 gathered) in 2.058346809 sec., 6182.145761035355 rows/sec., 986.90 KiB/sec. 2024.10.22 02:12:27.692609 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 3.49 MiB, will set to 1.59 GiB (RSS), difference: 99.53 MiB 2024.10.22 02:12:27.767764 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1406_1406_0 to 202410_1419_1419_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:27.818550 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 92873 2024.10.22 02:12:27.848907 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_843_216} MergedBlockOutputStream: filled checksums 202410_1_843_216 (state Temporary) 2024.10.22 02:12:27.917684 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vbx/uvdyckarmsgaowmqtdihzsufsjwex. 2024.10.22 02:12:28.136890 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 123 entries to flush up to offset 91209 2024.10.22 02:12:28.278990 [ 1201 ] {} DatabaseCatalog: Execution took 402 ms. 2024.10.22 02:12:28.359182 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vbx/uvdyckarmsgaowmqtdihzsufsjwex. 2024.10.22 02:12:28.663718 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ugz/fkaqfdcfgexwgsbvtpkfyzimnulbq. 2024.10.22 02:12:28.670536 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_843_216} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_843_216 to 202410_1_843_216 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:28.680082 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_843_216} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_842_215, 202410_843_843_0] -> 202410_1_843_216 2024.10.22 02:12:28.744291 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.45 MiB. 2024.10.22 02:12:28.796298 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:28.812764 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:28.957544 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ugz/fkaqfdcfgexwgsbvtpkfyzimnulbq. 2024.10.22 02:12:28.961420 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/dnd/mdvxhfatfeleyprlfonvshhvfhzaj. 2024.10.22 02:12:29.017162 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: df0f7b02353c159b75ae5ab4232f6eed, offset: 0, size: 0 2024.10.22 02:12:29.030568 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key df0f7b02353c159b75ae5ab4232f6eed for removal 2024.10.22 02:12:29.037120 [ 1476 ] {} CacheMetadata: Key df0f7b02353c159b75ae5ab4232f6eed is removed from metadata 2024.10.22 02:12:29.072431 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_820_820_0 (state Temporary) 2024.10.22 02:12:29.122592 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_15_15_0 (state Temporary) 2024.10.22 02:12:29.147554 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/dpf/himhnlnezetudzvmevsupwrwuwfvq (LogSeriesLimiter: on interval from 2024-10-22 02:12:22 to 2024-10-22 02:12:29 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:12:29.149562 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_820_820_0 to 202410_826_826_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:29.155400 [ 1680 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/sdh/fsjhgcedsqptjjbciwzdszcvmuubv, size 365 2024.10.22 02:12:29.195913 [ 1678 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/dpf/himhnlnezetudzvmevsupwrwuwfvq, size 4 2024.10.22 02:12:29.209618 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/dnd/mdvxhfatfeleyprlfonvshhvfhzaj. 2024.10.22 02:12:29.210814 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 91209 2024.10.22 02:12:29.752669 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 2.00 GiB, free memory in arenas 3.63 MiB, will set to 1.56 GiB (RSS), difference: 120.42 MiB 2024.10.22 02:12:29.758897 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 1048576 bytes for df0f7b02353c159b75ae5ab4232f6eed:0) 2024.10.22 02:12:29.762713 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a31fa211295ee67f2e9e6512d0dea7aa, offset: 0, size: 640 2024.10.22 02:12:29.766107 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a31fa211295ee67f2e9e6512d0dea7aa for removal 2024.10.22 02:12:29.768711 [ 1476 ] {} CacheMetadata: Key a31fa211295ee67f2e9e6512d0dea7aa is removed from metadata 2024.10.22 02:12:29.768890 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 273132e9a609f11d6fe7087d3cc14d3b, offset: 0, size: 57 2024.10.22 02:12:29.771825 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 273132e9a609f11d6fe7087d3cc14d3b for removal 2024.10.22 02:12:29.777513 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3f6db92dc4196b2f6e5ba5c6d260d5a9, offset: 0, size: 57 2024.10.22 02:12:29.787039 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3f6db92dc4196b2f6e5ba5c6d260d5a9 for removal 2024.10.22 02:12:29.792351 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4ad6c00b543c7ea90cff538ba9d0faa1, offset: 0, size: 452 2024.10.22 02:12:29.795356 [ 1476 ] {} CacheMetadata: Key 3f6db92dc4196b2f6e5ba5c6d260d5a9 is removed from metadata 2024.10.22 02:12:29.797328 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4ad6c00b543c7ea90cff538ba9d0faa1 for removal 2024.10.22 02:12:29.800812 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 22ecdf570076e6c0dd3a74f9d3b45e6d, offset: 0, size: 452 2024.10.22 02:12:29.805626 [ 1476 ] {} CacheMetadata: Key 4ad6c00b543c7ea90cff538ba9d0faa1 is removed from metadata 2024.10.22 02:12:29.809280 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 22ecdf570076e6c0dd3a74f9d3b45e6d for removal 2024.10.22 02:12:29.814497 [ 1476 ] {} CacheMetadata: Key 22ecdf570076e6c0dd3a74f9d3b45e6d is removed from metadata 2024.10.22 02:12:29.815206 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c6a201e507ba8a8a19fd74bee2a39e93, offset: 0, size: 57 2024.10.22 02:12:29.819957 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c6a201e507ba8a8a19fd74bee2a39e93 for removal 2024.10.22 02:12:29.822234 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 094c5b448040b68d5ed10e2821a5a8f5, offset: 0, size: 1154817 2024.10.22 02:12:29.823852 [ 1476 ] {} CacheMetadata: Key c6a201e507ba8a8a19fd74bee2a39e93 is removed from metadata 2024.10.22 02:12:29.826916 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 094c5b448040b68d5ed10e2821a5a8f5 for removal 2024.10.22 02:12:29.830227 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fcd3c9c2e9d86650fa03805a82ac0d9a, offset: 0, size: 57 2024.10.22 02:12:29.836124 [ 1476 ] {} CacheMetadata: Key 094c5b448040b68d5ed10e2821a5a8f5 is removed from metadata 2024.10.22 02:12:29.836122 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fcd3c9c2e9d86650fa03805a82ac0d9a for removal 2024.10.22 02:12:29.854532 [ 1476 ] {} CacheMetadata: Key fcd3c9c2e9d86650fa03805a82ac0d9a is removed from metadata 2024.10.22 02:12:29.962426 [ 1476 ] {} CacheMetadata: Key 273132e9a609f11d6fe7087d3cc14d3b is removed from metadata 2024.10.22 02:12:30.508816 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/tya/setmktoqqnaurzllfdsodugnsvrec. 2024.10.22 02:12:30.535606 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_165_169_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:12:30.557488 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b05b8be1fed58e2286a5400cf2247b40, offset: 0, size: 0 2024.10.22 02:12:30.559888 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b05b8be1fed58e2286a5400cf2247b40 for removal 2024.10.22 02:12:30.562350 [ 1476 ] {} CacheMetadata: Key b05b8be1fed58e2286a5400cf2247b40 is removed from metadata 2024.10.22 02:12:30.595762 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1203610 2024.10.22 02:12:30.617752 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/tya/setmktoqqnaurzllfdsodugnsvrec. 2024.10.22 02:12:30.859799 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 44 entries to flush up to offset 6769 2024.10.22 02:12:31.236165 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:31.240513 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:31.259531 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:12:31.260730 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:12:31.365816 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 694 2024.10.22 02:12:31.366763 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for b05b8be1fed58e2286a5400cf2247b40:0) 2024.10.22 02:12:31.368599 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fd1a5c52a55bfd4e138d55752f1be326, offset: 0, size: 141224 2024.10.22 02:12:31.371678 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fd1a5c52a55bfd4e138d55752f1be326 for removal 2024.10.22 02:12:31.378744 [ 1476 ] {} CacheMetadata: Key fd1a5c52a55bfd4e138d55752f1be326 is removed from metadata 2024.10.22 02:12:31.379611 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2e595d93bfa553bdd591a0822dedbe81, offset: 0, size: 161481 2024.10.22 02:12:31.383374 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2e595d93bfa553bdd591a0822dedbe81 for removal 2024.10.22 02:12:31.388403 [ 1476 ] {} CacheMetadata: Key 2e595d93bfa553bdd591a0822dedbe81 is removed from metadata 2024.10.22 02:12:31.387873 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 21df529796d5dc42d6bb742902574c03, offset: 0, size: 167 2024.10.22 02:12:31.400651 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 21df529796d5dc42d6bb742902574c03 for removal 2024.10.22 02:12:31.403008 [ 1476 ] {} CacheMetadata: Key 21df529796d5dc42d6bb742902574c03 is removed from metadata 2024.10.22 02:12:31.403026 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dad50c653803e622ca5b749959ab5bad, offset: 0, size: 168 2024.10.22 02:12:31.411145 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dad50c653803e622ca5b749959ab5bad for removal 2024.10.22 02:12:31.414960 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 4.42 MiB, will set to 1.56 GiB (RSS), difference: 1.28 MiB 2024.10.22 02:12:31.416486 [ 1476 ] {} CacheMetadata: Key dad50c653803e622ca5b749959ab5bad is removed from metadata 2024.10.22 02:12:31.455553 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_799_799_0 (state Temporary) 2024.10.22 02:12:31.545513 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:31.548143 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:31.664749 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_799_799_0 to 202410_806_806_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:31.694154 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_676_676_0 (state Temporary) 2024.10.22 02:12:31.699843 [ 1213 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_796_150 to 202410_797_802_1 2024.10.22 02:12:31.705697 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1203610 2024.10.22 02:12:31.715176 [ 1213 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:31.737557 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:31.767150 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_802_151} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_796_150 to 202410_797_802_1 into Wide with storage Full 2024.10.22 02:12:31.833755 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_802_151} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:31.851817 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 113 entries to flush up to offset 92986 2024.10.22 02:12:31.855560 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_802_151} MergeTreeSequentialSource: Reading 148 marks from part 202410_1_796_150, total 1201583 rows starting from the beginning of the part 2024.10.22 02:12:31.872220 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_802_151} MergeTreeSequentialSource: Reading 3 marks from part 202410_797_802_1, total 9842 rows starting from the beginning of the part 2024.10.22 02:12:31.972272 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_676_676_0 to 202410_686_686_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:32.159136 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 694 2024.10.22 02:12:32.168800 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:32.171766 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:32.290513 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1407_1407_0 (state Temporary) 2024.10.22 02:12:32.568050 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1407_1407_0 to 202410_1420_1420_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:32.624867 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 11 file segments (while reserving 1048576 bytes for b05b8be1fed58e2286a5400cf2247b40:0) 2024.10.22 02:12:32.632669 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6799854bd0156bf3de78b2ed3279b084, offset: 0, size: 535047 2024.10.22 02:12:32.641017 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6799854bd0156bf3de78b2ed3279b084 for removal 2024.10.22 02:12:32.648623 [ 1476 ] {} CacheMetadata: Key 6799854bd0156bf3de78b2ed3279b084 is removed from metadata 2024.10.22 02:12:32.648438 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 28bf2ed334127d5b9976c3b8a3a9eef2, offset: 0, size: 241 2024.10.22 02:12:32.653549 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 28bf2ed334127d5b9976c3b8a3a9eef2 for removal 2024.10.22 02:12:32.657588 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 92986 2024.10.22 02:12:32.749976 [ 1230 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 02:12:32.756394 [ 1476 ] {} CacheMetadata: Key 28bf2ed334127d5b9976c3b8a3a9eef2 is removed from metadata 2024.10.22 02:12:32.757362 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f787b1b3f3f4c8ba54252305cc84c9c9, offset: 0, size: 494 2024.10.22 02:12:32.760826 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f787b1b3f3f4c8ba54252305cc84c9c9 for removal 2024.10.22 02:12:32.763800 [ 1476 ] {} CacheMetadata: Key f787b1b3f3f4c8ba54252305cc84c9c9 is removed from metadata 2024.10.22 02:12:32.764072 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e011b68a18e07e50e4333111693a9bf2, offset: 0, size: 84153 2024.10.22 02:12:32.768288 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e011b68a18e07e50e4333111693a9bf2 for removal 2024.10.22 02:12:32.771920 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d2ddfe215412bc77583a80e985538386, offset: 0, size: 164 2024.10.22 02:12:32.788724 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d2ddfe215412bc77583a80e985538386 for removal 2024.10.22 02:12:32.795479 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cc8c2033ab645069f7b62ee049cfeab2, offset: 0, size: 554567 2024.10.22 02:12:32.803062 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cc8c2033ab645069f7b62ee049cfeab2 for removal 2024.10.22 02:12:32.812352 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fdc5c0ae26a21ff21351af83436467ef, offset: 0, size: 193 2024.10.22 02:12:32.871386 [ 1476 ] {} CacheMetadata: Key cc8c2033ab645069f7b62ee049cfeab2 is removed from metadata 2024.10.22 02:12:32.881194 [ 1476 ] {} CacheMetadata: Key d2ddfe215412bc77583a80e985538386 is removed from metadata 2024.10.22 02:12:32.898356 [ 1476 ] {} CacheMetadata: Key e011b68a18e07e50e4333111693a9bf2 is removed from metadata 2024.10.22 02:12:32.921155 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fdc5c0ae26a21ff21351af83436467ef for removal 2024.10.22 02:12:32.929074 [ 1476 ] {} CacheMetadata: Key fdc5c0ae26a21ff21351af83436467ef is removed from metadata 2024.10.22 02:12:32.932144 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d40462c5897d47801a5845686b8972b2, offset: 0, size: 496 2024.10.22 02:12:32.937842 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d40462c5897d47801a5845686b8972b2 for removal 2024.10.22 02:12:32.941261 [ 1476 ] {} CacheMetadata: Key d40462c5897d47801a5845686b8972b2 is removed from metadata 2024.10.22 02:12:32.943176 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8b031006a18a547a7ad6127f1e7b2dd9, offset: 0, size: 137518 2024.10.22 02:12:32.948067 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8b031006a18a547a7ad6127f1e7b2dd9 for removal 2024.10.22 02:12:32.952314 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3b99df8660aee744ac431ea9828c7923, offset: 0, size: 207157 2024.10.22 02:12:32.954615 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3b99df8660aee744ac431ea9828c7923 for removal 2024.10.22 02:12:32.956846 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e36bc03c47529725a0cdbe38d9811dc7, offset: 0, size: 164 2024.10.22 02:12:32.960990 [ 1476 ] {} CacheMetadata: Key 3b99df8660aee744ac431ea9828c7923 is removed from metadata 2024.10.22 02:12:32.960578 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e36bc03c47529725a0cdbe38d9811dc7 for removal 2024.10.22 02:12:32.968371 [ 1476 ] {} CacheMetadata: Key e36bc03c47529725a0cdbe38d9811dc7 is removed from metadata 2024.10.22 02:12:32.972978 [ 1476 ] {} CacheMetadata: Key 8b031006a18a547a7ad6127f1e7b2dd9 is removed from metadata 2024.10.22 02:12:33.123958 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 12700 2024.10.22 02:12:33.360793 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:33.364861 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:33.447317 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_835_835_0 (state Temporary) 2024.10.22 02:12:33.533358 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 5.84 MiB, will set to 1.57 GiB (RSS), difference: -10.78 MiB 2024.10.22 02:12:33.599037 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_835_835_0 to 202410_848_848_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:33.642806 [ 1175 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_843_216 to 202410_844_844_0 2024.10.22 02:12:33.666744 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12700 2024.10.22 02:12:33.675231 [ 1175 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:33.679961 [ 1175 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:33.736860 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_844_217} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_843_216 to 202410_844_844_0 into Compact with storage Full 2024.10.22 02:12:33.760360 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_844_217} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:33.772143 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_844_217} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_843_216, total 12725 rows starting from the beginning of the part 2024.10.22 02:12:33.803866 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_844_217} MergeTreeSequentialSource: Reading 2 marks from part 202410_844_844_0, total 21 rows starting from the beginning of the part 2024.10.22 02:12:33.861542 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 543882 bytes for b05b8be1fed58e2286a5400cf2247b40:0) 2024.10.22 02:12:33.865172 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5268562f0ea8a9752cc0aba8ed76626b, offset: 0, size: 523060 2024.10.22 02:12:33.885263 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5268562f0ea8a9752cc0aba8ed76626b for removal 2024.10.22 02:12:33.893495 [ 1476 ] {} CacheMetadata: Key 5268562f0ea8a9752cc0aba8ed76626b is removed from metadata 2024.10.22 02:12:34.081974 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/sxi/alumqirmcsartnhsdyvfqfyiwhlub. 2024.10.22 02:12:34.240859 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 37205d77145d65d8ed5b6247cee4ff5c, offset: 0, size: 0 2024.10.22 02:12:34.244733 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 37205d77145d65d8ed5b6247cee4ff5c for removal 2024.10.22 02:12:34.251351 [ 1476 ] {} CacheMetadata: Key 37205d77145d65d8ed5b6247cee4ff5c is removed from metadata 2024.10.22 02:12:34.328888 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/sxi/alumqirmcsartnhsdyvfqfyiwhlub. 2024.10.22 02:12:34.792425 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_844_217} MergeTask::MergeProjectionsStage: Merge sorted 12746 rows, containing 12 columns (12 merged, 0 gathered) in 1.091346907 sec., 11679.146125073501 rows/sec., 1.82 MiB/sec. 2024.10.22 02:12:34.946432 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_844_217} MergedBlockOutputStream: filled checksums 202410_1_844_217 (state Temporary) 2024.10.22 02:12:35.061604 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_844_217} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_844_217 to 202410_1_844_217 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:35.074265 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_844_217} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_843_216, 202410_844_844_0] -> 202410_1_844_217 2024.10.22 02:12:35.231367 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.46 MiB. 2024.10.22 02:12:35.373069 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 37205d77145d65d8ed5b6247cee4ff5c:0) 2024.10.22 02:12:35.374679 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1ca58f6febc896266271404bca92c4dc, offset: 0, size: 319372 2024.10.22 02:12:35.403077 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_784_173, 202410_785_785_0] 2024.10.22 02:12:35.426121 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_784_173, 202410_785_785_0] 2024.10.22 02:12:35.435472 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_784_173, 202410_785_785_0] 2024.10.22 02:12:35.451974 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 12.55 MiB, will set to 1.58 GiB (RSS), difference: 16.92 MiB 2024.10.22 02:12:35.481542 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1ca58f6febc896266271404bca92c4dc for removal 2024.10.22 02:12:35.490903 [ 1476 ] {} CacheMetadata: Key 1ca58f6febc896266271404bca92c4dc is removed from metadata 2024.10.22 02:12:35.491523 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8deae8e7bb10a85004b94b50a1a260b5, offset: 0, size: 491 2024.10.22 02:12:35.597030 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8deae8e7bb10a85004b94b50a1a260b5 for removal 2024.10.22 02:12:35.600447 [ 1476 ] {} CacheMetadata: Key 8deae8e7bb10a85004b94b50a1a260b5 is removed from metadata 2024.10.22 02:12:35.600557 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 322556128f75f7c4a9c59f52b730cf6b, offset: 0, size: 430759 2024.10.22 02:12:35.603919 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 322556128f75f7c4a9c59f52b730cf6b for removal 2024.10.22 02:12:35.620233 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dc05ddab31046edadce9a8e17d5ad59d, offset: 0, size: 240 2024.10.22 02:12:35.626520 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dc05ddab31046edadce9a8e17d5ad59d for removal 2024.10.22 02:12:35.731863 [ 1476 ] {} CacheMetadata: Key dc05ddab31046edadce9a8e17d5ad59d is removed from metadata 2024.10.22 02:12:35.840318 [ 1476 ] {} CacheMetadata: Key 322556128f75f7c4a9c59f52b730cf6b is removed from metadata 2024.10.22 02:12:36.045094 [ 1273 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:12:36.082689 [ 1273 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:12:36.428719 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 1048576 bytes for 37205d77145d65d8ed5b6247cee4ff5c:0) 2024.10.22 02:12:36.432557 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6fee2f126305d9ab359580a608883efb, offset: 0, size: 240 2024.10.22 02:12:36.439398 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6fee2f126305d9ab359580a608883efb for removal 2024.10.22 02:12:36.443061 [ 1476 ] {} CacheMetadata: Key 6fee2f126305d9ab359580a608883efb is removed from metadata 2024.10.22 02:12:36.445173 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7f2ae4f053ab5e229d25ba1a055f82aa, offset: 0, size: 229 2024.10.22 02:12:36.449298 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7f2ae4f053ab5e229d25ba1a055f82aa for removal 2024.10.22 02:12:36.452116 [ 1476 ] {} CacheMetadata: Key 7f2ae4f053ab5e229d25ba1a055f82aa is removed from metadata 2024.10.22 02:12:36.452093 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6952795899a4214856435905b13cab22, offset: 0, size: 236 2024.10.22 02:12:36.455609 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6952795899a4214856435905b13cab22 for removal 2024.10.22 02:12:36.459003 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2c5a4248da29d3132908c21b53b04429, offset: 0, size: 458780 2024.10.22 02:12:36.462316 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2c5a4248da29d3132908c21b53b04429 for removal 2024.10.22 02:12:36.465332 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fcba2fc821e6ceed6fbc6a9d33931fc3, offset: 0, size: 275501 2024.10.22 02:12:36.462443 [ 1476 ] {} CacheMetadata: Key 6952795899a4214856435905b13cab22 is removed from metadata 2024.10.22 02:12:36.473479 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fcba2fc821e6ceed6fbc6a9d33931fc3 for removal 2024.10.22 02:12:36.480530 [ 1476 ] {} CacheMetadata: Key fcba2fc821e6ceed6fbc6a9d33931fc3 is removed from metadata 2024.10.22 02:12:36.495323 [ 1476 ] {} CacheMetadata: Key 2c5a4248da29d3132908c21b53b04429 is removed from metadata 2024.10.22 02:12:36.483132 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 94d61c0d5d4aa276f9bc2cca11570daf, offset: 0, size: 549485 2024.10.22 02:12:36.620853 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 94d61c0d5d4aa276f9bc2cca11570daf for removal 2024.10.22 02:12:36.624589 [ 1476 ] {} CacheMetadata: Key 94d61c0d5d4aa276f9bc2cca11570daf is removed from metadata 2024.10.22 02:12:36.628251 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9eca1f2f93c172149b7f64572bbadaa9, offset: 0, size: 196 2024.10.22 02:12:36.632469 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9eca1f2f93c172149b7f64572bbadaa9 for removal 2024.10.22 02:12:36.634794 [ 1476 ] {} CacheMetadata: Key 9eca1f2f93c172149b7f64572bbadaa9 is removed from metadata 2024.10.22 02:12:36.635823 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9fb1b45c0672301cd7720185e7c70fb7, offset: 0, size: 9811 2024.10.22 02:12:36.639120 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9fb1b45c0672301cd7720185e7c70fb7 for removal 2024.10.22 02:12:36.643038 [ 1476 ] {} CacheMetadata: Key 9fb1b45c0672301cd7720185e7c70fb7 is removed from metadata 2024.10.22 02:12:36.763414 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 118 entries to flush up to offset 93104 2024.10.22 02:12:36.859277 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 119 entries to flush up to offset 91328 2024.10.22 02:12:36.959468 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:36.962311 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:37.047181 [ 1263 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 02:12:37.140465 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:37.144206 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:37.157243 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1408_1408_0 (state Temporary) 2024.10.22 02:12:37.177839 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_821_821_0 (state Temporary) 2024.10.22 02:12:37.225617 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1408_1408_0 to 202410_1421_1421_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:37.240021 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_802_151/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:12:37.248907 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 37205d77145d65d8ed5b6247cee4ff5c:0) 2024.10.22 02:12:37.251102 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8114fd847e6f969e010f6a0401181fcc, offset: 0, size: 677871 2024.10.22 02:12:37.252102 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_821_821_0 to 202410_827_827_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:37.253222 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8114fd847e6f969e010f6a0401181fcc for removal 2024.10.22 02:12:37.254386 [ 1476 ] {} CacheMetadata: Key 8114fd847e6f969e010f6a0401181fcc is removed from metadata 2024.10.22 02:12:37.255465 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7e5949d27d213b2c64cd53ce8b71a830, offset: 0, size: 246 2024.10.22 02:12:37.258645 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7e5949d27d213b2c64cd53ce8b71a830 for removal 2024.10.22 02:12:37.268912 [ 1476 ] {} CacheMetadata: Key 7e5949d27d213b2c64cd53ce8b71a830 is removed from metadata 2024.10.22 02:12:37.366967 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1416_42 to 202410_1421_1421_0 2024.10.22 02:12:37.370819 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_822_15 to 202410_827_827_0 2024.10.22 02:12:37.385359 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:37.392159 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 93104 2024.10.22 02:12:37.411781 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:37.393535 [ 1241 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:37.497140 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:37.511513 [ 1257 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 246 ms. 2024.10.22 02:12:37.549703 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1416_42 to 202410_1421_1421_0 into Compact with storage Full 2024.10.22 02:12:37.550577 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_822_15 to 202410_827_827_0 into Compact with storage Full 2024.10.22 02:12:37.570096 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 91328 2024.10.22 02:12:37.615938 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:37.661309 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_822_15, total 7409 rows starting from the beginning of the part 2024.10.22 02:12:37.734383 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:37.754672 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1416_42, total 13285 rows starting from the beginning of the part 2024.10.22 02:12:37.800215 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_1417_1417_0, total 174 rows starting from the beginning of the part 2024.10.22 02:12:37.812552 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 76083 bytes for 37205d77145d65d8ed5b6247cee4ff5c:0) 2024.10.22 02:12:37.853445 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: acd2e094fda37431f4a0024f9d064818, offset: 0, size: 1104759 2024.10.22 02:12:37.871180 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_1418_1418_0, total 168 rows starting from the beginning of the part 2024.10.22 02:12:37.878468 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key acd2e094fda37431f4a0024f9d064818 for removal 2024.10.22 02:12:37.887550 [ 1476 ] {} CacheMetadata: Key acd2e094fda37431f4a0024f9d064818 is removed from metadata 2024.10.22 02:12:37.888023 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5256f531d5bf87db73fdfefba2d9a6be, offset: 0, size: 252 2024.10.22 02:12:37.895456 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5256f531d5bf87db73fdfefba2d9a6be for removal 2024.10.22 02:12:37.899784 [ 1476 ] {} CacheMetadata: Key 5256f531d5bf87db73fdfefba2d9a6be is removed from metadata 2024.10.22 02:12:37.921176 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_1419_1419_0, total 105 rows starting from the beginning of the part 2024.10.22 02:12:37.946864 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mqg/zvdwetmgyrotxtoralhrtwuidtjsh. 2024.10.22 02:12:37.995677 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 914f14352ee5c8db5bbe9351f51dacb0, offset: 0, size: 0 2024.10.22 02:12:37.999318 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 914f14352ee5c8db5bbe9351f51dacb0 for removal 2024.10.22 02:12:38.003907 [ 1476 ] {} CacheMetadata: Key 914f14352ee5c8db5bbe9351f51dacb0 is removed from metadata 2024.10.22 02:12:38.165343 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_1420_1420_0, total 113 rows starting from the beginning of the part 2024.10.22 02:12:38.171592 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mqg/zvdwetmgyrotxtoralhrtwuidtjsh. 2024.10.22 02:12:38.241358 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_1421_1421_0, total 118 rows starting from the beginning of the part 2024.10.22 02:12:38.257699 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_823_823_0, total 127 rows starting from the beginning of the part 2024.10.22 02:12:38.453486 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_824_824_0, total 96 rows starting from the beginning of the part 2024.10.22 02:12:38.504138 [ 1153 ] {} DatabaseCatalog: Execution took 224 ms. 2024.10.22 02:12:38.565627 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_825_825_0, total 99 rows starting from the beginning of the part 2024.10.22 02:12:38.663752 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1317_22, 202410_1318_1318_0, 202410_1319_1319_0, 202410_1320_1320_0, 202410_1321_1321_0, 202410_1322_1322_0] 2024.10.22 02:12:38.665272 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1317_22, 202410_1318_1318_0, 202410_1319_1319_0, 202410_1320_1320_0, 202410_1321_1321_0, 202410_1322_1322_0] 2024.10.22 02:12:38.662577 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_826_826_0, total 123 rows starting from the beginning of the part 2024.10.22 02:12:38.707877 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1317_22, 202410_1318_1318_0, 202410_1319_1319_0, 202410_1320_1320_0, 202410_1321_1321_0, 202410_1322_1322_0] 2024.10.22 02:12:38.712117 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1205150 2024.10.22 02:12:38.828682 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} MergeTreeSequentialSource: Reading 2 marks from part 202410_827_827_0, total 119 rows starting from the beginning of the part 2024.10.22 02:12:39.365193 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 914f14352ee5c8db5bbe9351f51dacb0:0) 2024.10.22 02:12:39.367023 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 723dc7a7b2d3e0ad7b20e55b9094b531, offset: 0, size: 1124033 2024.10.22 02:12:39.370958 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 723dc7a7b2d3e0ad7b20e55b9094b531 for removal 2024.10.22 02:12:39.373753 [ 1476 ] {} CacheMetadata: Key 723dc7a7b2d3e0ad7b20e55b9094b531 is removed from metadata 2024.10.22 02:12:39.374579 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: bf269513a1b58ca26900bae968fb3eec, offset: 0, size: 641 2024.10.22 02:12:39.380213 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key bf269513a1b58ca26900bae968fb3eec for removal 2024.10.22 02:12:39.409805 [ 1476 ] {} CacheMetadata: Key bf269513a1b58ca26900bae968fb3eec is removed from metadata 2024.10.22 02:12:39.503347 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:39.514956 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:39.670880 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_800_800_0 (state Temporary) 2024.10.22 02:12:39.671650 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 12.17 MiB, will set to 1.58 GiB (RSS), difference: -17.41 MiB 2024.10.22 02:12:39.781830 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_800_800_0 to 202410_807_807_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:39.840784 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1205150 2024.10.22 02:12:39.882363 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_736_140, 202410_737_742_1] 2024.10.22 02:12:39.888878 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_736_140, 202410_737_742_1] 2024.10.22 02:12:39.905209 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_736_140, 202410_737_742_1] 2024.10.22 02:12:40.016268 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 914f14352ee5c8db5bbe9351f51dacb0:0) 2024.10.22 02:12:40.026307 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 58242d715d331feada47591edfd5c171, offset: 0, size: 1093778 2024.10.22 02:12:40.034744 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 58242d715d331feada47591edfd5c171 for removal 2024.10.22 02:12:40.041500 [ 1476 ] {} CacheMetadata: Key 58242d715d331feada47591edfd5c171 is removed from metadata 2024.10.22 02:12:40.143229 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 135bc705a33ca522c76ec91dadd0ec21, offset: 0, size: 252 2024.10.22 02:12:40.147454 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 135bc705a33ca522c76ec91dadd0ec21 for removal 2024.10.22 02:12:40.150450 [ 1476 ] {} CacheMetadata: Key 135bc705a33ca522c76ec91dadd0ec21 is removed from metadata 2024.10.22 02:12:40.330754 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} MergeTask::MergeProjectionsStage: Merge sorted 13963 rows, containing 13 columns (13 merged, 0 gathered) in 2.799157656 sec., 4988.286376106855 rows/sec., 1019.20 KiB/sec. 2024.10.22 02:12:40.560793 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 1048576 bytes for 914f14352ee5c8db5bbe9351f51dacb0:0) 2024.10.22 02:12:40.567584 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c3959736d3545f55856af0092ffe0022, offset: 0, size: 483 2024.10.22 02:12:40.572714 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c3959736d3545f55856af0092ffe0022 for removal 2024.10.22 02:12:40.575974 [ 1476 ] {} CacheMetadata: Key c3959736d3545f55856af0092ffe0022 is removed from metadata 2024.10.22 02:12:40.583118 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9705db9419e5474f75c9124ead1b97dc, offset: 0, size: 496998 2024.10.22 02:12:40.596118 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9705db9419e5474f75c9124ead1b97dc for removal 2024.10.22 02:12:40.601318 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2bef8608d018e1e0b379cbd2311477af, offset: 0, size: 678374 2024.10.22 02:12:40.609151 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2bef8608d018e1e0b379cbd2311477af for removal 2024.10.22 02:12:40.615526 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: bcaa7a92ea92ddd84ee076e44bab4042, offset: 0, size: 241 2024.10.22 02:12:40.620119 [ 1476 ] {} CacheMetadata: Key 2bef8608d018e1e0b379cbd2311477af is removed from metadata 2024.10.22 02:12:40.626341 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key bcaa7a92ea92ddd84ee076e44bab4042 for removal 2024.10.22 02:12:40.633545 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b544c2e7e3f23c358945db4015b81335, offset: 0, size: 246 2024.10.22 02:12:40.636212 [ 1476 ] {} CacheMetadata: Key bcaa7a92ea92ddd84ee076e44bab4042 is removed from metadata 2024.10.22 02:12:40.642831 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} MergeTask::MergeProjectionsStage: Merge sorted 7973 rows, containing 13 columns (13 merged, 0 gathered) in 3.100919089 sec., 2571.173181616671 rows/sec., 1.11 MiB/sec. 2024.10.22 02:12:40.652339 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b544c2e7e3f23c358945db4015b81335 for removal 2024.10.22 02:12:40.658317 [ 1476 ] {} CacheMetadata: Key b544c2e7e3f23c358945db4015b81335 is removed from metadata 2024.10.22 02:12:40.674936 [ 1476 ] {} CacheMetadata: Key 9705db9419e5474f75c9124ead1b97dc is removed from metadata 2024.10.22 02:12:40.722439 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} MergedBlockOutputStream: filled checksums 202410_1207_1421_43 (state Temporary) 2024.10.22 02:12:40.840328 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1421_43 to 202410_1207_1421_43 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:40.843687 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1421_43} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1416_42, 202410_1421_1421_0] -> 202410_1207_1421_43 2024.10.22 02:12:40.850214 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} MergedBlockOutputStream: filled checksums 202410_747_827_16 (state Temporary) 2024.10.22 02:12:40.883108 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.85 MiB. 2024.10.22 02:12:41.155841 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_827_16 to 202410_747_827_16 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:41.159856 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_827_16} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_822_15, 202410_827_827_0] -> 202410_747_827_16 2024.10.22 02:12:41.168952 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 12721 2024.10.22 02:12:41.188626 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.99 MiB. 2024.10.22 02:12:41.260722 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:12:41.264054 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:12:41.266876 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 915712 bytes for 914f14352ee5c8db5bbe9351f51dacb0:0) 2024.10.22 02:12:41.276299 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b1a7531bf59c11b1876e2361d05504fa, offset: 0, size: 485 2024.10.22 02:12:41.286730 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b1a7531bf59c11b1876e2361d05504fa for removal 2024.10.22 02:12:41.294388 [ 1476 ] {} CacheMetadata: Key b1a7531bf59c11b1876e2361d05504fa is removed from metadata 2024.10.22 02:12:41.295074 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e238bfa69efe4d2d15b75851149c8c64, offset: 0, size: 250 2024.10.22 02:12:41.300078 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e238bfa69efe4d2d15b75851149c8c64 for removal 2024.10.22 02:12:41.308302 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: eeb3c5981bb886a3fce8ac17b7974dd9, offset: 0, size: 961006 2024.10.22 02:12:41.317033 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key eeb3c5981bb886a3fce8ac17b7974dd9 for removal 2024.10.22 02:12:41.323580 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c9ec7d2502e4828e70695b34ecd60494, offset: 0, size: 23999 2024.10.22 02:12:41.331145 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c9ec7d2502e4828e70695b34ecd60494 for removal 2024.10.22 02:12:41.391980 [ 1682 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/eph/hdngfahzcnxwhxutuvvhvjtjnkqaa, size 54 2024.10.22 02:12:41.434204 [ 1476 ] {} CacheMetadata: Key c9ec7d2502e4828e70695b34ecd60494 is removed from metadata 2024.10.22 02:12:41.507748 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 124 entries to flush up to offset 93228 2024.10.22 02:12:41.516216 [ 1476 ] {} CacheMetadata: Key eeb3c5981bb886a3fce8ac17b7974dd9 is removed from metadata 2024.10.22 02:12:41.532627 [ 1476 ] {} CacheMetadata: Key e238bfa69efe4d2d15b75851149c8c64 is removed from metadata 2024.10.22 02:12:41.551224 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.20 MiB, will set to 1.58 GiB (RSS), difference: 22.38 MiB 2024.10.22 02:12:41.583880 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:41.592739 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:41.599886 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xpa/hebccyzvbkhjltamwqbgltswqaohs. 2024.10.22 02:12:41.749907 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_836_836_0 (state Temporary) 2024.10.22 02:12:41.752866 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 197b8031d4cc8fd87597911fa521a5bb, offset: 0, size: 0 2024.10.22 02:12:41.755789 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 197b8031d4cc8fd87597911fa521a5bb for removal 2024.10.22 02:12:41.762402 [ 1476 ] {} CacheMetadata: Key 197b8031d4cc8fd87597911fa521a5bb is removed from metadata 2024.10.22 02:12:41.808035 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsEvent 2024.10.22 02:12:41.809341 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xpa/hebccyzvbkhjltamwqbgltswqaohs. 2024.10.22 02:12:41.839492 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsEvent 2024.10.22 02:12:41.858446 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsEvent 2024.10.22 02:12:41.881217 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsEvent 2024.10.22 02:12:41.932224 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_836_836_0 to 202410_849_849_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:41.937990 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsEvent 2024.10.22 02:12:41.959226 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsEvent 2024.10.22 02:12:41.969829 [ 1165 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_844_217 to 202410_845_845_0 2024.10.22 02:12:41.971557 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12721 2024.10.22 02:12:41.980750 [ 1165 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:41.984092 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:42.017552 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_845_218} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_844_217 to 202410_845_845_0 into Compact with storage Full 2024.10.22 02:12:42.045912 [ 1251 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 02:12:42.065674 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_845_218} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:42.086526 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_845_218} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_844_217, total 12746 rows starting from the beginning of the part 2024.10.22 02:12:42.101728 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsEvent 2024.10.22 02:12:42.113432 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:42.116680 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:42.143388 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_845_218} MergeTreeSequentialSource: Reading 2 marks from part 202410_845_845_0, total 16 rows starting from the beginning of the part 2024.10.22 02:12:42.244760 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1409_1409_0 (state Temporary) 2024.10.22 02:12:42.248672 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_633_129, 202410_634_634_0, 202410_635_635_0, 202410_636_636_0, 202410_637_637_0, 202410_638_638_0] 2024.10.22 02:12:42.251865 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_633_129, 202410_634_634_0, 202410_635_635_0, 202410_636_636_0, 202410_637_637_0, 202410_638_638_0] 2024.10.22 02:12:42.276129 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zsx/hqbbbwqqipsspsqnecrmtxmzmaoqn (LogSeriesLimiter: on interval from 2024-10-22 02:12:29 to 2024-10-22 02:12:42 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:12:42.277165 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_633_129, 202410_634_634_0, 202410_635_635_0, 202410_636_636_0, 202410_637_637_0, 202410_638_638_0] 2024.10.22 02:12:42.409792 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1409_1409_0 to 202410_1422_1422_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:42.467803 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 1048576 bytes for 197b8031d4cc8fd87597911fa521a5bb:0) 2024.10.22 02:12:42.470890 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a47c5f9d5635cf17f16141e7114bd213, offset: 0, size: 246 2024.10.22 02:12:42.474511 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a47c5f9d5635cf17f16141e7114bd213 for removal 2024.10.22 02:12:42.479147 [ 1476 ] {} CacheMetadata: Key a47c5f9d5635cf17f16141e7114bd213 is removed from metadata 2024.10.22 02:12:42.480622 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f5cdd8b25fbad9b02fd50e10b6b03daf, offset: 0, size: 719064 2024.10.22 02:12:42.487766 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f5cdd8b25fbad9b02fd50e10b6b03daf for removal 2024.10.22 02:12:42.500245 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ed62a9761553b77e99d8e597c4bd2b1d, offset: 0, size: 301258 2024.10.22 02:12:42.506959 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ed62a9761553b77e99d8e597c4bd2b1d for removal 2024.10.22 02:12:42.520243 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3c3798746caa58c8b5ed6be7060ee8f5, offset: 0, size: 491 2024.10.22 02:12:42.529205 [ 1476 ] {} CacheMetadata: Key ed62a9761553b77e99d8e597c4bd2b1d is removed from metadata 2024.10.22 02:12:42.532039 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3c3798746caa58c8b5ed6be7060ee8f5 for removal 2024.10.22 02:12:42.539517 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 905e3e1d29f3083d31db4f49c1a1f53c, offset: 0, size: 33231 2024.10.22 02:12:42.543153 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 905e3e1d29f3083d31db4f49c1a1f53c for removal 2024.10.22 02:12:42.543153 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 93228 2024.10.22 02:12:42.543245 [ 1476 ] {} CacheMetadata: Key 3c3798746caa58c8b5ed6be7060ee8f5 is removed from metadata 2024.10.22 02:12:42.648441 [ 1476 ] {} CacheMetadata: Key 905e3e1d29f3083d31db4f49c1a1f53c is removed from metadata 2024.10.22 02:12:42.656053 [ 1476 ] {} CacheMetadata: Key f5cdd8b25fbad9b02fd50e10b6b03daf is removed from metadata 2024.10.22 02:12:42.800465 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_785_174, 202410_786_786_0] 2024.10.22 02:12:42.804807 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_785_174, 202410_786_786_0] 2024.10.22 02:12:42.819478 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_785_174, 202410_786_786_0] 2024.10.22 02:12:43.078471 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_165_169_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:12:43.133497 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_845_218} MergeTask::MergeProjectionsStage: Merge sorted 12762 rows, containing 12 columns (12 merged, 0 gathered) in 1.133915677 sec., 11254.80514897229 rows/sec., 1.75 MiB/sec. 2024.10.22 02:12:43.171358 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_845_218} MergedBlockOutputStream: filled checksums 202410_1_845_218 (state Temporary) 2024.10.22 02:12:43.208895 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 197b8031d4cc8fd87597911fa521a5bb:0) 2024.10.22 02:12:43.213593 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 40178c05a655880f8eec986d5941facc, offset: 0, size: 291330 2024.10.22 02:12:43.220032 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 40178c05a655880f8eec986d5941facc for removal 2024.10.22 02:12:43.225065 [ 1476 ] {} CacheMetadata: Key 40178c05a655880f8eec986d5941facc is removed from metadata 2024.10.22 02:12:43.290016 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 10.75 MiB, will set to 1.58 GiB (RSS), difference: -10.10 MiB 2024.10.22 02:12:43.327089 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d15fca726044c7d679ec41fbe683d151, offset: 0, size: 492 2024.10.22 02:12:43.331048 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d15fca726044c7d679ec41fbe683d151 for removal 2024.10.22 02:12:43.335427 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 156de19e6f496d9e807bb5401dc2e872, offset: 0, size: 454085 2024.10.22 02:12:43.335394 [ 1476 ] {} CacheMetadata: Key d15fca726044c7d679ec41fbe683d151 is removed from metadata 2024.10.22 02:12:43.341299 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 156de19e6f496d9e807bb5401dc2e872 for removal 2024.10.22 02:12:43.345563 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b2c37bc48389005e1b5ba4da7ae928a3, offset: 0, size: 238 2024.10.22 02:12:43.345797 [ 1476 ] {} CacheMetadata: Key 156de19e6f496d9e807bb5401dc2e872 is removed from metadata 2024.10.22 02:12:43.350716 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b2c37bc48389005e1b5ba4da7ae928a3 for removal 2024.10.22 02:12:43.354083 [ 1476 ] {} CacheMetadata: Key b2c37bc48389005e1b5ba4da7ae928a3 is removed from metadata 2024.10.22 02:12:43.561388 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_845_218} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_845_218 to 202410_1_845_218 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:43.564909 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_845_218} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_844_217, 202410_845_845_0] -> 202410_1_845_218 2024.10.22 02:12:43.587376 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.45 MiB. 2024.10.22 02:12:43.899665 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 197b8031d4cc8fd87597911fa521a5bb:0) 2024.10.22 02:12:43.903542 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e4d1058499dfd756c68e2ba8d3bda587, offset: 0, size: 795561 2024.10.22 02:12:43.912205 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e4d1058499dfd756c68e2ba8d3bda587 for removal 2024.10.22 02:12:43.917023 [ 1476 ] {} CacheMetadata: Key e4d1058499dfd756c68e2ba8d3bda587 is removed from metadata 2024.10.22 02:12:43.918423 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 632648139c704dafee0f6fe2704b0a44, offset: 0, size: 247 2024.10.22 02:12:43.933960 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 632648139c704dafee0f6fe2704b0a44 for removal 2024.10.22 02:12:43.943284 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 47711766063be8893904b72166173843, offset: 0, size: 240 2024.10.22 02:12:43.951071 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 47711766063be8893904b72166173843 for removal 2024.10.22 02:12:43.955128 [ 1476 ] {} CacheMetadata: Key 632648139c704dafee0f6fe2704b0a44 is removed from metadata 2024.10.22 02:12:43.958339 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5184208c1857f347561916b3d18a5028, offset: 0, size: 373296 2024.10.22 02:12:43.962718 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5184208c1857f347561916b3d18a5028 for removal 2024.10.22 02:12:43.966731 [ 1476 ] {} CacheMetadata: Key 5184208c1857f347561916b3d18a5028 is removed from metadata 2024.10.22 02:12:43.970341 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2da9a0b368cc06c361b128d1a166426f, offset: 0, size: 238 2024.10.22 02:12:43.974265 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2da9a0b368cc06c361b128d1a166426f for removal 2024.10.22 02:12:43.976075 [ 1476 ] {} CacheMetadata: Key 47711766063be8893904b72166173843 is removed from metadata 2024.10.22 02:12:43.978193 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 27e783b1eb3d349d868176c0e9aca581, offset: 0, size: 451940 2024.10.22 02:12:43.980845 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 27e783b1eb3d349d868176c0e9aca581 for removal 2024.10.22 02:12:44.003665 [ 1476 ] {} CacheMetadata: Key 27e783b1eb3d349d868176c0e9aca581 is removed from metadata 2024.10.22 02:12:44.040458 [ 1476 ] {} CacheMetadata: Key 2da9a0b368cc06c361b128d1a166426f is removed from metadata 2024.10.22 02:12:44.461585 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 1048576 bytes for 197b8031d4cc8fd87597911fa521a5bb:0) 2024.10.22 02:12:44.464663 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dc51548f816652d585e36e1b420684b4, offset: 0, size: 490496 2024.10.22 02:12:44.468337 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dc51548f816652d585e36e1b420684b4 for removal 2024.10.22 02:12:44.470500 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: db7b1102a76f161e10292323ab2faf55, offset: 0, size: 867 2024.10.22 02:12:44.470572 [ 1476 ] {} CacheMetadata: Key dc51548f816652d585e36e1b420684b4 is removed from metadata 2024.10.22 02:12:44.476235 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key db7b1102a76f161e10292323ab2faf55 for removal 2024.10.22 02:12:44.478915 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f4510d80cc058c028eec70b29023bc49, offset: 0, size: 241 2024.10.22 02:12:44.486786 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f4510d80cc058c028eec70b29023bc49 for removal 2024.10.22 02:12:44.589552 [ 1476 ] {} CacheMetadata: Key f4510d80cc058c028eec70b29023bc49 is removed from metadata 2024.10.22 02:12:44.611496 [ 1476 ] {} CacheMetadata: Key db7b1102a76f161e10292323ab2faf55 is removed from metadata 2024.10.22 02:12:44.743725 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.28 MiB, will set to 1.58 GiB (RSS), difference: 3.96 MiB 2024.10.22 02:12:45.064455 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.098204252 sec. 2024.10.22 02:12:45.233725 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 115 entries to flush up to offset 91443 2024.10.22 02:12:45.312605 [ 1156 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 02:12:45.376212 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:45.382679 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:45.398710 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 840976 bytes for 197b8031d4cc8fd87597911fa521a5bb:0) 2024.10.22 02:12:45.403821 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 228c9e1375ccb068f726f0c52a9ce0fa, offset: 0, size: 1901163 2024.10.22 02:12:45.410650 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 228c9e1375ccb068f726f0c52a9ce0fa for removal 2024.10.22 02:12:45.438440 [ 1476 ] {} CacheMetadata: Key 228c9e1375ccb068f726f0c52a9ce0fa is removed from metadata 2024.10.22 02:12:45.571279 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_822_822_0 (state Temporary) 2024.10.22 02:12:45.745499 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_822_822_0 to 202410_828_828_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:45.994187 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 91443 2024.10.22 02:12:46.555570 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 120 entries to flush up to offset 93348 2024.10.22 02:12:46.773147 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_15_15_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:12:46.845026 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1206690 2024.10.22 02:12:47.161589 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 695 2024.10.22 02:12:47.252174 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/brk/egwwtoukviprcmhaccjrdzfdiwmwe. 2024.10.22 02:12:47.367874 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/brk/egwwtoukviprcmhaccjrdzfdiwmwe. 2024.10.22 02:12:47.407439 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:47.414661 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:47.629391 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:47.639166 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:47.732594 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1410_1410_0 (state Temporary) 2024.10.22 02:12:47.755014 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:47.760231 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:47.812640 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1410_1410_0 to 202410_1423_1423_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:47.818099 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_677_677_0 (state Temporary) 2024.10.22 02:12:47.880864 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 93348 2024.10.22 02:12:47.931363 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_801_801_0 (state Temporary) 2024.10.22 02:12:48.010370 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_801_801_0 to 202410_808_808_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:48.018623 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_677_677_0 to 202410_687_687_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:48.071787 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1206690 2024.10.22 02:12:48.087200 [ 1185 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_803_803_0 to 202410_808_808_0 2024.10.22 02:12:48.211981 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 695 2024.10.22 02:12:48.216344 [ 1185 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:48.240713 [ 1185 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:48.247144 [ 1185 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 02:12:48.255659 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_803_803_0 to 202410_808_808_0 into Compact with storage Full 2024.10.22 02:12:48.390176 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:48.410498 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_803_803_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:12:48.422757 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_802_151/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:12:48.440148 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_804_804_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:12:48.454903 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_805_805_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:12:48.468842 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_806_806_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:12:48.484408 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_807_807_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:12:48.512938 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_808_808_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:12:48.868100 [ 1264 ] {} DatabaseCatalog: Execution took 360 ms. 2024.10.22 02:12:49.470094 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:49.473679 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 12730 2024.10.22 02:12:49.575741 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:49.796195 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:49.800734 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:50.061534 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_837_837_0 (state Temporary) 2024.10.22 02:12:50.249694 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_837_837_0 to 202410_850_850_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:50.271989 [ 1269 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_845_218 to 202410_846_846_0 2024.10.22 02:12:50.296458 [ 1269 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:50.303352 [ 1269 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:50.304958 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12730 2024.10.22 02:12:50.375441 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} MergeTask::MergeProjectionsStage: Merge sorted 8855 rows, containing 4 columns (4 merged, 0 gathered) in 2.024570936 sec., 4373.7662348818785 rows/sec., 68.34 KiB/sec. 2024.10.22 02:12:50.436104 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_846_219} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_845_218 to 202410_846_846_0 into Compact with storage Full 2024.10.22 02:12:50.474758 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_846_219} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:50.489202 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_846_219} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_845_218, total 12762 rows starting from the beginning of the part 2024.10.22 02:12:50.516308 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_846_219} MergeTreeSequentialSource: Reading 2 marks from part 202410_846_846_0, total 22 rows starting from the beginning of the part 2024.10.22 02:12:50.565825 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} MergedBlockOutputStream: filled checksums 202410_803_808_1 (state Temporary) 2024.10.22 02:12:50.652912 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_803_808_1 to 202410_803_808_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:50.656073 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_803_808_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_803_803_0, 202410_808_808_0] -> 202410_803_808_1 2024.10.22 02:12:50.896367 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:12:51.092520 [ 1229 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:12:51.142743 [ 1229 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:12:51.262363 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:12:51.264434 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:12:51.443531 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_786_175, 202410_787_787_0] 2024.10.22 02:12:51.446713 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_786_175, 202410_787_787_0] 2024.10.22 02:12:51.453669 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_786_175, 202410_787_787_0] 2024.10.22 02:12:51.532285 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_846_219} MergeTask::MergeProjectionsStage: Merge sorted 12784 rows, containing 12 columns (12 merged, 0 gathered) in 1.111864098 sec., 11497.80807114432 rows/sec., 1.79 MiB/sec. 2024.10.22 02:12:51.638942 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_846_219} MergedBlockOutputStream: filled checksums 202410_1_846_219 (state Temporary) 2024.10.22 02:12:52.003983 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 93412 2024.10.22 02:12:52.250933 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_846_219} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_846_219 to 202410_1_846_219 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:52.259497 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_846_219} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_845_218, 202410_846_846_0] -> 202410_1_846_219 2024.10.22 02:12:52.276636 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.46 MiB. 2024.10.22 02:12:52.385555 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:52.394354 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:52.457679 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1411_1411_0 (state Temporary) 2024.10.22 02:12:52.554283 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1411_1411_0 to 202410_1424_1424_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:52.602163 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 93412 2024.10.22 02:12:53.319319 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_168_168_0 (state Temporary) 2024.10.22 02:12:53.677522 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 97 entries to flush up to offset 91540 2024.10.22 02:12:54.021487 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:54.026860 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:12:54.146481 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_823_823_0 (state Temporary) 2024.10.22 02:12:54.209705 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_823_823_0 to 202410_829_829_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:54.350105 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 91540 2024.10.22 02:12:54.584389 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 12.45 MiB, will set to 1.57 GiB (RSS), difference: 5.25 MiB 2024.10.22 02:12:54.748917 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fgz/jskzccfiorchjqwhpkvotonuaguwp. 2024.10.22 02:12:54.840801 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fgz/jskzccfiorchjqwhpkvotonuaguwp. 2024.10.22 02:12:55.076131 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1895 entries to flush up to offset 1208585 2024.10.22 02:12:55.242479 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fya/cczmzaqgfnidjzjfpqrpvncgnfpor. 2024.10.22 02:12:55.816738 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 696 2024.10.22 02:12:56.026649 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:56.032637 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:56.142449 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 63ee9bf5d16cc4b7293c2d3c556ac111:2097152) 2024.10.22 02:12:56.149412 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 75049e6421ae2509a45ed246496e5aa1, offset: 0, size: 1923873 2024.10.22 02:12:56.153030 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 75049e6421ae2509a45ed246496e5aa1 for removal 2024.10.22 02:12:56.157794 [ 1476 ] {} CacheMetadata: Key 75049e6421ae2509a45ed246496e5aa1 is removed from metadata 2024.10.22 02:12:56.158666 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0779807631af6435d5cd2dfda99da55a, offset: 0, size: 2468 2024.10.22 02:12:56.162316 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0779807631af6435d5cd2dfda99da55a for removal 2024.10.22 02:12:56.165731 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 334d3a9380aa6d51b92f6d4a1e758703, offset: 0, size: 255 2024.10.22 02:12:56.167979 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 334d3a9380aa6d51b92f6d4a1e758703 for removal 2024.10.22 02:12:56.169185 [ 1476 ] {} CacheMetadata: Key 0779807631af6435d5cd2dfda99da55a is removed from metadata 2024.10.22 02:12:56.170731 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d08d473094dd6636a2a01aec83f5b50e, offset: 0, size: 163 2024.10.22 02:12:56.173134 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d08d473094dd6636a2a01aec83f5b50e for removal 2024.10.22 02:12:56.176242 [ 1476 ] {} CacheMetadata: Key 334d3a9380aa6d51b92f6d4a1e758703 is removed from metadata 2024.10.22 02:12:56.189000 [ 1476 ] {} CacheMetadata: Key d08d473094dd6636a2a01aec83f5b50e is removed from metadata 2024.10.22 02:12:56.256622 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_802_802_0 (state Temporary) 2024.10.22 02:12:56.309569 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:56.315684 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:56.376170 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_678_678_0 (state Temporary) 2024.10.22 02:12:56.400050 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vao/dixvkvueutjsuygphjsxsuqexdgrl. 2024.10.22 02:12:56.430509 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_802_802_0 to 202410_809_809_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:56.476362 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_678_678_0 to 202410_688_688_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:56.479460 [ 1673 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zsx/hqbbbwqqipsspsqnecrmtxmzmaoqn, size 54 2024.10.22 02:12:56.514322 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1208585 2024.10.22 02:12:56.520018 [ 1267 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_683_139 to 202410_688_688_0 2024.10.22 02:12:56.547295 [ 1267 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:56.553019 [ 1267 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:56.565466 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vao/dixvkvueutjsuygphjsxsuqexdgrl. 2024.10.22 02:12:56.586311 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 696 2024.10.22 02:12:56.591578 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 12.49 MiB, will set to 1.57 GiB (RSS), difference: 8.71 MiB 2024.10.22 02:12:56.618740 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 93464 2024.10.22 02:12:56.703660 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_683_139 to 202410_688_688_0 into Compact with storage Full 2024.10.22 02:12:57.003536 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:57.007199 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:57.099621 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:57.136452 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_683_139, total 727 rows starting from the beginning of the part 2024.10.22 02:12:57.271911 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1412_1412_0 (state Temporary) 2024.10.22 02:12:57.312250 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column IsParameter 2024.10.22 02:12:57.354634 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column IsParameter 2024.10.22 02:12:57.472424 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column IsParameter 2024.10.22 02:12:57.512890 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column IsParameter 2024.10.22 02:12:57.582456 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column IsParameter 2024.10.22 02:12:57.657402 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_684_684_0, total 1 rows starting from the beginning of the part 2024.10.22 02:12:57.669485 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column IsParameter 2024.10.22 02:12:57.686204 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column IsParameter 2024.10.22 02:12:57.759605 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qtt/rankgpbowrstjhgumolrimhtwtgep (LogSeriesLimiter: on interval from 2024-10-22 02:12:42 to 2024-10-22 02:12:57 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:12:57.808459 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 12745 2024.10.22 02:12:57.853842 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1412_1412_0 to 202410_1425_1425_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:57.875183 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_685_685_0, total 1 rows starting from the beginning of the part 2024.10.22 02:12:57.915132 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 93464 2024.10.22 02:12:57.979778 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:57.988431 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:57.998953 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_686_686_0, total 1 rows starting from the beginning of the part 2024.10.22 02:12:58.108680 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_687_687_0, total 1 rows starting from the beginning of the part 2024.10.22 02:12:58.147115 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_838_838_0 (state Temporary) 2024.10.22 02:12:58.199685 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_688_688_0, total 1 rows starting from the beginning of the part 2024.10.22 02:12:58.216732 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_838_838_0 to 202410_851_851_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:58.239284 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 497c362e0eabd9dd8b5457ecb4ead15c:1048576) 2024.10.22 02:12:58.244895 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a31f15e82f7c611c1971940927d7f5c3, offset: 0, size: 1255277 2024.10.22 02:12:58.251857 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a31f15e82f7c611c1971940927d7f5c3 for removal 2024.10.22 02:12:58.256232 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e42bf36701b53a7e30bb4fbe712be3f7, offset: 0, size: 511 2024.10.22 02:12:58.258794 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e42bf36701b53a7e30bb4fbe712be3f7 for removal 2024.10.22 02:12:58.261621 [ 1476 ] {} CacheMetadata: Key a31f15e82f7c611c1971940927d7f5c3 is removed from metadata 2024.10.22 02:12:58.372142 [ 1476 ] {} CacheMetadata: Key e42bf36701b53a7e30bb4fbe712be3f7 is removed from metadata 2024.10.22 02:12:58.372983 [ 1182 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_846_219 to 202410_847_847_0 2024.10.22 02:12:58.389315 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12745 2024.10.22 02:12:58.480827 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 11.53 MiB, will set to 1.57 GiB (RSS), difference: -11.81 MiB 2024.10.22 02:12:58.484450 [ 1182 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:12:58.490608 [ 1182 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:12:58.522311 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_847_220} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_846_219 to 202410_847_847_0 into Compact with storage Full 2024.10.22 02:12:58.581574 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_847_220} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:12:58.596204 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_847_220} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_846_219, total 12784 rows starting from the beginning of the part 2024.10.22 02:12:58.629003 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mbv/gkkabnyekbznvxfjwxfxfetjdyllt. 2024.10.22 02:12:58.629574 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_847_220} MergeTreeSequentialSource: Reading 2 marks from part 202410_847_847_0, total 10 rows starting from the beginning of the part 2024.10.22 02:12:58.814058 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_787_176, 202410_788_788_0] 2024.10.22 02:12:58.819677 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_787_176, 202410_788_788_0] 2024.10.22 02:12:58.847166 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_787_176, 202410_788_788_0] 2024.10.22 02:12:58.850683 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mbv/gkkabnyekbznvxfjwxfxfetjdyllt. 2024.10.22 02:12:59.159155 [ 1242 ] {} DatabaseCatalog: Execution took 289 ms. 2024.10.22 02:12:59.186048 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_802_151/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:12:59.392180 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pto/czkxbkwzclmpsyijafzaxxbfoafqg. 2024.10.22 02:12:59.527562 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_847_220} MergeTask::MergeProjectionsStage: Merge sorted 12794 rows, containing 12 columns (12 merged, 0 gathered) in 1.026780278 sec., 12460.309448990021 rows/sec., 1.94 MiB/sec. 2024.10.22 02:12:59.583543 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_847_220} MergedBlockOutputStream: filled checksums 202410_1_847_220 (state Temporary) 2024.10.22 02:12:59.691703 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pto/czkxbkwzclmpsyijafzaxxbfoafqg. 2024.10.22 02:12:59.845825 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_847_220} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_847_220 to 202410_1_847_220 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:12:59.853311 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_847_220} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_846_219, 202410_847_847_0] -> 202410_1_847_220 2024.10.22 02:12:59.873261 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.45 MiB. 2024.10.22 02:13:00.483101 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} MergeTask::MergeProjectionsStage: Merge sorted 732 rows, containing 32 columns (32 merged, 0 gathered) in 3.813665542 sec., 191.9413204798545 rows/sec., 17.35 KiB/sec. 2024.10.22 02:13:00.575601 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1322_23, 202410_1323_1323_0, 202410_1324_1324_0, 202410_1325_1325_0, 202410_1326_1326_0, 202410_1327_1327_0] 2024.10.22 02:13:00.583346 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1322_23, 202410_1323_1323_0, 202410_1324_1324_0, 202410_1325_1325_0, 202410_1326_1326_0, 202410_1327_1327_0] 2024.10.22 02:13:00.615711 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1322_23, 202410_1323_1323_0, 202410_1324_1324_0, 202410_1325_1325_0, 202410_1326_1326_0, 202410_1327_1327_0] 2024.10.22 02:13:00.718652 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} MergedBlockOutputStream: filled checksums 202410_1_688_140 (state Temporary) 2024.10.22 02:13:00.966579 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_688_140 to 202410_1_688_140 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:00.971277 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_688_140} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_683_139, 202410_688_688_0] -> 202410_1_688_140 2024.10.22 02:13:01.041486 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fqq/trfifnmaugyjxqbexltabhjsedkwj. 2024.10.22 02:13:01.152910 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:13:01.262925 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:13:01.264920 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:13:01.287886 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fqq/trfifnmaugyjxqbexltabhjsedkwj. 2024.10.22 02:13:01.708100 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 11.39 MiB, will set to 1.57 GiB (RSS), difference: 33.24 MiB 2024.10.22 02:13:01.712912 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qqj/grperotgusvxrhcanoqcvnlqypnoo. 2024.10.22 02:13:01.859873 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 99 entries to flush up to offset 91639 2024.10.22 02:13:01.945781 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 93536 2024.10.22 02:13:02.079872 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_168_168_0 to 202410_171_171_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:02.103636 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:02.121381 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:02.125838 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qqj/grperotgusvxrhcanoqcvnlqypnoo. 2024.10.22 02:13:02.174846 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_824_824_0 (state Temporary) 2024.10.22 02:13:02.216238 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_165_169_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:13:02.246546 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_824_824_0 to 202410_830_830_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:02.287238 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 974b61f4d096d6fe80b2d655a95b6c06, offset: 0, size: 0 2024.10.22 02:13:02.290614 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 974b61f4d096d6fe80b2d655a95b6c06 for removal 2024.10.22 02:13:02.301493 [ 1476 ] {} CacheMetadata: Key 974b61f4d096d6fe80b2d655a95b6c06 is removed from metadata 2024.10.22 02:13:02.371005 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:02.375337 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:02.378745 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 91639 2024.10.22 02:13:02.366732 [ 1418 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): /var/lib/clickhouse/disks/s3_disk/store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/tmp_merge_201403_17_24_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:13:02.518565 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1413_1413_0 (state Temporary) 2024.10.22 02:13:02.562340 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 172fb7139d6e3e143efb1afe07549d26, offset: 0, size: 0 2024.10.22 02:13:02.563883 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 172fb7139d6e3e143efb1afe07549d26 for removal 2024.10.22 02:13:02.566540 [ 1476 ] {} CacheMetadata: Key 172fb7139d6e3e143efb1afe07549d26 is removed from metadata 2024.10.22 02:13:02.581275 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1413_1413_0 to 202410_1426_1426_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:02.621726 [ 1251 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1421_43 to 202410_1426_1426_0 2024.10.22 02:13:02.653534 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/wym/xfuksglqhcbeivciwqnnmbxlqlkdf. 2024.10.22 02:13:02.655872 [ 1251 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:02.659349 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:02.684092 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1421_43 to 202410_1426_1426_0 into Compact with storage Full 2024.10.22 02:13:02.750579 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 93536 2024.10.22 02:13:02.751531 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:02.760157 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1421_43, total 13963 rows starting from the beginning of the part 2024.10.22 02:13:02.786924 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e3788eea9ccef41e41514bd0c9c29169, offset: 0, size: 0 2024.10.22 02:13:02.788674 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_1422_1422_0, total 124 rows starting from the beginning of the part 2024.10.22 02:13:02.790906 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e3788eea9ccef41e41514bd0c9c29169 for removal 2024.10.22 02:13:02.797201 [ 1476 ] {} CacheMetadata: Key e3788eea9ccef41e41514bd0c9c29169 is removed from metadata 2024.10.22 02:13:02.851317 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6b826047edc63f9d5549aa90972b4c6e, offset: 0, size: 0 2024.10.22 02:13:02.853303 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6b826047edc63f9d5549aa90972b4c6e for removal 2024.10.22 02:13:02.855922 [ 1476 ] {} CacheMetadata: Key 6b826047edc63f9d5549aa90972b4c6e is removed from metadata 2024.10.22 02:13:02.871471 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/wym/xfuksglqhcbeivciwqnnmbxlqlkdf. 2024.10.22 02:13:02.929434 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 911c3483032fd75f5c7c25209c5323b8, offset: 0, size: 0 2024.10.22 02:13:02.932020 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 911c3483032fd75f5c7c25209c5323b8 for removal 2024.10.22 02:13:02.951942 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_1423_1423_0, total 120 rows starting from the beginning of the part 2024.10.22 02:13:02.985350 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_1424_1424_0, total 64 rows starting from the beginning of the part 2024.10.22 02:13:03.023497 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_1425_1425_0, total 52 rows starting from the beginning of the part 2024.10.22 02:13:03.044243 [ 1476 ] {} CacheMetadata: Key 911c3483032fd75f5c7c25209c5323b8 is removed from metadata 2024.10.22 02:13:03.064361 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_1426_1426_0, total 72 rows starting from the beginning of the part 2024.10.22 02:13:03.400221 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/agd/zvonotuuivvdipouvoqunrxokewxw. 2024.10.22 02:13:03.536186 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1185 entries to flush up to offset 1209770 2024.10.22 02:13:03.552492 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fd3266c1aeb09f2411a16e3692a3c00e, offset: 0, size: 0 2024.10.22 02:13:03.554953 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fd3266c1aeb09f2411a16e3692a3c00e for removal 2024.10.22 02:13:03.555957 [ 1476 ] {} CacheMetadata: Key fd3266c1aeb09f2411a16e3692a3c00e is removed from metadata 2024.10.22 02:13:03.631983 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 9.47 MiB, will set to 1.57 GiB (RSS), difference: 2.67 MiB 2024.10.22 02:13:03.649981 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9e33abb83d2454027c46cc358f792683, offset: 0, size: 0 2024.10.22 02:13:03.654751 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9e33abb83d2454027c46cc358f792683 for removal 2024.10.22 02:13:03.658769 [ 1476 ] {} CacheMetadata: Key 9e33abb83d2454027c46cc358f792683 is removed from metadata 2024.10.22 02:13:03.769010 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/agd/zvonotuuivvdipouvoqunrxokewxw. 2024.10.22 02:13:03.844216 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f217dd9667706836223d543a5ab5e1e3, offset: 0, size: 0 2024.10.22 02:13:03.846794 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f217dd9667706836223d543a5ab5e1e3 for removal 2024.10.22 02:13:03.856462 [ 1476 ] {} CacheMetadata: Key f217dd9667706836223d543a5ab5e1e3 is removed from metadata 2024.10.22 02:13:04.032471 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ca393420dcd0c5217cd0e6529781412a, offset: 0, size: 0 2024.10.22 02:13:04.036816 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ca393420dcd0c5217cd0e6529781412a for removal 2024.10.22 02:13:04.047271 [ 1476 ] {} CacheMetadata: Key ca393420dcd0c5217cd0e6529781412a is removed from metadata 2024.10.22 02:13:04.090916 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 697 2024.10.22 02:13:04.093056 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f2a6e42ac3976d69bb059fe8977bd3fe, offset: 0, size: 0 2024.10.22 02:13:04.099870 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f2a6e42ac3976d69bb059fe8977bd3fe for removal 2024.10.22 02:13:04.108877 [ 1476 ] {} CacheMetadata: Key f2a6e42ac3976d69bb059fe8977bd3fe is removed from metadata 2024.10.22 02:13:04.176838 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b1645db16dbb6ef6bea73b11d3130c25, offset: 0, size: 0 2024.10.22 02:13:04.180856 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b1645db16dbb6ef6bea73b11d3130c25 for removal 2024.10.22 02:13:04.190422 [ 1476 ] {} CacheMetadata: Key b1645db16dbb6ef6bea73b11d3130c25 is removed from metadata 2024.10.22 02:13:04.443366 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3ee93c3d7f2a68e09c8b550402ccbd20, offset: 0, size: 0 2024.10.22 02:13:04.445407 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3ee93c3d7f2a68e09c8b550402ccbd20 for removal 2024.10.22 02:13:04.448400 [ 1476 ] {} CacheMetadata: Key 3ee93c3d7f2a68e09c8b550402ccbd20 is removed from metadata 2024.10.22 02:13:04.496182 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4f90a366a2b3732ae42e3f9188ba69c5, offset: 0, size: 0 2024.10.22 02:13:04.499497 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4f90a366a2b3732ae42e3f9188ba69c5 for removal 2024.10.22 02:13:04.502612 [ 1476 ] {} CacheMetadata: Key 4f90a366a2b3732ae42e3f9188ba69c5 is removed from metadata 2024.10.22 02:13:04.787235 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2b3474e0d743bdbf768e353d11a1e29f, offset: 0, size: 0 2024.10.22 02:13:04.794080 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2b3474e0d743bdbf768e353d11a1e29f for removal 2024.10.22 02:13:04.803384 [ 1476 ] {} CacheMetadata: Key 2b3474e0d743bdbf768e353d11a1e29f is removed from metadata 2024.10.22 02:13:04.876482 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 524e4e2d7af5d3bf04da0f8a80bdae97, offset: 0, size: 0 2024.10.22 02:13:04.877812 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:04.883985 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 524e4e2d7af5d3bf04da0f8a80bdae97 for removal 2024.10.22 02:13:04.893588 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:04.895314 [ 1476 ] {} CacheMetadata: Key 524e4e2d7af5d3bf04da0f8a80bdae97 is removed from metadata 2024.10.22 02:13:04.951719 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} MergeTask::MergeProjectionsStage: Merge sorted 14395 rows, containing 13 columns (13 merged, 0 gathered) in 2.282155172 sec., 6307.634194472733 rows/sec., 1.25 MiB/sec. 2024.10.22 02:13:04.928784 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6769 2024.10.22 02:13:05.094843 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: babc8a118b8021aa985cd34c605af1da, offset: 0, size: 0 2024.10.22 02:13:05.099010 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_679_679_0 (state Temporary) 2024.10.22 02:13:05.099574 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key babc8a118b8021aa985cd34c605af1da for removal 2024.10.22 02:13:05.112439 [ 1476 ] {} CacheMetadata: Key babc8a118b8021aa985cd34c605af1da is removed from metadata 2024.10.22 02:13:05.202348 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ae2ad8a9784c137101a6e339c732f01b, offset: 0, size: 0 2024.10.22 02:13:05.209073 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} MergedBlockOutputStream: filled checksums 202410_1207_1426_44 (state Temporary) 2024.10.22 02:13:05.309089 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ae2ad8a9784c137101a6e339c732f01b for removal 2024.10.22 02:13:05.315568 [ 1476 ] {} CacheMetadata: Key ae2ad8a9784c137101a6e339c732f01b is removed from metadata 2024.10.22 02:13:05.317152 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:05.323965 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:05.450484 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_679_679_0 to 202410_689_689_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:05.457904 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_803_803_0 (state Temporary) 2024.10.22 02:13:05.466207 [ 1179 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 02:13:05.547397 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_803_803_0 to 202410_810_810_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:05.559862 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: eabc1d53c0f6f1eead2c3140c83cbfdc, offset: 0, size: 0 2024.10.22 02:13:05.561034 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 697 2024.10.22 02:13:05.561481 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key eabc1d53c0f6f1eead2c3140c83cbfdc for removal 2024.10.22 02:13:05.565496 [ 1476 ] {} CacheMetadata: Key eabc1d53c0f6f1eead2c3140c83cbfdc is removed from metadata 2024.10.22 02:13:05.600159 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1209770 2024.10.22 02:13:05.631241 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 75e86ed9d18a10bee91339be9f52b975, offset: 0, size: 0 2024.10.22 02:13:05.646948 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 75e86ed9d18a10bee91339be9f52b975 for removal 2024.10.22 02:13:05.693194 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1426_44 to 202410_1207_1426_44 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:05.697444 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1426_44} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1421_43, 202410_1426_1426_0] -> 202410_1207_1426_44 2024.10.22 02:13:05.742474 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 287e3ac0203014b87d4a231a4390035f, offset: 0, size: 0 2024.10.22 02:13:05.745587 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 287e3ac0203014b87d4a231a4390035f for removal 2024.10.22 02:13:05.653654 [ 1476 ] {} CacheMetadata: Key 75e86ed9d18a10bee91339be9f52b975 is removed from metadata 2024.10.22 02:13:05.758525 [ 1476 ] {} CacheMetadata: Key 287e3ac0203014b87d4a231a4390035f is removed from metadata 2024.10.22 02:13:05.802339 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 12.24 MiB, will set to 1.56 GiB (RSS), difference: 49.32 MiB 2024.10.22 02:13:05.863997 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.80 MiB. 2024.10.22 02:13:05.883055 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3376c4421f3130dee889637d34e76ed9, offset: 0, size: 0 2024.10.22 02:13:05.893475 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3376c4421f3130dee889637d34e76ed9 for removal 2024.10.22 02:13:05.893510 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 24 entries to flush up to offset 12769 2024.10.22 02:13:05.897511 [ 1476 ] {} CacheMetadata: Key 3376c4421f3130dee889637d34e76ed9 is removed from metadata 2024.10.22 02:13:05.900993 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/baq/gvgkqkhmtgmbtyncgcudvwaegawmt. 2024.10.22 02:13:06.028645 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:06.031984 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:06.051127 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ed97b2210e88c9ca922d67830321ede7, offset: 0, size: 0 2024.10.22 02:13:06.053338 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ed97b2210e88c9ca922d67830321ede7 for removal 2024.10.22 02:13:06.056486 [ 1476 ] {} CacheMetadata: Key ed97b2210e88c9ca922d67830321ede7 is removed from metadata 2024.10.22 02:13:06.075181 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_839_839_0 (state Temporary) 2024.10.22 02:13:06.148941 [ 1155 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:13:06.209062 [ 1155 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:13:06.223195 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_839_839_0 to 202410_852_852_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:06.313713 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c339e32872a077820cb26a6c0e9e544e, offset: 0, size: 0 2024.10.22 02:13:06.317322 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c339e32872a077820cb26a6c0e9e544e for removal 2024.10.22 02:13:06.325080 [ 1476 ] {} CacheMetadata: Key c339e32872a077820cb26a6c0e9e544e is removed from metadata 2024.10.22 02:13:06.336033 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/baq/gvgkqkhmtgmbtyncgcudvwaegawmt. 2024.10.22 02:13:06.372800 [ 1156 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_847_220 to 202410_848_848_0 2024.10.22 02:13:06.380544 [ 1156 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:06.390871 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:06.396128 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 08ab16db79c939be63d26b97e598835a, offset: 0, size: 0 2024.10.22 02:13:06.403201 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 08ab16db79c939be63d26b97e598835a for removal 2024.10.22 02:13:06.410788 [ 1476 ] {} CacheMetadata: Key 08ab16db79c939be63d26b97e598835a is removed from metadata 2024.10.22 02:13:06.512647 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12769 2024.10.22 02:13:06.575358 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: abca8b03a9234238a95cd16040dbd779, offset: 0, size: 0 2024.10.22 02:13:06.579641 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key abca8b03a9234238a95cd16040dbd779 for removal 2024.10.22 02:13:06.587461 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/qpy/oirdylbiomlsoqlytlqqlosufvtzx. 2024.10.22 02:13:06.587291 [ 1476 ] {} CacheMetadata: Key abca8b03a9234238a95cd16040dbd779 is removed from metadata 2024.10.22 02:13:06.655491 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9cad1cfa2b652edaadd345b7ae793e5c, offset: 0, size: 0 2024.10.22 02:13:06.657175 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9cad1cfa2b652edaadd345b7ae793e5c for removal 2024.10.22 02:13:06.661208 [ 1476 ] {} CacheMetadata: Key 9cad1cfa2b652edaadd345b7ae793e5c is removed from metadata 2024.10.22 02:13:06.715573 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2d79faf8d553d46888f05be38d4616f3, offset: 0, size: 0 2024.10.22 02:13:06.719856 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2d79faf8d553d46888f05be38d4616f3 for removal 2024.10.22 02:13:06.729615 [ 1476 ] {} CacheMetadata: Key 2d79faf8d553d46888f05be38d4616f3 is removed from metadata 2024.10.22 02:13:06.758881 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 146 entries to flush up to offset 93682 2024.10.22 02:13:06.926936 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/qpy/oirdylbiomlsoqlytlqqlosufvtzx. 2024.10.22 02:13:06.981746 [ 1156 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 625 ms. 2024.10.22 02:13:06.986531 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9dd08dac8d0e2401e3a5dd51adc0ed95, offset: 0, size: 0 2024.10.22 02:13:06.996768 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9dd08dac8d0e2401e3a5dd51adc0ed95 for removal 2024.10.22 02:13:07.002573 [ 1476 ] {} CacheMetadata: Key 9dd08dac8d0e2401e3a5dd51adc0ed95 is removed from metadata 2024.10.22 02:13:07.007498 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_848_221} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_847_220 to 202410_848_848_0 into Compact with storage Full 2024.10.22 02:13:07.054614 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a5efe52cc12a0e595b951b34badad8d5, offset: 0, size: 0 2024.10.22 02:13:07.056656 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a5efe52cc12a0e595b951b34badad8d5 for removal 2024.10.22 02:13:07.059144 [ 1476 ] {} CacheMetadata: Key a5efe52cc12a0e595b951b34badad8d5 is removed from metadata 2024.10.22 02:13:07.060143 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kfv/mmdzlqjgnpuevmpmrxuimybbzvrbv. 2024.10.22 02:13:07.146224 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_848_221} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:07.160090 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_848_221} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_847_220, total 12794 rows starting from the beginning of the part 2024.10.22 02:13:07.181461 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_788_177, 202410_789_789_0] 2024.10.22 02:13:07.185206 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_788_177, 202410_789_789_0] 2024.10.22 02:13:07.195364 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_788_177, 202410_789_789_0] 2024.10.22 02:13:07.205726 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_848_221} MergeTreeSequentialSource: Reading 2 marks from part 202410_848_848_0, total 12 rows starting from the beginning of the part 2024.10.22 02:13:07.224259 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5eb64c19f39046b07406b4d9b888b91d, offset: 0, size: 0 2024.10.22 02:13:07.226146 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5eb64c19f39046b07406b4d9b888b91d for removal 2024.10.22 02:13:07.231345 [ 1476 ] {} CacheMetadata: Key 5eb64c19f39046b07406b4d9b888b91d is removed from metadata 2024.10.22 02:13:07.276619 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:07.277311 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: de4f1b99ee9a92393f79a307bbf89006, offset: 0, size: 0 2024.10.22 02:13:07.280587 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key de4f1b99ee9a92393f79a307bbf89006 for removal 2024.10.22 02:13:07.280905 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:07.295593 [ 1476 ] {} CacheMetadata: Key de4f1b99ee9a92393f79a307bbf89006 is removed from metadata 2024.10.22 02:13:07.295866 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kfv/mmdzlqjgnpuevmpmrxuimybbzvrbv. 2024.10.22 02:13:07.368170 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1414_1414_0 (state Temporary) 2024.10.22 02:13:07.439015 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 37ce6fa815099f067a67c8fb2b5141f0, offset: 0, size: 0 2024.10.22 02:13:07.441255 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 37ce6fa815099f067a67c8fb2b5141f0 for removal 2024.10.22 02:13:07.447107 [ 1476 ] {} CacheMetadata: Key 37ce6fa815099f067a67c8fb2b5141f0 is removed from metadata 2024.10.22 02:13:07.495301 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 64e6e2cd967a82f2168ea13282cbd3ab, offset: 0, size: 0 2024.10.22 02:13:07.497400 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 64e6e2cd967a82f2168ea13282cbd3ab for removal 2024.10.22 02:13:07.562813 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1414_1414_0 to 202410_1427_1427_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:07.601247 [ 1476 ] {} CacheMetadata: Key 64e6e2cd967a82f2168ea13282cbd3ab is removed from metadata 2024.10.22 02:13:07.602517 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fcv/xsqdanfzpjtlujwaihmeejyifktgy. 2024.10.22 02:13:07.654332 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 93682 2024.10.22 02:13:07.655498 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ca9e2f6cf20fbf99eba3b97517f9f7d8, offset: 0, size: 0 2024.10.22 02:13:07.658356 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ca9e2f6cf20fbf99eba3b97517f9f7d8 for removal 2024.10.22 02:13:07.660982 [ 1476 ] {} CacheMetadata: Key ca9e2f6cf20fbf99eba3b97517f9f7d8 is removed from metadata 2024.10.22 02:13:07.756801 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 2.00 GiB, free memory in arenas 9.70 MiB, will set to 1.56 GiB (RSS), difference: 69.62 MiB 2024.10.22 02:13:07.810983 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_767_4, 202410_768_768_0, 202410_769_769_0, 202410_770_770_0, 202410_771_771_0, 202410_772_772_0] 2024.10.22 02:13:07.815658 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_767_4, 202410_768_768_0, 202410_769_769_0, 202410_770_770_0, 202410_771_771_0, 202410_772_772_0] 2024.10.22 02:13:07.853065 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 43bd04e34ac04fd94bb09592809e5171, offset: 0, size: 0 2024.10.22 02:13:07.856445 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 43bd04e34ac04fd94bb09592809e5171 for removal 2024.10.22 02:13:07.860546 [ 1476 ] {} CacheMetadata: Key 43bd04e34ac04fd94bb09592809e5171 is removed from metadata 2024.10.22 02:13:07.862907 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_767_4, 202410_768_768_0, 202410_769_769_0, 202410_770_770_0, 202410_771_771_0, 202410_772_772_0] 2024.10.22 02:13:07.871990 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fcv/xsqdanfzpjtlujwaihmeejyifktgy. 2024.10.22 02:13:08.025476 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e15f7466b57b672d527ba58148035faf, offset: 0, size: 0 2024.10.22 02:13:08.026961 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e15f7466b57b672d527ba58148035faf for removal 2024.10.22 02:13:08.029533 [ 1476 ] {} CacheMetadata: Key e15f7466b57b672d527ba58148035faf is removed from metadata 2024.10.22 02:13:08.165772 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1225c9d632daf4efc6341dfd75d09871, offset: 0, size: 0 2024.10.22 02:13:08.167176 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1225c9d632daf4efc6341dfd75d09871 for removal 2024.10.22 02:13:08.169328 [ 1476 ] {} CacheMetadata: Key 1225c9d632daf4efc6341dfd75d09871 is removed from metadata 2024.10.22 02:13:08.170227 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/svu/afqixrkjudeplvuqaqwpnhssbwtft. 2024.10.22 02:13:08.198690 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_848_221} MergeTask::MergeProjectionsStage: Merge sorted 12806 rows, containing 12 columns (12 merged, 0 gathered) in 1.212733479 sec., 10559.616124855082 rows/sec., 1.65 MiB/sec. 2024.10.22 02:13:08.205779 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5341f09889b634dcf7e1496a9baa706b, offset: 0, size: 0 2024.10.22 02:13:08.208386 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5341f09889b634dcf7e1496a9baa706b for removal 2024.10.22 02:13:08.215231 [ 1476 ] {} CacheMetadata: Key 5341f09889b634dcf7e1496a9baa706b is removed from metadata 2024.10.22 02:13:08.257359 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3a01a6e9862ee4d4ca66c2ec6d67d1ce, offset: 0, size: 0 2024.10.22 02:13:08.258602 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3a01a6e9862ee4d4ca66c2ec6d67d1ce for removal 2024.10.22 02:13:08.260209 [ 1476 ] {} CacheMetadata: Key 3a01a6e9862ee4d4ca66c2ec6d67d1ce is removed from metadata 2024.10.22 02:13:08.281341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fya/cczmzaqgfnidjzjfpqrpvncgnfpor. 2024.10.22 02:13:08.298862 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_802_151} MergeTask::MergeProjectionsStage: Merge sorted 1211425 rows, containing 4 columns (4 merged, 0 gathered) in 36.537387101 sec., 33155.76444071569 rows/sec., 497.21 KiB/sec. 2024.10.22 02:13:08.346637 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_802_151} MergedBlockOutputStream: filled checksums 202410_1_802_151 (state Temporary) 2024.10.22 02:13:08.358509 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_848_221} MergedBlockOutputStream: filled checksums 202410_1_848_221 (state Temporary) 2024.10.22 02:13:08.368487 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/svu/afqixrkjudeplvuqaqwpnhssbwtft. 2024.10.22 02:13:08.403373 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f845286c1e24cc032e6be7fbc2a5c6b1, offset: 0, size: 0 2024.10.22 02:13:08.405512 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f845286c1e24cc032e6be7fbc2a5c6b1 for removal 2024.10.22 02:13:08.409333 [ 1476 ] {} CacheMetadata: Key f845286c1e24cc032e6be7fbc2a5c6b1 is removed from metadata 2024.10.22 02:13:08.561411 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d869f12035091d3314ac847bab514e22, offset: 0, size: 0 2024.10.22 02:13:08.564068 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d869f12035091d3314ac847bab514e22 for removal 2024.10.22 02:13:08.566257 [ 1476 ] {} CacheMetadata: Key d869f12035091d3314ac847bab514e22 is removed from metadata 2024.10.22 02:13:08.567882 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xox/ldsyqaqljqmhaavuufjmogombczae. 2024.10.22 02:13:08.716337 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 537179d766582fb3aaea09113b12ab20, offset: 0, size: 0 2024.10.22 02:13:08.718644 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 537179d766582fb3aaea09113b12ab20 for removal 2024.10.22 02:13:08.723058 [ 1476 ] {} CacheMetadata: Key 537179d766582fb3aaea09113b12ab20 is removed from metadata 2024.10.22 02:13:08.928782 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 42b265dfee6b192487ab79adc2cb16a9, offset: 0, size: 0 2024.10.22 02:13:08.933286 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 42b265dfee6b192487ab79adc2cb16a9 for removal 2024.10.22 02:13:08.943024 [ 1476 ] {} CacheMetadata: Key 42b265dfee6b192487ab79adc2cb16a9 is removed from metadata 2024.10.22 02:13:09.007713 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_848_221} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_848_221 to 202410_1_848_221 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:09.011812 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_848_221} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_847_220, 202410_848_848_0] -> 202410_1_848_221 2024.10.22 02:13:09.043455 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.45 MiB. 2024.10.22 02:13:09.056124 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xox/ldsyqaqljqmhaavuufjmogombczae. 2024.10.22 02:13:09.263986 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: dc8b8f7f720c98946eae05d2c896deeb, offset: 0, size: 0 2024.10.22 02:13:09.266331 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key dc8b8f7f720c98946eae05d2c896deeb for removal 2024.10.22 02:13:09.270273 [ 1476 ] {} CacheMetadata: Key dc8b8f7f720c98946eae05d2c896deeb is removed from metadata 2024.10.22 02:13:09.274162 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/jcc/pqraihiyvimmabzvykrjunforelqk. 2024.10.22 02:13:09.346573 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 202414651a10896f92770f368be4c06e, offset: 0, size: 0 2024.10.22 02:13:09.348250 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 202414651a10896f92770f368be4c06e for removal 2024.10.22 02:13:09.351195 [ 1476 ] {} CacheMetadata: Key 202414651a10896f92770f368be4c06e is removed from metadata 2024.10.22 02:13:09.421276 [ 1235 ] {} DatabaseCatalog: Execution took 261 ms. 2024.10.22 02:13:09.510952 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 26ad8d56cec4cd01cc7bff00d848021f, offset: 0, size: 0 2024.10.22 02:13:09.512573 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 26ad8d56cec4cd01cc7bff00d848021f for removal 2024.10.22 02:13:09.519084 [ 1476 ] {} CacheMetadata: Key 26ad8d56cec4cd01cc7bff00d848021f is removed from metadata 2024.10.22 02:13:09.552843 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/jcc/pqraihiyvimmabzvykrjunforelqk. 2024.10.22 02:13:09.604678 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.45 GiB, peak 2.00 GiB, free memory in arenas 11.21 MiB, will set to 1.55 GiB (RSS), difference: 96.91 MiB 2024.10.22 02:13:09.726968 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ed3c5fc983fb77b3521847057998c497, offset: 0, size: 0 2024.10.22 02:13:09.729216 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ed3c5fc983fb77b3521847057998c497 for removal 2024.10.22 02:13:09.732822 [ 1476 ] {} CacheMetadata: Key ed3c5fc983fb77b3521847057998c497 is removed from metadata 2024.10.22 02:13:09.868339 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7d6f4b6cd6134cf4b68221d511ac00d9, offset: 0, size: 0 2024.10.22 02:13:09.870448 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7d6f4b6cd6134cf4b68221d511ac00d9 for removal 2024.10.22 02:13:09.872747 [ 1476 ] {} CacheMetadata: Key 7d6f4b6cd6134cf4b68221d511ac00d9 is removed from metadata 2024.10.22 02:13:09.900577 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 98 entries to flush up to offset 91737 2024.10.22 02:13:10.029798 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:10.038289 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:10.236794 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_825_825_0 (state Temporary) 2024.10.22 02:13:10.393545 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_825_825_0 to 202410_831_831_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:10.425106 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 1048576 bytes for ed3c5fc983fb77b3521847057998c497:0) 2024.10.22 02:13:10.430872 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f55b3c44e54efd5a8f12e8d79a305e54, offset: 0, size: 78562 2024.10.22 02:13:10.460550 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 91737 2024.10.22 02:13:10.537082 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f55b3c44e54efd5a8f12e8d79a305e54 for removal 2024.10.22 02:13:10.542229 [ 1476 ] {} CacheMetadata: Key f55b3c44e54efd5a8f12e8d79a305e54 is removed from metadata 2024.10.22 02:13:10.542226 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c4069b8b57aa721adaab3632db86e59e, offset: 0, size: 48592 2024.10.22 02:13:10.544900 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c4069b8b57aa721adaab3632db86e59e for removal 2024.10.22 02:13:10.546873 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 20df8552942f873fd4517c61fd3e3293, offset: 0, size: 622 2024.10.22 02:13:10.548938 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 20df8552942f873fd4517c61fd3e3293 for removal 2024.10.22 02:13:10.552137 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 73649320ffab89c834b9d06db785b729, offset: 0, size: 573 2024.10.22 02:13:10.553770 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 73649320ffab89c834b9d06db785b729 for removal 2024.10.22 02:13:10.555548 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0c15c7136aab67db17142b7d7bb4df24, offset: 0, size: 12608 2024.10.22 02:13:10.559040 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0c15c7136aab67db17142b7d7bb4df24 for removal 2024.10.22 02:13:10.564834 [ 1476 ] {} CacheMetadata: Key 0c15c7136aab67db17142b7d7bb4df24 is removed from metadata 2024.10.22 02:13:10.575687 [ 1476 ] {} CacheMetadata: Key 73649320ffab89c834b9d06db785b729 is removed from metadata 2024.10.22 02:13:10.588784 [ 1476 ] {} CacheMetadata: Key 20df8552942f873fd4517c61fd3e3293 is removed from metadata 2024.10.22 02:13:10.605355 [ 1476 ] {} CacheMetadata: Key c4069b8b57aa721adaab3632db86e59e is removed from metadata 2024.10.22 02:13:10.662614 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_802_151/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:13:10.822254 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_749_749_0, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0, 202410_753_753_0, 202410_754_754_0] 2024.10.22 02:13:10.826369 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_749_749_0, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0, 202410_753_753_0, 202410_754_754_0] 2024.10.22 02:13:10.860886 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_749_749_0, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0, 202410_753_753_0, 202410_754_754_0] 2024.10.22 02:13:11.051235 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 21 file segments (while reserving 1048576 bytes for ed3c5fc983fb77b3521847057998c497:0) 2024.10.22 02:13:11.055332 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d94808e59a608ada036befb73fc51843, offset: 0, size: 262 2024.10.22 02:13:11.058263 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d94808e59a608ada036befb73fc51843 for removal 2024.10.22 02:13:11.060125 [ 1476 ] {} CacheMetadata: Key d94808e59a608ada036befb73fc51843 is removed from metadata 2024.10.22 02:13:11.060273 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 425f60facf0ec3084f136a3ab661234a, offset: 0, size: 438 2024.10.22 02:13:11.063972 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 425f60facf0ec3084f136a3ab661234a for removal 2024.10.22 02:13:11.066710 [ 1476 ] {} CacheMetadata: Key 425f60facf0ec3084f136a3ab661234a is removed from metadata 2024.10.22 02:13:11.067057 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d9d81a548164ac01c185913aebfb9f61, offset: 0, size: 355 2024.10.22 02:13:11.078182 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d9d81a548164ac01c185913aebfb9f61 for removal 2024.10.22 02:13:11.082251 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 64a2492e716b2a49b91221977b9b87ee, offset: 0, size: 68 2024.10.22 02:13:11.091279 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 64a2492e716b2a49b91221977b9b87ee for removal 2024.10.22 02:13:11.097303 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 188840efbd124642a188e5c780e88f2b, offset: 0, size: 688 2024.10.22 02:13:11.101346 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 188840efbd124642a188e5c780e88f2b for removal 2024.10.22 02:13:11.111144 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c16d0e8b1ee79e47b78319163114ffd5, offset: 0, size: 36028 2024.10.22 02:13:11.116275 [ 1476 ] {} CacheMetadata: Key 188840efbd124642a188e5c780e88f2b is removed from metadata 2024.10.22 02:13:11.117760 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c16d0e8b1ee79e47b78319163114ffd5 for removal 2024.10.22 02:13:11.124267 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 86ea4b0dae121d534c151e3b47a92a7f, offset: 0, size: 142 2024.10.22 02:13:11.131318 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 86ea4b0dae121d534c151e3b47a92a7f for removal 2024.10.22 02:13:11.140264 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 618e59f61674539413064755c5f5df83, offset: 0, size: 136 2024.10.22 02:13:11.144164 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 618e59f61674539413064755c5f5df83 for removal 2024.10.22 02:13:11.144923 [ 1476 ] {} CacheMetadata: Key c16d0e8b1ee79e47b78319163114ffd5 is removed from metadata 2024.10.22 02:13:11.150508 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fbd2e06e9976c39be9745fbe7b00b35f, offset: 0, size: 401 2024.10.22 02:13:11.154592 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fbd2e06e9976c39be9745fbe7b00b35f for removal 2024.10.22 02:13:11.157542 [ 1476 ] {} CacheMetadata: Key fbd2e06e9976c39be9745fbe7b00b35f is removed from metadata 2024.10.22 02:13:11.158187 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b968d623b345ba3595fd035d340c01ec, offset: 0, size: 544 2024.10.22 02:13:11.161558 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b968d623b345ba3595fd035d340c01ec for removal 2024.10.22 02:13:11.163617 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 23ea390ea3992a193356dfe9fcdb7a37, offset: 0, size: 71 2024.10.22 02:13:11.163705 [ 1476 ] {} CacheMetadata: Key b968d623b345ba3595fd035d340c01ec is removed from metadata 2024.10.22 02:13:11.165712 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 23ea390ea3992a193356dfe9fcdb7a37 for removal 2024.10.22 02:13:11.168993 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6907c37e8dc208e64448d34d7b975f15, offset: 0, size: 154 2024.10.22 02:13:11.169280 [ 1476 ] {} CacheMetadata: Key 23ea390ea3992a193356dfe9fcdb7a37 is removed from metadata 2024.10.22 02:13:11.171126 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6907c37e8dc208e64448d34d7b975f15 for removal 2024.10.22 02:13:11.173181 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 13411c39a7e52ef831819d165c0f9a0d, offset: 0, size: 445 2024.10.22 02:13:11.175185 [ 1476 ] {} CacheMetadata: Key 6907c37e8dc208e64448d34d7b975f15 is removed from metadata 2024.10.22 02:13:11.176198 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 13411c39a7e52ef831819d165c0f9a0d for removal 2024.10.22 02:13:11.179119 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 40d91b3cf883fae717c36e245f21c2eb, offset: 0, size: 2300177 2024.10.22 02:13:11.181545 [ 1476 ] {} CacheMetadata: Key 13411c39a7e52ef831819d165c0f9a0d is removed from metadata 2024.10.22 02:13:11.187369 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 40d91b3cf883fae717c36e245f21c2eb for removal 2024.10.22 02:13:11.191305 [ 1476 ] {} CacheMetadata: Key 618e59f61674539413064755c5f5df83 is removed from metadata 2024.10.22 02:13:11.194513 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8931c47fb6ef5b8a29d6c0b44ab7aa30, offset: 0, size: 25243 2024.10.22 02:13:11.199540 [ 1476 ] {} CacheMetadata: Key 40d91b3cf883fae717c36e245f21c2eb is removed from metadata 2024.10.22 02:13:11.199885 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8931c47fb6ef5b8a29d6c0b44ab7aa30 for removal 2024.10.22 02:13:11.203821 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6ce14267f7b69054e80d81cb6ffd3101, offset: 0, size: 440 2024.10.22 02:13:11.210614 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6ce14267f7b69054e80d81cb6ffd3101 for removal 2024.10.22 02:13:11.213149 [ 1476 ] {} CacheMetadata: Key 8931c47fb6ef5b8a29d6c0b44ab7aa30 is removed from metadata 2024.10.22 02:13:11.216573 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2210195faca1b9b3c7e6feeeb81e6075, offset: 0, size: 438 2024.10.22 02:13:11.220155 [ 1476 ] {} CacheMetadata: Key 6ce14267f7b69054e80d81cb6ffd3101 is removed from metadata 2024.10.22 02:13:11.220611 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2210195faca1b9b3c7e6feeeb81e6075 for removal 2024.10.22 02:13:11.228609 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 247806481a0c5a1b0f6dad804abdde1f, offset: 0, size: 660 2024.10.22 02:13:11.230616 [ 1476 ] {} CacheMetadata: Key 2210195faca1b9b3c7e6feeeb81e6075 is removed from metadata 2024.10.22 02:13:11.234767 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 247806481a0c5a1b0f6dad804abdde1f for removal 2024.10.22 02:13:11.242061 [ 1476 ] {} CacheMetadata: Key 86ea4b0dae121d534c151e3b47a92a7f is removed from metadata 2024.10.22 02:13:11.248278 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6f098524f50442e77ee64b10bddd718f, offset: 0, size: 540 2024.10.22 02:13:11.248677 [ 1476 ] {} CacheMetadata: Key 247806481a0c5a1b0f6dad804abdde1f is removed from metadata 2024.10.22 02:13:11.251611 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6f098524f50442e77ee64b10bddd718f for removal 2024.10.22 02:13:11.255495 [ 1476 ] {} CacheMetadata: Key 6f098524f50442e77ee64b10bddd718f is removed from metadata 2024.10.22 02:13:11.255985 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 358382d56c4b8aea4a37a395210e7540, offset: 0, size: 323 2024.10.22 02:13:11.260235 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 358382d56c4b8aea4a37a395210e7540 for removal 2024.10.22 02:13:11.261721 [ 1476 ] {} CacheMetadata: Key 64a2492e716b2a49b91221977b9b87ee is removed from metadata 2024.10.22 02:13:11.264571 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:13:11.266870 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6b4d9224858c1c7a3d96f9e49a4c8b72, offset: 0, size: 438 2024.10.22 02:13:11.268756 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:13:11.270954 [ 1476 ] {} CacheMetadata: Key 358382d56c4b8aea4a37a395210e7540 is removed from metadata 2024.10.22 02:13:11.270953 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6b4d9224858c1c7a3d96f9e49a4c8b72 for removal 2024.10.22 02:13:11.300504 [ 1476 ] {} CacheMetadata: Key 6b4d9224858c1c7a3d96f9e49a4c8b72 is removed from metadata 2024.10.22 02:13:11.314336 [ 1476 ] {} CacheMetadata: Key d9d81a548164ac01c185913aebfb9f61 is removed from metadata 2024.10.22 02:13:11.540302 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_802_151} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_802_151 to 202410_1_802_151 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:11.547639 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_802_151} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_796_150, 202410_797_802_1] -> 202410_1_802_151 2024.10.22 02:13:11.559704 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:13:11.616193 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 9.59 MiB, will set to 1.55 GiB (RSS), difference: 19.75 MiB 2024.10.22 02:13:11.680075 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 198 entries to flush up to offset 93880 2024.10.22 02:13:11.711673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for b3f0929e12b36be7481722bbb192018d:0) 2024.10.22 02:13:11.715908 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 747de55ef14e065537240817b74d03ff, offset: 0, size: 249 2024.10.22 02:13:11.721574 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 747de55ef14e065537240817b74d03ff for removal 2024.10.22 02:13:11.725259 [ 1476 ] {} CacheMetadata: Key 747de55ef14e065537240817b74d03ff is removed from metadata 2024.10.22 02:13:11.727497 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e6d9900edf8b9343d7ea34db41d388bc, offset: 0, size: 883088 2024.10.22 02:13:11.733085 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e6d9900edf8b9343d7ea34db41d388bc for removal 2024.10.22 02:13:11.737105 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e83736d078e3073fa5d4371d90162ff4, offset: 0, size: 670068 2024.10.22 02:13:11.737106 [ 1476 ] {} CacheMetadata: Key e6d9900edf8b9343d7ea34db41d388bc is removed from metadata 2024.10.22 02:13:11.741410 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e83736d078e3073fa5d4371d90162ff4 for removal 2024.10.22 02:13:11.743433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 257321d5e7af4e1f6c3b8aa0195f5918, offset: 0, size: 633 2024.10.22 02:13:11.744996 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 257321d5e7af4e1f6c3b8aa0195f5918 for removal 2024.10.22 02:13:11.771412 [ 1476 ] {} CacheMetadata: Key 257321d5e7af4e1f6c3b8aa0195f5918 is removed from metadata 2024.10.22 02:13:11.779374 [ 1476 ] {} CacheMetadata: Key e83736d078e3073fa5d4371d90162ff4 is removed from metadata 2024.10.22 02:13:11.903241 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for ed3c5fc983fb77b3521847057998c497:0) 2024.10.22 02:13:11.908799 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e92a51f7cbd183b911faf451db733549, offset: 0, size: 900459 2024.10.22 02:13:11.916785 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e92a51f7cbd183b911faf451db733549 for removal 2024.10.22 02:13:11.924123 [ 1476 ] {} CacheMetadata: Key e92a51f7cbd183b911faf451db733549 is removed from metadata 2024.10.22 02:13:11.926123 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: df157de9f7babe97767710714308be65, offset: 0, size: 505 2024.10.22 02:13:11.931323 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key df157de9f7babe97767710714308be65 for removal 2024.10.22 02:13:11.940243 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9560c2650f1cfd6d64f1c8ed05a8663b, offset: 0, size: 207943 2024.10.22 02:13:11.941529 [ 1476 ] {} CacheMetadata: Key df157de9f7babe97767710714308be65 is removed from metadata 2024.10.22 02:13:11.947069 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9560c2650f1cfd6d64f1c8ed05a8663b for removal 2024.10.22 02:13:11.953706 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b52f677b90b0f24df16429b2c336af83, offset: 0, size: 170 2024.10.22 02:13:11.958327 [ 1476 ] {} CacheMetadata: Key 9560c2650f1cfd6d64f1c8ed05a8663b is removed from metadata 2024.10.22 02:13:11.959190 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b52f677b90b0f24df16429b2c336af83 for removal 2024.10.22 02:13:11.967518 [ 1476 ] {} CacheMetadata: Key b52f677b90b0f24df16429b2c336af83 is removed from metadata 2024.10.22 02:13:12.276816 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:12.280679 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:12.373089 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1415_1415_0 (state Temporary) 2024.10.22 02:13:12.442378 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1415_1415_0 to 202410_1428_1428_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:12.481252 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for ed3c5fc983fb77b3521847057998c497:0) 2024.10.22 02:13:12.486164 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 3a009de3d5821f9639ed3193caf89aa4, offset: 0, size: 1486806 2024.10.22 02:13:12.494548 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 3a009de3d5821f9639ed3193caf89aa4 for removal 2024.10.22 02:13:12.498417 [ 1476 ] {} CacheMetadata: Key 3a009de3d5821f9639ed3193caf89aa4 is removed from metadata 2024.10.22 02:13:12.499707 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 46f5ab324c71650c46104464f092d870, offset: 0, size: 648 2024.10.22 02:13:12.506312 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 46f5ab324c71650c46104464f092d870 for removal 2024.10.22 02:13:12.510452 [ 1476 ] {} CacheMetadata: Key 46f5ab324c71650c46104464f092d870 is removed from metadata 2024.10.22 02:13:12.557546 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 6811 2024.10.22 02:13:12.609086 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1211695 2024.10.22 02:13:12.930423 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 93880 2024.10.22 02:13:12.973225 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for ed3c5fc983fb77b3521847057998c497:0) 2024.10.22 02:13:12.992552 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 94025f4a68c64eca351c2ef17bb93212, offset: 0, size: 1648659 2024.10.22 02:13:12.997470 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 94025f4a68c64eca351c2ef17bb93212 for removal 2024.10.22 02:13:13.000585 [ 1476 ] {} CacheMetadata: Key 94025f4a68c64eca351c2ef17bb93212 is removed from metadata 2024.10.22 02:13:13.002085 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8c4ea26bbfaed64da73da3792b48a04b, offset: 0, size: 650 2024.10.22 02:13:13.007716 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8c4ea26bbfaed64da73da3792b48a04b for removal 2024.10.22 02:13:13.063994 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 698 2024.10.22 02:13:13.113605 [ 1476 ] {} CacheMetadata: Key 8c4ea26bbfaed64da73da3792b48a04b is removed from metadata 2024.10.22 02:13:13.359447 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 57bb7d739ed1faf96c36085f030167c9:0) 2024.10.22 02:13:13.361788 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 41baa0e522ccd28c615f56e41060325e, offset: 0, size: 880218 2024.10.22 02:13:13.366394 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 41baa0e522ccd28c615f56e41060325e for removal 2024.10.22 02:13:13.367751 [ 1476 ] {} CacheMetadata: Key 41baa0e522ccd28c615f56e41060325e is removed from metadata 2024.10.22 02:13:13.369450 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2cf5b4cceb65005feb8446a0b08b7e4b, offset: 0, size: 638 2024.10.22 02:13:13.372154 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2cf5b4cceb65005feb8446a0b08b7e4b for removal 2024.10.22 02:13:13.374381 [ 1476 ] {} CacheMetadata: Key 2cf5b4cceb65005feb8446a0b08b7e4b is removed from metadata 2024.10.22 02:13:13.431873 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:13.434979 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:13.497515 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for ed3c5fc983fb77b3521847057998c497:0) 2024.10.22 02:13:13.500723 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:13.506391 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: eb57a587309c9d5d6b6f68ee27aa181a, offset: 0, size: 10329294 2024.10.22 02:13:13.514966 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:13.527259 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key eb57a587309c9d5d6b6f68ee27aa181a for removal 2024.10.22 02:13:13.531525 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 50cce7bddc13e44fa300d0ec9181d60e, offset: 0, size: 981 2024.10.22 02:13:13.539008 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 50cce7bddc13e44fa300d0ec9181d60e for removal 2024.10.22 02:13:13.633534 [ 1476 ] {} CacheMetadata: Key eb57a587309c9d5d6b6f68ee27aa181a is removed from metadata 2024.10.22 02:13:13.644435 [ 1476 ] {} CacheMetadata: Key 50cce7bddc13e44fa300d0ec9181d60e is removed from metadata 2024.10.22 02:13:13.658400 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_804_804_0 (state Temporary) 2024.10.22 02:13:13.676553 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_680_680_0 (state Temporary) 2024.10.22 02:13:13.736487 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_804_804_0 to 202410_811_811_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:13.760265 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_680_680_0 to 202410_690_690_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:13.764286 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1211695 2024.10.22 02:13:13.766933 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 8.18 MiB, will set to 1.54 GiB (RSS), difference: -3.23 MiB 2024.10.22 02:13:13.848161 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ksg/brnctonrigceoeohhiiyzrrhlbmea. 2024.10.22 02:13:13.927334 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 698 2024.10.22 02:13:14.005410 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 59d7f6e13394049dc663985c4824fa28, offset: 0, size: 0 2024.10.22 02:13:14.007394 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 59d7f6e13394049dc663985c4824fa28 for removal 2024.10.22 02:13:14.012013 [ 1476 ] {} CacheMetadata: Key 59d7f6e13394049dc663985c4824fa28 is removed from metadata 2024.10.22 02:13:14.015941 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 12781 2024.10.22 02:13:14.119612 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:14.128433 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:14.268568 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b26836c3e2d35a4b51ad218eb13d6fe2, offset: 0, size: 0 2024.10.22 02:13:14.271824 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b26836c3e2d35a4b51ad218eb13d6fe2 for removal 2024.10.22 02:13:14.272231 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_840_840_0 (state Temporary) 2024.10.22 02:13:14.274770 [ 1476 ] {} CacheMetadata: Key b26836c3e2d35a4b51ad218eb13d6fe2 is removed from metadata 2024.10.22 02:13:14.284875 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ksg/brnctonrigceoeohhiiyzrrhlbmea. 2024.10.22 02:13:14.435713 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_840_840_0 to 202410_853_853_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:14.458981 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_848_221 to 202410_849_849_0 2024.10.22 02:13:14.466900 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:14.469510 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:14.475826 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12781 2024.10.22 02:13:14.515634 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_849_222} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_848_221 to 202410_849_849_0 into Compact with storage Full 2024.10.22 02:13:14.706711 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_849_222} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:14.726323 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_849_222} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_848_221, total 12806 rows starting from the beginning of the part 2024.10.22 02:13:14.760444 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_849_222} MergeTreeSequentialSource: Reading 2 marks from part 202410_849_849_0, total 21 rows starting from the beginning of the part 2024.10.22 02:13:15.253578 [ 1710 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qtt/rankgpbowrstjhgumolrimhtwtgep, size 70074 2024.10.22 02:13:15.644886 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column DontCountHits 2024.10.22 02:13:15.662027 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column DontCountHits 2024.10.22 02:13:15.652102 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_849_222} MergeTask::MergeProjectionsStage: Merge sorted 12827 rows, containing 12 columns (12 merged, 0 gathered) in 1.160537345 sec., 11052.63872400418 rows/sec., 1.72 MiB/sec. 2024.10.22 02:13:15.676725 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column DontCountHits 2024.10.22 02:13:15.707374 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column DontCountHits 2024.10.22 02:13:15.735787 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column DontCountHits 2024.10.22 02:13:15.741938 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_849_222} MergedBlockOutputStream: filled checksums 202410_1_849_222 (state Temporary) 2024.10.22 02:13:15.748499 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column DontCountHits 2024.10.22 02:13:15.771467 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column DontCountHits 2024.10.22 02:13:15.815800 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/eoz/sgtvrctioarwhwbewgezpowlxrrcw (LogSeriesLimiter: on interval from 2024-10-22 02:12:57 to 2024-10-22 02:13:15 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:13:15.927793 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_849_222} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_849_222 to 202410_1_849_222 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:16.001840 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 59d7f6e13394049dc663985c4824fa28:0) 2024.10.22 02:13:16.006204 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 43e36fe9063f4a160b7186b6548005ba, offset: 0, size: 433028 2024.10.22 02:13:16.009385 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 43e36fe9063f4a160b7186b6548005ba for removal 2024.10.22 02:13:16.011297 [ 1476 ] {} CacheMetadata: Key 43e36fe9063f4a160b7186b6548005ba is removed from metadata 2024.10.22 02:13:16.011641 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 841c6ba4504b8760a79901a4639eada8, offset: 0, size: 495 2024.10.22 02:13:16.016726 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 841c6ba4504b8760a79901a4639eada8 for removal 2024.10.22 02:13:16.019413 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6d1f6e259eeb129818313861baf4cd20, offset: 0, size: 495866 2024.10.22 02:13:16.021425 [ 1476 ] {} CacheMetadata: Key 841c6ba4504b8760a79901a4639eada8 is removed from metadata 2024.10.22 02:13:16.022460 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6d1f6e259eeb129818313861baf4cd20 for removal 2024.10.22 02:13:16.028229 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 765e7a6dc95b33e280db7f02a2cc45e7, offset: 0, size: 496 2024.10.22 02:13:16.031193 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 765e7a6dc95b33e280db7f02a2cc45e7 for removal 2024.10.22 02:13:16.031563 [ 1476 ] {} CacheMetadata: Key 6d1f6e259eeb129818313861baf4cd20 is removed from metadata 2024.10.22 02:13:16.036210 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_849_222} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_848_221, 202410_849_849_0] -> 202410_1_849_222 2024.10.22 02:13:16.047002 [ 1476 ] {} CacheMetadata: Key 765e7a6dc95b33e280db7f02a2cc45e7 is removed from metadata 2024.10.22 02:13:16.067356 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.46 MiB. 2024.10.22 02:13:16.426996 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_789_178, 202410_790_790_0] 2024.10.22 02:13:16.428960 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for 59d7f6e13394049dc663985c4824fa28:0) 2024.10.22 02:13:16.431122 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_789_178, 202410_790_790_0] 2024.10.22 02:13:16.432466 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cf89f1f402501bfbc1f3539e1e95d5d2, offset: 0, size: 1256199 2024.10.22 02:13:16.451820 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cf89f1f402501bfbc1f3539e1e95d5d2 for removal 2024.10.22 02:13:16.454145 [ 1476 ] {} CacheMetadata: Key cf89f1f402501bfbc1f3539e1e95d5d2 is removed from metadata 2024.10.22 02:13:16.454529 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: e2a2970d43f3a71749170bb380a16fd9, offset: 0, size: 847 2024.10.22 02:13:16.457254 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key e2a2970d43f3a71749170bb380a16fd9 for removal 2024.10.22 02:13:16.541118 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_789_178, 202410_790_790_0] 2024.10.22 02:13:16.560513 [ 1476 ] {} CacheMetadata: Key e2a2970d43f3a71749170bb380a16fd9 is removed from metadata 2024.10.22 02:13:16.561283 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0a587d3c9177ec3e605c58c6e72f7ae1, offset: 0, size: 811946 2024.10.22 02:13:16.566784 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0a587d3c9177ec3e605c58c6e72f7ae1 for removal 2024.10.22 02:13:16.570687 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c3738ae5e1f45a9dacd27a5d3c8d85f7, offset: 0, size: 833 2024.10.22 02:13:16.571964 [ 1476 ] {} CacheMetadata: Key 0a587d3c9177ec3e605c58c6e72f7ae1 is removed from metadata 2024.10.22 02:13:16.573567 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c3738ae5e1f45a9dacd27a5d3c8d85f7 for removal 2024.10.22 02:13:16.609531 [ 1476 ] {} CacheMetadata: Key c3738ae5e1f45a9dacd27a5d3c8d85f7 is removed from metadata 2024.10.22 02:13:16.859248 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_165_169_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:13:16.865474 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_165_169_1} MergeTask::MergeProjectionsStage: Merge sorted 218 rows, containing 679 columns (679 merged, 0 gathered) in 84.211078748 sec., 2.5887330175683974 rows/sec., 3.82 KiB/sec. 2024.10.22 02:13:16.927403 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 78679 bytes for d601d27b25d4a6d31ff433635d37c41b:0) 2024.10.22 02:13:16.939662 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a45b46d3310a8ce7f855eff301d72a50, offset: 0, size: 869764 2024.10.22 02:13:16.958233 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 141 entries to flush up to offset 94021 2024.10.22 02:13:16.960088 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a45b46d3310a8ce7f855eff301d72a50 for removal 2024.10.22 02:13:16.964401 [ 1476 ] {} CacheMetadata: Key a45b46d3310a8ce7f855eff301d72a50 is removed from metadata 2024.10.22 02:13:16.964626 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 96aa6068289f2cd27e4a0e67dab366c2, offset: 0, size: 505 2024.10.22 02:13:16.970824 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 96aa6068289f2cd27e4a0e67dab366c2 for removal 2024.10.22 02:13:16.974589 [ 1476 ] {} CacheMetadata: Key 96aa6068289f2cd27e4a0e67dab366c2 is removed from metadata 2024.10.22 02:13:17.271617 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 59d7f6e13394049dc663985c4824fa28:0) 2024.10.22 02:13:17.276181 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9b115df86358e9199e28f4e246808406, offset: 0, size: 2621248 2024.10.22 02:13:17.308425 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9b115df86358e9199e28f4e246808406 for removal 2024.10.22 02:13:17.315469 [ 1476 ] {} CacheMetadata: Key 9b115df86358e9199e28f4e246808406 is removed from metadata 2024.10.22 02:13:17.315769 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1d7a79f63a949cc66c29613a5807dc99, offset: 0, size: 821 2024.10.22 02:13:17.320797 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1d7a79f63a949cc66c29613a5807dc99 for removal 2024.10.22 02:13:17.328298 [ 1476 ] {} CacheMetadata: Key 1d7a79f63a949cc66c29613a5807dc99 is removed from metadata 2024.10.22 02:13:17.519231 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:17.526053 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:17.608250 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1416_1416_0 (state Temporary) 2024.10.22 02:13:17.669803 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1416_1416_0 to 202410_1429_1429_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:17.725558 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94021 2024.10.22 02:13:17.799268 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 4.66 MiB, will set to 1.54 GiB (RSS), difference: 1.41 MiB 2024.10.22 02:13:17.806428 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 982f51aa958ece5823fdc2196e0e7d52, offset: 0, size: 0 2024.10.22 02:13:17.808394 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 982f51aa958ece5823fdc2196e0e7d52 for removal 2024.10.22 02:13:17.811705 [ 1476 ] {} CacheMetadata: Key 982f51aa958ece5823fdc2196e0e7d52 is removed from metadata 2024.10.22 02:13:17.836069 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_14_14_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:13:17.968903 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: f55d542d803c311f486819585b1e9b30, offset: 0, size: 0 2024.10.22 02:13:17.970880 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 75 entries to flush up to offset 91812 2024.10.22 02:13:17.971112 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key f55d542d803c311f486819585b1e9b30 for removal 2024.10.22 02:13:17.974287 [ 1476 ] {} CacheMetadata: Key f55d542d803c311f486819585b1e9b30 is removed from metadata 2024.10.22 02:13:17.975339 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ouv/zqzpxrmixbybpbkwyavgwlhisnogf. 2024.10.22 02:13:18.062198 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b1109b990b61c12ee188fe12776e61d5, offset: 0, size: 0 2024.10.22 02:13:18.063447 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b1109b990b61c12ee188fe12776e61d5 for removal 2024.10.22 02:13:18.068065 [ 1476 ] {} CacheMetadata: Key b1109b990b61c12ee188fe12776e61d5 is removed from metadata 2024.10.22 02:13:18.105455 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 69487ef70545e960a39cc2d75d020c09, offset: 0, size: 0 2024.10.22 02:13:18.108079 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 69487ef70545e960a39cc2d75d020c09 for removal 2024.10.22 02:13:18.110533 [ 1476 ] {} CacheMetadata: Key 69487ef70545e960a39cc2d75d020c09 is removed from metadata 2024.10.22 02:13:18.128551 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ouv/zqzpxrmixbybpbkwyavgwlhisnogf. 2024.10.22 02:13:18.199645 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:18.200938 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:18.249627 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_826_826_0 (state Temporary) 2024.10.22 02:13:18.257826 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d224758c8c0afca919b47ca3d8d0fb3e, offset: 0, size: 0 2024.10.22 02:13:18.259575 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d224758c8c0afca919b47ca3d8d0fb3e for removal 2024.10.22 02:13:18.262501 [ 1476 ] {} CacheMetadata: Key d224758c8c0afca919b47ca3d8d0fb3e is removed from metadata 2024.10.22 02:13:18.308505 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a21f419d1b14dddca517dda61f7bbd5a, offset: 0, size: 0 2024.10.22 02:13:18.312496 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a21f419d1b14dddca517dda61f7bbd5a for removal 2024.10.22 02:13:18.320185 [ 1476 ] {} CacheMetadata: Key a21f419d1b14dddca517dda61f7bbd5a is removed from metadata 2024.10.22 02:13:18.319467 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fko/zxdguxigfjxwjtrzyekycphwkybcr. 2024.10.22 02:13:18.344177 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_826_826_0 to 202410_832_832_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:18.354742 [ 1390 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_165_169_1} MergedBlockOutputStream: filled checksums 202410_165_169_1 (state Temporary) 2024.10.22 02:13:18.370390 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 6b7a886b8f02cc9fa206f703e8c6192c, offset: 0, size: 0 2024.10.22 02:13:18.372800 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 6b7a886b8f02cc9fa206f703e8c6192c for removal 2024.10.22 02:13:18.386843 [ 1259 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_827_16 to 202410_832_832_0 2024.10.22 02:13:18.402646 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 91812 2024.10.22 02:13:18.407592 [ 1259 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:18.413080 [ 1259 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:18.524069 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 071320489c6a35bd7699238e23399b37, offset: 0, size: 0 2024.10.22 02:13:18.531887 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 071320489c6a35bd7699238e23399b37 for removal 2024.10.22 02:13:18.537211 [ 1476 ] {} CacheMetadata: Key 6b7a886b8f02cc9fa206f703e8c6192c is removed from metadata 2024.10.22 02:13:18.553186 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_827_16 to 202410_832_832_0 into Compact with storage Full 2024.10.22 02:13:18.640587 [ 1476 ] {} CacheMetadata: Key 071320489c6a35bd7699238e23399b37 is removed from metadata 2024.10.22 02:13:18.650562 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fko/zxdguxigfjxwjtrzyekycphwkybcr. 2024.10.22 02:13:18.702930 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:18.712587 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_827_16, total 7973 rows starting from the beginning of the part 2024.10.22 02:13:18.742570 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_828_828_0, total 115 rows starting from the beginning of the part 2024.10.22 02:13:18.775134 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_829_829_0, total 97 rows starting from the beginning of the part 2024.10.22 02:13:18.819541 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_830_830_0, total 99 rows starting from the beginning of the part 2024.10.22 02:13:18.857552 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_831_831_0, total 98 rows starting from the beginning of the part 2024.10.22 02:13:18.889494 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 445904fdd952fd1b3f0d0112c9b321d0, offset: 0, size: 0 2024.10.22 02:13:18.897179 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} MergeTreeSequentialSource: Reading 2 marks from part 202410_832_832_0, total 75 rows starting from the beginning of the part 2024.10.22 02:13:18.996133 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 445904fdd952fd1b3f0d0112c9b321d0 for removal 2024.10.22 02:13:19.002693 [ 1476 ] {} CacheMetadata: Key 445904fdd952fd1b3f0d0112c9b321d0 is removed from metadata 2024.10.22 02:13:19.148388 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 2c6327ed4299cd2ad358240b960503e7, offset: 0, size: 0 2024.10.22 02:13:19.150501 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 2c6327ed4299cd2ad358240b960503e7 for removal 2024.10.22 02:13:19.152307 [ 1476 ] {} CacheMetadata: Key 2c6327ed4299cd2ad358240b960503e7 is removed from metadata 2024.10.22 02:13:19.152920 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/aif/ffscdlgarrairpcpmuobjkpywqmpb. 2024.10.22 02:13:19.182534 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7b64bb48e8731ad1b6f3f1afb99b1cc7, offset: 0, size: 0 2024.10.22 02:13:19.184015 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7b64bb48e8731ad1b6f3f1afb99b1cc7 for removal 2024.10.22 02:13:19.188275 [ 1476 ] {} CacheMetadata: Key 7b64bb48e8731ad1b6f3f1afb99b1cc7 is removed from metadata 2024.10.22 02:13:19.249288 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4ae8c71ca649ce0a8d198968cf4a0bf6, offset: 0, size: 0 2024.10.22 02:13:19.252304 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4ae8c71ca649ce0a8d198968cf4a0bf6 for removal 2024.10.22 02:13:19.255691 [ 1476 ] {} CacheMetadata: Key 4ae8c71ca649ce0a8d198968cf4a0bf6 is removed from metadata 2024.10.22 02:13:19.264652 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/aif/ffscdlgarrairpcpmuobjkpywqmpb. 2024.10.22 02:13:19.355791 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 4.82 MiB, will set to 1.54 GiB (RSS), difference: 1.06 MiB 2024.10.22 02:13:19.512736 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 22c8d0b56e1492a61d1de4ebd63ca86f, offset: 0, size: 0 2024.10.22 02:13:19.515163 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 22c8d0b56e1492a61d1de4ebd63ca86f for removal 2024.10.22 02:13:19.525373 [ 1476 ] {} CacheMetadata: Key 22c8d0b56e1492a61d1de4ebd63ca86f is removed from metadata 2024.10.22 02:13:19.557818 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7dadebb03892dc885f458696db38bfa0, offset: 0, size: 0 2024.10.22 02:13:19.559641 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7dadebb03892dc885f458696db38bfa0 for removal 2024.10.22 02:13:19.560953 [ 1476 ] {} CacheMetadata: Key 7dadebb03892dc885f458696db38bfa0 is removed from metadata 2024.10.22 02:13:19.607115 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 99693e6b820f7023bc1470827234a386, offset: 0, size: 0 2024.10.22 02:13:19.611801 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 99693e6b820f7023bc1470827234a386 for removal 2024.10.22 02:13:19.616781 [ 1476 ] {} CacheMetadata: Key 99693e6b820f7023bc1470827234a386 is removed from metadata 2024.10.22 02:13:19.658946 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c8cf284c1c7a182aaae198c49e765034, offset: 0, size: 0 2024.10.22 02:13:19.661710 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c8cf284c1c7a182aaae198c49e765034 for removal 2024.10.22 02:13:19.666943 [ 1476 ] {} CacheMetadata: Key c8cf284c1c7a182aaae198c49e765034 is removed from metadata 2024.10.22 02:13:19.732625 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 158da33bc5348cfb5d4dea48b95a29ed, offset: 0, size: 0 2024.10.22 02:13:19.735212 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 158da33bc5348cfb5d4dea48b95a29ed for removal 2024.10.22 02:13:19.750256 [ 1476 ] {} CacheMetadata: Key 158da33bc5348cfb5d4dea48b95a29ed is removed from metadata 2024.10.22 02:13:19.899385 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8bfc3182818aaf3294b4f8e6353548a5, offset: 0, size: 0 2024.10.22 02:13:19.901425 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8bfc3182818aaf3294b4f8e6353548a5 for removal 2024.10.22 02:13:19.904598 [ 1476 ] {} CacheMetadata: Key 8bfc3182818aaf3294b4f8e6353548a5 is removed from metadata 2024.10.22 02:13:19.904812 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/idh/wtjdhlvzkwbhrdycquoedqhanujtp. 2024.10.22 02:13:19.904944 [ 1244 ] {} DatabaseCatalog: Execution took 480 ms. 2024.10.22 02:13:19.956408 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 9cde6c63b0891173004b3f6ff09ce3e3, offset: 0, size: 0 2024.10.22 02:13:19.958837 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 9cde6c63b0891173004b3f6ff09ce3e3 for removal 2024.10.22 02:13:19.963351 [ 1476 ] {} CacheMetadata: Key 9cde6c63b0891173004b3f6ff09ce3e3 is removed from metadata 2024.10.22 02:13:19.995058 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c912855c504c101be42e57e4e54baa09, offset: 0, size: 0 2024.10.22 02:13:19.996860 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c912855c504c101be42e57e4e54baa09 for removal 2024.10.22 02:13:19.999759 [ 1476 ] {} CacheMetadata: Key c912855c504c101be42e57e4e54baa09 is removed from metadata 2024.10.22 02:13:20.019010 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/idh/wtjdhlvzkwbhrdycquoedqhanujtp. 2024.10.22 02:13:20.194515 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7192a021262a722a5e21e3bb39641993, offset: 0, size: 0 2024.10.22 02:13:20.200279 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7192a021262a722a5e21e3bb39641993 for removal 2024.10.22 02:13:20.202620 [ 1476 ] {} CacheMetadata: Key 7192a021262a722a5e21e3bb39641993 is removed from metadata 2024.10.22 02:13:20.560179 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 4f6e13f943fbe08dba9f2ae5ac75cee6, offset: 0, size: 0 2024.10.22 02:13:20.561504 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 4f6e13f943fbe08dba9f2ae5ac75cee6 for removal 2024.10.22 02:13:20.568279 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/rco/aucjajhfmgeqczfldlxfybwzilnyt. 2024.10.22 02:13:20.568745 [ 1476 ] {} CacheMetadata: Key 4f6e13f943fbe08dba9f2ae5ac75cee6 is removed from metadata 2024.10.22 02:13:20.646094 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: be4ad479f453141f3b6a20a4adb59d0c, offset: 0, size: 0 2024.10.22 02:13:20.657700 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key be4ad479f453141f3b6a20a4adb59d0c for removal 2024.10.22 02:13:20.660446 [ 1476 ] {} CacheMetadata: Key be4ad479f453141f3b6a20a4adb59d0c is removed from metadata 2024.10.22 02:13:20.770467 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1213235 2024.10.22 02:13:20.873814 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 26d4cd4913470ab4340d601fbbab4c7e, offset: 0, size: 0 2024.10.22 02:13:20.875724 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 26d4cd4913470ab4340d601fbbab4c7e for removal 2024.10.22 02:13:20.881421 [ 1476 ] {} CacheMetadata: Key 26d4cd4913470ab4340d601fbbab4c7e is removed from metadata 2024.10.22 02:13:20.951739 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/rco/aucjajhfmgeqczfldlxfybwzilnyt. 2024.10.22 02:13:21.020737 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} MergeTask::MergeProjectionsStage: Merge sorted 8457 rows, containing 13 columns (13 merged, 0 gathered) in 2.472910459 sec., 3419.856941937096 rows/sec., 1.48 MiB/sec. 2024.10.22 02:13:21.056096 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} MergedBlockOutputStream: filled checksums 202410_747_832_17 (state Temporary) 2024.10.22 02:13:21.192878 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: effa0a8575e1817ff0fb1a331f1c80ee, offset: 0, size: 0 2024.10.22 02:13:21.196254 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key effa0a8575e1817ff0fb1a331f1c80ee for removal 2024.10.22 02:13:21.201309 [ 1476 ] {} CacheMetadata: Key effa0a8575e1817ff0fb1a331f1c80ee is removed from metadata 2024.10.22 02:13:21.202732 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/njs/mfcewlufsigbfixowmpmqfkwppjnl. 2024.10.22 02:13:21.214280 [ 1232 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:13:21.254924 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7d30a2650fb4ed2506c5c4f382de8067, offset: 0, size: 0 2024.10.22 02:13:21.256875 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7d30a2650fb4ed2506c5c4f382de8067 for removal 2024.10.22 02:13:21.276570 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:13:21.278732 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_832_17 to 202410_747_832_17 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:21.280887 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:13:21.283085 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_832_17} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_827_16, 202410_832_832_0] -> 202410_747_832_17 2024.10.22 02:13:21.324947 [ 1476 ] {} CacheMetadata: Key 7d30a2650fb4ed2506c5c4f382de8067 is removed from metadata 2024.10.22 02:13:21.347471 [ 1232 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:13:21.415604 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 295570838d3db2f529a6e889b6f695d2, offset: 0, size: 0 2024.10.22 02:13:21.418680 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 295570838d3db2f529a6e889b6f695d2 for removal 2024.10.22 02:13:21.424423 [ 1476 ] {} CacheMetadata: Key 295570838d3db2f529a6e889b6f695d2 is removed from metadata 2024.10.22 02:13:21.555563 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} S3ObjectStorage: Objects with paths [common/rco/aucjajhfmgeqczfldlxfybwzilnyt] were removed from S3 2024.10.22 02:13:21.573085 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 3.09 MiB, will set to 1.54 GiB (RSS), difference: 8.93 MiB 2024.10.22 02:13:21.573316 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.18 MiB. 2024.10.22 02:13:21.592557 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1a83d9930e011d9489f15d66c9b83aba, offset: 0, size: 0 2024.10.22 02:13:21.594754 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1a83d9930e011d9489f15d66c9b83aba for removal 2024.10.22 02:13:21.599258 [ 1476 ] {} CacheMetadata: Key 1a83d9930e011d9489f15d66c9b83aba is removed from metadata 2024.10.22 02:13:21.726494 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:21.748285 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 148 entries to flush up to offset 94169 2024.10.22 02:13:21.751582 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} S3ObjectStorage: Objects with paths [common/liy/ctqcimmftcnoikcnhqkeqelkqwqxv] were removed from S3 2024.10.22 02:13:21.760712 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:21.774787 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/njs/mfcewlufsigbfixowmpmqfkwppjnl. 2024.10.22 02:13:21.870429 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1a150f66c4a6576129f351f646efeea4, offset: 0, size: 0 2024.10.22 02:13:21.872170 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1a150f66c4a6576129f351f646efeea4 for removal 2024.10.22 02:13:21.881566 [ 1476 ] {} CacheMetadata: Key 1a150f66c4a6576129f351f646efeea4 is removed from metadata 2024.10.22 02:13:21.978470 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 12794 2024.10.22 02:13:22.028073 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_805_805_0 (state Temporary) 2024.10.22 02:13:22.143983 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: cce833800d0365dff389c2ace19cabf4, offset: 0, size: 0 2024.10.22 02:13:22.146918 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key cce833800d0365dff389c2ace19cabf4 for removal 2024.10.22 02:13:22.149707 [ 1476 ] {} CacheMetadata: Key cce833800d0365dff389c2ace19cabf4 is removed from metadata 2024.10.22 02:13:22.191974 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ecdc2f899cbb9c16b05a74b96beb1a72, offset: 0, size: 0 2024.10.22 02:13:22.208311 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ecdc2f899cbb9c16b05a74b96beb1a72 for removal 2024.10.22 02:13:22.216647 [ 1476 ] {} CacheMetadata: Key ecdc2f899cbb9c16b05a74b96beb1a72 is removed from metadata 2024.10.22 02:13:22.227116 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:22.237207 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:22.252534 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} S3ObjectStorage: Objects with paths [common/mbn/qhxexlekllxsjpuesvllpwckfxqzl] were removed from S3 2024.10.22 02:13:22.256774 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_805_805_0 to 202410_812_812_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:22.280397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 237175 bytes for 2cc50a9a095d7b0f3d50c33026ef44c2:0) 2024.10.22 02:13:22.284407 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1259eab84f22c693f60df9e067fc4d44, offset: 0, size: 9572285 2024.10.22 02:13:22.300737 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1259eab84f22c693f60df9e067fc4d44 for removal 2024.10.22 02:13:22.306983 [ 1476 ] {} CacheMetadata: Key 1259eab84f22c693f60df9e067fc4d44 is removed from metadata 2024.10.22 02:13:22.308209 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3c0a59f3f81267b981d7a4484fc453e5, offset: 0, size: 975 2024.10.22 02:13:22.316132 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3c0a59f3f81267b981d7a4484fc453e5 for removal 2024.10.22 02:13:22.320086 [ 1476 ] {} CacheMetadata: Key 3c0a59f3f81267b981d7a4484fc453e5 is removed from metadata 2024.10.22 02:13:22.387174 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_841_841_0 (state Temporary) 2024.10.22 02:13:22.418162 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1213235 2024.10.22 02:13:22.422295 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 79035f84b076ec44f727c9361fd25d80, offset: 0, size: 0 2024.10.22 02:13:22.401675 [ 1150 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_802_151 to 202410_803_808_1 2024.10.22 02:13:22.447946 [ 1150 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:22.451276 [ 1150 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:22.471309 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_808_152} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_802_151 to 202410_803_808_1 into Wide with storage Full 2024.10.22 02:13:22.487905 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_808_152} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:22.498268 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_808_152} MergeTreeSequentialSource: Reading 149 marks from part 202410_1_802_151, total 1211425 rows starting from the beginning of the part 2024.10.22 02:13:22.531380 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 79035f84b076ec44f727c9361fd25d80 for removal 2024.10.22 02:13:22.560051 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_808_152} MergeTreeSequentialSource: Reading 3 marks from part 202410_803_808_1, total 8855 rows starting from the beginning of the part 2024.10.22 02:13:22.569833 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_841_841_0 to 202410_854_854_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:22.631763 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:22.639485 [ 1476 ] {} CacheMetadata: Key 79035f84b076ec44f727c9361fd25d80 is removed from metadata 2024.10.22 02:13:22.666318 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} S3ObjectStorage: Objects with paths [common/noh/jrskhtwgxajxsiublorehygpdcjvq] were removed from S3 2024.10.22 02:13:22.731302 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:22.812784 [ 1167 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_849_222 to 202410_850_850_0 2024.10.22 02:13:22.827444 [ 1167 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:22.836237 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:22.889158 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12794 2024.10.22 02:13:22.927639 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1417_1417_0 (state Temporary) 2024.10.22 02:13:22.953692 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 43f2e27ee77abb066acc73ce74177e40, offset: 0, size: 0 2024.10.22 02:13:22.960690 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 43f2e27ee77abb066acc73ce74177e40 for removal 2024.10.22 02:13:22.967048 [ 1476 ] {} CacheMetadata: Key 43f2e27ee77abb066acc73ce74177e40 is removed from metadata 2024.10.22 02:13:22.967467 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_850_223} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_849_222 to 202410_850_850_0 into Compact with storage Full 2024.10.22 02:13:23.075458 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1417_1417_0 to 202410_1430_1430_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:23.099001 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: c10982cf8dde3622665bf46facf3bcf0, offset: 0, size: 0 2024.10.22 02:13:23.103304 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key c10982cf8dde3622665bf46facf3bcf0 for removal 2024.10.22 02:13:23.107054 [ 1476 ] {} CacheMetadata: Key c10982cf8dde3622665bf46facf3bcf0 is removed from metadata 2024.10.22 02:13:23.124270 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_850_223} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:23.132611 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_850_223} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_849_222, total 12827 rows starting from the beginning of the part 2024.10.22 02:13:23.148941 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94169 2024.10.22 02:13:23.154387 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_850_223} MergeTreeSequentialSource: Reading 2 marks from part 202410_850_850_0, total 9 rows starting from the beginning of the part 2024.10.22 02:13:23.181197 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: a6fb733ea04a128cf230a5dce80a8785, offset: 0, size: 0 2024.10.22 02:13:23.191286 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key a6fb733ea04a128cf230a5dce80a8785 for removal 2024.10.22 02:13:23.197347 [ 1476 ] {} CacheMetadata: Key a6fb733ea04a128cf230a5dce80a8785 is removed from metadata 2024.10.22 02:13:23.433429 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} S3ObjectStorage: Objects with paths [common/ypk/asdpzeaamqiwfwutijpfzwjoztish] were removed from S3 2024.10.22 02:13:23.670419 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 2.76 MiB, will set to 1.54 GiB (RSS), difference: -15.16 MiB 2024.10.22 02:13:23.751054 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: eeea61b3ab48ccd8c4319dd3ba8607e8, offset: 0, size: 0 2024.10.22 02:13:23.752763 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key eeea61b3ab48ccd8c4319dd3ba8607e8 for removal 2024.10.22 02:13:23.754904 [ 1476 ] {} CacheMetadata: Key eeea61b3ab48ccd8c4319dd3ba8607e8 is removed from metadata 2024.10.22 02:13:23.776573 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} S3ObjectStorage: Objects with paths [common/ctk/aobbkvulsdfrpiyqgkowxczropeex] were removed from S3 2024.10.22 02:13:23.846458 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ec85081fc1371c498e1d5a8e3777c639, offset: 0, size: 0 2024.10.22 02:13:23.848360 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ec85081fc1371c498e1d5a8e3777c639 for removal 2024.10.22 02:13:23.851282 [ 1476 ] {} CacheMetadata: Key ec85081fc1371c498e1d5a8e3777c639 is removed from metadata 2024.10.22 02:13:23.881277 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b07fe82cc34ac381db91cd5ad4b46127, offset: 0, size: 0 2024.10.22 02:13:23.883376 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b07fe82cc34ac381db91cd5ad4b46127 for removal 2024.10.22 02:13:23.887761 [ 1476 ] {} CacheMetadata: Key b07fe82cc34ac381db91cd5ad4b46127 is removed from metadata 2024.10.22 02:13:23.892136 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/phm/vfykellvelazdywlknhkpefcfuybn. 2024.10.22 02:13:23.944930 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_790_179, 202410_791_791_0] 2024.10.22 02:13:23.948350 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_790_179, 202410_791_791_0] 2024.10.22 02:13:23.953037 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 1cef721a4bb0af14ab229b485e0758c3, offset: 0, size: 0 2024.10.22 02:13:23.955070 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 1cef721a4bb0af14ab229b485e0758c3 for removal 2024.10.22 02:13:23.956697 [ 1476 ] {} CacheMetadata: Key 1cef721a4bb0af14ab229b485e0758c3 is removed from metadata 2024.10.22 02:13:23.970229 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_790_179, 202410_791_791_0] 2024.10.22 02:13:24.071946 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} S3ObjectStorage: Objects with paths [common/jsi/qljcwixxgfkiguvdgthusmqeqmkjd] were removed from S3 2024.10.22 02:13:24.169345 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_850_223} MergeTask::MergeProjectionsStage: Merge sorted 12836 rows, containing 12 columns (12 merged, 0 gathered) in 1.209948386 sec., 10608.716990346065 rows/sec., 1.65 MiB/sec. 2024.10.22 02:13:24.239390 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d8dbbb2d37047102f59dbedc439cf01e, offset: 0, size: 0 2024.10.22 02:13:24.242040 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d8dbbb2d37047102f59dbedc439cf01e for removal 2024.10.22 02:13:24.244990 [ 1476 ] {} CacheMetadata: Key d8dbbb2d37047102f59dbedc439cf01e is removed from metadata 2024.10.22 02:13:24.270423 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} S3ObjectStorage: Objects with paths [common/dpr/tczlhweduhnummyqkzychyxzukvxo] were removed from S3 2024.10.22 02:13:24.391028 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/phm/vfykellvelazdywlknhkpefcfuybn. 2024.10.22 02:13:24.432246 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_850_223} MergedBlockOutputStream: filled checksums 202410_1_850_223 (state Temporary) 2024.10.22 02:13:24.474723 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 211593ca6bcf205729a7c16a879e921e, offset: 0, size: 0 2024.10.22 02:13:24.476885 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 211593ca6bcf205729a7c16a879e921e for removal 2024.10.22 02:13:24.482700 [ 1476 ] {} CacheMetadata: Key 211593ca6bcf205729a7c16a879e921e is removed from metadata 2024.10.22 02:13:24.843576 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 38cabb814b99b87a2fd8a70947666a3b, offset: 0, size: 0 2024.10.22 02:13:24.845301 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 38cabb814b99b87a2fd8a70947666a3b for removal 2024.10.22 02:13:24.848909 [ 1476 ] {} CacheMetadata: Key 38cabb814b99b87a2fd8a70947666a3b is removed from metadata 2024.10.22 02:13:24.985804 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 7642fe22b6cd460164f2cfbd6a9102c6, offset: 0, size: 0 2024.10.22 02:13:24.992061 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 7642fe22b6cd460164f2cfbd6a9102c6 for removal 2024.10.22 02:13:24.998647 [ 1476 ] {} CacheMetadata: Key 7642fe22b6cd460164f2cfbd6a9102c6 is removed from metadata 2024.10.22 02:13:25.036849 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} S3ObjectStorage: Objects with paths [common/phm/vfykellvelazdywlknhkpefcfuybn] were removed from S3 2024.10.22 02:13:25.089629 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 0cc095798131343869a0ca0a8fe068f3, offset: 0, size: 0 2024.10.22 02:13:25.092480 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 0cc095798131343869a0ca0a8fe068f3 for removal 2024.10.22 02:13:25.096339 [ 1476 ] {} CacheMetadata: Key 0cc095798131343869a0ca0a8fe068f3 is removed from metadata 2024.10.22 02:13:25.125119 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} S3ObjectStorage: Objects with paths [common/mpx/cyunxiyjppyrfqqpjjkiqmfnrmlfk] were removed from S3 2024.10.22 02:13:25.147745 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/syd/jgsfdgcknijtquhlfqwifrzmhqfwu. 2024.10.22 02:13:25.167846 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_850_223} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_850_223 to 202410_1_850_223 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:25.182846 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b3c63db6f609fa7d1dfd8b5b371312c2, offset: 0, size: 0 2024.10.22 02:13:25.186520 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b3c63db6f609fa7d1dfd8b5b371312c2 for removal 2024.10.22 02:13:25.193322 [ 1476 ] {} CacheMetadata: Key b3c63db6f609fa7d1dfd8b5b371312c2 is removed from metadata 2024.10.22 02:13:25.283390 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_850_223} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_849_222, 202410_850_850_0] -> 202410_1_850_223 2024.10.22 02:13:25.342767 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.45 MiB. 2024.10.22 02:13:25.359408 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: fc2a9f15dffa804d881554dca1bee8e2, offset: 0, size: 0 2024.10.22 02:13:25.366737 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key fc2a9f15dffa804d881554dca1bee8e2 for removal 2024.10.22 02:13:25.375712 [ 1476 ] {} CacheMetadata: Key fc2a9f15dffa804d881554dca1bee8e2 is removed from metadata 2024.10.22 02:13:25.386894 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/syd/jgsfdgcknijtquhlfqwifrzmhqfwu. 2024.10.22 02:13:25.393433 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nvs/gcovvqfodomvdpbaqmlvudxijguwy. 2024.10.22 02:13:25.446780 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: be026635971499dff795106d1ff92433, offset: 0, size: 0 2024.10.22 02:13:25.550687 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key be026635971499dff795106d1ff92433 for removal 2024.10.22 02:13:25.552304 [ 1476 ] {} CacheMetadata: Key be026635971499dff795106d1ff92433 is removed from metadata 2024.10.22 02:13:25.537089 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1327_24, 202410_1328_1328_0, 202410_1329_1329_0, 202410_1330_1330_0, 202410_1331_1331_0, 202410_1332_1332_0] 2024.10.22 02:13:25.554608 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1327_24, 202410_1328_1328_0, 202410_1329_1329_0, 202410_1330_1330_0, 202410_1331_1331_0, 202410_1332_1332_0] 2024.10.22 02:13:25.610468 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 41b0d3a72d8a8cb8c56779ed2329869e, offset: 0, size: 0 2024.10.22 02:13:25.610364 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 9.73 MiB, will set to 1.54 GiB (RSS), difference: 18.68 MiB 2024.10.22 02:13:25.613565 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 41b0d3a72d8a8cb8c56779ed2329869e for removal 2024.10.22 02:13:25.625666 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nvs/gcovvqfodomvdpbaqmlvudxijguwy. 2024.10.22 02:13:25.688522 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1327_24, 202410_1328_1328_0, 202410_1329_1329_0, 202410_1330_1330_0, 202410_1331_1331_0, 202410_1332_1332_0] 2024.10.22 02:13:25.720139 [ 1476 ] {} CacheMetadata: Key 41b0d3a72d8a8cb8c56779ed2329869e is removed from metadata 2024.10.22 02:13:25.924069 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 117 entries to flush up to offset 91929 2024.10.22 02:13:26.280829 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:26.290277 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:26.354270 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_827_827_0 (state Temporary) 2024.10.22 02:13:26.507385 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_827_827_0 to 202410_833_833_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:26.621013 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 91929 2024.10.22 02:13:26.945717 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vch/uzrpekbtmdwesjzhggatimtimlkxo. 2024.10.22 02:13:27.168367 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 134 entries to flush up to offset 94303 2024.10.22 02:13:27.188976 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: b0caf4f74b56c51bd6a35143a2025322, offset: 0, size: 0 2024.10.22 02:13:27.193429 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key b0caf4f74b56c51bd6a35143a2025322 for removal 2024.10.22 02:13:27.199200 [ 1476 ] {} CacheMetadata: Key b0caf4f74b56c51bd6a35143a2025322 is removed from metadata 2024.10.22 02:13:27.236747 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 5fd29699e006415d61b563e3f2ba1676, offset: 0, size: 0 2024.10.22 02:13:27.239988 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 5fd29699e006415d61b563e3f2ba1676 for removal 2024.10.22 02:13:27.246294 [ 1476 ] {} CacheMetadata: Key 5fd29699e006415d61b563e3f2ba1676 is removed from metadata 2024.10.22 02:13:27.351683 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vch/uzrpekbtmdwesjzhggatimtimlkxo. 2024.10.22 02:13:27.363552 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 10.07 MiB, will set to 1.54 GiB (RSS), difference: 4.51 MiB 2024.10.22 02:13:27.649134 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:27.654677 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:27.760536 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1418_1418_0 (state Temporary) 2024.10.22 02:13:27.772444 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_808_152/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:13:27.800257 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1418_1418_0 to 202410_1431_1431_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:27.849530 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 969018 bytes for b0caf4f74b56c51bd6a35143a2025322:0) 2024.10.22 02:13:27.852841 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: d8b50fd83031e036d2e6aa8fdff0742a, offset: 0, size: 967013 2024.10.22 02:13:27.855930 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key d8b50fd83031e036d2e6aa8fdff0742a for removal 2024.10.22 02:13:27.859326 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8697d1795fd323bda640c1cc81978fc7, offset: 0, size: 506 2024.10.22 02:13:27.859434 [ 1476 ] {} CacheMetadata: Key d8b50fd83031e036d2e6aa8fdff0742a is removed from metadata 2024.10.22 02:13:27.861682 [ 1386 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8697d1795fd323bda640c1cc81978fc7 for removal 2024.10.22 02:13:27.898929 [ 1476 ] {} CacheMetadata: Key 8697d1795fd323bda640c1cc81978fc7 is removed from metadata 2024.10.22 02:13:27.936890 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1426_44 to 202410_1431_1431_0 2024.10.22 02:13:28.048364 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:28.056781 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94303 2024.10.22 02:13:28.067300 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:28.104683 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1426_44 to 202410_1431_1431_0 into Compact with storage Full 2024.10.22 02:13:28.153350 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:28.161713 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1426_44, total 14395 rows starting from the beginning of the part 2024.10.22 02:13:28.186961 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_1427_1427_0, total 146 rows starting from the beginning of the part 2024.10.22 02:13:28.189755 [ 1191 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 275 ms. 2024.10.22 02:13:28.239770 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_1428_1428_0, total 198 rows starting from the beginning of the part 2024.10.22 02:13:28.266854 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_1429_1429_0, total 141 rows starting from the beginning of the part 2024.10.22 02:13:28.296529 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_1430_1430_0, total 148 rows starting from the beginning of the part 2024.10.22 02:13:28.338821 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_1431_1431_0, total 134 rows starting from the beginning of the part 2024.10.22 02:13:28.551762 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:28.554980 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:28.711995 [ 1392 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergeTask::MergeProjectionsStage: Merge sorted 2672554 rows, containing 133 columns (3 merged, 130 gathered) in 3046.974272303 sec., 877.1173502492355 rows/sec., 626.31 KiB/sec. 2024.10.22 02:13:28.929664 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 699 2024.10.22 02:13:28.929686 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_165_169_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:13:28.969695 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 368319 bytes for 089d88d23c80e0ee2e8b4877d25e1cfd:0) 2024.10.22 02:13:28.971720 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 392c35869a2db4990578a9e68bea478e, offset: 0, size: 10246423 2024.10.22 02:13:28.983042 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 392c35869a2db4990578a9e68bea478e for removal 2024.10.22 02:13:28.989706 [ 1476 ] {} CacheMetadata: Key 392c35869a2db4990578a9e68bea478e is removed from metadata 2024.10.22 02:13:28.991294 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 8471f8fe03e84a1ecaca27b0837643de, offset: 0, size: 31607 2024.10.22 02:13:28.996398 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 8471f8fe03e84a1ecaca27b0837643de for removal 2024.10.22 02:13:29.001377 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: 27b99a928414d3044c04a84bffd6e2f9, offset: 0, size: 981 2024.10.22 02:13:29.012304 [ 1476 ] {} CacheMetadata: Key 8471f8fe03e84a1ecaca27b0837643de is removed from metadata 2024.10.22 02:13:29.110294 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key 27b99a928414d3044c04a84bffd6e2f9 for removal 2024.10.22 02:13:29.115440 [ 1476 ] {} CacheMetadata: Key 27b99a928414d3044c04a84bffd6e2f9 is removed from metadata 2024.10.22 02:13:29.117814 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Remove from cache. Key: ba5e7351c77d231338f1f07b88190d28, offset: 0, size: 483 2024.10.22 02:13:29.123508 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} CacheMetadata: Submitting key ba5e7351c77d231338f1f07b88190d28 for removal 2024.10.22 02:13:29.131354 [ 1476 ] {} CacheMetadata: Key ba5e7351c77d231338f1f07b88190d28 is removed from metadata 2024.10.22 02:13:29.246747 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} MergedBlockOutputStream: filled checksums 201403_17_24_1 (state Temporary) 2024.10.22 02:13:29.359524 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nff/pjkpjkbthroavgzkadmvsmappktfp (LogSeriesLimiter: on interval from 2024-10-22 02:13:15 to 2024-10-22 02:13:29 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:13:29.432327 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1214775 2024.10.22 02:13:29.448239 [ 1709 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pms/facyzhdlagmrxdkcroaqoessnrueu, size 2733 2024.10.22 02:13:29.512211 [ 1746 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nff/pjkpjkbthroavgzkadmvsmappktfp, size 4 2024.10.22 02:13:29.580391 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:29.584206 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.79 GiB. 2024.10.22 02:13:30.029516 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_681_681_0 (state Temporary) 2024.10.22 02:13:30.113501 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_681_681_0 to 202410_691_691_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:30.164161 [ 1163 ] {} DatabaseCatalog: Execution took 255 ms. 2024.10.22 02:13:30.228041 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_638_130, 202410_639_639_0, 202410_640_640_0, 202410_641_641_0, 202410_642_642_0, 202410_643_643_0] 2024.10.22 02:13:30.218383 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} MergeTask::MergeProjectionsStage: Merge sorted 15162 rows, containing 13 columns (13 merged, 0 gathered) in 2.123342363 sec., 7140.628974490065 rows/sec., 1.40 MiB/sec. 2024.10.22 02:13:30.241146 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_638_130, 202410_639_639_0, 202410_640_640_0, 202410_641_641_0, 202410_642_642_0, 202410_643_643_0] 2024.10.22 02:13:30.278817 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_638_130, 202410_639_639_0, 202410_640_640_0, 202410_641_641_0, 202410_642_642_0, 202410_643_643_0] 2024.10.22 02:13:30.304138 [ 1693 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/eoz/sgtvrctioarwhwbewgezpowlxrrcw, size 892195 2024.10.22 02:13:30.323462 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column WithHash 2024.10.22 02:13:30.334203 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column WithHash 2024.10.22 02:13:30.345481 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 699 2024.10.22 02:13:30.350839 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column WithHash 2024.10.22 02:13:30.376471 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column WithHash 2024.10.22 02:13:30.393332 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 12803 2024.10.22 02:13:30.391374 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column WithHash 2024.10.22 02:13:30.454035 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column WithHash 2024.10.22 02:13:30.498547 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column WithHash 2024.10.22 02:13:30.511535 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} MergedBlockOutputStream: filled checksums 202410_1207_1431_45 (state Temporary) 2024.10.22 02:13:30.695122 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:30.701176 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:30.762388 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1431_45 to 202410_1207_1431_45 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:30.768333 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1431_45} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1426_44, 202410_1431_1431_0] -> 202410_1207_1431_45 2024.10.22 02:13:30.867315 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_842_842_0 (state Temporary) 2024.10.22 02:13:30.917126 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.99 MiB. 2024.10.22 02:13:31.025769 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_842_842_0 to 202410_855_855_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:31.070963 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12803 2024.10.22 02:13:31.072516 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:31.076090 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:31.259080 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_850_223 to 202410_851_851_0 2024.10.22 02:13:31.262591 [ 1220 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:31.264137 [ 1220 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:31.265821 [ 1220 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 02:13:31.274763 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_851_224} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_850_223 to 202410_851_851_0 into Compact with storage Full 2024.10.22 02:13:31.276420 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:13:31.280240 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:13:31.338598 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_851_224} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:31.348281 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_851_224} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_850_223, total 12836 rows starting from the beginning of the part 2024.10.22 02:13:31.367218 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_851_224} MergeTreeSequentialSource: Reading 2 marks from part 202410_851_851_0, total 15 rows starting from the beginning of the part 2024.10.22 02:13:31.448685 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_806_806_0 (state Temporary) 2024.10.22 02:13:31.455471 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 11.64 MiB, will set to 1.54 GiB (RSS), difference: -5.89 MiB 2024.10.22 02:13:31.671551 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_806_806_0 to 202410_813_813_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:31.705642 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1214775 2024.10.22 02:13:32.093172 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 101 entries to flush up to offset 94404 2024.10.22 02:13:32.109665 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_169_169_0 (state Temporary) 2024.10.22 02:13:32.150756 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_851_224} MergeTask::MergeProjectionsStage: Merge sorted 12851 rows, containing 12 columns (12 merged, 0 gathered) in 0.880037669 sec., 14602.78400878315 rows/sec., 2.28 MiB/sec. 2024.10.22 02:13:32.224682 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_851_224} MergedBlockOutputStream: filled checksums 202410_1_851_224 (state Temporary) 2024.10.22 02:13:32.575961 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_851_224} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_851_224 to 202410_1_851_224 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:32.584914 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_851_224} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_850_223, 202410_851_851_0] -> 202410_1_851_224 2024.10.22 02:13:32.646942 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.45 MiB. 2024.10.22 02:13:32.746293 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_165_169_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_165_169_1 to 202410_165_169_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:32.753905 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_165_169_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 5 parts: [202410_165_165_0, 202410_169_169_0] -> 202410_165_169_1 2024.10.22 02:13:32.764554 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:32.767015 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:32.811410 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_791_180, 202410_792_792_0] 2024.10.22 02:13:32.816609 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_791_180, 202410_792_792_0] 2024.10.22 02:13:32.827109 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_791_180, 202410_792_792_0] 2024.10.22 02:13:32.981734 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1419_1419_0 (state Temporary) 2024.10.22 02:13:33.039854 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1419_1419_0 to 202410_1432_1432_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:33.086662 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94404 2024.10.22 02:13:33.566506 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 11.72 MiB, will set to 1.53 GiB (RSS), difference: 4.09 MiB 2024.10.22 02:13:33.880040 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.75 MiB. 2024.10.22 02:13:33.969094 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/pms/facyzhdlagmrxdkcroaqoessnrueu. 2024.10.22 02:13:34.235296 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/nff/pjkpjkbthroavgzkadmvsmappktfp. 2024.10.22 02:13:34.247285 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 95 entries to flush up to offset 92024 2024.10.22 02:13:34.493526 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:34.498749 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:34.544451 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_828_828_0 (state Temporary) 2024.10.22 02:13:34.609697 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_828_828_0 to 202410_834_834_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:34.651522 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92024 2024.10.22 02:13:35.256105 [ 1394 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/nff/pjkpjkbthroavgzkadmvsmappktfp. 2024.10.22 02:13:35.263328 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Renaming temporary part tmp_merge_201403_17_24_1 to 201403_17_24_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:35.366917 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937) (MergerMutator): Merged 8 parts: [201403_17_17_0, 201403_24_24_0] -> 201403_17_24_1 2024.10.22 02:13:35.595201 [ 1393 ] {18eebfd3-a0d1-483d-948b-5d8c6de7d937::201403_17_24_1} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/pms/facyzhdlagmrxdkcroaqoessnrueu. 2024.10.22 02:13:35.658769 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 372.48 MiB. 2024.10.22 02:13:35.847812 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 11.57 MiB, will set to 1.54 GiB (RSS), difference: 13.88 MiB 2024.10.22 02:13:36.029214 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_742_141, 202410_743_748_1] 2024.10.22 02:13:36.034558 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_742_141, 202410_743_748_1] 2024.10.22 02:13:36.048403 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_742_141, 202410_743_748_1] 2024.10.22 02:13:36.353046 [ 1255 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:13:36.467747 [ 1255 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:13:36.661118 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 101540 bytes for b042b728d95a8ac2477aa943e4f44e76:0) 2024.10.22 02:13:36.667097 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 133253c8c4f70bd33d6ede93b87b42dc, offset: 0, size: 741641 2024.10.22 02:13:36.672481 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 133253c8c4f70bd33d6ede93b87b42dc for removal 2024.10.22 02:13:36.676951 [ 1476 ] {} CacheMetadata: Key 133253c8c4f70bd33d6ede93b87b42dc is removed from metadata 2024.10.22 02:13:36.679033 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e6714b10814d7a014fa5752216aa3e7c, offset: 0, size: 505 2024.10.22 02:13:36.687097 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e6714b10814d7a014fa5752216aa3e7c for removal 2024.10.22 02:13:36.715280 [ 1476 ] {} CacheMetadata: Key e6714b10814d7a014fa5752216aa3e7c is removed from metadata 2024.10.22 02:13:37.059051 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xka/bnqyqnnzmurgnbpvgebwtjklcgipl. 2024.10.22 02:13:37.101523 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 94450 2024.10.22 02:13:37.250895 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:37.254689 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:37.398672 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1420_1420_0 (state Temporary) 2024.10.22 02:13:37.539972 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xka/bnqyqnnzmurgnbpvgebwtjklcgipl. 2024.10.22 02:13:37.543538 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_14_14_0 to 201403_14_14_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:37.555276 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1420_1420_0 to 202410_1433_1433_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:37.565384 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 10.67 MiB, will set to 1.54 GiB (RSS), difference: -2.40 MiB 2024.10.22 02:13:37.597214 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94450 2024.10.22 02:13:37.714596 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_15_15_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:13:37.847607 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 700 2024.10.22 02:13:38.079712 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:38.088930 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:38.279590 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_682_682_0 (state Temporary) 2024.10.22 02:13:38.351437 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_682_682_0 to 202410_692_692_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:38.520152 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 700 2024.10.22 02:13:38.572577 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 32 entries to flush up to offset 12835 2024.10.22 02:13:38.713218 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1216700 2024.10.22 02:13:38.867862 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:38.869630 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:38.969246 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_843_843_0 (state Temporary) 2024.10.22 02:13:39.029175 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_843_843_0 to 202410_856_856_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:39.053100 [ 1257 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_851_224 to 202410_856_856_0 2024.10.22 02:13:39.069892 [ 1257 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:39.071854 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:39.122944 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_851_224 to 202410_856_856_0 into Compact with storage Full 2024.10.22 02:13:39.177160 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12835 2024.10.22 02:13:39.190715 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:39.199261 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_851_224, total 12851 rows starting from the beginning of the part 2024.10.22 02:13:39.231572 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} MergeTreeSequentialSource: Reading 2 marks from part 202410_852_852_0, total 24 rows starting from the beginning of the part 2024.10.22 02:13:39.264962 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} MergeTreeSequentialSource: Reading 2 marks from part 202410_853_853_0, total 12 rows starting from the beginning of the part 2024.10.22 02:13:39.312786 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} MergeTreeSequentialSource: Reading 2 marks from part 202410_854_854_0, total 13 rows starting from the beginning of the part 2024.10.22 02:13:39.453511 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} MergeTreeSequentialSource: Reading 2 marks from part 202410_855_855_0, total 9 rows starting from the beginning of the part 2024.10.22 02:13:39.461462 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.06 GiB, peak 2.00 GiB, free memory in arenas 8.94 MiB, will set to 1.28 GiB (RSS), difference: 227.12 MiB 2024.10.22 02:13:39.468799 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} MergeTreeSequentialSource: Reading 2 marks from part 202410_856_856_0, total 32 rows starting from the beginning of the part 2024.10.22 02:13:39.681084 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_169_169_0 to 202410_172_172_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:39.688280 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 2 entries to flush up to offset 57 2024.10.22 02:13:40.021467 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:40.048525 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:40.088989 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:40.090521 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_856_225/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:13:40.096655 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:40.152604 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_792_181, 202410_793_793_0] 2024.10.22 02:13:40.155076 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_792_181, 202410_793_793_0] 2024.10.22 02:13:40.165514 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_792_181, 202410_793_793_0] 2024.10.22 02:13:40.265124 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_42_42_0 (state Temporary) 2024.10.22 02:13:40.280588 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_807_807_0 (state Temporary) 2024.10.22 02:13:40.338518 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_808_152/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:13:40.356703 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_42_42_0 to 202410_42_42_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:40.387128 [ 1205 ] {} DatabaseCatalog: Execution took 220 ms. 2024.10.22 02:13:40.408674 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_807_807_0 to 202410_814_814_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:40.428950 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 57 2024.10.22 02:13:40.559845 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1216700 2024.10.22 02:13:40.650525 [ 1246 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_809_809_0 to 202410_814_814_0 2024.10.22 02:13:40.676869 [ 1246 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:40.681038 [ 1246 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:40.739030 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_809_809_0 to 202410_814_814_0 into Compact with storage Full 2024.10.22 02:13:40.768041 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:40.772144 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_809_809_0, total 1895 rows starting from the beginning of the part 2024.10.22 02:13:40.782732 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_810_810_0, total 1185 rows starting from the beginning of the part 2024.10.22 02:13:40.784574 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} MergeTask::MergeProjectionsStage: Merge sorted 12941 rows, containing 12 columns (12 merged, 0 gathered) in 1.690696318 sec., 7654.242729592316 rows/sec., 1.19 MiB/sec. 2024.10.22 02:13:40.809828 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_811_811_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:13:40.839188 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_812_812_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:13:40.859616 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_813_813_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:13:40.886204 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_814_814_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:13:40.960002 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} MergedBlockOutputStream: filled checksums 202410_1_856_225 (state Temporary) 2024.10.22 02:13:41.278742 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:13:41.281386 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_856_225 to 202410_1_856_225 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:41.282927 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:13:41.286969 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_856_225} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_851_224, 202410_856_856_0] -> 202410_1_856_225 2024.10.22 02:13:41.349471 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.66 MiB. 2024.10.22 02:13:41.405119 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 2.00 GiB, free memory in arenas 10.00 MiB, will set to 1.31 GiB (RSS), difference: -55.28 MiB 2024.10.22 02:13:41.605943 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 94522 2024.10.22 02:13:41.963292 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:41.967239 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:42.126588 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1421_1421_0 (state Temporary) 2024.10.22 02:13:42.169568 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 75 entries to flush up to offset 92099 2024.10.22 02:13:42.382463 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1421_1421_0 to 202410_1434_1434_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:42.446908 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94522 2024.10.22 02:13:42.453636 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 12.04 MiB, will set to 1.33 GiB (RSS), difference: -17.05 MiB 2024.10.22 02:13:42.644813 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6811 2024.10.22 02:13:42.677056 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:42.680555 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:42.742780 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_829_829_0 (state Temporary) 2024.10.22 02:13:42.849553 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} MergeTask::MergeProjectionsStage: Merge sorted 10010 rows, containing 4 columns (4 merged, 0 gathered) in 2.10574196 sec., 4753.668868335605 rows/sec., 74.28 KiB/sec. 2024.10.22 02:13:42.903180 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} MergedBlockOutputStream: filled checksums 202410_809_814_1 (state Temporary) 2024.10.22 02:13:42.911681 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_829_829_0 to 202410_835_835_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:42.961582 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92099 2024.10.22 02:13:42.967439 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_809_814_1 to 202410_809_814_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:42.973016 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_809_814_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_809_809_0, 202410_814_814_0] -> 202410_809_814_1 2024.10.22 02:13:43.019498 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:13:44.645036 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 17.65 MiB, will set to 1.37 GiB (RSS), difference: -11.89 MiB 2024.10.22 02:13:45.565437 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column HitColor 2024.10.22 02:13:45.581824 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column HitColor 2024.10.22 02:13:45.707021 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column HitColor 2024.10.22 02:13:45.719408 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column HitColor 2024.10.22 02:13:45.840819 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column HitColor 2024.10.22 02:13:45.855261 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column HitColor 2024.10.22 02:13:45.917812 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column HitColor 2024.10.22 02:13:45.949410 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/czz/svzoghkzsoemsjskefzdzkifamxvq (LogSeriesLimiter: on interval from 2024-10-22 02:13:29 to 2024-10-22 02:13:45 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:13:46.023697 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 701 2024.10.22 02:13:46.280432 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:46.283608 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:46.493117 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_683_683_0 (state Temporary) 2024.10.22 02:13:46.561329 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 31 entries to flush up to offset 94553 2024.10.22 02:13:46.654778 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_683_683_0 to 202410_693_693_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:46.674056 [ 1277 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_688_140 to 202410_693_693_0 2024.10.22 02:13:46.682033 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 12856 2024.10.22 02:13:46.706182 [ 1277 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:46.709999 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:46.737353 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 2.00 GiB, free memory in arenas 17.20 MiB, will set to 1.42 GiB (RSS), difference: -45.77 MiB 2024.10.22 02:13:46.754209 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_688_140 to 202410_693_693_0 into Compact with storage Full 2024.10.22 02:13:46.796047 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:46.803005 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:46.830836 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 701 2024.10.22 02:13:46.849625 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1422_1422_0 (state Temporary) 2024.10.22 02:13:46.867424 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:46.875104 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:46.879113 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:46.884706 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_688_140, total 732 rows starting from the beginning of the part 2024.10.22 02:13:46.919013 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1422_1422_0 to 202410_1435_1435_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:47.003528 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94553 2024.10.22 02:13:47.047186 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_689_689_0, total 1 rows starting from the beginning of the part 2024.10.22 02:13:47.049331 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_844_844_0 (state Temporary) 2024.10.22 02:13:47.108547 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_690_690_0, total 1 rows starting from the beginning of the part 2024.10.22 02:13:47.135658 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_844_844_0 to 202410_857_857_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:47.152941 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_691_691_0, total 1 rows starting from the beginning of the part 2024.10.22 02:13:47.230905 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_772_5, 202410_773_773_0, 202410_774_774_0, 202410_775_775_0, 202410_776_776_0, 202410_777_777_0] 2024.10.22 02:13:47.233124 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_772_5, 202410_773_773_0, 202410_774_774_0, 202410_775_775_0, 202410_776_776_0, 202410_777_777_0] 2024.10.22 02:13:47.236748 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12856 2024.10.22 02:13:47.270350 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_772_5, 202410_773_773_0, 202410_774_774_0, 202410_775_775_0, 202410_776_776_0, 202410_777_777_0] 2024.10.22 02:13:47.276960 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_692_692_0, total 1 rows starting from the beginning of the part 2024.10.22 02:13:47.556539 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_693_693_0, total 1 rows starting from the beginning of the part 2024.10.22 02:13:47.564378 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1218625 2024.10.22 02:13:48.336853 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_793_182, 202410_794_794_0] 2024.10.22 02:13:48.340467 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_793_182, 202410_794_794_0] 2024.10.22 02:13:48.353471 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_793_182, 202410_794_794_0] 2024.10.22 02:13:48.475298 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 2.00 GiB, free memory in arenas 13.93 MiB, will set to 1.45 GiB (RSS), difference: -9.89 MiB 2024.10.22 02:13:48.671965 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:48.681531 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:48.830980 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_808_808_0 (state Temporary) 2024.10.22 02:13:48.949053 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_808_808_0 to 202410_815_815_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:48.969385 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1218625 2024.10.22 02:13:49.047425 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_693_141/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:13:49.457356 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1332_25, 202410_1333_1333_0, 202410_1334_1334_0, 202410_1335_1335_0, 202410_1336_1336_0, 202410_1337_1337_0] 2024.10.22 02:13:49.560021 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1332_25, 202410_1333_1333_0, 202410_1334_1334_0, 202410_1335_1335_0, 202410_1336_1336_0, 202410_1337_1337_0] 2024.10.22 02:13:49.654567 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} MergeTask::MergeProjectionsStage: Merge sorted 737 rows, containing 32 columns (32 merged, 0 gathered) in 2.920308165 sec., 252.37062609794813 rows/sec., 22.81 KiB/sec. 2024.10.22 02:13:49.680849 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} MergedBlockOutputStream: filled checksums 202410_1_693_141 (state Temporary) 2024.10.22 02:13:49.683255 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1332_25, 202410_1333_1333_0, 202410_1334_1334_0, 202410_1335_1335_0, 202410_1336_1336_0, 202410_1337_1337_0] 2024.10.22 02:13:49.955869 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_15_15_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:13:50.062163 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_693_141 to 202410_1_693_141 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:50.064111 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_693_141} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_688_140, 202410_693_693_0] -> 202410_1_693_141 2024.10.22 02:13:50.122744 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:13:50.298369 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 6849 2024.10.22 02:13:50.375737 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 15.47 MiB, will set to 1.48 GiB (RSS), difference: -41.84 MiB 2024.10.22 02:13:50.469002 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 65 entries to flush up to offset 92164 2024.10.22 02:13:50.589064 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:50.591748 [ 1264 ] {} DatabaseCatalog: Execution took 204 ms. 2024.10.22 02:13:50.598762 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:50.664672 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_830_830_0 (state Temporary) 2024.10.22 02:13:50.780925 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_830_830_0 to 202410_836_836_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:51.014126 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 94611 2024.10.22 02:13:51.034729 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92164 2024.10.22 02:13:51.280988 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:13:51.287142 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:13:51.472397 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:51.473206 [ 1151 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:13:51.479305 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:51.535288 [ 1151 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:13:51.661638 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1423_1423_0 (state Temporary) 2024.10.22 02:13:52.131446 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1423_1423_0 to 202410_1436_1436_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:52.163765 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1207_1431_45 to 202410_1434_1434_0 2024.10.22 02:13:52.177656 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:52.176469 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94611 2024.10.22 02:13:52.182648 [ 1239 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:52.232600 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1434_46} MergeTask::PrepareStage: Merging 4 parts: from 202410_1207_1431_45 to 202410_1434_1434_0 into Compact with storage Full 2024.10.22 02:13:52.313206 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1434_46} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:52.355940 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1434_46} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1431_45, total 15162 rows starting from the beginning of the part 2024.10.22 02:13:52.420764 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 02:13:52.548431 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1434_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_1432_1432_0, total 101 rows starting from the beginning of the part 2024.10.22 02:13:52.562138 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_808_152/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:13:52.640729 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1434_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_1433_1433_0, total 46 rows starting from the beginning of the part 2024.10.22 02:13:52.664015 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 2.00 GiB, free memory in arenas 12.61 MiB, will set to 1.58 GiB (RSS), difference: -120.81 MiB 2024.10.22 02:13:52.840279 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1434_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_1434_1434_0, total 72 rows starting from the beginning of the part 2024.10.22 02:13:53.003224 [ 1209 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 02:13:54.300161 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1434_46} MergeTask::MergeProjectionsStage: Merge sorted 15381 rows, containing 13 columns (13 merged, 0 gathered) in 2.086663897 sec., 7371.096045756715 rows/sec., 1.44 MiB/sec. 2024.10.22 02:13:54.333692 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 702 2024.10.22 02:13:54.575014 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 2.00 GiB, free memory in arenas 12.55 MiB, will set to 1.71 GiB (RSS), difference: -59.30 MiB 2024.10.22 02:13:54.583778 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1434_46} MergedBlockOutputStream: filled checksums 202410_1207_1434_46 (state Temporary) 2024.10.22 02:13:54.660626 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1434_46} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1434_46 to 202410_1207_1434_46 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:54.665284 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1434_46} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1207_1431_45, 202410_1434_1434_0] -> 202410_1207_1434_46 2024.10.22 02:13:54.711042 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.89 MiB. 2024.10.22 02:13:54.741217 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 12873 2024.10.22 02:13:54.857082 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:54.871611 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:54.895452 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:54.899948 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:54.913417 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_845_845_0 (state Temporary) 2024.10.22 02:13:54.980946 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_684_684_0 (state Temporary) 2024.10.22 02:13:55.149292 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_845_845_0 to 202410_858_858_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:55.189194 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_684_684_0 to 202410_694_694_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:55.204827 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12873 2024.10.22 02:13:55.372718 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 702 2024.10.22 02:13:55.975056 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1220165 2024.10.22 02:13:56.195065 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 94654 2024.10.22 02:13:56.476397 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.20 GiB, peak 2.00 GiB, free memory in arenas 12.34 MiB, will set to 1.51 GiB (RSS), difference: 320.08 MiB 2024.10.22 02:13:56.537710 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:56.541084 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:56.622817 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:56.627201 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:56.658885 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_809_809_0 (state Temporary) 2024.10.22 02:13:56.668710 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1424_1424_0 (state Temporary) 2024.10.22 02:13:56.733481 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1424_1424_0 to 202410_1437_1437_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:56.801324 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_809_809_0 to 202410_816_816_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:56.810429 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94654 2024.10.22 02:13:56.928302 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1220165 2024.10.22 02:13:57.047744 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_794_183, 202410_795_795_0] 2024.10.22 02:13:57.050942 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_794_183, 202410_795_795_0] 2024.10.22 02:13:57.072091 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_794_183, 202410_795_795_0] 2024.10.22 02:13:57.959265 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 491.09 MiB using storage policy from min volume index 0 2024.10.22 02:13:58.548291 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 53 entries to flush up to offset 92217 2024.10.22 02:13:58.715468 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 13.17 MiB, will set to 1.52 GiB (RSS), difference: 7.51 MiB 2024.10.22 02:13:58.718510 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_808_152} MergeTask::MergeProjectionsStage: Merge sorted 1220280 rows, containing 4 columns (4 merged, 0 gathered) in 36.25267633 sec., 33660.41141051392 rows/sec., 504.96 KiB/sec. 2024.10.22 02:13:58.861676 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:58.865421 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:58.917072 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_808_152} MergedBlockOutputStream: filled checksums 202410_1_808_152 (state Temporary) 2024.10.22 02:13:59.011281 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_831_831_0 (state Temporary) 2024.10.22 02:13:59.239739 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xqk/hyabxbwxmwfyleknxnziaemrmzdol (LogSeriesLimiter: on interval from 2024-10-22 02:13:45 to 2024-10-22 02:13:59 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:13:59.256404 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_831_831_0 to 202410_837_837_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:13:59.272149 [ 1246 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_832_17 to 202410_837_837_0 2024.10.22 02:13:59.282625 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92217 2024.10.22 02:13:59.378799 [ 1246 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:13:59.384336 [ 1246 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:13:59.418993 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_832_17 to 202410_837_837_0 into Compact with storage Full 2024.10.22 02:13:59.559084 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:13:59.567110 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_832_17, total 8457 rows starting from the beginning of the part 2024.10.22 02:13:59.682998 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_833_833_0, total 117 rows starting from the beginning of the part 2024.10.22 02:13:59.709062 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_834_834_0, total 95 rows starting from the beginning of the part 2024.10.22 02:13:59.842919 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_835_835_0, total 75 rows starting from the beginning of the part 2024.10.22 02:13:59.858436 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_836_836_0, total 65 rows starting from the beginning of the part 2024.10.22 02:13:59.874972 [ 1722 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/czz/svzoghkzsoemsjskefzdzkifamxvq, size 470794 2024.10.22 02:13:59.896196 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} MergeTreeSequentialSource: Reading 2 marks from part 202410_837_837_0, total 53 rows starting from the beginning of the part 2024.10.22 02:14:00.011608 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_755_755_0, 202410_756_756_0, 202410_757_757_0, 202410_758_758_0, 202410_759_759_0, 202410_760_760_0] 2024.10.22 02:14:00.033498 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_755_755_0, 202410_756_756_0, 202410_757_757_0, 202410_758_758_0, 202410_759_759_0, 202410_760_760_0] 2024.10.22 02:14:00.168701 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_755_755_0, 202410_756_756_0, 202410_757_757_0, 202410_758_758_0, 202410_759_759_0, 202410_760_760_0] 2024.10.22 02:14:00.393653 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 02:14:00.415268 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 12.08 MiB, will set to 1.51 GiB (RSS), difference: -117.40 MiB 2024.10.22 02:14:00.521401 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 02:14:00.547676 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 02:14:00.555724 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 02:14:00.576595 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 02:14:00.613263 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 02:14:00.647265 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTCEventTime 2024.10.22 02:14:00.847372 [ 1204 ] {} DatabaseCatalog: Execution took 249 ms. 2024.10.22 02:14:00.841832 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 94703 2024.10.22 02:14:01.287518 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:14:01.297418 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:14:01.344067 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:01.346925 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:01.452484 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} MergeTask::MergeProjectionsStage: Merge sorted 8862 rows, containing 13 columns (13 merged, 0 gathered) in 2.038622345 sec., 4347.053303783934 rows/sec., 1.88 MiB/sec. 2024.10.22 02:14:01.591348 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1425_1425_0 (state Temporary) 2024.10.22 02:14:01.599297 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} MergedBlockOutputStream: filled checksums 202410_747_837_18 (state Temporary) 2024.10.22 02:14:01.744675 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1425_1425_0 to 202410_1438_1438_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:01.783945 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_837_18 to 202410_747_837_18 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:01.789393 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_837_18} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_832_17, 202410_837_837_0] -> 202410_747_837_18 2024.10.22 02:14:01.892503 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94703 2024.10.22 02:14:01.971193 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.10 MiB. 2024.10.22 02:14:02.390902 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 10.60 MiB, will set to 1.52 GiB (RSS), difference: -139.51 MiB 2024.10.22 02:14:02.709644 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 12886 2024.10.22 02:14:02.974415 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 703 2024.10.22 02:14:03.059562 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:03.064173 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:03.107820 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_846_846_0 (state Temporary) 2024.10.22 02:14:03.254942 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_808_152} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_808_152 to 202410_1_808_152 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:03.259432 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_808_152} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_802_151, 202410_803_808_1] -> 202410_1_808_152 2024.10.22 02:14:03.280211 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_846_846_0 to 202410_859_859_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:03.305932 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:03.314793 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:03.375772 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_685_685_0 (state Temporary) 2024.10.22 02:14:03.402939 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:14:03.545426 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12886 2024.10.22 02:14:03.590709 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_685_685_0 to 202410_695_695_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:03.661006 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 703 2024.10.22 02:14:03.936820 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1221705 2024.10.22 02:14:04.009315 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ylb/cwcmoewznqbtjsajgwobpgagwlyho (LogSeriesLimiter: on interval from 2024-10-22 02:13:59 to 2024-10-22 02:14:04 accepted series 1 / 275 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:14:04.143940 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 1048576 bytes for 8f14a51421bbd7059452f863bb190e22:0) 2024.10.22 02:14:04.146367 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a8e890157c02a05debd7322801c7cb98, offset: 0, size: 630290 2024.10.22 02:14:04.151225 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a8e890157c02a05debd7322801c7cb98 for removal 2024.10.22 02:14:04.153618 [ 1476 ] {} CacheMetadata: Key a8e890157c02a05debd7322801c7cb98 is removed from metadata 2024.10.22 02:14:04.153678 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 298a42f594a5185987db0b6d4b168db2, offset: 0, size: 501 2024.10.22 02:14:04.161273 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 298a42f594a5185987db0b6d4b168db2 for removal 2024.10.22 02:14:04.167453 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4d9e3853cff0ec8694bfc942315491d8, offset: 0, size: 498 2024.10.22 02:14:04.172072 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 4d9e3853cff0ec8694bfc942315491d8 for removal 2024.10.22 02:14:04.176162 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a0d24a49019615b2230383543e1b0ea2, offset: 0, size: 643102 2024.10.22 02:14:04.180980 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a0d24a49019615b2230383543e1b0ea2 for removal 2024.10.22 02:14:04.180374 [ 1476 ] {} CacheMetadata: Key 4d9e3853cff0ec8694bfc942315491d8 is removed from metadata 2024.10.22 02:14:04.184110 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 21455029cfb7dfd5c965b123e2be78d9, offset: 0, size: 4 2024.10.22 02:14:04.190181 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 21455029cfb7dfd5c965b123e2be78d9 for removal 2024.10.22 02:14:04.196873 [ 1476 ] {} CacheMetadata: Key a0d24a49019615b2230383543e1b0ea2 is removed from metadata 2024.10.22 02:14:04.226737 [ 1476 ] {} CacheMetadata: Key 21455029cfb7dfd5c965b123e2be78d9 is removed from metadata 2024.10.22 02:14:04.243202 [ 1476 ] {} CacheMetadata: Key 298a42f594a5185987db0b6d4b168db2 is removed from metadata 2024.10.22 02:14:04.271226 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 2.00 GiB, free memory in arenas 8.14 MiB, will set to 1.52 GiB (RSS), difference: -204.21 MiB 2024.10.22 02:14:04.583498 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_795_184, 202410_796_796_0] 2024.10.22 02:14:04.598157 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_795_184, 202410_796_796_0] 2024.10.22 02:14:04.619794 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_795_184, 202410_796_796_0] 2024.10.22 02:14:04.691544 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:04.695409 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:05.132876 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_810_810_0 (state Temporary) 2024.10.22 02:14:05.304590 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_810_810_0 to 202410_817_817_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:05.355325 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1221705 2024.10.22 02:14:05.899760 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 94760 2024.10.22 02:14:05.943041 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 8f14a51421bbd7059452f863bb190e22:0) 2024.10.22 02:14:05.947242 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6fa586a05d27218446720af217114c80, offset: 0, size: 874833 2024.10.22 02:14:05.951193 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 6fa586a05d27218446720af217114c80 for removal 2024.10.22 02:14:05.953127 [ 1476 ] {} CacheMetadata: Key 6fa586a05d27218446720af217114c80 is removed from metadata 2024.10.22 02:14:05.953446 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 09ec90228ce860fd44b144993c35ee34, offset: 0, size: 635 2024.10.22 02:14:05.955338 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 09ec90228ce860fd44b144993c35ee34 for removal 2024.10.22 02:14:05.957661 [ 1476 ] {} CacheMetadata: Key 09ec90228ce860fd44b144993c35ee34 is removed from metadata 2024.10.22 02:14:06.248983 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:06.251444 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:06.304903 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1426_1426_0 (state Temporary) 2024.10.22 02:14:06.366083 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1426_1426_0 to 202410_1439_1439_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:06.399957 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1434_46 to 202410_1435_1435_0 2024.10.22 02:14:06.431278 [ 1158 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:06.435386 [ 1158 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:06.521503 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94760 2024.10.22 02:14:06.537594 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 8.07 MiB, will set to 1.53 GiB (RSS), difference: -89.85 MiB 2024.10.22 02:14:06.545834 [ 1189 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:14:06.551704 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1435_47} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1434_46 to 202410_1435_1435_0 into Compact with storage Full 2024.10.22 02:14:06.592306 [ 1189 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:14:06.593716 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1435_47} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:06.610696 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1435_47} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1434_46, total 15381 rows starting from the beginning of the part 2024.10.22 02:14:06.763480 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1435_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_1435_1435_0, total 31 rows starting from the beginning of the part 2024.10.22 02:14:06.796724 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 99 entries to flush up to offset 92316 2024.10.22 02:14:06.903062 [ 1202 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Selected 5 parts from 202410_1_38_9 to 202410_42_42_0 2024.10.22 02:14:06.918282 [ 1202 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:06.923724 [ 1202 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:07.078808 [ 1202 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 213 ms. 2024.10.22 02:14:07.145169 [ 1391 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_42_10} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_38_9 to 202410_42_42_0 into Compact with storage Full 2024.10.22 02:14:07.260601 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:07.262344 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:07.311146 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_832_832_0 (state Temporary) 2024.10.22 02:14:07.364989 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_832_832_0 to 202410_838_838_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:07.514972 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92316 2024.10.22 02:14:07.518261 [ 1391 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_42_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:07.553509 [ 1391 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_42_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_38_9, total 44 rows starting from the beginning of the part 2024.10.22 02:14:07.599773 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:14:07.650676 [ 1391 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_42_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_39_39_0, total 1 rows starting from the beginning of the part 2024.10.22 02:14:07.740654 [ 1391 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_42_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_40_40_0, total 9 rows starting from the beginning of the part 2024.10.22 02:14:07.956915 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1435_47} MergeTask::MergeProjectionsStage: Merge sorted 15412 rows, containing 13 columns (13 merged, 0 gathered) in 1.511202428 sec., 10198.501348622767 rows/sec., 1.99 MiB/sec. 2024.10.22 02:14:07.976143 [ 1391 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_42_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_41_41_0, total 1 rows starting from the beginning of the part 2024.10.22 02:14:08.046823 [ 1391 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_42_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_42_42_0, total 2 rows starting from the beginning of the part 2024.10.22 02:14:08.060467 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1435_47} MergedBlockOutputStream: filled checksums 202410_1207_1435_47 (state Temporary) 2024.10.22 02:14:08.251688 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1435_47} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1435_47 to 202410_1207_1435_47 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:08.264139 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1435_47} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1434_46, 202410_1435_1435_0] -> 202410_1207_1435_47 2024.10.22 02:14:08.392183 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.86 MiB. 2024.10.22 02:14:08.640775 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 6.25 MiB, will set to 1.54 GiB (RSS), difference: -5.27 MiB 2024.10.22 02:14:08.661840 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:08.766654 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:09.217610 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 2097152 bytes for 8f14a51421bbd7059452f863bb190e22:0) 2024.10.22 02:14:09.230710 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 223815c382958e5b1d949e11249b7c7e, offset: 0, size: 2552763 2024.10.22 02:14:09.244913 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 223815c382958e5b1d949e11249b7c7e for removal 2024.10.22 02:14:09.248600 [ 1476 ] {} CacheMetadata: Key 223815c382958e5b1d949e11249b7c7e is removed from metadata 2024.10.22 02:14:09.248840 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: ad493f53f03973cfca0a94a55292cf4b, offset: 0, size: 372376 2024.10.22 02:14:09.253144 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key ad493f53f03973cfca0a94a55292cf4b for removal 2024.10.22 02:14:09.255775 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c3d1d86c9d7b4f29e0804fe4964abda1, offset: 0, size: 879 2024.10.22 02:14:09.258764 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c3d1d86c9d7b4f29e0804fe4964abda1 for removal 2024.10.22 02:14:09.264257 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 91d93e9ff3afcd8b1edf2fce819ad138, offset: 0, size: 494 2024.10.22 02:14:09.265480 [ 1476 ] {} CacheMetadata: Key c3d1d86c9d7b4f29e0804fe4964abda1 is removed from metadata 2024.10.22 02:14:09.270097 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 91d93e9ff3afcd8b1edf2fce819ad138 for removal 2024.10.22 02:14:09.273110 [ 1476 ] {} CacheMetadata: Key 91d93e9ff3afcd8b1edf2fce819ad138 is removed from metadata 2024.10.22 02:14:09.293384 [ 1476 ] {} CacheMetadata: Key ad493f53f03973cfca0a94a55292cf4b is removed from metadata 2024.10.22 02:14:09.624001 [ 1400 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_42_10} MergeTask::MergeProjectionsStage: Merge sorted 57 rows, containing 27 columns (27 merged, 0 gathered) in 2.547502797 sec., 22.37485276448943 rows/sec., 54.05 KiB/sec. 2024.10.22 02:14:09.842202 [ 1389 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_42_10} MergedBlockOutputStream: filled checksums 202410_1_42_10 (state Temporary) 2024.10.22 02:14:10.054818 [ 1387 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_42_10} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_merge_202410_1_42_10 to 202410_1_42_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:10.058040 [ 1387 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_42_10} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Merged 5 parts: [202410_1_38_9, 202410_42_42_0] -> 202410_1_42_10 2024.10.22 02:14:10.100306 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.24 MiB. 2024.10.22 02:14:10.453373 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 6.36 MiB, will set to 1.54 GiB (RSS), difference: -5.60 MiB 2024.10.22 02:14:10.542568 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 94828 2024.10.22 02:14:10.685313 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:10.691544 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:10.756284 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1427_1427_0 (state Temporary) 2024.10.22 02:14:10.868255 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1427_1427_0 to 202410_1440_1440_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:10.915004 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94828 2024.10.22 02:14:11.048238 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 12904 2024.10.22 02:14:11.038633 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1435_47 to 202410_1440_1440_0 2024.10.22 02:14:11.065451 [ 1207 ] {} DatabaseCatalog: Execution took 217 ms. 2024.10.22 02:14:11.074172 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:11.086194 [ 1214 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:11.103759 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 214 ms. 2024.10.22 02:14:11.151895 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1435_47 to 202410_1440_1440_0 into Compact with storage Full 2024.10.22 02:14:11.180684 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:11.200112 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1435_47, total 15412 rows starting from the beginning of the part 2024.10.22 02:14:11.215470 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:11.220150 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:11.284810 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:14:11.290534 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:14:11.328569 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_1436_1436_0, total 58 rows starting from the beginning of the part 2024.10.22 02:14:11.356190 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_847_847_0 (state Temporary) 2024.10.22 02:14:11.367319 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_1437_1437_0, total 43 rows starting from the beginning of the part 2024.10.22 02:14:11.391008 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_1438_1438_0, total 49 rows starting from the beginning of the part 2024.10.22 02:14:11.436387 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_1439_1439_0, total 57 rows starting from the beginning of the part 2024.10.22 02:14:11.466944 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_1440_1440_0, total 68 rows starting from the beginning of the part 2024.10.22 02:14:11.539856 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_847_847_0 to 202410_860_860_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:11.569802 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12904 2024.10.22 02:14:11.846925 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_170_170_0 (state Temporary) 2024.10.22 02:14:12.358303 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1223245 2024.10.22 02:14:12.635906 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 6.48 MiB, will set to 1.56 GiB (RSS), difference: -13.00 MiB 2024.10.22 02:14:12.935089 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_796_185, 202410_797_797_0] 2024.10.22 02:14:12.943021 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_796_185, 202410_797_797_0] 2024.10.22 02:14:12.964834 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_796_185, 202410_797_797_0] 2024.10.22 02:14:13.358220 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:13.361905 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:13.508863 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} MergeTask::MergeProjectionsStage: Merge sorted 15687 rows, containing 13 columns (13 merged, 0 gathered) in 2.364126789 sec., 6635.430922313363 rows/sec., 1.29 MiB/sec. 2024.10.22 02:14:13.593292 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} MergedBlockOutputStream: filled checksums 202410_1207_1440_48 (state Temporary) 2024.10.22 02:14:13.760867 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1440_48 to 202410_1207_1440_48 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:13.763703 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1440_48} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1435_47, 202410_1440_1440_0] -> 202410_1207_1440_48 2024.10.22 02:14:13.760789 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1337_26, 202410_1338_1338_0, 202410_1339_1339_0, 202410_1340_1340_0, 202410_1341_1341_0, 202410_1342_1342_0] 2024.10.22 02:14:13.774070 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1337_26, 202410_1338_1338_0, 202410_1339_1339_0, 202410_1340_1340_0, 202410_1341_1341_0, 202410_1342_1342_0] 2024.10.22 02:14:13.792897 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_811_811_0 (state Temporary) 2024.10.22 02:14:13.808509 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1337_26, 202410_1338_1338_0, 202410_1339_1339_0, 202410_1340_1340_0, 202410_1341_1341_0, 202410_1342_1342_0] 2024.10.22 02:14:13.812943 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.96 MiB. 2024.10.22 02:14:13.980900 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_811_811_0 to 202410_818_818_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:14.010849 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1223245 2024.10.22 02:14:14.024040 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_808_152 to 202410_809_814_1 2024.10.22 02:14:14.038821 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:14.044532 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:14.067557 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_814_153} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_808_152 to 202410_809_814_1 into Wide with storage Full 2024.10.22 02:14:14.207072 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_814_153} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:14.218981 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_814_153} MergeTreeSequentialSource: Reading 150 marks from part 202410_1_808_152, total 1220280 rows starting from the beginning of the part 2024.10.22 02:14:14.267836 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_814_153} MergeTreeSequentialSource: Reading 3 marks from part 202410_809_814_1, total 10010 rows starting from the beginning of the part 2024.10.22 02:14:14.306711 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 5.04 MiB, will set to 1.56 GiB (RSS), difference: 7.20 MiB 2024.10.22 02:14:14.939158 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 94882 2024.10.22 02:14:15.044893 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 107 entries to flush up to offset 92423 2024.10.22 02:14:15.067548 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_814_153/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:14:15.260641 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:15.262808 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:15.371474 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:15.376500 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:15.440263 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1428_1428_0 (state Temporary) 2024.10.22 02:14:15.449402 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_833_833_0 (state Temporary) 2024.10.22 02:14:15.502860 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_833_833_0 to 202410_839_839_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:15.561395 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92423 2024.10.22 02:14:15.569192 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1428_1428_0 to 202410_1441_1441_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:15.649801 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94882 2024.10.22 02:14:15.876615 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 5.59 MiB, will set to 1.57 GiB (RSS), difference: -5.19 MiB 2024.10.22 02:14:16.219092 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 4194304 bytes for 8f14a51421bbd7059452f863bb190e22:0) 2024.10.22 02:14:16.247232 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 08cf9264ad334000ebeafa9306abe0b2, offset: 0, size: 1429468 2024.10.22 02:14:16.253172 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 08cf9264ad334000ebeafa9306abe0b2 for removal 2024.10.22 02:14:16.256350 [ 1476 ] {} CacheMetadata: Key 08cf9264ad334000ebeafa9306abe0b2 is removed from metadata 2024.10.22 02:14:16.257261 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9984c4471e336290b002d97b63e1089a, offset: 0, size: 651 2024.10.22 02:14:16.260067 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 9984c4471e336290b002d97b63e1089a for removal 2024.10.22 02:14:16.364746 [ 1476 ] {} CacheMetadata: Key 9984c4471e336290b002d97b63e1089a is removed from metadata 2024.10.22 02:14:16.365878 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 48e2b9565d473cc0cc4b01baaf124c6c, offset: 0, size: 2576718 2024.10.22 02:14:16.368282 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 48e2b9565d473cc0cc4b01baaf124c6c for removal 2024.10.22 02:14:16.371384 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a2f761e2c0d311319af91e5da96c6d76, offset: 0, size: 47662 2024.10.22 02:14:16.372190 [ 1476 ] {} CacheMetadata: Key 48e2b9565d473cc0cc4b01baaf124c6c is removed from metadata 2024.10.22 02:14:16.375955 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a2f761e2c0d311319af91e5da96c6d76 for removal 2024.10.22 02:14:16.382701 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 630ad6d6ac2d9135ce5be44a6bdb36c2, offset: 0, size: 797 2024.10.22 02:14:16.383344 [ 1476 ] {} CacheMetadata: Key a2f761e2c0d311319af91e5da96c6d76 is removed from metadata 2024.10.22 02:14:16.389421 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 630ad6d6ac2d9135ce5be44a6bdb36c2 for removal 2024.10.22 02:14:16.395223 [ 1476 ] {} CacheMetadata: Key 630ad6d6ac2d9135ce5be44a6bdb36c2 is removed from metadata 2024.10.22 02:14:16.411675 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a0ffb58271b2eba1c12b2e193d3c857f, offset: 0, size: 883 2024.10.22 02:14:16.419589 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a0ffb58271b2eba1c12b2e193d3c857f for removal 2024.10.22 02:14:16.506511 [ 1476 ] {} CacheMetadata: Key a0ffb58271b2eba1c12b2e193d3c857f is removed from metadata 2024.10.22 02:14:17.324117 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_748_142, 202410_749_754_1] 2024.10.22 02:14:17.328135 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_748_142, 202410_749_754_1] 2024.10.22 02:14:17.345402 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_748_142, 202410_749_754_1] 2024.10.22 02:14:17.360358 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 10.59 MiB, will set to 1.58 GiB (RSS), difference: 7.31 MiB 2024.10.22 02:14:17.563322 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_170_170_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:14:18.664501 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 704 2024.10.22 02:14:19.071445 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 12915 2024.10.22 02:14:19.254473 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:19.256318 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:19.256561 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:19.259982 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:19.322677 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_686_686_0 (state Temporary) 2024.10.22 02:14:19.476340 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_686_686_0 to 202410_696_696_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:19.510383 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_848_848_0 (state Temporary) 2024.10.22 02:14:19.516230 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 2262876 bytes for 8f14a51421bbd7059452f863bb190e22:0) 2024.10.22 02:14:19.523519 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: df0f7b02353c159b75ae5ab4232f6eed, offset: 0, size: 2297058 2024.10.22 02:14:19.536076 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key df0f7b02353c159b75ae5ab4232f6eed for removal 2024.10.22 02:14:19.541808 [ 1476 ] {} CacheMetadata: Key df0f7b02353c159b75ae5ab4232f6eed is removed from metadata 2024.10.22 02:14:19.544684 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 335a203d85331f2fd74893f569dccaee, offset: 0, size: 877 2024.10.22 02:14:19.552259 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 335a203d85331f2fd74893f569dccaee for removal 2024.10.22 02:14:19.566618 [ 1476 ] {} CacheMetadata: Key 335a203d85331f2fd74893f569dccaee is removed from metadata 2024.10.22 02:14:19.569110 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_848_848_0 to 202410_861_861_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:19.642381 [ 1261 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_856_225 to 202410_857_857_0 2024.10.22 02:14:19.655932 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 94938 2024.10.22 02:14:19.657555 [ 1261 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:19.660474 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.78 GiB. 2024.10.22 02:14:19.684586 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_857_226} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_856_225 to 202410_857_857_0 into Compact with storage Full 2024.10.22 02:14:19.736717 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12915 2024.10.22 02:14:19.789436 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 12.36 MiB, will set to 1.58 GiB (RSS), difference: 9.83 MiB 2024.10.22 02:14:19.832540 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 704 2024.10.22 02:14:19.850326 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_857_226} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:19.965427 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_857_226} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_856_225, total 12941 rows starting from the beginning of the part 2024.10.22 02:14:19.990944 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_857_226} MergeTreeSequentialSource: Reading 2 marks from part 202410_857_857_0, total 21 rows starting from the beginning of the part 2024.10.22 02:14:19.990563 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_170_170_0 to 202410_173_173_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:20.187566 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:20.194487 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:20.262670 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Age 2024.10.22 02:14:20.408267 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Age 2024.10.22 02:14:20.424860 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1429_1429_0 (state Temporary) 2024.10.22 02:14:20.457797 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Age 2024.10.22 02:14:20.613840 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1429_1429_0 to 202410_1442_1442_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:20.701839 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Age 2024.10.22 02:14:20.791816 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94938 2024.10.22 02:14:20.829596 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Age 2024.10.22 02:14:20.873592 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Age 2024.10.22 02:14:20.891047 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Age 2024.10.22 02:14:20.958757 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fhv/volaoubivqrpucfyenyqqidjksktt (LogSeriesLimiter: on interval from 2024-10-22 02:14:04 to 2024-10-22 02:14:20 accepted series 1 / 80 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:14:20.974631 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_857_226} MergeTask::MergeProjectionsStage: Merge sorted 12962 rows, containing 12 columns (12 merged, 0 gathered) in 1.307005058 sec., 9917.329638979865 rows/sec., 1.55 MiB/sec. 2024.10.22 02:14:21.017042 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1225170 2024.10.22 02:14:21.051831 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_857_226} MergedBlockOutputStream: filled checksums 202410_1_857_226 (state Temporary) 2024.10.22 02:14:21.121521 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_797_186, 202410_798_798_0] 2024.10.22 02:14:21.128501 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_797_186, 202410_798_798_0] 2024.10.22 02:14:21.251507 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_797_186, 202410_798_798_0] 2024.10.22 02:14:21.289007 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:14:21.296781 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:14:21.410993 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_857_226} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_857_226 to 202410_1_857_226 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:21.440157 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_857_226} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_856_225, 202410_857_857_0] -> 202410_1_857_226 2024.10.22 02:14:21.459478 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:14:21.599827 [ 1150 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:14:21.670973 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 13.18 MiB, will set to 1.58 GiB (RSS), difference: 8.91 MiB 2024.10.22 02:14:21.679909 [ 1150 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:14:21.971562 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:21.974527 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:22.111160 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_812_812_0 (state Temporary) 2024.10.22 02:14:22.166355 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:14:22.175517 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_643_131, 202410_644_644_0, 202410_645_645_0, 202410_646_646_0, 202410_647_647_0, 202410_648_648_0] 2024.10.22 02:14:22.177765 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_643_131, 202410_644_644_0, 202410_645_645_0, 202410_646_646_0, 202410_647_647_0, 202410_648_648_0] 2024.10.22 02:14:22.184078 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_812_812_0 to 202410_819_819_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:22.271602 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1225170 2024.10.22 02:14:22.304604 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_643_131, 202410_644_644_0, 202410_645_645_0, 202410_646_646_0, 202410_647_647_0, 202410_648_648_0] 2024.10.22 02:14:23.079347 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 62 entries to flush up to offset 92485 2024.10.22 02:14:23.202663 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6849 2024.10.22 02:14:23.308075 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:23.310521 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:23.348366 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_834_834_0 (state Temporary) 2024.10.22 02:14:23.386193 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_834_834_0 to 202410_840_840_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:23.421259 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 15.23 MiB, will set to 1.59 GiB (RSS), difference: 7.81 MiB 2024.10.22 02:14:23.769496 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92485 2024.10.22 02:14:24.597110 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 5 old parts to remove. Parts: [202410_155_155_0, 202410_156_156_0, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0] 2024.10.22 02:14:24.599918 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from filesystem (serially): Parts: [202410_155_155_0, 202410_156_156_0, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0] 2024.10.22 02:14:24.643017 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from memory: Parts: [202410_155_155_0, 202410_156_156_0, 202410_157_157_0, 202410_158_158_0, 202410_159_159_0] 2024.10.22 02:14:24.806738 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 94996 2024.10.22 02:14:25.236281 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:25.243007 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:25.314531 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1430_1430_0 (state Temporary) 2024.10.22 02:14:25.354831 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1430_1430_0 to 202410_1443_1443_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:25.429397 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 94996 2024.10.22 02:14:25.760009 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 14.29 MiB, will set to 1.60 GiB (RSS), difference: 8.65 MiB 2024.10.22 02:14:25.782351 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_814_153/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:14:27.241595 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 12925 2024.10.22 02:14:27.336357 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 705 2024.10.22 02:14:27.462770 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:27.477388 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:27.697028 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:27.710745 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:27.711643 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 13.52 MiB, will set to 1.60 GiB (RSS), difference: 1.34 MiB 2024.10.22 02:14:27.713222 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_849_849_0 (state Temporary) 2024.10.22 02:14:27.886456 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_687_687_0 (state Temporary) 2024.10.22 02:14:27.960492 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_849_849_0 to 202410_862_862_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:27.977460 [ 1192 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_857_226 to 202410_858_858_0 2024.10.22 02:14:27.986751 [ 1192 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:27.990931 [ 1192 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:28.084305 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_858_227} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_857_226 to 202410_858_858_0 into Compact with storage Full 2024.10.22 02:14:28.099290 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_687_687_0 to 202410_697_697_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:28.128701 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12925 2024.10.22 02:14:28.311871 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_858_227} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:28.380359 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_858_227} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_857_226, total 12962 rows starting from the beginning of the part 2024.10.22 02:14:28.558929 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_858_227} MergeTreeSequentialSource: Reading 2 marks from part 202410_858_858_0, total 17 rows starting from the beginning of the part 2024.10.22 02:14:28.693118 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 705 2024.10.22 02:14:29.078748 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_777_6, 202410_778_778_0, 202410_779_779_0, 202410_780_780_0, 202410_781_781_0, 202410_782_782_0] 2024.10.22 02:14:29.080988 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_777_6, 202410_778_778_0, 202410_779_779_0, 202410_780_780_0, 202410_781_781_0, 202410_782_782_0] 2024.10.22 02:14:29.142233 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_777_6, 202410_778_778_0, 202410_779_779_0, 202410_780_780_0, 202410_781_781_0, 202410_782_782_0] 2024.10.22 02:14:29.379926 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1226710 2024.10.22 02:14:29.434326 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_798_187, 202410_799_799_0, 202410_800_800_0, 202410_801_801_0, 202410_802_802_0, 202410_803_803_0] 2024.10.22 02:14:29.439365 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 32 entries to flush up to offset 95028 2024.10.22 02:14:29.442851 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_798_187, 202410_799_799_0, 202410_800_800_0, 202410_801_801_0, 202410_802_802_0, 202410_803_803_0] 2024.10.22 02:14:29.460454 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_798_187, 202410_799_799_0, 202410_800_800_0, 202410_801_801_0, 202410_802_802_0, 202410_803_803_0] 2024.10.22 02:14:29.556326 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 12.80 MiB, will set to 1.60 GiB (RSS), difference: -8.74 MiB 2024.10.22 02:14:29.768364 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:29.771476 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:29.824690 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1431_1431_0 (state Temporary) 2024.10.22 02:14:29.864748 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1431_1431_0 to 202410_1444_1444_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:29.872091 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_858_227} MergeTask::MergeProjectionsStage: Merge sorted 12979 rows, containing 12 columns (12 merged, 0 gathered) in 1.827892785 sec., 7100.525865908487 rows/sec., 1.11 MiB/sec. 2024.10.22 02:14:29.947857 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_858_227} MergedBlockOutputStream: filled checksums 202410_1_858_227 (state Temporary) 2024.10.22 02:14:30.045770 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95028 2024.10.22 02:14:30.345722 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_858_227} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_858_227 to 202410_1_858_227 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:30.352000 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_858_227} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_857_226, 202410_858_858_0] -> 202410_1_858_227 2024.10.22 02:14:30.372218 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:14:30.433690 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:30.440970 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:30.732698 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 6889 2024.10.22 02:14:30.767805 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_813_813_0 (state Temporary) 2024.10.22 02:14:30.859139 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_813_813_0 to 202410_820_820_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:30.887184 [ 1208 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_815_815_0 to 202410_820_820_0 2024.10.22 02:14:30.898392 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1226710 2024.10.22 02:14:30.904848 [ 1208 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:30.908484 [ 1208 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:30.937590 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_815_815_0 to 202410_820_820_0 into Compact with storage Full 2024.10.22 02:14:30.968863 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:30.978185 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_815_815_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:14:30.985657 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_816_816_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:14:31.022688 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_817_817_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:14:31.038180 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_818_818_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:14:31.076076 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_819_819_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:14:31.115794 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_820_820_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:14:31.275967 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 63 entries to flush up to offset 92548 2024.10.22 02:14:31.289212 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:14:31.291339 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:14:31.397028 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:31.400815 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:31.546267 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_835_835_0 (state Temporary) 2024.10.22 02:14:31.809083 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_835_835_0 to 202410_841_841_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:31.895429 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 15.97 MiB, will set to 1.60 GiB (RSS), difference: -1.21 MiB 2024.10.22 02:14:32.147263 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92548 2024.10.22 02:14:32.815935 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} MergeTask::MergeProjectionsStage: Merge sorted 10010 rows, containing 4 columns (4 merged, 0 gathered) in 1.886818607 sec., 5305.226460489321 rows/sec., 82.89 KiB/sec. 2024.10.22 02:14:32.974190 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} MergedBlockOutputStream: filled checksums 202410_815_820_1 (state Temporary) 2024.10.22 02:14:33.182188 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_815_820_1 to 202410_815_820_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:33.191077 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_815_820_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_815_815_0, 202410_820_820_0] -> 202410_815_820_1 2024.10.22 02:14:33.265832 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:14:33.601108 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 13.00 MiB, will set to 1.60 GiB (RSS), difference: 11.86 MiB 2024.10.22 02:14:33.976932 [ 1209 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 02:14:34.051517 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 95076 2024.10.22 02:14:34.375337 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:34.379578 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:34.542736 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1432_1432_0 (state Temporary) 2024.10.22 02:14:34.567733 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1432_1432_0 to 202410_1445_1445_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:34.616395 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95076 2024.10.22 02:14:34.703398 [ 1175 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1207_1440_48 to 202410_1443_1443_0 2024.10.22 02:14:34.712925 [ 1175 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:34.715887 [ 1175 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.77 GiB. 2024.10.22 02:14:34.760538 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1443_49} MergeTask::PrepareStage: Merging 4 parts: from 202410_1207_1440_48 to 202410_1443_1443_0 into Compact with storage Full 2024.10.22 02:14:34.965325 [ 1272 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 02:14:35.028368 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1443_49} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:35.050448 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1443_49} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1440_48, total 15687 rows starting from the beginning of the part 2024.10.22 02:14:35.179348 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1443_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_1441_1441_0, total 54 rows starting from the beginning of the part 2024.10.22 02:14:35.215099 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1443_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_1442_1442_0, total 56 rows starting from the beginning of the part 2024.10.22 02:14:35.416868 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.47 MiB, will set to 1.60 GiB (RSS), difference: 31.89 MiB 2024.10.22 02:14:35.471824 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1443_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_1443_1443_0, total 58 rows starting from the beginning of the part 2024.10.22 02:14:35.588863 [ 1198 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 218 ms. 2024.10.22 02:14:35.630626 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 12941 2024.10.22 02:14:35.745668 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:35.848702 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.76 GiB. 2024.10.22 02:14:35.879932 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_850_850_0 (state Temporary) 2024.10.22 02:14:36.199397 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 706 2024.10.22 02:14:36.259122 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_850_850_0 to 202410_863_863_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:36.335299 [ 1248 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_858_227 to 202410_859_859_0 2024.10.22 02:14:36.343124 [ 1744 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fhv/volaoubivqrpucfyenyqqidjksktt, size 867453 2024.10.22 02:14:36.374882 [ 1248 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:36.375672 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_814_153/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:14:36.379839 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12941 2024.10.22 02:14:36.384808 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 02:14:36.496348 [ 1248 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 201 ms. 2024.10.22 02:14:36.689784 [ 1266 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:14:36.698381 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_859_228} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_858_227 to 202410_859_859_0 into Compact with storage Full 2024.10.22 02:14:36.856678 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:36.858874 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 02:14:36.861576 [ 1266 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:14:36.953732 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_859_228} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:36.980322 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_859_228} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_858_227, total 12979 rows starting from the beginning of the part 2024.10.22 02:14:37.032123 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_688_688_0 (state Temporary) 2024.10.22 02:14:37.083203 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_859_228} MergeTreeSequentialSource: Reading 2 marks from part 202410_859_859_0, total 13 rows starting from the beginning of the part 2024.10.22 02:14:37.170816 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_688_688_0 to 202410_698_698_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:37.239101 [ 1274 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_693_141 to 202410_698_698_0 2024.10.22 02:14:37.271317 [ 1274 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:37.276239 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 706 2024.10.22 02:14:37.387541 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Sex 2024.10.22 02:14:37.275840 [ 1274 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.75 GiB. 2024.10.22 02:14:37.418600 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Sex 2024.10.22 02:14:37.435329 [ 1274 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 02:14:37.567743 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_693_141 to 202410_698_698_0 into Compact with storage Full 2024.10.22 02:14:37.568937 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Sex 2024.10.22 02:14:37.595369 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Sex 2024.10.22 02:14:37.631981 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Sex 2024.10.22 02:14:37.652531 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Sex 2024.10.22 02:14:37.675715 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Sex 2024.10.22 02:14:37.752150 [ 1748 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ylb/cwcmoewznqbtjsajgwobpgagwlyho, size 1338 2024.10.22 02:14:37.753349 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:37.772968 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_693_141, total 737 rows starting from the beginning of the part 2024.10.22 02:14:37.821397 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 9.59 MiB, will set to 1.59 GiB (RSS), difference: 92.30 MiB 2024.10.22 02:14:37.862679 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_694_694_0, total 1 rows starting from the beginning of the part 2024.10.22 02:14:37.906363 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1228250 2024.10.22 02:14:37.923201 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ooc/fogsaumzqwtfibyjtpijkolopbodw (LogSeriesLimiter: on interval from 2024-10-22 02:14:20 to 2024-10-22 02:14:37 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:14:38.180687 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1443_49} MergeTask::MergeProjectionsStage: Merge sorted 15855 rows, containing 13 columns (13 merged, 0 gathered) in 3.425777201 sec., 4628.146861206226 rows/sec., 919.83 KiB/sec. 2024.10.22 02:14:38.267030 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1443_49/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:14:38.274526 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_695_695_0, total 1 rows starting from the beginning of the part 2024.10.22 02:14:38.341241 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_859_228} MergeTask::MergeProjectionsStage: Merge sorted 12992 rows, containing 12 columns (12 merged, 0 gathered) in 1.591599881 sec., 8162.855599007173 rows/sec., 1.27 MiB/sec. 2024.10.22 02:14:38.280261 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1342_27, 202410_1343_1343_0, 202410_1344_1344_0, 202410_1345_1345_0, 202410_1346_1346_0, 202410_1347_1347_0] 2024.10.22 02:14:38.346119 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1342_27, 202410_1343_1343_0, 202410_1344_1344_0, 202410_1345_1345_0, 202410_1346_1346_0, 202410_1347_1347_0] 2024.10.22 02:14:38.395187 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1342_27, 202410_1343_1343_0, 202410_1344_1344_0, 202410_1345_1345_0, 202410_1346_1346_0, 202410_1347_1347_0] 2024.10.22 02:14:38.445595 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_859_228} MergedBlockOutputStream: filled checksums 202410_1_859_228 (state Temporary) 2024.10.22 02:14:38.469142 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1443_49} MergedBlockOutputStream: filled checksums 202410_1207_1443_49 (state Temporary) 2024.10.22 02:14:38.541343 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_696_696_0, total 1 rows starting from the beginning of the part 2024.10.22 02:14:38.626978 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 95147 2024.10.22 02:14:38.648467 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_697_697_0, total 1 rows starting from the beginning of the part 2024.10.22 02:14:38.675413 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1443_49} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1443_49 to 202410_1207_1443_49 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:38.683177 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1443_49} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1207_1440_48, 202410_1443_1443_0] -> 202410_1207_1443_49 2024.10.22 02:14:38.785176 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.89 MiB. 2024.10.22 02:14:38.860804 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_698_698_0, total 1 rows starting from the beginning of the part 2024.10.22 02:14:38.953809 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:38.966998 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.74 GiB. 2024.10.22 02:14:39.008637 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:39.009580 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_859_228} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_859_228 to 202410_1_859_228 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:39.013276 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_859_228} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_858_227, 202410_859_859_0] -> 202410_1_859_228 2024.10.22 02:14:39.013250 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.74 GiB. 2024.10.22 02:14:39.161183 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:14:39.208074 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1433_1433_0 (state Temporary) 2024.10.22 02:14:39.274109 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1433_1433_0 to 202410_1446_1446_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:39.362838 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95147 2024.10.22 02:14:39.371297 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_814_814_0 (state Temporary) 2024.10.22 02:14:39.659074 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 121 entries to flush up to offset 92669 2024.10.22 02:14:39.660917 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_814_814_0 to 202410_821_821_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:39.718739 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1228250 2024.10.22 02:14:39.900779 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:39.916764 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 2.00 GiB, free memory in arenas 11.88 MiB, will set to 1.58 GiB (RSS), difference: 146.13 MiB 2024.10.22 02:14:39.919667 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.72 GiB. 2024.10.22 02:14:39.977394 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_836_836_0 (state Temporary) 2024.10.22 02:14:40.049800 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_698_142/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:14:40.202510 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_836_836_0 to 202410_842_842_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:40.287905 [ 1218 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_837_18 to 202410_842_842_0 2024.10.22 02:14:40.340528 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92669 2024.10.22 02:14:40.433700 [ 1218 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:40.439485 [ 1218 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.70 GiB. 2024.10.22 02:14:40.469475 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_837_18 to 202410_842_842_0 into Compact with storage Full 2024.10.22 02:14:40.665985 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:40.691022 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_837_18, total 8862 rows starting from the beginning of the part 2024.10.22 02:14:40.742805 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_16_16_0 (state Temporary) 2024.10.22 02:14:40.746654 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_838_838_0, total 99 rows starting from the beginning of the part 2024.10.22 02:14:40.764234 [ 1747 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xqk/hyabxbwxmwfyleknxnziaemrmzdol, size 717 2024.10.22 02:14:40.886285 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_839_839_0, total 107 rows starting from the beginning of the part 2024.10.22 02:14:40.933150 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_840_840_0, total 62 rows starting from the beginning of the part 2024.10.22 02:14:40.970172 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_841_841_0, total 63 rows starting from the beginning of the part 2024.10.22 02:14:41.032634 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} MergeTreeSequentialSource: Reading 2 marks from part 202410_842_842_0, total 121 rows starting from the beginning of the part 2024.10.22 02:14:41.288910 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:14:41.308675 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:14:41.517421 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} MergeTask::MergeProjectionsStage: Merge sorted 742 rows, containing 32 columns (32 merged, 0 gathered) in 3.958429558 sec., 187.44807483069022 rows/sec., 16.93 KiB/sec. 2024.10.22 02:14:41.564862 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} MergedBlockOutputStream: filled checksums 202410_1_698_142 (state Temporary) 2024.10.22 02:14:41.581416 [ 1154 ] {} DatabaseCatalog: Execution took 218 ms. 2024.10.22 02:14:41.958529 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_698_142 to 202410_1_698_142 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:41.971399 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_698_142} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_693_141, 202410_698_698_0] -> 202410_1_698_142 2024.10.22 02:14:42.014813 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:14:42.660560 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 2.00 GiB, free memory in arenas 10.20 MiB, will set to 1.54 GiB (RSS), difference: 60.35 MiB 2024.10.22 02:14:43.332744 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} MergeTask::MergeProjectionsStage: Merge sorted 9314 rows, containing 13 columns (13 merged, 0 gathered) in 2.86736461 sec., 3248.2789135072708 rows/sec., 1.41 MiB/sec. 2024.10.22 02:14:43.376329 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 95197 2024.10.22 02:14:43.500156 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} MergedBlockOutputStream: filled checksums 202410_747_842_19 (state Temporary) 2024.10.22 02:14:43.725734 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:43.732135 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 02:14:43.862527 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_842_19 to 202410_747_842_19 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:43.864834 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_842_19} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_837_18, 202410_842_842_0] -> 202410_747_842_19 2024.10.22 02:14:43.891199 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 12968 2024.10.22 02:14:43.941518 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1434_1434_0 (state Temporary) 2024.10.22 02:14:43.940674 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:14:43.942625 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.42 MiB. 2024.10.22 02:14:44.181221 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:44.187445 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 02:14:44.209536 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1434_1434_0 to 202410_1447_1447_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:44.252097 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_851_851_0 (state Temporary) 2024.10.22 02:14:44.332423 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_851_851_0 to 202410_864_864_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:44.366788 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95197 2024.10.22 02:14:44.355795 [ 1170 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_859_228 to 202410_860_860_0 2024.10.22 02:14:44.371346 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12968 2024.10.22 02:14:44.371619 [ 1170 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:44.374726 [ 1170 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 02:14:44.456082 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_860_229} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_859_228 to 202410_860_860_0 into Compact with storage Full 2024.10.22 02:14:44.463948 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 11.25 MiB, will set to 1.53 GiB (RSS), difference: 2.71 MiB 2024.10.22 02:14:44.582453 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_860_229} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:44.684523 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.084983793 sec. 2024.10.22 02:14:44.780063 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 707 2024.10.22 02:14:44.800656 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_860_229} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_859_228, total 12992 rows starting from the beginning of the part 2024.10.22 02:14:44.851505 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_860_229} MergeTreeSequentialSource: Reading 2 marks from part 202410_860_860_0, total 18 rows starting from the beginning of the part 2024.10.22 02:14:45.093629 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:45.098315 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 02:14:45.266803 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_689_689_0 (state Temporary) 2024.10.22 02:14:45.353649 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_689_689_0 to 202410_699_699_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:45.472732 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 707 2024.10.22 02:14:45.744063 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_860_229} MergeTask::MergeProjectionsStage: Merge sorted 13010 rows, containing 12 columns (12 merged, 0 gathered) in 1.306107133 sec., 9960.898054447727 rows/sec., 1.55 MiB/sec. 2024.10.22 02:14:45.806949 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_860_229} MergedBlockOutputStream: filled checksums 202410_1_860_229 (state Temporary) 2024.10.22 02:14:46.462536 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_860_229} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_860_229 to 202410_1_860_229 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:46.465180 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_860_229} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_859_228, 202410_860_860_0] -> 202410_1_860_229 2024.10.22 02:14:46.622959 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:14:46.729616 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 1229792 2024.10.22 02:14:47.855876 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 94 entries to flush up to offset 92763 2024.10.22 02:14:48.162996 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:48.166244 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 02:14:48.215923 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_837_837_0 (state Temporary) 2024.10.22 02:14:48.268809 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_837_837_0 to 202410_843_843_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:48.354658 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:48.366276 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 02:14:48.471629 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 95241 2024.10.22 02:14:48.531430 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_814_153/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:14:48.608094 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_815_815_0 (state Temporary) 2024.10.22 02:14:48.725140 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:48.733581 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 02:14:48.756010 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92763 2024.10.22 02:14:48.821789 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_815_815_0 to 202410_822_822_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:48.867384 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1229792 2024.10.22 02:14:48.873516 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1435_1435_0 (state Temporary) 2024.10.22 02:14:48.943776 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1435_1435_0 to 202410_1448_1448_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:48.971239 [ 1193 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1207_1443_49 to 202410_1446_1446_0 2024.10.22 02:14:48.981319 [ 1193 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:48.989321 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.69 GiB. 2024.10.22 02:14:48.996268 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/vgi/djselzodcomnpacpzggvbbyhygsgv. 2024.10.22 02:14:49.008517 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95241 2024.10.22 02:14:49.051762 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1446_50} MergeTask::PrepareStage: Merging 4 parts: from 202410_1207_1443_49 to 202410_1446_1446_0 into Compact with storage Full 2024.10.22 02:14:49.073388 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1446_50} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:49.087079 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1446_50} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1443_49, total 15855 rows starting from the beginning of the part 2024.10.22 02:14:49.104367 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1446_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_1444_1444_0, total 32 rows starting from the beginning of the part 2024.10.22 02:14:49.216581 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1446_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_1445_1445_0, total 48 rows starting from the beginning of the part 2024.10.22 02:14:49.249838 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1446_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_1446_1446_0, total 71 rows starting from the beginning of the part 2024.10.22 02:14:50.667043 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1446_50} MergeTask::MergeProjectionsStage: Merge sorted 16006 rows, containing 13 columns (13 merged, 0 gathered) in 1.623917441 sec., 9856.412398738441 rows/sec., 1.91 MiB/sec. 2024.10.22 02:14:50.740954 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 11.14 MiB, will set to 1.54 GiB (RSS), difference: -2.12 MiB 2024.10.22 02:14:50.872590 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1446_50} MergedBlockOutputStream: filled checksums 202410_1207_1446_50 (state Temporary) 2024.10.22 02:14:51.120621 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_814_153} MergeTask::MergeProjectionsStage: Merge sorted 1230290 rows, containing 4 columns (4 merged, 0 gathered) in 37.058035329 sec., 33199.00769367632 rows/sec., 497.96 KiB/sec. 2024.10.22 02:14:51.200144 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_814_153} MergedBlockOutputStream: filled checksums 202410_1_814_153 (state Temporary) 2024.10.22 02:14:51.295189 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:14:51.302875 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:14:51.504616 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1446_50} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1446_50 to 202410_1207_1446_50 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:51.510619 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1446_50} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1207_1443_49, 202410_1446_1446_0] -> 202410_1207_1446_50 2024.10.22 02:14:51.559149 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.89 MiB. 2024.10.22 02:14:51.813418 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_761_761_0, 202410_762_762_0, 202410_763_763_0, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0] 2024.10.22 02:14:51.815300 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_761_761_0, 202410_762_762_0, 202410_763_763_0, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0] 2024.10.22 02:14:51.839726 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:51.846628 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:14:51.859232 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_761_761_0, 202410_762_762_0, 202410_763_763_0, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0] 2024.10.22 02:14:51.866389 [ 1210 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:14:51.873609 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 12982 2024.10.22 02:14:51.897509 [ 1210 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:14:51.953610 [ 1208 ] {} DatabaseCatalog: Execution took 371 ms. 2024.10.22 02:14:52.333420 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:52.339563 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:14:52.380000 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_852_852_0 (state Temporary) 2024.10.22 02:14:52.551484 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 8.41 MiB, will set to 1.53 GiB (RSS), difference: 5.34 MiB 2024.10.22 02:14:52.649128 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_852_852_0 to 202410_865_865_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:52.665299 [ 1227 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_860_229 to 202410_861_861_0 2024.10.22 02:14:52.675294 [ 1227 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:52.680796 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:14:52.800277 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12982 2024.10.22 02:14:52.803368 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_861_230} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_860_229 to 202410_861_861_0 into Compact with storage Full 2024.10.22 02:14:52.970589 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_861_230} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:14:52.982700 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_861_230} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_860_229, total 13010 rows starting from the beginning of the part 2024.10.22 02:14:53.018652 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_861_230} MergeTreeSequentialSource: Reading 2 marks from part 202410_861_861_0, total 11 rows starting from the beginning of the part 2024.10.22 02:14:53.028052 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 95294 2024.10.22 02:14:53.075568 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 708 2024.10.22 02:14:53.268075 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:53.270412 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:14:53.274478 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:53.276997 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:14:53.355410 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_690_690_0 (state Temporary) 2024.10.22 02:14:53.440569 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1436_1436_0 (state Temporary) 2024.10.22 02:14:53.580383 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 717000 bytes for 0b07ceda215364972736011920085723:0) 2024.10.22 02:14:53.588537 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1436_1436_0 to 202410_1449_1449_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:53.591976 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: b05b8be1fed58e2286a5400cf2247b40, offset: 0, size: 2641034 2024.10.22 02:14:53.599752 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key b05b8be1fed58e2286a5400cf2247b40 for removal 2024.10.22 02:14:53.602259 [ 1476 ] {} CacheMetadata: Key b05b8be1fed58e2286a5400cf2247b40 is removed from metadata 2024.10.22 02:14:53.603272 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3d02f99a97b65a3eec68f117662c8be2, offset: 0, size: 883 2024.10.22 02:14:53.608076 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 3d02f99a97b65a3eec68f117662c8be2 for removal 2024.10.22 02:14:53.625568 [ 1476 ] {} CacheMetadata: Key 3d02f99a97b65a3eec68f117662c8be2 is removed from metadata 2024.10.22 02:14:53.631058 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_690_690_0 to 202410_700_700_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:53.659471 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95294 2024.10.22 02:14:53.695846 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 708 2024.10.22 02:14:53.781188 [ 1713 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ooc/fogsaumzqwtfibyjtpijkolopbodw, size 717000 2024.10.22 02:14:54.029304 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Income 2024.10.22 02:14:54.039310 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Income 2024.10.22 02:14:54.063702 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Income 2024.10.22 02:14:54.077150 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Income 2024.10.22 02:14:54.102443 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Income 2024.10.22 02:14:54.121944 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Income 2024.10.22 02:14:54.143588 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Income 2024.10.22 02:14:54.158588 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_861_230} MergeTask::MergeProjectionsStage: Merge sorted 13021 rows, containing 12 columns (12 merged, 0 gathered) in 1.375815174 sec., 9464.20729039001 rows/sec., 1.48 MiB/sec. 2024.10.22 02:14:54.194087 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hvy/fjfkwwbzmdgxnygduokwlrjlktahi (LogSeriesLimiter: on interval from 2024-10-22 02:14:37 to 2024-10-22 02:14:54 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:14:54.221008 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_861_230} MergedBlockOutputStream: filled checksums 202410_1_861_230 (state Temporary) 2024.10.22 02:14:54.349407 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_814_153} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_814_153 to 202410_1_814_153 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:54.359719 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_814_153} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_808_152, 202410_809_814_1] -> 202410_1_814_153 2024.10.22 02:14:54.376960 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.55 MiB. 2024.10.22 02:14:54.380493 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 7.20 MiB, will set to 1.53 GiB (RSS), difference: -5.74 MiB 2024.10.22 02:14:54.519811 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_861_230} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_861_230 to 202410_1_861_230 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:54.528529 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_861_230} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_860_229, 202410_861_861_0] -> 202410_1_861_230 2024.10.22 02:14:54.657612 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:14:55.004169 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_171_171_0 (state Temporary) 2024.10.22 02:14:55.977584 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1231717 2024.10.22 02:14:56.268372 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 92842 2024.10.22 02:14:56.621080 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:56.625149 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:14:56.646008 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 6.82 MiB, will set to 1.53 GiB (RSS), difference: -1.91 MiB 2024.10.22 02:14:56.672293 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_838_838_0 (state Temporary) 2024.10.22 02:14:56.708786 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_838_838_0 to 202410_844_844_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:56.774737 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92842 2024.10.22 02:14:57.139353 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:57.143947 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:14:57.265285 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_816_816_0 (state Temporary) 2024.10.22 02:14:57.560320 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_816_816_0 to 202410_823_823_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:57.667765 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 95345 2024.10.22 02:14:57.691489 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1231717 2024.10.22 02:14:58.191772 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:14:58.198347 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:14:58.243155 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1437_1437_0 (state Temporary) 2024.10.22 02:14:58.391174 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1437_1437_0 to 202410_1450_1450_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:14:58.443349 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95345 2024.10.22 02:14:59.244914 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 251 ms. 2024.10.22 02:14:59.888216 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_171_171_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:15:00.302100 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 12994 2024.10.22 02:15:00.443029 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:00.446250 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:00.575519 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_853_853_0 (state Temporary) 2024.10.22 02:15:00.858783 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_853_853_0 to 202410_866_866_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:00.896600 [ 1257 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_861_230 to 202410_862_862_0 2024.10.22 02:15:00.904587 [ 1257 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:00.909187 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:00.912214 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xid/zntugubdeidjhueqzmntmopyvromd. 2024.10.22 02:15:01.053149 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 12994 2024.10.22 02:15:01.053212 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_862_231} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_861_230 to 202410_862_862_0 into Compact with storage Full 2024.10.22 02:15:01.078926 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_862_231} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:01.092898 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_862_231} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_861_230, total 13021 rows starting from the beginning of the part 2024.10.22 02:15:01.225125 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_862_231} MergeTreeSequentialSource: Reading 2 marks from part 202410_862_862_0, total 10 rows starting from the beginning of the part 2024.10.22 02:15:01.294550 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:15:01.298940 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:15:01.653042 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 5.65 MiB, will set to 1.53 GiB (RSS), difference: -7.96 MiB 2024.10.22 02:15:01.742543 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_171_171_0 to 202410_174_174_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:01.866180 [ 1261 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 7 parts from 202410_1_164_27 to 202410_174_174_0 2024.10.22 02:15:01.885323 [ 1261 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 3.26 MiB using storage policy from min volume index 0 2024.10.22 02:15:01.913674 [ 1261 ] {} DiskLocal: Reserved 3.26 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:01.968405 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} MergeTask::PrepareStage: Merging 7 parts: from 202410_1_164_27 to 202410_174_174_0 into Compact with storage Full 2024.10.22 02:15:02.029705 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_862_231} MergeTask::MergeProjectionsStage: Merge sorted 13031 rows, containing 12 columns (12 merged, 0 gathered) in 1.063125609 sec., 12257.25341359922 rows/sec., 1.91 MiB/sec. 2024.10.22 02:15:02.070336 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_862_231} MergedBlockOutputStream: filled checksums 202410_1_862_231 (state Temporary) 2024.10.22 02:15:02.397403 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_862_231} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_862_231 to 202410_1_862_231 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:02.404720 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_862_231} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_861_230, 202410_862_862_0] -> 202410_1_862_231 2024.10.22 02:15:02.455210 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:15:02.557093 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 95380 2024.10.22 02:15:02.854753 [ 1217 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 02:15:02.977198 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:02.980910 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:03.130961 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1438_1438_0 (state Temporary) 2024.10.22 02:15:03.273310 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1438_1438_0 to 202410_1451_1451_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:03.312353 [ 1224 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1446_50 to 202410_1447_1447_0 2024.10.22 02:15:03.344784 [ 1224 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:03.350656 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:03.368766 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 4.57 MiB, will set to 1.53 GiB (RSS), difference: 9.81 MiB 2024.10.22 02:15:03.381079 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1447_51} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1446_50 to 202410_1447_1447_0 into Compact with storage Full 2024.10.22 02:15:03.416584 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95380 2024.10.22 02:15:03.542225 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1447_51} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:03.549342 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1447_51} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1446_50, total 16006 rows starting from the beginning of the part 2024.10.22 02:15:03.571171 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1447_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_1447_1447_0, total 50 rows starting from the beginning of the part 2024.10.22 02:15:04.260415 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1347_28, 202410_1348_1348_0, 202410_1349_1349_0, 202410_1350_1350_0, 202410_1351_1351_0, 202410_1352_1352_0] 2024.10.22 02:15:04.263391 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1347_28, 202410_1348_1348_0, 202410_1349_1349_0, 202410_1350_1350_0, 202410_1351_1351_0, 202410_1352_1352_0] 2024.10.22 02:15:04.305802 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 64 entries to flush up to offset 92906 2024.10.22 02:15:04.407277 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1347_28, 202410_1348_1348_0, 202410_1349_1349_0, 202410_1350_1350_0, 202410_1351_1351_0, 202410_1352_1352_0] 2024.10.22 02:15:04.429119 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:04.479706 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6889 2024.10.22 02:15:04.533535 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:04.697262 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1233642 2024.10.22 02:15:04.783456 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_839_839_0 (state Temporary) 2024.10.22 02:15:04.844666 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_839_839_0 to 202410_845_845_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:04.998142 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 92906 2024.10.22 02:15:05.088441 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1447_51} MergeTask::MergeProjectionsStage: Merge sorted 16056 rows, containing 13 columns (13 merged, 0 gathered) in 1.719473689 sec., 9337.741020822332 rows/sec., 1.81 MiB/sec. 2024.10.22 02:15:05.469519 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1447_51} MergedBlockOutputStream: filled checksums 202410_1207_1447_51 (state Temporary) 2024.10.22 02:15:05.700495 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:05.705600 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:05.901659 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1447_51} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1447_51 to 202410_1207_1447_51 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:05.909684 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1447_51} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1446_50, 202410_1447_1447_0] -> 202410_1207_1447_51 2024.10.22 02:15:05.945793 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.87 MiB. 2024.10.22 02:15:06.044446 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_817_817_0 (state Temporary) 2024.10.22 02:15:06.244656 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_817_817_0 to 202410_824_824_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:06.273535 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1233642 2024.10.22 02:15:06.280047 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_814_153 to 202410_815_820_1 2024.10.22 02:15:06.289773 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:06.299891 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:06.345207 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_820_154} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_814_153 to 202410_815_820_1 into Wide with storage Full 2024.10.22 02:15:06.472709 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_820_154} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:06.479533 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_820_154} MergeTreeSequentialSource: Reading 152 marks from part 202410_1_814_153, total 1230290 rows starting from the beginning of the part 2024.10.22 02:15:06.615102 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_820_154} MergeTreeSequentialSource: Reading 3 marks from part 202410_815_820_1, total 10010 rows starting from the beginning of the part 2024.10.22 02:15:06.985465 [ 1170 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:15:07.016120 [ 1170 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:15:07.420308 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 95426 2024.10.22 02:15:07.565464 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 8.01 MiB, will set to 1.53 GiB (RSS), difference: -8.36 MiB 2024.10.22 02:15:07.744720 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:07.751908 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:07.807544 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1439_1439_0 (state Temporary) 2024.10.22 02:15:07.851090 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1439_1439_0 to 202410_1452_1452_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:07.868435 [ 1160 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1447_51 to 202410_1448_1448_0 2024.10.22 02:15:07.872540 [ 1160 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:07.876838 [ 1160 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:07.891941 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95426 2024.10.22 02:15:07.945396 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1448_52} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1447_51 to 202410_1448_1448_0 into Compact with storage Full 2024.10.22 02:15:07.963259 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1448_52} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:07.968047 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1448_52} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1447_51, total 16056 rows starting from the beginning of the part 2024.10.22 02:15:07.981203 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1448_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_1448_1448_0, total 44 rows starting from the beginning of the part 2024.10.22 02:15:08.220277 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:08.557350 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 13006 2024.10.22 02:15:08.666888 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:08.669240 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:08.698177 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 709 2024.10.22 02:15:08.717772 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_854_854_0 (state Temporary) 2024.10.22 02:15:08.790617 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_854_854_0 to 202410_867_867_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:08.855080 [ 1156 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_862_231 to 202410_863_863_0 2024.10.22 02:15:08.866472 [ 1156 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:08.868385 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:08.901477 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_863_232} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_862_231 to 202410_863_863_0 into Compact with storage Full 2024.10.22 02:15:09.023557 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_164_27, total 6528 rows starting from the beginning of the part 2024.10.22 02:15:09.041650 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13006 2024.10.22 02:15:09.066544 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_863_232} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:09.072517 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_863_232} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_862_231, total 13031 rows starting from the beginning of the part 2024.10.22 02:15:09.096068 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1448_52} MergeTask::MergeProjectionsStage: Merge sorted 16100 rows, containing 13 columns (13 merged, 0 gathered) in 1.15716366 sec., 13913.330116156603 rows/sec., 2.69 MiB/sec. 2024.10.22 02:15:09.109153 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_863_232} MergeTreeSequentialSource: Reading 2 marks from part 202410_863_863_0, total 16 rows starting from the beginning of the part 2024.10.22 02:15:09.208151 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:09.214658 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:09.277990 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_691_691_0 (state Temporary) 2024.10.22 02:15:09.371205 [ 1743 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hvy/fjfkwwbzmdgxnygduokwlrjlktahi, size 695210 2024.10.22 02:15:09.631925 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1448_52} MergedBlockOutputStream: filled checksums 202410_1207_1448_52 (state Temporary) 2024.10.22 02:15:09.663215 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_691_691_0 to 202410_701_701_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:09.699762 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 10.68 MiB, will set to 1.53 GiB (RSS), difference: -6.87 MiB 2024.10.22 02:15:09.755234 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 709 2024.10.22 02:15:10.035176 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1448_52} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1448_52 to 202410_1207_1448_52 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:10.040662 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1448_52} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1447_51, 202410_1448_1448_0] -> 202410_1207_1448_52 2024.10.22 02:15:10.145108 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_863_232} MergeTask::MergeProjectionsStage: Merge sorted 13047 rows, containing 12 columns (12 merged, 0 gathered) in 1.252057205 sec., 10420.450397871396 rows/sec., 1.63 MiB/sec. 2024.10.22 02:15:10.154035 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Interests 2024.10.22 02:15:10.162333 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Interests 2024.10.22 02:15:10.162318 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.87 MiB. 2024.10.22 02:15:10.173194 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_863_232} MergedBlockOutputStream: filled checksums 202410_1_863_232 (state Temporary) 2024.10.22 02:15:10.176242 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Interests 2024.10.22 02:15:10.209343 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Interests 2024.10.22 02:15:10.226932 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Interests 2024.10.22 02:15:10.337822 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Interests 2024.10.22 02:15:10.342002 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 02:15:10.409135 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_782_7, 202410_783_783_0, 202410_784_784_0, 202410_785_785_0, 202410_786_786_0, 202410_787_787_0] 2024.10.22 02:15:10.453576 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_782_7, 202410_783_783_0, 202410_784_784_0, 202410_785_785_0, 202410_786_786_0, 202410_787_787_0] 2024.10.22 02:15:10.465456 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Interests 2024.10.22 02:15:10.494846 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_782_7, 202410_783_783_0, 202410_784_784_0, 202410_785_785_0, 202410_786_786_0, 202410_787_787_0] 2024.10.22 02:15:10.515406 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zgm/aaodxddanhiqhxzlvdwzqklgyrttn (LogSeriesLimiter: on interval from 2024-10-22 02:14:54 to 2024-10-22 02:15:10 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:15:10.607951 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_863_232} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_863_232 to 202410_1_863_232 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:10.625606 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_863_232} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_862_231, 202410_863_863_0] -> 202410_1_863_232 2024.10.22 02:15:10.651211 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:15:11.037773 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_803_188, 202410_804_804_0] 2024.10.22 02:15:11.040450 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_803_188, 202410_804_804_0] 2024.10.22 02:15:11.053003 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_803_188, 202410_804_804_0] 2024.10.22 02:15:11.148179 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_165_169_1, total 218 rows starting from the beginning of the part 2024.10.22 02:15:11.186437 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_820_154/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:15:11.297471 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:15:11.300602 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:15:11.356901 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_648_132, 202410_649_649_0, 202410_650_650_0, 202410_651_651_0, 202410_652_652_0, 202410_653_653_0] 2024.10.22 02:15:11.360112 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_648_132, 202410_649_649_0, 202410_650_650_0, 202410_651_651_0, 202410_652_652_0, 202410_653_653_0] 2024.10.22 02:15:11.397189 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_648_132, 202410_649_649_0, 202410_650_650_0, 202410_651_651_0, 202410_652_652_0, 202410_653_653_0] 2024.10.22 02:15:11.605508 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 14.50 MiB, will set to 1.54 GiB (RSS), difference: 7.57 MiB 2024.10.22 02:15:11.898581 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 95496 2024.10.22 02:15:12.161549 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 6931 2024.10.22 02:15:12.201716 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_170_170_0, total 41 rows starting from the beginning of the part 2024.10.22 02:15:12.264570 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:12.267177 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:12.515113 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 121 entries to flush up to offset 93027 2024.10.22 02:15:12.544685 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1440_1440_0 (state Temporary) 2024.10.22 02:15:12.612393 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1440_1440_0 to 202410_1453_1453_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:12.781805 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:12.800710 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:12.855196 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1448_52 to 202410_1453_1453_0 2024.10.22 02:15:12.867822 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95496 2024.10.22 02:15:12.872942 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_840_840_0 (state Temporary) 2024.10.22 02:15:12.877377 [ 1168 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:12.880064 [ 1168 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:12.944948 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1448_52 to 202410_1453_1453_0 into Compact with storage Full 2024.10.22 02:15:12.976015 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:12.987522 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1448_52, total 16100 rows starting from the beginning of the part 2024.10.22 02:15:13.016062 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_1449_1449_0, total 53 rows starting from the beginning of the part 2024.10.22 02:15:13.041632 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_1450_1450_0, total 51 rows starting from the beginning of the part 2024.10.22 02:15:13.055463 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_840_840_0 to 202410_846_846_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:13.062375 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_1451_1451_0, total 35 rows starting from the beginning of the part 2024.10.22 02:15:13.075835 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_1452_1452_0, total 46 rows starting from the beginning of the part 2024.10.22 02:15:13.081661 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 93027 2024.10.22 02:15:13.101722 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_1453_1453_0, total 70 rows starting from the beginning of the part 2024.10.22 02:15:13.279451 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1235182 2024.10.22 02:15:13.289525 [ 1211 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 02:15:13.541167 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 12.81 MiB, will set to 1.53 GiB (RSS), difference: 38.46 MiB 2024.10.22 02:15:13.550833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/vgi/djselzodcomnpacpzggvbbyhygsgv. 2024.10.22 02:15:13.554116 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_754_143, 202410_755_760_1] 2024.10.22 02:15:13.555707 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_754_143, 202410_755_760_1] 2024.10.22 02:15:13.564095 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_754_143, 202410_755_760_1] 2024.10.22 02:15:13.879001 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_171_171_0, total 44 rows starting from the beginning of the part 2024.10.22 02:15:14.055668 [ 1269 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 333 ms. 2024.10.22 02:15:14.281123 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:14.289610 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:14.451338 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:15:14.491749 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} MergeTask::MergeProjectionsStage: Merge sorted 16355 rows, containing 13 columns (13 merged, 0 gathered) in 1.554036176 sec., 10524.208028475136 rows/sec., 2.03 MiB/sec. 2024.10.22 02:15:14.604114 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_818_818_0 (state Temporary) 2024.10.22 02:15:14.714963 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_818_818_0 to 202410_825_825_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:14.763811 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1235182 2024.10.22 02:15:14.898487 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} MergedBlockOutputStream: filled checksums 202410_1207_1453_53 (state Temporary) 2024.10.22 02:15:15.066188 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1453_53 to 202410_1207_1453_53 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:15.069487 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1453_53} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1448_52, 202410_1453_1453_0] -> 202410_1207_1453_53 2024.10.22 02:15:15.111108 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.96 MiB. 2024.10.22 02:15:15.328008 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 2.00 GiB, free memory in arenas 13.60 MiB, will set to 1.54 GiB (RSS), difference: 73.58 MiB 2024.10.22 02:15:15.484879 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xid/zntugubdeidjhueqzmntmopyvromd. 2024.10.22 02:15:15.540554 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_172_172_0, total 42 rows starting from the beginning of the part 2024.10.22 02:15:16.546918 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 13024 2024.10.22 02:15:16.658426 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:16.662237 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:16.710970 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_855_855_0 (state Temporary) 2024.10.22 02:15:16.770809 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_855_855_0 to 202410_868_868_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:16.908125 [ 1272 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_863_232 to 202410_864_864_0 2024.10.22 02:15:16.941200 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13024 2024.10.22 02:15:16.979967 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 95549 2024.10.22 02:15:17.007748 [ 1272 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:17.023776 [ 1272 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:17.030163 [ 1272 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 02:15:17.057507 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_864_233} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_863_232 to 202410_864_864_0 into Compact with storage Full 2024.10.22 02:15:17.088930 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_864_233} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:17.103413 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_864_233} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_863_232, total 13047 rows starting from the beginning of the part 2024.10.22 02:15:17.146515 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_864_233} MergeTreeSequentialSource: Reading 2 marks from part 202410_864_864_0, total 27 rows starting from the beginning of the part 2024.10.22 02:15:17.148809 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:17.151480 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:17.211560 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_173_173_0, total 38 rows starting from the beginning of the part 2024.10.22 02:15:17.256554 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 710 2024.10.22 02:15:17.302998 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1441_1441_0 (state Temporary) 2024.10.22 02:15:17.472061 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1441_1441_0 to 202410_1454_1454_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:17.477250 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 2.00 GiB, free memory in arenas 13.50 MiB, will set to 1.53 GiB (RSS), difference: 62.81 MiB 2024.10.22 02:15:17.482650 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:17.490852 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:17.656931 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95549 2024.10.22 02:15:17.663406 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_692_692_0 (state Temporary) 2024.10.22 02:15:17.752059 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_692_692_0 to 202410_702_702_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:17.951252 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 710 2024.10.22 02:15:18.470697 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_864_233} MergeTask::MergeProjectionsStage: Merge sorted 13074 rows, containing 12 columns (12 merged, 0 gathered) in 1.41811993 sec., 9219.248473575855 rows/sec., 1.44 MiB/sec. 2024.10.22 02:15:18.750540 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_864_233} MergedBlockOutputStream: filled checksums 202410_1_864_233 (state Temporary) 2024.10.22 02:15:18.804789 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_174_174_0, total 40 rows starting from the beginning of the part 2024.10.22 02:15:18.860368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for cf367c5318829979b5f420c6417480f4:0) 2024.10.22 02:15:18.862937 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 37205d77145d65d8ed5b6247cee4ff5c, offset: 0, size: 3221811 2024.10.22 02:15:18.869264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 37205d77145d65d8ed5b6247cee4ff5c for removal 2024.10.22 02:15:18.871868 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d8abf6a6b7076e2ceeb6b28383ed35a9, offset: 0, size: 895 2024.10.22 02:15:18.871936 [ 1476 ] {} CacheMetadata: Key 37205d77145d65d8ed5b6247cee4ff5c is removed from metadata 2024.10.22 02:15:18.874769 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d8abf6a6b7076e2ceeb6b28383ed35a9 for removal 2024.10.22 02:15:18.907079 [ 1476 ] {} CacheMetadata: Key d8abf6a6b7076e2ceeb6b28383ed35a9 is removed from metadata 2024.10.22 02:15:19.367867 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_804_189, 202410_805_805_0, 202410_806_806_0, 202410_807_807_0, 202410_808_808_0, 202410_809_809_0] 2024.10.22 02:15:19.372711 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_804_189, 202410_805_805_0, 202410_806_806_0, 202410_807_807_0, 202410_808_808_0, 202410_809_809_0] 2024.10.22 02:15:19.471150 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 13.15 MiB, will set to 1.53 GiB (RSS), difference: 15.97 MiB 2024.10.22 02:15:19.560505 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_864_233} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_864_233 to 202410_1_864_233 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:19.568026 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_864_233} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_863_232, 202410_864_864_0] -> 202410_1_864_233 2024.10.22 02:15:19.596198 [ 1387 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {5276570e-a364-404e-a403-3e186cba01a8::202410_1_864_233}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 02:15:19.618554 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_804_189, 202410_805_805_0, 202410_806_806_0, 202410_807_807_0, 202410_808_808_0, 202410_809_809_0] 2024.10.22 02:15:19.628780 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:15:20.628853 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 103 entries to flush up to offset 93130 2024.10.22 02:15:20.764061 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:20.767189 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:20.835170 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_841_841_0 (state Temporary) 2024.10.22 02:15:20.883821 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_841_841_0 to 202410_847_847_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:21.024238 [ 1261 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_842_19 to 202410_847_847_0 2024.10.22 02:15:21.051471 [ 1261 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:21.055182 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:21.068952 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_842_19 to 202410_847_847_0 into Compact with storage Full 2024.10.22 02:15:21.145118 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 93130 2024.10.22 02:15:21.145385 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:21.152481 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_842_19, total 9314 rows starting from the beginning of the part 2024.10.22 02:15:21.172860 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_843_843_0, total 94 rows starting from the beginning of the part 2024.10.22 02:15:21.200625 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_844_844_0, total 79 rows starting from the beginning of the part 2024.10.22 02:15:21.331682 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_845_845_0, total 64 rows starting from the beginning of the part 2024.10.22 02:15:21.400094 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:15:21.454041 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_846_846_0, total 121 rows starting from the beginning of the part 2024.10.22 02:15:21.507288 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} MergeTreeSequentialSource: Reading 2 marks from part 202410_847_847_0, total 103 rows starting from the beginning of the part 2024.10.22 02:15:21.503825 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:15:21.644078 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 8.98 MiB, will set to 1.53 GiB (RSS), difference: -2.84 MiB 2024.10.22 02:15:21.664014 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 95610 2024.10.22 02:15:21.766553 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1236722 2024.10.22 02:15:21.935600 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:21.945384 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:22.002590 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1442_1442_0 (state Temporary) 2024.10.22 02:15:22.021790 [ 1217 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:15:22.057581 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1442_1442_0 to 202410_1455_1455_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:22.065915 [ 1217 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:15:22.133788 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95610 2024.10.22 02:15:22.471145 [ 1222 ] {} DatabaseCatalog: Execution took 207 ms. 2024.10.22 02:15:22.981225 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:22.987711 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:23.139373 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_820_154/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:15:23.139039 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_819_819_0 (state Temporary) 2024.10.22 02:15:23.151501 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 77404 bytes for 449e0fecec8d1d01c559adec26a178e5:0) 2024.10.22 02:15:23.157433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 914f14352ee5c8db5bbe9351f51dacb0, offset: 0, size: 5110016 2024.10.22 02:15:23.163907 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 914f14352ee5c8db5bbe9351f51dacb0 for removal 2024.10.22 02:15:23.170378 [ 1476 ] {} CacheMetadata: Key 914f14352ee5c8db5bbe9351f51dacb0 is removed from metadata 2024.10.22 02:15:23.171725 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 83a34b3e60abbf55c05c17ce4469e378, offset: 0, size: 928 2024.10.22 02:15:23.177316 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 83a34b3e60abbf55c05c17ce4469e378 for removal 2024.10.22 02:15:23.223062 [ 1476 ] {} CacheMetadata: Key 83a34b3e60abbf55c05c17ce4469e378 is removed from metadata 2024.10.22 02:15:23.331175 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} MergeTask::MergeProjectionsStage: Merge sorted 9775 rows, containing 13 columns (13 merged, 0 gathered) in 2.267745724 sec., 4310.447990949448 rows/sec., 1.87 MiB/sec. 2024.10.22 02:15:23.345365 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_819_819_0 to 202410_826_826_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:23.394757 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1236722 2024.10.22 02:15:23.384154 [ 1159 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_821_821_0 to 202410_826_826_0 2024.10.22 02:15:23.439499 [ 1159 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:23.443185 [ 1159 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:23.455955 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} MergedBlockOutputStream: filled checksums 202410_747_847_20 (state Temporary) 2024.10.22 02:15:23.463533 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_821_821_0 to 202410_826_826_0 into Compact with storage Full 2024.10.22 02:15:23.487963 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:23.499965 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_821_821_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:15:23.508530 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_822_822_0, total 1542 rows starting from the beginning of the part 2024.10.22 02:15:23.518531 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_823_823_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:15:23.536366 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_824_824_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:15:23.553251 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_825_825_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:15:23.564778 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_826_826_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:15:23.733419 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_847_20 to 202410_747_847_20 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:23.736913 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_847_20} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_842_19, 202410_847_847_0] -> 202410_747_847_20 2024.10.22 02:15:23.834500 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 12.38 MiB, will set to 1.53 GiB (RSS), difference: -14.92 MiB 2024.10.22 02:15:23.844863 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 02:15:23.993677 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.75 MiB. 2024.10.22 02:15:24.482526 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 13040 2024.10.22 02:15:24.758286 [ 1744 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zgm/aaodxddanhiqhxzlvdwzqklgyrttn, size 1153322 2024.10.22 02:15:25.003334 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:25.008195 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:25.046223 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_856_856_0 (state Temporary) 2024.10.22 02:15:25.251905 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column Robotness 2024.10.22 02:15:25.262329 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column Robotness 2024.10.22 02:15:25.286973 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} MergeTask::MergeProjectionsStage: Merge sorted 10012 rows, containing 4 columns (4 merged, 0 gathered) in 1.826937354 sec., 5480.2098047199925 rows/sec., 85.63 KiB/sec. 2024.10.22 02:15:25.288456 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column Robotness 2024.10.22 02:15:25.306639 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column Robotness 2024.10.22 02:15:25.310609 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_856_856_0 to 202410_869_869_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:25.334072 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column Robotness 2024.10.22 02:15:25.356660 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column Robotness 2024.10.22 02:15:25.362974 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} MergedBlockOutputStream: filled checksums 202410_821_826_1 (state Temporary) 2024.10.22 02:15:25.363740 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column Robotness 2024.10.22 02:15:25.366257 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13040 2024.10.22 02:15:25.452989 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 711 2024.10.22 02:15:25.453609 [ 1194 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_864_233 to 202410_865_865_0 2024.10.22 02:15:25.454397 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1352_29, 202410_1353_1353_0, 202410_1354_1354_0, 202410_1355_1355_0, 202410_1356_1356_0, 202410_1357_1357_0] 2024.10.22 02:15:25.456210 [ 1194 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:25.457710 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1352_29, 202410_1353_1353_0, 202410_1354_1354_0, 202410_1355_1355_0, 202410_1356_1356_0, 202410_1357_1357_0] 2024.10.22 02:15:25.457747 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:25.473325 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_865_234} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_864_233 to 202410_865_865_0 into Compact with storage Full 2024.10.22 02:15:25.542811 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 13.63 MiB, will set to 1.53 GiB (RSS), difference: 7.23 MiB 2024.10.22 02:15:25.598615 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1352_29, 202410_1353_1353_0, 202410_1354_1354_0, 202410_1355_1355_0, 202410_1356_1356_0, 202410_1357_1357_0] 2024.10.22 02:15:25.616451 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bsf/nuwvqsheyyooxqtmwblydwumnafxe (LogSeriesLimiter: on interval from 2024-10-22 02:15:10 to 2024-10-22 02:15:25 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:15:25.700337 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_821_826_1 to 202410_821_826_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:25.707790 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_821_826_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_821_821_0, 202410_826_826_0] -> 202410_821_826_1 2024.10.22 02:15:25.718075 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_865_234} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:25.746155 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_865_234} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_864_233, total 13074 rows starting from the beginning of the part 2024.10.22 02:15:25.766260 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_174_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:15:25.767621 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:15:25.844753 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_865_234} MergeTreeSequentialSource: Reading 2 marks from part 202410_865_865_0, total 14 rows starting from the beginning of the part 2024.10.22 02:15:25.857087 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:25.861602 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:26.046441 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_693_693_0 (state Temporary) 2024.10.22 02:15:26.146740 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 95687 2024.10.22 02:15:26.553705 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_693_693_0 to 202410_703_703_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:26.582538 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:26.657707 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:26.700971 [ 1178 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_698_142 to 202410_703_703_0 2024.10.22 02:15:26.720980 [ 1178 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:26.723520 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:26.755556 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1443_1443_0 (state Temporary) 2024.10.22 02:15:26.842891 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 711 2024.10.22 02:15:26.856193 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_698_142 to 202410_703_703_0 into Compact with storage Full 2024.10.22 02:15:26.919612 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1443_1443_0 to 202410_1456_1456_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:26.977352 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95687 2024.10.22 02:15:27.059926 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:27.070375 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_698_142, total 742 rows starting from the beginning of the part 2024.10.22 02:15:27.161166 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_865_234} MergeTask::MergeProjectionsStage: Merge sorted 13088 rows, containing 12 columns (12 merged, 0 gathered) in 1.697637193 sec., 7709.538913241753 rows/sec., 1.20 MiB/sec. 2024.10.22 02:15:27.217283 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} MergeTreeSequentialSource: Reading 2 marks from part 202410_699_699_0, total 1 rows starting from the beginning of the part 2024.10.22 02:15:27.315789 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_865_234} MergedBlockOutputStream: filled checksums 202410_1_865_234 (state Temporary) 2024.10.22 02:15:27.366878 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} MergeTreeSequentialSource: Reading 2 marks from part 202410_700_700_0, total 1 rows starting from the beginning of the part 2024.10.22 02:15:27.451259 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 12.25 MiB, will set to 1.53 GiB (RSS), difference: -2.44 MiB 2024.10.22 02:15:27.490858 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} MergeTreeSequentialSource: Reading 2 marks from part 202410_701_701_0, total 1 rows starting from the beginning of the part 2024.10.22 02:15:27.705102 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} MergeTreeSequentialSource: Reading 2 marks from part 202410_702_702_0, total 1 rows starting from the beginning of the part 2024.10.22 02:15:27.751881 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} MergeTreeSequentialSource: Reading 2 marks from part 202410_703_703_0, total 1 rows starting from the beginning of the part 2024.10.22 02:15:27.801747 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_865_234} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_865_234 to 202410_1_865_234 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:27.807403 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_865_234} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_864_233, 202410_865_865_0] -> 202410_1_865_234 2024.10.22 02:15:27.846918 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:15:28.666882 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 128 entries to flush up to offset 93258 2024.10.22 02:15:28.762220 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:28.764280 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:28.791967 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:28.795826 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:29.057478 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_842_842_0 (state Temporary) 2024.10.22 02:15:29.143057 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_842_842_0 to 202410_848_848_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:29.212968 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} MergeTask::MergeProjectionsStage: Merge sorted 747 rows, containing 32 columns (32 merged, 0 gathered) in 2.361175515 sec., 316.36784104124513 rows/sec., 28.57 KiB/sec. 2024.10.22 02:15:29.282195 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 93258 2024.10.22 02:15:29.360026 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} MergedBlockOutputStream: filled checksums 202410_1_703_143 (state Temporary) 2024.10.22 02:15:29.460292 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 11.38 MiB, will set to 1.53 GiB (RSS), difference: -2.10 MiB 2024.10.22 02:15:29.956358 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_703_143 to 202410_1_703_143 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:29.960873 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_703_143} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_698_142, 202410_703_703_0] -> 202410_1_703_143 2024.10.22 02:15:30.011174 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:15:30.402884 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1927 entries to flush up to offset 1238649 2024.10.22 02:15:30.907951 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 805561 bytes for 3d0daa716b765cb8b31c7908243a6485:0) 2024.10.22 02:15:30.911697 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 197b8031d4cc8fd87597911fa521a5bb, offset: 0, size: 5035280 2024.10.22 02:15:30.917005 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 197b8031d4cc8fd87597911fa521a5bb for removal 2024.10.22 02:15:30.920536 [ 1476 ] {} CacheMetadata: Key 197b8031d4cc8fd87597911fa521a5bb is removed from metadata 2024.10.22 02:15:30.922380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 209dfd22a34df5b198bc2e9f4b9c0d84, offset: 0, size: 928 2024.10.22 02:15:30.926620 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 209dfd22a34df5b198bc2e9f4b9c0d84 for removal 2024.10.22 02:15:30.995339 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 95733 2024.10.22 02:15:31.045158 [ 1476 ] {} CacheMetadata: Key 209dfd22a34df5b198bc2e9f4b9c0d84 is removed from metadata 2024.10.22 02:15:31.297543 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:15:31.301123 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:15:31.362454 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:31.367073 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:31.463887 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:31.560465 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1444_1444_0 (state Temporary) 2024.10.22 02:15:31.567891 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:31.585412 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1444_1444_0 to 202410_1457_1457_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:31.660904 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 13.54 MiB, will set to 1.54 GiB (RSS), difference: -1.76 MiB 2024.10.22 02:15:31.769355 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_820_820_0 (state Temporary) 2024.10.22 02:15:31.863632 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_820_820_0 to 202410_827_827_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:31.869765 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95733 2024.10.22 02:15:31.892155 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1238649 2024.10.22 02:15:32.135764 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 02:15:32.430327 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_172_172_0 (state Temporary) 2024.10.22 02:15:32.869401 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 13063 2024.10.22 02:15:32.997592 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:33.002785 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:33.152826 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_857_857_0 (state Temporary) 2024.10.22 02:15:33.205534 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_857_857_0 to 202410_870_870_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:33.253289 [ 1270 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_865_234 to 202410_866_866_0 2024.10.22 02:15:33.259657 [ 1270 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:33.261637 [ 1270 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:33.269825 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_866_235} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_865_234 to 202410_866_866_0 into Compact with storage Full 2024.10.22 02:15:33.301372 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_866_235} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:33.310456 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_866_235} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_865_234, total 13088 rows starting from the beginning of the part 2024.10.22 02:15:33.323646 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_866_235} MergeTreeSequentialSource: Reading 2 marks from part 202410_866_866_0, total 12 rows starting from the beginning of the part 2024.10.22 02:15:33.363837 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13063 2024.10.22 02:15:33.453133 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 15.18 MiB, will set to 1.54 GiB (RSS), difference: -5.15 MiB 2024.10.22 02:15:34.028940 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_820_154/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:15:34.102677 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_866_235} MergeTask::MergeProjectionsStage: Merge sorted 13100 rows, containing 12 columns (12 merged, 0 gathered) in 0.835045825 sec., 15687.761806365537 rows/sec., 2.45 MiB/sec. 2024.10.22 02:15:34.163967 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_866_235} MergedBlockOutputStream: filled checksums 202410_1_866_235 (state Temporary) 2024.10.22 02:15:34.346840 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 712 2024.10.22 02:15:34.567758 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_866_235} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_866_235 to 202410_1_866_235 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:34.571367 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_866_235} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_865_234, 202410_866_866_0] -> 202410_1_866_235 2024.10.22 02:15:34.663969 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:15:34.795111 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:34.852351 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:34.918033 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_694_694_0 (state Temporary) 2024.10.22 02:15:35.126199 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_694_694_0 to 202410_704_704_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:35.201945 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 712 2024.10.22 02:15:35.426664 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 15.46 MiB, will set to 1.54 GiB (RSS), difference: 6.71 MiB 2024.10.22 02:15:35.900954 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 95777 2024.10.22 02:15:36.280969 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:36.287497 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:36.465546 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1445_1445_0 (state Temporary) 2024.10.22 02:15:36.528220 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1445_1445_0 to 202410_1458_1458_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:36.660806 [ 1256 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1207_1453_53 to 202410_1456_1456_0 2024.10.22 02:15:36.671323 [ 1256 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:36.673228 [ 1256 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:36.786367 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95777 2024.10.22 02:15:36.901697 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 95 entries to flush up to offset 93353 2024.10.22 02:15:36.923037 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1456_54} MergeTask::PrepareStage: Merging 4 parts: from 202410_1207_1453_53 to 202410_1456_1456_0 into Compact with storage Full 2024.10.22 02:15:36.977378 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1456_54} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:37.002107 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1456_54} MergeTreeSequentialSource: Reading 3 marks from part 202410_1207_1453_53, total 16355 rows starting from the beginning of the part 2024.10.22 02:15:37.070855 [ 1255 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:15:37.082685 [ 1255 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:15:37.190677 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1456_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_1454_1454_0, total 53 rows starting from the beginning of the part 2024.10.22 02:15:37.346987 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1456_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_1455_1455_0, total 61 rows starting from the beginning of the part 2024.10.22 02:15:37.349379 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:37.353081 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:37.381041 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_843_843_0 (state Temporary) 2024.10.22 02:15:37.398106 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1456_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_1456_1456_0, total 77 rows starting from the beginning of the part 2024.10.22 02:15:37.445318 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_843_843_0 to 202410_849_849_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:37.499435 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 93353 2024.10.22 02:15:37.661458 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 16.39 MiB, will set to 1.55 GiB (RSS), difference: 3.84 MiB 2024.10.22 02:15:37.689067 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 429869 bytes for c695c49f80bc8c6c5742594be3f15c8e:0) 2024.10.22 02:15:37.698155 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 399a8231b7e1a5597c3d2f1f1a924a2d, offset: 0, size: 3149419 2024.10.22 02:15:37.702709 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 399a8231b7e1a5597c3d2f1f1a924a2d for removal 2024.10.22 02:15:37.705776 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5f1d1fabd810367067a4a2590467c254, offset: 0, size: 897 2024.10.22 02:15:37.707644 [ 1476 ] {} CacheMetadata: Key 399a8231b7e1a5597c3d2f1f1a924a2d is removed from metadata 2024.10.22 02:15:37.712879 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5f1d1fabd810367067a4a2590467c254 for removal 2024.10.22 02:15:37.849584 [ 1476 ] {} CacheMetadata: Key 5f1d1fabd810367067a4a2590467c254 is removed from metadata 2024.10.22 02:15:38.901497 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1240189 2024.10.22 02:15:39.060549 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_174_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:15:39.072298 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1456_54} MergeTask::MergeProjectionsStage: Merge sorted 16546 rows, containing 13 columns (13 merged, 0 gathered) in 2.262793929 sec., 7312.199218826878 rows/sec., 1.41 MiB/sec. 2024.10.22 02:15:39.149073 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1456_54/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:15:39.358533 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1456_54} MergedBlockOutputStream: filled checksums 202410_1207_1456_54 (state Temporary) 2024.10.22 02:15:39.360690 [ 1729 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bsf/nuwvqsheyyooxqtmwblydwumnafxe, size 563367 2024.10.22 02:15:39.506480 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 14.34 MiB, will set to 1.55 GiB (RSS), difference: -3.05 MiB 2024.10.22 02:15:39.533588 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1456_54} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1456_54 to 202410_1207_1456_54 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:39.544673 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1456_54} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1207_1453_53, 202410_1456_1456_0] -> 202410_1207_1456_54 2024.10.22 02:15:39.702201 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 9.91 MiB. 2024.10.22 02:15:39.736864 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 02:15:39.759304 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 02:15:39.769769 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 02:15:39.808121 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 02:15:39.846069 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 02:15:39.865736 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 02:15:39.895968 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column GeneralInterests 2024.10.22 02:15:40.105813 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qid/mhlbofjkmxykoqlkajjpobvomcjtt (LogSeriesLimiter: on interval from 2024-10-22 02:15:25 to 2024-10-22 02:15:40 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:15:40.254423 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:40.257641 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:40.444095 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_821_821_0 (state Temporary) 2024.10.22 02:15:40.519459 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_821_821_0 to 202410_828_828_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:40.564166 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1240189 2024.10.22 02:15:40.800117 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 95831 2024.10.22 02:15:40.865811 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 13076 2024.10.22 02:15:41.005998 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 858820 bytes for b3d1020c36eed3e05cb8f1ad5e250a02:0) 2024.10.22 02:15:41.011713 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 810f22ef34688d3ae2626571a4e17381, offset: 0, size: 2320384 2024.10.22 02:15:41.017673 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 810f22ef34688d3ae2626571a4e17381 for removal 2024.10.22 02:15:41.025031 [ 1476 ] {} CacheMetadata: Key 810f22ef34688d3ae2626571a4e17381 is removed from metadata 2024.10.22 02:15:41.025110 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:41.028151 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7b8edc429d27dea2c9cfffa64568fdb5, offset: 0, size: 878 2024.10.22 02:15:41.031714 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.68 GiB. 2024.10.22 02:15:41.034760 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7b8edc429d27dea2c9cfffa64568fdb5 for removal 2024.10.22 02:15:41.059138 [ 1476 ] {} CacheMetadata: Key 7b8edc429d27dea2c9cfffa64568fdb5 is removed from metadata 2024.10.22 02:15:41.079711 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_858_858_0 (state Temporary) 2024.10.22 02:15:41.124138 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:41.128504 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:41.128713 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_858_858_0 to 202410_871_871_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:41.156136 [ 1262 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_866_235 to 202410_867_867_0 2024.10.22 02:15:41.167364 [ 1262 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:41.169117 [ 1262 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:41.281126 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13076 2024.10.22 02:15:41.283887 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_867_236} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_866_235 to 202410_867_867_0 into Compact with storage Full 2024.10.22 02:15:41.299421 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:15:41.313800 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:15:41.317715 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1446_1446_0 (state Temporary) 2024.10.22 02:15:41.374712 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_867_236} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:41.395210 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_867_236} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_866_235, total 13100 rows starting from the beginning of the part 2024.10.22 02:15:41.448332 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_867_236} MergeTreeSequentialSource: Reading 2 marks from part 202410_867_867_0, total 12 rows starting from the beginning of the part 2024.10.22 02:15:41.467729 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_820_154} MergeTask::MergeProjectionsStage: Merge sorted 1240300 rows, containing 4 columns (4 merged, 0 gathered) in 35.129521159 sec., 35306.487509074446 rows/sec., 529.52 KiB/sec. 2024.10.22 02:15:41.470671 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1446_1446_0 to 202410_1459_1459_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:41.565543 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_172_172_0 to 202410_175_175_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:41.638614 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.27 MiB, will set to 1.55 GiB (RSS), difference: -23.59 MiB 2024.10.22 02:15:41.659702 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_820_154} MergedBlockOutputStream: filled checksums 202410_1_820_154 (state Temporary) 2024.10.22 02:15:41.709817 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95831 2024.10.22 02:15:42.223190 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_867_236} MergeTask::MergeProjectionsStage: Merge sorted 13112 rows, containing 12 columns (12 merged, 0 gathered) in 0.945221496 sec., 13871.880882404308 rows/sec., 2.16 MiB/sec. 2024.10.22 02:15:42.290167 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_867_236} MergedBlockOutputStream: filled checksums 202410_1_867_236 (state Temporary) 2024.10.22 02:15:42.393474 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_867_236} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_867_236 to 202410_1_867_236 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:42.399500 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_867_236} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_866_235, 202410_867_867_0] -> 202410_1_867_236 2024.10.22 02:15:42.434667 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:15:42.666679 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/sdh/fsjhgcedsqptjjbciwzdszcvmuubv. 2024.10.22 02:15:42.706868 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 713 2024.10.22 02:15:42.717531 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/dpf/himhnlnezetudzvmevsupwrwuwfvq. 2024.10.22 02:15:43.102849 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:43.105695 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:43.236672 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_767_767_0, 202410_768_768_0, 202410_769_769_0, 202410_770_770_0, 202410_771_771_0, 202410_772_772_0] 2024.10.22 02:15:43.258600 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_767_767_0, 202410_768_768_0, 202410_769_769_0, 202410_770_770_0, 202410_771_771_0, 202410_772_772_0] 2024.10.22 02:15:43.275705 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_695_695_0 (state Temporary) 2024.10.22 02:15:43.284887 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_767_767_0, 202410_768_768_0, 202410_769_769_0, 202410_770_770_0, 202410_771_771_0, 202410_772_772_0] 2024.10.22 02:15:43.347654 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_695_695_0 to 202410_705_705_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:43.351488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/dpf/himhnlnezetudzvmevsupwrwuwfvq. 2024.10.22 02:15:43.354952 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_15_15_0 to 201403_15_15_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:43.366865 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/sdh/fsjhgcedsqptjjbciwzdszcvmuubv. 2024.10.22 02:15:43.393537 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:15:43.428367 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 713 2024.10.22 02:15:43.664710 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 2.00 GiB, free memory in arenas 6.88 MiB, will set to 1.55 GiB (RSS), difference: 150.16 MiB 2024.10.22 02:15:44.412923 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6931 2024.10.22 02:15:45.013415 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 123 entries to flush up to offset 93476 2024.10.22 02:15:45.151143 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_820_154} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_820_154 to 202410_1_820_154 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:45.258752 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_820_154} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_814_153, 202410_815_820_1] -> 202410_1_820_154 2024.10.22 02:15:45.276246 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:15:45.451341 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:45.455448 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:45.490370 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_844_844_0 (state Temporary) 2024.10.22 02:15:45.560771 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_844_844_0 to 202410_850_850_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:45.607614 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 93476 2024.10.22 02:15:45.718875 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 95894 2024.10.22 02:15:45.751759 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 861.99 MiB, peak 2.00 GiB, free memory in arenas 8.77 MiB, will set to 1.07 GiB (RSS), difference: 231.74 MiB 2024.10.22 02:15:46.069223 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:46.080331 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:46.349616 [ 1173 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 258 ms. 2024.10.22 02:15:46.421468 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 1048576 bytes for 5b0ad389f11efc3192f1f0256a871cb4:0) 2024.10.22 02:15:46.426473 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 163535c645584a69033b089d7370229d, offset: 0, size: 4333113 2024.10.22 02:15:46.438920 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 163535c645584a69033b089d7370229d for removal 2024.10.22 02:15:46.444259 [ 1476 ] {} CacheMetadata: Key 163535c645584a69033b089d7370229d is removed from metadata 2024.10.22 02:15:46.446152 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1472bd7ec8c3f67498c9f16fadfe8b61, offset: 0, size: 912 2024.10.22 02:15:46.457157 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1447_1447_0 (state Temporary) 2024.10.22 02:15:46.552058 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 1472bd7ec8c3f67498c9f16fadfe8b61 for removal 2024.10.22 02:15:46.561014 [ 1476 ] {} CacheMetadata: Key 1472bd7ec8c3f67498c9f16fadfe8b61 is removed from metadata 2024.10.22 02:15:46.564874 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7a11bf69dff36648a9097357f627687c, offset: 0, size: 4 2024.10.22 02:15:46.570119 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 7a11bf69dff36648a9097357f627687c for removal 2024.10.22 02:15:46.572918 [ 1476 ] {} CacheMetadata: Key 7a11bf69dff36648a9097357f627687c is removed from metadata 2024.10.22 02:15:46.622584 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1447_1447_0 to 202410_1460_1460_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:46.963377 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95894 2024.10.22 02:15:47.433480 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.16 GiB, peak 2.00 GiB, free memory in arenas 8.71 MiB, will set to 1.10 GiB (RSS), difference: -62.62 MiB 2024.10.22 02:15:47.568915 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1241729 2024.10.22 02:15:47.948323 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 58 2024.10.22 02:15:48.263777 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:48.270695 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:48.354404 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_43_43_0 (state Temporary) 2024.10.22 02:15:48.371535 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:48.373788 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:48.443346 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_43_43_0 to 202410_43_43_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:48.603317 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_822_822_0 (state Temporary) 2024.10.22 02:15:48.664382 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 58 2024.10.22 02:15:48.695870 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_822_822_0 to 202410_829_829_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:48.735576 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1241729 2024.10.22 02:15:48.785798 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 13091 2024.10.22 02:15:48.997686 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:49.005350 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:49.043455 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_859_859_0 (state Temporary) 2024.10.22 02:15:49.074407 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_859_859_0 to 202410_872_872_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:49.117198 [ 1183 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_867_236 to 202410_868_868_0 2024.10.22 02:15:49.129525 [ 1183 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:49.146522 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13091 2024.10.22 02:15:49.149754 [ 1183 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:49.165403 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_868_237} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_867_236 to 202410_868_868_0 into Compact with storage Full 2024.10.22 02:15:49.354326 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_868_237} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:49.372530 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_868_237} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_867_236, total 13112 rows starting from the beginning of the part 2024.10.22 02:15:49.412918 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_868_237} MergeTreeSequentialSource: Reading 2 marks from part 202410_868_868_0, total 18 rows starting from the beginning of the part 2024.10.22 02:15:49.457799 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.17 GiB, peak 2.00 GiB, free memory in arenas 9.73 MiB, will set to 1.13 GiB (RSS), difference: -35.07 MiB 2024.10.22 02:15:50.362789 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1357_30, 202410_1358_1358_0, 202410_1359_1359_0, 202410_1360_1360_0, 202410_1361_1361_0, 202410_1362_1362_0] 2024.10.22 02:15:50.364817 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1357_30, 202410_1358_1358_0, 202410_1359_1359_0, 202410_1360_1360_0, 202410_1361_1361_0, 202410_1362_1362_0] 2024.10.22 02:15:50.471774 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_868_237} MergeTask::MergeProjectionsStage: Merge sorted 13130 rows, containing 12 columns (12 merged, 0 gathered) in 1.312314804 sec., 10005.221277683613 rows/sec., 1.56 MiB/sec. 2024.10.22 02:15:50.515511 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1357_30, 202410_1358_1358_0, 202410_1359_1359_0, 202410_1360_1360_0, 202410_1361_1361_0, 202410_1362_1362_0] 2024.10.22 02:15:50.566378 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.13 GiB, peak 2.00 GiB, free memory in arenas 10.00 MiB, will set to 1.14 GiB (RSS), difference: 6.87 MiB 2024.10.22 02:15:50.837407 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_868_237} MergedBlockOutputStream: filled checksums 202410_1_868_237 (state Temporary) 2024.10.22 02:15:50.984847 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 95945 2024.10.22 02:15:51.298985 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:15:51.301016 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:15:51.421015 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:51.425522 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:51.480119 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1448_1448_0 (state Temporary) 2024.10.22 02:15:51.519062 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1448_1448_0 to 202410_1461_1461_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:51.555478 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 5 parts from 202410_1207_1456_54 to 202410_1460_1460_0 2024.10.22 02:15:51.570179 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_868_237} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_868_237 to 202410_1_868_237 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:51.571479 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95945 2024.10.22 02:15:51.575917 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_868_237} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_867_236, 202410_868_868_0] -> 202410_1_868_237 2024.10.22 02:15:51.576089 [ 1247 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:51.580647 [ 1247 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:51.620615 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1460_55} MergeTask::PrepareStage: Merging 5 parts: from 202410_1207_1456_54 to 202410_1460_1460_0 into Compact with storage Full 2024.10.22 02:15:51.645348 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_174_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:15:51.696499 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1460_55} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:51.710288 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1460_55} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1456_54, total 16546 rows starting from the beginning of the part 2024.10.22 02:15:51.717646 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:15:51.939225 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 6970 2024.10.22 02:15:51.982512 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1460_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_1457_1457_0, total 46 rows starting from the beginning of the part 2024.10.22 02:15:52.030436 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1460_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_1458_1458_0, total 44 rows starting from the beginning of the part 2024.10.22 02:15:52.090630 [ 1197 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:15:52.117441 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1460_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_1459_1459_0, total 54 rows starting from the beginning of the part 2024.10.22 02:15:52.224613 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1460_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_1460_1460_0, total 63 rows starting from the beginning of the part 2024.10.22 02:15:52.341555 [ 1197 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:15:52.346843 [ 1197 ] {} DNSCacheUpdater: Execution took 257 ms. 2024.10.22 02:15:52.543783 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.17 GiB, peak 2.00 GiB, free memory in arenas 9.50 MiB, will set to 1.16 GiB (RSS), difference: -1.17 MiB 2024.10.22 02:15:53.144224 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 78 entries to flush up to offset 93554 2024.10.22 02:15:53.444438 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:53.446958 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:53.475091 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_845_845_0 (state Temporary) 2024.10.22 02:15:53.524799 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_845_845_0 to 202410_851_851_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:53.672586 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 93554 2024.10.22 02:15:53.695568 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_787_8, 202410_788_788_0, 202410_789_789_0, 202410_790_790_0, 202410_791_791_0, 202410_792_792_0] 2024.10.22 02:15:53.698350 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_787_8, 202410_788_788_0, 202410_789_789_0, 202410_790_790_0, 202410_791_791_0, 202410_792_792_0] 2024.10.22 02:15:53.730712 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_787_8, 202410_788_788_0, 202410_789_789_0, 202410_790_790_0, 202410_791_791_0, 202410_792_792_0] 2024.10.22 02:15:54.347002 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.24 GiB, peak 2.00 GiB, free memory in arenas 8.07 MiB, will set to 1.21 GiB (RSS), difference: -29.00 MiB 2024.10.22 02:15:55.074859 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1460_55} MergeTask::MergeProjectionsStage: Merge sorted 16753 rows, containing 13 columns (13 merged, 0 gathered) in 3.472279247 sec., 4824.784761903685 rows/sec., 950.89 KiB/sec. 2024.10.22 02:15:55.239646 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1460_55} MergedBlockOutputStream: filled checksums 202410_1207_1460_55 (state Temporary) 2024.10.22 02:15:55.346473 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1460_55} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1460_55 to 202410_1207_1460_55 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:55.352303 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1460_55} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 5 parts: [202410_1207_1456_54, 202410_1460_1460_0] -> 202410_1207_1460_55 2024.10.22 02:15:55.398156 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.54 MiB. 2024.10.22 02:15:55.580880 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 95987 2024.10.22 02:15:55.742403 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1243654 2024.10.22 02:15:55.845513 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:55.949168 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:56.129664 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1449_1449_0 (state Temporary) 2024.10.22 02:15:56.168858 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1449_1449_0 to 202410_1462_1462_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:56.237693 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 95987 2024.10.22 02:15:56.471866 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:56.473704 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:56.522924 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:15:56.648867 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 13104 2024.10.22 02:15:56.709352 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_823_823_0 (state Temporary) 2024.10.22 02:15:56.743917 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 9.29 MiB, will set to 1.26 GiB (RSS), difference: -47.89 MiB 2024.10.22 02:15:56.855432 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:56.857161 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:56.995872 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_860_860_0 (state Temporary) 2024.10.22 02:15:56.999805 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_823_823_0 to 202410_830_830_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:57.048006 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1243654 2024.10.22 02:15:57.050136 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_860_860_0 to 202410_873_873_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:57.141908 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_820_154 to 202410_821_826_1 2024.10.22 02:15:57.149412 [ 1243 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:57.154377 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:57.166399 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_826_155} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_820_154 to 202410_821_826_1 into Wide with storage Full 2024.10.22 02:15:57.171218 [ 1175 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_868_237 to 202410_869_869_0 2024.10.22 02:15:57.176821 [ 1175 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:57.180943 [ 1175 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:57.215519 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13104 2024.10.22 02:15:57.231078 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_869_238} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_868_237 to 202410_869_869_0 into Compact with storage Full 2024.10.22 02:15:57.404772 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_826_155} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:57.414406 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_826_155} MergeTreeSequentialSource: Reading 153 marks from part 202410_1_820_154, total 1240300 rows starting from the beginning of the part 2024.10.22 02:15:57.720763 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_869_238} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:15:57.750272 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_869_238} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_868_237, total 13130 rows starting from the beginning of the part 2024.10.22 02:15:57.776400 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_869_238} MergeTreeSequentialSource: Reading 2 marks from part 202410_869_869_0, total 16 rows starting from the beginning of the part 2024.10.22 02:15:58.166438 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_826_155} MergeTreeSequentialSource: Reading 3 marks from part 202410_821_826_1, total 10012 rows starting from the beginning of the part 2024.10.22 02:15:58.318670 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_809_190, 202410_810_810_0] 2024.10.22 02:15:58.322241 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_809_190, 202410_810_810_0] 2024.10.22 02:15:58.439293 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 714 2024.10.22 02:15:58.448094 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_809_190, 202410_810_810_0] 2024.10.22 02:15:58.543676 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_869_238} MergeTask::MergeProjectionsStage: Merge sorted 13146 rows, containing 12 columns (12 merged, 0 gathered) in 1.328152251 sec., 9897.961615546741 rows/sec., 1.54 MiB/sec. 2024.10.22 02:15:58.739301 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_869_238} MergedBlockOutputStream: filled checksums 202410_1_869_238 (state Temporary) 2024.10.22 02:15:58.826996 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:15:58.831273 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:15:58.897473 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 7.62 MiB, will set to 1.31 GiB (RSS), difference: -71.91 MiB 2024.10.22 02:15:58.999408 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_696_696_0 (state Temporary) 2024.10.22 02:15:59.048306 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_869_238} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_869_238 to 202410_1_869_238 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:59.052787 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_869_238} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_868_237, 202410_869_869_0] -> 202410_1_869_238 2024.10.22 02:15:59.071001 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:15:59.211531 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_696_696_0 to 202410_706_706_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:15:59.308254 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 714 2024.10.22 02:15:59.329176 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_653_133, 202410_654_654_0, 202410_655_655_0, 202410_656_656_0, 202410_657_657_0, 202410_658_658_0] 2024.10.22 02:15:59.331492 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_653_133, 202410_654_654_0, 202410_655_655_0, 202410_656_656_0, 202410_657_657_0, 202410_658_658_0] 2024.10.22 02:15:59.458488 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_653_133, 202410_654_654_0, 202410_655_655_0, 202410_656_656_0, 202410_657_657_0, 202410_658_658_0] 2024.10.22 02:16:00.129797 [ 1276 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 02:16:00.346694 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 96039 2024.10.22 02:16:00.748470 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:00.748890 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 6.30 MiB, will set to 1.34 GiB (RSS), difference: -22.41 MiB 2024.10.22 02:16:00.753365 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:00.809764 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1450_1450_0 (state Temporary) 2024.10.22 02:16:00.859767 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1450_1450_0 to 202410_1463_1463_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:00.951819 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96039 2024.10.22 02:16:01.282843 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 95 entries to flush up to offset 93649 2024.10.22 02:16:01.300572 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:16:01.302721 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:16:01.465103 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:01.468187 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:01.514160 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_846_846_0 (state Temporary) 2024.10.22 02:16:01.560821 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_846_846_0 to 202410_852_852_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:01.592584 [ 1205 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_847_20 to 202410_852_852_0 2024.10.22 02:16:01.615676 [ 1205 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:01.617732 [ 1205 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:01.658617 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_847_20 to 202410_852_852_0 into Compact with storage Full 2024.10.22 02:16:01.704614 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 93649 2024.10.22 02:16:01.713105 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:01.730724 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_847_20, total 9775 rows starting from the beginning of the part 2024.10.22 02:16:01.768829 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 2097152 bytes for 5b0ad389f11efc3192f1f0256a871cb4:0) 2024.10.22 02:16:01.771172 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 99b6d71215629df55493a12d0179937f, offset: 0, size: 3764584 2024.10.22 02:16:01.776673 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 99b6d71215629df55493a12d0179937f for removal 2024.10.22 02:16:01.779558 [ 1476 ] {} CacheMetadata: Key 99b6d71215629df55493a12d0179937f is removed from metadata 2024.10.22 02:16:01.779555 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6f3d38e9ac594443f09fdc4e3df817e1, offset: 0, size: 906 2024.10.22 02:16:01.786497 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 6f3d38e9ac594443f09fdc4e3df817e1 for removal 2024.10.22 02:16:01.788745 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_848_848_0, total 128 rows starting from the beginning of the part 2024.10.22 02:16:01.804091 [ 1476 ] {} CacheMetadata: Key 6f3d38e9ac594443f09fdc4e3df817e1 is removed from metadata 2024.10.22 02:16:01.867662 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_849_849_0, total 95 rows starting from the beginning of the part 2024.10.22 02:16:02.136653 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 277.20 MiB using storage policy from min volume index 0 2024.10.22 02:16:02.149810 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_850_850_0, total 123 rows starting from the beginning of the part 2024.10.22 02:16:02.318072 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_851_851_0, total 78 rows starting from the beginning of the part 2024.10.22 02:16:02.623716 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} MergeTreeSequentialSource: Reading 2 marks from part 202410_852_852_0, total 95 rows starting from the beginning of the part 2024.10.22 02:16:02.881474 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_826_155/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:16:03.057817 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_760_144, 202410_761_766_1] 2024.10.22 02:16:03.061029 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_760_144, 202410_761_766_1] 2024.10.22 02:16:03.076980 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_760_144, 202410_761_766_1] 2024.10.22 02:16:03.146211 [ 1232 ] {} DatabaseCatalog: Execution took 283 ms. 2024.10.22 02:16:03.262756 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uwt/tnnaflmjnvgnadflybmqrcwzknysr (LogSeriesLimiter: on interval from 2024-10-22 02:15:40 to 2024-10-22 02:16:03 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:16:03.457585 [ 1203 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 02:16:04.053576 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1245194 2024.10.22 02:16:04.182514 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_852_21/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:16:04.394738 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 2.00 GiB, free memory in arenas 13.42 MiB, will set to 1.36 GiB (RSS), difference: -77.30 MiB 2024.10.22 02:16:04.721276 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 13112 2024.10.22 02:16:04.843123 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RemoteIP 2024.10.22 02:16:04.861637 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RemoteIP 2024.10.22 02:16:04.862386 [ 1687 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qid/mhlbofjkmxykoqlkajjpobvomcjtt, size 1552087 2024.10.22 02:16:04.871198 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RemoteIP 2024.10.22 02:16:04.885405 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:04.901278 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:04.959989 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 96091 2024.10.22 02:16:04.999884 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RemoteIP 2024.10.22 02:16:05.031179 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RemoteIP 2024.10.22 02:16:05.037125 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} MergeTask::MergeProjectionsStage: Merge sorted 10294 rows, containing 13 columns (13 merged, 0 gathered) in 3.383618338 sec., 3042.3052991504387 rows/sec., 1.32 MiB/sec. 2024.10.22 02:16:05.067357 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:05.070295 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:05.074852 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RemoteIP 2024.10.22 02:16:05.093728 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RemoteIP 2024.10.22 02:16:05.118360 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:05.121047 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:05.138579 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_861_861_0 (state Temporary) 2024.10.22 02:16:05.196947 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} MergedBlockOutputStream: filled checksums 202410_747_852_21 (state Temporary) 2024.10.22 02:16:05.199110 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1451_1451_0 (state Temporary) 2024.10.22 02:16:05.207912 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_861_861_0 to 202410_874_874_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:05.244297 [ 1161 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_869_238 to 202410_870_870_0 2024.10.22 02:16:05.247867 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_824_824_0 (state Temporary) 2024.10.22 02:16:05.249483 [ 1161 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:05.252512 [ 1161 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:05.265112 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13112 2024.10.22 02:16:05.267902 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_870_239} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_869_238 to 202410_870_870_0 into Compact with storage Full 2024.10.22 02:16:05.349287 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_870_239} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:05.353133 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1451_1451_0 to 202410_1464_1464_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:05.363949 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_870_239} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_869_238, total 13146 rows starting from the beginning of the part 2024.10.22 02:16:05.370700 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_852_21 to 202410_747_852_21 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:05.384921 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_852_21} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_847_20, 202410_852_852_0] -> 202410_747_852_21 2024.10.22 02:16:05.430074 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.85 MiB. 2024.10.22 02:16:05.442643 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96091 2024.10.22 02:16:05.486782 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_824_824_0 to 202410_831_831_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:05.465712 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_870_239} MergeTreeSequentialSource: Reading 2 marks from part 202410_870_870_0, total 23 rows starting from the beginning of the part 2024.10.22 02:16:05.644284 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1245194 2024.10.22 02:16:06.570715 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_870_239} MergeTask::MergeProjectionsStage: Merge sorted 13169 rows, containing 12 columns (12 merged, 0 gathered) in 1.302156826 sec., 10113.221185848162 rows/sec., 1.58 MiB/sec. 2024.10.22 02:16:06.577120 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 11.19 MiB, will set to 1.35 GiB (RSS), difference: -177.11 MiB 2024.10.22 02:16:06.625748 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_870_239} MergedBlockOutputStream: filled checksums 202410_1_870_239 (state Temporary) 2024.10.22 02:16:06.653543 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_810_191, 202410_811_811_0, 202410_812_812_0, 202410_813_813_0, 202410_814_814_0, 202410_815_815_0] 2024.10.22 02:16:06.656507 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_810_191, 202410_811_811_0, 202410_812_812_0, 202410_813_813_0, 202410_814_814_0, 202410_815_815_0] 2024.10.22 02:16:06.677149 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_810_191, 202410_811_811_0, 202410_812_812_0, 202410_813_813_0, 202410_814_814_0, 202410_815_815_0] 2024.10.22 02:16:06.811740 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 715 2024.10.22 02:16:06.877595 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_870_239} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_870_239 to 202410_1_870_239 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:06.881679 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_870_239} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_869_238, 202410_870_870_0] -> 202410_1_870_239 2024.10.22 02:16:06.927556 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:16:07.107602 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:07.116428 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:07.167476 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_697_697_0 (state Temporary) 2024.10.22 02:16:07.334385 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_174_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:16:07.348182 [ 1157 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:16:07.369298 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_697_697_0 to 202410_707_707_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:07.370829 [ 1157 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:16:07.468411 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 715 2024.10.22 02:16:08.014592 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/veh/snvzcqwirfujishkdorybjlqmfuns (LogSeriesLimiter: on interval from 2024-10-22 02:16:03 to 2024-10-22 02:16:08 accepted series 1 / 266 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:16:08.208420 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:08.213321 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:08.649328 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 12.33 MiB, will set to 1.35 GiB (RSS), difference: -210.27 MiB 2024.10.22 02:16:09.338489 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 86 entries to flush up to offset 93735 2024.10.22 02:16:09.455551 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 96144 2024.10.22 02:16:09.903342 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:09.908483 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:10.148396 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:10.151487 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:10.181725 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_847_847_0 (state Temporary) 2024.10.22 02:16:10.302551 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1452_1452_0 (state Temporary) 2024.10.22 02:16:10.451545 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1452_1452_0 to 202410_1465_1465_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:10.469137 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_847_847_0 to 202410_853_853_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:10.519897 [ 1193 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1460_55 to 202410_1461_1461_0 2024.10.22 02:16:10.532402 [ 1193 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:10.567999 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:10.623833 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1461_56} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1460_55 to 202410_1461_1461_0 into Compact with storage Full 2024.10.22 02:16:10.630535 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 2.00 GiB, free memory in arenas 13.93 MiB, will set to 1.35 GiB (RSS), difference: -83.65 MiB 2024.10.22 02:16:10.653518 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96144 2024.10.22 02:16:10.808073 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1461_56} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:10.822941 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1461_56} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1460_55, total 16753 rows starting from the beginning of the part 2024.10.22 02:16:10.850273 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 93735 2024.10.22 02:16:11.066141 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1461_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_1461_1461_0, total 51 rows starting from the beginning of the part 2024.10.22 02:16:11.302338 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:16:11.308924 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:16:11.563627 [ 1184 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 02:16:11.759737 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_173_173_0 (state Temporary) 2024.10.22 02:16:12.653508 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1246734 2024.10.22 02:16:12.766913 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 13128 2024.10.22 02:16:12.914737 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1461_56} MergeTask::MergeProjectionsStage: Merge sorted 16804 rows, containing 13 columns (13 merged, 0 gathered) in 2.224838679 sec., 7552.907165185076 rows/sec., 1.45 MiB/sec. 2024.10.22 02:16:13.024420 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:13.029321 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.67 GiB. 2024.10.22 02:16:13.085164 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1461_56} MergedBlockOutputStream: filled checksums 202410_1207_1461_56 (state Temporary) 2024.10.22 02:16:13.171770 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1461_56} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1461_56 to 202410_1207_1461_56 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:13.175519 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1461_56} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1460_55, 202410_1461_1461_0] -> 202410_1207_1461_56 2024.10.22 02:16:13.177107 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_862_862_0 (state Temporary) 2024.10.22 02:16:13.218622 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.43 MiB. 2024.10.22 02:16:13.238680 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_862_862_0 to 202410_875_875_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:13.346829 [ 1419 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:16:13.353893 [ 1214 ] {} DatabaseCatalog: Execution took 208 ms. 2024.10.22 02:16:13.377110 [ 1211 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_870_239 to 202410_871_871_0 2024.10.22 02:16:13.383419 [ 1211 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:13.401255 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13128 2024.10.22 02:16:13.405139 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:13.535904 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_871_240} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_870_239 to 202410_871_871_0 into Compact with storage Full 2024.10.22 02:16:13.692192 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_871_240} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:13.705630 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:13.712194 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:13.747943 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 2.00 GiB, free memory in arenas 14.16 MiB, will set to 1.37 GiB (RSS), difference: -4.37 MiB 2024.10.22 02:16:13.860018 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_825_825_0 (state Temporary) 2024.10.22 02:16:13.906841 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_871_240} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_870_239, total 13169 rows starting from the beginning of the part 2024.10.22 02:16:13.931304 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_871_240} MergeTreeSequentialSource: Reading 2 marks from part 202410_871_871_0, total 13 rows starting from the beginning of the part 2024.10.22 02:16:13.956013 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_825_825_0 to 202410_832_832_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:13.975203 [ 1207 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_827_827_0 to 202410_832_832_0 2024.10.22 02:16:13.983270 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1246734 2024.10.22 02:16:13.988895 [ 1207 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:13.992583 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:14.020440 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_827_827_0 to 202410_832_832_0 into Compact with storage Full 2024.10.22 02:16:14.058281 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:14.063487 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_827_827_0, total 1927 rows starting from the beginning of the part 2024.10.22 02:16:14.078084 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_828_828_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:16:14.097302 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_829_829_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:16:14.123100 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_830_830_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:16:14.139034 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_831_831_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:16:14.148937 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_826_155/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:16:14.164278 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_832_832_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:16:14.659686 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 96206 2024.10.22 02:16:14.953915 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:14.955300 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_871_240} MergeTask::MergeProjectionsStage: Merge sorted 13182 rows, containing 12 columns (12 merged, 0 gathered) in 1.537347193 sec., 8574.510728624982 rows/sec., 1.34 MiB/sec. 2024.10.22 02:16:14.971111 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 716 2024.10.22 02:16:15.015908 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:15.135348 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_871_240} MergedBlockOutputStream: filled checksums 202410_1_871_240 (state Temporary) 2024.10.22 02:16:15.179458 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1453_1453_0 (state Temporary) 2024.10.22 02:16:15.248904 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1453_1453_0 to 202410_1466_1466_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:15.277168 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} MergeTask::MergeProjectionsStage: Merge sorted 10012 rows, containing 4 columns (4 merged, 0 gathered) in 1.266462987 sec., 7905.481725696892 rows/sec., 123.52 KiB/sec. 2024.10.22 02:16:15.289427 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1461_56 to 202410_1462_1462_0 2024.10.22 02:16:15.307316 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:15.313470 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:15.343167 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1462_57} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1461_56 to 202410_1462_1462_0 into Compact with storage Full 2024.10.22 02:16:15.413495 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1462_57} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:15.425289 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_871_240} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_871_240 to 202410_1_871_240 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:15.429252 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_871_240} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_870_239, 202410_871_871_0] -> 202410_1_871_240 2024.10.22 02:16:15.436088 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1462_57} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1461_56, total 16804 rows starting from the beginning of the part 2024.10.22 02:16:15.445740 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} MergedBlockOutputStream: filled checksums 202410_827_832_1 (state Temporary) 2024.10.22 02:16:15.457686 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:16:15.468720 [ 1229 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 02:16:15.474617 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1462_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_1462_1462_0, total 42 rows starting from the beginning of the part 2024.10.22 02:16:15.512554 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:15.523822 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:15.542331 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96206 2024.10.22 02:16:15.628368 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_827_832_1 to 202410_827_832_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:15.632452 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_827_832_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_827_827_0, 202410_832_832_0] -> 202410_827_832_1 2024.10.22 02:16:15.633057 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_698_698_0 (state Temporary) 2024.10.22 02:16:15.677430 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:16:15.813688 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_698_698_0 to 202410_708_708_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:15.859459 [ 1225 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_703_143 to 202410_708_708_0 2024.10.22 02:16:15.957421 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 716 2024.10.22 02:16:15.983000 [ 1225 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:15.985579 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 2.00 GiB, free memory in arenas 15.51 MiB, will set to 1.38 GiB (RSS), difference: -15.04 MiB 2024.10.22 02:16:16.088255 [ 1225 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:16.098144 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 255 ms. 2024.10.22 02:16:16.355425 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_703_143 to 202410_708_708_0 into Compact with storage Full 2024.10.22 02:16:16.426092 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1362_31, 202410_1363_1363_0, 202410_1364_1364_0, 202410_1365_1365_0, 202410_1366_1366_0, 202410_1367_1367_0] 2024.10.22 02:16:16.436291 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1362_31, 202410_1363_1363_0, 202410_1364_1364_0, 202410_1365_1365_0, 202410_1366_1366_0, 202410_1367_1367_0] 2024.10.22 02:16:16.459639 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1362_31, 202410_1363_1363_0, 202410_1364_1364_0, 202410_1365_1365_0, 202410_1366_1366_0, 202410_1367_1367_0] 2024.10.22 02:16:16.610248 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:16.637977 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_703_143, total 747 rows starting from the beginning of the part 2024.10.22 02:16:16.642834 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1462_57/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:16:16.685824 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1462_57} MergeTask::MergeProjectionsStage: Merge sorted 16846 rows, containing 13 columns (13 merged, 0 gathered) in 1.35704584 sec., 12413.729517051539 rows/sec., 2.39 MiB/sec. 2024.10.22 02:16:16.712944 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_704_704_0, total 1 rows starting from the beginning of the part 2024.10.22 02:16:16.877329 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1462_57} MergedBlockOutputStream: filled checksums 202410_1207_1462_57 (state Temporary) 2024.10.22 02:16:16.939331 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_705_705_0, total 1 rows starting from the beginning of the part 2024.10.22 02:16:16.987795 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1462_57} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1462_57 to 202410_1207_1462_57 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:16.996283 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1462_57} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1461_56, 202410_1462_1462_0] -> 202410_1207_1462_57 2024.10.22 02:16:17.009396 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_706_706_0, total 1 rows starting from the beginning of the part 2024.10.22 02:16:17.043022 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.43 MiB. 2024.10.22 02:16:17.071278 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_707_707_0, total 1 rows starting from the beginning of the part 2024.10.22 02:16:17.155752 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_708_708_0, total 1 rows starting from the beginning of the part 2024.10.22 02:16:17.335972 [ 1250 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 02:16:18.312917 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 2.00 GiB, free memory in arenas 14.54 MiB, will set to 1.39 GiB (RSS), difference: 15.44 MiB 2024.10.22 02:16:18.370606 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 138 entries to flush up to offset 93873 2024.10.22 02:16:18.611710 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:18.614839 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:18.774455 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_708_144/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:16:18.864974 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_848_848_0 (state Temporary) 2024.10.22 02:16:18.933266 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_848_848_0 to 202410_854_854_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:18.979879 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 93873 2024.10.22 02:16:19.176357 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} MergeTask::MergeProjectionsStage: Merge sorted 752 rows, containing 32 columns (32 merged, 0 gathered) in 3.06071367 sec., 245.69433180595425 rows/sec., 22.18 KiB/sec. 2024.10.22 02:16:19.234563 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} MergedBlockOutputStream: filled checksums 202410_1_708_144 (state Temporary) 2024.10.22 02:16:19.555648 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 96269 2024.10.22 02:16:19.796650 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:19.802456 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:19.819777 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 02:16:19.823847 [ 1274 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 209 ms. 2024.10.22 02:16:19.897202 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1454_1454_0 (state Temporary) 2024.10.22 02:16:20.040707 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1454_1454_0 to 202410_1467_1467_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:20.175796 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1462_57 to 202410_1463_1463_0 2024.10.22 02:16:20.192409 [ 1252 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:20.196995 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_708_144 to 202410_1_708_144 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:20.200640 [ 1252 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:20.202830 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_708_144} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_703_143, 202410_708_708_0] -> 202410_1_708_144 2024.10.22 02:16:20.221461 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1463_58} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1462_57 to 202410_1463_1463_0 into Compact with storage Full 2024.10.22 02:16:20.266677 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1463_58} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:20.270722 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:16:20.302709 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96269 2024.10.22 02:16:20.374637 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1463_58} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1462_57, total 16846 rows starting from the beginning of the part 2024.10.22 02:16:20.407400 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1463_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_1463_1463_0, total 52 rows starting from the beginning of the part 2024.10.22 02:16:20.471236 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 12.54 MiB, will set to 1.39 GiB (RSS), difference: 8.29 MiB 2024.10.22 02:16:20.908792 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 29 entries to flush up to offset 13157 2024.10.22 02:16:20.992830 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1492 entries to flush up to offset 1248226 2024.10.22 02:16:21.050563 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:21.059759 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:21.199581 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_863_863_0 (state Temporary) 2024.10.22 02:16:21.302567 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:16:21.307231 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:16:21.356418 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1463_58} MergeTask::MergeProjectionsStage: Merge sorted 16898 rows, containing 13 columns (13 merged, 0 gathered) in 1.139360437 sec., 14831.127579340375 rows/sec., 2.85 MiB/sec. 2024.10.22 02:16:21.480496 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1463_58} MergedBlockOutputStream: filled checksums 202410_1207_1463_58 (state Temporary) 2024.10.22 02:16:21.561469 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_863_863_0 to 202410_876_876_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:21.595166 [ 1153 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_871_240 to 202410_872_872_0 2024.10.22 02:16:21.704466 [ 1153 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:21.716722 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:21.727233 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:21.733271 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13157 2024.10.22 02:16:21.738944 [ 1153 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:21.754350 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_872_241} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_871_240 to 202410_872_872_0 into Compact with storage Full 2024.10.22 02:16:21.797576 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1463_58} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1463_58 to 202410_1207_1463_58 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:21.825055 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_872_241} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:21.835366 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_872_241} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_871_240, total 13182 rows starting from the beginning of the part 2024.10.22 02:16:21.861643 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_872_241} MergeTreeSequentialSource: Reading 2 marks from part 202410_872_872_0, total 15 rows starting from the beginning of the part 2024.10.22 02:16:21.801785 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1463_58} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1462_57, 202410_1463_1463_0] -> 202410_1207_1463_58 2024.10.22 02:16:21.951445 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.43 MiB. 2024.10.22 02:16:21.960467 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_826_826_0 (state Temporary) 2024.10.22 02:16:22.003838 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_826_826_0 to 202410_833_833_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:22.151794 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1248226 2024.10.22 02:16:22.326895 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 2.00 GiB, free memory in arenas 14.84 MiB, will set to 1.40 GiB (RSS), difference: 5.35 MiB 2024.10.22 02:16:22.373061 [ 1217 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:16:22.408551 [ 1217 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:16:22.500243 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 521689 bytes for b47a6787adc51844acde7645745b9727:0) 2024.10.22 02:16:22.507589 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7b87ad487230d7c9b36530b8f5d4abee, offset: 0, size: 2248593 2024.10.22 02:16:22.516565 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 7b87ad487230d7c9b36530b8f5d4abee for removal 2024.10.22 02:16:22.528030 [ 1476 ] {} CacheMetadata: Key 7b87ad487230d7c9b36530b8f5d4abee is removed from metadata 2024.10.22 02:16:22.529552 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7c4c1851a4256c9ce42bb8fd5b9a8aa4, offset: 0, size: 874 2024.10.22 02:16:22.501540 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_173_173_0 to 202410_176_176_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:22.542539 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 7c4c1851a4256c9ce42bb8fd5b9a8aa4 for removal 2024.10.22 02:16:22.559732 [ 1476 ] {} CacheMetadata: Key 7c4c1851a4256c9ce42bb8fd5b9a8aa4 is removed from metadata 2024.10.22 02:16:22.576205 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_174_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:16:22.791153 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 02:16:22.806522 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 02:16:22.828396 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_872_241} MergeTask::MergeProjectionsStage: Merge sorted 13197 rows, containing 12 columns (12 merged, 0 gathered) in 1.07792587 sec., 12242.956929867541 rows/sec., 1.91 MiB/sec. 2024.10.22 02:16:22.882522 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_872_241} MergedBlockOutputStream: filled checksums 202410_1_872_241 (state Temporary) 2024.10.22 02:16:22.924137 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 02:16:22.960077 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 02:16:23.088632 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 02:16:23.097214 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_872_241} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_872_241 to 202410_1_872_241 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:23.116385 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_872_241} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_871_240, 202410_872_872_0] -> 202410_1_872_241 2024.10.22 02:16:23.226663 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 02:16:23.263829 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:16:23.342806 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RemoteIP6 2024.10.22 02:16:23.368482 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vpa/bbhhtxzkjwmmwmslfzamudadaoeec (LogSeriesLimiter: on interval from 2024-10-22 02:16:08 to 2024-10-22 02:16:23 accepted series 1 / 93 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:16:23.459754 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 717 2024.10.22 02:16:23.661773 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:23.664985 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:23.727676 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_699_699_0 (state Temporary) 2024.10.22 02:16:23.934131 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_699_699_0 to 202410_709_709_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:23.934954 [ 1203 ] {} DatabaseCatalog: Execution took 572 ms. 2024.10.22 02:16:24.049486 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 717 2024.10.22 02:16:24.326187 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 96346 2024.10.22 02:16:24.642498 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:24.651095 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:24.849487 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1455_1455_0 (state Temporary) 2024.10.22 02:16:24.859908 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 2.00 GiB, free memory in arenas 14.53 MiB, will set to 1.41 GiB (RSS), difference: 40.83 MiB 2024.10.22 02:16:24.909731 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1455_1455_0 to 202410_1468_1468_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:24.997231 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96346 2024.10.22 02:16:25.059978 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1463_58 to 202410_1468_1468_0 2024.10.22 02:16:25.088025 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:25.094891 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.66 GiB. 2024.10.22 02:16:25.254303 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1463_58 to 202410_1468_1468_0 into Compact with storage Full 2024.10.22 02:16:25.260959 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 320 ms. 2024.10.22 02:16:25.304116 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:25.432249 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1463_58, total 16898 rows starting from the beginning of the part 2024.10.22 02:16:25.454632 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_826_155/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:16:25.492701 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_1464_1464_0, total 52 rows starting from the beginning of the part 2024.10.22 02:16:25.669404 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_1465_1465_0, total 53 rows starting from the beginning of the part 2024.10.22 02:16:25.743136 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_1466_1466_0, total 62 rows starting from the beginning of the part 2024.10.22 02:16:25.909095 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_1467_1467_0, total 63 rows starting from the beginning of the part 2024.10.22 02:16:25.987285 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_1468_1468_0, total 77 rows starting from the beginning of the part 2024.10.22 02:16:26.554272 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 136 entries to flush up to offset 94009 2024.10.22 02:16:26.595874 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 6970 2024.10.22 02:16:26.718910 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:26.723017 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.65 GiB. 2024.10.22 02:16:26.767031 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 15.41 MiB, will set to 1.41 GiB (RSS), difference: 80.76 MiB 2024.10.22 02:16:26.977330 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_849_849_0 (state Temporary) 2024.10.22 02:16:26.998964 [ 1699 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/veh/snvzcqwirfujishkdorybjlqmfuns, size 68 2024.10.22 02:16:27.154843 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1468_59/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:16:27.414969 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_849_849_0 to 202410_855_855_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:27.618416 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 94009 2024.10.22 02:16:27.676073 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:16:28.744580 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.21 GiB, peak 2.00 GiB, free memory in arenas 11.49 MiB, will set to 1.39 GiB (RSS), difference: 186.76 MiB 2024.10.22 02:16:28.763030 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} MergeTask::MergeProjectionsStage: Merge sorted 17205 rows, containing 13 columns (13 merged, 0 gathered) in 3.598606815 sec., 4781.016900286174 rows/sec., 936.94 KiB/sec. 2024.10.22 02:16:28.997466 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} MergedBlockOutputStream: filled checksums 202410_1207_1468_59 (state Temporary) 2024.10.22 02:16:29.032661 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 96380 2024.10.22 02:16:29.073234 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_17_17_0 (state Temporary) 2024.10.22 02:16:29.082303 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/feo/hqiztsyqrppzqmsqqujxxjcclfvpn (LogSeriesLimiter: on interval from 2024-10-22 02:16:23 to 2024-10-22 02:16:29 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:16:29.175544 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1347 entries to flush up to offset 1249573 2024.10.22 02:16:29.236865 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 13169 2024.10.22 02:16:29.253218 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:29.259367 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.62 GiB. 2024.10.22 02:16:29.270507 [ 1735 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/feo/hqiztsyqrppzqmsqqujxxjcclfvpn, size 4 2024.10.22 02:16:29.325282 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1456_1456_0 (state Temporary) 2024.10.22 02:16:29.361725 [ 1712 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uwt/tnnaflmjnvgnadflybmqrcwzknysr, size 384 2024.10.22 02:16:29.448614 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:29.451782 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:29.502023 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_864_864_0 (state Temporary) 2024.10.22 02:16:29.508338 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1456_1456_0 to 202410_1469_1469_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:29.514500 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1468_59 to 202410_1207_1468_59 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:29.517179 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1468_59} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1463_58, 202410_1468_1468_0] -> 202410_1207_1468_59 2024.10.22 02:16:29.558692 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.70 MiB. 2024.10.22 02:16:29.615216 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96380 2024.10.22 02:16:29.648027 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_864_864_0 to 202410_877_877_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:29.684550 [ 1202 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_872_241 to 202410_873_873_0 2024.10.22 02:16:29.719661 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13169 2024.10.22 02:16:29.898795 [ 1202 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:29.904754 [ 1202 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:29.911198 [ 1202 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 02:16:29.928294 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_873_242} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_872_241 to 202410_873_873_0 into Compact with storage Full 2024.10.22 02:16:29.986890 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_873_242} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:30.172659 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_873_242} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_872_241, total 13197 rows starting from the beginning of the part 2024.10.22 02:16:30.189628 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:30.192843 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:30.282829 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_827_827_0 (state Temporary) 2024.10.22 02:16:30.318692 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_873_242} MergeTreeSequentialSource: Reading 2 marks from part 202410_873_873_0, total 13 rows starting from the beginning of the part 2024.10.22 02:16:30.456273 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_827_827_0 to 202410_834_834_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:30.482477 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 13.05 MiB, will set to 1.38 GiB (RSS), difference: 30.25 MiB 2024.10.22 02:16:30.605407 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1249573 2024.10.22 02:16:31.116022 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_873_242} MergeTask::MergeProjectionsStage: Merge sorted 13210 rows, containing 12 columns (12 merged, 0 gathered) in 1.200901313 sec., 11000.071243989056 rows/sec., 1.72 MiB/sec. 2024.10.22 02:16:31.155687 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_873_242} MergedBlockOutputStream: filled checksums 202410_1_873_242 (state Temporary) 2024.10.22 02:16:31.302620 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:16:31.307168 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:16:31.552525 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 718 2024.10.22 02:16:31.662754 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_873_242} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_873_242 to 202410_1_873_242 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:31.667783 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_873_242} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_872_241, 202410_873_873_0] -> 202410_1_873_242 2024.10.22 02:16:31.799302 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:16:31.829105 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:31.839939 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:31.909332 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_700_700_0 (state Temporary) 2024.10.22 02:16:32.074932 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_700_700_0 to 202410_710_710_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:32.283049 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 718 2024.10.22 02:16:32.835294 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 15.35 MiB, will set to 1.39 GiB (RSS), difference: 6.79 MiB 2024.10.22 02:16:32.862642 [ 1247 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 02:16:33.727119 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 96429 2024.10.22 02:16:33.851475 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_826_155} MergeTask::MergeProjectionsStage: Merge sorted 1250312 rows, containing 4 columns (4 merged, 0 gathered) in 36.689539823 sec., 34078.159770654915 rows/sec., 511.05 KiB/sec. 2024.10.22 02:16:33.967652 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_826_155} MergedBlockOutputStream: filled checksums 202410_1_826_155 (state Temporary) 2024.10.22 02:16:34.085389 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:34.092447 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:34.182241 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 7013 2024.10.22 02:16:34.262810 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1457_1457_0 (state Temporary) 2024.10.22 02:16:34.330275 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_773_773_0, 202410_774_774_0, 202410_775_775_0, 202410_776_776_0, 202410_777_777_0, 202410_778_778_0] 2024.10.22 02:16:34.339161 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_773_773_0, 202410_774_774_0, 202410_775_775_0, 202410_776_776_0, 202410_777_777_0, 202410_778_778_0] 2024.10.22 02:16:34.369266 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_773_773_0, 202410_774_774_0, 202410_775_775_0, 202410_776_776_0, 202410_777_777_0, 202410_778_778_0] 2024.10.22 02:16:34.474756 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1457_1457_0 to 202410_1470_1470_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:34.522531 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 16.93 MiB, will set to 1.39 GiB (RSS), difference: 11.02 MiB 2024.10.22 02:16:34.551953 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96429 2024.10.22 02:16:34.702464 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_792_9, 202410_793_793_0, 202410_794_794_0, 202410_795_795_0, 202410_796_796_0, 202410_797_797_0] 2024.10.22 02:16:34.703840 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_792_9, 202410_793_793_0, 202410_794_794_0, 202410_795_795_0, 202410_796_796_0, 202410_797_797_0] 2024.10.22 02:16:34.740738 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_792_9, 202410_793_793_0, 202410_794_794_0, 202410_795_795_0, 202410_796_796_0, 202410_797_797_0] 2024.10.22 02:16:34.995581 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_174_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:16:35.150315 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 159 entries to flush up to offset 94168 2024.10.22 02:16:35.690879 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:35.696121 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:35.739720 [ 1151 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 244 ms. 2024.10.22 02:16:35.756461 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_850_850_0 (state Temporary) 2024.10.22 02:16:35.936325 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_850_850_0 to 202410_856_856_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:36.151576 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 94168 2024.10.22 02:16:36.300376 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_826_155/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:16:36.377754 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 2.00 GiB, free memory in arenas 13.85 MiB, will set to 1.38 GiB (RSS), difference: -4.82 MiB 2024.10.22 02:16:36.811562 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 2e7014336287a9018a47aa8341560a3c:0) 2024.10.22 02:16:36.814703 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a6882731fa7ce17c413a4aec19d1585e, offset: 0, size: 2246704 2024.10.22 02:16:36.825535 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a6882731fa7ce17c413a4aec19d1585e for removal 2024.10.22 02:16:36.832523 [ 1476 ] {} CacheMetadata: Key a6882731fa7ce17c413a4aec19d1585e is removed from metadata 2024.10.22 02:16:36.831989 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 05c4ec64ef3716db671d087dd5fd79ee, offset: 0, size: 874 2024.10.22 02:16:36.844640 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 05c4ec64ef3716db671d087dd5fd79ee for removal 2024.10.22 02:16:36.863022 [ 1476 ] {} CacheMetadata: Key 05c4ec64ef3716db671d087dd5fd79ee is removed from metadata 2024.10.22 02:16:37.155182 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_826_155} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_826_155 to 202410_1_826_155 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:37.159160 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_826_155} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_820_154, 202410_821_826_1] -> 202410_1_826_155 2024.10.22 02:16:37.189233 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:16:37.223767 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 13188 2024.10.22 02:16:37.327590 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 14.17 MiB, will set to 1.38 GiB (RSS), difference: 2.59 MiB 2024.10.22 02:16:37.416120 [ 1225 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:16:37.534937 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:37.538829 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:37.558461 [ 1225 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:16:37.573189 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_865_865_0 (state Temporary) 2024.10.22 02:16:37.657715 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1875 entries to flush up to offset 1251448 2024.10.22 02:16:37.728473 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_865_865_0 to 202410_878_878_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:37.751542 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_873_242 to 202410_874_874_0 2024.10.22 02:16:37.759548 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13188 2024.10.22 02:16:37.759658 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:37.763128 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:37.791187 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_874_243} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_873_242 to 202410_874_874_0 into Compact with storage Full 2024.10.22 02:16:37.823795 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_874_243} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:37.839388 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_874_243} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_873_242, total 13210 rows starting from the beginning of the part 2024.10.22 02:16:37.862958 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_874_243} MergeTreeSequentialSource: Reading 2 marks from part 202410_874_874_0, total 8 rows starting from the beginning of the part 2024.10.22 02:16:38.318819 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 2.00 GiB, free memory in arenas 11.56 MiB, will set to 1.39 GiB (RSS), difference: -5.73 MiB 2024.10.22 02:16:38.417545 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1367_32, 202410_1368_1368_0, 202410_1369_1369_0, 202410_1370_1370_0, 202410_1371_1371_0, 202410_1372_1372_0] 2024.10.22 02:16:38.421548 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1367_32, 202410_1368_1368_0, 202410_1369_1369_0, 202410_1370_1370_0, 202410_1371_1371_0, 202410_1372_1372_0] 2024.10.22 02:16:38.451870 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1367_32, 202410_1368_1368_0, 202410_1369_1369_0, 202410_1370_1370_0, 202410_1371_1371_0, 202410_1372_1372_0] 2024.10.22 02:16:38.559274 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 96486 2024.10.22 02:16:38.599843 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_874_243} MergeTask::MergeProjectionsStage: Merge sorted 13218 rows, containing 12 columns (12 merged, 0 gathered) in 0.821013755 sec., 16099.608465146848 rows/sec., 2.51 MiB/sec. 2024.10.22 02:16:38.663511 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_874_243} MergedBlockOutputStream: filled checksums 202410_1_874_243 (state Temporary) 2024.10.22 02:16:38.875532 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:38.877362 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:38.966262 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:38.968225 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:39.088641 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_828_828_0 (state Temporary) 2024.10.22 02:16:39.151966 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1458_1458_0 (state Temporary) 2024.10.22 02:16:39.213749 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1458_1458_0 to 202410_1471_1471_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:39.275996 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96486 2024.10.22 02:16:39.293735 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_874_243} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_874_243 to 202410_1_874_243 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:39.299716 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_874_243} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_873_242, 202410_874_874_0] -> 202410_1_874_243 2024.10.22 02:16:39.365165 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_828_828_0 to 202410_835_835_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:39.533438 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1251448 2024.10.22 02:16:39.556325 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:16:39.739835 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 2.00 GiB, free memory in arenas 8.47 MiB, will set to 1.39 GiB (RSS), difference: 3.58 MiB 2024.10.22 02:16:41.307966 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:16:41.309431 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:16:41.437467 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 8.18 MiB, will set to 1.39 GiB (RSS), difference: 5.85 MiB 2024.10.22 02:16:41.904025 [ 1692 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vpa/bbhhtxzkjwmmwmslfzamudadaoeec, size 2648463 2024.10.22 02:16:42.036189 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column WindowName 2024.10.22 02:16:42.044804 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column WindowName 2024.10.22 02:16:42.053236 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column WindowName 2024.10.22 02:16:42.068890 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column WindowName 2024.10.22 02:16:42.084117 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column WindowName 2024.10.22 02:16:42.225006 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column WindowName 2024.10.22 02:16:42.259695 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column WindowName 2024.10.22 02:16:42.417392 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mpj/lmkqvenqedfbebgzjfbznadmjfrup (LogSeriesLimiter: on interval from 2024-10-22 02:16:29 to 2024-10-22 02:16:42 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:16:43.287504 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 29 entries to flush up to offset 96515 2024.10.22 02:16:43.446967 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:43.450932 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:43.492307 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1459_1459_0 (state Temporary) 2024.10.22 02:16:43.544828 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1459_1459_0 to 202410_1472_1472_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:43.660671 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 13.68 MiB, will set to 1.37 GiB (RSS), difference: -10.31 MiB 2024.10.22 02:16:43.676032 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 84 entries to flush up to offset 94252 2024.10.22 02:16:43.695685 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96515 2024.10.22 02:16:43.837774 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:43.841528 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:43.847814 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/ylb/cwcmoewznqbtjsajgwobpgagwlyho. 2024.10.22 02:16:43.973814 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_851_851_0 (state Temporary) 2024.10.22 02:16:44.019565 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_851_851_0 to 202410_857_857_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:44.067262 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_852_21 to 202410_857_857_0 2024.10.22 02:16:44.072551 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:44.075264 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:44.077243 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 94252 2024.10.22 02:16:44.130941 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_852_21 to 202410_857_857_0 into Compact with storage Full 2024.10.22 02:16:44.181379 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:44.196734 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_852_21, total 10294 rows starting from the beginning of the part 2024.10.22 02:16:44.209743 [ 1217 ] {} DatabaseCatalog: Execution took 251 ms. 2024.10.22 02:16:44.238554 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_853_853_0, total 86 rows starting from the beginning of the part 2024.10.22 02:16:44.260667 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_854_854_0, total 138 rows starting from the beginning of the part 2024.10.22 02:16:44.484536 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_855_855_0, total 136 rows starting from the beginning of the part 2024.10.22 02:16:44.518146 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_856_856_0, total 159 rows starting from the beginning of the part 2024.10.22 02:16:44.554380 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} MergeTreeSequentialSource: Reading 2 marks from part 202410_857_857_0, total 84 rows starting from the beginning of the part 2024.10.22 02:16:45.264241 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 13196 2024.10.22 02:16:45.466682 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 12.57 MiB, will set to 1.38 GiB (RSS), difference: -1.64 MiB 2024.10.22 02:16:45.573288 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:45.576610 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:45.635034 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_866_866_0 (state Temporary) 2024.10.22 02:16:45.793317 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_866_866_0 to 202410_879_879_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:45.866768 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.051480451 sec. 2024.10.22 02:16:45.950567 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13196 2024.10.22 02:16:46.030329 [ 1204 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_874_243 to 202410_875_875_0 2024.10.22 02:16:46.259422 [ 1204 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:46.261696 [ 1204 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:46.264815 [ 1204 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 353 ms. 2024.10.22 02:16:46.285674 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_875_244} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_874_243 to 202410_875_875_0 into Compact with storage Full 2024.10.22 02:16:46.455424 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_875_244} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:46.461173 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_875_244} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_874_243, total 13218 rows starting from the beginning of the part 2024.10.22 02:16:46.477630 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_875_244} MergeTreeSequentialSource: Reading 2 marks from part 202410_875_875_0, total 16 rows starting from the beginning of the part 2024.10.22 02:16:46.552875 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2216 entries to flush up to offset 1253664 2024.10.22 02:16:46.739031 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} MergeTask::MergeProjectionsStage: Merge sorted 10897 rows, containing 13 columns (13 merged, 0 gathered) in 2.625761961 sec., 4150.033461468063 rows/sec., 1.80 MiB/sec. 2024.10.22 02:16:46.807845 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} MergedBlockOutputStream: filled checksums 202410_747_857_22 (state Temporary) 2024.10.22 02:16:46.929323 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_857_22 to 202410_747_857_22 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:46.937728 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_857_22} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_852_21, 202410_857_857_0] -> 202410_747_857_22 2024.10.22 02:16:47.047203 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_174_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:16:47.092947 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.37 MiB. 2024.10.22 02:16:47.287160 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 719 2024.10.22 02:16:47.494942 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_875_244} MergeTask::MergeProjectionsStage: Merge sorted 13234 rows, containing 12 columns (12 merged, 0 gathered) in 1.218964142 sec., 10856.759066174402 rows/sec., 1.69 MiB/sec. 2024.10.22 02:16:47.567936 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_875_244} MergedBlockOutputStream: filled checksums 202410_1_875_244 (state Temporary) 2024.10.22 02:16:47.594584 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:47.596991 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:47.707756 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 96569 2024.10.22 02:16:47.872315 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:47.874762 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:47.924088 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:47.937590 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:47.955408 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_829_829_0 (state Temporary) 2024.10.22 02:16:47.981567 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1460_1460_0 (state Temporary) 2024.10.22 02:16:48.032331 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_829_829_0 to 202410_836_836_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:48.058608 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_826_155 to 202410_827_832_1 2024.10.22 02:16:48.067314 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_701_701_0 (state Temporary) 2024.10.22 02:16:48.074435 [ 1251 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:48.091982 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:48.138387 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_832_156} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_826_155 to 202410_827_832_1 into Wide with storage Full 2024.10.22 02:16:48.136166 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1460_1460_0 to 202410_1473_1473_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:48.154524 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_875_244} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_875_244 to 202410_1_875_244 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:48.159312 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_875_244} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_874_243, 202410_875_875_0] -> 202410_1_875_244 2024.10.22 02:16:48.161620 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_832_156} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:48.165341 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1253664 2024.10.22 02:16:48.176570 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:16:48.181432 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_832_156} MergeTreeSequentialSource: Reading 154 marks from part 202410_1_826_155, total 1250312 rows starting from the beginning of the part 2024.10.22 02:16:48.188029 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1468_59 to 202410_1469_1469_0 2024.10.22 02:16:48.199937 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96569 2024.10.22 02:16:48.202492 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:48.209307 [ 1240 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:48.250442 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_832_156} MergeTreeSequentialSource: Reading 3 marks from part 202410_827_832_1, total 10012 rows starting from the beginning of the part 2024.10.22 02:16:48.346705 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1469_60} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1468_59 to 202410_1469_1469_0 into Compact with storage Full 2024.10.22 02:16:48.455803 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_701_701_0 to 202410_711_711_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:48.567547 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1469_60} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:48.755665 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 719 2024.10.22 02:16:48.790376 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1469_60} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1468_59, total 17205 rows starting from the beginning of the part 2024.10.22 02:16:48.864861 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_17_17_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:16:49.065119 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1469_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_1469_1469_0, total 34 rows starting from the beginning of the part 2024.10.22 02:16:49.542180 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_815_192, 202410_816_816_0] 2024.10.22 02:16:49.544920 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_815_192, 202410_816_816_0] 2024.10.22 02:16:49.554805 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_815_192, 202410_816_816_0] 2024.10.22 02:16:49.784685 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 2.00 GiB, free memory in arenas 10.04 MiB, will set to 1.38 GiB (RSS), difference: -13.26 MiB 2024.10.22 02:16:51.019886 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_658_134, 202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0, 202410_663_663_0] 2024.10.22 02:16:51.027329 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_658_134, 202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0, 202410_663_663_0] 2024.10.22 02:16:51.182771 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_658_134, 202410_659_659_0, 202410_660_660_0, 202410_661_661_0, 202410_662_662_0, 202410_663_663_0] 2024.10.22 02:16:51.311380 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:16:51.315010 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:16:51.600839 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 124 entries to flush up to offset 94376 2024.10.22 02:16:51.656762 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 2.00 GiB, free memory in arenas 11.98 MiB, will set to 1.39 GiB (RSS), difference: 9.32 MiB 2024.10.22 02:16:51.955324 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1469_60} MergeTask::MergeProjectionsStage: Merge sorted 17239 rows, containing 13 columns (13 merged, 0 gathered) in 3.591389972 sec., 4800.091366964479 rows/sec., 940.38 KiB/sec. 2024.10.22 02:16:52.011004 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1469_60} MergedBlockOutputStream: filled checksums 202410_1207_1469_60 (state Temporary) 2024.10.22 02:16:52.055016 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:52.058962 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:52.128430 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_852_852_0 (state Temporary) 2024.10.22 02:16:52.139919 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1469_60} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1469_60 to 202410_1207_1469_60 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:52.145130 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1469_60} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1468_59, 202410_1469_1469_0] -> 202410_1207_1469_60 2024.10.22 02:16:52.212493 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.49 MiB. 2024.10.22 02:16:52.214700 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 96619 2024.10.22 02:16:52.391389 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_852_852_0 to 202410_858_858_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:52.461519 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 94376 2024.10.22 02:16:52.565281 [ 1258 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:16:52.589166 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:52.603122 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:52.710264 [ 1258 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:16:52.760668 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1461_1461_0 (state Temporary) 2024.10.22 02:16:52.927697 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1461_1461_0 to 202410_1474_1474_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:52.961695 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1469_60 to 202410_1470_1470_0 2024.10.22 02:16:52.978731 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96619 2024.10.22 02:16:52.981819 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:52.991612 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:53.153697 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1470_61} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1469_60 to 202410_1470_1470_0 into Compact with storage Full 2024.10.22 02:16:53.195567 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1470_61} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:53.208002 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1470_61} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1469_60, total 17239 rows starting from the beginning of the part 2024.10.22 02:16:53.352627 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1470_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_1470_1470_0, total 49 rows starting from the beginning of the part 2024.10.22 02:16:53.452781 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 13215 2024.10.22 02:16:53.467823 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 2.00 GiB, free memory in arenas 23.08 MiB, will set to 1.39 GiB (RSS), difference: 11.30 MiB 2024.10.22 02:16:53.735983 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:53.746592 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:53.830957 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:53.841296 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:53.881288 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_867_867_0 (state Temporary) 2024.10.22 02:16:53.919086 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_832_156/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:16:54.054380 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_867_867_0 to 202410_880_880_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:54.268431 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13215 2024.10.22 02:16:54.269362 [ 1235 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_875_244 to 202410_876_876_0 2024.10.22 02:16:54.273097 [ 1235 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:54.276357 [ 1235 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:54.441406 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_876_245} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_875_244 to 202410_876_876_0 into Compact with storage Full 2024.10.22 02:16:54.469252 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1470_61} MergeTask::MergeProjectionsStage: Merge sorted 17288 rows, containing 13 columns (13 merged, 0 gathered) in 1.328321277 sec., 13014.92364787288 rows/sec., 2.49 MiB/sec. 2024.10.22 02:16:54.471602 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_876_245} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:54.476711 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_876_245} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_875_244, total 13234 rows starting from the beginning of the part 2024.10.22 02:16:54.537561 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_876_245} MergeTreeSequentialSource: Reading 2 marks from part 202410_876_876_0, total 29 rows starting from the beginning of the part 2024.10.22 02:16:54.654168 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1470_61} MergedBlockOutputStream: filled checksums 202410_1207_1470_61 (state Temporary) 2024.10.22 02:16:54.872018 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1470_61} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1470_61 to 202410_1207_1470_61 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:54.879836 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1470_61} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1469_60, 202410_1470_1470_0] -> 202410_1207_1470_61 2024.10.22 02:16:54.945216 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.56 MiB. 2024.10.22 02:16:55.171741 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1255204 2024.10.22 02:16:55.246622 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_766_145, 202410_767_772_1] 2024.10.22 02:16:55.249276 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_766_145, 202410_767_772_1] 2024.10.22 02:16:55.281527 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_766_145, 202410_767_772_1] 2024.10.22 02:16:55.530432 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_876_245} MergeTask::MergeProjectionsStage: Merge sorted 13263 rows, containing 12 columns (12 merged, 0 gathered) in 1.201650758 sec., 11037.31671760823 rows/sec., 1.72 MiB/sec. 2024.10.22 02:16:55.625350 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_876_245} MergedBlockOutputStream: filled checksums 202410_1_876_245 (state Temporary) 2024.10.22 02:16:55.689236 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 2.00 GiB, free memory in arenas 21.18 MiB, will set to 1.39 GiB (RSS), difference: -15.13 MiB 2024.10.22 02:16:55.742936 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_816_193, 202410_817_817_0] 2024.10.22 02:16:55.746723 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_816_193, 202410_817_817_0] 2024.10.22 02:16:55.762347 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_816_193, 202410_817_817_0] 2024.10.22 02:16:56.087755 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_876_245} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_876_245 to 202410_1_876_245 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:56.100260 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_876_245} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_875_244, 202410_876_876_0] -> 202410_1_876_245 2024.10.22 02:16:56.142768 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:16:56.260188 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 720 2024.10.22 02:16:56.526937 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:56.533184 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:56.792106 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_830_830_0 (state Temporary) 2024.10.22 02:16:56.886472 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_830_830_0 to 202410_837_837_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:56.932502 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:56.947459 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:56.955497 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1255204 2024.10.22 02:16:56.987483 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 96680 2024.10.22 02:16:57.166786 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:57.169082 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:57.226611 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_702_702_0 (state Temporary) 2024.10.22 02:16:57.316754 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_702_702_0 to 202410_712_712_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:57.404991 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1462_1462_0 (state Temporary) 2024.10.22 02:16:57.507776 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 720 2024.10.22 02:16:57.510667 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 2.00 GiB, free memory in arenas 18.92 MiB, will set to 1.39 GiB (RSS), difference: 3.95 MiB 2024.10.22 02:16:57.589381 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1462_1462_0 to 202410_1475_1475_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:16:57.617107 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1470_61 to 202410_1471_1471_0 2024.10.22 02:16:57.628349 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:16:57.631285 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:16:57.697117 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1471_62} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1470_61 to 202410_1471_1471_0 into Compact with storage Full 2024.10.22 02:16:57.751255 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96680 2024.10.22 02:16:57.755654 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1471_62} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:16:57.818643 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1471_62} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1470_61, total 17288 rows starting from the beginning of the part 2024.10.22 02:16:57.891405 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1471_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_1471_1471_0, total 57 rows starting from the beginning of the part 2024.10.22 02:16:58.348341 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_174_174_0 (state Temporary) 2024.10.22 02:16:58.876977 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 790957 bytes for 385b3316eab05e8e8768e9a64e94a7ab:0) 2024.10.22 02:16:58.880117 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: fb4763fa40288d4098bfca6554b133c5, offset: 0, size: 1862081 2024.10.22 02:16:58.884790 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key fb4763fa40288d4098bfca6554b133c5 for removal 2024.10.22 02:16:58.890231 [ 1476 ] {} CacheMetadata: Key fb4763fa40288d4098bfca6554b133c5 is removed from metadata 2024.10.22 02:16:58.891135 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6552b2aba9b4662ab52a242c9a2319ac, offset: 0, size: 865 2024.10.22 02:16:58.896374 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 6552b2aba9b4662ab52a242c9a2319ac for removal 2024.10.22 02:16:58.919414 [ 1476 ] {} CacheMetadata: Key 6552b2aba9b4662ab52a242c9a2319ac is removed from metadata 2024.10.22 02:16:59.344338 [ 1691 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mpj/lmkqvenqedfbebgzjfbznadmjfrup, size 1839533 2024.10.22 02:16:59.484265 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OpenerName 2024.10.22 02:16:59.517657 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OpenerName 2024.10.22 02:16:59.547562 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OpenerName 2024.10.22 02:16:59.565358 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 12.09 MiB, will set to 1.37 GiB (RSS), difference: 13.23 MiB 2024.10.22 02:16:59.659645 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OpenerName 2024.10.22 02:16:59.675611 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OpenerName 2024.10.22 02:16:59.797394 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OpenerName 2024.10.22 02:16:59.815832 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OpenerName 2024.10.22 02:16:59.856265 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/owk/dnjkxlhmgmnbjejylnpooaebvkeij (LogSeriesLimiter: on interval from 2024-10-22 02:16:42 to 2024-10-22 02:16:59 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:16:59.924531 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1471_62} MergeTask::MergeProjectionsStage: Merge sorted 17345 rows, containing 13 columns (13 merged, 0 gathered) in 2.242481811 sec., 7734.733862686388 rows/sec., 1.48 MiB/sec. 2024.10.22 02:16:59.980972 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 112 entries to flush up to offset 94488 2024.10.22 02:16:59.984553 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1471_62} MergedBlockOutputStream: filled checksums 202410_1207_1471_62 (state Temporary) 2024.10.22 02:17:00.089474 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1471_62} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1471_62 to 202410_1207_1471_62 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:00.127706 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:00.133328 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:00.170331 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_853_853_0 (state Temporary) 2024.10.22 02:17:00.196780 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1471_62} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1470_61, 202410_1471_1471_0] -> 202410_1207_1471_62 2024.10.22 02:17:00.340987 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_853_853_0 to 202410_859_859_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:00.357069 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.56 MiB. 2024.10.22 02:17:00.406211 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 94488 2024.10.22 02:17:01.314681 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:17:01.317241 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:17:01.652530 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 449 ms. 2024.10.22 02:17:01.760788 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 96729 2024.10.22 02:17:01.771518 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 13229 2024.10.22 02:17:01.940255 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 10.93 MiB, will set to 1.36 GiB (RSS), difference: -12.01 MiB 2024.10.22 02:17:02.031876 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:02.034445 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:02.187494 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:02.191880 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:02.294568 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1463_1463_0 (state Temporary) 2024.10.22 02:17:02.337569 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1463_1463_0 to 202410_1476_1476_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:02.353207 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_868_868_0 (state Temporary) 2024.10.22 02:17:02.468687 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1471_62 to 202410_1472_1472_0 2024.10.22 02:17:02.480130 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:02.505183 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_868_868_0 to 202410_881_881_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:02.518973 [ 1263 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:02.558998 [ 1263 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_876_245 to 202410_877_877_0 2024.10.22 02:17:02.564724 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1472_63} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1471_62 to 202410_1472_1472_0 into Compact with storage Full 2024.10.22 02:17:02.565570 [ 1263 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:02.571654 [ 1263 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:02.620090 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_877_246} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_876_245 to 202410_877_877_0 into Compact with storage Full 2024.10.22 02:17:02.573709 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13229 2024.10.22 02:17:02.682366 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96729 2024.10.22 02:17:02.767175 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_877_246} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:02.767468 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1472_63} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:02.772359 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_877_246} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_876_245, total 13263 rows starting from the beginning of the part 2024.10.22 02:17:02.797952 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1472_63} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1471_62, total 17345 rows starting from the beginning of the part 2024.10.22 02:17:02.816599 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_877_246} MergeTreeSequentialSource: Reading 2 marks from part 202410_877_877_0, total 12 rows starting from the beginning of the part 2024.10.22 02:17:02.917757 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1472_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_1472_1472_0, total 29 rows starting from the beginning of the part 2024.10.22 02:17:03.470764 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 17.11 MiB, will set to 1.36 GiB (RSS), difference: -13.58 MiB 2024.10.22 02:17:03.667821 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_877_246} MergeTask::MergeProjectionsStage: Merge sorted 13275 rows, containing 12 columns (12 merged, 0 gathered) in 1.086290626 sec., 12220.48656433863 rows/sec., 1.91 MiB/sec. 2024.10.22 02:17:03.824187 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_877_246} MergedBlockOutputStream: filled checksums 202410_1_877_246 (state Temporary) 2024.10.22 02:17:03.960583 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1800 entries to flush up to offset 1257004 2024.10.22 02:17:04.050020 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_877_246} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_877_246 to 202410_1_877_246 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:04.053607 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_877_246} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_876_245, 202410_877_877_0] -> 202410_1_877_246 2024.10.22 02:17:04.076533 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:17:04.079852 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1372_33, 202410_1373_1373_0, 202410_1374_1374_0, 202410_1375_1375_0, 202410_1376_1376_0, 202410_1377_1377_0] 2024.10.22 02:17:04.091219 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1372_33, 202410_1373_1373_0, 202410_1374_1374_0, 202410_1375_1375_0, 202410_1376_1376_0, 202410_1377_1377_0] 2024.10.22 02:17:04.406616 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_832_156/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:17:04.460928 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1372_33, 202410_1373_1373_0, 202410_1374_1374_0, 202410_1375_1375_0, 202410_1376_1376_0, 202410_1377_1377_0] 2024.10.22 02:17:04.562636 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1472_63} MergeTask::MergeProjectionsStage: Merge sorted 17374 rows, containing 13 columns (13 merged, 0 gathered) in 2.017177594 sec., 8613.024481175156 rows/sec., 1.65 MiB/sec. 2024.10.22 02:17:04.613035 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1472_63} MergedBlockOutputStream: filled checksums 202410_1207_1472_63 (state Temporary) 2024.10.22 02:17:04.707398 [ 1177 ] {} DatabaseCatalog: Execution took 355 ms. 2024.10.22 02:17:04.823883 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1472_63} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1472_63 to 202410_1207_1472_63 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:04.835217 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1472_63} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1471_62, 202410_1472_1472_0] -> 202410_1207_1472_63 2024.10.22 02:17:04.867190 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:04.872176 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:04.876709 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.57 MiB. 2024.10.22 02:17:05.010465 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 721 2024.10.22 02:17:05.135326 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_831_831_0 (state Temporary) 2024.10.22 02:17:05.283140 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:05.294421 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:05.313180 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_831_831_0 to 202410_838_838_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:05.365224 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1257004 2024.10.22 02:17:05.367125 [ 1232 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_833_833_0 to 202410_838_838_0 2024.10.22 02:17:05.372812 [ 1232 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:05.376016 [ 1232 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:05.409044 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_833_833_0 to 202410_838_838_0 into Compact with storage Full 2024.10.22 02:17:05.456190 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_703_703_0 (state Temporary) 2024.10.22 02:17:05.504864 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_817_194, 202410_818_818_0] 2024.10.22 02:17:05.510635 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_817_194, 202410_818_818_0] 2024.10.22 02:17:05.539258 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_817_194, 202410_818_818_0] 2024.10.22 02:17:05.541370 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_703_703_0 to 202410_713_713_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:05.546462 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:05.549432 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 17.33 MiB, will set to 1.36 GiB (RSS), difference: 51.48 MiB 2024.10.22 02:17:05.551627 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_833_833_0, total 1492 rows starting from the beginning of the part 2024.10.22 02:17:05.562549 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_834_834_0, total 1347 rows starting from the beginning of the part 2024.10.22 02:17:05.565037 [ 1188 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_708_144 to 202410_713_713_0 2024.10.22 02:17:05.572925 [ 1188 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:05.573273 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_835_835_0, total 1875 rows starting from the beginning of the part 2024.10.22 02:17:05.612973 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_836_836_0, total 2216 rows starting from the beginning of the part 2024.10.22 02:17:05.635502 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_837_837_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:17:05.675440 [ 1188 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:05.681266 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_838_838_0, total 1800 rows starting from the beginning of the part 2024.10.22 02:17:05.702403 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 721 2024.10.22 02:17:05.718641 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_708_144 to 202410_713_713_0 into Compact with storage Full 2024.10.22 02:17:06.058853 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:06.110433 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_708_144, total 752 rows starting from the beginning of the part 2024.10.22 02:17:06.167416 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/ylb/cwcmoewznqbtjsajgwobpgagwlyho. 2024.10.22 02:17:06.315045 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} MergeTreeSequentialSource: Reading 2 marks from part 202410_709_709_0, total 1 rows starting from the beginning of the part 2024.10.22 02:17:06.366275 [ 1237 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 211 ms. 2024.10.22 02:17:06.697262 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 96808 2024.10.22 02:17:06.728738 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} MergeTreeSequentialSource: Reading 2 marks from part 202410_710_710_0, total 1 rows starting from the beginning of the part 2024.10.22 02:17:06.912189 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_174_174_0 to 202410_177_177_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:06.993098 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_174_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:17:07.065782 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:07.069083 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:07.148353 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1464_1464_0 (state Temporary) 2024.10.22 02:17:07.150478 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} MergeTask::MergeProjectionsStage: Merge sorted 10270 rows, containing 4 columns (4 merged, 0 gathered) in 1.747538542 sec., 5876.837479215952 rows/sec., 91.83 KiB/sec. 2024.10.22 02:17:07.185580 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} MergedBlockOutputStream: filled checksums 202410_833_838_1 (state Temporary) 2024.10.22 02:17:07.247945 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_833_838_1 to 202410_833_838_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:07.253608 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_833_838_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_833_833_0, 202410_838_838_0] -> 202410_833_838_1 2024.10.22 02:17:07.297421 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1464_1464_0 to 202410_1477_1477_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:07.354239 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1472_63 to 202410_1473_1473_0 2024.10.22 02:17:07.362747 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:07.365203 [ 1171 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:07.399961 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96808 2024.10.22 02:17:07.463768 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} MergeTreeSequentialSource: Reading 2 marks from part 202410_711_711_0, total 1 rows starting from the beginning of the part 2024.10.22 02:17:07.465580 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:17:07.474644 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1473_64} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1472_63 to 202410_1473_1473_0 into Compact with storage Full 2024.10.22 02:17:07.502837 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1473_64} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:07.536710 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1473_64} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1472_63, total 17374 rows starting from the beginning of the part 2024.10.22 02:17:07.692599 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1473_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_1473_1473_0, total 54 rows starting from the beginning of the part 2024.10.22 02:17:07.718420 [ 1173 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:17:07.769566 [ 1173 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:17:07.797321 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.26 GiB, peak 2.00 GiB, free memory in arenas 12.58 MiB, will set to 1.35 GiB (RSS), difference: 86.09 MiB 2024.10.22 02:17:07.931809 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 103 entries to flush up to offset 94591 2024.10.22 02:17:07.946841 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} MergeTreeSequentialSource: Reading 2 marks from part 202410_712_712_0, total 1 rows starting from the beginning of the part 2024.10.22 02:17:08.255278 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:08.259446 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:08.267806 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} MergeTreeSequentialSource: Reading 2 marks from part 202410_713_713_0, total 1 rows starting from the beginning of the part 2024.10.22 02:17:08.526082 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_854_854_0 (state Temporary) 2024.10.22 02:17:08.597029 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_854_854_0 to 202410_860_860_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:08.758712 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 94591 2024.10.22 02:17:09.397520 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1473_64} MergeTask::MergeProjectionsStage: Merge sorted 17428 rows, containing 13 columns (13 merged, 0 gathered) in 1.935897575 sec., 9002.542399486192 rows/sec., 1.72 MiB/sec. 2024.10.22 02:17:09.470328 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1473_64} MergedBlockOutputStream: filled checksums 202410_1207_1473_64 (state Temporary) 2024.10.22 02:17:09.628336 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.29 GiB, peak 2.00 GiB, free memory in arenas 15.28 MiB, will set to 1.34 GiB (RSS), difference: 57.67 MiB 2024.10.22 02:17:09.659907 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1473_64} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1473_64 to 202410_1207_1473_64 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:09.673248 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1473_64} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1472_63, 202410_1473_1473_0] -> 202410_1207_1473_64 2024.10.22 02:17:09.724313 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.58 MiB. 2024.10.22 02:17:09.835294 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for c622cd6cd49bc4d911093d8feb7dad8a:0) 2024.10.22 02:17:09.840532 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 63ee9bf5d16cc4b7293c2d3c556ac111, offset: 0, size: 2097152 2024.10.22 02:17:09.849661 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: da1de5d47229b5b5f43e637c4fdd19dd, offset: 0, size: 911 2024.10.22 02:17:09.855096 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key da1de5d47229b5b5f43e637c4fdd19dd for removal 2024.10.22 02:17:09.857372 [ 1476 ] {} CacheMetadata: Key da1de5d47229b5b5f43e637c4fdd19dd is removed from metadata 2024.10.22 02:17:10.179899 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 13252 2024.10.22 02:17:10.359643 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} MergeTask::MergeProjectionsStage: Merge sorted 757 rows, containing 32 columns (32 merged, 0 gathered) in 4.631263731 sec., 163.45430620435553 rows/sec., 14.75 KiB/sec. 2024.10.22 02:17:10.375752 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 1048576 bytes for c622cd6cd49bc4d911093d8feb7dad8a:0) 2024.10.22 02:17:10.377391 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 63ee9bf5d16cc4b7293c2d3c556ac111, offset: 2097152, size: 2163384 2024.10.22 02:17:10.382489 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 63ee9bf5d16cc4b7293c2d3c556ac111 for removal 2024.10.22 02:17:10.386839 [ 1476 ] {} CacheMetadata: Key 63ee9bf5d16cc4b7293c2d3c556ac111 is removed from metadata 2024.10.22 02:17:10.416539 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:10.420202 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:10.457036 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_869_869_0 (state Temporary) 2024.10.22 02:17:10.493747 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} MergedBlockOutputStream: filled checksums 202410_1_713_145 (state Temporary) 2024.10.22 02:17:10.683015 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_869_869_0 to 202410_882_882_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:10.775622 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13252 2024.10.22 02:17:10.822453 [ 1196 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_877_246 to 202410_878_878_0 2024.10.22 02:17:10.839564 [ 1196 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:10.845030 [ 1196 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:10.857126 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_878_247} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_877_246 to 202410_878_878_0 into Compact with storage Full 2024.10.22 02:17:10.962939 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_878_247} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:10.975462 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_878_247} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_877_246, total 13275 rows starting from the beginning of the part 2024.10.22 02:17:11.019785 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_713_145 to 202410_1_713_145 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:11.027801 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_713_145} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_708_144, 202410_713_713_0] -> 202410_1_713_145 2024.10.22 02:17:11.035010 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_878_247} MergeTreeSequentialSource: Reading 2 marks from part 202410_878_878_0, total 19 rows starting from the beginning of the part 2024.10.22 02:17:11.090164 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:17:11.315184 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:17:11.316527 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:17:11.356577 [ 1193 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 216 ms. 2024.10.22 02:17:11.409610 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 96877 2024.10.22 02:17:11.660005 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7013 2024.10.22 02:17:11.821021 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:11.826666 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:11.926088 [ 1258 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 02:17:11.962194 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 16.62 MiB, will set to 1.34 GiB (RSS), difference: 3.03 MiB 2024.10.22 02:17:12.004467 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1465_1465_0 (state Temporary) 2024.10.22 02:17:12.042916 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_818_195, 202410_819_819_0] 2024.10.22 02:17:12.045180 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_818_195, 202410_819_819_0] 2024.10.22 02:17:12.048051 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_878_247} MergeTask::MergeProjectionsStage: Merge sorted 13294 rows, containing 12 columns (12 merged, 0 gathered) in 1.195800377 sec., 11117.240181301599 rows/sec., 1.73 MiB/sec. 2024.10.22 02:17:12.059019 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_818_195, 202410_819_819_0] 2024.10.22 02:17:12.075425 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1465_1465_0 to 202410_1478_1478_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:12.094399 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_878_247} MergedBlockOutputStream: filled checksums 202410_1_878_247 (state Temporary) 2024.10.22 02:17:12.121417 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1473_64 to 202410_1478_1478_0 2024.10.22 02:17:12.138489 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:12.143316 [ 1184 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:12.147783 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96877 2024.10.22 02:17:12.155999 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1473_64 to 202410_1478_1478_0 into Compact with storage Full 2024.10.22 02:17:12.194498 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:12.228768 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 4567180f243ee412b0953b3934c82fbc:0) 2024.10.22 02:17:12.231750 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 497c362e0eabd9dd8b5457ecb4ead15c, offset: 0, size: 1048576 2024.10.22 02:17:12.244885 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d291aa1d114f72433f487d1ab5723dbc, offset: 0, size: 876 2024.10.22 02:17:12.247885 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d291aa1d114f72433f487d1ab5723dbc for removal 2024.10.22 02:17:12.252092 [ 1476 ] {} CacheMetadata: Key d291aa1d114f72433f487d1ab5723dbc is removed from metadata 2024.10.22 02:17:12.305037 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1473_64, total 17428 rows starting from the beginning of the part 2024.10.22 02:17:12.327990 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_1474_1474_0, total 50 rows starting from the beginning of the part 2024.10.22 02:17:12.369540 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1353 entries to flush up to offset 1258357 2024.10.22 02:17:12.450795 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_1475_1475_0, total 61 rows starting from the beginning of the part 2024.10.22 02:17:12.468757 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_1476_1476_0, total 49 rows starting from the beginning of the part 2024.10.22 02:17:12.481353 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_878_247} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_878_247 to 202410_1_878_247 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:12.488184 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_878_247} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_877_246, 202410_878_878_0] -> 202410_1_878_247 2024.10.22 02:17:12.502408 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_1477_1477_0, total 79 rows starting from the beginning of the part 2024.10.22 02:17:12.525751 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:17:12.763285 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_1478_1478_0, total 69 rows starting from the beginning of the part 2024.10.22 02:17:13.081503 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:13.091129 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:13.117609 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 60829 bytes for c59514a927311cf4024bec70665bd6d4:0) 2024.10.22 02:17:13.120798 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 497c362e0eabd9dd8b5457ecb4ead15c, offset: 1048576, size: 1189165 2024.10.22 02:17:13.131316 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 497c362e0eabd9dd8b5457ecb4ead15c for removal 2024.10.22 02:17:13.135782 [ 1476 ] {} CacheMetadata: Key 497c362e0eabd9dd8b5457ecb4ead15c is removed from metadata 2024.10.22 02:17:13.205554 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 722 2024.10.22 02:17:13.331565 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.35 GiB, peak 2.00 GiB, free memory in arenas 19.76 MiB, will set to 1.34 GiB (RSS), difference: -4.39 MiB 2024.10.22 02:17:13.335016 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_832_832_0 (state Temporary) 2024.10.22 02:17:13.520855 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_832_832_0 to 202410_839_839_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:13.546489 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1258357 2024.10.22 02:17:13.572177 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:13.576232 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:13.652566 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_704_704_0 (state Temporary) 2024.10.22 02:17:13.822517 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_704_704_0 to 202410_714_714_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:13.915069 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 722 2024.10.22 02:17:14.328892 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} MergeTask::MergeProjectionsStage: Merge sorted 17736 rows, containing 13 columns (13 merged, 0 gathered) in 2.178026912 sec., 8143.1500695800405 rows/sec., 1.55 MiB/sec. 2024.10.22 02:17:14.431297 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 735763 bytes for b25e25398941d0f39d56746a0c823dd3:0) 2024.10.22 02:17:14.433360 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ed3c5fc983fb77b3521847057998c497, offset: 0, size: 9455484 2024.10.22 02:17:14.447163 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ed3c5fc983fb77b3521847057998c497 for removal 2024.10.22 02:17:14.452916 [ 1476 ] {} CacheMetadata: Key ed3c5fc983fb77b3521847057998c497 is removed from metadata 2024.10.22 02:17:14.455566 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7d6f4b6cd6134cf4b68221d511ac00d9, offset: 0, size: 967 2024.10.22 02:17:14.459501 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7d6f4b6cd6134cf4b68221d511ac00d9 for removal 2024.10.22 02:17:14.463333 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4960992b2d4294bcc7d0e6eecca60e42, offset: 0, size: 11916 2024.10.22 02:17:14.466385 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} MergedBlockOutputStream: filled checksums 202410_1207_1478_65 (state Temporary) 2024.10.22 02:17:14.467379 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4960992b2d4294bcc7d0e6eecca60e42 for removal 2024.10.22 02:17:14.467534 [ 1476 ] {} CacheMetadata: Key 7d6f4b6cd6134cf4b68221d511ac00d9 is removed from metadata 2024.10.22 02:17:14.470582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8fa13054d786fe0ed6b82ef263f800bc, offset: 0, size: 481 2024.10.22 02:17:14.475360 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8fa13054d786fe0ed6b82ef263f800bc for removal 2024.10.22 02:17:14.544335 [ 1476 ] {} CacheMetadata: Key 8fa13054d786fe0ed6b82ef263f800bc is removed from metadata 2024.10.22 02:17:14.552259 [ 1476 ] {} CacheMetadata: Key 4960992b2d4294bcc7d0e6eecca60e42 is removed from metadata 2024.10.22 02:17:14.567042 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1478_65 to 202410_1207_1478_65 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:14.570686 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1478_65} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1473_64, 202410_1478_1478_0] -> 202410_1207_1478_65 2024.10.22 02:17:14.617195 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.81 MiB. 2024.10.22 02:17:14.724618 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_797_10, 202410_798_798_0, 202410_799_799_0, 202410_800_800_0, 202410_801_801_0, 202410_802_802_0] 2024.10.22 02:17:14.726679 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_797_10, 202410_798_798_0, 202410_799_799_0, 202410_800_800_0, 202410_801_801_0, 202410_802_802_0] 2024.10.22 02:17:14.757097 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_797_10, 202410_798_798_0, 202410_799_799_0, 202410_800_800_0, 202410_801_801_0, 202410_802_802_0] 2024.10.22 02:17:15.146759 [ 1702 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/owk/dnjkxlhmgmnbjejylnpooaebvkeij, size 50972 2024.10.22 02:17:15.172502 [ 1270 ] {} DatabaseCatalog: Execution took 367 ms. 2024.10.22 02:17:15.412770 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column HistoryLength 2024.10.22 02:17:15.424015 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column HistoryLength 2024.10.22 02:17:15.556436 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column HistoryLength 2024.10.22 02:17:15.575632 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column HistoryLength 2024.10.22 02:17:15.591076 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column HistoryLength 2024.10.22 02:17:15.603811 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column HistoryLength 2024.10.22 02:17:15.623049 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column HistoryLength 2024.10.22 02:17:15.686137 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rzl/xkzpkxjpvtymueeisceiqmdgowhrf (LogSeriesLimiter: on interval from 2024-10-22 02:16:59 to 2024-10-22 02:17:15 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:17:15.720901 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 18.56 MiB, will set to 1.34 GiB (RSS), difference: 8.83 MiB 2024.10.22 02:17:16.165011 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 96959 2024.10.22 02:17:16.284561 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 124 entries to flush up to offset 94715 2024.10.22 02:17:16.663367 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:16.665004 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:16.703517 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1466_1466_0 (state Temporary) 2024.10.22 02:17:16.749402 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1466_1466_0 to 202410_1479_1479_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:16.819484 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96959 2024.10.22 02:17:16.831066 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_832_156/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:17:16.858986 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:16.861937 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:16.896399 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_855_855_0 (state Temporary) 2024.10.22 02:17:17.144125 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_855_855_0 to 202410_861_861_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:17.176391 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 94715 2024.10.22 02:17:17.507571 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 16.67 MiB, will set to 1.34 GiB (RSS), difference: -3.58 MiB 2024.10.22 02:17:18.277237 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 13275 2024.10.22 02:17:18.371100 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:18.372561 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:18.518446 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_870_870_0 (state Temporary) 2024.10.22 02:17:18.560117 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_870_870_0 to 202410_883_883_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:18.586333 [ 1278 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_878_247 to 202410_879_879_0 2024.10.22 02:17:18.596384 [ 1278 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:18.598806 [ 1278 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:18.616501 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13275 2024.10.22 02:17:18.631021 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_879_248} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_878_247 to 202410_879_879_0 into Compact with storage Full 2024.10.22 02:17:18.676016 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_879_248} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:18.691517 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_879_248} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_878_247, total 13294 rows starting from the beginning of the part 2024.10.22 02:17:18.727669 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_879_248} MergeTreeSequentialSource: Reading 2 marks from part 202410_879_879_0, total 8 rows starting from the beginning of the part 2024.10.22 02:17:19.231635 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 45 entries to flush up to offset 7058 2024.10.22 02:17:19.420210 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_879_248} MergeTask::MergeProjectionsStage: Merge sorted 13302 rows, containing 12 columns (12 merged, 0 gathered) in 0.809344292 sec., 16435.52704514533 rows/sec., 2.56 MiB/sec. 2024.10.22 02:17:19.508012 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_879_248} MergedBlockOutputStream: filled checksums 202410_1_879_248 (state Temporary) 2024.10.22 02:17:19.533275 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 13.91 MiB, will set to 1.34 GiB (RSS), difference: -3.71 MiB 2024.10.22 02:17:19.670326 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_879_248} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_879_248 to 202410_1_879_248 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:19.672242 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_879_248} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_878_247, 202410_879_879_0] -> 202410_1_879_248 2024.10.22 02:17:19.695413 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:17:19.846947 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_819_196, 202410_820_820_0] 2024.10.22 02:17:19.850177 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_819_196, 202410_820_820_0] 2024.10.22 02:17:19.961251 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_819_196, 202410_820_820_0] 2024.10.22 02:17:20.076874 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_16_16_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:17:20.552106 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1745 entries to flush up to offset 1260102 2024.10.22 02:17:20.833710 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 96998 2024.10.22 02:17:20.967256 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:20.970988 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:21.023964 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1467_1467_0 (state Temporary) 2024.10.22 02:17:21.170359 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1467_1467_0 to 202410_1480_1480_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:21.247156 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 96998 2024.10.22 02:17:21.315149 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:17:21.328073 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:17:21.417470 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 723 2024.10.22 02:17:21.618840 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 18.69 MiB, will set to 1.34 GiB (RSS), difference: 9.29 MiB 2024.10.22 02:17:21.724124 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:21.731369 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:21.797007 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:21.807732 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:21.823258 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_705_705_0 (state Temporary) 2024.10.22 02:17:21.964012 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_833_833_0 (state Temporary) 2024.10.22 02:17:22.009715 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_705_705_0 to 202410_715_715_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:22.034749 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_833_833_0 to 202410_840_840_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:22.170345 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1260102 2024.10.22 02:17:22.727808 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 723 2024.10.22 02:17:22.772630 [ 1223 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:17:22.795354 [ 1223 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:17:23.514920 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_174_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:17:23.658700 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 17.14 MiB, will set to 1.34 GiB (RSS), difference: 4.44 MiB 2024.10.22 02:17:24.702352 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 64 entries to flush up to offset 94779 2024.10.22 02:17:24.948224 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:24.952242 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:24.998385 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_856_856_0 (state Temporary) 2024.10.22 02:17:25.059031 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_856_856_0 to 202410_862_862_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:25.120397 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 94779 2024.10.22 02:17:25.186350 [ 1232 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_857_22 to 202410_862_862_0 2024.10.22 02:17:25.228081 [ 1232 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:25.243849 [ 1232 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:25.256818 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 33 entries to flush up to offset 97031 2024.10.22 02:17:25.274659 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_857_22 to 202410_862_862_0 into Compact with storage Full 2024.10.22 02:17:25.360644 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 15.24 MiB, will set to 1.34 GiB (RSS), difference: -3.34 MiB 2024.10.22 02:17:25.374332 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:25.378932 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:25.409025 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:25.421520 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_857_22, total 10897 rows starting from the beginning of the part 2024.10.22 02:17:25.441636 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1468_1468_0 (state Temporary) 2024.10.22 02:17:25.478995 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_858_858_0, total 124 rows starting from the beginning of the part 2024.10.22 02:17:25.621029 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1468_1468_0 to 202410_1481_1481_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:25.674658 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_859_859_0, total 112 rows starting from the beginning of the part 2024.10.22 02:17:25.722081 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97031 2024.10.22 02:17:25.859459 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_860_860_0, total 103 rows starting from the beginning of the part 2024.10.22 02:17:25.945798 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_861_861_0, total 124 rows starting from the beginning of the part 2024.10.22 02:17:26.118815 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 13284 2024.10.22 02:17:26.296297 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} MergeTreeSequentialSource: Reading 2 marks from part 202410_862_862_0, total 64 rows starting from the beginning of the part 2024.10.22 02:17:26.382108 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:26.389100 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:26.435770 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_871_871_0 (state Temporary) 2024.10.22 02:17:26.572184 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_779_779_0, 202410_780_780_0, 202410_781_781_0, 202410_782_782_0, 202410_783_783_0, 202410_784_784_0] 2024.10.22 02:17:26.575935 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_779_779_0, 202410_780_780_0, 202410_781_781_0, 202410_782_782_0, 202410_783_783_0, 202410_784_784_0] 2024.10.22 02:17:26.625335 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_871_871_0 to 202410_884_884_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:26.671625 [ 1165 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_879_248 to 202410_880_880_0 2024.10.22 02:17:26.683491 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13284 2024.10.22 02:17:26.687456 [ 1165 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:26.693431 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:26.732078 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_779_779_0, 202410_780_780_0, 202410_781_781_0, 202410_782_782_0, 202410_783_783_0, 202410_784_784_0] 2024.10.22 02:17:26.811317 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_880_249} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_879_248 to 202410_880_880_0 into Compact with storage Full 2024.10.22 02:17:26.865315 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_880_249} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:26.873391 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_880_249} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_879_248, total 13302 rows starting from the beginning of the part 2024.10.22 02:17:26.893797 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_880_249} MergeTreeSequentialSource: Reading 2 marks from part 202410_880_880_0, total 19 rows starting from the beginning of the part 2024.10.22 02:17:27.457185 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 218534 bytes for be1b10a0641fbef1c8a9cc32d1d51ae4:0) 2024.10.22 02:17:27.459570 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b3f0929e12b36be7481722bbb192018d, offset: 0, size: 2303665 2024.10.22 02:17:27.462338 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b3f0929e12b36be7481722bbb192018d for removal 2024.10.22 02:17:27.464970 [ 1476 ] {} CacheMetadata: Key b3f0929e12b36be7481722bbb192018d is removed from metadata 2024.10.22 02:17:27.465679 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bc66683192db2e3c9dd835e673f8765d, offset: 0, size: 212 2024.10.22 02:17:27.470479 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bc66683192db2e3c9dd835e673f8765d for removal 2024.10.22 02:17:27.488166 [ 1476 ] {} CacheMetadata: Key bc66683192db2e3c9dd835e673f8765d is removed from metadata 2024.10.22 02:17:27.516031 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_832_156/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:17:27.545116 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 13.99 MiB, will set to 1.34 GiB (RSS), difference: -21.41 MiB 2024.10.22 02:17:27.641529 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_880_249} MergeTask::MergeProjectionsStage: Merge sorted 13321 rows, containing 12 columns (12 merged, 0 gathered) in 0.839388439 sec., 15869.887385951952 rows/sec., 2.48 MiB/sec. 2024.10.22 02:17:27.704740 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_880_249} MergedBlockOutputStream: filled checksums 202410_1_880_249 (state Temporary) 2024.10.22 02:17:27.921226 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_880_249} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_880_249 to 202410_1_880_249 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:27.927758 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_880_249} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_879_248, 202410_880_880_0] -> 202410_1_880_249 2024.10.22 02:17:27.959345 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:17:28.315840 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_820_197, 202410_821_821_0] 2024.10.22 02:17:28.338353 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_820_197, 202410_821_821_0] 2024.10.22 02:17:28.355187 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_820_197, 202410_821_821_0] 2024.10.22 02:17:28.649597 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} MergeTask::MergeProjectionsStage: Merge sorted 11424 rows, containing 13 columns (13 merged, 0 gathered) in 3.393942918 sec., 3365.996504953593 rows/sec., 1.46 MiB/sec. 2024.10.22 02:17:28.910651 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} MergedBlockOutputStream: filled checksums 202410_747_862_23 (state Temporary) 2024.10.22 02:17:29.062679 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 6 file segments (while reserving 155803 bytes for a9623f6538ee01d5d2e5ebd45a9f9161:0) 2024.10.22 02:17:29.066308 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 57bb7d739ed1faf96c36085f030167c9, offset: 0, size: 1084077 2024.10.22 02:17:29.130202 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_862_23 to 202410_747_862_23 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:29.170735 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 57bb7d739ed1faf96c36085f030167c9 for removal 2024.10.22 02:17:29.173455 [ 1476 ] {} CacheMetadata: Key 57bb7d739ed1faf96c36085f030167c9 is removed from metadata 2024.10.22 02:17:29.173452 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6edbb97279062833cacc545197f1e9df, offset: 0, size: 2573 2024.10.22 02:17:29.174277 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1647 entries to flush up to offset 1261749 2024.10.22 02:17:29.178670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6edbb97279062833cacc545197f1e9df for removal 2024.10.22 02:17:29.193243 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6fe529f0c519521a19994b9faaf37736, offset: 0, size: 167 2024.10.22 02:17:29.198095 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6fe529f0c519521a19994b9faaf37736 for removal 2024.10.22 02:17:29.137820 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_862_23} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_857_22, 202410_862_862_0] -> 202410_747_862_23 2024.10.22 02:17:29.283553 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.66 MiB. 2024.10.22 02:17:29.304066 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e292d6d0404d169bfc608e8b4ed102f3, offset: 0, size: 169 2024.10.22 02:17:29.318910 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_832_156} MergeTask::MergeProjectionsStage: Merge sorted 1260324 rows, containing 4 columns (4 merged, 0 gathered) in 41.191676678 sec., 30596.56954127154 rows/sec., 458.80 KiB/sec. 2024.10.22 02:17:29.327317 [ 1476 ] {} CacheMetadata: Key 6fe529f0c519521a19994b9faaf37736 is removed from metadata 2024.10.22 02:17:29.344219 [ 1476 ] {} CacheMetadata: Key 6edbb97279062833cacc545197f1e9df is removed from metadata 2024.10.22 02:17:29.401045 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1377_34, 202410_1378_1378_0, 202410_1379_1379_0, 202410_1380_1380_0, 202410_1381_1381_0, 202410_1382_1382_0] 2024.10.22 02:17:29.405680 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1377_34, 202410_1378_1378_0, 202410_1379_1379_0, 202410_1380_1380_0, 202410_1381_1381_0, 202410_1382_1382_0] 2024.10.22 02:17:29.418928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e292d6d0404d169bfc608e8b4ed102f3 for removal 2024.10.22 02:17:29.424963 [ 1476 ] {} CacheMetadata: Key e292d6d0404d169bfc608e8b4ed102f3 is removed from metadata 2024.10.22 02:17:29.425202 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 94898b65abeb53d0d131fef79c1919ea, offset: 0, size: 52706 2024.10.22 02:17:29.429144 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_832_156} MergedBlockOutputStream: filled checksums 202410_1_832_156 (state Temporary) 2024.10.22 02:17:29.429799 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 94898b65abeb53d0d131fef79c1919ea for removal 2024.10.22 02:17:29.444905 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 63e31e603ceef117e6896cc212ca6641, offset: 0, size: 156 2024.10.22 02:17:29.448228 [ 1476 ] {} CacheMetadata: Key 94898b65abeb53d0d131fef79c1919ea is removed from metadata 2024.10.22 02:17:29.448514 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 63e31e603ceef117e6896cc212ca6641 for removal 2024.10.22 02:17:29.456174 [ 1476 ] {} CacheMetadata: Key 63e31e603ceef117e6896cc212ca6641 is removed from metadata 2024.10.22 02:17:29.547695 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1377_34, 202410_1378_1378_0, 202410_1379_1379_0, 202410_1380_1380_0, 202410_1381_1381_0, 202410_1382_1382_0] 2024.10.22 02:17:29.656999 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 15.58 MiB, will set to 1.34 GiB (RSS), difference: 29.06 MiB 2024.10.22 02:17:29.728475 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 97110 2024.10.22 02:17:29.963796 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 25aca65790494897fca7c52b319b2f00:0) 2024.10.22 02:17:29.965734 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8315cac438bb996f46013fa6c54618ba, offset: 0, size: 310566 2024.10.22 02:17:29.968527 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 8315cac438bb996f46013fa6c54618ba for removal 2024.10.22 02:17:29.972014 [ 1476 ] {} CacheMetadata: Key 8315cac438bb996f46013fa6c54618ba is removed from metadata 2024.10.22 02:17:29.972412 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: da4d8105f3a2c912539e7505dfdab521, offset: 0, size: 48111 2024.10.22 02:17:29.975844 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key da4d8105f3a2c912539e7505dfdab521 for removal 2024.10.22 02:17:29.980222 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 617c2cbcbf7b7711dba300f7f95c0cbe, offset: 0, size: 158 2024.10.22 02:17:29.985338 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 617c2cbcbf7b7711dba300f7f95c0cbe for removal 2024.10.22 02:17:29.990523 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: ca22bc399a2df3b29f6174220bc747c6, offset: 0, size: 167 2024.10.22 02:17:30.001242 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key ca22bc399a2df3b29f6174220bc747c6 for removal 2024.10.22 02:17:30.012515 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 98f73e743800071734f63c58c0e5b37f, offset: 0, size: 33673 2024.10.22 02:17:30.017158 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 98f73e743800071734f63c58c0e5b37f for removal 2024.10.22 02:17:30.118087 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:30.121343 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:30.127543 [ 1476 ] {} CacheMetadata: Key 98f73e743800071734f63c58c0e5b37f is removed from metadata 2024.10.22 02:17:30.128927 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6ebfb7482ce06a90ecb5909be34b5844, offset: 0, size: 151 2024.10.22 02:17:30.132496 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 6ebfb7482ce06a90ecb5909be34b5844 for removal 2024.10.22 02:17:30.140606 [ 1476 ] {} CacheMetadata: Key 6ebfb7482ce06a90ecb5909be34b5844 is removed from metadata 2024.10.22 02:17:30.156471 [ 1476 ] {} CacheMetadata: Key ca22bc399a2df3b29f6174220bc747c6 is removed from metadata 2024.10.22 02:17:30.165562 [ 1476 ] {} CacheMetadata: Key 617c2cbcbf7b7711dba300f7f95c0cbe is removed from metadata 2024.10.22 02:17:30.176485 [ 1476 ] {} CacheMetadata: Key da4d8105f3a2c912539e7505dfdab521 is removed from metadata 2024.10.22 02:17:30.281911 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:30.291564 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:30.431464 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1469_1469_0 (state Temporary) 2024.10.22 02:17:30.538356 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_834_834_0 (state Temporary) 2024.10.22 02:17:30.634700 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_834_834_0 to 202410_841_841_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:30.673301 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1261749 2024.10.22 02:17:30.699518 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 187288 bytes for c535492cebe4fc8d54ede82fa8beb88e:0) 2024.10.22 02:17:30.709024 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9efd8a90b6a33f10b2bee82a9b8c98a3, offset: 0, size: 155 2024.10.22 02:17:30.714387 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9efd8a90b6a33f10b2bee82a9b8c98a3 for removal 2024.10.22 02:17:30.719819 [ 1476 ] {} CacheMetadata: Key 9efd8a90b6a33f10b2bee82a9b8c98a3 is removed from metadata 2024.10.22 02:17:30.719554 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 59d7f6e13394049dc663985c4824fa28, offset: 0, size: 11261387 2024.10.22 02:17:30.743047 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 59d7f6e13394049dc663985c4824fa28 for removal 2024.10.22 02:17:30.746775 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b26836c3e2d35a4b51ad218eb13d6fe2, offset: 0, size: 990 2024.10.22 02:17:30.851854 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b26836c3e2d35a4b51ad218eb13d6fe2 for removal 2024.10.22 02:17:30.987438 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1469_1469_0 to 202410_1482_1482_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:30.998409 [ 1476 ] {} CacheMetadata: Key b26836c3e2d35a4b51ad218eb13d6fe2 is removed from metadata 2024.10.22 02:17:31.019230 [ 1476 ] {} CacheMetadata: Key 59d7f6e13394049dc663985c4824fa28 is removed from metadata 2024.10.22 02:17:31.047166 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97110 2024.10.22 02:17:31.416450 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:17:31.425442 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:17:31.883513 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.34 GiB, peak 2.00 GiB, free memory in arenas 10.43 MiB, will set to 1.33 GiB (RSS), difference: -4.96 MiB 2024.10.22 02:17:32.435025 [ 1724 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rzl/xkzpkxjpvtymueeisceiqmdgowhrf, size 1233357 2024.10.22 02:17:32.619570 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_832_156} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_832_156 to 202410_1_832_156 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:32.630418 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_832_156} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_826_155, 202410_827_832_1] -> 202410_1_832_156 2024.10.22 02:17:32.637220 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 80 entries to flush up to offset 94859 2024.10.22 02:17:32.657661 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:17:32.877751 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:32.882918 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:32.875334 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} MergeTask::MergeProjectionsStage: Merge sorted 6951 rows, containing 679 columns (679 merged, 0 gathered) in 150.912245958 sec., 46.05988040184966 rows/sec., 61.99 KiB/sec. 2024.10.22 02:17:32.952099 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_857_857_0 (state Temporary) 2024.10.22 02:17:33.013124 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 02:17:33.021825 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 02:17:33.048932 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 02:17:33.059249 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 02:17:33.071485 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 02:17:33.099083 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 02:17:33.117751 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column BrowserLanguage 2024.10.22 02:17:33.160473 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xgt/mwqqgowsscrtrrntcjstndulkvijx (LogSeriesLimiter: on interval from 2024-10-22 02:17:15 to 2024-10-22 02:17:33 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:17:33.261445 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_857_857_0 to 202410_863_863_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:33.317354 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 94859 2024.10.22 02:17:33.578307 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 6.05 MiB, will set to 1.33 GiB (RSS), difference: -2.89 MiB 2024.10.22 02:17:34.186944 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 13301 2024.10.22 02:17:34.290837 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} MergedBlockOutputStream: filled checksums 202410_1_174_28 (state Temporary) 2024.10.22 02:17:34.417728 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:34.421466 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:34.568391 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_872_872_0 (state Temporary) 2024.10.22 02:17:34.824100 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_872_872_0 to 202410_885_885_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:34.872257 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13301 2024.10.22 02:17:34.873477 [ 1211 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_880_249 to 202410_881_881_0 2024.10.22 02:17:34.891142 [ 1211 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:34.895729 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.61 GiB. 2024.10.22 02:17:34.963650 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_881_250} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_880_249 to 202410_881_881_0 into Compact with storage Full 2024.10.22 02:17:35.015843 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_881_250} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:35.036126 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_881_250} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_880_249, total 13321 rows starting from the beginning of the part 2024.10.22 02:17:35.060566 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 97186 2024.10.22 02:17:35.251333 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_881_250} MergeTreeSequentialSource: Reading 2 marks from part 202410_881_881_0, total 14 rows starting from the beginning of the part 2024.10.22 02:17:35.296615 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:35.300882 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:35.456416 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1470_1470_0 (state Temporary) 2024.10.22 02:17:35.492689 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 5.16 MiB, will set to 1.32 GiB (RSS), difference: -7.03 MiB 2024.10.22 02:17:35.515359 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1470_1470_0 to 202410_1483_1483_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:35.684054 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97186 2024.10.22 02:17:35.765500 [ 1269 ] {} DatabaseCatalog: Execution took 430 ms. 2024.10.22 02:17:35.659869 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1478_65 to 202410_1483_1483_0 2024.10.22 02:17:35.771161 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:35.773769 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:35.898209 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1478_65 to 202410_1483_1483_0 into Compact with storage Full 2024.10.22 02:17:36.064707 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:36.077179 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1478_65, total 17736 rows starting from the beginning of the part 2024.10.22 02:17:36.112345 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_881_250} MergeTask::MergeProjectionsStage: Merge sorted 13335 rows, containing 12 columns (12 merged, 0 gathered) in 1.158292557 sec., 11512.635490413499 rows/sec., 1.80 MiB/sec. 2024.10.22 02:17:36.351754 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_881_250} MergedBlockOutputStream: filled checksums 202410_1_881_250 (state Temporary) 2024.10.22 02:17:36.443587 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_1479_1479_0, total 82 rows starting from the beginning of the part 2024.10.22 02:17:36.595166 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_1480_1480_0, total 39 rows starting from the beginning of the part 2024.10.22 02:17:36.646258 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_1481_1481_0, total 33 rows starting from the beginning of the part 2024.10.22 02:17:36.726064 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_881_250} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_881_250 to 202410_1_881_250 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:36.731598 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_881_250} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_880_249, 202410_881_881_0] -> 202410_1_881_250 2024.10.22 02:17:36.759115 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:17:36.772686 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_1482_1482_0, total 79 rows starting from the beginning of the part 2024.10.22 02:17:36.812942 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_1483_1483_0, total 76 rows starting from the beginning of the part 2024.10.22 02:17:36.956310 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_174_28/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:17:37.404092 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 3.55 MiB, will set to 1.32 GiB (RSS), difference: -4.66 MiB 2024.10.22 02:17:37.677495 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1263289 2024.10.22 02:17:37.734212 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 724 2024.10.22 02:17:37.802449 [ 1254 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:17:37.933248 [ 1254 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:17:38.017205 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:38.022963 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:38.202101 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_706_706_0 (state Temporary) 2024.10.22 02:17:38.261062 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_821_198, 202410_822_822_0] 2024.10.22 02:17:38.268795 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_821_198, 202410_822_822_0] 2024.10.22 02:17:38.282251 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_821_198, 202410_822_822_0] 2024.10.22 02:17:38.359858 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_706_706_0 to 202410_716_716_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:38.477428 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1483_66/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:17:38.518219 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 724 2024.10.22 02:17:38.615827 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 502791 bytes for 28a0d8c0ffa651cc2011373216cef7d6:0) 2024.10.22 02:17:38.618720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f75e8977e203189353430c20b03213be, offset: 0, size: 435245 2024.10.22 02:17:38.622617 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f75e8977e203189353430c20b03213be for removal 2024.10.22 02:17:38.627112 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 40d516bf2f9b00358bc91b7984892317, offset: 0, size: 183 2024.10.22 02:17:38.627639 [ 1476 ] {} CacheMetadata: Key f75e8977e203189353430c20b03213be is removed from metadata 2024.10.22 02:17:38.631947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 40d516bf2f9b00358bc91b7984892317 for removal 2024.10.22 02:17:38.645631 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 33b61dc468c3e7c31d11aca833bc5392, offset: 0, size: 200434 2024.10.22 02:17:38.648679 [ 1476 ] {} CacheMetadata: Key 40d516bf2f9b00358bc91b7984892317 is removed from metadata 2024.10.22 02:17:38.649982 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 33b61dc468c3e7c31d11aca833bc5392 for removal 2024.10.22 02:17:38.658094 [ 1476 ] {} CacheMetadata: Key 33b61dc468c3e7c31d11aca833bc5392 is removed from metadata 2024.10.22 02:17:38.877545 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:38.882309 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:38.912868 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:38.916460 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:39.165491 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_835_835_0 (state Temporary) 2024.10.22 02:17:39.230719 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_835_835_0 to 202410_842_842_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:39.359613 [ 1250 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_832_156 to 202410_833_838_1 2024.10.22 02:17:39.365350 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1263289 2024.10.22 02:17:39.366146 [ 1250 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:39.371246 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:39.407898 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_838_157} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_832_156 to 202410_833_838_1 into Wide with storage Full 2024.10.22 02:17:39.527042 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_838_157} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:39.530518 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} MergeTask::MergeProjectionsStage: Merge sorted 18045 rows, containing 13 columns (13 merged, 0 gathered) in 3.749907759 sec., 4812.118366562733 rows/sec., 933.86 KiB/sec. 2024.10.22 02:17:39.533473 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_838_157} MergeTreeSequentialSource: Reading 155 marks from part 202410_1_832_156, total 1260324 rows starting from the beginning of the part 2024.10.22 02:17:39.582343 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} MergedBlockOutputStream: filled checksums 202410_1207_1483_66 (state Temporary) 2024.10.22 02:17:39.606248 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_838_157} MergeTreeSequentialSource: Reading 3 marks from part 202410_833_838_1, total 10270 rows starting from the beginning of the part 2024.10.22 02:17:39.703900 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 97252 2024.10.22 02:17:39.779786 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 18 file segments (while reserving 1048576 bytes for e66ba8ba0466cffd23bd372ed6525ce5:0) 2024.10.22 02:17:39.783633 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.33 GiB, peak 2.00 GiB, free memory in arenas 1.85 MiB, will set to 1.32 GiB (RSS), difference: -10.73 MiB 2024.10.22 02:17:39.786261 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d47b69caa5ff0926a56a1215379a26a1, offset: 0, size: 168 2024.10.22 02:17:39.801837 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d47b69caa5ff0926a56a1215379a26a1 for removal 2024.10.22 02:17:39.821553 [ 1476 ] {} CacheMetadata: Key d47b69caa5ff0926a56a1215379a26a1 is removed from metadata 2024.10.22 02:17:39.822843 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fd9806e21d039649ba61a73ded3ee387, offset: 0, size: 39997 2024.10.22 02:17:39.930151 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fd9806e21d039649ba61a73ded3ee387 for removal 2024.10.22 02:17:39.935460 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f96ce92df0a315b086619a81d3767824, offset: 0, size: 156 2024.10.22 02:17:39.938950 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f96ce92df0a315b086619a81d3767824 for removal 2024.10.22 02:17:39.944437 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4ebd26b71809e526e4550198dbf6f93d, offset: 0, size: 64014 2024.10.22 02:17:39.947436 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4ebd26b71809e526e4550198dbf6f93d for removal 2024.10.22 02:17:39.950057 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 047d255f8c32c59c70f42fe8288cc1c4, offset: 0, size: 156 2024.10.22 02:17:39.952264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 047d255f8c32c59c70f42fe8288cc1c4 for removal 2024.10.22 02:17:39.954608 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b2efa66575b226f90b264c8bfee017e5, offset: 0, size: 142676 2024.10.22 02:17:39.954790 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:39.956483 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b2efa66575b226f90b264c8bfee017e5 for removal 2024.10.22 02:17:39.956821 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:39.959540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 32ab0561e542b0da3a270d5f9cc8be70, offset: 0, size: 168 2024.10.22 02:17:39.961266 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 32ab0561e542b0da3a270d5f9cc8be70 for removal 2024.10.22 02:17:39.964390 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fb9b61137c4740441e9bb9b9fde4d3a3, offset: 0, size: 137920 2024.10.22 02:17:39.968176 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fb9b61137c4740441e9bb9b9fde4d3a3 for removal 2024.10.22 02:17:39.975244 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4e18432193ea57b682470571e9402c84, offset: 0, size: 168 2024.10.22 02:17:39.979376 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4e18432193ea57b682470571e9402c84 for removal 2024.10.22 02:17:39.972899 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1483_66 to 202410_1207_1483_66 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:39.983777 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d905d72b2e8b44cfed1343d9093cf732, offset: 0, size: 136739 2024.10.22 02:17:39.994238 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1483_66} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1478_65, 202410_1483_1483_0] -> 202410_1207_1483_66 2024.10.22 02:17:39.995482 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d905d72b2e8b44cfed1343d9093cf732 for removal 2024.10.22 02:17:39.998280 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: effdc3c1923af280f864aea922eab3d5, offset: 0, size: 91051 2024.10.22 02:17:40.003029 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key effdc3c1923af280f864aea922eab3d5 for removal 2024.10.22 02:17:40.012951 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f00be55a48f32a57db7d57e9e962429b, offset: 0, size: 157 2024.10.22 02:17:40.019708 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f00be55a48f32a57db7d57e9e962429b for removal 2024.10.22 02:17:40.027154 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 592a6b7d7e69dfe7417c9569cac8910e, offset: 0, size: 157 2024.10.22 02:17:40.034889 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 592a6b7d7e69dfe7417c9569cac8910e for removal 2024.10.22 02:17:40.039682 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d601d27b25d4a6d31ff433635d37c41b, offset: 0, size: 78679 2024.10.22 02:17:40.044471 [ 1476 ] {} CacheMetadata: Key 592a6b7d7e69dfe7417c9569cac8910e is removed from metadata 2024.10.22 02:17:40.044580 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.85 MiB. 2024.10.22 02:17:40.050606 [ 1476 ] {} CacheMetadata: Key f00be55a48f32a57db7d57e9e962429b is removed from metadata 2024.10.22 02:17:40.053105 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d601d27b25d4a6d31ff433635d37c41b for removal 2024.10.22 02:17:40.056127 [ 1476 ] {} CacheMetadata: Key d601d27b25d4a6d31ff433635d37c41b is removed from metadata 2024.10.22 02:17:40.057010 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 43df7856f67b1d44ebdad6be0c53e49f, offset: 0, size: 156 2024.10.22 02:17:40.060068 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 43df7856f67b1d44ebdad6be0c53e49f for removal 2024.10.22 02:17:40.062137 [ 1476 ] {} CacheMetadata: Key effdc3c1923af280f864aea922eab3d5 is removed from metadata 2024.10.22 02:17:40.062866 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d8da0d175cd3a7a4b11a25b1ff9b4424, offset: 0, size: 67051 2024.10.22 02:17:40.066598 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d8da0d175cd3a7a4b11a25b1ff9b4424 for removal 2024.10.22 02:17:40.069351 [ 1476 ] {} CacheMetadata: Key d8da0d175cd3a7a4b11a25b1ff9b4424 is removed from metadata 2024.10.22 02:17:40.069295 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 659d4a6d0e4f890e75fa972ed2fced38, offset: 0, size: 7208 2024.10.22 02:17:40.072031 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 659d4a6d0e4f890e75fa972ed2fced38 for removal 2024.10.22 02:17:40.080130 [ 1476 ] {} CacheMetadata: Key 659d4a6d0e4f890e75fa972ed2fced38 is removed from metadata 2024.10.22 02:17:40.079918 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d78bc1b69596024003566222c951cfd6, offset: 0, size: 156 2024.10.22 02:17:40.087939 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d78bc1b69596024003566222c951cfd6 for removal 2024.10.22 02:17:40.090748 [ 1476 ] {} CacheMetadata: Key d905d72b2e8b44cfed1343d9093cf732 is removed from metadata 2024.10.22 02:17:40.102875 [ 1476 ] {} CacheMetadata: Key d78bc1b69596024003566222c951cfd6 is removed from metadata 2024.10.22 02:17:40.108492 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1471_1471_0 (state Temporary) 2024.10.22 02:17:40.112886 [ 1476 ] {} CacheMetadata: Key 4e18432193ea57b682470571e9402c84 is removed from metadata 2024.10.22 02:17:40.129514 [ 1476 ] {} CacheMetadata: Key 43df7856f67b1d44ebdad6be0c53e49f is removed from metadata 2024.10.22 02:17:40.142475 [ 1476 ] {} CacheMetadata: Key fb9b61137c4740441e9bb9b9fde4d3a3 is removed from metadata 2024.10.22 02:17:40.355852 [ 1476 ] {} CacheMetadata: Key 32ab0561e542b0da3a270d5f9cc8be70 is removed from metadata 2024.10.22 02:17:40.364138 [ 1476 ] {} CacheMetadata: Key b2efa66575b226f90b264c8bfee017e5 is removed from metadata 2024.10.22 02:17:40.371547 [ 1476 ] {} CacheMetadata: Key 047d255f8c32c59c70f42fe8288cc1c4 is removed from metadata 2024.10.22 02:17:40.384979 [ 1476 ] {} CacheMetadata: Key 4ebd26b71809e526e4550198dbf6f93d is removed from metadata 2024.10.22 02:17:40.397108 [ 1476 ] {} CacheMetadata: Key f96ce92df0a315b086619a81d3767824 is removed from metadata 2024.10.22 02:17:40.414938 [ 1476 ] {} CacheMetadata: Key fd9806e21d039649ba61a73ded3ee387 is removed from metadata 2024.10.22 02:17:40.685223 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1471_1471_0 to 202410_1484_1484_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:40.722475 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 20 file segments (while reserving 898363 bytes for e66ba8ba0466cffd23bd372ed6525ce5:0) 2024.10.22 02:17:40.730194 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ba61397f46a7a2680bead63baff97adc, offset: 0, size: 170 2024.10.22 02:17:40.733703 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ba61397f46a7a2680bead63baff97adc for removal 2024.10.22 02:17:40.738179 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2acdf591f2c63a8d90a35f4f4b7f69c0, offset: 0, size: 94767 2024.10.22 02:17:40.738309 [ 1476 ] {} CacheMetadata: Key ba61397f46a7a2680bead63baff97adc is removed from metadata 2024.10.22 02:17:40.745213 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2acdf591f2c63a8d90a35f4f4b7f69c0 for removal 2024.10.22 02:17:40.747534 [ 1476 ] {} CacheMetadata: Key 2acdf591f2c63a8d90a35f4f4b7f69c0 is removed from metadata 2024.10.22 02:17:40.748604 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0d89b2beabfb6f56cfae71674b070a8c, offset: 0, size: 157 2024.10.22 02:17:40.835905 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 104 entries to flush up to offset 94963 2024.10.22 02:17:40.851984 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97252 2024.10.22 02:17:40.852847 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0d89b2beabfb6f56cfae71674b070a8c for removal 2024.10.22 02:17:40.856684 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f20a6d408b534e146dfbdcd058b62160, offset: 0, size: 168 2024.10.22 02:17:40.856733 [ 1476 ] {} CacheMetadata: Key 0d89b2beabfb6f56cfae71674b070a8c is removed from metadata 2024.10.22 02:17:40.860388 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f20a6d408b534e146dfbdcd058b62160 for removal 2024.10.22 02:17:40.863810 [ 1476 ] {} CacheMetadata: Key f20a6d408b534e146dfbdcd058b62160 is removed from metadata 2024.10.22 02:17:40.866478 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dc20a4c34c6811de332f1ccfb8a1750b, offset: 0, size: 158 2024.10.22 02:17:40.868687 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dc20a4c34c6811de332f1ccfb8a1750b for removal 2024.10.22 02:17:40.871423 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c4819030ae10f661d094edac36ad6437, offset: 0, size: 85316 2024.10.22 02:17:40.871773 [ 1476 ] {} CacheMetadata: Key dc20a4c34c6811de332f1ccfb8a1750b is removed from metadata 2024.10.22 02:17:40.873443 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c4819030ae10f661d094edac36ad6437 for removal 2024.10.22 02:17:40.877398 [ 1476 ] {} CacheMetadata: Key c4819030ae10f661d094edac36ad6437 is removed from metadata 2024.10.22 02:17:40.878176 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dddbcb9dc0ba31cffd299c9098dbb162, offset: 0, size: 234099 2024.10.22 02:17:40.883123 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dddbcb9dc0ba31cffd299c9098dbb162 for removal 2024.10.22 02:17:40.888018 [ 1476 ] {} CacheMetadata: Key dddbcb9dc0ba31cffd299c9098dbb162 is removed from metadata 2024.10.22 02:17:40.888152 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fbf56044cbb5673009e8eaf414f3090b, offset: 0, size: 156 2024.10.22 02:17:40.894376 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fbf56044cbb5673009e8eaf414f3090b for removal 2024.10.22 02:17:40.897757 [ 1476 ] {} CacheMetadata: Key fbf56044cbb5673009e8eaf414f3090b is removed from metadata 2024.10.22 02:17:40.898538 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6cdf353a10744d6e73a610cb361171b9, offset: 0, size: 30298 2024.10.22 02:17:40.903310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6cdf353a10744d6e73a610cb361171b9 for removal 2024.10.22 02:17:40.908922 [ 1476 ] {} CacheMetadata: Key 6cdf353a10744d6e73a610cb361171b9 is removed from metadata 2024.10.22 02:17:40.908584 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2cc50a9a095d7b0f3d50c33026ef44c2, offset: 0, size: 237175 2024.10.22 02:17:40.915973 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2cc50a9a095d7b0f3d50c33026ef44c2 for removal 2024.10.22 02:17:40.919816 [ 1476 ] {} CacheMetadata: Key 2cc50a9a095d7b0f3d50c33026ef44c2 is removed from metadata 2024.10.22 02:17:40.919787 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 05cc9f319b20530b74226c748137acf1, offset: 0, size: 147000 2024.10.22 02:17:40.929122 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 05cc9f319b20530b74226c748137acf1 for removal 2024.10.22 02:17:40.932109 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 11ddbe7ef919c8494f22487134206d19, offset: 0, size: 156 2024.10.22 02:17:40.932280 [ 1476 ] {} CacheMetadata: Key 05cc9f319b20530b74226c748137acf1 is removed from metadata 2024.10.22 02:17:40.937403 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 11ddbe7ef919c8494f22487134206d19 for removal 2024.10.22 02:17:40.942548 [ 1476 ] {} CacheMetadata: Key 11ddbe7ef919c8494f22487134206d19 is removed from metadata 2024.10.22 02:17:40.944085 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7455c91bbe6f29005c35114f41a6ec35, offset: 0, size: 169 2024.10.22 02:17:40.947975 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7455c91bbe6f29005c35114f41a6ec35 for removal 2024.10.22 02:17:40.951383 [ 1476 ] {} CacheMetadata: Key 7455c91bbe6f29005c35114f41a6ec35 is removed from metadata 2024.10.22 02:17:40.956246 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e5ccf9cf3454c9efc2a553a90567594a, offset: 0, size: 32400 2024.10.22 02:17:40.963823 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e5ccf9cf3454c9efc2a553a90567594a for removal 2024.10.22 02:17:40.968645 [ 1476 ] {} CacheMetadata: Key e5ccf9cf3454c9efc2a553a90567594a is removed from metadata 2024.10.22 02:17:40.969328 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a2394405b5058f85f03961c3ce9c5570, offset: 0, size: 156 2024.10.22 02:17:40.976455 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a2394405b5058f85f03961c3ce9c5570 for removal 2024.10.22 02:17:40.983395 [ 1476 ] {} CacheMetadata: Key a2394405b5058f85f03961c3ce9c5570 is removed from metadata 2024.10.22 02:17:40.985273 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1f51e2f459c276178b32aff64c61ed06, offset: 0, size: 156 2024.10.22 02:17:40.994336 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1f51e2f459c276178b32aff64c61ed06 for removal 2024.10.22 02:17:41.006956 [ 1476 ] {} CacheMetadata: Key 1f51e2f459c276178b32aff64c61ed06 is removed from metadata 2024.10.22 02:17:41.008493 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6a2ad5d327d467c21f75f1c8ee3b83fe, offset: 0, size: 195422 2024.10.22 02:17:41.016198 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6a2ad5d327d467c21f75f1c8ee3b83fe for removal 2024.10.22 02:17:41.019897 [ 1476 ] {} CacheMetadata: Key 6a2ad5d327d467c21f75f1c8ee3b83fe is removed from metadata 2024.10.22 02:17:41.022968 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 85efbff7d2f84b1169205ba626f98680, offset: 0, size: 2508 2024.10.22 02:17:41.030983 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 85efbff7d2f84b1169205ba626f98680 for removal 2024.10.22 02:17:41.039860 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2376848d51b984fe056ef6333e92e8ca, offset: 0, size: 156 2024.10.22 02:17:41.044448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2376848d51b984fe056ef6333e92e8ca for removal 2024.10.22 02:17:41.048435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2f6be83060f69ee03f486d8d7977c41d, offset: 0, size: 2070 2024.10.22 02:17:41.052206 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2f6be83060f69ee03f486d8d7977c41d for removal 2024.10.22 02:17:41.097422 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:41.101290 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:41.138518 [ 1476 ] {} CacheMetadata: Key 2376848d51b984fe056ef6333e92e8ca is removed from metadata 2024.10.22 02:17:41.153941 [ 1476 ] {} CacheMetadata: Key 85efbff7d2f84b1169205ba626f98680 is removed from metadata 2024.10.22 02:17:41.154740 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_858_858_0 (state Temporary) 2024.10.22 02:17:41.162852 [ 1476 ] {} CacheMetadata: Key 2f6be83060f69ee03f486d8d7977c41d is removed from metadata 2024.10.22 02:17:41.305235 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_858_858_0 to 202410_864_864_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:41.318006 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:17:41.320490 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:17:41.476137 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_663_135, 202410_664_664_0, 202410_665_665_0, 202410_666_666_0, 202410_667_667_0, 202410_668_668_0] 2024.10.22 02:17:41.477941 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_663_135, 202410_664_664_0, 202410_665_665_0, 202410_666_666_0, 202410_667_667_0, 202410_668_668_0] 2024.10.22 02:17:41.538091 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_663_135, 202410_664_664_0, 202410_665_665_0, 202410_666_666_0, 202410_667_667_0, 202410_668_668_0] 2024.10.22 02:17:41.659011 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 94963 2024.10.22 02:17:41.760279 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 6.67 MiB, will set to 1.32 GiB (RSS), difference: 2.94 MiB 2024.10.22 02:17:41.855631 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for b7a09612667d04d953baa3006633db97:0) 2024.10.22 02:17:41.858842 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 22f1b16a5d923d4e2a79bda3106618f5, offset: 0, size: 1099712 2024.10.22 02:17:41.861123 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 22f1b16a5d923d4e2a79bda3106618f5 for removal 2024.10.22 02:17:41.862968 [ 1476 ] {} CacheMetadata: Key 22f1b16a5d923d4e2a79bda3106618f5 is removed from metadata 2024.10.22 02:17:41.863333 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8508731f30d32abcf2ea5ee264be7214, offset: 0, size: 169 2024.10.22 02:17:41.866809 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8508731f30d32abcf2ea5ee264be7214 for removal 2024.10.22 02:17:41.870339 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 235813f180c77ffa7efbdb9315eef77b, offset: 0, size: 103272 2024.10.22 02:17:41.873382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 235813f180c77ffa7efbdb9315eef77b for removal 2024.10.22 02:17:41.876661 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: efa20fcc3803e76c352648bb91991afc, offset: 0, size: 168 2024.10.22 02:17:41.877242 [ 1476 ] {} CacheMetadata: Key 235813f180c77ffa7efbdb9315eef77b is removed from metadata 2024.10.22 02:17:41.891686 [ 1476 ] {} CacheMetadata: Key 8508731f30d32abcf2ea5ee264be7214 is removed from metadata 2024.10.22 02:17:41.981663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key efa20fcc3803e76c352648bb91991afc for removal 2024.10.22 02:17:41.985071 [ 1476 ] {} CacheMetadata: Key efa20fcc3803e76c352648bb91991afc is removed from metadata 2024.10.22 02:17:42.374205 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 13317 2024.10.22 02:17:42.464980 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 12 file segments (while reserving 1048576 bytes for b7a09612667d04d953baa3006633db97:0) 2024.10.22 02:17:42.466774 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 25cd05f95862a52ef82cd5828eed5659, offset: 0, size: 182 2024.10.22 02:17:42.483212 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 25cd05f95862a52ef82cd5828eed5659 for removal 2024.10.22 02:17:42.490909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 95068a2f5493f325f4d99e75b57a2c4e, offset: 0, size: 157 2024.10.22 02:17:42.490996 [ 1476 ] {} CacheMetadata: Key 25cd05f95862a52ef82cd5828eed5659 is removed from metadata 2024.10.22 02:17:42.494417 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 95068a2f5493f325f4d99e75b57a2c4e for removal 2024.10.22 02:17:42.499762 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: db0f3b9dbe2df0bee153332a92b130b9, offset: 0, size: 156 2024.10.22 02:17:42.508904 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key db0f3b9dbe2df0bee153332a92b130b9 for removal 2024.10.22 02:17:42.512750 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: aa242d9ba5cb383b0ff559efc145bbb4, offset: 0, size: 156 2024.10.22 02:17:42.514400 [ 1476 ] {} CacheMetadata: Key db0f3b9dbe2df0bee153332a92b130b9 is removed from metadata 2024.10.22 02:17:42.518092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key aa242d9ba5cb383b0ff559efc145bbb4 for removal 2024.10.22 02:17:42.533497 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c82c75ad6e3a5ed511747ef7807ddbcb, offset: 0, size: 153478 2024.10.22 02:17:42.534298 [ 1476 ] {} CacheMetadata: Key aa242d9ba5cb383b0ff559efc145bbb4 is removed from metadata 2024.10.22 02:17:42.544635 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c82c75ad6e3a5ed511747ef7807ddbcb for removal 2024.10.22 02:17:42.551365 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 27f49ff6cd7e8932734110b1aeba1355, offset: 0, size: 347125 2024.10.22 02:17:42.555944 [ 1476 ] {} CacheMetadata: Key c82c75ad6e3a5ed511747ef7807ddbcb is removed from metadata 2024.10.22 02:17:42.558591 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 27f49ff6cd7e8932734110b1aeba1355 for removal 2024.10.22 02:17:42.562605 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3009bc7580beb1d084d1080bdc3d5984, offset: 0, size: 155 2024.10.22 02:17:42.565398 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3009bc7580beb1d084d1080bdc3d5984 for removal 2024.10.22 02:17:42.565594 [ 1476 ] {} CacheMetadata: Key 27f49ff6cd7e8932734110b1aeba1355 is removed from metadata 2024.10.22 02:17:42.567551 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 01ee88098c1fc48758e1ebeb3a69b51b, offset: 0, size: 86102 2024.10.22 02:17:42.569405 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 01ee88098c1fc48758e1ebeb3a69b51b for removal 2024.10.22 02:17:42.570184 [ 1476 ] {} CacheMetadata: Key 3009bc7580beb1d084d1080bdc3d5984 is removed from metadata 2024.10.22 02:17:42.571261 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 904fbde0522794c8b96761cdc08df61d, offset: 0, size: 100967 2024.10.22 02:17:42.576126 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 904fbde0522794c8b96761cdc08df61d for removal 2024.10.22 02:17:42.580242 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:42.581346 [ 1476 ] {} CacheMetadata: Key 01ee88098c1fc48758e1ebeb3a69b51b is removed from metadata 2024.10.22 02:17:42.583069 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:42.583366 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7fbb8b1dc8bcd6129bdc084e5b21a294, offset: 0, size: 153 2024.10.22 02:17:42.591309 [ 1476 ] {} CacheMetadata: Key 904fbde0522794c8b96761cdc08df61d is removed from metadata 2024.10.22 02:17:42.581586 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_175_175_0 (state Temporary) 2024.10.22 02:17:42.690725 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7fbb8b1dc8bcd6129bdc084e5b21a294 for removal 2024.10.22 02:17:42.696213 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b8b480a5bac7b99a751ab6877da7095b, offset: 0, size: 53888 2024.10.22 02:17:42.697460 [ 1476 ] {} CacheMetadata: Key 7fbb8b1dc8bcd6129bdc084e5b21a294 is removed from metadata 2024.10.22 02:17:42.701540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b8b480a5bac7b99a751ab6877da7095b for removal 2024.10.22 02:17:42.706997 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 55c99e5b44ba65a1d2ac55660ceb4c8a, offset: 0, size: 62559 2024.10.22 02:17:42.710267 [ 1476 ] {} CacheMetadata: Key b8b480a5bac7b99a751ab6877da7095b is removed from metadata 2024.10.22 02:17:42.712860 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 55c99e5b44ba65a1d2ac55660ceb4c8a for removal 2024.10.22 02:17:42.717583 [ 1476 ] {} CacheMetadata: Key 55c99e5b44ba65a1d2ac55660ceb4c8a is removed from metadata 2024.10.22 02:17:42.733449 [ 1476 ] {} CacheMetadata: Key 95068a2f5493f325f4d99e75b57a2c4e is removed from metadata 2024.10.22 02:17:42.839027 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_873_873_0 (state Temporary) 2024.10.22 02:17:42.879830 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_873_873_0 to 202410_886_886_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:42.947790 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13317 2024.10.22 02:17:43.036576 [ 1263 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_881_250 to 202410_882_882_0 2024.10.22 02:17:43.049836 [ 1263 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:43.063603 [ 1263 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:43.073482 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_882_251} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_881_250 to 202410_882_882_0 into Compact with storage Full 2024.10.22 02:17:43.141094 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_882_251} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:43.151249 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_882_251} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_881_250, total 13335 rows starting from the beginning of the part 2024.10.22 02:17:43.323370 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_882_251} MergeTreeSequentialSource: Reading 2 marks from part 202410_882_882_0, total 23 rows starting from the beginning of the part 2024.10.22 02:17:43.464533 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 10.03 MiB, will set to 1.32 GiB (RSS), difference: -1.67 MiB 2024.10.22 02:17:43.507560 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 874103 bytes for 62bd81f01b1ab9e30375dd838f08314a:0) 2024.10.22 02:17:43.509655 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fa29c1a761482c48491e11769f75bfbb, offset: 0, size: 1032498 2024.10.22 02:17:43.514278 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fa29c1a761482c48491e11769f75bfbb for removal 2024.10.22 02:17:43.518071 [ 1476 ] {} CacheMetadata: Key fa29c1a761482c48491e11769f75bfbb is removed from metadata 2024.10.22 02:17:43.520543 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: df8865ddb86f0ea1d4581648788e5309, offset: 0, size: 168 2024.10.22 02:17:43.525331 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key df8865ddb86f0ea1d4581648788e5309 for removal 2024.10.22 02:17:43.528328 [ 1476 ] {} CacheMetadata: Key df8865ddb86f0ea1d4581648788e5309 is removed from metadata 2024.10.22 02:17:44.054985 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_882_251} MergeTask::MergeProjectionsStage: Merge sorted 13358 rows, containing 12 columns (12 merged, 0 gathered) in 0.985696411 sec., 13551.839948821726 rows/sec., 2.12 MiB/sec. 2024.10.22 02:17:44.072876 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 229250 bytes for 5596c47ff37100e0415dd93fab1557a3:0) 2024.10.22 02:17:44.073502 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_882_251} MergedBlockOutputStream: filled checksums 202410_1_882_251 (state Temporary) 2024.10.22 02:17:44.075540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a7b4c23992e53a1f065cad2af69373fe, offset: 0, size: 98218 2024.10.22 02:17:44.080664 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a7b4c23992e53a1f065cad2af69373fe for removal 2024.10.22 02:17:44.085630 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: aa4ecfdeb49ae82b2123d1dca02b731d, offset: 0, size: 157 2024.10.22 02:17:44.090785 [ 1476 ] {} CacheMetadata: Key a7b4c23992e53a1f065cad2af69373fe is removed from metadata 2024.10.22 02:17:44.092477 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key aa4ecfdeb49ae82b2123d1dca02b731d for removal 2024.10.22 02:17:44.161453 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xqk/hyabxbwxmwfyleknxnziaemrmzdol. 2024.10.22 02:17:44.234732 [ 1476 ] {} CacheMetadata: Key aa4ecfdeb49ae82b2123d1dca02b731d is removed from metadata 2024.10.22 02:17:44.648057 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_882_251} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_882_251 to 202410_1_882_251 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:44.653335 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_882_251} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_881_250, 202410_882_882_0] -> 202410_1_882_251 2024.10.22 02:17:44.756821 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_838_157/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:17:44.774679 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:17:44.860878 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 224 entries to flush up to offset 97476 2024.10.22 02:17:44.865575 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_16_16_0 to 201403_16_16_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:44.902293 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xqk/hyabxbwxmwfyleknxnziaemrmzdol. 2024.10.22 02:17:45.030662 [ 1217 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4) (MergerMutator): Selected 9 parts from 201403_8_8_0 to 201403_16_16_0 2024.10.22 02:17:45.038364 [ 1217 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 485.79 MiB using storage policy from min volume index 0 2024.10.22 02:17:45.069184 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTask::PrepareStage: Merging 9 parts: from 201403_8_8_0 to 201403_16_16_0 into Wide with storage Full 2024.10.22 02:17:45.079837 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_17_17_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:17:45.170777 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_822_199, 202410_823_823_0] 2024.10.22 02:17:45.173156 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_822_199, 202410_823_823_0] 2024.10.22 02:17:45.293825 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_822_199, 202410_823_823_0] 2024.10.22 02:17:45.562991 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 2.00 GiB, free memory in arenas 9.16 MiB, will set to 1.32 GiB (RSS), difference: 179.36 MiB 2024.10.22 02:17:45.937317 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Vertical 2024.10.22 02:17:45.980233 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:45.987490 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:46.021052 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 725 2024.10.22 02:17:46.056626 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part 2024.10.22 02:17:46.085411 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1472_1472_0 (state Temporary) 2024.10.22 02:17:46.092343 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part 2024.10.22 02:17:46.251867 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part 2024.10.22 02:17:46.328057 [ 1232 ] {} DatabaseCatalog: Execution took 458 ms. 2024.10.22 02:17:46.334361 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_174_28 to 202410_1_174_28 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:46.348524 [ 1392 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_174_28} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 7 parts: [202410_1_164_27, 202410_174_174_0] -> 202410_1_174_28 2024.10.22 02:17:46.370386 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1265214 2024.10.22 02:17:46.377495 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1472_1472_0 to 202410_1485_1485_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:46.372272 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part 2024.10.22 02:17:46.457431 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part 2024.10.22 02:17:46.538503 [ 1204 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 253 ms. 2024.10.22 02:17:46.542819 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part 2024.10.22 02:17:46.563448 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97476 2024.10.22 02:17:46.662457 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part 2024.10.22 02:17:46.718191 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part 2024.10.22 02:17:46.834822 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:46.844466 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:46.861601 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part 2024.10.22 02:17:46.904871 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_772_146, 202410_773_778_1] 2024.10.22 02:17:46.908211 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_772_146, 202410_773_778_1] 2024.10.22 02:17:46.957239 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_772_146, 202410_773_778_1] 2024.10.22 02:17:47.028640 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_707_707_0 (state Temporary) 2024.10.22 02:17:47.137193 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 69.88 MiB. 2024.10.22 02:17:47.169587 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_707_707_0 to 202410_717_717_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:47.308936 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wlj/djqluwbxyxonptsoezssgtnorqenx (LogSeriesLimiter: on interval from 2024-10-22 02:17:33 to 2024-10-22 02:17:47 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:17:47.402722 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 725 2024.10.22 02:17:47.381687 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:47.489422 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:47.678523 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1009.02 MiB, peak 2.00 GiB, free memory in arenas 9.96 MiB, will set to 1.07 GiB (RSS), difference: 82.02 MiB 2024.10.22 02:17:47.645679 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_836_836_0 (state Temporary) 2024.10.22 02:17:47.930311 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_836_836_0 to 202410_843_843_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:47.973825 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1265214 2024.10.22 02:17:48.576325 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_175_175_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:17:48.702546 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 206905 bytes for 36daf886e37c21bedee05f8d0c52fc53:0) 2024.10.22 02:17:48.705506 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: be026635971499dff795106d1ff92433, offset: 0, size: 4220766 2024.10.22 02:17:48.712596 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key be026635971499dff795106d1ff92433 for removal 2024.10.22 02:17:48.716035 [ 1476 ] {} CacheMetadata: Key be026635971499dff795106d1ff92433 is removed from metadata 2024.10.22 02:17:48.718754 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 41b0d3a72d8a8cb8c56779ed2329869e, offset: 0, size: 903 2024.10.22 02:17:48.723755 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 41b0d3a72d8a8cb8c56779ed2329869e for removal 2024.10.22 02:17:48.749345 [ 1476 ] {} CacheMetadata: Key 41b0d3a72d8a8cb8c56779ed2329869e is removed from metadata 2024.10.22 02:17:48.775024 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 59 2024.10.22 02:17:49.080672 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:49.091616 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:49.149760 [ 1181 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 260 ms. 2024.10.22 02:17:49.156101 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_44_44_0 (state Temporary) 2024.10.22 02:17:49.192330 [ 1699 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xgt/mwqqgowsscrtrrntcjstndulkvijx, size 206905 2024.10.22 02:17:49.254465 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 104 entries to flush up to offset 95067 2024.10.22 02:17:49.258539 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_44_44_0 to 202410_44_44_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:49.275253 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 02:17:49.305072 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 02:17:49.323577 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 02:17:49.533613 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 59 2024.10.22 02:17:49.550826 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 02:17:49.562509 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 02:17:49.628867 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.19 GiB, peak 2.00 GiB, free memory in arenas 10.81 MiB, will set to 1.11 GiB (RSS), difference: -85.75 MiB 2024.10.22 02:17:49.687633 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 02:17:49.711024 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column BrowserCountry 2024.10.22 02:17:49.741712 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:49.746974 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:49.950066 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_859_859_0 (state Temporary) 2024.10.22 02:17:49.974196 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_859_859_0 to 202410_865_865_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:50.253371 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 95067 2024.10.22 02:17:50.449351 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 13335 2024.10.22 02:17:50.628251 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 97555 2024.10.22 02:17:50.664029 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:50.668626 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:50.929832 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_874_874_0 (state Temporary) 2024.10.22 02:17:51.017631 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:51.024210 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:51.071292 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_874_874_0 to 202410_887_887_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:51.110713 [ 1155 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_882_251 to 202410_883_883_0 2024.10.22 02:17:51.127365 [ 1155 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:51.136287 [ 1155 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:51.145253 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13335 2024.10.22 02:17:51.187151 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1473_1473_0 (state Temporary) 2024.10.22 02:17:51.255851 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_883_252} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_882_251 to 202410_883_883_0 into Compact with storage Full 2024.10.22 02:17:51.288072 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_883_252} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:51.319868 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:17:51.329112 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:17:51.401211 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_883_252} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_882_251, total 13358 rows starting from the beginning of the part 2024.10.22 02:17:51.459338 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1382_35, 202410_1383_1383_0, 202410_1384_1384_0, 202410_1385_1385_0, 202410_1386_1386_0, 202410_1387_1387_0] 2024.10.22 02:17:51.459925 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1473_1473_0 to 202410_1486_1486_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:51.461513 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1382_35, 202410_1383_1383_0, 202410_1384_1384_0, 202410_1385_1385_0, 202410_1386_1386_0, 202410_1387_1387_0] 2024.10.22 02:17:51.503667 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.15 GiB, peak 2.00 GiB, free memory in arenas 9.71 MiB, will set to 1.14 GiB (RSS), difference: -5.61 MiB 2024.10.22 02:17:51.546135 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_883_252} MergeTreeSequentialSource: Reading 2 marks from part 202410_883_883_0, total 23 rows starting from the beginning of the part 2024.10.22 02:17:51.569194 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1382_35, 202410_1383_1383_0, 202410_1384_1384_0, 202410_1385_1385_0, 202410_1386_1386_0, 202410_1387_1387_0] 2024.10.22 02:17:51.767312 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97555 2024.10.22 02:17:52.487446 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_823_200, 202410_824_824_0] 2024.10.22 02:17:52.592310 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_823_200, 202410_824_824_0] 2024.10.22 02:17:52.627078 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_883_252} MergeTask::MergeProjectionsStage: Merge sorted 13381 rows, containing 12 columns (12 merged, 0 gathered) in 1.373641938 sec., 9741.257623134683 rows/sec., 1.52 MiB/sec. 2024.10.22 02:17:52.671513 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_883_252} MergedBlockOutputStream: filled checksums 202410_1_883_252 (state Temporary) 2024.10.22 02:17:52.709009 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_823_200, 202410_824_824_0] 2024.10.22 02:17:52.842866 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_175_175_0 to 202410_178_178_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:52.945584 [ 1212 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:17:53.071854 [ 1212 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:17:53.157574 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_883_252} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_883_252 to 202410_1_883_252 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:53.162880 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_883_252} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_882_251, 202410_883_883_0] -> 202410_1_883_252 2024.10.22 02:17:53.198310 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:17:53.441190 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.22 GiB, peak 2.00 GiB, free memory in arenas 10.41 MiB, will set to 1.18 GiB (RSS), difference: -36.07 MiB 2024.10.22 02:17:54.907108 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 726 2024.10.22 02:17:54.987066 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1266754 2024.10.22 02:17:55.019874 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7058 2024.10.22 02:17:55.281151 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:55.288463 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:55.465720 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_708_708_0 (state Temporary) 2024.10.22 02:17:55.473983 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_802_11, 202410_803_803_0, 202410_804_804_0, 202410_805_805_0, 202410_806_806_0, 202410_807_807_0] 2024.10.22 02:17:55.475136 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_802_11, 202410_803_803_0, 202410_804_804_0, 202410_805_805_0, 202410_806_806_0, 202410_807_807_0] 2024.10.22 02:17:55.549356 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_802_11, 202410_803_803_0, 202410_804_804_0, 202410_805_805_0, 202410_806_806_0, 202410_807_807_0] 2024.10.22 02:17:55.553455 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.26 GiB, peak 2.00 GiB, free memory in arenas 13.50 MiB, will set to 1.22 GiB (RSS), difference: -34.76 MiB 2024.10.22 02:17:55.659848 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_708_708_0 to 202410_718_718_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:55.706294 [ 1270 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_713_145 to 202410_718_718_0 2024.10.22 02:17:55.737321 [ 1270 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:55.748565 [ 1270 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:55.769890 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_713_145 to 202410_718_718_0 into Compact with storage Full 2024.10.22 02:17:55.780711 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 97605 2024.10.22 02:17:55.847688 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 726 2024.10.22 02:17:55.923948 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:55.927125 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:56.080662 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_insert_202410_837_837_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:17:56.157452 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:56.160312 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:56.176467 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:56.199368 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_713_145, total 757 rows starting from the beginning of the part 2024.10.22 02:17:56.251231 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1474_1474_0 (state Temporary) 2024.10.22 02:17:56.277791 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_837_837_0 (state Temporary) 2024.10.22 02:17:56.362568 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_837_837_0 to 202410_844_844_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:56.376785 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_714_714_0, total 1 rows starting from the beginning of the part 2024.10.22 02:17:56.397814 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1266754 2024.10.22 02:17:56.419940 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1474_1474_0 to 202410_1487_1487_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:56.490435 [ 1230 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_839_839_0 to 202410_844_844_0 2024.10.22 02:17:56.512504 [ 1230 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:56.555923 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_715_715_0, total 1 rows starting from the beginning of the part 2024.10.22 02:17:56.571876 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97605 2024.10.22 02:17:56.631487 [ 1230 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:56.656683 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_839_839_0 to 202410_844_844_0 into Compact with storage Full 2024.10.22 02:17:56.640925 [ 1230 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 263 ms. 2024.10.22 02:17:56.677348 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:56.689727 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_839_839_0, total 1353 rows starting from the beginning of the part 2024.10.22 02:17:56.696701 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_716_716_0, total 1 rows starting from the beginning of the part 2024.10.22 02:17:56.696080 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_840_840_0, total 1745 rows starting from the beginning of the part 2024.10.22 02:17:56.708935 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_841_841_0, total 1647 rows starting from the beginning of the part 2024.10.22 02:17:56.723012 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_842_842_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:17:56.747033 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_843_843_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:17:56.756214 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_717_717_0, total 1 rows starting from the beginning of the part 2024.10.22 02:17:56.769017 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_844_844_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:17:57.237170 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_718_718_0, total 1 rows starting from the beginning of the part 2024.10.22 02:17:57.434775 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.32 GiB, peak 2.00 GiB, free memory in arenas 17.23 MiB, will set to 1.27 GiB (RSS), difference: -54.18 MiB 2024.10.22 02:17:57.757763 [ 1256 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 02:17:57.761093 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 112 entries to flush up to offset 95179 2024.10.22 02:17:58.253209 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} MergeTask::MergeProjectionsStage: Merge sorted 9750 rows, containing 4 columns (4 merged, 0 gathered) in 1.605746848 sec., 6071.940924027896 rows/sec., 94.87 KiB/sec. 2024.10.22 02:17:58.333022 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:58.339240 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:58.373799 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_860_860_0 (state Temporary) 2024.10.22 02:17:58.277610 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} MergedBlockOutputStream: filled checksums 202410_839_844_1 (state Temporary) 2024.10.22 02:17:58.441722 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_839_844_1 to 202410_839_844_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:58.445973 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_839_844_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_839_839_0, 202410_844_844_0] -> 202410_839_844_1 2024.10.22 02:17:58.508360 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:17:58.544751 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_860_860_0 to 202410_866_866_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:58.560992 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 4 similar messages) 2024.10.22 02:17:58.608908 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 95179 2024.10.22 02:17:58.650968 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 13352 2024.10.22 02:17:58.661150 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} MergeTask::MergeProjectionsStage: Merge sorted 762 rows, containing 32 columns (32 merged, 0 gathered) in 2.895844043 sec., 263.1357174920901 rows/sec., 23.74 KiB/sec. 2024.10.22 02:17:58.680121 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} MergedBlockOutputStream: filled checksums 202410_1_718_146 (state Temporary) 2024.10.22 02:17:58.767935 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:58.771149 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:58.816362 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_875_875_0 (state Temporary) 2024.10.22 02:17:58.959754 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_875_875_0 to 202410_888_888_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:58.981367 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_883_252 to 202410_884_884_0 2024.10.22 02:17:59.000551 [ 1265 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:17:59.004859 [ 1265 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:17:59.028146 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_884_253} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_883_252 to 202410_884_884_0 into Compact with storage Full 2024.10.22 02:17:59.100591 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13352 2024.10.22 02:17:59.193470 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_884_253} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:17:59.335029 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_884_253} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_883_252, total 13381 rows starting from the beginning of the part 2024.10.22 02:17:59.455515 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.40 GiB, peak 2.00 GiB, free memory in arenas 16.99 MiB, will set to 1.30 GiB (RSS), difference: -106.60 MiB 2024.10.22 02:17:59.513439 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_884_253} MergeTreeSequentialSource: Reading 2 marks from part 202410_884_884_0, total 9 rows starting from the beginning of the part 2024.10.22 02:17:59.647254 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_718_146 to 202410_1_718_146 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:17:59.651893 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_718_146} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_713_145, 202410_718_718_0] -> 202410_1_718_146 2024.10.22 02:17:59.705430 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:18:00.577449 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 97671 2024.10.22 02:18:00.868849 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:00.871935 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:00.940107 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_884_253} MergeTask::MergeProjectionsStage: Merge sorted 13390 rows, containing 12 columns (12 merged, 0 gathered) in 1.921005802 sec., 6970.306901759165 rows/sec., 1.09 MiB/sec. 2024.10.22 02:18:01.002914 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_884_253} MergedBlockOutputStream: filled checksums 202410_1_884_253 (state Temporary) 2024.10.22 02:18:01.158062 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1475_1475_0 (state Temporary) 2024.10.22 02:18:01.232795 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1475_1475_0 to 202410_1488_1488_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:01.251308 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_884_253} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_884_253 to 202410_1_884_253 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:01.254944 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_884_253} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_883_252, 202410_884_884_0] -> 202410_1_884_253 2024.10.22 02:18:01.273442 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1207_1483_66 to 202410_1486_1486_0 2024.10.22 02:18:01.302306 [ 1153 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:01.304456 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97671 2024.10.22 02:18:01.305792 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:18:01.307990 [ 1153 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:01.319690 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:18:01.327398 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:18:01.347696 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1486_67} MergeTask::PrepareStage: Merging 4 parts: from 202410_1207_1483_66 to 202410_1486_1486_0 into Compact with storage Full 2024.10.22 02:18:01.391428 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1486_67} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:01.401717 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1486_67} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1483_66, total 18045 rows starting from the beginning of the part 2024.10.22 02:18:01.544240 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1486_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_1484_1484_0, total 66 rows starting from the beginning of the part 2024.10.22 02:18:01.569309 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1486_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_1485_1485_0, total 224 rows starting from the beginning of the part 2024.10.22 02:18:01.592407 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1486_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_1486_1486_0, total 79 rows starting from the beginning of the part 2024.10.22 02:18:01.695899 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.39 GiB, peak 2.00 GiB, free memory in arenas 16.90 MiB, will set to 1.33 GiB (RSS), difference: -61.24 MiB 2024.10.22 02:18:02.317965 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_824_201, 202410_825_825_0] 2024.10.22 02:18:02.319990 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_824_201, 202410_825_825_0] 2024.10.22 02:18:02.341399 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_824_201, 202410_825_825_0] 2024.10.22 02:18:02.560886 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 7101 2024.10.22 02:18:03.350747 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 727 2024.10.22 02:18:03.413280 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1268294 2024.10.22 02:18:03.490759 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:03.494466 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:03.672128 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.37 GiB, peak 2.00 GiB, free memory in arenas 16.17 MiB, will set to 1.37 GiB (RSS), difference: 5.09 MiB 2024.10.22 02:18:03.768106 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1486_67} MergeTask::MergeProjectionsStage: Merge sorted 18414 rows, containing 13 columns (13 merged, 0 gathered) in 2.43652404 sec., 7557.487509952908 rows/sec., 1.43 MiB/sec. 2024.10.22 02:18:03.851404 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_709_709_0 (state Temporary) 2024.10.22 02:18:03.934449 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_709_709_0 to 202410_719_719_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:04.016333 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1486_67} MergedBlockOutputStream: filled checksums 202410_1207_1486_67 (state Temporary) 2024.10.22 02:18:04.057237 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:04.060098 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:04.104875 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 727 2024.10.22 02:18:04.232102 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_838_838_0 (state Temporary) 2024.10.22 02:18:04.474252 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1486_67} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1486_67 to 202410_1207_1486_67 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:04.478524 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1486_67} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1207_1483_66, 202410_1486_1486_0] -> 202410_1207_1486_67 2024.10.22 02:18:04.534728 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.89 MiB. 2024.10.22 02:18:04.648197 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_838_838_0 to 202410_845_845_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:04.677609 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1268294 2024.10.22 02:18:05.427765 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 97716 2024.10.22 02:18:05.556157 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:05.559944 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:05.567350 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 02:18:05.578611 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 02:18:05.596831 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 02:18:05.619163 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1476_1476_0 (state Temporary) 2024.10.22 02:18:05.634928 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 13.54 MiB, will set to 1.39 GiB (RSS), difference: 8.78 MiB 2024.10.22 02:18:05.651498 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 02:18:05.869192 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 02:18:05.888579 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 02:18:05.908443 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SocialNetwork 2024.10.22 02:18:05.918661 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1476_1476_0 to 202410_1489_1489_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:06.066485 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kir/vsvnkbwmvxskiznevrlbvkemovboz (LogSeriesLimiter: on interval from 2024-10-22 02:17:47 to 2024-10-22 02:18:06 accepted series 1 / 9 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:18:06.095834 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97716 2024.10.22 02:18:06.276688 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 99 entries to flush up to offset 95278 2024.10.22 02:18:06.606492 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 13372 2024.10.22 02:18:06.609590 [ 1251 ] {} DatabaseCatalog: Execution took 247 ms. 2024.10.22 02:18:06.720354 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:06.728667 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:06.827818 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:06.835682 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:06.876437 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_876_876_0 (state Temporary) 2024.10.22 02:18:06.892226 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_861_861_0 (state Temporary) 2024.10.22 02:18:07.038517 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_876_876_0 to 202410_889_889_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:07.048181 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_861_861_0 to 202410_867_867_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:07.059485 [ 1170 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_884_253 to 202410_885_885_0 2024.10.22 02:18:07.066022 [ 1170 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:07.066225 [ 1190 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_862_23 to 202410_867_867_0 2024.10.22 02:18:07.068628 [ 1170 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:07.072942 [ 1190 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:07.073248 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 95278 2024.10.22 02:18:07.075066 [ 1190 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:07.083912 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13372 2024.10.22 02:18:07.096483 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_885_254} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_884_253 to 202410_885_885_0 into Compact with storage Full 2024.10.22 02:18:07.112634 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_862_23 to 202410_867_867_0 into Compact with storage Full 2024.10.22 02:18:07.165249 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:07.172596 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_862_23, total 11424 rows starting from the beginning of the part 2024.10.22 02:18:07.195484 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_885_254} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:07.208332 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_885_254} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_884_253, total 13390 rows starting from the beginning of the part 2024.10.22 02:18:07.266633 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_885_254} MergeTreeSequentialSource: Reading 2 marks from part 202410_885_885_0, total 17 rows starting from the beginning of the part 2024.10.22 02:18:07.327033 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_863_863_0, total 80 rows starting from the beginning of the part 2024.10.22 02:18:07.368009 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_864_864_0, total 104 rows starting from the beginning of the part 2024.10.22 02:18:07.387208 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 2.00 GiB, free memory in arenas 10.16 MiB, will set to 1.42 GiB (RSS), difference: -25.97 MiB 2024.10.22 02:18:07.399550 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_865_865_0, total 104 rows starting from the beginning of the part 2024.10.22 02:18:07.453441 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_866_866_0, total 112 rows starting from the beginning of the part 2024.10.22 02:18:07.490146 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} MergeTreeSequentialSource: Reading 2 marks from part 202410_867_867_0, total 99 rows starting from the beginning of the part 2024.10.22 02:18:07.668021 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_885_254} MergeTask::MergeProjectionsStage: Merge sorted 13407 rows, containing 12 columns (12 merged, 0 gathered) in 0.596054453 sec., 22492.911398482578 rows/sec., 3.51 MiB/sec. 2024.10.22 02:18:07.692131 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_838_157/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:18:07.830528 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_885_254} MergedBlockOutputStream: filled checksums 202410_1_885_254 (state Temporary) 2024.10.22 02:18:08.022703 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_885_254} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_885_254 to 202410_1_885_254 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:08.029596 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_885_254} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_884_253, 202410_885_885_0] -> 202410_1_885_254 2024.10.22 02:18:08.060370 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:18:08.075031 [ 1277 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:18:08.099782 [ 1277 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:18:08.167065 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_825_202, 202410_826_826_0] 2024.10.22 02:18:08.168175 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_825_202, 202410_826_826_0] 2024.10.22 02:18:08.281009 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_825_202, 202410_826_826_0] 2024.10.22 02:18:09.160048 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} MergeTask::MergeProjectionsStage: Merge sorted 11923 rows, containing 13 columns (13 merged, 0 gathered) in 2.067652709 sec., 5766.44227925803 rows/sec., 2.50 MiB/sec. 2024.10.22 02:18:09.419665 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} MergedBlockOutputStream: filled checksums 202410_747_867_24 (state Temporary) 2024.10.22 02:18:09.446598 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 2.00 GiB, free memory in arenas 8.75 MiB, will set to 1.46 GiB (RSS), difference: -38.45 MiB 2024.10.22 02:18:09.792215 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_867_24 to 202410_747_867_24 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:09.796346 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_867_24} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_862_23, 202410_867_867_0] -> 202410_747_867_24 2024.10.22 02:18:10.102917 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 97779 2024.10.22 02:18:10.149102 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.65 MiB. 2024.10.22 02:18:10.385794 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:10.399377 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:10.873507 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1477_1477_0 (state Temporary) 2024.10.22 02:18:11.215935 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1477_1477_0 to 202410_1490_1490_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:11.272202 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97779 2024.10.22 02:18:11.319939 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:18:11.327757 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:18:11.511807 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 12.79 MiB, will set to 1.48 GiB (RSS), difference: -20.46 MiB 2024.10.22 02:18:11.611313 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 728 2024.10.22 02:18:11.696703 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1635 entries to flush up to offset 1269929 2024.10.22 02:18:12.087265 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:12.091879 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:12.250491 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_710_710_0 (state Temporary) 2024.10.22 02:18:12.387762 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_710_710_0 to 202410_720_720_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:12.558315 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 728 2024.10.22 02:18:12.604526 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:12.610644 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:12.886390 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 02:18:13.179708 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_839_839_0 (state Temporary) 2024.10.22 02:18:13.389680 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_839_839_0 to 202410_846_846_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:13.449911 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 12.32 MiB, will set to 1.52 GiB (RSS), difference: -39.36 MiB 2024.10.22 02:18:13.538715 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1269929 2024.10.22 02:18:14.587322 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 13387 2024.10.22 02:18:14.603312 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 95 entries to flush up to offset 95373 2024.10.22 02:18:14.746462 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:14.749390 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:14.843467 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:14.857253 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:14.924310 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_862_862_0 (state Temporary) 2024.10.22 02:18:14.997547 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_877_877_0 (state Temporary) 2024.10.22 02:18:15.155430 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_877_877_0 to 202410_890_890_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:15.172063 [ 1221 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_885_254 to 202410_886_886_0 2024.10.22 02:18:15.181609 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_862_862_0 to 202410_868_868_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:15.187742 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13387 2024.10.22 02:18:15.203576 [ 1221 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:15.212066 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:15.256821 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_886_255} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_885_254 to 202410_886_886_0 into Compact with storage Full 2024.10.22 02:18:15.273050 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 95373 2024.10.22 02:18:15.261501 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1387_36, 202410_1388_1388_0, 202410_1389_1389_0, 202410_1390_1390_0, 202410_1391_1391_0, 202410_1392_1392_0] 2024.10.22 02:18:15.290158 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 39 entries to flush up to offset 97818 2024.10.22 02:18:15.293482 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1387_36, 202410_1388_1388_0, 202410_1389_1389_0, 202410_1390_1390_0, 202410_1391_1391_0, 202410_1392_1392_0] 2024.10.22 02:18:15.327819 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1387_36, 202410_1388_1388_0, 202410_1389_1389_0, 202410_1390_1390_0, 202410_1391_1391_0, 202410_1392_1392_0] 2024.10.22 02:18:15.385669 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_886_255} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:15.414171 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_886_255} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_885_254, total 13407 rows starting from the beginning of the part 2024.10.22 02:18:15.432666 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_785_785_0, 202410_786_786_0, 202410_787_787_0, 202410_788_788_0, 202410_789_789_0, 202410_790_790_0] 2024.10.22 02:18:15.439681 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_785_785_0, 202410_786_786_0, 202410_787_787_0, 202410_788_788_0, 202410_789_789_0, 202410_790_790_0] 2024.10.22 02:18:15.460042 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_886_255} MergeTreeSequentialSource: Reading 2 marks from part 202410_886_886_0, total 16 rows starting from the beginning of the part 2024.10.22 02:18:15.480388 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:15.486556 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:15.487225 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_785_785_0, 202410_786_786_0, 202410_787_787_0, 202410_788_788_0, 202410_789_789_0, 202410_790_790_0] 2024.10.22 02:18:15.550180 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 10.26 MiB, will set to 1.55 GiB (RSS), difference: -45.39 MiB 2024.10.22 02:18:15.598579 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1478_1478_0 (state Temporary) 2024.10.22 02:18:15.871736 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1478_1478_0 to 202410_1491_1491_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:15.932906 [ 1225 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1486_67 to 202410_1487_1487_0 2024.10.22 02:18:15.939873 [ 1225 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:15.944155 [ 1225 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:16.067687 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97818 2024.10.22 02:18:16.171655 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1487_68} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1486_67 to 202410_1487_1487_0 into Compact with storage Full 2024.10.22 02:18:16.285492 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_886_255} MergeTask::MergeProjectionsStage: Merge sorted 13423 rows, containing 12 columns (12 merged, 0 gathered) in 1.036208816 sec., 12953.952709856118 rows/sec., 2.02 MiB/sec. 2024.10.22 02:18:16.368430 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1487_68} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:16.482987 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1487_68} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1486_67, total 18414 rows starting from the beginning of the part 2024.10.22 02:18:16.530857 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1487_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_1487_1487_0, total 50 rows starting from the beginning of the part 2024.10.22 02:18:16.563798 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_886_255} MergedBlockOutputStream: filled checksums 202410_1_886_255 (state Temporary) 2024.10.22 02:18:16.574479 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 240.82 MiB using storage policy from min volume index 0 2024.10.22 02:18:16.957531 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_886_255} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_886_255 to 202410_1_886_255 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:16.966220 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_886_255} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_885_254, 202410_886_886_0] -> 202410_1_886_255 2024.10.22 02:18:16.995497 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:18:17.106193 [ 1216 ] {} DatabaseCatalog: Execution took 495 ms. 2024.10.22 02:18:17.267092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gqs/ycfwsjptikpgbrtglltbmeiueelmo (LogSeriesLimiter: on interval from 2024-10-22 02:18:06 to 2024-10-22 02:18:17 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:18:17.359776 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_826_203, 202410_827_827_0] 2024.10.22 02:18:17.361237 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_826_203, 202410_827_827_0] 2024.10.22 02:18:17.369586 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_826_203, 202410_827_827_0] 2024.10.22 02:18:17.873176 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 12.55 MiB, will set to 1.56 GiB (RSS), difference: -28.85 MiB 2024.10.22 02:18:17.938084 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_838_157} MergeTask::MergeProjectionsStage: Merge sorted 1270594 rows, containing 4 columns (4 merged, 0 gathered) in 38.542280781 sec., 32966.23796654915 rows/sec., 494.50 KiB/sec. 2024.10.22 02:18:17.947252 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1487_68} MergeTask::MergeProjectionsStage: Merge sorted 18464 rows, containing 13 columns (13 merged, 0 gathered) in 1.879267879 sec., 9825.102746834104 rows/sec., 1.85 MiB/sec. 2024.10.22 02:18:18.033739 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1487_68} MergedBlockOutputStream: filled checksums 202410_1207_1487_68 (state Temporary) 2024.10.22 02:18:18.105599 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_838_157} MergedBlockOutputStream: filled checksums 202410_1_838_157 (state Temporary) 2024.10.22 02:18:18.306545 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1487_68} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1487_68 to 202410_1207_1487_68 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:18.313345 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1487_68} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1486_67, 202410_1487_1487_0] -> 202410_1207_1487_68 2024.10.22 02:18:18.461665 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.80 MiB. 2024.10.22 02:18:19.267144 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_838_157/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:18:19.512366 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 10.35 MiB, will set to 1.55 GiB (RSS), difference: -69.79 MiB 2024.10.22 02:18:20.072051 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 97863 2024.10.22 02:18:20.296280 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:20.300248 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:20.356453 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1479_1479_0 (state Temporary) 2024.10.22 02:18:20.394712 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column WatchID 2024.10.22 02:18:20.423629 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column WatchID 2024.10.22 02:18:20.445308 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column WatchID 2024.10.22 02:18:20.464992 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:20.467694 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:20.555056 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column WatchID 2024.10.22 02:18:20.646961 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1830 entries to flush up to offset 1271759 2024.10.22 02:18:20.668810 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column WatchID 2024.10.22 02:18:20.674821 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column WatchID 2024.10.22 02:18:20.585829 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1479_1479_0 to 202410_1492_1492_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:20.690401 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column WatchID 2024.10.22 02:18:20.727442 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1487_68 to 202410_1488_1488_0 2024.10.22 02:18:20.738098 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:20.741248 [ 1262 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:20.757213 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97863 2024.10.22 02:18:20.839359 [ 1241 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 02:18:20.842549 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column WatchID 2024.10.22 02:18:20.845587 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1488_69} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1487_68 to 202410_1488_1488_0 into Compact with storage Full 2024.10.22 02:18:20.857683 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column WatchID 2024.10.22 02:18:20.875110 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1488_69} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:20.886079 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1488_69} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1487_68, total 18464 rows starting from the beginning of the part 2024.10.22 02:18:20.914340 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1488_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_1488_1488_0, total 66 rows starting from the beginning of the part 2024.10.22 02:18:21.049830 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:21.052490 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:21.208612 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_840_840_0 (state Temporary) 2024.10.22 02:18:21.321437 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:18:21.328301 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:18:21.516620 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_840_840_0 to 202410_847_847_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:21.651677 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 2.00 GiB, free memory in arenas 7.02 MiB, will set to 1.55 GiB (RSS), difference: -214.67 MiB 2024.10.22 02:18:21.660422 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1271759 2024.10.22 02:18:21.776063 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_838_157} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_838_157 to 202410_1_838_157 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:21.782525 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_838_157} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_832_156, 202410_833_838_1] -> 202410_1_838_157 2024.10.22 02:18:21.824308 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:18:22.082926 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xxz/qazsdnftjpycnqeewewepqipbpedt (LogSeriesLimiter: on interval from 2024-10-22 02:18:17 to 2024-10-22 02:18:22 accepted series 1 / 213 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:18:22.289090 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1488_69} MergeTask::MergeProjectionsStage: Merge sorted 18530 rows, containing 13 columns (13 merged, 0 gathered) in 1.528761904 sec., 12120.919517628168 rows/sec., 2.28 MiB/sec. 2024.10.22 02:18:22.449164 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1488_69} MergedBlockOutputStream: filled checksums 202410_1207_1488_69 (state Temporary) 2024.10.22 02:18:22.694999 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 13401 2024.10.22 02:18:22.766609 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1488_69} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1488_69 to 202410_1207_1488_69 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:22.786141 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1488_69} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1487_68, 202410_1488_1488_0] -> 202410_1207_1488_69 2024.10.22 02:18:22.811021 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 135 entries to flush up to offset 95508 2024.10.22 02:18:22.825092 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.80 MiB. 2024.10.22 02:18:23.064836 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:23.068502 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:23.083645 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:23.090792 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.59 GiB. 2024.10.22 02:18:23.104002 [ 1217 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:18:23.152898 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_878_878_0 (state Temporary) 2024.10.22 02:18:23.172298 [ 1217 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:18:23.309630 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_878_878_0 to 202410_891_891_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:23.345149 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_863_863_0 (state Temporary) 2024.10.22 02:18:23.371123 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13401 2024.10.22 02:18:23.369148 [ 1219 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_886_255 to 202410_887_887_0 2024.10.22 02:18:23.382005 [ 1219 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:23.391234 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.59 GiB. 2024.10.22 02:18:23.421684 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_887_256} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_886_255 to 202410_887_887_0 into Compact with storage Full 2024.10.22 02:18:23.497536 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_863_863_0 to 202410_869_869_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:23.563143 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_887_256} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:23.589279 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_887_256} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_886_255, total 13423 rows starting from the beginning of the part 2024.10.22 02:18:23.623896 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_887_256} MergeTreeSequentialSource: Reading 2 marks from part 202410_887_887_0, total 18 rows starting from the beginning of the part 2024.10.22 02:18:23.655198 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 95508 2024.10.22 02:18:23.729412 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 2.00 GiB, free memory in arenas 5.96 MiB, will set to 1.56 GiB (RSS), difference: -158.64 MiB 2024.10.22 02:18:23.830930 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_176_176_0 (state Temporary) 2024.10.22 02:18:23.904724 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for ddb09068ae1e2ea27ad31939c93a95d5:0) 2024.10.22 02:18:23.911796 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7afa091e12a1b7cf94c36ecb0c9f55e8, offset: 0, size: 4850 2024.10.22 02:18:23.919056 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 7afa091e12a1b7cf94c36ecb0c9f55e8 for removal 2024.10.22 02:18:23.922495 [ 1476 ] {} CacheMetadata: Key 7afa091e12a1b7cf94c36ecb0c9f55e8 is removed from metadata 2024.10.22 02:18:23.924088 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ecb82652626e6aa44d23fff018712b94, offset: 0, size: 1083151 2024.10.22 02:18:23.935251 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ecb82652626e6aa44d23fff018712b94 for removal 2024.10.22 02:18:23.940796 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e15f672d582ad9b203409f155be6fc7b, offset: 0, size: 169 2024.10.22 02:18:23.944067 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e15f672d582ad9b203409f155be6fc7b for removal 2024.10.22 02:18:23.945163 [ 1476 ] {} CacheMetadata: Key ecb82652626e6aa44d23fff018712b94 is removed from metadata 2024.10.22 02:18:23.948813 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 050ec815b0cd403c1d77fe5ba1dd16eb, offset: 0, size: 156 2024.10.22 02:18:23.952655 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 050ec815b0cd403c1d77fe5ba1dd16eb for removal 2024.10.22 02:18:23.966177 [ 1476 ] {} CacheMetadata: Key 050ec815b0cd403c1d77fe5ba1dd16eb is removed from metadata 2024.10.22 02:18:23.972501 [ 1476 ] {} CacheMetadata: Key e15f672d582ad9b203409f155be6fc7b is removed from metadata 2024.10.22 02:18:24.500943 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_887_256} MergeTask::MergeProjectionsStage: Merge sorted 13441 rows, containing 12 columns (12 merged, 0 gathered) in 1.087722802 sec., 12357.008582780449 rows/sec., 1.93 MiB/sec. 2024.10.22 02:18:24.630069 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_887_256/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:18:24.655916 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_887_256} MergedBlockOutputStream: filled checksums 202410_1_887_256 (state Temporary) 2024.10.22 02:18:24.737423 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_887_256} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_887_256 to 202410_1_887_256 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:24.742146 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_887_256} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_886_255, 202410_887_887_0] -> 202410_1_887_256 2024.10.22 02:18:24.762510 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:18:24.773624 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 97947 2024.10.22 02:18:25.147631 [ 1696 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kir/vsvnkbwmvxskiznevrlbvkemovboz, size 54 2024.10.22 02:18:25.331992 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:25.339285 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:25.415058 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1480_1480_0 (state Temporary) 2024.10.22 02:18:25.443205 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 5.71 MiB, will set to 1.57 GiB (RSS), difference: -84.41 MiB 2024.10.22 02:18:25.513440 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SocialAction 2024.10.22 02:18:25.528042 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SocialAction 2024.10.22 02:18:25.546692 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SocialAction 2024.10.22 02:18:25.566241 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SocialAction 2024.10.22 02:18:25.567957 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1480_1480_0 to 202410_1493_1493_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:25.602412 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SocialAction 2024.10.22 02:18:25.651790 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SocialAction 2024.10.22 02:18:25.663434 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SocialAction 2024.10.22 02:18:25.745655 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1488_69 to 202410_1493_1493_0 2024.10.22 02:18:25.755949 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:25.758062 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 97947 2024.10.22 02:18:25.773084 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:25.948032 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1488_69 to 202410_1493_1493_0 into Compact with storage Full 2024.10.22 02:18:26.072342 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:26.113665 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1488_69, total 18530 rows starting from the beginning of the part 2024.10.22 02:18:26.158127 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 18 file segments (while reserving 2097152 bytes for ddb09068ae1e2ea27ad31939c93a95d5:0) 2024.10.22 02:18:26.159717 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e9fae9ba11facf230ce9eb953d62b94c, offset: 0, size: 230574 2024.10.22 02:18:26.164320 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e9fae9ba11facf230ce9eb953d62b94c for removal 2024.10.22 02:18:26.167971 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: a832c37c8009c2cbdf03dcf88e503220, offset: 0, size: 4 2024.10.22 02:18:26.168560 [ 1476 ] {} CacheMetadata: Key e9fae9ba11facf230ce9eb953d62b94c is removed from metadata 2024.10.22 02:18:26.205120 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_1489_1489_0, total 45 rows starting from the beginning of the part 2024.10.22 02:18:26.270586 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key a832c37c8009c2cbdf03dcf88e503220 for removal 2024.10.22 02:18:26.272476 [ 1476 ] {} CacheMetadata: Key a832c37c8009c2cbdf03dcf88e503220 is removed from metadata 2024.10.22 02:18:26.272824 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 963daaa3e56574d35cb8f7b930bb9f07, offset: 0, size: 156 2024.10.22 02:18:26.276734 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 963daaa3e56574d35cb8f7b930bb9f07 for removal 2024.10.22 02:18:26.282095 [ 1476 ] {} CacheMetadata: Key 963daaa3e56574d35cb8f7b930bb9f07 is removed from metadata 2024.10.22 02:18:26.282614 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: db77264cfd99ac8fe8074fc2a7de6a76, offset: 0, size: 302489 2024.10.22 02:18:26.289521 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key db77264cfd99ac8fe8074fc2a7de6a76 for removal 2024.10.22 02:18:26.298452 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: fa36b249cf3bad0b0f17f8ed62004d88, offset: 0, size: 158 2024.10.22 02:18:26.300557 [ 1476 ] {} CacheMetadata: Key db77264cfd99ac8fe8074fc2a7de6a76 is removed from metadata 2024.10.22 02:18:26.303036 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key fa36b249cf3bad0b0f17f8ed62004d88 for removal 2024.10.22 02:18:26.318144 [ 1476 ] {} CacheMetadata: Key fa36b249cf3bad0b0f17f8ed62004d88 is removed from metadata 2024.10.22 02:18:26.354215 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_1490_1490_0, total 63 rows starting from the beginning of the part 2024.10.22 02:18:26.418820 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b6d11c167179a535dd5f76b449acd4e5, offset: 0, size: 2733 2024.10.22 02:18:26.426069 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_827_204, 202410_828_828_0] 2024.10.22 02:18:26.436480 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b6d11c167179a535dd5f76b449acd4e5 for removal 2024.10.22 02:18:26.447786 [ 1476 ] {} CacheMetadata: Key b6d11c167179a535dd5f76b449acd4e5 is removed from metadata 2024.10.22 02:18:26.448332 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f7683ac8cc4e424f6455f2ec43de267f, offset: 0, size: 47561 2024.10.22 02:18:26.454106 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f7683ac8cc4e424f6455f2ec43de267f for removal 2024.10.22 02:18:26.456321 [ 1476 ] {} CacheMetadata: Key f7683ac8cc4e424f6455f2ec43de267f is removed from metadata 2024.10.22 02:18:26.433108 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_827_204, 202410_828_828_0] 2024.10.22 02:18:26.460703 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4bfb77d3b13cbd2f54df0cffdeb8990f, offset: 0, size: 121188 2024.10.22 02:18:26.470389 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 4bfb77d3b13cbd2f54df0cffdeb8990f for removal 2024.10.22 02:18:26.472703 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_1491_1491_0, total 39 rows starting from the beginning of the part 2024.10.22 02:18:26.474305 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 396d6cea2ca27300bd77c01333a7bb8e, offset: 0, size: 156 2024.10.22 02:18:26.476416 [ 1476 ] {} CacheMetadata: Key 4bfb77d3b13cbd2f54df0cffdeb8990f is removed from metadata 2024.10.22 02:18:26.477660 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 396d6cea2ca27300bd77c01333a7bb8e for removal 2024.10.22 02:18:26.481699 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 62e437c90eab11288cc8788f1effe0f2, offset: 0, size: 157 2024.10.22 02:18:26.482520 [ 1476 ] {} CacheMetadata: Key 396d6cea2ca27300bd77c01333a7bb8e is removed from metadata 2024.10.22 02:18:26.484183 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 62e437c90eab11288cc8788f1effe0f2 for removal 2024.10.22 02:18:26.488989 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6cc96c37ecd2529964e71fd306f766b8, offset: 0, size: 75653 2024.10.22 02:18:26.493615 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6cc96c37ecd2529964e71fd306f766b8 for removal 2024.10.22 02:18:26.497221 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b0caf4f74b56c51bd6a35143a2025322, offset: 0, size: 969018 2024.10.22 02:18:26.502811 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b0caf4f74b56c51bd6a35143a2025322 for removal 2024.10.22 02:18:26.504265 [ 1476 ] {} CacheMetadata: Key 6cc96c37ecd2529964e71fd306f766b8 is removed from metadata 2024.10.22 02:18:26.519605 [ 1476 ] {} CacheMetadata: Key b0caf4f74b56c51bd6a35143a2025322 is removed from metadata 2024.10.22 02:18:26.506755 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b6808df8cae2a2dfd8cca869e71ebbda, offset: 0, size: 90544 2024.10.22 02:18:26.529711 [ 1476 ] {} CacheMetadata: Key 62e437c90eab11288cc8788f1effe0f2 is removed from metadata 2024.10.22 02:18:26.533318 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b6808df8cae2a2dfd8cca869e71ebbda for removal 2024.10.22 02:18:26.537487 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 657f99cc5e68e5c5f71d09f1777ac65b, offset: 0, size: 157 2024.10.22 02:18:26.544738 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 657f99cc5e68e5c5f71d09f1777ac65b for removal 2024.10.22 02:18:26.548900 [ 1476 ] {} CacheMetadata: Key 657f99cc5e68e5c5f71d09f1777ac65b is removed from metadata 2024.10.22 02:18:26.556020 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2964bc77fb9d0c981978e606f201726b, offset: 0, size: 157 2024.10.22 02:18:26.561780 [ 1476 ] {} CacheMetadata: Key b6808df8cae2a2dfd8cca869e71ebbda is removed from metadata 2024.10.22 02:18:26.562101 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 2964bc77fb9d0c981978e606f201726b for removal 2024.10.22 02:18:26.565645 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5fd29699e006415d61b563e3f2ba1676, offset: 0, size: 506 2024.10.22 02:18:26.579535 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_827_204, 202410_828_828_0] 2024.10.22 02:18:26.582673 [ 1476 ] {} CacheMetadata: Key 2964bc77fb9d0c981978e606f201726b is removed from metadata 2024.10.22 02:18:26.592925 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5fd29699e006415d61b563e3f2ba1676 for removal 2024.10.22 02:18:26.604040 [ 1476 ] {} CacheMetadata: Key 5fd29699e006415d61b563e3f2ba1676 is removed from metadata 2024.10.22 02:18:26.604617 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e38c34ad5aa462971464d2f9cad0578f, offset: 0, size: 78249 2024.10.22 02:18:26.608212 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_1492_1492_0, total 45 rows starting from the beginning of the part 2024.10.22 02:18:26.609810 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e38c34ad5aa462971464d2f9cad0578f for removal 2024.10.22 02:18:26.618556 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c0219d25573408feae41836d3e251995, offset: 0, size: 168 2024.10.22 02:18:26.624046 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c0219d25573408feae41836d3e251995 for removal 2024.10.22 02:18:26.633244 [ 1476 ] {} CacheMetadata: Key c0219d25573408feae41836d3e251995 is removed from metadata 2024.10.22 02:18:26.649541 [ 1476 ] {} CacheMetadata: Key e38c34ad5aa462971464d2f9cad0578f is removed from metadata 2024.10.22 02:18:26.661764 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_176_176_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:18:26.808705 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_1493_1493_0, total 84 rows starting from the beginning of the part 2024.10.22 02:18:27.560653 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 729 2024.10.22 02:18:27.608217 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 5.70 MiB, will set to 1.58 GiB (RSS), difference: -16.32 MiB 2024.10.22 02:18:27.767722 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:27.769299 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.59 GiB. 2024.10.22 02:18:27.862102 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_711_711_0 (state Temporary) 2024.10.22 02:18:28.072537 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_711_711_0 to 202410_721_721_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:28.263737 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 729 2024.10.22 02:18:28.764570 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1273299 2024.10.22 02:18:29.234920 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} MergeTask::MergeProjectionsStage: Merge sorted 18806 rows, containing 13 columns (13 merged, 0 gathered) in 3.443773636 sec., 5460.869960617818 rows/sec., 1.03 MiB/sec. 2024.10.22 02:18:29.469350 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 5.12 MiB, will set to 1.59 GiB (RSS), difference: 8.08 MiB 2024.10.22 02:18:29.514292 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} MergedBlockOutputStream: filled checksums 202410_1207_1493_70 (state Temporary) 2024.10.22 02:18:29.586431 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:29.591792 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.59 GiB. 2024.10.22 02:18:29.868702 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 99 entries to flush up to offset 98046 2024.10.22 02:18:29.793794 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1493_70 to 202410_1207_1493_70 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:29.900904 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1493_70} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1488_69, 202410_1493_1493_0] -> 202410_1207_1493_70 2024.10.22 02:18:29.919464 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_841_841_0 (state Temporary) 2024.10.22 02:18:30.000529 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_841_841_0 to 202410_848_848_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:30.060565 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.11 MiB. 2024.10.22 02:18:30.214679 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 31 file segments (while reserving 4194304 bytes for ddb09068ae1e2ea27ad31939c93a95d5:0) 2024.10.22 02:18:30.219662 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5908c79db9b85bc82626abc14c7f10e7, offset: 0, size: 239203 2024.10.22 02:18:30.236558 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5908c79db9b85bc82626abc14c7f10e7 for removal 2024.10.22 02:18:30.244708 [ 1476 ] {} CacheMetadata: Key 5908c79db9b85bc82626abc14c7f10e7 is removed from metadata 2024.10.22 02:18:30.245040 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 312e8aa2d7763371245d0925757ca957, offset: 0, size: 815 2024.10.22 02:18:30.250880 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 312e8aa2d7763371245d0925757ca957 for removal 2024.10.22 02:18:30.257004 [ 1476 ] {} CacheMetadata: Key 312e8aa2d7763371245d0925757ca957 is removed from metadata 2024.10.22 02:18:30.257259 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1273299 2024.10.22 02:18:30.258151 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 58c09aa98619b4eaa2077355a9dd8652, offset: 0, size: 337023 2024.10.22 02:18:30.264499 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 58c09aa98619b4eaa2077355a9dd8652 for removal 2024.10.22 02:18:30.266962 [ 1191 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_838_157 to 202410_839_844_1 2024.10.22 02:18:30.267504 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 43c9795675f51f153db895cf3221fd49, offset: 0, size: 1 2024.10.22 02:18:30.267683 [ 1476 ] {} CacheMetadata: Key 58c09aa98619b4eaa2077355a9dd8652 is removed from metadata 2024.10.22 02:18:30.270972 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 43c9795675f51f153db895cf3221fd49 for removal 2024.10.22 02:18:30.271589 [ 1191 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:30.273525 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 089d88d23c80e0ee2e8b4877d25e1cfd, offset: 0, size: 368319 2024.10.22 02:18:30.275521 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.59 GiB. 2024.10.22 02:18:30.277595 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 089d88d23c80e0ee2e8b4877d25e1cfd for removal 2024.10.22 02:18:30.283294 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: a3540e88b09f16b49ee357432f78eed3, offset: 0, size: 3702138 2024.10.22 02:18:30.292139 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key a3540e88b09f16b49ee357432f78eed3 for removal 2024.10.22 02:18:30.300323 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 354c4127bc96527c1853426d07cd819f, offset: 0, size: 620 2024.10.22 02:18:30.310315 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 354c4127bc96527c1853426d07cd819f for removal 2024.10.22 02:18:30.316744 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 924912b9a3ea72f0bdd543a09cfc5933, offset: 0, size: 3034 2024.10.22 02:18:30.273798 [ 1476 ] {} CacheMetadata: Key 43c9795675f51f153db895cf3221fd49 is removed from metadata 2024.10.22 02:18:30.394399 [ 1476 ] {} CacheMetadata: Key 354c4127bc96527c1853426d07cd819f is removed from metadata 2024.10.22 02:18:30.408890 [ 1476 ] {} CacheMetadata: Key a3540e88b09f16b49ee357432f78eed3 is removed from metadata 2024.10.22 02:18:30.416673 [ 1476 ] {} CacheMetadata: Key 089d88d23c80e0ee2e8b4877d25e1cfd is removed from metadata 2024.10.22 02:18:30.423526 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 924912b9a3ea72f0bdd543a09cfc5933 for removal 2024.10.22 02:18:30.428537 [ 1476 ] {} CacheMetadata: Key 924912b9a3ea72f0bdd543a09cfc5933 is removed from metadata 2024.10.22 02:18:30.429498 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8daa756afa4e9b72b2badce1333d64c9, offset: 0, size: 162 2024.10.22 02:18:30.432480 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8daa756afa4e9b72b2badce1333d64c9 for removal 2024.10.22 02:18:30.436440 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 16a62a86d83ef940fb3873aa9ce52c67, offset: 0, size: 273031 2024.10.22 02:18:30.438524 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:30.440042 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 16a62a86d83ef940fb3873aa9ce52c67 for removal 2024.10.22 02:18:30.441167 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:30.443906 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e639de5dfccff93f0cac2cc7ba77b451, offset: 0, size: 276089 2024.10.22 02:18:30.448939 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e639de5dfccff93f0cac2cc7ba77b451 for removal 2024.10.22 02:18:30.451641 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 839cf2e41e5ec374b8a33c0120d89ebf, offset: 0, size: 523925 2024.10.22 02:18:30.454134 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 839cf2e41e5ec374b8a33c0120d89ebf for removal 2024.10.22 02:18:30.456066 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6bd57dfa36b96f0d47315fa56cb95084, offset: 0, size: 158 2024.10.22 02:18:30.457000 [ 1476 ] {} CacheMetadata: Key 839cf2e41e5ec374b8a33c0120d89ebf is removed from metadata 2024.10.22 02:18:30.458431 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6bd57dfa36b96f0d47315fa56cb95084 for removal 2024.10.22 02:18:30.460885 [ 1476 ] {} CacheMetadata: Key 6bd57dfa36b96f0d47315fa56cb95084 is removed from metadata 2024.10.22 02:18:30.461045 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d0b0a497172393843e01f724a48568db, offset: 0, size: 318388 2024.10.22 02:18:30.464859 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d0b0a497172393843e01f724a48568db for removal 2024.10.22 02:18:30.466465 [ 1476 ] {} CacheMetadata: Key e639de5dfccff93f0cac2cc7ba77b451 is removed from metadata 2024.10.22 02:18:30.468759 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e8b6b4d8936d5b0d8a275d7a2902afce, offset: 0, size: 16456 2024.10.22 02:18:30.471835 [ 1476 ] {} CacheMetadata: Key d0b0a497172393843e01f724a48568db is removed from metadata 2024.10.22 02:18:30.474477 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e8b6b4d8936d5b0d8a275d7a2902afce for removal 2024.10.22 02:18:30.476784 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3e55aeab09b81c1d1b6394d32ea4c0bc, offset: 0, size: 168 2024.10.22 02:18:30.479302 [ 1476 ] {} CacheMetadata: Key e8b6b4d8936d5b0d8a275d7a2902afce is removed from metadata 2024.10.22 02:18:30.481904 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3e55aeab09b81c1d1b6394d32ea4c0bc for removal 2024.10.22 02:18:30.490819 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1481_1481_0 (state Temporary) 2024.10.22 02:18:30.513736 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_844_158} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_838_157 to 202410_839_844_1 into Wide with storage Full 2024.10.22 02:18:30.557534 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_844_158} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:30.565176 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_844_158} MergeTreeSequentialSource: Reading 157 marks from part 202410_1_838_157, total 1270594 rows starting from the beginning of the part 2024.10.22 02:18:30.591421 [ 1476 ] {} CacheMetadata: Key 3e55aeab09b81c1d1b6394d32ea4c0bc is removed from metadata 2024.10.22 02:18:30.592721 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: daf7f1f80f6912aa7557dbe1de417072, offset: 0, size: 10 2024.10.22 02:18:30.598844 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key daf7f1f80f6912aa7557dbe1de417072 for removal 2024.10.22 02:18:30.603891 [ 1476 ] {} CacheMetadata: Key 16a62a86d83ef940fb3873aa9ce52c67 is removed from metadata 2024.10.22 02:18:30.606690 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e775a887aad0ce0f4ed09af818948d3d, offset: 0, size: 158 2024.10.22 02:18:30.618989 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e775a887aad0ce0f4ed09af818948d3d for removal 2024.10.22 02:18:30.631293 [ 1476 ] {} CacheMetadata: Key e775a887aad0ce0f4ed09af818948d3d is removed from metadata 2024.10.22 02:18:30.628452 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ba1b1438d7100130b6227a642f5e2b91, offset: 0, size: 848 2024.10.22 02:18:30.644177 [ 1476 ] {} CacheMetadata: Key daf7f1f80f6912aa7557dbe1de417072 is removed from metadata 2024.10.22 02:18:30.644213 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ba1b1438d7100130b6227a642f5e2b91 for removal 2024.10.22 02:18:30.655113 [ 1476 ] {} CacheMetadata: Key ba1b1438d7100130b6227a642f5e2b91 is removed from metadata 2024.10.22 02:18:30.655156 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0430b5ed5ee83683bd6ba2602b9c190f, offset: 0, size: 7 2024.10.22 02:18:30.658267 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0430b5ed5ee83683bd6ba2602b9c190f for removal 2024.10.22 02:18:30.662769 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: a16c5964fd27e643bd367ac740043308, offset: 0, size: 161 2024.10.22 02:18:30.667850 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1481_1481_0 to 202410_1494_1494_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:30.668379 [ 1476 ] {} CacheMetadata: Key 0430b5ed5ee83683bd6ba2602b9c190f is removed from metadata 2024.10.22 02:18:30.670238 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key a16c5964fd27e643bd367ac740043308 for removal 2024.10.22 02:18:30.675786 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 49a9c33a8995d01f84a976c1a3c2eb96, offset: 0, size: 160 2024.10.22 02:18:30.683541 [ 1476 ] {} CacheMetadata: Key a16c5964fd27e643bd367ac740043308 is removed from metadata 2024.10.22 02:18:30.684252 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 49a9c33a8995d01f84a976c1a3c2eb96 for removal 2024.10.22 02:18:30.694415 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0864c6dfca29bf0594730094ea11c753, offset: 0, size: 524475 2024.10.22 02:18:30.698936 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0864c6dfca29bf0594730094ea11c753 for removal 2024.10.22 02:18:30.700872 [ 1476 ] {} CacheMetadata: Key 49a9c33a8995d01f84a976c1a3c2eb96 is removed from metadata 2024.10.22 02:18:30.703287 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b41c90ce33039fbb557203464d5ad2ef, offset: 0, size: 5128 2024.10.22 02:18:30.715835 [ 1476 ] {} CacheMetadata: Key 0864c6dfca29bf0594730094ea11c753 is removed from metadata 2024.10.22 02:18:30.716998 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b41c90ce33039fbb557203464d5ad2ef for removal 2024.10.22 02:18:30.720834 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9b28d974ae151491c0628cb2add0b69b, offset: 0, size: 205306 2024.10.22 02:18:30.727706 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_844_158} MergeTreeSequentialSource: Reading 3 marks from part 202410_839_844_1, total 9750 rows starting from the beginning of the part 2024.10.22 02:18:30.728130 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9b28d974ae151491c0628cb2add0b69b for removal 2024.10.22 02:18:30.729785 [ 1476 ] {} CacheMetadata: Key b41c90ce33039fbb557203464d5ad2ef is removed from metadata 2024.10.22 02:18:30.739674 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7bd2eff14747c84c37ff303e08fa9e5a, offset: 0, size: 158 2024.10.22 02:18:30.740774 [ 1476 ] {} CacheMetadata: Key 9b28d974ae151491c0628cb2add0b69b is removed from metadata 2024.10.22 02:18:30.743648 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 7bd2eff14747c84c37ff303e08fa9e5a for removal 2024.10.22 02:18:30.747506 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 98046 2024.10.22 02:18:30.749012 [ 1476 ] {} CacheMetadata: Key 7bd2eff14747c84c37ff303e08fa9e5a is removed from metadata 2024.10.22 02:18:30.749107 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 41fd3424e2d1b31d9be42d516cd2c6a1, offset: 0, size: 9726 2024.10.22 02:18:30.754796 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 41fd3424e2d1b31d9be42d516cd2c6a1 for removal 2024.10.22 02:18:30.756106 [ 1476 ] {} CacheMetadata: Key 8daa756afa4e9b72b2badce1333d64c9 is removed from metadata 2024.10.22 02:18:30.757766 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3209c899921364a2d7505c02dceb570c, offset: 0, size: 156 2024.10.22 02:18:30.761718 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3209c899921364a2d7505c02dceb570c for removal 2024.10.22 02:18:30.762674 [ 1476 ] {} CacheMetadata: Key 41fd3424e2d1b31d9be42d516cd2c6a1 is removed from metadata 2024.10.22 02:18:30.768842 [ 1476 ] {} CacheMetadata: Key 3209c899921364a2d7505c02dceb570c is removed from metadata 2024.10.22 02:18:30.764009 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: fcf7842290b9c4a1286c4e2937a9571b, offset: 0, size: 156 2024.10.22 02:18:30.857311 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key fcf7842290b9c4a1286c4e2937a9571b for removal 2024.10.22 02:18:30.859867 [ 1476 ] {} CacheMetadata: Key fcf7842290b9c4a1286c4e2937a9571b is removed from metadata 2024.10.22 02:18:30.860271 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e2261a34e4a8c5496a881bffa297ece6, offset: 0, size: 132376 2024.10.22 02:18:30.863736 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e2261a34e4a8c5496a881bffa297ece6 for removal 2024.10.22 02:18:30.867867 [ 1476 ] {} CacheMetadata: Key e2261a34e4a8c5496a881bffa297ece6 is removed from metadata 2024.10.22 02:18:30.868109 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 13f6914c001e554497ba539544132351, offset: 0, size: 151 2024.10.22 02:18:30.870343 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 13f6914c001e554497ba539544132351 for removal 2024.10.22 02:18:30.871905 [ 1476 ] {} CacheMetadata: Key 13f6914c001e554497ba539544132351 is removed from metadata 2024.10.22 02:18:30.872196 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 13420 2024.10.22 02:18:30.978899 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:30.985798 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:31.138495 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_879_879_0 (state Temporary) 2024.10.22 02:18:31.170720 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 119 entries to flush up to offset 95627 2024.10.22 02:18:31.177788 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_879_879_0 to 202410_892_892_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:31.228666 [ 1193 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_887_256 to 202410_888_888_0 2024.10.22 02:18:31.237225 [ 1193 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:31.241360 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:31.251261 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13420 2024.10.22 02:18:31.270264 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_888_257} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_887_256 to 202410_888_888_0 into Compact with storage Full 2024.10.22 02:18:31.319815 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:31.322570 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:18:31.324751 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:31.331431 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:18:31.347058 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_668_136, 202410_669_669_0, 202410_670_670_0, 202410_671_671_0, 202410_672_672_0, 202410_673_673_0] 2024.10.22 02:18:31.347690 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_888_257} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:31.351648 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_668_136, 202410_669_669_0, 202410_670_670_0, 202410_671_671_0, 202410_672_672_0, 202410_673_673_0] 2024.10.22 02:18:31.358299 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_888_257} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_887_256, total 13441 rows starting from the beginning of the part 2024.10.22 02:18:31.372014 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_668_136, 202410_669_669_0, 202410_670_670_0, 202410_671_671_0, 202410_672_672_0, 202410_673_673_0] 2024.10.22 02:18:31.535696 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_888_257} MergeTreeSequentialSource: Reading 2 marks from part 202410_888_888_0, total 17 rows starting from the beginning of the part 2024.10.22 02:18:31.570449 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_864_864_0 (state Temporary) 2024.10.22 02:18:31.687431 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_864_864_0 to 202410_870_870_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:31.856825 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 95627 2024.10.22 02:18:32.159180 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_176_176_0 to 202410_179_179_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:32.561175 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 6.57 MiB, will set to 1.60 GiB (RSS), difference: -5.44 MiB 2024.10.22 02:18:32.621088 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_888_257} MergeTask::MergeProjectionsStage: Merge sorted 13458 rows, containing 12 columns (12 merged, 0 gathered) in 1.357886124 sec., 9910.993095912952 rows/sec., 1.55 MiB/sec. 2024.10.22 02:18:32.679746 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_888_257} MergedBlockOutputStream: filled checksums 202410_1_888_257 (state Temporary) 2024.10.22 02:18:32.741744 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 6 similar messages) 2024.10.22 02:18:33.503728 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_888_257} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_888_257 to 202410_1_888_257 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:33.537625 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_888_257} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_887_256, 202410_888_888_0] -> 202410_1_888_257 2024.10.22 02:18:33.576912 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:18:34.008708 [ 1207 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 223 ms. 2024.10.22 02:18:34.404984 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 11.67 MiB, will set to 1.61 GiB (RSS), difference: 24.64 MiB 2024.10.22 02:18:34.767628 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 147 entries to flush up to offset 98193 2024.10.22 02:18:34.778929 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7101 2024.10.22 02:18:34.938435 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_844_158/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:18:35.365118 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_828_205, 202410_829_829_0] 2024.10.22 02:18:35.375929 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:35.378619 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_828_205, 202410_829_829_0] 2024.10.22 02:18:35.383910 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:35.481087 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1482_1482_0 (state Temporary) 2024.10.22 02:18:35.496437 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_828_205, 202410_829_829_0] 2024.10.22 02:18:35.767069 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 730 2024.10.22 02:18:35.847183 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1482_1482_0 to 202410_1495_1495_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:35.926372 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 98193 2024.10.22 02:18:36.160166 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:36.202757 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:36.281413 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_712_712_0 (state Temporary) 2024.10.22 02:18:36.453601 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_712_712_0 to 202410_722_722_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:36.521222 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 730 2024.10.22 02:18:36.813673 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 16.98 MiB, will set to 1.62 GiB (RSS), difference: 10.34 MiB 2024.10.22 02:18:37.261088 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1344 entries to flush up to offset 1274643 2024.10.22 02:18:37.582894 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 30 file segments (while reserving 8388608 bytes for ddb09068ae1e2ea27ad31939c93a95d5:0) 2024.10.22 02:18:37.587360 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9120ec8ccfb11987ae653302017384ff, offset: 0, size: 169 2024.10.22 02:18:37.596488 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9120ec8ccfb11987ae653302017384ff for removal 2024.10.22 02:18:37.603143 [ 1476 ] {} CacheMetadata: Key 9120ec8ccfb11987ae653302017384ff is removed from metadata 2024.10.22 02:18:37.604650 [ 1164 ] {} DatabaseCatalog: Execution took 234 ms. 2024.10.22 02:18:37.605128 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e1f90bb65501e8d72a862a404f89e8c3, offset: 0, size: 51112 2024.10.22 02:18:37.608265 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e1f90bb65501e8d72a862a404f89e8c3 for removal 2024.10.22 02:18:37.610165 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: fa4edd67f6c77bcd26b21b95de4e47cd, offset: 0, size: 156 2024.10.22 02:18:37.614356 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key fa4edd67f6c77bcd26b21b95de4e47cd for removal 2024.10.22 02:18:37.616827 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f986ff1b3e09a77b11e21155051666ba, offset: 0, size: 4 2024.10.22 02:18:37.620902 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f986ff1b3e09a77b11e21155051666ba for removal 2024.10.22 02:18:37.626878 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 03c2363c56af2c152bd974b4d2421f21, offset: 0, size: 192 2024.10.22 02:18:37.635086 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 03c2363c56af2c152bd974b4d2421f21 for removal 2024.10.22 02:18:37.643544 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 80ab5ef8a97a2c5b901cfa74f77d2108, offset: 0, size: 428734 2024.10.22 02:18:37.648710 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 80ab5ef8a97a2c5b901cfa74f77d2108 for removal 2024.10.22 02:18:37.653506 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b1fb81f3de2247e5be0a8b3bfd7fa388, offset: 0, size: 167 2024.10.22 02:18:37.656169 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b1fb81f3de2247e5be0a8b3bfd7fa388 for removal 2024.10.22 02:18:37.712251 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_807_12, 202410_808_808_0, 202410_809_809_0, 202410_810_810_0, 202410_811_811_0, 202410_812_812_0] 2024.10.22 02:18:37.715606 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_807_12, 202410_808_808_0, 202410_809_809_0, 202410_810_810_0, 202410_811_811_0, 202410_812_812_0] 2024.10.22 02:18:37.730210 [ 1476 ] {} CacheMetadata: Key b1fb81f3de2247e5be0a8b3bfd7fa388 is removed from metadata 2024.10.22 02:18:37.759946 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3808dc78bc7d439ad81ea899cd5d8f85, offset: 0, size: 985788 2024.10.22 02:18:37.759346 [ 1476 ] {} CacheMetadata: Key 80ab5ef8a97a2c5b901cfa74f77d2108 is removed from metadata 2024.10.22 02:18:37.762947 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3808dc78bc7d439ad81ea899cd5d8f85 for removal 2024.10.22 02:18:37.765117 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c1fa0f4bbe3dfc2d354f7c9012abe29c, offset: 0, size: 160 2024.10.22 02:18:37.765794 [ 1476 ] {} CacheMetadata: Key 3808dc78bc7d439ad81ea899cd5d8f85 is removed from metadata 2024.10.22 02:18:37.770924 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c1fa0f4bbe3dfc2d354f7c9012abe29c for removal 2024.10.22 02:18:37.782876 [ 1476 ] {} CacheMetadata: Key c1fa0f4bbe3dfc2d354f7c9012abe29c is removed from metadata 2024.10.22 02:18:37.798408 [ 1476 ] {} CacheMetadata: Key 03c2363c56af2c152bd974b4d2421f21 is removed from metadata 2024.10.22 02:18:37.811970 [ 1476 ] {} CacheMetadata: Key f986ff1b3e09a77b11e21155051666ba is removed from metadata 2024.10.22 02:18:37.829712 [ 1476 ] {} CacheMetadata: Key fa4edd67f6c77bcd26b21b95de4e47cd is removed from metadata 2024.10.22 02:18:37.842423 [ 1476 ] {} CacheMetadata: Key e1f90bb65501e8d72a862a404f89e8c3 is removed from metadata 2024.10.22 02:18:37.875062 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f1f4bbca4e9eac992f01488e2c58b3c9, offset: 0, size: 1300 2024.10.22 02:18:37.877443 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f1f4bbca4e9eac992f01488e2c58b3c9 for removal 2024.10.22 02:18:37.882784 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ef2de8dfb62bdfd86d3782bbab4ea833, offset: 0, size: 156 2024.10.22 02:18:37.894420 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ef2de8dfb62bdfd86d3782bbab4ea833 for removal 2024.10.22 02:18:37.901418 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 04a43b4eb839b590107d6cb571e3cb6c, offset: 0, size: 159 2024.10.22 02:18:37.905790 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 04a43b4eb839b590107d6cb571e3cb6c for removal 2024.10.22 02:18:37.913782 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 75ccd675d26964e08591bb7c216441a8, offset: 0, size: 363188 2024.10.22 02:18:37.920578 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 75ccd675d26964e08591bb7c216441a8 for removal 2024.10.22 02:18:37.949174 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_807_12, 202410_808_808_0, 202410_809_809_0, 202410_810_810_0, 202410_811_811_0, 202410_812_812_0] 2024.10.22 02:18:37.952800 [ 1476 ] {} CacheMetadata: Key 04a43b4eb839b590107d6cb571e3cb6c is removed from metadata 2024.10.22 02:18:38.035274 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 21e6e37f4b8122debc4e273f12ce02e7, offset: 0, size: 194 2024.10.22 02:18:38.040676 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 21e6e37f4b8122debc4e273f12ce02e7 for removal 2024.10.22 02:18:38.046978 [ 1476 ] {} CacheMetadata: Key 75ccd675d26964e08591bb7c216441a8 is removed from metadata 2024.10.22 02:18:38.055621 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9f3286d85c871032f7223186ee617839, offset: 0, size: 191414 2024.10.22 02:18:38.058794 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9f3286d85c871032f7223186ee617839 for removal 2024.10.22 02:18:38.059211 [ 1476 ] {} CacheMetadata: Key 21e6e37f4b8122debc4e273f12ce02e7 is removed from metadata 2024.10.22 02:18:38.063525 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 41aa9b3196cc7271b6644db060213d23, offset: 0, size: 156 2024.10.22 02:18:38.067398 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 41aa9b3196cc7271b6644db060213d23 for removal 2024.10.22 02:18:38.072535 [ 1476 ] {} CacheMetadata: Key 9f3286d85c871032f7223186ee617839 is removed from metadata 2024.10.22 02:18:38.073246 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b042b728d95a8ac2477aa943e4f44e76, offset: 0, size: 101540 2024.10.22 02:18:38.078219 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b042b728d95a8ac2477aa943e4f44e76 for removal 2024.10.22 02:18:38.176900 [ 1272 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:18:38.189110 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4d713f045ea361ee0bb543567b0b2731, offset: 0, size: 160852 2024.10.22 02:18:38.190913 [ 1476 ] {} CacheMetadata: Key b042b728d95a8ac2477aa943e4f44e76 is removed from metadata 2024.10.22 02:18:38.194301 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 4d713f045ea361ee0bb543567b0b2731 for removal 2024.10.22 02:18:38.201553 [ 1476 ] {} CacheMetadata: Key 41aa9b3196cc7271b6644db060213d23 is removed from metadata 2024.10.22 02:18:38.213025 [ 1272 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:18:38.301970 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: aa7f994c4405f3ac4355b8700cb61b8e, offset: 0, size: 155 2024.10.22 02:18:38.313686 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key aa7f994c4405f3ac4355b8700cb61b8e for removal 2024.10.22 02:18:38.319746 [ 1476 ] {} CacheMetadata: Key aa7f994c4405f3ac4355b8700cb61b8e is removed from metadata 2024.10.22 02:18:38.321630 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: cd19fee3191f22281cf6e0490d377bba, offset: 0, size: 156 2024.10.22 02:18:38.326310 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key cd19fee3191f22281cf6e0490d377bba for removal 2024.10.22 02:18:38.332209 [ 1476 ] {} CacheMetadata: Key 4d713f045ea361ee0bb543567b0b2731 is removed from metadata 2024.10.22 02:18:38.335811 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 23d6d5223f98d57e9fca3de257df6232, offset: 0, size: 1116534 2024.10.22 02:18:38.342680 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 23d6d5223f98d57e9fca3de257df6232 for removal 2024.10.22 02:18:38.343119 [ 1476 ] {} CacheMetadata: Key cd19fee3191f22281cf6e0490d377bba is removed from metadata 2024.10.22 02:18:38.346728 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6ee18762cd1e621da73e1c3a5bb3b89b, offset: 0, size: 2573 2024.10.22 02:18:38.351040 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6ee18762cd1e621da73e1c3a5bb3b89b for removal 2024.10.22 02:18:38.351822 [ 1476 ] {} CacheMetadata: Key 23d6d5223f98d57e9fca3de257df6232 is removed from metadata 2024.10.22 02:18:38.355561 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3e3afcd19b842abdc270a5977be43250, offset: 0, size: 258149 2024.10.22 02:18:38.357298 [ 1476 ] {} CacheMetadata: Key 6ee18762cd1e621da73e1c3a5bb3b89b is removed from metadata 2024.10.22 02:18:38.357726 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3e3afcd19b842abdc270a5977be43250 for removal 2024.10.22 02:18:38.363906 [ 1476 ] {} CacheMetadata: Key 3e3afcd19b842abdc270a5977be43250 is removed from metadata 2024.10.22 02:18:38.375467 [ 1476 ] {} CacheMetadata: Key ef2de8dfb62bdfd86d3782bbab4ea833 is removed from metadata 2024.10.22 02:18:38.388733 [ 1476 ] {} CacheMetadata: Key f1f4bbca4e9eac992f01488e2c58b3c9 is removed from metadata 2024.10.22 02:18:38.463441 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 89b29a6aeade40e4447fe861b2c11629, offset: 0, size: 191678 2024.10.22 02:18:38.465394 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 17.69 MiB, will set to 1.63 GiB (RSS), difference: 4.54 MiB 2024.10.22 02:18:38.465937 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:38.466685 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 89b29a6aeade40e4447fe861b2c11629 for removal 2024.10.22 02:18:38.472476 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.60 GiB. 2024.10.22 02:18:38.475542 [ 1476 ] {} CacheMetadata: Key 89b29a6aeade40e4447fe861b2c11629 is removed from metadata 2024.10.22 02:18:38.476581 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 524a38ec5a187656a74c766ead247c63, offset: 0, size: 155 2024.10.22 02:18:38.485530 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 524a38ec5a187656a74c766ead247c63 for removal 2024.10.22 02:18:38.497283 [ 1476 ] {} CacheMetadata: Key 524a38ec5a187656a74c766ead247c63 is removed from metadata 2024.10.22 02:18:38.499733 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9296ddd46e6e1d1d4a2b18f88960a010, offset: 0, size: 155 2024.10.22 02:18:38.503998 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9296ddd46e6e1d1d4a2b18f88960a010 for removal 2024.10.22 02:18:38.514686 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2287b405087b7fbb95ebd351ab7fc596, offset: 0, size: 159 2024.10.22 02:18:38.516425 [ 1476 ] {} CacheMetadata: Key 9296ddd46e6e1d1d4a2b18f88960a010 is removed from metadata 2024.10.22 02:18:38.525238 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 2287b405087b7fbb95ebd351ab7fc596 for removal 2024.10.22 02:18:38.537458 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 70fcaa569b011358f4b7d44feb4272a4, offset: 0, size: 393925 2024.10.22 02:18:38.555913 [ 1476 ] {} CacheMetadata: Key 2287b405087b7fbb95ebd351ab7fc596 is removed from metadata 2024.10.22 02:18:38.556843 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 70fcaa569b011358f4b7d44feb4272a4 for removal 2024.10.22 02:18:38.561629 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 65fec5bfbe23fc64d63e3b2c23bf8149, offset: 0, size: 1056992 2024.10.22 02:18:38.563936 [ 1476 ] {} CacheMetadata: Key 70fcaa569b011358f4b7d44feb4272a4 is removed from metadata 2024.10.22 02:18:38.565052 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 65fec5bfbe23fc64d63e3b2c23bf8149 for removal 2024.10.22 02:18:38.570321 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 297abc494f00f4c86e6da42b2996421b, offset: 0, size: 452250 2024.10.22 02:18:38.573424 [ 1476 ] {} CacheMetadata: Key 65fec5bfbe23fc64d63e3b2c23bf8149 is removed from metadata 2024.10.22 02:18:38.574841 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 297abc494f00f4c86e6da42b2996421b for removal 2024.10.22 02:18:38.584033 [ 1476 ] {} CacheMetadata: Key 297abc494f00f4c86e6da42b2996421b is removed from metadata 2024.10.22 02:18:38.713572 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_842_842_0 (state Temporary) 2024.10.22 02:18:38.796341 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 13428 2024.10.22 02:18:38.974365 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_842_842_0 to 202410_849_849_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:39.068619 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:39.073389 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.59 GiB. 2024.10.22 02:18:39.147530 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1274643 2024.10.22 02:18:39.251647 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_880_880_0 (state Temporary) 2024.10.22 02:18:39.354652 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_880_880_0 to 202410_893_893_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:39.390510 [ 1260 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_888_257 to 202410_889_889_0 2024.10.22 02:18:39.392897 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 66 entries to flush up to offset 95693 2024.10.22 02:18:39.401510 [ 1260 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:39.405724 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.59 GiB. 2024.10.22 02:18:39.413204 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13428 2024.10.22 02:18:39.457114 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_889_258} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_888_257 to 202410_889_889_0 into Compact with storage Full 2024.10.22 02:18:39.648210 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_889_258} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:39.653113 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:39.656115 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.59 GiB. 2024.10.22 02:18:39.659264 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_889_258} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_888_257, total 13458 rows starting from the beginning of the part 2024.10.22 02:18:39.683111 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_889_258} MergeTreeSequentialSource: Reading 2 marks from part 202410_889_889_0, total 20 rows starting from the beginning of the part 2024.10.22 02:18:39.821397 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_865_865_0 (state Temporary) 2024.10.22 02:18:39.947245 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 139 entries to flush up to offset 98332 2024.10.22 02:18:40.072587 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_865_865_0 to 202410_871_871_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:40.159533 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_778_147, 202410_779_784_1] 2024.10.22 02:18:40.164550 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_778_147, 202410_779_784_1] 2024.10.22 02:18:40.177233 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_778_147, 202410_779_784_1] 2024.10.22 02:18:40.317058 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 95693 2024.10.22 02:18:40.604746 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:40.618684 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.59 GiB. 2024.10.22 02:18:40.704453 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 17.38 MiB, will set to 1.62 GiB (RSS), difference: 89.87 MiB 2024.10.22 02:18:40.830092 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1483_1483_0 (state Temporary) 2024.10.22 02:18:40.947507 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_889_258} MergeTask::MergeProjectionsStage: Merge sorted 13478 rows, containing 12 columns (12 merged, 0 gathered) in 1.516294276 sec., 8888.77588824981 rows/sec., 1.39 MiB/sec. 2024.10.22 02:18:40.950212 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1483_1483_0 to 202410_1496_1496_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:41.122041 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_889_258} MergedBlockOutputStream: filled checksums 202410_1_889_258 (state Temporary) 2024.10.22 02:18:41.135036 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 98332 2024.10.22 02:18:41.276670 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1392_37, 202410_1393_1393_0, 202410_1394_1394_0, 202410_1395_1395_0, 202410_1396_1396_0, 202410_1397_1397_0] 2024.10.22 02:18:41.284594 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1392_37, 202410_1393_1393_0, 202410_1394_1394_0, 202410_1395_1395_0, 202410_1396_1396_0, 202410_1397_1397_0] 2024.10.22 02:18:41.323772 [ 1262 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 328 ms. 2024.10.22 02:18:41.329114 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:18:41.339287 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:18:41.461144 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1392_37, 202410_1393_1393_0, 202410_1394_1394_0, 202410_1395_1395_0, 202410_1396_1396_0, 202410_1397_1397_0] 2024.10.22 02:18:41.484481 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_829_206, 202410_830_830_0] 2024.10.22 02:18:41.489476 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_829_206, 202410_830_830_0] 2024.10.22 02:18:41.500595 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_829_206, 202410_830_830_0] 2024.10.22 02:18:41.731842 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_889_258} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_889_258 to 202410_1_889_258 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:41.746512 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_889_258} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_888_257, 202410_889_889_0] -> 202410_1_889_258 2024.10.22 02:18:41.806596 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:18:42.416110 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 7141 2024.10.22 02:18:42.600392 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 17.80 MiB, will set to 1.62 GiB (RSS), difference: 116.88 MiB 2024.10.22 02:18:42.828708 [ 1738 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xxz/qazsdnftjpycnqeewewepqipbpedt, size 195644 2024.10.22 02:18:44.025379 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 731 2024.10.22 02:18:44.145756 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_18_18_0 (state Temporary) 2024.10.22 02:18:44.157970 [ 1690 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gqs/ycfwsjptikpgbrtglltbmeiueelmo, size 356 2024.10.22 02:18:44.174089 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/seo/fswwgxnjyiervjouytcivembbzasm (LogSeriesLimiter: on interval from 2024-10-22 02:18:22 to 2024-10-22 02:18:44 accepted series 1 / 148 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:18:44.242721 [ 1685 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/seo/fswwgxnjyiervjouytcivembbzasm, size 4 2024.10.22 02:18:44.479385 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 2.00 GiB, free memory in arenas 16.80 MiB, will set to 1.60 GiB (RSS), difference: 127.33 MiB 2024.10.22 02:18:44.493596 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:44.506782 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.56 GiB. 2024.10.22 02:18:44.531618 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column HTTPError 2024.10.22 02:18:44.549641 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column HTTPError 2024.10.22 02:18:44.563445 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column HTTPError 2024.10.22 02:18:44.571422 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_713_713_0 (state Temporary) 2024.10.22 02:18:44.599357 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column HTTPError 2024.10.22 02:18:44.658894 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column HTTPError 2024.10.22 02:18:44.732124 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column HTTPError 2024.10.22 02:18:44.759960 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column HTTPError 2024.10.22 02:18:44.802336 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_713_713_0 to 202410_723_723_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:44.845427 [ 1204 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_718_146 to 202410_723_723_0 2024.10.22 02:18:44.954975 [ 1204 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:44.959509 [ 1204 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.56 GiB. 2024.10.22 02:18:44.980921 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_718_146 to 202410_723_723_0 into Compact with storage Full 2024.10.22 02:18:45.092920 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 731 2024.10.22 02:18:45.108410 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:45.151812 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 98384 2024.10.22 02:18:45.153671 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_718_146, total 762 rows starting from the beginning of the part 2024.10.22 02:18:45.211838 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_844_158/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:18:45.299624 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_719_719_0, total 1 rows starting from the beginning of the part 2024.10.22 02:18:45.319048 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:45.323660 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.55 GiB. 2024.10.22 02:18:45.384594 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1484_1484_0 (state Temporary) 2024.10.22 02:18:45.633774 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_720_720_0, total 1 rows starting from the beginning of the part 2024.10.22 02:18:45.771400 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1484_1484_0 to 202410_1497_1497_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:45.799349 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_721_721_0, total 1 rows starting from the beginning of the part 2024.10.22 02:18:45.868624 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 98384 2024.10.22 02:18:46.152864 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1736 entries to flush up to offset 1276379 2024.10.22 02:18:46.470954 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_722_722_0, total 1 rows starting from the beginning of the part 2024.10.22 02:18:46.571067 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 12.79 MiB, will set to 1.60 GiB (RSS), difference: -8.64 MiB 2024.10.22 02:18:46.657327 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_723_723_0, total 1 rows starting from the beginning of the part 2024.10.22 02:18:46.857595 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.166055752 sec. 2024.10.22 02:18:46.918364 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 13437 2024.10.22 02:18:47.024367 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:47.132569 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.55 GiB. 2024.10.22 02:18:47.176205 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_881_881_0 (state Temporary) 2024.10.22 02:18:47.298105 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:47.300837 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.55 GiB. 2024.10.22 02:18:47.575224 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_843_843_0 (state Temporary) 2024.10.22 02:18:47.602189 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_881_881_0 to 202410_894_894_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:47.665133 [ 1181 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 5 parts from 202410_175_175_0 to 202410_179_179_0 2024.10.22 02:18:47.674884 [ 1181 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:47.675132 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13437 2024.10.22 02:18:47.677745 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.55 GiB. 2024.10.22 02:18:47.766704 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_843_843_0 to 202410_850_850_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:47.820405 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1276379 2024.10.22 02:18:47.833735 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 105 entries to flush up to offset 95798 2024.10.22 02:18:47.741451 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_175_179_1} MergeTask::PrepareStage: Merging 5 parts: from 202410_175_175_0 to 202410_179_179_0 into Compact with storage Full 2024.10.22 02:18:47.852003 [ 1172 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_889_258 to 202410_890_890_0 2024.10.22 02:18:47.867123 [ 1172 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:47.871892 [ 1172 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.55 GiB. 2024.10.22 02:18:47.875568 [ 1172 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 258 ms. 2024.10.22 02:18:47.900678 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_890_259} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_889_258 to 202410_890_890_0 into Compact with storage Full 2024.10.22 02:18:47.924593 [ 1178 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_845_845_0 to 202410_850_850_0 2024.10.22 02:18:47.960658 [ 1178 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:47.966930 [ 1178 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.55 GiB. 2024.10.22 02:18:48.019032 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:48.018970 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_845_845_0 to 202410_850_850_0 into Compact with storage Full 2024.10.22 02:18:48.075324 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:48.090211 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_845_845_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:18:48.094488 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_890_259} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:48.108283 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_890_259} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_889_258, total 13478 rows starting from the beginning of the part 2024.10.22 02:18:48.108978 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.55 GiB. 2024.10.22 02:18:48.143969 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_846_846_0, total 1635 rows starting from the beginning of the part 2024.10.22 02:18:48.195526 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_890_259} MergeTreeSequentialSource: Reading 2 marks from part 202410_890_890_0, total 15 rows starting from the beginning of the part 2024.10.22 02:18:48.283302 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_847_847_0, total 1830 rows starting from the beginning of the part 2024.10.22 02:18:48.366756 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_866_866_0 (state Temporary) 2024.10.22 02:18:48.455400 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_848_848_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:18:48.497833 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_849_849_0, total 1344 rows starting from the beginning of the part 2024.10.22 02:18:48.506024 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:18:48.517780 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_866_866_0 to 202410_872_872_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:48.555712 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_867_24 to 202410_872_872_0 2024.10.22 02:18:48.561527 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:48.561519 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_850_850_0, total 1736 rows starting from the beginning of the part 2024.10.22 02:18:48.563267 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.55 GiB. 2024.10.22 02:18:48.575871 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 95798 2024.10.22 02:18:48.617608 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_867_24 to 202410_872_872_0 into Compact with storage Full 2024.10.22 02:18:48.675322 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 100 file segments (while reserving 10076676 bytes for ddb09068ae1e2ea27ad31939c93a95d5:0) 2024.10.22 02:18:48.677817 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c695c49f80bc8c6c5742594be3f15c8e, offset: 0, size: 429869 2024.10.22 02:18:48.785552 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c695c49f80bc8c6c5742594be3f15c8e for removal 2024.10.22 02:18:48.790775 [ 1476 ] {} CacheMetadata: Key c695c49f80bc8c6c5742594be3f15c8e is removed from metadata 2024.10.22 02:18:48.790953 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b7150e9f87f8f51ef8434b56ff0c89db, offset: 0, size: 160 2024.10.22 02:18:48.794850 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b7150e9f87f8f51ef8434b56ff0c89db for removal 2024.10.22 02:18:48.800333 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9f561636c8d9034bce5dade0b60cc9b8, offset: 0, size: 219189 2024.10.22 02:18:48.807734 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9f561636c8d9034bce5dade0b60cc9b8 for removal 2024.10.22 02:18:48.813743 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6ff2e6c281b04b2a4366e8561d1e0696, offset: 0, size: 160 2024.10.22 02:18:48.817688 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6ff2e6c281b04b2a4366e8561d1e0696 for removal 2024.10.22 02:18:48.821521 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1dbc0c1056b7f7c9a086f20c71abebc0, offset: 0, size: 452454 2024.10.22 02:18:48.918689 [ 1476 ] {} CacheMetadata: Key 6ff2e6c281b04b2a4366e8561d1e0696 is removed from metadata 2024.10.22 02:18:48.918530 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:48.928461 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 1dbc0c1056b7f7c9a086f20c71abebc0 for removal 2024.10.22 02:18:48.933629 [ 1476 ] {} CacheMetadata: Key 1dbc0c1056b7f7c9a086f20c71abebc0 is removed from metadata 2024.10.22 02:18:48.934909 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1c9b0ff07adeb16666b0d3ecb8ed038f, offset: 0, size: 160 2024.10.22 02:18:48.947461 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 1c9b0ff07adeb16666b0d3ecb8ed038f for removal 2024.10.22 02:18:48.951307 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 10f72d686e5157bcd8a5785ffc2e7fc7, offset: 0, size: 156 2024.10.22 02:18:48.953188 [ 1476 ] {} CacheMetadata: Key 1c9b0ff07adeb16666b0d3ecb8ed038f is removed from metadata 2024.10.22 02:18:48.955309 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_867_24, total 11923 rows starting from the beginning of the part 2024.10.22 02:18:48.956951 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 10f72d686e5157bcd8a5785ffc2e7fc7 for removal 2024.10.22 02:18:48.964015 [ 1476 ] {} CacheMetadata: Key 10f72d686e5157bcd8a5785ffc2e7fc7 is removed from metadata 2024.10.22 02:18:48.972010 [ 1476 ] {} CacheMetadata: Key 9f561636c8d9034bce5dade0b60cc9b8 is removed from metadata 2024.10.22 02:18:48.963820 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5dbf85b8516cd59fca739a6af676accf, offset: 0, size: 203276 2024.10.22 02:18:48.990496 [ 1476 ] {} CacheMetadata: Key b7150e9f87f8f51ef8434b56ff0c89db is removed from metadata 2024.10.22 02:18:48.992023 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5dbf85b8516cd59fca739a6af676accf for removal 2024.10.22 02:18:49.015265 [ 1476 ] {} CacheMetadata: Key 5dbf85b8516cd59fca739a6af676accf is removed from metadata 2024.10.22 02:18:49.009759 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 63507314ede3773aadff9a9b421d72ad, offset: 0, size: 155 2024.10.22 02:18:49.024047 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 63507314ede3773aadff9a9b421d72ad for removal 2024.10.22 02:18:49.034776 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3f0bbafc1515f364548ba38d8e3b12a1, offset: 0, size: 257460 2024.10.22 02:18:49.056815 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_868_868_0, total 95 rows starting from the beginning of the part 2024.10.22 02:18:49.136194 [ 1476 ] {} CacheMetadata: Key 63507314ede3773aadff9a9b421d72ad is removed from metadata 2024.10.22 02:18:49.144549 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3f0bbafc1515f364548ba38d8e3b12a1 for removal 2024.10.22 02:18:49.170501 [ 1476 ] {} CacheMetadata: Key 3f0bbafc1515f364548ba38d8e3b12a1 is removed from metadata 2024.10.22 02:18:49.170501 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3534976d5699c70df7fc4dd9538b003b, offset: 0, size: 141243 2024.10.22 02:18:49.192381 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3534976d5699c70df7fc4dd9538b003b for removal 2024.10.22 02:18:49.199262 [ 1476 ] {} CacheMetadata: Key 3534976d5699c70df7fc4dd9538b003b is removed from metadata 2024.10.22 02:18:49.201154 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b82a03bb45fccce80cb240437129cf1e, offset: 0, size: 54839 2024.10.22 02:18:49.207599 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b82a03bb45fccce80cb240437129cf1e for removal 2024.10.22 02:18:49.211120 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_869_869_0, total 135 rows starting from the beginning of the part 2024.10.22 02:18:49.213680 [ 1476 ] {} CacheMetadata: Key b82a03bb45fccce80cb240437129cf1e is removed from metadata 2024.10.22 02:18:49.217258 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 03f5eaf71b698dbd3d2df0046a40f05c, offset: 0, size: 168 2024.10.22 02:18:49.234618 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 03f5eaf71b698dbd3d2df0046a40f05c for removal 2024.10.22 02:18:49.254264 [ 1476 ] {} CacheMetadata: Key 03f5eaf71b698dbd3d2df0046a40f05c is removed from metadata 2024.10.22 02:18:49.255849 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d714c380d30d4ca283d88d911492eb14, offset: 0, size: 5128 2024.10.22 02:18:49.259690 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d714c380d30d4ca283d88d911492eb14 for removal 2024.10.22 02:18:49.262983 [ 1476 ] {} CacheMetadata: Key d714c380d30d4ca283d88d911492eb14 is removed from metadata 2024.10.22 02:18:49.263143 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 384ec9befa399eeb3ac5f61f6fad5a8d, offset: 0, size: 158 2024.10.22 02:18:49.269373 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 384ec9befa399eeb3ac5f61f6fad5a8d for removal 2024.10.22 02:18:49.277361 [ 1476 ] {} CacheMetadata: Key 384ec9befa399eeb3ac5f61f6fad5a8d is removed from metadata 2024.10.22 02:18:49.277170 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7716e2acedefbb0a4df2f901aa8e539b, offset: 0, size: 157 2024.10.22 02:18:49.285526 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 7716e2acedefbb0a4df2f901aa8e539b for removal 2024.10.22 02:18:49.301115 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 44724c50e9741b7505b2605330c723b4, offset: 0, size: 156 2024.10.22 02:18:49.304149 [ 1476 ] {} CacheMetadata: Key 7716e2acedefbb0a4df2f901aa8e539b is removed from metadata 2024.10.22 02:18:49.311579 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 44724c50e9741b7505b2605330c723b4 for removal 2024.10.22 02:18:49.317679 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4f8045395073552adb5d8a21ea9f6bb6, offset: 0, size: 156 2024.10.22 02:18:49.338705 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 4f8045395073552adb5d8a21ea9f6bb6 for removal 2024.10.22 02:18:49.343469 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9f375f8b481ca12f6cd9fe1a2994c738, offset: 0, size: 157 2024.10.22 02:18:49.348477 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9f375f8b481ca12f6cd9fe1a2994c738 for removal 2024.10.22 02:18:49.352903 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 17a666e325c3613479f7904d7af57618, offset: 0, size: 124300 2024.10.22 02:18:49.356924 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 17a666e325c3613479f7904d7af57618 for removal 2024.10.22 02:18:49.361674 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 82d40bc9825312e5cf445a211f618624, offset: 0, size: 75584 2024.10.22 02:18:49.366576 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 82d40bc9825312e5cf445a211f618624 for removal 2024.10.22 02:18:49.369979 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8416110ce765d5d8d32774421d178218, offset: 0, size: 157 2024.10.22 02:18:49.372393 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8416110ce765d5d8d32774421d178218 for removal 2024.10.22 02:18:49.377057 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b77939d70058a16bd2bdd0abb87913b1, offset: 0, size: 61700 2024.10.22 02:18:49.382851 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b77939d70058a16bd2bdd0abb87913b1 for removal 2024.10.22 02:18:49.397804 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8933d7d65593a870e6e3c9af680fc47f, offset: 0, size: 233195 2024.10.22 02:18:49.422986 [ 1476 ] {} CacheMetadata: Key b77939d70058a16bd2bdd0abb87913b1 is removed from metadata 2024.10.22 02:18:49.472724 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_870_870_0, total 119 rows starting from the beginning of the part 2024.10.22 02:18:49.507351 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8933d7d65593a870e6e3c9af680fc47f for removal 2024.10.22 02:18:49.513091 [ 1476 ] {} CacheMetadata: Key 8933d7d65593a870e6e3c9af680fc47f is removed from metadata 2024.10.22 02:18:49.545195 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_890_259} MergeTask::MergeProjectionsStage: Merge sorted 13493 rows, containing 12 columns (12 merged, 0 gathered) in 1.652555571 sec., 8164.9296621444755 rows/sec., 1.27 MiB/sec. 2024.10.22 02:18:49.524619 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: db5ebb0a388bfb5c360a1048cea97e3b, offset: 0, size: 307905 2024.10.22 02:18:49.558110 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key db5ebb0a388bfb5c360a1048cea97e3b for removal 2024.10.22 02:18:49.566942 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3020f12e7f267d397ac6f3d93ba509ed, offset: 0, size: 79316 2024.10.22 02:18:49.569724 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3020f12e7f267d397ac6f3d93ba509ed for removal 2024.10.22 02:18:49.575453 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8c8acc385bc5b609ac398caba05d08da, offset: 0, size: 450882 2024.10.22 02:18:49.581387 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8c8acc385bc5b609ac398caba05d08da for removal 2024.10.22 02:18:49.588893 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 233f9549a20b53ee6a60ee65c690d5af, offset: 0, size: 157 2024.10.22 02:18:49.600077 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 233f9549a20b53ee6a60ee65c690d5af for removal 2024.10.22 02:18:49.614927 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3103e364dfa03c60a102e9d7380bb8b9, offset: 0, size: 1057127 2024.10.22 02:18:49.628690 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3103e364dfa03c60a102e9d7380bb8b9 for removal 2024.10.22 02:18:49.629316 [ 1476 ] {} CacheMetadata: Key 233f9549a20b53ee6a60ee65c690d5af is removed from metadata 2024.10.22 02:18:49.631405 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_871_871_0, total 66 rows starting from the beginning of the part 2024.10.22 02:18:49.643264 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f5f6015101c1455ab4b78fca3106dcb1, offset: 0, size: 89883 2024.10.22 02:18:49.647200 [ 1476 ] {} CacheMetadata: Key 8c8acc385bc5b609ac398caba05d08da is removed from metadata 2024.10.22 02:18:49.651700 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f5f6015101c1455ab4b78fca3106dcb1 for removal 2024.10.22 02:18:49.656827 [ 1476 ] {} CacheMetadata: Key f5f6015101c1455ab4b78fca3106dcb1 is removed from metadata 2024.10.22 02:18:49.659867 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3d0daa716b765cb8b31c7908243a6485, offset: 0, size: 805561 2024.10.22 02:18:49.664179 [ 1476 ] {} CacheMetadata: Key 3020f12e7f267d397ac6f3d93ba509ed is removed from metadata 2024.10.22 02:18:49.666683 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3d0daa716b765cb8b31c7908243a6485 for removal 2024.10.22 02:18:49.672750 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5f808fa57b526cb2d854f3b7cc954c98, offset: 0, size: 404349 2024.10.22 02:18:49.677600 [ 1476 ] {} CacheMetadata: Key 3d0daa716b765cb8b31c7908243a6485 is removed from metadata 2024.10.22 02:18:49.681485 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5f808fa57b526cb2d854f3b7cc954c98 for removal 2024.10.22 02:18:49.697478 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 37c1f0f13d0bc1eaff5a2702875f50b2, offset: 0, size: 83261 2024.10.22 02:18:49.699772 [ 1476 ] {} CacheMetadata: Key 5f808fa57b526cb2d854f3b7cc954c98 is removed from metadata 2024.10.22 02:18:49.704140 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 37c1f0f13d0bc1eaff5a2702875f50b2 for removal 2024.10.22 02:18:49.711322 [ 1476 ] {} CacheMetadata: Key 37c1f0f13d0bc1eaff5a2702875f50b2 is removed from metadata 2024.10.22 02:18:49.710833 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2e747441de63943fc60f5854c620ec93, offset: 0, size: 314195 2024.10.22 02:18:49.718726 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 2e747441de63943fc60f5854c620ec93 for removal 2024.10.22 02:18:49.722622 [ 1476 ] {} CacheMetadata: Key db5ebb0a388bfb5c360a1048cea97e3b is removed from metadata 2024.10.22 02:18:49.728515 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 570c01152f1e64ce25fe73546004a1d6, offset: 0, size: 157 2024.10.22 02:18:49.740198 [ 1476 ] {} CacheMetadata: Key 2e747441de63943fc60f5854c620ec93 is removed from metadata 2024.10.22 02:18:49.741026 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 570c01152f1e64ce25fe73546004a1d6 for removal 2024.10.22 02:18:49.750704 [ 1476 ] {} CacheMetadata: Key 570c01152f1e64ce25fe73546004a1d6 is removed from metadata 2024.10.22 02:18:49.751827 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6710b965c30b9341af6b4bb494eeccb5, offset: 0, size: 242889 2024.10.22 02:18:49.760539 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6710b965c30b9341af6b4bb494eeccb5 for removal 2024.10.22 02:18:49.760873 [ 1476 ] {} CacheMetadata: Key 3103e364dfa03c60a102e9d7380bb8b9 is removed from metadata 2024.10.22 02:18:49.767592 [ 1476 ] {} CacheMetadata: Key 8416110ce765d5d8d32774421d178218 is removed from metadata 2024.10.22 02:18:49.767149 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} MergeTreeSequentialSource: Reading 2 marks from part 202410_872_872_0, total 105 rows starting from the beginning of the part 2024.10.22 02:18:49.786218 [ 1476 ] {} CacheMetadata: Key 6710b965c30b9341af6b4bb494eeccb5 is removed from metadata 2024.10.22 02:18:49.802321 [ 1476 ] {} CacheMetadata: Key 82d40bc9825312e5cf445a211f618624 is removed from metadata 2024.10.22 02:18:49.811706 [ 1476 ] {} CacheMetadata: Key 17a666e325c3613479f7904d7af57618 is removed from metadata 2024.10.22 02:18:49.822447 [ 1476 ] {} CacheMetadata: Key 9f375f8b481ca12f6cd9fe1a2994c738 is removed from metadata 2024.10.22 02:18:49.833037 [ 1476 ] {} CacheMetadata: Key 4f8045395073552adb5d8a21ea9f6bb6 is removed from metadata 2024.10.22 02:18:49.848714 [ 1476 ] {} CacheMetadata: Key 44724c50e9741b7505b2605330c723b4 is removed from metadata 2024.10.22 02:18:49.875145 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 178 entries to flush up to offset 98562 2024.10.22 02:18:49.781537 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 30318abfd9be0cbcc7572b2806a344a5, offset: 0, size: 4604 2024.10.22 02:18:49.894735 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 30318abfd9be0cbcc7572b2806a344a5 for removal 2024.10.22 02:18:49.903476 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_830_207, 202410_831_831_0] 2024.10.22 02:18:49.907057 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_830_207, 202410_831_831_0] 2024.10.22 02:18:49.926632 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_830_207, 202410_831_831_0] 2024.10.22 02:18:49.940559 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_890_259} MergedBlockOutputStream: filled checksums 202410_1_890_259 (state Temporary) 2024.10.22 02:18:49.969625 [ 1476 ] {} CacheMetadata: Key 30318abfd9be0cbcc7572b2806a344a5 is removed from metadata 2024.10.22 02:18:50.002983 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: fa600d10671394943968e601ce0db59d, offset: 0, size: 147563 2024.10.22 02:18:50.030820 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key fa600d10671394943968e601ce0db59d for removal 2024.10.22 02:18:50.038013 [ 1476 ] {} CacheMetadata: Key fa600d10671394943968e601ce0db59d is removed from metadata 2024.10.22 02:18:50.041815 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5d8fda3c9f25f1a1be43a4ea2a3c78cf, offset: 0, size: 155 2024.10.22 02:18:50.049528 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5d8fda3c9f25f1a1be43a4ea2a3c78cf for removal 2024.10.22 02:18:50.055129 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: af706fe28e5df0ab92f01097564dcba3, offset: 0, size: 1070995 2024.10.22 02:18:50.059915 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key af706fe28e5df0ab92f01097564dcba3 for removal 2024.10.22 02:18:50.065452 [ 1476 ] {} CacheMetadata: Key 5d8fda3c9f25f1a1be43a4ea2a3c78cf is removed from metadata 2024.10.22 02:18:50.069700 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c3d96c6e1627344206b06c30f4bc02ff, offset: 0, size: 156 2024.10.22 02:18:50.075316 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c3d96c6e1627344206b06c30f4bc02ff for removal 2024.10.22 02:18:50.076646 [ 1476 ] {} CacheMetadata: Key af706fe28e5df0ab92f01097564dcba3 is removed from metadata 2024.10.22 02:18:50.083325 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3edea1bfb53fc170ed9a0635e247f566, offset: 0, size: 92899 2024.10.22 02:18:50.094498 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3edea1bfb53fc170ed9a0635e247f566 for removal 2024.10.22 02:18:50.100749 [ 1476 ] {} CacheMetadata: Key c3d96c6e1627344206b06c30f4bc02ff is removed from metadata 2024.10.22 02:18:50.103388 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3a4d41c5950b999c9e17c9c9dcdc94a2, offset: 0, size: 126762 2024.10.22 02:18:50.116744 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3a4d41c5950b999c9e17c9c9dcdc94a2 for removal 2024.10.22 02:18:50.119315 [ 1476 ] {} CacheMetadata: Key 3edea1bfb53fc170ed9a0635e247f566 is removed from metadata 2024.10.22 02:18:50.124296 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 25ba2d415597ecb015439c55cbc00f76, offset: 0, size: 73720 2024.10.22 02:18:50.138796 [ 1476 ] {} CacheMetadata: Key 3a4d41c5950b999c9e17c9c9dcdc94a2 is removed from metadata 2024.10.22 02:18:50.145813 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 25ba2d415597ecb015439c55cbc00f76 for removal 2024.10.22 02:18:50.170686 [ 1476 ] {} CacheMetadata: Key 25ba2d415597ecb015439c55cbc00f76 is removed from metadata 2024.10.22 02:18:50.171324 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f54979468134b62b888fc51ce46ef9e4, offset: 0, size: 169 2024.10.22 02:18:50.181525 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f54979468134b62b888fc51ce46ef9e4 for removal 2024.10.22 02:18:50.188584 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9e8e72adb2fd17c8178fc1f65d2977dd, offset: 0, size: 176982 2024.10.22 02:18:50.196806 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9e8e72adb2fd17c8178fc1f65d2977dd for removal 2024.10.22 02:18:50.212973 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: a934eedc32445f078ebd67a343e6bb05, offset: 0, size: 167 2024.10.22 02:18:50.224532 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key a934eedc32445f078ebd67a343e6bb05 for removal 2024.10.22 02:18:50.235939 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9fbeaa735ceb088e650ba66009cf6184, offset: 0, size: 166 2024.10.22 02:18:50.244433 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9fbeaa735ceb088e650ba66009cf6184 for removal 2024.10.22 02:18:50.251062 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3b7a10771bf1da715633a54a46e5e113, offset: 0, size: 103794 2024.10.22 02:18:50.257226 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3b7a10771bf1da715633a54a46e5e113 for removal 2024.10.22 02:18:50.262059 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3d71e1272a4125ef41d0ac8caafc4284, offset: 0, size: 192781 2024.10.22 02:18:50.271989 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3d71e1272a4125ef41d0ac8caafc4284 for removal 2024.10.22 02:18:50.279832 [ 1476 ] {} CacheMetadata: Key 3b7a10771bf1da715633a54a46e5e113 is removed from metadata 2024.10.22 02:18:50.287703 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 342be57331a7b37e0e1b30f5df732080, offset: 0, size: 156 2024.10.22 02:18:50.294824 [ 1476 ] {} CacheMetadata: Key 3d71e1272a4125ef41d0ac8caafc4284 is removed from metadata 2024.10.22 02:18:50.300636 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 342be57331a7b37e0e1b30f5df732080 for removal 2024.10.22 02:18:50.304954 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8cf8bbcbd94bcbad69ba109d1e03a77c, offset: 0, size: 108241 2024.10.22 02:18:50.311860 [ 1476 ] {} CacheMetadata: Key 342be57331a7b37e0e1b30f5df732080 is removed from metadata 2024.10.22 02:18:50.313075 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8cf8bbcbd94bcbad69ba109d1e03a77c for removal 2024.10.22 02:18:50.320226 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0f25e1fa2a4c2656fdf683cb92a63f5a, offset: 0, size: 29314 2024.10.22 02:18:50.328107 [ 1476 ] {} CacheMetadata: Key 8cf8bbcbd94bcbad69ba109d1e03a77c is removed from metadata 2024.10.22 02:18:50.328847 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0f25e1fa2a4c2656fdf683cb92a63f5a for removal 2024.10.22 02:18:50.339661 [ 1476 ] {} CacheMetadata: Key 0f25e1fa2a4c2656fdf683cb92a63f5a is removed from metadata 2024.10.22 02:18:50.339765 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 044575ddbef8a9abfbbbb20891a95f70, offset: 0, size: 67896 2024.10.22 02:18:50.348017 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 044575ddbef8a9abfbbbb20891a95f70 for removal 2024.10.22 02:18:50.353684 [ 1476 ] {} CacheMetadata: Key 9fbeaa735ceb088e650ba66009cf6184 is removed from metadata 2024.10.22 02:18:50.366605 [ 1476 ] {} CacheMetadata: Key 044575ddbef8a9abfbbbb20891a95f70 is removed from metadata 2024.10.22 02:18:50.366801 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 14537af39476f86b5cb7086d4bf1ffe5, offset: 0, size: 169 2024.10.22 02:18:50.376096 [ 1476 ] {} CacheMetadata: Key a934eedc32445f078ebd67a343e6bb05 is removed from metadata 2024.10.22 02:18:50.396030 [ 1476 ] {} CacheMetadata: Key 9e8e72adb2fd17c8178fc1f65d2977dd is removed from metadata 2024.10.22 02:18:50.461692 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} MergeTask::MergeProjectionsStage: Merge sorted 9625 rows, containing 4 columns (4 merged, 0 gathered) in 2.472670979 sec., 3892.5518525285365 rows/sec., 60.82 KiB/sec. 2024.10.22 02:18:50.465733 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} MergeTask::MergeProjectionsStage: Merge sorted 767 rows, containing 32 columns (32 merged, 0 gathered) in 5.493233723 sec., 139.62631824467883 rows/sec., 12.59 KiB/sec. 2024.10.22 02:18:50.475163 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 14537af39476f86b5cb7086d4bf1ffe5 for removal 2024.10.22 02:18:50.485648 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b9fb1aeb0450cbc76c514a20be526c23, offset: 0, size: 156 2024.10.22 02:18:50.511995 [ 1476 ] {} CacheMetadata: Key 14537af39476f86b5cb7086d4bf1ffe5 is removed from metadata 2024.10.22 02:18:50.535395 [ 1476 ] {} CacheMetadata: Key f54979468134b62b888fc51ce46ef9e4 is removed from metadata 2024.10.22 02:18:50.561039 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} MergedBlockOutputStream: filled checksums 202410_1_723_147 (state Temporary) 2024.10.22 02:18:50.608585 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b9fb1aeb0450cbc76c514a20be526c23 for removal 2024.10.22 02:18:50.620490 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} MergedBlockOutputStream: filled checksums 202410_845_850_1 (state Temporary) 2024.10.22 02:18:50.608146 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_890_259} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_890_259 to 202410_1_890_259 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:50.628442 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_890_259} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_889_258, 202410_890_890_0] -> 202410_1_890_259 2024.10.22 02:18:50.647018 [ 1476 ] {} CacheMetadata: Key b9fb1aeb0450cbc76c514a20be526c23 is removed from metadata 2024.10.22 02:18:50.647134 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: dd453509ee18f82f034479fc473a5bab, offset: 0, size: 157 2024.10.22 02:18:50.653625 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key dd453509ee18f82f034479fc473a5bab for removal 2024.10.22 02:18:50.661815 [ 1476 ] {} CacheMetadata: Key dd453509ee18f82f034479fc473a5bab is removed from metadata 2024.10.22 02:18:50.662244 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:18:50.661921 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ae274a4659ebd9e7247f37d6d7d29478, offset: 0, size: 168 2024.10.22 02:18:50.667303 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ae274a4659ebd9e7247f37d6d7d29478 for removal 2024.10.22 02:18:50.671863 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: cf4e3532dd1b037b7ff1c769a17295d0, offset: 0, size: 91267 2024.10.22 02:18:50.672277 [ 1476 ] {} CacheMetadata: Key ae274a4659ebd9e7247f37d6d7d29478 is removed from metadata 2024.10.22 02:18:50.676731 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key cf4e3532dd1b037b7ff1c769a17295d0 for removal 2024.10.22 02:18:50.681815 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: cc29b0e76a9c911c9b7d3817bd7ce81b, offset: 0, size: 2019 2024.10.22 02:18:50.689083 [ 1476 ] {} CacheMetadata: Key cf4e3532dd1b037b7ff1c769a17295d0 is removed from metadata 2024.10.22 02:18:50.689439 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key cc29b0e76a9c911c9b7d3817bd7ce81b for removal 2024.10.22 02:18:50.706626 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 114650898257faeb99107e2d3245723d, offset: 0, size: 168 2024.10.22 02:18:50.708563 [ 1476 ] {} CacheMetadata: Key cc29b0e76a9c911c9b7d3817bd7ce81b is removed from metadata 2024.10.22 02:18:50.714715 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 114650898257faeb99107e2d3245723d for removal 2024.10.22 02:18:50.721467 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: df597e10b7e7b9c6d1d7ca7887911e2c, offset: 0, size: 306574 2024.10.22 02:18:50.727187 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key df597e10b7e7b9c6d1d7ca7887911e2c for removal 2024.10.22 02:18:50.734247 [ 1476 ] {} CacheMetadata: Key 114650898257faeb99107e2d3245723d is removed from metadata 2024.10.22 02:18:50.754228 [ 1476 ] {} CacheMetadata: Key df597e10b7e7b9c6d1d7ca7887911e2c is removed from metadata 2024.10.22 02:18:50.767552 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_845_850_1 to 202410_845_850_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:50.774363 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_845_850_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_845_845_0, 202410_850_850_0] -> 202410_845_850_1 2024.10.22 02:18:50.798480 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:50.742813 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5ba0a4e7e8f295a085a1d6e8f1e948c4, offset: 0, size: 156 2024.10.22 02:18:50.851770 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5ba0a4e7e8f295a085a1d6e8f1e948c4 for removal 2024.10.22 02:18:50.858605 [ 1476 ] {} CacheMetadata: Key 5ba0a4e7e8f295a085a1d6e8f1e948c4 is removed from metadata 2024.10.22 02:18:50.859184 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9f5379ec32e04e76c36c274c8f0f5013, offset: 0, size: 156 2024.10.22 02:18:50.866047 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9f5379ec32e04e76c36c274c8f0f5013 for removal 2024.10.22 02:18:50.872661 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: cc0e377a40e0ea628e1af4b7195dd863, offset: 0, size: 52311 2024.10.22 02:18:50.879938 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key cc0e377a40e0ea628e1af4b7195dd863 for removal 2024.10.22 02:18:50.889420 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8c634725e29c7b451a01e9a896c0fdde, offset: 0, size: 157 2024.10.22 02:18:50.896660 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8c634725e29c7b451a01e9a896c0fdde for removal 2024.10.22 02:18:50.909294 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0b893608a81f5952ee868fe8e9497e8c, offset: 0, size: 156 2024.10.22 02:18:50.914811 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.56 GiB. 2024.10.22 02:18:50.918526 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0b893608a81f5952ee868fe8e9497e8c for removal 2024.10.22 02:18:50.937494 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 449e0fecec8d1d01c559adec26a178e5, offset: 0, size: 77404 2024.10.22 02:18:50.952534 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 449e0fecec8d1d01c559adec26a178e5 for removal 2024.10.22 02:18:50.961442 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 54973871618470021b57bc93e03b676d, offset: 0, size: 26232 2024.10.22 02:18:50.969274 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 54973871618470021b57bc93e03b676d for removal 2024.10.22 02:18:50.972583 [ 1476 ] {} CacheMetadata: Key 449e0fecec8d1d01c559adec26a178e5 is removed from metadata 2024.10.22 02:18:50.977534 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2c2977e313853d8478093c3a0f274cbb, offset: 0, size: 168 2024.10.22 02:18:50.999682 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 2c2977e313853d8478093c3a0f274cbb for removal 2024.10.22 02:18:51.007571 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5e8334639d8f140688777476a1032768, offset: 0, size: 151 2024.10.22 02:18:51.011863 [ 1476 ] {} CacheMetadata: Key 54973871618470021b57bc93e03b676d is removed from metadata 2024.10.22 02:18:51.030403 [ 1476 ] {} CacheMetadata: Key 2c2977e313853d8478093c3a0f274cbb is removed from metadata 2024.10.22 02:18:51.034827 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5e8334639d8f140688777476a1032768 for removal 2024.10.22 02:18:51.046352 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 18146a423482968c061105e00f55897e, offset: 0, size: 209183 2024.10.22 02:18:51.053666 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 18146a423482968c061105e00f55897e for removal 2024.10.22 02:18:51.061373 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e0ff1d27dc0c2c074dd765d543169816, offset: 0, size: 87503 2024.10.22 02:18:51.067935 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e0ff1d27dc0c2c074dd765d543169816 for removal 2024.10.22 02:18:51.070946 [ 1476 ] {} CacheMetadata: Key 18146a423482968c061105e00f55897e is removed from metadata 2024.10.22 02:18:51.076394 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7520a8344e316438da07511a4fdb82ec, offset: 0, size: 156 2024.10.22 02:18:51.082252 [ 1476 ] {} CacheMetadata: Key 5e8334639d8f140688777476a1032768 is removed from metadata 2024.10.22 02:18:51.090418 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 7520a8344e316438da07511a4fdb82ec for removal 2024.10.22 02:18:51.102467 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:18:51.103843 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c8f8b66774b63a430255f7ecda95e4c3, offset: 0, size: 156 2024.10.22 02:18:51.109041 [ 1476 ] {} CacheMetadata: Key 7520a8344e316438da07511a4fdb82ec is removed from metadata 2024.10.22 02:18:51.117136 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c8f8b66774b63a430255f7ecda95e4c3 for removal 2024.10.22 02:18:51.127443 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 31b4885137f7de485a2bf029b22d86d1, offset: 0, size: 63560 2024.10.22 02:18:51.127141 [ 1476 ] {} CacheMetadata: Key c8f8b66774b63a430255f7ecda95e4c3 is removed from metadata 2024.10.22 02:18:51.132843 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 31b4885137f7de485a2bf029b22d86d1 for removal 2024.10.22 02:18:51.151375 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 79014ebe195fe62c409b8eb1a38afdf0, offset: 0, size: 169 2024.10.22 02:18:51.158379 [ 1476 ] {} CacheMetadata: Key 31b4885137f7de485a2bf029b22d86d1 is removed from metadata 2024.10.22 02:18:51.158821 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 79014ebe195fe62c409b8eb1a38afdf0 for removal 2024.10.22 02:18:51.199797 [ 1476 ] {} CacheMetadata: Key 79014ebe195fe62c409b8eb1a38afdf0 is removed from metadata 2024.10.22 02:18:51.215463 [ 1476 ] {} CacheMetadata: Key e0ff1d27dc0c2c074dd765d543169816 is removed from metadata 2024.10.22 02:18:51.268300 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e207e82a1ac11791ff682f406739dc85, offset: 0, size: 157 2024.10.22 02:18:51.275261 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e207e82a1ac11791ff682f406739dc85 for removal 2024.10.22 02:18:51.276454 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1485_1485_0 (state Temporary) 2024.10.22 02:18:51.281141 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 103000a58e881b25e5cd63bda33bebfa, offset: 0, size: 156 2024.10.22 02:18:51.286846 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 103000a58e881b25e5cd63bda33bebfa for removal 2024.10.22 02:18:51.300145 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4ddd317cd7b9ea5f8177ccc1e72dfe10, offset: 0, size: 2333 2024.10.22 02:18:51.315480 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 4ddd317cd7b9ea5f8177ccc1e72dfe10 for removal 2024.10.22 02:18:51.326843 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3f2e81663ceaf9f39fa23b660ee08aff, offset: 0, size: 156 2024.10.22 02:18:51.327584 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:18:51.330910 [ 1476 ] {} CacheMetadata: Key 4ddd317cd7b9ea5f8177ccc1e72dfe10 is removed from metadata 2024.10.22 02:18:51.331425 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:18:51.333383 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3f2e81663ceaf9f39fa23b660ee08aff for removal 2024.10.22 02:18:51.347194 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f3002b94a2a62e18d09aec62513d5110, offset: 0, size: 5373 2024.10.22 02:18:51.356113 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f3002b94a2a62e18d09aec62513d5110 for removal 2024.10.22 02:18:51.364230 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ca7b3aa687e4c1ba13de1a9a2aeb83df, offset: 0, size: 140327 2024.10.22 02:18:51.368813 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ca7b3aa687e4c1ba13de1a9a2aeb83df for removal 2024.10.22 02:18:51.378651 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 16217d31330a0187e4db6b5704f2a2b4, offset: 0, size: 153 2024.10.22 02:18:51.387463 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 16217d31330a0187e4db6b5704f2a2b4 for removal 2024.10.22 02:18:51.399584 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e93905395d05932cc91049fe0b7f8474, offset: 0, size: 157 2024.10.22 02:18:51.411199 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e93905395d05932cc91049fe0b7f8474 for removal 2024.10.22 02:18:51.419503 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 42c2111c826509f06062848c9ebecce4, offset: 0, size: 182 2024.10.22 02:18:51.427269 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 42c2111c826509f06062848c9ebecce4 for removal 2024.10.22 02:18:51.434648 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: fbac0f2a71241de88b54a4e9f292c11b, offset: 0, size: 168 2024.10.22 02:18:51.443991 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key fbac0f2a71241de88b54a4e9f292c11b for removal 2024.10.22 02:18:51.447644 [ 1476 ] {} CacheMetadata: Key 42c2111c826509f06062848c9ebecce4 is removed from metadata 2024.10.22 02:18:51.448921 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 60f154985d50a6576fa32cb6841c8ca3, offset: 0, size: 157 2024.10.22 02:18:51.452484 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 60f154985d50a6576fa32cb6841c8ca3 for removal 2024.10.22 02:18:51.456193 [ 1476 ] {} CacheMetadata: Key 60f154985d50a6576fa32cb6841c8ca3 is removed from metadata 2024.10.22 02:18:51.456164 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0411cf8ef2059bc778dc034ba958ea87, offset: 0, size: 168 2024.10.22 02:18:51.460170 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0411cf8ef2059bc778dc034ba958ea87 for removal 2024.10.22 02:18:51.461087 [ 1476 ] {} CacheMetadata: Key fbac0f2a71241de88b54a4e9f292c11b is removed from metadata 2024.10.22 02:18:51.547504 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1485_1485_0 to 202410_1498_1498_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:51.564112 [ 1476 ] {} CacheMetadata: Key 0411cf8ef2059bc778dc034ba958ea87 is removed from metadata 2024.10.22 02:18:51.565764 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e4eb2fe6027800fdda159b8e839ac138, offset: 0, size: 40507 2024.10.22 02:18:51.572054 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e4eb2fe6027800fdda159b8e839ac138 for removal 2024.10.22 02:18:51.573263 [ 1476 ] {} CacheMetadata: Key e93905395d05932cc91049fe0b7f8474 is removed from metadata 2024.10.22 02:18:51.582922 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e70e59c5b8ed4d5ed2996354ea4b9602, offset: 0, size: 314713 2024.10.22 02:18:51.590279 [ 1476 ] {} CacheMetadata: Key e4eb2fe6027800fdda159b8e839ac138 is removed from metadata 2024.10.22 02:18:51.591950 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e70e59c5b8ed4d5ed2996354ea4b9602 for removal 2024.10.22 02:18:51.602862 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1493_70 to 202410_1498_1498_0 2024.10.22 02:18:51.613275 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d660a343fe525e0ecfdadf65dfe7a9a5, offset: 0, size: 156 2024.10.22 02:18:51.617338 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 98562 2024.10.22 02:18:51.619041 [ 1476 ] {} CacheMetadata: Key e70e59c5b8ed4d5ed2996354ea4b9602 is removed from metadata 2024.10.22 02:18:51.620834 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d660a343fe525e0ecfdadf65dfe7a9a5 for removal 2024.10.22 02:18:51.641373 [ 1476 ] {} CacheMetadata: Key 16217d31330a0187e4db6b5704f2a2b4 is removed from metadata 2024.10.22 02:18:51.642543 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:51.650634 [ 1476 ] {} CacheMetadata: Key f3002b94a2a62e18d09aec62513d5110 is removed from metadata 2024.10.22 02:18:51.658420 [ 1235 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.56 GiB. 2024.10.22 02:18:51.658854 [ 1476 ] {} CacheMetadata: Key 3f2e81663ceaf9f39fa23b660ee08aff is removed from metadata 2024.10.22 02:18:51.667874 [ 1476 ] {} CacheMetadata: Key 103000a58e881b25e5cd63bda33bebfa is removed from metadata 2024.10.22 02:18:51.677765 [ 1476 ] {} CacheMetadata: Key e207e82a1ac11791ff682f406739dc85 is removed from metadata 2024.10.22 02:18:51.697663 [ 1476 ] {} CacheMetadata: Key 0b893608a81f5952ee868fe8e9497e8c is removed from metadata 2024.10.22 02:18:51.698158 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1493_70 to 202410_1498_1498_0 into Compact with storage Full 2024.10.22 02:18:51.701593 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} MergeTask::MergeProjectionsStage: Merge sorted 12443 rows, containing 13 columns (13 merged, 0 gathered) in 3.124748382 sec., 3982.080628212323 rows/sec., 1.73 MiB/sec. 2024.10.22 02:18:51.710550 [ 1476 ] {} CacheMetadata: Key 8c634725e29c7b451a01e9a896c0fdde is removed from metadata 2024.10.22 02:18:51.724326 [ 1476 ] {} CacheMetadata: Key ca7b3aa687e4c1ba13de1a9a2aeb83df is removed from metadata 2024.10.22 02:18:51.743518 [ 1476 ] {} CacheMetadata: Key cc0e377a40e0ea628e1af4b7195dd863 is removed from metadata 2024.10.22 02:18:51.845086 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 059cbb30bcc2231c4eca89fbdcd7173b, offset: 0, size: 158 2024.10.22 02:18:51.850558 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 059cbb30bcc2231c4eca89fbdcd7173b for removal 2024.10.22 02:18:51.852051 [ 1476 ] {} CacheMetadata: Key d660a343fe525e0ecfdadf65dfe7a9a5 is removed from metadata 2024.10.22 02:18:51.855825 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: acefddf0db1108c57cb43aa333aaad0f, offset: 0, size: 134820 2024.10.22 02:18:51.859667 [ 1476 ] {} CacheMetadata: Key 059cbb30bcc2231c4eca89fbdcd7173b is removed from metadata 2024.10.22 02:18:51.862982 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key acefddf0db1108c57cb43aa333aaad0f for removal 2024.10.22 02:18:51.873448 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2b6270a03fa390324d2e9ea2786d0eb3, offset: 0, size: 168 2024.10.22 02:18:51.874640 [ 1476 ] {} CacheMetadata: Key acefddf0db1108c57cb43aa333aaad0f is removed from metadata 2024.10.22 02:18:51.876609 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 2b6270a03fa390324d2e9ea2786d0eb3 for removal 2024.10.22 02:18:51.883437 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0130119a0a24c354180e9ef99e968679, offset: 0, size: 140576 2024.10.22 02:18:51.890006 [ 1476 ] {} CacheMetadata: Key 2b6270a03fa390324d2e9ea2786d0eb3 is removed from metadata 2024.10.22 02:18:51.896664 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0130119a0a24c354180e9ef99e968679 for removal 2024.10.22 02:18:51.904001 [ 1476 ] {} CacheMetadata: Key 9f5379ec32e04e76c36c274c8f0f5013 is removed from metadata 2024.10.22 02:18:51.932377 [ 1476 ] {} CacheMetadata: Key 0130119a0a24c354180e9ef99e968679 is removed from metadata 2024.10.22 02:18:51.933770 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9b27c706da6781f73986ab7fb36579d6, offset: 0, size: 156 2024.10.22 02:18:51.941758 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9b27c706da6781f73986ab7fb36579d6 for removal 2024.10.22 02:18:51.951741 [ 1476 ] {} CacheMetadata: Key 9b27c706da6781f73986ab7fb36579d6 is removed from metadata 2024.10.22 02:18:51.952615 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 60638594578a9ef85209229d6a775dbf, offset: 0, size: 156 2024.10.22 02:18:51.956584 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 60638594578a9ef85209229d6a775dbf for removal 2024.10.22 02:18:51.963468 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0c78f0eabead705f48e841727eb65b82, offset: 0, size: 156 2024.10.22 02:18:51.970602 [ 1476 ] {} CacheMetadata: Key 60638594578a9ef85209229d6a775dbf is removed from metadata 2024.10.22 02:18:51.980677 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0c78f0eabead705f48e841727eb65b82 for removal 2024.10.22 02:18:51.988859 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 526e2c61e1904ec6fc4ef704e0f43109, offset: 0, size: 158 2024.10.22 02:18:51.995674 [ 1476 ] {} CacheMetadata: Key 0c78f0eabead705f48e841727eb65b82 is removed from metadata 2024.10.22 02:18:51.999422 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 526e2c61e1904ec6fc4ef704e0f43109 for removal 2024.10.22 02:18:52.011878 [ 1476 ] {} CacheMetadata: Key 526e2c61e1904ec6fc4ef704e0f43109 is removed from metadata 2024.10.22 02:18:52.037425 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:52.063760 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1493_70, total 18806 rows starting from the beginning of the part 2024.10.22 02:18:52.067490 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_723_147 to 202410_1_723_147 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:52.071581 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_723_147} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_718_146, 202410_723_723_0] -> 202410_1_723_147 2024.10.22 02:18:52.084691 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} MergedBlockOutputStream: filled checksums 202410_747_872_25 (state Temporary) 2024.10.22 02:18:52.132975 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_1494_1494_0, total 99 rows starting from the beginning of the part 2024.10.22 02:18:52.135160 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:18:52.283209 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_1495_1495_0, total 147 rows starting from the beginning of the part 2024.10.22 02:18:52.355825 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_1496_1496_0, total 139 rows starting from the beginning of the part 2024.10.22 02:18:52.420808 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_1497_1497_0, total 52 rows starting from the beginning of the part 2024.10.22 02:18:52.462092 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_872_25 to 202410_747_872_25 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:52.464385 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_872_25} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_867_24, 202410_872_872_0] -> 202410_747_872_25 2024.10.22 02:18:52.499996 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_1498_1498_0, total 178 rows starting from the beginning of the part 2024.10.22 02:18:52.507422 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.66 MiB. 2024.10.22 02:18:52.609134 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 732 2024.10.22 02:18:52.890288 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:52.897188 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.55 GiB. 2024.10.22 02:18:52.945316 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column JavaEnable 2024.10.22 02:18:52.951125 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column JavaEnable 2024.10.22 02:18:52.971861 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_714_714_0 (state Temporary) 2024.10.22 02:18:52.971965 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column JavaEnable 2024.10.22 02:18:52.999132 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column JavaEnable 2024.10.22 02:18:53.010721 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column JavaEnable 2024.10.22 02:18:53.043582 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column JavaEnable 2024.10.22 02:18:53.064890 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column JavaEnable 2024.10.22 02:18:53.100086 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column JavaEnable 2024.10.22 02:18:53.128752 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_insert_202410_714_714_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:18:53.171573 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column JavaEnable 2024.10.22 02:18:53.238166 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ekf/zbygwzhkyaluhrpbiqmrqyutwqlqv (LogSeriesLimiter: on interval from 2024-10-22 02:18:44 to 2024-10-22 02:18:53 accepted series 1 / 12 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:18:53.295379 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_714_714_0 to 202410_724_724_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:53.315200 [ 1151 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:18:53.370192 [ 1151 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:18:53.405809 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_175_179_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:53.549172 [ 1270 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 02:18:53.531592 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 7.36 MiB, will set to 1.60 GiB (RSS), difference: 25.90 MiB 2024.10.22 02:18:53.611557 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 732 2024.10.22 02:18:53.991610 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_175_179_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_175_175_0, total 42 rows starting from the beginning of the part 2024.10.22 02:18:54.682667 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1498_71/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:18:54.754846 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} MergeTask::MergeProjectionsStage: Merge sorted 19421 rows, containing 13 columns (13 merged, 0 gathered) in 3.076449626 sec., 6312.79635976071 rows/sec., 1.18 MiB/sec. 2024.10.22 02:18:54.831835 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1277534 2024.10.22 02:18:54.900349 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} MergedBlockOutputStream: filled checksums 202410_1207_1498_71 (state Temporary) 2024.10.22 02:18:55.069738 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1498_71 to 202410_1207_1498_71 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:55.074856 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1498_71} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1493_70, 202410_1498_1498_0] -> 202410_1207_1498_71 2024.10.22 02:18:55.145742 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.28 MiB. 2024.10.22 02:18:55.177662 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 36 entries to flush up to offset 13473 2024.10.22 02:18:55.363893 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:55.366796 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:18:55.538176 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_882_882_0 (state Temporary) 2024.10.22 02:18:55.535954 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_175_179_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_176_176_0, total 39 rows starting from the beginning of the part 2024.10.22 02:18:55.753273 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_882_882_0 to 202410_895_895_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:55.756455 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 268 entries to flush up to offset 98830 2024.10.22 02:18:55.805625 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:55.811948 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:18:55.851701 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 8.71 MiB, will set to 1.58 GiB (RSS), difference: -21.71 MiB 2024.10.22 02:18:55.921548 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13473 2024.10.22 02:18:55.890436 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 6 parts from 202410_1_890_259 to 202410_895_895_0 2024.10.22 02:18:55.954689 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:55.958393 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:18:55.967864 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_844_844_0 (state Temporary) 2024.10.22 02:18:55.970601 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_890_259 to 202410_895_895_0 into Compact with storage Full 2024.10.22 02:18:56.007706 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:18:56.028796 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_890_259, total 13493 rows starting from the beginning of the part 2024.10.22 02:18:56.095718 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 160 entries to flush up to offset 95958 2024.10.22 02:18:56.103393 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_844_844_0 to 202410_851_851_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:56.152014 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} MergeTreeSequentialSource: Reading 2 marks from part 202410_891_891_0, total 14 rows starting from the beginning of the part 2024.10.22 02:18:56.167538 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1277534 2024.10.22 02:18:56.177119 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_844_158/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:18:56.186242 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} MergeTreeSequentialSource: Reading 2 marks from part 202410_892_892_0, total 19 rows starting from the beginning of the part 2024.10.22 02:18:56.246658 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} MergeTreeSequentialSource: Reading 2 marks from part 202410_893_893_0, total 8 rows starting from the beginning of the part 2024.10.22 02:18:56.270808 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} MergeTreeSequentialSource: Reading 2 marks from part 202410_894_894_0, total 9 rows starting from the beginning of the part 2024.10.22 02:18:56.310464 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} MergeTreeSequentialSource: Reading 2 marks from part 202410_895_895_0, total 36 rows starting from the beginning of the part 2024.10.22 02:18:56.518942 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:56.536838 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:18:56.592565 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_867_867_0 (state Temporary) 2024.10.22 02:18:56.818389 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:18:56.829736 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:18:56.960647 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1486_1486_0 (state Temporary) 2024.10.22 02:18:56.969512 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_867_867_0 to 202410_873_873_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:57.059805 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1486_1486_0 to 202410_1499_1499_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:57.078625 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_175_179_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_177_177_0, total 43 rows starting from the beginning of the part 2024.10.22 02:18:57.145645 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 95958 2024.10.22 02:18:57.221227 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 98830 2024.10.22 02:18:57.364517 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} MergeTask::MergeProjectionsStage: Merge sorted 13579 rows, containing 12 columns (12 merged, 0 gathered) in 1.397678649 sec., 9715.394886882901 rows/sec., 1.52 MiB/sec. 2024.10.22 02:18:57.366940 [ 1275 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 286 ms. 2024.10.22 02:18:57.439655 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} MergedBlockOutputStream: filled checksums 202410_1_895_260 (state Temporary) 2024.10.22 02:18:57.729644 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_895_260 to 202410_1_895_260 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:18:57.738074 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_895_260} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 6 parts: [202410_1_890_259, 202410_895_895_0] -> 202410_1_895_260 2024.10.22 02:18:57.772346 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.69 MiB. 2024.10.22 02:18:58.081054 [ 1260 ] {} DatabaseCatalog: Execution took 340 ms. 2024.10.22 02:18:58.279945 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_831_208, 202410_832_832_0] 2024.10.22 02:18:58.283158 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_831_208, 202410_832_832_0] 2024.10.22 02:18:58.299172 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_831_208, 202410_832_832_0] 2024.10.22 02:18:58.846992 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_175_179_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_178_178_0, total 45 rows starting from the beginning of the part 2024.10.22 02:18:59.717219 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 12.57 MiB, will set to 1.58 GiB (RSS), difference: 4.92 MiB 2024.10.22 02:18:59.860483 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_175_179_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_179_179_0, total 43 rows starting from the beginning of the part 2024.10.22 02:19:00.256638 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:00.259853 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:01.254812 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 98878 2024.10.22 02:19:01.328956 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:19:01.333504 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:19:01.381557 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 10.53 MiB, will set to 1.58 GiB (RSS), difference: -7.00 MiB 2024.10.22 02:19:01.609511 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:01.717323 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:01.772859 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1487_1487_0 (state Temporary) 2024.10.22 02:19:01.849224 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1487_1487_0 to 202410_1500_1500_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:01.925125 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 98878 2024.10.22 02:19:02.083511 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/veh/snvzcqwirfujishkdorybjlqmfuns. 2024.10.22 02:19:02.423847 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SendTiming 2024.10.22 02:19:02.447943 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SendTiming 2024.10.22 02:19:02.458187 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SendTiming 2024.10.22 02:19:02.467724 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SendTiming 2024.10.22 02:19:02.484259 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SendTiming 2024.10.22 02:19:02.625125 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SendTiming 2024.10.22 02:19:02.666686 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SendTiming 2024.10.22 02:19:02.921509 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nkn/iiabvgqftninxwslowqizlzojbanf (LogSeriesLimiter: on interval from 2024-10-22 02:18:53 to 2024-10-22 02:19:02 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:19:03.171184 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1279074 2024.10.22 02:19:03.427343 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 13485 2024.10.22 02:19:03.470810 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 9.80 MiB, will set to 1.58 GiB (RSS), difference: -3.36 MiB 2024.10.22 02:19:03.557348 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:03.562354 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:03.598965 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_883_883_0 (state Temporary) 2024.10.22 02:19:03.652847 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_883_883_0 to 202410_896_896_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:03.733800 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13485 2024.10.22 02:19:04.006926 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_177_177_0 (state Temporary) 2024.10.22 02:19:04.200506 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:04.271448 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:04.497788 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_845_845_0 (state Temporary) 2024.10.22 02:19:04.612833 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_845_845_0 to 202410_852_852_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:04.669560 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1279074 2024.10.22 02:19:04.671399 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 95 entries to flush up to offset 96053 2024.10.22 02:19:05.177326 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:05.181800 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:05.217692 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_832_209, 202410_833_833_0] 2024.10.22 02:19:05.324720 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_832_209, 202410_833_833_0] 2024.10.22 02:19:05.340983 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_832_209, 202410_833_833_0] 2024.10.22 02:19:05.356390 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_868_868_0 (state Temporary) 2024.10.22 02:19:05.419305 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 12.41 MiB, will set to 1.58 GiB (RSS), difference: -9.87 MiB 2024.10.22 02:19:05.419240 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_868_868_0 to 202410_874_874_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:05.488525 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 96053 2024.10.22 02:19:05.615571 [ 1162 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 227 ms. 2024.10.22 02:19:05.937437 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 98921 2024.10.22 02:19:06.097275 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1397_38, 202410_1398_1398_0, 202410_1399_1399_0, 202410_1400_1400_0, 202410_1401_1401_0, 202410_1402_1402_0] 2024.10.22 02:19:06.101754 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1397_38, 202410_1398_1398_0, 202410_1399_1399_0, 202410_1400_1400_0, 202410_1401_1401_0, 202410_1402_1402_0] 2024.10.22 02:19:06.137575 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1397_38, 202410_1398_1398_0, 202410_1399_1399_0, 202410_1400_1400_0, 202410_1401_1401_0, 202410_1402_1402_0] 2024.10.22 02:19:06.200347 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:06.209685 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:06.465653 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1488_1488_0 (state Temporary) 2024.10.22 02:19:06.608510 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1488_1488_0 to 202410_1501_1501_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:06.687676 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 98921 2024.10.22 02:19:07.355172 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_844_158/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:19:07.382381 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_791_791_0, 202410_792_792_0, 202410_793_793_0, 202410_794_794_0, 202410_795_795_0, 202410_796_796_0] 2024.10.22 02:19:07.389681 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_791_791_0, 202410_792_792_0, 202410_793_793_0, 202410_794_794_0, 202410_795_795_0, 202410_796_796_0] 2024.10.22 02:19:07.444465 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_791_791_0, 202410_792_792_0, 202410_793_793_0, 202410_794_794_0, 202410_795_795_0, 202410_796_796_0] 2024.10.22 02:19:07.671776 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 12.80 MiB, will set to 1.58 GiB (RSS), difference: -3.52 MiB 2024.10.22 02:19:08.275078 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_177_177_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:19:08.286201 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 7 old parts to remove. Parts: [202410_1_154_26, 202410_155_159_1, 202410_160_160_0, 202410_161_161_0, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0] 2024.10.22 02:19:08.288706 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from filesystem (serially): Parts: [202410_1_154_26, 202410_155_159_1, 202410_160_160_0, 202410_161_161_0, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0] 2024.10.22 02:19:08.345734 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from memory: Parts: [202410_1_154_26, 202410_155_159_1, 202410_160_160_0, 202410_161_161_0, 202410_162_162_0, 202410_163_163_0, 202410_164_164_0] 2024.10.22 02:19:08.372643 [ 1160 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:19:08.418953 [ 1160 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:19:08.618778 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 733 2024.10.22 02:19:09.059043 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:09.060537 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:09.123749 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_715_715_0 (state Temporary) 2024.10.22 02:19:09.222286 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_715_715_0 to 202410_725_725_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:09.342328 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 733 2024.10.22 02:19:09.603519 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 9.93 MiB, will set to 1.58 GiB (RSS), difference: 2.71 MiB 2024.10.22 02:19:10.156712 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 4 file segments (while reserving 501908 bytes for 33c96f39eb5056aa7161b353086bc52a:0) 2024.10.22 02:19:10.159267 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f5bc8514c9f81723122f291151decde3, offset: 0, size: 212437 2024.10.22 02:19:10.264584 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f5bc8514c9f81723122f291151decde3 for removal 2024.10.22 02:19:10.267255 [ 1476 ] {} CacheMetadata: Key f5bc8514c9f81723122f291151decde3 is removed from metadata 2024.10.22 02:19:10.267329 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ae3c96badf246689f8c2cf0709bb4004, offset: 0, size: 156 2024.10.22 02:19:10.272372 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ae3c96badf246689f8c2cf0709bb4004 for removal 2024.10.22 02:19:10.278266 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b66a71ef156f9ccff3d24c9f2251fae5, offset: 0, size: 338557 2024.10.22 02:19:10.286448 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b66a71ef156f9ccff3d24c9f2251fae5 for removal 2024.10.22 02:19:10.298922 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f24f1ba485762ce984fc4dda0c58d6cc, offset: 0, size: 158 2024.10.22 02:19:10.305051 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f24f1ba485762ce984fc4dda0c58d6cc for removal 2024.10.22 02:19:10.314417 [ 1476 ] {} CacheMetadata: Key f24f1ba485762ce984fc4dda0c58d6cc is removed from metadata 2024.10.22 02:19:10.336622 [ 1476 ] {} CacheMetadata: Key b66a71ef156f9ccff3d24c9f2251fae5 is removed from metadata 2024.10.22 02:19:10.354493 [ 1476 ] {} CacheMetadata: Key ae3c96badf246689f8c2cf0709bb4004 is removed from metadata 2024.10.22 02:19:10.424779 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_844_158} MergeTask::MergeProjectionsStage: Merge sorted 1280344 rows, containing 4 columns (4 merged, 0 gathered) in 40.018134808 sec., 31994.09483082773 rows/sec., 479.88 KiB/sec. 2024.10.22 02:19:10.545341 [ 1696 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ekf/zbygwzhkyaluhrpbiqmrqyutwqlqv, size 501908 2024.10.22 02:19:10.554847 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_844_158} MergedBlockOutputStream: filled checksums 202410_1_844_158 (state Temporary) 2024.10.22 02:19:10.699095 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 98964 2024.10.22 02:19:10.941456 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:10.945756 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:11.013542 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1489_1489_0 (state Temporary) 2024.10.22 02:19:11.069823 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1489_1489_0 to 202410_1502_1502_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:11.144569 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 98964 2024.10.22 02:19:11.175038 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Title 2024.10.22 02:19:11.189805 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Title 2024.10.22 02:19:11.310693 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Title 2024.10.22 02:19:11.333648 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:19:11.338514 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:19:11.340929 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Title 2024.10.22 02:19:11.342928 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 6 entries to flush up to offset 13491 2024.10.22 02:19:11.355218 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Title 2024.10.22 02:19:11.369107 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Title 2024.10.22 02:19:11.387796 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Title 2024.10.22 02:19:11.417002 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Title 2024.10.22 02:19:11.447851 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Title 2024.10.22 02:19:11.541664 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qmv/uwznpocuflslmsylotpnayfprdcvy (LogSeriesLimiter: on interval from 2024-10-22 02:19:02 to 2024-10-22 02:19:11 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:19:11.642966 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:11.646686 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:11.672744 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1280614 2024.10.22 02:19:11.697124 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_884_884_0 (state Temporary) 2024.10.22 02:19:11.959112 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_884_884_0 to 202410_897_897_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:12.054877 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13491 2024.10.22 02:19:12.171533 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:19:12.455520 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:12.457330 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:12.567085 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_846_846_0 (state Temporary) 2024.10.22 02:19:12.747255 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_846_846_0 to 202410_853_853_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:12.754152 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 7.31 MiB, will set to 1.58 GiB (RSS), difference: -19.70 MiB 2024.10.22 02:19:12.774182 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1280614 2024.10.22 02:19:12.857003 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 1048576 bytes for cc6c2047fccddf1ab016c43732324e88:0) 2024.10.22 02:19:12.859376 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8ef429fcc7660abb4d3d920374fb2b6b, offset: 0, size: 156 2024.10.22 02:19:12.862798 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 8ef429fcc7660abb4d3d920374fb2b6b for removal 2024.10.22 02:19:12.865775 [ 1476 ] {} CacheMetadata: Key 8ef429fcc7660abb4d3d920374fb2b6b is removed from metadata 2024.10.22 02:19:12.866430 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 247ca2c677c2ea7ea8450556e72472d4, offset: 0, size: 222948 2024.10.22 02:19:12.869589 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 247ca2c677c2ea7ea8450556e72472d4 for removal 2024.10.22 02:19:12.871647 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8f21db414db5b5ab52ee270c2cd7c029, offset: 0, size: 363126 2024.10.22 02:19:12.872538 [ 1476 ] {} CacheMetadata: Key 247ca2c677c2ea7ea8450556e72472d4 is removed from metadata 2024.10.22 02:19:12.877152 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 8f21db414db5b5ab52ee270c2cd7c029 for removal 2024.10.22 02:19:12.884198 [ 1476 ] {} CacheMetadata: Key 8f21db414db5b5ab52ee270c2cd7c029 is removed from metadata 2024.10.22 02:19:12.884788 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: f5b7b271ecf669ccd24805ed332b2c79, offset: 0, size: 158 2024.10.22 02:19:12.892203 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key f5b7b271ecf669ccd24805ed332b2c79 for removal 2024.10.22 02:19:12.898536 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c2882d01c92e003f636ec377b3f2f94e, offset: 0, size: 156 2024.10.22 02:19:12.899781 [ 1476 ] {} CacheMetadata: Key f5b7b271ecf669ccd24805ed332b2c79 is removed from metadata 2024.10.22 02:19:12.903545 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c2882d01c92e003f636ec377b3f2f94e for removal 2024.10.22 02:19:12.911132 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1176c8150a42089fe1a338679377a0db, offset: 0, size: 176851 2024.10.22 02:19:12.911363 [ 1476 ] {} CacheMetadata: Key c2882d01c92e003f636ec377b3f2f94e is removed from metadata 2024.10.22 02:19:12.914459 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 1176c8150a42089fe1a338679377a0db for removal 2024.10.22 02:19:12.917145 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4a2985be57c55ed38952f500b7549ba1, offset: 0, size: 155 2024.10.22 02:19:12.920712 [ 1476 ] {} CacheMetadata: Key 1176c8150a42089fe1a338679377a0db is removed from metadata 2024.10.22 02:19:12.921406 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 4a2985be57c55ed38952f500b7549ba1 for removal 2024.10.22 02:19:12.927636 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a0f593af88df2ffa7455ea7921df4b12, offset: 0, size: 211290 2024.10.22 02:19:12.932576 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a0f593af88df2ffa7455ea7921df4b12 for removal 2024.10.22 02:19:12.932469 [ 1476 ] {} CacheMetadata: Key 4a2985be57c55ed38952f500b7549ba1 is removed from metadata 2024.10.22 02:19:12.951531 [ 1476 ] {} CacheMetadata: Key a0f593af88df2ffa7455ea7921df4b12 is removed from metadata 2024.10.22 02:19:13.004716 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 68 entries to flush up to offset 96121 2024.10.22 02:19:13.130887 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_177_177_0 to 202410_180_180_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:13.139951 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:13.143100 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:13.177221 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_869_869_0 (state Temporary) 2024.10.22 02:19:13.225819 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_869_869_0 to 202410_875_875_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:13.475997 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 96121 2024.10.22 02:19:13.743245 [ 1210 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 280 ms. 2024.10.22 02:19:13.975863 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_844_158} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_844_158 to 202410_1_844_158 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:14.006381 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_844_158} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_838_157, 202410_839_844_1] -> 202410_1_844_158 2024.10.22 02:19:14.135775 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:19:14.456653 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 6.59 MiB, will set to 1.60 GiB (RSS), difference: 11.20 MiB 2024.10.22 02:19:15.256743 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 99033 2024.10.22 02:19:15.528983 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:15.536525 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:15.805023 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1490_1490_0 (state Temporary) 2024.10.22 02:19:16.011029 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_833_210, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0, 202410_837_837_0, 202410_838_838_0] 2024.10.22 02:19:16.014324 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_833_210, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0, 202410_837_837_0, 202410_838_838_0] 2024.10.22 02:19:16.047754 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_833_210, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0, 202410_837_837_0, 202410_838_838_0] 2024.10.22 02:19:16.062581 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1490_1490_0 to 202410_1503_1503_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:16.080370 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1498_71 to 202410_1499_1499_0 2024.10.22 02:19:16.108457 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 99033 2024.10.22 02:19:16.210425 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:16.224853 [ 1233 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:16.270354 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1499_72} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1498_71 to 202410_1499_1499_0 into Compact with storage Full 2024.10.22 02:19:16.371206 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1499_72} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:19:16.418379 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1499_72} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1498_71, total 19421 rows starting from the beginning of the part 2024.10.22 02:19:16.457799 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 156da8192cc7447bcd13b460024d9999:0) 2024.10.22 02:19:16.459727 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ae688763bcf55894b8335ac5bda43a0a, offset: 0, size: 1175705 2024.10.22 02:19:16.464071 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ae688763bcf55894b8335ac5bda43a0a for removal 2024.10.22 02:19:16.466703 [ 1476 ] {} CacheMetadata: Key ae688763bcf55894b8335ac5bda43a0a is removed from metadata 2024.10.22 02:19:16.467213 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b3d1020c36eed3e05cb8f1ad5e250a02, offset: 0, size: 858820 2024.10.22 02:19:16.469181 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b3d1020c36eed3e05cb8f1ad5e250a02 for removal 2024.10.22 02:19:16.471190 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 85318244e738d449f2442d1371ccec28, offset: 0, size: 190 2024.10.22 02:19:16.475923 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 85318244e738d449f2442d1371ccec28 for removal 2024.10.22 02:19:16.480011 [ 1476 ] {} CacheMetadata: Key 85318244e738d449f2442d1371ccec28 is removed from metadata 2024.10.22 02:19:16.480441 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ad566b83b0f43e866ab6221a2fa72975, offset: 0, size: 193 2024.10.22 02:19:16.483911 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ad566b83b0f43e866ab6221a2fa72975 for removal 2024.10.22 02:19:16.486592 [ 1476 ] {} CacheMetadata: Key b3d1020c36eed3e05cb8f1ad5e250a02 is removed from metadata 2024.10.22 02:19:16.488657 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: cf20bde7f1e27d3b0d7e29c7a2de0b87, offset: 0, size: 1300 2024.10.22 02:19:16.492603 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key cf20bde7f1e27d3b0d7e29c7a2de0b87 for removal 2024.10.22 02:19:16.493819 [ 1476 ] {} CacheMetadata: Key ad566b83b0f43e866ab6221a2fa72975 is removed from metadata 2024.10.22 02:19:16.498575 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8ffb495b2ae985fb0f0c03d496e716ad, offset: 0, size: 156 2024.10.22 02:19:16.502281 [ 1476 ] {} CacheMetadata: Key cf20bde7f1e27d3b0d7e29c7a2de0b87 is removed from metadata 2024.10.22 02:19:16.507254 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8ffb495b2ae985fb0f0c03d496e716ad for removal 2024.10.22 02:19:16.511064 [ 1476 ] {} CacheMetadata: Key 8ffb495b2ae985fb0f0c03d496e716ad is removed from metadata 2024.10.22 02:19:16.584507 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 6.28 MiB, will set to 1.60 GiB (RSS), difference: 4.80 MiB 2024.10.22 02:19:16.593744 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1499_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_1499_1499_0, total 268 rows starting from the beginning of the part 2024.10.22 02:19:16.744230 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7141 2024.10.22 02:19:16.845376 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 734 2024.10.22 02:19:17.107125 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_812_13, 202410_813_813_0, 202410_814_814_0, 202410_815_815_0, 202410_816_816_0, 202410_817_817_0] 2024.10.22 02:19:17.113115 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_812_13, 202410_813_813_0, 202410_814_814_0, 202410_815_815_0, 202410_816_816_0, 202410_817_817_0] 2024.10.22 02:19:17.137705 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_812_13, 202410_813_813_0, 202410_814_814_0, 202410_815_815_0, 202410_816_816_0, 202410_817_817_0] 2024.10.22 02:19:17.204524 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:17.212196 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:17.268704 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_716_716_0 (state Temporary) 2024.10.22 02:19:17.586700 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_716_716_0 to 202410_726_726_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:17.652101 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 734 2024.10.22 02:19:18.166732 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1499_72} MergeTask::MergeProjectionsStage: Merge sorted 19689 rows, containing 13 columns (13 merged, 0 gathered) in 1.901926977 sec., 10352.132462549323 rows/sec., 1.93 MiB/sec. 2024.10.22 02:19:18.341186 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1499_72} MergedBlockOutputStream: filled checksums 202410_1207_1499_72 (state Temporary) 2024.10.22 02:19:18.392437 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 6.87 MiB, will set to 1.60 GiB (RSS), difference: -5.88 MiB 2024.10.22 02:19:18.453118 [ 1266 ] {} DatabaseCatalog: Execution took 231 ms. 2024.10.22 02:19:18.869501 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1499_72} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1499_72 to 202410_1207_1499_72 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:18.873009 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1499_72} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1498_71, 202410_1499_1499_0] -> 202410_1207_1499_72 2024.10.22 02:19:18.923184 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.96 MiB. 2024.10.22 02:19:19.130595 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_175_179_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:19:19.556617 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 13503 2024.10.22 02:19:19.562414 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 2097152 bytes for 156da8192cc7447bcd13b460024d9999:0) 2024.10.22 02:19:19.565074 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4567180f243ee412b0953b3934c82fbc, offset: 0, size: 2144510 2024.10.22 02:19:19.576463 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 4567180f243ee412b0953b3934c82fbc for removal 2024.10.22 02:19:19.580320 [ 1476 ] {} CacheMetadata: Key 4567180f243ee412b0953b3934c82fbc is removed from metadata 2024.10.22 02:19:19.581482 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9d21b5de3f6a252cf53de3bf5ec6daa1, offset: 0, size: 269 2024.10.22 02:19:19.587372 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9d21b5de3f6a252cf53de3bf5ec6daa1 for removal 2024.10.22 02:19:19.591599 [ 1476 ] {} CacheMetadata: Key 9d21b5de3f6a252cf53de3bf5ec6daa1 is removed from metadata 2024.10.22 02:19:19.592459 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 01454703345520705539e0672f26253d, offset: 0, size: 5082 2024.10.22 02:19:19.596545 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 01454703345520705539e0672f26253d for removal 2024.10.22 02:19:19.601462 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ed8e2e8ba28b742c3010df649a0f2162, offset: 0, size: 518003 2024.10.22 02:19:19.601672 [ 1476 ] {} CacheMetadata: Key 01454703345520705539e0672f26253d is removed from metadata 2024.10.22 02:19:19.609354 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ed8e2e8ba28b742c3010df649a0f2162 for removal 2024.10.22 02:19:19.612461 [ 1476 ] {} CacheMetadata: Key ed8e2e8ba28b742c3010df649a0f2162 is removed from metadata 2024.10.22 02:19:19.613053 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: df110e59c184af4973547cef1cefda1b, offset: 0, size: 93558 2024.10.22 02:19:19.616862 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key df110e59c184af4973547cef1cefda1b for removal 2024.10.22 02:19:19.623729 [ 1476 ] {} CacheMetadata: Key df110e59c184af4973547cef1cefda1b is removed from metadata 2024.10.22 02:19:19.623874 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 25d2b3696fe9e07a8ef93ae19225f7f2, offset: 0, size: 162 2024.10.22 02:19:19.631478 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 25d2b3696fe9e07a8ef93ae19225f7f2 for removal 2024.10.22 02:19:19.638433 [ 1476 ] {} CacheMetadata: Key 25d2b3696fe9e07a8ef93ae19225f7f2 is removed from metadata 2024.10.22 02:19:19.637503 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 61d8b7c3d993da79a7fb4495afa20c03, offset: 0, size: 157 2024.10.22 02:19:19.642355 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 61d8b7c3d993da79a7fb4495afa20c03 for removal 2024.10.22 02:19:19.647770 [ 1476 ] {} CacheMetadata: Key 61d8b7c3d993da79a7fb4495afa20c03 is removed from metadata 2024.10.22 02:19:19.648633 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b7278d294152708613c87105bbce6835, offset: 0, size: 192 2024.10.22 02:19:19.652828 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b7278d294152708613c87105bbce6835 for removal 2024.10.22 02:19:19.657502 [ 1476 ] {} CacheMetadata: Key b7278d294152708613c87105bbce6835 is removed from metadata 2024.10.22 02:19:19.737643 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:19.743217 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:19.771474 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_885_885_0 (state Temporary) 2024.10.22 02:19:19.921840 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_885_885_0 to 202410_898_898_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:19.933464 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1282154 2024.10.22 02:19:20.124864 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 85 entries to flush up to offset 99118 2024.10.22 02:19:20.171206 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/veh/snvzcqwirfujishkdorybjlqmfuns. 2024.10.22 02:19:20.235162 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 823562 bytes for cc6c2047fccddf1ab016c43732324e88:0) 2024.10.22 02:19:20.242875 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: bc46a406204268feb302f947e938b866, offset: 0, size: 523627 2024.10.22 02:19:20.247697 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key bc46a406204268feb302f947e938b866 for removal 2024.10.22 02:19:20.250777 [ 1476 ] {} CacheMetadata: Key bc46a406204268feb302f947e938b866 is removed from metadata 2024.10.22 02:19:20.251946 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3a84d23b16e9311a03aaef68a0d49d2c, offset: 0, size: 246 2024.10.22 02:19:20.255625 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 3a84d23b16e9311a03aaef68a0d49d2c for removal 2024.10.22 02:19:20.270047 [ 1476 ] {} CacheMetadata: Key 3a84d23b16e9311a03aaef68a0d49d2c is removed from metadata 2024.10.22 02:19:20.352672 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13503 2024.10.22 02:19:20.491602 [ 1683 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nkn/iiabvgqftninxwslowqizlzojbanf, size 1872138 2024.10.22 02:19:20.578909 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:20.588973 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:20.695101 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:20.702743 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:20.721070 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 2.00 GiB, free memory in arenas 7.57 MiB, will set to 1.60 GiB (RSS), difference: 136.76 MiB 2024.10.22 02:19:20.911065 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column DNSTiming 2024.10.22 02:19:20.926808 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column DNSTiming 2024.10.22 02:19:20.945704 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1491_1491_0 (state Temporary) 2024.10.22 02:19:20.953608 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_847_847_0 (state Temporary) 2024.10.22 02:19:20.954864 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column DNSTiming 2024.10.22 02:19:20.976368 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column DNSTiming 2024.10.22 02:19:21.009801 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 96221 2024.10.22 02:19:21.021162 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column DNSTiming 2024.10.22 02:19:21.047310 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1491_1491_0 to 202410_1504_1504_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:21.046953 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column DNSTiming 2024.10.22 02:19:21.074125 [ 1261 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1499_72 to 202410_1500_1500_0 2024.10.22 02:19:21.082631 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column DNSTiming 2024.10.22 02:19:21.095411 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 99118 2024.10.22 02:19:21.115202 [ 1261 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:21.121247 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:21.165323 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1500_73} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1499_72 to 202410_1500_1500_0 into Compact with storage Full 2024.10.22 02:19:21.272677 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_847_847_0 to 202410_854_854_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:21.303242 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1500_73} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:19:21.317300 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1282154 2024.10.22 02:19:21.328235 [ 1223 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_844_158 to 202410_845_850_1 2024.10.22 02:19:21.331500 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1500_73} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1499_72, total 19689 rows starting from the beginning of the part 2024.10.22 02:19:21.335316 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:19:21.347902 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:19:21.348559 [ 1223 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:21.350688 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:21.466507 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1500_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_1500_1500_0, total 48 rows starting from the beginning of the part 2024.10.22 02:19:21.473345 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/dlj/tidnjuabrjfaekudlizlgdtjfalug (LogSeriesLimiter: on interval from 2024-10-22 02:19:11 to 2024-10-22 02:19:21 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:19:21.475328 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_850_159} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_844_158 to 202410_845_850_1 into Wide with storage Full 2024.10.22 02:19:21.516819 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:21.522906 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:21.604295 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_870_870_0 (state Temporary) 2024.10.22 02:19:21.632438 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_850_159} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:19:21.645447 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_850_159} MergeTreeSequentialSource: Reading 158 marks from part 202410_1_844_158, total 1280344 rows starting from the beginning of the part 2024.10.22 02:19:21.671545 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_850_159} MergeTreeSequentialSource: Reading 3 marks from part 202410_845_850_1, total 9625 rows starting from the beginning of the part 2024.10.22 02:19:21.760210 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_870_870_0 to 202410_876_876_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:21.949591 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 96221 2024.10.22 02:19:22.362923 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1500_73/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:19:22.589433 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 7.31 MiB, will set to 1.59 GiB (RSS), difference: 34.70 MiB 2024.10.22 02:19:22.853823 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 12 file segments (while reserving 1048576 bytes for a057cfe27d155e53e2a041f00570d893:0) 2024.10.22 02:19:22.857061 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 45ea09529056c6b7570f9e468816d03a, offset: 0, size: 170 2024.10.22 02:19:22.861281 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 45ea09529056c6b7570f9e468816d03a for removal 2024.10.22 02:19:22.964502 [ 1476 ] {} CacheMetadata: Key 45ea09529056c6b7570f9e468816d03a is removed from metadata 2024.10.22 02:19:22.965524 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1c613700dce817bda8b71e8b8b0e6251, offset: 0, size: 24591 2024.10.22 02:19:22.969083 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1500_73} MergeTask::MergeProjectionsStage: Merge sorted 19737 rows, containing 13 columns (13 merged, 0 gathered) in 1.813012924 sec., 10886.298568933975 rows/sec., 2.03 MiB/sec. 2024.10.22 02:19:22.969633 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1c613700dce817bda8b71e8b8b0e6251 for removal 2024.10.22 02:19:22.972182 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 875308274ca91935d3aa9859f6deccd7, offset: 0, size: 150079 2024.10.22 02:19:22.977517 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 875308274ca91935d3aa9859f6deccd7 for removal 2024.10.22 02:19:22.983717 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 05ca90bbef54afe7d82e20a06df3cd49, offset: 0, size: 241425 2024.10.22 02:19:22.991067 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 05ca90bbef54afe7d82e20a06df3cd49 for removal 2024.10.22 02:19:22.996471 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dc03916f402caea34bde215511d014a0, offset: 0, size: 137540 2024.10.22 02:19:23.002427 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dc03916f402caea34bde215511d014a0 for removal 2024.10.22 02:19:23.016777 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8454cb9bda625f59537b9f2068fbd8c7, offset: 0, size: 163 2024.10.22 02:19:23.021070 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8454cb9bda625f59537b9f2068fbd8c7 for removal 2024.10.22 02:19:23.027508 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 734e19be9da1ea7133b9cbb8baa3a431, offset: 0, size: 202 2024.10.22 02:19:23.035427 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 734e19be9da1ea7133b9cbb8baa3a431 for removal 2024.10.22 02:19:23.042655 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1500_73} MergedBlockOutputStream: filled checksums 202410_1207_1500_73 (state Temporary) 2024.10.22 02:19:23.042839 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ebb9a3605838234c4ebee83cde700787, offset: 0, size: 335266 2024.10.22 02:19:23.047263 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ebb9a3605838234c4ebee83cde700787 for removal 2024.10.22 02:19:23.050837 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3560212acfcd56614ffaa6f485716594, offset: 0, size: 9968 2024.10.22 02:19:23.055617 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3560212acfcd56614ffaa6f485716594 for removal 2024.10.22 02:19:23.059243 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f336f637ee405e510b28f4c0d8eed8e9, offset: 0, size: 163 2024.10.22 02:19:23.062294 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f336f637ee405e510b28f4c0d8eed8e9 for removal 2024.10.22 02:19:23.083027 [ 1476 ] {} CacheMetadata: Key 1c613700dce817bda8b71e8b8b0e6251 is removed from metadata 2024.10.22 02:19:23.098729 [ 1476 ] {} CacheMetadata: Key f336f637ee405e510b28f4c0d8eed8e9 is removed from metadata 2024.10.22 02:19:23.109304 [ 1476 ] {} CacheMetadata: Key 3560212acfcd56614ffaa6f485716594 is removed from metadata 2024.10.22 02:19:23.128060 [ 1476 ] {} CacheMetadata: Key ebb9a3605838234c4ebee83cde700787 is removed from metadata 2024.10.22 02:19:23.143157 [ 1476 ] {} CacheMetadata: Key 734e19be9da1ea7133b9cbb8baa3a431 is removed from metadata 2024.10.22 02:19:23.150091 [ 1476 ] {} CacheMetadata: Key 8454cb9bda625f59537b9f2068fbd8c7 is removed from metadata 2024.10.22 02:19:23.156356 [ 1476 ] {} CacheMetadata: Key dc03916f402caea34bde215511d014a0 is removed from metadata 2024.10.22 02:19:23.163382 [ 1476 ] {} CacheMetadata: Key 05ca90bbef54afe7d82e20a06df3cd49 is removed from metadata 2024.10.22 02:19:23.166971 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4f5d49c6cd1ff1aa682285ac72c53530, offset: 0, size: 168 2024.10.22 02:19:23.167935 [ 1476 ] {} CacheMetadata: Key 875308274ca91935d3aa9859f6deccd7 is removed from metadata 2024.10.22 02:19:23.171254 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4f5d49c6cd1ff1aa682285ac72c53530 for removal 2024.10.22 02:19:23.172998 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 24c75636c38af5650f0e55f68e779277, offset: 0, size: 242 2024.10.22 02:19:23.175720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 24c75636c38af5650f0e55f68e779277 for removal 2024.10.22 02:19:23.260733 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_850_159/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:19:23.295663 [ 1476 ] {} CacheMetadata: Key 24c75636c38af5650f0e55f68e779277 is removed from metadata 2024.10.22 02:19:23.315134 [ 1476 ] {} CacheMetadata: Key 4f5d49c6cd1ff1aa682285ac72c53530 is removed from metadata 2024.10.22 02:19:23.431216 [ 1188 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:19:23.438497 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1500_73} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1500_73 to 202410_1207_1500_73 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:23.443479 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1500_73} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1499_72, 202410_1500_1500_0] -> 202410_1207_1500_73 2024.10.22 02:19:23.459516 [ 1188 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:19:23.467371 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 10.95 MiB. 2024.10.22 02:19:23.869525 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 34 file segments (while reserving 4194304 bytes for 156da8192cc7447bcd13b460024d9999:0) 2024.10.22 02:19:23.882385 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9b88ba072c4d30bfb05907d5ca484701, offset: 0, size: 2164210 2024.10.22 02:19:23.887960 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9b88ba072c4d30bfb05907d5ca484701 for removal 2024.10.22 02:19:23.892859 [ 1476 ] {} CacheMetadata: Key 9b88ba072c4d30bfb05907d5ca484701 is removed from metadata 2024.10.22 02:19:23.895444 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3e7b58a55c453fdae8cceeecc970021f, offset: 0, size: 198 2024.10.22 02:19:23.901311 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3e7b58a55c453fdae8cceeecc970021f for removal 2024.10.22 02:19:23.907316 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d53f24c9ec195d3473258935f22c8bfa, offset: 0, size: 267 2024.10.22 02:19:23.913269 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d53f24c9ec195d3473258935f22c8bfa for removal 2024.10.22 02:19:23.914702 [ 1476 ] {} CacheMetadata: Key 3e7b58a55c453fdae8cceeecc970021f is removed from metadata 2024.10.22 02:19:23.926608 [ 1476 ] {} CacheMetadata: Key d53f24c9ec195d3473258935f22c8bfa is removed from metadata 2024.10.22 02:19:23.927917 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 22d06ce459c24c2feaccafc67ed3ac3f, offset: 0, size: 365020 2024.10.22 02:19:23.938493 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 22d06ce459c24c2feaccafc67ed3ac3f for removal 2024.10.22 02:19:23.946201 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6401a547ba06a047937ac60c001a8be2, offset: 0, size: 334084 2024.10.22 02:19:23.950491 [ 1476 ] {} CacheMetadata: Key 22d06ce459c24c2feaccafc67ed3ac3f is removed from metadata 2024.10.22 02:19:23.951044 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6401a547ba06a047937ac60c001a8be2 for removal 2024.10.22 02:19:23.953837 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5bd80e657178a65ce19479d7c7f58ae4, offset: 0, size: 197 2024.10.22 02:19:23.956015 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5bd80e657178a65ce19479d7c7f58ae4 for removal 2024.10.22 02:19:23.958625 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c303cdd196eb12ddedca5369b77be62a, offset: 0, size: 207338 2024.10.22 02:19:23.961014 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c303cdd196eb12ddedca5369b77be62a for removal 2024.10.22 02:19:23.964569 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: cec4936398d9eac61e7a3c83e6c0b199, offset: 0, size: 170 2024.10.22 02:19:23.966588 [ 1476 ] {} CacheMetadata: Key c303cdd196eb12ddedca5369b77be62a is removed from metadata 2024.10.22 02:19:23.967715 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key cec4936398d9eac61e7a3c83e6c0b199 for removal 2024.10.22 02:19:23.970221 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d42168c5f23b1fbd4d22f0241777ed85, offset: 0, size: 143614 2024.10.22 02:19:23.974593 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d42168c5f23b1fbd4d22f0241777ed85 for removal 2024.10.22 02:19:23.975515 [ 1476 ] {} CacheMetadata: Key cec4936398d9eac61e7a3c83e6c0b199 is removed from metadata 2024.10.22 02:19:23.985455 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 20d150a12ad40ad4fd6743fcb12073c0, offset: 0, size: 196 2024.10.22 02:19:23.993836 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 20d150a12ad40ad4fd6743fcb12073c0 for removal 2024.10.22 02:19:23.994978 [ 1476 ] {} CacheMetadata: Key d42168c5f23b1fbd4d22f0241777ed85 is removed from metadata 2024.10.22 02:19:23.999188 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f6365fe91278b8475dbede4acd989390, offset: 0, size: 5712 2024.10.22 02:19:24.002668 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f6365fe91278b8475dbede4acd989390 for removal 2024.10.22 02:19:24.003130 [ 1476 ] {} CacheMetadata: Key 20d150a12ad40ad4fd6743fcb12073c0 is removed from metadata 2024.10.22 02:19:24.007331 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: cbdde00d64aa09978cf319d5a0323054, offset: 0, size: 284204 2024.10.22 02:19:24.011377 [ 1476 ] {} CacheMetadata: Key f6365fe91278b8475dbede4acd989390 is removed from metadata 2024.10.22 02:19:24.011533 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key cbdde00d64aa09978cf319d5a0323054 for removal 2024.10.22 02:19:24.016436 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 70622b2ae9d4baf865af84183469025c, offset: 0, size: 114878 2024.10.22 02:19:24.019280 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 70622b2ae9d4baf865af84183469025c for removal 2024.10.22 02:19:24.020190 [ 1476 ] {} CacheMetadata: Key cbdde00d64aa09978cf319d5a0323054 is removed from metadata 2024.10.22 02:19:24.032365 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6c2b154e8c929a9643fa9f769b68b65f, offset: 0, size: 378517 2024.10.22 02:19:24.040248 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6c2b154e8c929a9643fa9f769b68b65f for removal 2024.10.22 02:19:24.047481 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5f5515be69c9c89cee3e00e35cce2381, offset: 0, size: 171 2024.10.22 02:19:24.048686 [ 1476 ] {} CacheMetadata: Key 70622b2ae9d4baf865af84183469025c is removed from metadata 2024.10.22 02:19:24.053465 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5f5515be69c9c89cee3e00e35cce2381 for removal 2024.10.22 02:19:24.056804 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8621cc92ea9513b5516e3001ed6d7d9e, offset: 0, size: 162 2024.10.22 02:19:24.059687 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8621cc92ea9513b5516e3001ed6d7d9e for removal 2024.10.22 02:19:24.064854 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b028dd2bc05082e12794106563b830cf, offset: 0, size: 243 2024.10.22 02:19:24.068548 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b028dd2bc05082e12794106563b830cf for removal 2024.10.22 02:19:24.071711 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f38b0d1747c10cdc15db359e411dd962, offset: 0, size: 202 2024.10.22 02:19:24.074554 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f38b0d1747c10cdc15db359e411dd962 for removal 2024.10.22 02:19:24.079641 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 65be5cee734e6226ed4c7ddd54ae37cc, offset: 0, size: 164 2024.10.22 02:19:24.083824 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 65be5cee734e6226ed4c7ddd54ae37cc for removal 2024.10.22 02:19:24.098161 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9f4148382d972e20428ab85a31690faf, offset: 0, size: 163 2024.10.22 02:19:24.102804 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9f4148382d972e20428ab85a31690faf for removal 2024.10.22 02:19:24.107333 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6be035a651ac596ef62a6d442e0867f8, offset: 0, size: 228202 2024.10.22 02:19:24.112021 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6be035a651ac596ef62a6d442e0867f8 for removal 2024.10.22 02:19:24.141650 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7944f9e12ac7efe0b80fb950cf34d132, offset: 0, size: 162 2024.10.22 02:19:24.146995 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 7944f9e12ac7efe0b80fb950cf34d132 for removal 2024.10.22 02:19:24.152721 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 379dd3e25b13993d6322173c83ce7ff7, offset: 0, size: 162 2024.10.22 02:19:24.156379 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 379dd3e25b13993d6322173c83ce7ff7 for removal 2024.10.22 02:19:24.057790 [ 1476 ] {} CacheMetadata: Key 5f5515be69c9c89cee3e00e35cce2381 is removed from metadata 2024.10.22 02:19:24.159689 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 87ca668eec2a48b130c9a5f14a222a5b, offset: 0, size: 163 2024.10.22 02:19:24.166284 [ 1476 ] {} CacheMetadata: Key 379dd3e25b13993d6322173c83ce7ff7 is removed from metadata 2024.10.22 02:19:24.167876 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 87ca668eec2a48b130c9a5f14a222a5b for removal 2024.10.22 02:19:24.172493 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e958c2c0229244429fe3d479abf65c58, offset: 0, size: 4577 2024.10.22 02:19:24.175245 [ 1476 ] {} CacheMetadata: Key 87ca668eec2a48b130c9a5f14a222a5b is removed from metadata 2024.10.22 02:19:24.179495 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e958c2c0229244429fe3d479abf65c58 for removal 2024.10.22 02:19:24.190686 [ 1476 ] {} CacheMetadata: Key e958c2c0229244429fe3d479abf65c58 is removed from metadata 2024.10.22 02:19:24.192646 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8ea214104dcae68c1c0df2c8cd243c1b, offset: 0, size: 202 2024.10.22 02:19:24.199743 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8ea214104dcae68c1c0df2c8cd243c1b for removal 2024.10.22 02:19:24.214364 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c4ffeea5155c1f440038d712a08ff70b, offset: 0, size: 69217 2024.10.22 02:19:24.219136 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c4ffeea5155c1f440038d712a08ff70b for removal 2024.10.22 02:19:24.231375 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4665b14a1898473f9867f2e766266c12, offset: 0, size: 110361 2024.10.22 02:19:24.243908 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 4665b14a1898473f9867f2e766266c12 for removal 2024.10.22 02:19:24.250595 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 2184a22de852227d67a422eef90381c5, offset: 0, size: 164 2024.10.22 02:19:24.254906 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 2184a22de852227d67a422eef90381c5 for removal 2024.10.22 02:19:24.259492 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e4e6c9f6cb68b5b87a5f7cc70eb35e62, offset: 0, size: 176967 2024.10.22 02:19:24.262265 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e4e6c9f6cb68b5b87a5f7cc70eb35e62 for removal 2024.10.22 02:19:24.267412 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1cd29f4d097f8f377e48ca40503a43f6, offset: 0, size: 53661 2024.10.22 02:19:24.271097 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 1cd29f4d097f8f377e48ca40503a43f6 for removal 2024.10.22 02:19:24.271941 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 7183 2024.10.22 02:19:24.275117 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d8ff7f6b050a448decdaefce65376d0d, offset: 0, size: 233007 2024.10.22 02:19:24.298586 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d8ff7f6b050a448decdaefce65376d0d for removal 2024.10.22 02:19:24.309823 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 769271f5a457eae71a6dfd0a80217955, offset: 0, size: 58157 2024.10.22 02:19:24.310832 [ 1476 ] {} CacheMetadata: Key d8ff7f6b050a448decdaefce65376d0d is removed from metadata 2024.10.22 02:19:24.314265 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 769271f5a457eae71a6dfd0a80217955 for removal 2024.10.22 02:19:24.323390 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f0cd38c2acebff75597dd54be925b75d, offset: 0, size: 200 2024.10.22 02:19:24.333588 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f0cd38c2acebff75597dd54be925b75d for removal 2024.10.22 02:19:24.424332 [ 1476 ] {} CacheMetadata: Key f0cd38c2acebff75597dd54be925b75d is removed from metadata 2024.10.22 02:19:24.436898 [ 1476 ] {} CacheMetadata: Key 769271f5a457eae71a6dfd0a80217955 is removed from metadata 2024.10.22 02:19:24.446715 [ 1476 ] {} CacheMetadata: Key 1cd29f4d097f8f377e48ca40503a43f6 is removed from metadata 2024.10.22 02:19:24.464475 [ 1476 ] {} CacheMetadata: Key e4e6c9f6cb68b5b87a5f7cc70eb35e62 is removed from metadata 2024.10.22 02:19:24.474983 [ 1476 ] {} CacheMetadata: Key 2184a22de852227d67a422eef90381c5 is removed from metadata 2024.10.22 02:19:24.491626 [ 1476 ] {} CacheMetadata: Key 4665b14a1898473f9867f2e766266c12 is removed from metadata 2024.10.22 02:19:24.503442 [ 1476 ] {} CacheMetadata: Key c4ffeea5155c1f440038d712a08ff70b is removed from metadata 2024.10.22 02:19:24.524312 [ 1476 ] {} CacheMetadata: Key 8ea214104dcae68c1c0df2c8cd243c1b is removed from metadata 2024.10.22 02:19:24.539207 [ 1476 ] {} CacheMetadata: Key 7944f9e12ac7efe0b80fb950cf34d132 is removed from metadata 2024.10.22 02:19:24.552982 [ 1476 ] {} CacheMetadata: Key 6be035a651ac596ef62a6d442e0867f8 is removed from metadata 2024.10.22 02:19:24.563441 [ 1476 ] {} CacheMetadata: Key 9f4148382d972e20428ab85a31690faf is removed from metadata 2024.10.22 02:19:24.571995 [ 1476 ] {} CacheMetadata: Key 65be5cee734e6226ed4c7ddd54ae37cc is removed from metadata 2024.10.22 02:19:24.591386 [ 1476 ] {} CacheMetadata: Key f38b0d1747c10cdc15db359e411dd962 is removed from metadata 2024.10.22 02:19:24.592167 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_673_137, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 02:19:24.606341 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_673_137, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 02:19:24.607725 [ 1476 ] {} CacheMetadata: Key b028dd2bc05082e12794106563b830cf is removed from metadata 2024.10.22 02:19:24.619569 [ 1476 ] {} CacheMetadata: Key 8621cc92ea9513b5516e3001ed6d7d9e is removed from metadata 2024.10.22 02:19:24.638226 [ 1204 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 244 ms. 2024.10.22 02:19:24.642107 [ 1476 ] {} CacheMetadata: Key 6c2b154e8c929a9643fa9f769b68b65f is removed from metadata 2024.10.22 02:19:24.655528 [ 1476 ] {} CacheMetadata: Key 5bd80e657178a65ce19479d7c7f58ae4 is removed from metadata 2024.10.22 02:19:24.668499 [ 1476 ] {} CacheMetadata: Key 6401a547ba06a047937ac60c001a8be2 is removed from metadata 2024.10.22 02:19:24.671706 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for a057cfe27d155e53e2a041f00570d893:0) 2024.10.22 02:19:24.698108 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 147874ab6bdf3c34a778bfcb6fb8140a, offset: 0, size: 103010 2024.10.22 02:19:24.702675 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 147874ab6bdf3c34a778bfcb6fb8140a for removal 2024.10.22 02:19:24.705246 [ 1476 ] {} CacheMetadata: Key 147874ab6bdf3c34a778bfcb6fb8140a is removed from metadata 2024.10.22 02:19:24.705675 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 86d9f6a60f19ca13a626b360f2dac8a7, offset: 0, size: 164 2024.10.22 02:19:24.710164 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 86d9f6a60f19ca13a626b360f2dac8a7 for removal 2024.10.22 02:19:24.717389 [ 1476 ] {} CacheMetadata: Key 86d9f6a60f19ca13a626b360f2dac8a7 is removed from metadata 2024.10.22 02:19:24.755545 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_673_137, 202410_674_674_0, 202410_675_675_0, 202410_676_676_0, 202410_677_677_0, 202410_678_678_0] 2024.10.22 02:19:25.111423 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 214 entries to flush up to offset 99332 2024.10.22 02:19:25.155395 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 735 2024.10.22 02:19:25.297122 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 272319 bytes for a057cfe27d155e53e2a041f00570d893:0) 2024.10.22 02:19:25.331086 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:25.335187 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d7742259c4e0b501bc814264b4c0232d, offset: 0, size: 165106 2024.10.22 02:19:25.336647 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:25.344199 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d7742259c4e0b501bc814264b4c0232d for removal 2024.10.22 02:19:25.347224 [ 1476 ] {} CacheMetadata: Key d7742259c4e0b501bc814264b4c0232d is removed from metadata 2024.10.22 02:19:25.348275 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 678d205a1b58f11ea543cb8e79304409, offset: 0, size: 89003 2024.10.22 02:19:25.353040 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 678d205a1b58f11ea543cb8e79304409 for removal 2024.10.22 02:19:25.356688 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a62954170d8010155cddb947adf7c53a, offset: 0, size: 234 2024.10.22 02:19:25.356848 [ 1476 ] {} CacheMetadata: Key 678d205a1b58f11ea543cb8e79304409 is removed from metadata 2024.10.22 02:19:25.363084 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a62954170d8010155cddb947adf7c53a for removal 2024.10.22 02:19:25.367690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9e64684045f8e7a67108719d8c8d076e, offset: 0, size: 168 2024.10.22 02:19:25.367445 [ 1476 ] {} CacheMetadata: Key a62954170d8010155cddb947adf7c53a is removed from metadata 2024.10.22 02:19:25.370886 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9e64684045f8e7a67108719d8c8d076e for removal 2024.10.22 02:19:25.386877 [ 1476 ] {} CacheMetadata: Key 9e64684045f8e7a67108719d8c8d076e is removed from metadata 2024.10.22 02:19:25.449639 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 13.02 MiB, will set to 1.60 GiB (RSS), difference: 7.91 MiB 2024.10.22 02:19:25.531304 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_717_717_0 (state Temporary) 2024.10.22 02:19:25.895471 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_717_717_0 to 202410_727_727_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:26.138738 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 735 2024.10.22 02:19:26.150732 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 49550 bytes for fb012c9b961ac0406e9e1ca53390342c:0) 2024.10.22 02:19:26.153558 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 08d803319b794982285cdf2fc91c2346, offset: 0, size: 269612 2024.10.22 02:19:26.156397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 08d803319b794982285cdf2fc91c2346 for removal 2024.10.22 02:19:26.167287 [ 1476 ] {} CacheMetadata: Key 08d803319b794982285cdf2fc91c2346 is removed from metadata 2024.10.22 02:19:26.169048 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0d36bc2a249820fcc23b1b5af1091f47, offset: 0, size: 239 2024.10.22 02:19:26.179585 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0d36bc2a249820fcc23b1b5af1091f47 for removal 2024.10.22 02:19:26.187992 [ 1476 ] {} CacheMetadata: Key 0d36bc2a249820fcc23b1b5af1091f47 is removed from metadata 2024.10.22 02:19:26.296712 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:26.298355 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:26.549688 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1492_1492_0 (state Temporary) 2024.10.22 02:19:26.599751 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1492_1492_0 to 202410_1505_1505_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:26.655049 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 99332 2024.10.22 02:19:26.762360 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1500_73 to 202410_1505_1505_0 2024.10.22 02:19:26.789517 [ 1173 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:26.798587 [ 1173 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:26.834856 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1500_73 to 202410_1505_1505_0 into Compact with storage Full 2024.10.22 02:19:26.899536 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:19:26.909641 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1500_73, total 19737 rows starting from the beginning of the part 2024.10.22 02:19:27.014687 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_1501_1501_0, total 43 rows starting from the beginning of the part 2024.10.22 02:19:27.269410 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_1502_1502_0, total 43 rows starting from the beginning of the part 2024.10.22 02:19:27.346581 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_1503_1503_0, total 69 rows starting from the beginning of the part 2024.10.22 02:19:27.357236 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 12.93 MiB, will set to 1.60 GiB (RSS), difference: 1.97 MiB 2024.10.22 02:19:27.377224 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_1504_1504_0, total 85 rows starting from the beginning of the part 2024.10.22 02:19:27.462655 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_1505_1505_0, total 214 rows starting from the beginning of the part 2024.10.22 02:19:27.617506 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 6b32873c1628cf704a61c88d6f83e715:0) 2024.10.22 02:19:27.622468 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c74feea6a3244eaf912e9a6c4d7c4f6c, offset: 0, size: 2007079 2024.10.22 02:19:27.632873 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c74feea6a3244eaf912e9a6c4d7c4f6c for removal 2024.10.22 02:19:27.641520 [ 1476 ] {} CacheMetadata: Key c74feea6a3244eaf912e9a6c4d7c4f6c is removed from metadata 2024.10.22 02:19:27.642471 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cff83981bfd637d135e5d3449643c194, offset: 0, size: 269 2024.10.22 02:19:27.647910 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cff83981bfd637d135e5d3449643c194 for removal 2024.10.22 02:19:27.653596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dd7ec0135aa1fe92c49ea163b0f5522a, offset: 0, size: 534466 2024.10.22 02:19:27.656537 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dd7ec0135aa1fe92c49ea163b0f5522a for removal 2024.10.22 02:19:27.658293 [ 1476 ] {} CacheMetadata: Key cff83981bfd637d135e5d3449643c194 is removed from metadata 2024.10.22 02:19:27.659175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fa035a2f1d85b6a09ccace8aaecbc9fc, offset: 0, size: 230 2024.10.22 02:19:27.660869 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fa035a2f1d85b6a09ccace8aaecbc9fc for removal 2024.10.22 02:19:27.662216 [ 1476 ] {} CacheMetadata: Key fa035a2f1d85b6a09ccace8aaecbc9fc is removed from metadata 2024.10.22 02:19:27.661956 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 317bbc8242222add9d4160e364dad132, offset: 0, size: 188041 2024.10.22 02:19:27.665830 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 317bbc8242222add9d4160e364dad132 for removal 2024.10.22 02:19:27.667999 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a5859a616fb3d37c3f9a8f3c5e157863, offset: 0, size: 165 2024.10.22 02:19:27.669341 [ 1476 ] {} CacheMetadata: Key dd7ec0135aa1fe92c49ea163b0f5522a is removed from metadata 2024.10.22 02:19:27.669909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a5859a616fb3d37c3f9a8f3c5e157863 for removal 2024.10.22 02:19:27.673987 [ 1476 ] {} CacheMetadata: Key a5859a616fb3d37c3f9a8f3c5e157863 is removed from metadata 2024.10.22 02:19:27.691914 [ 1476 ] {} CacheMetadata: Key 317bbc8242222add9d4160e364dad132 is removed from metadata 2024.10.22 02:19:27.854056 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 13512 2024.10.22 02:19:28.070565 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:28.074376 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:28.133165 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_886_886_0 (state Temporary) 2024.10.22 02:19:28.197325 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_886_886_0 to 202410_899_899_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:28.261495 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13512 2024.10.22 02:19:28.325287 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1481 entries to flush up to offset 1283635 2024.10.22 02:19:29.056698 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:29.059278 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:29.334579 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_848_848_0 (state Temporary) 2024.10.22 02:19:29.396353 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1402_39, 202410_1403_1403_0, 202410_1404_1404_0, 202410_1405_1405_0, 202410_1406_1406_0, 202410_1407_1407_0] 2024.10.22 02:19:29.402856 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1402_39, 202410_1403_1403_0, 202410_1404_1404_0, 202410_1405_1405_0, 202410_1406_1406_0, 202410_1407_1407_0] 2024.10.22 02:19:29.440894 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_848_848_0 to 202410_855_855_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:29.472961 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 142 entries to flush up to offset 96363 2024.10.22 02:19:29.492419 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1283635 2024.10.22 02:19:29.546435 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1402_39, 202410_1403_1403_0, 202410_1404_1404_0, 202410_1405_1405_0, 202410_1406_1406_0, 202410_1407_1407_0] 2024.10.22 02:19:29.596447 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:29.600857 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:29.656413 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 13.22 MiB, will set to 1.61 GiB (RSS), difference: 6.47 MiB 2024.10.22 02:19:29.762038 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_871_871_0 (state Temporary) 2024.10.22 02:19:29.847071 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} MergeTask::MergeProjectionsStage: Merge sorted 20191 rows, containing 13 columns (13 merged, 0 gathered) in 3.029225567 sec., 6665.3999688759395 rows/sec., 1.24 MiB/sec. 2024.10.22 02:19:29.899311 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} MergedBlockOutputStream: filled checksums 202410_1207_1505_74 (state Temporary) 2024.10.22 02:19:29.903332 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_871_871_0 to 202410_877_877_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:29.912221 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 400246 bytes for 8e23270d5db5cf82cc069001dfc9dcbb:0) 2024.10.22 02:19:29.915901 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: be1b10a0641fbef1c8a9cc32d1d51ae4, offset: 0, size: 218534 2024.10.22 02:19:29.925292 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key be1b10a0641fbef1c8a9cc32d1d51ae4 for removal 2024.10.22 02:19:29.934375 [ 1476 ] {} CacheMetadata: Key be1b10a0641fbef1c8a9cc32d1d51ae4 is removed from metadata 2024.10.22 02:19:29.936881 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fa9243816bed0104139b3179521631ed, offset: 0, size: 171 2024.10.22 02:19:29.941676 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fa9243816bed0104139b3179521631ed for removal 2024.10.22 02:19:29.947932 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_872_25 to 202410_877_877_0 2024.10.22 02:19:29.964357 [ 1476 ] {} CacheMetadata: Key fa9243816bed0104139b3179521631ed is removed from metadata 2024.10.22 02:19:29.974230 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 96363 2024.10.22 02:19:29.979336 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:29.987091 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:30.014571 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_872_25 to 202410_877_877_0 into Compact with storage Full 2024.10.22 02:19:30.118086 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:19:30.253599 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1505_74 to 202410_1207_1505_74 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:30.257198 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1505_74} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1500_73, 202410_1505_1505_0] -> 202410_1207_1505_74 2024.10.22 02:19:30.290203 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_175_179_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:19:30.350969 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_872_25, total 12443 rows starting from the beginning of the part 2024.10.22 02:19:30.368478 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.45 MiB. 2024.10.22 02:19:30.409831 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_873_873_0, total 160 rows starting from the beginning of the part 2024.10.22 02:19:30.475183 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_874_874_0, total 95 rows starting from the beginning of the part 2024.10.22 02:19:30.552802 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_875_875_0, total 68 rows starting from the beginning of the part 2024.10.22 02:19:30.600809 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_876_876_0, total 100 rows starting from the beginning of the part 2024.10.22 02:19:30.767278 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 108 entries to flush up to offset 99440 2024.10.22 02:19:30.925806 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} MergeTreeSequentialSource: Reading 2 marks from part 202410_877_877_0, total 142 rows starting from the beginning of the part 2024.10.22 02:19:30.962281 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:30.965296 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:31.053270 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1493_1493_0 (state Temporary) 2024.10.22 02:19:31.206836 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1493_1493_0 to 202410_1506_1506_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:31.253178 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 99440 2024.10.22 02:19:31.336522 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:19:31.340253 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:19:31.355861 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 34 file segments (while reserving 8388608 bytes for 156da8192cc7447bcd13b460024d9999:0) 2024.10.22 02:19:31.357281 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 044ae66eef8cb19f4e1f68eec74726ac, offset: 0, size: 698685 2024.10.22 02:19:31.359645 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 044ae66eef8cb19f4e1f68eec74726ac for removal 2024.10.22 02:19:31.361696 [ 1476 ] {} CacheMetadata: Key 044ae66eef8cb19f4e1f68eec74726ac is removed from metadata 2024.10.22 02:19:31.362252 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3902256ed3d92555d1bb5aa2f6443154, offset: 0, size: 1099246 2024.10.22 02:19:31.368912 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3902256ed3d92555d1bb5aa2f6443154 for removal 2024.10.22 02:19:31.371659 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 47b0419c258bedf7a2cd97a55cab21dc, offset: 0, size: 257 2024.10.22 02:19:31.374003 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 47b0419c258bedf7a2cd97a55cab21dc for removal 2024.10.22 02:19:31.376683 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1cbdb1d9e7d2ae2c541b3c291c9ba673, offset: 0, size: 1095115 2024.10.22 02:19:31.381548 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 1cbdb1d9e7d2ae2c541b3c291c9ba673 for removal 2024.10.22 02:19:31.385993 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e72f0132d792e2e1a9818c0150221163, offset: 0, size: 247 2024.10.22 02:19:31.386970 [ 1476 ] {} CacheMetadata: Key 1cbdb1d9e7d2ae2c541b3c291c9ba673 is removed from metadata 2024.10.22 02:19:31.390167 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e72f0132d792e2e1a9818c0150221163 for removal 2024.10.22 02:19:31.394579 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 69e7f93862f591cc58f7794216081d43, offset: 0, size: 583058 2024.10.22 02:19:31.395175 [ 1476 ] {} CacheMetadata: Key e72f0132d792e2e1a9818c0150221163 is removed from metadata 2024.10.22 02:19:31.399770 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 69e7f93862f591cc58f7794216081d43 for removal 2024.10.22 02:19:31.403398 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f8c47dd063d4ccf836b9e98cbe941d70, offset: 0, size: 466785 2024.10.22 02:19:31.407663 [ 1476 ] {} CacheMetadata: Key 69e7f93862f591cc58f7794216081d43 is removed from metadata 2024.10.22 02:19:31.409285 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f8c47dd063d4ccf836b9e98cbe941d70 for removal 2024.10.22 02:19:31.417367 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: eceefa70cfc9214acef7a638716776d8, offset: 0, size: 248 2024.10.22 02:19:31.424354 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key eceefa70cfc9214acef7a638716776d8 for removal 2024.10.22 02:19:31.426497 [ 1476 ] {} CacheMetadata: Key f8c47dd063d4ccf836b9e98cbe941d70 is removed from metadata 2024.10.22 02:19:31.437497 [ 1476 ] {} CacheMetadata: Key 47b0419c258bedf7a2cd97a55cab21dc is removed from metadata 2024.10.22 02:19:31.445468 [ 1476 ] {} CacheMetadata: Key 3902256ed3d92555d1bb5aa2f6443154 is removed from metadata 2024.10.22 02:19:31.532777 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 39bc8f58e1ac9b135df5d5ae8eb4fab1, offset: 0, size: 267720 2024.10.22 02:19:31.533517 [ 1476 ] {} CacheMetadata: Key eceefa70cfc9214acef7a638716776d8 is removed from metadata 2024.10.22 02:19:31.538780 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 39bc8f58e1ac9b135df5d5ae8eb4fab1 for removal 2024.10.22 02:19:31.542609 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 005ede693f2437f75e08ba2dd47e488b, offset: 0, size: 257 2024.10.22 02:19:31.543793 [ 1476 ] {} CacheMetadata: Key 39bc8f58e1ac9b135df5d5ae8eb4fab1 is removed from metadata 2024.10.22 02:19:31.544667 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 005ede693f2437f75e08ba2dd47e488b for removal 2024.10.22 02:19:31.548297 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0f8a20ca2c64c54da993b670629413fc, offset: 0, size: 10092 2024.10.22 02:19:31.550032 [ 1476 ] {} CacheMetadata: Key 005ede693f2437f75e08ba2dd47e488b is removed from metadata 2024.10.22 02:19:31.650749 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0f8a20ca2c64c54da993b670629413fc for removal 2024.10.22 02:19:31.653838 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b59c2cf57d57e76d2431a0dad668a93f, offset: 0, size: 234 2024.10.22 02:19:31.654145 [ 1476 ] {} CacheMetadata: Key 0f8a20ca2c64c54da993b670629413fc is removed from metadata 2024.10.22 02:19:31.656056 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b59c2cf57d57e76d2431a0dad668a93f for removal 2024.10.22 02:19:31.662728 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 25c676fec05ba16c95c35f0c312ab6ed, offset: 0, size: 242 2024.10.22 02:19:31.663705 [ 1476 ] {} CacheMetadata: Key b59c2cf57d57e76d2431a0dad668a93f is removed from metadata 2024.10.22 02:19:31.671140 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 25c676fec05ba16c95c35f0c312ab6ed for removal 2024.10.22 02:19:31.673507 [ 1476 ] {} CacheMetadata: Key 25c676fec05ba16c95c35f0c312ab6ed is removed from metadata 2024.10.22 02:19:31.673744 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 39db63860439a81694232755df3b9701, offset: 0, size: 245 2024.10.22 02:19:31.680557 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 39db63860439a81694232755df3b9701 for removal 2024.10.22 02:19:31.708945 [ 1476 ] {} CacheMetadata: Key 39db63860439a81694232755df3b9701 is removed from metadata 2024.10.22 02:19:31.709229 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 17.69 MiB, will set to 1.61 GiB (RSS), difference: -3.34 MiB 2024.10.22 02:19:31.709808 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: aa6081a11a955278d2a8f8f38472ff0b, offset: 0, size: 505771 2024.10.22 02:19:31.718273 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key aa6081a11a955278d2a8f8f38472ff0b for removal 2024.10.22 02:19:31.747253 [ 1476 ] {} CacheMetadata: Key aa6081a11a955278d2a8f8f38472ff0b is removed from metadata 2024.10.22 02:19:31.728268 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d5a61e57b137b42489148a90f022bf6f, offset: 0, size: 169 2024.10.22 02:19:31.834755 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d5a61e57b137b42489148a90f022bf6f for removal 2024.10.22 02:19:31.841566 [ 1476 ] {} CacheMetadata: Key d5a61e57b137b42489148a90f022bf6f is removed from metadata 2024.10.22 02:19:31.845013 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: acd15f680d467dcf37c73255e26f202f, offset: 0, size: 201 2024.10.22 02:19:31.854519 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key acd15f680d467dcf37c73255e26f202f for removal 2024.10.22 02:19:31.857028 [ 1476 ] {} CacheMetadata: Key acd15f680d467dcf37c73255e26f202f is removed from metadata 2024.10.22 02:19:31.857756 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1a8e0e86c18084a68b620db0ba22f8f8, offset: 0, size: 242 2024.10.22 02:19:31.861751 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 1a8e0e86c18084a68b620db0ba22f8f8 for removal 2024.10.22 02:19:31.865017 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 29619a0caf49ea822515550b95d37895, offset: 0, size: 2138880 2024.10.22 02:19:31.866357 [ 1476 ] {} CacheMetadata: Key 1a8e0e86c18084a68b620db0ba22f8f8 is removed from metadata 2024.10.22 02:19:31.869016 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 29619a0caf49ea822515550b95d37895 for removal 2024.10.22 02:19:31.875757 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c535492cebe4fc8d54ede82fa8beb88e, offset: 0, size: 187288 2024.10.22 02:19:31.881347 [ 1476 ] {} CacheMetadata: Key 29619a0caf49ea822515550b95d37895 is removed from metadata 2024.10.22 02:19:31.881711 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c535492cebe4fc8d54ede82fa8beb88e for removal 2024.10.22 02:19:31.888120 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c7d688e9cb6812c081f39af83c29e1f1, offset: 0, size: 244 2024.10.22 02:19:31.892436 [ 1476 ] {} CacheMetadata: Key c535492cebe4fc8d54ede82fa8beb88e is removed from metadata 2024.10.22 02:19:31.995451 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c7d688e9cb6812c081f39af83c29e1f1 for removal 2024.10.22 02:19:32.000917 [ 1476 ] {} CacheMetadata: Key c7d688e9cb6812c081f39af83c29e1f1 is removed from metadata 2024.10.22 02:19:32.001652 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e0936d34d1e7575d727a8163534c905f, offset: 0, size: 196 2024.10.22 02:19:32.108185 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e0936d34d1e7575d727a8163534c905f for removal 2024.10.22 02:19:32.141379 [ 1476 ] {} CacheMetadata: Key e0936d34d1e7575d727a8163534c905f is removed from metadata 2024.10.22 02:19:32.151732 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5a7344bb4e90df9c39c86457f42fbf82, offset: 0, size: 269 2024.10.22 02:19:32.155182 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5a7344bb4e90df9c39c86457f42fbf82 for removal 2024.10.22 02:19:32.157267 [ 1476 ] {} CacheMetadata: Key 5a7344bb4e90df9c39c86457f42fbf82 is removed from metadata 2024.10.22 02:19:32.157652 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d5dde0d7f2ea357f9b2e07f2fa6ad650, offset: 0, size: 131217 2024.10.22 02:19:32.161362 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d5dde0d7f2ea357f9b2e07f2fa6ad650 for removal 2024.10.22 02:19:32.166544 [ 1476 ] {} CacheMetadata: Key d5dde0d7f2ea357f9b2e07f2fa6ad650 is removed from metadata 2024.10.22 02:19:32.167536 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 50ca5f4f04690c620564510863b5c343, offset: 0, size: 164 2024.10.22 02:19:32.170581 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 50ca5f4f04690c620564510863b5c343 for removal 2024.10.22 02:19:32.172824 [ 1476 ] {} CacheMetadata: Key 50ca5f4f04690c620564510863b5c343 is removed from metadata 2024.10.22 02:19:32.173664 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0dfa9f6aa707d7dfeacd65a6f3166115, offset: 0, size: 171 2024.10.22 02:19:32.245838 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_784_148, 202410_785_790_1] 2024.10.22 02:19:32.247738 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_784_148, 202410_785_790_1] 2024.10.22 02:19:32.262030 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_784_148, 202410_785_790_1] 2024.10.22 02:19:32.279140 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0dfa9f6aa707d7dfeacd65a6f3166115 for removal 2024.10.22 02:19:32.282652 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 751e1220ac3588ab9144cc975da13dde, offset: 0, size: 130357 2024.10.22 02:19:32.291455 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 751e1220ac3588ab9144cc975da13dde for removal 2024.10.22 02:19:32.299002 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 89afcd05e15866e1468200791d2b9e6a, offset: 0, size: 700835 2024.10.22 02:19:32.303231 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 89afcd05e15866e1468200791d2b9e6a for removal 2024.10.22 02:19:32.306120 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: a3fdef4bcb5fd6c997d1c338da1934bc, offset: 0, size: 168 2024.10.22 02:19:32.308276 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key a3fdef4bcb5fd6c997d1c338da1934bc for removal 2024.10.22 02:19:32.311536 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: a9623f6538ee01d5d2e5ebd45a9f9161, offset: 0, size: 155803 2024.10.22 02:19:32.315615 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key a9623f6538ee01d5d2e5ebd45a9f9161 for removal 2024.10.22 02:19:32.317993 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ca4a9b59822e1277e5515e5f4d81a247, offset: 0, size: 440782 2024.10.22 02:19:32.325589 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ca4a9b59822e1277e5515e5f4d81a247 for removal 2024.10.22 02:19:32.331842 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3203588b316a829b6e8fe0a2213b143b, offset: 0, size: 77879 2024.10.22 02:19:32.336308 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3203588b316a829b6e8fe0a2213b143b for removal 2024.10.22 02:19:32.340613 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 68e2465702483da927b5d194d09aa9ff, offset: 0, size: 7038 2024.10.22 02:19:32.343893 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 68e2465702483da927b5d194d09aa9ff for removal 2024.10.22 02:19:32.347357 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f55ea60abd8bd494a4b6fee41c62488f, offset: 0, size: 163 2024.10.22 02:19:32.348385 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} MergeTask::MergeProjectionsStage: Merge sorted 13008 rows, containing 13 columns (13 merged, 0 gathered) in 2.343578829 sec., 5550.485368375889 rows/sec., 2.41 MiB/sec. 2024.10.22 02:19:32.349539 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f55ea60abd8bd494a4b6fee41c62488f for removal 2024.10.22 02:19:32.382975 [ 1476 ] {} CacheMetadata: Key 0dfa9f6aa707d7dfeacd65a6f3166115 is removed from metadata 2024.10.22 02:19:32.396370 [ 1476 ] {} CacheMetadata: Key f55ea60abd8bd494a4b6fee41c62488f is removed from metadata 2024.10.22 02:19:32.407865 [ 1476 ] {} CacheMetadata: Key 68e2465702483da927b5d194d09aa9ff is removed from metadata 2024.10.22 02:19:32.417024 [ 1476 ] {} CacheMetadata: Key 3203588b316a829b6e8fe0a2213b143b is removed from metadata 2024.10.22 02:19:32.434837 [ 1476 ] {} CacheMetadata: Key ca4a9b59822e1277e5515e5f4d81a247 is removed from metadata 2024.10.22 02:19:32.452410 [ 1476 ] {} CacheMetadata: Key a9623f6538ee01d5d2e5ebd45a9f9161 is removed from metadata 2024.10.22 02:19:32.458906 [ 1476 ] {} CacheMetadata: Key a3fdef4bcb5fd6c997d1c338da1934bc is removed from metadata 2024.10.22 02:19:32.464178 [ 1476 ] {} CacheMetadata: Key 89afcd05e15866e1468200791d2b9e6a is removed from metadata 2024.10.22 02:19:32.469709 [ 1476 ] {} CacheMetadata: Key 751e1220ac3588ab9144cc975da13dde is removed from metadata 2024.10.22 02:19:32.587858 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} MergedBlockOutputStream: filled checksums 202410_747_877_26 (state Temporary) 2024.10.22 02:19:32.978109 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_877_26 to 202410_747_877_26 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:32.983305 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_877_26} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_872_25, 202410_877_877_0] -> 202410_747_877_26 2024.10.22 02:19:33.037789 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.96 MiB. 2024.10.22 02:19:33.646479 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 736 2024.10.22 02:19:33.857468 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 123107 bytes for 8730abc7444b9874067b6b77dc7e43ed:0) 2024.10.22 02:19:33.860231 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b4504ba62229ac2f8236dec950c1e677, offset: 0, size: 524207 2024.10.22 02:19:33.863982 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b4504ba62229ac2f8236dec950c1e677 for removal 2024.10.22 02:19:33.865786 [ 1476 ] {} CacheMetadata: Key b4504ba62229ac2f8236dec950c1e677 is removed from metadata 2024.10.22 02:19:33.866881 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 48a9bcca7d772129b36f6e9f13e42cb1, offset: 0, size: 246 2024.10.22 02:19:33.868844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 48a9bcca7d772129b36f6e9f13e42cb1 for removal 2024.10.22 02:19:33.880816 [ 1476 ] {} CacheMetadata: Key 48a9bcca7d772129b36f6e9f13e42cb1 is removed from metadata 2024.10.22 02:19:34.032812 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_850_159/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:19:34.063023 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:34.067170 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:34.171988 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_718_718_0 (state Temporary) 2024.10.22 02:19:34.369721 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_718_718_0 to 202410_728_728_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:34.505624 [ 1213 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_723_147 to 202410_728_728_0 2024.10.22 02:19:34.559248 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 18.67 MiB, will set to 1.61 GiB (RSS), difference: -3.51 MiB 2024.10.22 02:19:34.607577 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 736 2024.10.22 02:19:34.634534 [ 1213 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:34.640654 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:34.664969 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_723_147 to 202410_728_728_0 into Compact with storage Full 2024.10.22 02:19:34.852027 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:19:34.983987 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_723_147, total 767 rows starting from the beginning of the part 2024.10.22 02:19:35.073927 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_724_724_0, total 1 rows starting from the beginning of the part 2024.10.22 02:19:35.164990 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_725_725_0, total 1 rows starting from the beginning of the part 2024.10.22 02:19:35.261629 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 145 entries to flush up to offset 99585 2024.10.22 02:19:35.283133 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_726_726_0, total 1 rows starting from the beginning of the part 2024.10.22 02:19:35.509399 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_727_727_0, total 1 rows starting from the beginning of the part 2024.10.22 02:19:35.616734 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_728_728_0, total 1 rows starting from the beginning of the part 2024.10.22 02:19:35.662442 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:35.667448 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:35.764067 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 13531 2024.10.22 02:19:35.780503 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1494_1494_0 (state Temporary) 2024.10.22 02:19:35.929519 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1494_1494_0 to 202410_1507_1507_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:35.989646 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 99585 2024.10.22 02:19:36.103352 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:36.112353 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:36.151254 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_887_887_0 (state Temporary) 2024.10.22 02:19:36.329280 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 680256c64a63d2136a549c7a3305acac:0) 2024.10.22 02:19:36.331376 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_887_887_0 to 202410_900_900_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:36.338363 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 53bb1df6362db3513aa9efc49f33000b, offset: 0, size: 1020775 2024.10.22 02:19:36.344078 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 53bb1df6362db3513aa9efc49f33000b for removal 2024.10.22 02:19:36.347995 [ 1476 ] {} CacheMetadata: Key 53bb1df6362db3513aa9efc49f33000b is removed from metadata 2024.10.22 02:19:36.348770 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: f990b3a32a69ac992d6d10f706930707, offset: 0, size: 256 2024.10.22 02:19:36.354462 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key f990b3a32a69ac992d6d10f706930707 for removal 2024.10.22 02:19:36.359817 [ 1164 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_895_260 to 202410_896_896_0 2024.10.22 02:19:36.363111 [ 1476 ] {} CacheMetadata: Key f990b3a32a69ac992d6d10f706930707 is removed from metadata 2024.10.22 02:19:36.374407 [ 1164 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:36.378427 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:36.428862 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_896_261} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_895_260 to 202410_896_896_0 into Compact with storage Full 2024.10.22 02:19:36.476290 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 15.39 MiB, will set to 1.62 GiB (RSS), difference: -5.28 MiB 2024.10.22 02:19:36.503399 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1214 entries to flush up to offset 1284849 2024.10.22 02:19:36.584954 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_896_261} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:19:36.707660 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_896_261} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_895_260, total 13579 rows starting from the beginning of the part 2024.10.22 02:19:36.749166 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_896_261} MergeTreeSequentialSource: Reading 2 marks from part 202410_896_896_0, total 12 rows starting from the beginning of the part 2024.10.22 02:19:36.824319 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13531 2024.10.22 02:19:37.444546 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:37.449532 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:37.509088 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 109 entries to flush up to offset 96472 2024.10.22 02:19:37.733015 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_849_849_0 (state Temporary) 2024.10.22 02:19:37.778359 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_849_849_0 to 202410_856_856_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:37.854696 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1284849 2024.10.22 02:19:37.846782 [ 1248 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_851_851_0 to 202410_856_856_0 2024.10.22 02:19:37.875500 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_896_261} MergeTask::MergeProjectionsStage: Merge sorted 13591 rows, containing 12 columns (12 merged, 0 gathered) in 1.456555239 sec., 9330.919718040299 rows/sec., 1.46 MiB/sec. 2024.10.22 02:19:37.896512 [ 1248 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:37.908705 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:37.944832 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_896_261} MergedBlockOutputStream: filled checksums 202410_1_896_261 (state Temporary) 2024.10.22 02:19:37.956293 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_851_851_0 to 202410_856_856_0 into Compact with storage Full 2024.10.22 02:19:37.963965 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 210113 bytes for 7101286de23025f3078161bb969f454a:0) 2024.10.22 02:19:37.965698 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a9b3485cf68b117ca70993e1acfa55b0, offset: 0, size: 862731 2024.10.22 02:19:37.967897 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a9b3485cf68b117ca70993e1acfa55b0 for removal 2024.10.22 02:19:37.969416 [ 1476 ] {} CacheMetadata: Key a9b3485cf68b117ca70993e1acfa55b0 is removed from metadata 2024.10.22 02:19:37.969695 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9b5d9dd2874b821f371cecc987ca41e6, offset: 0, size: 249 2024.10.22 02:19:37.971700 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:19:37.972548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9b5d9dd2874b821f371cecc987ca41e6 for removal 2024.10.22 02:19:37.978932 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_851_851_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:19:37.984360 [ 1476 ] {} CacheMetadata: Key 9b5d9dd2874b821f371cecc987ca41e6 is removed from metadata 2024.10.22 02:19:37.997308 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_852_852_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:19:38.016026 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_853_853_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:19:38.047229 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:38.050738 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_854_854_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:19:38.052902 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:38.062986 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_855_855_0, total 1481 rows starting from the beginning of the part 2024.10.22 02:19:38.080877 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_856_856_0, total 1214 rows starting from the beginning of the part 2024.10.22 02:19:38.091575 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} MergeTask::MergeProjectionsStage: Merge sorted 772 rows, containing 32 columns (32 merged, 0 gathered) in 3.430638858 sec., 225.03097293372988 rows/sec., 20.29 KiB/sec. 2024.10.22 02:19:38.116634 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} MergedBlockOutputStream: filled checksums 202410_1_728_148 (state Temporary) 2024.10.22 02:19:38.130214 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_896_261} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_896_261 to 202410_1_896_261 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:38.142002 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_896_261} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_895_260, 202410_896_896_0] -> 202410_1_896_261 2024.10.22 02:19:38.143288 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_872_872_0 (state Temporary) 2024.10.22 02:19:38.171314 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:19:38.350241 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_872_872_0 to 202410_878_878_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:38.416994 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 96472 2024.10.22 02:19:38.467720 [ 1266 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:19:38.601193 [ 1266 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:19:38.605743 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_728_148 to 202410_1_728_148 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:38.610554 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_728_148} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_723_147, 202410_728_728_0] -> 202410_1_728_148 2024.10.22 02:19:38.630629 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 8.00 MiB, will set to 1.62 GiB (RSS), difference: 1.13 MiB 2024.10.22 02:19:38.813306 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:19:38.903298 [ 1271 ] {} DatabaseCatalog: Execution took 329 ms. 2024.10.22 02:19:38.952425 [ 1723 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/dlj/tidnjuabrjfaekudlizlgdtjfalug, size 2363042 2024.10.22 02:19:39.169417 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 02:19:39.196561 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 02:19:39.213159 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 02:19:39.338383 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 02:19:39.346855 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} MergeTask::MergeProjectionsStage: Merge sorted 8470 rows, containing 4 columns (4 merged, 0 gathered) in 1.396112934 sec., 6066.844446267411 rows/sec., 94.79 KiB/sec. 2024.10.22 02:19:39.349397 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 02:19:39.364198 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 02:19:39.376647 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ConnectTiming 2024.10.22 02:19:39.405045 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} MergedBlockOutputStream: filled checksums 202410_851_856_1 (state Temporary) 2024.10.22 02:19:39.451099 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gyd/lticjnmdzxqwkrgrdzqxzoiwwwxpb (LogSeriesLimiter: on interval from 2024-10-22 02:19:21 to 2024-10-22 02:19:39 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:19:39.513060 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_851_856_1 to 202410_851_856_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:39.619431 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_851_856_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_851_851_0, 202410_856_856_0] -> 202410_851_856_1 2024.10.22 02:19:39.698292 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:19:40.052345 [ 1257 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 02:19:40.114337 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 99673 2024.10.22 02:19:40.175928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 176245 bytes for 0043e87303056e9c62affb15cad3eb60:0) 2024.10.22 02:19:40.181322 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 96e82f874a4f31856979e44663230386, offset: 0, size: 514177 2024.10.22 02:19:40.195624 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 96e82f874a4f31856979e44663230386 for removal 2024.10.22 02:19:40.201635 [ 1476 ] {} CacheMetadata: Key 96e82f874a4f31856979e44663230386 is removed from metadata 2024.10.22 02:19:40.201800 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 29abedfcccd6870fd84dcf6d4436e3bf, offset: 0, size: 246 2024.10.22 02:19:40.223149 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 29abedfcccd6870fd84dcf6d4436e3bf for removal 2024.10.22 02:19:40.241651 [ 1476 ] {} CacheMetadata: Key 29abedfcccd6870fd84dcf6d4436e3bf is removed from metadata 2024.10.22 02:19:40.415045 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:40.420970 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:40.690871 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 11.25 MiB, will set to 1.62 GiB (RSS), difference: 5.00 MiB 2024.10.22 02:19:40.695501 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1495_1495_0 (state Temporary) 2024.10.22 02:19:40.752687 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1495_1495_0 to 202410_1508_1508_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:40.808387 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 99673 2024.10.22 02:19:41.339611 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:19:41.342109 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:19:41.345242 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 1048576 bytes for a7040f410374dfe02c48a9e8db891d0e:0) 2024.10.22 02:19:41.450416 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5a5df5f034cac9ea6ec3132927cf2af5, offset: 0, size: 431565 2024.10.22 02:19:41.453562 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5a5df5f034cac9ea6ec3132927cf2af5 for removal 2024.10.22 02:19:41.459047 [ 1476 ] {} CacheMetadata: Key 5a5df5f034cac9ea6ec3132927cf2af5 is removed from metadata 2024.10.22 02:19:41.458716 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7858ae56f4f350a8ccf015696a18cca3, offset: 0, size: 512979 2024.10.22 02:19:41.461545 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7858ae56f4f350a8ccf015696a18cca3 for removal 2024.10.22 02:19:41.462397 [ 1476 ] {} CacheMetadata: Key 7858ae56f4f350a8ccf015696a18cca3 is removed from metadata 2024.10.22 02:19:41.462947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ad781a7835f3f81e9285bf385e8ebc41, offset: 0, size: 245 2024.10.22 02:19:41.465473 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ad781a7835f3f81e9285bf385e8ebc41 for removal 2024.10.22 02:19:41.567577 [ 1476 ] {} CacheMetadata: Key ad781a7835f3f81e9285bf385e8ebc41 is removed from metadata 2024.10.22 02:19:41.567970 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cf64ff2db327cc2d39449e6902b8638b, offset: 0, size: 246 2024.10.22 02:19:41.569991 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cf64ff2db327cc2d39449e6902b8638b for removal 2024.10.22 02:19:41.571679 [ 1476 ] {} CacheMetadata: Key cf64ff2db327cc2d39449e6902b8638b is removed from metadata 2024.10.22 02:19:42.110881 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 737 2024.10.22 02:19:42.274247 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:42.290231 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:42.357683 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_719_719_0 (state Temporary) 2024.10.22 02:19:42.546910 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_719_719_0 to 202410_729_729_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:42.633758 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 737 2024.10.22 02:19:43.040363 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:43.042878 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:43.227953 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 89720 bytes for 54cae4a2e079a4af8ceb0317d3e7b525:0) 2024.10.22 02:19:43.234804 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 85a6f5bf20c6d0d670879e7025c45b15, offset: 0, size: 978198 2024.10.22 02:19:43.239842 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 85a6f5bf20c6d0d670879e7025c45b15 for removal 2024.10.22 02:19:43.243457 [ 1476 ] {} CacheMetadata: Key 85a6f5bf20c6d0d670879e7025c45b15 is removed from metadata 2024.10.22 02:19:43.244034 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 543bd332328ac7b7f88ed854a4ee53bc, offset: 0, size: 255 2024.10.22 02:19:43.247454 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 543bd332328ac7b7f88ed854a4ee53bc for removal 2024.10.22 02:19:43.261130 [ 1476 ] {} CacheMetadata: Key 543bd332328ac7b7f88ed854a4ee53bc is removed from metadata 2024.10.22 02:19:43.745598 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 14.71 MiB, will set to 1.62 GiB (RSS), difference: 12.22 MiB 2024.10.22 02:19:44.312378 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_850_159/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:19:44.328227 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 13554 2024.10.22 02:19:44.835451 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 6 file segments (while reserving 990808 bytes for db49f6b5c905ee3a7a75f7971ac92076:0) 2024.10.22 02:19:44.863710 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1286389 2024.10.22 02:19:44.955532 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 99721 2024.10.22 02:19:44.956940 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e66ba8ba0466cffd23bd372ed6525ce5, offset: 0, size: 1946939 2024.10.22 02:19:44.959415 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e66ba8ba0466cffd23bd372ed6525ce5 for removal 2024.10.22 02:19:44.961239 [ 1476 ] {} CacheMetadata: Key e66ba8ba0466cffd23bd372ed6525ce5 is removed from metadata 2024.10.22 02:19:44.972189 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 65ba5213c9150cadaf05fb83f9b7e639, offset: 0, size: 260 2024.10.22 02:19:44.975260 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 65ba5213c9150cadaf05fb83f9b7e639 for removal 2024.10.22 02:19:44.980058 [ 1476 ] {} CacheMetadata: Key 65ba5213c9150cadaf05fb83f9b7e639 is removed from metadata 2024.10.22 02:19:44.981225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 63bf1a42a28a8da58c045a3326d7fc1f, offset: 0, size: 2532 2024.10.22 02:19:44.990194 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 63bf1a42a28a8da58c045a3326d7fc1f for removal 2024.10.22 02:19:44.994259 [ 1476 ] {} CacheMetadata: Key 63bf1a42a28a8da58c045a3326d7fc1f is removed from metadata 2024.10.22 02:19:44.995369 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0237d1216ef783f592be3bcea0226561, offset: 0, size: 163 2024.10.22 02:19:45.004078 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0237d1216ef783f592be3bcea0226561 for removal 2024.10.22 02:19:45.010290 [ 1476 ] {} CacheMetadata: Key 0237d1216ef783f592be3bcea0226561 is removed from metadata 2024.10.22 02:19:45.012015 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 28a0d8c0ffa651cc2011373216cef7d6, offset: 0, size: 502791 2024.10.22 02:19:45.017618 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 28a0d8c0ffa651cc2011373216cef7d6 for removal 2024.10.22 02:19:45.031796 [ 1476 ] {} CacheMetadata: Key 28a0d8c0ffa651cc2011373216cef7d6 is removed from metadata 2024.10.22 02:19:45.118012 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:45.122942 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:45.127198 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 503dc280eb3ab31f2ad81a8b1105b053, offset: 0, size: 245 2024.10.22 02:19:45.164550 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_888_888_0 (state Temporary) 2024.10.22 02:19:45.234672 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 503dc280eb3ab31f2ad81a8b1105b053 for removal 2024.10.22 02:19:45.241031 [ 1476 ] {} CacheMetadata: Key 503dc280eb3ab31f2ad81a8b1105b053 is removed from metadata 2024.10.22 02:19:45.336886 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:45.339721 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_888_888_0 to 202410_901_901_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:45.341197 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:45.363651 [ 1159 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_896_261 to 202410_897_897_0 2024.10.22 02:19:45.368156 [ 1159 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:45.371345 [ 1159 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:45.377493 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13554 2024.10.22 02:19:45.422814 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_897_262} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_896_261 to 202410_897_897_0 into Compact with storage Full 2024.10.22 02:19:45.467031 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 14.51 MiB, will set to 1.62 GiB (RSS), difference: -6.19 MiB 2024.10.22 02:19:45.493017 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_178_178_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:19:45.608281 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1496_1496_0 (state Temporary) 2024.10.22 02:19:45.657467 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1496_1496_0 to 202410_1509_1509_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:45.749241 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 99721 2024.10.22 02:19:45.760758 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:45.763170 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:45.797684 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 32 file segments (while reserving 16777216 bytes for 156da8192cc7447bcd13b460024d9999:0) 2024.10.22 02:19:45.801342 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6b32873c1628cf704a61c88d6f83e715, offset: 0, size: 1111843 2024.10.22 02:19:45.805614 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6b32873c1628cf704a61c88d6f83e715 for removal 2024.10.22 02:19:45.810469 [ 1476 ] {} CacheMetadata: Key 6b32873c1628cf704a61c88d6f83e715 is removed from metadata 2024.10.22 02:19:45.810632 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7413e8c19d2563aab9b55f8b325daddb, offset: 0, size: 164 2024.10.22 02:19:45.823013 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 7413e8c19d2563aab9b55f8b325daddb for removal 2024.10.22 02:19:45.830441 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6578ec25af9f08843e136eb673bb65ce, offset: 0, size: 2637 2024.10.22 02:19:45.838808 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6578ec25af9f08843e136eb673bb65ce for removal 2024.10.22 02:19:45.841432 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_897_262} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:19:45.845431 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8b1912e64a994fc3377cd1fbdef318cf, offset: 0, size: 164 2024.10.22 02:19:45.849414 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_897_262} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_896_261, total 13591 rows starting from the beginning of the part 2024.10.22 02:19:45.849451 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8b1912e64a994fc3377cd1fbdef318cf for removal 2024.10.22 02:19:45.855760 [ 1476 ] {} CacheMetadata: Key 6578ec25af9f08843e136eb673bb65ce is removed from metadata 2024.10.22 02:19:45.860032 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_897_262} MergeTreeSequentialSource: Reading 2 marks from part 202410_897_897_0, total 6 rows starting from the beginning of the part 2024.10.22 02:19:45.945515 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_850_850_0 (state Temporary) 2024.10.22 02:19:45.953660 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: fb012c9b961ac0406e9e1ca53390342c, offset: 0, size: 49550 2024.10.22 02:19:45.955663 [ 1476 ] {} CacheMetadata: Key 8b1912e64a994fc3377cd1fbdef318cf is removed from metadata 2024.10.22 02:19:45.956925 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key fb012c9b961ac0406e9e1ca53390342c for removal 2024.10.22 02:19:45.959387 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: fe96f669aee32eafea35806fce68acfd, offset: 0, size: 159 2024.10.22 02:19:45.960145 [ 1476 ] {} CacheMetadata: Key fb012c9b961ac0406e9e1ca53390342c is removed from metadata 2024.10.22 02:19:45.963405 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key fe96f669aee32eafea35806fce68acfd for removal 2024.10.22 02:19:45.970601 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: a057cfe27d155e53e2a041f00570d893, offset: 0, size: 2369471 2024.10.22 02:19:45.972169 [ 1476 ] {} CacheMetadata: Key fe96f669aee32eafea35806fce68acfd is removed from metadata 2024.10.22 02:19:45.974647 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key a057cfe27d155e53e2a041f00570d893 for removal 2024.10.22 02:19:45.981665 [ 1476 ] {} CacheMetadata: Key 7413e8c19d2563aab9b55f8b325daddb is removed from metadata 2024.10.22 02:19:45.983638 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 674ca8b4b72993c09f267aba6ab1abbb, offset: 0, size: 88092 2024.10.22 02:19:45.992485 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 674ca8b4b72993c09f267aba6ab1abbb for removal 2024.10.22 02:19:45.992701 [ 1476 ] {} CacheMetadata: Key a057cfe27d155e53e2a041f00570d893 is removed from metadata 2024.10.22 02:19:45.997251 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c622cd6cd49bc4d911093d8feb7dad8a, offset: 0, size: 4541507 2024.10.22 02:19:46.008732 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c622cd6cd49bc4d911093d8feb7dad8a for removal 2024.10.22 02:19:46.010741 [ 1476 ] {} CacheMetadata: Key 674ca8b4b72993c09f267aba6ab1abbb is removed from metadata 2024.10.22 02:19:46.023638 [ 1476 ] {} CacheMetadata: Key c622cd6cd49bc4d911093d8feb7dad8a is removed from metadata 2024.10.22 02:19:46.025294 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5c46639594dd58384cc638f55ab81abd, offset: 0, size: 286 2024.10.22 02:19:46.031664 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5c46639594dd58384cc638f55ab81abd for removal 2024.10.22 02:19:46.036296 [ 1476 ] {} CacheMetadata: Key 5c46639594dd58384cc638f55ab81abd is removed from metadata 2024.10.22 02:19:46.037031 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b25e25398941d0f39d56746a0c823dd3, offset: 0, size: 735763 2024.10.22 02:19:46.042616 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b25e25398941d0f39d56746a0c823dd3 for removal 2024.10.22 02:19:46.046766 [ 1476 ] {} CacheMetadata: Key b25e25398941d0f39d56746a0c823dd3 is removed from metadata 2024.10.22 02:19:46.046647 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c59514a927311cf4024bec70665bd6d4, offset: 0, size: 60829 2024.10.22 02:19:46.067246 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c59514a927311cf4024bec70665bd6d4 for removal 2024.10.22 02:19:46.069672 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9b02d33de6caaeb97180b0a8fc691a38, offset: 0, size: 83147 2024.10.22 02:19:46.069783 [ 1476 ] {} CacheMetadata: Key c59514a927311cf4024bec70665bd6d4 is removed from metadata 2024.10.22 02:19:46.158541 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:19:46.166450 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 96562 2024.10.22 02:19:46.249283 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_850_850_0 to 202410_857_857_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:46.340489 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9b02d33de6caaeb97180b0a8fc691a38 for removal 2024.10.22 02:19:46.344573 [ 1476 ] {} CacheMetadata: Key 9b02d33de6caaeb97180b0a8fc691a38 is removed from metadata 2024.10.22 02:19:46.345026 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 54835d5d73505c37618985aed8638026, offset: 0, size: 458220 2024.10.22 02:19:46.350245 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 54835d5d73505c37618985aed8638026 for removal 2024.10.22 02:19:46.352305 [ 1476 ] {} CacheMetadata: Key 54835d5d73505c37618985aed8638026 is removed from metadata 2024.10.22 02:19:46.352935 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b7a09612667d04d953baa3006633db97, offset: 0, size: 2235761 2024.10.22 02:19:46.356566 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b7a09612667d04d953baa3006633db97 for removal 2024.10.22 02:19:46.359087 [ 1476 ] {} CacheMetadata: Key b7a09612667d04d953baa3006633db97 is removed from metadata 2024.10.22 02:19:46.359156 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0021e93e4428e93d5ae5dd63726361a6, offset: 0, size: 167 2024.10.22 02:19:46.362483 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0021e93e4428e93d5ae5dd63726361a6 for removal 2024.10.22 02:19:46.364486 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 27b7c0aee0b1ad9dab7c9088aa2a75c8, offset: 0, size: 34151 2024.10.22 02:19:46.365228 [ 1476 ] {} CacheMetadata: Key 0021e93e4428e93d5ae5dd63726361a6 is removed from metadata 2024.10.22 02:19:46.368513 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 27b7c0aee0b1ad9dab7c9088aa2a75c8 for removal 2024.10.22 02:19:46.369586 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1286389 2024.10.22 02:19:46.372305 [ 1476 ] {} CacheMetadata: Key 27b7c0aee0b1ad9dab7c9088aa2a75c8 is removed from metadata 2024.10.22 02:19:46.372374 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 38606bf93174d6c2873edc44b475de14, offset: 0, size: 236 2024.10.22 02:19:46.375002 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 38606bf93174d6c2873edc44b475de14 for removal 2024.10.22 02:19:46.378076 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9880a02fd58c21ac417a1dfba00e0d57, offset: 0, size: 183 2024.10.22 02:19:46.378455 [ 1476 ] {} CacheMetadata: Key 38606bf93174d6c2873edc44b475de14 is removed from metadata 2024.10.22 02:19:46.380794 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9880a02fd58c21ac417a1dfba00e0d57 for removal 2024.10.22 02:19:46.384993 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 24cc9f7065a9b9ed9181ef2cfd72a8da, offset: 0, size: 171 2024.10.22 02:19:46.395132 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 24cc9f7065a9b9ed9181ef2cfd72a8da for removal 2024.10.22 02:19:46.395620 [ 1476 ] {} CacheMetadata: Key 9880a02fd58c21ac417a1dfba00e0d57 is removed from metadata 2024.10.22 02:19:46.397141 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0ce861953d23e8f93240b6b33c7cd457, offset: 0, size: 151 2024.10.22 02:19:46.399412 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0ce861953d23e8f93240b6b33c7cd457 for removal 2024.10.22 02:19:46.400189 [ 1476 ] {} CacheMetadata: Key 24cc9f7065a9b9ed9181ef2cfd72a8da is removed from metadata 2024.10.22 02:19:46.401034 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: cf367c5318829979b5f420c6417480f4, offset: 0, size: 2303665 2024.10.22 02:19:46.405526 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key cf367c5318829979b5f420c6417480f4 for removal 2024.10.22 02:19:46.407153 [ 1476 ] {} CacheMetadata: Key 0ce861953d23e8f93240b6b33c7cd457 is removed from metadata 2024.10.22 02:19:46.410868 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 62bd81f01b1ab9e30375dd838f08314a, offset: 0, size: 874103 2024.10.22 02:19:46.418366 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 62bd81f01b1ab9e30375dd838f08314a for removal 2024.10.22 02:19:46.425375 [ 1476 ] {} CacheMetadata: Key cf367c5318829979b5f420c6417480f4 is removed from metadata 2024.10.22 02:19:46.426594 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d50055706e8231f436584cf2653e8539, offset: 0, size: 265 2024.10.22 02:19:46.437255 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d50055706e8231f436584cf2653e8539 for removal 2024.10.22 02:19:46.442861 [ 1476 ] {} CacheMetadata: Key 62bd81f01b1ab9e30375dd838f08314a is removed from metadata 2024.10.22 02:19:46.454753 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 5596c47ff37100e0415dd93fab1557a3, offset: 0, size: 229250 2024.10.22 02:19:46.458894 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 5596c47ff37100e0415dd93fab1557a3 for removal 2024.10.22 02:19:46.467808 [ 1476 ] {} CacheMetadata: Key 5596c47ff37100e0415dd93fab1557a3 is removed from metadata 2024.10.22 02:19:46.470305 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7b0cc464ddfddf3827759d8df65b07aa, offset: 0, size: 231 2024.10.22 02:19:46.473069 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 7b0cc464ddfddf3827759d8df65b07aa for removal 2024.10.22 02:19:46.476492 [ 1476 ] {} CacheMetadata: Key 7b0cc464ddfddf3827759d8df65b07aa is removed from metadata 2024.10.22 02:19:46.477567 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: eb65cc2fca486fbae5c1913fc0fd2adc, offset: 0, size: 168 2024.10.22 02:19:46.491880 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key eb65cc2fca486fbae5c1913fc0fd2adc for removal 2024.10.22 02:19:46.496019 [ 1476 ] {} CacheMetadata: Key eb65cc2fca486fbae5c1913fc0fd2adc is removed from metadata 2024.10.22 02:19:46.496235 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 98c2aeaa54093db6ebd5ae4fe6a9262f, offset: 0, size: 196 2024.10.22 02:19:46.502693 [ 1476 ] {} CacheMetadata: Key d50055706e8231f436584cf2653e8539 is removed from metadata 2024.10.22 02:19:46.503502 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 98c2aeaa54093db6ebd5ae4fe6a9262f for removal 2024.10.22 02:19:46.510264 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 60f1a2cbede89ac6f035cecaf6841084, offset: 0, size: 254 2024.10.22 02:19:46.514339 [ 1476 ] {} CacheMetadata: Key 98c2aeaa54093db6ebd5ae4fe6a9262f is removed from metadata 2024.10.22 02:19:46.516758 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 60f1a2cbede89ac6f035cecaf6841084 for removal 2024.10.22 02:19:46.523411 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8fb8d6d16e76a659fb3155fd829e77a0, offset: 0, size: 217 2024.10.22 02:19:46.528260 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8fb8d6d16e76a659fb3155fd829e77a0 for removal 2024.10.22 02:19:46.541674 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:46.542813 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 04372cfa48e592bb6871f48ad67f7cf5, offset: 0, size: 212 2024.10.22 02:19:46.546404 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.54 GiB. 2024.10.22 02:19:46.546625 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 04372cfa48e592bb6871f48ad67f7cf5 for removal 2024.10.22 02:19:46.552201 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 20b73be6196120eea51c804b1db016b9, offset: 0, size: 24865 2024.10.22 02:19:46.555647 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 20b73be6196120eea51c804b1db016b9 for removal 2024.10.22 02:19:46.600816 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_873_873_0 (state Temporary) 2024.10.22 02:19:46.633573 [ 1476 ] {} CacheMetadata: Key 20b73be6196120eea51c804b1db016b9 is removed from metadata 2024.10.22 02:19:46.648329 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_873_873_0 to 202410_879_879_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:46.668764 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_178_178_0 (state Temporary) 2024.10.22 02:19:46.745253 [ 1476 ] {} CacheMetadata: Key 04372cfa48e592bb6871f48ad67f7cf5 is removed from metadata 2024.10.22 02:19:46.751659 [ 1476 ] {} CacheMetadata: Key 8fb8d6d16e76a659fb3155fd829e77a0 is removed from metadata 2024.10.22 02:19:46.755241 [ 1476 ] {} CacheMetadata: Key 60f1a2cbede89ac6f035cecaf6841084 is removed from metadata 2024.10.22 02:19:46.853082 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 96562 2024.10.22 02:19:46.993465 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_897_262} MergeTask::MergeProjectionsStage: Merge sorted 13597 rows, containing 12 columns (12 merged, 0 gathered) in 1.605309789 sec., 8470.016250551873 rows/sec., 1.32 MiB/sec. 2024.10.22 02:19:47.061403 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_897_262} MergedBlockOutputStream: filled checksums 202410_1_897_262 (state Temporary) 2024.10.22 02:19:47.532085 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 10 file segments (while reserving 1048576 bytes for cf8d2e671b0c35b060711501aa206ac3:0) 2024.10.22 02:19:47.537177 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8e23270d5db5cf82cc069001dfc9dcbb, offset: 0, size: 400246 2024.10.22 02:19:47.544727 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 8e23270d5db5cf82cc069001dfc9dcbb for removal 2024.10.22 02:19:47.547776 [ 1476 ] {} CacheMetadata: Key 8e23270d5db5cf82cc069001dfc9dcbb is removed from metadata 2024.10.22 02:19:47.548331 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: ea3c0aeb57d80a06f1b337270b764808, offset: 0, size: 182174 2024.10.22 02:19:47.552523 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key ea3c0aeb57d80a06f1b337270b764808 for removal 2024.10.22 02:19:47.558302 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 561ff0eb13a6a929bb3acb5597dc326c, offset: 0, size: 153 2024.10.22 02:19:47.563319 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 561ff0eb13a6a929bb3acb5597dc326c for removal 2024.10.22 02:19:47.566254 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a7e9d42f54031ce0acb3d2abe856cee2, offset: 0, size: 278175 2024.10.22 02:19:47.567172 [ 1476 ] {} CacheMetadata: Key 561ff0eb13a6a929bb3acb5597dc326c is removed from metadata 2024.10.22 02:19:47.569353 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a7e9d42f54031ce0acb3d2abe856cee2 for removal 2024.10.22 02:19:47.571278 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: be720102fb93cc90f99318baa60419f3, offset: 0, size: 34628 2024.10.22 02:19:47.571592 [ 1476 ] {} CacheMetadata: Key a7e9d42f54031ce0acb3d2abe856cee2 is removed from metadata 2024.10.22 02:19:47.576498 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key be720102fb93cc90f99318baa60419f3 for removal 2024.10.22 02:19:47.586814 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9ae41bc086106d15e43fc1a29f5a7b87, offset: 0, size: 151 2024.10.22 02:19:47.590993 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 9ae41bc086106d15e43fc1a29f5a7b87 for removal 2024.10.22 02:19:47.583979 [ 1476 ] {} CacheMetadata: Key be720102fb93cc90f99318baa60419f3 is removed from metadata 2024.10.22 02:19:47.595862 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 68acc37f91240bc92c71506e732978e6, offset: 0, size: 46330 2024.10.22 02:19:47.599533 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 68acc37f91240bc92c71506e732978e6 for removal 2024.10.22 02:19:47.607017 [ 1476 ] {} CacheMetadata: Key 68acc37f91240bc92c71506e732978e6 is removed from metadata 2024.10.22 02:19:47.608994 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 890852646fc515815c6352485a0d3cfe, offset: 0, size: 147 2024.10.22 02:19:47.616066 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 890852646fc515815c6352485a0d3cfe for removal 2024.10.22 02:19:47.621017 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 71b3c1748055db4e35dabcb1515e4427, offset: 0, size: 187 2024.10.22 02:19:47.621307 [ 1476 ] {} CacheMetadata: Key 890852646fc515815c6352485a0d3cfe is removed from metadata 2024.10.22 02:19:47.628426 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 71b3c1748055db4e35dabcb1515e4427 for removal 2024.10.22 02:19:47.639553 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3af0f99faf8fc3c596ffe046c827501f, offset: 0, size: 164 2024.10.22 02:19:47.647088 [ 1476 ] {} CacheMetadata: Key 71b3c1748055db4e35dabcb1515e4427 is removed from metadata 2024.10.22 02:19:47.647589 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 3af0f99faf8fc3c596ffe046c827501f for removal 2024.10.22 02:19:47.654203 [ 1476 ] {} CacheMetadata: Key 9ae41bc086106d15e43fc1a29f5a7b87 is removed from metadata 2024.10.22 02:19:47.674196 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_897_262} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_897_262 to 202410_1_897_262 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:47.678819 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_897_262} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_896_261, 202410_897_897_0] -> 202410_1_897_262 2024.10.22 02:19:47.684640 [ 1476 ] {} CacheMetadata: Key 3af0f99faf8fc3c596ffe046c827501f is removed from metadata 2024.10.22 02:19:47.700494 [ 1476 ] {} CacheMetadata: Key ea3c0aeb57d80a06f1b337270b764808 is removed from metadata 2024.10.22 02:19:47.723055 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.59 MiB. 2024.10.22 02:19:48.534446 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 30 file segments (while reserving 1048576 bytes for ed145d7d6f08c6c17bb8ba277bf3fe4b:0) 2024.10.22 02:19:48.538799 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b9acc0886abd520e8dbf6488da9a46b9, offset: 0, size: 131924 2024.10.22 02:19:48.553549 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b9acc0886abd520e8dbf6488da9a46b9 for removal 2024.10.22 02:19:48.555811 [ 1476 ] {} CacheMetadata: Key b9acc0886abd520e8dbf6488da9a46b9 is removed from metadata 2024.10.22 02:19:48.556325 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 96a02a94affab88f366e245bef6b0c2d, offset: 0, size: 160 2024.10.22 02:19:48.559841 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 96a02a94affab88f366e245bef6b0c2d for removal 2024.10.22 02:19:48.561730 [ 1476 ] {} CacheMetadata: Key 96a02a94affab88f366e245bef6b0c2d is removed from metadata 2024.10.22 02:19:48.562907 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1d70276be3b400d75f7b16e029683171, offset: 0, size: 159 2024.10.22 02:19:48.564765 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1d70276be3b400d75f7b16e029683171 for removal 2024.10.22 02:19:48.566713 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 291b1d7fd11d8363b1cb41c9f35a6f2f, offset: 0, size: 152 2024.10.22 02:19:48.569142 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 291b1d7fd11d8363b1cb41c9f35a6f2f for removal 2024.10.22 02:19:48.571548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a377e020ff6a28f0bd4f35f6a08cfd00, offset: 0, size: 34011 2024.10.22 02:19:48.575237 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a377e020ff6a28f0bd4f35f6a08cfd00 for removal 2024.10.22 02:19:48.578350 [ 1476 ] {} CacheMetadata: Key 291b1d7fd11d8363b1cb41c9f35a6f2f is removed from metadata 2024.10.22 02:19:48.579311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0b0ac836d17d9229cbb95f3ff4357330, offset: 0, size: 2368 2024.10.22 02:19:48.582968 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0b0ac836d17d9229cbb95f3ff4357330 for removal 2024.10.22 02:19:48.589722 [ 1476 ] {} CacheMetadata: Key 0b0ac836d17d9229cbb95f3ff4357330 is removed from metadata 2024.10.22 02:19:48.589518 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 17987e74aa1b467b09813179a40ae921, offset: 0, size: 159 2024.10.22 02:19:48.600696 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 17987e74aa1b467b09813179a40ae921 for removal 2024.10.22 02:19:48.611971 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 73eb015a4969bbf74cf3bf795a80b89d, offset: 0, size: 159 2024.10.22 02:19:48.612991 [ 1476 ] {} CacheMetadata: Key 17987e74aa1b467b09813179a40ae921 is removed from metadata 2024.10.22 02:19:48.617827 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 73eb015a4969bbf74cf3bf795a80b89d for removal 2024.10.22 02:19:48.625104 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 73547b926b05fb5f39236dd9dabbc7da, offset: 0, size: 122449 2024.10.22 02:19:48.625339 [ 1476 ] {} CacheMetadata: Key 73eb015a4969bbf74cf3bf795a80b89d is removed from metadata 2024.10.22 02:19:48.633048 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 73547b926b05fb5f39236dd9dabbc7da for removal 2024.10.22 02:19:48.642757 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: abc6087d78a79212bc9c1702571a9078, offset: 0, size: 159 2024.10.22 02:19:48.647214 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key abc6087d78a79212bc9c1702571a9078 for removal 2024.10.22 02:19:48.650629 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 53d6506d651c6d86360b356aa3d72c19, offset: 0, size: 35337 2024.10.22 02:19:48.654998 [ 1476 ] {} CacheMetadata: Key abc6087d78a79212bc9c1702571a9078 is removed from metadata 2024.10.22 02:19:48.657203 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 53d6506d651c6d86360b356aa3d72c19 for removal 2024.10.22 02:19:48.660211 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 38e76d2512369a4eb452db35db06a7a6, offset: 0, size: 31183 2024.10.22 02:19:48.665231 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 38e76d2512369a4eb452db35db06a7a6 for removal 2024.10.22 02:19:48.665608 [ 1476 ] {} CacheMetadata: Key 53d6506d651c6d86360b356aa3d72c19 is removed from metadata 2024.10.22 02:19:48.669111 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: da1868e209627204d454ee0532aa567b, offset: 0, size: 58857 2024.10.22 02:19:48.671199 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key da1868e209627204d454ee0532aa567b for removal 2024.10.22 02:19:48.674285 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6e73ca22bac6dce5df9d0ea09ac81b21, offset: 0, size: 159 2024.10.22 02:19:48.677212 [ 1476 ] {} CacheMetadata: Key da1868e209627204d454ee0532aa567b is removed from metadata 2024.10.22 02:19:48.680241 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6e73ca22bac6dce5df9d0ea09ac81b21 for removal 2024.10.22 02:19:48.683343 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c6b5dc6b9897338a9ce512145848ad40, offset: 0, size: 61072 2024.10.22 02:19:48.691042 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c6b5dc6b9897338a9ce512145848ad40 for removal 2024.10.22 02:19:48.697000 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8730abc7444b9874067b6b77dc7e43ed, offset: 0, size: 123107 2024.10.22 02:19:48.701524 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8730abc7444b9874067b6b77dc7e43ed for removal 2024.10.22 02:19:48.708730 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: da3d522d4b12cc42e29abd153155640a, offset: 0, size: 3252 2024.10.22 02:19:48.715760 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key da3d522d4b12cc42e29abd153155640a for removal 2024.10.22 02:19:48.722717 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f4facf13b8340cda2672eec7d941f91d, offset: 0, size: 73091 2024.10.22 02:19:48.728206 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f4facf13b8340cda2672eec7d941f91d for removal 2024.10.22 02:19:48.732945 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a8c13792467f76723ee92db6ba4d67ab, offset: 0, size: 165 2024.10.22 02:19:48.739760 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a8c13792467f76723ee92db6ba4d67ab for removal 2024.10.22 02:19:48.747296 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9a073e80d43beb3dd030195251fd8fe0, offset: 0, size: 159 2024.10.22 02:19:48.750970 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9a073e80d43beb3dd030195251fd8fe0 for removal 2024.10.22 02:19:48.764566 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7101286de23025f3078161bb969f454a, offset: 0, size: 210113 2024.10.22 02:19:48.768402 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7101286de23025f3078161bb969f454a for removal 2024.10.22 02:19:48.771505 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f741b8e14f0008deea49ae622204f887, offset: 0, size: 127252 2024.10.22 02:19:48.774664 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f741b8e14f0008deea49ae622204f887 for removal 2024.10.22 02:19:48.777688 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9278205860f5faa0dd7d3c789e6551b7, offset: 0, size: 165 2024.10.22 02:19:48.783650 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9278205860f5faa0dd7d3c789e6551b7 for removal 2024.10.22 02:19:48.793345 [ 1476 ] {} CacheMetadata: Key 6e73ca22bac6dce5df9d0ea09ac81b21 is removed from metadata 2024.10.22 02:19:48.796353 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4351ad87822ba2a3318898be08057121, offset: 0, size: 159 2024.10.22 02:19:48.802942 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4351ad87822ba2a3318898be08057121 for removal 2024.10.22 02:19:48.804790 [ 1476 ] {} CacheMetadata: Key 9278205860f5faa0dd7d3c789e6551b7 is removed from metadata 2024.10.22 02:19:48.808153 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 49cc455e6ad063e03cad63bb2a5755a1, offset: 0, size: 165 2024.10.22 02:19:48.811294 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 49cc455e6ad063e03cad63bb2a5755a1 for removal 2024.10.22 02:19:48.814219 [ 1476 ] {} CacheMetadata: Key 4351ad87822ba2a3318898be08057121 is removed from metadata 2024.10.22 02:19:48.814698 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8e8552845365ae4bd69c96fc3b3b8b7e, offset: 0, size: 160 2024.10.22 02:19:48.817557 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8e8552845365ae4bd69c96fc3b3b8b7e for removal 2024.10.22 02:19:48.833577 [ 1476 ] {} CacheMetadata: Key 49cc455e6ad063e03cad63bb2a5755a1 is removed from metadata 2024.10.22 02:19:48.942380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7e55c64036d7d244c1a2b8ebecb2641c, offset: 0, size: 82112 2024.10.22 02:19:48.946865 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7e55c64036d7d244c1a2b8ebecb2641c for removal 2024.10.22 02:19:48.949699 [ 1476 ] {} CacheMetadata: Key 7e55c64036d7d244c1a2b8ebecb2641c is removed from metadata 2024.10.22 02:19:48.949302 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 29c2e6ec726a5b88457e704a64a761b4, offset: 0, size: 160 2024.10.22 02:19:48.954850 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 29c2e6ec726a5b88457e704a64a761b4 for removal 2024.10.22 02:19:48.959896 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 14a7c1de3be9ff7e634b47e4fbc885ae, offset: 0, size: 5149 2024.10.22 02:19:48.961795 [ 1476 ] {} CacheMetadata: Key 29c2e6ec726a5b88457e704a64a761b4 is removed from metadata 2024.10.22 02:19:48.966598 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 14a7c1de3be9ff7e634b47e4fbc885ae for removal 2024.10.22 02:19:48.968383 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 225b0efd3efcb61576ef5ef9f1680bf0, offset: 0, size: 159 2024.10.22 02:19:48.970101 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 225b0efd3efcb61576ef5ef9f1680bf0 for removal 2024.10.22 02:19:48.971414 [ 1476 ] {} CacheMetadata: Key 14a7c1de3be9ff7e634b47e4fbc885ae is removed from metadata 2024.10.22 02:19:48.989393 [ 1476 ] {} CacheMetadata: Key 225b0efd3efcb61576ef5ef9f1680bf0 is removed from metadata 2024.10.22 02:19:49.007791 [ 1476 ] {} CacheMetadata: Key 8e8552845365ae4bd69c96fc3b3b8b7e is removed from metadata 2024.10.22 02:19:49.040743 [ 1476 ] {} CacheMetadata: Key f741b8e14f0008deea49ae622204f887 is removed from metadata 2024.10.22 02:19:49.055489 [ 1476 ] {} CacheMetadata: Key 7101286de23025f3078161bb969f454a is removed from metadata 2024.10.22 02:19:49.064142 [ 1476 ] {} CacheMetadata: Key 9a073e80d43beb3dd030195251fd8fe0 is removed from metadata 2024.10.22 02:19:49.103006 [ 1476 ] {} CacheMetadata: Key a8c13792467f76723ee92db6ba4d67ab is removed from metadata 2024.10.22 02:19:49.237397 [ 1172 ] {} DatabaseCatalog: Execution took 333 ms. 2024.10.22 02:19:49.340811 [ 1476 ] {} CacheMetadata: Key f4facf13b8340cda2672eec7d941f91d is removed from metadata 2024.10.22 02:19:49.349308 [ 1476 ] {} CacheMetadata: Key da3d522d4b12cc42e29abd153155640a is removed from metadata 2024.10.22 02:19:49.363166 [ 1476 ] {} CacheMetadata: Key 8730abc7444b9874067b6b77dc7e43ed is removed from metadata 2024.10.22 02:19:49.371547 [ 1476 ] {} CacheMetadata: Key c6b5dc6b9897338a9ce512145848ad40 is removed from metadata 2024.10.22 02:19:49.385210 [ 1476 ] {} CacheMetadata: Key 38e76d2512369a4eb452db35db06a7a6 is removed from metadata 2024.10.22 02:19:49.397424 [ 1476 ] {} CacheMetadata: Key 73547b926b05fb5f39236dd9dabbc7da is removed from metadata 2024.10.22 02:19:49.409019 [ 1476 ] {} CacheMetadata: Key a377e020ff6a28f0bd4f35f6a08cfd00 is removed from metadata 2024.10.22 02:19:49.527134 [ 1476 ] {} CacheMetadata: Key 1d70276be3b400d75f7b16e029683171 is removed from metadata 2024.10.22 02:19:49.548514 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 12.88 MiB, will set to 1.63 GiB (RSS), difference: -11.92 MiB 2024.10.22 02:19:49.753601 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 279 entries to flush up to offset 100000 2024.10.22 02:19:50.177389 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 83054 bytes for dc6c40a9e159068b2a4412b018bd2da9:0) 2024.10.22 02:19:50.183628 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d5782be805b66b3ad8d350f9841e1770, offset: 0, size: 81742 2024.10.22 02:19:50.189185 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d5782be805b66b3ad8d350f9841e1770 for removal 2024.10.22 02:19:50.192742 [ 1476 ] {} CacheMetadata: Key d5782be805b66b3ad8d350f9841e1770 is removed from metadata 2024.10.22 02:19:50.193681 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 16f66f2d4641a3ed33533bbcad06d3f8, offset: 0, size: 165 2024.10.22 02:19:50.197613 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 16f66f2d4641a3ed33533bbcad06d3f8 for removal 2024.10.22 02:19:50.224350 [ 1476 ] {} CacheMetadata: Key 16f66f2d4641a3ed33533bbcad06d3f8 is removed from metadata 2024.10.22 02:19:50.658828 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:50.661055 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:50.872125 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 69591 bytes for 1590af881c8e68ea4b36aac0379ce02e:0) 2024.10.22 02:19:50.873652 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 096ae4f272633e0630aa557bebc126e5, offset: 0, size: 87375 2024.10.22 02:19:50.878889 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 096ae4f272633e0630aa557bebc126e5 for removal 2024.10.22 02:19:50.882608 [ 1476 ] {} CacheMetadata: Key 096ae4f272633e0630aa557bebc126e5 is removed from metadata 2024.10.22 02:19:50.883804 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 04cc360bc9fe9c9c0512472b65e5f3f1, offset: 0, size: 160 2024.10.22 02:19:50.888932 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 04cc360bc9fe9c9c0512472b65e5f3f1 for removal 2024.10.22 02:19:50.902959 [ 1476 ] {} CacheMetadata: Key 04cc360bc9fe9c9c0512472b65e5f3f1 is removed from metadata 2024.10.22 02:19:51.101088 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1497_1497_0 (state Temporary) 2024.10.22 02:19:51.166699 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1497_1497_0 to 202410_1510_1510_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:51.206595 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1505_74 to 202410_1510_1510_0 2024.10.22 02:19:51.226978 [ 1202 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:51.241664 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100000 2024.10.22 02:19:51.239953 [ 1202 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:51.268508 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1505_74 to 202410_1510_1510_0 into Compact with storage Full 2024.10.22 02:19:51.337507 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:19:51.340591 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:19:51.343926 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:19:51.347036 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1505_74, total 20191 rows starting from the beginning of the part 2024.10.22 02:19:51.367071 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_1506_1506_0, total 108 rows starting from the beginning of the part 2024.10.22 02:19:51.444684 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 14.11 MiB, will set to 1.63 GiB (RSS), difference: 1.51 MiB 2024.10.22 02:19:51.468663 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 8 file segments (while reserving 1048576 bytes for cf8d2e671b0c35b060711501aa206ac3:0) 2024.10.22 02:19:51.472998 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a7040f410374dfe02c48a9e8db891d0e, offset: 0, size: 1122316 2024.10.22 02:19:51.481056 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a7040f410374dfe02c48a9e8db891d0e for removal 2024.10.22 02:19:51.488575 [ 1476 ] {} CacheMetadata: Key a7040f410374dfe02c48a9e8db891d0e is removed from metadata 2024.10.22 02:19:51.489493 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 298de8b3a3ca79b7ca4fa2f1a53c0cc5, offset: 0, size: 166 2024.10.22 02:19:51.511874 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_1507_1507_0, total 145 rows starting from the beginning of the part 2024.10.22 02:19:51.596677 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 298de8b3a3ca79b7ca4fa2f1a53c0cc5 for removal 2024.10.22 02:19:51.601039 [ 1476 ] {} CacheMetadata: Key 298de8b3a3ca79b7ca4fa2f1a53c0cc5 is removed from metadata 2024.10.22 02:19:51.600946 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 45547699b3261dc941302a618ef9dd00, offset: 0, size: 166 2024.10.22 02:19:51.610464 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 45547699b3261dc941302a618ef9dd00 for removal 2024.10.22 02:19:51.613145 [ 1476 ] {} CacheMetadata: Key 45547699b3261dc941302a618ef9dd00 is removed from metadata 2024.10.22 02:19:51.647696 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_1508_1508_0, total 88 rows starting from the beginning of the part 2024.10.22 02:19:51.670961 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_1509_1509_0, total 48 rows starting from the beginning of the part 2024.10.22 02:19:51.708294 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_1510_1510_0, total 279 rows starting from the beginning of the part 2024.10.22 02:19:51.715719 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3448c1d87221ec57f67ba96630bfcc5f, offset: 0, size: 165 2024.10.22 02:19:51.718869 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 3448c1d87221ec57f67ba96630bfcc5f for removal 2024.10.22 02:19:51.727034 [ 1476 ] {} CacheMetadata: Key 3448c1d87221ec57f67ba96630bfcc5f is removed from metadata 2024.10.22 02:19:51.727988 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3f88ce557dc445b9276fca9fc1557deb, offset: 0, size: 166 2024.10.22 02:19:51.736299 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 3f88ce557dc445b9276fca9fc1557deb for removal 2024.10.22 02:19:51.740948 [ 1476 ] {} CacheMetadata: Key 3f88ce557dc445b9276fca9fc1557deb is removed from metadata 2024.10.22 02:19:51.742484 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0043e87303056e9c62affb15cad3eb60, offset: 0, size: 176245 2024.10.22 02:19:51.846051 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 0043e87303056e9c62affb15cad3eb60 for removal 2024.10.22 02:19:51.848514 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d690bf7fa6a50c5b3e580d35f2d16d0d, offset: 0, size: 92925 2024.10.22 02:19:51.850029 [ 1476 ] {} CacheMetadata: Key 0043e87303056e9c62affb15cad3eb60 is removed from metadata 2024.10.22 02:19:51.850867 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d690bf7fa6a50c5b3e580d35f2d16d0d for removal 2024.10.22 02:19:51.853053 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 958736780885563df54173344c33e3f5, offset: 0, size: 164223 2024.10.22 02:19:51.854569 [ 1476 ] {} CacheMetadata: Key d690bf7fa6a50c5b3e580d35f2d16d0d is removed from metadata 2024.10.22 02:19:51.854928 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 958736780885563df54173344c33e3f5 for removal 2024.10.22 02:19:51.862808 [ 1476 ] {} CacheMetadata: Key 958736780885563df54173344c33e3f5 is removed from metadata 2024.10.22 02:19:52.881829 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 13562 2024.10.22 02:19:53.183640 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:53.193610 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:53.375554 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1157 entries to flush up to offset 1287546 2024.10.22 02:19:53.450997 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_889_889_0 (state Temporary) 2024.10.22 02:19:53.608132 [ 1199 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:19:53.649445 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_889_889_0 to 202410_902_902_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:53.649819 [ 1199 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:19:53.684647 [ 1163 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_897_262 to 202410_898_898_0 2024.10.22 02:19:53.692115 [ 1163 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:53.698578 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:53.716439 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} MergeTask::MergeProjectionsStage: Merge sorted 20859 rows, containing 13 columns (13 merged, 0 gathered) in 2.452834414 sec., 8504.039196834263 rows/sec., 1.57 MiB/sec. 2024.10.22 02:19:53.891490 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13562 2024.10.22 02:19:53.953520 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_898_263} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_897_262 to 202410_898_898_0 into Compact with storage Full 2024.10.22 02:19:53.981131 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:53.992801 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:54.007252 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} MergedBlockOutputStream: filled checksums 202410_1207_1510_75 (state Temporary) 2024.10.22 02:19:54.218978 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_898_263} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:19:54.241740 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_898_263} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_897_262, total 13597 rows starting from the beginning of the part 2024.10.22 02:19:54.266034 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_898_263} MergeTreeSequentialSource: Reading 2 marks from part 202410_898_898_0, total 12 rows starting from the beginning of the part 2024.10.22 02:19:54.328396 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_851_851_0 (state Temporary) 2024.10.22 02:19:54.381811 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 5 old parts to remove. Parts: [202410_1207_1407_40, 202410_1408_1408_0, 202410_1409_1409_0, 202410_1410_1410_0, 202410_1411_1411_0] 2024.10.22 02:19:54.381827 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1510_75 to 202410_1207_1510_75 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:54.389531 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from filesystem (serially): Parts: [202410_1207_1407_40, 202410_1408_1408_0, 202410_1409_1409_0, 202410_1410_1410_0, 202410_1411_1411_0] 2024.10.22 02:19:54.395562 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1510_75} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1505_74, 202410_1510_1510_0] -> 202410_1207_1510_75 2024.10.22 02:19:54.411555 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_851_851_0 to 202410_858_858_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:54.462364 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 107 entries to flush up to offset 96669 2024.10.22 02:19:54.489026 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 6 file segments (while reserving 212177 bytes for 1df799d85cdd64f2530e8b28ab758046:0) 2024.10.22 02:19:54.496299 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5a440211493d52fe575f3ffacc04fd48, offset: 0, size: 48174 2024.10.22 02:19:54.504391 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5a440211493d52fe575f3ffacc04fd48 for removal 2024.10.22 02:19:54.508804 [ 1476 ] {} CacheMetadata: Key 5a440211493d52fe575f3ffacc04fd48 is removed from metadata 2024.10.22 02:19:54.512056 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8d6a92c5a6890fe1cfebdc86ce29a777, offset: 0, size: 159 2024.10.22 02:19:54.518043 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8d6a92c5a6890fe1cfebdc86ce29a777 for removal 2024.10.22 02:19:54.526620 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from memory: Parts: [202410_1207_1407_40, 202410_1408_1408_0, 202410_1409_1409_0, 202410_1410_1410_0, 202410_1411_1411_0] 2024.10.22 02:19:54.527212 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 54cae4a2e079a4af8ceb0317d3e7b525, offset: 0, size: 89720 2024.10.22 02:19:54.534406 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 54cae4a2e079a4af8ceb0317d3e7b525 for removal 2024.10.22 02:19:54.546403 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b5b95ae87f7fd0915c5f0a5b8d2b4609, offset: 0, size: 149 2024.10.22 02:19:54.554297 [ 1476 ] {} CacheMetadata: Key 54cae4a2e079a4af8ceb0317d3e7b525 is removed from metadata 2024.10.22 02:19:54.554986 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b5b95ae87f7fd0915c5f0a5b8d2b4609 for removal 2024.10.22 02:19:54.557784 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6e1d9cf972b5454950a37cb752fd155f, offset: 0, size: 55431 2024.10.22 02:19:54.563408 [ 1476 ] {} CacheMetadata: Key b5b95ae87f7fd0915c5f0a5b8d2b4609 is removed from metadata 2024.10.22 02:19:54.574806 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6e1d9cf972b5454950a37cb752fd155f for removal 2024.10.22 02:19:54.582815 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a3df42bdabca169840e7d3c8b48fc46d, offset: 0, size: 159 2024.10.22 02:19:54.593106 [ 1476 ] {} CacheMetadata: Key 6e1d9cf972b5454950a37cb752fd155f is removed from metadata 2024.10.22 02:19:54.608433 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1287546 2024.10.22 02:19:54.613421 [ 1476 ] {} CacheMetadata: Key 8d6a92c5a6890fe1cfebdc86ce29a777 is removed from metadata 2024.10.22 02:19:54.634476 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_850_159/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:19:54.645445 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.51 MiB. 2024.10.22 02:19:54.698373 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a3df42bdabca169840e7d3c8b48fc46d for removal 2024.10.22 02:19:54.702284 [ 1476 ] {} CacheMetadata: Key a3df42bdabca169840e7d3c8b48fc46d is removed from metadata 2024.10.22 02:19:54.761428 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:54.764246 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:54.847657 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_874_874_0 (state Temporary) 2024.10.22 02:19:54.892471 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_874_874_0 to 202410_880_880_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:54.966750 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 96669 2024.10.22 02:19:55.059747 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_838_211, 202410_839_839_0] 2024.10.22 02:19:55.061191 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_838_211, 202410_839_839_0] 2024.10.22 02:19:55.071111 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_838_211, 202410_839_839_0] 2024.10.22 02:19:55.102798 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_175_179_1} MergeTask::MergeProjectionsStage: Merge sorted 212 rows, containing 679 columns (679 merged, 0 gathered) in 67.374438283 sec., 3.1465939516929833 rows/sec., 4.71 KiB/sec. 2024.10.22 02:19:55.252140 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_178_178_0 to 202410_181_181_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:55.262365 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_898_263} MergeTask::MergeProjectionsStage: Merge sorted 13609 rows, containing 12 columns (12 merged, 0 gathered) in 1.437319811 sec., 9468.317277650047 rows/sec., 1.48 MiB/sec. 2024.10.22 02:19:55.360935 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 118 entries to flush up to offset 100118 2024.10.22 02:19:55.445798 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_898_263} MergedBlockOutputStream: filled checksums 202410_1_898_263 (state Temporary) 2024.10.22 02:19:55.715097 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_898_263} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_898_263 to 202410_1_898_263 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:55.720165 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_898_263} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_897_262, 202410_898_898_0] -> 202410_1_898_263 2024.10.22 02:19:55.748456 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 155962 bytes for 927dd7d32d02f479ebd01a701eba1e50:0) 2024.10.22 02:19:55.758047 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:19:55.801494 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:55.815524 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.53 GiB. 2024.10.22 02:19:55.851639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c3ff62cfba76eb81eb8cb77ddd35c0c8, offset: 0, size: 135376 2024.10.22 02:19:55.857044 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c3ff62cfba76eb81eb8cb77ddd35c0c8 for removal 2024.10.22 02:19:55.859547 [ 1476 ] {} CacheMetadata: Key c3ff62cfba76eb81eb8cb77ddd35c0c8 is removed from metadata 2024.10.22 02:19:55.860451 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7a5d577ae31aa56c91cee10a5ebb4d15, offset: 0, size: 150 2024.10.22 02:19:55.865042 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7a5d577ae31aa56c91cee10a5ebb4d15 for removal 2024.10.22 02:19:55.873476 [ 1476 ] {} CacheMetadata: Key 7a5d577ae31aa56c91cee10a5ebb4d15 is removed from metadata 2024.10.22 02:19:55.878207 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1498_1498_0 (state Temporary) 2024.10.22 02:19:56.065940 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1498_1498_0 to 202410_1511_1511_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:56.212638 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100118 2024.10.22 02:19:56.315637 [ 1227 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 02:19:56.416165 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 12.07 MiB, will set to 1.63 GiB (RSS), difference: 15.06 MiB 2024.10.22 02:19:56.836633 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 203715 bytes for e12001a5de9b391ce8bc946cc9d3ac8d:0) 2024.10.22 02:19:56.843919 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2c20b46558483e28b7b7b7ed58f6d276, offset: 0, size: 185 2024.10.22 02:19:56.846427 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2c20b46558483e28b7b7b7ed58f6d276 for removal 2024.10.22 02:19:56.850460 [ 1476 ] {} CacheMetadata: Key 2c20b46558483e28b7b7b7ed58f6d276 is removed from metadata 2024.10.22 02:19:56.853380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: debc79067cf394d44207b36e83150889, offset: 0, size: 86351 2024.10.22 02:19:56.856908 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key debc79067cf394d44207b36e83150889 for removal 2024.10.22 02:19:56.860546 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 527cc077086d21bb727a1fe4b8e31139, offset: 0, size: 291845 2024.10.22 02:19:56.863526 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_175_179_1} MergedBlockOutputStream: filled checksums 202410_175_179_1 (state Temporary) 2024.10.22 02:19:56.865936 [ 1476 ] {} CacheMetadata: Key debc79067cf394d44207b36e83150889 is removed from metadata 2024.10.22 02:19:56.868987 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 527cc077086d21bb727a1fe4b8e31139 for removal 2024.10.22 02:19:56.871746 [ 1476 ] {} CacheMetadata: Key 527cc077086d21bb727a1fe4b8e31139 is removed from metadata 2024.10.22 02:19:56.871648 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ff0300bfa8a3f814df39648c67d13587, offset: 0, size: 160 2024.10.22 02:19:56.874783 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ff0300bfa8a3f814df39648c67d13587 for removal 2024.10.22 02:19:56.879312 [ 1476 ] {} CacheMetadata: Key ff0300bfa8a3f814df39648c67d13587 is removed from metadata 2024.10.22 02:19:57.639479 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 738 2024.10.22 02:19:57.924193 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 265631 bytes for 805cb0c1846ee94ab6c222e85b8ba3df:0) 2024.10.22 02:19:57.928568 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: db49f6b5c905ee3a7a75f7971ac92076, offset: 0, size: 990808 2024.10.22 02:19:57.938354 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key db49f6b5c905ee3a7a75f7971ac92076 for removal 2024.10.22 02:19:57.942799 [ 1476 ] {} CacheMetadata: Key db49f6b5c905ee3a7a75f7971ac92076 is removed from metadata 2024.10.22 02:19:57.943307 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6b47dcb783004be067d704def87bba02, offset: 0, size: 165 2024.10.22 02:19:57.950390 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6b47dcb783004be067d704def87bba02 for removal 2024.10.22 02:19:57.952843 [ 1476 ] {} CacheMetadata: Key 6b47dcb783004be067d704def87bba02 is removed from metadata 2024.10.22 02:19:58.069475 [ 1729 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gyd/lticjnmdzxqwkrgrdzqxzoiwwwxpb, size 2860177 2024.10.22 02:19:58.132463 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:19:58.138525 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.52 GiB. 2024.10.22 02:19:58.183299 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_720_720_0 (state Temporary) 2024.10.22 02:19:58.556898 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 02:19:58.615567 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_720_720_0 to 202410_730_730_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:19:58.672268 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 02:19:58.679425 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 02:19:58.703653 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 02:19:58.736299 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 02:19:58.756166 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 14.88 MiB, will set to 1.63 GiB (RSS), difference: 6.17 MiB 2024.10.22 02:19:58.757699 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 02:19:58.856109 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7183 2024.10.22 02:19:58.874805 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ResponseStartTiming 2024.10.22 02:19:58.934930 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 738 2024.10.22 02:19:59.030800 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/osl/bvvqlrxuyjwsmqqxqcvfnhapwveps (LogSeriesLimiter: on interval from 2024-10-22 02:19:39 to 2024-10-22 02:19:59 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:19:59.288193 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 331623 bytes for 445a7b44b660f0e30985c9c35993724c:0) 2024.10.22 02:19:59.392886 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ed145d7d6f08c6c17bb8ba277bf3fe4b, offset: 0, size: 1109144 2024.10.22 02:19:59.396844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ed145d7d6f08c6c17bb8ba277bf3fe4b for removal 2024.10.22 02:19:59.400190 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 57dfbe418641345bc05ffbfb0c5a9438, offset: 0, size: 110276 2024.10.22 02:19:59.400478 [ 1476 ] {} CacheMetadata: Key ed145d7d6f08c6c17bb8ba277bf3fe4b is removed from metadata 2024.10.22 02:19:59.405049 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 57dfbe418641345bc05ffbfb0c5a9438 for removal 2024.10.22 02:19:59.408855 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1772a27b63bab092ef0b86c1c1192988, offset: 0, size: 166 2024.10.22 02:19:59.411429 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1772a27b63bab092ef0b86c1c1192988 for removal 2024.10.22 02:19:59.416157 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bea73c7914d7188759212b8da11cb9f1, offset: 0, size: 160 2024.10.22 02:19:59.419714 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bea73c7914d7188759212b8da11cb9f1 for removal 2024.10.22 02:19:59.447651 [ 1476 ] {} CacheMetadata: Key bea73c7914d7188759212b8da11cb9f1 is removed from metadata 2024.10.22 02:19:59.460519 [ 1476 ] {} CacheMetadata: Key 1772a27b63bab092ef0b86c1c1192988 is removed from metadata 2024.10.22 02:19:59.467994 [ 1476 ] {} CacheMetadata: Key 57dfbe418641345bc05ffbfb0c5a9438 is removed from metadata 2024.10.22 02:19:59.492391 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_817_14, 202410_818_818_0, 202410_819_819_0, 202410_820_820_0, 202410_821_821_0, 202410_822_822_0] 2024.10.22 02:19:59.500509 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_817_14, 202410_818_818_0, 202410_819_819_0, 202410_820_820_0, 202410_821_821_0, 202410_822_822_0] 2024.10.22 02:19:59.547659 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_817_14, 202410_818_818_0, 202410_819_819_0, 202410_820_820_0, 202410_821_821_0, 202410_822_822_0] 2024.10.22 02:19:59.831310 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_175_179_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:20:00.059152 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 16777216 bytes for 156da8192cc7447bcd13b460024d9999:33554432) 2024.10.22 02:20:00.061229 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 156da8192cc7447bcd13b460024d9999, offset: 0, size: 33554432 2024.10.22 02:20:00.229765 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 100196 2024.10.22 02:20:00.359487 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/qmv/uwznpocuflslmsylotpnayfprdcvy, upload id 3d24f1af-6622-4074-b5e2-4850e3c7d2d5 2024.10.22 02:20:00.373325 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:00.375831 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.54 GiB. 2024.10.22 02:20:00.464018 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1499_1499_0 (state Temporary) 2024.10.22 02:20:00.539894 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1499_1499_0 to 202410_1512_1512_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:00.581125 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100196 2024.10.22 02:20:00.921337 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_850_159} MergeTask::MergeProjectionsStage: Merge sorted 1289969 rows, containing 4 columns (4 merged, 0 gathered) in 39.453277653 sec., 32696.11745177556 rows/sec., 490.36 KiB/sec. 2024.10.22 02:20:01.144095 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_850_159} MergedBlockOutputStream: filled checksums 202410_1_850_159 (state Temporary) 2024.10.22 02:20:01.340197 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_797_797_0, 202410_798_798_0, 202410_799_799_0, 202410_800_800_0, 202410_801_801_0, 202410_802_802_0] 2024.10.22 02:20:01.343150 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:20:01.345315 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_797_797_0, 202410_798_798_0, 202410_799_799_0, 202410_800_800_0, 202410_801_801_0, 202410_802_802_0] 2024.10.22 02:20:01.346659 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:20:01.383036 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_797_797_0, 202410_798_798_0, 202410_799_799_0, 202410_800_800_0, 202410_801_801_0, 202410_802_802_0] 2024.10.22 02:20:01.395386 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 13579 2024.10.22 02:20:01.719712 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1288701 2024.10.22 02:20:01.801598 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 14.91 MiB, will set to 1.62 GiB (RSS), difference: -15.24 MiB 2024.10.22 02:20:01.815399 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:01.834242 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.50 GiB. 2024.10.22 02:20:01.897213 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_890_890_0 (state Temporary) 2024.10.22 02:20:02.097373 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_890_890_0 to 202410_903_903_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:02.159340 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13579 2024.10.22 02:20:02.243842 [ 1267 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_898_263 to 202410_899_899_0 2024.10.22 02:20:02.263020 [ 1267 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:02.267030 [ 1267 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 02:20:02.310579 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_899_264} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_898_263 to 202410_899_899_0 into Compact with storage Full 2024.10.22 02:20:02.481494 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 94 entries to flush up to offset 96763 2024.10.22 02:20:02.614246 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_899_264} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:02.631834 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_899_264} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_898_263, total 13609 rows starting from the beginning of the part 2024.10.22 02:20:02.744716 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_899_264} MergeTreeSequentialSource: Reading 2 marks from part 202410_899_899_0, total 9 rows starting from the beginning of the part 2024.10.22 02:20:02.753231 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:02.757505 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 02:20:02.855555 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:02.862365 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 02:20:03.017182 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_852_852_0 (state Temporary) 2024.10.22 02:20:03.102406 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_852_852_0 to 202410_859_859_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:03.109052 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_899_264/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:20:03.119505 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_875_875_0 (state Temporary) 2024.10.22 02:20:03.154057 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1288701 2024.10.22 02:20:03.275391 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_875_875_0 to 202410_881_881_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:03.373630 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_839_212, 202410_840_840_0] 2024.10.22 02:20:03.376669 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_839_212, 202410_840_840_0] 2024.10.22 02:20:03.387896 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_839_212, 202410_840_840_0] 2024.10.22 02:20:03.402587 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_899_264} MergeTask::MergeProjectionsStage: Merge sorted 13618 rows, containing 12 columns (12 merged, 0 gathered) in 1.106666353 sec., 12305.425174519605 rows/sec., 1.92 MiB/sec. 2024.10.22 02:20:03.412009 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 13.32 MiB, will set to 1.58 GiB (RSS), difference: -8.41 MiB 2024.10.22 02:20:03.450019 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 96763 2024.10.22 02:20:03.858205 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_899_264} MergedBlockOutputStream: filled checksums 202410_1_899_264 (state Temporary) 2024.10.22 02:20:04.567035 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_899_264} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_899_264 to 202410_1_899_264 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:04.591878 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 100244 2024.10.22 02:20:04.670637 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_899_264} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_898_263, 202410_899_899_0] -> 202410_1_899_264 2024.10.22 02:20:04.700311 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:20:04.874261 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_850_159} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_850_159 to 202410_1_850_159 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:04.876977 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_850_159} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_844_158, 202410_845_850_1] -> 202410_1_850_159 2024.10.22 02:20:04.909516 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:04.915372 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 02:20:04.922585 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.53 MiB. 2024.10.22 02:20:05.103996 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1500_1500_0 (state Temporary) 2024.10.22 02:20:05.186340 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1500_1500_0 to 202410_1513_1513_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:05.251880 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100244 2024.10.22 02:20:05.496824 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.09 MiB, will set to 1.58 GiB (RSS), difference: 8.81 MiB 2024.10.22 02:20:05.716882 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/uwt/tnnaflmjnvgnadflybmqrcwzknysr. 2024.10.22 02:20:05.758822 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/feo/hqiztsyqrppzqmsqqujxxjcclfvpn. 2024.10.22 02:20:06.368610 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/feo/hqiztsyqrppzqmsqqujxxjcclfvpn. 2024.10.22 02:20:06.375837 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_17_17_0 to 201403_17_17_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:06.398512 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 7225 2024.10.22 02:20:06.411370 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/uwt/tnnaflmjnvgnadflybmqrcwzknysr. 2024.10.22 02:20:06.439853 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 739 2024.10.22 02:20:06.440704 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:20:06.612571 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:06.615411 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 02:20:06.767742 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_721_721_0 (state Temporary) 2024.10.22 02:20:06.895554 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_721_721_0 to 202410_731_731_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:07.079032 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 739 2024.10.22 02:20:07.349709 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.13 GiB, peak 2.00 GiB, free memory in arenas 8.23 MiB, will set to 1.52 GiB (RSS), difference: 393.51 MiB 2024.10.22 02:20:08.652689 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 2.00 GiB, free memory in arenas 7.95 MiB, will set to 1.35 GiB (RSS), difference: -139.08 MiB 2024.10.22 02:20:08.656557 [ 1161 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:20:08.693321 [ 1161 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:20:09.255868 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 29 entries to flush up to offset 100273 2024.10.22 02:20:09.430334 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 31 file segments (while reserving 11962998 bytes for 156da8192cc7447bcd13b460024d9999:33554432) 2024.10.22 02:20:09.432942 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 90e72220a349309c883a0cf0df7c4d91, offset: 0, size: 332251 2024.10.22 02:20:09.437690 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 90e72220a349309c883a0cf0df7c4d91 for removal 2024.10.22 02:20:09.442760 [ 1476 ] {} CacheMetadata: Key 90e72220a349309c883a0cf0df7c4d91 is removed from metadata 2024.10.22 02:20:09.443578 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 34f854cc39280225293dbdf73acf21ae, offset: 0, size: 155 2024.10.22 02:20:09.448755 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 34f854cc39280225293dbdf73acf21ae for removal 2024.10.22 02:20:09.454180 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 390b43acf9d3922608737b9e20990f24, offset: 0, size: 533336 2024.10.22 02:20:09.457363 [ 1476 ] {} CacheMetadata: Key 34f854cc39280225293dbdf73acf21ae is removed from metadata 2024.10.22 02:20:09.458868 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 390b43acf9d3922608737b9e20990f24 for removal 2024.10.22 02:20:09.463351 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: fda8e817d68ce3a83b9f1a81f985206f, offset: 0, size: 159 2024.10.22 02:20:09.464638 [ 1476 ] {} CacheMetadata: Key 390b43acf9d3922608737b9e20990f24 is removed from metadata 2024.10.22 02:20:09.466501 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key fda8e817d68ce3a83b9f1a81f985206f for removal 2024.10.22 02:20:09.468482 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d6af34481a6d56e1b358c6287e23a41f, offset: 0, size: 159 2024.10.22 02:20:09.469193 [ 1476 ] {} CacheMetadata: Key fda8e817d68ce3a83b9f1a81f985206f is removed from metadata 2024.10.22 02:20:09.470732 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d6af34481a6d56e1b358c6287e23a41f for removal 2024.10.22 02:20:09.472851 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 445a7b44b660f0e30985c9c35993724c, offset: 0, size: 331623 2024.10.22 02:20:09.474961 [ 1476 ] {} CacheMetadata: Key d6af34481a6d56e1b358c6287e23a41f is removed from metadata 2024.10.22 02:20:09.476453 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 445a7b44b660f0e30985c9c35993724c for removal 2024.10.22 02:20:09.480102 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1590af881c8e68ea4b36aac0379ce02e, offset: 0, size: 69591 2024.10.22 02:20:09.482846 [ 1476 ] {} CacheMetadata: Key 445a7b44b660f0e30985c9c35993724c is removed from metadata 2024.10.22 02:20:09.484773 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 1590af881c8e68ea4b36aac0379ce02e for removal 2024.10.22 02:20:09.491498 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6cb0ee134251115ba64e4f74424d72b0, offset: 0, size: 153 2024.10.22 02:20:09.493159 [ 1476 ] {} CacheMetadata: Key 1590af881c8e68ea4b36aac0379ce02e is removed from metadata 2024.10.22 02:20:09.495450 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6cb0ee134251115ba64e4f74424d72b0 for removal 2024.10.22 02:20:09.500042 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e12001a5de9b391ce8bc946cc9d3ac8d, offset: 0, size: 203715 2024.10.22 02:20:09.500264 [ 1476 ] {} CacheMetadata: Key 6cb0ee134251115ba64e4f74424d72b0 is removed from metadata 2024.10.22 02:20:09.502546 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e12001a5de9b391ce8bc946cc9d3ac8d for removal 2024.10.22 02:20:09.507374 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9d7b009975fd2fc19c3c1a9b2db8cd8e, offset: 0, size: 152 2024.10.22 02:20:09.508727 [ 1476 ] {} CacheMetadata: Key e12001a5de9b391ce8bc946cc9d3ac8d is removed from metadata 2024.10.22 02:20:09.512059 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9d7b009975fd2fc19c3c1a9b2db8cd8e for removal 2024.10.22 02:20:09.514643 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 96183840a24b33ff10e5eee0426f7c78, offset: 0, size: 155 2024.10.22 02:20:09.517395 [ 1476 ] {} CacheMetadata: Key 9d7b009975fd2fc19c3c1a9b2db8cd8e is removed from metadata 2024.10.22 02:20:09.517685 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 96183840a24b33ff10e5eee0426f7c78 for removal 2024.10.22 02:20:09.531916 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 1df799d85cdd64f2530e8b28ab758046, offset: 0, size: 212177 2024.10.22 02:20:09.541147 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 1df799d85cdd64f2530e8b28ab758046 for removal 2024.10.22 02:20:09.545047 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 621c006956b82bfd824133407fe8ffa9, offset: 0, size: 147 2024.10.22 02:20:09.548229 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 621c006956b82bfd824133407fe8ffa9 for removal 2024.10.22 02:20:09.552206 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 805cb0c1846ee94ab6c222e85b8ba3df, offset: 0, size: 265631 2024.10.22 02:20:09.557459 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 805cb0c1846ee94ab6c222e85b8ba3df for removal 2024.10.22 02:20:09.636307 [ 1476 ] {} CacheMetadata: Key 621c006956b82bfd824133407fe8ffa9 is removed from metadata 2024.10.22 02:20:09.660160 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: da5d09da8cc119631a4e9a535e3de678, offset: 0, size: 152 2024.10.22 02:20:09.662633 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key da5d09da8cc119631a4e9a535e3de678 for removal 2024.10.22 02:20:09.663529 [ 1476 ] {} CacheMetadata: Key 805cb0c1846ee94ab6c222e85b8ba3df is removed from metadata 2024.10.22 02:20:09.664784 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 13591 2024.10.22 02:20:09.666313 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ce3563d28783087cde242b3efe61f7c5, offset: 0, size: 166 2024.10.22 02:20:09.668715 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ce3563d28783087cde242b3efe61f7c5 for removal 2024.10.22 02:20:09.669181 [ 1476 ] {} CacheMetadata: Key da5d09da8cc119631a4e9a535e3de678 is removed from metadata 2024.10.22 02:20:09.671492 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: cb69ea0be56fb76126073e3be77fbfcd, offset: 0, size: 165 2024.10.22 02:20:09.674824 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key cb69ea0be56fb76126073e3be77fbfcd for removal 2024.10.22 02:20:09.678599 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:09.679844 [ 1476 ] {} CacheMetadata: Key ce3563d28783087cde242b3efe61f7c5 is removed from metadata 2024.10.22 02:20:09.681288 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 02:20:09.682527 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 59028abc7a3f80947125f76a7f959986, offset: 0, size: 154 2024.10.22 02:20:09.691047 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 59028abc7a3f80947125f76a7f959986 for removal 2024.10.22 02:20:09.698690 [ 1476 ] {} CacheMetadata: Key 59028abc7a3f80947125f76a7f959986 is removed from metadata 2024.10.22 02:20:09.699870 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8fc363fce4dd4750760ee72847f8581b, offset: 0, size: 44558 2024.10.22 02:20:09.704592 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8fc363fce4dd4750760ee72847f8581b for removal 2024.10.22 02:20:09.706386 [ 1476 ] {} CacheMetadata: Key cb69ea0be56fb76126073e3be77fbfcd is removed from metadata 2024.10.22 02:20:09.711816 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f71dbbc711f622056c2bed263f2a5611, offset: 0, size: 262471 2024.10.22 02:20:09.723524 [ 1476 ] {} CacheMetadata: Key 8fc363fce4dd4750760ee72847f8581b is removed from metadata 2024.10.22 02:20:09.729381 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f71dbbc711f622056c2bed263f2a5611 for removal 2024.10.22 02:20:09.736123 [ 1476 ] {} CacheMetadata: Key f71dbbc711f622056c2bed263f2a5611 is removed from metadata 2024.10.22 02:20:09.751759 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1501_1501_0 (state Temporary) 2024.10.22 02:20:09.755017 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:09.755851 [ 1476 ] {} CacheMetadata: Key 1df799d85cdd64f2530e8b28ab758046 is removed from metadata 2024.10.22 02:20:09.760691 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.44 GiB. 2024.10.22 02:20:09.768456 [ 1476 ] {} CacheMetadata: Key 96183840a24b33ff10e5eee0426f7c78 is removed from metadata 2024.10.22 02:20:09.801096 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1501_1501_0 to 202410_1514_1514_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:09.839103 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0c9de76cafd751db563bc017919d95d8, offset: 0, size: 4 2024.10.22 02:20:09.861439 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100273 2024.10.22 02:20:09.947130 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0c9de76cafd751db563bc017919d95d8 for removal 2024.10.22 02:20:09.951830 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0ae1915ff793d9089003df2fe24e9e5e, offset: 0, size: 69838 2024.10.22 02:20:09.951923 [ 1476 ] {} CacheMetadata: Key 0c9de76cafd751db563bc017919d95d8 is removed from metadata 2024.10.22 02:20:09.954596 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0ae1915ff793d9089003df2fe24e9e5e for removal 2024.10.22 02:20:09.956134 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ed49455c6e097ac0bc6ba1fc148675c0, offset: 0, size: 159 2024.10.22 02:20:09.956527 [ 1476 ] {} CacheMetadata: Key 0ae1915ff793d9089003df2fe24e9e5e is removed from metadata 2024.10.22 02:20:09.959326 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ed49455c6e097ac0bc6ba1fc148675c0 for removal 2024.10.22 02:20:09.962210 [ 1476 ] {} CacheMetadata: Key ed49455c6e097ac0bc6ba1fc148675c0 is removed from metadata 2024.10.22 02:20:09.962315 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 09b2ed8d3548451b91568fe4b744e494, offset: 0, size: 538890 2024.10.22 02:20:09.968063 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 09b2ed8d3548451b91568fe4b744e494 for removal 2024.10.22 02:20:09.971512 [ 1476 ] {} CacheMetadata: Key 09b2ed8d3548451b91568fe4b744e494 is removed from metadata 2024.10.22 02:20:09.971878 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: dc6c40a9e159068b2a4412b018bd2da9, offset: 0, size: 83054 2024.10.22 02:20:09.975542 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key dc6c40a9e159068b2a4412b018bd2da9 for removal 2024.10.22 02:20:09.979054 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: efe9a98fd9203b52e6cc322aefc15f86, offset: 0, size: 159 2024.10.22 02:20:09.980646 [ 1476 ] {} CacheMetadata: Key dc6c40a9e159068b2a4412b018bd2da9 is removed from metadata 2024.10.22 02:20:09.981803 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key efe9a98fd9203b52e6cc322aefc15f86 for removal 2024.10.22 02:20:09.987422 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 8de8192e36b6536a6b1b8261aa32a3f1, offset: 0, size: 97460 2024.10.22 02:20:09.991743 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 8de8192e36b6536a6b1b8261aa32a3f1 for removal 2024.10.22 02:20:09.995885 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 40fba0cb5be9a22fad4f4be8136f91fa, offset: 0, size: 159 2024.10.22 02:20:09.997526 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_891_891_0 (state Temporary) 2024.10.22 02:20:10.091288 [ 1476 ] {} CacheMetadata: Key 8de8192e36b6536a6b1b8261aa32a3f1 is removed from metadata 2024.10.22 02:20:10.100957 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 40fba0cb5be9a22fad4f4be8136f91fa for removal 2024.10.22 02:20:10.107894 [ 1476 ] {} CacheMetadata: Key efe9a98fd9203b52e6cc322aefc15f86 is removed from metadata 2024.10.22 02:20:10.124072 [ 1476 ] {} CacheMetadata: Key 40fba0cb5be9a22fad4f4be8136f91fa is removed from metadata 2024.10.22 02:20:10.107717 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b366d81ea3c77c60ed251f2d492a58dd, offset: 0, size: 5256 2024.10.22 02:20:10.141304 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b366d81ea3c77c60ed251f2d492a58dd for removal 2024.10.22 02:20:10.144957 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ff6ab88ca8251d1946a4935b054638fc, offset: 0, size: 160 2024.10.22 02:20:10.145239 [ 1476 ] {} CacheMetadata: Key b366d81ea3c77c60ed251f2d492a58dd is removed from metadata 2024.10.22 02:20:10.148379 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ff6ab88ca8251d1946a4935b054638fc for removal 2024.10.22 02:20:10.150425 [ 1476 ] {} CacheMetadata: Key ff6ab88ca8251d1946a4935b054638fc is removed from metadata 2024.10.22 02:20:10.151392 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 927dd7d32d02f479ebd01a701eba1e50, offset: 0, size: 155962 2024.10.22 02:20:10.153519 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 927dd7d32d02f479ebd01a701eba1e50 for removal 2024.10.22 02:20:10.154745 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_891_891_0 to 202410_904_904_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:10.155035 [ 1476 ] {} CacheMetadata: Key 927dd7d32d02f479ebd01a701eba1e50 is removed from metadata 2024.10.22 02:20:10.160756 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1290241 2024.10.22 02:20:10.167785 [ 1173 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_899_264 to 202410_900_900_0 2024.10.22 02:20:10.175134 [ 1173 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:10.183596 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13591 2024.10.22 02:20:10.240053 [ 1173 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.43 GiB. 2024.10.22 02:20:10.315748 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_900_265} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_899_264 to 202410_900_900_0 into Compact with storage Full 2024.10.22 02:20:10.369318 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_840_213, 202410_841_841_0] 2024.10.22 02:20:10.370630 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_840_213, 202410_841_841_0] 2024.10.22 02:20:10.384715 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_840_213, 202410_841_841_0] 2024.10.22 02:20:10.524921 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_900_265} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:10.532922 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column GoodEvent 2024.10.22 02:20:10.541942 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_900_265} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_899_264, total 13618 rows starting from the beginning of the part 2024.10.22 02:20:10.548353 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column GoodEvent 2024.10.22 02:20:10.559023 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column GoodEvent 2024.10.22 02:20:10.565407 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column GoodEvent 2024.10.22 02:20:10.573171 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column GoodEvent 2024.10.22 02:20:10.597762 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_900_265} MergeTreeSequentialSource: Reading 2 marks from part 202410_900_900_0, total 19 rows starting from the beginning of the part 2024.10.22 02:20:10.686717 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column GoodEvent 2024.10.22 02:20:10.746609 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column GoodEvent 2024.10.22 02:20:10.754565 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_678_138, 202410_679_679_0, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0, 202410_683_683_0] 2024.10.22 02:20:10.758748 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_678_138, 202410_679_679_0, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0, 202410_683_683_0] 2024.10.22 02:20:10.768233 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column GoodEvent 2024.10.22 02:20:10.790414 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column GoodEvent 2024.10.22 02:20:10.805596 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_678_138, 202410_679_679_0, 202410_680_680_0, 202410_681_681_0, 202410_682_682_0, 202410_683_683_0] 2024.10.22 02:20:10.864458 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ipg/phzjkrifhhqwckrqzagtqevhvrvla (LogSeriesLimiter: on interval from 2024-10-22 02:19:59 to 2024-10-22 02:20:10 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:20:10.967017 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 78 entries to flush up to offset 96841 2024.10.22 02:20:11.271462 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:11.284079 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 02:20:11.342965 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:20:11.346298 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:20:11.350182 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_876_876_0 (state Temporary) 2024.10.22 02:20:11.382623 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_876_876_0 to 202410_882_882_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:11.389503 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:11.396337 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 02:20:11.416731 [ 1169 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_877_26 to 202410_882_882_0 2024.10.22 02:20:11.455124 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 96841 2024.10.22 02:20:11.456631 [ 1169 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:11.460317 [ 1169 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:11.472298 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_877_26 to 202410_882_882_0 into Compact with storage Full 2024.10.22 02:20:11.544247 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_853_853_0 (state Temporary) 2024.10.22 02:20:11.550331 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:11.557831 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_877_26, total 13008 rows starting from the beginning of the part 2024.10.22 02:20:11.572916 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_878_878_0, total 109 rows starting from the beginning of the part 2024.10.22 02:20:11.620129 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_879_879_0, total 90 rows starting from the beginning of the part 2024.10.22 02:20:11.646627 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_880_880_0, total 107 rows starting from the beginning of the part 2024.10.22 02:20:11.666609 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_881_881_0, total 94 rows starting from the beginning of the part 2024.10.22 02:20:11.721708 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_853_853_0 to 202410_860_860_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:11.751331 [ 1191 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_850_159 to 202410_851_856_1 2024.10.22 02:20:11.756034 [ 1191 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:11.759278 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1290241 2024.10.22 02:20:11.793328 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} MergeTreeSequentialSource: Reading 2 marks from part 202410_882_882_0, total 78 rows starting from the beginning of the part 2024.10.22 02:20:11.860533 [ 1191 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 02:20:11.958217 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_856_160} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_850_159 to 202410_851_856_1 into Wide with storage Full 2024.10.22 02:20:12.052410 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 60 2024.10.22 02:20:12.098734 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_856_160} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:12.104346 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_856_160} MergeTreeSequentialSource: Reading 159 marks from part 202410_1_850_159, total 1289969 rows starting from the beginning of the part 2024.10.22 02:20:12.168027 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_856_160} MergeTreeSequentialSource: Reading 3 marks from part 202410_851_856_1, total 8470 rows starting from the beginning of the part 2024.10.22 02:20:12.219393 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:12.224203 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:12.293218 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_45_45_0 (state Temporary) 2024.10.22 02:20:12.299318 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_900_265} MergeTask::MergeProjectionsStage: Merge sorted 13637 rows, containing 12 columns (12 merged, 0 gathered) in 1.994132462 sec., 6838.562763439934 rows/sec., 1.07 MiB/sec. 2024.10.22 02:20:12.361750 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_900_265} MergedBlockOutputStream: filled checksums 202410_1_900_265 (state Temporary) 2024.10.22 02:20:12.460458 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_175_179_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_175_179_1 to 202410_175_179_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:12.471082 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 2.00 GiB, free memory in arenas 8.21 MiB, will set to 1.40 GiB (RSS), difference: -71.29 MiB 2024.10.22 02:20:12.565775 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_175_179_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 5 parts: [202410_175_175_0, 202410_179_179_0] -> 202410_175_179_1 2024.10.22 02:20:12.678721 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_45_45_0 to 202410_45_45_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:12.786485 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 60 2024.10.22 02:20:13.078331 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_900_265} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_900_265 to 202410_1_900_265 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:13.087344 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_900_265} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_899_264, 202410_900_900_0] -> 202410_1_900_265 2024.10.22 02:20:13.224957 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:20:13.330719 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_882_27/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:20:13.391860 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.75 MiB. 2024.10.22 02:20:13.460537 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} MergeTask::MergeProjectionsStage: Merge sorted 13486 rows, containing 13 columns (13 merged, 0 gathered) in 1.993127687 sec., 6766.249893552354 rows/sec., 2.93 MiB/sec. 2024.10.22 02:20:13.645115 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} MergedBlockOutputStream: filled checksums 202410_747_882_27 (state Temporary) 2024.10.22 02:20:13.672037 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 14 file segments (while reserving 2097152 bytes for 1ab449c26ea4a1444406bdba97ba9959:0) 2024.10.22 02:20:13.674867 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6aae294a146760366552fc4789cdcfed, offset: 0, size: 156 2024.10.22 02:20:13.684240 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 6aae294a146760366552fc4789cdcfed for removal 2024.10.22 02:20:13.688550 [ 1476 ] {} CacheMetadata: Key 6aae294a146760366552fc4789cdcfed is removed from metadata 2024.10.22 02:20:13.688841 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 8ce20dd7b51dae8c4aef8e97b680659c, offset: 0, size: 159719 2024.10.22 02:20:13.698369 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 8ce20dd7b51dae8c4aef8e97b680659c for removal 2024.10.22 02:20:13.701082 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a471f7e6d85023cc83f4d993e503d9e4, offset: 0, size: 190317 2024.10.22 02:20:13.705439 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a471f7e6d85023cc83f4d993e503d9e4 for removal 2024.10.22 02:20:13.716109 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d5e1ec9a0201ea6323e6f794a0d25ba5, offset: 0, size: 150 2024.10.22 02:20:13.721174 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d5e1ec9a0201ea6323e6f794a0d25ba5 for removal 2024.10.22 02:20:13.728898 [ 1476 ] {} CacheMetadata: Key d5e1ec9a0201ea6323e6f794a0d25ba5 is removed from metadata 2024.10.22 02:20:13.730352 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0bc4f7655a67ed1eddff0760a7bbe195, offset: 0, size: 191163 2024.10.22 02:20:13.740962 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 0bc4f7655a67ed1eddff0760a7bbe195 for removal 2024.10.22 02:20:13.744524 [ 1476 ] {} CacheMetadata: Key 0bc4f7655a67ed1eddff0760a7bbe195 is removed from metadata 2024.10.22 02:20:13.744723 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: bfe7d8bf092a652be2b42fa8483b95a4, offset: 0, size: 243043 2024.10.22 02:20:13.749675 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key bfe7d8bf092a652be2b42fa8483b95a4 for removal 2024.10.22 02:20:13.753467 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c2d6e0391bde98f3a345ffe9dbe2ea72, offset: 0, size: 156 2024.10.22 02:20:13.755587 [ 1476 ] {} CacheMetadata: Key bfe7d8bf092a652be2b42fa8483b95a4 is removed from metadata 2024.10.22 02:20:13.756651 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c2d6e0391bde98f3a345ffe9dbe2ea72 for removal 2024.10.22 02:20:13.766422 [ 1476 ] {} CacheMetadata: Key a471f7e6d85023cc83f4d993e503d9e4 is removed from metadata 2024.10.22 02:20:13.759189 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_882_27 to 202410_747_882_27 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:13.769836 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_882_27} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_877_26, 202410_882_882_0] -> 202410_747_882_27 2024.10.22 02:20:13.784551 [ 1476 ] {} CacheMetadata: Key 8ce20dd7b51dae8c4aef8e97b680659c is removed from metadata 2024.10.22 02:20:13.860695 [ 1476 ] {} CacheMetadata: Key c2d6e0391bde98f3a345ffe9dbe2ea72 is removed from metadata 2024.10.22 02:20:13.860950 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7f98533ba4976636bedf6843e861809a, offset: 0, size: 435135 2024.10.22 02:20:13.865694 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 7f98533ba4976636bedf6843e861809a for removal 2024.10.22 02:20:13.866802 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 204 entries to flush up to offset 100477 2024.10.22 02:20:13.869482 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: b4595f82c243f21a7ca7055af4bb6b41, offset: 0, size: 397869 2024.10.22 02:20:13.871707 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key b4595f82c243f21a7ca7055af4bb6b41 for removal 2024.10.22 02:20:13.873454 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4451a03b5398b72e8c06b9d8d9f27f9a, offset: 0, size: 150 2024.10.22 02:20:13.875482 [ 1476 ] {} CacheMetadata: Key b4595f82c243f21a7ca7055af4bb6b41 is removed from metadata 2024.10.22 02:20:13.877339 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 4451a03b5398b72e8c06b9d8d9f27f9a for removal 2024.10.22 02:20:13.880642 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: ae247816aead8d356098668dbd37a916, offset: 0, size: 150 2024.10.22 02:20:13.889406 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key ae247816aead8d356098668dbd37a916 for removal 2024.10.22 02:20:13.894756 [ 1476 ] {} CacheMetadata: Key 4451a03b5398b72e8c06b9d8d9f27f9a is removed from metadata 2024.10.22 02:20:13.897149 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 30a349c83e9032b5125ef60b2c7c52f7, offset: 0, size: 157 2024.10.22 02:20:13.901649 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 30a349c83e9032b5125ef60b2c7c52f7 for removal 2024.10.22 02:20:13.911660 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5f15cba51956c82944d2d7b332b42055, offset: 0, size: 424157 2024.10.22 02:20:13.913166 [ 1476 ] {} CacheMetadata: Key 30a349c83e9032b5125ef60b2c7c52f7 is removed from metadata 2024.10.22 02:20:13.916128 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 5f15cba51956c82944d2d7b332b42055 for removal 2024.10.22 02:20:13.920932 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 15b6e917ef27ff4ad332756c6350781c, offset: 0, size: 152 2024.10.22 02:20:13.921504 [ 1476 ] {} CacheMetadata: Key 5f15cba51956c82944d2d7b332b42055 is removed from metadata 2024.10.22 02:20:13.929663 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 15b6e917ef27ff4ad332756c6350781c for removal 2024.10.22 02:20:13.942531 [ 1476 ] {} CacheMetadata: Key 15b6e917ef27ff4ad332756c6350781c is removed from metadata 2024.10.22 02:20:13.948153 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 17.88 MiB. 2024.10.22 02:20:13.953150 [ 1476 ] {} CacheMetadata: Key ae247816aead8d356098668dbd37a916 is removed from metadata 2024.10.22 02:20:13.961032 [ 1476 ] {} CacheMetadata: Key 7f98533ba4976636bedf6843e861809a is removed from metadata 2024.10.22 02:20:14.512205 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:14.512135 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 2.00 GiB, free memory in arenas 6.88 MiB, will set to 1.43 GiB (RSS), difference: -45.18 MiB 2024.10.22 02:20:14.515449 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:14.580555 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1502_1502_0 (state Temporary) 2024.10.22 02:20:14.584021 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 740 2024.10.22 02:20:14.721698 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1411_41, 202410_1412_1412_0, 202410_1413_1413_0, 202410_1414_1414_0, 202410_1415_1415_0, 202410_1416_1416_0] 2024.10.22 02:20:14.722975 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1502_1502_0 to 202410_1515_1515_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:14.723920 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1411_41, 202410_1412_1412_0, 202410_1413_1413_0, 202410_1414_1414_0, 202410_1415_1415_0, 202410_1416_1416_0] 2024.10.22 02:20:14.755877 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1411_41, 202410_1412_1412_0, 202410_1413_1413_0, 202410_1414_1414_0, 202410_1415_1415_0, 202410_1416_1416_0] 2024.10.22 02:20:14.764564 [ 1220 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1510_75 to 202410_1515_1515_0 2024.10.22 02:20:14.765219 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100477 2024.10.22 02:20:14.774634 [ 1220 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:14.782348 [ 1220 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.41 GiB. 2024.10.22 02:20:14.816678 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1510_75 to 202410_1515_1515_0 into Compact with storage Full 2024.10.22 02:20:14.872765 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:14.892061 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:14.913152 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1510_75, total 20859 rows starting from the beginning of the part 2024.10.22 02:20:14.933149 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:15.007877 [ 1678 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/osl/bvvqlrxuyjwsmqqxqcvfnhapwveps, size 4309865 2024.10.22 02:20:15.100188 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_1511_1511_0, total 118 rows starting from the beginning of the part 2024.10.22 02:20:15.100289 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_722_722_0 (state Temporary) 2024.10.22 02:20:15.262665 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_1512_1512_0, total 78 rows starting from the beginning of the part 2024.10.22 02:20:15.263352 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 02:20:15.276411 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 02:20:15.289206 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_722_722_0 to 202410_732_732_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:15.300506 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 02:20:15.308155 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_1513_1513_0, total 48 rows starting from the beginning of the part 2024.10.22 02:20:15.315980 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 02:20:15.340309 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 02:20:15.354967 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 02:20:15.368207 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 740 2024.10.22 02:20:15.370758 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ResponseEndTiming 2024.10.22 02:20:15.482499 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_1514_1514_0, total 29 rows starting from the beginning of the part 2024.10.22 02:20:15.514411 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/uwm/kqqeaqduxmqmzeezbsttgaxwsaobc (LogSeriesLimiter: on interval from 2024-10-22 02:20:10 to 2024-10-22 02:20:15 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:20:15.536072 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_1515_1515_0, total 204 rows starting from the beginning of the part 2024.10.22 02:20:15.856712 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_856_160/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:20:16.558940 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 7.97 MiB, will set to 1.47 GiB (RSS), difference: -32.58 MiB 2024.10.22 02:20:16.845095 [ 1270 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 335 ms. 2024.10.22 02:20:17.687612 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 13614 2024.10.22 02:20:17.902075 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:17.908404 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:17.961133 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_892_892_0 (state Temporary) 2024.10.22 02:20:18.054331 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_892_892_0 to 202410_905_905_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:18.091448 [ 1239 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_900_265 to 202410_901_901_0 2024.10.22 02:20:18.098155 [ 1239 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:18.101429 [ 1239 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:18.123860 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_901_266} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_900_265 to 202410_901_901_0 into Compact with storage Full 2024.10.22 02:20:18.158138 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_901_266} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:18.164506 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_901_266} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_900_265, total 13637 rows starting from the beginning of the part 2024.10.22 02:20:18.215301 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_901_266} MergeTreeSequentialSource: Reading 2 marks from part 202410_901_901_0, total 23 rows starting from the beginning of the part 2024.10.22 02:20:18.220046 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13614 2024.10.22 02:20:18.576166 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 9.59 MiB, will set to 1.51 GiB (RSS), difference: -64.97 MiB 2024.10.22 02:20:18.673258 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} MergeTask::MergeProjectionsStage: Merge sorted 21336 rows, containing 13 columns (13 merged, 0 gathered) in 3.870769016 sec., 5512.082976743554 rows/sec., 1.01 MiB/sec. 2024.10.22 02:20:18.760416 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} MergedBlockOutputStream: filled checksums 202410_1207_1515_76 (state Temporary) 2024.10.22 02:20:18.764274 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1291781 2024.10.22 02:20:18.861003 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 100552 2024.10.22 02:20:18.944440 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_841_214, 202410_842_842_0] 2024.10.22 02:20:18.947964 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_841_214, 202410_842_842_0] 2024.10.22 02:20:18.957159 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_841_214, 202410_842_842_0] 2024.10.22 02:20:18.961612 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 154 entries to flush up to offset 96995 2024.10.22 02:20:19.016710 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_901_266} MergeTask::MergeProjectionsStage: Merge sorted 13660 rows, containing 12 columns (12 merged, 0 gathered) in 0.902962306 sec., 15127.984755545265 rows/sec., 2.36 MiB/sec. 2024.10.22 02:20:19.066472 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_901_266} MergedBlockOutputStream: filled checksums 202410_1_901_266 (state Temporary) 2024.10.22 02:20:19.095339 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1515_76 to 202410_1207_1515_76 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:19.100064 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1515_76} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1510_75, 202410_1515_1515_0] -> 202410_1207_1515_76 2024.10.22 02:20:19.143799 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_901_266} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_901_266 to 202410_1_901_266 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:19.148496 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_901_266} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_900_265, 202410_901_901_0] -> 202410_1_901_266 2024.10.22 02:20:19.151033 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.68 MiB. 2024.10.22 02:20:19.186545 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:20:19.244180 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:19.247460 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:19.267429 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:19.288625 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:19.306898 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_877_877_0 (state Temporary) 2024.10.22 02:20:19.365715 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1503_1503_0 (state Temporary) 2024.10.22 02:20:19.407985 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1503_1503_0 to 202410_1516_1516_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:19.459122 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_877_877_0 to 202410_883_883_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:19.520693 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1048576 bytes for 6d3d46d6fcc81e532d61fe2677299730:0) 2024.10.22 02:20:19.526989 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9f62aa06842091fae5699d4bfd77b2a5, offset: 0, size: 1149305 2024.10.22 02:20:19.538794 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 9f62aa06842091fae5699d4bfd77b2a5 for removal 2024.10.22 02:20:19.544558 [ 1476 ] {} CacheMetadata: Key 9f62aa06842091fae5699d4bfd77b2a5 is removed from metadata 2024.10.22 02:20:19.544825 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a22d8a4896dc2c059898b4a200cba56b, offset: 0, size: 1332 2024.10.22 02:20:19.548841 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a22d8a4896dc2c059898b4a200cba56b for removal 2024.10.22 02:20:19.552070 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9f5dd7b1c7fbf18c4f905f03053991cb, offset: 0, size: 198 2024.10.22 02:20:19.555307 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 9f5dd7b1c7fbf18c4f905f03053991cb for removal 2024.10.22 02:20:19.557477 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 4133244054de25d4c89cc951c07939f2, offset: 0, size: 159 2024.10.22 02:20:19.561320 [ 1476 ] {} CacheMetadata: Key 9f5dd7b1c7fbf18c4f905f03053991cb is removed from metadata 2024.10.22 02:20:19.561521 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 4133244054de25d4c89cc951c07939f2 for removal 2024.10.22 02:20:19.564529 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 75dc50db004ab40fd0848fe5bb8bccf0, offset: 0, size: 270262 2024.10.22 02:20:19.567019 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100552 2024.10.22 02:20:19.567535 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 75dc50db004ab40fd0848fe5bb8bccf0 for removal 2024.10.22 02:20:19.568953 [ 1476 ] {} CacheMetadata: Key 4133244054de25d4c89cc951c07939f2 is removed from metadata 2024.10.22 02:20:19.569512 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 07366c0abccf5fabf4483ea1f681ad1d, offset: 0, size: 154 2024.10.22 02:20:19.571987 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 07366c0abccf5fabf4483ea1f681ad1d for removal 2024.10.22 02:20:19.575027 [ 1476 ] {} CacheMetadata: Key 75dc50db004ab40fd0848fe5bb8bccf0 is removed from metadata 2024.10.22 02:20:19.593656 [ 1476 ] {} CacheMetadata: Key 07366c0abccf5fabf4483ea1f681ad1d is removed from metadata 2024.10.22 02:20:19.604500 [ 1476 ] {} CacheMetadata: Key a22d8a4896dc2c059898b4a200cba56b is removed from metadata 2024.10.22 02:20:19.648002 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 96995 2024.10.22 02:20:19.674453 [ 1256 ] {} DatabaseCatalog: Execution took 224 ms. 2024.10.22 02:20:19.705416 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:19.712810 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:19.929381 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_854_854_0 (state Temporary) 2024.10.22 02:20:20.056588 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_854_854_0 to 202410_861_861_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:20.217746 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1291781 2024.10.22 02:20:20.360900 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 10.02 MiB, will set to 1.56 GiB (RSS), difference: -21.84 MiB 2024.10.22 02:20:20.459841 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 02:20:21.260996 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 268.60 MiB using storage policy from min volume index 0 2024.10.22 02:20:21.344599 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:20:21.347743 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:20:21.368194 [ 1215 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 351 ms. 2024.10.22 02:20:21.758981 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 10.70 MiB, will set to 1.57 GiB (RSS), difference: 2.24 MiB 2024.10.22 02:20:21.974513 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mpm/imbvjpiaiyhuianzrduqwcrdtvgta (LogSeriesLimiter: on interval from 2024-10-22 02:20:15 to 2024-10-22 02:20:21 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:20:22.346588 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for 6d3d46d6fcc81e532d61fe2677299730:0) 2024.10.22 02:20:22.349384 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1b37e5550180900c8d7a9b0825903a67, offset: 0, size: 1351171 2024.10.22 02:20:22.354497 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 1b37e5550180900c8d7a9b0825903a67 for removal 2024.10.22 02:20:22.356337 [ 1476 ] {} CacheMetadata: Key 1b37e5550180900c8d7a9b0825903a67 is removed from metadata 2024.10.22 02:20:22.356994 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: b6b894ba5cf9a8dee5fd92f1fd0d4a59, offset: 0, size: 201 2024.10.22 02:20:22.360250 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key b6b894ba5cf9a8dee5fd92f1fd0d4a59 for removal 2024.10.22 02:20:22.372851 [ 1476 ] {} CacheMetadata: Key b6b894ba5cf9a8dee5fd92f1fd0d4a59 is removed from metadata 2024.10.22 02:20:22.871283 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 741 2024.10.22 02:20:23.142313 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:23.147047 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:23.216914 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_723_723_0 (state Temporary) 2024.10.22 02:20:23.267439 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_723_723_0 to 202410_733_733_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:23.303114 [ 1241 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_728_148 to 202410_733_733_0 2024.10.22 02:20:23.315848 [ 1241 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:23.319180 [ 1241 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:23.334548 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_728_148 to 202410_733_733_0 into Compact with storage Full 2024.10.22 02:20:23.366546 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 741 2024.10.22 02:20:23.498899 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 2.00 GiB, free memory in arenas 9.44 MiB, will set to 1.56 GiB (RSS), difference: -120.19 MiB 2024.10.22 02:20:23.500400 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:23.548507 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_728_148, total 772 rows starting from the beginning of the part 2024.10.22 02:20:23.578207 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 100630 2024.10.22 02:20:23.703964 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:23.709801 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:23.711193 [ 1223 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:20:23.759027 [ 1223 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:20:23.791568 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_729_729_0, total 1 rows starting from the beginning of the part 2024.10.22 02:20:23.967629 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1504_1504_0 (state Temporary) 2024.10.22 02:20:24.037698 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_730_730_0, total 1 rows starting from the beginning of the part 2024.10.22 02:20:24.128650 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1504_1504_0 to 202410_1517_1517_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:24.163912 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100630 2024.10.22 02:20:24.376528 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 9.83 MiB, will set to 1.56 GiB (RSS), difference: -50.21 MiB 2024.10.22 02:20:24.391522 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 02:20:24.539715 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_731_731_0, total 1 rows starting from the beginning of the part 2024.10.22 02:20:24.727795 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_790_149, 202410_791_796_1] 2024.10.22 02:20:24.729982 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_790_149, 202410_791_796_1] 2024.10.22 02:20:24.755666 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_790_149, 202410_791_796_1] 2024.10.22 02:20:24.810896 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_732_732_0, total 1 rows starting from the beginning of the part 2024.10.22 02:20:25.025314 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_733_733_0, total 1 rows starting from the beginning of the part 2024.10.22 02:20:25.727133 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 13630 2024.10.22 02:20:25.848442 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:25.851272 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:25.883332 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_893_893_0 (state Temporary) 2024.10.22 02:20:26.040407 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_893_893_0 to 202410_906_906_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:26.086817 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:26.088832 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fyb/cjntoihnssokzazgleoszuxcyzooi (LogSeriesLimiter: on interval from 2024-10-22 02:20:21 to 2024-10-22 02:20:26 accepted series 1 / 179 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:20:26.092716 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:26.110336 [ 1154 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_901_266 to 202410_902_902_0 2024.10.22 02:20:26.119102 [ 1154 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:26.139308 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13630 2024.10.22 02:20:26.152317 [ 1154 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:26.176122 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_902_267} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_901_266 to 202410_902_902_0 into Compact with storage Full 2024.10.22 02:20:26.268777 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_902_267} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:26.282551 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_902_267} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_901_266, total 13660 rows starting from the beginning of the part 2024.10.22 02:20:26.332132 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_902_267} MergeTreeSequentialSource: Reading 2 marks from part 202410_902_902_0, total 8 rows starting from the beginning of the part 2024.10.22 02:20:26.369323 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 2.00 GiB, free memory in arenas 10.51 MiB, will set to 1.57 GiB (RSS), difference: -127.29 MiB 2024.10.22 02:20:26.403404 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_856_160/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:20:27.157419 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_902_267} MergeTask::MergeProjectionsStage: Merge sorted 13668 rows, containing 12 columns (12 merged, 0 gathered) in 0.987244284 sec., 13844.597757124113 rows/sec., 2.16 MiB/sec. 2024.10.22 02:20:27.174977 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 97074 2024.10.22 02:20:27.227631 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 2296 entries to flush up to offset 1294077 2024.10.22 02:20:27.318541 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} MergeTask::MergeProjectionsStage: Merge sorted 777 rows, containing 32 columns (32 merged, 0 gathered) in 3.988641038 sec., 194.80319050961938 rows/sec., 14.74 KiB/sec. 2024.10.22 02:20:27.319026 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_902_267} MergedBlockOutputStream: filled checksums 202410_1_902_267 (state Temporary) 2024.10.22 02:20:27.350978 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:27.354461 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:27.418447 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_878_878_0 (state Temporary) 2024.10.22 02:20:27.433759 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_902_267} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_902_267 to 202410_1_902_267 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:27.439428 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_902_267} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_901_266, 202410_902_902_0] -> 202410_1_902_267 2024.10.22 02:20:27.468150 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} MergedBlockOutputStream: filled checksums 202410_1_733_149 (state Temporary) 2024.10.22 02:20:27.468557 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:20:27.591133 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_878_878_0 to 202410_884_884_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:27.763299 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 97074 2024.10.22 02:20:28.265677 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 100680 2024.10.22 02:20:28.288932 [ 1731 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ipg/phzjkrifhhqwckrqzagtqevhvrvla, size 29886 2024.10.22 02:20:28.488348 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:28.495058 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:28.627782 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column EventTime 2024.10.22 02:20:28.647221 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column EventTime 2024.10.22 02:20:28.654359 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.74 GiB, peak 2.00 GiB, free memory in arenas 13.29 MiB, will set to 1.57 GiB (RSS), difference: -173.09 MiB 2024.10.22 02:20:28.716935 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:28.721552 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:28.739902 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column EventTime 2024.10.22 02:20:28.753833 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column EventTime 2024.10.22 02:20:28.756292 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1505_1505_0 (state Temporary) 2024.10.22 02:20:28.763690 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column EventTime 2024.10.22 02:20:28.782280 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_733_149 to 202410_1_733_149 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:28.785267 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_733_149} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_728_148, 202410_733_733_0] -> 202410_1_733_149 2024.10.22 02:20:28.829119 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:20:28.881655 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column EventTime 2024.10.22 02:20:28.913523 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_855_855_0 (state Temporary) 2024.10.22 02:20:28.922853 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1505_1505_0 to 202410_1518_1518_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:28.928297 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column EventTime 2024.10.22 02:20:28.964465 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column EventTime 2024.10.22 02:20:28.991568 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column EventTime 2024.10.22 02:20:29.012701 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100680 2024.10.22 02:20:29.146962 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_855_855_0 to 202410_862_862_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:29.167658 [ 1181 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_857_857_0 to 202410_862_862_0 2024.10.22 02:20:29.182557 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1294077 2024.10.22 02:20:29.187302 [ 1181 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:29.190581 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:29.242536 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_857_857_0 to 202410_862_862_0 into Compact with storage Full 2024.10.22 02:20:29.274320 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:29.290973 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_857_857_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:20:29.329513 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_858_858_0, total 1157 rows starting from the beginning of the part 2024.10.22 02:20:29.353068 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_859_859_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:20:29.371540 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_860_860_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:20:29.381340 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_861_861_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:20:29.401660 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_179_179_0 (state Temporary) 2024.10.22 02:20:29.517642 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_862_862_0, total 2296 rows starting from the beginning of the part 2024.10.22 02:20:30.038986 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_842_215, 202410_843_843_0] 2024.10.22 02:20:30.053509 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_842_215, 202410_843_843_0] 2024.10.22 02:20:30.072529 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_842_215, 202410_843_843_0] 2024.10.22 02:20:30.569237 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 2.00 GiB, free memory in arenas 12.50 MiB, will set to 1.57 GiB (RSS), difference: -107.69 MiB 2024.10.22 02:20:30.928674 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} MergeTask::MergeProjectionsStage: Merge sorted 9228 rows, containing 4 columns (4 merged, 0 gathered) in 1.698976264 sec., 5431.5061343317275 rows/sec., 84.87 KiB/sec. 2024.10.22 02:20:31.086394 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} MergedBlockOutputStream: filled checksums 202410_857_862_1 (state Temporary) 2024.10.22 02:20:31.225665 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_857_862_1 to 202410_857_862_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:31.240705 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_857_862_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_857_857_0, 202410_862_862_0] -> 202410_857_862_1 2024.10.22 02:20:31.264787 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:20:31.345412 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:20:31.347689 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:20:31.787484 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(6d3d46d6fcc81e532d61fe2677299730) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6d3d46d6fcc81e532d61fe2677299730, state: DOWNLOADING, downloaded size: 2097152, reserved size: 2097152, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 2097152, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:20:31.791089 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 2097152, file segment info: File segment: [0, 33554431], key: 6d3d46d6fcc81e532d61fe2677299730, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 2097152, reserved size: 2097152, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 2097152, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:20:31.792720 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:20:32.424280 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 13.13 MiB, will set to 1.59 GiB (RSS), difference: 10.40 MiB 2024.10.22 02:20:32.773833 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_179_179_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:20:33.018625 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 100733 2024.10.22 02:20:33.048771 [ 1735 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/uwm/kqqeaqduxmqmzeezbsttgaxwsaobc, size 4320586 2024.10.22 02:20:33.227921 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FetchTiming 2024.10.22 02:20:33.236688 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:33.242310 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FetchTiming 2024.10.22 02:20:33.246489 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:33.276085 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FetchTiming 2024.10.22 02:20:33.310004 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FetchTiming 2024.10.22 02:20:33.312412 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1506_1506_0 (state Temporary) 2024.10.22 02:20:33.332072 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FetchTiming 2024.10.22 02:20:33.356304 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1506_1506_0 to 202410_1519_1519_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:33.367995 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FetchTiming 2024.10.22 02:20:33.408850 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FetchTiming 2024.10.22 02:20:33.454403 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100733 2024.10.22 02:20:33.591303 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/dpn/crqakkdkkxzlvshnbashjjjadyeeq (LogSeriesLimiter: on interval from 2024-10-22 02:20:26 to 2024-10-22 02:20:33 accepted series 1 / 180 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:20:33.643820 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 13652 2024.10.22 02:20:33.735146 [ 1199 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 361 ms. 2024.10.22 02:20:33.768909 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 1048576 bytes for ce6e56acb5b02707acc4835057a37f39:0) 2024.10.22 02:20:33.770549 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e1a2c3ee42ce6b2e391949b36742548f, offset: 0, size: 454784 2024.10.22 02:20:33.775023 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e1a2c3ee42ce6b2e391949b36742548f for removal 2024.10.22 02:20:33.783234 [ 1476 ] {} CacheMetadata: Key e1a2c3ee42ce6b2e391949b36742548f is removed from metadata 2024.10.22 02:20:33.784222 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 7317624b43399facc6e9f3fdb0fceb47, offset: 0, size: 157 2024.10.22 02:20:33.790539 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 7317624b43399facc6e9f3fdb0fceb47 for removal 2024.10.22 02:20:33.809002 [ 1476 ] {} CacheMetadata: Key 7317624b43399facc6e9f3fdb0fceb47 is removed from metadata 2024.10.22 02:20:33.844952 [ 1212 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 02:20:34.047999 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:34.051201 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:34.189250 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_894_894_0 (state Temporary) 2024.10.22 02:20:34.236911 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_894_894_0 to 202410_907_907_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:34.276684 [ 1163 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_902_267 to 202410_903_903_0 2024.10.22 02:20:34.318995 [ 1163 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:34.334171 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:34.362147 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_903_268} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_902_267 to 202410_903_903_0 into Compact with storage Full 2024.10.22 02:20:34.375600 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13652 2024.10.22 02:20:34.487641 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_903_268} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:34.561632 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_903_268} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_902_267, total 13668 rows starting from the beginning of the part 2024.10.22 02:20:34.567444 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 14.27 MiB, will set to 1.59 GiB (RSS), difference: -2.17 MiB 2024.10.22 02:20:34.655742 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_903_268} MergeTreeSequentialSource: Reading 2 marks from part 202410_903_903_0, total 17 rows starting from the beginning of the part 2024.10.22 02:20:35.375547 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_843_216, 202410_844_844_0] 2024.10.22 02:20:35.377174 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_843_216, 202410_844_844_0] 2024.10.22 02:20:35.392935 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_843_216, 202410_844_844_0] 2024.10.22 02:20:35.400037 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 97153 2024.10.22 02:20:35.471599 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_179_179_0 to 202410_182_182_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:35.727495 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_903_268} MergeTask::MergeProjectionsStage: Merge sorted 13685 rows, containing 12 columns (12 merged, 0 gathered) in 1.378312807 sec., 9928.805660441056 rows/sec., 1.55 MiB/sec. 2024.10.22 02:20:35.861410 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:35.868902 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:35.874781 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_903_268} MergedBlockOutputStream: filled checksums 202410_1_903_268 (state Temporary) 2024.10.22 02:20:35.932559 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_879_879_0 (state Temporary) 2024.10.22 02:20:35.964924 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_879_879_0 to 202410_885_885_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:36.131110 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 97153 2024.10.22 02:20:36.202990 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1554 entries to flush up to offset 1295631 2024.10.22 02:20:36.292809 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_903_268/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:20:36.489784 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_903_268} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_903_268 to 202410_1_903_268 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:36.496889 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_903_268} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_902_267, 202410_903_903_0] -> 202410_1_903_268 2024.10.22 02:20:36.549367 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:20:36.740382 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(ce6e56acb5b02707acc4835057a37f39) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ce6e56acb5b02707acc4835057a37f39, state: DOWNLOADING, downloaded size: 1048576, reserved size: 1048576, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 1048576, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:20:36.746466 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: ce6e56acb5b02707acc4835057a37f39, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 1048576, reserved size: 1048576, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 1048576, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:20:36.748877 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:20:37.268609 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:37.271715 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:37.387648 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_856_160/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:20:37.419246 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_856_856_0 (state Temporary) 2024.10.22 02:20:37.549618 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_856_856_0 to 202410_863_863_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:37.563122 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 100795 2024.10.22 02:20:37.682282 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7225 2024.10.22 02:20:37.712331 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1295631 2024.10.22 02:20:37.745556 [ 1270 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 267 ms. 2024.10.22 02:20:37.947420 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:37.949419 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:38.095253 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1507_1507_0 (state Temporary) 2024.10.22 02:20:38.153190 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1507_1507_0 to 202410_1520_1520_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:38.183316 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1515_76 to 202410_1516_1516_0 2024.10.22 02:20:38.194261 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:38.199080 [ 1209 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:38.211985 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100795 2024.10.22 02:20:38.330331 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 14.97 MiB, will set to 1.61 GiB (RSS), difference: 10.47 MiB 2024.10.22 02:20:38.361832 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1516_77} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1515_76 to 202410_1516_1516_0 into Compact with storage Full 2024.10.22 02:20:38.368080 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 742 2024.10.22 02:20:38.388460 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1516_77} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:38.404621 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1516_77} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1515_76, total 21336 rows starting from the beginning of the part 2024.10.22 02:20:38.529382 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1516_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_1516_1516_0, total 75 rows starting from the beginning of the part 2024.10.22 02:20:38.761432 [ 1164 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:20:38.772068 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:38.774640 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:38.963667 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_724_724_0 (state Temporary) 2024.10.22 02:20:38.998523 [ 1164 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:20:39.104196 [ 1164 ] {} DNSCacheUpdater: Execution took 240 ms. 2024.10.22 02:20:39.252173 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_724_724_0 to 202410_734_734_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:39.452416 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 742 2024.10.22 02:20:39.660903 [ 1419 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 6 similar messages) 2024.10.22 02:20:40.047508 [ 1268 ] {} DatabaseCatalog: Execution took 250 ms. 2024.10.22 02:20:40.197774 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1516_77} MergeTask::MergeProjectionsStage: Merge sorted 21411 rows, containing 13 columns (13 merged, 0 gathered) in 1.840521394 sec., 11633.116610216377 rows/sec., 2.13 MiB/sec. 2024.10.22 02:20:40.393758 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1516_77} MergedBlockOutputStream: filled checksums 202410_1207_1516_77 (state Temporary) 2024.10.22 02:20:40.483950 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1416_42, 202410_1417_1417_0, 202410_1418_1418_0, 202410_1419_1419_0, 202410_1420_1420_0, 202410_1421_1421_0] 2024.10.22 02:20:40.494260 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1416_42, 202410_1417_1417_0, 202410_1418_1418_0, 202410_1419_1419_0, 202410_1420_1420_0, 202410_1421_1421_0] 2024.10.22 02:20:40.536980 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 14.91 MiB, will set to 1.62 GiB (RSS), difference: -2.04 MiB 2024.10.22 02:20:40.535988 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(a5a2ad8307f9a61afaa6566c330bc4d1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a5a2ad8307f9a61afaa6566c330bc4d1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:20:40.545402 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1416_42, 202410_1417_1417_0, 202410_1418_1418_0, 202410_1419_1419_0, 202410_1420_1420_0, 202410_1421_1421_0] 2024.10.22 02:20:40.546645 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: a5a2ad8307f9a61afaa6566c330bc4d1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:20:40.551953 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:20:40.750891 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1516_77} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1516_77 to 202410_1207_1516_77 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:40.757720 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1516_77} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1515_76, 202410_1516_1516_0] -> 202410_1207_1516_77 2024.10.22 02:20:40.796877 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.42 MiB. 2024.10.22 02:20:41.345230 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:20:41.347396 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:20:41.682376 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_822_15, 202410_823_823_0, 202410_824_824_0, 202410_825_825_0, 202410_826_826_0, 202410_827_827_0] 2024.10.22 02:20:41.684021 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_822_15, 202410_823_823_0, 202410_824_824_0, 202410_825_825_0, 202410_826_826_0, 202410_827_827_0] 2024.10.22 02:20:41.813809 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_822_15, 202410_823_823_0, 202410_824_824_0, 202410_825_825_0, 202410_826_826_0, 202410_827_827_0] 2024.10.22 02:20:41.886606 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 13664 2024.10.22 02:20:42.059081 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:42.060998 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:42.100518 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_895_895_0 (state Temporary) 2024.10.22 02:20:42.229148 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 100845 2024.10.22 02:20:42.268170 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_895_895_0 to 202410_908_908_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:42.295079 [ 1276 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_903_268 to 202410_904_904_0 2024.10.22 02:20:42.302418 [ 1276 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:42.305501 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:42.332955 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_904_269} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_903_268 to 202410_904_904_0 into Compact with storage Full 2024.10.22 02:20:42.375896 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:42.379732 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:42.412537 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13664 2024.10.22 02:20:42.450285 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1508_1508_0 (state Temporary) 2024.10.22 02:20:42.481460 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1508_1508_0 to 202410_1521_1521_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:42.526274 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1516_77 to 202410_1517_1517_0 2024.10.22 02:20:42.540965 [ 1262 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:42.553427 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 17.66 MiB, will set to 1.62 GiB (RSS), difference: 13.81 MiB 2024.10.22 02:20:42.559565 [ 1262 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:42.598813 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1517_78} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1516_77 to 202410_1517_1517_0 into Compact with storage Full 2024.10.22 02:20:42.601247 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_904_269} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:42.618041 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_904_269} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_903_268, total 13685 rows starting from the beginning of the part 2024.10.22 02:20:42.635847 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1517_78} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:42.636513 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100845 2024.10.22 02:20:42.644943 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1517_78} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1516_77, total 21411 rows starting from the beginning of the part 2024.10.22 02:20:42.667834 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_904_269} MergeTreeSequentialSource: Reading 2 marks from part 202410_904_904_0, total 12 rows starting from the beginning of the part 2024.10.22 02:20:42.759353 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1517_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_1517_1517_0, total 78 rows starting from the beginning of the part 2024.10.22 02:20:43.649140 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 103 entries to flush up to offset 97256 2024.10.22 02:20:43.779055 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_904_269} MergeTask::MergeProjectionsStage: Merge sorted 13697 rows, containing 12 columns (12 merged, 0 gathered) in 1.458081321 sec., 9393.851908483504 rows/sec., 1.47 MiB/sec. 2024.10.22 02:20:43.813187 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_844_217, 202410_845_845_0] 2024.10.22 02:20:43.816005 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_844_217, 202410_845_845_0] 2024.10.22 02:20:43.835902 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_844_217, 202410_845_845_0] 2024.10.22 02:20:43.843613 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_904_269} MergedBlockOutputStream: filled checksums 202410_1_904_269 (state Temporary) 2024.10.22 02:20:43.900497 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:43.902728 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:44.054772 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_880_880_0 (state Temporary) 2024.10.22 02:20:44.185796 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_904_269} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_904_269 to 202410_1_904_269 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:44.196100 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_904_269} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_903_268, 202410_904_904_0] -> 202410_1_904_269 2024.10.22 02:20:44.202769 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_880_880_0 to 202410_886_886_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:44.241624 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:20:44.257806 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 97256 2024.10.22 02:20:44.642040 [ 1246 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 324 ms. 2024.10.22 02:20:44.662603 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1517_78} MergeTask::MergeProjectionsStage: Merge sorted 21489 rows, containing 13 columns (13 merged, 0 gathered) in 2.074525146 sec., 10358.515075815812 rows/sec., 1.90 MiB/sec. 2024.10.22 02:20:44.700217 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1517_78} MergedBlockOutputStream: filled checksums 202410_1207_1517_78 (state Temporary) 2024.10.22 02:20:44.718802 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1297171 2024.10.22 02:20:44.761275 [ 1219 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 02:20:44.970352 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1517_78} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1517_78 to 202410_1207_1517_78 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:44.974572 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1517_78} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1516_77, 202410_1517_1517_0] -> 202410_1207_1517_78 2024.10.22 02:20:45.000603 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.43 MiB. 2024.10.22 02:20:45.226530 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 7264 2024.10.22 02:20:45.739622 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:45.743885 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:46.160300 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_857_857_0 (state Temporary) 2024.10.22 02:20:46.361633 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_857_857_0 to 202410_864_864_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:46.400045 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1297171 2024.10.22 02:20:46.476459 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 19.61 MiB, will set to 1.63 GiB (RSS), difference: 6.52 MiB 2024.10.22 02:20:46.646417 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 100897 2024.10.22 02:20:46.774396 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.095523767 sec. 2024.10.22 02:20:46.954382 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 743 2024.10.22 02:20:47.014571 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:47.020400 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:47.090413 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1509_1509_0 (state Temporary) 2024.10.22 02:20:47.279722 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1509_1509_0 to 202410_1522_1522_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:47.299743 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:47.314898 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.40 GiB. 2024.10.22 02:20:47.351298 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1517_78 to 202410_1518_1518_0 2024.10.22 02:20:47.399519 [ 1253 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:47.410450 [ 1253 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.39 GiB. 2024.10.22 02:20:47.496586 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100897 2024.10.22 02:20:47.566127 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_725_725_0 (state Temporary) 2024.10.22 02:20:47.591866 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1518_79} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1517_78 to 202410_1518_1518_0 into Compact with storage Full 2024.10.22 02:20:48.002936 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_725_725_0 to 202410_735_735_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:48.011961 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1518_79} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:48.077679 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1518_79} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1517_78, total 21489 rows starting from the beginning of the part 2024.10.22 02:20:48.082187 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_856_160/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:20:48.148846 [ 1711 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fyb/cjntoihnssokzazgleoszuxcyzooi, size 112136 2024.10.22 02:20:48.166526 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 743 2024.10.22 02:20:48.441227 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1518_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_1518_1518_0, total 50 rows starting from the beginning of the part 2024.10.22 02:20:48.732353 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 15.94 MiB, will set to 1.63 GiB (RSS), difference: 128.65 MiB 2024.10.22 02:20:48.767270 [ 1170 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 02:20:49.915638 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 13676 2024.10.22 02:20:50.061838 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:50.074763 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.37 GiB. 2024.10.22 02:20:50.279924 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_896_896_0 (state Temporary) 2024.10.22 02:20:50.321577 [ 1160 ] {} DatabaseCatalog: Execution took 270 ms. 2024.10.22 02:20:50.568828 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 16.13 MiB, will set to 1.62 GiB (RSS), difference: 106.64 MiB 2024.10.22 02:20:50.624577 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_896_896_0 to 202410_909_909_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:50.701561 [ 1156 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_904_269 to 202410_905_905_0 2024.10.22 02:20:50.716302 [ 1156 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:50.748657 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.36 GiB. 2024.10.22 02:20:50.798755 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_905_270} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_904_269 to 202410_905_905_0 into Compact with storage Full 2024.10.22 02:20:50.879020 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_905_270} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:50.882093 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_803_803_0, 202410_804_804_0, 202410_805_805_0, 202410_806_806_0, 202410_807_807_0, 202410_808_808_0] 2024.10.22 02:20:50.887845 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_803_803_0, 202410_804_804_0, 202410_805_805_0, 202410_806_806_0, 202410_807_807_0, 202410_808_808_0] 2024.10.22 02:20:50.889210 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_905_270} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_904_269, total 13697 rows starting from the beginning of the part 2024.10.22 02:20:50.793637 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ClientIP 2024.10.22 02:20:50.903345 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ClientIP 2024.10.22 02:20:50.922314 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ClientIP 2024.10.22 02:20:50.927793 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13676 2024.10.22 02:20:50.941385 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_803_803_0, 202410_804_804_0, 202410_805_805_0, 202410_806_806_0, 202410_807_807_0, 202410_808_808_0] 2024.10.22 02:20:50.943011 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_905_270} MergeTreeSequentialSource: Reading 2 marks from part 202410_905_905_0, total 23 rows starting from the beginning of the part 2024.10.22 02:20:50.947807 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ClientIP 2024.10.22 02:20:50.970332 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ClientIP 2024.10.22 02:20:51.015905 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ClientIP 2024.10.22 02:20:51.059451 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ClientIP 2024.10.22 02:20:51.079455 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ClientIP 2024.10.22 02:20:51.112835 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ClientIP 2024.10.22 02:20:51.208878 [ 1696 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/dpn/crqakkdkkxzlvshnbashjjjadyeeq, size 3075347 2024.10.22 02:20:51.299372 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zfo/oqxhikuxfgwgbuabcinizmmwxrive (LogSeriesLimiter: on interval from 2024-10-22 02:20:33 to 2024-10-22 02:20:51 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:20:51.347318 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:20:51.353705 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:20:51.507260 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 100952 2024.10.22 02:20:51.630010 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1518_79} MergeTask::MergeProjectionsStage: Merge sorted 21539 rows, containing 13 columns (13 merged, 0 gathered) in 4.147008934 sec., 5193.86390113815 rows/sec., 974.47 KiB/sec. 2024.10.22 02:20:51.659602 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_19_19_0 (state Temporary) 2024.10.22 02:20:51.775323 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 153 entries to flush up to offset 97409 2024.10.22 02:20:51.778851 [ 1673 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mpm/imbvjpiaiyhuianzrduqwcrdtvgta, size 367 2024.10.22 02:20:51.877044 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1518_79} MergedBlockOutputStream: filled checksums 202410_1207_1518_79 (state Temporary) 2024.10.22 02:20:51.905589 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:51.917816 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:20:51.949797 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_856_160} MergeTask::MergeProjectionsStage: Merge sorted 1298439 rows, containing 4 columns (4 merged, 0 gathered) in 39.999399026 sec., 32461.462712377303 rows/sec., 486.80 KiB/sec. 2024.10.22 02:20:51.977455 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1510_1510_0 (state Temporary) 2024.10.22 02:20:52.037777 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1510_1510_0 to 202410_1523_1523_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:52.078569 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 02:20:52.079513 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 100952 2024.10.22 02:20:52.182668 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_905_270} MergeTask::MergeProjectionsStage: Merge sorted 13720 rows, containing 12 columns (12 merged, 0 gathered) in 1.398109919 sec., 9813.248453178308 rows/sec., 1.53 MiB/sec. 2024.10.22 02:20:52.244460 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:52.244505 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_856_160} MergedBlockOutputStream: filled checksums 202410_1_856_160 (state Temporary) 2024.10.22 02:20:52.247280 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:20:52.273353 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_905_270} MergedBlockOutputStream: filled checksums 202410_1_905_270 (state Temporary) 2024.10.22 02:20:52.303409 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1518_79} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1518_79 to 202410_1207_1518_79 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:52.307835 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1518_79} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1517_78, 202410_1518_1518_0] -> 202410_1207_1518_79 2024.10.22 02:20:52.313723 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_845_218, 202410_846_846_0] 2024.10.22 02:20:52.320002 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_845_218, 202410_846_846_0] 2024.10.22 02:20:52.325498 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 02:20:52.339680 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_845_218, 202410_846_846_0] 2024.10.22 02:20:52.354233 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_881_881_0 (state Temporary) 2024.10.22 02:20:52.359060 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.42 MiB. 2024.10.22 02:20:52.411140 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_881_881_0 to 202410_887_887_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:52.451371 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_882_27 to 202410_887_887_0 2024.10.22 02:20:52.451856 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_905_270} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_905_270 to 202410_1_905_270 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:52.453812 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_905_270} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_904_269, 202410_905_905_0] -> 202410_1_905_270 2024.10.22 02:20:52.460361 [ 1193 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:52.462259 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:20:52.471039 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_882_27 to 202410_887_887_0 into Compact with storage Full 2024.10.22 02:20:52.484553 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:20:52.555683 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 02:20:52.575359 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 2.00 GiB, free memory in arenas 9.47 MiB, will set to 1.58 GiB (RSS), difference: 117.06 MiB 2024.10.22 02:20:52.587939 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 02:20:52.613068 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 02:20:52.643502 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:52.648642 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 97409 2024.10.22 02:20:52.732818 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_882_27, total 13486 rows starting from the beginning of the part 2024.10.22 02:20:52.765119 [ 1258 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1518_79 to 202410_1519_1519_0 2024.10.22 02:20:52.767693 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 02:20:52.815446 [ 1258 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:52.817529 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RedirectTiming 2024.10.22 02:20:52.823182 [ 1258 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:20:52.844865 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1519_80} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1518_79 to 202410_1519_1519_0 into Compact with storage Full 2024.10.22 02:20:52.875867 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_883_883_0, total 154 rows starting from the beginning of the part 2024.10.22 02:20:52.914611 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1519_80} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:52.923988 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_884_884_0, total 79 rows starting from the beginning of the part 2024.10.22 02:20:52.926433 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1519_80} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1518_79, total 21539 rows starting from the beginning of the part 2024.10.22 02:20:53.020429 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_885_885_0, total 79 rows starting from the beginning of the part 2024.10.22 02:20:53.083018 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1519_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_1519_1519_0, total 53 rows starting from the beginning of the part 2024.10.22 02:20:53.273705 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_886_886_0, total 103 rows starting from the beginning of the part 2024.10.22 02:20:53.407141 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1927 entries to flush up to offset 1299098 2024.10.22 02:20:53.429613 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} MergeTreeSequentialSource: Reading 2 marks from part 202410_887_887_0, total 153 rows starting from the beginning of the part 2024.10.22 02:20:54.003910 [ 1247 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:20:54.030394 [ 1247 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:20:54.259820 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1519_80/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:20:54.358789 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 5.90 MiB, will set to 1.59 GiB (RSS), difference: -26.04 MiB 2024.10.22 02:20:54.417199 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:54.424342 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:20:54.748483 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_858_858_0 (state Temporary) 2024.10.22 02:20:54.841554 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1519_80} MergeTask::MergeProjectionsStage: Merge sorted 21592 rows, containing 13 columns (13 merged, 0 gathered) in 2.000714944 sec., 10792.142111375164 rows/sec., 1.98 MiB/sec. 2024.10.22 02:20:54.928305 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_858_858_0 to 202410_865_865_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:55.006153 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1519_80} MergedBlockOutputStream: filled checksums 202410_1207_1519_80 (state Temporary) 2024.10.22 02:20:55.015805 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1299098 2024.10.22 02:20:55.302935 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1519_80} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1519_80 to 202410_1207_1519_80 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:55.313131 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1519_80} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1518_79, 202410_1519_1519_0] -> 202410_1207_1519_80 2024.10.22 02:20:55.445640 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.42 MiB. 2024.10.22 02:20:55.459088 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} MergeTask::MergeProjectionsStage: Merge sorted 14054 rows, containing 13 columns (13 merged, 0 gathered) in 2.993054383 sec., 4695.537802394819 rows/sec., 2.04 MiB/sec. 2024.10.22 02:20:55.471431 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_856_160} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_856_160 to 202410_1_856_160 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:55.479355 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_856_160} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_850_159, 202410_851_856_1] -> 202410_1_856_160 2024.10.22 02:20:55.516758 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:20:55.681158 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 744 2024.10.22 02:20:55.735444 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} MergedBlockOutputStream: filled checksums 202410_747_887_28 (state Temporary) 2024.10.22 02:20:55.996036 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_887_28 to 202410_747_887_28 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:56.005440 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_887_28} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_882_27, 202410_887_887_0] -> 202410_747_887_28 2024.10.22 02:20:56.096093 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 101030 2024.10.22 02:20:56.113346 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:20:56.157740 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.18 MiB. 2024.10.22 02:20:56.165669 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:56.168873 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:20:56.235287 [ 1269 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 264 ms. 2024.10.22 02:20:56.245995 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_726_726_0 (state Temporary) 2024.10.22 02:20:56.469362 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 5.43 MiB, will set to 1.58 GiB (RSS), difference: 12.62 MiB 2024.10.22 02:20:56.624078 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_726_726_0 to 202410_736_736_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:56.912604 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 744 2024.10.22 02:20:56.940637 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:56.946638 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:20:57.084458 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1511_1511_0 (state Temporary) 2024.10.22 02:20:57.146347 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1511_1511_0 to 202410_1524_1524_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:57.168480 [ 1229 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1519_80 to 202410_1520_1520_0 2024.10.22 02:20:57.172509 [ 1229 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:57.175897 [ 1229 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:20:57.187145 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101030 2024.10.22 02:20:57.326488 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1520_81} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1519_80 to 202410_1520_1520_0 into Compact with storage Full 2024.10.22 02:20:57.413127 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1520_81} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:57.428762 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1520_81} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1519_80, total 21592 rows starting from the beginning of the part 2024.10.22 02:20:57.454737 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1520_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_1520_1520_0, total 62 rows starting from the beginning of the part 2024.10.22 02:20:58.431454 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 13701 2024.10.22 02:20:58.581531 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 4.53 MiB, will set to 1.58 GiB (RSS), difference: -3.31 MiB 2024.10.22 02:20:58.687839 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:58.696623 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:20:58.756384 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_897_897_0 (state Temporary) 2024.10.22 02:20:58.959123 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_897_897_0 to 202410_910_910_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:58.965764 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1520_81} MergeTask::MergeProjectionsStage: Merge sorted 21654 rows, containing 13 columns (13 merged, 0 gathered) in 1.754871593 sec., 12339.36436510543 rows/sec., 2.26 MiB/sec. 2024.10.22 02:20:59.027424 [ 1175 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_905_270 to 202410_906_906_0 2024.10.22 02:20:59.042610 [ 1175 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:20:59.046335 [ 1175 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:20:59.101970 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_906_271} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_905_270 to 202410_906_906_0 into Compact with storage Full 2024.10.22 02:20:59.141608 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13701 2024.10.22 02:20:59.192495 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_906_271} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:20:59.248472 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_906_271} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_905_270, total 13720 rows starting from the beginning of the part 2024.10.22 02:20:59.264226 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1520_81} MergedBlockOutputStream: filled checksums 202410_1207_1520_81 (state Temporary) 2024.10.22 02:20:59.300589 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_846_219, 202410_847_847_0] 2024.10.22 02:20:59.304757 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_846_219, 202410_847_847_0] 2024.10.22 02:20:59.316874 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_846_219, 202410_847_847_0] 2024.10.22 02:20:59.325306 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1520_81} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1520_81 to 202410_1207_1520_81 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:20:59.329489 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1520_81} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1519_80, 202410_1520_1520_0] -> 202410_1207_1520_81 2024.10.22 02:20:59.269839 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_906_271} MergeTreeSequentialSource: Reading 2 marks from part 202410_906_906_0, total 16 rows starting from the beginning of the part 2024.10.22 02:20:59.564914 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.43 MiB. 2024.10.22 02:21:00.155974 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_683_139, 202410_684_684_0, 202410_685_685_0, 202410_686_686_0, 202410_687_687_0, 202410_688_688_0] 2024.10.22 02:21:00.157518 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_683_139, 202410_684_684_0, 202410_685_685_0, 202410_686_686_0, 202410_687_687_0, 202410_688_688_0] 2024.10.22 02:21:00.162889 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 129 entries to flush up to offset 97538 2024.10.22 02:21:00.214546 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_683_139, 202410_684_684_0, 202410_685_685_0, 202410_686_686_0, 202410_687_687_0, 202410_688_688_0] 2024.10.22 02:21:00.304376 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:00.307299 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:21:00.354905 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_882_882_0 (state Temporary) 2024.10.22 02:21:00.418397 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_882_882_0 to 202410_888_888_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:00.467207 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_906_271} MergeTask::MergeProjectionsStage: Merge sorted 13736 rows, containing 12 columns (12 merged, 0 gathered) in 1.370986349 sec., 10019.064019141448 rows/sec., 1.56 MiB/sec. 2024.10.22 02:21:00.492493 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 97538 2024.10.22 02:21:00.529075 [ 1260 ] {} DatabaseCatalog: Execution took 207 ms. 2024.10.22 02:21:00.617356 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(52a718d08b11372ac424ff056f3ae105) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 52a718d08b11372ac424ff056f3ae105, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:21:00.625066 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 52a718d08b11372ac424ff056f3ae105, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:21:00.630043 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:21:00.634996 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_906_271} MergedBlockOutputStream: filled checksums 202410_1_906_271 (state Temporary) 2024.10.22 02:21:01.172053 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_906_271} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_906_271 to 202410_1_906_271 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:01.175254 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_906_271} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_905_270, 202410_906_906_0] -> 202410_1_906_271 2024.10.22 02:21:01.315185 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:21:01.349410 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:21:01.361814 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 101091 2024.10.22 02:21:01.363402 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:21:01.471452 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(82fa14920948c515ad3d6947c116976a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 82fa14920948c515ad3d6947c116976a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 02:21:01.473349 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 82fa14920948c515ad3d6947c116976a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 02:21:01.475081 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:21:01.572796 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:01.676763 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:21:01.725505 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1512_1512_0 (state Temporary) 2024.10.22 02:21:01.870361 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1512_1512_0 to 202410_1525_1525_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:01.900533 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1520_81 to 202410_1521_1521_0 2024.10.22 02:21:01.915190 [ 1257 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:01.918568 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:21:01.931924 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101091 2024.10.22 02:21:01.952127 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1521_82} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1520_81 to 202410_1521_1521_0 into Compact with storage Full 2024.10.22 02:21:01.975596 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1521_82} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:02.025733 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1533 entries to flush up to offset 1300631 2024.10.22 02:21:02.092274 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1521_82} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1520_81, total 21654 rows starting from the beginning of the part 2024.10.22 02:21:02.124072 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1521_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_1521_1521_0, total 50 rows starting from the beginning of the part 2024.10.22 02:21:02.918671 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:02.923476 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:21:03.343315 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_859_859_0 (state Temporary) 2024.10.22 02:21:03.401640 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 3.54 MiB, will set to 1.58 GiB (RSS), difference: -7.91 MiB 2024.10.22 02:21:03.450478 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1521_82} MergeTask::MergeProjectionsStage: Merge sorted 21704 rows, containing 13 columns (13 merged, 0 gathered) in 1.501905275 sec., 14450.977942001036 rows/sec., 2.65 MiB/sec. 2024.10.22 02:21:03.506077 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_859_859_0 to 202410_866_866_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:03.556137 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1300631 2024.10.22 02:21:03.554827 [ 1205 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_856_160 to 202410_857_862_1 2024.10.22 02:21:03.573270 [ 1205 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:03.576871 [ 1205 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:21:03.701420 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_862_161} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_856_160 to 202410_857_862_1 into Wide with storage Full 2024.10.22 02:21:03.759629 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:03.764587 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:21:03.837817 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_862_161} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:03.850815 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_862_161} MergeTreeSequentialSource: Reading 160 marks from part 202410_1_856_160, total 1298439 rows starting from the beginning of the part 2024.10.22 02:21:03.901421 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1521_82} MergedBlockOutputStream: filled checksums 202410_1207_1521_82 (state Temporary) 2024.10.22 02:21:04.018187 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_862_161} MergeTreeSequentialSource: Reading 3 marks from part 202410_857_862_1, total 9228 rows starting from the beginning of the part 2024.10.22 02:21:04.240231 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1521_82} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1521_82 to 202410_1207_1521_82 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:04.244763 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1521_82} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1520_81, 202410_1521_1521_0] -> 202410_1207_1521_82 2024.10.22 02:21:04.417814 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 745 2024.10.22 02:21:04.577779 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.43 MiB. 2024.10.22 02:21:04.955733 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:04.958679 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:05.249616 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_727_727_0 (state Temporary) 2024.10.22 02:21:05.571742 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_727_727_0 to 202410_737_737_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:05.572578 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 5.13 MiB, will set to 1.57 GiB (RSS), difference: -4.34 MiB 2024.10.22 02:21:05.657025 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 745 2024.10.22 02:21:05.946726 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 101138 2024.10.22 02:21:06.273420 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1421_43, 202410_1422_1422_0, 202410_1423_1423_0, 202410_1424_1424_0, 202410_1425_1425_0, 202410_1426_1426_0] 2024.10.22 02:21:06.275401 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1421_43, 202410_1422_1422_0, 202410_1423_1423_0, 202410_1424_1424_0, 202410_1425_1425_0, 202410_1426_1426_0] 2024.10.22 02:21:06.316217 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1421_43, 202410_1422_1422_0, 202410_1423_1423_0, 202410_1424_1424_0, 202410_1425_1425_0, 202410_1426_1426_0] 2024.10.22 02:21:06.331483 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:06.338662 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:21:06.471830 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1513_1513_0 (state Temporary) 2024.10.22 02:21:06.626496 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1513_1513_0 to 202410_1526_1526_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:06.657042 [ 1238 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1521_82 to 202410_1522_1522_0 2024.10.22 02:21:06.661792 [ 1238 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:06.664232 [ 1238 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.33 GiB. 2024.10.22 02:21:06.674488 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1522_83} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1521_82 to 202410_1522_1522_0 into Compact with storage Full 2024.10.22 02:21:06.744699 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 13716 2024.10.22 02:21:06.755365 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1522_83} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:06.774439 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1522_83} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1521_82, total 21704 rows starting from the beginning of the part 2024.10.22 02:21:06.775120 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101138 2024.10.22 02:21:06.937184 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1522_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_1522_1522_0, total 52 rows starting from the beginning of the part 2024.10.22 02:21:07.245775 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:07.254905 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:07.496147 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_898_898_0 (state Temporary) 2024.10.22 02:21:07.496922 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_180_180_0 (state Temporary) 2024.10.22 02:21:07.555985 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 7.86 MiB, will set to 1.58 GiB (RSS), difference: -15.92 MiB 2024.10.22 02:21:07.799294 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 02:21:07.824142 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 02:21:07.850594 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 02:21:07.952491 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 02:21:07.969724 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 02:21:07.978901 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_898_898_0 to 202410_911_911_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:08.002352 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 02:21:08.015559 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 110 entries to flush up to offset 97648 2024.10.22 02:21:08.025776 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_906_271 to 202410_907_907_0 2024.10.22 02:21:08.030688 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column DOMInteractiveTiming 2024.10.22 02:21:08.050616 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13716 2024.10.22 02:21:08.051079 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:08.053834 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:08.075276 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_907_272} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_906_271 to 202410_907_907_0 into Compact with storage Full 2024.10.22 02:21:08.097802 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fth/brjoreidltrtqqddylqcmdxsnitik (LogSeriesLimiter: on interval from 2024-10-22 02:20:51 to 2024-10-22 02:21:08 accepted series 1 / 12 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:21:08.231966 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_907_272} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:08.346070 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_907_272} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_906_271, total 13736 rows starting from the beginning of the part 2024.10.22 02:21:08.392557 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_907_272} MergeTreeSequentialSource: Reading 2 marks from part 202410_907_907_0, total 22 rows starting from the beginning of the part 2024.10.22 02:21:08.631968 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:08.635830 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:08.675918 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_883_883_0 (state Temporary) 2024.10.22 02:21:08.738323 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_883_883_0 to 202410_889_889_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:08.828572 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1522_83} MergeTask::MergeProjectionsStage: Merge sorted 21756 rows, containing 13 columns (13 merged, 0 gathered) in 2.1592106 sec., 10075.904592169007 rows/sec., 1.84 MiB/sec. 2024.10.22 02:21:08.916892 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 97648 2024.10.22 02:21:09.037076 [ 1157 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:21:09.048641 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1522_83} MergedBlockOutputStream: filled checksums 202410_1207_1522_83 (state Temporary) 2024.10.22 02:21:09.059675 [ 1157 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:21:09.222798 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1522_83} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1522_83 to 202410_1207_1522_83 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:09.227305 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1522_83} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1521_82, 202410_1522_1522_0] -> 202410_1207_1522_83 2024.10.22 02:21:09.267482 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.42 MiB. 2024.10.22 02:21:09.427450 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_907_272} MergeTask::MergeProjectionsStage: Merge sorted 13758 rows, containing 12 columns (12 merged, 0 gathered) in 1.357581964 sec., 10134.194740966668 rows/sec., 1.58 MiB/sec. 2024.10.22 02:21:09.479810 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_907_272} MergedBlockOutputStream: filled checksums 202410_1_907_272 (state Temporary) 2024.10.22 02:21:09.537329 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 10.21 MiB, will set to 1.59 GiB (RSS), difference: 11.84 MiB 2024.10.22 02:21:09.646538 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_847_220, 202410_848_848_0] 2024.10.22 02:21:09.651537 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_847_220, 202410_848_848_0] 2024.10.22 02:21:09.663246 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_847_220, 202410_848_848_0] 2024.10.22 02:21:10.052835 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_862_161/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:21:10.095820 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_907_272} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_907_272 to 202410_1_907_272 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:10.099393 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_907_272} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_906_271, 202410_907_907_0] -> 202410_1_907_272 2024.10.22 02:21:10.151536 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:21:10.565509 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1547 entries to flush up to offset 1302178 2024.10.22 02:21:10.793346 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 101201 2024.10.22 02:21:11.177098 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:11.181726 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:11.349808 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:21:11.351848 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:21:11.355609 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 12.06 MiB, will set to 1.59 GiB (RSS), difference: 2.02 MiB 2024.10.22 02:21:11.362823 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1514_1514_0 (state Temporary) 2024.10.22 02:21:11.417345 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1514_1514_0 to 202410_1527_1527_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:11.560280 [ 1192 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1522_83 to 202410_1523_1523_0 2024.10.22 02:21:11.578407 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101201 2024.10.22 02:21:11.625040 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:11.676815 [ 1192 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:11.683003 [ 1192 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:11.700924 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1523_84} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1522_83 to 202410_1523_1523_0 into Compact with storage Full 2024.10.22 02:21:11.724488 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:11.725637 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_796_150, 202410_797_802_1] 2024.10.22 02:21:11.728333 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_796_150, 202410_797_802_1] 2024.10.22 02:21:11.740367 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_796_150, 202410_797_802_1] 2024.10.22 02:21:11.769661 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1523_84} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:11.790814 [ 1192 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 252 ms. 2024.10.22 02:21:11.793154 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1523_84} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1522_83, total 21756 rows starting from the beginning of the part 2024.10.22 02:21:11.903734 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_860_860_0 (state Temporary) 2024.10.22 02:21:11.961290 [ 1689 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zfo/oqxhikuxfgwgbuabcinizmmwxrive, size 2928449 2024.10.22 02:21:12.066044 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1523_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_1523_1523_0, total 55 rows starting from the beginning of the part 2024.10.22 02:21:12.138669 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_860_860_0 to 202410_867_867_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:12.224842 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ClientIP6 2024.10.22 02:21:12.236975 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ClientIP6 2024.10.22 02:21:12.265593 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1302178 2024.10.22 02:21:12.307956 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ClientIP6 2024.10.22 02:21:12.340124 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ClientIP6 2024.10.22 02:21:12.356858 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ClientIP6 2024.10.22 02:21:12.392144 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ClientIP6 2024.10.22 02:21:12.417822 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ClientIP6 2024.10.22 02:21:12.544922 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ClientIP6 2024.10.22 02:21:12.558995 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ClientIP6 2024.10.22 02:21:13.159624 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 746 2024.10.22 02:21:13.270270 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_180_180_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:21:13.492017 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 15.02 MiB, will set to 1.59 GiB (RSS), difference: -6.06 MiB 2024.10.22 02:21:13.552363 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:13.555293 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:13.716839 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_728_728_0 (state Temporary) 2024.10.22 02:21:13.800787 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_728_728_0 to 202410_738_738_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:13.842144 [ 1247 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_733_149 to 202410_738_738_0 2024.10.22 02:21:13.864114 [ 1247 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:13.873074 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 746 2024.10.22 02:21:13.966501 [ 1247 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:14.015139 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_733_149 to 202410_738_738_0 into Compact with storage Full 2024.10.22 02:21:14.149588 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:14.166717 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_733_149, total 777 rows starting from the beginning of the part 2024.10.22 02:21:14.173015 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1523_84} MergeTask::MergeProjectionsStage: Merge sorted 21811 rows, containing 13 columns (13 merged, 0 gathered) in 2.479853988 sec., 8795.275893477323 rows/sec., 1.61 MiB/sec. 2024.10.22 02:21:14.199475 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(b5ff202fac9b50ebd1d51a9b1ed125a9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b5ff202fac9b50ebd1d51a9b1ed125a9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:21:14.206017 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: b5ff202fac9b50ebd1d51a9b1ed125a9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:21:14.216048 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:21:14.254556 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_734_734_0, total 1 rows starting from the beginning of the part 2024.10.22 02:21:14.356499 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1523_84} MergedBlockOutputStream: filled checksums 202410_1207_1523_84 (state Temporary) 2024.10.22 02:21:14.422799 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_735_735_0, total 1 rows starting from the beginning of the part 2024.10.22 02:21:14.505600 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1523_84} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1523_84 to 202410_1207_1523_84 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:14.510408 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1523_84} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1522_83, 202410_1523_1523_0] -> 202410_1207_1523_84 2024.10.22 02:21:14.527851 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_736_736_0, total 1 rows starting from the beginning of the part 2024.10.22 02:21:14.563027 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.42 MiB. 2024.10.22 02:21:14.598808 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_737_737_0, total 1 rows starting from the beginning of the part 2024.10.22 02:21:14.660154 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_738_738_0, total 1 rows starting from the beginning of the part 2024.10.22 02:21:15.457033 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 13.16 MiB, will set to 1.59 GiB (RSS), difference: -2.24 MiB 2024.10.22 02:21:15.552447 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 13730 2024.10.22 02:21:15.624687 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 101265 2024.10.22 02:21:15.704909 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:15.707813 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:15.767828 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_899_899_0 (state Temporary) 2024.10.22 02:21:15.846844 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_899_899_0 to 202410_912_912_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:15.853154 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:15.856951 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:15.889034 [ 1227 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_907_272 to 202410_908_908_0 2024.10.22 02:21:15.896837 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13730 2024.10.22 02:21:15.921652 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1515_1515_0 (state Temporary) 2024.10.22 02:21:15.926541 [ 1227 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:15.930461 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:15.947028 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_908_273} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_907_272 to 202410_908_908_0 into Compact with storage Full 2024.10.22 02:21:16.087066 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_908_273} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:16.114521 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_908_273} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_907_272, total 13758 rows starting from the beginning of the part 2024.10.22 02:21:16.158126 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_908_273} MergeTreeSequentialSource: Reading 2 marks from part 202410_908_908_0, total 12 rows starting from the beginning of the part 2024.10.22 02:21:16.271614 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1515_1515_0 to 202410_1528_1528_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:16.320393 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1523_84 to 202410_1524_1524_0 2024.10.22 02:21:16.351319 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101265 2024.10.22 02:21:16.355277 [ 1249 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:16.361714 [ 1249 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:16.458800 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 116 entries to flush up to offset 97764 2024.10.22 02:21:16.504513 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1524_85} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1523_84 to 202410_1524_1524_0 into Compact with storage Full 2024.10.22 02:21:16.550422 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1524_85} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:16.601208 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1524_85} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1523_84, total 21811 rows starting from the beginning of the part 2024.10.22 02:21:16.724840 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1524_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_1524_1524_0, total 78 rows starting from the beginning of the part 2024.10.22 02:21:16.835185 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_180_180_0 to 202410_183_183_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:16.895590 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:16.904218 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:16.980392 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_908_273} MergeTask::MergeProjectionsStage: Merge sorted 13770 rows, containing 12 columns (12 merged, 0 gathered) in 1.038130731 sec., 13264.225389740435 rows/sec., 2.07 MiB/sec. 2024.10.22 02:21:16.988636 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_884_884_0 (state Temporary) 2024.10.22 02:21:17.061405 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/xxz/qazsdnftjpycnqeewewepqipbpedt. 2024.10.22 02:21:17.175383 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_848_221, 202410_849_849_0] 2024.10.22 02:21:17.295948 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_848_221, 202410_849_849_0] 2024.10.22 02:21:17.310466 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_884_884_0 to 202410_890_890_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:17.368501 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 97764 2024.10.22 02:21:17.369105 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_908_273} MergedBlockOutputStream: filled checksums 202410_1_908_273 (state Temporary) 2024.10.22 02:21:17.414861 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_848_221, 202410_849_849_0] 2024.10.22 02:21:17.437208 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} MergeTask::MergeProjectionsStage: Merge sorted 782 rows, containing 32 columns (32 merged, 0 gathered) in 3.429345452 sec., 228.03185358416903 rows/sec., 17.25 KiB/sec. 2024.10.22 02:21:17.467902 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} MergedBlockOutputStream: filled checksums 202410_1_738_150 (state Temporary) 2024.10.22 02:21:17.617724 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_908_273} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_908_273 to 202410_1_908_273 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:17.632172 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_908_273} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_907_272, 202410_908_908_0] -> 202410_1_908_273 2024.10.22 02:21:17.655112 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:21:17.843452 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_738_150 to 202410_1_738_150 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:17.849229 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_738_150} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_733_149, 202410_738_738_0] -> 202410_1_738_150 2024.10.22 02:21:17.909021 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:21:17.987369 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 12.17 MiB, will set to 1.59 GiB (RSS), difference: -7.14 MiB 2024.10.22 02:21:18.774502 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1524_85} MergeTask::MergeProjectionsStage: Merge sorted 21889 rows, containing 13 columns (13 merged, 0 gathered) in 2.385312153 sec., 9176.576731255182 rows/sec., 1.68 MiB/sec. 2024.10.22 02:21:18.882639 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1524_85} MergedBlockOutputStream: filled checksums 202410_1207_1524_85 (state Temporary) 2024.10.22 02:21:19.268349 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1303718 2024.10.22 02:21:19.348597 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1524_85} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1524_85 to 202410_1207_1524_85 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:19.357159 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1524_85} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1523_84, 202410_1524_1524_0] -> 202410_1207_1524_85 2024.10.22 02:21:19.419323 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 10.60 MiB, will set to 1.59 GiB (RSS), difference: 17.00 MiB 2024.10.22 02:21:19.451435 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:21:19.491106 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.42 MiB. 2024.10.22 02:21:20.030998 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7264 2024.10.22 02:21:20.032860 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:20.041224 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:20.157199 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_861_861_0 (state Temporary) 2024.10.22 02:21:20.360720 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_861_861_0 to 202410_868_868_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:20.370358 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 101324 2024.10.22 02:21:20.409749 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1303718 2024.10.22 02:21:20.402917 [ 1215 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_863_863_0 to 202410_868_868_0 2024.10.22 02:21:20.506917 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 13.49 MiB, will set to 1.59 GiB (RSS), difference: -3.18 MiB 2024.10.22 02:21:20.541821 [ 1215 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:20.548262 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:20.587134 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_863_863_0 to 202410_868_868_0 into Compact with storage Full 2024.10.22 02:21:20.615098 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_862_161/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:21:20.650392 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:20.657320 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_863_863_0, total 1554 rows starting from the beginning of the part 2024.10.22 02:21:20.660255 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:20.666527 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:20.703572 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_864_864_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:21:20.817625 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_865_865_0, total 1927 rows starting from the beginning of the part 2024.10.22 02:21:20.841395 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1516_1516_0 (state Temporary) 2024.10.22 02:21:20.847021 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_866_866_0, total 1533 rows starting from the beginning of the part 2024.10.22 02:21:20.869281 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_867_867_0, total 1547 rows starting from the beginning of the part 2024.10.22 02:21:20.908422 [ 1252 ] {} DatabaseCatalog: Execution took 231 ms. 2024.10.22 02:21:20.989371 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_868_868_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:21:21.114512 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1516_1516_0 to 202410_1529_1529_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:21.151685 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1524_85 to 202410_1525_1525_0 2024.10.22 02:21:21.177719 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101324 2024.10.22 02:21:21.177808 [ 1276 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:21.193491 [ 1276 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:21.245291 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1525_86} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1524_85 to 202410_1525_1525_0 into Compact with storage Full 2024.10.22 02:21:21.307988 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1525_86} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:21.324273 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1525_86} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1524_85, total 21889 rows starting from the beginning of the part 2024.10.22 02:21:21.337822 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_827_16, 202410_828_828_0, 202410_829_829_0, 202410_830_830_0, 202410_831_831_0, 202410_832_832_0] 2024.10.22 02:21:21.340281 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_827_16, 202410_828_828_0, 202410_829_829_0, 202410_830_830_0, 202410_831_831_0, 202410_832_832_0] 2024.10.22 02:21:21.353528 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1525_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_1525_1525_0, total 61 rows starting from the beginning of the part 2024.10.22 02:21:21.359055 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:21:21.362110 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:21:21.364977 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_827_16, 202410_828_828_0, 202410_829_829_0, 202410_830_830_0, 202410_831_831_0, 202410_832_832_0] 2024.10.22 02:21:22.541051 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} MergeTask::MergeProjectionsStage: Merge sorted 9641 rows, containing 4 columns (4 merged, 0 gathered) in 1.984426482 sec., 4858.330649913188 rows/sec., 75.91 KiB/sec. 2024.10.22 02:21:22.602678 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 14.96 MiB, will set to 1.59 GiB (RSS), difference: -4.57 MiB 2024.10.22 02:21:22.667586 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} MergedBlockOutputStream: filled checksums 202410_863_868_1 (state Temporary) 2024.10.22 02:21:22.714575 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_863_868_1 to 202410_863_868_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:22.721636 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_863_868_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_863_863_0, 202410_868_868_0] -> 202410_863_868_1 2024.10.22 02:21:22.877437 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1525_86} MergeTask::MergeProjectionsStage: Merge sorted 21950 rows, containing 13 columns (13 merged, 0 gathered) in 1.664568545 sec., 13186.60025502284 rows/sec., 2.41 MiB/sec. 2024.10.22 02:21:22.970429 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:21:23.144024 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1525_86} MergedBlockOutputStream: filled checksums 202410_1207_1525_86 (state Temporary) 2024.10.22 02:21:23.401466 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 13756 2024.10.22 02:21:23.519939 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1525_86} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1525_86 to 202410_1207_1525_86 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:23.524643 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1525_86} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1524_85, 202410_1525_1525_0] -> 202410_1207_1525_86 2024.10.22 02:21:23.556811 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.42 MiB. 2024.10.22 02:21:23.704322 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(7ae2e9bb26c6a08a8afa24290b698a28) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7ae2e9bb26c6a08a8afa24290b698a28, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:21:23.712143 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 7ae2e9bb26c6a08a8afa24290b698a28, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:21:23.715309 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:21:23.821724 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:23.825724 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:23.962893 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_900_900_0 (state Temporary) 2024.10.22 02:21:24.068325 [ 1152 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:21:24.203294 [ 1152 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:21:24.237480 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_900_900_0 to 202410_913_913_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:24.263764 [ 1203 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_908_273 to 202410_909_909_0 2024.10.22 02:21:24.270572 [ 1203 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:24.285709 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13756 2024.10.22 02:21:24.287401 [ 1203 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:24.456096 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 17.75 MiB, will set to 1.59 GiB (RSS), difference: 10.04 MiB 2024.10.22 02:21:24.474090 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_909_274} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_908_273 to 202410_909_909_0 into Compact with storage Full 2024.10.22 02:21:24.893134 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 123 entries to flush up to offset 97887 2024.10.22 02:21:24.923239 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_909_274} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:25.045697 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_909_274} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_908_273, total 13770 rows starting from the beginning of the part 2024.10.22 02:21:25.117668 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_909_274} MergeTreeSequentialSource: Reading 2 marks from part 202410_909_909_0, total 12 rows starting from the beginning of the part 2024.10.22 02:21:25.284635 [ 1724 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fth/brjoreidltrtqqddylqcmdxsnitik, size 3466948 2024.10.22 02:21:25.287009 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:25.333663 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 101389 2024.10.22 02:21:25.341114 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:25.396599 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_885_885_0 (state Temporary) 2024.10.22 02:21:25.455472 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_885_885_0 to 202410_891_891_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:25.533094 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 97887 2024.10.22 02:21:25.816883 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_849_222, 202410_850_850_0] 2024.10.22 02:21:25.817723 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_849_222, 202410_850_850_0] 2024.10.22 02:21:25.830991 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_849_222, 202410_850_850_0] 2024.10.22 02:21:25.894673 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:25.897153 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 02:21:25.897328 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:25.944756 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 02:21:25.959503 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 02:21:25.988449 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 02:21:26.013059 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 02:21:26.048823 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 02:21:26.053065 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_909_274} MergeTask::MergeProjectionsStage: Merge sorted 13782 rows, containing 12 columns (12 merged, 0 gathered) in 1.688559913 sec., 8161.984596397321 rows/sec., 1.27 MiB/sec. 2024.10.22 02:21:26.064100 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column DOMContentLoadedTiming 2024.10.22 02:21:26.171743 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1517_1517_0 (state Temporary) 2024.10.22 02:21:26.201369 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_909_274} MergedBlockOutputStream: filled checksums 202410_1_909_274 (state Temporary) 2024.10.22 02:21:26.257171 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1517_1517_0 to 202410_1530_1530_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:26.275769 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1525_86 to 202410_1526_1526_0 2024.10.22 02:21:26.280412 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:26.282677 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:26.348547 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101389 2024.10.22 02:21:26.430830 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 02:21:26.431359 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/sqf/dsssnnwfpgrkcmiawyiaupbgnefor (LogSeriesLimiter: on interval from 2024-10-22 02:21:08 to 2024-10-22 02:21:26 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:21:26.449330 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1526_87} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1525_86 to 202410_1526_1526_0 into Compact with storage Full 2024.10.22 02:21:26.618869 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 18.97 MiB, will set to 1.59 GiB (RSS), difference: -4.38 MiB 2024.10.22 02:21:26.695354 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_909_274} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_909_274 to 202410_1_909_274 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:26.699123 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_909_274} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_908_273, 202410_909_909_0] -> 202410_1_909_274 2024.10.22 02:21:26.741506 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:21:26.765602 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1526_87} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:26.773084 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1526_87} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1525_86, total 21950 rows starting from the beginning of the part 2024.10.22 02:21:26.800734 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1526_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_1526_1526_0, total 47 rows starting from the beginning of the part 2024.10.22 02:21:27.515354 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1305643 2024.10.22 02:21:27.558799 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 7306 2024.10.22 02:21:28.261116 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1526_87} MergeTask::MergeProjectionsStage: Merge sorted 21997 rows, containing 13 columns (13 merged, 0 gathered) in 1.919825999 sec., 11457.809203259989 rows/sec., 2.09 MiB/sec. 2024.10.22 02:21:28.436598 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1526_87} MergedBlockOutputStream: filled checksums 202410_1207_1526_87 (state Temporary) 2024.10.22 02:21:28.508837 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:28.514498 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:28.631244 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_862_862_0 (state Temporary) 2024.10.22 02:21:28.640355 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 16.59 MiB, will set to 1.59 GiB (RSS), difference: 65.98 MiB 2024.10.22 02:21:28.698620 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_862_862_0 to 202410_869_869_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:28.754082 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1305643 2024.10.22 02:21:28.773321 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1526_87} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1526_87 to 202410_1207_1526_87 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:28.787757 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1526_87} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1525_86, 202410_1526_1526_0] -> 202410_1207_1526_87 2024.10.22 02:21:28.838804 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.42 MiB. 2024.10.22 02:21:28.875639 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 747 2024.10.22 02:21:29.147118 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:29.148950 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:29.206961 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_729_729_0 (state Temporary) 2024.10.22 02:21:29.485458 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_729_729_0 to 202410_739_739_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:29.574266 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 747 2024.10.22 02:21:29.657050 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/xxz/qazsdnftjpycnqeewewepqipbpedt. 2024.10.22 02:21:30.368004 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 101446 2024.10.22 02:21:30.532897 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:30.539540 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:30.545681 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 2.00 GiB, free memory in arenas 18.79 MiB, will set to 1.58 GiB (RSS), difference: 106.20 MiB 2024.10.22 02:21:30.574059 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1518_1518_0 (state Temporary) 2024.10.22 02:21:30.639974 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1518_1518_0 to 202410_1531_1531_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:30.660257 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1526_87 to 202410_1527_1527_0 2024.10.22 02:21:30.667964 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:30.671211 [ 1183 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:30.681008 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101446 2024.10.22 02:21:30.716786 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1527_88} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1526_87 to 202410_1527_1527_0 into Compact with storage Full 2024.10.22 02:21:30.861722 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1527_88} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:30.866396 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1527_88} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1526_87, total 21997 rows starting from the beginning of the part 2024.10.22 02:21:30.942921 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1527_88} MergeTreeSequentialSource: Reading 2 marks from part 202410_1527_1527_0, total 63 rows starting from the beginning of the part 2024.10.22 02:21:31.351898 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:21:31.353605 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:21:31.648270 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1426_44, 202410_1427_1427_0, 202410_1428_1428_0, 202410_1429_1429_0, 202410_1430_1430_0, 202410_1431_1431_0] 2024.10.22 02:21:31.650710 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1426_44, 202410_1427_1427_0, 202410_1428_1428_0, 202410_1429_1429_0, 202410_1430_1430_0, 202410_1431_1431_0] 2024.10.22 02:21:31.687543 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_862_161/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:21:31.772728 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1426_44, 202410_1427_1427_0, 202410_1428_1428_0, 202410_1429_1429_0, 202410_1430_1430_0, 202410_1431_1431_0] 2024.10.22 02:21:31.789164 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 13771 2024.10.22 02:21:31.942773 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a3246305c41cacaa19b2d911dd47ec53) : [0, 2172026]: Setting download as failed: File segment: [0, 2172026], key: a3246305c41cacaa19b2d911dd47ec53, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:21:31.946564 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a3246305c41cacaa19b2d911dd47ec53): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:21:32.024866 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:32.024620 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a3246305c41cacaa19b2d911dd47ec53, offset: 0, size: 0 2024.10.22 02:21:32.028111 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:32.028730 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a3246305c41cacaa19b2d911dd47ec53 for removal 2024.10.22 02:21:32.033766 [ 1476 ] {} CacheMetadata: Key a3246305c41cacaa19b2d911dd47ec53 is removed from metadata 2024.10.22 02:21:32.100911 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_901_901_0 (state Temporary) 2024.10.22 02:21:32.364776 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 5 old parts to remove. Parts: [202410_165_165_0, 202410_166_166_0, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0] 2024.10.22 02:21:32.366600 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from filesystem (serially): Parts: [202410_165_165_0, 202410_166_166_0, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0] 2024.10.22 02:21:32.497537 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from memory: Parts: [202410_165_165_0, 202410_166_166_0, 202410_167_167_0, 202410_168_168_0, 202410_169_169_0] 2024.10.22 02:21:32.553378 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 19.11 MiB, will set to 1.58 GiB (RSS), difference: 16.39 MiB 2024.10.22 02:21:32.559707 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_901_901_0 to 202410_914_914_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:32.595774 [ 1258 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_909_274 to 202410_910_910_0 2024.10.22 02:21:32.604974 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13771 2024.10.22 02:21:32.630074 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1527_88} MergeTask::MergeProjectionsStage: Merge sorted 22060 rows, containing 13 columns (13 merged, 0 gathered) in 1.905751037 sec., 11575.488913140756 rows/sec., 2.11 MiB/sec. 2024.10.22 02:21:32.629784 [ 1258 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:32.635885 [ 1258 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.32 GiB. 2024.10.22 02:21:32.679978 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_910_275} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_909_274 to 202410_910_910_0 into Compact with storage Full 2024.10.22 02:21:32.699782 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1527_88} MergedBlockOutputStream: filled checksums 202410_1207_1527_88 (state Temporary) 2024.10.22 02:21:32.720828 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_910_275} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:32.728821 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_910_275} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_909_274, total 13782 rows starting from the beginning of the part 2024.10.22 02:21:32.772022 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_910_275} MergeTreeSequentialSource: Reading 2 marks from part 202410_910_910_0, total 25 rows starting from the beginning of the part 2024.10.22 02:21:32.787947 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(31508334d6474b97dc7b7a42a13e7f1a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 31508334d6474b97dc7b7a42a13e7f1a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 02:21:32.808029 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 31508334d6474b97dc7b7a42a13e7f1a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 02:21:32.813194 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:21:33.055208 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 121 entries to flush up to offset 98008 2024.10.22 02:21:33.169489 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1527_88} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1527_88 to 202410_1207_1527_88 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:33.172025 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1527_88} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1526_87, 202410_1527_1527_0] -> 202410_1207_1527_88 2024.10.22 02:21:33.227045 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.42 MiB. 2024.10.22 02:21:33.253180 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RegionID 2024.10.22 02:21:33.270451 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RegionID 2024.10.22 02:21:33.291323 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RegionID 2024.10.22 02:21:33.314472 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RegionID 2024.10.22 02:21:33.438985 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RegionID 2024.10.22 02:21:33.448701 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_850_223, 202410_851_851_0] 2024.10.22 02:21:33.450848 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:33.450520 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_850_223, 202410_851_851_0] 2024.10.22 02:21:33.452818 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:33.458444 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_850_223, 202410_851_851_0] 2024.10.22 02:21:33.551055 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RegionID 2024.10.22 02:21:33.561347 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RegionID 2024.10.22 02:21:33.575492 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RegionID 2024.10.22 02:21:33.577705 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_886_886_0 (state Temporary) 2024.10.22 02:21:33.594299 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RegionID 2024.10.22 02:21:33.726407 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zxb/drxjkllojcfsmzelgzividleasphy (LogSeriesLimiter: on interval from 2024-10-22 02:21:26 to 2024-10-22 02:21:33 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:21:33.839435 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_886_886_0 to 202410_892_892_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:33.917823 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_910_275} MergeTask::MergeProjectionsStage: Merge sorted 13807 rows, containing 12 columns (12 merged, 0 gathered) in 1.244345484 sec., 11095.793071564682 rows/sec., 1.73 MiB/sec. 2024.10.22 02:21:33.991267 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 98008 2024.10.22 02:21:34.078212 [ 1247 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_887_28 to 202410_892_892_0 2024.10.22 02:21:34.094913 [ 1247 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:34.097151 [ 1247 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:34.151837 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_910_275} MergedBlockOutputStream: filled checksums 202410_1_910_275 (state Temporary) 2024.10.22 02:21:34.161470 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(83bc672490748ae915da33e510911f50) : [0, 1066976]: Setting download as failed: File segment: [0, 1066976], key: 83bc672490748ae915da33e510911f50, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:21:34.163656 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(83bc672490748ae915da33e510911f50): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:21:34.228310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 83bc672490748ae915da33e510911f50, offset: 0, size: 0 2024.10.22 02:21:34.231638 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 83bc672490748ae915da33e510911f50 for removal 2024.10.22 02:21:34.233533 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_887_28 to 202410_892_892_0 into Compact with storage Full 2024.10.22 02:21:34.234353 [ 1476 ] {} CacheMetadata: Key 83bc672490748ae915da33e510911f50 is removed from metadata 2024.10.22 02:21:34.319950 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:34.341795 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_887_28, total 14054 rows starting from the beginning of the part 2024.10.22 02:21:34.520320 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_910_275} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_910_275 to 202410_1_910_275 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:34.529692 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_910_275} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_909_274, 202410_910_910_0] -> 202410_1_910_275 2024.10.22 02:21:34.542051 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_888_888_0, total 129 rows starting from the beginning of the part 2024.10.22 02:21:34.555695 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:21:34.639970 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_889_889_0, total 110 rows starting from the beginning of the part 2024.10.22 02:21:34.698030 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 101533 2024.10.22 02:21:34.830536 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_890_890_0, total 116 rows starting from the beginning of the part 2024.10.22 02:21:34.897492 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_891_891_0, total 123 rows starting from the beginning of the part 2024.10.22 02:21:34.985059 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_892_892_0, total 121 rows starting from the beginning of the part 2024.10.22 02:21:35.033374 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:35.116364 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:35.227123 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1519_1519_0 (state Temporary) 2024.10.22 02:21:35.512667 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1519_1519_0 to 202410_1532_1532_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:35.558802 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1527_88 to 202410_1532_1532_0 2024.10.22 02:21:35.578527 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101533 2024.10.22 02:21:35.669427 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.02 MiB using storage policy from min volume index 0 2024.10.22 02:21:35.759317 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1307183 2024.10.22 02:21:35.773701 [ 1221 ] {} DiskLocal: Reserved 1.02 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:35.794157 [ 1221 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 253 ms. 2024.10.22 02:21:35.880882 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 30319 bytes for 4eff7c5d1ca122bf90ea251d050511c0:0) 2024.10.22 02:21:35.885179 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c228fffcbcae0b9e5404454d09795899, offset: 0, size: 98029 2024.10.22 02:21:35.890529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c228fffcbcae0b9e5404454d09795899 for removal 2024.10.22 02:21:35.895823 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7e0f5c348c0aedae31f30b58c341b7f0, offset: 0, size: 160 2024.10.22 02:21:35.913567 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7e0f5c348c0aedae31f30b58c341b7f0 for removal 2024.10.22 02:21:35.931782 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1527_88 to 202410_1532_1532_0 into Compact with storage Full 2024.10.22 02:21:35.979118 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:35.995044 [ 1476 ] {} CacheMetadata: Key c228fffcbcae0b9e5404454d09795899 is removed from metadata 2024.10.22 02:21:36.001623 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1527_88, total 22060 rows starting from the beginning of the part 2024.10.22 02:21:36.048703 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_1528_1528_0, total 64 rows starting from the beginning of the part 2024.10.22 02:21:36.050754 [ 1476 ] {} CacheMetadata: Key 7e0f5c348c0aedae31f30b58c341b7f0 is removed from metadata 2024.10.22 02:21:36.079480 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_1529_1529_0, total 59 rows starting from the beginning of the part 2024.10.22 02:21:36.250503 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_1530_1530_0, total 65 rows starting from the beginning of the part 2024.10.22 02:21:36.345201 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_1531_1531_0, total 57 rows starting from the beginning of the part 2024.10.22 02:21:36.552566 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} MergeTreeSequentialSource: Reading 2 marks from part 202410_1532_1532_0, total 87 rows starting from the beginning of the part 2024.10.22 02:21:36.977987 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:36.984419 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:37.078007 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 748 2024.10.22 02:21:37.146806 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_863_863_0 (state Temporary) 2024.10.22 02:21:37.212234 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1d5e5d4f9701f1a934051d6bf296b236) : [0, 301387]: Setting download as failed: File segment: [0, 301387], key: 1d5e5d4f9701f1a934051d6bf296b236, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:21:37.221779 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1d5e5d4f9701f1a934051d6bf296b236): No space left in cache to reserve 301388 bytes, will continue without cache download 2024.10.22 02:21:37.357016 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1d5e5d4f9701f1a934051d6bf296b236, offset: 0, size: 0 2024.10.22 02:21:37.359618 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1d5e5d4f9701f1a934051d6bf296b236 for removal 2024.10.22 02:21:37.363616 [ 1476 ] {} CacheMetadata: Key 1d5e5d4f9701f1a934051d6bf296b236 is removed from metadata 2024.10.22 02:21:37.393166 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:37.398762 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:37.469332 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_863_863_0 to 202410_870_870_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:37.529587 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1307183 2024.10.22 02:21:37.578334 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_730_730_0 (state Temporary) 2024.10.22 02:21:37.651978 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} MergeTask::MergeProjectionsStage: Merge sorted 14653 rows, containing 13 columns (13 merged, 0 gathered) in 3.435371802 sec., 4265.331627706013 rows/sec., 1.85 MiB/sec. 2024.10.22 02:21:37.678493 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_730_730_0 to 202410_740_740_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:37.819975 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} MergedBlockOutputStream: filled checksums 202410_747_892_29 (state Temporary) 2024.10.22 02:21:37.952362 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 748 2024.10.22 02:21:38.072993 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_892_29 to 202410_747_892_29 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:38.079901 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_892_29} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_887_28, 202410_892_892_0] -> 202410_747_892_29 2024.10.22 02:21:38.271969 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 18.26 MiB. 2024.10.22 02:21:38.453311 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 13.32 MiB, will set to 1.58 GiB (RSS), difference: -1.22 MiB 2024.10.22 02:21:38.745413 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 10 file segments (while reserving 199314 bytes for 2f7b19b8d72f5cd0b3ed31554158b9df:0) 2024.10.22 02:21:38.748020 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9d4cadc87dd7d71929fa1bb2bada37e5, offset: 0, size: 146 2024.10.22 02:21:38.752368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9d4cadc87dd7d71929fa1bb2bada37e5 for removal 2024.10.22 02:21:38.754483 [ 1476 ] {} CacheMetadata: Key 9d4cadc87dd7d71929fa1bb2bada37e5 is removed from metadata 2024.10.22 02:21:38.755600 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 25ac660b73fffbbe89e53cc95d4d0456, offset: 0, size: 40857 2024.10.22 02:21:38.758528 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 25ac660b73fffbbe89e53cc95d4d0456 for removal 2024.10.22 02:21:38.760968 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: de4a310a60118177bff4ebed00f70067, offset: 0, size: 162 2024.10.22 02:21:38.864158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key de4a310a60118177bff4ebed00f70067 for removal 2024.10.22 02:21:38.865674 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a55b409c397402ca3062523437d716eb, offset: 0, size: 163 2024.10.22 02:21:38.867013 [ 1476 ] {} CacheMetadata: Key 25ac660b73fffbbe89e53cc95d4d0456 is removed from metadata 2024.10.22 02:21:38.869501 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a55b409c397402ca3062523437d716eb for removal 2024.10.22 02:21:38.882746 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0b36414b51388ba06375b60b565f4ced, offset: 0, size: 204 2024.10.22 02:21:38.888703 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0b36414b51388ba06375b60b565f4ced for removal 2024.10.22 02:21:38.890874 [ 1476 ] {} CacheMetadata: Key a55b409c397402ca3062523437d716eb is removed from metadata 2024.10.22 02:21:38.895447 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1e796c7f3239d5bb628d1ba81faa36e1, offset: 0, size: 47209 2024.10.22 02:21:38.898398 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1e796c7f3239d5bb628d1ba81faa36e1 for removal 2024.10.22 02:21:38.899756 [ 1476 ] {} CacheMetadata: Key 0b36414b51388ba06375b60b565f4ced is removed from metadata 2024.10.22 02:21:38.901724 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 255c58684af99c00e01784b88951c66d, offset: 0, size: 2445 2024.10.22 02:21:38.905429 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 255c58684af99c00e01784b88951c66d for removal 2024.10.22 02:21:38.911296 [ 1476 ] {} CacheMetadata: Key 1e796c7f3239d5bb628d1ba81faa36e1 is removed from metadata 2024.10.22 02:21:38.925789 [ 1476 ] {} CacheMetadata: Key 255c58684af99c00e01784b88951c66d is removed from metadata 2024.10.22 02:21:38.914787 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4eff7c5d1ca122bf90ea251d050511c0, offset: 0, size: 30319 2024.10.22 02:21:38.932942 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4eff7c5d1ca122bf90ea251d050511c0 for removal 2024.10.22 02:21:38.937011 [ 1476 ] {} CacheMetadata: Key de4a310a60118177bff4ebed00f70067 is removed from metadata 2024.10.22 02:21:38.939124 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3d4c171138ebc99f125b2c94cb3dc2ff, offset: 0, size: 162 2024.10.22 02:21:38.943364 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3d4c171138ebc99f125b2c94cb3dc2ff for removal 2024.10.22 02:21:38.946163 [ 1476 ] {} CacheMetadata: Key 4eff7c5d1ca122bf90ea251d050511c0 is removed from metadata 2024.10.22 02:21:38.948027 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7a308d43e4497188218328ec1b17bb0e, offset: 0, size: 152 2024.10.22 02:21:38.953225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7a308d43e4497188218328ec1b17bb0e for removal 2024.10.22 02:21:38.954495 [ 1476 ] {} CacheMetadata: Key 3d4c171138ebc99f125b2c94cb3dc2ff is removed from metadata 2024.10.22 02:21:38.962550 [ 1476 ] {} CacheMetadata: Key 7a308d43e4497188218328ec1b17bb0e is removed from metadata 2024.10.22 02:21:39.092497 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} MergeTask::MergeProjectionsStage: Merge sorted 22392 rows, containing 13 columns (13 merged, 0 gathered) in 3.255824011 sec., 6877.521611839971 rows/sec., 1.25 MiB/sec. 2024.10.22 02:21:39.168169 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} MergedBlockOutputStream: filled checksums 202410_1207_1532_89 (state Temporary) 2024.10.22 02:21:39.216808 [ 1255 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:21:39.245639 [ 1255 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:21:39.480367 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1532_89 to 202410_1207_1532_89 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:39.506773 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1532_89} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1527_88, 202410_1532_1532_0] -> 202410_1207_1532_89 2024.10.22 02:21:39.600369 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 101621 2024.10.22 02:21:39.649752 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.67 MiB. 2024.10.22 02:21:39.860420 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:39.863284 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:40.051125 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1520_1520_0 (state Temporary) 2024.10.22 02:21:40.109467 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 13796 2024.10.22 02:21:40.195406 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1520_1520_0 to 202410_1533_1533_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:40.213144 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:40.222782 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:40.263466 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_902_902_0 (state Temporary) 2024.10.22 02:21:40.264864 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101621 2024.10.22 02:21:40.316951 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_902_902_0 to 202410_915_915_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:40.355277 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_910_275 to 202410_911_911_0 2024.10.22 02:21:40.361597 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:40.363913 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:40.368589 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13796 2024.10.22 02:21:40.402925 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_911_276} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_910_275 to 202410_911_911_0 into Compact with storage Full 2024.10.22 02:21:40.491546 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0573b91f3d4fecbb6660c806b9e26ecf) : [0, 426421]: Setting download as failed: File segment: [0, 426421], key: 0573b91f3d4fecbb6660c806b9e26ecf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:21:40.493559 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0573b91f3d4fecbb6660c806b9e26ecf): No space left in cache to reserve 426422 bytes, will continue without cache download 2024.10.22 02:21:40.544632 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_911_276} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:40.556652 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_911_276} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_910_275, total 13807 rows starting from the beginning of the part 2024.10.22 02:21:40.628977 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0573b91f3d4fecbb6660c806b9e26ecf, offset: 0, size: 0 2024.10.22 02:21:40.648059 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0573b91f3d4fecbb6660c806b9e26ecf for removal 2024.10.22 02:21:40.650715 [ 1476 ] {} CacheMetadata: Key 0573b91f3d4fecbb6660c806b9e26ecf is removed from metadata 2024.10.22 02:21:40.743495 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_911_276} MergeTreeSequentialSource: Reading 2 marks from part 202410_911_911_0, total 15 rows starting from the beginning of the part 2024.10.22 02:21:41.243159 [ 1274 ] {} DatabaseCatalog: Execution took 317 ms. 2024.10.22 02:21:41.352895 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:21:41.354373 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:21:41.452087 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 4489 bytes for 4fba7cfa9130f8356a4cb59e7422faf8:0) 2024.10.22 02:21:41.454337 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2f7b19b8d72f5cd0b3ed31554158b9df, offset: 0, size: 199314 2024.10.22 02:21:41.457828 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2f7b19b8d72f5cd0b3ed31554158b9df for removal 2024.10.22 02:21:41.460207 [ 1476 ] {} CacheMetadata: Key 2f7b19b8d72f5cd0b3ed31554158b9df is removed from metadata 2024.10.22 02:21:41.460831 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1fbb020be00b55a245f50828aae5e627, offset: 0, size: 150 2024.10.22 02:21:41.464319 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1fbb020be00b55a245f50828aae5e627 for removal 2024.10.22 02:21:41.466906 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b850bf329075f1396bc1fd408eefd347, offset: 0, size: 153 2024.10.22 02:21:41.468277 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_851_224, 202410_852_852_0, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0, 202410_856_856_0] 2024.10.22 02:21:41.469511 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b850bf329075f1396bc1fd408eefd347 for removal 2024.10.22 02:21:41.470923 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_851_224, 202410_852_852_0, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0, 202410_856_856_0] 2024.10.22 02:21:41.474224 [ 1476 ] {} CacheMetadata: Key b850bf329075f1396bc1fd408eefd347 is removed from metadata 2024.10.22 02:21:41.492827 [ 1476 ] {} CacheMetadata: Key 1fbb020be00b55a245f50828aae5e627 is removed from metadata 2024.10.22 02:21:41.524013 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 119 entries to flush up to offset 98127 2024.10.22 02:21:41.703962 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_851_224, 202410_852_852_0, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0, 202410_856_856_0] 2024.10.22 02:21:41.781378 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:41.786798 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:41.835655 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_887_887_0 (state Temporary) 2024.10.22 02:21:41.870951 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_911_276} MergeTask::MergeProjectionsStage: Merge sorted 13822 rows, containing 12 columns (12 merged, 0 gathered) in 1.501602127 sec., 9204.835123412156 rows/sec., 1.44 MiB/sec. 2024.10.22 02:21:41.876224 [ 1259 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 02:21:42.043103 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_911_276} MergedBlockOutputStream: filled checksums 202410_1_911_276 (state Temporary) 2024.10.22 02:21:42.055736 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_862_161} MergeTask::MergeProjectionsStage: Merge sorted 1307667 rows, containing 4 columns (4 merged, 0 gathered) in 38.362056532 sec., 34087.510373934194 rows/sec., 511.32 KiB/sec. 2024.10.22 02:21:42.107055 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_887_887_0 to 202410_893_893_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:42.124194 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_862_161} MergedBlockOutputStream: filled checksums 202410_1_862_161 (state Temporary) 2024.10.22 02:21:42.158719 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 98127 2024.10.22 02:21:42.260469 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_911_276} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_911_276 to 202410_1_911_276 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:42.266460 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_911_276} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_910_275, 202410_911_911_0] -> 202410_1_911_276 2024.10.22 02:21:42.332023 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 12.71 MiB, will set to 1.57 GiB (RSS), difference: 10.86 MiB 2024.10.22 02:21:42.391292 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:21:42.533032 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 7 file segments (while reserving 134536 bytes for 0e95737570d1f10f40c4ce1ce501da10:0) 2024.10.22 02:21:42.555380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e6a635127208a705d62bf3913a1fcb10, offset: 0, size: 79468 2024.10.22 02:21:42.560152 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e6a635127208a705d62bf3913a1fcb10 for removal 2024.10.22 02:21:42.562715 [ 1476 ] {} CacheMetadata: Key e6a635127208a705d62bf3913a1fcb10 is removed from metadata 2024.10.22 02:21:42.563132 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4759f972a3ede0a60a59192092b22ed0, offset: 0, size: 153 2024.10.22 02:21:42.574933 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4759f972a3ede0a60a59192092b22ed0 for removal 2024.10.22 02:21:42.579502 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3573cc74a7784f73a33d37033b570fd7, offset: 0, size: 73511 2024.10.22 02:21:42.579487 [ 1476 ] {} CacheMetadata: Key 4759f972a3ede0a60a59192092b22ed0 is removed from metadata 2024.10.22 02:21:42.584472 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3573cc74a7784f73a33d37033b570fd7 for removal 2024.10.22 02:21:42.590602 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ad377b19b6668beeb573b7ce3316599f, offset: 0, size: 153 2024.10.22 02:21:42.591884 [ 1476 ] {} CacheMetadata: Key 3573cc74a7784f73a33d37033b570fd7 is removed from metadata 2024.10.22 02:21:42.595446 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ad377b19b6668beeb573b7ce3316599f for removal 2024.10.22 02:21:42.598863 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4fba7cfa9130f8356a4cb59e7422faf8, offset: 0, size: 4489 2024.10.22 02:21:42.605357 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4fba7cfa9130f8356a4cb59e7422faf8 for removal 2024.10.22 02:21:42.612939 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5c6d76005a806f02b5efca0d357dad5d, offset: 0, size: 152 2024.10.22 02:21:42.621472 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5c6d76005a806f02b5efca0d357dad5d for removal 2024.10.22 02:21:42.640587 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4b19dd969de4d56c2a4d0166620ad7fb, offset: 0, size: 177 2024.10.22 02:21:42.646056 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4b19dd969de4d56c2a4d0166620ad7fb for removal 2024.10.22 02:21:42.601796 [ 1476 ] {} CacheMetadata: Key ad377b19b6668beeb573b7ce3316599f is removed from metadata 2024.10.22 02:21:42.717816 [ 1476 ] {} CacheMetadata: Key 4b19dd969de4d56c2a4d0166620ad7fb is removed from metadata 2024.10.22 02:21:42.733344 [ 1476 ] {} CacheMetadata: Key 5c6d76005a806f02b5efca0d357dad5d is removed from metadata 2024.10.22 02:21:42.743126 [ 1476 ] {} CacheMetadata: Key 4fba7cfa9130f8356a4cb59e7422faf8 is removed from metadata 2024.10.22 02:21:42.937248 [ 1738 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/sqf/dsssnnwfpgrkcmiawyiaupbgnefor, size 2927169 2024.10.22 02:21:42.959316 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:42.960776 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:42.979956 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 02:21:42.991804 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 02:21:43.001289 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 02:21:43.013660 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 02:21:43.031784 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 02:21:43.052836 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_862_161/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:21:43.059345 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 02:21:43.071223 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column DOMCompleteTiming 2024.10.22 02:21:43.123136 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/efj/vvnzcavtargbsquaakhysvleokncg (LogSeriesLimiter: on interval from 2024-10-22 02:21:33 to 2024-10-22 02:21:43 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:21:43.210925 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_809_809_0, 202410_810_810_0, 202410_811_811_0, 202410_812_812_0, 202410_813_813_0, 202410_814_814_0] 2024.10.22 02:21:43.214094 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_809_809_0, 202410_810_810_0, 202410_811_811_0, 202410_812_812_0, 202410_813_813_0, 202410_814_814_0] 2024.10.22 02:21:43.242676 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_809_809_0, 202410_810_810_0, 202410_811_811_0, 202410_812_812_0, 202410_813_813_0, 202410_814_814_0] 2024.10.22 02:21:43.933019 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 131343 bytes for ddac84491113f5a85d799c76343b7c64:0) 2024.10.22 02:21:43.937045 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0e95737570d1f10f40c4ce1ce501da10, offset: 0, size: 134536 2024.10.22 02:21:43.942186 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0e95737570d1f10f40c4ce1ce501da10 for removal 2024.10.22 02:21:43.947782 [ 1476 ] {} CacheMetadata: Key 0e95737570d1f10f40c4ce1ce501da10 is removed from metadata 2024.10.22 02:21:43.949004 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b93bb8c72c7653306a0c2ab046a7a781, offset: 0, size: 163 2024.10.22 02:21:43.952638 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b93bb8c72c7653306a0c2ab046a7a781 for removal 2024.10.22 02:21:43.957885 [ 1476 ] {} CacheMetadata: Key b93bb8c72c7653306a0c2ab046a7a781 is removed from metadata 2024.10.22 02:21:44.282726 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 100 entries to flush up to offset 101721 2024.10.22 02:21:44.438762 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 10.53 MiB, will set to 1.56 GiB (RSS), difference: -7.30 MiB 2024.10.22 02:21:44.535484 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1308723 2024.10.22 02:21:44.703385 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:44.712706 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:44.891661 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1521_1521_0 (state Temporary) 2024.10.22 02:21:44.905328 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_862_161} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_862_161 to 202410_1_862_161 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:44.910380 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_862_161} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_856_160, 202410_857_862_1] -> 202410_1_862_161 2024.10.22 02:21:44.941489 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:21:44.948983 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1521_1521_0 to 202410_1534_1534_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:44.988972 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101721 2024.10.22 02:21:45.074396 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 61371 bytes for 972b40982ba2820a587c17270c652b41:0) 2024.10.22 02:21:45.077169 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ddac84491113f5a85d799c76343b7c64, offset: 0, size: 131343 2024.10.22 02:21:45.083977 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ddac84491113f5a85d799c76343b7c64 for removal 2024.10.22 02:21:45.096481 [ 1476 ] {} CacheMetadata: Key ddac84491113f5a85d799c76343b7c64 is removed from metadata 2024.10.22 02:21:45.097326 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 99c47b9e73daa14c12a3f2b7d487485e, offset: 0, size: 163 2024.10.22 02:21:45.102563 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 99c47b9e73daa14c12a3f2b7d487485e for removal 2024.10.22 02:21:45.125140 [ 1476 ] {} CacheMetadata: Key 99c47b9e73daa14c12a3f2b7d487485e is removed from metadata 2024.10.22 02:21:45.246236 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:45.250240 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:45.454335 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 749 2024.10.22 02:21:45.554875 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_864_864_0 (state Temporary) 2024.10.22 02:21:45.634690 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:45.639625 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:45.657194 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_864_864_0 to 202410_871_871_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:45.697183 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_731_731_0 (state Temporary) 2024.10.22 02:21:45.804239 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1308723 2024.10.22 02:21:45.870210 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_731_731_0 to 202410_741_741_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:45.955815 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 749 2024.10.22 02:21:46.147287 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_181_181_0 (state Temporary) 2024.10.22 02:21:46.372220 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.98 MiB, will set to 1.56 GiB (RSS), difference: -5.65 MiB 2024.10.22 02:21:46.400379 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(85c5c9c37e793b938d23ea3e630b7a79) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 85c5c9c37e793b938d23ea3e630b7a79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 02:21:46.419078 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 85c5c9c37e793b938d23ea3e630b7a79, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 02:21:46.422073 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:21:47.801748 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 127996 bytes for 00e797d98589cafac6b1fb991b0472c7:0) 2024.10.22 02:21:47.810318 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 972b40982ba2820a587c17270c652b41, offset: 0, size: 61371 2024.10.22 02:21:47.815811 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 972b40982ba2820a587c17270c652b41 for removal 2024.10.22 02:21:47.817724 [ 1476 ] {} CacheMetadata: Key 972b40982ba2820a587c17270c652b41 is removed from metadata 2024.10.22 02:21:47.819258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 99c1ed3969c424353f9900a061519ca8, offset: 0, size: 59409 2024.10.22 02:21:47.827220 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 99c1ed3969c424353f9900a061519ca8 for removal 2024.10.22 02:21:47.837773 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2814fe2d7726f964920660c19d7941c0, offset: 0, size: 152 2024.10.22 02:21:47.844521 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2814fe2d7726f964920660c19d7941c0 for removal 2024.10.22 02:21:47.847503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 71314cf1d67bcf4795080fe1a2512553, offset: 0, size: 152 2024.10.22 02:21:47.848835 [ 1476 ] {} CacheMetadata: Key 2814fe2d7726f964920660c19d7941c0 is removed from metadata 2024.10.22 02:21:47.849225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 71314cf1d67bcf4795080fe1a2512553 for removal 2024.10.22 02:21:47.856924 [ 1476 ] {} CacheMetadata: Key 71314cf1d67bcf4795080fe1a2512553 is removed from metadata 2024.10.22 02:21:47.870537 [ 1476 ] {} CacheMetadata: Key 99c1ed3969c424353f9900a061519ca8 is removed from metadata 2024.10.22 02:21:47.871375 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 13808 2024.10.22 02:21:48.076091 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:48.078976 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:48.120149 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_903_903_0 (state Temporary) 2024.10.22 02:21:48.188069 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_903_903_0 to 202410_916_916_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:48.293261 [ 1173 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_911_276 to 202410_912_912_0 2024.10.22 02:21:48.321164 [ 1173 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:48.326312 [ 1173 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:48.350443 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_912_277} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_911_276 to 202410_912_912_0 into Compact with storage Full 2024.10.22 02:21:48.355890 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13808 2024.10.22 02:21:48.511319 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_912_277} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:48.525904 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_912_277} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_911_276, total 13822 rows starting from the beginning of the part 2024.10.22 02:21:48.555378 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_912_277} MergeTreeSequentialSource: Reading 2 marks from part 202410_912_912_0, total 14 rows starting from the beginning of the part 2024.10.22 02:21:48.999637 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 101781 2024.10.22 02:21:49.449548 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_912_277} MergeTask::MergeProjectionsStage: Merge sorted 13836 rows, containing 12 columns (12 merged, 0 gathered) in 1.106136258 sec., 12508.404728561027 rows/sec., 1.95 MiB/sec. 2024.10.22 02:21:49.473517 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:49.479588 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:49.647615 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1522_1522_0 (state Temporary) 2024.10.22 02:21:49.690564 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_912_277} MergedBlockOutputStream: filled checksums 202410_1_912_277 (state Temporary) 2024.10.22 02:21:49.700824 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1522_1522_0 to 202410_1535_1535_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:49.753306 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 31755 bytes for a70e0bdd1a6a51524fff5f39beeed3ce:0) 2024.10.22 02:21:49.755624 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dfbd713a6ad28f9256d3537e5a8164d5, offset: 0, size: 37114 2024.10.22 02:21:49.759504 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dfbd713a6ad28f9256d3537e5a8164d5 for removal 2024.10.22 02:21:49.761785 [ 1476 ] {} CacheMetadata: Key dfbd713a6ad28f9256d3537e5a8164d5 is removed from metadata 2024.10.22 02:21:49.763723 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1ec0983d47a83b97bbd958b82be13618, offset: 0, size: 152 2024.10.22 02:21:49.766597 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1ec0983d47a83b97bbd958b82be13618 for removal 2024.10.22 02:21:49.770318 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 60 entries to flush up to offset 98187 2024.10.22 02:21:49.785466 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.30 MiB, will set to 1.56 GiB (RSS), difference: -6.15 MiB 2024.10.22 02:21:49.791313 [ 1476 ] {} CacheMetadata: Key 1ec0983d47a83b97bbd958b82be13618 is removed from metadata 2024.10.22 02:21:49.899198 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101781 2024.10.22 02:21:50.132242 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:50.140360 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:50.145232 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_912_277} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_912_277 to 202410_1_912_277 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:50.149514 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_912_277} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_911_276, 202410_912_912_0] -> 202410_1_912_277 2024.10.22 02:21:50.166170 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:21:50.348997 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_888_888_0 (state Temporary) 2024.10.22 02:21:50.497255 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_888_888_0 to 202410_894_894_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:50.725237 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 28773 bytes for b8b39d594fbdfd46acd75f20b52d9c5d:0) 2024.10.22 02:21:50.728755 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a769cf7817f317dc1bf8b8526db97f38, offset: 0, size: 9433 2024.10.22 02:21:50.738280 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a769cf7817f317dc1bf8b8526db97f38 for removal 2024.10.22 02:21:50.742654 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 034cf89423e14738aae2240c893c1cdf, offset: 0, size: 152 2024.10.22 02:21:50.746886 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 034cf89423e14738aae2240c893c1cdf for removal 2024.10.22 02:21:50.775916 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 98187 2024.10.22 02:21:50.842668 [ 1476 ] {} CacheMetadata: Key a769cf7817f317dc1bf8b8526db97f38 is removed from metadata 2024.10.22 02:21:50.859650 [ 1476 ] {} CacheMetadata: Key 034cf89423e14738aae2240c893c1cdf is removed from metadata 2024.10.22 02:21:51.353310 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:21:51.356286 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:21:51.557272 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(842d28fc72335de1b3fa7e036cef16d9) : [0, 219680]: Setting download as failed: File segment: [0, 219680], key: 842d28fc72335de1b3fa7e036cef16d9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:21:51.558467 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(842d28fc72335de1b3fa7e036cef16d9): No space left in cache to reserve 219681 bytes, will continue without cache download 2024.10.22 02:21:51.572369 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 842d28fc72335de1b3fa7e036cef16d9, offset: 0, size: 0 2024.10.22 02:21:51.574291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 842d28fc72335de1b3fa7e036cef16d9 for removal 2024.10.22 02:21:51.579793 [ 1476 ] {} CacheMetadata: Key 842d28fc72335de1b3fa7e036cef16d9 is removed from metadata 2024.10.22 02:21:52.151135 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_181_181_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:21:52.460620 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_688_140, 202410_689_689_0, 202410_690_690_0, 202410_691_691_0, 202410_692_692_0, 202410_693_693_0] 2024.10.22 02:21:52.463091 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_688_140, 202410_689_689_0, 202410_690_690_0, 202410_691_691_0, 202410_692_692_0, 202410_693_693_0] 2024.10.22 02:21:52.490840 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_688_140, 202410_689_689_0, 202410_690_690_0, 202410_691_691_0, 202410_692_692_0, 202410_693_693_0] 2024.10.22 02:21:52.685304 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 7.08 MiB, will set to 1.56 GiB (RSS), difference: 6.69 MiB 2024.10.22 02:21:52.689606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 119876 bytes for 0dfadc33639be8cd6ca85f8d666ebcac:0) 2024.10.22 02:21:52.692398 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 00e797d98589cafac6b1fb991b0472c7, offset: 0, size: 127996 2024.10.22 02:21:52.798900 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 00e797d98589cafac6b1fb991b0472c7 for removal 2024.10.22 02:21:52.807245 [ 1476 ] {} CacheMetadata: Key 00e797d98589cafac6b1fb991b0472c7 is removed from metadata 2024.10.22 02:21:52.808926 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c5f0ef562bd6f7b463cfb314ee3a3ecd, offset: 0, size: 163 2024.10.22 02:21:52.812700 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1310263 2024.10.22 02:21:52.814316 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c5f0ef562bd6f7b463cfb314ee3a3ecd for removal 2024.10.22 02:21:52.822122 [ 1476 ] {} CacheMetadata: Key c5f0ef562bd6f7b463cfb314ee3a3ecd is removed from metadata 2024.10.22 02:21:52.994757 [ 1708 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zxb/drxjkllojcfsmzelgzividleasphy, size 1890901 2024.10.22 02:21:53.225597 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column CounterClass 2024.10.22 02:21:53.251147 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column CounterClass 2024.10.22 02:21:53.253195 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:21:53.267088 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column CounterClass 2024.10.22 02:21:53.315746 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column CounterClass 2024.10.22 02:21:53.458602 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 750 2024.10.22 02:21:53.516071 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column CounterClass 2024.10.22 02:21:53.555997 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column CounterClass 2024.10.22 02:21:53.560414 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(6193e42bf6e80576cd807afe397d4123) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6193e42bf6e80576cd807afe397d4123, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:21:53.563829 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 6193e42bf6e80576cd807afe397d4123, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:21:53.566019 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:21:53.565783 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column CounterClass 2024.10.22 02:21:53.592000 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column CounterClass 2024.10.22 02:21:53.629403 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column CounterClass 2024.10.22 02:21:53.685638 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:53.691865 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:53.726808 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bdr/ypnnfdszutkzrktiqmnaxrwqvjyvu (LogSeriesLimiter: on interval from 2024-10-22 02:21:43 to 2024-10-22 02:21:53 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:21:53.909448 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 101851 2024.10.22 02:21:53.950852 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_865_865_0 (state Temporary) 2024.10.22 02:21:53.996672 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_865_865_0 to 202410_872_872_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:54.038535 [ 1220 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_862_161 to 202410_863_868_1 2024.10.22 02:21:54.044473 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1310263 2024.10.22 02:21:54.047002 [ 1220 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.02 MiB using storage policy from min volume index 0 2024.10.22 02:21:54.047682 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:54.049142 [ 1220 ] {} DiskLocal: Reserved 1.02 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:21:54.050698 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:21:54.064941 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_868_162} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_862_161 to 202410_863_868_1 into Wide with storage Full 2024.10.22 02:21:54.079693 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_868_162} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:54.088472 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_868_162} MergeTreeSequentialSource: Reading 161 marks from part 202410_1_862_161, total 1307667 rows starting from the beginning of the part 2024.10.22 02:21:54.121594 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_868_162} MergeTreeSequentialSource: Reading 3 marks from part 202410_863_868_1, total 9641 rows starting from the beginning of the part 2024.10.22 02:21:54.233314 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_181_181_0 to 202410_184_184_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:54.252031 [ 1235 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:21:54.256384 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 11 file segments (while reserving 80365 bytes for 30a2062adacac03cdafde66c81b1ef2a:0) 2024.10.22 02:21:54.258774 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0dfadc33639be8cd6ca85f8d666ebcac, offset: 0, size: 119876 2024.10.22 02:21:54.263707 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0dfadc33639be8cd6ca85f8d666ebcac for removal 2024.10.22 02:21:54.265354 [ 1476 ] {} CacheMetadata: Key 0dfadc33639be8cd6ca85f8d666ebcac is removed from metadata 2024.10.22 02:21:54.266133 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8bb3b16ce9283e1a34a4eaf070dc0332, offset: 0, size: 151 2024.10.22 02:21:54.268959 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8bb3b16ce9283e1a34a4eaf070dc0332 for removal 2024.10.22 02:21:54.270391 [ 1476 ] {} CacheMetadata: Key 8bb3b16ce9283e1a34a4eaf070dc0332 is removed from metadata 2024.10.22 02:21:54.271483 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b8b39d594fbdfd46acd75f20b52d9c5d, offset: 0, size: 28773 2024.10.22 02:21:54.274502 [ 1185 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 7 parts from 202410_1_174_28 to 202410_184_184_0 2024.10.22 02:21:54.275406 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b8b39d594fbdfd46acd75f20b52d9c5d for removal 2024.10.22 02:21:54.279559 [ 1235 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:21:54.287425 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4035702fcfa9ec7780ac4d371cd0478e, offset: 0, size: 152 2024.10.22 02:21:54.277015 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:54.296706 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4035702fcfa9ec7780ac4d371cd0478e for removal 2024.10.22 02:21:54.301056 [ 1185 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 3.41 MiB using storage policy from min volume index 0 2024.10.22 02:21:54.302601 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a70e0bdd1a6a51524fff5f39beeed3ce, offset: 0, size: 31755 2024.10.22 02:21:54.309677 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a70e0bdd1a6a51524fff5f39beeed3ce for removal 2024.10.22 02:21:54.311921 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_732_732_0 (state Temporary) 2024.10.22 02:21:54.312830 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7c0a39901d5f90e0642292a9bb99599e, offset: 0, size: 153 2024.10.22 02:21:54.381546 [ 1476 ] {} CacheMetadata: Key a70e0bdd1a6a51524fff5f39beeed3ce is removed from metadata 2024.10.22 02:21:54.293491 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:21:54.398155 [ 1185 ] {} DiskLocal: Reserved 3.41 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:21:54.408946 [ 1476 ] {} CacheMetadata: Key 4035702fcfa9ec7780ac4d371cd0478e is removed from metadata 2024.10.22 02:21:54.416663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7c0a39901d5f90e0642292a9bb99599e for removal 2024.10.22 02:21:54.419159 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_732_732_0 to 202410_742_742_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:54.425341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ba82692db0ba17498b247ba7a7bb8c56, offset: 0, size: 152 2024.10.22 02:21:54.427585 [ 1476 ] {} CacheMetadata: Key 7c0a39901d5f90e0642292a9bb99599e is removed from metadata 2024.10.22 02:21:54.433145 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ba82692db0ba17498b247ba7a7bb8c56 for removal 2024.10.22 02:21:54.440647 [ 1476 ] {} CacheMetadata: Key ba82692db0ba17498b247ba7a7bb8c56 is removed from metadata 2024.10.22 02:21:54.438446 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c3097cd1286fc52facc43eebf4f75d04, offset: 0, size: 2840 2024.10.22 02:21:54.454335 [ 1476 ] {} CacheMetadata: Key b8b39d594fbdfd46acd75f20b52d9c5d is removed from metadata 2024.10.22 02:21:54.462958 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c3097cd1286fc52facc43eebf4f75d04 for removal 2024.10.22 02:21:54.468938 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9aaee899145afa21ed6d21bf75923019, offset: 0, size: 152 2024.10.22 02:21:54.476695 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9aaee899145afa21ed6d21bf75923019 for removal 2024.10.22 02:21:54.491922 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4c4b503e27d595c3d957b64c074a40b4, offset: 0, size: 3830 2024.10.22 02:21:54.494406 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1523_1523_0 (state Temporary) 2024.10.22 02:21:54.496757 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4c4b503e27d595c3d957b64c074a40b4 for removal 2024.10.22 02:21:54.500528 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e1a33b205476654184c87a8175c2a94f, offset: 0, size: 152 2024.10.22 02:21:54.527430 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1207_1431_45, 202410_1432_1432_0, 202410_1433_1433_0, 202410_1434_1434_0] 2024.10.22 02:21:54.533072 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 750 2024.10.22 02:21:54.536052 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1207_1431_45, 202410_1432_1432_0, 202410_1433_1433_0, 202410_1434_1434_0] 2024.10.22 02:21:54.561834 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1207_1431_45, 202410_1432_1432_0, 202410_1433_1433_0, 202410_1434_1434_0] 2024.10.22 02:21:54.563558 [ 1476 ] {} CacheMetadata: Key 4c4b503e27d595c3d957b64c074a40b4 is removed from metadata 2024.10.22 02:21:54.570906 [ 1476 ] {} CacheMetadata: Key 9aaee899145afa21ed6d21bf75923019 is removed from metadata 2024.10.22 02:21:54.584368 [ 1476 ] {} CacheMetadata: Key c3097cd1286fc52facc43eebf4f75d04 is removed from metadata 2024.10.22 02:21:54.605614 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e1a33b205476654184c87a8175c2a94f for removal 2024.10.22 02:21:54.611609 [ 1476 ] {} CacheMetadata: Key e1a33b205476654184c87a8175c2a94f is removed from metadata 2024.10.22 02:21:54.642277 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1523_1523_0 to 202410_1536_1536_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:54.646453 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} MergeTask::PrepareStage: Merging 7 parts: from 202410_1_174_28 to 202410_184_184_0 into Compact with storage Full 2024.10.22 02:21:54.843509 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101851 2024.10.22 02:21:54.846436 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 4.55 MiB, will set to 1.56 GiB (RSS), difference: -14.17 MiB 2024.10.22 02:21:55.858453 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 13818 2024.10.22 02:21:55.910483 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 155398 bytes for 7df66dd4d924ab427d57f533bed13ed1:0) 2024.10.22 02:21:55.915771 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 774dd7948546bb1b5fa50b2ab35e10ca, offset: 0, size: 81188 2024.10.22 02:21:55.923182 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 774dd7948546bb1b5fa50b2ab35e10ca for removal 2024.10.22 02:21:55.928256 [ 1476 ] {} CacheMetadata: Key 774dd7948546bb1b5fa50b2ab35e10ca is removed from metadata 2024.10.22 02:21:55.927948 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a92aeb592fb2e637c4acb61d19263f0e, offset: 0, size: 153 2024.10.22 02:21:55.932403 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a92aeb592fb2e637c4acb61d19263f0e for removal 2024.10.22 02:21:55.939946 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 30a2062adacac03cdafde66c81b1ef2a, offset: 0, size: 80365 2024.10.22 02:21:55.943883 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 30a2062adacac03cdafde66c81b1ef2a for removal 2024.10.22 02:21:55.947106 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7da4db147e2cba4102a101d5dc282ea3, offset: 0, size: 163 2024.10.22 02:21:55.951285 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7da4db147e2cba4102a101d5dc282ea3 for removal 2024.10.22 02:21:55.988095 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:55.994291 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:21:56.039522 [ 1476 ] {} CacheMetadata: Key 7da4db147e2cba4102a101d5dc282ea3 is removed from metadata 2024.10.22 02:21:56.046928 [ 1476 ] {} CacheMetadata: Key 30a2062adacac03cdafde66c81b1ef2a is removed from metadata 2024.10.22 02:21:56.046977 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_904_904_0 (state Temporary) 2024.10.22 02:21:56.052965 [ 1476 ] {} CacheMetadata: Key a92aeb592fb2e637c4acb61d19263f0e is removed from metadata 2024.10.22 02:21:56.081679 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_904_904_0 to 202410_917_917_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:56.102650 [ 1227 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_912_277 to 202410_913_913_0 2024.10.22 02:21:56.111575 [ 1227 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:56.113827 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:21:56.140565 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_913_278} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_912_277 to 202410_913_913_0 into Compact with storage Full 2024.10.22 02:21:56.147665 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13818 2024.10.22 02:21:56.197967 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_913_278} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:21:56.211095 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_913_278} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_912_277, total 13836 rows starting from the beginning of the part 2024.10.22 02:21:56.252847 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_913_278} MergeTreeSequentialSource: Reading 2 marks from part 202410_913_913_0, total 26 rows starting from the beginning of the part 2024.10.22 02:21:56.459622 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 7.69 MiB, will set to 1.56 GiB (RSS), difference: 1.53 MiB 2024.10.22 02:21:56.679600 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 173552 bytes for 92f987d9327e0138f9dc5b3065a07d2f:0) 2024.10.22 02:21:56.682772 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7df66dd4d924ab427d57f533bed13ed1, offset: 0, size: 155398 2024.10.22 02:21:56.687583 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7df66dd4d924ab427d57f533bed13ed1 for removal 2024.10.22 02:21:56.691764 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fb6832b34a81689455db6479f33b09fd, offset: 0, size: 164 2024.10.22 02:21:56.691756 [ 1476 ] {} CacheMetadata: Key 7df66dd4d924ab427d57f533bed13ed1 is removed from metadata 2024.10.22 02:21:56.694526 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fb6832b34a81689455db6479f33b09fd for removal 2024.10.22 02:21:56.827447 [ 1476 ] {} CacheMetadata: Key fb6832b34a81689455db6479f33b09fd is removed from metadata 2024.10.22 02:21:57.201255 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_913_278} MergeTask::MergeProjectionsStage: Merge sorted 13862 rows, containing 12 columns (12 merged, 0 gathered) in 1.078126637 sec., 12857.487723865597 rows/sec., 2.01 MiB/sec. 2024.10.22 02:21:57.293405 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_913_278} MergedBlockOutputStream: filled checksums 202410_1_913_278 (state Temporary) 2024.10.22 02:21:57.551409 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7306 2024.10.22 02:21:57.851042 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 106800 bytes for 083416636e744c15d55fb5b27c8df477:0) 2024.10.22 02:21:57.853185 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 92f987d9327e0138f9dc5b3065a07d2f, offset: 0, size: 173552 2024.10.22 02:21:57.855733 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 92f987d9327e0138f9dc5b3065a07d2f for removal 2024.10.22 02:21:57.859179 [ 1476 ] {} CacheMetadata: Key 92f987d9327e0138f9dc5b3065a07d2f is removed from metadata 2024.10.22 02:21:57.859291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dbb5f44e131df022e7412376d500d1cd, offset: 0, size: 164 2024.10.22 02:21:57.862723 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dbb5f44e131df022e7412376d500d1cd for removal 2024.10.22 02:21:57.872794 [ 1476 ] {} CacheMetadata: Key dbb5f44e131df022e7412376d500d1cd is removed from metadata 2024.10.22 02:21:58.118571 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_913_278} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_913_278 to 202410_1_913_278 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:58.138733 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_913_278} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_912_277, 202410_913_913_0] -> 202410_1_913_278 2024.10.22 02:21:58.156738 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:21:58.286822 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 99 entries to flush up to offset 98286 2024.10.22 02:21:58.408851 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 10.38 MiB, will set to 1.56 GiB (RSS), difference: 5.08 MiB 2024.10.22 02:21:58.556363 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:58.559910 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:21:58.600052 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_889_889_0 (state Temporary) 2024.10.22 02:21:58.652511 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_889_889_0 to 202410_895_895_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:58.683258 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 98286 2024.10.22 02:21:58.726011 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(770c42746763ebc2b31e4187969e11e9) : [0, 1071126]: Setting download as failed: File segment: [0, 1071126], key: 770c42746763ebc2b31e4187969e11e9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:21:58.729361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(770c42746763ebc2b31e4187969e11e9): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:21:58.771371 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 770c42746763ebc2b31e4187969e11e9, offset: 0, size: 0 2024.10.22 02:21:58.773410 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 770c42746763ebc2b31e4187969e11e9 for removal 2024.10.22 02:21:58.777224 [ 1476 ] {} CacheMetadata: Key 770c42746763ebc2b31e4187969e11e9 is removed from metadata 2024.10.22 02:21:58.848037 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 125 entries to flush up to offset 101976 2024.10.22 02:21:59.209709 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:21:59.216677 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:21:59.308145 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1524_1524_0 (state Temporary) 2024.10.22 02:21:59.370758 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1524_1524_0 to 202410_1537_1537_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:21:59.424630 [ 1720 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/efj/vvnzcavtargbsquaakhysvleokncg, size 1958583 2024.10.22 02:21:59.445292 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 101976 2024.10.22 02:21:59.570188 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 02:21:59.633821 [ 1275 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1532_89 to 202410_1537_1537_0 2024.10.22 02:21:59.644474 [ 1275 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.04 MiB using storage policy from min volume index 0 2024.10.22 02:21:59.646899 [ 1275 ] {} DiskLocal: Reserved 1.04 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:21:59.649762 [ 1275 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 257 ms. 2024.10.22 02:21:59.734811 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 02:21:59.743667 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 02:21:59.753014 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 02:21:59.869050 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 02:21:59.900207 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 02:21:59.928553 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 49665 bytes for 8c6e4512f87cecb908924158b000f498:0) 2024.10.22 02:21:59.931302 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column LoadEventStartTiming 2024.10.22 02:21:59.934690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 083416636e744c15d55fb5b27c8df477, offset: 0, size: 106800 2024.10.22 02:21:59.944801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 083416636e744c15d55fb5b27c8df477 for removal 2024.10.22 02:21:59.949438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7fa827898f2ae73122d8cb8906d2737b, offset: 0, size: 163 2024.10.22 02:21:59.953264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7fa827898f2ae73122d8cb8906d2737b for removal 2024.10.22 02:21:59.955013 [ 1476 ] {} CacheMetadata: Key 083416636e744c15d55fb5b27c8df477 is removed from metadata 2024.10.22 02:21:59.962161 [ 1476 ] {} CacheMetadata: Key 7fa827898f2ae73122d8cb8906d2737b is removed from metadata 2024.10.22 02:22:00.048497 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ckv/ngccaalmapvqwgdwtdiinwyewpuxw (LogSeriesLimiter: on interval from 2024-10-22 02:21:53 to 2024-10-22 02:22:00 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:22:00.059874 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1532_89 to 202410_1537_1537_0 into Compact with storage Full 2024.10.22 02:22:00.194563 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:00.254941 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_868_162/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:22:00.313826 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1532_89, total 22392 rows starting from the beginning of the part 2024.10.22 02:22:00.411398 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_1533_1533_0, total 88 rows starting from the beginning of the part 2024.10.22 02:22:00.516114 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 9.98 MiB, will set to 1.56 GiB (RSS), difference: -1.90 MiB 2024.10.22 02:22:00.600248 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 95294 bytes for ba3a3eae7a33ba45932d920fff31c37b:0) 2024.10.22 02:22:00.603017 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b30ab3712c7097e75ca780a529610a29, offset: 0, size: 57106 2024.10.22 02:22:00.608368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b30ab3712c7097e75ca780a529610a29 for removal 2024.10.22 02:22:00.614589 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4dd265a54e3bc1450de177f5649d9521, offset: 0, size: 152 2024.10.22 02:22:00.619672 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4dd265a54e3bc1450de177f5649d9521 for removal 2024.10.22 02:22:00.624124 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d968dac6082e1da3b3e9cc252b46cfdc, offset: 0, size: 163 2024.10.22 02:22:00.630751 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d968dac6082e1da3b3e9cc252b46cfdc for removal 2024.10.22 02:22:00.695657 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_1534_1534_0, total 100 rows starting from the beginning of the part 2024.10.22 02:22:00.716233 [ 1476 ] {} CacheMetadata: Key b30ab3712c7097e75ca780a529610a29 is removed from metadata 2024.10.22 02:22:00.764794 [ 1476 ] {} CacheMetadata: Key d968dac6082e1da3b3e9cc252b46cfdc is removed from metadata 2024.10.22 02:22:00.775390 [ 1476 ] {} CacheMetadata: Key 4dd265a54e3bc1450de177f5649d9521 is removed from metadata 2024.10.22 02:22:01.050160 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1790 entries to flush up to offset 1312053 2024.10.22 02:22:01.116723 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_1535_1535_0, total 60 rows starting from the beginning of the part 2024.10.22 02:22:01.355820 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:22:01.357758 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:22:01.446806 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_1536_1536_0, total 70 rows starting from the beginning of the part 2024.10.22 02:22:01.613200 [ 1263 ] {} DatabaseCatalog: Execution took 241 ms. 2024.10.22 02:22:01.628890 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} MergeTreeSequentialSource: Reading 2 marks from part 202410_1537_1537_0, total 125 rows starting from the beginning of the part 2024.10.22 02:22:01.688932 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:01.802514 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 141325 bytes for 32668db776c657f36050e3c8c0e4ce4c:0) 2024.10.22 02:22:01.811929 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 31f455c0d3c6df27a8c3d829a09c718a, offset: 0, size: 148 2024.10.22 02:22:01.829106 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 31f455c0d3c6df27a8c3d829a09c718a for removal 2024.10.22 02:22:01.833337 [ 1476 ] {} CacheMetadata: Key 31f455c0d3c6df27a8c3d829a09c718a is removed from metadata 2024.10.22 02:22:01.834494 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ba3a3eae7a33ba45932d920fff31c37b, offset: 0, size: 95294 2024.10.22 02:22:01.845579 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ba3a3eae7a33ba45932d920fff31c37b for removal 2024.10.22 02:22:01.851583 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8c6e4512f87cecb908924158b000f498, offset: 0, size: 49665 2024.10.22 02:22:01.852585 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_832_17, 202410_833_833_0, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0, 202410_837_837_0] 2024.10.22 02:22:01.854622 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_832_17, 202410_833_833_0, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0, 202410_837_837_0] 2024.10.22 02:22:01.854380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8c6e4512f87cecb908924158b000f498 for removal 2024.10.22 02:22:01.857104 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1fc0dab28c0f475279f5dc7419780dd3, offset: 0, size: 152 2024.10.22 02:22:01.862968 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1fc0dab28c0f475279f5dc7419780dd3 for removal 2024.10.22 02:22:01.885827 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_832_17, 202410_833_833_0, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0, 202410_837_837_0] 2024.10.22 02:22:01.948134 [ 1476 ] {} CacheMetadata: Key 1fc0dab28c0f475279f5dc7419780dd3 is removed from metadata 2024.10.22 02:22:01.956978 [ 1476 ] {} CacheMetadata: Key 8c6e4512f87cecb908924158b000f498 is removed from metadata 2024.10.22 02:22:01.964786 [ 1476 ] {} CacheMetadata: Key ba3a3eae7a33ba45932d920fff31c37b is removed from metadata 2024.10.22 02:22:02.035375 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 751 2024.10.22 02:22:02.217757 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:02.327361 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:02.360067 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:02.363795 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:02.390343 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_733_733_0 (state Temporary) 2024.10.22 02:22:02.421555 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} MergeTreeSequentialSource: Reading 5 marks from part 202410_1_174_28, total 6951 rows starting from the beginning of the part 2024.10.22 02:22:02.546064 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 12.01 MiB, will set to 1.56 GiB (RSS), difference: -18.47 MiB 2024.10.22 02:22:02.731629 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_866_866_0 (state Temporary) 2024.10.22 02:22:02.807806 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_733_733_0 to 202410_743_743_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:02.860396 [ 1223 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_738_150 to 202410_743_743_0 2024.10.22 02:22:02.869295 [ 1223 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:02.878684 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:02.898589 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 751 2024.10.22 02:22:02.928469 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_866_866_0 to 202410_873_873_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:02.965959 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1312053 2024.10.22 02:22:03.027173 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_738_150 to 202410_743_743_0 into Compact with storage Full 2024.10.22 02:22:03.030068 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 79064 bytes for 9f40b4013496590c320f30084dd19e22:0) 2024.10.22 02:22:03.038873 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 32668db776c657f36050e3c8c0e4ce4c, offset: 0, size: 141325 2024.10.22 02:22:03.048316 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 32668db776c657f36050e3c8c0e4ce4c for removal 2024.10.22 02:22:03.053035 [ 1476 ] {} CacheMetadata: Key 32668db776c657f36050e3c8c0e4ce4c is removed from metadata 2024.10.22 02:22:03.053932 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d8b0cb87267fd46f3a34a5f22aab93b0, offset: 0, size: 150 2024.10.22 02:22:03.059283 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d8b0cb87267fd46f3a34a5f22aab93b0 for removal 2024.10.22 02:22:03.072232 [ 1476 ] {} CacheMetadata: Key d8b0cb87267fd46f3a34a5f22aab93b0 is removed from metadata 2024.10.22 02:22:03.266774 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:03.282906 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_738_150, total 782 rows starting from the beginning of the part 2024.10.22 02:22:03.554895 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 102069 2024.10.22 02:22:03.557164 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} MergeTreeSequentialSource: Reading 2 marks from part 202410_739_739_0, total 1 rows starting from the beginning of the part 2024.10.22 02:22:03.650449 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 13826 2024.10.22 02:22:03.678180 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1537_90/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:22:03.730647 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} MergeTreeSequentialSource: Reading 2 marks from part 202410_740_740_0, total 1 rows starting from the beginning of the part 2024.10.22 02:22:03.895974 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:03.901036 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:03.934117 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} MergeTreeSequentialSource: Reading 2 marks from part 202410_741_741_0, total 1 rows starting from the beginning of the part 2024.10.22 02:22:03.943351 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_905_905_0 (state Temporary) 2024.10.22 02:22:03.978939 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ca615b81b6e5061867fc4722a7bac4ca) : [0, 298823]: Setting download as failed: File segment: [0, 298823], key: ca615b81b6e5061867fc4722a7bac4ca, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:03.981053 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ca615b81b6e5061867fc4722a7bac4ca): No space left in cache to reserve 298824 bytes, will continue without cache download 2024.10.22 02:22:04.000626 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_905_905_0 to 202410_918_918_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:04.033257 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} MergeTreeSequentialSource: Reading 2 marks from part 202410_742_742_0, total 1 rows starting from the beginning of the part 2024.10.22 02:22:04.064772 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:04.064789 [ 1222 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_913_278 to 202410_914_914_0 2024.10.22 02:22:04.067043 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:04.129411 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} MergeTreeSequentialSource: Reading 2 marks from part 202410_743_743_0, total 1 rows starting from the beginning of the part 2024.10.22 02:22:04.139720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ca615b81b6e5061867fc4722a7bac4ca, offset: 0, size: 0 2024.10.22 02:22:04.141474 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ca615b81b6e5061867fc4722a7bac4ca for removal 2024.10.22 02:22:04.147178 [ 1476 ] {} CacheMetadata: Key ca615b81b6e5061867fc4722a7bac4ca is removed from metadata 2024.10.22 02:22:04.167576 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1525_1525_0 (state Temporary) 2024.10.22 02:22:04.208348 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_175_179_1, total 212 rows starting from the beginning of the part 2024.10.22 02:22:04.279996 [ 1222 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:04.295342 [ 1222 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:04.303502 [ 1222 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 282 ms. 2024.10.22 02:22:04.339024 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_914_279} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_913_278 to 202410_914_914_0 into Compact with storage Full 2024.10.22 02:22:04.352726 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13826 2024.10.22 02:22:04.377820 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_914_279} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:04.396210 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_914_279} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_913_278, total 13862 rows starting from the beginning of the part 2024.10.22 02:22:04.427791 [ 1167 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 216 ms. 2024.10.22 02:22:04.420710 [ 1177 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 02:22:04.455846 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_914_279} MergeTreeSequentialSource: Reading 2 marks from part 202410_914_914_0, total 15 rows starting from the beginning of the part 2024.10.22 02:22:04.461542 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1525_1525_0 to 202410_1538_1538_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:04.467849 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 11.76 MiB, will set to 1.57 GiB (RSS), difference: 6.41 MiB 2024.10.22 02:22:04.535772 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} MergeTask::MergeProjectionsStage: Merge sorted 22835 rows, containing 13 columns (13 merged, 0 gathered) in 4.779224162 sec., 4777.972161582824 rows/sec., 889.75 KiB/sec. 2024.10.22 02:22:04.554018 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102069 2024.10.22 02:22:04.639444 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} MergedBlockOutputStream: filled checksums 202410_1207_1537_90 (state Temporary) 2024.10.22 02:22:04.640596 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_802_151, 202410_803_808_1] 2024.10.22 02:22:04.643896 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_802_151, 202410_803_808_1] 2024.10.22 02:22:04.846200 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1537_90 to 202410_1207_1537_90 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:04.851941 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1537_90} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1532_89, 202410_1537_1537_0] -> 202410_1207_1537_90 2024.10.22 02:22:04.872391 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_802_151, 202410_803_808_1] 2024.10.22 02:22:04.898708 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.82 MiB. 2024.10.22 02:22:04.981833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3bbcdb9d25c4afa863b47e9005a61b22) : [0, 1032014]: Setting download as failed: File segment: [0, 1032014], key: 3bbcdb9d25c4afa863b47e9005a61b22, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:04.987718 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3bbcdb9d25c4afa863b47e9005a61b22): No space left in cache to reserve 1032015 bytes, will continue without cache download 2024.10.22 02:22:05.030425 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3bbcdb9d25c4afa863b47e9005a61b22, offset: 0, size: 0 2024.10.22 02:22:05.032074 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3bbcdb9d25c4afa863b47e9005a61b22 for removal 2024.10.22 02:22:05.041205 [ 1476 ] {} CacheMetadata: Key 3bbcdb9d25c4afa863b47e9005a61b22 is removed from metadata 2024.10.22 02:22:05.127637 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 7344 2024.10.22 02:22:05.216884 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_914_279} MergeTask::MergeProjectionsStage: Merge sorted 13877 rows, containing 12 columns (12 merged, 0 gathered) in 0.891371492 sec., 15568.144286131152 rows/sec., 2.43 MiB/sec. 2024.10.22 02:22:05.365459 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_914_279} MergedBlockOutputStream: filled checksums 202410_1_914_279 (state Temporary) 2024.10.22 02:22:05.649475 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_914_279} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_914_279 to 202410_1_914_279 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:05.651828 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_914_279} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_913_278, 202410_914_914_0] -> 202410_1_914_279 2024.10.22 02:22:05.663715 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:22:05.832597 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_180_180_0, total 40 rows starting from the beginning of the part 2024.10.22 02:22:06.253384 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0531ab5b523d5ab89482a1cb58e180eb) : [0, 1070928]: Setting download as failed: File segment: [0, 1070928], key: 0531ab5b523d5ab89482a1cb58e180eb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:06.255598 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0531ab5b523d5ab89482a1cb58e180eb): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:22:06.260050 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 120 entries to flush up to offset 98406 2024.10.22 02:22:06.362879 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:06.364800 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:06.375938 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0531ab5b523d5ab89482a1cb58e180eb, offset: 0, size: 0 2024.10.22 02:22:06.377676 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0531ab5b523d5ab89482a1cb58e180eb for removal 2024.10.22 02:22:06.381499 [ 1476 ] {} CacheMetadata: Key 0531ab5b523d5ab89482a1cb58e180eb is removed from metadata 2024.10.22 02:22:06.396218 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_890_890_0 (state Temporary) 2024.10.22 02:22:06.552633 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} MergeTask::MergeProjectionsStage: Merge sorted 787 rows, containing 32 columns (32 merged, 0 gathered) in 3.53806413 sec., 222.4380257347116 rows/sec., 16.82 KiB/sec. 2024.10.22 02:22:06.570330 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} MergedBlockOutputStream: filled checksums 202410_1_743_151 (state Temporary) 2024.10.22 02:22:06.657518 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 8.15 MiB, will set to 1.56 GiB (RSS), difference: 8.42 MiB 2024.10.22 02:22:06.675826 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_890_890_0 to 202410_896_896_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:06.727496 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 98406 2024.10.22 02:22:06.959110 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 81730 bytes for 03352ff7e20d87e13d24620231b47ac6:0) 2024.10.22 02:22:06.963871 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9f40b4013496590c320f30084dd19e22, offset: 0, size: 79064 2024.10.22 02:22:06.968821 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9f40b4013496590c320f30084dd19e22 for removal 2024.10.22 02:22:06.975204 [ 1476 ] {} CacheMetadata: Key 9f40b4013496590c320f30084dd19e22 is removed from metadata 2024.10.22 02:22:06.978653 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 05220cd7e5318c36f3c44de6122a9bfc, offset: 0, size: 153 2024.10.22 02:22:06.992573 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 05220cd7e5318c36f3c44de6122a9bfc for removal 2024.10.22 02:22:07.016076 [ 1476 ] {} CacheMetadata: Key 05220cd7e5318c36f3c44de6122a9bfc is removed from metadata 2024.10.22 02:22:07.803724 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_181_181_0, total 42 rows starting from the beginning of the part 2024.10.22 02:22:07.870849 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_743_151 to 202410_1_743_151 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:07.875380 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_743_151} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_738_150, 202410_743_743_0] -> 202410_1_743_151 2024.10.22 02:22:08.037839 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:22:08.236380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 71809 bytes for 30ab1d2600b00176f852d81ce4f84443:0) 2024.10.22 02:22:08.240166 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 197cf0ef505376dd3e19808fa33bc5a2, offset: 0, size: 86174 2024.10.22 02:22:08.247080 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 197cf0ef505376dd3e19808fa33bc5a2 for removal 2024.10.22 02:22:08.248972 [ 1476 ] {} CacheMetadata: Key 197cf0ef505376dd3e19808fa33bc5a2 is removed from metadata 2024.10.22 02:22:08.250630 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a072025e79ac0b71d3f9c790819bc18f, offset: 0, size: 153 2024.10.22 02:22:08.354507 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a072025e79ac0b71d3f9c790819bc18f for removal 2024.10.22 02:22:08.357109 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a2f272d734576a18f27106ff7f4e716f, offset: 0, size: 163 2024.10.22 02:22:08.357348 [ 1476 ] {} CacheMetadata: Key a072025e79ac0b71d3f9c790819bc18f is removed from metadata 2024.10.22 02:22:08.360394 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a2f272d734576a18f27106ff7f4e716f for removal 2024.10.22 02:22:08.365053 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 98cd521817cb3e20899ea62fab2ca9e1, offset: 0, size: 175 2024.10.22 02:22:08.370610 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 98cd521817cb3e20899ea62fab2ca9e1 for removal 2024.10.22 02:22:08.393803 [ 1476 ] {} CacheMetadata: Key 98cd521817cb3e20899ea62fab2ca9e1 is removed from metadata 2024.10.22 02:22:08.410906 [ 1476 ] {} CacheMetadata: Key a2f272d734576a18f27106ff7f4e716f is removed from metadata 2024.10.22 02:22:08.575703 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 102162 2024.10.22 02:22:08.597663 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 10.04 MiB, will set to 1.56 GiB (RSS), difference: 6.55 MiB 2024.10.22 02:22:08.961799 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1434_46, 202410_1435_1435_0] 2024.10.22 02:22:08.965047 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1434_46, 202410_1435_1435_0] 2024.10.22 02:22:08.979858 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1434_46, 202410_1435_1435_0] 2024.10.22 02:22:09.147823 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:09.152799 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:09.247775 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_182_182_0, total 42 rows starting from the beginning of the part 2024.10.22 02:22:09.297365 [ 1158 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:22:09.312613 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(29386d99c5a1c0ed2904f09d61043d4b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 29386d99c5a1c0ed2904f09d61043d4b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:22:09.316386 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 29386d99c5a1c0ed2904f09d61043d4b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1386, kind: Regular, unbound: 0 2024.10.22 02:22:09.344467 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:22:09.431576 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1526_1526_0 (state Temporary) 2024.10.22 02:22:09.451139 [ 1158 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:22:09.814177 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 69791 bytes for 62d844166e92a482270fe12b8ca31cee:0) 2024.10.22 02:22:09.818787 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 03352ff7e20d87e13d24620231b47ac6, offset: 0, size: 81730 2024.10.22 02:22:09.826240 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 03352ff7e20d87e13d24620231b47ac6 for removal 2024.10.22 02:22:09.829266 [ 1476 ] {} CacheMetadata: Key 03352ff7e20d87e13d24620231b47ac6 is removed from metadata 2024.10.22 02:22:09.831229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7309d4e37a3ae82eb047ec7ef102927d, offset: 0, size: 153 2024.10.22 02:22:09.837656 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7309d4e37a3ae82eb047ec7ef102927d for removal 2024.10.22 02:22:09.883948 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1526_1526_0 to 202410_1539_1539_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:09.952413 [ 1476 ] {} CacheMetadata: Key 7309d4e37a3ae82eb047ec7ef102927d is removed from metadata 2024.10.22 02:22:09.952575 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dbf79f36e8c1ce21f4ed3fd921cc5ea2, offset: 0, size: 163 2024.10.22 02:22:09.969128 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1290 entries to flush up to offset 1313343 2024.10.22 02:22:10.011026 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102162 2024.10.22 02:22:10.057317 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dbf79f36e8c1ce21f4ed3fd921cc5ea2 for removal 2024.10.22 02:22:10.069599 [ 1476 ] {} CacheMetadata: Key dbf79f36e8c1ce21f4ed3fd921cc5ea2 is removed from metadata 2024.10.22 02:22:10.646506 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 562 bytes for 2ec9d677134c8fc7b8b483a0f2fa770f:0) 2024.10.22 02:22:10.649373 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 656181e0ade6317287ba79b2c8c9185c, offset: 0, size: 153 2024.10.22 02:22:10.654573 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 656181e0ade6317287ba79b2c8c9185c for removal 2024.10.22 02:22:10.658269 [ 1476 ] {} CacheMetadata: Key 656181e0ade6317287ba79b2c8c9185c is removed from metadata 2024.10.22 02:22:10.801467 [ 1712 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bdr/ypnnfdszutkzrktiqmnaxrwqvjyvu, size 56295 2024.10.22 02:22:10.857752 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_183_183_0, total 39 rows starting from the beginning of the part 2024.10.22 02:22:10.945472 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:10.950260 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:11.039278 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column OS 2024.10.22 02:22:11.150526 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column OS 2024.10.22 02:22:11.173960 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column OS 2024.10.22 02:22:11.187763 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column OS 2024.10.22 02:22:11.228044 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column OS 2024.10.22 02:22:11.251472 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column OS 2024.10.22 02:22:11.352609 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_867_867_0 (state Temporary) 2024.10.22 02:22:11.354109 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:22:11.357269 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:22:11.370670 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column OS 2024.10.22 02:22:11.402464 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column OS 2024.10.22 02:22:11.439546 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column OS 2024.10.22 02:22:11.479567 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 163 bytes for 026dcfb2fcc1b63aebe76e83a53f4339:0) 2024.10.22 02:22:11.487592 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 30ab1d2600b00176f852d81ce4f84443, offset: 0, size: 71809 2024.10.22 02:22:11.495720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 30ab1d2600b00176f852d81ce4f84443 for removal 2024.10.22 02:22:11.504656 [ 1476 ] {} CacheMetadata: Key 30ab1d2600b00176f852d81ce4f84443 is removed from metadata 2024.10.22 02:22:11.517185 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jku/bojjskujdydfdifmlpifqkzsdcyiw (LogSeriesLimiter: on interval from 2024-10-22 02:22:00 to 2024-10-22 02:22:11 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:22:11.558313 [ 1196 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 213 ms. 2024.10.22 02:22:11.571642 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_868_162/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:22:11.606393 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_867_867_0 to 202410_874_874_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:11.664267 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1313343 2024.10.22 02:22:11.748743 [ 1187 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_869_869_0 to 202410_874_874_0 2024.10.22 02:22:11.768363 [ 1187 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:11.770226 [ 1187 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:11.855654 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 13848 2024.10.22 02:22:11.898262 [ 1187 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 270 ms. 2024.10.22 02:22:12.023985 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_869_869_0 to 202410_874_874_0 into Compact with storage Full 2024.10.22 02:22:12.157412 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:12.162547 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_869_869_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:22:12.173022 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_870_870_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:22:12.204285 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_871_871_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:22:12.224101 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_872_872_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:22:12.347747 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_873_873_0, total 1790 rows starting from the beginning of the part 2024.10.22 02:22:12.364723 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:12.366738 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:12.456063 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_874_874_0, total 1290 rows starting from the beginning of the part 2024.10.22 02:22:12.499011 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_906_906_0 (state Temporary) 2024.10.22 02:22:12.563682 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 96332 bytes for 966d70826132e4a5b7b2d1e490fd80b8:0) 2024.10.22 02:22:12.565090 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 62d844166e92a482270fe12b8ca31cee, offset: 0, size: 69791 2024.10.22 02:22:12.567263 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 62d844166e92a482270fe12b8ca31cee for removal 2024.10.22 02:22:12.568682 [ 1476 ] {} CacheMetadata: Key 62d844166e92a482270fe12b8ca31cee is removed from metadata 2024.10.22 02:22:12.568917 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 561fb4f8ca05ad11d926a3dbf388c094, offset: 0, size: 153 2024.10.22 02:22:12.572232 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_906_906_0 to 202410_919_919_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:12.572287 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 561fb4f8ca05ad11d926a3dbf388c094 for removal 2024.10.22 02:22:12.590624 [ 1476 ] {} CacheMetadata: Key 561fb4f8ca05ad11d926a3dbf388c094 is removed from metadata 2024.10.22 02:22:12.611485 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 10.15 MiB, will set to 1.57 GiB (RSS), difference: -11.43 MiB 2024.10.22 02:22:12.619412 [ 1199 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_914_279 to 202410_915_915_0 2024.10.22 02:22:12.639245 [ 1199 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:12.647160 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13848 2024.10.22 02:22:12.653928 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} MergeTreeSequentialSource: Reading 2 marks from part 202410_184_184_0, total 42 rows starting from the beginning of the part 2024.10.22 02:22:12.663607 [ 1199 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:12.686856 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_915_280} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_914_279 to 202410_915_915_0 into Compact with storage Full 2024.10.22 02:22:12.841189 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_915_280} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:12.849207 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_915_280} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_914_279, total 13877 rows starting from the beginning of the part 2024.10.22 02:22:12.969396 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_915_280} MergeTreeSequentialSource: Reading 2 marks from part 202410_915_915_0, total 25 rows starting from the beginning of the part 2024.10.22 02:22:13.521921 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1435_47, 202410_1436_1436_0, 202410_1437_1437_0, 202410_1438_1438_0, 202410_1439_1439_0, 202410_1440_1440_0] 2024.10.22 02:22:13.550089 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1435_47, 202410_1436_1436_0, 202410_1437_1437_0, 202410_1438_1438_0, 202410_1439_1439_0, 202410_1440_1440_0] 2024.10.22 02:22:13.586476 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} MergeTask::MergeProjectionsStage: Merge sorted 9625 rows, containing 4 columns (4 merged, 0 gathered) in 1.671552211 sec., 5758.121066551596 rows/sec., 89.97 KiB/sec. 2024.10.22 02:22:13.683107 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1435_47, 202410_1436_1436_0, 202410_1437_1437_0, 202410_1438_1438_0, 202410_1439_1439_0, 202410_1440_1440_0] 2024.10.22 02:22:13.742161 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} MergedBlockOutputStream: filled checksums 202410_869_874_1 (state Temporary) 2024.10.22 02:22:13.804762 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_869_874_1 to 202410_869_874_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:13.821689 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_869_874_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_869_869_0, 202410_874_874_0] -> 202410_869_874_1 2024.10.22 02:22:13.826877 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_915_280} MergeTask::MergeProjectionsStage: Merge sorted 13902 rows, containing 12 columns (12 merged, 0 gathered) in 1.147082747 sec., 12119.439540310686 rows/sec., 1.89 MiB/sec. 2024.10.22 02:22:13.869010 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_915_280} MergedBlockOutputStream: filled checksums 202410_1_915_280 (state Temporary) 2024.10.22 02:22:13.874916 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:22:14.041805 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 100 entries to flush up to offset 102262 2024.10.22 02:22:14.064220 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6576f4f3d675981e96cb42e843f2c033) : [0, 302289]: Setting download as failed: File segment: [0, 302289], key: 6576f4f3d675981e96cb42e843f2c033, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:14.065200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6576f4f3d675981e96cb42e843f2c033): No space left in cache to reserve 302290 bytes, will continue without cache download 2024.10.22 02:22:14.114025 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6576f4f3d675981e96cb42e843f2c033, offset: 0, size: 0 2024.10.22 02:22:14.116612 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6576f4f3d675981e96cb42e843f2c033 for removal 2024.10.22 02:22:14.121130 [ 1476 ] {} CacheMetadata: Key 6576f4f3d675981e96cb42e843f2c033 is removed from metadata 2024.10.22 02:22:14.256307 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 96 entries to flush up to offset 98502 2024.10.22 02:22:14.391302 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_915_280} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_915_280 to 202410_1_915_280 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:14.400169 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_915_280} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_914_279, 202410_915_915_0] -> 202410_1_915_280 2024.10.22 02:22:14.449389 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:22:14.603408 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:14.694191 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 10.31 MiB, will set to 1.57 GiB (RSS), difference: 6.34 MiB 2024.10.22 02:22:14.708648 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:14.760766 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_891_891_0 (state Temporary) 2024.10.22 02:22:14.766370 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:14.768435 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:14.822212 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_891_891_0 to 202410_897_897_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:14.863787 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_892_29 to 202410_897_897_0 2024.10.22 02:22:14.870864 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 98502 2024.10.22 02:22:14.872010 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:14.874601 [ 1221 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:14.887818 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_892_29 to 202410_897_897_0 into Compact with storage Full 2024.10.22 02:22:14.951252 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:14.956736 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_892_29, total 14653 rows starting from the beginning of the part 2024.10.22 02:22:14.960036 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c5dc3bfd99551e6babcafe4817fdf2ba) : [0, 168236]: Setting download as failed: File segment: [0, 168236], key: c5dc3bfd99551e6babcafe4817fdf2ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:14.962234 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c5dc3bfd99551e6babcafe4817fdf2ba): No space left in cache to reserve 168237 bytes, will continue without cache download 2024.10.22 02:22:14.986261 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c5dc3bfd99551e6babcafe4817fdf2ba, offset: 0, size: 0 2024.10.22 02:22:14.988670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c5dc3bfd99551e6babcafe4817fdf2ba for removal 2024.10.22 02:22:14.992202 [ 1476 ] {} CacheMetadata: Key c5dc3bfd99551e6babcafe4817fdf2ba is removed from metadata 2024.10.22 02:22:15.024580 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_893_893_0, total 119 rows starting from the beginning of the part 2024.10.22 02:22:15.059920 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1527_1527_0 (state Temporary) 2024.10.22 02:22:15.162055 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1527_1527_0 to 202410_1540_1540_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:15.171781 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_894_894_0, total 60 rows starting from the beginning of the part 2024.10.22 02:22:15.254476 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_895_895_0, total 99 rows starting from the beginning of the part 2024.10.22 02:22:15.345645 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102262 2024.10.22 02:22:15.455420 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_896_896_0, total 120 rows starting from the beginning of the part 2024.10.22 02:22:15.630595 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} MergeTreeSequentialSource: Reading 2 marks from part 202410_897_897_0, total 96 rows starting from the beginning of the part 2024.10.22 02:22:15.707704 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 4836 bytes for 5e0a30193274bd02fd7ff8913327fd09:0) 2024.10.22 02:22:15.713053 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 966d70826132e4a5b7b2d1e490fd80b8, offset: 0, size: 96332 2024.10.22 02:22:15.722777 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 966d70826132e4a5b7b2d1e490fd80b8 for removal 2024.10.22 02:22:15.730459 [ 1476 ] {} CacheMetadata: Key 966d70826132e4a5b7b2d1e490fd80b8 is removed from metadata 2024.10.22 02:22:15.731874 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 026dcfb2fcc1b63aebe76e83a53f4339, offset: 0, size: 163 2024.10.22 02:22:15.739310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 026dcfb2fcc1b63aebe76e83a53f4339 for removal 2024.10.22 02:22:15.750064 [ 1476 ] {} CacheMetadata: Key 026dcfb2fcc1b63aebe76e83a53f4339 is removed from metadata 2024.10.22 02:22:15.824248 [ 1731 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ckv/ngccaalmapvqwgdwtdiinwyewpuxw, size 1956807 2024.10.22 02:22:16.065342 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 02:22:16.073606 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 02:22:16.095269 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 02:22:16.136182 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 02:22:16.157762 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 02:22:16.176758 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 02:22:16.193293 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column LoadEventEndTiming 2024.10.22 02:22:16.243041 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yds/pkngiwffskgzsydogewhjsttxurav (LogSeriesLimiter: on interval from 2024-10-22 02:22:11 to 2024-10-22 02:22:16 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:22:16.544599 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 10.15 MiB, will set to 1.57 GiB (RSS), difference: -12.56 MiB 2024.10.22 02:22:16.747191 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 95624 bytes for 1c30c58f7e471e966e9d1232815ede6b:0) 2024.10.22 02:22:16.749951 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2b92f63dee59803e0ac16dae9d57b477, offset: 0, size: 42922 2024.10.22 02:22:16.753137 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2b92f63dee59803e0ac16dae9d57b477 for removal 2024.10.22 02:22:16.758036 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6757d49f6665670c7b177f3b40a9b8a0, offset: 0, size: 152 2024.10.22 02:22:16.758509 [ 1476 ] {} CacheMetadata: Key 2b92f63dee59803e0ac16dae9d57b477 is removed from metadata 2024.10.22 02:22:16.761604 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6757d49f6665670c7b177f3b40a9b8a0 for removal 2024.10.22 02:22:16.766900 [ 1476 ] {} CacheMetadata: Key 6757d49f6665670c7b177f3b40a9b8a0 is removed from metadata 2024.10.22 02:22:17.347686 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} MergeTask::MergeProjectionsStage: Merge sorted 15147 rows, containing 13 columns (13 merged, 0 gathered) in 2.466531047 sec., 6141.013314396768 rows/sec., 2.66 MiB/sec. 2024.10.22 02:22:17.411566 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} MergedBlockOutputStream: filled checksums 202410_747_897_30 (state Temporary) 2024.10.22 02:22:17.907838 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 752 2024.10.22 02:22:17.962167 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_897_30 to 202410_747_897_30 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:17.964985 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_897_30} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_892_29, 202410_897_897_0] -> 202410_747_897_30 2024.10.22 02:22:18.002593 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.20 MiB. 2024.10.22 02:22:18.090689 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(78ce6d18d55f9975751633f0c0fd0c5d) : [0, 195643]: Setting download as failed: File segment: [0, 195643], key: 78ce6d18d55f9975751633f0c0fd0c5d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:18.094243 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(78ce6d18d55f9975751633f0c0fd0c5d): No space left in cache to reserve 195644 bytes, will continue without cache download 2024.10.22 02:22:18.110314 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 78ce6d18d55f9975751633f0c0fd0c5d, offset: 0, size: 0 2024.10.22 02:22:18.112333 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 78ce6d18d55f9975751633f0c0fd0c5d for removal 2024.10.22 02:22:18.117673 [ 1476 ] {} CacheMetadata: Key 78ce6d18d55f9975751633f0c0fd0c5d is removed from metadata 2024.10.22 02:22:18.321612 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:18.328898 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:18.496567 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_734_734_0 (state Temporary) 2024.10.22 02:22:18.524302 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.82 MiB, will set to 1.57 GiB (RSS), difference: 1.26 MiB 2024.10.22 02:22:18.667240 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1314883 2024.10.22 02:22:18.772624 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_734_734_0 to 202410_744_744_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:18.855739 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 752 2024.10.22 02:22:19.240614 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(aae7ad0985a06cc9408d5a4d3584ea31) : [0, 259251]: Setting download as failed: File segment: [0, 259251], key: aae7ad0985a06cc9408d5a4d3584ea31, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:19.244640 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(aae7ad0985a06cc9408d5a4d3584ea31): No space left in cache to reserve 259252 bytes, will continue without cache download 2024.10.22 02:22:19.468206 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: aae7ad0985a06cc9408d5a4d3584ea31, offset: 0, size: 0 2024.10.22 02:22:19.468685 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 102343 2024.10.22 02:22:19.470485 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key aae7ad0985a06cc9408d5a4d3584ea31 for removal 2024.10.22 02:22:19.474214 [ 1476 ] {} CacheMetadata: Key aae7ad0985a06cc9408d5a4d3584ea31 is removed from metadata 2024.10.22 02:22:19.706545 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:19.711653 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:19.859820 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:19.863595 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:19.945728 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_868_868_0 (state Temporary) 2024.10.22 02:22:20.041149 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1528_1528_0 (state Temporary) 2024.10.22 02:22:20.063696 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_868_868_0 to 202410_875_875_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:20.104203 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1314883 2024.10.22 02:22:20.150802 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 13870 2024.10.22 02:22:20.177432 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_184_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:22:20.195436 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1528_1528_0 to 202410_1541_1541_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:20.261331 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102343 2024.10.22 02:22:20.364150 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:20.366484 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:20.619538 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_907_907_0 (state Temporary) 2024.10.22 02:22:20.667936 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_907_907_0 to 202410_920_920_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:20.711514 [ 1267 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_915_280 to 202410_916_916_0 2024.10.22 02:22:20.749034 [ 1267 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:20.751538 [ 1267 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:20.835227 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.78 MiB, will set to 1.57 GiB (RSS), difference: 5.56 MiB 2024.10.22 02:22:20.875323 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_916_281} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_915_280 to 202410_916_916_0 into Compact with storage Full 2024.10.22 02:22:20.919829 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9a901d1089adf719a29d8aaba53b3ce7) : [0, 323536]: Setting download as failed: File segment: [0, 323536], key: 9a901d1089adf719a29d8aaba53b3ce7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:20.922967 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9a901d1089adf719a29d8aaba53b3ce7): No space left in cache to reserve 323537 bytes, will continue without cache download 2024.10.22 02:22:20.942782 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13870 2024.10.22 02:22:21.058392 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9a901d1089adf719a29d8aaba53b3ce7, offset: 0, size: 0 2024.10.22 02:22:21.060548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9a901d1089adf719a29d8aaba53b3ce7 for removal 2024.10.22 02:22:21.063997 [ 1476 ] {} CacheMetadata: Key 9a901d1089adf719a29d8aaba53b3ce7 is removed from metadata 2024.10.22 02:22:21.135444 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_916_281} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:21.163297 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_916_281} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_915_280, total 13902 rows starting from the beginning of the part 2024.10.22 02:22:21.270849 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_916_281} MergeTreeSequentialSource: Reading 2 marks from part 202410_916_916_0, total 12 rows starting from the beginning of the part 2024.10.22 02:22:21.358524 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:22:21.359262 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:22:21.808807 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9e815cc5624d5630616b401639c674db) : [0, 551085]: Setting download as failed: File segment: [0, 551085], key: 9e815cc5624d5630616b401639c674db, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:21.812288 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9e815cc5624d5630616b401639c674db): No space left in cache to reserve 551086 bytes, will continue without cache download 2024.10.22 02:22:21.860208 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9e815cc5624d5630616b401639c674db, offset: 0, size: 0 2024.10.22 02:22:21.861793 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9e815cc5624d5630616b401639c674db for removal 2024.10.22 02:22:21.864004 [ 1476 ] {} CacheMetadata: Key 9e815cc5624d5630616b401639c674db is removed from metadata 2024.10.22 02:22:22.058522 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_868_162/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:22:22.135196 [ 1262 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 252 ms. 2024.10.22 02:22:22.260280 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_856_225, 202410_857_857_0] 2024.10.22 02:22:22.264273 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_856_225, 202410_857_857_0] 2024.10.22 02:22:22.265832 [ 1276 ] {} DatabaseCatalog: Execution took 509 ms. 2024.10.22 02:22:22.267489 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_916_281} MergeTask::MergeProjectionsStage: Merge sorted 13914 rows, containing 12 columns (12 merged, 0 gathered) in 1.407532106 sec., 9885.387296451483 rows/sec., 1.54 MiB/sec. 2024.10.22 02:22:22.274514 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_856_225, 202410_857_857_0] 2024.10.22 02:22:22.393569 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 122 entries to flush up to offset 98624 2024.10.22 02:22:22.529480 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_916_281} MergedBlockOutputStream: filled checksums 202410_1_916_281 (state Temporary) 2024.10.22 02:22:22.610514 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_916_281} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_916_281 to 202410_1_916_281 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:22.639572 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_916_281} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_915_280, 202410_916_916_0] -> 202410_1_916_281 2024.10.22 02:22:22.656059 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 11.23 MiB, will set to 1.58 GiB (RSS), difference: 3.36 MiB 2024.10.22 02:22:22.662078 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:22:22.762546 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:22.764323 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:22.795766 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_892_892_0 (state Temporary) 2024.10.22 02:22:22.959787 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_892_892_0 to 202410_898_898_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:23.049266 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 98624 2024.10.22 02:22:23.092470 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(254e02382c7a77014259761cad539c8f) : [0, 554684]: Setting download as failed: File segment: [0, 554684], key: 254e02382c7a77014259761cad539c8f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:23.097169 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(254e02382c7a77014259761cad539c8f): No space left in cache to reserve 554685 bytes, will continue without cache download 2024.10.22 02:22:23.148148 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 254e02382c7a77014259761cad539c8f, offset: 0, size: 0 2024.10.22 02:22:23.149507 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 254e02382c7a77014259761cad539c8f for removal 2024.10.22 02:22:23.151925 [ 1476 ] {} CacheMetadata: Key 254e02382c7a77014259761cad539c8f is removed from metadata 2024.10.22 02:22:23.896296 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:23.901297 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:24.129386 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e6977e00f88c970ae744e94474895d84) : [0, 342833]: Setting download as failed: File segment: [0, 342833], key: e6977e00f88c970ae744e94474895d84, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:24.132291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e6977e00f88c970ae744e94474895d84): No space left in cache to reserve 342834 bytes, will continue without cache download 2024.10.22 02:22:24.288792 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 102411 2024.10.22 02:22:24.383495 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e6977e00f88c970ae744e94474895d84, offset: 0, size: 0 2024.10.22 02:22:24.386929 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e6977e00f88c970ae744e94474895d84 for removal 2024.10.22 02:22:24.392882 [ 1476 ] {} CacheMetadata: Key e6977e00f88c970ae744e94474895d84 is removed from metadata 2024.10.22 02:22:24.457268 [ 1222 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:22:24.493327 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:24.499220 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:24.522548 [ 1222 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:22:24.559440 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 14.50 MiB, will set to 1.58 GiB (RSS), difference: -3.06 MiB 2024.10.22 02:22:24.755976 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1529_1529_0 (state Temporary) 2024.10.22 02:22:24.896322 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1529_1529_0 to 202410_1542_1542_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:24.930777 [ 1167 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1207_1537_90 to 202410_1540_1540_0 2024.10.22 02:22:24.948895 [ 1167 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.04 MiB using storage policy from min volume index 0 2024.10.22 02:22:24.952381 [ 1167 ] {} DiskLocal: Reserved 1.04 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:24.963504 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102411 2024.10.22 02:22:25.041452 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1540_91} MergeTask::PrepareStage: Merging 4 parts: from 202410_1207_1537_90 to 202410_1540_1540_0 into Compact with storage Full 2024.10.22 02:22:25.339354 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1540_91} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:25.368804 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1540_91} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1537_90, total 22835 rows starting from the beginning of the part 2024.10.22 02:22:25.556586 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1540_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_1538_1538_0, total 93 rows starting from the beginning of the part 2024.10.22 02:22:25.656023 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1540_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_1539_1539_0, total 93 rows starting from the beginning of the part 2024.10.22 02:22:25.731892 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5ef717e067a2ea684133718f5d1feac5) : [0, 246828]: Setting download as failed: File segment: [0, 246828], key: 5ef717e067a2ea684133718f5d1feac5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:25.733824 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5ef717e067a2ea684133718f5d1feac5): No space left in cache to reserve 246829 bytes, will continue without cache download 2024.10.22 02:22:25.750143 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5ef717e067a2ea684133718f5d1feac5, offset: 0, size: 0 2024.10.22 02:22:25.754047 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5ef717e067a2ea684133718f5d1feac5 for removal 2024.10.22 02:22:25.762445 [ 1476 ] {} CacheMetadata: Key 5ef717e067a2ea684133718f5d1feac5 is removed from metadata 2024.10.22 02:22:25.840871 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1540_91} MergeTreeSequentialSource: Reading 2 marks from part 202410_1540_1540_0, total 100 rows starting from the beginning of the part 2024.10.22 02:22:26.312023 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 14.15 MiB, will set to 1.59 GiB (RSS), difference: -4.94 MiB 2024.10.22 02:22:26.313298 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(480852c0b69b4aeda7dde4d4245c9b5b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 480852c0b69b4aeda7dde4d4245c9b5b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:22:26.323209 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 480852c0b69b4aeda7dde4d4245c9b5b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1398, kind: Regular, unbound: 0 2024.10.22 02:22:26.328828 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:22:26.357610 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 753 2024.10.22 02:22:26.758531 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:26.763244 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:27.099215 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_735_735_0 (state Temporary) 2024.10.22 02:22:27.111129 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1316808 2024.10.22 02:22:27.226878 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(537c37c5afd41136f8872c9db907c7fa) : [0, 401794]: Setting download as failed: File segment: [0, 401794], key: 537c37c5afd41136f8872c9db907c7fa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:27.229400 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(537c37c5afd41136f8872c9db907c7fa): No space left in cache to reserve 401795 bytes, will continue without cache download 2024.10.22 02:22:27.375215 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 537c37c5afd41136f8872c9db907c7fa, offset: 0, size: 0 2024.10.22 02:22:27.377829 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 537c37c5afd41136f8872c9db907c7fa for removal 2024.10.22 02:22:27.391561 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_735_735_0 to 202410_745_745_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:27.486761 [ 1476 ] {} CacheMetadata: Key 537c37c5afd41136f8872c9db907c7fa is removed from metadata 2024.10.22 02:22:27.493713 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 753 2024.10.22 02:22:27.580469 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_182_182_0 (state Temporary) 2024.10.22 02:22:27.731110 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1540_91} MergeTask::MergeProjectionsStage: Merge sorted 23121 rows, containing 13 columns (13 merged, 0 gathered) in 2.710765582 sec., 8529.324761066706 rows/sec., 1.55 MiB/sec. 2024.10.22 02:22:28.140477 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1540_91} MergedBlockOutputStream: filled checksums 202410_1207_1540_91 (state Temporary) 2024.10.22 02:22:28.184771 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:28.194648 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:28.370429 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(758fb0e8b04a9a63eb39fae1870a085e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 758fb0e8b04a9a63eb39fae1870a085e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 02:22:28.373888 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 760401, file segment info: File segment: [0, 33554431], key: 758fb0e8b04a9a63eb39fae1870a085e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 02:22:28.381455 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:22:28.384439 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 10.61 MiB, will set to 1.59 GiB (RSS), difference: -3.59 MiB 2024.10.22 02:22:28.447089 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 13878 2024.10.22 02:22:28.468448 [ 1739 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jku/bojjskujdydfdifmlpifqkzsdcyiw, size 760401 2024.10.22 02:22:28.499633 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1540_91} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1540_91 to 202410_1207_1540_91 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:28.512355 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1540_91} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1207_1537_90, 202410_1540_1540_0] -> 202410_1207_1540_91 2024.10.22 02:22:28.558615 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_869_869_0 (state Temporary) 2024.10.22 02:22:28.599258 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:28.603139 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:28.647535 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 11.69 MiB. 2024.10.22 02:22:28.647865 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_908_908_0 (state Temporary) 2024.10.22 02:22:28.661400 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UserAgent 2024.10.22 02:22:28.672659 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UserAgent 2024.10.22 02:22:28.693768 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_869_869_0 to 202410_876_876_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:28.701225 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_908_908_0 to 202410_921_921_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:28.704570 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UserAgent 2024.10.22 02:22:28.754421 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1316808 2024.10.22 02:22:28.775737 [ 1170 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_916_281 to 202410_917_917_0 2024.10.22 02:22:28.782564 [ 1170 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:28.787830 [ 1170 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:22:28.795925 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13878 2024.10.22 02:22:28.970375 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_917_282} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_916_281 to 202410_917_917_0 into Compact with storage Full 2024.10.22 02:22:28.971314 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 102481 2024.10.22 02:22:29.027368 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UserAgent 2024.10.22 02:22:29.064481 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UserAgent 2024.10.22 02:22:29.099673 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UserAgent 2024.10.22 02:22:29.155315 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UserAgent 2024.10.22 02:22:29.191274 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UserAgent 2024.10.22 02:22:29.246408 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_917_282} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:29.251483 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UserAgent 2024.10.22 02:22:29.395573 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_917_282} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_916_281, total 13914 rows starting from the beginning of the part 2024.10.22 02:22:29.440091 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8d6ae6706a53295d3a91907210ae4b1f) : [0, 346109]: Setting download as failed: File segment: [0, 346109], key: 8d6ae6706a53295d3a91907210ae4b1f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:29.442865 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8d6ae6706a53295d3a91907210ae4b1f): No space left in cache to reserve 346110 bytes, will continue without cache download 2024.10.22 02:22:29.455922 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:29.459064 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:29.490546 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/srs/njaspvmfrabogwwkxupsrrjlqlzbh (LogSeriesLimiter: on interval from 2024-10-22 02:22:16 to 2024-10-22 02:22:29 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:22:29.558583 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1530_1530_0 (state Temporary) 2024.10.22 02:22:29.594466 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8d6ae6706a53295d3a91907210ae4b1f, offset: 0, size: 0 2024.10.22 02:22:29.597830 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8d6ae6706a53295d3a91907210ae4b1f for removal 2024.10.22 02:22:29.602903 [ 1476 ] {} CacheMetadata: Key 8d6ae6706a53295d3a91907210ae4b1f is removed from metadata 2024.10.22 02:22:29.746887 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1530_1530_0 to 202410_1543_1543_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:29.786885 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_917_282} MergeTreeSequentialSource: Reading 2 marks from part 202410_917_917_0, total 10 rows starting from the beginning of the part 2024.10.22 02:22:29.807801 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102481 2024.10.22 02:22:30.571420 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 120 entries to flush up to offset 98744 2024.10.22 02:22:30.708287 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:30.714889 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:30.770220 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_893_893_0 (state Temporary) 2024.10.22 02:22:30.850684 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9a5e546c6944b7c40852e64bb479886c) : [0, 199278]: Setting download as failed: File segment: [0, 199278], key: 9a5e546c6944b7c40852e64bb479886c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:30.852022 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9a5e546c6944b7c40852e64bb479886c): No space left in cache to reserve 199279 bytes, will continue without cache download 2024.10.22 02:22:30.945917 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_893_893_0 to 202410_899_899_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:30.992275 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9a5e546c6944b7c40852e64bb479886c, offset: 0, size: 0 2024.10.22 02:22:30.995950 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9a5e546c6944b7c40852e64bb479886c for removal 2024.10.22 02:22:31.000333 [ 1476 ] {} CacheMetadata: Key 9a5e546c6944b7c40852e64bb479886c is removed from metadata 2024.10.22 02:22:31.000756 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_917_282} MergeTask::MergeProjectionsStage: Merge sorted 13924 rows, containing 12 columns (12 merged, 0 gathered) in 2.074037431 sec., 6713.475751151957 rows/sec., 1.05 MiB/sec. 2024.10.22 02:22:31.115920 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 98744 2024.10.22 02:22:31.181775 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_917_282} MergedBlockOutputStream: filled checksums 202410_1_917_282 (state Temporary) 2024.10.22 02:22:31.356775 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:22:31.358856 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:22:31.523142 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 9.13 MiB, will set to 1.59 GiB (RSS), difference: 2.42 MiB 2024.10.22 02:22:31.563796 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_917_282} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_917_282 to 202410_1_917_282 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:31.563797 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_857_226, 202410_858_858_0] 2024.10.22 02:22:31.564939 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_857_226, 202410_858_858_0] 2024.10.22 02:22:31.565954 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_917_282} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_916_281, 202410_917_917_0] -> 202410_1_917_282 2024.10.22 02:22:31.588642 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_857_226, 202410_858_858_0] 2024.10.22 02:22:31.640770 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:22:31.980135 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e67146e3ebda4d447116d904af89d052) : [0, 198025]: Setting download as failed: File segment: [0, 198025], key: e67146e3ebda4d447116d904af89d052, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:31.984074 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e67146e3ebda4d447116d904af89d052): No space left in cache to reserve 198026 bytes, will continue without cache download 2024.10.22 02:22:32.055158 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_868_162} MergeTask::MergeProjectionsStage: Merge sorted 1317308 rows, containing 4 columns (4 merged, 0 gathered) in 37.995437073 sec., 34670.16309008574 rows/sec., 520.00 KiB/sec. 2024.10.22 02:22:32.245269 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e67146e3ebda4d447116d904af89d052, offset: 0, size: 0 2024.10.22 02:22:32.248999 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e67146e3ebda4d447116d904af89d052 for removal 2024.10.22 02:22:32.299513 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_868_162} MergedBlockOutputStream: filled checksums 202410_1_868_162 (state Temporary) 2024.10.22 02:22:32.359740 [ 1476 ] {} CacheMetadata: Key e67146e3ebda4d447116d904af89d052 is removed from metadata 2024.10.22 02:22:32.761244 [ 1193 ] {} DatabaseCatalog: Execution took 494 ms. 2024.10.22 02:22:33.515129 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_184_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:22:33.527576 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 8.40 MiB, will set to 1.59 GiB (RSS), difference: 10.10 MiB 2024.10.22 02:22:33.557892 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(533907b89ff001c5615945cec07c55ce) : [0, 168444]: Setting download as failed: File segment: [0, 168444], key: 533907b89ff001c5615945cec07c55ce, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:33.559252 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(533907b89ff001c5615945cec07c55ce): No space left in cache to reserve 168445 bytes, will continue without cache download 2024.10.22 02:22:33.616509 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 533907b89ff001c5615945cec07c55ce, offset: 0, size: 0 2024.10.22 02:22:33.619856 [ 1721 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yds/pkngiwffskgzsydogewhjsttxurav, size 1607669 2024.10.22 02:22:33.619765 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 533907b89ff001c5615945cec07c55ce for removal 2024.10.22 02:22:33.628524 [ 1476 ] {} CacheMetadata: Key 533907b89ff001c5615945cec07c55ce is removed from metadata 2024.10.22 02:22:33.772015 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 02:22:33.783231 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 02:22:33.799046 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 02:22:33.829158 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 102544 2024.10.22 02:22:33.830841 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 02:22:33.863806 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 02:22:33.897831 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 02:22:33.926985 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column NSToDOMContentLoadedTiming 2024.10.22 02:22:34.097468 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xoo/dztiangbotznznddibjfjqmpyatgf (LogSeriesLimiter: on interval from 2024-10-22 02:22:29 to 2024-10-22 02:22:34 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:22:34.246396 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:34.250087 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:34.417369 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_815_815_0, 202410_816_816_0, 202410_817_817_0, 202410_818_818_0, 202410_819_819_0, 202410_820_820_0] 2024.10.22 02:22:34.419642 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_815_815_0, 202410_816_816_0, 202410_817_817_0, 202410_818_818_0, 202410_819_819_0, 202410_820_820_0] 2024.10.22 02:22:34.488212 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c87a29a39b6387fe20ce9c47e274036d) : [0, 389978]: Setting download as failed: File segment: [0, 389978], key: c87a29a39b6387fe20ce9c47e274036d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:34.490459 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c87a29a39b6387fe20ce9c47e274036d): No space left in cache to reserve 389979 bytes, will continue without cache download 2024.10.22 02:22:34.518646 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1531_1531_0 (state Temporary) 2024.10.22 02:22:34.559451 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_815_815_0, 202410_816_816_0, 202410_817_817_0, 202410_818_818_0, 202410_819_819_0, 202410_820_820_0] 2024.10.22 02:22:34.590564 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1531_1531_0 to 202410_1544_1544_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:34.651596 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102544 2024.10.22 02:22:34.736365 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c87a29a39b6387fe20ce9c47e274036d, offset: 0, size: 0 2024.10.22 02:22:34.743594 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c87a29a39b6387fe20ce9c47e274036d for removal 2024.10.22 02:22:34.745603 [ 1476 ] {} CacheMetadata: Key c87a29a39b6387fe20ce9c47e274036d is removed from metadata 2024.10.22 02:22:35.000112 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 754 2024.10.22 02:22:35.271914 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:35.273555 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:35.350516 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_736_736_0 (state Temporary) 2024.10.22 02:22:35.448395 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 7.14 MiB, will set to 1.59 GiB (RSS), difference: -12.15 MiB 2024.10.22 02:22:35.516202 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_736_736_0 to 202410_746_746_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:35.590798 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 754 2024.10.22 02:22:35.760632 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1317963 2024.10.22 02:22:35.799292 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bc555bf8d560afe9206c1b063c777b6d) : [0, 277432]: Setting download as failed: File segment: [0, 277432], key: bc555bf8d560afe9206c1b063c777b6d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:35.801362 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bc555bf8d560afe9206c1b063c777b6d): No space left in cache to reserve 277433 bytes, will continue without cache download 2024.10.22 02:22:35.951614 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bc555bf8d560afe9206c1b063c777b6d, offset: 0, size: 0 2024.10.22 02:22:35.953066 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bc555bf8d560afe9206c1b063c777b6d for removal 2024.10.22 02:22:35.956462 [ 1476 ] {} CacheMetadata: Key bc555bf8d560afe9206c1b063c777b6d is removed from metadata 2024.10.22 02:22:35.992457 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_868_162} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_868_162 to 202410_1_868_162 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:35.997127 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_868_162} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_862_161, 202410_863_868_1] -> 202410_1_868_162 2024.10.22 02:22:36.128427 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:22:36.298239 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 13895 2024.10.22 02:22:36.433428 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:36.439890 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:36.456990 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:36.459249 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:36.472489 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_909_909_0 (state Temporary) 2024.10.22 02:22:36.608651 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_870_870_0 (state Temporary) 2024.10.22 02:22:36.641652 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_909_909_0 to 202410_922_922_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:36.673501 [ 1210 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_917_282 to 202410_918_918_0 2024.10.22 02:22:36.682831 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_870_870_0 to 202410_877_877_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:36.684057 [ 1210 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:36.688955 [ 1210 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:36.691889 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_182_182_0 to 202410_185_185_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:36.797546 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13895 2024.10.22 02:22:36.839340 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1317963 2024.10.22 02:22:36.835421 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_918_283} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_917_282 to 202410_918_918_0 into Compact with storage Full 2024.10.22 02:22:36.932809 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_918_283} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:36.946339 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_918_283} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_917_282, total 13924 rows starting from the beginning of the part 2024.10.22 02:22:36.990391 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_918_283} MergeTreeSequentialSource: Reading 2 marks from part 202410_918_918_0, total 8 rows starting from the beginning of the part 2024.10.22 02:22:37.268120 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(438dbb74e28b9f61a08a44aa3ff5b82c) : [0, 989572]: Setting download as failed: File segment: [0, 989572], key: 438dbb74e28b9f61a08a44aa3ff5b82c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:37.269309 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(438dbb74e28b9f61a08a44aa3ff5b82c): No space left in cache to reserve 989573 bytes, will continue without cache download 2024.10.22 02:22:37.339649 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 438dbb74e28b9f61a08a44aa3ff5b82c, offset: 0, size: 0 2024.10.22 02:22:37.343685 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 438dbb74e28b9f61a08a44aa3ff5b82c for removal 2024.10.22 02:22:37.348114 [ 1476 ] {} CacheMetadata: Key 438dbb74e28b9f61a08a44aa3ff5b82c is removed from metadata 2024.10.22 02:22:37.357665 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 7.90 MiB, will set to 1.58 GiB (RSS), difference: 7.62 MiB 2024.10.22 02:22:37.777314 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_918_283} MergeTask::MergeProjectionsStage: Merge sorted 13932 rows, containing 12 columns (12 merged, 0 gathered) in 1.062201927 sec., 13116.150183749383 rows/sec., 2.05 MiB/sec. 2024.10.22 02:22:38.155899 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_918_283} MergedBlockOutputStream: filled checksums 202410_1_918_283 (state Temporary) 2024.10.22 02:22:38.260191 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1207_1440_48, 202410_1441_1441_0, 202410_1442_1442_0, 202410_1443_1443_0] 2024.10.22 02:22:38.263696 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1207_1440_48, 202410_1441_1441_0, 202410_1442_1442_0, 202410_1443_1443_0] 2024.10.22 02:22:38.291242 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1207_1440_48, 202410_1441_1441_0, 202410_1442_1442_0, 202410_1443_1443_0] 2024.10.22 02:22:38.311660 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b59df9bd36399cad6a52696ad4fb2f4c) : [0, 1088994]: Setting download as failed: File segment: [0, 1088994], key: b59df9bd36399cad6a52696ad4fb2f4c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:38.314532 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b59df9bd36399cad6a52696ad4fb2f4c): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:22:38.363419 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b59df9bd36399cad6a52696ad4fb2f4c, offset: 0, size: 0 2024.10.22 02:22:38.364466 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b59df9bd36399cad6a52696ad4fb2f4c for removal 2024.10.22 02:22:38.367128 [ 1476 ] {} CacheMetadata: Key b59df9bd36399cad6a52696ad4fb2f4c is removed from metadata 2024.10.22 02:22:38.660460 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 102614 2024.10.22 02:22:38.670242 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_918_283} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_918_283 to 202410_1_918_283 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:38.672943 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_918_283} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_917_282, 202410_918_918_0] -> 202410_1_918_283 2024.10.22 02:22:38.699404 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:22:38.816445 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:38.828000 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:38.869747 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1532_1532_0 (state Temporary) 2024.10.22 02:22:39.009193 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 86 entries to flush up to offset 98830 2024.10.22 02:22:39.053231 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1532_1532_0 to 202410_1545_1545_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:39.098881 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1540_91 to 202410_1541_1541_0 2024.10.22 02:22:39.108775 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.02 MiB using storage policy from min volume index 0 2024.10.22 02:22:39.111512 [ 1250 ] {} DiskLocal: Reserved 1.02 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:39.131173 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102614 2024.10.22 02:22:39.138515 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1541_92} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1540_91 to 202410_1541_1541_0 into Compact with storage Full 2024.10.22 02:22:39.158435 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_858_227, 202410_859_859_0] 2024.10.22 02:22:39.161831 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_858_227, 202410_859_859_0] 2024.10.22 02:22:39.176981 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_858_227, 202410_859_859_0] 2024.10.22 02:22:39.182741 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1541_92} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:39.201320 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1541_92} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1540_91, total 23121 rows starting from the beginning of the part 2024.10.22 02:22:39.230514 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1541_92} MergeTreeSequentialSource: Reading 2 marks from part 202410_1541_1541_0, total 81 rows starting from the beginning of the part 2024.10.22 02:22:39.248520 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:39.253243 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:39.425353 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_894_894_0 (state Temporary) 2024.10.22 02:22:39.452343 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9ad8baff05b6d6a1f23e6e9ca8f022b2) : [0, 372376]: Setting download as failed: File segment: [0, 372376], key: 9ad8baff05b6d6a1f23e6e9ca8f022b2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:22:39.456606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9ad8baff05b6d6a1f23e6e9ca8f022b2): No space left in cache to reserve 372377 bytes, will continue without cache download 2024.10.22 02:22:39.482441 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_894_894_0 to 202410_900_900_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:39.489102 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9ad8baff05b6d6a1f23e6e9ca8f022b2, offset: 0, size: 0 2024.10.22 02:22:39.492703 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9ad8baff05b6d6a1f23e6e9ca8f022b2 for removal 2024.10.22 02:22:39.498775 [ 1476 ] {} CacheMetadata: Key 9ad8baff05b6d6a1f23e6e9ca8f022b2 is removed from metadata 2024.10.22 02:22:39.538223 [ 1256 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:22:39.594542 [ 1256 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:22:39.673821 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 98830 2024.10.22 02:22:39.668498 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 5.97 MiB, will set to 1.58 GiB (RSS), difference: -3.91 MiB 2024.10.22 02:22:39.877920 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7344 2024.10.22 02:22:40.320389 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(5bf5a59d2a58d56bcbedf96f434ac048) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5bf5a59d2a58d56bcbedf96f434ac048, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:22:40.326576 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 5bf5a59d2a58d56bcbedf96f434ac048, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:22:40.331925 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:22:40.653250 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1541_92} MergeTask::MergeProjectionsStage: Merge sorted 23202 rows, containing 13 columns (13 merged, 0 gathered) in 1.519785152 sec., 15266.631582409353 rows/sec., 2.77 MiB/sec. 2024.10.22 02:22:40.732488 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1541_92} MergedBlockOutputStream: filled checksums 202410_1207_1541_92 (state Temporary) 2024.10.22 02:22:41.065294 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 6 file segments (while reserving 83510 bytes for 4a3ba6638f8fedc22eb6539986dc1a17:0) 2024.10.22 02:22:41.067303 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: de543618484d2ed093800f94fbf77fcc, offset: 0, size: 163 2024.10.22 02:22:41.070923 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key de543618484d2ed093800f94fbf77fcc for removal 2024.10.22 02:22:41.085129 [ 1476 ] {} CacheMetadata: Key de543618484d2ed093800f94fbf77fcc is removed from metadata 2024.10.22 02:22:41.086444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5e0a30193274bd02fd7ff8913327fd09, offset: 0, size: 4836 2024.10.22 02:22:41.091841 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5e0a30193274bd02fd7ff8913327fd09 for removal 2024.10.22 02:22:41.101159 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0e8a44267a050940ba098c411438049d, offset: 0, size: 146 2024.10.22 02:22:41.108205 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0e8a44267a050940ba098c411438049d for removal 2024.10.22 02:22:41.109554 [ 1476 ] {} CacheMetadata: Key 5e0a30193274bd02fd7ff8913327fd09 is removed from metadata 2024.10.22 02:22:41.113832 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 77acdbc72bff907a374ef27cdc8c2325, offset: 0, size: 150 2024.10.22 02:22:41.113659 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1541_92} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1541_92 to 202410_1207_1541_92 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:41.122982 [ 1476 ] {} CacheMetadata: Key 0e8a44267a050940ba098c411438049d is removed from metadata 2024.10.22 02:22:41.124007 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 77acdbc72bff907a374ef27cdc8c2325 for removal 2024.10.22 02:22:41.124175 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1541_92} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1540_91, 202410_1541_1541_0] -> 202410_1207_1541_92 2024.10.22 02:22:41.132844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1c30c58f7e471e966e9d1232815ede6b, offset: 0, size: 95624 2024.10.22 02:22:41.136342 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1c30c58f7e471e966e9d1232815ede6b for removal 2024.10.22 02:22:41.140631 [ 1476 ] {} CacheMetadata: Key 77acdbc72bff907a374ef27cdc8c2325 is removed from metadata 2024.10.22 02:22:41.143202 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cfb54a1a42b67ce54c000b86bfc9e5ac, offset: 0, size: 153 2024.10.22 02:22:41.146157 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cfb54a1a42b67ce54c000b86bfc9e5ac for removal 2024.10.22 02:22:41.147536 [ 1476 ] {} CacheMetadata: Key 1c30c58f7e471e966e9d1232815ede6b is removed from metadata 2024.10.22 02:22:41.169553 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/gqs/ycfwsjptikpgbrtglltbmeiueelmo. 2024.10.22 02:22:41.198532 [ 1476 ] {} CacheMetadata: Key cfb54a1a42b67ce54c000b86bfc9e5ac is removed from metadata 2024.10.22 02:22:41.206954 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/seo/fswwgxnjyiervjouytcivembbzasm. 2024.10.22 02:22:41.255810 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.05 MiB. 2024.10.22 02:22:41.356726 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:22:41.357814 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:22:41.404937 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 6.88 MiB, will set to 1.58 GiB (RSS), difference: 13.45 MiB 2024.10.22 02:22:41.683982 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_693_141, 202410_694_694_0, 202410_695_695_0, 202410_696_696_0, 202410_697_697_0, 202410_698_698_0] 2024.10.22 02:22:41.688814 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_693_141, 202410_694_694_0, 202410_695_695_0, 202410_696_696_0, 202410_697_697_0, 202410_698_698_0] 2024.10.22 02:22:41.814346 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_693_141, 202410_694_694_0, 202410_695_695_0, 202410_696_696_0, 202410_697_697_0, 202410_698_698_0] 2024.10.22 02:22:42.471352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/seo/fswwgxnjyiervjouytcivembbzasm. 2024.10.22 02:22:42.498780 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_18_18_0 to 201403_18_18_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:42.525828 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/gqs/ycfwsjptikpgbrtglltbmeiueelmo. 2024.10.22 02:22:42.616701 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 7.09 MiB, will set to 1.58 GiB (RSS), difference: -3.19 MiB 2024.10.22 02:22:42.653670 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:22:42.913124 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 61 2024.10.22 02:22:43.095844 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 755 2024.10.22 02:22:43.148265 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 102690 2024.10.22 02:22:43.314576 [ 1259 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 215 ms. 2024.10.22 02:22:43.402218 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_837_18, 202410_838_838_0, 202410_839_839_0, 202410_840_840_0, 202410_841_841_0, 202410_842_842_0] 2024.10.22 02:22:43.405974 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_837_18, 202410_838_838_0, 202410_839_839_0, 202410_840_840_0, 202410_841_841_0, 202410_842_842_0] 2024.10.22 02:22:43.473508 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:43.476683 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:43.478812 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:43.481548 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:43.543384 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_46_46_0 (state Temporary) 2024.10.22 02:22:43.544673 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_837_18, 202410_838_838_0, 202410_839_839_0, 202410_840_840_0, 202410_841_841_0, 202410_842_842_0] 2024.10.22 02:22:43.704390 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:43.709534 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_46_46_0 to 202410_46_46_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:43.711919 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:43.752284 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_737_737_0 (state Temporary) 2024.10.22 02:22:43.779026 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1533_1533_0 (state Temporary) 2024.10.22 02:22:43.834399 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_737_737_0 to 202410_747_747_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:43.867186 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Found 5 old parts to remove. Parts: [202410_1_38_9, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0, 202410_42_42_0] 2024.10.22 02:22:43.870161 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from filesystem (serially): Parts: [202410_1_38_9, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0, 202410_42_42_0] 2024.10.22 02:22:43.891177 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 61 2024.10.22 02:22:43.915605 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1319888 2024.10.22 02:22:43.933186 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1533_1533_0 to 202410_1546_1546_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:43.936605 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 755 2024.10.22 02:22:43.969024 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102690 2024.10.22 02:22:44.062331 [ 1217 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1541_92 to 202410_1542_1542_0 2024.10.22 02:22:44.071984 [ 1217 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 02:22:44.079064 [ 1217 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:44.107634 [ 1420 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from memory: Parts: [202410_1_38_9, 202410_39_39_0, 202410_40_40_0, 202410_41_41_0, 202410_42_42_0] 2024.10.22 02:22:44.128316 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1542_93} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1541_92 to 202410_1542_1542_0 into Compact with storage Full 2024.10.22 02:22:44.310506 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1542_93} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:44.402258 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 13910 2024.10.22 02:22:44.439810 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.11 GiB, peak 2.00 GiB, free memory in arenas 3.63 MiB, will set to 1.32 GiB (RSS), difference: 211.60 MiB 2024.10.22 02:22:44.451549 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1542_93} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1541_92, total 23202 rows starting from the beginning of the part 2024.10.22 02:22:44.612429 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:44.615952 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:44.617551 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1542_93} MergeTreeSequentialSource: Reading 2 marks from part 202410_1542_1542_0, total 68 rows starting from the beginning of the part 2024.10.22 02:22:44.666550 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_910_910_0 (state Temporary) 2024.10.22 02:22:44.723500 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_910_910_0 to 202410_923_923_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:44.754414 [ 1167 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_918_283 to 202410_919_919_0 2024.10.22 02:22:44.760682 [ 1167 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:44.764599 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:44.815929 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_919_284} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_918_283 to 202410_919_919_0 into Compact with storage Full 2024.10.22 02:22:44.904469 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13910 2024.10.22 02:22:44.912026 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_919_284} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:44.922163 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_919_284} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_918_283, total 13932 rows starting from the beginning of the part 2024.10.22 02:22:44.943209 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:44.945793 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:45.094082 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_919_284} MergeTreeSequentialSource: Reading 2 marks from part 202410_919_919_0, total 22 rows starting from the beginning of the part 2024.10.22 02:22:45.216490 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_871_871_0 (state Temporary) 2024.10.22 02:22:45.290496 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_871_871_0 to 202410_878_878_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:45.448841 [ 1154 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_868_162 to 202410_869_874_1 2024.10.22 02:22:45.451078 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1319888 2024.10.22 02:22:45.455463 [ 1154 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 02:22:45.458484 [ 1154 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:45.497687 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_874_163} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_868_162 to 202410_869_874_1 into Wide with storage Full 2024.10.22 02:22:45.552930 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_874_163} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:45.559804 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_874_163} MergeTreeSequentialSource: Reading 162 marks from part 202410_1_868_162, total 1317308 rows starting from the beginning of the part 2024.10.22 02:22:45.596461 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_874_163} MergeTreeSequentialSource: Reading 3 marks from part 202410_869_874_1, total 9625 rows starting from the beginning of the part 2024.10.22 02:22:46.066724 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1542_93} MergeTask::MergeProjectionsStage: Merge sorted 23270 rows, containing 13 columns (13 merged, 0 gathered) in 1.953773144 sec., 11910.287574308064 rows/sec., 2.16 MiB/sec. 2024.10.22 02:22:46.261442 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1542_93} MergedBlockOutputStream: filled checksums 202410_1207_1542_93 (state Temporary) 2024.10.22 02:22:46.431981 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_919_284} MergeTask::MergeProjectionsStage: Merge sorted 13954 rows, containing 12 columns (12 merged, 0 gathered) in 1.636496578 sec., 8526.751713134348 rows/sec., 1.33 MiB/sec. 2024.10.22 02:22:46.524761 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.43 GiB, peak 2.00 GiB, free memory in arenas 3.64 MiB, will set to 1.37 GiB (RSS), difference: -65.44 MiB 2024.10.22 02:22:46.666442 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_184_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:22:46.679392 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_919_284} MergedBlockOutputStream: filled checksums 202410_1_919_284 (state Temporary) 2024.10.22 02:22:46.745038 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(b81a7bdfc0eb6392a2f335471636760e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b81a7bdfc0eb6392a2f335471636760e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:22:46.748452 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 863615, file segment info: File segment: [0, 33554431], key: b81a7bdfc0eb6392a2f335471636760e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:22:46.751756 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:22:46.786127 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1542_93} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1542_93 to 202410_1207_1542_93 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:46.793570 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1542_93} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1541_92, 202410_1542_1542_0] -> 202410_1207_1542_93 2024.10.22 02:22:46.831592 [ 1734 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/srs/njaspvmfrabogwwkxupsrrjlqlzbh, size 863615 2024.10.22 02:22:46.948161 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:22:47.138054 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_919_284} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_919_284 to 202410_1_919_284 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:47.143008 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_919_284} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_918_283, 202410_919_919_0] -> 202410_1_919_284 2024.10.22 02:22:47.202634 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:22:47.246893 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.413815757 sec. 2024.10.22 02:22:47.299691 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 151 entries to flush up to offset 98981 2024.10.22 02:22:47.514895 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column URL 2024.10.22 02:22:47.543460 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 7386 2024.10.22 02:22:47.635954 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column URL 2024.10.22 02:22:47.655750 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column URL 2024.10.22 02:22:47.669297 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column URL 2024.10.22 02:22:47.698508 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column URL 2024.10.22 02:22:47.702966 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:47.708455 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:47.720163 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column URL 2024.10.22 02:22:47.751707 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column URL 2024.10.22 02:22:47.783820 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column URL 2024.10.22 02:22:47.818630 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column URL 2024.10.22 02:22:47.952772 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_895_895_0 (state Temporary) 2024.10.22 02:22:47.964139 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_859_228, 202410_860_860_0] 2024.10.22 02:22:47.966114 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_859_228, 202410_860_860_0] 2024.10.22 02:22:47.975170 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_859_228, 202410_860_860_0] 2024.10.22 02:22:47.905220 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qsl/tibiwegtqryocqbvgxhglxqvbqnyi (LogSeriesLimiter: on interval from 2024-10-22 02:22:34 to 2024-10-22 02:22:47 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:22:48.048149 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 92 entries to flush up to offset 102782 2024.10.22 02:22:48.231340 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_895_895_0 to 202410_901_901_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:48.292164 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 98981 2024.10.22 02:22:48.670993 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:48.673305 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:48.878747 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 2.00 GiB, free memory in arenas 11.00 MiB, will set to 1.39 GiB (RSS), difference: -50.79 MiB 2024.10.22 02:22:49.157047 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1534_1534_0 (state Temporary) 2024.10.22 02:22:49.213353 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1534_1534_0 to 202410_1547_1547_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:49.340677 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1542_93 to 202410_1543_1543_0 2024.10.22 02:22:49.345313 [ 1186 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 02:22:49.347753 [ 1186 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:49.367575 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102782 2024.10.22 02:22:49.413684 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1543_94} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1542_93 to 202410_1543_1543_0 into Compact with storage Full 2024.10.22 02:22:49.499800 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1543_94} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:49.516814 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1543_94} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1542_93, total 23270 rows starting from the beginning of the part 2024.10.22 02:22:49.667887 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1543_94} MergeTreeSequentialSource: Reading 2 marks from part 202410_1543_1543_0, total 70 rows starting from the beginning of the part 2024.10.22 02:22:50.331386 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_874_163/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:22:50.650658 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 2.00 GiB, free memory in arenas 12.16 MiB, will set to 1.45 GiB (RSS), difference: -36.69 MiB 2024.10.22 02:22:51.276727 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1543_94} MergeTask::MergeProjectionsStage: Merge sorted 23340 rows, containing 13 columns (13 merged, 0 gathered) in 1.923896812 sec., 12131.627774639714 rows/sec., 2.20 MiB/sec. 2024.10.22 02:22:51.356978 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:22:51.358005 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:22:51.496137 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1543_94} MergedBlockOutputStream: filled checksums 202410_1207_1543_94 (state Temporary) 2024.10.22 02:22:51.540974 [ 1732 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xoo/dztiangbotznznddibjfjqmpyatgf, size 3224825 2024.10.22 02:22:51.541489 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 756 2024.10.22 02:22:51.572183 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1543_94} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1543_94 to 202410_1207_1543_94 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:51.578375 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1543_94} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1542_93, 202410_1543_1543_0] -> 202410_1207_1543_94 2024.10.22 02:22:51.580942 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 02:22:51.730439 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 02:22:51.742301 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:22:51.743432 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 02:22:51.751501 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 02:22:51.768873 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 02:22:51.790749 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 02:22:51.807270 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FirstPaintTiming 2024.10.22 02:22:52.277616 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1207_1443_49, 202410_1444_1444_0, 202410_1445_1445_0, 202410_1446_1446_0] 2024.10.22 02:22:52.281129 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1207_1443_49, 202410_1444_1444_0, 202410_1445_1445_0, 202410_1446_1446_0] 2024.10.22 02:22:52.307470 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1207_1443_49, 202410_1444_1444_0, 202410_1445_1445_0, 202410_1446_1446_0] 2024.10.22 02:22:52.325134 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:52.333042 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:52.407208 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 13923 2024.10.22 02:22:52.556386 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1321428 2024.10.22 02:22:52.597363 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_738_738_0 (state Temporary) 2024.10.22 02:22:52.610833 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:52.614059 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:52.740293 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_911_911_0 (state Temporary) 2024.10.22 02:22:52.795346 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_738_738_0 to 202410_748_748_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:52.810270 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_911_911_0 to 202410_924_924_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:52.855607 [ 1264 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_919_284 to 202410_920_920_0 2024.10.22 02:22:52.868223 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13923 2024.10.22 02:22:52.884202 [ 1264 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:52.889699 [ 1264 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:52.908525 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 756 2024.10.22 02:22:52.946589 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_743_151 to 202410_748_748_0 2024.10.22 02:22:52.959970 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(c91a7c7b2bfec5a2c0d5cd0de2e9d8fa) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c91a7c7b2bfec5a2c0d5cd0de2e9d8fa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 02:22:52.961190 [ 1211 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:52.964255 [ 1211 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:52.966651 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: c91a7c7b2bfec5a2c0d5cd0de2e9d8fa, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 02:22:52.967032 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_920_285} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_919_284 to 202410_920_920_0 into Compact with storage Full 2024.10.22 02:22:52.971832 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:22:52.984226 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_743_151 to 202410_748_748_0 into Compact with storage Full 2024.10.22 02:22:52.992679 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_920_285} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:53.004295 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_920_285} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_919_284, total 13954 rows starting from the beginning of the part 2024.10.22 02:22:53.064782 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_920_285} MergeTreeSequentialSource: Reading 2 marks from part 202410_920_920_0, total 22 rows starting from the beginning of the part 2024.10.22 02:22:53.069559 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:53.078834 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:53.238488 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_872_872_0 (state Temporary) 2024.10.22 02:22:53.248779 [ 1214 ] {} DatabaseCatalog: Execution took 384 ms. 2024.10.22 02:22:53.347765 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_872_872_0 to 202410_879_879_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:53.386045 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 102850 2024.10.22 02:22:53.406810 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1321428 2024.10.22 02:22:53.528182 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:53.536423 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:53.580609 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:53.598873 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1535_1535_0 (state Temporary) 2024.10.22 02:22:53.606869 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 11.55 MiB, will set to 1.51 GiB (RSS), difference: -49.91 MiB 2024.10.22 02:22:53.649384 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_743_151, total 787 rows starting from the beginning of the part 2024.10.22 02:22:53.753496 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1535_1535_0 to 202410_1548_1548_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:53.790776 [ 1245 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1543_94 to 202410_1544_1544_0 2024.10.22 02:22:53.809430 [ 1245 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 02:22:53.836882 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102850 2024.10.22 02:22:53.845096 [ 1245 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:53.871890 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1544_95} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1543_94 to 202410_1544_1544_0 into Compact with storage Full 2024.10.22 02:22:53.912850 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1544_95} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:53.931886 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1544_95} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1543_94, total 23340 rows starting from the beginning of the part 2024.10.22 02:22:53.964923 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1544_95} MergeTreeSequentialSource: Reading 2 marks from part 202410_1544_1544_0, total 63 rows starting from the beginning of the part 2024.10.22 02:22:54.062876 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_920_285} MergeTask::MergeProjectionsStage: Merge sorted 13976 rows, containing 12 columns (12 merged, 0 gathered) in 1.099635288 sec., 12709.668516931042 rows/sec., 1.99 MiB/sec. 2024.10.22 02:22:54.075151 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_744_744_0, total 1 rows starting from the beginning of the part 2024.10.22 02:22:54.155426 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_920_285} MergedBlockOutputStream: filled checksums 202410_1_920_285 (state Temporary) 2024.10.22 02:22:54.416940 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_745_745_0, total 1 rows starting from the beginning of the part 2024.10.22 02:22:54.488529 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_746_746_0, total 1 rows starting from the beginning of the part 2024.10.22 02:22:54.577566 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_920_285} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_920_285 to 202410_1_920_285 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:54.592526 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_920_285} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_919_284, 202410_920_920_0] -> 202410_1_920_285 2024.10.22 02:22:54.613586 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_747_747_0, total 1 rows starting from the beginning of the part 2024.10.22 02:22:54.627225 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:22:54.627160 [ 1243 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:22:54.658890 [ 1243 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:22:54.726066 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} MergeTreeSequentialSource: Reading 2 marks from part 202410_748_748_0, total 1 rows starting from the beginning of the part 2024.10.22 02:22:54.868579 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_860_229, 202410_861_861_0] 2024.10.22 02:22:54.870063 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_860_229, 202410_861_861_0] 2024.10.22 02:22:54.877478 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_860_229, 202410_861_861_0] 2024.10.22 02:22:55.312672 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_808_152, 202410_809_814_1] 2024.10.22 02:22:55.314691 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_808_152, 202410_809_814_1] 2024.10.22 02:22:55.332443 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_808_152, 202410_809_814_1] 2024.10.22 02:22:55.569153 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.64 MiB, will set to 1.55 GiB (RSS), difference: -25.96 MiB 2024.10.22 02:22:55.847597 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1544_95} MergeTask::MergeProjectionsStage: Merge sorted 23403 rows, containing 13 columns (13 merged, 0 gathered) in 1.985720849 sec., 11785.644498714733 rows/sec., 2.14 MiB/sec. 2024.10.22 02:22:55.850880 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 140 entries to flush up to offset 99121 2024.10.22 02:22:55.956160 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1544_95} MergedBlockOutputStream: filled checksums 202410_1207_1544_95 (state Temporary) 2024.10.22 02:22:56.054628 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:56.058531 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:56.190724 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1544_95} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1544_95 to 202410_1207_1544_95 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:56.195893 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1544_95} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1543_94, 202410_1544_1544_0] -> 202410_1207_1544_95 2024.10.22 02:22:56.250525 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_19_19_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 02:22:56.255911 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:22:56.339606 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_896_896_0 (state Temporary) 2024.10.22 02:22:56.530451 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_896_896_0 to 202410_902_902_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:56.596369 [ 1260 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_897_30 to 202410_902_902_0 2024.10.22 02:22:56.721276 [ 1260 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:56.728437 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:56.755100 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_897_30 to 202410_902_902_0 into Compact with storage Full 2024.10.22 02:22:56.779434 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 99121 2024.10.22 02:22:56.842458 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} MergeTask::MergeProjectionsStage: Merge sorted 792 rows, containing 32 columns (32 merged, 0 gathered) in 3.851442086 sec., 205.6372606195798 rows/sec., 15.54 KiB/sec. 2024.10.22 02:22:56.913575 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:56.915087 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} MergedBlockOutputStream: filled checksums 202410_1_748_152 (state Temporary) 2024.10.22 02:22:56.933012 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_897_30, total 15147 rows starting from the beginning of the part 2024.10.22 02:22:57.056813 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_898_898_0, total 122 rows starting from the beginning of the part 2024.10.22 02:22:57.079098 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_899_899_0, total 120 rows starting from the beginning of the part 2024.10.22 02:22:57.101528 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_900_900_0, total 86 rows starting from the beginning of the part 2024.10.22 02:22:57.239036 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_901_901_0, total 151 rows starting from the beginning of the part 2024.10.22 02:22:57.258773 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} MergeTreeSequentialSource: Reading 2 marks from part 202410_902_902_0, total 140 rows starting from the beginning of the part 2024.10.22 02:22:57.556044 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_748_152 to 202410_1_748_152 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:57.560235 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_748_152} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_743_151, 202410_748_748_0] -> 202410_1_748_152 2024.10.22 02:22:57.613017 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:22:57.623161 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 2.00 GiB, free memory in arenas 12.86 MiB, will set to 1.60 GiB (RSS), difference: -88.58 MiB 2024.10.22 02:22:57.852092 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 102915 2024.10.22 02:22:58.367508 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:22:58.382539 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:58.640809 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1536_1536_0 (state Temporary) 2024.10.22 02:22:58.729211 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1536_1536_0 to 202410_1549_1549_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:58.734911 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 237.72 MiB using storage policy from min volume index 0 2024.10.22 02:22:58.869448 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1544_95 to 202410_1545_1545_0 2024.10.22 02:22:58.883747 [ 1259 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 02:22:58.892573 [ 1259 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:22:58.905298 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1545_96} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1544_95 to 202410_1545_1545_0 into Compact with storage Full 2024.10.22 02:22:58.949698 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} MergeTask::MergeProjectionsStage: Merge sorted 15766 rows, containing 13 columns (13 merged, 0 gathered) in 2.202319473 sec., 7158.816054295497 rows/sec., 3.11 MiB/sec. 2024.10.22 02:22:58.978409 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1545_96} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:22:58.992061 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1545_96} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1544_95, total 23403 rows starting from the beginning of the part 2024.10.22 02:22:59.042099 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102915 2024.10.22 02:22:59.126318 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1545_96} MergeTreeSequentialSource: Reading 2 marks from part 202410_1545_1545_0, total 70 rows starting from the beginning of the part 2024.10.22 02:22:59.157796 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} MergedBlockOutputStream: filled checksums 202410_747_902_31 (state Temporary) 2024.10.22 02:22:59.384634 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_902_31 to 202410_747_902_31 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:22:59.422576 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_902_31} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_897_30, 202410_902_902_0] -> 202410_747_902_31 2024.10.22 02:22:59.473048 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.23 MiB. 2024.10.22 02:22:59.568485 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 11.32 MiB, will set to 1.62 GiB (RSS), difference: 29.40 MiB 2024.10.22 02:22:59.650216 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wfc/drfvqxuqzqvzezjlfnoldbmjljlsl (LogSeriesLimiter: on interval from 2024-10-22 02:22:47 to 2024-10-22 02:22:59 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:22:59.864015 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(ee709b2d721250985ca92849a6ff394a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ee709b2d721250985ca92849a6ff394a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 02:22:59.869375 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: ee709b2d721250985ca92849a6ff394a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1394, kind: Regular, unbound: 0 2024.10.22 02:22:59.871587 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:23:00.370753 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 13949 2024.10.22 02:23:00.479411 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:00.482680 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:00.441761 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1472 entries to flush up to offset 1322900 2024.10.22 02:23:00.617483 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_912_912_0 (state Temporary) 2024.10.22 02:23:00.669595 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1545_96} MergeTask::MergeProjectionsStage: Merge sorted 23473 rows, containing 13 columns (13 merged, 0 gathered) in 1.770586751 sec., 13257.187193309119 rows/sec., 2.40 MiB/sec. 2024.10.22 02:23:00.684021 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_912_912_0 to 202410_925_925_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:00.740601 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1545_96} MergedBlockOutputStream: filled checksums 202410_1207_1545_96 (state Temporary) 2024.10.22 02:23:00.837540 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1545_96} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1545_96 to 202410_1207_1545_96 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:00.840659 [ 1246 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_920_285 to 202410_921_921_0 2024.10.22 02:23:00.842620 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1545_96} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1544_95, 202410_1545_1545_0] -> 202410_1207_1545_96 2024.10.22 02:23:00.848065 [ 1246 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:00.851414 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13949 2024.10.22 02:23:00.867097 [ 1246 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:00.879109 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:23:00.919042 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_921_286} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_920_285 to 202410_921_921_0 into Compact with storage Full 2024.10.22 02:23:00.942574 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_874_163/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:23:01.073352 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_921_286} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:01.086095 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_921_286} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_920_285, total 13976 rows starting from the beginning of the part 2024.10.22 02:23:01.103971 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_921_286} MergeTreeSequentialSource: Reading 2 marks from part 202410_921_921_0, total 8 rows starting from the beginning of the part 2024.10.22 02:23:01.357732 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:23:01.359711 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:23:01.385549 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:01.391608 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:01.560308 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_873_873_0 (state Temporary) 2024.10.22 02:23:01.565003 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 2.00 GiB, free memory in arenas 9.30 MiB, will set to 1.61 GiB (RSS), difference: -146.53 MiB 2024.10.22 02:23:01.743355 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_921_286} MergeTask::MergeProjectionsStage: Merge sorted 13984 rows, containing 12 columns (12 merged, 0 gathered) in 0.837440596 sec., 16698.497859781328 rows/sec., 2.61 MiB/sec. 2024.10.22 02:23:01.794549 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_873_873_0 to 202410_880_880_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:01.807327 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_921_286} MergedBlockOutputStream: filled checksums 202410_1_921_286 (state Temporary) 2024.10.22 02:23:01.849238 [ 1177 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_875_875_0 to 202410_880_880_0 2024.10.22 02:23:01.857391 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1322900 2024.10.22 02:23:01.860703 [ 1177 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:01.869335 [ 1177 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:01.884943 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_875_875_0 to 202410_880_880_0 into Compact with storage Full 2024.10.22 02:23:02.020460 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:02.030978 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_875_875_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:23:02.046086 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_876_876_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:23:02.052360 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_877_877_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:23:02.063033 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_878_878_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:23:02.141504 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_921_286} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_921_286 to 202410_1_921_286 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:02.141827 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_861_230, 202410_862_862_0] 2024.10.22 02:23:02.147530 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_921_286} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_920_285, 202410_921_921_0] -> 202410_1_921_286 2024.10.22 02:23:02.146035 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_861_230, 202410_862_862_0] 2024.10.22 02:23:02.163887 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_861_230, 202410_862_862_0] 2024.10.22 02:23:02.174633 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_879_879_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:23:02.206212 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_880_880_0, total 1472 rows starting from the beginning of the part 2024.10.22 02:23:02.315379 [ 1387 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {5276570e-a364-404e-a403-3e186cba01a8::202410_1_921_286}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 02:23:02.344076 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:23:02.477086 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_184_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:23:03.050924 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 102988 2024.10.22 02:23:03.360022 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} MergeTask::MergeProjectionsStage: Merge sorted 9557 rows, containing 4 columns (4 merged, 0 gathered) in 1.48247895 sec., 6446.634537374039 rows/sec., 100.73 KiB/sec. 2024.10.22 02:23:03.444006 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} MergedBlockOutputStream: filled checksums 202410_875_880_1 (state Temporary) 2024.10.22 02:23:03.451179 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:03.458784 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:03.557276 [ 1161 ] {} DatabaseCatalog: Execution took 307 ms. 2024.10.22 02:23:03.632641 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_875_880_1 to 202410_875_880_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:03.637807 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_875_880_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_875_875_0, 202410_880_880_0] -> 202410_875_880_1 2024.10.22 02:23:03.645078 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1537_1537_0 (state Temporary) 2024.10.22 02:23:03.678669 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.76 GiB, peak 2.00 GiB, free memory in arenas 12.69 MiB, will set to 1.62 GiB (RSS), difference: -140.90 MiB 2024.10.22 02:23:03.680044 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:23:03.916999 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1537_1537_0 to 202410_1550_1550_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:03.955919 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1545_96 to 202410_1550_1550_0 2024.10.22 02:23:03.979066 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.08 MiB using storage policy from min volume index 0 2024.10.22 02:23:03.989308 [ 1171 ] {} DiskLocal: Reserved 1.08 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:04.004156 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 102988 2024.10.22 02:23:04.006894 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kpy/vexkocplusvrntzylomimexppqryr (LogSeriesLimiter: on interval from 2024-10-22 02:22:59 to 2024-10-22 02:23:04 accepted series 1 / 188 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:23:04.059973 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 02:23:04.013781 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1545_96 to 202410_1550_1550_0 into Compact with storage Full 2024.10.22 02:23:04.162146 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:04.170278 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1545_96, total 23473 rows starting from the beginning of the part 2024.10.22 02:23:04.304266 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_1546_1546_0, total 76 rows starting from the beginning of the part 2024.10.22 02:23:04.315631 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 163 entries to flush up to offset 99284 2024.10.22 02:23:04.322964 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_1547_1547_0, total 92 rows starting from the beginning of the part 2024.10.22 02:23:04.355974 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_1548_1548_0, total 68 rows starting from the beginning of the part 2024.10.22 02:23:04.375192 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_1549_1549_0, total 65 rows starting from the beginning of the part 2024.10.22 02:23:04.512433 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} MergeTreeSequentialSource: Reading 2 marks from part 202410_1550_1550_0, total 73 rows starting from the beginning of the part 2024.10.22 02:23:04.674481 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:04.679919 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:04.742614 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_897_897_0 (state Temporary) 2024.10.22 02:23:04.812416 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_897_897_0 to 202410_903_903_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:05.129596 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 99284 2024.10.22 02:23:05.168244 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1446_50, 202410_1447_1447_0] 2024.10.22 02:23:05.170430 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1446_50, 202410_1447_1447_0] 2024.10.22 02:23:05.184526 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1446_50, 202410_1447_1447_0] 2024.10.22 02:23:05.506575 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.77 GiB, peak 2.00 GiB, free memory in arenas 10.29 MiB, will set to 1.63 GiB (RSS), difference: -148.76 MiB 2024.10.22 02:23:06.121728 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} MergeTask::MergeProjectionsStage: Merge sorted 23847 rows, containing 13 columns (13 merged, 0 gathered) in 2.116782432 sec., 11265.683066666721 rows/sec., 2.04 MiB/sec. 2024.10.22 02:23:06.305586 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} MergedBlockOutputStream: filled checksums 202410_1207_1550_97 (state Temporary) 2024.10.22 02:23:06.553128 [ 1156 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 245 ms. 2024.10.22 02:23:06.585614 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1550_97 to 202410_1207_1550_97 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:06.595403 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1550_97} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1545_96, 202410_1550_1550_0] -> 202410_1207_1550_97 2024.10.22 02:23:06.735678 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.33 MiB. 2024.10.22 02:23:07.616182 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.73 GiB, peak 2.00 GiB, free memory in arenas 12.62 MiB, will set to 1.62 GiB (RSS), difference: -114.13 MiB 2024.10.22 02:23:07.911879 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 757 2024.10.22 02:23:08.036851 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 02:23:08.058953 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:08.135591 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 103032 2024.10.22 02:23:08.139566 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:08.298671 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:08.304629 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:08.353667 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 24 entries to flush up to offset 13973 2024.10.22 02:23:08.422211 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_739_739_0 (state Temporary) 2024.10.22 02:23:08.497819 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:08.505634 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:08.579259 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_913_913_0 (state Temporary) 2024.10.22 02:23:08.601002 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_739_739_0 to 202410_749_749_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:08.735728 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_913_913_0 to 202410_926_926_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:08.774730 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 757 2024.10.22 02:23:08.783661 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:08.798579 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:08.805018 [ 1193 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_921_286 to 202410_922_922_0 2024.10.22 02:23:08.810701 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13973 2024.10.22 02:23:08.863333 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1608 entries to flush up to offset 1324508 2024.10.22 02:23:08.863966 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1538_1538_0 (state Temporary) 2024.10.22 02:23:08.921485 [ 1193 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:08.925378 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:08.953716 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1538_1538_0 to 202410_1551_1551_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:08.956915 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_922_287} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_921_286 to 202410_922_922_0 into Compact with storage Full 2024.10.22 02:23:09.031173 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_922_287} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:09.047585 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_922_287} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_921_286, total 13984 rows starting from the beginning of the part 2024.10.22 02:23:09.059051 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103032 2024.10.22 02:23:09.069286 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_922_287} MergeTreeSequentialSource: Reading 2 marks from part 202410_922_922_0, total 17 rows starting from the beginning of the part 2024.10.22 02:23:09.668102 [ 1212 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:23:09.695175 [ 1212 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:23:09.978875 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:09.986607 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:09.991170 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RedirectCount 2024.10.22 02:23:10.023927 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RedirectCount 2024.10.22 02:23:10.052117 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RedirectCount 2024.10.22 02:23:10.066915 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RedirectCount 2024.10.22 02:23:10.117485 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_922_287} MergeTask::MergeProjectionsStage: Merge sorted 14001 rows, containing 12 columns (12 merged, 0 gathered) in 1.164238415 sec., 12025.887326523236 rows/sec., 1.88 MiB/sec. 2024.10.22 02:23:10.246271 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RedirectCount 2024.10.22 02:23:10.253168 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_874_874_0 (state Temporary) 2024.10.22 02:23:10.272698 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RedirectCount 2024.10.22 02:23:10.308804 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_922_287} MergedBlockOutputStream: filled checksums 202410_1_922_287 (state Temporary) 2024.10.22 02:23:10.312808 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RedirectCount 2024.10.22 02:23:10.375212 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tuy/aqweuqzndzbgraiyusqhbaqqzgcdu (LogSeriesLimiter: on interval from 2024-10-22 02:23:04 to 2024-10-22 02:23:10 accepted series 1 / 165 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:23:10.496886 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_874_874_0 to 202410_881_881_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:10.559690 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1324508 2024.10.22 02:23:11.064095 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_862_231, 202410_863_863_0] 2024.10.22 02:23:11.064222 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_922_287} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_922_287 to 202410_1_922_287 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:11.067291 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_862_231, 202410_863_863_0] 2024.10.22 02:23:11.068380 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_922_287} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_921_286, 202410_922_922_0] -> 202410_1_922_287 2024.10.22 02:23:11.097776 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_862_231, 202410_863_863_0] 2024.10.22 02:23:11.122426 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:23:11.216283 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1447_51, 202410_1448_1448_0] 2024.10.22 02:23:11.229049 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1447_51, 202410_1448_1448_0] 2024.10.22 02:23:11.358081 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:23:11.363137 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1447_51, 202410_1448_1448_0] 2024.10.22 02:23:11.363832 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:23:11.529573 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 15.90 MiB, will set to 1.65 GiB (RSS), difference: 13.91 MiB 2024.10.22 02:23:11.760077 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_874_163/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:23:12.077221 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_183_183_0 (state Temporary) 2024.10.22 02:23:12.163453 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_184_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:23:12.663004 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 114 entries to flush up to offset 99398 2024.10.22 02:23:13.065578 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 103093 2024.10.22 02:23:13.148460 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:13.151426 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:13.340762 [ 1249 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Selected 5 parts from 202410_1_42_10 to 202410_46_46_0 2024.10.22 02:23:13.354438 [ 1249 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:13.361421 [ 1249 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:13.392126 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_898_898_0 (state Temporary) 2024.10.22 02:23:13.391728 [ 1393 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_46_11} MergeTask::PrepareStage: Merging 5 parts: from 202410_1_42_10 to 202410_46_46_0 into Compact with storage Full 2024.10.22 02:23:13.464287 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:13.467995 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:13.512892 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 2.00 GiB, free memory in arenas 16.74 MiB, will set to 1.67 GiB (RSS), difference: 4.38 MiB 2024.10.22 02:23:13.568996 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_898_898_0 to 202410_904_904_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:13.578923 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1539_1539_0 (state Temporary) 2024.10.22 02:23:13.656403 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1539_1539_0 to 202410_1552_1552_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:13.756533 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103093 2024.10.22 02:23:13.865891 [ 1229 ] {} DatabaseCatalog: Execution took 307 ms. 2024.10.22 02:23:13.919344 [ 1393 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_46_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:13.945743 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 99398 2024.10.22 02:23:13.971364 [ 1393 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_46_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_42_10, total 57 rows starting from the beginning of the part 2024.10.22 02:23:14.247536 [ 1254 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 02:23:14.264369 [ 1393 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_46_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_43_43_0, total 1 rows starting from the beginning of the part 2024.10.22 02:23:14.451955 [ 1393 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_46_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_44_44_0, total 1 rows starting from the beginning of the part 2024.10.22 02:23:14.653336 [ 1393 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_46_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_45_45_0, total 1 rows starting from the beginning of the part 2024.10.22 02:23:14.852362 [ 1393 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_46_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_46_46_0, total 1 rows starting from the beginning of the part 2024.10.22 02:23:15.613295 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 6 similar messages) 2024.10.22 02:23:15.667198 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1448_52, 202410_1449_1449_0, 202410_1450_1450_0, 202410_1451_1451_0, 202410_1452_1452_0, 202410_1453_1453_0] 2024.10.22 02:23:15.670680 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1448_52, 202410_1449_1449_0, 202410_1450_1450_0, 202410_1451_1451_0, 202410_1452_1452_0, 202410_1453_1453_0] 2024.10.22 02:23:15.715774 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.69 GiB, peak 2.00 GiB, free memory in arenas 16.37 MiB, will set to 1.68 GiB (RSS), difference: -16.75 MiB 2024.10.22 02:23:15.813702 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1448_52, 202410_1449_1449_0, 202410_1450_1450_0, 202410_1451_1451_0, 202410_1452_1452_0, 202410_1453_1453_0] 2024.10.22 02:23:16.276709 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 758 2024.10.22 02:23:16.426214 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 13982 2024.10.22 02:23:16.734640 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:16.744690 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:16.850326 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_914_914_0 (state Temporary) 2024.10.22 02:23:16.895583 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_914_914_0 to 202410_927_927_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:16.944237 [ 1226 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_922_287 to 202410_923_923_0 2024.10.22 02:23:16.951104 [ 1226 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:16.962131 [ 1226 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:17.017082 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:17.026715 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:17.058973 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13982 2024.10.22 02:23:17.130979 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_923_288} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_922_287 to 202410_923_923_0 into Compact with storage Full 2024.10.22 02:23:17.169518 [ 1393 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_46_11} MergeTask::MergeProjectionsStage: Merge sorted 61 rows, containing 27 columns (27 merged, 0 gathered) in 3.786172499 sec., 16.1112574812984 rows/sec., 39.96 KiB/sec. 2024.10.22 02:23:17.299908 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_923_288} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:17.318221 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_923_288} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_922_287, total 14001 rows starting from the beginning of the part 2024.10.22 02:23:17.321718 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_740_740_0 (state Temporary) 2024.10.22 02:23:17.358441 [ 1396 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_46_11} MergedBlockOutputStream: filled checksums 202410_1_46_11 (state Temporary) 2024.10.22 02:23:17.438832 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_923_288} MergeTreeSequentialSource: Reading 2 marks from part 202410_923_923_0, total 15 rows starting from the beginning of the part 2024.10.22 02:23:17.531747 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_740_740_0 to 202410_750_750_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:17.564138 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1326048 2024.10.22 02:23:17.765149 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 758 2024.10.22 02:23:17.767790 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 50 entries to flush up to offset 103143 2024.10.22 02:23:17.921078 [ 1387 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_46_11} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_merge_202410_1_46_11 to 202410_1_46_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:17.929351 [ 1387 ] {2d8f770f-485b-41a3-9b48-1e3c86e7559f::202410_1_46_11} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f) (MergerMutator): Merged 5 parts: [202410_1_42_10, 202410_46_46_0] -> 202410_1_46_11 2024.10.22 02:23:18.060519 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.24 MiB. 2024.10.22 02:23:18.061448 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:18.068268 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:18.258889 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1540_1540_0 (state Temporary) 2024.10.22 02:23:18.319701 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1540_1540_0 to 202410_1553_1553_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:18.386944 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103143 2024.10.22 02:23:18.569278 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:18.574939 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:18.680057 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_875_875_0 (state Temporary) 2024.10.22 02:23:18.711588 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_923_288} MergeTask::MergeProjectionsStage: Merge sorted 14016 rows, containing 12 columns (12 merged, 0 gathered) in 1.697210272 sec., 8258.257819453027 rows/sec., 1.29 MiB/sec. 2024.10.22 02:23:18.911437 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_875_875_0 to 202410_882_882_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:18.960917 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1326048 2024.10.22 02:23:19.006910 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_183_183_0 to 202410_186_186_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:19.099986 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_923_288} MergedBlockOutputStream: filled checksums 202410_1_923_288 (state Temporary) 2024.10.22 02:23:19.348139 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_863_232, 202410_864_864_0] 2024.10.22 02:23:19.349495 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_863_232, 202410_864_864_0] 2024.10.22 02:23:19.361643 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_863_232, 202410_864_864_0] 2024.10.22 02:23:19.821614 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_923_288} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_923_288 to 202410_1_923_288 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:19.831809 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_923_288} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_922_287, 202410_923_923_0] -> 202410_1_923_288 2024.10.22 02:23:19.964283 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:23:21.358866 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:23:21.359779 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:23:21.461154 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 159 entries to flush up to offset 99557 2024.10.22 02:23:21.547879 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 2.00 GiB, free memory in arenas 13.21 MiB, will set to 1.69 GiB (RSS), difference: 9.74 MiB 2024.10.22 02:23:22.095199 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:22.099636 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:23:22.389339 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_899_899_0 (state Temporary) 2024.10.22 02:23:22.411414 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 103173 2024.10.22 02:23:22.479632 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_899_899_0 to 202410_905_905_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:22.480432 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_874_163/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:23:22.551453 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 99557 2024.10.22 02:23:22.568203 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:22.570194 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:22.651327 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1541_1541_0 (state Temporary) 2024.10.22 02:23:22.737448 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1541_1541_0 to 202410_1554_1554_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:22.932653 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7386 2024.10.22 02:23:23.230615 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103173 2024.10.22 02:23:23.454842 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 10.71 MiB, will set to 1.68 GiB (RSS), difference: 74.61 MiB 2024.10.22 02:23:23.773004 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_842_19, 202410_843_843_0, 202410_844_844_0, 202410_845_845_0, 202410_846_846_0, 202410_847_847_0] 2024.10.22 02:23:23.774698 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_842_19, 202410_843_843_0, 202410_844_844_0, 202410_845_845_0, 202410_846_846_0, 202410_847_847_0] 2024.10.22 02:23:23.815635 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_842_19, 202410_843_843_0, 202410_844_844_0, 202410_845_845_0, 202410_846_846_0, 202410_847_847_0] 2024.10.22 02:23:24.031838 [ 1165 ] {} DatabaseCatalog: Execution took 220 ms. 2024.10.22 02:23:24.571520 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 13998 2024.10.22 02:23:24.699082 [ 1182 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:23:24.740012 [ 1182 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:23:24.834295 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:24.842109 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:24.881571 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_915_915_0 (state Temporary) 2024.10.22 02:23:25.037085 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_915_915_0 to 202410_928_928_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:25.066610 [ 1181 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_923_288 to 202410_924_924_0 2024.10.22 02:23:25.074126 [ 1181 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:25.080858 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:23:25.222783 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 13998 2024.10.22 02:23:25.249121 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_924_289} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_923_288 to 202410_924_924_0 into Compact with storage Full 2024.10.22 02:23:25.272130 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 759 2024.10.22 02:23:25.278455 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_924_289} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:25.288972 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_924_289} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_923_288, total 14016 rows starting from the beginning of the part 2024.10.22 02:23:25.312153 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_924_289} MergeTreeSequentialSource: Reading 2 marks from part 202410_924_924_0, total 13 rows starting from the beginning of the part 2024.10.22 02:23:25.427025 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/qsl/tibiwegtqryocqbvgxhglxqvbqnyi, upload id 2f07a0bf-b565-4ac0-a3a2-1cce40b173bb 2024.10.22 02:23:25.653190 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:25.659522 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_874_163} MergeTask::MergeProjectionsStage: Merge sorted 1326933 rows, containing 4 columns (4 merged, 0 gathered) in 40.176653316 sec., 33027.46472094928 rows/sec., 495.31 KiB/sec. 2024.10.22 02:23:25.661393 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:23:25.936687 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_741_741_0 (state Temporary) 2024.10.22 02:23:25.950803 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 10.10 MiB, will set to 1.69 GiB (RSS), difference: 154.70 MiB 2024.10.22 02:23:25.967589 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1327588 2024.10.22 02:23:25.979075 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_874_163} MergedBlockOutputStream: filled checksums 202410_1_874_163 (state Temporary) 2024.10.22 02:23:26.023113 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_741_741_0 to 202410_751_751_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:26.166054 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 759 2024.10.22 02:23:26.340137 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_924_289} MergeTask::MergeProjectionsStage: Merge sorted 14029 rows, containing 12 columns (12 merged, 0 gathered) in 1.10057981 sec., 12746.917463441385 rows/sec., 1.99 MiB/sec. 2024.10.22 02:23:26.446158 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_184_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:23:26.448802 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_821_821_0, 202410_822_822_0, 202410_823_823_0, 202410_824_824_0, 202410_825_825_0, 202410_826_826_0] 2024.10.22 02:23:26.456405 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_821_821_0, 202410_822_822_0, 202410_823_823_0, 202410_824_824_0, 202410_825_825_0, 202410_826_826_0] 2024.10.22 02:23:26.513092 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_924_289} MergedBlockOutputStream: filled checksums 202410_1_924_289 (state Temporary) 2024.10.22 02:23:26.522503 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_821_821_0, 202410_822_822_0, 202410_823_823_0, 202410_824_824_0, 202410_825_825_0, 202410_826_826_0] 2024.10.22 02:23:26.669283 [ 1688 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tuy/aqweuqzndzbgraiyusqhbaqqzgcdu, size 12752 2024.10.22 02:23:27.240155 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 103222 2024.10.22 02:23:27.325244 [ 1746 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kpy/vexkocplusvrntzylomimexppqryr, size 155 2024.10.22 02:23:27.358097 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_20_20_0 (state Temporary) 2024.10.22 02:23:27.374345 [ 1709 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wfc/drfvqxuqzqvzezjlfnoldbmjljlsl, size 344 2024.10.22 02:23:27.377040 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 02:23:27.381426 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bhj/bdjswkdslqasbgbvesxaeuploafkb (LogSeriesLimiter: on interval from 2024-10-22 02:23:10 to 2024-10-22 02:23:27 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:23:27.405122 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:27.410936 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:23:27.433328 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 02:23:27.461155 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 02:23:27.473759 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1542_1542_0 (state Temporary) 2024.10.22 02:23:27.488581 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:27.492387 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:23:27.500730 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 02:23:27.513683 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_924_289} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_924_289 to 202410_1_924_289 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:27.541779 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_924_289} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_923_288, 202410_924_924_0] -> 202410_1_924_289 2024.10.22 02:23:27.570709 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:23:27.513743 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_864_233, 202410_865_865_0] 2024.10.22 02:23:27.616772 [ 1728 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bhj/bdjswkdslqasbgbvesxaeuploafkb, size 4 2024.10.22 02:23:27.616052 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_864_233, 202410_865_865_0] 2024.10.22 02:23:27.646668 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1542_1542_0 to 202410_1555_1555_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:27.665212 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 02:23:27.674954 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 7.44 MiB, will set to 1.62 GiB (RSS), difference: 89.87 MiB 2024.10.22 02:23:27.727596 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_864_233, 202410_865_865_0] 2024.10.22 02:23:27.736464 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_876_876_0 (state Temporary) 2024.10.22 02:23:27.755296 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1550_97 to 202410_1551_1551_0 2024.10.22 02:23:27.762884 [ 1196 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.05 MiB using storage policy from min volume index 0 2024.10.22 02:23:27.767580 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 02:23:27.768304 [ 1196 ] {} DiskLocal: Reserved 1.05 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:23:27.799943 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_876_876_0 to 202410_883_883_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:27.820847 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1551_98} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1550_97 to 202410_1551_1551_0 into Compact with storage Full 2024.10.22 02:23:27.941023 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SocialSourceNetworkID 2024.10.22 02:23:27.974874 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1551_98} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:27.996128 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1551_98} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1550_97, total 23847 rows starting from the beginning of the part 2024.10.22 02:23:28.038949 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1551_98} MergeTreeSequentialSource: Reading 2 marks from part 202410_1551_1551_0, total 44 rows starting from the beginning of the part 2024.10.22 02:23:28.047737 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1327588 2024.10.22 02:23:28.088641 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103222 2024.10.22 02:23:29.092656 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_698_142, 202410_699_699_0, 202410_700_700_0, 202410_701_701_0, 202410_702_702_0, 202410_703_703_0] 2024.10.22 02:23:29.095387 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_698_142, 202410_699_699_0, 202410_700_700_0, 202410_701_701_0, 202410_702_702_0, 202410_703_703_0] 2024.10.22 02:23:29.182800 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_874_163} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_874_163 to 202410_1_874_163 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:29.188091 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_874_163} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_868_162, 202410_869_874_1] -> 202410_1_874_163 2024.10.22 02:23:29.240828 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:23:29.340367 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_698_142, 202410_699_699_0, 202410_700_700_0, 202410_701_701_0, 202410_702_702_0, 202410_703_703_0] 2024.10.22 02:23:29.416171 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 17.85 MiB, will set to 1.61 GiB (RSS), difference: -4.47 MiB 2024.10.22 02:23:29.760082 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1551_98} MergeTask::MergeProjectionsStage: Merge sorted 23891 rows, containing 13 columns (13 merged, 0 gathered) in 1.949271822 sec., 12256.371702683957 rows/sec., 2.22 MiB/sec. 2024.10.22 02:23:29.865780 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1551_98} MergedBlockOutputStream: filled checksums 202410_1207_1551_98 (state Temporary) 2024.10.22 02:23:30.107273 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 174 entries to flush up to offset 99731 2024.10.22 02:23:30.288702 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1551_98} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1551_98 to 202410_1207_1551_98 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:30.293095 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1551_98} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1550_97, 202410_1551_1551_0] -> 202410_1207_1551_98 2024.10.22 02:23:30.439150 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.05 MiB. 2024.10.22 02:23:30.584631 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 7429 2024.10.22 02:23:30.714896 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:30.716877 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:23:30.770055 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_900_900_0 (state Temporary) 2024.10.22 02:23:30.856172 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_900_900_0 to 202410_906_906_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:31.021533 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 99731 2024.10.22 02:23:31.291277 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:23:31.360174 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:23:31.368212 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:23:31.510327 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 15.75 MiB, will set to 1.61 GiB (RSS), difference: 6.05 MiB 2024.10.22 02:23:32.094917 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 103282 2024.10.22 02:23:32.349100 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:32.354334 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:23:32.444290 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1543_1543_0 (state Temporary) 2024.10.22 02:23:32.726192 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 14012 2024.10.22 02:23:32.754369 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1543_1543_0 to 202410_1556_1556_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:32.778744 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1551_98 to 202410_1552_1552_0 2024.10.22 02:23:32.800261 [ 1239 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.05 MiB using storage policy from min volume index 0 2024.10.22 02:23:32.809298 [ 1239 ] {} DiskLocal: Reserved 1.05 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:23:32.841278 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103282 2024.10.22 02:23:32.855326 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1552_99} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1551_98 to 202410_1552_1552_0 into Compact with storage Full 2024.10.22 02:23:32.882208 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:32.895250 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:23:33.046286 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_916_916_0 (state Temporary) 2024.10.22 02:23:33.115349 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1552_99} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:33.137621 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1552_99} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1551_98, total 23891 rows starting from the beginning of the part 2024.10.22 02:23:33.155157 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1552_99} MergeTreeSequentialSource: Reading 2 marks from part 202410_1552_1552_0, total 61 rows starting from the beginning of the part 2024.10.22 02:23:33.320015 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_916_916_0 to 202410_929_929_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:33.420782 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14012 2024.10.22 02:23:33.474688 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 13.53 MiB, will set to 1.61 GiB (RSS), difference: -8.00 MiB 2024.10.22 02:23:33.565116 [ 1229 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_924_289 to 202410_925_925_0 2024.10.22 02:23:33.572886 [ 1229 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:33.577264 [ 1229 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:23:33.601455 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Referer 2024.10.22 02:23:33.606383 [ 1229 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 267 ms. 2024.10.22 02:23:33.651523 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_925_290} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_924_289 to 202410_925_925_0 into Compact with storage Full 2024.10.22 02:23:33.659701 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Referer 2024.10.22 02:23:33.667596 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Referer 2024.10.22 02:23:33.667962 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 760 2024.10.22 02:23:33.694494 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Referer 2024.10.22 02:23:33.714385 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Referer 2024.10.22 02:23:33.751507 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Referer 2024.10.22 02:23:33.769516 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Referer 2024.10.22 02:23:33.770371 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_925_290} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:33.798896 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Referer 2024.10.22 02:23:33.807933 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_925_290} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_924_289, total 14029 rows starting from the beginning of the part 2024.10.22 02:23:33.853469 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Referer 2024.10.22 02:23:34.091810 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_925_290} MergeTreeSequentialSource: Reading 2 marks from part 202410_925_925_0, total 26 rows starting from the beginning of the part 2024.10.22 02:23:34.186601 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/cnl/muixpjekxuetvbcxohaswoycebitk (LogSeriesLimiter: on interval from 2024-10-22 02:23:27 to 2024-10-22 02:23:34 accepted series 1 / 12 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:23:34.319534 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:34.323751 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:34.397235 [ 1216 ] {} DatabaseCatalog: Execution took 374 ms. 2024.10.22 02:23:34.497353 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_742_742_0 (state Temporary) 2024.10.22 02:23:34.763875 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1552_99/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:23:34.794159 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_742_742_0 to 202410_752_752_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:35.056125 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1329513 2024.10.22 02:23:35.104244 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 760 2024.10.22 02:23:35.289058 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_925_290} MergeTask::MergeProjectionsStage: Merge sorted 14055 rows, containing 12 columns (12 merged, 0 gathered) in 1.609069589 sec., 8734.861497652728 rows/sec., 1.36 MiB/sec. 2024.10.22 02:23:35.295687 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1552_99} MergeTask::MergeProjectionsStage: Merge sorted 23952 rows, containing 13 columns (13 merged, 0 gathered) in 2.447508277 sec., 9786.279468422816 rows/sec., 1.77 MiB/sec. 2024.10.22 02:23:35.392100 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_865_234, 202410_866_866_0] 2024.10.22 02:23:35.395022 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_865_234, 202410_866_866_0] 2024.10.22 02:23:35.407356 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_865_234, 202410_866_866_0] 2024.10.22 02:23:35.525611 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_925_290} MergedBlockOutputStream: filled checksums 202410_1_925_290 (state Temporary) 2024.10.22 02:23:35.527134 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1552_99} MergedBlockOutputStream: filled checksums 202410_1207_1552_99 (state Temporary) 2024.10.22 02:23:35.555535 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 12.96 MiB, will set to 1.62 GiB (RSS), difference: -10.93 MiB 2024.10.22 02:23:35.708087 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1552_99} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1552_99 to 202410_1207_1552_99 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:35.712488 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1552_99} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1551_98, 202410_1552_1552_0] -> 202410_1207_1552_99 2024.10.22 02:23:35.758816 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:23:35.867787 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_925_290} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_925_290 to 202410_1_925_290 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:35.871095 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_925_290} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_924_289, 202410_925_925_0] -> 202410_1_925_290 2024.10.22 02:23:35.917340 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:23:36.031684 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:36.042610 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:36.043068 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/fyb/cjntoihnssokzazgleoszuxcyzooi. 2024.10.22 02:23:36.283431 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_877_877_0 (state Temporary) 2024.10.22 02:23:36.343125 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_877_877_0 to 202410_884_884_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:36.363799 [ 1252 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_874_163 to 202410_875_880_1 2024.10.22 02:23:36.366268 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1329513 2024.10.22 02:23:36.384672 [ 1252 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.04 MiB using storage policy from min volume index 0 2024.10.22 02:23:36.392613 [ 1252 ] {} DiskLocal: Reserved 1.04 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:36.420863 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_880_164} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_874_163 to 202410_875_880_1 into Wide with storage Full 2024.10.22 02:23:36.540061 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_880_164} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:36.544297 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_880_164} MergeTreeSequentialSource: Reading 163 marks from part 202410_1_874_163, total 1326933 rows starting from the beginning of the part 2024.10.22 02:23:36.577017 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_880_164} MergeTreeSequentialSource: Reading 3 marks from part 202410_875_880_1, total 9557 rows starting from the beginning of the part 2024.10.22 02:23:36.853422 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 103356 2024.10.22 02:23:37.376579 [ 1262 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 02:23:37.403002 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:37.407802 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:37.595096 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 18.53 MiB, will set to 1.63 GiB (RSS), difference: 17.97 MiB 2024.10.22 02:23:37.760471 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1544_1544_0 (state Temporary) 2024.10.22 02:23:37.844802 [ 1259 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 261 ms. 2024.10.22 02:23:38.019485 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1544_1544_0 to 202410_1557_1557_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:38.056904 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1552_99 to 202410_1553_1553_0 2024.10.22 02:23:38.080484 [ 1241 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.05 MiB using storage policy from min volume index 0 2024.10.22 02:23:38.088898 [ 1241 ] {} DiskLocal: Reserved 1.05 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:38.093126 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103356 2024.10.22 02:23:38.143909 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1553_100} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1552_99 to 202410_1553_1553_0 into Compact with storage Full 2024.10.22 02:23:38.235668 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1553_100} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:38.360205 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1553_100} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1552_99, total 23952 rows starting from the beginning of the part 2024.10.22 02:23:38.447056 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1553_100} MergeTreeSequentialSource: Reading 2 marks from part 202410_1553_1553_0, total 50 rows starting from the beginning of the part 2024.10.22 02:23:38.456476 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_880_164/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:23:38.557297 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 123 entries to flush up to offset 99854 2024.10.22 02:23:38.746552 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(a5938b87e2635c315706a9675a94615c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a5938b87e2635c315706a9675a94615c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 02:23:38.775064 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: a5938b87e2635c315706a9675a94615c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 02:23:38.881663 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:23:38.932576 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:38.936942 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:38.977131 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_901_901_0 (state Temporary) 2024.10.22 02:23:39.264691 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_901_901_0 to 202410_907_907_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:39.300188 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_902_31 to 202410_907_907_0 2024.10.22 02:23:39.325292 [ 1224 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:39.339379 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:39.341056 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 99854 2024.10.22 02:23:39.371579 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_902_31 to 202410_907_907_0 into Compact with storage Full 2024.10.22 02:23:39.525766 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:39.747256 [ 1222 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:23:39.752195 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_902_31, total 15766 rows starting from the beginning of the part 2024.10.22 02:23:39.796882 [ 1222 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:23:39.799781 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_903_903_0, total 163 rows starting from the beginning of the part 2024.10.22 02:23:39.848538 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 25.80 MiB, will set to 1.63 GiB (RSS), difference: -10.67 MiB 2024.10.22 02:23:39.868571 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_904_904_0, total 114 rows starting from the beginning of the part 2024.10.22 02:23:39.997122 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_905_905_0, total 159 rows starting from the beginning of the part 2024.10.22 02:23:40.067617 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_906_906_0, total 174 rows starting from the beginning of the part 2024.10.22 02:23:40.124043 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1553_100} MergeTask::MergeProjectionsStage: Merge sorted 24002 rows, containing 13 columns (13 merged, 0 gathered) in 2.000820092 sec., 11996.081054947743 rows/sec., 2.17 MiB/sec. 2024.10.22 02:23:40.141345 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} MergeTreeSequentialSource: Reading 2 marks from part 202410_907_907_0, total 123 rows starting from the beginning of the part 2024.10.22 02:23:40.332677 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1553_100} MergedBlockOutputStream: filled checksums 202410_1207_1553_100 (state Temporary) 2024.10.22 02:23:40.926732 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 14024 2024.10.22 02:23:40.934179 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1553_100} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1553_100 to 202410_1207_1553_100 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:40.935386 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1207_1453_53, 202410_1454_1454_0, 202410_1455_1455_0, 202410_1456_1456_0] 2024.10.22 02:23:40.938664 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1207_1453_53, 202410_1454_1454_0, 202410_1455_1455_0, 202410_1456_1456_0] 2024.10.22 02:23:41.042474 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1553_100} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1552_99, 202410_1553_1553_0] -> 202410_1207_1553_100 2024.10.22 02:23:41.042859 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1207_1453_53, 202410_1454_1454_0, 202410_1455_1455_0, 202410_1456_1456_0] 2024.10.22 02:23:41.067236 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:23:41.130186 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:41.136560 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:41.268228 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_917_917_0 (state Temporary) 2024.10.22 02:23:41.359752 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:23:41.361457 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:23:41.385699 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_917_917_0 to 202410_930_930_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:41.416816 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_184_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:23:41.437022 [ 1204 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_925_290 to 202410_926_926_0 2024.10.22 02:23:41.450548 [ 1204 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:41.524265 [ 1204 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:41.543014 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 19.54 MiB, will set to 1.65 GiB (RSS), difference: 4.39 MiB 2024.10.22 02:23:41.556549 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14024 2024.10.22 02:23:41.558861 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_926_291} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_925_290 to 202410_926_926_0 into Compact with storage Full 2024.10.22 02:23:41.645025 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_926_291} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:41.677772 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_926_291} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_925_290, total 14055 rows starting from the beginning of the part 2024.10.22 02:23:41.721107 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_926_291} MergeTreeSequentialSource: Reading 2 marks from part 202410_926_926_0, total 24 rows starting from the beginning of the part 2024.10.22 02:23:42.109527 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 65 entries to flush up to offset 103421 2024.10.22 02:23:42.427752 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} MergeTask::MergeProjectionsStage: Merge sorted 16499 rows, containing 13 columns (13 merged, 0 gathered) in 3.060978416 sec., 5390.106612238196 rows/sec., 2.34 MiB/sec. 2024.10.22 02:23:42.459734 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:42.463728 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:42.486125 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} MergedBlockOutputStream: filled checksums 202410_747_907_32 (state Temporary) 2024.10.22 02:23:42.521469 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1545_1545_0 (state Temporary) 2024.10.22 02:23:42.607308 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 761 2024.10.22 02:23:42.699103 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_907_32 to 202410_747_907_32 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:42.709229 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_907_32} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_902_31, 202410_907_907_0] -> 202410_747_907_32 2024.10.22 02:23:42.759728 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.29 MiB. 2024.10.22 02:23:42.840677 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_866_235, 202410_867_867_0] 2024.10.22 02:23:42.842075 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_866_235, 202410_867_867_0] 2024.10.22 02:23:42.851213 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_866_235, 202410_867_867_0] 2024.10.22 02:23:42.941093 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1545_1545_0 to 202410_1558_1558_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:43.059538 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1553_100 to 202410_1554_1554_0 2024.10.22 02:23:43.068171 [ 1240 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.05 MiB using storage policy from min volume index 0 2024.10.22 02:23:43.071627 [ 1240 ] {} DiskLocal: Reserved 1.05 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:43.076091 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:43.086655 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:43.095643 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103421 2024.10.22 02:23:43.110666 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1554_101} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1553_100 to 202410_1554_1554_0 into Compact with storage Full 2024.10.22 02:23:43.152300 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_926_291} MergeTask::MergeProjectionsStage: Merge sorted 14079 rows, containing 12 columns (12 merged, 0 gathered) in 1.603242954 sec., 8781.576095421904 rows/sec., 1.37 MiB/sec. 2024.10.22 02:23:43.165557 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1554_101} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:43.217246 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_926_291} MergedBlockOutputStream: filled checksums 202410_1_926_291 (state Temporary) 2024.10.22 02:23:43.358487 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_743_743_0 (state Temporary) 2024.10.22 02:23:43.371297 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1331053 2024.10.22 02:23:43.439838 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1554_101} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1553_100, total 24002 rows starting from the beginning of the part 2024.10.22 02:23:43.441990 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_926_291} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_926_291 to 202410_1_926_291 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:43.448197 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_926_291} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_925_290, 202410_926_926_0] -> 202410_1_926_291 2024.10.22 02:23:43.498371 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1554_101} MergeTreeSequentialSource: Reading 2 marks from part 202410_1554_1554_0, total 30 rows starting from the beginning of the part 2024.10.22 02:23:43.645307 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_743_743_0 to 202410_753_753_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:43.521699 [ 1399 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {5276570e-a364-404e-a403-3e186cba01a8::202410_1_926_291}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 02:23:43.698611 [ 1179 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_748_152 to 202410_753_753_0 2024.10.22 02:23:43.713500 [ 1179 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:43.719627 [ 1179 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:43.737211 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:23:43.794375 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 19.16 MiB, will set to 1.63 GiB (RSS), difference: 24.68 MiB 2024.10.22 02:23:43.849456 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 761 2024.10.22 02:23:43.864157 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_748_152 to 202410_753_753_0 into Compact with storage Full 2024.10.22 02:23:44.177194 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:44.214837 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_748_152, total 792 rows starting from the beginning of the part 2024.10.22 02:23:44.347304 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} MergeTreeSequentialSource: Reading 2 marks from part 202410_749_749_0, total 1 rows starting from the beginning of the part 2024.10.22 02:23:44.447799 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:44.453466 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:44.454762 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} MergeTreeSequentialSource: Reading 2 marks from part 202410_750_750_0, total 1 rows starting from the beginning of the part 2024.10.22 02:23:44.552348 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} MergeTreeSequentialSource: Reading 2 marks from part 202410_751_751_0, total 1 rows starting from the beginning of the part 2024.10.22 02:23:44.653776 [ 1158 ] {} DatabaseCatalog: Execution took 260 ms. 2024.10.22 02:23:44.835663 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_878_878_0 (state Temporary) 2024.10.22 02:23:44.894219 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} MergeTreeSequentialSource: Reading 2 marks from part 202410_752_752_0, total 1 rows starting from the beginning of the part 2024.10.22 02:23:45.015636 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} MergeTreeSequentialSource: Reading 2 marks from part 202410_753_753_0, total 1 rows starting from the beginning of the part 2024.10.22 02:23:45.120323 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_878_878_0 to 202410_885_885_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:45.243768 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1554_101} MergeTask::MergeProjectionsStage: Merge sorted 24032 rows, containing 13 columns (13 merged, 0 gathered) in 2.146194774 sec., 11197.492553394877 rows/sec., 2.03 MiB/sec. 2024.10.22 02:23:45.271839 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1331053 2024.10.22 02:23:45.313268 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1554_101} MergedBlockOutputStream: filled checksums 202410_1207_1554_101 (state Temporary) 2024.10.22 02:23:45.340583 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_814_153, 202410_815_820_1] 2024.10.22 02:23:45.344291 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_814_153, 202410_815_820_1] 2024.10.22 02:23:45.357818 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_814_153, 202410_815_820_1] 2024.10.22 02:23:45.499215 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 16.89 MiB, will set to 1.62 GiB (RSS), difference: -27.73 MiB 2024.10.22 02:23:45.663578 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1554_101} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1554_101 to 202410_1207_1554_101 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:45.674795 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1554_101} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1553_100, 202410_1554_1554_0] -> 202410_1207_1554_101 2024.10.22 02:23:45.707331 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.05 MiB. 2024.10.22 02:23:45.910965 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 02:23:45.926186 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 02:23:45.941149 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 02:23:45.968787 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 02:23:46.085403 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 02:23:46.147024 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 02:23:46.175979 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column SocialSourcePage 2024.10.22 02:23:46.358429 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/sjb/lwuuvhlrdwkexojtfhtxpavplzfux (LogSeriesLimiter: on interval from 2024-10-22 02:23:34 to 2024-10-22 02:23:46 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:23:46.856769 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 190 entries to flush up to offset 100044 2024.10.22 02:23:47.101461 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 103491 2024.10.22 02:23:47.567932 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:47.569388 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:47.580926 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:47.588977 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:47.683902 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1546_1546_0 (state Temporary) 2024.10.22 02:23:47.710622 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} MergeTask::MergeProjectionsStage: Merge sorted 797 rows, containing 32 columns (32 merged, 0 gathered) in 3.960199457 sec., 201.2524895914605 rows/sec., 15.20 KiB/sec. 2024.10.22 02:23:47.758546 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_902_902_0 (state Temporary) 2024.10.22 02:23:47.763203 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1546_1546_0 to 202410_1559_1559_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:47.814295 [ 1220 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1554_101 to 202410_1555_1555_0 2024.10.22 02:23:47.827412 [ 1220 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 02:23:47.831772 [ 1220 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:47.831766 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_902_902_0 to 202410_908_908_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:47.860944 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} MergedBlockOutputStream: filled checksums 202410_1_753_153 (state Temporary) 2024.10.22 02:23:47.864822 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1555_102} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1554_101 to 202410_1555_1555_0 into Compact with storage Full 2024.10.22 02:23:47.942769 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103491 2024.10.22 02:23:47.996362 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 100044 2024.10.22 02:23:47.998177 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1555_102} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:48.012669 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1555_102} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1554_101, total 24032 rows starting from the beginning of the part 2024.10.22 02:23:48.034055 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1555_102} MergeTreeSequentialSource: Reading 2 marks from part 202410_1555_1555_0, total 49 rows starting from the beginning of the part 2024.10.22 02:23:48.084622 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_753_153 to 202410_1_753_153 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:48.089630 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_753_153} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_748_152, 202410_753_753_0] -> 202410_1_753_153 2024.10.22 02:23:48.153112 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:23:49.058626 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 29 entries to flush up to offset 14053 2024.10.22 02:23:49.404321 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:49.406777 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:49.505798 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 10.98 MiB, will set to 1.62 GiB (RSS), difference: -15.56 MiB 2024.10.22 02:23:49.592650 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_918_918_0 (state Temporary) 2024.10.22 02:23:49.598268 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1555_102} MergeTask::MergeProjectionsStage: Merge sorted 24081 rows, containing 13 columns (13 merged, 0 gathered) in 1.741703812 sec., 13826.1166072478 rows/sec., 2.50 MiB/sec. 2024.10.22 02:23:49.599172 [ 1211 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 272 ms. 2024.10.22 02:23:49.618461 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_880_164/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:23:49.854857 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_918_918_0 to 202410_931_931_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:49.911758 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1555_102} MergedBlockOutputStream: filled checksums 202410_1207_1555_102 (state Temporary) 2024.10.22 02:23:49.912965 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_926_291 to 202410_927_927_0 2024.10.22 02:23:49.920690 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:49.931513 [ 1228 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:50.043023 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14053 2024.10.22 02:23:50.048614 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_927_292} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_926_291 to 202410_927_927_0 into Compact with storage Full 2024.10.22 02:23:50.117438 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_927_292} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:50.300092 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1555_102} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1555_102 to 202410_1207_1555_102 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:50.316580 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1555_102} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1554_101, 202410_1555_1555_0] -> 202410_1207_1555_102 2024.10.22 02:23:50.336706 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_927_292} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_926_291, total 14079 rows starting from the beginning of the part 2024.10.22 02:23:50.358682 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:23:50.380177 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_927_292} MergeTreeSequentialSource: Reading 2 marks from part 202410_927_927_0, total 9 rows starting from the beginning of the part 2024.10.22 02:23:50.652230 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:50.655057 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:51.171690 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_927_292} MergeTask::MergeProjectionsStage: Merge sorted 14088 rows, containing 12 columns (12 merged, 0 gathered) in 1.126400225 sec., 12507.099774416327 rows/sec., 1.95 MiB/sec. 2024.10.22 02:23:51.219738 [ 1245 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 02:23:51.360674 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:23:51.361244 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_927_292} MergedBlockOutputStream: filled checksums 202410_1_927_292 (state Temporary) 2024.10.22 02:23:51.362739 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:23:51.554115 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_927_292} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_927_292 to 202410_1_927_292 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:51.558505 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 11.45 MiB, will set to 1.62 GiB (RSS), difference: 13.03 MiB 2024.10.22 02:23:51.560044 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_927_292} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_926_291, 202410_927_927_0] -> 202410_1_927_292 2024.10.22 02:23:51.592838 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:23:51.960170 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 103546 2024.10.22 02:23:52.034571 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_867_236, 202410_868_868_0] 2024.10.22 02:23:52.037975 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_867_236, 202410_868_868_0] 2024.10.22 02:23:52.047899 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_867_236, 202410_868_868_0] 2024.10.22 02:23:52.213447 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:52.219172 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:52.277132 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1702 entries to flush up to offset 1332755 2024.10.22 02:23:52.508448 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1547_1547_0 (state Temporary) 2024.10.22 02:23:52.709184 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1547_1547_0 to 202410_1560_1560_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:52.761036 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1555_102 to 202410_1556_1556_0 2024.10.22 02:23:52.765637 [ 1254 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 02:23:52.771898 [ 1254 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:52.807882 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1556_103} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1555_102 to 202410_1556_1556_0 into Compact with storage Full 2024.10.22 02:23:52.867876 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1556_103} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:52.902999 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1556_103} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1555_102, total 24081 rows starting from the beginning of the part 2024.10.22 02:23:52.954855 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1556_103} MergeTreeSequentialSource: Reading 2 marks from part 202410_1556_1556_0, total 60 rows starting from the beginning of the part 2024.10.22 02:23:53.064512 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:53.069571 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:53.095786 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103546 2024.10.22 02:23:53.243065 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_879_879_0 (state Temporary) 2024.10.22 02:23:53.309695 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_879_879_0 to 202410_886_886_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:53.346137 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_881_881_0 to 202410_886_886_0 2024.10.22 02:23:53.359298 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:53.379451 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:53.542853 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_881_881_0 to 202410_886_886_0 into Compact with storage Full 2024.10.22 02:23:53.551801 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1332755 2024.10.22 02:23:53.566845 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:53.575482 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_881_881_0, total 1608 rows starting from the beginning of the part 2024.10.22 02:23:53.636129 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_882_882_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:23:53.656724 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_883_883_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:23:53.671188 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_884_884_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:23:53.679206 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 17.05 MiB, will set to 1.62 GiB (RSS), difference: -4.09 MiB 2024.10.22 02:23:53.689355 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_885_885_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:23:53.712800 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_886_886_0, total 1702 rows starting from the beginning of the part 2024.10.22 02:23:54.805692 [ 1260 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:23:54.820072 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} MergeTask::MergeProjectionsStage: Merge sorted 9855 rows, containing 4 columns (4 merged, 0 gathered) in 1.399544658 sec., 7041.575946624777 rows/sec., 110.02 KiB/sec. 2024.10.22 02:23:54.855887 [ 1260 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:23:54.874054 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} MergedBlockOutputStream: filled checksums 202410_881_886_1 (state Temporary) 2024.10.22 02:23:54.898909 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_184_184_0 (state Temporary) 2024.10.22 02:23:55.039077 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_881_886_1 to 202410_881_886_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:55.046242 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_881_881_0, 202410_886_886_0] -> 202410_881_886_1 2024.10.22 02:23:55.154270 [ 1393 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_881_886_1}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 02:23:55.220106 [ 1266 ] {} DatabaseCatalog: Execution took 559 ms. 2024.10.22 02:23:55.246753 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:23:55.388180 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:23:55.656626 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 152 entries to flush up to offset 100196 2024.10.22 02:23:55.669426 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1556_103} MergeTask::MergeProjectionsStage: Merge sorted 24141 rows, containing 13 columns (13 merged, 0 gathered) in 2.878643029 sec., 8386.243016865568 rows/sec., 1.52 MiB/sec. 2024.10.22 02:23:55.763144 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 20.82 MiB, will set to 1.64 GiB (RSS), difference: 5.41 MiB 2024.10.22 02:23:55.862577 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:55.864596 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1556_103} MergedBlockOutputStream: filled checksums 202410_1207_1556_103 (state Temporary) 2024.10.22 02:23:55.866400 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:55.998973 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1556_103} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1556_103 to 202410_1207_1556_103 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:56.063404 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_903_903_0 (state Temporary) 2024.10.22 02:23:56.112158 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1556_103} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1555_102, 202410_1556_1556_0] -> 202410_1207_1556_103 2024.10.22 02:23:56.221532 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_903_903_0 to 202410_909_909_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:56.246923 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:23:56.368978 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 100196 2024.10.22 02:23:56.784488 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 5 old parts to remove. Parts: [202410_1207_1456_54, 202410_1457_1457_0, 202410_1458_1458_0, 202410_1459_1459_0, 202410_1460_1460_0] 2024.10.22 02:23:56.791389 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from filesystem (serially): Parts: [202410_1207_1456_54, 202410_1457_1457_0, 202410_1458_1458_0, 202410_1459_1459_0, 202410_1460_1460_0] 2024.10.22 02:23:56.828184 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 5 parts from memory: Parts: [202410_1207_1456_54, 202410_1457_1457_0, 202410_1458_1458_0, 202410_1459_1459_0, 202410_1460_1460_0] 2024.10.22 02:23:57.105415 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 103606 2024.10.22 02:23:57.472694 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:57.477425 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:57.532933 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1548_1548_0 (state Temporary) 2024.10.22 02:23:57.544709 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 14073 2024.10.22 02:23:57.571306 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1548_1548_0 to 202410_1561_1561_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:57.650173 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103606 2024.10.22 02:23:57.718475 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1556_103 to 202410_1557_1557_0 2024.10.22 02:23:57.746521 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 17.60 MiB, will set to 1.64 GiB (RSS), difference: -12.89 MiB 2024.10.22 02:23:57.750520 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 02:23:57.751992 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:57.754006 [ 1218 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:57.756280 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:57.907535 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1557_104} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1556_103 to 202410_1557_1557_0 into Compact with storage Full 2024.10.22 02:23:57.914353 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_919_919_0 (state Temporary) 2024.10.22 02:23:57.993022 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_919_919_0 to 202410_932_932_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:58.036830 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_927_292 to 202410_928_928_0 2024.10.22 02:23:58.067948 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:58.069775 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1557_104} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:58.070913 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:58.091317 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1557_104} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1556_103, total 24141 rows starting from the beginning of the part 2024.10.22 02:23:58.142493 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_928_293} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_927_292 to 202410_928_928_0 into Compact with storage Full 2024.10.22 02:23:58.153562 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14073 2024.10.22 02:23:58.207518 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_928_293} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:23:58.225771 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_928_293} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_927_292, total 14088 rows starting from the beginning of the part 2024.10.22 02:23:58.239080 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1557_104} MergeTreeSequentialSource: Reading 2 marks from part 202410_1557_1557_0, total 74 rows starting from the beginning of the part 2024.10.22 02:23:58.357186 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_928_293} MergeTreeSequentialSource: Reading 2 marks from part 202410_928_928_0, total 16 rows starting from the beginning of the part 2024.10.22 02:23:58.852292 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 762 2024.10.22 02:23:59.026171 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:23:59.041802 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:23:59.111434 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_744_744_0 (state Temporary) 2024.10.22 02:23:59.202202 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_744_744_0 to 202410_754_754_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:59.311618 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 762 2024.10.22 02:23:59.373207 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_928_293} MergeTask::MergeProjectionsStage: Merge sorted 14104 rows, containing 12 columns (12 merged, 0 gathered) in 1.291115914 sec., 10923.88363203151 rows/sec., 1.71 MiB/sec. 2024.10.22 02:23:59.429372 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_928_293} MergedBlockOutputStream: filled checksums 202410_1_928_293 (state Temporary) 2024.10.22 02:23:59.558133 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 11.96 MiB, will set to 1.64 GiB (RSS), difference: 2.64 MiB 2024.10.22 02:23:59.751973 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1557_104} MergeTask::MergeProjectionsStage: Merge sorted 24215 rows, containing 13 columns (13 merged, 0 gathered) in 1.855715626 sec., 13048.874332214089 rows/sec., 2.36 MiB/sec. 2024.10.22 02:23:59.761673 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_928_293} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_928_293 to 202410_1_928_293 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:23:59.768226 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_928_293} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_927_292, 202410_928_928_0] -> 202410_1_928_293 2024.10.22 02:23:59.841199 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:23:59.930876 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1557_104} MergedBlockOutputStream: filled checksums 202410_1207_1557_104 (state Temporary) 2024.10.22 02:24:00.014027 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_880_164/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:24:00.139711 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1557_104} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1557_104 to 202410_1207_1557_104 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:00.143102 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1557_104} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1556_103, 202410_1557_1557_0] -> 202410_1207_1557_104 2024.10.22 02:24:00.163205 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:24:00.243069 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_868_237, 202410_869_869_0] 2024.10.22 02:24:00.245315 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_868_237, 202410_869_869_0] 2024.10.22 02:24:00.261073 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_868_237, 202410_869_869_0] 2024.10.22 02:24:00.556592 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1763 entries to flush up to offset 1334518 2024.10.22 02:24:01.362945 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:24:01.365281 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:24:01.370448 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:01.373085 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:24:01.441532 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 10.84 MiB, will set to 1.64 GiB (RSS), difference: 41.11 MiB 2024.10.22 02:24:01.663278 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 103660 2024.10.22 02:24:01.739039 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/fyb/cjntoihnssokzazgleoszuxcyzooi. 2024.10.22 02:24:01.762551 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_880_880_0 (state Temporary) 2024.10.22 02:24:01.900866 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:01.919686 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:24:02.068257 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_880_880_0 to 202410_887_887_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:02.088433 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1549_1549_0 (state Temporary) 2024.10.22 02:24:02.152457 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1549_1549_0 to 202410_1562_1562_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:02.217558 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103660 2024.10.22 02:24:02.235484 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_184_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:24:02.309190 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1334518 2024.10.22 02:24:02.339045 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1557_104 to 202410_1558_1558_0 2024.10.22 02:24:02.344329 [ 1170 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 02:24:02.347770 [ 1170 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:24:02.362605 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1558_105} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1557_104 to 202410_1558_1558_0 into Compact with storage Full 2024.10.22 02:24:02.380053 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1558_105} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:02.395257 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1558_105} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1557_104, total 24215 rows starting from the beginning of the part 2024.10.22 02:24:02.513840 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 11.21 MiB, will set to 1.63 GiB (RSS), difference: 32.68 MiB 2024.10.22 02:24:02.527067 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1558_105} MergeTreeSequentialSource: Reading 2 marks from part 202410_1558_1558_0, total 65 rows starting from the beginning of the part 2024.10.22 02:24:03.906359 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 132 entries to flush up to offset 100328 2024.10.22 02:24:04.057000 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:04.059822 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:24:04.224026 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_184_184_0 to 202410_187_187_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:04.227217 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_904_904_0 (state Temporary) 2024.10.22 02:24:04.379604 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_904_904_0 to 202410_910_910_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:04.485480 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 100328 2024.10.22 02:24:04.580110 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1558_105} MergeTask::MergeProjectionsStage: Merge sorted 24280 rows, containing 13 columns (13 merged, 0 gathered) in 2.223781968 sec., 10918.336576780805 rows/sec., 1.98 MiB/sec. 2024.10.22 02:24:04.671664 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1558_105} MergedBlockOutputStream: filled checksums 202410_1207_1558_105 (state Temporary) 2024.10.22 02:24:04.756058 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 8.97 MiB, will set to 1.64 GiB (RSS), difference: 73.50 MiB 2024.10.22 02:24:05.013190 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1558_105} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1558_105 to 202410_1207_1558_105 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:05.120071 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1558_105} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1557_104, 202410_1558_1558_0] -> 202410_1207_1558_105 2024.10.22 02:24:05.331278 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Multipart upload has created. Details: bucket test, key common/cnl/muixpjekxuetvbcxohaswoycebitk, upload id 38630a0d-e9c7-48ef-891f-64cfc695e544 2024.10.22 02:24:05.428166 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:24:05.576633 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_847_20, 202410_848_848_0, 202410_849_849_0, 202410_850_850_0, 202410_851_851_0, 202410_852_852_0] 2024.10.22 02:24:05.579844 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_847_20, 202410_848_848_0, 202410_849_849_0, 202410_850_850_0, 202410_851_851_0, 202410_852_852_0] 2024.10.22 02:24:05.641208 [ 1155 ] {} DatabaseCatalog: Execution took 424 ms. 2024.10.22 02:24:05.655074 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 14089 2024.10.22 02:24:05.746973 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_847_20, 202410_848_848_0, 202410_849_849_0, 202410_850_850_0, 202410_851_851_0, 202410_852_852_0] 2024.10.22 02:24:05.823084 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:05.830505 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.12 GiB. 2024.10.22 02:24:05.899954 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_920_920_0 (state Temporary) 2024.10.22 02:24:05.971246 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_920_920_0 to 202410_933_933_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:06.050519 [ 1192 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_928_293 to 202410_929_929_0 2024.10.22 02:24:06.063830 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14089 2024.10.22 02:24:06.074203 [ 1192 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:06.082414 [ 1192 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:24:06.149371 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_929_294} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_928_293 to 202410_929_929_0 into Compact with storage Full 2024.10.22 02:24:06.251848 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 47 entries to flush up to offset 103707 2024.10.22 02:24:06.293229 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_929_294} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:06.309451 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_929_294} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_928_293, total 14104 rows starting from the beginning of the part 2024.10.22 02:24:06.362430 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_869_238, 202410_870_870_0] 2024.10.22 02:24:06.366737 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_929_294} MergeTreeSequentialSource: Reading 2 marks from part 202410_929_929_0, total 14 rows starting from the beginning of the part 2024.10.22 02:24:06.370558 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_869_238, 202410_870_870_0] 2024.10.22 02:24:06.398463 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_869_238, 202410_870_870_0] 2024.10.22 02:24:06.724671 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 10.90 MiB, will set to 1.61 GiB (RSS), difference: 78.90 MiB 2024.10.22 02:24:06.767112 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:06.775580 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.03 GiB. 2024.10.22 02:24:06.814322 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 763 2024.10.22 02:24:06.957766 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1550_1550_0 (state Temporary) 2024.10.22 02:24:06.999853 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1550_1550_0 to 202410_1563_1563_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:07.148048 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103707 2024.10.22 02:24:07.156812 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1558_105 to 202410_1559_1559_0 2024.10.22 02:24:07.163465 [ 1235 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.07 MiB using storage policy from min volume index 0 2024.10.22 02:24:07.165334 [ 1235 ] {} DiskLocal: Reserved 1.07 MiB on local disk `default`, having unreserved 296.03 GiB. 2024.10.22 02:24:07.211945 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1559_106} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1558_105 to 202410_1559_1559_0 into Compact with storage Full 2024.10.22 02:24:07.240000 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_929_294} MergeTask::MergeProjectionsStage: Merge sorted 14118 rows, containing 12 columns (12 merged, 0 gathered) in 1.105053388 sec., 12775.853323749097 rows/sec., 2.00 MiB/sec. 2024.10.22 02:24:07.300707 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:07.309717 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.03 GiB. 2024.10.22 02:24:07.319390 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_929_294} MergedBlockOutputStream: filled checksums 202410_1_929_294 (state Temporary) 2024.10.22 02:24:07.360254 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1559_106} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:07.376996 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1559_106} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1558_105, total 24280 rows starting from the beginning of the part 2024.10.22 02:24:07.487722 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_745_745_0 (state Temporary) 2024.10.22 02:24:07.529688 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1559_106} MergeTreeSequentialSource: Reading 2 marks from part 202410_1559_1559_0, total 70 rows starting from the beginning of the part 2024.10.22 02:24:07.572489 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_745_745_0 to 202410_755_755_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:07.741504 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_929_294} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_929_294 to 202410_1_929_294 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:07.847539 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_929_294} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_928_293, 202410_929_929_0] -> 202410_1_929_294 2024.10.22 02:24:07.934231 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(129f74b1bda3b2d90f43191046cca496) : [0, 2303660]: Setting download as failed: File segment: [0, 2303660], key: 129f74b1bda3b2d90f43191046cca496, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:07.946722 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(129f74b1bda3b2d90f43191046cca496): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:24:07.962582 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:24:08.067778 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 763 2024.10.22 02:24:08.097564 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 129f74b1bda3b2d90f43191046cca496, offset: 0, size: 0 2024.10.22 02:24:08.102081 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 129f74b1bda3b2d90f43191046cca496 for removal 2024.10.22 02:24:08.111010 [ 1476 ] {} CacheMetadata: Key 129f74b1bda3b2d90f43191046cca496 is removed from metadata 2024.10.22 02:24:08.511770 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 15.54 MiB, will set to 1.59 GiB (RSS), difference: -23.02 MiB 2024.10.22 02:24:08.734304 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7429 2024.10.22 02:24:09.067801 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1559_106} MergeTask::MergeProjectionsStage: Merge sorted 24350 rows, containing 13 columns (13 merged, 0 gathered) in 1.875181235 sec., 12985.411514103596 rows/sec., 2.35 MiB/sec. 2024.10.22 02:24:09.150220 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1559_106} MergedBlockOutputStream: filled checksums 202410_1207_1559_106 (state Temporary) 2024.10.22 02:24:09.254171 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 22 file segments (while reserving 50387 bytes for a845e1d3851131c305b92671ef1836e0:0) 2024.10.22 02:24:09.255936 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 49ee2b827e1e751f393ef5ec89d3da31, offset: 0, size: 153 2024.10.22 02:24:09.258631 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 49ee2b827e1e751f393ef5ec89d3da31 for removal 2024.10.22 02:24:09.260297 [ 1476 ] {} CacheMetadata: Key 49ee2b827e1e751f393ef5ec89d3da31 is removed from metadata 2024.10.22 02:24:09.261282 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b31861a40b93059354efbcdef85a3a8f, offset: 0, size: 154 2024.10.22 02:24:09.265280 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b31861a40b93059354efbcdef85a3a8f for removal 2024.10.22 02:24:09.268198 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 91693b8e28e0beac6c548f8b7d094999, offset: 0, size: 152 2024.10.22 02:24:09.271109 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 91693b8e28e0beac6c548f8b7d094999 for removal 2024.10.22 02:24:09.273978 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c8c6c466513af68b54e88138e9c79247, offset: 0, size: 153 2024.10.22 02:24:09.279624 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c8c6c466513af68b54e88138e9c79247 for removal 2024.10.22 02:24:09.285476 [ 1476 ] {} CacheMetadata: Key 91693b8e28e0beac6c548f8b7d094999 is removed from metadata 2024.10.22 02:24:09.287697 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fa9cf967225dc26cd277ff652b4a82e9, offset: 0, size: 154 2024.10.22 02:24:09.295928 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fa9cf967225dc26cd277ff652b4a82e9 for removal 2024.10.22 02:24:09.300222 [ 1476 ] {} CacheMetadata: Key c8c6c466513af68b54e88138e9c79247 is removed from metadata 2024.10.22 02:24:09.300670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4a3ba6638f8fedc22eb6539986dc1a17, offset: 0, size: 83510 2024.10.22 02:24:09.307264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4a3ba6638f8fedc22eb6539986dc1a17 for removal 2024.10.22 02:24:09.312088 [ 1476 ] {} CacheMetadata: Key 4a3ba6638f8fedc22eb6539986dc1a17 is removed from metadata 2024.10.22 02:24:09.313263 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e66d701e0b24541308d51fdaace0d1ca, offset: 0, size: 150 2024.10.22 02:24:09.314907 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1336058 2024.10.22 02:24:09.317244 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e66d701e0b24541308d51fdaace0d1ca for removal 2024.10.22 02:24:09.321375 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8c0232a64979dd38010e9ad191801630, offset: 0, size: 154 2024.10.22 02:24:09.324531 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8c0232a64979dd38010e9ad191801630 for removal 2024.10.22 02:24:09.326411 [ 1476 ] {} CacheMetadata: Key e66d701e0b24541308d51fdaace0d1ca is removed from metadata 2024.10.22 02:24:09.327799 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 35f4968b81335bced29cd0b799111e13, offset: 0, size: 158 2024.10.22 02:24:09.331751 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 35f4968b81335bced29cd0b799111e13 for removal 2024.10.22 02:24:09.339804 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e2317569a6822abb39b144cb81a8c48b, offset: 0, size: 186 2024.10.22 02:24:09.345007 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e2317569a6822abb39b144cb81a8c48b for removal 2024.10.22 02:24:09.346797 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4959bd61faa3bbeba6e781940f1eeb0e, offset: 0, size: 151 2024.10.22 02:24:09.348953 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4959bd61faa3bbeba6e781940f1eeb0e for removal 2024.10.22 02:24:09.351387 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 68f5abd6c805f8b69c0ff1bdbd4b37f2, offset: 0, size: 154 2024.10.22 02:24:09.353621 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 68f5abd6c805f8b69c0ff1bdbd4b37f2 for removal 2024.10.22 02:24:09.358885 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2757e5291bfd50fd0c3d7ed309cb010f, offset: 0, size: 151 2024.10.22 02:24:09.362441 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2757e5291bfd50fd0c3d7ed309cb010f for removal 2024.10.22 02:24:09.366217 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6523f225eb03e0bcd3c2896b2c182852, offset: 0, size: 154 2024.10.22 02:24:09.369490 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1559_106} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1559_106 to 202410_1207_1559_106 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:09.374328 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1559_106} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1558_105, 202410_1559_1559_0] -> 202410_1207_1559_106 2024.10.22 02:24:09.382478 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6523f225eb03e0bcd3c2896b2c182852 for removal 2024.10.22 02:24:09.391470 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d390649a985c0482a9c1c1e499b157cf, offset: 0, size: 150 2024.10.22 02:24:09.397633 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d390649a985c0482a9c1c1e499b157cf for removal 2024.10.22 02:24:09.411573 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 09754ff4ea9020a60b1fc68759530a45, offset: 0, size: 1236 2024.10.22 02:24:09.435532 [ 1476 ] {} CacheMetadata: Key 8c0232a64979dd38010e9ad191801630 is removed from metadata 2024.10.22 02:24:09.446344 [ 1476 ] {} CacheMetadata: Key d390649a985c0482a9c1c1e499b157cf is removed from metadata 2024.10.22 02:24:09.451632 [ 1476 ] {} CacheMetadata: Key 6523f225eb03e0bcd3c2896b2c182852 is removed from metadata 2024.10.22 02:24:09.468288 [ 1476 ] {} CacheMetadata: Key 2757e5291bfd50fd0c3d7ed309cb010f is removed from metadata 2024.10.22 02:24:09.513397 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.08 MiB. 2024.10.22 02:24:09.515091 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 09754ff4ea9020a60b1fc68759530a45 for removal 2024.10.22 02:24:09.529188 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 550dece6efb92664a3c2dfbb56e2d3ba, offset: 0, size: 158 2024.10.22 02:24:09.534679 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 550dece6efb92664a3c2dfbb56e2d3ba for removal 2024.10.22 02:24:09.538986 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2c54aca2d24194cba85e5e94a2dc8cb5, offset: 0, size: 150 2024.10.22 02:24:09.542708 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2c54aca2d24194cba85e5e94a2dc8cb5 for removal 2024.10.22 02:24:09.547214 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8e5da73431b1877aa602c268b410c2ae, offset: 0, size: 150 2024.10.22 02:24:09.550535 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8e5da73431b1877aa602c268b410c2ae for removal 2024.10.22 02:24:09.553671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 36b8a34e7ec9fc646684c8d1429f8935, offset: 0, size: 153 2024.10.22 02:24:09.586650 [ 1476 ] {} CacheMetadata: Key 4959bd61faa3bbeba6e781940f1eeb0e is removed from metadata 2024.10.22 02:24:09.605838 [ 1476 ] {} CacheMetadata: Key 8e5da73431b1877aa602c268b410c2ae is removed from metadata 2024.10.22 02:24:09.622147 [ 1476 ] {} CacheMetadata: Key 2c54aca2d24194cba85e5e94a2dc8cb5 is removed from metadata 2024.10.22 02:24:09.635773 [ 1476 ] {} CacheMetadata: Key 550dece6efb92664a3c2dfbb56e2d3ba is removed from metadata 2024.10.22 02:24:09.646768 [ 1476 ] {} CacheMetadata: Key 09754ff4ea9020a60b1fc68759530a45 is removed from metadata 2024.10.22 02:24:09.655072 [ 1476 ] {} CacheMetadata: Key 68f5abd6c805f8b69c0ff1bdbd4b37f2 is removed from metadata 2024.10.22 02:24:09.660521 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 36b8a34e7ec9fc646684c8d1429f8935 for removal 2024.10.22 02:24:09.667010 [ 1476 ] {} CacheMetadata: Key e2317569a6822abb39b144cb81a8c48b is removed from metadata 2024.10.22 02:24:09.667765 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 523c69afff605b94b1dd1794a1f8475c, offset: 0, size: 185 2024.10.22 02:24:09.669817 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 523c69afff605b94b1dd1794a1f8475c for removal 2024.10.22 02:24:09.673033 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5fb41c0b8507041dc74088ab4d27f4fd, offset: 0, size: 4 2024.10.22 02:24:09.678582 [ 1476 ] {} CacheMetadata: Key 523c69afff605b94b1dd1794a1f8475c is removed from metadata 2024.10.22 02:24:09.681225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5fb41c0b8507041dc74088ab4d27f4fd for removal 2024.10.22 02:24:09.697311 [ 1476 ] {} CacheMetadata: Key 5fb41c0b8507041dc74088ab4d27f4fd is removed from metadata 2024.10.22 02:24:09.712770 [ 1476 ] {} CacheMetadata: Key 36b8a34e7ec9fc646684c8d1429f8935 is removed from metadata 2024.10.22 02:24:09.740879 [ 1476 ] {} CacheMetadata: Key 35f4968b81335bced29cd0b799111e13 is removed from metadata 2024.10.22 02:24:09.751513 [ 1476 ] {} CacheMetadata: Key fa9cf967225dc26cd277ff652b4a82e9 is removed from metadata 2024.10.22 02:24:09.763650 [ 1476 ] {} CacheMetadata: Key b31861a40b93059354efbcdef85a3a8f is removed from metadata 2024.10.22 02:24:09.859115 [ 1221 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:24:09.893801 [ 1221 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:24:10.273750 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:10.277662 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.03 GiB. 2024.10.22 02:24:10.605084 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_881_881_0 (state Temporary) 2024.10.22 02:24:10.709382 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_881_881_0 to 202410_888_888_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:10.752737 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1336058 2024.10.22 02:24:10.865244 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_880_164/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:24:11.152751 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 119 entries to flush up to offset 103826 2024.10.22 02:24:11.365262 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:24:11.368490 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:24:11.527206 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:11.530655 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.03 GiB. 2024.10.22 02:24:11.591754 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5c522d8ca65dda2952ca53c7eb777feb) : [0, 1121086]: Setting download as failed: File segment: [0, 1121086], key: 5c522d8ca65dda2952ca53c7eb777feb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:11.594880 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5c522d8ca65dda2952ca53c7eb777feb): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:24:11.638604 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 18.19 MiB, will set to 1.59 GiB (RSS), difference: 20.01 MiB 2024.10.22 02:24:11.649123 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5c522d8ca65dda2952ca53c7eb777feb, offset: 0, size: 0 2024.10.22 02:24:11.650966 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5c522d8ca65dda2952ca53c7eb777feb for removal 2024.10.22 02:24:11.651443 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1551_1551_0 (state Temporary) 2024.10.22 02:24:11.652994 [ 1476 ] {} CacheMetadata: Key 5c522d8ca65dda2952ca53c7eb777feb is removed from metadata 2024.10.22 02:24:11.743609 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1551_1551_0 to 202410_1564_1564_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:11.767443 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1559_106 to 202410_1560_1560_0 2024.10.22 02:24:11.771530 [ 1209 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.07 MiB using storage policy from min volume index 0 2024.10.22 02:24:11.784607 [ 1209 ] {} DiskLocal: Reserved 1.07 MiB on local disk `default`, having unreserved 296.03 GiB. 2024.10.22 02:24:11.816183 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1560_107} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1559_106 to 202410_1560_1560_0 into Compact with storage Full 2024.10.22 02:24:11.892366 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103826 2024.10.22 02:24:12.002120 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1560_107} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:12.029809 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1560_107} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1559_106, total 24350 rows starting from the beginning of the part 2024.10.22 02:24:12.037499 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 103 entries to flush up to offset 100431 2024.10.22 02:24:12.077649 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1560_107} MergeTreeSequentialSource: Reading 2 marks from part 202410_1560_1560_0, total 55 rows starting from the beginning of the part 2024.10.22 02:24:12.178528 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:12.188473 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.03 GiB. 2024.10.22 02:24:12.248441 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_905_905_0 (state Temporary) 2024.10.22 02:24:12.422630 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 3 file segments (while reserving 42229 bytes for abdb8012c9e96c48d172034c8193616e:0) 2024.10.22 02:24:12.430342 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a845e1d3851131c305b92671ef1836e0, offset: 0, size: 50387 2024.10.22 02:24:12.437675 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a845e1d3851131c305b92671ef1836e0 for removal 2024.10.22 02:24:12.452555 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: e0081082d4909f293a447e44522c6ee4, offset: 0, size: 156 2024.10.22 02:24:12.455377 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key e0081082d4909f293a447e44522c6ee4 for removal 2024.10.22 02:24:12.456015 [ 1476 ] {} CacheMetadata: Key a845e1d3851131c305b92671ef1836e0 is removed from metadata 2024.10.22 02:24:12.460012 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5a5d9be16db803203558bddf637aa8a7, offset: 0, size: 212 2024.10.22 02:24:12.464168 [ 1476 ] {} CacheMetadata: Key e0081082d4909f293a447e44522c6ee4 is removed from metadata 2024.10.22 02:24:12.466414 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 5a5d9be16db803203558bddf637aa8a7 for removal 2024.10.22 02:24:12.469298 [ 1476 ] {} CacheMetadata: Key 5a5d9be16db803203558bddf637aa8a7 is removed from metadata 2024.10.22 02:24:12.525064 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_905_905_0 to 202410_911_911_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:12.568714 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 100431 2024.10.22 02:24:12.730203 [ 1216 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 205 ms. 2024.10.22 02:24:13.537424 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1560_107} MergeTask::MergeProjectionsStage: Merge sorted 24405 rows, containing 13 columns (13 merged, 0 gathered) in 1.730507508 sec., 14102.799258123761 rows/sec., 2.56 MiB/sec. 2024.10.22 02:24:13.565398 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 14101 2024.10.22 02:24:13.694533 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 10342 bytes for 2aeae08f55e5646058ab29367f9670ce:0) 2024.10.22 02:24:13.697403 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: ca7738060be7ec4e3a766c832a87dffd, offset: 0, size: 2573 2024.10.22 02:24:13.705030 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key ca7738060be7ec4e3a766c832a87dffd for removal 2024.10.22 02:24:13.709740 [ 1476 ] {} CacheMetadata: Key ca7738060be7ec4e3a766c832a87dffd is removed from metadata 2024.10.22 02:24:13.711819 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d9919a490e6813bd65eaaf4720bb09b8, offset: 0, size: 167 2024.10.22 02:24:13.715705 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d9919a490e6813bd65eaaf4720bb09b8 for removal 2024.10.22 02:24:13.732276 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1560_107} MergedBlockOutputStream: filled checksums 202410_1207_1560_107 (state Temporary) 2024.10.22 02:24:13.751925 [ 1476 ] {} CacheMetadata: Key d9919a490e6813bd65eaaf4720bb09b8 is removed from metadata 2024.10.22 02:24:13.771649 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:13.778005 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.03 GiB. 2024.10.22 02:24:13.942796 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_921_921_0 (state Temporary) 2024.10.22 02:24:14.039116 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_921_921_0 to 202410_934_934_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:14.046164 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1460_55, 202410_1461_1461_0] 2024.10.22 02:24:14.049823 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1460_55, 202410_1461_1461_0] 2024.10.22 02:24:14.044576 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1560_107} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1560_107 to 202410_1207_1560_107 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:14.054936 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1560_107} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1559_106, 202410_1560_1560_0] -> 202410_1207_1560_107 2024.10.22 02:24:14.061637 [ 1275 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_929_294 to 202410_930_930_0 2024.10.22 02:24:14.064510 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1460_55, 202410_1461_1461_0] 2024.10.22 02:24:14.067640 [ 1275 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:14.069929 [ 1275 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.03 GiB. 2024.10.22 02:24:14.084220 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:24:14.097489 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14101 2024.10.22 02:24:14.181018 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_930_295} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_929_294 to 202410_930_930_0 into Compact with storage Full 2024.10.22 02:24:14.193479 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} MergeTask::MergeProjectionsStage: Merge sorted 7368 rows, containing 679 columns (679 merged, 0 gathered) in 139.764352583 sec., 52.71730497677842 rows/sec., 70.99 KiB/sec. 2024.10.22 02:24:14.347660 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_930_295} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:14.373803 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_930_295} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_929_294, total 14118 rows starting from the beginning of the part 2024.10.22 02:24:14.459602 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_930_295} MergeTreeSequentialSource: Reading 2 marks from part 202410_930_930_0, total 12 rows starting from the beginning of the part 2024.10.22 02:24:14.646486 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 14.66 MiB, will set to 1.58 GiB (RSS), difference: -3.69 MiB 2024.10.22 02:24:14.868828 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 29921 bytes for 2bd72bf5824c6a8a7fa3c28a1f9f4ae1:0) 2024.10.22 02:24:14.871247 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 96566ad0034c9a8891a3310b68e4cefd, offset: 0, size: 39222 2024.10.22 02:24:14.876210 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 96566ad0034c9a8891a3310b68e4cefd for removal 2024.10.22 02:24:14.882293 [ 1476 ] {} CacheMetadata: Key 96566ad0034c9a8891a3310b68e4cefd is removed from metadata 2024.10.22 02:24:14.884801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f413a6816147416dbd37e438a78e2cfa, offset: 0, size: 167 2024.10.22 02:24:14.889092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f413a6816147416dbd37e438a78e2cfa for removal 2024.10.22 02:24:14.894133 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: aa0768875810c08731596345862fc02f, offset: 0, size: 169 2024.10.22 02:24:14.898310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key aa0768875810c08731596345862fc02f for removal 2024.10.22 02:24:14.909583 [ 1476 ] {} CacheMetadata: Key aa0768875810c08731596345862fc02f is removed from metadata 2024.10.22 02:24:14.927806 [ 1476 ] {} CacheMetadata: Key f413a6816147416dbd37e438a78e2cfa is removed from metadata 2024.10.22 02:24:15.088810 [ 1691 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/sjb/lwuuvhlrdwkexojtfhtxpavplzfux, size 10342 2024.10.22 02:24:15.464306 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParamPrice 2024.10.22 02:24:15.483809 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParamPrice 2024.10.22 02:24:15.515262 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParamPrice 2024.10.22 02:24:15.572822 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 764 2024.10.22 02:24:15.662961 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParamPrice 2024.10.22 02:24:15.700045 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_827_827_0, 202410_828_828_0, 202410_829_829_0, 202410_830_830_0, 202410_831_831_0, 202410_832_832_0] 2024.10.22 02:24:15.709424 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_827_827_0, 202410_828_828_0, 202410_829_829_0, 202410_830_830_0, 202410_831_831_0, 202410_832_832_0] 2024.10.22 02:24:15.735567 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParamPrice 2024.10.22 02:24:15.735946 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_870_239, 202410_871_871_0] 2024.10.22 02:24:15.741398 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_870_239, 202410_871_871_0] 2024.10.22 02:24:15.752389 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_827_827_0, 202410_828_828_0, 202410_829_829_0, 202410_830_830_0, 202410_831_831_0, 202410_832_832_0] 2024.10.22 02:24:15.769265 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParamPrice 2024.10.22 02:24:15.769829 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_870_239, 202410_871_871_0] 2024.10.22 02:24:15.853250 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParamPrice 2024.10.22 02:24:15.948988 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:15.984290 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_930_295} MergeTask::MergeProjectionsStage: Merge sorted 14130 rows, containing 12 columns (12 merged, 0 gathered) in 1.822501907 sec., 7753.078307204208 rows/sec., 1.21 MiB/sec. 2024.10.22 02:24:15.986852 [ 1273 ] {} DatabaseCatalog: Execution took 304 ms. 2024.10.22 02:24:16.005610 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 103919 2024.10.22 02:24:16.010655 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.00 GiB. 2024.10.22 02:24:16.037018 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yxw/rljofxlyeddhfpdhxigqcrfmimghd (LogSeriesLimiter: on interval from 2024-10-22 02:23:46 to 2024-10-22 02:24:16 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:24:16.091131 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_746_746_0 (state Temporary) 2024.10.22 02:24:16.168208 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_930_295} MergedBlockOutputStream: filled checksums 202410_1_930_295 (state Temporary) 2024.10.22 02:24:16.173526 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a2a0ff7eeafda83541ff2ead642664cf) : [0, 285540]: Setting download as failed: File segment: [0, 285540], key: a2a0ff7eeafda83541ff2ead642664cf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:16.173630 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_184_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:24:16.176548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a2a0ff7eeafda83541ff2ead642664cf): No space left in cache to reserve 285541 bytes, will continue without cache download 2024.10.22 02:24:16.262804 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 46 entries to flush up to offset 7475 2024.10.22 02:24:16.337031 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a2a0ff7eeafda83541ff2ead642664cf, offset: 0, size: 0 2024.10.22 02:24:16.343871 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a2a0ff7eeafda83541ff2ead642664cf for removal 2024.10.22 02:24:16.352513 [ 1476 ] {} CacheMetadata: Key a2a0ff7eeafda83541ff2ead642664cf is removed from metadata 2024.10.22 02:24:16.354910 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_746_746_0 to 202410_756_756_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:16.472394 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_930_295} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_930_295 to 202410_1_930_295 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:16.487032 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_930_295} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_929_294, 202410_930_930_0] -> 202410_1_930_295 2024.10.22 02:24:16.498451 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:16.507264 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.00 GiB. 2024.10.22 02:24:16.507739 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 764 2024.10.22 02:24:16.538634 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:24:16.579955 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1552_1552_0 (state Temporary) 2024.10.22 02:24:16.653165 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 15.88 MiB, will set to 1.57 GiB (RSS), difference: -17.83 MiB 2024.10.22 02:24:16.656688 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1552_1552_0 to 202410_1565_1565_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:16.704245 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103919 2024.10.22 02:24:16.712326 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1560_107 to 202410_1561_1561_0 2024.10.22 02:24:16.727126 [ 1244 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.07 MiB using storage policy from min volume index 0 2024.10.22 02:24:16.832706 [ 1244 ] {} DiskLocal: Reserved 1.07 MiB on local disk `default`, having unreserved 296.00 GiB. 2024.10.22 02:24:16.944023 [ 1244 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 278 ms. 2024.10.22 02:24:16.969654 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1561_108} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1560_107 to 202410_1561_1561_0 into Compact with storage Full 2024.10.22 02:24:17.105309 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1561_108} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:17.146907 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1561_108} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1560_107, total 24405 rows starting from the beginning of the part 2024.10.22 02:24:17.239679 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bdc9d01067448c249ef158b85cc8d656) : [0, 185836]: Setting download as failed: File segment: [0, 185836], key: bdc9d01067448c249ef158b85cc8d656, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:17.242664 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bdc9d01067448c249ef158b85cc8d656): No space left in cache to reserve 185837 bytes, will continue without cache download 2024.10.22 02:24:17.270475 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bdc9d01067448c249ef158b85cc8d656, offset: 0, size: 0 2024.10.22 02:24:17.272645 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bdc9d01067448c249ef158b85cc8d656 for removal 2024.10.22 02:24:17.277526 [ 1476 ] {} CacheMetadata: Key bdc9d01067448c249ef158b85cc8d656 is removed from metadata 2024.10.22 02:24:17.464722 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1561_108} MergeTreeSequentialSource: Reading 2 marks from part 202410_1561_1561_0, total 60 rows starting from the beginning of the part 2024.10.22 02:24:17.497675 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_880_164} MergeTask::MergeProjectionsStage: Merge sorted 1336490 rows, containing 4 columns (4 merged, 0 gathered) in 41.089226093 sec., 32526.53133390813 rows/sec., 487.75 KiB/sec. 2024.10.22 02:24:17.799227 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_880_164} MergedBlockOutputStream: filled checksums 202410_1_880_164 (state Temporary) 2024.10.22 02:24:17.785781 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1337598 2024.10.22 02:24:18.380501 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} MergedBlockOutputStream: filled checksums 202410_1_184_29 (state Temporary) 2024.10.22 02:24:18.458245 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1461_56, 202410_1462_1462_0] 2024.10.22 02:24:18.461706 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1461_56, 202410_1462_1462_0] 2024.10.22 02:24:18.471269 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1461_56, 202410_1462_1462_0] 2024.10.22 02:24:18.512264 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 14.15 MiB, will set to 1.57 GiB (RSS), difference: 15.18 MiB 2024.10.22 02:24:18.737037 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(741c1da003a75f939aa44db08885f936) : [0, 418558]: Setting download as failed: File segment: [0, 418558], key: 741c1da003a75f939aa44db08885f936, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:18.739105 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(741c1da003a75f939aa44db08885f936): No space left in cache to reserve 418559 bytes, will continue without cache download 2024.10.22 02:24:18.806419 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:18.812322 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.00 GiB. 2024.10.22 02:24:18.875580 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 741c1da003a75f939aa44db08885f936, offset: 0, size: 0 2024.10.22 02:24:18.918178 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_882_882_0 (state Temporary) 2024.10.22 02:24:18.981625 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 741c1da003a75f939aa44db08885f936 for removal 2024.10.22 02:24:18.987175 [ 1476 ] {} CacheMetadata: Key 741c1da003a75f939aa44db08885f936 is removed from metadata 2024.10.22 02:24:19.154066 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_882_882_0 to 202410_889_889_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:19.213030 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1337598 2024.10.22 02:24:19.458581 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1561_108} MergeTask::MergeProjectionsStage: Merge sorted 24465 rows, containing 13 columns (13 merged, 0 gathered) in 2.49309681 sec., 9813.0966683159 rows/sec., 1.78 MiB/sec. 2024.10.22 02:24:19.528247 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1561_108} MergedBlockOutputStream: filled checksums 202410_1207_1561_108 (state Temporary) 2024.10.22 02:24:19.813416 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1561_108} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1561_108 to 202410_1207_1561_108 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:19.817446 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1561_108} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1560_107, 202410_1561_1561_0] -> 202410_1207_1561_108 2024.10.22 02:24:19.852167 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:24:20.100911 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 140 entries to flush up to offset 100571 2024.10.22 02:24:20.157061 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f31e8b956ba03f8314cde16dbd51fffd) : [0, 74536]: Setting download as failed: File segment: [0, 74536], key: f31e8b956ba03f8314cde16dbd51fffd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:20.159196 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f31e8b956ba03f8314cde16dbd51fffd): No space left in cache to reserve 74537 bytes, will continue without cache download 2024.10.22 02:24:20.178909 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f31e8b956ba03f8314cde16dbd51fffd, offset: 0, size: 0 2024.10.22 02:24:20.182995 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f31e8b956ba03f8314cde16dbd51fffd for removal 2024.10.22 02:24:20.187750 [ 1476 ] {} CacheMetadata: Key f31e8b956ba03f8314cde16dbd51fffd is removed from metadata 2024.10.22 02:24:20.519768 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:20.528739 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.00 GiB. 2024.10.22 02:24:20.547091 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_703_143, 202410_704_704_0, 202410_705_705_0, 202410_706_706_0, 202410_707_707_0, 202410_708_708_0] 2024.10.22 02:24:20.550487 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_703_143, 202410_704_704_0, 202410_705_705_0, 202410_706_706_0, 202410_707_707_0, 202410_708_708_0] 2024.10.22 02:24:20.574953 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_906_906_0 (state Temporary) 2024.10.22 02:24:20.605827 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_703_143, 202410_704_704_0, 202410_705_705_0, 202410_706_706_0, 202410_707_707_0, 202410_708_708_0] 2024.10.22 02:24:20.717677 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 103993 2024.10.22 02:24:20.758340 [ 1161 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 277 ms. 2024.10.22 02:24:20.826304 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_906_906_0 to 202410_912_912_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:20.854465 [ 1171 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_907_32 to 202410_912_912_0 2024.10.22 02:24:20.866862 [ 1171 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.03 MiB using storage policy from min volume index 0 2024.10.22 02:24:20.871486 [ 1171 ] {} DiskLocal: Reserved 1.03 MiB on local disk `default`, having unreserved 296.00 GiB. 2024.10.22 02:24:20.876879 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:20.883940 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.00 GiB. 2024.10.22 02:24:20.887265 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 100571 2024.10.22 02:24:20.900466 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_907_32 to 202410_912_912_0 into Compact with storage Full 2024.10.22 02:24:20.955645 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1553_1553_0 (state Temporary) 2024.10.22 02:24:20.999963 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1553_1553_0 to 202410_1566_1566_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:21.042598 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1561_108 to 202410_1562_1562_0 2024.10.22 02:24:21.052553 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 103993 2024.10.22 02:24:21.056558 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.07 MiB using storage policy from min volume index 0 2024.10.22 02:24:21.058571 [ 1165 ] {} DiskLocal: Reserved 1.07 MiB on local disk `default`, having unreserved 296.00 GiB. 2024.10.22 02:24:21.179270 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1562_109} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1561_108 to 202410_1562_1562_0 into Compact with storage Full 2024.10.22 02:24:21.218579 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:21.242452 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1562_109} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:21.243858 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} MergeTreeSequentialSource: Reading 3 marks from part 202410_747_907_32, total 16499 rows starting from the beginning of the part 2024.10.22 02:24:21.250896 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1562_109} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1561_108, total 24465 rows starting from the beginning of the part 2024.10.22 02:24:21.259204 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a5d2717004eff2ebd6cd153daf4d1b4e) : [0, 83142]: Setting download as failed: File segment: [0, 83142], key: a5d2717004eff2ebd6cd153daf4d1b4e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:21.260829 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a5d2717004eff2ebd6cd153daf4d1b4e): No space left in cache to reserve 83143 bytes, will continue without cache download 2024.10.22 02:24:21.268027 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_908_908_0, total 190 rows starting from the beginning of the part 2024.10.22 02:24:21.279063 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_880_164/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:24:21.309822 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_909_909_0, total 152 rows starting from the beginning of the part 2024.10.22 02:24:21.347853 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_910_910_0, total 132 rows starting from the beginning of the part 2024.10.22 02:24:21.364216 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:24:21.365749 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:24:21.371138 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a5d2717004eff2ebd6cd153daf4d1b4e, offset: 0, size: 0 2024.10.22 02:24:21.372013 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a5d2717004eff2ebd6cd153daf4d1b4e for removal 2024.10.22 02:24:21.375589 [ 1476 ] {} CacheMetadata: Key a5d2717004eff2ebd6cd153daf4d1b4e is removed from metadata 2024.10.22 02:24:21.378736 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_911_911_0, total 103 rows starting from the beginning of the part 2024.10.22 02:24:21.394330 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1562_109} MergeTreeSequentialSource: Reading 2 marks from part 202410_1562_1562_0, total 54 rows starting from the beginning of the part 2024.10.22 02:24:21.422488 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} MergeTreeSequentialSource: Reading 2 marks from part 202410_912_912_0, total 140 rows starting from the beginning of the part 2024.10.22 02:24:21.603256 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 14116 2024.10.22 02:24:21.664435 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_880_164} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_880_164 to 202410_1_880_164 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:21.668430 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_880_164} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_874_163, 202410_875_880_1] -> 202410_1_880_164 2024.10.22 02:24:21.757444 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:24:21.858364 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1462_57, 202410_1463_1463_0] 2024.10.22 02:24:21.860078 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1462_57, 202410_1463_1463_0] 2024.10.22 02:24:21.869095 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1462_57, 202410_1463_1463_0] 2024.10.22 02:24:21.909838 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:21.914845 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.00 GiB. 2024.10.22 02:24:21.953437 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_922_922_0 (state Temporary) 2024.10.22 02:24:22.208640 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_922_922_0 to 202410_935_935_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:22.228991 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3d9fda6eced794e2a8204bc5d6124a94) : [0, 132635]: Setting download as failed: File segment: [0, 132635], key: 3d9fda6eced794e2a8204bc5d6124a94, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:22.243072 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3d9fda6eced794e2a8204bc5d6124a94): No space left in cache to reserve 132636 bytes, will continue without cache download 2024.10.22 02:24:22.255400 [ 1157 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_930_295 to 202410_931_931_0 2024.10.22 02:24:22.266165 [ 1157 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:22.269240 [ 1157 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.00 GiB. 2024.10.22 02:24:22.294523 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14116 2024.10.22 02:24:22.303608 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_931_296} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_930_295 to 202410_931_931_0 into Compact with storage Full 2024.10.22 02:24:22.367148 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3d9fda6eced794e2a8204bc5d6124a94, offset: 0, size: 0 2024.10.22 02:24:22.368471 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3d9fda6eced794e2a8204bc5d6124a94 for removal 2024.10.22 02:24:22.370528 [ 1476 ] {} CacheMetadata: Key 3d9fda6eced794e2a8204bc5d6124a94 is removed from metadata 2024.10.22 02:24:22.555424 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_931_296} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:22.667044 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_931_296} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_930_295, total 14130 rows starting from the beginning of the part 2024.10.22 02:24:22.801608 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_931_296} MergeTreeSequentialSource: Reading 2 marks from part 202410_931_931_0, total 29 rows starting from the beginning of the part 2024.10.22 02:24:23.039071 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} MergeTask::MergeProjectionsStage: Merge sorted 17216 rows, containing 13 columns (13 merged, 0 gathered) in 2.142650808 sec., 8034.90701131549 rows/sec., 3.49 MiB/sec. 2024.10.22 02:24:23.067789 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1562_109} MergeTask::MergeProjectionsStage: Merge sorted 24519 rows, containing 13 columns (13 merged, 0 gathered) in 1.996535818 sec., 12280.771413638622 rows/sec., 2.23 MiB/sec. 2024.10.22 02:24:23.067922 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} MergedBlockOutputStream: filled checksums 202410_747_912_33 (state Temporary) 2024.10.22 02:24:23.115890 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1562_109/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:24:23.155089 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1562_109} MergedBlockOutputStream: filled checksums 202410_1207_1562_109 (state Temporary) 2024.10.22 02:24:23.222241 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_912_33 to 202410_747_912_33 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:23.229151 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_912_33} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_907_32, 202410_912_912_0] -> 202410_747_912_33 2024.10.22 02:24:23.258348 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4415fdfd9677ece7d77fd45d59c49516) : [0, 128577]: Setting download as failed: File segment: [0, 128577], key: 4415fdfd9677ece7d77fd45d59c49516, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:23.258728 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.93 MiB. 2024.10.22 02:24:23.259674 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4415fdfd9677ece7d77fd45d59c49516): No space left in cache to reserve 128578 bytes, will continue without cache download 2024.10.22 02:24:23.276626 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4415fdfd9677ece7d77fd45d59c49516, offset: 0, size: 0 2024.10.22 02:24:23.280965 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4415fdfd9677ece7d77fd45d59c49516 for removal 2024.10.22 02:24:23.284457 [ 1476 ] {} CacheMetadata: Key 4415fdfd9677ece7d77fd45d59c49516 is removed from metadata 2024.10.22 02:24:23.473421 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1562_109} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1562_109 to 202410_1207_1562_109 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:23.479209 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1562_109} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1561_108, 202410_1562_1562_0] -> 202410_1207_1562_109 2024.10.22 02:24:23.517230 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.06 MiB. 2024.10.22 02:24:23.609352 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 4.72 MiB, will set to 1.57 GiB (RSS), difference: 6.16 MiB 2024.10.22 02:24:23.677140 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_931_296} MergeTask::MergeProjectionsStage: Merge sorted 14159 rows, containing 12 columns (12 merged, 0 gathered) in 1.383149196 sec., 10236.78431867447 rows/sec., 1.60 MiB/sec. 2024.10.22 02:24:23.746018 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_931_296} MergedBlockOutputStream: filled checksums 202410_1_931_296 (state Temporary) 2024.10.22 02:24:24.011548 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 765 2024.10.22 02:24:24.040521 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_931_296/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:24:24.148659 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_931_296} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_931_296 to 202410_1_931_296 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:24.149419 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_871_240, 202410_872_872_0] 2024.10.22 02:24:24.151723 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_931_296} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_930_295, 202410_931_931_0] -> 202410_1_931_296 2024.10.22 02:24:24.152733 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_871_240, 202410_872_872_0] 2024.10.22 02:24:24.168669 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_871_240, 202410_872_872_0] 2024.10.22 02:24:24.181191 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:24:24.491340 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f6c044b2b54c6c5953105552341d0ae1) : [0, 62577]: Setting download as failed: File segment: [0, 62577], key: f6c044b2b54c6c5953105552341d0ae1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:24.497824 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f6c044b2b54c6c5953105552341d0ae1): No space left in cache to reserve 62578 bytes, will continue without cache download 2024.10.22 02:24:24.526973 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f6c044b2b54c6c5953105552341d0ae1, offset: 0, size: 0 2024.10.22 02:24:24.531278 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f6c044b2b54c6c5953105552341d0ae1 for removal 2024.10.22 02:24:24.537195 [ 1476 ] {} CacheMetadata: Key f6c044b2b54c6c5953105552341d0ae1 is removed from metadata 2024.10.22 02:24:24.560238 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:24.566948 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.97 GiB. 2024.10.22 02:24:24.879479 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_747_747_0 (state Temporary) 2024.10.22 02:24:25.000499 [ 1201 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:24:25.041367 [ 1201 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:24:25.058515 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_747_747_0 to 202410_757_757_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:25.063110 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 104089 2024.10.22 02:24:25.203562 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 765 2024.10.22 02:24:25.255630 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:25.259640 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.97 GiB. 2024.10.22 02:24:25.341754 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1554_1554_0 (state Temporary) 2024.10.22 02:24:25.585578 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1554_1554_0 to 202410_1567_1567_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:25.593607 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c6f778c0a3da08910212f49861e1f8d6) : [0, 60157]: Setting download as failed: File segment: [0, 60157], key: c6f778c0a3da08910212f49861e1f8d6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:25.598342 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c6f778c0a3da08910212f49861e1f8d6): No space left in cache to reserve 60158 bytes, will continue without cache download 2024.10.22 02:24:25.647162 [ 1220 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1562_109 to 202410_1563_1563_0 2024.10.22 02:24:25.651389 [ 1220 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.07 MiB using storage policy from min volume index 0 2024.10.22 02:24:25.663295 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104089 2024.10.22 02:24:25.655406 [ 1220 ] {} DiskLocal: Reserved 1.07 MiB on local disk `default`, having unreserved 295.97 GiB. 2024.10.22 02:24:25.680567 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1563_110} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1562_109 to 202410_1563_1563_0 into Compact with storage Full 2024.10.22 02:24:25.832476 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c6f778c0a3da08910212f49861e1f8d6, offset: 0, size: 0 2024.10.22 02:24:25.836226 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c6f778c0a3da08910212f49861e1f8d6 for removal 2024.10.22 02:24:25.839739 [ 1476 ] {} CacheMetadata: Key c6f778c0a3da08910212f49861e1f8d6 is removed from metadata 2024.10.22 02:24:25.849916 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1563_110} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:25.962981 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1563_110} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1562_109, total 24519 rows starting from the beginning of the part 2024.10.22 02:24:26.099207 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1563_110} MergeTreeSequentialSource: Reading 2 marks from part 202410_1563_1563_0, total 47 rows starting from the beginning of the part 2024.10.22 02:24:26.229333 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1338753 2024.10.22 02:24:26.699826 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 4.49 MiB, will set to 1.55 GiB (RSS), difference: -24.63 MiB 2024.10.22 02:24:26.748726 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_1_184_29/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:24:26.934304 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 35000 bytes for 4548a4006d0eb25c4decaf6b43454c5f:0) 2024.10.22 02:24:26.956680 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2bd72bf5824c6a8a7fa3c28a1f9f4ae1, offset: 0, size: 29921 2024.10.22 02:24:26.961007 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2bd72bf5824c6a8a7fa3c28a1f9f4ae1 for removal 2024.10.22 02:24:26.963590 [ 1476 ] {} CacheMetadata: Key 2bd72bf5824c6a8a7fa3c28a1f9f4ae1 is removed from metadata 2024.10.22 02:24:26.964399 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 490aede1fd8e98b6ab612da7b0742697, offset: 0, size: 151 2024.10.22 02:24:26.967705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 490aede1fd8e98b6ab612da7b0742697 for removal 2024.10.22 02:24:26.981730 [ 1476 ] {} CacheMetadata: Key 490aede1fd8e98b6ab612da7b0742697 is removed from metadata 2024.10.22 02:24:27.290549 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1563_110} MergeTask::MergeProjectionsStage: Merge sorted 24566 rows, containing 13 columns (13 merged, 0 gathered) in 1.619767409 sec., 15166.375038479368 rows/sec., 2.75 MiB/sec. 2024.10.22 02:24:27.296521 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:27.300339 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.97 GiB. 2024.10.22 02:24:27.416912 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_883_883_0 (state Temporary) 2024.10.22 02:24:27.465147 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1563_110} MergedBlockOutputStream: filled checksums 202410_1207_1563_110 (state Temporary) 2024.10.22 02:24:27.521485 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_883_883_0 to 202410_890_890_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:27.556400 [ 1207 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_880_164 to 202410_881_886_1 2024.10.22 02:24:27.561517 [ 1207 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.04 MiB using storage policy from min volume index 0 2024.10.22 02:24:27.605638 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1338753 2024.10.22 02:24:27.664150 [ 1207 ] {} DiskLocal: Reserved 1.04 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:27.696798 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 5 file segments (while reserving 1357 bytes for 06652ded8dc6f9dde323755fad57972c:0) 2024.10.22 02:24:27.699618 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4a381f929862350f60cf327a0b4bccaa, offset: 0, size: 156 2024.10.22 02:24:27.705055 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 4a381f929862350f60cf327a0b4bccaa for removal 2024.10.22 02:24:27.708658 [ 1476 ] {} CacheMetadata: Key 4a381f929862350f60cf327a0b4bccaa is removed from metadata 2024.10.22 02:24:27.709555 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ee07104c47219ca89e4659f6c08d4543, offset: 0, size: 183 2024.10.22 02:24:27.713781 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ee07104c47219ca89e4659f6c08d4543 for removal 2024.10.22 02:24:27.716243 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1563_110} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1563_110 to 202410_1207_1563_110 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:27.718349 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_886_165} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_880_164 to 202410_881_886_1 into Wide with storage Full 2024.10.22 02:24:27.720325 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: b83eefc59cfaeeb4225f34e7f5aa18e2, offset: 0, size: 4650 2024.10.22 02:24:27.721711 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1563_110} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1562_109, 202410_1563_1563_0] -> 202410_1207_1563_110 2024.10.22 02:24:27.722203 [ 1476 ] {} CacheMetadata: Key ee07104c47219ca89e4659f6c08d4543 is removed from metadata 2024.10.22 02:24:27.728571 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key b83eefc59cfaeeb4225f34e7f5aa18e2 for removal 2024.10.22 02:24:27.735795 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 254fccf78e55afe97d90fba421cc01e9, offset: 0, size: 155 2024.10.22 02:24:27.740362 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 254fccf78e55afe97d90fba421cc01e9 for removal 2024.10.22 02:24:27.744730 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 9537f6377d38e6597bb78b76f172b90d, offset: 0, size: 158 2024.10.22 02:24:27.748574 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 9537f6377d38e6597bb78b76f172b90d for removal 2024.10.22 02:24:27.753055 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.05 MiB. 2024.10.22 02:24:27.838979 [ 1476 ] {} CacheMetadata: Key 9537f6377d38e6597bb78b76f172b90d is removed from metadata 2024.10.22 02:24:27.848165 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_886_165} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:27.848948 [ 1476 ] {} CacheMetadata: Key 254fccf78e55afe97d90fba421cc01e9 is removed from metadata 2024.10.22 02:24:27.854471 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_886_165} MergeTreeSequentialSource: Reading 165 marks from part 202410_1_880_164, total 1336490 rows starting from the beginning of the part 2024.10.22 02:24:27.921398 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_886_165} MergeTreeSequentialSource: Reading 3 marks from part 202410_881_886_1, total 9855 rows starting from the beginning of the part 2024.10.22 02:24:27.958999 [ 1476 ] {} CacheMetadata: Key b83eefc59cfaeeb4225f34e7f5aa18e2 is removed from metadata 2024.10.22 02:24:28.412285 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 172 entries to flush up to offset 100743 2024.10.22 02:24:28.429808 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column URLDomain 2024.10.22 02:24:28.468000 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(38987a82c8164f6c539a4d874b00eb68) : [0, 126645]: Setting download as failed: File segment: [0, 126645], key: 38987a82c8164f6c539a4d874b00eb68, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:28.469078 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(38987a82c8164f6c539a4d874b00eb68): No space left in cache to reserve 126646 bytes, will continue without cache download 2024.10.22 02:24:28.498256 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 38987a82c8164f6c539a4d874b00eb68, offset: 0, size: 0 2024.10.22 02:24:28.500198 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 38987a82c8164f6c539a4d874b00eb68 for removal 2024.10.22 02:24:28.503367 [ 1476 ] {} CacheMetadata: Key 38987a82c8164f6c539a4d874b00eb68 is removed from metadata 2024.10.22 02:24:28.547589 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column URLDomain 2024.10.22 02:24:28.553745 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column URLDomain 2024.10.22 02:24:28.563397 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column URLDomain 2024.10.22 02:24:28.571603 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column URLDomain 2024.10.22 02:24:28.615431 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column URLDomain 2024.10.22 02:24:28.636567 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column URLDomain 2024.10.22 02:24:28.644306 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column URLDomain 2024.10.22 02:24:28.657812 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column URLDomain 2024.10.22 02:24:28.658484 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:28.661686 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:28.831471 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_907_907_0 (state Temporary) 2024.10.22 02:24:28.938398 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ebx/ipbflbqidqdfxrmhpmlgbjdqdvqig (LogSeriesLimiter: on interval from 2024-10-22 02:24:16 to 2024-10-22 02:24:28 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:24:28.985030 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_907_907_0 to 202410_913_913_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:29.036925 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:24:29.256960 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 100743 2024.10.22 02:24:29.344089 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1463_58, 202410_1464_1464_0, 202410_1465_1465_0, 202410_1466_1466_0, 202410_1467_1467_0, 202410_1468_1468_0] 2024.10.22 02:24:29.347939 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1463_58, 202410_1464_1464_0, 202410_1465_1465_0, 202410_1466_1466_0, 202410_1467_1467_0, 202410_1468_1468_0] 2024.10.22 02:24:29.373660 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1463_58, 202410_1464_1464_0, 202410_1465_1465_0, 202410_1466_1466_0, 202410_1467_1467_0, 202410_1468_1468_0] 2024.10.22 02:24:29.472878 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 5.96 MiB, will set to 1.54 GiB (RSS), difference: 9.80 MiB 2024.10.22 02:24:29.674862 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 104178 2024.10.22 02:24:29.798424 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 24 entries to flush up to offset 14140 2024.10.22 02:24:29.800566 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 8 file segments (while reserving 5598 bytes for ff65dcf2467d0fc5f2d44f7389f49b24:0) 2024.10.22 02:24:29.808748 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4548a4006d0eb25c4decaf6b43454c5f, offset: 0, size: 35000 2024.10.22 02:24:29.815341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4548a4006d0eb25c4decaf6b43454c5f for removal 2024.10.22 02:24:29.818845 [ 1476 ] {} CacheMetadata: Key 4548a4006d0eb25c4decaf6b43454c5f is removed from metadata 2024.10.22 02:24:29.819292 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d50645ca1d8d6e293de3bb17be41d9cf, offset: 0, size: 156 2024.10.22 02:24:29.824825 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d50645ca1d8d6e293de3bb17be41d9cf for removal 2024.10.22 02:24:29.830548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cb9236db9a2b8500643bc4e6253b928d, offset: 0, size: 168 2024.10.22 02:24:29.835996 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cb9236db9a2b8500643bc4e6253b928d for removal 2024.10.22 02:24:29.842100 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a8fe3a4a4e3653e6153e5b625d8eb54f, offset: 0, size: 157 2024.10.22 02:24:29.847671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a8fe3a4a4e3653e6153e5b625d8eb54f for removal 2024.10.22 02:24:29.848808 [ 1476 ] {} CacheMetadata: Key a8fe3a4a4e3653e6153e5b625d8eb54f is removed from metadata 2024.10.22 02:24:29.848964 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2065405f628703ee83ee48c9aa2e36bc, offset: 0, size: 168 2024.10.22 02:24:29.852018 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2065405f628703ee83ee48c9aa2e36bc for removal 2024.10.22 02:24:29.853052 [ 1476 ] {} CacheMetadata: Key cb9236db9a2b8500643bc4e6253b928d is removed from metadata 2024.10.22 02:24:29.858732 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c4a97fc0a47aa4c4626ae0bc53dc1909, offset: 0, size: 156 2024.10.22 02:24:29.860804 [ 1476 ] {} CacheMetadata: Key 2065405f628703ee83ee48c9aa2e36bc is removed from metadata 2024.10.22 02:24:29.862111 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c4a97fc0a47aa4c4626ae0bc53dc1909 for removal 2024.10.22 02:24:29.865228 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 16f2bd80dd0859b5803bf430ca8d669b, offset: 0, size: 156 2024.10.22 02:24:29.870181 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 16f2bd80dd0859b5803bf430ca8d669b for removal 2024.10.22 02:24:29.871044 [ 1476 ] {} CacheMetadata: Key c4a97fc0a47aa4c4626ae0bc53dc1909 is removed from metadata 2024.10.22 02:24:29.873221 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ca1397758d2764f62a1a3667c6ab0463, offset: 0, size: 157 2024.10.22 02:24:29.878885 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ca1397758d2764f62a1a3667c6ab0463 for removal 2024.10.22 02:24:29.886013 [ 1476 ] {} CacheMetadata: Key ca1397758d2764f62a1a3667c6ab0463 is removed from metadata 2024.10.22 02:24:29.893722 [ 1476 ] {} CacheMetadata: Key 16f2bd80dd0859b5803bf430ca8d669b is removed from metadata 2024.10.22 02:24:29.908187 [ 1476 ] {} CacheMetadata: Key d50645ca1d8d6e293de3bb17be41d9cf is removed from metadata 2024.10.22 02:24:29.936085 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:29.939606 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:30.121035 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1555_1555_0 (state Temporary) 2024.10.22 02:24:30.129829 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:30.136809 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:30.167241 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1555_1555_0 to 202410_1568_1568_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:30.181442 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_923_923_0 (state Temporary) 2024.10.22 02:24:30.227900 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1563_110 to 202410_1564_1564_0 2024.10.22 02:24:30.248841 [ 1250 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.08 MiB using storage policy from min volume index 0 2024.10.22 02:24:30.249515 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104178 2024.10.22 02:24:30.251361 [ 1250 ] {} DiskLocal: Reserved 1.08 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:30.264551 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1564_111} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1563_110 to 202410_1564_1564_0 into Compact with storage Full 2024.10.22 02:24:30.345412 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1564_111} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:30.358585 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1564_111} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1563_110, total 24566 rows starting from the beginning of the part 2024.10.22 02:24:30.447160 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_923_923_0 to 202410_936_936_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:30.485540 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_931_296 to 202410_932_932_0 2024.10.22 02:24:30.493684 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:30.498396 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:30.498669 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14140 2024.10.22 02:24:30.544096 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_932_297} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_931_296 to 202410_932_932_0 into Compact with storage Full 2024.10.22 02:24:30.544845 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1564_111} MergeTreeSequentialSource: Reading 2 marks from part 202410_1564_1564_0, total 119 rows starting from the beginning of the part 2024.10.22 02:24:30.568127 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_932_297} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:30.585713 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_932_297} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_931_296, total 14159 rows starting from the beginning of the part 2024.10.22 02:24:30.640293 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_932_297} MergeTreeSequentialSource: Reading 2 marks from part 202410_932_932_0, total 20 rows starting from the beginning of the part 2024.10.22 02:24:31.030353 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 3 file segments (while reserving 33570 bytes for b80447b25315ec22e55f65d03c352b1a:0) 2024.10.22 02:24:31.039381 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b8e8175a6528a98f9f28107b999ef4f1, offset: 0, size: 156 2024.10.22 02:24:31.042444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b8e8175a6528a98f9f28107b999ef4f1 for removal 2024.10.22 02:24:31.044055 [ 1476 ] {} CacheMetadata: Key b8e8175a6528a98f9f28107b999ef4f1 is removed from metadata 2024.10.22 02:24:31.044315 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1baa7b41bf63c57df455af59c098a44f, offset: 0, size: 2633 2024.10.22 02:24:31.053341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1baa7b41bf63c57df455af59c098a44f for removal 2024.10.22 02:24:31.158162 [ 1476 ] {} CacheMetadata: Key 1baa7b41bf63c57df455af59c098a44f is removed from metadata 2024.10.22 02:24:31.159178 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7562d4e93c8f6a8405a9616d09e9946a, offset: 0, size: 168 2024.10.22 02:24:31.169311 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7562d4e93c8f6a8405a9616d09e9946a for removal 2024.10.22 02:24:31.171175 [ 1476 ] {} CacheMetadata: Key 7562d4e93c8f6a8405a9616d09e9946a is removed from metadata 2024.10.22 02:24:31.363967 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:24:31.365236 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:24:31.448279 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 8.66 MiB, will set to 1.55 GiB (RSS), difference: -11.48 MiB 2024.10.22 02:24:31.533080 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:31.540237 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:31.571257 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_872_241, 202410_873_873_0] 2024.10.22 02:24:31.573003 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_872_241, 202410_873_873_0] 2024.10.22 02:24:31.591233 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_872_241, 202410_873_873_0] 2024.10.22 02:24:31.616975 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_932_297} MergeTask::MergeProjectionsStage: Merge sorted 14179 rows, containing 12 columns (12 merged, 0 gathered) in 1.081426254 sec., 13111.388730904624 rows/sec., 2.05 MiB/sec. 2024.10.22 02:24:31.864728 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_932_297} MergedBlockOutputStream: filled checksums 202410_1_932_297 (state Temporary) 2024.10.22 02:24:32.132551 [ 1271 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 02:24:32.248835 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1564_111} MergeTask::MergeProjectionsStage: Merge sorted 24685 rows, containing 13 columns (13 merged, 0 gathered) in 1.991081711 sec., 12397.783508142524 rows/sec., 2.25 MiB/sec. 2024.10.22 02:24:32.256181 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 4 file segments (while reserving 31148 bytes for 585cfdb3b86b1c840c4b62032c8bd6a6:0) 2024.10.22 02:24:32.261407 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b80447b25315ec22e55f65d03c352b1a, offset: 0, size: 33570 2024.10.22 02:24:32.264818 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b80447b25315ec22e55f65d03c352b1a for removal 2024.10.22 02:24:32.266768 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1564_111} MergedBlockOutputStream: filled checksums 202410_1207_1564_111 (state Temporary) 2024.10.22 02:24:32.320514 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_932_297} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_932_297 to 202410_1_932_297 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:32.337775 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_932_297} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_931_296, 202410_932_932_0] -> 202410_1_932_297 2024.10.22 02:24:32.366460 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:24:32.367210 [ 1476 ] {} CacheMetadata: Key b80447b25315ec22e55f65d03c352b1a is removed from metadata 2024.10.22 02:24:32.370621 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: df6c614467d3273b327ff3966ce49f46, offset: 0, size: 156 2024.10.22 02:24:32.373656 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key df6c614467d3273b327ff3966ce49f46 for removal 2024.10.22 02:24:32.378415 [ 1476 ] {} CacheMetadata: Key df6c614467d3273b327ff3966ce49f46 is removed from metadata 2024.10.22 02:24:32.380565 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ff65dcf2467d0fc5f2d44f7389f49b24, offset: 0, size: 5598 2024.10.22 02:24:32.454776 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ff65dcf2467d0fc5f2d44f7389f49b24 for removal 2024.10.22 02:24:32.458928 [ 1476 ] {} CacheMetadata: Key ff65dcf2467d0fc5f2d44f7389f49b24 is removed from metadata 2024.10.22 02:24:32.459601 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 19212206122fce863e8c9f4b308e95c5, offset: 0, size: 156 2024.10.22 02:24:32.463198 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 19212206122fce863e8c9f4b308e95c5 for removal 2024.10.22 02:24:32.467961 [ 1476 ] {} CacheMetadata: Key 19212206122fce863e8c9f4b308e95c5 is removed from metadata 2024.10.22 02:24:32.706833 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 766 2024.10.22 02:24:32.742963 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1564_111} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1564_111 to 202410_1207_1564_111 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:32.748153 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1564_111} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1563_110, 202410_1564_1564_0] -> 202410_1207_1564_111 2024.10.22 02:24:32.788899 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.14 MiB. 2024.10.22 02:24:32.793291 [ 1674 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yxw/rljofxlyeddhfpdhxigqcrfmimghd, size 1086 2024.10.22 02:24:32.799695 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_886_165/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:24:32.922438 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 02:24:32.950463 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 02:24:32.977528 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 02:24:33.009489 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 02:24:33.054690 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 02:24:33.098761 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 02:24:33.129567 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:33.137236 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:33.145415 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParamOrderID 2024.10.22 02:24:33.209488 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_748_748_0 (state Temporary) 2024.10.22 02:24:33.266048 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/nwd/hvuviirdsgvkckgyvinjpbnojrrlw (LogSeriesLimiter: on interval from 2024-10-22 02:24:28 to 2024-10-22 02:24:33 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:24:33.384357 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_748_748_0 to 202410_758_758_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:33.430336 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cef77aaddd5c975ff8fc9a34860ebd56) : [0, 212211]: Setting download as failed: File segment: [0, 212211], key: cef77aaddd5c975ff8fc9a34860ebd56, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:33.435080 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cef77aaddd5c975ff8fc9a34860ebd56): No space left in cache to reserve 212212 bytes, will continue without cache download 2024.10.22 02:24:33.449052 [ 1257 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_753_153 to 202410_758_758_0 2024.10.22 02:24:33.465553 [ 1257 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:33.484755 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:33.507424 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_753_153 to 202410_758_758_0 into Compact with storage Full 2024.10.22 02:24:33.567604 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cef77aaddd5c975ff8fc9a34860ebd56, offset: 0, size: 0 2024.10.22 02:24:33.568794 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cef77aaddd5c975ff8fc9a34860ebd56 for removal 2024.10.22 02:24:33.570943 [ 1476 ] {} CacheMetadata: Key cef77aaddd5c975ff8fc9a34860ebd56 is removed from metadata 2024.10.22 02:24:33.658759 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 766 2024.10.22 02:24:33.670809 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:33.795176 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_753_153, total 797 rows starting from the beginning of the part 2024.10.22 02:24:33.870075 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} MergeTreeSequentialSource: Reading 2 marks from part 202410_754_754_0, total 1 rows starting from the beginning of the part 2024.10.22 02:24:34.046789 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} MergeTreeSequentialSource: Reading 2 marks from part 202410_755_755_0, total 1 rows starting from the beginning of the part 2024.10.22 02:24:34.259743 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 122 entries to flush up to offset 104300 2024.10.22 02:24:34.317569 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} MergeTreeSequentialSource: Reading 2 marks from part 202410_756_756_0, total 1 rows starting from the beginning of the part 2024.10.22 02:24:34.390422 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} MergeTreeSequentialSource: Reading 2 marks from part 202410_757_757_0, total 1 rows starting from the beginning of the part 2024.10.22 02:24:34.457965 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} MergeTreeSequentialSource: Reading 2 marks from part 202410_758_758_0, total 1 rows starting from the beginning of the part 2024.10.22 02:24:34.488605 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 12.12 MiB, will set to 1.55 GiB (RSS), difference: 3.78 MiB 2024.10.22 02:24:34.597319 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1526c6620ddde2d015e889dad3e7aa78) : [0, 141937]: Setting download as failed: File segment: [0, 141937], key: 1526c6620ddde2d015e889dad3e7aa78, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:34.666405 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1339908 2024.10.22 02:24:34.702955 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1526c6620ddde2d015e889dad3e7aa78): No space left in cache to reserve 141938 bytes, will continue without cache download 2024.10.22 02:24:34.783172 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:34.787770 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:34.846177 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1526c6620ddde2d015e889dad3e7aa78, offset: 0, size: 0 2024.10.22 02:24:34.856712 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1526c6620ddde2d015e889dad3e7aa78 for removal 2024.10.22 02:24:34.861622 [ 1476 ] {} CacheMetadata: Key 1526c6620ddde2d015e889dad3e7aa78 is removed from metadata 2024.10.22 02:24:35.130371 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1556_1556_0 (state Temporary) 2024.10.22 02:24:35.203197 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1556_1556_0 to 202410_1569_1569_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:35.244276 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1564_111 to 202410_1569_1569_0 2024.10.22 02:24:35.268892 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.12 MiB using storage policy from min volume index 0 2024.10.22 02:24:35.271827 [ 1223 ] {} DiskLocal: Reserved 1.12 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:35.342642 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1564_111 to 202410_1569_1569_0 into Compact with storage Full 2024.10.22 02:24:35.355505 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104300 2024.10.22 02:24:35.467613 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:35.503403 [ 1224 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 225 ms. 2024.10.22 02:24:35.525227 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_758_154/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:24:35.560159 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:35.563204 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:35.606628 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} MergeTreeSequentialSource: Reading 4 marks from part 202410_1207_1564_111, total 24685 rows starting from the beginning of the part 2024.10.22 02:24:35.675294 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_1565_1565_0, total 93 rows starting from the beginning of the part 2024.10.22 02:24:35.723579 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_884_884_0 (state Temporary) 2024.10.22 02:24:35.838362 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_884_884_0 to 202410_891_891_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:35.872345 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_1566_1566_0, total 74 rows starting from the beginning of the part 2024.10.22 02:24:35.883916 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1339908 2024.10.22 02:24:36.075781 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_1567_1567_0, total 96 rows starting from the beginning of the part 2024.10.22 02:24:36.107407 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e86a84125e747a756f2d6ce45ab811be) : [0, 92070]: Setting download as failed: File segment: [0, 92070], key: e86a84125e747a756f2d6ce45ab811be, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:36.112955 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e86a84125e747a756f2d6ce45ab811be): No space left in cache to reserve 92071 bytes, will continue without cache download 2024.10.22 02:24:36.153420 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e86a84125e747a756f2d6ce45ab811be, offset: 0, size: 0 2024.10.22 02:24:36.155922 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e86a84125e747a756f2d6ce45ab811be for removal 2024.10.22 02:24:36.159086 [ 1476 ] {} CacheMetadata: Key e86a84125e747a756f2d6ce45ab811be is removed from metadata 2024.10.22 02:24:36.247128 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_1568_1568_0, total 89 rows starting from the beginning of the part 2024.10.22 02:24:36.258294 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_185_185_0 (state Temporary) 2024.10.22 02:24:36.279803 [ 1167 ] {} DatabaseCatalog: Execution took 237 ms. 2024.10.22 02:24:36.306528 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} MergeTreeSequentialSource: Reading 2 marks from part 202410_1569_1569_0, total 122 rows starting from the beginning of the part 2024.10.22 02:24:36.437596 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 10.00 MiB, will set to 1.55 GiB (RSS), difference: -8.50 MiB 2024.10.22 02:24:36.880805 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 145 entries to flush up to offset 100888 2024.10.22 02:24:37.003700 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} MergeTask::MergeProjectionsStage: Merge sorted 802 rows, containing 32 columns (32 merged, 0 gathered) in 3.507326604 sec., 228.6641908641594 rows/sec., 17.26 KiB/sec. 2024.10.22 02:24:37.046701 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} MergedBlockOutputStream: filled checksums 202410_1_758_154 (state Temporary) 2024.10.22 02:24:37.305207 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:37.308941 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:37.356865 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_908_908_0 (state Temporary) 2024.10.22 02:24:37.417165 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_908_908_0 to 202410_914_914_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:37.467439 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 100888 2024.10.22 02:24:37.547269 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_758_154 to 202410_1_758_154 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:37.551813 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_758_154} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_753_153, 202410_758_758_0] -> 202410_1_758_154 2024.10.22 02:24:37.590846 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:24:37.725018 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(411eabd0b0311e59352e5500658df944) : [0, 107345]: Setting download as failed: File segment: [0, 107345], key: 411eabd0b0311e59352e5500658df944, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:37.727679 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(411eabd0b0311e59352e5500658df944): No space left in cache to reserve 107346 bytes, will continue without cache download 2024.10.22 02:24:37.759251 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_1_184_29 to 202410_1_184_29 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:37.772667 [ 1397 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_1_184_29} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 7 parts: [202410_1_174_28, 202410_184_184_0] -> 202410_1_184_29 2024.10.22 02:24:37.848256 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 411eabd0b0311e59352e5500658df944, offset: 0, size: 0 2024.10.22 02:24:37.856137 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 411eabd0b0311e59352e5500658df944 for removal 2024.10.22 02:24:37.857659 [ 1476 ] {} CacheMetadata: Key 411eabd0b0311e59352e5500658df944 is removed from metadata 2024.10.22 02:24:38.004369 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 27 entries to flush up to offset 14167 2024.10.22 02:24:38.279610 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:38.285483 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:38.395323 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 70.89 MiB. 2024.10.22 02:24:38.406902 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 7.83 MiB, will set to 1.56 GiB (RSS), difference: 5.78 MiB 2024.10.22 02:24:38.439421 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_924_924_0 (state Temporary) 2024.10.22 02:24:38.614727 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_820_154, 202410_821_826_1] 2024.10.22 02:24:38.623400 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_820_154, 202410_821_826_1] 2024.10.22 02:24:38.648202 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_820_154, 202410_821_826_1] 2024.10.22 02:24:38.768821 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_924_924_0 to 202410_937_937_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:38.821736 [ 1161 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_932_297 to 202410_933_933_0 2024.10.22 02:24:38.827776 [ 1161 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:38.835131 [ 1161 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:38.838171 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14167 2024.10.22 02:24:38.952111 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} MergeTask::MergeProjectionsStage: Merge sorted 25159 rows, containing 13 columns (13 merged, 0 gathered) in 3.635821336 sec., 6919.756961347014 rows/sec., 1.25 MiB/sec. 2024.10.22 02:24:38.954787 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_933_298} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_932_297 to 202410_933_933_0 into Compact with storage Full 2024.10.22 02:24:39.045634 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_933_298} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:39.054763 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_933_298} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_932_297, total 14179 rows starting from the beginning of the part 2024.10.22 02:24:39.087333 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} MergedBlockOutputStream: filled checksums 202410_1207_1569_112 (state Temporary) 2024.10.22 02:24:39.148849 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1569_112 to 202410_1207_1569_112 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:39.158688 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1569_112} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1564_111, 202410_1569_1569_0] -> 202410_1207_1569_112 2024.10.22 02:24:39.195148 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_933_298} MergeTreeSequentialSource: Reading 2 marks from part 202410_933_933_0, total 16 rows starting from the beginning of the part 2024.10.22 02:24:39.196095 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.77 MiB. 2024.10.22 02:24:39.367726 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3f484f90b6292e1d2592a0a564d4385b) : [0, 168755]: Setting download as failed: File segment: [0, 168755], key: 3f484f90b6292e1d2592a0a564d4385b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:39.371604 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 104383 2024.10.22 02:24:39.372660 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3f484f90b6292e1d2592a0a564d4385b): No space left in cache to reserve 168756 bytes, will continue without cache download 2024.10.22 02:24:39.502819 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3f484f90b6292e1d2592a0a564d4385b, offset: 0, size: 0 2024.10.22 02:24:39.505803 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3f484f90b6292e1d2592a0a564d4385b for removal 2024.10.22 02:24:39.516055 [ 1476 ] {} CacheMetadata: Key 3f484f90b6292e1d2592a0a564d4385b is removed from metadata 2024.10.22 02:24:39.664333 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:39.666365 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:39.857688 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1557_1557_0 (state Temporary) 2024.10.22 02:24:39.968830 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_873_242, 202410_874_874_0] 2024.10.22 02:24:39.971099 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_873_242, 202410_874_874_0] 2024.10.22 02:24:40.044743 [ 1182 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:24:40.054429 [ 1182 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:24:40.090500 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_873_242, 202410_874_874_0] 2024.10.22 02:24:40.189423 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1557_1557_0 to 202410_1570_1570_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:40.247443 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104383 2024.10.22 02:24:40.263715 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_933_298/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:24:40.352611 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 10.49 MiB, will set to 1.55 GiB (RSS), difference: 2.46 MiB 2024.10.22 02:24:40.486401 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_933_298} MergeTask::MergeProjectionsStage: Merge sorted 14195 rows, containing 12 columns (12 merged, 0 gathered) in 1.641592452 sec., 8647.091415841865 rows/sec., 1.35 MiB/sec. 2024.10.22 02:24:40.627293 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_933_298} MergedBlockOutputStream: filled checksums 202410_1_933_298 (state Temporary) 2024.10.22 02:24:40.766594 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(58d01533e79300db4f4a609cb3ffbd5a) : [0, 185576]: Setting download as failed: File segment: [0, 185576], key: 58d01533e79300db4f4a609cb3ffbd5a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:40.767772 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(58d01533e79300db4f4a609cb3ffbd5a): No space left in cache to reserve 185577 bytes, will continue without cache download 2024.10.22 02:24:40.786942 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 58d01533e79300db4f4a609cb3ffbd5a, offset: 0, size: 0 2024.10.22 02:24:40.790964 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 58d01533e79300db4f4a609cb3ffbd5a for removal 2024.10.22 02:24:40.797515 [ 1476 ] {} CacheMetadata: Key 58d01533e79300db4f4a609cb3ffbd5a is removed from metadata 2024.10.22 02:24:41.253268 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_933_298} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_933_298 to 202410_1_933_298 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:41.260755 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_933_298} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_932_297, 202410_933_933_0] -> 202410_1_933_298 2024.10.22 02:24:41.364791 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:24:41.436385 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:24:41.436376 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:24:41.650631 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(194c6dcaf69177ac037543aa7bc63a7b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 194c6dcaf69177ac037543aa7bc63a7b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:24:41.652876 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 194c6dcaf69177ac037543aa7bc63a7b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:24:41.655904 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:24:42.574451 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3bcbeed7cdde91ba6fbdd53065487c94) : [0, 102589]: Setting download as failed: File segment: [0, 102589], key: 3bcbeed7cdde91ba6fbdd53065487c94, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:42.577365 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3bcbeed7cdde91ba6fbdd53065487c94): No space left in cache to reserve 102590 bytes, will continue without cache download 2024.10.22 02:24:42.613175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3bcbeed7cdde91ba6fbdd53065487c94, offset: 0, size: 0 2024.10.22 02:24:42.625397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3bcbeed7cdde91ba6fbdd53065487c94 for removal 2024.10.22 02:24:42.629344 [ 1476 ] {} CacheMetadata: Key 3bcbeed7cdde91ba6fbdd53065487c94 is removed from metadata 2024.10.22 02:24:42.889377 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1341448 2024.10.22 02:24:43.353560 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 12.16 MiB, will set to 1.55 GiB (RSS), difference: 2.30 MiB 2024.10.22 02:24:43.641408 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:43.643821 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:43.750592 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_885_885_0 (state Temporary) 2024.10.22 02:24:43.849974 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_885_885_0 to 202410_892_892_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:43.876660 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1341448 2024.10.22 02:24:43.934396 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(79bd790131d91460dfe19c087d90c66b) : [0, 1119800]: Setting download as failed: File segment: [0, 1119800], key: 79bd790131d91460dfe19c087d90c66b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:43.938352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(79bd790131d91460dfe19c087d90c66b): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:24:43.970997 [ 1193 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_887_887_0 to 202410_892_892_0 2024.10.22 02:24:43.994444 [ 1193 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:43.996704 [ 1193 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:44.050603 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_887_887_0 to 202410_892_892_0 into Compact with storage Full 2024.10.22 02:24:44.064291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 79bd790131d91460dfe19c087d90c66b, offset: 0, size: 0 2024.10.22 02:24:44.066143 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 79bd790131d91460dfe19c087d90c66b for removal 2024.10.22 02:24:44.068359 [ 1476 ] {} CacheMetadata: Key 79bd790131d91460dfe19c087d90c66b is removed from metadata 2024.10.22 02:24:44.078186 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:44.099031 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_887_887_0, total 1763 rows starting from the beginning of the part 2024.10.22 02:24:44.137118 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_886_165/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:24:44.253565 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 57 entries to flush up to offset 104440 2024.10.22 02:24:44.455135 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_888_888_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:24:44.471674 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:44.475274 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:44.525282 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1558_1558_0 (state Temporary) 2024.10.22 02:24:44.558978 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_889_889_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:24:44.594803 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1558_1558_0 to 202410_1571_1571_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:44.611641 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_890_890_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:24:44.761693 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104440 2024.10.22 02:24:44.850472 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_891_891_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:24:44.873521 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_892_892_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:24:45.091295 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 81 entries to flush up to offset 100969 2024.10.22 02:24:45.251801 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6a88b92aa9eadea5dc5e9ef602126623) : [0, 55178]: Setting download as failed: File segment: [0, 55178], key: 6a88b92aa9eadea5dc5e9ef602126623, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:45.253380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6a88b92aa9eadea5dc5e9ef602126623): No space left in cache to reserve 55179 bytes, will continue without cache download 2024.10.22 02:24:45.262829 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6a88b92aa9eadea5dc5e9ef602126623, offset: 0, size: 0 2024.10.22 02:24:45.265142 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6a88b92aa9eadea5dc5e9ef602126623 for removal 2024.10.22 02:24:45.268786 [ 1476 ] {} CacheMetadata: Key 6a88b92aa9eadea5dc5e9ef602126623 is removed from metadata 2024.10.22 02:24:45.410911 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:45.415025 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:45.559237 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_909_909_0 (state Temporary) 2024.10.22 02:24:45.641020 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 13.31 MiB, will set to 1.55 GiB (RSS), difference: -1.53 MiB 2024.10.22 02:24:45.726618 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_909_909_0 to 202410_915_915_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:45.809708 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_185_185_0 to 202410_188_188_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:45.868798 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 100969 2024.10.22 02:24:46.341026 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 14183 2024.10.22 02:24:46.544776 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} MergeTask::MergeProjectionsStage: Merge sorted 8693 rows, containing 4 columns (4 merged, 0 gathered) in 2.524312253 sec., 3443.7102579797206 rows/sec., 53.81 KiB/sec. 2024.10.22 02:24:46.561191 [ 1197 ] {} DatabaseCatalog: Execution took 276 ms. 2024.10.22 02:24:46.568724 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1458f3a50b27db5b30bfc364ff47e62d) : [0, 48033]: Setting download as failed: File segment: [0, 48033], key: 1458f3a50b27db5b30bfc364ff47e62d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:46.571206 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1458f3a50b27db5b30bfc364ff47e62d): No space left in cache to reserve 48034 bytes, will continue without cache download 2024.10.22 02:24:46.574928 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} MergedBlockOutputStream: filled checksums 202410_887_892_1 (state Temporary) 2024.10.22 02:24:46.598177 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1458f3a50b27db5b30bfc364ff47e62d, offset: 0, size: 0 2024.10.22 02:24:46.600536 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1458f3a50b27db5b30bfc364ff47e62d for removal 2024.10.22 02:24:46.607466 [ 1476 ] {} CacheMetadata: Key 1458f3a50b27db5b30bfc364ff47e62d is removed from metadata 2024.10.22 02:24:46.647903 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_887_892_1 to 202410_887_892_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:46.656709 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_887_892_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_887_887_0, 202410_892_892_0] -> 202410_887_892_1 2024.10.22 02:24:46.745765 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:46.750411 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:46.822433 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_925_925_0 (state Temporary) 2024.10.22 02:24:46.828550 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:24:46.871082 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_925_925_0 to 202410_938_938_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:47.132273 [ 1207 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_933_298 to 202410_934_934_0 2024.10.22 02:24:47.149659 [ 1207 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:47.152570 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14183 2024.10.22 02:24:47.154558 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:47.192886 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_934_299} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_933_298 to 202410_934_934_0 into Compact with storage Full 2024.10.22 02:24:47.366717 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_934_299} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:47.376435 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_934_299} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_933_298, total 14195 rows starting from the beginning of the part 2024.10.22 02:24:47.444065 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_934_299} MergeTreeSequentialSource: Reading 2 marks from part 202410_934_934_0, total 12 rows starting from the beginning of the part 2024.10.22 02:24:47.906934 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_852_21, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0, 202410_856_856_0, 202410_857_857_0] 2024.10.22 02:24:47.911502 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_852_21, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0, 202410_856_856_0, 202410_857_857_0] 2024.10.22 02:24:47.947240 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_852_21, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0, 202410_856_856_0, 202410_857_857_0] 2024.10.22 02:24:47.953060 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a37ce1df15a8662e88194ff81f0ff9dd) : [0, 91878]: Setting download as failed: File segment: [0, 91878], key: a37ce1df15a8662e88194ff81f0ff9dd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:47.954062 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a37ce1df15a8662e88194ff81f0ff9dd): No space left in cache to reserve 91879 bytes, will continue without cache download 2024.10.22 02:24:47.967971 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a37ce1df15a8662e88194ff81f0ff9dd, offset: 0, size: 0 2024.10.22 02:24:47.969240 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a37ce1df15a8662e88194ff81f0ff9dd for removal 2024.10.22 02:24:47.971698 [ 1476 ] {} CacheMetadata: Key a37ce1df15a8662e88194ff81f0ff9dd is removed from metadata 2024.10.22 02:24:48.291700 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_934_299} MergeTask::MergeProjectionsStage: Merge sorted 14207 rows, containing 12 columns (12 merged, 0 gathered) in 1.108427133 sec., 12817.26112346981 rows/sec., 2.00 MiB/sec. 2024.10.22 02:24:48.528724 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_934_299} MergedBlockOutputStream: filled checksums 202410_1_934_299 (state Temporary) 2024.10.22 02:24:48.566608 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 14.09 MiB, will set to 1.55 GiB (RSS), difference: 9.79 MiB 2024.10.22 02:24:48.661387 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 767 2024.10.22 02:24:48.768048 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 104500 2024.10.22 02:24:48.788019 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7475 2024.10.22 02:24:48.883983 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:48.888517 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:48.991164 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_934_299} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_934_299 to 202410_1_934_299 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:48.993288 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:48.995809 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:48.996781 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_934_299} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_933_298, 202410_934_934_0] -> 202410_1_934_299 2024.10.22 02:24:49.039771 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1559_1559_0 (state Temporary) 2024.10.22 02:24:49.072987 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1559_1559_0 to 202410_1572_1572_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:49.139898 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:24:49.165465 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_749_749_0 (state Temporary) 2024.10.22 02:24:49.167600 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dfc92e23d312aa8ab32f36cdc4448574) : [0, 135971]: Setting download as failed: File segment: [0, 135971], key: dfc92e23d312aa8ab32f36cdc4448574, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:49.169614 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dfc92e23d312aa8ab32f36cdc4448574): No space left in cache to reserve 135972 bytes, will continue without cache download 2024.10.22 02:24:49.253208 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104500 2024.10.22 02:24:49.295165 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dfc92e23d312aa8ab32f36cdc4448574, offset: 0, size: 0 2024.10.22 02:24:49.300379 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dfc92e23d312aa8ab32f36cdc4448574 for removal 2024.10.22 02:24:49.318873 [ 1476 ] {} CacheMetadata: Key dfc92e23d312aa8ab32f36cdc4448574 is removed from metadata 2024.10.22 02:24:49.233794 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_749_749_0 to 202410_759_759_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:49.384980 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.145723076 sec. 2024.10.22 02:24:49.774374 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 767 2024.10.22 02:24:49.974520 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_874_243, 202410_875_875_0] 2024.10.22 02:24:49.977312 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_874_243, 202410_875_875_0] 2024.10.22 02:24:50.004995 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_874_243, 202410_875_875_0] 2024.10.22 02:24:50.884565 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1342603 2024.10.22 02:24:51.063657 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b9368b14d21bcdafbaad5397bcc79023) : [0, 99549]: Setting download as failed: File segment: [0, 99549], key: b9368b14d21bcdafbaad5397bcc79023, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:51.064706 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b9368b14d21bcdafbaad5397bcc79023): No space left in cache to reserve 99550 bytes, will continue without cache download 2024.10.22 02:24:51.101181 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b9368b14d21bcdafbaad5397bcc79023, offset: 0, size: 0 2024.10.22 02:24:51.107670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b9368b14d21bcdafbaad5397bcc79023 for removal 2024.10.22 02:24:51.114936 [ 1476 ] {} CacheMetadata: Key b9368b14d21bcdafbaad5397bcc79023 is removed from metadata 2024.10.22 02:24:51.365611 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:24:51.367776 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:24:51.633821 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:51.641156 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:52.142972 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_886_886_0 (state Temporary) 2024.10.22 02:24:52.275959 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5c361610a6551acada98bcb2b07a8b1a) : [0, 325852]: Setting download as failed: File segment: [0, 325852], key: 5c361610a6551acada98bcb2b07a8b1a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:52.278542 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5c361610a6551acada98bcb2b07a8b1a): No space left in cache to reserve 325853 bytes, will continue without cache download 2024.10.22 02:24:52.323463 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5c361610a6551acada98bcb2b07a8b1a, offset: 0, size: 0 2024.10.22 02:24:52.325147 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5c361610a6551acada98bcb2b07a8b1a for removal 2024.10.22 02:24:52.329332 [ 1476 ] {} CacheMetadata: Key 5c361610a6551acada98bcb2b07a8b1a is removed from metadata 2024.10.22 02:24:52.451341 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_886_886_0 to 202410_893_893_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:52.528833 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1342603 2024.10.22 02:24:52.539765 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1468_59, 202410_1469_1469_0] 2024.10.22 02:24:52.544438 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1468_59, 202410_1469_1469_0] 2024.10.22 02:24:52.554416 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1468_59, 202410_1469_1469_0] 2024.10.22 02:24:53.265813 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 104545 2024.10.22 02:24:53.383654 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 74 entries to flush up to offset 101043 2024.10.22 02:24:53.479612 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8560654ce07678a1a1880bbbd38ee8aa) : [0, 1032588]: Setting download as failed: File segment: [0, 1032588], key: 8560654ce07678a1a1880bbbd38ee8aa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:53.484972 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8560654ce07678a1a1880bbbd38ee8aa): No space left in cache to reserve 1032589 bytes, will continue without cache download 2024.10.22 02:24:53.526814 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8560654ce07678a1a1880bbbd38ee8aa, offset: 0, size: 0 2024.10.22 02:24:53.528667 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8560654ce07678a1a1880bbbd38ee8aa for removal 2024.10.22 02:24:53.532868 [ 1476 ] {} CacheMetadata: Key 8560654ce07678a1a1880bbbd38ee8aa is removed from metadata 2024.10.22 02:24:53.554967 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 16.89 MiB, will set to 1.55 GiB (RSS), difference: 3.14 MiB 2024.10.22 02:24:53.596931 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:53.605315 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:53.653746 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:53.655787 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:53.659314 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1560_1560_0 (state Temporary) 2024.10.22 02:24:53.693800 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_910_910_0 (state Temporary) 2024.10.22 02:24:53.762333 [ 1719 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ebx/ipbflbqidqdfxrmhpmlgbjdqdvqig, size 2261708 2024.10.22 02:24:53.781321 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1560_1560_0 to 202410_1573_1573_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:53.858834 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104545 2024.10.22 02:24:53.948293 [ 1274 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 02:24:54.022145 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_910_910_0 to 202410_916_916_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:54.183145 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 101043 2024.10.22 02:24:54.203373 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RefererDomain 2024.10.22 02:24:54.337259 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RefererDomain 2024.10.22 02:24:54.455219 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RefererDomain 2024.10.22 02:24:54.484060 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RefererDomain 2024.10.22 02:24:54.522779 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RefererDomain 2024.10.22 02:24:54.654482 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 14199 2024.10.22 02:24:54.658272 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RefererDomain 2024.10.22 02:24:54.671728 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RefererDomain 2024.10.22 02:24:54.706672 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RefererDomain 2024.10.22 02:24:54.736488 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RefererDomain 2024.10.22 02:24:54.765667 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(48b7dbddb4b827b559c840a45eceb2c2) : [0, 114680]: Setting download as failed: File segment: [0, 114680], key: 48b7dbddb4b827b559c840a45eceb2c2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:54.767025 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(48b7dbddb4b827b559c840a45eceb2c2): No space left in cache to reserve 114681 bytes, will continue without cache download 2024.10.22 02:24:54.788810 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 48b7dbddb4b827b559c840a45eceb2c2, offset: 0, size: 0 2024.10.22 02:24:54.796646 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 48b7dbddb4b827b559c840a45eceb2c2 for removal 2024.10.22 02:24:54.799798 [ 1476 ] {} CacheMetadata: Key 48b7dbddb4b827b559c840a45eceb2c2 is removed from metadata 2024.10.22 02:24:54.836058 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mrk/xkrgyvtvzvjavubgdtxcnwoqgniqd (LogSeriesLimiter: on interval from 2024-10-22 02:24:33 to 2024-10-22 02:24:54 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:24:54.891095 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:54.894775 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:55.034592 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_926_926_0 (state Temporary) 2024.10.22 02:24:55.056939 [ 1241 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:24:55.119641 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1469_60, 202410_1470_1470_0] 2024.10.22 02:24:55.123871 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1469_60, 202410_1470_1470_0] 2024.10.22 02:24:55.161955 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1469_60, 202410_1470_1470_0] 2024.10.22 02:24:55.197754 [ 1241 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:24:55.408494 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_926_926_0 to 202410_939_939_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:55.433031 [ 1218 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_934_299 to 202410_935_935_0 2024.10.22 02:24:55.444447 [ 1218 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:55.447184 [ 1218 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:55.496189 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 15.75 MiB, will set to 1.55 GiB (RSS), difference: -6.80 MiB 2024.10.22 02:24:55.537929 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_886_165/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:24:55.559931 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14199 2024.10.22 02:24:55.568039 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_935_300} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_934_299 to 202410_935_935_0 into Compact with storage Full 2024.10.22 02:24:55.608906 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 4449 bytes for cfd667f02e53df20ca6178e53ea0dde9:0) 2024.10.22 02:24:55.615356 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 585cfdb3b86b1c840c4b62032c8bd6a6, offset: 0, size: 31148 2024.10.22 02:24:55.656281 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_935_300} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:55.662177 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_935_300} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_934_299, total 14207 rows starting from the beginning of the part 2024.10.22 02:24:55.690950 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_935_300} MergeTreeSequentialSource: Reading 2 marks from part 202410_935_935_0, total 15 rows starting from the beginning of the part 2024.10.22 02:24:55.722321 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 585cfdb3b86b1c840c4b62032c8bd6a6 for removal 2024.10.22 02:24:55.727232 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 1da9bd77494534a01cc5c11fb3d1dc3e, offset: 0, size: 156 2024.10.22 02:24:55.728992 [ 1476 ] {} CacheMetadata: Key 585cfdb3b86b1c840c4b62032c8bd6a6 is removed from metadata 2024.10.22 02:24:55.732418 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 1da9bd77494534a01cc5c11fb3d1dc3e for removal 2024.10.22 02:24:55.862295 [ 1476 ] {} CacheMetadata: Key 1da9bd77494534a01cc5c11fb3d1dc3e is removed from metadata 2024.10.22 02:24:56.356706 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 7515 2024.10.22 02:24:56.357390 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_935_300} MergeTask::MergeProjectionsStage: Merge sorted 14222 rows, containing 12 columns (12 merged, 0 gathered) in 0.893588209 sec., 15915.60839406734 rows/sec., 2.49 MiB/sec. 2024.10.22 02:24:56.407514 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_935_300} MergedBlockOutputStream: filled checksums 202410_1_935_300 (state Temporary) 2024.10.22 02:24:56.673569 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dc624b6ea8c1a641c9f3df22ad84ce8b) : [0, 1121004]: Setting download as failed: File segment: [0, 1121004], key: dc624b6ea8c1a641c9f3df22ad84ce8b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:56.677135 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dc624b6ea8c1a641c9f3df22ad84ce8b): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:24:56.713568 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_935_300} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_935_300 to 202410_1_935_300 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:56.727994 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_935_300} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_934_299, 202410_935_935_0] -> 202410_1_935_300 2024.10.22 02:24:56.730580 [ 1675 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/nwd/hvuviirdsgvkckgyvinjpbnojrrlw, size 1042 2024.10.22 02:24:56.810429 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_875_244, 202410_876_876_0] 2024.10.22 02:24:56.812796 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_875_244, 202410_876_876_0] 2024.10.22 02:24:56.835155 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_875_244, 202410_876_876_0] 2024.10.22 02:24:56.848396 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:24:56.911214 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dc624b6ea8c1a641c9f3df22ad84ce8b, offset: 0, size: 0 2024.10.22 02:24:56.913567 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dc624b6ea8c1a641c9f3df22ad84ce8b for removal 2024.10.22 02:24:56.918737 [ 1476 ] {} CacheMetadata: Key dc624b6ea8c1a641c9f3df22ad84ce8b is removed from metadata 2024.10.22 02:24:57.190608 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 02:24:57.199850 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 02:24:57.232568 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 02:24:57.251143 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 02:24:57.264631 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 02:24:57.280207 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 02:24:57.303075 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParamCurrency 2024.10.22 02:24:57.379313 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 768 2024.10.22 02:24:57.541327 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 16.51 MiB, will set to 1.55 GiB (RSS), difference: -1.46 MiB 2024.10.22 02:24:57.668658 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:57.670769 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.96 GiB. 2024.10.22 02:24:57.820237 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_750_750_0 (state Temporary) 2024.10.22 02:24:57.860879 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(29091cad4e6e8e8f783dc691a24af12c) : [0, 83276]: Setting download as failed: File segment: [0, 83276], key: 29091cad4e6e8e8f783dc691a24af12c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:57.861405 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 104634 2024.10.22 02:24:57.862979 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(29091cad4e6e8e8f783dc691a24af12c): No space left in cache to reserve 83277 bytes, will continue without cache download 2024.10.22 02:24:57.883335 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 29091cad4e6e8e8f783dc691a24af12c, offset: 0, size: 0 2024.10.22 02:24:57.888812 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 29091cad4e6e8e8f783dc691a24af12c for removal 2024.10.22 02:24:57.988641 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_750_750_0 to 202410_760_760_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:57.994858 [ 1476 ] {} CacheMetadata: Key 29091cad4e6e8e8f783dc691a24af12c is removed from metadata 2024.10.22 02:24:58.059102 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 768 2024.10.22 02:24:58.244088 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:24:58.245550 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:58.320786 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1561_1561_0 (state Temporary) 2024.10.22 02:24:58.351386 [ 1204 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 02:24:58.499634 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1561_1561_0 to 202410_1574_1574_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:24:58.566283 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104634 2024.10.22 02:24:58.649277 [ 1237 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1569_112 to 202410_1570_1570_0 2024.10.22 02:24:58.653308 [ 1237 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 02:24:58.658365 [ 1237 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:24:58.706982 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1570_113} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1569_112 to 202410_1570_1570_0 into Compact with storage Full 2024.10.22 02:24:58.752876 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1570_113} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:24:58.864422 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1570_113} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1569_112, total 25159 rows starting from the beginning of the part 2024.10.22 02:24:58.899556 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1570_113} MergeTreeSequentialSource: Reading 2 marks from part 202410_1570_1570_0, total 83 rows starting from the beginning of the part 2024.10.22 02:24:59.526735 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f9886fa58e075defefd91ffbb340b631) : [0, 71709]: Setting download as failed: File segment: [0, 71709], key: f9886fa58e075defefd91ffbb340b631, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:24:59.528619 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f9886fa58e075defefd91ffbb340b631): No space left in cache to reserve 71710 bytes, will continue without cache download 2024.10.22 02:24:59.534619 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1344143 2024.10.22 02:24:59.545380 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 17.75 MiB, will set to 1.55 GiB (RSS), difference: -13.18 MiB 2024.10.22 02:24:59.572200 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f9886fa58e075defefd91ffbb340b631, offset: 0, size: 0 2024.10.22 02:24:59.573762 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f9886fa58e075defefd91ffbb340b631 for removal 2024.10.22 02:24:59.579025 [ 1476 ] {} CacheMetadata: Key f9886fa58e075defefd91ffbb340b631 is removed from metadata 2024.10.22 02:25:00.388898 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:00.395272 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:00.598987 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6f7989fba5b7a39782e6e87bcc72090d) : [0, 69634]: Setting download as failed: File segment: [0, 69634], key: 6f7989fba5b7a39782e6e87bcc72090d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:00.605158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6f7989fba5b7a39782e6e87bcc72090d): No space left in cache to reserve 69635 bytes, will continue without cache download 2024.10.22 02:25:00.634183 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6f7989fba5b7a39782e6e87bcc72090d, offset: 0, size: 0 2024.10.22 02:25:00.636838 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6f7989fba5b7a39782e6e87bcc72090d for removal 2024.10.22 02:25:00.642031 [ 1476 ] {} CacheMetadata: Key 6f7989fba5b7a39782e6e87bcc72090d is removed from metadata 2024.10.22 02:25:00.723207 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_887_887_0 (state Temporary) 2024.10.22 02:25:00.879587 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1570_113} MergeTask::MergeProjectionsStage: Merge sorted 25242 rows, containing 13 columns (13 merged, 0 gathered) in 2.183004125 sec., 11562.964866133727 rows/sec., 2.09 MiB/sec. 2024.10.22 02:25:01.048898 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1570_113} MergedBlockOutputStream: filled checksums 202410_1207_1570_113 (state Temporary) 2024.10.22 02:25:01.097331 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_887_887_0 to 202410_894_894_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:01.142204 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1344143 2024.10.22 02:25:01.336796 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1570_113} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1570_113 to 202410_1207_1570_113 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:01.343934 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1570_113} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1569_112, 202410_1570_1570_0] -> 202410_1207_1570_113 2024.10.22 02:25:01.366344 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:25:01.367725 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:25:01.367995 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.03 MiB. 2024.10.22 02:25:01.799418 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 101 entries to flush up to offset 101144 2024.10.22 02:25:01.813214 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1470_61, 202410_1471_1471_0] 2024.10.22 02:25:01.819969 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1470_61, 202410_1471_1471_0] 2024.10.22 02:25:01.919954 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 14.84 MiB, will set to 1.56 GiB (RSS), difference: 14.35 MiB 2024.10.22 02:25:01.936233 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1470_61, 202410_1471_1471_0] 2024.10.22 02:25:02.042574 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(720c5b361d96eac206cf9ef78a075a2e) : [0, 112135]: Setting download as failed: File segment: [0, 112135], key: 720c5b361d96eac206cf9ef78a075a2e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:02.044136 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(720c5b361d96eac206cf9ef78a075a2e): No space left in cache to reserve 112136 bytes, will continue without cache download 2024.10.22 02:25:02.060595 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 720c5b361d96eac206cf9ef78a075a2e, offset: 0, size: 0 2024.10.22 02:25:02.063356 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 720c5b361d96eac206cf9ef78a075a2e for removal 2024.10.22 02:25:02.066677 [ 1476 ] {} CacheMetadata: Key 720c5b361d96eac206cf9ef78a075a2e is removed from metadata 2024.10.22 02:25:02.324842 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:02.329305 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:02.476435 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_911_911_0 (state Temporary) 2024.10.22 02:25:02.588511 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 104692 2024.10.22 02:25:02.652760 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_911_911_0 to 202410_917_917_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:02.672642 [ 1158 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 3 parts from 202410_747_912_33 to 202410_914_914_0 2024.10.22 02:25:02.678559 [ 1158 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.04 MiB using storage policy from min volume index 0 2024.10.22 02:25:02.699646 [ 1158 ] {} DiskLocal: Reserved 1.04 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:02.750672 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 101144 2024.10.22 02:25:02.753287 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_914_34} MergeTask::PrepareStage: Merging 3 parts: from 202410_747_912_33 to 202410_914_914_0 into Compact with storage Full 2024.10.22 02:25:02.790115 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_914_34} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:02.812933 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_914_34} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_912_33, total 17216 rows starting from the beginning of the part 2024.10.22 02:25:02.834141 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:02.847932 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:02.899546 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1562_1562_0 (state Temporary) 2024.10.22 02:25:02.951459 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_914_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_913_913_0, total 172 rows starting from the beginning of the part 2024.10.22 02:25:03.063816 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 14210 2024.10.22 02:25:03.084807 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_914_34} MergeTreeSequentialSource: Reading 2 marks from part 202410_914_914_0, total 145 rows starting from the beginning of the part 2024.10.22 02:25:03.167563 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:03.172342 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:03.231423 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_927_927_0 (state Temporary) 2024.10.22 02:25:03.250397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b438a6bc0719a03c12512b7f69d117b2) : [0, 47722]: Setting download as failed: File segment: [0, 47722], key: b438a6bc0719a03c12512b7f69d117b2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:03.251567 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b438a6bc0719a03c12512b7f69d117b2): No space left in cache to reserve 47723 bytes, will continue without cache download 2024.10.22 02:25:03.282960 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b438a6bc0719a03c12512b7f69d117b2, offset: 0, size: 0 2024.10.22 02:25:03.294491 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b438a6bc0719a03c12512b7f69d117b2 for removal 2024.10.22 02:25:03.299556 [ 1476 ] {} CacheMetadata: Key b438a6bc0719a03c12512b7f69d117b2 is removed from metadata 2024.10.22 02:25:03.303351 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1562_1562_0 to 202410_1575_1575_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:03.347710 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1570_113 to 202410_1571_1571_0 2024.10.22 02:25:03.363666 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104692 2024.10.22 02:25:03.396045 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_927_927_0 to 202410_940_940_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:03.427416 [ 1180 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_935_300 to 202410_936_936_0 2024.10.22 02:25:03.443795 [ 1183 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 02:25:03.448366 [ 1183 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:03.457675 [ 1180 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:03.459226 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1571_114} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1570_113 to 202410_1571_1571_0 into Compact with storage Full 2024.10.22 02:25:03.461516 [ 1180 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:03.463742 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14210 2024.10.22 02:25:03.547187 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1571_114} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:03.561206 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1571_114} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1570_113, total 25242 rows starting from the beginning of the part 2024.10.22 02:25:03.583557 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 15.15 MiB, will set to 1.55 GiB (RSS), difference: -17.86 MiB 2024.10.22 02:25:03.634757 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_936_301} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_935_300 to 202410_936_936_0 into Compact with storage Full 2024.10.22 02:25:03.708536 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1571_114} MergeTreeSequentialSource: Reading 2 marks from part 202410_1571_1571_0, total 57 rows starting from the beginning of the part 2024.10.22 02:25:03.734424 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_936_301} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:03.752909 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_936_301} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_935_300, total 14222 rows starting from the beginning of the part 2024.10.22 02:25:03.913092 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_936_301} MergeTreeSequentialSource: Reading 2 marks from part 202410_936_936_0, total 24 rows starting from the beginning of the part 2024.10.22 02:25:04.176234 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(ee11a53fba1f0d3293b3b5cf174118f2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ee11a53fba1f0d3293b3b5cf174118f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 02:25:04.280384 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: ee11a53fba1f0d3293b3b5cf174118f2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 02:25:04.286646 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:25:04.619821 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1471_62, 202410_1472_1472_0] 2024.10.22 02:25:04.625760 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1471_62, 202410_1472_1472_0] 2024.10.22 02:25:04.643795 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1471_62, 202410_1472_1472_0] 2024.10.22 02:25:04.691404 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_914_34} MergeTask::MergeProjectionsStage: Merge sorted 17533 rows, containing 13 columns (13 merged, 0 gathered) in 1.945231227 sec., 9013.324357865657 rows/sec., 3.92 MiB/sec. 2024.10.22 02:25:04.750582 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_914_34} MergedBlockOutputStream: filled checksums 202410_747_914_34 (state Temporary) 2024.10.22 02:25:04.860733 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_876_245, 202410_877_877_0] 2024.10.22 02:25:04.862940 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_876_245, 202410_877_877_0] 2024.10.22 02:25:04.872293 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_876_245, 202410_877_877_0] 2024.10.22 02:25:05.016359 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_936_301} MergeTask::MergeProjectionsStage: Merge sorted 14246 rows, containing 12 columns (12 merged, 0 gathered) in 1.502090994 sec., 9484.112518419108 rows/sec., 1.48 MiB/sec. 2024.10.22 02:25:05.171916 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_936_301} MergedBlockOutputStream: filled checksums 202410_1_936_301 (state Temporary) 2024.10.22 02:25:05.379441 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 12.24 MiB, will set to 1.56 GiB (RSS), difference: 5.16 MiB 2024.10.22 02:25:05.447722 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_914_34} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_914_34 to 202410_747_914_34 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:05.449659 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_936_301} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_936_301 to 202410_1_936_301 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:05.452282 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_936_301} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_935_300, 202410_936_936_0] -> 202410_1_936_301 2024.10.22 02:25:05.452383 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_914_34} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 3 parts: [202410_747_912_33, 202410_914_914_0] -> 202410_747_914_34 2024.10.22 02:25:05.488110 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 20.94 MiB. 2024.10.22 02:25:05.511686 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:25:05.513103 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(10524e29e56b31934c2c6c34aed5741c) : [0, 283426]: Setting download as failed: File segment: [0, 283426], key: 10524e29e56b31934c2c6c34aed5741c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:05.521363 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(10524e29e56b31934c2c6c34aed5741c): No space left in cache to reserve 283427 bytes, will continue without cache download 2024.10.22 02:25:05.560703 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 769 2024.10.22 02:25:05.672042 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 10524e29e56b31934c2c6c34aed5741c, offset: 0, size: 0 2024.10.22 02:25:05.675827 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 10524e29e56b31934c2c6c34aed5741c for removal 2024.10.22 02:25:05.679620 [ 1476 ] {} CacheMetadata: Key 10524e29e56b31934c2c6c34aed5741c is removed from metadata 2024.10.22 02:25:05.766819 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:05.768753 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:05.907877 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1571_114} MergeTask::MergeProjectionsStage: Merge sorted 25299 rows, containing 13 columns (13 merged, 0 gathered) in 2.417931358 sec., 10463.076181337981 rows/sec., 1.89 MiB/sec. 2024.10.22 02:25:05.950306 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_751_751_0 (state Temporary) 2024.10.22 02:25:06.091700 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1571_114} MergedBlockOutputStream: filled checksums 202410_1207_1571_114 (state Temporary) 2024.10.22 02:25:06.134774 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_751_751_0 to 202410_761_761_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:06.150723 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1571_114} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1571_114 to 202410_1207_1571_114 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:06.154538 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1571_114} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1570_113, 202410_1571_1571_0] -> 202410_1207_1571_114 2024.10.22 02:25:06.176924 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.05 MiB. 2024.10.22 02:25:06.274342 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_886_165/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:25:06.325714 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 769 2024.10.22 02:25:07.346348 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 12.91 MiB, will set to 1.55 GiB (RSS), difference: 10.09 MiB 2024.10.22 02:25:07.355591 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_886_165} MergeTask::MergeProjectionsStage: Merge sorted 1346345 rows, containing 4 columns (4 merged, 0 gathered) in 39.668808435 sec., 33939.6380459997 rows/sec., 509.09 KiB/sec. 2024.10.22 02:25:07.374414 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 104773 2024.10.22 02:25:07.535851 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_886_165} MergedBlockOutputStream: filled checksums 202410_1_886_165 (state Temporary) 2024.10.22 02:25:07.546824 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2243d7706bd84a19d5055e26a1a6d625) : [0, 223374]: Setting download as failed: File segment: [0, 223374], key: 2243d7706bd84a19d5055e26a1a6d625, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:07.548396 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2243d7706bd84a19d5055e26a1a6d625): No space left in cache to reserve 223375 bytes, will continue without cache download 2024.10.22 02:25:07.563525 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2243d7706bd84a19d5055e26a1a6d625, offset: 0, size: 0 2024.10.22 02:25:07.575144 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2243d7706bd84a19d5055e26a1a6d625 for removal 2024.10.22 02:25:07.578962 [ 1476 ] {} CacheMetadata: Key 2243d7706bd84a19d5055e26a1a6d625 is removed from metadata 2024.10.22 02:25:07.648939 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:07.652479 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:07.724175 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1563_1563_0 (state Temporary) 2024.10.22 02:25:07.866572 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1563_1563_0 to 202410_1576_1576_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:07.880951 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_833_833_0, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0, 202410_837_837_0, 202410_838_838_0] 2024.10.22 02:25:07.883953 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_833_833_0, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0, 202410_837_837_0, 202410_838_838_0] 2024.10.22 02:25:07.896545 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1571_114 to 202410_1572_1572_0 2024.10.22 02:25:07.904590 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 02:25:07.911646 [ 1210 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:07.947088 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_833_833_0, 202410_834_834_0, 202410_835_835_0, 202410_836_836_0, 202410_837_837_0, 202410_838_838_0] 2024.10.22 02:25:07.954511 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1572_115} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1571_114 to 202410_1572_1572_0 into Compact with storage Full 2024.10.22 02:25:08.006298 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1572_115} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:08.016851 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1572_115} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1571_114, total 25299 rows starting from the beginning of the part 2024.10.22 02:25:08.044294 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104773 2024.10.22 02:25:08.146795 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1346068 2024.10.22 02:25:08.158786 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1572_115} MergeTreeSequentialSource: Reading 2 marks from part 202410_1572_1572_0, total 60 rows starting from the beginning of the part 2024.10.22 02:25:08.810363 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:08.814614 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:08.946592 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_888_888_0 (state Temporary) 2024.10.22 02:25:08.961475 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d0988c8e5ce1bbcd5cdf0a935ed5d1af) : [0, 161402]: Setting download as failed: File segment: [0, 161402], key: d0988c8e5ce1bbcd5cdf0a935ed5d1af, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:08.964645 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d0988c8e5ce1bbcd5cdf0a935ed5d1af): No space left in cache to reserve 161403 bytes, will continue without cache download 2024.10.22 02:25:09.047319 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_888_888_0 to 202410_895_895_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:09.095010 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1346068 2024.10.22 02:25:09.111092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d0988c8e5ce1bbcd5cdf0a935ed5d1af, offset: 0, size: 0 2024.10.22 02:25:09.113104 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d0988c8e5ce1bbcd5cdf0a935ed5d1af for removal 2024.10.22 02:25:09.118849 [ 1476 ] {} CacheMetadata: Key d0988c8e5ce1bbcd5cdf0a935ed5d1af is removed from metadata 2024.10.22 02:25:09.358415 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 10.14 MiB, will set to 1.55 GiB (RSS), difference: 3.78 MiB 2024.10.22 02:25:09.957302 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1572_115} MergeTask::MergeProjectionsStage: Merge sorted 25359 rows, containing 13 columns (13 merged, 0 gathered) in 2.011593222 sec., 12606.425455533772 rows/sec., 2.28 MiB/sec. 2024.10.22 02:25:10.024341 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1472_63, 202410_1473_1473_0] 2024.10.22 02:25:10.121092 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1572_115} MergedBlockOutputStream: filled checksums 202410_1207_1572_115 (state Temporary) 2024.10.22 02:25:10.131141 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1472_63, 202410_1473_1473_0] 2024.10.22 02:25:10.141559 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1472_63, 202410_1473_1473_0] 2024.10.22 02:25:10.205462 [ 1229 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:25:10.266794 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 143 entries to flush up to offset 101287 2024.10.22 02:25:10.294642 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1572_115} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1572_115 to 202410_1207_1572_115 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:10.309271 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1572_115} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1571_114, 202410_1572_1572_0] -> 202410_1207_1572_115 2024.10.22 02:25:10.339587 [ 1229 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:25:10.349663 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.05 MiB. 2024.10.22 02:25:10.591961 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:10.596205 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:10.679337 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7f4435eb543556a8566ef798fe366b7c) : [0, 231967]: Setting download as failed: File segment: [0, 231967], key: 7f4435eb543556a8566ef798fe366b7c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:10.681331 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7f4435eb543556a8566ef798fe366b7c): No space left in cache to reserve 231968 bytes, will continue without cache download 2024.10.22 02:25:10.704162 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7f4435eb543556a8566ef798fe366b7c, offset: 0, size: 0 2024.10.22 02:25:10.707503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7f4435eb543556a8566ef798fe366b7c for removal 2024.10.22 02:25:10.713195 [ 1476 ] {} CacheMetadata: Key 7f4435eb543556a8566ef798fe366b7c is removed from metadata 2024.10.22 02:25:10.746033 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_912_912_0 (state Temporary) 2024.10.22 02:25:10.887347 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_912_912_0 to 202410_918_918_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:10.976762 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 101287 2024.10.22 02:25:11.066951 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 14229 2024.10.22 02:25:11.271815 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_708_144, 202410_709_709_0, 202410_710_710_0, 202410_711_711_0, 202410_712_712_0, 202410_713_713_0] 2024.10.22 02:25:11.273492 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_708_144, 202410_709_709_0, 202410_710_710_0, 202410_711_711_0, 202410_712_712_0, 202410_713_713_0] 2024.10.22 02:25:11.366167 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:25:11.368246 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:25:11.412115 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_708_144, 202410_709_709_0, 202410_710_710_0, 202410_711_711_0, 202410_712_712_0, 202410_713_713_0] 2024.10.22 02:25:11.444964 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 10.06 MiB, will set to 1.55 GiB (RSS), difference: 1.18 MiB 2024.10.22 02:25:11.467671 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:11.549964 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:11.562063 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Found 8 old parts to remove. Parts: [201403_17_17_0, 201403_18_18_0, 201403_19_19_0, 201403_20_20_0, 201403_21_21_0, 201403_22_22_0, 201403_23_23_0, 201403_24_24_0] 2024.10.22 02:25:11.565087 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Removing 8 parts from filesystem (serially): Parts: [201403_17_17_0, 201403_18_18_0, 201403_19_19_0, 201403_20_20_0, 201403_21_21_0, 201403_22_22_0, 201403_23_23_0, 201403_24_24_0] 2024.10.22 02:25:11.736565 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_928_928_0 (state Temporary) 2024.10.22 02:25:11.775716 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_928_928_0 to 202410_941_941_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:11.790660 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_886_165} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_886_165 to 202410_1_886_165 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:11.807675 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_886_165} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_880_164, 202410_881_886_1] -> 202410_1_886_165 2024.10.22 02:25:11.813657 [ 1248 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_936_301 to 202410_937_937_0 2024.10.22 02:25:11.838297 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.52 MiB. 2024.10.22 02:25:11.850233 [ 1248 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:11.852730 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:11.884782 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_937_302} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_936_301 to 202410_937_937_0 into Compact with storage Full 2024.10.22 02:25:11.932176 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14229 2024.10.22 02:25:11.944798 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_937_302} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:11.950581 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_937_302} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_936_301, total 14246 rows starting from the beginning of the part 2024.10.22 02:25:11.969685 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_937_302} MergeTreeSequentialSource: Reading 2 marks from part 202410_937_937_0, total 27 rows starting from the beginning of the part 2024.10.22 02:25:12.021254 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(32d5c2e98a020b1f76406382e1785d4c) : [0, 293305]: Setting download as failed: File segment: [0, 293305], key: 32d5c2e98a020b1f76406382e1785d4c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:12.029092 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(32d5c2e98a020b1f76406382e1785d4c): No space left in cache to reserve 293306 bytes, will continue without cache download 2024.10.22 02:25:12.054140 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 104854 2024.10.22 02:25:12.064273 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 32d5c2e98a020b1f76406382e1785d4c, offset: 0, size: 0 2024.10.22 02:25:12.066254 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 32d5c2e98a020b1f76406382e1785d4c for removal 2024.10.22 02:25:12.080835 [ 1476 ] {} CacheMetadata: Key 32d5c2e98a020b1f76406382e1785d4c is removed from metadata 2024.10.22 02:25:12.455924 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_877_246, 202410_878_878_0] 2024.10.22 02:25:12.457562 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_877_246, 202410_878_878_0] 2024.10.22 02:25:12.567665 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_877_246, 202410_878_878_0] 2024.10.22 02:25:12.678721 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:12.684297 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:12.759741 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1564_1564_0 (state Temporary) 2024.10.22 02:25:12.807838 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1564_1564_0 to 202410_1577_1577_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:12.857073 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:12.846149 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1572_115 to 202410_1573_1573_0 2024.10.22 02:25:12.861478 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:12.865093 [ 1215 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 02:25:12.869157 [ 1215 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:12.882547 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1573_116} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1572_115 to 202410_1573_1573_0 into Compact with storage Full 2024.10.22 02:25:12.937264 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1573_116} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:12.954940 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1573_116} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1572_115, total 25359 rows starting from the beginning of the part 2024.10.22 02:25:13.068888 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_937_302} MergeTask::MergeProjectionsStage: Merge sorted 14273 rows, containing 12 columns (12 merged, 0 gathered) in 1.184803007 sec., 12046.728372288813 rows/sec., 1.88 MiB/sec. 2024.10.22 02:25:13.109441 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1573_116} MergeTreeSequentialSource: Reading 2 marks from part 202410_1573_1573_0, total 45 rows starting from the beginning of the part 2024.10.22 02:25:13.227489 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104854 2024.10.22 02:25:13.343241 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_937_302} MergedBlockOutputStream: filled checksums 202410_1_937_302 (state Temporary) 2024.10.22 02:25:13.393626 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 10.14 MiB, will set to 1.55 GiB (RSS), difference: -2.40 MiB 2024.10.22 02:25:13.465771 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(cdf830cabb339271171f69fa5dc94ee7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cdf830cabb339271171f69fa5dc94ee7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 02:25:13.467251 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 41661, file segment info: File segment: [0, 33554431], key: cdf830cabb339271171f69fa5dc94ee7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1389, kind: Regular, unbound: 0 2024.10.22 02:25:13.471799 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:25:13.830824 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 770 2024.10.22 02:25:13.856465 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_937_302} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_937_302 to 202410_1_937_302 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:13.959993 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_937_302} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_936_301, 202410_937_937_0] -> 202410_1_937_302 2024.10.22 02:25:13.994923 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:25:14.110881 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:14.126613 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:14.215307 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d2e5f40d7318de35a44d4af5cee9d1b0) : [0, 380055]: Setting download as failed: File segment: [0, 380055], key: d2e5f40d7318de35a44d4af5cee9d1b0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:14.217593 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d2e5f40d7318de35a44d4af5cee9d1b0): No space left in cache to reserve 380056 bytes, will continue without cache download 2024.10.22 02:25:14.363036 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d2e5f40d7318de35a44d4af5cee9d1b0, offset: 0, size: 0 2024.10.22 02:25:14.367981 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d2e5f40d7318de35a44d4af5cee9d1b0 for removal 2024.10.22 02:25:14.372852 [ 1476 ] {} CacheMetadata: Key d2e5f40d7318de35a44d4af5cee9d1b0 is removed from metadata 2024.10.22 02:25:14.387917 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_752_752_0 (state Temporary) 2024.10.22 02:25:14.791003 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_752_752_0 to 202410_762_762_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:14.866575 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1473_64, 202410_1474_1474_0, 202410_1475_1475_0, 202410_1476_1476_0, 202410_1477_1477_0, 202410_1478_1478_0] 2024.10.22 02:25:14.869493 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1473_64, 202410_1474_1474_0, 202410_1475_1475_0, 202410_1476_1476_0, 202410_1477_1477_0, 202410_1478_1478_0] 2024.10.22 02:25:15.006486 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 02:25:15.016789 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1473_64, 202410_1474_1474_0, 202410_1475_1475_0, 202410_1476_1476_0, 202410_1477_1477_0, 202410_1478_1478_0] 2024.10.22 02:25:15.033180 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 02:25:15.038296 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1573_116} MergeTask::MergeProjectionsStage: Merge sorted 25404 rows, containing 13 columns (13 merged, 0 gathered) in 2.164259706 sec., 11737.96283762629 rows/sec., 2.12 MiB/sec. 2024.10.22 02:25:15.058465 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 02:25:15.110178 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1573_116} MergedBlockOutputStream: filled checksums 202410_1207_1573_116 (state Temporary) 2024.10.22 02:25:15.124013 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 02:25:15.149999 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 02:25:15.171348 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 02:25:15.187677 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 770 2024.10.22 02:25:15.214681 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParamCurrencyID 2024.10.22 02:25:15.441442 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1573_116} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1573_116 to 202410_1207_1573_116 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:15.446825 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1573_116} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1572_115, 202410_1573_1573_0] -> 202410_1207_1573_116 2024.10.22 02:25:15.495954 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vgs/xfjqnpvbnwkntncspfhvmenanjyqu (LogSeriesLimiter: on interval from 2024-10-22 02:24:54 to 2024-10-22 02:25:15 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:25:15.557798 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.05 MiB. 2024.10.22 02:25:15.628038 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_186_186_0 (state Temporary) 2024.10.22 02:25:16.064195 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c1b19662ca001275b8fb09a1efd4da79) : [0, 562955]: Setting download as failed: File segment: [0, 562955], key: c1b19662ca001275b8fb09a1efd4da79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:16.066964 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c1b19662ca001275b8fb09a1efd4da79): No space left in cache to reserve 562956 bytes, will continue without cache download 2024.10.22 02:25:16.099774 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c1b19662ca001275b8fb09a1efd4da79, offset: 0, size: 0 2024.10.22 02:25:16.099943 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1347223 2024.10.22 02:25:16.103808 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c1b19662ca001275b8fb09a1efd4da79 for removal 2024.10.22 02:25:16.108874 [ 1476 ] {} CacheMetadata: Key c1b19662ca001275b8fb09a1efd4da79 is removed from metadata 2024.10.22 02:25:16.686664 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 5.86 MiB, will set to 1.54 GiB (RSS), difference: -12.92 MiB 2024.10.22 02:25:16.823150 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:16.837053 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:16.931883 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_889_889_0 (state Temporary) 2024.10.22 02:25:17.103095 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_889_889_0 to 202410_896_896_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:17.149172 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1347223 2024.10.22 02:25:17.152276 [ 1232 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_886_165 to 202410_887_892_1 2024.10.22 02:25:17.178993 [ 1232 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 02:25:17.181214 [ 1232 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:17.235956 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f983879a67fcefc666498d5c92d68a4e) : [0, 557792]: Setting download as failed: File segment: [0, 557792], key: f983879a67fcefc666498d5c92d68a4e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:17.243217 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 104926 2024.10.22 02:25:17.243810 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f983879a67fcefc666498d5c92d68a4e): No space left in cache to reserve 557793 bytes, will continue without cache download 2024.10.22 02:25:17.367794 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f983879a67fcefc666498d5c92d68a4e, offset: 0, size: 0 2024.10.22 02:25:17.372066 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f983879a67fcefc666498d5c92d68a4e for removal 2024.10.22 02:25:17.376281 [ 1476 ] {} CacheMetadata: Key f983879a67fcefc666498d5c92d68a4e is removed from metadata 2024.10.22 02:25:17.407666 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_892_166} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_886_165 to 202410_887_892_1 into Wide with storage Full 2024.10.22 02:25:17.444417 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_892_166} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:17.452280 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_892_166} MergeTreeSequentialSource: Reading 166 marks from part 202410_1_886_165, total 1346345 rows starting from the beginning of the part 2024.10.22 02:25:17.490910 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_892_166} MergeTreeSequentialSource: Reading 3 marks from part 202410_887_892_1, total 8693 rows starting from the beginning of the part 2024.10.22 02:25:17.560806 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:17.563593 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:17.629225 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1565_1565_0 (state Temporary) 2024.10.22 02:25:17.800573 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1565_1565_0 to 202410_1578_1578_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:17.838671 [ 1154 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1573_116 to 202410_1574_1574_0 2024.10.22 02:25:17.852538 [ 1154 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.12 MiB using storage policy from min volume index 0 2024.10.22 02:25:17.854814 [ 1154 ] {} DiskLocal: Reserved 1.12 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:17.959085 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 104926 2024.10.22 02:25:18.076881 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1574_117} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1573_116 to 202410_1574_1574_0 into Compact with storage Full 2024.10.22 02:25:18.242766 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1574_117} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:18.249576 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1574_117} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1573_116, total 25404 rows starting from the beginning of the part 2024.10.22 02:25:18.264032 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1574_117} MergeTreeSequentialSource: Reading 2 marks from part 202410_1574_1574_0, total 89 rows starting from the beginning of the part 2024.10.22 02:25:18.500168 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 135 entries to flush up to offset 101422 2024.10.22 02:25:18.527361 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 5.50 MiB, will set to 1.54 GiB (RSS), difference: -16.71 MiB 2024.10.22 02:25:18.757752 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:25:18.871655 [ 1679 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mrk/xkrgyvtvzvjavubgdtxcnwoqgniqd, size 3869948 2024.10.22 02:25:18.969113 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:18.971680 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:19.142457 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_913_913_0 (state Temporary) 2024.10.22 02:25:19.252704 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Refresh 2024.10.22 02:25:19.341312 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_913_913_0 to 202410_919_919_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:19.368369 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Refresh 2024.10.22 02:25:19.425260 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Refresh 2024.10.22 02:25:19.435730 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 14244 2024.10.22 02:25:19.471839 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Refresh 2024.10.22 02:25:19.521260 [ 1194 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_914_34 to 202410_915_915_0 2024.10.22 02:25:19.530327 [ 1194 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.04 MiB using storage policy from min volume index 0 2024.10.22 02:25:19.533702 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Refresh 2024.10.22 02:25:19.539124 [ 1194 ] {} DiskLocal: Reserved 1.04 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:19.555595 [ 1194 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 258 ms. 2024.10.22 02:25:19.588554 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_915_35} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_914_34 to 202410_915_915_0 into Compact with storage Full 2024.10.22 02:25:19.643273 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Refresh 2024.10.22 02:25:19.665664 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Refresh 2024.10.22 02:25:19.710744 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Refresh 2024.10.22 02:25:19.722544 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_915_35} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:19.749244 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 101422 2024.10.22 02:25:19.752540 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_915_35} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_914_34, total 17533 rows starting from the beginning of the part 2024.10.22 02:25:19.754356 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Refresh 2024.10.22 02:25:19.831825 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:19.839750 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:19.871545 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_915_35} MergeTreeSequentialSource: Reading 2 marks from part 202410_915_915_0, total 81 rows starting from the beginning of the part 2024.10.22 02:25:19.894823 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_929_929_0 (state Temporary) 2024.10.22 02:25:19.939534 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1574_117} MergeTask::MergeProjectionsStage: Merge sorted 25493 rows, containing 13 columns (13 merged, 0 gathered) in 1.8702773 sec., 13630.599056086496 rows/sec., 2.46 MiB/sec. 2024.10.22 02:25:19.952252 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_186_186_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:25:19.977027 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1574_117} MergedBlockOutputStream: filled checksums 202410_1207_1574_117 (state Temporary) 2024.10.22 02:25:20.212413 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1574_117} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1574_117 to 202410_1207_1574_117 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:20.215008 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1574_117} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1573_116, 202410_1574_1574_0] -> 202410_1207_1574_117 2024.10.22 02:25:20.274999 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_929_929_0 to 202410_942_942_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:20.303102 [ 1206 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_937_302 to 202410_938_938_0 2024.10.22 02:25:20.338425 [ 1206 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:20.343414 [ 1206 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:20.344602 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.08 MiB. 2024.10.22 02:25:20.347900 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14244 2024.10.22 02:25:20.350490 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 6.58 MiB, will set to 1.54 GiB (RSS), difference: -1.90 MiB 2024.10.22 02:25:20.358836 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_938_303} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_937_302 to 202410_938_938_0 into Compact with storage Full 2024.10.22 02:25:20.449051 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_878_247, 202410_879_879_0] 2024.10.22 02:25:20.451290 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_878_247, 202410_879_879_0] 2024.10.22 02:25:20.463498 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_878_247, 202410_879_879_0] 2024.10.22 02:25:20.552060 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_938_303} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:20.571656 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_938_303} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_937_302, total 14273 rows starting from the beginning of the part 2024.10.22 02:25:20.674155 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_938_303} MergeTreeSequentialSource: Reading 2 marks from part 202410_938_938_0, total 16 rows starting from the beginning of the part 2024.10.22 02:25:21.260687 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_915_35} MergeTask::MergeProjectionsStage: Merge sorted 17614 rows, containing 13 columns (13 merged, 0 gathered) in 1.686697811 sec., 10442.890175779092 rows/sec., 4.54 MiB/sec. 2024.10.22 02:25:21.337535 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_915_35} MergedBlockOutputStream: filled checksums 202410_747_915_35 (state Temporary) 2024.10.22 02:25:21.366289 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:25:21.368167 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:25:21.545642 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_915_35} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_915_35 to 202410_747_915_35 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:21.550074 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_915_35} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_914_34, 202410_915_915_0] -> 202410_747_915_35 2024.10.22 02:25:21.560657 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_938_303} MergeTask::MergeProjectionsStage: Merge sorted 14289 rows, containing 12 columns (12 merged, 0 gathered) in 1.209884907 sec., 11810.214275199649 rows/sec., 1.85 MiB/sec. 2024.10.22 02:25:21.577778 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.11 MiB. 2024.10.22 02:25:21.630690 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_938_303} MergedBlockOutputStream: filled checksums 202410_1_938_303 (state Temporary) 2024.10.22 02:25:21.904097 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_938_303} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_938_303 to 202410_1_938_303 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:21.927211 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_938_303} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_937_302, 202410_938_938_0] -> 202410_1_938_303 2024.10.22 02:25:21.955089 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:25:21.966804 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 105010 2024.10.22 02:25:22.149577 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:22.152173 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:22.208333 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1566_1566_0 (state Temporary) 2024.10.22 02:25:22.261464 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1566_1566_0 to 202410_1579_1579_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:22.409005 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1574_117 to 202410_1575_1575_0 2024.10.22 02:25:22.417091 [ 1191 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.12 MiB using storage policy from min volume index 0 2024.10.22 02:25:22.419403 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105010 2024.10.22 02:25:22.424301 [ 1191 ] {} DiskLocal: Reserved 1.12 MiB on local disk `default`, having unreserved 295.95 GiB. 2024.10.22 02:25:22.446654 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1575_118} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1574_117 to 202410_1575_1575_0 into Compact with storage Full 2024.10.22 02:25:22.474669 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1575_118} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:22.489218 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1575_118} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1574_117, total 25493 rows starting from the beginning of the part 2024.10.22 02:25:22.527208 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1575_118} MergeTreeSequentialSource: Reading 2 marks from part 202410_1575_1575_0, total 58 rows starting from the beginning of the part 2024.10.22 02:25:22.532564 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 9.27 MiB, will set to 1.54 GiB (RSS), difference: 11.48 MiB 2024.10.22 02:25:22.689829 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 771 2024.10.22 02:25:22.917015 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_892_166/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:25:23.005622 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:23.010350 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.94 GiB. 2024.10.22 02:25:23.096288 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_753_753_0 (state Temporary) 2024.10.22 02:25:23.169419 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1575_118/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:25:23.173333 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_753_753_0 to 202410_763_763_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:23.215125 [ 1231 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_758_154 to 202410_763_763_0 2024.10.22 02:25:23.242354 [ 1231 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:23.247121 [ 1231 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.94 GiB. 2024.10.22 02:25:23.272899 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_758_154 to 202410_763_763_0 into Compact with storage Full 2024.10.22 02:25:23.560914 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:23.654987 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 771 2024.10.22 02:25:23.688924 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_758_154, total 802 rows starting from the beginning of the part 2024.10.22 02:25:23.770160 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} MergeTreeSequentialSource: Reading 2 marks from part 202410_759_759_0, total 1 rows starting from the beginning of the part 2024.10.22 02:25:23.870114 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} MergeTreeSequentialSource: Reading 2 marks from part 202410_760_760_0, total 1 rows starting from the beginning of the part 2024.10.22 02:25:23.956589 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} MergeTreeSequentialSource: Reading 2 marks from part 202410_761_761_0, total 1 rows starting from the beginning of the part 2024.10.22 02:25:24.073648 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} MergeTreeSequentialSource: Reading 2 marks from part 202410_762_762_0, total 1 rows starting from the beginning of the part 2024.10.22 02:25:24.165301 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1348763 2024.10.22 02:25:24.194576 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(361d96365a4989e8d1dea71312fd466d) : [0, 351757]: Setting download as failed: File segment: [0, 351757], key: 361d96365a4989e8d1dea71312fd466d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:24.198228 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(361d96365a4989e8d1dea71312fd466d): No space left in cache to reserve 351758 bytes, will continue without cache download 2024.10.22 02:25:24.279129 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} MergeTreeSequentialSource: Reading 2 marks from part 202410_763_763_0, total 1 rows starting from the beginning of the part 2024.10.22 02:25:24.358825 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 361d96365a4989e8d1dea71312fd466d, offset: 0, size: 0 2024.10.22 02:25:24.363012 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 361d96365a4989e8d1dea71312fd466d for removal 2024.10.22 02:25:24.365274 [ 1476 ] {} CacheMetadata: Key 361d96365a4989e8d1dea71312fd466d is removed from metadata 2024.10.22 02:25:24.370722 [ 1420 ] {} S3ObjectStorage: Objects with paths [common/asl/kubshwsxvaefatntouilxzncqzbse, common/smc/ejlqxtrdcemmwhzviqaemrnrvttpi, common/lej/guecwahmhncclrbxcltxuyhizberj, common/fhc/zzcvlovkrzabcpnxheokssmkrtvzu, common/kmd/fwslanvaahtmqvcpvlrttinrclfvs, common/qct/egfmqjegkmfobayngiosuxpnbcgiq, common/nei/dtklecjahxndvvxkosvpcudrjwboa, common/gad/kzdjtqkgjdctgbokoeqanaykpkvtn, common/eig/xklkoeusibskjtraejypfskayufzj, common/jqk/nftxkepoxfcldrtdvnpwbbvlzprxd, common/kcz/pjkguwgzdlexcopswclwbdygfyvhm, common/wla/mywzjbnscygqspjmnjvzfphnqyhpd, common/eza/qefqtlinrqdqksvryjhsyvsjzzntz, common/heb/kwmxsfrfotqhkuvuqmukqqakuevjs, common/aet/prjsmfxmlvjbyrsbgwwzgplebtfek, common/acj/fyyzpmbkvyluotfxtytfzgbereqdr, common/pai/rmoasumnsuruqmfeuiuinukldcpcf, common/vyf/uqkxvnfqqruwpklyuaypuxsbwbpag, common/tfx/jswentttnjgpstgfgeksocwjhprim, common/ibz/jklaiahlygmlaqdptmeatviotznwj, common/fey/kkadcdfplfcxscxjhtopmzvntlyrr, common/rns/njnhoakqftmqzhrrvzoeixlbpbylo, common/pez/mzpjubqqqofxwpioqjfbqhwcgwurd, common/mfd/pigplidkvgpblzgrdubrfglbykdeg, common/ehc/vegvratksvmwsxcgjgvjfcfnhmgwe, common/uvq/dqqfaikjzcllbenklwhuicvrocgpw, common/kxv/iftxhdprvilmmpxsprcqtdnvabste, common/ysa/uzmfvknhukgrcyqynqqgafpjqheet, common/zzl/gmnnraesnxnfqohvevxlqbhipqprt, common/iet/vnflzvhqsamsxbgvwwagdcsbhlkzi, common/glw/bjmvwvmkjrxxobnompascvippefya, common/gcz/sajdrzulyidmkesdrjulgevjeofum, common/cin/ttpmeemkfjgbatkhextdcsiadoxqa, common/duh/eqduyitkodbydczpkisnflokvnzbu, common/zmd/uuhphcarkbdzcfvhofbrkejoujpsi, common/peb/tswyjfduedhgzzktnfyyxjxluipwv, common/wzw/xmzfeghhwrocnmnikejwpfpprzmus, common/iau/nkhqtnaervyugbyrksoxvqqwoaeso, common/ehs/orhwkrssvgmefusjkjzeomvnnxynu, common/zub/xjqsilqdpzyercagjpouzmlqmuzgq, common/pek/jfmkyfcyonhiqmdpajoouvabwvszw, common/vlv/kgtjerarnpwukvwmgbjiexlnpzwll, common/kfe/deztwtnvqkxqoaeinphlfyyfbefyv, common/kod/hewjglsfacucdwgawehjswdxokqgf, common/gkl/yspchpcizszlnxidbbqaxhwsycflp, common/urf/mlrjyoeevlftcjxugzsohkukrgzsq, common/gax/rpglqapwlwtzptsowzxlnuuvoisrh, common/njc/oszamrahoomrphmquovjtmsbvljjj, common/usv/dvxoypcanrjacigkwcsqeayaxgacc, common/nsq/wcluszyinytfnuflmdlmbtkoulawr, common/ezs/oqpbqkxbxfqmizanalywlavlrqsmq, common/ece/vhiyuuuwibnohdlwqcalfxvvluysu, common/zxu/xjftdrrfqzuvwinfvpnztwddbatfp, common/loq/bawtuvbiumbjwleqwqqljoeybtteu, common/cxg/ijtxvtinsdrnjmctsidsrueriuobq, common/sfn/yhwexkffynizxwaghhalnsnvgurly, common/gar/dmicyzjtgrnabqmugcoitjxqhqjtz, common/frl/jcyahpukkehwlgpetmwljqijwhdif, common/sig/coaemyypdczixovnwoyhdvnkwgsgy, common/feh/adjxxeuypvdiwlnrafwsooqgmuryx, common/vdm/cvviopspzcelasfcludznjxepashi, common/fbn/mlwfnxynjidtbmaotxcrjajvkhjad, common/pgx/xsuwwnwzgrivuutehbhuhwzmbjtsd, common/oov/vonndoiyyjxcrusmltiigmguoqmox, common/eau/ejnltkutmpltkddiotfydtctpbprm, common/ols/ejyohufinabyhhbfxdwrjlqzkfvfl, common/uhp/nrnygbmyeiorogaodpjkscejkvacc, common/hch/afqjphmogdyzjdtapztcmvodrvaom, common/yek/izerqyvrlsonaosyqysfzamwszkof, common/mag/ynytterpvusnmtuzhoqbzcxoqqrmz, common/edg/fiblgurflaypplgrbcyfecdjgkuzs, common/aya/xcunypbtrdoyyihevkxixrdnpyfiv, common/lrd/dgsylhnksaktsbufuhchqvmoxzvsj, common/bgw/cfvyiufdcbgtmjrtkcpsafmonwlwd, common/xrj/fjqpewgzfhzozggltvxjsffomdagk, common/bwu/zeyrtrtjplrqmrvryplqixvnvbgra, common/jmx/mjdtcpfuyodknlujgtwfhbvdjergc, common/mea/jaxgtzzdfjszohmrbbvfwpawgjtyq, common/jva/oaqtjxhxyvwvedmbgfknpfqzeucor, common/pwt/rvrocoidfgztuxurncsoxqnuwslaz, common/ixy/oxjjwonavfsxcjkvfknyzgevxwcsn, common/rhy/euhrzhurunhafjsnmvcgnoehkznqx, common/zgc/hgbspvdsbxyktlrmkzwikmpwwkcph, common/dxz/xczwuxccgptfqzpqjzayhfopuzqco, common/kkq/vhuxzoccymzxahncjwpjxdesixpkt, common/amu/lywqvrrqosylswssqgqkdagoaszmz, common/mba/huydcoasohrwsksonkuqkginlyabf, common/swe/uwopnyfuqlektatiurtcsbpvynbbu, common/xjm/dzjcszjlzxtlpzctjaklvpvqbpkdk, common/boz/fqhxhwrekllxmqlpllcawmddskhzp, common/oev/vfmawhtnbpypicqjryeohkesqgsbo, common/elr/fhkzpojidfabjsxhcrwyplogsiawu, common/eni/bfszbzdibxkicofwaldpyljumfkjb, common/btx/tznxwxzezgrealdkpseqerxkrvjaj, common/xot/lqspdcqrhdjnrbgdfprxqpzmsgbgg, common/qkc/sgzqkliwojostaeznwhaltyleebhl, common/atw/nkdvziqgpofkbwwopdlatqbrgybvo, common/yay/jhiidaicfiveznanoncrfzjaxkwgr, common/ahc/fukrsdglkfzthgmjgnmcnjdzyafwv, common/ecj/mszecpywhzjkiahyaezoixnklzolv, common/xus/epgbqfxmkhuzxktptoplfvsfnsnee, common/bmd/kcmdegjubuwnaoldzshypebfbalke, common/uaf/raqknprpwkzjtdjpjbysevxmprfvj, common/lln/unzmpvtgtzuvrhglnbrnghgnsbcgd, common/wyi/srskehobpulwqksioflziqxnydjni, common/rpo/mkvnpuycpqhawzapbbmranzmkkacp, common/dtr/ueisyypgylkfanmnvgwrygeyfptlt, common/epe/ktiaplazymzxundbfqqxhbxvhance, common/cpm/pxtfirakdwlaqxtudyefcwuuumuho, common/jpu/mlxzbhfiturhwmdlryovrwndfufrt, common/anx/yttfknrblwlsqftksemwwmxegxalc, common/nzf/ilfxwxzgcpguovkqcukygkolguzik, common/kxr/pqdmjjyyyvhegfoemzmhznwlernmf, common/hkb/aoddilahadumtdfhhhfvqshmvajok, common/tns/wwlyntsegkxlzcsdgjpjdaygywqov, common/lbs/sjwuwbzondhmhhuvfbbvqoqqinxnd, common/mth/tledinnhfjqcjkukyjrabicarjhzc, common/stm/ehofeoyklkawzeinjdmsfmbvkkrbi, common/gmu/gvsfpadxlvhezhyhoyntepzclqknc, common/dxb/fwpgvsxokobaeabizidsqhgycixxn, common/wkb/urvixqccgoerqegvktgzsyyrqoary, common/gzi/kehjqugmmkhabjuaqmfzoshyyvurt, common/vjv/toczcntynisghtgdbhbveeuknplgs, common/thx/lmnudjzpazgnmrctvyqpnmeixcbzf, common/lhz/rkcphjjjejtkcnbeiyqppsquasqov, common/zrp/tuaxwckgeddacwyncptffxejgnxgj, common/iqg/segkuxkmvockdtyifdfcyhbppafda, common/zcx/zbobfcxzguwgfivnmsdemvndmrqbp, common/iuy/twzimqcucbectqfrwmwgkkizcextt, common/gxf/vakqnepawsrlmzksfqjyxglmpkrrs, common/jbf/dfaxsuucrnqsnetykryoufyagtmju, common/ljg/gzwjkxchkggrbnledlpanhekbxrwp, common/bmv/ddhlvrbulqmkpgklkargovtoaloeb, common/wiy/ygghafzbaidyhbncryrlasmanepvt, common/lot/vekxfkvrsstkauirmkitfevdowfqp, common/bex/xnsncvrqyofyzbuoytbidpgzhoxpk, common/uqn/lzqpwllcpvyurcgynqywmbxctjuye, common/ubu/koftnnhbbzrfqupecexlkqoqsqrbc, common/gbt/spfsmrvqkgifhjknsefqnxtafzhvq, common/kyr/hwdsozqprwwzpychkubyyzsdywlps, common/wht/zbzonueucoaqdrugdlyjqgubykodj, common/swx/ivkulsdxtjiablsrcyyxvmqehsncj, common/aat/bvxgqlzhseklavcsnlifchnpjatuc, common/ewy/smiqzivtetfghefksxxtjhmkoqxua, common/fxd/cjvttsoypvuvtziuwckhnohsijdmh, common/gsy/tcduvxoghrjyxleoxjhkmouawrqld, common/dfx/sqsxebauwopqpwkreamujvlidhmfv, common/sxv/azsvvfdhwnzglqmxsnsqeawhkniuv, common/anb/vpekjzmfgepqanzlulzrekktfdwvj, common/wei/bordrhizfhlbjmcpbzmuvdzifrzog, common/hnn/tqroqqigbjrfplhyfiiumxwofmymy, common/fxy/uxyrawyajhasnvaopshebsqphaehm, common/hds/ingzlzbicoefsforhphkivvwwhygu, common/tff/lskpwyygpnkqbahgzqumkkesgyxtr, common/znh/mxrxzivnnnhwacggnpvufwaxqxkof, common/jjr/udjyzksntwlcdfammnibdjdsjsupj, common/jnm/mwykybuxmsnyoedtkkdjhgsjpihhq, common/ial/kgkppuqwkjwdgjywsijlfgkehzvur, common/tnh/kqxqzrqtlvpmctptfcssmyiczkcmn, common/zma/iltdbvosbljhbwpwjnogxfhutpjff, common/jam/chjhxkvwvajweddqrxeiouzirdowc, common/fcq/rnxvuapfwgnfyqhqyrmzinkfuizkk, common/mkf/loaqnbvesfaintlqflsqdusvemovn, common/iug/vimgejyokepqpsooacgaghparimax, common/lme/tsxqsfywicemsdspbxagemapkxglr, common/kon/yrzckeiwgxefcxwncxivhgicmrhjc, common/uiu/qzcyjjoutpvhectaxyjrrwbzqjhim, common/cfu/barjqgylxhtldaaegnjxttaasdenq, common/vjl/klvddnxnbpootyqkruaqbgboguhbp, common/qjv/fmkprcdmukaebmnrbmflmlbsbxdhj, common/gsz/ueihknxtfmerfetabrtukofdismho, common/xsr/nattdgtmiydvhmwcbrhevmvmziatt, common/sxj/hstylplzmazvpyopvkqscqcxzlwfw, common/qbw/gejzyuftmcrbavalfggxkbrhggcku, common/xan/cfvtavtfupzfrtnzgfmgikmkgzotm, common/poy/wxqggbkiymjcmxkvgrspeojipvdca, common/ktd/mnjisyzqxsterzsfsxvzttgxdsqqw, common/ugf/scrwkuypautxdvrokjxayjtgepcdj, common/btd/lcsdowoknjonxrqlfkugjoaucwfru, common/wui/xgygmrhxpqbihkolcmudofhwlgwnq, common/uym/zqfcrflmvtrdelcyypvmejoypchrz, common/mbs/umgsziqqgvsfutededqmofbykngqp, common/kdh/epzrrrxrczcsczebxyhxohdoqyuus, common/hvu/dqomjnueooraqvbcqfrefnakxbzhu, common/tyv/kidulgohaezmwcemdvvfdnyqmsqdh, common/cic/mepiutmylfgywaptrizcmwoiyvzlq, common/rfz/pfjcrlljqrohcgwtxwzzgdgidbudi, common/geo/tahgcnrbatkwikpykfowigqdmvfrf, common/xnq/ddiwlkgvpsravcdjlfnyfeptpdhpq, common/zjs/mbpbuaftlacfurnqervvkrzrqwkpx, common/jzv/kakhufbwiekyzlpzvxphjqdrgdetb, common/vly/kxwoxsosmbvtcvilrrwqoofaudkeg, common/aky/bagbzrodorrnzggxjtyujzidhxryx, common/jcf/arfbwrdswxqyagjevzopavvxhdwgu, common/qyl/injssnxpivamqrpvkvnbdquumktov, common/puq/ylcdcnajfptcrmihdczwacufjstlj, common/nfb/noqmixlpwrqwwzmltpezpqrmamwmc, common/tqs/dzivstcvlxarmosgubyhjqilpbovo, common/mbv/fikynaogqsbeeueojfcnwwhkbtgyl, common/sek/wxvkbafdlzcmfmxelljolybpptqpz, common/yzn/gehvsarrdctdqfxffpatsoususpmc, common/peq/cxfhzcsmkbsgqjwkpdmzyroddlwym, common/fbo/crvcgkadmrcqryyyhwnapfqoeffxd, common/svh/gkrqzbelptpwtdatmjqpvpcybssoy, common/yeu/ansbsaalbeykoygzvcyavuofgulph, common/gcw/qspwhrxmwsitipxckgjiqtqpcbubd, common/xui/xzukqbwaxeowlwrjpqwvuphfmymkv, common/iis/bwototxrizcideimfwvpiomowvajj, common/zbv/ojgorubgdngytcmldcewommrpsrno, common/bvu/uksevjebygghluzmjgyqkmbhxzghf, common/pqr/furfbguansidexeioamzqtjrkuziu, common/oli/qfmopayimertndeqocytjrjnvfejv, common/pmw/syejlucohbbczlddottntbsfwqdxb, common/txs/khfgyqwgzzxskhapvefpmzlbtctvx, common/vmi/bfsjvkcwgyrjnbizslnbwpdewvomi, common/bah/whzofxccjdfhuqssesdpcxnngawxc, common/ccc/bknafiwnlumsircbvurzsijkcbkhv, common/uvy/ezxxhwfobosafjiyeojtvhnavqbvq, common/jsh/qnzvixvfoineahwetyipwrtrlhtnj, common/pwt/sbeukucthahevtnbbxpcucefhwlyx, common/riu/qvhcjkzhxhspvzbilyupszekyyrjm, common/zow/kuwbusvmlygeafilpfodoreunfxxp, common/vyh/dqxsjoayidqeklgwwwgimgzdattse, common/zzn/bjrooyfhfyczzcynwwgsjolrkuyap, common/uib/qhpkwilhkvqlozttwgtlxdbqannfs, common/ilm/rlgwxkttihlnxwcoluyiqgandpgxd, common/liu/qonrekdsxuyvtkaopybnookdpbcyy, common/tvf/ubrmndqwkkbyjbvyjaiifqwjagmta, common/cso/jwprdoobgnfedyorziqyblrgyghmv, common/scf/iprqjbfrtgtnzwokptjsnwgxboxox, common/cfs/coyyfvqzlqbwvdxedqnofvfkxoddh, common/quu/qojmexilgqnxznvjbkqzxfvwgfhle, common/vqy/ttsabwghohdeulmjaqrtpezncqiwv, common/msq/urbxxbltxizgdrvlujdqguoxmwmmf, common/apk/rziahqkcdhewyjfflnoukqdlsruyw, common/rtr/ipukyawkmklrzwdxmpruspnbzqjsy, common/nsb/kailjrexnijqzhtsrpbnjdrakdnql, common/tjd/tywcopyfualptjducexmiyfgvkrrl, common/fko/abtlrjgphqowmhdpkutnsnvferhch, common/ewt/frhxvmkgopqphdkvjetfxqnnkloel, common/ihm/qpqiofbnpqadmxqjbpztkvqbkuzeo, common/xop/uriwxdfuqezzqaxeercfmsxnyscal, common/bsl/mgpanbzbozyyontrwhigxfbbqoiaa, common/pdl/eqgixjetcsdnbixwbfvrwkrayepsk, common/gyl/tlnxpxfavoaehmljwhigahqjilctv, common/nql/wwwpcyrpqoiwavshcsksaqkmrewqi, common/cfd/ehqssxkizhrxizymzbfkzpwbsqrmt, common/xzu/qswmufggzdpthbupdonsvvzxwcnam, common/tfa/oaeutuiecksssoipgzleetstbmtiq, common/ldl/esrtsbidpozzbyklsfyqvjpgxksoh, common/urm/tsncjcwzymawhzuhbxovjocjitzss, common/ahb/asisgqwclvkekmcbhjwthrrxcaojt, common/vkz/xioerzwhgpjfufcjkujjdqnuxyofg, common/psx/swoyglhjnbposwroufctnrtbuzeep, common/bbf/tkhahllzbbgiotaaofhisfjneekao, common/kvh/wimmvvucucflnwreeuepnsflqlieh, common/zjb/iixjdnwpyfhawyqonkpmuzbexgdcl, common/fjq/tdplnnrszlapmoqjybovuiojcvxqf, common/phm/huugnwobubfnioihlodlkvrknyblh, common/cqk/cwhaprruftjiqocvcfdmaweandbji, common/gxo/lwcydvlmmuullzwpozemeutvvqhiy, common/rrn/rzlviyrxufzygiwlrgjgbcwpqozzh, common/ofx/ggbngskegqezyhmixbztyqbgkjzce, common/tcd/xjcezhgpngnpnicltrgxmmkbymabi, common/bsi/qateturggdopzfjsctvmgfsshzwtt, common/hbf/mcjhzeltkljntxudbadmwahwhkyjk, common/fkc/ijlhvewgqglhurymrbvihqozfqewx, common/aps/nomvvcysfgfcwufrrajvqwuiczuaj, common/afg/fihqefemsihuxicglaxallwflpgzv, common/zhx/rvrofvnfwxqputmjncxqvcatniuee, common/qat/vujnftkgetyouuraprrjabkygjkvp, common/yso/ectkqdvfizgtkswopzwyirjfibcul, common/cyb/jzuuirhbmnzqdrulqzsvrnybgarkz, common/mxw/nugehxgdldvhmgipbtvxbilsejbom, common/ogs/oghuevywhqqvjkpsixiqedqpnozcx, common/jio/pwzpevgeeteunkvthqkjsntypupum, common/anf/noeynmpvdazzfprctxptucjqylfln, common/jsq/kxckmrhauvuycdtoqipdlliglgtrq, common/vyn/hkghugbudgjfgaznhflgfqsztmiyt, common/rsa/mjawlwxvunzqpxlkowlibbdcszbdw, common/ctk/bymddygutzwrgvihizydznnvpufco, common/khg/wwgvyecqilcwdphtawogbrbzljxvz, common/gup/qborqxnatidlilrnlfbmzxvnmtspb, common/jhp/ogbilfoetyxpuwgmcdzmxwhprisnk, common/sqs/mckjljohqrklapurvdwruqitigcvf, common/yry/bdpjdchpmdhzyhefadnifxxktpjal, common/hwx/deiatxcyihrrhfzgwnwnrhewurfyi, common/the/cbdxwfjjebmbyuthgsbrrwvwkixbj, common/siq/qgupvvauxaplbgbjnvgkkrmxohvqq, common/pxq/vmyiirjndryiiyegmpkrsfbvgmemb, common/qxd/mnlstzwrnyusfgxnalprskdukprmp, common/frp/bmqpegrbobrxecpqvcczxeatbylzm, common/lza/prjzxlmnjhawljjsrdvladcmofvkn, common/apq/tqkmndmcqffbsgxvxpdktjvpfhgjt, common/mry/nnbkxbohwrtajnjmjuosjqewzaxdu, common/yjt/jntansefmmljfgazzumrugsqkvlzp, common/mxt/ewgjctedzodxosviyuzvfvokgbnwg, common/ntl/zdkewptkduasopfgdlpzrlebofyfq, common/yeb/ldydigwdyueuqoswctykkaxnzjnhi, common/hys/pagteuijcytnjjbolesxxtytspmwo, common/mvg/bidofiunfbfzhcplbejujivmkuzgs, common/mak/rkjhzxrrjhcvdxqblfsobcdvkndbs, common/pga/jbmlyimnxjfldvihlozyoqsiatzsn, common/lnb/hkbcqimvfypnjdbqmeesmqirckohu, common/utf/veormoaczveklqapygcpfuxekrspi, common/rwy/lyswofnexwfoxvprsiiginhcnayll, common/met/tjzybjsqhdmytfroiuiylhtmwhcqe, common/oku/nzcvesinoonxybclyseoyxklrncfs, common/ain/jlrmxfxcdmsipqquxbjnomrjekqam, common/kky/nnsbgdaekutjaxiemvynmpiqzppmd, common/whv/kgdedbwcaypscyhowrzpgxrisfdcx, common/glk/crtheevhesoevlndtvawrdhnputxa, common/ucm/qkxxprxloywpxrxluoemklbvqjatp, common/sno/qzsbrnqgrnuuotaqmkqkoglbfxqoe, common/ugi/hrditupxluhpnrlrmosgfndnemhcp, common/qjf/pyblnyrvhpkxzvggicdmomuqwpeln, common/qsz/swqrlzbdkixvcoiyahxeusslttady, common/xkn/adcwdhonvtnomlyvbpqlxpmnxrtqo, common/cam/mzufmgezddhqdtehcvsihrlbrcegb, common/uuy/amqlslwbblffqcsnrdtdwevnuyvib, common/xqg/jfifuleazvbzknxpswdlobhnbqmpm, common/ygz/eirgqalitvixqurdqqqalrmstnrmw, common/woh/pkdigaslyfmzfjdjkjyfuedzbqxav, common/dbv/ruetgsrzigkrckgqgzrntseljcirx, common/xkj/bqpcghuefwcezeobesbjbtljwspfb, common/ext/xqdswmtobkxfxojpjrmmajxmbieeg, common/pps/jeakirrcopaxcbgrqntsefzceudil, common/xav/hujbmjfajeaxpfuzwhqryvzekduwl, common/zna/kmpstlhctxowfgicvgbabumcvzlxm, common/vmd/upqgarxdtvbwclowyhgitbbcagfvl, common/ssp/tapowfkprefiazpkpwyaugrvzlysi, common/qmg/jsjqilexwnocsaoupepkbmbrietwo, common/mwl/yxizxllyvlhbtoaaeseolandbkdak, common/zcg/mdcsmjazaptcwfdpqqscsyfncwhvu, common/dys/bynavgpdinpbkfvkuxafsxtifqglm, common/tna/pifjsinauraukihexdsdoinzxvklq, common/bto/zsjasympslmeddgfuigfzppfwgfhx, common/hvz/uswxudpyrrhrlnvyugnteomhhjlxq, common/tim/joelshxtjmwtmoapbrgogvydvajhq, common/sxf/oalgbhukobocwhztnzipytfxjjgfa, common/mhp/apbmqpqudlttdwuxyjsrsgoptxjgb, common/iey/bkxvamftyqvyyptmicjhkhsidmniz, common/zig/pzranrnaavezpqjhxyacnvbbbfwjv, common/eyc/isoqqhchdabazwafycnxyhjzfluty, common/wrr/mvcyspyjxuzvpneuethwojahngdck, common/emd/zqugflhayvncznyjrjntopxhnibji, common/gab/nurgynyoijybsidylodsbqkdqtqgu, common/ypu/huwtdzfoekcqotvkaywlkfcuhocbk, common/ubb/pmjuaxyplnhnvryvtigtdvrthkioq, common/efo/kqqixavxivrshkqxrqvwwyacxdhwb, common/wue/gwxmssuiedrbfixbqwwycsdvvafzt, common/nev/ixulfekgfvccsmjdqojzaxmijzgpj, common/exx/omfwxcgazfmlojwksynjcccfxhxuj, common/cps/udocgglxoymeanaytgezhwropmjpu, common/qol/dqfuwrmmxfveuabqzrjqblfypbcho, common/bcb/xvdkqsqeetuqzswwvtwkppsvkdorl, common/mma/ydhdcowosuudffqyhicsaeuehmytf, common/rzn/ydkcicapxbxlbfiaimlseegqoyhfi, common/xqq/fmmqgwagedxcjtjwgzsxrebhfrurr, common/upl/zoivpqkkghgtnmtjmumketpdbibpf, common/bgq/zapjavbfitsyvwtsbhxtuugywnqtg, common/yoi/otbfrghsffeslnywenpiqwqccgixe, common/vzq/wkhwfrirfwakzykdvebqgbyszmexk, common/jia/opzawttvjvimdfiaykvyxakjmarqx, common/mhz/zjytfajfnummlvpsojhaytlsuexly] were removed from S3 2024.10.22 02:25:24.430657 [ 1420 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WithHash.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/WithHash.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_17_17_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:25:24.667432 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1575_118} MergeTask::MergeProjectionsStage: Merge sorted 25551 rows, containing 13 columns (13 merged, 0 gathered) in 2.233558851 sec., 11439.591120941544 rows/sec., 2.07 MiB/sec. 2024.10.22 02:25:24.719641 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1575_118} MergedBlockOutputStream: filled checksums 202410_1207_1575_118 (state Temporary) 2024.10.22 02:25:24.724164 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 9.26 MiB, will set to 1.55 GiB (RSS), difference: -4.30 MiB 2024.10.22 02:25:24.748279 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 02:25:24.767493 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1575_118} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1575_118 to 202410_1207_1575_118 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:24.770109 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1575_118} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1574_117, 202410_1575_1575_0] -> 202410_1207_1575_118 2024.10.22 02:25:24.816900 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.35 MiB. 2024.10.22 02:25:25.050641 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:25.053522 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:25.210581 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_186_186_0 to 202410_189_189_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:25.216371 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_890_890_0 (state Temporary) 2024.10.22 02:25:25.304805 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_890_890_0 to 202410_897_897_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:25.344848 [ 1197 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:25:25.352305 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1348763 2024.10.22 02:25:25.367234 [ 1197 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:25:25.567231 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_763_155/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:25:25.628918 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(24a5a67e9050a3a5ac7f371486dc9c17) : [0, 259831]: Setting download as failed: File segment: [0, 259831], key: 24a5a67e9050a3a5ac7f371486dc9c17, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:25.633740 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(24a5a67e9050a3a5ac7f371486dc9c17): No space left in cache to reserve 259832 bytes, will continue without cache download 2024.10.22 02:25:25.667040 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 24a5a67e9050a3a5ac7f371486dc9c17, offset: 0, size: 0 2024.10.22 02:25:25.668828 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 24a5a67e9050a3a5ac7f371486dc9c17 for removal 2024.10.22 02:25:25.678979 [ 1476 ] {} CacheMetadata: Key 24a5a67e9050a3a5ac7f371486dc9c17 is removed from metadata 2024.10.22 02:25:26.221579 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} MergeTask::MergeProjectionsStage: Merge sorted 807 rows, containing 32 columns (32 merged, 0 gathered) in 2.968208914 sec., 271.8811321513335 rows/sec., 20.51 KiB/sec. 2024.10.22 02:25:26.274312 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} MergedBlockOutputStream: filled checksums 202410_1_763_155 (state Temporary) 2024.10.22 02:25:26.448340 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 105074 2024.10.22 02:25:26.708098 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:26.711792 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:26.871712 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_763_155 to 202410_1_763_155 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:26.877621 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_763_155} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_758_154, 202410_763_763_0] -> 202410_1_763_155 2024.10.22 02:25:26.900408 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1567_1567_0 (state Temporary) 2024.10.22 02:25:27.144464 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:25:27.181164 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1567_1567_0 to 202410_1580_1580_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:27.252335 [ 1269 ] {} DatabaseCatalog: Execution took 222 ms. 2024.10.22 02:25:27.262494 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 216 entries to flush up to offset 101638 2024.10.22 02:25:27.304910 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b1f7cd5aea475bc7e22ad68313aab5dc) : [0, 530534]: Setting download as failed: File segment: [0, 530534], key: b1f7cd5aea475bc7e22ad68313aab5dc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:27.307141 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b1f7cd5aea475bc7e22ad68313aab5dc): No space left in cache to reserve 530535 bytes, will continue without cache download 2024.10.22 02:25:27.331809 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1575_118 to 202410_1580_1580_0 2024.10.22 02:25:27.349448 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.18 MiB using storage policy from min volume index 0 2024.10.22 02:25:27.352756 [ 1227 ] {} DiskLocal: Reserved 1.18 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:27.361820 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b1f7cd5aea475bc7e22ad68313aab5dc, offset: 0, size: 0 2024.10.22 02:25:27.363335 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b1f7cd5aea475bc7e22ad68313aab5dc for removal 2024.10.22 02:25:27.363684 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1575_118 to 202410_1580_1580_0 into Compact with storage Full 2024.10.22 02:25:27.365190 [ 1476 ] {} CacheMetadata: Key b1f7cd5aea475bc7e22ad68313aab5dc is removed from metadata 2024.10.22 02:25:27.376803 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105074 2024.10.22 02:25:27.493235 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 10.45 MiB, will set to 1.55 GiB (RSS), difference: 10.48 MiB 2024.10.22 02:25:27.570730 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:27.616780 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:27.628022 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:27.698560 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1575_118, total 25551 rows starting from the beginning of the part 2024.10.22 02:25:27.708534 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_914_914_0 (state Temporary) 2024.10.22 02:25:27.769364 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_1576_1576_0, total 81 rows starting from the beginning of the part 2024.10.22 02:25:27.788490 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_914_914_0 to 202410_920_920_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:27.844300 [ 1230 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_915_35 to 202410_920_920_0 2024.10.22 02:25:27.850374 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 14270 2024.10.22 02:25:27.858436 [ 1230 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.08 MiB using storage policy from min volume index 0 2024.10.22 02:25:27.860858 [ 1230 ] {} DiskLocal: Reserved 1.08 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:27.863355 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 101638 2024.10.22 02:25:27.869091 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_1577_1577_0, total 81 rows starting from the beginning of the part 2024.10.22 02:25:27.920165 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_915_35 to 202410_920_920_0 into Compact with storage Full 2024.10.22 02:25:27.949338 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_1578_1578_0, total 72 rows starting from the beginning of the part 2024.10.22 02:25:27.967180 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:27.978113 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_915_35, total 17614 rows starting from the beginning of the part 2024.10.22 02:25:27.993395 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_1579_1579_0, total 84 rows starting from the beginning of the part 2024.10.22 02:25:28.050759 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} MergeTreeSequentialSource: Reading 2 marks from part 202410_1580_1580_0, total 64 rows starting from the beginning of the part 2024.10.22 02:25:28.072474 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:28.075614 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:28.107238 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_916_916_0, total 74 rows starting from the beginning of the part 2024.10.22 02:25:28.124837 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_930_930_0 (state Temporary) 2024.10.22 02:25:28.148177 [ 1243 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 02:25:28.148503 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_917_917_0, total 101 rows starting from the beginning of the part 2024.10.22 02:25:28.228698 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_879_248, 202410_880_880_0] 2024.10.22 02:25:28.231299 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_930_930_0 to 202410_943_943_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:28.234543 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_879_248, 202410_880_880_0] 2024.10.22 02:25:28.250457 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_879_248, 202410_880_880_0] 2024.10.22 02:25:28.262769 [ 1233 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_938_303 to 202410_939_939_0 2024.10.22 02:25:28.272137 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14270 2024.10.22 02:25:28.278193 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_918_918_0, total 143 rows starting from the beginning of the part 2024.10.22 02:25:28.279442 [ 1233 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:28.287639 [ 1233 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:28.337567 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_919_919_0, total 135 rows starting from the beginning of the part 2024.10.22 02:25:28.356020 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} MergeTreeSequentialSource: Reading 2 marks from part 202410_920_920_0, total 216 rows starting from the beginning of the part 2024.10.22 02:25:28.430356 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_939_304} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_938_303 to 202410_939_939_0 into Compact with storage Full 2024.10.22 02:25:28.477657 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_939_304} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:28.487774 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_939_304} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_938_303, total 14289 rows starting from the beginning of the part 2024.10.22 02:25:28.525173 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_939_304} MergeTreeSequentialSource: Reading 2 marks from part 202410_939_939_0, total 16 rows starting from the beginning of the part 2024.10.22 02:25:28.672770 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5f59437dbfbc1f5c944fa5c881c8617e) : [0, 490048]: Setting download as failed: File segment: [0, 490048], key: 5f59437dbfbc1f5c944fa5c881c8617e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:28.675290 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5f59437dbfbc1f5c944fa5c881c8617e): No space left in cache to reserve 490049 bytes, will continue without cache download 2024.10.22 02:25:28.851198 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5f59437dbfbc1f5c944fa5c881c8617e, offset: 0, size: 0 2024.10.22 02:25:28.863849 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5f59437dbfbc1f5c944fa5c881c8617e for removal 2024.10.22 02:25:28.865442 [ 1476 ] {} CacheMetadata: Key 5f59437dbfbc1f5c944fa5c881c8617e is removed from metadata 2024.10.22 02:25:29.371312 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_939_304} MergeTask::MergeProjectionsStage: Merge sorted 14305 rows, containing 12 columns (12 merged, 0 gathered) in 1.055498908 sec., 13552.832590898333 rows/sec., 2.12 MiB/sec. 2024.10.22 02:25:29.451908 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_939_304} MergedBlockOutputStream: filled checksums 202410_1_939_304 (state Temporary) 2024.10.22 02:25:29.446194 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7515 2024.10.22 02:25:29.599196 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 11.88 MiB, will set to 1.56 GiB (RSS), difference: -15.90 MiB 2024.10.22 02:25:30.120599 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_920_36/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:25:30.162413 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_857_22, 202410_858_858_0, 202410_859_859_0, 202410_860_860_0, 202410_861_861_0, 202410_862_862_0] 2024.10.22 02:25:30.170036 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_939_304} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_939_304 to 202410_1_939_304 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:30.170790 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_857_22, 202410_858_858_0, 202410_859_859_0, 202410_860_860_0, 202410_861_861_0, 202410_862_862_0] 2024.10.22 02:25:30.173703 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_939_304} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_938_303, 202410_939_939_0] -> 202410_1_939_304 2024.10.22 02:25:30.224064 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_857_22, 202410_858_858_0, 202410_859_859_0, 202410_860_860_0, 202410_861_861_0, 202410_862_862_0] 2024.10.22 02:25:30.248758 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:25:30.888275 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} MergeTask::MergeProjectionsStage: Merge sorted 25933 rows, containing 13 columns (13 merged, 0 gathered) in 3.529423249 sec., 7347.659424906791 rows/sec., 1.34 MiB/sec. 2024.10.22 02:25:30.962552 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} MergedBlockOutputStream: filled checksums 202410_1207_1580_119 (state Temporary) 2024.10.22 02:25:30.981202 [ 1252 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 02:25:31.150391 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1580_119 to 202410_1207_1580_119 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:31.158724 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1580_119} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1575_118, 202410_1580_1580_0] -> 202410_1207_1580_119 2024.10.22 02:25:31.222490 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.65 MiB. 2024.10.22 02:25:31.249503 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} MergeTask::MergeProjectionsStage: Merge sorted 18283 rows, containing 13 columns (13 merged, 0 gathered) in 3.3444051 sec., 5466.74205227112 rows/sec., 2.38 MiB/sec. 2024.10.22 02:25:31.373051 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:25:31.377568 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:25:31.492286 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 105160 2024.10.22 02:25:31.509153 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} MergedBlockOutputStream: filled checksums 202410_747_920_36 (state Temporary) 2024.10.22 02:25:31.600998 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 13.61 MiB, will set to 1.55 GiB (RSS), difference: 15.17 MiB 2024.10.22 02:25:31.658276 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:31.661574 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:31.744748 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1568_1568_0 (state Temporary) 2024.10.22 02:25:31.795696 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1568_1568_0 to 202410_1581_1581_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:31.872960 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105160 2024.10.22 02:25:31.999696 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_920_36 to 202410_747_920_36 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:32.008487 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_920_36} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_915_35, 202410_920_920_0] -> 202410_747_920_36 2024.10.22 02:25:32.058439 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.59 MiB. 2024.10.22 02:25:32.357175 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1350303 2024.10.22 02:25:32.896474 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_826_155, 202410_827_832_1] 2024.10.22 02:25:32.899811 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:32.899877 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_826_155, 202410_827_832_1] 2024.10.22 02:25:32.905346 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:32.922788 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_826_155, 202410_827_832_1] 2024.10.22 02:25:33.154376 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_891_891_0 (state Temporary) 2024.10.22 02:25:33.368871 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 15.35 MiB, will set to 1.55 GiB (RSS), difference: -4.64 MiB 2024.10.22 02:25:33.469555 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_891_891_0 to 202410_898_898_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:33.508280 [ 1223 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_893_893_0 to 202410_898_898_0 2024.10.22 02:25:33.514192 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1350303 2024.10.22 02:25:33.521481 [ 1223 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:33.545814 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:33.592625 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_893_893_0 to 202410_898_898_0 into Compact with storage Full 2024.10.22 02:25:33.747969 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:33.853410 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_893_893_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:25:33.966517 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_894_894_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:25:33.992849 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_895_895_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:25:34.027823 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_896_896_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:25:34.148787 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_897_897_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:25:34.164639 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_898_898_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:25:34.304709 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 16.02 MiB, will set to 1.55 GiB (RSS), difference: 8.35 MiB 2024.10.22 02:25:34.764109 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_892_166/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:25:35.572135 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(88c54b385359da648b8edb881b93f309) : [0, 229787]: Setting download as failed: File segment: [0, 229787], key: 88c54b385359da648b8edb881b93f309, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:35.574234 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(88c54b385359da648b8edb881b93f309): No space left in cache to reserve 229788 bytes, will continue without cache download 2024.10.22 02:25:35.629437 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 88c54b385359da648b8edb881b93f309, offset: 0, size: 0 2024.10.22 02:25:35.636310 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 14.25 MiB, will set to 1.55 GiB (RSS), difference: -6.32 MiB 2024.10.22 02:25:35.636305 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 88c54b385359da648b8edb881b93f309 for removal 2024.10.22 02:25:35.642400 [ 1476 ] {} CacheMetadata: Key 88c54b385359da648b8edb881b93f309 is removed from metadata 2024.10.22 02:25:35.708457 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 126 entries to flush up to offset 101764 2024.10.22 02:25:35.715694 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} MergeTask::MergeProjectionsStage: Merge sorted 8855 rows, containing 4 columns (4 merged, 0 gathered) in 2.122922326 sec., 4171.137064955433 rows/sec., 65.17 KiB/sec. 2024.10.22 02:25:35.767867 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} MergedBlockOutputStream: filled checksums 202410_893_898_1 (state Temporary) 2024.10.22 02:25:35.825085 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_893_898_1 to 202410_893_898_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:35.849635 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_893_898_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_893_893_0, 202410_898_898_0] -> 202410_893_898_1 2024.10.22 02:25:35.866521 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 14290 2024.10.22 02:25:35.891345 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 46 entries to flush up to offset 105206 2024.10.22 02:25:35.902930 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:25:35.967537 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:35.970174 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:36.063646 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:36.064934 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:36.106813 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_931_931_0 (state Temporary) 2024.10.22 02:25:36.128671 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_915_915_0 (state Temporary) 2024.10.22 02:25:36.243324 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:36.258595 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:36.262716 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_931_931_0 to 202410_944_944_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:36.312185 [ 1204 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_939_304 to 202410_940_940_0 2024.10.22 02:25:36.320953 [ 1204 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:36.326652 [ 1204 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 295.99 GiB. 2024.10.22 02:25:36.339291 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14290 2024.10.22 02:25:36.394343 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_915_915_0 to 202410_921_921_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:36.436444 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 101764 2024.10.22 02:25:36.436948 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1569_1569_0 (state Temporary) 2024.10.22 02:25:36.448336 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_940_305} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_939_304 to 202410_940_940_0 into Compact with storage Full 2024.10.22 02:25:36.472209 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1569_1569_0 to 202410_1582_1582_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:36.472159 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_880_249, 202410_881_881_0] 2024.10.22 02:25:36.477311 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_880_249, 202410_881_881_0] 2024.10.22 02:25:36.492553 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_880_249, 202410_881_881_0] 2024.10.22 02:25:36.550939 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105206 2024.10.22 02:25:36.608620 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_940_305} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:36.638916 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_940_305} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_939_304, total 14305 rows starting from the beginning of the part 2024.10.22 02:25:36.772563 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_940_305} MergeTreeSequentialSource: Reading 2 marks from part 202410_940_940_0, total 11 rows starting from the beginning of the part 2024.10.22 02:25:36.893457 [ 1420 ] {} S3ObjectStorage: Objects with paths [common/ojf/oozwgeucjntvqfjveuetughmwdvme, common/mhc/hbqjgrwwhlkkgdyhrdwrhselzloid, common/csr/qnelxhathoeoovwqsfdqivngpvekd, common/wqc/ytetddmvxwniuddexdkbuxsscxqgq, common/bch/vsbywdncwhhybsebrfxzmzaqtieui, common/tma/nfkheddhzvwnfdmwficssfbemyhch, common/vac/agzgabqeiczrptruunfdzhautrbdj, common/xqg/bltpkoygrmsyozfrrwtdsucqkgqbv, common/lhx/sadmipkeizvmnnonsbsayczaokoyg, common/sss/mzpqevqlsougjddxmrofvvumqvhns, common/tyi/dhsoibudfbiwidusciwuzojaitlcf, common/mbf/aqykzaviclbgcrruxvusowxspbivi, common/dyr/zfxkdoucnygaiaqadzzjvyyllnhah, common/sab/bnaauzmswagflkskhawcqjwsuarsf, common/dqt/ielxhvjhmkgdslybmvdghjzlzweyg, common/vtp/lzlqqytwqnlsfsvhnguahauybgkgc, common/nim/jizhevtqjbssqkyvptfjzukxovtty, common/cfm/yttxqfqjmzppzedbwzclzmgmpdypj, common/yle/uefnfrvhhckzgtrzhacokbbubduur, common/vmo/zfejizudsfqvbjewxwyviimtupapl, common/uhe/xkhoensdgarfndjwmqtvzabchbfge, common/bsp/oduleoefqscmrwbbusmomzfwrocrc, common/gvv/nmdgipcygecmfxekrosxxrpcmqlii, common/zey/rfxkolpzdemcyicjvochahmdclwaf, common/rjs/ivtzoevtsoyydrvqnydqysylfzvdl, common/nfs/znrmnfodunztnjvlexlfzpguclsgq, common/hkt/sgqcqsukcowqlgxhjjyjbnzxsomwn, common/vga/tikmkawflarenqipklpujmnocjmfv, common/gpc/yfznydzqooetkxggxmtqlsoocnhnp, common/ban/gpxixevtdsnzreyvnxxpsmwvcaqmi, common/bqw/vskjjtirepsuxjyhjtclsckviluql, common/mdn/lvyjifszeyncdujjipvnqtpzbrqqd, common/ebt/qbmwsitndzyduqatunhqwalrkjukf, common/plt/bgwokzsybqriuefvjailbfdwmwxum, common/pak/vluugttqrebezzhzkrgitoajjoils, common/vub/fqhiiopooizqwmviebdnocpelzrye, common/kjt/ghprlilfbwwnrnrdpfpmcbrhncxrf, common/pgg/vpwggokndydkcspfxwpkfkbzdfkjg, common/ryl/crenkygzewemkcbltladvijyvovfc, common/hky/egrfduqulxrmqwzvgiekmpltpyfkk, common/pcp/wvzxzowuvbwwwtwfngkynxehdvmbb, common/klg/jjvjnujcckfpkxzpxfxydfrszeebc, common/vum/oamsunixdwxaygibvrlhaxzwwlfau, common/blh/wpwziriuijqctuhwmclzikxgpwpeg, common/jir/gfzuyqedryvvtiqalhdturewvdvuz, common/vjp/gnsukumbtojqmrarnlxfpzsykylms, common/opg/miohazzqdimfggcqwvdsgodhjmpln, common/tqa/ikydnarmgxekudwyksqjkucwyjssk, common/gkt/yvsxxzkwtprjwavhfkqcgqzrqoiak, common/crj/etbrgzoajebrytjogerzedbevylyb, common/pyf/mxwysvycoxderlcawqxnxkjtvvotl, common/oum/licfaaciqleojrnmmumboevidlfeh, common/usy/jyrpznqgmoallzypprkuwfcvjpiqe, common/kar/smgurrtookcypjhswhhweudhgvlwr, common/gme/ukphnwfnujsuvnwhgbzmzoxwqemtf, common/ptl/rocmhfmvtxvwdfpvtvzrcohkjuhkb, common/hrz/kdjydfhwripioikjvepozujdtfuth, common/oci/hbrqbtkflmiduntzsnyjcgrdeilly, common/zhs/boneeunbxmryaokiwubqmanhtmhma, common/gxe/ytrnvspqxyumaskecwcqqhxobmcsv, common/ybw/uyrwvagwdihvcauisgooxrqszavrn, common/btb/kspaksthvottsfwtfzyvupnntqkiy, common/xmm/cymwcxlvlkupmiizsyasvgkwoyeum, common/yan/rscegqfktcqhvpfrrclgcaltkfonn, common/wbl/ztdeykldtyxcwsrtannncmrofsrph, common/jhm/wloplxpxaqsciookwlhvqawsmkjie, common/ccy/rxwlqlfeozldbzkoebclscpjzlytl, common/hxx/dokdkdgcmptccnpsqmlfdrptqxnzi, common/sqx/diksuucfzdudlpywseabvwlbqvkge, common/wjh/dgxxccfkayaxptrbiilifnsuensms, common/nsn/ysikrrnvaeanermjsinaijzcrcpcf, common/stb/jwnugkqgbydhsjpfazpwlfuillhli, common/krl/emsreholojtbcularirravsrwmvcu, common/tya/aghyelbpxgyocgtzjnncvlxxizgnx, common/rnz/nzgalwpxmrkbpwjqrazpyavymvncc, common/uig/qatsqhqamehtkuvrlkkbhjirfprym, common/rsl/nifedyibirgqzbzyrxqxlsngnmcua, common/jlv/honyifzgurkidleuzdltnfwtsceyy, common/egd/nlzhqpvhercdknwbpxhrsioaennkz, common/vcz/hbminapzxmldntrlmzgxzauutwnrx, common/oxd/sxjwscakcwksgdqyqhfgbwpuuezrh, common/hso/bgpmabevmhkmtlepununjnkmkykkw, common/nfb/xwyydnbfdxegnoakulvfkykpdfopx, common/otq/rbtemgdplkaoautahgobsycewydxn, common/sqm/dosdmmxtublwzwavxyjakkkjeizzh, common/pth/boaqrafjdbxmgvsweafsddmweyrco, common/tny/yqbqlvjiwkxvkltbmoqcgcqjteekf, common/ady/wmbvkzaemribajxwibbohydzaowjk, common/ozg/vpzckhddjkuqgujranpimjxwqpcbk, common/tqa/tjhzmppyhwycszhmcgxvtutjucrcb, common/lsu/kuyriygygltcxnourptyzukaepoih, common/meh/kyhdiqwtxvlduynrgkkxryzyvrlxp, common/mun/ktvgsnsaqqgokkckdxobvmmdcjdzk, common/cgg/cvwoaumiphzkqgqyhksmfszivtqik, common/uhg/nqriaolkxczpdmsbovfpltgnrwkvu, common/uup/geggheghmngebamiwbykgklyrppob, common/neb/yjyyjhmieuwbrovopgyyddroueish, common/erz/lmzapeexfvsfihoomlvnbixrxmbwd, common/jru/pubvtmhiffyouxkbsfuqocwpywtfv, common/phx/igiwlmwhamxfgxkuvkiirialpmdbi, common/nxu/grbkwyoivyhhdacezzmikdgqazwrq, common/hra/utrktylcbtoywlrjferppiywdysag, common/htq/hcacjxxqlohohufvrujxyynoezusz, common/mqd/gxtomcffxydlfehcpeofmewgcmzbr, common/fpa/ygxfjhetlmnzrmlvcfheyuwfkhect, common/vli/idmbaiivpgvlntruenmllmtpvethn, common/lbo/hjndexfhaftmmwzpnkrijbleoztgh, common/tqg/hgdiuebkerqkaaxisrdscjffqksiy, common/bwl/ygatesfygtxzdlcsyuefeipotyvrd, common/etz/gxyknhuskxsxnzdymujohmfigvyff, common/gnx/gvnbycgnczhdgggtwatqrjawrwejp, common/eov/soqjgstikubnxpyijnguddydfxabz, common/zkz/zgvymumhyulrvanzvjrjtwocwqzuv, common/pks/cbeymmlohszjpqbfkhpvrethggkqk, common/kys/ngerfefvvwuqhewdixrvtmvnkdcrp, common/chy/ucznwqqwzwjphqnmcufkbhtwqvqfo, common/qzl/ayeuqilmjrfhjsjjmeggnvnupbrxv, common/lte/ftbaszuggmfewxadrewnplmsnhfsr, common/guo/ewgnghewbmvapomqecunduedfrgwk, common/lez/yrajklzawpaclmttfhsbdopgtbcbu, common/buv/hybyclamugtmagyizyywwjcyhpckn, common/kgq/upuquorpkisnmyctndliqiojmrcml, common/mku/ohkvfdvtkyqhmpkubriljebhbakhc, common/qzx/uoomstzjsowligoqvkbxcruoopbtf, common/fjt/gxrrofsgiyvudphxgfnjkakjubfcx, common/nkl/vfhhwyclimphdhksntsyoqxolrqph, common/mtq/imufvsimvqhkaeneeiqzehlykmivi, common/pua/kqokrpiuyrulzswhhxmvrrlvlunqn, common/pxt/naaioedlxbjyiqngbygpvowptmezq, common/kuh/pmbyxkefjhxcjezgtttpxuzlnfoxn, common/uzk/jjjdgygwymjmjjdnvzzyzyhvgdakq, common/eha/bbkuzxpntwpzhwgkvuntfupsxxtkn, common/wdp/ghgxyzqcvvapzjskrzksjqvchzvai, common/kpe/vtlexplpuhfoytbubilqjiraidiht, common/rwj/nljpdbdugajfexeymgypgzithhhgn, common/jqt/seboqlumkqbrymhcsysxlswvwerjo, common/bdn/kxdvgxxywobxjtubhoxqmffstcbdq, common/nbc/qqriaqqxzlkenwzpwfynoobbdixxz, common/rao/hgujfhnrwmlpuwknzvmuwfcrauxfv, common/zoq/xuvmzpqwpxdxxvvukhlogtzlyiqwi, common/xfr/rdltsricefjlajbwqwrnmsyzqhxpe, common/ssj/sslfqpukzrmnhjuqpcynpxwhhfgag, common/dbu/xbojkukejwqysokjmlepxofkcgwvh, common/dto/zlnvslghdtcqeslbamuvuoajrczkh, common/abp/ubxsazddkfwrwbvsrnaqnqtzkpzpt, common/alu/rviayegouklslityjuzynizmeyjra, common/uvo/dbamzrlugwkdbkwmtvtedwxkemzpa, common/cow/irxawzxrxdgfzqpyuxivwgpkqxmhg, common/zyr/flxuasooinjxmeivnmhejfyhfrlif, common/uqx/oerzprithlkzvxgohpkhlpwqtdyfn, common/skq/fivnjplrhxwjgxdieqddovlndvljq, common/rdf/sdssrqsjxtwjyplccgbunvqafbabk, common/qig/qbrksjxhgxukzbvfsmguqeilqyqhy, common/jvt/upoovuvfrblgilwitutxcsphmfzql, common/bea/gsisvjzutefzlnjiroqflgoilqmln, common/eib/mcaohozyqboqdgumpufeofnkcogfc, common/esu/ngwjnyprziuopezvnkkmxpxqaaxit, common/jrz/aoduknpafycerimjldcdxxqxozaav, common/mkv/rnkefjygbboejgiyektlilfzoicly, common/cxn/xaqbfyfzyqnbqtrrcuxsemiogdfme, common/xhy/xbeuwhlmrqnztgimiaswbfbnsvrmw, common/sua/sncwmewoonvzyvmjbypkuojsqxnpo, common/epb/iwuuokziviosnclkznfowymzjepyk, common/yki/gnlqiqnhmegnmvhtdctqpeyzrsdwr, common/efj/zldyorpbfswxroxbefksarvmrhram, common/dyr/setmkxlwoxufumqysdcprxyuexflh, common/yvv/jxjujriazmsdrtoqythkdntfftpte, common/tdt/knkaxidpqymupcgywomvgberarrhy, common/qmj/ryfaujywskrftfuahbkekkdkwdteu, common/tvv/ifnrswnqijffzsovayhejwgwynhkl, common/xjw/azkeuwvaliojkbmobzmrebpqvjycf, common/zrt/xenltfexgflipvvzbcsflsddpghis, common/wfs/zhdllwhetfftjivbzspyrslnfjblk, common/giw/eeeutwulgnodhixcocxoxjnruxfda, common/cpu/anvfagrfucznhfxkjrrueowxrryae, common/iuv/toacydpqhhiysigfqehfrthhbebtq, common/ysq/bwmbtnrzuofdpaotrerjbhpzwhyfj, common/tik/wroclkerhrobkeymmkutwqzudoham, common/cmt/yfhapsvflgdhzubxatlhfrashoyfr, common/ecn/bsbyrdkqhhzuhnrtwfownelkukenp, common/agd/opdigcquivsafrcntzsqsfbluuxqo, common/qsq/odswrelblqukuyacxiehktsfnhtbz, common/ugk/ysmeplwljahnyzivfeascloekpeum, common/bmf/bqzhtsdmstvsxiihzkkwpwyigieux, common/kyj/aemoqcwzugiuayliqevucnlpmxebw, common/tgx/bktbkenhpzzibnevguwhkwxjxcaio, common/mgq/boisiqaspvxktjibcuypwdxnwxngn, common/ixw/htojhsqpccfmdxnsvleqfpohtnlnj, common/wda/hiyyevijchwhutttnrnchipsgnuoc, common/ual/cdofmpvihldrifepodtctuzspdmrt, common/yqk/ujzwpphlkrgajnkfnhpfcvbuvklfb, common/qet/uhcokqfnxfnuuntvdcgnzqsgiwtkm, common/rlk/zrxpawcmgevjtkyyibyupmbqmjmpi, common/dlu/ivfkiqumgojvhubhlirdkbjeeignb, common/qir/kqxydxuaoriwjgiiktpjfhrydkggr, common/xjh/juwpajdfjcndhogmoipwhonuawjsk, common/lic/svtphmzqtozgpfbvgdlgvztjlwirs, common/uxp/tolicslszyzvjczppberhmemeduzf, common/ozw/bfpukxizmrygnzdjfprhrcwdzocmo, common/ueo/wgcjnrevyflxzbmbgmvswwvvhtqpf, common/tcy/cllgrgfzyuljqstkhipewontxbayc, common/bjv/dsywesdhozfiwdugwknftxfkdstuh, common/roh/llibstnpwlfopyzpxxytudszejufr, common/fbo/zclvqvullktctqwguqsfxuezuodwh, common/icr/cyqeonhkbxjaivpovczhcrfnyauls, common/kjo/oojmcsgktfwwjcdhvdrccwdyaislc, common/pjh/ofsdzugzwrufqoawhgsjmjkqmnpwp, common/yix/mmyytbqkrtyyasavohackqxxzvnqb, common/lro/hdrbwbphlxrbyvdutijxuurrohsub, common/ute/woiqmmrjgunwkjechargxykxatify, common/nqp/jgjcjvlvfpewwcgstvdmiymzukqdu, common/ovh/lafnvvfsnhjcsvrkqzhzbfvdjjedt, common/oka/qgbvozpppnpusbtmigezsaxpmcelk, common/jvi/qldynwfcbawzopocyuxttkredxkhi, common/vtn/efiewadidtugkuzajlbhjhsnlkwuk, common/czx/ioojhiffcdypmgpsoccuzlejvdomy, common/nzu/ksikhdlflsuhivsubrwqqbsmwpvwg, common/bjy/ffmikaaiojvlflvlsflxqbfdlamfk, common/ulu/rrtmuforrvgkrfcjwfhopkgsxwwvo, common/ffo/latqroockrncqcpgrsdqmykjkowqd, common/krt/pdlduvxdaufvuynnrqykadtlhhhlz, common/zfe/ucngntqyccxsiqwrxnfniauzwostu, common/qjb/vdxqjmabvxpnxxihxetwaymkwznvm, common/gbm/orcehmifhloklvwflxaczjoreyiwh, common/qne/twyzviqtiriorrswgcvdqwrkzlutu, common/fyp/eirdmszijtnfcubrrdvpijdenuzep, common/pba/flesrsjmbwmpohtftzkmcdsvltcat, common/tuc/xytubrqzzxlbtwugecbvrpcnivwrr, common/jvh/oqsrusvckhzhbyifauqhldlkxcivr, common/zod/arqapgzfffezsqnwsoxliyxqjvotv, common/ryd/stlkwohrrsfqrkqpvhqhygwhrkaoo, common/jga/cmpbufnfworwdduvkgnznhfkuwfyn, common/ats/otmqkvujkbsnrpmebdqboanocqhit, common/wzd/eqitgoctmdgxuinlxcfkxozuukium, common/itw/swhkchmnywgcghitdwcueqltflugf, common/zqy/hqzedzcfeoryelexnddzgppexbblp, common/joi/lcdouctxszfnnasqmgnhnmwsfkecq, common/vua/xkmupgfmuhwduromtduntqcdpnubj, common/bnq/ophaxgkrecsgjwefjrcqwunehkpvm, common/mrl/sdbzubdwxatnurxaqboerjlydlhil, common/lnh/bkfnuvsetmhxbvnsbixciptzdgpzp, common/loc/ljjxoouulpbqkkcooqjvbfwecbgrs, common/fxj/mqjvdyrfyptaqxrnuckouoycicupk, common/bho/zmemjfshhwfyhgnovlsucnhsjidfq, common/ahb/vaekocqntfkmwswrqkpjogopmnvle, common/xdp/lbcawwbrqqatuirwduppifrkzzvgp, common/nwa/zulnoqxfypoovpogcakhmfwihwjqx, common/jku/hvipiejqcupoilpualvptritthkrk, common/bmy/haxokzbvyptfqjcrdeozftgzvstoq, common/pzl/rtuhwogoxsjtmlvuzofqzdtvwckcm, common/gbx/ullblravkjikyyqmoyuczcxclsyut, common/lfg/llbwhemhnsxozspqvtlmmwafrocay, common/oty/qiiwkoadajierfztgembgiinihgpt, common/vie/qoastnxvprngqchqiojsiogftkoyb, common/tlv/aenddnhbdofttwhbatgxperkxxngr, common/jut/skfyxhfrwiwcehbwtdeipbpivjuom, common/heu/bckeitdpvxfdwrrzhgyttfltxjxit, common/bez/qedxfofdqorfgofxirctahejdwhuh, common/fka/hjvrstwmzuxrolkxneqoghbrdezwm, common/xni/fpwhfarljxkxmihhlduawvacydtwo, common/fuz/atdvbyqagjlaghwyjuztpmvyjmekk, common/tyo/zuzkymezyzsxsgxsxzggcvhdywhhx, common/zmm/caczmjmtpwgsybvukexojszcgnfmr, common/rol/eiauidrljcfxmzuogsjwulzldllcm, common/dez/kwjgbmffpprwsulzvjhrzuzxrrees, common/sje/rusmabgozlcfsaqirnymztofblgzq, common/wgq/axpidwucioovpunmzjejagbpmocck, common/rha/fsutvgxwnywakshjfwztrcmulgaun, common/udw/vlwcutaouukxbgpqckerwfrkrbaxe, common/yme/ftwmjoflvclnjnpauznksoooinfvl, common/srd/piypumlkwtfczdtszrrvtuytgklvd, common/qdz/ntbzxwkxhcahbxeuddrewkvbqkkln, common/ava/xilpvccpiimnafgrjubngrealmgmo, common/qhq/umadxfqgcdtrmhlkaespdetbyonnz, common/fxp/tjpvnkzupgnldwqtmlqsndecoaqvj, common/xog/pnqwmugmjohdqbdonhxzzjjbmcrct, common/zxr/dfggexoevfazqvqvbtcegecygcztl, common/pmu/xrnkptpduemhoqsipqgnbeanepvne, common/foa/lvkbuvrhhhfrlpvdvkmgvewexvewq, common/axd/xeqfxwnhrybamrtwswssqciioyfby, common/udl/endaenrvvxbmtqexydaglosvhwnbj, common/azt/vpwfwkoptmhbaxwzuiuczgljjzcdb, common/xph/qcakmzkmwwcahsinyxjrvfzzhbtka, common/vja/ehtrjetjwgbvroykkozjkmmvemqyh, common/gbd/ovbbksancoxltzmimixnriumvbcuu, common/qil/pkzobdiwqjhnfocouztrvzxhivheo, common/ktj/ggyqbtgrzljjfitagvgqqebiyoafe, common/fab/wznnwrtdtiaaktnenmjreifpnwgtu, common/zqz/nffahtcjjfeqgdbkicuoiacyykldk, common/ykw/yaenvoisiviwuohpdzxdoxjezunod, common/ewo/gmjleeldywlyyyyjfjuxiuxucyuzq, common/qwy/gfzsehwgmqhsyfospvjqtsqjbmqum, common/pdn/wyzlojesmeaeeykcddsagotsfjkzs, common/xwg/qixjndrqkpdxqwmisolskjesxoyrn, common/hsu/yivrkgzqjijndftcueacbthtbuvsz, common/qjy/lvzfyugemoossostfimhwptofkiez, common/jit/ubqzlecmkovorztsiekhrowujvkrv, common/pmu/hvzlxybnsksfycutigojanlngbvho, common/dgi/eagmsxnyzujmmohjtssgocxemppya, common/xis/srpfdlgtqiuzpcffvkwzbiumnjtao, common/sym/wkivjvaqpapsafaooylemapsxnrle, common/idq/omqravhskxqirutphocufbismshqm, common/vyt/ndtiahxqivirnfyyzoyujchklezoz, common/oan/uupjmolshxrvacsbsuqtrditscdkk, common/erl/pigixnktvsulqqsmuerfobpisozpe, common/ouo/ukijphgnrlixaxngetdtzqiykwxcp, common/rqi/ntwjkwatbdytntyfjhjtzsxdvetfv, common/uoy/ebanxegykaslwqjwzfufyavfvjgse, common/hbt/bsdbeagbjdbospnxcudxdlkpvtecw, common/lop/mkuceyxhegkjgolksqikeyrcyubnd, common/kjp/hpexledrzwolsqqbbyekngwmoiuye, common/akb/fhcwdruixachfziglnitxbxqcbcqr, common/adp/sxubvekvlrinsjsjgacdsffxldpjn, common/fjh/zboriqydfefugffuxlqtjgjudeagj, common/qle/bolchsuqbrllzqwljtyvotpxozvxj, common/awl/qvkqibzbwirxivmtntelsryjtnwgf, common/wmw/tedfugndabzjstfasigyvjzqtoeds, common/our/ntxyimseppkqmanqurpqdvcugcujy, common/zeo/otvgostxjbhivozbwlqaijaxtbnwf, common/vyw/ihdwiviwwkiyiudeccytpkpnhgtmv, common/qnt/cuabhomvkzvbqukpybnoqakzzyfod, common/hqr/faoymtpotneudiohgygyrmanrqqpj, common/odw/gtmexqhwgaljbqungwcqzcpflvvqv, common/iqd/wmaijtklhrqwolmenefozeeerloef, common/cyh/yisrfjssyfeglxmnvihbbvxzzvnnz, common/mcs/mcclxvgclksudukiafpakahxcrcyn, common/nia/hyatslkhwbkgjbuyhbikmyitkzlvq, common/wga/ixhxrejiypbivecxvptoaorvhfhvx, common/ogl/gcmeifschtpnbcsilmqeujgcxzvik, common/pgc/sdgbxlixchuvnruznbpvraoaxitna, common/bdr/igtrvckbpdrcqtbilqqztbuytigsl, common/ini/wpnbvyvtgrdprskgnqbqihspbovpb, common/vzm/jgvpuzhctonujdoxilkgczqbjthyy, common/fpg/czukqiooeguccfdlbzxfypyivsgkx, common/jvh/efjsgiszdtzsejxiplfzywwbljxik, common/azm/eiqtraczmufucxgxncpdlynzmzigp, common/qsy/jbnqsgdzjzojncsgdelsizzidzyin, common/con/xbpsrwknryqhdosqxatuxetnkklor, common/lcg/ypqanstzypahxvrgtsffysrsomhwb, common/abx/udxukdwymcvudjhhergdzhnelqmnm, common/pxm/nkfcvufcdmdfbnicjdiqdyqkeldhu, common/dkr/srxexsnhppbhryrgrezddjfsllxoe, common/bbn/whhhhgndunlywsmsxnaiiuvcgaxrs, common/pjj/ywlaxlwfakvxnimigltawscffgfbb, common/vvt/jpgihbszmiycirqrewguzpkfjlgnm, common/qrv/xplnwvfhxmltfauxdnineuyolfzlo, common/wpt/xqlradcredptbwnowleoohhfnqjis, common/oja/nrayxcuuovvbakqbkqfqajsprhshe, common/lol/gihpejpfqysotvdyyipqjsfowptfl, common/rng/ozbbuambunnnqdacqjncxksokgrcs, common/wus/dmwkncdhszkpcyvfrcpgntdvfsrdg, common/pvm/rhxylnuckaxelnugrshgkrgtuukui, common/yfw/qrkahawqhkhjfsdandjxppazvjstc, common/jor/jojtauzyhaxlhqdmdotmrrppxzlup, common/kwn/kggrwggamwtukmxnwnuehdhkszobl, common/tnc/lhtxzdwpqnngxqjfjyiasmupccygp, common/dgg/fgdqcazuvghcauxaefollusyxhhts, common/aoi/ntrldwssizqezwwrqwtahhdeyvxoc, common/nex/dbjbyhpajjlndrhslxpsyppfisvpl, common/ygi/ykonxchhfcqdpwjncjvykgtbyocno, common/bai/bfqwpbxcvxyoosmhjwsyrcuwkapnh, common/hqi/zzgagybryefitpckxwqmeovtukavg, common/gny/zydyfrirhqzjykdgpvjudmobvdtkm] were removed from S3 2024.10.22 02:25:36.936492 [ 1420 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WithHash.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/WithHash.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_18_18_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:25:36.969151 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 7556 2024.10.22 02:25:37.196681 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(194daba54b07b0f31c4b90c53bee0ecd) : [0, 228546]: Setting download as failed: File segment: [0, 228546], key: 194daba54b07b0f31c4b90c53bee0ecd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:37.198601 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(194daba54b07b0f31c4b90c53bee0ecd): No space left in cache to reserve 228547 bytes, will continue without cache download 2024.10.22 02:25:37.358586 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 194daba54b07b0f31c4b90c53bee0ecd, offset: 0, size: 0 2024.10.22 02:25:37.363955 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 194daba54b07b0f31c4b90c53bee0ecd for removal 2024.10.22 02:25:37.366253 [ 1476 ] {} CacheMetadata: Key 194daba54b07b0f31c4b90c53bee0ecd is removed from metadata 2024.10.22 02:25:37.576633 [ 1161 ] {} DatabaseCatalog: Execution took 309 ms. 2024.10.22 02:25:37.630868 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 13.38 MiB, will set to 1.55 GiB (RSS), difference: -1.63 MiB 2024.10.22 02:25:37.849164 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_940_305} MergeTask::MergeProjectionsStage: Merge sorted 14316 rows, containing 12 columns (12 merged, 0 gathered) in 1.40758769 sec., 10170.591929515951 rows/sec., 1.59 MiB/sec. 2024.10.22 02:25:38.104905 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_940_305} MergedBlockOutputStream: filled checksums 202410_1_940_305 (state Temporary) 2024.10.22 02:25:38.397343 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(eb836d553b44b5158f0e343aa90da041) : [0, 190451]: Setting download as failed: File segment: [0, 190451], key: eb836d553b44b5158f0e343aa90da041, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:38.399613 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(eb836d553b44b5158f0e343aa90da041): No space left in cache to reserve 190452 bytes, will continue without cache download 2024.10.22 02:25:38.450172 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: eb836d553b44b5158f0e343aa90da041, offset: 0, size: 0 2024.10.22 02:25:38.454705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key eb836d553b44b5158f0e343aa90da041 for removal 2024.10.22 02:25:38.456064 [ 1476 ] {} CacheMetadata: Key eb836d553b44b5158f0e343aa90da041 is removed from metadata 2024.10.22 02:25:38.460932 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_940_305} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_940_305 to 202410_1_940_305 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:38.465917 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_940_305} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_939_304, 202410_940_940_0] -> 202410_1_940_305 2024.10.22 02:25:38.488910 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:25:38.658793 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 772 2024.10.22 02:25:38.952484 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:38.954815 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.04 GiB. 2024.10.22 02:25:39.017298 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_754_754_0 (state Temporary) 2024.10.22 02:25:39.311450 [ 1160 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 5 parts from 202410_185_185_0 to 202410_189_189_0 2024.10.22 02:25:39.315184 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_754_754_0 to 202410_764_764_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:39.341725 [ 1160 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:39.346442 [ 1160 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.04 GiB. 2024.10.22 02:25:39.443559 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_189_1} MergeTask::PrepareStage: Merging 5 parts: from 202410_185_185_0 to 202410_189_189_0 into Compact with storage Full 2024.10.22 02:25:39.506596 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 772 2024.10.22 02:25:39.666593 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(12a331aaa90c7e46244749cbe8dd3a86) : [0, 518133]: Setting download as failed: File segment: [0, 518133], key: 12a331aaa90c7e46244749cbe8dd3a86, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:39.678369 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(12a331aaa90c7e46244749cbe8dd3a86): No space left in cache to reserve 518134 bytes, will continue without cache download 2024.10.22 02:25:39.930363 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 12a331aaa90c7e46244749cbe8dd3a86, offset: 0, size: 0 2024.10.22 02:25:39.935555 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 12a331aaa90c7e46244749cbe8dd3a86 for removal 2024.10.22 02:25:40.043307 [ 1476 ] {} CacheMetadata: Key 12a331aaa90c7e46244749cbe8dd3a86 is removed from metadata 2024.10.22 02:25:40.196575 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1478_65, 202410_1479_1479_0, 202410_1480_1480_0, 202410_1481_1481_0, 202410_1482_1482_0, 202410_1483_1483_0] 2024.10.22 02:25:40.199971 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1478_65, 202410_1479_1479_0, 202410_1480_1480_0, 202410_1481_1481_0, 202410_1482_1482_0, 202410_1483_1483_0] 2024.10.22 02:25:40.248120 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1478_65, 202410_1479_1479_0, 202410_1480_1480_0, 202410_1481_1481_0, 202410_1482_1482_0, 202410_1483_1483_0] 2024.10.22 02:25:40.370292 [ 1191 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:25:40.420214 [ 1191 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:25:40.517776 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1351843 2024.10.22 02:25:40.558846 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 105274 2024.10.22 02:25:40.682244 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 12.98 MiB, will set to 1.55 GiB (RSS), difference: 8.85 MiB 2024.10.22 02:25:40.758829 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:40.762018 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.04 GiB. 2024.10.22 02:25:40.856409 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1570_1570_0 (state Temporary) 2024.10.22 02:25:40.909011 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1570_1570_0 to 202410_1583_1583_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:41.057184 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105274 2024.10.22 02:25:41.239462 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:41.242139 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.04 GiB. 2024.10.22 02:25:41.359281 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_892_892_0 (state Temporary) 2024.10.22 02:25:41.375405 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:25:41.379436 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:25:41.397259 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dd627111e231c3a5fa88b8bbf601bb2f) : [0, 284373]: Setting download as failed: File segment: [0, 284373], key: dd627111e231c3a5fa88b8bbf601bb2f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:41.399855 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dd627111e231c3a5fa88b8bbf601bb2f): No space left in cache to reserve 284374 bytes, will continue without cache download 2024.10.22 02:25:41.499883 [ 1726 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vgs/xfjqnpvbnwkntncspfhvmenanjyqu, size 1042 2024.10.22 02:25:41.503492 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_892_892_0 to 202410_899_899_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:41.550253 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dd627111e231c3a5fa88b8bbf601bb2f, offset: 0, size: 0 2024.10.22 02:25:41.551663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dd627111e231c3a5fa88b8bbf601bb2f for removal 2024.10.22 02:25:41.553966 [ 1476 ] {} CacheMetadata: Key dd627111e231c3a5fa88b8bbf601bb2f is removed from metadata 2024.10.22 02:25:41.557166 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1351843 2024.10.22 02:25:41.964384 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column GoalsReached 2024.10.22 02:25:41.974284 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column GoalsReached 2024.10.22 02:25:42.009408 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column GoalsReached 2024.10.22 02:25:42.028343 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column GoalsReached 2024.10.22 02:25:42.066910 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column GoalsReached 2024.10.22 02:25:42.218487 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column GoalsReached 2024.10.22 02:25:42.262624 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column GoalsReached 2024.10.22 02:25:42.411561 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wlw/lseftjwrefqfblzyopwbwnktzpvdg (LogSeriesLimiter: on interval from 2024-10-22 02:25:15 to 2024-10-22 02:25:42 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:25:43.462849 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 10.68 MiB, will set to 1.54 GiB (RSS), difference: -8.80 MiB 2024.10.22 02:25:43.842662 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 14306 2024.10.22 02:25:43.951294 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 107 entries to flush up to offset 101871 2024.10.22 02:25:44.265662 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:44.268239 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.04 GiB. 2024.10.22 02:25:44.279206 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:44.285275 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.03 GiB. 2024.10.22 02:25:44.540779 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_916_916_0 (state Temporary) 2024.10.22 02:25:44.545762 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_932_932_0 (state Temporary) 2024.10.22 02:25:44.596313 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_932_932_0 to 202410_945_945_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:44.613561 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_916_916_0 to 202410_922_922_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:44.632810 [ 1197 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_940_305 to 202410_941_941_0 2024.10.22 02:25:44.653628 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14306 2024.10.22 02:25:44.655434 [ 1197 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:44.657159 [ 1197 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.04 GiB. 2024.10.22 02:25:44.760205 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 101871 2024.10.22 02:25:44.795136 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_941_306} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_940_305 to 202410_941_941_0 into Compact with storage Full 2024.10.22 02:25:44.796458 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_881_250, 202410_882_882_0] 2024.10.22 02:25:44.798617 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_881_250, 202410_882_882_0] 2024.10.22 02:25:44.809839 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_881_250, 202410_882_882_0] 2024.10.22 02:25:45.061300 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 105322 2024.10.22 02:25:45.167306 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_941_306} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:45.215492 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_941_306} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_940_305, total 14316 rows starting from the beginning of the part 2024.10.22 02:25:45.326616 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(5301038850fc01251d78bb94a7126feb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5301038850fc01251d78bb94a7126feb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 02:25:45.344313 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 102989, file segment info: File segment: [0, 33554431], key: 5301038850fc01251d78bb94a7126feb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 02:25:45.348820 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:25:45.369440 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 12.07 MiB, will set to 1.55 GiB (RSS), difference: 5.80 MiB 2024.10.22 02:25:45.389780 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:45.396509 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.04 GiB. 2024.10.22 02:25:45.468425 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1571_1571_0 (state Temporary) 2024.10.22 02:25:45.507718 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_941_306} MergeTreeSequentialSource: Reading 2 marks from part 202410_941_941_0, total 19 rows starting from the beginning of the part 2024.10.22 02:25:45.631726 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1571_1571_0 to 202410_1584_1584_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:45.686671 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105322 2024.10.22 02:25:45.763295 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IsRobot 2024.10.22 02:25:45.812562 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IsRobot 2024.10.22 02:25:45.926579 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IsRobot 2024.10.22 02:25:46.072478 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IsRobot 2024.10.22 02:25:46.216317 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IsRobot 2024.10.22 02:25:46.252940 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IsRobot 2024.10.22 02:25:46.270120 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IsRobot 2024.10.22 02:25:46.311298 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IsRobot 2024.10.22 02:25:46.353828 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IsRobot 2024.10.22 02:25:46.447444 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_892_166/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:25:46.469054 [ 1213 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 224 ms. 2024.10.22 02:25:46.505362 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_941_306/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:25:46.804364 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_941_306} MergeTask::MergeProjectionsStage: Merge sorted 14335 rows, containing 12 columns (12 merged, 0 gathered) in 2.036108801 sec., 7040.3899796315445 rows/sec., 1.10 MiB/sec. 2024.10.22 02:25:46.863586 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_941_306} MergedBlockOutputStream: filled checksums 202410_1_941_306 (state Temporary) 2024.10.22 02:25:47.011183 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 773 2024.10.22 02:25:47.216135 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:47.220596 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.04 GiB. 2024.10.22 02:25:47.345832 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_755_755_0 (state Temporary) 2024.10.22 02:25:47.468625 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_755_755_0 to 202410_765_765_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:47.571909 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 773 2024.10.22 02:25:47.597977 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 10.32 MiB, will set to 1.55 GiB (RSS), difference: -8.72 MiB 2024.10.22 02:25:47.737466 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_941_306} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_941_306 to 202410_1_941_306 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:47.740655 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_941_306} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_940_305, 202410_941_941_0] -> 202410_1_941_306 2024.10.22 02:25:47.865787 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:25:47.913269 [ 1229 ] {} DatabaseCatalog: Execution took 334 ms. 2024.10.22 02:25:48.560144 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1353383 2024.10.22 02:25:48.856467 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 7 old parts to remove. Parts: [202410_1_164_27, 202410_165_169_1, 202410_170_170_0, 202410_171_171_0, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0] 2024.10.22 02:25:48.868131 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from filesystem (serially): Parts: [202410_1_164_27, 202410_165_169_1, 202410_170_170_0, 202410_171_171_0, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0] 2024.10.22 02:25:49.008307 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from memory: Parts: [202410_1_164_27, 202410_165_169_1, 202410_170_170_0, 202410_171_171_0, 202410_172_172_0, 202410_173_173_0, 202410_174_174_0] 2024.10.22 02:25:49.248918 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:49.250387 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.04 GiB. 2024.10.22 02:25:49.396979 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_893_893_0 (state Temporary) 2024.10.22 02:25:49.405622 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(138f2bc71764293ccc215be3c6fbdb37) : [0, 1041708]: Setting download as failed: File segment: [0, 1041708], key: 138f2bc71764293ccc215be3c6fbdb37, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:49.410701 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(138f2bc71764293ccc215be3c6fbdb37): No space left in cache to reserve 1041709 bytes, will continue without cache download 2024.10.22 02:25:49.656208 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 6.81 MiB, will set to 1.55 GiB (RSS), difference: -4.29 MiB 2024.10.22 02:25:49.659711 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_893_893_0 to 202410_900_900_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:49.697041 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 49 entries to flush up to offset 105371 2024.10.22 02:25:49.703111 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1353383 2024.10.22 02:25:49.762479 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 138f2bc71764293ccc215be3c6fbdb37, offset: 0, size: 0 2024.10.22 02:25:49.763711 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 138f2bc71764293ccc215be3c6fbdb37 for removal 2024.10.22 02:25:49.765317 [ 1476 ] {} CacheMetadata: Key 138f2bc71764293ccc215be3c6fbdb37 is removed from metadata 2024.10.22 02:25:49.951289 [ 1420 ] {} S3ObjectStorage: Objects with paths [common/mct/vrcljoitiqtpepbudyuqkcaltrgyu, common/xgj/qzbfdqklsxvcadyovqdctriowengs, common/dhs/lurabdyypbomxwnuwbtetiwyjvagj, common/mzy/vcwhwbdkojxjywswaozcunxkltdsn, common/rpu/jklifsdpbhykwhzknggmnvgpyovrn, common/dww/jmlxjjmyqvheahidlkexykmrotzkp, common/yez/aidjhnkmypojrdqtcrfsflwkbsknc, common/mml/rssmbfkxtkzemtmrcoearwypvkfpi, common/ope/jdrygebwdjoszlywlttxfqrbuecpy, common/eyo/lfujtdxcemslhnotxtouqheckawid, common/yka/skkxqxgsgrjizbnhplqeadzknivhm, common/oqo/tykfmnhehjznuaeaykzytzcyqgkob, common/tia/gbusutdxpjmottssgefkvxfgwgpyo, common/ohb/uzkqctoinwtvcudkfrdfqjkvqusno, common/viz/rtfsgcnvtvobfuprjgegqtjhwfsdo, common/msg/fzgbwxaxomhygrzpjrronulsoazun, common/svl/gizjessemcxyqoegepqugtvccjsnb, common/zrs/nugldnslkakspkjjwdofrpvypvqox, common/ubr/kfgnzsrdxjcwsjloclqvcvzdewrof, common/tfo/pbwwmhunjmnvshhdmykchkauezlwg, common/fwp/clojdrzpebxagrmabofyfqdstypdr, common/crz/puxxqyzzxpawnxkjprxjhjkqhlzag, common/zal/qyqswoeddymdtzsyuddxfjbvcygbz, common/aht/zivygacrvupiobgfkdvsczweeymkk, common/udv/gjpeazvppgidpxlizygibowakcmja, common/oeb/lwslijqhviqdxjpafqzwoguboxatz, common/cvw/icxqakassvoditsrrdvumrkyzbiwc, common/yhn/ynvwoinslmfqgoazcyqvhurzchnmq, common/bsq/yiticvuxsgkiarolvagzdticjjsch, common/dwn/apmrbbwrabirbgsxqvmwajordkkay, common/qfu/vvbekpzlbvhzzwyzzkipsobzzrvpk, common/fqa/oyevgrtsulvtqsdqthqhhjhgfakhj, common/fhw/jdabvzwxljbwjsajazfehheroxkws, common/xre/prkmqgefapyrzpqsmnailytalgtxb, common/fdu/lgomyqopzjingbivlmhjltwclpewh, common/dkc/uhbrlvviocvwjkfvscjwmpyjuxcqg, common/kok/yehqtrntideakjqxkqosesztpkqss, common/yqe/mtdwctnvskjvtnsbsaekqqcaqzhkz, common/pod/egdhvyrxikgmaoudbhibdzdkfoojo, common/zzm/vjbxulsxlihcuscldfhcoovkythjx, common/xns/relykelzqcoxwkbufgywuufhxkzaw, common/ebx/rovmzdhqprdqpwlvmmcvhkfquebjb, common/nes/afzqluazobpccqdgcdxuxfygpxthr, common/kel/zdsifyurfzwykyhbbicnouqslibqi, common/qdr/igqqaravjgztagfllplrqvkfyvczl, common/ywx/vbdwdcfdrxlrelqzvrtvjjuwuuhus, common/mgf/frrjodmzrgipfeiytbbkfexgaynxx, common/fgd/getphbinfgmskfmtmoptuealphcwp, common/isk/rsadslzptbjktlsoxbxbtontflbdo, common/kyh/khhnqmpiwqeowilffbpgdugrrxebv, common/luh/goczjcwcykrslndmerqppyqogybmu, common/nfp/dfzeasgnelhpntfergzhdponjyjzr, common/usz/fitvcnbdjbjrmcwswxmdwzzrtmfng, common/hyu/qbdnybsarhvplrphqfuhmrtzuwcmw, common/cyq/qirjiwvvhmbwxfptbqammakuiduvg, common/vyt/bleevkxofgkrermwkgshrrxmfkzkk, common/gio/vqxlkesgjjandcslwhprodwruxfem, common/zpw/lwzkeaqyebqcixdxwhamidwihsqrq, common/iza/fgfzhuyhijpibirxgquokgquwvqwk, common/nxg/azfhfkoobdyvzrzfnsnjshjyzjtof, common/dhe/mpyxoffbtvdidcaxfzulhtawysusy, common/hei/chkcavdadwuoaqreddzshvgegzafs, common/duy/pnjjjyrrwouxbqsedaxxkixkbyxys, common/rty/bnftsnouuhovanelxovqrgqulnwut, common/izy/fjstdtecscdlgnqitaekwygcdigrz, common/drg/comkekjevnwtabwyjubqgloegqjpw, common/wpn/myupedmqhozjjokmnrbbdhzjwfpuv, common/exd/viqpewfyfodulnsdcwuoauoiljynm, common/gzx/gopllpnthbutbqnhxzlvkoblpxfva, common/qfm/ygcoadvrxvmyvevahuwqxckresrkx, common/odm/tfihpfzzqbdoetirgzhzvylxddzfx, common/kyz/rqxdkrsuqkdlkvcoyglsjsodnyfjz, common/kjo/jxmczydouacixpnppbsbydpreherh, common/jkv/fznzrmnxtbrsxhesycefmwvzvmzqw, common/yby/bmsgqjvjxbjdsgfuiftlwgzqkdpvv, common/xsx/pnkbolrnabtrwctzyhkjznzyxcado, common/ixw/birxxcspjxcsvoazdweanghgadfix, common/vpv/zaqpjpxxncamhigfzgvkzwedyldqr, common/qgh/pdzszcofnbcrkagnvqymbrkulgyyw, common/lcs/argxibuuofhnfkcpoidawtwzeixsn, common/cdo/ybuzknabpjnxzyuzpqmgiakzovbxd, common/ucj/yqfzarwksujsjuqpxqygvxmncyqbi, common/sds/bofwbnscqneapxhldfvmakmrrvgrb, common/czb/jwlnljreupnvimtvnxeeahcvijzzt, common/qze/nzbhwknxtcclywduzkdbtktwmcoph, common/adx/bvholhibfqgabyqvsswvkpxgueggj, common/mur/rnkhosaeulxqjwwszbwvohgsbkmuq, common/fty/qfvvxleayjnbylvkqswekxnxetssv, common/sfj/yevlzyjldzurcaixpvllwyfzkvhnq, common/ceq/hvhbedyhxdbwgvkwwattubutzmafz, common/gxt/xigszxscdenpsjlqzihuylzivscvx, common/oob/wnqfkzsdsqlzxkflsqxpywzeayldr, common/acl/miqmwwllwrmnwacixfkcilqlryjxa, common/nfm/arqnhtqbberqvyfiaxaubzrjwhcwl, common/xjq/futxytpfkvgblxkvzsagxdgxvcwdc, common/fix/okjjpxtdzumrusrsehbkpxarbpoei, common/fuq/hmfjmywygquvtiinebwljskztxgrh, common/tlj/ilossfutmjdahygevyuagnigtakvo, common/rkq/jxrepdecosctnmghbcuhjkzpggptu, common/ekj/lmedseppqeknukjlqayvljsdgvkqy, common/eia/slrjmnahtjplsmkvzdiimrpjfzyht, common/bno/krpkxsjqbrluuzozxizgkyqsqsthj, common/nvd/dnhkmqyvibpuhsvlxbfikogtkkteq, common/meg/ejoinbnijfozxpaysdutvqxzavcpz, common/ytg/qoavzpvhrcyukyfrklsxdqjyooxyc, common/ley/aqadjhnnoldljxdeuznbalzvkgxtj, common/vwg/gpsdtvfrqefhwoitoxvrnadqsjlsg, common/usw/krbsjxieopnvczcmybiaibkuabzgp, common/ihn/ymhwojylectifqepsalezctcjyviw, common/nxf/ehspulangckpxljcsmivkgktlcagt, common/yyo/tezgyznrvmtwfdldeihafawxbbigp, common/hga/kfmvnbamywjdjpcvggazjgpnejsgp, common/pll/hhphnymwlsqzpyxeairpqtltjlrbv, common/kie/dhlnwboxdagsykhguxkowvrjibhkq, common/olw/fqtvzcnnsjxlfgmikadeeyuoekwdi, common/ikh/ccrcyspbsqatqkymoynopersdkavk, common/meo/jhzqqnxljjdlqagizbbdmbbidzftr, common/zej/awkcwhldmfytllhnrqofizueisflb, common/ukr/fefnwgvgqerokennbbvoaufiwvorj, common/eay/xpwlmsjcypsozedegmufbnzysgffk, common/pwk/cogjvfqcibskvktmgscxizmapixof, common/tfr/ewhtuocwoiosdhajqjovbxiqjarpi, common/fbk/uuoabiwwclhyqhalvsvelamhsqphy, common/hpb/dzeiqpdtfbzkgeuvvxngjdecufjwp, common/wsr/onwttxfkcxgnbzqmsdnaehobktwod, common/tiz/frgmtoxmfcofydpevezssogdxjknb, common/hfv/nwalusguahcqzwkahihrhimhwkryy, common/ikh/ucvjyxhtaabjgkakoreondhbkuhtw, common/fiu/kcmafbmnpbsuexmbhzmqxseedxedl, common/bme/cfgrnohqlhgtpqjcptleigrykvmdm, common/kue/eainmlxzptwkuyuhfwilgbhgqqorr, common/omd/fahawxpjjwaujatlxrneyewvgtvfd, common/fen/haycmfoofywqglhajufyaldhecfaw, common/yrx/tzayuwojpbchksiansqyzghmjiqvi, common/vpg/nfescfqekebsbcwdptrjphocbhdvc, common/ygb/sbumjxbehvglxslmmvcovifwacskn, common/ouy/dnmkjqaxediobzalgusmudnketwyc, common/uii/ikfwysmfkeqwihzpvqmqnnbstflps, common/apm/eogbpaizpgunvzkjxojochxjaqkkl, common/ieb/mdvrheftuoliwmgwirqdgykbwchvn, common/upn/gptrbdhscgwewmubrulpyenrbtyck, common/vny/cbhhcuvgtrknsqhbnmhxqdikbfzjn, common/buu/tygsppbazlnikujdcdokghoqzvtru, common/uso/uyvfuloisdldsleifeyaikfgmqren, common/cwg/azefkdiwymqjbhjaalibfemwiknoq, common/zwd/lbrmautrbxasrfoqaiaatkoyowzqm, common/cnf/wybyfdalskpegpcdcrujomlekveav, common/uvs/mzthfldzqwvbqazdfoxvyorslhopx, common/qnb/hebotzvbitytmjcscisohuecadbpr, common/wsf/yocsggpxnowngzswieqplkggurqon, common/eni/fpopodluictmlxqomgzhfqwvnbjez, common/kto/abpcavgxugexujinbxxufxzwrgsul, common/nvk/zscqyzqqqghkvpqtlucpwjzuxlcvd, common/mka/hhuahpdwaxqojaiapgblrxntrgflg, common/cgh/ycvleezudguxjwjzvifzrfinudzfj, common/peq/hsegmmldwdemcmizgrogglymmasxq, common/akt/wvgketfobhxrryfdnpvmgnbxczeuk, common/pzr/kdtcxbwgppttuywvtbbevkixwdmgp, common/osc/bjzsfresiiydsdjpvykweaoimqdag, common/hot/sheuzgfcxigitdxkcwuvkvyxhokug, common/pdw/vtunmrqdwzndokshvrsdkevdndflp, common/zdk/ptdbrikjoodnkcdslqsmkrwyfrsrv, common/num/dbcomhyjtzugbmhpwhfbzporuhomc, common/ink/vimyoxguimtyxpeboewjxcshoxerp, common/oqb/oxmtnkyroffwrfgednyooejsaemsk, common/hbe/zkeengrtwuduxeupxjlxovqjamrcy, common/hha/zpdfnmpyygrdxslhiiellvynfasyh, common/yoa/htoryhkydxhchrnrgwnvcxticgzxh, common/bdi/dsziiyxmwevzfbmkowvlizhnnnluw, common/vav/qelwaawykpzcttzfenijfgyeygtsm, common/nhb/zmkwckwjrpomojodfrzipijfrcoap, common/cnl/yyngermkmdkzcwwpoldofhjgcmcad, common/vvg/ikqbhbimrllnhlfvcmhtaydbegcdg, common/cip/hjpggzuplgukitsxyncyjlbvpurha, common/rbj/wccptopkdjnbdbzogniyhmydpgasi, common/tdr/xyjvvplprntzbrejvvhfnwqlcywvu, common/qrp/gekqsrkdstbmgcrfztbfbgtmjksni, common/wax/kxefqcgroxnqisvoatmyoexipgnjx, common/icw/nqomebanfyionssfvukeqpgfskrmz, common/ezj/dzkxzgefijncicoulxqpqjiwnudpc, common/axq/ywpakwxjztvvqpwxpkdnulmemfyfs, common/vsm/wrmanwxllkkyexjkpglvralbwjeql, common/kqm/atlygqpctzoievaisrrqgegrbfgei, common/gcx/vrqjkrpuweqkibfixxlptofwoaprc, common/lzv/qrchivxnwiyaebvknlvwuoaurqthu, common/mki/xotxlilqccjeuxizvuowwwenykjlc, common/xss/sxsqwvzwzprwrdrjkxncgdbyzfecj, common/btk/hdnqcvalgbujcjlxapbnrnjnbisqy, common/uhh/swlkrulasanfvwmogqtaatktgbxaj, common/bgj/zqyxsmdzekgslekzhjvajcltmqure, common/hwm/xefgvevluxlwyvtktitpromwewxgi, common/qxj/amkuwyliarpczokowgrqkabkdybli, common/zpx/okuiplvgryeprelclzivungagxqln, common/shz/hereyhbykqbkwnekbuowzjpugkuor, common/xnr/gywotjgdrisxcwcqrzslbtopqbhgv, common/hwx/rxvcvawpxeyiiuuuadksvwebwrgeq, common/pji/hlebxaygzduutiuahclydczqyuota, common/lxe/kpdxduwviabfupjkcdvemviswuwpc, common/xaj/qnfoqzrnsuangaaidxyinvgscagrw, common/vpu/tfvrqzdkeqobraiwnuvpxywntuiug, common/jdb/alnerxeuetjtonbgwbqqgvugctgdu, common/ahy/efbzjqihzrogykrmconxayialwnif, common/qqm/pjwjltrhkkjleudshtkbctqgjkdpc, common/dyn/rcxcdlxbwxahbmcalogfhevlwkwwc, common/gba/hsjuazmpgaonsvajdfvjezpgzzfwe, common/ppm/ceikipoistikfybvjekqpycxhnoot, common/ani/pnqzuimgjoxmfeycjzifqbcyhkenv, common/xva/fvumxwepdbnycwzkurricsmguznvi, common/owj/xzytubxwnhivitmyyhboubmbybspe, common/jsy/gqincctwanffgryedrebpitfexikp, common/qtx/mxcexksbcusrdickrmnbiqqnuruta, common/mkq/ozqcnhwkfznligvnbnlvhldfwjowz, common/rnv/emxbbszkccdeebmhlsxanjrvviwlf, common/hcl/hxalwphftsglnuaxxoflatalekuok, common/kbf/yagskesghvxxryyeubfeqzvztozjt, common/nrg/rrxxsgjiwdbbylgyfcsoylfukjese, common/jnf/zkvfmqapbucxmdhowdwapszwtttiu, common/hdm/wjqfkobqycztecxbosjllmqjzpqvq, common/wal/mecbnanhhfaynaxuyidtnmqtgtfkd, common/ndo/ogggmesoljyvqilwixnaklorpjcwj, common/jgo/zhjgqddmbbqtbjqmyqnjtxtggladc, common/zut/qsnyfynwdxthifrsltmtutlzhypbd, common/wsr/aogpmazarqftdgwfdahvyuugdmnok, common/lzm/icvhbigfpdgzqshonmyqbcffbkonc, common/kuk/evnvgsikucnohxpvoouycelnjmugv, common/csj/sljbotfaszsimsgzxziueloghctqz, common/gba/jatnjgqskvebknxrjmizhnqsjcyqg, common/afn/yjqormgvgvvqiibwmtqycdqznvbuu, common/tiz/ksxznrrqgorbxcjfbxemmskhdcncr, common/fgs/tbbcgzfmmnfzqghunjrpnbhglahik, common/ele/qgkxbkfseeqfjoaziqxkslfgcekkl, common/wvq/xsvpgthuuvauilkmcjvgahjvomdfn, common/eta/gcdhoaowhitkudjqlsmbrvemeebvi, common/gwc/jkmcccvyalfwmwprhawndxdfneusi, common/usy/apenkpppndfakkqwutiqajlfwhmug, common/rcm/jzhgrouzdhjshqrxwqsbsjbwipcng, common/rjk/addbpmbllagmaohfjscawtrczzxvl, common/taa/gmnzwpzpedxrgkfxhrithyvmzmmcu, common/flr/kfnqkacblhndsoazbysnkvlmzqdqc, common/sxe/lytectczrvghbeuqbxlaxhpbvvlns, common/crl/sdedourrhtftzhouhsfclibxcclfy, common/hjw/ipstmrbezbtzpokcagbjlosuzyskh, common/tys/soflecwwrswbbcdtrsmwocdtyowtx, common/rnj/xyjtomamtqntidafzkojzltyuicae, common/tvp/dchgjkcncklamavfqeelxfvqkygki, common/qkm/ipjmhcmkgcnwzekvkycesiruzfehh, common/iqj/byhwqjvyutgbcxscpzvfxnpgfciez, common/hco/kjumijiwbxcnwrzvxmmmbgbsjvgjc, common/fbf/mqqmgzzukgdtgnwjcyzrovjdidpze, common/ruk/pheqqsngkjzcgpicfzyebpztwemkt, common/skj/tejjhikgrtruucedadfbhxhsvizzv, common/ohb/znetrwntpvyuwqqkybqoxpsdustlx, common/koe/dyvtfoaawffbxsrwykewddxhtutxp, common/yyp/bppoacihcnhofnpzmhndxbttqzphq, common/alx/minwjhhosrboxckpdjhblcojhknjy, common/gxt/gduocbnnqxwjnzkxugzowzqbezgzi, common/zpl/htvrsaqydzxkselkzjqcomduygoge, common/otz/ltvxvumrjmwhpnuyjwnqogpcbwcsw, common/pls/lozhdtwqxvfzzfleneqcuowphkghz, common/ayy/wbzfpivpqhenbxjskgjrpcoqyjpoq, common/wib/vpyjnzwzoontjbccmlsjklqvlfvtq, common/mso/ltvnrevlmcgnbrqujkiznavkxajwm, common/jgp/bijgdmvrkiuthnmhdouxtrtlintud, common/dti/zbwmkkvysgnesuycfhypfmcgyohla, common/zau/mcjcabvpugrmddruleuygxagctkce, common/mqa/lvdxofjfxnzcnchvbnwbtrmwidcnr, common/xoo/rvcpxnezygkcfpiekjjepasdvaofp, common/luv/wdziomnezwyuqehrsojmdwzlkzgzz, common/aqx/iedpfyuyxrpvrjdseeexztbefqfva, common/emy/njgvzyhjotwgwbisqwwajyeluaswa, common/zpr/pvspzmmlhiptepqzzkpfdcfomofqy, common/zhm/pcnsgdxvespzrqhqqaqulemwrfogq, common/vuw/lktqucatpkpuzjghybfehzkjbjcts, common/waw/rvugxtgkszufogfuboltuovsvseot, common/mch/ruodukzgqscjjqbvftloyombjzeks, common/qlv/vohwkdgjjissmmgyunisohhfvtvjw, common/ppq/igwyzhdjkuuaoeykimlugreoioknd, common/boz/ptvbsxqjhajeyihgzvmvufjgdlsww, common/zqb/zjdyencbiqgjkrxlwsujnizvuccfo, common/uxy/nfwdaeomcfiyrkhlxtveskrtkjkkz, common/cew/fytftgvegndhjxobcracigajwtxqz, common/zyr/cubcvggglhokudlnqfdzxcqhtafzd, common/xzq/nnburjvbzfrlgpqthcfcgngzekpdf, common/qpq/kadtwbzfqoxzjbrenubijuijqmphi, common/vtg/dvgozkvseaoqwkhwiirlnpfwynrtu, common/fmm/xsulfonptmbutzxmehtvfcjugkgxs, common/ogo/pjvjpkzyocilyzugzzflcaxnkbqod, common/thw/ikbugmwkqtkrcmrvzkypssfyztxln, common/qaz/ccpiqmbxhpzjvjnpdrlchmnvkqkhq, common/hlx/ipaqlfzpaecoxzhqauzncvslhrbxn, common/opt/twcxabhlocvejlgcaqfveonutepci, common/vkf/vpskwcewyteniufqxemqwhaserkak, common/yjz/pqrvolyfyrtgkvhgugubgoxrrqpib, common/sit/ekwqxynyadusvtnfkloasqtbkcljq, common/pfx/kpykengfpslpvrpredwisghmyklhe, common/pka/ozucldnbeywpmzenidcuvfwdxvdgp, common/cxv/ibtmkzhkzwgkdsfdeslwkstngzoyz, common/vbx/kdaouxpovzbzsnqltyxdbzvhihgbi, common/zet/ahcqdrnbbucmpkyfjppsjvyxdmszj, common/oer/cvkjvpbuyjtwwmwzzwhxrzawdaqcw, common/izu/qzlnbexjhqcufbszyzhtrcgottcgc, common/wdp/pfxeilebdczbyqiwqasqypeyaylhv, common/tej/jufvcbbacyxnqqoctatpgipcwbgkj, common/vic/rjukgjeqlgjuqvukadbjubihsydjq, common/dvz/buchxengpqbtczvnkmfaffztqpmsl, common/mfy/zdxpyvupzfvwkyzerlastawltqofg, common/cpy/lsylregjjfgdmpvsslvytdxneqxrc, common/nyf/psoewlemysqzquovohwjwfrwelpsj, common/rwm/ojeaxukwpbnogfrlnkradvtbuzfbl, common/hur/fnffymefiqxpmmlzdqeenvbezjiqb, common/qwk/rnruynihtftikoxdszfmsspyecamx, common/sfa/uiwuljroevcpcqffaeziihkjdcwsg, common/nzd/xtepmnlltcbevlnwisgnzkvhevnyn, common/ktg/sfdekfaeshnvuxicpcgycrhvyteao, common/idc/vtwnylanukesvlxwqmdqktxpfrikf, common/usv/zdhpzomwrugyahielbonlitodkuva, common/snr/jznqgccdzelqmexfetehyvistbgxp, common/tyz/opudoqiejvgamwmkppuylkivzgbyr, common/kpk/qsdlnxgvxmevyqytuespmymgzxoyo, common/xjb/oxtccyfwfkikrlvnywjntvuaehubu, common/soy/nqrnagmjkgqjryjjtaeoxyltygtnd, common/nlr/dwbmqkdzkvcrmizmobzpisopqxwip, common/kgv/uknnuonqjsqeeamdllesafwxtfeyx, common/nox/irfuurrhvwdpxwfqlzdshivayokdi, common/jzk/qdrenilwauljpxlycppesqtpqiqip, common/bgc/phuupuhcrmvulenctepimdzilvfdh, common/bzm/qygoszwhjuwdcvnkbitzrxjlhoklg, common/vvt/qnxfydshtpglpbcangdervmhejoew, common/mal/otkqhntebolaljizntoqtiagqmtpi, common/ony/xevjfdgudfpowrstfmymgkptrdbmv, common/ckt/lotjmogcvjrpgjzvhkcfjmozzduah, common/bbc/vnzhdwzppuhrenhqukikafssmwpbz, common/qsg/iphqdnhrzaihfeymuqgvwqcauvejv, common/qzr/btuacsxwihjvpwslgoedyeqjdwmlx, common/fwo/yohqtmdnqjlkzgmqtycrejzshdbgp, common/uwp/zccbbsgktvhqotcstqrsecznqwfzr, common/zro/bxlisdptpaiqzeoklthsikbnzktkr, common/eiv/svrtnbupvggkceboytwvvcusrgsdy, common/lqs/tvimeqdyymrmnjtdxwwjcewlqulca, common/pdz/moyxyyybkvtrgoeglwsxjrpsoywaf, common/bln/znjuuuitsvfafynhtdmhxhgarnhyd, common/ozb/tthufkjiaptiyxcbmjxnfjlfuhnlz, common/xzk/yunmwoimczbklnimswymicvgikjho, common/dnl/ratubuodkruefybunyxmeijaogbit, common/kgo/ntnpswalidqijvrimcmdhiophtcnx, common/ovd/lzqsjssxthpttundjikyfoloqbzfw, common/nxo/agsaqeetohseyqznlfheoihhndxqq, common/jio/reuurrgsrxkfmywodhnxlrecendir, common/yfl/afrwqbvtvuvzthcyvpnpjjewtzcfo, common/lhl/hoyfhgyifdaihfhfumwghrcxlsmog, common/xhh/yxpcubnyrwemoscwvuolwxrosftsz, common/mih/uvmivnqqqskwrbhbeloulyjncoxzd, common/uat/lnohamhdbnzgbqwjugjfhhwlxsmcr, common/zui/lftmdidkljgjzupixitrwlckiugzk, common/sbr/zipirmkuseeffwfcebqspazxptcxg, common/prt/xwwegffhuicyhhubfhpazxrushyyi, common/sdz/hwuqfljjeetxcrbcfsovvadzynhqv, common/gnp/clwvxqmaexupvfdiezguzuhkompyp, common/kob/agijcaqoztrfowehzyequatrpwhsp, common/iil/ifrrlfesabmgkswdsaluavtzxftui, common/mzw/udqrixkpxothdhjyccujrecldijgr, common/yss/kdtvhnottettweeojkbzzirdfltgy, common/wtf/mehuliawikkiuzugsybjejzznflgv, common/uha/gylblquruneidnbwuhdsxvghgelal, common/jai/puvnemgxfwmmhnxqooylyektspkpv] were removed from S3 2024.10.22 02:25:49.959900 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:49.962240 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:25:50.139146 [ 1420 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WithHash.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/WithHash.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_19_19_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:25:50.254172 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1572_1572_0 (state Temporary) 2024.10.22 02:25:50.338090 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1572_1572_0 to 202410_1585_1585_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:50.368068 [ 1260 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1207_1580_119 to 202410_1583_1583_0 2024.10.22 02:25:50.401271 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105371 2024.10.22 02:25:50.406739 [ 1260 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.19 MiB using storage policy from min volume index 0 2024.10.22 02:25:50.413289 [ 1260 ] {} DiskLocal: Reserved 1.19 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:25:50.435855 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1583_120} MergeTask::PrepareStage: Merging 4 parts: from 202410_1207_1580_119 to 202410_1583_1583_0 into Compact with storage Full 2024.10.22 02:25:50.471195 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1583_120} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:50.492081 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1583_120} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1580_119, total 25933 rows starting from the beginning of the part 2024.10.22 02:25:50.575019 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1583_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_1581_1581_0, total 86 rows starting from the beginning of the part 2024.10.22 02:25:50.625739 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1583_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_1582_1582_0, total 46 rows starting from the beginning of the part 2024.10.22 02:25:50.749607 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1583_120} MergeTreeSequentialSource: Reading 2 marks from part 202410_1583_1583_0, total 68 rows starting from the beginning of the part 2024.10.22 02:25:51.150589 [ 1266 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 253 ms. 2024.10.22 02:25:51.308576 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(10194cd6b40efe1de39a0e70e41b01ba) : [0, 1354771]: Setting download as failed: File segment: [0, 1354771], key: 10194cd6b40efe1de39a0e70e41b01ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:51.310978 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(10194cd6b40efe1de39a0e70e41b01ba): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:25:51.357545 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 10194cd6b40efe1de39a0e70e41b01ba, offset: 0, size: 0 2024.10.22 02:25:51.360076 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 10194cd6b40efe1de39a0e70e41b01ba for removal 2024.10.22 02:25:51.362080 [ 1476 ] {} CacheMetadata: Key 10194cd6b40efe1de39a0e70e41b01ba is removed from metadata 2024.10.22 02:25:51.375268 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:25:51.379371 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:25:52.156413 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 14315 2024.10.22 02:25:52.269354 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 193 entries to flush up to offset 102064 2024.10.22 02:25:52.448221 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:52.452140 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:25:52.599267 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_933_933_0 (state Temporary) 2024.10.22 02:25:52.637584 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:52.643904 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:25:52.713206 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(12cf7479b676aa480305484fd8f666e4) : [0, 428160]: Setting download as failed: File segment: [0, 428160], key: 12cf7479b676aa480305484fd8f666e4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:52.715603 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_917_917_0 (state Temporary) 2024.10.22 02:25:52.720342 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(12cf7479b676aa480305484fd8f666e4): No space left in cache to reserve 428161 bytes, will continue without cache download 2024.10.22 02:25:52.735751 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_189_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:52.752738 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_933_933_0 to 202410_946_946_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:52.897118 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14315 2024.10.22 02:25:52.956929 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 12cf7479b676aa480305484fd8f666e4, offset: 0, size: 0 2024.10.22 02:25:52.958565 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 12cf7479b676aa480305484fd8f666e4 for removal 2024.10.22 02:25:52.960351 [ 1476 ] {} CacheMetadata: Key 12cf7479b676aa480305484fd8f666e4 is removed from metadata 2024.10.22 02:25:52.994691 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_917_917_0 to 202410_923_923_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:53.047620 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 102064 2024.10.22 02:25:53.153415 [ 1273 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_941_306 to 202410_942_942_0 2024.10.22 02:25:53.169249 [ 1273 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:53.173138 [ 1273 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:25:53.188717 [ 1273 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 425 ms. 2024.10.22 02:25:53.207175 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1583_120} MergeTask::MergeProjectionsStage: Merge sorted 26133 rows, containing 13 columns (13 merged, 0 gathered) in 2.779109494 sec., 9403.371855776188 rows/sec., 1.73 MiB/sec. 2024.10.22 02:25:53.231825 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_942_307} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_941_306 to 202410_942_942_0 into Compact with storage Full 2024.10.22 02:25:53.277192 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_942_307} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:25:53.293193 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_942_307} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_941_306, total 14335 rows starting from the beginning of the part 2024.10.22 02:25:53.328432 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_942_307} MergeTreeSequentialSource: Reading 2 marks from part 202410_942_942_0, total 15 rows starting from the beginning of the part 2024.10.22 02:25:53.371885 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1583_120} MergedBlockOutputStream: filled checksums 202410_1207_1583_120 (state Temporary) 2024.10.22 02:25:53.492988 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1583_120} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1583_120 to 202410_1207_1583_120 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:53.498838 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1583_120} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1207_1580_119, 202410_1583_1583_0] -> 202410_1207_1583_120 2024.10.22 02:25:53.651614 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.60 MiB. 2024.10.22 02:25:54.398435 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_189_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_185_185_0, total 38 rows starting from the beginning of the part 2024.10.22 02:25:54.412768 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 105432 2024.10.22 02:25:54.544286 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_882_251, 202410_883_883_0] 2024.10.22 02:25:54.563656 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 7.47 MiB, will set to 1.55 GiB (RSS), difference: 4.29 MiB 2024.10.22 02:25:54.579403 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_942_307} MergeTask::MergeProjectionsStage: Merge sorted 14350 rows, containing 12 columns (12 merged, 0 gathered) in 1.367380366 sec., 10494.519562232766 rows/sec., 1.64 MiB/sec. 2024.10.22 02:25:54.649351 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_882_251, 202410_883_883_0] 2024.10.22 02:25:54.664997 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:54.666919 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_882_251, 202410_883_883_0] 2024.10.22 02:25:54.719060 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:54.729288 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:25:54.767525 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:25:54.800773 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_942_307} MergedBlockOutputStream: filled checksums 202410_1_942_307 (state Temporary) 2024.10.22 02:25:54.858897 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e8a16444a319e53c3285559dbf951b6b) : [0, 116062]: Setting download as failed: File segment: [0, 116062], key: e8a16444a319e53c3285559dbf951b6b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:25:54.860227 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e8a16444a319e53c3285559dbf951b6b): No space left in cache to reserve 116063 bytes, will continue without cache download 2024.10.22 02:25:54.873235 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e8a16444a319e53c3285559dbf951b6b, offset: 0, size: 0 2024.10.22 02:25:54.875259 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e8a16444a319e53c3285559dbf951b6b for removal 2024.10.22 02:25:54.883057 [ 1476 ] {} CacheMetadata: Key e8a16444a319e53c3285559dbf951b6b is removed from metadata 2024.10.22 02:25:54.964856 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1573_1573_0 (state Temporary) 2024.10.22 02:25:54.985783 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/mpm/imbvjpiaiyhuianzrduqwcrdtvgta. 2024.10.22 02:25:55.087077 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 774 2024.10.22 02:25:55.231541 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1573_1573_0 to 202410_1586_1586_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:55.368986 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:55.371646 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:25:55.411198 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105432 2024.10.22 02:25:55.412197 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_942_307} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_942_307 to 202410_1_942_307 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:55.417806 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_942_307} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_941_306, 202410_942_942_0] -> 202410_1_942_307 2024.10.22 02:25:55.433982 [ 1150 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:25:55.446688 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_756_756_0 (state Temporary) 2024.10.22 02:25:55.458837 [ 1150 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:25:55.462439 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:25:55.720683 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_756_756_0 to 202410_766_766_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:55.794084 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 774 2024.10.22 02:25:56.443921 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 8.02 MiB, will set to 1.55 GiB (RSS), difference: 3.44 MiB 2024.10.22 02:25:56.711784 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1338 entries to flush up to offset 1354721 2024.10.22 02:25:57.437323 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:57.443600 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:25:57.589284 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_894_894_0 (state Temporary) 2024.10.22 02:25:57.741271 [ 1240 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 234 ms. 2024.10.22 02:25:57.844496 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_894_894_0 to 202410_901_901_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:57.869741 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1354721 2024.10.22 02:25:57.896053 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_892_166/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:25:57.939728 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_187_187_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:25:57.950730 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_189_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_186_186_0, total 42 rows starting from the beginning of the part 2024.10.22 02:25:58.744518 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_187_187_0 (state Temporary) 2024.10.22 02:25:58.764376 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 11.57 MiB, will set to 1.55 GiB (RSS), difference: -6.86 MiB 2024.10.22 02:25:59.183668 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_839_839_0, 202410_840_840_0, 202410_841_841_0, 202410_842_842_0, 202410_843_843_0, 202410_844_844_0] 2024.10.22 02:25:59.187745 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_839_839_0, 202410_840_840_0, 202410_841_841_0, 202410_842_842_0, 202410_843_843_0, 202410_844_844_0] 2024.10.22 02:25:59.219111 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_839_839_0, 202410_840_840_0, 202410_841_841_0, 202410_842_842_0, 202410_843_843_0, 202410_844_844_0] 2024.10.22 02:25:59.335608 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_713_145, 202410_714_714_0, 202410_715_715_0, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0] 2024.10.22 02:25:59.337645 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_713_145, 202410_714_714_0, 202410_715_715_0, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0] 2024.10.22 02:25:59.424405 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 105480 2024.10.22 02:25:59.534661 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:25:59.538242 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:25:59.562728 [ 1421 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_713_145, 202410_714_714_0, 202410_715_715_0, 202410_716_716_0, 202410_717_717_0, 202410_718_718_0] 2024.10.22 02:25:59.588460 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1574_1574_0 (state Temporary) 2024.10.22 02:25:59.752057 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1574_1574_0 to 202410_1587_1587_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:25:59.805143 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105480 2024.10.22 02:25:59.961569 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_892_166} MergeTask::MergeProjectionsStage: Merge sorted 1355038 rows, containing 4 columns (4 merged, 0 gathered) in 42.661566692 sec., 31762.499717435363 rows/sec., 476.40 KiB/sec. 2024.10.22 02:26:00.046324 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_892_166} MergedBlockOutputStream: filled checksums 202410_1_892_166 (state Temporary) 2024.10.22 02:26:00.176198 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_189_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_187_187_0, total 43 rows starting from the beginning of the part 2024.10.22 02:26:00.306671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_19_19_0 to 201403_19_19_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:00.327274 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/mpm/imbvjpiaiyhuianzrduqwcrdtvgta. 2024.10.22 02:26:00.367249 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:26:00.401011 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 14330 2024.10.22 02:26:00.583544 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.38 GiB, peak 2.00 GiB, free memory in arenas 14.20 MiB, will set to 1.55 GiB (RSS), difference: 172.02 MiB 2024.10.22 02:26:00.593262 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 72 entries to flush up to offset 102136 2024.10.22 02:26:00.760285 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:00.765424 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:26:00.823892 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:00.870920 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:26:00.908700 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_934_934_0 (state Temporary) 2024.10.22 02:26:01.047431 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_918_918_0 (state Temporary) 2024.10.22 02:26:01.088392 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_934_934_0 to 202410_947_947_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:01.123442 [ 1196 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_942_307 to 202410_943_943_0 2024.10.22 02:26:01.137200 [ 1196 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:01.144625 [ 1196 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.09 GiB. 2024.10.22 02:26:01.150756 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14330 2024.10.22 02:26:01.176973 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_943_308} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_942_307 to 202410_943_943_0 into Compact with storage Full 2024.10.22 02:26:01.347038 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_883_252, 202410_884_884_0] 2024.10.22 02:26:01.349105 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_883_252, 202410_884_884_0] 2024.10.22 02:26:01.363036 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_943_308} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:01.364434 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_883_252, 202410_884_884_0] 2024.10.22 02:26:01.372892 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_943_308} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_942_307, total 14350 rows starting from the beginning of the part 2024.10.22 02:26:01.376799 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:26:01.383017 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:26:01.422482 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_918_918_0 to 202410_924_924_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:01.424644 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_943_308} MergeTreeSequentialSource: Reading 2 marks from part 202410_943_943_0, total 26 rows starting from the beginning of the part 2024.10.22 02:26:01.484105 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 102136 2024.10.22 02:26:01.675084 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_189_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_188_188_0, total 46 rows starting from the beginning of the part 2024.10.22 02:26:02.450432 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_943_308} MergeTask::MergeProjectionsStage: Merge sorted 14376 rows, containing 12 columns (12 merged, 0 gathered) in 1.280017283 sec., 11231.098353849336 rows/sec., 1.76 MiB/sec. 2024.10.22 02:26:02.514886 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_943_308} MergedBlockOutputStream: filled checksums 202410_1_943_308 (state Temporary) 2024.10.22 02:26:02.746538 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.22 GiB, peak 2.00 GiB, free memory in arenas 9.55 MiB, will set to 1.33 GiB (RSS), difference: 115.73 MiB 2024.10.22 02:26:02.911374 [ 1420 ] {} S3ObjectStorage: Objects with paths [common/ssk/zuyyjoiulbyfxzenuahsgfzovmdyd, common/inp/pjtrsbxdthwidnharehdnnymokvvr, common/mcx/alhyymcbakzldftsbavgiublscwif, common/fld/lfmtpjjdurujrrczuaecjseunnkeg, common/bxe/nostqlulrwbphbrjipsvelkisozdo, common/zgl/hvxtfnyqqadyqnwrcvaonpcfejoqn, common/zpd/uaoogrzzxeitsxgkhafdzhlwyyvdm, common/yco/pcbnrdleloktfxopdlguhqkxnfjph, common/ygx/qgpfqtdcqmkassxyidugeqzdtzfil, common/afy/rgtjrahwqeogwmcwhfyksbaxcweyw, common/tsd/sgbjqfmopjxtnnqzzsrhzebidaqnn, common/ggr/fktdmacjzdxjzntrldmduvxrlmnpy, common/nsc/xhbkfpmimxbcxzzluqeaqjmxyvrxv, common/xex/ylydrrnjhrlffalohypesxmkpfldq, common/zum/rdlwajfsqxbgyqbsvqjdsqbmkjwht, common/gao/vddufqxexllzisqmnncjoaqxaujes, common/bwo/qznjitxiwteckwkvwocbilxylyvgv, common/pwx/hrpbvphmrdbvljvaiwxlpqjtwvthm, common/ztm/qnzdtdjaeegjkkwarvkeeitdqufhz, common/jaq/wwvwisqpdjgrcqgbgrxwuindhktga, common/hzh/pojffmyuachlsujputjabkermrtfa, common/ntl/ogecqscuqpbllslkbohmmavrtndce, common/kbv/ozplnuahluspsfuigqyrwgihcypfw, common/ovt/xpdeyhkmmfdaenpsmusrfcuzwgzkb, common/eva/ytvnkyiqqmhmyywxumvofkajcvhvf, common/kzx/hrbdxrxzsrllnronclmqioxvpbvoq, common/ber/visvyveqsrpafeeobalfurkcitsbp, common/gby/cogxphkvdmzjvevwxndhkacfxcbso, common/xcj/kpigijqwmhrcifktseefgzavtpnoo, common/ayy/rziptcxpdsxjlbrzqmxrryygwbkcf, common/ujg/fctttnsytthptaqcemvaxocfzuggt, common/blf/blytxqwcwxmwehkullqkvaylqowrh, common/ofv/chgtrsuoqikgvkywezunofegnmpng, common/wzs/ipoycrhbviqnuzyhgemkpiafrzbfo, common/xcw/vttpgvznwqbiuqkhzjqiqngqjdpzz, common/jcw/hheflapotnidtorstduqgshczbyus, common/scw/pgwjfgjntkeyupfsacxuhuakutaes, common/qha/bvgcfcarxtybytlkyfxtyddvirtax, common/mrn/axerymzxfwjagfceljqfhynzzmnxm, common/jld/hqpsfpndhtvrqpxgzgsupsuisjcoi, common/cil/jioudumdzqhhghkzudluooqrywqqg, common/exn/eoprsvqqtplsxbjvnprzhwjlqvxwp, common/qui/kobzmtmxpeuvrlibaagwumfiablnw, common/whd/weqqwumnrmfofryhfgebwdmqhvmeg, common/cvl/ylfuhdjmbeceghksvsonkxprnlcna, common/xym/diytibjxjvungtuwfmlymexzchwpw, common/jny/lyfpxkgopjdrudyujqnpuusmajadw, common/shg/wyssupgzcavazshfixlewtjhqwhxm, common/aon/dhgsrlhozldcrpbpsvffpdwvvfwgg, common/tji/xsoqqvzbjiadjzpftwkgmlzygmzan, common/cnp/mqhrqlsvafluwgeistgtoolfsjdwh, common/tjv/fzdugljuvoghcpfhtswekuvcjucdy, common/agq/fuoyepngamaxcjguajevqkkrqffpp, common/qor/rpslvjhytyvwakobzbrebbdbvydwu, common/ljb/zcpbmzavpzwcgwyjkbgctphsnxwei, common/eok/uqshmdgoqbohqqoewoazhywrupgtu, common/muy/jhjwfuqqgnskvzlrblfkmvssjzyho, common/cxk/emvpcikcrvymugedohwawellsgnee, common/gah/limbmdzkzbwbehmbyjcsleqdqntok, common/ttg/tdzqzxrstdchsyfatjsdxunmtzwvv, common/lgm/aabvjjwydqviikpvuxzhtvrdxwhlb, common/xiq/vjohxfokhztyhpdpdumwhnmwnvrie, common/kes/phuxtxqqtywlazukifoiwyvjigpwm, common/cjh/soncnechlrgnkphrpaukeauahrtys, common/ozw/zblfeqrcjqodfwaajhtzwxzgdotzr, common/wob/opytovkssosfdezfumitvbeoropvy, common/ddm/ouaaglfanlpahkbfystsndprujhax, common/opq/imtmetvbdqqbmqhahqvlwytheupqz, common/oej/tgjycxmuqdiyzyejtikjmruufpyzr, common/qiy/kphrtwlkuyzdjmvqajkopeopffdku, common/pre/tcnnsmkbcnaukmiqksqutuqvovchh, common/kkt/hbtndwbzriidymuvansqvriumqmpf, common/igz/dtagghonepciahgfqpocmwzlmveom, common/yyu/qbteddrabnikhczwupuyylclidksk, common/wyz/sovoghdkgsvrehzpmdavoyaljfoun, common/vhz/hsudhlrkzcofltyylkbdvydspmylf, common/uhs/wszyyidjrmlulconpxvctqmwjyqqy, common/ojh/qvefuwkfnwslazaqqpmjfbmcixkep, common/vsv/achtigfzoohwxbckiiczfbecnuhcz, common/ogy/twiappsgzlixuimfdxsbzbldqnzjt, common/gyc/kceoexomnmuocamurthvftpmnkqhe, common/gin/cqivrsvduqgcpidocrvcvaqtuqmaq, common/kwz/lkmvejyzfsucjxfgyzgbbooexsnel, common/acq/brkbgzcyplqgmbbptyfrdfdwckslq, common/aib/ewvnrotmalzxmgbrctpfifrqtfmbx, common/yff/xxldhvivntalhxtykqnihkqxznvdx, common/jtu/akgnahjamzgphsydghfholjhhxusp, common/lff/ljgbyytxdtofgrxfahuwinoevkatj, common/xjq/sawwnuzthuuhcrcpcfruxsvwtdofg, common/wkr/mdqsclvoodboueiecrxafevvkiijv, common/xlz/kpihenpnqceucjzbllbnptjblyxhi, common/jyq/jnjwfbzafzisjanwbifpvlukmkcge, common/vnc/jcdwpqfakfhksysfvwbdcqlwimceg, common/npn/bvmqttluvzlrpghvapzoosejuykvy, common/xnq/mstltlvhalkhdtucbkwezcoxgqguy, common/jil/gymrvhetqpemvzcywkxybyaeriucf, common/fgj/bvdectsilnxdljfpgvvxgzscdziui, common/qkh/atbfapsshawhvxnlqchkverwjesus, common/wzo/bsunvcegooyykpedzhpapcnelxmkr, common/jtr/zrmmhvdjchmwaslaiwdatdwglyurj, common/sgf/yvfsuzkadksofhcxfgwwrxfvroqgn, common/oit/wzpqltyiawvnriiptsfqtxjnqukez, common/huz/wjdicmkdkjaoeparfqamdsemnpkdn, common/yaj/qbyfkzrsoizipmthgrjdghomurwja, common/kvh/fwfkpaqmueumlsotkeiwjrmiebheq, common/wis/unexadbmjsgrsrltlolhnhyrtijjw, common/lzr/dpbkdlcnmsvoshyannpbysspzihfg, common/raj/mfddhswxzdirmwafyzagsxjuiwpih, common/rut/dykaspamrqlkpkhvndlkulijbvact, common/yzw/tmdxwpbctddusitpjwxhtloyxvicz, common/sri/hmakarsukrjfgzgwupwhvvvijxqyh, common/xya/jwsnmhttztjzwpsvwhwlerumctdnf, common/ilw/wydbloelmqycnxttirewlcctosjcz, common/jly/ppebfyfxpwmjsrsdriwcpkdsfumci, common/ufd/srynebjslmixnnmdvbojgoehwgnla, common/otf/gimyhlrcsjewtozedttfwetympwbt, common/nfn/lkwfawsfvxbuiydqpzdboadtfemmr, common/wwh/pclstumdtkobzbpaeuoardzabefvo, common/jko/qgqgccnstqesjeyxzqfbbvfxszumb, common/sgc/crdtvbdqtidszmqukysrxdivtkeua, common/jru/jknrvtykjhgqbvpnizzlvjdsxinwm, common/kse/gmbixkczmiorjbwaixsbuxsrjgbwc, common/eoy/pptgbpqyripsdzfppdijxyjbgilgm, common/epq/sauzcamaexeoqdiflaexkiwmnqajb, common/osb/aaqgyowpqexgjpugylunjewjysamb, common/owf/jysfpdoarucuybcjdxzifnlgvffil, common/mon/byqtbonaqqidizlptmreuzzfoefcc, common/gci/qiyolpyypdyhvbdjscozgtwbgmqqx, common/ujj/srjxeplslaonafkfldopaqlodjdni, common/ntr/jnbucyuuocmqgaoecrhjvphqzpcwj, common/buo/ntdemzalzurttpmxvsygcfnnazmnu, common/ble/nrtstfpthayjmgbnyjlruypdffxkj, common/kkk/mboadfmtxoyasjxgexqgyldwheniv, common/eiv/niqjofhjepaxozouwbgolegglvrpe, common/tct/unokxnxvlmkhkylxuetwokxvpptyy, common/pyn/xngisosopzjqmiazrnimkpbfcqymo, common/ebz/zlrgifoklxhphosfnwsvakxdcscyp, common/mzg/ftlhkvhehbdeocuqtpatumpahlcma, common/kjo/gkdhtzttrusefivxtvaqdslinvetp, common/bdt/hylvmahywwodvmyrfgqklfvwjkwhd, common/rbx/qtvtfgvynfmuzwrblnvhpigwjlmjx, common/dzf/shzbgruqaupfstjcgyejdcxnclxgs, common/hfz/elakagykdracmifacvonutopdtzbp, common/vbg/wegjtzdflqpioriksmvoaltawijcw, common/gow/kpcvszfuzewgzgarclmfgiatbyhqm, common/yfr/xddsfsosbxfjxffxbbnsidjotsswj, common/ftp/snbdbximrxyypqahnndhbkdezcajo, common/jjo/inqewxnnsmfqmxyqiiwaejrlmiwql, common/cse/auyoymcanoouhrkuxoonjaqflytrq, common/efr/iaffzojqyimevihkewcvzppcsnsqa, common/lby/rzjebtrviartpetbkpixfdvmwwpph, common/fli/ybzkwfiliueobkxrriormggomgeiq, common/usg/bfhrzuxddlohkdgmsaudowlfmdqzd, common/vxw/xenmnhkxzwavjjvhbzxpwsumhbbzp, common/woo/sqpnxatogrjuwlkkhtkthtmkjsvme, common/mor/lvytzaygsccwxfpmkgigojettgygd, common/ffz/mgluueqalmtlyscqwvumqrritarvl, common/lbj/jmcogtwoojqsmmiyydoyckmybwtcc, common/tgo/mkrsvsvvkhsafsfiprjhmlgebfycu, common/qnp/xgciajqhsxqbrbllhhxmjaxvegmre, common/btg/iehftqezuhjtlzenbgknbqaplwfgw, common/evu/mghyhwgpnrgwzodfztulyvrdunmgi, common/tak/iwihsxchfnxzbkaoalwppkghimkfd, common/ljg/uzaguaqtbebppmjdtnmewgtuyqocb, common/vey/befjowopefrwdcoesqohsezplldht, common/kje/hueddwjadpwriizdoejjwtxesvdsz, common/jyu/gembruxmbiydfzkbaxkshkpuhaqsi, common/hvp/nekzuotdbtayghqwsxxycvklvaxbn, common/kem/hqsycqvodgxmipukrmvcwhvcbmkni, common/tqi/ixhhdnwzzvozvidhyfauliwcfvvha, common/fbt/mubuhtpzcybdadppftqmnarcdgyqj, common/nhk/hevyqxekosmljwllsfpgtkiajnogq, common/tab/myxylarnkolrzblucunzhhxowhbyi, common/etb/pooonasuxewydymkhphbxqptunnzv, common/clb/aapglyeeehgnpgleimappvuqvoaka, common/pys/klsirhvbkgwctcgmovtuikmrkfgql, common/aui/ykwomlayeeeqzgaevvzkqvthnbahy, common/ewq/yvoukadbahbozkkkvhsybfhwzevgg, common/kpq/vofbinrciguqdpccotvuhsimlqqxq, common/rvv/rsrbfzwdblqkajnkymobdyvlhfvqm, common/lvq/hyrzzhyngaseuwbzutughfsxcwwoa, common/zlo/liidbaxlefbipajgumommtaqjgjkw, common/gsg/nwqaocdvhbpjuodlyiuxltpyagomn, common/ibm/dbmpzmpyatbbylqlqnblzugrdiryv, common/sji/bsbtvcgpsxshoivnexasjfomsbzug, common/llh/rgkslrtdujmyecpvfklxnqhavhkec, common/yzb/edfcgmrmkvfcyujygrakllxmayeno, common/hjr/pzstyqysswsmyxvbmgbygwhuyxhcn, common/isa/traedsskifcbjwsywgfxznxmlwxfa, common/oab/fjepulzxuctycfarztgbjglxaxvoq, common/alf/uzaojxzimfxmncheafmyoynwwusae, common/odz/ajtprxuhkhomjnflnfmahvjhblffs, common/otz/lqpxhphflnzolvtwtvfioyvykkxol, common/poh/kwwpvjiqvcborvnannegfacjwisjf, common/dyk/besbhypehdxucmeowfywswsmrjmft, common/npd/ncrluhemzbelpjinlcawgebvzykqg, common/sed/avsqdpholpmbaantiiqsfxntnxfrh, common/lqz/xpwmeybafqwrsmckwqvhkurcyxboc, common/iim/gvfxioestputyvhlxitswjuimbgqb, common/tig/iyvtksaxzpfrqeoijhlcjwjpngrca, common/vqu/ojemogufxzdnctpcmhsvyjghtprcy, common/ool/qsrtjfwnlqcojvcpipwpcatncdkbb, common/pya/gapjsukauizwfkdrzgknjqqyrtmwf, common/uum/lslryeegvivvdqltzsnbwizrxkghv, common/ckf/rkdsnyttptfvkazragfflydbghdzn, common/wgl/jvqmpxkpjjyhnwiacdquwgdmomxyh, common/muu/yzhjbkkwmtovliwhuufgmpfpimrtz, common/uxe/zdwkzcejgozflznfqjkpnbpmtqguh, common/iqb/xcewtbcylukmcdvykedmmezswjtur, common/nry/orwmftllvonibccweraittkebqkvn, common/ckj/rprfxnufwfmcampoomkpcxxrohyex, common/qyy/kodigvgkmdatpzxljsnpiwqxcgvwu, common/tmh/pkuytssggaumljyxrtfpufdffpxqy, common/pnc/znkefqocszpnftngoxyhwualgvjsq, common/vie/dlunnggwvniethywvurhncvxxpmlp, common/zzs/tetneaqayohslqychklqjehcjgnlw, common/bmu/kadiuslugflfvsvvfdnhyuzvqvpyq, common/ieg/jnhhthfgutiyzbnomfygwbluhkdhu, common/wzc/rtcfztubbqxfothwijklmksykkdya, common/rxo/vgqpqkowiesfhhmrqhzskezcyydrk, common/sfp/mnhfqawonnodtdfmglxhnwtaqhwfm, common/uda/ruwmuooqdbnczqyfphbpkauefyukq, common/voz/cntylhrqmrdoemkaqpwoxftlauslb, common/vfg/jqhzpqrdplsxvqnmlccacedfppotp, common/buo/ggawqhfwyufrfzyfnphydkkbeletc, common/amv/bevdwxurrufyzpazbibxiolhxpqnu, common/xkn/mqwtmiwbirrylwplbhgfukybfzvvb, common/loq/wqrdrwaojzlwqxqssvjkrubjvxunc, common/mkt/epptwlhzcppyffahrzrhuszibfzmi, common/zkw/vshdkhtgxmfkawsroqpsgpnoxtrxa, common/esd/cmavjofymdmjrqywkcrdkprjxbvhx, common/udn/pajcbvmqexzcspmkhheoywtgjnkpg, common/otu/cgqfmbuujqnvwsjaevzrlrkpyaaxs, common/ogz/qdbglajgxskmehcxwvuopjqvchpqh, common/vdv/gakzxuxuxcrqpnagodabqwgtblegr, common/aew/jyozzbwpfuabksgxcwwgjgnhtjewo, common/rfq/dofnzosgwozfznxrddzmisqymdjvi, common/lzl/clxioiovwddqlvwgspymvypoqosen, common/dbo/hljexvkwpzerfqnxlgmwsbayagyji, common/gxc/deenwhrdaarwzxbpfqiyexponuzaa, common/wlj/yxairwdklegxufwohxhumikgisigo, common/zhy/fasykoembmtohfqlmrrjogtgjuiem, common/gzh/neegagazgfsclgvhjbvuwoakfbwpf, common/egh/murkeveqihytchonwkrpzzkyhwoov, common/efi/momvhtxdqyraqpocshpogpcpoyamm, common/aqg/clwpfdpyukadrtopxtbccjxhdcbkz, common/qup/rwqaheatrdydygnmrkcarudvfmerd, common/dxi/nbutpqbhxnirsgnmsobzzoooehmbx, common/oln/fexxhabovbxzhmqjwegftindiohgp, common/kes/qjitruuxepebpqevqnsnydjfkcpfp, common/tof/keviztfqqtawosipaswsqbxqakizz, common/uqs/fppozstmnyzpezwmdtvfbqvthywgc, common/ink/eeqtuwypgnrvoaezcukbzuonzfzxp, common/kcu/orojlsptgsdbnjvatybrvsihedxau, common/nkl/qqfszbrqpfjpwntdpjdilljweecra, common/bep/nxzykwxumpoqffiqmqjxoyjlpbrqm, common/hqz/zwxtwumyqpsyreeuokqasbztyzrhq, common/urn/neviygthsybqtoxpsfwlnmsvdlybo, common/nzw/bzdnclccboqdjevosqdklqodbspax, common/jgg/nkavlwtskdjzzcmmagmlfctwsssik, common/zqb/dvhggkhidafpnlkpkryapupcbwvnd, common/uib/smuvdxzynozmfkytitjiuhoausujt, common/dyg/ebrjfsvtchpwlbxdczljdthcmeczk, common/quj/bcefxfseepreqslessjweoibdlshk, common/dbo/wiqwvljwcpmzuptayyevhjtwbshse, common/hwo/tewifnqhcwkxuseskwtpwdokssuko, common/nsb/ekzukxpaashkcbqshmcsmsvvntwox, common/ckp/hhbgeoipoozbffddakqrxoghccoct, common/inf/rqbtlpgrhcfdfwoavszgbwmoaszgt, common/giv/rtlofacgvajvhdqxnnjtveepwlasz, common/aam/umtdeckgztrbqmgimfpwlsirobahc, common/ihe/innnbioxmtyiwvwbewnvvfbcdgfxm, common/ack/ptswhidxtnpkbyhzskxlwzxwfooah, common/jxf/nlvvidphldhugtcexrqgvwosvznbh, common/gir/iilafnhpuxefpadwllhneoccrygva, common/kjk/uujwlcrinrsvhczcugkciroxzjbft, common/hoo/cphvjwwjzmatsdcshlpxppywhtafg, common/bux/ctjygtyapnwjacxjoofehwfdhbehy, common/yqv/uwjiynpvhjyyzxjpcdjvkirtjibkr, common/rnc/jmdxkpuqjsihbkihjrwzurblkhmoj, common/ufu/vegifzojrpestrdhgekrnpvlwkjzv, common/bvr/wicmxcqplehrgtfvcsvnwxrnorqzv, common/ajs/drkganvvlcbhjxqljqbkfwrkbrskd, common/vgx/bdxnjmkbztdwkejkqoygkqaxhyyyh, common/nvv/czfhzmicoglkviomickbyvhntomgy, common/tfi/xqreaptjorioygshkdjlsqytcvuri, common/xud/vannbojqtstpvgcepvrczokysskpy, common/tpn/qgtvcikiehfbfbofadyvrcpkgqcde, common/ncs/swxvzixekxfmvujlzlksgrzzfkkrm, common/lue/akqnbldjmiomxycforalkdrggzeht, common/pxm/erobgnbjjqzzwldpqkjfgtzgxojhg, common/slr/zoqtvsoqifbutvcscmwesqknrthcb, common/lfi/fprshityqbmbawwfsspdymctgqsdr, common/exg/xkrvocrmkvuvxhjscapxiewmuyqil, common/ksu/ifxefvibwwszlfhlvszaawobkxufr, common/ksd/nxrhgazwjafbkfrvpvrbbjxkunhwu, common/qqh/wbbilxhhqxofritjspmzehemqgiwh, common/ogl/gxxbfghtyafdydqzrrmvfghcvxmbl, common/qet/xaxlqzgujguuwtizxgnsppzcxlxyx, common/vnt/ohctlfrgassagmjkkggpbcwtckomb, common/dzs/weoqfudovdennfglacqjalvusphum, common/zzu/lbhqustkppqgprcnuqlydeeoewism, common/yiv/tcxomehlsjzsdjywwrhdkfxehscug, common/bcy/jgranuwqtdlhppxzbvfonzdwshovl, common/pyz/nqbbdjbgxmgtmzwwvzmwohaycyvwt, common/hzs/bxxpckkuwyjqyuunugenbpfiljwei, common/fsy/lwvognyugiavqfdakierpqcsmjphr, common/buf/scvvyfojluyotkmpdomficpnkowks, common/mmb/akaheimbhqffrhufoecsmhbiswcfg, common/osk/wjdnwornhmsewttksqavkrikgswrm, common/mvh/wkknsefbrmovnaqndapqpapiqvvki, common/zpd/padumetkwvpdxjbvlbqawbeevgzsl, common/sxt/gxuoffptnvircjafjgwhyrkydmqil, common/lac/mhgxduckzwhuxlqjwhcnnibhfluql, common/nyl/zeitvhpfosppfakagwicuknrdsrzq, common/thj/jluufkatrtgojzcynkodayzeooqom, common/dlx/olnmuumhvgrlskyuerobiscnoiziq, common/cvb/kretkfvewgysoxamhnhxxhekpgfsl, common/pad/gyatnkdliislmeuahgktyznirilxj, common/eer/bxztsgvzrsvrvwxndzpsyjtgbmiwk, common/afy/lozygtcfdjbqafqqbpvabxbxuyhbl, common/oqx/myicpwtempdoqnczlwlziapmtmogv, common/zuf/wymuhqbcssodvbvfqxnlezptriwbe, common/zlg/jlhohncznozntzdywttohtkypsgwp, common/ceu/fnoufwiwddcqedgdigwlqqgxhdwhw, common/jxg/zqhfxkdneiboejskfnmlhqvgnjzif, common/fwq/oxpncrohahrpfutlgqbpgodenfngr, common/rrs/fhoitlskpqbvepruztcfbexxaywoh, common/etg/nidlpsglgendocckxpltqsoyngjqf, common/lga/agrslzwipscgjxvashvlpsnxrhcbo, common/oht/shfdlfcuszucfnpbieijniszamfdu, common/yma/eyydnuocrbwopnpsvwrtqfjsmbjcu, common/nsi/szziicvlxlnmzcimtlactyrkyxaor, common/jdd/cdwdicssaharapodcfhktnzqstvjj, common/orz/jnhyfqvsijydcgxpzrtxioyitsffq, common/qsu/fkmdjjdeyaclxfwczlsqfrxxepxkl, common/zki/efozckadtequejsxwybakpymqdxxk, common/rtq/cibymrpxaxitzcvvlzxtkwdhaptkr, common/mbj/sypwidbvgjdohsqxfcrubzsamakwy, common/rme/zxcqankhmhanzjsicfbolfzxykpeh, common/ljd/phbanmoemwxcpqlimaajqwgnptads, common/qes/gnuhrxlnzcrvrxpklrynuayfxojan, common/amu/qlmirecokcqttczcsactpxeszbvdc, common/erz/chnmiimonpvbqexsaewijpapftynf, common/vag/aaeztdwdhagrhgvhgiwogufujdqak, common/atc/kifersqcvrvjhjzrdjygsqrbzfakc, common/xbg/vrrukbpwdszaipztctrjtebcnovhw, common/hum/wquuwwcnkyoxwzijrhaubafeqjjrb, common/roa/jhtbquhiydcftbgkewffhgaawlmdx, common/bkg/tafqrhqxfcfwjujfhiljixnxzzuuw, common/iom/mbdqtskrcvinxfyhtsahivrqqbbwp, common/tdi/qcytkxxlojgczmilsdhmmonietkjl, common/qdn/lkucdqlynmoyrrnyyobwjgifqtfzg, common/acu/qjrtgeuzqrthvsgylelxgkuwaxofa, common/cvi/iktplgljlyxawgmxysgqjrhnkblpb, common/cpr/jledbnmyvgvdeuajsxaguzpqldccl, common/ujm/tuypwbfxowimewdpzltihyglnakys, common/yfp/ttuwewfltqvtllcmetnvveelpgxef, common/sdo/xddeqodtksqoydmwomxxgcotvcghl, common/rby/tkfzfnavpiuqrremphyefgoquuntn, common/gxr/mzxvifccycbdldgwuxhytfrikajoj] were removed from S3 2024.10.22 02:26:02.957591 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_943_308} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_943_308 to 202410_1_943_308 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:02.962983 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_943_308} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_942_307, 202410_943_943_0] -> 202410_1_943_308 2024.10.22 02:26:03.010864 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:26:03.060136 [ 1420 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CounterClass.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CounterClass.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_20_20_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:26:03.296956 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 775 2024.10.22 02:26:03.763897 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_892_166} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_892_166 to 202410_1_892_166 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:03.768105 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_892_166} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_886_165, 202410_887_892_1] -> 202410_1_892_166 2024.10.22 02:26:03.793817 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:26:03.810184 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 105532 2024.10.22 02:26:03.845110 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:03.847775 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:03.916134 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_757_757_0 (state Temporary) 2024.10.22 02:26:04.162943 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:04.165728 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:04.177356 [ 1387 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_189_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_189_189_0, total 40 rows starting from the beginning of the part 2024.10.22 02:26:04.215876 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1575_1575_0 (state Temporary) 2024.10.22 02:26:04.230784 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_757_757_0 to 202410_767_767_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:04.365742 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1575_1575_0 to 202410_1588_1588_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:04.422675 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1583_120 to 202410_1588_1588_0 2024.10.22 02:26:04.447783 [ 1210 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.24 MiB using storage policy from min volume index 0 2024.10.22 02:26:04.449651 [ 1210 ] {} DiskLocal: Reserved 1.24 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:04.468097 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105532 2024.10.22 02:26:04.476693 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1583_120 to 202410_1588_1588_0 into Compact with storage Full 2024.10.22 02:26:04.588660 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1207_1483_66, 202410_1484_1484_0, 202410_1485_1485_0, 202410_1486_1486_0] 2024.10.22 02:26:04.592706 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1207_1483_66, 202410_1484_1484_0, 202410_1485_1485_0, 202410_1486_1486_0] 2024.10.22 02:26:04.615449 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1207_1483_66, 202410_1484_1484_0, 202410_1485_1485_0, 202410_1486_1486_0] 2024.10.22 02:26:04.645114 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 775 2024.10.22 02:26:04.668857 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:04.700398 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1583_120, total 26133 rows starting from the beginning of the part 2024.10.22 02:26:04.750485 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.48 GiB, peak 2.00 GiB, free memory in arenas 8.09 MiB, will set to 1.36 GiB (RSS), difference: -115.91 MiB 2024.10.22 02:26:04.873567 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1357 entries to flush up to offset 1356078 2024.10.22 02:26:04.920382 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_1584_1584_0, total 48 rows starting from the beginning of the part 2024.10.22 02:26:05.050622 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_187_187_0 to 202410_190_190_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:05.128136 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_1585_1585_0, total 49 rows starting from the beginning of the part 2024.10.22 02:26:05.210842 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_1586_1586_0, total 61 rows starting from the beginning of the part 2024.10.22 02:26:05.417118 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_1587_1587_0, total 48 rows starting from the beginning of the part 2024.10.22 02:26:05.607290 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} MergeTreeSequentialSource: Reading 2 marks from part 202410_1588_1588_0, total 52 rows starting from the beginning of the part 2024.10.22 02:26:05.908514 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:05.912470 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:06.242841 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_895_895_0 (state Temporary) 2024.10.22 02:26:06.330278 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_895_895_0 to 202410_902_902_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:06.373521 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1356078 2024.10.22 02:26:06.367210 [ 1209 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_892_166 to 202410_893_898_1 2024.10.22 02:26:06.396992 [ 1209 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.06 MiB using storage policy from min volume index 0 2024.10.22 02:26:06.411890 [ 1209 ] {} DiskLocal: Reserved 1.06 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:06.436801 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_898_167} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_892_166 to 202410_893_898_1 into Wide with storage Full 2024.10.22 02:26:06.478973 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_898_167} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:06.491562 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_898_167} MergeTreeSequentialSource: Reading 167 marks from part 202410_1_892_166, total 1355038 rows starting from the beginning of the part 2024.10.22 02:26:06.530340 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 62 2024.10.22 02:26:06.543804 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 3.25 MiB, will set to 1.38 GiB (RSS), difference: 14.57 MiB 2024.10.22 02:26:06.754224 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_898_167} MergeTreeSequentialSource: Reading 3 marks from part 202410_893_898_1, total 8855 rows starting from the beginning of the part 2024.10.22 02:26:06.942414 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:06.946490 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:07.109301 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_47_47_0 (state Temporary) 2024.10.22 02:26:07.168238 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_47_47_0 to 202410_47_47_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:07.354274 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 62 2024.10.22 02:26:07.683211 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7556 2024.10.22 02:26:08.344391 [ 1255 ] {} DatabaseCatalog: Execution took 262 ms. 2024.10.22 02:26:08.452062 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 2.00 GiB, free memory in arenas 9.19 MiB, will set to 1.43 GiB (RSS), difference: -35.14 MiB 2024.10.22 02:26:08.474713 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 105583 2024.10.22 02:26:08.557994 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_898_167/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:26:08.662143 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 14343 2024.10.22 02:26:08.668428 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:08.673091 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:08.743788 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1576_1576_0 (state Temporary) 2024.10.22 02:26:08.767805 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:08.773046 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:08.797276 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1576_1576_0 to 202410_1589_1589_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:08.836392 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_935_935_0 (state Temporary) 2024.10.22 02:26:08.963386 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105583 2024.10.22 02:26:08.990608 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_884_253, 202410_885_885_0] 2024.10.22 02:26:08.991773 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_935_935_0 to 202410_948_948_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:08.997666 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_884_253, 202410_885_885_0] 2024.10.22 02:26:09.005709 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 136 entries to flush up to offset 102272 2024.10.22 02:26:09.011489 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_884_253, 202410_885_885_0] 2024.10.22 02:26:09.044974 [ 1260 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_943_308 to 202410_944_944_0 2024.10.22 02:26:09.051621 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14343 2024.10.22 02:26:09.155863 [ 1260 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:09.157492 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:09.219836 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:09.230683 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:09.251732 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_862_23, 202410_863_863_0, 202410_864_864_0, 202410_865_865_0, 202410_866_866_0, 202410_867_867_0] 2024.10.22 02:26:09.276143 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_919_919_0 (state Temporary) 2024.10.22 02:26:09.254994 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_862_23, 202410_863_863_0, 202410_864_864_0, 202410_865_865_0, 202410_866_866_0, 202410_867_867_0] 2024.10.22 02:26:09.327275 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_862_23, 202410_863_863_0, 202410_864_864_0, 202410_865_865_0, 202410_866_866_0, 202410_867_867_0] 2024.10.22 02:26:09.331871 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_944_309} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_943_308 to 202410_944_944_0 into Compact with storage Full 2024.10.22 02:26:09.489611 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_944_309} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:09.542669 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} MergeTask::MergeProjectionsStage: Merge sorted 26391 rows, containing 13 columns (13 merged, 0 gathered) in 5.074142294 sec., 5201.0760579588905 rows/sec., 996.41 KiB/sec. 2024.10.22 02:26:09.547860 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_944_309} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_943_308, total 14376 rows starting from the beginning of the part 2024.10.22 02:26:09.551257 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_919_919_0 to 202410_925_925_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:09.565570 [ 1274 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 4 parts from 202410_747_920_36 to 202410_923_923_0 2024.10.22 02:26:09.569394 [ 1274 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.10 MiB using storage policy from min volume index 0 2024.10.22 02:26:09.590824 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 102272 2024.10.22 02:26:09.592616 [ 1274 ] {} DiskLocal: Reserved 1.10 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:09.653723 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(de1e4672fc2bed563e6bc22e93225913) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: de1e4672fc2bed563e6bc22e93225913, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 02:26:09.659664 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 41019, file segment info: File segment: [0, 33554431], key: de1e4672fc2bed563e6bc22e93225913, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 02:26:09.662507 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:26:09.671026 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_923_37} MergeTask::PrepareStage: Merging 4 parts: from 202410_747_920_36 to 202410_923_923_0 into Compact with storage Full 2024.10.22 02:26:09.720237 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} MergedBlockOutputStream: filled checksums 202410_1207_1588_121 (state Temporary) 2024.10.22 02:26:09.729114 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_944_309} MergeTreeSequentialSource: Reading 2 marks from part 202410_944_944_0, total 20 rows starting from the beginning of the part 2024.10.22 02:26:09.870552 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_923_37} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:09.892018 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_923_37} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_920_36, total 18283 rows starting from the beginning of the part 2024.10.22 02:26:09.894957 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RefererCategories 2024.10.22 02:26:09.940392 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RefererCategories 2024.10.22 02:26:09.955917 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_923_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_921_921_0, total 126 rows starting from the beginning of the part 2024.10.22 02:26:09.958995 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RefererCategories 2024.10.22 02:26:10.083430 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RefererCategories 2024.10.22 02:26:10.124899 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_923_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_922_922_0, total 107 rows starting from the beginning of the part 2024.10.22 02:26:10.169255 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_923_37} MergeTreeSequentialSource: Reading 2 marks from part 202410_923_923_0, total 193 rows starting from the beginning of the part 2024.10.22 02:26:10.230695 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RefererCategories 2024.10.22 02:26:10.320823 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RefererCategories 2024.10.22 02:26:10.340946 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1588_121 to 202410_1207_1588_121 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:10.354161 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1588_121} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1583_120, 202410_1588_1588_0] -> 202410_1207_1588_121 2024.10.22 02:26:10.365609 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RefererCategories 2024.10.22 02:26:10.415054 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RefererCategories 2024.10.22 02:26:10.463464 [ 1153 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:26:10.535439 [ 1153 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:26:10.549429 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RefererCategories 2024.10.22 02:26:10.622326 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.75 MiB. 2024.10.22 02:26:10.631886 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/bfa/avxxnnxigpxiwpdzqkobgmogxhcag (LogSeriesLimiter: on interval from 2024-10-22 02:25:42 to 2024-10-22 02:26:10 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:26:11.353770 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_923_37} MergeTask::MergeProjectionsStage: Merge sorted 18709 rows, containing 13 columns (13 merged, 0 gathered) in 1.697224889 sec., 11023.288735191298 rows/sec., 4.79 MiB/sec. 2024.10.22 02:26:11.377382 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:26:11.380326 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_944_309} MergeTask::MergeProjectionsStage: Merge sorted 14396 rows, containing 12 columns (12 merged, 0 gathered) in 2.061725004 sec., 6982.5025025500445 rows/sec., 1.09 MiB/sec. 2024.10.22 02:26:11.383685 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:26:11.430333 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_923_37} MergedBlockOutputStream: filled checksums 202410_747_923_37 (state Temporary) 2024.10.22 02:26:11.472929 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.72 MiB, will set to 1.49 GiB (RSS), difference: -80.74 MiB 2024.10.22 02:26:11.497011 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_189_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:26:11.499336 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(6813534881e7527502a9611ec80f30e7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6813534881e7527502a9611ec80f30e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:26:11.512124 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 334564, file segment info: File segment: [0, 33554431], key: 6813534881e7527502a9611ec80f30e7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:26:11.520927 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:26:11.654388 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_944_309} MergedBlockOutputStream: filled checksums 202410_1_944_309 (state Temporary) 2024.10.22 02:26:11.797533 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_923_37} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_923_37 to 202410_747_923_37 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:11.903415 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_923_37} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 4 parts: [202410_747_920_36, 202410_923_923_0] -> 202410_747_923_37 2024.10.22 02:26:11.962986 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 21.98 MiB. 2024.10.22 02:26:12.070546 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_944_309} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_944_309 to 202410_1_944_309 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:12.077494 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_944_309} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_943_308, 202410_944_944_0] -> 202410_1_944_309 2024.10.22 02:26:12.148502 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 776 2024.10.22 02:26:12.315006 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:26:12.567023 [ 1203 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 240 ms. 2024.10.22 02:26:12.591855 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:12.595785 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:12.755288 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_758_758_0 (state Temporary) 2024.10.22 02:26:12.927301 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(4dd4dd2e12ea628a6a394bacc727c54d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4dd4dd2e12ea628a6a394bacc727c54d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:26:12.933234 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 852893, file segment info: File segment: [0, 33554431], key: 4dd4dd2e12ea628a6a394bacc727c54d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:26:12.936079 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:26:12.991290 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 105669 2024.10.22 02:26:13.001623 [ 1684 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wlw/lseftjwrefqfblzyopwbwnktzpvdg, size 852893 2024.10.22 02:26:13.024610 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_758_758_0 to 202410_768_768_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:13.117398 [ 1246 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_763_155 to 202410_768_768_0 2024.10.22 02:26:13.190965 [ 1246 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:13.196664 [ 1246 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:13.276634 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 776 2024.10.22 02:26:13.338927 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_763_155 to 202410_768_768_0 into Compact with storage Full 2024.10.22 02:26:13.405398 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 02:26:13.410769 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1292 entries to flush up to offset 1357370 2024.10.22 02:26:13.453785 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 02:26:13.467247 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 02:26:13.497549 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 02:26:13.544370 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 02:26:13.545407 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:13.547995 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:13.588357 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 02:26:13.643030 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OpenstatServiceName 2024.10.22 02:26:13.665261 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1577_1577_0 (state Temporary) 2024.10.22 02:26:13.742167 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1577_1577_0 to 202410_1590_1590_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:13.763380 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:13.870621 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105669 2024.10.22 02:26:13.986796 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_763_155, total 807 rows starting from the beginning of the part 2024.10.22 02:26:14.295982 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} MergeTreeSequentialSource: Reading 2 marks from part 202410_764_764_0, total 1 rows starting from the beginning of the part 2024.10.22 02:26:14.460149 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:14.462703 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.13 GiB. 2024.10.22 02:26:14.639868 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} MergeTreeSequentialSource: Reading 2 marks from part 202410_765_765_0, total 1 rows starting from the beginning of the part 2024.10.22 02:26:14.867613 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_896_896_0 (state Temporary) 2024.10.22 02:26:15.077731 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_896_896_0 to 202410_903_903_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:15.153803 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} MergeTreeSequentialSource: Reading 2 marks from part 202410_766_766_0, total 1 rows starting from the beginning of the part 2024.10.22 02:26:15.249304 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1357370 2024.10.22 02:26:15.281808 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 7594 2024.10.22 02:26:15.421035 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} MergeTreeSequentialSource: Reading 2 marks from part 202410_767_767_0, total 1 rows starting from the beginning of the part 2024.10.22 02:26:15.517016 [ 1420 ] {} S3ObjectStorage: Objects with paths [common/rko/zdsyldybzatrzbqtfpeziuzlofdmh, common/lnq/qqptijmofrijpgskicblsdgdaqqnj, common/aek/yognvwmmkpvegsynxoukumgiowknf, common/wrl/zpbvgiggmzugnjfzpnahghcgzdcip, common/cvj/ytricmlfgxqgrqmhwgwusiwgoxvjo, common/efj/zllnecrdsxgikfqiuicjcdjpumnmy, common/zkp/sxrdwvxozyesfumbioyfsehyiijui, common/yhc/yuwemxugjxcucyfnijiuzgesubrdq, common/bkx/jtyiwazytxeelpiqzesjwhwwzpivm, common/ytw/chbhoewkejfseiqnbviqvwxhcowru, common/lyz/depskbexalcuaoptzwmqomejneokc, common/jie/pdwhjcsaypkyeibugqysjrllczgmn, common/vtx/rmuyjvsdsdrdtuggjwijdrpffubnn, common/ioz/xxogrldfhxoahbhxshrsvlmzxafgl, common/wwn/gbinsssllchijwpatqlashclskbir, common/mhv/xkwjdushldkpummirdgcthuuuwixy, common/qae/axrfbaxnazrdaecsoollqurtuspez, common/mmz/lynfpzwkjdxwyoxvohmceekujoqji, common/dyz/xvuklitlaouhdmdkwrxhvzwrpjfta, common/jho/nctpxkcgvnytavyhhgqgpxgcwheqd, common/unz/dfnbazlohibncnbubfzwqeknmxkzq, common/pyu/wgjmjwrovgiwrwkedaapqgqjstaic, common/fns/kvfdnnhmqoohqakmjaxvpnsukzilq, common/njp/uqnblqdaqudowgnjqzxkgqcxqtnkp, common/obh/atvqyizphosxvdnlcxscnztqhkulr, common/ldj/wofdjvxdnrlrurkrgaldmmhmreawb, common/wel/tjwdnzmxppeacdxyfmacsbgfvfrcm, common/jep/xxdjqcmogdhtrfajkktuurtebdpgh, common/srx/usneprotynmenorbuudkncszugttk, common/sop/ewhxvqnxyiheazmxbbduktcidsfgf, common/slg/qkoimgswsrslvpopmyjhmopfloail, common/crg/odrrqpsziiynjzyqlesbfgxxjmwfw, common/xyw/fshxrhfpcdxpdlkogeadnkkfpudwo, common/bcq/jpsxrrutryzyhrvymsyxlcvojbdhh, common/wrz/zazfxnxgpbnnfggvxoxbbthzqwnse, common/kty/brwfkoykozzhpppmimplludiwpumf, common/rtu/bxnahwhnvgzjzqubpwysgzpjwqrlb, common/ypp/ffmqyviyqxffcwvcdbrwxobjxtrry, common/ttc/nlhcamdnwfrawpgijhbiactrvghqw, common/kkj/qnbewnazhiqxhddhvibtbnxabphxq, common/djb/majmqvshkfpsgqzazntljohxbdsqk, common/iki/svjoztjzqedtyzwumkvlpziqbfwsu, common/lvm/chrgcopxwvidyndkcbnvaagxcwzrz, common/wrk/eblztgtkpdblhjmtdbqbxjlwbilax, common/vjx/xnhegflcufmhpghvdpljlzirujsek, common/atm/frguigvjmaqmwxkuizzxlnlmvynba, common/iyf/sidzyncffjkdndcftioekqhpogzyn, common/ohb/mzobrwfrtfwfqjqgbdoaxtuxuarlz, common/qfw/pdetredrjfufhhxmngzakpvvzbwqu, common/cvr/mmncblmhvpsbcnjqhpgvbdohahjmo, common/onk/lrbsfruzyjurltghpfxnfzceochko, common/alx/lrtbygwxtueyibcuzjkwiocjshcva, common/hle/fjtaonsvnpvuwbugaejvjdantyiyb, common/ozv/edowyntzcafbwpexzuojvpnzexuxo, common/gpr/mhtrebvrczoxhshpglgkqhnbyqgdh, common/nib/ybcepiyhjfshoqdgwiweqkgfmaijp, common/dtm/sworepospcdtaloubqaxkrenpeeij, common/wqa/kpfnmzdckqxaoraclqodyzksxyljd, common/rkg/plkanlksyrcnhmhldbshjgfmtunyi, common/cjg/ukjrdsnfguqsnrssudhliwmnrffie, common/fkr/eptajewmfvcktfrozpzobzjmstsia, common/ely/uzymfirxrpmfoswlbuvcswevqziov, common/uqn/qorcfecnrghapladmtyesatueheas, common/bar/pxpnmehkfpoyvfuppmyffzphjiuee, common/ujt/ahnskpmvhefnawodkeqlprmwfdman, common/mvl/bmxsgdyvmvjuyskxdggeoyopxyvja, common/uur/ojnhdaeosjhaxfbkssueetykllitc, common/kkr/ovlwtaqbqrmdcjqwuewwkvklqiqmx, common/kea/vaeojqgxqdvugummwtzzrlgjprkak, common/uqu/brzczoumgxmrtigoseuqdxosstnwi, common/hwn/lpatprnlvtfklfpfzyvzfrporyeyf, common/dph/vptdkxgouvhrdvovwhvsmydnuvkgp, common/jug/tlgixmpvyjusaicdpiktwpfzboela, common/xof/mdlwmcvcxfhvriyhqlnlymnqwwlkj, common/kuq/wpeaudkleleoruwliwhhqkshtqyur, common/ned/laeieyzsioncpoigfetjctadkpjsy, common/kth/fmxkausuymkfnmtokenvpqixpiaqz, common/leb/wubcfwrtqiajdxqxkpxbmhputypke, common/kyy/fexnxpfayxovwjceobhuwdkfmvhfw, common/cpp/jjlklyuezdggbsxlvhbptluscxhgb, common/ncn/rravwtipmkwpwxlobsguffeomuksy, common/lmd/clcwjajdehrcfakigpllbjfedzrzx, common/tuk/fpusnddiagfublgfiogngttiebvfm, common/qxt/arekcqnsitmsbzuouskmcaedomoxj, common/ice/ocmqjkdxishlkjhmplahescaywpah, common/mfp/ieypvsutzwecxhlbijyvlrzouwain, common/vgd/fswijuuizxabbijyshrsytqvvyini, common/rrr/djpuardwlqunnacbuoggfylmatciq, common/nnv/wmvewtuoupqowrlidtxprsgwsagxg, common/wmg/bknidbjnslluxafixjvzhbcsztcwf, common/ych/ehiqxubjaqaocugvqxgmkfyteaunh, common/zjf/dfxwoyytkvvfkplkiwdfobtdrkscz, common/uof/ipsqedjykuilmyawgdvhuokjurgyb, common/kka/vvckzjefbmtqvllhybddjynjztvnn, common/lfn/rzrwgfbxbsjvfgyotaoyzbqvgjozb, common/zzk/tjifnnpekobonfuwqcjbmoglkvuja, common/ajw/xqdjmapkuqrloenbjoqfenkpkmilv, common/yes/ojdvmonwikwpetuqzvjujokxqxjzi, common/qhe/omfavvbhbhyxamaxwlgwyuzdgggcr, common/qxq/nsbfllkstvamkenttexzhhzihfeit, common/lhp/bxlaetwjcnxrknqcwxkarmbontwsr, common/ufe/qodzwanwcowgbtkwnmfrjxaynuyoc, common/bep/yjdikvnnmyeuzkapalulzjzqlcpnn, common/osv/rnphuocpmyyttebgqbnufbpleamth, common/qov/oyesfzgwzbekynogmsbrevqofmwcn, common/esp/lmhpwdrijlmoblkaswxtxayeffuli, common/nyv/rhycslpaupozpozcdjuswiebmhahy, common/fxm/wjbptlzffwqfttgrrurkvnethrvko, common/hzo/fhccajzvtmwnedtgddlkapxhmxodn, common/mpn/xhvaflqpzrebpqzfgnubyuuwhhkcy, common/foz/hlcrkzeosqdbgjevbzodzrpocgxut, common/xvh/tmhckwuiguypcitqiitnkmmlwepsf, common/owa/ivydvnjmulbuorzqllmzguyaxnmyy, common/npc/vgzkakwvmqnlbdregrdxacarluquc, common/iqx/noqhblyzuxwaeosdqsmnlfismahyv, common/ics/fjsxzrnuakjvguebrgjilvpttnaam, common/bbs/nmilbenyxojrydbhpccwdionrndbp, common/vjk/rxndjwzaruwjusfpvaghiwcaxjnjm, common/yws/eorklchcfmtvweinvyyfvsypnxdng, common/lfu/ojuozujxyakbwwoaqqnhkwlozzihe, common/cit/dieocmqsrmeqeozlsnreaaeypbbmm, common/aqg/ftkaewxcehvqliipkguayqityzhou, common/ayb/uyznfepfuacmaawkpflxlfuvnehhz, common/qij/gmindbiuzfqsjalwwzaqwndtffgeg, common/psv/xwcokyamcdvddrahxcshakmunompc, common/jtm/behmjtkwzlwqqlicbxefoufwutluo, common/bqp/vvvdcnxchenqdjyxfwmsxprumreee, common/enp/grdmylassyowqhfrlyxxihkxpnanm, common/afb/htfwibobkbthkkuyxmtcqzoilajda, common/ort/uuqcukwjrukpeyqeklttbxzevfghy, common/vbf/nhlqkxcstwkohjixpowmvcrhowhgg, common/txg/smjuidyryxqjmuogfcuxhwwxdrahd, common/ztl/houtyakpabqrewqtgfunomdqborlr, common/jng/ghokuxunjgcdilwragqdnjxpyyaak, common/jhq/jxlbkfouehphjftlsufsrfjerocrk, common/jtb/zcatqfwepyrhfbnjgnfbohasvykaa, common/yva/gkzgequqsggilrpoolpngvlinoigu, common/ogw/xqdxdbdixfgwlgmmznhzduthkmuzp, common/mgz/eqtttvqjuggjncfrqccktrghtpqqz, common/lmx/zkbcqqhytxnidrzhnlqdrrgmcvojf, common/dwp/puzoztvantxveupofepytndpncdkt, common/anu/xvahhzuuzzcujzsqcnnjsbhtufliv, common/vkh/wonplnofdvikkpmqveukicukewsum, common/qqv/arpeifjflgvpjokwamgasspqywcux, common/yxz/kfoyzsdwhqpfwtbjcmkxrwyhqtlix, common/fmb/rzeufpfcdtppzdkbqedplaazfzavg, common/uer/dyodrgvyycrwyvcduazbulghdixry, common/okn/iipilpzitsnuqriwpofvurcofmwoq, common/rng/vhkgsdlkxkrnfmqyaoyuswqsxhmml, common/gzc/kygdpohqnrckjdjsgcpyvpthztrpj, common/bjn/bdkjcylcnkosppehhreclkpytgcsw, common/gcd/tmtutejpfxvkygdydunnpggqrkatd, common/hwv/wkilssyouajskecysbiwqdeycfnsi, common/xsj/uuvhwgbjlmdgkjgqxomsjgwyhxwlh, common/nij/ylwhlvfjtxgyqcjeurovlaqefalsb, common/pzn/ubekfwmrmzxnbuxwfpftgihdxbilt, common/kbs/nwkhhljchmvrtckveyynfuokrptaq, common/zyf/cquavpbiciecinvsyqwdzqukeumqq, common/ynw/jcxtywtporyzzwaygvxsbyfpmnwic, common/fri/szjkqieaxejoocbqygltqvdqhhamk, common/bqd/vcawwlobmniyryutjdfblujcjsyoq, common/tpg/qpdmhsrbhhbjswjktqamgnqcizpfl, common/dlp/itikjsrvyyyfjhtaqcfialqhulcsf, common/bic/qpknvupqfobdqacpczuebdzeqodps, common/ddu/tlvamruqfequrldemqaqepmvqgzth, common/rgz/vjqatirtgqdstyhdoswgvjdwhopgh, common/juo/vhfplrfekldnanuwdnotwzymntbqg, common/xsh/kvgsgwhrijeyilavodkofyoxtjtaq, common/ypq/xedxluaymxpdlqbjhkdposzhjsqdy, common/qin/cbhujjrecvehpkeersuoeonnhirsx, common/ttx/hnlxzplmkevrisryfqenbtogkvwpk, common/hfp/hohkdlfqjseilmcxddkfhzgmcjdpg, common/uvw/wotugrjogywxhwragztdqiclejsqn, common/eyx/zczytkyheyzciwodbnsgqhaeqansf, common/jrt/lqmubzbjvkgtuekaejjzfqyktvuie, common/chd/vndhjacflqhrcqhgfhtrtmecnwvea, common/fza/sffgseczwcqsrznrbscpttsrqgkxy, common/ory/qorciwqucshuafbnuvyfebqwixcuv, common/egk/buykefztonmdsyetomqrfqdbmbieb, common/epf/ksjjgfyphhjpjmgbbkncvwklcczce, common/uag/vrsicqkircxtngsltjlwfvtgpgeel, common/zln/efxtdlaxkznseiudgrfuvhsesqiqf, common/rwv/qqhxahkhtgurtdurjbdnxchzrusgk, common/vav/uxmsyxfrfkbiruqzbfilfswmvmiwf, common/vgl/dqxlkjxusiycsgbavhisgpngchrvn, common/rpq/gefyuyaogieytplgwkqvslkmtzrsf, common/agl/fxojddmtkpsutmdkocbyfzfdcbifz, common/okn/yzkauzmefgnipjgxazthwwilmckpw, common/xbm/lielhnxdoegprhfavepsvhbmdsdao, common/tqm/imiwpbqdscvujumootdrgctwnfyyb, common/zfd/xcejukeevzqqvvgieacbmwlgmiykh, common/pqn/aiskygemuqdwhtdxyaoeuirefbcfh, common/wki/xfsojqobqllseedsndyisswrqlqpf, common/mtt/wttgceluaypbjvnudyngwkapswbhh, common/viv/vecpcqatigaqujwqqcltdlvxwyscm, common/cna/actrnridzmuhsydplmcnjcygccivx, common/ojp/dhnbbclwjeiphnbuyskjjoyoxvqkz, common/cwi/grljzxhowhnndjejqhdkwxrznhpdn, common/jwb/loefqcpfticychemdsojbsupsuend, common/ctk/pyucpbtvdxvafeveiabkrncrwlaep, common/rru/iqiswaopmwwegakeksszbeibmipcx, common/gbz/lfzgcvqljjvnudfsyzuiwwptpujeb, common/otf/hdkbucoiziemdyifmehucanlemrqv, common/sdc/yloetdedwfphstkgvzyywfoikacvy, common/gsy/sxcdfqwcstsumiiqpmeexeqchrgti, common/ujj/mnyhwdurqyobqivqztcvezaimzden, common/ffs/ajvmsfkygcuegmrkynpfyrwulfeiu, common/imz/prglqggtdbxzlwtcenaqsrugxkvlx, common/sah/qckuxfkwqklzrneqnulfqbzmdnsiq, common/whe/cudoyvdceeurrhiavkleziarwjtdg, common/mht/tsrgammxpcqsnymaloifhreznmmub, common/nqb/xogcbjhkbxrfxxqtwykfoogorygni, common/eet/ldwvdiyhwcrgnhmasnugtshusfkcf, common/pjf/knkqkrnbnrdghdxwgaerusltydxrm, common/igr/sixxtogexcinmflnfcvydkmfhcdko, common/rjq/eomcvxmzugcqscokxjkcfeyvebedb, common/jvo/mrpfogxcootxxyhufulfrjxphazln, common/xtj/condznkjyguauppyrmilxqsqthqrk, common/xoc/wqdalerkmrvojskgkjhtzmhdypnao, common/pxb/ewfujjqsmzhrxtvfwliiuvechzneh, common/glw/asyalxmsgpddbaobfiszymrscibfl, common/nbw/gxtbddxhwxnbasqbkkeifzrhncacd, common/mwp/qrihstxjrzumlzcxstfpdwvgewxhk, common/fue/hhatjvfdwnhpmptbaqhsngjecoqiw, common/ugc/djfnsjbohxbvboxrshackgfnuixos, common/fem/eluccgqyckedgsysajfqxfldqweoo, common/gwy/ioijpahdcfsclaeowvnyljxwtpgbq, common/oxo/xssyirrcupwbepmmtorwgaaajgwql, common/zpm/iqltejavfkifkyjczwfgcytygdwks, common/neu/korinajeoarxaplgerkvzvsvpbdlh, common/joo/fnqdxotexknpcwqebdhinleqrpvuo, common/aft/rodjapoyrklqeuvsetnhkgfkcjxwc, common/ngw/ucvjyvdlmhkvrzctympdssomkamzg, common/eja/hpjotisvylekmqhmuxlrtxduejapg, common/lse/jrvzjrggprafradbkyhfuhvgoqmpx, common/gcv/ipztqfzfurhbjylclrvwwhdyztcpv, common/pct/rxhzlgqscwuxrxezoeerxwtytfomj, common/hcw/syvwxvskaeohjgomdfhwequhexmqo, common/uyr/ptrlylydojqbyvxehnwyziyqryqna, common/rfc/gusdccnrkobifqnowrugybxeodzut, common/ief/mesgyynfjjcjdybthanrnipzcdwol, common/keb/syktlzyaastfsoumqfuwnltxtvzkj, common/fmq/pboboiiibynvsgtsdbdemfkrgycoq, common/fjf/llcnwfruxiastrasyukhywumkmpgu, common/wfm/nqebvmfarmgtbsvnoeqwpokyddvei, common/sra/wgprvjlltmhjonzkryuisaxanitqw, common/nwx/dsucrkayalajenwvunehligczavdn, common/rij/gtzafyzytowfdpxptxswrxpfhuhfy, common/ylm/yigyfbkrfbezduubyssjhxhxvqqou, common/yie/lgijihmgiusfoxljueounqhupmzfb, common/dpz/zhgoweqhkgggblxznxjzfidzwahmh, common/xds/ihkuksqpuuemwqxszeuwolgaxxxuf, common/mhx/ystcrkpnfxwssurzahjovtxttmthy, common/cru/vazokoeqwmdokhoufaqvomnwsfynq, common/yal/lfgnooowvtrmrfypluorvbbvavabw, common/wit/ztgwuzoypljsqagwfxsompqnpxuky, common/khz/tikabwosaghqglpknwiavehlscaes, common/ugs/nukzhncnnzsriwjbrdoxrnhnpjcyx, common/cik/opfslbswvekekwoohiajgsdxmtqjr, common/wmn/bqmcvbpymamdohvhliothurgeeukj, common/ixo/xiimqyqywxuduiqhipyhbomvvqura, common/uya/swuzbdqqvudgwfakgjviaherbbeyk, common/bej/gkehvkkyzlqgcnbwrcwhpwhmnjfjb, common/wwd/vfmfdzllmbhgfmqrqrujwaaivxucz, common/fze/feuskqrfdxsqnmdzgmqqgqubljtjb, common/ijm/eqnipjoqqtthmpdgisesabrsruzhp, common/okh/eyczkqattkdfoboipceirxjcxhafp, common/nor/frkycjnmzyspnmyfmzqrxnuidvcep, common/sgc/ztlhkkgeisqzrpbskvxsigdjzhttt, common/fry/lkmrtumsnosjefygjfzrrakmqlwuq, common/ggo/kpduojlzqygmbdflbwnwmanlfcnau, common/cou/vynqhbsbnmattapbogfqusdbpzwrc, common/etg/lhwiadhzzvzncignudxqeslosrpvb, common/gej/fduykfprdcnvwtbgwpfqivqcrwoka, common/cbs/thotbkmpngdupgoicyudknyoakugg, common/xep/pwxcxxsdxmwmrwflobblylcvbmehn, common/kpd/paswfbzlgwfoeeflwrojxzslrjiqa, common/iia/wecpfyagfhdnyvcmllnmkcvapnpij, common/pps/uvoplmhoqiayfvsrfhqwiwibdqper, common/yfs/wzmcmcnfudpgvsmqbdblxbzaxnqcw, common/ufa/ovpqtvgncbooebcxjyzagwdxmunwv, common/fav/fmwnhmbyozfgdfwzvwskaosoyohdf, common/cle/ecxerhrriuzsyrhmphftnqkrhwwhy, common/rlf/zsbrtktcgrgopfzjonujgaskymdji, common/pea/lqhllpmukkjullpmksughnpsscpni, common/wse/leyrsyfxfvkitljlshavyjdbrwjby, common/zmn/bpifsijuefdwlyhmqtnqgpdioqhfb, common/apm/exseorjqvazfsatvfleginmthebws, common/vfo/bkamwpbgfxwtjbfpixqjlflvhwkba, common/ekn/khtttvmowoyystvkhpalzymlflxmo, common/jcz/tnzxzaysloskuwhrqhvmcrqpmbkth, common/yrz/bxyujhmrkikzsbfngjmvtvnrbjxgj, common/glt/uoczqwqmpotxaomzpibnqbhybhsjt, common/afm/dliutzyvqckqbyhqcjsizqqymdyrm, common/mvk/yevyvswpmjmxdzewidyvsgmmuafbw, common/nle/acpkaebchgdslvwchozpfdivnfcyn, common/isj/pyvxdhmeeibrelpqeeoointmlyniz, common/gih/mfqzsvukxmybvyxqvqeiqfehmaoxe, common/msl/fyhfmbnnhtkwhcrlgstzjtwdprdot, common/tfe/llxjxszwgzterksytibxuwmaywter, common/irc/zlhtkfdiovprjyjxhwudabxuqakuf, common/gtk/wigfxqambequourjrxzsajtgfzzca, common/fns/ewrifbzfmjnjumogxmnlrfeygsnsu, common/ogh/taxycxyekzzxkvghlsupdyutexdmt, common/wny/zbwimcseiaabcsnvjfoazlbiyzteu, common/yat/chlanwvqmkgadxcrbeloueimrysls, common/hjr/cqtpbzhihhvyhelahhlraqdpfwlgo, common/gbq/sgpdxznbpltggdruzapfxqgukkikg, common/zev/fxmscmczrdukhlkjlfikovaisxlkg, common/ngc/olknksierfdggqtzhszienvegbjmx, common/xbp/gjwwjprniuijefxcibyazzvsqawod, common/qjg/wegthottwvsjsqxywgbwatkjzbnjk, common/sld/irirxyppylqtfazishyekdkqttolb, common/kju/fqxzuripitupplnllbuvdogwbbsht, common/sbl/tvsltytqzhchseaxnalqwyhktywtc, common/vdk/vcmjtavlazfapzjxuwupqyvdyjeog, common/mkd/asjmrsnyyrwpweajjxumqukzmjanw, common/npr/qafstglsmraojdttuiafdqykyhtgh, common/uem/qfsxqzbvyimeryiezlfkgjztzcalk, common/sed/ejzjjouxkiiirnjgcixezkdlewypy, common/qln/ehobmcveaekyvqsllrzumzmzwxmhe, common/sib/ggapwbeplkofvcotsponvbbsbquuo, common/unl/rnbjldpvozipwfarglzqaxtevowxj, common/jqt/vvinvakkitughhxxurzbvqwjydggj, common/bcm/bkltvtodmkewcdoimfjqxzaaclrka, common/bpw/fzkninrvcfdhxqbfeavozidnlvsmb, common/vsl/haptzumqaqimrwsmxeowsbbzvzann, common/uqz/bencyhxaxtdttysaefivfykvjxavd, common/vlm/ddmontrpbwsgiyqurpvfuxggjhxju, common/zor/orviqrsheffxhuxjlxvddzcktmprj, common/ohz/vtofvadeinmqyjalnelnkejsmnklr, common/uit/ukgpbdlkreyqcbwdxusojxxcmgqej, common/mwt/kcfinrxrfvmesxwlajvkectbhhexh, common/sac/xihyplxeuqajuqiyayvcvalkkazxm, common/lnh/fcggcrlzssrvvdorvoujkrnwvvcbg, common/bku/favuvijtfuqhlowfdjgyuomtbrcjx, common/yxx/fkzruuusrczekmkzioifrtsdmvcfd, common/dko/optfscrgwunvwqczzvyjmzzacatvn, common/tia/quhdwfhismfjqdrlnallnhuczbgup, common/pfc/enuhrikswzqtbqqikbcyoqajxyqzk, common/xye/qmfyvstngpcxgbclrzlpofnyktbvn, common/mtl/owpnivpukxeiamxhldxwiokcsshki, common/pdf/umgeibdpuozmydaprwjrbfqmkwfyz, common/azp/ijrrsxkieqmffkzuryqncykmiyvuu, common/mew/tnzlqrrvxsadjaauqliyhnxrplvfw, common/wbm/dkndqczgrnsorxufbqepihixivxbl, common/lqd/ogernkkshacmrhttzscjpicyzufts, common/kom/onlnaesouvezpwtcihypyuqaijaqf, common/mii/fmciobnusoakxtogmomfuxmicpcxh, common/cqn/edqqgrxtkvlprlffqnudbttbzseff, common/sta/mapfcpumpthyyoffmnzckkuvgloig, common/vgg/ulvbyyuldgmowqowbzgxcovydixhj, common/sje/cyydcodmawjzyfcqrmvritcypkyxj, common/osz/rsorqmlvwbskfbuibcptvfkibmffs, common/fap/wsninwrlzwmisdpkogsaigzockjtq, common/txu/vvirwqmdkxuxkhpnnynsznrzcrrup, common/uvg/vgiucangefjchixzyrdkcxgcxzjit, common/fud/rvydlhrtexqbmpzoljuljulwygnde, common/vrz/kbeoutvwvunfiukqwtaulyulikxwb, common/noe/lutmnkwmxgzowcxnhlylxiacvwtmc, common/qfx/tcfmmxgzewrcasrubfoljftfogdln, common/nxf/xqggoubbmvueioxsihkimizntkdrl, common/lcw/mgyeobbkxrjnmqejsdnuoketxeioa, common/zby/ysmmmobmcomhehwqzyscgniugfzul, common/rkz/pepcrqtepypqfkxbsegksxjjsnomv] were removed from S3 2024.10.22 02:26:15.643968 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 8.66 MiB, will set to 1.56 GiB (RSS), difference: -81.65 MiB 2024.10.22 02:26:15.686005 [ 1420 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CounterClass.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SearchEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CounterClass.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SearchEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WithHash.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/WithHash.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_21_21_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:26:15.694649 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} MergeTreeSequentialSource: Reading 2 marks from part 202410_768_768_0, total 1 rows starting from the beginning of the part 2024.10.22 02:26:15.762513 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 257.85 MiB using storage policy from min volume index 0 2024.10.22 02:26:16.553503 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 14364 2024.10.22 02:26:16.576419 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/brc/fqgbmjrrtdmofuddqgjfwfiekmddt (LogSeriesLimiter: on interval from 2024-10-22 02:26:10 to 2024-10-22 02:26:16 accepted series 1 / 8 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:26:16.663709 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:16.666635 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:26:16.916927 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_936_936_0 (state Temporary) 2024.10.22 02:26:16.920619 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 5 similar messages) 2024.10.22 02:26:17.058694 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_768_156/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:26:17.114722 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 185 entries to flush up to offset 102457 2024.10.22 02:26:17.149788 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_885_254, 202410_886_886_0] 2024.10.22 02:26:17.153344 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_936_936_0 to 202410_949_949_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:17.154993 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_885_254, 202410_886_886_0] 2024.10.22 02:26:17.167673 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_885_254, 202410_886_886_0] 2024.10.22 02:26:17.176392 [ 1277 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_944_309 to 202410_945_945_0 2024.10.22 02:26:17.295373 [ 1277 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:17.299709 [ 1277 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:26:17.415352 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14364 2024.10.22 02:26:17.469247 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_945_310} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_944_309 to 202410_945_945_0 into Compact with storage Full 2024.10.22 02:26:17.535837 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_945_310} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:17.548434 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_945_310} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_944_309, total 14396 rows starting from the beginning of the part 2024.10.22 02:26:17.559893 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_945_310} MergeTreeSequentialSource: Reading 2 marks from part 202410_945_945_0, total 16 rows starting from the beginning of the part 2024.10.22 02:26:17.735683 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 9.91 MiB, will set to 1.56 GiB (RSS), difference: -71.60 MiB 2024.10.22 02:26:17.819922 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:17.827817 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:26:17.895160 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 105730 2024.10.22 02:26:18.014925 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_920_920_0 (state Temporary) 2024.10.22 02:26:18.163869 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:18.168182 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:26:18.190306 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_920_920_0 to 202410_926_926_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:18.253624 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1578_1578_0 (state Temporary) 2024.10.22 02:26:18.282491 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_945_310} MergeTask::MergeProjectionsStage: Merge sorted 14412 rows, containing 12 columns (12 merged, 0 gathered) in 0.820011294 sec., 17575.36768755773 rows/sec., 2.75 MiB/sec. 2024.10.22 02:26:18.347665 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_945_310} MergedBlockOutputStream: filled checksums 202410_1_945_310 (state Temporary) 2024.10.22 02:26:18.352730 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 102457 2024.10.22 02:26:18.380987 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1486_67, 202410_1487_1487_0] 2024.10.22 02:26:18.384650 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1486_67, 202410_1487_1487_0] 2024.10.22 02:26:18.406351 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1486_67, 202410_1487_1487_0] 2024.10.22 02:26:18.422507 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1578_1578_0 to 202410_1591_1591_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:18.428875 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} MergeTask::MergeProjectionsStage: Merge sorted 812 rows, containing 32 columns (32 merged, 0 gathered) in 5.097965525 sec., 159.27922541218047 rows/sec., 12.01 KiB/sec. 2024.10.22 02:26:18.492934 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105730 2024.10.22 02:26:18.585207 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} MergedBlockOutputStream: filled checksums 202410_1_768_156 (state Temporary) 2024.10.22 02:26:18.587117 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_945_310} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_945_310 to 202410_1_945_310 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:18.609629 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_945_310} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_944_309, 202410_945_945_0] -> 202410_1_945_310 2024.10.22 02:26:18.655928 [ 1191 ] {} DatabaseCatalog: Execution took 308 ms. 2024.10.22 02:26:18.663791 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:26:18.963645 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_768_156 to 202410_1_768_156 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:18.965785 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_768_156} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_763_155, 202410_768_768_0] -> 202410_1_768_156 2024.10.22 02:26:19.120265 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:26:19.318938 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_898_167/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:26:19.361355 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.68 GiB, peak 2.00 GiB, free memory in arenas 9.54 MiB, will set to 1.56 GiB (RSS), difference: -120.48 MiB 2024.10.22 02:26:21.011334 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/hlk/ownsabaaotdxbywjohcirjbkfxhvp (LogSeriesLimiter: on interval from 2024-10-22 02:26:16 to 2024-10-22 02:26:21 accepted series 1 / 200 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:26:21.375060 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:26:21.377173 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:26:21.475296 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.71 GiB, peak 2.00 GiB, free memory in arenas 8.84 MiB, will set to 1.56 GiB (RSS), difference: -151.79 MiB 2024.10.22 02:26:22.255667 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1778 entries to flush up to offset 1359148 2024.10.22 02:26:22.522979 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 30 entries to flush up to offset 105760 2024.10.22 02:26:22.749280 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:22.751918 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:26:22.828498 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1579_1579_0 (state Temporary) 2024.10.22 02:26:22.905400 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:22.908906 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:26:23.029185 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_832_156, 202410_833_838_1] 2024.10.22 02:26:23.035352 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_832_156, 202410_833_838_1] 2024.10.22 02:26:23.065210 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_832_156, 202410_833_838_1] 2024.10.22 02:26:23.083582 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1487_68, 202410_1488_1488_0] 2024.10.22 02:26:23.084252 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1579_1579_0 to 202410_1592_1592_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:23.089732 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1487_68, 202410_1488_1488_0] 2024.10.22 02:26:23.114508 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1487_68, 202410_1488_1488_0] 2024.10.22 02:26:23.133774 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_897_897_0 (state Temporary) 2024.10.22 02:26:23.158286 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105760 2024.10.22 02:26:23.210977 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_897_897_0 to 202410_904_904_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:23.361680 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1359148 2024.10.22 02:26:23.356616 [ 1228 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_899_899_0 to 202410_904_904_0 2024.10.22 02:26:23.394970 [ 1228 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:23.397641 [ 1228 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:26:23.536419 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_899_899_0 to 202410_904_904_0 into Compact with storage Full 2024.10.22 02:26:23.560034 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:23.564100 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_899_899_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:26:23.569467 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_900_900_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:26:23.576697 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_901_901_0, total 1338 rows starting from the beginning of the part 2024.10.22 02:26:23.598611 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_902_902_0, total 1357 rows starting from the beginning of the part 2024.10.22 02:26:23.617276 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_903_903_0, total 1292 rows starting from the beginning of the part 2024.10.22 02:26:23.645054 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_904_904_0, total 1778 rows starting from the beginning of the part 2024.10.22 02:26:23.852124 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.75 GiB, peak 2.00 GiB, free memory in arenas 12.46 MiB, will set to 1.57 GiB (RSS), difference: -190.77 MiB 2024.10.22 02:26:24.188871 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(41cf368cfc33b47ef4c3347d020deea2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 41cf368cfc33b47ef4c3347d020deea2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 02:26:24.195652 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 41cf368cfc33b47ef4c3347d020deea2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1386, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1386, kind: Regular, unbound: 0 2024.10.22 02:26:24.203998 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:26:24.766658 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_886_255, 202410_887_887_0] 2024.10.22 02:26:24.769012 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_886_255, 202410_887_887_0] 2024.10.22 02:26:24.783671 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_886_255, 202410_887_887_0] 2024.10.22 02:26:24.829307 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} MergeTask::MergeProjectionsStage: Merge sorted 8845 rows, containing 4 columns (4 merged, 0 gathered) in 1.29994026 sec., 6804.158831114285 rows/sec., 106.31 KiB/sec. 2024.10.22 02:26:24.874054 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} MergedBlockOutputStream: filled checksums 202410_899_904_1 (state Temporary) 2024.10.22 02:26:24.918608 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 14379 2024.10.22 02:26:24.945496 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_899_904_1 to 202410_899_904_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:24.949791 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_899_904_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_899_899_0, 202410_904_904_0] -> 202410_899_904_1 2024.10.22 02:26:25.068828 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:26:25.258504 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:25.260534 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:26:25.304959 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_937_937_0 (state Temporary) 2024.10.22 02:26:25.355386 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_937_937_0 to 202410_950_950_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:25.460022 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14379 2024.10.22 02:26:25.543335 [ 1254 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:26:25.558156 [ 1254 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:26:25.606684 [ 1233 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_945_310 to 202410_946_946_0 2024.10.22 02:26:25.692203 [ 1233 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:25.699588 [ 1233 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:26:25.760781 [ 1233 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 358 ms. 2024.10.22 02:26:25.763006 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_946_311} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_945_310 to 202410_946_946_0 into Compact with storage Full 2024.10.22 02:26:25.859770 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_946_311} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:25.889017 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 93 entries to flush up to offset 102550 2024.10.22 02:26:26.036351 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:26.042409 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:26:26.087024 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_921_921_0 (state Temporary) 2024.10.22 02:26:26.085694 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_946_311} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_945_310, total 14412 rows starting from the beginning of the part 2024.10.22 02:26:26.121632 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:26:26.147768 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_946_311} MergeTreeSequentialSource: Reading 2 marks from part 202410_946_946_0, total 9 rows starting from the beginning of the part 2024.10.22 02:26:26.305735 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_921_921_0 to 202410_927_927_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:26.360735 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 102550 2024.10.22 02:26:26.507908 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 13.95 MiB, will set to 1.59 GiB (RSS), difference: 5.46 MiB 2024.10.22 02:26:26.753064 [ 1420 ] {} S3ObjectStorage: Objects with paths [common/djd/ucubxjjiouahkomoyozqowrykxfmf, common/nnj/ukgthnelvbprfiysqifljulpkktom, common/wma/kjtfboptsknylwnejnruywerlesaj, common/fht/qsqeurbqwstvxnmeqhabzinjudouf, common/zag/voodseharyifccodhqattphxkdoqk, common/ihz/banqhaxcahkiniwedsraulijpasjo, common/mlw/voltkxdqhecjfwfvizasmfnkevxsg, common/cql/jofgbcgjdyavfewlksxjbmkgtbtsp, common/jvx/durpbheyxuirbhjflzfabyrsjcony, common/bmt/qgpueprrwkaxpidgllotzmgbcjuak, common/zhi/xguanjaiewpdqxnggndksuaeiggxt, common/qit/fcxylawdfzaetwbdppglrouozhiky, common/wwd/scyjavhofcfsaxvcnzfczegsrjihq, common/xtw/ftyyjlnvjvkmilxhhaqypijhhuagd, common/kuu/vklofmowccreafdgswzejiyzeslie, common/dda/nowjevnygjcbukqcdqjrfxjecbvip, common/lji/lnbjripdvekosdrllxasbhzeznbij, common/xpl/wimfvlqmpqcxvjxbwibvihupcnpce, common/aok/yybytbmgetlwacbpvgdjeptjobwrv, common/dxl/llisbfrubwplwxuamittxjxtagrsd, common/rcs/chhknsczsolylxtpqndpjpaztleww, common/yij/cbacnojaaesunxnbprgzlzmelrbbn, common/fyd/lloohckxwlypcpqguvcicpbvkhbcl, common/wpu/oagvenyywlifesrwprdyvkdgincke, common/whp/vmbmdqcqbdvlpevtivparayhgxham, common/xzw/mablogrkreeyoomqivmtztixmuowv, common/htl/qblgmgiwiulwokkiygnasedezaudu, common/bvs/kirvpisbopnmmsftmeaujrhstzhsu, common/mzw/crhcewgtpnddvkhbatdqgktmyxlca, common/luw/fbhbiqlitkbvntoyuyukxtuomstnx, common/uqx/ylxkjkonsjijqoqtumgxomapsdwlq, common/upq/utzyqappsvztjhpuifonlqtthztxj, common/mvn/axjogaewicihbygycsajwqrafyxsr, common/mwb/txpsxhhetjwrxuwpfngfypftywsaz, common/ypz/teilzfwnzkobnabitkqeknlljqevw, common/ajo/ddewvuljgjudloibsmzydeddjbeic, common/xti/knrshdgoeugnhbivpaqhtskvmbhpb, common/fwi/mwilpbsxezhawcxjzyzoxglakyhww, common/atd/ndtbhyfwphcvekcumxlcdyiidfsri, common/xqt/nicgggurufyjjyrwugwpwztowfhhq, common/axm/ocprfasdxvwaxfapipbcpwbfkztoq, common/bsi/aohbocxpfpurefvikzxxvkjxuvyvu, common/rrr/kvknqglminqettovvxwxcfohnaxba, common/wfb/qsrsblktwqavggjpqzqeokqhzimcz, common/fqq/eatlvpllucwvdndsocpvanpeprqsc, common/ihu/sfjdycqxyojvosvsxedeujpknnthx, common/dqm/onecgiklhfiokdocajohuewwoivsg, common/cnp/fsgqegfbvyztycfwswfkybijlqnxb, common/snf/ecknmgxctfdkzsnxbrlkvpuyyrdem, common/wtm/wtpyflqulqapnmmeinzzqokktvdwx, common/afw/izdgvjisfnststxxjmstwowhwbrrp, common/vos/jxqpehyuvrgtjietomvqarvhsavca, common/qia/ivtlrtnwxxgcxxpjeolhojgkhuokz, common/qzm/gxrylngapkjeaeltlyshtrvqauzhx, common/ggn/qbhqyohwjqqynauqdpeihalvxtfhg, common/kro/octkldlgwvxsshrpuldzalpbftlhu, common/lik/tmfadpvuguwgqfybdsqnqexdsgorv, common/cts/riksbfmcmsaqbmgekjkiipwebdgxv, common/eyt/evascaqiivzjwrbfrsurkxdeqvjxv, common/rtk/xvkazkyvbkufsmlfwtdcmqlmxmfqz, common/lzv/myoxaiagpourwurgzkyaxnizzvubg, common/spj/mgemajmrvvstyvfkvocgfyyyrjngl, common/hwh/vjaimddfdwozlrphdvaplmegdbmqa, common/edx/kbgptoikznufqofdvnhbucrbosmel, common/tst/boxfytajroxsqygjkwlxuxitvueqe, common/jtq/vkdztdbbwlvoqcptrtvvzqvitzsbf, common/oaq/hueznztvkswziwkfpbtibekyizcti, common/fex/ndhdpgmwrccmcovyxrlbapquadwjp, common/wrd/fmtzwnhznqtgaxshomrkhfyueginy, common/iof/fzsirkinggghvnimmuzuydtivcegz, common/wqx/psweogtotausigmzebyqkmafrgjwe, common/gpc/xnzrfgtkayeeqetnrbuhgprdpjzdp, common/jeo/yabxcyowkhnlqxxegwxtzdopgpfaa, common/sbi/jpbfuqqupavclmldzrhlfzcqdhdsv, common/ulz/ybrfzbkcbqcjvmipaxpkkqqxvzdoz, common/wjn/qfoagngzzmlypkxbacedsxnwilkrl, common/zri/uapcjslqbtmgsuqdfqshfpvkoials, common/wcn/izwicbjkmhqbeikfczimhasswkwge, common/zxt/zbiwdzmywzpogygcyiqpylkgmhrcp, common/twc/jclnnsvwdwbhgeyugxzxcxptezyyd, common/eww/osmaltzsvssavyeisglvprjhpinso, common/dfi/cvlcryxertmitrvbbiwigmbwgshui, common/gmc/wnxtfpbfzjwbnpehicfozwwhswziz, common/oqv/ymkkygatjdvhxfmdkxzqinumgxzgv, common/vwr/pfrzybytlxhxywwjyrdrszqxmxdrs, common/tss/avranzmbllnzimzbqguorhnepetqn, common/yyv/irzpqarlecanobwqnllaoeoftpuow, common/pxw/jervzguqdtdyczfoaetvpxebikddf, common/swq/uvheiqvgpsgblqaqqxbssedrfiyka, common/lzv/pokukryvzikugigroceegstfuadmy, common/gjm/fapsulzvdimqveznptjhqrnmgqgdi, common/pbv/jkknsbvkikhqvujilpwioxzaffmrr, common/lgo/ceqqarvnbhmiznfldauvkiwvejcct, common/nlz/ybdzyvdauwtkcemlxcctomnzewfof, common/jca/zlvksjxbnjlsgmtpzxvvbczqmxmjp, common/qdt/braufwdrqbzxalhjtiglnlxkkkvaf, common/uqc/mcjeoqpkzwyluwzugkuyggyabdtkk, common/ahg/omfazdlrskubqkiaaydqqkrxynfiz, common/tln/yytnfvrmfznunsazhluzdhpebjuma, common/obp/htdqtetqwkmouzzikgrcxfaouxwae, common/rtl/cmvyweklouipbwmgeyoslwcuajkix, common/zqo/cishgifsitcuzuhqatpjwozheyycq, common/dqf/qqhjuzhnaedtngnsnydopvivdlfbb, common/pqm/mbwmybzbwbucltpfxhybktzckohkh, common/chd/ugrvtamiqaxyrbqbiygmvrzciavpr, common/tjq/jqkhkhzhonssuprddcmianhyshlki, common/ifh/tcajdleawdhnbpqreyhbhcwnnxgeq, common/gdy/jceadxhgyhtaorakxgqsubhmbmtzh, common/pdn/pstisxlbhliuxkndchczqkszdfimr, common/xwc/cjkueaodarsumhukcfalkukybvwbl, common/asl/xfhgevggjglwaunxrxyhlxuxjnzmx, common/txq/uqpdhrvssxyuokgvtmjqdkgpjvcfc, common/koa/bknoffekllapdddsaqldtgdhtalab, common/kbq/pvyxpatkdemrcbyvxbwcaqxqvembl, common/qeb/vvxratflgipabmaaezysegwovokfd, common/tbf/fpcfeyiregwmwiapwmnmzougjgpvy, common/pjy/zpnpiodibqeynoqwyjyjefjhswbwa, common/yzz/bgrmrhxlejaupexdydzvjdqaenqie, common/hwy/qqalwnuahmqoveeutcxlkosxibhhb, common/mmv/jhmgvjnfrudivmnlawanehcoairng, common/jyx/qvwnehsdqlhdobnacqqietyrbjgcs, common/iny/ssrerypiijhatbecteyadihjlgztx, common/owj/thbbvocinsltbepvrihrfueqshpec, common/fsm/azgtfykpzxqfminulwpqptldmgukv, common/tbr/gncfjzaojbeixltaxhlbqofszthos, common/bro/wjugkusodvhouksxspqocgrqofosq, common/nlf/llxtikevsvjfbudqkddvvqxiroxti, common/boh/ezeogrfbmrrvfurnteusawghtacod, common/zuf/pxssvsgdtlcrfhbuvvijgielabsaf, common/gvc/iicjhlcqcieavpqedmiljfotweqqm, common/hpt/bvwpavwzyirtlqldkejweftqfwesy, common/skh/jjefzeympwjjzipplueguatuxysif, common/afj/vyvbhccbuexsepqsjziipzwgqmbnx, common/zxt/zercblbjimvafazzfzotknvpmlkzu, common/mcp/eadrkefyolzbslhskbjldmdfabuoa, common/yit/iuykhqegkidxwbzwvqckutjkhidyn, common/euu/vzcrtgqwkdhaxyrnafiiyikbpyrsd, common/tvn/qafhdabjndfhdxkelkdznewnhwhuy, common/zmf/smyhktqwufgbvegjvsqaeeogvvdau, common/qii/bckyvpkwzmfpkuwkhaimpjqmndtcn, common/kha/zpckfjhyfsxbvyohdprufxtfxxvge, common/vmw/kpbpigcmlepwdpjlgkqmioqjzpwvh, common/kxp/bcavxnzimacbyhpmrxyvvhzixwncl, common/kac/faphvhqxmwecfzmownvuhvofwvwmc, common/iep/tirexiobssqtrtjgmkxveltwupgas, common/uac/rllkjuachjfjgvhwwysiabdshdvlv, common/kqz/swtvjwbzxxnlhkuhhizrpbwirvxmi, common/exr/qasypmhzbaugvzvirmgihqwrtumjv, common/xoi/llwoqclqqskfduuaapyooqwmgqlrt, common/mrh/kxljdsvzqrxccszyplvwewhqimquh, common/xut/xgjcdlhdnuiqoxjxxkxirqjhczncz, common/izu/ilgbwaexgbjjfhmfvlyfzskrigjrs, common/zdc/srqqspgeumeqfmijtpdkapbewlnpt, common/rqn/yfdlwcbenaxrlyjzizmzyyssggnrm, common/nph/rzukhrkhcesythuqnyotmbnwzveie, common/ikv/xfhowgmiotrlzfitnwtuyyknocnhr, common/ouf/ryjfjqpfnglnwttaasmafnobgpthi, common/yhk/fhvnnxvdmwhauuajwxujfmefrlusb, common/uwt/weadvtpqeysunnpuyditptlyrjlxq, common/ine/tllwkznlloiqdpizwpfpmelyftema, common/pek/gauxcqrqsfazchcyvvqaqvfucjgdi, common/iev/kphfyivgbpaiiwaokcddujhhxrxhe, common/qon/azzhpzmitkkjghxnrerlyecizxvnn, common/law/tlugkevxvhaosjyfxitfyopkxbjjf, common/mqj/yzjxdpkysggjjgoxgajmlamzyyofo, common/nzp/ajvsgnhfrpboejpmowlfpipowqpgn, common/lwm/gpnzszxzcamvisrfmlxqnjbegrtxz, common/sxv/kqzscrttlixuweyaqjzrnwzbcwnsv, common/kdj/ujmwtgyperbatkkxkowfedjefdxde, common/uzh/zuqzksbscwatvbtppscelkhdooxwo, common/mot/uickcpjmocrmrvaebcjtuhkjbqtyt, common/efs/mehdxbzujmxzqbzzghvcthcptvuiy, common/qcx/leqnydmcbzpirrwkrojegzsrqsjqm, common/llr/ybeicxfxueqyegjhxaqyjfaibcxtb, common/usr/lrescwvwcryzcudegxghwviymcgzo, common/roe/exelsnccdrfsdamktztccvlnqhfdn, common/myd/uprnhvkzukkzctcoqzamwkrlumwau, common/qqu/sfghfpczfkddstsghifzcqkydtajc, common/xkt/cpaleailysbbdahkqllmmahtpjplc, common/trc/clwlzlyyqycpxhtgwuwrsdncanygf, common/hjm/tdrlcxwjkbhummbyohawavjfsofdj, common/zmz/pmultgdefskcgvjiaitirssjicgcz, common/vzh/uamxvudrvrdphdcofwausdcuzrcio, common/coz/vrwaebuenkndsqaodflicmeosziec, common/axx/dubekortcgnukblmlhungbfpyxcmj, common/jqz/tzlpkdsvosgclhfedsfpvryxwppsa, common/wwi/bojikdmwbglplcnurcfydlisldjuw, common/xch/znzlodrxleohfjanlbervgztpqqik, common/oby/qwrrckuufktrhwrdgzmvxoewfasaw, common/txg/kwxzkravkqsdthuztwdzcvvvoqtfz, common/dul/foovaptdbgkwhgbdkqkapditkygfm, common/dkf/cbcwbgbvprgkotvropohyvdclcxjh, common/akh/psyjdtihrrgtuetinoefopctpjukx, common/hou/orfnnpkkbvrkclhvuxznezyzridlr, common/hsc/pewvxslfiypbaeqcolnhhkwpzhkdt, common/bvd/upwnxejjvuuqbueeyktbwoaxyfkff, common/gkx/dyroqenhrocslninrlwlpzuegeieu, common/lfj/dnscnjsrbepjociptwbtcwzqxtkmi, common/tib/axsjcvslegxnktdtebevolsixifbg, common/cuz/isitifrmrpvszejdgejjighjtnprn, common/iqj/psnvkieenxtezhurnytoxyzkufpau, common/qbj/etfgwovbdskhnfbjmkeeqrhpmpljx, common/wvl/iczcmqgzmlhsydssddzgezffgokoq, common/kmt/aoygbkrvwjekgiriovgdhcexbqctf, common/iae/jtkclaepyerddpsyilyxoxdrfbmvk, common/pjj/vccjcaqduakyjvxebzspqcnrxspig, common/lfb/bphfotheireqsaelxdtwzpjkmwozw, common/elj/iqlmjxpfsvbrktpfivjsinjwkvjrl, common/gvf/lvmtevakzboawnvackoiorjvcadyo, common/nat/yiwudmttyvjtgqbrpgdrfqauncatp, common/bya/wsziffbtojlwsnvvhybwqzdcxlbon, common/vmb/zfousaovsdrenpawlppqfhxhcyiod, common/cwq/ohauovtdruhyvbugqwqjmgthpoedc, common/ecv/zqmqfohtkbzwfxaazofxqdbuddllh, common/jck/gwoqmbnqbxunkqunfsigulowubswb, common/owi/knntxxpnuwbwnljthjppkfpjruddm, common/mae/soryrvujbvnxkofzsqpsebhbtgyvu, common/oyh/xflickxgfcwqwildrkkbpgjsjdsrd, common/sad/hbqrhwilniuxkuhiomausaslrnunl, common/knj/bgfdcvowmgzzevivwewulcrmhukzc, common/klt/etigohcstrqrujshxezbbcnuggimr, common/kyo/rmkeihsexnjhqdqzxwvunvphxtnlf, common/utp/txpsslcnihcoegvhkalxlybdiumxs, common/tbm/qxqswyciepvhoekxuckdzphwiftnh, common/jwe/jfjzchlusudsyobemkhxafcpzchwc, common/etd/ripqfxvfjpnglbfeslqholncupzti, common/dpm/diphvvljxfymoqpcofdakpjspyzsm, common/qqp/xxausudwrowfsktmhxsfxwepiugxc, common/raf/nrsffskhgigqvpalbxnifoegepfni, common/pqu/qdhsdumwgxkhlxafjlcfqkzmuhcmb, common/mcj/xwhnnksbnqbcbtfxztakjljeqlcno, common/wfp/jesisqikoxmnmkcozxizgcyguogzw, common/uzr/ozzoxdyqydsruqtqzxkjvemqytnmc, common/kwc/qfjqpbxihizerflmdmcwcihynasoi, common/ryt/btbbqppgxjcchqnnqdizokglmpzqm, common/wkl/fkdohymxluywrtpvmfisrmeyiqrhp, common/dpq/fayiiemggdspkakirmcgyvlzcnrov, common/iox/funxcbfpynguzuuqatuihmqjmyvjf, common/aad/navsxdujqppnyrdsiylgzlqbigpjb, common/fyb/dbvsusafnhrfjmpevzwilsaqnrozt, common/xrc/tjtmthmydbllfrepyjsmgyeencnne, common/uue/fareqipjooanchbrdalslrhhyhljn, common/mzz/kzpjdcfhjfloiqsruelrfnvxpktff, common/byl/nldmbpubdbhrtbozlzdainfnanbxd, common/wqn/lquzopiueawufmtbwkkgtzqeczseu, common/zrh/bybmmbrmkverqyxxouwhbclxaanlt, common/ivh/kyqyokvkfkinebfrnrosxzglubxnr, common/ako/nczhhrcnjeqncfhuxlbbuufteuybg, common/otw/zowffqvrckwpshhrebweradxiglxl, common/eyh/gwzbbyzzxnsbbwrncpoejfqeyttvy, common/swy/qnawormastaydyhxfqvscpougfbng, common/vqb/qksxcaqaswyuwtrzohciezmdykevb, common/unq/cuomsmuqpiblluaeslyxbjifeqspz, common/gaq/udbqtwsrkihkwiuuzubdbcytagknk, common/zjy/ludtpxiueoegnbqaskufjynervcqz, common/jwp/feqdhaenuvirjabegkcvtiuiaayre, common/bbu/donvkdgfwvjciewvlriegjwhyiuss, common/eza/xcbukicpmjplwfkapbzwlalhilbyh, common/fax/vecbarvetahhzxbpksdaotzmestoz, common/pjp/gtjogksxjsfjcjzowkhnqhgrpibag, common/dpk/wzbsbgsaixyxvpbziffzxintyvcal, common/shp/furgrglimxeqaliycrillywhowfbf, common/tym/vfrpxytxirbkhuodsekhpyjhtmdpt, common/try/lvhirvcfpsptfvyeoyktgdpvjklac, common/pej/cwdvaxywzwjoayjnvcymatnfovxny, common/ucd/vbauqgvsgetpgdgmjmmziqetoegiq, common/lfn/nfqvvdwtaijtxmoziodgtieajwsss, common/tsr/cgllfhuffuqyaoqfunlncnxlodbea, common/wwn/qmjbqhjumtororrlfsxmwkmpkipqs, common/ayi/ayhbcllfeksptnnwegnvnjijpfpuz, common/zka/lexofhassqqhbisnlbmmeienvdlki, common/far/kvggzinlnxororphxtfwgaugheoiq, common/opk/jhgdlleiszyaaavgofjwikdltlowk, common/gat/cgqohmplcnfhsudokzxwgknpykfvu, common/fvn/zdfrlzxpzhwpgzqlpcqeozshcuowj, common/ajf/glczvunnpexailrhpbgepleriyvec, common/yhj/nkhmeguomhkfukezkvoxashuzvebi, common/zkv/gekjrzdahkcejtklyavukoghruchr, common/lpk/tnntacpjpubcigcwyaxumcxlldkik, common/qgb/nybivqcewvdlffvjsbivflgduxlio, common/vct/kqcyueiinxgbcmkddewlgckxlgrsl, common/iot/ojmvbetxatkkjjbrqfozgvcicmiux, common/aud/hctoetmikcxescfaqpontbdftbemi, common/jtm/zewybmmohykcpqyebmeglhplmptbr, common/ndz/gyvaxixctiowxqwagtxgzpjizyywc, common/jgt/teespiknljkwqkaizsgyswjyyndjq, common/uii/mgcrbpzanaadagoeouknednuywyqi, common/dfx/yiocjbvbpenrjyvonvoqwdsvuiutd, common/qmg/zvitjdfufmilmzfljquvpeyqhluju, common/gak/afsmavzzvtplebdifamtuuhbbeohu, common/ghv/jfgwvxiqkssvjsodiivoaojuovtrh, common/yvh/sbvfmgdnegmhbnagrmvkokiwqknnq, common/tyc/rliatdpfjqcboymbkswnckyypccig, common/zcg/rjuxzlbcnfeyykkozjgxscblcatoe, common/ewq/kcjdkibrbjsculkvoaxzsjvwfjlmn, common/akj/qbzydhcfjnvvngtcnyckzdfxvmyxt, common/bys/qtcszcegzbcdnamdewwgsiijponyc, common/obe/uuyundlkyusgyalroxzvtquctrqna, common/pou/jpniokixwhgiigzdjmabudhddlwpc, common/uml/macljmkmxhkhrzesbmzuzlllvrwrn, common/sgq/zwkvwiepgbgzhslckbiovuzyckdfh, common/oqf/mrosobyprhlnjkrrcrujsajeutibf, common/kew/xpwvcvstkzfyhayzcteskcwkdaclv, common/eoq/nfufhaiohjxkatjmkglnswzhapflp, common/tnu/kvsuituyeenneihfhspnlxvjoyanc, common/cva/njvocqklpoilnvsizgsruvkvsbzhl, common/ijl/paotgdlylelqsxqwkpuuudadeazcm, common/tmz/cjdtzkfsjgepzzivhspryhyxaqfmb, common/ygs/mlukujqeapgaujpqanbbrwpzmjahd, common/ftu/zatvhkjhfyuqkkkmfizrcoltyegrx, common/kqw/mgmgyddvqmkxuwwwvbenjojrkqppr, common/ple/ocmnhlkdaozobuhzrqfjkesigansf, common/qwa/aaibfpfcrxewwesxuaqyhixmwhcjy, common/nuc/nsuoqwaccvmiagasvqsmkfvwpjdoi, common/inv/dvitprliotniwwlxyjgdglshoeqew, common/vfi/pofpchsjpcrasootnoqxmvyipoyix, common/myi/wrlbkegnggycmckykjyidsfdfetxg, common/ctw/aiwzcxsytqlhampfkvruruoinopaf, common/ueu/hnxnwjtiuaciczwllpkvldlxxmsdd, common/mxd/jmukdzinyjolbtzldrijkfkqfgklr, common/aox/yvugxpztsncttxeaxqseuuddumsam, common/rcd/ipeotossqbmqfpodefcihjuocxkuz, common/qew/lnfvawgwcejraguzxpiiwwracqbec, common/byp/nayyypzxqhxgethwlzezyyilzsues, common/tqp/adppnoclfyickzhpohontzvygeroz, common/qeu/tksgyulvetuyesqdkprsktmmfldyy, common/xpy/knaohopxpzpyiyrarxzltyjnqewqe, common/bes/axxhrrusbrtpzhujcmaecnijdkgdz, common/zmq/xzzfsgenqfyazizmcwkhjxurhduki, common/ylr/pdbyeycnnehjegucgcvtzqignapih, common/mvf/ftzodkknppnkexqcnhrjdwzumgdya, common/wud/snyvpklrianvdphthulihqaplrurd, common/jif/ikqousqjklazstfcxmsbnfkaczefy, common/qxi/barzxeuhvqtopyctsmotghgrxbkei, common/lkq/ygxaldpjidwpaehekiwxktrcnfwjr, common/jsr/vznjdgtvzpqvdkqmtyahfikgtwiqc, common/who/hugafivywilporddaxzqkfwsopfpg, common/csv/gjkagelcmpqajpggvjmzwogmnddwm, common/ztb/ttybiwrjyouwpvdduinefpyzlpnqp, common/arq/aydvbxkvknkzmjpbksawdkwkadeqj, common/maz/aepeidowmqltbshljxswwmdkowqoz, common/ogm/umgnvkdrgbuejtxownluqbwqppglt, common/bbb/docxtakajoamzsayjwgmfyisdkxzj, common/bob/nuqelyrfgswcmqxdyivxwidjepbjp, common/qdc/pnsasagwkyxnvoeuihxfbohzhabzb, common/sue/xhzlcflilayyuhmheohwrpzfsomuc, common/tuk/qrycmrijdrfdlvmogleftownxbxpw, common/oxy/bttowfrkjxyrdjifjcxovmstlgzdb, common/isb/ohhpzgurwpawgpnafwoiflnvqopew, common/tvr/bxjdraxpvrnauwtrusemlufpgnwnj, common/erj/gzyhuaahngqrtnoavyqtglcwadeay, common/evx/bdugqaqejrpaejglmhxdfyxlvwfow, common/urp/smsrucveulaksdhwzkuvpyxibuahb, common/shz/pedpjdzlsycwndmxksnjctgtbzjtf, common/jhi/mebdfvajhaqooenxrtczkwiclcefl, common/ndz/ycrvyvhnjavstkkghneptqxwhetlp, common/dhj/iecdpcmrvpxzbptaohorpsnbutdkw, common/wpm/puicreeujtmagrdirsahrtverpkun, common/mnj/wjcbtdibqofgjicpgcgnvcmtwbwpv, common/obw/taqjigesqcbxbyvcklmhurcieuehw, common/nmy/moxwzvughzvqdcyxeghshhehiysit] were removed from S3 2024.10.22 02:26:26.797116 [ 1420 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_22_22_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:26:26.952999 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_946_311} MergeTask::MergeProjectionsStage: Merge sorted 14421 rows, containing 12 columns (12 merged, 0 gathered) in 1.215117699 sec., 11867.986131605181 rows/sec., 1.86 MiB/sec. 2024.10.22 02:26:27.161231 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 105827 2024.10.22 02:26:27.243427 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_946_311} MergedBlockOutputStream: filled checksums 202410_1_946_311 (state Temporary) 2024.10.22 02:26:27.430447 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_946_311} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_946_311 to 202410_1_946_311 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:27.457316 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_946_311} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_945_310, 202410_946_946_0] -> 202410_1_946_311 2024.10.22 02:26:27.469232 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:27.479328 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:27.503662 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:26:27.654223 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1580_1580_0 (state Temporary) 2024.10.22 02:26:27.692396 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1580_1580_0 to 202410_1593_1593_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:27.723934 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1207_1588_121 to 202410_1593_1593_0 2024.10.22 02:26:27.742201 [ 1267 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.29 MiB using storage policy from min volume index 0 2024.10.22 02:26:27.746547 [ 1267 ] {} DiskLocal: Reserved 1.29 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:27.754061 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105827 2024.10.22 02:26:27.761818 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} MergeTask::PrepareStage: Merging 6 parts: from 202410_1207_1588_121 to 202410_1593_1593_0 into Compact with storage Full 2024.10.22 02:26:27.861912 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_189_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:26:27.868062 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:28.016303 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1588_121, total 26391 rows starting from the beginning of the part 2024.10.22 02:26:28.279781 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 777 2024.10.22 02:26:28.403353 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_1589_1589_0, total 51 rows starting from the beginning of the part 2024.10.22 02:26:28.439716 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 12.33 MiB, will set to 1.59 GiB (RSS), difference: 9.23 MiB 2024.10.22 02:26:28.502567 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_1590_1590_0, total 86 rows starting from the beginning of the part 2024.10.22 02:26:28.579973 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:28.586360 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:28.627785 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_1591_1591_0, total 61 rows starting from the beginning of the part 2024.10.22 02:26:28.664205 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_759_759_0 (state Temporary) 2024.10.22 02:26:28.748552 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_759_759_0 to 202410_769_769_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:28.822348 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 777 2024.10.22 02:26:28.859268 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_1592_1592_0, total 30 rows starting from the beginning of the part 2024.10.22 02:26:28.902722 [ 1274 ] {} DatabaseCatalog: Execution took 255 ms. 2024.10.22 02:26:28.926184 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} MergeTreeSequentialSource: Reading 2 marks from part 202410_1593_1593_0, total 67 rows starting from the beginning of the part 2024.10.22 02:26:30.366810 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1165 entries to flush up to offset 1360313 2024.10.22 02:26:30.526839 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 10.45 MiB, will set to 1.60 GiB (RSS), difference: -4.50 MiB 2024.10.22 02:26:30.669215 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1488_69, 202410_1489_1489_0, 202410_1490_1490_0, 202410_1491_1491_0, 202410_1492_1492_0, 202410_1493_1493_0] 2024.10.22 02:26:30.672784 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1488_69, 202410_1489_1489_0, 202410_1490_1490_0, 202410_1491_1491_0, 202410_1492_1492_0, 202410_1493_1493_0] 2024.10.22 02:26:30.706286 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1488_69, 202410_1489_1489_0, 202410_1490_1490_0, 202410_1491_1491_0, 202410_1492_1492_0, 202410_1493_1493_0] 2024.10.22 02:26:30.849361 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_898_167/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:26:31.365061 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:31.367654 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:31.377568 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:26:31.380645 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:26:31.495513 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_898_898_0 (state Temporary) 2024.10.22 02:26:31.640591 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_898_898_0 to 202410_905_905_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:31.658186 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1360313 2024.10.22 02:26:31.758212 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 105870 2024.10.22 02:26:31.878522 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} MergeTask::MergeProjectionsStage: Merge sorted 26686 rows, containing 13 columns (13 merged, 0 gathered) in 4.121334802 sec., 6475.086660527999 rows/sec., 1.23 MiB/sec. 2024.10.22 02:26:31.974454 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} MergedBlockOutputStream: filled checksums 202410_1207_1593_122 (state Temporary) 2024.10.22 02:26:31.996905 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:32.005267 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:32.057186 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1581_1581_0 (state Temporary) 2024.10.22 02:26:32.313257 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1593_122 to 202410_1207_1593_122 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:32.318310 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1593_122} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1207_1588_121, 202410_1593_1593_0] -> 202410_1207_1593_122 2024.10.22 02:26:32.393613 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.95 MiB. 2024.10.22 02:26:32.425176 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1581_1581_0 to 202410_1594_1594_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:32.500381 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105870 2024.10.22 02:26:32.504747 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 9.63 MiB, will set to 1.60 GiB (RSS), difference: 17.29 MiB 2024.10.22 02:26:32.564143 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(8cbcf60b3f8a77ed9c7da6e01f97de42) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8cbcf60b3f8a77ed9c7da6e01f97de42, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 02:26:32.569419 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 8cbcf60b3f8a77ed9c7da6e01f97de42, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 02:26:32.572513 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:26:32.951195 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 02:26:32.962630 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 23 entries to flush up to offset 14402 2024.10.22 02:26:33.074795 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 02:26:33.120988 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 02:26:33.167756 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 02:26:33.197268 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 02:26:33.247569 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 02:26:33.255999 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_887_256, 202410_888_888_0] 2024.10.22 02:26:33.258426 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_887_256, 202410_888_888_0] 2024.10.22 02:26:33.268557 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_887_256, 202410_888_888_0] 2024.10.22 02:26:33.304217 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OpenstatCampaignID 2024.10.22 02:26:33.316132 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:33.323396 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:33.367703 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_938_938_0 (state Temporary) 2024.10.22 02:26:33.496803 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ppv/eemfyvacbugyrpitvvstlsnuwvqwi (LogSeriesLimiter: on interval from 2024-10-22 02:26:21 to 2024-10-22 02:26:33 accepted series 1 / 157 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:26:33.537930 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_938_938_0 to 202410_951_951_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:33.586632 [ 1195 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_946_311 to 202410_947_947_0 2024.10.22 02:26:33.597132 [ 1195 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:33.604340 [ 1195 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:33.620840 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14402 2024.10.22 02:26:33.681601 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_947_312} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_946_311 to 202410_947_947_0 into Compact with storage Full 2024.10.22 02:26:33.746102 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_947_312} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:33.757468 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_947_312} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_946_311, total 14421 rows starting from the beginning of the part 2024.10.22 02:26:33.855112 [ 1238 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 02:26:33.875392 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 131 entries to flush up to offset 102681 2024.10.22 02:26:33.965152 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_947_312} MergeTreeSequentialSource: Reading 2 marks from part 202410_947_947_0, total 15 rows starting from the beginning of the part 2024.10.22 02:26:34.230098 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:34.239692 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:34.271831 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:34.276141 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:34.852322 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 8.70 MiB, will set to 1.60 GiB (RSS), difference: -15.94 MiB 2024.10.22 02:26:34.960396 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_922_922_0 (state Temporary) 2024.10.22 02:26:35.115153 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_922_922_0 to 202410_928_928_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:35.187528 [ 1240 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 4 parts from 202410_747_923_37 to 202410_926_926_0 2024.10.22 02:26:35.208133 [ 1240 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.12 MiB using storage policy from min volume index 0 2024.10.22 02:26:35.214194 [ 1240 ] {} DiskLocal: Reserved 1.12 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:35.223410 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 102681 2024.10.22 02:26:35.365081 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_926_38} MergeTask::PrepareStage: Merging 4 parts: from 202410_747_923_37 to 202410_926_926_0 into Compact with storage Full 2024.10.22 02:26:35.572006 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_947_312} MergeTask::MergeProjectionsStage: Merge sorted 14436 rows, containing 12 columns (12 merged, 0 gathered) in 1.925534683 sec., 7497.138393533678 rows/sec., 1.17 MiB/sec. 2024.10.22 02:26:35.623623 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_926_38} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:35.643255 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_926_38} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_923_37, total 18709 rows starting from the beginning of the part 2024.10.22 02:26:35.665059 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_947_312} MergedBlockOutputStream: filled checksums 202410_1_947_312 (state Temporary) 2024.10.22 02:26:35.698413 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_926_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_924_924_0, total 72 rows starting from the beginning of the part 2024.10.22 02:26:35.750104 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_926_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_925_925_0, total 136 rows starting from the beginning of the part 2024.10.22 02:26:35.796563 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_926_38} MergeTreeSequentialSource: Reading 2 marks from part 202410_926_926_0, total 185 rows starting from the beginning of the part 2024.10.22 02:26:36.334700 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 778 2024.10.22 02:26:36.512478 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 105931 2024.10.22 02:26:36.514781 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_947_312} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_947_312 to 202410_1_947_312 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:36.549177 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_947_312} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_946_311, 202410_947_947_0] -> 202410_1_947_312 2024.10.22 02:26:36.584630 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:26:36.782913 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 12.76 MiB, will set to 1.61 GiB (RSS), difference: -4.03 MiB 2024.10.22 02:26:36.936666 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:36.942380 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:37.028265 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:37.033231 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:37.155632 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1582_1582_0 (state Temporary) 2024.10.22 02:26:37.270263 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_760_760_0 (state Temporary) 2024.10.22 02:26:37.352667 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1582_1582_0 to 202410_1595_1595_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:37.419288 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_760_760_0 to 202410_770_770_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:37.537043 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105931 2024.10.22 02:26:37.673346 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_926_38/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:26:37.890931 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 778 2024.10.22 02:26:37.891437 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_926_38} MergeTask::MergeProjectionsStage: Merge sorted 19102 rows, containing 13 columns (13 merged, 0 gathered) in 2.532243845 sec., 7543.507327589141 rows/sec., 3.28 MiB/sec. 2024.10.22 02:26:38.096404 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_926_38} MergedBlockOutputStream: filled checksums 202410_747_926_38 (state Temporary) 2024.10.22 02:26:38.175839 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_188_188_0 (state Temporary) 2024.10.22 02:26:38.614796 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_926_38} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_926_38 to 202410_747_926_38 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:38.623536 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_926_38} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 4 parts: [202410_747_923_37, 202410_926_926_0] -> 202410_747_926_38 2024.10.22 02:26:38.666675 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.03 MiB. 2024.10.22 02:26:38.667469 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1361853 2024.10.22 02:26:38.927118 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 11.22 MiB, will set to 1.61 GiB (RSS), difference: 10.74 MiB 2024.10.22 02:26:39.603694 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:39.623545 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:39.775197 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_899_899_0 (state Temporary) 2024.10.22 02:26:39.838619 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:26:40.177185 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_899_899_0 to 202410_906_906_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:40.240234 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1361853 2024.10.22 02:26:40.339091 [ 1420 ] {} S3ObjectStorage: Objects with paths [common/wpu/gyiwxuazxhxntfydxrogblfiugkxw, common/jqa/qdlkhpdarfvaedpelarqhxydvvdtu, common/ztd/vbslxihdffckogxepkyntvmgdrmqj, common/eoq/ntjherotbfqmhugbxkaywawzqgrju, common/kan/ajvyhnkxlkizixxjrhyacijvflxuo, common/job/ujgpbqykawelqnthpmqdqmwiifwpp, common/awl/atmyzxprffvacpvonxewxbpykqwcs, common/rma/rmwgcmiwkfsqwmowjzvrwubsbqfcq, common/qtc/ksutglrtjieakbbwayyihehsisvpi, common/fwb/isfvzbbbncpvjtmjawcntubkovhzm, common/rjf/wlqkjppnkxouzzjubmmmcdamuvqei, common/rxh/yrjkiwtngahynwksldaubeuxtvpng, common/chv/ktvptdhtjdkxcoleathxrzinsaaut, common/qpp/rgmyieqozosairdujqhsypcqixfsf, common/hsw/ywvlwaojlfcscqfcywnsggjafeemx, common/ady/lzmwzdpwmlfqzkwbzyuagogskexdx, common/nql/cjlvsyarnmlqeyzlnmlwijrwewpzv, common/bho/cojwihbhdoejnhyumpbgsximvegqj, common/lot/lbysmhdwyusnlhkexwonyesqhknyj, common/dxq/ievgtipyetcywlnnahootxahjecut, common/fqy/byzdlfywocnsxtrwuwdowhmlgiolo, common/bwc/hwmnwalnxkkmyqzjswpzuysuzcaqs, common/vsm/brqyqvvmrwojjvrvqnlmwuxrcqeoy, common/yig/uqscepjqzgxpgjlwbciobhlcnrtsc, common/mxo/tzoirlhnlrkslhfavdgnrkewzflxu, common/ljg/alxcjmeqlygdqrwfcmjckknxqazpa, common/cyv/qlgoolqcsyghgjstpcfemqvzolcrt, common/niy/qwxtdkfevgrerntztcirpvspldgsi, common/hca/lswqtlsbkijbsrytiqwblroxappll, common/ffw/nyozyjocsvokfykuiumbvptkwdjto, common/fbp/rwdfvkfynmxceffofrcamqsouqyiw, common/iri/yjrxuoxdbnecwzzjpikhelzoklduz, common/dll/iagjjvcxhurmcjugaywumsebkctqp, common/hvy/jpxmbvecdgdzkzmrlfkdydcmilrnf, common/mvw/lqyuajjnjrhkccjcjfywbffnyubup, common/hal/xofjxcajxfmfbeantnpgyanzlrqkv, common/yho/vbnrdzitlnlfjgrjyszrkrknboztq, common/wzh/slcqfsefcebhrdwshryqvspimgbrb, common/qgc/zpnfjhgjbtzpnarclgprjkmlrwrga, common/mpw/wghmkyjxthhagwhokdpkmfpkjofnf, common/kzz/iqvcplwdnkezxfefzanpowqiqosgk, common/jsj/gqbhvoiisjzejgoqqkovbovttmfds, common/iwx/kqxkyjmjdnveooiizxtelpcxxhirt, common/vlw/lrcaluqosujtjcnluyntrdmmhhsjb, common/gkq/zbhthekqagtdnquyamgvduncphtyt, common/zcn/uvpyxenddyuiemqwfisgclznbypbm, common/dpw/uflnliovzaenzmxmwwajtpiopduka, common/dey/siplsdmgmbwezyazprfzahvxfwkkh, common/avq/jqfohaeeijbtyryofmunhityucgdp, common/hmm/fkqdozfphuzmeyjllpztkcavhpxqi, common/gsf/khklciupwcnxwiopunsfqryugdete, common/vmp/mdrwocakflxidsnmecgtnhitxjnyb, common/jre/pgkescoitgoggfpmqczobxyiyeoka, common/ldo/zhuutwwqckvbmzbgtyaurykieamct, common/gex/chmjsvbwhaerndesnnemimpqyyhtl, common/gjn/dmduisitkgfkuutiqnwdzgxgoefqf, common/los/mnuspzctldxqptppuxlnndtfedces, common/gwr/dsjtuzzoykapczirfzpniqwxgopmq, common/dbu/msllunpomthetaarobdnwspzgqacl, common/oqg/lstvxlawtbtdrozvlzrynblkohbeu, common/vtk/skhveatsbqiofaeaabdrrbwwgotry, common/cmh/hisoyzxwehezexdzmizrkerhngflj, common/yds/nzbxlepyhnpgynwqsitvdgdhyltlk, common/hqq/kbgmotevhfpbqkebcruqcqzludyge, common/thp/oebpyyyufbqwqxnhghhcrlbqcbqcg, common/dpm/vedcxsjifsfhkksowjjtgyyfjgeqt, common/mae/mdyxioxdxskzpailoltxzdunebngl, common/cff/qfiumavljfgetvdnnohmeyzvxfowy, common/tgv/nldqzygouvfkznapwxtwfuszbrjhd, common/ewa/tvbttatxwgmergbpzpsuuzqtircdb, common/dgs/cyeupbrwxlhajgmkiqpyeanaisgwd, common/gnc/efezkofmssetxmjzplhtevgaujmjz, common/sid/ffrqlqtqpcecagrabhhtvnjohqvlr, common/lup/vbziefgawfjewulrighfebikcqora, common/mih/mbbayzsphwquouevevclnaknzdxur, common/ezx/upvcozjmxrcxozzkvxgdmhwmnrndz, common/pvk/cwfuxspntsnyoudebeuddpjqhbvnp, common/qxn/evqigdjrkhocqdbnlkgruelxqaoah, common/zpw/zmiepyascmddprdmybfbrrydypcoj, common/ane/ulwutcfpizcubqdafizvvrawoxbzs, common/rng/fsgxuvjhaoyijowmvphrpkklgmlfn, common/jzf/znsqtoorxkssaqppncvnusngonjmv, common/rhy/qijpkiynqaotrjwkihzuxnzuwgsui, common/qut/ricpsozqbqxnjuavkysuhebpvyrvl, common/svw/dwyqllaprzagnerbkmsbzmkhkamyr, common/oaf/snmmqsgwhklgwucixlhsglsighnjn, common/ffw/psiajnewjwfysapkhskfdcklzuozk, common/gcr/wkvlhptuafklchdkfuvipbhkazjbr, common/hyb/gygqzgfukaolhyychanzxexkrtxxn, common/wtg/ofurtnmtceecaciachiuphdphbcxd, common/mlh/cndfffevdqcpsmrivwfzscjpdkeyq, common/cah/ydpufgwxlbktydqabgfermbxtgebl, common/yru/yhudnonstbjqmajprdiiofecnqtdk, common/wod/gpytmcqkdomuxwghhkfdnpoftuqgo, common/cjy/jkpiosfcgbyrqwntcgyeuowndwymq, common/ijb/fyegodjgnobwuigavwwvcsyfnsmzn, common/nws/itzkpofrkxtfyxxjrboexrkrhwoie, common/pmh/yfyfsexskwetvuayumbykoejputcg, common/hnu/yxnummqzubygejavcuffwuyjkutgw, common/dwu/djhjwprzflkfknywgetpndqnyjbjq, common/kcr/hwkqqlsccdiarhkocgxicagamtkue, common/iqk/ubeeylzovpbenfpqvrfyjhgfjocrf, common/tyb/vathvqmsbndepkpttotkfswukwcsv, common/cev/edkxauppojirbonhkcvyhrbmidsnz, common/gsa/yzmimqwcyhubzwlynzgkcoyqjvuke, common/xth/pysrlndhjpmdesmbpcridyyjadbry, common/cfn/pwavagzriespxklvmuxubujmgwibj, common/kiy/zttbptfezlazmsekmgjtuswwkyohh, common/krl/syyskstaqezznmpgprojpdrfhmrtd, common/khk/htgzkelosgkhzsqpflfysucgelcrf, common/esa/ktyfiziftfurdvoyjpyvdxhuytwsl, common/whd/omlmehknlsisdvbmktgxhtxbcxhmo, common/jbh/mqvuqkffturadusncwemwzljblakp, common/vto/idefkknzhlzrxbgijmorraepspykl, common/cey/ghqcswaqgrtmkpqdebyewlshkzcno, common/dkf/hnadfcfxeekkcalqgqrpkxtlclkos, common/tyq/hhhfftxysvsykwkrmmnmmyqyimibx, common/htq/sbfruvktftiilocqrvmnqhfrbnolp, common/yvk/mafkwjvswenbimdiovhikgnpqraky, common/kwy/xeuujiusdbchgbzxvanlkpyqvfppc, common/cxz/nbwtivqfafswrpmpjaqdfgnvywscs, common/zqb/kbfixhmkbzvvqyeqssqcswhuokwig, common/osh/ywwkkrekefvjtutoywotzxlinfqqy, common/gep/bmccbdkwaljxveoyeyoodxtcsovmt, common/zgy/qpjokcitcekvoezaajoiqgremzsgg, common/mvz/tmvtqcyotfneusvugivuflhgvvtgq, common/kcz/shoppvmrykqetrthedbqulxguwmmx, common/oqx/sqcpeokttmffzgpagafplhutrelaq, common/obw/wqvgayhhzeujrzurtdrficzwxktdc, common/xol/cuktudesvogppcodngvjapsxtlvpd, common/auf/izblaftdshldbspkcoadzgusrcvkj, common/ane/zvbuuhdadqctxgedzoawljzupnrxm, common/mtv/qgmklqjgdijnttmsysjkekbszzpzg, common/jpw/tibqwhiibzuaphhkuryxvapznfwqs, common/pac/dlsnjjwplqdbryuhjtouswzecwpiw, common/awr/anotxumkvdrgjdtzechrielkzeyiw, common/qdd/iadfhxsmrsuphegxnfswxodphrxzg, common/rtn/bdaiwqfuitmafgvadtsfsgmwciyhv, common/ovh/pohoerrahnjigctwmtiepbrqrwddh, common/lhj/ceucibjfzorzreucskkbxjukalwza, common/xwn/zbyccbmxxeoblxqpbsbyzzbiwehnk, common/spg/lgzqpjzygvkodiiofmzgkmevfhugw, common/aut/acpbdfhwdgqvxghdektnytxnjmfrd, common/zyq/eaagpcqxevqbnnophevrlkwxfawji, common/mav/xtiggjefbatxrqkqjlqiwztoyqphn, common/czf/ehpwyndfnhdczzqzkrwrpurihyxvx, common/lvk/wdpmymegfftbkjdvgzhpoahybwrqg, common/rkr/uddtapjzdkhdasplzfttxsnxjuuzc, common/rkf/bkgjcoxbhrjlycxjtfffuxzklzoth, common/mtp/dzvehuehpltmcigaxhxzrounykkpl, common/ylb/pwbubnyitthpwnkycqlinxxszrxuo, common/gdt/rvmmezxvcwtazlopagqbzbrlzssrm, common/mpw/bpezmhmwhzjtjvklcvaajpgmbdtzj, common/lqw/flosomkmuyqqsmhuhknwurkblzoio, common/foc/fegxmzmreehgmidodrgjyrjivbjbj, common/qcu/qkiswdvwfqmrllwhtjaeaijzaikuh, common/upl/qhjnqsqigzuduqhsfjzoyeqzijbns, common/szi/zthxpqgwsjkonwkkteqtpjsnuxeff, common/knk/dyjdqngtumudkkpeslhrurtxjxstb, common/lnr/cygbscewgnylrnsjcdrafhvefoxew, common/hfc/squehdmsgpggfcnntupfwtyjcdsfj, common/qci/kbcpkhdekohztesewdzolucoyggaz, common/ada/hjlqzdcztnhqvxvpzqjgdpmzzqbzn, common/oif/vvmeyweebnqidubuxlbaezooqfahd, common/tbf/pggfkhrgkxlragmfifrawyrdoccvn, common/asa/eatakbvhonycgjwjjcwnziqiglmbx, common/vzv/jdzyezljrmuhtdkrkgocxiykdobiu, common/okp/enbkakrimrcbswbdbqdnbenmapjkq, common/ktx/lruvtwahrtqvogsldgtxrtcmkavmx, common/flf/dtnewbgilomakckldyixeafrpgoqi, common/lkz/wgsmgnxvpfjihowqktwithaxhghrq, common/sud/robgjthclzvzznciuhmrmdvgsdpio, common/tvg/fjdppnjdswqvbrdeynwctlkizdlmv, common/rys/fcdtxayefuljopbnsxechofehrmvr, common/vgk/bpnwsehozhxrtgbogzvveignsdqyo, common/pdz/lqknyssgxerfuqmikararzmcjtnok, common/kni/qwfrioqnmnkeqqkvadjzyvcvtdjal, common/pgh/lxulqfmpovjrkdtpwrajdfbnjopgi, common/byh/rdjgdqxgdukjuwtfzvikptkuwlihr, common/tju/nkqnbbreiqokayqwgdnijynmkkrdy, common/xho/lbhvkfrmlpxwedsdftcwnmduzsffw, common/hte/wfgkhpkjesgdrqqlptdjtdpghekpf, common/ike/ndfexndqqazfubdmxhivutypgzhlo, common/dhk/mnmrpgzzeupyhodezlndauuihaceh, common/qyf/zvcarxnecqvkgnynytxuossculhma, common/chb/isckbjgrsftgnigfwjigcprppcbdj, common/avh/ksqfnipotzakpqnmsvakcrfuwiizr, common/brz/cmlpgnjsrxqwzdilfecnhvjlvilhz, common/trj/ygpyagpcagkjbcvgcajgesstugqir, common/hun/cukrkdeluwlodvrcajodcwjzhoggc, common/jjh/ufjaxaypsflfkuutmhpoljlftannv, common/shl/fuduchdsemufekkdararvgaxgucle, common/hzt/fpwgitojylrlapmeuxfiwhuehmxaj, common/avw/xegdpkkyanikxhocdrrwzvwnxxxvu, common/svd/qurcwdmhpzmeymdxgjptuynaghjct, common/wym/fvlzbxelmuagribaqhrxafhqxsbty, common/sxz/lyqhtjavfjvkpctmpiogkomednzss, common/plr/rdhuiyrizcdvshedlaxhsqzfdvcar, common/crq/welukjndqhghszejgpaypxuqqrrbx, common/sdu/ogkzhkbwtqqvevccyyibjhvxmqpnb, common/cek/uksoohxicjlputaflkqpccgvtpqea, common/ynm/llacmgafyjtamemzuupctohocihuw, common/ixa/ktjxpiveqrtvfehkeofyyqzjhmszf, common/xso/zakkzsqrzgcytdwgtstmvfbbqzkpg, common/cll/szhhpmphvqrigmcridqeclilxkqtp, common/ksm/iinjpcohyqvyfmooicqsbztzaqbsl, common/ykc/cnijeqwncyfdmkpogjteecyeatjuo, common/hdj/fvlcomughvvrqvrdelrrduailyjzr, common/uop/nfdufjgzfbewmkvlujmdgthnupunc, common/hmn/zlrkswgkmouijkxwaoozryfbzqqrw, common/shz/uhnftkayiabnqdlwdtcnplqsttlcv, common/xeq/yrduxfihfxkogkceqyyrbuqmdlcei, common/dzz/vsegbtwysmkkljsjbmzlkvgzobmms, common/oat/idsifkzawfiprnpvpporwvuotizud, common/ehn/aroizgorpgnrdgngyonhykahrmoqz, common/mvl/qdqrsltzpgoxsoaarjrzmrnkpdgfy, common/pth/nscaxvbldyebhcknztquewvslqtfe, common/qbo/cjvtalcszwhrjnsfdwipchnpbqxvr, common/ouk/wxpwgtxwpylkzqhmvqfwcfimhffia, common/zli/aptmjogttubxmakcoufznodzqtdab, common/acf/qdjstlrnwfxyajmnvukqheaqxpkcj, common/umq/nldcaotxclyxvrcdrynoirpxqamlj, common/zvv/jjwhnpbufmkftlamygqnxnmvqnybu, common/lnc/wmgnliavugnfqodzqlgjhfxpxwtmu, common/ohn/iqrxrickxuapjdqbhwuyvdnndptbe, common/mbs/fsktslzrqymefnthugufvzyxgshxm, common/ymz/hgzcpxqobefywlrerqbkorizirvda, common/rvs/mnsdiakdlfbwgopzuweflvrajdlsu, common/wxx/gwggfaqtksmjyqmgvctwtksdbresa, common/bru/qwxckngiqcsiqqbzifyreuxtmqbcq, common/luz/wajapdyrsdimsenjvnrnqisupvzxo, common/wln/oxczoldhfntdrixbnsprgzeewqoro, common/dii/miejjnkpuagkyyfwofljydxfeexdp, common/jfm/gzmlrxsetwvyeichxxqjoykkwgoly, common/ghu/ioxvqmpxylmhnveipkhitaopcnrio, common/vah/afiirlcatpzphzosaslrmtwbeyjek, common/pqr/ayyhjixkezgzreinuudadfcnvccgl, common/rph/pqnpjpodntmpeuymlcsuhbzzvyxzc, common/kgc/vwufomdzadrriunnlgtrpztkgvfst, common/pwy/lxqehxstfmihojzyabemwbfsadetw, common/wte/qxxuoxiukimuwoysndpogpfzuinff, common/ggs/ztyfzljujossfqqjyfwjbsrywkdmh, common/roj/skwabhxppadjfsxgukancpxfufkfn, common/kqe/ksedqpnnyrtmekotkxdxasykbvaig, common/iaj/elgkvkwdjyxfaxiiepmzrqnnbvxbt, common/lzg/wbugvayeypfmqmaphckywzeqfptav, common/cfq/tbedzmprynloifhpecmsphgyjjhou, common/fbg/ptgfqwjfwvvmokdaczfoktzerkndj, common/onq/czlagzbcbfgxvjmkqjlhkmakjnrvr, common/wrt/qxrtgjauqudqdgeqfdhlqezjkzqtp, common/cnh/uiakxtosonvvosenqtxnqrvavoblb, common/lcc/jxdnqrklvknbizvwoafgymnxxhgnf, common/rhd/viktfidhzeunbyafkqftlthonlbak, common/xrb/ukvmplskwhzbjgdtnhyhkilqigaxs, common/kmg/ufarkqokednjzsfcmzurcpegyfmqd, common/yiv/ahanrivurvsnchjemfesemzjfkbbf, common/xjj/xoltljltmlsuasddrskvszfhnbgha, common/zag/nzeoiickdkzmyazrazyxppmbmvbnj, common/pzn/wcygakrbwcymsrknprslyzazzuzvv, common/sqs/gwmroeazwdzxzmuvghndrkeeackad, common/xqy/vjrodheyxqqtiyxeemavccwfktavp, common/ess/bocrkvqmucovvelirnexzjdlqudmz, common/bsg/dehhgzpppfcsovtgudcpklwjntxkh, common/mdw/fbmvovurfywhatfvykzjyifnipgln, common/kwy/zjqgrblmjoajesvjfygppglyolpgi, common/kry/rniqglxuyuxvghlxudksgfhjypmlv, common/ttt/unuqgbeajsuvnocsuqxwmrvjbekul, common/dvo/caqdpxjnqbwsvcqcckvirsikrcttz, common/tes/mbuhbovevhonxzmhgnxdbveuvslzs, common/bqt/lmfvrebjtiqngxrmzgrgagjiyvmog, common/vwd/kgvxafugcxbqtkpgkqeuilwrawtgx, common/vue/ovpelyupvrzkoefpuzyyhpgdtbotc, common/tic/hbcupmklqgnjrvblvhlphchdpfcmb, common/ivl/weopymilwnnsxpynnjtedtofwgpru, common/axf/ljkweckcrfwnwrgovbrtmoasxrzpy, common/dqv/fdmilkjmxjtitxqbxignvvvxbcrss, common/ghe/lfajgovaxevhnepfbcfcxifvkixms, common/mvs/mbdtompaecqwueopktbhmeonjsphh, common/yzr/xttafnwmcsumhbklqdrnhmnzdngzv, common/vhh/mrdkbdexsfbdaolxusbxfcecqrmua, common/ydu/wnndsmrtipdmialtylervondtmnjm, common/xld/dzqcbeuhyqqskisiiubqhziwaqqab, common/zyj/vgbslguyxdiqwneukloeohdleknlh, common/ste/mcclznibgalrthjwufqwvltxjgkth, common/phl/igsdtdskkxvzzhikeovwdkbxseqlc, common/qum/vjhaijuedevqtefbfsgggxtclefeh, common/afj/qquplwlfbyxknxixxyoihorpzovuc, common/ewd/qxxbhwjivmtiydgvigbsnmdschbvh, common/doc/eibpxhidhkrjrjigtacsgbcaduvhv, common/tst/ipocwlywlhpdfnzstiawfnbkkylrt, common/sff/ychwdsdwajhacakrxbjioqynfbimc, common/cbz/vgqbiompvsqvflghpoplknkshzhky, common/emy/evruaokzxoqjkvogsgszccabrunbw, common/pxk/vfpwaisycztmbmyixroagiupyrvlb, common/gfh/uxyemcyhvcebdlewymgjeetjxsfxg, common/byk/xixazmzmggtxstmypgitouizbmyka, common/ryp/kvdkcfckffftuoimbxgxssbezklat, common/hvf/rdqbwooxfeuuafkwveaumjuembunb, common/muv/qdajfbyozksmvpugratkhxycuxvrw, common/dgf/lovlnqukjmxmtkttobetoqfninbkt, common/mcw/zppvmnujzlryuzxzhwbconqgcdqxa, common/xqi/hyljbwybiynxqrnjshomplfyhqqvs, common/kdl/jkybfhglevylwihdrmfrsnvjemehx, common/kic/fqnougblecchejiykimpcqgavzigu, common/rkx/mphkrxaequbrikinihjtwoorbvmrm, common/bho/txuefmwmurflotfhrvfkbbczgbzns, common/nmm/wormfvyypurvihxtobmcxiqkuuulp, common/mnv/obdscdskrhaiimuvmwvnptostqcvs, common/fzf/nrmuqvaddruzsygrcecajxdetwhso, common/eul/phlkjgsqfarjwblfzxyeauscxkctk, common/pfo/dmnslwhghjgeweuzrgeohwagrxbrb, common/voo/azwddrixsmjffgnzssuiuorfluogi, common/gnj/wruyjwscokvlfmivetkttmyjnfbdx, common/ual/vuthgttawriovhfkhrouxqbsfebsu, common/vbv/viakjgiuokotflioiitddgpjsanpn, common/oaf/vycdnlchhxndssfgkrcmfsixcndzf, common/lse/vrvunrdrktzsrujnfgcmftssedder, common/lyk/fxzhgoeleqpqdcfjvnmpujzmpflmb, common/ovs/govtzczxnxujoocejfjayeqpdcblz, common/dwh/gcvlnpvumkjelfhonwlubrrrymnme, common/xho/kvckxxspsmnpgssvmsvomczmwtvlh, common/tpd/jieytktmedsjqtslxcyjongticgnh, common/gqa/ssoyrjpvacxwatwmjjhjgqxjoxuzl, common/ndo/mvbexpttjunydhdlythswbniwtuuq, common/wzp/vfajjdzycljgjglhqgovgoapcwtdz, common/lsm/vhijjxgrwtrjburebrontyzmtxhcb, common/loy/dyyfdqifvlyjiqbijflxrgrhkzujg, common/fxt/wdtrgppwtpfhzfqnriskogaeqflym, common/scv/ezejlifjcawbyujtnukbsfpecmpqa, common/mjk/hyzzogkvnbjbnhqfkuzvimrjbszha, common/lcj/uyznxohujbytrsbhqurdfyxlcoeld, common/ywf/cgseqrbfytxpydvymgpmgikmzmjlo, common/bhr/jzuqpmcixilqfbdrjucwrewmbtpdi, common/urc/ksusotfzdydzuekumzwgqjhkkwlvj, common/vgi/fqhfwuthnmbejbcwceiyufyyhmenv, common/udv/shnmpoiusiixtptojckztabjddicm, common/njf/uzygqonrhyzksvekueigbstzfrehm, common/dml/ybwggrlzukfvruutvlsdzqgyspajp, common/xfq/yxgpuvrauwdbamvbxucidwfnfyvnq, common/ahj/cvgcrfhofnczkrcbzrljdlrdcqhrj, common/gdf/dhsxtbrzxewhvrgbupxgtmosywiya, common/uce/feunnkjmdmwewmxqohfhjvpyhriht, common/dvl/fpewsrznldrmugwiiliamqfpxfyfs, common/jon/rxaclftwizlxcppbveazxcfvtujnb, common/gup/igtyzetvjxwbzipbsgnrhnvgchkgu, common/iak/rzlanykvveurmcfvynetobggyxiii, common/yra/fqiqyeazirtwvasaucujydwqzhcap, common/mbo/yswjssvmwuwznaclyskbuiouovhjv, common/uir/azpzcclecfuqkifxwcoqtwilyuuhx, common/khh/qkeyrawlbnraimhdbsxxsaxsrmjys, common/yvz/obgirowpentorzownjnpdysttlsxy, common/bfh/bddzhdxidnadunbaxotwixstrgjfj, common/nga/qizarpsmwhuxatsapshgjtcfsjejb, common/uli/iwvxfikgjugukwfdhlzzcwqdzquxi, common/clz/yoztjkguarquzogbkloyfhqbwgpqw, common/hss/zdwxghcylpdcfojssmkfuozrznfyq, common/oiv/pyuxskfxfuxcilfsyeqbnewfwtxgk, common/yqh/xsqgkblhfzrfuatqvfmtzijapkvwy, common/vcl/eqaspdffccuupahjbzsrontrczjel, common/dgd/tftuidagalxssshveliioklzmwxwp, common/spt/pnozcexztpteolodvizzdcopyfvzo, common/ugg/sxxmbwwvleuhjlfhnynsdnstaexbt, common/xsy/ybpdomyvhfeuzwhaeeahzqcpygmik] were removed from S3 2024.10.22 02:26:40.376641 [ 1420 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialSourceNetworkID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialSourcePage.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialSourceNetworkID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/SocialSourcePage.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_23_23_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:26:40.560815 [ 1201 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:26:40.610373 [ 1201 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:26:40.678149 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 12.24 MiB, will set to 1.61 GiB (RSS), difference: -1.51 MiB 2024.10.22 02:26:41.001378 [ 1418 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_188_188_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:26:41.131770 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 14415 2024.10.22 02:26:41.239171 [ 1718 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/hlk/ownsabaaotdxbywjohcirjbkfxhvp, size 167 2024.10.22 02:26:41.382198 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:26:41.396171 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:26:41.456244 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_898_167/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:26:41.460516 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:41.464561 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:26:41.547932 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_939_939_0 (state Temporary) 2024.10.22 02:26:41.647081 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 44 entries to flush up to offset 105975 2024.10.22 02:26:41.998862 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_939_939_0 to 202410_952_952_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:41.998862 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_888_257, 202410_889_889_0] 2024.10.22 02:26:42.006879 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_888_257, 202410_889_889_0] 2024.10.22 02:26:42.038815 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_888_257, 202410_889_889_0] 2024.10.22 02:26:42.071710 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:42.096350 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14415 2024.10.22 02:26:42.098688 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:26:42.130836 [ 1185 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_947_312 to 202410_948_948_0 2024.10.22 02:26:42.156776 [ 1185 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:42.164093 [ 1185 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:26:42.223593 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1583_1583_0 (state Temporary) 2024.10.22 02:26:42.371901 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_948_313} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_947_312 to 202410_948_948_0 into Compact with storage Full 2024.10.22 02:26:42.400266 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1583_1583_0 to 202410_1596_1596_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:42.453062 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_948_313} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:42.470108 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_948_313} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_947_312, total 14436 rows starting from the beginning of the part 2024.10.22 02:26:42.477389 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 105975 2024.10.22 02:26:42.640591 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_948_313} MergeTreeSequentialSource: Reading 2 marks from part 202410_948_948_0, total 13 rows starting from the beginning of the part 2024.10.22 02:26:42.749896 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 167 entries to flush up to offset 102848 2024.10.22 02:26:42.919902 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 12.95 MiB, will set to 1.61 GiB (RSS), difference: 112.36 MiB 2024.10.22 02:26:43.324804 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:43.329515 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:26:43.477259 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_923_923_0 (state Temporary) 2024.10.22 02:26:43.754414 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_923_923_0 to 202410_929_929_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:43.874350 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 102848 2024.10.22 02:26:44.090718 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_948_313} MergeTask::MergeProjectionsStage: Merge sorted 14449 rows, containing 12 columns (12 merged, 0 gathered) in 1.740726038 sec., 8300.559470346705 rows/sec., 1.30 MiB/sec. 2024.10.22 02:26:44.199883 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_948_313} MergedBlockOutputStream: filled checksums 202410_1_948_313 (state Temporary) 2024.10.22 02:26:44.558107 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_948_313} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_948_313 to 202410_1_948_313 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:44.561006 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_948_313} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_947_312, 202410_948_948_0] -> 202410_1_948_313 2024.10.22 02:26:44.593595 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 2.00 GiB, free memory in arenas 7.43 MiB, will set to 1.61 GiB (RSS), difference: 135.91 MiB 2024.10.22 02:26:44.725704 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:26:45.398279 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 779 2024.10.22 02:26:45.857337 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:45.861357 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:26:45.970873 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 9 file segments (while reserving 1264 bytes for 36a835248c40aafdb98b5aafc1360fec:0) 2024.10.22 02:26:45.976185 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 848a325e6fd713a72d5d1f628a462cf6, offset: 0, size: 168 2024.10.22 02:26:45.982201 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_761_761_0 (state Temporary) 2024.10.22 02:26:45.983896 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 848a325e6fd713a72d5d1f628a462cf6 for removal 2024.10.22 02:26:45.992324 [ 1476 ] {} CacheMetadata: Key 848a325e6fd713a72d5d1f628a462cf6 is removed from metadata 2024.10.22 02:26:45.993334 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d65e686b85da67ce359421941ab94244, offset: 0, size: 168 2024.10.22 02:26:45.999607 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d65e686b85da67ce359421941ab94244 for removal 2024.10.22 02:26:46.009210 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 0a1e7856892e177fded5d2a7850436e5, offset: 0, size: 169 2024.10.22 02:26:46.018931 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 0a1e7856892e177fded5d2a7850436e5 for removal 2024.10.22 02:26:46.033057 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 65e6a8b2c5a0c812d8b8196a8435c9ef, offset: 0, size: 157 2024.10.22 02:26:46.045150 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 65e6a8b2c5a0c812d8b8196a8435c9ef for removal 2024.10.22 02:26:46.110179 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_21_21_0 (state Temporary) 2024.10.22 02:26:46.139864 [ 1476 ] {} CacheMetadata: Key 0a1e7856892e177fded5d2a7850436e5 is removed from metadata 2024.10.22 02:26:46.136912 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pki/nvvgbrsevnlxhrxerdppxczrjhmrq (LogSeriesLimiter: on interval from 2024-10-22 02:26:33 to 2024-10-22 02:26:46 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:26:46.153154 [ 1476 ] {} CacheMetadata: Key 65e6a8b2c5a0c812d8b8196a8435c9ef is removed from metadata 2024.10.22 02:26:46.167932 [ 1476 ] {} CacheMetadata: Key d65e686b85da67ce359421941ab94244 is removed from metadata 2024.10.22 02:26:46.221704 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 241 ms. 2024.10.22 02:26:46.240313 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: e58b2a8acf1409629c13ab38c721497e, offset: 0, size: 169 2024.10.22 02:26:46.245217 [ 1743 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/brc/fqgbmjrrtdmofuddqgjfwfiekmddt, size 353 2024.10.22 02:26:46.246740 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key e58b2a8acf1409629c13ab38c721497e for removal 2024.10.22 02:26:46.250322 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 4d1db6ed0feb6154b70d3d8cabc64e36, offset: 0, size: 157 2024.10.22 02:26:46.259152 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 4d1db6ed0feb6154b70d3d8cabc64e36 for removal 2024.10.22 02:26:46.262797 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 3033f44c56b5bda00f8cf683f09ab646, offset: 0, size: 156 2024.10.22 02:26:46.266136 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 3033f44c56b5bda00f8cf683f09ab646 for removal 2024.10.22 02:26:46.268864 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c676e512255ea0564b9457d8ff41b16c, offset: 0, size: 168 2024.10.22 02:26:46.297167 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_761_761_0 to 202410_771_771_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:46.300121 [ 1476 ] {} CacheMetadata: Key 3033f44c56b5bda00f8cf683f09ab646 is removed from metadata 2024.10.22 02:26:46.376166 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c676e512255ea0564b9457d8ff41b16c for removal 2024.10.22 02:26:46.487087 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 779 2024.10.22 02:26:46.488448 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 62 entries to flush up to offset 106037 2024.10.22 02:26:46.489020 [ 1476 ] {} CacheMetadata: Key 4d1db6ed0feb6154b70d3d8cabc64e36 is removed from metadata 2024.10.22 02:26:46.489364 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: ad4c78df4058a1930580f4b3d1e2f475, offset: 0, size: 4 2024.10.22 02:26:46.502586 [ 1476 ] {} CacheMetadata: Key c676e512255ea0564b9457d8ff41b16c is removed from metadata 2024.10.22 02:26:46.506779 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key ad4c78df4058a1930580f4b3d1e2f475 for removal 2024.10.22 02:26:46.516496 [ 1476 ] {} CacheMetadata: Key ad4c78df4058a1930580f4b3d1e2f475 is removed from metadata 2024.10.22 02:26:46.539058 [ 1476 ] {} CacheMetadata: Key e58b2a8acf1409629c13ab38c721497e is removed from metadata 2024.10.22 02:26:46.574795 [ 1747 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pki/nvvgbrsevnlxhrxerdppxczrjhmrq, size 4 2024.10.22 02:26:46.961345 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:47.025662 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:47.108164 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1584_1584_0 (state Temporary) 2024.10.22 02:26:47.170999 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1584_1584_0 to 202410_1597_1597_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:47.251158 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1413 entries to flush up to offset 1363266 2024.10.22 02:26:47.347421 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106037 2024.10.22 02:26:47.425127 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileCache(s3_cache): Will evict 6 file segments (while reserving 1007 bytes for 15d8d1e9afddd83fb2fbd96671c6c42e:0) 2024.10.22 02:26:47.430504 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c3974a6e8380d0191ac5d97e29360f70, offset: 0, size: 181 2024.10.22 02:26:47.435983 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c3974a6e8380d0191ac5d97e29360f70 for removal 2024.10.22 02:26:47.446645 [ 1476 ] {} CacheMetadata: Key c3974a6e8380d0191ac5d97e29360f70 is removed from metadata 2024.10.22 02:26:47.446380 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: d1e9f3238ed63da49f05860f2bfc988c, offset: 0, size: 157 2024.10.22 02:26:47.465308 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key d1e9f3238ed63da49f05860f2bfc988c for removal 2024.10.22 02:26:47.467062 [ 1476 ] {} CacheMetadata: Key d1e9f3238ed63da49f05860f2bfc988c is removed from metadata 2024.10.22 02:26:47.467322 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: c07af8472cbd6654d744e1ce9b8fe8dc, offset: 0, size: 155 2024.10.22 02:26:47.469074 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key c07af8472cbd6654d744e1ce9b8fe8dc for removal 2024.10.22 02:26:47.472357 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: f95918152eb607816d7e31abea147610, offset: 0, size: 156 2024.10.22 02:26:47.472679 [ 1476 ] {} CacheMetadata: Key c07af8472cbd6654d744e1ce9b8fe8dc is removed from metadata 2024.10.22 02:26:47.474294 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key f95918152eb607816d7e31abea147610 for removal 2024.10.22 02:26:47.479543 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 68669cfc0d579172400ed988fad8de19, offset: 0, size: 152 2024.10.22 02:26:47.487355 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 68669cfc0d579172400ed988fad8de19 for removal 2024.10.22 02:26:47.504875 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Remove from cache. Key: 6eb8a1f2de2285fbd289a2e50d7004d6, offset: 0, size: 156 2024.10.22 02:26:47.511781 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CacheMetadata: Submitting key 6eb8a1f2de2285fbd289a2e50d7004d6 for removal 2024.10.22 02:26:47.595020 [ 1476 ] {} CacheMetadata: Key 6eb8a1f2de2285fbd289a2e50d7004d6 is removed from metadata 2024.10.22 02:26:47.603144 [ 1476 ] {} CacheMetadata: Key 68669cfc0d579172400ed988fad8de19 is removed from metadata 2024.10.22 02:26:47.617045 [ 1476 ] {} CacheMetadata: Key f95918152eb607816d7e31abea147610 is removed from metadata 2024.10.22 02:26:47.703581 [ 1684 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/bfa/avxxnnxigpxiwpdzqkobgmogxhcag, size 3318905 2024.10.22 02:26:47.803457 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_188_188_0 to 202410_191_191_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:47.960064 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:47.992882 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:48.016720 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column URLCategories 2024.10.22 02:26:48.122986 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column URLCategories 2024.10.22 02:26:48.152969 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column URLCategories 2024.10.22 02:26:48.178869 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column URLCategories 2024.10.22 02:26:48.210959 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column URLCategories 2024.10.22 02:26:48.237488 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_900_900_0 (state Temporary) 2024.10.22 02:26:48.346471 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column URLCategories 2024.10.22 02:26:48.362183 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.49 GiB, peak 2.00 GiB, free memory in arenas 5.34 MiB, will set to 1.57 GiB (RSS), difference: 83.51 MiB 2024.10.22 02:26:48.371773 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column URLCategories 2024.10.22 02:26:48.450143 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_900_900_0 to 202410_907_907_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:48.473362 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1363266 2024.10.22 02:26:48.505090 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column URLCategories 2024.10.22 02:26:48.639650 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.074318084 sec. 2024.10.22 02:26:48.639540 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column URLCategories 2024.10.22 02:26:49.203817 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_898_167} MergeTask::MergeProjectionsStage: Merge sorted 1363893 rows, containing 4 columns (4 merged, 0 gathered) in 42.777874781 sec., 31883.140688554722 rows/sec., 478.15 KiB/sec. 2024.10.22 02:26:49.392195 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_898_167} MergedBlockOutputStream: filled checksums 202410_1_898_167 (state Temporary) 2024.10.22 02:26:49.617430 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 14425 2024.10.22 02:26:49.743254 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:49.745796 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:49.772907 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_940_940_0 (state Temporary) 2024.10.22 02:26:49.828491 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_940_940_0 to 202410_953_953_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:49.868244 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_948_313 to 202410_949_949_0 2024.10.22 02:26:49.869480 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14425 2024.10.22 02:26:49.875754 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:49.881303 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:49.923678 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_949_314} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_948_313 to 202410_949_949_0 into Compact with storage Full 2024.10.22 02:26:49.969737 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_949_314} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:49.995453 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_949_314} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_948_313, total 14449 rows starting from the beginning of the part 2024.10.22 02:26:50.034292 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_949_314} MergeTreeSequentialSource: Reading 2 marks from part 202410_949_949_0, total 21 rows starting from the beginning of the part 2024.10.22 02:26:50.695992 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 9.02 MiB, will set to 1.57 GiB (RSS), difference: 6.82 MiB 2024.10.22 02:26:50.765731 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_889_258, 202410_890_890_0] 2024.10.22 02:26:50.768126 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_889_258, 202410_890_890_0] 2024.10.22 02:26:50.783872 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_889_258, 202410_890_890_0] 2024.10.22 02:26:50.898600 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7594 2024.10.22 02:26:51.137291 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_845_845_0, 202410_846_846_0, 202410_847_847_0, 202410_848_848_0, 202410_849_849_0, 202410_850_850_0] 2024.10.22 02:26:51.140375 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_845_845_0, 202410_846_846_0, 202410_847_847_0, 202410_848_848_0, 202410_849_849_0, 202410_850_850_0] 2024.10.22 02:26:51.172396 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_845_845_0, 202410_846_846_0, 202410_847_847_0, 202410_848_848_0, 202410_849_849_0, 202410_850_850_0] 2024.10.22 02:26:51.314990 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_949_314} MergeTask::MergeProjectionsStage: Merge sorted 14470 rows, containing 12 columns (12 merged, 0 gathered) in 1.399799767 sec., 10337.192747939642 rows/sec., 1.62 MiB/sec. 2024.10.22 02:26:51.356011 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 106113 2024.10.22 02:26:51.385308 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:26:51.390404 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_949_314} MergedBlockOutputStream: filled checksums 202410_1_949_314 (state Temporary) 2024.10.22 02:26:51.392589 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:26:51.444373 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 170 entries to flush up to offset 103018 2024.10.22 02:26:51.551377 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:51.556681 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:51.576209 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:51.617579 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1585_1585_0 (state Temporary) 2024.10.22 02:26:51.750872 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:51.757788 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1585_1585_0 to 202410_1598_1598_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:51.779159 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_949_314} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_949_314 to 202410_1_949_314 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:51.783768 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_949_314} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_948_313, 202410_949_949_0] -> 202410_1_949_314 2024.10.22 02:26:51.789353 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1207_1593_122 to 202410_1596_1596_0 2024.10.22 02:26:51.791223 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_924_924_0 (state Temporary) 2024.10.22 02:26:51.801498 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.30 MiB using storage policy from min volume index 0 2024.10.22 02:26:51.808956 [ 1218 ] {} DiskLocal: Reserved 1.30 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:26:51.809998 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:26:51.812199 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106113 2024.10.22 02:26:51.843663 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1596_123} MergeTask::PrepareStage: Merging 4 parts: from 202410_1207_1593_122 to 202410_1596_1596_0 into Compact with storage Full 2024.10.22 02:26:51.945505 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1596_123} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:51.963018 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1596_123} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1593_122, total 26686 rows starting from the beginning of the part 2024.10.22 02:26:52.020062 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1596_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_1594_1594_0, total 43 rows starting from the beginning of the part 2024.10.22 02:26:52.066904 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1596_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_1595_1595_0, total 61 rows starting from the beginning of the part 2024.10.22 02:26:52.110950 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_924_924_0 to 202410_930_930_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:52.154654 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1596_123} MergeTreeSequentialSource: Reading 2 marks from part 202410_1596_1596_0, total 44 rows starting from the beginning of the part 2024.10.22 02:26:52.257431 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 103018 2024.10.22 02:26:52.341049 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_867_24, 202410_868_868_0, 202410_869_869_0, 202410_870_870_0, 202410_871_871_0, 202410_872_872_0] 2024.10.22 02:26:52.344281 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_867_24, 202410_868_868_0, 202410_869_869_0, 202410_870_870_0, 202410_871_871_0, 202410_872_872_0] 2024.10.22 02:26:52.365508 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_867_24, 202410_868_868_0, 202410_869_869_0, 202410_870_870_0, 202410_871_871_0, 202410_872_872_0] 2024.10.22 02:26:52.665251 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_898_167/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:26:52.741773 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.37 MiB, will set to 1.57 GiB (RSS), difference: -5.15 MiB 2024.10.22 02:26:53.830620 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_718_146, 202410_719_719_0, 202410_720_720_0, 202410_721_721_0, 202410_722_722_0, 202410_723_723_0] 2024.10.22 02:26:53.836441 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_718_146, 202410_719_719_0, 202410_720_720_0, 202410_721_721_0, 202410_722_722_0, 202410_723_723_0] 2024.10.22 02:26:53.878441 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_718_146, 202410_719_719_0, 202410_720_720_0, 202410_721_721_0, 202410_722_722_0, 202410_723_723_0] 2024.10.22 02:26:53.992303 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 780 2024.10.22 02:26:54.063545 [ 1420 ] {} S3ObjectStorage: Objects with paths [common/zsf/xsyjeshyptfcswbsrhillhfournnd, common/hqv/bdvgznsefkvndpcrtqcdcrakdifmu, common/qii/ukptiweeoifpltdnxslrzdkokyrel, common/azk/lzeeewnayoplcvbkpypvsvaoofdgt, common/kre/lfwzolylanbitevrdjxzkmhywkygw, common/vqa/nuwbinvlglcpbtzgxlnqzlghcobhu, common/zko/zfwnpbmyrfbzftjmiimqzphvxqola, common/lfp/yxmtjuoyqcvddstmhccpotorimdff, common/frn/ytmnieddvebpaacrlsfjdffxvxscg, common/bma/fvfajuwdhhkfbkjhrgfyvwbhjnkzh, common/ttk/cmcryvvbchkmjtxvzfpvkaxvhptnp, common/yvm/lcuflsddcqwhbzyvdwytydlpfetgz, common/xjo/gceaezqhcfaegkfamueeeopwewtjg, common/zna/hxrqjynkisycuintbpxtdposbltbl, common/iqa/askkytscedxugcubjhsbzayshbqma, common/muv/fqcasvfhcvcrvombfxcqgddbpmlld, common/syr/rqkdfvqplhrpaziryzfwioikedgvj, common/hik/pbwzxgecrsrzkbhqwadzguxdnfvyk, common/npe/fpvcqnuoosfgbcvhsynhjjoagihoo, common/xuq/jrxvqzxjtcrigbbxkohzwvlqmxnje, common/jsn/jhrywpoticqwvzvsbqtfhrrwamtgk, common/sbz/rydfmbaikmncqbohilqddtfnwtvib, common/vrb/tbfzyglginofzktdgzsegfonriktq, common/xsd/miavctxxnobfiwemsowbxhuwqcxsu, common/khi/pvbxpiqcoiexuytcjhalsbudhxwdw, common/emx/opgmoknuyepyppuroplhgtlqmjyyw, common/xzd/qcxovlniuknulaachzmacxxvsvspj, common/ttp/iwlqmjshyfpizlebyxchgqvswqtaj, common/cnk/lxmruzsnzigbntxdolxmicytbqtfy, common/zsx/irihubvwflxekqlgatdmtftoqvukx, common/oxn/mvkjsdsvmlgwzmhrescuksbekajey, common/csv/qphpgctkkmkugwheioymxcuahvpda, common/wsr/nydolwdsqrgoyhtlloigldekhkwlp, common/vnm/zyvkmwyoejtpvdchplrxahoyrhcqe, common/jty/ngvhixzupscbisxgdqmsswmgpwayx, common/epz/nuiukevchudyrmawjuvgjpcovbpuz, common/lsv/aujapkatjqqxvfupnmuofulqbscdb, common/caf/njbfevkgntmtyuxxhbksxfvxxrnbl, common/hhg/dwzfhlrxswlsmihdriugmltvoyroi, common/ngb/eyqnlpolnajvekfknrmefsekaxkws, common/ajw/zcsqbhxqefzkrmcrtrzamtyagftik, common/hhu/wrqcxrouhyrdjmfiyjfudaygtuyxe, common/fdu/nnvifzgcvixtsaonxdgvexozffqac, common/pwi/ivvocurornfvquxrotezcalxyddsf, common/xej/olcutyrwvpetfkbklovylbzumsvam, common/zzu/nazrbvzxkswisznmudvhsruklzljq, common/ndb/zegodlhywkiocxnzoptgxqsipiikz, common/rnk/eibryyggzaqhkzfxawdhwbpgupuok, common/svh/pnfkrnvwtnkqipwokfsyfqapqnxmn, common/boc/ijsnagqsrgduzttvrvaroyhpslium, common/zzy/lepnxgxrppvddohquojbkuwytxzjj, common/zii/kqyuuxekeqeuugfroyboqzyhtgzlw, common/xjq/wgrasaoicrmdszkgffpxrtcipcsoy, common/kjw/iuydockngobxkwjbydbvmckfeaztl, common/jqg/vmavtbeqawtdvhebhfkyxqgxppmtc, common/sir/pczainlrlpigwxnmgmthkjqwccnct, common/nuj/lajhgxhisfffxbvfpjscbpwxthwvx, common/shk/itgcdmsxhnmmcmhzcrfsftyyrjdin, common/rst/ppwpocjcecvqyiwngpzfahzvgwuls, common/ney/fogxcxzgtieaffzzwjglyfomwxioq, common/qur/ncpyayezzscwnotumrbsvroyeainz, common/ahz/ozigmeibgxmgnxuvjmjtcgtqivgjz, common/ryv/ehdsqmapgdkixuzlncvwiyobkuuhk, common/wma/gmamggmpuqxulauaysiccvniilmpo, common/gbu/ocxjfivhvykcnxxzocrdmjxsvbrsr, common/tyx/oxosbemrxwtlcptufjdawxqlymrhx, common/oco/fnmoflsxpppackhvkcatwgcaqqqyh, common/yip/rqngfqylrlrgjocgfmktvidtquzqd, common/wbx/opsibaudlvolvtnetqydlwthxyqug, common/ost/lgsnwrvylpaqnagsjqhrhfnmiufai, common/lso/xqibvivkaozgrzecnxchkbkuyuavm, common/yrr/qdonqlzorodvrzkdiajuhrawjqesg, common/gnb/cexffpxhcecscyscgdybgcypyuxfi, common/yug/boryxvxfxoeiacmgwdqcfgeguzpzm, common/hja/orqkafcaqbqlsnjpwiyeizighdquy, common/tvh/wmswlowskuibmmjekzobxzasbtpje, common/sml/fdecedmkencrkvzbjvvehbljyvukh, common/jdt/ouojbczgogcqcqfdnzxnsbgqqiohu, common/ylo/mrwovyrdgdbbkjsifbxutbrrfjylz, common/odd/cyidbqpyrvalfncsfnoicelnbrqcr, common/dms/vvxgbzeuwubyamuomqniquxmgcgeg, common/eya/flfwgjdcvaiuaaewtzvwbeziwunqd, common/mpl/whugjczldojegzkufnfnhhfzxvyjk, common/xio/uufykozhrcimehkwspyprzpczksmv, common/teh/kfymfszqcmvhbsihmygmxukdygkgj, common/hpf/ohlahrgsjnhiiobkdsompntfghwoj, common/lou/uzerjtfpjfvfjycwyztkdqjgrvjrm, common/xiz/iewygfiammyhjhiplskbyohjlfhlu, common/txb/danavztdimpegzwcescllrwsvysdu, common/twq/chzedczhvujkodjkponypxtyzdkqx, common/mlq/lpiovvpgjkitjvswhzirkbnrwdnmi, common/djw/qjhuccgjfzannnkabqzcffmxguvmk, common/upp/teezlwtotaywemejuqfytyiggyjmg, common/gfx/alkldzhaosirclaqwanhsgvcyldyt, common/zet/jwsjunrxjdinlyqizhezqeoatogsg, common/pxf/ikffahumqzrnsvfaedzkxeunerogs, common/dtf/slwjgbnifpjzwknlxpzqwlqasuiyk, common/bgc/flzttobdmwglfszofbncctmjsriaq, common/vld/rascgazbagkipndtrvsqnegrzxosc, common/kuo/ccocdoqjndvjrhkkbocbxcmoeuprr, common/dts/xdhlyctrmeegbleowzghfpgguszso, common/qmk/uamxybmvwrjuwqlveveqejimybtod, common/oqj/qsvnegnyqlnxbybvwfusmpluuprja, common/omk/yndwoyubsecpdozssmkgssgapaunv, common/lcz/pytcowqcnfuqjexbbixiyoxdppyxw, common/myr/azfauzhxtktczdipkuhttnxhxngpz, common/vor/dlrctnrzxetfoeunwwxbgdyyolpnl, common/eak/kzlphmizhdqipulahduokcruwgntw, common/avk/qidspmgithzrbhuazvzxorwsxculx, common/isj/hkriwdyupnqoezwpgztgojvwcpqda, common/eet/zhexksgivoarfunojajsgvsrvqnph, common/jvp/ximxbtjedqapolyjkxmwavvyodimj, common/jnp/lcmpebylsoenfmjgzkonabxcthxgs, common/qht/lovgodflkstmkdjnoesaobbamhquf, common/ogv/hiprxqgcbhmxucxzjxlgeuxhwhxwe, common/azh/nfnexshsbqawmjuywdbvlcxngzxqh, common/kmp/urlylpotfakqsfhgurxdqabxwkdmv, common/ssd/pdggwzztlqzdbkfogykdsrvykzjlc, common/lfd/dqmngtrbvhdrqawdlcautbpivzghr, common/wrl/agqkmsnhfofjooxwcaztnnktkbmwf, common/mah/cubkvgopkztkgfbimucgjmuwdrptk, common/yxg/fwojyxkxxodadbrzqkmhjetwqwmyf, common/nrg/hdudrcefxlmripmbuapdwypwgqdjv, common/unq/mueglhyhrfyooppjmamtzvzkgyieb, common/fmh/uxznrzacbrfjxlkintyrocyjkokwy, common/fwb/ussgstqeuhgsolkqcvdnnnohfjfhq, common/nrl/vxinjlqoqhusluizhgeuzbrhzxsuc, common/noy/swhknhkkogkoviogwxkhkvyrpaalc, common/mbq/rvnesejflcmmlteysblukvmuqqxub, common/jht/sutykmdzsemdwbcetqcekyhapeuqd, common/xvh/ipackjwnvkccyhmxogtwzoehwvllp, common/lrt/aldwvmhqbxfidqbinddhcflpugeuc, common/aks/vqfvijqpvbpypyzisrcztqzawehnu, common/yao/buuuadjnboamanzvgydtaploodgps, common/clp/qmfbijbuscolzyweahxkihsuztyqp, common/tow/gcsyfbkkwfjeupicskqqmtmdvahan, common/vks/gnublvepvcsjjcpzibydrprbyhxsu, common/jez/rugirpvesbktujddqzeqcpeasupeg, common/hov/muzhjdakxtkqaqxcmjhgrgxrlncyv, common/hty/oyhkkzxdbqndkfovnuehnkrpurwrt, common/zoa/dyczvixwdarmgovqawbdfjjyprjpg, common/qni/uorafakohdvgctbcbqeuxifcrwtos, common/qgt/gqdgibgwjtxgtijezxraczvawxvvl, common/ysx/nshclippzevqeokrtddyckwvuhdjy, common/akd/gefmybarssbmnyedsvhqklcvvcypb, common/rul/mbkjqtbjfbhjpupnlutbeguvetdbc, common/snk/bngcmrkquythjddwqkrakqxmvcpsa, common/ydm/tpcnxccdktppmmwzsrvpilpjczzky, common/wxl/bibfldkverkkjhteochpdkzazviop, common/jdf/zxricqgqpklgkrxbzqhuctcxeelya, common/bix/ktpsufiilxtttibucaxvwaptmlqxn, common/ava/lgodsdnnmxrftysfjzisbpsjfuloc, common/cws/mwwtlvgzipxwgqlgdbiwsfqtndxpv, common/voe/mmyfpboiflsjqqbawtwyusqzotqdj, common/ewo/vcufpezmarrkdynheotolrilzwfpo, common/zuc/teffxkgmdiugyppmbaocumajimrtj, common/fuv/qvaelgatbcnxeqhsybgcbtykmdyjx, common/oeq/ukgjhbgeciplofmnebwmbgihxbuim, common/ssw/qsqjfqtwuirszyykuyhrlgyxaizij, common/xfb/ewrvufsnnyqsqhdcndwgahfcwmfzt, common/ssk/xsznpisdqlqkftzplzepcatxcphol, common/gcv/hygglbboxbdhvyxpfqtamdmchahen, common/fxo/rvgzviecalihdcprwubcvyknvwqyg, common/xfg/cuhxwtqznoeytqzqebqjhbavfkspa, common/sla/egzstbltavyabsfsebrpwsilijzdj, common/qtx/vgtyhyccbsbahmrrknydpgretstsr, common/dvb/ksxhfxhjhxiragxkioipzobdfvjvn, common/aht/rpetpltnbhnnvkaixfmxqalmqgyqs, common/lgv/qnkyutmjvkfwzskrahpcbsmwfivga, common/eid/iparalikfjpfleympuhgjsgzcznuv, common/mqs/asmomovuawzwijywlrfbvbklmakbi, common/pba/xzmxnssszfuszsvtkcddmxqrayfpa, common/lje/asbmdsbitrfmmohfnmhueqgeutndc, common/udi/qxfdpbexzbitrbhkqczpkagbxslrz, common/aea/mtxprqhcnwjycucflyyihfqcltdwy, common/csd/dvsqedrwagkgapwmdsuyuvbgwktou, common/cdn/pqltmuhpwyxdtngiagccnwhuivxbg, common/zeq/jpaixddejbbygqcxzargtmcddpwkv, common/cqz/oijqaadqlijblqkeczgrxvhowiprg, common/rmr/cdwzbuxqqaxxopwldssreqbxjoivh, common/vxg/pyqsvgojcmhxrwiyvdgfuruchbegj, common/yjw/vixlyuurwaaojcrpfcytfhkneofqc, common/zrr/rsrfsbgcfuwzghhneapocuqgfvlko, common/aud/pizxbksxdadvybxkrooqgwxdtubzk, common/cud/ngpzjzolrkhlogdxmnhmyzmvttrjl, common/pmz/ejxuzxjgecvpyozztcktxbcvvdunb, common/tbf/fpzcebublaolejxsmmtcowsyxkzrg, common/por/bycltiemxdwngqlpxhboonoujafba, common/und/xxbtvykhqbfxwejczckyhcrfcbddi, common/gge/mhpdnoeezixotpnkafabrlwdupbgo, common/sjr/tekzfalgbvpihvwwjsjchhuffvcja, common/vna/dklbozwwqgfnqgtvdefosoydzytzu, common/lun/ehgmhxzwexwpmudgtusxeyoggfhpo, common/zku/wlboopopvpjasqgiflqymnhdhugef, common/hny/wavjhykbfpigsxkxmykeyohmfrkyp, common/eul/kdtjmaldbzvpxdzlkumqhajkfnvjf, common/dew/djhokjnivfhlltjnlllfwzvkgorqu, common/zsz/vuyfbvoydfbjjqxfrclcdjxwrzqau, common/ytr/ufkdlzhmmznlpoqwisondstaouefj, common/zjo/zavauqobtzhupclrvqyndclkdmmfl, common/jch/fxhofncyqywzhhczldsbdgdjoqvbx, common/pij/hwbrqnvlrntcronekzfwxrbltrklk, common/hkq/scacxixgrteobwrenljayuolocznb, common/xwi/ykcaayqstwordxpjtlxukdnfdojsr, common/jgr/jeuqsetvuahminrfdaiahqaygqyes, common/uhr/hezdjzlgxmomfvbsuntunvujktrwj, common/nqp/gplqdfudthpffynovgjrcsqvmzkqd, common/ggv/jwcmdihyecctruarbyqnojzeqbrce, common/pwm/dhmjcewirllhiuzbhfixaeahbibdg, common/oxy/skmjptkaiuhecxchxqgqolmptxrvi, common/fhj/mpnneelioxaiuzmwwxmxmlpdafhyg, common/npt/qltbcagstdfeqvpqzxxneixpnlcko, common/hmk/jewylbsdyafntngkbleyjoaopfbtu, common/cms/sdkkmlkljsslechxadkagrsffeftu, common/ujl/dhkfirzehkaxxjsyavsswxgcdlhkt, common/dyt/phkxgaxbskjkhdezhhhnvmvbfwsxm, common/bef/aqonwbrfdgvnunixlxffqihaznxwh, common/gkn/osvcvbjijwlwafriiumajxpacqjif, common/ttj/xvwdpasdebbxrokiufztsubltnydk, common/ttx/qcxltcplskryycnrpyancmxsrndqe, common/rxs/zzbvpgqobuwjnsncpbyqlqshmirkt, common/scr/txqhkuaczivbntsulukcqucawfiwm, common/ejo/nxgsjqcszwyyqkhtbamzxaxlovnhh, common/bdk/sohpsyexihbgjefpnnhwjouxmvhqe, common/cfr/tqmzpdybxiuinsmqpzaeldhyunael, common/syo/qxkpqsxniglbshadkbwgbpfvyjhap, common/hnh/qxjicktxwcaysqetpsbnaxjjtzsmi, common/kop/bmykvnipapsjfqmpodqqocnbvunju, common/byh/tpywsokuhhmblllgzafvedmqjaybk, common/xuf/zntnphdqjouwagypsuendbmunquyg, common/mkd/orrqoiybtmsewmmlkljvkrsggqbeu, common/osb/mnynpaayuiihenlselgvmvuvnnmse, common/zau/vfizigmdrsnhnbxgpqhhjypulyxds, common/tkz/edieinnjtwcxzcbkegilytqaksakn, common/udv/nfzscnlyisewkmptlilqtrttaubjk, common/okm/xkmfhfusmvaplizzekzzxtmxksnfm, common/vtj/zgdqcyehovnsxedyrrfcydadvgcet, common/uxm/laxoggjizfhlbtdpdeuezzdnnslpa, common/csq/dclxfbtkzqlpterckhnytjaxvxzti, common/ndu/pcxyehgbthmqlwbojmwyouxkwkhrp, common/qcp/iuwmmvyzwhgdtokfueuzjidgzizjy, common/zak/tkvqgckqsijhgpcspprpimkvomlrv, common/dzp/sqjsqvlzpkbqdqtydsrffahajgezx, common/enz/xsmogvhwxsqkqbdwallrppiprbwgn, common/pzp/ujpkqkzsirevzwbvxwrvxyodyqyha, common/afp/jyctcyavuqsmxdpuawxnijptuypnx, common/bpx/rvxitjlzvzprndwxwtmwfsndpxqkh, common/xic/ggmuggidskaohonlhooyjmtphkndk, common/cik/marxaxmywuykutoxovpbnkiquxqau, common/yqs/cnyzmjcqvvnyacgxinkljuhphtgcj, common/fqe/xaawvhvfgwdlqxlqwkgzlwlwyldnk, common/umv/utznssernglrknrpgocvdyasnkqme, common/oyy/jwwirkkhuuwuarssflbktqqtlamxp, common/fua/qjstfuixczufqtifsjlrkswpicupx, common/fma/wucrznjngotdxbpiqvicukpbeemyr, common/ahq/tnkdyddkbgiukkzapcdvkcgjyhiao, common/xob/zfmleotuhdorrodizzntxzvrbkzzv, common/lho/wzdfmojgvvgrhahjdmpewqhllqtab, common/uab/doryohtabhpkakdhfixzmyczyvmhh, common/toj/akrmmbqtwcsdoyuoapcjggqkhhqpi, common/kkp/npgwgvuhoxuywsdljzvdwbehvuyed, common/fwd/jjqvntracbhgvlqriareekrlvyzsw, common/tty/idwkcajpnefrholuzgfwberuvinnd, common/ddu/jwlnnzdwlykqdoffvsuqgloyuguhv, common/gwk/hbmxlmzqtgkqbahcpzeioupghztyx, common/vyj/zoiozwjjwyiovfncyudyddreityke, common/oka/vlegdnckbzieelzplgljlttlvpvax, common/odx/pdrrrbgohponcprwsmdwavlirgjwb, common/jfw/vjhahhlfbfoiyyvcmutejhzysphaf, common/gan/qxubapqvwkabhyoxeuvnnmuhgiats, common/dgj/srrxfemjddffcblvpdnauguenseue, common/sim/kugpentejtbruhdsxtpskgbivfnzw, common/cbg/zvjgnsdaysbmdfxujyemfuvnrxhda, common/ivj/blbcnesgfqiixyeayrrgepqgobjic, common/rbw/hudjtcougyesjjqvvwjedbpvkjvsy, common/kew/hehnxzmdhcgrmijgtcsvyitwcksmo, common/daf/fxuzqunjrmcepxkplarxeosxyeaud, common/swd/crtrcpoeeafgibtcltekzmxsjbdtw, common/oyi/zpzodqgdylfsonbglsmvlwjnjrppv, common/ipy/vzvbhvuckrlsajenvgvwfrpwrzmxb, common/ddz/slpcfexfeznclkbengfjrappufbfp, common/vcc/vigtvrbtghekkgxqzdafiqpaqclrh, common/qzr/ctpzljtemcbdpwuplbpkmqovzbudr, common/wbl/sqystoqmnhkpkbtsinkylpqerxpnf, common/ash/pcwbjwgsppjlhcywkpacgvozsagha, common/cxf/qohtqkdzbodhrsprhpbvditzxezgh, common/kjm/rfguwtlznrplsldqxwbfqwiwzmfmo, common/ngl/dmddfnddnfaqjhlztzvwuvcbuzhkn, common/lxk/jamernjwwkudkowgxcqcaotlzlnqc, common/irq/jfgrvatcdjixjvlvkxsxemqmmggom, common/klk/xtzgyzzlgzyftrzcaoxsjikbnrird, common/ejb/wvgkabagpuvnmofaqximavwceuzbk, common/epg/onrjjtpvrjkkicaajgfjyceuljckp, common/mfr/fnjigdqpigmyaphffekwhceurdhev, common/xpe/gbdttrhjhnjixyebwtzihqnxlelcb, common/udd/xsuhxeelghqmlcbjkstmxknlsiztd, common/kgy/cqwgpbgzpfuyobgousfqmtiffgubk, common/izr/hpahlwrljypolkvmegalzwqhpuuka, common/vti/nvpjuraqzsxzjrmzhixcsojjgwxsj, common/szm/qjbclxourtvicmfzjjnkqahmxwphw, common/mmx/zovnoalzpmgbueqnftvbuaubapfdi, common/bvb/qdiqibswcorjbgqubepeoychfljxk, common/anw/ityhmmiimpajyqvngrwsfrniatrxo, common/bgq/xetzutlqtbnahheghrawkqfsekgxk, common/xrk/fbrvrocvrbrrvtrbikwmejqnvxxmt, common/urg/zqefuwxtmjaplfdaunebpeoczunwy, common/mjh/mwqmphwkmdheqdxiavhmlmuasodhi, common/yji/yyddirtfvefwfkfxbfmfibfvpnoic, common/vif/hivaiilwqegdadnrqndfeqgwxbfqu, common/tdn/ohlqcuqvknxqazpuyjrwbmnxkulgm, common/oeq/qidmiejndiyeuwwnflxndybkvfdej, common/mfb/ayjsuarffrfmtrxyqttgxktojmqap, common/vgw/xftqenizvhzfbzmctmkxplygmhhif, common/oeq/qbqbgszwcyygtwnmzpysjuezzrkdn, common/tth/ecsnibzfscfbgwvnwsqgwdjsweghl, common/sjo/zdgafvfrpceapqjyasrrfconjgxxf, common/dno/tkpvzrdgwgoamxdtyinwvjhwaxbdi, common/xnr/sebfqbhwadkdxjhprsiuzuckvhnhs, common/jcc/cbobluhjismrqjgmpyjelzzdblmtg, common/bwa/jrhkigpfrofvzwqqnostofdckxoyd, common/snl/xxnflnuehkvysqbvjasxffyupatmt, common/hly/pulyurittqcltxefwuweqiilpjepi, common/pdm/mnyhqchaxzayquxccphysljfxydlj, common/hzw/gnaokqvtuzoxjbqahakqtkgalwrjh, common/hxd/hdqcaakkoamlzoziifjbjoddjivpk, common/cip/tsxkvgzqxxhiipqpxgdrjmahjyqoq, common/egw/ktzshhkxyesfldvkgfbvalwwpaycu, common/gso/vfpemgaziuuillqeflftlbnkamgen, common/spf/drtksgigksecoznmyqcaajqyiornx, common/duy/opulruqcnydrjfnrlqdimljqfocwy, common/dac/tgytjipdeknvexociaznobabijzwp, common/kxl/frivnuwcmoreazgarzzjpwlwensxp, common/xpu/dgihopfbdvhowklkfnrfuptkkulze, common/wxf/bfnfzepaoaqavdxmqgaeztsmnfcpb, common/atj/pwsuqfowgowqybyksznaspldoqkms, common/vcz/dzwlfpsmzvwcwtweehucqyomlqxpr, common/mwa/gspprfwfxzgzirunyfrgfxpfxzgwe, common/vix/oefxdovznnkweqrlhergsvyfnhoay, common/txt/cedagrsvgrgffjvtbafkkoncfwadw, common/pnu/zqqilgtimmtttezegmiqiwqvldwzx, common/dbj/vwfxkmtijqgkcupurrvkbvfkztzcm, common/dfx/cqpttzzpofzjhxnkkhztvavxvndzm, common/gjc/ivxqxvrurlkfnrlumngcjxfuebqjz, common/wqx/zoeealrtsrarnivsoecxlnjocupvo, common/ntl/wynklbkndyrgjdhyyxzrxidcuqgub, common/qre/prosnzgrmogijgkfslfsejtbnpdqp, common/qdj/uwxnplgaikgprxddchkbzelwriwih, common/qjw/ihqrwiqlotgdtmviiqeyhcopmgbpz, common/yyb/jnclxsvovqlohzalfnqorkaygevws, common/bjn/cjulpnkxllxahvkekorozfzrwxrdu, common/ild/vcitmrfennkyvcohdrofwbkiwklgp, common/nha/joqnvvbgcaslzroyoatluecygtlmq, common/uvb/vwhnglnrxuabuvhhxecrlumxpgqym, common/vey/schungisgcdztwipdoqadkaqkqexw, common/nyg/qirsclrbadjgosdofvkmanvladich, common/mck/ojpftlvmlinbbdikekwnhrhrhovll, common/tsa/zhklslvfejahxjjbuuebjxsevkqjt, common/lcv/ahwqmjqruyjwjxeavyfrllthtaciz, common/soe/hybyyhhdlivtlzmekghhtuoglcshr, common/zeb/wwjqixwulqawwttreuujnfvwidhfb, common/gcq/qqpaokbtdxfiippvhnnmvbzqdhtpm, common/uma/ybelrvtugvpuuaspznobpscmgrspn, common/azr/kqrabonvklkkmgfpugkbmtxbkbjcf] were removed from S3 2024.10.22 02:26:54.126805 [ 1420 ] {} RemoveManyObjectStorageOperation: metadata and objects were removed for [store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMTerm.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HistoryLength.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CounterClass.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsRobot.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ClientIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EKey3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsMobile.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/AdvEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FUniqID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/BrowserLanguage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SilverlightVersion4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EKey4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialAction.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/AdvEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Refresh.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/NSToDOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FirstPaintTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UserAgentMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/BrowserCountry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/NetMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamOrderID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Income.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/BrowserLanguage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/JavascriptEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WindowClientHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/AdvEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/DOMCompleteTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMSource.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/LoadEventStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatCampaignID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ClientIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMMedium.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/BrowserCountry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CookieEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/EventDate.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/DontCountHits.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UserAgentMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/columns.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Age.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HasGCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ClientTimeZone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenerName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/partition.dat, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CodeVersion.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/EventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HTTPError.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialAction.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/DOMInteractiveTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UserID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CodeVersion.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/DNSTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EKey5.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FromTag.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ResolutionHeight.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsRobot.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ClientTimeZone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RemoteIP.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WindowClientHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ConnectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Interests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialNetwork.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WindowClientWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Age.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamCurrency.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CookieEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/DOMContentLoadedTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/MobilePhoneModel.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/DOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/GoalsReached.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HasGCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/DOMInteractiveTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EValueDouble.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HasGCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/DontCountHits.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatServiceName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/GeneralInterests.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RedirectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FlashMajor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsLink.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/EventDate.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamCurrencyID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FetchTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsOldCounter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FlashMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FlashMinor2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/PageCharset.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FlashMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/DOMCompleteTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FromTag.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FlashMinor2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/EventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsNotBounce.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FlashMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/NetMinor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FromTag.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SearchEngineID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SearchPhrase.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/LoadEventEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HitColor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatSourceID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Robotness.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FromTag.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareURL.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/GoalsReached.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/GeneralInterests.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/GeneralInterests.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HTTPError.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareService.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/MobilePhone.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WindowName.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Interests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ClientIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareTitle.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ClientEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsParameter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/AdvEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WithHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/GoalsReached.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FetchTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ConnectTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RedirectCount.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsParameter.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EValueDouble.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMSource.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HTTPError.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMMedium.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HTTPError.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareService.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/NSToDOMContentLoadedTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialNetwork.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMCampaign.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HasGCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMContent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HistoryLength.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/GoodEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/HitColor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/YCLID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ClientEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SearchEngineID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamPrice.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IPNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IPNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Income.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RequestTry.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/GoalsReached.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ResponseEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatAdID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsArtifical.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsDownload.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsDownload.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/NetMajor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsDownload.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsEvent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatCampaignID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsEvent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/GoodEvent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMTerm.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsEvent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsArtifical.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsLink.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsOldCounter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsParameter.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatCampaignID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Sex.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsLink.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Params.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsLink.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CounterID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsMobile.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsDownload.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/GeneralInterests.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMTerm.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatAdID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RegionID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatSourceID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsNotBounce.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsOldCounter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsOldCounter.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialAction.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Title.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsRobot.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CounterID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/checksums.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatAdID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EKey3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/CounterClass.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsParameter.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RegionID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/YCLID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamOrderID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareService.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IsRobot.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IslandID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ClientIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/JavaEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/JavaEnable.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ResolutionHeight.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialSourcePage.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/LoadEventStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SearchPhrase.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/MobilePhone.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/count.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/NetMajor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/IslandID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/NetMajor.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamPrice.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/JavascriptEnable.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FUniqID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/NetMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/NetMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UserAgentMinor.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/NetMinor.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OS.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/LoadEventEndTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OS.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatAdID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/TraficSourceID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Params.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatCampaignID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Params.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatServiceName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ResponseStartTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Robotness.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareService.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamCurrencyID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/MobilePhoneModel.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EKey2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatServiceName.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamCurrency.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamCurrencyID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EKey4.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/DNSTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SearchEngineID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamCurrencyID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamOrderID.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamOrderID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenstatServiceName.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/PageCharset.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SilverlightVersion4.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamPrice.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EKey1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/OpenerName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EKey1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParamPrice.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Params.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ResolutionDepth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EKey2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/FirstPaintTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams%2EKey5.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Sex.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ParsedParams.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RedirectCount.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RedirectTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Referer.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Referer.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialSourcePage.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererCategories.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SilverlightVersion3.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UserID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererDomain.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererRegions.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RefererRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareTitle.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Refresh.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Refresh.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Refresh.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RemoteIP.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RemoteIP6.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RemoteIP6.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RequestNum.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RequestNum.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/RequestTry.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ResolutionDepth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ResolutionWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialAction.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ResolutionWidth.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/YCLID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ResponseEndTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ResponseStartTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SearchEngineID.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SearchPhrase.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/minmax_EventDate.idx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SearchPhrase.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SendTiming.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SendTiming.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareTitle.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareTitle.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareURL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareURL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SilverlightVersion2.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/ShareURL.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLRegions.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WindowName.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SilverlightVersion1.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SilverlightVersion1.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SilverlightVersion2.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SilverlightVersion3.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SilverlightVersion4.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SilverlightVersion4.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialNetwork.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialNetwork.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WatchID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialSourceNetworkID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/SocialSourceNetworkID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/Title.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/TraficSourceID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URL.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URL.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLCategories.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMCampaign.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLCategories.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLCategories.size0.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLDomain.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLHash.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLRegions.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/URLRegions.size0.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTCEventTime.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTCEventTime.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMCampaign.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMCampaign.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMContent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WindowClientWidth.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMContent.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMContent.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMMedium.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMMedium.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMSource.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMSource.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UTMTerm.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UserAgent.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UserAgent.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/UserAgentMinor.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WatchID.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WithHash.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WithHash.sparse.idx.bin, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/WithHash.sparse.idx.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/YCLID.cmrk2, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/primary.cidx, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/serialization.json, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/default_compression_codec.txt, store/18e/18eebfd3-a0d1-483d-948b-5d8c6de7d937/delete_tmp_201403_24_24_0/metadata_version.txt], only metadata were removed for []. 2024.10.22 02:26:54.135200 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_898_167} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_898_167 to 202410_1_898_167 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:54.365637 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:54.370341 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:26:54.374768 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_898_167} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_892_166, 202410_893_898_1] -> 202410_1_898_167 2024.10.22 02:26:54.385813 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_189_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:26:54.429760 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.57 MiB. 2024.10.22 02:26:54.468899 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_762_762_0 (state Temporary) 2024.10.22 02:26:54.470174 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1596_123} MergeTask::MergeProjectionsStage: Merge sorted 26834 rows, containing 13 columns (13 merged, 0 gathered) in 2.64773358 sec., 10134.705471386587 rows/sec., 1.94 MiB/sec. 2024.10.22 02:26:54.469818 [ 1420 ] {} test.hits_s3 (18eebfd3-a0d1-483d-948b-5d8c6de7d937): Removing 8 parts from memory: Parts: [201403_17_17_0, 201403_18_18_0, 201403_19_19_0, 201403_20_20_0, 201403_21_21_0, 201403_22_22_0, 201403_23_23_0, 201403_24_24_0] 2024.10.22 02:26:54.572584 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_762_762_0 to 202410_772_772_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:54.583240 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 6.30 MiB, will set to 1.57 GiB (RSS), difference: 1.78 MiB 2024.10.22 02:26:54.668160 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1596_123} MergedBlockOutputStream: filled checksums 202410_1207_1596_123 (state Temporary) 2024.10.22 02:26:54.784978 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1596_123} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1596_123 to 202410_1207_1596_123 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:54.793087 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1596_123} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1207_1593_122, 202410_1596_1596_0] -> 202410_1207_1596_123 2024.10.22 02:26:54.851520 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.78 MiB. 2024.10.22 02:26:54.863959 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 780 2024.10.22 02:26:55.490419 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1667 entries to flush up to offset 1364933 2024.10.22 02:26:55.617720 [ 1238 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:26:55.647756 [ 1238 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:26:55.725939 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1493_70, 202410_1494_1494_0, 202410_1495_1495_0, 202410_1496_1496_0, 202410_1497_1497_0, 202410_1498_1498_0] 2024.10.22 02:26:55.733402 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1493_70, 202410_1494_1494_0, 202410_1495_1495_0, 202410_1496_1496_0, 202410_1497_1497_0, 202410_1498_1498_0] 2024.10.22 02:26:55.835872 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 106172 2024.10.22 02:26:55.963414 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1493_70, 202410_1494_1494_0, 202410_1495_1495_0, 202410_1496_1496_0, 202410_1497_1497_0, 202410_1498_1498_0] 2024.10.22 02:26:56.067568 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:56.069429 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:26:56.351206 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 3.84 MiB, will set to 1.57 GiB (RSS), difference: 4.60 MiB 2024.10.22 02:26:56.534393 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1586_1586_0 (state Temporary) 2024.10.22 02:26:56.592373 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1586_1586_0 to 202410_1599_1599_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:56.650267 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:56.653552 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:26:56.679777 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106172 2024.10.22 02:26:57.004994 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_901_901_0 (state Temporary) 2024.10.22 02:26:57.095228 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_901_901_0 to 202410_908_908_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:57.128003 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1364933 2024.10.22 02:26:57.128563 [ 1190 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_898_167 to 202410_899_904_1 2024.10.22 02:26:57.151290 [ 1190 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.07 MiB using storage policy from min volume index 0 2024.10.22 02:26:57.154605 [ 1190 ] {} DiskLocal: Reserved 1.07 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:26:57.167461 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_904_168} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_898_167 to 202410_899_904_1 into Wide with storage Full 2024.10.22 02:26:57.211739 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_904_168} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:57.223044 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_904_168} MergeTreeSequentialSource: Reading 168 marks from part 202410_1_898_167, total 1363893 rows starting from the beginning of the part 2024.10.22 02:26:57.250577 [ 1390 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_904_168} MergeTreeSequentialSource: Reading 3 marks from part 202410_899_904_1, total 8845 rows starting from the beginning of the part 2024.10.22 02:26:57.371760 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 14442 2024.10.22 02:26:57.525385 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 4.59 MiB, will set to 1.57 GiB (RSS), difference: -12.82 MiB 2024.10.22 02:26:57.551109 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 6 old parts to remove. Parts: [202410_1_890_259, 202410_891_891_0, 202410_892_892_0, 202410_893_893_0, 202410_894_894_0, 202410_895_895_0] 2024.10.22 02:26:57.593530 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:57.635794 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:26:57.635296 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from filesystem (serially): Parts: [202410_1_890_259, 202410_891_891_0, 202410_892_892_0, 202410_893_893_0, 202410_894_894_0, 202410_895_895_0] 2024.10.22 02:26:57.693292 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 6 parts from memory: Parts: [202410_1_890_259, 202410_891_891_0, 202410_892_892_0, 202410_893_893_0, 202410_894_894_0, 202410_895_895_0] 2024.10.22 02:26:57.785290 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_941_941_0 (state Temporary) 2024.10.22 02:26:57.848085 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_941_941_0 to 202410_954_954_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:57.890573 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14442 2024.10.22 02:26:57.922026 [ 1172 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_949_314 to 202410_950_950_0 2024.10.22 02:26:57.957723 [ 1172 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:26:57.961178 [ 1172 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:26:58.090763 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_950_315} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_949_314 to 202410_950_950_0 into Compact with storage Full 2024.10.22 02:26:58.139601 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_950_315} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:26:58.151004 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_950_315} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_949_314, total 14470 rows starting from the beginning of the part 2024.10.22 02:26:58.271951 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_950_315} MergeTreeSequentialSource: Reading 2 marks from part 202410_950_950_0, total 15 rows starting from the beginning of the part 2024.10.22 02:26:58.544418 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 7637 2024.10.22 02:26:58.915125 [ 1420 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:26:59.276279 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_950_315/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:26:59.347806 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_950_315} MergeTask::MergeProjectionsStage: Merge sorted 14485 rows, containing 12 columns (12 merged, 0 gathered) in 1.380621526 sec., 10491.651569396145 rows/sec., 1.64 MiB/sec. 2024.10.22 02:26:59.406272 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_950_315} MergedBlockOutputStream: filled checksums 202410_1_950_315 (state Temporary) 2024.10.22 02:26:59.466463 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 8.84 MiB, will set to 1.57 GiB (RSS), difference: 1.80 MiB 2024.10.22 02:26:59.902777 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 128 entries to flush up to offset 103146 2024.10.22 02:26:59.981331 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_950_315} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_950_315 to 202410_1_950_315 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:26:59.990281 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_950_315} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_949_314, 202410_950_950_0] -> 202410_1_950_315 2024.10.22 02:27:00.034439 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:27:00.045411 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 8 entries to flush up to offset 70 2024.10.22 02:27:00.192633 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:00.194549 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:00.302958 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:00.308019 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:00.348273 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_48_48_0 (state Temporary) 2024.10.22 02:27:00.389603 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_925_925_0 (state Temporary) 2024.10.22 02:27:00.633272 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_48_48_0 to 202410_48_48_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:00.667110 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_925_925_0 to 202410_931_931_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:00.702462 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 106228 2024.10.22 02:27:00.722259 [ 1199 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 5 parts from 202410_747_926_38 to 202410_930_930_0 2024.10.22 02:27:00.745153 [ 1199 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.15 MiB using storage policy from min volume index 0 2024.10.22 02:27:00.750269 [ 1199 ] {} DiskLocal: Reserved 1.15 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:00.761265 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_930_39} MergeTask::PrepareStage: Merging 5 parts: from 202410_747_926_38 to 202410_930_930_0 into Compact with storage Full 2024.10.22 02:27:00.841483 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 70 2024.10.22 02:27:00.857040 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_930_39} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:00.865564 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:00.865671 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 103146 2024.10.22 02:27:00.945177 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:00.963115 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_930_39} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_926_38, total 19102 rows starting from the beginning of the part 2024.10.22 02:27:01.112864 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1587_1587_0 (state Temporary) 2024.10.22 02:27:01.263631 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_930_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_927_927_0, total 93 rows starting from the beginning of the part 2024.10.22 02:27:01.299900 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1587_1587_0 to 202410_1600_1600_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:01.351862 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 29 file segments (while reserving 9955 bytes for fc68d579ed74ccf3e3c53af0d19caa96:0) 2024.10.22 02:27:01.408077 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 9b73e4ecff1d3bcd45653c496ebd769c, offset: 0, size: 1300 2024.10.22 02:27:01.424095 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 9b73e4ecff1d3bcd45653c496ebd769c for removal 2024.10.22 02:27:01.435366 [ 1476 ] {} CacheMetadata: Key 9b73e4ecff1d3bcd45653c496ebd769c is removed from metadata 2024.10.22 02:27:01.436860 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 99ffeb728cf49a72f98ab296275f3e55, offset: 0, size: 158 2024.10.22 02:27:01.449659 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 99ffeb728cf49a72f98ab296275f3e55 for removal 2024.10.22 02:27:01.462836 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d19de1f5cd6d7246e6e20e95679789cf, offset: 0, size: 155 2024.10.22 02:27:01.466665 [ 1476 ] {} CacheMetadata: Key 99ffeb728cf49a72f98ab296275f3e55 is removed from metadata 2024.10.22 02:27:01.466780 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d19de1f5cd6d7246e6e20e95679789cf for removal 2024.10.22 02:27:01.478168 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_930_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_928_928_0, total 131 rows starting from the beginning of the part 2024.10.22 02:27:01.489287 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106228 2024.10.22 02:27:01.489252 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:27:01.507950 [ 1476 ] {} CacheMetadata: Key d19de1f5cd6d7246e6e20e95679789cf is removed from metadata 2024.10.22 02:27:01.511283 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:27:01.513017 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 6ac694069b852d4f6b82017446497df0, offset: 0, size: 156 2024.10.22 02:27:01.520893 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 6ac694069b852d4f6b82017446497df0 for removal 2024.10.22 02:27:01.532640 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d93468c7b2031328311fcfd71c441196, offset: 0, size: 159 2024.10.22 02:27:01.533806 [ 1476 ] {} CacheMetadata: Key 6ac694069b852d4f6b82017446497df0 is removed from metadata 2024.10.22 02:27:01.538881 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d93468c7b2031328311fcfd71c441196 for removal 2024.10.22 02:27:01.544440 [ 1476 ] {} CacheMetadata: Key d93468c7b2031328311fcfd71c441196 is removed from metadata 2024.10.22 02:27:01.554614 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 10.73 MiB, will set to 1.58 GiB (RSS), difference: 9.53 MiB 2024.10.22 02:27:01.549699 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: dcc2c7c26d57ac01b9ea04b0b2d94387, offset: 0, size: 163 2024.10.22 02:27:01.659543 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key dcc2c7c26d57ac01b9ea04b0b2d94387 for removal 2024.10.22 02:27:01.665892 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5cdcffc4a20710215141d515b899a1ef, offset: 0, size: 156 2024.10.22 02:27:01.666264 [ 1476 ] {} CacheMetadata: Key dcc2c7c26d57ac01b9ea04b0b2d94387 is removed from metadata 2024.10.22 02:27:01.677147 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 5cdcffc4a20710215141d515b899a1ef for removal 2024.10.22 02:27:01.692607 [ 1476 ] {} CacheMetadata: Key 5cdcffc4a20710215141d515b899a1ef is removed from metadata 2024.10.22 02:27:01.692839 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c4fd2772f1351d9d773449bd88658e1e, offset: 0, size: 162 2024.10.22 02:27:01.700042 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c4fd2772f1351d9d773449bd88658e1e for removal 2024.10.22 02:27:01.739323 [ 1476 ] {} CacheMetadata: Key c4fd2772f1351d9d773449bd88658e1e is removed from metadata 2024.10.22 02:27:01.820673 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_930_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_929_929_0, total 167 rows starting from the beginning of the part 2024.10.22 02:27:01.816885 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: f93450928950c37ccb4b2283627a1869, offset: 0, size: 169 2024.10.22 02:27:01.831901 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key f93450928950c37ccb4b2283627a1869 for removal 2024.10.22 02:27:01.843548 [ 1476 ] {} CacheMetadata: Key f93450928950c37ccb4b2283627a1869 is removed from metadata 2024.10.22 02:27:01.843859 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 157e1742070a2957629ccc34c02ddd5c, offset: 0, size: 5128 2024.10.22 02:27:01.851264 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 157e1742070a2957629ccc34c02ddd5c for removal 2024.10.22 02:27:01.956301 [ 1476 ] {} CacheMetadata: Key 157e1742070a2957629ccc34c02ddd5c is removed from metadata 2024.10.22 02:27:01.956511 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 2ee0ba80fccc39ec8f2d855cbe0e7fe6, offset: 0, size: 156 2024.10.22 02:27:01.959828 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 2ee0ba80fccc39ec8f2d855cbe0e7fe6 for removal 2024.10.22 02:27:01.961932 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 14b0adfe8470fd740fc0e87402ef5fb7, offset: 0, size: 156 2024.10.22 02:27:01.964839 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 14b0adfe8470fd740fc0e87402ef5fb7 for removal 2024.10.22 02:27:01.967315 [ 1476 ] {} CacheMetadata: Key 2ee0ba80fccc39ec8f2d855cbe0e7fe6 is removed from metadata 2024.10.22 02:27:01.969808 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c51916353f52437c308de76ba44f9ed4, offset: 0, size: 159 2024.10.22 02:27:01.973098 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c51916353f52437c308de76ba44f9ed4 for removal 2024.10.22 02:27:01.976653 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 3ed1b4207e3e489410fcc55b8354ad04, offset: 0, size: 158 2024.10.22 02:27:01.978553 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 3ed1b4207e3e489410fcc55b8354ad04 for removal 2024.10.22 02:27:01.979770 [ 1476 ] {} CacheMetadata: Key c51916353f52437c308de76ba44f9ed4 is removed from metadata 2024.10.22 02:27:01.995249 [ 1476 ] {} CacheMetadata: Key 3ed1b4207e3e489410fcc55b8354ad04 is removed from metadata 2024.10.22 02:27:02.011601 [ 1476 ] {} CacheMetadata: Key 14b0adfe8470fd740fc0e87402ef5fb7 is removed from metadata 2024.10.22 02:27:02.036046 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_930_39} MergeTreeSequentialSource: Reading 2 marks from part 202410_930_930_0, total 170 rows starting from the beginning of the part 2024.10.22 02:27:02.088959 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: e59275196a154061cfae5daa68b4225c, offset: 0, size: 157 2024.10.22 02:27:02.095253 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key e59275196a154061cfae5daa68b4225c for removal 2024.10.22 02:27:02.102606 [ 1476 ] {} CacheMetadata: Key e59275196a154061cfae5daa68b4225c is removed from metadata 2024.10.22 02:27:02.106460 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 24ab4a8e1f40520e542b481430ff29c7, offset: 0, size: 169 2024.10.22 02:27:02.115965 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 24ab4a8e1f40520e542b481430ff29c7 for removal 2024.10.22 02:27:02.122666 [ 1476 ] {} CacheMetadata: Key 24ab4a8e1f40520e542b481430ff29c7 is removed from metadata 2024.10.22 02:27:02.125444 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0f12d9dc25f7fce1a9350a8efd1d5867, offset: 0, size: 168 2024.10.22 02:27:02.134641 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 0f12d9dc25f7fce1a9350a8efd1d5867 for removal 2024.10.22 02:27:02.145541 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 0172f887d1fd504cc5659d3b27ee0622, offset: 0, size: 156 2024.10.22 02:27:02.150939 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 0172f887d1fd504cc5659d3b27ee0622 for removal 2024.10.22 02:27:02.160089 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 535e107d29370ead6ceb148b13b27892, offset: 0, size: 163 2024.10.22 02:27:02.163394 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 535e107d29370ead6ceb148b13b27892 for removal 2024.10.22 02:27:02.183185 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7c13ba8697d263ded4a87722f3ff426a, offset: 0, size: 158 2024.10.22 02:27:02.198969 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 7c13ba8697d263ded4a87722f3ff426a for removal 2024.10.22 02:27:02.211373 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: a182cef1a08564a35559fa05d47c579c, offset: 0, size: 151 2024.10.22 02:27:02.215150 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key a182cef1a08564a35559fa05d47c579c for removal 2024.10.22 02:27:02.228008 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 73adfbe9101baa2cec741be890a06288, offset: 0, size: 162 2024.10.22 02:27:02.235962 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 73adfbe9101baa2cec741be890a06288 for removal 2024.10.22 02:27:02.242892 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: d90f4c4b7a0cc612f4744137b0173fa7, offset: 0, size: 156 2024.10.22 02:27:02.242696 [ 1476 ] {} CacheMetadata: Key 73adfbe9101baa2cec741be890a06288 is removed from metadata 2024.10.22 02:27:02.247370 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key d90f4c4b7a0cc612f4744137b0173fa7 for removal 2024.10.22 02:27:02.252046 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5bbc42d71a93e8275de40c915a43af38, offset: 0, size: 156 2024.10.22 02:27:02.255082 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 5bbc42d71a93e8275de40c915a43af38 for removal 2024.10.22 02:27:02.258223 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 24ac5947d3ec8621b461df1d741f20e7, offset: 0, size: 169 2024.10.22 02:27:02.263446 [ 1476 ] {} CacheMetadata: Key 5bbc42d71a93e8275de40c915a43af38 is removed from metadata 2024.10.22 02:27:02.271696 [ 1476 ] {} CacheMetadata: Key d90f4c4b7a0cc612f4744137b0173fa7 is removed from metadata 2024.10.22 02:27:02.273233 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 24ac5947d3ec8621b461df1d741f20e7 for removal 2024.10.22 02:27:02.284453 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 518535bbb70c86026abcd1a086ea4834, offset: 0, size: 157 2024.10.22 02:27:02.285104 [ 1476 ] {} CacheMetadata: Key 24ac5947d3ec8621b461df1d741f20e7 is removed from metadata 2024.10.22 02:27:02.294998 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 518535bbb70c86026abcd1a086ea4834 for removal 2024.10.22 02:27:02.303213 [ 1476 ] {} CacheMetadata: Key 518535bbb70c86026abcd1a086ea4834 is removed from metadata 2024.10.22 02:27:02.303482 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 7fb4cf0af14ddeb7bb309f8464e12a0b, offset: 0, size: 162 2024.10.22 02:27:02.313628 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 7fb4cf0af14ddeb7bb309f8464e12a0b for removal 2024.10.22 02:27:02.323083 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 175fc4a68794ccb3b1277f1f13e77b4e, offset: 0, size: 156 2024.10.22 02:27:02.328881 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 175fc4a68794ccb3b1277f1f13e77b4e for removal 2024.10.22 02:27:02.335325 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: ffe0aa594f56aad975c4b692b406eff7, offset: 0, size: 156 2024.10.22 02:27:02.341078 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key ffe0aa594f56aad975c4b692b406eff7 for removal 2024.10.22 02:27:02.365045 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 781 2024.10.22 02:27:02.428383 [ 1476 ] {} CacheMetadata: Key ffe0aa594f56aad975c4b692b406eff7 is removed from metadata 2024.10.22 02:27:02.443355 [ 1476 ] {} CacheMetadata: Key 175fc4a68794ccb3b1277f1f13e77b4e is removed from metadata 2024.10.22 02:27:02.551605 [ 1476 ] {} CacheMetadata: Key 7fb4cf0af14ddeb7bb309f8464e12a0b is removed from metadata 2024.10.22 02:27:02.555911 [ 1476 ] {} CacheMetadata: Key a182cef1a08564a35559fa05d47c579c is removed from metadata 2024.10.22 02:27:02.561839 [ 1476 ] {} CacheMetadata: Key 7c13ba8697d263ded4a87722f3ff426a is removed from metadata 2024.10.22 02:27:02.570878 [ 1476 ] {} CacheMetadata: Key 535e107d29370ead6ceb148b13b27892 is removed from metadata 2024.10.22 02:27:02.597035 [ 1476 ] {} CacheMetadata: Key 0172f887d1fd504cc5659d3b27ee0622 is removed from metadata 2024.10.22 02:27:02.621532 [ 1476 ] {} CacheMetadata: Key 0f12d9dc25f7fce1a9350a8efd1d5867 is removed from metadata 2024.10.22 02:27:02.655557 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:02.667571 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:02.750191 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_763_763_0 (state Temporary) 2024.10.22 02:27:02.835127 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_763_763_0 to 202410_773_773_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:02.919716 [ 1261 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_768_156 to 202410_773_773_0 2024.10.22 02:27:02.946700 [ 1261 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:02.950041 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:02.986976 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_768_156 to 202410_773_773_0 into Compact with storage Full 2024.10.22 02:27:03.190460 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 781 2024.10.22 02:27:03.451739 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:03.465002 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_768_156, total 812 rows starting from the beginning of the part 2024.10.22 02:27:03.546998 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_769_769_0, total 1 rows starting from the beginning of the part 2024.10.22 02:27:03.554316 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(288f1fa55aedb6a2af6fc455fcd2581c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 288f1fa55aedb6a2af6fc455fcd2581c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:27:03.569776 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4878, file segment info: File segment: [0, 33554431], key: 288f1fa55aedb6a2af6fc455fcd2581c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:27:03.573231 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:03.715001 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_770_770_0, total 1 rows starting from the beginning of the part 2024.10.22 02:27:03.791966 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_771_771_0, total 1 rows starting from the beginning of the part 2024.10.22 02:27:03.864569 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_772_772_0, total 1 rows starting from the beginning of the part 2024.10.22 02:27:03.971086 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} MergeTreeSequentialSource: Reading 2 marks from part 202410_773_773_0, total 1 rows starting from the beginning of the part 2024.10.22 02:27:03.977030 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_904_168/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:27:04.236422 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1366473 2024.10.22 02:27:04.555430 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 498 bytes for 38b728c7c1f3666456cc15e29446424c:0) 2024.10.22 02:27:04.571160 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c7c756e4384ce706027380591d286026, offset: 0, size: 192 2024.10.22 02:27:04.579722 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c7c756e4384ce706027380591d286026 for removal 2024.10.22 02:27:04.591693 [ 1476 ] {} CacheMetadata: Key c7c756e4384ce706027380591d286026 is removed from metadata 2024.10.22 02:27:04.746770 [ 1732 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ppv/eemfyvacbugyrpitvvstlsnuwvqwi, size 4878 2024.10.22 02:27:05.019013 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:05.024337 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:05.060700 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_930_39} MergeTask::MergeProjectionsStage: Merge sorted 19663 rows, containing 13 columns (13 merged, 0 gathered) in 4.30523295 sec., 4567.232535001387 rows/sec., 1.99 MiB/sec. 2024.10.22 02:27:05.129321 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 02:27:05.208009 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_902_902_0 (state Temporary) 2024.10.22 02:27:05.246780 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 02:27:05.251882 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_930_39} MergedBlockOutputStream: filled checksums 202410_747_930_39 (state Temporary) 2024.10.22 02:27:05.264240 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 02:27:05.394602 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 14450 2024.10.22 02:27:05.395002 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 02:27:05.508749 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_902_902_0 to 202410_909_909_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:05.510639 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 146 entries to flush up to offset 106374 2024.10.22 02:27:05.530914 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 02:27:05.563544 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1366473 2024.10.22 02:27:05.564378 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 02:27:05.614644 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OpenstatAdID 2024.10.22 02:27:05.713561 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:05.717655 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:05.743618 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zhn/jekflyncjmtnmhhleulsxbnfqgudj (LogSeriesLimiter: on interval from 2024-10-22 02:26:46 to 2024-10-22 02:27:05 accepted series 1 / 12 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:27:05.766159 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_942_942_0 (state Temporary) 2024.10.22 02:27:05.807564 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_930_39} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_930_39 to 202410_747_930_39 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:05.813379 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_930_39} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 5 parts: [202410_747_926_38, 202410_930_930_0] -> 202410_747_930_39 2024.10.22 02:27:05.845095 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_942_942_0 to 202410_955_955_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:05.867302 [ 1229 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_950_315 to 202410_951_951_0 2024.10.22 02:27:05.880249 [ 1229 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:05.889333 [ 1229 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:05.900969 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14450 2024.10.22 02:27:05.947136 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_951_316} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_950_315 to 202410_951_951_0 into Compact with storage Full 2024.10.22 02:27:06.080159 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.28 MiB. 2024.10.22 02:27:06.235067 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_951_316} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:06.272871 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:06.278776 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:06.371582 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1588_1588_0 (state Temporary) 2024.10.22 02:27:06.387696 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_951_316} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_950_315, total 14485 rows starting from the beginning of the part 2024.10.22 02:27:06.648579 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1588_1588_0 to 202410_1601_1601_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:06.655444 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} MergeTask::MergeProjectionsStage: Merge sorted 817 rows, containing 32 columns (32 merged, 0 gathered) in 3.682242599 sec., 221.87565811711474 rows/sec., 16.72 KiB/sec. 2024.10.22 02:27:06.658670 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 10.40 MiB, will set to 1.58 GiB (RSS), difference: -5.64 MiB 2024.10.22 02:27:06.679557 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 4 parts from 202410_1207_1596_123 to 202410_1599_1599_0 2024.10.22 02:27:06.700652 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106374 2024.10.22 02:27:06.710719 [ 1278 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 02:27:06.719570 [ 1278 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:06.839460 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1599_124} MergeTask::PrepareStage: Merging 4 parts: from 202410_1207_1596_123 to 202410_1599_1599_0 into Compact with storage Full 2024.10.22 02:27:06.851635 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_951_316} MergeTreeSequentialSource: Reading 2 marks from part 202410_951_951_0, total 23 rows starting from the beginning of the part 2024.10.22 02:27:06.876595 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1599_124} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:06.900318 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1599_124} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1596_123, total 26834 rows starting from the beginning of the part 2024.10.22 02:27:06.965163 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} MergedBlockOutputStream: filled checksums 202410_1_773_157 (state Temporary) 2024.10.22 02:27:07.148663 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1599_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_1597_1597_0, total 62 rows starting from the beginning of the part 2024.10.22 02:27:07.192930 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1599_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_1598_1598_0, total 76 rows starting from the beginning of the part 2024.10.22 02:27:07.277089 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1599_124} MergeTreeSequentialSource: Reading 2 marks from part 202410_1599_1599_0, total 59 rows starting from the beginning of the part 2024.10.22 02:27:07.629354 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_773_157 to 202410_1_773_157 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:07.634425 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_773_157} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_768_156, 202410_773_773_0] -> 202410_1_773_157 2024.10.22 02:27:07.695347 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:27:08.113835 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_189_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:27:08.128956 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_951_316} MergeTask::MergeProjectionsStage: Merge sorted 14508 rows, containing 12 columns (12 merged, 0 gathered) in 2.227447191 sec., 6513.285728442664 rows/sec., 1.02 MiB/sec. 2024.10.22 02:27:08.185065 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_951_316} MergedBlockOutputStream: filled checksums 202410_1_951_316 (state Temporary) 2024.10.22 02:27:08.455203 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 10.15 MiB, will set to 1.58 GiB (RSS), difference: -9.47 MiB 2024.10.22 02:27:08.463958 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 160 entries to flush up to offset 103306 2024.10.22 02:27:08.692348 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:08.698365 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:08.856600 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_926_926_0 (state Temporary) 2024.10.22 02:27:08.947491 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_926_926_0 to 202410_932_932_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:09.008698 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_951_316} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_951_316 to 202410_1_951_316 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:09.013072 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_951_316} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_950_315, 202410_951_951_0] -> 202410_1_951_316 2024.10.22 02:27:09.034354 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 103306 2024.10.22 02:27:09.059563 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:27:09.952185 [ 1259 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 247 ms. 2024.10.22 02:27:09.979172 [ 1274 ] {} DatabaseCatalog: Execution took 576 ms. 2024.10.22 02:27:10.551555 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1599_124} MergeTask::MergeProjectionsStage: Merge sorted 27031 rows, containing 13 columns (13 merged, 0 gathered) in 3.812073475 sec., 7090.891657065975 rows/sec., 1.37 MiB/sec. 2024.10.22 02:27:10.655424 [ 1218 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:27:10.692991 [ 1218 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:27:10.734066 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 106434 2024.10.22 02:27:10.740414 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1599_124} MergedBlockOutputStream: filled checksums 202410_1207_1599_124 (state Temporary) 2024.10.22 02:27:10.751339 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.36 MiB, will set to 1.59 GiB (RSS), difference: 17.02 MiB 2024.10.22 02:27:10.868892 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:10.872550 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:10.961939 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1589_1589_0 (state Temporary) 2024.10.22 02:27:11.267379 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1599_124} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1599_124 to 202410_1207_1599_124 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:11.274549 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1599_124} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 4 parts: [202410_1207_1596_123, 202410_1599_1599_0] -> 202410_1207_1599_124 2024.10.22 02:27:11.275565 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1589_1589_0 to 202410_1602_1602_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:11.358184 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.38 MiB. 2024.10.22 02:27:11.391045 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:27:11.472119 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:27:11.578760 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106434 2024.10.22 02:27:12.571966 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1368013 2024.10.22 02:27:13.407181 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 14476 2024.10.22 02:27:13.537189 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:13.543067 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:13.704916 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:13.709047 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:13.759768 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_943_943_0 (state Temporary) 2024.10.22 02:27:13.847888 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_903_903_0 (state Temporary) 2024.10.22 02:27:13.925002 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_943_943_0 to 202410_956_956_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:13.962040 [ 1232 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_951_316 to 202410_952_952_0 2024.10.22 02:27:13.962536 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_903_903_0 to 202410_910_910_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:13.965105 [ 1232 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:13.966595 [ 1232 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:13.986273 [ 1150 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_905_905_0 to 202410_910_910_0 2024.10.22 02:27:13.986733 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14476 2024.10.22 02:27:14.005144 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_952_317} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_951_316 to 202410_952_952_0 into Compact with storage Full 2024.10.22 02:27:14.054878 [ 1150 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:14.057014 [ 1150 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:14.079970 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_952_317} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:14.080578 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_905_905_0 to 202410_910_910_0 into Compact with storage Full 2024.10.22 02:27:14.101723 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_952_317} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_951_316, total 14508 rows starting from the beginning of the part 2024.10.22 02:27:14.138369 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:14.148020 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_905_905_0, total 1165 rows starting from the beginning of the part 2024.10.22 02:27:14.154879 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_952_317} MergeTreeSequentialSource: Reading 2 marks from part 202410_952_952_0, total 13 rows starting from the beginning of the part 2024.10.22 02:27:14.169607 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_906_906_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:27:14.179914 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_838_157, 202410_839_844_1] 2024.10.22 02:27:14.184080 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_838_157, 202410_839_844_1] 2024.10.22 02:27:14.194011 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_838_157, 202410_839_844_1] 2024.10.22 02:27:14.215846 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1368013 2024.10.22 02:27:14.215259 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_907_907_0, total 1413 rows starting from the beginning of the part 2024.10.22 02:27:14.255715 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_908_908_0, total 1667 rows starting from the beginning of the part 2024.10.22 02:27:14.288007 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_909_909_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:27:14.329512 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_910_910_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:27:15.563119 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_904_168/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:27:15.497016 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_952_317} MergeTask::MergeProjectionsStage: Merge sorted 14521 rows, containing 12 columns (12 merged, 0 gathered) in 1.507184434 sec., 9634.520946757602 rows/sec., 1.51 MiB/sec. 2024.10.22 02:27:15.701269 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 48 entries to flush up to offset 106482 2024.10.22 02:27:15.833786 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 10.92 MiB, will set to 1.58 GiB (RSS), difference: -6.48 MiB 2024.10.22 02:27:15.856555 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} MergeTask::MergeProjectionsStage: Merge sorted 8865 rows, containing 4 columns (4 merged, 0 gathered) in 1.75982441 sec., 5037.434388127393 rows/sec., 78.71 KiB/sec. 2024.10.22 02:27:15.863415 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_952_317} MergedBlockOutputStream: filled checksums 202410_1_952_317 (state Temporary) 2024.10.22 02:27:15.912187 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} MergedBlockOutputStream: filled checksums 202410_905_910_1 (state Temporary) 2024.10.22 02:27:16.065416 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:16.071469 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:16.102736 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_905_910_1 to 202410_905_910_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:16.119821 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_905_910_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_905_905_0, 202410_910_910_0] -> 202410_905_910_1 2024.10.22 02:27:16.138636 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1590_1590_0 (state Temporary) 2024.10.22 02:27:16.165735 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:27:16.203613 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1590_1590_0 to 202410_1603_1603_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:16.243233 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_952_317} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_952_317 to 202410_1_952_317 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:16.247781 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_952_317} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_951_316, 202410_952_952_0] -> 202410_1_952_317 2024.10.22 02:27:16.272107 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106482 2024.10.22 02:27:16.280276 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:27:16.344180 [ 1398 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_189_1} MergeTask::MergeProjectionsStage: Merge sorted 209 rows, containing 679 columns (679 merged, 0 gathered) in 96.991025539 sec., 2.154838541386092 rows/sec., 3.29 KiB/sec. 2024.10.22 02:27:16.528047 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:16.531732 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:16.557942 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 100 entries to flush up to offset 103406 2024.10.22 02:27:16.948821 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:16.951026 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:17.006682 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_927_927_0 (state Temporary) 2024.10.22 02:27:17.178180 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_927_927_0 to 202410_933_933_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:17.548685 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 103406 2024.10.22 02:27:17.564497 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.05 MiB, will set to 1.58 GiB (RSS), difference: 7.96 MiB 2024.10.22 02:27:17.972787 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_189_1} MergedBlockOutputStream: filled checksums 202410_185_189_1 (state Temporary) 2024.10.22 02:27:18.199199 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 782 2024.10.22 02:27:18.684003 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:18.692006 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:18.810541 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1498_71, 202410_1499_1499_0] 2024.10.22 02:27:18.814386 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1498_71, 202410_1499_1499_0] 2024.10.22 02:27:18.849829 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1498_71, 202410_1499_1499_0] 2024.10.22 02:27:18.960725 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_764_764_0 (state Temporary) 2024.10.22 02:27:19.271941 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_764_764_0 to 202410_774_774_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:19.563088 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 782 2024.10.22 02:27:19.566060 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 10.62 MiB, will set to 1.58 GiB (RSS), difference: -1.09 MiB 2024.10.22 02:27:19.763295 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_189_189_0 (state Temporary) 2024.10.22 02:27:20.291067 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 106520 2024.10.22 02:27:20.415762 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(c269e240fad9eccd4e43b451804666a3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c269e240fad9eccd4e43b451804666a3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 02:27:20.419509 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 996827, file segment info: File segment: [0, 33554431], key: c269e240fad9eccd4e43b451804666a3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 02:27:20.423481 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:20.544051 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:20.548188 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:20.585194 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1591_1591_0 (state Temporary) 2024.10.22 02:27:20.735649 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1591_1591_0 to 202410_1604_1604_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:20.775531 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1599_124 to 202410_1600_1600_0 2024.10.22 02:27:20.790451 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106520 2024.10.22 02:27:20.888859 [ 1200 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.32 MiB using storage policy from min volume index 0 2024.10.22 02:27:20.894593 [ 1200 ] {} DiskLocal: Reserved 1.32 MiB on local disk `default`, having unreserved 296.31 GiB. 2024.10.22 02:27:21.024347 [ 1200 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 275 ms. 2024.10.22 02:27:21.049433 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1600_125} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1599_124 to 202410_1600_1600_0 into Compact with storage Full 2024.10.22 02:27:21.114194 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1600_125} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:21.148452 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1600_125} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1599_124, total 27031 rows starting from the beginning of the part 2024.10.22 02:27:21.221577 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1498 entries to flush up to offset 1369511 2024.10.22 02:27:21.244900 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1600_125} MergeTreeSequentialSource: Reading 2 marks from part 202410_1600_1600_0, total 56 rows starting from the beginning of the part 2024.10.22 02:27:21.342734 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(ebde5680b6a6cf71f5a2af887d26cd48) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ebde5680b6a6cf71f5a2af887d26cd48, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 02:27:21.346364 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1214, file segment info: File segment: [0, 33554431], key: ebde5680b6a6cf71f5a2af887d26cd48, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 02:27:21.359583 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:21.390489 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:27:21.392692 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:27:21.494524 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 14492 2024.10.22 02:27:21.746850 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:21.749129 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:21.803850 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_944_944_0 (state Temporary) 2024.10.22 02:27:21.960337 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_944_944_0 to 202410_957_957_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:21.996247 [ 1199 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_952_317 to 202410_953_953_0 2024.10.22 02:27:22.013563 [ 1199 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:22.019292 [ 1199 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:22.049422 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14492 2024.10.22 02:27:22.066560 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_953_318} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_952_317 to 202410_953_953_0 into Compact with storage Full 2024.10.22 02:27:22.221356 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:22.228688 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:22.382242 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_953_318} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:22.426193 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_953_318} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_952_317, total 14521 rows starting from the beginning of the part 2024.10.22 02:27:22.476370 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(5f24590f6505877b43eda9091118c6c7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5f24590f6505877b43eda9091118c6c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 02:27:22.480171 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 963962, file segment info: File segment: [0, 33554431], key: 5f24590f6505877b43eda9091118c6c7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 02:27:22.487275 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:22.540299 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_953_318} MergeTreeSequentialSource: Reading 2 marks from part 202410_953_953_0, total 10 rows starting from the beginning of the part 2024.10.22 02:27:22.552212 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 10.16 MiB, will set to 1.58 GiB (RSS), difference: -20.96 MiB 2024.10.22 02:27:22.671597 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_904_904_0 (state Temporary) 2024.10.22 02:27:22.845243 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_904_904_0 to 202410_911_911_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:22.865262 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1369511 2024.10.22 02:27:23.428247 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_953_318} MergeTask::MergeProjectionsStage: Merge sorted 14531 rows, containing 12 columns (12 merged, 0 gathered) in 1.366921175 sec., 10630.459360613826 rows/sec., 1.66 MiB/sec. 2024.10.22 02:27:23.574967 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1600_125/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:27:23.652179 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1600_125} MergeTask::MergeProjectionsStage: Merge sorted 27087 rows, containing 13 columns (13 merged, 0 gathered) in 2.608448176 sec., 10384.335118950816 rows/sec., 2.01 MiB/sec. 2024.10.22 02:27:23.693554 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_953_318} MergedBlockOutputStream: filled checksums 202410_1_953_318 (state Temporary) 2024.10.22 02:27:23.704620 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/kpy/vexkocplusvrntzylomimexppqryr. 2024.10.22 02:27:23.709513 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(ae42a4fce965f1b20f32f176fc0e2f2c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ae42a4fce965f1b20f32f176fc0e2f2c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 02:27:23.729015 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1499_72, 202410_1500_1500_0] 2024.10.22 02:27:23.800836 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 940, file segment info: File segment: [0, 33554431], key: ae42a4fce965f1b20f32f176fc0e2f2c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1385, kind: Regular, unbound: 0 2024.10.22 02:27:23.805332 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:23.807259 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1499_72, 202410_1500_1500_0] 2024.10.22 02:27:23.816334 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1499_72, 202410_1500_1500_0] 2024.10.22 02:27:23.942700 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_953_318} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_953_318 to 202410_1_953_318 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:23.946746 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_953_318} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_952_317, 202410_953_953_0] -> 202410_1_953_318 2024.10.22 02:27:23.951444 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1600_125} MergedBlockOutputStream: filled checksums 202410_1207_1600_125 (state Temporary) 2024.10.22 02:27:23.985716 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:27:24.112442 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1600_125} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1600_125 to 202410_1207_1600_125 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:24.128455 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1600_125} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1599_124, 202410_1600_1600_0] -> 202410_1207_1600_125 2024.10.22 02:27:24.193483 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.21 MiB. 2024.10.22 02:27:24.337313 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column URLRegions 2024.10.22 02:27:24.350540 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column URLRegions 2024.10.22 02:27:24.395918 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column URLRegions 2024.10.22 02:27:24.546350 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column URLRegions 2024.10.22 02:27:24.570824 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column URLRegions 2024.10.22 02:27:24.607456 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column URLRegions 2024.10.22 02:27:24.646410 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column URLRegions 2024.10.22 02:27:24.673793 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column URLRegions 2024.10.22 02:27:24.808902 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column URLRegions 2024.10.22 02:27:24.811583 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 106591 2024.10.22 02:27:25.074331 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/qch/axbiezydykehnudkmxltequphczfo (LogSeriesLimiter: on interval from 2024-10-22 02:27:05 to 2024-10-22 02:27:25 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:27:25.075646 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 111 entries to flush up to offset 103517 2024.10.22 02:27:25.317028 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:25.323231 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:25.361573 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_928_928_0 (state Temporary) 2024.10.22 02:27:25.422415 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:25.426800 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:25.590779 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1592_1592_0 (state Temporary) 2024.10.22 02:27:25.596692 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 8.14 MiB, will set to 1.57 GiB (RSS), difference: 55.35 MiB 2024.10.22 02:27:25.648011 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1592_1592_0 to 202410_1605_1605_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:25.656520 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_928_928_0 to 202410_934_934_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:25.696642 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1600_125 to 202410_1601_1601_0 2024.10.22 02:27:25.702591 [ 1245 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:27:25.710558 [ 1227 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.32 MiB using storage policy from min volume index 0 2024.10.22 02:27:25.716926 [ 1227 ] {} DiskLocal: Reserved 1.32 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:25.720718 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 103517 2024.10.22 02:27:25.734095 [ 1245 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:27:25.739425 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1601_126} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1600_125 to 202410_1601_1601_0 into Compact with storage Full 2024.10.22 02:27:25.741616 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106591 2024.10.22 02:27:25.760670 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1601_126} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:25.766895 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1601_126} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1600_125, total 27087 rows starting from the beginning of the part 2024.10.22 02:27:25.889712 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1601_126} MergeTreeSequentialSource: Reading 2 marks from part 202410_1601_1601_0, total 146 rows starting from the beginning of the part 2024.10.22 02:27:26.870635 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_189_189_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:27:27.064599 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 783 2024.10.22 02:27:27.294472 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:27:27.366313 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_904_168/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:27:27.399973 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(3b964bfcfa132e87f27e79f8f32a6978) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3b964bfcfa132e87f27e79f8f32a6978, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:27:27.405402 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 14241, file segment info: File segment: [0, 33554431], key: 3b964bfcfa132e87f27e79f8f32a6978, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:27:27.409753 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:27.659568 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:27.663334 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:27.739287 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_765_765_0 (state Temporary) 2024.10.22 02:27:27.774551 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 2.00 GiB, free memory in arenas 13.26 MiB, will set to 1.57 GiB (RSS), difference: 106.87 MiB 2024.10.22 02:27:27.858853 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1601_126} MergeTask::MergeProjectionsStage: Merge sorted 27233 rows, containing 13 columns (13 merged, 0 gathered) in 2.12942569 sec., 12788.894267543094 rows/sec., 2.47 MiB/sec. 2024.10.22 02:27:27.914545 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1601_126} MergedBlockOutputStream: filled checksums 202410_1207_1601_126 (state Temporary) 2024.10.22 02:27:27.914277 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_765_765_0 to 202410_775_775_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:28.101674 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 783 2024.10.22 02:27:28.240045 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/kpy/vexkocplusvrntzylomimexppqryr. 2024.10.22 02:27:28.288572 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 365 bytes for 795cd4570071a71064503ab96f1ffaeb:0) 2024.10.22 02:27:28.307478 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: f05f42aee76b3db513db0b936da92f43, offset: 0, size: 197 2024.10.22 02:27:28.312614 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key f05f42aee76b3db513db0b936da92f43 for removal 2024.10.22 02:27:28.319797 [ 1476 ] {} CacheMetadata: Key f05f42aee76b3db513db0b936da92f43 is removed from metadata 2024.10.22 02:27:28.541424 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1601_126} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1601_126 to 202410_1207_1601_126 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:28.557056 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1601_126} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1600_125, 202410_1601_1601_0] -> 202410_1207_1601_126 2024.10.22 02:27:28.647016 [ 1216 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 02:27:28.688332 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.24 MiB. 2024.10.22 02:27:29.120846 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_189_189_0 to 202410_192_192_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:29.196403 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(ba291891b2335e368290fed3822e5d6f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ba291891b2335e368290fed3822e5d6f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:27:29.203629 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4887, file segment info: File segment: [0, 33554431], key: ba291891b2335e368290fed3822e5d6f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:27:29.211132 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:29.458560 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 14.19 MiB, will set to 1.56 GiB (RSS), difference: 35.38 MiB 2024.10.22 02:27:29.552972 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 14507 2024.10.22 02:27:29.752553 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 106645 2024.10.22 02:27:29.787840 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:29.792271 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:29.799308 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_189_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_185_189_1 to 202410_185_189_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:29.828577 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_189_1} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 5 parts: [202410_185_185_0, 202410_189_189_0] -> 202410_185_189_1 2024.10.22 02:27:29.864065 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_945_945_0 (state Temporary) 2024.10.22 02:27:29.868483 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1197 entries to flush up to offset 1370708 2024.10.22 02:27:29.954075 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_945_945_0 to 202410_958_958_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:30.033798 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14507 2024.10.22 02:27:30.092354 [ 1248 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_953_318 to 202410_954_954_0 2024.10.22 02:27:30.114417 [ 1248 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:30.117255 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:30.153070 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:30.155950 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:30.156973 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_954_319} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_953_318 to 202410_954_954_0 into Compact with storage Full 2024.10.22 02:27:30.227560 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_954_319} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:30.315342 [ 1267 ] {} DatabaseCatalog: Execution took 202 ms. 2024.10.22 02:27:30.317075 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1593_1593_0 (state Temporary) 2024.10.22 02:27:30.351605 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_954_319} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_953_318, total 14531 rows starting from the beginning of the part 2024.10.22 02:27:30.392738 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_954_319} MergeTreeSequentialSource: Reading 2 marks from part 202410_954_954_0, total 17 rows starting from the beginning of the part 2024.10.22 02:27:30.480736 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 204 bytes for 07a8a78a8342201d3a3acbb626aa1996:0) 2024.10.22 02:27:30.491997 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6a8463ea8057a45a1ec4506c3dfb3db4, offset: 0, size: 157 2024.10.22 02:27:30.499872 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6a8463ea8057a45a1ec4506c3dfb3db4 for removal 2024.10.22 02:27:30.507812 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3b44933443dacb89e1d34d58ba3d16f7, offset: 0, size: 160 2024.10.22 02:27:30.508476 [ 1476 ] {} CacheMetadata: Key 6a8463ea8057a45a1ec4506c3dfb3db4 is removed from metadata 2024.10.22 02:27:30.513102 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3b44933443dacb89e1d34d58ba3d16f7 for removal 2024.10.22 02:27:30.536349 [ 1476 ] {} CacheMetadata: Key 3b44933443dacb89e1d34d58ba3d16f7 is removed from metadata 2024.10.22 02:27:30.591005 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 12.75 MiB. 2024.10.22 02:27:30.652232 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1500_73, 202410_1501_1501_0, 202410_1502_1502_0, 202410_1503_1503_0, 202410_1504_1504_0, 202410_1505_1505_0] 2024.10.22 02:27:30.652765 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1593_1593_0 to 202410_1606_1606_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:30.654082 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1500_73, 202410_1501_1501_0, 202410_1502_1502_0, 202410_1503_1503_0, 202410_1504_1504_0, 202410_1505_1505_0] 2024.10.22 02:27:30.685132 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1500_73, 202410_1501_1501_0, 202410_1502_1502_0, 202410_1503_1503_0, 202410_1504_1504_0, 202410_1505_1505_0] 2024.10.22 02:27:30.739060 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106645 2024.10.22 02:27:30.727887 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1601_126 to 202410_1602_1602_0 2024.10.22 02:27:30.815895 [ 1266 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 02:27:30.821176 [ 1266 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:30.849087 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1602_127} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1601_126 to 202410_1602_1602_0 into Compact with storage Full 2024.10.22 02:27:31.014865 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1602_127} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:31.034071 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1602_127} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1601_126, total 27233 rows starting from the beginning of the part 2024.10.22 02:27:31.055496 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:31.058266 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:31.167857 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1602_127} MergeTreeSequentialSource: Reading 2 marks from part 202410_1602_1602_0, total 60 rows starting from the beginning of the part 2024.10.22 02:27:31.200129 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_905_905_0 (state Temporary) 2024.10.22 02:27:31.273244 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_905_905_0 to 202410_912_912_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:31.321537 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1370708 2024.10.22 02:27:31.393368 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:27:31.398610 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:27:31.431209 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(f269c7a9f9ca68306968898004b88dee) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f269c7a9f9ca68306968898004b88dee, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:27:31.438762 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 498, file segment info: File segment: [0, 33554431], key: f269c7a9f9ca68306968898004b88dee, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:27:31.442519 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:31.469446 [ 1748 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zhn/jekflyncjmtnmhhleulsxbnfqgudj, size 4887 2024.10.22 02:27:31.736878 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_954_319} MergeTask::MergeProjectionsStage: Merge sorted 14548 rows, containing 12 columns (12 merged, 0 gathered) in 1.585582027 sec., 9175.17968308807 rows/sec., 1.44 MiB/sec. 2024.10.22 02:27:31.775926 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_954_319} MergedBlockOutputStream: filled checksums 202410_1_954_319 (state Temporary) 2024.10.22 02:27:31.999092 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_954_319} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_954_319 to 202410_1_954_319 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:32.008667 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_954_319} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_953_318, 202410_954_954_0] -> 202410_1_954_319 2024.10.22 02:27:32.150690 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0e6b1c11504342820fd747ec8bebbc90) : [0, 2172022]: Setting download as failed: File segment: [0, 2172022], key: 0e6b1c11504342820fd747ec8bebbc90, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:32.151991 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0e6b1c11504342820fd747ec8bebbc90): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:27:32.154673 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:27:32.243131 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 02:27:32.250727 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 02:27:32.346369 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0e6b1c11504342820fd747ec8bebbc90, offset: 0, size: 0 2024.10.22 02:27:32.348387 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0e6b1c11504342820fd747ec8bebbc90 for removal 2024.10.22 02:27:32.329351 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7637 2024.10.22 02:27:32.351281 [ 1476 ] {} CacheMetadata: Key 0e6b1c11504342820fd747ec8bebbc90 is removed from metadata 2024.10.22 02:27:32.358062 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 02:27:32.357759 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_872_25, 202410_873_873_0, 202410_874_874_0, 202410_875_875_0, 202410_876_876_0, 202410_877_877_0] 2024.10.22 02:27:32.361896 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_872_25, 202410_873_873_0, 202410_874_874_0, 202410_875_875_0, 202410_876_876_0, 202410_877_877_0] 2024.10.22 02:27:32.363993 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 02:27:32.479312 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 02:27:32.497045 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_872_25, 202410_873_873_0, 202410_874_874_0, 202410_875_875_0, 202410_876_876_0, 202410_877_877_0] 2024.10.22 02:27:32.671499 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1602_127} MergeTask::MergeProjectionsStage: Merge sorted 27293 rows, containing 13 columns (13 merged, 0 gathered) in 1.82986277 sec., 14915.326136724449 rows/sec., 2.88 MiB/sec. 2024.10.22 02:27:32.738667 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 02:27:32.780060 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column OpenstatSourceID 2024.10.22 02:27:32.842861 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xbf/vozsdlylaccenofrufapachvckcxc (LogSeriesLimiter: on interval from 2024-10-22 02:27:25 to 2024-10-22 02:27:32 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:27:33.098221 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1602_127} MergedBlockOutputStream: filled checksums 202410_1207_1602_127 (state Temporary) 2024.10.22 02:27:33.239023 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 137 entries to flush up to offset 103654 2024.10.22 02:27:33.265043 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1602_127} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1602_127 to 202410_1207_1602_127 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:33.268083 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1602_127} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1601_126, 202410_1602_1602_0] -> 202410_1207_1602_127 2024.10.22 02:27:33.306686 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.21 MiB. 2024.10.22 02:27:33.552779 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 12.28 MiB, will set to 1.56 GiB (RSS), difference: 10.82 MiB 2024.10.22 02:27:33.639044 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for a0196d67a95bb7c5705304e44f6d1f72:0) 2024.10.22 02:27:33.641397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 07a8a78a8342201d3a3acbb626aa1996, offset: 0, size: 204 2024.10.22 02:27:33.649741 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 07a8a78a8342201d3a3acbb626aa1996 for removal 2024.10.22 02:27:33.654623 [ 1476 ] {} CacheMetadata: Key 07a8a78a8342201d3a3acbb626aa1996 is removed from metadata 2024.10.22 02:27:33.744118 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:33.848571 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:33.900171 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_929_929_0 (state Temporary) 2024.10.22 02:27:33.969161 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_929_929_0 to 202410_935_935_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:34.039189 [ 1152 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 3 parts from 202410_747_930_39 to 202410_932_932_0 2024.10.22 02:27:34.041531 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 103654 2024.10.22 02:27:34.044612 [ 1152 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.17 MiB using storage policy from min volume index 0 2024.10.22 02:27:34.047097 [ 1152 ] {} DiskLocal: Reserved 1.17 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:34.090568 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_932_40} MergeTask::PrepareStage: Merging 3 parts: from 202410_747_930_39 to 202410_932_932_0 into Compact with storage Full 2024.10.22 02:27:34.189254 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_932_40} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:34.225685 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_932_40} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_930_39, total 19663 rows starting from the beginning of the part 2024.10.22 02:27:34.455501 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_932_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_931_931_0, total 128 rows starting from the beginning of the part 2024.10.22 02:27:34.747554 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 106741 2024.10.22 02:27:34.657806 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_932_40} MergeTreeSequentialSource: Reading 2 marks from part 202410_932_932_0, total 160 rows starting from the beginning of the part 2024.10.22 02:27:34.865671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ec9668d02c429091bbbb1ab69313faa8) : [0, 40587]: Setting download as failed: File segment: [0, 40587], key: ec9668d02c429091bbbb1ab69313faa8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:34.867636 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ec9668d02c429091bbbb1ab69313faa8): No space left in cache to reserve 40588 bytes, will continue without cache download 2024.10.22 02:27:34.915226 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ec9668d02c429091bbbb1ab69313faa8, offset: 0, size: 0 2024.10.22 02:27:34.920184 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ec9668d02c429091bbbb1ab69313faa8 for removal 2024.10.22 02:27:34.928539 [ 1476 ] {} CacheMetadata: Key ec9668d02c429091bbbb1ab69313faa8 is removed from metadata 2024.10.22 02:27:35.609693 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 784 2024.10.22 02:27:35.614466 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:35.621222 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:35.834223 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:35.839759 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:35.889820 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_766_766_0 (state Temporary) 2024.10.22 02:27:35.951865 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1594_1594_0 (state Temporary) 2024.10.22 02:27:36.047552 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 13.72 MiB, will set to 1.56 GiB (RSS), difference: -25.38 MiB 2024.10.22 02:27:36.076487 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_766_766_0 to 202410_776_776_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:36.093322 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1594_1594_0 to 202410_1607_1607_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:36.132792 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1602_127 to 202410_1603_1603_0 2024.10.22 02:27:36.142257 [ 1171 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 02:27:36.145481 [ 1171 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:36.205361 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1603_128} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1602_127 to 202410_1603_1603_0 into Compact with storage Full 2024.10.22 02:27:36.237599 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106741 2024.10.22 02:27:36.259056 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 784 2024.10.22 02:27:36.267585 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1603_128} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:36.301087 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a6f9c141071dff21d11c0b07d60173ab) : [0, 2444]: Setting download as failed: File segment: [0, 2444], key: a6f9c141071dff21d11c0b07d60173ab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:36.303076 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a6f9c141071dff21d11c0b07d60173ab): No space left in cache to reserve 2445 bytes, will continue without cache download 2024.10.22 02:27:36.398195 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1603_128} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1602_127, total 27293 rows starting from the beginning of the part 2024.10.22 02:27:36.434744 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_932_40} MergeTask::MergeProjectionsStage: Merge sorted 19951 rows, containing 13 columns (13 merged, 0 gathered) in 2.369971878 sec., 8418.243349299355 rows/sec., 3.66 MiB/sec. 2024.10.22 02:27:36.526263 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a6f9c141071dff21d11c0b07d60173ab, offset: 0, size: 0 2024.10.22 02:27:36.530790 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a6f9c141071dff21d11c0b07d60173ab for removal 2024.10.22 02:27:36.544550 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1603_128} MergeTreeSequentialSource: Reading 2 marks from part 202410_1603_1603_0, total 48 rows starting from the beginning of the part 2024.10.22 02:27:36.561566 [ 1476 ] {} CacheMetadata: Key a6f9c141071dff21d11c0b07d60173ab is removed from metadata 2024.10.22 02:27:36.687831 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_932_40} MergedBlockOutputStream: filled checksums 202410_747_932_40 (state Temporary) 2024.10.22 02:27:37.229551 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_932_40} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_932_40 to 202410_747_932_40 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:37.236978 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_932_40} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 3 parts: [202410_747_930_39, 202410_932_932_0] -> 202410_747_932_40 2024.10.22 02:27:37.271217 [ 1390 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_932_40}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 02:27:37.311339 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.10 MiB. 2024.10.22 02:27:37.472446 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 15.25 MiB, will set to 1.57 GiB (RSS), difference: 16.61 MiB 2024.10.22 02:27:37.539505 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 14529 2024.10.22 02:27:37.764862 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:37.767190 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:37.784079 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 163 bytes for e9085e933357ee91e76c77afae87405f:0) 2024.10.22 02:27:37.791629 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dfc09baa7d299916c6a3801a47ea4f7c, offset: 0, size: 162 2024.10.22 02:27:37.800198 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dfc09baa7d299916c6a3801a47ea4f7c for removal 2024.10.22 02:27:37.804328 [ 1476 ] {} CacheMetadata: Key dfc09baa7d299916c6a3801a47ea4f7c is removed from metadata 2024.10.22 02:27:37.805588 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a0196d67a95bb7c5705304e44f6d1f72, offset: 0, size: 152 2024.10.22 02:27:37.810503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a0196d67a95bb7c5705304e44f6d1f72 for removal 2024.10.22 02:27:37.814148 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_946_946_0 (state Temporary) 2024.10.22 02:27:37.850094 [ 1476 ] {} CacheMetadata: Key a0196d67a95bb7c5705304e44f6d1f72 is removed from metadata 2024.10.22 02:27:37.969249 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_946_946_0 to 202410_959_959_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:38.007958 [ 1181 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_954_319 to 202410_955_955_0 2024.10.22 02:27:38.020293 [ 1181 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:38.034456 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:38.060534 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_955_320} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_954_319 to 202410_955_955_0 into Compact with storage Full 2024.10.22 02:27:38.138958 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14529 2024.10.22 02:27:38.234560 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_955_320} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:38.246029 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_955_320} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_954_319, total 14548 rows starting from the beginning of the part 2024.10.22 02:27:38.287901 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_955_320} MergeTreeSequentialSource: Reading 2 marks from part 202410_955_955_0, total 8 rows starting from the beginning of the part 2024.10.22 02:27:38.371745 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1245 entries to flush up to offset 1371953 2024.10.22 02:27:38.404943 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1603_128} MergeTask::MergeProjectionsStage: Merge sorted 27341 rows, containing 13 columns (13 merged, 0 gathered) in 2.212792692 sec., 12355.879562892194 rows/sec., 2.38 MiB/sec. 2024.10.22 02:27:38.472920 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(861e266a911c30a946fb7cc8cd694e06) : [0, 1041557]: Setting download as failed: File segment: [0, 1041557], key: 861e266a911c30a946fb7cc8cd694e06, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:38.474746 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(861e266a911c30a946fb7cc8cd694e06): No space left in cache to reserve 1041558 bytes, will continue without cache download 2024.10.22 02:27:38.611133 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1603_128} MergedBlockOutputStream: filled checksums 202410_1207_1603_128 (state Temporary) 2024.10.22 02:27:38.809058 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 861e266a911c30a946fb7cc8cd694e06, offset: 0, size: 0 2024.10.22 02:27:38.810468 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 861e266a911c30a946fb7cc8cd694e06 for removal 2024.10.22 02:27:38.812893 [ 1476 ] {} CacheMetadata: Key 861e266a911c30a946fb7cc8cd694e06 is removed from metadata 2024.10.22 02:27:38.975552 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1603_128} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1603_128 to 202410_1207_1603_128 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:38.981272 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1603_128} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1602_127, 202410_1603_1603_0] -> 202410_1207_1603_128 2024.10.22 02:27:39.023415 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.20 MiB. 2024.10.22 02:27:39.256001 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_955_320/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:27:39.287548 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_723_147, 202410_724_724_0, 202410_725_725_0, 202410_726_726_0, 202410_727_727_0, 202410_728_728_0] 2024.10.22 02:27:39.291184 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_723_147, 202410_724_724_0, 202410_725_725_0, 202410_726_726_0, 202410_727_727_0, 202410_728_728_0] 2024.10.22 02:27:39.331698 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_723_147, 202410_724_724_0, 202410_725_725_0, 202410_726_726_0, 202410_727_727_0, 202410_728_728_0] 2024.10.22 02:27:39.442661 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:39.442853 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_895_260, 202410_896_896_0] 2024.10.22 02:27:39.446178 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_895_260, 202410_896_896_0] 2024.10.22 02:27:39.455132 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:39.455321 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_895_260, 202410_896_896_0] 2024.10.22 02:27:39.466930 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 12.58 MiB, will set to 1.56 GiB (RSS), difference: 7.38 MiB 2024.10.22 02:27:39.461232 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_955_320} MergeTask::MergeProjectionsStage: Merge sorted 14556 rows, containing 12 columns (12 merged, 0 gathered) in 1.404911071 sec., 10360.798131969452 rows/sec., 1.62 MiB/sec. 2024.10.22 02:27:39.548861 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_955_320} MergedBlockOutputStream: filled checksums 202410_1_955_320 (state Temporary) 2024.10.22 02:27:39.729067 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 162 bytes for 61093cfd4cd8d99e74af4af76ccd2ed3:0) 2024.10.22 02:27:39.738686 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e9085e933357ee91e76c77afae87405f, offset: 0, size: 163 2024.10.22 02:27:39.742817 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e9085e933357ee91e76c77afae87405f for removal 2024.10.22 02:27:39.746682 [ 1476 ] {} CacheMetadata: Key e9085e933357ee91e76c77afae87405f is removed from metadata 2024.10.22 02:27:39.753173 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_955_320} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_955_320 to 202410_1_955_320 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:39.758433 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_955_320} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_954_319, 202410_955_955_0] -> 202410_1_955_320 2024.10.22 02:27:39.781498 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:27:39.872601 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 7678 2024.10.22 02:27:39.934673 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_906_906_0 (state Temporary) 2024.10.22 02:27:40.048956 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_904_168/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:27:40.110889 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_851_851_0, 202410_852_852_0, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0, 202410_856_856_0] 2024.10.22 02:27:40.115361 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_906_906_0 to 202410_913_913_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:40.116616 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_851_851_0, 202410_852_852_0, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0, 202410_856_856_0] 2024.10.22 02:27:40.233225 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_851_851_0, 202410_852_852_0, 202410_853_853_0, 202410_854_854_0, 202410_855_855_0, 202410_856_856_0] 2024.10.22 02:27:40.252970 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 95 entries to flush up to offset 106836 2024.10.22 02:27:40.392434 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1371953 2024.10.22 02:27:40.407848 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:40.415779 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:40.507610 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1595_1595_0 (state Temporary) 2024.10.22 02:27:40.650348 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ff15f24b726b2e88a91bd741fbc56874) : [0, 41657]: Setting download as failed: File segment: [0, 41657], key: ff15f24b726b2e88a91bd741fbc56874, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:40.652161 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ff15f24b726b2e88a91bd741fbc56874): No space left in cache to reserve 41658 bytes, will continue without cache download 2024.10.22 02:27:40.661218 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ff15f24b726b2e88a91bd741fbc56874, offset: 0, size: 0 2024.10.22 02:27:40.662807 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ff15f24b726b2e88a91bd741fbc56874 for removal 2024.10.22 02:27:40.665578 [ 1476 ] {} CacheMetadata: Key ff15f24b726b2e88a91bd741fbc56874 is removed from metadata 2024.10.22 02:27:40.668383 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1595_1595_0 to 202410_1608_1608_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:40.709606 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1603_128 to 202410_1604_1604_0 2024.10.22 02:27:40.741705 [ 1220 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:27:40.746931 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106836 2024.10.22 02:27:40.757482 [ 1220 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:27:40.922473 [ 1201 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 02:27:40.947117 [ 1201 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:40.951906 [ 1201 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 267 ms. 2024.10.22 02:27:40.958521 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1604_129} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1603_128 to 202410_1604_1604_0 into Compact with storage Full 2024.10.22 02:27:40.871166 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_904_168} MergeTask::MergeProjectionsStage: Merge sorted 1372738 rows, containing 4 columns (4 merged, 0 gathered) in 43.708093316 sec., 31406.952256539837 rows/sec., 470.95 KiB/sec. 2024.10.22 02:27:41.003991 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1604_129} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:41.053680 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1604_129} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1603_128, total 27341 rows starting from the beginning of the part 2024.10.22 02:27:41.109541 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_904_168} MergedBlockOutputStream: filled checksums 202410_1_904_168 (state Temporary) 2024.10.22 02:27:41.262821 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1604_129} MergeTreeSequentialSource: Reading 2 marks from part 202410_1604_1604_0, total 38 rows starting from the beginning of the part 2024.10.22 02:27:41.393460 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:27:41.405594 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:27:41.533766 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 12.69 MiB, will set to 1.55 GiB (RSS), difference: 7.88 MiB 2024.10.22 02:27:41.566375 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 135 entries to flush up to offset 103789 2024.10.22 02:27:41.856788 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e5a1737b950a87b3784a81911ae51a80) : [0, 30922]: Setting download as failed: File segment: [0, 30922], key: e5a1737b950a87b3784a81911ae51a80, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:41.858663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e5a1737b950a87b3784a81911ae51a80): No space left in cache to reserve 30923 bytes, will continue without cache download 2024.10.22 02:27:41.862633 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:41.866995 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:41.888705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e5a1737b950a87b3784a81911ae51a80, offset: 0, size: 0 2024.10.22 02:27:41.895501 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e5a1737b950a87b3784a81911ae51a80 for removal 2024.10.22 02:27:41.903251 [ 1476 ] {} CacheMetadata: Key e5a1737b950a87b3784a81911ae51a80 is removed from metadata 2024.10.22 02:27:41.947551 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_930_930_0 (state Temporary) 2024.10.22 02:27:42.095672 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_930_930_0 to 202410_936_936_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:42.355136 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 103789 2024.10.22 02:27:42.606690 [ 1264 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 02:27:43.040195 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 151 bytes for 28e222b8360d5bb63d0186549d8d0100:0) 2024.10.22 02:27:43.042442 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 61093cfd4cd8d99e74af4af76ccd2ed3, offset: 0, size: 162 2024.10.22 02:27:43.045970 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 61093cfd4cd8d99e74af4af76ccd2ed3 for removal 2024.10.22 02:27:43.048652 [ 1476 ] {} CacheMetadata: Key 61093cfd4cd8d99e74af4af76ccd2ed3 is removed from metadata 2024.10.22 02:27:43.126763 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1604_129} MergeTask::MergeProjectionsStage: Merge sorted 27379 rows, containing 13 columns (13 merged, 0 gathered) in 2.172551436 sec., 12602.233275732671 rows/sec., 2.43 MiB/sec. 2024.10.22 02:27:43.213378 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1604_129} MergedBlockOutputStream: filled checksums 202410_1207_1604_129 (state Temporary) 2024.10.22 02:27:43.307989 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1604_129} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1604_129 to 202410_1207_1604_129 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:43.312035 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1604_129} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1603_128, 202410_1604_1604_0] -> 202410_1207_1604_129 2024.10.22 02:27:43.351835 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.20 MiB. 2024.10.22 02:27:43.570849 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 11.20 MiB, will set to 1.56 GiB (RSS), difference: 5.55 MiB 2024.10.22 02:27:43.760843 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 785 2024.10.22 02:27:44.043198 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:44.046564 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:44.176613 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(31696c2e49bc6f86d30970e43f54b9fd) : [0, 248557]: Setting download as failed: File segment: [0, 248557], key: 31696c2e49bc6f86d30970e43f54b9fd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:44.182438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(31696c2e49bc6f86d30970e43f54b9fd): No space left in cache to reserve 248558 bytes, will continue without cache download 2024.10.22 02:27:44.214236 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 31696c2e49bc6f86d30970e43f54b9fd, offset: 0, size: 0 2024.10.22 02:27:44.217085 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 31696c2e49bc6f86d30970e43f54b9fd for removal 2024.10.22 02:27:44.224613 [ 1476 ] {} CacheMetadata: Key 31696c2e49bc6f86d30970e43f54b9fd is removed from metadata 2024.10.22 02:27:44.324130 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_767_767_0 (state Temporary) 2024.10.22 02:27:44.406820 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_904_168} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_904_168 to 202410_1_904_168 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:44.455072 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_904_168} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_898_167, 202410_899_904_1] -> 202410_1_904_168 2024.10.22 02:27:44.484517 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.57 MiB. 2024.10.22 02:27:44.760049 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 106897 2024.10.22 02:27:44.805768 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_767_767_0 to 202410_777_777_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:44.995738 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:45.001789 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:45.057408 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1596_1596_0 (state Temporary) 2024.10.22 02:27:45.103995 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 785 2024.10.22 02:27:45.212287 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1596_1596_0 to 202410_1609_1609_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:45.249781 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1604_129 to 202410_1605_1605_0 2024.10.22 02:27:45.262376 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106897 2024.10.22 02:27:45.264257 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 02:27:45.267235 [ 1243 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:45.276057 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1605_130} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1604_129 to 202410_1605_1605_0 into Compact with storage Full 2024.10.22 02:27:45.333481 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1605_130} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:45.346201 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1605_130} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1604_129, total 27379 rows starting from the beginning of the part 2024.10.22 02:27:45.367130 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1605_130} MergeTreeSequentialSource: Reading 2 marks from part 202410_1605_1605_0, total 71 rows starting from the beginning of the part 2024.10.22 02:27:45.448899 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 150 bytes for 8cde6570ff6f31c4c2661d0a89794500:0) 2024.10.22 02:27:45.451389 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 17481c2a6533e1e5c7161a9c4b3f78ef, offset: 0, size: 145 2024.10.22 02:27:45.456850 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 17481c2a6533e1e5c7161a9c4b3f78ef for removal 2024.10.22 02:27:45.461167 [ 1476 ] {} CacheMetadata: Key 17481c2a6533e1e5c7161a9c4b3f78ef is removed from metadata 2024.10.22 02:27:45.511501 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 7.26 MiB, will set to 1.55 GiB (RSS), difference: -1.07 MiB 2024.10.22 02:27:45.642737 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 14547 2024.10.22 02:27:46.199048 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:46.205049 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:46.252884 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_947_947_0 (state Temporary) 2024.10.22 02:27:46.356924 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_947_947_0 to 202410_960_960_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:46.378740 [ 1164 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_955_320 to 202410_956_956_0 2024.10.22 02:27:46.396897 [ 1164 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:46.399728 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:46.400438 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14547 2024.10.22 02:27:46.517354 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_956_321} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_955_320 to 202410_956_956_0 into Compact with storage Full 2024.10.22 02:27:46.554310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(aa2c7b17e469aabbe5f6b366df0f83bc) : [0, 161301]: Setting download as failed: File segment: [0, 161301], key: aa2c7b17e469aabbe5f6b366df0f83bc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:46.555525 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(aa2c7b17e469aabbe5f6b366df0f83bc): No space left in cache to reserve 161302 bytes, will continue without cache download 2024.10.22 02:27:46.559749 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_956_321} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:46.564683 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_956_321} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_955_320, total 14556 rows starting from the beginning of the part 2024.10.22 02:27:46.610773 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_956_321} MergeTreeSequentialSource: Reading 2 marks from part 202410_956_956_0, total 26 rows starting from the beginning of the part 2024.10.22 02:27:46.700587 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: aa2c7b17e469aabbe5f6b366df0f83bc, offset: 0, size: 0 2024.10.22 02:27:46.702565 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key aa2c7b17e469aabbe5f6b366df0f83bc for removal 2024.10.22 02:27:46.808556 [ 1476 ] {} CacheMetadata: Key aa2c7b17e469aabbe5f6b366df0f83bc is removed from metadata 2024.10.22 02:27:47.166516 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_896_261, 202410_897_897_0] 2024.10.22 02:27:47.168365 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_896_261, 202410_897_897_0] 2024.10.22 02:27:47.287996 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_896_261, 202410_897_897_0] 2024.10.22 02:27:47.360803 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1605_130} MergeTask::MergeProjectionsStage: Merge sorted 27450 rows, containing 13 columns (13 merged, 0 gathered) in 2.088694064 sec., 13142.183181883167 rows/sec., 2.53 MiB/sec. 2024.10.22 02:27:47.500994 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1835 entries to flush up to offset 1373788 2024.10.22 02:27:47.550421 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1605_130} MergedBlockOutputStream: filled checksums 202410_1207_1605_130 (state Temporary) 2024.10.22 02:27:47.688129 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_956_321} MergeTask::MergeProjectionsStage: Merge sorted 14582 rows, containing 12 columns (12 merged, 0 gathered) in 1.185053015 sec., 12304.934729017164 rows/sec., 1.92 MiB/sec. 2024.10.22 02:27:47.805062 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1605_130} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1605_130 to 202410_1207_1605_130 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:47.816560 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1605_130} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1604_129, 202410_1605_1605_0] -> 202410_1207_1605_130 2024.10.22 02:27:47.855245 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_956_321} MergedBlockOutputStream: filled checksums 202410_1_956_321 (state Temporary) 2024.10.22 02:27:47.871244 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.22 MiB. 2024.10.22 02:27:48.092934 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 176 bytes for 12eb932dd412e1efcc1636c881233d14:0) 2024.10.22 02:27:48.096446 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8cde6570ff6f31c4c2661d0a89794500, offset: 0, size: 150 2024.10.22 02:27:48.099291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8cde6570ff6f31c4c2661d0a89794500 for removal 2024.10.22 02:27:48.103863 [ 1476 ] {} CacheMetadata: Key 8cde6570ff6f31c4c2661d0a89794500 is removed from metadata 2024.10.22 02:27:48.105800 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 28e222b8360d5bb63d0186549d8d0100, offset: 0, size: 151 2024.10.22 02:27:48.113177 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 28e222b8360d5bb63d0186549d8d0100 for removal 2024.10.22 02:27:48.118227 [ 1476 ] {} CacheMetadata: Key 28e222b8360d5bb63d0186549d8d0100 is removed from metadata 2024.10.22 02:27:48.339085 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_956_321} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_956_321 to 202410_1_956_321 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:48.343422 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_956_321} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_955_320, 202410_956_956_0] -> 202410_1_956_321 2024.10.22 02:27:48.365190 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:27:48.389280 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:48.395832 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:48.596008 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 5.21 MiB, will set to 1.55 GiB (RSS), difference: -5.45 MiB 2024.10.22 02:27:48.663552 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_907_907_0 (state Temporary) 2024.10.22 02:27:48.759642 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_907_907_0 to 202410_914_914_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:48.839829 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1373788 2024.10.22 02:27:48.849231 [ 1263 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_904_168 to 202410_905_910_1 2024.10.22 02:27:48.859388 [ 1263 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.08 MiB using storage policy from min volume index 0 2024.10.22 02:27:48.862963 [ 1263 ] {} DiskLocal: Reserved 1.08 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:48.872098 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_910_169} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_904_168 to 202410_905_910_1 into Wide with storage Full 2024.10.22 02:27:48.907423 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_910_169} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:48.919700 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_910_169} MergeTreeSequentialSource: Reading 169 marks from part 202410_1_904_168, total 1372738 rows starting from the beginning of the part 2024.10.22 02:27:48.971098 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_910_169} MergeTreeSequentialSource: Reading 3 marks from part 202410_905_910_1, total 8865 rows starting from the beginning of the part 2024.10.22 02:27:49.245361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ef92d9b753fe194445b241bf57554742) : [0, 358461]: Setting download as failed: File segment: [0, 358461], key: ef92d9b753fe194445b241bf57554742, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:49.247117 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ef92d9b753fe194445b241bf57554742): No space left in cache to reserve 358462 bytes, will continue without cache download 2024.10.22 02:27:49.275407 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 106971 2024.10.22 02:27:49.272996 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ef92d9b753fe194445b241bf57554742, offset: 0, size: 0 2024.10.22 02:27:49.305637 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ef92d9b753fe194445b241bf57554742 for removal 2024.10.22 02:27:49.311804 [ 1476 ] {} CacheMetadata: Key ef92d9b753fe194445b241bf57554742 is removed from metadata 2024.10.22 02:27:49.656006 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:49.658299 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:49.713977 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1597_1597_0 (state Temporary) 2024.10.22 02:27:49.961798 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1597_1597_0 to 202410_1610_1610_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:50.093458 [ 1273 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1605_130 to 202410_1606_1606_0 2024.10.22 02:27:50.103675 [ 1273 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.34 MiB using storage policy from min volume index 0 2024.10.22 02:27:50.127724 [ 1273 ] {} DiskLocal: Reserved 1.34 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:50.135531 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 106971 2024.10.22 02:27:50.161466 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1606_131} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1605_130 to 202410_1606_1606_0 into Compact with storage Full 2024.10.22 02:27:50.165731 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 130 entries to flush up to offset 103919 2024.10.22 02:27:50.343921 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1606_131} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:50.373782 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1606_131} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1605_130, total 27450 rows starting from the beginning of the part 2024.10.22 02:27:50.447069 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for f62fbe5089155e1e6c43ee5fe9f77ae8:0) 2024.10.22 02:27:50.450771 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 12eb932dd412e1efcc1636c881233d14, offset: 0, size: 176 2024.10.22 02:27:50.453368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 12eb932dd412e1efcc1636c881233d14 for removal 2024.10.22 02:27:50.456674 [ 1476 ] {} CacheMetadata: Key 12eb932dd412e1efcc1636c881233d14 is removed from metadata 2024.10.22 02:27:50.648785 [ 1244 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 302 ms. 2024.10.22 02:27:50.676066 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:50.679683 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:50.694554 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 5.80 MiB, will set to 1.55 GiB (RSS), difference: -9.50 MiB 2024.10.22 02:27:50.759142 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_931_931_0 (state Temporary) 2024.10.22 02:27:50.815821 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_931_931_0 to 202410_937_937_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:50.849610 [ 1217 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_932_40 to 202410_933_933_0 2024.10.22 02:27:50.855395 [ 1217 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.17 MiB using storage policy from min volume index 0 2024.10.22 02:27:50.857091 [ 1217 ] {} DiskLocal: Reserved 1.17 MiB on local disk `default`, having unreserved 296.30 GiB. 2024.10.22 02:27:50.861809 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1606_131} MergeTreeSequentialSource: Reading 2 marks from part 202410_1606_1606_0, total 54 rows starting from the beginning of the part 2024.10.22 02:27:50.888886 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_933_41} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_932_40 to 202410_933_933_0 into Compact with storage Full 2024.10.22 02:27:50.949636 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_933_41} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:50.956372 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 103919 2024.10.22 02:27:50.956883 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_933_41} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_932_40, total 19951 rows starting from the beginning of the part 2024.10.22 02:27:50.959286 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_910_169/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:27:51.023266 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_933_41} MergeTreeSequentialSource: Reading 2 marks from part 202410_933_933_0, total 100 rows starting from the beginning of the part 2024.10.22 02:27:51.395111 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:27:51.406154 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:27:51.654958 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8994c9ec6dacead63763cd3771ee24d6) : [0, 65155]: Setting download as failed: File segment: [0, 65155], key: 8994c9ec6dacead63763cd3771ee24d6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:51.656490 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8994c9ec6dacead63763cd3771ee24d6): No space left in cache to reserve 65156 bytes, will continue without cache download 2024.10.22 02:27:51.668291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8994c9ec6dacead63763cd3771ee24d6, offset: 0, size: 0 2024.10.22 02:27:51.678546 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8994c9ec6dacead63763cd3771ee24d6 for removal 2024.10.22 02:27:51.683066 [ 1476 ] {} CacheMetadata: Key 8994c9ec6dacead63763cd3771ee24d6 is removed from metadata 2024.10.22 02:27:52.553433 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_933_41} MergeTask::MergeProjectionsStage: Merge sorted 20051 rows, containing 13 columns (13 merged, 0 gathered) in 1.678813457 sec., 11943.554488674796 rows/sec., 5.19 MiB/sec. 2024.10.22 02:27:52.611601 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_933_41} MergedBlockOutputStream: filled checksums 202410_747_933_41 (state Temporary) 2024.10.22 02:27:52.635075 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1606_131} MergeTask::MergeProjectionsStage: Merge sorted 27504 rows, containing 13 columns (13 merged, 0 gathered) in 2.48062273 sec., 11087.538490788562 rows/sec., 2.14 MiB/sec. 2024.10.22 02:27:52.706954 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 786 2024.10.22 02:27:52.715935 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(53d79d5cc9a7a1030cd96db2c647be66) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 53d79d5cc9a7a1030cd96db2c647be66, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:27:52.721634 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 647657, file segment info: File segment: [0, 33554431], key: 53d79d5cc9a7a1030cd96db2c647be66, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:27:52.726617 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:52.757828 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_933_41} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_933_41 to 202410_747_933_41 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:52.778197 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_933_41} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_932_40, 202410_933_933_0] -> 202410_747_933_41 2024.10.22 02:27:52.916593 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1606_131} MergedBlockOutputStream: filled checksums 202410_1207_1606_131 (state Temporary) 2024.10.22 02:27:53.033468 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.97 MiB. 2024.10.22 02:27:53.174593 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:53.177553 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:27:53.241059 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1606_131} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1606_131 to 202410_1207_1606_131 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:53.244555 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1606_131} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1605_130, 202410_1606_1606_0] -> 202410_1207_1606_131 2024.10.22 02:27:53.270769 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.22 MiB. 2024.10.22 02:27:53.373320 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 10.71 MiB, will set to 1.55 GiB (RSS), difference: 8.40 MiB 2024.10.22 02:27:53.562399 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_768_768_0 (state Temporary) 2024.10.22 02:27:53.827893 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_768_768_0 to 202410_778_778_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:53.908623 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 14564 2024.10.22 02:27:53.951132 [ 1194 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_773_157 to 202410_778_778_0 2024.10.22 02:27:53.962556 [ 1194 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:53.967507 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:27:54.055766 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 786 2024.10.22 02:27:54.110027 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_773_157 to 202410_778_778_0 into Compact with storage Full 2024.10.22 02:27:54.119183 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(646e0fdc4071ca3634f2095da83546a9) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 646e0fdc4071ca3634f2095da83546a9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:27:54.123750 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:54.148159 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 107041 2024.10.22 02:27:54.224087 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1273, file segment info: File segment: [0, 33554431], key: 646e0fdc4071ca3634f2095da83546a9, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:27:54.228168 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:54.230475 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:27:54.322786 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_948_948_0 (state Temporary) 2024.10.22 02:27:54.490475 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_948_948_0 to 202410_961_961_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:54.533202 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:54.560900 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_773_157, total 817 rows starting from the beginning of the part 2024.10.22 02:27:54.595977 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1505_74, 202410_1506_1506_0, 202410_1507_1507_0, 202410_1508_1508_0, 202410_1509_1509_0, 202410_1510_1510_0] 2024.10.22 02:27:54.607453 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1505_74, 202410_1506_1506_0, 202410_1507_1507_0, 202410_1508_1508_0, 202410_1509_1509_0, 202410_1510_1510_0] 2024.10.22 02:27:54.636766 [ 1173 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_956_321 to 202410_957_957_0 2024.10.22 02:27:54.643509 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_774_774_0, total 1 rows starting from the beginning of the part 2024.10.22 02:27:54.644925 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14564 2024.10.22 02:27:54.729049 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:54.736443 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:27:54.751034 [ 1173 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:54.753073 [ 1173 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:27:54.768609 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_957_322} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_956_321 to 202410_957_957_0 into Compact with storage Full 2024.10.22 02:27:54.787308 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1598_1598_0 (state Temporary) 2024.10.22 02:27:54.831075 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_957_322} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:54.843235 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_957_322} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_956_321, total 14582 rows starting from the beginning of the part 2024.10.22 02:27:54.846587 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1505_74, 202410_1506_1506_0, 202410_1507_1507_0, 202410_1508_1508_0, 202410_1509_1509_0, 202410_1510_1510_0] 2024.10.22 02:27:54.858696 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_775_775_0, total 1 rows starting from the beginning of the part 2024.10.22 02:27:54.875853 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_957_322} MergeTreeSequentialSource: Reading 2 marks from part 202410_957_957_0, total 16 rows starting from the beginning of the part 2024.10.22 02:27:54.962174 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1598_1598_0 to 202410_1611_1611_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:55.003533 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1606_131 to 202410_1607_1607_0 2024.10.22 02:27:55.007177 [ 1179 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.34 MiB using storage policy from min volume index 0 2024.10.22 02:27:55.017676 [ 1179 ] {} DiskLocal: Reserved 1.34 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:27:55.051181 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(eb6d10d153d6b666f05ef900b43e331b) : [0, 68720]: Setting download as failed: File segment: [0, 68720], key: eb6d10d153d6b666f05ef900b43e331b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:55.052649 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(eb6d10d153d6b666f05ef900b43e331b): No space left in cache to reserve 68721 bytes, will continue without cache download 2024.10.22 02:27:55.062213 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: eb6d10d153d6b666f05ef900b43e331b, offset: 0, size: 0 2024.10.22 02:27:55.070770 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key eb6d10d153d6b666f05ef900b43e331b for removal 2024.10.22 02:27:55.072919 [ 1476 ] {} CacheMetadata: Key eb6d10d153d6b666f05ef900b43e331b is removed from metadata 2024.10.22 02:27:55.215502 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1607_132} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1606_131 to 202410_1607_1607_0 into Compact with storage Full 2024.10.22 02:27:55.219457 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_776_776_0, total 1 rows starting from the beginning of the part 2024.10.22 02:27:55.250327 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107041 2024.10.22 02:27:55.309399 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_777_777_0, total 1 rows starting from the beginning of the part 2024.10.22 02:27:55.376822 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1607_132} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:55.399913 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1607_132} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1606_131, total 27504 rows starting from the beginning of the part 2024.10.22 02:27:55.447711 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1607_132} MergeTreeSequentialSource: Reading 2 marks from part 202410_1607_1607_0, total 96 rows starting from the beginning of the part 2024.10.22 02:27:55.485303 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} MergeTreeSequentialSource: Reading 2 marks from part 202410_778_778_0, total 1 rows starting from the beginning of the part 2024.10.22 02:27:55.732728 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_957_322} MergeTask::MergeProjectionsStage: Merge sorted 14598 rows, containing 12 columns (12 merged, 0 gathered) in 0.942713634 sec., 15485.084201084122 rows/sec., 2.42 MiB/sec. 2024.10.22 02:27:55.760691 [ 1169 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:27:55.805374 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_957_322} MergedBlockOutputStream: filled checksums 202410_1_957_322 (state Temporary) 2024.10.22 02:27:55.844767 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1374558 2024.10.22 02:27:55.915135 [ 1169 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:27:55.951608 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(c444e66be8f18cbf01f75d1f4700301b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c444e66be8f18cbf01f75d1f4700301b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:27:55.965665 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 807640, file segment info: File segment: [0, 33554431], key: c444e66be8f18cbf01f75d1f4700301b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:27:55.968709 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:56.185093 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_957_322} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_957_322 to 202410_1_957_322 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:56.189640 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_957_322} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_956_321, 202410_957_957_0] -> 202410_1_957_322 2024.10.22 02:27:56.228455 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:27:56.297408 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1607_132/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:27:56.358444 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_897_262, 202410_898_898_0] 2024.10.22 02:27:56.361004 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_897_262, 202410_898_898_0] 2024.10.22 02:27:56.369238 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_897_262, 202410_898_898_0] 2024.10.22 02:27:56.457382 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:56.460559 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:27:56.807575 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_908_908_0 (state Temporary) 2024.10.22 02:27:57.015553 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 2d675df339c68880a4224ff6df91a647:0) 2024.10.22 02:27:57.042555 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_908_908_0 to 202410_915_915_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:57.121537 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f62fbe5089155e1e6c43ee5fe9f77ae8, offset: 0, size: 152 2024.10.22 02:27:57.127780 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f62fbe5089155e1e6c43ee5fe9f77ae8 for removal 2024.10.22 02:27:57.156046 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1374558 2024.10.22 02:27:57.239772 [ 1476 ] {} CacheMetadata: Key f62fbe5089155e1e6c43ee5fe9f77ae8 is removed from metadata 2024.10.22 02:27:57.255903 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1607_132} MergeTask::MergeProjectionsStage: Merge sorted 27600 rows, containing 13 columns (13 merged, 0 gathered) in 2.22723384 sec., 12392.053094882935 rows/sec., 2.39 MiB/sec. 2024.10.22 02:27:57.414932 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 9.46 MiB, will set to 1.56 GiB (RSS), difference: -6.93 MiB 2024.10.22 02:27:57.756882 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} MergeTask::MergeProjectionsStage: Merge sorted 822 rows, containing 32 columns (32 merged, 0 gathered) in 3.756596922 sec., 218.81506508884905 rows/sec., 16.48 KiB/sec. 2024.10.22 02:27:57.853753 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1607_132} MergedBlockOutputStream: filled checksums 202410_1207_1607_132 (state Temporary) 2024.10.22 02:27:58.006563 [ 1391 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} MergedBlockOutputStream: filled checksums 202410_1_778_158 (state Temporary) 2024.10.22 02:27:58.142992 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(90a1d924b53a856da946099265082b14) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 90a1d924b53a856da946099265082b14, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:27:58.151778 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 936, file segment info: File segment: [0, 33554431], key: 90a1d924b53a856da946099265082b14, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:27:58.154563 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:27:58.209510 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1607_132} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1607_132 to 202410_1207_1607_132 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:58.214900 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1607_132} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1606_131, 202410_1607_1607_0] -> 202410_1207_1607_132 2024.10.22 02:27:58.249695 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.23 MiB. 2024.10.22 02:27:58.271783 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column RefererRegions 2024.10.22 02:27:58.293119 [ 1717 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/qch/axbiezydykehnudkmxltequphczfo, size 807640 2024.10.22 02:27:58.400677 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column RefererRegions 2024.10.22 02:27:58.437094 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column RefererRegions 2024.10.22 02:27:58.453681 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column RefererRegions 2024.10.22 02:27:58.475435 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 166 entries to flush up to offset 104085 2024.10.22 02:27:58.478678 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column RefererRegions 2024.10.22 02:27:58.612956 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column RefererRegions 2024.10.22 02:27:58.635357 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column RefererRegions 2024.10.22 02:27:58.658913 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column RefererRegions 2024.10.22 02:27:58.789445 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column RefererRegions 2024.10.22 02:27:58.866410 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/cer/leiyjctmmaczkeqtftndfnhlonjnu (LogSeriesLimiter: on interval from 2024-10-22 02:27:32 to 2024-10-22 02:27:58 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:27:59.004359 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:59.007798 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:27:59.063003 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_932_932_0 (state Temporary) 2024.10.22 02:27:59.057661 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_778_158 to 202410_1_778_158 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:59.067275 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_778_158} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_773_157, 202410_778_778_0] -> 202410_1_778_158 2024.10.22 02:27:59.105752 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:27:59.184380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6eea31bd443725b1dc2c67318648f8de) : [0, 23814]: Setting download as failed: File segment: [0, 23814], key: 6eea31bd443725b1dc2c67318648f8de, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:27:59.186684 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6eea31bd443725b1dc2c67318648f8de): No space left in cache to reserve 23815 bytes, will continue without cache download 2024.10.22 02:27:59.208565 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6eea31bd443725b1dc2c67318648f8de, offset: 0, size: 0 2024.10.22 02:27:59.229745 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6eea31bd443725b1dc2c67318648f8de for removal 2024.10.22 02:27:59.236646 [ 1476 ] {} CacheMetadata: Key 6eea31bd443725b1dc2c67318648f8de is removed from metadata 2024.10.22 02:27:59.240262 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_932_932_0 to 202410_938_938_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:27:59.260375 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 100 entries to flush up to offset 107141 2024.10.22 02:27:59.260574 [ 1220 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_933_41 to 202410_934_934_0 2024.10.22 02:27:59.268066 [ 1220 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.18 MiB using storage policy from min volume index 0 2024.10.22 02:27:59.271349 [ 1220 ] {} DiskLocal: Reserved 1.18 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:27:59.293349 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 104085 2024.10.22 02:27:59.295316 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_934_42} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_933_41 to 202410_934_934_0 into Compact with storage Full 2024.10.22 02:27:59.440528 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_934_42} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:27:59.448030 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_934_42} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_933_41, total 20051 rows starting from the beginning of the part 2024.10.22 02:27:59.465416 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_934_42} MergeTreeSequentialSource: Reading 2 marks from part 202410_934_934_0, total 111 rows starting from the beginning of the part 2024.10.22 02:27:59.835865 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:59.875568 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:27:59.939263 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:27:59.944609 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:00.011363 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1599_1599_0 (state Temporary) 2024.10.22 02:28:00.090697 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1599_1599_0 to 202410_1612_1612_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:00.141304 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1607_132 to 202410_1608_1608_0 2024.10.22 02:28:00.150907 [ 1185 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.34 MiB using storage policy from min volume index 0 2024.10.22 02:28:00.153420 [ 1185 ] {} DiskLocal: Reserved 1.34 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:00.175458 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1608_133} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1607_132 to 202410_1608_1608_0 into Compact with storage Full 2024.10.22 02:28:00.205565 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(12c5691810bec1f1cbfd4513dd241bf1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 12c5691810bec1f1cbfd4513dd241bf1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:28:00.229921 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1608_133} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:00.254686 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107141 2024.10.22 02:28:00.336694 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1608_133} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1607_132, total 27600 rows starting from the beginning of the part 2024.10.22 02:28:00.357639 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1608_133} MergeTreeSequentialSource: Reading 2 marks from part 202410_1608_1608_0, total 95 rows starting from the beginning of the part 2024.10.22 02:28:00.309182 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 8612, file segment info: File segment: [0, 33554431], key: 12c5691810bec1f1cbfd4513dd241bf1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:28:00.439253 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:00.464651 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_934_42/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:28:00.718675 [ 1168 ] {} DatabaseCatalog: Execution took 212 ms. 2024.10.22 02:28:01.056126 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_934_42} MergeTask::MergeProjectionsStage: Merge sorted 20162 rows, containing 13 columns (13 merged, 0 gathered) in 1.776811563 sec., 11347.292205797065 rows/sec., 4.93 MiB/sec. 2024.10.22 02:28:01.242708 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_934_42} MergedBlockOutputStream: filled checksums 202410_747_934_42 (state Temporary) 2024.10.22 02:28:01.253107 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 163 bytes for 900ea2e3b3f584e4aab05d4eb7d69005:0) 2024.10.22 02:28:01.258797 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fdea327dae9022c8cf9f44527b867026, offset: 0, size: 153 2024.10.22 02:28:01.260593 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fdea327dae9022c8cf9f44527b867026 for removal 2024.10.22 02:28:01.261947 [ 1476 ] {} CacheMetadata: Key fdea327dae9022c8cf9f44527b867026 is removed from metadata 2024.10.22 02:28:01.393732 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:28:01.396418 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:28:01.446314 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_934_42} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_934_42 to 202410_747_934_42 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:01.448365 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_934_42} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_933_41, 202410_934_934_0] -> 202410_747_934_42 2024.10.22 02:28:01.449790 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 10.40 MiB, will set to 1.57 GiB (RSS), difference: -21.76 MiB 2024.10.22 02:28:01.459116 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.97 MiB. 2024.10.22 02:28:01.546107 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_910_169/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:28:01.558701 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 787 2024.10.22 02:28:02.088474 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:02.096648 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:02.148175 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 14585 2024.10.22 02:28:02.165365 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(a29c55b005b5ed05580d6c061c39132e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a29c55b005b5ed05580d6c061c39132e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:28:02.180369 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 370, file segment info: File segment: [0, 33554431], key: a29c55b005b5ed05580d6c061c39132e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:28:02.181034 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_769_769_0 (state Temporary) 2024.10.22 02:28:02.186895 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:02.260224 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_190_190_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:28:02.367395 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1608_133} MergeTask::MergeProjectionsStage: Merge sorted 27695 rows, containing 13 columns (13 merged, 0 gathered) in 2.20225918 sec., 12575.72235435068 rows/sec., 2.43 MiB/sec. 2024.10.22 02:28:02.396643 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_769_769_0 to 202410_779_779_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:02.556628 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1608_133} MergedBlockOutputStream: filled checksums 202410_1207_1608_133 (state Temporary) 2024.10.22 02:28:02.562599 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:02.564652 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:02.610233 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_949_949_0 (state Temporary) 2024.10.22 02:28:02.623836 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 787 2024.10.22 02:28:02.644177 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_949_949_0 to 202410_962_962_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:02.655604 [ 1227 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_957_322 to 202410_958_958_0 2024.10.22 02:28:02.657607 [ 1227 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:02.659858 [ 1227 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:02.671284 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_958_323} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_957_322 to 202410_958_958_0 into Compact with storage Full 2024.10.22 02:28:02.739451 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_958_323} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:02.751281 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_958_323} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_957_322, total 14598 rows starting from the beginning of the part 2024.10.22 02:28:02.766417 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14585 2024.10.22 02:28:02.767261 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1608_133} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1608_133 to 202410_1207_1608_133 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:02.770667 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1608_133} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1607_132, 202410_1608_1608_0] -> 202410_1207_1608_133 2024.10.22 02:28:02.795816 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.24 MiB. 2024.10.22 02:28:02.874509 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_958_323} MergeTreeSequentialSource: Reading 2 marks from part 202410_958_958_0, total 15 rows starting from the beginning of the part 2024.10.22 02:28:03.236407 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_190_190_0 (state Temporary) 2024.10.22 02:28:03.349651 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(48c781d02da68609dab921cd4fde55df) : [0, 113367]: Setting download as failed: File segment: [0, 113367], key: 48c781d02da68609dab921cd4fde55df, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:03.351380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(48c781d02da68609dab921cd4fde55df): No space left in cache to reserve 113368 bytes, will continue without cache download 2024.10.22 02:28:03.480387 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 48c781d02da68609dab921cd4fde55df, offset: 0, size: 0 2024.10.22 02:28:03.662755 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_958_323} MergeTask::MergeProjectionsStage: Merge sorted 14613 rows, containing 12 columns (12 merged, 0 gathered) in 0.99541317 sec., 14680.336206522163 rows/sec., 2.30 MiB/sec. 2024.10.22 02:28:03.698477 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 48c781d02da68609dab921cd4fde55df for removal 2024.10.22 02:28:03.702579 [ 1476 ] {} CacheMetadata: Key 48c781d02da68609dab921cd4fde55df is removed from metadata 2024.10.22 02:28:03.804684 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_958_323} MergedBlockOutputStream: filled checksums 202410_1_958_323 (state Temporary) 2024.10.22 02:28:04.136478 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(8cec3326158d7b059e1ae4d3a5a992a4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8cec3326158d7b059e1ae4d3a5a992a4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:28:04.158138 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1375328 2024.10.22 02:28:04.159099 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4521, file segment info: File segment: [0, 33554431], key: 8cec3326158d7b059e1ae4d3a5a992a4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:28:04.166431 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:04.446041 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 107222 2024.10.22 02:28:04.529721 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_958_323} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_958_323 to 202410_1_958_323 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:04.537575 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_958_323} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_957_322, 202410_958_958_0] -> 202410_1_958_323 2024.10.22 02:28:04.585818 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:28:04.645686 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:04.648850 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:04.745340 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:04.750383 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:04.863458 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_898_263, 202410_899_899_0] 2024.10.22 02:28:04.867116 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_898_263, 202410_899_899_0] 2024.10.22 02:28:04.870564 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_909_909_0 (state Temporary) 2024.10.22 02:28:04.884851 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_898_263, 202410_899_899_0] 2024.10.22 02:28:04.944808 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1600_1600_0 (state Temporary) 2024.10.22 02:28:05.020010 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1600_1600_0 to 202410_1613_1613_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:05.069744 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1608_133 to 202410_1609_1609_0 2024.10.22 02:28:05.081077 [ 1233 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.35 MiB using storage policy from min volume index 0 2024.10.22 02:28:05.086985 [ 1233 ] {} DiskLocal: Reserved 1.35 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:05.098923 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107222 2024.10.22 02:28:05.123360 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1609_134} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1608_133 to 202410_1609_1609_0 into Compact with storage Full 2024.10.22 02:28:05.135195 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_909_909_0 to 202410_916_916_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:05.165116 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1609_134} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:05.177368 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1375328 2024.10.22 02:28:05.201810 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_911_911_0 to 202410_916_916_0 2024.10.22 02:28:05.221175 [ 1164 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:05.224639 [ 1164 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:05.278566 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1609_134} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1608_133, total 27695 rows starting from the beginning of the part 2024.10.22 02:28:05.292322 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_911_911_0 to 202410_916_916_0 into Compact with storage Full 2024.10.22 02:28:05.307756 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1609_134} MergeTreeSequentialSource: Reading 2 marks from part 202410_1609_1609_0, total 61 rows starting from the beginning of the part 2024.10.22 02:28:05.327118 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_844_158, 202410_845_850_1] 2024.10.22 02:28:05.330250 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_844_158, 202410_845_850_1] 2024.10.22 02:28:05.330947 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:05.343956 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_911_911_0, total 1498 rows starting from the beginning of the part 2024.10.22 02:28:05.346130 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_844_158, 202410_845_850_1] 2024.10.22 02:28:05.354434 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_912_912_0, total 1197 rows starting from the beginning of the part 2024.10.22 02:28:05.365701 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_913_913_0, total 1245 rows starting from the beginning of the part 2024.10.22 02:28:05.376172 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 163 bytes for 567dfe4f74d8ba4aa20081b229fb931f:0) 2024.10.22 02:28:05.380709 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_914_914_0, total 1835 rows starting from the beginning of the part 2024.10.22 02:28:05.383141 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 900ea2e3b3f584e4aab05d4eb7d69005, offset: 0, size: 163 2024.10.22 02:28:05.418164 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_915_915_0, total 770 rows starting from the beginning of the part 2024.10.22 02:28:05.443031 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_916_916_0, total 770 rows starting from the beginning of the part 2024.10.22 02:28:05.489020 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 900ea2e3b3f584e4aab05d4eb7d69005 for removal 2024.10.22 02:28:05.496610 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2d675df339c68880a4224ff6df91a647, offset: 0, size: 152 2024.10.22 02:28:05.492203 [ 1476 ] {} CacheMetadata: Key 900ea2e3b3f584e4aab05d4eb7d69005 is removed from metadata 2024.10.22 02:28:05.503572 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2d675df339c68880a4224ff6df91a647 for removal 2024.10.22 02:28:05.536931 [ 1476 ] {} CacheMetadata: Key 2d675df339c68880a4224ff6df91a647 is removed from metadata 2024.10.22 02:28:05.683388 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 11.14 MiB, will set to 1.56 GiB (RSS), difference: -1.98 MiB 2024.10.22 02:28:06.534041 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d06b3be6b595dadf005cf32c30fb14fc) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d06b3be6b595dadf005cf32c30fb14fc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:28:06.536688 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 495, file segment info: File segment: [0, 33554431], key: d06b3be6b595dadf005cf32c30fb14fc, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:28:06.548781 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:06.594985 [ 1733 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xbf/vozsdlylaccenofrufapachvckcxc, size 4521 2024.10.22 02:28:06.751521 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} MergeTask::MergeProjectionsStage: Merge sorted 7315 rows, containing 4 columns (4 merged, 0 gathered) in 1.514614146 sec., 4829.61288808668 rows/sec., 75.46 KiB/sec. 2024.10.22 02:28:06.855007 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 161 entries to flush up to offset 104246 2024.10.22 02:28:06.870860 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:28:06.901106 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:28:06.925282 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:28:06.947901 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:28:06.977833 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:28:06.996147 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1609_134} MergeTask::MergeProjectionsStage: Merge sorted 27756 rows, containing 13 columns (13 merged, 0 gathered) in 1.887292956 sec., 14706.778781619105 rows/sec., 2.84 MiB/sec. 2024.10.22 02:28:07.009037 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} MergedBlockOutputStream: filled checksums 202410_911_916_1 (state Temporary) 2024.10.22 02:28:07.027923 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:28:07.068325 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTMSource 2024.10.22 02:28:07.143111 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kdt/tglwyqxpirjetzletwvngbsnykgdd (LogSeriesLimiter: on interval from 2024-10-22 02:27:58 to 2024-10-22 02:28:07 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:28:07.154550 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_911_916_1 to 202410_911_916_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:07.161107 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_911_916_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_911_911_0, 202410_916_916_0] -> 202410_911_916_1 2024.10.22 02:28:07.173650 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:07.176976 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:07.238921 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1609_134} MergedBlockOutputStream: filled checksums 202410_1207_1609_134 (state Temporary) 2024.10.22 02:28:07.355711 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_933_933_0 (state Temporary) 2024.10.22 02:28:07.363715 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1609_134} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1609_134 to 202410_1207_1609_134 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:07.366999 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1609_134} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1608_133, 202410_1609_1609_0] -> 202410_1207_1609_134 2024.10.22 02:28:07.419892 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:28:07.423385 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_933_933_0 to 202410_939_939_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:07.469322 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 104246 2024.10.22 02:28:07.476504 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_934_42 to 202410_935_935_0 2024.10.22 02:28:07.493324 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.18 MiB using storage policy from min volume index 0 2024.10.22 02:28:07.496868 [ 1257 ] {} DiskLocal: Reserved 1.18 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:07.554425 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_935_43} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_934_42 to 202410_935_935_0 into Compact with storage Full 2024.10.22 02:28:07.558585 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5a7e80fa3d3ede78ff905220a2fd54cd) : [0, 109599]: Setting download as failed: File segment: [0, 109599], key: 5a7e80fa3d3ede78ff905220a2fd54cd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:07.560614 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5a7e80fa3d3ede78ff905220a2fd54cd): No space left in cache to reserve 109600 bytes, will continue without cache download 2024.10.22 02:28:07.600788 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.21 MiB. 2024.10.22 02:28:07.602312 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_935_43} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:07.612204 [ 1266 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 02:28:07.612724 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_935_43} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_934_42, total 20162 rows starting from the beginning of the part 2024.10.22 02:28:07.679732 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5a7e80fa3d3ede78ff905220a2fd54cd, offset: 0, size: 0 2024.10.22 02:28:07.683725 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5a7e80fa3d3ede78ff905220a2fd54cd for removal 2024.10.22 02:28:07.688150 [ 1476 ] {} CacheMetadata: Key 5a7e80fa3d3ede78ff905220a2fd54cd is removed from metadata 2024.10.22 02:28:07.736880 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_935_43} MergeTreeSequentialSource: Reading 2 marks from part 202410_935_935_0, total 137 rows starting from the beginning of the part 2024.10.22 02:28:08.556957 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2ffdb4a73ecec4e08364f3c33576a325) : [0, 52139]: Setting download as failed: File segment: [0, 52139], key: 2ffdb4a73ecec4e08364f3c33576a325, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:08.559595 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2ffdb4a73ecec4e08364f3c33576a325): No space left in cache to reserve 52140 bytes, will continue without cache download 2024.10.22 02:28:08.573264 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2ffdb4a73ecec4e08364f3c33576a325, offset: 0, size: 0 2024.10.22 02:28:08.577348 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2ffdb4a73ecec4e08364f3c33576a325 for removal 2024.10.22 02:28:08.583343 [ 1476 ] {} CacheMetadata: Key 2ffdb4a73ecec4e08364f3c33576a325 is removed from metadata 2024.10.22 02:28:08.756809 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:28:09.111768 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 107315 2024.10.22 02:28:09.116218 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_935_43} MergeTask::MergeProjectionsStage: Merge sorted 20299 rows, containing 13 columns (13 merged, 0 gathered) in 1.571271063 sec., 12918.840343971891 rows/sec., 5.62 MiB/sec. 2024.10.22 02:28:09.376808 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_935_43} MergedBlockOutputStream: filled checksums 202410_747_935_43 (state Temporary) 2024.10.22 02:28:09.506600 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:09.511091 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:09.598875 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_935_43} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_935_43 to 202410_747_935_43 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:09.604040 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 3aa7d7a95add65a07842b08383a41e09:0) 2024.10.22 02:28:09.607291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 567dfe4f74d8ba4aa20081b229fb931f, offset: 0, size: 163 2024.10.22 02:28:09.617135 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_935_43} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_934_42, 202410_935_935_0] -> 202410_747_935_43 2024.10.22 02:28:09.617539 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 567dfe4f74d8ba4aa20081b229fb931f for removal 2024.10.22 02:28:09.636967 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 14.88 MiB, will set to 1.57 GiB (RSS), difference: 13.60 MiB 2024.10.22 02:28:09.649672 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.98 MiB. 2024.10.22 02:28:09.723697 [ 1476 ] {} CacheMetadata: Key 567dfe4f74d8ba4aa20081b229fb931f is removed from metadata 2024.10.22 02:28:09.908365 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1601_1601_0 (state Temporary) 2024.10.22 02:28:10.158476 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1601_1601_0 to 202410_1614_1614_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:10.193597 [ 1224 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1609_134 to 202410_1610_1610_0 2024.10.22 02:28:10.200311 [ 1224 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.35 MiB using storage policy from min volume index 0 2024.10.22 02:28:10.205351 [ 1224 ] {} DiskLocal: Reserved 1.35 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:10.232647 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1610_135} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1609_134 to 202410_1610_1610_0 into Compact with storage Full 2024.10.22 02:28:10.267695 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 14610 2024.10.22 02:28:10.333493 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1610_135} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:10.333567 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107315 2024.10.22 02:28:10.346931 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1610_135} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1609_134, total 27756 rows starting from the beginning of the part 2024.10.22 02:28:10.408553 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:10.413782 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:10.473735 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1610_135} MergeTreeSequentialSource: Reading 2 marks from part 202410_1610_1610_0, total 74 rows starting from the beginning of the part 2024.10.22 02:28:10.474600 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_950_950_0 (state Temporary) 2024.10.22 02:28:10.699055 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(926e862cd38695968fc9c443790b0c4e) : [0, 52576]: Setting download as failed: File segment: [0, 52576], key: 926e862cd38695968fc9c443790b0c4e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:10.703745 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(926e862cd38695968fc9c443790b0c4e): No space left in cache to reserve 52577 bytes, will continue without cache download 2024.10.22 02:28:10.742491 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 926e862cd38695968fc9c443790b0c4e, offset: 0, size: 0 2024.10.22 02:28:10.745644 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 926e862cd38695968fc9c443790b0c4e for removal 2024.10.22 02:28:10.750506 [ 1476 ] {} CacheMetadata: Key 926e862cd38695968fc9c443790b0c4e is removed from metadata 2024.10.22 02:28:10.756965 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_950_950_0 to 202410_963_963_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:10.827050 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14610 2024.10.22 02:28:10.852066 [ 1177 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_958_323 to 202410_959_959_0 2024.10.22 02:28:10.857258 [ 1177 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:10.873711 [ 1177 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:10.903749 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_959_324} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_958_323 to 202410_959_959_0 into Compact with storage Full 2024.10.22 02:28:10.926313 [ 1159 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:28:10.949301 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_959_324} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:10.960711 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_959_324} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_958_323, total 14613 rows starting from the beginning of the part 2024.10.22 02:28:10.966599 [ 1152 ] {} DatabaseCatalog: Execution took 246 ms. 2024.10.22 02:28:10.982737 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_959_324} MergeTreeSequentialSource: Reading 2 marks from part 202410_959_959_0, total 22 rows starting from the beginning of the part 2024.10.22 02:28:10.991539 [ 1159 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:28:11.395788 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:28:11.399455 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:28:11.702875 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(24fdb9daf5d9a974ed6a8ad89bed4e88) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 24fdb9daf5d9a974ed6a8ad89bed4e88, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 02:28:11.707285 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 24fdb9daf5d9a974ed6a8ad89bed4e88, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 02:28:11.710558 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:12.056645 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_959_324} MergeTask::MergeProjectionsStage: Merge sorted 14635 rows, containing 12 columns (12 merged, 0 gathered) in 1.162466113 sec., 12589.614300438536 rows/sec., 1.97 MiB/sec. 2024.10.22 02:28:12.060364 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1610_135} MergeTask::MergeProjectionsStage: Merge sorted 27830 rows, containing 13 columns (13 merged, 0 gathered) in 1.843299485 sec., 15097.926422954542 rows/sec., 2.91 MiB/sec. 2024.10.22 02:28:12.185788 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1376483 2024.10.22 02:28:12.209772 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_959_324} MergedBlockOutputStream: filled checksums 202410_1_959_324 (state Temporary) 2024.10.22 02:28:12.254943 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1610_135} MergedBlockOutputStream: filled checksums 202410_1207_1610_135 (state Temporary) 2024.10.22 02:28:12.355912 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1610_135} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1610_135 to 202410_1207_1610_135 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:12.360334 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1610_135} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1609_134, 202410_1610_1610_0] -> 202410_1207_1610_135 2024.10.22 02:28:12.372376 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_959_324} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_959_324 to 202410_1_959_324 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:12.387566 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_959_324} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_958_323, 202410_959_959_0] -> 202410_1_959_324 2024.10.22 02:28:12.394728 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.38 MiB. 2024.10.22 02:28:12.442309 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:28:12.672498 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 12.90 MiB, will set to 1.57 GiB (RSS), difference: 4.82 MiB 2024.10.22 02:28:12.741674 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_190_190_0 to 202410_193_193_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:12.836623 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 40ab3f5d8f9fdce22596bcc658e4874c:0) 2024.10.22 02:28:12.842388 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d1bcd6b0bba77d0a918d1a4f49032386, offset: 0, size: 152 2024.10.22 02:28:12.845370 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d1bcd6b0bba77d0a918d1a4f49032386 for removal 2024.10.22 02:28:12.854241 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_910_169/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:28:12.942833 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 5 old parts to remove. Parts: [202410_175_175_0, 202410_176_176_0, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0] 2024.10.22 02:28:12.950461 [ 1476 ] {} CacheMetadata: Key d1bcd6b0bba77d0a918d1a4f49032386 is removed from metadata 2024.10.22 02:28:12.960871 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from filesystem (serially): Parts: [202410_175_175_0, 202410_176_176_0, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0] 2024.10.22 02:28:13.038532 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:13.045320 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:13.106521 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 5 parts from memory: Parts: [202410_175_175_0, 202410_176_176_0, 202410_177_177_0, 202410_178_178_0, 202410_179_179_0] 2024.10.22 02:28:13.202537 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_899_264, 202410_900_900_0] 2024.10.22 02:28:13.205827 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_899_264, 202410_900_900_0] 2024.10.22 02:28:13.222218 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_910_910_0 (state Temporary) 2024.10.22 02:28:13.224045 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_899_264, 202410_900_900_0] 2024.10.22 02:28:13.347606 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_910_910_0 to 202410_917_917_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:13.369027 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1376483 2024.10.22 02:28:14.059327 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_877_26, 202410_878_878_0, 202410_879_879_0, 202410_880_880_0, 202410_881_881_0, 202410_882_882_0] 2024.10.22 02:28:14.061723 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_877_26, 202410_878_878_0, 202410_879_879_0, 202410_880_880_0, 202410_881_881_0, 202410_882_882_0] 2024.10.22 02:28:14.062765 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(61dbabc10dc23ed8e21beffc80ebf0cc) : [0, 31073]: Setting download as failed: File segment: [0, 31073], key: 61dbabc10dc23ed8e21beffc80ebf0cc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:14.064248 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(61dbabc10dc23ed8e21beffc80ebf0cc): No space left in cache to reserve 31074 bytes, will continue without cache download 2024.10.22 02:28:14.094822 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 61dbabc10dc23ed8e21beffc80ebf0cc, offset: 0, size: 0 2024.10.22 02:28:14.096524 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 61dbabc10dc23ed8e21beffc80ebf0cc for removal 2024.10.22 02:28:14.098027 [ 1476 ] {} CacheMetadata: Key 61dbabc10dc23ed8e21beffc80ebf0cc is removed from metadata 2024.10.22 02:28:14.101646 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_877_26, 202410_878_878_0, 202410_879_879_0, 202410_880_880_0, 202410_881_881_0, 202410_882_882_0] 2024.10.22 02:28:14.352606 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 86 entries to flush up to offset 107401 2024.10.22 02:28:14.393044 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 8.46 MiB, will set to 1.56 GiB (RSS), difference: -2.44 MiB 2024.10.22 02:28:14.639518 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:14.642306 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:14.809745 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1602_1602_0 (state Temporary) 2024.10.22 02:28:14.935261 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1602_1602_0 to 202410_1615_1615_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:14.965034 [ 1181 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1610_135 to 202410_1611_1611_0 2024.10.22 02:28:14.978331 [ 1181 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.35 MiB using storage policy from min volume index 0 2024.10.22 02:28:14.985284 [ 1181 ] {} DiskLocal: Reserved 1.35 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:15.002216 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107401 2024.10.22 02:28:15.003327 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 134 entries to flush up to offset 104380 2024.10.22 02:28:15.012015 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1611_136} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1610_135 to 202410_1611_1611_0 into Compact with storage Full 2024.10.22 02:28:15.042857 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1611_136} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:15.047510 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 151 bytes for 8d732af64140e87aff46fb58f76382c8:0) 2024.10.22 02:28:15.050281 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3aa7d7a95add65a07842b08383a41e09, offset: 0, size: 152 2024.10.22 02:28:15.053355 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1611_136} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1610_135, total 27830 rows starting from the beginning of the part 2024.10.22 02:28:15.054429 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3aa7d7a95add65a07842b08383a41e09 for removal 2024.10.22 02:28:15.056256 [ 1476 ] {} CacheMetadata: Key 3aa7d7a95add65a07842b08383a41e09 is removed from metadata 2024.10.22 02:28:15.085511 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1611_136} MergeTreeSequentialSource: Reading 2 marks from part 202410_1611_1611_0, total 70 rows starting from the beginning of the part 2024.10.22 02:28:15.233738 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7678 2024.10.22 02:28:15.385174 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:15.394560 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:15.450598 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_934_934_0 (state Temporary) 2024.10.22 02:28:15.813284 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_934_934_0 to 202410_940_940_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:15.846357 [ 1156 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_935_43 to 202410_936_936_0 2024.10.22 02:28:15.851512 [ 1156 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.19 MiB using storage policy from min volume index 0 2024.10.22 02:28:15.854495 [ 1156 ] {} DiskLocal: Reserved 1.19 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:15.958446 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_936_44} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_935_43 to 202410_936_936_0 into Compact with storage Full 2024.10.22 02:28:15.962233 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 104380 2024.10.22 02:28:15.984085 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_936_44} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:15.994821 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_936_44} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_935_43, total 20299 rows starting from the beginning of the part 2024.10.22 02:28:16.025528 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_936_44} MergeTreeSequentialSource: Reading 2 marks from part 202410_936_936_0, total 135 rows starting from the beginning of the part 2024.10.22 02:28:16.264341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(dbe3af98113da7e74a6c9404456be050) : [0, 9544]: Setting download as failed: File segment: [0, 9544], key: dbe3af98113da7e74a6c9404456be050, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:16.265357 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(dbe3af98113da7e74a6c9404456be050): No space left in cache to reserve 9545 bytes, will continue without cache download 2024.10.22 02:28:16.274390 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: dbe3af98113da7e74a6c9404456be050, offset: 0, size: 0 2024.10.22 02:28:16.295540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key dbe3af98113da7e74a6c9404456be050 for removal 2024.10.22 02:28:16.298770 [ 1476 ] {} CacheMetadata: Key dbe3af98113da7e74a6c9404456be050 is removed from metadata 2024.10.22 02:28:16.587486 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 13.95 MiB, will set to 1.57 GiB (RSS), difference: -15.75 MiB 2024.10.22 02:28:17.121731 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1611_136} MergeTask::MergeProjectionsStage: Merge sorted 27900 rows, containing 13 columns (13 merged, 0 gathered) in 2.122771924 sec., 13143.192485524885 rows/sec., 2.53 MiB/sec. 2024.10.22 02:28:17.328727 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1611_136} MergedBlockOutputStream: filled checksums 202410_1207_1611_136 (state Temporary) 2024.10.22 02:28:17.558930 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 163 bytes for f3786bf8ab8dce228698903aa379c611:0) 2024.10.22 02:28:17.561089 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 40ab3f5d8f9fdce22596bcc658e4874c, offset: 0, size: 152 2024.10.22 02:28:17.565022 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 40ab3f5d8f9fdce22596bcc658e4874c for removal 2024.10.22 02:28:17.570316 [ 1476 ] {} CacheMetadata: Key 40ab3f5d8f9fdce22596bcc658e4874c is removed from metadata 2024.10.22 02:28:17.628191 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 788 2024.10.22 02:28:17.672324 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_936_44/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:28:17.841412 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1611_136} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1611_136 to 202410_1207_1611_136 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:17.847224 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1611_136} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1610_135, 202410_1611_1611_0] -> 202410_1207_1611_136 2024.10.22 02:28:17.881725 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.39 MiB. 2024.10.22 02:28:17.915605 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_936_44} MergeTask::MergeProjectionsStage: Merge sorted 20434 rows, containing 13 columns (13 merged, 0 gathered) in 2.025047811 sec., 10090.62595411482 rows/sec., 4.39 MiB/sec. 2024.10.22 02:28:17.936360 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:17.941655 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:18.197234 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_936_44} MergedBlockOutputStream: filled checksums 202410_747_936_44 (state Temporary) 2024.10.22 02:28:18.272831 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_936_44} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_936_44 to 202410_747_936_44 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:18.283354 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_936_44} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_935_43, 202410_936_936_0] -> 202410_747_936_44 2024.10.22 02:28:18.300649 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_770_770_0 (state Temporary) 2024.10.22 02:28:18.330139 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 14627 2024.10.22 02:28:18.333621 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.98 MiB. 2024.10.22 02:28:18.486915 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_770_770_0 to 202410_780_780_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:18.565226 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:18.567275 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:18.583223 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 788 2024.10.22 02:28:18.592449 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(21a17f82776082960c2e8b96556b4016) : [0, 105748]: Setting download as failed: File segment: [0, 105748], key: 21a17f82776082960c2e8b96556b4016, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:18.594025 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(21a17f82776082960c2e8b96556b4016): No space left in cache to reserve 105749 bytes, will continue without cache download 2024.10.22 02:28:18.716557 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_951_951_0 (state Temporary) 2024.10.22 02:28:18.717808 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 21a17f82776082960c2e8b96556b4016, offset: 0, size: 0 2024.10.22 02:28:18.720465 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 21a17f82776082960c2e8b96556b4016 for removal 2024.10.22 02:28:18.739449 [ 1476 ] {} CacheMetadata: Key 21a17f82776082960c2e8b96556b4016 is removed from metadata 2024.10.22 02:28:18.868401 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_951_951_0 to 202410_964_964_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:18.913081 [ 1163 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_959_324 to 202410_960_960_0 2024.10.22 02:28:18.923765 [ 1163 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:18.927812 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:18.948100 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14627 2024.10.22 02:28:19.013252 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 107474 2024.10.22 02:28:19.049896 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_960_325} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_959_324 to 202410_960_960_0 into Compact with storage Full 2024.10.22 02:28:19.069773 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_960_325} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:19.098185 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_960_325} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_959_324, total 14635 rows starting from the beginning of the part 2024.10.22 02:28:19.121963 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_960_325} MergeTreeSequentialSource: Reading 2 marks from part 202410_960_960_0, total 18 rows starting from the beginning of the part 2024.10.22 02:28:19.269204 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:19.272137 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:19.554647 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1603_1603_0 (state Temporary) 2024.10.22 02:28:19.540527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 5bad7cc4078a0e7282b47650800a4413:0) 2024.10.22 02:28:19.557581 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8d732af64140e87aff46fb58f76382c8, offset: 0, size: 151 2024.10.22 02:28:19.561501 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8d732af64140e87aff46fb58f76382c8 for removal 2024.10.22 02:28:19.565792 [ 1476 ] {} CacheMetadata: Key 8d732af64140e87aff46fb58f76382c8 is removed from metadata 2024.10.22 02:28:19.788898 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 14.78 MiB, will set to 1.56 GiB (RSS), difference: 11.58 MiB 2024.10.22 02:28:19.879879 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1510_75, 202410_1511_1511_0, 202410_1512_1512_0, 202410_1513_1513_0, 202410_1514_1514_0, 202410_1515_1515_0] 2024.10.22 02:28:19.880935 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1603_1603_0 to 202410_1616_1616_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:19.888499 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1510_75, 202410_1511_1511_0, 202410_1512_1512_0, 202410_1513_1513_0, 202410_1514_1514_0, 202410_1515_1515_0] 2024.10.22 02:28:19.918403 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1510_75, 202410_1511_1511_0, 202410_1512_1512_0, 202410_1513_1513_0, 202410_1514_1514_0, 202410_1515_1515_0] 2024.10.22 02:28:20.134693 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1611_136 to 202410_1612_1612_0 2024.10.22 02:28:20.150170 [ 1214 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.36 MiB using storage policy from min volume index 0 2024.10.22 02:28:20.154414 [ 1214 ] {} DiskLocal: Reserved 1.36 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:20.159335 [ 1214 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 259 ms. 2024.10.22 02:28:20.178657 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1612_137} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1611_136 to 202410_1612_1612_0 into Compact with storage Full 2024.10.22 02:28:20.230960 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1612_137} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:20.241732 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1612_137} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1611_136, total 27900 rows starting from the beginning of the part 2024.10.22 02:28:20.251618 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107474 2024.10.22 02:28:20.256858 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1612_137} MergeTreeSequentialSource: Reading 2 marks from part 202410_1612_1612_0, total 100 rows starting from the beginning of the part 2024.10.22 02:28:20.298850 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_960_325} MergeTask::MergeProjectionsStage: Merge sorted 14653 rows, containing 12 columns (12 merged, 0 gathered) in 1.25818754 sec., 11646.117557323767 rows/sec., 1.82 MiB/sec. 2024.10.22 02:28:20.357805 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_960_325} MergedBlockOutputStream: filled checksums 202410_1_960_325 (state Temporary) 2024.10.22 02:28:20.373209 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1377638 2024.10.22 02:28:20.528600 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_960_325} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_960_325 to 202410_1_960_325 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:20.550020 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_960_325} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_959_324, 202410_960_960_0] -> 202410_1_960_325 2024.10.22 02:28:20.583433 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:28:20.861166 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_900_265, 202410_901_901_0] 2024.10.22 02:28:20.866194 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_900_265, 202410_901_901_0] 2024.10.22 02:28:20.872547 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b39dee2eb8c41bc239ca06ab0e367176) : [0, 2626]: Setting download as failed: File segment: [0, 2626], key: b39dee2eb8c41bc239ca06ab0e367176, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:20.874352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b39dee2eb8c41bc239ca06ab0e367176): No space left in cache to reserve 2627 bytes, will continue without cache download 2024.10.22 02:28:20.885166 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_900_265, 202410_901_901_0] 2024.10.22 02:28:20.932330 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b39dee2eb8c41bc239ca06ab0e367176, offset: 0, size: 0 2024.10.22 02:28:20.938888 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b39dee2eb8c41bc239ca06ab0e367176 for removal 2024.10.22 02:28:20.940841 [ 1476 ] {} CacheMetadata: Key b39dee2eb8c41bc239ca06ab0e367176 is removed from metadata 2024.10.22 02:28:21.130663 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:21.140643 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:21.287794 [ 1211 ] {} DatabaseCatalog: Execution took 324 ms. 2024.10.22 02:28:21.397516 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:28:21.400504 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:28:21.555400 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_911_911_0 (state Temporary) 2024.10.22 02:28:21.646152 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_911_911_0 to 202410_918_918_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:21.798304 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1377638 2024.10.22 02:28:22.425307 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 17.92 MiB, will set to 1.57 GiB (RSS), difference: 13.81 MiB 2024.10.22 02:28:22.559511 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for a3d16c1d7564e2a683c267f3f935eb5e:0) 2024.10.22 02:28:22.563201 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f3786bf8ab8dce228698903aa379c611, offset: 0, size: 163 2024.10.22 02:28:22.573366 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f3786bf8ab8dce228698903aa379c611 for removal 2024.10.22 02:28:22.576221 [ 1476 ] {} CacheMetadata: Key f3786bf8ab8dce228698903aa379c611 is removed from metadata 2024.10.22 02:28:22.652681 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1612_137} MergeTask::MergeProjectionsStage: Merge sorted 28000 rows, containing 13 columns (13 merged, 0 gathered) in 2.491583172 sec., 11237.834768937026 rows/sec., 2.17 MiB/sec. 2024.10.22 02:28:22.767135 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 43 entries to flush up to offset 7721 2024.10.22 02:28:22.855315 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1612_137} MergedBlockOutputStream: filled checksums 202410_1207_1612_137 (state Temporary) 2024.10.22 02:28:22.919267 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1612_137} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1612_137 to 202410_1207_1612_137 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:22.923886 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1612_137} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1611_136, 202410_1612_1612_0] -> 202410_1207_1612_137 2024.10.22 02:28:22.947242 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.40 MiB. 2024.10.22 02:28:23.483148 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 149 entries to flush up to offset 104529 2024.10.22 02:28:23.631848 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:23.636170 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:23.891380 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_935_935_0 (state Temporary) 2024.10.22 02:28:23.920269 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0733187a27d62383b47e21b8e610a9c7) : [0, 3612]: Setting download as failed: File segment: [0, 3612], key: 0733187a27d62383b47e21b8e610a9c7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:23.921599 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0733187a27d62383b47e21b8e610a9c7): No space left in cache to reserve 3613 bytes, will continue without cache download 2024.10.22 02:28:23.942950 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0733187a27d62383b47e21b8e610a9c7, offset: 0, size: 0 2024.10.22 02:28:23.945119 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0733187a27d62383b47e21b8e610a9c7 for removal 2024.10.22 02:28:23.948250 [ 1476 ] {} CacheMetadata: Key 0733187a27d62383b47e21b8e610a9c7 is removed from metadata 2024.10.22 02:28:24.251295 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_935_935_0 to 202410_941_941_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:24.260561 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 107543 2024.10.22 02:28:24.269290 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_936_44 to 202410_937_937_0 2024.10.22 02:28:24.274137 [ 1268 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.19 MiB using storage policy from min volume index 0 2024.10.22 02:28:24.277084 [ 1268 ] {} DiskLocal: Reserved 1.19 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:24.296397 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 104529 2024.10.22 02:28:24.333973 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_937_45} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_936_44 to 202410_937_937_0 into Compact with storage Full 2024.10.22 02:28:24.371051 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_937_45} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:24.372855 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 11.63 MiB, will set to 1.56 GiB (RSS), difference: -4.90 MiB 2024.10.22 02:28:24.385410 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_937_45} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_936_44, total 20434 rows starting from the beginning of the part 2024.10.22 02:28:24.420347 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:24.424432 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:24.483116 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_910_169/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:28:24.515068 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1604_1604_0 (state Temporary) 2024.10.22 02:28:24.523373 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_937_45} MergeTreeSequentialSource: Reading 2 marks from part 202410_937_937_0, total 130 rows starting from the beginning of the part 2024.10.22 02:28:24.752917 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 7b79e8bd60dd0bc2c5375e53ee6e1426:0) 2024.10.22 02:28:24.755014 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5bad7cc4078a0e7282b47650800a4413, offset: 0, size: 152 2024.10.22 02:28:24.756777 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5bad7cc4078a0e7282b47650800a4413 for removal 2024.10.22 02:28:24.759271 [ 1476 ] {} CacheMetadata: Key 5bad7cc4078a0e7282b47650800a4413 is removed from metadata 2024.10.22 02:28:24.973289 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1604_1604_0 to 202410_1617_1617_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:25.029081 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1612_137 to 202410_1613_1613_0 2024.10.22 02:28:25.041688 [ 1248 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.36 MiB using storage policy from min volume index 0 2024.10.22 02:28:25.046587 [ 1248 ] {} DiskLocal: Reserved 1.36 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:25.051501 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107543 2024.10.22 02:28:25.057067 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1613_138} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1612_137 to 202410_1613_1613_0 into Compact with storage Full 2024.10.22 02:28:25.195711 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1613_138} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:25.443134 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1613_138} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1612_137, total 28000 rows starting from the beginning of the part 2024.10.22 02:28:25.664030 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1613_138} MergeTreeSequentialSource: Reading 2 marks from part 202410_1613_1613_0, total 81 rows starting from the beginning of the part 2024.10.22 02:28:25.905312 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_937_45} MergeTask::MergeProjectionsStage: Merge sorted 20564 rows, containing 13 columns (13 merged, 0 gathered) in 1.583922115 sec., 12982.961602250247 rows/sec., 5.65 MiB/sec. 2024.10.22 02:28:25.998110 [ 1262 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:28:26.063405 [ 1262 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:28:26.090539 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 789 2024.10.22 02:28:26.175340 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_937_45} MergedBlockOutputStream: filled checksums 202410_747_937_45 (state Temporary) 2024.10.22 02:28:26.290451 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(d2f156c2ce6cf6a3c29e6f63a98cd8e1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d2f156c2ce6cf6a3c29e6f63a98cd8e1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 02:28:26.297959 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: d2f156c2ce6cf6a3c29e6f63a98cd8e1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1392, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1392, kind: Regular, unbound: 0 2024.10.22 02:28:26.304077 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:26.518356 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:26.529410 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:26.550453 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 14639 2024.10.22 02:28:26.572751 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_937_45} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_937_45 to 202410_747_937_45 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:26.576470 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_937_45} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_936_44, 202410_937_937_0] -> 202410_747_937_45 2024.10.22 02:28:26.597558 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_771_771_0 (state Temporary) 2024.10.22 02:28:26.623312 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 22.98 MiB. 2024.10.22 02:28:26.681707 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:26.689021 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:26.693196 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_771_771_0 to 202410_781_781_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:26.740591 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_952_952_0 (state Temporary) 2024.10.22 02:28:26.766090 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 789 2024.10.22 02:28:26.941661 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_952_952_0 to 202410_965_965_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:27.002881 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14639 2024.10.22 02:28:27.108220 [ 1273 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_960_325 to 202410_961_961_0 2024.10.22 02:28:27.119396 [ 1273 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:27.125917 [ 1273 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:27.151248 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_961_326} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_960_325 to 202410_961_961_0 into Compact with storage Full 2024.10.22 02:28:27.239671 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8854ab9a3da69cbe43005dc2763ef12d) : [0, 30457]: Setting download as failed: File segment: [0, 30457], key: 8854ab9a3da69cbe43005dc2763ef12d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:27.242252 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8854ab9a3da69cbe43005dc2763ef12d): No space left in cache to reserve 30458 bytes, will continue without cache download 2024.10.22 02:28:27.294595 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_961_326} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:27.325435 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_961_326} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_960_325, total 14653 rows starting from the beginning of the part 2024.10.22 02:28:27.253622 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8854ab9a3da69cbe43005dc2763ef12d, offset: 0, size: 0 2024.10.22 02:28:27.363227 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8854ab9a3da69cbe43005dc2763ef12d for removal 2024.10.22 02:28:27.365266 [ 1476 ] {} CacheMetadata: Key 8854ab9a3da69cbe43005dc2763ef12d is removed from metadata 2024.10.22 02:28:27.411240 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_961_326} MergeTreeSequentialSource: Reading 2 marks from part 202410_961_961_0, total 17 rows starting from the beginning of the part 2024.10.22 02:28:27.964352 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1613_138} MergeTask::MergeProjectionsStage: Merge sorted 28081 rows, containing 13 columns (13 merged, 0 gathered) in 2.911823385 sec., 9643.78545232406 rows/sec., 1.86 MiB/sec. 2024.10.22 02:28:28.012300 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1613_138} MergedBlockOutputStream: filled checksums 202410_1207_1613_138 (state Temporary) 2024.10.22 02:28:28.096168 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_961_326} MergeTask::MergeProjectionsStage: Merge sorted 14670 rows, containing 12 columns (12 merged, 0 gathered) in 0.950433629 sec., 15435.059905692795 rows/sec., 2.41 MiB/sec. 2024.10.22 02:28:28.148695 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_961_326} MergedBlockOutputStream: filled checksums 202410_1_961_326 (state Temporary) 2024.10.22 02:28:28.206555 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1613_138} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1613_138 to 202410_1207_1613_138 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:28.211750 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1613_138} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1612_137, 202410_1613_1613_0] -> 202410_1207_1613_138 2024.10.22 02:28:28.266806 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 16.47 MiB, will set to 1.57 GiB (RSS), difference: 5.60 MiB 2024.10.22 02:28:28.349797 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.40 MiB. 2024.10.22 02:28:28.442432 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_901_266, 202410_902_902_0] 2024.10.22 02:28:28.444283 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_901_266, 202410_902_902_0] 2024.10.22 02:28:28.499666 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_961_326} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_961_326 to 202410_1_961_326 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:28.511478 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_961_326} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_960_325, 202410_961_961_0] -> 202410_1_961_326 2024.10.22 02:28:28.553470 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_901_266, 202410_902_902_0] 2024.10.22 02:28:28.638802 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:28:28.667813 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(334029ea98403bc22d6c23ffecdc6ff6) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 334029ea98403bc22d6c23ffecdc6ff6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:28:28.671487 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 38350, file segment info: File segment: [0, 33554431], key: 334029ea98403bc22d6c23ffecdc6ff6, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:28:28.700902 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:28.752977 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_728_148, 202410_729_729_0, 202410_730_730_0, 202410_731_731_0, 202410_732_732_0, 202410_733_733_0] 2024.10.22 02:28:28.754788 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_728_148, 202410_729_729_0, 202410_730_730_0, 202410_731_731_0, 202410_732_732_0, 202410_733_733_0] 2024.10.22 02:28:28.789485 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_728_148, 202410_729_729_0, 202410_730_730_0, 202410_731_731_0, 202410_732_732_0, 202410_733_733_0] 2024.10.22 02:28:28.809325 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1378793 2024.10.22 02:28:29.062359 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 107625 2024.10.22 02:28:29.437935 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:29.440432 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.29 GiB. 2024.10.22 02:28:29.512987 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:29.521802 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:29.650552 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_912_912_0 (state Temporary) 2024.10.22 02:28:29.691618 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1605_1605_0 (state Temporary) 2024.10.22 02:28:29.804719 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 925cf71b1459b3e2c9fd119ecb5e1874:0) 2024.10.22 02:28:29.810955 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a3d16c1d7564e2a683c267f3f935eb5e, offset: 0, size: 152 2024.10.22 02:28:29.816350 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a3d16c1d7564e2a683c267f3f935eb5e for removal 2024.10.22 02:28:29.824813 [ 1476 ] {} CacheMetadata: Key a3d16c1d7564e2a683c267f3f935eb5e is removed from metadata 2024.10.22 02:28:29.835470 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_912_912_0 to 202410_919_919_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:29.871074 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1378793 2024.10.22 02:28:30.097594 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1605_1605_0 to 202410_1618_1618_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:30.150213 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1613_138 to 202410_1614_1614_0 2024.10.22 02:28:30.156367 [ 1195 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.36 MiB using storage policy from min volume index 0 2024.10.22 02:28:30.158054 [ 1195 ] {} DiskLocal: Reserved 1.36 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:30.159383 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107625 2024.10.22 02:28:30.284000 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1614_139} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1613_138 to 202410_1614_1614_0 into Compact with storage Full 2024.10.22 02:28:30.373217 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(4f8a6ecf638b6378e872708a55a76697) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4f8a6ecf638b6378e872708a55a76697, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:28:30.376760 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 467, file segment info: File segment: [0, 33554431], key: 4f8a6ecf638b6378e872708a55a76697, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:28:30.383478 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:30.432669 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_910_169} MergeTask::MergeProjectionsStage: Merge sorted 1381603 rows, containing 4 columns (4 merged, 0 gathered) in 41.563909894 sec., 33240.44834866324 rows/sec., 498.58 KiB/sec. 2024.10.22 02:28:30.517083 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1614_139} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:30.517964 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_910_169} MergedBlockOutputStream: filled checksums 202410_1_910_169 (state Temporary) 2024.10.22 02:28:30.528817 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1614_139} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1613_138, total 28081 rows starting from the beginning of the part 2024.10.22 02:28:30.551241 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1614_139} MergeTreeSequentialSource: Reading 2 marks from part 202410_1614_1614_0, total 93 rows starting from the beginning of the part 2024.10.22 02:28:31.381638 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(bd38d535e1795075f614bf930251a9a0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: bd38d535e1795075f614bf930251a9a0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:28:31.389052 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1243, file segment info: File segment: [0, 33554431], key: bd38d535e1795075f614bf930251a9a0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:28:31.395545 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:31.396339 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:28:31.399454 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:28:31.562455 [ 1170 ] {} DatabaseCatalog: Execution took 273 ms. 2024.10.22 02:28:31.834289 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 150 entries to flush up to offset 104679 2024.10.22 02:28:31.883856 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_857_857_0, 202410_858_858_0, 202410_859_859_0, 202410_860_860_0, 202410_861_861_0, 202410_862_862_0] 2024.10.22 02:28:31.891821 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_857_857_0, 202410_858_858_0, 202410_859_859_0, 202410_860_860_0, 202410_861_861_0, 202410_862_862_0] 2024.10.22 02:28:31.949363 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_857_857_0, 202410_858_858_0, 202410_859_859_0, 202410_860_860_0, 202410_861_861_0, 202410_862_862_0] 2024.10.22 02:28:31.965545 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:31.967172 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:32.016611 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_936_936_0 (state Temporary) 2024.10.22 02:28:32.082572 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_936_936_0 to 202410_942_942_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:32.133578 [ 1179 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_937_45 to 202410_938_938_0 2024.10.22 02:28:32.143015 [ 1179 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.20 MiB using storage policy from min volume index 0 2024.10.22 02:28:32.146770 [ 1179 ] {} DiskLocal: Reserved 1.20 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:32.151820 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 104679 2024.10.22 02:28:32.156505 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_938_46} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_937_45 to 202410_938_938_0 into Compact with storage Full 2024.10.22 02:28:32.161939 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(e91f9eb96fba07aea91d0efdd59241f3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e91f9eb96fba07aea91d0efdd59241f3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:28:32.164215 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 19244, file segment info: File segment: [0, 33554431], key: e91f9eb96fba07aea91d0efdd59241f3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:28:32.166590 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:32.312470 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_938_46} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:32.326338 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_938_46} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_937_45, total 20564 rows starting from the beginning of the part 2024.10.22 02:28:32.419205 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1614_139} MergeTask::MergeProjectionsStage: Merge sorted 28174 rows, containing 13 columns (13 merged, 0 gathered) in 2.146286836 sec., 13126.856824275841 rows/sec., 2.53 MiB/sec. 2024.10.22 02:28:32.456343 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 14.88 MiB, will set to 1.56 GiB (RSS), difference: 9.18 MiB 2024.10.22 02:28:32.461751 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_938_46} MergeTreeSequentialSource: Reading 2 marks from part 202410_938_938_0, total 166 rows starting from the beginning of the part 2024.10.22 02:28:32.498079 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1614_139} MergedBlockOutputStream: filled checksums 202410_1207_1614_139 (state Temporary) 2024.10.22 02:28:32.820050 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1614_139} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1614_139 to 202410_1207_1614_139 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:32.823098 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1614_139} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1613_138, 202410_1614_1614_0] -> 202410_1207_1614_139 2024.10.22 02:28:32.907747 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d828c9256f94787cc4c6dc1c8b3cbdbf) : [0, 29352]: Setting download as failed: File segment: [0, 29352], key: d828c9256f94787cc4c6dc1c8b3cbdbf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:32.910184 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d828c9256f94787cc4c6dc1c8b3cbdbf): No space left in cache to reserve 29353 bytes, will continue without cache download 2024.10.22 02:28:32.950736 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.40 MiB. 2024.10.22 02:28:33.035588 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d828c9256f94787cc4c6dc1c8b3cbdbf, offset: 0, size: 0 2024.10.22 02:28:33.037517 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d828c9256f94787cc4c6dc1c8b3cbdbf for removal 2024.10.22 02:28:33.039688 [ 1476 ] {} CacheMetadata: Key d828c9256f94787cc4c6dc1c8b3cbdbf is removed from metadata 2024.10.22 02:28:33.348235 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_938_46/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:28:33.661540 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(7fae0fd3b5272a5f5f520113b3012e2e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7fae0fd3b5272a5f5f520113b3012e2e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:28:33.663129 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 562, file segment info: File segment: [0, 33554431], key: 7fae0fd3b5272a5f5f520113b3012e2e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1400, kind: Regular, unbound: 0 2024.10.22 02:28:33.664021 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:33.791264 [ 1677 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kdt/tglwyqxpirjetzletwvngbsnykgdd, size 19244 2024.10.22 02:28:33.985839 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:28:34.009179 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:28:34.076040 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_938_46} MergeTask::MergeProjectionsStage: Merge sorted 20730 rows, containing 13 columns (13 merged, 0 gathered) in 1.922863124 sec., 10780.798560886022 rows/sec., 4.69 MiB/sec. 2024.10.22 02:28:34.145741 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:28:34.156974 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:28:34.168537 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 107698 2024.10.22 02:28:34.172530 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:28:34.208625 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:28:34.268020 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 790 2024.10.22 02:28:34.350576 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTMMedium 2024.10.22 02:28:34.402740 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/vdz/mtxtstnbdlqyfjanyawibhysonspw (LogSeriesLimiter: on interval from 2024-10-22 02:28:07 to 2024-10-22 02:28:34 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:28:34.443002 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_910_169} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_910_169 to 202410_1_910_169 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:34.447381 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_910_169} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_904_168, 202410_905_910_1] -> 202410_1_910_169 2024.10.22 02:28:34.455763 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:34.457655 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:34.465626 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_938_46} MergedBlockOutputStream: filled checksums 202410_747_938_46 (state Temporary) 2024.10.22 02:28:34.470732 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:28:34.506369 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 14659 2024.10.22 02:28:34.556078 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1606_1606_0 (state Temporary) 2024.10.22 02:28:34.557786 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_938_46} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_938_46 to 202410_747_938_46 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:34.562581 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_938_46} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_937_45, 202410_938_938_0] -> 202410_747_938_46 2024.10.22 02:28:34.637180 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1606_1606_0 to 202410_1619_1619_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:34.690815 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.04 MiB. 2024.10.22 02:28:34.703749 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:34.710786 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:34.739781 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 150 bytes for a93f6d9db9ddf20aeedc191f5b90f46f:0) 2024.10.22 02:28:34.743234 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7b79e8bd60dd0bc2c5375e53ee6e1426, offset: 0, size: 152 2024.10.22 02:28:34.745794 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7b79e8bd60dd0bc2c5375e53ee6e1426 for removal 2024.10.22 02:28:34.748917 [ 1476 ] {} CacheMetadata: Key 7b79e8bd60dd0bc2c5375e53ee6e1426 is removed from metadata 2024.10.22 02:28:34.762720 [ 1155 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1614_139 to 202410_1615_1615_0 2024.10.22 02:28:34.770091 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:34.772506 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:34.786887 [ 1155 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 02:28:34.797661 [ 1155 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:34.799936 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107698 2024.10.22 02:28:34.906984 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_772_772_0 (state Temporary) 2024.10.22 02:28:34.943456 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1615_140} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1614_139 to 202410_1615_1615_0 into Compact with storage Full 2024.10.22 02:28:34.953803 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_953_953_0 (state Temporary) 2024.10.22 02:28:34.978536 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1615_140} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:34.993339 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1615_140} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1614_139, total 28174 rows starting from the beginning of the part 2024.10.22 02:28:35.005801 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_953_953_0 to 202410_966_966_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:35.044248 [ 1258 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_961_326 to 202410_962_962_0 2024.10.22 02:28:35.050596 [ 1258 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:35.052643 [ 1258 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:35.058089 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14659 2024.10.22 02:28:35.076532 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_772_772_0 to 202410_782_782_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:35.118634 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_962_327} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_961_326 to 202410_962_962_0 into Compact with storage Full 2024.10.22 02:28:35.121924 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1615_140} MergeTreeSequentialSource: Reading 2 marks from part 202410_1615_1615_0, total 86 rows starting from the beginning of the part 2024.10.22 02:28:35.123748 [ 1192 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 02:28:35.212340 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 790 2024.10.22 02:28:35.223389 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_962_327} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:35.337456 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_962_327} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_961_326, total 14670 rows starting from the beginning of the part 2024.10.22 02:28:35.548265 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_962_327} MergeTreeSequentialSource: Reading 2 marks from part 202410_962_962_0, total 21 rows starting from the beginning of the part 2024.10.22 02:28:35.579211 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(0034b50a8d88de22144e78bdb766d39b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0034b50a8d88de22144e78bdb766d39b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:28:35.596835 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1004, file segment info: File segment: [0, 33554431], key: 0034b50a8d88de22144e78bdb766d39b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:28:35.607226 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:35.672729 [ 1683 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/cer/leiyjctmmaczkeqtftndfnhlonjnu, size 3088250 2024.10.22 02:28:36.329439 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 02:28:36.342229 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 02:28:36.361416 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 02:28:36.396925 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 02:28:36.421754 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 02:28:36.448086 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 02:28:36.477590 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.12 MiB, will set to 1.56 GiB (RSS), difference: -9.41 MiB 2024.10.22 02:28:36.570872 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 02:28:36.635950 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 02:28:36.657580 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(51e0a3dc9f3e305323c1d3153cda8a14) : [0, 184008]: Setting download as failed: File segment: [0, 184008], key: 51e0a3dc9f3e305323c1d3153cda8a14, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:36.660278 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(51e0a3dc9f3e305323c1d3153cda8a14): No space left in cache to reserve 184009 bytes, will continue without cache download 2024.10.22 02:28:36.687236 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ResolutionWidth 2024.10.22 02:28:36.746282 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_962_327} MergeTask::MergeProjectionsStage: Merge sorted 14691 rows, containing 12 columns (12 merged, 0 gathered) in 1.629192912 sec., 9017.34833965445 rows/sec., 1.41 MiB/sec. 2024.10.22 02:28:36.793041 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 51e0a3dc9f3e305323c1d3153cda8a14, offset: 0, size: 0 2024.10.22 02:28:36.797887 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 51e0a3dc9f3e305323c1d3153cda8a14 for removal 2024.10.22 02:28:36.806357 [ 1476 ] {} CacheMetadata: Key 51e0a3dc9f3e305323c1d3153cda8a14 is removed from metadata 2024.10.22 02:28:36.874974 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 794 entries to flush up to offset 1379587 2024.10.22 02:28:36.882402 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_962_327} MergedBlockOutputStream: filled checksums 202410_1_962_327 (state Temporary) 2024.10.22 02:28:36.954244 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_902_267, 202410_903_903_0] 2024.10.22 02:28:36.955044 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_902_267, 202410_903_903_0] 2024.10.22 02:28:36.960193 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_902_267, 202410_903_903_0] 2024.10.22 02:28:37.361016 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1615_140} MergeTask::MergeProjectionsStage: Merge sorted 28260 rows, containing 13 columns (13 merged, 0 gathered) in 2.426364621 sec., 11647.054097068456 rows/sec., 2.24 MiB/sec. 2024.10.22 02:28:37.443626 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1615_140} MergedBlockOutputStream: filled checksums 202410_1207_1615_140 (state Temporary) 2024.10.22 02:28:37.607985 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1615_140} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1615_140 to 202410_1207_1615_140 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:37.639814 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1615_140} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1614_139, 202410_1615_1615_0] -> 202410_1207_1615_140 2024.10.22 02:28:37.674375 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:37.684765 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:37.765834 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.40 MiB. 2024.10.22 02:28:37.789815 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_962_327} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_962_327 to 202410_1_962_327 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:37.800453 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_962_327} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_961_326, 202410_962_962_0] -> 202410_1_962_327 2024.10.22 02:28:37.946841 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_913_913_0 (state Temporary) 2024.10.22 02:28:37.949227 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:28:38.034683 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 153 bytes for d33f47a68fa39034f09f58e9763016b1:0) 2024.10.22 02:28:38.038144 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 925cf71b1459b3e2c9fd119ecb5e1874, offset: 0, size: 152 2024.10.22 02:28:38.052416 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 925cf71b1459b3e2c9fd119ecb5e1874 for removal 2024.10.22 02:28:38.055508 [ 1476 ] {} CacheMetadata: Key 925cf71b1459b3e2c9fd119ecb5e1874 is removed from metadata 2024.10.22 02:28:38.113010 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_913_913_0 to 202410_920_920_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:38.245297 [ 1261 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_910_169 to 202410_911_916_1 2024.10.22 02:28:38.260338 [ 1261 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.08 MiB using storage policy from min volume index 0 2024.10.22 02:28:38.262636 [ 1261 ] {} DiskLocal: Reserved 1.08 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:38.274438 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_916_170} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_910_169 to 202410_911_916_1 into Wide with storage Full 2024.10.22 02:28:38.293682 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 7.48 MiB, will set to 1.55 GiB (RSS), difference: -2.56 MiB 2024.10.22 02:28:38.313225 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_916_170} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:38.348476 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1379587 2024.10.22 02:28:38.426496 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_916_170} MergeTreeSequentialSource: Reading 170 marks from part 202410_1_910_169, total 1381603 rows starting from the beginning of the part 2024.10.22 02:28:38.454603 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_916_170} MergeTreeSequentialSource: Reading 2 marks from part 202410_911_916_1, total 7315 rows starting from the beginning of the part 2024.10.22 02:28:38.914005 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 98 entries to flush up to offset 107796 2024.10.22 02:28:39.211525 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:39.215314 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:39.372462 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1607_1607_0 (state Temporary) 2024.10.22 02:28:39.466743 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1607_1607_0 to 202410_1620_1620_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:39.543428 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9eef52bb0772defdb86f61c72df7dc6c) : [0, 126918]: Setting download as failed: File segment: [0, 126918], key: 9eef52bb0772defdb86f61c72df7dc6c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:39.545307 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107796 2024.10.22 02:28:39.545576 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9eef52bb0772defdb86f61c72df7dc6c): No space left in cache to reserve 126919 bytes, will continue without cache download 2024.10.22 02:28:39.547543 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1615_140 to 202410_1616_1616_0 2024.10.22 02:28:39.557574 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9eef52bb0772defdb86f61c72df7dc6c, offset: 0, size: 0 2024.10.22 02:28:39.562246 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 02:28:39.575804 [ 1213 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:39.662488 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1616_141} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1615_140 to 202410_1616_1616_0 into Compact with storage Full 2024.10.22 02:28:39.663241 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9eef52bb0772defdb86f61c72df7dc6c for removal 2024.10.22 02:28:39.671030 [ 1476 ] {} CacheMetadata: Key 9eef52bb0772defdb86f61c72df7dc6c is removed from metadata 2024.10.22 02:28:39.695422 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1616_141} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:39.713512 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1616_141} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1615_140, total 28260 rows starting from the beginning of the part 2024.10.22 02:28:39.717231 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 161 entries to flush up to offset 104840 2024.10.22 02:28:39.744420 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1616_141} MergeTreeSequentialSource: Reading 2 marks from part 202410_1616_1616_0, total 73 rows starting from the beginning of the part 2024.10.22 02:28:40.085461 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:40.089297 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:40.156960 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_937_937_0 (state Temporary) 2024.10.22 02:28:40.165708 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1616_141/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:28:40.426901 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_937_937_0 to 202410_943_943_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:40.456577 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 104840 2024.10.22 02:28:40.461518 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.45 MiB, will set to 1.55 GiB (RSS), difference: -22.02 MiB 2024.10.22 02:28:40.555523 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_938_46 to 202410_939_939_0 2024.10.22 02:28:40.562134 [ 1254 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.21 MiB using storage policy from min volume index 0 2024.10.22 02:28:40.565588 [ 1254 ] {} DiskLocal: Reserved 1.21 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:40.693070 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_939_47} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_938_46 to 202410_939_939_0 into Compact with storage Full 2024.10.22 02:28:40.757030 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_939_47} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:40.762750 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_939_47} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_938_46, total 20730 rows starting from the beginning of the part 2024.10.22 02:28:40.796095 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_939_47} MergeTreeSequentialSource: Reading 2 marks from part 202410_939_939_0, total 161 rows starting from the beginning of the part 2024.10.22 02:28:40.798089 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1515_76, 202410_1516_1516_0] 2024.10.22 02:28:40.805081 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1515_76, 202410_1516_1516_0] 2024.10.22 02:28:40.819030 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1515_76, 202410_1516_1516_0] 2024.10.22 02:28:41.094706 [ 1164 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:28:41.139702 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 162 bytes for 1ea4a679a11caa83c889e663703038c4:0) 2024.10.22 02:28:41.142533 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a93f6d9db9ddf20aeedc191f5b90f46f, offset: 0, size: 150 2024.10.22 02:28:41.144865 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a93f6d9db9ddf20aeedc191f5b90f46f for removal 2024.10.22 02:28:41.146558 [ 1476 ] {} CacheMetadata: Key a93f6d9db9ddf20aeedc191f5b90f46f is removed from metadata 2024.10.22 02:28:41.252517 [ 1164 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:28:41.347060 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_916_170/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:28:41.397266 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:28:41.402568 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:28:41.863952 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:41.868782 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.28 GiB. 2024.10.22 02:28:41.921718 [ 1272 ] {} DatabaseCatalog: Execution took 352 ms. 2024.10.22 02:28:41.923283 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1616_141} MergeTask::MergeProjectionsStage: Merge sorted 28333 rows, containing 13 columns (13 merged, 0 gathered) in 2.332163473 sec., 12148.805316615984 rows/sec., 2.34 MiB/sec. 2024.10.22 02:28:42.106719 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1616_141} MergedBlockOutputStream: filled checksums 202410_1207_1616_141 (state Temporary) 2024.10.22 02:28:42.334258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(16bcbb472f023125f9ff1d7329ec5ea1) : [0, 60099]: Setting download as failed: File segment: [0, 60099], key: 16bcbb472f023125f9ff1d7329ec5ea1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:42.345895 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(16bcbb472f023125f9ff1d7329ec5ea1): No space left in cache to reserve 60100 bytes, will continue without cache download 2024.10.22 02:28:42.353619 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1616_141} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1616_141 to 202410_1207_1616_141 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:42.356411 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1616_141} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1615_140, 202410_1616_1616_0] -> 202410_1207_1616_141 2024.10.22 02:28:42.360375 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 16bcbb472f023125f9ff1d7329ec5ea1, offset: 0, size: 0 2024.10.22 02:28:42.365606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 16bcbb472f023125f9ff1d7329ec5ea1 for removal 2024.10.22 02:28:42.368121 [ 1476 ] {} CacheMetadata: Key 16bcbb472f023125f9ff1d7329ec5ea1 is removed from metadata 2024.10.22 02:28:42.374294 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.38 MiB. 2024.10.22 02:28:42.382784 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_939_47} MergeTask::MergeProjectionsStage: Merge sorted 20891 rows, containing 13 columns (13 merged, 0 gathered) in 1.805328966 sec., 11571.852218317534 rows/sec., 5.04 MiB/sec. 2024.10.22 02:28:42.504851 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 10.85 MiB, will set to 1.56 GiB (RSS), difference: 18.71 MiB 2024.10.22 02:28:42.557000 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_939_47} MergedBlockOutputStream: filled checksums 202410_747_939_47 (state Temporary) 2024.10.22 02:28:42.561201 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 14677 2024.10.22 02:28:42.714952 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 791 2024.10.22 02:28:42.741013 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_939_47} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_939_47 to 202410_747_939_47 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:42.749325 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_939_47} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_938_46, 202410_939_939_0] -> 202410_747_939_47 2024.10.22 02:28:42.780544 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:42.792955 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:42.839149 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_954_954_0 (state Temporary) 2024.10.22 02:28:42.900730 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_954_954_0 to 202410_967_967_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:42.904403 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.29 MiB. 2024.10.22 02:28:42.936514 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_962_327 to 202410_963_963_0 2024.10.22 02:28:42.944279 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:42.947038 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:43.064118 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14677 2024.10.22 02:28:43.122621 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_963_328} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_962_327 to 202410_963_963_0 into Compact with storage Full 2024.10.22 02:28:43.126707 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:43.134669 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:43.194790 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_773_773_0 (state Temporary) 2024.10.22 02:28:43.279053 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_963_328} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:43.296286 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_963_328} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_962_327, total 14691 rows starting from the beginning of the part 2024.10.22 02:28:43.312479 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_963_328} MergeTreeSequentialSource: Reading 2 marks from part 202410_963_963_0, total 25 rows starting from the beginning of the part 2024.10.22 02:28:43.332837 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 153 bytes for fef3bd9e7ee34ab676281dd2c57ddfb1:0) 2024.10.22 02:28:43.340955 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d33f47a68fa39034f09f58e9763016b1, offset: 0, size: 153 2024.10.22 02:28:43.344277 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d33f47a68fa39034f09f58e9763016b1 for removal 2024.10.22 02:28:43.346559 [ 1476 ] {} CacheMetadata: Key d33f47a68fa39034f09f58e9763016b1 is removed from metadata 2024.10.22 02:28:43.460182 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_773_773_0 to 202410_783_783_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:43.494938 [ 1263 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_778_158 to 202410_783_783_0 2024.10.22 02:28:43.504748 [ 1263 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:43.507141 [ 1263 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:43.645772 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 791 2024.10.22 02:28:43.648852 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_778_158 to 202410_783_783_0 into Compact with storage Full 2024.10.22 02:28:43.663883 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 89 entries to flush up to offset 107885 2024.10.22 02:28:43.837267 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:43.863729 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_778_158, total 822 rows starting from the beginning of the part 2024.10.22 02:28:44.061480 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} MergeTreeSequentialSource: Reading 2 marks from part 202410_779_779_0, total 1 rows starting from the beginning of the part 2024.10.22 02:28:44.195082 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_191_191_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:28:44.245156 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:44.248693 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} MergeTreeSequentialSource: Reading 2 marks from part 202410_780_780_0, total 1 rows starting from the beginning of the part 2024.10.22 02:28:44.249152 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:44.356901 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_963_328} MergeTask::MergeProjectionsStage: Merge sorted 14716 rows, containing 12 columns (12 merged, 0 gathered) in 1.240725898 sec., 11860.798604850272 rows/sec., 1.86 MiB/sec. 2024.10.22 02:28:44.483335 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(32c96398976f63fe9489a4df6521567f) : [0, 98836]: Setting download as failed: File segment: [0, 98836], key: 32c96398976f63fe9489a4df6521567f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:44.484897 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(32c96398976f63fe9489a4df6521567f): No space left in cache to reserve 98837 bytes, will continue without cache download 2024.10.22 02:28:44.502327 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 32c96398976f63fe9489a4df6521567f, offset: 0, size: 0 2024.10.22 02:28:44.506438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 32c96398976f63fe9489a4df6521567f for removal 2024.10.22 02:28:44.510845 [ 1476 ] {} CacheMetadata: Key 32c96398976f63fe9489a4df6521567f is removed from metadata 2024.10.22 02:28:44.583640 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1516_77, 202410_1517_1517_0] 2024.10.22 02:28:44.593656 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1516_77, 202410_1517_1517_0] 2024.10.22 02:28:44.604342 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1516_77, 202410_1517_1517_0] 2024.10.22 02:28:44.653133 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1608_1608_0 (state Temporary) 2024.10.22 02:28:44.659821 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} MergeTreeSequentialSource: Reading 2 marks from part 202410_781_781_0, total 1 rows starting from the beginning of the part 2024.10.22 02:28:44.660258 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_963_328} MergedBlockOutputStream: filled checksums 202410_1_963_328 (state Temporary) 2024.10.22 02:28:44.747106 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} MergeTreeSequentialSource: Reading 2 marks from part 202410_782_782_0, total 1 rows starting from the beginning of the part 2024.10.22 02:28:44.847568 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1608_1608_0 to 202410_1621_1621_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:44.840856 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} MergeTreeSequentialSource: Reading 2 marks from part 202410_783_783_0, total 1 rows starting from the beginning of the part 2024.10.22 02:28:44.982622 [ 1205 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1616_141 to 202410_1617_1617_0 2024.10.22 02:28:44.992241 [ 1205 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 02:28:44.996814 [ 1205 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:45.090543 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107885 2024.10.22 02:28:45.147053 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1617_142} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1616_141 to 202410_1617_1617_0 into Compact with storage Full 2024.10.22 02:28:45.161571 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1617_142} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:45.169748 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1617_142} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1616_141, total 28333 rows starting from the beginning of the part 2024.10.22 02:28:45.173029 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_963_328} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_963_328 to 202410_1_963_328 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:45.173523 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_903_268, 202410_904_904_0] 2024.10.22 02:28:45.177035 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_963_328} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_962_327, 202410_963_963_0] -> 202410_1_963_328 2024.10.22 02:28:45.177288 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_903_268, 202410_904_904_0] 2024.10.22 02:28:45.188010 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1617_142} MergeTreeSequentialSource: Reading 2 marks from part 202410_1617_1617_0, total 69 rows starting from the beginning of the part 2024.10.22 02:28:45.203710 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_903_268, 202410_904_904_0] 2024.10.22 02:28:45.287939 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 11.81 MiB, will set to 1.56 GiB (RSS), difference: -8.92 MiB 2024.10.22 02:28:45.288920 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:28:45.451071 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1516 entries to flush up to offset 1381103 2024.10.22 02:28:45.488480 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 163 bytes for 88b1c79011ea23a5eb9769ba1723db13:0) 2024.10.22 02:28:45.498917 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1ea4a679a11caa83c889e663703038c4, offset: 0, size: 162 2024.10.22 02:28:45.605542 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1ea4a679a11caa83c889e663703038c4 for removal 2024.10.22 02:28:45.610668 [ 1476 ] {} CacheMetadata: Key 1ea4a679a11caa83c889e663703038c4 is removed from metadata 2024.10.22 02:28:46.176657 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_191_191_0 (state Temporary) 2024.10.22 02:28:46.738329 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:46.741178 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:47.077410 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1617_142} MergeTask::MergeProjectionsStage: Merge sorted 28402 rows, containing 13 columns (13 merged, 0 gathered) in 1.944245328 sec., 14608.23878086233 rows/sec., 2.81 MiB/sec. 2024.10.22 02:28:47.108063 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_914_914_0 (state Temporary) 2024.10.22 02:28:47.110662 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(074c6188ba759a3c52bbf53be54577a9) : [0, 131681]: Setting download as failed: File segment: [0, 131681], key: 074c6188ba759a3c52bbf53be54577a9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:47.113030 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(074c6188ba759a3c52bbf53be54577a9): No space left in cache to reserve 131682 bytes, will continue without cache download 2024.10.22 02:28:47.163906 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1617_142} MergedBlockOutputStream: filled checksums 202410_1207_1617_142 (state Temporary) 2024.10.22 02:28:47.222866 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_914_914_0 to 202410_921_921_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:47.252363 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 074c6188ba759a3c52bbf53be54577a9, offset: 0, size: 0 2024.10.22 02:28:47.256154 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1381103 2024.10.22 02:28:47.253789 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 074c6188ba759a3c52bbf53be54577a9 for removal 2024.10.22 02:28:47.357412 [ 1476 ] {} CacheMetadata: Key 074c6188ba759a3c52bbf53be54577a9 is removed from metadata 2024.10.22 02:28:47.428850 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.68 MiB, will set to 1.56 GiB (RSS), difference: -2.24 MiB 2024.10.22 02:28:47.544487 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1617_142} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1617_142 to 202410_1207_1617_142 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:47.547424 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1617_142} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1616_141, 202410_1617_1617_0] -> 202410_1207_1617_142 2024.10.22 02:28:47.579084 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.39 MiB. 2024.10.22 02:28:47.924505 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} MergeTask::MergeProjectionsStage: Merge sorted 827 rows, containing 32 columns (32 merged, 0 gathered) in 4.28105291 sec., 193.17677622442653 rows/sec., 14.55 KiB/sec. 2024.10.22 02:28:48.072034 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 164 entries to flush up to offset 105004 2024.10.22 02:28:48.082849 [ 1396 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} MergedBlockOutputStream: filled checksums 202410_1_783_159 (state Temporary) 2024.10.22 02:28:48.424503 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 164 bytes for 0d4f7b184d04c0cb60d19f138b257a80:0) 2024.10.22 02:28:48.429372 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 88b1c79011ea23a5eb9769ba1723db13, offset: 0, size: 163 2024.10.22 02:28:48.438533 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 88b1c79011ea23a5eb9769ba1723db13 for removal 2024.10.22 02:28:48.441374 [ 1476 ] {} CacheMetadata: Key 88b1c79011ea23a5eb9769ba1723db13 is removed from metadata 2024.10.22 02:28:48.441638 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fef3bd9e7ee34ab676281dd2c57ddfb1, offset: 0, size: 153 2024.10.22 02:28:48.448572 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fef3bd9e7ee34ab676281dd2c57ddfb1 for removal 2024.10.22 02:28:48.480527 [ 1476 ] {} CacheMetadata: Key fef3bd9e7ee34ab676281dd2c57ddfb1 is removed from metadata 2024.10.22 02:28:48.667566 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:48.672858 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:48.747407 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_938_938_0 (state Temporary) 2024.10.22 02:28:48.826764 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_938_938_0 to 202410_944_944_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:48.963628 [ 1166 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_939_47 to 202410_940_940_0 2024.10.22 02:28:48.967068 [ 1166 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.22 MiB using storage policy from min volume index 0 2024.10.22 02:28:48.970065 [ 1166 ] {} DiskLocal: Reserved 1.22 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:48.972059 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 105004 2024.10.22 02:28:48.987653 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_940_48} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_939_47 to 202410_940_940_0 into Compact with storage Full 2024.10.22 02:28:49.098325 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 107966 2024.10.22 02:28:49.172592 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_783_159 to 202410_1_783_159 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:49.177570 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_778_158, 202410_783_783_0] -> 202410_1_783_159 2024.10.22 02:28:49.295950 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_940_48} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:49.313627 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:49.321500 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:49.327442 [ 1392 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_783_159}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 02:28:49.386382 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1609_1609_0 (state Temporary) 2024.10.22 02:28:49.437370 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_940_48} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_939_47, total 20891 rows starting from the beginning of the part 2024.10.22 02:28:49.451886 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:28:49.498157 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_940_48} MergeTreeSequentialSource: Reading 2 marks from part 202410_940_940_0, total 134 rows starting from the beginning of the part 2024.10.22 02:28:49.609512 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(83af34edba53fca37395bfe62747adf5) : [0, 142257]: Setting download as failed: File segment: [0, 142257], key: 83af34edba53fca37395bfe62747adf5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:49.611637 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(83af34edba53fca37395bfe62747adf5): No space left in cache to reserve 142258 bytes, will continue without cache download 2024.10.22 02:28:49.755793 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 83af34edba53fca37395bfe62747adf5, offset: 0, size: 0 2024.10.22 02:28:49.758293 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 83af34edba53fca37395bfe62747adf5 for removal 2024.10.22 02:28:49.761024 [ 1476 ] {} CacheMetadata: Key 83af34edba53fca37395bfe62747adf5 is removed from metadata 2024.10.22 02:28:49.765247 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1609_1609_0 to 202410_1622_1622_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:49.800713 [ 1190 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1617_142 to 202410_1618_1618_0 2024.10.22 02:28:49.823923 [ 1190 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 02:28:49.829267 [ 1190 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:49.831263 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 107966 2024.10.22 02:28:49.864065 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1618_143} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1617_142 to 202410_1618_1618_0 into Compact with storage Full 2024.10.22 02:28:49.908962 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1618_143} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:49.923914 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1618_143} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1617_142, total 28402 rows starting from the beginning of the part 2024.10.22 02:28:49.958428 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1618_143} MergeTreeSequentialSource: Reading 2 marks from part 202410_1618_1618_0, total 82 rows starting from the beginning of the part 2024.10.22 02:28:50.107575 [ 1165 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 02:28:50.509113 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.93 MiB, will set to 1.56 GiB (RSS), difference: -14.13 MiB 2024.10.22 02:28:50.567334 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 14699 2024.10.22 02:28:50.704486 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:50.715577 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:50.764397 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_191_191_0 to 202410_194_194_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:50.827693 [ 1179 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_185_189_1 to 202410_194_194_0 2024.10.22 02:28:50.842236 [ 1179 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:50.843826 [ 1179 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:50.867042 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} MergeTask::PrepareStage: Merging 6 parts: from 202410_185_189_1 to 202410_194_194_0 into Compact with storage Full 2024.10.22 02:28:50.882152 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_955_955_0 (state Temporary) 2024.10.22 02:28:50.937419 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_955_955_0 to 202410_968_968_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:50.971899 [ 1181 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_963_328 to 202410_964_964_0 2024.10.22 02:28:50.988514 [ 1181 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:50.995904 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:51.010964 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14699 2024.10.22 02:28:51.101647 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 163 bytes for 9a108c53d8a81511833193735407026f:0) 2024.10.22 02:28:51.103613 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_964_329} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_963_328 to 202410_964_964_0 into Compact with storage Full 2024.10.22 02:28:51.107030 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0d4f7b184d04c0cb60d19f138b257a80, offset: 0, size: 164 2024.10.22 02:28:51.123008 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0d4f7b184d04c0cb60d19f138b257a80 for removal 2024.10.22 02:28:51.132297 [ 1476 ] {} CacheMetadata: Key 0d4f7b184d04c0cb60d19f138b257a80 is removed from metadata 2024.10.22 02:28:51.162833 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): /var/lib/clickhouse/store/ffa/ffa745eb-8a28-47e2-b73e-5e0c8034f61b/tmp_merge_202410_1207_1618_143/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:28:51.180935 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_964_329} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:51.197362 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_964_329} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_963_328, total 14716 rows starting from the beginning of the part 2024.10.22 02:28:51.344465 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_964_329} MergeTreeSequentialSource: Reading 2 marks from part 202410_964_964_0, total 17 rows starting from the beginning of the part 2024.10.22 02:28:51.397599 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:28:51.401631 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:28:51.638831 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.19594712 sec. 2024.10.22 02:28:51.881132 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1618_143} MergeTask::MergeProjectionsStage: Merge sorted 28484 rows, containing 13 columns (13 merged, 0 gathered) in 2.025024878 sec., 14066.00003261787 rows/sec., 2.70 MiB/sec. 2024.10.22 02:28:51.948523 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_940_48} MergeTask::MergeProjectionsStage: Merge sorted 21025 rows, containing 13 columns (13 merged, 0 gathered) in 2.970714823 sec., 7077.4211772935305 rows/sec., 3.08 MiB/sec. 2024.10.22 02:28:52.138299 [ 1186 ] {} DatabaseCatalog: Execution took 215 ms. 2024.10.22 02:28:52.140669 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_964_329/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:28:52.142635 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(40c6771ea50bf9bc89904abab9bd3635) : [0, 76561]: Setting download as failed: File segment: [0, 76561], key: 40c6771ea50bf9bc89904abab9bd3635, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:52.145932 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(40c6771ea50bf9bc89904abab9bd3635): No space left in cache to reserve 76562 bytes, will continue without cache download 2024.10.22 02:28:52.156973 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_940_48} MergedBlockOutputStream: filled checksums 202410_747_940_48 (state Temporary) 2024.10.22 02:28:52.196780 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 40c6771ea50bf9bc89904abab9bd3635, offset: 0, size: 0 2024.10.22 02:28:52.204400 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1618_143} MergedBlockOutputStream: filled checksums 202410_1207_1618_143 (state Temporary) 2024.10.22 02:28:52.261350 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_904_269, 202410_905_905_0] 2024.10.22 02:28:52.264001 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_904_269, 202410_905_905_0] 2024.10.22 02:28:52.265912 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_940_48} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_940_48 to 202410_747_940_48 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:52.269695 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_940_48} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_939_47, 202410_940_940_0] -> 202410_747_940_48 2024.10.22 02:28:52.295704 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1618_143} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1618_143 to 202410_1207_1618_143 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:52.300337 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 40c6771ea50bf9bc89904abab9bd3635 for removal 2024.10.22 02:28:52.302142 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1618_143} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1617_142, 202410_1618_1618_0] -> 202410_1207_1618_143 2024.10.22 02:28:52.306947 [ 1476 ] {} CacheMetadata: Key 40c6771ea50bf9bc89904abab9bd3635 is removed from metadata 2024.10.22 02:28:52.311281 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_964_329} MergeTask::MergeProjectionsStage: Merge sorted 14733 rows, containing 12 columns (12 merged, 0 gathered) in 1.303473836 sec., 11302.873592930331 rows/sec., 1.77 MiB/sec. 2024.10.22 02:28:52.392206 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.28 MiB. 2024.10.22 02:28:52.449340 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.40 MiB. 2024.10.22 02:28:52.483206 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_904_269, 202410_905_905_0] 2024.10.22 02:28:52.565454 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_964_329} MergedBlockOutputStream: filled checksums 202410_1_964_329 (state Temporary) 2024.10.22 02:28:52.859046 [ 1176 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 02:28:52.890203 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1517_78, 202410_1518_1518_0] 2024.10.22 02:28:52.892718 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1517_78, 202410_1518_1518_0] 2024.10.22 02:28:52.902272 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1517_78, 202410_1518_1518_0] 2024.10.22 02:28:53.050854 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_964_329} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_964_329 to 202410_1_964_329 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:53.058613 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_964_329} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_963_328, 202410_964_964_0] -> 202410_1_964_329 2024.10.22 02:28:53.067305 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(d93abbf4ceb3ca3c2d4d866c6ce86046) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d93abbf4ceb3ca3c2d4d866c6ce86046, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 02:28:53.072472 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: d93abbf4ceb3ca3c2d4d866c6ce86046, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 02:28:53.079149 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:53.087527 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:28:53.246971 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_916_170/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:28:53.557487 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 6.63 MiB, will set to 1.55 GiB (RSS), difference: 22.37 MiB 2024.10.22 02:28:53.867868 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 108053 2024.10.22 02:28:54.167211 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 163 bytes for 93b4383f406a3f8a0db6819b63d0c0a4:0) 2024.10.22 02:28:54.169810 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9a108c53d8a81511833193735407026f, offset: 0, size: 163 2024.10.22 02:28:54.173807 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9a108c53d8a81511833193735407026f for removal 2024.10.22 02:28:54.177363 [ 1476 ] {} CacheMetadata: Key 9a108c53d8a81511833193735407026f is removed from metadata 2024.10.22 02:28:54.261466 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1382258 2024.10.22 02:28:54.348414 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7721 2024.10.22 02:28:54.407047 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:54.409784 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:54.480014 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1610_1610_0 (state Temporary) 2024.10.22 02:28:54.537905 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1610_1610_0 to 202410_1623_1623_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:54.603644 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 108053 2024.10.22 02:28:54.659002 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1618_143 to 202410_1619_1619_0 2024.10.22 02:28:54.667152 [ 1163 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 02:28:54.669232 [ 1163 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:54.792574 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1619_144} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1618_143 to 202410_1619_1619_0 into Compact with storage Full 2024.10.22 02:28:54.851821 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1619_144} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:54.862014 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1619_144} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1618_143, total 28484 rows starting from the beginning of the part 2024.10.22 02:28:54.881351 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1619_144} MergeTreeSequentialSource: Reading 2 marks from part 202410_1619_1619_0, total 73 rows starting from the beginning of the part 2024.10.22 02:28:55.088465 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(4a9bef799cab7ab747e19440c9b7a541) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4a9bef799cab7ab747e19440c9b7a541, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:28:55.113693 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 28232, file segment info: File segment: [0, 33554431], key: 4a9bef799cab7ab747e19440c9b7a541, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:28:55.118681 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:55.253006 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:55.310061 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:55.374467 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_850_159, 202410_851_856_1] 2024.10.22 02:28:55.376897 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_850_159, 202410_851_856_1] 2024.10.22 02:28:55.410625 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_915_915_0 (state Temporary) 2024.10.22 02:28:55.509307 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1518_79, 202410_1519_1519_0] 2024.10.22 02:28:55.511059 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1518_79, 202410_1519_1519_0] 2024.10.22 02:28:55.522208 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1518_79, 202410_1519_1519_0] 2024.10.22 02:28:55.582137 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_915_915_0 to 202410_922_922_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:55.594406 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_850_159, 202410_851_856_1] 2024.10.22 02:28:55.617527 [ 1214 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_917_917_0 to 202410_922_922_0 2024.10.22 02:28:55.627753 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1382258 2024.10.22 02:28:55.638745 [ 1214 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:55.643228 [ 1214 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:55.661185 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_917_917_0 to 202410_922_922_0 into Compact with storage Full 2024.10.22 02:28:55.784343 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:55.793150 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_917_917_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:28:55.813387 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_918_918_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:28:55.827033 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_919_919_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:28:55.846347 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_920_920_0, total 794 rows starting from the beginning of the part 2024.10.22 02:28:55.870413 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_921_921_0, total 1516 rows starting from the beginning of the part 2024.10.22 02:28:56.028785 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_922_922_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:28:56.260951 [ 1269 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:28:56.312582 [ 1269 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:28:56.396531 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(85e1cb497cd5abb27eaafea6a14aada7) : [0, 1060915]: Setting download as failed: File segment: [0, 1060915], key: 85e1cb497cd5abb27eaafea6a14aada7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:56.398443 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(85e1cb497cd5abb27eaafea6a14aada7): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:28:56.473390 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 85e1cb497cd5abb27eaafea6a14aada7, offset: 0, size: 0 2024.10.22 02:28:56.475970 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 85e1cb497cd5abb27eaafea6a14aada7 for removal 2024.10.22 02:28:56.480317 [ 1476 ] {} CacheMetadata: Key 85e1cb497cd5abb27eaafea6a14aada7 is removed from metadata 2024.10.22 02:28:56.590792 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_882_27, 202410_883_883_0, 202410_884_884_0, 202410_885_885_0, 202410_886_886_0, 202410_887_887_0] 2024.10.22 02:28:56.598067 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_882_27, 202410_883_883_0, 202410_884_884_0, 202410_885_885_0, 202410_886_886_0, 202410_887_887_0] 2024.10.22 02:28:56.617313 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 179 entries to flush up to offset 105183 2024.10.22 02:28:56.722160 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.45 MiB, will set to 1.56 GiB (RSS), difference: 3.88 MiB 2024.10.22 02:28:56.847503 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_882_27, 202410_883_883_0, 202410_884_884_0, 202410_885_885_0, 202410_886_886_0, 202410_887_887_0] 2024.10.22 02:28:56.892548 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:56.896680 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:56.900707 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1619_144} MergeTask::MergeProjectionsStage: Merge sorted 28557 rows, containing 13 columns (13 merged, 0 gathered) in 2.226996945 sec., 12823.097967922899 rows/sec., 2.47 MiB/sec. 2024.10.22 02:28:57.069367 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_939_939_0 (state Temporary) 2024.10.22 02:28:57.112960 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1619_144} MergedBlockOutputStream: filled checksums 202410_1207_1619_144 (state Temporary) 2024.10.22 02:28:57.235093 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_939_939_0 to 202410_945_945_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:57.261295 [ 1249 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_940_48 to 202410_941_941_0 2024.10.22 02:28:57.265621 [ 1249 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.22 MiB using storage policy from min volume index 0 2024.10.22 02:28:57.267625 [ 1249 ] {} DiskLocal: Reserved 1.22 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:57.324309 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 105183 2024.10.22 02:28:57.394893 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_941_49} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_940_48 to 202410_941_941_0 into Compact with storage Full 2024.10.22 02:28:57.455062 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_941_49} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:57.462573 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_941_49} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_940_48, total 21025 rows starting from the beginning of the part 2024.10.22 02:28:57.487844 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_941_49} MergeTreeSequentialSource: Reading 2 marks from part 202410_941_941_0, total 149 rows starting from the beginning of the part 2024.10.22 02:28:57.487123 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(0cee3b7c7a355743889b4bb0fb6ee9b5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0cee3b7c7a355743889b4bb0fb6ee9b5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 02:28:57.512448 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 764, file segment info: File segment: [0, 33554431], key: 0cee3b7c7a355743889b4bb0fb6ee9b5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1393, kind: Regular, unbound: 0 2024.10.22 02:28:57.517098 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:57.544153 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} MergeTask::MergeProjectionsStage: Merge sorted 6930 rows, containing 4 columns (4 merged, 0 gathered) in 1.855912195 sec., 3734.0128582968873 rows/sec., 58.34 KiB/sec. 2024.10.22 02:28:57.599286 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1619_144} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1619_144 to 202410_1207_1619_144 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:57.609020 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1619_144} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1618_143, 202410_1619_1619_0] -> 202410_1207_1619_144 2024.10.22 02:28:57.647873 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.40 MiB. 2024.10.22 02:28:57.725201 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 02:28:57.736558 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} MergedBlockOutputStream: filled checksums 202410_917_922_1 (state Temporary) 2024.10.22 02:28:57.763489 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 02:28:57.781041 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 02:28:57.788197 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_917_922_1 to 202410_917_922_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:57.796881 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 02:28:57.811370 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 02:28:57.823308 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 02:28:57.843384 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 02:28:57.862445 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 02:28:57.876463 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ResolutionHeight 2024.10.22 02:28:57.896897 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_917_922_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_917_917_0, 202410_922_922_0] -> 202410_917_922_1 2024.10.22 02:28:57.934306 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gou/oxbsgwszkkdwjgdbnjetmqygkpret (LogSeriesLimiter: on interval from 2024-10-22 02:28:34 to 2024-10-22 02:28:57 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:28:57.995500 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:28:58.515306 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 24 entries to flush up to offset 14723 2024.10.22 02:28:58.595913 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:58.606556 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(05b14d0a34aff77c556b4628ae63b131) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 05b14d0a34aff77c556b4628ae63b131, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:28:58.614973 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 93 entries to flush up to offset 108146 2024.10.22 02:28:58.622348 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 458, file segment info: File segment: [0, 33554431], key: 05b14d0a34aff77c556b4628ae63b131, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:28:58.628648 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:28:58.651001 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:58.665305 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 792 2024.10.22 02:28:58.693834 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_956_956_0 (state Temporary) 2024.10.22 02:28:58.863613 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:58.865793 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:58.922376 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_774_774_0 (state Temporary) 2024.10.22 02:28:58.948333 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_956_956_0 to 202410_969_969_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:58.986003 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_964_329 to 202410_965_965_0 2024.10.22 02:28:58.993766 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_941_49} MergeTask::MergeProjectionsStage: Merge sorted 21174 rows, containing 13 columns (13 merged, 0 gathered) in 1.60738971 sec., 13172.910009483636 rows/sec., 5.73 MiB/sec. 2024.10.22 02:28:58.996385 [ 1216 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:59.004033 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:59.005965 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14723 2024.10.22 02:28:59.126674 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_965_330} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_964_329 to 202410_965_965_0 into Compact with storage Full 2024.10.22 02:28:59.308533 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_965_330} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:28:59.321424 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_965_330} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_964_329, total 14733 rows starting from the beginning of the part 2024.10.22 02:28:59.347082 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_965_330} MergeTreeSequentialSource: Reading 2 marks from part 202410_965_965_0, total 12 rows starting from the beginning of the part 2024.10.22 02:28:59.360380 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_774_774_0 to 202410_784_784_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:59.556749 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 792 2024.10.22 02:28:59.557741 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:28:59.560975 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:28:59.564345 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_941_49} MergedBlockOutputStream: filled checksums 202410_747_941_49 (state Temporary) 2024.10.22 02:28:59.582634 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9a297de8072708b890e57796630ab058) : [0, 47829]: Setting download as failed: File segment: [0, 47829], key: 9a297de8072708b890e57796630ab058, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:28:59.584581 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 9.73 MiB, will set to 1.56 GiB (RSS), difference: -3.14 MiB 2024.10.22 02:28:59.586290 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9a297de8072708b890e57796630ab058): No space left in cache to reserve 47830 bytes, will continue without cache download 2024.10.22 02:28:59.626448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9a297de8072708b890e57796630ab058, offset: 0, size: 0 2024.10.22 02:28:59.629010 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9a297de8072708b890e57796630ab058 for removal 2024.10.22 02:28:59.634373 [ 1476 ] {} CacheMetadata: Key 9a297de8072708b890e57796630ab058 is removed from metadata 2024.10.22 02:28:59.693102 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_941_49} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_941_49 to 202410_747_941_49 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:28:59.697388 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_941_49} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_940_48, 202410_941_941_0] -> 202410_747_941_49 2024.10.22 02:28:59.838762 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.28 MiB. 2024.10.22 02:28:59.949517 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1519_80, 202410_1520_1520_0] 2024.10.22 02:28:59.951107 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1519_80, 202410_1520_1520_0] 2024.10.22 02:28:59.960680 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1611_1611_0 (state Temporary) 2024.10.22 02:28:59.962999 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1519_80, 202410_1520_1520_0] 2024.10.22 02:29:00.073234 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_965_330} MergeTask::MergeProjectionsStage: Merge sorted 14745 rows, containing 12 columns (12 merged, 0 gathered) in 0.95961194 sec., 15365.586218112292 rows/sec., 2.40 MiB/sec. 2024.10.22 02:29:00.127844 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_965_330} MergedBlockOutputStream: filled checksums 202410_1_965_330 (state Temporary) 2024.10.22 02:29:00.137542 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1611_1611_0 to 202410_1624_1624_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:00.163877 [ 1260 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1619_144 to 202410_1620_1620_0 2024.10.22 02:29:00.167189 [ 1260 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 02:29:00.172879 [ 1260 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:00.195007 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1620_145} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1619_144 to 202410_1620_1620_0 into Compact with storage Full 2024.10.22 02:29:00.254207 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1620_145} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:00.265057 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1620_145} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1619_144, total 28557 rows starting from the beginning of the part 2024.10.22 02:29:00.266261 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_965_330} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_965_330 to 202410_1_965_330 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:00.268754 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_965_330} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_964_329, 202410_965_965_0] -> 202410_1_965_330 2024.10.22 02:29:00.300825 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1620_145} MergeTreeSequentialSource: Reading 2 marks from part 202410_1620_1620_0, total 98 rows starting from the beginning of the part 2024.10.22 02:29:00.314401 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 108146 2024.10.22 02:29:00.403705 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:29:00.496999 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d71bbee2329e095903b68a9a6b57ebe2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d71bbee2329e095903b68a9a6b57ebe2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:29:00.511147 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 18728, file segment info: File segment: [0, 33554431], key: d71bbee2329e095903b68a9a6b57ebe2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:29:00.518813 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:01.401144 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:29:01.408298 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:29:01.477324 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 9687758b4d1783cf0545d78a89c868db:0) 2024.10.22 02:29:01.484763 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 93b4383f406a3f8a0db6819b63d0c0a4, offset: 0, size: 163 2024.10.22 02:29:01.488783 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 93b4383f406a3f8a0db6819b63d0c0a4 for removal 2024.10.22 02:29:01.493555 [ 1476 ] {} CacheMetadata: Key 93b4383f406a3f8a0db6819b63d0c0a4 is removed from metadata 2024.10.22 02:29:01.871853 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 7760 2024.10.22 02:29:02.041207 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:02.048952 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1620_145} MergeTask::MergeProjectionsStage: Merge sorted 28655 rows, containing 13 columns (13 merged, 0 gathered) in 1.864432491 sec., 15369.288047877084 rows/sec., 2.95 MiB/sec. 2024.10.22 02:29:02.138485 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_905_270, 202410_906_906_0] 2024.10.22 02:29:02.143593 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_905_270, 202410_906_906_0] 2024.10.22 02:29:02.152916 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_905_270, 202410_906_906_0] 2024.10.22 02:29:02.155611 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(0d321c9977a6c836625b458a86b279e4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0d321c9977a6c836625b458a86b279e4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:29:02.158463 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 564, file segment info: File segment: [0, 33554431], key: 0d321c9977a6c836625b458a86b279e4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1387, kind: Regular, unbound: 0 2024.10.22 02:29:02.161777 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:02.201055 [ 1341 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/vdz/mtxtstnbdlqyfjanyawibhysonspw, size 18728 2024.10.22 02:29:02.243838 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:29:02.262118 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:29:02.272053 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:29:02.382742 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:29:02.385419 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1620_145} MergedBlockOutputStream: filled checksums 202410_1207_1620_145 (state Temporary) 2024.10.22 02:29:02.429820 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:29:02.463163 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:29:02.492427 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTMCampaign 2024.10.22 02:29:02.570266 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1620_145} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1620_145 to 202410_1207_1620_145 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:02.573277 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1620_145} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1619_144, 202410_1620_1620_0] -> 202410_1207_1620_145 2024.10.22 02:29:02.634643 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1383413 2024.10.22 02:29:02.634869 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lvh/etrwdeczclkqxuvaqyiysokvkqdtl (LogSeriesLimiter: on interval from 2024-10-22 02:28:57 to 2024-10-22 02:29:02 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:29:02.674556 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 11.99 MiB, will set to 1.56 GiB (RSS), difference: 7.77 MiB 2024.10.22 02:29:02.815406 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.40 MiB. 2024.10.22 02:29:02.950504 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_185_189_1, total 209 rows starting from the beginning of the part 2024.10.22 02:29:03.111331 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(af6bac7a311ea80f4424ea5b2e0745f2) : [0, 41993]: Setting download as failed: File segment: [0, 41993], key: af6bac7a311ea80f4424ea5b2e0745f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:03.113365 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(af6bac7a311ea80f4424ea5b2e0745f2): No space left in cache to reserve 41994 bytes, will continue without cache download 2024.10.22 02:29:03.229760 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: af6bac7a311ea80f4424ea5b2e0745f2, offset: 0, size: 0 2024.10.22 02:29:03.233042 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key af6bac7a311ea80f4424ea5b2e0745f2 for removal 2024.10.22 02:29:03.240060 [ 1476 ] {} CacheMetadata: Key af6bac7a311ea80f4424ea5b2e0745f2 is removed from metadata 2024.10.22 02:29:03.467966 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:03.473800 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:03.707253 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_916_916_0 (state Temporary) 2024.10.22 02:29:03.771161 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_916_916_0 to 202410_923_923_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:03.803383 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1383413 2024.10.22 02:29:03.819834 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_916_170/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:29:04.368744 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 97 entries to flush up to offset 108243 2024.10.22 02:29:04.568889 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:04.572469 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:04.631108 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 10.08 MiB, will set to 1.55 GiB (RSS), difference: -8.83 MiB 2024.10.22 02:29:04.698910 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 148 bytes for 9e17f901aab7a17c9051bc2238dd1b4f:0) 2024.10.22 02:29:04.712772 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 556268c4d7729cf55dc5720e4b02eb67, offset: 0, size: 152 2024.10.22 02:29:04.726932 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 556268c4d7729cf55dc5720e4b02eb67 for removal 2024.10.22 02:29:04.740632 [ 1476 ] {} CacheMetadata: Key 556268c4d7729cf55dc5720e4b02eb67 is removed from metadata 2024.10.22 02:29:04.888208 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1612_1612_0 (state Temporary) 2024.10.22 02:29:04.912215 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_190_190_0, total 41 rows starting from the beginning of the part 2024.10.22 02:29:04.926805 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1520_81, 202410_1521_1521_0] 2024.10.22 02:29:04.930323 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1520_81, 202410_1521_1521_0] 2024.10.22 02:29:04.942386 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 158 entries to flush up to offset 105341 2024.10.22 02:29:05.149833 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1520_81, 202410_1521_1521_0] 2024.10.22 02:29:05.199631 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1612_1612_0 to 202410_1625_1625_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:05.241548 [ 1237 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1620_145 to 202410_1621_1621_0 2024.10.22 02:29:05.243769 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:05.250081 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:05.260356 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 108243 2024.10.22 02:29:05.251752 [ 1237 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 02:29:05.264140 [ 1237 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:05.271713 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1621_146} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1620_145 to 202410_1621_1621_0 into Compact with storage Full 2024.10.22 02:29:05.294628 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_940_940_0 (state Temporary) 2024.10.22 02:29:05.374849 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_940_940_0 to 202410_946_946_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:05.434435 [ 1166 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_941_49 to 202410_942_942_0 2024.10.22 02:29:05.442732 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1621_146} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:05.457557 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 105341 2024.10.22 02:29:05.458591 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1621_146} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1620_145, total 28655 rows starting from the beginning of the part 2024.10.22 02:29:05.460217 [ 1166 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.23 MiB using storage policy from min volume index 0 2024.10.22 02:29:05.465492 [ 1166 ] {} DiskLocal: Reserved 1.23 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:05.477264 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1621_146} MergeTreeSequentialSource: Reading 2 marks from part 202410_1621_1621_0, total 89 rows starting from the beginning of the part 2024.10.22 02:29:05.597687 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_942_50} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_941_49 to 202410_942_942_0 into Compact with storage Full 2024.10.22 02:29:05.675023 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_942_50} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:05.791941 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_942_50} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_941_49, total 21174 rows starting from the beginning of the part 2024.10.22 02:29:05.814581 [ 1253 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 02:29:05.826415 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_942_50} MergeTreeSequentialSource: Reading 2 marks from part 202410_942_942_0, total 150 rows starting from the beginning of the part 2024.10.22 02:29:05.941772 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(19a003997742cabbebe031212e72aa68) : [0, 78595]: Setting download as failed: File segment: [0, 78595], key: 19a003997742cabbebe031212e72aa68, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:05.946280 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(19a003997742cabbebe031212e72aa68): No space left in cache to reserve 78596 bytes, will continue without cache download 2024.10.22 02:29:06.073361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 19a003997742cabbebe031212e72aa68, offset: 0, size: 0 2024.10.22 02:29:06.078384 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 19a003997742cabbebe031212e72aa68 for removal 2024.10.22 02:29:06.084537 [ 1476 ] {} CacheMetadata: Key 19a003997742cabbebe031212e72aa68 is removed from metadata 2024.10.22 02:29:06.511725 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 14738 2024.10.22 02:29:06.656015 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 8.48 MiB, will set to 1.56 GiB (RSS), difference: -15.51 MiB 2024.10.22 02:29:06.762887 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:06.766970 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:06.950894 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_957_957_0 (state Temporary) 2024.10.22 02:29:06.966516 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_191_191_0, total 38 rows starting from the beginning of the part 2024.10.22 02:29:06.996659 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1621_146} MergeTask::MergeProjectionsStage: Merge sorted 28744 rows, containing 13 columns (13 merged, 0 gathered) in 1.727483226 sec., 16639.23537281282 rows/sec., 3.20 MiB/sec. 2024.10.22 02:29:07.062555 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 793 2024.10.22 02:29:07.063355 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_957_957_0 to 202410_970_970_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:07.120130 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1621_146} MergedBlockOutputStream: filled checksums 202410_1207_1621_146 (state Temporary) 2024.10.22 02:29:07.144893 [ 1190 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_965_330 to 202410_966_966_0 2024.10.22 02:29:07.163124 [ 1190 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:07.168459 [ 1190 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:07.230835 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_966_331} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_965_330 to 202410_966_966_0 into Compact with storage Full 2024.10.22 02:29:07.248755 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14738 2024.10.22 02:29:07.446541 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_966_331} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:07.472300 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_966_331} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_965_330, total 14745 rows starting from the beginning of the part 2024.10.22 02:29:07.548500 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_966_331} MergeTreeSequentialSource: Reading 2 marks from part 202410_966_966_0, total 20 rows starting from the beginning of the part 2024.10.22 02:29:07.567281 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1621_146} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1621_146 to 202410_1207_1621_146 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:07.571240 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1621_146} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1620_145, 202410_1621_1621_0] -> 202410_1207_1621_146 2024.10.22 02:29:07.637763 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.41 MiB. 2024.10.22 02:29:07.692617 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:07.704297 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:07.762808 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_942_50} MergeTask::MergeProjectionsStage: Merge sorted 21324 rows, containing 13 columns (13 merged, 0 gathered) in 2.191530202 sec., 9730.187601585241 rows/sec., 4.23 MiB/sec. 2024.10.22 02:29:07.863409 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_775_775_0 (state Temporary) 2024.10.22 02:29:07.868057 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 148 bytes for b217d3532a0059ec4c6b41a03a3ed3b4:0) 2024.10.22 02:29:07.952585 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_942_50} MergedBlockOutputStream: filled checksums 202410_747_942_50 (state Temporary) 2024.10.22 02:29:07.983145 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9687758b4d1783cf0545d78a89c868db, offset: 0, size: 152 2024.10.22 02:29:07.986719 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9687758b4d1783cf0545d78a89c868db for removal 2024.10.22 02:29:07.992172 [ 1476 ] {} CacheMetadata: Key 9687758b4d1783cf0545d78a89c868db is removed from metadata 2024.10.22 02:29:08.056226 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_775_775_0 to 202410_785_785_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:08.059810 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:29:08.153250 [ 1182 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 297 ms. 2024.10.22 02:29:08.260983 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 793 2024.10.22 02:29:08.555116 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_942_50} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_942_50 to 202410_747_942_50 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:08.595524 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_942_50} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_941_49, 202410_942_942_0] -> 202410_747_942_50 2024.10.22 02:29:08.732613 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.29 MiB. 2024.10.22 02:29:08.746218 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_966_331} MergeTask::MergeProjectionsStage: Merge sorted 14765 rows, containing 12 columns (12 merged, 0 gathered) in 1.514343678 sec., 9750.098484579272 rows/sec., 1.53 MiB/sec. 2024.10.22 02:29:08.987886 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_966_331} MergedBlockOutputStream: filled checksums 202410_1_966_331 (state Temporary) 2024.10.22 02:29:09.073926 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_966_331} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_966_331 to 202410_1_966_331 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:09.082261 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_966_331} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_965_330, 202410_966_966_0] -> 202410_1_966_331 2024.10.22 02:29:09.155352 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_192_192_0, total 43 rows starting from the beginning of the part 2024.10.22 02:29:09.282486 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 108326 2024.10.22 02:29:09.337491 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:29:09.512007 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1521_82, 202410_1522_1522_0] 2024.10.22 02:29:09.515843 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1521_82, 202410_1522_1522_0] 2024.10.22 02:29:09.531854 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1521_82, 202410_1522_1522_0] 2024.10.22 02:29:09.536758 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 10.56 MiB, will set to 1.56 GiB (RSS), difference: 17.01 MiB 2024.10.22 02:29:09.872991 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(18e2e78b400f9d693c2002452d7edaa6) : [0, 125155]: Setting download as failed: File segment: [0, 125155], key: 18e2e78b400f9d693c2002452d7edaa6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:09.874540 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(18e2e78b400f9d693c2002452d7edaa6): No space left in cache to reserve 125156 bytes, will continue without cache download 2024.10.22 02:29:09.992998 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:09.999334 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:10.001112 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 18e2e78b400f9d693c2002452d7edaa6, offset: 0, size: 0 2024.10.22 02:29:10.004453 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 18e2e78b400f9d693c2002452d7edaa6 for removal 2024.10.22 02:29:10.011370 [ 1476 ] {} CacheMetadata: Key 18e2e78b400f9d693c2002452d7edaa6 is removed from metadata 2024.10.22 02:29:10.265165 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1613_1613_0 (state Temporary) 2024.10.22 02:29:10.535735 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1613_1613_0 to 202410_1626_1626_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:10.559972 [ 1155 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1621_146 to 202410_1622_1622_0 2024.10.22 02:29:10.565315 [ 1155 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.39 MiB using storage policy from min volume index 0 2024.10.22 02:29:10.567553 [ 1155 ] {} DiskLocal: Reserved 1.39 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:10.596850 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 108326 2024.10.22 02:29:10.616891 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1622_147} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1621_146 to 202410_1622_1622_0 into Compact with storage Full 2024.10.22 02:29:10.802961 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1622_147} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:10.812135 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1384953 2024.10.22 02:29:10.820998 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_193_193_0, total 41 rows starting from the beginning of the part 2024.10.22 02:29:10.835412 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1622_147} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1621_146, total 28744 rows starting from the beginning of the part 2024.10.22 02:29:10.977441 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1622_147} MergeTreeSequentialSource: Reading 2 marks from part 202410_1622_1622_0, total 81 rows starting from the beginning of the part 2024.10.22 02:29:11.060605 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 153 bytes for 90504164664914a4f8b56ca2b0ddf057:0) 2024.10.22 02:29:11.064679 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9e17f901aab7a17c9051bc2238dd1b4f, offset: 0, size: 148 2024.10.22 02:29:11.070329 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9e17f901aab7a17c9051bc2238dd1b4f for removal 2024.10.22 02:29:11.073057 [ 1476 ] {} CacheMetadata: Key 9e17f901aab7a17c9051bc2238dd1b4f is removed from metadata 2024.10.22 02:29:11.317700 [ 1220 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:29:11.352827 [ 1220 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:29:11.401574 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:29:11.405584 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:29:11.886765 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 11.14 MiB, will set to 1.56 GiB (RSS), difference: -5.14 MiB 2024.10.22 02:29:11.944879 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1fb59b45a346c177a642e943641e0929) : [0, 97613]: Setting download as failed: File segment: [0, 97613], key: 1fb59b45a346c177a642e943641e0929, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:11.947256 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1fb59b45a346c177a642e943641e0929): No space left in cache to reserve 97614 bytes, will continue without cache download 2024.10.22 02:29:11.967691 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1fb59b45a346c177a642e943641e0929, offset: 0, size: 0 2024.10.22 02:29:11.969529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1fb59b45a346c177a642e943641e0929 for removal 2024.10.22 02:29:11.971527 [ 1476 ] {} CacheMetadata: Key 1fb59b45a346c177a642e943641e0929 is removed from metadata 2024.10.22 02:29:12.046437 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_906_271, 202410_907_907_0] 2024.10.22 02:29:12.049070 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_906_271, 202410_907_907_0] 2024.10.22 02:29:12.061336 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:12.062561 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_906_271, 202410_907_907_0] 2024.10.22 02:29:12.067667 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:12.386666 [ 1391 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_194_194_0, total 43 rows starting from the beginning of the part 2024.10.22 02:29:12.463040 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_917_917_0 (state Temporary) 2024.10.22 02:29:12.537501 [ 1212 ] {} DatabaseCatalog: Execution took 275 ms. 2024.10.22 02:29:12.590924 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_917_917_0 to 202410_924_924_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:12.653991 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1384953 2024.10.22 02:29:12.833448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 174 bytes for d7133c7420fa36a07010a9a0126acac7:0) 2024.10.22 02:29:12.840784 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 90504164664914a4f8b56ca2b0ddf057, offset: 0, size: 153 2024.10.22 02:29:12.846204 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 90504164664914a4f8b56ca2b0ddf057 for removal 2024.10.22 02:29:12.849295 [ 1476 ] {} CacheMetadata: Key 90504164664914a4f8b56ca2b0ddf057 is removed from metadata 2024.10.22 02:29:12.851404 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b217d3532a0059ec4c6b41a03a3ed3b4, offset: 0, size: 148 2024.10.22 02:29:12.856382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b217d3532a0059ec4c6b41a03a3ed3b4 for removal 2024.10.22 02:29:12.878708 [ 1476 ] {} CacheMetadata: Key b217d3532a0059ec4c6b41a03a3ed3b4 is removed from metadata 2024.10.22 02:29:12.972473 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 163 entries to flush up to offset 105504 2024.10.22 02:29:13.319927 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1622_147} MergeTask::MergeProjectionsStage: Merge sorted 28825 rows, containing 13 columns (13 merged, 0 gathered) in 2.690836503 sec., 10712.282209589157 rows/sec., 2.06 MiB/sec. 2024.10.22 02:29:13.660025 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:13.662245 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:13.713021 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_941_941_0 (state Temporary) 2024.10.22 02:29:13.750780 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1622_147} MergedBlockOutputStream: filled checksums 202410_1207_1622_147 (state Temporary) 2024.10.22 02:29:13.787507 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(06803a29c04b9061c276b01c520cc82d) : [0, 230193]: Setting download as failed: File segment: [0, 230193], key: 06803a29c04b9061c276b01c520cc82d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:13.793169 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(06803a29c04b9061c276b01c520cc82d): No space left in cache to reserve 230194 bytes, will continue without cache download 2024.10.22 02:29:13.806446 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_941_941_0 to 202410_947_947_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:13.822415 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 06803a29c04b9061c276b01c520cc82d, offset: 0, size: 0 2024.10.22 02:29:13.826781 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 06803a29c04b9061c276b01c520cc82d for removal 2024.10.22 02:29:13.832397 [ 1476 ] {} CacheMetadata: Key 06803a29c04b9061c276b01c520cc82d is removed from metadata 2024.10.22 02:29:13.843517 [ 1187 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_942_50 to 202410_943_943_0 2024.10.22 02:29:13.848534 [ 1187 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.24 MiB using storage policy from min volume index 0 2024.10.22 02:29:13.850179 [ 1187 ] {} DiskLocal: Reserved 1.24 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:13.876436 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_943_51} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_942_50 to 202410_943_943_0 into Compact with storage Full 2024.10.22 02:29:13.939068 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_943_51} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:13.953085 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_943_51} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_942_50, total 21324 rows starting from the beginning of the part 2024.10.22 02:29:13.970490 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 105504 2024.10.22 02:29:14.004521 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_943_51} MergeTreeSequentialSource: Reading 2 marks from part 202410_943_943_0, total 161 rows starting from the beginning of the part 2024.10.22 02:29:14.004982 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1622_147} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1622_147 to 202410_1207_1622_147 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:14.011411 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1622_147} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1621_146, 202410_1622_1622_0] -> 202410_1207_1622_147 2024.10.22 02:29:14.055274 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.42 MiB. 2024.10.22 02:29:14.549725 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_916_170/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:29:14.615251 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 108406 2024.10.22 02:29:14.652934 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(ae2ca864039261446abed7a25e9ecc29) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ae2ca864039261446abed7a25e9ecc29, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 02:29:14.658492 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: ae2ca864039261446abed7a25e9ecc29, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 02:29:14.660719 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:14.663357 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 15.48 MiB, will set to 1.56 GiB (RSS), difference: -7.89 MiB 2024.10.22 02:29:14.753307 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 14755 2024.10.22 02:29:14.960672 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:14.963698 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:15.023511 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:15.028054 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:15.051777 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1614_1614_0 (state Temporary) 2024.10.22 02:29:15.162254 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_958_958_0 (state Temporary) 2024.10.22 02:29:15.212605 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1522_83, 202410_1523_1523_0] 2024.10.22 02:29:15.212516 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_958_958_0 to 202410_971_971_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:15.213497 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1614_1614_0 to 202410_1627_1627_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:15.215198 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1522_83, 202410_1523_1523_0] 2024.10.22 02:29:15.232160 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1522_83, 202410_1523_1523_0] 2024.10.22 02:29:15.246060 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_966_331 to 202410_967_967_0 2024.10.22 02:29:15.250545 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:15.253013 [ 1228 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:15.265730 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1622_147 to 202410_1623_1623_0 2024.10.22 02:29:15.275155 [ 1221 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.39 MiB using storage policy from min volume index 0 2024.10.22 02:29:15.282923 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 108406 2024.10.22 02:29:15.296845 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_967_332} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_966_331 to 202410_967_967_0 into Compact with storage Full 2024.10.22 02:29:15.299313 [ 1221 ] {} DiskLocal: Reserved 1.39 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:15.344496 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1623_148} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1622_147 to 202410_1623_1623_0 into Compact with storage Full 2024.10.22 02:29:15.393458 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1623_148} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:15.394705 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_967_332} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:15.412892 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_967_332} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_966_331, total 14765 rows starting from the beginning of the part 2024.10.22 02:29:15.425352 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14755 2024.10.22 02:29:15.424593 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1623_148} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1622_147, total 28825 rows starting from the beginning of the part 2024.10.22 02:29:15.443538 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 163 bytes for 03639382dabcad3d213074071cfcdc99:0) 2024.10.22 02:29:15.446523 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d7133c7420fa36a07010a9a0126acac7, offset: 0, size: 174 2024.10.22 02:29:15.452327 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d7133c7420fa36a07010a9a0126acac7 for removal 2024.10.22 02:29:15.445402 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_943_51} MergeTask::MergeProjectionsStage: Merge sorted 21485 rows, containing 13 columns (13 merged, 0 gathered) in 1.579314946 sec., 13603.999667334245 rows/sec., 5.92 MiB/sec. 2024.10.22 02:29:15.455076 [ 1476 ] {} CacheMetadata: Key d7133c7420fa36a07010a9a0126acac7 is removed from metadata 2024.10.22 02:29:15.455896 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_967_332} MergeTreeSequentialSource: Reading 2 marks from part 202410_967_967_0, total 18 rows starting from the beginning of the part 2024.10.22 02:29:15.543712 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_943_51} MergedBlockOutputStream: filled checksums 202410_747_943_51 (state Temporary) 2024.10.22 02:29:15.566198 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1623_148} MergeTreeSequentialSource: Reading 2 marks from part 202410_1623_1623_0, total 87 rows starting from the beginning of the part 2024.10.22 02:29:15.763188 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 794 2024.10.22 02:29:15.828444 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_943_51} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_943_51 to 202410_747_943_51 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:15.837066 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_943_51} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_942_50, 202410_943_943_0] -> 202410_747_943_51 2024.10.22 02:29:15.939990 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.29 MiB. 2024.10.22 02:29:16.065684 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:16.071831 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:16.151517 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_776_776_0 (state Temporary) 2024.10.22 02:29:16.439437 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_776_776_0 to 202410_786_786_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:16.508718 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d56dff6b6297819f05fd116f97b601f8) : [0, 972185]: Setting download as failed: File segment: [0, 972185], key: d56dff6b6297819f05fd116f97b601f8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:16.511465 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d56dff6b6297819f05fd116f97b601f8): No space left in cache to reserve 972186 bytes, will continue without cache download 2024.10.22 02:29:16.516087 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 794 2024.10.22 02:29:16.650834 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d56dff6b6297819f05fd116f97b601f8, offset: 0, size: 0 2024.10.22 02:29:16.653578 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d56dff6b6297819f05fd116f97b601f8 for removal 2024.10.22 02:29:16.655639 [ 1476 ] {} CacheMetadata: Key d56dff6b6297819f05fd116f97b601f8 is removed from metadata 2024.10.22 02:29:16.676997 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_967_332} MergeTask::MergeProjectionsStage: Merge sorted 14783 rows, containing 12 columns (12 merged, 0 gathered) in 1.390965062 sec., 10627.872981039693 rows/sec., 1.66 MiB/sec. 2024.10.22 02:29:16.743168 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_967_332} MergedBlockOutputStream: filled checksums 202410_1_967_332 (state Temporary) 2024.10.22 02:29:17.104345 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_967_332} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_967_332 to 202410_1_967_332 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:17.109976 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_967_332} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_966_331, 202410_967_967_0] -> 202410_1_967_332 2024.10.22 02:29:17.145494 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:29:17.615111 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_907_272, 202410_908_908_0] 2024.10.22 02:29:17.622348 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_907_272, 202410_908_908_0] 2024.10.22 02:29:17.650325 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_907_272, 202410_908_908_0] 2024.10.22 02:29:17.810854 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 12.34 MiB, will set to 1.57 GiB (RSS), difference: 13.79 MiB 2024.10.22 02:29:17.836000 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1623_148} MergeTask::MergeProjectionsStage: Merge sorted 28912 rows, containing 13 columns (13 merged, 0 gathered) in 2.523020369 sec., 11459.281246888735 rows/sec., 2.20 MiB/sec. 2024.10.22 02:29:17.983112 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_733_149, 202410_734_734_0, 202410_735_735_0, 202410_736_736_0, 202410_737_737_0, 202410_738_738_0] 2024.10.22 02:29:17.989776 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_733_149, 202410_734_734_0, 202410_735_735_0, 202410_736_736_0, 202410_737_737_0, 202410_738_738_0] 2024.10.22 02:29:18.022305 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1623_148} MergedBlockOutputStream: filled checksums 202410_1207_1623_148 (state Temporary) 2024.10.22 02:29:18.037612 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_733_149, 202410_734_734_0, 202410_735_735_0, 202410_736_736_0, 202410_737_737_0, 202410_738_738_0] 2024.10.22 02:29:18.048396 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_916_170} MergeTask::MergeProjectionsStage: Merge sorted 1388918 rows, containing 4 columns (4 merged, 0 gathered) in 39.78027198 sec., 34914.743687481445 rows/sec., 523.62 KiB/sec. 2024.10.22 02:29:18.055417 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:18.059390 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.27 GiB. 2024.10.22 02:29:18.063784 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1623_148} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1623_148 to 202410_1207_1623_148 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:18.067702 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1623_148} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1622_147, 202410_1623_1623_0] -> 202410_1207_1623_148 2024.10.22 02:29:18.111282 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.63 MiB. 2024.10.22 02:29:18.144384 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fdd99b5fbec624325279598f3ec33787) : [0, 103744]: Setting download as failed: File segment: [0, 103744], key: fdd99b5fbec624325279598f3ec33787, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:18.147597 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fdd99b5fbec624325279598f3ec33787): No space left in cache to reserve 103745 bytes, will continue without cache download 2024.10.22 02:29:18.160520 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fdd99b5fbec624325279598f3ec33787, offset: 0, size: 0 2024.10.22 02:29:18.162610 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fdd99b5fbec624325279598f3ec33787 for removal 2024.10.22 02:29:18.164470 [ 1476 ] {} CacheMetadata: Key fdd99b5fbec624325279598f3ec33787 is removed from metadata 2024.10.22 02:29:18.221150 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_916_170} MergedBlockOutputStream: filled checksums 202410_1_916_170 (state Temporary) 2024.10.22 02:29:19.256635 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(7af33b90e5f8510b6268341c3ced8102) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7af33b90e5f8510b6268341c3ced8102, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:29:19.260381 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 767, file segment info: File segment: [0, 33554431], key: 7af33b90e5f8510b6268341c3ced8102, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:29:19.266228 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:19.311537 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 108488 2024.10.22 02:29:19.365828 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 14.06 MiB, will set to 1.56 GiB (RSS), difference: 10.25 MiB 2024.10.22 02:29:19.440429 [ 1741 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gou/oxbsgwszkkdwjgdbnjetmqygkpret, size 1300759 2024.10.22 02:29:19.600081 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:19.604535 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:19.658902 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1386108 2024.10.22 02:29:19.698483 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:29:19.712254 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:29:19.731738 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:29:19.748999 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:29:19.790760 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1615_1615_0 (state Temporary) 2024.10.22 02:29:19.859010 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:29:19.874781 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:29:19.876216 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1523_84, 202410_1524_1524_0] 2024.10.22 02:29:19.887163 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1523_84, 202410_1524_1524_0] 2024.10.22 02:29:19.908635 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1523_84, 202410_1524_1524_0] 2024.10.22 02:29:19.911117 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:29:19.947640 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:29:19.970961 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1615_1615_0 to 202410_1628_1628_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:19.988798 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column ResolutionDepth 2024.10.22 02:29:20.062976 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1623_148 to 202410_1624_1624_0 2024.10.22 02:29:20.080587 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ffc/mojvwohblvotahlymeocieorjchej (LogSeriesLimiter: on interval from 2024-10-22 02:29:02 to 2024-10-22 02:29:20 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:29:20.167415 [ 1263 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.40 MiB using storage policy from min volume index 0 2024.10.22 02:29:20.173730 [ 1263 ] {} DiskLocal: Reserved 1.40 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:20.197955 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1624_149} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1623_148 to 202410_1624_1624_0 into Compact with storage Full 2024.10.22 02:29:20.226679 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 108488 2024.10.22 02:29:20.273712 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1624_149} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:20.294082 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1624_149} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1623_148, total 28912 rows starting from the beginning of the part 2024.10.22 02:29:20.412843 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1624_149} MergeTreeSequentialSource: Reading 2 marks from part 202410_1624_1624_0, total 93 rows starting from the beginning of the part 2024.10.22 02:29:20.495763 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:20.500450 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:20.561114 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for a9583bb7a0cf4a1df2ed90a171929188:0) 2024.10.22 02:29:20.564644 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 03639382dabcad3d213074071cfcdc99, offset: 0, size: 163 2024.10.22 02:29:20.569326 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 03639382dabcad3d213074071cfcdc99 for removal 2024.10.22 02:29:20.572099 [ 1476 ] {} CacheMetadata: Key 03639382dabcad3d213074071cfcdc99 is removed from metadata 2024.10.22 02:29:20.653019 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_918_918_0 (state Temporary) 2024.10.22 02:29:20.742937 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_918_918_0 to 202410_925_925_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:20.767165 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1386108 2024.10.22 02:29:21.266255 [ 1241 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 02:29:21.405709 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:29:21.409464 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:29:21.503146 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 172 entries to flush up to offset 105676 2024.10.22 02:29:21.571488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fff89cfdc506fc34307f887a0dc50aa9) : [0, 3683]: Setting download as failed: File segment: [0, 3683], key: fff89cfdc506fc34307f887a0dc50aa9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:21.574826 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fff89cfdc506fc34307f887a0dc50aa9): No space left in cache to reserve 3684 bytes, will continue without cache download 2024.10.22 02:29:21.719127 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fff89cfdc506fc34307f887a0dc50aa9, offset: 0, size: 0 2024.10.22 02:29:21.727291 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fff89cfdc506fc34307f887a0dc50aa9 for removal 2024.10.22 02:29:21.731912 [ 1476 ] {} CacheMetadata: Key fff89cfdc506fc34307f887a0dc50aa9 is removed from metadata 2024.10.22 02:29:21.940673 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:21.944025 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:21.999312 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_942_942_0 (state Temporary) 2024.10.22 02:29:22.138059 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_192_192_0 (state Temporary) 2024.10.22 02:29:22.161033 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_942_942_0 to 202410_948_948_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:22.222854 [ 1222 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_943_51 to 202410_944_944_0 2024.10.22 02:29:22.240559 [ 1222 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.25 MiB using storage policy from min volume index 0 2024.10.22 02:29:22.245228 [ 1222 ] {} DiskLocal: Reserved 1.25 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:22.280808 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_944_52} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_943_51 to 202410_944_944_0 into Compact with storage Full 2024.10.22 02:29:22.313699 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 105676 2024.10.22 02:29:22.403532 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_944_52} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:22.442283 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_944_52} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_943_51, total 21485 rows starting from the beginning of the part 2024.10.22 02:29:22.543067 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 11.28 MiB, will set to 1.57 GiB (RSS), difference: -13.78 MiB 2024.10.22 02:29:22.559615 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_944_52} MergeTreeSequentialSource: Reading 2 marks from part 202410_944_944_0, total 164 rows starting from the beginning of the part 2024.10.22 02:29:22.661494 [ 1244 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 220 ms. 2024.10.22 02:29:22.883467 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_863_863_0, 202410_864_864_0, 202410_865_865_0, 202410_866_866_0, 202410_867_867_0, 202410_868_868_0] 2024.10.22 02:29:22.888775 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_863_863_0, 202410_864_864_0, 202410_865_865_0, 202410_866_866_0, 202410_867_867_0, 202410_868_868_0] 2024.10.22 02:29:22.918459 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_863_863_0, 202410_864_864_0, 202410_865_865_0, 202410_866_866_0, 202410_867_867_0, 202410_868_868_0] 2024.10.22 02:29:22.928591 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 14770 2024.10.22 02:29:22.972627 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 163 bytes for 374fb8780823c0f1754cc3b564c086f1:0) 2024.10.22 02:29:22.978438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a726410caeca61b416e6b23a17b4035a, offset: 0, size: 153 2024.10.22 02:29:22.989238 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a726410caeca61b416e6b23a17b4035a for removal 2024.10.22 02:29:22.996966 [ 1476 ] {} CacheMetadata: Key a726410caeca61b416e6b23a17b4035a is removed from metadata 2024.10.22 02:29:23.115322 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_916_170} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_916_170 to 202410_1_916_170 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:23.241236 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_916_170} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_910_169, 202410_911_916_1] -> 202410_1_916_170 2024.10.22 02:29:23.271261 [ 1398 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_916_170}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 02:29:23.294805 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:29:23.448656 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1624_149} MergeTask::MergeProjectionsStage: Merge sorted 29005 rows, containing 13 columns (13 merged, 0 gathered) in 3.261274712 sec., 8893.761661130497 rows/sec., 1.71 MiB/sec. 2024.10.22 02:29:23.537182 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1624_149} MergedBlockOutputStream: filled checksums 202410_1207_1624_149 (state Temporary) 2024.10.22 02:29:23.551161 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:23.555284 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:23.689679 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_959_959_0 (state Temporary) 2024.10.22 02:29:23.723350 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1624_149} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1624_149 to 202410_1207_1624_149 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:23.742312 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1624_149} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1623_148, 202410_1624_1624_0] -> 202410_1207_1624_149 2024.10.22 02:29:23.762654 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.62 MiB. 2024.10.22 02:29:23.832620 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(41468ec3a4e6b8c2af71ccc338148c36) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 41468ec3a4e6b8c2af71ccc338148c36, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:29:23.943594 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 53467, file segment info: File segment: [0, 33554431], key: 41468ec3a4e6b8c2af71ccc338148c36, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:29:23.947708 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_959_959_0 to 202410_972_972_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:23.948585 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:23.969340 [ 1154 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_967_332 to 202410_968_968_0 2024.10.22 02:29:23.988285 [ 1154 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:23.991577 [ 1154 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:24.030988 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 795 2024.10.22 02:29:24.049939 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_968_333} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_967_332 to 202410_968_968_0 into Compact with storage Full 2024.10.22 02:29:24.053486 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14770 2024.10.22 02:29:24.075675 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_968_333} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:24.189556 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_968_333} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_967_332, total 14783 rows starting from the beginning of the part 2024.10.22 02:29:24.246451 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 94 entries to flush up to offset 108582 2024.10.22 02:29:24.304892 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:24.309080 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_968_333} MergeTreeSequentialSource: Reading 2 marks from part 202410_968_968_0, total 22 rows starting from the beginning of the part 2024.10.22 02:29:24.310729 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:24.325511 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1524_85, 202410_1525_1525_0] 2024.10.22 02:29:24.328566 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1524_85, 202410_1525_1525_0] 2024.10.22 02:29:24.342968 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1524_85, 202410_1525_1525_0] 2024.10.22 02:29:24.378196 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_777_777_0 (state Temporary) 2024.10.22 02:29:24.468612 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_777_777_0 to 202410_787_787_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:24.555321 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 795 2024.10.22 02:29:24.566892 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:24.569827 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:24.654844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1f7f64911c2ecb7499f158e5503426b8) : [0, 1041537]: Setting download as failed: File segment: [0, 1041537], key: 1f7f64911c2ecb7499f158e5503426b8, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:24.657286 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1f7f64911c2ecb7499f158e5503426b8): No space left in cache to reserve 1041538 bytes, will continue without cache download 2024.10.22 02:29:24.781397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1f7f64911c2ecb7499f158e5503426b8, offset: 0, size: 0 2024.10.22 02:29:24.785070 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1f7f64911c2ecb7499f158e5503426b8 for removal 2024.10.22 02:29:24.794794 [ 1476 ] {} CacheMetadata: Key 1f7f64911c2ecb7499f158e5503426b8 is removed from metadata 2024.10.22 02:29:24.842870 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_968_333} MergeTask::MergeProjectionsStage: Merge sorted 14805 rows, containing 12 columns (12 merged, 0 gathered) in 0.799261546 sec., 18523.348300807655 rows/sec., 2.90 MiB/sec. 2024.10.22 02:29:24.850978 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1616_1616_0 (state Temporary) 2024.10.22 02:29:24.909151 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1616_1616_0 to 202410_1629_1629_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:24.947822 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1207_1624_149 to 202410_1625_1625_0 2024.10.22 02:29:24.964773 [ 1161 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.40 MiB using storage policy from min volume index 0 2024.10.22 02:29:24.972910 [ 1161 ] {} DiskLocal: Reserved 1.40 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:24.996057 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_968_333} MergedBlockOutputStream: filled checksums 202410_1_968_333 (state Temporary) 2024.10.22 02:29:25.073812 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 108582 2024.10.22 02:29:25.090574 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_192_192_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:29:25.107311 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1625_150} MergeTask::PrepareStage: Merging 2 parts: from 202410_1207_1624_149 to 202410_1625_1625_0 into Compact with storage Full 2024.10.22 02:29:25.151430 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1625_150} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:25.157216 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1625_150} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1624_149, total 29005 rows starting from the beginning of the part 2024.10.22 02:29:25.192011 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1625_150} MergeTreeSequentialSource: Reading 2 marks from part 202410_1625_1625_0, total 97 rows starting from the beginning of the part 2024.10.22 02:29:25.204695 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_944_52} MergeTask::MergeProjectionsStage: Merge sorted 21649 rows, containing 13 columns (13 merged, 0 gathered) in 2.941573179 sec., 7359.6673217423295 rows/sec., 3.20 MiB/sec. 2024.10.22 02:29:25.256569 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_968_333} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_968_333 to 202410_1_968_333 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:25.260833 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_968_333} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_967_332, 202410_968_968_0] -> 202410_1_968_333 2024.10.22 02:29:25.389546 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:29:25.501418 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_944_52} MergedBlockOutputStream: filled checksums 202410_747_944_52 (state Temporary) 2024.10.22 02:29:25.762563 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(506866258ca00de6ded9f6cbdb481853) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 506866258ca00de6ded9f6cbdb481853, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:29:25.764588 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 464, file segment info: File segment: [0, 33554431], key: 506866258ca00de6ded9f6cbdb481853, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:29:25.766431 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:25.927176 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_944_52} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_944_52 to 202410_747_944_52 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:25.932982 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_944_52} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_943_51, 202410_944_944_0] -> 202410_747_944_52 2024.10.22 02:29:25.955480 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.54 MiB. 2024.10.22 02:29:26.357112 [ 1172 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:29:26.385534 [ 1172 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:29:26.570544 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1625_150} MergeTask::MergeProjectionsStage: Merge sorted 29102 rows, containing 13 columns (13 merged, 0 gathered) in 1.476847279 sec., 19705.490482201716 rows/sec., 3.79 MiB/sec. 2024.10.22 02:29:26.621593 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 5cd1d3f22225ddcfc32dbd49380d8a7f:0) 2024.10.22 02:29:26.625700 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a9583bb7a0cf4a1df2ed90a171929188, offset: 0, size: 152 2024.10.22 02:29:26.631568 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a9583bb7a0cf4a1df2ed90a171929188 for removal 2024.10.22 02:29:26.637739 [ 1476 ] {} CacheMetadata: Key a9583bb7a0cf4a1df2ed90a171929188 is removed from metadata 2024.10.22 02:29:26.760875 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 9.43 MiB, will set to 1.57 GiB (RSS), difference: 8.28 MiB 2024.10.22 02:29:26.857749 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1625_150} MergedBlockOutputStream: filled checksums 202410_1207_1625_150 (state Temporary) 2024.10.22 02:29:27.045406 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1625_150} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1207_1625_150 to 202410_1207_1625_150 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:27.048920 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1207_1625_150} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1207_1624_149, 202410_1625_1625_0] -> 202410_1207_1625_150 2024.10.22 02:29:27.067605 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 15.62 MiB. 2024.10.22 02:29:27.468585 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(8d71a2588984941560f87ef28eb4049c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8d71a2588984941560f87ef28eb4049c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:29:27.470615 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 17914, file segment info: File segment: [0, 33554431], key: 8d71a2588984941560f87ef28eb4049c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:29:27.472127 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:27.728196 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_908_273, 202410_909_909_0] 2024.10.22 02:29:27.742406 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_908_273, 202410_909_909_0] 2024.10.22 02:29:27.754612 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_908_273, 202410_909_909_0] 2024.10.22 02:29:27.772981 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1386878 2024.10.22 02:29:28.367933 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1525_86, 202410_1526_1526_0] 2024.10.22 02:29:28.375143 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1525_86, 202410_1526_1526_0] 2024.10.22 02:29:28.393461 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1525_86, 202410_1526_1526_0] 2024.10.22 02:29:28.566728 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4a2d9c5e435c10409022ee811b61ad2f) : [0, 66225]: Setting download as failed: File segment: [0, 66225], key: 4a2d9c5e435c10409022ee811b61ad2f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:28.567871 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4a2d9c5e435c10409022ee811b61ad2f): No space left in cache to reserve 66226 bytes, will continue without cache download 2024.10.22 02:29:28.585229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4a2d9c5e435c10409022ee811b61ad2f, offset: 0, size: 0 2024.10.22 02:29:28.587953 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4a2d9c5e435c10409022ee811b61ad2f for removal 2024.10.22 02:29:28.592410 [ 1476 ] {} CacheMetadata: Key 4a2d9c5e435c10409022ee811b61ad2f is removed from metadata 2024.10.22 02:29:28.641712 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:28.643957 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:28.749642 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_919_919_0 (state Temporary) 2024.10.22 02:29:28.844150 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_919_919_0 to 202410_926_926_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:28.871815 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1386878 2024.10.22 02:29:29.085479 [ 1258 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_916_170 to 202410_917_922_1 2024.10.22 02:29:29.092071 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 108655 2024.10.22 02:29:29.095104 [ 1258 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.08 MiB using storage policy from min volume index 0 2024.10.22 02:29:29.097458 [ 1258 ] {} DiskLocal: Reserved 1.08 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:29.113216 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_922_171} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_916_170 to 202410_917_922_1 into Wide with storage Full 2024.10.22 02:29:29.151299 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_922_171} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:29.158950 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_922_171} MergeTreeSequentialSource: Reading 171 marks from part 202410_1_916_170, total 1388918 rows starting from the beginning of the part 2024.10.22 02:29:29.205156 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_922_171} MergeTreeSequentialSource: Reading 2 marks from part 202410_917_922_1, total 6930 rows starting from the beginning of the part 2024.10.22 02:29:29.202751 [ 1258 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 239 ms. 2024.10.22 02:29:29.365806 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(abbfd2211a734b39f4a2a201a4274916) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: abbfd2211a734b39f4a2a201a4274916, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:29:29.367484 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 559, file segment info: File segment: [0, 33554431], key: abbfd2211a734b39f4a2a201a4274916, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1391, kind: Regular, unbound: 0 2024.10.22 02:29:29.375932 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:29.381789 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:29.370088 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:29.526560 [ 1725 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lvh/etrwdeczclkqxuvaqyiysokvkqdtl, size 17914 2024.10.22 02:29:29.551355 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1617_1617_0 (state Temporary) 2024.10.22 02:29:29.614141 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:29:29.650935 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:29:29.663028 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:29:29.698292 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:29:29.733245 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:29:29.760401 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:29:29.828107 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1617_1617_0 to 202410_1630_1630_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:29.850803 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 163 entries to flush up to offset 105839 2024.10.22 02:29:29.856237 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 2 parts from 202410_1_1206_506 to 202410_1207_1625_150 2024.10.22 02:29:29.884801 [ 1184 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 4.33 MiB using storage policy from min volume index 0 2024.10.22 02:29:29.888966 [ 1184 ] {} DiskLocal: Reserved 4.33 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:29.944441 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_192_192_0 to 202410_195_195_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:29.973431 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTMContent 2024.10.22 02:29:29.990293 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 108655 2024.10.22 02:29:30.028156 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1625_507} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1206_506 to 202410_1207_1625_150 into Wide with storage Full 2024.10.22 02:29:30.046732 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ssb/rrrypdmpuyyszhycnypbgawulosks (LogSeriesLimiter: on interval from 2024-10-22 02:29:20 to 2024-10-22 02:29:30 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:29:30.061480 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1625_507} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:30.078264 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1625_507} MergeTreeSequentialSource: Reading 11 marks from part 202410_1_1206_506, total 79972 rows starting from the beginning of the part 2024.10.22 02:29:30.255456 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1625_507} MergeTreeSequentialSource: Reading 5 marks from part 202410_1207_1625_150, total 29102 rows starting from the beginning of the part 2024.10.22 02:29:30.350956 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_922_171/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:29:30.511413 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 11.84 MiB, will set to 1.56 GiB (RSS), difference: -18.62 MiB 2024.10.22 02:29:30.648560 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:30.652142 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:30.789780 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for ac7b7813a0d3c845e8b84a0cb495dd4a:0) 2024.10.22 02:29:30.797808 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 374fb8780823c0f1754cc3b564c086f1, offset: 0, size: 163 2024.10.22 02:29:30.800173 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_943_943_0 (state Temporary) 2024.10.22 02:29:30.808582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 374fb8780823c0f1754cc3b564c086f1 for removal 2024.10.22 02:29:30.816192 [ 1476 ] {} CacheMetadata: Key 374fb8780823c0f1754cc3b564c086f1 is removed from metadata 2024.10.22 02:29:30.852710 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_943_943_0 to 202410_949_949_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:30.877388 [ 1235 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_944_52 to 202410_945_945_0 2024.10.22 02:29:30.896572 [ 1235 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.25 MiB using storage policy from min volume index 0 2024.10.22 02:29:30.902388 [ 1235 ] {} DiskLocal: Reserved 1.25 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:30.972142 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 105839 2024.10.22 02:29:31.038644 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_945_53} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_944_52 to 202410_945_945_0 into Compact with storage Full 2024.10.22 02:29:31.070051 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_945_53} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:31.083687 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_945_53} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_944_52, total 21649 rows starting from the beginning of the part 2024.10.22 02:29:31.114882 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_945_53} MergeTreeSequentialSource: Reading 2 marks from part 202410_945_945_0, total 179 rows starting from the beginning of the part 2024.10.22 02:29:31.407851 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:29:31.410990 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:29:31.558170 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 22 entries to flush up to offset 14792 2024.10.22 02:29:31.665750 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:31.670881 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:31.728644 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_960_960_0 (state Temporary) 2024.10.22 02:29:31.793542 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_960_960_0 to 202410_973_973_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:31.854908 [ 1210 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_968_333 to 202410_969_969_0 2024.10.22 02:29:31.859145 [ 1210 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:31.860268 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14792 2024.10.22 02:29:31.862524 [ 1210 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:31.872287 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_969_334} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_968_333 to 202410_969_969_0 into Compact with storage Full 2024.10.22 02:29:31.964488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(93bdc7f095e02acf1ac1dbd7783dd0fc) : [0, 58110]: Setting download as failed: File segment: [0, 58110], key: 93bdc7f095e02acf1ac1dbd7783dd0fc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:31.965761 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(93bdc7f095e02acf1ac1dbd7783dd0fc): No space left in cache to reserve 58111 bytes, will continue without cache download 2024.10.22 02:29:31.980689 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 93bdc7f095e02acf1ac1dbd7783dd0fc, offset: 0, size: 0 2024.10.22 02:29:31.986523 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 93bdc7f095e02acf1ac1dbd7783dd0fc for removal 2024.10.22 02:29:32.048384 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_969_334} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:32.063685 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 796 2024.10.22 02:29:32.097020 [ 1476 ] {} CacheMetadata: Key 93bdc7f095e02acf1ac1dbd7783dd0fc is removed from metadata 2024.10.22 02:29:32.109772 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_969_334} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_968_333, total 14805 rows starting from the beginning of the part 2024.10.22 02:29:32.339020 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_969_334} MergeTreeSequentialSource: Reading 2 marks from part 202410_969_969_0, total 24 rows starting from the beginning of the part 2024.10.22 02:29:32.350824 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_945_53} MergeTask::MergeProjectionsStage: Merge sorted 21828 rows, containing 13 columns (13 merged, 0 gathered) in 1.343544212 sec., 16246.581098739458 rows/sec., 7.07 MiB/sec. 2024.10.22 02:29:32.460276 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_945_53} MergedBlockOutputStream: filled checksums 202410_747_945_53 (state Temporary) 2024.10.22 02:29:32.476633 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:32.485998 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:32.523904 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 15.69 MiB, will set to 1.58 GiB (RSS), difference: -32.65 MiB 2024.10.22 02:29:32.556479 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_778_778_0 (state Temporary) 2024.10.22 02:29:32.661136 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7760 2024.10.22 02:29:32.750454 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_945_53} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_945_53 to 202410_747_945_53 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:32.764472 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_945_53} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_944_52, 202410_945_945_0] -> 202410_747_945_53 2024.10.22 02:29:32.823955 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 23.54 MiB. 2024.10.22 02:29:32.837972 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_778_778_0 to 202410_788_788_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:32.863371 [ 1245 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_783_159 to 202410_788_788_0 2024.10.22 02:29:32.877614 [ 1245 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:32.885490 [ 1245 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:32.902885 [ 1227 ] {} DatabaseCatalog: Execution took 238 ms. 2024.10.22 02:29:32.912254 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_783_159 to 202410_788_788_0 into Compact with storage Full 2024.10.22 02:29:32.915122 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 796 2024.10.22 02:29:33.465676 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_969_334} MergeTask::MergeProjectionsStage: Merge sorted 14829 rows, containing 12 columns (12 merged, 0 gathered) in 1.598831592 sec., 9274.898040668688 rows/sec., 1.45 MiB/sec. 2024.10.22 02:29:33.487381 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:33.514529 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_783_159, total 827 rows starting from the beginning of the part 2024.10.22 02:29:33.578820 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} MergeTreeSequentialSource: Reading 2 marks from part 202410_784_784_0, total 1 rows starting from the beginning of the part 2024.10.22 02:29:33.671660 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for a3a68a1158d11da6e3144dc6312227ff:0) 2024.10.22 02:29:33.674332 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5cd1d3f22225ddcfc32dbd49380d8a7f, offset: 0, size: 152 2024.10.22 02:29:33.680826 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5cd1d3f22225ddcfc32dbd49380d8a7f for removal 2024.10.22 02:29:33.685653 [ 1476 ] {} CacheMetadata: Key 5cd1d3f22225ddcfc32dbd49380d8a7f is removed from metadata 2024.10.22 02:29:33.746666 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_969_334} MergedBlockOutputStream: filled checksums 202410_1_969_334 (state Temporary) 2024.10.22 02:29:33.762080 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} MergeTreeSequentialSource: Reading 2 marks from part 202410_785_785_0, total 1 rows starting from the beginning of the part 2024.10.22 02:29:33.809599 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} MergeTreeSequentialSource: Reading 2 marks from part 202410_786_786_0, total 1 rows starting from the beginning of the part 2024.10.22 02:29:33.824455 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_969_334} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_969_334 to 202410_1_969_334 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:33.842919 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_969_334} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_968_333, 202410_969_969_0] -> 202410_1_969_334 2024.10.22 02:29:33.876806 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:29:33.944628 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} MergeTreeSequentialSource: Reading 2 marks from part 202410_787_787_0, total 1 rows starting from the beginning of the part 2024.10.22 02:29:33.973365 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} MergeTreeSequentialSource: Reading 2 marks from part 202410_788_788_0, total 1 rows starting from the beginning of the part 2024.10.22 02:29:34.005979 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 107 entries to flush up to offset 108762 2024.10.22 02:29:34.440532 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:34.443398 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:34.665760 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_909_274, 202410_910_910_0] 2024.10.22 02:29:34.667256 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_909_274, 202410_910_910_0] 2024.10.22 02:29:34.679227 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_909_274, 202410_910_910_0] 2024.10.22 02:29:34.769198 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1526_87, 202410_1527_1527_0] 2024.10.22 02:29:34.772002 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1526_87, 202410_1527_1527_0] 2024.10.22 02:29:34.772351 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1618_1618_0 (state Temporary) 2024.10.22 02:29:34.788284 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1526_87, 202410_1527_1527_0] 2024.10.22 02:29:34.834732 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1618_1618_0 to 202410_1631_1631_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:34.878673 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1626_0 to 202410_1631_1631_0 2024.10.22 02:29:34.894700 [ 1268 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:34.913497 [ 1268 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:34.937707 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ccbcda4de2de3524fc87a20fe1ea8f73) : [0, 58285]: Setting download as failed: File segment: [0, 58285], key: ccbcda4de2de3524fc87a20fe1ea8f73, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:34.939229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ccbcda4de2de3524fc87a20fe1ea8f73): No space left in cache to reserve 58286 bytes, will continue without cache download 2024.10.22 02:29:34.944236 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1626_0 to 202410_1631_1631_0 into Compact with storage Full 2024.10.22 02:29:34.965841 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ccbcda4de2de3524fc87a20fe1ea8f73, offset: 0, size: 0 2024.10.22 02:29:34.968977 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ccbcda4de2de3524fc87a20fe1ea8f73 for removal 2024.10.22 02:29:34.973750 [ 1476 ] {} CacheMetadata: Key ccbcda4de2de3524fc87a20fe1ea8f73 is removed from metadata 2024.10.22 02:29:35.012308 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 108762 2024.10.22 02:29:35.095499 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:35.113061 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1626_0, total 83 rows starting from the beginning of the part 2024.10.22 02:29:35.371755 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1627_1627_0, total 80 rows starting from the beginning of the part 2024.10.22 02:29:35.399571 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 18.08 MiB, will set to 1.58 GiB (RSS), difference: 7.88 MiB 2024.10.22 02:29:35.565445 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1628_1628_0, total 82 rows starting from the beginning of the part 2024.10.22 02:29:35.614426 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1629_1629_0, total 94 rows starting from the beginning of the part 2024.10.22 02:29:35.749265 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_788_160/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:29:35.757803 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1630_1630_0, total 73 rows starting from the beginning of the part 2024.10.22 02:29:35.873552 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1388033 2024.10.22 02:29:35.953119 [ 1396 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_1631_1631_0, total 107 rows starting from the beginning of the part 2024.10.22 02:29:35.962769 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 163 bytes for 59a15b5bc327fa6e99a34935989c360c:0) 2024.10.22 02:29:35.965424 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ac7b7813a0d3c845e8b84a0cb495dd4a, offset: 0, size: 152 2024.10.22 02:29:35.970608 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ac7b7813a0d3c845e8b84a0cb495dd4a for removal 2024.10.22 02:29:35.977527 [ 1476 ] {} CacheMetadata: Key ac7b7813a0d3c845e8b84a0cb495dd4a is removed from metadata 2024.10.22 02:29:36.267904 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1625_507} MergeTask::MergeProjectionsStage: Merge sorted 109074 rows, containing 13 columns (13 merged, 0 gathered) in 6.367833035 sec., 17128.903883077393 rows/sec., 2.95 MiB/sec. 2024.10.22 02:29:36.333068 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} MergeTask::MergeProjectionsStage: Merge sorted 832 rows, containing 32 columns (32 merged, 0 gathered) in 3.427948272 sec., 242.71078032183328 rows/sec., 18.27 KiB/sec. 2024.10.22 02:29:36.387935 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} MergedBlockOutputStream: filled checksums 202410_1_788_160 (state Temporary) 2024.10.22 02:29:36.581606 [ 1394 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1625_507} MergedBlockOutputStream: filled checksums 202410_1_1625_507 (state Temporary) 2024.10.22 02:29:36.827809 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_788_160 to 202410_1_788_160 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:36.836988 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_788_160} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_783_159, 202410_788_788_0] -> 202410_1_788_160 2024.10.22 02:29:36.859826 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(950c94451d262b67942c7ed1167d589d) : [0, 72608]: Setting download as failed: File segment: [0, 72608], key: 950c94451d262b67942c7ed1167d589d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:36.861575 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(950c94451d262b67942c7ed1167d589d): No space left in cache to reserve 72609 bytes, will continue without cache download 2024.10.22 02:29:36.878649 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 950c94451d262b67942c7ed1167d589d, offset: 0, size: 0 2024.10.22 02:29:36.882898 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 950c94451d262b67942c7ed1167d589d for removal 2024.10.22 02:29:36.887098 [ 1476 ] {} CacheMetadata: Key 950c94451d262b67942c7ed1167d589d is removed from metadata 2024.10.22 02:29:36.928276 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:36.936127 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:36.985007 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} MergeTask::MergeProjectionsStage: Merge sorted 519 rows, containing 13 columns (13 merged, 0 gathered) in 2.061320797 sec., 251.7803152014674 rows/sec., 47.01 KiB/sec. 2024.10.22 02:29:37.019497 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} MergedBlockOutputStream: filled checksums 202410_1626_1631_1 (state Temporary) 2024.10.22 02:29:37.062346 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_920_920_0 (state Temporary) 2024.10.22 02:29:37.248415 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:29:37.343304 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1631_1 to 202410_1626_1631_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:37.345727 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_920_920_0 to 202410_927_927_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:37.356505 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1631_1} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1626_0, 202410_1631_1631_0] -> 202410_1626_1631_1 2024.10.22 02:29:37.397753 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1388033 2024.10.22 02:29:37.411226 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:29:37.565731 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 15.63 MiB, will set to 1.56 GiB (RSS), difference: 33.75 MiB 2024.10.22 02:29:37.596653 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1625_507} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1_1625_507 to 202410_1_1625_507 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:37.600233 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1_1625_507} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 2 parts: [202410_1_1206_506, 202410_1207_1625_150] -> 202410_1_1625_507 2024.10.22 02:29:37.618290 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.33 MiB. 2024.10.22 02:29:38.142767 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for e6945f917497649d9c011f18d8c42820:0) 2024.10.22 02:29:38.145408 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a3a68a1158d11da6e3144dc6312227ff, offset: 0, size: 152 2024.10.22 02:29:38.147766 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a3a68a1158d11da6e3144dc6312227ff for removal 2024.10.22 02:29:38.149381 [ 1476 ] {} CacheMetadata: Key a3a68a1158d11da6e3144dc6312227ff is removed from metadata 2024.10.22 02:29:38.498434 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 257 entries to flush up to offset 106096 2024.10.22 02:29:38.727308 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(66c7c04b9dc009b7f1362bb8ba0f0ce7) : [0, 27283]: Setting download as failed: File segment: [0, 27283], key: 66c7c04b9dc009b7f1362bb8ba0f0ce7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:38.732438 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(66c7c04b9dc009b7f1362bb8ba0f0ce7): No space left in cache to reserve 27284 bytes, will continue without cache download 2024.10.22 02:29:38.752462 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 66c7c04b9dc009b7f1362bb8ba0f0ce7, offset: 0, size: 0 2024.10.22 02:29:38.754904 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 66c7c04b9dc009b7f1362bb8ba0f0ce7 for removal 2024.10.22 02:29:38.757139 [ 1476 ] {} CacheMetadata: Key 66c7c04b9dc009b7f1362bb8ba0f0ce7 is removed from metadata 2024.10.22 02:29:38.876386 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:38.885123 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:39.037392 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 73 entries to flush up to offset 108835 2024.10.22 02:29:39.150545 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_944_944_0 (state Temporary) 2024.10.22 02:29:39.210775 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_944_944_0 to 202410_950_950_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:39.361435 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 28 entries to flush up to offset 14820 2024.10.22 02:29:39.426123 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:39.432766 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:39.435811 [ 1218 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_945_53 to 202410_950_950_0 2024.10.22 02:29:39.446256 [ 1218 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 02:29:39.449346 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 106096 2024.10.22 02:29:39.451115 [ 1218 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:39.464881 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_945_53 to 202410_950_950_0 into Compact with storage Full 2024.10.22 02:29:39.512709 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1619_1619_0 (state Temporary) 2024.10.22 02:29:39.600295 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:39.610410 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:39.643505 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:39.567567 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1527_88, 202410_1528_1528_0, 202410_1529_1529_0, 202410_1530_1530_0, 202410_1531_1531_0, 202410_1532_1532_0] 2024.10.22 02:29:39.653018 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1527_88, 202410_1528_1528_0, 202410_1529_1529_0, 202410_1530_1530_0, 202410_1531_1531_0, 202410_1532_1532_0] 2024.10.22 02:29:39.658240 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_945_53, total 21828 rows starting from the beginning of the part 2024.10.22 02:29:39.658529 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_887_28, 202410_888_888_0, 202410_889_889_0, 202410_890_890_0, 202410_891_891_0, 202410_892_892_0] 2024.10.22 02:29:39.665761 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1619_1619_0 to 202410_1632_1632_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:39.664738 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_887_28, 202410_888_888_0, 202410_889_889_0, 202410_890_890_0, 202410_891_891_0, 202410_892_892_0] 2024.10.22 02:29:39.675171 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_961_961_0 (state Temporary) 2024.10.22 02:29:39.680482 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1527_88, 202410_1528_1528_0, 202410_1529_1529_0, 202410_1530_1530_0, 202410_1531_1531_0, 202410_1532_1532_0] 2024.10.22 02:29:39.667563 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(7abe29d8ca8f9636d17658d211f936b0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7abe29d8ca8f9636d17658d211f936b0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:29:39.707362 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 641310, file segment info: File segment: [0, 33554431], key: 7abe29d8ca8f9636d17658d211f936b0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:29:39.710208 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:39.747161 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 108835 2024.10.22 02:29:39.758916 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_961_961_0 to 202410_974_974_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:39.798613 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_887_28, 202410_888_888_0, 202410_889_889_0, 202410_890_890_0, 202410_891_891_0, 202410_892_892_0] 2024.10.22 02:29:39.852311 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_946_946_0, total 158 rows starting from the beginning of the part 2024.10.22 02:29:39.880659 [ 1167 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_969_334 to 202410_970_970_0 2024.10.22 02:29:39.908596 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14820 2024.10.22 02:29:39.909365 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_947_947_0, total 163 rows starting from the beginning of the part 2024.10.22 02:29:39.912592 [ 1167 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:39.920321 [ 1167 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:39.959255 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_948_948_0, total 172 rows starting from the beginning of the part 2024.10.22 02:29:39.962283 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_970_335} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_969_334 to 202410_970_970_0 into Compact with storage Full 2024.10.22 02:29:40.038250 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_949_949_0, total 163 rows starting from the beginning of the part 2024.10.22 02:29:40.151181 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_970_335} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:40.158271 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_970_335} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_969_334, total 14829 rows starting from the beginning of the part 2024.10.22 02:29:40.172452 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} MergeTreeSequentialSource: Reading 2 marks from part 202410_950_950_0, total 257 rows starting from the beginning of the part 2024.10.22 02:29:40.192384 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_970_335} MergeTreeSequentialSource: Reading 2 marks from part 202410_970_970_0, total 15 rows starting from the beginning of the part 2024.10.22 02:29:40.180304 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 39 entries to flush up to offset 7799 2024.10.22 02:29:40.371327 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_194_2/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:29:40.629087 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 151 bytes for 430a16c41c7d5ad0fbc92502de780bf4:0) 2024.10.22 02:29:40.634280 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 59a15b5bc327fa6e99a34935989c360c, offset: 0, size: 163 2024.10.22 02:29:40.637647 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 59a15b5bc327fa6e99a34935989c360c for removal 2024.10.22 02:29:40.643483 [ 1476 ] {} CacheMetadata: Key 59a15b5bc327fa6e99a34935989c360c is removed from metadata 2024.10.22 02:29:41.048263 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_922_171/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:29:41.155836 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_970_335} MergeTask::MergeProjectionsStage: Merge sorted 14844 rows, containing 12 columns (12 merged, 0 gathered) in 1.206098678 sec., 12307.450684395824 rows/sec., 1.93 MiB/sec. 2024.10.22 02:29:41.318211 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_970_335} MergedBlockOutputStream: filled checksums 202410_1_970_335 (state Temporary) 2024.10.22 02:29:41.390302 [ 1237 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:29:41.409767 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:29:41.412605 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:29:41.439046 [ 1237 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:29:41.549549 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(9122400da8c3a8e82a19da2551a657bd) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9122400da8c3a8e82a19da2551a657bd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:29:41.551911 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 582, file segment info: File segment: [0, 33554431], key: 9122400da8c3a8e82a19da2551a657bd, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:29:41.554286 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:41.724889 [ 1711 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ffc/mojvwohblvotahlymeocieorjchej, size 641310 2024.10.22 02:29:41.807418 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_970_335} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_970_335 to 202410_1_970_335 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:41.811272 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_970_335} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_969_334, 202410_970_970_0] -> 202410_1_970_335 2024.10.22 02:29:41.862273 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:29:41.964404 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:29:41.975096 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} MergeTask::MergeProjectionsStage: Merge sorted 22741 rows, containing 13 columns (13 merged, 0 gathered) in 2.517728326 sec., 9032.348631565581 rows/sec., 3.94 MiB/sec. 2024.10.22 02:29:41.981619 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:29:41.990944 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:29:42.013645 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:29:42.056522 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:29:42.058251 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} MergedBlockOutputStream: filled checksums 202410_747_950_54 (state Temporary) 2024.10.22 02:29:42.226802 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:29:42.230285 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_950_54 to 202410_747_950_54 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:42.250521 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:29:42.266593 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:29:42.292627 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column FlashMajor 2024.10.22 02:29:42.349017 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_950_54} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_945_53, 202410_950_950_0] -> 202410_747_950_54 2024.10.22 02:29:42.365085 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 16.30 MiB, will set to 1.57 GiB (RSS), difference: 3.02 MiB 2024.10.22 02:29:42.370125 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 24.08 MiB. 2024.10.22 02:29:42.447491 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/kqb/jukywyofxxdhlhvqzluiaekccvvyw (LogSeriesLimiter: on interval from 2024-10-22 02:29:30 to 2024-10-22 02:29:42 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:29:42.706837 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c17c4a7929f0f996caf00aac9c3718ad) : [0, 229036]: Setting download as failed: File segment: [0, 229036], key: c17c4a7929f0f996caf00aac9c3718ad, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:42.709287 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c17c4a7929f0f996caf00aac9c3718ad): No space left in cache to reserve 229037 bytes, will continue without cache download 2024.10.22 02:29:42.743339 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c17c4a7929f0f996caf00aac9c3718ad, offset: 0, size: 0 2024.10.22 02:29:42.745458 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c17c4a7929f0f996caf00aac9c3718ad for removal 2024.10.22 02:29:42.748620 [ 1476 ] {} CacheMetadata: Key c17c4a7929f0f996caf00aac9c3718ad is removed from metadata 2024.10.22 02:29:42.983183 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_910_275, 202410_911_911_0] 2024.10.22 02:29:42.988321 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_910_275, 202410_911_911_0] 2024.10.22 02:29:42.997076 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_910_275, 202410_911_911_0] 2024.10.22 02:29:43.169069 [ 1161 ] {} DatabaseCatalog: Execution took 269 ms. 2024.10.22 02:29:43.752814 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 108922 2024.10.22 02:29:44.015932 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:44.019416 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:44.188796 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for 359b526483b3a809ecdb5036c4928318:0) 2024.10.22 02:29:44.193084 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e6945f917497649d9c011f18d8c42820, offset: 0, size: 152 2024.10.22 02:29:44.198936 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e6945f917497649d9c011f18d8c42820 for removal 2024.10.22 02:29:44.202287 [ 1476 ] {} CacheMetadata: Key e6945f917497649d9c011f18d8c42820 is removed from metadata 2024.10.22 02:29:44.507043 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1389573 2024.10.22 02:29:44.538880 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1620_1620_0 (state Temporary) 2024.10.22 02:29:44.593837 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1620_1620_0 to 202410_1633_1633_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:44.687573 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 108922 2024.10.22 02:29:44.752734 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 11.95 MiB, will set to 1.57 GiB (RSS), difference: -8.50 MiB 2024.10.22 02:29:45.247505 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_856_160, 202410_857_862_1] 2024.10.22 02:29:45.249405 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_856_160, 202410_857_862_1] 2024.10.22 02:29:45.257962 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6b840ee2c77c5cbf20ed7c022f8d65af) : [0, 422601]: Setting download as failed: File segment: [0, 422601], key: 6b840ee2c77c5cbf20ed7c022f8d65af, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:45.259127 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6b840ee2c77c5cbf20ed7c022f8d65af): No space left in cache to reserve 422602 bytes, will continue without cache download 2024.10.22 02:29:45.293442 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6b840ee2c77c5cbf20ed7c022f8d65af, offset: 0, size: 0 2024.10.22 02:29:45.294988 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6b840ee2c77c5cbf20ed7c022f8d65af for removal 2024.10.22 02:29:45.297755 [ 1476 ] {} CacheMetadata: Key 6b840ee2c77c5cbf20ed7c022f8d65af is removed from metadata 2024.10.22 02:29:45.362245 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_856_160, 202410_857_862_1] 2024.10.22 02:29:45.456806 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:45.459950 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:45.607272 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_921_921_0 (state Temporary) 2024.10.22 02:29:45.741126 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_921_921_0 to 202410_928_928_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:45.765399 [ 1156 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_923_923_0 to 202410_928_928_0 2024.10.22 02:29:45.776724 [ 1156 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:45.788480 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:45.819378 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_923_923_0 to 202410_928_928_0 into Compact with storage Full 2024.10.22 02:29:45.839604 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1389573 2024.10.22 02:29:45.854129 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:45.867938 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_923_923_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:29:45.888257 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_924_924_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:29:45.913686 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_925_925_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:29:45.944798 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_926_926_0, total 770 rows starting from the beginning of the part 2024.10.22 02:29:45.956901 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_927_927_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:29:45.978870 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_928_928_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:29:46.337479 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 146 bytes for cb607479a3947b91bb3f45e6891d976e:0) 2024.10.22 02:29:46.339520 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 430a16c41c7d5ad0fbc92502de780bf4, offset: 0, size: 151 2024.10.22 02:29:46.342927 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 430a16c41c7d5ad0fbc92502de780bf4 for removal 2024.10.22 02:29:46.346490 [ 1476 ] {} CacheMetadata: Key 430a16c41c7d5ad0fbc92502de780bf4 is removed from metadata 2024.10.22 02:29:46.541075 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.27 MiB, will set to 1.57 GiB (RSS), difference: -7.44 MiB 2024.10.22 02:29:46.860474 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} MergeTask::MergeProjectionsStage: Merge sorted 7315 rows, containing 4 columns (4 merged, 0 gathered) in 1.054544347 sec., 6936.64521630592 rows/sec., 108.39 KiB/sec. 2024.10.22 02:29:46.908424 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} MergedBlockOutputStream: filled checksums 202410_923_928_1 (state Temporary) 2024.10.22 02:29:47.067020 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_923_928_1 to 202410_923_928_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:47.070548 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_923_928_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_923_923_0, 202410_928_928_0] -> 202410_923_928_1 2024.10.22 02:29:47.199436 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 119 entries to flush up to offset 106215 2024.10.22 02:29:47.218300 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(70ee73587414bad2f1c907477f095527) : [0, 4835]: Setting download as failed: File segment: [0, 4835], key: 70ee73587414bad2f1c907477f095527, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:47.222246 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(70ee73587414bad2f1c907477f095527): No space left in cache to reserve 4836 bytes, will continue without cache download 2024.10.22 02:29:47.353825 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 70ee73587414bad2f1c907477f095527, offset: 0, size: 0 2024.10.22 02:29:47.356373 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 70ee73587414bad2f1c907477f095527 for removal 2024.10.22 02:29:47.356849 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:29:47.359642 [ 1476 ] {} CacheMetadata: Key 70ee73587414bad2f1c907477f095527 is removed from metadata 2024.10.22 02:29:47.416757 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 14841 2024.10.22 02:29:47.452384 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:47.455148 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:47.502211 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_945_945_0 (state Temporary) 2024.10.22 02:29:47.561090 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:47.564078 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:47.575207 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_945_945_0 to 202410_951_951_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:47.627826 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 106215 2024.10.22 02:29:47.904954 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_962_962_0 (state Temporary) 2024.10.22 02:29:47.920095 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 797 2024.10.22 02:29:48.049436 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_962_962_0 to 202410_975_975_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:48.069497 [ 1201 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_970_335 to 202410_971_971_0 2024.10.22 02:29:48.075706 [ 1201 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:48.084665 [ 1201 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:48.101248 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14841 2024.10.22 02:29:48.144967 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_971_336} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_970_335 to 202410_971_971_0 into Compact with storage Full 2024.10.22 02:29:48.181435 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:48.192477 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:48.351973 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_779_779_0 (state Temporary) 2024.10.22 02:29:48.415616 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_779_779_0 to 202410_789_789_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:48.450481 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_971_336} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:48.461749 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_971_336} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_970_335, total 14844 rows starting from the beginning of the part 2024.10.22 02:29:48.503681 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_971_336} MergeTreeSequentialSource: Reading 2 marks from part 202410_971_971_0, total 17 rows starting from the beginning of the part 2024.10.22 02:29:48.570475 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 164 bytes for 3b6180a8047b5138a740ae01d215edd6:0) 2024.10.22 02:29:48.573741 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 359b526483b3a809ecdb5036c4928318, offset: 0, size: 155 2024.10.22 02:29:48.578285 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 359b526483b3a809ecdb5036c4928318 for removal 2024.10.22 02:29:48.585541 [ 1476 ] {} CacheMetadata: Key 359b526483b3a809ecdb5036c4928318 is removed from metadata 2024.10.22 02:29:48.703436 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 109001 2024.10.22 02:29:48.852998 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 797 2024.10.22 02:29:49.120395 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:49.131971 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:49.289061 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1621_1621_0 (state Temporary) 2024.10.22 02:29:49.349650 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_971_336} MergeTask::MergeProjectionsStage: Merge sorted 14861 rows, containing 12 columns (12 merged, 0 gathered) in 1.229897934 sec., 12083.116484038259 rows/sec., 1.89 MiB/sec. 2024.10.22 02:29:49.410709 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1621_1621_0 to 202410_1634_1634_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:49.498461 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_971_336} MergedBlockOutputStream: filled checksums 202410_1_971_336 (state Temporary) 2024.10.22 02:29:49.538980 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109001 2024.10.22 02:29:49.607052 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_971_336} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_971_336 to 202410_1_971_336 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:49.641721 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_971_336} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_970_335, 202410_971_971_0] -> 202410_1_971_336 2024.10.22 02:29:49.666238 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:29:49.736864 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.02 MiB, will set to 1.57 GiB (RSS), difference: 11.50 MiB 2024.10.22 02:29:49.747571 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(90670ada79db51fe63b9b1edc12329e0) : [0, 183864]: Setting download as failed: File segment: [0, 183864], key: 90670ada79db51fe63b9b1edc12329e0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:49.748720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(90670ada79db51fe63b9b1edc12329e0): No space left in cache to reserve 183865 bytes, will continue without cache download 2024.10.22 02:29:50.215848 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 90670ada79db51fe63b9b1edc12329e0, offset: 0, size: 0 2024.10.22 02:29:50.218803 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 90670ada79db51fe63b9b1edc12329e0 for removal 2024.10.22 02:29:50.224665 [ 1476 ] {} CacheMetadata: Key 90670ada79db51fe63b9b1edc12329e0 is removed from metadata 2024.10.22 02:29:50.550568 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_911_276, 202410_912_912_0] 2024.10.22 02:29:50.553767 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_911_276, 202410_912_912_0] 2024.10.22 02:29:50.568872 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_911_276, 202410_912_912_0] 2024.10.22 02:29:51.253356 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 153 bytes for 08b5aaafada9bee2a3cfab673474e9e5:0) 2024.10.22 02:29:51.255397 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cb607479a3947b91bb3f45e6891d976e, offset: 0, size: 146 2024.10.22 02:29:51.257729 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cb607479a3947b91bb3f45e6891d976e for removal 2024.10.22 02:29:51.260098 [ 1476 ] {} CacheMetadata: Key cb607479a3947b91bb3f45e6891d976e is removed from metadata 2024.10.22 02:29:51.411407 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:29:51.415426 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:29:51.456549 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 15.52 MiB, will set to 1.57 GiB (RSS), difference: -1.30 MiB 2024.10.22 02:29:51.750979 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_922_171/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:29:52.289840 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(fa64405f7d0494874571fc0d08c3ae56) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fa64405f7d0494874571fc0d08c3ae56, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:29:52.297566 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 25578, file segment info: File segment: [0, 33554431], key: fa64405f7d0494874571fc0d08c3ae56, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:29:52.302607 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:52.844983 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1390728 2024.10.22 02:29:53.265183 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f15b350293beca5e90d0205bd2781894) : [0, 280772]: Setting download as failed: File segment: [0, 280772], key: f15b350293beca5e90d0205bd2781894, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:53.267629 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f15b350293beca5e90d0205bd2781894): No space left in cache to reserve 280773 bytes, will continue without cache download 2024.10.22 02:29:53.434346 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f15b350293beca5e90d0205bd2781894, offset: 0, size: 0 2024.10.22 02:29:53.438991 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f15b350293beca5e90d0205bd2781894 for removal 2024.10.22 02:29:53.452638 [ 1476 ] {} CacheMetadata: Key f15b350293beca5e90d0205bd2781894 is removed from metadata 2024.10.22 02:29:53.548915 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 38 entries to flush up to offset 109039 2024.10.22 02:29:53.686673 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:53.692622 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:53.792636 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_922_922_0 (state Temporary) 2024.10.22 02:29:53.804022 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:53.807665 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:53.856337 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_922_922_0 to 202410_929_929_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:53.860210 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1622_1622_0 (state Temporary) 2024.10.22 02:29:53.898835 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1390728 2024.10.22 02:29:53.905770 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1622_1622_0 to 202410_1635_1635_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:54.173635 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109039 2024.10.22 02:29:54.220158 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_194_2/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:29:54.523744 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(3792130e0f406cd907723962b0e9c4e3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3792130e0f406cd907723962b0e9c4e3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:29:54.643231 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 393, file segment info: File segment: [0, 33554431], key: 3792130e0f406cd907723962b0e9c4e3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:29:54.646078 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:55.152134 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 74 entries to flush up to offset 106289 2024.10.22 02:29:55.359342 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:55.361786 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:55.412715 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_946_946_0 (state Temporary) 2024.10.22 02:29:55.564959 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_946_946_0 to 202410_952_952_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:55.603995 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 106289 2024.10.22 02:29:55.605803 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 14851 2024.10.22 02:29:55.758868 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:55.760770 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:55.839829 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:55.843375 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:55.846866 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 153 bytes for 0bb5f8eb06d62431e75f6a573d95b0ec:0) 2024.10.22 02:29:55.848204 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3b6180a8047b5138a740ae01d215edd6, offset: 0, size: 164 2024.10.22 02:29:55.850097 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3b6180a8047b5138a740ae01d215edd6 for removal 2024.10.22 02:29:55.851440 [ 1476 ] {} CacheMetadata: Key 3b6180a8047b5138a740ae01d215edd6 is removed from metadata 2024.10.22 02:29:55.935112 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_963_963_0 (state Temporary) 2024.10.22 02:29:55.980110 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_963_963_0 to 202410_976_976_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:56.030468 [ 1275 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_971_336 to 202410_972_972_0 2024.10.22 02:29:56.042681 [ 1275 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:56.049535 [ 1275 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:56.052286 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14851 2024.10.22 02:29:56.092263 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_972_337} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_971_336 to 202410_972_972_0 into Compact with storage Full 2024.10.22 02:29:56.301717 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_972_337} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:56.354667 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 798 2024.10.22 02:29:56.424505 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_972_337} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_971_336, total 14861 rows starting from the beginning of the part 2024.10.22 02:29:56.445191 [ 1260 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:29:56.457694 [ 1260 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:29:56.483966 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_972_337} MergeTreeSequentialSource: Reading 2 marks from part 202410_972_972_0, total 15 rows starting from the beginning of the part 2024.10.22 02:29:56.681193 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:56.686856 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:56.784684 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(ede7c25c4cdce2230719a89794792167) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ede7c25c4cdce2230719a89794792167, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:29:56.791584 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 8263, file segment info: File segment: [0, 33554431], key: ede7c25c4cdce2230719a89794792167, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:29:56.797906 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:56.951506 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_780_780_0 (state Temporary) 2024.10.22 02:29:57.052365 [ 1425 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_insert_202410_780_780_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:29:57.352501 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_780_780_0 to 202410_790_790_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:57.555185 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 798 2024.10.22 02:29:57.678858 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_972_337} MergeTask::MergeProjectionsStage: Merge sorted 14876 rows, containing 12 columns (12 merged, 0 gathered) in 1.612151818 sec., 9227.418803803997 rows/sec., 1.44 MiB/sec. 2024.10.22 02:29:57.759363 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a11bbfa9201d19a13057f2bef6c390eb) : [0, 269318]: Setting download as failed: File segment: [0, 269318], key: a11bbfa9201d19a13057f2bef6c390eb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:29:57.761054 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a11bbfa9201d19a13057f2bef6c390eb): No space left in cache to reserve 269319 bytes, will continue without cache download 2024.10.22 02:29:57.774945 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a11bbfa9201d19a13057f2bef6c390eb, offset: 0, size: 0 2024.10.22 02:29:57.779196 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a11bbfa9201d19a13057f2bef6c390eb for removal 2024.10.22 02:29:57.784671 [ 1476 ] {} CacheMetadata: Key a11bbfa9201d19a13057f2bef6c390eb is removed from metadata 2024.10.22 02:29:58.064678 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_972_337} MergedBlockOutputStream: filled checksums 202410_1_972_337 (state Temporary) 2024.10.22 02:29:58.188277 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 59 entries to flush up to offset 109098 2024.10.22 02:29:58.332487 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_972_337} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_972_337 to 202410_1_972_337 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:58.337674 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_972_337} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_971_336, 202410_972_972_0] -> 202410_1_972_337 2024.10.22 02:29:58.354888 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:29:58.540322 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:58.544225 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:58.591890 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1623_1623_0 (state Temporary) 2024.10.22 02:29:58.911559 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1623_1623_0 to 202410_1636_1636_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:29:58.940448 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d8041adb582fdef9758dad3171132ed2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d8041adb582fdef9758dad3171132ed2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:29:58.943054 [ 1205 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1631_1 to 202410_1636_1636_0 2024.10.22 02:29:58.944557 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 521, file segment info: File segment: [0, 33554431], key: d8041adb582fdef9758dad3171132ed2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:29:58.950210 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:29:58.969184 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109098 2024.10.22 02:29:58.977407 [ 1205 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:29:58.983512 [ 1205 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:29:58.999489 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1631_1 to 202410_1636_1636_0 into Compact with storage Full 2024.10.22 02:29:59.028834 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:29:59.039778 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1631_1, total 519 rows starting from the beginning of the part 2024.10.22 02:29:59.056668 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1632_1632_0, total 73 rows starting from the beginning of the part 2024.10.22 02:29:59.076987 [ 1697 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ssb/rrrypdmpuyyszhycnypbgawulosks, size 8263 2024.10.22 02:29:59.099659 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1633_1633_0, total 87 rows starting from the beginning of the part 2024.10.22 02:29:59.114768 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1634_1634_0, total 79 rows starting from the beginning of the part 2024.10.22 02:29:59.141419 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1635_1635_0, total 38 rows starting from the beginning of the part 2024.10.22 02:29:59.154692 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} MergeTreeSequentialSource: Reading 2 marks from part 202410_1636_1636_0, total 59 rows starting from the beginning of the part 2024.10.22 02:29:59.301685 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_193_193_0 (state Temporary) 2024.10.22 02:29:59.320986 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:29:59.360819 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:29:59.374441 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:29:59.405342 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:29:59.441735 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:29:59.452234 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_912_277, 202410_913_913_0] 2024.10.22 02:29:59.454713 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_912_277, 202410_913_913_0] 2024.10.22 02:29:59.465179 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_912_277, 202410_913_913_0] 2024.10.22 02:29:59.465534 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:29:59.496814 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column UTMTerm 2024.10.22 02:29:59.554882 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ixi/guiepckiywzaivgimlpatihnxwsnu (LogSeriesLimiter: on interval from 2024-10-22 02:29:42 to 2024-10-22 02:29:59 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:30:00.037836 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 153 bytes for 4d00d21c00453cb8091ecfec809b9122:0) 2024.10.22 02:30:00.040378 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 08b5aaafada9bee2a3cfab673474e9e5, offset: 0, size: 153 2024.10.22 02:30:00.044209 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 08b5aaafada9bee2a3cfab673474e9e5 for removal 2024.10.22 02:30:00.046902 [ 1476 ] {} CacheMetadata: Key 08b5aaafada9bee2a3cfab673474e9e5 is removed from metadata 2024.10.22 02:30:00.579206 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} MergeTask::MergeProjectionsStage: Merge sorted 855 rows, containing 13 columns (13 merged, 0 gathered) in 1.583354862 sec., 539.9926576914127 rows/sec., 97.41 KiB/sec. 2024.10.22 02:30:00.612745 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} MergedBlockOutputStream: filled checksums 202410_1626_1636_2 (state Temporary) 2024.10.22 02:30:00.905303 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1391498 2024.10.22 02:30:00.915699 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(a7886eeed464cce95da4304eb7b17470) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a7886eeed464cce95da4304eb7b17470, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:30:00.923702 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 601454, file segment info: File segment: [0, 33554431], key: a7886eeed464cce95da4304eb7b17470, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:30:00.925651 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1636_2 to 202410_1626_1636_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:00.930068 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1636_2} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1631_1, 202410_1636_1636_0] -> 202410_1626_1636_2 2024.10.22 02:30:00.930127 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:01.058338 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:30:01.410749 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:30:01.413456 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:30:01.489240 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:01.500495 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:01.607946 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_923_923_0 (state Temporary) 2024.10.22 02:30:01.711062 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_923_923_0 to 202410_930_930_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:01.741583 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1391498 2024.10.22 02:30:01.980539 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8469a53d4e458ba8d3f2e8abc123f3a1) : [0, 312450]: Setting download as failed: File segment: [0, 312450], key: 8469a53d4e458ba8d3f2e8abc123f3a1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:01.986564 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8469a53d4e458ba8d3f2e8abc123f3a1): No space left in cache to reserve 312451 bytes, will continue without cache download 2024.10.22 02:30:02.046196 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8469a53d4e458ba8d3f2e8abc123f3a1, offset: 0, size: 0 2024.10.22 02:30:02.048680 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8469a53d4e458ba8d3f2e8abc123f3a1 for removal 2024.10.22 02:30:02.051076 [ 1476 ] {} CacheMetadata: Key 8469a53d4e458ba8d3f2e8abc123f3a1 is removed from metadata 2024.10.22 02:30:02.550579 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 14.77 MiB, will set to 1.57 GiB (RSS), difference: -12.02 MiB 2024.10.22 02:30:02.827464 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(42d308a39e04dfc1ac9b5c5acb163849) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 42d308a39e04dfc1ac9b5c5acb163849, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:30:02.847919 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 583, file segment info: File segment: [0, 33554431], key: 42d308a39e04dfc1ac9b5c5acb163849, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:30:02.850120 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:02.894864 [ 1688 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/kqb/jukywyofxxdhlhvqzluiaekccvvyw, size 601454 2024.10.22 02:30:02.980222 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 109164 2024.10.22 02:30:03.071586 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_922_171/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:30:03.116099 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 91 entries to flush up to offset 106380 2024.10.22 02:30:03.121126 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:30:03.144229 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:30:03.155012 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:30:03.165614 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:30:03.191117 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:30:03.222904 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:30:03.243095 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:30:03.283697 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:30:03.418198 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column FlashMinor 2024.10.22 02:30:03.433327 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:03.440801 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:03.492261 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_947_947_0 (state Temporary) 2024.10.22 02:30:03.551148 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:03.553466 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:03.554761 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 14865 2024.10.22 02:30:03.559570 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_947_947_0 to 202410_953_953_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:03.615187 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1624_1624_0 (state Temporary) 2024.10.22 02:30:03.699998 [ 1238 ] {} DatabaseCatalog: Execution took 408 ms. 2024.10.22 02:30:03.704314 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 106380 2024.10.22 02:30:03.765685 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1624_1624_0 to 202410_1637_1637_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:03.817585 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:03.822872 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:03.850266 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 157 bytes for 798e0081035e9b3b997ef616ed3411ce:0) 2024.10.22 02:30:03.856244 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0bb5f8eb06d62431e75f6a573d95b0ec, offset: 0, size: 153 2024.10.22 02:30:03.857676 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_964_964_0 (state Temporary) 2024.10.22 02:30:03.862427 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0bb5f8eb06d62431e75f6a573d95b0ec for removal 2024.10.22 02:30:03.864852 [ 1476 ] {} CacheMetadata: Key 0bb5f8eb06d62431e75f6a573d95b0ec is removed from metadata 2024.10.22 02:30:03.948855 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109164 2024.10.22 02:30:04.166804 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_964_964_0 to 202410_977_977_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:04.312816 [ 1240 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_972_337 to 202410_973_973_0 2024.10.22 02:30:04.323017 [ 1240 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:04.334043 [ 1240 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:04.397664 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14865 2024.10.22 02:30:04.410954 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_973_338} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_972_337 to 202410_973_973_0 into Compact with storage Full 2024.10.22 02:30:04.501472 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_973_338} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:04.517456 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_973_338} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_972_337, total 14876 rows starting from the beginning of the part 2024.10.22 02:30:04.550486 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_973_338} MergeTreeSequentialSource: Reading 2 marks from part 202410_973_973_0, total 22 rows starting from the beginning of the part 2024.10.22 02:30:05.056694 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 799 2024.10.22 02:30:05.058018 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(717a72b2c67e44cfbcab6b078b0228fa) : [0, 517461]: Setting download as failed: File segment: [0, 517461], key: 717a72b2c67e44cfbcab6b078b0228fa, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:05.060175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(717a72b2c67e44cfbcab6b078b0228fa): No space left in cache to reserve 517462 bytes, will continue without cache download 2024.10.22 02:30:05.215781 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 717a72b2c67e44cfbcab6b078b0228fa, offset: 0, size: 0 2024.10.22 02:30:05.218938 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 717a72b2c67e44cfbcab6b078b0228fa for removal 2024.10.22 02:30:05.225058 [ 1476 ] {} CacheMetadata: Key 717a72b2c67e44cfbcab6b078b0228fa is removed from metadata 2024.10.22 02:30:05.365833 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_973_338} MergeTask::MergeProjectionsStage: Merge sorted 14898 rows, containing 12 columns (12 merged, 0 gathered) in 0.965267555 sec., 15434.062735072454 rows/sec., 2.42 MiB/sec. 2024.10.22 02:30:05.450925 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_973_338} MergedBlockOutputStream: filled checksums 202410_1_973_338 (state Temporary) 2024.10.22 02:30:05.534664 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 8.74 MiB, will set to 1.57 GiB (RSS), difference: -7.60 MiB 2024.10.22 02:30:05.555281 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:05.559017 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:05.639643 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_781_781_0 (state Temporary) 2024.10.22 02:30:05.743021 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_781_781_0 to 202410_791_791_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:05.855679 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_973_338} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_973_338 to 202410_1_973_338 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:05.873584 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1532_89, 202410_1533_1533_0, 202410_1534_1534_0, 202410_1535_1535_0, 202410_1536_1536_0, 202410_1537_1537_0] 2024.10.22 02:30:05.877161 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1532_89, 202410_1533_1533_0, 202410_1534_1534_0, 202410_1535_1535_0, 202410_1536_1536_0, 202410_1537_1537_0] 2024.10.22 02:30:05.950631 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 799 2024.10.22 02:30:05.977028 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_973_338} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_972_337, 202410_973_973_0] -> 202410_1_973_338 2024.10.22 02:30:06.010863 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:30:06.012109 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1532_89, 202410_1533_1533_0, 202410_1534_1534_0, 202410_1535_1535_0, 202410_1536_1536_0, 202410_1537_1537_0] 2024.10.22 02:30:06.099058 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_913_278, 202410_914_914_0] 2024.10.22 02:30:06.103166 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_913_278, 202410_914_914_0] 2024.10.22 02:30:06.193336 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 157 bytes for cbd50784d8a95bda9b13847f0b117618:0) 2024.10.22 02:30:06.196850 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4d00d21c00453cb8091ecfec809b9122, offset: 0, size: 153 2024.10.22 02:30:06.200810 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4d00d21c00453cb8091ecfec809b9122 for removal 2024.10.22 02:30:06.206304 [ 1476 ] {} CacheMetadata: Key 4d00d21c00453cb8091ecfec809b9122 is removed from metadata 2024.10.22 02:30:06.219317 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_913_278, 202410_914_914_0] 2024.10.22 02:30:07.469743 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 10.99 MiB, will set to 1.57 GiB (RSS), difference: 4.53 MiB 2024.10.22 02:30:07.525412 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8b8b25d9bba63e578caaf6eb9ef66bf2) : [0, 483395]: Setting download as failed: File segment: [0, 483395], key: 8b8b25d9bba63e578caaf6eb9ef66bf2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:07.528234 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8b8b25d9bba63e578caaf6eb9ef66bf2): No space left in cache to reserve 483396 bytes, will continue without cache download 2024.10.22 02:30:07.566639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8b8b25d9bba63e578caaf6eb9ef66bf2, offset: 0, size: 0 2024.10.22 02:30:07.568997 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8b8b25d9bba63e578caaf6eb9ef66bf2 for removal 2024.10.22 02:30:07.573134 [ 1476 ] {} CacheMetadata: Key 8b8b25d9bba63e578caaf6eb9ef66bf2 is removed from metadata 2024.10.22 02:30:07.615190 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_193_193_0 to 202410_196_196_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:07.670809 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_738_150, 202410_739_739_0, 202410_740_740_0, 202410_741_741_0, 202410_742_742_0, 202410_743_743_0] 2024.10.22 02:30:07.724738 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_738_150, 202410_739_739_0, 202410_740_740_0, 202410_741_741_0, 202410_742_742_0, 202410_743_743_0] 2024.10.22 02:30:07.771488 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_194_2/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:30:07.895295 [ 1424 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_738_150, 202410_739_739_0, 202410_740_740_0, 202410_741_741_0, 202410_742_742_0, 202410_743_743_0] 2024.10.22 02:30:07.955917 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 109242 2024.10.22 02:30:08.113799 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:08.122950 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:08.304264 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1625_1625_0 (state Temporary) 2024.10.22 02:30:08.366506 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1625_1625_0 to 202410_1638_1638_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:08.554301 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 153 bytes for 6a0d8cd83faad7fffcdd7990bff394e3:0) 2024.10.22 02:30:08.556443 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 798e0081035e9b3b997ef616ed3411ce, offset: 0, size: 157 2024.10.22 02:30:08.562521 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109242 2024.10.22 02:30:08.562303 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 798e0081035e9b3b997ef616ed3411ce for removal 2024.10.22 02:30:08.567614 [ 1476 ] {} CacheMetadata: Key 798e0081035e9b3b997ef616ed3411ce is removed from metadata 2024.10.22 02:30:08.745701 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 1393040 2024.10.22 02:30:09.251140 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:09.255050 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:09.451341 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.78 MiB, will set to 1.56 GiB (RSS), difference: -4.51 MiB 2024.10.22 02:30:09.566644 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_924_924_0 (state Temporary) 2024.10.22 02:30:09.657276 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_924_924_0 to 202410_931_931_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:09.749816 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7d527c75836049a7dddd347401345134) : [0, 300093]: Setting download as failed: File segment: [0, 300093], key: 7d527c75836049a7dddd347401345134, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:09.751808 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7d527c75836049a7dddd347401345134): No space left in cache to reserve 300094 bytes, will continue without cache download 2024.10.22 02:30:09.805159 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1393040 2024.10.22 02:30:09.992749 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7d527c75836049a7dddd347401345134, offset: 0, size: 0 2024.10.22 02:30:09.995893 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7d527c75836049a7dddd347401345134 for removal 2024.10.22 02:30:09.999032 [ 1476 ] {} CacheMetadata: Key 7d527c75836049a7dddd347401345134 is removed from metadata 2024.10.22 02:30:10.921409 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 4eb4e0cc5a20c36bcb04793d3ec63fd8:0) 2024.10.22 02:30:10.927329 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cbd50784d8a95bda9b13847f0b117618, offset: 0, size: 157 2024.10.22 02:30:10.931608 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cbd50784d8a95bda9b13847f0b117618 for removal 2024.10.22 02:30:10.936831 [ 1476 ] {} CacheMetadata: Key cbd50784d8a95bda9b13847f0b117618 is removed from metadata 2024.10.22 02:30:11.127100 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7799 2024.10.22 02:30:11.222630 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 70 entries to flush up to offset 106450 2024.10.22 02:30:11.300823 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_922_171} MergeTask::MergeProjectionsStage: Merge sorted 1395848 rows, containing 4 columns (4 merged, 0 gathered) in 42.1929987 sec., 33082.45545486674 rows/sec., 496.06 KiB/sec. 2024.10.22 02:30:11.413493 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:30:11.419156 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:30:11.435870 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_922_171} MergedBlockOutputStream: filled checksums 202410_1_922_171 (state Temporary) 2024.10.22 02:30:11.462836 [ 1197 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:30:11.479372 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 12.68 MiB, will set to 1.56 GiB (RSS), difference: 6.08 MiB 2024.10.22 02:30:11.485020 [ 1197 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:30:11.525154 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:11.529324 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:11.821793 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_948_948_0 (state Temporary) 2024.10.22 02:30:11.861469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4697b66d32fb28f4cd043d83e6d51a79) : [0, 238416]: Setting download as failed: File segment: [0, 238416], key: 4697b66d32fb28f4cd043d83e6d51a79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:11.862565 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4697b66d32fb28f4cd043d83e6d51a79): No space left in cache to reserve 238417 bytes, will continue without cache download 2024.10.22 02:30:11.886795 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4697b66d32fb28f4cd043d83e6d51a79, offset: 0, size: 0 2024.10.22 02:30:11.892515 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4697b66d32fb28f4cd043d83e6d51a79 for removal 2024.10.22 02:30:11.904518 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 14875 2024.10.22 02:30:12.014445 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_948_948_0 to 202410_954_954_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:12.066890 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 106450 2024.10.22 02:30:11.997826 [ 1476 ] {} CacheMetadata: Key 4697b66d32fb28f4cd043d83e6d51a79 is removed from metadata 2024.10.22 02:30:12.139316 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:12.145450 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:12.177355 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_965_965_0 (state Temporary) 2024.10.22 02:30:12.327845 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_965_965_0 to 202410_978_978_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:12.360799 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14875 2024.10.22 02:30:12.364193 [ 1261 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_973_338 to 202410_974_974_0 2024.10.22 02:30:12.368517 [ 1261 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:12.372801 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:12.407427 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_974_339} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_973_338 to 202410_974_974_0 into Compact with storage Full 2024.10.22 02:30:12.561591 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_974_339} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:12.568109 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_974_339} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_973_338, total 14898 rows starting from the beginning of the part 2024.10.22 02:30:12.570119 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 109298 2024.10.22 02:30:12.624018 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_974_339} MergeTreeSequentialSource: Reading 2 marks from part 202410_974_974_0, total 28 rows starting from the beginning of the part 2024.10.22 02:30:12.841612 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:12.852289 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:12.938390 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1626_1626_0 (state Temporary) 2024.10.22 02:30:13.100402 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1626_1626_0 to 202410_1639_1639_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:13.260797 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109298 2024.10.22 02:30:13.265899 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 154 bytes for 02532c30223616b1acd9f141a7cb09b2:0) 2024.10.22 02:30:13.267836 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6a0d8cd83faad7fffcdd7990bff394e3, offset: 0, size: 153 2024.10.22 02:30:13.271415 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6a0d8cd83faad7fffcdd7990bff394e3 for removal 2024.10.22 02:30:13.273754 [ 1476 ] {} CacheMetadata: Key 6a0d8cd83faad7fffcdd7990bff394e3 is removed from metadata 2024.10.22 02:30:13.348313 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 10.46 MiB, will set to 1.56 GiB (RSS), difference: 5.73 MiB 2024.10.22 02:30:13.452602 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 800 2024.10.22 02:30:13.506971 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_974_339} MergeTask::MergeProjectionsStage: Merge sorted 14926 rows, containing 12 columns (12 merged, 0 gathered) in 1.110895532 sec., 13436.00687017616 rows/sec., 2.10 MiB/sec. 2024.10.22 02:30:13.565064 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_922_171/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:30:13.672587 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_974_339} MergedBlockOutputStream: filled checksums 202410_1_974_339 (state Temporary) 2024.10.22 02:30:13.761740 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:13.766535 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:13.926295 [ 1213 ] {} DatabaseCatalog: Execution took 226 ms. 2024.10.22 02:30:13.929642 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_782_782_0 (state Temporary) 2024.10.22 02:30:14.017764 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_insert_202410_782_782_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:30:14.136247 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_782_782_0 to 202410_792_792_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:14.323265 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 800 2024.10.22 02:30:14.337719 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_974_339} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_974_339 to 202410_1_974_339 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:14.343126 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_974_339} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_973_338, 202410_974_974_0] -> 202410_1_974_339 2024.10.22 02:30:14.364068 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:30:14.476492 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(350bb280ec0511d3efbb77e5c934fb83) : [0, 466231]: Setting download as failed: File segment: [0, 466231], key: 350bb280ec0511d3efbb77e5c934fb83, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:14.481222 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(350bb280ec0511d3efbb77e5c934fb83): No space left in cache to reserve 466232 bytes, will continue without cache download 2024.10.22 02:30:14.619778 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 350bb280ec0511d3efbb77e5c934fb83, offset: 0, size: 0 2024.10.22 02:30:14.621623 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 350bb280ec0511d3efbb77e5c934fb83 for removal 2024.10.22 02:30:14.625544 [ 1476 ] {} CacheMetadata: Key 350bb280ec0511d3efbb77e5c934fb83 is removed from metadata 2024.10.22 02:30:15.124478 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_914_279, 202410_915_915_0] 2024.10.22 02:30:15.137421 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_914_279, 202410_915_915_0] 2024.10.22 02:30:15.148966 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_914_279, 202410_915_915_0] 2024.10.22 02:30:15.360969 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 7.52 MiB, will set to 1.56 GiB (RSS), difference: 3.21 MiB 2024.10.22 02:30:15.446684 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_869_869_0, 202410_870_870_0, 202410_871_871_0, 202410_872_872_0, 202410_873_873_0, 202410_874_874_0] 2024.10.22 02:30:15.449569 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_869_869_0, 202410_870_870_0, 202410_871_871_0, 202410_872_872_0, 202410_873_873_0, 202410_874_874_0] 2024.10.22 02:30:15.472790 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_922_171} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_922_171 to 202410_1_922_171 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:15.475556 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_922_171} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_916_170, 202410_917_922_1] -> 202410_1_922_171 2024.10.22 02:30:15.478543 [ 1420 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_869_869_0, 202410_870_870_0, 202410_871_871_0, 202410_872_872_0, 202410_873_873_0, 202410_874_874_0] 2024.10.22 02:30:15.504963 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:30:15.814411 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 154 bytes for 967f0b45b0397e24cfc1a829d9ccd43b:0) 2024.10.22 02:30:15.817103 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4eb4e0cc5a20c36bcb04793d3ec63fd8, offset: 0, size: 152 2024.10.22 02:30:15.823115 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4eb4e0cc5a20c36bcb04793d3ec63fd8 for removal 2024.10.22 02:30:15.827037 [ 1476 ] {} CacheMetadata: Key 4eb4e0cc5a20c36bcb04793d3ec63fd8 is removed from metadata 2024.10.22 02:30:16.827317 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1394580 2024.10.22 02:30:16.951056 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(918f2c6cf8dcc18cb57212f87c48ae41) : [0, 350357]: Setting download as failed: File segment: [0, 350357], key: 918f2c6cf8dcc18cb57212f87c48ae41, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:16.952582 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(918f2c6cf8dcc18cb57212f87c48ae41): No space left in cache to reserve 350358 bytes, will continue without cache download 2024.10.22 02:30:16.974478 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 918f2c6cf8dcc18cb57212f87c48ae41, offset: 0, size: 0 2024.10.22 02:30:16.975480 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 918f2c6cf8dcc18cb57212f87c48ae41 for removal 2024.10.22 02:30:16.978578 [ 1476 ] {} CacheMetadata: Key 918f2c6cf8dcc18cb57212f87c48ae41 is removed from metadata 2024.10.22 02:30:17.265681 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:17.279106 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 109350 2024.10.22 02:30:17.285275 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:17.387971 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:17.491956 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:17.514200 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_892_29, 202410_893_893_0, 202410_894_894_0, 202410_895_895_0, 202410_896_896_0, 202410_897_897_0] 2024.10.22 02:30:17.519045 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_892_29, 202410_893_893_0, 202410_894_894_0, 202410_895_895_0, 202410_896_896_0, 202410_897_897_0] 2024.10.22 02:30:17.563926 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_925_925_0 (state Temporary) 2024.10.22 02:30:17.650274 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1627_1627_0 (state Temporary) 2024.10.22 02:30:17.652528 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_892_29, 202410_893_893_0, 202410_894_894_0, 202410_895_895_0, 202410_896_896_0, 202410_897_897_0] 2024.10.22 02:30:17.778255 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 7.14 MiB, will set to 1.56 GiB (RSS), difference: -11.91 MiB 2024.10.22 02:30:17.800628 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1627_1627_0 to 202410_1640_1640_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:17.843132 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_925_925_0 to 202410_932_932_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:17.855386 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109350 2024.10.22 02:30:17.868601 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1394580 2024.10.22 02:30:17.872656 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 151 bytes for 4b91c413e5801218a39a77665b5d9eb3:0) 2024.10.22 02:30:17.865033 [ 1266 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_922_171 to 202410_923_928_1 2024.10.22 02:30:17.877069 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 02532c30223616b1acd9f141a7cb09b2, offset: 0, size: 154 2024.10.22 02:30:17.880613 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 02532c30223616b1acd9f141a7cb09b2 for removal 2024.10.22 02:30:17.884694 [ 1266 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.10 MiB using storage policy from min volume index 0 2024.10.22 02:30:17.885478 [ 1476 ] {} CacheMetadata: Key 02532c30223616b1acd9f141a7cb09b2 is removed from metadata 2024.10.22 02:30:17.890858 [ 1266 ] {} DiskLocal: Reserved 1.10 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:17.934304 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_928_172} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_922_171 to 202410_923_928_1 into Wide with storage Full 2024.10.22 02:30:18.062584 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_928_172} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:18.069485 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_928_172} MergeTreeSequentialSource: Reading 172 marks from part 202410_1_922_171, total 1395848 rows starting from the beginning of the part 2024.10.22 02:30:18.236173 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_928_172} MergeTreeSequentialSource: Reading 2 marks from part 202410_923_928_1, total 7315 rows starting from the beginning of the part 2024.10.22 02:30:18.670288 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(db5b6df32b3691e32d984a4c567c7689) : [0, 250296]: Setting download as failed: File segment: [0, 250296], key: db5b6df32b3691e32d984a4c567c7689, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:18.672594 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(db5b6df32b3691e32d984a4c567c7689): No space left in cache to reserve 250297 bytes, will continue without cache download 2024.10.22 02:30:18.723819 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 38 entries to flush up to offset 7837 2024.10.22 02:30:18.801010 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: db5b6df32b3691e32d984a4c567c7689, offset: 0, size: 0 2024.10.22 02:30:18.802768 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key db5b6df32b3691e32d984a4c567c7689 for removal 2024.10.22 02:30:18.804697 [ 1476 ] {} CacheMetadata: Key db5b6df32b3691e32d984a4c567c7689 is removed from metadata 2024.10.22 02:30:19.558400 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 7.84 MiB, will set to 1.55 GiB (RSS), difference: -1.48 MiB 2024.10.22 02:30:19.734769 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 104 entries to flush up to offset 106554 2024.10.22 02:30:19.862513 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 14887 2024.10.22 02:30:19.940562 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 151 bytes for 7976eae7ef5c0d214f6f4ed5ada6d39e:0) 2024.10.22 02:30:19.943489 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 967f0b45b0397e24cfc1a829d9ccd43b, offset: 0, size: 154 2024.10.22 02:30:19.946439 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 967f0b45b0397e24cfc1a829d9ccd43b for removal 2024.10.22 02:30:19.949389 [ 1476 ] {} CacheMetadata: Key 967f0b45b0397e24cfc1a829d9ccd43b is removed from metadata 2024.10.22 02:30:19.967528 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:19.968953 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:19.996195 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_966_966_0 (state Temporary) 2024.10.22 02:30:20.036061 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:20.042843 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:20.149395 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_966_966_0 to 202410_979_979_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:20.186213 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_974_339 to 202410_975_975_0 2024.10.22 02:30:20.193271 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_949_949_0 (state Temporary) 2024.10.22 02:30:20.211745 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:20.214938 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14887 2024.10.22 02:30:20.219125 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:20.348083 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_975_340} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_974_339 to 202410_975_975_0 into Compact with storage Full 2024.10.22 02:30:20.502241 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_949_949_0 to 202410_955_955_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:20.538693 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_950_54 to 202410_951_951_0 2024.10.22 02:30:20.542166 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_975_340} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:20.543587 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 02:30:20.545800 [ 1219 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:20.548932 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_975_340} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_974_339, total 14926 rows starting from the beginning of the part 2024.10.22 02:30:20.550769 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 106554 2024.10.22 02:30:20.555871 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_951_55} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_950_54 to 202410_951_951_0 into Compact with storage Full 2024.10.22 02:30:20.560234 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_975_340} MergeTreeSequentialSource: Reading 2 marks from part 202410_975_975_0, total 21 rows starting from the beginning of the part 2024.10.22 02:30:20.604083 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_951_55} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:20.619821 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_951_55} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_950_54, total 22741 rows starting from the beginning of the part 2024.10.22 02:30:20.665573 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_951_55} MergeTreeSequentialSource: Reading 2 marks from part 202410_951_951_0, total 119 rows starting from the beginning of the part 2024.10.22 02:30:20.999180 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(72fa1f4b24bd12f0f8b7060d61e0131b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 72fa1f4b24bd12f0f8b7060d61e0131b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:30:21.003672 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 14679, file segment info: File segment: [0, 33554431], key: 72fa1f4b24bd12f0f8b7060d61e0131b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:30:21.008519 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:21.264913 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_975_340} MergeTask::MergeProjectionsStage: Merge sorted 14947 rows, containing 12 columns (12 merged, 0 gathered) in 1.024676152 sec., 14587.047791466508 rows/sec., 2.28 MiB/sec. 2024.10.22 02:30:21.413239 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:30:21.416720 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:30:21.438961 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_975_340} MergedBlockOutputStream: filled checksums 202410_1_975_340 (state Temporary) 2024.10.22 02:30:21.824936 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 801 2024.10.22 02:30:21.861804 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 71 entries to flush up to offset 109421 2024.10.22 02:30:21.910720 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_975_340} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_975_340 to 202410_1_975_340 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:21.915948 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_975_340} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_974_339, 202410_975_975_0] -> 202410_1_975_340 2024.10.22 02:30:22.021916 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:22.029012 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:22.059372 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:30:22.113257 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_915_280, 202410_916_916_0] 2024.10.22 02:30:22.118594 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_915_280, 202410_916_916_0] 2024.10.22 02:30:22.248723 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:22.250184 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_915_280, 202410_916_916_0] 2024.10.22 02:30:22.251143 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:22.251576 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_951_55} MergeTask::MergeProjectionsStage: Merge sorted 22860 rows, containing 13 columns (13 merged, 0 gathered) in 1.70030782 sec., 13444.624397481159 rows/sec., 5.86 MiB/sec. 2024.10.22 02:30:22.271078 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(706cd6193ade52701e2a81eb06a01f1a) : [0, 249552]: Setting download as failed: File segment: [0, 249552], key: 706cd6193ade52701e2a81eb06a01f1a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:22.273412 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(706cd6193ade52701e2a81eb06a01f1a): No space left in cache to reserve 249553 bytes, will continue without cache download 2024.10.22 02:30:22.307134 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 706cd6193ade52701e2a81eb06a01f1a, offset: 0, size: 0 2024.10.22 02:30:22.310174 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 706cd6193ade52701e2a81eb06a01f1a for removal 2024.10.22 02:30:22.313353 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_783_783_0 (state Temporary) 2024.10.22 02:30:22.315944 [ 1476 ] {} CacheMetadata: Key 706cd6193ade52701e2a81eb06a01f1a is removed from metadata 2024.10.22 02:30:22.343181 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1628_1628_0 (state Temporary) 2024.10.22 02:30:22.462680 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1628_1628_0 to 202410_1641_1641_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:22.506726 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_783_783_0 to 202410_793_793_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:22.511333 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.69 MiB, will set to 1.57 GiB (RSS), difference: 3.77 MiB 2024.10.22 02:30:22.491955 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1636_2 to 202410_1641_1641_0 2024.10.22 02:30:22.529108 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109421 2024.10.22 02:30:22.538642 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:22.543174 [ 1150 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:22.557528 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1636_2 to 202410_1641_1641_0 into Compact with storage Full 2024.10.22 02:30:22.605536 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:22.638453 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 801 2024.10.22 02:30:22.653098 [ 1254 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_788_160 to 202410_793_793_0 2024.10.22 02:30:22.664935 [ 1254 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:22.665252 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_951_55} MergedBlockOutputStream: filled checksums 202410_747_951_55 (state Temporary) 2024.10.22 02:30:22.666611 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:22.703632 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_788_160 to 202410_793_793_0 into Compact with storage Full 2024.10.22 02:30:22.724638 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1636_2, total 855 rows starting from the beginning of the part 2024.10.22 02:30:22.770734 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1637_1637_0, total 66 rows starting from the beginning of the part 2024.10.22 02:30:22.849629 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:22.856477 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_788_160, total 832 rows starting from the beginning of the part 2024.10.22 02:30:22.941094 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1638_1638_0, total 78 rows starting from the beginning of the part 2024.10.22 02:30:22.943753 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_789_789_0, total 1 rows starting from the beginning of the part 2024.10.22 02:30:22.966068 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_951_55} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_951_55 to 202410_747_951_55 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:22.966233 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_194_2/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:30:22.968207 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_951_55} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_950_54, 202410_951_951_0] -> 202410_747_951_55 2024.10.22 02:30:22.994974 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_790_790_0, total 1 rows starting from the beginning of the part 2024.10.22 02:30:23.059152 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_791_791_0, total 1 rows starting from the beginning of the part 2024.10.22 02:30:23.068101 [ 1399 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} MergeTask::MergeProjectionsStage: Merge sorted 415 rows, containing 679 columns (679 merged, 0 gathered) in 92.207220087 sec., 4.50073215100115 rows/sec., 6.60 KiB/sec. 2024.10.22 02:30:23.072382 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1639_1639_0, total 56 rows starting from the beginning of the part 2024.10.22 02:30:23.109017 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.53 MiB. 2024.10.22 02:30:23.122590 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1640_1640_0, total 52 rows starting from the beginning of the part 2024.10.22 02:30:23.210883 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_792_792_0, total 1 rows starting from the beginning of the part 2024.10.22 02:30:23.268569 [ 1398 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} MergeTreeSequentialSource: Reading 2 marks from part 202410_793_793_0, total 1 rows starting from the beginning of the part 2024.10.22 02:30:23.272390 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_1641_1641_0, total 71 rows starting from the beginning of the part 2024.10.22 02:30:23.440070 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(9a640a2963ab43c0341827fc03a896fe) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 9a640a2963ab43c0341827fc03a896fe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:30:23.446189 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 374538, file segment info: File segment: [0, 33554431], key: 9a640a2963ab43c0341827fc03a896fe, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:30:23.450429 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:23.893348 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_928_172/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:30:24.208961 [ 1190 ] {} DatabaseCatalog: Execution took 217 ms. 2024.10.22 02:30:24.340860 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(355a699e1256424addec6de9e33760d1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 355a699e1256424addec6de9e33760d1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:30:24.343634 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 342, file segment info: File segment: [0, 33554431], key: 355a699e1256424addec6de9e33760d1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:30:24.346562 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:24.351943 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_793_161/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:30:24.872311 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1083 entries to flush up to offset 1395663 2024.10.22 02:30:24.886892 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} MergeTask::MergeProjectionsStage: Merge sorted 1178 rows, containing 13 columns (13 merged, 0 gathered) in 2.332878684 sec., 504.955533298533 rows/sec., 89.35 KiB/sec. 2024.10.22 02:30:25.029794 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} MergedBlockOutputStream: filled checksums 202410_1626_1641_3 (state Temporary) 2024.10.22 02:30:25.094547 [ 1393 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} MergedBlockOutputStream: filled checksums 202410_185_194_2 (state Temporary) 2024.10.22 02:30:25.117720 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1641_3 to 202410_1626_1641_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:25.125405 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1641_3} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1636_2, 202410_1641_1641_0] -> 202410_1626_1641_3 2024.10.22 02:30:25.260615 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:30:25.333807 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(31e3e075491f25dda340d7e994037c1f) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 31e3e075491f25dda340d7e994037c1f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:30:25.338757 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 577, file segment info: File segment: [0, 33554431], key: 31e3e075491f25dda340d7e994037c1f, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:30:25.341298 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:25.452459 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} MergeTask::MergeProjectionsStage: Merge sorted 837 rows, containing 32 columns (32 merged, 0 gathered) in 2.756285927 sec., 303.6695111348657 rows/sec., 22.84 KiB/sec. 2024.10.22 02:30:25.456382 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 16.72 MiB, will set to 1.56 GiB (RSS), difference: 7.73 MiB 2024.10.22 02:30:25.464167 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} MergedBlockOutputStream: filled checksums 202410_1_793_161 (state Temporary) 2024.10.22 02:30:25.473471 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 02:30:25.649758 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:25.698472 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:30:25.717562 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:30:25.733523 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:30:25.738770 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:25.762736 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:30:25.867579 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_926_926_0 (state Temporary) 2024.10.22 02:30:25.880606 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:30:25.904487 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:30:25.931309 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:30:25.947781 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:30:25.961719 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_926_926_0 to 202410_933_933_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:25.964663 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column FlashMinor2 2024.10.22 02:30:26.017603 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1395663 2024.10.22 02:30:26.098530 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_793_161 to 202410_1_793_161 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:26.103067 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_793_161} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_788_160, 202410_793_793_0] -> 202410_1_793_161 2024.10.22 02:30:26.110263 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/crv/lvjujajkuylqkaxbyloltcdignlxr (LogSeriesLimiter: on interval from 2024-10-22 02:29:59 to 2024-10-22 02:30:26 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:30:26.158795 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(ecb8818f1ebb0538de3250f9de585597) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: ecb8818f1ebb0538de3250f9de585597, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:30:26.165331 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4164, file segment info: File segment: [0, 33554431], key: ecb8818f1ebb0538de3250f9de585597, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:30:26.168177 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:26.350359 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:30:26.491025 [ 1180 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:30:26.538204 [ 1180 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:30:26.544000 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 97 entries to flush up to offset 109518 2024.10.22 02:30:27.252455 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:27.255575 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:27.433085 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1629_1629_0 (state Temporary) 2024.10.22 02:30:27.489572 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 151 bytes for 10788d3ba123f6b5dba1caa8110abef1:0) 2024.10.22 02:30:27.492546 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4b91c413e5801218a39a77665b5d9eb3, offset: 0, size: 151 2024.10.22 02:30:27.498695 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4b91c413e5801218a39a77665b5d9eb3 for removal 2024.10.22 02:30:27.503428 [ 1476 ] {} CacheMetadata: Key 4b91c413e5801218a39a77665b5d9eb3 is removed from metadata 2024.10.22 02:30:27.630736 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1629_1629_0 to 202410_1642_1642_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:27.721678 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 14913 2024.10.22 02:30:27.957585 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109518 2024.10.22 02:30:28.061430 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 130 entries to flush up to offset 106684 2024.10.22 02:30:28.162990 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:28.167664 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:28.262233 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_967_967_0 (state Temporary) 2024.10.22 02:30:28.347154 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:28.350683 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:28.395211 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_950_950_0 (state Temporary) 2024.10.22 02:30:28.452306 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_950_950_0 to 202410_956_956_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:28.529666 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_967_967_0 to 202410_980_980_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:28.608084 [ 1234 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_951_55 to 202410_952_952_0 2024.10.22 02:30:28.614279 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 106684 2024.10.22 02:30:28.616156 [ 1234 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.31 MiB using storage policy from min volume index 0 2024.10.22 02:30:28.643721 [ 1234 ] {} DiskLocal: Reserved 1.31 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:28.654409 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14913 2024.10.22 02:30:28.667519 [ 1184 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_975_340 to 202410_976_976_0 2024.10.22 02:30:28.677294 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_952_56} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_951_55 to 202410_952_952_0 into Compact with storage Full 2024.10.22 02:30:28.680672 [ 1184 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:28.688707 [ 1184 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:28.721114 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_952_56} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:28.738838 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_952_56} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_951_55, total 22860 rows starting from the beginning of the part 2024.10.22 02:30:28.742332 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_976_341} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_975_340 to 202410_976_976_0 into Compact with storage Full 2024.10.22 02:30:28.810201 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_976_341} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:28.814374 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_952_56} MergeTreeSequentialSource: Reading 2 marks from part 202410_952_952_0, total 74 rows starting from the beginning of the part 2024.10.22 02:30:28.826835 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_976_341} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_975_340, total 14947 rows starting from the beginning of the part 2024.10.22 02:30:28.866800 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_976_341} MergeTreeSequentialSource: Reading 2 marks from part 202410_976_976_0, total 10 rows starting from the beginning of the part 2024.10.22 02:30:28.996843 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(0b386c6f26156275c173d685791425bb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0b386c6f26156275c173d685791425bb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:30:29.001179 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 495, file segment info: File segment: [0, 33554431], key: 0b386c6f26156275c173d685791425bb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:30:29.006494 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:29.049682 [ 1742 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ixi/guiepckiywzaivgimlpatihnxwsnu, size 4164 2024.10.22 02:30:29.835110 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_976_341} MergeTask::MergeProjectionsStage: Merge sorted 14957 rows, containing 12 columns (12 merged, 0 gathered) in 1.096342554 sec., 13642.633814978233 rows/sec., 2.14 MiB/sec. 2024.10.22 02:30:29.846475 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1207_1537_90, 202410_1538_1538_0, 202410_1539_1539_0, 202410_1540_1540_0] 2024.10.22 02:30:29.863300 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1207_1537_90, 202410_1538_1538_0, 202410_1539_1539_0, 202410_1540_1540_0] 2024.10.22 02:30:29.881745 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1207_1537_90, 202410_1538_1538_0, 202410_1539_1539_0, 202410_1540_1540_0] 2024.10.22 02:30:29.952300 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column FromTag 2024.10.22 02:30:29.977726 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column FromTag 2024.10.22 02:30:30.002823 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column FromTag 2024.10.22 02:30:30.034228 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column FromTag 2024.10.22 02:30:30.063405 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column FromTag 2024.10.22 02:30:30.096832 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_976_341} MergedBlockOutputStream: filled checksums 202410_1_976_341 (state Temporary) 2024.10.22 02:30:30.108451 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column FromTag 2024.10.22 02:30:30.270155 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column FromTag 2024.10.22 02:30:30.378960 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f398dcfe249ec3597803c7dfcc31be36) : [0, 209139]: Setting download as failed: File segment: [0, 209139], key: f398dcfe249ec3597803c7dfcc31be36, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:30.383762 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f398dcfe249ec3597803c7dfcc31be36): No space left in cache to reserve 209140 bytes, will continue without cache download 2024.10.22 02:30:30.393745 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_976_341} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_976_341 to 202410_1_976_341 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:30.500752 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_976_341} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_975_340, 202410_976_976_0] -> 202410_1_976_341 2024.10.22 02:30:30.536551 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f398dcfe249ec3597803c7dfcc31be36, offset: 0, size: 0 2024.10.22 02:30:30.538528 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f398dcfe249ec3597803c7dfcc31be36 for removal 2024.10.22 02:30:30.548193 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_952_56} MergeTask::MergeProjectionsStage: Merge sorted 22934 rows, containing 13 columns (13 merged, 0 gathered) in 1.882779113 sec., 12180.929691458714 rows/sec., 5.31 MiB/sec. 2024.10.22 02:30:30.643221 [ 1476 ] {} CacheMetadata: Key f398dcfe249ec3597803c7dfcc31be36 is removed from metadata 2024.10.22 02:30:30.660381 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:30:30.733166 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.29 MiB, will set to 1.58 GiB (RSS), difference: 1.15 MiB 2024.10.22 02:30:30.944373 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_952_56} MergedBlockOutputStream: filled checksums 202410_747_952_56 (state Temporary) 2024.10.22 02:30:31.000512 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_952_56} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_952_56 to 202410_747_952_56 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:31.009210 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_952_56} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_951_55, 202410_952_952_0] -> 202410_747_952_56 2024.10.22 02:30:31.077030 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.48 MiB. 2024.10.22 02:30:31.415459 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:30:31.419764 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:30:31.601238 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_916_281, 202410_917_917_0] 2024.10.22 02:30:31.604487 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_916_281, 202410_917_917_0] 2024.10.22 02:30:31.618257 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_916_281, 202410_917_917_0] 2024.10.22 02:30:31.988000 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 109590 2024.10.22 02:30:32.043362 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 154 bytes for 222b95f90badfe8594400eee8bffac8f:0) 2024.10.22 02:30:32.046710 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7976eae7ef5c0d214f6f4ed5ada6d39e, offset: 0, size: 151 2024.10.22 02:30:32.054511 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7976eae7ef5c0d214f6f4ed5ada6d39e for removal 2024.10.22 02:30:32.059472 [ 1476 ] {} CacheMetadata: Key 7976eae7ef5c0d214f6f4ed5ada6d39e is removed from metadata 2024.10.22 02:30:32.293493 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:32.299331 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:32.465081 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1630_1630_0 (state Temporary) 2024.10.22 02:30:32.634331 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1630_1630_0 to 202410_1643_1643_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:32.667964 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109590 2024.10.22 02:30:33.025166 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 844 entries to flush up to offset 1396507 2024.10.22 02:30:33.373111 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(383b0160f5b6d83d0e81a340f1f3f70a) : [0, 463785]: Setting download as failed: File segment: [0, 463785], key: 383b0160f5b6d83d0e81a340f1f3f70a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:33.375730 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(383b0160f5b6d83d0e81a340f1f3f70a): No space left in cache to reserve 463786 bytes, will continue without cache download 2024.10.22 02:30:33.433770 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 8.49 MiB, will set to 1.56 GiB (RSS), difference: -8.98 MiB 2024.10.22 02:30:33.443272 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 383b0160f5b6d83d0e81a340f1f3f70a, offset: 0, size: 0 2024.10.22 02:30:33.446549 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 383b0160f5b6d83d0e81a340f1f3f70a for removal 2024.10.22 02:30:33.450954 [ 1476 ] {} CacheMetadata: Key 383b0160f5b6d83d0e81a340f1f3f70a is removed from metadata 2024.10.22 02:30:34.067821 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:34.074504 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:34.313352 [ 1256 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 250 ms. 2024.10.22 02:30:34.362594 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_927_927_0 (state Temporary) 2024.10.22 02:30:34.403744 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 150 bytes for 9250cca17c928b46d5e2b0524694adff:0) 2024.10.22 02:30:34.411836 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 10788d3ba123f6b5dba1caa8110abef1, offset: 0, size: 151 2024.10.22 02:30:34.417252 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 10788d3ba123f6b5dba1caa8110abef1 for removal 2024.10.22 02:30:34.421237 [ 1476 ] {} CacheMetadata: Key 10788d3ba123f6b5dba1caa8110abef1 is removed from metadata 2024.10.22 02:30:34.955597 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_927_927_0 to 202410_934_934_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:34.976225 [ 1216 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_929_929_0 to 202410_934_934_0 2024.10.22 02:30:34.987990 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1396507 2024.10.22 02:30:34.991934 [ 1216 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:35.002692 [ 1216 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:35.015036 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_929_929_0 to 202410_934_934_0 into Compact with storage Full 2024.10.22 02:30:35.049494 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_928_172/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:30:35.063196 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:35.068920 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_929_929_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:30:35.094081 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_930_930_0, total 770 rows starting from the beginning of the part 2024.10.22 02:30:35.113197 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_931_931_0, total 1542 rows starting from the beginning of the part 2024.10.22 02:30:35.134642 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_932_932_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:30:35.153695 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_933_933_0, total 1083 rows starting from the beginning of the part 2024.10.22 02:30:35.162484 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:35.164902 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:35.192671 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_934_934_0, total 844 rows starting from the beginning of the part 2024.10.22 02:30:35.561326 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 10.94 MiB, will set to 1.56 GiB (RSS), difference: -5.30 MiB 2024.10.22 02:30:35.609054 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c86ff4eebe60dcd2d8ee920ce18cb503) : [0, 169787]: Setting download as failed: File segment: [0, 169787], key: c86ff4eebe60dcd2d8ee920ce18cb503, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:35.664079 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c86ff4eebe60dcd2d8ee920ce18cb503): No space left in cache to reserve 169788 bytes, will continue without cache download 2024.10.22 02:30:35.688198 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c86ff4eebe60dcd2d8ee920ce18cb503, offset: 0, size: 0 2024.10.22 02:30:35.690996 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c86ff4eebe60dcd2d8ee920ce18cb503 for removal 2024.10.22 02:30:35.696363 [ 1476 ] {} CacheMetadata: Key c86ff4eebe60dcd2d8ee920ce18cb503 is removed from metadata 2024.10.22 02:30:35.933679 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_insert_201403_20_20_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:30:36.147121 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 125 entries to flush up to offset 106809 2024.10.22 02:30:36.158290 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 14923 2024.10.22 02:30:36.388971 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} MergeTask::MergeProjectionsStage: Merge sorted 6934 rows, containing 4 columns (4 merged, 0 gathered) in 1.378603693 sec., 5029.726842607551 rows/sec., 78.59 KiB/sec. 2024.10.22 02:30:36.409320 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:36.416303 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:36.503942 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:36.511169 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:36.562765 [ 1420 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_194_2/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:30:36.562224 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_951_951_0 (state Temporary) 2024.10.22 02:30:36.578252 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_968_968_0 (state Temporary) 2024.10.22 02:30:36.603088 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} MergedBlockOutputStream: filled checksums 202410_929_934_1 (state Temporary) 2024.10.22 02:30:36.656848 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_968_968_0 to 202410_981_981_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:36.685667 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 109650 2024.10.22 02:30:36.700596 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_862_161, 202410_863_868_1] 2024.10.22 02:30:36.711168 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_862_161, 202410_863_868_1] 2024.10.22 02:30:36.700969 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_929_934_1 to 202410_929_934_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:36.737663 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_862_161, 202410_863_868_1] 2024.10.22 02:30:36.739330 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_929_934_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_929_929_0, 202410_934_934_0] -> 202410_929_934_1 2024.10.22 02:30:36.747837 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14923 2024.10.22 02:30:36.786734 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:30:36.807612 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_976_341 to 202410_977_977_0 2024.10.22 02:30:36.816917 [ 1215 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:36.823715 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_951_951_0 to 202410_957_957_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:36.824414 [ 1215 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:36.857275 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_952_56 to 202410_953_953_0 2024.10.22 02:30:36.862383 [ 1257 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.32 MiB using storage policy from min volume index 0 2024.10.22 02:30:36.866297 [ 1257 ] {} DiskLocal: Reserved 1.32 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:36.873523 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:36.874726 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 106809 2024.10.22 02:30:36.877039 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:30:36.911711 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 152 bytes for 72805e68d500870fcbf59367a41dacba:0) 2024.10.22 02:30:36.921037 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 222b95f90badfe8594400eee8bffac8f, offset: 0, size: 154 2024.10.22 02:30:36.925488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 222b95f90badfe8594400eee8bffac8f for removal 2024.10.22 02:30:36.929684 [ 1476 ] {} CacheMetadata: Key 222b95f90badfe8594400eee8bffac8f is removed from metadata 2024.10.22 02:30:36.942534 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_977_342} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_976_341 to 202410_977_977_0 into Compact with storage Full 2024.10.22 02:30:37.000809 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_977_342} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:37.044564 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_953_57} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_952_56 to 202410_953_953_0 into Compact with storage Full 2024.10.22 02:30:37.106711 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_953_57} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:37.114988 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_977_342} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_976_341, total 14957 rows starting from the beginning of the part 2024.10.22 02:30:37.120946 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_953_57} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_952_56, total 22934 rows starting from the beginning of the part 2024.10.22 02:30:37.139277 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1631_1631_0 (state Temporary) 2024.10.22 02:30:37.151023 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_977_342} MergeTreeSequentialSource: Reading 2 marks from part 202410_977_977_0, total 14 rows starting from the beginning of the part 2024.10.22 02:30:37.158380 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_953_57} MergeTreeSequentialSource: Reading 2 marks from part 202410_953_953_0, total 91 rows starting from the beginning of the part 2024.10.22 02:30:37.379057 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1631_1631_0 to 202410_1644_1644_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:37.446283 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109650 2024.10.22 02:30:37.643432 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 802 2024.10.22 02:30:37.727651 [ 1153 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 328 ms. 2024.10.22 02:30:37.761024 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 13.23 MiB, will set to 1.56 GiB (RSS), difference: -23.40 MiB 2024.10.22 02:30:38.036682 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:38.039506 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:38.106878 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_784_784_0 (state Temporary) 2024.10.22 02:30:38.200553 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(066defc7e40c520dda0e76c89f2386d0) : [0, 1235]: Setting download as failed: File segment: [0, 1235], key: 066defc7e40c520dda0e76c89f2386d0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:38.203365 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(066defc7e40c520dda0e76c89f2386d0): No space left in cache to reserve 1236 bytes, will continue without cache download 2024.10.22 02:30:38.221653 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 066defc7e40c520dda0e76c89f2386d0, offset: 0, size: 0 2024.10.22 02:30:38.225037 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 066defc7e40c520dda0e76c89f2386d0 for removal 2024.10.22 02:30:38.229412 [ 1476 ] {} CacheMetadata: Key 066defc7e40c520dda0e76c89f2386d0 is removed from metadata 2024.10.22 02:30:38.209824 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_784_784_0 to 202410_794_794_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:38.344289 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_977_342} MergeTask::MergeProjectionsStage: Merge sorted 14971 rows, containing 12 columns (12 merged, 0 gathered) in 1.405639557 sec., 10650.667822661453 rows/sec., 1.67 MiB/sec. 2024.10.22 02:30:38.396191 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_977_342} MergedBlockOutputStream: filled checksums 202410_1_977_342 (state Temporary) 2024.10.22 02:30:38.410955 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 802 2024.10.22 02:30:38.488397 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_917_282, 202410_918_918_0] 2024.10.22 02:30:38.492291 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_917_282, 202410_918_918_0] 2024.10.22 02:30:38.508813 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_917_282, 202410_918_918_0] 2024.10.22 02:30:38.562680 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_977_342} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_977_342 to 202410_1_977_342 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:38.565371 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_953_57} MergeTask::MergeProjectionsStage: Merge sorted 23025 rows, containing 13 columns (13 merged, 0 gathered) in 1.624972639 sec., 14169.469348215898 rows/sec., 6.17 MiB/sec. 2024.10.22 02:30:38.565467 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_977_342} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_976_341, 202410_977_977_0] -> 202410_1_977_342 2024.10.22 02:30:38.599799 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:30:38.647615 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_953_57} MergedBlockOutputStream: filled checksums 202410_747_953_57 (state Temporary) 2024.10.22 02:30:38.666246 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_194_194_0 (state Temporary) 2024.10.22 02:30:39.140321 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_953_57} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_953_57 to 202410_747_953_57 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:39.145611 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_953_57} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_952_56, 202410_953_953_0] -> 202410_747_953_57 2024.10.22 02:30:39.266544 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.49 MiB. 2024.10.22 02:30:39.456744 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_185_194_2 to 202410_185_194_2 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:39.474098 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 9.60 MiB, will set to 1.56 GiB (RSS), difference: 25.63 MiB 2024.10.22 02:30:39.515578 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 2 file segments (while reserving 185 bytes for ea06d639df091279f22eeb06d3414a33:0) 2024.10.22 02:30:39.564247 [ 1400 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_194_2} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_185_189_1, 202410_194_194_0] -> 202410_185_194_2 2024.10.22 02:30:39.564038 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 72805e68d500870fcbf59367a41dacba, offset: 0, size: 152 2024.10.22 02:30:39.571272 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 72805e68d500870fcbf59367a41dacba for removal 2024.10.22 02:30:39.573585 [ 1476 ] {} CacheMetadata: Key 72805e68d500870fcbf59367a41dacba is removed from metadata 2024.10.22 02:30:39.574405 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9250cca17c928b46d5e2b0524694adff, offset: 0, size: 150 2024.10.22 02:30:39.584181 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9250cca17c928b46d5e2b0524694adff for removal 2024.10.22 02:30:39.626338 [ 1476 ] {} CacheMetadata: Key 9250cca17c928b46d5e2b0524694adff is removed from metadata 2024.10.22 02:30:40.209768 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 14.59 MiB. 2024.10.22 02:30:40.950180 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2f2410fd62ab3c9e9e092ffca92426b3) : [0, 953655]: Setting download as failed: File segment: [0, 953655], key: 2f2410fd62ab3c9e9e092ffca92426b3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:40.951841 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2f2410fd62ab3c9e9e092ffca92426b3): No space left in cache to reserve 953656 bytes, will continue without cache download 2024.10.22 02:30:40.987648 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2f2410fd62ab3c9e9e092ffca92426b3, offset: 0, size: 0 2024.10.22 02:30:40.993612 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2f2410fd62ab3c9e9e092ffca92426b3 for removal 2024.10.22 02:30:41.105328 [ 1476 ] {} CacheMetadata: Key 2f2410fd62ab3c9e9e092ffca92426b3 is removed from metadata 2024.10.22 02:30:41.418253 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:30:41.432520 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:30:41.456792 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 109728 2024.10.22 02:30:41.548989 [ 1162 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:30:41.572524 [ 1162 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:30:41.755876 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:41.760207 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:41.846764 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 11.45 MiB, will set to 1.56 GiB (RSS), difference: 2.21 MiB 2024.10.22 02:30:41.962129 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1632_1632_0 (state Temporary) 2024.10.22 02:30:41.991408 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1544 entries to flush up to offset 1398051 2024.10.22 02:30:41.993770 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1540_91, 202410_1541_1541_0] 2024.10.22 02:30:41.998271 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1540_91, 202410_1541_1541_0] 2024.10.22 02:30:42.038765 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1540_91, 202410_1541_1541_0] 2024.10.22 02:30:42.055142 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1632_1632_0 to 202410_1645_1645_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:42.208484 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109728 2024.10.22 02:30:42.440315 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 187 bytes for f164c568d64f6abdebe42dd4700dbc37:0) 2024.10.22 02:30:42.445158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ea06d639df091279f22eeb06d3414a33, offset: 0, size: 185 2024.10.22 02:30:42.448760 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ea06d639df091279f22eeb06d3414a33 for removal 2024.10.22 02:30:42.451771 [ 1476 ] {} CacheMetadata: Key ea06d639df091279f22eeb06d3414a33 is removed from metadata 2024.10.22 02:30:43.004206 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:43.009021 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:43.139482 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_928_928_0 (state Temporary) 2024.10.22 02:30:43.241576 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_928_928_0 to 202410_935_935_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:43.293223 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1398051 2024.10.22 02:30:43.810946 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2d35ed4dc2be705314e1ff5dfb1b6516) : [0, 1100519]: Setting download as failed: File segment: [0, 1100519], key: 2d35ed4dc2be705314e1ff5dfb1b6516, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:43.818116 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2d35ed4dc2be705314e1ff5dfb1b6516): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:30:44.075380 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2d35ed4dc2be705314e1ff5dfb1b6516, offset: 0, size: 0 2024.10.22 02:30:44.087500 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2d35ed4dc2be705314e1ff5dfb1b6516 for removal 2024.10.22 02:30:44.095476 [ 1476 ] {} CacheMetadata: Key 2d35ed4dc2be705314e1ff5dfb1b6516 is removed from metadata 2024.10.22 02:30:44.249564 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 14949 2024.10.22 02:30:44.374593 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:44.378737 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:44.416820 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 11.76 MiB, will set to 1.56 GiB (RSS), difference: 6.25 MiB 2024.10.22 02:30:44.421254 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 96 entries to flush up to offset 106905 2024.10.22 02:30:44.536727 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_969_969_0 (state Temporary) 2024.10.22 02:30:44.590951 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:44.596483 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:44.729820 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_969_969_0 to 202410_982_982_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:44.758908 [ 1172 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_977_342 to 202410_978_978_0 2024.10.22 02:30:44.766954 [ 1172 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:44.771061 [ 1172 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:44.851906 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_952_952_0 (state Temporary) 2024.10.22 02:30:44.853559 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_978_343} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_977_342 to 202410_978_978_0 into Compact with storage Full 2024.10.22 02:30:44.877270 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_978_343} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:44.887753 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_978_343} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_977_342, total 14971 rows starting from the beginning of the part 2024.10.22 02:30:44.907624 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_978_343} MergeTreeSequentialSource: Reading 2 marks from part 202410_978_978_0, total 10 rows starting from the beginning of the part 2024.10.22 02:30:44.989160 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14949 2024.10.22 02:30:45.040354 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_952_952_0 to 202410_958_958_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:45.067632 [ 1160 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_953_57 to 202410_954_954_0 2024.10.22 02:30:45.083691 [ 1160 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.32 MiB using storage policy from min volume index 0 2024.10.22 02:30:45.089291 [ 1160 ] {} DiskLocal: Reserved 1.32 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:45.181733 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 106905 2024.10.22 02:30:45.303225 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileCache(s3_cache): Will evict 1 file segments (while reserving 155 bytes for c543fbc33571b5267a71fdaddf899661:0) 2024.10.22 02:30:45.306514 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f164c568d64f6abdebe42dd4700dbc37, offset: 0, size: 187 2024.10.22 02:30:45.311539 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f164c568d64f6abdebe42dd4700dbc37 for removal 2024.10.22 02:30:45.326835 [ 1476 ] {} CacheMetadata: Key f164c568d64f6abdebe42dd4700dbc37 is removed from metadata 2024.10.22 02:30:45.348299 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_954_58} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_953_57 to 202410_954_954_0 into Compact with storage Full 2024.10.22 02:30:45.460975 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_954_58} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:45.479264 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_954_58} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_953_57, total 23025 rows starting from the beginning of the part 2024.10.22 02:30:45.650228 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_954_58} MergeTreeSequentialSource: Reading 2 marks from part 202410_954_954_0, total 70 rows starting from the beginning of the part 2024.10.22 02:30:45.819094 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_978_343} MergeTask::MergeProjectionsStage: Merge sorted 14981 rows, containing 12 columns (12 merged, 0 gathered) in 0.970820747 sec., 15431.27301954951 rows/sec., 2.42 MiB/sec. 2024.10.22 02:30:45.920185 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 803 2024.10.22 02:30:46.117606 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_978_343} MergedBlockOutputStream: filled checksums 202410_1_978_343 (state Temporary) 2024.10.22 02:30:46.246153 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 61 entries to flush up to offset 109789 2024.10.22 02:30:46.360485 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:46.363205 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:30:46.429264 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:46.437265 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:30:46.442301 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_785_785_0 (state Temporary) 2024.10.22 02:30:46.519704 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1633_1633_0 (state Temporary) 2024.10.22 02:30:46.536270 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_785_785_0 to 202410_795_795_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:46.566576 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_978_343} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_978_343 to 202410_1_978_343 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:46.567123 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_928_172/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:30:46.574212 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_978_343} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_977_342, 202410_978_978_0] -> 202410_1_978_343 2024.10.22 02:30:46.611423 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1633_1633_0 to 202410_1646_1646_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:46.627730 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:30:46.656063 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1641_3 to 202410_1646_1646_0 2024.10.22 02:30:46.670320 [ 1180 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:46.676623 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 11.57 MiB, will set to 1.56 GiB (RSS), difference: -25.96 MiB 2024.10.22 02:30:46.680163 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109789 2024.10.22 02:30:46.681683 [ 1180 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:30:46.687352 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bdd9ec4067e41ce71f38079c81bcb6be) : [0, 435423]: Setting download as failed: File segment: [0, 435423], key: bdd9ec4067e41ce71f38079c81bcb6be, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:46.694646 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bdd9ec4067e41ce71f38079c81bcb6be): No space left in cache to reserve 435424 bytes, will continue without cache download 2024.10.22 02:30:46.749236 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bdd9ec4067e41ce71f38079c81bcb6be, offset: 0, size: 0 2024.10.22 02:30:46.754232 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bdd9ec4067e41ce71f38079c81bcb6be for removal 2024.10.22 02:30:46.759920 [ 1476 ] {} CacheMetadata: Key bdd9ec4067e41ce71f38079c81bcb6be is removed from metadata 2024.10.22 02:30:46.790041 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1641_3 to 202410_1646_1646_0 into Compact with storage Full 2024.10.22 02:30:46.954651 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:46.966485 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1641_3, total 1178 rows starting from the beginning of the part 2024.10.22 02:30:47.016491 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 803 2024.10.22 02:30:47.035513 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1642_1642_0, total 97 rows starting from the beginning of the part 2024.10.22 02:30:47.121796 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1643_1643_0, total 72 rows starting from the beginning of the part 2024.10.22 02:30:47.150470 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1644_1644_0, total 60 rows starting from the beginning of the part 2024.10.22 02:30:47.274854 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1645_1645_0, total 78 rows starting from the beginning of the part 2024.10.22 02:30:47.310079 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} MergeTreeSequentialSource: Reading 2 marks from part 202410_1646_1646_0, total 61 rows starting from the beginning of the part 2024.10.22 02:30:47.689723 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_918_283, 202410_919_919_0] 2024.10.22 02:30:47.693523 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_918_283, 202410_919_919_0] 2024.10.22 02:30:47.705601 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_918_283, 202410_919_919_0] 2024.10.22 02:30:47.714304 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_954_58} MergeTask::MergeProjectionsStage: Merge sorted 23095 rows, containing 13 columns (13 merged, 0 gathered) in 2.377556811 sec., 9713.753165917515 rows/sec., 4.23 MiB/sec. 2024.10.22 02:30:47.890394 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_954_58} MergedBlockOutputStream: filled checksums 202410_747_954_58 (state Temporary) 2024.10.22 02:30:48.040260 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1541_92, 202410_1542_1542_0] 2024.10.22 02:30:48.042091 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1541_92, 202410_1542_1542_0] 2024.10.22 02:30:48.084654 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_954_58} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_954_58 to 202410_747_954_58 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:48.092046 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_954_58} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_953_57, 202410_954_954_0] -> 202410_747_954_58 2024.10.22 02:30:48.218973 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1541_92, 202410_1542_1542_0] 2024.10.22 02:30:48.251558 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.48 MiB. 2024.10.22 02:30:48.314732 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3b6597c0d8ccc9d87bf4b719014f4e87) : [0, 107918]: Setting download as failed: File segment: [0, 107918], key: 3b6597c0d8ccc9d87bf4b719014f4e87, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:30:48.317006 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3b6597c0d8ccc9d87bf4b719014f4e87): No space left in cache to reserve 107919 bytes, will continue without cache download 2024.10.22 02:30:48.359094 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3b6597c0d8ccc9d87bf4b719014f4e87, offset: 0, size: 0 2024.10.22 02:30:48.360724 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3b6597c0d8ccc9d87bf4b719014f4e87 for removal 2024.10.22 02:30:48.364767 [ 1476 ] {} CacheMetadata: Key 3b6597c0d8ccc9d87bf4b719014f4e87 is removed from metadata 2024.10.22 02:30:48.365327 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/wfc/drfvqxuqzqvzezjlfnoldbmjljlsl. 2024.10.22 02:30:48.441105 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} MergeTask::MergeProjectionsStage: Merge sorted 1546 rows, containing 13 columns (13 merged, 0 gathered) in 1.673649474 sec., 923.7298634014926 rows/sec., 161.74 KiB/sec. 2024.10.22 02:30:48.512682 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/bhj/bdjswkdslqasbgbvesxaeuploafkb. 2024.10.22 02:30:48.585646 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} MergedBlockOutputStream: filled checksums 202410_1626_1646_4 (state Temporary) 2024.10.22 02:30:48.671611 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 10.12 MiB, will set to 1.56 GiB (RSS), difference: 19.18 MiB 2024.10.22 02:30:48.749265 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1646_4 to 202410_1626_1646_4 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:48.755699 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1646_4} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1641_3, 202410_1646_1646_0] -> 202410_1626_1646_4 2024.10.22 02:30:48.792037 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:30:48.983662 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_194_194_0 to 202410_197_197_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:49.585339 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/bhj/bdjswkdslqasbgbvesxaeuploafkb. 2024.10.22 02:30:49.592460 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Renaming temporary part tmp_insert_201403_20_20_0 to 201403_20_20_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:49.607987 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/wfc/drfvqxuqzqvzezjlfnoldbmjljlsl. 2024.10.22 02:30:49.659975 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(13cec3f94b125a1b8bdab3985fcd63eb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 13cec3f94b125a1b8bdab3985fcd63eb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 02:30:49.660621 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:30:49.765072 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 948310, file segment info: File segment: [0, 33554431], key: 13cec3f94b125a1b8bdab3985fcd63eb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 02:30:49.768544 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:50.300386 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1399591 2024.10.22 02:30:50.530012 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1000.86 MiB, peak 2.00 GiB, free memory in arenas 10.71 MiB, will set to 1.46 GiB (RSS), difference: 494.58 MiB 2024.10.22 02:30:50.647274 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(6f411625dfbe66254dbfbe6312403f30) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6f411625dfbe66254dbfbe6312403f30, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 02:30:50.649658 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1234, file segment info: File segment: [0, 33554431], key: 6f411625dfbe66254dbfbe6312403f30, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1387, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1387, kind: Regular, unbound: 0 2024.10.22 02:30:50.652330 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:50.697630 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 109858 2024.10.22 02:30:50.807126 [ 1721 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/crv/lvjujajkuylqkaxbyloltcdignlxr, size 948310 2024.10.22 02:30:50.894337 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:51.005246 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:51.088375 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1634_1634_0 (state Temporary) 2024.10.22 02:30:51.125975 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:51.129558 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:51.145613 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.213474216 sec. 2024.10.22 02:30:51.252240 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1634_1634_0 to 202410_1647_1647_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:51.257464 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:30:51.268971 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:30:51.313589 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:30:51.343751 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109858 2024.10.22 02:30:51.348077 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_929_929_0 (state Temporary) 2024.10.22 02:30:51.358833 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:30:51.417376 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:30:51.421183 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:30:51.479135 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:30:51.557348 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:30:51.561547 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_929_929_0 to 202410_936_936_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:51.590677 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:30:51.648006 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1399591 2024.10.22 02:30:51.664319 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:30:51.698648 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column NetMajor 2024.10.22 02:30:51.755975 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jym/auigyqzuouwelnsbtevujzenpfdfl (LogSeriesLimiter: on interval from 2024-10-22 02:30:26 to 2024-10-22 02:30:51 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:30:52.012109 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7837 2024.10.22 02:30:52.217743 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(fbd086f2829bae4cedf93b15658ef0ab) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fbd086f2829bae4cedf93b15658ef0ab, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:30:52.224658 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 225349, file segment info: File segment: [0, 33554431], key: fbd086f2829bae4cedf93b15658ef0ab, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:30:52.229973 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:52.360766 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1542_93, 202410_1543_1543_0] 2024.10.22 02:30:52.363053 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1542_93, 202410_1543_1543_0] 2024.10.22 02:30:52.376787 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1542_93, 202410_1543_1543_0] 2024.10.22 02:30:52.493499 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 14970 2024.10.22 02:30:52.541298 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 14.60 MiB, will set to 1.32 GiB (RSS), difference: -297.98 MiB 2024.10.22 02:30:52.711738 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 108 entries to flush up to offset 107013 2024.10.22 02:30:52.725577 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:52.828155 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:52.912077 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:52.924045 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:52.994614 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_970_970_0 (state Temporary) 2024.10.22 02:30:53.041174 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_953_953_0 (state Temporary) 2024.10.22 02:30:53.053375 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_970_970_0 to 202410_983_983_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:53.071128 [ 1209 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_978_343 to 202410_979_979_0 2024.10.22 02:30:53.072686 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_953_953_0 to 202410_959_959_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:53.075577 [ 1209 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:53.078073 [ 1209 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:53.111662 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14970 2024.10.22 02:30:53.116925 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_979_344} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_978_343 to 202410_979_979_0 into Compact with storage Full 2024.10.22 02:30:53.128336 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 107013 2024.10.22 02:30:53.207223 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_979_344} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:53.215348 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(073dd9ced206aebc204bc2c632c67da3) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 073dd9ced206aebc204bc2c632c67da3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:30:53.218927 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 649, file segment info: File segment: [0, 33554431], key: 073dd9ced206aebc204bc2c632c67da3, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:30:53.222032 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_979_344} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_978_343, total 14981 rows starting from the beginning of the part 2024.10.22 02:30:53.222806 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:53.222892 [ 1209 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_954_58 to 202410_955_955_0 2024.10.22 02:30:53.250819 [ 1209 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 02:30:53.254238 [ 1209 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:53.268982 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_979_344} MergeTreeSequentialSource: Reading 2 marks from part 202410_979_979_0, total 12 rows starting from the beginning of the part 2024.10.22 02:30:53.288269 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_955_59} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_954_58 to 202410_955_955_0 into Compact with storage Full 2024.10.22 02:30:53.336179 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_955_59} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:30:53.351212 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_955_59} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_954_58, total 23095 rows starting from the beginning of the part 2024.10.22 02:30:53.371091 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_955_59} MergeTreeSequentialSource: Reading 2 marks from part 202410_955_955_0, total 104 rows starting from the beginning of the part 2024.10.22 02:30:53.463600 [ 1545 ] {} SystemLog (system.part_log): Flushing system log, 1 entries to flush up to offset 71 2024.10.22 02:30:54.031705 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:54.040449 [ 1545 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:30:54.198514 [ 1545 ] {} MergedBlockOutputStream: filled checksums 202410_49_49_0 (state Temporary) 2024.10.22 02:30:54.243477 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(4ca69eea07ba69a4b32a57c860711d75) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 4ca69eea07ba69a4b32a57c860711d75, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:30:54.267653 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 46263, file segment info: File segment: [0, 33554431], key: 4ca69eea07ba69a4b32a57c860711d75, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:30:54.270912 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:54.283359 [ 1545 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Renaming temporary part tmp_insert_202410_49_49_0 to 202410_49_49_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:54.327141 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_919_284, 202410_920_920_0] 2024.10.22 02:30:54.332064 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_919_284, 202410_920_920_0] 2024.10.22 02:30:54.345292 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_919_284, 202410_920_920_0] 2024.10.22 02:30:54.404008 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_979_344} MergeTask::MergeProjectionsStage: Merge sorted 14993 rows, containing 12 columns (12 merged, 0 gathered) in 1.292746472 sec., 11597.788371299459 rows/sec., 1.82 MiB/sec. 2024.10.22 02:30:54.471625 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_979_344} MergedBlockOutputStream: filled checksums 202410_1_979_344 (state Temporary) 2024.10.22 02:30:54.520207 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 804 2024.10.22 02:30:54.560034 [ 1545 ] {} SystemLog (system.part_log): Flushed system log up to offset 71 2024.10.22 02:30:54.598343 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.42 GiB, peak 2.00 GiB, free memory in arenas 16.86 MiB, will set to 1.39 GiB (RSS), difference: -27.94 MiB 2024.10.22 02:30:54.771306 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:54.774891 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:54.856616 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_786_786_0 (state Temporary) 2024.10.22 02:30:55.096787 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_786_786_0 to 202410_796_796_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:55.177724 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_955_59} MergeTask::MergeProjectionsStage: Merge sorted 23199 rows, containing 13 columns (13 merged, 0 gathered) in 1.90661099 sec., 12167.6630008306 rows/sec., 5.30 MiB/sec. 2024.10.22 02:30:55.273465 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 804 2024.10.22 02:30:55.361805 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_979_344} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_979_344 to 202410_1_979_344 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:55.364622 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 84 entries to flush up to offset 109942 2024.10.22 02:30:55.367625 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_979_344} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_978_343, 202410_979_979_0] -> 202410_1_979_344 2024.10.22 02:30:55.391844 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:30:55.462446 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_955_59} MergedBlockOutputStream: filled checksums 202410_747_955_59 (state Temporary) 2024.10.22 02:30:55.458479 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(2c75b0c4d20db9c520b1c1d4304cf0f7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2c75b0c4d20db9c520b1c1d4304cf0f7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:30:55.473040 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 621, file segment info: File segment: [0, 33554431], key: 2c75b0c4d20db9c520b1c1d4304cf0f7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:30:55.483181 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:30:55.734729 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:55.740544 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:30:55.808623 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:30:55.830543 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:30:55.848743 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:30:55.972417 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:30:56.005064 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:30:56.017636 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1635_1635_0 (state Temporary) 2024.10.22 02:30:56.045236 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:30:56.081605 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1635_1635_0 to 202410_1648_1648_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:56.139989 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column HasGCLID 2024.10.22 02:30:56.188899 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_955_59} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_955_59 to 202410_747_955_59 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:30:56.194911 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_955_59} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_954_58, 202410_955_955_0] -> 202410_747_955_59 2024.10.22 02:30:56.195456 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/yem/ccebndaxzxjxfrfqsjqosmsidnnpx (LogSeriesLimiter: on interval from 2024-10-22 02:30:51 to 2024-10-22 02:30:56 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:30:56.239620 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109942 2024.10.22 02:30:56.351919 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.52 MiB. 2024.10.22 02:30:56.517287 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1543_94, 202410_1544_1544_0] 2024.10.22 02:30:56.520967 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1543_94, 202410_1544_1544_0] 2024.10.22 02:30:56.534189 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1543_94, 202410_1544_1544_0] 2024.10.22 02:30:56.577424 [ 1252 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:30:56.627643 [ 1252 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:30:57.358353 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.47 GiB, peak 2.00 GiB, free memory in arenas 16.50 MiB, will set to 1.43 GiB (RSS), difference: -40.63 MiB 2024.10.22 02:30:57.642534 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_928_172/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:30:57.798529 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_743_151, 202410_744_744_0, 202410_745_745_0, 202410_746_746_0, 202410_747_747_0, 202410_748_748_0] 2024.10.22 02:30:57.803350 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_743_151, 202410_744_744_0, 202410_745_745_0, 202410_746_746_0, 202410_747_747_0, 202410_748_748_0] 2024.10.22 02:30:57.844150 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_743_151, 202410_744_744_0, 202410_745_745_0, 202410_746_746_0, 202410_747_747_0, 202410_748_748_0] 2024.10.22 02:30:58.753049 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 1401133 2024.10.22 02:30:59.223151 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_897_30, 202410_898_898_0, 202410_899_899_0, 202410_900_900_0, 202410_901_901_0, 202410_902_902_0] 2024.10.22 02:30:59.232464 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_897_30, 202410_898_898_0, 202410_899_899_0, 202410_900_900_0, 202410_901_901_0, 202410_902_902_0] 2024.10.22 02:30:59.355105 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_928_172} MergeTask::MergeProjectionsStage: Merge sorted 1403163 rows, containing 4 columns (4 merged, 0 gathered) in 41.44927991 sec., 33852.5302018932 rows/sec., 507.72 KiB/sec. 2024.10.22 02:30:59.358056 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_897_30, 202410_898_898_0, 202410_899_899_0, 202410_900_900_0, 202410_901_901_0, 202410_902_902_0] 2024.10.22 02:30:59.441563 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 16.33 MiB, will set to 1.48 GiB (RSS), difference: -36.06 MiB 2024.10.22 02:30:59.572071 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 7878 2024.10.22 02:30:59.644761 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_928_172} MergedBlockOutputStream: filled checksums 202410_1_928_172 (state Temporary) 2024.10.22 02:30:59.920482 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:30:59.925155 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:00.184987 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_930_930_0 (state Temporary) 2024.10.22 02:31:00.255938 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_930_930_0 to 202410_937_937_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:00.321651 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 45 entries to flush up to offset 109987 2024.10.22 02:31:00.364594 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1401133 2024.10.22 02:31:00.471057 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:00.478643 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:00.503539 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 17.05 MiB, will set to 1.49 GiB (RSS), difference: -21.69 MiB 2024.10.22 02:31:00.546214 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1636_1636_0 (state Temporary) 2024.10.22 02:31:00.586937 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1636_1636_0 to 202410_1649_1649_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:00.616361 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 14983 2024.10.22 02:31:00.647403 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 90 entries to flush up to offset 107103 2024.10.22 02:31:00.648566 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 109987 2024.10.22 02:31:00.896816 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:00.902821 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:01.000312 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_954_954_0 (state Temporary) 2024.10.22 02:31:01.005566 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:01.009074 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:01.064491 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_954_954_0 to 202410_960_960_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:01.071641 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_971_971_0 (state Temporary) 2024.10.22 02:31:01.125777 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_955_59 to 202410_956_956_0 2024.10.22 02:31:01.135163 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_971_971_0 to 202410_984_984_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:01.135776 [ 1154 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 02:31:01.141312 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 107103 2024.10.22 02:31:01.150984 [ 1154 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:01.177398 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14983 2024.10.22 02:31:01.256508 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_956_60} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_955_59 to 202410_956_956_0 into Compact with storage Full 2024.10.22 02:31:01.262120 [ 1150 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_979_344 to 202410_980_980_0 2024.10.22 02:31:01.266418 [ 1150 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:01.268666 [ 1150 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:01.281246 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_956_60} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:01.408245 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_956_60} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_955_59, total 23199 rows starting from the beginning of the part 2024.10.22 02:31:01.410353 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_980_345} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_979_344 to 202410_980_980_0 into Compact with storage Full 2024.10.22 02:31:01.422267 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:31:01.426727 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:31:01.434354 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_956_60} MergeTreeSequentialSource: Reading 2 marks from part 202410_956_956_0, total 130 rows starting from the beginning of the part 2024.10.22 02:31:01.446186 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_980_345} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:01.462207 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_980_345} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_979_344, total 14993 rows starting from the beginning of the part 2024.10.22 02:31:01.478655 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_980_345} MergeTreeSequentialSource: Reading 2 marks from part 202410_980_980_0, total 26 rows starting from the beginning of the part 2024.10.22 02:31:01.552981 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1544_95, 202410_1545_1545_0] 2024.10.22 02:31:01.555024 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1544_95, 202410_1545_1545_0] 2024.10.22 02:31:01.564270 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1544_95, 202410_1545_1545_0] 2024.10.22 02:31:01.758316 [ 1425 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:31:02.555502 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 15.48 MiB, will set to 1.55 GiB (RSS), difference: -90.93 MiB 2024.10.22 02:31:02.721556 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_956_60} MergeTask::MergeProjectionsStage: Merge sorted 23329 rows, containing 13 columns (13 merged, 0 gathered) in 1.470106981 sec., 15868.913148164964 rows/sec., 6.91 MiB/sec. 2024.10.22 02:31:02.775849 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 805 2024.10.22 02:31:02.820551 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_956_60/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:31:03.041697 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_956_60} MergedBlockOutputStream: filled checksums 202410_747_956_60 (state Temporary) 2024.10.22 02:31:03.099479 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:03.109035 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:03.166782 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_980_345} MergeTask::MergeProjectionsStage: Merge sorted 15019 rows, containing 12 columns (12 merged, 0 gathered) in 1.767001417 sec., 8499.710218398766 rows/sec., 1.33 MiB/sec. 2024.10.22 02:31:03.204183 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_787_787_0 (state Temporary) 2024.10.22 02:31:03.314547 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_980_345} MergedBlockOutputStream: filled checksums 202410_1_980_345 (state Temporary) 2024.10.22 02:31:03.344154 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_928_172} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_928_172 to 202410_1_928_172 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:03.344073 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_875_875_0, 202410_876_876_0, 202410_877_877_0, 202410_878_878_0, 202410_879_879_0, 202410_880_880_0] 2024.10.22 02:31:03.356074 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_875_875_0, 202410_876_876_0, 202410_877_877_0, 202410_878_878_0, 202410_879_879_0, 202410_880_880_0] 2024.10.22 02:31:03.349001 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_928_172} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_922_171, 202410_923_928_1] -> 202410_1_928_172 2024.10.22 02:31:03.382940 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:31:03.401019 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_787_787_0 to 202410_797_797_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:03.506174 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_875_875_0, 202410_876_876_0, 202410_877_877_0, 202410_878_878_0, 202410_879_879_0, 202410_880_880_0] 2024.10.22 02:31:03.550499 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_956_60} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_956_60 to 202410_747_956_60 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:03.555990 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_956_60} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_955_59, 202410_956_956_0] -> 202410_747_956_60 2024.10.22 02:31:03.579485 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 805 2024.10.22 02:31:03.753656 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 25.53 MiB. 2024.10.22 02:31:03.863004 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_980_345} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_980_345 to 202410_1_980_345 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:03.867896 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_980_345} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_979_344, 202410_980_980_0] -> 202410_1_980_345 2024.10.22 02:31:03.904437 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:31:04.298709 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_920_285, 202410_921_921_0] 2024.10.22 02:31:04.303593 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_920_285, 202410_921_921_0] 2024.10.22 02:31:04.316528 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_920_285, 202410_921_921_0] 2024.10.22 02:31:04.587951 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 2.00 GiB, free memory in arenas 14.18 MiB, will set to 1.63 GiB (RSS), difference: -46.13 MiB 2024.10.22 02:31:04.684314 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 110056 2024.10.22 02:31:05.368487 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:05.370935 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:05.479332 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1637_1637_0 (state Temporary) 2024.10.22 02:31:05.959741 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1637_1637_0 to 202410_1650_1650_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:06.123423 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110056 2024.10.22 02:31:06.614743 [ 1192 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 02:31:06.645585 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.31 GiB, peak 2.00 GiB, free memory in arenas 11.97 MiB, will set to 1.73 GiB (RSS), difference: 429.87 MiB 2024.10.22 02:31:07.370573 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1925 entries to flush up to offset 1403058 2024.10.22 02:31:07.439688 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1545_96, 202410_1546_1546_0, 202410_1547_1547_0, 202410_1548_1548_0, 202410_1549_1549_0, 202410_1550_1550_0] 2024.10.22 02:31:07.443412 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1545_96, 202410_1546_1546_0, 202410_1547_1547_0, 202410_1548_1548_0, 202410_1549_1549_0, 202410_1550_1550_0] 2024.10.22 02:31:07.483887 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1545_96, 202410_1546_1546_0, 202410_1547_1547_0, 202410_1548_1548_0, 202410_1549_1549_0, 202410_1550_1550_0] 2024.10.22 02:31:08.131692 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:08.137078 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:08.152302 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): Trying to reserve 495.65 MiB using storage policy from min volume index 0 2024.10.22 02:31:08.345462 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_931_931_0 (state Temporary) 2024.10.22 02:31:08.390440 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_931_931_0 to 202410_938_938_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:08.424839 [ 1198 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_928_172 to 202410_929_934_1 2024.10.22 02:31:08.430427 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1403058 2024.10.22 02:31:08.434427 [ 1198 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.09 MiB using storage policy from min volume index 0 2024.10.22 02:31:08.436872 [ 1198 ] {} DiskLocal: Reserved 1.09 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:08.465342 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_934_173} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_928_172 to 202410_929_934_1 into Wide with storage Full 2024.10.22 02:31:08.478675 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_934_173} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:08.498533 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_934_173} MergeTreeSequentialSource: Reading 173 marks from part 202410_1_928_172, total 1403163 rows starting from the beginning of the part 2024.10.22 02:31:08.552005 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.67 GiB, peak 2.00 GiB, free memory in arenas 10.64 MiB, will set to 1.55 GiB (RSS), difference: -120.91 MiB 2024.10.22 02:31:08.656688 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_934_173} MergeTreeSequentialSource: Reading 2 marks from part 202410_929_934_1, total 6934 rows starting from the beginning of the part 2024.10.22 02:31:08.747967 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 14997 2024.10.22 02:31:08.750082 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 88 entries to flush up to offset 107191 2024.10.22 02:31:09.051812 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:09.056692 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:09.160493 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:09.166562 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:09.214852 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_955_955_0 (state Temporary) 2024.10.22 02:31:09.259694 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_955_955_0 to 202410_961_961_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:09.335939 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 107191 2024.10.22 02:31:09.408305 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_972_972_0 (state Temporary) 2024.10.22 02:31:09.409623 [ 1210 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_956_60 to 202410_957_957_0 2024.10.22 02:31:09.415149 [ 1210 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.34 MiB using storage policy from min volume index 0 2024.10.22 02:31:09.431136 [ 1210 ] {} DiskLocal: Reserved 1.34 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:09.440225 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 7.46 MiB, will set to 1.55 GiB (RSS), difference: -17.28 MiB 2024.10.22 02:31:09.480603 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_972_972_0 to 202410_985_985_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:09.630685 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_957_61} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_956_60 to 202410_957_957_0 into Wide with storage Full 2024.10.22 02:31:09.663303 [ 1269 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_980_345 to 202410_981_981_0 2024.10.22 02:31:09.668060 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 14997 2024.10.22 02:31:09.684962 [ 1269 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:09.691594 [ 1269 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:09.692696 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/ypm/tvhcpycqfhqxxcyyaudionezsveyt (LogSeriesLimiter: on interval from 2024-10-22 02:30:56 to 2024-10-22 02:31:09 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:31:09.709832 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_981_346} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_980_345 to 202410_981_981_0 into Compact with storage Full 2024.10.22 02:31:09.717295 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_957_61} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:09.742897 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_957_61} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_956_60, total 23329 rows starting from the beginning of the part 2024.10.22 02:31:09.870625 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_981_346} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:09.879056 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_981_346} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_980_345, total 15019 rows starting from the beginning of the part 2024.10.22 02:31:10.024193 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_981_346} MergeTreeSequentialSource: Reading 2 marks from part 202410_981_981_0, total 10 rows starting from the beginning of the part 2024.10.22 02:31:10.039665 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_957_61} MergeTreeSequentialSource: Reading 2 marks from part 202410_957_957_0, total 125 rows starting from the beginning of the part 2024.10.22 02:31:10.137388 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 110110 2024.10.22 02:31:10.368201 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:10.375492 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:10.436577 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1638_1638_0 (state Temporary) 2024.10.22 02:31:10.578382 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1638_1638_0 to 202410_1651_1651_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:10.623311 [ 1270 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1646_4 to 202410_1651_1651_0 2024.10.22 02:31:10.644123 [ 1270 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:10.647899 [ 1270 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:10.704552 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1646_4 to 202410_1651_1651_0 into Compact with storage Full 2024.10.22 02:31:10.777558 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:10.799428 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1646_4, total 1546 rows starting from the beginning of the part 2024.10.22 02:31:10.855016 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1647_1647_0, total 69 rows starting from the beginning of the part 2024.10.22 02:31:10.896952 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110110 2024.10.22 02:31:10.931087 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1648_1648_0, total 84 rows starting from the beginning of the part 2024.10.22 02:31:11.014683 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1649_1649_0, total 45 rows starting from the beginning of the part 2024.10.22 02:31:11.077015 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1650_1650_0, total 69 rows starting from the beginning of the part 2024.10.22 02:31:11.174963 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_981_346} MergeTask::MergeProjectionsStage: Merge sorted 15029 rows, containing 12 columns (12 merged, 0 gathered) in 1.477401378 sec., 10172.591026241753 rows/sec., 1.59 MiB/sec. 2024.10.22 02:31:11.252228 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} MergeTreeSequentialSource: Reading 2 marks from part 202410_1651_1651_0, total 54 rows starting from the beginning of the part 2024.10.22 02:31:11.422192 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:31:11.429005 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:31:11.507649 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.72 GiB, peak 2.00 GiB, free memory in arenas 8.67 MiB, will set to 1.56 GiB (RSS), difference: -164.88 MiB 2024.10.22 02:31:11.612019 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_981_346} MergedBlockOutputStream: filled checksums 202410_1_981_346 (state Temporary) 2024.10.22 02:31:11.639717 [ 1225 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:31:11.666473 [ 1225 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:31:12.395047 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_921_286, 202410_922_922_0] 2024.10.22 02:31:12.403349 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_921_286, 202410_922_922_0] 2024.10.22 02:31:12.420784 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_921_286, 202410_922_922_0] 2024.10.22 02:31:12.484840 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_981_346} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_981_346 to 202410_1_981_346 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:12.494809 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_957_61} MergeTask::MergeProjectionsStage: Merge sorted 23454 rows, containing 13 columns (13 merged, 0 gathered) in 2.873926345 sec., 8160.96071522668 rows/sec., 3.56 MiB/sec. 2024.10.22 02:31:12.495787 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_981_346} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_980_345, 202410_981_981_0] -> 202410_1_981_346 2024.10.22 02:31:12.642445 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:31:12.782453 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_957_61} MergedBlockOutputStream: filled checksums 202410_747_957_61 (state Temporary) 2024.10.22 02:31:12.907041 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} MergeTask::MergeProjectionsStage: Merge sorted 1867 rows, containing 13 columns (13 merged, 0 gathered) in 2.213285284 sec., 843.542408878186 rows/sec., 147.13 KiB/sec. 2024.10.22 02:31:13.107740 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} MergedBlockOutputStream: filled checksums 202410_1626_1651_5 (state Temporary) 2024.10.22 02:31:13.350921 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 2 file segments (while reserving 47 bytes for b872b35f9b3b1be02ad15120a521fabc:0) 2024.10.22 02:31:13.353794 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: c543fbc33571b5267a71fdaddf899661, offset: 0, size: 155 2024.10.22 02:31:13.358895 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key c543fbc33571b5267a71fdaddf899661 for removal 2024.10.22 02:31:13.361218 [ 1476 ] {} CacheMetadata: Key c543fbc33571b5267a71fdaddf899661 is removed from metadata 2024.10.22 02:31:13.361603 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 48fdc0444459d6b99453c457e536b397, offset: 0, size: 4 2024.10.22 02:31:13.364667 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 48fdc0444459d6b99453c457e536b397 for removal 2024.10.22 02:31:13.369728 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1651_5 to 202410_1626_1651_5 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:13.389315 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1651_5} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1646_4, 202410_1651_1651_0] -> 202410_1626_1651_5 2024.10.22 02:31:13.420881 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_957_61} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_957_61 to 202410_747_957_61 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:13.428023 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_957_61} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_956_60, 202410_957_957_0] -> 202410_747_957_61 2024.10.22 02:31:13.449795 [ 1476 ] {} CacheMetadata: Key 48fdc0444459d6b99453c457e536b397 is removed from metadata 2024.10.22 02:31:13.450298 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:31:13.461516 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.07 MiB. 2024.10.22 02:31:13.534863 [ 1265 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 243 ms. 2024.10.22 02:31:13.558245 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_934_173/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:31:13.567527 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.65 GiB, peak 2.00 GiB, free memory in arenas 8.94 MiB, will set to 1.56 GiB (RSS), difference: -96.92 MiB 2024.10.22 02:31:14.008774 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rvh/xqvhbvyuyvflgqkikxixsfjkkbqhd (LogSeriesLimiter: on interval from 2024-10-22 02:31:09 to 2024-10-22 02:31:14 accepted series 1 / 201 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:31:14.142339 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(d5cfe3db3eb393e40fc8008950d7e76d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d5cfe3db3eb393e40fc8008950d7e76d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:31:14.152886 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 7142, file segment info: File segment: [0, 33554431], key: d5cfe3db3eb393e40fc8008950d7e76d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:31:14.155886 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:14.903399 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 54 entries to flush up to offset 110164 2024.10.22 02:31:15.168872 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileCache(s3_cache): Will evict 1 file segments (while reserving 273 bytes for ddd3e4650b28247749f34df9413e4990:0) 2024.10.22 02:31:15.172261 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Remove from cache. Key: 5d371d54d2754e3843768ae4d4a52a43, offset: 0, size: 153 2024.10.22 02:31:15.176085 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CacheMetadata: Submitting key 5d371d54d2754e3843768ae4d4a52a43 for removal 2024.10.22 02:31:15.222222 [ 1476 ] {} CacheMetadata: Key 5d371d54d2754e3843768ae4d4a52a43 is removed from metadata 2024.10.22 02:31:15.256228 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:15.261183 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:15.277307 [ 1263 ] {} DatabaseCatalog: Execution took 333 ms. 2024.10.22 02:31:15.439747 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1404213 2024.10.22 02:31:15.548925 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1639_1639_0 (state Temporary) 2024.10.22 02:31:15.635223 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1639_1639_0 to 202410_1652_1652_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:15.881376 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110164 2024.10.22 02:31:16.347701 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:16.349806 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:16.388981 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(59c4cf87c75a9b47d885a0efcfc09e81) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 59c4cf87c75a9b47d885a0efcfc09e81, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:31:16.411322 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 880, file segment info: File segment: [0, 33554431], key: 59c4cf87c75a9b47d885a0efcfc09e81, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:31:16.414612 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:16.580864 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_932_932_0 (state Temporary) 2024.10.22 02:31:16.845535 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 191 entries to flush up to offset 107382 2024.10.22 02:31:16.870149 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_932_932_0 to 202410_939_939_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:16.913535 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1404213 2024.10.22 02:31:17.165242 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:17.170005 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 15015 2024.10.22 02:31:17.171221 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:17.186879 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(b9202767003a447ebcb82fd95252215e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b9202767003a447ebcb82fd95252215e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:31:17.225739 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4052, file segment info: File segment: [0, 33554431], key: b9202767003a447ebcb82fd95252215e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:31:17.233161 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:17.261561 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_956_956_0 (state Temporary) 2024.10.22 02:31:17.432438 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_956_956_0 to 202410_962_962_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:17.478859 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.82 GiB, peak 2.00 GiB, free memory in arenas 10.53 MiB, will set to 1.56 GiB (RSS), difference: -275.27 MiB 2024.10.22 02:31:17.583862 [ 1223 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_957_61 to 202410_958_958_0 2024.10.22 02:31:17.592170 [ 1223 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.33 MiB using storage policy from min volume index 0 2024.10.22 02:31:17.596759 [ 1223 ] {} DiskLocal: Reserved 1.33 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:17.613559 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_958_62} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_957_61 to 202410_958_958_0 into Wide with storage Full 2024.10.22 02:31:17.649613 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_958_62} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:17.663478 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:17.668621 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:17.673245 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_958_62} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_957_61, total 23454 rows starting from the beginning of the part 2024.10.22 02:31:17.710663 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 107382 2024.10.22 02:31:17.740507 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_973_973_0 (state Temporary) 2024.10.22 02:31:17.896149 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_973_973_0 to 202410_986_986_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:17.936774 [ 1217 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_981_346 to 202410_982_982_0 2024.10.22 02:31:17.946575 [ 1217 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:17.950725 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15015 2024.10.22 02:31:17.949260 [ 1217 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:17.979297 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_982_347} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_981_346 to 202410_982_982_0 into Compact with storage Full 2024.10.22 02:31:18.006506 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_958_62} MergeTreeSequentialSource: Reading 2 marks from part 202410_958_958_0, total 96 rows starting from the beginning of the part 2024.10.22 02:31:18.162676 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_982_347} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:18.167789 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_982_347} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_981_346, total 15029 rows starting from the beginning of the part 2024.10.22 02:31:18.169444 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(cfcfb3496d25a4da25c5d11572987d22) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: cfcfb3496d25a4da25c5d11572987d22, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:31:18.186491 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 42394, file segment info: File segment: [0, 33554431], key: cfcfb3496d25a4da25c5d11572987d22, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:31:18.193411 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:18.234766 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_982_347} MergeTreeSequentialSource: Reading 2 marks from part 202410_982_982_0, total 26 rows starting from the beginning of the part 2024.10.22 02:31:18.585143 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 806 2024.10.22 02:31:18.975895 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:18.985489 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:19.027109 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:19.030276 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:19.040521 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_982_347} MergeTask::MergeProjectionsStage: Merge sorted 15055 rows, containing 12 columns (12 merged, 0 gathered) in 1.071678577 sec., 14048.055380694617 rows/sec., 2.20 MiB/sec. 2024.10.22 02:31:19.206191 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_982_347} MergedBlockOutputStream: filled checksums 202410_1_982_347 (state Temporary) 2024.10.22 02:31:19.263267 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_788_788_0 (state Temporary) 2024.10.22 02:31:19.339213 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(1981aca0754654ba3593162a87efb44a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 1981aca0754654ba3593162a87efb44a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:31:19.350202 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 491, file segment info: File segment: [0, 33554431], key: 1981aca0754654ba3593162a87efb44a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:31:19.363106 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:19.407250 [ 1724 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/yem/ccebndaxzxjxfrfqsjqosmsidnnpx, size 4052 2024.10.22 02:31:19.620708 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_788_788_0 to 202410_798_798_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:19.669672 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_958_62/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:31:19.724777 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_982_347} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_982_347 to 202410_1_982_347 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:19.758842 [ 1154 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_793_161 to 202410_798_798_0 2024.10.22 02:31:19.796381 [ 1419 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Found 5 old parts to remove. Parts: [202410_1_42_10, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 02:31:19.800324 [ 1419 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from filesystem (serially): Parts: [202410_1_42_10, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 02:31:19.807473 [ 1154 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:19.817158 [ 1154 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:19.817682 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 806 2024.10.22 02:31:19.836007 [ 1419 ] {} system.part_log (2d8f770f-485b-41a3-9b48-1e3c86e7559f): Removing 5 parts from memory: Parts: [202410_1_42_10, 202410_43_43_0, 202410_44_44_0, 202410_45_45_0, 202410_46_46_0] 2024.10.22 02:31:19.844817 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_982_347} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_981_346, 202410_982_982_0] -> 202410_1_982_347 2024.10.22 02:31:19.872826 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_793_161 to 202410_798_798_0 into Compact with storage Full 2024.10.22 02:31:19.875034 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_958_62} MergeTask::MergeProjectionsStage: Merge sorted 23550 rows, containing 13 columns (13 merged, 0 gathered) in 2.268334879 sec., 10382.064931427612 rows/sec., 4.52 MiB/sec. 2024.10.22 02:31:19.940873 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_922_287, 202410_923_923_0] 2024.10.22 02:31:19.962714 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:31:19.986903 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 110242 2024.10.22 02:31:19.989629 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_922_287, 202410_923_923_0] 2024.10.22 02:31:20.002019 [ 1422 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_922_287, 202410_923_923_0] 2024.10.22 02:31:20.046487 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:31:20.051681 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:31:20.066202 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_958_62} MergedBlockOutputStream: filled checksums 202410_747_958_62 (state Temporary) 2024.10.22 02:31:20.075129 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:31:20.092000 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:31:20.107509 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:31:20.127591 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:31:20.192263 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column RefererHash 2024.10.22 02:31:20.229213 [ 1386 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/zwe/bjfsituuwibwtntworxkxobzykltp (LogSeriesLimiter: on interval from 2024-10-22 02:31:14 to 2024-10-22 02:31:20 accepted series 1 / 152 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:31:20.312900 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:20.321649 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:20.396478 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:20.455230 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_793_161, total 837 rows starting from the beginning of the part 2024.10.22 02:31:20.501260 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1640_1640_0 (state Temporary) 2024.10.22 02:31:20.615873 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(758c3c799d606177dae9cf9f00c8d5e7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 758c3c799d606177dae9cf9f00c8d5e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:31:20.627545 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 899, file segment info: File segment: [0, 33554431], key: 758c3c799d606177dae9cf9f00c8d5e7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1394, kind: Regular, unbound: 0 2024.10.22 02:31:20.734276 [ 1394 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:20.762736 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} MergeTreeSequentialSource: Reading 2 marks from part 202410_794_794_0, total 1 rows starting from the beginning of the part 2024.10.22 02:31:20.803548 [ 1708 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jym/auigyqzuouwelnsbtevujzenpfdfl, size 42394 2024.10.22 02:31:20.862137 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1640_1640_0 to 202410_1653_1653_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:21.017654 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110242 2024.10.22 02:31:21.049721 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_958_62} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_958_62 to 202410_747_958_62 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:21.051971 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_958_62} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_957_61, 202410_958_958_0] -> 202410_747_958_62 2024.10.22 02:31:21.171043 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.54 MiB. 2024.10.22 02:31:21.334301 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} MergeTreeSequentialSource: Reading 2 marks from part 202410_795_795_0, total 1 rows starting from the beginning of the part 2024.10.22 02:31:21.421528 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:31:21.425720 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:31:21.483985 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 9.72 MiB, will set to 1.58 GiB (RSS), difference: -14.80 MiB 2024.10.22 02:31:21.566534 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} MergeTreeSequentialSource: Reading 2 marks from part 202410_796_796_0, total 1 rows starting from the beginning of the part 2024.10.22 02:31:21.614810 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:31:21.644797 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:31:21.758802 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:31:21.963138 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:31:21.963285 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} MergeTreeSequentialSource: Reading 2 marks from part 202410_797_797_0, total 1 rows starting from the beginning of the part 2024.10.22 02:31:22.006191 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:31:22.156331 [ 1172 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 251 ms. 2024.10.22 02:31:22.220219 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:31:22.255348 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:31:22.398236 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:31:22.465784 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column NetMinor 2024.10.22 02:31:22.551232 [ 1395 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} MergeTreeSequentialSource: Reading 2 marks from part 202410_798_798_0, total 1 rows starting from the beginning of the part 2024.10.22 02:31:22.952615 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_195_195_0 (state Temporary) 2024.10.22 02:31:22.991629 [ 1423 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 6 similar messages) 2024.10.22 02:31:23.826171 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 13.00 MiB, will set to 1.59 GiB (RSS), difference: -18.29 MiB 2024.10.22 02:31:23.924219 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1405368 2024.10.22 02:31:24.051289 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(b4fb5cb011eac58b5ecc7f7d1741a42a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b4fb5cb011eac58b5ecc7f7d1741a42a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:31:24.054176 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: b4fb5cb011eac58b5ecc7f7d1741a42a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:31:24.058920 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:25.027419 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 53 entries to flush up to offset 110295 2024.10.22 02:31:25.028260 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:25.030315 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:25.122120 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_934_173/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:31:25.122620 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_933_933_0 (state Temporary) 2024.10.22 02:31:25.231160 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_933_933_0 to 202410_940_940_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:25.250366 [ 1176 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_935_935_0 to 202410_940_940_0 2024.10.22 02:31:25.255455 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 209 entries to flush up to offset 107591 2024.10.22 02:31:25.266797 [ 1176 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:25.271586 [ 1176 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:25.282247 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1405368 2024.10.22 02:31:25.295251 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_935_935_0 to 202410_940_940_0 into Compact with storage Full 2024.10.22 02:31:25.334785 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:25.356007 [ 1387 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} MergeTask::MergeProjectionsStage: Merge sorted 842 rows, containing 32 columns (32 merged, 0 gathered) in 5.490424124 sec., 153.3579157062585 rows/sec., 11.53 KiB/sec. 2024.10.22 02:31:25.387169 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} MergedBlockOutputStream: filled checksums 202410_1_798_162 (state Temporary) 2024.10.22 02:31:25.445793 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_935_935_0, total 1544 rows starting from the beginning of the part 2024.10.22 02:31:25.452787 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 15026 2024.10.22 02:31:25.461705 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_936_936_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:31:25.493762 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_937_937_0, total 1542 rows starting from the beginning of the part 2024.10.22 02:31:25.535217 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_938_938_0, total 1925 rows starting from the beginning of the part 2024.10.22 02:31:25.552215 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_939_939_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:31:25.564571 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:25.567133 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:25.572869 [ 1394 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_940_940_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:31:25.590706 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:25.594614 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:25.727074 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1641_1641_0 (state Temporary) 2024.10.22 02:31:25.795630 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_974_974_0 (state Temporary) 2024.10.22 02:31:25.841761 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:25.846808 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:25.849631 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_974_974_0 to 202410_987_987_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:25.859912 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1641_1641_0 to 202410_1654_1654_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:25.864486 [ 1231 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_982_347 to 202410_983_983_0 2024.10.22 02:31:25.867183 [ 1231 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:25.868860 [ 1231 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:25.881370 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_957_957_0 (state Temporary) 2024.10.22 02:31:25.883879 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15026 2024.10.22 02:31:25.897583 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110295 2024.10.22 02:31:26.038474 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_983_348} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_982_347 to 202410_983_983_0 into Compact with storage Full 2024.10.22 02:31:26.055335 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_957_957_0 to 202410_963_963_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:26.133776 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_983_348} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:26.140024 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 107591 2024.10.22 02:31:26.141506 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_983_348} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_982_347, total 15055 rows starting from the beginning of the part 2024.10.22 02:31:26.172221 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_983_348} MergeTreeSequentialSource: Reading 2 marks from part 202410_983_983_0, total 21 rows starting from the beginning of the part 2024.10.22 02:31:26.179317 [ 1199 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 6 parts from 202410_747_958_62 to 202410_963_963_0 2024.10.22 02:31:26.195935 [ 1199 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 02:31:26.199505 [ 1199 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:26.350507 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} MergeTask::PrepareStage: Merging 6 parts: from 202410_747_958_62 to 202410_963_963_0 into Wide with storage Full 2024.10.22 02:31:26.452405 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:26.460586 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 11.50 MiB, will set to 1.60 GiB (RSS), difference: -17.08 MiB 2024.10.22 02:31:26.467101 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_958_62, total 23550 rows starting from the beginning of the part 2024.10.22 02:31:26.540794 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} MergeTask::MergeProjectionsStage: Merge sorted 8861 rows, containing 4 columns (4 merged, 0 gathered) in 1.252704007 sec., 7073.498568285493 rows/sec., 110.52 KiB/sec. 2024.10.22 02:31:26.558465 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} MergedBlockOutputStream: filled checksums 202410_935_940_1 (state Temporary) 2024.10.22 02:31:26.570859 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_798_162 to 202410_1_798_162 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:26.574206 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_798_162} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_793_161, 202410_798_798_0] -> 202410_1_798_162 2024.10.22 02:31:26.603960 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_935_940_1 to 202410_935_940_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:26.613772 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_935_940_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_935_935_0, 202410_940_940_0] -> 202410_935_940_1 2024.10.22 02:31:26.672358 [ 1208 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:31:26.688108 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:31:26.734457 [ 1208 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:31:26.743189 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:31:26.981534 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_959_959_0, total 108 rows starting from the beginning of the part 2024.10.22 02:31:27.057143 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_960_960_0, total 90 rows starting from the beginning of the part 2024.10.22 02:31:27.140978 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_961_961_0, total 88 rows starting from the beginning of the part 2024.10.22 02:31:27.183803 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_962_962_0, total 191 rows starting from the beginning of the part 2024.10.22 02:31:27.251946 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} MergeTreeSequentialSource: Reading 2 marks from part 202410_963_963_0, total 209 rows starting from the beginning of the part 2024.10.22 02:31:27.322107 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 807 2024.10.22 02:31:27.609809 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_983_348} MergeTask::MergeProjectionsStage: Merge sorted 15076 rows, containing 12 columns (12 merged, 0 gathered) in 1.595928749 sec., 9446.53701454187 rows/sec., 1.48 MiB/sec. 2024.10.22 02:31:27.627753 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:27.632885 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:27.688331 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_789_789_0 (state Temporary) 2024.10.22 02:31:27.795554 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_789_789_0 to 202410_799_799_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:27.964557 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_983_348} MergedBlockOutputStream: filled checksums 202410_1_983_348 (state Temporary) 2024.10.22 02:31:27.984347 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 807 2024.10.22 02:31:28.044505 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_923_288, 202410_924_924_0] 2024.10.22 02:31:28.047994 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_923_288, 202410_924_924_0] 2024.10.22 02:31:28.142501 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_983_348} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_983_348 to 202410_1_983_348 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:28.155565 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_983_348} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_982_347, 202410_983_983_0] -> 202410_1_983_348 2024.10.22 02:31:28.159929 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_923_288, 202410_924_924_0] 2024.10.22 02:31:28.298167 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:31:28.838327 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 11.64 MiB, will set to 1.61 GiB (RSS), difference: -22.60 MiB 2024.10.22 02:31:29.566942 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_868_162, 202410_869_874_1] 2024.10.22 02:31:29.568923 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_868_162, 202410_869_874_1] 2024.10.22 02:31:29.593770 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_868_162, 202410_869_874_1] 2024.10.22 02:31:29.671358 [ 1391 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} MergeTask::MergeProjectionsStage: Merge sorted 24236 rows, containing 13 columns (13 merged, 0 gathered) in 3.324466433 sec., 7290.1924228873695 rows/sec., 3.18 MiB/sec. 2024.10.22 02:31:29.749081 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_195_195_0 to 202410_198_198_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:29.752867 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} MergedBlockOutputStream: filled checksums 202410_747_963_63 (state Temporary) 2024.10.22 02:31:30.103950 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 87 entries to flush up to offset 110382 2024.10.22 02:31:30.517636 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_963_63 to 202410_747_963_63 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:30.520876 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_963_63} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 6 parts: [202410_747_958_62, 202410_963_963_0] -> 202410_747_963_63 2024.10.22 02:31:30.555188 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 7.97 MiB, will set to 1.61 GiB (RSS), difference: 56.17 MiB 2024.10.22 02:31:30.609740 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:30.612403 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:30.687026 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 56.03 MiB. 2024.10.22 02:31:30.699574 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1642_1642_0 (state Temporary) 2024.10.22 02:31:30.888566 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1550_97, 202410_1551_1551_0] 2024.10.22 02:31:30.890857 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1642_1642_0 to 202410_1655_1655_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:30.893448 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1550_97, 202410_1551_1551_0] 2024.10.22 02:31:30.907521 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1550_97, 202410_1551_1551_0] 2024.10.22 02:31:31.005827 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110382 2024.10.22 02:31:31.426080 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:31:31.430404 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:31:32.391933 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1406908 2024.10.22 02:31:32.559490 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7878 2024.10.22 02:31:33.242412 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:33.348062 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:33.447351 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_934_934_0 (state Temporary) 2024.10.22 02:31:33.488671 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 30 entries to flush up to offset 15056 2024.10.22 02:31:33.628572 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_934_934_0 to 202410_941_941_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:33.662673 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1406908 2024.10.22 02:31:33.672014 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 169 entries to flush up to offset 107760 2024.10.22 02:31:33.840442 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:33.842808 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:33.895928 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_958_958_0 (state Temporary) 2024.10.22 02:31:33.928640 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:33.939270 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:33.964231 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_975_975_0 (state Temporary) 2024.10.22 02:31:34.023948 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_975_975_0 to 202410_988_988_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:34.062231 [ 1236 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_983_348 to 202410_984_984_0 2024.10.22 02:31:34.067108 [ 1236 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:34.073705 [ 1236 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:34.088991 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15056 2024.10.22 02:31:34.092982 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_984_349} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_983_348 to 202410_984_984_0 into Compact with storage Full 2024.10.22 02:31:34.130000 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_984_349} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:34.142747 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_984_349} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_983_348, total 15076 rows starting from the beginning of the part 2024.10.22 02:31:34.161580 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_984_349} MergeTreeSequentialSource: Reading 2 marks from part 202410_984_984_0, total 13 rows starting from the beginning of the part 2024.10.22 02:31:34.192671 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_958_958_0 to 202410_964_964_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:34.256567 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 107760 2024.10.22 02:31:34.668137 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 14.00 MiB, will set to 1.63 GiB (RSS), difference: 3.05 MiB 2024.10.22 02:31:35.029080 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 42 entries to flush up to offset 110424 2024.10.22 02:31:35.074598 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_984_349} MergeTask::MergeProjectionsStage: Merge sorted 15089 rows, containing 12 columns (12 merged, 0 gathered) in 0.987782032 sec., 15275.637247064238 rows/sec., 2.39 MiB/sec. 2024.10.22 02:31:35.161019 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:35.164190 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:35.380566 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_984_349} MergedBlockOutputStream: filled checksums 202410_1_984_349 (state Temporary) 2024.10.22 02:31:35.421473 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1643_1643_0 (state Temporary) 2024.10.22 02:31:35.488979 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 808 2024.10.22 02:31:35.650592 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_984_349} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_984_349 to 202410_1_984_349 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:35.653524 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_984_349} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_983_348, 202410_984_984_0] -> 202410_1_984_349 2024.10.22 02:31:35.672750 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1551_98, 202410_1552_1552_0] 2024.10.22 02:31:35.673767 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1643_1643_0 to 202410_1656_1656_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:35.676026 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1551_98, 202410_1552_1552_0] 2024.10.22 02:31:35.680724 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:31:35.696602 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1551_98, 202410_1552_1552_0] 2024.10.22 02:31:35.724325 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1651_5 to 202410_1656_1656_0 2024.10.22 02:31:35.739792 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110424 2024.10.22 02:31:35.847272 [ 1218 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:35.853571 [ 1218 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:35.966169 [ 1218 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 274 ms. 2024.10.22 02:31:35.996000 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1651_5 to 202410_1656_1656_0 into Compact with storage Full 2024.10.22 02:31:36.061795 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:36.129109 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:36.138838 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:36.214136 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_924_289, 202410_925_925_0] 2024.10.22 02:31:36.216179 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_924_289, 202410_925_925_0] 2024.10.22 02:31:36.327908 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_790_790_0 (state Temporary) 2024.10.22 02:31:36.330317 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_924_289, 202410_925_925_0] 2024.10.22 02:31:36.427195 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1651_5, total 1867 rows starting from the beginning of the part 2024.10.22 02:31:36.508440 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_790_790_0 to 202410_800_800_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:36.663073 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 808 2024.10.22 02:31:36.687915 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_934_173/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:31:36.637833 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1652_1652_0, total 54 rows starting from the beginning of the part 2024.10.22 02:31:36.841505 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1653_1653_0, total 78 rows starting from the beginning of the part 2024.10.22 02:31:36.960947 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 12.93 MiB, will set to 1.63 GiB (RSS), difference: 15.09 MiB 2024.10.22 02:31:37.058954 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1654_1654_0, total 53 rows starting from the beginning of the part 2024.10.22 02:31:37.162373 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1655_1655_0, total 87 rows starting from the beginning of the part 2024.10.22 02:31:37.351287 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} MergeTreeSequentialSource: Reading 2 marks from part 202410_1656_1656_0, total 42 rows starting from the beginning of the part 2024.10.22 02:31:38.350693 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.63 GiB, peak 2.00 GiB, free memory in arenas 15.59 MiB, will set to 1.64 GiB (RSS), difference: 3.12 MiB 2024.10.22 02:31:38.691152 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} MergeTask::MergeProjectionsStage: Merge sorted 2181 rows, containing 13 columns (13 merged, 0 gathered) in 2.702412128 sec., 807.056768803844 rows/sec., 140.41 KiB/sec. 2024.10.22 02:31:38.863735 [ 1389 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} MergedBlockOutputStream: filled checksums 202410_1626_1656_6 (state Temporary) 2024.10.22 02:31:38.941231 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1656_6 to 202410_1626_1656_6 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:38.944763 [ 1385 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1656_6} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1651_5, 202410_1656_1656_0] -> 202410_1626_1656_6 2024.10.22 02:31:39.066821 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:31:39.746255 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 43 entries to flush up to offset 110467 2024.10.22 02:31:40.060198 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:40.063013 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.26 GiB. 2024.10.22 02:31:40.214669 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 7919 2024.10.22 02:31:40.302726 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1644_1644_0 (state Temporary) 2024.10.22 02:31:40.340144 [ 1422 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:31:40.551693 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 18.86 MiB, will set to 1.65 GiB (RSS), difference: 10.15 MiB 2024.10.22 02:31:40.666848 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1408448 2024.10.22 02:31:40.706670 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1644_1644_0 to 202410_1657_1657_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:40.753757 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110467 2024.10.22 02:31:40.771172 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(8cd22c3de3bd4c04227e91fe05fc4978) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8cd22c3de3bd4c04227e91fe05fc4978, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 02:31:40.773367 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 990, file segment info: File segment: [0, 33554431], key: 8cd22c3de3bd4c04227e91fe05fc4978, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1399, kind: Regular, unbound: 0 2024.10.22 02:31:40.780483 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:41.110661 [ 1712 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/zwe/bjfsituuwibwtntworxkxobzykltp, size 8209791 2024.10.22 02:31:41.172456 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column URLHash 2024.10.22 02:31:41.178466 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column URLHash 2024.10.22 02:31:41.220036 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column URLHash 2024.10.22 02:31:41.244665 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column URLHash 2024.10.22 02:31:41.311565 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column URLHash 2024.10.22 02:31:41.336596 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column URLHash 2024.10.22 02:31:41.379001 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column URLHash 2024.10.22 02:31:41.436252 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:31:41.439880 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:31:41.476608 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xev/moffpartbwvxgeaszpbyrmfaptoyh (LogSeriesLimiter: on interval from 2024-10-22 02:31:20 to 2024-10-22 02:31:41 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:31:41.637183 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 15072 2024.10.22 02:31:41.738691 [ 1169 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:31:41.790596 [ 1169 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:31:41.807307 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 79 entries to flush up to offset 107839 2024.10.22 02:31:41.856959 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:41.861294 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:42.045121 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:42.050885 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:42.060256 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:42.062437 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:42.128776 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_959_959_0 (state Temporary) 2024.10.22 02:31:42.170558 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1552_99, 202410_1553_1553_0] 2024.10.22 02:31:42.172684 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1552_99, 202410_1553_1553_0] 2024.10.22 02:31:42.177572 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_959_959_0 to 202410_965_965_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:42.181008 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1552_99, 202410_1553_1553_0] 2024.10.22 02:31:42.212782 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_976_976_0 (state Temporary) 2024.10.22 02:31:42.212778 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_935_935_0 (state Temporary) 2024.10.22 02:31:42.286794 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 107839 2024.10.22 02:31:42.395099 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_935_935_0 to 202410_942_942_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:42.432715 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1408448 2024.10.22 02:31:42.495997 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_976_976_0 to 202410_989_989_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:42.538625 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_902_31, 202410_903_903_0, 202410_904_904_0, 202410_905_905_0, 202410_906_906_0, 202410_907_907_0] 2024.10.22 02:31:42.543474 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_902_31, 202410_903_903_0, 202410_904_904_0, 202410_905_905_0, 202410_906_906_0, 202410_907_907_0] 2024.10.22 02:31:42.550988 [ 1183 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_984_349 to 202410_985_985_0 2024.10.22 02:31:42.578368 [ 1183 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:42.585520 [ 1183 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:42.585118 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15072 2024.10.22 02:31:42.686141 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_902_31, 202410_903_903_0, 202410_904_904_0, 202410_905_905_0, 202410_906_906_0, 202410_907_907_0] 2024.10.22 02:31:42.751673 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_985_350} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_984_349 to 202410_985_985_0 into Compact with storage Full 2024.10.22 02:31:42.765322 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.66 GiB, peak 2.00 GiB, free memory in arenas 14.05 MiB, will set to 1.64 GiB (RSS), difference: -18.39 MiB 2024.10.22 02:31:42.874416 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_985_350} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:42.922402 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_985_350} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_984_349, total 15089 rows starting from the beginning of the part 2024.10.22 02:31:43.010085 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_985_350} MergeTreeSequentialSource: Reading 2 marks from part 202410_985_985_0, total 14 rows starting from the beginning of the part 2024.10.22 02:31:43.651351 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_985_350} MergeTask::MergeProjectionsStage: Merge sorted 15103 rows, containing 12 columns (12 merged, 0 gathered) in 1.030733641 sec., 14652.670097531045 rows/sec., 2.29 MiB/sec. 2024.10.22 02:31:43.801104 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_985_350} MergedBlockOutputStream: filled checksums 202410_1_985_350 (state Temporary) 2024.10.22 02:31:44.068663 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_985_350} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_985_350 to 202410_1_985_350 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:44.075215 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_985_350} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_984_349, 202410_985_985_0] -> 202410_1_985_350 2024.10.22 02:31:44.120067 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:31:44.166312 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 809 2024.10.22 02:31:44.353542 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_925_290, 202410_926_926_0] 2024.10.22 02:31:44.356031 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_925_290, 202410_926_926_0] 2024.10.22 02:31:44.365557 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_925_290, 202410_926_926_0] 2024.10.22 02:31:44.479876 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:44.483803 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:44.499476 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.64 GiB, peak 2.00 GiB, free memory in arenas 11.30 MiB, will set to 1.65 GiB (RSS), difference: 4.45 MiB 2024.10.22 02:31:44.654642 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_791_791_0 (state Temporary) 2024.10.22 02:31:44.758318 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 110537 2024.10.22 02:31:44.764494 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_791_791_0 to 202410_801_801_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:44.921768 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:44.927224 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.25 GiB. 2024.10.22 02:31:45.011145 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1645_1645_0 (state Temporary) 2024.10.22 02:31:45.045729 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 809 2024.10.22 02:31:45.066200 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1645_1645_0 to 202410_1658_1658_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:45.125311 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110537 2024.10.22 02:31:45.963067 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(7b2df0fa287f88ca526c483d3ed61d48) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7b2df0fa287f88ca526c483d3ed61d48, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:31:46.075952 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1445, file segment info: File segment: [0, 33554431], key: 7b2df0fa287f88ca526c483d3ed61d48, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:31:46.091410 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:46.226306 [ 1163 ] {} DatabaseCatalog: Execution took 681 ms. 2024.10.22 02:31:46.449610 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1553_100, 202410_1554_1554_0] 2024.10.22 02:31:46.451513 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1553_100, 202410_1554_1554_0] 2024.10.22 02:31:46.564156 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1553_100, 202410_1554_1554_0] 2024.10.22 02:31:47.268132 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 13.74 MiB, will set to 1.64 GiB (RSS), difference: 110.00 MiB 2024.10.22 02:31:47.471364 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(31bf2160cf3fd734ae72e0fb3986043b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 31bf2160cf3fd734ae72e0fb3986043b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:31:47.532383 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 31bf2160cf3fd734ae72e0fb3986043b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1390, kind: Regular, unbound: 0 2024.10.22 02:31:47.636443 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:47.860824 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_934_173/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:31:48.585074 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_748_152, 202410_749_749_0, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0, 202410_753_753_0] 2024.10.22 02:31:48.595356 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_748_152, 202410_749_749_0, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0, 202410_753_753_0] 2024.10.22 02:31:48.624733 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(fa65774a2957b66b357a837f124d166d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fa65774a2957b66b357a837f124d166d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:31:48.635166 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 876, file segment info: File segment: [0, 33554431], key: fa65774a2957b66b357a837f124d166d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:31:48.640856 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:48.749613 [ 1419 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_748_152, 202410_749_749_0, 202410_750_750_0, 202410_751_751_0, 202410_752_752_0, 202410_753_753_0] 2024.10.22 02:31:49.145009 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 26 entries to flush up to offset 110563 2024.10.22 02:31:49.511510 [ 1239 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 02:31:49.539567 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1409988 2024.10.22 02:31:49.662457 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:49.665284 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.20 GiB. 2024.10.22 02:31:49.719740 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(adb32d4d09dc616b5ca01d8b1064f514) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: adb32d4d09dc616b5ca01d8b1064f514, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:31:49.735217 [ 1709 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rvh/xqvhbvyuyvflgqkikxixsfjkkbqhd, size 44 2024.10.22 02:31:49.740205 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 39426, file segment info: File segment: [0, 33554431], key: adb32d4d09dc616b5ca01d8b1064f514, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:31:49.746816 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:49.801776 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 101 entries to flush up to offset 107940 2024.10.22 02:31:49.891885 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1646_1646_0 (state Temporary) 2024.10.22 02:31:50.042313 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1646_1646_0 to 202410_1659_1659_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:50.100322 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 9 entries to flush up to offset 15081 2024.10.22 02:31:50.164884 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110563 2024.10.22 02:31:50.305411 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:50.310729 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:31:50.315272 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:50.327268 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.19 GiB. 2024.10.22 02:31:50.456065 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1554_101, 202410_1555_1555_0] 2024.10.22 02:31:50.459859 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1554_101, 202410_1555_1555_0] 2024.10.22 02:31:50.467401 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 366 ms. 2024.10.22 02:31:50.470490 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1554_101, 202410_1555_1555_0] 2024.10.22 02:31:50.518942 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_960_960_0 (state Temporary) 2024.10.22 02:31:50.580553 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_977_977_0 (state Temporary) 2024.10.22 02:31:50.647050 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_960_960_0 to 202410_966_966_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:50.709444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} MergedBlockOutputStream: filled checksums 201403_22_22_0 (state Temporary) 2024.10.22 02:31:50.746480 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fpd/prluihixybsdfaemzmcyqaxtpaeau (LogSeriesLimiter: on interval from 2024-10-22 02:31:41 to 2024-10-22 02:31:50 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:31:50.750920 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 107940 2024.10.22 02:31:50.758579 [ 1689 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/ypm/tvhcpycqfhqxxcyyaudionezsveyt, size 710 2024.10.22 02:31:50.842267 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_977_977_0 to 202410_990_990_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:50.846457 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.36 GiB, peak 2.00 GiB, free memory in arenas 11.18 MiB, will set to 1.61 GiB (RSS), difference: 251.91 MiB 2024.10.22 02:31:50.906271 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15081 2024.10.22 02:31:51.004144 [ 1255 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_985_350 to 202410_986_986_0 2024.10.22 02:31:51.012866 [ 1255 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:51.016623 [ 1255 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:31:51.054606 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_986_351} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_985_350 to 202410_986_986_0 into Compact with storage Full 2024.10.22 02:31:51.311139 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_986_351} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:51.314314 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:51.318464 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(a5a56bc2785a6b94618aa5a9eb64f7af) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a5a56bc2785a6b94618aa5a9eb64f7af, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:31:51.320169 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:31:51.324396 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 890, file segment info: File segment: [0, 33554431], key: a5a56bc2785a6b94618aa5a9eb64f7af, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:31:51.326768 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:51.343631 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_986_351} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_985_350, total 15103 rows starting from the beginning of the part 2024.10.22 02:31:51.438435 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:31:51.441611 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:31:51.553231 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_986_351} MergeTreeSequentialSource: Reading 2 marks from part 202410_986_986_0, total 18 rows starting from the beginning of the part 2024.10.22 02:31:51.621743 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_936_936_0 (state Temporary) 2024.10.22 02:31:51.646762 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:31:51.654810 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:31:51.679385 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:31:51.711325 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:31:51.727254 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:31:51.868093 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_936_936_0 to 202410_943_943_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:51.875108 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:31:51.940086 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1409988 2024.10.22 02:31:52.002554 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:31:52.024447 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:31:52.144743 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UserAgentMajor 2024.10.22 02:31:52.176680 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_934_173} MergeTask::MergeProjectionsStage: Merge sorted 1410097 rows, containing 4 columns (4 merged, 0 gathered) in 43.715520573 sec., 32256.209728654532 rows/sec., 483.69 KiB/sec. 2024.10.22 02:31:52.360017 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b1f0f1e53602efd439e608db3577ba79) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b1f0f1e53602efd439e608db3577ba79, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:31:52.363371 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegmentRangeWriter: Failed to reserve space in cache (size: 4, file segment info: File segment: [0, 33554431], key: b1f0f1e53602efd439e608db3577ba79, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:31:52.365262 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:31:52.400499 [ 1709 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fpd/prluihixybsdfaemzmcyqaxtpaeau, size 4 2024.10.22 02:31:52.482402 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_986_351/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:31:52.550268 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 810 2024.10.22 02:31:52.605395 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_926_291, 202410_927_927_0] 2024.10.22 02:31:52.610174 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_926_291, 202410_927_927_0] 2024.10.22 02:31:52.618303 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_934_173} MergedBlockOutputStream: filled checksums 202410_1_934_173 (state Temporary) 2024.10.22 02:31:52.624642 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_926_291, 202410_927_927_0] 2024.10.22 02:31:52.872324 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:52.875469 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:31:53.149446 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_792_792_0 (state Temporary) 2024.10.22 02:31:53.222754 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_986_351} MergeTask::MergeProjectionsStage: Merge sorted 15121 rows, containing 12 columns (12 merged, 0 gathered) in 2.176917402 sec., 6946.0605102002855 rows/sec., 1.09 MiB/sec. 2024.10.22 02:31:53.445655 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_792_792_0 to 202410_802_802_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:53.483124 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_986_351} MergedBlockOutputStream: filled checksums 202410_1_986_351 (state Temporary) 2024.10.22 02:31:53.648549 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 810 2024.10.22 02:31:54.069736 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: finalizeImpl WriteBufferFromS3. Details: bucket test, key common/hlk/ownsabaaotdxbywjohcirjbkfxhvp. 2024.10.22 02:31:54.142541 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_986_351} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_986_351 to 202410_1_986_351 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:54.172926 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 78 entries to flush up to offset 110641 2024.10.22 02:31:54.166282 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_986_351} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_985_350, 202410_986_986_0] -> 202410_1_986_351 2024.10.22 02:31:54.241481 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:31:54.375665 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:54.380442 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:31:54.645655 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1647_1647_0 (state Temporary) 2024.10.22 02:31:54.754532 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1647_1647_0 to 202410_1660_1660_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:54.775118 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 11.16 MiB, will set to 1.58 GiB (RSS), difference: -18.47 MiB 2024.10.22 02:31:54.856648 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110641 2024.10.22 02:31:55.628021 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_881_881_0, 202410_882_882_0, 202410_883_883_0, 202410_884_884_0, 202410_885_885_0, 202410_886_886_0] 2024.10.22 02:31:55.642839 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_881_881_0, 202410_882_882_0, 202410_883_883_0, 202410_884_884_0, 202410_885_885_0, 202410_886_886_0] 2024.10.22 02:31:55.670355 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_881_881_0, 202410_882_882_0, 202410_883_883_0, 202410_884_884_0, 202410_885_885_0, 202410_886_886_0] 2024.10.22 02:31:56.372805 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 10.67 MiB, will set to 1.58 GiB (RSS), difference: 4.07 MiB 2024.10.22 02:31:56.440274 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_934_173} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_934_173 to 202410_1_934_173 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:56.445765 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_934_173} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_928_172, 202410_929_934_1] -> 202410_1_934_173 2024.10.22 02:31:56.477916 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:31:56.552294 [ 1203 ] {} DatabaseCatalog: Execution took 332 ms. 2024.10.22 02:31:56.802895 [ 1188 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:31:56.955544 [ 1188 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:31:58.056880 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1555_102, 202410_1556_1556_0] 2024.10.22 02:31:58.059951 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1555_102, 202410_1556_1556_0] 2024.10.22 02:31:58.074641 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1555_102, 202410_1556_1556_0] 2024.10.22 02:31:58.272436 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 81 entries to flush up to offset 108021 2024.10.22 02:31:58.332574 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.52 MiB, will set to 1.58 GiB (RSS), difference: 4.42 MiB 2024.10.22 02:31:58.407697 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 11 entries to flush up to offset 15092 2024.10.22 02:31:58.408823 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:58.411668 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:31:58.457272 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_961_961_0 (state Temporary) 2024.10.22 02:31:58.505787 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_961_961_0 to 202410_967_967_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:58.566820 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 108021 2024.10.22 02:31:58.651297 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:58.653825 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:31:58.680654 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_978_978_0 (state Temporary) 2024.10.22 02:31:58.748077 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_978_978_0 to 202410_991_991_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:58.871236 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 34 entries to flush up to offset 110675 2024.10.22 02:31:58.883655 [ 1254 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_986_351 to 202410_987_987_0 2024.10.22 02:31:58.890308 [ 1254 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:58.910198 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15092 2024.10.22 02:31:58.935077 [ 1254 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:31:58.944666 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1411528 2024.10.22 02:31:58.974192 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_987_352} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_986_351 to 202410_987_987_0 into Compact with storage Full 2024.10.22 02:31:59.021249 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:59.026264 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:31:59.036240 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_987_352} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:59.045043 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_987_352} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_986_351, total 15121 rows starting from the beginning of the part 2024.10.22 02:31:59.065112 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_987_352} MergeTreeSequentialSource: Reading 2 marks from part 202410_987_987_0, total 11 rows starting from the beginning of the part 2024.10.22 02:31:59.276556 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1648_1648_0 (state Temporary) 2024.10.22 02:31:59.339365 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1648_1648_0 to 202410_1661_1661_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:31:59.432526 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110675 2024.10.22 02:31:59.493714 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1656_6 to 202410_1661_1661_0 2024.10.22 02:31:59.512892 [ 1194 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:31:59.516107 [ 1194 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:31:59.551654 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1656_6 to 202410_1661_1661_0 into Compact with storage Full 2024.10.22 02:31:59.587553 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 6.94 MiB, will set to 1.58 GiB (RSS), difference: -7.15 MiB 2024.10.22 02:31:59.618032 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:31:59.636013 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1656_6, total 2181 rows starting from the beginning of the part 2024.10.22 02:31:59.672493 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1657_1657_0, total 43 rows starting from the beginning of the part 2024.10.22 02:31:59.729444 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1658_1658_0, total 70 rows starting from the beginning of the part 2024.10.22 02:31:59.765431 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1659_1659_0, total 26 rows starting from the beginning of the part 2024.10.22 02:31:59.933432 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1660_1660_0, total 78 rows starting from the beginning of the part 2024.10.22 02:31:59.939338 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_987_352} MergeTask::MergeProjectionsStage: Merge sorted 15132 rows, containing 12 columns (12 merged, 0 gathered) in 0.972713938 sec., 15556.474939706273 rows/sec., 2.44 MiB/sec. 2024.10.22 02:31:59.967485 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} MergeTreeSequentialSource: Reading 2 marks from part 202410_1661_1661_0, total 34 rows starting from the beginning of the part 2024.10.22 02:32:00.002984 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_987_352} MergedBlockOutputStream: filled checksums 202410_1_987_352 (state Temporary) 2024.10.22 02:32:00.046281 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:00.048946 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:32:00.214744 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:00.219487 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:32:00.367902 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_937_937_0 (state Temporary) 2024.10.22 02:32:00.457455 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_937_937_0 to 202410_944_944_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:00.468793 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_987_352} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_987_352 to 202410_1_987_352 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:00.471836 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_987_352} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_986_351, 202410_987_987_0] -> 202410_1_987_352 2024.10.22 02:32:00.494780 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1411528 2024.10.22 02:32:00.474860 [ 1179 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_934_173 to 202410_935_940_1 2024.10.22 02:32:00.508945 [ 1179 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.11 MiB using storage policy from min volume index 0 2024.10.22 02:32:00.511737 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:32:00.512576 [ 1179 ] {} DiskLocal: Reserved 1.11 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:32:00.544695 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_940_174} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_934_173 to 202410_935_940_1 into Wide with storage Full 2024.10.22 02:32:00.595085 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_940_174} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:00.614608 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_940_174} MergeTreeSequentialSource: Reading 174 marks from part 202410_1_934_173, total 1410097 rows starting from the beginning of the part 2024.10.22 02:32:00.696582 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_940_174} MergeTreeSequentialSource: Reading 3 marks from part 202410_935_940_1, total 8861 rows starting from the beginning of the part 2024.10.22 02:32:00.813830 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1556_103, 202410_1557_1557_0] 2024.10.22 02:32:00.818028 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1556_103, 202410_1557_1557_0] 2024.10.22 02:32:00.832233 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1556_103, 202410_1557_1557_0] 2024.10.22 02:32:01.062896 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_927_292, 202410_928_928_0] 2024.10.22 02:32:01.067629 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_927_292, 202410_928_928_0] 2024.10.22 02:32:01.078344 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_927_292, 202410_928_928_0] 2024.10.22 02:32:01.545750 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:32:01.550516 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:32:01.556278 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} MergeTask::MergeProjectionsStage: Merge sorted 2432 rows, containing 13 columns (13 merged, 0 gathered) in 2.008660934 sec., 1210.7568573840747 rows/sec., 210.28 KiB/sec. 2024.10.22 02:32:01.669598 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 7.45 MiB, will set to 1.58 GiB (RSS), difference: -9.61 MiB 2024.10.22 02:32:01.692510 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} MergedBlockOutputStream: filled checksums 202410_1626_1661_7 (state Temporary) 2024.10.22 02:32:02.442516 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1661_7 to 202410_1626_1661_7 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:02.446988 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1661_7} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1656_6, 202410_1661_1661_0] -> 202410_1626_1661_7 2024.10.22 02:32:02.490579 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:32:03.437003 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 60 entries to flush up to offset 110735 2024.10.22 02:32:03.637418 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_196_196_0 (state Temporary) 2024.10.22 02:32:03.828892 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 10.68 MiB, will set to 1.58 GiB (RSS), difference: 2.20 MiB 2024.10.22 02:32:03.895575 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:03.902757 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:32:04.114891 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_940_174/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:32:04.174993 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1649_1649_0 (state Temporary) 2024.10.22 02:32:04.350660 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1649_1649_0 to 202410_1662_1662_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:04.419888 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110735 2024.10.22 02:32:05.813293 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 12.11 MiB, will set to 1.58 GiB (RSS), difference: 3.56 MiB 2024.10.22 02:32:05.849592 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1557_104, 202410_1558_1558_0] 2024.10.22 02:32:05.851349 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1557_104, 202410_1558_1558_0] 2024.10.22 02:32:05.864040 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1557_104, 202410_1558_1558_0] 2024.10.22 02:32:06.073566 [ 1418 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:32:06.093355 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 95 entries to flush up to offset 108116 2024.10.22 02:32:06.413796 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 15107 2024.10.22 02:32:06.596436 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:06.606418 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:32:06.661734 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_962_962_0 (state Temporary) 2024.10.22 02:32:06.757362 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:06.760410 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:32:06.804144 [ 1247 ] {} DatabaseCatalog: Execution took 246 ms. 2024.10.22 02:32:06.894505 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_979_979_0 (state Temporary) 2024.10.22 02:32:06.944597 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_962_962_0 to 202410_968_968_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:06.979510 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 108116 2024.10.22 02:32:06.969775 [ 1198 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_963_63 to 202410_964_964_0 2024.10.22 02:32:07.094129 [ 1198 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 02:32:07.097178 [ 1198 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:32:07.126205 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_964_64} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_963_63 to 202410_964_964_0 into Wide with storage Full 2024.10.22 02:32:07.348000 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_964_64} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:07.358495 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_964_64} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_963_63, total 24236 rows starting from the beginning of the part 2024.10.22 02:32:07.415981 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_928_293, 202410_929_929_0] 2024.10.22 02:32:07.419923 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_979_979_0 to 202410_992_992_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:07.420492 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_928_293, 202410_929_929_0] 2024.10.22 02:32:07.468387 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15107 2024.10.22 02:32:07.449525 [ 1163 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_987_352 to 202410_988_988_0 2024.10.22 02:32:07.487616 [ 1163 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:07.500828 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1413068 2024.10.22 02:32:07.501540 [ 1163 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:32:07.535548 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_928_293, 202410_929_929_0] 2024.10.22 02:32:07.549827 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_964_64} MergeTreeSequentialSource: Reading 2 marks from part 202410_964_964_0, total 169 rows starting from the beginning of the part 2024.10.22 02:32:07.550535 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(17c32e65c44acb575d1c33f698bcfbcb) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 17c32e65c44acb575d1c33f698bcfbcb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:32:07.558291 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1060, file segment info: File segment: [0, 33554431], key: 17c32e65c44acb575d1c33f698bcfbcb, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1396, kind: Regular, unbound: 0 2024.10.22 02:32:07.560068 [ 1396 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:07.561253 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_988_353} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_987_352 to 202410_988_988_0 into Compact with storage Full 2024.10.22 02:32:07.606827 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_988_353} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:07.624515 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_988_353} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_987_352, total 15132 rows starting from the beginning of the part 2024.10.22 02:32:07.664104 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_988_353} MergeTreeSequentialSource: Reading 2 marks from part 202410_988_988_0, total 30 rows starting from the beginning of the part 2024.10.22 02:32:07.971239 [ 1680 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xev/moffpartbwvxgeaszpbyrmfaptoyh, size 12656903 2024.10.22 02:32:08.150276 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column CLID 2024.10.22 02:32:08.347364 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column CLID 2024.10.22 02:32:08.443961 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 51 entries to flush up to offset 110786 2024.10.22 02:32:08.452676 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column CLID 2024.10.22 02:32:08.495145 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column CLID 2024.10.22 02:32:08.609384 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column CLID 2024.10.22 02:32:08.651922 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 811 2024.10.22 02:32:08.704897 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:08.719740 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:08.745695 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column CLID 2024.10.22 02:32:08.760995 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_988_353/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:32:08.764919 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:08.765119 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1650_1650_0 (state Temporary) 2024.10.22 02:32:08.770562 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:08.817582 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(32a85387c16fec68d8d8f8fa9a0090e0) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 32a85387c16fec68d8d8f8fa9a0090e0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 02:32:08.823887 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: 32a85387c16fec68d8d8f8fa9a0090e0, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1398, kind: Regular, unbound: 0 2024.10.22 02:32:08.826705 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:08.853717 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column CLID 2024.10.22 02:32:08.854668 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1650_1650_0 to 202410_1663_1663_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:08.909660 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/tpl/amugnvbimujvcpnmayyedospckhna (LogSeriesLimiter: on interval from 2024-10-22 02:31:50 to 2024-10-22 02:32:08 accepted series 1 / 10 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:32:08.974603 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_938_938_0 (state Temporary) 2024.10.22 02:32:08.996245 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110786 2024.10.22 02:32:09.006937 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_988_353} MergeTask::MergeProjectionsStage: Merge sorted 15162 rows, containing 12 columns (12 merged, 0 gathered) in 1.453923275 sec., 10428.335704303241 rows/sec., 1.63 MiB/sec. 2024.10.22 02:32:09.060759 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_988_353} MergedBlockOutputStream: filled checksums 202410_1_988_353 (state Temporary) 2024.10.22 02:32:09.066808 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_938_938_0 to 202410_945_945_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:09.154079 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1413068 2024.10.22 02:32:09.351017 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:09.355616 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:09.420336 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_793_793_0 (state Temporary) 2024.10.22 02:32:09.537164 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_988_353} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_988_353 to 202410_1_988_353 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:09.543566 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_988_353} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_987_352, 202410_988_988_0] -> 202410_1_988_353 2024.10.22 02:32:09.562120 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_196_196_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:32:09.564179 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:32:09.574479 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_964_64/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:32:09.661165 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_793_793_0 to 202410_803_803_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:09.704390 [ 1213 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_798_162 to 202410_803_803_0 2024.10.22 02:32:09.746220 [ 1213 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:09.749649 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:09.784728 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_798_162 to 202410_803_803_0 into Compact with storage Full 2024.10.22 02:32:09.860326 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 811 2024.10.22 02:32:10.032524 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_964_64} MergeTask::MergeProjectionsStage: Merge sorted 24405 rows, containing 13 columns (13 merged, 0 gathered) in 2.925828316 sec., 8341.227633398843 rows/sec., 3.64 MiB/sec. 2024.10.22 02:32:10.062073 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1558_105, 202410_1559_1559_0] 2024.10.22 02:32:10.064542 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1558_105, 202410_1559_1559_0] 2024.10.22 02:32:10.075983 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1558_105, 202410_1559_1559_0] 2024.10.22 02:32:10.170064 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:10.191970 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_798_162, total 842 rows starting from the beginning of the part 2024.10.22 02:32:10.284893 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} MergeTreeSequentialSource: Reading 2 marks from part 202410_799_799_0, total 1 rows starting from the beginning of the part 2024.10.22 02:32:10.345237 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} MergeTreeSequentialSource: Reading 2 marks from part 202410_800_800_0, total 1 rows starting from the beginning of the part 2024.10.22 02:32:10.386457 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 11.51 MiB, will set to 1.58 GiB (RSS), difference: -26.42 MiB 2024.10.22 02:32:10.399108 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} MergeTreeSequentialSource: Reading 2 marks from part 202410_801_801_0, total 1 rows starting from the beginning of the part 2024.10.22 02:32:10.463525 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} MergeTreeSequentialSource: Reading 2 marks from part 202410_802_802_0, total 1 rows starting from the beginning of the part 2024.10.22 02:32:10.518925 [ 1389 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} MergeTreeSequentialSource: Reading 2 marks from part 202410_803_803_0, total 1 rows starting from the beginning of the part 2024.10.22 02:32:10.583189 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_964_64} MergedBlockOutputStream: filled checksums 202410_747_964_64 (state Temporary) 2024.10.22 02:32:11.245013 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_964_64} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_964_64 to 202410_747_964_64 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:11.256309 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_964_64} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_963_63, 202410_964_964_0] -> 202410_747_964_64 2024.10.22 02:32:11.273323 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.57 MiB. 2024.10.22 02:32:11.542017 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:32:11.544317 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:32:11.827219 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_196_196_0 to 202410_199_199_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:11.963033 [ 1272 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:32:11.970840 [ 1278 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Selected 6 parts from 202410_185_194_2 to 202410_199_199_0 2024.10.22 02:32:11.989298 [ 1272 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:32:11.991410 [ 1278 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:12.009471 [ 1278 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:12.032371 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} MergeTask::PrepareStage: Merging 6 parts: from 202410_185_194_2 to 202410_199_199_0 into Compact with storage Full 2024.10.22 02:32:12.399073 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 9.02 MiB, will set to 1.57 GiB (RSS), difference: 34.57 MiB 2024.10.22 02:32:12.809670 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(2ce1b8537a84d1fdcbc98cad1b17c2ca) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2ce1b8537a84d1fdcbc98cad1b17c2ca, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:32:12.814318 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 763, file segment info: File segment: [0, 33554431], key: 2ce1b8537a84d1fdcbc98cad1b17c2ca, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1399, kind: Regular, unbound: 0 2024.10.22 02:32:12.820050 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:12.863738 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} MergeTask::MergeProjectionsStage: Merge sorted 847 rows, containing 32 columns (32 merged, 0 gathered) in 3.089467847 sec., 274.15724712023524 rows/sec., 20.61 KiB/sec. 2024.10.22 02:32:12.948680 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:32:12.948870 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} MergedBlockOutputStream: filled checksums 202410_1_803_163 (state Temporary) 2024.10.22 02:32:12.955243 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:32:12.969129 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:32:13.010426 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 110858 2024.10.22 02:32:13.016765 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:32:13.050374 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:32:13.063425 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:32:13.082994 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:32:13.100697 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:32:13.135003 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column UserAgentMinor 2024.10.22 02:32:13.160838 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rjq/yheqkuznwxtfuwsufknfjgwqtuqur (LogSeriesLimiter: on interval from 2024-10-22 02:32:08 to 2024-10-22 02:32:13 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:32:13.353812 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:13.356152 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:13.436325 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1651_1651_0 (state Temporary) 2024.10.22 02:32:13.502305 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_803_163 to 202410_1_803_163 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:13.522490 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_803_163} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_798_162, 202410_803_803_0] -> 202410_1_803_163 2024.10.22 02:32:13.581027 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:32:13.584139 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1651_1651_0 to 202410_1664_1664_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:13.651893 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110858 2024.10.22 02:32:14.488607 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 9.64 MiB, will set to 1.57 GiB (RSS), difference: 1.48 MiB 2024.10.22 02:32:14.712546 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 174 entries to flush up to offset 108290 2024.10.22 02:32:14.827554 [ 1259 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 236 ms. 2024.10.22 02:32:14.712546 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7919 2024.10.22 02:32:14.972235 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 15127 2024.10.22 02:32:15.012896 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1559_106, 202410_1560_1560_0] 2024.10.22 02:32:15.017503 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1559_106, 202410_1560_1560_0] 2024.10.22 02:32:15.031162 [ 1424 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1559_106, 202410_1560_1560_0] 2024.10.22 02:32:15.071416 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:15.080173 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:15.160128 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_940_174/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:32:15.210893 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:15.213008 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:15.250207 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_980_980_0 (state Temporary) 2024.10.22 02:32:15.251345 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_963_963_0 (state Temporary) 2024.10.22 02:32:15.308232 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_963_963_0 to 202410_969_969_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:15.347182 [ 1222 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_964_64 to 202410_965_965_0 2024.10.22 02:32:15.355048 [ 1222 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.37 MiB using storage policy from min volume index 0 2024.10.22 02:32:15.358789 [ 1222 ] {} DiskLocal: Reserved 1.37 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:15.360956 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 108290 2024.10.22 02:32:15.407635 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_965_65} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_964_64 to 202410_965_965_0 into Wide with storage Full 2024.10.22 02:32:15.431193 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_965_65} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:15.438688 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_980_980_0 to 202410_993_993_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:15.440391 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_965_65} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_964_64, total 24405 rows starting from the beginning of the part 2024.10.22 02:32:15.464673 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_988_353 to 202410_989_989_0 2024.10.22 02:32:15.470729 [ 1224 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:15.479929 [ 1224 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:15.546411 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_989_354} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_988_353 to 202410_989_989_0 into Compact with storage Full 2024.10.22 02:32:15.578174 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15127 2024.10.22 02:32:15.596525 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_965_65} MergeTreeSequentialSource: Reading 2 marks from part 202410_965_965_0, total 79 rows starting from the beginning of the part 2024.10.22 02:32:15.643519 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_989_354} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:15.673614 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_989_354} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_988_353, total 15162 rows starting from the beginning of the part 2024.10.22 02:32:15.851994 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_989_354} MergeTreeSequentialSource: Reading 2 marks from part 202410_989_989_0, total 16 rows starting from the beginning of the part 2024.10.22 02:32:16.161263 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1414608 2024.10.22 02:32:16.344982 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 13.19 MiB, will set to 1.57 GiB (RSS), difference: -48.68 MiB 2024.10.22 02:32:16.574459 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_929_294, 202410_930_930_0] 2024.10.22 02:32:16.576893 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_929_294, 202410_930_930_0] 2024.10.22 02:32:16.593182 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_929_294, 202410_930_930_0] 2024.10.22 02:32:16.597391 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_989_354} MergeTask::MergeProjectionsStage: Merge sorted 15178 rows, containing 12 columns (12 merged, 0 gathered) in 1.092302268 sec., 13895.421116163057 rows/sec., 2.18 MiB/sec. 2024.10.22 02:32:16.654524 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_989_354} MergedBlockOutputStream: filled checksums 202410_1_989_354 (state Temporary) 2024.10.22 02:32:16.970998 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:16.973491 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:17.310569 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_989_354} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_989_354 to 202410_1_989_354 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:17.317658 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_989_354} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_988_353, 202410_989_989_0] -> 202410_1_989_354 2024.10.22 02:32:17.352021 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:32:17.362109 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 812 2024.10.22 02:32:17.402182 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_939_939_0 (state Temporary) 2024.10.22 02:32:17.470059 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_939_939_0 to 202410_946_946_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:17.509610 [ 1237 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_941_941_0 to 202410_946_946_0 2024.10.22 02:32:17.515697 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1414608 2024.10.22 02:32:17.526233 [ 1237 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:17.530012 [ 1237 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:17.573034 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_941_941_0 to 202410_946_946_0 into Compact with storage Full 2024.10.22 02:32:17.620349 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:17.635613 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_941_941_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:32:17.643891 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:17.646820 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_942_942_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:32:17.648046 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:17.654944 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_943_943_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:32:17.665401 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 110932 2024.10.22 02:32:17.673668 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_944_944_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:32:17.796410 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_945_945_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:32:17.843420 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:17.847373 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:17.916399 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_794_794_0 (state Temporary) 2024.10.22 02:32:17.927771 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1652_1652_0 (state Temporary) 2024.10.22 02:32:17.806767 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_965_65} MergeTask::MergeProjectionsStage: Merge sorted 24484 rows, containing 13 columns (13 merged, 0 gathered) in 2.410420333 sec., 10157.564498108639 rows/sec., 4.43 MiB/sec. 2024.10.22 02:32:17.967608 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1652_1652_0 to 202410_1665_1665_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:18.027097 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_946_946_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:32:18.040786 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_965_65} MergedBlockOutputStream: filled checksums 202410_747_965_65 (state Temporary) 2024.10.22 02:32:18.120554 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_794_794_0 to 202410_804_804_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:18.239045 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110932 2024.10.22 02:32:18.318185 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 812 2024.10.22 02:32:18.581549 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 14.77 MiB, will set to 1.57 GiB (RSS), difference: 44.78 MiB 2024.10.22 02:32:18.828540 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_965_65} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_965_65 to 202410_747_965_65 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:18.836510 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_965_65} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_964_64, 202410_965_965_0] -> 202410_747_965_65 2024.10.22 02:32:18.855243 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.52 MiB. 2024.10.22 02:32:19.059766 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} MergeTask::MergeProjectionsStage: Merge sorted 9240 rows, containing 4 columns (4 merged, 0 gathered) in 1.494672273 sec., 6181.957186811346 rows/sec., 96.59 KiB/sec. 2024.10.22 02:32:19.197979 [ 1389 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} MergedBlockOutputStream: filled checksums 202410_941_946_1 (state Temporary) 2024.10.22 02:32:19.337133 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_941_946_1 to 202410_941_946_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:19.343845 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_941_946_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_941_941_0, 202410_946_946_0] -> 202410_941_946_1 2024.10.22 02:32:19.466194 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1560_107, 202410_1561_1561_0] 2024.10.22 02:32:19.469044 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1560_107, 202410_1561_1561_0] 2024.10.22 02:32:19.492430 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1560_107, 202410_1561_1561_0] 2024.10.22 02:32:19.579914 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:32:19.602882 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:20.149701 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_185_194_2, total 415 rows starting from the beginning of the part 2024.10.22 02:32:20.496428 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 15.21 MiB, will set to 1.57 GiB (RSS), difference: 8.68 MiB 2024.10.22 02:32:21.543365 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:32:21.558611 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:32:21.688897 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_874_163, 202410_875_880_1] 2024.10.22 02:32:21.693101 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_874_163, 202410_875_880_1] 2024.10.22 02:32:21.855367 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_874_163, 202410_875_880_1] 2024.10.22 02:32:22.252097 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 35 entries to flush up to offset 110967 2024.10.22 02:32:22.364819 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 42 entries to flush up to offset 7961 2024.10.22 02:32:22.459910 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.07 MiB, will set to 1.57 GiB (RSS), difference: -2.28 MiB 2024.10.22 02:32:22.492026 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:22.495049 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:22.661998 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1653_1653_0 (state Temporary) 2024.10.22 02:32:22.730215 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1653_1653_0 to 202410_1666_1666_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:22.741608 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_195_195_0, total 39 rows starting from the beginning of the part 2024.10.22 02:32:22.852821 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 110967 2024.10.22 02:32:22.856268 [ 1197 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1661_7 to 202410_1666_1666_0 2024.10.22 02:32:22.877186 [ 1197 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:22.885655 [ 1197 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:22.898561 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 187 entries to flush up to offset 108477 2024.10.22 02:32:22.911968 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1661_7 to 202410_1666_1666_0 into Compact with storage Full 2024.10.22 02:32:23.051818 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:23.062968 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1661_7, total 2432 rows starting from the beginning of the part 2024.10.22 02:32:23.086139 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 15146 2024.10.22 02:32:23.185565 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1662_1662_0, total 60 rows starting from the beginning of the part 2024.10.22 02:32:23.187930 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:23.192555 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:23.339908 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:23.342667 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1663_1663_0, total 51 rows starting from the beginning of the part 2024.10.22 02:32:23.346328 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:23.364883 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_964_964_0 (state Temporary) 2024.10.22 02:32:23.376315 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1664_1664_0, total 72 rows starting from the beginning of the part 2024.10.22 02:32:23.499225 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1665_1665_0, total 74 rows starting from the beginning of the part 2024.10.22 02:32:23.553840 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_964_964_0 to 202410_970_970_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:23.576225 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_965_65 to 202410_966_966_0 2024.10.22 02:32:23.581449 [ 1191 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 02:32:23.587933 [ 1191 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:23.601832 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_981_981_0 (state Temporary) 2024.10.22 02:32:23.641705 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} MergeTreeSequentialSource: Reading 2 marks from part 202410_1666_1666_0, total 35 rows starting from the beginning of the part 2024.10.22 02:32:23.668405 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_981_981_0 to 202410_994_994_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:23.699663 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 108477 2024.10.22 02:32:23.748801 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_966_66} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_965_65 to 202410_966_966_0 into Wide with storage Full 2024.10.22 02:32:23.812871 [ 1258 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_989_354 to 202410_990_990_0 2024.10.22 02:32:23.825344 [ 1258 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:23.843660 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15146 2024.10.22 02:32:23.855884 [ 1258 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:23.907019 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_990_355} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_989_354 to 202410_990_990_0 into Compact with storage Full 2024.10.22 02:32:23.941572 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_966_66} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:23.956715 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_966_66} MergeTreeSequentialSource: Reading 4 marks from part 202410_747_965_65, total 24484 rows starting from the beginning of the part 2024.10.22 02:32:23.995810 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_990_355} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:24.015231 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_990_355} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_989_354, total 15178 rows starting from the beginning of the part 2024.10.22 02:32:24.041101 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_907_32, 202410_908_908_0, 202410_909_909_0, 202410_910_910_0, 202410_911_911_0, 202410_912_912_0] 2024.10.22 02:32:24.104637 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_990_355} MergeTreeSequentialSource: Reading 2 marks from part 202410_990_990_0, total 9 rows starting from the beginning of the part 2024.10.22 02:32:24.154126 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_966_66} MergeTreeSequentialSource: Reading 2 marks from part 202410_966_966_0, total 101 rows starting from the beginning of the part 2024.10.22 02:32:24.332652 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1561_108, 202410_1562_1562_0] 2024.10.22 02:32:24.336942 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1561_108, 202410_1562_1562_0] 2024.10.22 02:32:24.348769 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1561_108, 202410_1562_1562_0] 2024.10.22 02:32:24.044135 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_907_32, 202410_908_908_0, 202410_909_909_0, 202410_910_910_0, 202410_911_911_0, 202410_912_912_0] 2024.10.22 02:32:24.477178 [ 1422 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_907_32, 202410_908_908_0, 202410_909_909_0, 202410_910_910_0, 202410_911_911_0, 202410_912_912_0] 2024.10.22 02:32:24.626588 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1542 entries to flush up to offset 1416150 2024.10.22 02:32:24.692233 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_930_295, 202410_931_931_0] 2024.10.22 02:32:24.758439 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 9.43 MiB, will set to 1.57 GiB (RSS), difference: -36.96 MiB 2024.10.22 02:32:24.799070 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_930_295, 202410_931_931_0] 2024.10.22 02:32:24.833662 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_930_295, 202410_931_931_0] 2024.10.22 02:32:25.025457 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_196_196_0, total 39 rows starting from the beginning of the part 2024.10.22 02:32:25.198378 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} MergeTask::MergeProjectionsStage: Merge sorted 2724 rows, containing 13 columns (13 merged, 0 gathered) in 2.297220282 sec., 1185.78092895316 rows/sec., 204.65 KiB/sec. 2024.10.22 02:32:25.369911 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} MergedBlockOutputStream: filled checksums 202410_1626_1666_8 (state Temporary) 2024.10.22 02:32:25.377201 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:25.385087 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:25.350639 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_990_355} MergeTask::MergeProjectionsStage: Merge sorted 15187 rows, containing 12 columns (12 merged, 0 gathered) in 1.453060345 sec., 10451.733854177955 rows/sec., 1.64 MiB/sec. 2024.10.22 02:32:25.453057 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1666_8 to 202410_1626_1666_8 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:25.464139 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1666_8} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1661_7, 202410_1666_1666_0] -> 202410_1626_1666_8 2024.10.22 02:32:25.509230 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:32:25.588402 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_990_355} MergedBlockOutputStream: filled checksums 202410_1_990_355 (state Temporary) 2024.10.22 02:32:25.729743 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_940_940_0 (state Temporary) 2024.10.22 02:32:25.804931 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_940_174/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:32:25.823441 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 813 2024.10.22 02:32:25.964115 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_990_355} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_990_355 to 202410_1_990_355 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:25.977632 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_990_355} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_989_354, 202410_990_990_0] -> 202410_1_990_355 2024.10.22 02:32:26.044063 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:32:26.168311 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:26.175196 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:26.184644 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_940_940_0 to 202410_947_947_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:26.248267 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_795_795_0 (state Temporary) 2024.10.22 02:32:26.336542 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1416150 2024.10.22 02:32:26.352920 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_795_795_0 to 202410_805_805_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:26.456609 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 813 2024.10.22 02:32:26.857912 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 111047 2024.10.22 02:32:27.002128 [ 1228 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:32:27.041141 [ 1228 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:32:27.065246 [ 1261 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 02:32:27.174103 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} WriteBufferFromS3: Close WriteBufferFromS3. Details: bucket test, key common/hlk/ownsabaaotdxbywjohcirjbkfxhvp. 2024.10.22 02:32:27.176378 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(801ff329077e91970a3879ec61588ac1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 801ff329077e91970a3879ec61588ac1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:32:27.202059 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 581928, file segment info: File segment: [0, 33554431], key: 801ff329077e91970a3879ec61588ac1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:32:27.310932 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:27.351037 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_197_197_0, total 38 rows starting from the beginning of the part 2024.10.22 02:32:27.369756 [ 1269 ] {} DatabaseCatalog: Execution took 363 ms. 2024.10.22 02:32:27.387189 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_966_66/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:32:27.498813 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1562_109, 202410_1563_1563_0] 2024.10.22 02:32:27.523479 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1562_109, 202410_1563_1563_0] 2024.10.22 02:32:27.542304 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1562_109, 202410_1563_1563_0] 2024.10.22 02:32:27.545722 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:27.551127 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:27.751825 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1654_1654_0 (state Temporary) 2024.10.22 02:32:27.768706 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_966_66} MergeTask::MergeProjectionsStage: Merge sorted 24585 rows, containing 13 columns (13 merged, 0 gathered) in 4.023338219 sec., 6110.597385001999 rows/sec., 2.66 MiB/sec. 2024.10.22 02:32:27.834449 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1654_1654_0 to 202410_1667_1667_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:27.865407 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_966_66} MergedBlockOutputStream: filled checksums 202410_747_966_66 (state Temporary) 2024.10.22 02:32:27.868378 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111047 2024.10.22 02:32:28.260532 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_966_66} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_966_66 to 202410_747_966_66 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:28.262610 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_966_66} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_965_65, 202410_966_966_0] -> 202410_747_966_66 2024.10.22 02:32:28.272050 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 55.57 MiB. 2024.10.22 02:32:28.295967 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(8b858b8d7988a8540b292410e1ebcb94) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8b858b8d7988a8540b292410e1ebcb94, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:32:28.314287 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 867, file segment info: File segment: [0, 33554431], key: 8b858b8d7988a8540b292410e1ebcb94, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:32:28.321607 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:28.393119 [ 1693 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/tpl/amugnvbimujvcpnmayyedospckhna, size 581928 2024.10.22 02:32:28.748156 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column YCLID 2024.10.22 02:32:28.779919 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column YCLID 2024.10.22 02:32:28.919177 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column YCLID 2024.10.22 02:32:28.955101 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column YCLID 2024.10.22 02:32:29.029026 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column YCLID 2024.10.22 02:32:29.067012 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column YCLID 2024.10.22 02:32:29.172249 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column YCLID 2024.10.22 02:32:29.288364 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xkt/gdbmfvzjqucccxrhsugzltjiruazl (LogSeriesLimiter: on interval from 2024-10-22 02:32:13 to 2024-10-22 02:32:29 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:32:29.488975 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_198_198_0, total 41 rows starting from the beginning of the part 2024.10.22 02:32:29.636287 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.44 GiB, peak 2.00 GiB, free memory in arenas 8.35 MiB, will set to 1.57 GiB (RSS), difference: 128.24 MiB 2024.10.22 02:32:31.255106 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 184 entries to flush up to offset 108661 2024.10.22 02:32:31.347478 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 15164 2024.10.22 02:32:31.349656 [ 1396 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} MergeTreeSequentialSource: Reading 2 marks from part 202410_199_199_0, total 41 rows starting from the beginning of the part 2024.10.22 02:32:31.493358 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:31.496803 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:31.533545 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.46 GiB, peak 2.00 GiB, free memory in arenas 12.88 MiB, will set to 1.56 GiB (RSS), difference: 102.30 MiB 2024.10.22 02:32:31.545479 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:32:31.552089 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_965_965_0 (state Temporary) 2024.10.22 02:32:31.553399 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:32:31.643984 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:31.646813 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_965_965_0 to 202410_971_971_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:31.647477 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:31.673660 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_966_66 to 202410_967_967_0 2024.10.22 02:32:31.680276 [ 1248 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.38 MiB using storage policy from min volume index 0 2024.10.22 02:32:31.687461 [ 1248 ] {} DiskLocal: Reserved 1.38 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:31.694465 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 108661 2024.10.22 02:32:31.699630 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_982_982_0 (state Temporary) 2024.10.22 02:32:31.707262 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_967_67} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_966_66 to 202410_967_967_0 into Wide with storage Full 2024.10.22 02:32:31.873304 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_967_67} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:31.892162 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 56 entries to flush up to offset 111103 2024.10.22 02:32:31.899666 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_967_67} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_966_66, total 24585 rows starting from the beginning of the part 2024.10.22 02:32:31.900205 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_982_982_0 to 202410_995_995_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:31.945115 [ 1267 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_990_355 to 202410_991_991_0 2024.10.22 02:32:31.967134 [ 1267 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:31.969220 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15164 2024.10.22 02:32:31.973490 [ 1267 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:32.011793 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_991_356} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_990_355 to 202410_991_991_0 into Compact with storage Full 2024.10.22 02:32:32.094125 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_991_356} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:32.119677 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_991_356} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_990_355, total 15187 rows starting from the beginning of the part 2024.10.22 02:32:32.202696 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_967_67} MergeTreeSequentialSource: Reading 2 marks from part 202410_967_967_0, total 81 rows starting from the beginning of the part 2024.10.22 02:32:32.206880 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:32.211112 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:32.266765 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1655_1655_0 (state Temporary) 2024.10.22 02:32:32.313180 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1655_1655_0 to 202410_1668_1668_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:32.362329 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111103 2024.10.22 02:32:32.366852 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_991_356} MergeTreeSequentialSource: Reading 2 marks from part 202410_991_991_0, total 11 rows starting from the beginning of the part 2024.10.22 02:32:32.465198 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(731947aacb5c3ccdc13ebcce9d176b82) : [0, 211]: Setting download as failed: File segment: [0, 211], key: 731947aacb5c3ccdc13ebcce9d176b82, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:32.466416 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(731947aacb5c3ccdc13ebcce9d176b82): No space left in cache to reserve 212 bytes, will continue without cache download 2024.10.22 02:32:32.719281 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1563_110, 202410_1564_1564_0] 2024.10.22 02:32:32.727241 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1563_110, 202410_1564_1564_0] 2024.10.22 02:32:32.744905 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1563_110, 202410_1564_1564_0] 2024.10.22 02:32:33.148074 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(24b7070f14ebf221b82636c83758901d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 24b7070f14ebf221b82636c83758901d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 02:32:33.150495 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 589172, file segment info: File segment: [0, 33554431], key: 24b7070f14ebf221b82636c83758901d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 02:32:33.151980 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:33.254583 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_931_296, 202410_932_932_0] 2024.10.22 02:32:33.255892 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_931_296, 202410_932_932_0] 2024.10.22 02:32:33.265632 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_931_296, 202410_932_932_0] 2024.10.22 02:32:33.343301 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1417690 2024.10.22 02:32:33.418964 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_991_356} MergeTask::MergeProjectionsStage: Merge sorted 15198 rows, containing 12 columns (12 merged, 0 gathered) in 1.416950877 sec., 10725.84819043095 rows/sec., 1.68 MiB/sec. 2024.10.22 02:32:33.616047 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_991_356} MergedBlockOutputStream: filled checksums 202410_1_991_356 (state Temporary) 2024.10.22 02:32:33.848660 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 10.72 MiB, will set to 1.57 GiB (RSS), difference: -57.63 MiB 2024.10.22 02:32:33.927892 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_991_356} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_991_356 to 202410_1_991_356 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:33.932548 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_991_356} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_990_355, 202410_991_991_0] -> 202410_1_991_356 2024.10.22 02:32:33.937284 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ea9f02da2fb667bfe8582c7ac21726cd) : [0, 2303664]: Setting download as failed: File segment: [0, 2303664], key: ea9f02da2fb667bfe8582c7ac21726cd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:33.939696 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ea9f02da2fb667bfe8582c7ac21726cd): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:32:33.958825 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 814 2024.10.22 02:32:33.967548 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:32:33.974557 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:33.990321 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:34.166742 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:34.183324 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:34.232599 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ea9f02da2fb667bfe8582c7ac21726cd, offset: 0, size: 0 2024.10.22 02:32:34.235943 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ea9f02da2fb667bfe8582c7ac21726cd for removal 2024.10.22 02:32:34.244608 [ 1476 ] {} CacheMetadata: Key ea9f02da2fb667bfe8582c7ac21726cd is removed from metadata 2024.10.22 02:32:34.247526 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 731947aacb5c3ccdc13ebcce9d176b82, offset: 0, size: 0 2024.10.22 02:32:34.249779 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 731947aacb5c3ccdc13ebcce9d176b82 for removal 2024.10.22 02:32:34.252333 [ 1476 ] {} CacheMetadata: Key 731947aacb5c3ccdc13ebcce9d176b82 is removed from metadata 2024.10.22 02:32:34.256445 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_796_796_0 (state Temporary) 2024.10.22 02:32:34.257559 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_941_941_0 (state Temporary) 2024.10.22 02:32:34.397365 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_941_941_0 to 202410_948_948_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:34.444552 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_796_796_0 to 202410_806_806_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:34.542459 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1417690 2024.10.22 02:32:34.603858 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 814 2024.10.22 02:32:34.663444 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_967_67} MergeTask::MergeProjectionsStage: Merge sorted 24666 rows, containing 13 columns (13 merged, 0 gathered) in 2.965960148 sec., 8316.36258384413 rows/sec., 3.62 MiB/sec. 2024.10.22 02:32:34.747045 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_967_67} MergedBlockOutputStream: filled checksums 202410_747_967_67 (state Temporary) 2024.10.22 02:32:34.885178 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(13afe892f3cef51aa730e43b5948cf56) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 13afe892f3cef51aa730e43b5948cf56, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 02:32:34.892911 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 743, file segment info: File segment: [0, 33554431], key: 13afe892f3cef51aa730e43b5948cf56, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1389, kind: Regular, unbound: 0 2024.10.22 02:32:34.901066 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:34.966119 [ 1690 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/rjq/yheqkuznwxtfuwsufknfjgwqtuqur, size 589172 2024.10.22 02:32:35.042472 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:32:35.045783 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:32:35.057201 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:32:35.066090 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:32:35.091905 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:32:35.131597 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:32:35.166146 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:32:35.199957 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:32:35.239495 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column CookieEnable 2024.10.22 02:32:35.295099 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xta/ehhrqldgmuchohctclyimyetwlpej (LogSeriesLimiter: on interval from 2024-10-22 02:32:29 to 2024-10-22 02:32:35 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:32:35.626116 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_967_67} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_967_67 to 202410_747_967_67 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:35.636846 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_967_67} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_966_66, 202410_967_967_0] -> 202410_747_967_67 2024.10.22 02:32:35.713613 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 58.32 MiB. 2024.10.22 02:32:35.941147 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(823eb24f894d3179d6dacb95303f439c) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 823eb24f894d3179d6dacb95303f439c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:35.943467 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(823eb24f894d3179d6dacb95303f439c): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:32:36.472942 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 76 entries to flush up to offset 111179 2024.10.22 02:32:36.737924 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:36.741794 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:36.928434 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1656_1656_0 (state Temporary) 2024.10.22 02:32:36.990220 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ad54968d8d1df93dac9a005f6cc39257) : [0, 44173]: Setting download as failed: File segment: [0, 44173], key: ad54968d8d1df93dac9a005f6cc39257, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:36.992702 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ad54968d8d1df93dac9a005f6cc39257): No space left in cache to reserve 44174 bytes, will continue without cache download 2024.10.22 02:32:37.118992 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ad54968d8d1df93dac9a005f6cc39257, offset: 0, size: 0 2024.10.22 02:32:37.122272 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ad54968d8d1df93dac9a005f6cc39257 for removal 2024.10.22 02:32:37.133737 [ 1476 ] {} CacheMetadata: Key ad54968d8d1df93dac9a005f6cc39257 is removed from metadata 2024.10.22 02:32:37.217110 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1656_1656_0 to 202410_1669_1669_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:37.223482 [ 1422 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_940_174/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:32:37.233724 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 823eb24f894d3179d6dacb95303f439c, offset: 0, size: 0 2024.10.22 02:32:37.235594 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 823eb24f894d3179d6dacb95303f439c for removal 2024.10.22 02:32:37.238638 [ 1476 ] {} CacheMetadata: Key 823eb24f894d3179d6dacb95303f439c is removed from metadata 2024.10.22 02:32:37.383016 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111179 2024.10.22 02:32:37.684949 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 13.70 MiB, will set to 1.56 GiB (RSS), difference: 49.70 MiB 2024.10.22 02:32:37.785309 [ 1254 ] {} DatabaseCatalog: Execution took 415 ms. 2024.10.22 02:32:38.208130 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_753_153, 202410_754_754_0, 202410_755_755_0, 202410_756_756_0, 202410_757_757_0, 202410_758_758_0] 2024.10.22 02:32:38.210241 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_753_153, 202410_754_754_0, 202410_755_755_0, 202410_756_756_0, 202410_757_757_0, 202410_758_758_0] 2024.10.22 02:32:38.252192 [ 1422 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_753_153, 202410_754_754_0, 202410_755_755_0, 202410_756_756_0, 202410_757_757_0, 202410_758_758_0] 2024.10.22 02:32:38.552174 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ab0a2e258965a6657992a5ccf5076661) : [0, 166]: Setting download as failed: File segment: [0, 166], key: ab0a2e258965a6657992a5ccf5076661, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:38.553604 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ab0a2e258965a6657992a5ccf5076661): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 02:32:39.238059 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 178 entries to flush up to offset 108839 2024.10.22 02:32:39.266938 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:39.271915 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:39.473719 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 15176 2024.10.22 02:32:39.574248 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(02ae50d5601d646459845c0c7124a82a) : [0, 2572]: Setting download as failed: File segment: [0, 2572], key: 02ae50d5601d646459845c0c7124a82a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:39.576357 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(02ae50d5601d646459845c0c7124a82a): No space left in cache to reserve 2573 bytes, will continue without cache download 2024.10.22 02:32:39.608880 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 02ae50d5601d646459845c0c7124a82a, offset: 0, size: 0 2024.10.22 02:32:39.612919 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 02ae50d5601d646459845c0c7124a82a for removal 2024.10.22 02:32:39.616669 [ 1476 ] {} CacheMetadata: Key 02ae50d5601d646459845c0c7124a82a is removed from metadata 2024.10.22 02:32:39.618744 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ab0a2e258965a6657992a5ccf5076661, offset: 0, size: 0 2024.10.22 02:32:39.623040 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ab0a2e258965a6657992a5ccf5076661 for removal 2024.10.22 02:32:39.628151 [ 1476 ] {} CacheMetadata: Key ab0a2e258965a6657992a5ccf5076661 is removed from metadata 2024.10.22 02:32:39.665461 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:39.673643 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:39.697012 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:39.701008 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:39.748946 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_983_983_0 (state Temporary) 2024.10.22 02:32:39.753207 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_966_966_0 (state Temporary) 2024.10.22 02:32:39.810717 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_983_983_0 to 202410_996_996_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:39.943816 [ 1172 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_991_356 to 202410_992_992_0 2024.10.22 02:32:39.950374 [ 1172 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:39.970807 [ 1172 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:39.978105 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15176 2024.10.22 02:32:40.024113 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_966_966_0 to 202410_972_972_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:40.025594 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1564_111, 202410_1565_1565_0, 202410_1566_1566_0, 202410_1567_1567_0, 202410_1568_1568_0, 202410_1569_1569_0] 2024.10.22 02:32:40.027797 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1564_111, 202410_1565_1565_0, 202410_1566_1566_0, 202410_1567_1567_0, 202410_1568_1568_0, 202410_1569_1569_0] 2024.10.22 02:32:40.128995 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_992_357} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_991_356 to 202410_992_992_0 into Compact with storage Full 2024.10.22 02:32:40.162569 [ 1181 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_967_67 to 202410_968_968_0 2024.10.22 02:32:40.168619 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1564_111, 202410_1565_1565_0, 202410_1566_1566_0, 202410_1567_1567_0, 202410_1568_1568_0, 202410_1569_1569_0] 2024.10.22 02:32:40.173159 [ 1181 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.39 MiB using storage policy from min volume index 0 2024.10.22 02:32:40.181171 [ 1181 ] {} DiskLocal: Reserved 1.39 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:40.256305 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_968_68} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_967_67 to 202410_968_968_0 into Wide with storage Full 2024.10.22 02:32:40.279179 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 108839 2024.10.22 02:32:40.316346 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_992_357} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:40.344747 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_992_357} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_991_356, total 15198 rows starting from the beginning of the part 2024.10.22 02:32:40.365650 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_968_68} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:40.460021 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 13.10 MiB, will set to 1.57 GiB (RSS), difference: -1.69 MiB 2024.10.22 02:32:40.507084 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_968_68} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_967_67, total 24666 rows starting from the beginning of the part 2024.10.22 02:32:40.526376 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(69ba6ba2710cb33d5547494f42e0800d) : [0, 168]: Setting download as failed: File segment: [0, 168], key: 69ba6ba2710cb33d5547494f42e0800d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:40.528603 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(69ba6ba2710cb33d5547494f42e0800d): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 02:32:40.658760 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_992_357} MergeTreeSequentialSource: Reading 2 marks from part 202410_992_992_0, total 15 rows starting from the beginning of the part 2024.10.22 02:32:40.878497 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_968_68} MergeTreeSequentialSource: Reading 2 marks from part 202410_968_968_0, total 95 rows starting from the beginning of the part 2024.10.22 02:32:41.403666 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 111243 2024.10.22 02:32:41.543936 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:32:41.545824 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:32:41.547386 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1418845 2024.10.22 02:32:41.745142 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_992_357} MergeTask::MergeProjectionsStage: Merge sorted 15213 rows, containing 12 columns (12 merged, 0 gathered) in 1.731508174 sec., 8785.982202357192 rows/sec., 1.38 MiB/sec. 2024.10.22 02:32:41.796053 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_992_357} MergedBlockOutputStream: filled checksums 202410_1_992_357 (state Temporary) 2024.10.22 02:32:41.846957 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0cb1b8726d807efd3941fdbae8cb5de3) : [0, 1050940]: Setting download as failed: File segment: [0, 1050940], key: 0cb1b8726d807efd3941fdbae8cb5de3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:41.848100 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0cb1b8726d807efd3941fdbae8cb5de3): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:32:41.877807 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0cb1b8726d807efd3941fdbae8cb5de3, offset: 0, size: 0 2024.10.22 02:32:41.880838 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0cb1b8726d807efd3941fdbae8cb5de3 for removal 2024.10.22 02:32:41.886004 [ 1476 ] {} CacheMetadata: Key 0cb1b8726d807efd3941fdbae8cb5de3 is removed from metadata 2024.10.22 02:32:41.889723 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 69ba6ba2710cb33d5547494f42e0800d, offset: 0, size: 0 2024.10.22 02:32:41.893633 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 69ba6ba2710cb33d5547494f42e0800d for removal 2024.10.22 02:32:41.898755 [ 1476 ] {} CacheMetadata: Key 69ba6ba2710cb33d5547494f42e0800d is removed from metadata 2024.10.22 02:32:41.905161 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_932_297, 202410_933_933_0] 2024.10.22 02:32:41.908735 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_932_297, 202410_933_933_0] 2024.10.22 02:32:41.927396 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_932_297, 202410_933_933_0] 2024.10.22 02:32:41.995499 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:41.999459 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:42.046187 [ 1220 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:32:42.062568 [ 1220 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:32:42.106891 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 815 2024.10.22 02:32:42.220084 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_992_357} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_992_357 to 202410_1_992_357 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:42.226094 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_992_357} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_991_356, 202410_992_992_0] -> 202410_1_992_357 2024.10.22 02:32:42.255318 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:32:42.337739 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:42.336801 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1657_1657_0 (state Temporary) 2024.10.22 02:32:42.340580 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:42.394912 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_940_174} MergeTask::MergeProjectionsStage: Merge sorted 1418958 rows, containing 4 columns (4 merged, 0 gathered) in 41.856539493 sec., 33900.50914833281 rows/sec., 508.46 KiB/sec. 2024.10.22 02:32:42.477026 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_197_197_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:32:42.478219 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Found 7 old parts to remove. Parts: [202410_1_174_28, 202410_175_179_1, 202410_180_180_0, 202410_181_181_0, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0] 2024.10.22 02:32:42.491812 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from filesystem (serially): Parts: [202410_1_174_28, 202410_175_179_1, 202410_180_180_0, 202410_181_181_0, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0] 2024.10.22 02:32:42.504746 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:42.510356 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:42.510772 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_942_942_0 (state Temporary) 2024.10.22 02:32:42.510924 [ 1396 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_940_174} MergedBlockOutputStream: filled checksums 202410_1_940_174 (state Temporary) 2024.10.22 02:32:42.570907 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_797_797_0 (state Temporary) 2024.10.22 02:32:42.596478 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_942_942_0 to 202410_949_949_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:42.601399 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1657_1657_0 to 202410_1670_1670_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:42.635182 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Removing 7 parts from memory: Parts: [202410_1_174_28, 202410_175_179_1, 202410_180_180_0, 202410_181_181_0, 202410_182_182_0, 202410_183_183_0, 202410_184_184_0] 2024.10.22 02:32:42.641508 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 12.01 MiB, will set to 1.57 GiB (RSS), difference: -54.97 MiB 2024.10.22 02:32:42.643126 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1418845 2024.10.22 02:32:42.668123 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111243 2024.10.22 02:32:42.749371 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_797_797_0 to 202410_807_807_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:42.933283 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 815 2024.10.22 02:32:43.139178 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a19a87fc6e44bee741613656c6a59495) : [0, 166]: Setting download as failed: File segment: [0, 166], key: a19a87fc6e44bee741613656c6a59495, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:43.140947 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a19a87fc6e44bee741613656c6a59495): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 02:32:43.373604 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_197_197_0 (state Temporary) 2024.10.22 02:32:43.498396 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_968_68} MergeTask::MergeProjectionsStage: Merge sorted 24761 rows, containing 13 columns (13 merged, 0 gathered) in 3.245317471 sec., 7629.762025214204 rows/sec., 3.32 MiB/sec. 2024.10.22 02:32:43.923100 [ 1424 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 2 similar messages) 2024.10.22 02:32:43.983810 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_968_68} MergedBlockOutputStream: filled checksums 202410_747_968_68 (state Temporary) 2024.10.22 02:32:44.256140 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(33972cc97a125ae5be13484e0470569c) : [0, 31313]: Setting download as failed: File segment: [0, 31313], key: 33972cc97a125ae5be13484e0470569c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:44.257642 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(33972cc97a125ae5be13484e0470569c): No space left in cache to reserve 31314 bytes, will continue without cache download 2024.10.22 02:32:44.287433 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 33972cc97a125ae5be13484e0470569c, offset: 0, size: 0 2024.10.22 02:32:44.290649 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 33972cc97a125ae5be13484e0470569c for removal 2024.10.22 02:32:44.296133 [ 1476 ] {} CacheMetadata: Key 33972cc97a125ae5be13484e0470569c is removed from metadata 2024.10.22 02:32:44.297130 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a19a87fc6e44bee741613656c6a59495, offset: 0, size: 0 2024.10.22 02:32:44.299444 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a19a87fc6e44bee741613656c6a59495 for removal 2024.10.22 02:32:44.304855 [ 1476 ] {} CacheMetadata: Key a19a87fc6e44bee741613656c6a59495 is removed from metadata 2024.10.22 02:32:44.516502 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_968_68} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_968_68 to 202410_747_968_68 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:44.522899 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_968_68} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_967_67, 202410_968_968_0] -> 202410_747_968_68 2024.10.22 02:32:44.547161 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 58.30 MiB. 2024.10.22 02:32:44.567642 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 5.04 MiB, will set to 1.56 GiB (RSS), difference: 47.19 MiB 2024.10.22 02:32:45.144057 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f2c87977014fb6a9115da5d3b31b2efb) : [0, 150]: Setting download as failed: File segment: [0, 150], key: f2c87977014fb6a9115da5d3b31b2efb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:45.146668 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f2c87977014fb6a9115da5d3b31b2efb): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 02:32:46.294952 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_887_887_0, 202410_888_888_0, 202410_889_889_0, 202410_890_890_0, 202410_891_891_0, 202410_892_892_0] 2024.10.22 02:32:46.315057 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_887_887_0, 202410_888_888_0, 202410_889_889_0, 202410_890_890_0, 202410_891_891_0, 202410_892_892_0] 2024.10.22 02:32:46.341340 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_887_887_0, 202410_888_888_0, 202410_889_889_0, 202410_890_890_0, 202410_891_891_0, 202410_892_892_0] 2024.10.22 02:32:46.374894 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_940_174} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_940_174 to 202410_1_940_174 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:46.484201 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_940_174} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_934_173, 202410_935_940_1] -> 202410_1_940_174 2024.10.22 02:32:46.514411 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(62fa8074b56e7a5f357c9b336cbcc8b7) : [0, 24112]: Setting download as failed: File segment: [0, 24112], key: 62fa8074b56e7a5f357c9b336cbcc8b7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:46.521408 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(62fa8074b56e7a5f357c9b336cbcc8b7): No space left in cache to reserve 24113 bytes, will continue without cache download 2024.10.22 02:32:46.533597 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 62fa8074b56e7a5f357c9b336cbcc8b7, offset: 0, size: 0 2024.10.22 02:32:46.536869 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 62fa8074b56e7a5f357c9b336cbcc8b7 for removal 2024.10.22 02:32:46.542451 [ 1476 ] {} CacheMetadata: Key 62fa8074b56e7a5f357c9b336cbcc8b7 is removed from metadata 2024.10.22 02:32:46.542812 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:32:46.547974 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f2c87977014fb6a9115da5d3b31b2efb, offset: 0, size: 0 2024.10.22 02:32:46.550874 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f2c87977014fb6a9115da5d3b31b2efb for removal 2024.10.22 02:32:46.555020 [ 1476 ] {} CacheMetadata: Key f2c87977014fb6a9115da5d3b31b2efb is removed from metadata 2024.10.22 02:32:46.676228 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 79 entries to flush up to offset 111322 2024.10.22 02:32:46.893673 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:46.896951 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:47.061618 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1658_1658_0 (state Temporary) 2024.10.22 02:32:47.436584 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1658_1658_0 to 202410_1671_1671_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:47.471203 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1666_8 to 202410_1671_1671_0 2024.10.22 02:32:47.485268 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 15 entries to flush up to offset 15191 2024.10.22 02:32:47.488186 [ 1213 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:47.493392 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:47.600549 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111322 2024.10.22 02:32:47.630002 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d0dfc837fb4802f524bbb00ae1e0fa1c) : [0, 155]: Setting download as failed: File segment: [0, 155], key: d0dfc837fb4802f524bbb00ae1e0fa1c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:47.634201 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d0dfc837fb4802f524bbb00ae1e0fa1c): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:32:47.640127 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1666_8 to 202410_1671_1671_0 into Compact with storage Full 2024.10.22 02:32:47.711281 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:47.832516 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1666_8, total 2724 rows starting from the beginning of the part 2024.10.22 02:32:47.855375 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 176 entries to flush up to offset 109015 2024.10.22 02:32:47.867679 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1667_1667_0, total 80 rows starting from the beginning of the part 2024.10.22 02:32:47.875557 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:47.879517 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:47.905692 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1668_1668_0, total 56 rows starting from the beginning of the part 2024.10.22 02:32:47.918795 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_984_984_0 (state Temporary) 2024.10.22 02:32:47.954374 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1669_1669_0, total 76 rows starting from the beginning of the part 2024.10.22 02:32:47.965538 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_984_984_0 to 202410_997_997_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:47.985554 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1670_1670_0, total 64 rows starting from the beginning of the part 2024.10.22 02:32:47.987941 [ 1230 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_992_357 to 202410_993_993_0 2024.10.22 02:32:47.991292 [ 1230 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:47.993231 [ 1230 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:48.013109 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_993_358} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_992_357 to 202410_993_993_0 into Compact with storage Full 2024.10.22 02:32:48.023991 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} MergeTreeSequentialSource: Reading 2 marks from part 202410_1671_1671_0, total 79 rows starting from the beginning of the part 2024.10.22 02:32:48.059817 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_993_358} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:48.066275 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_993_358} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_992_357, total 15213 rows starting from the beginning of the part 2024.10.22 02:32:48.110162 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15191 2024.10.22 02:32:48.149312 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:48.152241 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:48.089467 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_993_358} MergeTreeSequentialSource: Reading 2 marks from part 202410_993_993_0, total 20 rows starting from the beginning of the part 2024.10.22 02:32:48.235889 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_967_967_0 (state Temporary) 2024.10.22 02:32:48.405026 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(7490f349ec05d3b38fd8fceabe35deec) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7490f349ec05d3b38fd8fceabe35deec, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:32:48.406315 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_967_967_0 to 202410_973_973_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:48.411634 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 19002, file segment info: File segment: [0, 33554431], key: 7490f349ec05d3b38fd8fceabe35deec, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:32:48.415370 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:48.442445 [ 1216 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_968_68 to 202410_969_969_0 2024.10.22 02:32:48.446567 [ 1216 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.40 MiB using storage policy from min volume index 0 2024.10.22 02:32:48.449386 [ 1216 ] {} DiskLocal: Reserved 1.40 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:48.547918 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 109015 2024.10.22 02:32:48.737552 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_969_69} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_968_68 to 202410_969_969_0 into Wide with storage Full 2024.10.22 02:32:48.835367 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_933_298, 202410_934_934_0] 2024.10.22 02:32:48.840997 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_933_298, 202410_934_934_0] 2024.10.22 02:32:48.862127 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_933_298, 202410_934_934_0] 2024.10.22 02:32:48.942369 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_969_69} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:48.978643 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_969_69} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_968_68, total 24761 rows starting from the beginning of the part 2024.10.22 02:32:49.019261 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_993_358} MergeTask::MergeProjectionsStage: Merge sorted 15233 rows, containing 12 columns (12 merged, 0 gathered) in 1.021264745 sec., 14915.818914320791 rows/sec., 2.34 MiB/sec. 2024.10.22 02:32:49.110756 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} MergeTask::MergeProjectionsStage: Merge sorted 3079 rows, containing 13 columns (13 merged, 0 gathered) in 1.491436565 sec., 2064.452536739301 rows/sec., 356.10 KiB/sec. 2024.10.22 02:32:49.194485 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_993_358} MergedBlockOutputStream: filled checksums 202410_1_993_358 (state Temporary) 2024.10.22 02:32:49.252066 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} MergedBlockOutputStream: filled checksums 202410_1626_1671_9 (state Temporary) 2024.10.22 02:32:49.317639 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f7c69e99fcea046fe6b306cb4287bd15) : [0, 235582]: Setting download as failed: File segment: [0, 235582], key: f7c69e99fcea046fe6b306cb4287bd15, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:49.321519 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f7c69e99fcea046fe6b306cb4287bd15): No space left in cache to reserve 235583 bytes, will continue without cache download 2024.10.22 02:32:49.354931 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_969_69} MergeTreeSequentialSource: Reading 2 marks from part 202410_969_969_0, total 174 rows starting from the beginning of the part 2024.10.22 02:32:49.362246 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f7c69e99fcea046fe6b306cb4287bd15, offset: 0, size: 0 2024.10.22 02:32:49.363334 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f7c69e99fcea046fe6b306cb4287bd15 for removal 2024.10.22 02:32:49.364632 [ 1476 ] {} CacheMetadata: Key f7c69e99fcea046fe6b306cb4287bd15 is removed from metadata 2024.10.22 02:32:49.365663 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d0dfc837fb4802f524bbb00ae1e0fa1c, offset: 0, size: 0 2024.10.22 02:32:49.367900 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d0dfc837fb4802f524bbb00ae1e0fa1c for removal 2024.10.22 02:32:49.370297 [ 1476 ] {} CacheMetadata: Key d0dfc837fb4802f524bbb00ae1e0fa1c is removed from metadata 2024.10.22 02:32:49.497362 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_993_358} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_993_358 to 202410_1_993_358 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:49.510687 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_993_358} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_992_357, 202410_993_993_0] -> 202410_1_993_358 2024.10.22 02:32:49.547532 [ 1391 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:32:49.650819 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 948 entries to flush up to offset 1419793 2024.10.22 02:32:49.679496 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1671_9 to 202410_1626_1671_9 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:49.693273 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1671_9} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1666_8, 202410_1671_1671_0] -> 202410_1626_1671_9 2024.10.22 02:32:49.846519 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:32:50.198610 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(52c3af816c4a976d58cf5d28de8c828c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 52c3af816c4a976d58cf5d28de8c828c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:32:50.200778 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:50.204317 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 355, file segment info: File segment: [0, 33554431], key: 52c3af816c4a976d58cf5d28de8c828c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:32:50.207131 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:50.210801 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:50.428128 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_943_943_0 (state Temporary) 2024.10.22 02:32:50.479312 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_943_943_0 to 202410_950_950_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:50.611082 [ 1252 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_940_174 to 202410_941_946_1 2024.10.22 02:32:50.614928 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1419793 2024.10.22 02:32:50.623004 [ 1252 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.12 MiB using storage policy from min volume index 0 2024.10.22 02:32:50.651974 [ 1252 ] {} DiskLocal: Reserved 1.12 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:50.672868 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_946_175} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_940_174 to 202410_941_946_1 into Wide with storage Full 2024.10.22 02:32:50.709645 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_946_175} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:50.730143 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_946_175} MergeTreeSequentialSource: Reading 175 marks from part 202410_1_940_174, total 1418958 rows starting from the beginning of the part 2024.10.22 02:32:50.762419 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_946_175} MergeTreeSequentialSource: Reading 3 marks from part 202410_941_946_1, total 9240 rows starting from the beginning of the part 2024.10.22 02:32:50.843594 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 4.51 MiB, will set to 1.56 GiB (RSS), difference: -50.68 MiB 2024.10.22 02:32:51.100737 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(17aba3c52198b9f953e86c30c8eea2f2) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 17aba3c52198b9f953e86c30c8eea2f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:51.104545 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(17aba3c52198b9f953e86c30c8eea2f2): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 02:32:51.546057 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:32:51.547493 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:32:51.714604 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 90 entries to flush up to offset 111412 2024.10.22 02:32:51.886867 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(0862d38847f4d10ca42a9977b9b05a7e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 0862d38847f4d10ca42a9977b9b05a7e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:32:51.894832 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 5185, file segment info: File segment: [0, 33554431], key: 0862d38847f4d10ca42a9977b9b05a7e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:32:51.897209 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:52.051809 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.080587329 sec. 2024.10.22 02:32:52.185427 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:52.192520 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:52.249450 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_197_197_0 to 202410_200_200_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:52.407130 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1659_1659_0 (state Temporary) 2024.10.22 02:32:52.460926 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1659_1659_0 to 202410_1672_1672_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:52.516975 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111412 2024.10.22 02:32:52.607458 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_969_69} MergeTask::MergeProjectionsStage: Merge sorted 24935 rows, containing 13 columns (13 merged, 0 gathered) in 3.977472486 sec., 6269.056564883049 rows/sec., 2.73 MiB/sec. 2024.10.22 02:32:52.658642 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_969_69} MergedBlockOutputStream: filled checksums 202410_747_969_69 (state Temporary) 2024.10.22 02:32:52.947927 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0b2aab8079cf56bedebba45238bc51e9) : [0, 154140]: Setting download as failed: File segment: [0, 154140], key: 0b2aab8079cf56bedebba45238bc51e9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:52.949705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0b2aab8079cf56bedebba45238bc51e9): No space left in cache to reserve 154141 bytes, will continue without cache download 2024.10.22 02:32:52.961302 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0b2aab8079cf56bedebba45238bc51e9, offset: 0, size: 0 2024.10.22 02:32:52.963389 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0b2aab8079cf56bedebba45238bc51e9 for removal 2024.10.22 02:32:52.965082 [ 1476 ] {} CacheMetadata: Key 0b2aab8079cf56bedebba45238bc51e9 is removed from metadata 2024.10.22 02:32:52.966624 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 17aba3c52198b9f953e86c30c8eea2f2, offset: 0, size: 0 2024.10.22 02:32:52.968430 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 17aba3c52198b9f953e86c30c8eea2f2 for removal 2024.10.22 02:32:52.970700 [ 1476 ] {} CacheMetadata: Key 17aba3c52198b9f953e86c30c8eea2f2 is removed from metadata 2024.10.22 02:32:53.476655 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_969_69} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_969_69 to 202410_747_969_69 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:53.481030 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_969_69} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_968_68, 202410_969_969_0] -> 202410_747_969_69 2024.10.22 02:32:53.608415 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 58.40 MiB. 2024.10.22 02:32:53.600403 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 8.41 MiB, will set to 1.56 GiB (RSS), difference: 52.35 MiB 2024.10.22 02:32:53.885094 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(e9a6a3befe14fb100b5ae4a5ec925194) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e9a6a3befe14fb100b5ae4a5ec925194, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:32:53.896589 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 504, file segment info: File segment: [0, 33554431], key: e9a6a3befe14fb100b5ae4a5ec925194, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1385, kind: Regular, unbound: 0 2024.10.22 02:32:53.897819 [ 1385 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:54.047183 [ 1702 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xkt/gdbmfvzjqucccxrhsugzltjiruazl, size 5185 2024.10.22 02:32:54.096979 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ShareService 2024.10.22 02:32:54.225310 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ShareService 2024.10.22 02:32:54.243850 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ShareService 2024.10.22 02:32:54.244102 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_946_175/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:32:54.261766 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ShareService 2024.10.22 02:32:54.287897 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ShareService 2024.10.22 02:32:54.512982 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ShareService 2024.10.22 02:32:54.526550 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ShareService 2024.10.22 02:32:54.660783 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(8d98e9e579c67cb2ed176681aba0fdc7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8d98e9e579c67cb2ed176681aba0fdc7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 02:32:54.662831 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 25115, file segment info: File segment: [0, 33554431], key: 8d98e9e579c67cb2ed176681aba0fdc7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 02:32:54.664348 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:54.695296 [ 1399 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/fph/ovxcdcmhkpihzsicxbkmhqcxnmdvw (LogSeriesLimiter: on interval from 2024-10-22 02:32:35 to 2024-10-22 02:32:54 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:32:54.713988 [ 1424 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_199_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:32:55.346886 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(0c67f66865b84a52d59209a064cb9b8a) : [0, 181]: Setting download as failed: File segment: [0, 181], key: 0c67f66865b84a52d59209a064cb9b8a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:55.348514 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(0c67f66865b84a52d59209a064cb9b8a): No space left in cache to reserve 182 bytes, will continue without cache download 2024.10.22 02:32:55.466795 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 7961 2024.10.22 02:32:55.613564 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 15209 2024.10.22 02:32:56.036860 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:56.041707 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:56.076620 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 200 entries to flush up to offset 109215 2024.10.22 02:32:56.081207 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_985_985_0 (state Temporary) 2024.10.22 02:32:56.139497 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(a570c18310c1716afd4e132981c009e7) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a570c18310c1716afd4e132981c009e7, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 02:32:56.142075 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 567, file segment info: File segment: [0, 33554431], key: a570c18310c1716afd4e132981c009e7, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 02:32:56.146101 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:32:56.187518 [ 1701 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/xta/ehhrqldgmuchohctclyimyetwlpej, size 25115 2024.10.22 02:32:56.257444 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_934_299, 202410_935_935_0] 2024.10.22 02:32:56.260810 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_985_985_0 to 202410_998_998_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:56.261122 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_934_299, 202410_935_935_0] 2024.10.22 02:32:56.277253 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_934_299, 202410_935_935_0] 2024.10.22 02:32:56.399261 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_993_358 to 202410_994_994_0 2024.10.22 02:32:56.409735 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:56.410418 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15209 2024.10.22 02:32:56.411605 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:56.437567 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_994_359} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_993_358 to 202410_994_994_0 into Compact with storage Full 2024.10.22 02:32:56.451224 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:56.451242 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 15.95 MiB, will set to 1.57 GiB (RSS), difference: 4.27 MiB 2024.10.22 02:32:56.452626 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:56.496333 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_968_968_0 (state Temporary) 2024.10.22 02:32:56.546188 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 111479 2024.10.22 02:32:56.573623 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_994_359} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:56.595069 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_994_359} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_993_358, total 15233 rows starting from the beginning of the part 2024.10.22 02:32:56.660754 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_968_968_0 to 202410_974_974_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:56.736589 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:32:56.755320 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:32:56.762160 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 109215 2024.10.22 02:32:56.772680 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:32:56.775060 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_994_359} MergeTreeSequentialSource: Reading 2 marks from part 202410_994_994_0, total 19 rows starting from the beginning of the part 2024.10.22 02:32:56.797370 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:32:56.801042 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:56.808733 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:56.814080 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:32:56.843174 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:32:56.964437 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:32:56.970289 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1660_1660_0 (state Temporary) 2024.10.22 02:32:56.979731 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:32:57.002949 [ 1389 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column JavascriptEnable 2024.10.22 02:32:57.069771 [ 1240 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:32:57.097659 [ 1240 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:32:57.116848 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(20f5e1e62b5671be435672e5cabe9723) : [0, 347746]: Setting download as failed: File segment: [0, 347746], key: 20f5e1e62b5671be435672e5cabe9723, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:57.119329 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(20f5e1e62b5671be435672e5cabe9723): No space left in cache to reserve 347747 bytes, will continue without cache download 2024.10.22 02:32:57.163708 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1660_1660_0 to 202410_1673_1673_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:57.261521 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 20f5e1e62b5671be435672e5cabe9723, offset: 0, size: 0 2024.10.22 02:32:57.263176 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 20f5e1e62b5671be435672e5cabe9723 for removal 2024.10.22 02:32:57.264409 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111479 2024.10.22 02:32:57.268365 [ 1476 ] {} CacheMetadata: Key 20f5e1e62b5671be435672e5cabe9723 is removed from metadata 2024.10.22 02:32:57.268366 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 0c67f66865b84a52d59209a064cb9b8a, offset: 0, size: 0 2024.10.22 02:32:57.272658 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 0c67f66865b84a52d59209a064cb9b8a for removal 2024.10.22 02:32:57.276352 [ 1476 ] {} CacheMetadata: Key 0c67f66865b84a52d59209a064cb9b8a is removed from metadata 2024.10.22 02:32:57.403465 [ 1170 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 219 ms. 2024.10.22 02:32:57.569609 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_994_359} MergeTask::MergeProjectionsStage: Merge sorted 15252 rows, containing 12 columns (12 merged, 0 gathered) in 1.144153244 sec., 13330.382166883932 rows/sec., 2.09 MiB/sec. 2024.10.22 02:32:57.620709 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1286 entries to flush up to offset 1421079 2024.10.22 02:32:57.639533 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_994_359} MergedBlockOutputStream: filled checksums 202410_1_994_359 (state Temporary) 2024.10.22 02:32:57.937270 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 816 2024.10.22 02:32:58.269624 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_994_359} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_994_359 to 202410_1_994_359 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:58.277067 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_994_359} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_993_358, 202410_994_994_0] -> 202410_1_994_359 2024.10.22 02:32:58.315871 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:32:58.366866 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f34a251e8d6578d1da27b039052275f5) : [0, 155]: Setting download as failed: File segment: [0, 155], key: f34a251e8d6578d1da27b039052275f5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:58.367970 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f34a251e8d6578d1da27b039052275f5): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:32:58.463110 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:58.466732 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:58.560067 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_798_798_0 (state Temporary) 2024.10.22 02:32:58.572307 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:58.579806 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:58.727251 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_798_798_0 to 202410_808_808_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:58.760688 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_944_944_0 (state Temporary) 2024.10.22 02:32:58.761752 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 17.74 MiB, will set to 1.57 GiB (RSS), difference: -4.57 MiB 2024.10.22 02:32:58.863036 [ 1169 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_803_163 to 202410_808_808_0 2024.10.22 02:32:58.873573 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_944_944_0 to 202410_951_951_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:32:58.874711 [ 1169 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:32:58.877290 [ 1169 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:32:58.907124 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 816 2024.10.22 02:32:59.021118 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1421079 2024.10.22 02:32:59.032489 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_803_163 to 202410_808_808_0 into Compact with storage Full 2024.10.22 02:32:59.262381 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:32:59.300113 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_803_163, total 847 rows starting from the beginning of the part 2024.10.22 02:32:59.358475 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_804_804_0, total 1 rows starting from the beginning of the part 2024.10.22 02:32:59.548163 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_805_805_0, total 1 rows starting from the beginning of the part 2024.10.22 02:32:59.601525 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(932491a6944fdca2f600770df3abb957) : [0, 67719]: Setting download as failed: File segment: [0, 67719], key: 932491a6944fdca2f600770df3abb957, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:32:59.606660 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(932491a6944fdca2f600770df3abb957): No space left in cache to reserve 67720 bytes, will continue without cache download 2024.10.22 02:32:59.616606 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_806_806_0, total 1 rows starting from the beginning of the part 2024.10.22 02:32:59.639580 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 932491a6944fdca2f600770df3abb957, offset: 0, size: 0 2024.10.22 02:32:59.642810 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 932491a6944fdca2f600770df3abb957 for removal 2024.10.22 02:32:59.645297 [ 1476 ] {} CacheMetadata: Key 932491a6944fdca2f600770df3abb957 is removed from metadata 2024.10.22 02:32:59.646485 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f34a251e8d6578d1da27b039052275f5, offset: 0, size: 0 2024.10.22 02:32:59.648709 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f34a251e8d6578d1da27b039052275f5 for removal 2024.10.22 02:32:59.650260 [ 1476 ] {} CacheMetadata: Key f34a251e8d6578d1da27b039052275f5 is removed from metadata 2024.10.22 02:32:59.662070 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_807_807_0, total 1 rows starting from the beginning of the part 2024.10.22 02:32:59.999182 [ 1397 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} MergeTreeSequentialSource: Reading 2 marks from part 202410_808_808_0, total 1 rows starting from the beginning of the part 2024.10.22 02:33:00.871857 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(447faa67c613613f13bd5709b9ec2cfb) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 447faa67c613613f13bd5709b9ec2cfb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:00.873329 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(447faa67c613613f13bd5709b9ec2cfb): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 02:33:01.298899 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 72 entries to flush up to offset 111551 2024.10.22 02:33:01.459483 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 14.54 MiB, will set to 1.57 GiB (RSS), difference: -2.64 MiB 2024.10.22 02:33:01.547369 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:33:01.549667 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:33:01.554741 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:01.556361 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:01.717558 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1661_1661_0 (state Temporary) 2024.10.22 02:33:01.723709 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4082efc4f96045a6a276f27f5f4d97ef) : [0, 74473]: Setting download as failed: File segment: [0, 74473], key: 4082efc4f96045a6a276f27f5f4d97ef, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:01.726740 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4082efc4f96045a6a276f27f5f4d97ef): No space left in cache to reserve 74474 bytes, will continue without cache download 2024.10.22 02:33:01.748873 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4082efc4f96045a6a276f27f5f4d97ef, offset: 0, size: 0 2024.10.22 02:33:01.752727 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4082efc4f96045a6a276f27f5f4d97ef for removal 2024.10.22 02:33:01.755107 [ 1476 ] {} CacheMetadata: Key 4082efc4f96045a6a276f27f5f4d97ef is removed from metadata 2024.10.22 02:33:01.756419 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 447faa67c613613f13bd5709b9ec2cfb, offset: 0, size: 0 2024.10.22 02:33:01.759096 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_808_164/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:33:01.759167 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 447faa67c613613f13bd5709b9ec2cfb for removal 2024.10.22 02:33:01.762609 [ 1476 ] {} CacheMetadata: Key 447faa67c613613f13bd5709b9ec2cfb is removed from metadata 2024.10.22 02:33:02.057126 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1661_1661_0 to 202410_1674_1674_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:02.057673 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1569_112, 202410_1570_1570_0] 2024.10.22 02:33:02.061182 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1569_112, 202410_1570_1570_0] 2024.10.22 02:33:02.076334 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1569_112, 202410_1570_1570_0] 2024.10.22 02:33:02.142877 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111551 2024.10.22 02:33:02.243887 [ 1394 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} MergeTask::MergeProjectionsStage: Merge sorted 852 rows, containing 32 columns (32 merged, 0 gathered) in 3.227015292 sec., 264.0210606104559 rows/sec., 19.83 KiB/sec. 2024.10.22 02:33:02.295303 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} MergedBlockOutputStream: filled checksums 202410_1_808_164 (state Temporary) 2024.10.22 02:33:02.480220 [ 1247 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 02:33:02.730882 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(25a78c716b9881796733cbab3be1b552) : [0, 167]: Setting download as failed: File segment: [0, 167], key: 25a78c716b9881796733cbab3be1b552, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:02.735378 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(25a78c716b9881796733cbab3be1b552): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 02:33:03.029380 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 8002 2024.10.22 02:33:03.042990 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_808_164 to 202410_1_808_164 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:03.055046 [ 1399 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_808_164} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_803_163, 202410_808_808_0] -> 202410_1_808_164 2024.10.22 02:33:03.088655 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:33:03.403178 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 14.76 MiB, will set to 1.56 GiB (RSS), difference: 2.62 MiB 2024.10.22 02:33:03.740429 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7580041948a3cd563d2c8a0ac2c3bcef) : [0, 113283]: Setting download as failed: File segment: [0, 113283], key: 7580041948a3cd563d2c8a0ac2c3bcef, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:03.742016 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7580041948a3cd563d2c8a0ac2c3bcef): No space left in cache to reserve 113284 bytes, will continue without cache download 2024.10.22 02:33:03.772982 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7580041948a3cd563d2c8a0ac2c3bcef, offset: 0, size: 0 2024.10.22 02:33:03.774168 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7580041948a3cd563d2c8a0ac2c3bcef for removal 2024.10.22 02:33:03.778993 [ 1476 ] {} CacheMetadata: Key 7580041948a3cd563d2c8a0ac2c3bcef is removed from metadata 2024.10.22 02:33:03.778736 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 25a78c716b9881796733cbab3be1b552, offset: 0, size: 0 2024.10.22 02:33:03.784862 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 25a78c716b9881796733cbab3be1b552 for removal 2024.10.22 02:33:03.790227 [ 1476 ] {} CacheMetadata: Key 25a78c716b9881796733cbab3be1b552 is removed from metadata 2024.10.22 02:33:03.915303 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 16 entries to flush up to offset 15225 2024.10.22 02:33:04.105350 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:04.111768 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:04.154389 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_986_986_0 (state Temporary) 2024.10.22 02:33:04.212445 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_986_986_0 to 202410_999_999_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:04.352193 [ 1207 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_994_359 to 202410_995_995_0 2024.10.22 02:33:04.366264 [ 1207 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:04.370304 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15225 2024.10.22 02:33:04.370602 [ 1207 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:04.393510 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 86 entries to flush up to offset 109301 2024.10.22 02:33:04.515672 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_995_360} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_994_359 to 202410_995_995_0 into Compact with storage Full 2024.10.22 02:33:04.551794 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_995_360} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:04.556877 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_995_360} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_994_359, total 15252 rows starting from the beginning of the part 2024.10.22 02:33:04.571809 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_995_360} MergeTreeSequentialSource: Reading 2 marks from part 202410_995_995_0, total 18 rows starting from the beginning of the part 2024.10.22 02:33:04.861562 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:04.864180 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:04.899024 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(fb50048ebd683b7d2a37b98ba54dec28) : [0, 167]: Setting download as failed: File segment: [0, 167], key: fb50048ebd683b7d2a37b98ba54dec28, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:04.903632 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(fb50048ebd683b7d2a37b98ba54dec28): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 02:33:05.051778 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_969_969_0 (state Temporary) 2024.10.22 02:33:05.094529 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_969_969_0 to 202410_975_975_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:05.237646 [ 1211 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_969_69 to 202410_970_970_0 2024.10.22 02:33:05.245794 [ 1211 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.41 MiB using storage policy from min volume index 0 2024.10.22 02:33:05.250342 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 109301 2024.10.22 02:33:05.250917 [ 1211 ] {} DiskLocal: Reserved 1.41 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:05.278493 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_970_70} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_969_69 to 202410_970_970_0 into Wide with storage Full 2024.10.22 02:33:05.363003 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_970_70} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:05.374230 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_970_70} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_969_69, total 24935 rows starting from the beginning of the part 2024.10.22 02:33:05.501704 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 13.83 MiB, will set to 1.56 GiB (RSS), difference: -9.11 MiB 2024.10.22 02:33:05.695947 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 3 old parts to remove. Parts: [202410_747_912_33, 202410_913_913_0, 202410_914_914_0] 2024.10.22 02:33:05.702158 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 3 parts from filesystem (serially): Parts: [202410_747_912_33, 202410_913_913_0, 202410_914_914_0] 2024.10.22 02:33:05.707186 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_970_70} MergeTreeSequentialSource: Reading 2 marks from part 202410_970_970_0, total 187 rows starting from the beginning of the part 2024.10.22 02:33:05.741136 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_935_300, 202410_936_936_0] 2024.10.22 02:33:05.746894 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_935_300, 202410_936_936_0] 2024.10.22 02:33:05.809096 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 3 parts from memory: Parts: [202410_747_912_33, 202410_913_913_0, 202410_914_914_0] 2024.10.22 02:33:05.857678 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_935_300, 202410_936_936_0] 2024.10.22 02:33:05.969759 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_946_175/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:33:06.044236 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1231 entries to flush up to offset 1422310 2024.10.22 02:33:06.112985 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_995_360} MergeTask::MergeProjectionsStage: Merge sorted 15270 rows, containing 12 columns (12 merged, 0 gathered) in 1.705076793 sec., 8955.608370654776 rows/sec., 1.40 MiB/sec. 2024.10.22 02:33:06.160577 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 111628 2024.10.22 02:33:06.245558 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1570_113, 202410_1571_1571_0] 2024.10.22 02:33:06.248549 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1570_113, 202410_1571_1571_0] 2024.10.22 02:33:06.261373 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1570_113, 202410_1571_1571_0] 2024.10.22 02:33:06.300709 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_995_360} MergedBlockOutputStream: filled checksums 202410_1_995_360 (state Temporary) 2024.10.22 02:33:06.325238 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(966d2ed1da0c9b04bbdf0d20cbfd1e08) : [0, 109246]: Setting download as failed: File segment: [0, 109246], key: 966d2ed1da0c9b04bbdf0d20cbfd1e08, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:06.329790 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(966d2ed1da0c9b04bbdf0d20cbfd1e08): No space left in cache to reserve 109247 bytes, will continue without cache download 2024.10.22 02:33:06.410601 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 817 2024.10.22 02:33:06.466426 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 966d2ed1da0c9b04bbdf0d20cbfd1e08, offset: 0, size: 0 2024.10.22 02:33:06.470322 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 966d2ed1da0c9b04bbdf0d20cbfd1e08 for removal 2024.10.22 02:33:06.473186 [ 1476 ] {} CacheMetadata: Key 966d2ed1da0c9b04bbdf0d20cbfd1e08 is removed from metadata 2024.10.22 02:33:06.474156 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: fb50048ebd683b7d2a37b98ba54dec28, offset: 0, size: 0 2024.10.22 02:33:06.478999 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key fb50048ebd683b7d2a37b98ba54dec28 for removal 2024.10.22 02:33:06.485745 [ 1476 ] {} CacheMetadata: Key fb50048ebd683b7d2a37b98ba54dec28 is removed from metadata 2024.10.22 02:33:06.568051 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:06.677090 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:06.709225 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_995_360} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_995_360 to 202410_1_995_360 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:06.716204 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_995_360} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_994_359, 202410_995_995_0] -> 202410_1_995_360 2024.10.22 02:33:06.765776 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:33:06.789207 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:06.796949 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:06.803419 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:06.806719 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:06.876659 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1662_1662_0 (state Temporary) 2024.10.22 02:33:06.972476 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1662_1662_0 to 202410_1675_1675_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:07.030577 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_945_945_0 (state Temporary) 2024.10.22 02:33:07.032312 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_799_799_0 (state Temporary) 2024.10.22 02:33:07.053134 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111628 2024.10.22 02:33:07.164349 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_945_945_0 to 202410_952_952_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:07.220106 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1422310 2024.10.22 02:33:07.215772 [ 1175 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_947_947_0 to 202410_952_952_0 2024.10.22 02:33:07.260474 [ 1175 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:07.263934 [ 1175 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:07.327320 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_947_947_0 to 202410_952_952_0 into Compact with storage Full 2024.10.22 02:33:07.380811 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:07.457514 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_199_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:33:07.475394 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_799_799_0 to 202410_809_809_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:07.504446 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_947_947_0, total 1542 rows starting from the beginning of the part 2024.10.22 02:33:07.561089 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_948_948_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:33:07.624602 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_949_949_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:33:07.658989 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5094bc70b2368f54c4e00dfcf23acacf) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 5094bc70b2368f54c4e00dfcf23acacf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:07.661120 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5094bc70b2368f54c4e00dfcf23acacf): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:33:07.678928 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 817 2024.10.22 02:33:07.715781 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_950_950_0, total 948 rows starting from the beginning of the part 2024.10.22 02:33:07.768627 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_951_951_0, total 1286 rows starting from the beginning of the part 2024.10.22 02:33:08.033303 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_952_952_0, total 1231 rows starting from the beginning of the part 2024.10.22 02:33:08.295343 [ 1164 ] {} DatabaseCatalog: Execution took 271 ms. 2024.10.22 02:33:08.491803 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 11.26 MiB, will set to 1.57 GiB (RSS), difference: -30.29 MiB 2024.10.22 02:33:08.688893 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_970_70} MergeTask::MergeProjectionsStage: Merge sorted 25122 rows, containing 13 columns (13 merged, 0 gathered) in 3.417319372 sec., 7351.376112469478 rows/sec., 3.20 MiB/sec. 2024.10.22 02:33:08.828415 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ef03fa630734dbe24eeefcb6fdc61efd) : [0, 53874]: Setting download as failed: File segment: [0, 53874], key: ef03fa630734dbe24eeefcb6fdc61efd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:08.831538 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ef03fa630734dbe24eeefcb6fdc61efd): No space left in cache to reserve 53875 bytes, will continue without cache download 2024.10.22 02:33:08.869459 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_970_70} MergedBlockOutputStream: filled checksums 202410_747_970_70 (state Temporary) 2024.10.22 02:33:08.863539 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ef03fa630734dbe24eeefcb6fdc61efd, offset: 0, size: 0 2024.10.22 02:33:08.874233 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ef03fa630734dbe24eeefcb6fdc61efd for removal 2024.10.22 02:33:08.877254 [ 1476 ] {} CacheMetadata: Key ef03fa630734dbe24eeefcb6fdc61efd is removed from metadata 2024.10.22 02:33:08.879245 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5094bc70b2368f54c4e00dfcf23acacf, offset: 0, size: 0 2024.10.22 02:33:08.883337 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5094bc70b2368f54c4e00dfcf23acacf for removal 2024.10.22 02:33:08.886610 [ 1476 ] {} CacheMetadata: Key 5094bc70b2368f54c4e00dfcf23acacf is removed from metadata 2024.10.22 02:33:09.263617 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} MergeTask::MergeProjectionsStage: Merge sorted 7702 rows, containing 4 columns (4 merged, 0 gathered) in 1.970789442 sec., 3908.0785779854 rows/sec., 61.06 KiB/sec. 2024.10.22 02:33:09.326464 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} MergedBlockOutputStream: filled checksums 202410_947_952_1 (state Temporary) 2024.10.22 02:33:09.472445 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_947_952_1 to 202410_947_952_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:09.476785 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_947_952_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_947_947_0, 202410_952_952_0] -> 202410_947_952_1 2024.10.22 02:33:09.523342 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:33:09.668768 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(50862ad37a4fa17bbb1cf3d747f48997) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 50862ad37a4fa17bbb1cf3d747f48997, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:09.670705 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(50862ad37a4fa17bbb1cf3d747f48997): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:33:09.724788 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_970_70} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_970_70 to 202410_747_970_70 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:09.739189 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_970_70} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_969_69, 202410_970_970_0] -> 202410_747_970_70 2024.10.22 02:33:09.757312 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 58.56 MiB. 2024.10.22 02:33:10.247372 [ 1267 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 02:33:10.575692 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1571_114, 202410_1572_1572_0] 2024.10.22 02:33:10.580553 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 13.68 MiB, will set to 1.57 GiB (RSS), difference: 42.64 MiB 2024.10.22 02:33:10.681302 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1571_114, 202410_1572_1572_0] 2024.10.22 02:33:10.776601 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2bb3aad861173a5085433e4166547725) : [0, 52365]: Setting download as failed: File segment: [0, 52365], key: 2bb3aad861173a5085433e4166547725, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:10.781502 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2bb3aad861173a5085433e4166547725): No space left in cache to reserve 52366 bytes, will continue without cache download 2024.10.22 02:33:10.793136 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1571_114, 202410_1572_1572_0] 2024.10.22 02:33:10.814446 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2bb3aad861173a5085433e4166547725, offset: 0, size: 0 2024.10.22 02:33:10.821300 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2bb3aad861173a5085433e4166547725 for removal 2024.10.22 02:33:10.826831 [ 1476 ] {} CacheMetadata: Key 2bb3aad861173a5085433e4166547725 is removed from metadata 2024.10.22 02:33:10.834322 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 50862ad37a4fa17bbb1cf3d747f48997, offset: 0, size: 0 2024.10.22 02:33:10.837000 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 50862ad37a4fa17bbb1cf3d747f48997 for removal 2024.10.22 02:33:10.839870 [ 1476 ] {} CacheMetadata: Key 50862ad37a4fa17bbb1cf3d747f48997 is removed from metadata 2024.10.22 02:33:11.059321 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 111709 2024.10.22 02:33:11.370601 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:11.374312 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:11.547737 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1663_1663_0 (state Temporary) 2024.10.22 02:33:11.547938 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:33:11.551256 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:33:11.696435 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1663_1663_0 to 202410_1676_1676_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:11.719021 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1671_9 to 202410_1676_1676_0 2024.10.22 02:33:11.733700 [ 1223 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:11.736488 [ 1223 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:11.737168 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111709 2024.10.22 02:33:11.813284 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1671_9 to 202410_1676_1676_0 into Compact with storage Full 2024.10.22 02:33:11.842484 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:11.851551 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1671_9, total 3079 rows starting from the beginning of the part 2024.10.22 02:33:11.874379 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 15244 2024.10.22 02:33:11.879790 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1672_1672_0, total 90 rows starting from the beginning of the part 2024.10.22 02:33:11.928843 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1673_1673_0, total 67 rows starting from the beginning of the part 2024.10.22 02:33:12.040563 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1cca5866cb3e4b1083b8acbebea1a08a) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 1cca5866cb3e4b1083b8acbebea1a08a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:12.044354 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1cca5866cb3e4b1083b8acbebea1a08a): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:33:12.113427 [ 1222 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:33:12.175001 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1674_1674_0, total 72 rows starting from the beginning of the part 2024.10.22 02:33:12.245568 [ 1222 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:33:12.289546 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_880_164, 202410_881_886_1] 2024.10.22 02:33:12.294587 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_880_164, 202410_881_886_1] 2024.10.22 02:33:12.309589 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_880_164, 202410_881_886_1] 2024.10.22 02:33:12.356780 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1675_1675_0, total 77 rows starting from the beginning of the part 2024.10.22 02:33:12.405166 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:12.409536 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:12.460774 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_987_987_0 (state Temporary) 2024.10.22 02:33:12.481590 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 14.73 MiB, will set to 1.56 GiB (RSS), difference: -5.35 MiB 2024.10.22 02:33:12.500706 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} MergeTreeSequentialSource: Reading 2 marks from part 202410_1676_1676_0, total 81 rows starting from the beginning of the part 2024.10.22 02:33:12.798838 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8886cd04e6c17ca60f3c7938089a792e) : [0, 27872]: Setting download as failed: File segment: [0, 27872], key: 8886cd04e6c17ca60f3c7938089a792e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:12.801325 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8886cd04e6c17ca60f3c7938089a792e): No space left in cache to reserve 27873 bytes, will continue without cache download 2024.10.22 02:33:12.819195 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_987_987_0 to 202410_1000_1000_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:12.849274 [ 1219 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_995_360 to 202410_996_996_0 2024.10.22 02:33:12.846283 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8886cd04e6c17ca60f3c7938089a792e, offset: 0, size: 0 2024.10.22 02:33:12.860814 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8886cd04e6c17ca60f3c7938089a792e for removal 2024.10.22 02:33:12.869185 [ 1476 ] {} CacheMetadata: Key 8886cd04e6c17ca60f3c7938089a792e is removed from metadata 2024.10.22 02:33:12.870574 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1cca5866cb3e4b1083b8acbebea1a08a, offset: 0, size: 0 2024.10.22 02:33:12.874612 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1cca5866cb3e4b1083b8acbebea1a08a for removal 2024.10.22 02:33:12.881411 [ 1476 ] {} CacheMetadata: Key 1cca5866cb3e4b1083b8acbebea1a08a is removed from metadata 2024.10.22 02:33:12.882177 [ 1219 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:12.893837 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 187 entries to flush up to offset 109488 2024.10.22 02:33:12.992175 [ 1219 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:13.009307 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15244 2024.10.22 02:33:13.048825 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_996_361} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_995_360 to 202410_996_996_0 into Compact with storage Full 2024.10.22 02:33:13.114871 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_996_361} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:13.135457 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_996_361} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_995_360, total 15270 rows starting from the beginning of the part 2024.10.22 02:33:13.180100 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_996_361} MergeTreeSequentialSource: Reading 2 marks from part 202410_996_996_0, total 12 rows starting from the beginning of the part 2024.10.22 02:33:13.318891 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:13.327846 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:13.388876 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_936_301, 202410_937_937_0] 2024.10.22 02:33:13.394955 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_936_301, 202410_937_937_0] 2024.10.22 02:33:13.406529 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_970_970_0 (state Temporary) 2024.10.22 02:33:13.416276 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_936_301, 202410_937_937_0] 2024.10.22 02:33:13.488844 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_970_970_0 to 202410_976_976_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:13.545077 [ 1159 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_970_70 to 202410_971_971_0 2024.10.22 02:33:13.546720 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 109488 2024.10.22 02:33:13.550096 [ 1159 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.42 MiB using storage policy from min volume index 0 2024.10.22 02:33:13.552213 [ 1159 ] {} DiskLocal: Reserved 1.42 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:13.564438 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_971_71} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_970_70 to 202410_971_971_0 into Wide with storage Full 2024.10.22 02:33:13.715267 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_971_71} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:13.849743 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_971_71} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_970_70, total 25122 rows starting from the beginning of the part 2024.10.22 02:33:14.226002 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1423850 2024.10.22 02:33:14.392009 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} MergeTask::MergeProjectionsStage: Merge sorted 3466 rows, containing 13 columns (13 merged, 0 gathered) in 2.637982436 sec., 1313.8828950110567 rows/sec., 227.17 KiB/sec. 2024.10.22 02:33:14.467253 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_971_71} MergeTreeSequentialSource: Reading 2 marks from part 202410_971_971_0, total 184 rows starting from the beginning of the part 2024.10.22 02:33:14.474086 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} MergedBlockOutputStream: filled checksums 202410_1626_1676_10 (state Temporary) 2024.10.22 02:33:14.543409 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(7bb7335ac86b7144eae2bb924799d702) : [0, 167]: Setting download as failed: File segment: [0, 167], key: 7bb7335ac86b7144eae2bb924799d702, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:14.550986 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(7bb7335ac86b7144eae2bb924799d702): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 02:33:14.592062 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1676_10 to 202410_1626_1676_10 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:14.596995 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1676_10} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1671_9, 202410_1676_1676_0] -> 202410_1626_1676_10 2024.10.22 02:33:14.650407 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:33:14.740970 [ 1237 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 235 ms. 2024.10.22 02:33:14.825413 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 14.06 MiB, will set to 1.57 GiB (RSS), difference: -11.16 MiB 2024.10.22 02:33:14.946113 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_996_361} MergeTask::MergeProjectionsStage: Merge sorted 15282 rows, containing 12 columns (12 merged, 0 gathered) in 1.903256029 sec., 8029.397919747769 rows/sec., 1.26 MiB/sec. 2024.10.22 02:33:15.183780 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 818 2024.10.22 02:33:15.346023 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_996_361} MergedBlockOutputStream: filled checksums 202410_1_996_361 (state Temporary) 2024.10.22 02:33:15.485437 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:15.491881 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:15.570719 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_800_800_0 (state Temporary) 2024.10.22 02:33:15.643077 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_996_361} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_996_361 to 202410_1_996_361 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:15.647331 [ 1390 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_996_361} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_995_360, 202410_996_996_0] -> 202410_1_996_361 2024.10.22 02:33:15.674420 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:33:15.743184 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 83 entries to flush up to offset 111792 2024.10.22 02:33:15.780893 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_800_800_0 to 202410_810_810_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:15.795500 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:15.804075 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:15.895676 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 818 2024.10.22 02:33:15.947054 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4436fcf79d48085e36b40e35adf1ce16) : [0, 105755]: Setting download as failed: File segment: [0, 105755], key: 4436fcf79d48085e36b40e35adf1ce16, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:15.947987 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4436fcf79d48085e36b40e35adf1ce16): No space left in cache to reserve 105756 bytes, will continue without cache download 2024.10.22 02:33:15.972056 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1572_115, 202410_1573_1573_0] 2024.10.22 02:33:15.978893 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1572_115, 202410_1573_1573_0] 2024.10.22 02:33:16.015630 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1572_115, 202410_1573_1573_0] 2024.10.22 02:33:15.971051 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4436fcf79d48085e36b40e35adf1ce16, offset: 0, size: 0 2024.10.22 02:33:16.050529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4436fcf79d48085e36b40e35adf1ce16 for removal 2024.10.22 02:33:16.054614 [ 1476 ] {} CacheMetadata: Key 4436fcf79d48085e36b40e35adf1ce16 is removed from metadata 2024.10.22 02:33:16.056435 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 7bb7335ac86b7144eae2bb924799d702, offset: 0, size: 0 2024.10.22 02:33:16.059871 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 7bb7335ac86b7144eae2bb924799d702 for removal 2024.10.22 02:33:16.066663 [ 1476 ] {} CacheMetadata: Key 7bb7335ac86b7144eae2bb924799d702 is removed from metadata 2024.10.22 02:33:16.229038 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_946_946_0 (state Temporary) 2024.10.22 02:33:16.273322 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:16.277455 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:16.344216 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_946_946_0 to 202410_953_953_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:16.353663 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_971_71} MergeTask::MergeProjectionsStage: Merge sorted 25306 rows, containing 13 columns (13 merged, 0 gathered) in 2.79425896 sec., 9056.426180342283 rows/sec., 3.95 MiB/sec. 2024.10.22 02:33:16.357002 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1664_1664_0 (state Temporary) 2024.10.22 02:33:16.413055 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1664_1664_0 to 202410_1677_1677_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:16.580620 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1423850 2024.10.22 02:33:16.652863 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_971_71} MergedBlockOutputStream: filled checksums 202410_747_971_71 (state Temporary) 2024.10.22 02:33:16.772496 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111792 2024.10.22 02:33:16.804852 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(6081af361c21db7c99a372939bb28c27) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 6081af361c21db7c99a372939bb28c27, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:33:16.809570 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 141, file segment info: File segment: [0, 33554431], key: 6081af361c21db7c99a372939bb28c27, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:33:16.817425 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:17.509817 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_971_71} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_971_71 to 202410_747_971_71 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:17.519597 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_971_71} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_970_70, 202410_971_971_0] -> 202410_747_971_71 2024.10.22 02:33:17.587825 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_946_175/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:33:17.648527 [ 1390 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 58.68 MiB. 2024.10.22 02:33:18.304643 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(2f3f05add11fd722adf46381a8f1c6a2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2f3f05add11fd722adf46381a8f1c6a2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 02:33:18.309122 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 28948, file segment info: File segment: [0, 33554431], key: 2f3f05add11fd722adf46381a8f1c6a2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 02:33:18.314666 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:18.499950 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 13.36 MiB, will set to 1.56 GiB (RSS), difference: 10.78 MiB 2024.10.22 02:33:18.565520 [ 1189 ] {} DatabaseCatalog: Execution took 265 ms. 2024.10.22 02:33:19.589633 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d09d1a6cd437ab49ad6353a4c048fb61) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d09d1a6cd437ab49ad6353a4c048fb61, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:33:19.593188 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 152, file segment info: File segment: [0, 33554431], key: d09d1a6cd437ab49ad6353a4c048fb61, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:33:19.595986 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:20.456345 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(01bdad8f7cc303ecb9751ad62f54c5de) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 01bdad8f7cc303ecb9751ad62f54c5de, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 02:33:20.458433 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 564, file segment info: File segment: [0, 33554431], key: 01bdad8f7cc303ecb9751ad62f54c5de, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 02:33:20.461271 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:20.512633 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 15262 2024.10.22 02:33:20.548711 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:33:20.556590 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:33:20.562813 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:33:20.571899 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:33:20.602099 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:33:20.614264 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:33:20.771116 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:33:20.779811 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:33:20.791805 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:20.801078 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 52 entries to flush up to offset 111844 2024.10.22 02:33:20.803297 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:20.898426 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IsMobile 2024.10.22 02:33:20.907771 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_988_988_0 (state Temporary) 2024.10.22 02:33:20.946983 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1573_116, 202410_1574_1574_0] 2024.10.22 02:33:20.948207 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1573_116, 202410_1574_1574_0] 2024.10.22 02:33:20.958574 [ 1420 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1573_116, 202410_1574_1574_0] 2024.10.22 02:33:20.961566 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_988_988_0 to 202410_1001_1001_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:20.970334 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/pzl/tytlcvpbzgsbszxzipktfctsmorpx (LogSeriesLimiter: on interval from 2024-10-22 02:32:54 to 2024-10-22 02:33:20 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:33:21.074970 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 191 entries to flush up to offset 109679 2024.10.22 02:33:21.088240 [ 1258 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_996_361 to 202410_997_997_0 2024.10.22 02:33:21.096923 [ 1258 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:21.108765 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15262 2024.10.22 02:33:21.158527 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(de361ab8160e37cc729528b63c9b2581) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: de361ab8160e37cc729528b63c9b2581, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:33:21.163018 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 359, file segment info: File segment: [0, 33554431], key: de361ab8160e37cc729528b63c9b2581, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:33:21.164488 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:21.207526 [ 1258 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:21.250656 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_997_362} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_996_361 to 202410_997_997_0 into Compact with storage Full 2024.10.22 02:33:21.285095 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_997_362} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:21.343991 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:21.395193 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:21.397372 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_997_362} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_996_361, total 15282 rows starting from the beginning of the part 2024.10.22 02:33:21.431527 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_997_362} MergeTreeSequentialSource: Reading 2 marks from part 202410_997_997_0, total 15 rows starting from the beginning of the part 2024.10.22 02:33:21.445761 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:21.448389 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:21.511831 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1665_1665_0 (state Temporary) 2024.10.22 02:33:21.548043 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:33:21.550342 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:33:21.650033 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_971_971_0 (state Temporary) 2024.10.22 02:33:21.668632 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1665_1665_0 to 202410_1678_1678_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:21.704315 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_971_971_0 to 202410_977_977_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:21.749071 [ 1201 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_971_71 to 202410_972_972_0 2024.10.22 02:33:21.751778 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 17.49 MiB, will set to 1.57 GiB (RSS), difference: -2.66 MiB 2024.10.22 02:33:21.751937 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111844 2024.10.22 02:33:21.755529 [ 1201 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.43 MiB using storage policy from min volume index 0 2024.10.22 02:33:21.757176 [ 1201 ] {} DiskLocal: Reserved 1.43 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:21.768426 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_972_72} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_971_71 to 202410_972_972_0 into Wide with storage Full 2024.10.22 02:33:21.769247 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 109679 2024.10.22 02:33:21.865167 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_937_302, 202410_938_938_0] 2024.10.22 02:33:21.866846 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_937_302, 202410_938_938_0] 2024.10.22 02:33:21.976515 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_937_302, 202410_938_938_0] 2024.10.22 02:33:21.977038 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 2 old parts to remove. Parts: [202410_747_914_34, 202410_915_915_0] 2024.10.22 02:33:21.986816 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from filesystem (serially): Parts: [202410_747_914_34, 202410_915_915_0] 2024.10.22 02:33:22.016229 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 2 parts from memory: Parts: [202410_747_914_34, 202410_915_915_0] 2024.10.22 02:33:22.032816 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_972_72} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:22.096707 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_972_72} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_971_71, total 25306 rows starting from the beginning of the part 2024.10.22 02:33:22.293265 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_972_72} MergeTreeSequentialSource: Reading 2 marks from part 202410_972_972_0, total 178 rows starting from the beginning of the part 2024.10.22 02:33:22.324882 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_997_362} MergeTask::MergeProjectionsStage: Merge sorted 15297 rows, containing 12 columns (12 merged, 0 gathered) in 1.082105979 sec., 14136.32333326198 rows/sec., 2.21 MiB/sec. 2024.10.22 02:33:22.365519 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_997_362} MergedBlockOutputStream: filled checksums 202410_1_997_362 (state Temporary) 2024.10.22 02:33:22.440209 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_997_362} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_997_362 to 202410_1_997_362 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:22.444866 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_997_362} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_996_361, 202410_997_997_0] -> 202410_1_997_362 2024.10.22 02:33:22.461657 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:33:22.469889 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(740b45716eff23744b719c8bd249d54b) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 740b45716eff23744b719c8bd249d54b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:22.472421 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(740b45716eff23744b719c8bd249d54b): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:33:23.264784 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:23.266903 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:23.283130 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(f78c290b1a174302b802cabf8127ca8c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f78c290b1a174302b802cabf8127ca8c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:33:23.289213 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 464, file segment info: File segment: [0, 33554431], key: f78c290b1a174302b802cabf8127ca8c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1393, kind: Regular, unbound: 0 2024.10.22 02:33:23.291785 [ 1393 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:23.350747 [ 1696 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/fph/ovxcdcmhkpihzsicxbkmhqcxnmdvw, size 359 2024.10.22 02:33:23.398618 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 819 2024.10.22 02:33:23.587894 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 943 entries to flush up to offset 1424793 2024.10.22 02:33:23.668533 [ 1422 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_199_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:33:23.670325 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:23.676227 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:23.796529 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:33:23.837700 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:33:23.859777 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:33:23.899308 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:33:23.944123 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:33:24.016373 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:33:24.047130 [ 1398 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ShareURL 2024.10.22 02:33:24.156465 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_801_801_0 (state Temporary) 2024.10.22 02:33:24.248218 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_801_801_0 to 202410_811_811_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:24.312126 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(777f8785f55547c4f7915b97f2377a01) : [0, 2431]: Setting download as failed: File segment: [0, 2431], key: 777f8785f55547c4f7915b97f2377a01, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:24.314341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(777f8785f55547c4f7915b97f2377a01): No space left in cache to reserve 2432 bytes, will continue without cache download 2024.10.22 02:33:24.316333 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1574_117, 202410_1575_1575_0] 2024.10.22 02:33:24.321420 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1574_117, 202410_1575_1575_0] 2024.10.22 02:33:24.345400 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1574_117, 202410_1575_1575_0] 2024.10.22 02:33:24.345074 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 777f8785f55547c4f7915b97f2377a01, offset: 0, size: 0 2024.10.22 02:33:24.355540 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 819 2024.10.22 02:33:24.355612 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 777f8785f55547c4f7915b97f2377a01 for removal 2024.10.22 02:33:24.357713 [ 1476 ] {} CacheMetadata: Key 777f8785f55547c4f7915b97f2377a01 is removed from metadata 2024.10.22 02:33:24.358249 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 740b45716eff23744b719c8bd249d54b, offset: 0, size: 0 2024.10.22 02:33:24.360368 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 740b45716eff23744b719c8bd249d54b for removal 2024.10.22 02:33:24.363084 [ 1476 ] {} CacheMetadata: Key 740b45716eff23744b719c8bd249d54b is removed from metadata 2024.10.22 02:33:24.363666 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:24.366966 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:24.704203 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_947_947_0 (state Temporary) 2024.10.22 02:33:24.884739 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_947_947_0 to 202410_954_954_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:24.922670 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1424793 2024.10.22 02:33:24.966136 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_972_72} MergeTask::MergeProjectionsStage: Merge sorted 25484 rows, containing 13 columns (13 merged, 0 gathered) in 3.201588665 sec., 7959.798295950051 rows/sec., 3.47 MiB/sec. 2024.10.22 02:33:25.048435 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_972_72} MergedBlockOutputStream: filled checksums 202410_747_972_72 (state Temporary) 2024.10.22 02:33:25.366985 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ef468bbbc1bdba380e79d9b196b92e2b) : [0, 155]: Setting download as failed: File segment: [0, 155], key: ef468bbbc1bdba380e79d9b196b92e2b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:25.368012 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ef468bbbc1bdba380e79d9b196b92e2b): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:33:25.707977 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 17.95 MiB, will set to 1.57 GiB (RSS), difference: -2.47 MiB 2024.10.22 02:33:25.752466 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_972_72} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_972_72 to 202410_747_972_72 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:25.756381 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_972_72} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_971_71, 202410_972_972_0] -> 202410_747_972_72 2024.10.22 02:33:25.757225 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_198_198_0 (state Temporary) 2024.10.22 02:33:25.757279 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 99 entries to flush up to offset 111943 2024.10.22 02:33:25.774797 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 58.69 MiB. 2024.10.22 02:33:26.053462 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:26.058867 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:26.255051 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1666_1666_0 (state Temporary) 2024.10.22 02:33:26.314149 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1666_1666_0 to 202410_1679_1679_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:26.466445 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(80b0c48f90939d9194cfdb10b1f4599f) : [0, 2606]: Setting download as failed: File segment: [0, 2606], key: 80b0c48f90939d9194cfdb10b1f4599f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:26.470210 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(80b0c48f90939d9194cfdb10b1f4599f): No space left in cache to reserve 2607 bytes, will continue without cache download 2024.10.22 02:33:26.506426 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 111943 2024.10.22 02:33:26.596561 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 80b0c48f90939d9194cfdb10b1f4599f, offset: 0, size: 0 2024.10.22 02:33:26.600245 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 80b0c48f90939d9194cfdb10b1f4599f for removal 2024.10.22 02:33:26.605198 [ 1476 ] {} CacheMetadata: Key 80b0c48f90939d9194cfdb10b1f4599f is removed from metadata 2024.10.22 02:33:26.611060 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ef468bbbc1bdba380e79d9b196b92e2b, offset: 0, size: 0 2024.10.22 02:33:26.612359 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ef468bbbc1bdba380e79d9b196b92e2b for removal 2024.10.22 02:33:26.619648 [ 1476 ] {} CacheMetadata: Key ef468bbbc1bdba380e79d9b196b92e2b is removed from metadata 2024.10.22 02:33:27.114681 [ 1225 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 237 ms. 2024.10.22 02:33:27.163819 [ 1250 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 226 ms. 2024.10.22 02:33:27.217070 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_758_154, 202410_759_759_0, 202410_760_760_0, 202410_761_761_0, 202410_762_762_0, 202410_763_763_0] 2024.10.22 02:33:27.249319 [ 1216 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:33:27.225658 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_758_154, 202410_759_759_0, 202410_760_760_0, 202410_761_761_0, 202410_762_762_0, 202410_763_763_0] 2024.10.22 02:33:27.276605 [ 1420 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_758_154, 202410_759_759_0, 202410_760_760_0, 202410_761_761_0, 202410_762_762_0, 202410_763_763_0] 2024.10.22 02:33:27.280677 [ 1216 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:33:27.656511 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9bf91b5bb65783ef8daca1d2afdf024b) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 9bf91b5bb65783ef8daca1d2afdf024b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:27.658621 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9bf91b5bb65783ef8daca1d2afdf024b): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:33:28.386528 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(54ff7005555303c41ca701947802be27) : [0, 34090]: Setting download as failed: File segment: [0, 34090], key: 54ff7005555303c41ca701947802be27, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:28.388704 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(54ff7005555303c41ca701947802be27): No space left in cache to reserve 34091 bytes, will continue without cache download 2024.10.22 02:33:28.587232 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 18.08 MiB, will set to 1.57 GiB (RSS), difference: -1.33 MiB 2024.10.22 02:33:28.611411 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 15274 2024.10.22 02:33:28.398842 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 54ff7005555303c41ca701947802be27, offset: 0, size: 0 2024.10.22 02:33:28.628785 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 54ff7005555303c41ca701947802be27 for removal 2024.10.22 02:33:28.644412 [ 1476 ] {} CacheMetadata: Key 54ff7005555303c41ca701947802be27 is removed from metadata 2024.10.22 02:33:28.749228 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9bf91b5bb65783ef8daca1d2afdf024b, offset: 0, size: 0 2024.10.22 02:33:28.751226 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9bf91b5bb65783ef8daca1d2afdf024b for removal 2024.10.22 02:33:28.753412 [ 1476 ] {} CacheMetadata: Key 9bf91b5bb65783ef8daca1d2afdf024b is removed from metadata 2024.10.22 02:33:28.802185 [ 1260 ] {} DatabaseCatalog: Execution took 236 ms. 2024.10.22 02:33:28.924790 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:28.931068 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:29.015640 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_946_175/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:33:29.265721 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_989_989_0 (state Temporary) 2024.10.22 02:33:29.435444 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_989_989_0 to 202410_1002_1002_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:29.439431 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 178 entries to flush up to offset 109857 2024.10.22 02:33:29.458641 [ 1257 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_997_362 to 202410_998_998_0 2024.10.22 02:33:29.466935 [ 1257 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:29.470279 [ 1257 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:29.483693 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15274 2024.10.22 02:33:29.518266 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_998_363} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_997_362 to 202410_998_998_0 into Compact with storage Full 2024.10.22 02:33:29.651327 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_998_363} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:29.659133 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_998_363} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_997_362, total 15297 rows starting from the beginning of the part 2024.10.22 02:33:29.693285 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_998_363} MergeTreeSequentialSource: Reading 2 marks from part 202410_998_998_0, total 18 rows starting from the beginning of the part 2024.10.22 02:33:29.829158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(365fd5246da56739b2b1438eedb85c38) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 365fd5246da56739b2b1438eedb85c38, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:29.830779 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(365fd5246da56739b2b1438eedb85c38): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:33:29.964930 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:29.966470 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:30.037069 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_972_972_0 (state Temporary) 2024.10.22 02:33:30.138484 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_972_972_0 to 202410_978_978_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:30.161616 [ 1263 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_972_72 to 202410_973_973_0 2024.10.22 02:33:30.168148 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 109857 2024.10.22 02:33:30.269485 [ 1263 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.43 MiB using storage policy from min volume index 0 2024.10.22 02:33:30.282552 [ 1263 ] {} DiskLocal: Reserved 1.43 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:30.334753 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_973_73} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_972_72 to 202410_973_973_0 into Wide with storage Full 2024.10.22 02:33:30.460398 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 19.54 MiB, will set to 1.57 GiB (RSS), difference: 2.27 MiB 2024.10.22 02:33:30.513760 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_938_303, 202410_939_939_0] 2024.10.22 02:33:30.516294 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_998_363} MergeTask::MergeProjectionsStage: Merge sorted 15315 rows, containing 12 columns (12 merged, 0 gathered) in 1.03360258 sec., 14817.1069774226 rows/sec., 2.32 MiB/sec. 2024.10.22 02:33:30.519572 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 112006 2024.10.22 02:33:30.534309 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_973_73} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:30.550447 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_998_363} MergedBlockOutputStream: filled checksums 202410_1_998_363 (state Temporary) 2024.10.22 02:33:30.623585 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_938_303, 202410_939_939_0] 2024.10.22 02:33:30.636367 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_938_303, 202410_939_939_0] 2024.10.22 02:33:30.663383 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_973_73} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_972_72, total 25484 rows starting from the beginning of the part 2024.10.22 02:33:30.675270 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9e1645a65db4e75cce3086ef19b89286) : [0, 32211]: Setting download as failed: File segment: [0, 32211], key: 9e1645a65db4e75cce3086ef19b89286, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:30.677659 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9e1645a65db4e75cce3086ef19b89286): No space left in cache to reserve 32212 bytes, will continue without cache download 2024.10.22 02:33:30.931321 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_973_73} MergeTreeSequentialSource: Reading 2 marks from part 202410_973_973_0, total 176 rows starting from the beginning of the part 2024.10.22 02:33:30.718144 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9e1645a65db4e75cce3086ef19b89286, offset: 0, size: 0 2024.10.22 02:33:30.943297 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9e1645a65db4e75cce3086ef19b89286 for removal 2024.10.22 02:33:30.945167 [ 1476 ] {} CacheMetadata: Key 9e1645a65db4e75cce3086ef19b89286 is removed from metadata 2024.10.22 02:33:30.945611 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 365fd5246da56739b2b1438eedb85c38, offset: 0, size: 0 2024.10.22 02:33:30.946780 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 365fd5246da56739b2b1438eedb85c38 for removal 2024.10.22 02:33:30.948667 [ 1476 ] {} CacheMetadata: Key 365fd5246da56739b2b1438eedb85c38 is removed from metadata 2024.10.22 02:33:30.998690 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:31.003331 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:31.049113 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1667_1667_0 (state Temporary) 2024.10.22 02:33:31.087780 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_998_363} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_998_363 to 202410_1_998_363 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:31.109367 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_998_363} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_997_362, 202410_998_998_0] -> 202410_1_998_363 2024.10.22 02:33:31.153739 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:33:31.199628 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1667_1667_0 to 202410_1680_1680_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:31.348822 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112006 2024.10.22 02:33:31.547724 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:33:31.549093 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:33:31.662222 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(340d9f6d90399960e2ecf97df10b9ad0) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 340d9f6d90399960e2ecf97df10b9ad0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:31.663798 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(340d9f6d90399960e2ecf97df10b9ad0): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 02:33:31.857040 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 820 2024.10.22 02:33:32.034322 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1367 entries to flush up to offset 1426160 2024.10.22 02:33:32.036598 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:32.042248 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:32.113682 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_802_802_0 (state Temporary) 2024.10.22 02:33:32.287066 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_802_802_0 to 202410_812_812_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:32.415574 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 820 2024.10.22 02:33:32.423147 [ 1424 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_973_73/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:33:32.455372 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 17.77 MiB, will set to 1.58 GiB (RSS), difference: -40.82 MiB 2024.10.22 02:33:32.510427 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 6 old parts to remove. Parts: [202410_747_915_35, 202410_916_916_0, 202410_917_917_0, 202410_918_918_0, 202410_919_919_0, 202410_920_920_0] 2024.10.22 02:33:32.513255 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from filesystem (serially): Parts: [202410_747_915_35, 202410_916_916_0, 202410_917_917_0, 202410_918_918_0, 202410_919_919_0, 202410_920_920_0] 2024.10.22 02:33:32.559002 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1575_118, 202410_1576_1576_0, 202410_1577_1577_0, 202410_1578_1578_0, 202410_1579_1579_0, 202410_1580_1580_0] 2024.10.22 02:33:32.560601 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1575_118, 202410_1576_1576_0, 202410_1577_1577_0, 202410_1578_1578_0, 202410_1579_1579_0, 202410_1580_1580_0] 2024.10.22 02:33:32.571129 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 6 parts from memory: Parts: [202410_747_915_35, 202410_916_916_0, 202410_917_917_0, 202410_918_918_0, 202410_919_919_0, 202410_920_920_0] 2024.10.22 02:33:32.654432 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(063c16737d5fb87db3774079bf9334d9) : [0, 177781]: Setting download as failed: File segment: [0, 177781], key: 063c16737d5fb87db3774079bf9334d9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:32.655891 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(063c16737d5fb87db3774079bf9334d9): No space left in cache to reserve 177782 bytes, will continue without cache download 2024.10.22 02:33:32.687654 [ 1421 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1575_118, 202410_1576_1576_0, 202410_1577_1577_0, 202410_1578_1578_0, 202410_1579_1579_0, 202410_1580_1580_0] 2024.10.22 02:33:32.805714 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 063c16737d5fb87db3774079bf9334d9, offset: 0, size: 0 2024.10.22 02:33:32.808998 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 063c16737d5fb87db3774079bf9334d9 for removal 2024.10.22 02:33:32.924042 [ 1476 ] {} CacheMetadata: Key 063c16737d5fb87db3774079bf9334d9 is removed from metadata 2024.10.22 02:33:32.926448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 340d9f6d90399960e2ecf97df10b9ad0, offset: 0, size: 0 2024.10.22 02:33:32.928364 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 340d9f6d90399960e2ecf97df10b9ad0 for removal 2024.10.22 02:33:33.034371 [ 1476 ] {} CacheMetadata: Key 340d9f6d90399960e2ecf97df10b9ad0 is removed from metadata 2024.10.22 02:33:33.096836 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_946_175} MergeTask::MergeProjectionsStage: Merge sorted 1428198 rows, containing 4 columns (4 merged, 0 gathered) in 42.429501745 sec., 33660.49426136149 rows/sec., 504.80 KiB/sec. 2024.10.22 02:33:33.099629 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:33.103863 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.17 GiB. 2024.10.22 02:33:33.248457 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_946_175} MergedBlockOutputStream: filled checksums 202410_1_946_175 (state Temporary) 2024.10.22 02:33:33.347764 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_198_198_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:33:33.356167 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_973_73} MergeTask::MergeProjectionsStage: Merge sorted 25660 rows, containing 13 columns (13 merged, 0 gathered) in 3.045564504 sec., 8425.36743723488 rows/sec., 3.67 MiB/sec. 2024.10.22 02:33:33.364159 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_948_948_0 (state Temporary) 2024.10.22 02:33:33.511194 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_948_948_0 to 202410_955_955_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:33.548369 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_973_73} MergedBlockOutputStream: filled checksums 202410_747_973_73 (state Temporary) 2024.10.22 02:33:33.555244 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1426160 2024.10.22 02:33:33.718419 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_198_198_0 to 202410_201_201_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:33.772879 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6a5ea1ff08595597a6bf14bb902b68d0) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 6a5ea1ff08595597a6bf14bb902b68d0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:33.776262 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6a5ea1ff08595597a6bf14bb902b68d0): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 02:33:34.724238 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_973_73} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_973_73 to 202410_747_973_73 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:34.730150 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_973_73} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_972_72, 202410_973_973_0] -> 202410_747_973_73 2024.10.22 02:33:34.760185 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 58.94 MiB. 2024.10.22 02:33:34.819702 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 10.98 MiB, will set to 1.57 GiB (RSS), difference: 55.36 MiB 2024.10.22 02:33:35.014720 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a694eb57dfcd76c99871421a1119541b) : [0, 143122]: Setting download as failed: File segment: [0, 143122], key: a694eb57dfcd76c99871421a1119541b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:35.018862 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a694eb57dfcd76c99871421a1119541b): No space left in cache to reserve 143123 bytes, will continue without cache download 2024.10.22 02:33:35.145550 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a694eb57dfcd76c99871421a1119541b, offset: 0, size: 0 2024.10.22 02:33:35.161077 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a694eb57dfcd76c99871421a1119541b for removal 2024.10.22 02:33:35.163889 [ 1476 ] {} CacheMetadata: Key a694eb57dfcd76c99871421a1119541b is removed from metadata 2024.10.22 02:33:35.165104 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6a5ea1ff08595597a6bf14bb902b68d0, offset: 0, size: 0 2024.10.22 02:33:35.166561 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6a5ea1ff08595597a6bf14bb902b68d0 for removal 2024.10.22 02:33:35.171137 [ 1476 ] {} CacheMetadata: Key 6a5ea1ff08595597a6bf14bb902b68d0 is removed from metadata 2024.10.22 02:33:35.397472 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 77 entries to flush up to offset 112083 2024.10.22 02:33:35.804440 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_893_893_0, 202410_894_894_0, 202410_895_895_0, 202410_896_896_0, 202410_897_897_0, 202410_898_898_0] 2024.10.22 02:33:35.828798 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_893_893_0, 202410_894_894_0, 202410_895_895_0, 202410_896_896_0, 202410_897_897_0, 202410_898_898_0] 2024.10.22 02:33:35.866117 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_893_893_0, 202410_894_894_0, 202410_895_895_0, 202410_896_896_0, 202410_897_897_0, 202410_898_898_0] 2024.10.22 02:33:36.147437 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:36.150375 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:36.222605 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1668_1668_0 (state Temporary) 2024.10.22 02:33:36.402382 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e4ccffafc1a3d3cb60f97c6db94052cf) : [0, 156]: Setting download as failed: File segment: [0, 156], key: e4ccffafc1a3d3cb60f97c6db94052cf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:36.405525 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e4ccffafc1a3d3cb60f97c6db94052cf): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 02:33:36.470131 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1668_1668_0 to 202410_1681_1681_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:36.523109 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1676_10 to 202410_1681_1681_0 2024.10.22 02:33:36.544065 [ 1165 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:36.546954 [ 1165 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:36.568360 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112083 2024.10.22 02:33:36.571917 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 9.79 MiB, will set to 1.56 GiB (RSS), difference: -3.81 MiB 2024.10.22 02:33:36.586885 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1676_10 to 202410_1681_1681_0 into Compact with storage Full 2024.10.22 02:33:36.640267 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:36.658424 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1676_10, total 3466 rows starting from the beginning of the part 2024.10.22 02:33:36.727647 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1677_1677_0, total 83 rows starting from the beginning of the part 2024.10.22 02:33:36.767302 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1678_1678_0, total 52 rows starting from the beginning of the part 2024.10.22 02:33:36.827576 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1679_1679_0, total 99 rows starting from the beginning of the part 2024.10.22 02:33:37.060592 [ 1395 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} MergeTask::MergeProjectionsStage: Merge sorted 613 rows, containing 679 columns (679 merged, 0 gathered) in 85.034914603 sec., 7.208803617454019 rows/sec., 10.37 KiB/sec. 2024.10.22 02:33:37.061362 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1680_1680_0, total 63 rows starting from the beginning of the part 2024.10.22 02:33:37.095904 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 13 entries to flush up to offset 15287 2024.10.22 02:33:37.143790 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} MergeTreeSequentialSource: Reading 2 marks from part 202410_1681_1681_0, total 77 rows starting from the beginning of the part 2024.10.22 02:33:37.182403 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 8002 2024.10.22 02:33:37.509675 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_946_175} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_946_175 to 202410_1_946_175 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:37.515286 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_946_175} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_940_174, 202410_941_946_1] -> 202410_1_946_175 2024.10.22 02:33:37.645598 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:37.648784 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:37.678346 [ 1386 ] {} MergeTreeBackgroundExecutor: Exception while executing background task {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_946_175}: Code: 241. DB::Exception: Memory tracker (for thread): fault injected (at specific point). (MEMORY_LIMIT_EXCEEDED), Stack trace (when copying this message, always include the lines below): 0. /build/contrib/llvm-project/libcxx/include/exception:134: std::exception::capture() @ 0x000000000b1a6742 in /usr/bin/clickhouse 1. /build/contrib/llvm-project/libcxx/include/exception:112: std::exception::exception[abi:v15000]() @ 0x000000000b1a670d in /usr/bin/clickhouse 2. /build/base/poco/Foundation/src/Exception.cpp:27: Poco::Exception::Exception(String const&, int) @ 0x000000002495b180 in /usr/bin/clickhouse 3. /build/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x00000000138e1a6e in /usr/bin/clickhouse 4. /build/src/Common/Exception.h:73: DB::Exception::Exception(String&&, int, bool) @ 0x000000000b19878a in /usr/bin/clickhouse 5. /build/src/Common/Exception.h:101: DB::Exception::Exception(int, FormatStringHelperImpl::type, std::type_identity::type>, char const*&&, char const*&&) @ 0x000000001390092c in /usr/bin/clickhouse 6. /build/src/Common/MemoryTracker.cpp:184: MemoryTracker::injectFault() const @ 0x00000000138fe061 in /usr/bin/clickhouse 7. /build/src/Common/CurrentMemoryTracker.cpp:129: CurrentMemoryTracker::injectFault() @ 0x00000000138822e5 in /usr/bin/clickhouse 8. /build/src/Common/ThreadFuzzer.cpp:242: DB::ThreadFuzzer::maybeInjectMemoryLimitException() @ 0x000000001394e9a0 in /usr/bin/clickhouse 9. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:150: DB::MergePlainMergeTreeTask::finish() @ 0x000000001e3b0918 in /usr/bin/clickhouse 10. /build/src/Storages/MergeTree/MergePlainMergeTreeTask.cpp:72: DB::MergePlainMergeTreeTask::executeStep() @ 0x000000001e3b00b6 in /usr/bin/clickhouse 11. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:273: DB::MergeTreeBackgroundExecutor::routine(std::shared_ptr) @ 0x000000001de56fed in /usr/bin/clickhouse 12. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:338: DB::MergeTreeBackgroundExecutor::threadFunction() @ 0x000000001de57a4e in /usr/bin/clickhouse 13. /build/src/Storages/MergeTree/MergeTreeBackgroundExecutor.cpp:56: DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()::operator()() const @ 0x000000001de5ff38 in /usr/bin/clickhouse 14. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>()()) std::__invoke[abi:v15000]::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5ff15 in /usr/bin/clickhouse 15. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&>(DB::MergeTreeBackgroundExecutor::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'()&) @ 0x000000001de5fef5 in /usr/bin/clickhouse 16. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x000000001de5fed5 in /usr/bin/clickhouse 17. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::MergeTreeBackgroundExecutor(String, unsigned long, unsigned long, StrongTypedef, StrongTypedef, std::basic_string_view>)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x000000001de5fe9d in /usr/bin/clickhouse 18. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 19. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse 20. /build/src/Common/ThreadPool.cpp:426: ThreadPoolImpl>::worker(std::__list_iterator, void*>) @ 0x0000000013a5ff63 in /usr/bin/clickhouse 21. /build/src/Common/ThreadPool.cpp:179: void ThreadPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()::operator()() const @ 0x0000000013a6a7c4 in /usr/bin/clickhouse 22. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&>(void&&) @ 0x0000000013a6a795 in /usr/bin/clickhouse 23. /build/contrib/llvm-project/libcxx/include/tuple:1789: decltype(auto) std::__apply_tuple_impl[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&, std::__tuple_indices<>) @ 0x0000000013a6a779 in /usr/bin/clickhouse 24. /build/contrib/llvm-project/libcxx/include/tuple:1798: decltype(auto) std::apply[abi:v15000]>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()&, std::tuple<>&>(void&&, std::tuple<>&) @ 0x0000000013a6a6dd in /usr/bin/clickhouse 25. /build/src/Common/ThreadPool.h:242: ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()::operator()() @ 0x0000000013a6a5e5 in /usr/bin/clickhouse 26. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:394: decltype(std::declval()()) std::__invoke[abi:v15000]::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(void&&) @ 0x0000000013a6a555 in /usr/bin/clickhouse 27. /build/contrib/llvm-project/libcxx/include/__functional/invoke.h:480: void std::__invoke_void_return_wrapper::__call::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&>(ThreadFromGlobalPoolImpl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'()&) @ 0x0000000013a6a535 in /usr/bin/clickhouse 28. /build/contrib/llvm-project/libcxx/include/__functional/function.h:235: std::__function::__default_alloc_func::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>::operator()[abi:v15000]() @ 0x0000000013a6a515 in /usr/bin/clickhouse 29. /build/contrib/llvm-project/libcxx/include/__functional/function.h:716: void std::__function::__policy_invoker::__call_impl::ThreadFromGlobalPoolImpl>::scheduleImpl(std::function, Priority, std::optional, bool)::'lambda0'()>(void&&)::'lambda'(), void ()>>(std::__function::__policy_storage const*) @ 0x0000000013a6a4e0 in /usr/bin/clickhouse 30. /build/contrib/llvm-project/libcxx/include/__functional/function.h:848: std::__function::__policy_func::operator()[abi:v15000]() const @ 0x0000000013953756 in /usr/bin/clickhouse 31. /build/contrib/llvm-project/libcxx/include/__functional/function.h:1187: std::function::operator()() const @ 0x0000000013952c35 in /usr/bin/clickhouse (version 23.8.16.42.altinitystable (altinity build)) 2024.10.22 02:33:37.751836 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:33:37.788446 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_990_990_0 (state Temporary) 2024.10.22 02:33:37.843871 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(06d91f4b310e6a27d1a83f583c66729f) : [0, 111150]: Setting download as failed: File segment: [0, 111150], key: 06d91f4b310e6a27d1a83f583c66729f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:37.845908 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(06d91f4b310e6a27d1a83f583c66729f): No space left in cache to reserve 111151 bytes, will continue without cache download 2024.10.22 02:33:37.848252 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_990_990_0 to 202410_1003_1003_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:37.986708 [ 1156 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_998_363 to 202410_999_999_0 2024.10.22 02:33:37.970111 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 06d91f4b310e6a27d1a83f583c66729f, offset: 0, size: 0 2024.10.22 02:33:37.991930 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 06d91f4b310e6a27d1a83f583c66729f for removal 2024.10.22 02:33:37.996704 [ 1156 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:37.999697 [ 1476 ] {} CacheMetadata: Key 06d91f4b310e6a27d1a83f583c66729f is removed from metadata 2024.10.22 02:33:38.000468 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e4ccffafc1a3d3cb60f97c6db94052cf, offset: 0, size: 0 2024.10.22 02:33:38.001545 [ 1156 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:38.015274 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e4ccffafc1a3d3cb60f97c6db94052cf for removal 2024.10.22 02:33:38.024020 [ 1476 ] {} CacheMetadata: Key e4ccffafc1a3d3cb60f97c6db94052cf is removed from metadata 2024.10.22 02:33:38.059203 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_999_364} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_998_363 to 202410_999_999_0 into Compact with storage Full 2024.10.22 02:33:38.072048 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 177 entries to flush up to offset 110034 2024.10.22 02:33:38.112767 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15287 2024.10.22 02:33:38.120710 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_999_364} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:38.252658 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_999_364} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_998_363, total 15315 rows starting from the beginning of the part 2024.10.22 02:33:38.417469 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_999_364} MergeTreeSequentialSource: Reading 2 marks from part 202410_999_999_0, total 16 rows starting from the beginning of the part 2024.10.22 02:33:38.715476 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:38.724098 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:38.885548 [ 1421 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:33:38.946120 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_973_973_0 (state Temporary) 2024.10.22 02:33:38.968963 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(2ffb3d8ba7fbde1d02a63d616fef08c2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2ffb3d8ba7fbde1d02a63d616fef08c2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:33:39.012076 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 313303, file segment info: File segment: [0, 33554431], key: 2ffb3d8ba7fbde1d02a63d616fef08c2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:33:39.019230 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:39.042257 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_973_973_0 to 202410_979_979_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:39.099768 [ 1196 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_973_73 to 202410_974_974_0 2024.10.22 02:33:39.111580 [ 1196 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.44 MiB using storage policy from min volume index 0 2024.10.22 02:33:39.118901 [ 1196 ] {} DiskLocal: Reserved 1.44 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:39.123679 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 110034 2024.10.22 02:33:39.158605 [ 1197 ] {} DatabaseCatalog: Execution took 349 ms. 2024.10.22 02:33:39.157225 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_974_74} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_973_73 to 202410_974_974_0 into Wide with storage Full 2024.10.22 02:33:39.252978 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_999_364} MergeTask::MergeProjectionsStage: Merge sorted 15331 rows, containing 12 columns (12 merged, 0 gathered) in 1.202011705 sec., 12754.451505112422 rows/sec., 2.00 MiB/sec. 2024.10.22 02:33:39.327715 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_974_74} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:39.327769 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_999_364} MergedBlockOutputStream: filled checksums 202410_1_999_364 (state Temporary) 2024.10.22 02:33:39.360663 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_974_74} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_973_73, total 25660 rows starting from the beginning of the part 2024.10.22 02:33:39.420140 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_939_304, 202410_940_940_0] 2024.10.22 02:33:39.464761 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_939_304, 202410_940_940_0] 2024.10.22 02:33:39.473103 [ 1419 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_939_304, 202410_940_940_0] 2024.10.22 02:33:39.522951 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} MergeTask::MergeProjectionsStage: Merge sorted 3840 rows, containing 13 columns (13 merged, 0 gathered) in 2.836386934 sec., 1353.8350335666862 rows/sec., 234.13 KiB/sec. 2024.10.22 02:33:39.649755 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_999_364} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_999_364 to 202410_1_999_364 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:39.654922 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_999_364} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_998_363, 202410_999_999_0] -> 202410_1_999_364 2024.10.22 02:33:39.688124 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} MergedBlockOutputStream: filled checksums 202410_1626_1681_11 (state Temporary) 2024.10.22 02:33:39.692078 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:33:39.763925 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(399e8cf42fdb5649581ebe7bf5b8bbb3) : [0, 167]: Setting download as failed: File segment: [0, 167], key: 399e8cf42fdb5649581ebe7bf5b8bbb3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:39.766007 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(399e8cf42fdb5649581ebe7bf5b8bbb3): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 02:33:39.846103 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_974_74} MergeTreeSequentialSource: Reading 2 marks from part 202410_974_974_0, total 200 rows starting from the beginning of the part 2024.10.22 02:33:40.172509 [ 1234 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 215 ms. 2024.10.22 02:33:40.126425 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1681_11 to 202410_1626_1681_11 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:40.250232 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1681_11} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1676_10, 202410_1681_1681_0] -> 202410_1626_1681_11 2024.10.22 02:33:40.372896 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:33:40.492014 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(96b1ef95eea8b47b208ca8943bd453bd) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 96b1ef95eea8b47b208ca8943bd453bd, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:33:40.496789 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 577, file segment info: File segment: [0, 33554431], key: 96b1ef95eea8b47b208ca8943bd453bd, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:33:40.503292 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:40.558517 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1157 entries to flush up to offset 1427317 2024.10.22 02:33:40.570642 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 8.11 MiB, will set to 1.55 GiB (RSS), difference: -31.12 MiB 2024.10.22 02:33:40.573158 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 112171 2024.10.22 02:33:40.636151 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} MergedBlockOutputStream: filled checksums 202410_185_199_3 (state Temporary) 2024.10.22 02:33:40.647925 [ 1732 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/pzl/tytlcvpbzgsbszxzipktfctsmorpx, size 313303 2024.10.22 02:33:41.273419 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:41.275475 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:41.438980 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:41.443589 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:33:41.444737 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:41.463161 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:33:41.470063 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1669_1669_0 (state Temporary) 2024.10.22 02:33:41.487259 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:33:41.543663 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1669_1669_0 to 202410_1682_1682_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:41.549114 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:33:41.550314 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:33:41.570459 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_949_949_0 (state Temporary) 2024.10.22 02:33:41.596382 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112171 2024.10.22 02:33:41.621684 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ff875cb4d93c074d48bb6ee40aa3b09e) : [0, 128819]: Setting download as failed: File segment: [0, 128819], key: ff875cb4d93c074d48bb6ee40aa3b09e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:41.624217 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:33:41.624995 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ff875cb4d93c074d48bb6ee40aa3b09e): No space left in cache to reserve 128820 bytes, will continue without cache download 2024.10.22 02:33:41.644687 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_949_949_0 to 202410_956_956_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:41.652956 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:33:41.673830 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1427317 2024.10.22 02:33:41.665654 [ 1235 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_946_175 to 202410_947_952_1 2024.10.22 02:33:41.690082 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:33:41.693817 [ 1235 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.13 MiB using storage policy from min volume index 0 2024.10.22 02:33:41.710592 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:33:41.746840 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:33:41.767291 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column MobilePhone 2024.10.22 02:33:41.796177 [ 1235 ] {} DiskLocal: Reserved 1.13 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:41.844458 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_952_176} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_946_175 to 202410_947_952_1 into Wide with storage Full 2024.10.22 02:33:41.853426 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ff875cb4d93c074d48bb6ee40aa3b09e, offset: 0, size: 0 2024.10.22 02:33:41.855689 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ff875cb4d93c074d48bb6ee40aa3b09e for removal 2024.10.22 02:33:41.858581 [ 1476 ] {} CacheMetadata: Key ff875cb4d93c074d48bb6ee40aa3b09e is removed from metadata 2024.10.22 02:33:41.859434 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 399e8cf42fdb5649581ebe7bf5b8bbb3, offset: 0, size: 0 2024.10.22 02:33:41.863392 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 399e8cf42fdb5649581ebe7bf5b8bbb3 for removal 2024.10.22 02:33:41.871049 [ 1476 ] {} CacheMetadata: Key 399e8cf42fdb5649581ebe7bf5b8bbb3 is removed from metadata 2024.10.22 02:33:41.880176 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_952_176} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:41.893147 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_952_176} MergeTreeSequentialSource: Reading 176 marks from part 202410_1_946_175, total 1428198 rows starting from the beginning of the part 2024.10.22 02:33:41.900410 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/gmc/afkpeoighpcjfymbauohlxiyuynhy (LogSeriesLimiter: on interval from 2024-10-22 02:33:20 to 2024-10-22 02:33:41 accepted series 1 / 6 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:33:42.039059 [ 1385 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_952_176} MergeTreeSequentialSource: Reading 2 marks from part 202410_947_952_1, total 7702 rows starting from the beginning of the part 2024.10.22 02:33:42.294764 [ 1207 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:33:42.341559 [ 1207 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:33:42.550775 [ 1264 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 231 ms. 2024.10.22 02:33:42.967361 [ 1151 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 238 ms. 2024.10.22 02:33:43.058653 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_974_74/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:33:43.263160 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(29f5216dceed9376fbeddb331ac42bf0) : [0, 168]: Setting download as failed: File segment: [0, 168], key: 29f5216dceed9376fbeddb331ac42bf0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:43.264548 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(29f5216dceed9376fbeddb331ac42bf0): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 02:33:43.509234 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_974_74} MergeTask::MergeProjectionsStage: Merge sorted 25860 rows, containing 13 columns (13 merged, 0 gathered) in 4.32773272 sec., 5975.415228507919 rows/sec., 2.61 MiB/sec. 2024.10.22 02:33:43.571568 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.63 MiB, will set to 1.56 GiB (RSS), difference: -10.23 MiB 2024.10.22 02:33:43.702244 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_974_74} MergedBlockOutputStream: filled checksums 202410_747_974_74 (state Temporary) 2024.10.22 02:33:44.157743 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_974_74} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_974_74 to 202410_747_974_74 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:44.161512 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_974_74} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_973_73, 202410_974_974_0] -> 202410_747_974_74 2024.10.22 02:33:44.174698 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.13 MiB. 2024.10.22 02:33:44.385526 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(568ad75e77f23395354b5e8c9ac24a90) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 568ad75e77f23395354b5e8c9ac24a90, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:33:44.388746 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 598, file segment info: File segment: [0, 33554431], key: 568ad75e77f23395354b5e8c9ac24a90, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:33:44.391906 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:44.746373 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 8043 2024.10.22 02:33:44.924627 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_952_176/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:33:45.045218 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9cf25fbbf6fd58fe3560a0313a89dcb6) : [0, 148739]: Setting download as failed: File segment: [0, 148739], key: 9cf25fbbf6fd58fe3560a0313a89dcb6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:45.047216 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9cf25fbbf6fd58fe3560a0313a89dcb6): No space left in cache to reserve 148740 bytes, will continue without cache download 2024.10.22 02:33:45.066041 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9cf25fbbf6fd58fe3560a0313a89dcb6, offset: 0, size: 0 2024.10.22 02:33:45.082315 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9cf25fbbf6fd58fe3560a0313a89dcb6 for removal 2024.10.22 02:33:45.087757 [ 1476 ] {} CacheMetadata: Key 9cf25fbbf6fd58fe3560a0313a89dcb6 is removed from metadata 2024.10.22 02:33:45.089188 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 29f5216dceed9376fbeddb331ac42bf0, offset: 0, size: 0 2024.10.22 02:33:45.093262 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 29f5216dceed9376fbeddb331ac42bf0 for removal 2024.10.22 02:33:45.096596 [ 1476 ] {} CacheMetadata: Key 29f5216dceed9376fbeddb331ac42bf0 is removed from metadata 2024.10.22 02:33:45.615523 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 20 entries to flush up to offset 15307 2024.10.22 02:33:45.652576 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(c2cf1510626fa5aeb5160fb92d39c949) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: c2cf1510626fa5aeb5160fb92d39c949, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:33:45.709790 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 67 entries to flush up to offset 112238 2024.10.22 02:33:45.829271 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 156, file segment info: File segment: [0, 33554431], key: c2cf1510626fa5aeb5160fb92d39c949, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:33:45.832831 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:45.853141 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:45.857128 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:45.915294 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:45.918253 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:45.966197 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_991_991_0 (state Temporary) 2024.10.22 02:33:46.040622 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1670_1670_0 (state Temporary) 2024.10.22 02:33:46.071032 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1670_1670_0 to 202410_1683_1683_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:46.203049 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_991_991_0 to 202410_1004_1004_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:46.252580 [ 1250 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_999_364 to 202410_1000_1000_0 2024.10.22 02:33:46.264233 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15307 2024.10.22 02:33:46.273165 [ 1250 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:46.276894 [ 1250 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:46.312460 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1000_365} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_999_364 to 202410_1000_1000_0 into Compact with storage Full 2024.10.22 02:33:46.345451 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112238 2024.10.22 02:33:46.461218 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1000_365} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:46.466681 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c546253874f8181f3b615d855cdd1b28) : [0, 167]: Setting download as failed: File segment: [0, 167], key: c546253874f8181f3b615d855cdd1b28, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:46.469171 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c546253874f8181f3b615d855cdd1b28): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 02:33:46.566148 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1000_365} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_999_364, total 15331 rows starting from the beginning of the part 2024.10.22 02:33:46.580773 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1000_365} MergeTreeSequentialSource: Reading 2 marks from part 202410_1000_1000_0, total 19 rows starting from the beginning of the part 2024.10.22 02:33:46.653492 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 204 entries to flush up to offset 110238 2024.10.22 02:33:46.671340 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 15.29 MiB, will set to 1.56 GiB (RSS), difference: 35.02 MiB 2024.10.22 02:33:47.158641 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(08b5e9edb2bd020fcf77839f6ad9973c) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 08b5e9edb2bd020fcf77839f6ad9973c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:33:47.171625 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 359, file segment info: File segment: [0, 33554431], key: 08b5e9edb2bd020fcf77839f6ad9973c, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:33:47.174457 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:47.297222 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:47.303659 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:47.361703 [ 1419 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_199_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:33:47.420591 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 821 2024.10.22 02:33:47.534313 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1000_365} MergeTask::MergeProjectionsStage: Merge sorted 15350 rows, containing 12 columns (12 merged, 0 gathered) in 1.22805608 sec., 12499.429179162567 rows/sec., 1.96 MiB/sec. 2024.10.22 02:33:47.568421 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_974_974_0 (state Temporary) 2024.10.22 02:33:47.670960 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:47.674942 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:47.742490 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_974_974_0 to 202410_980_980_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:47.758310 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1000_365} MergedBlockOutputStream: filled checksums 202410_1_1000_365 (state Temporary) 2024.10.22 02:33:47.779959 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 110238 2024.10.22 02:33:47.875283 [ 1272 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_974_74 to 202410_975_975_0 2024.10.22 02:33:47.887215 [ 1272 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.45 MiB using storage policy from min volume index 0 2024.10.22 02:33:47.912226 [ 1272 ] {} DiskLocal: Reserved 1.45 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:47.940304 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_975_75} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_974_74 to 202410_975_975_0 into Wide with storage Full 2024.10.22 02:33:47.947331 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_803_803_0 (state Temporary) 2024.10.22 02:33:47.987871 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_975_75} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:48.004102 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_975_75} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_974_74, total 25860 rows starting from the beginning of the part 2024.10.22 02:33:48.016777 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_803_803_0 to 202410_813_813_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:48.051482 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bca8abde26275fd3e7f2172649845cf6) : [0, 81797]: Setting download as failed: File segment: [0, 81797], key: bca8abde26275fd3e7f2172649845cf6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:48.053735 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bca8abde26275fd3e7f2172649845cf6): No space left in cache to reserve 81798 bytes, will continue without cache download 2024.10.22 02:33:48.054362 [ 1226 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_808_164 to 202410_813_813_0 2024.10.22 02:33:48.085111 [ 1226 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:48.092988 [ 1226 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:48.114392 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_808_164 to 202410_813_813_0 into Compact with storage Full 2024.10.22 02:33:48.117832 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bca8abde26275fd3e7f2172649845cf6, offset: 0, size: 0 2024.10.22 02:33:48.121834 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bca8abde26275fd3e7f2172649845cf6 for removal 2024.10.22 02:33:48.122996 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_940_305, 202410_941_941_0] 2024.10.22 02:33:48.130265 [ 1476 ] {} CacheMetadata: Key bca8abde26275fd3e7f2172649845cf6 is removed from metadata 2024.10.22 02:33:48.129734 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_940_305, 202410_941_941_0] 2024.10.22 02:33:48.134483 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c546253874f8181f3b615d855cdd1b28, offset: 0, size: 0 2024.10.22 02:33:48.189095 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:48.236309 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c546253874f8181f3b615d855cdd1b28 for removal 2024.10.22 02:33:48.241592 [ 1476 ] {} CacheMetadata: Key c546253874f8181f3b615d855cdd1b28 is removed from metadata 2024.10.22 02:33:48.242839 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_940_305, 202410_941_941_0] 2024.10.22 02:33:48.268782 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_808_164, total 852 rows starting from the beginning of the part 2024.10.22 02:33:48.274426 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1000_365} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1000_365 to 202410_1_1000_365 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:48.284226 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1000_365} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_999_364, 202410_1000_1000_0] -> 202410_1_1000_365 2024.10.22 02:33:48.287614 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_975_75} MergeTreeSequentialSource: Reading 2 marks from part 202410_975_975_0, total 86 rows starting from the beginning of the part 2024.10.22 02:33:48.325213 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:33:48.353284 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} MergeTreeSequentialSource: Reading 2 marks from part 202410_809_809_0, total 1 rows starting from the beginning of the part 2024.10.22 02:33:48.391163 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} MergeTreeSequentialSource: Reading 2 marks from part 202410_810_810_0, total 1 rows starting from the beginning of the part 2024.10.22 02:33:48.496872 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 821 2024.10.22 02:33:48.544893 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} MergeTreeSequentialSource: Reading 2 marks from part 202410_811_811_0, total 1 rows starting from the beginning of the part 2024.10.22 02:33:48.655572 [ 1267 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 217 ms. 2024.10.22 02:33:48.681182 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1428087 2024.10.22 02:33:48.690215 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} MergeTreeSequentialSource: Reading 2 marks from part 202410_812_812_0, total 1 rows starting from the beginning of the part 2024.10.22 02:33:48.747500 [ 1386 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} MergeTreeSequentialSource: Reading 2 marks from part 202410_813_813_0, total 1 rows starting from the beginning of the part 2024.10.22 02:33:49.251363 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(975861830e307cb69aa547f96c01f7f2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 975861830e307cb69aa547f96c01f7f2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:33:49.262194 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 464, file segment info: File segment: [0, 33554431], key: 975861830e307cb69aa547f96c01f7f2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1388, kind: Regular, unbound: 0 2024.10.22 02:33:49.264574 [ 1388 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:33:49.356444 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:49.440341 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:49.627885 [ 1263 ] {} DatabaseCatalog: Execution took 470 ms. 2024.10.22 02:33:49.665428 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_950_950_0 (state Temporary) 2024.10.22 02:33:49.856733 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:33:49.907222 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_950_950_0 to 202410_957_957_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:49.967633 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1428087 2024.10.22 02:33:49.983588 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:33:50.005325 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:33:50.041554 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:33:50.058509 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:33:50.078376 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:33:50.122449 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ShareTitle 2024.10.22 02:33:50.271434 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/jif/jtlqsmrhdoqjrgwzbygmyjerupfgu (LogSeriesLimiter: on interval from 2024-10-22 02:33:41 to 2024-10-22 02:33:50 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:33:50.301779 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a630e6cb0e6786089adc956a7e38924c) : [0, 168]: Setting download as failed: File segment: [0, 168], key: a630e6cb0e6786089adc956a7e38924c, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:50.306885 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a630e6cb0e6786089adc956a7e38924c): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 02:33:50.374886 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 112334 2024.10.22 02:33:50.489055 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 13.03 MiB, will set to 1.57 GiB (RSS), difference: -42.89 MiB 2024.10.22 02:33:50.825378 [ 1390 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} MergeTask::MergeProjectionsStage: Merge sorted 857 rows, containing 32 columns (32 merged, 0 gathered) in 2.724029253 sec., 314.6074878073272 rows/sec., 23.62 KiB/sec. 2024.10.22 02:33:50.912397 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} MergedBlockOutputStream: filled checksums 202410_1_813_165 (state Temporary) 2024.10.22 02:33:50.914955 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_975_75} MergeTask::MergeProjectionsStage: Merge sorted 25946 rows, containing 13 columns (13 merged, 0 gathered) in 2.994383585 sec., 8664.88853665019 rows/sec., 3.78 MiB/sec. 2024.10.22 02:33:50.969554 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_975_75} MergedBlockOutputStream: filled checksums 202410_747_975_75 (state Temporary) 2024.10.22 02:33:50.972852 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:50.975116 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:51.059952 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1671_1671_0 (state Temporary) 2024.10.22 02:33:51.125569 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1671_1671_0 to 202410_1684_1684_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:51.183029 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112334 2024.10.22 02:33:51.222127 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(55eacc6831e287ff860d0e07a9a47464) : [0, 1049087]: Setting download as failed: File segment: [0, 1049087], key: 55eacc6831e287ff860d0e07a9a47464, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:51.224566 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(55eacc6831e287ff860d0e07a9a47464): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:33:51.257707 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 55eacc6831e287ff860d0e07a9a47464, offset: 0, size: 0 2024.10.22 02:33:51.261532 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 55eacc6831e287ff860d0e07a9a47464 for removal 2024.10.22 02:33:51.263884 [ 1476 ] {} CacheMetadata: Key 55eacc6831e287ff860d0e07a9a47464 is removed from metadata 2024.10.22 02:33:51.265780 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a630e6cb0e6786089adc956a7e38924c, offset: 0, size: 0 2024.10.22 02:33:51.268780 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a630e6cb0e6786089adc956a7e38924c for removal 2024.10.22 02:33:51.281131 [ 1476 ] {} CacheMetadata: Key a630e6cb0e6786089adc956a7e38924c is removed from metadata 2024.10.22 02:33:51.549609 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:33:51.551062 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:33:51.565332 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_813_165 to 202410_1_813_165 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:51.569435 [ 1400 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_813_165} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_808_164, 202410_813_813_0] -> 202410_1_813_165 2024.10.22 02:33:51.608565 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:33:51.735254 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_975_75} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_975_75 to 202410_747_975_75 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:51.739844 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_975_75} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_974_74, 202410_975_975_0] -> 202410_747_975_75 2024.10.22 02:33:51.760576 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.02 MiB. 2024.10.22 02:33:52.055459 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(306fa462c314f665d2140adb380a6947) : [0, 154]: Setting download as failed: File segment: [0, 154], key: 306fa462c314f665d2140adb380a6947, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:52.059529 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(306fa462c314f665d2140adb380a6947): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 02:33:52.755477 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 11.78 MiB, will set to 1.56 GiB (RSS), difference: 36.07 MiB 2024.10.22 02:33:52.897626 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e9bbfa6bde726615ea251934bdc1c5c6) : [0, 50373]: Setting download as failed: File segment: [0, 50373], key: e9bbfa6bde726615ea251934bdc1c5c6, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:52.900620 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e9bbfa6bde726615ea251934bdc1c5c6): No space left in cache to reserve 50374 bytes, will continue without cache download 2024.10.22 02:33:53.040262 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e9bbfa6bde726615ea251934bdc1c5c6, offset: 0, size: 0 2024.10.22 02:33:53.042786 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e9bbfa6bde726615ea251934bdc1c5c6 for removal 2024.10.22 02:33:53.046511 [ 1476 ] {} CacheMetadata: Key e9bbfa6bde726615ea251934bdc1c5c6 is removed from metadata 2024.10.22 02:33:53.049013 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 306fa462c314f665d2140adb380a6947, offset: 0, size: 0 2024.10.22 02:33:53.051030 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 306fa462c314f665d2140adb380a6947 for removal 2024.10.22 02:33:53.054018 [ 1476 ] {} CacheMetadata: Key 306fa462c314f665d2140adb380a6947 is removed from metadata 2024.10.22 02:33:53.768530 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 19 entries to flush up to offset 15326 2024.10.22 02:33:53.981270 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a37d173f0f996a89620959ea12fcb2ae) : [0, 155]: Setting download as failed: File segment: [0, 155], key: a37d173f0f996a89620959ea12fcb2ae, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:53.985826 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a37d173f0f996a89620959ea12fcb2ae): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:33:54.151314 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:54.153624 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:54.193534 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_992_992_0 (state Temporary) 2024.10.22 02:33:54.249728 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_992_992_0 to 202410_1005_1005_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:54.270807 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1000_365 to 202410_1001_1001_0 2024.10.22 02:33:54.276671 [ 1251 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:54.284196 [ 1251 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:54.306769 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1001_366} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1000_365 to 202410_1001_1001_0 into Compact with storage Full 2024.10.22 02:33:54.364802 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1001_366} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:54.372192 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1001_366} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1000_365, total 15350 rows starting from the beginning of the part 2024.10.22 02:33:54.397266 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1001_366} MergeTreeSequentialSource: Reading 2 marks from part 202410_1001_1001_0, total 18 rows starting from the beginning of the part 2024.10.22 02:33:54.474023 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.46 MiB, will set to 1.56 GiB (RSS), difference: 2.84 MiB 2024.10.22 02:33:54.499976 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15326 2024.10.22 02:33:54.665704 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1207_1580_119, 202410_1581_1581_0, 202410_1582_1582_0, 202410_1583_1583_0] 2024.10.22 02:33:54.667196 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1207_1580_119, 202410_1581_1581_0, 202410_1582_1582_0, 202410_1583_1583_0] 2024.10.22 02:33:54.686305 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1207_1580_119, 202410_1581_1581_0, 202410_1582_1582_0, 202410_1583_1583_0] 2024.10.22 02:33:55.068134 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_1001_366/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:33:55.187698 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_941_306, 202410_942_942_0] 2024.10.22 02:33:55.197830 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 112392 2024.10.22 02:33:55.200029 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_941_306, 202410_942_942_0] 2024.10.22 02:33:55.210213 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a07390c35c28eb3e409f89d98c4d51ac) : [0, 43994]: Setting download as failed: File segment: [0, 43994], key: a07390c35c28eb3e409f89d98c4d51ac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:55.212882 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a07390c35c28eb3e409f89d98c4d51ac): No space left in cache to reserve 43995 bytes, will continue without cache download 2024.10.22 02:33:55.215052 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_941_306, 202410_942_942_0] 2024.10.22 02:33:55.249633 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a07390c35c28eb3e409f89d98c4d51ac, offset: 0, size: 0 2024.10.22 02:33:55.254258 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a07390c35c28eb3e409f89d98c4d51ac for removal 2024.10.22 02:33:55.256949 [ 1476 ] {} CacheMetadata: Key a07390c35c28eb3e409f89d98c4d51ac is removed from metadata 2024.10.22 02:33:55.259275 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a37d173f0f996a89620959ea12fcb2ae, offset: 0, size: 0 2024.10.22 02:33:55.260848 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a37d173f0f996a89620959ea12fcb2ae for removal 2024.10.22 02:33:55.262983 [ 1476 ] {} CacheMetadata: Key a37d173f0f996a89620959ea12fcb2ae is removed from metadata 2024.10.22 02:33:55.267434 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1001_366} MergeTask::MergeProjectionsStage: Merge sorted 15368 rows, containing 12 columns (12 merged, 0 gathered) in 0.970016138 sec., 15843.035386695805 rows/sec., 2.48 MiB/sec. 2024.10.22 02:33:55.476162 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:55.478342 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:55.538049 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 212 entries to flush up to offset 110450 2024.10.22 02:33:55.556761 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1672_1672_0 (state Temporary) 2024.10.22 02:33:55.558038 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1001_366} MergedBlockOutputStream: filled checksums 202410_1_1001_366 (state Temporary) 2024.10.22 02:33:55.617108 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_952_176/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:33:55.706479 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1672_1672_0 to 202410_1685_1685_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:55.795692 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112392 2024.10.22 02:33:55.999616 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 822 2024.10.22 02:33:56.103091 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1001_366} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1001_366 to 202410_1_1001_366 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:56.107725 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1001_366} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1000_365, 202410_1001_1001_0] -> 202410_1_1001_366 2024.10.22 02:33:56.149522 [ 1395 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:33:56.195570 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b3a021fd58f136b7857187ba5616d8fb) : [0, 152]: Setting download as failed: File segment: [0, 152], key: b3a021fd58f136b7857187ba5616d8fb, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:56.197415 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b3a021fd58f136b7857187ba5616d8fb): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 02:33:56.268277 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:56.275108 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:56.336182 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:56.348259 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:56.464692 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_975_975_0 (state Temporary) 2024.10.22 02:33:56.518505 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_804_804_0 (state Temporary) 2024.10.22 02:33:56.639368 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_975_975_0 to 202410_981_981_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:56.662664 [ 1190 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_975_75 to 202410_976_976_0 2024.10.22 02:33:56.667360 [ 1190 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.46 MiB using storage policy from min volume index 0 2024.10.22 02:33:56.670378 [ 1190 ] {} DiskLocal: Reserved 1.46 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:56.672665 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 110450 2024.10.22 02:33:56.712301 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_804_804_0 to 202410_814_814_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:56.726822 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_976_76} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_975_75 to 202410_976_976_0 into Wide with storage Full 2024.10.22 02:33:56.806097 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 822 2024.10.22 02:33:56.843640 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_976_76} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:56.865299 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_976_76} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_975_75, total 25946 rows starting from the beginning of the part 2024.10.22 02:33:56.942267 [ 1160 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 02:33:56.971626 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1429627 2024.10.22 02:33:57.251116 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_976_76} MergeTreeSequentialSource: Reading 2 marks from part 202410_976_976_0, total 187 rows starting from the beginning of the part 2024.10.22 02:33:57.347963 [ 1202 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:33:57.369128 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1643a3e0e660bcb076e53a45f4213123) : [0, 76585]: Setting download as failed: File segment: [0, 76585], key: 1643a3e0e660bcb076e53a45f4213123, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:57.370794 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1643a3e0e660bcb076e53a45f4213123): No space left in cache to reserve 76586 bytes, will continue without cache download 2024.10.22 02:33:57.375306 [ 1202 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:33:57.501186 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1643a3e0e660bcb076e53a45f4213123, offset: 0, size: 0 2024.10.22 02:33:57.513550 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1643a3e0e660bcb076e53a45f4213123 for removal 2024.10.22 02:33:57.620233 [ 1476 ] {} CacheMetadata: Key 1643a3e0e660bcb076e53a45f4213123 is removed from metadata 2024.10.22 02:33:57.621006 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b3a021fd58f136b7857187ba5616d8fb, offset: 0, size: 0 2024.10.22 02:33:57.624749 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b3a021fd58f136b7857187ba5616d8fb for removal 2024.10.22 02:33:57.628882 [ 1476 ] {} CacheMetadata: Key b3a021fd58f136b7857187ba5616d8fb is removed from metadata 2024.10.22 02:33:57.723915 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.21 MiB, will set to 1.56 GiB (RSS), difference: -9.26 MiB 2024.10.22 02:33:58.056110 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:58.059099 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:58.168141 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_951_951_0 (state Temporary) 2024.10.22 02:33:58.234749 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_951_951_0 to 202410_958_958_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:58.292551 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1429627 2024.10.22 02:33:58.280456 [ 1186 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_953_953_0 to 202410_958_958_0 2024.10.22 02:33:58.317211 [ 1186 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:33:58.320849 [ 1186 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:33:58.376386 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_953_953_0 to 202410_958_958_0 into Compact with storage Full 2024.10.22 02:33:58.413103 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:33:58.428641 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_953_953_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:33:58.454965 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_954_954_0, total 943 rows starting from the beginning of the part 2024.10.22 02:33:58.470662 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_955_955_0, total 1367 rows starting from the beginning of the part 2024.10.22 02:33:58.500806 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_956_956_0, total 1157 rows starting from the beginning of the part 2024.10.22 02:33:58.732061 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_957_957_0, total 770 rows starting from the beginning of the part 2024.10.22 02:33:58.858315 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_958_958_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:33:59.206488 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5114801ee943ba4a1714008b2465cd2f) : [0, 152]: Setting download as failed: File segment: [0, 152], key: 5114801ee943ba4a1714008b2465cd2f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:33:59.212007 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5114801ee943ba4a1714008b2465cd2f): No space left in cache to reserve 153 bytes, will continue without cache download 2024.10.22 02:33:59.376488 [ 1425 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_merge_202410_185_199_3/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:33:59.439340 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_merge_202410_185_199_3 to 202410_185_199_3 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:33:59.444490 [ 1389 ] {92e05502-7cd1-42a9-b81b-a85674c93a09::202410_185_199_3} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09) (MergerMutator): Merged 6 parts: [202410_185_194_2, 202410_199_199_0] -> 202410_185_199_3 2024.10.22 02:33:59.477563 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_976_76} MergeTask::MergeProjectionsStage: Merge sorted 26133 rows, containing 13 columns (13 merged, 0 gathered) in 2.798359429 sec., 9338.685991934455 rows/sec., 4.07 MiB/sec. 2024.10.22 02:33:59.578253 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_976_76} MergedBlockOutputStream: filled checksums 202410_747_976_76 (state Temporary) 2024.10.22 02:33:59.804527 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 112472 2024.10.22 02:33:59.860496 [ 1193 ] {} DatabaseCatalog: Execution took 230 ms. 2024.10.22 02:34:00.106394 [ 1388 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} MergeTask::MergeProjectionsStage: Merge sorted 7317 rows, containing 4 columns (4 merged, 0 gathered) in 1.741021513 sec., 4202.705104655419 rows/sec., 65.67 KiB/sec. 2024.10.22 02:34:00.168279 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:00.171514 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:00.193005 [ 1395 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} MergedBlockOutputStream: filled checksums 202410_953_958_1 (state Temporary) 2024.10.22 02:34:00.253221 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1673_1673_0 (state Temporary) 2024.10.22 02:34:00.263181 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_953_958_1 to 202410_953_958_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:00.265781 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_953_958_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_953_953_0, 202410_958_958_0] -> 202410_953_958_1 2024.10.22 02:34:00.278445 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 16.12 MiB. 2024.10.22 02:34:00.322310 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:34:00.419818 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_976_76} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_976_76 to 202410_747_976_76 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:00.424226 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_976_76} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_975_75, 202410_976_976_0] -> 202410_747_976_76 2024.10.22 02:34:00.426813 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(d23a1bb90c95e6c7952d89d04c71adc5) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d23a1bb90c95e6c7952d89d04c71adc5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:00.440924 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 302507, file segment info: File segment: [0, 33554431], key: d23a1bb90c95e6c7952d89d04c71adc5, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:00.443560 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:00.451945 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.13 MiB. 2024.10.22 02:34:00.464994 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1673_1673_0 to 202410_1686_1686_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:00.613681 [ 1208 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1681_11 to 202410_1686_1686_0 2024.10.22 02:34:00.645041 [ 1208 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:00.654883 [ 1208 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:00.740513 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112472 2024.10.22 02:34:00.819624 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1681_11 to 202410_1686_1686_0 into Compact with storage Full 2024.10.22 02:34:00.863177 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:00.871264 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1681_11, total 3840 rows starting from the beginning of the part 2024.10.22 02:34:00.939597 [ 1198 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 209 ms. 2024.10.22 02:34:01.022853 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1682_1682_0, total 88 rows starting from the beginning of the part 2024.10.22 02:34:01.150655 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1683_1683_0, total 67 rows starting from the beginning of the part 2024.10.22 02:34:01.183501 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1684_1684_0, total 96 rows starting from the beginning of the part 2024.10.22 02:34:01.219232 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1685_1685_0, total 58 rows starting from the beginning of the part 2024.10.22 02:34:01.250188 [ 1399 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} MergeTreeSequentialSource: Reading 2 marks from part 202410_1686_1686_0, total 80 rows starting from the beginning of the part 2024.10.22 02:34:01.546527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(187c420ba124e3957c65b7cf60120e87) : [0, 111992]: Setting download as failed: File segment: [0, 111992], key: 187c420ba124e3957c65b7cf60120e87, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:01.549647 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(187c420ba124e3957c65b7cf60120e87): No space left in cache to reserve 111993 bytes, will continue without cache download 2024.10.22 02:34:01.551657 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:34:01.552613 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:34:01.570833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 187c420ba124e3957c65b7cf60120e87, offset: 0, size: 0 2024.10.22 02:34:01.573437 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 187c420ba124e3957c65b7cf60120e87 for removal 2024.10.22 02:34:01.576979 [ 1476 ] {} CacheMetadata: Key 187c420ba124e3957c65b7cf60120e87 is removed from metadata 2024.10.22 02:34:01.578781 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5114801ee943ba4a1714008b2465cd2f, offset: 0, size: 0 2024.10.22 02:34:01.581604 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5114801ee943ba4a1714008b2465cd2f for removal 2024.10.22 02:34:01.585115 [ 1476 ] {} CacheMetadata: Key 5114801ee943ba4a1714008b2465cd2f is removed from metadata 2024.10.22 02:34:01.895411 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:01.898322 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:02.007549 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 15352 2024.10.22 02:34:02.060021 [ 1391 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} MergeTask::MergeProjectionsStage: Merge sorted 4229 rows, containing 13 columns (13 merged, 0 gathered) in 1.320508268 sec., 3202.5547302366454 rows/sec., 565.45 KiB/sec. 2024.10.22 02:34:02.204538 [ 1395 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} MergedBlockOutputStream: filled checksums 202410_1626_1686_12 (state Temporary) 2024.10.22 02:34:02.220289 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_942_307, 202410_943_943_0] 2024.10.22 02:34:02.227234 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_942_307, 202410_943_943_0] 2024.10.22 02:34:02.241420 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_942_307, 202410_943_943_0] 2024.10.22 02:34:02.304488 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:02.306928 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:02.308213 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 13.45 MiB, will set to 1.56 GiB (RSS), difference: 12.50 MiB 2024.10.22 02:34:02.359082 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_993_993_0 (state Temporary) 2024.10.22 02:34:02.451008 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1686_12 to 202410_1626_1686_12 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:02.456195 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1686_12} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1681_11, 202410_1686_1686_0] -> 202410_1626_1686_12 2024.10.22 02:34:02.508029 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:34:02.531182 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_993_993_0 to 202410_1006_1006_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:02.554537 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1001_366 to 202410_1002_1002_0 2024.10.22 02:34:02.563932 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15352 2024.10.22 02:34:02.625492 [ 1228 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:02.635323 [ 1228 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:02.657102 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1002_367} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1001_366 to 202410_1002_1002_0 into Compact with storage Full 2024.10.22 02:34:02.677331 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(a0270bc39da51189ee86092424e5cfe1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a0270bc39da51189ee86092424e5cfe1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:02.688986 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 577, file segment info: File segment: [0, 33554431], key: a0270bc39da51189ee86092424e5cfe1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:02.695492 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1002_367} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:02.696815 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:02.705316 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1002_367} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1001_366, total 15368 rows starting from the beginning of the part 2024.10.22 02:34:02.735868 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1002_367} MergeTreeSequentialSource: Reading 2 marks from part 202410_1002_1002_0, total 12 rows starting from the beginning of the part 2024.10.22 02:34:02.748305 [ 1729 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/gmc/afkpeoighpcjfymbauohlxiyuynhy, size 302507 2024.10.22 02:34:03.012920 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:34:03.053037 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:34:03.063227 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:34:03.077928 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:34:03.197117 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:34:03.244772 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:34:03.259376 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:34:03.272726 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:34:03.289802 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column MobilePhoneModel 2024.10.22 02:34:03.326684 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/lvr/evdqmzaxpadfwyckqxzyufdsiiuiz (LogSeriesLimiter: on interval from 2024-10-22 02:33:50 to 2024-10-22 02:34:03 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:34:03.663269 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1002_367} MergeTask::MergeProjectionsStage: Merge sorted 15380 rows, containing 12 columns (12 merged, 0 gathered) in 1.011947084 sec., 15198.423161818213 rows/sec., 2.38 MiB/sec. 2024.10.22 02:34:03.711051 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_886_165, 202410_887_892_1] 2024.10.22 02:34:03.716189 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_886_165, 202410_887_892_1] 2024.10.22 02:34:03.728229 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1002_367} MergedBlockOutputStream: filled checksums 202410_1_1002_367 (state Temporary) 2024.10.22 02:34:03.835816 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_886_165, 202410_887_892_1] 2024.10.22 02:34:03.959864 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ebec21afe1f9ad5501efcfc46d29f648) : [0, 156]: Setting download as failed: File segment: [0, 156], key: ebec21afe1f9ad5501efcfc46d29f648, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:03.966621 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ebec21afe1f9ad5501efcfc46d29f648): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 02:34:04.268733 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 204 entries to flush up to offset 110654 2024.10.22 02:34:04.309356 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 823 2024.10.22 02:34:04.540181 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:04.544623 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:04.549102 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1002_367} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1002_367 to 202410_1_1002_367 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:04.553426 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1002_367} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1001_366, 202410_1002_1002_0] -> 202410_1_1002_367 2024.10.22 02:34:04.593442 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:04.597159 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:04.670460 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:34:04.697640 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_976_976_0 (state Temporary) 2024.10.22 02:34:04.768786 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_976_976_0 to 202410_982_982_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:04.772838 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_805_805_0 (state Temporary) 2024.10.22 02:34:04.849243 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 96 entries to flush up to offset 112568 2024.10.22 02:34:04.887225 [ 1209 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_976_76 to 202410_977_977_0 2024.10.22 02:34:04.900783 [ 1209 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.47 MiB using storage policy from min volume index 0 2024.10.22 02:34:04.909035 [ 1209 ] {} DiskLocal: Reserved 1.47 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:04.927003 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 110654 2024.10.22 02:34:04.961246 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_977_77} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_976_76 to 202410_977_977_0 into Wide with storage Full 2024.10.22 02:34:05.044740 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_977_77} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:05.047431 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_805_805_0 to 202410_815_815_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:05.053990 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_977_77} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_976_76, total 26133 rows starting from the beginning of the part 2024.10.22 02:34:05.133029 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(e429e012127ff29940f97bf8f62207a2) : [0, 110003]: Setting download as failed: File segment: [0, 110003], key: e429e012127ff29940f97bf8f62207a2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:05.137177 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(e429e012127ff29940f97bf8f62207a2): No space left in cache to reserve 110004 bytes, will continue without cache download 2024.10.22 02:34:05.173946 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: e429e012127ff29940f97bf8f62207a2, offset: 0, size: 0 2024.10.22 02:34:05.177191 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key e429e012127ff29940f97bf8f62207a2 for removal 2024.10.22 02:34:05.185107 [ 1476 ] {} CacheMetadata: Key e429e012127ff29940f97bf8f62207a2 is removed from metadata 2024.10.22 02:34:05.193371 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ebec21afe1f9ad5501efcfc46d29f648, offset: 0, size: 0 2024.10.22 02:34:05.196514 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ebec21afe1f9ad5501efcfc46d29f648 for removal 2024.10.22 02:34:05.200405 [ 1476 ] {} CacheMetadata: Key ebec21afe1f9ad5501efcfc46d29f648 is removed from metadata 2024.10.22 02:34:05.259145 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 823 2024.10.22 02:34:05.286631 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_199_199_0 (state Temporary) 2024.10.22 02:34:05.297399 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1430397 2024.10.22 02:34:05.311840 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_977_77} MergeTreeSequentialSource: Reading 2 marks from part 202410_977_977_0, total 191 rows starting from the beginning of the part 2024.10.22 02:34:05.652269 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 11.94 MiB, will set to 1.56 GiB (RSS), difference: -48.64 MiB 2024.10.22 02:34:05.710187 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:05.719818 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:06.045743 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1674_1674_0 (state Temporary) 2024.10.22 02:34:06.097016 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1674_1674_0 to 202410_1687_1687_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:06.173650 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112568 2024.10.22 02:34:06.195355 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:06.198869 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:06.262708 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ba207c28c3381966467d4f708c2b282b) : [0, 179]: Setting download as failed: File segment: [0, 179], key: ba207c28c3381966467d4f708c2b282b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:06.264751 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ba207c28c3381966467d4f708c2b282b): No space left in cache to reserve 180 bytes, will continue without cache download 2024.10.22 02:34:06.339529 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_952_952_0 (state Temporary) 2024.10.22 02:34:06.549216 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_952_952_0 to 202410_959_959_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:06.573584 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1430397 2024.10.22 02:34:06.593798 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_952_176/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:34:07.340596 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_977_77} MergeTask::MergeProjectionsStage: Merge sorted 26324 rows, containing 13 columns (13 merged, 0 gathered) in 2.386628642 sec., 11029.784666432408 rows/sec., 4.81 MiB/sec. 2024.10.22 02:34:07.348895 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d446f1bcc2dd90cc9cafad269c2038c1) : [0, 242827]: Setting download as failed: File segment: [0, 242827], key: d446f1bcc2dd90cc9cafad269c2038c1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:07.350759 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d446f1bcc2dd90cc9cafad269c2038c1): No space left in cache to reserve 242828 bytes, will continue without cache download 2024.10.22 02:34:07.383127 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 14.60 MiB, will set to 1.57 GiB (RSS), difference: 7.38 MiB 2024.10.22 02:34:07.493404 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d446f1bcc2dd90cc9cafad269c2038c1, offset: 0, size: 0 2024.10.22 02:34:07.498660 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d446f1bcc2dd90cc9cafad269c2038c1 for removal 2024.10.22 02:34:07.503030 [ 1389 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_977_77} MergedBlockOutputStream: filled checksums 202410_747_977_77 (state Temporary) 2024.10.22 02:34:07.504887 [ 1476 ] {} CacheMetadata: Key d446f1bcc2dd90cc9cafad269c2038c1 is removed from metadata 2024.10.22 02:34:07.507800 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ba207c28c3381966467d4f708c2b282b, offset: 0, size: 0 2024.10.22 02:34:07.509596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ba207c28c3381966467d4f708c2b282b for removal 2024.10.22 02:34:07.516729 [ 1476 ] {} CacheMetadata: Key ba207c28c3381966467d4f708c2b282b is removed from metadata 2024.10.22 02:34:08.264872 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_977_77} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_977_77 to 202410_747_977_77 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:08.269324 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_977_77} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_976_76, 202410_977_977_0] -> 202410_747_977_77 2024.10.22 02:34:08.290886 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.64 MiB. 2024.10.22 02:34:08.402951 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2b75a4e4981ef4605cdac07800ec9831) : [0, 166]: Setting download as failed: File segment: [0, 166], key: 2b75a4e4981ef4605cdac07800ec9831, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:08.404236 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2b75a4e4981ef4605cdac07800ec9831): No space left in cache to reserve 167 bytes, will continue without cache download 2024.10.22 02:34:09.425349 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 13.68 MiB, will set to 1.56 GiB (RSS), difference: 28.66 MiB 2024.10.22 02:34:09.507111 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2b7e845ca554a25750453dc785bfbf22) : [0, 936602]: Setting download as failed: File segment: [0, 936602], key: 2b7e845ca554a25750453dc785bfbf22, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:09.510611 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2b7e845ca554a25750453dc785bfbf22): No space left in cache to reserve 936603 bytes, will continue without cache download 2024.10.22 02:34:09.541449 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2b7e845ca554a25750453dc785bfbf22, offset: 0, size: 0 2024.10.22 02:34:09.542860 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2b7e845ca554a25750453dc785bfbf22 for removal 2024.10.22 02:34:09.545439 [ 1476 ] {} CacheMetadata: Key 2b7e845ca554a25750453dc785bfbf22 is removed from metadata 2024.10.22 02:34:09.547860 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2b75a4e4981ef4605cdac07800ec9831, offset: 0, size: 0 2024.10.22 02:34:09.549497 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2b75a4e4981ef4605cdac07800ec9831 for removal 2024.10.22 02:34:09.552562 [ 1476 ] {} CacheMetadata: Key 2b75a4e4981ef4605cdac07800ec9831 is removed from metadata 2024.10.22 02:34:10.066029 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 18 entries to flush up to offset 15370 2024.10.22 02:34:10.095213 [ 1259 ] {} DatabaseCatalog: Execution took 234 ms. 2024.10.22 02:34:10.229558 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(e1ea803eebb2b113494d570944e7acc4) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: e1ea803eebb2b113494d570944e7acc4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:10.281573 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:10.302584 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 64 entries to flush up to offset 112632 2024.10.22 02:34:10.304669 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 993, file segment info: File segment: [0, 33554431], key: e1ea803eebb2b113494d570944e7acc4, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:10.307653 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:10.308550 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:10.352895 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_994_994_0 (state Temporary) 2024.10.22 02:34:10.505564 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_994_994_0 to 202410_1007_1007_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:10.552100 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:10.554209 [ 1205 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1002_367 to 202410_1003_1003_0 2024.10.22 02:34:10.555644 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:10.560874 [ 1205 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:10.572441 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15370 2024.10.22 02:34:10.563022 [ 1205 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:10.714685 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1675_1675_0 (state Temporary) 2024.10.22 02:34:10.769638 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1003_368} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1002_367 to 202410_1003_1003_0 into Compact with storage Full 2024.10.22 02:34:10.847309 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1003_368} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:10.858368 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1003_368} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1002_367, total 15380 rows starting from the beginning of the part 2024.10.22 02:34:10.959363 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1675_1675_0 to 202410_1688_1688_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:10.987225 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1003_368} MergeTreeSequentialSource: Reading 2 marks from part 202410_1003_1003_0, total 13 rows starting from the beginning of the part 2024.10.22 02:34:11.012724 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112632 2024.10.22 02:34:11.152036 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1583_120, 202410_1584_1584_0, 202410_1585_1585_0, 202410_1586_1586_0, 202410_1587_1587_0, 202410_1588_1588_0] 2024.10.22 02:34:11.157190 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1583_120, 202410_1584_1584_0, 202410_1585_1585_0, 202410_1586_1586_0, 202410_1587_1587_0, 202410_1588_1588_0] 2024.10.22 02:34:11.193303 [ 1418 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1583_120, 202410_1584_1584_0, 202410_1585_1585_0, 202410_1586_1586_0, 202410_1587_1587_0, 202410_1588_1588_0] 2024.10.22 02:34:11.392900 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1359ce57dbe5524ae47502b64b867402) : [0, 156]: Setting download as failed: File segment: [0, 156], key: 1359ce57dbe5524ae47502b64b867402, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:11.396112 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1359ce57dbe5524ae47502b64b867402): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 02:34:11.550759 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:34:11.552050 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:34:12.083160 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1003_368} MergeTask::MergeProjectionsStage: Merge sorted 15393 rows, containing 12 columns (12 merged, 0 gathered) in 1.435116449 sec., 10725.958866073872 rows/sec., 1.68 MiB/sec. 2024.10.22 02:34:12.263471 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1003_368} MergedBlockOutputStream: filled checksums 202410_1_1003_368 (state Temporary) 2024.10.22 02:34:12.387459 [ 1219 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:34:12.411115 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.16 MiB, will set to 1.56 GiB (RSS), difference: 1.47 MiB 2024.10.22 02:34:12.428568 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(a2d2da6a78bcf596c624c8000c6a29ba) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a2d2da6a78bcf596c624c8000c6a29ba, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:12.433107 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 161, file segment info: File segment: [0, 33554431], key: a2d2da6a78bcf596c624c8000c6a29ba, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:12.435656 [ 1219 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:34:12.436467 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:12.462464 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 158 entries to flush up to offset 110812 2024.10.22 02:34:12.423255 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 4 old parts to remove. Parts: [202410_747_920_36, 202410_921_921_0, 202410_922_922_0, 202410_923_923_0] 2024.10.22 02:34:12.509106 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from filesystem (serially): Parts: [202410_747_920_36, 202410_921_921_0, 202410_922_922_0, 202410_923_923_0] 2024.10.22 02:34:12.526968 [ 1423 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from memory: Parts: [202410_747_920_36, 202410_921_921_0, 202410_922_922_0, 202410_923_923_0] 2024.10.22 02:34:12.705770 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_943_308, 202410_944_944_0] 2024.10.22 02:34:12.705794 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1003_368} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1003_368 to 202410_1_1003_368 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:12.707514 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_943_308, 202410_944_944_0] 2024.10.22 02:34:12.709003 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1003_368} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1002_367, 202410_1003_1003_0] -> 202410_1_1003_368 2024.10.22 02:34:12.724856 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_943_308, 202410_944_944_0] 2024.10.22 02:34:12.737317 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:12.747546 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:12.748508 [ 1400 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:34:12.762790 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 824 2024.10.22 02:34:12.795679 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_977_977_0 (state Temporary) 2024.10.22 02:34:12.885022 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_977_977_0 to 202410_983_983_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:13.008136 [ 1162 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_977_77 to 202410_978_978_0 2024.10.22 02:34:13.016050 [ 1162 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.48 MiB using storage policy from min volume index 0 2024.10.22 02:34:13.038251 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 110812 2024.10.22 02:34:13.108062 [ 1162 ] {} DiskLocal: Reserved 1.48 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:13.155780 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:13.157678 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:13.158659 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_978_78} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_977_77 to 202410_978_978_0 into Wide with storage Full 2024.10.22 02:34:13.314081 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_978_78} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:13.334741 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_978_78} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_977_77, total 26324 rows starting from the beginning of the part 2024.10.22 02:34:13.353479 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_199_199_0 to 202410_202_202_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:13.422965 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_806_806_0 (state Temporary) 2024.10.22 02:34:13.494308 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ea11f1468a21a677ec23f9b574abe12a) : [0, 122661]: Setting download as failed: File segment: [0, 122661], key: ea11f1468a21a677ec23f9b574abe12a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:13.497831 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ea11f1468a21a677ec23f9b574abe12a): No space left in cache to reserve 122662 bytes, will continue without cache download 2024.10.22 02:34:13.523421 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_806_806_0 to 202410_816_816_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:13.550687 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ea11f1468a21a677ec23f9b574abe12a, offset: 0, size: 0 2024.10.22 02:34:13.555306 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ea11f1468a21a677ec23f9b574abe12a for removal 2024.10.22 02:34:13.559612 [ 1476 ] {} CacheMetadata: Key ea11f1468a21a677ec23f9b574abe12a is removed from metadata 2024.10.22 02:34:13.561527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1359ce57dbe5524ae47502b64b867402, offset: 0, size: 0 2024.10.22 02:34:13.562825 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1359ce57dbe5524ae47502b64b867402 for removal 2024.10.22 02:34:13.566373 [ 1476 ] {} CacheMetadata: Key 1359ce57dbe5524ae47502b64b867402 is removed from metadata 2024.10.22 02:34:13.576958 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1431552 2024.10.22 02:34:13.642031 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 824 2024.10.22 02:34:13.764024 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_978_78} MergeTreeSequentialSource: Reading 2 marks from part 202410_978_978_0, total 178 rows starting from the beginning of the part 2024.10.22 02:34:14.166942 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(7e5951cd6bd14584ea72f31726bbc0b1) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 7e5951cd6bd14584ea72f31726bbc0b1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:14.171847 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 359, file segment info: File segment: [0, 33554431], key: 7e5951cd6bd14584ea72f31726bbc0b1, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:14.177294 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:14.500673 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:14.508282 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:14.847283 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_953_953_0 (state Temporary) 2024.10.22 02:34:14.938006 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(acd1592c94030bb4a43f1ffe4d6af1d0) : [0, 168]: Setting download as failed: File segment: [0, 168], key: acd1592c94030bb4a43f1ffe4d6af1d0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:14.940699 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(acd1592c94030bb4a43f1ffe4d6af1d0): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 02:34:15.027184 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 82 entries to flush up to offset 112714 2024.10.22 02:34:15.027331 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_953_953_0 to 202410_960_960_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:15.075255 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1431552 2024.10.22 02:34:15.605070 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:15.615870 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:15.797091 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 8.90 MiB, will set to 1.57 GiB (RSS), difference: -31.52 MiB 2024.10.22 02:34:15.810727 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1676_1676_0 (state Temporary) 2024.10.22 02:34:16.027299 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1676_1676_0 to 202410_1689_1689_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:16.110849 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(d232407e6c578b2839996ef5d310225a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: d232407e6c578b2839996ef5d310225a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:16.116634 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 464, file segment info: File segment: [0, 33554431], key: d232407e6c578b2839996ef5d310225a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:16.123525 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:16.174160 [ 1747 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/jif/jtlqsmrhdoqjrgwzbygmyjerupfgu, size 359 2024.10.22 02:34:16.175962 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112714 2024.10.22 02:34:16.195009 [ 1425 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_978_78/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:34:16.254443 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:34:16.266481 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:34:16.320323 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:34:16.356383 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:34:16.395225 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:34:16.450822 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:34:16.516291 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParsedParams.Key1 2024.10.22 02:34:16.651361 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/wwk/rlbzrheubzimqgzwkhknkeenunegp (LogSeriesLimiter: on interval from 2024-10-22 02:34:03 to 2024-10-22 02:34:16 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:34:16.813333 [ 1398 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_978_78} MergeTask::MergeProjectionsStage: Merge sorted 26502 rows, containing 13 columns (13 merged, 0 gathered) in 3.66091067 sec., 7239.182375351432 rows/sec., 3.16 MiB/sec. 2024.10.22 02:34:17.136693 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_978_78} MergedBlockOutputStream: filled checksums 202410_747_978_78 (state Temporary) 2024.10.22 02:34:17.355606 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(168884de1a4e9919ac043c4319110fe1) : [0, 1057949]: Setting download as failed: File segment: [0, 1057949], key: 168884de1a4e9919ac043c4319110fe1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:17.357736 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(168884de1a4e9919ac043c4319110fe1): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:34:17.528946 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 8043 2024.10.22 02:34:17.537175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 168884de1a4e9919ac043c4319110fe1, offset: 0, size: 0 2024.10.22 02:34:17.539670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 168884de1a4e9919ac043c4319110fe1 for removal 2024.10.22 02:34:17.542574 [ 1476 ] {} CacheMetadata: Key 168884de1a4e9919ac043c4319110fe1 is removed from metadata 2024.10.22 02:34:17.545662 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: acd1592c94030bb4a43f1ffe4d6af1d0, offset: 0, size: 0 2024.10.22 02:34:17.547957 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key acd1592c94030bb4a43f1ffe4d6af1d0 for removal 2024.10.22 02:34:17.550189 [ 1476 ] {} CacheMetadata: Key acd1592c94030bb4a43f1ffe4d6af1d0 is removed from metadata 2024.10.22 02:34:18.076910 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 10 entries to flush up to offset 15380 2024.10.22 02:34:18.170925 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_978_78} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_978_78 to 202410_747_978_78 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:18.178604 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_978_78} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_977_77, 202410_978_978_0] -> 202410_747_978_78 2024.10.22 02:34:18.297193 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:18.304536 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.58 MiB. 2024.10.22 02:34:18.416108 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(bb00c57c03a402a90448595e81debda0) : [0, 156]: Setting download as failed: File segment: [0, 156], key: bb00c57c03a402a90448595e81debda0, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:18.420392 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(bb00c57c03a402a90448595e81debda0): No space left in cache to reserve 157 bytes, will continue without cache download 2024.10.22 02:34:18.475850 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_952_176/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:34:18.571041 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 14.75 MiB, will set to 1.56 GiB (RSS), difference: 40.49 MiB 2024.10.22 02:34:18.638463 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_763_155, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0, 202410_767_767_0, 202410_768_768_0] 2024.10.22 02:34:18.642867 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_763_155, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0, 202410_767_767_0, 202410_768_768_0] 2024.10.22 02:34:18.659227 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:18.710484 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_995_995_0 (state Temporary) 2024.10.22 02:34:18.764458 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_995_995_0 to 202410_1008_1008_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:18.788966 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_763_155, 202410_764_764_0, 202410_765_765_0, 202410_766_766_0, 202410_767_767_0, 202410_768_768_0] 2024.10.22 02:34:18.803749 [ 1196 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1003_368 to 202410_1004_1004_0 2024.10.22 02:34:18.813720 [ 1196 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:18.837083 [ 1196 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:18.852922 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1004_369} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1003_368 to 202410_1004_1004_0 into Compact with storage Full 2024.10.22 02:34:18.896251 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1004_369} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:18.905720 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1004_369} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1003_368, total 15393 rows starting from the beginning of the part 2024.10.22 02:34:18.921425 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15380 2024.10.22 02:34:19.030687 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1004_369} MergeTreeSequentialSource: Reading 2 marks from part 202410_1004_1004_0, total 20 rows starting from the beginning of the part 2024.10.22 02:34:19.102860 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ff160cdb893c8704e7b80d1c7c0d0585) : [0, 76151]: Setting download as failed: File segment: [0, 76151], key: ff160cdb893c8704e7b80d1c7c0d0585, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:19.105376 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ff160cdb893c8704e7b80d1c7c0d0585): No space left in cache to reserve 76152 bytes, will continue without cache download 2024.10.22 02:34:19.262114 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ff160cdb893c8704e7b80d1c7c0d0585, offset: 0, size: 0 2024.10.22 02:34:19.265820 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ff160cdb893c8704e7b80d1c7c0d0585 for removal 2024.10.22 02:34:19.271180 [ 1476 ] {} CacheMetadata: Key ff160cdb893c8704e7b80d1c7c0d0585 is removed from metadata 2024.10.22 02:34:19.271670 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: bb00c57c03a402a90448595e81debda0, offset: 0, size: 0 2024.10.22 02:34:19.273703 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key bb00c57c03a402a90448595e81debda0 for removal 2024.10.22 02:34:19.282225 [ 1476 ] {} CacheMetadata: Key bb00c57c03a402a90448595e81debda0 is removed from metadata 2024.10.22 02:34:19.403264 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_944_309, 202410_945_945_0] 2024.10.22 02:34:19.405217 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_944_309, 202410_945_945_0] 2024.10.22 02:34:19.418287 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_944_309, 202410_945_945_0] 2024.10.22 02:34:19.760791 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1004_369} MergeTask::MergeProjectionsStage: Merge sorted 15413 rows, containing 12 columns (12 merged, 0 gathered) in 0.911736688 sec., 16905.100126891022 rows/sec., 2.65 MiB/sec. 2024.10.22 02:34:19.809439 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1004_369} MergedBlockOutputStream: filled checksums 202410_1_1004_369 (state Temporary) 2024.10.22 02:34:19.893247 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1004_369} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1004_369 to 202410_1_1004_369 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:19.896963 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1004_369} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1003_368, 202410_1004_1004_0] -> 202410_1_1004_369 2024.10.22 02:34:19.933352 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:34:20.093725 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1501b8ceec253ece7a2c07bb40b91cfc) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 1501b8ceec253ece7a2c07bb40b91cfc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:20.097647 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1501b8ceec253ece7a2c07bb40b91cfc): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:34:20.186329 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 74 entries to flush up to offset 112788 2024.10.22 02:34:20.290746 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.07 MiB, will set to 1.56 GiB (RSS), difference: -5.70 MiB 2024.10.22 02:34:20.301413 [ 1201 ] {} DatabaseCatalog: Execution took 204 ms. 2024.10.22 02:34:20.339722 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:20.343898 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:20.402452 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1677_1677_0 (state Temporary) 2024.10.22 02:34:20.464842 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1677_1677_0 to 202410_1690_1690_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:20.516659 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112788 2024.10.22 02:34:20.556703 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 167 entries to flush up to offset 110979 2024.10.22 02:34:20.912822 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(c10347ef43441b06d509150481dc8c5a) : [0, 66549]: Setting download as failed: File segment: [0, 66549], key: c10347ef43441b06d509150481dc8c5a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:20.915865 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(c10347ef43441b06d509150481dc8c5a): No space left in cache to reserve 66550 bytes, will continue without cache download 2024.10.22 02:34:20.960174 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:20.962525 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:21.047361 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: c10347ef43441b06d509150481dc8c5a, offset: 0, size: 0 2024.10.22 02:34:21.049199 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key c10347ef43441b06d509150481dc8c5a for removal 2024.10.22 02:34:21.053454 [ 1476 ] {} CacheMetadata: Key c10347ef43441b06d509150481dc8c5a is removed from metadata 2024.10.22 02:34:21.053733 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1501b8ceec253ece7a2c07bb40b91cfc, offset: 0, size: 0 2024.10.22 02:34:21.055009 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1501b8ceec253ece7a2c07bb40b91cfc for removal 2024.10.22 02:34:21.058185 [ 1476 ] {} CacheMetadata: Key 1501b8ceec253ece7a2c07bb40b91cfc is removed from metadata 2024.10.22 02:34:21.150702 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_978_978_0 (state Temporary) 2024.10.22 02:34:21.228373 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_978_978_0 to 202410_984_984_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:21.273061 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 110979 2024.10.22 02:34:21.366540 [ 1235 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_978_78 to 202410_979_979_0 2024.10.22 02:34:21.393746 [ 1235 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.48 MiB using storage policy from min volume index 0 2024.10.22 02:34:21.397387 [ 1235 ] {} DiskLocal: Reserved 1.48 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:21.450628 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_979_79} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_978_78 to 202410_979_979_0 into Wide with storage Full 2024.10.22 02:34:21.551644 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:34:21.555360 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:34:21.639225 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_979_79} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:21.656055 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_979_79} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_978_78, total 26502 rows starting from the beginning of the part 2024.10.22 02:34:21.872145 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(cd64d2c7de90598eac8e0acf40cea3b9) : [0, 155]: Setting download as failed: File segment: [0, 155], key: cd64d2c7de90598eac8e0acf40cea3b9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:21.874502 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(cd64d2c7de90598eac8e0acf40cea3b9): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:34:22.051074 [ 1386 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_979_79} MergeTreeSequentialSource: Reading 2 marks from part 202410_979_979_0, total 177 rows starting from the beginning of the part 2024.10.22 02:34:22.084093 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1540 entries to flush up to offset 1433092 2024.10.22 02:34:22.611647 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.59 GiB, peak 2.00 GiB, free memory in arenas 12.58 MiB, will set to 1.55 GiB (RSS), difference: -38.80 MiB 2024.10.22 02:34:23.003678 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_952_176} MergeTask::MergeProjectionsStage: Merge sorted 1435900 rows, containing 4 columns (4 merged, 0 gathered) in 41.170443473 sec., 34876.96218141731 rows/sec., 523.00 KiB/sec. 2024.10.22 02:34:23.016469 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(25d3ed52702467ea88de954d405ff01b) : [0, 64120]: Setting download as failed: File segment: [0, 64120], key: 25d3ed52702467ea88de954d405ff01b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:23.022283 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(25d3ed52702467ea88de954d405ff01b): No space left in cache to reserve 64121 bytes, will continue without cache download 2024.10.22 02:34:23.046600 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 25d3ed52702467ea88de954d405ff01b, offset: 0, size: 0 2024.10.22 02:34:23.048791 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 25d3ed52702467ea88de954d405ff01b for removal 2024.10.22 02:34:23.050506 [ 1476 ] {} CacheMetadata: Key 25d3ed52702467ea88de954d405ff01b is removed from metadata 2024.10.22 02:34:23.051347 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: cd64d2c7de90598eac8e0acf40cea3b9, offset: 0, size: 0 2024.10.22 02:34:23.054844 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key cd64d2c7de90598eac8e0acf40cea3b9 for removal 2024.10.22 02:34:23.058738 [ 1476 ] {} CacheMetadata: Key cd64d2c7de90598eac8e0acf40cea3b9 is removed from metadata 2024.10.22 02:34:23.100609 [ 1398 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_952_176} MergedBlockOutputStream: filled checksums 202410_1_952_176 (state Temporary) 2024.10.22 02:34:23.249463 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:23.252407 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:23.678393 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_954_954_0 (state Temporary) 2024.10.22 02:34:23.751152 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(2e760fbc0fb08f066199075af8599aac) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 2e760fbc0fb08f066199075af8599aac, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 02:34:23.762548 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 487824, file segment info: File segment: [0, 33554431], key: 2e760fbc0fb08f066199075af8599aac, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 02:34:23.765753 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:23.877011 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_954_954_0 to 202410_961_961_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:24.026210 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1433092 2024.10.22 02:34:24.164357 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_979_79} MergeTask::MergeProjectionsStage: Merge sorted 26679 rows, containing 13 columns (13 merged, 0 gathered) in 2.722869878 sec., 9798.11786657842 rows/sec., 4.27 MiB/sec. 2024.10.22 02:34:24.339551 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_979_79} MergedBlockOutputStream: filled checksums 202410_747_979_79 (state Temporary) 2024.10.22 02:34:24.539726 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 55 entries to flush up to offset 112843 2024.10.22 02:34:24.593565 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(305f2cc2cdc2ee15e9aca8d0572a28fe) : [0, 167]: Setting download as failed: File segment: [0, 167], key: 305f2cc2cdc2ee15e9aca8d0572a28fe, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:24.596785 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(305f2cc2cdc2ee15e9aca8d0572a28fe): No space left in cache to reserve 168 bytes, will continue without cache download 2024.10.22 02:34:24.742915 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_979_79} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_979_79 to 202410_747_979_79 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:24.746997 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_979_79} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_978_78, 202410_979_979_0] -> 202410_747_979_79 2024.10.22 02:34:24.762294 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.58 MiB. 2024.10.22 02:34:24.852525 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:24.854939 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:24.939814 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1678_1678_0 (state Temporary) 2024.10.22 02:34:25.090506 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1678_1678_0 to 202410_1691_1691_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:25.195661 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112843 2024.10.22 02:34:25.231870 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 41 entries to flush up to offset 8084 2024.10.22 02:34:25.346478 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1686_12 to 202410_1691_1691_0 2024.10.22 02:34:25.358701 [ 1243 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:25.360601 [ 1243 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:25.363547 [ 1243 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 259 ms. 2024.10.22 02:34:25.374737 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1686_12 to 202410_1691_1691_0 into Compact with storage Full 2024.10.22 02:34:25.448455 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:25.469484 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1686_12, total 4229 rows starting from the beginning of the part 2024.10.22 02:34:25.681366 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.51 GiB, peak 2.00 GiB, free memory in arenas 11.29 MiB, will set to 1.55 GiB (RSS), difference: 46.55 MiB 2024.10.22 02:34:25.736073 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1687_1687_0, total 96 rows starting from the beginning of the part 2024.10.22 02:34:25.782339 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1688_1688_0, total 64 rows starting from the beginning of the part 2024.10.22 02:34:25.846482 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(86ffdd7998302b8749933b526cf2a343) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 86ffdd7998302b8749933b526cf2a343, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 02:34:25.848704 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1154, file segment info: File segment: [0, 33554431], key: 86ffdd7998302b8749933b526cf2a343, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1390, kind: Regular, unbound: 0 2024.10.22 02:34:25.850894 [ 1390 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:25.854600 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1689_1689_0, total 82 rows starting from the beginning of the part 2024.10.22 02:34:25.924428 [ 1742 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/lvr/evdqmzaxpadfwyckqxzyufdsiiuiz, size 487824 2024.10.22 02:34:25.940113 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1690_1690_0, total 74 rows starting from the beginning of the part 2024.10.22 02:34:26.050902 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column Params 2024.10.22 02:34:26.060655 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column Params 2024.10.22 02:34:26.067392 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_899_899_0, 202410_900_900_0, 202410_901_901_0, 202410_902_902_0, 202410_903_903_0, 202410_904_904_0] 2024.10.22 02:34:26.068370 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column Params 2024.10.22 02:34:26.077571 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_899_899_0, 202410_900_900_0, 202410_901_901_0, 202410_902_902_0, 202410_903_903_0, 202410_904_904_0] 2024.10.22 02:34:26.128020 [ 1425 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_899_899_0, 202410_900_900_0, 202410_901_901_0, 202410_902_902_0, 202410_903_903_0, 202410_904_904_0] 2024.10.22 02:34:26.131945 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column Params 2024.10.22 02:34:26.159922 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column Params 2024.10.22 02:34:26.162555 [ 1387 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} MergeTreeSequentialSource: Reading 2 marks from part 202410_1691_1691_0, total 55 rows starting from the beginning of the part 2024.10.22 02:34:26.197722 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column Params 2024.10.22 02:34:26.219076 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column Params 2024.10.22 02:34:26.240321 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column Params 2024.10.22 02:34:26.423561 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 15394 2024.10.22 02:34:26.464960 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column Params 2024.10.22 02:34:26.557520 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:26.560947 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:26.608871 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_996_996_0 (state Temporary) 2024.10.22 02:34:26.635359 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/chx/owxelszvxfgnevwziaeydivgajmfq (LogSeriesLimiter: on interval from 2024-10-22 02:34:16 to 2024-10-22 02:34:26 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:34:26.801003 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_952_176} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_952_176 to 202410_1_952_176 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:26.881135 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_996_996_0 to 202410_1009_1009_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:26.886157 [ 1399 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_952_176} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_946_175, 202410_947_952_1] -> 202410_1_952_176 2024.10.22 02:34:26.913336 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:34:26.915323 [ 1271 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1004_369 to 202410_1005_1005_0 2024.10.22 02:34:26.923588 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(63e9131997180332d600539e678c2f44) : [0, 77738]: Setting download as failed: File segment: [0, 77738], key: 63e9131997180332d600539e678c2f44, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:26.926448 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(63e9131997180332d600539e678c2f44): No space left in cache to reserve 77739 bytes, will continue without cache download 2024.10.22 02:34:26.928523 [ 1271 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:26.938303 [ 1271 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:26.956499 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 63e9131997180332d600539e678c2f44, offset: 0, size: 0 2024.10.22 02:34:26.958362 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1005_370} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1004_369 to 202410_1005_1005_0 into Compact with storage Full 2024.10.22 02:34:26.958600 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 63e9131997180332d600539e678c2f44 for removal 2024.10.22 02:34:26.961324 [ 1476 ] {} CacheMetadata: Key 63e9131997180332d600539e678c2f44 is removed from metadata 2024.10.22 02:34:26.964146 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 305f2cc2cdc2ee15e9aca8d0572a28fe, offset: 0, size: 0 2024.10.22 02:34:26.966134 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 305f2cc2cdc2ee15e9aca8d0572a28fe for removal 2024.10.22 02:34:26.970474 [ 1476 ] {} CacheMetadata: Key 305f2cc2cdc2ee15e9aca8d0572a28fe is removed from metadata 2024.10.22 02:34:27.128920 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1005_370} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:27.143772 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15394 2024.10.22 02:34:27.174102 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_945_310, 202410_946_946_0] 2024.10.22 02:34:27.176428 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_945_310, 202410_946_946_0] 2024.10.22 02:34:27.225303 [ 1424 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_945_310, 202410_946_946_0] 2024.10.22 02:34:27.249154 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1005_370} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1004_369, total 15413 rows starting from the beginning of the part 2024.10.22 02:34:27.286395 [ 1386 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1005_370} MergeTreeSequentialSource: Reading 2 marks from part 202410_1005_1005_0, total 19 rows starting from the beginning of the part 2024.10.22 02:34:27.543156 [ 1260 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:34:27.559608 [ 1260 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:34:27.746003 [ 1397 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} MergeTask::MergeProjectionsStage: Merge sorted 4600 rows, containing 13 columns (13 merged, 0 gathered) in 2.374859954 sec., 1936.956321256828 rows/sec., 342.41 KiB/sec. 2024.10.22 02:34:27.917569 [ 1390 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} MergedBlockOutputStream: filled checksums 202410_1626_1691_13 (state Temporary) 2024.10.22 02:34:28.071022 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1005_370} MergeTask::MergeProjectionsStage: Merge sorted 15432 rows, containing 12 columns (12 merged, 0 gathered) in 1.117291586 sec., 13811.97190900532 rows/sec., 2.16 MiB/sec. 2024.10.22 02:34:28.154413 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1005_370} MergedBlockOutputStream: filled checksums 202410_1_1005_370 (state Temporary) 2024.10.22 02:34:28.165004 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(61fc03df8642f6aae0aac9f788f330e9) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 61fc03df8642f6aae0aac9f788f330e9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:28.166727 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(61fc03df8642f6aae0aac9f788f330e9): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:34:28.400626 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1691_13 to 202410_1626_1691_13 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:28.507720 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1691_13} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1686_12, 202410_1691_1691_0] -> 202410_1626_1691_13 2024.10.22 02:34:28.544267 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:34:28.551608 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1005_370} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1005_370 to 202410_1_1005_370 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:28.565422 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1005_370} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1004_369, 202410_1005_1005_0] -> 202410_1_1005_370 2024.10.22 02:34:28.593085 [ 1398 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:34:28.646212 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 825 2024.10.22 02:34:29.020501 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 174 entries to flush up to offset 111153 2024.10.22 02:34:29.060228 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:29.063124 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:29.149466 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_807_807_0 (state Temporary) 2024.10.22 02:34:29.211845 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 88 entries to flush up to offset 112931 2024.10.22 02:34:29.280249 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:29.289224 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:29.324819 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_807_807_0 to 202410_817_817_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:29.345119 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_979_979_0 (state Temporary) 2024.10.22 02:34:29.375485 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(895b1cd6bf09f74cdd5029dd07cce7bf) : [0, 32152]: Setting download as failed: File segment: [0, 32152], key: 895b1cd6bf09f74cdd5029dd07cce7bf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:29.378535 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(895b1cd6bf09f74cdd5029dd07cce7bf): No space left in cache to reserve 32153 bytes, will continue without cache download 2024.10.22 02:34:29.403301 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_979_979_0 to 202410_985_985_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:29.451822 [ 1153 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_979_79 to 202410_980_980_0 2024.10.22 02:34:29.463302 [ 1153 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.49 MiB using storage policy from min volume index 0 2024.10.22 02:34:29.467192 [ 1153 ] {} DiskLocal: Reserved 1.49 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:29.478378 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_980_80} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_979_79 to 202410_980_980_0 into Wide with storage Full 2024.10.22 02:34:29.480675 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 111153 2024.10.22 02:34:29.508058 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 825 2024.10.22 02:34:29.548060 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_980_80} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:29.556665 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_980_80} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_979_79, total 26679 rows starting from the beginning of the part 2024.10.22 02:34:29.609614 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 895b1cd6bf09f74cdd5029dd07cce7bf, offset: 0, size: 0 2024.10.22 02:34:29.635091 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 895b1cd6bf09f74cdd5029dd07cce7bf for removal 2024.10.22 02:34:29.640522 [ 1476 ] {} CacheMetadata: Key 895b1cd6bf09f74cdd5029dd07cce7bf is removed from metadata 2024.10.22 02:34:29.642918 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 61fc03df8642f6aae0aac9f788f330e9, offset: 0, size: 0 2024.10.22 02:34:29.644767 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 61fc03df8642f6aae0aac9f788f330e9 for removal 2024.10.22 02:34:29.646680 [ 1476 ] {} CacheMetadata: Key 61fc03df8642f6aae0aac9f788f330e9 is removed from metadata 2024.10.22 02:34:29.651438 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:29.654793 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:29.673821 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_980_80} MergeTreeSequentialSource: Reading 2 marks from part 202410_980_980_0, total 204 rows starting from the beginning of the part 2024.10.22 02:34:29.738837 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1679_1679_0 (state Temporary) 2024.10.22 02:34:29.775061 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1679_1679_0 to 202410_1692_1692_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:29.829637 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112931 2024.10.22 02:34:31.039441 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1434247 2024.10.22 02:34:31.226175 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d18f47204940cdfb45ba3c26fc1e18dc) : [0, 155]: Setting download as failed: File segment: [0, 155], key: d18f47204940cdfb45ba3c26fc1e18dc, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:31.231083 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d18f47204940cdfb45ba3c26fc1e18dc): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:34:31.527494 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 4.88 MiB, will set to 1.57 GiB (RSS), difference: -40.22 MiB 2024.10.22 02:34:31.551604 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:34:31.553691 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:34:31.555139 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:31.556540 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:31.773596 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_955_955_0 (state Temporary) 2024.10.22 02:34:31.850448 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_955_955_0 to 202410_962_962_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:31.867901 [ 1203 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_952_176 to 202410_953_958_1 2024.10.22 02:34:31.868802 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1434247 2024.10.22 02:34:31.871640 [ 1203 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.14 MiB using storage policy from min volume index 0 2024.10.22 02:34:31.873459 [ 1203 ] {} DiskLocal: Reserved 1.14 MiB on local disk `default`, having unreserved 296.16 GiB. 2024.10.22 02:34:31.901188 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_958_177} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_952_176 to 202410_953_958_1 into Wide with storage Full 2024.10.22 02:34:31.955553 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8fa8a2d49b085dced6d9bbf6e102d207) : [0, 216398]: Setting download as failed: File segment: [0, 216398], key: 8fa8a2d49b085dced6d9bbf6e102d207, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:31.957370 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8fa8a2d49b085dced6d9bbf6e102d207): No space left in cache to reserve 216399 bytes, will continue without cache download 2024.10.22 02:34:31.963321 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_958_177} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:32.063316 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_958_177} MergeTreeSequentialSource: Reading 177 marks from part 202410_1_952_176, total 1435900 rows starting from the beginning of the part 2024.10.22 02:34:32.071550 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_980_80/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:34:32.074348 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8fa8a2d49b085dced6d9bbf6e102d207, offset: 0, size: 0 2024.10.22 02:34:32.078155 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8fa8a2d49b085dced6d9bbf6e102d207 for removal 2024.10.22 02:34:32.083559 [ 1476 ] {} CacheMetadata: Key 8fa8a2d49b085dced6d9bbf6e102d207 is removed from metadata 2024.10.22 02:34:32.087011 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d18f47204940cdfb45ba3c26fc1e18dc, offset: 0, size: 0 2024.10.22 02:34:32.089695 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d18f47204940cdfb45ba3c26fc1e18dc for removal 2024.10.22 02:34:32.093168 [ 1476 ] {} CacheMetadata: Key d18f47204940cdfb45ba3c26fc1e18dc is removed from metadata 2024.10.22 02:34:32.132414 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_958_177} MergeTreeSequentialSource: Reading 2 marks from part 202410_953_958_1, total 7317 rows starting from the beginning of the part 2024.10.22 02:34:32.245754 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_980_80} MergeTask::MergeProjectionsStage: Merge sorted 26883 rows, containing 13 columns (13 merged, 0 gathered) in 2.772652636 sec., 9695.769189025812 rows/sec., 4.23 MiB/sec. 2024.10.22 02:34:32.319637 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_980_80} MergedBlockOutputStream: filled checksums 202410_747_980_80 (state Temporary) 2024.10.22 02:34:32.669467 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(abfe6137a345c6c4d92a73766c5abf2e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: abfe6137a345c6c4d92a73766c5abf2e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:34:32.673621 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: abfe6137a345c6c4d92a73766c5abf2e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:34:32.676915 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:33.175983 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_980_80} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_980_80 to 202410_747_980_80 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:33.181783 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_980_80} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_979_79, 202410_980_980_0] -> 202410_747_980_80 2024.10.22 02:34:33.207769 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.89 MiB. 2024.10.22 02:34:33.570995 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(321aa87243ab826b1b631b6d1d10fe32) : [0, 158]: Setting download as failed: File segment: [0, 158], key: 321aa87243ab826b1b631b6d1d10fe32, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:33.573063 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(321aa87243ab826b1b631b6d1d10fe32): No space left in cache to reserve 159 bytes, will continue without cache download 2024.10.22 02:34:33.575208 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 6 old parts to remove. Parts: [202410_1207_1588_121, 202410_1589_1589_0, 202410_1590_1590_0, 202410_1591_1591_0, 202410_1592_1592_0, 202410_1593_1593_0] 2024.10.22 02:34:33.577533 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from filesystem (serially): Parts: [202410_1207_1588_121, 202410_1589_1589_0, 202410_1590_1590_0, 202410_1591_1591_0, 202410_1592_1592_0, 202410_1593_1593_0] 2024.10.22 02:34:33.629127 [ 1419 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 6 parts from memory: Parts: [202410_1207_1588_121, 202410_1589_1589_0, 202410_1590_1590_0, 202410_1591_1591_0, 202410_1592_1592_0, 202410_1593_1593_0] 2024.10.22 02:34:33.844071 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 112999 2024.10.22 02:34:34.255533 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:34.257634 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:34.417820 [ 1423 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_958_177/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:34:34.426568 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1680_1680_0 (state Temporary) 2024.10.22 02:34:34.647508 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 21 entries to flush up to offset 15415 2024.10.22 02:34:34.706971 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1680_1680_0 to 202410_1693_1693_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:34.752306 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 112999 2024.10.22 02:34:34.857478 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:34.861092 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:34.967549 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(ada27d35a1b409d3a9b3670c60668366) : [0, 346477]: Setting download as failed: File segment: [0, 346477], key: ada27d35a1b409d3a9b3670c60668366, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:34.980594 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(ada27d35a1b409d3a9b3670c60668366): No space left in cache to reserve 346478 bytes, will continue without cache download 2024.10.22 02:34:35.019174 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_997_997_0 (state Temporary) 2024.10.22 02:34:35.028775 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: ada27d35a1b409d3a9b3670c60668366, offset: 0, size: 0 2024.10.22 02:34:35.034803 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key ada27d35a1b409d3a9b3670c60668366 for removal 2024.10.22 02:34:35.039584 [ 1476 ] {} CacheMetadata: Key ada27d35a1b409d3a9b3670c60668366 is removed from metadata 2024.10.22 02:34:35.043158 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 321aa87243ab826b1b631b6d1d10fe32, offset: 0, size: 0 2024.10.22 02:34:35.044596 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 321aa87243ab826b1b631b6d1d10fe32 for removal 2024.10.22 02:34:35.049203 [ 1476 ] {} CacheMetadata: Key 321aa87243ab826b1b631b6d1d10fe32 is removed from metadata 2024.10.22 02:34:35.149431 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_997_997_0 to 202410_1010_1010_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:35.177350 [ 1240 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1005_370 to 202410_1006_1006_0 2024.10.22 02:34:35.185289 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15415 2024.10.22 02:34:35.186793 [ 1240 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:35.193084 [ 1240 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:35.245110 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1006_371} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1005_370 to 202410_1006_1006_0 into Compact with storage Full 2024.10.22 02:34:35.295042 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1006_371} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:35.413281 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1006_371} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1005_370, total 15432 rows starting from the beginning of the part 2024.10.22 02:34:35.449158 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1006_371} MergeTreeSequentialSource: Reading 2 marks from part 202410_1006_1006_0, total 26 rows starting from the beginning of the part 2024.10.22 02:34:35.462608 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.53 GiB, peak 2.00 GiB, free memory in arenas 8.24 MiB, will set to 1.56 GiB (RSS), difference: 24.86 MiB 2024.10.22 02:34:36.355182 [ 1385 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1006_371} MergeTask::MergeProjectionsStage: Merge sorted 15458 rows, containing 12 columns (12 merged, 0 gathered) in 1.123979456 sec., 13752.9204092499 rows/sec., 2.15 MiB/sec. 2024.10.22 02:34:36.424692 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(57c8c15124685782658fe76bc084939f) : [0, 150]: Setting download as failed: File segment: [0, 150], key: 57c8c15124685782658fe76bc084939f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:36.430784 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(57c8c15124685782658fe76bc084939f): No space left in cache to reserve 151 bytes, will continue without cache download 2024.10.22 02:34:36.493219 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1006_371} MergedBlockOutputStream: filled checksums 202410_1_1006_371 (state Temporary) 2024.10.22 02:34:36.662137 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_946_311, 202410_947_947_0] 2024.10.22 02:34:36.664501 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_946_311, 202410_947_947_0] 2024.10.22 02:34:36.674575 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_946_311, 202410_947_947_0] 2024.10.22 02:34:36.999949 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1006_371} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1006_371 to 202410_1_1006_371 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:37.006580 [ 1387 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1006_371} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1005_370, 202410_1006_1006_0] -> 202410_1_1006_371 2024.10.22 02:34:37.015421 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 826 2024.10.22 02:34:37.037267 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 183 entries to flush up to offset 111336 2024.10.22 02:34:37.139217 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:34:37.351427 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:37.353487 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:37.417056 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:37.423624 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:37.501714 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_980_980_0 (state Temporary) 2024.10.22 02:34:37.666468 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_980_980_0 to 202410_986_986_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:37.677703 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_808_808_0 (state Temporary) 2024.10.22 02:34:37.712220 [ 1169 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_980_80 to 202410_981_981_0 2024.10.22 02:34:37.723587 [ 1169 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.50 MiB using storage policy from min volume index 0 2024.10.22 02:34:37.731428 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 111336 2024.10.22 02:34:37.732449 [ 1169 ] {} DiskLocal: Reserved 1.50 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:37.759421 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_981_81} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_980_80 to 202410_981_981_0 into Wide with storage Full 2024.10.22 02:34:37.763993 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_808_808_0 to 202410_818_818_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:37.823012 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_981_81} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:37.837190 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_981_81} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_980_80, total 26883 rows starting from the beginning of the part 2024.10.22 02:34:37.928695 [ 1189 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_813_165 to 202410_818_818_0 2024.10.22 02:34:37.951469 [ 1189 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:37.954822 [ 1189 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:37.963156 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 826 2024.10.22 02:34:37.977161 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(6e78a517e37048d734e172e5cba976b3) : [0, 5127]: Setting download as failed: File segment: [0, 5127], key: 6e78a517e37048d734e172e5cba976b3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:37.983205 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(6e78a517e37048d734e172e5cba976b3): No space left in cache to reserve 5128 bytes, will continue without cache download 2024.10.22 02:34:37.992698 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_813_165 to 202410_818_818_0 into Compact with storage Full 2024.10.22 02:34:38.110487 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 6e78a517e37048d734e172e5cba976b3, offset: 0, size: 0 2024.10.22 02:34:38.109468 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_981_81} MergeTreeSequentialSource: Reading 2 marks from part 202410_981_981_0, total 212 rows starting from the beginning of the part 2024.10.22 02:34:38.113091 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 6e78a517e37048d734e172e5cba976b3 for removal 2024.10.22 02:34:38.120268 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 57c8c15124685782658fe76bc084939f, offset: 0, size: 0 2024.10.22 02:34:38.124499 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 57c8c15124685782658fe76bc084939f for removal 2024.10.22 02:34:38.138850 [ 1476 ] {} CacheMetadata: Key 6e78a517e37048d734e172e5cba976b3 is removed from metadata 2024.10.22 02:34:38.154538 [ 1476 ] {} CacheMetadata: Key 57c8c15124685782658fe76bc084939f is removed from metadata 2024.10.22 02:34:38.224087 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:38.269806 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_813_165, total 857 rows starting from the beginning of the part 2024.10.22 02:34:38.538172 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} MergeTreeSequentialSource: Reading 2 marks from part 202410_814_814_0, total 1 rows starting from the beginning of the part 2024.10.22 02:34:38.595713 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.58 GiB, peak 2.00 GiB, free memory in arenas 12.21 MiB, will set to 1.56 GiB (RSS), difference: -22.06 MiB 2024.10.22 02:34:38.713400 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} MergeTreeSequentialSource: Reading 2 marks from part 202410_815_815_0, total 1 rows starting from the beginning of the part 2024.10.22 02:34:38.867340 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 75 entries to flush up to offset 113074 2024.10.22 02:34:38.872820 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1435017 2024.10.22 02:34:39.016630 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} MergeTreeSequentialSource: Reading 2 marks from part 202410_816_816_0, total 1 rows starting from the beginning of the part 2024.10.22 02:34:39.056591 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:39.060562 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:39.150453 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1681_1681_0 (state Temporary) 2024.10.22 02:34:39.218972 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3ea54ff78c9176fe81d0aed511aa5ef5) : [0, 168]: Setting download as failed: File segment: [0, 168], key: 3ea54ff78c9176fe81d0aed511aa5ef5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:39.222960 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3ea54ff78c9176fe81d0aed511aa5ef5): No space left in cache to reserve 169 bytes, will continue without cache download 2024.10.22 02:34:39.224498 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1681_1681_0 to 202410_1694_1694_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:39.271280 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 113074 2024.10.22 02:34:39.310797 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} MergeTreeSequentialSource: Reading 2 marks from part 202410_817_817_0, total 1 rows starting from the beginning of the part 2024.10.22 02:34:39.536114 [ 1388 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} MergeTreeSequentialSource: Reading 2 marks from part 202410_818_818_0, total 1 rows starting from the beginning of the part 2024.10.22 02:34:39.739616 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:39.743055 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:39.773438 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 4 old parts to remove. Parts: [202410_747_923_37, 202410_924_924_0, 202410_925_925_0, 202410_926_926_0] 2024.10.22 02:34:39.776158 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from filesystem (serially): Parts: [202410_747_923_37, 202410_924_924_0, 202410_925_925_0, 202410_926_926_0] 2024.10.22 02:34:39.799080 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 4 parts from memory: Parts: [202410_747_923_37, 202410_924_924_0, 202410_925_925_0, 202410_926_926_0] 2024.10.22 02:34:39.996332 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_956_956_0 (state Temporary) 2024.10.22 02:34:40.230695 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_956_956_0 to 202410_963_963_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:40.267788 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1435017 2024.10.22 02:34:40.273218 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b68d4dc702d15b1d5dc993cfd6cf2e10) : [0, 163702]: Setting download as failed: File segment: [0, 163702], key: b68d4dc702d15b1d5dc993cfd6cf2e10, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:40.274282 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b68d4dc702d15b1d5dc993cfd6cf2e10): No space left in cache to reserve 163703 bytes, will continue without cache download 2024.10.22 02:34:40.299229 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b68d4dc702d15b1d5dc993cfd6cf2e10, offset: 0, size: 0 2024.10.22 02:34:40.302470 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b68d4dc702d15b1d5dc993cfd6cf2e10 for removal 2024.10.22 02:34:40.308386 [ 1476 ] {} CacheMetadata: Key b68d4dc702d15b1d5dc993cfd6cf2e10 is removed from metadata 2024.10.22 02:34:40.310786 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3ea54ff78c9176fe81d0aed511aa5ef5, offset: 0, size: 0 2024.10.22 02:34:40.314778 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3ea54ff78c9176fe81d0aed511aa5ef5 for removal 2024.10.22 02:34:40.321381 [ 1476 ] {} CacheMetadata: Key 3ea54ff78c9176fe81d0aed511aa5ef5 is removed from metadata 2024.10.22 02:34:40.339238 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_981_81} MergeTask::MergeProjectionsStage: Merge sorted 27095 rows, containing 13 columns (13 merged, 0 gathered) in 2.59581086 sec., 10437.971586265727 rows/sec., 4.55 MiB/sec. 2024.10.22 02:34:40.545530 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_981_81} MergedBlockOutputStream: filled checksums 202410_747_981_81 (state Temporary) 2024.10.22 02:34:40.571916 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.27 MiB, will set to 1.56 GiB (RSS), difference: -7.62 MiB 2024.10.22 02:34:40.869303 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:40.871297 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:41.190438 [ 1193 ] {} DatabaseCatalog: Execution took 739 ms. 2024.10.22 02:34:41.480310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3a21461790b260fe525a3d92439a6489) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 3a21461790b260fe525a3d92439a6489, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:41.487429 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3a21461790b260fe525a3d92439a6489): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 02:34:41.552839 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:34:41.555153 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:34:41.563227 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} MergeTask::MergeProjectionsStage: Merge sorted 862 rows, containing 32 columns (32 merged, 0 gathered) in 3.589991008 sec., 240.11202202988915 rows/sec., 18.02 KiB/sec. 2024.10.22 02:34:41.631224 [ 1385 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} MergedBlockOutputStream: filled checksums 202410_1_818_166 (state Temporary) 2024.10.22 02:34:41.639150 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_981_81} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_981_81 to 202410_747_981_81 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:41.643908 [ 1388 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_981_81} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_980_80, 202410_981_981_0] -> 202410_747_981_81 2024.10.22 02:34:41.673310 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 59.95 MiB. 2024.10.22 02:34:42.562898 [ 1207 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:34:42.688799 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 12 entries to flush up to offset 15427 2024.10.22 02:34:42.712083 [ 1207 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:34:42.796704 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:42.809979 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:42.827645 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8c4b68a557e032eb972e17d48471c7cf) : [0, 293645]: Setting download as failed: File segment: [0, 293645], key: 8c4b68a557e032eb972e17d48471c7cf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:42.833523 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8c4b68a557e032eb972e17d48471c7cf): No space left in cache to reserve 293646 bytes, will continue without cache download 2024.10.22 02:34:42.863016 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8c4b68a557e032eb972e17d48471c7cf, offset: 0, size: 0 2024.10.22 02:34:42.864619 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8c4b68a557e032eb972e17d48471c7cf for removal 2024.10.22 02:34:42.866458 [ 1476 ] {} CacheMetadata: Key 8c4b68a557e032eb972e17d48471c7cf is removed from metadata 2024.10.22 02:34:42.868525 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3a21461790b260fe525a3d92439a6489, offset: 0, size: 0 2024.10.22 02:34:42.870139 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3a21461790b260fe525a3d92439a6489 for removal 2024.10.22 02:34:42.872460 [ 1476 ] {} CacheMetadata: Key 3a21461790b260fe525a3d92439a6489 is removed from metadata 2024.10.22 02:34:42.951701 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_merge_202410_1_818_166 to 202410_1_818_166 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:43.045824 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_998_998_0 (state Temporary) 2024.10.22 02:34:43.063366 [ 1393 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_818_166} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Merged 6 parts: [202410_1_813_165, 202410_818_818_0] -> 202410_1_818_166 2024.10.22 02:34:43.096494 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_998_998_0 to 202410_1011_1011_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:43.110561 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.26 MiB. 2024.10.22 02:34:43.140564 [ 1261 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1006_371 to 202410_1007_1007_0 2024.10.22 02:34:43.155132 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15427 2024.10.22 02:34:43.167947 [ 1261 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:43.172239 [ 1261 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:43.194639 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1007_372} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1006_371 to 202410_1007_1007_0 into Compact with storage Full 2024.10.22 02:34:43.292338 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 66 entries to flush up to offset 113140 2024.10.22 02:34:43.341143 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1007_372} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:43.347955 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1007_372} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1006_371, total 15458 rows starting from the beginning of the part 2024.10.22 02:34:43.363802 [ 1394 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1007_372} MergeTreeSequentialSource: Reading 2 marks from part 202410_1007_1007_0, total 18 rows starting from the beginning of the part 2024.10.22 02:34:43.418730 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 13.95 MiB, will set to 1.56 GiB (RSS), difference: 39.88 MiB 2024.10.22 02:34:43.515528 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:43.519310 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:43.568335 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1682_1682_0 (state Temporary) 2024.10.22 02:34:43.654600 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1682_1682_0 to 202410_1695_1695_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:43.738298 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 113140 2024.10.22 02:34:43.750522 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(956ad36f82eaf013184def68d24076de) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 956ad36f82eaf013184def68d24076de, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:43.754242 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 289760, file segment info: File segment: [0, 33554431], key: 956ad36f82eaf013184def68d24076de, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:43.760011 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:43.831017 [ 1221 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 228 ms. 2024.10.22 02:34:44.288634 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_200_200_0 (state Temporary) 2024.10.22 02:34:44.290736 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1007_372} MergeTask::MergeProjectionsStage: Merge sorted 15476 rows, containing 12 columns (12 merged, 0 gathered) in 1.110397488 sec., 13937.351414469338 rows/sec., 2.18 MiB/sec. 2024.10.22 02:34:44.292660 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_1007_372/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:34:44.337778 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(17efd384242d83fe335d810153aa4cd5) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 17efd384242d83fe335d810153aa4cd5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:44.341692 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(17efd384242d83fe335d810153aa4cd5): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 02:34:44.462255 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1007_372} MergedBlockOutputStream: filled checksums 202410_1_1007_372 (state Temporary) 2024.10.22 02:34:44.496986 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_947_312, 202410_948_948_0] 2024.10.22 02:34:44.502262 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_947_312, 202410_948_948_0] 2024.10.22 02:34:44.525247 [ 1423 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_947_312, 202410_948_948_0] 2024.10.22 02:34:44.555633 [ 1239 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 02:34:45.197438 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(b97e59c74354a075b44100aeb642e569) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b97e59c74354a075b44100aeb642e569, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:45.204513 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 574, file segment info: File segment: [0, 33554431], key: b97e59c74354a075b44100aeb642e569, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:45.208223 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:45.222448 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1007_372} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1007_372 to 202410_1_1007_372 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:45.233470 [ 1396 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1007_372} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1006_371, 202410_1007_1007_0] -> 202410_1_1007_372 2024.10.22 02:34:45.253489 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:34:45.258794 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 175 entries to flush up to offset 111511 2024.10.22 02:34:45.476440 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 827 2024.10.22 02:34:45.672950 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:45.675151 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:45.843840 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_981_981_0 (state Temporary) 2024.10.22 02:34:45.895708 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(687b9ca21bfed47d8a951da05ad32367) : [0, 287444]: Setting download as failed: File segment: [0, 287444], key: 687b9ca21bfed47d8a951da05ad32367, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:45.897810 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(687b9ca21bfed47d8a951da05ad32367): No space left in cache to reserve 287445 bytes, will continue without cache download 2024.10.22 02:34:46.066780 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:46.069533 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:46.070404 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_958_177/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:34:46.108920 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_981_981_0 to 202410_987_987_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:46.154300 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 687b9ca21bfed47d8a951da05ad32367, offset: 0, size: 0 2024.10.22 02:34:46.156864 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 687b9ca21bfed47d8a951da05ad32367 for removal 2024.10.22 02:34:46.160289 [ 1476 ] {} CacheMetadata: Key 687b9ca21bfed47d8a951da05ad32367 is removed from metadata 2024.10.22 02:34:46.161522 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 17efd384242d83fe335d810153aa4cd5, offset: 0, size: 0 2024.10.22 02:34:46.163468 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 17efd384242d83fe335d810153aa4cd5 for removal 2024.10.22 02:34:46.239940 [ 1476 ] {} CacheMetadata: Key 17efd384242d83fe335d810153aa4cd5 is removed from metadata 2024.10.22 02:34:46.250772 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_981_81 to 202410_982_982_0 2024.10.22 02:34:46.258307 [ 1250 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.51 MiB using storage policy from min volume index 0 2024.10.22 02:34:46.259623 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 111511 2024.10.22 02:34:46.260763 [ 1250 ] {} DiskLocal: Reserved 1.51 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:46.306926 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_982_82} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_981_81 to 202410_982_982_0 into Wide with storage Full 2024.10.22 02:34:46.345266 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_982_82} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:46.352466 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_982_82} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_981_81, total 27095 rows starting from the beginning of the part 2024.10.22 02:34:46.377526 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_809_809_0 (state Temporary) 2024.10.22 02:34:46.391413 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 12.70 MiB, will set to 1.56 GiB (RSS), difference: -10.89 MiB 2024.10.22 02:34:46.598457 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_809_809_0 to 202410_819_819_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:46.651311 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_982_82} MergeTreeSequentialSource: Reading 2 marks from part 202410_982_982_0, total 204 rows starting from the beginning of the part 2024.10.22 02:34:46.682484 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 827 2024.10.22 02:34:47.019910 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(3681885df01e5475688b2da18a58d076) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 3681885df01e5475688b2da18a58d076, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:47.025384 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 975558, file segment info: File segment: [0, 33554431], key: 3681885df01e5475688b2da18a58d076, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:47.030228 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:47.371486 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1436172 2024.10.22 02:34:47.773685 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:47.812615 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b49d9d856f5c9dcfbf4746bbe8095902) : [0, 157]: Setting download as failed: File segment: [0, 157], key: b49d9d856f5c9dcfbf4746bbe8095902, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:47.861296 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 63 entries to flush up to offset 113203 2024.10.22 02:34:47.861691 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b49d9d856f5c9dcfbf4746bbe8095902): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 02:34:47.874802 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:48.113567 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:48.118448 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:48.123166 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_957_957_0 (state Temporary) 2024.10.22 02:34:48.156878 [ 1418 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_982_82/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:34:48.320378 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_957_957_0 to 202410_964_964_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:48.350189 [ 1260 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 6 parts from 202410_959_959_0 to 202410_964_964_0 2024.10.22 02:34:48.359007 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1436172 2024.10.22 02:34:48.396497 [ 1260 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:48.407503 [ 1260 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:48.424698 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1683_1683_0 (state Temporary) 2024.10.22 02:34:48.440517 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} MergeTask::PrepareStage: Merging 6 parts: from 202410_959_959_0 to 202410_964_964_0 into Compact with storage Full 2024.10.22 02:34:48.558836 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1683_1683_0 to 202410_1696_1696_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:48.583791 [ 1236 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1691_13 to 202410_1696_1696_0 2024.10.22 02:34:48.734343 [ 1236 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:48.748802 [ 1236 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:48.773588 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:48.787912 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_959_959_0, total 770 rows starting from the beginning of the part 2024.10.22 02:34:48.792554 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1691_13 to 202410_1696_1696_0 into Compact with storage Full 2024.10.22 02:34:48.822575 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_960_960_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:34:48.827797 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 113203 2024.10.22 02:34:48.839350 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_961_961_0, total 1540 rows starting from the beginning of the part 2024.10.22 02:34:48.858466 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_962_962_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:34:48.875879 [ 1423 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_200_200_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:34:48.963700 [ 1399 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_982_82} MergeTask::MergeProjectionsStage: Merge sorted 27299 rows, containing 13 columns (13 merged, 0 gathered) in 2.668413557 sec., 10230.423214717614 rows/sec., 4.46 MiB/sec. 2024.10.22 02:34:49.006578 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_963_963_0, total 770 rows starting from the beginning of the part 2024.10.22 02:34:49.050345 [ 1391 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} MergeTreeSequentialSource: Reading 2 marks from part 202410_964_964_0, total 1155 rows starting from the beginning of the part 2024.10.22 02:34:49.059908 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:49.075024 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1691_13, total 4600 rows starting from the beginning of the part 2024.10.22 02:34:49.152894 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(82e1994433051140412149c7a8fe1cb2) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 82e1994433051140412149c7a8fe1cb2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:49.155412 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 818, file segment info: File segment: [0, 33554431], key: 82e1994433051140412149c7a8fe1cb2, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1397, kind: Regular, unbound: 0 2024.10.22 02:34:49.158679 [ 1397 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:49.213352 [ 1683 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/wwk/rlbzrheubzimqgzwkhknkeenunegp, size 975558 2024.10.22 02:34:49.247485 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_982_82} MergedBlockOutputStream: filled checksums 202410_747_982_82 (state Temporary) 2024.10.22 02:34:49.255719 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1692_1692_0, total 88 rows starting from the beginning of the part 2024.10.22 02:34:49.413423 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1693_1693_0, total 68 rows starting from the beginning of the part 2024.10.22 02:34:49.679730 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:34:49.705078 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:34:49.736281 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1694_1694_0, total 75 rows starting from the beginning of the part 2024.10.22 02:34:49.765546 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:34:49.849114 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:34:49.853264 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_982_82} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_982_82 to 202410_747_982_82 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:49.857372 [ 1394 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_982_82} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_981_81, 202410_982_982_0] -> 202410_747_982_82 2024.10.22 02:34:49.871909 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1695_1695_0, total 66 rows starting from the beginning of the part 2024.10.22 02:34:49.888678 [ 1394 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.02 MiB. 2024.10.22 02:34:49.983795 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} MergeTreeSequentialSource: Reading 2 marks from part 202410_1696_1696_0, total 63 rows starting from the beginning of the part 2024.10.22 02:34:50.041185 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(aa0855f51d042ce14633003a7da4e650) : [0, 326558]: Setting download as failed: File segment: [0, 326558], key: aa0855f51d042ce14633003a7da4e650, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:50.043776 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(aa0855f51d042ce14633003a7da4e650): No space left in cache to reserve 326559 bytes, will continue without cache download 2024.10.22 02:34:50.084279 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:34:50.115032 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:34:50.125082 [ 1392 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} MergeTask::MergeProjectionsStage: Merge sorted 6545 rows, containing 4 columns (4 merged, 0 gathered) in 1.696755219 sec., 3857.3625274347837 rows/sec., 60.27 KiB/sec. 2024.10.22 02:34:50.171558 [ 1400 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} MergedBlockOutputStream: filled checksums 202410_959_964_1 (state Temporary) 2024.10.22 02:34:50.172771 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: aa0855f51d042ce14633003a7da4e650, offset: 0, size: 0 2024.10.22 02:34:50.173629 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key aa0855f51d042ce14633003a7da4e650 for removal 2024.10.22 02:34:50.175322 [ 1476 ] {} CacheMetadata: Key aa0855f51d042ce14633003a7da4e650 is removed from metadata 2024.10.22 02:34:50.177070 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b49d9d856f5c9dcfbf4746bbe8095902, offset: 0, size: 0 2024.10.22 02:34:50.177505 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParsedParams.Key2 2024.10.22 02:34:50.178232 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b49d9d856f5c9dcfbf4746bbe8095902 for removal 2024.10.22 02:34:50.185900 [ 1476 ] {} CacheMetadata: Key b49d9d856f5c9dcfbf4746bbe8095902 is removed from metadata 2024.10.22 02:34:50.305608 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_959_964_1 to 202410_959_964_1 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:50.310182 [ 1386 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_959_964_1} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 6 parts: [202410_959_959_0, 202410_964_964_0] -> 202410_959_964_1 2024.10.22 02:34:50.350652 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.20 MiB. 2024.10.22 02:34:50.360768 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/irj/vvhvhpjjkdoritxtcsilnnwyxvgua (LogSeriesLimiter: on interval from 2024-10-22 02:34:26 to 2024-10-22 02:34:50 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:34:50.367010 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 14.02 MiB, will set to 1.56 GiB (RSS), difference: 6.19 MiB 2024.10.22 02:34:50.662225 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 26 entries to flush up to offset 15453 2024.10.22 02:34:50.881182 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(aa603ada49bc4138b6155ef2b3df780f) : [0, 161]: Setting download as failed: File segment: [0, 161], key: aa603ada49bc4138b6155ef2b3df780f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:50.985393 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(aa603ada49bc4138b6155ef2b3df780f): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 02:34:51.094603 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:51.102453 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:51.149700 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_948_313, 202410_949_949_0] 2024.10.22 02:34:51.152662 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_948_313, 202410_949_949_0] 2024.10.22 02:34:51.166073 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_948_313, 202410_949_949_0] 2024.10.22 02:34:51.173120 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_999_999_0 (state Temporary) 2024.10.22 02:34:51.175999 [ 1510 ] {} AsynchronousMetrics: Update heavy metrics. Update period 1 sec. Update heavy metrics period 120 sec. Heavy metrics calculation elapsed: 0.225108061 sec. 2024.10.22 02:34:51.253631 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_999_999_0 to 202410_1012_1012_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:51.273349 [ 1217 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1007_372 to 202410_1008_1008_0 2024.10.22 02:34:51.284069 [ 1217 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:51.288808 [ 1217 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:51.312201 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1008_373} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1007_372 to 202410_1008_1008_0 into Compact with storage Full 2024.10.22 02:34:51.400359 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1008_373} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:51.418066 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1008_373} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1007_372, total 15476 rows starting from the beginning of the part 2024.10.22 02:34:51.420973 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15453 2024.10.22 02:34:51.446207 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1008_373} MergeTreeSequentialSource: Reading 2 marks from part 202410_1008_1008_0, total 10 rows starting from the beginning of the part 2024.10.22 02:34:51.453447 [ 1224 ] {} DatabaseCatalog: Execution took 263 ms. 2024.10.22 02:34:51.553431 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:34:51.555775 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:34:51.683443 [ 1393 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} MergeTask::MergeProjectionsStage: Merge sorted 4960 rows, containing 13 columns (13 merged, 0 gathered) in 2.925376308 sec., 1695.5083646626701 rows/sec., 299.37 KiB/sec. 2024.10.22 02:34:51.895159 [ 1398 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} MergedBlockOutputStream: filled checksums 202410_1626_1696_14 (state Temporary) 2024.10.22 02:34:52.262866 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1696_14 to 202410_1626_1696_14 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:52.265510 [ 1388 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1696_14} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1691_13, 202410_1696_1696_0] -> 202410_1626_1696_14 2024.10.22 02:34:52.271713 [ 1397 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1008_373} MergeTask::MergeProjectionsStage: Merge sorted 15486 rows, containing 12 columns (12 merged, 0 gathered) in 0.974023864 sec., 15898.994441885667 rows/sec., 2.49 MiB/sec. 2024.10.22 02:34:52.424471 [ 1388 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:34:52.436183 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d09394f0a30404108b87884f84aa43e2) : [0, 516717]: Setting download as failed: File segment: [0, 516717], key: d09394f0a30404108b87884f84aa43e2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:52.539860 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d09394f0a30404108b87884f84aa43e2): No space left in cache to reserve 516718 bytes, will continue without cache download 2024.10.22 02:34:52.540865 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1008_373} MergedBlockOutputStream: filled checksums 202410_1_1008_373 (state Temporary) 2024.10.22 02:34:52.564755 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d09394f0a30404108b87884f84aa43e2, offset: 0, size: 0 2024.10.22 02:34:52.566829 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d09394f0a30404108b87884f84aa43e2 for removal 2024.10.22 02:34:52.551220 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 14.34 MiB, will set to 1.56 GiB (RSS), difference: -8.35 MiB 2024.10.22 02:34:52.572483 [ 1476 ] {} CacheMetadata: Key d09394f0a30404108b87884f84aa43e2 is removed from metadata 2024.10.22 02:34:52.581012 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: aa603ada49bc4138b6155ef2b3df780f, offset: 0, size: 0 2024.10.22 02:34:52.586899 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key aa603ada49bc4138b6155ef2b3df780f for removal 2024.10.22 02:34:52.590704 [ 1476 ] {} CacheMetadata: Key aa603ada49bc4138b6155ef2b3df780f is removed from metadata 2024.10.22 02:34:52.658952 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1008_373} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1008_373 to 202410_1_1008_373 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:52.663193 [ 1392 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1008_373} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1007_372, 202410_1008_1008_0] -> 202410_1_1008_373 2024.10.22 02:34:52.844867 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 108 entries to flush up to offset 113311 2024.10.22 02:34:52.992970 [ 1392 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:34:53.251685 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:53.257020 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:53.348494 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1684_1684_0 (state Temporary) 2024.10.22 02:34:53.506901 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1684_1684_0 to 202410_1697_1697_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:53.567428 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 113311 2024.10.22 02:34:53.777718 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 191 entries to flush up to offset 111702 2024.10.22 02:34:53.924493 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(9252bff745055da5857839dc26cf981f) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 9252bff745055da5857839dc26cf981f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:53.928685 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(9252bff745055da5857839dc26cf981f): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 02:34:54.189780 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 828 2024.10.22 02:34:54.250669 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:54.253207 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:54.426349 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_982_982_0 (state Temporary) 2024.10.22 02:34:54.496254 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 14.55 MiB, will set to 1.56 GiB (RSS), difference: 3.83 MiB 2024.10.22 02:34:54.628282 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:54.637031 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:54.653883 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_982_982_0 to 202410_988_988_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:54.666464 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(8a3dca67b7c110e3ef7e8a8c17d9b54a) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 8a3dca67b7c110e3ef7e8a8c17d9b54a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 02:34:54.672676 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1106, file segment info: File segment: [0, 33554431], key: 8a3dca67b7c110e3ef7e8a8c17d9b54a, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1400, kind: Regular, unbound: 0 2024.10.22 02:34:54.676059 [ 1400 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:34:54.678114 [ 1171 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_982_82 to 202410_983_983_0 2024.10.22 02:34:54.696792 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_810_810_0 (state Temporary) 2024.10.22 02:34:54.705699 [ 1171 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.52 MiB using storage policy from min volume index 0 2024.10.22 02:34:54.711210 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 111702 2024.10.22 02:34:54.715863 [ 1171 ] {} DiskLocal: Reserved 1.52 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:54.763237 [ 1725 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/chx/owxelszvxfgnevwziaeydivgajmfq, size 1657823 2024.10.22 02:34:54.771823 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_983_83} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_982_82 to 202410_983_983_0 into Wide with storage Full 2024.10.22 02:34:54.814618 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_810_810_0 to 202410_820_820_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:54.819014 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_983_83} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:34:54.829470 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_983_83} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_982_82, total 27299 rows starting from the beginning of the part 2024.10.22 02:34:54.907257 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 828 2024.10.22 02:34:54.968726 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_983_83} MergeTreeSequentialSource: Reading 2 marks from part 202410_983_983_0, total 158 rows starting from the beginning of the part 2024.10.22 02:34:55.189829 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 2 old parts to remove. Parts: [202410_1_892_166, 202410_893_898_1] 2024.10.22 02:34:55.192522 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from filesystem (serially): Parts: [202410_1_892_166, 202410_893_898_1] 2024.10.22 02:34:55.200283 [ 1424 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 2 parts from memory: Parts: [202410_1_892_166, 202410_893_898_1] 2024.10.22 02:34:55.236533 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Renaming temporary part tmp_insert_202410_200_200_0 to 202410_203_203_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:55.262963 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:34:55.264033 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(1f48346a008a82cceb400968df36c54f) : [0, 485749]: Setting download as failed: File segment: [0, 485749], key: 1f48346a008a82cceb400968df36c54f, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:55.265346 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(1f48346a008a82cceb400968df36c54f): No space left in cache to reserve 485750 bytes, will continue without cache download 2024.10.22 02:34:55.281006 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:34:55.295235 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:34:55.318533 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:34:55.349725 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:34:55.359382 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:34:55.366313 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:34:55.366452 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1437327 2024.10.22 02:34:55.380242 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:34:55.401199 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column IPNetworkID 2024.10.22 02:34:55.445326 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/mlf/bnncrmwesvuqffkkmaoltwjpfoouv (LogSeriesLimiter: on interval from 2024-10-22 02:34:50 to 2024-10-22 02:34:55 accepted series 1 / 4 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:34:55.523889 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 1f48346a008a82cceb400968df36c54f, offset: 0, size: 0 2024.10.22 02:34:55.528315 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 1f48346a008a82cceb400968df36c54f for removal 2024.10.22 02:34:55.540093 [ 1476 ] {} CacheMetadata: Key 1f48346a008a82cceb400968df36c54f is removed from metadata 2024.10.22 02:34:55.544177 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 9252bff745055da5857839dc26cf981f, offset: 0, size: 0 2024.10.22 02:34:55.546775 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 9252bff745055da5857839dc26cf981f for removal 2024.10.22 02:34:55.548762 [ 1476 ] {} CacheMetadata: Key 9252bff745055da5857839dc26cf981f is removed from metadata 2024.10.22 02:34:55.754550 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1207_1593_122, 202410_1594_1594_0, 202410_1595_1595_0, 202410_1596_1596_0] 2024.10.22 02:34:55.766497 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1207_1593_122, 202410_1594_1594_0, 202410_1595_1595_0, 202410_1596_1596_0] 2024.10.22 02:34:55.814527 [ 1422 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1207_1593_122, 202410_1594_1594_0, 202410_1595_1595_0, 202410_1596_1596_0] 2024.10.22 02:34:56.135502 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:56.139040 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:56.471037 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_958_958_0 (state Temporary) 2024.10.22 02:34:56.612539 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(928e615bec8c6b20395a1984bbca2af2) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 928e615bec8c6b20395a1984bbca2af2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:56.616310 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(928e615bec8c6b20395a1984bbca2af2): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 02:34:56.896417 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_958_958_0 to 202410_965_965_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:57.043955 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1437327 2024.10.22 02:34:57.069829 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_958_177/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:34:57.539022 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.60 GiB, peak 2.00 GiB, free memory in arenas 12.39 MiB, will set to 1.57 GiB (RSS), difference: -29.00 MiB 2024.10.22 02:34:57.581596 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 68 entries to flush up to offset 113379 2024.10.22 02:34:57.590410 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2db82d11c366d64c2869349393016cc5) : [0, 315326]: Setting download as failed: File segment: [0, 315326], key: 2db82d11c366d64c2869349393016cc5, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:57.592766 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2db82d11c366d64c2869349393016cc5): No space left in cache to reserve 315327 bytes, will continue without cache download 2024.10.22 02:34:57.734498 [ 1183 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:34:57.775992 [ 1183 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:34:57.805365 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:57.814141 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:57.840536 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2db82d11c366d64c2869349393016cc5, offset: 0, size: 0 2024.10.22 02:34:57.842153 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2db82d11c366d64c2869349393016cc5 for removal 2024.10.22 02:34:57.845382 [ 1476 ] {} CacheMetadata: Key 2db82d11c366d64c2869349393016cc5 is removed from metadata 2024.10.22 02:34:57.845788 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 928e615bec8c6b20395a1984bbca2af2, offset: 0, size: 0 2024.10.22 02:34:57.847839 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 928e615bec8c6b20395a1984bbca2af2 for removal 2024.10.22 02:34:57.850936 [ 1476 ] {} CacheMetadata: Key 928e615bec8c6b20395a1984bbca2af2 is removed from metadata 2024.10.22 02:34:57.963351 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1685_1685_0 (state Temporary) 2024.10.22 02:34:57.984923 [ 1390 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_983_83} MergeTask::MergeProjectionsStage: Merge sorted 27457 rows, containing 13 columns (13 merged, 0 gathered) in 3.248393358 sec., 8452.486190559437 rows/sec., 3.69 MiB/sec. 2024.10.22 02:34:57.993801 [ 1549 ] {} SystemLog (system.metric_log): Flushed system log up to offset 8084 2024.10.22 02:34:58.087707 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1685_1685_0 to 202410_1698_1698_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:58.154659 [ 1419 ] {} test.hits (5e446213-61fa-4dfd-ac7e-377a973d53b4): /var/lib/clickhouse/disks/s3_disk/store/5e4/5e446213-61fa-4dfd-ac7e-377a973d53b4/tmp_merge_201403_8_16_1/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 3 similar messages) 2024.10.22 02:34:58.157506 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 113379 2024.10.22 02:34:58.165315 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_983_83} MergedBlockOutputStream: filled checksums 202410_747_983_83 (state Temporary) 2024.10.22 02:34:58.930232 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 17 entries to flush up to offset 15470 2024.10.22 02:34:58.961061 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(91da078bc0edbe6d2f0a0f8ec0607fef) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 91da078bc0edbe6d2f0a0f8ec0607fef, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:58.961829 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(91da078bc0edbe6d2f0a0f8ec0607fef): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:34:58.976372 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_983_83} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_983_83 to 202410_747_983_83 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:58.979985 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_983_83} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_982_82, 202410_983_983_0] -> 202410_747_983_83 2024.10.22 02:34:59.004350 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.03 MiB. 2024.10.22 02:34:59.142185 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:59.145509 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:59.284712 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_1000_1000_0 (state Temporary) 2024.10.22 02:34:59.659777 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.54 GiB, peak 2.00 GiB, free memory in arenas 14.60 MiB, will set to 1.56 GiB (RSS), difference: 24.20 MiB 2024.10.22 02:34:59.687362 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b75432d25583d9a8d17507868bfdf7f4) : [0, 264559]: Setting download as failed: File segment: [0, 264559], key: b75432d25583d9a8d17507868bfdf7f4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:34:59.690396 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b75432d25583d9a8d17507868bfdf7f4): No space left in cache to reserve 264560 bytes, will continue without cache download 2024.10.22 02:34:59.738412 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b75432d25583d9a8d17507868bfdf7f4, offset: 0, size: 0 2024.10.22 02:34:59.740366 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b75432d25583d9a8d17507868bfdf7f4 for removal 2024.10.22 02:34:59.743886 [ 1476 ] {} CacheMetadata: Key b75432d25583d9a8d17507868bfdf7f4 is removed from metadata 2024.10.22 02:34:59.744180 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 91da078bc0edbe6d2f0a0f8ec0607fef, offset: 0, size: 0 2024.10.22 02:34:59.746645 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 91da078bc0edbe6d2f0a0f8ec0607fef for removal 2024.10.22 02:34:59.748755 [ 1476 ] {} CacheMetadata: Key 91da078bc0edbe6d2f0a0f8ec0607fef is removed from metadata 2024.10.22 02:34:59.815547 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_1000_1000_0 to 202410_1013_1013_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:34:59.844751 [ 1184 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1008_373 to 202410_1009_1009_0 2024.10.22 02:34:59.848408 [ 1184 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:34:59.850456 [ 1184 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:34:59.852396 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15470 2024.10.22 02:34:59.908751 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1009_374} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1008_373 to 202410_1009_1009_0 into Compact with storage Full 2024.10.22 02:34:59.976818 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_949_314, 202410_950_950_0] 2024.10.22 02:34:59.978327 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_949_314, 202410_950_950_0] 2024.10.22 02:34:59.988705 [ 1418 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_949_314, 202410_950_950_0] 2024.10.22 02:35:00.063383 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1009_374} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:35:00.067379 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1009_374} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1008_373, total 15486 rows starting from the beginning of the part 2024.10.22 02:35:00.293148 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1009_374} MergeTreeSequentialSource: Reading 2 marks from part 202410_1009_1009_0, total 14 rows starting from the beginning of the part 2024.10.22 02:35:00.799557 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(264db0f4078717ffbff02750e1215093) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 264db0f4078717ffbff02750e1215093, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:00.804296 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(264db0f4078717ffbff02750e1215093): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 02:35:01.246312 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): /var/lib/clickhouse/store/527/5276570e-a364-404e-a403-3e186cba01a8/tmp_merge_202410_1_1009_374/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:35:01.393674 [ 1388 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1009_374} MergeTask::MergeProjectionsStage: Merge sorted 15500 rows, containing 12 columns (12 merged, 0 gathered) in 1.494870022 sec., 10368.794458305085 rows/sec., 1.62 MiB/sec. 2024.10.22 02:35:01.545718 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1009_374} MergedBlockOutputStream: filled checksums 202410_1_1009_374 (state Temporary) 2024.10.22 02:35:01.554991 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:35:01.556547 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:35:01.721809 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 15.32 MiB, will set to 1.56 GiB (RSS), difference: 4.65 MiB 2024.10.22 02:35:01.741780 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1009_374} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1009_374 to 202410_1_1009_374 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:01.855172 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1009_374} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1008_373, 202410_1009_1009_0] -> 202410_1_1009_374 2024.10.22 02:35:01.867489 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(52eb0b0e41586a004240c8281b068390) : [0, 495459]: Setting download as failed: File segment: [0, 495459], key: 52eb0b0e41586a004240c8281b068390, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:01.870943 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(52eb0b0e41586a004240c8281b068390): No space left in cache to reserve 495460 bytes, will continue without cache download 2024.10.22 02:35:01.895786 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:35:02.022993 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 52eb0b0e41586a004240c8281b068390, offset: 0, size: 0 2024.10.22 02:35:02.026004 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 52eb0b0e41586a004240c8281b068390 for removal 2024.10.22 02:35:02.030704 [ 1476 ] {} CacheMetadata: Key 52eb0b0e41586a004240c8281b068390 is removed from metadata 2024.10.22 02:35:02.039199 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 264db0f4078717ffbff02750e1215093, offset: 0, size: 0 2024.10.22 02:35:02.041328 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 264db0f4078717ffbff02750e1215093 for removal 2024.10.22 02:35:02.044431 [ 1476 ] {} CacheMetadata: Key 264db0f4078717ffbff02750e1215093 is removed from metadata 2024.10.22 02:35:02.194254 [ 1187 ] {} DatabaseCatalog: Execution took 741 ms. 2024.10.22 02:35:02.253713 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 168 entries to flush up to offset 111870 2024.10.22 02:35:02.288543 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 113448 2024.10.22 02:35:02.410757 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 829 2024.10.22 02:35:02.602143 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:02.605420 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:02.635468 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:02.640027 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:02.674508 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_811_811_0 (state Temporary) 2024.10.22 02:35:02.704507 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_983_983_0 (state Temporary) 2024.10.22 02:35:02.766332 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_811_811_0 to 202410_821_821_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:02.868607 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 829 2024.10.22 02:35:02.990372 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_983_983_0 to 202410_989_989_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:03.090831 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 111870 2024.10.22 02:35:03.108107 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:03.110227 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:03.123567 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_983_83 to 202410_984_984_0 2024.10.22 02:35:03.130325 [ 1221 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.53 MiB using storage policy from min volume index 0 2024.10.22 02:35:03.134382 [ 1221 ] {} DiskLocal: Reserved 1.53 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:03.152654 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_984_84} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_983_83 to 202410_984_984_0 into Wide with storage Full 2024.10.22 02:35:03.167006 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8c1ae23b5f0536b1247f351d70b4186b) : [0, 159]: Setting download as failed: File segment: [0, 159], key: 8c1ae23b5f0536b1247f351d70b4186b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:03.169666 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8c1ae23b5f0536b1247f351d70b4186b): No space left in cache to reserve 160 bytes, will continue without cache download 2024.10.22 02:35:03.178463 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1686_1686_0 (state Temporary) 2024.10.22 02:35:03.238826 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1686_1686_0 to 202410_1699_1699_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:03.296418 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_984_84} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:35:03.317102 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_984_84} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_983_83, total 27457 rows starting from the beginning of the part 2024.10.22 02:35:03.464858 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_984_84} MergeTreeSequentialSource: Reading 2 marks from part 202410_984_984_0, total 167 rows starting from the beginning of the part 2024.10.22 02:35:03.478227 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 113448 2024.10.22 02:35:04.149075 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1905 entries to flush up to offset 1439232 2024.10.22 02:35:04.663527 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(d2a397ae5071e83476e20c6017c3552a) : [0, 416450]: Setting download as failed: File segment: [0, 416450], key: d2a397ae5071e83476e20c6017c3552a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:04.664407 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(d2a397ae5071e83476e20c6017c3552a): No space left in cache to reserve 416451 bytes, will continue without cache download 2024.10.22 02:35:04.797579 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: d2a397ae5071e83476e20c6017c3552a, offset: 0, size: 0 2024.10.22 02:35:04.800149 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key d2a397ae5071e83476e20c6017c3552a for removal 2024.10.22 02:35:04.805166 [ 1476 ] {} CacheMetadata: Key d2a397ae5071e83476e20c6017c3552a is removed from metadata 2024.10.22 02:35:04.810701 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8c1ae23b5f0536b1247f351d70b4186b, offset: 0, size: 0 2024.10.22 02:35:04.812943 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8c1ae23b5f0536b1247f351d70b4186b for removal 2024.10.22 02:35:04.816620 [ 1476 ] {} CacheMetadata: Key 8c1ae23b5f0536b1247f351d70b4186b is removed from metadata 2024.10.22 02:35:05.373657 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.61 GiB, peak 2.00 GiB, free memory in arenas 11.60 MiB, will set to 1.57 GiB (RSS), difference: -35.53 MiB 2024.10.22 02:35:05.464015 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:05.487526 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:05.571891 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Found 5 old parts to remove. Parts: [202410_747_926_38, 202410_927_927_0, 202410_928_928_0, 202410_929_929_0, 202410_930_930_0] 2024.10.22 02:35:05.577176 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from filesystem (serially): Parts: [202410_747_926_38, 202410_927_927_0, 202410_928_928_0, 202410_929_929_0, 202410_930_930_0] 2024.10.22 02:35:05.587619 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_959_959_0 (state Temporary) 2024.10.22 02:35:05.649037 [ 1549 ] {} SystemLog (system.metric_log): Flushing system log, 40 entries to flush up to offset 8124 2024.10.22 02:35:05.726618 [ 1420 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Removing 5 parts from memory: Parts: [202410_747_926_38, 202410_927_927_0, 202410_928_928_0, 202410_929_929_0, 202410_930_930_0] 2024.10.22 02:35:06.027224 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_959_959_0 to 202410_966_966_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:06.061289 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1439232 2024.10.22 02:35:06.126548 [ 1271 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 247 ms. 2024.10.22 02:35:06.134408 [ 1419 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_984_84/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:35:06.210500 [ 1397 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_984_84} MergeTask::MergeProjectionsStage: Merge sorted 27624 rows, containing 13 columns (13 merged, 0 gathered) in 3.062234462 sec., 9020.86379824694 rows/sec., 3.94 MiB/sec. 2024.10.22 02:35:06.275362 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_984_84} MergedBlockOutputStream: filled checksums 202410_747_984_84 (state Temporary) 2024.10.22 02:35:06.784698 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_984_84} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_984_84 to 202410_747_984_84 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:06.788504 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_984_84} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_983_83, 202410_984_984_0] -> 202410_747_984_84 2024.10.22 02:35:06.811785 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b156a2aa2ff652d3b8f5026487f99ecf) : [0, 157]: Setting download as failed: File segment: [0, 157], key: b156a2aa2ff652d3b8f5026487f99ecf, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:06.815131 [ 1385 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.08 MiB. 2024.10.22 02:35:06.827162 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b156a2aa2ff652d3b8f5026487f99ecf): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 02:35:07.356510 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 14 entries to flush up to offset 15484 2024.10.22 02:35:07.444236 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:07.492265 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 58 entries to flush up to offset 113506 2024.10.22 02:35:07.500285 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Found 6 old parts to remove. Parts: [202410_1_768_156, 202410_769_769_0, 202410_770_770_0, 202410_771_771_0, 202410_772_772_0, 202410_773_773_0] 2024.10.22 02:35:07.503311 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from filesystem (serially): Parts: [202410_1_768_156, 202410_769_769_0, 202410_770_770_0, 202410_771_771_0, 202410_772_772_0, 202410_773_773_0] 2024.10.22 02:35:07.547421 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:07.571846 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(f4137cf0da9114c9ac95a51fc2c03c68) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: f4137cf0da9114c9ac95a51fc2c03c68, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 02:35:07.575160 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1048576, file segment info: File segment: [0, 33554431], key: f4137cf0da9114c9ac95a51fc2c03c68, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1395, kind: Regular, unbound: 0 2024.10.22 02:35:07.578713 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:35:07.590667 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_1001_1001_0 (state Temporary) 2024.10.22 02:35:07.645287 [ 1418 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Removing 6 parts from memory: Parts: [202410_1_768_156, 202410_769_769_0, 202410_770_770_0, 202410_771_771_0, 202410_772_772_0, 202410_773_773_0] 2024.10.22 02:35:07.662352 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_1001_1001_0 to 202410_1014_1014_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:07.705617 [ 1236 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1009_374 to 202410_1010_1010_0 2024.10.22 02:35:07.721176 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15484 2024.10.22 02:35:07.735699 [ 1236 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:07.738675 [ 1236 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:07.775451 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:07.782576 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:07.787468 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1010_375} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1009_374 to 202410_1010_1010_0 into Compact with storage Full 2024.10.22 02:35:07.857822 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1010_375} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:35:07.859178 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1687_1687_0 (state Temporary) 2024.10.22 02:35:07.876228 [ 1419 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_958_177/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:35:07.876528 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1010_375} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1009_374, total 15500 rows starting from the beginning of the part 2024.10.22 02:35:07.955557 [ 1400 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1010_375} MergeTreeSequentialSource: Reading 2 marks from part 202410_1010_1010_0, total 21 rows starting from the beginning of the part 2024.10.22 02:35:08.232261 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1687_1687_0 to 202410_1700_1700_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:08.274304 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 113506 2024.10.22 02:35:08.379166 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(296cafdfd5860317f67d1be1f32b3f06) : [0, 302508]: Setting download as failed: File segment: [0, 302508], key: 296cafdfd5860317f67d1be1f32b3f06, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:08.382373 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(296cafdfd5860317f67d1be1f32b3f06): No space left in cache to reserve 302509 bytes, will continue without cache download 2024.10.22 02:35:08.417497 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 296cafdfd5860317f67d1be1f32b3f06, offset: 0, size: 0 2024.10.22 02:35:08.421154 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 296cafdfd5860317f67d1be1f32b3f06 for removal 2024.10.22 02:35:08.426464 [ 1476 ] {} CacheMetadata: Key 296cafdfd5860317f67d1be1f32b3f06 is removed from metadata 2024.10.22 02:35:08.428792 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b156a2aa2ff652d3b8f5026487f99ecf, offset: 0, size: 0 2024.10.22 02:35:08.434405 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b156a2aa2ff652d3b8f5026487f99ecf for removal 2024.10.22 02:35:08.437605 [ 1476 ] {} CacheMetadata: Key b156a2aa2ff652d3b8f5026487f99ecf is removed from metadata 2024.10.22 02:35:09.357294 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.52 GiB, peak 2.00 GiB, free memory in arenas 13.20 MiB, will set to 1.56 GiB (RSS), difference: 38.00 MiB 2024.10.22 02:35:09.421208 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_950_315, 202410_951_951_0] 2024.10.22 02:35:09.424304 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_950_315, 202410_951_951_0] 2024.10.22 02:35:09.436273 [ 1420 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_950_315, 202410_951_951_0] 2024.10.22 02:35:09.441183 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1010_375} MergeTask::MergeProjectionsStage: Merge sorted 15521 rows, containing 12 columns (12 merged, 0 gathered) in 1.665002739 sec., 9321.90658696568 rows/sec., 1.46 MiB/sec. 2024.10.22 02:35:09.510137 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1010_375} MergedBlockOutputStream: filled checksums 202410_1_1010_375 (state Temporary) 2024.10.22 02:35:09.837247 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(284bc5ff20c4ee8d1e24cf37c272739a) : [0, 157]: Setting download as failed: File segment: [0, 157], key: 284bc5ff20c4ee8d1e24cf37c272739a, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:09.842993 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(284bc5ff20c4ee8d1e24cf37c272739a): No space left in cache to reserve 158 bytes, will continue without cache download 2024.10.22 02:35:09.924677 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1010_375} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1010_375 to 202410_1_1010_375 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:09.930688 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1010_375} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1009_374, 202410_1010_1010_0] -> 202410_1_1010_375 2024.10.22 02:35:10.070950 [ 1393 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:35:10.619453 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 158 entries to flush up to offset 112028 2024.10.22 02:35:10.651654 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(4a6cc93b61fe82e67544ea7fbaa6886e) : [0, 302578]: Setting download as failed: File segment: [0, 302578], key: 4a6cc93b61fe82e67544ea7fbaa6886e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:10.653666 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(4a6cc93b61fe82e67544ea7fbaa6886e): No space left in cache to reserve 302579 bytes, will continue without cache download 2024.10.22 02:35:10.676012 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 4a6cc93b61fe82e67544ea7fbaa6886e, offset: 0, size: 0 2024.10.22 02:35:10.682275 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 4a6cc93b61fe82e67544ea7fbaa6886e for removal 2024.10.22 02:35:10.689619 [ 1476 ] {} CacheMetadata: Key 4a6cc93b61fe82e67544ea7fbaa6886e is removed from metadata 2024.10.22 02:35:10.695824 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 284bc5ff20c4ee8d1e24cf37c272739a, offset: 0, size: 0 2024.10.22 02:35:10.699840 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 284bc5ff20c4ee8d1e24cf37c272739a for removal 2024.10.22 02:35:10.705432 [ 1476 ] {} CacheMetadata: Key 284bc5ff20c4ee8d1e24cf37c272739a is removed from metadata 2024.10.22 02:35:10.751059 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:10.763930 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:10.913716 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_984_984_0 (state Temporary) 2024.10.22 02:35:10.964049 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_984_984_0 to 202410_990_990_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:10.993018 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_984_84 to 202410_985_985_0 2024.10.22 02:35:11.003073 [ 1219 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.54 MiB using storage policy from min volume index 0 2024.10.22 02:35:11.007485 [ 1219 ] {} DiskLocal: Reserved 1.54 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:11.010727 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 112028 2024.10.22 02:35:11.131969 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_985_85} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_984_84 to 202410_985_985_0 into Wide with storage Full 2024.10.22 02:35:11.163320 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_985_85} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:35:11.170812 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_985_85} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_984_84, total 27624 rows starting from the beginning of the part 2024.10.22 02:35:11.434907 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 13.81 MiB, will set to 1.56 GiB (RSS), difference: 5.98 MiB 2024.10.22 02:35:11.470884 [ 1393 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_985_85} MergeTreeSequentialSource: Reading 2 marks from part 202410_985_985_0, total 174 rows starting from the beginning of the part 2024.10.22 02:35:11.558855 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:35:11.560887 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:35:11.663017 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(956712b12366852771b2eaddeed145d4) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 956712b12366852771b2eaddeed145d4, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:11.665228 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(956712b12366852771b2eaddeed145d4): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:35:12.332341 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 69 entries to flush up to offset 113575 2024.10.22 02:35:12.441136 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 4 old parts to remove. Parts: [202410_1207_1596_123, 202410_1597_1597_0, 202410_1598_1598_0, 202410_1599_1599_0] 2024.10.22 02:35:12.454447 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from filesystem (serially): Parts: [202410_1207_1596_123, 202410_1597_1597_0, 202410_1598_1598_0, 202410_1599_1599_0] 2024.10.22 02:35:12.473786 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 4 parts from memory: Parts: [202410_1207_1596_123, 202410_1597_1597_0, 202410_1598_1598_0, 202410_1599_1599_0] 2024.10.22 02:35:12.523617 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:12.532661 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:12.596541 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1688_1688_0 (state Temporary) 2024.10.22 02:35:12.790404 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1688_1688_0 to 202410_1701_1701_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:12.791617 [ 1238 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:35:12.820554 [ 1238 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:35:12.898090 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Selected 6 parts from 202410_1626_1696_14 to 202410_1701_1701_0 2024.10.22 02:35:12.927033 [ 1150 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:12.931926 [ 1150 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:12.935724 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(28e3e6bcc4d7b4ab1b9564c0bc69ee13) : [0, 239201]: Setting download as failed: File segment: [0, 239201], key: 28e3e6bcc4d7b4ab1b9564c0bc69ee13, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:12.937343 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(28e3e6bcc4d7b4ab1b9564c0bc69ee13): No space left in cache to reserve 239202 bytes, will continue without cache download 2024.10.22 02:35:12.940607 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} MergeTask::PrepareStage: Merging 6 parts: from 202410_1626_1696_14 to 202410_1701_1701_0 into Compact with storage Full 2024.10.22 02:35:12.972019 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 113575 2024.10.22 02:35:13.055994 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 28e3e6bcc4d7b4ab1b9564c0bc69ee13, offset: 0, size: 0 2024.10.22 02:35:13.057188 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 28e3e6bcc4d7b4ab1b9564c0bc69ee13 for removal 2024.10.22 02:35:13.060507 [ 1476 ] {} CacheMetadata: Key 28e3e6bcc4d7b4ab1b9564c0bc69ee13 is removed from metadata 2024.10.22 02:35:13.061349 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 956712b12366852771b2eaddeed145d4, offset: 0, size: 0 2024.10.22 02:35:13.062554 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 956712b12366852771b2eaddeed145d4 for removal 2024.10.22 02:35:13.064363 [ 1476 ] {} CacheMetadata: Key 956712b12366852771b2eaddeed145d4 is removed from metadata 2024.10.22 02:35:13.064844 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1175 entries to flush up to offset 1440407 2024.10.22 02:35:13.100390 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:35:13.118469 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1626_1696_14, total 4960 rows starting from the beginning of the part 2024.10.22 02:35:13.157714 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1697_1697_0, total 108 rows starting from the beginning of the part 2024.10.22 02:35:13.202427 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1698_1698_0, total 68 rows starting from the beginning of the part 2024.10.22 02:35:13.278517 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1699_1699_0, total 69 rows starting from the beginning of the part 2024.10.22 02:35:13.335065 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1700_1700_0, total 58 rows starting from the beginning of the part 2024.10.22 02:35:13.494847 [ 1400 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} MergeTreeSequentialSource: Reading 2 marks from part 202410_1701_1701_0, total 69 rows starting from the beginning of the part 2024.10.22 02:35:13.544192 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:13.549808 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:13.658231 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 15.05 MiB, will set to 1.57 GiB (RSS), difference: -50.05 MiB 2024.10.22 02:35:13.786796 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_960_960_0 (state Temporary) 2024.10.22 02:35:13.866229 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_960_960_0 to 202410_967_967_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:13.941707 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_958_177} MergeTask::MergeProjectionsStage: Merge sorted 1443217 rows, containing 4 columns (4 merged, 0 gathered) in 42.045422699 sec., 34325.1870799797 rows/sec., 514.83 KiB/sec. 2024.10.22 02:35:14.139374 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1440407 2024.10.22 02:35:14.154805 [ 1387 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_958_177} MergedBlockOutputStream: filled checksums 202410_1_958_177 (state Temporary) 2024.10.22 02:35:14.157081 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(2436c0c0740a6e7c92af80673f13d4a1) : [0, 161]: Setting download as failed: File segment: [0, 161], key: 2436c0c0740a6e7c92af80673f13d4a1, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:14.159904 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(2436c0c0740a6e7c92af80673f13d4a1): No space left in cache to reserve 162 bytes, will continue without cache download 2024.10.22 02:35:14.966456 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_985_85} MergeTask::MergeProjectionsStage: Merge sorted 27798 rows, containing 13 columns (13 merged, 0 gathered) in 3.950540996 sec., 7036.504627631005 rows/sec., 3.07 MiB/sec. 2024.10.22 02:35:15.036051 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_985_85} MergedBlockOutputStream: filled checksums 202410_747_985_85 (state Temporary) 2024.10.22 02:35:15.146719 [ 1392 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} MergeTask::MergeProjectionsStage: Merge sorted 5332 rows, containing 13 columns (13 merged, 0 gathered) in 2.209034024 sec., 2413.7247059441397 rows/sec., 425.95 KiB/sec. 2024.10.22 02:35:15.225348 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 8 entries to flush up to offset 15492 2024.10.22 02:35:15.396000 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} MergedBlockOutputStream: filled checksums 202410_1626_1701_15 (state Temporary) 2024.10.22 02:35:15.466651 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b817504213f1742f51e3c90bda1d7fb2) : [0, 495095]: Setting download as failed: File segment: [0, 495095], key: b817504213f1742f51e3c90bda1d7fb2, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:15.467833 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b817504213f1742f51e3c90bda1d7fb2): No space left in cache to reserve 495096 bytes, will continue without cache download 2024.10.22 02:35:15.542838 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:15.662659 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_merge_202410_1626_1701_15 to 202410_1626_1701_15 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:15.666603 [ 1386 ] {ffa745eb-8a28-47e2-b73e-5e0c8034f61b::202410_1626_1701_15} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b) (MergerMutator): Merged 6 parts: [202410_1626_1696_14, 202410_1701_1701_0] -> 202410_1626_1701_15 2024.10.22 02:35:15.728222 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:15.744589 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b817504213f1742f51e3c90bda1d7fb2, offset: 0, size: 0 2024.10.22 02:35:15.746627 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b817504213f1742f51e3c90bda1d7fb2 for removal 2024.10.22 02:35:15.750181 [ 1476 ] {} CacheMetadata: Key b817504213f1742f51e3c90bda1d7fb2 is removed from metadata 2024.10.22 02:35:15.753096 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 2436c0c0740a6e7c92af80673f13d4a1, offset: 0, size: 0 2024.10.22 02:35:15.754814 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 2436c0c0740a6e7c92af80673f13d4a1 for removal 2024.10.22 02:35:15.758389 [ 1476 ] {} CacheMetadata: Key 2436c0c0740a6e7c92af80673f13d4a1 is removed from metadata 2024.10.22 02:35:15.764813 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_1002_1002_0 (state Temporary) 2024.10.22 02:35:15.815063 [ 1386 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 5.20 MiB. 2024.10.22 02:35:15.837089 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_985_85} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_985_85 to 202410_747_985_85 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:15.839869 [ 1396 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_985_85} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_984_84, 202410_985_985_0] -> 202410_747_985_85 2024.10.22 02:35:15.853183 [ 1396 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.84 MiB. 2024.10.22 02:35:15.921533 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_1002_1002_0 to 202410_1015_1015_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:15.951881 [ 1181 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1010_375 to 202410_1011_1011_0 2024.10.22 02:35:15.955994 [ 1181 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:15.966651 [ 1181 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:16.080132 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15492 2024.10.22 02:35:16.103285 [ 1182 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 02:35:16.120964 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1011_376} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1010_375 to 202410_1011_1011_0 into Compact with storage Full 2024.10.22 02:35:16.361457 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Found 6 old parts to remove. Parts: [202410_905_905_0, 202410_906_906_0, 202410_907_907_0, 202410_908_908_0, 202410_909_909_0, 202410_910_910_0] 2024.10.22 02:35:16.368406 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from filesystem (serially): Parts: [202410_905_905_0, 202410_906_906_0, 202410_907_907_0, 202410_908_908_0, 202410_909_909_0, 202410_910_910_0] 2024.10.22 02:35:16.414775 [ 1421 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Removing 6 parts from memory: Parts: [202410_905_905_0, 202410_906_906_0, 202410_907_907_0, 202410_908_908_0, 202410_909_909_0, 202410_910_910_0] 2024.10.22 02:35:16.427997 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1011_376} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:35:16.435974 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.50 GiB, peak 2.00 GiB, free memory in arenas 10.59 MiB, will set to 1.56 GiB (RSS), difference: 56.22 MiB 2024.10.22 02:35:16.445374 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1011_376} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1010_375, total 15521 rows starting from the beginning of the part 2024.10.22 02:35:16.465777 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1011_376} MergeTreeSequentialSource: Reading 2 marks from part 202410_1011_1011_0, total 12 rows starting from the beginning of the part 2024.10.22 02:35:16.763573 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegment(a3540bdb73339f59504ece0b78f8205b) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: a3540bdb73339f59504ece0b78f8205b, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:35:16.765738 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 1020, file segment info: File segment: [0, 33554431], key: a3540bdb73339f59504ece0b78f8205b, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1:1391, kind: Regular, unbound: 0 2024.10.22 02:35:16.769534 [ 1391 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:35:16.944834 [ 1741 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/mlf/bnncrmwesvuqffkkmaoltwjpfoouv, size 2203514 2024.10.22 02:35:16.994591 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 81 entries to flush up to offset 113656 2024.10.22 02:35:17.058115 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_951_316, 202410_952_952_0] 2024.10.22 02:35:17.060676 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_951_316, 202410_952_952_0] 2024.10.22 02:35:17.071537 [ 1425 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_951_316, 202410_952_952_0] 2024.10.22 02:35:17.376583 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_8_8_0, total 278528 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:35:17.389993 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_9_9_0, total 286720 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:35:17.489424 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:17.493522 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:17.546153 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_10_10_0, total 278528 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:35:17.562608 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_11_11_0, total 532480 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:35:17.595518 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 35 marks from part 201403_12_12_0, total 278528 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:35:17.595134 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1689_1689_0 (state Temporary) 2024.10.22 02:35:17.617642 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 68 marks from part 201403_13_13_0, total 548864 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:35:17.644098 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_14_14_0, total 286720 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:35:17.666803 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 36 marks from part 201403_15_15_0, total 286720 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:35:17.762183 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1689_1689_0 to 202410_1702_1702_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:17.764442 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(a9259a9c375a4ca648714f79f2c34783) : [0, 154]: Setting download as failed: File segment: [0, 154], key: a9259a9c375a4ca648714f79f2c34783, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:17.768796 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(a9259a9c375a4ca648714f79f2c34783): No space left in cache to reserve 155 bytes, will continue without cache download 2024.10.22 02:35:17.773738 [ 1393 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1011_376} MergeTask::MergeProjectionsStage: Merge sorted 15533 rows, containing 12 columns (12 merged, 0 gathered) in 1.66345653 sec., 9337.785340263747 rows/sec., 1.46 MiB/sec. 2024.10.22 02:35:17.834396 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_16_16_0, total 565248 rows starting from the beginning of the part, column TraficSourceID 2024.10.22 02:35:17.847505 [ 1391 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1011_376} MergedBlockOutputStream: filled checksums 202410_1_1011_376 (state Temporary) 2024.10.22 02:35:17.870799 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 830 2024.10.22 02:35:17.895621 [ 1392 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_8_16_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/rvg/cfdytnpfvyhjzngjdaszttpddjgil (LogSeriesLimiter: on interval from 2024-10-22 02:34:55 to 2024-10-22 02:35:17 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:35:17.911473 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 113656 2024.10.22 02:35:18.122521 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1011_376} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1011_376 to 202410_1_1011_376 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:18.128879 [ 1389 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1011_376} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1010_375, 202410_1011_1011_0] -> 202410_1_1011_376 2024.10.22 02:35:18.151347 [ 1389 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.48 MiB. 2024.10.22 02:35:18.440561 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:18.445504 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:18.505946 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_812_812_0 (state Temporary) 2024.10.22 02:35:18.607364 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_958_177} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_merge_202410_1_958_177 to 202410_1_958_177 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:18.655625 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 173 entries to flush up to offset 112201 2024.10.22 02:35:18.673068 [ 1397 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_958_177} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Merged 2 parts: [202410_1_952_176, 202410_953_958_1] -> 202410_1_958_177 2024.10.22 02:35:18.700433 [ 1397 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 13.56 MiB. 2024.10.22 02:35:18.714303 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_812_812_0 to 202410_822_822_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:18.724134 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(66eb94bc1dde464a9149ffcaa0ba6751) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 66eb94bc1dde464a9149ffcaa0ba6751, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:35:18.743796 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 396373, file segment info: File segment: [0, 33554431], key: 66eb94bc1dde464a9149ffcaa0ba6751, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:35:18.745689 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:35:18.781749 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 830 2024.10.22 02:35:19.171224 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:19.186949 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:19.249525 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_985_985_0 (state Temporary) 2024.10.22 02:35:19.515425 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_985_985_0 to 202410_991_991_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:19.657910 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.55 GiB, peak 2.00 GiB, free memory in arenas 3.64 MiB, will set to 1.55 GiB (RSS), difference: -4.83 MiB 2024.10.22 02:35:19.721197 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(be8df2445acc2089fef5208c007cd939) : [0, 158650]: Setting download as failed: File segment: [0, 158650], key: be8df2445acc2089fef5208c007cd939, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:19.726003 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(be8df2445acc2089fef5208c007cd939): No space left in cache to reserve 158651 bytes, will continue without cache download 2024.10.22 02:35:19.735681 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 112201 2024.10.22 02:35:19.751866 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_985_85 to 202410_986_986_0 2024.10.22 02:35:19.756968 [ 1200 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.54 MiB using storage policy from min volume index 0 2024.10.22 02:35:19.785335 [ 1200 ] {} DiskLocal: Reserved 1.54 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:19.828871 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_986_86} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_985_85 to 202410_986_986_0 into Wide with storage Full 2024.10.22 02:35:19.864124 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: be8df2445acc2089fef5208c007cd939, offset: 0, size: 0 2024.10.22 02:35:19.865471 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_986_86} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:35:19.866036 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key be8df2445acc2089fef5208c007cd939 for removal 2024.10.22 02:35:19.870647 [ 1476 ] {} CacheMetadata: Key be8df2445acc2089fef5208c007cd939 is removed from metadata 2024.10.22 02:35:19.870757 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: a9259a9c375a4ca648714f79f2c34783, offset: 0, size: 0 2024.10.22 02:35:19.871883 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_986_86} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_985_85, total 27798 rows starting from the beginning of the part 2024.10.22 02:35:19.872721 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key a9259a9c375a4ca648714f79f2c34783 for removal 2024.10.22 02:35:19.876480 [ 1476 ] {} CacheMetadata: Key a9259a9c375a4ca648714f79f2c34783 is removed from metadata 2024.10.22 02:35:20.007818 [ 1385 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_986_86} MergeTreeSequentialSource: Reading 2 marks from part 202410_986_986_0, total 183 rows starting from the beginning of the part 2024.10.22 02:35:20.605642 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(5622223ca9fb541f3de31d5802c44b4e) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: 5622223ca9fb541f3de31d5802c44b4e, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:35:20.609716 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 519, file segment info: File segment: [0, 33554431], key: 5622223ca9fb541f3de31d5802c44b4e, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:35:20.615278 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:35:21.145518 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 770 entries to flush up to offset 1441177 2024.10.22 02:35:21.237319 [ 1549 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:21.243496 [ 1549 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.14 GiB. 2024.10.22 02:35:21.555730 [ 1040 ] {} KeeperTCPHandler: Received heartbeat for session #4 2024.10.22 02:35:21.557833 [ 1319 ] {} KeeperDispatcher: Processing requests batch, size: 1, bytes: 0 2024.10.22 02:35:21.641232 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5afde881421623e23bdce67d89aaa750) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 5afde881421623e23bdce67d89aaa750, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:21.645230 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5afde881421623e23bdce67d89aaa750): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:35:22.029028 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 70 entries to flush up to offset 113726 2024.10.22 02:35:22.157054 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:22.159288 [ 1550 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:22.343337 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:22.356195 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.14 GiB. 2024.10.22 02:35:22.379688 [ 1550 ] {} MergedBlockOutputStream: filled checksums 202410_961_961_0 (state Temporary) 2024.10.22 02:35:22.561661 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1690_1690_0 (state Temporary) 2024.10.22 02:35:22.573497 [ 1392 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_986_86} MergeTask::MergeProjectionsStage: Merge sorted 27981 rows, containing 13 columns (13 merged, 0 gathered) in 2.758827693 sec., 10142.351431006893 rows/sec., 4.43 MiB/sec. 2024.10.22 02:35:22.621697 [ 1550 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Renaming temporary part tmp_insert_202410_961_961_0 to 202410_968_968_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:22.649796 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1690_1690_0 to 202410_1703_1703_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:22.654505 [ 1151 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1) (MergerMutator): Selected 2 parts from 202410_1_958_177 to 202410_959_964_1 2024.10.22 02:35:22.658299 [ 1151 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): Trying to reserve 1.13 MiB using storage policy from min volume index 0 2024.10.22 02:35:22.660387 [ 1151 ] {} DiskLocal: Reserved 1.13 MiB on local disk `default`, having unreserved 296.15 GiB. 2024.10.22 02:35:22.673651 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_964_178} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_958_177 to 202410_959_964_1 into Wide with storage Full 2024.10.22 02:35:22.705151 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 113726 2024.10.22 02:35:22.766306 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushed system log up to offset 1441177 2024.10.22 02:35:22.766707 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(fe367fe938e0b22d70fcd3e5f5ebcc2d) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: fe367fe938e0b22d70fcd3e5f5ebcc2d, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:35:22.816271 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_964_178} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:35:22.838055 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_964_178} MergeTreeSequentialSource: Reading 178 marks from part 202410_1_958_177, total 1443217 rows starting from the beginning of the part 2024.10.22 02:35:22.891388 [ 1393 ] {77c672cd-125d-412b-9cb8-29507f0d4ac1::202410_1_964_178} MergeTreeSequentialSource: Reading 2 marks from part 202410_959_964_1, total 6545 rows starting from the beginning of the part 2024.10.22 02:35:22.968752 [ 1400 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_986_86} MergedBlockOutputStream: filled checksums 202410_747_986_86 (state Temporary) 2024.10.22 02:35:22.972921 [ 1211 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 232 ms. 2024.10.22 02:35:23.002584 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 975558, file segment info: File segment: [0, 33554431], key: fe367fe938e0b22d70fcd3e5f5ebcc2d, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:35:23.004160 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:35:23.351525 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Found 2 old parts to remove. Parts: [202410_1_952_317, 202410_953_953_0] 2024.10.22 02:35:23.355099 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from filesystem (serially): Parts: [202410_1_952_317, 202410_953_953_0] 2024.10.22 02:35:23.362403 [ 1421 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Removing 2 parts from memory: Parts: [202410_1_952_317, 202410_953_953_0] 2024.10.22 02:35:23.464436 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_986_86} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_merge_202410_747_986_86 to 202410_747_986_86 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:23.468300 [ 1387 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_986_86} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Merged 2 parts: [202410_747_985_85, 202410_986_986_0] -> 202410_747_986_86 2024.10.22 02:35:23.488077 [ 1387 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 60.85 MiB. 2024.10.22 02:35:23.509591 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(b32c9d042a49f3b44e4f35c65e7041be) : [0, 1299]: Setting download as failed: File segment: [0, 1299], key: b32c9d042a49f3b44e4f35c65e7041be, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:23.512007 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(b32c9d042a49f3b44e4f35c65e7041be): No space left in cache to reserve 1300 bytes, will continue without cache download 2024.10.22 02:35:23.529121 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: b32c9d042a49f3b44e4f35c65e7041be, offset: 0, size: 0 2024.10.22 02:35:23.536769 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key b32c9d042a49f3b44e4f35c65e7041be for removal 2024.10.22 02:35:23.542792 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5afde881421623e23bdce67d89aaa750, offset: 0, size: 0 2024.10.22 02:35:23.541806 [ 1476 ] {} CacheMetadata: Key b32c9d042a49f3b44e4f35c65e7041be is removed from metadata 2024.10.22 02:35:23.544943 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5afde881421623e23bdce67d89aaa750 for removal 2024.10.22 02:35:23.549127 [ 1476 ] {} CacheMetadata: Key 5afde881421623e23bdce67d89aaa750 is removed from metadata 2024.10.22 02:35:23.584214 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushing system log, 25 entries to flush up to offset 15517 2024.10.22 02:35:23.759146 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.56 GiB, peak 2.00 GiB, free memory in arenas 4.67 MiB, will set to 1.55 GiB (RSS), difference: -11.14 MiB 2024.10.22 02:35:23.817364 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:23.820861 [ 1556 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.14 GiB. 2024.10.22 02:35:24.060227 [ 1556 ] {} MergedBlockOutputStream: filled checksums 202410_1003_1003_0 (state Temporary) 2024.10.22 02:35:24.148730 [ 1556 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_insert_202410_1003_1003_0 to 202410_1016_1016_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:24.186553 [ 1213 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Selected 2 parts from 202410_1_1011_376 to 202410_1012_1012_0 2024.10.22 02:35:24.196134 [ 1213 ] {} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:24.202742 [ 1213 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.14 GiB. 2024.10.22 02:35:24.232378 [ 1556 ] {} SystemLog (system.transactions_info_log): Flushed system log up to offset 15517 2024.10.22 02:35:24.322612 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1012_377} MergeTask::PrepareStage: Merging 2 parts: from 202410_1_1011_376 to 202410_1012_1012_0 into Compact with storage Full 2024.10.22 02:35:24.377466 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1012_377} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:35:24.403031 [ 1418 ] {} system.asynchronous_metric_log (77c672cd-125d-412b-9cb8-29507f0d4ac1): /var/lib/clickhouse/store/77c/77c672cd-125d-412b-9cb8-29507f0d4ac1/tmp_merge_202410_1_964_178/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) (skipped 1 similar messages) 2024.10.22 02:35:24.407934 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1012_377} MergeTreeSequentialSource: Reading 3 marks from part 202410_1_1011_376, total 15533 rows starting from the beginning of the part 2024.10.22 02:35:24.457270 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1012_377} MergeTreeSequentialSource: Reading 2 marks from part 202410_1012_1012_0, total 26 rows starting from the beginning of the part 2024.10.22 02:35:24.464844 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegment(b485d94586fe07a04871f1132631e5ef) : [0, 33554431]: Setting download as failed: File segment: [0, 33554431], key: b485d94586fe07a04871f1132631e5ef, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:35:24.468030 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} FileSegmentRangeWriter: Failed to reserve space in cache (size: 818, file segment info: File segment: [0, 33554431], key: b485d94586fe07a04871f1132631e5ef, state: PARTIALLY_DOWNLOADED_NO_CONTINUATION, downloaded size: 0, reserved size: 0, downloader id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, current write offset: 0, caller id: 5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1:1395, kind: Regular, unbound: 0 2024.10.22 02:35:24.471320 [ 1395 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} CachedOnDiskWriteBufferFromFile: Write-through cache is stopped as cache limit is reached and nothing can be evicted 2024.10.22 02:35:24.538158 [ 1549 ] {} MergedBlockOutputStream: filled checksums 202410_201_201_0 (state Temporary) 2024.10.22 02:35:24.538988 [ 1743 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Single part upload has completed. Details: bucket test, key common/irj/vvhvhpjjkdoritxtcsilnnwyxvgua, size 975558 2024.10.22 02:35:24.841126 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 49 marks from part 201403_1_1_0, total 393216 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:35:24.881324 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 66 marks from part 201403_2_2_0, total 532480 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:35:24.922828 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 70 marks from part 201403_3_3_0, total 565248 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:35:25.112093 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 43 marks from part 201403_4_4_0, total 344064 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:35:25.162929 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 32 marks from part 201403_5_5_0, total 253952 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:35:25.197571 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 33 marks from part 201403_6_6_0, total 262144 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:35:25.214175 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1599_124, 202410_1600_1600_0] 2024.10.22 02:35:25.221169 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1599_124, 202410_1600_1600_0] 2024.10.22 02:35:25.257527 [ 1423 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1599_124, 202410_1600_1600_0] 2024.10.22 02:35:25.352209 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} MergeTreeSequentialSource: Reading 63 marks from part 201403_7_7_0, total 507904 rows starting from the beginning of the part, column ParsedParams.Key3 2024.10.22 02:35:25.537560 [ 1387 ] {5e446213-61fa-4dfd-ac7e-377a973d53b4::201403_1_7_1} WriteBufferFromS3: Create WriteBufferFromS3, Details: bucket test, key common/xkk/zsqzoqmxiskalxiaqcexysfepiosc (LogSeriesLimiter: on interval from 2024-10-22 02:35:17 to 2024-10-22 02:35:25 accepted series 1 / 2 for the logger WriteBufferFromS3 : 1.7491308770058435e+38) 2024.10.22 02:35:25.637307 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(8eee364b4c26ff664e16fb9b458d2111) : [0, 155]: Setting download as failed: File segment: [0, 155], key: 8eee364b4c26ff664e16fb9b458d2111, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:25.638770 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(8eee364b4c26ff664e16fb9b458d2111): No space left in cache to reserve 156 bytes, will continue without cache download 2024.10.22 02:35:25.745792 [ 1398 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1012_377} MergeTask::MergeProjectionsStage: Merge sorted 15559 rows, containing 12 columns (12 merged, 0 gathered) in 1.432150799 sec., 10864.079404811337 rows/sec., 1.70 MiB/sec. 2024.10.22 02:35:25.801769 [ 1395 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1012_377} MergedBlockOutputStream: filled checksums 202410_1_1012_377 (state Temporary) 2024.10.22 02:35:25.928131 [ 1229 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 02:35:26.149176 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1012_377} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8): Renaming temporary part tmp_merge_202410_1_1012_377 to 202410_1_1012_377 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:26.152180 [ 1399 ] {5276570e-a364-404e-a403-3e186cba01a8::202410_1_1012_377} system.transactions_info_log (5276570e-a364-404e-a403-3e186cba01a8) (MergerMutator): Merged 2 parts: [202410_1_1011_376, 202410_1012_1012_0] -> 202410_1_1012_377 2024.10.22 02:35:26.173132 [ 1399 ] {} MemoryTracker: Peak memory usage background process to apply mutate/merge in table: 8.49 MiB. 2024.10.22 02:35:26.285067 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushing system log, 1 entries to flush up to offset 831 2024.10.22 02:35:26.670746 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:26.674620 [ 1554 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.14 GiB. 2024.10.22 02:35:26.714475 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(f815bfd81d8b953aaa488881fb7f3ea9) : [0, 30230]: Setting download as failed: File segment: [0, 30230], key: f815bfd81d8b953aaa488881fb7f3ea9, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:26.720404 [ 1548 ] {} SystemLog (system.text_log): Flushing system log, 80 entries to flush up to offset 113806 2024.10.22 02:35:26.721538 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(f815bfd81d8b953aaa488881fb7f3ea9): No space left in cache to reserve 30231 bytes, will continue without cache download 2024.10.22 02:35:26.748117 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: f815bfd81d8b953aaa488881fb7f3ea9, offset: 0, size: 0 2024.10.22 02:35:26.751201 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key f815bfd81d8b953aaa488881fb7f3ea9 for removal 2024.10.22 02:35:26.754654 [ 1476 ] {} CacheMetadata: Key f815bfd81d8b953aaa488881fb7f3ea9 is removed from metadata 2024.10.22 02:35:26.856341 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 8eee364b4c26ff664e16fb9b458d2111, offset: 0, size: 0 2024.10.22 02:35:26.858997 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 8eee364b4c26ff664e16fb9b458d2111 for removal 2024.10.22 02:35:26.863468 [ 1476 ] {} CacheMetadata: Key 8eee364b4c26ff664e16fb9b458d2111 is removed from metadata 2024.10.22 02:35:26.884244 [ 1554 ] {} MergedBlockOutputStream: filled checksums 202410_813_813_0 (state Temporary) 2024.10.22 02:35:26.983892 [ 1554 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Renaming temporary part tmp_insert_202410_813_813_0 to 202410_823_823_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:27.044206 [ 1248 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e) (MergerMutator): Selected 6 parts from 202410_1_818_166 to 202410_823_823_0 2024.10.22 02:35:27.054878 [ 1248 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:27.057570 [ 1248 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.14 GiB. 2024.10.22 02:35:27.128614 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_823_167} MergeTask::PrepareStage: Merging 6 parts: from 202410_1_818_166 to 202410_823_823_0 into Compact with storage Full 2024.10.22 02:35:27.143915 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:27.149160 [ 1548 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.14 GiB. 2024.10.22 02:35:27.181700 [ 1554 ] {} SystemLog (system.zookeeper_log): Flushed system log up to offset 831 2024.10.22 02:35:27.223146 [ 1548 ] {} MergedBlockOutputStream: filled checksums 202410_1691_1691_0 (state Temporary) 2024.10.22 02:35:27.262675 [ 1546 ] {} SystemLog (system.trace_log): Flushing system log, 189 entries to flush up to offset 112390 2024.10.22 02:35:27.308218 [ 1275 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 242 ms. 2024.10.22 02:35:27.336910 [ 1548 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Renaming temporary part tmp_insert_202410_1691_1691_0 to 202410_1704_1704_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:27.410736 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.57 GiB, peak 2.00 GiB, free memory in arenas 11.96 MiB, will set to 1.56 GiB (RSS), difference: -7.92 MiB 2024.10.22 02:35:27.444676 [ 1510 ] {} AsynchronousMetrics: Scanning /sys/block 2024.10.22 02:35:27.463619 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.00 MiB using storage policy from min volume index 0 2024.10.22 02:35:27.477741 [ 1546 ] {} DiskLocal: Reserved 1.00 MiB on local disk `default`, having unreserved 296.14 GiB. 2024.10.22 02:35:27.517310 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_823_167} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:35:27.549322 [ 1546 ] {} MergedBlockOutputStream: filled checksums 202410_986_986_0 (state Temporary) 2024.10.22 02:35:27.549324 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_823_167} MergeTreeSequentialSource: Reading 2 marks from part 202410_1_818_166, total 862 rows starting from the beginning of the part 2024.10.22 02:35:27.612280 [ 1546 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Renaming temporary part tmp_insert_202410_986_986_0 to 202410_992_992_0 with tid (1, 1, 00000000-0000-0000-0000-000000000000). 2024.10.22 02:35:27.631479 [ 1548 ] {} SystemLog (system.text_log): Flushed system log up to offset 113806 2024.10.22 02:35:27.635188 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_823_167} MergeTreeSequentialSource: Reading 2 marks from part 202410_819_819_0, total 1 rows starting from the beginning of the part 2024.10.22 02:35:27.674727 [ 1546 ] {} SystemLog (system.trace_log): Flushed system log up to offset 112390 2024.10.22 02:35:27.793319 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_823_167} MergeTreeSequentialSource: Reading 2 marks from part 202410_820_820_0, total 1 rows starting from the beginning of the part 2024.10.22 02:35:27.809535 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(5909b9825b34e96f2cba9bc6eb680ea3) : [0, 192]: Setting download as failed: File segment: [0, 192], key: 5909b9825b34e96f2cba9bc6eb680ea3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:27.813256 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(5909b9825b34e96f2cba9bc6eb680ea3): No space left in cache to reserve 193 bytes, will continue without cache download 2024.10.22 02:35:27.713751 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac) (MergerMutator): Selected 2 parts from 202410_747_986_86 to 202410_987_987_0 2024.10.22 02:35:27.826937 [ 1215 ] {} DNSResolver: Updating DNS cache 2024.10.22 02:35:27.828234 [ 1265 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): Trying to reserve 1.55 MiB using storage policy from min volume index 0 2024.10.22 02:35:27.845376 [ 1265 ] {} DiskLocal: Reserved 1.55 MiB on local disk `default`, having unreserved 296.14 GiB. 2024.10.22 02:35:27.862154 [ 1215 ] {} DNSResolver: Updated DNS cache 2024.10.22 02:35:27.864329 [ 1265 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 229 ms. 2024.10.22 02:35:27.869021 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_987_87} MergeTask::PrepareStage: Merging 2 parts: from 202410_747_986_86 to 202410_987_987_0 into Wide with storage Full 2024.10.22 02:35:27.890141 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_987_87} MergeTask::PrepareStage: Selected MergeAlgorithm: Horizontal 2024.10.22 02:35:27.904133 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_823_167} MergeTreeSequentialSource: Reading 2 marks from part 202410_821_821_0, total 1 rows starting from the beginning of the part 2024.10.22 02:35:27.910865 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_987_87} MergeTreeSequentialSource: Reading 5 marks from part 202410_747_986_86, total 27981 rows starting from the beginning of the part 2024.10.22 02:35:28.151435 [ 1395 ] {b1d331ed-9402-47f3-92ba-69560d00c0ac::202410_747_987_87} MergeTreeSequentialSource: Reading 2 marks from part 202410_987_987_0, total 175 rows starting from the beginning of the part 2024.10.22 02:35:28.217140 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_823_167} MergeTreeSequentialSource: Reading 2 marks from part 202410_822_822_0, total 1 rows starting from the beginning of the part 2024.10.22 02:35:28.349160 [ 1392 ] {4de457e0-9d34-4a1d-91af-81e270dc2c3e::202410_1_823_167} MergeTreeSequentialSource: Reading 2 marks from part 202410_823_823_0, total 1 rows starting from the beginning of the part 2024.10.22 02:35:28.667958 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Found 2 old parts to remove. Parts: [202410_1207_1600_125, 202410_1601_1601_0] 2024.10.22 02:35:28.670571 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from filesystem (serially): Parts: [202410_1207_1600_125, 202410_1601_1601_0] 2024.10.22 02:35:28.684581 [ 1425 ] {} system.text_log (ffa745eb-8a28-47e2-b73e-5e0c8034f61b): Removing 2 parts from memory: Parts: [202410_1207_1600_125, 202410_1601_1601_0] 2024.10.22 02:35:28.813460 [ 1421 ] {} system.metric_log (92e05502-7cd1-42a9-b81b-a85674c93a09): /var/lib/clickhouse/store/92e/92e05502-7cd1-42a9-b81b-a85674c93a09/tmp_insert_202410_201_201_0/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:35:29.079485 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} FileSegment(3c7d1d667fa653893c765541ed7716a3) : [0, 1048786]: Setting download as failed: File segment: [0, 1048786], key: 3c7d1d667fa653893c765541ed7716a3, state: DOWNLOADING, downloaded size: 0, reserved size: 0, downloader id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, current write offset: 0, caller id: a62859f6-69a4-4c1f-82f3-460a7a455cae:1448, kind: Regular, unbound: 0 2024.10.22 02:35:29.083094 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CachedOnDiskReadBufferFromFile(3c7d1d667fa653893c765541ed7716a3): No space left in cache to reserve 1048576 bytes, will continue without cache download 2024.10.22 02:35:29.265742 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 3c7d1d667fa653893c765541ed7716a3, offset: 0, size: 0 2024.10.22 02:35:29.267495 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 3c7d1d667fa653893c765541ed7716a3 for removal 2024.10.22 02:35:29.281452 [ 1476 ] {} CacheMetadata: Key 3c7d1d667fa653893c765541ed7716a3 is removed from metadata 2024.10.22 02:35:29.292694 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Remove from cache. Key: 5909b9825b34e96f2cba9bc6eb680ea3, offset: 0, size: 0 2024.10.22 02:35:29.300973 [ 1448 ] {a62859f6-69a4-4c1f-82f3-460a7a455cae} CacheMetadata: Submitting key 5909b9825b34e96f2cba9bc6eb680ea3 for removal 2024.10.22 02:35:29.304742 [ 1476 ] {} CacheMetadata: Key 5909b9825b34e96f2cba9bc6eb680ea3 is removed from metadata 2024.10.22 02:35:29.311100 [ 1276 ] {} BackgroundJobsAssignee:DataProcessing: Execution took 230 ms. 2024.10.22 02:35:29.352266 [ 1423 ] {} system.zookeeper_log (4de457e0-9d34-4a1d-91af-81e270dc2c3e): /var/lib/clickhouse/store/4de/4de457e0-9d34-4a1d-91af-81e270dc2c3e/tmp_merge_202410_1_823_167/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:35:29.729365 [ 1510 ] {} AsynchronousMetrics: MemoryTracking: was 1.62 GiB, peak 2.00 GiB, free memory in arenas 11.83 MiB, will set to 1.57 GiB (RSS), difference: -42.81 MiB 2024.10.22 02:35:29.739386 [ 1421 ] {} system.trace_log (b1d331ed-9402-47f3-92ba-69560d00c0ac): /var/lib/clickhouse/store/b1d/b1d331ed-9402-47f3-92ba-69560d00c0ac/tmp_merge_202410_747_987_87/ is in use (by merge/mutation/INSERT) (consider increasing temporary_directories_lifetime setting) 2024.10.22 02:35:29.769037 [ 1550 ] {} SystemLog (system.asynchronous_metric_log): Flushing system log, 1155 entries to flush up to offset 1442332